-- Logs begin at Sat 2022-07-16 20:01:20 UTC, end at Sat 2022-07-16 22:45:07 UTC. -- Jul 16 20:56:32.120073 ncdv-master-1 systemd[1]: Starting Devstack devstack@n-api.service... Jul 16 20:56:32.128803 ncdv-master-1 devstack@n-api.service[122765]: [uWSGI] getting INI configuration from /etc/nova/nova-api-uwsgi.ini Jul 16 20:56:32.130778 ncdv-master-1 devstack@n-api.service[122765]: *** Starting uWSGI 2.0.18-debian (64bit) on [Sat Jul 16 20:56:32 2022] *** Jul 16 20:56:32.130778 ncdv-master-1 devstack@n-api.service[122765]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jul 16 20:56:32.130778 ncdv-master-1 devstack@n-api.service[122765]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jul 16 20:56:32.130778 ncdv-master-1 devstack@n-api.service[122765]: nodename: ncdv-master-1 Jul 16 20:56:32.130778 ncdv-master-1 devstack@n-api.service[122765]: machine: x86_64 Jul 16 20:56:32.130778 ncdv-master-1 devstack@n-api.service[122765]: clock source: unix Jul 16 20:56:32.130778 ncdv-master-1 devstack@n-api.service[122765]: pcre jit disabled Jul 16 20:56:32.130778 ncdv-master-1 devstack@n-api.service[122765]: detected number of CPU cores: 4 Jul 16 20:56:32.130778 ncdv-master-1 devstack@n-api.service[122765]: current working directory: / Jul 16 20:56:32.130778 ncdv-master-1 devstack@n-api.service[122765]: detected binary path: /usr/bin/uwsgi-core Jul 16 20:56:32.131530 ncdv-master-1 devstack@n-api.service[122765]: your processes number limit is 47680 Jul 16 20:56:32.131530 ncdv-master-1 devstack@n-api.service[122765]: your memory page size is 4096 bytes Jul 16 20:56:32.131530 ncdv-master-1 devstack@n-api.service[122765]: detected max file descriptor number: 2048 Jul 16 20:56:32.131530 ncdv-master-1 devstack@n-api.service[122765]: lock engine: pthread robust mutexes Jul 16 20:56:32.131530 ncdv-master-1 devstack@n-api.service[122765]: thunder lock: enabled Jul 16 20:56:32.131530 ncdv-master-1 devstack@n-api.service[122765]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/nova-api-wsgi.socket fd 4 Jul 16 20:56:32.131530 ncdv-master-1 devstack@n-api.service[122765]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Jul 16 20:56:32.176184 ncdv-master-1 devstack@n-api.service[122765]: Python main interpreter initialized at 0x55e2e22a79c0 Jul 16 20:56:32.176184 ncdv-master-1 devstack@n-api.service[122765]: python threads support enabled Jul 16 20:56:32.176184 ncdv-master-1 devstack@n-api.service[122765]: your server socket listen backlog is limited to 100 connections Jul 16 20:56:32.176184 ncdv-master-1 devstack@n-api.service[122765]: your mercy for graceful operations on workers is 90 seconds Jul 16 20:56:32.176567 ncdv-master-1 devstack@n-api.service[122765]: mapped 403077 bytes (393 KB) for 2 cores Jul 16 20:56:32.176631 ncdv-master-1 devstack@n-api.service[122765]: *** Operational MODE: preforking *** Jul 16 20:56:32.176849 ncdv-master-1 devstack@n-api.service[122765]: *** uWSGI is running in multiple interpreter mode *** Jul 16 20:56:32.176849 ncdv-master-1 devstack@n-api.service[122765]: spawned uWSGI master process (pid: 122765) Jul 16 20:56:32.177058 ncdv-master-1 systemd[1]: Started Devstack devstack@n-api.service. Jul 16 20:56:32.178466 ncdv-master-1 devstack@n-api.service[122765]: spawned uWSGI worker 1 (pid: 122766, cores: 1) Jul 16 20:56:32.178466 ncdv-master-1 devstack@n-api.service[122765]: spawned uWSGI worker 2 (pid: 122767, cores: 1) Jul 16 20:56:32.178466 ncdv-master-1 devstack@n-api.service[122765]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jul 16 20:56:33.453590 ncdv-master-1 devstack@n-api.service[122766]: Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code. Jul 16 20:56:33.454246 ncdv-master-1 devstack@n-api.service[122767]: Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code. Jul 16 20:56:34.146291 ncdv-master-1 devstack@n-api.service[122766]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 16 20:56:34.146291 ncdv-master-1 devstack@n-api.service[122766]: from cryptography.utils import int_from_bytes Jul 16 20:56:34.146535 ncdv-master-1 devstack@n-api.service[122766]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 16 20:56:34.146535 ncdv-master-1 devstack@n-api.service[122766]: from cryptography.utils import int_from_bytes Jul 16 20:56:34.147782 ncdv-master-1 devstack@n-api.service[122767]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 16 20:56:34.147782 ncdv-master-1 devstack@n-api.service[122767]: from cryptography.utils import int_from_bytes Jul 16 20:56:34.148121 ncdv-master-1 devstack@n-api.service[122767]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 16 20:56:34.148121 ncdv-master-1 devstack@n-api.service[122767]: from cryptography.utils import int_from_bytes Jul 16 20:56:36.793770 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 16 20:56:36.794343 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] Configuration options gathered from: {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 16 20:56:36.794549 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] command line args: [] {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 16 20:56:36.794916 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] config files: ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 16 20:56:36.795200 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] ================================================================================ {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 16 20:56:36.795599 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 16 20:56:36.795976 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] allow_resize_to_same_host = True {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.796060 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] Configuration options gathered from: {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 16 20:56:36.796136 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] arq_binding_timeout = 300 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.796236 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries = 60 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.796393 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] command line args: [] {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 16 20:56:36.796557 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries_interval = 3 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.796766 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] config files: ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 16 20:56:36.796888 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] cert = self.pem {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.797031 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] ================================================================================ {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 16 20:56:36.797200 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] compute_driver = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.797428 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] allow_resize_to_same_host = True {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.797525 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] compute_monitors = [] {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.797760 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] arq_binding_timeout = 300 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.797842 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] config_dir = [] {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.798088 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries = 60 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.798180 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] config_drive_format = iso9660 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.798395 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] config_file = ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.798485 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries_interval = 3 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.798711 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] config_source = [] {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.798803 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] cert = self.pem {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.799059 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] compute_driver = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.799238 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] console_host = ncdv-master-1 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.799389 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] control_exchange = nova {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.799512 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] compute_monitors = [] {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.799698 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] cpu_allocation_ratio = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.799820 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] config_dir = [] {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.800016 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] daemon = False {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.800164 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] config_drive_format = iso9660 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.800286 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] config_file = ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.800409 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] debug = True {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.800582 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] config_source = [] {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.800704 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] default_access_ip_network_name = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.800911 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] console_host = ncdv-master-1 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.801043 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] default_availability_zone = nova {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.801241 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] default_ephemeral_format = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.801370 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] control_exchange = nova {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.801547 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] cpu_allocation_ratio = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.801734 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.801848 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] daemon = False {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.802177 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] debug = True {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.802274 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] default_schedule_zone = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.802482 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] disk_allocation_ratio = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.802575 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] default_access_ip_network_name = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.802795 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] enable_new_services = True {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.802893 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] default_availability_zone = nova {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.803130 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] default_ephemeral_format = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.803219 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] enabled_apis = ['osapi_compute'] {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.803475 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] enabled_ssl_apis = [] {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.803635 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.803787 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] flat_injected = False {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.803940 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] default_schedule_zone = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.804112 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] force_config_drive = True {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.804243 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] disk_allocation_ratio = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.804423 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] force_raw_images = True {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.804541 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] enable_new_services = True {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.804747 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] graceful_shutdown_timeout = 5 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.804876 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] enabled_apis = ['osapi_compute'] {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.805056 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] heal_instance_info_cache_interval = 60 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.805185 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] enabled_ssl_apis = [] {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.805456 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] host = ncdv-master-1 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.805568 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] flat_injected = False {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.805773 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] initial_cpu_allocation_ratio = 16.0 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.805878 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] force_config_drive = True {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.806083 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] initial_disk_allocation_ratio = 1.0 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.806676 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] force_raw_images = True {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.807062 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] initial_ram_allocation_ratio = 1.5 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.807209 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] graceful_shutdown_timeout = 5 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.807317 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] heal_instance_info_cache_interval = 60 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.807444 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] injected_network_template = /opt/stack/nova/nova/virt/interfaces.template {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.807640 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] host = ncdv-master-1 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.807771 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] instance_build_timeout = 0 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.807961 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] initial_cpu_allocation_ratio = 16.0 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.808083 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] instance_delete_interval = 300 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.808278 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] initial_disk_allocation_ratio = 1.0 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.808413 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] instance_format = [instance: %(uuid)s] {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.808595 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] initial_ram_allocation_ratio = 1.5 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.808717 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] instance_name_template = instance-%08x {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.808987 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit = False {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.809120 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] injected_network_template = /opt/stack/nova/nova/virt/interfaces.template {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.809313 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] instance_build_timeout = 0 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.809435 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit_period = month {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.809623 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] instance_delete_interval = 300 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.809778 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.809949 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] instance_format = [instance: %(uuid)s] {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.810085 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] instances_path = /opt/stack/data/nova/instances {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.810280 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] internal_service_availability_zone = internal {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.810406 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] instance_name_template = instance-%08x {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.810565 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] key = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.810688 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit = False {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.810873 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] live_migration_retry_count = 30 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.811036 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit_period = month {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.811251 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] log_config_append = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.811399 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.811521 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.811641 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] instances_path = /opt/stack/data/nova/instances {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.811809 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] log_dir = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.812044 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] internal_service_availability_zone = internal {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.812128 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] log_file = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.812219 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] key = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.812354 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] log_options = True {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.812485 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] live_migration_retry_count = 30 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.812661 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval = 1 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.812786 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] log_config_append = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.812981 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval_type = days {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.813110 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.813306 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] log_rotation_type = none {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.813405 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] log_dir = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.813630 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] log_file = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.813723 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.813881 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.813984 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] log_options = True {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.814132 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.814235 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval = 1 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.814452 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.814546 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval_type = days {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.814771 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.814864 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] log_rotation_type = none {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.815153 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.815260 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] long_rpc_timeout = 1800 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.815412 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.815504 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_builds = 10 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.815668 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.815760 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_live_migrations = 1 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.815990 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.816081 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_snapshots = 5 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.816316 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.816400 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] max_local_block_devices = 3 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.816627 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_count = 30 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.816708 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] long_rpc_timeout = 1800 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.816933 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_size_mb = 200 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.817017 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_builds = 10 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.817289 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_live_migrations = 1 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.817385 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] maximum_instance_delete_attempts = 5 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.817546 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_snapshots = 5 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.817643 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen = 0.0.0.0 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.817853 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] max_local_block_devices = 3 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.817938 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen_port = 8775 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.818162 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_count = 30 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.818254 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] metadata_workers = 2 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.818466 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] migrate_max_retries = -1 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.818569 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_size_mb = 200 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.818774 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] maximum_instance_delete_attempts = 5 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.818865 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] mkisofs_cmd = genisoimage {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.819123 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen = 0.0.0.0 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.819213 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] my_block_storage_ip = 10.222.0.16 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.819440 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] my_ip = 10.222.0.16 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.819541 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen_port = 8775 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.819749 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] network_allocate_retries = 0 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.819863 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] metadata_workers = 2 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.820063 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] migrate_max_retries = -1 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.820173 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.820388 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] mkisofs_cmd = genisoimage {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.820508 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen = 0.0.0.0 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.820697 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen_port = 8774 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.820807 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] my_block_storage_ip = 10.222.0.16 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.821019 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_unique_server_name_scope = {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.821142 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] my_ip = 10.222.0.16 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.821336 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] network_allocate_retries = 0 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.821453 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_workers = 2 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.821638 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] password_length = 12 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.821764 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.822537 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen = 0.0.0.0 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.822728 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] periodic_enable = True {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.822940 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen_port = 8774 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.823157 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] periodic_fuzzy_delay = 60 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.823288 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_unique_server_name_scope = {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.823410 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] pointer_model = usbtablet {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.823547 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_workers = 2 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.823669 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] preallocate_images = none {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.823801 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] password_length = 12 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.823953 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] publish_errors = False {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.824085 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] periodic_enable = True {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.824206 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] pybasedir = /opt/stack/nova {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.824398 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] periodic_fuzzy_delay = 60 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.824521 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] ram_allocation_ratio = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.824729 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] pointer_model = usbtablet {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.824853 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_burst = 0 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.825059 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] preallocate_images = none {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.825182 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_except_level = CRITICAL {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.825371 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] publish_errors = False {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.825496 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_interval = 0 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.825625 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] pybasedir = /opt/stack/nova {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.825745 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] reboot_timeout = 0 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.825922 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] ram_allocation_ratio = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.826009 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] reclaim_instance_interval = 0 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.826232 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_burst = 0 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.826398 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] record = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.826558 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_except_level = CRITICAL {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.826643 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] report_interval = 10 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.826864 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] rescue_timeout = 0 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.826946 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_interval = 0 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.827197 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_cpus = 0 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.827297 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] reboot_timeout = 0 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.827506 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_disk_mb = 0 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.827635 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] reclaim_instance_interval = 0 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.827810 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] record = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.827892 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_memory_mb = 512 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.828111 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] reserved_huge_pages = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.828216 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] report_interval = 10 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.828421 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] resize_confirm_window = 0 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.828516 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] rescue_timeout = 0 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.828725 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] resize_fs_using_block_device = False {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.828815 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_cpus = 0 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.829031 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_disk_mb = 0 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.829125 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] resume_guests_state_on_host_boot = False {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.829341 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_memory_mb = 512 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.829425 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] rootwrap_config = /etc/nova/rootwrap.conf {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.829650 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] reserved_huge_pages = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.829722 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_action = reap {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.829958 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_poll_interval = 1800 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.830038 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] resize_confirm_window = 0 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.830263 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_timeout = 0 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.830352 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] resize_fs_using_block_device = False {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.830567 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] scheduler_instance_sync_interval = 120 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.830664 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] resume_guests_state_on_host_boot = False {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.830872 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] service_down_time = 60 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.830964 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] rootwrap_config = /etc/nova/rootwrap.conf {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.831218 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] servicegroup_driver = db {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.831304 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_action = reap {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.831527 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_poll_interval = 1800 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.831625 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] shelved_offload_time = 0 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.831830 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] shelved_poll_interval = 3600 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.831913 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_timeout = 0 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.832137 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] scheduler_instance_sync_interval = 120 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.832219 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] shutdown_timeout = 0 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.832445 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] source_is_ipv6 = False {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.832529 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] service_down_time = 60 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.832750 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] ssl_only = False {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.832842 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] servicegroup_driver = db {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.833060 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] shelved_offload_time = 0 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.833143 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] state_path = /opt/stack/data/nova {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.833365 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_interval = 600 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.833455 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] shelved_poll_interval = 3600 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.833673 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_pool_size = 1000 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.833759 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] shutdown_timeout = 0 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.833982 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] source_is_ipv6 = False {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.834070 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] syslog_log_facility = LOG_USER {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.834280 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] tempdir = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.834361 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] ssl_only = False {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.834584 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] timeout_nbd = 10 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.834669 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] state_path = /opt/stack/data/nova {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.834902 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_interval = 600 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.834991 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] transport_url = **** {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.835229 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] update_resources_interval = 0 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.835311 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_pool_size = 1000 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.835535 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] use_cow_images = True {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.835621 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] syslog_log_facility = LOG_USER {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.835839 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] use_eventlog = False {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.835977 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] tempdir = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.836147 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] timeout_nbd = 10 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.836232 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] use_journal = False {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.836449 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] use_json = False {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.836533 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] transport_url = **** {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.836758 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] use_rootwrap_daemon = False {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.836850 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] update_resources_interval = 0 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.837063 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] use_stderr = False {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.837155 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] use_cow_images = True {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.837386 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] use_syslog = False {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.837469 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] use_eventlog = False {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.837771 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] vcpu_pin_set = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.837927 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] use_journal = False {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.838094 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_is_fatal = True {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.838225 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] use_json = False {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.838420 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_timeout = 300 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.838559 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] use_rootwrap_daemon = False {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.838714 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] virt_mkfs = [] {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.838828 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] use_stderr = False {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.839057 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] volume_usage_poll_interval = 0 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.839159 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] use_syslog = False {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.839352 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] watch_log_file = False {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.839461 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] vcpu_pin_set = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.839672 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] web = /usr/share/spice-html5 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.839788 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_is_fatal = True {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.840019 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.disable_process_locking = False {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.840117 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_timeout = 300 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.840357 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.lock_path = /opt/stack/data/nova {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.840442 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] virt_mkfs = [] {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.840703 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] api.auth_strategy = keystone {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.840787 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] volume_usage_poll_interval = 0 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.841011 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] watch_log_file = False {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.841106 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] api.compute_link_prefix = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.841337 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] web = /usr/share/spice-html5 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 20:56:36.841420 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.841667 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] api.dhcp_domain = novalocal {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.841759 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.disable_process_locking = False {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.841990 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] api.enable_instance_password = True {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.842075 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.lock_path = /opt/stack/data/nova {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.842305 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] api.glance_link_prefix = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.842395 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] api.auth_strategy = keystone {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.842621 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_fixed_size = 100 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.842705 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] api.compute_link_prefix = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.842950 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_strategy = distributed {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.843049 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.843289 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_per_project_cells = False {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.843376 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] api.dhcp_domain = novalocal {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.843603 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] api.list_records_by_skipping_down_cells = True {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.843698 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] api.enable_instance_password = True {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.843917 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] api.local_metadata_per_cell = False {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.844002 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] api.glance_link_prefix = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.844237 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] api.max_limit = 1000 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.844330 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_fixed_size = 100 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.844558 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] api.metadata_cache_expiration = 15 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.844654 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_strategy = distributed {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.844885 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_per_project_cells = False {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.844974 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] api.neutron_default_tenant_id = default {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.845201 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] api.use_forwarded_for = False {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.845283 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] api.list_records_by_skipping_down_cells = True {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.845516 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] api.local_metadata_per_cell = False {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.845609 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] api.use_neutron_default_nets = False {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.845834 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_connect_timeout = 5 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.845917 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] api.max_limit = 1000 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.846148 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_failure_fatal = False {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.846273 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] api.metadata_cache_expiration = 15 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.846466 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_read_timeout = 5 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.846558 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] api.neutron_default_tenant_id = default {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.846789 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] api.use_forwarded_for = False {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.846881 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_ssl_certfile = {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.847138 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] api.use_neutron_default_nets = False {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.847226 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_targets = [] {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.847464 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_connect_timeout = 5 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.847628 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_jsonfile_path = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.847783 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_failure_fatal = False {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.847875 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_providers = ['StaticJSON'] {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.848105 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_read_timeout = 5 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.848189 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend = dogpile.cache.memcached {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.848439 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_ssl_certfile = {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.848536 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend_argument = **** {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.848765 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_targets = [] {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.848853 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] cache.config_prefix = cache.oslo {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.849083 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] cache.dead_timeout = 60.0 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.849165 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_jsonfile_path = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.849397 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] cache.debug_cache_backend = False {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.849498 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_providers = ['StaticJSON'] {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.849709 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_retry_client = False {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.849794 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend = dogpile.cache.memcached {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.850026 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_socket_keepalive = False {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.850109 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend_argument = **** {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.850350 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] cache.enabled = True {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.850436 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] cache.config_prefix = cache.oslo {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.850678 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] cache.expiration_time = 600 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.850761 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] cache.dead_timeout = 60.0 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.850992 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_attempts = 2 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.851090 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] cache.debug_cache_backend = False {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.851332 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_retry_client = False {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.851417 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_delay = 1.0 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.851643 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_dead_retry = 300 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.851734 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_socket_keepalive = False {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.851953 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.852036 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] cache.enabled = True {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.852263 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.852348 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] cache.expiration_time = 600 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.852574 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_maxsize = 10 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.852658 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_attempts = 2 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.852884 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_unused_timeout = 60 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.852967 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_delay = 1.0 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.853257 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_dead_retry = 300 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.853410 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_servers = ['localhost:11211'] {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.853684 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_socket_timeout = 1.0 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.853827 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.853997 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] cache.proxies = [] {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.854096 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.854312 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_attempts = 2 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.854397 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_maxsize = 10 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.854633 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_delay = 0.0 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.854728 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_unused_timeout = 60 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.854945 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_count = 1 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.855068 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_servers = ['localhost:11211'] {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.855285 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_idle = 1 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.855369 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_socket_timeout = 1.0 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.855598 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_interval = 1 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.855686 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] cache.proxies = [] {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.855906 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_allowed_ciphers = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.855999 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_attempts = 2 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.856213 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_cafile = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.856310 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_delay = 0.0 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.856520 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_certfile = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.856607 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_count = 1 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.856832 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_enabled = False {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.856930 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_idle = 1 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.857138 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_keyfile = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.857223 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_interval = 1 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.857456 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_section = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.857539 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_allowed_ciphers = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.857788 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_type = password {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.857880 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_cafile = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.858094 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_certfile = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.858177 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cafile = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.858410 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_enabled = False {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.858493 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] cinder.catalog_info = volumev3::publicURL {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.858717 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] cinder.certfile = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.858801 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_keyfile = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.859487 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] cinder.collect_timing = False {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.859622 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_section = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.859727 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cross_az_attach = True {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.859843 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_type = password {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.859958 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] cinder.debug = False {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.860061 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cafile = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.860162 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] cinder.endpoint_template = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.860264 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] cinder.catalog_info = volumev3::publicURL {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.860374 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] cinder.certfile = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.860476 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] cinder.http_retries = 3 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.860591 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] cinder.collect_timing = False {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.860694 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] cinder.insecure = False {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.860913 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cross_az_attach = True {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.861025 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] cinder.keyfile = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.861229 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] cinder.debug = False {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.861340 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] cinder.os_region_name = RegionOne {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.861541 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] cinder.endpoint_template = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.861652 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] cinder.split_loggers = False {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.861858 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] cinder.http_retries = 3 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.861969 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] cinder.timeout = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.862172 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] cinder.insecure = False {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.862291 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] compute.consecutive_build_service_disable_threshold = 10 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.862479 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_dedicated_set = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.862587 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] cinder.keyfile = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.862788 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_shared_set = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.862907 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] cinder.os_region_name = RegionOne {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.863125 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] compute.image_type_exclude_list = [] {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.863250 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] cinder.split_loggers = False {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.863437 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] cinder.timeout = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.863555 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] compute.live_migration_wait_for_vif_plug = True {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.863745 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_concurrent_disk_ops = 0 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.863854 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] compute.consecutive_build_service_disable_threshold = 10 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.864057 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_disk_devices_to_attach = -1 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.864178 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_dedicated_set = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.864386 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_shared_set = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.864466 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] compute.packing_host_numa_cells_allocation_strategy = True {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.864675 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] compute.image_type_exclude_list = [] {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.864759 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] compute.provider_config_location = /etc/nova/provider_config/ {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.865097 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] compute.resource_provider_association_refresh = 300 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.995236 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] compute.live_migration_wait_for_vif_plug = True {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.995236 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_concurrent_disk_ops = 0 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.995236 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_disk_devices_to_attach = -1 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.995236 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] compute.packing_host_numa_cells_allocation_strategy = True {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.995236 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] compute.provider_config_location = /etc/nova/provider_config/ {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.995236 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] compute.resource_provider_association_refresh = 300 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.995236 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] compute.shutdown_retry_interval = 10 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.995236 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] conductor.workers = 2 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.996021 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] console.allowed_origins = [] {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.996110 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] compute.shutdown_retry_interval = 10 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.996206 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_ciphers = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.996292 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] conductor.workers = 2 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.996380 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_minimum_version = default {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.996459 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] console.allowed_origins = [] {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.996459 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_ciphers = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.996611 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] consoleauth.token_ttl = 600 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.996698 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_minimum_version = default {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.996782 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.cafile = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.996867 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] consoleauth.token_ttl = 600 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.996951 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.certfile = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.997035 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.cafile = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.997119 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.collect_timing = False {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.997209 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.certfile = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.997293 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retries = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.997376 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.collect_timing = False {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.997459 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retry_delay = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.997546 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retries = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.997635 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.endpoint_override = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.997719 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retry_delay = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.997810 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.insecure = False {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.997898 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.endpoint_override = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.997982 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.keyfile = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.998071 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.insecure = False {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.998157 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.max_version = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.998236 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.keyfile = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.998318 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.min_version = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.998409 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.max_version = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.998493 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.region_name = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.998578 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.min_version = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.998667 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_name = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.998751 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.region_name = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.998840 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_type = accelerator {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.998920 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_name = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.999034 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.split_loggers = False {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.999129 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_type = accelerator {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.999224 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retries = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.999314 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.split_loggers = False {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.999402 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retry_delay = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.999486 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retries = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.999569 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.timeout = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.999662 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retry_delay = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.999662 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.timeout = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.999813 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.valid_interfaces = ['internal', 'public'] {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.999895 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.valid_interfaces = ['internal', 'public'] {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.999982 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.version = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:36.999982 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] database.backend = sqlalchemy {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.000143 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.version = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.000143 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] database.backend = sqlalchemy {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.000298 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] database.connection = **** {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.000377 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] database.connection = **** {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.000454 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_debug = 0 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.000536 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_debug = 0 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.000625 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_parameters = {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.000625 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_recycle_time = 3600 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.000786 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_parameters = {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.000871 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_trace = False {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.000959 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_recycle_time = 3600 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.001038 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] database.db_inc_retry_interval = True {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.001129 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_trace = False {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.001129 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] database.db_inc_retry_interval = True {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.001411 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retries = 20 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.001498 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retries = 20 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.001587 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retry_interval = 10 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.001755 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retry_interval = 10 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.001846 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] database.db_retry_interval = 1 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.001925 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] database.db_retry_interval = 1 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.001925 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] database.max_overflow = 50 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.002076 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] database.max_overflow = 50 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.002159 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] database.max_pool_size = 5 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.002248 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] database.max_pool_size = 5 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.002333 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] database.max_retries = 10 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.002418 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] database.max_retries = 10 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.002502 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_enable_ndb = False {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.002502 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_sql_mode = TRADITIONAL {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.002662 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_enable_ndb = False {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.002743 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] database.pool_timeout = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.002828 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_sql_mode = TRADITIONAL {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.002915 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] database.retry_interval = 10 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.002998 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] database.pool_timeout = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.008520 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] database.slave_connection = **** {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.008621 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] database.retry_interval = 10 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.008702 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] database.sqlite_synchronous = True {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.008789 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] database.slave_connection = **** {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.008876 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] api_database.backend = sqlalchemy {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.008952 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] database.sqlite_synchronous = True {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.009035 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection = **** {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.009119 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] api_database.backend = sqlalchemy {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.009209 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_debug = 0 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.009297 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection = **** {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.009375 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_parameters = {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.009451 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_debug = 0 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.009527 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_recycle_time = 3600 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.009611 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_parameters = {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.009694 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_trace = False {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.009828 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_recycle_time = 3600 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.010027 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_inc_retry_interval = True {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.010111 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_trace = False {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.010195 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retries = 20 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.010290 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_inc_retry_interval = True {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.010376 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retry_interval = 10 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.010454 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retries = 20 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.010537 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_retry_interval = 1 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.010603 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retry_interval = 10 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.010679 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_overflow = 50 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.010764 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_retry_interval = 1 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.010850 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_pool_size = 5 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.010935 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_overflow = 50 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.011031 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_retries = 10 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.011122 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_pool_size = 5 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.011202 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_enable_ndb = False {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.011287 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_retries = 10 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.011380 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_sql_mode = TRADITIONAL {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.011380 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] api_database.pool_timeout = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.011522 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_enable_ndb = False {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.011522 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_sql_mode = TRADITIONAL {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.011673 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] api_database.retry_interval = 10 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.011745 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] api_database.pool_timeout = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.011861 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] api_database.slave_connection = **** {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.011939 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] api_database.retry_interval = 10 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.012027 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] api_database.sqlite_synchronous = True {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.012106 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] api_database.slave_connection = **** {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.012191 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] devices.enabled_mdev_types = [] {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.012275 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] api_database.sqlite_synchronous = True {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.012352 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.cipher = aes-xts-plain64 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.012352 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.enabled = False {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.012503 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] devices.enabled_mdev_types = [] {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.012590 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.key_size = 512 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.012676 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.cipher = aes-xts-plain64 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.012743 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] glance.api_servers = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.012829 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.enabled = False {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.012902 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] glance.cafile = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.012988 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.key_size = 512 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.013074 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] glance.certfile = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.013162 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] glance.api_servers = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.013240 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] glance.collect_timing = False {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.013321 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] glance.cafile = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.013399 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retries = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.013471 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] glance.certfile = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.013560 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retry_delay = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.013639 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] glance.collect_timing = False {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.013721 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] glance.debug = False {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.013803 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retries = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.013881 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] glance.default_trusted_certificate_ids = [] {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.013957 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retry_delay = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.014046 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_certificate_validation = False {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.014130 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] glance.debug = False {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.014208 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_rbd_download = False {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.014284 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] glance.default_trusted_certificate_ids = [] {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.014361 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] glance.endpoint_override = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.014444 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_certificate_validation = False {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.014522 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] glance.insecure = False {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.014522 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] glance.keyfile = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.014673 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_rbd_download = False {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.014757 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] glance.max_version = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.014839 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] glance.endpoint_override = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.014924 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] glance.min_version = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.015037 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] glance.insecure = False {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.015129 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] glance.num_retries = 3 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.015208 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] glance.keyfile = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.015285 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_ceph_conf = {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.015369 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] glance.max_version = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.015453 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_connect_timeout = 5 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.015539 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] glance.min_version = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.015613 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_pool = {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.015691 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] glance.num_retries = 3 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.015774 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_user = {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.015774 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] glance.region_name = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.015929 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_ceph_conf = {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.015997 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_name = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.016079 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_connect_timeout = 5 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.016157 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_type = image {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.016157 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] glance.split_loggers = False {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.016302 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_pool = {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.016379 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retries = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.016463 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_user = {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.016547 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retry_delay = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.016631 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] glance.region_name = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.016710 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] glance.timeout = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.016792 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_name = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.016879 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] glance.valid_interfaces = ['internal', 'public'] {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.016964 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_type = image {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.017048 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] glance.verify_glance_signatures = False {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.017132 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] glance.split_loggers = False {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.017216 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] glance.version = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.017306 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retries = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.017392 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] guestfs.debug = False {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.017470 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retry_delay = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.017552 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_cdrom = False {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.017629 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] glance.timeout = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.017694 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_inject_password = False {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.017782 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] glance.valid_interfaces = ['internal', 'public'] {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.017867 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.dynamic_memory_ratio = 1.0 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.017950 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] glance.verify_glance_signatures = False {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.018037 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_instance_metrics_collection = False {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.018115 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] glance.version = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.018194 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_remotefx = False {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.018194 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.instances_path_share = {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.018352 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] guestfs.debug = False {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.018435 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.iscsi_initiator_list = [] {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.018522 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_cdrom = False {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.018601 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.limit_cpu_features = False {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.018677 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_inject_password = False {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.018761 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_count = 10 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.018835 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.dynamic_memory_ratio = 1.0 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.018912 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_interval = 5 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.018994 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_instance_metrics_collection = False {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.019121 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_check_timeframe = 60 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.019208 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_remotefx = False {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.019296 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_event_polling_interval = 2 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.019375 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.instances_path_share = {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.019457 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.qemu_img_cmd = qemu-img.exe {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.019457 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.use_multipath_io = False {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.019592 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.iscsi_initiator_list = [] {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.019680 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_count = 10 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.019758 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.limit_cpu_features = False {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.019833 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_interval = 5 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.019919 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_count = 10 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.020003 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.vswitch_name = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.020080 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_interval = 5 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.020158 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.wait_soft_reboot_seconds = 60 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.020158 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] mks.enabled = False {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.020307 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_check_timeframe = 60 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.020307 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_event_polling_interval = 2 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.020449 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] mks.mksproxy_base_url = http://127.0.0.1:6090/ {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.020514 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.qemu_img_cmd = qemu-img.exe {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.020601 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.manager_interval = 2400 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.020679 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.use_multipath_io = False {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.020761 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.precache_concurrency = 1 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.020761 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_base_images = True {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.020896 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_count = 10 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.020978 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_original_minimum_age_seconds = 86400 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.021066 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_interval = 5 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.021151 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_resized_minimum_age_seconds = 3600 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.021236 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.vswitch_name = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.021303 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.subdirectory_name = _base {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.021378 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.wait_soft_reboot_seconds = 60 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.021463 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_max_retries = 60 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.021542 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] mks.enabled = False {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.021633 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_retry_interval = 2 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.021633 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_section = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.021807 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] mks.mksproxy_base_url = http://127.0.0.1:6090/ {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.021885 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_type = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.021962 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.manager_interval = 2400 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.022039 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] ironic.cafile = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.022161 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.precache_concurrency = 1 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.022249 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] ironic.certfile = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.022329 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_base_images = True {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.022406 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] ironic.collect_timing = False {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.022492 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_original_minimum_age_seconds = 86400 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.022579 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retries = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.022653 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_resized_minimum_age_seconds = 3600 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.022731 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retry_delay = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.022813 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.subdirectory_name = _base {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.022891 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] ironic.endpoint_override = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.022961 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_max_retries = 60 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.023058 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] ironic.insecure = False {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.023199 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_retry_interval = 2 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.023297 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] ironic.keyfile = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.023387 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_section = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.023541 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] ironic.max_version = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.023668 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_type = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.023767 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] ironic.min_version = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.023767 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] ironic.partition_key = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.024014 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] ironic.cafile = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.024103 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] ironic.peer_list = [] {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.024181 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] ironic.certfile = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.024268 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] ironic.region_name = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.024346 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] ironic.collect_timing = False {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.024434 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] ironic.serial_console_state_timeout = 10 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.024520 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retries = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.024598 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_name = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.024675 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retry_delay = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.024757 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_type = baremetal {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.024866 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] ironic.endpoint_override = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.024961 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] ironic.split_loggers = False {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.025041 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] ironic.insecure = False {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.025118 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retries = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.025195 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] ironic.keyfile = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.025278 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retry_delay = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.025278 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] ironic.timeout = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.025443 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] ironic.max_version = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.025571 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] ironic.valid_interfaces = ['internal', 'public'] {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.025680 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] ironic.min_version = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.025769 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] ironic.version = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.025858 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] ironic.partition_key = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.025944 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.backend = nova.keymgr.conf_key_mgr.ConfKeyManager {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.026033 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] ironic.peer_list = [] {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.026111 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.fixed_key = **** {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.026238 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] ironic.region_name = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.026320 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.026411 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] ironic.serial_console_state_timeout = 10 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.026505 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_api_version = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.026591 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_name = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.026669 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.026746 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_type = baremetal {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.026822 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint_type = public {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.026898 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] ironic.split_loggers = False {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.026981 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_region_name = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.027081 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retries = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.027160 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] barbican.cafile = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.027236 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retry_delay = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.027322 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] barbican.certfile = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.027481 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] ironic.timeout = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.027569 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] barbican.collect_timing = False {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.027569 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] barbican.insecure = False {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.027712 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] ironic.valid_interfaces = ['internal', 'public'] {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.027834 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] barbican.keyfile = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.027938 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] ironic.version = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.028051 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] barbican.number_of_retries = 60 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.028161 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.backend = nova.keymgr.conf_key_mgr.ConfKeyManager {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.028266 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] barbican.retry_delay = 1 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.028368 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.fixed_key = **** {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.028480 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] barbican.send_service_user_token = False {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.028584 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.028679 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] barbican.split_loggers = False {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.028748 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_api_version = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.028836 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] barbican.timeout = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.028920 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.029005 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl = True {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.029095 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint_type = public {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.029174 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl_path = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.029246 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_region_name = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.029311 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_section = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.029374 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] barbican.cafile = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.029449 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_type = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.029534 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] barbican.certfile = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.029612 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.cafile = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.029612 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.certfile = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.029753 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] barbican.collect_timing = False {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.029753 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] barbican.insecure = False {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.029883 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.collect_timing = False {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.029883 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.insecure = False {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.030022 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] barbican.keyfile = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.030022 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] barbican.number_of_retries = 60 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.030173 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.keyfile = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.030257 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] barbican.retry_delay = 1 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.030257 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] barbican.send_service_user_token = False {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.030394 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.split_loggers = False {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.030472 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] barbican.split_loggers = False {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.030549 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.timeout = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.030705 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] barbican.timeout = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.030782 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_role_id = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.030857 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl = True {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.030941 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_secret_id = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.031063 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl_path = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.031499 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] vault.cafile = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.031746 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_section = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.031841 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] vault.certfile = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.031985 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_type = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.032069 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] vault.collect_timing = False {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.032147 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.cafile = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.032147 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.certfile = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.032342 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] vault.insecure = False {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.032424 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.collect_timing = False {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.032511 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] vault.keyfile = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.032591 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.insecure = False {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.032676 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_mountpoint = secret {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.032755 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.keyfile = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.032755 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.split_loggers = False {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.032901 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_version = 2 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.032987 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.timeout = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.033073 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] vault.namespace = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.033162 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_role_id = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.033162 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_secret_id = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.033307 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] vault.root_token_id = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.033386 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] vault.cafile = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.033471 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] vault.split_loggers = False {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.033549 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] vault.certfile = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.033620 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] vault.ssl_ca_crt_file = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.033710 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] vault.collect_timing = False {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.033837 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] vault.timeout = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.033941 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] vault.insecure = False {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.034022 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] vault.use_ssl = False {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.034100 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] vault.keyfile = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.034100 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_mountpoint = secret {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.034249 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] vault.vault_url = http://127.0.0.1:8200 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.034328 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_version = 2 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.034413 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] keystone.cafile = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.034492 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] vault.namespace = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.034605 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] keystone.certfile = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.034691 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] vault.root_token_id = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.034758 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] keystone.collect_timing = False {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.034883 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] vault.split_loggers = False {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.034975 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retries = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.035120 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] vault.ssl_ca_crt_file = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.035208 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retry_delay = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.035366 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] vault.timeout = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.035455 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] keystone.endpoint_override = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.035455 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] keystone.insecure = False {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.035609 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] vault.use_ssl = False {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.035689 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] keystone.keyfile = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.035771 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] vault.vault_url = http://127.0.0.1:8200 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.035850 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] keystone.max_version = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.035932 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] keystone.cafile = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.036019 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] keystone.min_version = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.036098 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] keystone.certfile = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.036183 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] keystone.region_name = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.036261 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] keystone.collect_timing = False {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.036337 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_name = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.036424 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retries = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.036583 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_type = identity {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.036665 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retry_delay = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.036742 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] keystone.split_loggers = False {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.036818 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] keystone.endpoint_override = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.036894 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retries = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.036980 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] keystone.insecure = False {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.037079 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retry_delay = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.037159 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] keystone.keyfile = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.037275 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] keystone.timeout = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.037372 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] keystone.max_version = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.037496 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] keystone.valid_interfaces = ['internal', 'public'] {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.037584 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] keystone.min_version = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.037671 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] keystone.version = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.037756 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] keystone.region_name = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.037834 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.connection_uri = {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.037910 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_name = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.037993 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_mode = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.038078 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_type = identity {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.038166 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_model_extra_flags = [] {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.038251 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] keystone.split_loggers = False {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.038362 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_models = [] {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.038450 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retries = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.038681 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_attempts = 8 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.038777 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retry_delay = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.038994 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] keystone.timeout = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.039167 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_timeout = 20 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.039336 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_cachemodes = [] {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.039422 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] keystone.valid_interfaces = ['internal', 'public'] {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.039923 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] keystone.version = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.040028 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_prefix = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.040242 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.enabled_perf_events = [] {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.040328 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.connection_uri = {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.041153 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_mode = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.041357 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.file_backed_memory = 0 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.041506 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.gid_maps = [] {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.041624 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_model_extra_flags = [] {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.041732 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_models = [] {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.041835 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_disk_discard = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.042086 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_attempts = 8 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.042189 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_machine_type = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.042366 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_timeout = 20 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.042465 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_ceph_conf = {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.042690 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_cachemodes = [] {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.042776 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_poll_interval = 15 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.043025 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_prefix = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.043127 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_timeout = 600 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.043416 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_store_name = {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.043522 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.enabled_perf_events = [] {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.043676 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_pool = rbd {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.043769 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.file_backed_memory = 0 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.044054 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_type = default {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.044155 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.gid_maps = [] {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.044315 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_volume_group = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.044404 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_disk_discard = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.044625 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_machine_type = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.044722 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_key = False {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.044937 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_partition = -2 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.045029 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_ceph_conf = {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.045249 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_password = False {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.045342 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_poll_interval = 15 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.045562 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iscsi_iface = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.045655 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_timeout = 600 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.045875 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iser_use_multipath = False {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.045973 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_store_name = {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.046193 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_bandwidth = 0 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.046334 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_pool = rbd {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.046508 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_completion_timeout = 800 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.046594 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_type = default {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.046821 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime = 500 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.046920 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_volume_group = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.047169 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_delay = 75 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.052440 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_key = False {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.052580 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_steps = 10 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.052675 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_partition = -2 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.052762 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_inbound_addr = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.052850 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_password = False {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.052930 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_auto_converge = False {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.053008 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iscsi_iface = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.053091 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_post_copy = False {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.053169 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iser_use_multipath = False {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.053247 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_scheme = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.053341 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_bandwidth = 0 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.053424 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_timeout_action = abort {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.053599 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_completion_timeout = 800 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.053681 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_tunnelled = False {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.053758 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime = 500 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.053845 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_uri = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.053924 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_delay = 75 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.054011 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_with_native_tls = False {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.054077 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_steps = 10 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.054141 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.max_queues = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.054218 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_inbound_addr = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.054303 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.mem_stats_period_seconds = 10 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.054383 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_auto_converge = False {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.054460 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_options = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.054543 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_post_copy = False {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.054543 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_scheme = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.054692 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.054778 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_timeout_action = abort {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.054857 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_aoe_discover_tries = 3 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.054933 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_tunnelled = False {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.055025 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_iser_scan_tries = 5 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.055121 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_uri = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.055200 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_memory_encrypted_guests = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.055285 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_with_native_tls = False {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.055371 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_nvme_discover_tries = 5 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.055450 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.max_queues = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.055532 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_pcie_ports = 0 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.055532 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_volume_scan_tries = 5 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.055681 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.mem_stats_period_seconds = 10 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.055811 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.pmem_namespaces = [] {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.055893 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_options = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.055972 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_client_cfg = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.056051 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.056149 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_mount_point_base = /opt/stack/data/nova/mnt {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.056855 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_aoe_discover_tries = 3 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.056954 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_connect_timeout = 5 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.057032 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_iser_scan_tries = 5 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.057105 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retries = 12 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.057171 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_memory_encrypted_guests = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.057250 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retry_interval = 5 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.057325 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_nvme_discover_tries = 5 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.057392 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_secret_uuid = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.057540 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_pcie_ports = 0 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.057635 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_user = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.057715 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_volume_scan_tries = 5 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.057782 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.realtime_scheduler_priority = 1 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.057861 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.pmem_namespaces = [] {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.057861 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_client_cfg = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.058004 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.remote_filesystem_transport = ssh {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.058004 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_image_id = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.058152 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_mount_point_base = /opt/stack/data/nova/mnt {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.058236 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_kernel_id = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.058327 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_connect_timeout = 5 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.058413 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_ramdisk_id = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.058491 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retries = 12 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.058574 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rng_dev_path = /dev/urandom {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.058653 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retry_interval = 5 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.058737 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rx_queue_size = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.058816 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_secret_uuid = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.058899 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_options = {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.058977 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_user = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.059174 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.059256 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.realtime_scheduler_priority = 1 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.059488 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_compression = False {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.059580 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.remote_filesystem_transport = ssh {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.059801 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_image_format = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.059894 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_image_id = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.060138 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_kernel_id = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.060284 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshots_directory = /opt/stack/data/nova/instances/snapshots {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.060456 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_ramdisk_id = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.060556 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sparse_logical_volumes = False {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.060787 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rng_dev_path = /dev/urandom {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.060880 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_enabled = False {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.061103 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rx_queue_size = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.061199 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_group = tss {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.061433 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_options = {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.061525 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_user = tss {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.061767 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sysinfo_serial = unique {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.061851 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.062078 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.tx_queue_size = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.062172 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_compression = False {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.062398 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.uid_maps = [] {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.062484 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_image_format = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.062741 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.use_virtio_for_bridges = True {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.062880 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshots_directory = /opt/stack/data/nova/instances/snapshots {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.063095 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.virt_type = kvm {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.063233 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sparse_logical_volumes = False {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.063426 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear = zero {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.063674 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_enabled = False {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.063776 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear_size = 0 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.063968 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_group = tss {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.064069 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_use_multipath = False {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.064253 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_user = tss {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.064395 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_cache_path = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.064581 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sysinfo_serial = unique {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.064680 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_log_path = /var/log/vstorage/%(cluster_name)s/nova.log.gz {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.064895 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.tx_queue_size = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.064983 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_group = qemu {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.065219 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.uid_maps = [] {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.065312 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_opts = [] {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.065539 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.use_virtio_for_bridges = True {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.065633 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_perms = 0770 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.065870 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.virt_type = kvm {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.065999 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_point_base = /opt/stack/data/nova/mnt {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.066200 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear = zero {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.066326 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_user = stack {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.066520 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear_size = 0 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.066641 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.wait_soft_reboot_seconds = 120 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.066843 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_use_multipath = False {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.066970 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_section = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.067184 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_cache_path = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.067330 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_type = password {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.067512 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_log_path = /var/log/vstorage/%(cluster_name)s/nova.log.gz {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.067639 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] neutron.cafile = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.067873 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_group = qemu {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.067967 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] neutron.certfile = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.068195 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_opts = [] {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.068292 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] neutron.collect_timing = False {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.068527 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_perms = 0770 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.068615 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retries = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.068835 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retry_delay = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.068956 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_point_base = /opt/stack/data/nova/mnt {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.069161 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] neutron.default_floating_pool = public {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.069288 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_user = stack {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.069471 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] neutron.endpoint_override = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.069605 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.wait_soft_reboot_seconds = 120 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.069786 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] neutron.extension_sync_interval = 600 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.069939 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_section = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.070100 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] neutron.http_retries = 3 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.070277 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_type = password {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.070413 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] neutron.insecure = False {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.070594 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] neutron.cafile = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.070720 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] neutron.keyfile = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.070908 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] neutron.certfile = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.071043 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] neutron.max_version = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.071832 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] neutron.collect_timing = False {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.071968 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] neutron.metadata_proxy_shared_secret = **** {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.072098 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retries = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.072330 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] neutron.min_version = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.072422 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retry_delay = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.072512 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] neutron.ovs_bridge = br-int {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.072592 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] neutron.default_floating_pool = public {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.072678 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] neutron.physnets = [] {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.072757 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] neutron.endpoint_override = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.072834 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] neutron.region_name = RegionOne {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.072910 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] neutron.extension_sync_interval = 600 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.073046 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_metadata_proxy = True {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.073156 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] neutron.http_retries = 3 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.073360 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_name = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.073475 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] neutron.insecure = False {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.073693 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_type = network {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.073932 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] neutron.keyfile = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.074017 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] neutron.split_loggers = False {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.074105 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] neutron.max_version = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.074302 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retries = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.074402 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] neutron.metadata_proxy_shared_secret = **** {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.074609 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retry_delay = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.074847 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] neutron.min_version = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.074991 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] neutron.timeout = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.075173 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] neutron.ovs_bridge = br-int {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.075404 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] neutron.valid_interfaces = ['internal', 'public'] {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.075498 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] neutron.physnets = [] {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.075716 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] neutron.version = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.075823 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] neutron.region_name = RegionOne {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.076044 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] notifications.bdms_in_notifications = False {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.076138 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_metadata_proxy = True {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.076381 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] notifications.default_level = INFO {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.076473 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_name = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.076716 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notification_format = unversioned {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.076853 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_type = network {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.077043 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] neutron.split_loggers = False {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.077127 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notify_on_state_change = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.077399 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retries = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.077495 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] notifications.versioned_notifications_topics = ['versioned_notifications'] {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.077693 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retry_delay = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.077789 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] pci.alias = [] {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.078006 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] neutron.timeout = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.078156 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] pci.passthrough_whitelist = [] {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.078329 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_section = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.078474 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] neutron.valid_interfaces = ['internal', 'public'] {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.078639 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] neutron.version = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.078734 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_type = password {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.078954 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] placement.cafile = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.079074 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] notifications.bdms_in_notifications = False {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.079290 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] placement.certfile = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.079389 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] notifications.default_level = INFO {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.079609 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] placement.collect_timing = False {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.079695 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notification_format = unversioned {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.079921 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retries = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.080152 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notify_on_state_change = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.080242 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retry_delay = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.080319 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] notifications.versioned_notifications_topics = ['versioned_notifications'] {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.080529 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] placement.endpoint_override = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.080636 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] pci.alias = [] {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.080833 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] placement.insecure = False {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.080919 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] pci.passthrough_whitelist = [] {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.081147 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] placement.keyfile = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.081244 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_section = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.081454 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] placement.max_version = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.081569 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_type = password {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.081887 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] placement.min_version = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.081980 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] placement.cafile = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.082098 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] placement.region_name = RegionOne {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.082193 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] placement.certfile = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.082400 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_name = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.082488 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] placement.collect_timing = False {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.082726 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_type = placement {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.082821 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retries = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.083060 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] placement.split_loggers = False {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.083294 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retry_delay = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.083376 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] placement.endpoint_override = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.083447 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retries = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.083694 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] placement.insecure = False {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.083779 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retry_delay = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.084020 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] placement.keyfile = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.084180 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] placement.timeout = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.084296 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] placement.max_version = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.084536 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] placement.valid_interfaces = ['internal', 'public'] {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.084621 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] placement.min_version = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.084688 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] placement.version = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.084928 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] placement.region_name = RegionOne {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.085014 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] powervm.disk_driver = localdisk {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.085238 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_name = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.085328 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] powervm.proc_units_factor = 0.1 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.085567 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_type = placement {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.085657 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] powervm.volume_group_name = {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.085888 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] placement.split_loggers = False {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.086078 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] quota.cores = 20 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.086379 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retries = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.086564 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] quota.count_usage_from_placement = False {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.086708 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retry_delay = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.086847 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] quota.driver = nova.quota.DbQuotaDriver {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.086999 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] placement.timeout = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.087212 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_content_bytes = 10240 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.087384 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] placement.valid_interfaces = ['internal', 'public'] {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.087476 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_path_length = 255 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.087598 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] placement.version = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.087678 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_files = 5 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.087945 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] powervm.disk_driver = localdisk {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.088079 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] quota.instances = 10 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.088273 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] powervm.proc_units_factor = 0.1 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.088580 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] quota.key_pairs = 100 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.088727 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] powervm.volume_group_name = {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.088874 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] quota.metadata_items = 128 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.089017 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] quota.cores = 20 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.089152 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] quota.ram = 51200 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.089306 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] quota.recheck_quota = True {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.089441 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] quota.count_usage_from_placement = False {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.089578 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_group_members = 10 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.089721 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] quota.driver = nova.quota.DbQuotaDriver {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.089856 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_groups = 10 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.089998 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_content_bytes = 10240 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.090139 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] rdp.enabled = False {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.090304 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_path_length = 255 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.090476 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_files = 5 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.090708 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] rdp.html5_proxy_base_url = http://127.0.0.1:6083/ {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.090796 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] quota.instances = 10 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.091083 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.discover_hosts_in_cells_interval = -1 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.091175 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] quota.key_pairs = 100 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.091412 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.enable_isolated_aggregate_filtering = False {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.091497 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] quota.metadata_items = 128 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.091736 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.image_metadata_prefilter = False {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.091827 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] quota.ram = 51200 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.092055 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.limit_tenants_to_placement_aggregate = False {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.092148 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] quota.recheck_quota = True {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.092380 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_attempts = 3 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.092484 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_group_members = 10 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.092697 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_placement_results = 1000 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.092784 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_groups = 10 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.093017 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.placement_aggregate_required_for_tenants = False {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.093112 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] rdp.enabled = False {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.093333 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_availability_zone = True {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.093589 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] rdp.html5_proxy_base_url = http://127.0.0.1:6083/ {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.093677 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_image_type_support = False {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.093926 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.discover_hosts_in_cells_interval = -1 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.094012 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_routed_network_aggregates = False {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.094249 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.enable_isolated_aggregate_filtering = False {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.094338 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.workers = 2 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.094574 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.image_metadata_prefilter = False {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.094669 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_namespace = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.094896 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.limit_tenants_to_placement_aggregate = False {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.094982 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_separator = . {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.095247 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_attempts = 3 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.098574 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.098725 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_placement_results = 1000 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.098822 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.build_failure_weight_multiplier = 1000000.0 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.098910 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.placement_aggregate_required_for_tenants = False {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.098991 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cpu_weight_multiplier = 1.0 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.099100 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_availability_zone = True {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.099194 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.099275 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_image_type_support = False {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.099354 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.disk_weight_multiplier = 1.0 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.099438 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_routed_network_aggregates = False {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.099519 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.enabled_filters = ['AvailabilityZoneFilter', 'ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter', 'SameHostFilter', 'DifferentHostFilter'] {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.099601 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.workers = 2 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.099698 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.host_subset_size = 1 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.099788 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_namespace = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.099875 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.image_properties_default_architecture = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.099955 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_separator = . {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.100034 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.io_ops_weight_multiplier = -1.0 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.100120 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.100200 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_hosts = [] {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.100277 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.build_failure_weight_multiplier = 1000000.0 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.100343 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_images = [] {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.100343 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_instances_per_host = 50 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.100468 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cpu_weight_multiplier = 1.0 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.100541 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_io_ops_per_host = 8 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.100628 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.100707 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.pci_weight_multiplier = 1.0 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.100783 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.disk_weight_multiplier = 1.0 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.100860 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.ram_weight_multiplier = 1.0 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.100944 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.enabled_filters = ['AvailabilityZoneFilter', 'ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter', 'SameHostFilter', 'DifferentHostFilter'] {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.101033 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.101113 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.host_subset_size = 1 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.101190 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.shuffle_best_same_weighed_hosts = False {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.101274 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.image_properties_default_architecture = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.101352 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_affinity_weight_multiplier = 1.0 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.101437 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.io_ops_weight_multiplier = -1.0 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.101516 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.101600 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_hosts = [] {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.101679 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.track_instance_changes = False {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.101756 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_images = [] {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.101842 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.101930 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_instances_per_host = 50 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.102015 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] metrics.required = True {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.102094 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_io_ops_per_host = 8 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.102171 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_multiplier = 1.0 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.102248 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.pci_weight_multiplier = 1.0 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.102370 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_of_unavailable = -10000.0 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.102458 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.ram_weight_multiplier = 1.0 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.102686 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_setting = [] {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.102778 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.103570 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.shuffle_best_same_weighed_hosts = False {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.103900 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_affinity_weight_multiplier = 1.0 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.104228 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.104565 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.track_instance_changes = False {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.104912 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.105237 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] metrics.required = True {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.105579 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_multiplier = 1.0 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.105956 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_of_unavailable = -10000.0 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.106065 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.base_url = ws://127.0.0.1:6083/ {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.106313 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.enabled = False {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.106408 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_setting = [] {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.106663 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.port_range = 10000:20000 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.106872 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.base_url = ws://127.0.0.1:6083/ {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.107038 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.proxyclient_address = 127.0.0.1 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.107222 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.enabled = False {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.107363 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_host = 0.0.0.0 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.107562 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.port_range = 10000:20000 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.107696 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_port = 6083 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.107894 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.proxyclient_address = 127.0.0.1 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.108026 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_section = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.108229 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_host = 0.0.0.0 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.108367 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_type = password {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.108552 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_port = 6083 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.108685 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] service_user.cafile = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.108881 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_section = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.109000 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] service_user.certfile = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.109242 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_type = password {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.109374 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] service_user.collect_timing = False {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.109558 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] service_user.cafile = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.109652 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] service_user.insecure = False {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.109868 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] service_user.certfile = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.109962 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] service_user.keyfile = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.110192 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] service_user.collect_timing = False {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.110368 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] service_user.send_service_user_token = True {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.110549 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] service_user.insecure = False {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.110651 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] service_user.split_loggers = False {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.110868 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] service_user.keyfile = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.110956 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] service_user.timeout = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.111223 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] service_user.send_service_user_token = True {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.111321 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] spice.agent_enabled = True {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.111541 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] service_user.split_loggers = False {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.111629 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] spice.enabled = False {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.111852 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] service_user.timeout = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.112139 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_base_url = http://127.0.0.1:6082/spice_auto.html {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.112265 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] spice.agent_enabled = True {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.112479 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] spice.enabled = False {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.112568 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_host = 0.0.0.0 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.112802 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_port = 6082 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.113023 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_base_url = http://127.0.0.1:6082/spice_auto.html {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.113118 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_listen = 127.0.0.1 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.113404 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_host = 0.0.0.0 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.113498 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_proxyclient_address = 127.0.0.1 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.113738 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_port = 6082 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.113828 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.baseapi = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.114058 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.cert = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.114157 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_listen = 127.0.0.1 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.114384 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.compute = auto {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.114471 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_proxyclient_address = 127.0.0.1 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.114697 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.conductor = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.114792 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.baseapi = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.115063 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.scheduler = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.117617 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.cert = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.117747 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_section = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.117835 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.compute = auto {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.117922 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_type = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.118011 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.conductor = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.118098 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.cafile = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.118178 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.scheduler = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.118269 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.certfile = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.118350 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_section = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.118427 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.collect_timing = False {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.118512 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_type = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.118702 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.insecure = False {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.118791 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.cafile = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.119374 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.keyfile = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.119454 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.certfile = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.119666 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.split_loggers = False {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.119775 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.collect_timing = False {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.119857 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.timeout = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.119943 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.insecure = False {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.120028 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] vmware.api_retry_count = 10 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.120107 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.keyfile = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.120193 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] vmware.ca_file = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.120272 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.split_loggers = False {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.120357 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cache_prefix = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.120437 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.timeout = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.120503 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cluster_name = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.120567 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] vmware.api_retry_count = 10 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.120696 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] vmware.connection_pool_size = 10 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.120784 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] vmware.ca_file = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.120851 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] vmware.console_delay_seconds = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.120932 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cache_prefix = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.121018 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] vmware.datastore_regex = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.121096 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cluster_name = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.121186 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_ip = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.121265 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] vmware.connection_pool_size = 10 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.121341 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_password = **** {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.121418 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] vmware.console_delay_seconds = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.121494 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_port = 443 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.121570 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] vmware.datastore_regex = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.121656 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_username = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.121735 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_ip = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.121936 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] vmware.insecure = False {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.122036 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_password = **** {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.122244 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] vmware.integration_bridge = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.122339 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_port = 443 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.122558 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] vmware.maximum_objects = 100 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.122644 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_username = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.122864 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_default_policy = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.122950 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] vmware.insecure = False {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.123228 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_enabled = False {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.123396 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] vmware.integration_bridge = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.123565 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] vmware.maximum_objects = 100 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.123651 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_wsdl_location = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.123880 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_default_policy = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.123966 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_log_dir = /opt/vmware/vspc {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.124193 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_proxy_uri = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.124285 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_enabled = False {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.124499 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_service_uri = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.124592 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_wsdl_location = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.124818 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] vmware.task_poll_interval = 0.5 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.124905 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_log_dir = /opt/vmware/vspc {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.125132 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] vmware.use_linked_clone = True {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.125237 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_proxy_uri = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.125455 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_keymap = en-us {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.125561 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_service_uri = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.125767 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port = 5900 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.125853 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] vmware.task_poll_interval = 0.5 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.126080 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] vmware.use_linked_clone = True {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.126166 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port_total = 10000 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.126467 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] vnc.auth_schemes = ['none'] {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.126599 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_keymap = en-us {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.126747 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port = 5900 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.126943 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] vnc.enabled = True {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.127151 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port_total = 10000 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.127319 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_base_url = http://127.0.0.1:6080/vnc_auto.html {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.127431 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] vnc.auth_schemes = ['none'] {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.127655 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_host = 0.0.0.0 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.127742 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] vnc.enabled = True {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.127981 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_port = 6080 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.128317 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_listen = 127.0.0.1 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.128413 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_base_url = http://127.0.0.1:6080/vnc_auto.html {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.128651 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_proxyclient_address = 127.0.0.1 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.128745 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_host = 0.0.0.0 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.128961 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_ca_certs = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.129047 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_port = 6080 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.129270 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_cert = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.129358 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_listen = 127.0.0.1 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.129579 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_key = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.129666 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_proxyclient_address = 127.0.0.1 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.129904 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_compute_service_check_for_ffu = False {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.130003 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_ca_certs = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.130216 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_fallback_pcpu_query = False {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.130301 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_cert = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.130529 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_group_policy_check_upcall = False {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.130614 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_key = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.130843 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_libvirt_livesnapshot = False {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.130947 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_compute_service_check_for_ffu = False {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.131176 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_rootwrap = False {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.131266 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_fallback_pcpu_query = False {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.131489 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_numa_live_migration = False {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.131575 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_group_policy_check_upcall = False {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.131801 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_qemu_monitor_announce_self = False {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.131904 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_libvirt_livesnapshot = False {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.132117 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.132237 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_rootwrap = False {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.132427 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.handle_virt_lifecycle_events = True {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.132519 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_numa_live_migration = False {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.132739 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_qemu_monitor_announce_self = False {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.132826 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.libvirt_disable_apic = False {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.133047 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.never_download_image_if_on_rbd = False {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.133131 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.133363 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.handle_virt_lifecycle_events = True {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.133495 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.reserve_disk_resource_for_image_cache = False {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.133671 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_cpu_compare_on_dest = False {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.133759 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.libvirt_disable_apic = False {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.133982 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.unified_limits_count_pcpu_as_vcpu = False {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.134067 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.never_download_image_if_on_rbd = False {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.134317 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.reserve_disk_resource_for_image_cache = False {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.134406 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.wait_for_vif_plugged_event_during_hard_reboot = [] {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.134634 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_cpu_compare_on_dest = False {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.134896 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.api_paste_config = /etc/nova/api-paste.ini {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.134997 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.unified_limits_count_pcpu_as_vcpu = False {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.135259 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.client_socket_timeout = 900 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.135366 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.wait_for_vif_plugged_event_during_hard_reboot = [] {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.135555 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.default_pool_size = 1000 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.135692 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.api_paste_config = /etc/nova/api-paste.ini {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.135882 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.keep_alive = True {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.136014 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.client_socket_timeout = 900 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.136208 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.max_header_line = 16384 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.136355 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.default_pool_size = 1000 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.136527 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.secure_proxy_ssl_header = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.136665 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.keep_alive = True {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.136843 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_ca_file = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.136986 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.max_header_line = 16384 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.137159 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_cert_file = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.137295 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.secure_proxy_ssl_header = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.137496 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_key_file = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.137632 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_ca_file = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.137793 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.tcp_keepidle = 600 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.137924 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_cert_file = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.138108 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_key_file = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.138244 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.138430 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.tcp_keepidle = 600 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.138567 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] zvm.ca_file = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.138737 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] zvm.cloud_connector_url = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.138873 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.139092 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] zvm.ca_file = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.139196 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] zvm.image_tmp_path = /opt/stack/data/nova/images {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.139404 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] zvm.cloud_connector_url = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.139491 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] zvm.reachable_timeout = 300 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.139806 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.139892 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] zvm.image_tmp_path = /opt/stack/data/nova/images {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.140132 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.140228 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] zvm.reachable_timeout = 300 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.140447 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.140533 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.140772 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.140905 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_process_name = {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.141090 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.141190 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.141416 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_process_name = {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.141502 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.141744 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.141830 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_new_defaults = False {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.142061 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.142148 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_scope = False {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.142388 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_new_defaults = False {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.142473 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_default_rule = default {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.142703 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_scope = False {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.142795 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.143057 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_default_rule = default {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.143148 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_file = policy.yaml {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.143406 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.143491 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.143722 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.143822 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_file = policy.yaml {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.144032 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.144119 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.144340 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.144433 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.144653 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.144750 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.144974 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.145059 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_section = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.145283 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_type = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.145367 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.145592 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.cafile = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.145696 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_section = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.145897 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.certfile = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.145993 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_type = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.146203 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.cafile = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.146289 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.collect_timing = False {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.146505 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retries = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.146602 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.certfile = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.146810 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retry_delay = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.146896 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.collect_timing = False {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.147138 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_id = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.147230 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retries = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.147447 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retry_delay = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.147548 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_override = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.147754 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.insecure = False {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.147838 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_id = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.148057 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.keyfile = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.148150 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_override = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.148364 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.max_version = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.148451 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.insecure = False {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.148668 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.min_version = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.148753 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.keyfile = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.148974 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.max_version = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.149057 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.region_name = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.149295 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_name = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.149403 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.min_version = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.149583 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_type = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.149684 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.region_name = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.149956 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.split_loggers = False {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.150086 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_name = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.150283 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retries = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.150370 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_type = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.150592 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retry_delay = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.150724 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.split_loggers = False {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.150904 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.timeout = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.151098 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retries = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.151236 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.valid_interfaces = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.151372 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retry_delay = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.151541 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.version = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.151683 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.timeout = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.151876 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.151999 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.valid_interfaces = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.152213 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] profiler.connection_string = messaging:// {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.152299 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.version = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.152529 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] profiler.enabled = False {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.152615 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.152857 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_doc_type = notification {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.153022 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] profiler.connection_string = messaging:// {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.153175 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_size = 10000 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.153263 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] profiler.enabled = False {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.153499 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_time = 2m {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.153589 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_doc_type = notification {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.153810 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] profiler.filter_error_trace = False {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.153910 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_size = 10000 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.154131 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] profiler.hmac_keys = SECRET_KEY {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.154224 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_time = 2m {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.154453 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] profiler.sentinel_service_name = mymaster {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.154548 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] profiler.filter_error_trace = False {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.154770 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] profiler.socket_timeout = 0.1 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.154871 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] profiler.hmac_keys = SECRET_KEY {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.155104 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] profiler.trace_sqlalchemy = False {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.155196 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] profiler.sentinel_service_name = mymaster {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.157845 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.157993 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] profiler.socket_timeout = 0.1 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.158087 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.158167 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] profiler.trace_sqlalchemy = False {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.158245 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.158329 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.158416 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.158495 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.158581 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.158661 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.158739 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.158817 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.158894 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_in_pthread = True {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.158982 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.159142 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.159227 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.159313 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 600 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.159392 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_in_pthread = True {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.159469 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.159555 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.159642 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.159721 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 600 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.159799 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.159877 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.159954 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.160043 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.160130 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.160209 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.160287 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.160438 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.160526 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.160603 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.160682 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.160759 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.160845 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_delivery_limit = 0 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.160924 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.161014 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.161132 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.161258 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_bytes = 0 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.161451 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_delivery_limit = 0 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.161539 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_length = 0 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.161769 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.161869 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.162088 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_bytes = 0 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.162174 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.162397 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.162489 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_length = 0 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.162714 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.162813 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.163057 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.163165 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl = False {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.163364 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.163459 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.163691 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.163814 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.164005 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.164101 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl = False {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.164331 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.164415 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.164904 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_version = {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.164998 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.165224 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.165307 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.driver = ['messagingv2'] {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.166100 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.166199 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.retry = -1 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.166577 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.166767 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_version = {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.166945 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.transport_url = **** {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.167093 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.driver = ['messagingv2'] {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.167261 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.167371 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.retry = -1 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.167552 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler_interval = 1 {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.167729 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.167849 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.log_dir = None {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.168070 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.transport_url = **** {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.168183 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=122766) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 16 20:56:37.168398 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.168721 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler_interval = 1 {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.169065 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.log_dir = None {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 20:56:37.169318 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=122767) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 16 20:56:37.234346 ncdv-master-1 devstack@n-api.service[122766]: INFO dbcounter [None req-ba6d99ed-058e-4ecf-a296-91ac2c64ec19 None None] Registered counter for database nova_api Jul 16 20:56:37.235352 ncdv-master-1 devstack@n-api.service[122767]: INFO dbcounter [None req-f91db710-7d5d-436a-bec3-1d06bcfa51cd None None] Registered counter for database nova_api Jul 16 20:56:37.255885 ncdv-master-1 devstack@n-api.service[122766]: DEBUG oslo_db.sqlalchemy.engines [None req-ba6d99ed-058e-4ecf-a296-91ac2c64ec19 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=122766) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 16 20:56:37.257470 ncdv-master-1 devstack@n-api.service[122767]: DEBUG oslo_db.sqlalchemy.engines [None req-f91db710-7d5d-436a-bec3-1d06bcfa51cd None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=122767) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 16 20:56:37.263567 ncdv-master-1 devstack@n-api.service[122766]: DEBUG dbcounter [-] [122766] Writer thread running {{(pid=122766) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 16 20:56:37.264535 ncdv-master-1 devstack@n-api.service[122767]: DEBUG dbcounter [-] [122767] Writer thread running {{(pid=122767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 16 20:56:37.321686 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.context [None req-ba6d99ed-058e-4ecf-a296-91ac2c64ec19 None None] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),327e8df1-39cc-4b36-9ea6-6a9b88ed54fc(cell1) {{(pid=122766) load_cells /opt/stack/nova/nova/context.py:464}} Jul 16 20:56:37.322936 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.context [None req-f91db710-7d5d-436a-bec3-1d06bcfa51cd None None] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),327e8df1-39cc-4b36-9ea6-6a9b88ed54fc(cell1) {{(pid=122767) load_cells /opt/stack/nova/nova/context.py:464}} Jul 16 20:56:37.325969 ncdv-master-1 devstack@n-api.service[122766]: DEBUG oslo_concurrency.lockutils [None req-ba6d99ed-058e-4ecf-a296-91ac2c64ec19 None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=122766) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 20:56:37.327126 ncdv-master-1 devstack@n-api.service[122767]: DEBUG oslo_concurrency.lockutils [None req-f91db710-7d5d-436a-bec3-1d06bcfa51cd None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=122767) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 20:56:37.327596 ncdv-master-1 devstack@n-api.service[122766]: DEBUG oslo_concurrency.lockutils [None req-ba6d99ed-058e-4ecf-a296-91ac2c64ec19 None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=122766) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 20:56:37.328395 ncdv-master-1 devstack@n-api.service[122766]: DEBUG oslo_concurrency.lockutils [None req-ba6d99ed-058e-4ecf-a296-91ac2c64ec19 None None] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=122766) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 20:56:37.328723 ncdv-master-1 devstack@n-api.service[122767]: DEBUG oslo_concurrency.lockutils [None req-f91db710-7d5d-436a-bec3-1d06bcfa51cd None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=122767) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 20:56:37.329545 ncdv-master-1 devstack@n-api.service[122767]: DEBUG oslo_concurrency.lockutils [None req-f91db710-7d5d-436a-bec3-1d06bcfa51cd None None] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=122767) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 20:56:37.330369 ncdv-master-1 devstack@n-api.service[122766]: DEBUG oslo_concurrency.lockutils [None req-ba6d99ed-058e-4ecf-a296-91ac2c64ec19 None None] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=122766) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 20:56:37.331551 ncdv-master-1 devstack@n-api.service[122767]: DEBUG oslo_concurrency.lockutils [None req-f91db710-7d5d-436a-bec3-1d06bcfa51cd None None] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=122767) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 20:56:37.332410 ncdv-master-1 devstack@n-api.service[122766]: INFO dbcounter [None req-ba6d99ed-058e-4ecf-a296-91ac2c64ec19 None None] Registered counter for database nova_cell0 Jul 16 20:56:37.333564 ncdv-master-1 devstack@n-api.service[122767]: INFO dbcounter [None req-f91db710-7d5d-436a-bec3-1d06bcfa51cd None None] Registered counter for database nova_cell0 Jul 16 20:56:37.336473 ncdv-master-1 devstack@n-api.service[122766]: INFO dbcounter [None req-ba6d99ed-058e-4ecf-a296-91ac2c64ec19 None None] Registered counter for database nova_cell1 Jul 16 20:56:37.338310 ncdv-master-1 devstack@n-api.service[122767]: INFO dbcounter [None req-f91db710-7d5d-436a-bec3-1d06bcfa51cd None None] Registered counter for database nova_cell1 Jul 16 20:56:37.344508 ncdv-master-1 devstack@n-api.service[122766]: DEBUG oslo_db.sqlalchemy.engines [None req-ba6d99ed-058e-4ecf-a296-91ac2c64ec19 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=122766) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 16 20:56:37.348078 ncdv-master-1 devstack@n-api.service[122766]: DEBUG oslo_db.sqlalchemy.engines [None req-ba6d99ed-058e-4ecf-a296-91ac2c64ec19 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=122766) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 16 20:56:37.348474 ncdv-master-1 devstack@n-api.service[122767]: DEBUG oslo_db.sqlalchemy.engines [None req-f91db710-7d5d-436a-bec3-1d06bcfa51cd None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=122767) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 16 20:56:37.350924 ncdv-master-1 devstack@n-api.service[122767]: DEBUG oslo_db.sqlalchemy.engines [None req-f91db710-7d5d-436a-bec3-1d06bcfa51cd None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=122767) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 16 20:56:37.355795 ncdv-master-1 devstack@n-api.service[122766]: DEBUG dbcounter [-] [122766] Writer thread running {{(pid=122766) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 16 20:56:37.356701 ncdv-master-1 devstack@n-api.service[122767]: DEBUG dbcounter [-] [122767] Writer thread running {{(pid=122767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 16 20:56:37.357139 ncdv-master-1 devstack@n-api.service[122766]: DEBUG dbcounter [-] [122766] Writer thread running {{(pid=122766) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 16 20:56:37.359215 ncdv-master-1 devstack@n-api.service[122767]: DEBUG dbcounter [-] [122767] Writer thread running {{(pid=122767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 16 20:56:37.501461 ncdv-master-1 devstack@n-api.service[122766]: INFO dbcounter [None req-ba6d99ed-058e-4ecf-a296-91ac2c64ec19 None None] Registered counter for database nova_cell0 Jul 16 20:56:37.504400 ncdv-master-1 devstack@n-api.service[122767]: INFO dbcounter [None req-f91db710-7d5d-436a-bec3-1d06bcfa51cd None None] Registered counter for database nova_cell0 Jul 16 20:56:37.510681 ncdv-master-1 devstack@n-api.service[122766]: DEBUG oslo_db.sqlalchemy.engines [None req-ba6d99ed-058e-4ecf-a296-91ac2c64ec19 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=122766) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 16 20:56:37.514524 ncdv-master-1 devstack@n-api.service[122767]: DEBUG oslo_db.sqlalchemy.engines [None req-f91db710-7d5d-436a-bec3-1d06bcfa51cd None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=122767) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 16 20:56:37.516097 ncdv-master-1 devstack@n-api.service[122766]: DEBUG dbcounter [-] [122766] Writer thread running {{(pid=122766) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 16 20:56:37.518842 ncdv-master-1 devstack@n-api.service[122767]: DEBUG dbcounter [-] [122767] Writer thread running {{(pid=122767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 16 20:56:37.925592 ncdv-master-1 devstack@n-api.service[122767]: WARNING keystonemiddleware.auth_token [None req-f91db710-7d5d-436a-bec3-1d06bcfa51cd None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Jul 16 20:56:37.926590 ncdv-master-1 devstack@n-api.service[122766]: WARNING keystonemiddleware.auth_token [None req-ba6d99ed-058e-4ecf-a296-91ac2c64ec19 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Jul 16 20:56:37.933987 ncdv-master-1 devstack@n-api.service[122767]: WARNING keystonemiddleware.auth_token [None req-f91db710-7d5d-436a-bec3-1d06bcfa51cd None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Jul 16 20:56:37.934437 ncdv-master-1 devstack@n-api.service[122766]: WARNING keystonemiddleware.auth_token [None req-ba6d99ed-058e-4ecf-a296-91ac2c64ec19 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Jul 16 20:56:38.277034 ncdv-master-1 devstack@n-api.service[122766]: WARNING keystonemiddleware.auth_token [None req-ba6d99ed-058e-4ecf-a296-91ac2c64ec19 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Jul 16 20:56:38.282950 ncdv-master-1 devstack@n-api.service[122766]: WARNING keystonemiddleware.auth_token [None req-ba6d99ed-058e-4ecf-a296-91ac2c64ec19 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Jul 16 20:56:38.289260 ncdv-master-1 devstack@n-api.service[122766]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55e2e22a79c0 pid: 122766 (default app) Jul 16 20:56:38.292202 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi [None req-ba6d99ed-058e-4ecf-a296-91ac2c64ec19 None None] Calling method '>' {{(pid=122766) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 20:56:38.293377 ncdv-master-1 devstack@n-api.service[122766]: INFO nova.api.openstack.requestlog [None req-ba6d99ed-058e-4ecf-a296-91ac2c64ec19 None None] 10.222.0.16 "GET /compute/v2.1/" status: 200 len: 388 microversion: 2.1 time: 0.001753 Jul 16 20:56:38.293775 ncdv-master-1 devstack@n-api.service[122766]: [pid: 122766|app: 0|req: 1/1] 10.222.0.16 () {50 vars in 788 bytes} [Sat Jul 16 20:56:38 2022] GET /compute/v2.1/ => generated 388 bytes in 4 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 20:56:38.295754 ncdv-master-1 devstack@n-api.service[122767]: WARNING keystonemiddleware.auth_token [None req-f91db710-7d5d-436a-bec3-1d06bcfa51cd None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Jul 16 20:56:38.302257 ncdv-master-1 devstack@n-api.service[122767]: WARNING keystonemiddleware.auth_token [None req-f91db710-7d5d-436a-bec3-1d06bcfa51cd None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Jul 16 20:56:38.309390 ncdv-master-1 devstack@n-api.service[122767]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55e2e22a79c0 pid: 122767 (default app) Jul 16 20:58:04.982786 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi [None req-ba6d99ed-058e-4ecf-a296-91ac2c64ec19 None None] Calling method '>' {{(pid=122766) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 20:58:04.983821 ncdv-master-1 devstack@n-api.service[122766]: INFO nova.api.openstack.requestlog [None req-ba6d99ed-058e-4ecf-a296-91ac2c64ec19 None None] 10.222.0.16 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001467 Jul 16 20:58:04.984418 ncdv-master-1 devstack@n-api.service[122766]: [pid: 122766|app: 0|req: 2/2] 10.222.0.16 () {54 vars in 930 bytes} [Sat Jul 16 20:58:04 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 20:58:04.997614 ncdv-master-1 devstack@n-api.service[122767]: DEBUG dbcounter [-] [122767] Writing DB stats nova_api:SELECT=3 {{(pid=122767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 20:58:04.999708 ncdv-master-1 devstack@n-api.service[122767]: DEBUG dbcounter [-] [122767] Writing DB stats nova_cell0:SELECT=3 {{(pid=122767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 20:58:05.000678 ncdv-master-1 devstack@n-api.service[122767]: DEBUG dbcounter [-] [122767] Writing DB stats nova_cell1:SELECT=3 {{(pid=122767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 20:58:05.001610 ncdv-master-1 devstack@n-api.service[122767]: DEBUG dbcounter [-] [122767] Writing DB stats nova_cell0:SELECT=6,nova_cell0:INSERT=1 {{(pid=122767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 20:58:05.974445 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi [None req-47dff7b6-cd95-4e64-8db0-dc793b479254 admin admin] Calling method '>' {{(pid=122767) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 20:58:06.004739 ncdv-master-1 devstack@n-api.service[122767]: INFO nova.api.openstack.requestlog [None req-47dff7b6-cd95-4e64-8db0-dc793b479254 admin admin] 10.222.0.16 "GET /compute/v2.1/flavors/detail?is_public=True" status: 200 len: 15 microversion: 2.61 time: 1.012460 Jul 16 20:58:06.005785 ncdv-master-1 devstack@n-api.service[122767]: [pid: 122767|app: 0|req: 1/3] 10.222.0.16 () {60 vars in 1291 bytes} [Sat Jul 16 20:58:04 2022] GET /compute/v2.1/flavors/detail?is_public=True => generated 15 bytes in 1015 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 20:58:08.558079 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi [None req-ba6d99ed-058e-4ecf-a296-91ac2c64ec19 None None] Calling method '>' {{(pid=122766) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 20:58:08.559997 ncdv-master-1 devstack@n-api.service[122766]: INFO nova.api.openstack.requestlog [None req-ba6d99ed-058e-4ecf-a296-91ac2c64ec19 None None] 10.222.0.16 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.002407 Jul 16 20:58:08.563233 ncdv-master-1 devstack@n-api.service[122766]: [pid: 122766|app: 0|req: 3/4] 10.222.0.16 () {54 vars in 930 bytes} [Sat Jul 16 20:58:08 2022] GET /compute/v2.1 => generated 388 bytes in 5 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 20:58:09.050336 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi [None req-53edf0c4-0dd5-4fd0-8aa8-055e15be03be admin admin] Action: 'create', calling method: >, body: {"flavor": {"disk": 1, "name": "cirros256", "os-flavor-access:is_public": true, "id": "c1", "rxtx_factor": 1.0, "vcpus": 1, "OS-FLV-EXT-DATA:ephemeral": 0, "swap": 0, "ram": 256}} {{(pid=122767) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 20:58:09.071068 ncdv-master-1 devstack@n-api.service[122767]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-flavor-manage:create": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 20:58:09.071068 ncdv-master-1 devstack@n-api.service[122767]: warnings.warn(msg) Jul 16 20:58:09.160247 ncdv-master-1 devstack@n-api.service[122767]: INFO nova.api.openstack.requestlog [None req-53edf0c4-0dd5-4fd0-8aa8-055e15be03be admin admin] 10.222.0.16 "POST /compute/v2.1/flavors" status: 200 len: 407 microversion: 2.61 time: 0.592377 Jul 16 20:58:09.160814 ncdv-master-1 devstack@n-api.service[122767]: [pid: 122767|app: 0|req: 2/5] 10.222.0.16 () {64 vars in 1282 bytes} [Sat Jul 16 20:58:08 2022] POST /compute/v2.1/flavors => generated 407 bytes in 594 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 16 20:58:09.176144 ncdv-master-1 devstack@n-api.service[122766]: DEBUG dbcounter [-] [122766] Writing DB stats nova_api:SELECT=3 {{(pid=122766) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 20:58:09.177971 ncdv-master-1 devstack@n-api.service[122766]: DEBUG dbcounter [-] [122766] Writing DB stats nova_cell0:SELECT=3 {{(pid=122766) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 20:58:09.178979 ncdv-master-1 devstack@n-api.service[122766]: DEBUG dbcounter [-] [122766] Writing DB stats nova_cell1:SELECT=3 {{(pid=122766) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 20:58:09.180092 ncdv-master-1 devstack@n-api.service[122766]: DEBUG dbcounter [-] [122766] Writing DB stats nova_cell0:SELECT=6,nova_cell0:INSERT=1 {{(pid=122766) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 20:58:09.392638 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi [None req-e9b9cbb8-701d-4090-9cc9-fbf9e2a0759c admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=122766) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 20:58:09.397015 ncdv-master-1 devstack@n-api.service[122766]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-flavor-extra-specs:create": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 20:58:09.397015 ncdv-master-1 devstack@n-api.service[122766]: warnings.warn(msg) Jul 16 20:58:09.525361 ncdv-master-1 devstack@n-api.service[122766]: INFO nova.api.openstack.requestlog [None req-e9b9cbb8-701d-4090-9cc9-fbf9e2a0759c admin admin] 10.222.0.16 "POST /compute/v2.1/flavors/c1/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.355037 Jul 16 20:58:09.526065 ncdv-master-1 devstack@n-api.service[122766]: [pid: 122766|app: 0|req: 4/6] 10.222.0.16 () {64 vars in 1335 bytes} [Sat Jul 16 20:58:09 2022] POST /compute/v2.1/flavors/c1/os-extra_specs => generated 43 bytes in 356 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 20:58:12.106489 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi [None req-53edf0c4-0dd5-4fd0-8aa8-055e15be03be admin admin] Calling method '>' {{(pid=122767) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 20:58:12.107516 ncdv-master-1 devstack@n-api.service[122767]: INFO nova.api.openstack.requestlog [None req-53edf0c4-0dd5-4fd0-8aa8-055e15be03be admin admin] 10.222.0.16 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001542 Jul 16 20:58:12.107925 ncdv-master-1 devstack@n-api.service[122767]: [pid: 122767|app: 0|req: 3/7] 10.222.0.16 () {54 vars in 930 bytes} [Sat Jul 16 20:58:12 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 20:58:12.912056 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi [None req-1769ce96-57ef-475e-a5d3-ecb0a37177ee admin admin] Action: 'create', calling method: >, body: {"flavor": {"vcpus": 1, "disk": 5, "ram": 512, "os-flavor-access:is_public": true, "swap": 0, "name": "ds512M", "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "id": "d1"}} {{(pid=122766) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 20:58:12.931994 ncdv-master-1 devstack@n-api.service[122766]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-flavor-manage:create": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 20:58:12.931994 ncdv-master-1 devstack@n-api.service[122766]: warnings.warn(msg) Jul 16 20:58:13.060263 ncdv-master-1 devstack@n-api.service[122766]: INFO nova.api.openstack.requestlog [None req-1769ce96-57ef-475e-a5d3-ecb0a37177ee admin admin] 10.222.0.16 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.61 time: 0.945143 Jul 16 20:58:13.063962 ncdv-master-1 devstack@n-api.service[122766]: [pid: 122766|app: 0|req: 5/8] 10.222.0.16 () {64 vars in 1282 bytes} [Sat Jul 16 20:58:12 2022] POST /compute/v2.1/flavors => generated 404 bytes in 950 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 16 20:58:13.076350 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi [None req-38410ba1-f803-4135-8c96-8f6fb56a14eb admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=122767) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 20:58:13.078495 ncdv-master-1 devstack@n-api.service[122767]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-flavor-extra-specs:create": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 20:58:13.078495 ncdv-master-1 devstack@n-api.service[122767]: warnings.warn(msg) Jul 16 20:58:13.201586 ncdv-master-1 devstack@n-api.service[122767]: INFO nova.api.openstack.requestlog [None req-38410ba1-f803-4135-8c96-8f6fb56a14eb admin admin] 10.222.0.16 "POST /compute/v2.1/flavors/d1/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.130990 Jul 16 20:58:13.219123 ncdv-master-1 devstack@n-api.service[122767]: [pid: 122767|app: 0|req: 4/9] 10.222.0.16 () {64 vars in 1335 bytes} [Sat Jul 16 20:58:13 2022] POST /compute/v2.1/flavors/d1/os-extra_specs => generated 43 bytes in 134 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 20:58:16.046366 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi [None req-1769ce96-57ef-475e-a5d3-ecb0a37177ee admin admin] Calling method '>' {{(pid=122766) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 20:58:16.048185 ncdv-master-1 devstack@n-api.service[122766]: INFO nova.api.openstack.requestlog [None req-1769ce96-57ef-475e-a5d3-ecb0a37177ee admin admin] 10.222.0.16 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.002353 Jul 16 20:58:16.065892 ncdv-master-1 devstack@n-api.service[122766]: [pid: 122766|app: 0|req: 6/10] 10.222.0.16 () {54 vars in 930 bytes} [Sat Jul 16 20:58:16 2022] GET /compute/v2.1 => generated 388 bytes in 9 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 20:58:16.638627 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi [None req-16a8bdbe-ca1e-4432-baa1-91349ef672a5 admin admin] Action: 'create', calling method: >, body: {"flavor": {"OS-FLV-EXT-DATA:ephemeral": 0, "swap": 0, "id": "d2", "vcpus": 1, "disk": 10, "name": "ds1G", "rxtx_factor": 1.0, "os-flavor-access:is_public": true, "ram": 1024}} {{(pid=122767) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 20:58:16.808821 ncdv-master-1 devstack@n-api.service[122767]: INFO nova.api.openstack.requestlog [None req-16a8bdbe-ca1e-4432-baa1-91349ef672a5 admin admin] 10.222.0.16 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.61 time: 0.753071 Jul 16 20:58:16.810220 ncdv-master-1 devstack@n-api.service[122767]: [pid: 122767|app: 0|req: 5/11] 10.222.0.16 () {64 vars in 1282 bytes} [Sat Jul 16 20:58:16 2022] POST /compute/v2.1/flavors => generated 404 bytes in 755 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 16 20:58:16.823030 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi [None req-4cae074f-31bb-4e93-935a-1401bb5e1e8d admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=122766) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 20:58:16.908146 ncdv-master-1 devstack@n-api.service[122766]: INFO nova.api.openstack.requestlog [None req-4cae074f-31bb-4e93-935a-1401bb5e1e8d admin admin] 10.222.0.16 "POST /compute/v2.1/flavors/d2/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.088724 Jul 16 20:58:16.908737 ncdv-master-1 devstack@n-api.service[122766]: [pid: 122766|app: 0|req: 7/12] 10.222.0.16 () {64 vars in 1335 bytes} [Sat Jul 16 20:58:16 2022] POST /compute/v2.1/flavors/d2/os-extra_specs => generated 43 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 20:58:19.619885 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi [None req-16a8bdbe-ca1e-4432-baa1-91349ef672a5 admin admin] Calling method '>' {{(pid=122767) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 20:58:19.621055 ncdv-master-1 devstack@n-api.service[122767]: INFO nova.api.openstack.requestlog [None req-16a8bdbe-ca1e-4432-baa1-91349ef672a5 admin admin] 10.222.0.16 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001790 Jul 16 20:58:19.631348 ncdv-master-1 devstack@n-api.service[122767]: [pid: 122767|app: 0|req: 6/13] 10.222.0.16 () {54 vars in 930 bytes} [Sat Jul 16 20:58:19 2022] GET /compute/v2.1 => generated 388 bytes in 12 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 20:58:20.152726 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi [None req-f2835a5c-2adc-4e72-8aa4-873e14b0305b admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "ds2G", "vcpus": 2, "swap": 0, "ram": 2048, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "disk": 10, "id": "d3", "os-flavor-access:is_public": true}} {{(pid=122766) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 20:58:20.231673 ncdv-master-1 devstack@n-api.service[122766]: INFO nova.api.openstack.requestlog [None req-f2835a5c-2adc-4e72-8aa4-873e14b0305b admin admin] 10.222.0.16 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.61 time: 0.603750 Jul 16 20:58:20.232363 ncdv-master-1 devstack@n-api.service[122766]: [pid: 122766|app: 0|req: 8/14] 10.222.0.16 () {64 vars in 1282 bytes} [Sat Jul 16 20:58:19 2022] POST /compute/v2.1/flavors => generated 404 bytes in 605 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 16 20:58:20.245565 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi [None req-d82d4e59-9355-46d7-b9b5-5d13a25747f6 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=122767) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 20:58:20.405460 ncdv-master-1 devstack@n-api.service[122767]: INFO nova.api.openstack.requestlog [None req-d82d4e59-9355-46d7-b9b5-5d13a25747f6 admin admin] 10.222.0.16 "POST /compute/v2.1/flavors/d3/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.163480 Jul 16 20:58:20.406067 ncdv-master-1 devstack@n-api.service[122767]: [pid: 122767|app: 0|req: 7/15] 10.222.0.16 () {64 vars in 1335 bytes} [Sat Jul 16 20:58:20 2022] POST /compute/v2.1/flavors/d3/os-extra_specs => generated 43 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 20:58:23.039585 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi [None req-f2835a5c-2adc-4e72-8aa4-873e14b0305b admin admin] Calling method '>' {{(pid=122766) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 20:58:23.040756 ncdv-master-1 devstack@n-api.service[122766]: INFO nova.api.openstack.requestlog [None req-f2835a5c-2adc-4e72-8aa4-873e14b0305b admin admin] 10.222.0.16 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001634 Jul 16 20:58:23.041429 ncdv-master-1 devstack@n-api.service[122766]: [pid: 122766|app: 0|req: 9/16] 10.222.0.16 () {54 vars in 930 bytes} [Sat Jul 16 20:58:23 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 20:58:23.501555 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi [None req-e2e02182-02a7-4fac-80c1-62ef9cfc0fd3 admin admin] Action: 'create', calling method: >, body: {"flavor": {"ram": 4096, "os-flavor-access:is_public": true, "rxtx_factor": 1.0, "vcpus": 4, "swap": 0, "name": "ds4G", "OS-FLV-EXT-DATA:ephemeral": 0, "id": "d4", "disk": 20}} {{(pid=122767) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 20:58:23.602195 ncdv-master-1 devstack@n-api.service[122767]: INFO nova.api.openstack.requestlog [None req-e2e02182-02a7-4fac-80c1-62ef9cfc0fd3 admin admin] 10.222.0.16 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.61 time: 0.553551 Jul 16 20:58:23.602759 ncdv-master-1 devstack@n-api.service[122767]: [pid: 122767|app: 0|req: 8/17] 10.222.0.16 () {64 vars in 1282 bytes} [Sat Jul 16 20:58:23 2022] POST /compute/v2.1/flavors => generated 404 bytes in 555 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 16 20:58:23.615339 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi [None req-4d874221-54e0-40cd-af50-4c3464fcb25f admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=122766) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 20:58:23.752914 ncdv-master-1 devstack@n-api.service[122766]: INFO nova.api.openstack.requestlog [None req-4d874221-54e0-40cd-af50-4c3464fcb25f admin admin] 10.222.0.16 "POST /compute/v2.1/flavors/d4/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.141634 Jul 16 20:58:23.753522 ncdv-master-1 devstack@n-api.service[122766]: [pid: 122766|app: 0|req: 10/18] 10.222.0.16 () {64 vars in 1335 bytes} [Sat Jul 16 20:58:23 2022] POST /compute/v2.1/flavors/d4/os-extra_specs => generated 43 bytes in 143 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 20:58:26.386509 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi [None req-e2e02182-02a7-4fac-80c1-62ef9cfc0fd3 admin admin] Calling method '>' {{(pid=122767) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 20:58:26.388295 ncdv-master-1 devstack@n-api.service[122767]: INFO nova.api.openstack.requestlog [None req-e2e02182-02a7-4fac-80c1-62ef9cfc0fd3 admin admin] 10.222.0.16 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.002218 Jul 16 20:58:26.391820 ncdv-master-1 devstack@n-api.service[122767]: [pid: 122767|app: 0|req: 9/19] 10.222.0.16 () {54 vars in 930 bytes} [Sat Jul 16 20:58:26 2022] GET /compute/v2.1 => generated 388 bytes in 6 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 20:58:26.857267 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi [None req-8b2295f8-5613-44ca-abb3-4859902fd2de admin admin] Calling method '>' {{(pid=122766) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 20:58:26.882671 ncdv-master-1 devstack@n-api.service[122766]: INFO nova.api.openstack.requestlog [None req-8b2295f8-5613-44ca-abb3-4859902fd2de admin admin] 10.222.0.16 "GET /compute/v2.1/flavors/detail?is_public=True" status: 200 len: 2106 microversion: 2.61 time: 0.486728 Jul 16 20:58:26.883441 ncdv-master-1 devstack@n-api.service[122766]: [pid: 122766|app: 0|req: 11/20] 10.222.0.16 () {60 vars in 1291 bytes} [Sat Jul 16 20:58:26 2022] GET /compute/v2.1/flavors/detail?is_public=True => generated 2106 bytes in 489 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 20:58:29.649962 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi [None req-e2e02182-02a7-4fac-80c1-62ef9cfc0fd3 admin admin] Calling method '>' {{(pid=122767) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 20:58:29.651222 ncdv-master-1 devstack@n-api.service[122767]: INFO nova.api.openstack.requestlog [None req-e2e02182-02a7-4fac-80c1-62ef9cfc0fd3 admin admin] 10.222.0.16 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001661 Jul 16 20:58:29.651710 ncdv-master-1 devstack@n-api.service[122767]: [pid: 122767|app: 0|req: 10/21] 10.222.0.16 () {54 vars in 930 bytes} [Sat Jul 16 20:58:29 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 20:58:30.115236 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi [None req-fce08d2b-d026-46ab-b503-3845ad8e4a29 admin admin] Action: 'create', calling method: >, body: {"flavor": {"swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "disk": 1, "rxtx_factor": 1.0, "name": "m1.tiny", "vcpus": 1, "os-flavor-access:is_public": true, "id": "1", "ram": 512}} {{(pid=122766) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 20:58:30.242847 ncdv-master-1 devstack@n-api.service[122766]: INFO nova.api.openstack.requestlog [None req-fce08d2b-d026-46ab-b503-3845ad8e4a29 admin admin] 10.222.0.16 "POST /compute/v2.1/flavors" status: 200 len: 402 microversion: 2.61 time: 0.584784 Jul 16 20:58:30.243357 ncdv-master-1 devstack@n-api.service[122766]: [pid: 122766|app: 0|req: 12/22] 10.222.0.16 () {64 vars in 1282 bytes} [Sat Jul 16 20:58:29 2022] POST /compute/v2.1/flavors => generated 402 bytes in 586 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Jul 16 20:58:30.256091 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi [None req-2239f721-dfb8-40e5-803b-9408519e1a6d admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=122767) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 20:58:30.390960 ncdv-master-1 devstack@n-api.service[122767]: INFO nova.api.openstack.requestlog [None req-2239f721-dfb8-40e5-803b-9408519e1a6d admin admin] 10.222.0.16 "POST /compute/v2.1/flavors/1/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.138675 Jul 16 20:58:30.392082 ncdv-master-1 devstack@n-api.service[122767]: [pid: 122767|app: 0|req: 11/23] 10.222.0.16 () {64 vars in 1332 bytes} [Sat Jul 16 20:58:30 2022] POST /compute/v2.1/flavors/1/os-extra_specs => generated 43 bytes in 141 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 20:58:32.886229 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi [None req-fce08d2b-d026-46ab-b503-3845ad8e4a29 admin admin] Calling method '>' {{(pid=122766) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 20:58:32.887289 ncdv-master-1 devstack@n-api.service[122766]: INFO nova.api.openstack.requestlog [None req-fce08d2b-d026-46ab-b503-3845ad8e4a29 admin admin] 10.222.0.16 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001813 Jul 16 20:58:32.888018 ncdv-master-1 devstack@n-api.service[122766]: [pid: 122766|app: 0|req: 13/24] 10.222.0.16 () {54 vars in 930 bytes} [Sat Jul 16 20:58:32 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 20:58:33.388804 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi [None req-013bab2e-e87f-4f2d-b92f-3af7c181b30d admin admin] Action: 'create', calling method: >, body: {"flavor": {"os-flavor-access:is_public": true, "vcpus": 1, "rxtx_factor": 1.0, "name": "m1.small", "id": "2", "OS-FLV-EXT-DATA:ephemeral": 0, "ram": 2048, "disk": 20, "swap": 0}} {{(pid=122767) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 20:58:33.550961 ncdv-master-1 devstack@n-api.service[122767]: INFO nova.api.openstack.requestlog [None req-013bab2e-e87f-4f2d-b92f-3af7c181b30d admin admin] 10.222.0.16 "POST /compute/v2.1/flavors" status: 200 len: 405 microversion: 2.61 time: 0.656164 Jul 16 20:58:33.551850 ncdv-master-1 devstack@n-api.service[122767]: [pid: 122767|app: 0|req: 12/25] 10.222.0.16 () {64 vars in 1282 bytes} [Sat Jul 16 20:58:32 2022] POST /compute/v2.1/flavors => generated 405 bytes in 658 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 16 20:58:33.563970 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi [None req-7e2ccc70-19be-45aa-8642-8d69154a46bc admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=122766) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 20:58:33.633363 ncdv-master-1 devstack@n-api.service[122766]: INFO nova.api.openstack.requestlog [None req-7e2ccc70-19be-45aa-8642-8d69154a46bc admin admin] 10.222.0.16 "POST /compute/v2.1/flavors/2/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.073063 Jul 16 20:58:33.633948 ncdv-master-1 devstack@n-api.service[122766]: [pid: 122766|app: 0|req: 14/26] 10.222.0.16 () {64 vars in 1332 bytes} [Sat Jul 16 20:58:33 2022] POST /compute/v2.1/flavors/2/os-extra_specs => generated 43 bytes in 74 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 20:58:36.106235 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi [None req-013bab2e-e87f-4f2d-b92f-3af7c181b30d admin admin] Calling method '>' {{(pid=122767) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 20:58:36.107284 ncdv-master-1 devstack@n-api.service[122767]: INFO nova.api.openstack.requestlog [None req-013bab2e-e87f-4f2d-b92f-3af7c181b30d admin admin] 10.222.0.16 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001473 Jul 16 20:58:36.107874 ncdv-master-1 devstack@n-api.service[122767]: [pid: 122767|app: 0|req: 13/27] 10.222.0.16 () {54 vars in 930 bytes} [Sat Jul 16 20:58:36 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 20:58:36.494433 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi [None req-b36de221-ae36-49dc-ade9-a8ce6adb8d20 admin admin] Action: 'create', calling method: >, body: {"flavor": {"disk": 40, "ram": 4096, "id": "3", "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true, "vcpus": 2, "name": "m1.medium"}} {{(pid=122766) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 20:58:36.613578 ncdv-master-1 devstack@n-api.service[122766]: INFO nova.api.openstack.requestlog [None req-b36de221-ae36-49dc-ade9-a8ce6adb8d20 admin admin] 10.222.0.16 "POST /compute/v2.1/flavors" status: 200 len: 406 microversion: 2.61 time: 0.499054 Jul 16 20:58:36.614106 ncdv-master-1 devstack@n-api.service[122766]: [pid: 122766|app: 0|req: 15/28] 10.222.0.16 () {64 vars in 1282 bytes} [Sat Jul 16 20:58:36 2022] POST /compute/v2.1/flavors => generated 406 bytes in 501 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 16 20:58:36.626057 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi [None req-ca666983-b52e-47d5-8b37-28b31e1e0be6 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=122767) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 20:58:36.629807 ncdv-master-1 devstack@n-api.service[122767]: DEBUG dbcounter [-] [122767] Writing DB stats nova_api:SELECT=33,nova_api:INSERT=7,nova_api:SAVEPOINT=3,nova_api:RELEASE=3 {{(pid=122767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 20:58:36.755979 ncdv-master-1 devstack@n-api.service[122767]: INFO nova.api.openstack.requestlog [None req-ca666983-b52e-47d5-8b37-28b31e1e0be6 admin admin] 10.222.0.16 "POST /compute/v2.1/flavors/3/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.133480 Jul 16 20:58:36.756480 ncdv-master-1 devstack@n-api.service[122767]: [pid: 122767|app: 0|req: 14/29] 10.222.0.16 () {64 vars in 1332 bytes} [Sat Jul 16 20:58:36 2022] POST /compute/v2.1/flavors/3/os-extra_specs => generated 43 bytes in 135 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 20:58:39.328343 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi [None req-b36de221-ae36-49dc-ade9-a8ce6adb8d20 admin admin] Calling method '>' {{(pid=122766) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 20:58:39.328343 ncdv-master-1 devstack@n-api.service[122766]: INFO nova.api.openstack.requestlog [None req-b36de221-ae36-49dc-ade9-a8ce6adb8d20 admin admin] 10.222.0.16 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001452 Jul 16 20:58:39.328343 ncdv-master-1 devstack@n-api.service[122766]: [pid: 122766|app: 0|req: 16/30] 10.222.0.16 () {54 vars in 930 bytes} [Sat Jul 16 20:58:39 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 20:58:39.838965 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi [None req-15ac1d35-4068-46cc-8e41-c465563fc26b admin admin] Action: 'create', calling method: >, body: {"flavor": {"rxtx_factor": 1.0, "id": "4", "ram": 8192, "os-flavor-access:is_public": true, "OS-FLV-EXT-DATA:ephemeral": 0, "swap": 0, "vcpus": 4, "disk": 80, "name": "m1.large"}} {{(pid=122767) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 20:58:39.917243 ncdv-master-1 devstack@n-api.service[122767]: INFO nova.api.openstack.requestlog [None req-15ac1d35-4068-46cc-8e41-c465563fc26b admin admin] 10.222.0.16 "POST /compute/v2.1/flavors" status: 200 len: 405 microversion: 2.61 time: 0.619959 Jul 16 20:58:39.918110 ncdv-master-1 devstack@n-api.service[122767]: [pid: 122767|app: 0|req: 15/31] 10.222.0.16 () {64 vars in 1282 bytes} [Sat Jul 16 20:58:39 2022] POST /compute/v2.1/flavors => generated 405 bytes in 621 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 16 20:58:39.931588 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi [None req-26fa6846-01ab-42aa-8f4b-235e668b8f42 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=122766) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 20:58:40.032161 ncdv-master-1 devstack@n-api.service[122766]: INFO nova.api.openstack.requestlog [None req-26fa6846-01ab-42aa-8f4b-235e668b8f42 admin admin] 10.222.0.16 "POST /compute/v2.1/flavors/4/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.104376 Jul 16 20:58:40.033318 ncdv-master-1 devstack@n-api.service[122766]: [pid: 122766|app: 0|req: 17/32] 10.222.0.16 () {64 vars in 1332 bytes} [Sat Jul 16 20:58:39 2022] POST /compute/v2.1/flavors/4/os-extra_specs => generated 43 bytes in 106 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 20:58:42.617518 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi [None req-15ac1d35-4068-46cc-8e41-c465563fc26b admin admin] Calling method '>' {{(pid=122767) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 20:58:42.618554 ncdv-master-1 devstack@n-api.service[122767]: INFO nova.api.openstack.requestlog [None req-15ac1d35-4068-46cc-8e41-c465563fc26b admin admin] 10.222.0.16 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001472 Jul 16 20:58:42.618964 ncdv-master-1 devstack@n-api.service[122767]: [pid: 122767|app: 0|req: 16/33] 10.222.0.16 () {54 vars in 930 bytes} [Sat Jul 16 20:58:42 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 20:58:43.152914 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi [None req-2290eb84-3334-40c0-ae7e-7f39fbc0bdd7 admin admin] Action: 'create', calling method: >, body: {"flavor": {"swap": 0, "rxtx_factor": 1.0, "name": "m1.xlarge", "vcpus": 8, "disk": 160, "OS-FLV-EXT-DATA:ephemeral": 0, "ram": 16384, "os-flavor-access:is_public": true, "id": "5"}} {{(pid=122766) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 20:58:43.157592 ncdv-master-1 devstack@n-api.service[122766]: DEBUG dbcounter [-] [122766] Writing DB stats nova_api:SELECT=45,nova_api:SAVEPOINT=5,nova_api:INSERT=9,nova_api:RELEASE=5 {{(pid=122766) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 20:58:43.225944 ncdv-master-1 devstack@n-api.service[122766]: INFO nova.api.openstack.requestlog [None req-2290eb84-3334-40c0-ae7e-7f39fbc0bdd7 admin admin] 10.222.0.16 "POST /compute/v2.1/flavors" status: 200 len: 408 microversion: 2.61 time: 0.600564 Jul 16 20:58:43.227259 ncdv-master-1 devstack@n-api.service[122766]: [pid: 122766|app: 0|req: 18/34] 10.222.0.16 () {64 vars in 1282 bytes} [Sat Jul 16 20:58:42 2022] POST /compute/v2.1/flavors => generated 408 bytes in 603 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Jul 16 20:58:43.240178 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi [None req-21db01f9-2938-4ce4-bbb0-e9b66e867a3f admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=122767) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 20:58:43.326537 ncdv-master-1 devstack@n-api.service[122767]: INFO nova.api.openstack.requestlog [None req-21db01f9-2938-4ce4-bbb0-e9b66e867a3f admin admin] 10.222.0.16 "POST /compute/v2.1/flavors/5/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.091381 Jul 16 20:58:43.327397 ncdv-master-1 devstack@n-api.service[122767]: [pid: 122767|app: 0|req: 17/35] 10.222.0.16 () {64 vars in 1332 bytes} [Sat Jul 16 20:58:43 2022] POST /compute/v2.1/flavors/5/os-extra_specs => generated 43 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 20:59:06.457117 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi [None req-2290eb84-3334-40c0-ae7e-7f39fbc0bdd7 admin admin] Calling method '>' {{(pid=122766) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 20:59:06.458171 ncdv-master-1 devstack@n-api.service[122766]: INFO nova.api.openstack.requestlog [None req-2290eb84-3334-40c0-ae7e-7f39fbc0bdd7 admin admin] 10.222.0.16 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.002155 Jul 16 20:59:06.458578 ncdv-master-1 devstack@n-api.service[122766]: [pid: 122766|app: 0|req: 19/36] 10.222.0.16 () {54 vars in 930 bytes} [Sat Jul 16 20:59:06 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 20:59:06.469804 ncdv-master-1 devstack@n-api.service[122767]: DEBUG dbcounter [-] [122767] Writing DB stats nova_api:SELECT=14,nova_api:SAVEPOINT=2,nova_api:INSERT=3,nova_api:RELEASE=2 {{(pid=122767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 20:59:06.864374 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi [None req-df26022d-8cf6-4d95-96bc-e1cc0998a2f5 admin admin] Calling method '>' {{(pid=122767) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 20:59:06.882575 ncdv-master-1 devstack@n-api.service[122767]: INFO nova.api.openstack.requestlog [None req-df26022d-8cf6-4d95-96bc-e1cc0998a2f5 admin admin] 10.222.0.16 "GET /compute/v2.1/flavors/detail?is_public=True" status: 200 len: 4202 microversion: 2.61 time: 0.417493 Jul 16 20:59:06.883069 ncdv-master-1 devstack@n-api.service[122767]: [pid: 122767|app: 0|req: 18/37] 10.222.0.16 () {60 vars in 1291 bytes} [Sat Jul 16 20:59:06 2022] GET /compute/v2.1/flavors/detail?is_public=True => generated 4202 bytes in 419 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 20:59:08.023863 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi [None req-2290eb84-3334-40c0-ae7e-7f39fbc0bdd7 admin admin] Calling method '>' {{(pid=122766) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 20:59:08.024870 ncdv-master-1 devstack@n-api.service[122766]: INFO nova.api.openstack.requestlog [None req-2290eb84-3334-40c0-ae7e-7f39fbc0bdd7 admin admin] 10.222.0.16 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001491 Jul 16 20:59:08.025267 ncdv-master-1 devstack@n-api.service[122766]: [pid: 122766|app: 0|req: 20/38] 10.222.0.16 () {56 vars in 1074 bytes} [Sat Jul 16 20:59:08 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 20:59:08.613269 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi [None req-705b0462-8504-4f66-9155-a0fa18233b59 admin admin] Calling method '>' {{(pid=122767) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 20:59:08.618906 ncdv-master-1 devstack@n-api.service[122767]: INFO nova.api.openstack.wsgi [None req-705b0462-8504-4f66-9155-a0fa18233b59 admin admin] HTTP exception thrown: Flavor 42 could not be found. Jul 16 20:59:08.632897 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi [None req-705b0462-8504-4f66-9155-a0fa18233b59 admin admin] Request method failure captured: Jul 16 20:59:08.632897 ncdv-master-1 devstack@n-api.service[122767]: request: GET /compute/v2.1/flavors/42 HTTP/1.1 Jul 16 20:59:08.632897 ncdv-master-1 devstack@n-api.service[122767]: Accept: application/json Jul 16 20:59:08.632897 ncdv-master-1 devstack@n-api.service[122767]: Accept-Encoding: gzip, deflate Jul 16 20:59:08.632897 ncdv-master-1 devstack@n-api.service[122767]: Connection: keep-alive Jul 16 20:59:08.632897 ncdv-master-1 devstack@n-api.service[122767]: Content-Length: 0 Jul 16 20:59:08.632897 ncdv-master-1 devstack@n-api.service[122767]: Host: 10.222.0.16 Jul 16 20:59:08.632897 ncdv-master-1 devstack@n-api.service[122767]: Openstack-Api-Version: compute 2.90 Jul 16 20:59:08.632897 ncdv-master-1 devstack@n-api.service[122767]: Openstack-System-Scope: None Jul 16 20:59:08.632897 ncdv-master-1 devstack@n-api.service[122767]: User-Agent: python-novaclient Jul 16 20:59:08.632897 ncdv-master-1 devstack@n-api.service[122767]: X-Auth-Token: gAAAAABi0yabpCRKMcJmfew4IDTb-aqZfwt_DZq7MFwclfMSKwJKdpAKK5uKP5Tz2DQRrsE6M0PpwVMPZ-2HrLwodsWMyA6cUXDQFA2BB9ZG3vHY3aB4LxMsuyPPJXydJ0VIrAuXp9pNsXzLGV1ww64DrlZiEXtTm4zvbHPbz5pPZOfUB1fzUxI Jul 16 20:59:08.632897 ncdv-master-1 devstack@n-api.service[122767]: X-Domain-Id: None Jul 16 20:59:08.632897 ncdv-master-1 devstack@n-api.service[122767]: X-Domain-Name: None Jul 16 20:59:08.632897 ncdv-master-1 devstack@n-api.service[122767]: X-Identity-Status: Confirmed Jul 16 20:59:08.632897 ncdv-master-1 devstack@n-api.service[122767]: X-Is-Admin-Project: True Jul 16 20:59:08.632897 ncdv-master-1 devstack@n-api.service[122767]: X-Openstack-Nova-Api-Version: 2.90 Jul 16 20:59:08.632897 ncdv-master-1 devstack@n-api.service[122767]: X-Project-Domain-Id: default Jul 16 20:59:08.632897 ncdv-master-1 devstack@n-api.service[122767]: X-Project-Domain-Name: Default Jul 16 20:59:08.632897 ncdv-master-1 devstack@n-api.service[122767]: X-Project-Id: 455f5961d0804544b3b78f843a889637 Jul 16 20:59:08.632897 ncdv-master-1 devstack@n-api.service[122767]: X-Project-Name: admin Jul 16 20:59:08.632897 ncdv-master-1 devstack@n-api.service[122767]: X-Role: member,admin,reader Jul 16 20:59:08.632897 ncdv-master-1 devstack@n-api.service[122767]: X-Roles: member,admin,reader Jul 16 20:59:08.634413 ncdv-master-1 devstack@n-api.service[122767]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/455f5961d0804544b3b78f843a889637"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/455f5961d0804544b3b78f843a889637"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_455f5961d0804544b3b78f843a889637"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/455f5961d0804544b3b78f843a889637"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 20:59:08.634413 ncdv-master-1 devstack@n-api.service[122767]: X-Tenant: admin Jul 16 20:59:08.634413 ncdv-master-1 devstack@n-api.service[122767]: X-Tenant-Id: 455f5961d0804544b3b78f843a889637 Jul 16 20:59:08.634413 ncdv-master-1 devstack@n-api.service[122767]: X-Tenant-Name: admin Jul 16 20:59:08.634413 ncdv-master-1 devstack@n-api.service[122767]: X-User: admin Jul 16 20:59:08.634413 ncdv-master-1 devstack@n-api.service[122767]: X-User-Domain-Id: default Jul 16 20:59:08.634413 ncdv-master-1 devstack@n-api.service[122767]: X-User-Domain-Name: Default Jul 16 20:59:08.634413 ncdv-master-1 devstack@n-api.service[122767]: X-User-Id: d6cd66b16dc14b84918d5f636a683873 Jul 16 20:59:08.634413 ncdv-master-1 devstack@n-api.service[122767]: X-User-Name: admin Jul 16 20:59:08.634413 ncdv-master-1 devstack@n-api.service[122767]: method: > Jul 16 20:59:08.634413 ncdv-master-1 devstack@n-api.service[122767]: exception: Flavor 42 could not be found. Jul 16 20:59:08.634413 ncdv-master-1 devstack@n-api.service[122767]: action_args: {'id': '42'} Jul 16 20:59:08.634413 ncdv-master-1 devstack@n-api.service[122767]: {{(pid=122767) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 20:59:08.634413 ncdv-master-1 devstack@n-api.service[122767]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 20:59:08.635480 ncdv-master-1 devstack@n-api.service[122767]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Jul 16 20:59:08.635480 ncdv-master-1 devstack@n-api.service[122767]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Jul 16 20:59:08.635480 ncdv-master-1 devstack@n-api.service[122767]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Jul 16 20:59:08.635480 ncdv-master-1 devstack@n-api.service[122767]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Jul 16 20:59:08.635480 ncdv-master-1 devstack@n-api.service[122767]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Jul 16 20:59:08.635480 ncdv-master-1 devstack@n-api.service[122767]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Jul 16 20:59:08.635480 ncdv-master-1 devstack@n-api.service[122767]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Jul 16 20:59:08.635480 ncdv-master-1 devstack@n-api.service[122767]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Jul 16 20:59:08.635480 ncdv-master-1 devstack@n-api.service[122767]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Jul 16 20:59:08.635480 ncdv-master-1 devstack@n-api.service[122767]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 20:59:08.635480 ncdv-master-1 devstack@n-api.service[122767]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Jul 16 20:59:08.635480 ncdv-master-1 devstack@n-api.service[122767]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Jul 16 20:59:08.635480 ncdv-master-1 devstack@n-api.service[122767]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 42 could not be found. Jul 16 20:59:08.635480 ncdv-master-1 devstack@n-api.service[122767]: ERROR nova.api.openstack.wsgi Jul 16 20:59:08.635480 ncdv-master-1 devstack@n-api.service[122767]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 20:59:08.635480 ncdv-master-1 devstack@n-api.service[122767]: ERROR nova.api.openstack.wsgi Jul 16 20:59:08.635480 ncdv-master-1 devstack@n-api.service[122767]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 20:59:08.635480 ncdv-master-1 devstack@n-api.service[122767]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 20:59:08.635480 ncdv-master-1 devstack@n-api.service[122767]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 20:59:08.636908 ncdv-master-1 devstack@n-api.service[122767]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 20:59:08.636908 ncdv-master-1 devstack@n-api.service[122767]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 20:59:08.636908 ncdv-master-1 devstack@n-api.service[122767]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 20:59:08.636908 ncdv-master-1 devstack@n-api.service[122767]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 20:59:08.636908 ncdv-master-1 devstack@n-api.service[122767]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Jul 16 20:59:08.636908 ncdv-master-1 devstack@n-api.service[122767]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Jul 16 20:59:08.636908 ncdv-master-1 devstack@n-api.service[122767]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 42 could not be found. Jul 16 20:59:08.636908 ncdv-master-1 devstack@n-api.service[122767]: ERROR nova.api.openstack.wsgi Jul 16 20:59:08.636908 ncdv-master-1 devstack@n-api.service[122767]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 20:59:08.636908 ncdv-master-1 devstack@n-api.service[122767]: ERROR nova.api.openstack.wsgi Jul 16 20:59:08.636908 ncdv-master-1 devstack@n-api.service[122767]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 20:59:08.636908 ncdv-master-1 devstack@n-api.service[122767]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 20:59:08.636908 ncdv-master-1 devstack@n-api.service[122767]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 20:59:08.636908 ncdv-master-1 devstack@n-api.service[122767]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 20:59:08.636908 ncdv-master-1 devstack@n-api.service[122767]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 20:59:08.636908 ncdv-master-1 devstack@n-api.service[122767]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 42 could not be found. Jul 16 20:59:08.636908 ncdv-master-1 devstack@n-api.service[122767]: ERROR nova.api.openstack.wsgi Jul 16 20:59:08.636908 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi [None req-705b0462-8504-4f66-9155-a0fa18233b59 admin admin] Returning 404 to user: Flavor 42 could not be found. {{(pid=122767) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 20:59:08.638174 ncdv-master-1 devstack@n-api.service[122767]: INFO nova.api.openstack.requestlog [None req-705b0462-8504-4f66-9155-a0fa18233b59 admin admin] 10.222.0.16 "GET /compute/v2.1/flavors/42" status: 404 len: 75 microversion: 2.90 time: 0.421782 Jul 16 20:59:08.638174 ncdv-master-1 devstack@n-api.service[122767]: [pid: 122767|app: 0|req: 19/39] 10.222.0.16 () {60 vars in 1190 bytes} [Sat Jul 16 20:59:08 2022] GET /compute/v2.1/flavors/42 => generated 75 bytes in 423 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Jul 16 20:59:08.644106 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi [None req-fbe3bfab-312c-45b7-be49-3323f713411a admin admin] Calling method '>' {{(pid=122766) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 20:59:08.665653 ncdv-master-1 devstack@n-api.service[122766]: INFO nova.api.openstack.requestlog [None req-fbe3bfab-312c-45b7-be49-3323f713411a admin admin] 10.222.0.16 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.90 time: 0.025030 Jul 16 20:59:08.666853 ncdv-master-1 devstack@n-api.service[122766]: [pid: 122766|app: 0|req: 21/40] 10.222.0.16 () {60 vars in 1210 bytes} [Sat Jul 16 20:59:08 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 26 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 20:59:08.683400 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi [None req-c9f33089-b406-4062-96ce-c1e4c6c375de admin admin] Calling method '>' {{(pid=122767) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 20:59:08.705847 ncdv-master-1 devstack@n-api.service[122767]: INFO nova.api.openstack.requestlog [None req-c9f33089-b406-4062-96ce-c1e4c6c375de admin admin] 10.222.0.16 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.90 time: 0.026083 Jul 16 20:59:08.706983 ncdv-master-1 devstack@n-api.service[122767]: [pid: 122767|app: 0|req: 20/41] 10.222.0.16 () {60 vars in 1210 bytes} [Sat Jul 16 20:59:08 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 28 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 20:59:08.716959 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi [None req-704f200d-6a9e-4a84-b5a3-abd20f241d71 admin admin] Calling method '>' {{(pid=122766) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 20:59:08.723309 ncdv-master-1 devstack@n-api.service[122766]: INFO nova.api.openstack.wsgi [None req-704f200d-6a9e-4a84-b5a3-abd20f241d71 admin admin] HTTP exception thrown: Flavor 42 could not be found. Jul 16 20:59:08.726657 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi [None req-704f200d-6a9e-4a84-b5a3-abd20f241d71 admin admin] Request method failure captured: Jul 16 20:59:08.726657 ncdv-master-1 devstack@n-api.service[122766]: request: GET /compute/v2.1/flavors/42 HTTP/1.1 Jul 16 20:59:08.726657 ncdv-master-1 devstack@n-api.service[122766]: Accept: application/json Jul 16 20:59:08.726657 ncdv-master-1 devstack@n-api.service[122766]: Accept-Encoding: gzip, deflate Jul 16 20:59:08.726657 ncdv-master-1 devstack@n-api.service[122766]: Connection: keep-alive Jul 16 20:59:08.726657 ncdv-master-1 devstack@n-api.service[122766]: Content-Length: 0 Jul 16 20:59:08.726657 ncdv-master-1 devstack@n-api.service[122766]: Host: 10.222.0.16 Jul 16 20:59:08.726657 ncdv-master-1 devstack@n-api.service[122766]: Openstack-Api-Version: compute 2.90 Jul 16 20:59:08.726657 ncdv-master-1 devstack@n-api.service[122766]: Openstack-System-Scope: None Jul 16 20:59:08.726657 ncdv-master-1 devstack@n-api.service[122766]: User-Agent: python-novaclient Jul 16 20:59:08.726657 ncdv-master-1 devstack@n-api.service[122766]: X-Auth-Token: gAAAAABi0yabpCRKMcJmfew4IDTb-aqZfwt_DZq7MFwclfMSKwJKdpAKK5uKP5Tz2DQRrsE6M0PpwVMPZ-2HrLwodsWMyA6cUXDQFA2BB9ZG3vHY3aB4LxMsuyPPJXydJ0VIrAuXp9pNsXzLGV1ww64DrlZiEXtTm4zvbHPbz5pPZOfUB1fzUxI Jul 16 20:59:08.726657 ncdv-master-1 devstack@n-api.service[122766]: X-Domain-Id: None Jul 16 20:59:08.726657 ncdv-master-1 devstack@n-api.service[122766]: X-Domain-Name: None Jul 16 20:59:08.726657 ncdv-master-1 devstack@n-api.service[122766]: X-Identity-Status: Confirmed Jul 16 20:59:08.726657 ncdv-master-1 devstack@n-api.service[122766]: X-Is-Admin-Project: True Jul 16 20:59:08.726657 ncdv-master-1 devstack@n-api.service[122766]: X-Openstack-Nova-Api-Version: 2.90 Jul 16 20:59:08.726657 ncdv-master-1 devstack@n-api.service[122766]: X-Project-Domain-Id: default Jul 16 20:59:08.726657 ncdv-master-1 devstack@n-api.service[122766]: X-Project-Domain-Name: Default Jul 16 20:59:08.726657 ncdv-master-1 devstack@n-api.service[122766]: X-Project-Id: 455f5961d0804544b3b78f843a889637 Jul 16 20:59:08.726657 ncdv-master-1 devstack@n-api.service[122766]: X-Project-Name: admin Jul 16 20:59:08.726657 ncdv-master-1 devstack@n-api.service[122766]: X-Role: member,admin,reader Jul 16 20:59:08.726657 ncdv-master-1 devstack@n-api.service[122766]: X-Roles: member,admin,reader Jul 16 20:59:08.728194 ncdv-master-1 devstack@n-api.service[122766]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/455f5961d0804544b3b78f843a889637"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/455f5961d0804544b3b78f843a889637"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_455f5961d0804544b3b78f843a889637"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/455f5961d0804544b3b78f843a889637"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 20:59:08.728194 ncdv-master-1 devstack@n-api.service[122766]: X-Tenant: admin Jul 16 20:59:08.728194 ncdv-master-1 devstack@n-api.service[122766]: X-Tenant-Id: 455f5961d0804544b3b78f843a889637 Jul 16 20:59:08.728194 ncdv-master-1 devstack@n-api.service[122766]: X-Tenant-Name: admin Jul 16 20:59:08.728194 ncdv-master-1 devstack@n-api.service[122766]: X-User: admin Jul 16 20:59:08.728194 ncdv-master-1 devstack@n-api.service[122766]: X-User-Domain-Id: default Jul 16 20:59:08.728194 ncdv-master-1 devstack@n-api.service[122766]: X-User-Domain-Name: Default Jul 16 20:59:08.728194 ncdv-master-1 devstack@n-api.service[122766]: X-User-Id: d6cd66b16dc14b84918d5f636a683873 Jul 16 20:59:08.728194 ncdv-master-1 devstack@n-api.service[122766]: X-User-Name: admin Jul 16 20:59:08.728194 ncdv-master-1 devstack@n-api.service[122766]: method: > Jul 16 20:59:08.728194 ncdv-master-1 devstack@n-api.service[122766]: exception: Flavor 42 could not be found. Jul 16 20:59:08.728194 ncdv-master-1 devstack@n-api.service[122766]: action_args: {'id': '42'} Jul 16 20:59:08.728194 ncdv-master-1 devstack@n-api.service[122766]: {{(pid=122766) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 20:59:08.728194 ncdv-master-1 devstack@n-api.service[122766]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 20:59:08.729097 ncdv-master-1 devstack@n-api.service[122766]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Jul 16 20:59:08.729097 ncdv-master-1 devstack@n-api.service[122766]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Jul 16 20:59:08.729097 ncdv-master-1 devstack@n-api.service[122766]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Jul 16 20:59:08.729097 ncdv-master-1 devstack@n-api.service[122766]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Jul 16 20:59:08.729097 ncdv-master-1 devstack@n-api.service[122766]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Jul 16 20:59:08.729097 ncdv-master-1 devstack@n-api.service[122766]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Jul 16 20:59:08.729097 ncdv-master-1 devstack@n-api.service[122766]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Jul 16 20:59:08.729097 ncdv-master-1 devstack@n-api.service[122766]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Jul 16 20:59:08.729097 ncdv-master-1 devstack@n-api.service[122766]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Jul 16 20:59:08.729097 ncdv-master-1 devstack@n-api.service[122766]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 20:59:08.729097 ncdv-master-1 devstack@n-api.service[122766]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Jul 16 20:59:08.729097 ncdv-master-1 devstack@n-api.service[122766]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Jul 16 20:59:08.729097 ncdv-master-1 devstack@n-api.service[122766]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 42 could not be found. Jul 16 20:59:08.729097 ncdv-master-1 devstack@n-api.service[122766]: ERROR nova.api.openstack.wsgi Jul 16 20:59:08.729097 ncdv-master-1 devstack@n-api.service[122766]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 20:59:08.729097 ncdv-master-1 devstack@n-api.service[122766]: ERROR nova.api.openstack.wsgi Jul 16 20:59:08.729097 ncdv-master-1 devstack@n-api.service[122766]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 20:59:08.729097 ncdv-master-1 devstack@n-api.service[122766]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 20:59:08.729097 ncdv-master-1 devstack@n-api.service[122766]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 20:59:08.730117 ncdv-master-1 devstack@n-api.service[122766]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 20:59:08.730117 ncdv-master-1 devstack@n-api.service[122766]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 20:59:08.730117 ncdv-master-1 devstack@n-api.service[122766]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 20:59:08.730117 ncdv-master-1 devstack@n-api.service[122766]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 20:59:08.730117 ncdv-master-1 devstack@n-api.service[122766]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Jul 16 20:59:08.730117 ncdv-master-1 devstack@n-api.service[122766]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Jul 16 20:59:08.730117 ncdv-master-1 devstack@n-api.service[122766]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 42 could not be found. Jul 16 20:59:08.730117 ncdv-master-1 devstack@n-api.service[122766]: ERROR nova.api.openstack.wsgi Jul 16 20:59:08.730117 ncdv-master-1 devstack@n-api.service[122766]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 20:59:08.730117 ncdv-master-1 devstack@n-api.service[122766]: ERROR nova.api.openstack.wsgi Jul 16 20:59:08.730117 ncdv-master-1 devstack@n-api.service[122766]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 20:59:08.730117 ncdv-master-1 devstack@n-api.service[122766]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 20:59:08.730117 ncdv-master-1 devstack@n-api.service[122766]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 20:59:08.730117 ncdv-master-1 devstack@n-api.service[122766]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 20:59:08.730117 ncdv-master-1 devstack@n-api.service[122766]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 20:59:08.730117 ncdv-master-1 devstack@n-api.service[122766]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 42 could not be found. Jul 16 20:59:08.730117 ncdv-master-1 devstack@n-api.service[122766]: ERROR nova.api.openstack.wsgi Jul 16 20:59:08.730117 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi [None req-704f200d-6a9e-4a84-b5a3-abd20f241d71 admin admin] Returning 404 to user: Flavor 42 could not be found. {{(pid=122766) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 20:59:08.731112 ncdv-master-1 devstack@n-api.service[122766]: INFO nova.api.openstack.requestlog [None req-704f200d-6a9e-4a84-b5a3-abd20f241d71 admin admin] 10.222.0.16 "GET /compute/v2.1/flavors/42" status: 404 len: 75 microversion: 2.90 time: 0.015056 Jul 16 20:59:08.731112 ncdv-master-1 devstack@n-api.service[122766]: [pid: 122766|app: 0|req: 22/42] 10.222.0.16 () {60 vars in 1190 bytes} [Sat Jul 16 20:59:08 2022] GET /compute/v2.1/flavors/42 => generated 75 bytes in 16 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Jul 16 20:59:09.713924 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi [None req-c9f33089-b406-4062-96ce-c1e4c6c375de admin admin] Calling method '>' {{(pid=122767) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 20:59:09.715717 ncdv-master-1 devstack@n-api.service[122767]: INFO nova.api.openstack.requestlog [None req-c9f33089-b406-4062-96ce-c1e4c6c375de admin admin] 10.222.0.16 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.002455 Jul 16 20:59:09.716473 ncdv-master-1 devstack@n-api.service[122767]: [pid: 122767|app: 0|req: 21/43] 10.222.0.16 () {56 vars in 1074 bytes} [Sat Jul 16 20:59:09 2022] GET /compute/v2.1 => generated 388 bytes in 4 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 20:59:10.279546 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi [None req-bf5ed4b7-ee43-4444-be14-4f92cedd5656 admin admin] Calling method '>' {{(pid=122766) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 20:59:10.286384 ncdv-master-1 devstack@n-api.service[122766]: INFO nova.api.openstack.wsgi [None req-bf5ed4b7-ee43-4444-be14-4f92cedd5656 admin admin] HTTP exception thrown: Flavor 84 could not be found. Jul 16 20:59:10.287949 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi [None req-bf5ed4b7-ee43-4444-be14-4f92cedd5656 admin admin] Request method failure captured: Jul 16 20:59:10.287949 ncdv-master-1 devstack@n-api.service[122766]: request: GET /compute/v2.1/flavors/84 HTTP/1.1 Jul 16 20:59:10.287949 ncdv-master-1 devstack@n-api.service[122766]: Accept: application/json Jul 16 20:59:10.287949 ncdv-master-1 devstack@n-api.service[122766]: Accept-Encoding: gzip, deflate Jul 16 20:59:10.287949 ncdv-master-1 devstack@n-api.service[122766]: Connection: keep-alive Jul 16 20:59:10.287949 ncdv-master-1 devstack@n-api.service[122766]: Content-Length: 0 Jul 16 20:59:10.287949 ncdv-master-1 devstack@n-api.service[122766]: Host: 10.222.0.16 Jul 16 20:59:10.287949 ncdv-master-1 devstack@n-api.service[122766]: Openstack-Api-Version: compute 2.90 Jul 16 20:59:10.287949 ncdv-master-1 devstack@n-api.service[122766]: Openstack-System-Scope: None Jul 16 20:59:10.287949 ncdv-master-1 devstack@n-api.service[122766]: User-Agent: python-novaclient Jul 16 20:59:10.287949 ncdv-master-1 devstack@n-api.service[122766]: X-Auth-Token: gAAAAABi0yadbjLGmLBD2V1ZlTvXDNu6Bg5p81XaEM6RbEDkHS5aTk4oNM9I87IZcHmmRcw9LxUpwmQ1GjVkf4mTYDqNO2DAi-EWGb2JwtH735SJ7oXb3KYojENaBaDqhzWFW31Uo1xB_YipAEqDpiIZ3-5V1_g2Xd9Pn-3Bx12vDCim70j9AxI Jul 16 20:59:10.287949 ncdv-master-1 devstack@n-api.service[122766]: X-Domain-Id: None Jul 16 20:59:10.287949 ncdv-master-1 devstack@n-api.service[122766]: X-Domain-Name: None Jul 16 20:59:10.287949 ncdv-master-1 devstack@n-api.service[122766]: X-Identity-Status: Confirmed Jul 16 20:59:10.287949 ncdv-master-1 devstack@n-api.service[122766]: X-Is-Admin-Project: True Jul 16 20:59:10.287949 ncdv-master-1 devstack@n-api.service[122766]: X-Openstack-Nova-Api-Version: 2.90 Jul 16 20:59:10.287949 ncdv-master-1 devstack@n-api.service[122766]: X-Project-Domain-Id: default Jul 16 20:59:10.287949 ncdv-master-1 devstack@n-api.service[122766]: X-Project-Domain-Name: Default Jul 16 20:59:10.287949 ncdv-master-1 devstack@n-api.service[122766]: X-Project-Id: 455f5961d0804544b3b78f843a889637 Jul 16 20:59:10.287949 ncdv-master-1 devstack@n-api.service[122766]: X-Project-Name: admin Jul 16 20:59:10.287949 ncdv-master-1 devstack@n-api.service[122766]: X-Role: member,admin,reader Jul 16 20:59:10.287949 ncdv-master-1 devstack@n-api.service[122766]: X-Roles: member,admin,reader Jul 16 20:59:10.289588 ncdv-master-1 devstack@n-api.service[122766]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/455f5961d0804544b3b78f843a889637"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/455f5961d0804544b3b78f843a889637"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_455f5961d0804544b3b78f843a889637"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/455f5961d0804544b3b78f843a889637"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 20:59:10.289588 ncdv-master-1 devstack@n-api.service[122766]: X-Tenant: admin Jul 16 20:59:10.289588 ncdv-master-1 devstack@n-api.service[122766]: X-Tenant-Id: 455f5961d0804544b3b78f843a889637 Jul 16 20:59:10.289588 ncdv-master-1 devstack@n-api.service[122766]: X-Tenant-Name: admin Jul 16 20:59:10.289588 ncdv-master-1 devstack@n-api.service[122766]: X-User: admin Jul 16 20:59:10.289588 ncdv-master-1 devstack@n-api.service[122766]: X-User-Domain-Id: default Jul 16 20:59:10.289588 ncdv-master-1 devstack@n-api.service[122766]: X-User-Domain-Name: Default Jul 16 20:59:10.289588 ncdv-master-1 devstack@n-api.service[122766]: X-User-Id: d6cd66b16dc14b84918d5f636a683873 Jul 16 20:59:10.289588 ncdv-master-1 devstack@n-api.service[122766]: X-User-Name: admin Jul 16 20:59:10.289588 ncdv-master-1 devstack@n-api.service[122766]: method: > Jul 16 20:59:10.289588 ncdv-master-1 devstack@n-api.service[122766]: exception: Flavor 84 could not be found. Jul 16 20:59:10.289588 ncdv-master-1 devstack@n-api.service[122766]: action_args: {'id': '84'} Jul 16 20:59:10.289588 ncdv-master-1 devstack@n-api.service[122766]: {{(pid=122766) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 20:59:10.289588 ncdv-master-1 devstack@n-api.service[122766]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 20:59:10.290463 ncdv-master-1 devstack@n-api.service[122766]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Jul 16 20:59:10.290463 ncdv-master-1 devstack@n-api.service[122766]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Jul 16 20:59:10.290463 ncdv-master-1 devstack@n-api.service[122766]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Jul 16 20:59:10.290463 ncdv-master-1 devstack@n-api.service[122766]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Jul 16 20:59:10.290463 ncdv-master-1 devstack@n-api.service[122766]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Jul 16 20:59:10.290463 ncdv-master-1 devstack@n-api.service[122766]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Jul 16 20:59:10.290463 ncdv-master-1 devstack@n-api.service[122766]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Jul 16 20:59:10.290463 ncdv-master-1 devstack@n-api.service[122766]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Jul 16 20:59:10.290463 ncdv-master-1 devstack@n-api.service[122766]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Jul 16 20:59:10.290463 ncdv-master-1 devstack@n-api.service[122766]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 20:59:10.290463 ncdv-master-1 devstack@n-api.service[122766]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Jul 16 20:59:10.290463 ncdv-master-1 devstack@n-api.service[122766]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Jul 16 20:59:10.290463 ncdv-master-1 devstack@n-api.service[122766]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 84 could not be found. Jul 16 20:59:10.290463 ncdv-master-1 devstack@n-api.service[122766]: ERROR nova.api.openstack.wsgi Jul 16 20:59:10.290463 ncdv-master-1 devstack@n-api.service[122766]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 20:59:10.290463 ncdv-master-1 devstack@n-api.service[122766]: ERROR nova.api.openstack.wsgi Jul 16 20:59:10.290463 ncdv-master-1 devstack@n-api.service[122766]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 20:59:10.290463 ncdv-master-1 devstack@n-api.service[122766]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 20:59:10.290463 ncdv-master-1 devstack@n-api.service[122766]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 20:59:10.291616 ncdv-master-1 devstack@n-api.service[122766]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 20:59:10.291616 ncdv-master-1 devstack@n-api.service[122766]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 20:59:10.291616 ncdv-master-1 devstack@n-api.service[122766]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 20:59:10.291616 ncdv-master-1 devstack@n-api.service[122766]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 20:59:10.291616 ncdv-master-1 devstack@n-api.service[122766]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Jul 16 20:59:10.291616 ncdv-master-1 devstack@n-api.service[122766]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Jul 16 20:59:10.291616 ncdv-master-1 devstack@n-api.service[122766]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 84 could not be found. Jul 16 20:59:10.291616 ncdv-master-1 devstack@n-api.service[122766]: ERROR nova.api.openstack.wsgi Jul 16 20:59:10.291616 ncdv-master-1 devstack@n-api.service[122766]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 20:59:10.291616 ncdv-master-1 devstack@n-api.service[122766]: ERROR nova.api.openstack.wsgi Jul 16 20:59:10.291616 ncdv-master-1 devstack@n-api.service[122766]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 20:59:10.291616 ncdv-master-1 devstack@n-api.service[122766]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 20:59:10.291616 ncdv-master-1 devstack@n-api.service[122766]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 20:59:10.291616 ncdv-master-1 devstack@n-api.service[122766]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 20:59:10.291616 ncdv-master-1 devstack@n-api.service[122766]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 20:59:10.291616 ncdv-master-1 devstack@n-api.service[122766]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 84 could not be found. Jul 16 20:59:10.291616 ncdv-master-1 devstack@n-api.service[122766]: ERROR nova.api.openstack.wsgi Jul 16 20:59:10.291616 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi [None req-bf5ed4b7-ee43-4444-be14-4f92cedd5656 admin admin] Returning 404 to user: Flavor 84 could not be found. {{(pid=122766) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 20:59:10.292631 ncdv-master-1 devstack@n-api.service[122766]: INFO nova.api.openstack.requestlog [None req-bf5ed4b7-ee43-4444-be14-4f92cedd5656 admin admin] 10.222.0.16 "GET /compute/v2.1/flavors/84" status: 404 len: 75 microversion: 2.90 time: 0.419882 Jul 16 20:59:10.292631 ncdv-master-1 devstack@n-api.service[122766]: [pid: 122766|app: 0|req: 23/44] 10.222.0.16 () {60 vars in 1190 bytes} [Sat Jul 16 20:59:09 2022] GET /compute/v2.1/flavors/84 => generated 75 bytes in 421 msecs (HTTP/1.1 404) 9 headers in 380 bytes (2 switches on core 0) Jul 16 20:59:10.299316 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi [None req-df10e31c-7fbe-4738-a782-f943f7b7c44b admin admin] Calling method '>' {{(pid=122767) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 20:59:10.313585 ncdv-master-1 devstack@n-api.service[122767]: INFO nova.api.openstack.requestlog [None req-df10e31c-7fbe-4738-a782-f943f7b7c44b admin admin] 10.222.0.16 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.90 time: 0.017791 Jul 16 20:59:10.314144 ncdv-master-1 devstack@n-api.service[122767]: [pid: 122767|app: 0|req: 22/45] 10.222.0.16 () {60 vars in 1210 bytes} [Sat Jul 16 20:59:10 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 19 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 20:59:10.324772 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi [None req-8142bf18-5080-4071-b764-3f9d158755d9 admin admin] Calling method '>' {{(pid=122766) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 20:59:10.339695 ncdv-master-1 devstack@n-api.service[122766]: INFO nova.api.openstack.requestlog [None req-8142bf18-5080-4071-b764-3f9d158755d9 admin admin] 10.222.0.16 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.90 time: 0.018538 Jul 16 20:59:10.340261 ncdv-master-1 devstack@n-api.service[122766]: [pid: 122766|app: 0|req: 24/46] 10.222.0.16 () {60 vars in 1210 bytes} [Sat Jul 16 20:59:10 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 20 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 16 20:59:10.349930 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi [None req-f689c567-72cd-4dae-957b-2ec3a2398c46 admin admin] Calling method '>' {{(pid=122767) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 20:59:10.355497 ncdv-master-1 devstack@n-api.service[122767]: INFO nova.api.openstack.wsgi [None req-f689c567-72cd-4dae-957b-2ec3a2398c46 admin admin] HTTP exception thrown: Flavor 84 could not be found. Jul 16 20:59:10.357041 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi [None req-f689c567-72cd-4dae-957b-2ec3a2398c46 admin admin] Request method failure captured: Jul 16 20:59:10.357041 ncdv-master-1 devstack@n-api.service[122767]: request: GET /compute/v2.1/flavors/84 HTTP/1.1 Jul 16 20:59:10.357041 ncdv-master-1 devstack@n-api.service[122767]: Accept: application/json Jul 16 20:59:10.357041 ncdv-master-1 devstack@n-api.service[122767]: Accept-Encoding: gzip, deflate Jul 16 20:59:10.357041 ncdv-master-1 devstack@n-api.service[122767]: Connection: keep-alive Jul 16 20:59:10.357041 ncdv-master-1 devstack@n-api.service[122767]: Content-Length: 0 Jul 16 20:59:10.357041 ncdv-master-1 devstack@n-api.service[122767]: Host: 10.222.0.16 Jul 16 20:59:10.357041 ncdv-master-1 devstack@n-api.service[122767]: Openstack-Api-Version: compute 2.90 Jul 16 20:59:10.357041 ncdv-master-1 devstack@n-api.service[122767]: Openstack-System-Scope: None Jul 16 20:59:10.357041 ncdv-master-1 devstack@n-api.service[122767]: User-Agent: python-novaclient Jul 16 20:59:10.357041 ncdv-master-1 devstack@n-api.service[122767]: X-Auth-Token: gAAAAABi0yadbjLGmLBD2V1ZlTvXDNu6Bg5p81XaEM6RbEDkHS5aTk4oNM9I87IZcHmmRcw9LxUpwmQ1GjVkf4mTYDqNO2DAi-EWGb2JwtH735SJ7oXb3KYojENaBaDqhzWFW31Uo1xB_YipAEqDpiIZ3-5V1_g2Xd9Pn-3Bx12vDCim70j9AxI Jul 16 20:59:10.357041 ncdv-master-1 devstack@n-api.service[122767]: X-Domain-Id: None Jul 16 20:59:10.357041 ncdv-master-1 devstack@n-api.service[122767]: X-Domain-Name: None Jul 16 20:59:10.357041 ncdv-master-1 devstack@n-api.service[122767]: X-Identity-Status: Confirmed Jul 16 20:59:10.357041 ncdv-master-1 devstack@n-api.service[122767]: X-Is-Admin-Project: True Jul 16 20:59:10.357041 ncdv-master-1 devstack@n-api.service[122767]: X-Openstack-Nova-Api-Version: 2.90 Jul 16 20:59:10.357041 ncdv-master-1 devstack@n-api.service[122767]: X-Project-Domain-Id: default Jul 16 20:59:10.357041 ncdv-master-1 devstack@n-api.service[122767]: X-Project-Domain-Name: Default Jul 16 20:59:10.357041 ncdv-master-1 devstack@n-api.service[122767]: X-Project-Id: 455f5961d0804544b3b78f843a889637 Jul 16 20:59:10.357041 ncdv-master-1 devstack@n-api.service[122767]: X-Project-Name: admin Jul 16 20:59:10.357041 ncdv-master-1 devstack@n-api.service[122767]: X-Role: member,admin,reader Jul 16 20:59:10.357041 ncdv-master-1 devstack@n-api.service[122767]: X-Roles: member,admin,reader Jul 16 20:59:10.358554 ncdv-master-1 devstack@n-api.service[122767]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/455f5961d0804544b3b78f843a889637"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/455f5961d0804544b3b78f843a889637"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_455f5961d0804544b3b78f843a889637"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/455f5961d0804544b3b78f843a889637"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 20:59:10.358554 ncdv-master-1 devstack@n-api.service[122767]: X-Tenant: admin Jul 16 20:59:10.358554 ncdv-master-1 devstack@n-api.service[122767]: X-Tenant-Id: 455f5961d0804544b3b78f843a889637 Jul 16 20:59:10.358554 ncdv-master-1 devstack@n-api.service[122767]: X-Tenant-Name: admin Jul 16 20:59:10.358554 ncdv-master-1 devstack@n-api.service[122767]: X-User: admin Jul 16 20:59:10.358554 ncdv-master-1 devstack@n-api.service[122767]: X-User-Domain-Id: default Jul 16 20:59:10.358554 ncdv-master-1 devstack@n-api.service[122767]: X-User-Domain-Name: Default Jul 16 20:59:10.358554 ncdv-master-1 devstack@n-api.service[122767]: X-User-Id: d6cd66b16dc14b84918d5f636a683873 Jul 16 20:59:10.358554 ncdv-master-1 devstack@n-api.service[122767]: X-User-Name: admin Jul 16 20:59:10.358554 ncdv-master-1 devstack@n-api.service[122767]: method: > Jul 16 20:59:10.358554 ncdv-master-1 devstack@n-api.service[122767]: exception: Flavor 84 could not be found. Jul 16 20:59:10.358554 ncdv-master-1 devstack@n-api.service[122767]: action_args: {'id': '84'} Jul 16 20:59:10.358554 ncdv-master-1 devstack@n-api.service[122767]: {{(pid=122767) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 20:59:10.358554 ncdv-master-1 devstack@n-api.service[122767]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 20:59:10.362691 ncdv-master-1 devstack@n-api.service[122767]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Jul 16 20:59:10.362691 ncdv-master-1 devstack@n-api.service[122767]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Jul 16 20:59:10.362691 ncdv-master-1 devstack@n-api.service[122767]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Jul 16 20:59:10.362691 ncdv-master-1 devstack@n-api.service[122767]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Jul 16 20:59:10.362691 ncdv-master-1 devstack@n-api.service[122767]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Jul 16 20:59:10.362691 ncdv-master-1 devstack@n-api.service[122767]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Jul 16 20:59:10.362691 ncdv-master-1 devstack@n-api.service[122767]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Jul 16 20:59:10.362691 ncdv-master-1 devstack@n-api.service[122767]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Jul 16 20:59:10.362691 ncdv-master-1 devstack@n-api.service[122767]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Jul 16 20:59:10.362691 ncdv-master-1 devstack@n-api.service[122767]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 20:59:10.362691 ncdv-master-1 devstack@n-api.service[122767]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Jul 16 20:59:10.362691 ncdv-master-1 devstack@n-api.service[122767]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Jul 16 20:59:10.362691 ncdv-master-1 devstack@n-api.service[122767]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 84 could not be found. Jul 16 20:59:10.362691 ncdv-master-1 devstack@n-api.service[122767]: ERROR nova.api.openstack.wsgi Jul 16 20:59:10.362691 ncdv-master-1 devstack@n-api.service[122767]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 20:59:10.362691 ncdv-master-1 devstack@n-api.service[122767]: ERROR nova.api.openstack.wsgi Jul 16 20:59:10.362691 ncdv-master-1 devstack@n-api.service[122767]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 20:59:10.362691 ncdv-master-1 devstack@n-api.service[122767]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 20:59:10.362691 ncdv-master-1 devstack@n-api.service[122767]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 20:59:10.364208 ncdv-master-1 devstack@n-api.service[122767]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 20:59:10.364208 ncdv-master-1 devstack@n-api.service[122767]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 20:59:10.364208 ncdv-master-1 devstack@n-api.service[122767]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 20:59:10.364208 ncdv-master-1 devstack@n-api.service[122767]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 20:59:10.364208 ncdv-master-1 devstack@n-api.service[122767]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Jul 16 20:59:10.364208 ncdv-master-1 devstack@n-api.service[122767]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Jul 16 20:59:10.364208 ncdv-master-1 devstack@n-api.service[122767]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 84 could not be found. Jul 16 20:59:10.364208 ncdv-master-1 devstack@n-api.service[122767]: ERROR nova.api.openstack.wsgi Jul 16 20:59:10.364208 ncdv-master-1 devstack@n-api.service[122767]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 20:59:10.364208 ncdv-master-1 devstack@n-api.service[122767]: ERROR nova.api.openstack.wsgi Jul 16 20:59:10.364208 ncdv-master-1 devstack@n-api.service[122767]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 20:59:10.364208 ncdv-master-1 devstack@n-api.service[122767]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 20:59:10.364208 ncdv-master-1 devstack@n-api.service[122767]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 20:59:10.364208 ncdv-master-1 devstack@n-api.service[122767]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 20:59:10.364208 ncdv-master-1 devstack@n-api.service[122767]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 20:59:10.364208 ncdv-master-1 devstack@n-api.service[122767]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 84 could not be found. Jul 16 20:59:10.364208 ncdv-master-1 devstack@n-api.service[122767]: ERROR nova.api.openstack.wsgi Jul 16 20:59:10.364208 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi [None req-f689c567-72cd-4dae-957b-2ec3a2398c46 admin admin] Returning 404 to user: Flavor 84 could not be found. {{(pid=122767) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 20:59:10.365277 ncdv-master-1 devstack@n-api.service[122767]: INFO nova.api.openstack.requestlog [None req-f689c567-72cd-4dae-957b-2ec3a2398c46 admin admin] 10.222.0.16 "GET /compute/v2.1/flavors/84" status: 404 len: 75 microversion: 2.90 time: 0.012151 Jul 16 20:59:10.365277 ncdv-master-1 devstack@n-api.service[122767]: [pid: 122767|app: 0|req: 23/47] 10.222.0.16 () {60 vars in 1190 bytes} [Sat Jul 16 20:59:10 2022] GET /compute/v2.1/flavors/84 => generated 75 bytes in 13 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Jul 16 20:59:11.371159 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi [None req-8142bf18-5080-4071-b764-3f9d158755d9 admin admin] Calling method '>' {{(pid=122766) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 20:59:11.372181 ncdv-master-1 devstack@n-api.service[122766]: INFO nova.api.openstack.requestlog [None req-8142bf18-5080-4071-b764-3f9d158755d9 admin admin] 10.222.0.16 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001499 Jul 16 20:59:11.372582 ncdv-master-1 devstack@n-api.service[122766]: [pid: 122766|app: 0|req: 25/48] 10.222.0.16 () {56 vars in 1074 bytes} [Sat Jul 16 20:59:11 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 20:59:11.920026 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi [None req-44841d68-6ea0-4a94-a5f1-eac19b9595ff admin admin] Calling method '>' {{(pid=122767) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 20:59:11.926358 ncdv-master-1 devstack@n-api.service[122767]: INFO nova.api.openstack.wsgi [None req-44841d68-6ea0-4a94-a5f1-eac19b9595ff admin admin] HTTP exception thrown: Flavor 451 could not be found. Jul 16 20:59:11.927911 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi [None req-44841d68-6ea0-4a94-a5f1-eac19b9595ff admin admin] Request method failure captured: Jul 16 20:59:11.927911 ncdv-master-1 devstack@n-api.service[122767]: request: GET /compute/v2.1/flavors/451 HTTP/1.1 Jul 16 20:59:11.927911 ncdv-master-1 devstack@n-api.service[122767]: Accept: application/json Jul 16 20:59:11.927911 ncdv-master-1 devstack@n-api.service[122767]: Accept-Encoding: gzip, deflate Jul 16 20:59:11.927911 ncdv-master-1 devstack@n-api.service[122767]: Connection: keep-alive Jul 16 20:59:11.927911 ncdv-master-1 devstack@n-api.service[122767]: Content-Length: 0 Jul 16 20:59:11.927911 ncdv-master-1 devstack@n-api.service[122767]: Host: 10.222.0.16 Jul 16 20:59:11.927911 ncdv-master-1 devstack@n-api.service[122767]: Openstack-Api-Version: compute 2.90 Jul 16 20:59:11.927911 ncdv-master-1 devstack@n-api.service[122767]: Openstack-System-Scope: None Jul 16 20:59:11.927911 ncdv-master-1 devstack@n-api.service[122767]: User-Agent: python-novaclient Jul 16 20:59:11.927911 ncdv-master-1 devstack@n-api.service[122767]: X-Auth-Token: gAAAAABi0yafQKO19u6zUsvLzkGONChtL7-A0xmxbmewUPHFHeQSECrxRuIDy7tn8LirgKqBwlzRUTS5DPdUg8SWjhtnpywrDAlJk93AswV6poDQJ-hq_8_nVWA1biZelxKfsx0o5j5OBEOXVLaNGcoJpW8RCLaaFjhobaDM6dkTjqpB8PrLA-A Jul 16 20:59:11.927911 ncdv-master-1 devstack@n-api.service[122767]: X-Domain-Id: None Jul 16 20:59:11.927911 ncdv-master-1 devstack@n-api.service[122767]: X-Domain-Name: None Jul 16 20:59:11.927911 ncdv-master-1 devstack@n-api.service[122767]: X-Identity-Status: Confirmed Jul 16 20:59:11.927911 ncdv-master-1 devstack@n-api.service[122767]: X-Is-Admin-Project: True Jul 16 20:59:11.927911 ncdv-master-1 devstack@n-api.service[122767]: X-Openstack-Nova-Api-Version: 2.90 Jul 16 20:59:11.927911 ncdv-master-1 devstack@n-api.service[122767]: X-Project-Domain-Id: default Jul 16 20:59:11.927911 ncdv-master-1 devstack@n-api.service[122767]: X-Project-Domain-Name: Default Jul 16 20:59:11.927911 ncdv-master-1 devstack@n-api.service[122767]: X-Project-Id: 455f5961d0804544b3b78f843a889637 Jul 16 20:59:11.927911 ncdv-master-1 devstack@n-api.service[122767]: X-Project-Name: admin Jul 16 20:59:11.927911 ncdv-master-1 devstack@n-api.service[122767]: X-Role: member,admin,reader Jul 16 20:59:11.927911 ncdv-master-1 devstack@n-api.service[122767]: X-Roles: member,admin,reader Jul 16 20:59:11.929446 ncdv-master-1 devstack@n-api.service[122767]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/455f5961d0804544b3b78f843a889637"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/455f5961d0804544b3b78f843a889637"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_455f5961d0804544b3b78f843a889637"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/455f5961d0804544b3b78f843a889637"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 20:59:11.929446 ncdv-master-1 devstack@n-api.service[122767]: X-Tenant: admin Jul 16 20:59:11.929446 ncdv-master-1 devstack@n-api.service[122767]: X-Tenant-Id: 455f5961d0804544b3b78f843a889637 Jul 16 20:59:11.929446 ncdv-master-1 devstack@n-api.service[122767]: X-Tenant-Name: admin Jul 16 20:59:11.929446 ncdv-master-1 devstack@n-api.service[122767]: X-User: admin Jul 16 20:59:11.929446 ncdv-master-1 devstack@n-api.service[122767]: X-User-Domain-Id: default Jul 16 20:59:11.929446 ncdv-master-1 devstack@n-api.service[122767]: X-User-Domain-Name: Default Jul 16 20:59:11.929446 ncdv-master-1 devstack@n-api.service[122767]: X-User-Id: d6cd66b16dc14b84918d5f636a683873 Jul 16 20:59:11.929446 ncdv-master-1 devstack@n-api.service[122767]: X-User-Name: admin Jul 16 20:59:11.929446 ncdv-master-1 devstack@n-api.service[122767]: method: > Jul 16 20:59:11.929446 ncdv-master-1 devstack@n-api.service[122767]: exception: Flavor 451 could not be found. Jul 16 20:59:11.929446 ncdv-master-1 devstack@n-api.service[122767]: action_args: {'id': '451'} Jul 16 20:59:11.929446 ncdv-master-1 devstack@n-api.service[122767]: {{(pid=122767) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 20:59:11.929446 ncdv-master-1 devstack@n-api.service[122767]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 20:59:11.930454 ncdv-master-1 devstack@n-api.service[122767]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Jul 16 20:59:11.930454 ncdv-master-1 devstack@n-api.service[122767]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Jul 16 20:59:11.930454 ncdv-master-1 devstack@n-api.service[122767]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Jul 16 20:59:11.930454 ncdv-master-1 devstack@n-api.service[122767]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Jul 16 20:59:11.930454 ncdv-master-1 devstack@n-api.service[122767]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Jul 16 20:59:11.930454 ncdv-master-1 devstack@n-api.service[122767]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Jul 16 20:59:11.930454 ncdv-master-1 devstack@n-api.service[122767]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Jul 16 20:59:11.930454 ncdv-master-1 devstack@n-api.service[122767]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Jul 16 20:59:11.930454 ncdv-master-1 devstack@n-api.service[122767]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Jul 16 20:59:11.930454 ncdv-master-1 devstack@n-api.service[122767]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 20:59:11.930454 ncdv-master-1 devstack@n-api.service[122767]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Jul 16 20:59:11.930454 ncdv-master-1 devstack@n-api.service[122767]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Jul 16 20:59:11.930454 ncdv-master-1 devstack@n-api.service[122767]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 451 could not be found. Jul 16 20:59:11.930454 ncdv-master-1 devstack@n-api.service[122767]: ERROR nova.api.openstack.wsgi Jul 16 20:59:11.930454 ncdv-master-1 devstack@n-api.service[122767]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 20:59:11.930454 ncdv-master-1 devstack@n-api.service[122767]: ERROR nova.api.openstack.wsgi Jul 16 20:59:11.930454 ncdv-master-1 devstack@n-api.service[122767]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 20:59:11.930454 ncdv-master-1 devstack@n-api.service[122767]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 20:59:11.930454 ncdv-master-1 devstack@n-api.service[122767]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 20:59:11.932029 ncdv-master-1 devstack@n-api.service[122767]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 20:59:11.932029 ncdv-master-1 devstack@n-api.service[122767]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 20:59:11.932029 ncdv-master-1 devstack@n-api.service[122767]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 20:59:11.932029 ncdv-master-1 devstack@n-api.service[122767]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 20:59:11.932029 ncdv-master-1 devstack@n-api.service[122767]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Jul 16 20:59:11.932029 ncdv-master-1 devstack@n-api.service[122767]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Jul 16 20:59:11.932029 ncdv-master-1 devstack@n-api.service[122767]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 451 could not be found. Jul 16 20:59:11.932029 ncdv-master-1 devstack@n-api.service[122767]: ERROR nova.api.openstack.wsgi Jul 16 20:59:11.932029 ncdv-master-1 devstack@n-api.service[122767]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 20:59:11.932029 ncdv-master-1 devstack@n-api.service[122767]: ERROR nova.api.openstack.wsgi Jul 16 20:59:11.932029 ncdv-master-1 devstack@n-api.service[122767]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 20:59:11.932029 ncdv-master-1 devstack@n-api.service[122767]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 20:59:11.932029 ncdv-master-1 devstack@n-api.service[122767]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 20:59:11.932029 ncdv-master-1 devstack@n-api.service[122767]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 20:59:11.932029 ncdv-master-1 devstack@n-api.service[122767]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 20:59:11.932029 ncdv-master-1 devstack@n-api.service[122767]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 451 could not be found. Jul 16 20:59:11.932029 ncdv-master-1 devstack@n-api.service[122767]: ERROR nova.api.openstack.wsgi Jul 16 20:59:11.932029 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi [None req-44841d68-6ea0-4a94-a5f1-eac19b9595ff admin admin] Returning 404 to user: Flavor 451 could not be found. {{(pid=122767) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 20:59:11.933258 ncdv-master-1 devstack@n-api.service[122767]: INFO nova.api.openstack.requestlog [None req-44841d68-6ea0-4a94-a5f1-eac19b9595ff admin admin] 10.222.0.16 "GET /compute/v2.1/flavors/451" status: 404 len: 76 microversion: 2.90 time: 0.411176 Jul 16 20:59:11.933258 ncdv-master-1 devstack@n-api.service[122767]: [pid: 122767|app: 0|req: 24/49] 10.222.0.16 () {60 vars in 1193 bytes} [Sat Jul 16 20:59:11 2022] GET /compute/v2.1/flavors/451 => generated 76 bytes in 412 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Jul 16 20:59:11.939280 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi [None req-ac9177c3-5431-4555-9b5b-e87af021c645 admin admin] Calling method '>' {{(pid=122766) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 20:59:11.953415 ncdv-master-1 devstack@n-api.service[122766]: INFO nova.api.openstack.requestlog [None req-ac9177c3-5431-4555-9b5b-e87af021c645 admin admin] 10.222.0.16 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.90 time: 0.017786 Jul 16 20:59:11.953963 ncdv-master-1 devstack@n-api.service[122766]: [pid: 122766|app: 0|req: 26/50] 10.222.0.16 () {60 vars in 1210 bytes} [Sat Jul 16 20:59:11 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 19 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 20:59:11.964494 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi [None req-47fdff52-391e-421b-90fc-341f4b0e8c9e admin admin] Calling method '>' {{(pid=122767) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 20:59:11.978818 ncdv-master-1 devstack@n-api.service[122767]: INFO nova.api.openstack.requestlog [None req-47fdff52-391e-421b-90fc-341f4b0e8c9e admin admin] 10.222.0.16 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.90 time: 0.017901 Jul 16 20:59:11.979390 ncdv-master-1 devstack@n-api.service[122767]: [pid: 122767|app: 0|req: 25/51] 10.222.0.16 () {60 vars in 1210 bytes} [Sat Jul 16 20:59:11 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 19 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 20:59:11.989307 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi [None req-6baedb8d-b14a-4701-9a61-aa6a67ce1556 admin admin] Calling method '>' {{(pid=122766) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 20:59:11.994702 ncdv-master-1 devstack@n-api.service[122766]: INFO nova.api.openstack.wsgi [None req-6baedb8d-b14a-4701-9a61-aa6a67ce1556 admin admin] HTTP exception thrown: Flavor 451 could not be found. Jul 16 20:59:11.996238 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi [None req-6baedb8d-b14a-4701-9a61-aa6a67ce1556 admin admin] Request method failure captured: Jul 16 20:59:11.996238 ncdv-master-1 devstack@n-api.service[122766]: request: GET /compute/v2.1/flavors/451 HTTP/1.1 Jul 16 20:59:11.996238 ncdv-master-1 devstack@n-api.service[122766]: Accept: application/json Jul 16 20:59:11.996238 ncdv-master-1 devstack@n-api.service[122766]: Accept-Encoding: gzip, deflate Jul 16 20:59:11.996238 ncdv-master-1 devstack@n-api.service[122766]: Connection: keep-alive Jul 16 20:59:11.996238 ncdv-master-1 devstack@n-api.service[122766]: Content-Length: 0 Jul 16 20:59:11.996238 ncdv-master-1 devstack@n-api.service[122766]: Host: 10.222.0.16 Jul 16 20:59:11.996238 ncdv-master-1 devstack@n-api.service[122766]: Openstack-Api-Version: compute 2.90 Jul 16 20:59:11.996238 ncdv-master-1 devstack@n-api.service[122766]: Openstack-System-Scope: None Jul 16 20:59:11.996238 ncdv-master-1 devstack@n-api.service[122766]: User-Agent: python-novaclient Jul 16 20:59:11.996238 ncdv-master-1 devstack@n-api.service[122766]: X-Auth-Token: gAAAAABi0yafQKO19u6zUsvLzkGONChtL7-A0xmxbmewUPHFHeQSECrxRuIDy7tn8LirgKqBwlzRUTS5DPdUg8SWjhtnpywrDAlJk93AswV6poDQJ-hq_8_nVWA1biZelxKfsx0o5j5OBEOXVLaNGcoJpW8RCLaaFjhobaDM6dkTjqpB8PrLA-A Jul 16 20:59:11.996238 ncdv-master-1 devstack@n-api.service[122766]: X-Domain-Id: None Jul 16 20:59:11.996238 ncdv-master-1 devstack@n-api.service[122766]: X-Domain-Name: None Jul 16 20:59:11.996238 ncdv-master-1 devstack@n-api.service[122766]: X-Identity-Status: Confirmed Jul 16 20:59:11.996238 ncdv-master-1 devstack@n-api.service[122766]: X-Is-Admin-Project: True Jul 16 20:59:11.996238 ncdv-master-1 devstack@n-api.service[122766]: X-Openstack-Nova-Api-Version: 2.90 Jul 16 20:59:11.996238 ncdv-master-1 devstack@n-api.service[122766]: X-Project-Domain-Id: default Jul 16 20:59:11.996238 ncdv-master-1 devstack@n-api.service[122766]: X-Project-Domain-Name: Default Jul 16 20:59:11.996238 ncdv-master-1 devstack@n-api.service[122766]: X-Project-Id: 455f5961d0804544b3b78f843a889637 Jul 16 20:59:11.996238 ncdv-master-1 devstack@n-api.service[122766]: X-Project-Name: admin Jul 16 20:59:11.996238 ncdv-master-1 devstack@n-api.service[122766]: X-Role: member,admin,reader Jul 16 20:59:11.996238 ncdv-master-1 devstack@n-api.service[122766]: X-Roles: member,admin,reader Jul 16 20:59:11.999414 ncdv-master-1 devstack@n-api.service[122766]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/455f5961d0804544b3b78f843a889637"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/455f5961d0804544b3b78f843a889637"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_455f5961d0804544b3b78f843a889637"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/455f5961d0804544b3b78f843a889637"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 20:59:11.999414 ncdv-master-1 devstack@n-api.service[122766]: X-Tenant: admin Jul 16 20:59:11.999414 ncdv-master-1 devstack@n-api.service[122766]: X-Tenant-Id: 455f5961d0804544b3b78f843a889637 Jul 16 20:59:11.999414 ncdv-master-1 devstack@n-api.service[122766]: X-Tenant-Name: admin Jul 16 20:59:11.999414 ncdv-master-1 devstack@n-api.service[122766]: X-User: admin Jul 16 20:59:11.999414 ncdv-master-1 devstack@n-api.service[122766]: X-User-Domain-Id: default Jul 16 20:59:11.999414 ncdv-master-1 devstack@n-api.service[122766]: X-User-Domain-Name: Default Jul 16 20:59:11.999414 ncdv-master-1 devstack@n-api.service[122766]: X-User-Id: d6cd66b16dc14b84918d5f636a683873 Jul 16 20:59:11.999414 ncdv-master-1 devstack@n-api.service[122766]: X-User-Name: admin Jul 16 20:59:11.999414 ncdv-master-1 devstack@n-api.service[122766]: method: > Jul 16 20:59:11.999414 ncdv-master-1 devstack@n-api.service[122766]: exception: Flavor 451 could not be found. Jul 16 20:59:11.999414 ncdv-master-1 devstack@n-api.service[122766]: action_args: {'id': '451'} Jul 16 20:59:11.999414 ncdv-master-1 devstack@n-api.service[122766]: {{(pid=122766) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 20:59:11.999414 ncdv-master-1 devstack@n-api.service[122766]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 20:59:12.000203 ncdv-master-1 devstack@n-api.service[122766]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Jul 16 20:59:12.000203 ncdv-master-1 devstack@n-api.service[122766]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Jul 16 20:59:12.000203 ncdv-master-1 devstack@n-api.service[122766]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Jul 16 20:59:12.000203 ncdv-master-1 devstack@n-api.service[122766]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Jul 16 20:59:12.000203 ncdv-master-1 devstack@n-api.service[122766]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Jul 16 20:59:12.000203 ncdv-master-1 devstack@n-api.service[122766]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Jul 16 20:59:12.000203 ncdv-master-1 devstack@n-api.service[122766]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Jul 16 20:59:12.000203 ncdv-master-1 devstack@n-api.service[122766]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Jul 16 20:59:12.000203 ncdv-master-1 devstack@n-api.service[122766]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Jul 16 20:59:12.000203 ncdv-master-1 devstack@n-api.service[122766]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 20:59:12.000203 ncdv-master-1 devstack@n-api.service[122766]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Jul 16 20:59:12.000203 ncdv-master-1 devstack@n-api.service[122766]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Jul 16 20:59:12.000203 ncdv-master-1 devstack@n-api.service[122766]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 451 could not be found. Jul 16 20:59:12.000203 ncdv-master-1 devstack@n-api.service[122766]: ERROR nova.api.openstack.wsgi Jul 16 20:59:12.000203 ncdv-master-1 devstack@n-api.service[122766]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 20:59:12.000203 ncdv-master-1 devstack@n-api.service[122766]: ERROR nova.api.openstack.wsgi Jul 16 20:59:12.000203 ncdv-master-1 devstack@n-api.service[122766]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 20:59:12.000203 ncdv-master-1 devstack@n-api.service[122766]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 20:59:12.000203 ncdv-master-1 devstack@n-api.service[122766]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 20:59:12.001240 ncdv-master-1 devstack@n-api.service[122766]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 20:59:12.001240 ncdv-master-1 devstack@n-api.service[122766]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 20:59:12.001240 ncdv-master-1 devstack@n-api.service[122766]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 20:59:12.001240 ncdv-master-1 devstack@n-api.service[122766]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 20:59:12.001240 ncdv-master-1 devstack@n-api.service[122766]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Jul 16 20:59:12.001240 ncdv-master-1 devstack@n-api.service[122766]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Jul 16 20:59:12.001240 ncdv-master-1 devstack@n-api.service[122766]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 451 could not be found. Jul 16 20:59:12.001240 ncdv-master-1 devstack@n-api.service[122766]: ERROR nova.api.openstack.wsgi Jul 16 20:59:12.001240 ncdv-master-1 devstack@n-api.service[122766]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 20:59:12.001240 ncdv-master-1 devstack@n-api.service[122766]: ERROR nova.api.openstack.wsgi Jul 16 20:59:12.001240 ncdv-master-1 devstack@n-api.service[122766]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 20:59:12.001240 ncdv-master-1 devstack@n-api.service[122766]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 20:59:12.001240 ncdv-master-1 devstack@n-api.service[122766]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 20:59:12.001240 ncdv-master-1 devstack@n-api.service[122766]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 20:59:12.001240 ncdv-master-1 devstack@n-api.service[122766]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 20:59:12.001240 ncdv-master-1 devstack@n-api.service[122766]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 451 could not be found. Jul 16 20:59:12.001240 ncdv-master-1 devstack@n-api.service[122766]: ERROR nova.api.openstack.wsgi Jul 16 20:59:12.001240 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi [None req-6baedb8d-b14a-4701-9a61-aa6a67ce1556 admin admin] Returning 404 to user: Flavor 451 could not be found. {{(pid=122766) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 20:59:12.003337 ncdv-master-1 devstack@n-api.service[122766]: INFO nova.api.openstack.requestlog [None req-6baedb8d-b14a-4701-9a61-aa6a67ce1556 admin admin] 10.222.0.16 "GET /compute/v2.1/flavors/451" status: 404 len: 76 microversion: 2.90 time: 0.012210 Jul 16 20:59:12.003337 ncdv-master-1 devstack@n-api.service[122766]: [pid: 122766|app: 0|req: 27/52] 10.222.0.16 () {60 vars in 1193 bytes} [Sat Jul 16 20:59:11 2022] GET /compute/v2.1/flavors/451 => generated 76 bytes in 13 msecs (HTTP/1.1 404) 9 headers in 380 bytes (2 switches on core 0) Jul 16 20:59:12.989424 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi [None req-47fdff52-391e-421b-90fc-341f4b0e8c9e admin admin] Calling method '>' {{(pid=122767) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 20:59:12.990435 ncdv-master-1 devstack@n-api.service[122767]: INFO nova.api.openstack.requestlog [None req-47fdff52-391e-421b-90fc-341f4b0e8c9e admin admin] 10.222.0.16 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001451 Jul 16 20:59:12.990838 ncdv-master-1 devstack@n-api.service[122767]: [pid: 122767|app: 0|req: 26/53] 10.222.0.16 () {56 vars in 1074 bytes} [Sat Jul 16 20:59:12 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 20:59:13.531631 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi [None req-70d3fc38-5c45-443d-bda6-4092e2e1c99f admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.nano", "ram": 256, "vcpus": 1, "disk": 1, "id": "42", "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true}} {{(pid=122766) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 20:59:13.612580 ncdv-master-1 devstack@n-api.service[122766]: INFO nova.api.openstack.requestlog [None req-70d3fc38-5c45-443d-bda6-4092e2e1c99f admin admin] 10.222.0.16 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.90 time: 0.476443 Jul 16 20:59:13.613762 ncdv-master-1 devstack@n-api.service[122766]: [pid: 122766|app: 0|req: 28/54] 10.222.0.16 () {64 vars in 1235 bytes} [Sat Jul 16 20:59:13 2022] POST /compute/v2.1/flavors => generated 404 bytes in 479 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 16 20:59:14.623672 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi [None req-47fdff52-391e-421b-90fc-341f4b0e8c9e admin admin] Calling method '>' {{(pid=122767) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 20:59:14.624690 ncdv-master-1 devstack@n-api.service[122767]: INFO nova.api.openstack.requestlog [None req-47fdff52-391e-421b-90fc-341f4b0e8c9e admin admin] 10.222.0.16 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001512 Jul 16 20:59:14.625085 ncdv-master-1 devstack@n-api.service[122767]: [pid: 122767|app: 0|req: 27/55] 10.222.0.16 () {56 vars in 1074 bytes} [Sat Jul 16 20:59:14 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 20:59:15.312112 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi [None req-a56081ca-1dbf-4496-b454-45303055702d admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.micro", "ram": 256, "vcpus": 1, "disk": 2, "id": "84", "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true}} {{(pid=122766) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 20:59:15.421998 ncdv-master-1 devstack@n-api.service[122766]: INFO nova.api.openstack.requestlog [None req-a56081ca-1dbf-4496-b454-45303055702d admin admin] 10.222.0.16 "POST /compute/v2.1/flavors" status: 200 len: 405 microversion: 2.90 time: 0.647488 Jul 16 20:59:15.422575 ncdv-master-1 devstack@n-api.service[122766]: [pid: 122766|app: 0|req: 29/56] 10.222.0.16 () {64 vars in 1235 bytes} [Sat Jul 16 20:59:14 2022] POST /compute/v2.1/flavors => generated 405 bytes in 649 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 16 20:59:16.447351 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi [None req-47fdff52-391e-421b-90fc-341f4b0e8c9e admin admin] Calling method '>' {{(pid=122767) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 20:59:16.448360 ncdv-master-1 devstack@n-api.service[122767]: INFO nova.api.openstack.requestlog [None req-47fdff52-391e-421b-90fc-341f4b0e8c9e admin admin] 10.222.0.16 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001483 Jul 16 20:59:16.448759 ncdv-master-1 devstack@n-api.service[122767]: [pid: 122767|app: 0|req: 28/57] 10.222.0.16 () {56 vars in 1074 bytes} [Sat Jul 16 20:59:16 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 20:59:17.025855 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi [None req-af887a1e-d47d-4fa9-8201-9529423ec432 admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.heat", "ram": 512, "vcpus": 1, "disk": 1, "id": "451", "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true}} {{(pid=122766) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 20:59:17.113469 ncdv-master-1 devstack@n-api.service[122766]: INFO nova.api.openstack.requestlog [None req-af887a1e-d47d-4fa9-8201-9529423ec432 admin admin] 10.222.0.16 "POST /compute/v2.1/flavors" status: 200 len: 407 microversion: 2.90 time: 0.515824 Jul 16 20:59:17.113957 ncdv-master-1 devstack@n-api.service[122766]: [pid: 122766|app: 0|req: 30/58] 10.222.0.16 () {64 vars in 1235 bytes} [Sat Jul 16 20:59:16 2022] POST /compute/v2.1/flavors => generated 407 bytes in 518 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 16 20:59:18.110678 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi [None req-47fdff52-391e-421b-90fc-341f4b0e8c9e admin admin] Calling method '>' {{(pid=122767) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 20:59:18.111729 ncdv-master-1 devstack@n-api.service[122767]: INFO nova.api.openstack.requestlog [None req-47fdff52-391e-421b-90fc-341f4b0e8c9e admin admin] 10.222.0.16 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001494 Jul 16 20:59:18.112138 ncdv-master-1 devstack@n-api.service[122767]: [pid: 122767|app: 0|req: 29/59] 10.222.0.16 () {56 vars in 1074 bytes} [Sat Jul 16 20:59:18 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 20:59:18.678950 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi [None req-d28dc954-2217-40f3-8f67-5fefa5b8636d admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.heat2", "ram": 1024, "vcpus": 1, "disk": 1, "id": "452", "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true}} {{(pid=122766) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 20:59:18.764531 ncdv-master-1 devstack@n-api.service[122766]: INFO nova.api.openstack.requestlog [None req-d28dc954-2217-40f3-8f67-5fefa5b8636d admin admin] 10.222.0.16 "POST /compute/v2.1/flavors" status: 200 len: 409 microversion: 2.90 time: 0.498472 Jul 16 20:59:18.765090 ncdv-master-1 devstack@n-api.service[122766]: [pid: 122766|app: 0|req: 31/60] 10.222.0.16 () {64 vars in 1235 bytes} [Sat Jul 16 20:59:18 2022] POST /compute/v2.1/flavors => generated 409 bytes in 500 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Jul 16 20:59:19.765069 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi [None req-47fdff52-391e-421b-90fc-341f4b0e8c9e admin admin] Calling method '>' {{(pid=122767) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 20:59:19.766127 ncdv-master-1 devstack@n-api.service[122767]: INFO nova.api.openstack.requestlog [None req-47fdff52-391e-421b-90fc-341f4b0e8c9e admin admin] 10.222.0.16 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001504 Jul 16 20:59:19.766599 ncdv-master-1 devstack@n-api.service[122767]: [pid: 122767|app: 0|req: 30/61] 10.222.0.16 () {56 vars in 1074 bytes} [Sat Jul 16 20:59:19 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 20:59:20.436705 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi [None req-7c5e60f8-6668-4c70-881f-7cf34c187708 admin admin] Calling method '>' {{(pid=122766) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 20:59:20.457012 ncdv-master-1 devstack@n-api.service[122766]: INFO nova.api.openstack.requestlog [None req-7c5e60f8-6668-4c70-881f-7cf34c187708 admin admin] 10.222.0.16 "GET /compute/v2.1/flavors/detail" status: 200 len: 5777 microversion: 2.90 time: 0.540286 Jul 16 20:59:20.457547 ncdv-master-1 devstack@n-api.service[122766]: [pid: 122766|app: 0|req: 32/62] 10.222.0.16 () {60 vars in 1202 bytes} [Sat Jul 16 20:59:19 2022] GET /compute/v2.1/flavors/detail => generated 5777 bytes in 542 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:00:20.136451 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi [None req-47fdff52-391e-421b-90fc-341f4b0e8c9e admin admin] Calling method '>' {{(pid=122767) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:00:20.137425 ncdv-master-1 devstack@n-api.service[122767]: INFO nova.api.openstack.requestlog [None req-47fdff52-391e-421b-90fc-341f4b0e8c9e admin admin] 10.222.0.16 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001506 Jul 16 21:00:20.137821 ncdv-master-1 devstack@n-api.service[122767]: [pid: 122767|app: 0|req: 31/63] 10.222.0.16 () {54 vars in 930 bytes} [Sat Jul 16 21:00:20 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:00:20.148338 ncdv-master-1 devstack@n-api.service[122766]: DEBUG dbcounter [-] [122766] Writing DB stats nova_api:INSERT=5,nova_api:SELECT=28 {{(pid=122766) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:00:20.539595 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi [None req-ad51f103-eff6-43a1-9efd-02a90686244c admin admin] Calling method '>' {{(pid=122766) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:00:20.558478 ncdv-master-1 devstack@n-api.service[122766]: INFO nova.api.openstack.requestlog [None req-ad51f103-eff6-43a1-9efd-02a90686244c admin admin] 10.222.0.16 "GET /compute/v2.1/flavors/detail?is_public=True" status: 200 len: 5791 microversion: 2.61 time: 0.414566 Jul 16 21:00:20.559028 ncdv-master-1 devstack@n-api.service[122766]: [pid: 122766|app: 0|req: 33/64] 10.222.0.16 () {60 vars in 1291 bytes} [Sat Jul 16 21:00:20 2022] GET /compute/v2.1/flavors/detail?is_public=True => generated 5791 bytes in 416 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 16 21:00:20.580338 ncdv-master-1 devstack@n-api.service[122767]: DEBUG dbcounter [-] [122767] Writing DB stats nova_api:SELECT=14 {{(pid=122767) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:00:20.584076 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi [None req-fb6ebec6-e138-4e96-9feb-09fc747ec57e admin admin] Calling method '>' {{(pid=122767) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:00:20.591717 ncdv-master-1 devstack@n-api.service[122767]: INFO nova.api.openstack.requestlog [None req-fb6ebec6-e138-4e96-9feb-09fc747ec57e admin admin] 10.222.0.16 "GET /compute/v2.1/flavors/42/os-extra_specs" status: 200 len: 19 microversion: 2.61 time: 0.013414 Jul 16 21:00:20.596054 ncdv-master-1 devstack@n-api.service[122767]: [pid: 122767|app: 0|req: 32/65] 10.222.0.16 () {60 vars in 1282 bytes} [Sat Jul 16 21:00:20 2022] GET /compute/v2.1/flavors/42/os-extra_specs => generated 19 bytes in 14 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:00:20.604715 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi [None req-33f0e79e-7c55-4713-b517-fd89d7611bcc admin admin] Calling method '>' {{(pid=122766) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:00:20.613388 ncdv-master-1 devstack@n-api.service[122766]: INFO nova.api.openstack.requestlog [None req-33f0e79e-7c55-4713-b517-fd89d7611bcc admin admin] 10.222.0.16 "GET /compute/v2.1/flavors/451/os-extra_specs" status: 200 len: 19 microversion: 2.61 time: 0.012220 Jul 16 21:00:20.613916 ncdv-master-1 devstack@n-api.service[122766]: [pid: 122766|app: 0|req: 34/66] 10.222.0.16 () {60 vars in 1285 bytes} [Sat Jul 16 21:00:20 2022] GET /compute/v2.1/flavors/451/os-extra_specs => generated 19 bytes in 13 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:00:20.626279 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi [None req-885111ce-ab9d-454b-bea7-ab88cda45bf1 admin admin] Calling method '>' {{(pid=122767) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:00:20.634249 ncdv-master-1 devstack@n-api.service[122767]: INFO nova.api.openstack.requestlog [None req-885111ce-ab9d-454b-bea7-ab88cda45bf1 admin admin] 10.222.0.16 "GET /compute/v2.1/flavors/452/os-extra_specs" status: 200 len: 19 microversion: 2.61 time: 0.011665 Jul 16 21:00:20.634769 ncdv-master-1 devstack@n-api.service[122767]: [pid: 122767|app: 0|req: 33/67] 10.222.0.16 () {60 vars in 1285 bytes} [Sat Jul 16 21:00:20 2022] GET /compute/v2.1/flavors/452/os-extra_specs => generated 19 bytes in 13 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:00:20.646790 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi [None req-ba6c75d9-0df5-4dd6-b8b4-faa7ad41e1d5 admin admin] Calling method '>' {{(pid=122766) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:00:20.654453 ncdv-master-1 devstack@n-api.service[122766]: INFO nova.api.openstack.requestlog [None req-ba6c75d9-0df5-4dd6-b8b4-faa7ad41e1d5 admin admin] 10.222.0.16 "GET /compute/v2.1/flavors/84/os-extra_specs" status: 200 len: 19 microversion: 2.61 time: 0.011286 Jul 16 21:00:20.655104 ncdv-master-1 devstack@n-api.service[122766]: [pid: 122766|app: 0|req: 35/68] 10.222.0.16 () {60 vars in 1282 bytes} [Sat Jul 16 21:00:20 2022] GET /compute/v2.1/flavors/84/os-extra_specs => generated 19 bytes in 12 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:06:16.662307 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi [None req-885111ce-ab9d-454b-bea7-ab88cda45bf1 admin admin] Calling method '>' {{(pid=122767) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:06:16.663424 ncdv-master-1 devstack@n-api.service[122767]: INFO nova.api.openstack.requestlog [None req-885111ce-ab9d-454b-bea7-ab88cda45bf1 admin admin] 10.222.0.16 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001444 Jul 16 21:06:16.664055 ncdv-master-1 devstack@n-api.service[122767]: [pid: 122767|app: 0|req: 34/69] 10.222.0.16 () {56 vars in 1074 bytes} [Sat Jul 16 21:06:16 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:06:16.811381 ncdv-master-1 devstack@n-api.service[122766]: DEBUG dbcounter [-] [122766] Writing DB stats nova_api:SELECT=6 {{(pid=122766) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:06:16.959460 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi [None req-8dd3bfac-ba23-4a57-abb8-d2b7ac9d37e3 admin admin] Calling method '>' {{(pid=122766) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:06:16.960652 ncdv-master-1 devstack@n-api.service[122766]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-services:list": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:06:16.960652 ncdv-master-1 devstack@n-api.service[122766]: warnings.warn(msg) Jul 16 21:06:16.961959 ncdv-master-1 devstack@n-api.service[122766]: DEBUG oslo_concurrency.lockutils [None req-8dd3bfac-ba23-4a57-abb8-d2b7ac9d37e3 admin admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=122766) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:06:16.962403 ncdv-master-1 devstack@n-api.service[122766]: DEBUG oslo_concurrency.lockutils [None req-8dd3bfac-ba23-4a57-abb8-d2b7ac9d37e3 admin admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=122766) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:06:16.963426 ncdv-master-1 devstack@n-api.service[122766]: DEBUG oslo_concurrency.lockutils [None req-8dd3bfac-ba23-4a57-abb8-d2b7ac9d37e3 admin admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=122766) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:06:16.963858 ncdv-master-1 devstack@n-api.service[122766]: DEBUG oslo_concurrency.lockutils [None req-8dd3bfac-ba23-4a57-abb8-d2b7ac9d37e3 admin admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=122766) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:06:17.006123 ncdv-master-1 devstack@n-api.service[122766]: INFO nova.api.openstack.requestlog [None req-8dd3bfac-ba23-4a57-abb8-d2b7ac9d37e3 admin admin] 10.222.0.16 "GET /compute/v2.1/os-services" status: 200 len: 1242 microversion: 2.90 time: 0.196162 Jul 16 21:06:17.006685 ncdv-master-1 devstack@n-api.service[122766]: [pid: 122766|app: 0|req: 36/70] 10.222.0.16 () {60 vars in 1193 bytes} [Sat Jul 16 21:06:16 2022] GET /compute/v2.1/os-services => generated 1242 bytes in 197 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:06:17.849523 ncdv-master-1 devstack@n-api.service[122767]: DEBUG nova.api.openstack.wsgi [None req-885111ce-ab9d-454b-bea7-ab88cda45bf1 admin admin] Calling method '>' {{(pid=122767) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:06:17.851214 ncdv-master-1 devstack@n-api.service[122767]: INFO nova.api.openstack.requestlog [None req-885111ce-ab9d-454b-bea7-ab88cda45bf1 admin admin] 10.222.0.16 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001461 Jul 16 21:06:17.851825 ncdv-master-1 devstack@n-api.service[122767]: [pid: 122767|app: 0|req: 35/71] 10.222.0.16 () {56 vars in 1074 bytes} [Sat Jul 16 21:06:17 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:06:18.054636 ncdv-master-1 devstack@n-api.service[122766]: DEBUG nova.api.openstack.wsgi [None req-1426547d-cdf9-4804-8224-3279e87c4234 admin admin] Calling method '>' {{(pid=122766) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:06:18.055820 ncdv-master-1 devstack@n-api.service[122766]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-services:list": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:06:18.055820 ncdv-master-1 devstack@n-api.service[122766]: warnings.warn(msg) Jul 16 21:06:18.057016 ncdv-master-1 devstack@n-api.service[122766]: DEBUG oslo_concurrency.lockutils [None req-1426547d-cdf9-4804-8224-3279e87c4234 admin admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=122766) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:06:18.057470 ncdv-master-1 devstack@n-api.service[122766]: DEBUG oslo_concurrency.lockutils [None req-1426547d-cdf9-4804-8224-3279e87c4234 admin admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=122766) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:06:18.058401 ncdv-master-1 devstack@n-api.service[122766]: DEBUG oslo_concurrency.lockutils [None req-1426547d-cdf9-4804-8224-3279e87c4234 admin admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=122766) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:06:18.058778 ncdv-master-1 devstack@n-api.service[122766]: DEBUG oslo_concurrency.lockutils [None req-1426547d-cdf9-4804-8224-3279e87c4234 admin admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=122766) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:06:18.081939 ncdv-master-1 devstack@n-api.service[122766]: INFO nova.api.openstack.requestlog [None req-1426547d-cdf9-4804-8224-3279e87c4234 admin admin] 10.222.0.16 "GET /compute/v2.1/os-services" status: 200 len: 1242 microversion: 2.90 time: 0.082116 Jul 16 21:06:18.082513 ncdv-master-1 devstack@n-api.service[122766]: [pid: 122766|app: 0|req: 37/72] 10.222.0.16 () {60 vars in 1193 bytes} [Sat Jul 16 21:06:17 2022] GET /compute/v2.1/os-services => generated 1242 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:07:04.508742 ncdv-master-1 devstack@n-api.service[122765]: Sat Jul 16 21:07:04 2022 - graceful shutdown triggered... Jul 16 21:07:04.508878 ncdv-master-1 systemd[1]: Stopping Devstack devstack@n-api.service... Jul 16 21:07:04.509835 ncdv-master-1 devstack@n-api.service[122766]: Gracefully killing worker 1 (pid: 122766)... Jul 16 21:07:04.510296 ncdv-master-1 devstack@n-api.service[122767]: Gracefully killing worker 2 (pid: 122767)... Jul 16 21:07:05.510735 ncdv-master-1 devstack@n-api.service[122765]: worker 1 buried after 1 seconds Jul 16 21:07:05.510735 ncdv-master-1 devstack@n-api.service[122765]: worker 2 buried after 1 seconds Jul 16 21:07:05.510735 ncdv-master-1 devstack@n-api.service[122765]: goodbye to uWSGI. Jul 16 21:07:05.512415 ncdv-master-1 systemd[1]: devstack@n-api.service: Succeeded. Jul 16 21:07:05.513009 ncdv-master-1 systemd[1]: Stopped Devstack devstack@n-api.service. Jul 16 21:07:05.515976 ncdv-master-1 systemd[1]: Starting Devstack devstack@n-api.service... Jul 16 21:07:05.553963 ncdv-master-1 devstack@n-api.service[138670]: [uWSGI] getting INI configuration from /etc/nova/nova-api-uwsgi.ini Jul 16 21:07:05.556480 ncdv-master-1 devstack@n-api.service[138670]: *** Starting uWSGI 2.0.18-debian (64bit) on [Sat Jul 16 21:07:05 2022] *** Jul 16 21:07:05.556480 ncdv-master-1 devstack@n-api.service[138670]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jul 16 21:07:05.556480 ncdv-master-1 devstack@n-api.service[138670]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jul 16 21:07:05.556480 ncdv-master-1 devstack@n-api.service[138670]: nodename: ncdv-master-1 Jul 16 21:07:05.556480 ncdv-master-1 devstack@n-api.service[138670]: machine: x86_64 Jul 16 21:07:05.556480 ncdv-master-1 devstack@n-api.service[138670]: clock source: unix Jul 16 21:07:05.556480 ncdv-master-1 devstack@n-api.service[138670]: pcre jit disabled Jul 16 21:07:05.556480 ncdv-master-1 devstack@n-api.service[138670]: detected number of CPU cores: 4 Jul 16 21:07:05.556480 ncdv-master-1 devstack@n-api.service[138670]: current working directory: / Jul 16 21:07:05.556480 ncdv-master-1 devstack@n-api.service[138670]: detected binary path: /usr/bin/uwsgi-core Jul 16 21:07:05.556480 ncdv-master-1 devstack@n-api.service[138670]: your processes number limit is 47680 Jul 16 21:07:05.557246 ncdv-master-1 devstack@n-api.service[138670]: your memory page size is 4096 bytes Jul 16 21:07:05.557246 ncdv-master-1 devstack@n-api.service[138670]: detected max file descriptor number: 2048 Jul 16 21:07:05.557246 ncdv-master-1 devstack@n-api.service[138670]: lock engine: pthread robust mutexes Jul 16 21:07:05.557246 ncdv-master-1 devstack@n-api.service[138670]: thunder lock: enabled Jul 16 21:07:05.557246 ncdv-master-1 devstack@n-api.service[138670]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/nova-api-wsgi.socket fd 4 Jul 16 21:07:05.557246 ncdv-master-1 devstack@n-api.service[138670]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Jul 16 21:07:05.609700 ncdv-master-1 devstack@n-api.service[138670]: Python main interpreter initialized at 0x55ba469e09c0 Jul 16 21:07:05.609700 ncdv-master-1 devstack@n-api.service[138670]: python threads support enabled Jul 16 21:07:05.609700 ncdv-master-1 devstack@n-api.service[138670]: your server socket listen backlog is limited to 100 connections Jul 16 21:07:05.609700 ncdv-master-1 devstack@n-api.service[138670]: your mercy for graceful operations on workers is 90 seconds Jul 16 21:07:05.610146 ncdv-master-1 devstack@n-api.service[138670]: mapped 403077 bytes (393 KB) for 2 cores Jul 16 21:07:05.610146 ncdv-master-1 devstack@n-api.service[138670]: *** Operational MODE: preforking *** Jul 16 21:07:05.610433 ncdv-master-1 devstack@n-api.service[138670]: *** uWSGI is running in multiple interpreter mode *** Jul 16 21:07:05.610433 ncdv-master-1 devstack@n-api.service[138670]: spawned uWSGI master process (pid: 138670) Jul 16 21:07:05.610641 ncdv-master-1 systemd[1]: Started Devstack devstack@n-api.service. Jul 16 21:07:05.611281 ncdv-master-1 devstack@n-api.service[138670]: spawned uWSGI worker 1 (pid: 138680, cores: 1) Jul 16 21:07:05.611647 ncdv-master-1 devstack@n-api.service[138670]: spawned uWSGI worker 2 (pid: 138681, cores: 1) Jul 16 21:07:05.611818 ncdv-master-1 devstack@n-api.service[138670]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jul 16 21:07:06.922057 ncdv-master-1 devstack@n-api.service[138681]: Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code. Jul 16 21:07:06.944689 ncdv-master-1 devstack@n-api.service[138680]: Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code. Jul 16 21:07:07.748947 ncdv-master-1 devstack@n-api.service[138680]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 16 21:07:07.748947 ncdv-master-1 devstack@n-api.service[138680]: from cryptography.utils import int_from_bytes Jul 16 21:07:07.749350 ncdv-master-1 devstack@n-api.service[138680]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 16 21:07:07.749350 ncdv-master-1 devstack@n-api.service[138680]: from cryptography.utils import int_from_bytes Jul 16 21:07:07.863666 ncdv-master-1 devstack@n-api.service[138681]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 16 21:07:07.863666 ncdv-master-1 devstack@n-api.service[138681]: from cryptography.utils import int_from_bytes Jul 16 21:07:07.863666 ncdv-master-1 devstack@n-api.service[138681]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 16 21:07:07.863666 ncdv-master-1 devstack@n-api.service[138681]: from cryptography.utils import int_from_bytes Jul 16 21:07:10.346091 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 16 21:07:10.346673 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] Configuration options gathered from: {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 16 21:07:10.346884 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] command line args: [] {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 16 21:07:10.347813 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] config files: ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 16 21:07:10.348111 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] ================================================================================ {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 16 21:07:10.348516 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] allow_resize_to_same_host = True {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.348850 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] arq_binding_timeout = 300 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.349188 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries = 60 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.349530 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries_interval = 3 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.349865 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] cert = self.pem {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.350189 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] compute_driver = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.350522 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] compute_monitors = [] {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.350856 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] config_dir = [] {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.351219 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] config_drive_format = iso9660 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.351489 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] config_file = ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.351815 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] config_source = [] {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.352179 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] console_host = ncdv-master-1 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.352484 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] control_exchange = nova {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.352807 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] cpu_allocation_ratio = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.353189 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] daemon = False {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.353530 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] debug = True {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.353847 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] default_access_ip_network_name = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.354182 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] default_availability_zone = nova {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.354502 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] default_ephemeral_format = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.355069 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.355517 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] default_schedule_zone = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.355836 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] disk_allocation_ratio = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.356214 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] enable_new_services = True {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.356504 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] enabled_apis = ['osapi_compute'] {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.356832 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] enabled_ssl_apis = [] {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.357158 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] flat_injected = False {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.357494 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] force_config_drive = True {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.357818 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] force_raw_images = True {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.358150 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] graceful_shutdown_timeout = 5 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.358473 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] heal_instance_info_cache_interval = 60 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.358890 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] host = ncdv-master-1 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.359248 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] initial_cpu_allocation_ratio = 16.0 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.359662 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] initial_disk_allocation_ratio = 1.0 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.359924 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] initial_ram_allocation_ratio = 1.5 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.360357 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] injected_network_template = /opt/stack/nova/nova/virt/interfaces.template {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.360682 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] instance_build_timeout = 0 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.361005 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] instance_delete_interval = 300 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.361354 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] instance_format = [instance: %(uuid)s] {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.361677 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] instance_name_template = instance-%08x {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.361997 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit = False {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.362337 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit_period = month {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.362671 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.363047 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] instances_path = /opt/stack/data/nova/instances {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.363386 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] internal_service_availability_zone = internal {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.363704 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] key = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.364024 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] live_migration_retry_count = 30 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.364444 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] log_config_append = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.364781 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.365102 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] log_dir = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.365415 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] log_file = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.365675 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] log_options = True {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.366016 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval = 1 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.366352 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval_type = days {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.366686 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] log_rotation_type = none {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.367037 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.367303 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.367564 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.367895 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.368230 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.368554 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] long_rpc_timeout = 1800 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.368872 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_builds = 10 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.369206 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_live_migrations = 1 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.369506 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_snapshots = 5 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.369829 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] max_local_block_devices = 3 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.370136 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_count = 30 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.370452 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_size_mb = 200 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.370770 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] maximum_instance_delete_attempts = 5 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.371126 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen = 0.0.0.0 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.371454 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen_port = 8775 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.371786 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] metadata_workers = 2 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.372103 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] migrate_max_retries = -1 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.372431 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] mkisofs_cmd = genisoimage {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.372849 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] my_block_storage_ip = 10.222.0.16 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.373112 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] my_ip = 10.222.0.16 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.373430 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] network_allocate_retries = 0 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.373780 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.374113 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen = 0.0.0.0 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.374431 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen_port = 8774 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.374760 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_unique_server_name_scope = {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.375119 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_workers = 2 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.375463 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] password_length = 12 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.375760 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] periodic_enable = True {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.376078 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] periodic_fuzzy_delay = 60 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.376414 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] pointer_model = usbtablet {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.376745 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] preallocate_images = none {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.377069 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] publish_errors = False {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.377328 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] pybasedir = /opt/stack/nova {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.377645 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] ram_allocation_ratio = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.377962 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_burst = 0 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.378289 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_except_level = CRITICAL {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.378850 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_interval = 0 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.379216 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] reboot_timeout = 0 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.379537 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] reclaim_instance_interval = 0 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.379862 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] record = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.380209 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] report_interval = 10 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.380492 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] rescue_timeout = 0 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.380816 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_cpus = 0 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.381154 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_disk_mb = 0 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.381471 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_memory_mb = 512 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.381784 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] reserved_huge_pages = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.382102 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] resize_confirm_window = 0 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.382420 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] resize_fs_using_block_device = False {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.382735 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] resume_guests_state_on_host_boot = False {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.383161 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] rootwrap_config = /etc/nova/rootwrap.conf {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.383499 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_action = reap {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.383817 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_poll_interval = 1800 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.384133 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_timeout = 0 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.384451 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] scheduler_instance_sync_interval = 120 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.384771 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] service_down_time = 60 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.385125 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] servicegroup_driver = db {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.385445 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] shelved_offload_time = 0 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.385763 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] shelved_poll_interval = 3600 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.386095 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] shutdown_timeout = 0 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.386414 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] source_is_ipv6 = False {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.386735 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] ssl_only = False {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.387113 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] state_path = /opt/stack/data/nova {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.387439 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_interval = 600 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.387778 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_pool_size = 1000 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.388115 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] syslog_log_facility = LOG_USER {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.388424 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] tempdir = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.388770 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] timeout_nbd = 10 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.389095 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] transport_url = **** {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.389410 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] update_resources_interval = 0 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.389743 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] use_cow_images = True {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.390059 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] use_eventlog = False {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.390378 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] use_journal = False {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.390697 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] use_json = False {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.391054 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] use_rootwrap_daemon = False {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.391378 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] use_stderr = False {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.391695 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] use_syslog = False {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.392009 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] vcpu_pin_set = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.392336 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_is_fatal = True {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.392921 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_timeout = 300 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.393248 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] virt_mkfs = [] {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.393794 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] volume_usage_poll_interval = 0 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.393906 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] watch_log_file = False {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.395140 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] web = /usr/share/spice-html5 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.395140 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.disable_process_locking = False {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.395779 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.lock_path = /opt/stack/data/nova {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.396090 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] api.auth_strategy = keystone {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.396420 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] api.compute_link_prefix = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.396771 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.397121 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] api.dhcp_domain = novalocal {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.397455 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] api.enable_instance_password = True {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.397782 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] api.glance_link_prefix = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.398112 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_fixed_size = 100 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.398452 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_strategy = distributed {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.399056 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_per_project_cells = False {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.399392 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] api.list_records_by_skipping_down_cells = True {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.399719 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] api.local_metadata_per_cell = False {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.400052 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] api.max_limit = 1000 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.400389 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] api.metadata_cache_expiration = 15 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.400734 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] api.neutron_default_tenant_id = default {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.401065 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] api.use_forwarded_for = False {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.401397 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] api.use_neutron_default_nets = False {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.401728 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_connect_timeout = 5 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.401881 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 16 21:07:10.402361 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_failure_fatal = False {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.402559 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] Configuration options gathered from: {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 16 21:07:10.402727 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_read_timeout = 5 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.402867 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] command line args: [] {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 16 21:07:10.403028 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_ssl_certfile = {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.403185 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] config files: ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 16 21:07:10.403367 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_targets = [] {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.403515 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] ================================================================================ {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 16 21:07:10.403661 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_jsonfile_path = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.403811 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] allow_resize_to_same_host = True {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.403957 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_providers = ['StaticJSON'] {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.404096 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] arq_binding_timeout = 300 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.404243 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend = dogpile.cache.memcached {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.404418 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries = 60 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.404561 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend_argument = **** {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.404699 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries_interval = 3 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.404865 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] cache.config_prefix = cache.oslo {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.405019 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] cert = self.pem {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.405165 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] cache.dead_timeout = 60.0 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.405312 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] compute_driver = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.405452 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] cache.debug_cache_backend = False {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.405590 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] compute_monitors = [] {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.405737 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_retry_client = False {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.405890 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] config_dir = [] {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.406030 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_socket_keepalive = False {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.406169 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] config_drive_format = iso9660 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.406306 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] cache.enabled = True {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.406444 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] config_file = ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.406596 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] cache.expiration_time = 600 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.406735 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] config_source = [] {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.406872 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_attempts = 2 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.407335 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] console_host = ncdv-master-1 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.407496 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_delay = 1.0 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.407637 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] control_exchange = nova {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.407813 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_dead_retry = 300 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.407955 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] cpu_allocation_ratio = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.408102 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.408514 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] daemon = False {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.408660 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.408798 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] debug = True {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.409012 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_maxsize = 10 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.409155 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] default_access_ip_network_name = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.409299 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_unused_timeout = 60 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.409445 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] default_availability_zone = nova {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.409592 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_servers = ['localhost:11211'] {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.410283 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] default_ephemeral_format = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.410994 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_socket_timeout = 1.0 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.411194 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.411363 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] cache.proxies = [] {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.411504 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] default_schedule_zone = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.411643 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_attempts = 2 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.411791 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] disk_allocation_ratio = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.411929 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_delay = 0.0 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.412066 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] enable_new_services = True {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.412203 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_count = 1 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.412352 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] enabled_apis = ['osapi_compute'] {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.412533 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_idle = 1 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.412692 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] enabled_ssl_apis = [] {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.412852 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_interval = 1 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.412992 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] flat_injected = False {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.413131 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_allowed_ciphers = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.413274 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] force_config_drive = True {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.413420 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_cafile = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.413572 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] force_raw_images = True {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.413721 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_certfile = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.413868 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] graceful_shutdown_timeout = 5 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.414007 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_enabled = False {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.414154 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] heal_instance_info_cache_interval = 60 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.414303 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_keyfile = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.414449 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] host = ncdv-master-1 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.414588 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_section = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.414742 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] initial_cpu_allocation_ratio = 16.0 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.414889 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_type = password {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.415219 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] initial_disk_allocation_ratio = 1.0 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.415368 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cafile = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.415513 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] initial_ram_allocation_ratio = 1.5 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.415660 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] cinder.catalog_info = volumev3::publicURL {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.415660 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] cinder.certfile = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.415924 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] injected_network_template = /opt/stack/nova/nova/virt/interfaces.template {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.416077 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] cinder.collect_timing = False {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.416222 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] instance_build_timeout = 0 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.416222 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] instance_delete_interval = 300 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.416524 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cross_az_attach = True {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.416670 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] instance_format = [instance: %(uuid)s] {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.416821 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] cinder.debug = False {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.416821 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] cinder.endpoint_template = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.417111 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] instance_name_template = instance-%08x {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.417111 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit = False {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.417709 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] cinder.http_retries = 3 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.417709 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] cinder.insecure = False {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.417996 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit_period = month {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.418147 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] cinder.keyfile = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.418287 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.418435 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] cinder.os_region_name = RegionOne {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.418581 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] instances_path = /opt/stack/data/nova/instances {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.418721 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] cinder.split_loggers = False {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.418879 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] internal_service_availability_zone = internal {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.419047 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] cinder.timeout = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.419204 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] key = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.419355 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] compute.consecutive_build_service_disable_threshold = 10 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.419508 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] live_migration_retry_count = 30 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.419655 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_dedicated_set = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.419802 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] log_config_append = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.419941 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_shared_set = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.420079 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.420225 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] compute.image_type_exclude_list = [] {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.420364 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] log_dir = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.420501 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] compute.live_migration_wait_for_vif_plug = True {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.420700 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] log_file = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.420700 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] log_options = True {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.420990 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_concurrent_disk_ops = 0 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.421130 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval = 1 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.421278 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_disk_devices_to_attach = -1 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.421426 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval_type = days {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.421564 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] compute.packing_host_numa_cells_allocation_strategy = True {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.421701 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] log_rotation_type = none {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.421845 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] compute.provider_config_location = /etc/nova/provider_config/ {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.421983 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.422125 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] compute.resource_provider_association_refresh = 300 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.422271 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.422413 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] compute.shutdown_retry_interval = 10 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.422413 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] conductor.workers = 2 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.422688 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.422836 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] console.allowed_origins = [] {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.422987 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.423203 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_ciphers = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.423351 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.423497 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_minimum_version = default {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.423643 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] long_rpc_timeout = 1800 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.423643 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_builds = 10 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.423928 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] consoleauth.token_ttl = 600 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.424076 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_live_migrations = 1 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.424222 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.cafile = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.424380 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_snapshots = 5 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.424519 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.certfile = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.424656 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] max_local_block_devices = 3 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.424809 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.collect_timing = False {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.424949 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_count = 30 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.425086 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retries = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.425232 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_size_mb = 200 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.425377 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retry_delay = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.425596 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] maximum_instance_delete_attempts = 5 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.426392 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.endpoint_override = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.426560 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen = 0.0.0.0 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.426702 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.insecure = False {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.426843 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen_port = 8775 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.427001 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.keyfile = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.427001 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.max_version = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.427993 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] metadata_workers = 2 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.428144 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.min_version = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.428284 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] migrate_max_retries = -1 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.428500 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.region_name = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.428684 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] mkisofs_cmd = genisoimage {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.428830 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_name = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.428976 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] my_block_storage_ip = 10.222.0.16 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.429129 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_type = accelerator {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.429269 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] my_ip = 10.222.0.16 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.429419 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.split_loggers = False {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.429566 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] network_allocate_retries = 0 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.429717 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retries = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.429864 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.430004 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retry_delay = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.430150 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen = 0.0.0.0 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.430289 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.timeout = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.430434 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen_port = 8774 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.430574 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.valid_interfaces = ['internal', 'public'] {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.430728 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_unique_server_name_scope = {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.430879 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.version = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.431058 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_workers = 2 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.431214 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] database.backend = sqlalchemy {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.431354 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] password_length = 12 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.431354 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] periodic_enable = True {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.431637 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] database.connection = **** {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.431776 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] periodic_fuzzy_delay = 60 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.431921 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_debug = 0 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.432067 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] pointer_model = usbtablet {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.432240 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_parameters = {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.432421 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] preallocate_images = none {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.432581 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_recycle_time = 3600 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.432722 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] publish_errors = False {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.432860 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_trace = False {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.433008 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] pybasedir = /opt/stack/nova {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.433155 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] database.db_inc_retry_interval = True {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.433307 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] ram_allocation_ratio = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.433457 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retries = 20 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.433605 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_burst = 0 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.433744 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retry_interval = 10 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.433890 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_except_level = CRITICAL {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.434042 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] database.db_retry_interval = 1 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.434190 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_interval = 0 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.434330 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] database.max_overflow = 50 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.434474 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] reboot_timeout = 0 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.434621 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] database.max_pool_size = 5 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.434761 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] reclaim_instance_interval = 0 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.434899 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] database.max_retries = 10 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.435068 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] record = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.435215 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_enable_ndb = False {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.435362 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] report_interval = 10 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.435508 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_sql_mode = TRADITIONAL {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.435647 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] rescue_timeout = 0 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.435647 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_cpus = 0 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.435922 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] database.pool_timeout = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.436069 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_disk_mb = 0 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.436208 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] database.retry_interval = 10 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.436345 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_memory_mb = 512 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.436490 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] database.slave_connection = **** {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.436629 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] reserved_huge_pages = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.436848 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] database.sqlite_synchronous = True {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.436997 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] resize_confirm_window = 0 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.437137 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] api_database.backend = sqlalchemy {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.437292 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] resize_fs_using_block_device = False {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.437432 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection = **** {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.437893 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] resume_guests_state_on_host_boot = False {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.438044 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_debug = 0 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.438182 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] rootwrap_config = /etc/nova/rootwrap.conf {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.438321 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_parameters = {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.438468 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_action = reap {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.438608 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_recycle_time = 3600 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.438758 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_poll_interval = 1800 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.438943 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_trace = False {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.439202 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_timeout = 0 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.439345 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_inc_retry_interval = True {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.439492 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] scheduler_instance_sync_interval = 120 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.439654 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retries = 20 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.439809 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] service_down_time = 60 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.439958 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retry_interval = 10 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.440098 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] servicegroup_driver = db {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.440254 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_retry_interval = 1 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.440410 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] shelved_offload_time = 0 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.440554 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_overflow = 50 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.440692 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] shelved_poll_interval = 3600 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.440829 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_pool_size = 5 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.440968 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] shutdown_timeout = 0 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.440968 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] source_is_ipv6 = False {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.441785 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_retries = 10 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.441939 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] ssl_only = False {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.442099 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_enable_ndb = False {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.442242 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] state_path = /opt/stack/data/nova {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.442391 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_sql_mode = TRADITIONAL {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.442540 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_interval = 600 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.442697 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] api_database.pool_timeout = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.442837 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_pool_size = 1000 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.442984 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] api_database.retry_interval = 10 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.442984 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] api_database.slave_connection = **** {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.443293 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] syslog_log_facility = LOG_USER {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.443441 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] api_database.sqlite_synchronous = True {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.443587 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] tempdir = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.443727 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] devices.enabled_mdev_types = [] {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.443876 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] timeout_nbd = 10 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.444016 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.cipher = aes-xts-plain64 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.444169 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] transport_url = **** {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.444317 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.enabled = False {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.444463 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] update_resources_interval = 0 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.444602 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.key_size = 512 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.444782 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] use_cow_images = True {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.444941 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] glance.api_servers = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.445095 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] use_eventlog = False {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.445236 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] glance.cafile = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.445374 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] use_journal = False {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.445524 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] glance.certfile = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.445679 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] use_json = False {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.445829 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] glance.collect_timing = False {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.445977 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] use_rootwrap_daemon = False {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.446124 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retries = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.446264 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] use_stderr = False {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.446410 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retry_delay = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.446556 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] use_syslog = False {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.446708 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] glance.debug = False {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.446849 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] vcpu_pin_set = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.447054 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] glance.default_trusted_certificate_ids = [] {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.447213 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_is_fatal = True {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.447362 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_certificate_validation = False {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.447502 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_timeout = 300 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.447672 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_rbd_download = False {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.448134 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] virt_mkfs = [] {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.448293 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] glance.endpoint_override = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.448441 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] volume_usage_poll_interval = 0 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.448441 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] watch_log_file = False {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.448748 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] glance.insecure = False {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.448748 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] glance.keyfile = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.449028 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] web = /usr/share/spice-html5 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 16 21:07:10.449168 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] glance.max_version = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.449316 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.disable_process_locking = False {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.449463 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] glance.min_version = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.449610 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.lock_path = /opt/stack/data/nova {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.449762 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] glance.num_retries = 3 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.449901 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] api.auth_strategy = keystone {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.450053 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_ceph_conf = {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.450200 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] api.compute_link_prefix = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.450340 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_connect_timeout = 5 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.450498 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.450650 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_pool = {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.450796 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] api.dhcp_domain = novalocal {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.450935 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_user = {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.451108 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] api.enable_instance_password = True {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.451259 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] glance.region_name = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.451412 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] api.glance_link_prefix = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.451561 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_name = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.451700 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_fixed_size = 100 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.451857 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_type = image {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.451996 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_strategy = distributed {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.452142 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] glance.split_loggers = False {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.452290 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_per_project_cells = False {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.452437 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retries = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.452578 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] api.list_records_by_skipping_down_cells = True {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.452719 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retry_delay = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.452864 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] api.local_metadata_per_cell = False {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.453013 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] glance.timeout = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.453152 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] api.max_limit = 1000 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.453297 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] glance.valid_interfaces = ['internal', 'public'] {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.453442 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] api.metadata_cache_expiration = 15 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.453590 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] glance.verify_glance_signatures = False {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.453747 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] api.neutron_default_tenant_id = default {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.453901 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] glance.version = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.454052 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] api.use_forwarded_for = False {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.454200 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] guestfs.debug = False {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.454345 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] api.use_neutron_default_nets = False {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.454492 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_cdrom = False {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.454639 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_connect_timeout = 5 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.454792 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_inject_password = False {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.454931 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_failure_fatal = False {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.455113 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.dynamic_memory_ratio = 1.0 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.455263 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_read_timeout = 5 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.455403 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_instance_metrics_collection = False {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.455542 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_ssl_certfile = {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.455704 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_remotefx = False {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.455852 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_targets = [] {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.455991 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.instances_path_share = {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.456129 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_jsonfile_path = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.456275 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.iscsi_initiator_list = [] {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.456427 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_providers = ['StaticJSON'] {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.457078 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.limit_cpu_features = False {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.457695 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend = dogpile.cache.memcached {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.457853 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_count = 10 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.458311 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend_argument = **** {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.458470 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_interval = 5 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.458621 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] cache.config_prefix = cache.oslo {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.458776 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_check_timeframe = 60 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.458929 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] cache.dead_timeout = 60.0 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.459091 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_event_polling_interval = 2 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.459235 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] cache.debug_cache_backend = False {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.459373 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.qemu_img_cmd = qemu-img.exe {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.459373 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.use_multipath_io = False {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.459788 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_retry_client = False {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.459958 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_count = 10 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.460101 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_socket_keepalive = False {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.460254 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_interval = 5 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.460405 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] cache.enabled = True {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.460555 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.vswitch_name = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.460694 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] cache.expiration_time = 600 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.460883 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.wait_soft_reboot_seconds = 60 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.461042 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_attempts = 2 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.461190 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] mks.enabled = False {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.461343 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_delay = 1.0 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.461483 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] mks.mksproxy_base_url = http://127.0.0.1:6090/ {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.461621 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_dead_retry = 300 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.461621 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.461900 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.manager_interval = 2400 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.462048 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.462200 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.precache_concurrency = 1 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.462339 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_maxsize = 10 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.462484 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_base_images = True {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.462623 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_unused_timeout = 60 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.462771 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_original_minimum_age_seconds = 86400 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.462910 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_servers = ['localhost:11211'] {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.463107 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_resized_minimum_age_seconds = 3600 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.463261 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_socket_timeout = 1.0 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.463409 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.subdirectory_name = _base {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.463548 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] cache.proxies = [] {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.463694 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_max_retries = 60 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.463841 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_attempts = 2 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.463994 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_retry_interval = 2 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.464136 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_delay = 0.0 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.464287 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_section = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.464435 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_count = 1 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.464627 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_type = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.464770 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_idle = 1 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.464909 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] ironic.cafile = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.465047 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_interval = 1 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.465193 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] ironic.certfile = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.465418 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_allowed_ciphers = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.465567 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] ironic.collect_timing = False {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.465707 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_cafile = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.465845 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retries = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.465983 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_certfile = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.466129 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retry_delay = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.466277 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_enabled = False {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.466416 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] ironic.endpoint_override = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.466565 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_keyfile = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.466710 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] ironic.insecure = False {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.466849 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_section = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.466995 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] ironic.keyfile = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.467183 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_type = password {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.467324 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] ironic.max_version = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.467470 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cafile = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.467609 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] ironic.min_version = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.467747 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] cinder.catalog_info = volumev3::publicURL {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.467898 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] ironic.partition_key = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.468045 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] cinder.certfile = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.468527 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] ironic.peer_list = [] {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.468686 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] cinder.collect_timing = False {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.468839 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] ironic.region_name = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.468978 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cross_az_attach = True {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.469131 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] ironic.serial_console_state_timeout = 10 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.469281 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] cinder.debug = False {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.469419 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_name = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.469558 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] cinder.endpoint_template = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.469704 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_type = baremetal {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.469852 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] cinder.http_retries = 3 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.470005 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] ironic.split_loggers = False {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.470153 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] cinder.insecure = False {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.470301 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retries = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.470449 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] cinder.keyfile = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.470595 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retry_delay = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.470748 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] cinder.os_region_name = RegionOne {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.470901 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] ironic.timeout = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.471097 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] cinder.split_loggers = False {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.471255 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] ironic.valid_interfaces = ['internal', 'public'] {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.471397 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] cinder.timeout = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.471536 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] ironic.version = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.471536 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.backend = nova.keymgr.conf_key_mgr.ConfKeyManager {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.471809 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] compute.consecutive_build_service_disable_threshold = 10 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.471809 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_dedicated_set = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.472071 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.fixed_key = **** {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.472782 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_shared_set = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.473238 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.473794 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] compute.image_type_exclude_list = [] {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.473949 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_api_version = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.473949 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.473949 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint_type = public {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.474356 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] compute.live_migration_wait_for_vif_plug = True {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.474496 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_region_name = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.474649 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_concurrent_disk_ops = 0 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.474798 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] barbican.cafile = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.474798 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] barbican.certfile = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.475115 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_disk_devices_to_attach = -1 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.475276 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] barbican.collect_timing = False {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.475429 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] compute.packing_host_numa_cells_allocation_strategy = True {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.475698 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] barbican.insecure = False {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.475865 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] compute.provider_config_location = /etc/nova/provider_config/ {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.476019 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] barbican.keyfile = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.476019 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] barbican.number_of_retries = 60 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.476311 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] compute.resource_provider_association_refresh = 300 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.476471 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] barbican.retry_delay = 1 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.476618 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] compute.shutdown_retry_interval = 10 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.476764 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] barbican.send_service_user_token = False {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.476953 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] conductor.workers = 2 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.477112 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] barbican.split_loggers = False {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.477112 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] barbican.timeout = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.477398 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] console.allowed_origins = [] {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.477543 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl = True {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.477695 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_ciphers = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.477854 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl_path = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.477999 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_minimum_version = default {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.478143 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_section = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.478314 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] consoleauth.token_ttl = 600 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.478720 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_type = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.478954 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.cafile = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.479155 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.cafile = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.479303 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.certfile = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.479460 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.certfile = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.479617 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.collect_timing = False {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.479765 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.collect_timing = False {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.479917 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retries = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.480070 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.insecure = False {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.480236 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retry_delay = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.480436 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.keyfile = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.480596 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.endpoint_override = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.480749 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.split_loggers = False {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.480911 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.insecure = False {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.481068 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.timeout = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.481222 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.keyfile = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.481381 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_role_id = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.481528 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.max_version = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.481681 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_secret_id = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.481835 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.min_version = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.481980 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] vault.cafile = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.482136 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.region_name = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.482294 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] vault.certfile = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.482441 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_name = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.482595 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] vault.collect_timing = False {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.482741 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_type = accelerator {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.482886 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] vault.insecure = False {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.483065 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.split_loggers = False {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.483234 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] vault.keyfile = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.483381 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retries = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.483525 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_mountpoint = secret {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.483670 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retry_delay = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.483813 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_version = 2 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.483967 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.timeout = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.484135 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] vault.namespace = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.484282 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.valid_interfaces = ['internal', 'public'] {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.484426 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] vault.root_token_id = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.484577 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.version = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.484722 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] vault.split_loggers = False {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.484881 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] database.backend = sqlalchemy {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.485027 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] vault.ssl_ca_crt_file = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.485180 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] database.connection = **** {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.485324 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] vault.timeout = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.485476 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_debug = 0 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.485620 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] vault.use_ssl = False {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.485776 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_parameters = {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.485921 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] vault.vault_url = http://127.0.0.1:8200 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.486077 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_recycle_time = 3600 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.486222 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] keystone.cafile = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.486373 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_trace = False {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.486518 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] keystone.certfile = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.486675 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] database.db_inc_retry_interval = True {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.486821 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] keystone.collect_timing = False {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.486972 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retries = 20 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.487156 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retries = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.487261 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retry_interval = 10 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.487369 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retry_delay = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.487468 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] database.db_retry_interval = 1 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.487555 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] keystone.endpoint_override = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.487638 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] database.max_overflow = 50 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.487748 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] keystone.insecure = False {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.487855 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] database.max_pool_size = 5 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.487962 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] keystone.keyfile = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.488044 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] database.max_retries = 10 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.488120 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] keystone.max_version = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.488195 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_enable_ndb = False {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.488270 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] keystone.min_version = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.488458 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_sql_mode = TRADITIONAL {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.488557 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] keystone.region_name = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.488630 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] database.pool_timeout = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.488705 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_name = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.488782 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] database.retry_interval = 10 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.488893 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_type = identity {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.488991 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] database.slave_connection = **** {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.489063 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] keystone.split_loggers = False {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.489139 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] database.sqlite_synchronous = True {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.489208 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retries = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.489283 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] api_database.backend = sqlalchemy {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.489352 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retry_delay = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.489510 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection = **** {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.489694 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] keystone.timeout = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.489847 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_debug = 0 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.490000 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] keystone.valid_interfaces = ['internal', 'public'] {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.490156 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_parameters = {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.490303 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] keystone.version = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.490474 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_recycle_time = 3600 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.490621 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.connection_uri = {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.490773 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_trace = False {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.490919 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_mode = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.491101 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_inc_retry_interval = True {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.491271 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_model_extra_flags = [] {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.491428 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retries = 20 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.491594 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_models = [] {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.491749 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retry_interval = 10 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.491894 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_attempts = 8 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.492038 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_retry_interval = 1 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.492220 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_timeout = 20 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.492375 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_overflow = 50 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.492531 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_cachemodes = [] {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.492676 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_pool_size = 5 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.492843 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_prefix = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.493028 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_retries = 10 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.493187 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.enabled_perf_events = [] {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.493343 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_enable_ndb = False {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.493498 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.file_backed_memory = 0 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.493652 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_sql_mode = TRADITIONAL {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.493798 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.gid_maps = [] {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.493950 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] api_database.pool_timeout = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.494105 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_disk_discard = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.494260 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] api_database.retry_interval = 10 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.494407 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_machine_type = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.494569 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] api_database.slave_connection = **** {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.494716 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_ceph_conf = {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.494861 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] api_database.sqlite_synchronous = True {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.495044 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_poll_interval = 15 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.495210 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] devices.enabled_mdev_types = [] {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.495364 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_timeout = 600 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.495512 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.cipher = aes-xts-plain64 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.495656 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_store_name = {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.495801 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.enabled = False {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.495960 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_pool = rbd {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.496107 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.key_size = 512 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.496258 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_type = default {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.496415 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] glance.api_servers = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.496572 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_volume_group = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.496834 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] glance.cafile = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.497000 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_key = False {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.497147 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] glance.certfile = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.497291 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_partition = -2 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.497435 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] glance.collect_timing = False {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.497590 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_password = False {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.497747 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retries = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.497899 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iscsi_iface = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.498052 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retry_delay = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.498198 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iser_use_multipath = False {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.498342 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] glance.debug = False {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.498493 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_bandwidth = 0 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.498661 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] glance.default_trusted_certificate_ids = [] {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.498814 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_completion_timeout = 800 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.498977 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_certificate_validation = False {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.499380 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime = 500 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.499534 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_rbd_download = False {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.499693 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_delay = 75 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.499852 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] glance.endpoint_override = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.500007 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_steps = 10 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.500160 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] glance.insecure = False {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.500305 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_inbound_addr = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.500451 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] glance.keyfile = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.500610 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_auto_converge = False {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.500778 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] glance.max_version = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.500924 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_post_copy = False {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.501076 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] glance.min_version = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.501240 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_scheme = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.501387 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] glance.num_retries = 3 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.501541 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_timeout_action = abort {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.501710 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_ceph_conf = {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.501871 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_tunnelled = False {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.692564 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_connect_timeout = 5 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.692829 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_uri = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.692962 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_pool = {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.693048 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_with_native_tls = False {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.693146 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_user = {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.693239 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.max_queues = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.693306 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] glance.region_name = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.693375 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.mem_stats_period_seconds = 10 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.693446 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_name = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.693507 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_options = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.693578 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_type = image {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.693578 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] glance.split_loggers = False {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.693671 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.693747 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retries = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.693815 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_aoe_discover_tries = 3 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.693886 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retry_delay = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.693969 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_iser_scan_tries = 5 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.694043 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] glance.timeout = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.694127 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_memory_encrypted_guests = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.694198 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] glance.valid_interfaces = ['internal', 'public'] {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.694264 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_nvme_discover_tries = 5 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.694346 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] glance.verify_glance_signatures = False {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.694416 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_pcie_ports = 0 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.694472 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] glance.version = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.694524 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_volume_scan_tries = 5 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.694574 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] guestfs.debug = False {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.694699 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.pmem_namespaces = [] {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.694756 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_cdrom = False {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.694832 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_client_cfg = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.694911 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_inject_password = False {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.694990 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_mount_point_base = /opt/stack/data/nova/mnt {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.695128 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.dynamic_memory_ratio = 1.0 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.695208 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_connect_timeout = 5 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.695280 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_instance_metrics_collection = False {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.695343 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retries = 12 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.695408 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_remotefx = False {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.695481 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retry_interval = 5 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.695559 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.instances_path_share = {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.695621 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_secret_uuid = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.695698 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.iscsi_initiator_list = [] {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.695756 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_user = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.695809 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.limit_cpu_features = False {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.695862 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.realtime_scheduler_priority = 1 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.695925 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_count = 10 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.695998 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.remote_filesystem_transport = ssh {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.696066 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_interval = 5 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.696137 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_image_id = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.696193 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_check_timeframe = 60 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.696267 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_kernel_id = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.696336 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_event_polling_interval = 2 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.696402 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_ramdisk_id = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.696480 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.qemu_img_cmd = qemu-img.exe {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.696550 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rng_dev_path = /dev/urandom {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.696616 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.use_multipath_io = False {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.696692 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rx_queue_size = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.696769 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_count = 10 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.696845 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_options = {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.696920 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_interval = 5 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.696920 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.vswitch_name = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.697051 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.697051 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_compression = False {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.697281 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.wait_soft_reboot_seconds = 60 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.697365 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_image_format = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.697441 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] mks.enabled = False {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.697526 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshots_directory = /opt/stack/data/nova/instances/snapshots {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.697605 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] mks.mksproxy_base_url = http://127.0.0.1:6090/ {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.697683 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sparse_logical_volumes = False {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.697758 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.manager_interval = 2400 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.697837 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_enabled = False {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.697912 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.precache_concurrency = 1 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.697992 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_group = tss {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.698071 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_base_images = True {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.698215 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_user = tss {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.698308 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_original_minimum_age_seconds = 86400 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.698391 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sysinfo_serial = unique {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.698471 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_resized_minimum_age_seconds = 3600 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.698538 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.tx_queue_size = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.698612 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.subdirectory_name = _base {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.698671 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.uid_maps = [] {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.698724 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_max_retries = 60 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.698784 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.use_virtio_for_bridges = True {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.698839 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_retry_interval = 2 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.698892 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.virt_type = kvm {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.698945 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_section = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.698998 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear = zero {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.699119 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_type = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.699189 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear_size = 0 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.699256 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] ironic.cafile = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.699321 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_use_multipath = False {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.699386 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] ironic.certfile = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.699452 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_cache_path = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.699526 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] ironic.collect_timing = False {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.699594 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_log_path = /var/log/vstorage/%(cluster_name)s/nova.log.gz {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.699663 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retries = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.699718 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_group = qemu {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.699782 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retry_delay = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.699837 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_opts = [] {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.699913 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] ironic.endpoint_override = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.699982 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_perms = 0770 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.700048 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] ironic.insecure = False {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.700048 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] ironic.keyfile = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.700159 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_point_base = /opt/stack/data/nova/mnt {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.700221 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] ironic.max_version = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.700288 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_user = stack {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.700354 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] ironic.min_version = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.700420 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.wait_soft_reboot_seconds = 120 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.700475 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] ironic.partition_key = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.700539 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_section = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.700601 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] ironic.peer_list = [] {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.700657 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_type = password {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.700731 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] ironic.region_name = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.700809 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] neutron.cafile = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.700864 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] ironic.serial_console_state_timeout = 10 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.700916 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] neutron.certfile = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.700988 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_name = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.701056 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] neutron.collect_timing = False {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.701056 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retries = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.701210 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_type = baremetal {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.701288 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retry_delay = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.701358 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] ironic.split_loggers = False {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.701425 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] neutron.default_floating_pool = public {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.701501 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retries = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.701570 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] neutron.endpoint_override = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.701570 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] neutron.extension_sync_interval = 600 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.701835 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retry_delay = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.701957 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] neutron.http_retries = 3 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.702104 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] ironic.timeout = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.702184 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] neutron.insecure = False {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.702254 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] ironic.valid_interfaces = ['internal', 'public'] {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.702323 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] neutron.keyfile = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.702391 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] ironic.version = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.702460 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] neutron.max_version = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.702525 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.backend = nova.keymgr.conf_key_mgr.ConfKeyManager {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.702595 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] neutron.metadata_proxy_shared_secret = **** {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.702651 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.fixed_key = **** {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.702796 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] neutron.min_version = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.702874 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.702943 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] neutron.ovs_bridge = br-int {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.703035 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_api_version = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.703110 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] neutron.physnets = [] {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.703178 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.703245 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] neutron.region_name = RegionOne {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.703312 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint_type = public {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.703368 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_metadata_proxy = True {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.703446 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_region_name = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.703518 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_name = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.703586 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] barbican.cafile = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.703654 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_type = network {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.703721 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] barbican.certfile = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.703797 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] neutron.split_loggers = False {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.703867 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] barbican.collect_timing = False {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.703935 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retries = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.704002 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] barbican.insecure = False {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.704070 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retry_delay = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.704138 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] barbican.keyfile = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.704213 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] neutron.timeout = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.704283 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] barbican.number_of_retries = 60 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.704351 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] neutron.valid_interfaces = ['internal', 'public'] {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.704419 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] barbican.retry_delay = 1 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.704486 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] neutron.version = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.704542 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] barbican.send_service_user_token = False {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.704617 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] notifications.bdms_in_notifications = False {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.704687 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] barbican.split_loggers = False {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.704687 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] barbican.timeout = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.704798 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] notifications.default_level = INFO {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.704852 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl = True {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.704918 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notification_format = unversioned {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.704993 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl_path = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.705063 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notify_on_state_change = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.705130 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_section = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.705197 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] notifications.versioned_notifications_topics = ['versioned_notifications'] {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.705264 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_type = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.705338 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] pci.alias = [] {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.705408 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.cafile = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.705475 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] pci.passthrough_whitelist = [] {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.705542 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.certfile = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.705609 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_section = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.705675 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.collect_timing = False {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.705750 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_type = password {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.705819 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.insecure = False {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.705886 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] placement.cafile = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.705953 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.keyfile = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.706020 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] placement.certfile = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.706086 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.split_loggers = False {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.706161 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] placement.collect_timing = False {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.706230 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.timeout = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.706298 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retries = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.706365 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_role_id = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.706438 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retry_delay = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.706581 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_secret_id = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.706762 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] placement.endpoint_override = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.707881 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] vault.cafile = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.708099 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] placement.insecure = False {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.708179 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] vault.certfile = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.708249 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] placement.keyfile = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.708328 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] vault.collect_timing = False {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.708388 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] placement.max_version = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.708461 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] vault.insecure = False {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.708536 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] placement.min_version = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.708593 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] vault.keyfile = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.708659 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] placement.region_name = RegionOne {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.708734 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_mountpoint = secret {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.708805 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_name = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.708872 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_version = 2 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.708939 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_type = placement {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.709005 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] vault.namespace = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.709005 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] vault.root_token_id = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.709112 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] placement.split_loggers = False {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.709181 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] vault.split_loggers = False {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.709248 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retries = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.709315 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] vault.ssl_ca_crt_file = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.709370 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retry_delay = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.709448 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] vault.timeout = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.709537 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] placement.timeout = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.709601 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] vault.use_ssl = False {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.709655 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] placement.valid_interfaces = ['internal', 'public'] {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.709709 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] vault.vault_url = http://127.0.0.1:8200 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.709775 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] placement.version = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.709839 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] keystone.cafile = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.709896 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] powervm.disk_driver = localdisk {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.709962 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] keystone.certfile = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.709962 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] keystone.collect_timing = False {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.710083 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] powervm.proc_units_factor = 0.1 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.710150 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retries = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.710150 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retry_delay = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.710279 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] powervm.volume_group_name = {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.710434 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] keystone.endpoint_override = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.710516 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] quota.cores = 20 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.710586 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] keystone.insecure = False {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.710654 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] quota.count_usage_from_placement = False {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.710730 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] keystone.keyfile = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.710801 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] quota.driver = nova.quota.DbQuotaDriver {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.710877 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] keystone.max_version = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.710877 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] keystone.min_version = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.710999 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_content_bytes = 10240 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.711089 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] keystone.region_name = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.711168 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_path_length = 255 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.711242 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_name = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.711299 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_files = 5 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.711469 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_type = identity {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.711547 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] quota.instances = 10 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.711617 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] keystone.split_loggers = False {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.711692 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] quota.key_pairs = 100 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.711763 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retries = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.711763 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retry_delay = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.711874 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] quota.metadata_items = 128 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.711942 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] keystone.timeout = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.712009 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] quota.ram = 51200 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.712084 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] keystone.valid_interfaces = ['internal', 'public'] {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.712156 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] quota.recheck_quota = True {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.718691 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] keystone.version = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.718762 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_group_members = 10 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.718914 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.connection_uri = {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.718975 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_groups = 10 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.719135 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_mode = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.719202 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] rdp.enabled = False {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.719265 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_model_extra_flags = [] {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.719265 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_models = [] {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.719365 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] rdp.html5_proxy_base_url = http://127.0.0.1:6083/ {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.719420 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_attempts = 8 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.719484 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.discover_hosts_in_cells_interval = -1 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.719541 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_timeout = 20 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.719541 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_cachemodes = [] {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.719664 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.enable_isolated_aggregate_filtering = False {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.719725 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_prefix = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.719788 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.image_metadata_prefilter = False {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.719966 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.enabled_perf_events = [] {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.720100 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.limit_tenants_to_placement_aggregate = False {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.720187 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.file_backed_memory = 0 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.720187 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.gid_maps = [] {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.720371 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_attempts = 3 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.720441 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_disk_discard = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.720499 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_placement_results = 1000 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.720554 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_machine_type = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.720626 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.placement_aggregate_required_for_tenants = False {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.720683 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_ceph_conf = {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.720738 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_availability_zone = True {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.720801 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_poll_interval = 15 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.720857 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_image_type_support = False {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.720912 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_timeout = 600 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.721108 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_routed_network_aggregates = False {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.721188 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_store_name = {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.721325 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.workers = 2 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.721416 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_pool = rbd {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.721416 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_type = default {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.721535 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_namespace = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.721591 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_volume_group = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.721702 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_separator = . {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.721775 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_key = False {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.721834 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.721912 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_partition = -2 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.721912 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_password = False {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.722011 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.build_failure_weight_multiplier = 1000000.0 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.722065 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iscsi_iface = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.722188 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cpu_weight_multiplier = 1.0 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.722294 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iser_use_multipath = False {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.725758 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.725852 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_bandwidth = 0 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.725950 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.disk_weight_multiplier = 1.0 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.726053 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_completion_timeout = 800 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.726859 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.enabled_filters = ['AvailabilityZoneFilter', 'ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter', 'SameHostFilter', 'DifferentHostFilter'] {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.726940 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime = 500 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.727000 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.host_subset_size = 1 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.727123 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_delay = 75 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.727189 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.image_properties_default_architecture = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.727276 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_steps = 10 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.727353 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.io_ops_weight_multiplier = -1.0 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.727410 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_inbound_addr = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.727480 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_hosts = [] {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.727539 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_auto_converge = False {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.727595 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_images = [] {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.727689 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_post_copy = False {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.727750 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_instances_per_host = 50 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.727821 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_scheme = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.727880 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_io_ops_per_host = 8 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.727934 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_timeout_action = abort {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.728038 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.pci_weight_multiplier = 1.0 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.728137 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_tunnelled = False {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.728203 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.ram_weight_multiplier = 1.0 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.729449 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_uri = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.729663 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.729819 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_with_native_tls = False {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.730060 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.shuffle_best_same_weighed_hosts = False {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.730219 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.max_queues = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.730629 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_affinity_weight_multiplier = 1.0 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.730868 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.mem_stats_period_seconds = 10 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.731089 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.731277 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_options = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.731426 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.track_instance_changes = False {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.731581 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.731713 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.731844 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_aoe_discover_tries = 3 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.731973 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] metrics.required = True {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.731973 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_multiplier = 1.0 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.732230 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_iser_scan_tries = 5 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.732230 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_memory_encrypted_guests = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.732491 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_of_unavailable = -10000.0 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.732621 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_nvme_discover_tries = 5 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.732793 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_setting = [] {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.732942 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_pcie_ports = 0 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.732942 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_volume_scan_tries = 5 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.733212 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.base_url = ws://127.0.0.1:6083/ {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.733410 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.pmem_namespaces = [] {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.733410 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_client_cfg = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.733668 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.enabled = False {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.733822 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_mount_point_base = /opt/stack/data/nova/mnt {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.733994 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.port_range = 10000:20000 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.734160 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_connect_timeout = 5 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.734397 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.proxyclient_address = 127.0.0.1 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.734554 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retries = 12 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.734554 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retry_interval = 5 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.735046 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_host = 0.0.0.0 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.735258 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_secret_uuid = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.735920 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_port = 6083 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.736109 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_user = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.736267 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_section = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.736460 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.realtime_scheduler_priority = 1 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.736599 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_type = password {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.736803 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.remote_filesystem_transport = ssh {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.736803 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_image_id = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.737196 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] service_user.cafile = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.737411 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_kernel_id = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.737411 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_ramdisk_id = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.737411 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rng_dev_path = /dev/urandom {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.737411 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rx_queue_size = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.738053 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] service_user.certfile = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.738833 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_options = {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.738833 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.739315 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] service_user.collect_timing = False {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.739450 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_compression = False {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.739450 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_image_format = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.739774 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] service_user.insecure = False {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.740009 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshots_directory = /opt/stack/data/nova/instances/snapshots {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.740260 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] service_user.keyfile = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.740424 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sparse_logical_volumes = False {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.740424 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_enabled = False {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.740678 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] service_user.send_service_user_token = True {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.740857 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_group = tss {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.741069 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] service_user.split_loggers = False {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.741218 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_user = tss {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.741218 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sysinfo_serial = unique {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.741525 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] service_user.timeout = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.741694 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.tx_queue_size = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.741694 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.uid_maps = [] {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.741694 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.use_virtio_for_bridges = True {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.742103 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] spice.agent_enabled = True {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.742289 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.virt_type = kvm {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.742289 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear = zero {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.742289 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear_size = 0 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.742790 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] spice.enabled = False {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.742970 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_use_multipath = False {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.742970 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_cache_path = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.742970 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_log_path = /var/log/vstorage/%(cluster_name)s/nova.log.gz {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.743609 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_base_url = http://127.0.0.1:6082/spice_auto.html {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.744467 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_group = qemu {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.744621 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_host = 0.0.0.0 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.744782 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_opts = [] {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.744782 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_perms = 0770 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.744782 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_point_base = /opt/stack/data/nova/mnt {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.745225 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_port = 6082 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.745376 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_user = stack {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.746116 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_listen = 127.0.0.1 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.746296 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.wait_soft_reboot_seconds = 120 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.746468 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_proxyclient_address = 127.0.0.1 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.746633 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_section = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.746633 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_type = password {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.746633 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] neutron.cafile = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.746633 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] neutron.certfile = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.747346 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.baseapi = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.747504 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] neutron.collect_timing = False {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.747504 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retries = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.747504 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retry_delay = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.747994 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.cert = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.748162 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] neutron.default_floating_pool = public {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.748162 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] neutron.endpoint_override = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.748479 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.compute = auto {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.748668 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] neutron.extension_sync_interval = 600 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.748668 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] neutron.http_retries = 3 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.748668 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] neutron.insecure = False {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.749250 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.conductor = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.749431 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] neutron.keyfile = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.749649 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.scheduler = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.749822 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] neutron.max_version = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.749822 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] neutron.metadata_proxy_shared_secret = **** {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.750211 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_section = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.750390 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] neutron.min_version = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.750529 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_type = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.750662 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] neutron.ovs_bridge = br-int {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.750794 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.cafile = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.750934 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] neutron.physnets = [] {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.751148 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.certfile = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.751670 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] neutron.region_name = RegionOne {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.751854 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.collect_timing = False {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.752032 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_metadata_proxy = True {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.752205 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.insecure = False {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.752388 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_name = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.752388 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_type = network {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.752656 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.keyfile = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.752823 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] neutron.split_loggers = False {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.752963 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.split_loggers = False {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.753119 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retries = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.753119 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retry_delay = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.754001 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.timeout = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.754842 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] neutron.timeout = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.755146 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] vmware.api_retry_count = 10 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.755338 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] neutron.valid_interfaces = ['internal', 'public'] {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.755512 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] vmware.ca_file = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.755780 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] neutron.version = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.756359 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cache_prefix = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.756514 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] notifications.bdms_in_notifications = False {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.756514 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] notifications.default_level = INFO {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.756768 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cluster_name = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.756910 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notification_format = unversioned {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.757047 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] vmware.connection_pool_size = 10 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.757189 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notify_on_state_change = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.757189 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] notifications.versioned_notifications_topics = ['versioned_notifications'] {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.757389 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] vmware.console_delay_seconds = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.757449 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] pci.alias = [] {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.757555 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] vmware.datastore_regex = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.757671 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] pci.passthrough_whitelist = [] {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.757765 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_ip = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.757860 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_section = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.757860 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_type = password {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.758022 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_password = **** {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.758102 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] placement.cafile = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.758175 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_port = 443 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.758284 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] placement.certfile = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.758284 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] placement.collect_timing = False {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.758425 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_username = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.758495 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retries = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.758573 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] vmware.insecure = False {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.758645 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retry_delay = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.758729 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] vmware.integration_bridge = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.758810 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] placement.endpoint_override = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.758881 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] vmware.maximum_objects = 100 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.758958 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] placement.insecure = False {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.758958 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] placement.keyfile = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.759108 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_default_policy = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.759178 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] placement.max_version = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.759303 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_enabled = False {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.759381 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] placement.min_version = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.759381 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] placement.region_name = RegionOne {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.762147 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_wsdl_location = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.762227 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_name = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.762298 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_log_dir = /opt/vmware/vspc {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.762364 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_type = placement {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.762364 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] placement.split_loggers = False {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.762475 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_proxy_uri = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.762546 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retries = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.762616 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_service_uri = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.762767 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retry_delay = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.762844 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] vmware.task_poll_interval = 0.5 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.762913 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] placement.timeout = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.762990 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] vmware.use_linked_clone = True {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.763088 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] placement.valid_interfaces = ['internal', 'public'] {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.763088 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] placement.version = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.763215 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_keymap = en-us {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.763284 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] powervm.disk_driver = localdisk {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.763366 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port = 5900 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.763438 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] powervm.proc_units_factor = 0.1 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.763507 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port_total = 10000 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.763576 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] powervm.volume_group_name = {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.763576 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] quota.cores = 20 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.763700 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] vnc.auth_schemes = ['none'] {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.763776 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] quota.count_usage_from_placement = False {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.763848 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] vnc.enabled = True {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.763916 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] quota.driver = nova.quota.DbQuotaDriver {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.763916 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_content_bytes = 10240 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.763916 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_path_length = 255 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.764095 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_base_url = http://127.0.0.1:6080/vnc_auto.html {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.764172 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_files = 5 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.764243 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_host = 0.0.0.0 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.764312 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] quota.instances = 10 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.764312 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] quota.key_pairs = 100 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.764439 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_port = 6080 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.764507 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] quota.metadata_items = 128 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.764507 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] quota.ram = 51200 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.764642 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_listen = 127.0.0.1 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.764711 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] quota.recheck_quota = True {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.764711 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_group_members = 10 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.764833 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_proxyclient_address = 127.0.0.1 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.764901 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_groups = 10 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.764978 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_ca_certs = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.765048 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] rdp.enabled = False {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.765118 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_cert = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.765187 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] rdp.html5_proxy_base_url = http://127.0.0.1:6083/ {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.765255 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_key = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.765331 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.discover_hosts_in_cells_interval = -1 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.765491 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_compute_service_check_for_ffu = False {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.765569 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.enable_isolated_aggregate_filtering = False {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.765569 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.image_metadata_prefilter = False {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.765569 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.limit_tenants_to_placement_aggregate = False {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.765569 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_attempts = 3 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.765907 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_fallback_pcpu_query = False {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.765971 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_placement_results = 1000 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.766139 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_group_policy_check_upcall = False {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.766290 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.placement_aggregate_required_for_tenants = False {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.766496 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_libvirt_livesnapshot = False {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.766623 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_availability_zone = True {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.766623 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_image_type_support = False {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.766766 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_rootwrap = False {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.766843 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_routed_network_aggregates = False {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.766916 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_numa_live_migration = False {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.766985 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.workers = 2 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.767109 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_qemu_monitor_announce_self = False {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.767185 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_namespace = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.767254 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.767323 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_separator = . {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.767392 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.handle_virt_lifecycle_events = True {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.767461 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.767538 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.libvirt_disable_apic = False {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.767609 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.build_failure_weight_multiplier = 1000000.0 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.767678 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.never_download_image_if_on_rbd = False {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.767748 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cpu_weight_multiplier = 1.0 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.767748 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.767861 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.reserve_disk_resource_for_image_cache = False {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.767980 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.disk_weight_multiplier = 1.0 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.768059 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_cpu_compare_on_dest = False {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.768129 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.enabled_filters = ['AvailabilityZoneFilter', 'ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter', 'SameHostFilter', 'DifferentHostFilter'] {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.768200 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.unified_limits_count_pcpu_as_vcpu = False {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.768268 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.host_subset_size = 1 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.768268 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.image_properties_default_architecture = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.768404 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.wait_for_vif_plugged_event_during_hard_reboot = [] {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.768473 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.io_ops_weight_multiplier = -1.0 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.768540 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.api_paste_config = /etc/nova/api-paste.ini {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.768608 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_hosts = [] {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.768677 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.client_socket_timeout = 900 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.768753 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_images = [] {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.768813 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.default_pool_size = 1000 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.768870 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_instances_per_host = 50 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.768928 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.keep_alive = True {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.769839 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_io_ops_per_host = 8 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.770461 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.max_header_line = 16384 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.770540 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.pci_weight_multiplier = 1.0 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.770540 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.ram_weight_multiplier = 1.0 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.770646 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.secure_proxy_ssl_header = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.770702 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.770759 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_ca_file = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.770823 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.shuffle_best_same_weighed_hosts = False {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.770887 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_cert_file = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.770945 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_affinity_weight_multiplier = 1.0 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.771002 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_key_file = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.771091 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.771149 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.tcp_keepidle = 600 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.771219 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.track_instance_changes = False {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.771279 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.771338 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.771394 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] zvm.ca_file = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.771450 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] metrics.required = True {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.771516 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] zvm.cloud_connector_url = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.771582 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_multiplier = 1.0 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.771582 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_of_unavailable = -10000.0 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.771684 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] zvm.image_tmp_path = /opt/stack/data/nova/images {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.771740 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_setting = [] {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.771796 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] zvm.reachable_timeout = 300 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.771796 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.771906 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.base_url = ws://127.0.0.1:6083/ {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.771967 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.772026 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.enabled = False {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.772082 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.772139 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.port_range = 10000:20000 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.772203 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_process_name = {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.772262 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.proxyclient_address = 127.0.0.1 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.772318 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.772375 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_host = 0.0.0.0 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.772432 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.772495 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_port = 6083 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.772554 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_new_defaults = False {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.772611 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_section = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.772667 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_scope = False {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.772722 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_type = password {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.772778 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_default_rule = default {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.772840 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] service_user.cafile = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.772899 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.772956 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] service_user.certfile = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.773012 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_file = policy.yaml {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.773067 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] service_user.collect_timing = False {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.773123 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.773186 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] service_user.insecure = False {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.773244 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.773300 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] service_user.keyfile = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.773356 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.773412 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] service_user.send_service_user_token = True {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.773467 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.773531 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] service_user.split_loggers = False {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.773596 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.773653 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] service_user.timeout = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.773709 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_section = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.774014 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] spice.agent_enabled = True {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.774086 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_type = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.774145 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] spice.enabled = False {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.774211 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.cafile = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.774211 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.certfile = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.774501 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_base_url = http://127.0.0.1:6082/spice_auto.html {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.774560 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.collect_timing = False {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.774625 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_host = 0.0.0.0 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.774684 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retries = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.774740 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_port = 6082 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.774796 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retry_delay = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.774851 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_listen = 127.0.0.1 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.774907 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_id = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.774970 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_proxyclient_address = 127.0.0.1 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.775060 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_override = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.775134 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.baseapi = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.775194 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.insecure = False {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.775250 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.cert = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.775306 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.keyfile = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.775361 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.compute = auto {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.775424 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.max_version = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.775482 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.conductor = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.775538 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.min_version = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.775594 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.scheduler = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.775649 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.region_name = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.775705 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_section = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.775769 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_name = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.775826 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_type = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.775995 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_type = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.776065 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.cafile = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.776237 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.split_loggers = False {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.776307 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.certfile = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.776614 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.collect_timing = False {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.776693 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retries = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.776867 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.insecure = False {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.776947 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retry_delay = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.777163 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.timeout = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.777238 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.keyfile = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.777487 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.valid_interfaces = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.777577 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.split_loggers = False {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.777799 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.timeout = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.777875 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.version = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.778139 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] vmware.api_retry_count = 10 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.778224 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.778459 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] vmware.ca_file = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.778537 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] profiler.connection_string = messaging:// {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.778776 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cache_prefix = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.778854 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] profiler.enabled = False {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.779119 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cluster_name = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.779218 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_doc_type = notification {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.779446 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] vmware.connection_pool_size = 10 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.779532 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_size = 10000 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.779762 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] vmware.console_delay_seconds = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.779840 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_time = 2m {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.780079 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] vmware.datastore_regex = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.780166 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] profiler.filter_error_trace = False {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.780395 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_ip = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.780473 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] profiler.hmac_keys = SECRET_KEY {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.780718 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_password = **** {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.780795 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] profiler.sentinel_service_name = mymaster {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.781044 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_port = 443 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.781123 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] profiler.socket_timeout = 0.1 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.781422 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] profiler.trace_sqlalchemy = False {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.781592 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_username = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.781779 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.782147 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.782485 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.782769 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] vmware.insecure = False {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.782920 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.783126 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] vmware.integration_bridge = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.783271 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.783580 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] vmware.maximum_objects = 100 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.783747 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.783911 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_default_policy = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.784052 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_in_pthread = True {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.784241 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_enabled = False {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.784388 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.785332 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 600 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.785932 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_wsdl_location = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.786174 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.786409 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_log_dir = /opt/vmware/vspc {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.786648 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.786907 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_proxy_uri = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.787165 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.787308 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_service_uri = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.787453 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.787586 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] vmware.task_poll_interval = 0.5 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.787718 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.787851 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] vmware.use_linked_clone = True {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.787991 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.788127 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_keymap = en-us {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.788258 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.788390 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port = 5900 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.788522 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.788654 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port_total = 10000 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.788794 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_delivery_limit = 0 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.788928 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] vnc.auth_schemes = ['none'] {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.789072 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.789206 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] vnc.enabled = True {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.789338 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_bytes = 0 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.789338 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_length = 0 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.789598 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_base_url = http://127.0.0.1:6080/vnc_auto.html {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.789731 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.789862 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_host = 0.0.0.0 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.789994 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.790126 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_port = 6080 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.790267 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.790400 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_listen = 127.0.0.1 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.790532 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.790664 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_proxyclient_address = 127.0.0.1 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.790805 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl = False {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.790938 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_ca_certs = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.791187 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.791327 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_cert = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.791459 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.791591 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_key = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.791722 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.791862 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_compute_service_check_for_ffu = False {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.791996 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.792204 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_fallback_pcpu_query = False {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.792346 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_version = {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.792527 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_group_policy_check_upcall = False {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.792680 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.driver = ['messagingv2'] {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.792850 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_libvirt_livesnapshot = False {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.792992 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.retry = -1 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.793382 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_rootwrap = False {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.793535 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.793725 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_numa_live_migration = False {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.793866 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.transport_url = **** {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.794115 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_qemu_monitor_announce_self = False {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.794259 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.794471 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.794625 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler_interval = 1 {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.794810 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.log_dir = None {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.794949 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.handle_virt_lifecycle_events = True {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.795185 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=138680) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 16 21:07:10.795341 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.libvirt_disable_apic = False {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.795558 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.never_download_image_if_on_rbd = False {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.795882 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.reserve_disk_resource_for_image_cache = False {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.796209 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_cpu_compare_on_dest = False {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.796532 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.unified_limits_count_pcpu_as_vcpu = False {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.796859 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.wait_for_vif_plugged_event_during_hard_reboot = [] {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.797215 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.api_paste_config = /etc/nova/api-paste.ini {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.797547 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.client_socket_timeout = 900 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.797878 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.default_pool_size = 1000 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.798210 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.keep_alive = True {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.798542 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.max_header_line = 16384 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.798866 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.secure_proxy_ssl_header = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.799216 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_ca_file = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.799542 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_cert_file = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.799863 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_key_file = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.800997 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.tcp_keepidle = 600 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.801645 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.801806 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] zvm.ca_file = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.802001 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] zvm.cloud_connector_url = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.802443 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] zvm.image_tmp_path = /opt/stack/data/nova/images {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.802772 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] zvm.reachable_timeout = 300 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.803131 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.803470 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.803796 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.804133 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_process_name = {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.804469 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.804793 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.805126 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_new_defaults = False {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.805452 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_scope = False {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.805795 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_default_rule = default {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.806150 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.806497 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_file = policy.yaml {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.806833 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.807260 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.807512 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.807830 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.808156 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.808487 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_section = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.808809 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_type = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.809126 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.cafile = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.809440 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.certfile = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.809765 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.collect_timing = False {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.810076 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retries = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.810421 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retry_delay = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.810735 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_id = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.811074 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_override = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.811398 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.insecure = False {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.811711 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.keyfile = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.812024 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.max_version = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.812435 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.min_version = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.812636 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.region_name = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.812999 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_name = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.813329 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_type = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.813657 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.split_loggers = False {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.813992 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retries = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.814311 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retry_delay = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.814625 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.timeout = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.814938 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.valid_interfaces = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.815280 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.version = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.815625 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.816043 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] profiler.connection_string = messaging:// {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.816691 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] profiler.enabled = False {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.816996 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_doc_type = notification {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.817149 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_size = 10000 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.817400 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_time = 2m {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.817707 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] profiler.filter_error_trace = False {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.818098 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] profiler.hmac_keys = SECRET_KEY {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.818503 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] profiler.sentinel_service_name = mymaster {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.819111 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] profiler.socket_timeout = 0.1 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.819205 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] profiler.trace_sqlalchemy = False {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.820113 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.820751 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.821095 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.821423 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.821753 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.822081 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.822408 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_in_pthread = True {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.822753 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.823118 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 600 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.823644 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.824868 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.824868 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.824868 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.827858 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.827858 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.827858 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.827858 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.827858 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_delivery_limit = 0 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.827858 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.827858 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_bytes = 0 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.827858 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_length = 0 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.829048 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.829048 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.829048 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.829048 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.829048 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl = False {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.829652 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.829652 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.829911 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.830081 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.830421 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_version = {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.830782 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.driver = ['messagingv2'] {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.831131 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.retry = -1 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.831532 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.831968 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.transport_url = **** {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.832660 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.832660 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler_interval = 1 {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.832922 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.log_dir = None {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 16 21:07:10.833064 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=138681) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 16 21:07:10.866423 ncdv-master-1 devstack@n-api.service[138680]: INFO dbcounter [None req-8c8ac9b0-f73d-4b52-ae46-7cc47b15c28a None None] Registered counter for database nova_api Jul 16 21:07:10.889593 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_db.sqlalchemy.engines [None req-8c8ac9b0-f73d-4b52-ae46-7cc47b15c28a None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=138680) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 16 21:07:10.896357 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writer thread running {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 16 21:07:10.907355 ncdv-master-1 devstack@n-api.service[138681]: INFO dbcounter [None req-aaa1e639-273f-48e3-9fb2-ee67fd25324a None None] Registered counter for database nova_api Jul 16 21:07:10.925327 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_db.sqlalchemy.engines [None req-aaa1e639-273f-48e3-9fb2-ee67fd25324a None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=138681) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 16 21:07:10.933550 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writer thread running {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 16 21:07:10.956497 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.context [None req-8c8ac9b0-f73d-4b52-ae46-7cc47b15c28a None None] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),327e8df1-39cc-4b36-9ea6-6a9b88ed54fc(cell1) {{(pid=138680) load_cells /opt/stack/nova/nova/context.py:464}} Jul 16 21:07:10.962032 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8c8ac9b0-f73d-4b52-ae46-7cc47b15c28a None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:07:10.963702 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8c8ac9b0-f73d-4b52-ae46-7cc47b15c28a None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:07:10.964538 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8c8ac9b0-f73d-4b52-ae46-7cc47b15c28a None None] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:07:10.966616 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8c8ac9b0-f73d-4b52-ae46-7cc47b15c28a None None] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:07:10.968735 ncdv-master-1 devstack@n-api.service[138680]: INFO dbcounter [None req-8c8ac9b0-f73d-4b52-ae46-7cc47b15c28a None None] Registered counter for database nova_cell0 Jul 16 21:07:10.973518 ncdv-master-1 devstack@n-api.service[138680]: INFO dbcounter [None req-8c8ac9b0-f73d-4b52-ae46-7cc47b15c28a None None] Registered counter for database nova_cell1 Jul 16 21:07:10.980956 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_db.sqlalchemy.engines [None req-8c8ac9b0-f73d-4b52-ae46-7cc47b15c28a None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=138680) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 16 21:07:10.984268 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_db.sqlalchemy.engines [None req-8c8ac9b0-f73d-4b52-ae46-7cc47b15c28a None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=138680) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 16 21:07:10.991865 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writer thread running {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 16 21:07:10.993528 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writer thread running {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 16 21:07:10.996380 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.context [None req-aaa1e639-273f-48e3-9fb2-ee67fd25324a None None] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),327e8df1-39cc-4b36-9ea6-6a9b88ed54fc(cell1) {{(pid=138681) load_cells /opt/stack/nova/nova/context.py:464}} Jul 16 21:07:11.001780 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-aaa1e639-273f-48e3-9fb2-ee67fd25324a None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:07:11.003951 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-aaa1e639-273f-48e3-9fb2-ee67fd25324a None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:07:11.005457 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-aaa1e639-273f-48e3-9fb2-ee67fd25324a None None] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:07:11.007514 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-aaa1e639-273f-48e3-9fb2-ee67fd25324a None None] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:07:11.009566 ncdv-master-1 devstack@n-api.service[138681]: INFO dbcounter [None req-aaa1e639-273f-48e3-9fb2-ee67fd25324a None None] Registered counter for database nova_cell0 Jul 16 21:07:11.013764 ncdv-master-1 devstack@n-api.service[138681]: INFO dbcounter [None req-aaa1e639-273f-48e3-9fb2-ee67fd25324a None None] Registered counter for database nova_cell1 Jul 16 21:07:11.022305 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_db.sqlalchemy.engines [None req-aaa1e639-273f-48e3-9fb2-ee67fd25324a None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=138681) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 16 21:07:11.024832 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_db.sqlalchemy.engines [None req-aaa1e639-273f-48e3-9fb2-ee67fd25324a None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=138681) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 16 21:07:11.032258 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writer thread running {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 16 21:07:11.033596 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writer thread running {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 16 21:07:11.147033 ncdv-master-1 devstack@n-api.service[138680]: INFO dbcounter [None req-8c8ac9b0-f73d-4b52-ae46-7cc47b15c28a None None] Registered counter for database nova_cell0 Jul 16 21:07:11.155766 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_db.sqlalchemy.engines [None req-8c8ac9b0-f73d-4b52-ae46-7cc47b15c28a None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=138680) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 16 21:07:11.160614 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writer thread running {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 16 21:07:11.194676 ncdv-master-1 devstack@n-api.service[138681]: INFO dbcounter [None req-aaa1e639-273f-48e3-9fb2-ee67fd25324a None None] Registered counter for database nova_cell0 Jul 16 21:07:11.201576 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_db.sqlalchemy.engines [None req-aaa1e639-273f-48e3-9fb2-ee67fd25324a None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=138681) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 16 21:07:11.206429 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writer thread running {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 16 21:07:11.489134 ncdv-master-1 devstack@n-api.service[138680]: WARNING keystonemiddleware.auth_token [None req-8c8ac9b0-f73d-4b52-ae46-7cc47b15c28a None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Jul 16 21:07:11.496723 ncdv-master-1 devstack@n-api.service[138680]: WARNING keystonemiddleware.auth_token [None req-8c8ac9b0-f73d-4b52-ae46-7cc47b15c28a None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Jul 16 21:07:11.535954 ncdv-master-1 devstack@n-api.service[138681]: WARNING keystonemiddleware.auth_token [None req-aaa1e639-273f-48e3-9fb2-ee67fd25324a None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Jul 16 21:07:11.543795 ncdv-master-1 devstack@n-api.service[138681]: WARNING keystonemiddleware.auth_token [None req-aaa1e639-273f-48e3-9fb2-ee67fd25324a None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Jul 16 21:07:11.855605 ncdv-master-1 devstack@n-api.service[138680]: WARNING keystonemiddleware.auth_token [None req-8c8ac9b0-f73d-4b52-ae46-7cc47b15c28a None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Jul 16 21:07:11.862335 ncdv-master-1 devstack@n-api.service[138680]: WARNING keystonemiddleware.auth_token [None req-8c8ac9b0-f73d-4b52-ae46-7cc47b15c28a None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Jul 16 21:07:11.869045 ncdv-master-1 devstack@n-api.service[138680]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55ba469e09c0 pid: 138680 (default app) Jul 16 21:07:11.900710 ncdv-master-1 devstack@n-api.service[138681]: WARNING keystonemiddleware.auth_token [None req-aaa1e639-273f-48e3-9fb2-ee67fd25324a None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Jul 16 21:07:11.906842 ncdv-master-1 devstack@n-api.service[138681]: WARNING keystonemiddleware.auth_token [None req-aaa1e639-273f-48e3-9fb2-ee67fd25324a None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Jul 16 21:07:11.912884 ncdv-master-1 devstack@n-api.service[138681]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55ba469e09c0 pid: 138681 (default app) Jul 16 21:08:00.359825 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_api:SELECT=3 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:08:00.361931 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=3 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:08:00.363608 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell0:SELECT=3 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:08:00.364970 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell0:SELECT=3 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:08:00.482927 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_api:SELECT=3 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:08:00.483753 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell0:SELECT=3 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:08:00.484986 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=3 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:08:00.485737 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell0:SELECT=3 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:08:01.994353 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-691652f1-ff99-4ecc-92d5-958b506d213e tempest-AZAdminV2TestJSON-1980545772 tempest-AZAdminV2TestJSON-1980545772-project-admin] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:08:01.996327 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-691652f1-ff99-4ecc-92d5-958b506d213e tempest-AZAdminV2TestJSON-1980545772 tempest-AZAdminV2TestJSON-1980545772-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:08:01.996759 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-691652f1-ff99-4ecc-92d5-958b506d213e tempest-AZAdminV2TestJSON-1980545772 tempest-AZAdminV2TestJSON-1980545772-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:08:01.997572 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-691652f1-ff99-4ecc-92d5-958b506d213e tempest-AZAdminV2TestJSON-1980545772 tempest-AZAdminV2TestJSON-1980545772-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:08:01.998017 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-691652f1-ff99-4ecc-92d5-958b506d213e tempest-AZAdminV2TestJSON-1980545772 tempest-AZAdminV2TestJSON-1980545772-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:08:02.057408 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-691652f1-ff99-4ecc-92d5-958b506d213e tempest-AZAdminV2TestJSON-1980545772 tempest-AZAdminV2TestJSON-1980545772-project-admin] 10.222.0.16 "GET /compute/v2.1/os-availability-zone" status: 200 len: 97 microversion: 2.1 time: 1.706631 Jul 16 21:08:02.067432 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1/1] 10.222.0.16 () {58 vars in 1163 bytes} [Sat Jul 16 21:08:00 2022] GET /compute/v2.1/os-availability-zone => generated 97 bytes in 1718 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Jul 16 21:08:02.072949 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-e140ddd4-f2ca-4829-acbe-ed5184d9df98 tempest-AZAdminV2TestJSON-1980545772 tempest-AZAdminV2TestJSON-1980545772-project-admin] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:08:02.074649 ncdv-master-1 devstack@n-api.service[138680]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-availability-zone:detail": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:08:02.074649 ncdv-master-1 devstack@n-api.service[138680]: warnings.warn(msg) Jul 16 21:08:02.076109 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e140ddd4-f2ca-4829-acbe-ed5184d9df98 tempest-AZAdminV2TestJSON-1980545772 tempest-AZAdminV2TestJSON-1980545772-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:08:02.076796 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e140ddd4-f2ca-4829-acbe-ed5184d9df98 tempest-AZAdminV2TestJSON-1980545772 tempest-AZAdminV2TestJSON-1980545772-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:08:02.077980 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e140ddd4-f2ca-4829-acbe-ed5184d9df98 tempest-AZAdminV2TestJSON-1980545772 tempest-AZAdminV2TestJSON-1980545772-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:08:02.078727 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e140ddd4-f2ca-4829-acbe-ed5184d9df98 tempest-AZAdminV2TestJSON-1980545772 tempest-AZAdminV2TestJSON-1980545772-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:08:02.104965 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-e140ddd4-f2ca-4829-acbe-ed5184d9df98 tempest-AZAdminV2TestJSON-1980545772 tempest-AZAdminV2TestJSON-1980545772-project-admin] 10.222.0.16 "GET /compute/v2.1/os-availability-zone/detail" status: 200 len: 613 microversion: 2.1 time: 0.035661 Jul 16 21:08:02.113598 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2/2] 10.222.0.16 () {58 vars in 1184 bytes} [Sat Jul 16 21:08:02 2022] GET /compute/v2.1/os-availability-zone/detail => generated 613 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:08:02.216520 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-d11e266d-a719-4229-bff5-4626fcf77b98 tempest-FlavorsAccessTestJSON-378486216 tempest-FlavorsAccessTestJSON-378486216-project-admin] Action: 'create', calling method: >, body: {"flavor": {"ram": 512, "vcpus": 1, "disk": 10, "name": "tempest-FlavorsAccessTestJSON-flavor-440207227", "id": 2138490358, "os-flavor-access:is_public": "False"}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:08:02.236173 ncdv-master-1 devstack@n-api.service[138681]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-flavor-manage:create": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:08:02.236173 ncdv-master-1 devstack@n-api.service[138681]: warnings.warn(msg) Jul 16 21:08:02.348921 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-d11e266d-a719-4229-bff5-4626fcf77b98 tempest-FlavorsAccessTestJSON-378486216 tempest-FlavorsAccessTestJSON-378486216-project-admin] 10.222.0.16 "POST /compute/v2.1/flavors" status: 200 len: 430 microversion: 2.1 time: 1.874945 Jul 16 21:08:02.349707 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1/3] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 21:08:00 2022] POST /compute/v2.1/flavors => generated 430 bytes in 1878 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:08:02.364943 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-561e227f-b85c-4de2-b27a-85579241899c tempest-FlavorsAccessTestJSON-378486216 tempest-FlavorsAccessTestJSON-378486216-project-admin] Action: 'action', calling method: >, body: {"addTenantAccess": {"tenant": "ba432093c99945c1af32f04d896809b6"}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:08:02.366610 ncdv-master-1 devstack@n-api.service[138680]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-flavor-access:add_tenant_access": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:08:02.366610 ncdv-master-1 devstack@n-api.service[138680]: warnings.warn(msg) Jul 16 21:08:03.559837 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-561e227f-b85c-4de2-b27a-85579241899c tempest-FlavorsAccessTestJSON-378486216 tempest-FlavorsAccessTestJSON-378486216-project-admin] 10.222.0.16 "POST /compute/v2.1/flavors/2138490358/action" status: 200 len: 97 microversion: 2.1 time: 1.202075 Jul 16 21:08:03.571289 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3/4] 10.222.0.16 () {60 vars in 1199 bytes} [Sat Jul 16 21:08:02 2022] POST /compute/v2.1/flavors/2138490358/action => generated 97 bytes in 1214 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Jul 16 21:08:03.745599 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-b1e323d3-5440-4b44-8f76-1a56cda25ff1 tempest-AZAdminNegativeTestJSON-1439419830 tempest-AZAdminNegativeTestJSON-1439419830-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:08:03.746394 ncdv-master-1 devstack@n-api.service[138681]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-availability-zone:detail": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:08:03.746394 ncdv-master-1 devstack@n-api.service[138681]: warnings.warn(msg) Jul 16 21:08:03.747612 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-b1e323d3-5440-4b44-8f76-1a56cda25ff1 tempest-AZAdminNegativeTestJSON-1439419830 tempest-AZAdminNegativeTestJSON-1439419830-project] Policy check for os_compute_api:os-availability-zone:detail failed with credentials {'is_admin': False, 'user_id': '6f34858458b7434fa08eb5be6b0ce1cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '57887ca847c64efba6b864955e6ee14b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:08:03.779148 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-b1e323d3-5440-4b44-8f76-1a56cda25ff1 tempest-AZAdminNegativeTestJSON-1439419830 tempest-AZAdminNegativeTestJSON-1439419830-project] Request method failure captured: Jul 16 21:08:03.779148 ncdv-master-1 devstack@n-api.service[138681]: request: GET /compute/v2.1/os-availability-zone/detail HTTP/1.1 Jul 16 21:08:03.779148 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:08:03.779148 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:08:03.779148 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:08:03.779148 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 21:08:03.779148 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:08:03.779148 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:08:03.779148 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:08:03.779148 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:08:03.779148 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0yivC2gbgyK1RIpcMgC6j6eZA6Scp8HxcWh6G0d4_cT2Cb3SSCTgQAt69C7oJVFTpD-LJmJrDH4J3JDqsNEph2z6IOLAusOrKDGHSyBcQL6sNL880aoTAkk46ABe6shltJr-MY4xsvMsX4KGEEIeKkzG9jpJh6Ktl3TjdFMO4F3p_Yc Jul 16 21:08:03.779148 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:08:03.779148 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:08:03.779148 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:08:03.779148 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:08:03.779148 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:08:03.779148 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:08:03.779148 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: 57887ca847c64efba6b864955e6ee14b Jul 16 21:08:03.779148 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-AZAdminNegativeTestJSON-1439419830 Jul 16 21:08:03.779148 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:08:03.779148 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:08:03.780202 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/57887ca847c64efba6b864955e6ee14b"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/57887ca847c64efba6b864955e6ee14b"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_57887ca847c64efba6b864955e6ee14b"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/57887ca847c64efba6b864955e6ee14b"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:08:03.780202 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-AZAdminNegativeTestJSON-1439419830 Jul 16 21:08:03.780202 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: 57887ca847c64efba6b864955e6ee14b Jul 16 21:08:03.780202 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-AZAdminNegativeTestJSON-1439419830 Jul 16 21:08:03.780202 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-AZAdminNegativeTestJSON-1439419830-project Jul 16 21:08:03.780202 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:08:03.780202 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:08:03.780202 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: 6f34858458b7434fa08eb5be6b0ce1cf Jul 16 21:08:03.780202 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-AZAdminNegativeTestJSON-1439419830-project Jul 16 21:08:03.780202 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:08:03.780202 ncdv-master-1 devstack@n-api.service[138681]: exception: Policy doesn't allow os_compute_api:os-availability-zone:detail to be performed. Jul 16 21:08:03.780202 ncdv-master-1 devstack@n-api.service[138681]: action_args: {} Jul 16 21:08:03.780960 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:08:03.780960 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:08:03.780960 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:08:03.780960 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:08:03.780960 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:08:03.780960 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:08:03.780960 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:08:03.780960 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:08:03.780960 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/availability_zone.py", line 117, in detail Jul 16 21:08:03.780960 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi context.can(az_policies.POLICY_ROOT % 'detail', target={}) Jul 16 21:08:03.780960 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/context.py", line 240, in can Jul 16 21:08:03.780960 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return policy.authorize(self, action, target) Jul 16 21:08:03.780960 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 199, in authorize Jul 16 21:08:03.780960 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi LOG.debug('Policy check for %(action)s failed with credentials ' Jul 16 21:08:03.780960 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 16 21:08:03.780960 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi self.force_reraise() Jul 16 21:08:03.780960 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 16 21:08:03.780960 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise self.value Jul 16 21:08:03.780960 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 186, in authorize Jul 16 21:08:03.781998 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi result = _ENFORCER.authorize(action, target, context, Jul 16 21:08:03.781998 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1184, in authorize Jul 16 21:08:03.781998 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return self.enforce( Jul 16 21:08:03.781998 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1088, in enforce Jul 16 21:08:03.781998 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc(*args, **kwargs) Jul 16 21:08:03.781998 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.PolicyNotAuthorized: Policy doesn't allow os_compute_api:os-availability-zone:detail to be performed. Jul 16 21:08:03.781998 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:08:03.781998 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:08:03.781998 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:08:03.781998 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:08:03.781998 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:08:03.781998 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:08:03.781998 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 354, in __exit__ Jul 16 21:08:03.781998 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(webob.exc.HTTPForbidden( Jul 16 21:08:03.781998 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Policy doesn't allow os_compute_api:os-availability-zone:detail to be performed. Jul 16 21:08:03.781998 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:08:03.781998 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-b1e323d3-5440-4b44-8f76-1a56cda25ff1 tempest-AZAdminNegativeTestJSON-1439419830 tempest-AZAdminNegativeTestJSON-1439419830-project] Returning 403 to user: Policy doesn't allow os_compute_api:os-availability-zone:detail to be performed. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:08:03.783459 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-b1e323d3-5440-4b44-8f76-1a56cda25ff1 tempest-AZAdminNegativeTestJSON-1439419830 tempest-AZAdminNegativeTestJSON-1439419830-project] 10.222.0.16 "GET /compute/v2.1/os-availability-zone/detail" status: 403 len: 123 microversion: 2.1 time: 0.669712 Jul 16 21:08:03.786560 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2/5] 10.222.0.16 () {58 vars in 1184 bytes} [Sat Jul 16 21:08:03 2022] GET /compute/v2.1/os-availability-zone/detail => generated 123 bytes in 676 msecs (HTTP/1.1 403) 9 headers in 379 bytes (2 switches on core 0) Jul 16 21:08:03.987543 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-6dfb0be0-c552-4e11-bbec-816e70f2fadd tempest-AggregatesAdminNegativeTestJSON-578260511 tempest-AggregatesAdminNegativeTestJSON-578260511-project-admin] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:08:03.989189 ncdv-master-1 devstack@n-api.service[138680]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-services:list": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:08:03.989189 ncdv-master-1 devstack@n-api.service[138680]: warnings.warn(msg) Jul 16 21:08:03.990471 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6dfb0be0-c552-4e11-bbec-816e70f2fadd tempest-AggregatesAdminNegativeTestJSON-578260511 tempest-AggregatesAdminNegativeTestJSON-578260511-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:08:03.990947 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6dfb0be0-c552-4e11-bbec-816e70f2fadd tempest-AggregatesAdminNegativeTestJSON-578260511 tempest-AggregatesAdminNegativeTestJSON-578260511-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:08:03.991787 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6dfb0be0-c552-4e11-bbec-816e70f2fadd tempest-AggregatesAdminNegativeTestJSON-578260511 tempest-AggregatesAdminNegativeTestJSON-578260511-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:08:03.992220 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6dfb0be0-c552-4e11-bbec-816e70f2fadd tempest-AggregatesAdminNegativeTestJSON-578260511 tempest-AggregatesAdminNegativeTestJSON-578260511-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:08:04.017072 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-6dfb0be0-c552-4e11-bbec-816e70f2fadd tempest-AggregatesAdminNegativeTestJSON-578260511 tempest-AggregatesAdminNegativeTestJSON-578260511-project-admin] 10.222.0.16 "GET /compute/v2.1/os-services?binary=nova-compute" status: 200 len: 380 microversion: 2.1 time: 0.444571 Jul 16 21:08:04.018292 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 4/6] 10.222.0.16 () {58 vars in 1175 bytes} [Sat Jul 16 21:08:03 2022] GET /compute/v2.1/os-services?binary=nova-compute => generated 380 bytes in 446 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:08:04.123432 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-1875fd94-fed4-4d6b-b5f3-f19164c94364 tempest-FlavorsAccessTestJSON-1911343943 tempest-FlavorsAccessTestJSON-1911343943-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:08:04.164929 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-1875fd94-fed4-4d6b-b5f3-f19164c94364 tempest-FlavorsAccessTestJSON-1911343943 tempest-FlavorsAccessTestJSON-1911343943-project] 10.222.0.16 "GET /compute/v2.1/flavors/detail" status: 200 len: 5411 microversion: 2.1 time: 0.376954 Jul 16 21:08:04.165728 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3/7] 10.222.0.16 () {58 vars in 1145 bytes} [Sat Jul 16 21:08:03 2022] GET /compute/v2.1/flavors/detail => generated 5411 bytes in 378 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:08:04.220664 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-5242a34c-4aee-412c-8416-495f12a4ca5c tempest-HypervisorAdminUnderV252Test-686734338 tempest-HypervisorAdminUnderV252Test-686734338-project-admin] Calling method '.version_select at 0x7fdd9f9d7f70>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:08:04.221481 ncdv-master-1 devstack@n-api.service[138680]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-hypervisors:list": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:08:04.221481 ncdv-master-1 devstack@n-api.service[138680]: warnings.warn(msg) Jul 16 21:08:04.229929 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-5242a34c-4aee-412c-8416-495f12a4ca5c tempest-HypervisorAdminUnderV252Test-686734338 tempest-HypervisorAdminUnderV252Test-686734338-project-admin] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),327e8df1-39cc-4b36-9ea6-6a9b88ed54fc(cell1) {{(pid=138680) load_cells /opt/stack/nova/nova/compute/api.py:306}} Jul 16 21:08:04.230986 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5242a34c-4aee-412c-8416-495f12a4ca5c tempest-HypervisorAdminUnderV252Test-686734338 tempest-HypervisorAdminUnderV252Test-686734338-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:08:04.231552 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5242a34c-4aee-412c-8416-495f12a4ca5c tempest-HypervisorAdminUnderV252Test-686734338 tempest-HypervisorAdminUnderV252Test-686734338-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:08:04.262581 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5242a34c-4aee-412c-8416-495f12a4ca5c tempest-HypervisorAdminUnderV252Test-686734338 tempest-HypervisorAdminUnderV252Test-686734338-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:08:04.263204 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5242a34c-4aee-412c-8416-495f12a4ca5c tempest-HypervisorAdminUnderV252Test-686734338 tempest-HypervisorAdminUnderV252Test-686734338-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:08:04.279344 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5242a34c-4aee-412c-8416-495f12a4ca5c tempest-HypervisorAdminUnderV252Test-686734338 tempest-HypervisorAdminUnderV252Test-686734338-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:08:04.279775 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5242a34c-4aee-412c-8416-495f12a4ca5c tempest-HypervisorAdminUnderV252Test-686734338 tempest-HypervisorAdminUnderV252Test-686734338-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:08:04.286886 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-5242a34c-4aee-412c-8416-495f12a4ca5c tempest-HypervisorAdminUnderV252Test-686734338 tempest-HypervisorAdminUnderV252Test-686734338-project-admin] 10.222.0.16 "GET /compute/v2.1/os-hypervisors" status: 200 len: 191 microversion: 2.1 time: 0.267402 Jul 16 21:08:04.287532 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 5/8] 10.222.0.16 () {58 vars in 1145 bytes} [Sat Jul 16 21:08:04 2022] GET /compute/v2.1/os-hypervisors => generated 191 bytes in 269 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:08:04.568004 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-d6d355be-3cc1-40e3-a738-0491621d44a0 tempest-AggregatesAdminTestV241-531976310 tempest-AggregatesAdminTestV241-531976310-project-admin] Calling method '.version_select at 0x7fdd9fac0d30>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:08:04.569353 ncdv-master-1 devstack@n-api.service[138681]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-hypervisors:list-detail": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:08:04.569353 ncdv-master-1 devstack@n-api.service[138681]: warnings.warn(msg) Jul 16 21:08:04.575149 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-cb8e446e-ae36-46c6-b200-eef9e3748e83 tempest-AggregatesAdminTestJSON-1464138310 tempest-AggregatesAdminTestJSON-1464138310-project-admin] Calling method '.version_select at 0x7fdd9f9d7820>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:08:04.575951 ncdv-master-1 devstack@n-api.service[138680]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-hypervisors:list-detail": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:08:04.575951 ncdv-master-1 devstack@n-api.service[138680]: warnings.warn(msg) Jul 16 21:08:04.577186 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cb8e446e-ae36-46c6-b200-eef9e3748e83 tempest-AggregatesAdminTestJSON-1464138310 tempest-AggregatesAdminTestJSON-1464138310-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:08:04.577634 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cb8e446e-ae36-46c6-b200-eef9e3748e83 tempest-AggregatesAdminTestJSON-1464138310 tempest-AggregatesAdminTestJSON-1464138310-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:08:04.588705 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-d6d355be-3cc1-40e3-a738-0491621d44a0 tempest-AggregatesAdminTestV241-531976310 tempest-AggregatesAdminTestV241-531976310-project-admin] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),327e8df1-39cc-4b36-9ea6-6a9b88ed54fc(cell1) {{(pid=138681) load_cells /opt/stack/nova/nova/compute/api.py:306}} Jul 16 21:08:04.590345 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d6d355be-3cc1-40e3-a738-0491621d44a0 tempest-AggregatesAdminTestV241-531976310 tempest-AggregatesAdminTestV241-531976310-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:08:04.591002 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d6d355be-3cc1-40e3-a738-0491621d44a0 tempest-AggregatesAdminTestV241-531976310 tempest-AggregatesAdminTestV241-531976310-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:08:04.592167 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cb8e446e-ae36-46c6-b200-eef9e3748e83 tempest-AggregatesAdminTestJSON-1464138310 tempest-AggregatesAdminTestJSON-1464138310-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:08:04.592794 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cb8e446e-ae36-46c6-b200-eef9e3748e83 tempest-AggregatesAdminTestJSON-1464138310 tempest-AggregatesAdminTestJSON-1464138310-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:08:04.605228 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cb8e446e-ae36-46c6-b200-eef9e3748e83 tempest-AggregatesAdminTestJSON-1464138310 tempest-AggregatesAdminTestJSON-1464138310-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:08:04.605742 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cb8e446e-ae36-46c6-b200-eef9e3748e83 tempest-AggregatesAdminTestJSON-1464138310 tempest-AggregatesAdminTestJSON-1464138310-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:08:04.613877 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-cb8e446e-ae36-46c6-b200-eef9e3748e83 tempest-AggregatesAdminTestJSON-1464138310 tempest-AggregatesAdminTestJSON-1464138310-project-admin] 10.222.0.16 "GET /compute/v2.1/os-hypervisors/detail" status: 200 len: 1503 microversion: 2.1 time: 0.320278 Jul 16 21:08:04.622995 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 6/9] 10.222.0.16 () {58 vars in 1166 bytes} [Sat Jul 16 21:08:04 2022] GET /compute/v2.1/os-hypervisors/detail => generated 1503 bytes in 335 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:08:04.628951 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-c52b294c-af70-4ae6-8f2a-6d241bc66eca tempest-AggregatesAdminNegativeTestJSON-578260511 tempest-AggregatesAdminNegativeTestJSON-578260511-project-admin] Action: 'create', calling method: >, body: {"aggregate": {"name": "tempest-test_aggregate-489481267"}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:08:04.633240 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d6d355be-3cc1-40e3-a738-0491621d44a0 tempest-AggregatesAdminTestV241-531976310 tempest-AggregatesAdminTestV241-531976310-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:08:04.633699 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d6d355be-3cc1-40e3-a738-0491621d44a0 tempest-AggregatesAdminTestV241-531976310 tempest-AggregatesAdminTestV241-531976310-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:08:04.649587 ncdv-master-1 devstack@n-api.service[138680]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-aggregates:create": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:08:04.649587 ncdv-master-1 devstack@n-api.service[138680]: warnings.warn(msg) Jul 16 21:08:04.650190 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d6d355be-3cc1-40e3-a738-0491621d44a0 tempest-AggregatesAdminTestV241-531976310 tempest-AggregatesAdminTestV241-531976310-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:08:04.651001 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d6d355be-3cc1-40e3-a738-0491621d44a0 tempest-AggregatesAdminTestV241-531976310 tempest-AggregatesAdminTestV241-531976310-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:08:04.651902 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.objects.aggregate [None req-c52b294c-af70-4ae6-8f2a-6d241bc66eca tempest-AggregatesAdminNegativeTestJSON-578260511 tempest-AggregatesAdminNegativeTestJSON-578260511-project-admin] Generated uuid bb034d9a-7e57-4815-8284-9fe41ba7dec3 for aggregate {{(pid=138680) create /opt/stack/nova/nova/objects/aggregate.py:298}} Jul 16 21:08:04.662618 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-d6d355be-3cc1-40e3-a738-0491621d44a0 tempest-AggregatesAdminTestV241-531976310 tempest-AggregatesAdminTestV241-531976310-project-admin] 10.222.0.16 "GET /compute/v2.1/os-hypervisors/detail" status: 200 len: 1423 microversion: 2.41 time: 0.493907 Jul 16 21:08:04.666719 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 4/10] 10.222.0.16 () {60 vars in 1207 bytes} [Sat Jul 16 21:08:04 2022] GET /compute/v2.1/os-hypervisors/detail => generated 1423 bytes in 500 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:08:04.675758 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-68137b3d-c5b2-463a-b95a-ca970fd5b1c5 tempest-FlavorsAccessTestJSON-378486216 tempest-FlavorsAccessTestJSON-378486216-project-admin] Action: 'action', calling method: >, body: {"removeTenantAccess": {"tenant": "ba432093c99945c1af32f04d896809b6"}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:08:04.677336 ncdv-master-1 devstack@n-api.service[138681]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-flavor-access:remove_tenant_access": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:08:04.677336 ncdv-master-1 devstack@n-api.service[138681]: warnings.warn(msg) Jul 16 21:08:04.872277 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-c52b294c-af70-4ae6-8f2a-6d241bc66eca tempest-AggregatesAdminNegativeTestJSON-578260511 tempest-AggregatesAdminNegativeTestJSON-578260511-project-admin] 10.222.0.16 "POST /compute/v2.1/os-aggregates" status: 200 len: 197 microversion: 2.1 time: 0.247850 Jul 16 21:08:04.879801 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 7/11] 10.222.0.16 () {60 vars in 1163 bytes} [Sat Jul 16 21:08:04 2022] POST /compute/v2.1/os-aggregates => generated 197 bytes in 256 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:08:04.885076 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-6a4df28b-c539-4f90-ac8f-a969ceb3d01c tempest-HypervisorAdminUnderV252Test-686734338 tempest-HypervisorAdminUnderV252Test-686734338-project-admin] Calling method '.version_select at 0x7fdd9f9cd040>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:08:04.885908 ncdv-master-1 devstack@n-api.service[138680]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-hypervisors:servers": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:08:04.885908 ncdv-master-1 devstack@n-api.service[138680]: warnings.warn(msg) Jul 16 21:08:04.892126 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6a4df28b-c539-4f90-ac8f-a969ceb3d01c tempest-HypervisorAdminUnderV252Test-686734338 tempest-HypervisorAdminUnderV252Test-686734338-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:08:04.892126 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6a4df28b-c539-4f90-ac8f-a969ceb3d01c tempest-HypervisorAdminUnderV252Test-686734338 tempest-HypervisorAdminUnderV252Test-686734338-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:08:04.907771 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6a4df28b-c539-4f90-ac8f-a969ceb3d01c tempest-HypervisorAdminUnderV252Test-686734338 tempest-HypervisorAdminUnderV252Test-686734338-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:08:04.908350 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6a4df28b-c539-4f90-ac8f-a969ceb3d01c tempest-HypervisorAdminUnderV252Test-686734338 tempest-HypervisorAdminUnderV252Test-686734338-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:08:04.946617 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6a4df28b-c539-4f90-ac8f-a969ceb3d01c tempest-HypervisorAdminUnderV252Test-686734338 tempest-HypervisorAdminUnderV252Test-686734338-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:08:04.947178 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6a4df28b-c539-4f90-ac8f-a969ceb3d01c tempest-HypervisorAdminUnderV252Test-686734338 tempest-HypervisorAdminUnderV252Test-686734338-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:08:04.953258 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-6a4df28b-c539-4f90-ac8f-a969ceb3d01c tempest-HypervisorAdminUnderV252Test-686734338 tempest-HypervisorAdminUnderV252Test-686734338-project-admin] 10.222.0.16 "GET /compute/v2.1/os-hypervisors/nch1-master-1/servers" status: 200 len: 104 microversion: 2.1 time: 0.072088 Jul 16 21:08:04.954166 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 8/12] 10.222.0.16 () {58 vars in 1211 bytes} [Sat Jul 16 21:08:04 2022] GET /compute/v2.1/os-hypervisors/nch1-master-1/servers => generated 104 bytes in 73 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:08:04.959952 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-ec11f92f-228f-4002-851b-e89d83c367a8 tempest-AggregatesAdminNegativeTestJSON-578260511 tempest-AggregatesAdminNegativeTestJSON-578260511-project-admin] Action: 'action', calling method: >, body: {"add_host": {"host": "nch1-master-1"}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:08:04.960551 ncdv-master-1 devstack@n-api.service[138680]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-aggregates:add_host": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:08:04.960551 ncdv-master-1 devstack@n-api.service[138680]: warnings.warn(msg) Jul 16 21:08:04.971335 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ec11f92f-228f-4002-851b-e89d83c367a8 tempest-AggregatesAdminNegativeTestJSON-578260511 tempest-AggregatesAdminNegativeTestJSON-578260511-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:08:04.978186 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ec11f92f-228f-4002-851b-e89d83c367a8 tempest-AggregatesAdminNegativeTestJSON-578260511 tempest-AggregatesAdminNegativeTestJSON-578260511-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:08:05.309885 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-68137b3d-c5b2-463a-b95a-ca970fd5b1c5 tempest-FlavorsAccessTestJSON-378486216 tempest-FlavorsAccessTestJSON-378486216-project-admin] 10.222.0.16 "POST /compute/v2.1/flavors/2138490358/action" status: 200 len: 21 microversion: 2.1 time: 0.641933 Jul 16 21:08:05.310491 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 5/13] 10.222.0.16 () {60 vars in 1199 bytes} [Sat Jul 16 21:08:04 2022] POST /compute/v2.1/flavors/2138490358/action => generated 21 bytes in 643 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Jul 16 21:08:05.315650 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-27e6ba45-c6da-4cb0-ba95-23924c4816ad tempest-HypervisorAdminUnderV252Test-686734338 tempest-HypervisorAdminUnderV252Test-686734338-project-admin] Calling method '.version_select at 0x7fdd9f9c7c10>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:08:05.316142 ncdv-master-1 devstack@n-api.service[138681]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-hypervisors:list": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:08:05.316142 ncdv-master-1 devstack@n-api.service[138681]: warnings.warn(msg) Jul 16 21:08:05.317385 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-27e6ba45-c6da-4cb0-ba95-23924c4816ad tempest-HypervisorAdminUnderV252Test-686734338 tempest-HypervisorAdminUnderV252Test-686734338-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:08:05.317816 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-27e6ba45-c6da-4cb0-ba95-23924c4816ad tempest-HypervisorAdminUnderV252Test-686734338 tempest-HypervisorAdminUnderV252Test-686734338-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:08:05.334380 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-27e6ba45-c6da-4cb0-ba95-23924c4816ad tempest-HypervisorAdminUnderV252Test-686734338 tempest-HypervisorAdminUnderV252Test-686734338-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:08:05.334756 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-27e6ba45-c6da-4cb0-ba95-23924c4816ad tempest-HypervisorAdminUnderV252Test-686734338 tempest-HypervisorAdminUnderV252Test-686734338-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:08:05.344838 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-27e6ba45-c6da-4cb0-ba95-23924c4816ad tempest-HypervisorAdminUnderV252Test-686734338 tempest-HypervisorAdminUnderV252Test-686734338-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:08:05.345288 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-27e6ba45-c6da-4cb0-ba95-23924c4816ad tempest-HypervisorAdminUnderV252Test-686734338 tempest-HypervisorAdminUnderV252Test-686734338-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:08:05.351711 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-27e6ba45-c6da-4cb0-ba95-23924c4816ad tempest-HypervisorAdminUnderV252Test-686734338 tempest-HypervisorAdminUnderV252Test-686734338-project-admin] 10.222.0.16 "GET /compute/v2.1/os-hypervisors" status: 200 len: 191 microversion: 2.1 time: 0.040241 Jul 16 21:08:05.352298 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 6/14] 10.222.0.16 () {58 vars in 1145 bytes} [Sat Jul 16 21:08:05 2022] GET /compute/v2.1/os-hypervisors => generated 191 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:08:05.357097 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-0087c3da-6a98-47e5-8218-cd49d9f8b58a tempest-FlavorsAccessTestJSON-1911343943 tempest-FlavorsAccessTestJSON-1911343943-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:08:05.385163 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-0087c3da-6a98-47e5-8218-cd49d9f8b58a tempest-FlavorsAccessTestJSON-1911343943 tempest-FlavorsAccessTestJSON-1911343943-project] 10.222.0.16 "GET /compute/v2.1/flavors/detail" status: 200 len: 4991 microversion: 2.1 time: 0.031657 Jul 16 21:08:05.386422 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 7/15] 10.222.0.16 () {58 vars in 1145 bytes} [Sat Jul 16 21:08:05 2022] GET /compute/v2.1/flavors/detail => generated 4991 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:08:05.390818 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-cce6330f-78a0-4cc6-967a-c709e2581a49 tempest-HypervisorAdminUnderV252Test-686734338 tempest-HypervisorAdminUnderV252Test-686734338-project-admin] Calling method '.version_select at 0x7fdd9f9c7ee0>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:08:05.391767 ncdv-master-1 devstack@n-api.service[138681]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-hypervisors:search": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:08:05.391767 ncdv-master-1 devstack@n-api.service[138681]: warnings.warn(msg) Jul 16 21:08:05.393544 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cce6330f-78a0-4cc6-967a-c709e2581a49 tempest-HypervisorAdminUnderV252Test-686734338 tempest-HypervisorAdminUnderV252Test-686734338-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:08:05.394468 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cce6330f-78a0-4cc6-967a-c709e2581a49 tempest-HypervisorAdminUnderV252Test-686734338 tempest-HypervisorAdminUnderV252Test-686734338-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:08:05.414352 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cce6330f-78a0-4cc6-967a-c709e2581a49 tempest-HypervisorAdminUnderV252Test-686734338 tempest-HypervisorAdminUnderV252Test-686734338-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:08:05.414811 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cce6330f-78a0-4cc6-967a-c709e2581a49 tempest-HypervisorAdminUnderV252Test-686734338 tempest-HypervisorAdminUnderV252Test-686734338-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:08:05.424201 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-cce6330f-78a0-4cc6-967a-c709e2581a49 tempest-HypervisorAdminUnderV252Test-686734338 tempest-HypervisorAdminUnderV252Test-686734338-project-admin] 10.222.0.16 "GET /compute/v2.1/os-hypervisors/nch1-master-1/search" status: 200 len: 104 microversion: 2.1 time: 0.036965 Jul 16 21:08:05.425136 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 8/16] 10.222.0.16 () {58 vars in 1208 bytes} [Sat Jul 16 21:08:05 2022] GET /compute/v2.1/os-hypervisors/nch1-master-1/search => generated 104 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:08:05.434388 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-f3a2d337-1079-4464-90c6-db38bdc5a15e tempest-FlavorsAccessTestJSON-378486216 tempest-FlavorsAccessTestJSON-378486216-project-admin] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:08:05.435205 ncdv-master-1 devstack@n-api.service[138681]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-flavor-manage:delete": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:08:05.435205 ncdv-master-1 devstack@n-api.service[138681]: warnings.warn(msg) Jul 16 21:08:05.549847 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-f3a2d337-1079-4464-90c6-db38bdc5a15e tempest-FlavorsAccessTestJSON-378486216 tempest-FlavorsAccessTestJSON-378486216-project-admin] 10.222.0.16 "DELETE /compute/v2.1/flavors/2138490358" status: 202 len: 0 microversion: 2.1 time: 0.123484 Jul 16 21:08:05.550463 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 9/17] 10.222.0.16 () {58 vars in 1160 bytes} [Sat Jul 16 21:08:05 2022] DELETE /compute/v2.1/flavors/2138490358 => generated 0 bytes in 125 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 21:08:05.558409 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-e093dc74-e71a-4b71-8486-37be7de9b6c8 tempest-FlavorsAccessTestJSON-378486216 tempest-FlavorsAccessTestJSON-378486216-project-admin] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:08:05.587406 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-e093dc74-e71a-4b71-8486-37be7de9b6c8 tempest-FlavorsAccessTestJSON-378486216 tempest-FlavorsAccessTestJSON-378486216-project-admin] 10.222.0.16 "GET /compute/v2.1/flavors/detail" status: 200 len: 4991 microversion: 2.1 time: 0.032357 Jul 16 21:08:05.588154 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 10/18] 10.222.0.16 () {58 vars in 1145 bytes} [Sat Jul 16 21:08:05 2022] GET /compute/v2.1/flavors/detail => generated 4991 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:08:05.604081 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-1cc15c1a-41e4-404e-b829-23da6a7c9410 tempest-FlavorsAccessTestJSON-378486216 tempest-FlavorsAccessTestJSON-378486216-project-admin] Action: 'create', calling method: >, body: {"flavor": {"ram": 512, "vcpus": 1, "disk": 10, "name": "tempest-FlavorsAccessTestJSON-flavor-1682708547", "id": 448286780, "os-flavor-access:is_public": "False"}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:08:05.605727 ncdv-master-1 devstack@n-api.service[138681]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-flavor-manage:create": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:08:05.605727 ncdv-master-1 devstack@n-api.service[138681]: warnings.warn(msg) Jul 16 21:08:05.670464 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-1cc15c1a-41e4-404e-b829-23da6a7c9410 tempest-FlavorsAccessTestJSON-378486216 tempest-FlavorsAccessTestJSON-378486216-project-admin] 10.222.0.16 "POST /compute/v2.1/flavors" status: 200 len: 428 microversion: 2.1 time: 0.069983 Jul 16 21:08:05.680601 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 11/19] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 21:08:05 2022] POST /compute/v2.1/flavors => generated 428 bytes in 71 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:08:05.701616 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-a06fafbe-8e25-4f8a-83d1-eca4d37d35db tempest-FlavorsAccessTestJSON-378486216 tempest-FlavorsAccessTestJSON-378486216-project-admin] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:08:05.708743 ncdv-master-1 devstack@n-api.service[138681]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-flavor-access": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:08:05.708743 ncdv-master-1 devstack@n-api.service[138681]: warnings.warn(msg) Jul 16 21:08:05.741153 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-a06fafbe-8e25-4f8a-83d1-eca4d37d35db tempest-FlavorsAccessTestJSON-378486216 tempest-FlavorsAccessTestJSON-378486216-project-admin] 10.222.0.16 "GET /compute/v2.1/flavors/448286780/os-flavor-access" status: 200 len: 21 microversion: 2.1 time: 0.053320 Jul 16 21:08:05.742044 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 12/20] 10.222.0.16 () {58 vars in 1205 bytes} [Sat Jul 16 21:08:05 2022] GET /compute/v2.1/flavors/448286780/os-flavor-access => generated 21 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Jul 16 21:08:05.754363 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-d9eb17d0-6baa-42c5-a8a4-7ab2941fed22 tempest-FlavorsAccessTestJSON-378486216 tempest-FlavorsAccessTestJSON-378486216-project-admin] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:08:05.858998 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-d9eb17d0-6baa-42c5-a8a4-7ab2941fed22 tempest-FlavorsAccessTestJSON-378486216 tempest-FlavorsAccessTestJSON-378486216-project-admin] 10.222.0.16 "DELETE /compute/v2.1/flavors/448286780" status: 202 len: 0 microversion: 2.1 time: 0.111361 Jul 16 21:08:05.859867 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 13/21] 10.222.0.16 () {58 vars in 1157 bytes} [Sat Jul 16 21:08:05 2022] DELETE /compute/v2.1/flavors/448286780 => generated 0 bytes in 113 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Jul 16 21:08:05.868123 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-eca17644-2ffd-47e5-b453-8fd1b1fbeed8 tempest-FlavorsAccessTestJSON-378486216 tempest-FlavorsAccessTestJSON-378486216-project-admin] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:08:05.887667 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-eca17644-2ffd-47e5-b453-8fd1b1fbeed8 tempest-FlavorsAccessTestJSON-378486216 tempest-FlavorsAccessTestJSON-378486216-project-admin] 10.222.0.16 "GET /compute/v2.1/flavors/detail" status: 200 len: 4991 microversion: 2.1 time: 0.023076 Jul 16 21:08:05.888253 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 14/22] 10.222.0.16 () {58 vars in 1145 bytes} [Sat Jul 16 21:08:05 2022] GET /compute/v2.1/flavors/detail => generated 4991 bytes in 24 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:08:05.974479 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-ec11f92f-228f-4002-851b-e89d83c367a8 tempest-AggregatesAdminNegativeTestJSON-578260511 tempest-AggregatesAdminNegativeTestJSON-578260511-project-admin] 10.222.0.16 "POST /compute/v2.1/os-aggregates/1/action" status: 200 len: 241 microversion: 2.1 time: 1.019028 Jul 16 21:08:05.975189 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 9/23] 10.222.0.16 () {60 vars in 1190 bytes} [Sat Jul 16 21:08:04 2022] POST /compute/v2.1/os-aggregates/1/action => generated 241 bytes in 1020 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:08:05.989030 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-8ce6c6aa-7586-4bfc-8a05-0915b452c95e tempest-AggregatesAdminNegativeTestJSON-578260511 tempest-AggregatesAdminNegativeTestJSON-578260511-project-admin] Action: 'action', calling method: >, body: {"add_host": {"host": "nch1-master-1"}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:08:05.990682 ncdv-master-1 devstack@n-api.service[138681]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-aggregates:add_host": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:08:05.990682 ncdv-master-1 devstack@n-api.service[138681]: warnings.warn(msg) Jul 16 21:08:06.024515 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8ce6c6aa-7586-4bfc-8a05-0915b452c95e tempest-AggregatesAdminNegativeTestJSON-578260511 tempest-AggregatesAdminNegativeTestJSON-578260511-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:08:06.024979 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8ce6c6aa-7586-4bfc-8a05-0915b452c95e tempest-AggregatesAdminNegativeTestJSON-578260511 tempest-AggregatesAdminNegativeTestJSON-578260511-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:08:06.173912 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-8ce6c6aa-7586-4bfc-8a05-0915b452c95e tempest-AggregatesAdminNegativeTestJSON-578260511 tempest-AggregatesAdminNegativeTestJSON-578260511-project-admin] HTTP exception thrown: Aggregate 1 already has host nch1-master-1. Jul 16 21:08:06.231134 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-8ce6c6aa-7586-4bfc-8a05-0915b452c95e tempest-AggregatesAdminNegativeTestJSON-578260511 tempest-AggregatesAdminNegativeTestJSON-578260511-project-admin] Request method failure captured: Jul 16 21:08:06.231134 ncdv-master-1 devstack@n-api.service[138681]: request: POST /compute/v2.1/os-aggregates/1/action HTTP/1.1 Jul 16 21:08:06.231134 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:08:06.231134 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:08:06.231134 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:08:06.231134 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 39 Jul 16 21:08:06.231134 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:08:06.231134 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:08:06.231134 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:08:06.231134 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:08:06.231134 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0yiz0unej4HpWPJrVBlkA1u9-AWCKDSpHymJUf5DT-hiCYEGZJzMCi_CgXajssbkUJoCwaGruhY3wft9OVrurmz0_pRbvWF03aApzlFrXwt0GhLH6-kKTcDhg1N2k0bLKCTgcUoad_Ma5kdyMmhxUVTCB2mCAuoF3iPjsftSucch2dg Jul 16 21:08:06.231134 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:08:06.231134 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:08:06.231134 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:08:06.231134 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:08:06.231134 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:08:06.231134 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:08:06.231134 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: b6be58dcb13b4cec8f0bfc58f0080173 Jul 16 21:08:06.231134 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-AggregatesAdminNegativeTestJSON-578260511 Jul 16 21:08:06.231134 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,admin,reader Jul 16 21:08:06.231134 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,admin,reader Jul 16 21:08:06.232235 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/b6be58dcb13b4cec8f0bfc58f0080173"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/b6be58dcb13b4cec8f0bfc58f0080173"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_b6be58dcb13b4cec8f0bfc58f0080173"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/b6be58dcb13b4cec8f0bfc58f0080173"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:08:06.232235 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-AggregatesAdminNegativeTestJSON-578260511 Jul 16 21:08:06.232235 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: b6be58dcb13b4cec8f0bfc58f0080173 Jul 16 21:08:06.232235 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-AggregatesAdminNegativeTestJSON-578260511 Jul 16 21:08:06.232235 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-AggregatesAdminNegativeTestJSON-578260511-project-admin Jul 16 21:08:06.232235 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:08:06.232235 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:08:06.232235 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: cadb63bb5d6b4d7597987780f7d9ef21 Jul 16 21:08:06.232235 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-AggregatesAdminNegativeTestJSON-578260511-project-admin Jul 16 21:08:06.232235 ncdv-master-1 devstack@n-api.service[138681]: Jul 16 21:08:06.232235 ncdv-master-1 devstack@n-api.service[138681]: {"add_host": {"host": "nch1-master-1"}} Jul 16 21:08:06.232235 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:08:06.232235 ncdv-master-1 devstack@n-api.service[138681]: exception: Aggregate 1 already has host nch1-master-1. Jul 16 21:08:06.232877 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': '1', 'body': {'add_host': {'host': 'nch1-master-1'}}} Jul 16 21:08:06.232877 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:08:06.232877 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:08:06.232877 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 16 21:08:06.232877 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi self.dialect.do_execute( Jul 16 21:08:06.232877 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 16 21:08:06.232877 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi cursor.execute(statement, parameters) Jul 16 21:08:06.232877 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 16 21:08:06.232877 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi result = self._query(query) Jul 16 21:08:06.232877 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 16 21:08:06.232877 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi conn.query(q) Jul 16 21:08:06.232877 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 16 21:08:06.232877 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 16 21:08:06.232877 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 16 21:08:06.232877 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi result.read() Jul 16 21:08:06.232877 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 16 21:08:06.232877 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi first_packet = self.connection._read_packet() Jul 16 21:08:06.232877 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 16 21:08:06.232877 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi packet.raise_for_error() Jul 16 21:08:06.233869 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 16 21:08:06.233869 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi err.raise_mysql_exception(self._data) Jul 16 21:08:06.233869 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 16 21:08:06.233869 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise errorclass(errno, errval) Jul 16 21:08:06.233869 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi pymysql.err.IntegrityError: (1062, "Duplicate entry 'nch1-master-1-1' for key 'aggregate_hosts.uniq_aggregate_hosts0host0aggregate_id'") Jul 16 21:08:06.233869 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:08:06.233869 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi The above exception was the direct cause of the following exception: Jul 16 21:08:06.233869 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:08:06.233869 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:08:06.233869 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/aggregate.py", line 73, in _host_add_to_db Jul 16 21:08:06.233869 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi host_ref.save(context.session) Jul 16 21:08:06.233869 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/models.py", line 38, in save Jul 16 21:08:06.233869 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi session.flush() Jul 16 21:08:06.233869 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 16 21:08:06.233869 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi self._flush(objects) Jul 16 21:08:06.233869 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 16 21:08:06.233869 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi transaction.rollback(_capture_exception=True) Jul 16 21:08:06.233869 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 16 21:08:06.233869 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi compat.raise_( Jul 16 21:08:06.235723 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 16 21:08:06.235723 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception Jul 16 21:08:06.235723 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 16 21:08:06.235723 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi flush_context.execute() Jul 16 21:08:06.235723 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 16 21:08:06.235723 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi rec.execute(self) Jul 16 21:08:06.235723 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 16 21:08:06.235723 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi util.preloaded.orm_persistence.save_obj( Jul 16 21:08:06.235723 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 16 21:08:06.235723 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi _emit_insert_statements( Jul 16 21:08:06.235723 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jul 16 21:08:06.235723 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi result = connection._execute_20( Jul 16 21:08:06.235723 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 16 21:08:06.235723 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return meth(self, args_10style, kwargs_10style, execution_options) Jul 16 21:08:06.235723 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 16 21:08:06.235723 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return connection._execute_clauseelement( Jul 16 21:08:06.235723 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 16 21:08:06.235723 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi ret = self._execute_context( Jul 16 21:08:06.237312 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 16 21:08:06.237312 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi self._handle_dbapi_exception( Jul 16 21:08:06.237312 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 16 21:08:06.237312 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 16 21:08:06.237312 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 16 21:08:06.237312 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception Jul 16 21:08:06.237312 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 16 21:08:06.237312 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi self.dialect.do_execute( Jul 16 21:08:06.237312 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 16 21:08:06.237312 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi cursor.execute(statement, parameters) Jul 16 21:08:06.237312 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 16 21:08:06.237312 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi result = self._query(query) Jul 16 21:08:06.237312 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 16 21:08:06.237312 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi conn.query(q) Jul 16 21:08:06.237312 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 16 21:08:06.237312 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 16 21:08:06.237312 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 16 21:08:06.237312 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi result.read() Jul 16 21:08:06.239239 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 16 21:08:06.239239 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi first_packet = self.connection._read_packet() Jul 16 21:08:06.239239 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 16 21:08:06.239239 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi packet.raise_for_error() Jul 16 21:08:06.239239 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 16 21:08:06.239239 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi err.raise_mysql_exception(self._data) Jul 16 21:08:06.239239 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 16 21:08:06.239239 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise errorclass(errno, errval) Jul 16 21:08:06.239239 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'nch1-master-1-1' for key 'aggregate_hosts.uniq_aggregate_hosts0host0aggregate_id'") Jul 16 21:08:06.239239 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi [SQL: INSERT INTO aggregate_hosts (created_at, updated_at, host, aggregate_id) VALUES (%(created_at)s, %(updated_at)s, %(host)s, %(aggregate_id)s)] Jul 16 21:08:06.239239 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi [parameters: {'created_at': datetime.datetime(2022, 7, 16, 21, 8, 6, 59090), 'updated_at': None, 'host': 'nch1-master-1', 'aggregate_id': 1}] Jul 16 21:08:06.239239 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 16 21:08:06.239239 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:08:06.239239 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:08:06.239239 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:08:06.239239 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:08:06.239239 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/aggregates.py", line 180, in _add_host Jul 16 21:08:06.241653 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi aggregate = self.api.add_host_to_aggregate(context, id, host) Jul 16 21:08:06.241653 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/exception_wrapper.py", line 71, in wrapped Jul 16 21:08:06.241653 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi _emit_versioned_exception_notification( Jul 16 21:08:06.241653 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 16 21:08:06.241653 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi self.force_reraise() Jul 16 21:08:06.241653 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 16 21:08:06.241653 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise self.value Jul 16 21:08:06.241653 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/exception_wrapper.py", line 63, in wrapped Jul 16 21:08:06.241653 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(self, context, *args, **kw) Jul 16 21:08:06.241653 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 6513, in add_host_to_aggregate Jul 16 21:08:06.241653 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi aggregate.add_host(host_name) Jul 16 21:08:06.241653 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 226, in wrapper Jul 16 21:08:06.241653 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return fn(self, *args, **kwargs) Jul 16 21:08:06.241653 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/aggregate.py", line 395, in add_host Jul 16 21:08:06.241653 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi _host_add_to_db(self._context, self.id, host) Jul 16 21:08:06.241653 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/aggregate.py", line 76, in _host_add_to_db Jul 16 21:08:06.241653 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.AggregateHostExists(host=host, Jul 16 21:08:06.241653 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.AggregateHostExists: Aggregate 1 already has host nch1-master-1. Jul 16 21:08:06.241653 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:08:06.241653 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:08:06.244825 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:08:06.244825 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:08:06.244825 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:08:06.244825 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:08:06.244825 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:08:06.244825 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:08:06.244825 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:08:06.244825 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:08:06.244825 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 16 21:08:06.244825 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 16 21:08:06.244825 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/aggregates.py", line 187, in _add_host Jul 16 21:08:06.244825 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPConflict(explanation=e.format_message()) Jul 16 21:08:06.244825 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPConflict: Aggregate 1 already has host nch1-master-1. Jul 16 21:08:06.244825 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:08:06.244825 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:08:06.244825 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:08:06.244825 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:08:06.244825 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:08:06.244825 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:08:06.244825 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:08:06.244825 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:08:06.247457 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Aggregate 1 already has host nch1-master-1. Jul 16 21:08:06.247457 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:08:06.247457 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-8ce6c6aa-7586-4bfc-8a05-0915b452c95e tempest-AggregatesAdminNegativeTestJSON-578260511 tempest-AggregatesAdminNegativeTestJSON-578260511-project-admin] Returning 409 to user: Aggregate 1 already has host nch1-master-1. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:08:06.247457 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-8ce6c6aa-7586-4bfc-8a05-0915b452c95e tempest-AggregatesAdminNegativeTestJSON-578260511 tempest-AggregatesAdminNegativeTestJSON-578260511-project-admin] 10.222.0.16 "POST /compute/v2.1/os-aggregates/1/action" status: 409 len: 95 microversion: 2.1 time: 0.248991 Jul 16 21:08:06.251116 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-17c0be52-5572-424d-8a5a-ece000916bdf tempest-AggregatesAdminNegativeTestJSON-578260511 tempest-AggregatesAdminNegativeTestJSON-578260511-project-admin] Action: 'action', calling method: >, body: {"remove_host": {"host": "nch1-master-1"}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:08:06.251116 ncdv-master-1 devstack@n-api.service[138680]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-aggregates:remove_host": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:08:06.251116 ncdv-master-1 devstack@n-api.service[138680]: warnings.warn(msg) Jul 16 21:08:06.251520 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 15/24] 10.222.0.16 () {60 vars in 1190 bytes} [Sat Jul 16 21:08:05 2022] POST /compute/v2.1/os-aggregates/1/action => generated 95 bytes in 254 msecs (HTTP/1.1 409) 9 headers in 377 bytes (1 switches on core 0) Jul 16 21:08:06.254859 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-17c0be52-5572-424d-8a5a-ece000916bdf tempest-AggregatesAdminNegativeTestJSON-578260511 tempest-AggregatesAdminNegativeTestJSON-578260511-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:08:06.255437 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-17c0be52-5572-424d-8a5a-ece000916bdf tempest-AggregatesAdminNegativeTestJSON-578260511 tempest-AggregatesAdminNegativeTestJSON-578260511-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:08:06.578370 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-17c0be52-5572-424d-8a5a-ece000916bdf tempest-AggregatesAdminNegativeTestJSON-578260511 tempest-AggregatesAdminNegativeTestJSON-578260511-project-admin] 10.222.0.16 "POST /compute/v2.1/os-aggregates/1/action" status: 200 len: 226 microversion: 2.1 time: 0.339171 Jul 16 21:08:06.579290 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 10/25] 10.222.0.16 () {60 vars in 1190 bytes} [Sat Jul 16 21:08:06 2022] POST /compute/v2.1/os-aggregates/1/action => generated 226 bytes in 341 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 16 21:08:06.589688 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-44ce7067-01ab-47e5-9daf-92cca270e1e1 tempest-AggregatesAdminNegativeTestJSON-578260511 tempest-AggregatesAdminNegativeTestJSON-578260511-project-admin] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:08:06.590450 ncdv-master-1 devstack@n-api.service[138681]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-aggregates:delete": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:08:06.590450 ncdv-master-1 devstack@n-api.service[138681]: warnings.warn(msg) Jul 16 21:08:06.714598 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-44ce7067-01ab-47e5-9daf-92cca270e1e1 tempest-AggregatesAdminNegativeTestJSON-578260511 tempest-AggregatesAdminNegativeTestJSON-578260511-project-admin] 10.222.0.16 "DELETE /compute/v2.1/os-aggregates/1" status: 200 len: 0 microversion: 2.1 time: 0.128085 Jul 16 21:08:06.715166 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 16/26] 10.222.0.16 () {58 vars in 1151 bytes} [Sat Jul 16 21:08:06 2022] DELETE /compute/v2.1/os-aggregates/1 => generated 0 bytes in 130 msecs (HTTP/1.1 200) 9 headers in 355 bytes (1 switches on core 0) Jul 16 21:08:06.728000 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-a48388ba-0fc8-41d1-8db3-fe7020142bc6 tempest-AggregatesAdminNegativeTestJSON-578260511 tempest-AggregatesAdminNegativeTestJSON-578260511-project-admin] Action: 'create', calling method: >, body: {"aggregate": {"name": "tempest-test_aggregate-1528344378"}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:08:06.730132 ncdv-master-1 devstack@n-api.service[138680]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-aggregates:create": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:08:06.730132 ncdv-master-1 devstack@n-api.service[138680]: warnings.warn(msg) Jul 16 21:08:06.731650 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.objects.aggregate [None req-a48388ba-0fc8-41d1-8db3-fe7020142bc6 tempest-AggregatesAdminNegativeTestJSON-578260511 tempest-AggregatesAdminNegativeTestJSON-578260511-project-admin] Generated uuid 97e1462a-ecca-46ae-9ebc-c2d2d1dd897d for aggregate {{(pid=138680) create /opt/stack/nova/nova/objects/aggregate.py:298}} Jul 16 21:08:06.742200 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-9a3fbc56-6729-4e5c-a61b-1348c8cd20b9 tempest-AggregatesAdminTestV241-531976310 tempest-AggregatesAdminTestV241-531976310-project-admin] Action: 'create', calling method: >, body: {"aggregate": {"name": "tempest-test_aggregate-2089114722"}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:08:06.743833 ncdv-master-1 devstack@n-api.service[138681]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-aggregates:create": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:08:06.743833 ncdv-master-1 devstack@n-api.service[138681]: warnings.warn(msg) Jul 16 21:08:06.744843 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.objects.aggregate [None req-9a3fbc56-6729-4e5c-a61b-1348c8cd20b9 tempest-AggregatesAdminTestV241-531976310 tempest-AggregatesAdminTestV241-531976310-project-admin] Generated uuid a22c9190-ba93-44d6-a54c-3aa60ba316d2 for aggregate {{(pid=138681) create /opt/stack/nova/nova/objects/aggregate.py:298}} Jul 16 21:08:06.856206 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-a48388ba-0fc8-41d1-8db3-fe7020142bc6 tempest-AggregatesAdminNegativeTestJSON-578260511 tempest-AggregatesAdminNegativeTestJSON-578260511-project-admin] 10.222.0.16 "POST /compute/v2.1/os-aggregates" status: 200 len: 198 microversion: 2.1 time: 0.133704 Jul 16 21:08:06.859513 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 11/27] 10.222.0.16 () {60 vars in 1163 bytes} [Sat Jul 16 21:08:06 2022] POST /compute/v2.1/os-aggregates => generated 198 bytes in 139 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:08:06.863934 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-9a3fbc56-6729-4e5c-a61b-1348c8cd20b9 tempest-AggregatesAdminTestV241-531976310 tempest-AggregatesAdminTestV241-531976310-project-admin] 10.222.0.16 "POST /compute/v2.1/os-aggregates" status: 200 len: 246 microversion: 2.41 time: 0.128953 Jul 16 21:08:06.872299 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 17/28] 10.222.0.16 () {62 vars in 1204 bytes} [Sat Jul 16 21:08:06 2022] POST /compute/v2.1/os-aggregates => generated 246 bytes in 138 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 16 21:08:06.880147 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-73f79a54-4b30-458b-9bea-9edd0922c61f tempest-AggregatesAdminTestV241-531976310 tempest-AggregatesAdminTestV241-531976310-project-admin] Action: 'update', calling method: >, body: {"aggregate": {"name": "tempest-test_aggregate-765880380"}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:08:06.959554 ncdv-master-1 devstack@n-api.service[138681]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-aggregates:update": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:08:06.959554 ncdv-master-1 devstack@n-api.service[138681]: warnings.warn(msg) Jul 16 21:08:07.056713 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-73f79a54-4b30-458b-9bea-9edd0922c61f tempest-AggregatesAdminTestV241-531976310 tempest-AggregatesAdminTestV241-531976310-project-admin] 10.222.0.16 "PUT /compute/v2.1/os-aggregates/3" status: 200 len: 298 microversion: 2.41 time: 0.183296 Jul 16 21:08:07.057518 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 18/29] 10.222.0.16 () {62 vars in 1209 bytes} [Sat Jul 16 21:08:06 2022] PUT /compute/v2.1/os-aggregates/3 => generated 298 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 16 21:08:07.067965 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-bd3eeddd-ea50-4547-a3fe-e13c363a58db tempest-AggregatesAdminTestV241-531976310 tempest-AggregatesAdminTestV241-531976310-project-admin] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:08:07.068916 ncdv-master-1 devstack@n-api.service[138681]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-aggregates:show": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:08:07.068916 ncdv-master-1 devstack@n-api.service[138681]: warnings.warn(msg) Jul 16 21:08:07.076191 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-bd3eeddd-ea50-4547-a3fe-e13c363a58db tempest-AggregatesAdminTestV241-531976310 tempest-AggregatesAdminTestV241-531976310-project-admin] 10.222.0.16 "GET /compute/v2.1/os-aggregates/3" status: 200 len: 298 microversion: 2.41 time: 0.012320 Jul 16 21:08:07.076742 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 19/30] 10.222.0.16 () {60 vars in 1189 bytes} [Sat Jul 16 21:08:07 2022] GET /compute/v2.1/os-aggregates/3 => generated 298 bytes in 14 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Jul 16 21:08:07.087647 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-3d7a7e6d-e0bc-4ac5-bc03-d85cf33672de tempest-AggregatesAdminTestV241-531976310 tempest-AggregatesAdminTestV241-531976310-project-admin] Action: 'action', calling method: >, body: {"add_host": {"host": "nch1-master-1"}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:08:07.089360 ncdv-master-1 devstack@n-api.service[138681]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-aggregates:add_host": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:08:07.089360 ncdv-master-1 devstack@n-api.service[138681]: warnings.warn(msg) Jul 16 21:08:07.102509 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3d7a7e6d-e0bc-4ac5-bc03-d85cf33672de tempest-AggregatesAdminTestV241-531976310 tempest-AggregatesAdminTestV241-531976310-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:08:07.103380 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3d7a7e6d-e0bc-4ac5-bc03-d85cf33672de tempest-AggregatesAdminTestV241-531976310 tempest-AggregatesAdminTestV241-531976310-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:08:07.389184 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-f5669124-23f1-4a17-a894-2ce8269be7d5 tempest-AggregatesAdminNegativeTestJSON-1271440314 tempest-AggregatesAdminNegativeTestJSON-1271440314-project] Action: 'action', calling method: >, body: {"add_host": {"host": "nch1-master-1"}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:08:07.390510 ncdv-master-1 devstack@n-api.service[138680]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-aggregates:add_host": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:08:07.390510 ncdv-master-1 devstack@n-api.service[138680]: warnings.warn(msg) Jul 16 21:08:07.391721 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-f5669124-23f1-4a17-a894-2ce8269be7d5 tempest-AggregatesAdminNegativeTestJSON-1271440314 tempest-AggregatesAdminNegativeTestJSON-1271440314-project] Policy check for os_compute_api:os-aggregates:add_host failed with credentials {'is_admin': False, 'user_id': '35a4905d463a4db2960294407f747f20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '49044125b0414a4ab0e2ab9eb7d66e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:08:07.396921 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-f5669124-23f1-4a17-a894-2ce8269be7d5 tempest-AggregatesAdminNegativeTestJSON-1271440314 tempest-AggregatesAdminNegativeTestJSON-1271440314-project] Request method failure captured: Jul 16 21:08:07.396921 ncdv-master-1 devstack@n-api.service[138680]: request: POST /compute/v2.1/os-aggregates/2/action HTTP/1.1 Jul 16 21:08:07.396921 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:08:07.396921 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:08:07.396921 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:08:07.396921 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 39 Jul 16 21:08:07.396921 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:08:07.396921 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:08:07.396921 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:08:07.396921 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:08:07.396921 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0yiwx37gJG3UPf6cs7N4UkgaksWm_yG7GacwHzT4X_Y36InoB6bLW8dxMeWTnPNUr5MkHyB6zCa2ShiCwvtwzG8FdDgsivxSANP9O8r63IrJOV2dcu5-Fla-kUEiqkTUeLu0mggNWQr19K_Q1rkm8xG96kdc5KRDvRt2-aIVSJpj_Ug Jul 16 21:08:07.396921 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:08:07.396921 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:08:07.396921 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:08:07.396921 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:08:07.396921 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:08:07.396921 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:08:07.396921 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: 49044125b0414a4ab0e2ab9eb7d66e6e Jul 16 21:08:07.396921 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-AggregatesAdminNegativeTestJSON-1271440314 Jul 16 21:08:07.396921 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:08:07.396921 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:08:07.398336 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/49044125b0414a4ab0e2ab9eb7d66e6e"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/49044125b0414a4ab0e2ab9eb7d66e6e"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_49044125b0414a4ab0e2ab9eb7d66e6e"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/49044125b0414a4ab0e2ab9eb7d66e6e"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:08:07.398336 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-AggregatesAdminNegativeTestJSON-1271440314 Jul 16 21:08:07.398336 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: 49044125b0414a4ab0e2ab9eb7d66e6e Jul 16 21:08:07.398336 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-AggregatesAdminNegativeTestJSON-1271440314 Jul 16 21:08:07.398336 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-AggregatesAdminNegativeTestJSON-1271440314-project Jul 16 21:08:07.398336 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:08:07.398336 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:08:07.398336 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: 35a4905d463a4db2960294407f747f20 Jul 16 21:08:07.398336 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-AggregatesAdminNegativeTestJSON-1271440314-project Jul 16 21:08:07.398336 ncdv-master-1 devstack@n-api.service[138680]: Jul 16 21:08:07.398336 ncdv-master-1 devstack@n-api.service[138680]: {"add_host": {"host": "nch1-master-1"}} Jul 16 21:08:07.398336 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 21:08:07.398336 ncdv-master-1 devstack@n-api.service[138680]: exception: Policy doesn't allow os_compute_api:os-aggregates:add_host to be performed. Jul 16 21:08:07.399940 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': '2', 'body': {'add_host': {'host': 'nch1-master-1'}}} Jul 16 21:08:07.399940 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:08:07.399940 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:08:07.399940 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:08:07.399940 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:08:07.399940 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:08:07.399940 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:08:07.399940 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:08:07.399940 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:08:07.399940 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 16 21:08:07.399940 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 16 21:08:07.399940 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/aggregates.py", line 172, in _add_host Jul 16 21:08:07.399940 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi context.can(aggr_policies.POLICY_ROOT % 'add_host', target={}) Jul 16 21:08:07.399940 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/context.py", line 240, in can Jul 16 21:08:07.399940 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return policy.authorize(self, action, target) Jul 16 21:08:07.399940 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 199, in authorize Jul 16 21:08:07.399940 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi LOG.debug('Policy check for %(action)s failed with credentials ' Jul 16 21:08:07.399940 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 16 21:08:07.399940 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi self.force_reraise() Jul 16 21:08:07.400892 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 16 21:08:07.400892 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise self.value Jul 16 21:08:07.400892 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 186, in authorize Jul 16 21:08:07.400892 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi result = _ENFORCER.authorize(action, target, context, Jul 16 21:08:07.400892 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1184, in authorize Jul 16 21:08:07.400892 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return self.enforce( Jul 16 21:08:07.400892 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1088, in enforce Jul 16 21:08:07.400892 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc(*args, **kwargs) Jul 16 21:08:07.400892 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.PolicyNotAuthorized: Policy doesn't allow os_compute_api:os-aggregates:add_host to be performed. Jul 16 21:08:07.400892 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:08:07.400892 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:08:07.400892 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:08:07.400892 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:08:07.400892 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:08:07.400892 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:08:07.400892 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 354, in __exit__ Jul 16 21:08:07.400892 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(webob.exc.HTTPForbidden( Jul 16 21:08:07.400892 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Policy doesn't allow os_compute_api:os-aggregates:add_host to be performed. Jul 16 21:08:07.400892 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:08:07.401769 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-f5669124-23f1-4a17-a894-2ce8269be7d5 tempest-AggregatesAdminNegativeTestJSON-1271440314 tempest-AggregatesAdminNegativeTestJSON-1271440314-project] Returning 403 to user: Policy doesn't allow os_compute_api:os-aggregates:add_host to be performed. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:08:07.401769 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-f5669124-23f1-4a17-a894-2ce8269be7d5 tempest-AggregatesAdminNegativeTestJSON-1271440314 tempest-AggregatesAdminNegativeTestJSON-1271440314-project] 10.222.0.16 "POST /compute/v2.1/os-aggregates/2/action" status: 403 len: 118 microversion: 2.1 time: 0.532654 Jul 16 21:08:07.403161 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 12/31] 10.222.0.16 () {60 vars in 1190 bytes} [Sat Jul 16 21:08:06 2022] POST /compute/v2.1/os-aggregates/2/action => generated 118 bytes in 538 msecs (HTTP/1.1 403) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:08:07.411872 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-4b0dcbd6-9670-4a0e-af6b-336e5aa8e47e tempest-AggregatesAdminNegativeTestJSON-578260511 tempest-AggregatesAdminNegativeTestJSON-578260511-project-admin] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:08:07.420308 ncdv-master-1 devstack@n-api.service[138680]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-aggregates:delete": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:08:07.420308 ncdv-master-1 devstack@n-api.service[138680]: warnings.warn(msg) Jul 16 21:08:07.520782 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-4b0dcbd6-9670-4a0e-af6b-336e5aa8e47e tempest-AggregatesAdminNegativeTestJSON-578260511 tempest-AggregatesAdminNegativeTestJSON-578260511-project-admin] 10.222.0.16 "DELETE /compute/v2.1/os-aggregates/2" status: 200 len: 0 microversion: 2.1 time: 0.112975 Jul 16 21:08:07.523467 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 13/32] 10.222.0.16 () {58 vars in 1151 bytes} [Sat Jul 16 21:08:07 2022] DELETE /compute/v2.1/os-aggregates/2 => generated 0 bytes in 120 msecs (HTTP/1.1 200) 9 headers in 355 bytes (1 switches on core 0) Jul 16 21:08:07.538668 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-9ffa2085-cef4-4013-bbca-495073b70a37 tempest-AggregatesAdminNegativeTestJSON-578260511 tempest-AggregatesAdminNegativeTestJSON-578260511-project-admin] Action: 'create', calling method: >, body: {"aggregate": {"name": "tempest-test_aggregate-2002803285"}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:08:07.541495 ncdv-master-1 devstack@n-api.service[138680]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-aggregates:create": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:08:07.541495 ncdv-master-1 devstack@n-api.service[138680]: warnings.warn(msg) Jul 16 21:08:07.544158 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.objects.aggregate [None req-9ffa2085-cef4-4013-bbca-495073b70a37 tempest-AggregatesAdminNegativeTestJSON-578260511 tempest-AggregatesAdminNegativeTestJSON-578260511-project-admin] Generated uuid f8120357-e06e-42c2-bae6-3d8568e686ca for aggregate {{(pid=138680) create /opt/stack/nova/nova/objects/aggregate.py:298}} Jul 16 21:08:07.770959 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-9ffa2085-cef4-4013-bbca-495073b70a37 tempest-AggregatesAdminNegativeTestJSON-578260511 tempest-AggregatesAdminNegativeTestJSON-578260511-project-admin] 10.222.0.16 "POST /compute/v2.1/os-aggregates" status: 200 len: 198 microversion: 2.1 time: 0.241691 Jul 16 21:08:07.777485 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 14/33] 10.222.0.16 () {60 vars in 1163 bytes} [Sat Jul 16 21:08:07 2022] POST /compute/v2.1/os-aggregates => generated 198 bytes in 249 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:08:07.782553 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-b095f738-4fd5-4845-8fe7-3917346da840 tempest-AggregatesAdminNegativeTestJSON-578260511 tempest-AggregatesAdminNegativeTestJSON-578260511-project-admin] Action: 'action', calling method: >, body: {"add_host": {"host": "tempest-nonexist_host-445474147"}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:08:07.784421 ncdv-master-1 devstack@n-api.service[138680]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-aggregates:add_host": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:08:07.784421 ncdv-master-1 devstack@n-api.service[138680]: warnings.warn(msg) Jul 16 21:08:07.793019 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b095f738-4fd5-4845-8fe7-3917346da840 tempest-AggregatesAdminNegativeTestJSON-578260511 tempest-AggregatesAdminNegativeTestJSON-578260511-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:08:07.793661 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b095f738-4fd5-4845-8fe7-3917346da840 tempest-AggregatesAdminNegativeTestJSON-578260511 tempest-AggregatesAdminNegativeTestJSON-578260511-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:08:07.802129 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b095f738-4fd5-4845-8fe7-3917346da840 tempest-AggregatesAdminNegativeTestJSON-578260511 tempest-AggregatesAdminNegativeTestJSON-578260511-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:08:07.802610 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b095f738-4fd5-4845-8fe7-3917346da840 tempest-AggregatesAdminNegativeTestJSON-578260511 tempest-AggregatesAdminNegativeTestJSON-578260511-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:08:07.816827 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-b095f738-4fd5-4845-8fe7-3917346da840 tempest-AggregatesAdminNegativeTestJSON-578260511 tempest-AggregatesAdminNegativeTestJSON-578260511-project-admin] HTTP exception thrown: Compute host tempest-nonexist_host-445474147 could not be found. Jul 16 21:08:07.822081 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-b095f738-4fd5-4845-8fe7-3917346da840 tempest-AggregatesAdminNegativeTestJSON-578260511 tempest-AggregatesAdminNegativeTestJSON-578260511-project-admin] Request method failure captured: Jul 16 21:08:07.822081 ncdv-master-1 devstack@n-api.service[138680]: request: POST /compute/v2.1/os-aggregates/4/action HTTP/1.1 Jul 16 21:08:07.822081 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:08:07.822081 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:08:07.822081 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:08:07.822081 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 57 Jul 16 21:08:07.822081 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:08:07.822081 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:08:07.822081 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:08:07.822081 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:08:07.822081 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0yiz0unej4HpWPJrVBlkA1u9-AWCKDSpHymJUf5DT-hiCYEGZJzMCi_CgXajssbkUJoCwaGruhY3wft9OVrurmz0_pRbvWF03aApzlFrXwt0GhLH6-kKTcDhg1N2k0bLKCTgcUoad_Ma5kdyMmhxUVTCB2mCAuoF3iPjsftSucch2dg Jul 16 21:08:07.822081 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:08:07.822081 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:08:07.822081 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:08:07.822081 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:08:07.822081 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:08:07.822081 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:08:07.822081 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: b6be58dcb13b4cec8f0bfc58f0080173 Jul 16 21:08:07.822081 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-AggregatesAdminNegativeTestJSON-578260511 Jul 16 21:08:07.822081 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,admin,reader Jul 16 21:08:07.822081 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,admin,reader Jul 16 21:08:07.823266 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/b6be58dcb13b4cec8f0bfc58f0080173"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/b6be58dcb13b4cec8f0bfc58f0080173"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_b6be58dcb13b4cec8f0bfc58f0080173"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/b6be58dcb13b4cec8f0bfc58f0080173"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:08:07.823266 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-AggregatesAdminNegativeTestJSON-578260511 Jul 16 21:08:07.823266 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: b6be58dcb13b4cec8f0bfc58f0080173 Jul 16 21:08:07.823266 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-AggregatesAdminNegativeTestJSON-578260511 Jul 16 21:08:07.823266 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-AggregatesAdminNegativeTestJSON-578260511-project-admin Jul 16 21:08:07.823266 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:08:07.823266 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:08:07.823266 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: cadb63bb5d6b4d7597987780f7d9ef21 Jul 16 21:08:07.823266 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-AggregatesAdminNegativeTestJSON-578260511-project-admin Jul 16 21:08:07.823266 ncdv-master-1 devstack@n-api.service[138680]: Jul 16 21:08:07.823266 ncdv-master-1 devstack@n-api.service[138680]: {"add_host": {"host": "tempest-nonexist_host-445474147"}} Jul 16 21:08:07.823266 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 21:08:07.823266 ncdv-master-1 devstack@n-api.service[138680]: exception: Compute host tempest-nonexist_host-445474147 could not be found. Jul 16 21:08:07.823889 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': '4', 'body': {'add_host': {'host': 'tempest-nonexist_host-445474147'}}} Jul 16 21:08:07.823889 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:08:07.823889 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:08:07.823889 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5868, in _get_service_in_cell_by_host Jul 16 21:08:07.823889 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi mapping = objects.HostMapping.get_by_host(context, host_name) Jul 16 21:08:07.823889 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Jul 16 21:08:07.823889 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Jul 16 21:08:07.823889 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/host_mapping.py", line 105, in get_by_host Jul 16 21:08:07.823889 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi db_mapping = cls._get_by_host_from_db(context, host) Jul 16 21:08:07.823889 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1010, in wrapper Jul 16 21:08:07.823889 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return fn(*args, **kwargs) Jul 16 21:08:07.823889 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/host_mapping.py", line 100, in _get_by_host_from_db Jul 16 21:08:07.823889 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.HostMappingNotFound(name=host) Jul 16 21:08:07.823889 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.HostMappingNotFound: Host 'tempest-nonexist_host-445474147' is not mapped to any cell Jul 16 21:08:07.823889 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:08:07.823889 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:08:07.823889 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:08:07.823889 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:08:07.823889 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5874, in _get_service_in_cell_by_host Jul 16 21:08:07.824828 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi service = _find_service_in_cell(context, service_host=host_name) Jul 16 21:08:07.824828 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5936, in _find_service_in_cell Jul 16 21:08:07.824828 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.NotFound() Jul 16 21:08:07.824828 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.NotFound: Resource could not be found. Jul 16 21:08:07.824828 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:08:07.824828 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:08:07.824828 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:08:07.824828 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:08:07.824828 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/aggregates.py", line 180, in _add_host Jul 16 21:08:07.824828 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi aggregate = self.api.add_host_to_aggregate(context, id, host) Jul 16 21:08:07.824828 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/exception_wrapper.py", line 71, in wrapped Jul 16 21:08:07.824828 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi _emit_versioned_exception_notification( Jul 16 21:08:07.824828 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 16 21:08:07.824828 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi self.force_reraise() Jul 16 21:08:07.824828 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 16 21:08:07.824828 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise self.value Jul 16 21:08:07.824828 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/exception_wrapper.py", line 63, in wrapped Jul 16 21:08:07.824828 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(self, context, *args, **kw) Jul 16 21:08:07.824828 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 6493, in add_host_to_aggregate Jul 16 21:08:07.824828 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi service = _get_service_in_cell_by_host(context, host_name) Jul 16 21:08:07.825803 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5876, in _get_service_in_cell_by_host Jul 16 21:08:07.825803 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.ComputeHostNotFound(host=host_name) Jul 16 21:08:07.825803 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.ComputeHostNotFound: Compute host tempest-nonexist_host-445474147 could not be found. Jul 16 21:08:07.825803 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:08:07.825803 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:08:07.825803 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:08:07.825803 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:08:07.825803 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:08:07.825803 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:08:07.825803 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:08:07.825803 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:08:07.825803 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:08:07.825803 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:08:07.825803 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 16 21:08:07.825803 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 16 21:08:07.825803 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/aggregates.py", line 184, in _add_host Jul 16 21:08:07.825803 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:08:07.825803 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Compute host tempest-nonexist_host-445474147 could not be found. Jul 16 21:08:07.825803 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:08:07.825803 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:08:07.825803 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:08:07.826777 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:08:07.826777 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:08:07.826777 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:08:07.826777 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:08:07.826777 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:08:07.826777 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Compute host tempest-nonexist_host-445474147 could not be found. Jul 16 21:08:07.826777 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:08:07.829283 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-b095f738-4fd5-4845-8fe7-3917346da840 tempest-AggregatesAdminNegativeTestJSON-578260511 tempest-AggregatesAdminNegativeTestJSON-578260511-project-admin] Returning 404 to user: Compute host tempest-nonexist_host-445474147 could not be found. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:08:07.830248 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-b095f738-4fd5-4845-8fe7-3917346da840 tempest-AggregatesAdminNegativeTestJSON-578260511 tempest-AggregatesAdminNegativeTestJSON-578260511-project-admin] 10.222.0.16 "POST /compute/v2.1/os-aggregates/4/action" status: 404 len: 110 microversion: 2.1 time: 0.051588 Jul 16 21:08:07.831670 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 15/34] 10.222.0.16 () {60 vars in 1190 bytes} [Sat Jul 16 21:08:07 2022] POST /compute/v2.1/os-aggregates/4/action => generated 110 bytes in 53 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:08:07.841007 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-c22a5192-aa0d-419c-ab8c-9f14acffc2a6 tempest-AggregatesAdminNegativeTestJSON-578260511 tempest-AggregatesAdminNegativeTestJSON-578260511-project-admin] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:08:07.841991 ncdv-master-1 devstack@n-api.service[138680]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-aggregates:delete": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:08:07.841991 ncdv-master-1 devstack@n-api.service[138680]: warnings.warn(msg) Jul 16 21:08:08.022973 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-c22a5192-aa0d-419c-ab8c-9f14acffc2a6 tempest-AggregatesAdminNegativeTestJSON-578260511 tempest-AggregatesAdminNegativeTestJSON-578260511-project-admin] 10.222.0.16 "DELETE /compute/v2.1/os-aggregates/4" status: 200 len: 0 microversion: 2.1 time: 0.186255 Jul 16 21:08:08.032897 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 16/35] 10.222.0.16 () {58 vars in 1151 bytes} [Sat Jul 16 21:08:07 2022] DELETE /compute/v2.1/os-aggregates/4 => generated 0 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 355 bytes (2 switches on core 0) Jul 16 21:08:08.032897 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-2a2680ed-fd9a-442c-8dee-eca92e927658 tempest-AggregatesAdminNegativeTestJSON-578260511 tempest-AggregatesAdminNegativeTestJSON-578260511-project-admin] Action: 'create', calling method: >, body: {"aggregate": {"name": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:08:08.047149 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-2a2680ed-fd9a-442c-8dee-eca92e927658 tempest-AggregatesAdminNegativeTestJSON-578260511 tempest-AggregatesAdminNegativeTestJSON-578260511-project-admin] Request method failure captured: Jul 16 21:08:08.047149 ncdv-master-1 devstack@n-api.service[138680]: request: POST /compute/v2.1/os-aggregates HTTP/1.1 Jul 16 21:08:08.047149 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:08:08.047149 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:08:08.047149 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:08:08.047149 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 283 Jul 16 21:08:08.047149 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:08:08.047149 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:08:08.047149 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:08:08.047149 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:08:08.047149 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0yiz0unej4HpWPJrVBlkA1u9-AWCKDSpHymJUf5DT-hiCYEGZJzMCi_CgXajssbkUJoCwaGruhY3wft9OVrurmz0_pRbvWF03aApzlFrXwt0GhLH6-kKTcDhg1N2k0bLKCTgcUoad_Ma5kdyMmhxUVTCB2mCAuoF3iPjsftSucch2dg Jul 16 21:08:08.047149 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:08:08.047149 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:08:08.047149 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:08:08.047149 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:08:08.047149 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:08:08.047149 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:08:08.047149 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: b6be58dcb13b4cec8f0bfc58f0080173 Jul 16 21:08:08.047149 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-AggregatesAdminNegativeTestJSON-578260511 Jul 16 21:08:08.047149 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,admin,reader Jul 16 21:08:08.047149 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,admin,reader Jul 16 21:08:08.048433 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/b6be58dcb13b4cec8f0bfc58f0080173"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/b6be58dcb13b4cec8f0bfc58f0080173"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_b6be58dcb13b4cec8f0bfc58f0080173"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/b6be58dcb13b4cec8f0bfc58f0080173"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:08:08.048433 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-AggregatesAdminNegativeTestJSON-578260511 Jul 16 21:08:08.048433 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: b6be58dcb13b4cec8f0bfc58f0080173 Jul 16 21:08:08.048433 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-AggregatesAdminNegativeTestJSON-578260511 Jul 16 21:08:08.048433 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-AggregatesAdminNegativeTestJSON-578260511-project-admin Jul 16 21:08:08.048433 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:08:08.048433 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:08:08.048433 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: cadb63bb5d6b4d7597987780f7d9ef21 Jul 16 21:08:08.048433 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-AggregatesAdminNegativeTestJSON-578260511-project-admin Jul 16 21:08:08.048433 ncdv-master-1 devstack@n-api.service[138680]: Jul 16 21:08:08.048433 ncdv-master-1 devstack@n-api.service[138680]: {"aggregate": {"name": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"}} Jul 16 21:08:08.049206 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 21:08:08.049206 ncdv-master-1 devstack@n-api.service[138680]: exception: Invalid input for field/attribute name. Value: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa. 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' is too long Jul 16 21:08:08.049206 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'body': {'aggregate': {'name': 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa'}}} Jul 16 21:08:08.049206 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:08:08.049206 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:08:08.049206 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 279, in validate Jul 16 21:08:08.049206 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Jul 16 21:08:08.049206 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/lib/python3/dist-packages/jsonschema/validators.py", line 353, in validate Jul 16 21:08:08.049206 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise error Jul 16 21:08:08.049206 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' is too long Jul 16 21:08:08.049206 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:08:08.049972 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Failed validating 'maxLength' in schema['properties']['aggregate']['properties']['name']: Jul 16 21:08:08.049972 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi {'format': 'name', 'maxLength': 255, 'minLength': 1, 'type': 'string'} Jul 16 21:08:08.049972 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:08:08.049972 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi On instance['aggregate']: Jul 16 21:08:08.049972 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' Jul 16 21:08:08.049972 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:08:08.049972 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:08:08.049972 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:08:08.049972 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:08:08.049972 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:08:08.049972 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:08:08.049972 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:08:08.049972 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:08:08.049972 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:08:08.049972 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:08:08.049972 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 16 21:08:08.049972 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 16 21:08:08.049972 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Jul 16 21:08:08.049972 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Jul 16 21:08:08.051145 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Jul 16 21:08:08.051145 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Jul 16 21:08:08.051145 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 308, in validate Jul 16 21:08:08.051145 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Jul 16 21:08:08.051145 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute name. Value: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa. 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' is too long Jul 16 21:08:08.051145 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:08:08.051145 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:08:08.051145 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:08:08.051145 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:08:08.051145 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:08:08.051145 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:08:08.051145 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Jul 16 21:08:08.051145 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Jul 16 21:08:08.051840 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute name. Value: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa. 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' is too long Jul 16 21:08:08.051840 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:08:08.051840 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-2a2680ed-fd9a-442c-8dee-eca92e927658 tempest-AggregatesAdminNegativeTestJSON-578260511 tempest-AggregatesAdminNegativeTestJSON-578260511-project-admin] Returning 400 to user: Invalid input for field/attribute name. Value: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa. 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' is too long {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:08:08.051840 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-2a2680ed-fd9a-442c-8dee-eca92e927658 tempest-AggregatesAdminNegativeTestJSON-578260511 tempest-AggregatesAdminNegativeTestJSON-578260511-project-admin] 10.222.0.16 "POST /compute/v2.1/os-aggregates" status: 400 len: 619 microversion: 2.1 time: 0.019202 Jul 16 21:08:08.052086 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 17/36] 10.222.0.16 () {60 vars in 1164 bytes} [Sat Jul 16 21:08:08 2022] POST /compute/v2.1/os-aggregates => generated 619 bytes in 20 msecs (HTTP/1.1 400) 9 headers in 381 bytes (2 switches on core 0) Jul 16 21:08:08.057401 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-06eb54fd-33b5-428d-9ed0-3bb2c2924c93 tempest-AggregatesAdminNegativeTestJSON-578260511 tempest-AggregatesAdminNegativeTestJSON-578260511-project-admin] Action: 'create', calling method: >, body: {"aggregate": {"name": ""}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:08:08.059703 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-06eb54fd-33b5-428d-9ed0-3bb2c2924c93 tempest-AggregatesAdminNegativeTestJSON-578260511 tempest-AggregatesAdminNegativeTestJSON-578260511-project-admin] Request method failure captured: Jul 16 21:08:08.059703 ncdv-master-1 devstack@n-api.service[138680]: request: POST /compute/v2.1/os-aggregates HTTP/1.1 Jul 16 21:08:08.059703 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:08:08.059703 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:08:08.059703 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:08:08.059703 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 27 Jul 16 21:08:08.059703 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:08:08.059703 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:08:08.059703 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:08:08.059703 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:08:08.059703 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0yiz0unej4HpWPJrVBlkA1u9-AWCKDSpHymJUf5DT-hiCYEGZJzMCi_CgXajssbkUJoCwaGruhY3wft9OVrurmz0_pRbvWF03aApzlFrXwt0GhLH6-kKTcDhg1N2k0bLKCTgcUoad_Ma5kdyMmhxUVTCB2mCAuoF3iPjsftSucch2dg Jul 16 21:08:08.059703 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:08:08.059703 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:08:08.059703 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:08:08.059703 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:08:08.059703 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:08:08.059703 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:08:08.059703 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: b6be58dcb13b4cec8f0bfc58f0080173 Jul 16 21:08:08.059703 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-AggregatesAdminNegativeTestJSON-578260511 Jul 16 21:08:08.059703 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,admin,reader Jul 16 21:08:08.059703 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,admin,reader Jul 16 21:08:08.061074 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/b6be58dcb13b4cec8f0bfc58f0080173"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/b6be58dcb13b4cec8f0bfc58f0080173"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_b6be58dcb13b4cec8f0bfc58f0080173"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/b6be58dcb13b4cec8f0bfc58f0080173"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:08:08.061074 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-AggregatesAdminNegativeTestJSON-578260511 Jul 16 21:08:08.061074 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: b6be58dcb13b4cec8f0bfc58f0080173 Jul 16 21:08:08.061074 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-AggregatesAdminNegativeTestJSON-578260511 Jul 16 21:08:08.061074 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-AggregatesAdminNegativeTestJSON-578260511-project-admin Jul 16 21:08:08.061074 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:08:08.061074 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:08:08.061074 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: cadb63bb5d6b4d7597987780f7d9ef21 Jul 16 21:08:08.061074 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-AggregatesAdminNegativeTestJSON-578260511-project-admin Jul 16 21:08:08.061074 ncdv-master-1 devstack@n-api.service[138680]: Jul 16 21:08:08.061074 ncdv-master-1 devstack@n-api.service[138680]: {"aggregate": {"name": ""}} Jul 16 21:08:08.061074 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 21:08:08.061074 ncdv-master-1 devstack@n-api.service[138680]: exception: Invalid input for field/attribute name. Value: . '' is too short Jul 16 21:08:08.061940 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'body': {'aggregate': {'name': ''}}} Jul 16 21:08:08.061940 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:08:08.061940 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:08:08.061940 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 279, in validate Jul 16 21:08:08.061940 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Jul 16 21:08:08.061940 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/lib/python3/dist-packages/jsonschema/validators.py", line 353, in validate Jul 16 21:08:08.061940 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise error Jul 16 21:08:08.061940 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: '' is too short Jul 16 21:08:08.061940 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:08:08.061940 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Failed validating 'minLength' in schema['properties']['aggregate']['properties']['name']: Jul 16 21:08:08.061940 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi {'format': 'name', 'maxLength': 255, 'minLength': 1, 'type': 'string'} Jul 16 21:08:08.061940 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:08:08.061940 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi On instance['aggregate']: Jul 16 21:08:08.061940 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi '' Jul 16 21:08:08.061940 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:08:08.061940 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:08:08.061940 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:08:08.061940 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:08:08.061940 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:08:08.061940 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:08:08.061940 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:08:08.061940 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:08:08.061940 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:08:08.063265 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:08:08.063265 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 16 21:08:08.063265 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 16 21:08:08.063265 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Jul 16 21:08:08.063265 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Jul 16 21:08:08.063265 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Jul 16 21:08:08.063265 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Jul 16 21:08:08.063265 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 308, in validate Jul 16 21:08:08.063265 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Jul 16 21:08:08.063265 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute name. Value: . '' is too short Jul 16 21:08:08.063265 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:08:08.063265 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:08:08.063265 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:08:08.063265 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:08:08.063265 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:08:08.063265 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:08:08.063265 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Jul 16 21:08:08.063265 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Jul 16 21:08:08.063265 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute name. Value: . '' is too short Jul 16 21:08:08.063265 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:08:08.066281 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-06eb54fd-33b5-428d-9ed0-3bb2c2924c93 tempest-AggregatesAdminNegativeTestJSON-578260511 tempest-AggregatesAdminNegativeTestJSON-578260511-project-admin] Returning 400 to user: Invalid input for field/attribute name. Value: . '' is too short {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:08:08.066281 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-06eb54fd-33b5-428d-9ed0-3bb2c2924c93 tempest-AggregatesAdminNegativeTestJSON-578260511 tempest-AggregatesAdminNegativeTestJSON-578260511-project-admin] 10.222.0.16 "POST /compute/v2.1/os-aggregates" status: 400 len: 108 microversion: 2.1 time: 0.007194 Jul 16 21:08:08.066281 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 18/37] 10.222.0.16 () {60 vars in 1163 bytes} [Sat Jul 16 21:08:08 2022] POST /compute/v2.1/os-aggregates => generated 108 bytes in 8 msecs (HTTP/1.1 400) 9 headers in 381 bytes (2 switches on core 0) Jul 16 21:08:08.072686 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-f1420da1-9cf4-4322-a142-efa9a3b94765 tempest-AggregatesAdminNegativeTestJSON-1271440314 tempest-AggregatesAdminNegativeTestJSON-1271440314-project] Action: 'create', calling method: >, body: {"aggregate": {"name": "tempest-test_aggregate-264847656"}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:08:08.075197 ncdv-master-1 devstack@n-api.service[138680]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-aggregates:create": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:08:08.075197 ncdv-master-1 devstack@n-api.service[138680]: warnings.warn(msg) Jul 16 21:08:08.076102 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-f1420da1-9cf4-4322-a142-efa9a3b94765 tempest-AggregatesAdminNegativeTestJSON-1271440314 tempest-AggregatesAdminNegativeTestJSON-1271440314-project] Policy check for os_compute_api:os-aggregates:create failed with credentials {'is_admin': False, 'user_id': '35a4905d463a4db2960294407f747f20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '49044125b0414a4ab0e2ab9eb7d66e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:08:08.077882 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-f1420da1-9cf4-4322-a142-efa9a3b94765 tempest-AggregatesAdminNegativeTestJSON-1271440314 tempest-AggregatesAdminNegativeTestJSON-1271440314-project] Request method failure captured: Jul 16 21:08:08.077882 ncdv-master-1 devstack@n-api.service[138680]: request: POST /compute/v2.1/os-aggregates HTTP/1.1 Jul 16 21:08:08.077882 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:08:08.077882 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:08:08.077882 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:08:08.077882 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 59 Jul 16 21:08:08.077882 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:08:08.077882 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:08:08.077882 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:08:08.077882 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:08:08.077882 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0yiwx37gJG3UPf6cs7N4UkgaksWm_yG7GacwHzT4X_Y36InoB6bLW8dxMeWTnPNUr5MkHyB6zCa2ShiCwvtwzG8FdDgsivxSANP9O8r63IrJOV2dcu5-Fla-kUEiqkTUeLu0mggNWQr19K_Q1rkm8xG96kdc5KRDvRt2-aIVSJpj_Ug Jul 16 21:08:08.077882 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:08:08.077882 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:08:08.077882 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:08:08.077882 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:08:08.077882 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:08:08.077882 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:08:08.077882 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: 49044125b0414a4ab0e2ab9eb7d66e6e Jul 16 21:08:08.077882 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-AggregatesAdminNegativeTestJSON-1271440314 Jul 16 21:08:08.077882 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:08:08.077882 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:08:08.079293 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/49044125b0414a4ab0e2ab9eb7d66e6e"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/49044125b0414a4ab0e2ab9eb7d66e6e"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_49044125b0414a4ab0e2ab9eb7d66e6e"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/49044125b0414a4ab0e2ab9eb7d66e6e"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:08:08.079293 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-AggregatesAdminNegativeTestJSON-1271440314 Jul 16 21:08:08.079293 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: 49044125b0414a4ab0e2ab9eb7d66e6e Jul 16 21:08:08.079293 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-AggregatesAdminNegativeTestJSON-1271440314 Jul 16 21:08:08.079293 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-AggregatesAdminNegativeTestJSON-1271440314-project Jul 16 21:08:08.079293 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:08:08.079293 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:08:08.079293 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: 35a4905d463a4db2960294407f747f20 Jul 16 21:08:08.079293 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-AggregatesAdminNegativeTestJSON-1271440314-project Jul 16 21:08:08.079293 ncdv-master-1 devstack@n-api.service[138680]: Jul 16 21:08:08.079293 ncdv-master-1 devstack@n-api.service[138680]: {"aggregate": {"name": "tempest-test_aggregate-264847656"}} Jul 16 21:08:08.079293 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 21:08:08.079293 ncdv-master-1 devstack@n-api.service[138680]: exception: Policy doesn't allow os_compute_api:os-aggregates:create to be performed. Jul 16 21:08:08.080077 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'body': {'aggregate': {'name': 'tempest-test_aggregate-264847656'}}} Jul 16 21:08:08.080077 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:08:08.080077 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:08:08.080077 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:08:08.080077 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:08:08.080077 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:08:08.080077 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:08:08.080077 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:08:08.080077 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:08:08.080077 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 16 21:08:08.080077 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 16 21:08:08.080077 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 16 21:08:08.080077 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 16 21:08:08.080077 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/aggregates.py", line 70, in create Jul 16 21:08:08.080077 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi context.can(aggr_policies.POLICY_ROOT % 'create', target={}) Jul 16 21:08:08.080077 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/context.py", line 240, in can Jul 16 21:08:08.080077 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return policy.authorize(self, action, target) Jul 16 21:08:08.080077 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 199, in authorize Jul 16 21:08:08.080077 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi LOG.debug('Policy check for %(action)s failed with credentials ' Jul 16 21:08:08.082568 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 16 21:08:08.082568 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi self.force_reraise() Jul 16 21:08:08.082568 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 16 21:08:08.082568 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise self.value Jul 16 21:08:08.082568 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 186, in authorize Jul 16 21:08:08.082568 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi result = _ENFORCER.authorize(action, target, context, Jul 16 21:08:08.082568 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1184, in authorize Jul 16 21:08:08.082568 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return self.enforce( Jul 16 21:08:08.082568 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1088, in enforce Jul 16 21:08:08.082568 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc(*args, **kwargs) Jul 16 21:08:08.082568 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.PolicyNotAuthorized: Policy doesn't allow os_compute_api:os-aggregates:create to be performed. Jul 16 21:08:08.082568 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:08:08.082568 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:08:08.082568 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:08:08.082568 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:08:08.082568 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:08:08.082568 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:08:08.082568 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 354, in __exit__ Jul 16 21:08:08.082568 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(webob.exc.HTTPForbidden( Jul 16 21:08:08.082568 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Policy doesn't allow os_compute_api:os-aggregates:create to be performed. Jul 16 21:08:08.083804 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:08:08.083804 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-f1420da1-9cf4-4322-a142-efa9a3b94765 tempest-AggregatesAdminNegativeTestJSON-1271440314 tempest-AggregatesAdminNegativeTestJSON-1271440314-project] Returning 403 to user: Policy doesn't allow os_compute_api:os-aggregates:create to be performed. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:08:08.083804 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-f1420da1-9cf4-4322-a142-efa9a3b94765 tempest-AggregatesAdminNegativeTestJSON-1271440314 tempest-AggregatesAdminNegativeTestJSON-1271440314-project] 10.222.0.16 "POST /compute/v2.1/os-aggregates" status: 403 len: 116 microversion: 2.1 time: 0.011155 Jul 16 21:08:08.084720 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 19/38] 10.222.0.16 () {60 vars in 1163 bytes} [Sat Jul 16 21:08:08 2022] POST /compute/v2.1/os-aggregates => generated 116 bytes in 17 msecs (HTTP/1.1 403) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:08:08.090215 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-6600e69a-b024-4535-bbb1-8127916e9ce6 tempest-AggregatesAdminNegativeTestJSON-578260511 tempest-AggregatesAdminNegativeTestJSON-578260511-project-admin] Action: 'create', calling method: >, body: {"aggregate": {"name": "tempest-test_aggregate-1033759764"}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:08:08.097475 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.objects.aggregate [None req-6600e69a-b024-4535-bbb1-8127916e9ce6 tempest-AggregatesAdminNegativeTestJSON-578260511 tempest-AggregatesAdminNegativeTestJSON-578260511-project-admin] Generated uuid def900f2-90ea-4498-b107-659ee4922010 for aggregate {{(pid=138680) create /opt/stack/nova/nova/objects/aggregate.py:298}} Jul 16 21:08:08.218589 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-6600e69a-b024-4535-bbb1-8127916e9ce6 tempest-AggregatesAdminNegativeTestJSON-578260511 tempest-AggregatesAdminNegativeTestJSON-578260511-project-admin] 10.222.0.16 "POST /compute/v2.1/os-aggregates" status: 200 len: 198 microversion: 2.1 time: 0.132201 Jul 16 21:08:08.220785 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 20/39] 10.222.0.16 () {60 vars in 1163 bytes} [Sat Jul 16 21:08:08 2022] POST /compute/v2.1/os-aggregates => generated 198 bytes in 135 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:08:08.231065 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-69f4b3bd-46ac-4095-a960-a0b44855dea1 tempest-AggregatesAdminNegativeTestJSON-578260511 tempest-AggregatesAdminNegativeTestJSON-578260511-project-admin] Action: 'create', calling method: >, body: {"aggregate": {"name": "tempest-test_aggregate-1033759764"}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:08:08.232414 ncdv-master-1 devstack@n-api.service[138680]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-aggregates:create": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:08:08.232414 ncdv-master-1 devstack@n-api.service[138680]: warnings.warn(msg) Jul 16 21:08:08.233384 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.objects.aggregate [None req-69f4b3bd-46ac-4095-a960-a0b44855dea1 tempest-AggregatesAdminNegativeTestJSON-578260511 tempest-AggregatesAdminNegativeTestJSON-578260511-project-admin] Generated uuid 68d51a40-1927-4de5-98db-41f9683ade65 for aggregate {{(pid=138680) create /opt/stack/nova/nova/objects/aggregate.py:298}} Jul 16 21:08:08.245078 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-69f4b3bd-46ac-4095-a960-a0b44855dea1 tempest-AggregatesAdminNegativeTestJSON-578260511 tempest-AggregatesAdminNegativeTestJSON-578260511-project-admin] HTTP exception thrown: Aggregate tempest-test_aggregate-1033759764 already exists. Jul 16 21:08:08.247646 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-69f4b3bd-46ac-4095-a960-a0b44855dea1 tempest-AggregatesAdminNegativeTestJSON-578260511 tempest-AggregatesAdminNegativeTestJSON-578260511-project-admin] Request method failure captured: Jul 16 21:08:08.247646 ncdv-master-1 devstack@n-api.service[138680]: request: POST /compute/v2.1/os-aggregates HTTP/1.1 Jul 16 21:08:08.247646 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:08:08.247646 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:08:08.247646 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:08:08.247646 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 60 Jul 16 21:08:08.247646 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:08:08.247646 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:08:08.247646 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:08:08.247646 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:08:08.247646 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0yiz0unej4HpWPJrVBlkA1u9-AWCKDSpHymJUf5DT-hiCYEGZJzMCi_CgXajssbkUJoCwaGruhY3wft9OVrurmz0_pRbvWF03aApzlFrXwt0GhLH6-kKTcDhg1N2k0bLKCTgcUoad_Ma5kdyMmhxUVTCB2mCAuoF3iPjsftSucch2dg Jul 16 21:08:08.247646 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:08:08.247646 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:08:08.247646 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:08:08.247646 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:08:08.247646 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:08:08.247646 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:08:08.247646 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: b6be58dcb13b4cec8f0bfc58f0080173 Jul 16 21:08:08.247646 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-AggregatesAdminNegativeTestJSON-578260511 Jul 16 21:08:08.247646 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,admin,reader Jul 16 21:08:08.247646 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,admin,reader Jul 16 21:08:08.248762 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/b6be58dcb13b4cec8f0bfc58f0080173"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/b6be58dcb13b4cec8f0bfc58f0080173"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_b6be58dcb13b4cec8f0bfc58f0080173"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/b6be58dcb13b4cec8f0bfc58f0080173"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:08:08.248762 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-AggregatesAdminNegativeTestJSON-578260511 Jul 16 21:08:08.248762 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: b6be58dcb13b4cec8f0bfc58f0080173 Jul 16 21:08:08.248762 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-AggregatesAdminNegativeTestJSON-578260511 Jul 16 21:08:08.248762 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-AggregatesAdminNegativeTestJSON-578260511-project-admin Jul 16 21:08:08.248762 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:08:08.248762 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:08:08.248762 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: cadb63bb5d6b4d7597987780f7d9ef21 Jul 16 21:08:08.248762 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-AggregatesAdminNegativeTestJSON-578260511-project-admin Jul 16 21:08:08.248762 ncdv-master-1 devstack@n-api.service[138680]: Jul 16 21:08:08.248762 ncdv-master-1 devstack@n-api.service[138680]: {"aggregate": {"name": "tempest-test_aggregate-1033759764"}} Jul 16 21:08:08.248762 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 21:08:08.248762 ncdv-master-1 devstack@n-api.service[138680]: exception: Aggregate tempest-test_aggregate-1033759764 already exists. Jul 16 21:08:08.249459 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'body': {'aggregate': {'name': 'tempest-test_aggregate-1033759764'}}} Jul 16 21:08:08.249459 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:08:08.249459 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:08:08.249459 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/aggregates.py", line 78, in create Jul 16 21:08:08.249459 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi aggregate = self.api.create_aggregate(context, name, avail_zone) Jul 16 21:08:08.249459 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/exception_wrapper.py", line 71, in wrapped Jul 16 21:08:08.249459 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi _emit_versioned_exception_notification( Jul 16 21:08:08.249459 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 16 21:08:08.249459 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi self.force_reraise() Jul 16 21:08:08.249459 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 16 21:08:08.249459 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise self.value Jul 16 21:08:08.249459 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/exception_wrapper.py", line 63, in wrapped Jul 16 21:08:08.249459 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(self, context, *args, **kw) Jul 16 21:08:08.249459 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 6330, in create_aggregate Jul 16 21:08:08.249459 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi aggregate.create() Jul 16 21:08:08.249459 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 226, in wrapper Jul 16 21:08:08.249459 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return fn(self, *args, **kwargs) Jul 16 21:08:08.249459 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/aggregate.py", line 310, in create Jul 16 21:08:08.249459 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi db_aggregate = _aggregate_create_in_db(self._context, updates, Jul 16 21:08:08.251276 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1010, in wrapper Jul 16 21:08:08.251276 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return fn(*args, **kwargs) Jul 16 21:08:08.251276 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/aggregate.py", line 175, in _aggregate_create_in_db Jul 16 21:08:08.251276 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.AggregateNameExists(aggregate_name=values['name']) Jul 16 21:08:08.251276 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.AggregateNameExists: Aggregate tempest-test_aggregate-1033759764 already exists. Jul 16 21:08:08.251276 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:08:08.251276 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:08:08.251276 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:08:08.251276 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:08:08.251276 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:08:08.251276 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:08:08.251276 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:08:08.251276 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:08:08.251276 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:08:08.251276 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:08:08.251276 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 16 21:08:08.251276 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 16 21:08:08.251276 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 16 21:08:08.251276 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 16 21:08:08.251276 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/aggregates.py", line 80, in create Jul 16 21:08:08.252817 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPConflict(explanation=e.format_message()) Jul 16 21:08:08.252817 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPConflict: Aggregate tempest-test_aggregate-1033759764 already exists. Jul 16 21:08:08.252817 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:08:08.252817 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:08:08.252817 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:08:08.252817 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:08:08.252817 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:08:08.252817 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:08:08.252817 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:08:08.252817 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:08:08.252817 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Aggregate tempest-test_aggregate-1033759764 already exists. Jul 16 21:08:08.252817 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:08:08.252817 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-69f4b3bd-46ac-4095-a960-a0b44855dea1 tempest-AggregatesAdminNegativeTestJSON-578260511 tempest-AggregatesAdminNegativeTestJSON-578260511-project-admin] Returning 409 to user: Aggregate tempest-test_aggregate-1033759764 already exists. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:08:08.252817 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-69f4b3bd-46ac-4095-a960-a0b44855dea1 tempest-AggregatesAdminNegativeTestJSON-578260511 tempest-AggregatesAdminNegativeTestJSON-578260511-project-admin] 10.222.0.16 "POST /compute/v2.1/os-aggregates" status: 409 len: 111 microversion: 2.1 time: 0.022993 Jul 16 21:08:08.254461 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 21/40] 10.222.0.16 () {60 vars in 1163 bytes} [Sat Jul 16 21:08:08 2022] POST /compute/v2.1/os-aggregates => generated 111 bytes in 24 msecs (HTTP/1.1 409) 9 headers in 378 bytes (2 switches on core 0) Jul 16 21:08:08.259696 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-2c1d7589-c189-40a4-afcf-be1c87fba907 tempest-AggregatesAdminNegativeTestJSON-578260511 tempest-AggregatesAdminNegativeTestJSON-578260511-project-admin] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:08:08.260468 ncdv-master-1 devstack@n-api.service[138680]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-aggregates:delete": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:08:08.260468 ncdv-master-1 devstack@n-api.service[138680]: warnings.warn(msg) Jul 16 21:08:08.350161 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-2c1d7589-c189-40a4-afcf-be1c87fba907 tempest-AggregatesAdminNegativeTestJSON-578260511 tempest-AggregatesAdminNegativeTestJSON-578260511-project-admin] 10.222.0.16 "DELETE /compute/v2.1/os-aggregates/5" status: 200 len: 0 microversion: 2.1 time: 0.095077 Jul 16 21:08:08.350813 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 22/41] 10.222.0.16 () {58 vars in 1151 bytes} [Sat Jul 16 21:08:08 2022] DELETE /compute/v2.1/os-aggregates/5 => generated 0 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 355 bytes (1 switches on core 0) Jul 16 21:08:08.362207 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-0cac077f-b9e1-413c-9b52-5b37e1ddcd7d tempest-AggregatesAdminNegativeTestJSON-578260511 tempest-AggregatesAdminNegativeTestJSON-578260511-project-admin] Action: 'create', calling method: >, body: {"aggregate": {"name": "tempest-test_aggregate-153555352"}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:08:08.363546 ncdv-master-1 devstack@n-api.service[138680]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-aggregates:create": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:08:08.363546 ncdv-master-1 devstack@n-api.service[138680]: warnings.warn(msg) Jul 16 21:08:08.364649 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.objects.aggregate [None req-0cac077f-b9e1-413c-9b52-5b37e1ddcd7d tempest-AggregatesAdminNegativeTestJSON-578260511 tempest-AggregatesAdminNegativeTestJSON-578260511-project-admin] Generated uuid e7be55ff-83f7-4c49-8b48-b91f242bd87e for aggregate {{(pid=138680) create /opt/stack/nova/nova/objects/aggregate.py:298}} Jul 16 21:08:08.470870 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-3d7a7e6d-e0bc-4ac5-bc03-d85cf33672de tempest-AggregatesAdminTestV241-531976310 tempest-AggregatesAdminTestV241-531976310-project-admin] 10.222.0.16 "POST /compute/v2.1/os-aggregates/3/action" status: 200 len: 313 microversion: 2.41 time: 1.386590 Jul 16 21:08:08.472148 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 20/42] 10.222.0.16 () {62 vars in 1231 bytes} [Sat Jul 16 21:08:07 2022] POST /compute/v2.1/os-aggregates/3/action => generated 313 bytes in 1389 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Jul 16 21:08:08.508367 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-0cac077f-b9e1-413c-9b52-5b37e1ddcd7d tempest-AggregatesAdminNegativeTestJSON-578260511 tempest-AggregatesAdminNegativeTestJSON-578260511-project-admin] 10.222.0.16 "POST /compute/v2.1/os-aggregates" status: 200 len: 197 microversion: 2.1 time: 0.144941 Jul 16 21:08:08.508367 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 23/43] 10.222.0.16 () {60 vars in 1163 bytes} [Sat Jul 16 21:08:08 2022] POST /compute/v2.1/os-aggregates => generated 197 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:08:08.508367 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-af4238ee-068f-4390-8eef-9b27ebed0c17 tempest-AggregatesAdminTestV241-531976310 tempest-AggregatesAdminTestV241-531976310-project-admin] Action: 'action', calling method: >, body: {"remove_host": {"host": "nch1-master-1"}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:08:08.509270 ncdv-master-1 devstack@n-api.service[138680]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-aggregates:remove_host": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:08:08.509270 ncdv-master-1 devstack@n-api.service[138680]: warnings.warn(msg) Jul 16 21:08:08.517610 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-af4238ee-068f-4390-8eef-9b27ebed0c17 tempest-AggregatesAdminTestV241-531976310 tempest-AggregatesAdminTestV241-531976310-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:08:08.518220 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-af4238ee-068f-4390-8eef-9b27ebed0c17 tempest-AggregatesAdminTestV241-531976310 tempest-AggregatesAdminTestV241-531976310-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:08:08.857283 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-af4238ee-068f-4390-8eef-9b27ebed0c17 tempest-AggregatesAdminTestV241-531976310 tempest-AggregatesAdminTestV241-531976310-project-admin] 10.222.0.16 "POST /compute/v2.1/os-aggregates/3/action" status: 200 len: 298 microversion: 2.41 time: 0.345715 Jul 16 21:08:08.857283 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 24/44] 10.222.0.16 () {62 vars in 1231 bytes} [Sat Jul 16 21:08:08 2022] POST /compute/v2.1/os-aggregates/3/action => generated 298 bytes in 347 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 16 21:08:08.857283 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-8e3809db-fcc7-42d2-bf24-ee19a0570fca tempest-AggregatesAdminNegativeTestJSON-1271440314 tempest-AggregatesAdminNegativeTestJSON-1271440314-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:08:08.857283 ncdv-master-1 devstack@n-api.service[138680]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-aggregates:delete": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:08:08.857283 ncdv-master-1 devstack@n-api.service[138680]: warnings.warn(msg) Jul 16 21:08:08.857671 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-8e3809db-fcc7-42d2-bf24-ee19a0570fca tempest-AggregatesAdminNegativeTestJSON-1271440314 tempest-AggregatesAdminNegativeTestJSON-1271440314-project] Policy check for os_compute_api:os-aggregates:delete failed with credentials {'is_admin': False, 'user_id': '35a4905d463a4db2960294407f747f20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '49044125b0414a4ab0e2ab9eb7d66e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:08:08.858694 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-8e3809db-fcc7-42d2-bf24-ee19a0570fca tempest-AggregatesAdminNegativeTestJSON-1271440314 tempest-AggregatesAdminNegativeTestJSON-1271440314-project] Request method failure captured: Jul 16 21:08:08.858694 ncdv-master-1 devstack@n-api.service[138680]: request: DELETE /compute/v2.1/os-aggregates/6 HTTP/1.1 Jul 16 21:08:08.858694 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:08:08.858694 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:08:08.858694 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:08:08.858694 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 0 Jul 16 21:08:08.858694 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:08:08.858694 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:08:08.858694 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:08:08.858694 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:08:08.858694 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0yiwx37gJG3UPf6cs7N4UkgaksWm_yG7GacwHzT4X_Y36InoB6bLW8dxMeWTnPNUr5MkHyB6zCa2ShiCwvtwzG8FdDgsivxSANP9O8r63IrJOV2dcu5-Fla-kUEiqkTUeLu0mggNWQr19K_Q1rkm8xG96kdc5KRDvRt2-aIVSJpj_Ug Jul 16 21:08:08.858694 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:08:08.858694 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:08:08.858694 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:08:08.858694 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:08:08.858694 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:08:08.858694 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:08:08.858694 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: 49044125b0414a4ab0e2ab9eb7d66e6e Jul 16 21:08:08.858694 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-AggregatesAdminNegativeTestJSON-1271440314 Jul 16 21:08:08.858694 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:08:08.858694 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:08:08.859853 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/49044125b0414a4ab0e2ab9eb7d66e6e"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/49044125b0414a4ab0e2ab9eb7d66e6e"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_49044125b0414a4ab0e2ab9eb7d66e6e"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/49044125b0414a4ab0e2ab9eb7d66e6e"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:08:08.859853 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-AggregatesAdminNegativeTestJSON-1271440314 Jul 16 21:08:08.859853 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: 49044125b0414a4ab0e2ab9eb7d66e6e Jul 16 21:08:08.859853 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-AggregatesAdminNegativeTestJSON-1271440314 Jul 16 21:08:08.859853 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-AggregatesAdminNegativeTestJSON-1271440314-project Jul 16 21:08:08.859853 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:08:08.859853 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:08:08.859853 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: 35a4905d463a4db2960294407f747f20 Jul 16 21:08:08.859853 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-AggregatesAdminNegativeTestJSON-1271440314-project Jul 16 21:08:08.859853 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 21:08:08.859853 ncdv-master-1 devstack@n-api.service[138680]: exception: Policy doesn't allow os_compute_api:os-aggregates:delete to be performed. Jul 16 21:08:08.859853 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': '6'} Jul 16 21:08:08.860434 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:08:08.860434 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:08:08.860434 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:08:08.860434 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:08:08.860434 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:08:08.860434 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:08:08.860434 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:08:08.860434 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:08:08.860434 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/aggregates.py", line 147, in delete Jul 16 21:08:08.860434 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi context.can(aggr_policies.POLICY_ROOT % 'delete', target={}) Jul 16 21:08:08.860434 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/context.py", line 240, in can Jul 16 21:08:08.860434 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return policy.authorize(self, action, target) Jul 16 21:08:08.860434 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 199, in authorize Jul 16 21:08:08.860434 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi LOG.debug('Policy check for %(action)s failed with credentials ' Jul 16 21:08:08.860434 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 16 21:08:08.860434 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi self.force_reraise() Jul 16 21:08:08.860434 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 16 21:08:08.860434 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise self.value Jul 16 21:08:08.860434 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 186, in authorize Jul 16 21:08:08.861489 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi result = _ENFORCER.authorize(action, target, context, Jul 16 21:08:08.861489 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1184, in authorize Jul 16 21:08:08.861489 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return self.enforce( Jul 16 21:08:08.861489 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1088, in enforce Jul 16 21:08:08.861489 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc(*args, **kwargs) Jul 16 21:08:08.861489 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.PolicyNotAuthorized: Policy doesn't allow os_compute_api:os-aggregates:delete to be performed. Jul 16 21:08:08.861489 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:08:08.861489 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:08:08.861489 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:08:08.861489 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:08:08.861489 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:08:08.861489 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:08:08.861489 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 354, in __exit__ Jul 16 21:08:08.861489 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(webob.exc.HTTPForbidden( Jul 16 21:08:08.861489 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Policy doesn't allow os_compute_api:os-aggregates:delete to be performed. Jul 16 21:08:08.861489 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:08:08.861489 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-8e3809db-fcc7-42d2-bf24-ee19a0570fca tempest-AggregatesAdminNegativeTestJSON-1271440314 tempest-AggregatesAdminNegativeTestJSON-1271440314-project] Returning 403 to user: Policy doesn't allow os_compute_api:os-aggregates:delete to be performed. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:08:08.862769 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-8e3809db-fcc7-42d2-bf24-ee19a0570fca tempest-AggregatesAdminNegativeTestJSON-1271440314 tempest-AggregatesAdminNegativeTestJSON-1271440314-project] 10.222.0.16 "DELETE /compute/v2.1/os-aggregates/6" status: 403 len: 116 microversion: 2.1 time: 0.008692 Jul 16 21:08:08.863844 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 25/45] 10.222.0.16 () {58 vars in 1151 bytes} [Sat Jul 16 21:08:08 2022] DELETE /compute/v2.1/os-aggregates/6 => generated 116 bytes in 13 msecs (HTTP/1.1 403) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:08:08.869342 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-7c33754e-17a9-4054-aef6-2af6925b9048 tempest-AggregatesAdminTestV241-531976310 tempest-AggregatesAdminTestV241-531976310-project-admin] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:08:08.881269 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_api:SELECT=85,nova_api:INSERT=7,nova_api:DELETE=8 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:08:09.019482 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-7c33754e-17a9-4054-aef6-2af6925b9048 tempest-AggregatesAdminTestV241-531976310 tempest-AggregatesAdminTestV241-531976310-project-admin] 10.222.0.16 "DELETE /compute/v2.1/os-aggregates/3" status: 200 len: 0 microversion: 2.41 time: 0.154078 Jul 16 21:08:09.020782 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 26/46] 10.222.0.16 () {60 vars in 1192 bytes} [Sat Jul 16 21:08:08 2022] DELETE /compute/v2.1/os-aggregates/3 => generated 0 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:08:09.026078 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-81b3d80d-301a-48ea-98f5-7e96de0397e1 tempest-AggregatesAdminNegativeTestJSON-578260511 tempest-AggregatesAdminNegativeTestJSON-578260511-project-admin] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:08:09.026841 ncdv-master-1 devstack@n-api.service[138680]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-aggregates:delete": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:08:09.026841 ncdv-master-1 devstack@n-api.service[138680]: warnings.warn(msg) Jul 16 21:08:09.137200 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-81b3d80d-301a-48ea-98f5-7e96de0397e1 tempest-AggregatesAdminNegativeTestJSON-578260511 tempest-AggregatesAdminNegativeTestJSON-578260511-project-admin] 10.222.0.16 "DELETE /compute/v2.1/os-aggregates/6" status: 200 len: 0 microversion: 2.1 time: 0.115169 Jul 16 21:08:09.137793 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 27/47] 10.222.0.16 () {58 vars in 1151 bytes} [Sat Jul 16 21:08:09 2022] DELETE /compute/v2.1/os-aggregates/6 => generated 0 bytes in 116 msecs (HTTP/1.1 200) 9 headers in 355 bytes (1 switches on core 0) Jul 16 21:08:09.143331 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-ad9cd963-a1a0-4bf9-852a-d2af3561aa45 tempest-AggregatesAdminTestJSON-1464138310 tempest-AggregatesAdminTestJSON-1464138310-project-admin] Action: 'create', calling method: >, body: {"aggregate": {"availability_zone": "tempest-test_az-2083597543", "name": "tempest-test_aggregate-2105494649"}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:08:09.163224 ncdv-master-1 devstack@n-api.service[138680]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-aggregates:create": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:08:09.163224 ncdv-master-1 devstack@n-api.service[138680]: warnings.warn(msg) Jul 16 21:08:09.164315 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.objects.aggregate [None req-ad9cd963-a1a0-4bf9-852a-d2af3561aa45 tempest-AggregatesAdminTestJSON-1464138310 tempest-AggregatesAdminTestJSON-1464138310-project-admin] Generated uuid 777e98ff-8cc7-4aaf-bfc6-a764234bd65b for aggregate {{(pid=138680) create /opt/stack/nova/nova/objects/aggregate.py:298}} Jul 16 21:08:09.189570 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-63a41d4c-ecc2-4fe0-9555-d6cf0b510ac1 tempest-HostsAdminNegativeTestJSON-1188407738 tempest-HostsAdminNegativeTestJSON-1188407738-project-admin] Calling method '.version_select at 0x7fdd9ee3d3a0>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:08:09.195507 ncdv-master-1 devstack@n-api.service[138681]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-hosts:list": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:08:09.195507 ncdv-master-1 devstack@n-api.service[138681]: warnings.warn(msg) Jul 16 21:08:09.197112 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-63a41d4c-ecc2-4fe0-9555-d6cf0b510ac1 tempest-HostsAdminNegativeTestJSON-1188407738 tempest-HostsAdminNegativeTestJSON-1188407738-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:08:09.197713 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-63a41d4c-ecc2-4fe0-9555-d6cf0b510ac1 tempest-HostsAdminNegativeTestJSON-1188407738 tempest-HostsAdminNegativeTestJSON-1188407738-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:08:09.198749 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-63a41d4c-ecc2-4fe0-9555-d6cf0b510ac1 tempest-HostsAdminNegativeTestJSON-1188407738 tempest-HostsAdminNegativeTestJSON-1188407738-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:08:09.199364 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-63a41d4c-ecc2-4fe0-9555-d6cf0b510ac1 tempest-HostsAdminNegativeTestJSON-1188407738 tempest-HostsAdminNegativeTestJSON-1188407738-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:08:09.271219 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-63a41d4c-ecc2-4fe0-9555-d6cf0b510ac1 tempest-HostsAdminNegativeTestJSON-1188407738 tempest-HostsAdminNegativeTestJSON-1188407738-project-admin] 10.222.0.16 "GET /compute/v2.1/os-hosts" status: 200 len: 379 microversion: 2.1 time: 0.797571 Jul 16 21:08:09.271813 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 21/48] 10.222.0.16 () {58 vars in 1127 bytes} [Sat Jul 16 21:08:08 2022] GET /compute/v2.1/os-hosts => generated 379 bytes in 799 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:08:09.278561 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-87519b34-5fc7-44eb-945b-a8a0755f46d4 tempest-AggregatesAdminNegativeTestJSON-578260511 tempest-AggregatesAdminNegativeTestJSON-578260511-project-admin] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:08:09.279375 ncdv-master-1 devstack@n-api.service[138681]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-aggregates:delete": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:08:09.279375 ncdv-master-1 devstack@n-api.service[138681]: warnings.warn(msg) Jul 16 21:08:09.284435 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-ad9cd963-a1a0-4bf9-852a-d2af3561aa45 tempest-AggregatesAdminTestJSON-1464138310 tempest-AggregatesAdminTestJSON-1464138310-project-admin] 10.222.0.16 "POST /compute/v2.1/os-aggregates" status: 200 len: 222 microversion: 2.1 time: 0.145505 Jul 16 21:08:09.285209 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 28/49] 10.222.0.16 () {60 vars in 1164 bytes} [Sat Jul 16 21:08:09 2022] POST /compute/v2.1/os-aggregates => generated 222 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:08:09.294028 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-87519b34-5fc7-44eb-945b-a8a0755f46d4 tempest-AggregatesAdminNegativeTestJSON-578260511 tempest-AggregatesAdminNegativeTestJSON-578260511-project-admin] HTTP exception thrown: Aggregate -1 could not be found. Jul 16 21:08:09.296461 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-87519b34-5fc7-44eb-945b-a8a0755f46d4 tempest-AggregatesAdminNegativeTestJSON-578260511 tempest-AggregatesAdminNegativeTestJSON-578260511-project-admin] Request method failure captured: Jul 16 21:08:09.296461 ncdv-master-1 devstack@n-api.service[138681]: request: DELETE /compute/v2.1/os-aggregates/-1 HTTP/1.1 Jul 16 21:08:09.296461 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:08:09.296461 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:08:09.296461 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:08:09.296461 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 21:08:09.296461 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:08:09.296461 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:08:09.296461 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:08:09.296461 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:08:09.296461 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0yiz0unej4HpWPJrVBlkA1u9-AWCKDSpHymJUf5DT-hiCYEGZJzMCi_CgXajssbkUJoCwaGruhY3wft9OVrurmz0_pRbvWF03aApzlFrXwt0GhLH6-kKTcDhg1N2k0bLKCTgcUoad_Ma5kdyMmhxUVTCB2mCAuoF3iPjsftSucch2dg Jul 16 21:08:09.296461 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:08:09.296461 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:08:09.296461 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:08:09.296461 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:08:09.296461 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:08:09.296461 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:08:09.296461 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: b6be58dcb13b4cec8f0bfc58f0080173 Jul 16 21:08:09.296461 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-AggregatesAdminNegativeTestJSON-578260511 Jul 16 21:08:09.296461 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,admin,reader Jul 16 21:08:09.296461 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,admin,reader Jul 16 21:08:09.297534 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/b6be58dcb13b4cec8f0bfc58f0080173"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/b6be58dcb13b4cec8f0bfc58f0080173"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_b6be58dcb13b4cec8f0bfc58f0080173"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/b6be58dcb13b4cec8f0bfc58f0080173"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:08:09.297534 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-AggregatesAdminNegativeTestJSON-578260511 Jul 16 21:08:09.297534 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: b6be58dcb13b4cec8f0bfc58f0080173 Jul 16 21:08:09.297534 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-AggregatesAdminNegativeTestJSON-578260511 Jul 16 21:08:09.297534 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-AggregatesAdminNegativeTestJSON-578260511-project-admin Jul 16 21:08:09.297534 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:08:09.297534 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:08:09.297534 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: cadb63bb5d6b4d7597987780f7d9ef21 Jul 16 21:08:09.297534 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-AggregatesAdminNegativeTestJSON-578260511-project-admin Jul 16 21:08:09.297534 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:08:09.297534 ncdv-master-1 devstack@n-api.service[138681]: exception: Aggregate -1 could not be found. Jul 16 21:08:09.297534 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': '-1'} Jul 16 21:08:09.298127 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:08:09.298127 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:08:09.298127 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/aggregates.py", line 155, in delete Jul 16 21:08:09.298127 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi self.api.delete_aggregate(context, id) Jul 16 21:08:09.298127 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/exception_wrapper.py", line 71, in wrapped Jul 16 21:08:09.298127 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi _emit_versioned_exception_notification( Jul 16 21:08:09.298127 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 16 21:08:09.298127 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi self.force_reraise() Jul 16 21:08:09.298127 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 16 21:08:09.298127 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise self.value Jul 16 21:08:09.298127 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/exception_wrapper.py", line 63, in wrapped Jul 16 21:08:09.298127 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(self, context, *args, **kw) Jul 16 21:08:09.298127 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 6389, in delete_aggregate Jul 16 21:08:09.298127 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi aggregate = objects.Aggregate.get_by_id(context, aggregate_id) Jul 16 21:08:09.298127 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Jul 16 21:08:09.298127 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Jul 16 21:08:09.298127 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/aggregate.py", line 274, in get_by_id Jul 16 21:08:09.298127 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi db_aggregate = _aggregate_get_from_db(context, aggregate_id) Jul 16 21:08:09.298127 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1010, in wrapper Jul 16 21:08:09.299645 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return fn(*args, **kwargs) Jul 16 21:08:09.299645 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/aggregate.py", line 45, in _aggregate_get_from_db Jul 16 21:08:09.299645 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.AggregateNotFound(aggregate_id=aggregate_id) Jul 16 21:08:09.299645 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.AggregateNotFound: Aggregate -1 could not be found. Jul 16 21:08:09.299645 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:08:09.299645 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:08:09.299645 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:08:09.299645 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:08:09.299645 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:08:09.299645 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:08:09.299645 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:08:09.299645 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:08:09.299645 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:08:09.299645 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:08:09.299645 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/aggregates.py", line 157, in delete Jul 16 21:08:09.299645 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:08:09.299645 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Aggregate -1 could not be found. Jul 16 21:08:09.299645 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:08:09.299645 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:08:09.299645 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:08:09.299645 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:08:09.300700 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:08:09.300700 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:08:09.300700 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:08:09.300700 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:08:09.300700 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Aggregate -1 could not be found. Jul 16 21:08:09.300700 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:08:09.300700 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-87519b34-5fc7-44eb-945b-a8a0755f46d4 tempest-AggregatesAdminNegativeTestJSON-578260511 tempest-AggregatesAdminNegativeTestJSON-578260511-project-admin] Returning 404 to user: Aggregate -1 could not be found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:08:09.300700 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-87519b34-5fc7-44eb-945b-a8a0755f46d4 tempest-AggregatesAdminNegativeTestJSON-578260511 tempest-AggregatesAdminNegativeTestJSON-578260511-project-admin] 10.222.0.16 "DELETE /compute/v2.1/os-aggregates/-1" status: 404 len: 78 microversion: 2.1 time: 0.024906 Jul 16 21:08:09.300700 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 22/50] 10.222.0.16 () {58 vars in 1154 bytes} [Sat Jul 16 21:08:09 2022] DELETE /compute/v2.1/os-aggregates/-1 => generated 78 bytes in 26 msecs (HTTP/1.1 404) 9 headers in 378 bytes (1 switches on core 0) Jul 16 21:08:09.305376 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-a655253d-89ac-4346-95e8-b3e4dbe69fdd tempest-AggregatesAdminTestJSON-1464138310 tempest-AggregatesAdminTestJSON-1464138310-project-admin] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:08:09.306356 ncdv-master-1 devstack@n-api.service[138681]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-aggregates:index": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:08:09.306356 ncdv-master-1 devstack@n-api.service[138681]: warnings.warn(msg) Jul 16 21:08:09.317792 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-a655253d-89ac-4346-95e8-b3e4dbe69fdd tempest-AggregatesAdminTestJSON-1464138310 tempest-AggregatesAdminTestJSON-1464138310-project-admin] 10.222.0.16 "GET /compute/v2.1/os-aggregates" status: 200 len: 303 microversion: 2.1 time: 0.018317 Jul 16 21:08:09.318289 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 23/51] 10.222.0.16 () {58 vars in 1142 bytes} [Sat Jul 16 21:08:09 2022] GET /compute/v2.1/os-aggregates => generated 303 bytes in 19 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:08:09.322822 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-70fb2f2d-85b6-4313-b665-eb05a0ff7de8 tempest-AggregatesAdminNegativeTestJSON-578260511 tempest-AggregatesAdminNegativeTestJSON-578260511-project-admin] Action: 'create', calling method: >, body: {"aggregate": {"name": "tempest-test_aggregate-754348076"}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:08:09.324312 ncdv-master-1 devstack@n-api.service[138681]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-aggregates:create": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:08:09.324312 ncdv-master-1 devstack@n-api.service[138681]: warnings.warn(msg) Jul 16 21:08:09.325312 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.objects.aggregate [None req-70fb2f2d-85b6-4313-b665-eb05a0ff7de8 tempest-AggregatesAdminNegativeTestJSON-578260511 tempest-AggregatesAdminNegativeTestJSON-578260511-project-admin] Generated uuid 77bc57fa-fad3-4f32-92bc-bc6090ccaa9d for aggregate {{(pid=138681) create /opt/stack/nova/nova/objects/aggregate.py:298}} Jul 16 21:08:09.468467 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-70fb2f2d-85b6-4313-b665-eb05a0ff7de8 tempest-AggregatesAdminNegativeTestJSON-578260511 tempest-AggregatesAdminNegativeTestJSON-578260511-project-admin] 10.222.0.16 "POST /compute/v2.1/os-aggregates" status: 200 len: 197 microversion: 2.1 time: 0.148975 Jul 16 21:08:09.470085 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 24/52] 10.222.0.16 () {60 vars in 1163 bytes} [Sat Jul 16 21:08:09 2022] POST /compute/v2.1/os-aggregates => generated 197 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:08:09.474939 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-66a584f2-d5d5-4103-b11b-3cbd54070a3c tempest-AggregatesAdminTestJSON-1464138310 tempest-AggregatesAdminTestJSON-1464138310-project-admin] Action: 'action', calling method: >, body: {"add_host": {"host": "nch1-master-1"}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:08:09.476244 ncdv-master-1 devstack@n-api.service[138681]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-aggregates:add_host": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:08:09.476244 ncdv-master-1 devstack@n-api.service[138681]: warnings.warn(msg) Jul 16 21:08:09.484681 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-66a584f2-d5d5-4103-b11b-3cbd54070a3c tempest-AggregatesAdminTestJSON-1464138310 tempest-AggregatesAdminTestJSON-1464138310-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:08:09.485133 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-66a584f2-d5d5-4103-b11b-3cbd54070a3c tempest-AggregatesAdminTestJSON-1464138310 tempest-AggregatesAdminTestJSON-1464138310-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:08:09.492474 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_api:SELECT=84,nova_api:INSERT=6,nova_api:DELETE=9,nova_api:UPDATE=1 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:08:09.928689 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-66a584f2-d5d5-4103-b11b-3cbd54070a3c tempest-AggregatesAdminTestJSON-1464138310 tempest-AggregatesAdminTestJSON-1464138310-project-admin] 10.222.0.16 "POST /compute/v2.1/os-aggregates/7/action" status: 200 len: 315 microversion: 2.1 time: 0.457473 Jul 16 21:08:09.943061 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 25/53] 10.222.0.16 () {60 vars in 1190 bytes} [Sat Jul 16 21:08:09 2022] POST /compute/v2.1/os-aggregates/7/action => generated 315 bytes in 459 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:08:09.947812 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-baacdf52-585d-4bd5-9ee6-4916b0dd1764 tempest-AggregatesAdminNegativeTestJSON-1271440314 tempest-AggregatesAdminNegativeTestJSON-1271440314-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:08:09.948785 ncdv-master-1 devstack@n-api.service[138681]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-aggregates:show": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:08:09.948785 ncdv-master-1 devstack@n-api.service[138681]: warnings.warn(msg) Jul 16 21:08:09.950187 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-baacdf52-585d-4bd5-9ee6-4916b0dd1764 tempest-AggregatesAdminNegativeTestJSON-1271440314 tempest-AggregatesAdminNegativeTestJSON-1271440314-project] Policy check for os_compute_api:os-aggregates:show failed with credentials {'is_admin': False, 'user_id': '35a4905d463a4db2960294407f747f20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '49044125b0414a4ab0e2ab9eb7d66e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:08:09.951846 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-baacdf52-585d-4bd5-9ee6-4916b0dd1764 tempest-AggregatesAdminNegativeTestJSON-1271440314 tempest-AggregatesAdminNegativeTestJSON-1271440314-project] Request method failure captured: Jul 16 21:08:09.951846 ncdv-master-1 devstack@n-api.service[138681]: request: GET /compute/v2.1/os-aggregates/8 HTTP/1.1 Jul 16 21:08:09.951846 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:08:09.951846 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:08:09.951846 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:08:09.951846 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 21:08:09.951846 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:08:09.951846 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:08:09.951846 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:08:09.951846 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:08:09.951846 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0yiwx37gJG3UPf6cs7N4UkgaksWm_yG7GacwHzT4X_Y36InoB6bLW8dxMeWTnPNUr5MkHyB6zCa2ShiCwvtwzG8FdDgsivxSANP9O8r63IrJOV2dcu5-Fla-kUEiqkTUeLu0mggNWQr19K_Q1rkm8xG96kdc5KRDvRt2-aIVSJpj_Ug Jul 16 21:08:09.951846 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:08:09.951846 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:08:09.951846 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:08:09.951846 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:08:09.951846 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:08:09.951846 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:08:09.951846 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: 49044125b0414a4ab0e2ab9eb7d66e6e Jul 16 21:08:09.951846 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-AggregatesAdminNegativeTestJSON-1271440314 Jul 16 21:08:09.951846 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:08:09.951846 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:08:09.952960 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/49044125b0414a4ab0e2ab9eb7d66e6e"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/49044125b0414a4ab0e2ab9eb7d66e6e"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_49044125b0414a4ab0e2ab9eb7d66e6e"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/49044125b0414a4ab0e2ab9eb7d66e6e"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:08:09.952960 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-AggregatesAdminNegativeTestJSON-1271440314 Jul 16 21:08:09.952960 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: 49044125b0414a4ab0e2ab9eb7d66e6e Jul 16 21:08:09.952960 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-AggregatesAdminNegativeTestJSON-1271440314 Jul 16 21:08:09.952960 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-AggregatesAdminNegativeTestJSON-1271440314-project Jul 16 21:08:09.952960 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:08:09.952960 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:08:09.952960 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: 35a4905d463a4db2960294407f747f20 Jul 16 21:08:09.952960 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-AggregatesAdminNegativeTestJSON-1271440314-project Jul 16 21:08:09.952960 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:08:09.952960 ncdv-master-1 devstack@n-api.service[138681]: exception: Policy doesn't allow os_compute_api:os-aggregates:show to be performed. Jul 16 21:08:09.952960 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': '8'} Jul 16 21:08:09.953573 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:08:09.953573 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:08:09.953573 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:08:09.953573 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:08:09.953573 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:08:09.953573 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:08:09.953573 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:08:09.953573 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:08:09.953573 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/aggregates.py", line 100, in show Jul 16 21:08:09.953573 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi context.can(aggr_policies.POLICY_ROOT % 'show', target={}) Jul 16 21:08:09.953573 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/context.py", line 240, in can Jul 16 21:08:09.953573 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return policy.authorize(self, action, target) Jul 16 21:08:09.953573 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 199, in authorize Jul 16 21:08:09.953573 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi LOG.debug('Policy check for %(action)s failed with credentials ' Jul 16 21:08:09.953573 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 16 21:08:09.953573 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi self.force_reraise() Jul 16 21:08:09.953573 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 16 21:08:09.953573 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise self.value Jul 16 21:08:09.953573 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 186, in authorize Jul 16 21:08:09.954486 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi result = _ENFORCER.authorize(action, target, context, Jul 16 21:08:09.954486 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1184, in authorize Jul 16 21:08:09.954486 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return self.enforce( Jul 16 21:08:09.954486 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1088, in enforce Jul 16 21:08:09.954486 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc(*args, **kwargs) Jul 16 21:08:09.954486 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.PolicyNotAuthorized: Policy doesn't allow os_compute_api:os-aggregates:show to be performed. Jul 16 21:08:09.954486 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:08:09.954486 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:08:09.954486 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:08:09.954486 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:08:09.954486 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:08:09.954486 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:08:09.954486 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 354, in __exit__ Jul 16 21:08:09.954486 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(webob.exc.HTTPForbidden( Jul 16 21:08:09.954486 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Policy doesn't allow os_compute_api:os-aggregates:show to be performed. Jul 16 21:08:09.954486 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:08:09.958091 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-baacdf52-585d-4bd5-9ee6-4916b0dd1764 tempest-AggregatesAdminNegativeTestJSON-1271440314 tempest-AggregatesAdminNegativeTestJSON-1271440314-project] Returning 403 to user: Policy doesn't allow os_compute_api:os-aggregates:show to be performed. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:08:09.959267 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-baacdf52-585d-4bd5-9ee6-4916b0dd1764 tempest-AggregatesAdminNegativeTestJSON-1271440314 tempest-AggregatesAdminNegativeTestJSON-1271440314-project] 10.222.0.16 "GET /compute/v2.1/os-aggregates/8" status: 403 len: 114 microversion: 2.1 time: 0.025319 Jul 16 21:08:09.959854 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 26/54] 10.222.0.16 () {58 vars in 1148 bytes} [Sat Jul 16 21:08:09 2022] GET /compute/v2.1/os-aggregates/8 => generated 114 bytes in 30 msecs (HTTP/1.1 403) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:08:09.977913 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-f3dec801-1aa1-43ef-aac6-491f66704184 tempest-AggregatesAdminNegativeTestJSON-578260511 tempest-AggregatesAdminNegativeTestJSON-578260511-project-admin] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:08:09.978783 ncdv-master-1 devstack@n-api.service[138681]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-aggregates:delete": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:08:09.978783 ncdv-master-1 devstack@n-api.service[138681]: warnings.warn(msg) Jul 16 21:08:09.998463 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-cbfdf186-1072-486f-b682-d2582b80c0ca tempest-HostsAdminNegativeTestJSON-1773216254 tempest-HostsAdminNegativeTestJSON-1773216254-project] Calling method '.version_select at 0x7fdd9df6eb80>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:08:09.999634 ncdv-master-1 devstack@n-api.service[138680]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-hosts:list": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:08:09.999634 ncdv-master-1 devstack@n-api.service[138680]: warnings.warn(msg) Jul 16 21:08:10.001079 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-cbfdf186-1072-486f-b682-d2582b80c0ca tempest-HostsAdminNegativeTestJSON-1773216254 tempest-HostsAdminNegativeTestJSON-1773216254-project] Policy check for os_compute_api:os-hosts:list failed with credentials {'is_admin': False, 'user_id': '9b61d9df6f51477f9e1c2072a03a073c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '964640b04de845039e4b41cda5f555f1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:08:10.009347 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-cbfdf186-1072-486f-b682-d2582b80c0ca tempest-HostsAdminNegativeTestJSON-1773216254 tempest-HostsAdminNegativeTestJSON-1773216254-project] Request method failure captured: Jul 16 21:08:10.009347 ncdv-master-1 devstack@n-api.service[138680]: request: GET /compute/v2.1/os-hosts HTTP/1.1 Jul 16 21:08:10.009347 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:08:10.009347 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:08:10.009347 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:08:10.009347 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 0 Jul 16 21:08:10.009347 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:08:10.009347 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:08:10.009347 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:08:10.009347 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:08:10.009347 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0yi13dQlIKkatjUrCe7DvLmujmbnviq_zuHjC9uzP2yiL7VHHGKkoVizD2sGeU719CNvsihfESqr0I1nPCCiV-fuL1q1OyNyD0kkegizPVQfmW6dBo4n9rHFKAyoAJjcO7WEXpG0OPIOkEasF-pn9GBJh_XdaUkbSC1p0Pr7n_bQijk Jul 16 21:08:10.009347 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:08:10.009347 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:08:10.009347 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:08:10.009347 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:08:10.009347 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:08:10.009347 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:08:10.009347 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: 964640b04de845039e4b41cda5f555f1 Jul 16 21:08:10.009347 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-HostsAdminNegativeTestJSON-1773216254 Jul 16 21:08:10.009347 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:08:10.009347 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:08:10.010492 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/964640b04de845039e4b41cda5f555f1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/964640b04de845039e4b41cda5f555f1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_964640b04de845039e4b41cda5f555f1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/964640b04de845039e4b41cda5f555f1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:08:10.010492 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-HostsAdminNegativeTestJSON-1773216254 Jul 16 21:08:10.010492 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: 964640b04de845039e4b41cda5f555f1 Jul 16 21:08:10.010492 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-HostsAdminNegativeTestJSON-1773216254 Jul 16 21:08:10.010492 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-HostsAdminNegativeTestJSON-1773216254-project Jul 16 21:08:10.010492 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:08:10.010492 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:08:10.010492 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: 9b61d9df6f51477f9e1c2072a03a073c Jul 16 21:08:10.010492 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-HostsAdminNegativeTestJSON-1773216254-project Jul 16 21:08:10.010492 ncdv-master-1 devstack@n-api.service[138680]: method: Jul 16 21:08:10.010492 ncdv-master-1 devstack@n-api.service[138680]: exception: Policy doesn't allow os_compute_api:os-hosts:list to be performed. Jul 16 21:08:10.010492 ncdv-master-1 devstack@n-api.service[138680]: action_args: {} Jul 16 21:08:10.010492 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:08:10.011158 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:08:10.011158 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:08:10.011158 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:08:10.011158 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:08:10.011158 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:08:10.011158 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Jul 16 21:08:10.011158 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Jul 16 21:08:10.011158 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper Jul 16 21:08:10.011158 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 16 21:08:10.011158 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:08:10.011158 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:08:10.011158 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/hosts.py", line 74, in index Jul 16 21:08:10.011158 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi context.can(hosts_policies.POLICY_NAME % 'list', Jul 16 21:08:10.011158 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/context.py", line 240, in can Jul 16 21:08:10.011158 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return policy.authorize(self, action, target) Jul 16 21:08:10.011158 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 199, in authorize Jul 16 21:08:10.011158 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi LOG.debug('Policy check for %(action)s failed with credentials ' Jul 16 21:08:10.011158 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 16 21:08:10.011158 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi self.force_reraise() Jul 16 21:08:10.012060 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 16 21:08:10.012060 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise self.value Jul 16 21:08:10.012060 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 186, in authorize Jul 16 21:08:10.012060 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi result = _ENFORCER.authorize(action, target, context, Jul 16 21:08:10.012060 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1184, in authorize Jul 16 21:08:10.012060 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return self.enforce( Jul 16 21:08:10.012060 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1088, in enforce Jul 16 21:08:10.012060 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc(*args, **kwargs) Jul 16 21:08:10.012060 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.PolicyNotAuthorized: Policy doesn't allow os_compute_api:os-hosts:list to be performed. Jul 16 21:08:10.012060 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:08:10.012060 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:08:10.012060 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:08:10.012060 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:08:10.012060 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:08:10.012060 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:08:10.012060 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 354, in __exit__ Jul 16 21:08:10.012060 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(webob.exc.HTTPForbidden( Jul 16 21:08:10.012060 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Policy doesn't allow os_compute_api:os-hosts:list to be performed. Jul 16 21:08:10.012060 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:08:10.014963 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-cbfdf186-1072-486f-b682-d2582b80c0ca tempest-HostsAdminNegativeTestJSON-1773216254 tempest-HostsAdminNegativeTestJSON-1773216254-project] Returning 403 to user: Policy doesn't allow os_compute_api:os-hosts:list to be performed. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:08:10.016129 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-cbfdf186-1072-486f-b682-d2582b80c0ca tempest-HostsAdminNegativeTestJSON-1773216254 tempest-HostsAdminNegativeTestJSON-1773216254-project] 10.222.0.16 "GET /compute/v2.1/os-hosts" status: 403 len: 109 microversion: 2.1 time: 0.729939 Jul 16 21:08:10.017338 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 29/55] 10.222.0.16 () {58 vars in 1127 bytes} [Sat Jul 16 21:08:09 2022] GET /compute/v2.1/os-hosts => generated 109 bytes in 731 msecs (HTTP/1.1 403) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:08:10.027590 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-e154eee7-dcd7-45b8-8799-411a4033e697 tempest-HostsAdminNegativeTestJSON-1773216254 tempest-HostsAdminNegativeTestJSON-1773216254-project] Calling method '.version_select at 0x7fdd9df29ee0>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:08:10.028188 ncdv-master-1 devstack@n-api.service[138680]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-hosts:reboot": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:08:10.028188 ncdv-master-1 devstack@n-api.service[138680]: warnings.warn(msg) Jul 16 21:08:10.029297 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-e154eee7-dcd7-45b8-8799-411a4033e697 tempest-HostsAdminNegativeTestJSON-1773216254 tempest-HostsAdminNegativeTestJSON-1773216254-project] Policy check for os_compute_api:os-hosts:reboot failed with credentials {'is_admin': False, 'user_id': '9b61d9df6f51477f9e1c2072a03a073c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '964640b04de845039e4b41cda5f555f1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:08:10.030837 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-e154eee7-dcd7-45b8-8799-411a4033e697 tempest-HostsAdminNegativeTestJSON-1773216254 tempest-HostsAdminNegativeTestJSON-1773216254-project] Request method failure captured: Jul 16 21:08:10.030837 ncdv-master-1 devstack@n-api.service[138680]: request: GET /compute/v2.1/os-hosts/ncdv-master-1/reboot HTTP/1.1 Jul 16 21:08:10.030837 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:08:10.030837 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:08:10.030837 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:08:10.030837 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 0 Jul 16 21:08:10.030837 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:08:10.030837 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:08:10.030837 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:08:10.030837 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:08:10.030837 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0yi13dQlIKkatjUrCe7DvLmujmbnviq_zuHjC9uzP2yiL7VHHGKkoVizD2sGeU719CNvsihfESqr0I1nPCCiV-fuL1q1OyNyD0kkegizPVQfmW6dBo4n9rHFKAyoAJjcO7WEXpG0OPIOkEasF-pn9GBJh_XdaUkbSC1p0Pr7n_bQijk Jul 16 21:08:10.030837 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:08:10.030837 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:08:10.030837 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:08:10.030837 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:08:10.030837 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:08:10.030837 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:08:10.030837 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: 964640b04de845039e4b41cda5f555f1 Jul 16 21:08:10.030837 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-HostsAdminNegativeTestJSON-1773216254 Jul 16 21:08:10.030837 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:08:10.030837 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:08:10.032097 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/964640b04de845039e4b41cda5f555f1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/964640b04de845039e4b41cda5f555f1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_964640b04de845039e4b41cda5f555f1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/964640b04de845039e4b41cda5f555f1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:08:10.032097 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-HostsAdminNegativeTestJSON-1773216254 Jul 16 21:08:10.032097 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: 964640b04de845039e4b41cda5f555f1 Jul 16 21:08:10.032097 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-HostsAdminNegativeTestJSON-1773216254 Jul 16 21:08:10.032097 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-HostsAdminNegativeTestJSON-1773216254-project Jul 16 21:08:10.032097 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:08:10.032097 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:08:10.032097 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: 9b61d9df6f51477f9e1c2072a03a073c Jul 16 21:08:10.032097 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-HostsAdminNegativeTestJSON-1773216254-project Jul 16 21:08:10.032097 ncdv-master-1 devstack@n-api.service[138680]: method: Jul 16 21:08:10.032097 ncdv-master-1 devstack@n-api.service[138680]: exception: Policy doesn't allow os_compute_api:os-hosts:reboot to be performed. Jul 16 21:08:10.032097 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': 'ncdv-master-1'} Jul 16 21:08:10.032097 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:08:10.032788 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:08:10.032788 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:08:10.032788 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:08:10.032788 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:08:10.032788 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:08:10.032788 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Jul 16 21:08:10.032788 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Jul 16 21:08:10.032788 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:08:10.032788 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:08:10.032788 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/hosts.py", line 203, in reboot Jul 16 21:08:10.032788 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi context.can(hosts_policies.POLICY_NAME % 'reboot', Jul 16 21:08:10.032788 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/context.py", line 240, in can Jul 16 21:08:10.032788 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return policy.authorize(self, action, target) Jul 16 21:08:10.032788 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 199, in authorize Jul 16 21:08:10.032788 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi LOG.debug('Policy check for %(action)s failed with credentials ' Jul 16 21:08:10.032788 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 16 21:08:10.032788 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi self.force_reraise() Jul 16 21:08:10.032788 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 16 21:08:10.032788 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise self.value Jul 16 21:08:10.034713 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 186, in authorize Jul 16 21:08:10.034713 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi result = _ENFORCER.authorize(action, target, context, Jul 16 21:08:10.034713 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1184, in authorize Jul 16 21:08:10.034713 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return self.enforce( Jul 16 21:08:10.034713 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1088, in enforce Jul 16 21:08:10.034713 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc(*args, **kwargs) Jul 16 21:08:10.034713 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.PolicyNotAuthorized: Policy doesn't allow os_compute_api:os-hosts:reboot to be performed. Jul 16 21:08:10.034713 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:08:10.034713 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:08:10.034713 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:08:10.034713 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:08:10.034713 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:08:10.034713 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:08:10.034713 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 354, in __exit__ Jul 16 21:08:10.034713 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(webob.exc.HTTPForbidden( Jul 16 21:08:10.034713 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Policy doesn't allow os_compute_api:os-hosts:reboot to be performed. Jul 16 21:08:10.034713 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:08:10.036003 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-e154eee7-dcd7-45b8-8799-411a4033e697 tempest-HostsAdminNegativeTestJSON-1773216254 tempest-HostsAdminNegativeTestJSON-1773216254-project] Returning 403 to user: Policy doesn't allow os_compute_api:os-hosts:reboot to be performed. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:08:10.036003 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-e154eee7-dcd7-45b8-8799-411a4033e697 tempest-HostsAdminNegativeTestJSON-1773216254 tempest-HostsAdminNegativeTestJSON-1773216254-project] 10.222.0.16 "GET /compute/v2.1/os-hosts/ncdv-master-1/reboot" status: 403 len: 111 microversion: 2.1 time: 0.009119 Jul 16 21:08:10.039237 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 30/56] 10.222.0.16 () {58 vars in 1190 bytes} [Sat Jul 16 21:08:10 2022] GET /compute/v2.1/os-hosts/ncdv-master-1/reboot => generated 111 bytes in 17 msecs (HTTP/1.1 403) 9 headers in 379 bytes (2 switches on core 0) Jul 16 21:08:10.045039 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-41fb19ec-b193-45e4-a81f-0dad73ba09b5 tempest-HostsAdminNegativeTestJSON-1188407738 tempest-HostsAdminNegativeTestJSON-1188407738-project-admin] Calling method '.version_select at 0x7fdda0396940>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:08:10.055763 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-41fb19ec-b193-45e4-a81f-0dad73ba09b5 tempest-HostsAdminNegativeTestJSON-1188407738 tempest-HostsAdminNegativeTestJSON-1188407738-project-admin] HTTP exception thrown: Host 'nonexistent_hostname' is not mapped to any cell Jul 16 21:08:10.057514 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-41fb19ec-b193-45e4-a81f-0dad73ba09b5 tempest-HostsAdminNegativeTestJSON-1188407738 tempest-HostsAdminNegativeTestJSON-1188407738-project-admin] Request method failure captured: Jul 16 21:08:10.057514 ncdv-master-1 devstack@n-api.service[138680]: request: GET /compute/v2.1/os-hosts/nonexistent_hostname/reboot HTTP/1.1 Jul 16 21:08:10.057514 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:08:10.057514 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:08:10.057514 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:08:10.057514 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 0 Jul 16 21:08:10.057514 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:08:10.057514 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:08:10.057514 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:08:10.057514 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:08:10.057514 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0yi4GY1omV82JLqFw7XbMWXHwUhmlIsKhZldqcHctGOoDzX3t98Zr3azT-ncB48LiB8oaCQdFSaGPU9jBj_RszLzWyb-FMfOh_oq62khRvIgXX8_ruDGT6Tp-p-JyUZIB57WzkCgKQfmJzxy1gy0lEig7mob5FG67unax5enTQCW78Q Jul 16 21:08:10.057514 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:08:10.057514 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:08:10.057514 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:08:10.057514 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:08:10.057514 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:08:10.057514 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:08:10.057514 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: 64f567689acb4ab7b9a6d8af6459f87a Jul 16 21:08:10.057514 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-HostsAdminNegativeTestJSON-1188407738 Jul 16 21:08:10.057514 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,admin,reader Jul 16 21:08:10.057514 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,admin,reader Jul 16 21:08:10.058878 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/64f567689acb4ab7b9a6d8af6459f87a"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/64f567689acb4ab7b9a6d8af6459f87a"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_64f567689acb4ab7b9a6d8af6459f87a"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/64f567689acb4ab7b9a6d8af6459f87a"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:08:10.058878 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-HostsAdminNegativeTestJSON-1188407738 Jul 16 21:08:10.058878 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: 64f567689acb4ab7b9a6d8af6459f87a Jul 16 21:08:10.058878 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-HostsAdminNegativeTestJSON-1188407738 Jul 16 21:08:10.058878 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-HostsAdminNegativeTestJSON-1188407738-project-admin Jul 16 21:08:10.058878 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:08:10.058878 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:08:10.058878 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: 17538442f109479896480a0c97445aba Jul 16 21:08:10.058878 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-HostsAdminNegativeTestJSON-1188407738-project-admin Jul 16 21:08:10.058878 ncdv-master-1 devstack@n-api.service[138680]: method: Jul 16 21:08:10.058878 ncdv-master-1 devstack@n-api.service[138680]: exception: Host 'nonexistent_hostname' is not mapped to any cell Jul 16 21:08:10.058878 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': 'nonexistent_hostname'} Jul 16 21:08:10.058878 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:08:10.059720 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:08:10.059720 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/hosts.py", line 174, in _host_power_action Jul 16 21:08:10.059720 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi result = self.api.host_power_action(context, host_name, action) Jul 16 21:08:10.059720 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/exception_wrapper.py", line 71, in wrapped Jul 16 21:08:10.059720 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi _emit_versioned_exception_notification( Jul 16 21:08:10.059720 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 16 21:08:10.059720 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi self.force_reraise() Jul 16 21:08:10.059720 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 16 21:08:10.059720 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise self.value Jul 16 21:08:10.059720 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/exception_wrapper.py", line 63, in wrapped Jul 16 21:08:10.059720 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(self, context, *args, **kw) Jul 16 21:08:10.059720 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5859, in targeted Jul 16 21:08:10.059720 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi mapping = objects.HostMapping.get_by_host(context, host) Jul 16 21:08:10.059720 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Jul 16 21:08:10.059720 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Jul 16 21:08:10.059720 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/host_mapping.py", line 105, in get_by_host Jul 16 21:08:10.059720 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi db_mapping = cls._get_by_host_from_db(context, host) Jul 16 21:08:10.059720 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1010, in wrapper Jul 16 21:08:10.059720 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return fn(*args, **kwargs) Jul 16 21:08:10.060787 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/host_mapping.py", line 100, in _get_by_host_from_db Jul 16 21:08:10.060787 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.HostMappingNotFound(name=host) Jul 16 21:08:10.060787 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.HostMappingNotFound: Host 'nonexistent_hostname' is not mapped to any cell Jul 16 21:08:10.060787 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:08:10.060787 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:08:10.060787 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:08:10.060787 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:08:10.060787 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:08:10.060787 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:08:10.060787 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:08:10.060787 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:08:10.060787 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Jul 16 21:08:10.060787 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Jul 16 21:08:10.060787 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:08:10.060787 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:08:10.060787 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/hosts.py", line 205, in reboot Jul 16 21:08:10.060787 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return self._host_power_action(req, host_name=id, action="reboot") Jul 16 21:08:10.060787 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/hosts.py", line 178, in _host_power_action Jul 16 21:08:10.060787 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:08:10.061849 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Host 'nonexistent_hostname' is not mapped to any cell Jul 16 21:08:10.061849 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:08:10.061849 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:08:10.061849 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:08:10.061849 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:08:10.061849 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:08:10.061849 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:08:10.061849 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:08:10.061849 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:08:10.061849 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Host 'nonexistent_hostname' is not mapped to any cell Jul 16 21:08:10.061849 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:08:10.061849 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-41fb19ec-b193-45e4-a81f-0dad73ba09b5 tempest-HostsAdminNegativeTestJSON-1188407738 tempest-HostsAdminNegativeTestJSON-1188407738-project-admin] Returning 404 to user: Host 'nonexistent_hostname' is not mapped to any cell {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:08:10.061849 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-41fb19ec-b193-45e4-a81f-0dad73ba09b5 tempest-HostsAdminNegativeTestJSON-1188407738 tempest-HostsAdminNegativeTestJSON-1188407738-project-admin] 10.222.0.16 "GET /compute/v2.1/os-hosts/nonexistent_hostname/reboot" status: 404 len: 99 microversion: 2.1 time: 0.018724 Jul 16 21:08:10.061849 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 31/57] 10.222.0.16 () {58 vars in 1211 bytes} [Sat Jul 16 21:08:10 2022] GET /compute/v2.1/os-hosts/nonexistent_hostname/reboot => generated 99 bytes in 20 msecs (HTTP/1.1 404) 9 headers in 378 bytes (1 switches on core 0) Jul 16 21:08:10.070664 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-f3dec801-1aa1-43ef-aac6-491f66704184 tempest-AggregatesAdminNegativeTestJSON-578260511 tempest-AggregatesAdminNegativeTestJSON-578260511-project-admin] 10.222.0.16 "DELETE /compute/v2.1/os-aggregates/8" status: 200 len: 0 microversion: 2.1 time: 0.104414 Jul 16 21:08:10.071871 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 27/58] 10.222.0.16 () {58 vars in 1151 bytes} [Sat Jul 16 21:08:09 2022] DELETE /compute/v2.1/os-aggregates/8 => generated 0 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 355 bytes (1 switches on core 0) Jul 16 21:08:10.074650 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-482f350c-4ab0-4495-9285-0973930932f8 tempest-HostsAdminNegativeTestJSON-1773216254 tempest-HostsAdminNegativeTestJSON-1773216254-project] Calling method '.version_select at 0x7fdd9df6e040>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:08:10.075720 ncdv-master-1 devstack@n-api.service[138680]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-hosts:show": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:08:10.075720 ncdv-master-1 devstack@n-api.service[138680]: warnings.warn(msg) Jul 16 21:08:10.076758 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-482f350c-4ab0-4495-9285-0973930932f8 tempest-HostsAdminNegativeTestJSON-1773216254 tempest-HostsAdminNegativeTestJSON-1773216254-project] Policy check for os_compute_api:os-hosts:show failed with credentials {'is_admin': False, 'user_id': '9b61d9df6f51477f9e1c2072a03a073c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '964640b04de845039e4b41cda5f555f1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:08:10.082690 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-35dd4113-44b2-4491-b9ef-0f1f988fd672 tempest-AggregatesAdminNegativeTestJSON-578260511 tempest-AggregatesAdminNegativeTestJSON-578260511-project-admin] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:08:10.083191 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-482f350c-4ab0-4495-9285-0973930932f8 tempest-HostsAdminNegativeTestJSON-1773216254 tempest-HostsAdminNegativeTestJSON-1773216254-project] Request method failure captured: Jul 16 21:08:10.083191 ncdv-master-1 devstack@n-api.service[138680]: request: GET /compute/v2.1/os-hosts/ncdv-master-1 HTTP/1.1 Jul 16 21:08:10.083191 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:08:10.083191 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:08:10.083191 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:08:10.083191 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 0 Jul 16 21:08:10.083191 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:08:10.083191 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:08:10.083191 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:08:10.083191 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:08:10.083191 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0yi13dQlIKkatjUrCe7DvLmujmbnviq_zuHjC9uzP2yiL7VHHGKkoVizD2sGeU719CNvsihfESqr0I1nPCCiV-fuL1q1OyNyD0kkegizPVQfmW6dBo4n9rHFKAyoAJjcO7WEXpG0OPIOkEasF-pn9GBJh_XdaUkbSC1p0Pr7n_bQijk Jul 16 21:08:10.083191 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:08:10.083191 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:08:10.083191 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:08:10.083191 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:08:10.083191 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:08:10.083191 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:08:10.083191 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: 964640b04de845039e4b41cda5f555f1 Jul 16 21:08:10.083191 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-HostsAdminNegativeTestJSON-1773216254 Jul 16 21:08:10.083191 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:08:10.083191 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:08:10.084555 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/964640b04de845039e4b41cda5f555f1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/964640b04de845039e4b41cda5f555f1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_964640b04de845039e4b41cda5f555f1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/964640b04de845039e4b41cda5f555f1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:08:10.084555 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-HostsAdminNegativeTestJSON-1773216254 Jul 16 21:08:10.084555 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: 964640b04de845039e4b41cda5f555f1 Jul 16 21:08:10.084555 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-HostsAdminNegativeTestJSON-1773216254 Jul 16 21:08:10.084555 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-HostsAdminNegativeTestJSON-1773216254-project Jul 16 21:08:10.084555 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:08:10.084555 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:08:10.084555 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: 9b61d9df6f51477f9e1c2072a03a073c Jul 16 21:08:10.084555 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-HostsAdminNegativeTestJSON-1773216254-project Jul 16 21:08:10.084555 ncdv-master-1 devstack@n-api.service[138680]: method: Jul 16 21:08:10.084555 ncdv-master-1 devstack@n-api.service[138680]: exception: Policy doesn't allow os_compute_api:os-hosts:show to be performed. Jul 16 21:08:10.084555 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': 'ncdv-master-1'} Jul 16 21:08:10.084555 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:08:10.085307 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:08:10.085307 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:08:10.085307 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:08:10.085307 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:08:10.085307 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:08:10.085307 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Jul 16 21:08:10.085307 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Jul 16 21:08:10.085307 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:08:10.085307 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:08:10.085307 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/hosts.py", line 270, in show Jul 16 21:08:10.085307 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi context.can(hosts_policies.POLICY_NAME % 'show', Jul 16 21:08:10.085307 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/context.py", line 240, in can Jul 16 21:08:10.085307 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return policy.authorize(self, action, target) Jul 16 21:08:10.085307 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 199, in authorize Jul 16 21:08:10.085307 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi LOG.debug('Policy check for %(action)s failed with credentials ' Jul 16 21:08:10.085307 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 16 21:08:10.085307 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi self.force_reraise() Jul 16 21:08:10.085307 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 16 21:08:10.085307 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise self.value Jul 16 21:08:10.087274 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 186, in authorize Jul 16 21:08:10.087274 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi result = _ENFORCER.authorize(action, target, context, Jul 16 21:08:10.087274 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1184, in authorize Jul 16 21:08:10.087274 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return self.enforce( Jul 16 21:08:10.087274 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1088, in enforce Jul 16 21:08:10.087274 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc(*args, **kwargs) Jul 16 21:08:10.087274 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.PolicyNotAuthorized: Policy doesn't allow os_compute_api:os-hosts:show to be performed. Jul 16 21:08:10.087274 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:08:10.087274 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:08:10.087274 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:08:10.087274 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:08:10.087274 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:08:10.087274 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:08:10.087274 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 354, in __exit__ Jul 16 21:08:10.087274 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(webob.exc.HTTPForbidden( Jul 16 21:08:10.087274 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Policy doesn't allow os_compute_api:os-hosts:show to be performed. Jul 16 21:08:10.087274 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:08:10.088120 ncdv-master-1 devstack@n-api.service[138681]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-aggregates:show": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:08:10.088120 ncdv-master-1 devstack@n-api.service[138681]: warnings.warn(msg) Jul 16 21:08:10.088261 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-482f350c-4ab0-4495-9285-0973930932f8 tempest-HostsAdminNegativeTestJSON-1773216254 tempest-HostsAdminNegativeTestJSON-1773216254-project] Returning 403 to user: Policy doesn't allow os_compute_api:os-hosts:show to be performed. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:08:10.089310 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-482f350c-4ab0-4495-9285-0973930932f8 tempest-HostsAdminNegativeTestJSON-1773216254 tempest-HostsAdminNegativeTestJSON-1773216254-project] 10.222.0.16 "GET /compute/v2.1/os-hosts/ncdv-master-1" status: 403 len: 109 microversion: 2.1 time: 0.018987 Jul 16 21:08:10.089953 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 32/59] 10.222.0.16 () {58 vars in 1169 bytes} [Sat Jul 16 21:08:10 2022] GET /compute/v2.1/os-hosts/ncdv-master-1 => generated 109 bytes in 23 msecs (HTTP/1.1 403) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:08:10.092938 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-35dd4113-44b2-4491-b9ef-0f1f988fd672 tempest-AggregatesAdminNegativeTestJSON-578260511 tempest-AggregatesAdminNegativeTestJSON-578260511-project-admin] HTTP exception thrown: Aggregate -1 could not be found. Jul 16 21:08:10.094726 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-35dd4113-44b2-4491-b9ef-0f1f988fd672 tempest-AggregatesAdminNegativeTestJSON-578260511 tempest-AggregatesAdminNegativeTestJSON-578260511-project-admin] Request method failure captured: Jul 16 21:08:10.094726 ncdv-master-1 devstack@n-api.service[138681]: request: GET /compute/v2.1/os-aggregates/-1 HTTP/1.1 Jul 16 21:08:10.094726 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:08:10.094726 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:08:10.094726 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:08:10.094726 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 21:08:10.094726 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:08:10.094726 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:08:10.094726 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:08:10.094726 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:08:10.094726 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0yiz0unej4HpWPJrVBlkA1u9-AWCKDSpHymJUf5DT-hiCYEGZJzMCi_CgXajssbkUJoCwaGruhY3wft9OVrurmz0_pRbvWF03aApzlFrXwt0GhLH6-kKTcDhg1N2k0bLKCTgcUoad_Ma5kdyMmhxUVTCB2mCAuoF3iPjsftSucch2dg Jul 16 21:08:10.094726 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:08:10.094726 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:08:10.094726 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:08:10.094726 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:08:10.094726 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:08:10.094726 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:08:10.094726 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: b6be58dcb13b4cec8f0bfc58f0080173 Jul 16 21:08:10.094726 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-AggregatesAdminNegativeTestJSON-578260511 Jul 16 21:08:10.094726 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,admin,reader Jul 16 21:08:10.094726 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,admin,reader Jul 16 21:08:10.096612 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/b6be58dcb13b4cec8f0bfc58f0080173"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/b6be58dcb13b4cec8f0bfc58f0080173"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_b6be58dcb13b4cec8f0bfc58f0080173"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/b6be58dcb13b4cec8f0bfc58f0080173"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:08:10.096612 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-AggregatesAdminNegativeTestJSON-578260511 Jul 16 21:08:10.096612 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: b6be58dcb13b4cec8f0bfc58f0080173 Jul 16 21:08:10.096612 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-AggregatesAdminNegativeTestJSON-578260511 Jul 16 21:08:10.096612 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-AggregatesAdminNegativeTestJSON-578260511-project-admin Jul 16 21:08:10.096612 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:08:10.096612 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:08:10.096612 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: cadb63bb5d6b4d7597987780f7d9ef21 Jul 16 21:08:10.096612 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-AggregatesAdminNegativeTestJSON-578260511-project-admin Jul 16 21:08:10.096612 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:08:10.096612 ncdv-master-1 devstack@n-api.service[138681]: exception: Aggregate -1 could not be found. Jul 16 21:08:10.096612 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': '-1'} Jul 16 21:08:10.099777 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:08:10.099777 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:08:10.099777 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/aggregates.py", line 108, in show Jul 16 21:08:10.099777 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi aggregate = self.api.get_aggregate(context, id) Jul 16 21:08:10.099777 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 6336, in get_aggregate Jul 16 21:08:10.099777 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return objects.Aggregate.get_by_id(context, aggregate_id) Jul 16 21:08:10.099777 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Jul 16 21:08:10.099777 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Jul 16 21:08:10.099777 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/aggregate.py", line 274, in get_by_id Jul 16 21:08:10.099777 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi db_aggregate = _aggregate_get_from_db(context, aggregate_id) Jul 16 21:08:10.099777 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1010, in wrapper Jul 16 21:08:10.099777 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return fn(*args, **kwargs) Jul 16 21:08:10.099777 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/aggregate.py", line 45, in _aggregate_get_from_db Jul 16 21:08:10.099777 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.AggregateNotFound(aggregate_id=aggregate_id) Jul 16 21:08:10.099777 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.AggregateNotFound: Aggregate -1 could not be found. Jul 16 21:08:10.099777 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:08:10.099777 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:08:10.099777 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:08:10.099777 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:08:10.099777 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:08:10.101040 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:08:10.101040 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:08:10.101040 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:08:10.101040 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:08:10.101040 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:08:10.101040 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/aggregates.py", line 110, in show Jul 16 21:08:10.101040 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:08:10.101040 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Aggregate -1 could not be found. Jul 16 21:08:10.101040 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:08:10.101040 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:08:10.101040 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:08:10.101040 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:08:10.101040 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:08:10.101040 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:08:10.101040 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:08:10.101040 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:08:10.101040 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Aggregate -1 could not be found. Jul 16 21:08:10.101040 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:08:10.102102 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-35dd4113-44b2-4491-b9ef-0f1f988fd672 tempest-AggregatesAdminNegativeTestJSON-578260511 tempest-AggregatesAdminNegativeTestJSON-578260511-project-admin] Returning 404 to user: Aggregate -1 could not be found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:08:10.102102 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-35dd4113-44b2-4491-b9ef-0f1f988fd672 tempest-AggregatesAdminNegativeTestJSON-578260511 tempest-AggregatesAdminNegativeTestJSON-578260511-project-admin] 10.222.0.16 "GET /compute/v2.1/os-aggregates/-1" status: 404 len: 78 microversion: 2.1 time: 0.019178 Jul 16 21:08:10.102102 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 28/60] 10.222.0.16 () {58 vars in 1151 bytes} [Sat Jul 16 21:08:10 2022] GET /compute/v2.1/os-aggregates/-1 => generated 78 bytes in 20 msecs (HTTP/1.1 404) 9 headers in 378 bytes (2 switches on core 0) Jul 16 21:08:10.102311 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-e29bc77d-e9c8-480a-a5c1-16da348937eb tempest-HostsAdminNegativeTestJSON-1188407738 tempest-HostsAdminNegativeTestJSON-1188407738-project-admin] Calling method '.version_select at 0x7fdd9df53790>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:08:10.109576 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-1173bc68-ae22-455d-8bbf-5d3d878f66de tempest-AggregatesAdminNegativeTestJSON-1271440314 tempest-AggregatesAdminNegativeTestJSON-1271440314-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:08:10.109576 ncdv-master-1 devstack@n-api.service[138681]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-aggregates:index": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:08:10.109576 ncdv-master-1 devstack@n-api.service[138681]: warnings.warn(msg) Jul 16 21:08:10.109897 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-e29bc77d-e9c8-480a-a5c1-16da348937eb tempest-HostsAdminNegativeTestJSON-1188407738 tempest-HostsAdminNegativeTestJSON-1188407738-project-admin] HTTP exception thrown: Host 'nonexistent_hostname' is not mapped to any cell Jul 16 21:08:10.110942 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-1173bc68-ae22-455d-8bbf-5d3d878f66de tempest-AggregatesAdminNegativeTestJSON-1271440314 tempest-AggregatesAdminNegativeTestJSON-1271440314-project] Policy check for os_compute_api:os-aggregates:index failed with credentials {'is_admin': False, 'user_id': '35a4905d463a4db2960294407f747f20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '49044125b0414a4ab0e2ab9eb7d66e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:08:10.112694 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-e29bc77d-e9c8-480a-a5c1-16da348937eb tempest-HostsAdminNegativeTestJSON-1188407738 tempest-HostsAdminNegativeTestJSON-1188407738-project-admin] Request method failure captured: Jul 16 21:08:10.112694 ncdv-master-1 devstack@n-api.service[138680]: request: GET /compute/v2.1/os-hosts/nonexistent_hostname HTTP/1.1 Jul 16 21:08:10.112694 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:08:10.112694 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:08:10.112694 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:08:10.112694 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 0 Jul 16 21:08:10.112694 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:08:10.112694 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:08:10.112694 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:08:10.112694 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:08:10.112694 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0yi4GY1omV82JLqFw7XbMWXHwUhmlIsKhZldqcHctGOoDzX3t98Zr3azT-ncB48LiB8oaCQdFSaGPU9jBj_RszLzWyb-FMfOh_oq62khRvIgXX8_ruDGT6Tp-p-JyUZIB57WzkCgKQfmJzxy1gy0lEig7mob5FG67unax5enTQCW78Q Jul 16 21:08:10.112694 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:08:10.112694 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:08:10.112694 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:08:10.112694 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:08:10.112694 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:08:10.112694 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:08:10.112694 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: 64f567689acb4ab7b9a6d8af6459f87a Jul 16 21:08:10.112694 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-HostsAdminNegativeTestJSON-1188407738 Jul 16 21:08:10.112694 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,admin,reader Jul 16 21:08:10.112694 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,admin,reader Jul 16 21:08:10.114062 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/64f567689acb4ab7b9a6d8af6459f87a"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/64f567689acb4ab7b9a6d8af6459f87a"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_64f567689acb4ab7b9a6d8af6459f87a"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/64f567689acb4ab7b9a6d8af6459f87a"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:08:10.114062 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-HostsAdminNegativeTestJSON-1188407738 Jul 16 21:08:10.114062 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: 64f567689acb4ab7b9a6d8af6459f87a Jul 16 21:08:10.114062 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-HostsAdminNegativeTestJSON-1188407738 Jul 16 21:08:10.114062 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-HostsAdminNegativeTestJSON-1188407738-project-admin Jul 16 21:08:10.114062 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:08:10.114062 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:08:10.114062 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: 17538442f109479896480a0c97445aba Jul 16 21:08:10.114062 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-HostsAdminNegativeTestJSON-1188407738-project-admin Jul 16 21:08:10.114062 ncdv-master-1 devstack@n-api.service[138680]: method: Jul 16 21:08:10.114062 ncdv-master-1 devstack@n-api.service[138680]: exception: Host 'nonexistent_hostname' is not mapped to any cell Jul 16 21:08:10.114062 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': 'nonexistent_hostname'} Jul 16 21:08:10.114062 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:08:10.114837 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:08:10.114837 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/hosts.py", line 274, in show Jul 16 21:08:10.114837 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi mapping = objects.HostMapping.get_by_host(context, host_name) Jul 16 21:08:10.114837 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Jul 16 21:08:10.114837 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Jul 16 21:08:10.114837 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/host_mapping.py", line 105, in get_by_host Jul 16 21:08:10.114837 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi db_mapping = cls._get_by_host_from_db(context, host) Jul 16 21:08:10.114837 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1010, in wrapper Jul 16 21:08:10.114837 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return fn(*args, **kwargs) Jul 16 21:08:10.114837 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/host_mapping.py", line 100, in _get_by_host_from_db Jul 16 21:08:10.114837 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.HostMappingNotFound(name=host) Jul 16 21:08:10.114837 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.HostMappingNotFound: Host 'nonexistent_hostname' is not mapped to any cell Jul 16 21:08:10.114837 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:08:10.114837 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:08:10.114837 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:08:10.114837 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:08:10.114837 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:08:10.114837 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:08:10.114837 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:08:10.115978 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:08:10.115978 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Jul 16 21:08:10.115978 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Jul 16 21:08:10.115978 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:08:10.115978 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:08:10.115978 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/hosts.py", line 282, in show Jul 16 21:08:10.115978 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:08:10.115978 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Host 'nonexistent_hostname' is not mapped to any cell Jul 16 21:08:10.115978 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:08:10.115978 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:08:10.115978 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:08:10.115978 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:08:10.115978 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:08:10.115978 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:08:10.115978 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:08:10.115978 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:08:10.115978 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Host 'nonexistent_hostname' is not mapped to any cell Jul 16 21:08:10.115978 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:08:10.120157 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-e29bc77d-e9c8-480a-a5c1-16da348937eb tempest-HostsAdminNegativeTestJSON-1188407738 tempest-HostsAdminNegativeTestJSON-1188407738-project-admin] Returning 404 to user: Host 'nonexistent_hostname' is not mapped to any cell {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:08:10.120157 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-e29bc77d-e9c8-480a-a5c1-16da348937eb tempest-HostsAdminNegativeTestJSON-1188407738 tempest-HostsAdminNegativeTestJSON-1188407738-project-admin] 10.222.0.16 "GET /compute/v2.1/os-hosts/nonexistent_hostname" status: 404 len: 99 microversion: 2.1 time: 0.018455 Jul 16 21:08:10.120802 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-1173bc68-ae22-455d-8bbf-5d3d878f66de tempest-AggregatesAdminNegativeTestJSON-1271440314 tempest-AggregatesAdminNegativeTestJSON-1271440314-project] Request method failure captured: Jul 16 21:08:10.120802 ncdv-master-1 devstack@n-api.service[138681]: request: GET /compute/v2.1/os-aggregates HTTP/1.1 Jul 16 21:08:10.120802 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:08:10.120802 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:08:10.120802 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:08:10.120802 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 21:08:10.120802 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:08:10.120802 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:08:10.120802 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:08:10.120802 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:08:10.120802 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0yiwx37gJG3UPf6cs7N4UkgaksWm_yG7GacwHzT4X_Y36InoB6bLW8dxMeWTnPNUr5MkHyB6zCa2ShiCwvtwzG8FdDgsivxSANP9O8r63IrJOV2dcu5-Fla-kUEiqkTUeLu0mggNWQr19K_Q1rkm8xG96kdc5KRDvRt2-aIVSJpj_Ug Jul 16 21:08:10.120802 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:08:10.120802 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:08:10.120802 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:08:10.120802 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:08:10.120802 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:08:10.120802 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:08:10.120802 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: 49044125b0414a4ab0e2ab9eb7d66e6e Jul 16 21:08:10.120802 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-AggregatesAdminNegativeTestJSON-1271440314 Jul 16 21:08:10.120802 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:08:10.120802 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:08:10.122082 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/49044125b0414a4ab0e2ab9eb7d66e6e"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/49044125b0414a4ab0e2ab9eb7d66e6e"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_49044125b0414a4ab0e2ab9eb7d66e6e"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/49044125b0414a4ab0e2ab9eb7d66e6e"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:08:10.122082 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-AggregatesAdminNegativeTestJSON-1271440314 Jul 16 21:08:10.122082 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: 49044125b0414a4ab0e2ab9eb7d66e6e Jul 16 21:08:10.122082 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-AggregatesAdminNegativeTestJSON-1271440314 Jul 16 21:08:10.122082 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-AggregatesAdminNegativeTestJSON-1271440314-project Jul 16 21:08:10.122082 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:08:10.122082 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:08:10.122082 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: 35a4905d463a4db2960294407f747f20 Jul 16 21:08:10.122082 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-AggregatesAdminNegativeTestJSON-1271440314-project Jul 16 21:08:10.122082 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:08:10.122082 ncdv-master-1 devstack@n-api.service[138681]: exception: Policy doesn't allow os_compute_api:os-aggregates:index to be performed. Jul 16 21:08:10.122082 ncdv-master-1 devstack@n-api.service[138681]: action_args: {} Jul 16 21:08:10.122804 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:08:10.122804 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:08:10.122804 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:08:10.122804 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:08:10.122804 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:08:10.122804 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:08:10.122804 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:08:10.122804 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:08:10.122804 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/aggregates.py", line 55, in index Jul 16 21:08:10.122804 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi context.can(aggr_policies.POLICY_ROOT % 'index', target={}) Jul 16 21:08:10.122804 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/context.py", line 240, in can Jul 16 21:08:10.122804 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return policy.authorize(self, action, target) Jul 16 21:08:10.122804 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 199, in authorize Jul 16 21:08:10.122804 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi LOG.debug('Policy check for %(action)s failed with credentials ' Jul 16 21:08:10.122804 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 16 21:08:10.122804 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi self.force_reraise() Jul 16 21:08:10.122804 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 16 21:08:10.122804 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise self.value Jul 16 21:08:10.122804 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 186, in authorize Jul 16 21:08:10.123942 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi result = _ENFORCER.authorize(action, target, context, Jul 16 21:08:10.123942 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1184, in authorize Jul 16 21:08:10.123942 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return self.enforce( Jul 16 21:08:10.123942 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1088, in enforce Jul 16 21:08:10.123942 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc(*args, **kwargs) Jul 16 21:08:10.123942 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.PolicyNotAuthorized: Policy doesn't allow os_compute_api:os-aggregates:index to be performed. Jul 16 21:08:10.123942 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:08:10.123942 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:08:10.123942 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:08:10.123942 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:08:10.123942 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:08:10.123942 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:08:10.123942 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 354, in __exit__ Jul 16 21:08:10.123942 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(webob.exc.HTTPForbidden( Jul 16 21:08:10.123942 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Policy doesn't allow os_compute_api:os-aggregates:index to be performed. Jul 16 21:08:10.123942 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:08:10.123942 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-1173bc68-ae22-455d-8bbf-5d3d878f66de tempest-AggregatesAdminNegativeTestJSON-1271440314 tempest-AggregatesAdminNegativeTestJSON-1271440314-project] Returning 403 to user: Policy doesn't allow os_compute_api:os-aggregates:index to be performed. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:08:10.124968 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 33/61] 10.222.0.16 () {58 vars in 1190 bytes} [Sat Jul 16 21:08:10 2022] GET /compute/v2.1/os-hosts/nonexistent_hostname => generated 99 bytes in 25 msecs (HTTP/1.1 404) 9 headers in 378 bytes (2 switches on core 0) Jul 16 21:08:10.125281 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-1173bc68-ae22-455d-8bbf-5d3d878f66de tempest-AggregatesAdminNegativeTestJSON-1271440314 tempest-AggregatesAdminNegativeTestJSON-1271440314-project] 10.222.0.16 "GET /compute/v2.1/os-aggregates" status: 403 len: 115 microversion: 2.1 time: 0.017318 Jul 16 21:08:10.125675 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 29/62] 10.222.0.16 () {58 vars in 1142 bytes} [Sat Jul 16 21:08:10 2022] GET /compute/v2.1/os-aggregates => generated 115 bytes in 22 msecs (HTTP/1.1 403) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:08:10.131621 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-ca427b3a-9807-4d7b-900e-ef67d5a9a0cd tempest-HostsAdminNegativeTestJSON-1773216254 tempest-HostsAdminNegativeTestJSON-1773216254-project] Calling method '.version_select at 0x7fdd9df6eca0>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:08:10.132764 ncdv-master-1 devstack@n-api.service[138680]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-hosts:shutdown": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:08:10.132764 ncdv-master-1 devstack@n-api.service[138680]: warnings.warn(msg) Jul 16 21:08:10.133790 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-ca427b3a-9807-4d7b-900e-ef67d5a9a0cd tempest-HostsAdminNegativeTestJSON-1773216254 tempest-HostsAdminNegativeTestJSON-1773216254-project] Policy check for os_compute_api:os-hosts:shutdown failed with credentials {'is_admin': False, 'user_id': '9b61d9df6f51477f9e1c2072a03a073c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '964640b04de845039e4b41cda5f555f1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:08:10.135466 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-ca427b3a-9807-4d7b-900e-ef67d5a9a0cd tempest-HostsAdminNegativeTestJSON-1773216254 tempest-HostsAdminNegativeTestJSON-1773216254-project] Request method failure captured: Jul 16 21:08:10.135466 ncdv-master-1 devstack@n-api.service[138680]: request: GET /compute/v2.1/os-hosts/ncdv-master-1/shutdown HTTP/1.1 Jul 16 21:08:10.135466 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:08:10.135466 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:08:10.135466 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:08:10.135466 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 0 Jul 16 21:08:10.135466 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:08:10.135466 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:08:10.135466 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:08:10.135466 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:08:10.135466 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0yi13dQlIKkatjUrCe7DvLmujmbnviq_zuHjC9uzP2yiL7VHHGKkoVizD2sGeU719CNvsihfESqr0I1nPCCiV-fuL1q1OyNyD0kkegizPVQfmW6dBo4n9rHFKAyoAJjcO7WEXpG0OPIOkEasF-pn9GBJh_XdaUkbSC1p0Pr7n_bQijk Jul 16 21:08:10.135466 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:08:10.135466 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:08:10.135466 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:08:10.135466 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:08:10.135466 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:08:10.135466 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:08:10.135466 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: 964640b04de845039e4b41cda5f555f1 Jul 16 21:08:10.135466 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-HostsAdminNegativeTestJSON-1773216254 Jul 16 21:08:10.135466 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:08:10.135466 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:08:10.138810 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/964640b04de845039e4b41cda5f555f1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/964640b04de845039e4b41cda5f555f1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_964640b04de845039e4b41cda5f555f1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/964640b04de845039e4b41cda5f555f1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:08:10.138810 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-HostsAdminNegativeTestJSON-1773216254 Jul 16 21:08:10.138810 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: 964640b04de845039e4b41cda5f555f1 Jul 16 21:08:10.138810 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-HostsAdminNegativeTestJSON-1773216254 Jul 16 21:08:10.138810 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-HostsAdminNegativeTestJSON-1773216254-project Jul 16 21:08:10.138810 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:08:10.138810 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:08:10.138810 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: 9b61d9df6f51477f9e1c2072a03a073c Jul 16 21:08:10.138810 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-HostsAdminNegativeTestJSON-1773216254-project Jul 16 21:08:10.138810 ncdv-master-1 devstack@n-api.service[138680]: method: Jul 16 21:08:10.138810 ncdv-master-1 devstack@n-api.service[138680]: exception: Policy doesn't allow os_compute_api:os-hosts:shutdown to be performed. Jul 16 21:08:10.138810 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': 'ncdv-master-1'} Jul 16 21:08:10.138810 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:08:10.140681 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:08:10.140681 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:08:10.140681 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:08:10.140681 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:08:10.140681 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:08:10.140681 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Jul 16 21:08:10.140681 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Jul 16 21:08:10.140681 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:08:10.140681 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:08:10.140681 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/hosts.py", line 195, in shutdown Jul 16 21:08:10.140681 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi context.can(hosts_policies.POLICY_NAME % 'shutdown', Jul 16 21:08:10.140681 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/context.py", line 240, in can Jul 16 21:08:10.140681 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return policy.authorize(self, action, target) Jul 16 21:08:10.140681 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 199, in authorize Jul 16 21:08:10.140681 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi LOG.debug('Policy check for %(action)s failed with credentials ' Jul 16 21:08:10.140681 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 16 21:08:10.140681 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi self.force_reraise() Jul 16 21:08:10.140681 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 16 21:08:10.140681 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise self.value Jul 16 21:08:10.142348 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 186, in authorize Jul 16 21:08:10.142348 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi result = _ENFORCER.authorize(action, target, context, Jul 16 21:08:10.142348 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1184, in authorize Jul 16 21:08:10.142348 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return self.enforce( Jul 16 21:08:10.142348 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1088, in enforce Jul 16 21:08:10.142348 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc(*args, **kwargs) Jul 16 21:08:10.142348 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.PolicyNotAuthorized: Policy doesn't allow os_compute_api:os-hosts:shutdown to be performed. Jul 16 21:08:10.142348 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:08:10.142348 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:08:10.142348 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:08:10.142348 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:08:10.142348 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:08:10.142348 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:08:10.142348 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 354, in __exit__ Jul 16 21:08:10.142348 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(webob.exc.HTTPForbidden( Jul 16 21:08:10.142348 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Policy doesn't allow os_compute_api:os-hosts:shutdown to be performed. Jul 16 21:08:10.142348 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:08:10.143564 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-ca427b3a-9807-4d7b-900e-ef67d5a9a0cd tempest-HostsAdminNegativeTestJSON-1773216254 tempest-HostsAdminNegativeTestJSON-1773216254-project] Returning 403 to user: Policy doesn't allow os_compute_api:os-hosts:shutdown to be performed. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:08:10.143564 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-ca427b3a-9807-4d7b-900e-ef67d5a9a0cd tempest-HostsAdminNegativeTestJSON-1773216254 tempest-HostsAdminNegativeTestJSON-1773216254-project] 10.222.0.16 "GET /compute/v2.1/os-hosts/ncdv-master-1/shutdown" status: 403 len: 113 microversion: 2.1 time: 0.015981 Jul 16 21:08:10.143564 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 34/63] 10.222.0.16 () {58 vars in 1196 bytes} [Sat Jul 16 21:08:10 2022] GET /compute/v2.1/os-hosts/ncdv-master-1/shutdown => generated 113 bytes in 17 msecs (HTTP/1.1 403) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:08:10.148942 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-a2546bd8-aacd-4c4f-8d7a-d79c4cf6d0d0 tempest-HostsAdminNegativeTestJSON-1188407738 tempest-HostsAdminNegativeTestJSON-1188407738-project-admin] Calling method '.version_select at 0x7fdd9f91bca0>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:08:10.149746 ncdv-master-1 devstack@n-api.service[138681]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-hosts:shutdown": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:08:10.149746 ncdv-master-1 devstack@n-api.service[138681]: warnings.warn(msg) Jul 16 21:08:10.157696 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-a2546bd8-aacd-4c4f-8d7a-d79c4cf6d0d0 tempest-HostsAdminNegativeTestJSON-1188407738 tempest-HostsAdminNegativeTestJSON-1188407738-project-admin] HTTP exception thrown: Host 'nonexistent_hostname' is not mapped to any cell Jul 16 21:08:10.160632 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-a2546bd8-aacd-4c4f-8d7a-d79c4cf6d0d0 tempest-HostsAdminNegativeTestJSON-1188407738 tempest-HostsAdminNegativeTestJSON-1188407738-project-admin] Request method failure captured: Jul 16 21:08:10.160632 ncdv-master-1 devstack@n-api.service[138681]: request: GET /compute/v2.1/os-hosts/nonexistent_hostname/shutdown HTTP/1.1 Jul 16 21:08:10.160632 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:08:10.160632 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:08:10.160632 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:08:10.160632 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 21:08:10.160632 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:08:10.160632 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:08:10.160632 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:08:10.160632 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:08:10.160632 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0yi4GY1omV82JLqFw7XbMWXHwUhmlIsKhZldqcHctGOoDzX3t98Zr3azT-ncB48LiB8oaCQdFSaGPU9jBj_RszLzWyb-FMfOh_oq62khRvIgXX8_ruDGT6Tp-p-JyUZIB57WzkCgKQfmJzxy1gy0lEig7mob5FG67unax5enTQCW78Q Jul 16 21:08:10.160632 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:08:10.160632 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:08:10.160632 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:08:10.160632 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:08:10.160632 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:08:10.160632 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:08:10.160632 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: 64f567689acb4ab7b9a6d8af6459f87a Jul 16 21:08:10.160632 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-HostsAdminNegativeTestJSON-1188407738 Jul 16 21:08:10.160632 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,admin,reader Jul 16 21:08:10.160632 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,admin,reader Jul 16 21:08:10.161910 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/64f567689acb4ab7b9a6d8af6459f87a"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/64f567689acb4ab7b9a6d8af6459f87a"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_64f567689acb4ab7b9a6d8af6459f87a"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/64f567689acb4ab7b9a6d8af6459f87a"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:08:10.161910 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-HostsAdminNegativeTestJSON-1188407738 Jul 16 21:08:10.161910 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: 64f567689acb4ab7b9a6d8af6459f87a Jul 16 21:08:10.161910 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-HostsAdminNegativeTestJSON-1188407738 Jul 16 21:08:10.161910 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-HostsAdminNegativeTestJSON-1188407738-project-admin Jul 16 21:08:10.161910 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:08:10.161910 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:08:10.161910 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: 17538442f109479896480a0c97445aba Jul 16 21:08:10.161910 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-HostsAdminNegativeTestJSON-1188407738-project-admin Jul 16 21:08:10.161910 ncdv-master-1 devstack@n-api.service[138681]: method: Jul 16 21:08:10.161910 ncdv-master-1 devstack@n-api.service[138681]: exception: Host 'nonexistent_hostname' is not mapped to any cell Jul 16 21:08:10.161910 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': 'nonexistent_hostname'} Jul 16 21:08:10.161910 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:08:10.162705 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:08:10.162705 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/hosts.py", line 174, in _host_power_action Jul 16 21:08:10.162705 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi result = self.api.host_power_action(context, host_name, action) Jul 16 21:08:10.162705 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/exception_wrapper.py", line 71, in wrapped Jul 16 21:08:10.162705 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi _emit_versioned_exception_notification( Jul 16 21:08:10.162705 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 16 21:08:10.162705 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi self.force_reraise() Jul 16 21:08:10.162705 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 16 21:08:10.162705 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise self.value Jul 16 21:08:10.162705 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/exception_wrapper.py", line 63, in wrapped Jul 16 21:08:10.162705 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(self, context, *args, **kw) Jul 16 21:08:10.162705 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5859, in targeted Jul 16 21:08:10.162705 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi mapping = objects.HostMapping.get_by_host(context, host) Jul 16 21:08:10.162705 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Jul 16 21:08:10.162705 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Jul 16 21:08:10.162705 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/host_mapping.py", line 105, in get_by_host Jul 16 21:08:10.162705 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi db_mapping = cls._get_by_host_from_db(context, host) Jul 16 21:08:10.162705 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1010, in wrapper Jul 16 21:08:10.162705 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return fn(*args, **kwargs) Jul 16 21:08:10.163938 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/host_mapping.py", line 100, in _get_by_host_from_db Jul 16 21:08:10.163938 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.HostMappingNotFound(name=host) Jul 16 21:08:10.163938 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.HostMappingNotFound: Host 'nonexistent_hostname' is not mapped to any cell Jul 16 21:08:10.163938 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:08:10.163938 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:08:10.163938 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:08:10.163938 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:08:10.163938 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:08:10.163938 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:08:10.163938 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:08:10.163938 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:08:10.163938 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Jul 16 21:08:10.163938 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Jul 16 21:08:10.163938 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:08:10.163938 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:08:10.163938 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/hosts.py", line 197, in shutdown Jul 16 21:08:10.163938 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return self._host_power_action(req, host_name=id, action="shutdown") Jul 16 21:08:10.163938 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/hosts.py", line 178, in _host_power_action Jul 16 21:08:10.163938 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:08:10.165252 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Host 'nonexistent_hostname' is not mapped to any cell Jul 16 21:08:10.165252 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:08:10.165252 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:08:10.165252 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:08:10.165252 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:08:10.165252 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:08:10.165252 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:08:10.165252 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:08:10.165252 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:08:10.165252 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Host 'nonexistent_hostname' is not mapped to any cell Jul 16 21:08:10.165252 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:08:10.165252 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-a2546bd8-aacd-4c4f-8d7a-d79c4cf6d0d0 tempest-HostsAdminNegativeTestJSON-1188407738 tempest-HostsAdminNegativeTestJSON-1188407738-project-admin] Returning 404 to user: Host 'nonexistent_hostname' is not mapped to any cell {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:08:10.165252 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-a2546bd8-aacd-4c4f-8d7a-d79c4cf6d0d0 tempest-HostsAdminNegativeTestJSON-1188407738 tempest-HostsAdminNegativeTestJSON-1188407738-project-admin] 10.222.0.16 "GET /compute/v2.1/os-hosts/nonexistent_hostname/shutdown" status: 404 len: 99 microversion: 2.1 time: 0.017190 Jul 16 21:08:10.169601 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 30/64] 10.222.0.16 () {58 vars in 1217 bytes} [Sat Jul 16 21:08:10 2022] GET /compute/v2.1/os-hosts/nonexistent_hostname/shutdown => generated 99 bytes in 24 msecs (HTTP/1.1 404) 9 headers in 378 bytes (1 switches on core 0) Jul 16 21:08:10.173571 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-d985ba12-f804-4a57-93ac-b76556fb4ecf tempest-HostsAdminNegativeTestJSON-1773216254 tempest-HostsAdminNegativeTestJSON-1773216254-project] Calling method '.version_select at 0x7fdd9df295e0>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:08:10.175437 ncdv-master-1 devstack@n-api.service[138680]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-hosts:start": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:08:10.175437 ncdv-master-1 devstack@n-api.service[138680]: warnings.warn(msg) Jul 16 21:08:10.176690 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-d985ba12-f804-4a57-93ac-b76556fb4ecf tempest-HostsAdminNegativeTestJSON-1773216254 tempest-HostsAdminNegativeTestJSON-1773216254-project] Policy check for os_compute_api:os-hosts:start failed with credentials {'is_admin': False, 'user_id': '9b61d9df6f51477f9e1c2072a03a073c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '964640b04de845039e4b41cda5f555f1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:08:10.178378 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-d985ba12-f804-4a57-93ac-b76556fb4ecf tempest-HostsAdminNegativeTestJSON-1773216254 tempest-HostsAdminNegativeTestJSON-1773216254-project] Request method failure captured: Jul 16 21:08:10.178378 ncdv-master-1 devstack@n-api.service[138680]: request: GET /compute/v2.1/os-hosts/ncdv-master-1/startup HTTP/1.1 Jul 16 21:08:10.178378 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:08:10.178378 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:08:10.178378 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:08:10.178378 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 0 Jul 16 21:08:10.178378 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:08:10.178378 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:08:10.178378 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:08:10.178378 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:08:10.178378 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0yi13dQlIKkatjUrCe7DvLmujmbnviq_zuHjC9uzP2yiL7VHHGKkoVizD2sGeU719CNvsihfESqr0I1nPCCiV-fuL1q1OyNyD0kkegizPVQfmW6dBo4n9rHFKAyoAJjcO7WEXpG0OPIOkEasF-pn9GBJh_XdaUkbSC1p0Pr7n_bQijk Jul 16 21:08:10.178378 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:08:10.178378 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:08:10.178378 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:08:10.178378 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:08:10.178378 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:08:10.178378 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:08:10.178378 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: 964640b04de845039e4b41cda5f555f1 Jul 16 21:08:10.178378 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-HostsAdminNegativeTestJSON-1773216254 Jul 16 21:08:10.178378 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:08:10.178378 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:08:10.179724 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/964640b04de845039e4b41cda5f555f1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/964640b04de845039e4b41cda5f555f1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_964640b04de845039e4b41cda5f555f1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/964640b04de845039e4b41cda5f555f1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:08:10.179724 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-HostsAdminNegativeTestJSON-1773216254 Jul 16 21:08:10.179724 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: 964640b04de845039e4b41cda5f555f1 Jul 16 21:08:10.179724 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-HostsAdminNegativeTestJSON-1773216254 Jul 16 21:08:10.179724 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-HostsAdminNegativeTestJSON-1773216254-project Jul 16 21:08:10.179724 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:08:10.179724 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:08:10.179724 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: 9b61d9df6f51477f9e1c2072a03a073c Jul 16 21:08:10.179724 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-HostsAdminNegativeTestJSON-1773216254-project Jul 16 21:08:10.179724 ncdv-master-1 devstack@n-api.service[138680]: method: Jul 16 21:08:10.179724 ncdv-master-1 devstack@n-api.service[138680]: exception: Policy doesn't allow os_compute_api:os-hosts:start to be performed. Jul 16 21:08:10.179724 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': 'ncdv-master-1'} Jul 16 21:08:10.179724 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:08:10.180583 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:08:10.180583 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:08:10.180583 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:08:10.180583 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:08:10.180583 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:08:10.180583 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Jul 16 21:08:10.180583 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Jul 16 21:08:10.180583 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:08:10.180583 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:08:10.180583 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/hosts.py", line 187, in startup Jul 16 21:08:10.180583 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi context.can(hosts_policies.POLICY_NAME % 'start', Jul 16 21:08:10.180583 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/context.py", line 240, in can Jul 16 21:08:10.180583 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return policy.authorize(self, action, target) Jul 16 21:08:10.180583 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 199, in authorize Jul 16 21:08:10.180583 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi LOG.debug('Policy check for %(action)s failed with credentials ' Jul 16 21:08:10.180583 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 16 21:08:10.180583 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi self.force_reraise() Jul 16 21:08:10.180583 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 16 21:08:10.180583 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise self.value Jul 16 21:08:10.181896 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 186, in authorize Jul 16 21:08:10.181896 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi result = _ENFORCER.authorize(action, target, context, Jul 16 21:08:10.181896 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1184, in authorize Jul 16 21:08:10.181896 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return self.enforce( Jul 16 21:08:10.181896 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1088, in enforce Jul 16 21:08:10.181896 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc(*args, **kwargs) Jul 16 21:08:10.181896 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.PolicyNotAuthorized: Policy doesn't allow os_compute_api:os-hosts:start to be performed. Jul 16 21:08:10.181896 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:08:10.181896 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:08:10.181896 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:08:10.181896 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:08:10.181896 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:08:10.181896 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:08:10.181896 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 354, in __exit__ Jul 16 21:08:10.181896 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(webob.exc.HTTPForbidden( Jul 16 21:08:10.181896 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Policy doesn't allow os_compute_api:os-hosts:start to be performed. Jul 16 21:08:10.181896 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:08:10.182872 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-d985ba12-f804-4a57-93ac-b76556fb4ecf tempest-HostsAdminNegativeTestJSON-1773216254 tempest-HostsAdminNegativeTestJSON-1773216254-project] Returning 403 to user: Policy doesn't allow os_compute_api:os-hosts:start to be performed. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:08:10.182872 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-d985ba12-f804-4a57-93ac-b76556fb4ecf tempest-HostsAdminNegativeTestJSON-1773216254 tempest-HostsAdminNegativeTestJSON-1773216254-project] 10.222.0.16 "GET /compute/v2.1/os-hosts/ncdv-master-1/startup" status: 403 len: 110 microversion: 2.1 time: 0.011061 Jul 16 21:08:10.182872 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 35/65] 10.222.0.16 () {58 vars in 1193 bytes} [Sat Jul 16 21:08:10 2022] GET /compute/v2.1/os-hosts/ncdv-master-1/startup => generated 110 bytes in 12 msecs (HTTP/1.1 403) 9 headers in 379 bytes (2 switches on core 0) Jul 16 21:08:10.202259 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-3dc61ad2-a376-47f7-a77a-cc989fbe0ee5 tempest-HostsAdminNegativeTestJSON-1188407738 tempest-HostsAdminNegativeTestJSON-1188407738-project-admin] Calling method '.version_select at 0x7fdd9ee17940>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:08:10.203570 ncdv-master-1 devstack@n-api.service[138681]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-hosts:start": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:08:10.203570 ncdv-master-1 devstack@n-api.service[138681]: warnings.warn(msg) Jul 16 21:08:10.216441 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-3dc61ad2-a376-47f7-a77a-cc989fbe0ee5 tempest-HostsAdminNegativeTestJSON-1188407738 tempest-HostsAdminNegativeTestJSON-1188407738-project-admin] HTTP exception thrown: Host 'nonexistent_hostname' is not mapped to any cell Jul 16 21:08:10.218325 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-3dc61ad2-a376-47f7-a77a-cc989fbe0ee5 tempest-HostsAdminNegativeTestJSON-1188407738 tempest-HostsAdminNegativeTestJSON-1188407738-project-admin] Request method failure captured: Jul 16 21:08:10.218325 ncdv-master-1 devstack@n-api.service[138681]: request: GET /compute/v2.1/os-hosts/nonexistent_hostname/startup HTTP/1.1 Jul 16 21:08:10.218325 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:08:10.218325 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:08:10.218325 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:08:10.218325 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 21:08:10.218325 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:08:10.218325 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:08:10.218325 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:08:10.218325 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:08:10.218325 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0yi4GY1omV82JLqFw7XbMWXHwUhmlIsKhZldqcHctGOoDzX3t98Zr3azT-ncB48LiB8oaCQdFSaGPU9jBj_RszLzWyb-FMfOh_oq62khRvIgXX8_ruDGT6Tp-p-JyUZIB57WzkCgKQfmJzxy1gy0lEig7mob5FG67unax5enTQCW78Q Jul 16 21:08:10.218325 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:08:10.218325 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:08:10.218325 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:08:10.218325 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:08:10.218325 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:08:10.218325 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:08:10.218325 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: 64f567689acb4ab7b9a6d8af6459f87a Jul 16 21:08:10.218325 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-HostsAdminNegativeTestJSON-1188407738 Jul 16 21:08:10.218325 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,admin,reader Jul 16 21:08:10.218325 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,admin,reader Jul 16 21:08:10.219623 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/64f567689acb4ab7b9a6d8af6459f87a"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/64f567689acb4ab7b9a6d8af6459f87a"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_64f567689acb4ab7b9a6d8af6459f87a"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/64f567689acb4ab7b9a6d8af6459f87a"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:08:10.219623 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-HostsAdminNegativeTestJSON-1188407738 Jul 16 21:08:10.219623 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: 64f567689acb4ab7b9a6d8af6459f87a Jul 16 21:08:10.219623 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-HostsAdminNegativeTestJSON-1188407738 Jul 16 21:08:10.219623 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-HostsAdminNegativeTestJSON-1188407738-project-admin Jul 16 21:08:10.219623 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:08:10.219623 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:08:10.219623 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: 17538442f109479896480a0c97445aba Jul 16 21:08:10.219623 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-HostsAdminNegativeTestJSON-1188407738-project-admin Jul 16 21:08:10.219623 ncdv-master-1 devstack@n-api.service[138681]: method: Jul 16 21:08:10.219623 ncdv-master-1 devstack@n-api.service[138681]: exception: Host 'nonexistent_hostname' is not mapped to any cell Jul 16 21:08:10.219623 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': 'nonexistent_hostname'} Jul 16 21:08:10.219623 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:08:10.220539 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:08:10.220539 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/hosts.py", line 174, in _host_power_action Jul 16 21:08:10.220539 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi result = self.api.host_power_action(context, host_name, action) Jul 16 21:08:10.220539 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/exception_wrapper.py", line 71, in wrapped Jul 16 21:08:10.220539 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi _emit_versioned_exception_notification( Jul 16 21:08:10.220539 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 16 21:08:10.220539 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi self.force_reraise() Jul 16 21:08:10.220539 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 16 21:08:10.220539 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise self.value Jul 16 21:08:10.220539 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/exception_wrapper.py", line 63, in wrapped Jul 16 21:08:10.220539 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(self, context, *args, **kw) Jul 16 21:08:10.220539 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5859, in targeted Jul 16 21:08:10.220539 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi mapping = objects.HostMapping.get_by_host(context, host) Jul 16 21:08:10.220539 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Jul 16 21:08:10.220539 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Jul 16 21:08:10.220539 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/host_mapping.py", line 105, in get_by_host Jul 16 21:08:10.220539 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi db_mapping = cls._get_by_host_from_db(context, host) Jul 16 21:08:10.220539 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1010, in wrapper Jul 16 21:08:10.220539 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return fn(*args, **kwargs) Jul 16 21:08:10.223383 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/host_mapping.py", line 100, in _get_by_host_from_db Jul 16 21:08:10.223383 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.HostMappingNotFound(name=host) Jul 16 21:08:10.223383 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.HostMappingNotFound: Host 'nonexistent_hostname' is not mapped to any cell Jul 16 21:08:10.223383 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:08:10.223383 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:08:10.223383 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:08:10.223383 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:08:10.223383 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:08:10.223383 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:08:10.223383 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:08:10.223383 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:08:10.223383 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Jul 16 21:08:10.223383 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Jul 16 21:08:10.223383 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:08:10.223383 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:08:10.223383 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/hosts.py", line 189, in startup Jul 16 21:08:10.223383 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return self._host_power_action(req, host_name=id, action="startup") Jul 16 21:08:10.223383 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/hosts.py", line 178, in _host_power_action Jul 16 21:08:10.223383 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:08:10.225786 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Host 'nonexistent_hostname' is not mapped to any cell Jul 16 21:08:10.225786 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:08:10.225786 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:08:10.225786 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:08:10.225786 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:08:10.225786 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:08:10.225786 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:08:10.225786 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:08:10.225786 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:08:10.225786 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Host 'nonexistent_hostname' is not mapped to any cell Jul 16 21:08:10.225786 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:08:10.225786 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-3dc61ad2-a376-47f7-a77a-cc989fbe0ee5 tempest-HostsAdminNegativeTestJSON-1188407738 tempest-HostsAdminNegativeTestJSON-1188407738-project-admin] Returning 404 to user: Host 'nonexistent_hostname' is not mapped to any cell {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:08:10.225786 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-3dc61ad2-a376-47f7-a77a-cc989fbe0ee5 tempest-HostsAdminNegativeTestJSON-1188407738 tempest-HostsAdminNegativeTestJSON-1188407738-project-admin] 10.222.0.16 "GET /compute/v2.1/os-hosts/nonexistent_hostname/startup" status: 404 len: 99 microversion: 2.1 time: 0.024199 Jul 16 21:08:10.225786 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 31/66] 10.222.0.16 () {58 vars in 1214 bytes} [Sat Jul 16 21:08:10 2022] GET /compute/v2.1/os-hosts/nonexistent_hostname/startup => generated 99 bytes in 39 msecs (HTTP/1.1 404) 9 headers in 378 bytes (1 switches on core 0) Jul 16 21:08:10.232723 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-e5ea12e1-b7ef-44c5-a5d4-137bd2bb017c tempest-HostsAdminNegativeTestJSON-1188407738 tempest-HostsAdminNegativeTestJSON-1188407738-project-admin] Action: 'update', calling method: .version_select at 0x7fdd9df29ca0>, body: {"status": "enable", "maintenance_mode": "invalid"} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:08:10.235917 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-e5ea12e1-b7ef-44c5-a5d4-137bd2bb017c tempest-HostsAdminNegativeTestJSON-1188407738 tempest-HostsAdminNegativeTestJSON-1188407738-project-admin] Request method failure captured: Jul 16 21:08:10.235917 ncdv-master-1 devstack@n-api.service[138680]: request: PUT /compute/v2.1/os-hosts/ncdv-master-1 HTTP/1.1 Jul 16 21:08:10.235917 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:08:10.235917 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:08:10.235917 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:08:10.235917 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 51 Jul 16 21:08:10.235917 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:08:10.235917 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:08:10.235917 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:08:10.235917 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:08:10.235917 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0yi4GY1omV82JLqFw7XbMWXHwUhmlIsKhZldqcHctGOoDzX3t98Zr3azT-ncB48LiB8oaCQdFSaGPU9jBj_RszLzWyb-FMfOh_oq62khRvIgXX8_ruDGT6Tp-p-JyUZIB57WzkCgKQfmJzxy1gy0lEig7mob5FG67unax5enTQCW78Q Jul 16 21:08:10.235917 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:08:10.235917 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:08:10.235917 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:08:10.235917 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:08:10.235917 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:08:10.235917 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:08:10.235917 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: 64f567689acb4ab7b9a6d8af6459f87a Jul 16 21:08:10.235917 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-HostsAdminNegativeTestJSON-1188407738 Jul 16 21:08:10.235917 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,admin,reader Jul 16 21:08:10.235917 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,admin,reader Jul 16 21:08:10.238145 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/64f567689acb4ab7b9a6d8af6459f87a"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/64f567689acb4ab7b9a6d8af6459f87a"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_64f567689acb4ab7b9a6d8af6459f87a"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/64f567689acb4ab7b9a6d8af6459f87a"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:08:10.238145 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-HostsAdminNegativeTestJSON-1188407738 Jul 16 21:08:10.238145 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: 64f567689acb4ab7b9a6d8af6459f87a Jul 16 21:08:10.238145 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-HostsAdminNegativeTestJSON-1188407738 Jul 16 21:08:10.238145 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-HostsAdminNegativeTestJSON-1188407738-project-admin Jul 16 21:08:10.238145 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:08:10.238145 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:08:10.238145 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: 17538442f109479896480a0c97445aba Jul 16 21:08:10.238145 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-HostsAdminNegativeTestJSON-1188407738-project-admin Jul 16 21:08:10.238145 ncdv-master-1 devstack@n-api.service[138680]: Jul 16 21:08:10.238145 ncdv-master-1 devstack@n-api.service[138680]: {"status": "enable", "maintenance_mode": "invalid"} Jul 16 21:08:10.238145 ncdv-master-1 devstack@n-api.service[138680]: method: Jul 16 21:08:10.238145 ncdv-master-1 devstack@n-api.service[138680]: exception: Invalid input for field/attribute maintenance_mode. Value: invalid. 'invalid' is not one of ['enable', 'disable', 'Enable', 'Disable', 'ENABLE', 'DISABLE'] Jul 16 21:08:10.239524 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': 'ncdv-master-1', 'body': {'status': 'enable', 'maintenance_mode': 'invalid'}} Jul 16 21:08:10.239524 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:08:10.239524 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:08:10.239524 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 279, in validate Jul 16 21:08:10.239524 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Jul 16 21:08:10.239524 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/lib/python3/dist-packages/jsonschema/validators.py", line 353, in validate Jul 16 21:08:10.239524 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise error Jul 16 21:08:10.239524 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: 'invalid' is not one of ['enable', 'disable', 'Enable', 'Disable', 'ENABLE', 'DISABLE'] Jul 16 21:08:10.239524 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:08:10.239524 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Failed validating 'enum' in schema['properties']['maintenance_mode']: Jul 16 21:08:10.239524 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi {'enum': ['enable', Jul 16 21:08:10.239524 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi 'disable', Jul 16 21:08:10.239524 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi 'Enable', Jul 16 21:08:10.239524 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi 'Disable', Jul 16 21:08:10.239524 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi 'ENABLE', Jul 16 21:08:10.239524 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi 'DISABLE'], Jul 16 21:08:10.239524 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi 'type': 'string'} Jul 16 21:08:10.239524 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:08:10.239524 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi On instance: Jul 16 21:08:10.239524 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi 'invalid' Jul 16 21:08:10.239524 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:08:10.239524 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:08:10.239524 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:08:10.239524 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:08:10.239524 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:08:10.242651 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:08:10.242651 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:08:10.242651 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:08:10.242651 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Jul 16 21:08:10.242651 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Jul 16 21:08:10.242651 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:08:10.242651 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:08:10.242651 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Jul 16 21:08:10.242651 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Jul 16 21:08:10.242651 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Jul 16 21:08:10.242651 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Jul 16 21:08:10.242651 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 308, in validate Jul 16 21:08:10.242651 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Jul 16 21:08:10.242651 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute maintenance_mode. Value: invalid. 'invalid' is not one of ['enable', 'disable', 'Enable', 'Disable', 'ENABLE', 'DISABLE'] Jul 16 21:08:10.242651 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:08:10.242651 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:08:10.242651 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:08:10.242651 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:08:10.242651 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:08:10.244053 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:08:10.244053 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Jul 16 21:08:10.244053 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Jul 16 21:08:10.244053 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute maintenance_mode. Value: invalid. 'invalid' is not one of ['enable', 'disable', 'Enable', 'Disable', 'ENABLE', 'DISABLE'] Jul 16 21:08:10.244053 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:08:10.244053 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-e5ea12e1-b7ef-44c5-a5d4-137bd2bb017c tempest-HostsAdminNegativeTestJSON-1188407738 tempest-HostsAdminNegativeTestJSON-1188407738-project-admin] Returning 400 to user: Invalid input for field/attribute maintenance_mode. Value: invalid. 'invalid' is not one of ['enable', 'disable', 'Enable', 'Disable', 'ENABLE', 'DISABLE'] {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:08:10.244053 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-e5ea12e1-b7ef-44c5-a5d4-137bd2bb017c tempest-HostsAdminNegativeTestJSON-1188407738 tempest-HostsAdminNegativeTestJSON-1188407738-project-admin] 10.222.0.16 "PUT /compute/v2.1/os-hosts/ncdv-master-1" status: 400 len: 199 microversion: 2.1 time: 0.009755 Jul 16 21:08:10.244053 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 36/67] 10.222.0.16 () {60 vars in 1189 bytes} [Sat Jul 16 21:08:10 2022] PUT /compute/v2.1/os-hosts/ncdv-master-1 => generated 199 bytes in 15 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Jul 16 21:08:10.249093 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-44135b8c-693e-4d75-b078-7b44dca56e8a tempest-HostsAdminNegativeTestJSON-1188407738 tempest-HostsAdminNegativeTestJSON-1188407738-project-admin] Action: 'update', calling method: .version_select at 0x7fdd9ddcc700>, body: {"status": "invalid", "maintenance_mode": "enable"} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:08:10.253503 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-44135b8c-693e-4d75-b078-7b44dca56e8a tempest-HostsAdminNegativeTestJSON-1188407738 tempest-HostsAdminNegativeTestJSON-1188407738-project-admin] Request method failure captured: Jul 16 21:08:10.253503 ncdv-master-1 devstack@n-api.service[138681]: request: PUT /compute/v2.1/os-hosts/ncdv-master-1 HTTP/1.1 Jul 16 21:08:10.253503 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:08:10.253503 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:08:10.253503 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:08:10.253503 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 51 Jul 16 21:08:10.253503 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:08:10.253503 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:08:10.253503 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:08:10.253503 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:08:10.253503 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0yi4GY1omV82JLqFw7XbMWXHwUhmlIsKhZldqcHctGOoDzX3t98Zr3azT-ncB48LiB8oaCQdFSaGPU9jBj_RszLzWyb-FMfOh_oq62khRvIgXX8_ruDGT6Tp-p-JyUZIB57WzkCgKQfmJzxy1gy0lEig7mob5FG67unax5enTQCW78Q Jul 16 21:08:10.253503 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:08:10.253503 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:08:10.253503 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:08:10.253503 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:08:10.253503 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:08:10.253503 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:08:10.253503 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: 64f567689acb4ab7b9a6d8af6459f87a Jul 16 21:08:10.253503 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-HostsAdminNegativeTestJSON-1188407738 Jul 16 21:08:10.253503 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,admin,reader Jul 16 21:08:10.253503 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,admin,reader Jul 16 21:08:10.255350 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/64f567689acb4ab7b9a6d8af6459f87a"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/64f567689acb4ab7b9a6d8af6459f87a"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_64f567689acb4ab7b9a6d8af6459f87a"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/64f567689acb4ab7b9a6d8af6459f87a"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:08:10.255350 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-HostsAdminNegativeTestJSON-1188407738 Jul 16 21:08:10.255350 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: 64f567689acb4ab7b9a6d8af6459f87a Jul 16 21:08:10.255350 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-HostsAdminNegativeTestJSON-1188407738 Jul 16 21:08:10.255350 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-HostsAdminNegativeTestJSON-1188407738-project-admin Jul 16 21:08:10.255350 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:08:10.255350 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:08:10.255350 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: 17538442f109479896480a0c97445aba Jul 16 21:08:10.255350 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-HostsAdminNegativeTestJSON-1188407738-project-admin Jul 16 21:08:10.255350 ncdv-master-1 devstack@n-api.service[138681]: Jul 16 21:08:10.255350 ncdv-master-1 devstack@n-api.service[138681]: {"status": "invalid", "maintenance_mode": "enable"} Jul 16 21:08:10.255350 ncdv-master-1 devstack@n-api.service[138681]: method: Jul 16 21:08:10.255350 ncdv-master-1 devstack@n-api.service[138681]: exception: Invalid input for field/attribute status. Value: invalid. 'invalid' is not one of ['enable', 'disable', 'Enable', 'Disable', 'ENABLE', 'DISABLE'] Jul 16 21:08:10.256437 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': 'ncdv-master-1', 'body': {'status': 'invalid', 'maintenance_mode': 'enable'}} Jul 16 21:08:10.256437 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:08:10.256437 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:08:10.256437 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 279, in validate Jul 16 21:08:10.256437 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Jul 16 21:08:10.256437 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/lib/python3/dist-packages/jsonschema/validators.py", line 353, in validate Jul 16 21:08:10.256437 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise error Jul 16 21:08:10.256437 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: 'invalid' is not one of ['enable', 'disable', 'Enable', 'Disable', 'ENABLE', 'DISABLE'] Jul 16 21:08:10.256437 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:08:10.256437 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Failed validating 'enum' in schema['properties']['status']: Jul 16 21:08:10.256437 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi {'enum': ['enable', Jul 16 21:08:10.256437 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi 'disable', Jul 16 21:08:10.256437 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi 'Enable', Jul 16 21:08:10.256437 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi 'Disable', Jul 16 21:08:10.256437 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi 'ENABLE', Jul 16 21:08:10.256437 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi 'DISABLE'], Jul 16 21:08:10.256437 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi 'type': 'string'} Jul 16 21:08:10.256437 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:08:10.256437 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi On instance: Jul 16 21:08:10.256437 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi 'invalid' Jul 16 21:08:10.256437 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:08:10.256437 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:08:10.256437 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:08:10.256437 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:08:10.256437 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:08:10.258208 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:08:10.258208 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:08:10.258208 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:08:10.258208 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Jul 16 21:08:10.258208 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Jul 16 21:08:10.258208 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:08:10.258208 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:08:10.258208 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Jul 16 21:08:10.258208 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Jul 16 21:08:10.258208 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Jul 16 21:08:10.258208 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Jul 16 21:08:10.258208 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 308, in validate Jul 16 21:08:10.258208 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Jul 16 21:08:10.258208 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute status. Value: invalid. 'invalid' is not one of ['enable', 'disable', 'Enable', 'Disable', 'ENABLE', 'DISABLE'] Jul 16 21:08:10.258208 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:08:10.258208 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:08:10.258208 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:08:10.258208 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:08:10.258208 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:08:10.260370 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:08:10.260370 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Jul 16 21:08:10.260370 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Jul 16 21:08:10.260370 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute status. Value: invalid. 'invalid' is not one of ['enable', 'disable', 'Enable', 'Disable', 'ENABLE', 'DISABLE'] Jul 16 21:08:10.260370 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:08:10.260370 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-44135b8c-693e-4d75-b078-7b44dca56e8a tempest-HostsAdminNegativeTestJSON-1188407738 tempest-HostsAdminNegativeTestJSON-1188407738-project-admin] Returning 400 to user: Invalid input for field/attribute status. Value: invalid. 'invalid' is not one of ['enable', 'disable', 'Enable', 'Disable', 'ENABLE', 'DISABLE'] {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:08:10.260370 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-44135b8c-693e-4d75-b078-7b44dca56e8a tempest-HostsAdminNegativeTestJSON-1188407738 tempest-HostsAdminNegativeTestJSON-1188407738-project-admin] 10.222.0.16 "PUT /compute/v2.1/os-hosts/ncdv-master-1" status: 400 len: 189 microversion: 2.1 time: 0.015291 Jul 16 21:08:10.261118 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 32/68] 10.222.0.16 () {60 vars in 1189 bytes} [Sat Jul 16 21:08:10 2022] PUT /compute/v2.1/os-hosts/ncdv-master-1 => generated 189 bytes in 18 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Jul 16 21:08:10.271188 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-3290c918-a52d-4e3d-967b-4bce8b87cb87 tempest-HostsAdminNegativeTestJSON-1773216254 tempest-HostsAdminNegativeTestJSON-1773216254-project] Action: 'update', calling method: .version_select at 0x7fdd9df29c10>, body: {"status": "enable", "maintenance_mode": "enable"} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:08:10.272653 ncdv-master-1 devstack@n-api.service[138680]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-hosts:update": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:08:10.272653 ncdv-master-1 devstack@n-api.service[138680]: warnings.warn(msg) Jul 16 21:08:10.273763 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-3290c918-a52d-4e3d-967b-4bce8b87cb87 tempest-HostsAdminNegativeTestJSON-1773216254 tempest-HostsAdminNegativeTestJSON-1773216254-project] Policy check for os_compute_api:os-hosts:update failed with credentials {'is_admin': False, 'user_id': '9b61d9df6f51477f9e1c2072a03a073c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '964640b04de845039e4b41cda5f555f1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:08:10.275350 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-3290c918-a52d-4e3d-967b-4bce8b87cb87 tempest-HostsAdminNegativeTestJSON-1773216254 tempest-HostsAdminNegativeTestJSON-1773216254-project] Request method failure captured: Jul 16 21:08:10.275350 ncdv-master-1 devstack@n-api.service[138680]: request: PUT /compute/v2.1/os-hosts/ncdv-master-1 HTTP/1.1 Jul 16 21:08:10.275350 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:08:10.275350 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:08:10.275350 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:08:10.275350 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 50 Jul 16 21:08:10.275350 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:08:10.275350 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:08:10.275350 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:08:10.275350 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:08:10.275350 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0yi13dQlIKkatjUrCe7DvLmujmbnviq_zuHjC9uzP2yiL7VHHGKkoVizD2sGeU719CNvsihfESqr0I1nPCCiV-fuL1q1OyNyD0kkegizPVQfmW6dBo4n9rHFKAyoAJjcO7WEXpG0OPIOkEasF-pn9GBJh_XdaUkbSC1p0Pr7n_bQijk Jul 16 21:08:10.275350 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:08:10.275350 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:08:10.275350 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:08:10.275350 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:08:10.275350 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:08:10.275350 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:08:10.275350 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: 964640b04de845039e4b41cda5f555f1 Jul 16 21:08:10.275350 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-HostsAdminNegativeTestJSON-1773216254 Jul 16 21:08:10.275350 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:08:10.275350 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:08:10.276730 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/964640b04de845039e4b41cda5f555f1"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/964640b04de845039e4b41cda5f555f1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_964640b04de845039e4b41cda5f555f1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/964640b04de845039e4b41cda5f555f1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:08:10.276730 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-HostsAdminNegativeTestJSON-1773216254 Jul 16 21:08:10.276730 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: 964640b04de845039e4b41cda5f555f1 Jul 16 21:08:10.276730 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-HostsAdminNegativeTestJSON-1773216254 Jul 16 21:08:10.276730 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-HostsAdminNegativeTestJSON-1773216254-project Jul 16 21:08:10.276730 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:08:10.276730 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:08:10.276730 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: 9b61d9df6f51477f9e1c2072a03a073c Jul 16 21:08:10.276730 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-HostsAdminNegativeTestJSON-1773216254-project Jul 16 21:08:10.276730 ncdv-master-1 devstack@n-api.service[138680]: Jul 16 21:08:10.276730 ncdv-master-1 devstack@n-api.service[138680]: {"status": "enable", "maintenance_mode": "enable"} Jul 16 21:08:10.276730 ncdv-master-1 devstack@n-api.service[138680]: method: Jul 16 21:08:10.276730 ncdv-master-1 devstack@n-api.service[138680]: exception: Policy doesn't allow os_compute_api:os-hosts:update to be performed. Jul 16 21:08:10.276730 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': 'ncdv-master-1', 'body': {'status': 'enable', 'maintenance_mode': 'enable'}} Jul 16 21:08:10.277686 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:08:10.277686 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:08:10.277686 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:08:10.277686 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:08:10.277686 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:08:10.277686 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:08:10.277686 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Jul 16 21:08:10.277686 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Jul 16 21:08:10.277686 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:08:10.277686 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:08:10.277686 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 16 21:08:10.277686 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 16 21:08:10.277686 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/hosts.py", line 112, in update Jul 16 21:08:10.277686 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi context.can(hosts_policies.POLICY_NAME % 'update', Jul 16 21:08:10.277686 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/context.py", line 240, in can Jul 16 21:08:10.277686 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return policy.authorize(self, action, target) Jul 16 21:08:10.277686 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 199, in authorize Jul 16 21:08:10.277686 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi LOG.debug('Policy check for %(action)s failed with credentials ' Jul 16 21:08:10.277686 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 16 21:08:10.277686 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi self.force_reraise() Jul 16 21:08:10.278867 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 16 21:08:10.278867 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise self.value Jul 16 21:08:10.278867 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 186, in authorize Jul 16 21:08:10.278867 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi result = _ENFORCER.authorize(action, target, context, Jul 16 21:08:10.278867 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1184, in authorize Jul 16 21:08:10.278867 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return self.enforce( Jul 16 21:08:10.278867 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1088, in enforce Jul 16 21:08:10.278867 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc(*args, **kwargs) Jul 16 21:08:10.278867 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.PolicyNotAuthorized: Policy doesn't allow os_compute_api:os-hosts:update to be performed. Jul 16 21:08:10.278867 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:08:10.278867 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:08:10.278867 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:08:10.278867 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:08:10.278867 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:08:10.278867 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:08:10.278867 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 354, in __exit__ Jul 16 21:08:10.278867 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(webob.exc.HTTPForbidden( Jul 16 21:08:10.278867 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Policy doesn't allow os_compute_api:os-hosts:update to be performed. Jul 16 21:08:10.278867 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:08:10.280341 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-3290c918-a52d-4e3d-967b-4bce8b87cb87 tempest-HostsAdminNegativeTestJSON-1773216254 tempest-HostsAdminNegativeTestJSON-1773216254-project] Returning 403 to user: Policy doesn't allow os_compute_api:os-hosts:update to be performed. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:08:10.280341 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-3290c918-a52d-4e3d-967b-4bce8b87cb87 tempest-HostsAdminNegativeTestJSON-1773216254 tempest-HostsAdminNegativeTestJSON-1773216254-project] 10.222.0.16 "PUT /compute/v2.1/os-hosts/ncdv-master-1" status: 403 len: 111 microversion: 2.1 time: 0.010218 Jul 16 21:08:10.280341 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 37/69] 10.222.0.16 () {60 vars in 1189 bytes} [Sat Jul 16 21:08:10 2022] PUT /compute/v2.1/os-hosts/ncdv-master-1 => generated 111 bytes in 11 msecs (HTTP/1.1 403) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:08:10.289334 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-55d25be7-b4e7-4ecc-95fd-b7c878e5fa3f tempest-HostsAdminNegativeTestJSON-1188407738 tempest-HostsAdminNegativeTestJSON-1188407738-project-admin] Action: 'update', calling method: .version_select at 0x7fdd9dd55af0>, body: {"status": null, "maintenance_mode": null} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:08:10.292111 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-55d25be7-b4e7-4ecc-95fd-b7c878e5fa3f tempest-HostsAdminNegativeTestJSON-1188407738 tempest-HostsAdminNegativeTestJSON-1188407738-project-admin] Request method failure captured: Jul 16 21:08:10.292111 ncdv-master-1 devstack@n-api.service[138681]: request: PUT /compute/v2.1/os-hosts/ncdv-master-1 HTTP/1.1 Jul 16 21:08:10.292111 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:08:10.292111 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:08:10.292111 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:08:10.292111 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 42 Jul 16 21:08:10.292111 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:08:10.292111 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:08:10.292111 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:08:10.292111 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:08:10.292111 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0yi4GY1omV82JLqFw7XbMWXHwUhmlIsKhZldqcHctGOoDzX3t98Zr3azT-ncB48LiB8oaCQdFSaGPU9jBj_RszLzWyb-FMfOh_oq62khRvIgXX8_ruDGT6Tp-p-JyUZIB57WzkCgKQfmJzxy1gy0lEig7mob5FG67unax5enTQCW78Q Jul 16 21:08:10.292111 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:08:10.292111 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:08:10.292111 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:08:10.292111 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:08:10.292111 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:08:10.292111 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:08:10.292111 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: 64f567689acb4ab7b9a6d8af6459f87a Jul 16 21:08:10.292111 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-HostsAdminNegativeTestJSON-1188407738 Jul 16 21:08:10.292111 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,admin,reader Jul 16 21:08:10.292111 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,admin,reader Jul 16 21:08:10.293169 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/64f567689acb4ab7b9a6d8af6459f87a"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/64f567689acb4ab7b9a6d8af6459f87a"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_64f567689acb4ab7b9a6d8af6459f87a"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/64f567689acb4ab7b9a6d8af6459f87a"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:08:10.293169 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-HostsAdminNegativeTestJSON-1188407738 Jul 16 21:08:10.293169 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: 64f567689acb4ab7b9a6d8af6459f87a Jul 16 21:08:10.293169 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-HostsAdminNegativeTestJSON-1188407738 Jul 16 21:08:10.293169 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-HostsAdminNegativeTestJSON-1188407738-project-admin Jul 16 21:08:10.293169 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:08:10.293169 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:08:10.293169 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: 17538442f109479896480a0c97445aba Jul 16 21:08:10.293169 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-HostsAdminNegativeTestJSON-1188407738-project-admin Jul 16 21:08:10.293169 ncdv-master-1 devstack@n-api.service[138681]: Jul 16 21:08:10.293169 ncdv-master-1 devstack@n-api.service[138681]: {"status": null, "maintenance_mode": null} Jul 16 21:08:10.293169 ncdv-master-1 devstack@n-api.service[138681]: method: Jul 16 21:08:10.293169 ncdv-master-1 devstack@n-api.service[138681]: exception: Invalid input for field/attribute status. Value: None. None is not of type 'string' Jul 16 21:08:10.293169 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': 'ncdv-master-1', 'body': {'status': None, 'maintenance_mode': None}} Jul 16 21:08:10.293908 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:08:10.293908 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:08:10.293908 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 279, in validate Jul 16 21:08:10.293908 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Jul 16 21:08:10.293908 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/lib/python3/dist-packages/jsonschema/validators.py", line 353, in validate Jul 16 21:08:10.293908 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise error Jul 16 21:08:10.293908 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: None is not of type 'string' Jul 16 21:08:10.293908 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:08:10.293908 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Failed validating 'type' in schema['properties']['status']: Jul 16 21:08:10.293908 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi {'enum': ['enable', Jul 16 21:08:10.293908 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi 'disable', Jul 16 21:08:10.293908 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi 'Enable', Jul 16 21:08:10.293908 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi 'Disable', Jul 16 21:08:10.293908 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi 'ENABLE', Jul 16 21:08:10.293908 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi 'DISABLE'], Jul 16 21:08:10.293908 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi 'type': 'string'} Jul 16 21:08:10.293908 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:08:10.293908 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi On instance: Jul 16 21:08:10.293908 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi None Jul 16 21:08:10.293908 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:08:10.293908 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:08:10.293908 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:08:10.293908 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:08:10.293908 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:08:10.293908 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:08:10.295905 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:08:10.295905 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:08:10.295905 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Jul 16 21:08:10.295905 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Jul 16 21:08:10.295905 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:08:10.295905 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:08:10.295905 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Jul 16 21:08:10.295905 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Jul 16 21:08:10.295905 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Jul 16 21:08:10.295905 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Jul 16 21:08:10.295905 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 308, in validate Jul 16 21:08:10.295905 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Jul 16 21:08:10.295905 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute status. Value: None. None is not of type 'string' Jul 16 21:08:10.295905 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:08:10.295905 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:08:10.295905 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:08:10.295905 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:08:10.295905 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:08:10.295905 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:08:10.299807 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Jul 16 21:08:10.299807 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Jul 16 21:08:10.299807 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute status. Value: None. None is not of type 'string' Jul 16 21:08:10.299807 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:08:10.299807 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-55d25be7-b4e7-4ecc-95fd-b7c878e5fa3f tempest-HostsAdminNegativeTestJSON-1188407738 tempest-HostsAdminNegativeTestJSON-1188407738-project-admin] Returning 400 to user: Invalid input for field/attribute status. Value: None. None is not of type 'string' {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:08:10.299807 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-55d25be7-b4e7-4ecc-95fd-b7c878e5fa3f tempest-HostsAdminNegativeTestJSON-1188407738 tempest-HostsAdminNegativeTestJSON-1188407738-project-admin] 10.222.0.16 "PUT /compute/v2.1/os-hosts/ncdv-master-1" status: 400 len: 127 microversion: 2.1 time: 0.008117 Jul 16 21:08:10.299807 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 33/70] 10.222.0.16 () {60 vars in 1189 bytes} [Sat Jul 16 21:08:10 2022] PUT /compute/v2.1/os-hosts/ncdv-master-1 => generated 127 bytes in 10 msecs (HTTP/1.1 400) 9 headers in 381 bytes (2 switches on core 0) Jul 16 21:08:10.304920 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-e4beb22f-8579-4056-8ab2-347faa3683f0 tempest-HostsAdminNegativeTestJSON-1188407738 tempest-HostsAdminNegativeTestJSON-1188407738-project-admin] Action: 'update', calling method: .version_select at 0x7fdd9dd6e820>, body: {"status": "enable", "maintenance_mode": "enable"} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:08:10.306130 ncdv-master-1 devstack@n-api.service[138681]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-hosts:update": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:08:10.306130 ncdv-master-1 devstack@n-api.service[138681]: warnings.warn(msg) Jul 16 21:08:10.306918 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.hosts [None req-e4beb22f-8579-4056-8ab2-347faa3683f0 tempest-HostsAdminNegativeTestJSON-1188407738 tempest-HostsAdminNegativeTestJSON-1188407738-project-admin] Enabling host nonexistent_hostname. Jul 16 21:08:10.315137 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-e4beb22f-8579-4056-8ab2-347faa3683f0 tempest-HostsAdminNegativeTestJSON-1188407738 tempest-HostsAdminNegativeTestJSON-1188407738-project-admin] HTTP exception thrown: Host 'nonexistent_hostname' is not mapped to any cell Jul 16 21:08:10.317183 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-e4beb22f-8579-4056-8ab2-347faa3683f0 tempest-HostsAdminNegativeTestJSON-1188407738 tempest-HostsAdminNegativeTestJSON-1188407738-project-admin] Request method failure captured: Jul 16 21:08:10.317183 ncdv-master-1 devstack@n-api.service[138681]: request: PUT /compute/v2.1/os-hosts/nonexistent_hostname HTTP/1.1 Jul 16 21:08:10.317183 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:08:10.317183 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:08:10.317183 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:08:10.317183 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 50 Jul 16 21:08:10.317183 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:08:10.317183 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:08:10.317183 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:08:10.317183 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:08:10.317183 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0yi4GY1omV82JLqFw7XbMWXHwUhmlIsKhZldqcHctGOoDzX3t98Zr3azT-ncB48LiB8oaCQdFSaGPU9jBj_RszLzWyb-FMfOh_oq62khRvIgXX8_ruDGT6Tp-p-JyUZIB57WzkCgKQfmJzxy1gy0lEig7mob5FG67unax5enTQCW78Q Jul 16 21:08:10.317183 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:08:10.317183 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:08:10.317183 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:08:10.317183 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:08:10.317183 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:08:10.317183 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:08:10.317183 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: 64f567689acb4ab7b9a6d8af6459f87a Jul 16 21:08:10.317183 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-HostsAdminNegativeTestJSON-1188407738 Jul 16 21:08:10.317183 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,admin,reader Jul 16 21:08:10.317183 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,admin,reader Jul 16 21:08:10.318291 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/64f567689acb4ab7b9a6d8af6459f87a"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/64f567689acb4ab7b9a6d8af6459f87a"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_64f567689acb4ab7b9a6d8af6459f87a"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/64f567689acb4ab7b9a6d8af6459f87a"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:08:10.318291 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-HostsAdminNegativeTestJSON-1188407738 Jul 16 21:08:10.318291 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: 64f567689acb4ab7b9a6d8af6459f87a Jul 16 21:08:10.318291 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-HostsAdminNegativeTestJSON-1188407738 Jul 16 21:08:10.318291 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-HostsAdminNegativeTestJSON-1188407738-project-admin Jul 16 21:08:10.318291 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:08:10.318291 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:08:10.318291 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: 17538442f109479896480a0c97445aba Jul 16 21:08:10.318291 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-HostsAdminNegativeTestJSON-1188407738-project-admin Jul 16 21:08:10.318291 ncdv-master-1 devstack@n-api.service[138681]: Jul 16 21:08:10.318291 ncdv-master-1 devstack@n-api.service[138681]: {"status": "enable", "maintenance_mode": "enable"} Jul 16 21:08:10.318291 ncdv-master-1 devstack@n-api.service[138681]: method: Jul 16 21:08:10.318291 ncdv-master-1 devstack@n-api.service[138681]: exception: Host 'nonexistent_hostname' is not mapped to any cell Jul 16 21:08:10.318291 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': 'nonexistent_hostname', 'body': {'status': 'enable', 'maintenance_mode': 'enable'}} Jul 16 21:08:10.319061 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:08:10.319061 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:08:10.319061 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/hosts.py", line 159, in _set_enabled_status Jul 16 21:08:10.319061 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi result = self.api.set_host_enabled(context, host_name, enabled) Jul 16 21:08:10.319061 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/exception_wrapper.py", line 71, in wrapped Jul 16 21:08:10.319061 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi _emit_versioned_exception_notification( Jul 16 21:08:10.319061 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 16 21:08:10.319061 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi self.force_reraise() Jul 16 21:08:10.319061 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 16 21:08:10.319061 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise self.value Jul 16 21:08:10.319061 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/exception_wrapper.py", line 63, in wrapped Jul 16 21:08:10.319061 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(self, context, *args, **kw) Jul 16 21:08:10.319061 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5859, in targeted Jul 16 21:08:10.319061 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi mapping = objects.HostMapping.get_by_host(context, host) Jul 16 21:08:10.319061 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Jul 16 21:08:10.319061 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Jul 16 21:08:10.319061 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/host_mapping.py", line 105, in get_by_host Jul 16 21:08:10.319061 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi db_mapping = cls._get_by_host_from_db(context, host) Jul 16 21:08:10.319935 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1010, in wrapper Jul 16 21:08:10.319935 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return fn(*args, **kwargs) Jul 16 21:08:10.319935 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/host_mapping.py", line 100, in _get_by_host_from_db Jul 16 21:08:10.319935 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.HostMappingNotFound(name=host) Jul 16 21:08:10.319935 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.HostMappingNotFound: Host 'nonexistent_hostname' is not mapped to any cell Jul 16 21:08:10.319935 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:08:10.319935 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:08:10.319935 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:08:10.319935 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:08:10.319935 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:08:10.319935 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:08:10.319935 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:08:10.319935 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:08:10.319935 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Jul 16 21:08:10.319935 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Jul 16 21:08:10.319935 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:08:10.319935 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:08:10.319935 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 16 21:08:10.319935 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 16 21:08:10.319935 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/hosts.py", line 124, in update Jul 16 21:08:10.321267 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi result['status'] = self._set_enabled_status(context, id, status) Jul 16 21:08:10.321267 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/hosts.py", line 163, in _set_enabled_status Jul 16 21:08:10.321267 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:08:10.321267 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Host 'nonexistent_hostname' is not mapped to any cell Jul 16 21:08:10.321267 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:08:10.321267 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:08:10.321267 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:08:10.321267 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:08:10.321267 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:08:10.321267 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:08:10.321267 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:08:10.321267 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:08:10.321267 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Host 'nonexistent_hostname' is not mapped to any cell Jul 16 21:08:10.321267 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:08:10.321267 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-e4beb22f-8579-4056-8ab2-347faa3683f0 tempest-HostsAdminNegativeTestJSON-1188407738 tempest-HostsAdminNegativeTestJSON-1188407738-project-admin] Returning 404 to user: Host 'nonexistent_hostname' is not mapped to any cell {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:08:10.322274 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-e4beb22f-8579-4056-8ab2-347faa3683f0 tempest-HostsAdminNegativeTestJSON-1188407738 tempest-HostsAdminNegativeTestJSON-1188407738-project-admin] 10.222.0.16 "PUT /compute/v2.1/os-hosts/nonexistent_hostname" status: 404 len: 99 microversion: 2.1 time: 0.018829 Jul 16 21:08:10.322274 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 34/71] 10.222.0.16 () {60 vars in 1210 bytes} [Sat Jul 16 21:08:10 2022] PUT /compute/v2.1/os-hosts/nonexistent_hostname => generated 99 bytes in 21 msecs (HTTP/1.1 404) 9 headers in 378 bytes (1 switches on core 0) Jul 16 21:08:10.842666 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-6afadc81-b54d-4480-baad-48637e02b8c4 tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AggregatesAdminTestJSON-server-62033026", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452", "availability_zone": "tempest-test_az-2083597543"}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:08:10.876025 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6afadc81-b54d-4480-baad-48637e02b8c4 tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:08:10.876025 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6afadc81-b54d-4480-baad-48637e02b8c4 tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:08:10.876989 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6afadc81-b54d-4480-baad-48637e02b8c4 tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:08:10.877533 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6afadc81-b54d-4480-baad-48637e02b8c4 tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:08:12.115312 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.network.neutron [None req-6afadc81-b54d-4480-baad-48637e02b8c4 tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] validate_networks() for None {{(pid=138680) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 16 21:08:12.386379 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-6afadc81-b54d-4480-baad-48637e02b8c4 tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] Flavor limits 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 16 21:08:12.386907 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-6afadc81-b54d-4480-baad-48637e02b8c4 tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] Image limits 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 16 21:08:12.387431 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-6afadc81-b54d-4480-baad-48637e02b8c4 tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] Flavor pref 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 16 21:08:12.387916 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-6afadc81-b54d-4480-baad-48637e02b8c4 tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] Image pref 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 16 21:08:12.388718 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-6afadc81-b54d-4480-baad-48637e02b8c4 tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 16 21:08:12.408059 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6afadc81-b54d-4480-baad-48637e02b8c4 tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:08:12.408489 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6afadc81-b54d-4480-baad-48637e02b8c4 tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:08:12.409289 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6afadc81-b54d-4480-baad-48637e02b8c4 tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:08:12.410440 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6afadc81-b54d-4480-baad-48637e02b8c4 tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:08:12.442529 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-6afadc81-b54d-4480-baad-48637e02b8c4 tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] Getting quotas for project e7c343c974bd4ec0a6a765f818b895be. Resources: {'ram', 'instances', 'cores'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:08:12.448940 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-6afadc81-b54d-4480-baad-48637e02b8c4 tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] Getting quotas for user db6c21ea09c54ea492d806f42d2f69a7 and project e7c343c974bd4ec0a6a765f818b895be. Resources: {'ram', 'instances', 'cores'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:08:12.461011 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-6afadc81-b54d-4480-baad-48637e02b8c4 tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] Going to run 1 instances... {{(pid=138680) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 16 21:08:12.463929 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-6afadc81-b54d-4480-baad-48637e02b8c4 tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] [instance: 7895aae7-b50f-48cf-a3e1-5b33dc6e16a3] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138680) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 16 21:08:12.606247 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-6afadc81-b54d-4480-baad-48637e02b8c4 tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 2.305496 Jul 16 21:08:12.609535 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 38/72] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 21:08:10 2022] POST /compute/v2.1/servers => generated 380 bytes in 2315 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 16 21:08:12.618623 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-8cd3257d-8e57-49f1-9d6d-01bf3326be0d tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:08:12.619752 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-8cd3257d-8e57-49f1-9d6d-01bf3326be0d tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] [instance: 7895aae7-b50f-48cf-a3e1-5b33dc6e16a3] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:08:14.198587 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-8cd3257d-8e57-49f1-9d6d-01bf3326be0d tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'db6c21ea09c54ea492d806f42d2f69a7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e7c343c974bd4ec0a6a765f818b895be', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:08:14.231523 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-8cd3257d-8e57-49f1-9d6d-01bf3326be0d tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] 10.222.0.16 "GET /compute/v2.1/servers/7895aae7-b50f-48cf-a3e1-5b33dc6e16a3" status: 200 len: 1241 microversion: 2.1 time: 1.616933 Jul 16 21:08:14.232433 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 35/73] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:08:12 2022] GET /compute/v2.1/servers/7895aae7-b50f-48cf-a3e1-5b33dc6e16a3 => generated 1241 bytes in 1619 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:08:15.017173 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-07b093e2-77d8-48e2-81f1-e2fbd05c853b tempest-HypervisorAdminV253TestBase-1151073637 tempest-HypervisorAdminV253TestBase-1151073637-project-admin] Calling method '.version_select at 0x7fdd9df6ee50>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:08:15.017173 ncdv-master-1 devstack@n-api.service[138680]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-hypervisors:list-detail": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:08:15.017173 ncdv-master-1 devstack@n-api.service[138680]: warnings.warn(msg) Jul 16 21:08:15.021138 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-07b093e2-77d8-48e2-81f1-e2fbd05c853b tempest-HypervisorAdminV253TestBase-1151073637 tempest-HypervisorAdminV253TestBase-1151073637-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:08:15.023065 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-07b093e2-77d8-48e2-81f1-e2fbd05c853b tempest-HypervisorAdminV253TestBase-1151073637 tempest-HypervisorAdminV253TestBase-1151073637-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:08:15.055465 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-07b093e2-77d8-48e2-81f1-e2fbd05c853b tempest-HypervisorAdminV253TestBase-1151073637 tempest-HypervisorAdminV253TestBase-1151073637-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:08:15.056006 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-07b093e2-77d8-48e2-81f1-e2fbd05c853b tempest-HypervisorAdminV253TestBase-1151073637 tempest-HypervisorAdminV253TestBase-1151073637-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:08:15.099517 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-07b093e2-77d8-48e2-81f1-e2fbd05c853b tempest-HypervisorAdminV253TestBase-1151073637 tempest-HypervisorAdminV253TestBase-1151073637-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:08:15.100613 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-07b093e2-77d8-48e2-81f1-e2fbd05c853b tempest-HypervisorAdminV253TestBase-1151073637 tempest-HypervisorAdminV253TestBase-1151073637-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:08:15.132029 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-07b093e2-77d8-48e2-81f1-e2fbd05c853b tempest-HypervisorAdminV253TestBase-1151073637 tempest-HypervisorAdminV253TestBase-1151073637-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:08:15.133480 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-07b093e2-77d8-48e2-81f1-e2fbd05c853b tempest-HypervisorAdminV253TestBase-1151073637 tempest-HypervisorAdminV253TestBase-1151073637-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:08:15.165961 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-07b093e2-77d8-48e2-81f1-e2fbd05c853b tempest-HypervisorAdminV253TestBase-1151073637 tempest-HypervisorAdminV253TestBase-1151073637-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:08:15.166790 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-07b093e2-77d8-48e2-81f1-e2fbd05c853b tempest-HypervisorAdminV253TestBase-1151073637 tempest-HypervisorAdminV253TestBase-1151073637-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:08:15.179896 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-07b093e2-77d8-48e2-81f1-e2fbd05c853b tempest-HypervisorAdminV253TestBase-1151073637 tempest-HypervisorAdminV253TestBase-1151073637-project-admin] 10.222.0.16 "GET /compute/v2.1/os-hypervisors/detail?with_servers=True" status: 200 len: 1571 microversion: 2.53 time: 0.725703 Jul 16 21:08:15.180878 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 39/74] 10.222.0.16 () {60 vars in 1242 bytes} [Sat Jul 16 21:08:14 2022] GET /compute/v2.1/os-hypervisors/detail?with_servers=True => generated 1571 bytes in 729 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:08:15.196101 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-a5da6da0-5f39-4151-b5b3-2f99e7bc616f tempest-HypervisorAdminV253TestBase-1151073637 tempest-HypervisorAdminV253TestBase-1151073637-project-admin] Calling method '.version_select at 0x7fdd9dd55d30>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:08:15.197637 ncdv-master-1 devstack@n-api.service[138681]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-hypervisors:list": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:08:15.197637 ncdv-master-1 devstack@n-api.service[138681]: warnings.warn(msg) Jul 16 21:08:15.198897 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a5da6da0-5f39-4151-b5b3-2f99e7bc616f tempest-HypervisorAdminV253TestBase-1151073637 tempest-HypervisorAdminV253TestBase-1151073637-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:08:15.199590 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a5da6da0-5f39-4151-b5b3-2f99e7bc616f tempest-HypervisorAdminV253TestBase-1151073637 tempest-HypervisorAdminV253TestBase-1151073637-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:08:15.236397 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a5da6da0-5f39-4151-b5b3-2f99e7bc616f tempest-HypervisorAdminV253TestBase-1151073637 tempest-HypervisorAdminV253TestBase-1151073637-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:08:15.237090 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a5da6da0-5f39-4151-b5b3-2f99e7bc616f tempest-HypervisorAdminV253TestBase-1151073637 tempest-HypervisorAdminV253TestBase-1151073637-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:08:15.250036 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-7291816d-80b1-4b9f-9606-fb545384c006 tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:08:15.250809 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-7291816d-80b1-4b9f-9606-fb545384c006 tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] [instance: 7895aae7-b50f-48cf-a3e1-5b33dc6e16a3] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:08:15.288672 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a5da6da0-5f39-4151-b5b3-2f99e7bc616f tempest-HypervisorAdminV253TestBase-1151073637 tempest-HypervisorAdminV253TestBase-1151073637-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:08:15.289123 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a5da6da0-5f39-4151-b5b3-2f99e7bc616f tempest-HypervisorAdminV253TestBase-1151073637 tempest-HypervisorAdminV253TestBase-1151073637-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:08:15.317309 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a5da6da0-5f39-4151-b5b3-2f99e7bc616f tempest-HypervisorAdminV253TestBase-1151073637 tempest-HypervisorAdminV253TestBase-1151073637-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:08:15.317779 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a5da6da0-5f39-4151-b5b3-2f99e7bc616f tempest-HypervisorAdminV253TestBase-1151073637 tempest-HypervisorAdminV253TestBase-1151073637-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:08:15.341587 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a5da6da0-5f39-4151-b5b3-2f99e7bc616f tempest-HypervisorAdminV253TestBase-1151073637 tempest-HypervisorAdminV253TestBase-1151073637-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:08:15.342247 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a5da6da0-5f39-4151-b5b3-2f99e7bc616f tempest-HypervisorAdminV253TestBase-1151073637 tempest-HypervisorAdminV253TestBase-1151073637-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:08:15.353558 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-a5da6da0-5f39-4151-b5b3-2f99e7bc616f tempest-HypervisorAdminV253TestBase-1151073637 tempest-HypervisorAdminV253TestBase-1151073637-project-admin] 10.222.0.16 "GET /compute/v2.1/os-hypervisors?with_servers=True" status: 200 len: 265 microversion: 2.53 time: 0.159312 Jul 16 21:08:15.353558 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 36/75] 10.222.0.16 () {60 vars in 1221 bytes} [Sat Jul 16 21:08:15 2022] GET /compute/v2.1/os-hypervisors?with_servers=True => generated 265 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 16 21:08:15.363278 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-5db58eb1-df5a-4278-b4a2-9178376ce80e tempest-HypervisorAdminV253TestBase-1151073637 tempest-HypervisorAdminV253TestBase-1151073637-project-admin] Calling method '.version_select at 0x7fdd9dc931f0>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:08:15.364935 ncdv-master-1 devstack@n-api.service[138681]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-hypervisors:show": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:08:15.364935 ncdv-master-1 devstack@n-api.service[138681]: warnings.warn(msg) Jul 16 21:08:15.366701 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5db58eb1-df5a-4278-b4a2-9178376ce80e tempest-HypervisorAdminV253TestBase-1151073637 tempest-HypervisorAdminV253TestBase-1151073637-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:08:15.367526 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5db58eb1-df5a-4278-b4a2-9178376ce80e tempest-HypervisorAdminV253TestBase-1151073637 tempest-HypervisorAdminV253TestBase-1151073637-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:08:15.373990 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-7291816d-80b1-4b9f-9606-fb545384c006 tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'db6c21ea09c54ea492d806f42d2f69a7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e7c343c974bd4ec0a6a765f818b895be', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:08:15.398044 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-7291816d-80b1-4b9f-9606-fb545384c006 tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] 10.222.0.16 "GET /compute/v2.1/servers/7895aae7-b50f-48cf-a3e1-5b33dc6e16a3" status: 200 len: 1241 microversion: 2.1 time: 0.152639 Jul 16 21:08:15.403753 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5db58eb1-df5a-4278-b4a2-9178376ce80e tempest-HypervisorAdminV253TestBase-1151073637 tempest-HypervisorAdminV253TestBase-1151073637-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:08:15.404425 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5db58eb1-df5a-4278-b4a2-9178376ce80e tempest-HypervisorAdminV253TestBase-1151073637 tempest-HypervisorAdminV253TestBase-1151073637-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:08:15.408884 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 40/76] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:08:15 2022] GET /compute/v2.1/servers/7895aae7-b50f-48cf-a3e1-5b33dc6e16a3 => generated 1241 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:08:15.412654 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-5db58eb1-df5a-4278-b4a2-9178376ce80e tempest-HypervisorAdminV253TestBase-1151073637 tempest-HypervisorAdminV253TestBase-1151073637-project-admin] 10.222.0.16 "GET /compute/v2.1/os-hypervisors/fcc75857-7dde-4ac0-88e4-871d39001997" status: 200 len: 791 microversion: 2.53 time: 0.053743 Jul 16 21:08:15.413266 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 37/77] 10.222.0.16 () {60 vars in 1297 bytes} [Sat Jul 16 21:08:15 2022] GET /compute/v2.1/os-hypervisors/fcc75857-7dde-4ac0-88e4-871d39001997 => generated 791 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 16 21:08:16.422476 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-612db21c-6b46-4a3e-80c2-87113813ca30 tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:08:16.426451 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-612db21c-6b46-4a3e-80c2-87113813ca30 tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] [instance: 7895aae7-b50f-48cf-a3e1-5b33dc6e16a3] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:08:16.433965 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-612db21c-6b46-4a3e-80c2-87113813ca30 tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:08:16.434542 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-612db21c-6b46-4a3e-80c2-87113813ca30 tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:08:16.436847 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-612db21c-6b46-4a3e-80c2-87113813ca30 tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:08:16.437384 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-612db21c-6b46-4a3e-80c2-87113813ca30 tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:08:16.606945 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-612db21c-6b46-4a3e-80c2-87113813ca30 tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'db6c21ea09c54ea492d806f42d2f69a7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e7c343c974bd4ec0a6a765f818b895be', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:08:16.627210 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-612db21c-6b46-4a3e-80c2-87113813ca30 tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] 10.222.0.16 "GET /compute/v2.1/servers/7895aae7-b50f-48cf-a3e1-5b33dc6e16a3" status: 200 len: 1289 microversion: 2.1 time: 0.213986 Jul 16 21:08:16.638890 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 41/78] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:08:16 2022] GET /compute/v2.1/servers/7895aae7-b50f-48cf-a3e1-5b33dc6e16a3 => generated 1289 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:08:17.647798 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-fb641a4f-5411-43e7-b454-66be8dfbec4b tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:08:17.649450 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-fb641a4f-5411-43e7-b454-66be8dfbec4b tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] [instance: 7895aae7-b50f-48cf-a3e1-5b33dc6e16a3] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:08:17.663494 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-fb641a4f-5411-43e7-b454-66be8dfbec4b tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:08:17.673820 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-fb641a4f-5411-43e7-b454-66be8dfbec4b tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.010s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:08:17.675961 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-fb641a4f-5411-43e7-b454-66be8dfbec4b tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:08:17.676676 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-fb641a4f-5411-43e7-b454-66be8dfbec4b tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:08:17.905498 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-fb641a4f-5411-43e7-b454-66be8dfbec4b tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'db6c21ea09c54ea492d806f42d2f69a7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e7c343c974bd4ec0a6a765f818b895be', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:08:17.933499 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-fb641a4f-5411-43e7-b454-66be8dfbec4b tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] 10.222.0.16 "GET /compute/v2.1/servers/7895aae7-b50f-48cf-a3e1-5b33dc6e16a3" status: 200 len: 1295 microversion: 2.1 time: 0.285312 Jul 16 21:08:17.939119 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 42/79] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:08:17 2022] GET /compute/v2.1/servers/7895aae7-b50f-48cf-a3e1-5b33dc6e16a3 => generated 1295 bytes in 298 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:08:18.360415 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-9358d941-dc77-41cf-9476-0815cee5f25a tempest-InstanceUsageAuditLogNegativeTestJSON-152173970 tempest-InstanceUsageAuditLogNegativeTestJSON-152173970-project-admin] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:08:18.363989 ncdv-master-1 devstack@n-api.service[138681]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-instance-usage-audit-log:show": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:08:18.363989 ncdv-master-1 devstack@n-api.service[138681]: warnings.warn(msg) Jul 16 21:08:18.363989 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-9358d941-dc77-41cf-9476-0815cee5f25a tempest-InstanceUsageAuditLogNegativeTestJSON-152173970 tempest-InstanceUsageAuditLogNegativeTestJSON-152173970-project-admin] HTTP exception thrown: Invalid timestamp for date invalid_time Jul 16 21:08:18.375206 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-9358d941-dc77-41cf-9476-0815cee5f25a tempest-InstanceUsageAuditLogNegativeTestJSON-152173970 tempest-InstanceUsageAuditLogNegativeTestJSON-152173970-project-admin] Request method failure captured: Jul 16 21:08:18.375206 ncdv-master-1 devstack@n-api.service[138681]: request: GET /compute/v2.1/os-instance_usage_audit_log/invalid_time HTTP/1.1 Jul 16 21:08:18.375206 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:08:18.375206 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:08:18.375206 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:08:18.375206 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 21:08:18.375206 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:08:18.375206 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:08:18.375206 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:08:18.375206 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:08:18.375206 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0yjBl-s3PY2HZBg2P8FRT5G8eHYNqVFWS8RvOWGQ5LklVWT-lLVxSsigb700ogfFwA1ew1oSNhKAylHHSZTfP_4XCkkTw9bJyoOHXTGX4GZIyCdyaYjikLo3cec7m4ufV9IlwjyuBPIt-tGil5ozelDXr0ae1t3GA8v6X3aU7N_neLU Jul 16 21:08:18.375206 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:08:18.375206 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:08:18.375206 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:08:18.375206 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:08:18.375206 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:08:18.375206 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:08:18.375206 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: 427515e67f884dbe988c183fa82424eb Jul 16 21:08:18.375206 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-InstanceUsageAuditLogNegativeTestJSON-152173970 Jul 16 21:08:18.375206 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,admin,reader Jul 16 21:08:18.375206 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,admin,reader Jul 16 21:08:18.376701 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/427515e67f884dbe988c183fa82424eb"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/427515e67f884dbe988c183fa82424eb"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_427515e67f884dbe988c183fa82424eb"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/427515e67f884dbe988c183fa82424eb"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:08:18.376701 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-InstanceUsageAuditLogNegativeTestJSON-152173970 Jul 16 21:08:18.376701 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: 427515e67f884dbe988c183fa82424eb Jul 16 21:08:18.376701 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-InstanceUsageAuditLogNegativeTestJSON-152173970 Jul 16 21:08:18.376701 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-InstanceUsageAuditLogNegativeTestJSON-152173970-project-admin Jul 16 21:08:18.376701 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:08:18.376701 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:08:18.376701 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: eeef7e8219ec4bd1b463112fe6dfb740 Jul 16 21:08:18.376701 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-InstanceUsageAuditLogNegativeTestJSON-152173970-project-admin Jul 16 21:08:18.376701 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:08:18.376701 ncdv-master-1 devstack@n-api.service[138681]: exception: Invalid timestamp for date invalid_time Jul 16 21:08:18.376701 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': 'invalid_time'} Jul 16 21:08:18.377692 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:08:18.377692 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:08:18.377692 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/instance_usage_audit_log.py", line 51, in show Jul 16 21:08:18.377692 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi before_date = datetime.datetime.strptime(str(id), Jul 16 21:08:18.377692 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/lib/python3.8/_strptime.py", line 568, in _strptime_datetime Jul 16 21:08:18.377692 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi tt, fraction, gmtoff_fraction = _strptime(data_string, format) Jul 16 21:08:18.377692 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/lib/python3.8/_strptime.py", line 349, in _strptime Jul 16 21:08:18.377692 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise ValueError("time data %r does not match format %r" % Jul 16 21:08:18.377692 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi ValueError: time data 'invalid_time' does not match format '%Y-%m-%d %H:%M:%S' Jul 16 21:08:18.377692 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:08:18.377692 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:08:18.377692 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:08:18.377692 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:08:18.377692 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:08:18.377692 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:08:18.377692 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:08:18.377692 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:08:18.377692 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:08:18.377692 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:08:18.377692 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/instance_usage_audit_log.py", line 55, in show Jul 16 21:08:18.379103 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPBadRequest(explanation=msg) Jul 16 21:08:18.379103 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid timestamp for date invalid_time Jul 16 21:08:18.379103 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:08:18.379103 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:08:18.379103 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:08:18.379103 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:08:18.379103 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:08:18.379103 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:08:18.379103 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:08:18.379103 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:08:18.379103 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid timestamp for date invalid_time Jul 16 21:08:18.379103 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:08:18.381789 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-9358d941-dc77-41cf-9476-0815cee5f25a tempest-InstanceUsageAuditLogNegativeTestJSON-152173970 tempest-InstanceUsageAuditLogNegativeTestJSON-152173970-project-admin] Returning 400 to user: Invalid timestamp for date invalid_time {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:08:18.383575 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-9358d941-dc77-41cf-9476-0815cee5f25a tempest-InstanceUsageAuditLogNegativeTestJSON-152173970 tempest-InstanceUsageAuditLogNegativeTestJSON-152173970-project-admin] 10.222.0.16 "GET /compute/v2.1/os-instance_usage_audit_log/invalid_time" status: 400 len: 83 microversion: 2.1 time: 0.854270 Jul 16 21:08:18.385076 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 38/80] 10.222.0.16 () {58 vars in 1223 bytes} [Sat Jul 16 21:08:17 2022] GET /compute/v2.1/os-instance_usage_audit_log/invalid_time => generated 83 bytes in 857 msecs (HTTP/1.1 400) 9 headers in 380 bytes (1 switches on core 0) Jul 16 21:08:18.978158 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-3a93a820-3164-4aa6-85ca-270f464327fd tempest-InstanceUsageAuditLogNegativeTestJSON-562700382 tempest-InstanceUsageAuditLogNegativeTestJSON-562700382-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:08:18.979041 ncdv-master-1 devstack@n-api.service[138680]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-instance-usage-audit-log:list": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:08:18.979041 ncdv-master-1 devstack@n-api.service[138680]: warnings.warn(msg) Jul 16 21:08:18.980448 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-3a93a820-3164-4aa6-85ca-270f464327fd tempest-InstanceUsageAuditLogNegativeTestJSON-562700382 tempest-InstanceUsageAuditLogNegativeTestJSON-562700382-project] Policy check for os_compute_api:os-instance-usage-audit-log:list failed with credentials {'is_admin': False, 'user_id': '134751891f45482c8c5e7584ac1ea522', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ed6998e80f4044fc8bb56a23ac75a4a6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:08:18.987996 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-3a93a820-3164-4aa6-85ca-270f464327fd tempest-InstanceUsageAuditLogNegativeTestJSON-562700382 tempest-InstanceUsageAuditLogNegativeTestJSON-562700382-project] Request method failure captured: Jul 16 21:08:18.987996 ncdv-master-1 devstack@n-api.service[138680]: request: GET /compute/v2.1/os-instance_usage_audit_log HTTP/1.1 Jul 16 21:08:18.987996 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:08:18.987996 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:08:18.987996 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:08:18.987996 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 0 Jul 16 21:08:18.987996 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:08:18.987996 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:08:18.987996 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:08:18.987996 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:08:18.987996 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0yi-bMPEf-wh4F6wE_dooNvPGF8ySLjMR-xbpCc65BU9DA4VT7o8IVv11ZpXG5iRQ3TeMhWPsA_rL3HU8w58Uaz7OPLy5I-vKnvAehEvInhT7tG8GuqgsGATCpY3FW2aDdMmeIO4svZ4l2PIM3ABaXoFNQJW6XehAIPC7t5weoVic4I Jul 16 21:08:18.987996 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:08:18.987996 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:08:18.987996 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:08:18.987996 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:08:18.987996 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:08:18.987996 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:08:18.987996 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: ed6998e80f4044fc8bb56a23ac75a4a6 Jul 16 21:08:18.987996 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-InstanceUsageAuditLogNegativeTestJSON-562700382 Jul 16 21:08:18.987996 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:08:18.987996 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:08:18.989078 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/ed6998e80f4044fc8bb56a23ac75a4a6"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/ed6998e80f4044fc8bb56a23ac75a4a6"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_ed6998e80f4044fc8bb56a23ac75a4a6"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/ed6998e80f4044fc8bb56a23ac75a4a6"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:08:18.989078 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-InstanceUsageAuditLogNegativeTestJSON-562700382 Jul 16 21:08:18.989078 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: ed6998e80f4044fc8bb56a23ac75a4a6 Jul 16 21:08:18.989078 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-InstanceUsageAuditLogNegativeTestJSON-562700382 Jul 16 21:08:18.989078 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-InstanceUsageAuditLogNegativeTestJSON-562700382-project Jul 16 21:08:18.989078 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:08:18.989078 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:08:18.989078 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: 134751891f45482c8c5e7584ac1ea522 Jul 16 21:08:18.989078 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-InstanceUsageAuditLogNegativeTestJSON-562700382-project Jul 16 21:08:18.989078 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 21:08:18.989078 ncdv-master-1 devstack@n-api.service[138680]: exception: Policy doesn't allow os_compute_api:os-instance-usage-audit-log:list to be performed. Jul 16 21:08:18.989664 ncdv-master-1 devstack@n-api.service[138680]: action_args: {} Jul 16 21:08:18.989664 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:08:18.989664 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:08:18.989664 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:08:18.989664 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:08:18.989664 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:08:18.989664 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:08:18.989664 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:08:18.989664 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:08:18.989664 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/instance_usage_audit_log.py", line 38, in index Jul 16 21:08:18.989664 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi context.can(iual_policies.BASE_POLICY_NAME % 'list', target={}) Jul 16 21:08:18.989664 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/context.py", line 240, in can Jul 16 21:08:18.989664 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return policy.authorize(self, action, target) Jul 16 21:08:18.989664 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 199, in authorize Jul 16 21:08:18.989664 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi LOG.debug('Policy check for %(action)s failed with credentials ' Jul 16 21:08:18.989664 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 16 21:08:18.989664 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi self.force_reraise() Jul 16 21:08:18.989664 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 16 21:08:18.989664 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise self.value Jul 16 21:08:18.989664 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 186, in authorize Jul 16 21:08:18.990659 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi result = _ENFORCER.authorize(action, target, context, Jul 16 21:08:18.990659 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1184, in authorize Jul 16 21:08:18.990659 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return self.enforce( Jul 16 21:08:18.990659 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1088, in enforce Jul 16 21:08:18.990659 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc(*args, **kwargs) Jul 16 21:08:18.990659 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.PolicyNotAuthorized: Policy doesn't allow os_compute_api:os-instance-usage-audit-log:list to be performed. Jul 16 21:08:18.990659 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:08:18.990659 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:08:18.990659 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:08:18.990659 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:08:18.990659 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:08:18.990659 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:08:18.990659 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 354, in __exit__ Jul 16 21:08:18.990659 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(webob.exc.HTTPForbidden( Jul 16 21:08:18.990659 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Policy doesn't allow os_compute_api:os-instance-usage-audit-log:list to be performed. Jul 16 21:08:18.990659 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:08:18.990659 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-3a93a820-3164-4aa6-85ca-270f464327fd tempest-InstanceUsageAuditLogNegativeTestJSON-562700382 tempest-InstanceUsageAuditLogNegativeTestJSON-562700382-project] Returning 403 to user: Policy doesn't allow os_compute_api:os-instance-usage-audit-log:list to be performed. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:08:18.998060 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-3a93a820-3164-4aa6-85ca-270f464327fd tempest-InstanceUsageAuditLogNegativeTestJSON-562700382 tempest-InstanceUsageAuditLogNegativeTestJSON-562700382-project] 10.222.0.16 "GET /compute/v2.1/os-instance_usage_audit_log" status: 403 len: 128 microversion: 2.1 time: 0.606792 Jul 16 21:08:18.999228 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 43/81] 10.222.0.16 () {58 vars in 1184 bytes} [Sat Jul 16 21:08:18 2022] GET /compute/v2.1/os-instance_usage_audit_log => generated 128 bytes in 609 msecs (HTTP/1.1 403) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:08:19.009711 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-1e6bd085-1024-4aa9-a053-b0665bd9d8b6 tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:08:19.011794 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-1e6bd085-1024-4aa9-a053-b0665bd9d8b6 tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] [instance: 7895aae7-b50f-48cf-a3e1-5b33dc6e16a3] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:08:19.025582 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1e6bd085-1024-4aa9-a053-b0665bd9d8b6 tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:08:19.029013 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1e6bd085-1024-4aa9-a053-b0665bd9d8b6 tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:08:19.029013 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1e6bd085-1024-4aa9-a053-b0665bd9d8b6 tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:08:19.029329 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1e6bd085-1024-4aa9-a053-b0665bd9d8b6 tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:08:19.140272 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-1e6bd085-1024-4aa9-a053-b0665bd9d8b6 tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'db6c21ea09c54ea492d806f42d2f69a7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e7c343c974bd4ec0a6a765f818b895be', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:08:19.160495 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-1e6bd085-1024-4aa9-a053-b0665bd9d8b6 tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] 10.222.0.16 "GET /compute/v2.1/servers/7895aae7-b50f-48cf-a3e1-5b33dc6e16a3" status: 200 len: 1295 microversion: 2.1 time: 0.159482 Jul 16 21:08:19.166095 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 44/82] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:08:18 2022] GET /compute/v2.1/servers/7895aae7-b50f-48cf-a3e1-5b33dc6e16a3 => generated 1295 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:08:19.181199 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-2bc48f2a-86a9-4479-95ea-7403ee5cabc3 tempest-InstanceUsageAuditLogNegativeTestJSON-562700382 tempest-InstanceUsageAuditLogNegativeTestJSON-562700382-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:08:19.182353 ncdv-master-1 devstack@n-api.service[138680]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-instance-usage-audit-log:show": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:08:19.182353 ncdv-master-1 devstack@n-api.service[138680]: warnings.warn(msg) Jul 16 21:08:19.186526 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-2bc48f2a-86a9-4479-95ea-7403ee5cabc3 tempest-InstanceUsageAuditLogNegativeTestJSON-562700382 tempest-InstanceUsageAuditLogNegativeTestJSON-562700382-project] Policy check for os_compute_api:os-instance-usage-audit-log:show failed with credentials {'is_admin': False, 'user_id': '134751891f45482c8c5e7584ac1ea522', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ed6998e80f4044fc8bb56a23ac75a4a6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:08:19.189273 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-2bc48f2a-86a9-4479-95ea-7403ee5cabc3 tempest-InstanceUsageAuditLogNegativeTestJSON-562700382 tempest-InstanceUsageAuditLogNegativeTestJSON-562700382-project] Request method failure captured: Jul 16 21:08:19.189273 ncdv-master-1 devstack@n-api.service[138680]: request: GET /compute/v2.1/os-instance_usage_audit_log/2022-07-16%2021:08:19 HTTP/1.1 Jul 16 21:08:19.189273 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:08:19.189273 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:08:19.189273 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:08:19.189273 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 0 Jul 16 21:08:19.189273 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:08:19.189273 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:08:19.189273 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:08:19.189273 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:08:19.189273 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0yi-bMPEf-wh4F6wE_dooNvPGF8ySLjMR-xbpCc65BU9DA4VT7o8IVv11ZpXG5iRQ3TeMhWPsA_rL3HU8w58Uaz7OPLy5I-vKnvAehEvInhT7tG8GuqgsGATCpY3FW2aDdMmeIO4svZ4l2PIM3ABaXoFNQJW6XehAIPC7t5weoVic4I Jul 16 21:08:19.189273 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:08:19.189273 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:08:19.189273 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:08:19.189273 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:08:19.189273 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:08:19.189273 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:08:19.189273 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: ed6998e80f4044fc8bb56a23ac75a4a6 Jul 16 21:08:19.189273 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-InstanceUsageAuditLogNegativeTestJSON-562700382 Jul 16 21:08:19.189273 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:08:19.189273 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:08:19.193523 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/ed6998e80f4044fc8bb56a23ac75a4a6"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/ed6998e80f4044fc8bb56a23ac75a4a6"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_ed6998e80f4044fc8bb56a23ac75a4a6"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/ed6998e80f4044fc8bb56a23ac75a4a6"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:08:19.193523 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-InstanceUsageAuditLogNegativeTestJSON-562700382 Jul 16 21:08:19.193523 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: ed6998e80f4044fc8bb56a23ac75a4a6 Jul 16 21:08:19.193523 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-InstanceUsageAuditLogNegativeTestJSON-562700382 Jul 16 21:08:19.193523 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-InstanceUsageAuditLogNegativeTestJSON-562700382-project Jul 16 21:08:19.193523 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:08:19.193523 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:08:19.193523 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: 134751891f45482c8c5e7584ac1ea522 Jul 16 21:08:19.193523 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-InstanceUsageAuditLogNegativeTestJSON-562700382-project Jul 16 21:08:19.193523 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 21:08:19.193523 ncdv-master-1 devstack@n-api.service[138680]: exception: Policy doesn't allow os_compute_api:os-instance-usage-audit-log:show to be performed. Jul 16 21:08:19.194213 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': '2022-07-16 21:08:19'} Jul 16 21:08:19.194213 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:08:19.194213 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:08:19.194213 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:08:19.194213 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:08:19.194213 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:08:19.194213 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:08:19.194213 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:08:19.194213 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:08:19.194213 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/instance_usage_audit_log.py", line 45, in show Jul 16 21:08:19.194213 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi context.can(iual_policies.BASE_POLICY_NAME % 'show', target={}) Jul 16 21:08:19.194213 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/context.py", line 240, in can Jul 16 21:08:19.194213 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return policy.authorize(self, action, target) Jul 16 21:08:19.194213 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 199, in authorize Jul 16 21:08:19.194213 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi LOG.debug('Policy check for %(action)s failed with credentials ' Jul 16 21:08:19.194213 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 16 21:08:19.194213 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi self.force_reraise() Jul 16 21:08:19.194213 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 16 21:08:19.194213 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise self.value Jul 16 21:08:19.194213 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 186, in authorize Jul 16 21:08:19.195259 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi result = _ENFORCER.authorize(action, target, context, Jul 16 21:08:19.195259 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1184, in authorize Jul 16 21:08:19.195259 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return self.enforce( Jul 16 21:08:19.195259 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1088, in enforce Jul 16 21:08:19.195259 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc(*args, **kwargs) Jul 16 21:08:19.195259 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.PolicyNotAuthorized: Policy doesn't allow os_compute_api:os-instance-usage-audit-log:show to be performed. Jul 16 21:08:19.195259 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:08:19.195259 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:08:19.195259 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:08:19.195259 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:08:19.195259 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:08:19.195259 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:08:19.195259 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 354, in __exit__ Jul 16 21:08:19.195259 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(webob.exc.HTTPForbidden( Jul 16 21:08:19.195259 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Policy doesn't allow os_compute_api:os-instance-usage-audit-log:show to be performed. Jul 16 21:08:19.195259 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:08:19.200386 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-2bc48f2a-86a9-4479-95ea-7403ee5cabc3 tempest-InstanceUsageAuditLogNegativeTestJSON-562700382 tempest-InstanceUsageAuditLogNegativeTestJSON-562700382-project] Returning 403 to user: Policy doesn't allow os_compute_api:os-instance-usage-audit-log:show to be performed. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:08:19.201270 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-2bc48f2a-86a9-4479-95ea-7403ee5cabc3 tempest-InstanceUsageAuditLogNegativeTestJSON-562700382 tempest-InstanceUsageAuditLogNegativeTestJSON-562700382-project] 10.222.0.16 "GET /compute/v2.1/os-instance_usage_audit_log/2022-07-16 21:08:19" status: 403 len: 128 microversion: 2.1 time: 0.033680 Jul 16 21:08:19.202155 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 45/83] 10.222.0.16 () {58 vars in 1250 bytes} [Sat Jul 16 21:08:19 2022] GET /compute/v2.1/os-instance_usage_audit_log/2022-07-16%2021%3A08%3A19 => generated 128 bytes in 35 msecs (HTTP/1.1 403) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:08:19.235709 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell0:SELECT=2 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:08:19.379301 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-24c036ab-c3f1-4523-bcc7-10d4f3b0c226 tempest-FlavorsExtraSpecsTestJSON-2120042937 tempest-FlavorsExtraSpecsTestJSON-2120042937-project-admin] Action: 'create', calling method: >, body: {"flavor": {"name": "tempest-test_flavor-25915267", "ram": 512, "vcpus": 1, "disk": 10, "id": 480530849, "swap": 1024, "rxtx_factor": 1, "OS-FLV-EXT-DATA:ephemeral": 10}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:08:19.382502 ncdv-master-1 devstack@n-api.service[138681]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-flavor-manage:create": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:08:19.382502 ncdv-master-1 devstack@n-api.service[138681]: warnings.warn(msg) Jul 16 21:08:19.472638 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-24c036ab-c3f1-4523-bcc7-10d4f3b0c226 tempest-FlavorsExtraSpecsTestJSON-2120042937 tempest-FlavorsExtraSpecsTestJSON-2120042937-project-admin] 10.222.0.16 "POST /compute/v2.1/flavors" status: 200 len: 411 microversion: 2.1 time: 0.926993 Jul 16 21:08:19.486684 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-08f3c092-5662-45c2-964a-617a66e8e11a tempest-FlavorsExtraSpecsTestJSON-2120042937 tempest-FlavorsExtraSpecsTestJSON-2120042937-project-admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw:numa_nodes": "1", "hw:cpu_policy": "shared"}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:08:19.493922 ncdv-master-1 devstack@n-api.service[138680]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-flavor-extra-specs:create": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:08:19.493922 ncdv-master-1 devstack@n-api.service[138680]: warnings.warn(msg) Jul 16 21:08:19.494246 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 39/84] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 21:08:18 2022] POST /compute/v2.1/flavors => generated 411 bytes in 948 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:08:19.608204 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-08f3c092-5662-45c2-964a-617a66e8e11a tempest-FlavorsExtraSpecsTestJSON-2120042937 tempest-FlavorsExtraSpecsTestJSON-2120042937-project-admin] 10.222.0.16 "POST /compute/v2.1/flavors/480530849/os-extra_specs" status: 200 len: 66 microversion: 2.1 time: 0.125615 Jul 16 21:08:19.621013 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 46/85] 10.222.0.16 () {60 vars in 1220 bytes} [Sat Jul 16 21:08:19 2022] POST /compute/v2.1/flavors/480530849/os-extra_specs => generated 66 bytes in 133 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Jul 16 21:08:20.198950 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-15d3e776-0eb7-41d8-b785-6b196b98e409 tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:08:20.202217 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-15d3e776-0eb7-41d8-b785-6b196b98e409 tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] [instance: 7895aae7-b50f-48cf-a3e1-5b33dc6e16a3] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:08:20.215750 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-15d3e776-0eb7-41d8-b785-6b196b98e409 tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:08:20.220988 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-15d3e776-0eb7-41d8-b785-6b196b98e409 tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:08:20.220988 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-15d3e776-0eb7-41d8-b785-6b196b98e409 tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:08:20.220988 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-15d3e776-0eb7-41d8-b785-6b196b98e409 tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:08:20.563379 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-15d3e776-0eb7-41d8-b785-6b196b98e409 tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'db6c21ea09c54ea492d806f42d2f69a7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e7c343c974bd4ec0a6a765f818b895be', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:08:20.576227 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-15d3e776-0eb7-41d8-b785-6b196b98e409 tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] 10.222.0.16 "GET /compute/v2.1/servers/7895aae7-b50f-48cf-a3e1-5b33dc6e16a3" status: 200 len: 1337 microversion: 2.1 time: 0.394131 Jul 16 21:08:20.587231 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 47/86] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:08:20 2022] GET /compute/v2.1/servers/7895aae7-b50f-48cf-a3e1-5b33dc6e16a3 => generated 1337 bytes in 406 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:08:21.116214 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-22e9afdf-88e4-4eb9-b65c-2941bb873497 tempest-FlavorsExtraSpecsTestJSON-988725370 tempest-FlavorsExtraSpecsTestJSON-988725370-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:08:21.160249 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-22e9afdf-88e4-4eb9-b65c-2941bb873497 tempest-FlavorsExtraSpecsTestJSON-988725370 tempest-FlavorsExtraSpecsTestJSON-988725370-project] 10.222.0.16 "GET /compute/v2.1/flavors/480530849/os-extra_specs" status: 200 len: 66 microversion: 2.1 time: 1.544466 Jul 16 21:08:21.167989 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 40/87] 10.222.0.16 () {58 vars in 1199 bytes} [Sat Jul 16 21:08:19 2022] GET /compute/v2.1/flavors/480530849/os-extra_specs => generated 66 bytes in 1553 msecs (HTTP/1.1 200) 9 headers in 356 bytes (2 switches on core 0) Jul 16 21:08:21.183748 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-de31c986-de5f-4f45-af84-9d4ca156bee6 tempest-FlavorsExtraSpecsTestJSON-2120042937 tempest-FlavorsExtraSpecsTestJSON-2120042937-project-admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw:numa_nodes": "1", "hw:cpu_policy": "shared"}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:08:21.185475 ncdv-master-1 devstack@n-api.service[138680]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-flavor-extra-specs:create": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:08:21.185475 ncdv-master-1 devstack@n-api.service[138680]: warnings.warn(msg) Jul 16 21:08:21.248853 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-de31c986-de5f-4f45-af84-9d4ca156bee6 tempest-FlavorsExtraSpecsTestJSON-2120042937 tempest-FlavorsExtraSpecsTestJSON-2120042937-project-admin] 10.222.0.16 "POST /compute/v2.1/flavors/480530849/os-extra_specs" status: 200 len: 66 microversion: 2.1 time: 0.067571 Jul 16 21:08:21.249860 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 48/88] 10.222.0.16 () {60 vars in 1220 bytes} [Sat Jul 16 21:08:21 2022] POST /compute/v2.1/flavors/480530849/os-extra_specs => generated 66 bytes in 71 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Jul 16 21:08:21.260763 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-d0437291-6cc1-4578-af8e-f5a7a25f1461 tempest-FlavorsExtraSpecsTestJSON-988725370 tempest-FlavorsExtraSpecsTestJSON-988725370-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:08:21.276056 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-d0437291-6cc1-4578-af8e-f5a7a25f1461 tempest-FlavorsExtraSpecsTestJSON-988725370 tempest-FlavorsExtraSpecsTestJSON-988725370-project] 10.222.0.16 "GET /compute/v2.1/flavors/480530849/os-extra_specs/hw:numa_nodes" status: 200 len: 22 microversion: 2.1 time: 0.019425 Jul 16 21:08:21.279365 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 41/89] 10.222.0.16 () {58 vars in 1241 bytes} [Sat Jul 16 21:08:21 2022] GET /compute/v2.1/flavors/480530849/os-extra_specs/hw:numa_nodes => generated 22 bytes in 23 msecs (HTTP/1.1 200) 9 headers in 356 bytes (2 switches on core 0) Jul 16 21:08:21.293972 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-07ed4ac6-5cf9-4d52-b7d0-542ff80cb202 tempest-FlavorsExtraSpecsTestJSON-2120042937 tempest-FlavorsExtraSpecsTestJSON-2120042937-project-admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw:numa_nodes": "1", "hw:cpu_policy": "shared"}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:08:21.335266 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-07ed4ac6-5cf9-4d52-b7d0-542ff80cb202 tempest-FlavorsExtraSpecsTestJSON-2120042937 tempest-FlavorsExtraSpecsTestJSON-2120042937-project-admin] 10.222.0.16 "POST /compute/v2.1/flavors/480530849/os-extra_specs" status: 200 len: 66 microversion: 2.1 time: 0.048436 Jul 16 21:08:21.336415 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 49/90] 10.222.0.16 () {60 vars in 1220 bytes} [Sat Jul 16 21:08:21 2022] POST /compute/v2.1/flavors/480530849/os-extra_specs => generated 66 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Jul 16 21:08:21.346530 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-13931c67-7e5e-4827-a4b7-51903a45bf2d tempest-FlavorsExtraSpecsTestJSON-2120042937 tempest-FlavorsExtraSpecsTestJSON-2120042937-project-admin] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:08:21.358221 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-13931c67-7e5e-4827-a4b7-51903a45bf2d tempest-FlavorsExtraSpecsTestJSON-2120042937 tempest-FlavorsExtraSpecsTestJSON-2120042937-project-admin] 10.222.0.16 "GET /compute/v2.1/flavors/480530849/os-extra_specs" status: 200 len: 66 microversion: 2.1 time: 0.015588 Jul 16 21:08:21.366782 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 42/91] 10.222.0.16 () {58 vars in 1199 bytes} [Sat Jul 16 21:08:21 2022] GET /compute/v2.1/flavors/480530849/os-extra_specs => generated 66 bytes in 25 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Jul 16 21:08:21.385148 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-304ee728-5e80-46b6-b772-53108a97a56f tempest-FlavorsExtraSpecsTestJSON-2120042937 tempest-FlavorsExtraSpecsTestJSON-2120042937-project-admin] Action: 'update', calling method: >, body: {"hw:numa_nodes": "2"} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:08:21.387943 ncdv-master-1 devstack@n-api.service[138680]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-flavor-extra-specs:update": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:08:21.387943 ncdv-master-1 devstack@n-api.service[138680]: warnings.warn(msg) Jul 16 21:08:21.436009 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_api:DELETE=4,nova_api:SELECT=72,nova_api:INSERT=7,nova_api:SAVEPOINT=8,nova_api:RELEASE=8,nova_api:UPDATE=1 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:08:21.560224 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-304ee728-5e80-46b6-b772-53108a97a56f tempest-FlavorsExtraSpecsTestJSON-2120042937 tempest-FlavorsExtraSpecsTestJSON-2120042937-project-admin] 10.222.0.16 "PUT /compute/v2.1/flavors/480530849/os-extra_specs/hw:numa_nodes" status: 200 len: 22 microversion: 2.1 time: 0.184736 Jul 16 21:08:21.562563 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 50/92] 10.222.0.16 () {60 vars in 1261 bytes} [Sat Jul 16 21:08:21 2022] PUT /compute/v2.1/flavors/480530849/os-extra_specs/hw:numa_nodes => generated 22 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Jul 16 21:08:21.571129 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-5d03d522-d556-4a54-80b1-b20254320782 tempest-FlavorsExtraSpecsTestJSON-2120042937 tempest-FlavorsExtraSpecsTestJSON-2120042937-project-admin] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:08:21.604080 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-5d03d522-d556-4a54-80b1-b20254320782 tempest-FlavorsExtraSpecsTestJSON-2120042937 tempest-FlavorsExtraSpecsTestJSON-2120042937-project-admin] 10.222.0.16 "GET /compute/v2.1/flavors/480530849/os-extra_specs" status: 200 len: 66 microversion: 2.1 time: 0.034436 Jul 16 21:08:21.605082 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-c0438cbd-d4ed-4dc8-83d2-354a696f57ab tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:08:21.605933 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 43/93] 10.222.0.16 () {58 vars in 1199 bytes} [Sat Jul 16 21:08:21 2022] GET /compute/v2.1/flavors/480530849/os-extra_specs => generated 66 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Jul 16 21:08:21.606072 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-c0438cbd-d4ed-4dc8-83d2-354a696f57ab tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] [instance: 7895aae7-b50f-48cf-a3e1-5b33dc6e16a3] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:08:21.621234 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c0438cbd-d4ed-4dc8-83d2-354a696f57ab tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:08:21.622202 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c0438cbd-d4ed-4dc8-83d2-354a696f57ab tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:08:21.623774 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c0438cbd-d4ed-4dc8-83d2-354a696f57ab tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:08:21.625253 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-0258855e-c493-4485-866a-4481126a9a7b tempest-FlavorsExtraSpecsTestJSON-2120042937 tempest-FlavorsExtraSpecsTestJSON-2120042937-project-admin] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:08:21.626826 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c0438cbd-d4ed-4dc8-83d2-354a696f57ab tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:08:21.639508 ncdv-master-1 devstack@n-api.service[138681]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-flavor-extra-specs:delete": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:08:21.639508 ncdv-master-1 devstack@n-api.service[138681]: warnings.warn(msg) Jul 16 21:08:21.802314 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-0258855e-c493-4485-866a-4481126a9a7b tempest-FlavorsExtraSpecsTestJSON-2120042937 tempest-FlavorsExtraSpecsTestJSON-2120042937-project-admin] 10.222.0.16 "DELETE /compute/v2.1/flavors/480530849/os-extra_specs/hw:numa_nodes" status: 200 len: 0 microversion: 2.1 time: 0.188596 Jul 16 21:08:21.802948 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 44/94] 10.222.0.16 () {58 vars in 1244 bytes} [Sat Jul 16 21:08:21 2022] DELETE /compute/v2.1/flavors/480530849/os-extra_specs/hw:numa_nodes => generated 0 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 355 bytes (1 switches on core 0) Jul 16 21:08:21.811415 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-60192936-a78f-42bc-90c9-69992c48c0d0 tempest-FlavorsExtraSpecsTestJSON-2120042937 tempest-FlavorsExtraSpecsTestJSON-2120042937-project-admin] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:08:21.942671 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-60192936-a78f-42bc-90c9-69992c48c0d0 tempest-FlavorsExtraSpecsTestJSON-2120042937 tempest-FlavorsExtraSpecsTestJSON-2120042937-project-admin] 10.222.0.16 "DELETE /compute/v2.1/flavors/480530849/os-extra_specs/hw:cpu_policy" status: 200 len: 0 microversion: 2.1 time: 0.134889 Jul 16 21:08:21.943611 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 45/95] 10.222.0.16 () {58 vars in 1244 bytes} [Sat Jul 16 21:08:21 2022] DELETE /compute/v2.1/flavors/480530849/os-extra_specs/hw:cpu_policy => generated 0 bytes in 136 msecs (HTTP/1.1 200) 9 headers in 355 bytes (2 switches on core 0) Jul 16 21:08:21.963738 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-be3dbf73-9f18-46e8-be04-315dab12ea9e tempest-FlavorsExtraSpecsTestJSON-2120042937 tempest-FlavorsExtraSpecsTestJSON-2120042937-project-admin] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:08:21.999118 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-be3dbf73-9f18-46e8-be04-315dab12ea9e tempest-FlavorsExtraSpecsTestJSON-2120042937 tempest-FlavorsExtraSpecsTestJSON-2120042937-project-admin] 10.222.0.16 "GET /compute/v2.1/flavors/480530849/os-extra_specs" status: 200 len: 19 microversion: 2.1 time: 0.051276 Jul 16 21:08:22.004058 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 46/96] 10.222.0.16 () {58 vars in 1199 bytes} [Sat Jul 16 21:08:21 2022] GET /compute/v2.1/flavors/480530849/os-extra_specs => generated 19 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Jul 16 21:08:22.027050 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-3a7e465d-2a65-4adb-afdc-3a2e04065033 tempest-FlavorsExtraSpecsTestJSON-2120042937 tempest-FlavorsExtraSpecsTestJSON-2120042937-project-admin] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:08:22.030124 ncdv-master-1 devstack@n-api.service[138681]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-flavor-manage:delete": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:08:22.030124 ncdv-master-1 devstack@n-api.service[138681]: warnings.warn(msg) Jul 16 21:08:22.156025 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-3a7e465d-2a65-4adb-afdc-3a2e04065033 tempest-FlavorsExtraSpecsTestJSON-2120042937 tempest-FlavorsExtraSpecsTestJSON-2120042937-project-admin] 10.222.0.16 "DELETE /compute/v2.1/flavors/480530849" status: 202 len: 0 microversion: 2.1 time: 0.143615 Jul 16 21:08:22.157887 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 47/97] 10.222.0.16 () {58 vars in 1157 bytes} [Sat Jul 16 21:08:22 2022] DELETE /compute/v2.1/flavors/480530849 => generated 0 bytes in 146 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 21:08:22.166197 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-3d850fa4-868f-44fa-9112-a0677528ce47 tempest-FlavorsExtraSpecsTestJSON-2120042937 tempest-FlavorsExtraSpecsTestJSON-2120042937-project-admin] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:08:22.207912 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-3d850fa4-868f-44fa-9112-a0677528ce47 tempest-FlavorsExtraSpecsTestJSON-2120042937 tempest-FlavorsExtraSpecsTestJSON-2120042937-project-admin] 10.222.0.16 "GET /compute/v2.1/flavors/detail" status: 200 len: 4991 microversion: 2.1 time: 0.044425 Jul 16 21:08:22.210480 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 48/98] 10.222.0.16 () {58 vars in 1145 bytes} [Sat Jul 16 21:08:22 2022] GET /compute/v2.1/flavors/detail => generated 4991 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:08:22.433939 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell0:SELECT=12 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:08:22.702146 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-c0438cbd-d4ed-4dc8-83d2-354a696f57ab tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'db6c21ea09c54ea492d806f42d2f69a7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e7c343c974bd4ec0a6a765f818b895be', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:08:22.713977 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-c0438cbd-d4ed-4dc8-83d2-354a696f57ab tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] 10.222.0.16 "GET /compute/v2.1/servers/7895aae7-b50f-48cf-a3e1-5b33dc6e16a3" status: 200 len: 1337 microversion: 2.1 time: 1.113802 Jul 16 21:08:22.718018 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 51/99] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:08:21 2022] GET /compute/v2.1/servers/7895aae7-b50f-48cf-a3e1-5b33dc6e16a3 => generated 1337 bytes in 1118 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:08:23.512793 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-4a004775-8833-406a-8071-a04aa140a3e2 tempest-FlavorsV255TestJSON-1136979884 tempest-FlavorsV255TestJSON-1136979884-project-admin] Action: 'create', calling method: >, body: {"flavor": {"ram": 512, "vcpus": 1, "disk": 10, "name": "tempest-FlavorsV255TestJSON-flavor-1737767730", "id": 6346025, "os-flavor-access:is_public": "True"}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:08:23.630623 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-4a004775-8833-406a-8071-a04aa140a3e2 tempest-FlavorsV255TestJSON-1136979884 tempest-FlavorsV255TestJSON-1136979884-project-admin] 10.222.0.16 "POST /compute/v2.1/flavors" status: 200 len: 440 microversion: 2.55 time: 0.974490 Jul 16 21:08:23.634331 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 49/100] 10.222.0.16 () {62 vars in 1187 bytes} [Sat Jul 16 21:08:22 2022] POST /compute/v2.1/flavors => generated 440 bytes in 978 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 16 21:08:23.737344 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-ba284aff-9dcf-444e-85d9-847058db6f06 tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:08:23.738328 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-ba284aff-9dcf-444e-85d9-847058db6f06 tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] [instance: 7895aae7-b50f-48cf-a3e1-5b33dc6e16a3] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:08:23.767164 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ba284aff-9dcf-444e-85d9-847058db6f06 tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:08:23.767776 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ba284aff-9dcf-444e-85d9-847058db6f06 tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:08:23.781754 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ba284aff-9dcf-444e-85d9-847058db6f06 tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:08:23.782260 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ba284aff-9dcf-444e-85d9-847058db6f06 tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.013s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:08:24.231922 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell0:SELECT=2 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:08:24.709062 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-ba284aff-9dcf-444e-85d9-847058db6f06 tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'db6c21ea09c54ea492d806f42d2f69a7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e7c343c974bd4ec0a6a765f818b895be', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:08:24.728525 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-ba284aff-9dcf-444e-85d9-847058db6f06 tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] 10.222.0.16 "GET /compute/v2.1/servers/7895aae7-b50f-48cf-a3e1-5b33dc6e16a3" status: 200 len: 1337 microversion: 2.1 time: 0.997117 Jul 16 21:08:24.732353 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 50/101] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:08:23 2022] GET /compute/v2.1/servers/7895aae7-b50f-48cf-a3e1-5b33dc6e16a3 => generated 1337 bytes in 1002 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:08:24.799676 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-d7807641-8e81-4144-a97d-acfbe2b5cc50 tempest-FlavorsV255TestJSON-911094004 tempest-FlavorsV255TestJSON-911094004-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:08:24.826645 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-d7807641-8e81-4144-a97d-acfbe2b5cc50 tempest-FlavorsV255TestJSON-911094004 tempest-FlavorsV255TestJSON-911094004-project] 10.222.0.16 "GET /compute/v2.1/flavors/6346025" status: 200 len: 440 microversion: 2.55 time: 1.185911 Jul 16 21:08:24.827286 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 52/102] 10.222.0.16 () {60 vars in 1189 bytes} [Sat Jul 16 21:08:23 2022] GET /compute/v2.1/flavors/6346025 => generated 440 bytes in 1187 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Jul 16 21:08:24.844279 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-86be6d65-02f9-4dec-b30c-bc61a00433ed tempest-FlavorsV255TestJSON-1136979884 tempest-FlavorsV255TestJSON-1136979884-project-admin] Action: 'update', calling method: .version_select at 0x7fdd9ff2ae50>, body: {"flavor": {"description": "new"}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:08:24.864000 ncdv-master-1 devstack@n-api.service[138680]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-flavor-manage:update": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:08:24.864000 ncdv-master-1 devstack@n-api.service[138680]: warnings.warn(msg) Jul 16 21:08:25.057288 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-86be6d65-02f9-4dec-b30c-bc61a00433ed tempest-FlavorsV255TestJSON-1136979884 tempest-FlavorsV255TestJSON-1136979884-project-admin] 10.222.0.16 "PUT /compute/v2.1/flavors/6346025" status: 200 len: 441 microversion: 2.55 time: 0.211070 Jul 16 21:08:25.058835 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 53/103] 10.222.0.16 () {62 vars in 1209 bytes} [Sat Jul 16 21:08:24 2022] PUT /compute/v2.1/flavors/6346025 => generated 441 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Jul 16 21:08:25.072292 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-42250a67-4195-4c30-8c1b-58efc71abdcb tempest-FlavorsV255TestJSON-911094004 tempest-FlavorsV255TestJSON-911094004-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:08:25.218445 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-42250a67-4195-4c30-8c1b-58efc71abdcb tempest-FlavorsV255TestJSON-911094004 tempest-FlavorsV255TestJSON-911094004-project] 10.222.0.16 "GET /compute/v2.1/flavors/detail" status: 200 len: 5716 microversion: 2.55 time: 0.151231 Jul 16 21:08:25.222261 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 54/104] 10.222.0.16 () {60 vars in 1186 bytes} [Sat Jul 16 21:08:25 2022] GET /compute/v2.1/flavors/detail => generated 5716 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 16 21:08:25.251797 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-71b7955e-28b0-42cd-ad5e-be1f81b23431 tempest-FlavorsV255TestJSON-911094004 tempest-FlavorsV255TestJSON-911094004-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:08:25.322577 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-71b7955e-28b0-42cd-ad5e-be1f81b23431 tempest-FlavorsV255TestJSON-911094004 tempest-FlavorsV255TestJSON-911094004-project] 10.222.0.16 "GET /compute/v2.1/flavors" status: 200 len: 3148 microversion: 2.55 time: 0.085551 Jul 16 21:08:25.328686 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 55/105] 10.222.0.16 () {60 vars in 1165 bytes} [Sat Jul 16 21:08:25 2022] GET /compute/v2.1/flavors => generated 3148 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 16 21:08:25.356409 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-437eca05-c425-41db-baff-1dabd3d4a93b tempest-FlavorsV255TestJSON-1136979884 tempest-FlavorsV255TestJSON-1136979884-project-admin] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:08:25.357533 ncdv-master-1 devstack@n-api.service[138680]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-flavor-manage:delete": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:08:25.357533 ncdv-master-1 devstack@n-api.service[138680]: warnings.warn(msg) Jul 16 21:08:25.402624 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell0:SELECT=8 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:08:25.557117 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-010441f7-79fa-4f8c-874c-5df9c483bc6c tempest-QuotaClassesAdmin257Test-219312212 tempest-QuotaClassesAdmin257Test-219312212-project-admin] Calling method '.version_select at 0x7fdd9dc6a940>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:08:25.557956 ncdv-master-1 devstack@n-api.service[138681]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-quota-class-sets:show": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:08:25.557956 ncdv-master-1 devstack@n-api.service[138681]: warnings.warn(msg) Jul 16 21:08:25.562206 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-437eca05-c425-41db-baff-1dabd3d4a93b tempest-FlavorsV255TestJSON-1136979884 tempest-FlavorsV255TestJSON-1136979884-project-admin] 10.222.0.16 "DELETE /compute/v2.1/flavors/6346025" status: 202 len: 0 microversion: 2.55 time: 0.222321 Jul 16 21:08:25.562754 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 56/106] 10.222.0.16 () {60 vars in 1192 bytes} [Sat Jul 16 21:08:25 2022] DELETE /compute/v2.1/flavors/6346025 => generated 0 bytes in 224 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Jul 16 21:08:25.574828 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-351b4a70-74bf-4bee-bfc6-1674933b4873 tempest-FlavorsV255TestJSON-1136979884 tempest-FlavorsV255TestJSON-1136979884-project-admin] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:08:25.633932 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-010441f7-79fa-4f8c-874c-5df9c483bc6c tempest-QuotaClassesAdmin257Test-219312212 tempest-QuotaClassesAdmin257Test-219312212-project-admin] 10.222.0.16 "GET /compute/v2.1/os-quota-class-sets/default" status: 200 len: 172 microversion: 2.57 time: 0.898649 Jul 16 21:08:25.647150 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 51/107] 10.222.0.16 () {60 vars in 1225 bytes} [Sat Jul 16 21:08:24 2022] GET /compute/v2.1/os-quota-class-sets/default => generated 172 bytes in 912 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 16 21:08:25.663399 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-351b4a70-74bf-4bee-bfc6-1674933b4873 tempest-FlavorsV255TestJSON-1136979884 tempest-FlavorsV255TestJSON-1136979884-project-admin] 10.222.0.16 "GET /compute/v2.1/flavors/detail" status: 200 len: 5285 microversion: 2.55 time: 0.095989 Jul 16 21:08:25.671987 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-357e8210-ce2c-4e1f-81d7-65573d5f228b tempest-QuotaClassesAdmin257Test-219312212 tempest-QuotaClassesAdmin257Test-219312212-project-admin] Action: 'update', calling method: .version_select at 0x7fdd9dc6a790>, body: {"quota_class_set": {"cores": 120, "instances": 110, "key_pairs": 200, "metadata_items": 228, "ram": 51300, "server_group_members": 110, "server_groups": 110}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:08:25.679187 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 57/108] 10.222.0.16 () {60 vars in 1186 bytes} [Sat Jul 16 21:08:25 2022] GET /compute/v2.1/flavors/detail => generated 5285 bytes in 112 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:08:25.680828 ncdv-master-1 devstack@n-api.service[138681]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-quota-class-sets:update": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:08:25.680828 ncdv-master-1 devstack@n-api.service[138681]: warnings.warn(msg) Jul 16 21:08:25.751296 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-c2159900-3b40-4a65-8a07-87c683d624ad tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:08:25.751686 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-c2159900-3b40-4a65-8a07-87c683d624ad tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] [instance: 7895aae7-b50f-48cf-a3e1-5b33dc6e16a3] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:08:25.770953 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c2159900-3b40-4a65-8a07-87c683d624ad tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:08:25.771459 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c2159900-3b40-4a65-8a07-87c683d624ad tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:08:25.772509 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c2159900-3b40-4a65-8a07-87c683d624ad tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:08:25.772972 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c2159900-3b40-4a65-8a07-87c683d624ad tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:08:26.128498 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-c2159900-3b40-4a65-8a07-87c683d624ad tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'db6c21ea09c54ea492d806f42d2f69a7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e7c343c974bd4ec0a6a765f818b895be', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:08:26.138935 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-c2159900-3b40-4a65-8a07-87c683d624ad tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] 10.222.0.16 "GET /compute/v2.1/servers/7895aae7-b50f-48cf-a3e1-5b33dc6e16a3" status: 200 len: 1337 microversion: 2.1 time: 0.392859 Jul 16 21:08:26.150176 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 58/109] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:08:25 2022] GET /compute/v2.1/servers/7895aae7-b50f-48cf-a3e1-5b33dc6e16a3 => generated 1337 bytes in 404 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:08:26.318365 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_api:SELECT=81,nova_api:INSERT=6,nova_api:DELETE=7,nova_api:SAVEPOINT=1,nova_api:RELEASE=1,nova_api:UPDATE=4 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:08:26.788511 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-357e8210-ce2c-4e1f-81d7-65573d5f228b tempest-QuotaClassesAdmin257Test-219312212 tempest-QuotaClassesAdmin257Test-219312212-project-admin] 10.222.0.16 "PUT /compute/v2.1/os-quota-class-sets/default" status: 200 len: 159 microversion: 2.57 time: 1.139853 Jul 16 21:08:26.791393 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 52/110] 10.222.0.16 () {62 vars in 1246 bytes} [Sat Jul 16 21:08:25 2022] PUT /compute/v2.1/os-quota-class-sets/default => generated 159 bytes in 1144 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 16 21:08:26.804546 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-b728f44a-9945-4038-8d01-c6b7307bb17f tempest-QuotaClassesAdmin257Test-219312212 tempest-QuotaClassesAdmin257Test-219312212-project-admin] Calling method '.version_select at 0x7fdd9ddd29d0>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:08:26.805821 ncdv-master-1 devstack@n-api.service[138680]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-quota-class-sets:show": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:08:26.805821 ncdv-master-1 devstack@n-api.service[138680]: warnings.warn(msg) Jul 16 21:08:26.850036 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-b728f44a-9945-4038-8d01-c6b7307bb17f tempest-QuotaClassesAdmin257Test-219312212 tempest-QuotaClassesAdmin257Test-219312212-project-admin] 10.222.0.16 "GET /compute/v2.1/os-quota-class-sets/default" status: 200 len: 176 microversion: 2.57 time: 0.046764 Jul 16 21:08:26.855081 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 59/111] 10.222.0.16 () {60 vars in 1225 bytes} [Sat Jul 16 21:08:26 2022] GET /compute/v2.1/os-quota-class-sets/default => generated 176 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 16 21:08:26.862715 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-b1415f32-2f0b-4330-9dfe-609428bd0d8a tempest-QuotaClassesAdmin257Test-219312212 tempest-QuotaClassesAdmin257Test-219312212-project-admin] Action: 'update', calling method: .version_select at 0x7fdd9dc38820>, body: {"quota_class_set": {"cores": 20, "instances": 10, "key_pairs": 100, "metadata_items": 128, "ram": 51200, "server_group_members": 10, "server_groups": 10}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:08:27.157987 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-62d1bcfb-c1e1-4dd4-aa28-0e8bc5a24a61 tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:08:27.158612 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-62d1bcfb-c1e1-4dd4-aa28-0e8bc5a24a61 tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] [instance: 7895aae7-b50f-48cf-a3e1-5b33dc6e16a3] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:08:27.168788 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-62d1bcfb-c1e1-4dd4-aa28-0e8bc5a24a61 tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:08:27.169808 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-62d1bcfb-c1e1-4dd4-aa28-0e8bc5a24a61 tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:08:27.170040 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-62d1bcfb-c1e1-4dd4-aa28-0e8bc5a24a61 tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:08:27.171250 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-62d1bcfb-c1e1-4dd4-aa28-0e8bc5a24a61 tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:08:27.611415 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-b1415f32-2f0b-4330-9dfe-609428bd0d8a tempest-QuotaClassesAdmin257Test-219312212 tempest-QuotaClassesAdmin257Test-219312212-project-admin] 10.222.0.16 "PUT /compute/v2.1/os-quota-class-sets/default" status: 200 len: 155 microversion: 2.57 time: 0.753211 Jul 16 21:08:27.627923 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 53/112] 10.222.0.16 () {62 vars in 1246 bytes} [Sat Jul 16 21:08:26 2022] PUT /compute/v2.1/os-quota-class-sets/default => generated 155 bytes in 770 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 16 21:08:27.765022 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-62d1bcfb-c1e1-4dd4-aa28-0e8bc5a24a61 tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'db6c21ea09c54ea492d806f42d2f69a7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e7c343c974bd4ec0a6a765f818b895be', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:08:27.779779 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-62d1bcfb-c1e1-4dd4-aa28-0e8bc5a24a61 tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] 10.222.0.16 "GET /compute/v2.1/servers/7895aae7-b50f-48cf-a3e1-5b33dc6e16a3" status: 200 len: 1337 microversion: 2.1 time: 0.626065 Jul 16 21:08:27.802922 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 60/113] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:08:27 2022] GET /compute/v2.1/servers/7895aae7-b50f-48cf-a3e1-5b33dc6e16a3 => generated 1337 bytes in 649 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:08:28.413828 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-17397902-0f1e-468c-87e7-a55e2f3e9ff5 req-b9dd086f-5d44-4799-bf0f-6d71d484073c service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "7895aae7-b50f-48cf-a3e1-5b33dc6e16a3", "tag": "ce71fe98-d12f-40ec-b5fa-d43e42a5edeb"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:08:28.431318 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-17397902-0f1e-468c-87e7-a55e2f3e9ff5 req-b9dd086f-5d44-4799-bf0f-6d71d484073c service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:08:28.432559 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-17397902-0f1e-468c-87e7-a55e2f3e9ff5 req-b9dd086f-5d44-4799-bf0f-6d71d484073c service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:08:28.466416 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-17397902-0f1e-468c-87e7-a55e2f3e9ff5 req-b9dd086f-5d44-4799-bf0f-6d71d484073c service nova] Creating event network-changed:ce71fe98-d12f-40ec-b5fa-d43e42a5edeb for instance 7895aae7-b50f-48cf-a3e1-5b33dc6e16a3 on nch1-master-1 Jul 16 21:08:28.467416 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-17397902-0f1e-468c-87e7-a55e2f3e9ff5 req-b9dd086f-5d44-4799-bf0f-6d71d484073c service nova] Acquired lock "compute-rpcapi-router" {{(pid=138681) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:294}} Jul 16 21:08:28.469374 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6a52eeff-e45b-431d-b71e-750ccbe4d669 None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:08:28.470087 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6a52eeff-e45b-431d-b71e-750ccbe4d669 None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:08:28.472050 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6a52eeff-e45b-431d-b71e-750ccbe4d669 None None] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:08:28.472843 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6a52eeff-e45b-431d-b71e-750ccbe4d669 None None] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:08:28.490907 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.compute.rpcapi [None req-6a52eeff-e45b-431d-b71e-750ccbe4d669 None None] Automatically selected compute RPC version 6.0 from minimum service version 61 Jul 16 21:08:28.493121 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6a52eeff-e45b-431d-b71e-750ccbe4d669 None None] Releasing lock "compute-rpcapi-router" {{(pid=138681) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:312}} Jul 16 21:08:28.529774 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-6a52eeff-e45b-431d-b71e-750ccbe4d669 None None] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.900237 Jul 16 21:08:28.537142 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 54/114] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:08:27 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 908 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:08:28.821587 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-23b2e8eb-7763-43df-99cb-a0baea8a158a tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:08:28.823407 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-23b2e8eb-7763-43df-99cb-a0baea8a158a tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] [instance: 7895aae7-b50f-48cf-a3e1-5b33dc6e16a3] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:08:28.854771 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-23b2e8eb-7763-43df-99cb-a0baea8a158a tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:08:28.854771 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-23b2e8eb-7763-43df-99cb-a0baea8a158a tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:08:28.856195 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-23b2e8eb-7763-43df-99cb-a0baea8a158a tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:08:28.857655 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-23b2e8eb-7763-43df-99cb-a0baea8a158a tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:08:29.551522 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-23b2e8eb-7763-43df-99cb-a0baea8a158a tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'db6c21ea09c54ea492d806f42d2f69a7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e7c343c974bd4ec0a6a765f818b895be', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:08:29.555469 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=100 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:08:29.571801 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-23b2e8eb-7763-43df-99cb-a0baea8a158a tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] 10.222.0.16 "GET /compute/v2.1/servers/7895aae7-b50f-48cf-a3e1-5b33dc6e16a3" status: 200 len: 1337 microversion: 2.1 time: 0.754593 Jul 16 21:08:29.573073 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 61/115] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:08:28 2022] GET /compute/v2.1/servers/7895aae7-b50f-48cf-a3e1-5b33dc6e16a3 => generated 1337 bytes in 759 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:08:30.591861 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-98700c8e-7476-46d7-a500-3594b875c91e tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:08:30.592604 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-98700c8e-7476-46d7-a500-3594b875c91e tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] [instance: 7895aae7-b50f-48cf-a3e1-5b33dc6e16a3] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:08:30.607799 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-98700c8e-7476-46d7-a500-3594b875c91e tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:08:30.610044 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-98700c8e-7476-46d7-a500-3594b875c91e tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:08:30.611299 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-98700c8e-7476-46d7-a500-3594b875c91e tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:08:30.612352 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-98700c8e-7476-46d7-a500-3594b875c91e tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:08:30.829233 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-98700c8e-7476-46d7-a500-3594b875c91e tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'db6c21ea09c54ea492d806f42d2f69a7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e7c343c974bd4ec0a6a765f818b895be', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:08:30.832484 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=57 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:08:30.843107 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-98700c8e-7476-46d7-a500-3594b875c91e tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] 10.222.0.16 "GET /compute/v2.1/servers/7895aae7-b50f-48cf-a3e1-5b33dc6e16a3" status: 200 len: 1337 microversion: 2.1 time: 0.255992 Jul 16 21:08:30.844513 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 55/116] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:08:30 2022] GET /compute/v2.1/servers/7895aae7-b50f-48cf-a3e1-5b33dc6e16a3 => generated 1337 bytes in 258 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:08:31.871103 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-6aaf40a1-b0e0-46fc-bfaf-9eaafe0b2689 tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:08:31.872533 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-6aaf40a1-b0e0-46fc-bfaf-9eaafe0b2689 tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] [instance: 7895aae7-b50f-48cf-a3e1-5b33dc6e16a3] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:08:31.882705 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6aaf40a1-b0e0-46fc-bfaf-9eaafe0b2689 tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:08:31.883319 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6aaf40a1-b0e0-46fc-bfaf-9eaafe0b2689 tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:08:31.884051 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6aaf40a1-b0e0-46fc-bfaf-9eaafe0b2689 tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:08:31.884491 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6aaf40a1-b0e0-46fc-bfaf-9eaafe0b2689 tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:08:32.160888 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-6aaf40a1-b0e0-46fc-bfaf-9eaafe0b2689 tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'db6c21ea09c54ea492d806f42d2f69a7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e7c343c974bd4ec0a6a765f818b895be', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:08:32.169589 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-6aaf40a1-b0e0-46fc-bfaf-9eaafe0b2689 tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] 10.222.0.16 "GET /compute/v2.1/servers/7895aae7-b50f-48cf-a3e1-5b33dc6e16a3" status: 200 len: 1337 microversion: 2.1 time: 0.309198 Jul 16 21:08:32.192337 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 62/117] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:08:31 2022] GET /compute/v2.1/servers/7895aae7-b50f-48cf-a3e1-5b33dc6e16a3 => generated 1337 bytes in 334 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:08:33.189595 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-0edaaa5a-baf9-4d2e-b757-5148907f87d5 tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:08:33.190692 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-0edaaa5a-baf9-4d2e-b757-5148907f87d5 tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] [instance: 7895aae7-b50f-48cf-a3e1-5b33dc6e16a3] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:08:33.198056 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0edaaa5a-baf9-4d2e-b757-5148907f87d5 tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:08:33.198506 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0edaaa5a-baf9-4d2e-b757-5148907f87d5 tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:08:33.199708 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0edaaa5a-baf9-4d2e-b757-5148907f87d5 tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:08:33.200349 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0edaaa5a-baf9-4d2e-b757-5148907f87d5 tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:08:33.451250 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-0edaaa5a-baf9-4d2e-b757-5148907f87d5 tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'db6c21ea09c54ea492d806f42d2f69a7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e7c343c974bd4ec0a6a765f818b895be', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:08:33.460313 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-0edaaa5a-baf9-4d2e-b757-5148907f87d5 tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] 10.222.0.16 "GET /compute/v2.1/servers/7895aae7-b50f-48cf-a3e1-5b33dc6e16a3" status: 200 len: 1337 microversion: 2.1 time: 0.276861 Jul 16 21:08:33.461829 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 56/118] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:08:33 2022] GET /compute/v2.1/servers/7895aae7-b50f-48cf-a3e1-5b33dc6e16a3 => generated 1337 bytes in 279 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:08:34.478690 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-db7f50be-cf04-40cd-b6d7-b46b3a53a868 tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:08:34.479426 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-db7f50be-cf04-40cd-b6d7-b46b3a53a868 tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] [instance: 7895aae7-b50f-48cf-a3e1-5b33dc6e16a3] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:08:34.488132 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-db7f50be-cf04-40cd-b6d7-b46b3a53a868 tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:08:34.488588 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-db7f50be-cf04-40cd-b6d7-b46b3a53a868 tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:08:34.489375 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-db7f50be-cf04-40cd-b6d7-b46b3a53a868 tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:08:34.490017 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-db7f50be-cf04-40cd-b6d7-b46b3a53a868 tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:08:34.960875 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-d28b24a0-99e2-45fd-81ba-7695ec63834b tempest-HypervisorAdminNegativeTestJSON-405949889 tempest-HypervisorAdminNegativeTestJSON-405949889-project] Calling method '.version_select at 0x7fdd9dd635e0>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:08:34.961944 ncdv-master-1 devstack@n-api.service[138680]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-hypervisors:list-detail": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:08:34.961944 ncdv-master-1 devstack@n-api.service[138680]: warnings.warn(msg) Jul 16 21:08:34.963197 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-d28b24a0-99e2-45fd-81ba-7695ec63834b tempest-HypervisorAdminNegativeTestJSON-405949889 tempest-HypervisorAdminNegativeTestJSON-405949889-project] Policy check for os_compute_api:os-hypervisors:list-detail failed with credentials {'is_admin': False, 'user_id': '3f63a1fb362b462fa8f36f5a53cb6cf1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55dc74e9c8884796818fb12bd31f449a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:08:34.967256 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-db7f50be-cf04-40cd-b6d7-b46b3a53a868 tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'db6c21ea09c54ea492d806f42d2f69a7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e7c343c974bd4ec0a6a765f818b895be', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:08:34.970521 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-d28b24a0-99e2-45fd-81ba-7695ec63834b tempest-HypervisorAdminNegativeTestJSON-405949889 tempest-HypervisorAdminNegativeTestJSON-405949889-project] Request method failure captured: Jul 16 21:08:34.970521 ncdv-master-1 devstack@n-api.service[138680]: request: GET /compute/v2.1/os-hypervisors/detail HTTP/1.1 Jul 16 21:08:34.970521 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:08:34.970521 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:08:34.970521 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:08:34.970521 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 0 Jul 16 21:08:34.970521 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:08:34.970521 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:08:34.970521 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:08:34.970521 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:08:34.970521 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0yjM6liv1yp4mDCVvKiko1fgf31eifSlomvSV2MXttjD4_OpO5ge4d2xQumH0tIylebwVbwXYOZJuv5e3a4Un9EvjwJbpZLZEnEvJm2m_TI54EydFD_mwjadcMbsubo2MADqUVLILyWEJ19voqly0yyvKuOp9kP1vn0swY38nYHTwSo Jul 16 21:08:34.970521 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:08:34.970521 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:08:34.970521 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:08:34.970521 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:08:34.970521 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:08:34.970521 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:08:34.970521 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: 55dc74e9c8884796818fb12bd31f449a Jul 16 21:08:34.970521 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-HypervisorAdminNegativeTestJSON-405949889 Jul 16 21:08:34.970521 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:08:34.970521 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:08:34.972424 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/55dc74e9c8884796818fb12bd31f449a"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/55dc74e9c8884796818fb12bd31f449a"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_55dc74e9c8884796818fb12bd31f449a"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/55dc74e9c8884796818fb12bd31f449a"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:08:34.972424 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-HypervisorAdminNegativeTestJSON-405949889 Jul 16 21:08:34.972424 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: 55dc74e9c8884796818fb12bd31f449a Jul 16 21:08:34.972424 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-HypervisorAdminNegativeTestJSON-405949889 Jul 16 21:08:34.972424 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-HypervisorAdminNegativeTestJSON-405949889-project Jul 16 21:08:34.972424 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:08:34.972424 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:08:34.972424 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: 3f63a1fb362b462fa8f36f5a53cb6cf1 Jul 16 21:08:34.972424 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-HypervisorAdminNegativeTestJSON-405949889-project Jul 16 21:08:34.972424 ncdv-master-1 devstack@n-api.service[138680]: method: Jul 16 21:08:34.972424 ncdv-master-1 devstack@n-api.service[138680]: exception: Policy doesn't allow os_compute_api:os-hypervisors:list-detail to be performed. Jul 16 21:08:34.972424 ncdv-master-1 devstack@n-api.service[138680]: action_args: {} Jul 16 21:08:34.972424 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:08:34.973298 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:08:34.973298 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:08:34.973298 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:08:34.973298 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:08:34.973298 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:08:34.973298 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Jul 16 21:08:34.973298 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Jul 16 21:08:34.973298 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:08:34.973298 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:08:34.973298 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/hypervisors.py", line 285, in detail Jul 16 21:08:34.973298 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return self._detail(req) Jul 16 21:08:34.973298 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/hypervisors.py", line 289, in _detail Jul 16 21:08:34.973298 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi context.can(hv_policies.BASE_POLICY_NAME % 'list-detail', target={}) Jul 16 21:08:34.973298 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/context.py", line 240, in can Jul 16 21:08:34.973298 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return policy.authorize(self, action, target) Jul 16 21:08:34.973298 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 199, in authorize Jul 16 21:08:34.973298 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi LOG.debug('Policy check for %(action)s failed with credentials ' Jul 16 21:08:34.973298 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 16 21:08:34.973298 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi self.force_reraise() Jul 16 21:08:34.974467 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 16 21:08:34.974467 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise self.value Jul 16 21:08:34.974467 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 186, in authorize Jul 16 21:08:34.974467 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi result = _ENFORCER.authorize(action, target, context, Jul 16 21:08:34.974467 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1184, in authorize Jul 16 21:08:34.974467 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return self.enforce( Jul 16 21:08:34.974467 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1088, in enforce Jul 16 21:08:34.974467 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc(*args, **kwargs) Jul 16 21:08:34.974467 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.PolicyNotAuthorized: Policy doesn't allow os_compute_api:os-hypervisors:list-detail to be performed. Jul 16 21:08:34.974467 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:08:34.974467 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:08:34.974467 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:08:34.974467 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:08:34.974467 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:08:34.974467 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:08:34.974467 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 354, in __exit__ Jul 16 21:08:34.974467 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(webob.exc.HTTPForbidden( Jul 16 21:08:34.974467 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Policy doesn't allow os_compute_api:os-hypervisors:list-detail to be performed. Jul 16 21:08:34.974467 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:08:34.975774 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-d28b24a0-99e2-45fd-81ba-7695ec63834b tempest-HypervisorAdminNegativeTestJSON-405949889 tempest-HypervisorAdminNegativeTestJSON-405949889-project] Returning 403 to user: Policy doesn't allow os_compute_api:os-hypervisors:list-detail to be performed. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:08:34.975774 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-d28b24a0-99e2-45fd-81ba-7695ec63834b tempest-HypervisorAdminNegativeTestJSON-405949889 tempest-HypervisorAdminNegativeTestJSON-405949889-project] 10.222.0.16 "GET /compute/v2.1/os-hypervisors/detail" status: 403 len: 122 microversion: 2.1 time: 0.802156 Jul 16 21:08:34.976905 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 63/119] 10.222.0.16 () {58 vars in 1166 bytes} [Sat Jul 16 21:08:34 2022] GET /compute/v2.1/os-hypervisors/detail => generated 122 bytes in 806 msecs (HTTP/1.1 403) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:08:34.988860 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-db7f50be-cf04-40cd-b6d7-b46b3a53a868 tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] 10.222.0.16 "GET /compute/v2.1/servers/7895aae7-b50f-48cf-a3e1-5b33dc6e16a3" status: 200 len: 1337 microversion: 2.1 time: 0.514567 Jul 16 21:08:34.999047 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 57/120] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:08:34 2022] GET /compute/v2.1/servers/7895aae7-b50f-48cf-a3e1-5b33dc6e16a3 => generated 1337 bytes in 517 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:08:35.000581 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-4230aca5-4729-4c4d-b98a-1ca92221a6d7 tempest-HypervisorAdminNegativeTestJSON-405949889 tempest-HypervisorAdminNegativeTestJSON-405949889-project] Calling method '.version_select at 0x7fdd9dd3f040>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:08:35.001792 ncdv-master-1 devstack@n-api.service[138680]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-hypervisors:list": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:08:35.001792 ncdv-master-1 devstack@n-api.service[138680]: warnings.warn(msg) Jul 16 21:08:35.006022 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-4230aca5-4729-4c4d-b98a-1ca92221a6d7 tempest-HypervisorAdminNegativeTestJSON-405949889 tempest-HypervisorAdminNegativeTestJSON-405949889-project] Policy check for os_compute_api:os-hypervisors:list failed with credentials {'is_admin': False, 'user_id': '3f63a1fb362b462fa8f36f5a53cb6cf1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55dc74e9c8884796818fb12bd31f449a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:08:35.015171 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-4230aca5-4729-4c4d-b98a-1ca92221a6d7 tempest-HypervisorAdminNegativeTestJSON-405949889 tempest-HypervisorAdminNegativeTestJSON-405949889-project] Request method failure captured: Jul 16 21:08:35.015171 ncdv-master-1 devstack@n-api.service[138680]: request: GET /compute/v2.1/os-hypervisors HTTP/1.1 Jul 16 21:08:35.015171 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:08:35.015171 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:08:35.015171 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:08:35.015171 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 0 Jul 16 21:08:35.015171 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:08:35.015171 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:08:35.015171 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:08:35.015171 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:08:35.015171 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0yjM6liv1yp4mDCVvKiko1fgf31eifSlomvSV2MXttjD4_OpO5ge4d2xQumH0tIylebwVbwXYOZJuv5e3a4Un9EvjwJbpZLZEnEvJm2m_TI54EydFD_mwjadcMbsubo2MADqUVLILyWEJ19voqly0yyvKuOp9kP1vn0swY38nYHTwSo Jul 16 21:08:35.015171 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:08:35.015171 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:08:35.015171 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:08:35.015171 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:08:35.015171 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:08:35.015171 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:08:35.015171 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: 55dc74e9c8884796818fb12bd31f449a Jul 16 21:08:35.015171 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-HypervisorAdminNegativeTestJSON-405949889 Jul 16 21:08:35.015171 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:08:35.015171 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:08:35.017561 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/55dc74e9c8884796818fb12bd31f449a"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/55dc74e9c8884796818fb12bd31f449a"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_55dc74e9c8884796818fb12bd31f449a"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/55dc74e9c8884796818fb12bd31f449a"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:08:35.017561 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-HypervisorAdminNegativeTestJSON-405949889 Jul 16 21:08:35.017561 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: 55dc74e9c8884796818fb12bd31f449a Jul 16 21:08:35.017561 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-HypervisorAdminNegativeTestJSON-405949889 Jul 16 21:08:35.017561 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-HypervisorAdminNegativeTestJSON-405949889-project Jul 16 21:08:35.017561 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:08:35.017561 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:08:35.017561 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: 3f63a1fb362b462fa8f36f5a53cb6cf1 Jul 16 21:08:35.017561 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-HypervisorAdminNegativeTestJSON-405949889-project Jul 16 21:08:35.017561 ncdv-master-1 devstack@n-api.service[138680]: method: Jul 16 21:08:35.017561 ncdv-master-1 devstack@n-api.service[138680]: exception: Policy doesn't allow os_compute_api:os-hypervisors:list to be performed. Jul 16 21:08:35.017561 ncdv-master-1 devstack@n-api.service[138680]: action_args: {} Jul 16 21:08:35.017561 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:08:35.019881 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:08:35.019881 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:08:35.019881 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:08:35.019881 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:08:35.019881 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:08:35.019881 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Jul 16 21:08:35.019881 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Jul 16 21:08:35.019881 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:08:35.019881 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:08:35.019881 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/hypervisors.py", line 253, in index Jul 16 21:08:35.019881 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return self._index(req) Jul 16 21:08:35.019881 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/hypervisors.py", line 257, in _index Jul 16 21:08:35.019881 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi context.can(hv_policies.BASE_POLICY_NAME % 'list', target={}) Jul 16 21:08:35.019881 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/context.py", line 240, in can Jul 16 21:08:35.019881 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return policy.authorize(self, action, target) Jul 16 21:08:35.019881 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 199, in authorize Jul 16 21:08:35.019881 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi LOG.debug('Policy check for %(action)s failed with credentials ' Jul 16 21:08:35.019881 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 16 21:08:35.019881 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi self.force_reraise() Jul 16 21:08:35.021076 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 16 21:08:35.021076 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise self.value Jul 16 21:08:35.021076 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 186, in authorize Jul 16 21:08:35.021076 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi result = _ENFORCER.authorize(action, target, context, Jul 16 21:08:35.021076 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1184, in authorize Jul 16 21:08:35.021076 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return self.enforce( Jul 16 21:08:35.021076 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1088, in enforce Jul 16 21:08:35.021076 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc(*args, **kwargs) Jul 16 21:08:35.021076 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.PolicyNotAuthorized: Policy doesn't allow os_compute_api:os-hypervisors:list to be performed. Jul 16 21:08:35.021076 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:08:35.021076 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:08:35.021076 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:08:35.021076 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:08:35.021076 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:08:35.021076 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:08:35.021076 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 354, in __exit__ Jul 16 21:08:35.021076 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(webob.exc.HTTPForbidden( Jul 16 21:08:35.021076 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Policy doesn't allow os_compute_api:os-hypervisors:list to be performed. Jul 16 21:08:35.021076 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:08:35.022435 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-4230aca5-4729-4c4d-b98a-1ca92221a6d7 tempest-HypervisorAdminNegativeTestJSON-405949889 tempest-HypervisorAdminNegativeTestJSON-405949889-project] Returning 403 to user: Policy doesn't allow os_compute_api:os-hypervisors:list to be performed. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:08:35.022435 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-4230aca5-4729-4c4d-b98a-1ca92221a6d7 tempest-HypervisorAdminNegativeTestJSON-405949889 tempest-HypervisorAdminNegativeTestJSON-405949889-project] 10.222.0.16 "GET /compute/v2.1/os-hypervisors" status: 403 len: 115 microversion: 2.1 time: 0.034197 Jul 16 21:08:35.026311 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 64/121] 10.222.0.16 () {58 vars in 1145 bytes} [Sat Jul 16 21:08:34 2022] GET /compute/v2.1/os-hypervisors => generated 115 bytes in 44 msecs (HTTP/1.1 403) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:08:35.031106 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-0e45ea99-5ad9-4a0d-b749-4c554e3a2912 tempest-HypervisorAdminNegativeTestJSON-405949889 tempest-HypervisorAdminNegativeTestJSON-405949889-project] Calling method '.version_select at 0x7fdd9dbfcee0>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:08:35.032133 ncdv-master-1 devstack@n-api.service[138681]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-hypervisors:statistics": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:08:35.032133 ncdv-master-1 devstack@n-api.service[138681]: warnings.warn(msg) Jul 16 21:08:35.033940 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-0e45ea99-5ad9-4a0d-b749-4c554e3a2912 tempest-HypervisorAdminNegativeTestJSON-405949889 tempest-HypervisorAdminNegativeTestJSON-405949889-project] Policy check for os_compute_api:os-hypervisors:statistics failed with credentials {'is_admin': False, 'user_id': '3f63a1fb362b462fa8f36f5a53cb6cf1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55dc74e9c8884796818fb12bd31f449a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:08:35.036101 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-0e45ea99-5ad9-4a0d-b749-4c554e3a2912 tempest-HypervisorAdminNegativeTestJSON-405949889 tempest-HypervisorAdminNegativeTestJSON-405949889-project] Request method failure captured: Jul 16 21:08:35.036101 ncdv-master-1 devstack@n-api.service[138681]: request: GET /compute/v2.1/os-hypervisors/statistics HTTP/1.1 Jul 16 21:08:35.036101 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:08:35.036101 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:08:35.036101 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:08:35.036101 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 21:08:35.036101 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:08:35.036101 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:08:35.036101 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:08:35.036101 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:08:35.036101 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0yjM6liv1yp4mDCVvKiko1fgf31eifSlomvSV2MXttjD4_OpO5ge4d2xQumH0tIylebwVbwXYOZJuv5e3a4Un9EvjwJbpZLZEnEvJm2m_TI54EydFD_mwjadcMbsubo2MADqUVLILyWEJ19voqly0yyvKuOp9kP1vn0swY38nYHTwSo Jul 16 21:08:35.036101 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:08:35.036101 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:08:35.036101 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:08:35.036101 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:08:35.036101 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:08:35.036101 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:08:35.036101 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: 55dc74e9c8884796818fb12bd31f449a Jul 16 21:08:35.036101 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-HypervisorAdminNegativeTestJSON-405949889 Jul 16 21:08:35.036101 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:08:35.036101 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:08:35.037274 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/55dc74e9c8884796818fb12bd31f449a"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/55dc74e9c8884796818fb12bd31f449a"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_55dc74e9c8884796818fb12bd31f449a"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/55dc74e9c8884796818fb12bd31f449a"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:08:35.037274 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-HypervisorAdminNegativeTestJSON-405949889 Jul 16 21:08:35.037274 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: 55dc74e9c8884796818fb12bd31f449a Jul 16 21:08:35.037274 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-HypervisorAdminNegativeTestJSON-405949889 Jul 16 21:08:35.037274 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-HypervisorAdminNegativeTestJSON-405949889-project Jul 16 21:08:35.037274 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:08:35.037274 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:08:35.037274 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: 3f63a1fb362b462fa8f36f5a53cb6cf1 Jul 16 21:08:35.037274 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-HypervisorAdminNegativeTestJSON-405949889-project Jul 16 21:08:35.037274 ncdv-master-1 devstack@n-api.service[138681]: method: Jul 16 21:08:35.037274 ncdv-master-1 devstack@n-api.service[138681]: exception: Policy doesn't allow os_compute_api:os-hypervisors:statistics to be performed. Jul 16 21:08:35.037274 ncdv-master-1 devstack@n-api.service[138681]: action_args: {} Jul 16 21:08:35.037274 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:08:35.037967 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:08:35.037967 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:08:35.037967 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:08:35.037967 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:08:35.037967 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:08:35.037967 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Jul 16 21:08:35.037967 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Jul 16 21:08:35.037967 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:08:35.037967 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:08:35.037967 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/hypervisors.py", line 521, in statistics Jul 16 21:08:35.037967 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi context.can(hv_policies.BASE_POLICY_NAME % 'statistics', target={}) Jul 16 21:08:35.037967 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/context.py", line 240, in can Jul 16 21:08:35.037967 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return policy.authorize(self, action, target) Jul 16 21:08:35.037967 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 199, in authorize Jul 16 21:08:35.037967 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi LOG.debug('Policy check for %(action)s failed with credentials ' Jul 16 21:08:35.037967 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 16 21:08:35.037967 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi self.force_reraise() Jul 16 21:08:35.037967 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 16 21:08:35.037967 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise self.value Jul 16 21:08:35.038915 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 186, in authorize Jul 16 21:08:35.038915 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi result = _ENFORCER.authorize(action, target, context, Jul 16 21:08:35.038915 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1184, in authorize Jul 16 21:08:35.038915 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return self.enforce( Jul 16 21:08:35.038915 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1088, in enforce Jul 16 21:08:35.038915 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc(*args, **kwargs) Jul 16 21:08:35.038915 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.PolicyNotAuthorized: Policy doesn't allow os_compute_api:os-hypervisors:statistics to be performed. Jul 16 21:08:35.038915 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:08:35.038915 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:08:35.038915 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:08:35.038915 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:08:35.038915 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:08:35.038915 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:08:35.038915 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 354, in __exit__ Jul 16 21:08:35.038915 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(webob.exc.HTTPForbidden( Jul 16 21:08:35.038915 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Policy doesn't allow os_compute_api:os-hypervisors:statistics to be performed. Jul 16 21:08:35.038915 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:08:35.041320 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-0e45ea99-5ad9-4a0d-b749-4c554e3a2912 tempest-HypervisorAdminNegativeTestJSON-405949889 tempest-HypervisorAdminNegativeTestJSON-405949889-project] Returning 403 to user: Policy doesn't allow os_compute_api:os-hypervisors:statistics to be performed. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:08:35.041320 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-0e45ea99-5ad9-4a0d-b749-4c554e3a2912 tempest-HypervisorAdminNegativeTestJSON-405949889 tempest-HypervisorAdminNegativeTestJSON-405949889-project] 10.222.0.16 "GET /compute/v2.1/os-hypervisors/statistics" status: 403 len: 121 microversion: 2.1 time: 0.013284 Jul 16 21:08:35.043083 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 58/122] 10.222.0.16 () {58 vars in 1178 bytes} [Sat Jul 16 21:08:35 2022] GET /compute/v2.1/os-hypervisors/statistics => generated 121 bytes in 17 msecs (HTTP/1.1 403) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:08:35.574638 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-c288522b-7e45-41f3-811c-890071f2ffd6 tempest-HypervisorAdminNegativeTestJSON-1085479963 tempest-HypervisorAdminNegativeTestJSON-1085479963-project-admin] Calling method '.version_select at 0x7fdd9dd3caf0>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:08:35.577370 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c288522b-7e45-41f3-811c-890071f2ffd6 tempest-HypervisorAdminNegativeTestJSON-1085479963 tempest-HypervisorAdminNegativeTestJSON-1085479963-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:08:35.578127 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c288522b-7e45-41f3-811c-890071f2ffd6 tempest-HypervisorAdminNegativeTestJSON-1085479963 tempest-HypervisorAdminNegativeTestJSON-1085479963-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:08:35.609408 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c288522b-7e45-41f3-811c-890071f2ffd6 tempest-HypervisorAdminNegativeTestJSON-1085479963 tempest-HypervisorAdminNegativeTestJSON-1085479963-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:08:35.609408 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c288522b-7e45-41f3-811c-890071f2ffd6 tempest-HypervisorAdminNegativeTestJSON-1085479963 tempest-HypervisorAdminNegativeTestJSON-1085479963-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:08:35.644681 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c288522b-7e45-41f3-811c-890071f2ffd6 tempest-HypervisorAdminNegativeTestJSON-1085479963 tempest-HypervisorAdminNegativeTestJSON-1085479963-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:08:35.646171 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c288522b-7e45-41f3-811c-890071f2ffd6 tempest-HypervisorAdminNegativeTestJSON-1085479963 tempest-HypervisorAdminNegativeTestJSON-1085479963-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:08:35.669147 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-c288522b-7e45-41f3-811c-890071f2ffd6 tempest-HypervisorAdminNegativeTestJSON-1085479963 tempest-HypervisorAdminNegativeTestJSON-1085479963-project-admin] 10.222.0.16 "GET /compute/v2.1/os-hypervisors" status: 200 len: 191 microversion: 2.1 time: 0.615427 Jul 16 21:08:35.669801 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 65/123] 10.222.0.16 () {58 vars in 1145 bytes} [Sat Jul 16 21:08:35 2022] GET /compute/v2.1/os-hypervisors => generated 191 bytes in 619 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:08:35.681739 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-403479aa-5fac-4e97-b00c-f8f1729d65ac tempest-HypervisorAdminNegativeTestJSON-405949889 tempest-HypervisorAdminNegativeTestJSON-405949889-project] Calling method '.version_select at 0x7fdd9db85dc0>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:08:35.683392 ncdv-master-1 devstack@n-api.service[138681]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-hypervisors:uptime": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:08:35.683392 ncdv-master-1 devstack@n-api.service[138681]: warnings.warn(msg) Jul 16 21:08:35.685679 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-403479aa-5fac-4e97-b00c-f8f1729d65ac tempest-HypervisorAdminNegativeTestJSON-405949889 tempest-HypervisorAdminNegativeTestJSON-405949889-project] Policy check for os_compute_api:os-hypervisors:uptime failed with credentials {'is_admin': False, 'user_id': '3f63a1fb362b462fa8f36f5a53cb6cf1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55dc74e9c8884796818fb12bd31f449a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:08:35.687663 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-403479aa-5fac-4e97-b00c-f8f1729d65ac tempest-HypervisorAdminNegativeTestJSON-405949889 tempest-HypervisorAdminNegativeTestJSON-405949889-project] Request method failure captured: Jul 16 21:08:35.687663 ncdv-master-1 devstack@n-api.service[138681]: request: GET /compute/v2.1/os-hypervisors/1/uptime HTTP/1.1 Jul 16 21:08:35.687663 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:08:35.687663 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:08:35.687663 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:08:35.687663 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 21:08:35.687663 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:08:35.687663 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:08:35.687663 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:08:35.687663 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:08:35.687663 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0yjM6liv1yp4mDCVvKiko1fgf31eifSlomvSV2MXttjD4_OpO5ge4d2xQumH0tIylebwVbwXYOZJuv5e3a4Un9EvjwJbpZLZEnEvJm2m_TI54EydFD_mwjadcMbsubo2MADqUVLILyWEJ19voqly0yyvKuOp9kP1vn0swY38nYHTwSo Jul 16 21:08:35.687663 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:08:35.687663 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:08:35.687663 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:08:35.687663 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:08:35.687663 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:08:35.687663 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:08:35.687663 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: 55dc74e9c8884796818fb12bd31f449a Jul 16 21:08:35.687663 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-HypervisorAdminNegativeTestJSON-405949889 Jul 16 21:08:35.687663 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:08:35.687663 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:08:35.688997 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/55dc74e9c8884796818fb12bd31f449a"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/55dc74e9c8884796818fb12bd31f449a"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_55dc74e9c8884796818fb12bd31f449a"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/55dc74e9c8884796818fb12bd31f449a"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:08:35.688997 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-HypervisorAdminNegativeTestJSON-405949889 Jul 16 21:08:35.688997 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: 55dc74e9c8884796818fb12bd31f449a Jul 16 21:08:35.688997 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-HypervisorAdminNegativeTestJSON-405949889 Jul 16 21:08:35.688997 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-HypervisorAdminNegativeTestJSON-405949889-project Jul 16 21:08:35.688997 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:08:35.688997 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:08:35.688997 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: 3f63a1fb362b462fa8f36f5a53cb6cf1 Jul 16 21:08:35.688997 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-HypervisorAdminNegativeTestJSON-405949889-project Jul 16 21:08:35.688997 ncdv-master-1 devstack@n-api.service[138681]: method: Jul 16 21:08:35.688997 ncdv-master-1 devstack@n-api.service[138681]: exception: Policy doesn't allow os_compute_api:os-hypervisors:uptime to be performed. Jul 16 21:08:35.688997 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': '1'} Jul 16 21:08:35.688997 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:08:35.690082 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:08:35.690082 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:08:35.690082 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:08:35.690082 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:08:35.690082 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:08:35.690082 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Jul 16 21:08:35.690082 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Jul 16 21:08:35.690082 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:08:35.690082 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:08:35.690082 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/hypervisors.py", line 387, in uptime Jul 16 21:08:35.690082 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi context.can(hv_policies.BASE_POLICY_NAME % 'uptime', target={}) Jul 16 21:08:35.690082 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/context.py", line 240, in can Jul 16 21:08:35.690082 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return policy.authorize(self, action, target) Jul 16 21:08:35.690082 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 199, in authorize Jul 16 21:08:35.690082 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi LOG.debug('Policy check for %(action)s failed with credentials ' Jul 16 21:08:35.690082 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 16 21:08:35.690082 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi self.force_reraise() Jul 16 21:08:35.690082 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 16 21:08:35.690082 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise self.value Jul 16 21:08:35.691327 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 186, in authorize Jul 16 21:08:35.691327 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi result = _ENFORCER.authorize(action, target, context, Jul 16 21:08:35.691327 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1184, in authorize Jul 16 21:08:35.691327 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return self.enforce( Jul 16 21:08:35.691327 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1088, in enforce Jul 16 21:08:35.691327 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc(*args, **kwargs) Jul 16 21:08:35.691327 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.PolicyNotAuthorized: Policy doesn't allow os_compute_api:os-hypervisors:uptime to be performed. Jul 16 21:08:35.691327 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:08:35.691327 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:08:35.691327 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:08:35.691327 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:08:35.691327 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:08:35.691327 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:08:35.691327 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 354, in __exit__ Jul 16 21:08:35.691327 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(webob.exc.HTTPForbidden( Jul 16 21:08:35.691327 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Policy doesn't allow os_compute_api:os-hypervisors:uptime to be performed. Jul 16 21:08:35.691327 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:08:35.693321 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-403479aa-5fac-4e97-b00c-f8f1729d65ac tempest-HypervisorAdminNegativeTestJSON-405949889 tempest-HypervisorAdminNegativeTestJSON-405949889-project] Returning 403 to user: Policy doesn't allow os_compute_api:os-hypervisors:uptime to be performed. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:08:35.694319 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-403479aa-5fac-4e97-b00c-f8f1729d65ac tempest-HypervisorAdminNegativeTestJSON-405949889 tempest-HypervisorAdminNegativeTestJSON-405949889-project] 10.222.0.16 "GET /compute/v2.1/os-hypervisors/1/uptime" status: 403 len: 117 microversion: 2.1 time: 0.017515 Jul 16 21:08:35.699913 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 59/124] 10.222.0.16 () {58 vars in 1172 bytes} [Sat Jul 16 21:08:35 2022] GET /compute/v2.1/os-hypervisors/1/uptime => generated 117 bytes in 24 msecs (HTTP/1.1 403) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:08:35.713180 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-a99aade1-5bbd-48ee-b3c2-b4d3e626901c tempest-HypervisorAdminNegativeTestJSON-1085479963 tempest-HypervisorAdminNegativeTestJSON-1085479963-project-admin] Calling method '.version_select at 0x7fdd9de5edc0>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:08:35.713180 ncdv-master-1 devstack@n-api.service[138680]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-hypervisors:uptime": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:08:35.713180 ncdv-master-1 devstack@n-api.service[138680]: warnings.warn(msg) Jul 16 21:08:35.714106 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-a99aade1-5bbd-48ee-b3c2-b4d3e626901c tempest-HypervisorAdminNegativeTestJSON-1085479963 tempest-HypervisorAdminNegativeTestJSON-1085479963-project-admin] HTTP exception thrown: Hypervisor with ID 'ad239a9b-b564-4590-8323-2fa8e911a86a' could not be found. Jul 16 21:08:35.730170 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-a99aade1-5bbd-48ee-b3c2-b4d3e626901c tempest-HypervisorAdminNegativeTestJSON-1085479963 tempest-HypervisorAdminNegativeTestJSON-1085479963-project-admin] Request method failure captured: Jul 16 21:08:35.730170 ncdv-master-1 devstack@n-api.service[138680]: request: GET /compute/v2.1/os-hypervisors/ad239a9b-b564-4590-8323-2fa8e911a86a/uptime HTTP/1.1 Jul 16 21:08:35.730170 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:08:35.730170 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:08:35.730170 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:08:35.730170 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 0 Jul 16 21:08:35.730170 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:08:35.730170 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:08:35.730170 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:08:35.730170 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:08:35.730170 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0yjSU6JosviZCoKXTDugqVSYZ3qqOxyZ1jU-6pszawZnN78Kq3kilGVYs8zwdMnvfVqmrZGE5WctmpUHTAIrrsPU93UcfQyhoxhT5odmt6wmH55F8grh7F_SSEjh8AWgYeH-FID4rTYThdiS2W4B83yUC-Iq1-HrUpyppNkskPLOtyg Jul 16 21:08:35.730170 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:08:35.730170 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:08:35.730170 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:08:35.730170 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:08:35.730170 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:08:35.730170 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:08:35.730170 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: 531521baa8f1419398a42b7e7cf47b33 Jul 16 21:08:35.730170 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-HypervisorAdminNegativeTestJSON-1085479963 Jul 16 21:08:35.730170 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,admin,reader Jul 16 21:08:35.730170 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,admin,reader Jul 16 21:08:35.732047 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/531521baa8f1419398a42b7e7cf47b33"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/531521baa8f1419398a42b7e7cf47b33"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_531521baa8f1419398a42b7e7cf47b33"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/531521baa8f1419398a42b7e7cf47b33"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:08:35.732047 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-HypervisorAdminNegativeTestJSON-1085479963 Jul 16 21:08:35.732047 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: 531521baa8f1419398a42b7e7cf47b33 Jul 16 21:08:35.732047 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-HypervisorAdminNegativeTestJSON-1085479963 Jul 16 21:08:35.732047 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-HypervisorAdminNegativeTestJSON-1085479963-project-admin Jul 16 21:08:35.732047 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:08:35.732047 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:08:35.732047 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: 10ccde987997492c9dc816f3edb9d6c1 Jul 16 21:08:35.732047 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-HypervisorAdminNegativeTestJSON-1085479963-project-admin Jul 16 21:08:35.732047 ncdv-master-1 devstack@n-api.service[138680]: method: Jul 16 21:08:35.732047 ncdv-master-1 devstack@n-api.service[138680]: exception: Hypervisor with ID 'ad239a9b-b564-4590-8323-2fa8e911a86a' could not be found. Jul 16 21:08:35.732047 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': 'ad239a9b-b564-4590-8323-2fa8e911a86a'} Jul 16 21:08:35.733142 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:08:35.733142 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:08:35.733142 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/strutils.py", line 501, in validate_integer Jul 16 21:08:35.733142 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi value = int(str(value)) Jul 16 21:08:35.733142 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi ValueError: invalid literal for int() with base 10: 'ad239a9b-b564-4590-8323-2fa8e911a86a' Jul 16 21:08:35.733142 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:08:35.733142 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:08:35.733142 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:08:35.733142 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:08:35.733142 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/utils.py", line 614, in validate_integer Jul 16 21:08:35.733142 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return strutils.validate_integer(value, name, min_value, max_value) Jul 16 21:08:35.733142 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/strutils.py", line 505, in validate_integer Jul 16 21:08:35.733142 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise ValueError(msg) Jul 16 21:08:35.733142 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi ValueError: id must be an integer Jul 16 21:08:35.733142 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:08:35.733142 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:08:35.733142 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:08:35.733142 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:08:35.733142 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/hypervisors.py", line 313, in _validate_id Jul 16 21:08:35.733142 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi utils.validate_integer(hypervisor_id, 'id') Jul 16 21:08:35.733142 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/utils.py", line 616, in validate_integer Jul 16 21:08:35.733142 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.InvalidInput(reason=str(e)) Jul 16 21:08:35.734741 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: id must be an integer Jul 16 21:08:35.734741 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:08:35.734741 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:08:35.734741 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:08:35.734741 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:08:35.734741 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:08:35.734741 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:08:35.734741 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:08:35.734741 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:08:35.734741 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Jul 16 21:08:35.734741 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Jul 16 21:08:35.734741 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:08:35.734741 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:08:35.734741 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/hypervisors.py", line 389, in uptime Jul 16 21:08:35.734741 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi self._validate_id(req, id) Jul 16 21:08:35.734741 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/hypervisors.py", line 317, in _validate_id Jul 16 21:08:35.734741 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=msg) Jul 16 21:08:35.734741 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Hypervisor with ID 'ad239a9b-b564-4590-8323-2fa8e911a86a' could not be found. Jul 16 21:08:35.734741 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:08:35.734741 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:08:35.734741 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:08:35.736075 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:08:35.736075 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:08:35.736075 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:08:35.736075 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:08:35.736075 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:08:35.736075 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Hypervisor with ID 'ad239a9b-b564-4590-8323-2fa8e911a86a' could not be found. Jul 16 21:08:35.736075 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:08:35.737573 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-a99aade1-5bbd-48ee-b3c2-b4d3e626901c tempest-HypervisorAdminNegativeTestJSON-1085479963 tempest-HypervisorAdminNegativeTestJSON-1085479963-project-admin] Returning 404 to user: Hypervisor with ID 'ad239a9b-b564-4590-8323-2fa8e911a86a' could not be found. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:08:35.738610 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-a99aade1-5bbd-48ee-b3c2-b4d3e626901c tempest-HypervisorAdminNegativeTestJSON-1085479963 tempest-HypervisorAdminNegativeTestJSON-1085479963-project-admin] 10.222.0.16 "GET /compute/v2.1/os-hypervisors/ad239a9b-b564-4590-8323-2fa8e911a86a/uptime" status: 404 len: 123 microversion: 2.1 time: 0.037423 Jul 16 21:08:35.740775 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 66/125] 10.222.0.16 () {58 vars in 1277 bytes} [Sat Jul 16 21:08:35 2022] GET /compute/v2.1/os-hypervisors/ad239a9b-b564-4590-8323-2fa8e911a86a/uptime => generated 123 bytes in 40 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:08:35.751416 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-52f8d1d2-53a6-473c-8b42-e668a584ccab tempest-HypervisorAdminNegativeTestJSON-1085479963 tempest-HypervisorAdminNegativeTestJSON-1085479963-project-admin] Calling method '.version_select at 0x7fdd9dd6e820>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:08:35.752838 ncdv-master-1 devstack@n-api.service[138681]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-hypervisors:list": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:08:35.752838 ncdv-master-1 devstack@n-api.service[138681]: warnings.warn(msg) Jul 16 21:08:35.754170 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-52f8d1d2-53a6-473c-8b42-e668a584ccab tempest-HypervisorAdminNegativeTestJSON-1085479963 tempest-HypervisorAdminNegativeTestJSON-1085479963-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:08:35.754629 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-52f8d1d2-53a6-473c-8b42-e668a584ccab tempest-HypervisorAdminNegativeTestJSON-1085479963 tempest-HypervisorAdminNegativeTestJSON-1085479963-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:08:35.785573 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-52f8d1d2-53a6-473c-8b42-e668a584ccab tempest-HypervisorAdminNegativeTestJSON-1085479963 tempest-HypervisorAdminNegativeTestJSON-1085479963-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:08:35.786618 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-52f8d1d2-53a6-473c-8b42-e668a584ccab tempest-HypervisorAdminNegativeTestJSON-1085479963 tempest-HypervisorAdminNegativeTestJSON-1085479963-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:08:35.800401 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-52f8d1d2-53a6-473c-8b42-e668a584ccab tempest-HypervisorAdminNegativeTestJSON-1085479963 tempest-HypervisorAdminNegativeTestJSON-1085479963-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:08:35.801199 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-52f8d1d2-53a6-473c-8b42-e668a584ccab tempest-HypervisorAdminNegativeTestJSON-1085479963 tempest-HypervisorAdminNegativeTestJSON-1085479963-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:08:35.808090 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-52f8d1d2-53a6-473c-8b42-e668a584ccab tempest-HypervisorAdminNegativeTestJSON-1085479963 tempest-HypervisorAdminNegativeTestJSON-1085479963-project-admin] 10.222.0.16 "GET /compute/v2.1/os-hypervisors" status: 200 len: 191 microversion: 2.1 time: 0.062293 Jul 16 21:08:35.808854 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 60/126] 10.222.0.16 () {58 vars in 1145 bytes} [Sat Jul 16 21:08:35 2022] GET /compute/v2.1/os-hypervisors => generated 191 bytes in 64 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:08:35.822312 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-d49f3ecd-36b9-473c-be87-e7337252ec14 tempest-HypervisorAdminNegativeTestJSON-405949889 tempest-HypervisorAdminNegativeTestJSON-405949889-project] Calling method '.version_select at 0x7fdd9dd3f700>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:08:35.824955 ncdv-master-1 devstack@n-api.service[138680]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-hypervisors:show": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:08:35.824955 ncdv-master-1 devstack@n-api.service[138680]: warnings.warn(msg) Jul 16 21:08:35.826986 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-d49f3ecd-36b9-473c-be87-e7337252ec14 tempest-HypervisorAdminNegativeTestJSON-405949889 tempest-HypervisorAdminNegativeTestJSON-405949889-project] Policy check for os_compute_api:os-hypervisors:show failed with credentials {'is_admin': False, 'user_id': '3f63a1fb362b462fa8f36f5a53cb6cf1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55dc74e9c8884796818fb12bd31f449a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:08:35.829654 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-d49f3ecd-36b9-473c-be87-e7337252ec14 tempest-HypervisorAdminNegativeTestJSON-405949889 tempest-HypervisorAdminNegativeTestJSON-405949889-project] Request method failure captured: Jul 16 21:08:35.829654 ncdv-master-1 devstack@n-api.service[138680]: request: GET /compute/v2.1/os-hypervisors/1 HTTP/1.1 Jul 16 21:08:35.829654 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:08:35.829654 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:08:35.829654 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:08:35.829654 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 0 Jul 16 21:08:35.829654 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:08:35.829654 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:08:35.829654 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:08:35.829654 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:08:35.829654 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0yjM6liv1yp4mDCVvKiko1fgf31eifSlomvSV2MXttjD4_OpO5ge4d2xQumH0tIylebwVbwXYOZJuv5e3a4Un9EvjwJbpZLZEnEvJm2m_TI54EydFD_mwjadcMbsubo2MADqUVLILyWEJ19voqly0yyvKuOp9kP1vn0swY38nYHTwSo Jul 16 21:08:35.829654 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:08:35.829654 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:08:35.829654 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:08:35.829654 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:08:35.829654 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:08:35.829654 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:08:35.829654 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: 55dc74e9c8884796818fb12bd31f449a Jul 16 21:08:35.829654 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-HypervisorAdminNegativeTestJSON-405949889 Jul 16 21:08:35.829654 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:08:35.829654 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:08:35.834063 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/55dc74e9c8884796818fb12bd31f449a"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/55dc74e9c8884796818fb12bd31f449a"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_55dc74e9c8884796818fb12bd31f449a"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/55dc74e9c8884796818fb12bd31f449a"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:08:35.834063 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-HypervisorAdminNegativeTestJSON-405949889 Jul 16 21:08:35.834063 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: 55dc74e9c8884796818fb12bd31f449a Jul 16 21:08:35.834063 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-HypervisorAdminNegativeTestJSON-405949889 Jul 16 21:08:35.834063 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-HypervisorAdminNegativeTestJSON-405949889-project Jul 16 21:08:35.834063 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:08:35.834063 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:08:35.834063 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: 3f63a1fb362b462fa8f36f5a53cb6cf1 Jul 16 21:08:35.834063 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-HypervisorAdminNegativeTestJSON-405949889-project Jul 16 21:08:35.834063 ncdv-master-1 devstack@n-api.service[138680]: method: Jul 16 21:08:35.834063 ncdv-master-1 devstack@n-api.service[138680]: exception: Policy doesn't allow os_compute_api:os-hypervisors:show to be performed. Jul 16 21:08:35.834063 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': '1'} Jul 16 21:08:35.834063 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:08:35.834748 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:08:35.834748 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:08:35.834748 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:08:35.834748 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:08:35.834748 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:08:35.834748 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Jul 16 21:08:35.834748 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Jul 16 21:08:35.834748 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:08:35.834748 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:08:35.834748 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/hypervisors.py", line 337, in show Jul 16 21:08:35.834748 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return self._show(req, id) Jul 16 21:08:35.834748 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/hypervisors.py", line 341, in _show Jul 16 21:08:35.834748 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi context.can(hv_policies.BASE_POLICY_NAME % 'show', target={}) Jul 16 21:08:35.834748 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/context.py", line 240, in can Jul 16 21:08:35.834748 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return policy.authorize(self, action, target) Jul 16 21:08:35.834748 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 199, in authorize Jul 16 21:08:35.834748 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi LOG.debug('Policy check for %(action)s failed with credentials ' Jul 16 21:08:35.834748 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 16 21:08:35.834748 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi self.force_reraise() Jul 16 21:08:35.835724 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 16 21:08:35.835724 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise self.value Jul 16 21:08:35.835724 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 186, in authorize Jul 16 21:08:35.835724 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi result = _ENFORCER.authorize(action, target, context, Jul 16 21:08:35.835724 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1184, in authorize Jul 16 21:08:35.835724 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return self.enforce( Jul 16 21:08:35.835724 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1088, in enforce Jul 16 21:08:35.835724 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc(*args, **kwargs) Jul 16 21:08:35.835724 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.PolicyNotAuthorized: Policy doesn't allow os_compute_api:os-hypervisors:show to be performed. Jul 16 21:08:35.835724 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:08:35.835724 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:08:35.835724 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:08:35.835724 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:08:35.835724 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:08:35.835724 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:08:35.835724 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 354, in __exit__ Jul 16 21:08:35.835724 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(webob.exc.HTTPForbidden( Jul 16 21:08:35.835724 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Policy doesn't allow os_compute_api:os-hypervisors:show to be performed. Jul 16 21:08:35.835724 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:08:35.836675 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-d49f3ecd-36b9-473c-be87-e7337252ec14 tempest-HypervisorAdminNegativeTestJSON-405949889 tempest-HypervisorAdminNegativeTestJSON-405949889-project] Returning 403 to user: Policy doesn't allow os_compute_api:os-hypervisors:show to be performed. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:08:35.836675 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-d49f3ecd-36b9-473c-be87-e7337252ec14 tempest-HypervisorAdminNegativeTestJSON-405949889 tempest-HypervisorAdminNegativeTestJSON-405949889-project] 10.222.0.16 "GET /compute/v2.1/os-hypervisors/1" status: 403 len: 115 microversion: 2.1 time: 0.014968 Jul 16 21:08:35.836675 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 67/127] 10.222.0.16 () {58 vars in 1151 bytes} [Sat Jul 16 21:08:35 2022] GET /compute/v2.1/os-hypervisors/1 => generated 115 bytes in 17 msecs (HTTP/1.1 403) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:08:35.843236 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-cd07e9e5-c094-4446-b413-0bc64e10f5db tempest-HypervisorAdminNegativeTestJSON-1085479963 tempest-HypervisorAdminNegativeTestJSON-1085479963-project-admin] Calling method '.version_select at 0x7fdd9dbf14c0>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:08:35.844092 ncdv-master-1 devstack@n-api.service[138681]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-hypervisors:show": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:08:35.844092 ncdv-master-1 devstack@n-api.service[138681]: warnings.warn(msg) Jul 16 21:08:35.845124 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-cd07e9e5-c094-4446-b413-0bc64e10f5db tempest-HypervisorAdminNegativeTestJSON-1085479963 tempest-HypervisorAdminNegativeTestJSON-1085479963-project-admin] HTTP exception thrown: Hypervisor with ID 'e9e2b0a5-6fdb-46d9-a42a-628d9f689826' could not be found. Jul 16 21:08:35.847737 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-cd07e9e5-c094-4446-b413-0bc64e10f5db tempest-HypervisorAdminNegativeTestJSON-1085479963 tempest-HypervisorAdminNegativeTestJSON-1085479963-project-admin] Request method failure captured: Jul 16 21:08:35.847737 ncdv-master-1 devstack@n-api.service[138681]: request: GET /compute/v2.1/os-hypervisors/e9e2b0a5-6fdb-46d9-a42a-628d9f689826 HTTP/1.1 Jul 16 21:08:35.847737 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:08:35.847737 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:08:35.847737 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:08:35.847737 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 21:08:35.847737 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:08:35.847737 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:08:35.847737 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:08:35.847737 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:08:35.847737 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0yjSU6JosviZCoKXTDugqVSYZ3qqOxyZ1jU-6pszawZnN78Kq3kilGVYs8zwdMnvfVqmrZGE5WctmpUHTAIrrsPU93UcfQyhoxhT5odmt6wmH55F8grh7F_SSEjh8AWgYeH-FID4rTYThdiS2W4B83yUC-Iq1-HrUpyppNkskPLOtyg Jul 16 21:08:35.847737 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:08:35.847737 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:08:35.847737 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:08:35.847737 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:08:35.847737 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:08:35.847737 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:08:35.847737 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: 531521baa8f1419398a42b7e7cf47b33 Jul 16 21:08:35.847737 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-HypervisorAdminNegativeTestJSON-1085479963 Jul 16 21:08:35.847737 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,admin,reader Jul 16 21:08:35.847737 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,admin,reader Jul 16 21:08:35.849096 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/531521baa8f1419398a42b7e7cf47b33"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/531521baa8f1419398a42b7e7cf47b33"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_531521baa8f1419398a42b7e7cf47b33"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/531521baa8f1419398a42b7e7cf47b33"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:08:35.849096 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-HypervisorAdminNegativeTestJSON-1085479963 Jul 16 21:08:35.849096 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: 531521baa8f1419398a42b7e7cf47b33 Jul 16 21:08:35.849096 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-HypervisorAdminNegativeTestJSON-1085479963 Jul 16 21:08:35.849096 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-HypervisorAdminNegativeTestJSON-1085479963-project-admin Jul 16 21:08:35.849096 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:08:35.849096 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:08:35.849096 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: 10ccde987997492c9dc816f3edb9d6c1 Jul 16 21:08:35.849096 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-HypervisorAdminNegativeTestJSON-1085479963-project-admin Jul 16 21:08:35.849096 ncdv-master-1 devstack@n-api.service[138681]: method: Jul 16 21:08:35.849096 ncdv-master-1 devstack@n-api.service[138681]: exception: Hypervisor with ID 'e9e2b0a5-6fdb-46d9-a42a-628d9f689826' could not be found. Jul 16 21:08:35.849096 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': 'e9e2b0a5-6fdb-46d9-a42a-628d9f689826'} Jul 16 21:08:35.849792 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:08:35.849792 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:08:35.849792 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/strutils.py", line 501, in validate_integer Jul 16 21:08:35.849792 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi value = int(str(value)) Jul 16 21:08:35.849792 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi ValueError: invalid literal for int() with base 10: 'e9e2b0a5-6fdb-46d9-a42a-628d9f689826' Jul 16 21:08:35.849792 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:08:35.849792 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:08:35.849792 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:08:35.849792 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:08:35.849792 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/utils.py", line 614, in validate_integer Jul 16 21:08:35.849792 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return strutils.validate_integer(value, name, min_value, max_value) Jul 16 21:08:35.849792 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/strutils.py", line 505, in validate_integer Jul 16 21:08:35.849792 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise ValueError(msg) Jul 16 21:08:35.849792 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi ValueError: id must be an integer Jul 16 21:08:35.849792 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:08:35.849792 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:08:35.849792 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:08:35.849792 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:08:35.849792 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/hypervisors.py", line 313, in _validate_id Jul 16 21:08:35.849792 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi utils.validate_integer(hypervisor_id, 'id') Jul 16 21:08:35.849792 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/utils.py", line 616, in validate_integer Jul 16 21:08:35.849792 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.InvalidInput(reason=str(e)) Jul 16 21:08:35.850920 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: id must be an integer Jul 16 21:08:35.850920 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:08:35.850920 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:08:35.850920 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:08:35.850920 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:08:35.850920 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:08:35.850920 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:08:35.850920 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:08:35.850920 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:08:35.850920 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Jul 16 21:08:35.850920 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Jul 16 21:08:35.850920 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:08:35.850920 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:08:35.850920 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/hypervisors.py", line 337, in show Jul 16 21:08:35.850920 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return self._show(req, id) Jul 16 21:08:35.850920 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/hypervisors.py", line 343, in _show Jul 16 21:08:35.850920 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi self._validate_id(req, id) Jul 16 21:08:35.850920 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/hypervisors.py", line 317, in _validate_id Jul 16 21:08:35.850920 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=msg) Jul 16 21:08:35.850920 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Hypervisor with ID 'e9e2b0a5-6fdb-46d9-a42a-628d9f689826' could not be found. Jul 16 21:08:35.855808 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:08:35.855808 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:08:35.855808 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:08:35.855808 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:08:35.855808 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:08:35.855808 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:08:35.855808 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:08:35.855808 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:08:35.855808 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Hypervisor with ID 'e9e2b0a5-6fdb-46d9-a42a-628d9f689826' could not be found. Jul 16 21:08:35.855808 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:08:35.855808 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-cd07e9e5-c094-4446-b413-0bc64e10f5db tempest-HypervisorAdminNegativeTestJSON-1085479963 tempest-HypervisorAdminNegativeTestJSON-1085479963-project-admin] Returning 404 to user: Hypervisor with ID 'e9e2b0a5-6fdb-46d9-a42a-628d9f689826' could not be found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:08:35.855808 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-cd07e9e5-c094-4446-b413-0bc64e10f5db tempest-HypervisorAdminNegativeTestJSON-1085479963 tempest-HypervisorAdminNegativeTestJSON-1085479963-project-admin] 10.222.0.16 "GET /compute/v2.1/os-hypervisors/e9e2b0a5-6fdb-46d9-a42a-628d9f689826" status: 404 len: 123 microversion: 2.1 time: 0.010675 Jul 16 21:08:35.855808 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 61/128] 10.222.0.16 () {58 vars in 1256 bytes} [Sat Jul 16 21:08:35 2022] GET /compute/v2.1/os-hypervisors/e9e2b0a5-6fdb-46d9-a42a-628d9f689826 => generated 123 bytes in 16 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:08:36.013220 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-9b68b668-8514-4b32-bbf2-85debe5cdc3a tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:08:36.015845 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-9b68b668-8514-4b32-bbf2-85debe5cdc3a tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] [instance: 7895aae7-b50f-48cf-a3e1-5b33dc6e16a3] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:08:36.026624 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9b68b668-8514-4b32-bbf2-85debe5cdc3a tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:08:36.030680 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9b68b668-8514-4b32-bbf2-85debe5cdc3a tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:08:36.031611 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9b68b668-8514-4b32-bbf2-85debe5cdc3a tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:08:36.032079 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9b68b668-8514-4b32-bbf2-85debe5cdc3a tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:08:36.355891 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-9b68b668-8514-4b32-bbf2-85debe5cdc3a tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'db6c21ea09c54ea492d806f42d2f69a7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e7c343c974bd4ec0a6a765f818b895be', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:08:36.365415 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-9b68b668-8514-4b32-bbf2-85debe5cdc3a tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] 10.222.0.16 "GET /compute/v2.1/servers/7895aae7-b50f-48cf-a3e1-5b33dc6e16a3" status: 200 len: 1337 microversion: 2.1 time: 0.356710 Jul 16 21:08:36.378511 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 68/129] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:08:36 2022] GET /compute/v2.1/servers/7895aae7-b50f-48cf-a3e1-5b33dc6e16a3 => generated 1337 bytes in 368 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:08:37.383980 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-959682ec-beed-4906-a720-7c1d1766146b tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:08:37.384622 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-959682ec-beed-4906-a720-7c1d1766146b tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] [instance: 7895aae7-b50f-48cf-a3e1-5b33dc6e16a3] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:08:37.400432 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-959682ec-beed-4906-a720-7c1d1766146b tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:08:37.400991 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-959682ec-beed-4906-a720-7c1d1766146b tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:08:37.401690 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-959682ec-beed-4906-a720-7c1d1766146b tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:08:37.402135 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-959682ec-beed-4906-a720-7c1d1766146b tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:08:37.491481 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell0:SELECT=2 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:08:37.608964 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell0:SELECT=27,nova_cell0:UPDATE=7 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:08:37.807020 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-959682ec-beed-4906-a720-7c1d1766146b tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'db6c21ea09c54ea492d806f42d2f69a7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e7c343c974bd4ec0a6a765f818b895be', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:08:37.831674 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-959682ec-beed-4906-a720-7c1d1766146b tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] 10.222.0.16 "GET /compute/v2.1/servers/7895aae7-b50f-48cf-a3e1-5b33dc6e16a3" status: 200 len: 1337 microversion: 2.1 time: 0.451872 Jul 16 21:08:37.842280 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 62/130] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:08:37 2022] GET /compute/v2.1/servers/7895aae7-b50f-48cf-a3e1-5b33dc6e16a3 => generated 1337 bytes in 463 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:08:38.047266 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-a1a7c65b-b71b-4fee-a929-ebac16efc9a0 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-MigrationsAdminTest-server-918410943", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452"}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:08:38.145215 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.network.neutron [None req-a1a7c65b-b71b-4fee-a929-ebac16efc9a0 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] validate_networks() for None {{(pid=138680) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 16 21:08:38.598555 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-a1a7c65b-b71b-4fee-a929-ebac16efc9a0 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Flavor limits 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 16 21:08:38.598555 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-a1a7c65b-b71b-4fee-a929-ebac16efc9a0 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Image limits 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 16 21:08:38.598555 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-a1a7c65b-b71b-4fee-a929-ebac16efc9a0 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Flavor pref 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 16 21:08:38.598555 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-a1a7c65b-b71b-4fee-a929-ebac16efc9a0 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Image pref 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 16 21:08:38.600025 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-a1a7c65b-b71b-4fee-a929-ebac16efc9a0 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 16 21:08:38.619493 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a1a7c65b-b71b-4fee-a929-ebac16efc9a0 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:08:38.624309 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a1a7c65b-b71b-4fee-a929-ebac16efc9a0 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:08:38.624309 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a1a7c65b-b71b-4fee-a929-ebac16efc9a0 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:08:38.625487 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a1a7c65b-b71b-4fee-a929-ebac16efc9a0 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:08:38.693620 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-a1a7c65b-b71b-4fee-a929-ebac16efc9a0 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Getting quotas for project 55a19b7a4682457a9132f5fc80110712. Resources: {'ram', 'instances', 'cores'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:08:38.706510 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-a1a7c65b-b71b-4fee-a929-ebac16efc9a0 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Getting quotas for user a9b68dacc4b8463ba07fae6372131117 and project 55a19b7a4682457a9132f5fc80110712. Resources: {'ram', 'instances', 'cores'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:08:38.755062 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-a1a7c65b-b71b-4fee-a929-ebac16efc9a0 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Going to run 1 instances... {{(pid=138680) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 16 21:08:38.757542 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-a1a7c65b-b71b-4fee-a929-ebac16efc9a0 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: 1703f309-9e2e-4c5f-8588-eb4e43f21d91] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138680) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 16 21:08:38.849816 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-c3402971-b91e-45e0-939a-5c76aa6470ea tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:08:38.851370 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-c3402971-b91e-45e0-939a-5c76aa6470ea tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] [instance: 7895aae7-b50f-48cf-a3e1-5b33dc6e16a3] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:08:38.874036 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell0:SELECT=2 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:08:38.880600 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c3402971-b91e-45e0-939a-5c76aa6470ea tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:08:38.883596 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c3402971-b91e-45e0-939a-5c76aa6470ea tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:08:38.885681 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c3402971-b91e-45e0-939a-5c76aa6470ea tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:08:38.887446 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c3402971-b91e-45e0-939a-5c76aa6470ea tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:08:39.060642 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-a1a7c65b-b71b-4fee-a929-ebac16efc9a0 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 1.573818 Jul 16 21:08:39.065402 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 69/131] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 21:08:37 2022] POST /compute/v2.1/servers => generated 380 bytes in 1580 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 16 21:08:39.554156 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-c3402971-b91e-45e0-939a-5c76aa6470ea tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'db6c21ea09c54ea492d806f42d2f69a7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e7c343c974bd4ec0a6a765f818b895be', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:08:39.561227 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-8a684fb7-60cf-499c-bcc5-f209e0493679 req-e4c74c60-f4ef-4b8b-bceb-e9664efe03dd service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "7895aae7-b50f-48cf-a3e1-5b33dc6e16a3", "name": "network-vif-plugged", "status": "completed", "tag": "ce71fe98-d12f-40ec-b5fa-d43e42a5edeb"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:08:39.566723 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-c3402971-b91e-45e0-939a-5c76aa6470ea tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] 10.222.0.16 "GET /compute/v2.1/servers/7895aae7-b50f-48cf-a3e1-5b33dc6e16a3" status: 200 len: 1337 microversion: 2.1 time: 0.717570 Jul 16 21:08:39.567154 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 63/132] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:08:38 2022] GET /compute/v2.1/servers/7895aae7-b50f-48cf-a3e1-5b33dc6e16a3 => generated 1337 bytes in 719 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:08:39.589128 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-11fd5483-aac7-48d1-86f6-3f95f7442f80 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:08:39.589763 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-11fd5483-aac7-48d1-86f6-3f95f7442f80 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: 1703f309-9e2e-4c5f-8588-eb4e43f21d91] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:08:39.603376 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-8a684fb7-60cf-499c-bcc5-f209e0493679 req-e4c74c60-f4ef-4b8b-bceb-e9664efe03dd service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:08:39.604067 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-8a684fb7-60cf-499c-bcc5-f209e0493679 req-e4c74c60-f4ef-4b8b-bceb-e9664efe03dd service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:08:39.671450 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-8a684fb7-60cf-499c-bcc5-f209e0493679 req-e4c74c60-f4ef-4b8b-bceb-e9664efe03dd service nova] Creating event network-vif-plugged:ce71fe98-d12f-40ec-b5fa-d43e42a5edeb for instance 7895aae7-b50f-48cf-a3e1-5b33dc6e16a3 on nch1-master-1 Jul 16 21:08:39.671995 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-8a684fb7-60cf-499c-bcc5-f209e0493679 req-e4c74c60-f4ef-4b8b-bceb-e9664efe03dd service nova] Acquired lock "compute-rpcapi-router" {{(pid=138680) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:294}} Jul 16 21:08:39.673131 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-03568e28-9c53-4493-bdcf-564a2e3b2b3c None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:08:39.673576 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-03568e28-9c53-4493-bdcf-564a2e3b2b3c None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:08:39.696346 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-03568e28-9c53-4493-bdcf-564a2e3b2b3c None None] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:08:39.696346 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-03568e28-9c53-4493-bdcf-564a2e3b2b3c None None] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:08:39.723322 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.compute.rpcapi [None req-03568e28-9c53-4493-bdcf-564a2e3b2b3c None None] Automatically selected compute RPC version 6.0 from minimum service version 61 Jul 16 21:08:39.724241 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-03568e28-9c53-4493-bdcf-564a2e3b2b3c None None] Releasing lock "compute-rpcapi-router" {{(pid=138680) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:312}} Jul 16 21:08:39.770204 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-11fd5483-aac7-48d1-86f6-3f95f7442f80 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b68dacc4b8463ba07fae6372131117', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55a19b7a4682457a9132f5fc80110712', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:08:39.775621 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-03568e28-9c53-4493-bdcf-564a2e3b2b3c None None] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.708295 Jul 16 21:08:39.776577 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 70/133] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:08:39 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 710 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:08:39.803211 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-11fd5483-aac7-48d1-86f6-3f95f7442f80 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "GET /compute/v2.1/servers/1703f309-9e2e-4c5f-8588-eb4e43f21d91" status: 200 len: 1212 microversion: 2.1 time: 0.234719 Jul 16 21:08:39.817849 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 64/134] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:08:39 2022] GET /compute/v2.1/servers/1703f309-9e2e-4c5f-8588-eb4e43f21d91 => generated 1212 bytes in 243 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:08:40.382208 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-29d648bf-2b3b-4265-a50b-13b98599baa9 tempest-FlavorsV261TestJSON-508540900 tempest-FlavorsV261TestJSON-508540900-project-admin] Action: 'create', calling method: >, body: {"flavor": {"ram": 512, "vcpus": 1, "disk": 10, "name": "tempest-FlavorsV261TestJSON-flavor-1611329066", "id": 1741380053, "os-flavor-access:is_public": "True"}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:08:40.394110 ncdv-master-1 devstack@n-api.service[138680]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-flavor-manage:create": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:08:40.394110 ncdv-master-1 devstack@n-api.service[138680]: warnings.warn(msg) Jul 16 21:08:40.504020 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-29d648bf-2b3b-4265-a50b-13b98599baa9 tempest-FlavorsV261TestJSON-508540900 tempest-FlavorsV261TestJSON-508540900-project-admin] 10.222.0.16 "POST /compute/v2.1/flavors" status: 200 len: 468 microversion: 2.61 time: 0.726027 Jul 16 21:08:40.507577 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 71/135] 10.222.0.16 () {62 vars in 1187 bytes} [Sat Jul 16 21:08:39 2022] POST /compute/v2.1/flavors => generated 468 bytes in 730 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 16 21:08:40.591409 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-a7ccd1a2-eb31-43a4-aa7d-0d2f2696909b tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:08:40.592722 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-a7ccd1a2-eb31-43a4-aa7d-0d2f2696909b tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] [instance: 7895aae7-b50f-48cf-a3e1-5b33dc6e16a3] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:08:40.601498 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a7ccd1a2-eb31-43a4-aa7d-0d2f2696909b tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:08:40.602105 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a7ccd1a2-eb31-43a4-aa7d-0d2f2696909b tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:08:40.603104 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a7ccd1a2-eb31-43a4-aa7d-0d2f2696909b tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:08:40.603612 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a7ccd1a2-eb31-43a4-aa7d-0d2f2696909b tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:08:40.995284 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-a7ccd1a2-eb31-43a4-aa7d-0d2f2696909b tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'db6c21ea09c54ea492d806f42d2f69a7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e7c343c974bd4ec0a6a765f818b895be', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:08:41.009491 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-a7ccd1a2-eb31-43a4-aa7d-0d2f2696909b tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] 10.222.0.16 "GET /compute/v2.1/servers/7895aae7-b50f-48cf-a3e1-5b33dc6e16a3" status: 200 len: 1337 microversion: 2.1 time: 0.429112 Jul 16 21:08:41.012496 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 72/136] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:08:40 2022] GET /compute/v2.1/servers/7895aae7-b50f-48cf-a3e1-5b33dc6e16a3 => generated 1337 bytes in 436 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:08:41.024139 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-a13a9b89-cfee-415b-918e-197dca9f0e57 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:08:41.030554 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-a13a9b89-cfee-415b-918e-197dca9f0e57 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: 1703f309-9e2e-4c5f-8588-eb4e43f21d91] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:08:41.073949 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-d23d1221-8d84-4e0c-9b11-ec6beaa7de4c tempest-FlavorsV261TestJSON-573190856 tempest-FlavorsV261TestJSON-573190856-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:08:41.096155 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-d23d1221-8d84-4e0c-9b11-ec6beaa7de4c tempest-FlavorsV261TestJSON-573190856 tempest-FlavorsV261TestJSON-573190856-project] 10.222.0.16 "GET /compute/v2.1/flavors/1741380053" status: 200 len: 468 microversion: 2.61 time: 0.582094 Jul 16 21:08:41.103526 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 65/137] 10.222.0.16 () {60 vars in 1198 bytes} [Sat Jul 16 21:08:40 2022] GET /compute/v2.1/flavors/1741380053 => generated 468 bytes in 584 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 16 21:08:41.385637 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-67909b29-369e-45f8-8980-7f7cc0178e16 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:08:41.400130 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-67909b29-369e-45f8-8980-7f7cc0178e16 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] 10.222.0.16 "GET /compute/v2.1/flavors/452" status: 200 len: 370 microversion: 2.1 time: 0.301708 Jul 16 21:08:41.402349 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 66/138] 10.222.0.16 () {58 vars in 1136 bytes} [Sat Jul 16 21:08:41 2022] GET /compute/v2.1/flavors/452 => generated 370 bytes in 304 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:08:41.409051 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-e570aaca-8b8b-42a6-aadc-10150f5ccaa3 tempest-FlavorsV261TestJSON-508540900 tempest-FlavorsV261TestJSON-508540900-project-admin] Action: 'update', calling method: .version_select at 0x7fdd9ff29e50>, body: {"flavor": {"description": "new"}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:08:41.449689 ncdv-master-1 devstack@n-api.service[138681]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-flavor-manage:update": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:08:41.449689 ncdv-master-1 devstack@n-api.service[138681]: warnings.warn(msg) Jul 16 21:08:41.453423 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-a13a9b89-cfee-415b-918e-197dca9f0e57 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b68dacc4b8463ba07fae6372131117', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55a19b7a4682457a9132f5fc80110712', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:08:41.467764 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-a13a9b89-cfee-415b-918e-197dca9f0e57 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "GET /compute/v2.1/servers/1703f309-9e2e-4c5f-8588-eb4e43f21d91" status: 200 len: 1212 microversion: 2.1 time: 0.453868 Jul 16 21:08:41.468362 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 73/139] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:08:41 2022] GET /compute/v2.1/servers/1703f309-9e2e-4c5f-8588-eb4e43f21d91 => generated 1212 bytes in 455 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:08:41.592617 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-e570aaca-8b8b-42a6-aadc-10150f5ccaa3 tempest-FlavorsV261TestJSON-508540900 tempest-FlavorsV261TestJSON-508540900-project-admin] 10.222.0.16 "PUT /compute/v2.1/flavors/1741380053" status: 200 len: 469 microversion: 2.61 time: 0.188817 Jul 16 21:08:41.608664 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 67/140] 10.222.0.16 () {62 vars in 1218 bytes} [Sat Jul 16 21:08:41 2022] PUT /compute/v2.1/flavors/1741380053 => generated 469 bytes in 205 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 16 21:08:41.778139 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-29111460-b158-46c3-a901-13e5cad6056c tempest-ServersWithSpecificFlavorTestJSON-2008354030 tempest-ServersWithSpecificFlavorTestJSON-2008354030-project-admin] Action: 'create', calling method: >, body: {"flavor": {"ram": 1024, "vcpus": 1, "disk": 1, "name": "tempest-flavor_with_ephemeral_1-1450913785", "id": 2089714046, "OS-FLV-EXT-DATA:ephemeral": 1, "os-flavor-access:is_public": "True"}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:08:41.779747 ncdv-master-1 devstack@n-api.service[138680]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-flavor-manage:create": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:08:41.779747 ncdv-master-1 devstack@n-api.service[138680]: warnings.warn(msg) Jul 16 21:08:41.846703 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-9053e0cb-3ba0-451f-a717-7527b104c60a tempest-QuotasAdminTestJSON-972835255 tempest-QuotasAdminTestJSON-972835255-project-admin] Calling method '.version_select at 0x7fdd9dbb64c0>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:08:41.866378 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-29111460-b158-46c3-a901-13e5cad6056c tempest-ServersWithSpecificFlavorTestJSON-2008354030 tempest-ServersWithSpecificFlavorTestJSON-2008354030-project-admin] 10.222.0.16 "POST /compute/v2.1/flavors" status: 200 len: 425 microversion: 2.1 time: 0.396394 Jul 16 21:08:41.867745 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 74/141] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 21:08:41 2022] POST /compute/v2.1/flavors => generated 425 bytes in 398 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:08:41.876421 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-fb6a0a6d-e716-4f77-8a84-96d7bf50b9b1 tempest-FlavorsV261TestJSON-573190856 tempest-FlavorsV261TestJSON-573190856-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:08:41.914758 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-fb6a0a6d-e716-4f77-8a84-96d7bf50b9b1 tempest-FlavorsV261TestJSON-573190856 tempest-FlavorsV261TestJSON-573190856-project] 10.222.0.16 "GET /compute/v2.1/flavors/detail" status: 200 len: 6705 microversion: 2.61 time: 0.043885 Jul 16 21:08:41.915643 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 75/142] 10.222.0.16 () {60 vars in 1186 bytes} [Sat Jul 16 21:08:41 2022] GET /compute/v2.1/flavors/detail => generated 6705 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:08:41.921622 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-0dd0d664-2a2d-4803-b3cf-4f069f00a9f9 tempest-ServersWithSpecificFlavorTestJSON-2008354030 tempest-ServersWithSpecificFlavorTestJSON-2008354030-project-admin] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:08:41.931924 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-0dd0d664-2a2d-4803-b3cf-4f069f00a9f9 tempest-ServersWithSpecificFlavorTestJSON-2008354030 tempest-ServersWithSpecificFlavorTestJSON-2008354030-project-admin] 10.222.0.16 "GET /compute/v2.1/flavors/452/os-extra_specs" status: 200 len: 19 microversion: 2.1 time: 0.015050 Jul 16 21:08:41.933057 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 76/143] 10.222.0.16 () {58 vars in 1181 bytes} [Sat Jul 16 21:08:41 2022] GET /compute/v2.1/flavors/452/os-extra_specs => generated 19 bytes in 17 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Jul 16 21:08:41.938440 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-13d49bc9-4363-4e62-8dec-a104d2b48313 tempest-FlavorsV261TestJSON-573190856 tempest-FlavorsV261TestJSON-573190856-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:08:41.978267 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-13d49bc9-4363-4e62-8dec-a104d2b48313 tempest-FlavorsV261TestJSON-573190856 tempest-FlavorsV261TestJSON-573190856-project] 10.222.0.16 "GET /compute/v2.1/flavors" status: 200 len: 3422 microversion: 2.61 time: 0.043874 Jul 16 21:08:41.981832 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-9053e0cb-3ba0-451f-a717-7527b104c60a tempest-QuotasAdminTestJSON-972835255 tempest-QuotasAdminTestJSON-972835255-project-admin] 10.222.0.16 "GET /compute/v2.1/os-quota-sets/ab7c7acf71184ab983a12fa486de5850" status: 200 len: 371 microversion: 2.1 time: 0.369717 Jul 16 21:08:41.982066 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 77/144] 10.222.0.16 () {60 vars in 1165 bytes} [Sat Jul 16 21:08:41 2022] GET /compute/v2.1/flavors => generated 3422 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:08:41.982143 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 68/145] 10.222.0.16 () {58 vars in 1241 bytes} [Sat Jul 16 21:08:41 2022] GET /compute/v2.1/os-quota-sets/ab7c7acf71184ab983a12fa486de5850 => generated 371 bytes in 371 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:08:41.986011 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-0050872d-73ab-413a-b30c-6d6438064a4f tempest-ServersWithSpecificFlavorTestJSON-2008354030 tempest-ServersWithSpecificFlavorTestJSON-2008354030-project-admin] Action: 'create', calling method: >, body: {"flavor": {"ram": 1024, "vcpus": 1, "disk": 1, "name": "tempest-flavor_with_ephemeral_0-20379067", "id": 1856122662, "OS-FLV-EXT-DATA:ephemeral": 0, "os-flavor-access:is_public": "True"}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:08:41.993402 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-e8d9cc31-28a6-48a0-a38b-236a83ced857 tempest-QuotasAdminTestJSON-972835255 tempest-QuotasAdminTestJSON-972835255-project-admin] Action: 'update', calling method: .version_select at 0x7fdd9dbfc940>, body: {"quota_set": {"ram": "5120"}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:08:42.116019 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-0050872d-73ab-413a-b30c-6d6438064a4f tempest-ServersWithSpecificFlavorTestJSON-2008354030 tempest-ServersWithSpecificFlavorTestJSON-2008354030-project-admin] 10.222.0.16 "POST /compute/v2.1/flavors" status: 200 len: 423 microversion: 2.1 time: 0.134646 Jul 16 21:08:42.117475 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 78/146] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 21:08:41 2022] POST /compute/v2.1/flavors => generated 423 bytes in 137 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:08:42.124072 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-7b3fa290-e860-4274-b48c-625cd9814906 tempest-FlavorsV261TestJSON-508540900 tempest-FlavorsV261TestJSON-508540900-project-admin] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:08:42.125278 ncdv-master-1 devstack@n-api.service[138680]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-flavor-manage:delete": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:08:42.125278 ncdv-master-1 devstack@n-api.service[138680]: warnings.warn(msg) Jul 16 21:08:42.264070 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-7b3fa290-e860-4274-b48c-625cd9814906 tempest-FlavorsV261TestJSON-508540900 tempest-FlavorsV261TestJSON-508540900-project-admin] 10.222.0.16 "DELETE /compute/v2.1/flavors/1741380053" status: 202 len: 0 microversion: 2.61 time: 0.145213 Jul 16 21:08:42.264819 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 79/147] 10.222.0.16 () {60 vars in 1201 bytes} [Sat Jul 16 21:08:42 2022] DELETE /compute/v2.1/flavors/1741380053 => generated 0 bytes in 147 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Jul 16 21:08:42.270573 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-007e0512-943e-4f23-b08d-8042d77bbb99 tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:08:42.273235 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-007e0512-943e-4f23-b08d-8042d77bbb99 tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] [instance: 7895aae7-b50f-48cf-a3e1-5b33dc6e16a3] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:08:42.284605 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-007e0512-943e-4f23-b08d-8042d77bbb99 tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:08:42.286041 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-007e0512-943e-4f23-b08d-8042d77bbb99 tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:08:42.288405 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-007e0512-943e-4f23-b08d-8042d77bbb99 tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:08:42.290146 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-007e0512-943e-4f23-b08d-8042d77bbb99 tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:08:42.317753 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e8d9cc31-28a6-48a0-a38b-236a83ced857 tempest-QuotasAdminTestJSON-972835255 tempest-QuotasAdminTestJSON-972835255-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:08:42.319295 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e8d9cc31-28a6-48a0-a38b-236a83ced857 tempest-QuotasAdminTestJSON-972835255 tempest-QuotasAdminTestJSON-972835255-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:08:42.321673 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e8d9cc31-28a6-48a0-a38b-236a83ced857 tempest-QuotasAdminTestJSON-972835255 tempest-QuotasAdminTestJSON-972835255-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:08:42.322333 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e8d9cc31-28a6-48a0-a38b-236a83ced857 tempest-QuotasAdminTestJSON-972835255 tempest-QuotasAdminTestJSON-972835255-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:08:42.530243 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-007e0512-943e-4f23-b08d-8042d77bbb99 tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'db6c21ea09c54ea492d806f42d2f69a7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e7c343c974bd4ec0a6a765f818b895be', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:08:42.542203 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-007e0512-943e-4f23-b08d-8042d77bbb99 tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] 10.222.0.16 "GET /compute/v2.1/servers/7895aae7-b50f-48cf-a3e1-5b33dc6e16a3" status: 200 len: 1337 microversion: 2.1 time: 0.276141 Jul 16 21:08:42.549849 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-e8d9cc31-28a6-48a0-a38b-236a83ced857 tempest-QuotasAdminTestJSON-972835255 tempest-QuotasAdminTestJSON-972835255-project-admin] 10.222.0.16 "PUT /compute/v2.1/os-quota-sets/ab7c7acf71184ab983a12fa486de5850" status: 200 len: 328 microversion: 2.1 time: 0.560329 Jul 16 21:08:42.551844 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 69/148] 10.222.0.16 () {60 vars in 1261 bytes} [Sat Jul 16 21:08:41 2022] PUT /compute/v2.1/os-quota-sets/ab7c7acf71184ab983a12fa486de5850 => generated 328 bytes in 564 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:08:42.553722 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 80/149] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:08:42 2022] GET /compute/v2.1/servers/7895aae7-b50f-48cf-a3e1-5b33dc6e16a3 => generated 1337 bytes in 288 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:08:42.561204 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-af339ff6-d488-4ba9-bbea-8f9262eecc16 tempest-FlavorsV261TestJSON-508540900 tempest-FlavorsV261TestJSON-508540900-project-admin] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:08:42.565824 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-ff4e7d54-fd8b-4138-bf5c-52cb3dbaf904 tempest-ServersWithSpecificFlavorTestJSON-2008354030 tempest-ServersWithSpecificFlavorTestJSON-2008354030-project-admin] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:08:42.585262 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-ff4e7d54-fd8b-4138-bf5c-52cb3dbaf904 tempest-ServersWithSpecificFlavorTestJSON-2008354030 tempest-ServersWithSpecificFlavorTestJSON-2008354030-project-admin] 10.222.0.16 "GET /compute/v2.1/flavors/452/os-extra_specs" status: 200 len: 19 microversion: 2.1 time: 0.031984 Jul 16 21:08:42.593244 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 70/150] 10.222.0.16 () {58 vars in 1181 bytes} [Sat Jul 16 21:08:42 2022] GET /compute/v2.1/flavors/452/os-extra_specs => generated 19 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Jul 16 21:08:42.618933 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-e699733b-7932-4230-88fd-2b92a4e8b00c tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:08:42.620681 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-e699733b-7932-4230-88fd-2b92a4e8b00c tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: 1703f309-9e2e-4c5f-8588-eb4e43f21d91] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:08:42.635548 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-af339ff6-d488-4ba9-bbea-8f9262eecc16 tempest-FlavorsV261TestJSON-508540900 tempest-FlavorsV261TestJSON-508540900-project-admin] 10.222.0.16 "GET /compute/v2.1/flavors/detail" status: 200 len: 6699 microversion: 2.61 time: 0.080560 Jul 16 21:08:42.636175 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 81/151] 10.222.0.16 () {60 vars in 1186 bytes} [Sat Jul 16 21:08:42 2022] GET /compute/v2.1/flavors/detail => generated 6699 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:08:42.648570 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-6241f1f5-7b76-499b-bd55-7334ed2bef5d tempest-QuotasAdminTestJSON-972835255 tempest-QuotasAdminTestJSON-972835255-project-admin] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:08:42.673418 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e699733b-7932-4230-88fd-2b92a4e8b00c tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:08:42.675164 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e699733b-7932-4230-88fd-2b92a4e8b00c tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:08:42.678496 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e699733b-7932-4230-88fd-2b92a4e8b00c tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:08:42.679231 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e699733b-7932-4230-88fd-2b92a4e8b00c tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:08:42.794528 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-6241f1f5-7b76-499b-bd55-7334ed2bef5d tempest-QuotasAdminTestJSON-972835255 tempest-QuotasAdminTestJSON-972835255-project-admin] 10.222.0.16 "DELETE /compute/v2.1/os-quota-sets/ab7c7acf71184ab983a12fa486de5850" status: 202 len: 0 microversion: 2.1 time: 0.154386 Jul 16 21:08:42.796051 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 82/152] 10.222.0.16 () {58 vars in 1244 bytes} [Sat Jul 16 21:08:42 2022] DELETE /compute/v2.1/os-quota-sets/ab7c7acf71184ab983a12fa486de5850 => generated 0 bytes in 159 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 21:08:42.801590 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-e04228aa-b318-41db-b9d4-40cf8735d509 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Action: 'create', calling method: .version_select at 0x7fdd9dce48b0>, body: {"keypair": {"name": "tempest-keypair-461056992", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBFDsEuQzmWsS8AVGmabJgwO9e8ZzezU82qVZb70yLsg+18osVyb+BOUsm3BhuJNHe6g8UXPE2154ll+Xfs79mCBrZRzI9HhSL4z/ET4SUO46CyNOme5DlG9tA4f2CMvxxA=="}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:08:42.863892 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_api:SELECT=85,nova_api:UPDATE=1,nova_api:DELETE=8,nova_api:INSERT=6 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:08:42.893165 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-e04228aa-b318-41db-b9d4-40cf8735d509 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Getting quotas for project d0ee02e4c8b448f2b77798ae504129b9. Resources: {'key_pairs'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:08:42.896483 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-e699733b-7932-4230-88fd-2b92a4e8b00c tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b68dacc4b8463ba07fae6372131117', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55a19b7a4682457a9132f5fc80110712', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:08:42.905936 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-e04228aa-b318-41db-b9d4-40cf8735d509 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Getting quotas for user ef9cbce2b2e64665b7f50767342dcb46 and project d0ee02e4c8b448f2b77798ae504129b9. Resources: {'key_pairs'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:08:42.919737 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-e699733b-7932-4230-88fd-2b92a4e8b00c tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "GET /compute/v2.1/servers/1703f309-9e2e-4c5f-8588-eb4e43f21d91" status: 200 len: 1264 microversion: 2.1 time: 0.325207 Jul 16 21:08:42.920850 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 71/153] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:08:42 2022] GET /compute/v2.1/servers/1703f309-9e2e-4c5f-8588-eb4e43f21d91 => generated 1264 bytes in 327 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:08:42.927025 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-4104c342-17cb-4ced-9de1-37d4ade17c95 tempest-QuotasAdminTestJSON-972835255 tempest-QuotasAdminTestJSON-972835255-project-admin] Calling method '.version_select at 0x7fdd9dbb6310>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:08:43.345220 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-e04228aa-b318-41db-b9d4-40cf8735d509 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] 10.222.0.16 "POST /compute/v2.1/os-keypairs" status: 200 len: 385 microversion: 2.1 time: 0.547513 Jul 16 21:08:43.346990 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 83/154] 10.222.0.16 () {60 vars in 1158 bytes} [Sat Jul 16 21:08:42 2022] POST /compute/v2.1/os-keypairs => generated 385 bytes in 550 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:08:43.540706 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-4104c342-17cb-4ced-9de1-37d4ade17c95 tempest-QuotasAdminTestJSON-972835255 tempest-QuotasAdminTestJSON-972835255-project-admin] 10.222.0.16 "GET /compute/v2.1/os-quota-sets/ab7c7acf71184ab983a12fa486de5850" status: 200 len: 371 microversion: 2.1 time: 0.616648 Jul 16 21:08:43.542767 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 72/155] 10.222.0.16 () {58 vars in 1241 bytes} [Sat Jul 16 21:08:42 2022] GET /compute/v2.1/os-quota-sets/ab7c7acf71184ab983a12fa486de5850 => generated 371 bytes in 621 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:08:43.563759 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-809f56dd-d400-41b6-a023-e1b0dadf89fd tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:08:43.564642 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-809f56dd-d400-41b6-a023-e1b0dadf89fd tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] [instance: 7895aae7-b50f-48cf-a3e1-5b33dc6e16a3] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:08:43.587872 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-809f56dd-d400-41b6-a023-e1b0dadf89fd tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:08:43.588542 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-809f56dd-d400-41b6-a023-e1b0dadf89fd tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:08:43.589520 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-809f56dd-d400-41b6-a023-e1b0dadf89fd tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:08:43.591873 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-809f56dd-d400-41b6-a023-e1b0dadf89fd tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:08:43.938562 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-933c2a6a-2447-46bf-a189-9930cf82c3fb tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:08:43.940607 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-933c2a6a-2447-46bf-a189-9930cf82c3fb tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: 1703f309-9e2e-4c5f-8588-eb4e43f21d91] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:08:43.954267 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-933c2a6a-2447-46bf-a189-9930cf82c3fb tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:08:43.957245 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-933c2a6a-2447-46bf-a189-9930cf82c3fb tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:08:43.961857 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-933c2a6a-2447-46bf-a189-9930cf82c3fb tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:08:43.971480 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-933c2a6a-2447-46bf-a189-9930cf82c3fb tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.009s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:08:44.203508 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-933c2a6a-2447-46bf-a189-9930cf82c3fb tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b68dacc4b8463ba07fae6372131117', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55a19b7a4682457a9132f5fc80110712', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:08:44.221260 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-933c2a6a-2447-46bf-a189-9930cf82c3fb tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "GET /compute/v2.1/servers/1703f309-9e2e-4c5f-8588-eb4e43f21d91" status: 200 len: 1272 microversion: 2.1 time: 0.287081 Jul 16 21:08:44.222151 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 73/156] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:08:43 2022] GET /compute/v2.1/servers/1703f309-9e2e-4c5f-8588-eb4e43f21d91 => generated 1272 bytes in 289 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:08:44.226896 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-1382abc0-8750-447f-884b-c29eeb2e72ed tempest-QuotasAdminTestJSON-972835255 tempest-QuotasAdminTestJSON-972835255-project-admin] Calling method '.version_select at 0x7fdda03d6ee0>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:08:44.400758 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-809f56dd-d400-41b6-a023-e1b0dadf89fd tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'db6c21ea09c54ea492d806f42d2f69a7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e7c343c974bd4ec0a6a765f818b895be', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:08:44.418945 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-809f56dd-d400-41b6-a023-e1b0dadf89fd tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] 10.222.0.16 "GET /compute/v2.1/servers/7895aae7-b50f-48cf-a3e1-5b33dc6e16a3" status: 200 len: 1487 microversion: 2.1 time: 0.861218 Jul 16 21:08:44.427636 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 84/157] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:08:43 2022] GET /compute/v2.1/servers/7895aae7-b50f-48cf-a3e1-5b33dc6e16a3 => generated 1487 bytes in 870 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:08:44.447144 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-a412fa16-af76-457c-b0ff-bb70a643dda4 tempest-AggregatesAdminTestJSON-1464138310 tempest-AggregatesAdminTestJSON-1464138310-project-admin] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:08:44.447695 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-a412fa16-af76-457c-b0ff-bb70a643dda4 tempest-AggregatesAdminTestJSON-1464138310 tempest-AggregatesAdminTestJSON-1464138310-project-admin] [instance: 7895aae7-b50f-48cf-a3e1-5b33dc6e16a3] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:08:44.484777 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a412fa16-af76-457c-b0ff-bb70a643dda4 tempest-AggregatesAdminTestJSON-1464138310 tempest-AggregatesAdminTestJSON-1464138310-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:08:44.485249 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a412fa16-af76-457c-b0ff-bb70a643dda4 tempest-AggregatesAdminTestJSON-1464138310 tempest-AggregatesAdminTestJSON-1464138310-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:08:44.492468 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a412fa16-af76-457c-b0ff-bb70a643dda4 tempest-AggregatesAdminTestJSON-1464138310 tempest-AggregatesAdminTestJSON-1464138310-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:08:44.493145 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a412fa16-af76-457c-b0ff-bb70a643dda4 tempest-AggregatesAdminTestJSON-1464138310 tempest-AggregatesAdminTestJSON-1464138310-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:08:45.072861 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-1382abc0-8750-447f-884b-c29eeb2e72ed tempest-QuotasAdminTestJSON-972835255 tempest-QuotasAdminTestJSON-972835255-project-admin] 10.222.0.16 "GET /compute/v2.1/os-quota-sets/9dcf95f2466344d1a28724d545a8fed6/defaults" status: 200 len: 371 microversion: 2.1 time: 0.848598 Jul 16 21:08:45.080616 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 74/158] 10.222.0.16 () {58 vars in 1268 bytes} [Sat Jul 16 21:08:44 2022] GET /compute/v2.1/os-quota-sets/9dcf95f2466344d1a28724d545a8fed6/defaults => generated 371 bytes in 858 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:08:45.241214 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-1dc60577-0654-4245-a3d6-a725e5250352 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:08:45.241847 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-1dc60577-0654-4245-a3d6-a725e5250352 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: 1703f309-9e2e-4c5f-8588-eb4e43f21d91] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:08:45.257254 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1dc60577-0654-4245-a3d6-a725e5250352 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:08:45.257867 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1dc60577-0654-4245-a3d6-a725e5250352 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:08:45.258824 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1dc60577-0654-4245-a3d6-a725e5250352 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:08:45.259481 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1dc60577-0654-4245-a3d6-a725e5250352 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:08:45.471731 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-1dc60577-0654-4245-a3d6-a725e5250352 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b68dacc4b8463ba07fae6372131117', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55a19b7a4682457a9132f5fc80110712', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:08:45.486897 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-1dc60577-0654-4245-a3d6-a725e5250352 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "GET /compute/v2.1/servers/1703f309-9e2e-4c5f-8588-eb4e43f21d91" status: 200 len: 1270 microversion: 2.1 time: 0.250706 Jul 16 21:08:45.488261 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 75/159] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:08:45 2022] GET /compute/v2.1/servers/1703f309-9e2e-4c5f-8588-eb4e43f21d91 => generated 1270 bytes in 253 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:08:45.555404 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-05870d1f-7cb9-4c90-be7d-108ca56feba6 tempest-QuotasAdminTestJSON-972835255 tempest-QuotasAdminTestJSON-972835255-project-admin] Action: 'update', calling method: .version_select at 0x7fdd9daed700>, body: {"quota_set": {"ram": "5120"}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:08:45.766120 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_api:SELECT=84,nova_api:INSERT=5,nova_api:UPDATE=11 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:08:45.773958 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-05870d1f-7cb9-4c90-be7d-108ca56feba6 tempest-QuotasAdminTestJSON-972835255 tempest-QuotasAdminTestJSON-972835255-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:08:45.774745 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-05870d1f-7cb9-4c90-be7d-108ca56feba6 tempest-QuotasAdminTestJSON-972835255 tempest-QuotasAdminTestJSON-972835255-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:08:45.775922 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-05870d1f-7cb9-4c90-be7d-108ca56feba6 tempest-QuotasAdminTestJSON-972835255 tempest-QuotasAdminTestJSON-972835255-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:08:45.776574 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-05870d1f-7cb9-4c90-be7d-108ca56feba6 tempest-QuotasAdminTestJSON-972835255 tempest-QuotasAdminTestJSON-972835255-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:08:45.979492 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-05870d1f-7cb9-4c90-be7d-108ca56feba6 tempest-QuotasAdminTestJSON-972835255 tempest-QuotasAdminTestJSON-972835255-project-admin] 10.222.0.16 "PUT /compute/v2.1/os-quota-sets/b74bc8cc72c8405a9a5b124902016aff" status: 200 len: 328 microversion: 2.1 time: 0.427943 Jul 16 21:08:45.984154 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 76/160] 10.222.0.16 () {60 vars in 1261 bytes} [Sat Jul 16 21:08:45 2022] PUT /compute/v2.1/os-quota-sets/b74bc8cc72c8405a9a5b124902016aff => generated 328 bytes in 433 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 16 21:08:45.994437 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-ef93616d-f7a3-4d9d-b4d3-077457ce51d9 tempest-QuotasAdminTestJSON-972835255 tempest-QuotasAdminTestJSON-972835255-project-admin] Calling method '.version_select at 0x7fdd9daed820>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:08:46.173046 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-a412fa16-af76-457c-b0ff-bb70a643dda4 tempest-AggregatesAdminTestJSON-1464138310 tempest-AggregatesAdminTestJSON-1464138310-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/7895aae7-b50f-48cf-a3e1-5b33dc6e16a3" status: 200 len: 1638 microversion: 2.1 time: 1.735004 Jul 16 21:08:46.182481 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 85/161] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:08:44 2022] GET /compute/v2.1/servers/7895aae7-b50f-48cf-a3e1-5b33dc6e16a3 => generated 1638 bytes in 1747 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:08:46.187879 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ef93616d-f7a3-4d9d-b4d3-077457ce51d9 tempest-QuotasAdminTestJSON-972835255 tempest-QuotasAdminTestJSON-972835255-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:08:46.188448 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ef93616d-f7a3-4d9d-b4d3-077457ce51d9 tempest-QuotasAdminTestJSON-972835255 tempest-QuotasAdminTestJSON-972835255-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:08:46.189751 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ef93616d-f7a3-4d9d-b4d3-077457ce51d9 tempest-QuotasAdminTestJSON-972835255 tempest-QuotasAdminTestJSON-972835255-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:08:46.190471 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ef93616d-f7a3-4d9d-b4d3-077457ce51d9 tempest-QuotasAdminTestJSON-972835255 tempest-QuotasAdminTestJSON-972835255-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:08:46.203957 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-9023ba92-46d5-4960-9cff-3d8dd2e8d849 tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:08:46.207225 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-9023ba92-46d5-4960-9cff-3d8dd2e8d849 tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] [instance: 7895aae7-b50f-48cf-a3e1-5b33dc6e16a3] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:08:46.233021 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9023ba92-46d5-4960-9cff-3d8dd2e8d849 tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:08:46.236210 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9023ba92-46d5-4960-9cff-3d8dd2e8d849 tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:08:46.239774 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9023ba92-46d5-4960-9cff-3d8dd2e8d849 tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:08:46.240285 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9023ba92-46d5-4960-9cff-3d8dd2e8d849 tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:08:46.264616 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-ef93616d-f7a3-4d9d-b4d3-077457ce51d9 tempest-QuotasAdminTestJSON-972835255 tempest-QuotasAdminTestJSON-972835255-project-admin] 10.222.0.16 "GET /compute/v2.1/os-quota-sets/b74bc8cc72c8405a9a5b124902016aff/detail" status: 200 len: 916 microversion: 2.1 time: 0.273886 Jul 16 21:08:46.280411 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 77/162] 10.222.0.16 () {58 vars in 1262 bytes} [Sat Jul 16 21:08:45 2022] GET /compute/v2.1/os-quota-sets/b74bc8cc72c8405a9a5b124902016aff/detail => generated 916 bytes in 289 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:08:46.327684 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-9023ba92-46d5-4960-9cff-3d8dd2e8d849 tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] [instance: 7895aae7-b50f-48cf-a3e1-5b33dc6e16a3] Going to try to terminate instance {{(pid=138680) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 16 21:08:46.505890 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-cc93f22f-c60a-47cd-a0d6-c16065bd6ff3 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:08:46.506767 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-cc93f22f-c60a-47cd-a0d6-c16065bd6ff3 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: 1703f309-9e2e-4c5f-8588-eb4e43f21d91] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:08:46.531399 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cc93f22f-c60a-47cd-a0d6-c16065bd6ff3 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:08:46.540100 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cc93f22f-c60a-47cd-a0d6-c16065bd6ff3 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.009s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:08:46.544424 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cc93f22f-c60a-47cd-a0d6-c16065bd6ff3 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:08:46.547417 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cc93f22f-c60a-47cd-a0d6-c16065bd6ff3 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:08:46.802490 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-cc93f22f-c60a-47cd-a0d6-c16065bd6ff3 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b68dacc4b8463ba07fae6372131117', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55a19b7a4682457a9132f5fc80110712', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:08:46.813398 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-cc93f22f-c60a-47cd-a0d6-c16065bd6ff3 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "GET /compute/v2.1/servers/1703f309-9e2e-4c5f-8588-eb4e43f21d91" status: 200 len: 1312 microversion: 2.1 time: 0.311079 Jul 16 21:08:46.816358 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 78/163] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:08:46 2022] GET /compute/v2.1/servers/1703f309-9e2e-4c5f-8588-eb4e43f21d91 => generated 1312 bytes in 316 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:08:46.828328 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-f702120a-3dc4-4126-8d9d-ed838a98d85f tempest-QuotasAdminTestJSON-972835255 tempest-QuotasAdminTestJSON-972835255-project-admin] Action: 'update', calling method: .version_select at 0x7fdda03d6ee0>, body: {"quota_set": {"ram": "2048"}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:08:46.947889 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f702120a-3dc4-4126-8d9d-ed838a98d85f tempest-QuotasAdminTestJSON-972835255 tempest-QuotasAdminTestJSON-972835255-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:08:46.948522 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f702120a-3dc4-4126-8d9d-ed838a98d85f tempest-QuotasAdminTestJSON-972835255 tempest-QuotasAdminTestJSON-972835255-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:08:46.949641 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f702120a-3dc4-4126-8d9d-ed838a98d85f tempest-QuotasAdminTestJSON-972835255 tempest-QuotasAdminTestJSON-972835255-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:08:46.950183 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f702120a-3dc4-4126-8d9d-ed838a98d85f tempest-QuotasAdminTestJSON-972835255 tempest-QuotasAdminTestJSON-972835255-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:08:47.015915 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f702120a-3dc4-4126-8d9d-ed838a98d85f tempest-QuotasAdminTestJSON-972835255 tempest-QuotasAdminTestJSON-972835255-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:08:47.016699 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f702120a-3dc4-4126-8d9d-ed838a98d85f tempest-QuotasAdminTestJSON-972835255 tempest-QuotasAdminTestJSON-972835255-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:08:47.018631 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f702120a-3dc4-4126-8d9d-ed838a98d85f tempest-QuotasAdminTestJSON-972835255 tempest-QuotasAdminTestJSON-972835255-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:08:47.019251 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f702120a-3dc4-4126-8d9d-ed838a98d85f tempest-QuotasAdminTestJSON-972835255 tempest-QuotasAdminTestJSON-972835255-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:08:47.115823 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-9023ba92-46d5-4960-9cff-3d8dd2e8d849 tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] 10.222.0.16 "DELETE /compute/v2.1/servers/7895aae7-b50f-48cf-a3e1-5b33dc6e16a3" status: 204 len: 0 microversion: 2.1 time: 0.916543 Jul 16 21:08:47.117573 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 86/164] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:08:46 2022] DELETE /compute/v2.1/servers/7895aae7-b50f-48cf-a3e1-5b33dc6e16a3 => generated 0 bytes in 919 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Jul 16 21:08:47.132911 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-b3d0f915-d3b8-47f9-9e78-f22a96a3a7d7 tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:08:47.133869 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-b3d0f915-d3b8-47f9-9e78-f22a96a3a7d7 tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] [instance: 7895aae7-b50f-48cf-a3e1-5b33dc6e16a3] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:08:47.150192 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b3d0f915-d3b8-47f9-9e78-f22a96a3a7d7 tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:08:47.151719 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b3d0f915-d3b8-47f9-9e78-f22a96a3a7d7 tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:08:47.153072 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b3d0f915-d3b8-47f9-9e78-f22a96a3a7d7 tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:08:47.153520 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b3d0f915-d3b8-47f9-9e78-f22a96a3a7d7 tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:08:47.296570 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-f702120a-3dc4-4126-8d9d-ed838a98d85f tempest-QuotasAdminTestJSON-972835255 tempest-QuotasAdminTestJSON-972835255-project-admin] 10.222.0.16 "PUT /compute/v2.1/os-quota-sets/b74bc8cc72c8405a9a5b124902016aff?user_id=674d28367979435aa78462118d6a7256" status: 200 len: 328 microversion: 2.1 time: 0.478782 Jul 16 21:08:47.307421 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 79/165] 10.222.0.16 () {60 vars in 1342 bytes} [Sat Jul 16 21:08:46 2022] PUT /compute/v2.1/os-quota-sets/b74bc8cc72c8405a9a5b124902016aff?user_id=674d28367979435aa78462118d6a7256 => generated 328 bytes in 490 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:08:47.318175 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-9002d8e9-f7b9-432c-8a11-d1631c7c1864 tempest-QuotasAdminTestJSON-972835255 tempest-QuotasAdminTestJSON-972835255-project-admin] Calling method '.version_select at 0x7fdd9daed310>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:08:47.534605 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-b3d0f915-d3b8-47f9-9e78-f22a96a3a7d7 tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'db6c21ea09c54ea492d806f42d2f69a7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e7c343c974bd4ec0a6a765f818b895be', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:08:47.541894 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-9002d8e9-f7b9-432c-8a11-d1631c7c1864 tempest-QuotasAdminTestJSON-972835255 tempest-QuotasAdminTestJSON-972835255-project-admin] 10.222.0.16 "GET /compute/v2.1/os-quota-sets/b74bc8cc72c8405a9a5b124902016aff?user_id=674d28367979435aa78462118d6a7256" status: 200 len: 370 microversion: 2.1 time: 0.229693 Jul 16 21:08:47.543991 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 80/166] 10.222.0.16 () {58 vars in 1322 bytes} [Sat Jul 16 21:08:47 2022] GET /compute/v2.1/os-quota-sets/b74bc8cc72c8405a9a5b124902016aff?user_id=674d28367979435aa78462118d6a7256 => generated 370 bytes in 232 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:08:47.550982 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-b3d0f915-d3b8-47f9-9e78-f22a96a3a7d7 tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] 10.222.0.16 "GET /compute/v2.1/servers/7895aae7-b50f-48cf-a3e1-5b33dc6e16a3" status: 200 len: 1493 microversion: 2.1 time: 0.428109 Jul 16 21:08:47.552186 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 87/167] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:08:47 2022] GET /compute/v2.1/servers/7895aae7-b50f-48cf-a3e1-5b33dc6e16a3 => generated 1493 bytes in 431 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:08:47.850964 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-0b76385d-f4e9-4599-89a2-e7afccf985a0 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:08:47.851412 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-0b76385d-f4e9-4599-89a2-e7afccf985a0 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: 1703f309-9e2e-4c5f-8588-eb4e43f21d91] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:08:47.878420 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0b76385d-f4e9-4599-89a2-e7afccf985a0 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:08:47.879064 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0b76385d-f4e9-4599-89a2-e7afccf985a0 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:08:47.880070 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0b76385d-f4e9-4599-89a2-e7afccf985a0 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:08:47.880663 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0b76385d-f4e9-4599-89a2-e7afccf985a0 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:08:48.552499 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-0b76385d-f4e9-4599-89a2-e7afccf985a0 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b68dacc4b8463ba07fae6372131117', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55a19b7a4682457a9132f5fc80110712', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:08:48.565049 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-0b76385d-f4e9-4599-89a2-e7afccf985a0 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "GET /compute/v2.1/servers/1703f309-9e2e-4c5f-8588-eb4e43f21d91" status: 200 len: 1312 microversion: 2.1 time: 0.734137 Jul 16 21:08:48.566477 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 81/168] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:08:47 2022] GET /compute/v2.1/servers/1703f309-9e2e-4c5f-8588-eb4e43f21d91 => generated 1312 bytes in 738 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:08:48.576515 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-c58b4c5a-74f9-484e-907f-392c1eda55ca tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:08:48.576515 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-c58b4c5a-74f9-484e-907f-392c1eda55ca tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] [instance: 7895aae7-b50f-48cf-a3e1-5b33dc6e16a3] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:08:48.577648 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c58b4c5a-74f9-484e-907f-392c1eda55ca tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:08:48.578277 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c58b4c5a-74f9-484e-907f-392c1eda55ca tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:08:48.579317 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c58b4c5a-74f9-484e-907f-392c1eda55ca tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:08:48.580022 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c58b4c5a-74f9-484e-907f-392c1eda55ca tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:08:49.098729 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-b1e26cf7-72ba-4077-b560-bb2f37b7b923 tempest-QuotasAdminTestJSON-972835255 tempest-QuotasAdminTestJSON-972835255-project-admin] Calling method '.version_select at 0x7fdd9da8dd30>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:08:49.197033 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-c58b4c5a-74f9-484e-907f-392c1eda55ca tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'db6c21ea09c54ea492d806f42d2f69a7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e7c343c974bd4ec0a6a765f818b895be', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:08:49.215476 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-c58b4c5a-74f9-484e-907f-392c1eda55ca tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] 10.222.0.16 "GET /compute/v2.1/servers/7895aae7-b50f-48cf-a3e1-5b33dc6e16a3" status: 200 len: 1493 microversion: 2.1 time: 0.649958 Jul 16 21:08:49.223371 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 88/169] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:08:48 2022] GET /compute/v2.1/servers/7895aae7-b50f-48cf-a3e1-5b33dc6e16a3 => generated 1493 bytes in 659 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:08:49.592115 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-b1e26cf7-72ba-4077-b560-bb2f37b7b923 tempest-QuotasAdminTestJSON-972835255 tempest-QuotasAdminTestJSON-972835255-project-admin] 10.222.0.16 "GET /compute/v2.1/os-quota-sets/9dcf95f2466344d1a28724d545a8fed6/defaults" status: 200 len: 371 microversion: 2.1 time: 0.497882 Jul 16 21:08:49.592944 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 82/170] 10.222.0.16 () {58 vars in 1268 bytes} [Sat Jul 16 21:08:49 2022] GET /compute/v2.1/os-quota-sets/9dcf95f2466344d1a28724d545a8fed6/defaults => generated 371 bytes in 499 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:08:49.594947 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-c81fbf41-3a2d-45bd-a0c8-088f9c2916ad tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:08:49.597499 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-c81fbf41-3a2d-45bd-a0c8-088f9c2916ad tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: 1703f309-9e2e-4c5f-8588-eb4e43f21d91] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:08:49.604368 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-bfea9d5b-0f3e-4aca-b28a-79c671a19906 tempest-QuotasAdminTestJSON-972835255 tempest-QuotasAdminTestJSON-972835255-project-admin] Action: 'update', calling method: .version_select at 0x7fdd9db6c5e0>, body: {"quota_set": {"force": true, "metadata_items": 256, "ram": 10240, "key_pairs": 200, "instances": 20, "server_groups": 20, "server_group_members": 20, "cores": 2, "fixed_ips": 10, "floating_ips": 20, "security_group_rules": 20, "security_groups": 20, "injected_file_content_bytes": 20480, "injected_file_path_bytes": 512, "injected_files": 10}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:08:49.606320 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c81fbf41-3a2d-45bd-a0c8-088f9c2916ad tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:08:49.606958 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c81fbf41-3a2d-45bd-a0c8-088f9c2916ad tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:08:49.607991 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c81fbf41-3a2d-45bd-a0c8-088f9c2916ad tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:08:49.608621 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c81fbf41-3a2d-45bd-a0c8-088f9c2916ad tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:08:49.696152 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell0:SELECT=4 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:08:49.924760 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-bfea9d5b-0f3e-4aca-b28a-79c671a19906 tempest-QuotasAdminTestJSON-972835255 tempest-QuotasAdminTestJSON-972835255-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:08:49.928654 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-bfea9d5b-0f3e-4aca-b28a-79c671a19906 tempest-QuotasAdminTestJSON-972835255 tempest-QuotasAdminTestJSON-972835255-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:08:49.933896 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-bfea9d5b-0f3e-4aca-b28a-79c671a19906 tempest-QuotasAdminTestJSON-972835255 tempest-QuotasAdminTestJSON-972835255-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:08:49.934548 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-bfea9d5b-0f3e-4aca-b28a-79c671a19906 tempest-QuotasAdminTestJSON-972835255 tempest-QuotasAdminTestJSON-972835255-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:08:50.393792 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-c81fbf41-3a2d-45bd-a0c8-088f9c2916ad tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b68dacc4b8463ba07fae6372131117', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55a19b7a4682457a9132f5fc80110712', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:08:50.417557 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-c81fbf41-3a2d-45bd-a0c8-088f9c2916ad tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "GET /compute/v2.1/servers/1703f309-9e2e-4c5f-8588-eb4e43f21d91" status: 200 len: 1312 microversion: 2.1 time: 0.823905 Jul 16 21:08:50.427652 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 89/171] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:08:49 2022] GET /compute/v2.1/servers/1703f309-9e2e-4c5f-8588-eb4e43f21d91 => generated 1312 bytes in 839 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:08:50.432615 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-23d10023-7092-4285-af52-631925c48ba0 req-2eab3e49-7eec-4994-9226-a5fb4e8315d0 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "7895aae7-b50f-48cf-a3e1-5b33dc6e16a3", "name": "network-vif-deleted", "tag": "ce71fe98-d12f-40ec-b5fa-d43e42a5edeb"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:08:50.453035 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-23d10023-7092-4285-af52-631925c48ba0 req-2eab3e49-7eec-4994-9226-a5fb4e8315d0 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:08:50.453035 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-23d10023-7092-4285-af52-631925c48ba0 req-2eab3e49-7eec-4994-9226-a5fb4e8315d0 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:08:50.482763 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-23d10023-7092-4285-af52-631925c48ba0 req-2eab3e49-7eec-4994-9226-a5fb4e8315d0 service nova] Creating event network-vif-deleted:ce71fe98-d12f-40ec-b5fa-d43e42a5edeb for instance 7895aae7-b50f-48cf-a3e1-5b33dc6e16a3 on nch1-master-1 Jul 16 21:08:50.498883 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-23d10023-7092-4285-af52-631925c48ba0 req-2eab3e49-7eec-4994-9226-a5fb4e8315d0 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.069357 Jul 16 21:08:50.500122 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 90/172] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:08:50 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 72 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:08:50.511460 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-480128dc-af41-41ea-94aa-36f28164a9c6 tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:08:50.513088 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-480128dc-af41-41ea-94aa-36f28164a9c6 tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] [instance: 7895aae7-b50f-48cf-a3e1-5b33dc6e16a3] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:08:50.524943 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-480128dc-af41-41ea-94aa-36f28164a9c6 tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:08:50.525749 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-480128dc-af41-41ea-94aa-36f28164a9c6 tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:08:50.527933 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-480128dc-af41-41ea-94aa-36f28164a9c6 tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:08:50.528789 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-480128dc-af41-41ea-94aa-36f28164a9c6 tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:08:50.580208 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-480128dc-af41-41ea-94aa-36f28164a9c6 tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:08:50.580208 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-480128dc-af41-41ea-94aa-36f28164a9c6 tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:08:50.581054 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.objects.instance [None req-480128dc-af41-41ea-94aa-36f28164a9c6 tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] Lazy-loading 'fault' on Instance uuid 7895aae7-b50f-48cf-a3e1-5b33dc6e16a3 {{(pid=138680) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:08:51.088369 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-480128dc-af41-41ea-94aa-36f28164a9c6 tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'db6c21ea09c54ea492d806f42d2f69a7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e7c343c974bd4ec0a6a765f818b895be', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:08:51.106706 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-480128dc-af41-41ea-94aa-36f28164a9c6 tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] 10.222.0.16 "GET /compute/v2.1/servers/7895aae7-b50f-48cf-a3e1-5b33dc6e16a3" status: 200 len: 1327 microversion: 2.1 time: 0.602767 Jul 16 21:08:51.107640 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 91/173] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:08:50 2022] GET /compute/v2.1/servers/7895aae7-b50f-48cf-a3e1-5b33dc6e16a3 => generated 1327 bytes in 605 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:08:51.440782 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-b25d7742-7822-462b-be47-0dbfb5a952ae tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:08:51.441808 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-b25d7742-7822-462b-be47-0dbfb5a952ae tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: 1703f309-9e2e-4c5f-8588-eb4e43f21d91] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:08:51.491631 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b25d7742-7822-462b-be47-0dbfb5a952ae tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:08:51.499204 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b25d7742-7822-462b-be47-0dbfb5a952ae tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:08:51.502281 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b25d7742-7822-462b-be47-0dbfb5a952ae tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:08:51.503724 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b25d7742-7822-462b-be47-0dbfb5a952ae tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:08:51.720039 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-bfea9d5b-0f3e-4aca-b28a-79c671a19906 tempest-QuotasAdminTestJSON-972835255 tempest-QuotasAdminTestJSON-972835255-project-admin] 10.222.0.16 "PUT /compute/v2.1/os-quota-sets/9dcf95f2466344d1a28724d545a8fed6" status: 200 len: 329 microversion: 2.1 time: 2.119712 Jul 16 21:08:51.722329 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 83/174] 10.222.0.16 () {60 vars in 1262 bytes} [Sat Jul 16 21:08:49 2022] PUT /compute/v2.1/os-quota-sets/9dcf95f2466344d1a28724d545a8fed6 => generated 329 bytes in 2123 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:08:51.735054 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-8524e499-248d-4232-ac88-6f0945bdeac9 tempest-QuotasAdminTestJSON-972835255 tempest-QuotasAdminTestJSON-972835255-project-admin] Action: 'update', calling method: .version_select at 0x7fdda0401310>, body: {"quota_set": {"cores": 20, "fixed_ips": -1, "floating_ips": -1, "injected_file_content_bytes": 10240, "injected_file_path_bytes": 255, "injected_files": 5, "instances": 10, "key_pairs": 100, "metadata_items": 128, "ram": 51200, "security_group_rules": -1, "security_groups": -1, "server_group_members": 10, "server_groups": 10}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:08:52.000793 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-b25d7742-7822-462b-be47-0dbfb5a952ae tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b68dacc4b8463ba07fae6372131117', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55a19b7a4682457a9132f5fc80110712', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:08:52.018380 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-b25d7742-7822-462b-be47-0dbfb5a952ae tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "GET /compute/v2.1/servers/1703f309-9e2e-4c5f-8588-eb4e43f21d91" status: 200 len: 1312 microversion: 2.1 time: 0.584026 Jul 16 21:08:52.018980 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 92/175] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:08:51 2022] GET /compute/v2.1/servers/1703f309-9e2e-4c5f-8588-eb4e43f21d91 => generated 1312 bytes in 585 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:08:52.125756 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-4544e10c-54e7-4271-ae2f-2370a74fbfdd tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:08:52.126904 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-4544e10c-54e7-4271-ae2f-2370a74fbfdd tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] [instance: 7895aae7-b50f-48cf-a3e1-5b33dc6e16a3] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:08:52.133721 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4544e10c-54e7-4271-ae2f-2370a74fbfdd tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:08:52.134177 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4544e10c-54e7-4271-ae2f-2370a74fbfdd tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:08:52.134972 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4544e10c-54e7-4271-ae2f-2370a74fbfdd tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:08:52.135462 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4544e10c-54e7-4271-ae2f-2370a74fbfdd tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:08:52.145517 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:08:52.158531 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-4544e10c-54e7-4271-ae2f-2370a74fbfdd tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] HTTP exception thrown: Instance 7895aae7-b50f-48cf-a3e1-5b33dc6e16a3 could not be found. Jul 16 21:08:52.163500 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-4544e10c-54e7-4271-ae2f-2370a74fbfdd tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] Request method failure captured: Jul 16 21:08:52.163500 ncdv-master-1 devstack@n-api.service[138680]: request: GET /compute/v2.1/servers/7895aae7-b50f-48cf-a3e1-5b33dc6e16a3 HTTP/1.1 Jul 16 21:08:52.163500 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:08:52.163500 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:08:52.163500 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:08:52.163500 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 0 Jul 16 21:08:52.163500 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:08:52.163500 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:08:52.163500 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:08:52.163500 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:08:52.163500 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0yiwEDc6jCrIXaeq91y19seHWVuAydqowazCbU3Ptgxp4XZeouI5kZu6lb6qVjBtzG4PXc_eGUoHi4NASCxzS51pZnxcShhc_TTetcJdupN3R2A3t1jzrc5iKUJ5B24npTLiwYERA2Q_WmIZ5KSaFY0acD8RGBvhEitiUAFojMftS2o Jul 16 21:08:52.163500 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:08:52.163500 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:08:52.163500 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:08:52.163500 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:08:52.163500 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:08:52.163500 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:08:52.163500 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: e7c343c974bd4ec0a6a765f818b895be Jul 16 21:08:52.163500 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-AggregatesAdminTestJSON-412945753 Jul 16 21:08:52.163500 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:08:52.163500 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:08:52.164934 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/e7c343c974bd4ec0a6a765f818b895be"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/e7c343c974bd4ec0a6a765f818b895be"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_e7c343c974bd4ec0a6a765f818b895be"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/e7c343c974bd4ec0a6a765f818b895be"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:08:52.164934 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-AggregatesAdminTestJSON-412945753 Jul 16 21:08:52.164934 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: e7c343c974bd4ec0a6a765f818b895be Jul 16 21:08:52.164934 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-AggregatesAdminTestJSON-412945753 Jul 16 21:08:52.164934 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-AggregatesAdminTestJSON-412945753-project Jul 16 21:08:52.164934 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:08:52.164934 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:08:52.164934 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: db6c21ea09c54ea492d806f42d2f69a7 Jul 16 21:08:52.164934 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-AggregatesAdminTestJSON-412945753-project Jul 16 21:08:52.164934 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 21:08:52.164934 ncdv-master-1 devstack@n-api.service[138680]: exception: Instance 7895aae7-b50f-48cf-a3e1-5b33dc6e16a3 could not be found. Jul 16 21:08:52.164934 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': '7895aae7-b50f-48cf-a3e1-5b33dc6e16a3'} Jul 16 21:08:52.165730 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:08:52.165730 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:08:52.165730 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:08:52.165730 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:08:52.165730 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:08:52.165730 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:08:52.165730 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:08:52.165730 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:08:52.165730 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:08:52.165730 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:08:52.165730 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 7895aae7-b50f-48cf-a3e1-5b33dc6e16a3 could not be found. Jul 16 21:08:52.165730 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:08:52.165730 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:08:52.165730 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:08:52.165730 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:08:52.165730 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:08:52.165730 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:08:52.165730 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:08:52.165730 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:08:52.165730 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:08:52.166978 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:08:52.166978 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:08:52.166978 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:08:52.166978 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:08:52.166978 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:08:52.166978 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:08:52.166978 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:08:52.166978 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 7895aae7-b50f-48cf-a3e1-5b33dc6e16a3 could not be found. Jul 16 21:08:52.166978 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:08:52.166978 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:08:52.166978 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:08:52.166978 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:08:52.166978 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:08:52.166978 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:08:52.166978 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:08:52.166978 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:08:52.166978 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 7895aae7-b50f-48cf-a3e1-5b33dc6e16a3 could not be found. Jul 16 21:08:52.166978 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:08:52.169087 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-4544e10c-54e7-4271-ae2f-2370a74fbfdd tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] Returning 404 to user: Instance 7895aae7-b50f-48cf-a3e1-5b33dc6e16a3 could not be found. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:08:52.170467 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-4544e10c-54e7-4271-ae2f-2370a74fbfdd tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] 10.222.0.16 "GET /compute/v2.1/servers/7895aae7-b50f-48cf-a3e1-5b33dc6e16a3" status: 404 len: 111 microversion: 2.1 time: 0.049634 Jul 16 21:08:52.171751 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 93/176] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:08:52 2022] GET /compute/v2.1/servers/7895aae7-b50f-48cf-a3e1-5b33dc6e16a3 => generated 111 bytes in 51 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Jul 16 21:08:52.181887 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-84cfc9ca-38a5-4155-befc-59e5f118bfcd tempest-AggregatesAdminTestJSON-1464138310 tempest-AggregatesAdminTestJSON-1464138310-project-admin] Action: 'action', calling method: >, body: {"remove_host": {"host": "nch1-master-1"}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:08:52.183821 ncdv-master-1 devstack@n-api.service[138680]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-aggregates:remove_host": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:08:52.183821 ncdv-master-1 devstack@n-api.service[138680]: warnings.warn(msg) Jul 16 21:08:52.194379 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-84cfc9ca-38a5-4155-befc-59e5f118bfcd tempest-AggregatesAdminTestJSON-1464138310 tempest-AggregatesAdminTestJSON-1464138310-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:08:52.196091 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-84cfc9ca-38a5-4155-befc-59e5f118bfcd tempest-AggregatesAdminTestJSON-1464138310 tempest-AggregatesAdminTestJSON-1464138310-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:08:52.237005 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8524e499-248d-4232-ac88-6f0945bdeac9 tempest-QuotasAdminTestJSON-972835255 tempest-QuotasAdminTestJSON-972835255-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:08:52.240570 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8524e499-248d-4232-ac88-6f0945bdeac9 tempest-QuotasAdminTestJSON-972835255 tempest-QuotasAdminTestJSON-972835255-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:08:52.241706 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8524e499-248d-4232-ac88-6f0945bdeac9 tempest-QuotasAdminTestJSON-972835255 tempest-QuotasAdminTestJSON-972835255-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:08:52.242238 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8524e499-248d-4232-ac88-6f0945bdeac9 tempest-QuotasAdminTestJSON-972835255 tempest-QuotasAdminTestJSON-972835255-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:08:52.309888 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_api:SELECT=84,nova_api:INSERT=16 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:08:52.667771 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-84cfc9ca-38a5-4155-befc-59e5f118bfcd tempest-AggregatesAdminTestJSON-1464138310 tempest-AggregatesAdminTestJSON-1464138310-project-admin] 10.222.0.16 "POST /compute/v2.1/os-aggregates/7/action" status: 200 len: 300 microversion: 2.1 time: 0.489232 Jul 16 21:08:52.680556 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 94/177] 10.222.0.16 () {60 vars in 1190 bytes} [Sat Jul 16 21:08:52 2022] POST /compute/v2.1/os-aggregates/7/action => generated 300 bytes in 504 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:08:52.686683 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-da17a894-f6cc-40d3-8dd1-cc9c492315f8 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServersWithSpecificFlavorTestJSON-server-1570144937", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "1856122662", "adminPass": "***", "networks": [{"uuid": "7ee2ff5a-751a-444f-a143-2ba55b7e64b7"}], "security_groups": [{"name": "tempest-securitygroup--1281138757"}], "key_name": "tempest-keypair-461056992", "user_data": "}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:08:52.945509 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell0:SELECT=12 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:08:53.153582 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.network.neutron [None req-da17a894-f6cc-40d3-8dd1-cc9c492315f8 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] validate_networks() for [('7ee2ff5a-751a-444f-a143-2ba55b7e64b7', None, None, None, None, None)] {{(pid=138680) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 16 21:08:54.122780 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell0:SELECT=100 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:08:54.936171 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-8524e499-248d-4232-ac88-6f0945bdeac9 tempest-QuotasAdminTestJSON-972835255 tempest-QuotasAdminTestJSON-972835255-project-admin] 10.222.0.16 "PUT /compute/v2.1/os-quota-sets/9dcf95f2466344d1a28724d545a8fed6" status: 200 len: 329 microversion: 2.1 time: 3.200127 Jul 16 21:08:54.940258 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 84/178] 10.222.0.16 () {60 vars in 1262 bytes} [Sat Jul 16 21:08:51 2022] PUT /compute/v2.1/os-quota-sets/9dcf95f2466344d1a28724d545a8fed6 => generated 329 bytes in 3210 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:08:54.946647 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-145529d2-9176-47df-98b2-20bfd7a56e0b tempest-AggregatesAdminTestJSON-1464138310 tempest-AggregatesAdminTestJSON-1464138310-project-admin] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:08:54.949623 ncdv-master-1 devstack@n-api.service[138681]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-aggregates:delete": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:08:54.949623 ncdv-master-1 devstack@n-api.service[138681]: warnings.warn(msg) Jul 16 21:08:55.068013 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-145529d2-9176-47df-98b2-20bfd7a56e0b tempest-AggregatesAdminTestJSON-1464138310 tempest-AggregatesAdminTestJSON-1464138310-project-admin] 10.222.0.16 "DELETE /compute/v2.1/os-aggregates/7" status: 200 len: 0 microversion: 2.1 time: 0.121615 Jul 16 21:08:55.068013 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 85/179] 10.222.0.16 () {58 vars in 1151 bytes} [Sat Jul 16 21:08:54 2022] DELETE /compute/v2.1/os-aggregates/7 => generated 0 bytes in 124 msecs (HTTP/1.1 200) 9 headers in 355 bytes (1 switches on core 0) Jul 16 21:08:55.088116 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-a5923492-bb85-494b-b2b3-4cdf86f14e0e req-b02036d1-f683-4f1c-a430-4768242c8ae0 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "1703f309-9e2e-4c5f-8588-eb4e43f21d91", "tag": "d97c4e64-34a5-4e3d-b81c-2020594e3e0b"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:08:55.110830 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-a5923492-bb85-494b-b2b3-4cdf86f14e0e req-b02036d1-f683-4f1c-a430-4768242c8ae0 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:08:55.115704 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-a5923492-bb85-494b-b2b3-4cdf86f14e0e req-b02036d1-f683-4f1c-a430-4768242c8ae0 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:08:55.149801 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-a5923492-bb85-494b-b2b3-4cdf86f14e0e req-b02036d1-f683-4f1c-a430-4768242c8ae0 service nova] Creating event network-changed:d97c4e64-34a5-4e3d-b81c-2020594e3e0b for instance 1703f309-9e2e-4c5f-8588-eb4e43f21d91 on nch2-master-1 Jul 16 21:08:55.156331 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-a5923492-bb85-494b-b2b3-4cdf86f14e0e req-b02036d1-f683-4f1c-a430-4768242c8ae0 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.089709 Jul 16 21:08:55.157241 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 86/180] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:08:55 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:08:55.165346 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-a3dd48ed-34e3-4d79-8985-7f311ca8f817 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:08:55.168042 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-a3dd48ed-34e3-4d79-8985-7f311ca8f817 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: 1703f309-9e2e-4c5f-8588-eb4e43f21d91] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:08:55.195927 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a3dd48ed-34e3-4d79-8985-7f311ca8f817 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:08:55.200250 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a3dd48ed-34e3-4d79-8985-7f311ca8f817 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:08:55.201721 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a3dd48ed-34e3-4d79-8985-7f311ca8f817 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:08:55.202355 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a3dd48ed-34e3-4d79-8985-7f311ca8f817 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:08:55.656051 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-a3dd48ed-34e3-4d79-8985-7f311ca8f817 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b68dacc4b8463ba07fae6372131117', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55a19b7a4682457a9132f5fc80110712', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:08:55.675089 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-da17a894-f6cc-40d3-8dd1-cc9c492315f8 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Flavor limits 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 16 21:08:55.675089 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-da17a894-f6cc-40d3-8dd1-cc9c492315f8 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Image limits 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 16 21:08:55.675089 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-da17a894-f6cc-40d3-8dd1-cc9c492315f8 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Flavor pref 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 16 21:08:55.675089 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-da17a894-f6cc-40d3-8dd1-cc9c492315f8 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Image pref 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 16 21:08:55.676355 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-da17a894-f6cc-40d3-8dd1-cc9c492315f8 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 16 21:08:55.696320 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-a3dd48ed-34e3-4d79-8985-7f311ca8f817 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "GET /compute/v2.1/servers/1703f309-9e2e-4c5f-8588-eb4e43f21d91" status: 200 len: 1312 microversion: 2.1 time: 0.527052 Jul 16 21:08:55.701724 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 87/181] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:08:55 2022] GET /compute/v2.1/servers/1703f309-9e2e-4c5f-8588-eb4e43f21d91 => generated 1312 bytes in 543 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:08:55.713559 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-da17a894-f6cc-40d3-8dd1-cc9c492315f8 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:08:55.714334 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-da17a894-f6cc-40d3-8dd1-cc9c492315f8 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:08:55.719517 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-da17a894-f6cc-40d3-8dd1-cc9c492315f8 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:08:55.720297 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-da17a894-f6cc-40d3-8dd1-cc9c492315f8 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:08:55.770612 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-da17a894-f6cc-40d3-8dd1-cc9c492315f8 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Getting quotas for project d0ee02e4c8b448f2b77798ae504129b9. Resources: {'ram', 'instances', 'cores'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:08:55.780238 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-da17a894-f6cc-40d3-8dd1-cc9c492315f8 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Getting quotas for user ef9cbce2b2e64665b7f50767342dcb46 and project d0ee02e4c8b448f2b77798ae504129b9. Resources: {'ram', 'instances', 'cores'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:08:55.811696 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-da17a894-f6cc-40d3-8dd1-cc9c492315f8 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Going to run 1 instances... {{(pid=138680) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 16 21:08:55.816662 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-da17a894-f6cc-40d3-8dd1-cc9c492315f8 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] [instance: e506a6e0-fbcd-4e53-a6a0-d31761654c92] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138680) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 16 21:08:56.005433 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-da17a894-f6cc-40d3-8dd1-cc9c492315f8 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 402 microversion: 2.1 time: 3.321586 Jul 16 21:08:56.007714 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 95/182] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 21:08:52 2022] POST /compute/v2.1/servers => generated 402 bytes in 3326 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 16 21:08:56.025089 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-b21b9e67-88d7-43a2-953d-f0a5c88b9bb4 tempest-AggregatesAdminTestJSON-1464138310 tempest-AggregatesAdminTestJSON-1464138310-project-admin] Action: 'create', calling method: >, body: {"aggregate": {"name": "tempest-test_aggregate-1636292066"}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:08:56.029864 ncdv-master-1 devstack@n-api.service[138680]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-aggregates:create": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:08:56.029864 ncdv-master-1 devstack@n-api.service[138680]: warnings.warn(msg) Jul 16 21:08:56.036097 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.objects.aggregate [None req-b21b9e67-88d7-43a2-953d-f0a5c88b9bb4 tempest-AggregatesAdminTestJSON-1464138310 tempest-AggregatesAdminTestJSON-1464138310-project-admin] Generated uuid 4b317a5c-0724-4dec-942b-ec13e7c2bcb4 for aggregate {{(pid=138680) create /opt/stack/nova/nova/objects/aggregate.py:298}} Jul 16 21:08:56.204074 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-d869fd8b-78db-4e3d-8f5b-2674943d4a25 tempest-HypervisorAdminV228Test-620836045 tempest-HypervisorAdminV228Test-620836045-project-admin] Calling method '.version_select at 0x7fdd9da52790>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:08:56.205030 ncdv-master-1 devstack@n-api.service[138681]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-hypervisors:list": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:08:56.205030 ncdv-master-1 devstack@n-api.service[138681]: warnings.warn(msg) Jul 16 21:08:56.207155 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d869fd8b-78db-4e3d-8f5b-2674943d4a25 tempest-HypervisorAdminV228Test-620836045 tempest-HypervisorAdminV228Test-620836045-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:08:56.207794 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d869fd8b-78db-4e3d-8f5b-2674943d4a25 tempest-HypervisorAdminV228Test-620836045 tempest-HypervisorAdminV228Test-620836045-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:08:56.221546 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-b21b9e67-88d7-43a2-953d-f0a5c88b9bb4 tempest-AggregatesAdminTestJSON-1464138310 tempest-AggregatesAdminTestJSON-1464138310-project-admin] 10.222.0.16 "POST /compute/v2.1/os-aggregates" status: 200 len: 198 microversion: 2.1 time: 0.212413 Jul 16 21:08:56.222175 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 96/183] 10.222.0.16 () {60 vars in 1163 bytes} [Sat Jul 16 21:08:56 2022] POST /compute/v2.1/os-aggregates => generated 198 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:08:56.227769 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-33b36b97-b66e-47fe-99c1-5cc878833fe3 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:08:56.231852 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-33b36b97-b66e-47fe-99c1-5cc878833fe3 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] [instance: e506a6e0-fbcd-4e53-a6a0-d31761654c92] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:08:56.259865 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d869fd8b-78db-4e3d-8f5b-2674943d4a25 tempest-HypervisorAdminV228Test-620836045 tempest-HypervisorAdminV228Test-620836045-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:08:56.260344 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d869fd8b-78db-4e3d-8f5b-2674943d4a25 tempest-HypervisorAdminV228Test-620836045 tempest-HypervisorAdminV228Test-620836045-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:08:56.304993 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d869fd8b-78db-4e3d-8f5b-2674943d4a25 tempest-HypervisorAdminV228Test-620836045 tempest-HypervisorAdminV228Test-620836045-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:08:56.305516 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d869fd8b-78db-4e3d-8f5b-2674943d4a25 tempest-HypervisorAdminV228Test-620836045 tempest-HypervisorAdminV228Test-620836045-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:08:56.321047 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-d869fd8b-78db-4e3d-8f5b-2674943d4a25 tempest-HypervisorAdminV228Test-620836045 tempest-HypervisorAdminV228Test-620836045-project-admin] 10.222.0.16 "GET /compute/v2.1/os-hypervisors" status: 200 len: 191 microversion: 2.28 time: 0.617726 Jul 16 21:08:56.326239 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 88/184] 10.222.0.16 () {60 vars in 1186 bytes} [Sat Jul 16 21:08:55 2022] GET /compute/v2.1/os-hypervisors => generated 191 bytes in 619 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 16 21:08:56.340381 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-750df69e-534a-4049-a67e-86e964034e6c tempest-AggregatesAdminTestJSON-1464138310 tempest-AggregatesAdminTestJSON-1464138310-project-admin] Action: 'action', calling method: >, body: {"add_host": {"host": "nch1-master-1"}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:08:56.343320 ncdv-master-1 devstack@n-api.service[138681]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-aggregates:add_host": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:08:56.343320 ncdv-master-1 devstack@n-api.service[138681]: warnings.warn(msg) Jul 16 21:08:56.355585 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-750df69e-534a-4049-a67e-86e964034e6c tempest-AggregatesAdminTestJSON-1464138310 tempest-AggregatesAdminTestJSON-1464138310-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:08:56.356451 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-750df69e-534a-4049-a67e-86e964034e6c tempest-AggregatesAdminTestJSON-1464138310 tempest-AggregatesAdminTestJSON-1464138310-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:08:56.381132 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-33b36b97-b66e-47fe-99c1-5cc878833fe3 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ef9cbce2b2e64665b7f50767342dcb46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0ee02e4c8b448f2b77798ae504129b9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:08:56.389445 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-33b36b97-b66e-47fe-99c1-5cc878833fe3 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] 10.222.0.16 "GET /compute/v2.1/servers/e506a6e0-fbcd-4e53-a6a0-d31761654c92" status: 200 len: 1264 microversion: 2.1 time: 0.166189 Jul 16 21:08:56.393028 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 97/185] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:08:56 2022] GET /compute/v2.1/servers/e506a6e0-fbcd-4e53-a6a0-d31761654c92 => generated 1264 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:08:56.413456 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-12cdcf8b-19bf-4bc7-bde3-1f16f3e96e9b tempest-HypervisorAdminV228Test-620836045 tempest-HypervisorAdminV228Test-620836045-project-admin] Calling method '.version_select at 0x7fdd9dd92b80>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:08:56.414577 ncdv-master-1 devstack@n-api.service[138680]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-hypervisors:show": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:08:56.414577 ncdv-master-1 devstack@n-api.service[138680]: warnings.warn(msg) Jul 16 21:08:56.416242 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-12cdcf8b-19bf-4bc7-bde3-1f16f3e96e9b tempest-HypervisorAdminV228Test-620836045 tempest-HypervisorAdminV228Test-620836045-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:08:56.419160 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-12cdcf8b-19bf-4bc7-bde3-1f16f3e96e9b tempest-HypervisorAdminV228Test-620836045 tempest-HypervisorAdminV228Test-620836045-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:08:56.443526 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-12cdcf8b-19bf-4bc7-bde3-1f16f3e96e9b tempest-HypervisorAdminV228Test-620836045 tempest-HypervisorAdminV228Test-620836045-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:08:56.444019 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-12cdcf8b-19bf-4bc7-bde3-1f16f3e96e9b tempest-HypervisorAdminV228Test-620836045 tempest-HypervisorAdminV228Test-620836045-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:08:56.451802 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-12cdcf8b-19bf-4bc7-bde3-1f16f3e96e9b tempest-HypervisorAdminV228Test-620836045 tempest-HypervisorAdminV228Test-620836045-project-admin] 10.222.0.16 "GET /compute/v2.1/os-hypervisors/1" status: 200 len: 717 microversion: 2.28 time: 0.046744 Jul 16 21:08:56.461029 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 98/186] 10.222.0.16 () {60 vars in 1192 bytes} [Sat Jul 16 21:08:56 2022] GET /compute/v2.1/os-hypervisors/1 => generated 717 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 16 21:08:56.718227 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-4be3b5e9-b157-4ad2-b0ea-41a771523eef tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:08:56.719273 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-4be3b5e9-b157-4ad2-b0ea-41a771523eef tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: 1703f309-9e2e-4c5f-8588-eb4e43f21d91] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:08:56.725844 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-750df69e-534a-4049-a67e-86e964034e6c tempest-AggregatesAdminTestJSON-1464138310 tempest-AggregatesAdminTestJSON-1464138310-project-admin] 10.222.0.16 "POST /compute/v2.1/os-aggregates/9/action" status: 200 len: 242 microversion: 2.1 time: 0.402733 Jul 16 21:08:56.727333 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 89/187] 10.222.0.16 () {60 vars in 1190 bytes} [Sat Jul 16 21:08:56 2022] POST /compute/v2.1/os-aggregates/9/action => generated 242 bytes in 405 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:08:56.739370 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4be3b5e9-b157-4ad2-b0ea-41a771523eef tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:08:56.739879 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4be3b5e9-b157-4ad2-b0ea-41a771523eef tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:08:56.740711 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-7c8b3162-006a-4a94-b312-6e17cb8927bd tempest-AggregatesAdminTestJSON-1464138310 tempest-AggregatesAdminTestJSON-1464138310-project-admin] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:08:56.745140 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4be3b5e9-b157-4ad2-b0ea-41a771523eef tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:08:56.745587 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4be3b5e9-b157-4ad2-b0ea-41a771523eef tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:08:56.747289 ncdv-master-1 devstack@n-api.service[138681]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-aggregates:show": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:08:56.747289 ncdv-master-1 devstack@n-api.service[138681]: warnings.warn(msg) Jul 16 21:08:56.774934 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-7c8b3162-006a-4a94-b312-6e17cb8927bd tempest-AggregatesAdminTestJSON-1464138310 tempest-AggregatesAdminTestJSON-1464138310-project-admin] 10.222.0.16 "GET /compute/v2.1/os-aggregates/9" status: 200 len: 242 microversion: 2.1 time: 0.041161 Jul 16 21:08:56.775947 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 90/188] 10.222.0.16 () {58 vars in 1148 bytes} [Sat Jul 16 21:08:56 2022] GET /compute/v2.1/os-aggregates/9 => generated 242 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:08:56.792568 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-5ab3629d-b603-4c56-819f-bd32d7a2e74f tempest-AggregatesAdminTestJSON-1464138310 tempest-AggregatesAdminTestJSON-1464138310-project-admin] Action: 'action', calling method: >, body: {"remove_host": {"host": "nch1-master-1"}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:08:56.793665 ncdv-master-1 devstack@n-api.service[138681]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-aggregates:remove_host": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:08:56.793665 ncdv-master-1 devstack@n-api.service[138681]: warnings.warn(msg) Jul 16 21:08:56.810198 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5ab3629d-b603-4c56-819f-bd32d7a2e74f tempest-AggregatesAdminTestJSON-1464138310 tempest-AggregatesAdminTestJSON-1464138310-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:08:56.810664 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5ab3629d-b603-4c56-819f-bd32d7a2e74f tempest-AggregatesAdminTestJSON-1464138310 tempest-AggregatesAdminTestJSON-1464138310-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:08:56.983903 ncdv-master-1 devstack@n-api.service[138681]: WARNING nova.scheduler.client.report [None req-5ab3629d-b603-4c56-819f-bd32d7a2e74f tempest-AggregatesAdminTestJSON-1464138310 tempest-AggregatesAdminTestJSON-1464138310-project-admin] [req-cc1c46cc-7f42-4749-a646-746c01309201] Failed to update aggregates to [] for resource provider with UUID fcc75857-7dde-4ac0-88e4-871d39001997. Got 409: {"errors": [{"status": 409, "title": "Conflict", "detail": "There was a conflict when trying to complete your request.\n\n Resource provider's generation already changed. Please update the generation and try again. ", "request_id": "req-cc1c46cc-7f42-4749-a646-746c01309201"}]} Jul 16 21:08:57.450751 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-5ab3629d-b603-4c56-819f-bd32d7a2e74f tempest-AggregatesAdminTestJSON-1464138310 tempest-AggregatesAdminTestJSON-1464138310-project-admin] 10.222.0.16 "POST /compute/v2.1/os-aggregates/9/action" status: 200 len: 227 microversion: 2.1 time: 0.661999 Jul 16 21:08:57.453824 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 91/189] 10.222.0.16 () {60 vars in 1190 bytes} [Sat Jul 16 21:08:56 2022] POST /compute/v2.1/os-aggregates/9/action => generated 227 bytes in 666 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:08:57.468706 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-a40a1196-548d-44bd-8c72-cb7397b0bb95 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:08:57.474173 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-a40a1196-548d-44bd-8c72-cb7397b0bb95 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] [instance: e506a6e0-fbcd-4e53-a6a0-d31761654c92] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:08:57.813433 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-4be3b5e9-b157-4ad2-b0ea-41a771523eef tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b68dacc4b8463ba07fae6372131117', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55a19b7a4682457a9132f5fc80110712', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:08:57.814070 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-a40a1196-548d-44bd-8c72-cb7397b0bb95 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ef9cbce2b2e64665b7f50767342dcb46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0ee02e4c8b448f2b77798ae504129b9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:08:57.835521 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-a40a1196-548d-44bd-8c72-cb7397b0bb95 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] 10.222.0.16 "GET /compute/v2.1/servers/e506a6e0-fbcd-4e53-a6a0-d31761654c92" status: 200 len: 1264 microversion: 2.1 time: 0.371198 Jul 16 21:08:57.850841 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-4be3b5e9-b157-4ad2-b0ea-41a771523eef tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "GET /compute/v2.1/servers/1703f309-9e2e-4c5f-8588-eb4e43f21d91" status: 200 len: 1312 microversion: 2.1 time: 1.137539 Jul 16 21:08:57.858402 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 92/190] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:08:57 2022] GET /compute/v2.1/servers/e506a6e0-fbcd-4e53-a6a0-d31761654c92 => generated 1264 bytes in 404 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:08:57.880325 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 99/191] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:08:56 2022] GET /compute/v2.1/servers/1703f309-9e2e-4c5f-8588-eb4e43f21d91 => generated 1312 bytes in 1168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:08:57.893245 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-00f13ed7-4cc7-49a2-9888-b0c9ecc8344b tempest-AggregatesAdminTestJSON-1464138310 tempest-AggregatesAdminTestJSON-1464138310-project-admin] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:08:57.894849 ncdv-master-1 devstack@n-api.service[138681]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-aggregates:delete": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:08:57.894849 ncdv-master-1 devstack@n-api.service[138681]: warnings.warn(msg) Jul 16 21:08:58.040150 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-00f13ed7-4cc7-49a2-9888-b0c9ecc8344b tempest-AggregatesAdminTestJSON-1464138310 tempest-AggregatesAdminTestJSON-1464138310-project-admin] 10.222.0.16 "DELETE /compute/v2.1/os-aggregates/9" status: 200 len: 0 microversion: 2.1 time: 0.179421 Jul 16 21:08:58.041063 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 93/192] 10.222.0.16 () {58 vars in 1151 bytes} [Sat Jul 16 21:08:57 2022] DELETE /compute/v2.1/os-aggregates/9 => generated 0 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 355 bytes (1 switches on core 0) Jul 16 21:08:58.052464 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-8d132b90-8406-49b4-9a29-5dc96a539969 tempest-AggregatesAdminTestJSON-1464138310 tempest-AggregatesAdminTestJSON-1464138310-project-admin] Action: 'create', calling method: >, body: {"aggregate": {"name": "tempest-test_aggregate-1005088982"}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:08:58.054735 ncdv-master-1 devstack@n-api.service[138680]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-aggregates:create": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:08:58.054735 ncdv-master-1 devstack@n-api.service[138680]: warnings.warn(msg) Jul 16 21:08:58.056084 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.objects.aggregate [None req-8d132b90-8406-49b4-9a29-5dc96a539969 tempest-AggregatesAdminTestJSON-1464138310 tempest-AggregatesAdminTestJSON-1464138310-project-admin] Generated uuid 25a412e1-d64f-4bd0-bb32-606b174c9011 for aggregate {{(pid=138680) create /opt/stack/nova/nova/objects/aggregate.py:298}} Jul 16 21:08:58.223659 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-8d132b90-8406-49b4-9a29-5dc96a539969 tempest-AggregatesAdminTestJSON-1464138310 tempest-AggregatesAdminTestJSON-1464138310-project-admin] 10.222.0.16 "POST /compute/v2.1/os-aggregates" status: 200 len: 199 microversion: 2.1 time: 0.175658 Jul 16 21:08:58.224281 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 100/193] 10.222.0.16 () {60 vars in 1163 bytes} [Sat Jul 16 21:08:58 2022] POST /compute/v2.1/os-aggregates => generated 199 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:08:58.242505 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-a10fee10-9a06-4898-8fb3-545be9e032d2 tempest-AggregatesAdminTestJSON-1464138310 tempest-AggregatesAdminTestJSON-1464138310-project-admin] Action: 'action', calling method: >, body: {"add_host": {"host": "nch1-master-1"}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:08:58.245218 ncdv-master-1 devstack@n-api.service[138681]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-aggregates:add_host": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:08:58.245218 ncdv-master-1 devstack@n-api.service[138681]: warnings.warn(msg) Jul 16 21:08:58.256794 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_api:SELECT=66,nova_api:INSERT=15,nova_api:UPDATE=14,nova_api:DELETE=5 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:08:58.265572 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a10fee10-9a06-4898-8fb3-545be9e032d2 tempest-AggregatesAdminTestJSON-1464138310 tempest-AggregatesAdminTestJSON-1464138310-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:08:58.266287 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a10fee10-9a06-4898-8fb3-545be9e032d2 tempest-AggregatesAdminTestJSON-1464138310 tempest-AggregatesAdminTestJSON-1464138310-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:08:58.297145 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=100 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:08:58.754535 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-a10fee10-9a06-4898-8fb3-545be9e032d2 tempest-AggregatesAdminTestJSON-1464138310 tempest-AggregatesAdminTestJSON-1464138310-project-admin] 10.222.0.16 "POST /compute/v2.1/os-aggregates/10/action" status: 200 len: 243 microversion: 2.1 time: 0.523541 Jul 16 21:08:58.760921 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 94/194] 10.222.0.16 () {60 vars in 1193 bytes} [Sat Jul 16 21:08:58 2022] POST /compute/v2.1/os-aggregates/10/action => generated 243 bytes in 531 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:08:58.765459 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-35ad7c21-3c16-435e-8468-9ffdb0a3af14 tempest-AggregatesAdminTestJSON-1464138310 tempest-AggregatesAdminTestJSON-1464138310-project-admin] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:08:58.766420 ncdv-master-1 devstack@n-api.service[138680]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-aggregates:index": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:08:58.766420 ncdv-master-1 devstack@n-api.service[138680]: warnings.warn(msg) Jul 16 21:08:58.793544 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-35ad7c21-3c16-435e-8468-9ffdb0a3af14 tempest-AggregatesAdminTestJSON-1464138310 tempest-AggregatesAdminTestJSON-1464138310-project-admin] 10.222.0.16 "GET /compute/v2.1/os-aggregates" status: 200 len: 246 microversion: 2.1 time: 0.032062 Jul 16 21:08:58.796840 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 101/195] 10.222.0.16 () {58 vars in 1142 bytes} [Sat Jul 16 21:08:58 2022] GET /compute/v2.1/os-aggregates => generated 246 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:08:58.808752 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-c66ea6c9-810f-4abb-aa96-41baf3f84bc7 tempest-AggregatesAdminTestJSON-1464138310 tempest-AggregatesAdminTestJSON-1464138310-project-admin] Action: 'action', calling method: >, body: {"remove_host": {"host": "nch1-master-1"}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:08:58.810042 ncdv-master-1 devstack@n-api.service[138681]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-aggregates:remove_host": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:08:58.810042 ncdv-master-1 devstack@n-api.service[138681]: warnings.warn(msg) Jul 16 21:08:58.827802 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c66ea6c9-810f-4abb-aa96-41baf3f84bc7 tempest-AggregatesAdminTestJSON-1464138310 tempest-AggregatesAdminTestJSON-1464138310-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:08:58.828263 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c66ea6c9-810f-4abb-aa96-41baf3f84bc7 tempest-AggregatesAdminTestJSON-1464138310 tempest-AggregatesAdminTestJSON-1464138310-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:08:58.856855 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-93f7e536-5b14-4a07-8bc5-a8fe2189491d tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:08:58.858124 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-93f7e536-5b14-4a07-8bc5-a8fe2189491d tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] [instance: e506a6e0-fbcd-4e53-a6a0-d31761654c92] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:08:58.877085 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-93f7e536-5b14-4a07-8bc5-a8fe2189491d tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:08:58.879984 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-93f7e536-5b14-4a07-8bc5-a8fe2189491d tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:08:58.881051 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-93f7e536-5b14-4a07-8bc5-a8fe2189491d tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:08:58.881699 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-93f7e536-5b14-4a07-8bc5-a8fe2189491d tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:08:59.081275 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-93f7e536-5b14-4a07-8bc5-a8fe2189491d tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ef9cbce2b2e64665b7f50767342dcb46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0ee02e4c8b448f2b77798ae504129b9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:08:59.103358 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-93f7e536-5b14-4a07-8bc5-a8fe2189491d tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] 10.222.0.16 "GET /compute/v2.1/servers/e506a6e0-fbcd-4e53-a6a0-d31761654c92" status: 200 len: 1260 microversion: 2.1 time: 0.249340 Jul 16 21:08:59.105087 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 102/196] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:08:58 2022] GET /compute/v2.1/servers/e506a6e0-fbcd-4e53-a6a0-d31761654c92 => generated 1260 bytes in 253 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:08:59.117528 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-59b42305-19ab-42a7-8c9c-cd426302099c tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:08:59.119588 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-59b42305-19ab-42a7-8c9c-cd426302099c tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: 1703f309-9e2e-4c5f-8588-eb4e43f21d91] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:08:59.139790 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-59b42305-19ab-42a7-8c9c-cd426302099c tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:08:59.142742 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-59b42305-19ab-42a7-8c9c-cd426302099c tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:08:59.143800 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-59b42305-19ab-42a7-8c9c-cd426302099c tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:08:59.152176 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-59b42305-19ab-42a7-8c9c-cd426302099c tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:08:59.210477 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-c66ea6c9-810f-4abb-aa96-41baf3f84bc7 tempest-AggregatesAdminTestJSON-1464138310 tempest-AggregatesAdminTestJSON-1464138310-project-admin] 10.222.0.16 "POST /compute/v2.1/os-aggregates/10/action" status: 200 len: 228 microversion: 2.1 time: 0.405962 Jul 16 21:08:59.223358 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 95/197] 10.222.0.16 () {60 vars in 1193 bytes} [Sat Jul 16 21:08:58 2022] POST /compute/v2.1/os-aggregates/10/action => generated 228 bytes in 409 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:08:59.233721 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-10c840fd-3cba-4390-ad88-7ed8c19186a0 tempest-AggregatesAdminTestJSON-1464138310 tempest-AggregatesAdminTestJSON-1464138310-project-admin] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:08:59.237509 ncdv-master-1 devstack@n-api.service[138681]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-aggregates:delete": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:08:59.237509 ncdv-master-1 devstack@n-api.service[138681]: warnings.warn(msg) Jul 16 21:08:59.377742 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-10c840fd-3cba-4390-ad88-7ed8c19186a0 tempest-AggregatesAdminTestJSON-1464138310 tempest-AggregatesAdminTestJSON-1464138310-project-admin] 10.222.0.16 "DELETE /compute/v2.1/os-aggregates/10" status: 200 len: 0 microversion: 2.1 time: 0.152917 Jul 16 21:08:59.378335 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 96/198] 10.222.0.16 () {58 vars in 1154 bytes} [Sat Jul 16 21:08:59 2022] DELETE /compute/v2.1/os-aggregates/10 => generated 0 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 355 bytes (1 switches on core 0) Jul 16 21:08:59.413096 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-81460a42-6051-4814-b9cf-d71b33439553 tempest-AggregatesAdminTestJSON-1464138310 tempest-AggregatesAdminTestJSON-1464138310-project-admin] Action: 'create', calling method: >, body: {"aggregate": {"name": "tempest-test_aggregate-459773065"}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:08:59.416236 ncdv-master-1 devstack@n-api.service[138681]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-aggregates:create": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:08:59.416236 ncdv-master-1 devstack@n-api.service[138681]: warnings.warn(msg) Jul 16 21:08:59.420052 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.objects.aggregate [None req-81460a42-6051-4814-b9cf-d71b33439553 tempest-AggregatesAdminTestJSON-1464138310 tempest-AggregatesAdminTestJSON-1464138310-project-admin] Generated uuid c2585843-0993-4b0f-a6be-f5f9465241a0 for aggregate {{(pid=138681) create /opt/stack/nova/nova/objects/aggregate.py:298}} Jul 16 21:08:59.559351 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-81460a42-6051-4814-b9cf-d71b33439553 tempest-AggregatesAdminTestJSON-1464138310 tempest-AggregatesAdminTestJSON-1464138310-project-admin] 10.222.0.16 "POST /compute/v2.1/os-aggregates" status: 200 len: 198 microversion: 2.1 time: 0.169873 Jul 16 21:08:59.561013 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 97/199] 10.222.0.16 () {60 vars in 1163 bytes} [Sat Jul 16 21:08:59 2022] POST /compute/v2.1/os-aggregates => generated 198 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:08:59.573382 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-68b24931-5a47-45ad-87ab-cf51cf1627cb tempest-AggregatesAdminTestJSON-1464138310 tempest-AggregatesAdminTestJSON-1464138310-project-admin] Action: 'action', calling method: >, body: {"add_host": {"host": "nch1-master-1"}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:08:59.575495 ncdv-master-1 devstack@n-api.service[138681]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-aggregates:add_host": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:08:59.575495 ncdv-master-1 devstack@n-api.service[138681]: warnings.warn(msg) Jul 16 21:08:59.586688 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-68b24931-5a47-45ad-87ab-cf51cf1627cb tempest-AggregatesAdminTestJSON-1464138310 tempest-AggregatesAdminTestJSON-1464138310-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:08:59.589968 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-68b24931-5a47-45ad-87ab-cf51cf1627cb tempest-AggregatesAdminTestJSON-1464138310 tempest-AggregatesAdminTestJSON-1464138310-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:08:59.609063 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-59b42305-19ab-42a7-8c9c-cd426302099c tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b68dacc4b8463ba07fae6372131117', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55a19b7a4682457a9132f5fc80110712', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:08:59.624141 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-59b42305-19ab-42a7-8c9c-cd426302099c tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "GET /compute/v2.1/servers/1703f309-9e2e-4c5f-8588-eb4e43f21d91" status: 200 len: 1312 microversion: 2.1 time: 0.517932 Jul 16 21:08:59.624737 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 103/200] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:08:59 2022] GET /compute/v2.1/servers/1703f309-9e2e-4c5f-8588-eb4e43f21d91 => generated 1312 bytes in 519 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:09:00.088236 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-68b24931-5a47-45ad-87ab-cf51cf1627cb tempest-AggregatesAdminTestJSON-1464138310 tempest-AggregatesAdminTestJSON-1464138310-project-admin] 10.222.0.16 "POST /compute/v2.1/os-aggregates/11/action" status: 200 len: 242 microversion: 2.1 time: 0.520498 Jul 16 21:09:00.089067 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 98/201] 10.222.0.16 () {60 vars in 1193 bytes} [Sat Jul 16 21:08:59 2022] POST /compute/v2.1/os-aggregates/11/action => generated 242 bytes in 522 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:09:00.099577 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-43e1c40e-9d04-4ba3-b430-e04aec64935f tempest-AggregatesAdminTestJSON-1464138310 tempest-AggregatesAdminTestJSON-1464138310-project-admin] Action: 'action', calling method: >, body: {"remove_host": {"host": "nch1-master-1"}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:09:00.100419 ncdv-master-1 devstack@n-api.service[138680]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-aggregates:remove_host": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:09:00.100419 ncdv-master-1 devstack@n-api.service[138680]: warnings.warn(msg) Jul 16 21:09:00.118628 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-43e1c40e-9d04-4ba3-b430-e04aec64935f tempest-AggregatesAdminTestJSON-1464138310 tempest-AggregatesAdminTestJSON-1464138310-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:00.119122 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-43e1c40e-9d04-4ba3-b430-e04aec64935f tempest-AggregatesAdminTestJSON-1464138310 tempest-AggregatesAdminTestJSON-1464138310-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:00.123861 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-a55ba068-2fef-438f-a42f-f84b23765684 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:00.124957 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-a55ba068-2fef-438f-a42f-f84b23765684 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] [instance: e506a6e0-fbcd-4e53-a6a0-d31761654c92] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:09:00.143425 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a55ba068-2fef-438f-a42f-f84b23765684 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:00.146579 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a55ba068-2fef-438f-a42f-f84b23765684 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:00.151694 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a55ba068-2fef-438f-a42f-f84b23765684 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:00.152805 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a55ba068-2fef-438f-a42f-f84b23765684 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:00.355530 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-a55ba068-2fef-438f-a42f-f84b23765684 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ef9cbce2b2e64665b7f50767342dcb46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0ee02e4c8b448f2b77798ae504129b9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:00.364719 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-a55ba068-2fef-438f-a42f-f84b23765684 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] 10.222.0.16 "GET /compute/v2.1/servers/e506a6e0-fbcd-4e53-a6a0-d31761654c92" status: 200 len: 1334 microversion: 2.1 time: 0.245377 Jul 16 21:09:00.380472 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 99/202] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:09:00 2022] GET /compute/v2.1/servers/e506a6e0-fbcd-4e53-a6a0-d31761654c92 => generated 1334 bytes in 262 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:09:00.480839 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-43e1c40e-9d04-4ba3-b430-e04aec64935f tempest-AggregatesAdminTestJSON-1464138310 tempest-AggregatesAdminTestJSON-1464138310-project-admin] 10.222.0.16 "POST /compute/v2.1/os-aggregates/11/action" status: 200 len: 227 microversion: 2.1 time: 0.385684 Jul 16 21:09:00.481806 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 104/203] 10.222.0.16 () {60 vars in 1193 bytes} [Sat Jul 16 21:09:00 2022] POST /compute/v2.1/os-aggregates/11/action => generated 227 bytes in 387 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:09:00.494179 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-2ed7be53-86c6-42b4-920a-c00e505a22bb tempest-AggregatesAdminTestJSON-1464138310 tempest-AggregatesAdminTestJSON-1464138310-project-admin] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:00.494971 ncdv-master-1 devstack@n-api.service[138681]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-aggregates:delete": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:09:00.494971 ncdv-master-1 devstack@n-api.service[138681]: warnings.warn(msg) Jul 16 21:09:00.604821 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-2ed7be53-86c6-42b4-920a-c00e505a22bb tempest-AggregatesAdminTestJSON-1464138310 tempest-AggregatesAdminTestJSON-1464138310-project-admin] 10.222.0.16 "DELETE /compute/v2.1/os-aggregates/11" status: 200 len: 0 microversion: 2.1 time: 0.114225 Jul 16 21:09:00.606585 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 100/204] 10.222.0.16 () {58 vars in 1154 bytes} [Sat Jul 16 21:09:00 2022] DELETE /compute/v2.1/os-aggregates/11 => generated 0 bytes in 117 msecs (HTTP/1.1 200) 9 headers in 355 bytes (2 switches on core 0) Jul 16 21:09:00.618117 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-5f4433c1-73bc-4681-aabe-ad50adb0c123 tempest-AggregatesAdminTestJSON-1464138310 tempest-AggregatesAdminTestJSON-1464138310-project-admin] Action: 'create', calling method: >, body: {"aggregate": {"name": "tempest-test_aggregate-1361920941"}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:09:00.618643 ncdv-master-1 devstack@n-api.service[138680]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-aggregates:create": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:09:00.618643 ncdv-master-1 devstack@n-api.service[138680]: warnings.warn(msg) Jul 16 21:09:00.619630 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.objects.aggregate [None req-5f4433c1-73bc-4681-aabe-ad50adb0c123 tempest-AggregatesAdminTestJSON-1464138310 tempest-AggregatesAdminTestJSON-1464138310-project-admin] Generated uuid defc3f31-530d-453b-a21a-8d05bca47dd2 for aggregate {{(pid=138680) create /opt/stack/nova/nova/objects/aggregate.py:298}} Jul 16 21:09:00.644974 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-3923b434-b6af-4ea4-89e5-6aa4becdeffa tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:00.646072 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-3923b434-b6af-4ea4-89e5-6aa4becdeffa tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: 1703f309-9e2e-4c5f-8588-eb4e43f21d91] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:09:00.659716 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3923b434-b6af-4ea4-89e5-6aa4becdeffa tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:00.665512 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_api:SELECT=90,nova_api:INSERT=7,nova_api:UPDATE=1,nova_api:DELETE=2 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:09:00.670561 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3923b434-b6af-4ea4-89e5-6aa4becdeffa tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.009s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:00.671573 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3923b434-b6af-4ea4-89e5-6aa4becdeffa tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:00.673920 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3923b434-b6af-4ea4-89e5-6aa4becdeffa tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:00.734416 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-5f4433c1-73bc-4681-aabe-ad50adb0c123 tempest-AggregatesAdminTestJSON-1464138310 tempest-AggregatesAdminTestJSON-1464138310-project-admin] 10.222.0.16 "POST /compute/v2.1/os-aggregates" status: 200 len: 199 microversion: 2.1 time: 0.121812 Jul 16 21:09:00.736755 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 105/205] 10.222.0.16 () {60 vars in 1163 bytes} [Sat Jul 16 21:09:00 2022] POST /compute/v2.1/os-aggregates => generated 199 bytes in 125 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:09:00.751995 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-a0fd2e92-d71c-4824-ab15-a9bde07a20f8 tempest-AggregatesAdminNegativeTestJSON-578260511 tempest-AggregatesAdminNegativeTestJSON-578260511-project-admin] Action: 'create', calling method: >, body: {"aggregate": {"name": "tempest-test_aggregate-1530919956"}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:09:00.753986 ncdv-master-1 devstack@n-api.service[138680]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-aggregates:create": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:09:00.753986 ncdv-master-1 devstack@n-api.service[138680]: warnings.warn(msg) Jul 16 21:09:00.755056 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.objects.aggregate [None req-a0fd2e92-d71c-4824-ab15-a9bde07a20f8 tempest-AggregatesAdminNegativeTestJSON-578260511 tempest-AggregatesAdminNegativeTestJSON-578260511-project-admin] Generated uuid a040d07e-fff3-4dd6-b303-4bab64072697 for aggregate {{(pid=138680) create /opt/stack/nova/nova/objects/aggregate.py:298}} Jul 16 21:09:00.908747 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-a0fd2e92-d71c-4824-ab15-a9bde07a20f8 tempest-AggregatesAdminNegativeTestJSON-578260511 tempest-AggregatesAdminNegativeTestJSON-578260511-project-admin] 10.222.0.16 "POST /compute/v2.1/os-aggregates" status: 200 len: 199 microversion: 2.1 time: 0.169272 Jul 16 21:09:00.910079 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 106/206] 10.222.0.16 () {60 vars in 1163 bytes} [Sat Jul 16 21:09:00 2022] POST /compute/v2.1/os-aggregates => generated 199 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:09:00.921575 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-b36d1477-dcea-4b2b-b3c7-894dc6cdbb56 tempest-AggregatesAdminTestJSON-1464138310 tempest-AggregatesAdminTestJSON-1464138310-project-admin] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:00.925982 ncdv-master-1 devstack@n-api.service[138680]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-aggregates:delete": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:09:00.925982 ncdv-master-1 devstack@n-api.service[138680]: warnings.warn(msg) Jul 16 21:09:01.054506 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-b36d1477-dcea-4b2b-b3c7-894dc6cdbb56 tempest-AggregatesAdminTestJSON-1464138310 tempest-AggregatesAdminTestJSON-1464138310-project-admin] 10.222.0.16 "DELETE /compute/v2.1/os-aggregates/12" status: 200 len: 0 microversion: 2.1 time: 0.141532 Jul 16 21:09:01.056695 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 107/207] 10.222.0.16 () {58 vars in 1154 bytes} [Sat Jul 16 21:09:00 2022] DELETE /compute/v2.1/os-aggregates/12 => generated 0 bytes in 146 msecs (HTTP/1.1 200) 9 headers in 355 bytes (1 switches on core 0) Jul 16 21:09:01.073985 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-886830b9-041e-417c-a71c-dad52d3956a4 tempest-AggregatesAdminNegativeTestJSON-578260511 tempest-AggregatesAdminNegativeTestJSON-578260511-project-admin] Action: 'action', calling method: >, body: {"add_host": {"host": "nch1-master-1"}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:09:01.076359 ncdv-master-1 devstack@n-api.service[138680]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-aggregates:add_host": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:09:01.076359 ncdv-master-1 devstack@n-api.service[138680]: warnings.warn(msg) Jul 16 21:09:01.098180 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-886830b9-041e-417c-a71c-dad52d3956a4 tempest-AggregatesAdminNegativeTestJSON-578260511 tempest-AggregatesAdminNegativeTestJSON-578260511-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:01.102921 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-886830b9-041e-417c-a71c-dad52d3956a4 tempest-AggregatesAdminNegativeTestJSON-578260511 tempest-AggregatesAdminNegativeTestJSON-578260511-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:01.597205 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-3923b434-b6af-4ea4-89e5-6aa4becdeffa tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b68dacc4b8463ba07fae6372131117', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55a19b7a4682457a9132f5fc80110712', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:01.617272 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-3923b434-b6af-4ea4-89e5-6aa4becdeffa tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "GET /compute/v2.1/servers/1703f309-9e2e-4c5f-8588-eb4e43f21d91" status: 200 len: 1312 microversion: 2.1 time: 0.975771 Jul 16 21:09:01.625409 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 101/208] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:09:00 2022] GET /compute/v2.1/servers/1703f309-9e2e-4c5f-8588-eb4e43f21d91 => generated 1312 bytes in 986 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:09:01.630681 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-ca277274-9fef-46f7-bae9-bfa91f89cf19 tempest-AggregatesAdminTestJSON-1464138310 tempest-AggregatesAdminTestJSON-1464138310-project-admin] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:01.632612 ncdv-master-1 devstack@n-api.service[138681]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-aggregates:show": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:09:01.632612 ncdv-master-1 devstack@n-api.service[138681]: warnings.warn(msg) Jul 16 21:09:01.650613 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-ca277274-9fef-46f7-bae9-bfa91f89cf19 tempest-AggregatesAdminTestJSON-1464138310 tempest-AggregatesAdminTestJSON-1464138310-project-admin] HTTP exception thrown: Aggregate 12 could not be found. Jul 16 21:09:01.652800 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-ca277274-9fef-46f7-bae9-bfa91f89cf19 tempest-AggregatesAdminTestJSON-1464138310 tempest-AggregatesAdminTestJSON-1464138310-project-admin] Request method failure captured: Jul 16 21:09:01.652800 ncdv-master-1 devstack@n-api.service[138681]: request: GET /compute/v2.1/os-aggregates/12 HTTP/1.1 Jul 16 21:09:01.652800 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:09:01.652800 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:09:01.652800 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:09:01.652800 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 21:09:01.652800 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:09:01.652800 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:09:01.652800 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:09:01.652800 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:09:01.652800 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0yizDAIMDUK3k-KWik016QtqhsacLSD0saiv-JOeGfYuNvqsavIUAkZ3iBuHBSfP7qFl5ZoaybYHJVK0ceO6C1sb4NOOxGeHPq_diZrK7gsGhKDcg10XIFJ7sD9WC1LOTi7KhM7YeGVI_dihO0AmMnpFYju7juyvUZsnsFHzskWIAKU Jul 16 21:09:01.652800 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:09:01.652800 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:09:01.652800 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:09:01.652800 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:09:01.652800 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:09:01.652800 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:09:01.652800 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: 09da0450b1464083990b4b3566862cd6 Jul 16 21:09:01.652800 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-AggregatesAdminTestJSON-1464138310 Jul 16 21:09:01.652800 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,admin,reader Jul 16 21:09:01.652800 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,admin,reader Jul 16 21:09:01.654049 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/09da0450b1464083990b4b3566862cd6"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/09da0450b1464083990b4b3566862cd6"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_09da0450b1464083990b4b3566862cd6"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/09da0450b1464083990b4b3566862cd6"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:09:01.654049 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-AggregatesAdminTestJSON-1464138310 Jul 16 21:09:01.654049 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: 09da0450b1464083990b4b3566862cd6 Jul 16 21:09:01.654049 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-AggregatesAdminTestJSON-1464138310 Jul 16 21:09:01.654049 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-AggregatesAdminTestJSON-1464138310-project-admin Jul 16 21:09:01.654049 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:09:01.654049 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:09:01.654049 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: ceb1a15568df445abebf4220c6894ca9 Jul 16 21:09:01.654049 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-AggregatesAdminTestJSON-1464138310-project-admin Jul 16 21:09:01.654049 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:09:01.654049 ncdv-master-1 devstack@n-api.service[138681]: exception: Aggregate 12 could not be found. Jul 16 21:09:01.654049 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': '12'} Jul 16 21:09:01.654049 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:09:01.654821 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:09:01.654821 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/aggregates.py", line 108, in show Jul 16 21:09:01.654821 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi aggregate = self.api.get_aggregate(context, id) Jul 16 21:09:01.654821 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 6336, in get_aggregate Jul 16 21:09:01.654821 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return objects.Aggregate.get_by_id(context, aggregate_id) Jul 16 21:09:01.654821 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Jul 16 21:09:01.654821 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Jul 16 21:09:01.654821 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/aggregate.py", line 274, in get_by_id Jul 16 21:09:01.654821 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi db_aggregate = _aggregate_get_from_db(context, aggregate_id) Jul 16 21:09:01.654821 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1010, in wrapper Jul 16 21:09:01.654821 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return fn(*args, **kwargs) Jul 16 21:09:01.654821 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/aggregate.py", line 45, in _aggregate_get_from_db Jul 16 21:09:01.654821 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.AggregateNotFound(aggregate_id=aggregate_id) Jul 16 21:09:01.654821 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.AggregateNotFound: Aggregate 12 could not be found. Jul 16 21:09:01.654821 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:09:01.654821 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:09:01.654821 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:09:01.654821 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:09:01.654821 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:09:01.656408 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:09:01.656408 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:09:01.656408 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:09:01.656408 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:09:01.656408 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:09:01.656408 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/aggregates.py", line 110, in show Jul 16 21:09:01.656408 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:09:01.656408 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Aggregate 12 could not be found. Jul 16 21:09:01.656408 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:09:01.656408 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:09:01.656408 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:09:01.656408 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:09:01.656408 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:09:01.656408 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:09:01.656408 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:09:01.656408 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:09:01.656408 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Aggregate 12 could not be found. Jul 16 21:09:01.656408 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:09:01.662545 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-ca277274-9fef-46f7-bae9-bfa91f89cf19 tempest-AggregatesAdminTestJSON-1464138310 tempest-AggregatesAdminTestJSON-1464138310-project-admin] Returning 404 to user: Aggregate 12 could not be found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:09:01.663844 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-ca277274-9fef-46f7-bae9-bfa91f89cf19 tempest-AggregatesAdminTestJSON-1464138310 tempest-AggregatesAdminTestJSON-1464138310-project-admin] 10.222.0.16 "GET /compute/v2.1/os-aggregates/12" status: 404 len: 78 microversion: 2.1 time: 0.037181 Jul 16 21:09:01.671401 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 102/209] 10.222.0.16 () {58 vars in 1151 bytes} [Sat Jul 16 21:09:01 2022] GET /compute/v2.1/os-aggregates/12 => generated 78 bytes in 45 msecs (HTTP/1.1 404) 9 headers in 378 bytes (1 switches on core 0) Jul 16 21:09:01.682685 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-716f9a75-c50d-43e7-8f29-4fe1831c702b tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:01.686444 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-716f9a75-c50d-43e7-8f29-4fe1831c702b tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] [instance: e506a6e0-fbcd-4e53-a6a0-d31761654c92] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:09:01.705364 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-716f9a75-c50d-43e7-8f29-4fe1831c702b tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:01.708222 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-716f9a75-c50d-43e7-8f29-4fe1831c702b tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:01.709425 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-716f9a75-c50d-43e7-8f29-4fe1831c702b tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:01.710418 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-716f9a75-c50d-43e7-8f29-4fe1831c702b tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:01.937300 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-716f9a75-c50d-43e7-8f29-4fe1831c702b tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ef9cbce2b2e64665b7f50767342dcb46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0ee02e4c8b448f2b77798ae504129b9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:01.952925 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-716f9a75-c50d-43e7-8f29-4fe1831c702b tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] 10.222.0.16 "GET /compute/v2.1/servers/e506a6e0-fbcd-4e53-a6a0-d31761654c92" status: 200 len: 1322 microversion: 2.1 time: 0.271992 Jul 16 21:09:01.958287 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 103/210] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:09:01 2022] GET /compute/v2.1/servers/e506a6e0-fbcd-4e53-a6a0-d31761654c92 => generated 1322 bytes in 286 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:09:01.975578 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-0be89395-0977-4a89-b342-efe9180cbf72 tempest-AggregatesAdminTestJSON-1464138310 tempest-AggregatesAdminTestJSON-1464138310-project-admin] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:01.979123 ncdv-master-1 devstack@n-api.service[138681]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-aggregates:delete": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:09:01.979123 ncdv-master-1 devstack@n-api.service[138681]: warnings.warn(msg) Jul 16 21:09:01.980386 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-886830b9-041e-417c-a71c-dad52d3956a4 tempest-AggregatesAdminNegativeTestJSON-578260511 tempest-AggregatesAdminNegativeTestJSON-578260511-project-admin] 10.222.0.16 "POST /compute/v2.1/os-aggregates/13/action" status: 200 len: 243 microversion: 2.1 time: 0.920278 Jul 16 21:09:01.980386 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 108/211] 10.222.0.16 () {60 vars in 1193 bytes} [Sat Jul 16 21:09:01 2022] POST /compute/v2.1/os-aggregates/13/action => generated 243 bytes in 923 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:09:01.994181 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-16610027-db7a-4699-a523-263bcf06b34a tempest-AggregatesAdminNegativeTestJSON-1271440314 tempest-AggregatesAdminNegativeTestJSON-1271440314-project] Action: 'action', calling method: >, body: {"remove_host": {"host": "nch1-master-1"}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:09:01.997144 ncdv-master-1 devstack@n-api.service[138680]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-aggregates:remove_host": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:09:01.997144 ncdv-master-1 devstack@n-api.service[138680]: warnings.warn(msg) Jul 16 21:09:02.002938 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-16610027-db7a-4699-a523-263bcf06b34a tempest-AggregatesAdminNegativeTestJSON-1271440314 tempest-AggregatesAdminNegativeTestJSON-1271440314-project] Policy check for os_compute_api:os-aggregates:remove_host failed with credentials {'is_admin': False, 'user_id': '35a4905d463a4db2960294407f747f20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '49044125b0414a4ab0e2ab9eb7d66e6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:02.010366 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-16610027-db7a-4699-a523-263bcf06b34a tempest-AggregatesAdminNegativeTestJSON-1271440314 tempest-AggregatesAdminNegativeTestJSON-1271440314-project] Request method failure captured: Jul 16 21:09:02.010366 ncdv-master-1 devstack@n-api.service[138680]: request: POST /compute/v2.1/os-aggregates/13/action HTTP/1.1 Jul 16 21:09:02.010366 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:09:02.010366 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:09:02.010366 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:09:02.010366 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 42 Jul 16 21:09:02.010366 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:09:02.010366 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:09:02.010366 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:09:02.010366 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:09:02.010366 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0yiwx37gJG3UPf6cs7N4UkgaksWm_yG7GacwHzT4X_Y36InoB6bLW8dxMeWTnPNUr5MkHyB6zCa2ShiCwvtwzG8FdDgsivxSANP9O8r63IrJOV2dcu5-Fla-kUEiqkTUeLu0mggNWQr19K_Q1rkm8xG96kdc5KRDvRt2-aIVSJpj_Ug Jul 16 21:09:02.010366 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:09:02.010366 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:09:02.010366 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:09:02.010366 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:09:02.010366 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:09:02.010366 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:09:02.010366 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: 49044125b0414a4ab0e2ab9eb7d66e6e Jul 16 21:09:02.010366 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-AggregatesAdminNegativeTestJSON-1271440314 Jul 16 21:09:02.010366 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:09:02.010366 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:09:02.011740 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/49044125b0414a4ab0e2ab9eb7d66e6e"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/49044125b0414a4ab0e2ab9eb7d66e6e"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_49044125b0414a4ab0e2ab9eb7d66e6e"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/49044125b0414a4ab0e2ab9eb7d66e6e"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:09:02.011740 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-AggregatesAdminNegativeTestJSON-1271440314 Jul 16 21:09:02.011740 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: 49044125b0414a4ab0e2ab9eb7d66e6e Jul 16 21:09:02.011740 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-AggregatesAdminNegativeTestJSON-1271440314 Jul 16 21:09:02.011740 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-AggregatesAdminNegativeTestJSON-1271440314-project Jul 16 21:09:02.011740 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:09:02.011740 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:09:02.011740 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: 35a4905d463a4db2960294407f747f20 Jul 16 21:09:02.011740 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-AggregatesAdminNegativeTestJSON-1271440314-project Jul 16 21:09:02.011740 ncdv-master-1 devstack@n-api.service[138680]: Jul 16 21:09:02.011740 ncdv-master-1 devstack@n-api.service[138680]: {"remove_host": {"host": "nch1-master-1"}} Jul 16 21:09:02.011740 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 21:09:02.011740 ncdv-master-1 devstack@n-api.service[138680]: exception: Policy doesn't allow os_compute_api:os-aggregates:remove_host to be performed. Jul 16 21:09:02.012506 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': '13', 'body': {'remove_host': {'host': 'nch1-master-1'}}} Jul 16 21:09:02.012506 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:09:02.012506 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:09:02.012506 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:09:02.012506 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:09:02.012506 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:09:02.012506 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:09:02.012506 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:09:02.012506 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:09:02.012506 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 16 21:09:02.012506 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 16 21:09:02.012506 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/aggregates.py", line 201, in _remove_host Jul 16 21:09:02.012506 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi context.can(aggr_policies.POLICY_ROOT % 'remove_host', target={}) Jul 16 21:09:02.012506 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/context.py", line 240, in can Jul 16 21:09:02.012506 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return policy.authorize(self, action, target) Jul 16 21:09:02.012506 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 199, in authorize Jul 16 21:09:02.012506 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi LOG.debug('Policy check for %(action)s failed with credentials ' Jul 16 21:09:02.012506 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 16 21:09:02.012506 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi self.force_reraise() Jul 16 21:09:02.013559 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 16 21:09:02.013559 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise self.value Jul 16 21:09:02.013559 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 186, in authorize Jul 16 21:09:02.013559 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi result = _ENFORCER.authorize(action, target, context, Jul 16 21:09:02.013559 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1184, in authorize Jul 16 21:09:02.013559 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return self.enforce( Jul 16 21:09:02.013559 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1088, in enforce Jul 16 21:09:02.013559 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc(*args, **kwargs) Jul 16 21:09:02.013559 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.PolicyNotAuthorized: Policy doesn't allow os_compute_api:os-aggregates:remove_host to be performed. Jul 16 21:09:02.013559 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:09:02.013559 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:09:02.013559 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:09:02.013559 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:09:02.013559 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:09:02.013559 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:09:02.013559 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 354, in __exit__ Jul 16 21:09:02.013559 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(webob.exc.HTTPForbidden( Jul 16 21:09:02.013559 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Policy doesn't allow os_compute_api:os-aggregates:remove_host to be performed. Jul 16 21:09:02.013559 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:09:02.018073 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-16610027-db7a-4699-a523-263bcf06b34a tempest-AggregatesAdminNegativeTestJSON-1271440314 tempest-AggregatesAdminNegativeTestJSON-1271440314-project] Returning 403 to user: Policy doesn't allow os_compute_api:os-aggregates:remove_host to be performed. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:09:02.019595 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-16610027-db7a-4699-a523-263bcf06b34a tempest-AggregatesAdminNegativeTestJSON-1271440314 tempest-AggregatesAdminNegativeTestJSON-1271440314-project] 10.222.0.16 "POST /compute/v2.1/os-aggregates/13/action" status: 403 len: 121 microversion: 2.1 time: 0.032147 Jul 16 21:09:02.020961 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 109/212] 10.222.0.16 () {60 vars in 1193 bytes} [Sat Jul 16 21:09:01 2022] POST /compute/v2.1/os-aggregates/13/action => generated 121 bytes in 34 msecs (HTTP/1.1 403) 9 headers in 379 bytes (2 switches on core 0) Jul 16 21:09:02.023555 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-0be89395-0977-4a89-b342-efe9180cbf72 tempest-AggregatesAdminTestJSON-1464138310 tempest-AggregatesAdminTestJSON-1464138310-project-admin] HTTP exception thrown: Aggregate 12 could not be found. Jul 16 21:09:02.026222 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-0be89395-0977-4a89-b342-efe9180cbf72 tempest-AggregatesAdminTestJSON-1464138310 tempest-AggregatesAdminTestJSON-1464138310-project-admin] Request method failure captured: Jul 16 21:09:02.026222 ncdv-master-1 devstack@n-api.service[138681]: request: DELETE /compute/v2.1/os-aggregates/12 HTTP/1.1 Jul 16 21:09:02.026222 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:09:02.026222 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:09:02.026222 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:09:02.026222 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 21:09:02.026222 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:09:02.026222 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:09:02.026222 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:09:02.026222 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:09:02.026222 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0yizDAIMDUK3k-KWik016QtqhsacLSD0saiv-JOeGfYuNvqsavIUAkZ3iBuHBSfP7qFl5ZoaybYHJVK0ceO6C1sb4NOOxGeHPq_diZrK7gsGhKDcg10XIFJ7sD9WC1LOTi7KhM7YeGVI_dihO0AmMnpFYju7juyvUZsnsFHzskWIAKU Jul 16 21:09:02.026222 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:09:02.026222 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:09:02.026222 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:09:02.026222 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:09:02.026222 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:09:02.026222 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:09:02.026222 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: 09da0450b1464083990b4b3566862cd6 Jul 16 21:09:02.026222 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-AggregatesAdminTestJSON-1464138310 Jul 16 21:09:02.026222 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,admin,reader Jul 16 21:09:02.026222 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,admin,reader Jul 16 21:09:02.027559 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/09da0450b1464083990b4b3566862cd6"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/09da0450b1464083990b4b3566862cd6"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_09da0450b1464083990b4b3566862cd6"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/09da0450b1464083990b4b3566862cd6"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:09:02.027559 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-AggregatesAdminTestJSON-1464138310 Jul 16 21:09:02.027559 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: 09da0450b1464083990b4b3566862cd6 Jul 16 21:09:02.027559 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-AggregatesAdminTestJSON-1464138310 Jul 16 21:09:02.027559 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-AggregatesAdminTestJSON-1464138310-project-admin Jul 16 21:09:02.027559 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:09:02.027559 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:09:02.027559 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: ceb1a15568df445abebf4220c6894ca9 Jul 16 21:09:02.027559 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-AggregatesAdminTestJSON-1464138310-project-admin Jul 16 21:09:02.027559 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:09:02.027559 ncdv-master-1 devstack@n-api.service[138681]: exception: Aggregate 12 could not be found. Jul 16 21:09:02.027559 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': '12'} Jul 16 21:09:02.027559 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:09:02.028250 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:09:02.028250 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/aggregates.py", line 155, in delete Jul 16 21:09:02.028250 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi self.api.delete_aggregate(context, id) Jul 16 21:09:02.028250 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/exception_wrapper.py", line 71, in wrapped Jul 16 21:09:02.028250 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi _emit_versioned_exception_notification( Jul 16 21:09:02.028250 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 16 21:09:02.028250 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi self.force_reraise() Jul 16 21:09:02.028250 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 16 21:09:02.028250 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise self.value Jul 16 21:09:02.028250 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/exception_wrapper.py", line 63, in wrapped Jul 16 21:09:02.028250 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(self, context, *args, **kw) Jul 16 21:09:02.028250 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 6389, in delete_aggregate Jul 16 21:09:02.028250 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi aggregate = objects.Aggregate.get_by_id(context, aggregate_id) Jul 16 21:09:02.028250 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Jul 16 21:09:02.028250 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Jul 16 21:09:02.028250 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/aggregate.py", line 274, in get_by_id Jul 16 21:09:02.028250 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi db_aggregate = _aggregate_get_from_db(context, aggregate_id) Jul 16 21:09:02.028250 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1010, in wrapper Jul 16 21:09:02.028250 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return fn(*args, **kwargs) Jul 16 21:09:02.029311 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/aggregate.py", line 45, in _aggregate_get_from_db Jul 16 21:09:02.029311 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.AggregateNotFound(aggregate_id=aggregate_id) Jul 16 21:09:02.029311 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.AggregateNotFound: Aggregate 12 could not be found. Jul 16 21:09:02.029311 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:09:02.029311 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:09:02.029311 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:09:02.029311 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:09:02.029311 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:09:02.029311 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:09:02.029311 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:09:02.029311 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:09:02.029311 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:09:02.029311 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:09:02.029311 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/aggregates.py", line 157, in delete Jul 16 21:09:02.029311 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:09:02.029311 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Aggregate 12 could not be found. Jul 16 21:09:02.029311 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:09:02.029311 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:09:02.029311 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:09:02.029311 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:09:02.029311 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:09:02.030897 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:09:02.030897 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:09:02.030897 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:09:02.030897 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Aggregate 12 could not be found. Jul 16 21:09:02.030897 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:09:02.033926 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-0be89395-0977-4a89-b342-efe9180cbf72 tempest-AggregatesAdminTestJSON-1464138310 tempest-AggregatesAdminTestJSON-1464138310-project-admin] Returning 404 to user: Aggregate 12 could not be found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:09:02.035164 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-0be89395-0977-4a89-b342-efe9180cbf72 tempest-AggregatesAdminTestJSON-1464138310 tempest-AggregatesAdminTestJSON-1464138310-project-admin] 10.222.0.16 "DELETE /compute/v2.1/os-aggregates/12" status: 404 len: 78 microversion: 2.1 time: 0.064729 Jul 16 21:09:02.036086 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-a4ccfd59-bc78-40ed-a9c8-c175dfebeac0 tempest-AggregatesAdminNegativeTestJSON-578260511 tempest-AggregatesAdminNegativeTestJSON-578260511-project-admin] Action: 'action', calling method: >, body: {"remove_host": {"host": "nch1-master-1"}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:09:02.041971 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 104/213] 10.222.0.16 () {58 vars in 1154 bytes} [Sat Jul 16 21:09:01 2022] DELETE /compute/v2.1/os-aggregates/12 => generated 78 bytes in 82 msecs (HTTP/1.1 404) 9 headers in 378 bytes (1 switches on core 0) Jul 16 21:09:02.055942 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-8e4e4c75-cfba-4651-abdf-da9723512985 tempest-AggregatesAdminTestJSON-1464138310 tempest-AggregatesAdminTestJSON-1464138310-project-admin] Action: 'create', calling method: >, body: {"aggregate": {"availability_zone": "tempest-test_az-2064921022", "name": "tempest-test_aggregate-1594896182"}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:09:02.070190 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a4ccfd59-bc78-40ed-a9c8-c175dfebeac0 tempest-AggregatesAdminNegativeTestJSON-578260511 tempest-AggregatesAdminNegativeTestJSON-578260511-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:02.072052 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a4ccfd59-bc78-40ed-a9c8-c175dfebeac0 tempest-AggregatesAdminNegativeTestJSON-578260511 tempest-AggregatesAdminNegativeTestJSON-578260511-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:02.082547 ncdv-master-1 devstack@n-api.service[138681]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-aggregates:create": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:09:02.082547 ncdv-master-1 devstack@n-api.service[138681]: warnings.warn(msg) Jul 16 21:09:02.084509 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.objects.aggregate [None req-8e4e4c75-cfba-4651-abdf-da9723512985 tempest-AggregatesAdminTestJSON-1464138310 tempest-AggregatesAdminTestJSON-1464138310-project-admin] Generated uuid 862bd373-91f4-4628-b330-155cc3434e36 for aggregate {{(pid=138681) create /opt/stack/nova/nova/objects/aggregate.py:298}} Jul 16 21:09:02.236551 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-8e4e4c75-cfba-4651-abdf-da9723512985 tempest-AggregatesAdminTestJSON-1464138310 tempest-AggregatesAdminTestJSON-1464138310-project-admin] 10.222.0.16 "POST /compute/v2.1/os-aggregates" status: 200 len: 223 microversion: 2.1 time: 0.192768 Jul 16 21:09:02.244449 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 105/214] 10.222.0.16 () {60 vars in 1164 bytes} [Sat Jul 16 21:09:02 2022] POST /compute/v2.1/os-aggregates => generated 223 bytes in 201 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:09:02.280664 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-2d27a89c-c674-4953-ac88-6eb3daaa2913 tempest-AggregatesAdminTestJSON-1464138310 tempest-AggregatesAdminTestJSON-1464138310-project-admin] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:02.289766 ncdv-master-1 devstack@n-api.service[138681]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-aggregates:delete": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:09:02.289766 ncdv-master-1 devstack@n-api.service[138681]: warnings.warn(msg) Jul 16 21:09:02.304506 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell0:SELECT=15 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:09:02.466582 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-2d27a89c-c674-4953-ac88-6eb3daaa2913 tempest-AggregatesAdminTestJSON-1464138310 tempest-AggregatesAdminTestJSON-1464138310-project-admin] 10.222.0.16 "DELETE /compute/v2.1/os-aggregates/14" status: 200 len: 0 microversion: 2.1 time: 0.218427 Jul 16 21:09:02.466582 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 106/215] 10.222.0.16 () {58 vars in 1154 bytes} [Sat Jul 16 21:09:02 2022] DELETE /compute/v2.1/os-aggregates/14 => generated 0 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 355 bytes (1 switches on core 0) Jul 16 21:09:02.478710 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-e9049562-9709-41a8-9742-124501407e9d tempest-AggregatesAdminTestJSON-1464138310 tempest-AggregatesAdminTestJSON-1464138310-project-admin] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:02.484259 ncdv-master-1 devstack@n-api.service[138681]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-aggregates:show": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:09:02.484259 ncdv-master-1 devstack@n-api.service[138681]: warnings.warn(msg) Jul 16 21:09:02.509395 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-e9049562-9709-41a8-9742-124501407e9d tempest-AggregatesAdminTestJSON-1464138310 tempest-AggregatesAdminTestJSON-1464138310-project-admin] HTTP exception thrown: Aggregate 14 could not be found. Jul 16 21:09:02.515626 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-e9049562-9709-41a8-9742-124501407e9d tempest-AggregatesAdminTestJSON-1464138310 tempest-AggregatesAdminTestJSON-1464138310-project-admin] Request method failure captured: Jul 16 21:09:02.515626 ncdv-master-1 devstack@n-api.service[138681]: request: GET /compute/v2.1/os-aggregates/14 HTTP/1.1 Jul 16 21:09:02.515626 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:09:02.515626 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:09:02.515626 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:09:02.515626 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 21:09:02.515626 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:09:02.515626 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:09:02.515626 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:09:02.515626 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:09:02.515626 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0yizDAIMDUK3k-KWik016QtqhsacLSD0saiv-JOeGfYuNvqsavIUAkZ3iBuHBSfP7qFl5ZoaybYHJVK0ceO6C1sb4NOOxGeHPq_diZrK7gsGhKDcg10XIFJ7sD9WC1LOTi7KhM7YeGVI_dihO0AmMnpFYju7juyvUZsnsFHzskWIAKU Jul 16 21:09:02.515626 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:09:02.515626 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:09:02.515626 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:09:02.515626 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:09:02.515626 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:09:02.515626 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:09:02.515626 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: 09da0450b1464083990b4b3566862cd6 Jul 16 21:09:02.515626 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-AggregatesAdminTestJSON-1464138310 Jul 16 21:09:02.515626 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,admin,reader Jul 16 21:09:02.515626 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,admin,reader Jul 16 21:09:02.517687 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/09da0450b1464083990b4b3566862cd6"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/09da0450b1464083990b4b3566862cd6"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_09da0450b1464083990b4b3566862cd6"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/09da0450b1464083990b4b3566862cd6"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:09:02.517687 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-AggregatesAdminTestJSON-1464138310 Jul 16 21:09:02.517687 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: 09da0450b1464083990b4b3566862cd6 Jul 16 21:09:02.517687 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-AggregatesAdminTestJSON-1464138310 Jul 16 21:09:02.517687 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-AggregatesAdminTestJSON-1464138310-project-admin Jul 16 21:09:02.517687 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:09:02.517687 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:09:02.517687 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: ceb1a15568df445abebf4220c6894ca9 Jul 16 21:09:02.517687 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-AggregatesAdminTestJSON-1464138310-project-admin Jul 16 21:09:02.517687 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:09:02.517687 ncdv-master-1 devstack@n-api.service[138681]: exception: Aggregate 14 could not be found. Jul 16 21:09:02.517687 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': '14'} Jul 16 21:09:02.517687 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:09:02.518676 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:09:02.518676 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/aggregates.py", line 108, in show Jul 16 21:09:02.518676 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi aggregate = self.api.get_aggregate(context, id) Jul 16 21:09:02.518676 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 6336, in get_aggregate Jul 16 21:09:02.518676 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return objects.Aggregate.get_by_id(context, aggregate_id) Jul 16 21:09:02.518676 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Jul 16 21:09:02.518676 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Jul 16 21:09:02.518676 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/aggregate.py", line 274, in get_by_id Jul 16 21:09:02.518676 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi db_aggregate = _aggregate_get_from_db(context, aggregate_id) Jul 16 21:09:02.518676 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1010, in wrapper Jul 16 21:09:02.518676 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return fn(*args, **kwargs) Jul 16 21:09:02.518676 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/aggregate.py", line 45, in _aggregate_get_from_db Jul 16 21:09:02.518676 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.AggregateNotFound(aggregate_id=aggregate_id) Jul 16 21:09:02.518676 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.AggregateNotFound: Aggregate 14 could not be found. Jul 16 21:09:02.518676 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:09:02.518676 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:09:02.518676 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:09:02.518676 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:09:02.518676 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:09:02.519985 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:09:02.519985 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:09:02.519985 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:09:02.519985 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:09:02.519985 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:09:02.519985 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/aggregates.py", line 110, in show Jul 16 21:09:02.519985 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:09:02.519985 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Aggregate 14 could not be found. Jul 16 21:09:02.519985 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:09:02.519985 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:09:02.519985 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:09:02.519985 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:09:02.519985 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:09:02.519985 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:09:02.519985 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:09:02.519985 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:09:02.519985 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Aggregate 14 could not be found. Jul 16 21:09:02.519985 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:09:02.524160 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-e9049562-9709-41a8-9742-124501407e9d tempest-AggregatesAdminTestJSON-1464138310 tempest-AggregatesAdminTestJSON-1464138310-project-admin] Returning 404 to user: Aggregate 14 could not be found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:09:02.537748 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-e9049562-9709-41a8-9742-124501407e9d tempest-AggregatesAdminTestJSON-1464138310 tempest-AggregatesAdminTestJSON-1464138310-project-admin] 10.222.0.16 "GET /compute/v2.1/os-aggregates/14" status: 404 len: 78 microversion: 2.1 time: 0.055486 Jul 16 21:09:02.537748 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 107/216] 10.222.0.16 () {58 vars in 1151 bytes} [Sat Jul 16 21:09:02 2022] GET /compute/v2.1/os-aggregates/14 => generated 78 bytes in 64 msecs (HTTP/1.1 404) 9 headers in 378 bytes (1 switches on core 0) Jul 16 21:09:02.537996 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-a4ccfd59-bc78-40ed-a9c8-c175dfebeac0 tempest-AggregatesAdminNegativeTestJSON-578260511 tempest-AggregatesAdminNegativeTestJSON-578260511-project-admin] 10.222.0.16 "POST /compute/v2.1/os-aggregates/13/action" status: 200 len: 228 microversion: 2.1 time: 0.507856 Jul 16 21:09:02.540538 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 110/217] 10.222.0.16 () {60 vars in 1193 bytes} [Sat Jul 16 21:09:02 2022] POST /compute/v2.1/os-aggregates/13/action => generated 228 bytes in 515 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:09:02.550334 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-e553c5db-fb31-4c75-ad1b-908c38033789 tempest-AggregatesAdminTestJSON-1464138310 tempest-AggregatesAdminTestJSON-1464138310-project-admin] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:02.551866 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-5811cf88-4a77-48c4-9081-fb5e33b0f8c0 tempest-AggregatesAdminNegativeTestJSON-578260511 tempest-AggregatesAdminNegativeTestJSON-578260511-project-admin] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:02.554298 ncdv-master-1 devstack@n-api.service[138681]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-aggregates:delete": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:09:02.554298 ncdv-master-1 devstack@n-api.service[138681]: warnings.warn(msg) Jul 16 21:09:02.575292 ncdv-master-1 devstack@n-api.service[138680]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-aggregates:delete": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:09:02.575292 ncdv-master-1 devstack@n-api.service[138680]: warnings.warn(msg) Jul 16 21:09:02.597801 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-e553c5db-fb31-4c75-ad1b-908c38033789 tempest-AggregatesAdminTestJSON-1464138310 tempest-AggregatesAdminTestJSON-1464138310-project-admin] HTTP exception thrown: Aggregate 14 could not be found. Jul 16 21:09:02.599644 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-e553c5db-fb31-4c75-ad1b-908c38033789 tempest-AggregatesAdminTestJSON-1464138310 tempest-AggregatesAdminTestJSON-1464138310-project-admin] Request method failure captured: Jul 16 21:09:02.599644 ncdv-master-1 devstack@n-api.service[138681]: request: DELETE /compute/v2.1/os-aggregates/14 HTTP/1.1 Jul 16 21:09:02.599644 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:09:02.599644 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:09:02.599644 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:09:02.599644 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 21:09:02.599644 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:09:02.599644 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:09:02.599644 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:09:02.599644 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:09:02.599644 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0yizDAIMDUK3k-KWik016QtqhsacLSD0saiv-JOeGfYuNvqsavIUAkZ3iBuHBSfP7qFl5ZoaybYHJVK0ceO6C1sb4NOOxGeHPq_diZrK7gsGhKDcg10XIFJ7sD9WC1LOTi7KhM7YeGVI_dihO0AmMnpFYju7juyvUZsnsFHzskWIAKU Jul 16 21:09:02.599644 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:09:02.599644 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:09:02.599644 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:09:02.599644 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:09:02.599644 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:09:02.599644 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:09:02.599644 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: 09da0450b1464083990b4b3566862cd6 Jul 16 21:09:02.599644 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-AggregatesAdminTestJSON-1464138310 Jul 16 21:09:02.599644 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,admin,reader Jul 16 21:09:02.599644 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,admin,reader Jul 16 21:09:02.600803 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/09da0450b1464083990b4b3566862cd6"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/09da0450b1464083990b4b3566862cd6"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_09da0450b1464083990b4b3566862cd6"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/09da0450b1464083990b4b3566862cd6"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:09:02.600803 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-AggregatesAdminTestJSON-1464138310 Jul 16 21:09:02.600803 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: 09da0450b1464083990b4b3566862cd6 Jul 16 21:09:02.600803 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-AggregatesAdminTestJSON-1464138310 Jul 16 21:09:02.600803 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-AggregatesAdminTestJSON-1464138310-project-admin Jul 16 21:09:02.600803 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:09:02.600803 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:09:02.600803 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: ceb1a15568df445abebf4220c6894ca9 Jul 16 21:09:02.600803 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-AggregatesAdminTestJSON-1464138310-project-admin Jul 16 21:09:02.600803 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:09:02.600803 ncdv-master-1 devstack@n-api.service[138681]: exception: Aggregate 14 could not be found. Jul 16 21:09:02.600803 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': '14'} Jul 16 21:09:02.600803 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:09:02.601520 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:09:02.601520 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/aggregates.py", line 155, in delete Jul 16 21:09:02.601520 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi self.api.delete_aggregate(context, id) Jul 16 21:09:02.601520 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/exception_wrapper.py", line 71, in wrapped Jul 16 21:09:02.601520 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi _emit_versioned_exception_notification( Jul 16 21:09:02.601520 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 16 21:09:02.601520 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi self.force_reraise() Jul 16 21:09:02.601520 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 16 21:09:02.601520 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise self.value Jul 16 21:09:02.601520 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/exception_wrapper.py", line 63, in wrapped Jul 16 21:09:02.601520 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(self, context, *args, **kw) Jul 16 21:09:02.601520 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 6389, in delete_aggregate Jul 16 21:09:02.601520 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi aggregate = objects.Aggregate.get_by_id(context, aggregate_id) Jul 16 21:09:02.601520 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Jul 16 21:09:02.601520 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Jul 16 21:09:02.601520 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/aggregate.py", line 274, in get_by_id Jul 16 21:09:02.601520 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi db_aggregate = _aggregate_get_from_db(context, aggregate_id) Jul 16 21:09:02.601520 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1010, in wrapper Jul 16 21:09:02.601520 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return fn(*args, **kwargs) Jul 16 21:09:02.602584 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/aggregate.py", line 45, in _aggregate_get_from_db Jul 16 21:09:02.602584 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.AggregateNotFound(aggregate_id=aggregate_id) Jul 16 21:09:02.602584 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.AggregateNotFound: Aggregate 14 could not be found. Jul 16 21:09:02.602584 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:09:02.602584 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:09:02.602584 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:09:02.602584 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:09:02.602584 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:09:02.602584 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:09:02.602584 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:09:02.602584 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:09:02.602584 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:09:02.602584 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:09:02.602584 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/aggregates.py", line 157, in delete Jul 16 21:09:02.602584 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:09:02.602584 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Aggregate 14 could not be found. Jul 16 21:09:02.602584 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:09:02.602584 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:09:02.602584 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:09:02.602584 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:09:02.602584 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:09:02.603800 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:09:02.603800 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:09:02.603800 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:09:02.603800 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Aggregate 14 could not be found. Jul 16 21:09:02.603800 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:09:02.603800 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-e553c5db-fb31-4c75-ad1b-908c38033789 tempest-AggregatesAdminTestJSON-1464138310 tempest-AggregatesAdminTestJSON-1464138310-project-admin] Returning 404 to user: Aggregate 14 could not be found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:09:02.603800 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-e553c5db-fb31-4c75-ad1b-908c38033789 tempest-AggregatesAdminTestJSON-1464138310 tempest-AggregatesAdminTestJSON-1464138310-project-admin] 10.222.0.16 "DELETE /compute/v2.1/os-aggregates/14" status: 404 len: 78 microversion: 2.1 time: 0.059892 Jul 16 21:09:02.603800 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 108/218] 10.222.0.16 () {58 vars in 1154 bytes} [Sat Jul 16 21:09:02 2022] DELETE /compute/v2.1/os-aggregates/14 => generated 78 bytes in 61 msecs (HTTP/1.1 404) 9 headers in 378 bytes (1 switches on core 0) Jul 16 21:09:02.611149 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-1ad21679-d6d8-4d91-9b48-566c1b52ebbe tempest-AggregatesAdminTestJSON-1464138310 tempest-AggregatesAdminTestJSON-1464138310-project-admin] Action: 'create', calling method: >, body: {"aggregate": {"name": "tempest-test_aggregate-244735356"}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:09:02.614463 ncdv-master-1 devstack@n-api.service[138681]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-aggregates:create": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:09:02.614463 ncdv-master-1 devstack@n-api.service[138681]: warnings.warn(msg) Jul 16 21:09:02.616389 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.objects.aggregate [None req-1ad21679-d6d8-4d91-9b48-566c1b52ebbe tempest-AggregatesAdminTestJSON-1464138310 tempest-AggregatesAdminTestJSON-1464138310-project-admin] Generated uuid 3db24aaf-958e-44a5-844e-45a55608423c for aggregate {{(pid=138681) create /opt/stack/nova/nova/objects/aggregate.py:298}} Jul 16 21:09:02.784472 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-1ad21679-d6d8-4d91-9b48-566c1b52ebbe tempest-AggregatesAdminTestJSON-1464138310 tempest-AggregatesAdminTestJSON-1464138310-project-admin] 10.222.0.16 "POST /compute/v2.1/os-aggregates" status: 200 len: 198 microversion: 2.1 time: 0.176946 Jul 16 21:09:02.791771 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 109/219] 10.222.0.16 () {60 vars in 1163 bytes} [Sat Jul 16 21:09:02 2022] POST /compute/v2.1/os-aggregates => generated 198 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:09:02.792929 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-0a98ab91-abff-46f1-9bd1-43239a557a5d tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:02.794437 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-0a98ab91-abff-46f1-9bd1-43239a557a5d tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: 1703f309-9e2e-4c5f-8588-eb4e43f21d91] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:09:02.796897 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-5811cf88-4a77-48c4-9081-fb5e33b0f8c0 tempest-AggregatesAdminNegativeTestJSON-578260511 tempest-AggregatesAdminNegativeTestJSON-578260511-project-admin] 10.222.0.16 "DELETE /compute/v2.1/os-aggregates/13" status: 200 len: 0 microversion: 2.1 time: 0.254549 Jul 16 21:09:02.801846 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 111/220] 10.222.0.16 () {58 vars in 1154 bytes} [Sat Jul 16 21:09:02 2022] DELETE /compute/v2.1/os-aggregates/13 => generated 0 bytes in 261 msecs (HTTP/1.1 200) 9 headers in 355 bytes (1 switches on core 0) Jul 16 21:09:02.813003 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-ad467a2d-665b-4931-a300-43353716042f req-f247f99c-982f-4535-a955-bfb23f784048 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "1703f309-9e2e-4c5f-8588-eb4e43f21d91", "name": "network-vif-plugged", "status": "completed", "tag": "d97c4e64-34a5-4e3d-b81c-2020594e3e0b"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:09:02.831348 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0a98ab91-abff-46f1-9bd1-43239a557a5d tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:02.832019 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0a98ab91-abff-46f1-9bd1-43239a557a5d tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:02.832774 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0a98ab91-abff-46f1-9bd1-43239a557a5d tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:02.833223 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0a98ab91-abff-46f1-9bd1-43239a557a5d tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:02.834674 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-ad467a2d-665b-4931-a300-43353716042f req-f247f99c-982f-4535-a955-bfb23f784048 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:02.835724 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-ad467a2d-665b-4931-a300-43353716042f req-f247f99c-982f-4535-a955-bfb23f784048 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:02.850238 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-ad467a2d-665b-4931-a300-43353716042f req-f247f99c-982f-4535-a955-bfb23f784048 service nova] Creating event network-vif-plugged:d97c4e64-34a5-4e3d-b81c-2020594e3e0b for instance 1703f309-9e2e-4c5f-8588-eb4e43f21d91 on nch2-master-1 Jul 16 21:09:02.893292 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-ad467a2d-665b-4931-a300-43353716042f req-f247f99c-982f-4535-a955-bfb23f784048 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.062514 Jul 16 21:09:02.896486 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 112/221] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:09:02 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:09:02.936813 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-d3b521f9-70a6-44d7-bac4-33c21e9d7fea tempest-AggregatesAdminTestJSON-1464138310 tempest-AggregatesAdminTestJSON-1464138310-project-admin] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:02.952462 ncdv-master-1 devstack@n-api.service[138680]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-aggregates:show": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:09:02.952462 ncdv-master-1 devstack@n-api.service[138680]: warnings.warn(msg) Jul 16 21:09:02.979474 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-d3b521f9-70a6-44d7-bac4-33c21e9d7fea tempest-AggregatesAdminTestJSON-1464138310 tempest-AggregatesAdminTestJSON-1464138310-project-admin] 10.222.0.16 "GET /compute/v2.1/os-aggregates/15" status: 200 len: 227 microversion: 2.1 time: 0.075798 Jul 16 21:09:02.982163 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 113/222] 10.222.0.16 () {58 vars in 1151 bytes} [Sat Jul 16 21:09:02 2022] GET /compute/v2.1/os-aggregates/15 => generated 227 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:09:02.991444 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-61575d19-198e-4dbd-8e65-ad2bb5a47615 tempest-AggregatesAdminNegativeTestJSON-578260511 tempest-AggregatesAdminNegativeTestJSON-578260511-project-admin] Action: 'create', calling method: >, body: {"aggregate": {"name": "tempest-test_aggregate-1073008423"}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:09:02.993257 ncdv-master-1 devstack@n-api.service[138680]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-aggregates:create": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:09:02.993257 ncdv-master-1 devstack@n-api.service[138680]: warnings.warn(msg) Jul 16 21:09:02.995269 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.objects.aggregate [None req-61575d19-198e-4dbd-8e65-ad2bb5a47615 tempest-AggregatesAdminNegativeTestJSON-578260511 tempest-AggregatesAdminNegativeTestJSON-578260511-project-admin] Generated uuid b1bbedbd-b7f8-44df-8b3e-ce6a903cac44 for aggregate {{(pid=138680) create /opt/stack/nova/nova/objects/aggregate.py:298}} Jul 16 21:09:03.126529 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-61575d19-198e-4dbd-8e65-ad2bb5a47615 tempest-AggregatesAdminNegativeTestJSON-578260511 tempest-AggregatesAdminNegativeTestJSON-578260511-project-admin] 10.222.0.16 "POST /compute/v2.1/os-aggregates" status: 200 len: 199 microversion: 2.1 time: 0.140954 Jul 16 21:09:03.129645 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 114/223] 10.222.0.16 () {60 vars in 1163 bytes} [Sat Jul 16 21:09:02 2022] POST /compute/v2.1/os-aggregates => generated 199 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:09:03.155526 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-c211bc6c-9725-44f9-9430-82f97e9ad69d tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:03.160587 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-c211bc6c-9725-44f9-9430-82f97e9ad69d tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] [instance: e506a6e0-fbcd-4e53-a6a0-d31761654c92] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:09:03.189920 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c211bc6c-9725-44f9-9430-82f97e9ad69d tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:03.192645 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c211bc6c-9725-44f9-9430-82f97e9ad69d tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:03.198279 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c211bc6c-9725-44f9-9430-82f97e9ad69d tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:03.199739 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c211bc6c-9725-44f9-9430-82f97e9ad69d tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:04.172372 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-0a98ab91-abff-46f1-9bd1-43239a557a5d tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b68dacc4b8463ba07fae6372131117', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55a19b7a4682457a9132f5fc80110712', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:04.188154 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-0a98ab91-abff-46f1-9bd1-43239a557a5d tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "GET /compute/v2.1/servers/1703f309-9e2e-4c5f-8588-eb4e43f21d91" status: 200 len: 1312 microversion: 2.1 time: 1.397311 Jul 16 21:09:04.189778 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 110/224] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:09:02 2022] GET /compute/v2.1/servers/1703f309-9e2e-4c5f-8588-eb4e43f21d91 => generated 1312 bytes in 1402 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:09:04.200013 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-b01bdafa-8aa9-4ad6-a4dc-c7b7015fb587 tempest-AggregatesAdminTestJSON-1464138310 tempest-AggregatesAdminTestJSON-1464138310-project-admin] Action: 'action', calling method: >, body: {"set_metadata": {"metadata": {"key": "value"}}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:09:04.206013 ncdv-master-1 devstack@n-api.service[138681]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-aggregates:set_metadata": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:09:04.206013 ncdv-master-1 devstack@n-api.service[138681]: warnings.warn(msg) Jul 16 21:09:04.240005 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-c211bc6c-9725-44f9-9430-82f97e9ad69d tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ef9cbce2b2e64665b7f50767342dcb46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0ee02e4c8b448f2b77798ae504129b9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:04.256342 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-c211bc6c-9725-44f9-9430-82f97e9ad69d tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] 10.222.0.16 "GET /compute/v2.1/servers/e506a6e0-fbcd-4e53-a6a0-d31761654c92" status: 200 len: 1390 microversion: 2.1 time: 1.121023 Jul 16 21:09:04.256342 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 115/225] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:09:03 2022] GET /compute/v2.1/servers/e506a6e0-fbcd-4e53-a6a0-d31761654c92 => generated 1390 bytes in 1122 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:09:04.280799 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-6e6a98f6-651e-41db-8bd9-67dd2326f618 tempest-AggregatesAdminNegativeTestJSON-578260511 tempest-AggregatesAdminNegativeTestJSON-578260511-project-admin] Action: 'action', calling method: >, body: {"remove_host": {"host": "nonexist_host"}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:09:04.282805 ncdv-master-1 devstack@n-api.service[138680]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-aggregates:remove_host": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:09:04.282805 ncdv-master-1 devstack@n-api.service[138680]: warnings.warn(msg) Jul 16 21:09:04.319535 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-b01bdafa-8aa9-4ad6-a4dc-c7b7015fb587 tempest-AggregatesAdminTestJSON-1464138310 tempest-AggregatesAdminTestJSON-1464138310-project-admin] 10.222.0.16 "POST /compute/v2.1/os-aggregates/15/action" status: 200 len: 265 microversion: 2.1 time: 0.128322 Jul 16 21:09:04.320576 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 111/226] 10.222.0.16 () {60 vars in 1193 bytes} [Sat Jul 16 21:09:04 2022] POST /compute/v2.1/os-aggregates/15/action => generated 265 bytes in 130 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:09:04.323224 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6e6a98f6-651e-41db-8bd9-67dd2326f618 tempest-AggregatesAdminNegativeTestJSON-578260511 tempest-AggregatesAdminNegativeTestJSON-578260511-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:04.323998 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6e6a98f6-651e-41db-8bd9-67dd2326f618 tempest-AggregatesAdminNegativeTestJSON-578260511 tempest-AggregatesAdminNegativeTestJSON-578260511-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:04.333862 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-f5912015-9a61-42fe-92f5-0ced82c7cfbe tempest-AggregatesAdminTestJSON-1464138310 tempest-AggregatesAdminTestJSON-1464138310-project-admin] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:04.335110 ncdv-master-1 devstack@n-api.service[138681]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-aggregates:show": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:09:04.335110 ncdv-master-1 devstack@n-api.service[138681]: warnings.warn(msg) Jul 16 21:09:04.350853 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6e6a98f6-651e-41db-8bd9-67dd2326f618 tempest-AggregatesAdminNegativeTestJSON-578260511 tempest-AggregatesAdminNegativeTestJSON-578260511-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:04.351357 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6e6a98f6-651e-41db-8bd9-67dd2326f618 tempest-AggregatesAdminNegativeTestJSON-578260511 tempest-AggregatesAdminNegativeTestJSON-578260511-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:04.366678 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-f5912015-9a61-42fe-92f5-0ced82c7cfbe tempest-AggregatesAdminTestJSON-1464138310 tempest-AggregatesAdminTestJSON-1464138310-project-admin] 10.222.0.16 "GET /compute/v2.1/os-aggregates/15" status: 200 len: 241 microversion: 2.1 time: 0.039199 Jul 16 21:09:04.380546 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 112/227] 10.222.0.16 () {58 vars in 1151 bytes} [Sat Jul 16 21:09:04 2022] GET /compute/v2.1/os-aggregates/15 => generated 241 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:09:04.388884 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.compute.aggregates [None req-6e6a98f6-651e-41db-8bd9-67dd2326f618 tempest-AggregatesAdminNegativeTestJSON-578260511 tempest-AggregatesAdminNegativeTestJSON-578260511-project-admin] Failed to remove host nonexist_host from aggregate 16. Error: Compute host nonexist_host could not be found.: nova.exception.ComputeHostNotFound: Compute host nonexist_host could not be found. Jul 16 21:09:04.388884 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-6e6a98f6-651e-41db-8bd9-67dd2326f618 tempest-AggregatesAdminNegativeTestJSON-578260511 tempest-AggregatesAdminNegativeTestJSON-578260511-project-admin] HTTP exception thrown: Cannot remove host nonexist_host in aggregate 16 Jul 16 21:09:04.388884 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-6e6a98f6-651e-41db-8bd9-67dd2326f618 tempest-AggregatesAdminNegativeTestJSON-578260511 tempest-AggregatesAdminNegativeTestJSON-578260511-project-admin] Request method failure captured: Jul 16 21:09:04.388884 ncdv-master-1 devstack@n-api.service[138680]: request: POST /compute/v2.1/os-aggregates/16/action HTTP/1.1 Jul 16 21:09:04.388884 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:09:04.388884 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:09:04.388884 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:09:04.388884 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 42 Jul 16 21:09:04.388884 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:09:04.388884 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:09:04.388884 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:09:04.388884 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:09:04.388884 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0yiz0unej4HpWPJrVBlkA1u9-AWCKDSpHymJUf5DT-hiCYEGZJzMCi_CgXajssbkUJoCwaGruhY3wft9OVrurmz0_pRbvWF03aApzlFrXwt0GhLH6-kKTcDhg1N2k0bLKCTgcUoad_Ma5kdyMmhxUVTCB2mCAuoF3iPjsftSucch2dg Jul 16 21:09:04.388884 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:09:04.388884 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:09:04.388884 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:09:04.388884 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:09:04.388884 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:09:04.388884 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:09:04.388884 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: b6be58dcb13b4cec8f0bfc58f0080173 Jul 16 21:09:04.388884 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-AggregatesAdminNegativeTestJSON-578260511 Jul 16 21:09:04.388884 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,admin,reader Jul 16 21:09:04.388884 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,admin,reader Jul 16 21:09:04.390480 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/b6be58dcb13b4cec8f0bfc58f0080173"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/b6be58dcb13b4cec8f0bfc58f0080173"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_b6be58dcb13b4cec8f0bfc58f0080173"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/b6be58dcb13b4cec8f0bfc58f0080173"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:09:04.390480 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-AggregatesAdminNegativeTestJSON-578260511 Jul 16 21:09:04.390480 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: b6be58dcb13b4cec8f0bfc58f0080173 Jul 16 21:09:04.390480 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-AggregatesAdminNegativeTestJSON-578260511 Jul 16 21:09:04.390480 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-AggregatesAdminNegativeTestJSON-578260511-project-admin Jul 16 21:09:04.390480 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:09:04.390480 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:09:04.390480 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: cadb63bb5d6b4d7597987780f7d9ef21 Jul 16 21:09:04.390480 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-AggregatesAdminNegativeTestJSON-578260511-project-admin Jul 16 21:09:04.390480 ncdv-master-1 devstack@n-api.service[138680]: Jul 16 21:09:04.390480 ncdv-master-1 devstack@n-api.service[138680]: {"remove_host": {"host": "nonexist_host"}} Jul 16 21:09:04.390480 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 21:09:04.390480 ncdv-master-1 devstack@n-api.service[138680]: exception: Cannot remove host nonexist_host in aggregate 16 Jul 16 21:09:04.391729 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': '16', 'body': {'remove_host': {'host': 'nonexist_host'}}} Jul 16 21:09:04.391729 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:09:04.391729 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:09:04.391729 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5868, in _get_service_in_cell_by_host Jul 16 21:09:04.391729 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi mapping = objects.HostMapping.get_by_host(context, host_name) Jul 16 21:09:04.391729 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Jul 16 21:09:04.391729 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Jul 16 21:09:04.391729 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/host_mapping.py", line 105, in get_by_host Jul 16 21:09:04.391729 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi db_mapping = cls._get_by_host_from_db(context, host) Jul 16 21:09:04.391729 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1010, in wrapper Jul 16 21:09:04.391729 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return fn(*args, **kwargs) Jul 16 21:09:04.391729 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/host_mapping.py", line 100, in _get_by_host_from_db Jul 16 21:09:04.391729 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.HostMappingNotFound(name=host) Jul 16 21:09:04.391729 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.HostMappingNotFound: Host 'nonexist_host' is not mapped to any cell Jul 16 21:09:04.391729 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:09:04.391729 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:09:04.391729 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:09:04.391729 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:09:04.391729 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5874, in _get_service_in_cell_by_host Jul 16 21:09:04.391729 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi service = _find_service_in_cell(context, service_host=host_name) Jul 16 21:09:04.393058 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5936, in _find_service_in_cell Jul 16 21:09:04.393058 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.NotFound() Jul 16 21:09:04.393058 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.NotFound: Resource could not be found. Jul 16 21:09:04.393058 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:09:04.393058 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:09:04.393058 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:09:04.393058 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:09:04.393058 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/aggregates.py", line 209, in _remove_host Jul 16 21:09:04.393058 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi aggregate = self.api.remove_host_from_aggregate(context, id, host) Jul 16 21:09:04.393058 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/exception_wrapper.py", line 71, in wrapped Jul 16 21:09:04.393058 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi _emit_versioned_exception_notification( Jul 16 21:09:04.393058 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 16 21:09:04.393058 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi self.force_reraise() Jul 16 21:09:04.393058 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 16 21:09:04.393058 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise self.value Jul 16 21:09:04.393058 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/exception_wrapper.py", line 63, in wrapped Jul 16 21:09:04.393058 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(self, context, *args, **kw) Jul 16 21:09:04.393058 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 6555, in remove_host_from_aggregate Jul 16 21:09:04.393058 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi _get_service_in_cell_by_host(context, host_name) Jul 16 21:09:04.393058 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5876, in _get_service_in_cell_by_host Jul 16 21:09:04.394323 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.ComputeHostNotFound(host=host_name) Jul 16 21:09:04.394323 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.ComputeHostNotFound: Compute host nonexist_host could not be found. Jul 16 21:09:04.394323 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:09:04.394323 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:09:04.394323 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:09:04.394323 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:09:04.394323 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:09:04.394323 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:09:04.394323 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:09:04.394323 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:09:04.394323 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:09:04.394323 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:09:04.394323 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 16 21:09:04.394323 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 16 21:09:04.394323 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/aggregates.py", line 217, in _remove_host Jul 16 21:09:04.394323 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=msg) Jul 16 21:09:04.394323 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Cannot remove host nonexist_host in aggregate 16 Jul 16 21:09:04.394323 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:09:04.394323 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:09:04.394323 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:09:04.394323 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:09:04.397680 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:09:04.397680 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:09:04.397680 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:09:04.397680 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:09:04.397680 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Cannot remove host nonexist_host in aggregate 16 Jul 16 21:09:04.397680 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:09:04.397680 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-6e6a98f6-651e-41db-8bd9-67dd2326f618 tempest-AggregatesAdminNegativeTestJSON-578260511 tempest-AggregatesAdminNegativeTestJSON-578260511-project-admin] Returning 404 to user: Cannot remove host nonexist_host in aggregate 16 {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:09:04.397680 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-6e6a98f6-651e-41db-8bd9-67dd2326f618 tempest-AggregatesAdminNegativeTestJSON-578260511 tempest-AggregatesAdminNegativeTestJSON-578260511-project-admin] 10.222.0.16 "POST /compute/v2.1/os-aggregates/16/action" status: 404 len: 94 microversion: 2.1 time: 0.124001 Jul 16 21:09:04.397680 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 116/228] 10.222.0.16 () {60 vars in 1193 bytes} [Sat Jul 16 21:09:04 2022] POST /compute/v2.1/os-aggregates/16/action => generated 94 bytes in 143 msecs (HTTP/1.1 404) 9 headers in 378 bytes (1 switches on core 0) Jul 16 21:09:04.400477 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-2586ca13-fae6-4ef3-b8b1-45e6406f83be tempest-AggregatesAdminTestJSON-1464138310 tempest-AggregatesAdminTestJSON-1464138310-project-admin] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:04.403166 ncdv-master-1 devstack@n-api.service[138681]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-aggregates:delete": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:09:04.403166 ncdv-master-1 devstack@n-api.service[138681]: warnings.warn(msg) Jul 16 21:09:04.408210 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-9a255324-f15d-4b7d-bc3c-eef0cfb29918 tempest-AggregatesAdminNegativeTestJSON-578260511 tempest-AggregatesAdminNegativeTestJSON-578260511-project-admin] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:04.410752 ncdv-master-1 devstack@n-api.service[138680]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-aggregates:delete": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:09:04.410752 ncdv-master-1 devstack@n-api.service[138680]: warnings.warn(msg) Jul 16 21:09:04.548251 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-2586ca13-fae6-4ef3-b8b1-45e6406f83be tempest-AggregatesAdminTestJSON-1464138310 tempest-AggregatesAdminTestJSON-1464138310-project-admin] 10.222.0.16 "DELETE /compute/v2.1/os-aggregates/15" status: 200 len: 0 microversion: 2.1 time: 0.164608 Jul 16 21:09:04.558998 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 113/229] 10.222.0.16 () {58 vars in 1154 bytes} [Sat Jul 16 21:09:04 2022] DELETE /compute/v2.1/os-aggregates/15 => generated 0 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 355 bytes (1 switches on core 0) Jul 16 21:09:04.561738 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-0368a2f3-928e-407f-acd5-4954cd9be1fe tempest-AggregatesAdminTestJSON-1464138310 tempest-AggregatesAdminTestJSON-1464138310-project-admin] Action: 'create', calling method: >, body: {"aggregate": {"name": "tempest-test_aggregate-281160594", "availability_zone": "tempest-test_az-1990832677"}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:09:04.563342 ncdv-master-1 devstack@n-api.service[138681]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-aggregates:create": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:09:04.563342 ncdv-master-1 devstack@n-api.service[138681]: warnings.warn(msg) Jul 16 21:09:04.564338 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.objects.aggregate [None req-0368a2f3-928e-407f-acd5-4954cd9be1fe tempest-AggregatesAdminTestJSON-1464138310 tempest-AggregatesAdminTestJSON-1464138310-project-admin] Generated uuid 5f109389-da02-4d78-83c1-53790445e67b for aggregate {{(pid=138681) create /opt/stack/nova/nova/objects/aggregate.py:298}} Jul 16 21:09:04.604608 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-9a255324-f15d-4b7d-bc3c-eef0cfb29918 tempest-AggregatesAdminNegativeTestJSON-578260511 tempest-AggregatesAdminNegativeTestJSON-578260511-project-admin] 10.222.0.16 "DELETE /compute/v2.1/os-aggregates/16" status: 200 len: 0 microversion: 2.1 time: 0.203521 Jul 16 21:09:04.606506 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 117/230] 10.222.0.16 () {58 vars in 1154 bytes} [Sat Jul 16 21:09:04 2022] DELETE /compute/v2.1/os-aggregates/16 => generated 0 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 355 bytes (1 switches on core 0) Jul 16 21:09:04.728210 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-0368a2f3-928e-407f-acd5-4954cd9be1fe tempest-AggregatesAdminTestJSON-1464138310 tempest-AggregatesAdminTestJSON-1464138310-project-admin] 10.222.0.16 "POST /compute/v2.1/os-aggregates" status: 200 len: 222 microversion: 2.1 time: 0.170553 Jul 16 21:09:04.730314 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 114/231] 10.222.0.16 () {60 vars in 1164 bytes} [Sat Jul 16 21:09:04 2022] POST /compute/v2.1/os-aggregates => generated 222 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:09:04.741439 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-e824f52c-b5de-42b6-b623-afc60010eb64 tempest-AggregatesAdminTestJSON-1464138310 tempest-AggregatesAdminTestJSON-1464138310-project-admin] Action: 'update', calling method: >, body: {"aggregate": {"name": "tempest-test_aggregate-281160594_new", "availability_zone": "tempest-test_az-1990832677_new"}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:09:04.879515 ncdv-master-1 devstack@n-api.service[138680]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-aggregates:update": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:09:04.879515 ncdv-master-1 devstack@n-api.service[138680]: warnings.warn(msg) Jul 16 21:09:05.021255 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e824f52c-b5de-42b6-b623-afc60010eb64 tempest-AggregatesAdminTestJSON-1464138310 tempest-AggregatesAdminTestJSON-1464138310-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:05.021953 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e824f52c-b5de-42b6-b623-afc60010eb64 tempest-AggregatesAdminTestJSON-1464138310 tempest-AggregatesAdminTestJSON-1464138310-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:05.172501 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-e824f52c-b5de-42b6-b623-afc60010eb64 tempest-AggregatesAdminTestJSON-1464138310 tempest-AggregatesAdminTestJSON-1464138310-project-admin] 10.222.0.16 "PUT /compute/v2.1/os-aggregates/17" status: 200 len: 336 microversion: 2.1 time: 0.435305 Jul 16 21:09:05.176045 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 118/232] 10.222.0.16 () {60 vars in 1172 bytes} [Sat Jul 16 21:09:04 2022] PUT /compute/v2.1/os-aggregates/17 => generated 336 bytes in 440 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:09:05.190859 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-3c84019e-cd81-4cc9-a1b2-c7269ef7b8ce tempest-AggregatesAdminTestJSON-1464138310 tempest-AggregatesAdminTestJSON-1464138310-project-admin] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:05.191950 ncdv-master-1 devstack@n-api.service[138681]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-aggregates:index": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:09:05.191950 ncdv-master-1 devstack@n-api.service[138681]: warnings.warn(msg) Jul 16 21:09:05.213521 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-48887d13-17c5-481f-bcfc-a7a786c47c9f tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:05.214143 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-48887d13-17c5-481f-bcfc-a7a786c47c9f tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: 1703f309-9e2e-4c5f-8588-eb4e43f21d91] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:09:05.278188 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-3c84019e-cd81-4cc9-a1b2-c7269ef7b8ce tempest-AggregatesAdminTestJSON-1464138310 tempest-AggregatesAdminTestJSON-1464138310-project-admin] 10.222.0.16 "GET /compute/v2.1/os-aggregates" status: 200 len: 339 microversion: 2.1 time: 0.082874 Jul 16 21:09:05.283836 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 115/233] 10.222.0.16 () {58 vars in 1142 bytes} [Sat Jul 16 21:09:05 2022] GET /compute/v2.1/os-aggregates => generated 339 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:09:05.295171 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-48887d13-17c5-481f-bcfc-a7a786c47c9f tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:05.301309 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-48887d13-17c5-481f-bcfc-a7a786c47c9f tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:05.303215 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-48887d13-17c5-481f-bcfc-a7a786c47c9f tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:05.303677 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-48887d13-17c5-481f-bcfc-a7a786c47c9f tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:05.312422 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-e658df2b-974a-4af0-ac6c-88f116477108 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:05.317867 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-e658df2b-974a-4af0-ac6c-88f116477108 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] [instance: e506a6e0-fbcd-4e53-a6a0-d31761654c92] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:09:05.344193 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e658df2b-974a-4af0-ac6c-88f116477108 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:05.345489 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e658df2b-974a-4af0-ac6c-88f116477108 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:05.352940 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e658df2b-974a-4af0-ac6c-88f116477108 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:05.352940 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e658df2b-974a-4af0-ac6c-88f116477108 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:05.901222 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-e658df2b-974a-4af0-ac6c-88f116477108 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ef9cbce2b2e64665b7f50767342dcb46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0ee02e4c8b448f2b77798ae504129b9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:05.911715 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-48887d13-17c5-481f-bcfc-a7a786c47c9f tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b68dacc4b8463ba07fae6372131117', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55a19b7a4682457a9132f5fc80110712', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:05.919354 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-e658df2b-974a-4af0-ac6c-88f116477108 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] 10.222.0.16 "GET /compute/v2.1/servers/e506a6e0-fbcd-4e53-a6a0-d31761654c92" status: 200 len: 1390 microversion: 2.1 time: 0.633692 Jul 16 21:09:05.919949 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 116/234] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:09:05 2022] GET /compute/v2.1/servers/e506a6e0-fbcd-4e53-a6a0-d31761654c92 => generated 1390 bytes in 636 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:09:05.928053 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-440359bb-2a05-4e53-8e8f-4c0e718b58ae tempest-AggregatesAdminTestJSON-1464138310 tempest-AggregatesAdminTestJSON-1464138310-project-admin] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:05.928821 ncdv-master-1 devstack@n-api.service[138681]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-aggregates:delete": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:09:05.928821 ncdv-master-1 devstack@n-api.service[138681]: warnings.warn(msg) Jul 16 21:09:05.933241 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-48887d13-17c5-481f-bcfc-a7a786c47c9f tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "GET /compute/v2.1/servers/1703f309-9e2e-4c5f-8588-eb4e43f21d91" status: 200 len: 1312 microversion: 2.1 time: 0.731016 Jul 16 21:09:05.933958 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 119/235] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:09:05 2022] GET /compute/v2.1/servers/1703f309-9e2e-4c5f-8588-eb4e43f21d91 => generated 1312 bytes in 732 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:09:06.091205 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-440359bb-2a05-4e53-8e8f-4c0e718b58ae tempest-AggregatesAdminTestJSON-1464138310 tempest-AggregatesAdminTestJSON-1464138310-project-admin] 10.222.0.16 "DELETE /compute/v2.1/os-aggregates/17" status: 200 len: 0 microversion: 2.1 time: 0.170111 Jul 16 21:09:06.092401 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 117/236] 10.222.0.16 () {58 vars in 1154 bytes} [Sat Jul 16 21:09:05 2022] DELETE /compute/v2.1/os-aggregates/17 => generated 0 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 355 bytes (1 switches on core 0) Jul 16 21:09:06.103376 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-c07e19ab-5189-422d-b388-010768b71da4 tempest-AggregatesAdminTestJSON-1464138310 tempest-AggregatesAdminTestJSON-1464138310-project-admin] Action: 'create', calling method: >, body: {"aggregate": {"name": "tempest-test_aggregate-97038034"}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:09:06.104741 ncdv-master-1 devstack@n-api.service[138680]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-aggregates:create": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:09:06.104741 ncdv-master-1 devstack@n-api.service[138680]: warnings.warn(msg) Jul 16 21:09:06.105790 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.objects.aggregate [None req-c07e19ab-5189-422d-b388-010768b71da4 tempest-AggregatesAdminTestJSON-1464138310 tempest-AggregatesAdminTestJSON-1464138310-project-admin] Generated uuid 9da694d4-817a-44b3-933a-14ccaa27fe90 for aggregate {{(pid=138680) create /opt/stack/nova/nova/objects/aggregate.py:298}} Jul 16 21:09:06.275044 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-c07e19ab-5189-422d-b388-010768b71da4 tempest-AggregatesAdminTestJSON-1464138310 tempest-AggregatesAdminTestJSON-1464138310-project-admin] 10.222.0.16 "POST /compute/v2.1/os-aggregates" status: 200 len: 197 microversion: 2.1 time: 0.176134 Jul 16 21:09:06.275727 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 120/237] 10.222.0.16 () {60 vars in 1163 bytes} [Sat Jul 16 21:09:06 2022] POST /compute/v2.1/os-aggregates => generated 197 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:09:06.293908 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-425280fd-7485-4a80-b50a-76ea1e2fe201 tempest-AggregatesAdminTestJSON-1464138310 tempest-AggregatesAdminTestJSON-1464138310-project-admin] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:06.295303 ncdv-master-1 devstack@n-api.service[138681]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-aggregates:index": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:09:06.295303 ncdv-master-1 devstack@n-api.service[138681]: warnings.warn(msg) Jul 16 21:09:06.307212 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_api:SELECT=80,nova_api:INSERT=9,nova_api:DELETE=11 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:09:06.338930 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-425280fd-7485-4a80-b50a-76ea1e2fe201 tempest-AggregatesAdminTestJSON-1464138310 tempest-AggregatesAdminTestJSON-1464138310-project-admin] 10.222.0.16 "GET /compute/v2.1/os-aggregates" status: 200 len: 229 microversion: 2.1 time: 0.052507 Jul 16 21:09:06.344126 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 118/238] 10.222.0.16 () {58 vars in 1142 bytes} [Sat Jul 16 21:09:06 2022] GET /compute/v2.1/os-aggregates => generated 229 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:09:06.356841 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-156a8910-5447-4764-91fe-a0f6f3ed099e tempest-AggregatesAdminTestJSON-1464138310 tempest-AggregatesAdminTestJSON-1464138310-project-admin] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:06.358377 ncdv-master-1 devstack@n-api.service[138680]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-aggregates:delete": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:09:06.358377 ncdv-master-1 devstack@n-api.service[138680]: warnings.warn(msg) Jul 16 21:09:06.394534 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell0:SELECT=8 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:09:06.510565 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-156a8910-5447-4764-91fe-a0f6f3ed099e tempest-AggregatesAdminTestJSON-1464138310 tempest-AggregatesAdminTestJSON-1464138310-project-admin] 10.222.0.16 "DELETE /compute/v2.1/os-aggregates/18" status: 200 len: 0 microversion: 2.1 time: 0.157845 Jul 16 21:09:06.512197 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 121/239] 10.222.0.16 () {58 vars in 1154 bytes} [Sat Jul 16 21:09:06 2022] DELETE /compute/v2.1/os-aggregates/18 => generated 0 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 355 bytes (1 switches on core 0) Jul 16 21:09:06.527398 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-a55e5391-b7fa-44c9-aba1-97c62700acd5 tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:06.528067 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-a55e5391-b7fa-44c9-aba1-97c62700acd5 tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] [instance: 7895aae7-b50f-48cf-a3e1-5b33dc6e16a3] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:09:06.559137 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a55e5391-b7fa-44c9-aba1-97c62700acd5 tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:06.559763 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a55e5391-b7fa-44c9-aba1-97c62700acd5 tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:06.561370 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a55e5391-b7fa-44c9-aba1-97c62700acd5 tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:06.568298 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a55e5391-b7fa-44c9-aba1-97c62700acd5 tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:06.640293 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-a55e5391-b7fa-44c9-aba1-97c62700acd5 tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] HTTP exception thrown: Instance 7895aae7-b50f-48cf-a3e1-5b33dc6e16a3 could not be found. Jul 16 21:09:06.663928 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-a55e5391-b7fa-44c9-aba1-97c62700acd5 tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] Request method failure captured: Jul 16 21:09:06.663928 ncdv-master-1 devstack@n-api.service[138681]: request: DELETE /compute/v2.1/servers/7895aae7-b50f-48cf-a3e1-5b33dc6e16a3 HTTP/1.1 Jul 16 21:09:06.663928 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:09:06.663928 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:09:06.663928 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:09:06.663928 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 21:09:06.663928 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:09:06.663928 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:09:06.663928 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:09:06.663928 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:09:06.663928 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0yiwEDc6jCrIXaeq91y19seHWVuAydqowazCbU3Ptgxp4XZeouI5kZu6lb6qVjBtzG4PXc_eGUoHi4NASCxzS51pZnxcShhc_TTetcJdupN3R2A3t1jzrc5iKUJ5B24npTLiwYERA2Q_WmIZ5KSaFY0acD8RGBvhEitiUAFojMftS2o Jul 16 21:09:06.663928 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:09:06.663928 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:09:06.663928 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:09:06.663928 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:09:06.663928 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:09:06.663928 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:09:06.663928 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: e7c343c974bd4ec0a6a765f818b895be Jul 16 21:09:06.663928 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-AggregatesAdminTestJSON-412945753 Jul 16 21:09:06.663928 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:09:06.663928 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:09:06.666893 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/e7c343c974bd4ec0a6a765f818b895be"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/e7c343c974bd4ec0a6a765f818b895be"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_e7c343c974bd4ec0a6a765f818b895be"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/e7c343c974bd4ec0a6a765f818b895be"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:09:06.666893 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-AggregatesAdminTestJSON-412945753 Jul 16 21:09:06.666893 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: e7c343c974bd4ec0a6a765f818b895be Jul 16 21:09:06.666893 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-AggregatesAdminTestJSON-412945753 Jul 16 21:09:06.666893 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-AggregatesAdminTestJSON-412945753-project Jul 16 21:09:06.666893 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:09:06.666893 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:09:06.666893 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: db6c21ea09c54ea492d806f42d2f69a7 Jul 16 21:09:06.666893 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-AggregatesAdminTestJSON-412945753-project Jul 16 21:09:06.666893 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:09:06.666893 ncdv-master-1 devstack@n-api.service[138681]: exception: Instance 7895aae7-b50f-48cf-a3e1-5b33dc6e16a3 could not be found. Jul 16 21:09:06.666893 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': '7895aae7-b50f-48cf-a3e1-5b33dc6e16a3'} Jul 16 21:09:06.667584 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:09:06.667584 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:09:06.667584 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:09:06.667584 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:09:06.667584 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:09:06.667584 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:09:06.667584 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:09:06.667584 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:09:06.667584 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:09:06.667584 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:09:06.667584 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 7895aae7-b50f-48cf-a3e1-5b33dc6e16a3 could not be found. Jul 16 21:09:06.667584 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:09:06.667584 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:09:06.667584 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:09:06.667584 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:09:06.667584 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:09:06.667584 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:09:06.667584 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:09:06.667584 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:09:06.667584 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:09:06.668610 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:09:06.668610 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Jul 16 21:09:06.668610 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Jul 16 21:09:06.668610 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Jul 16 21:09:06.668610 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Jul 16 21:09:06.668610 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:09:06.668610 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:09:06.668610 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:09:06.668610 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:09:06.668610 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 7895aae7-b50f-48cf-a3e1-5b33dc6e16a3 could not be found. Jul 16 21:09:06.668610 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:09:06.668610 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:09:06.668610 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:09:06.668610 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:09:06.668610 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:09:06.668610 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:09:06.668610 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:09:06.668610 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:09:06.668610 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 7895aae7-b50f-48cf-a3e1-5b33dc6e16a3 could not be found. Jul 16 21:09:06.668610 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:09:06.670226 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-a55e5391-b7fa-44c9-aba1-97c62700acd5 tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] Returning 404 to user: Instance 7895aae7-b50f-48cf-a3e1-5b33dc6e16a3 could not be found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:09:06.670226 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-a55e5391-b7fa-44c9-aba1-97c62700acd5 tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] 10.222.0.16 "DELETE /compute/v2.1/servers/7895aae7-b50f-48cf-a3e1-5b33dc6e16a3" status: 404 len: 111 microversion: 2.1 time: 0.147658 Jul 16 21:09:06.670226 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 119/240] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:09:06 2022] DELETE /compute/v2.1/servers/7895aae7-b50f-48cf-a3e1-5b33dc6e16a3 => generated 111 bytes in 152 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:09:06.676777 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-94308888-80a5-41ed-8744-250916de0e12 tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:06.677681 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-94308888-80a5-41ed-8744-250916de0e12 tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] [instance: 7895aae7-b50f-48cf-a3e1-5b33dc6e16a3] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:09:06.690359 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-94308888-80a5-41ed-8744-250916de0e12 tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:06.691733 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-94308888-80a5-41ed-8744-250916de0e12 tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:06.695698 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-94308888-80a5-41ed-8744-250916de0e12 tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:06.703260 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-94308888-80a5-41ed-8744-250916de0e12 tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:06.735944 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-94308888-80a5-41ed-8744-250916de0e12 tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] HTTP exception thrown: Instance 7895aae7-b50f-48cf-a3e1-5b33dc6e16a3 could not be found. Jul 16 21:09:06.740267 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-94308888-80a5-41ed-8744-250916de0e12 tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] Request method failure captured: Jul 16 21:09:06.740267 ncdv-master-1 devstack@n-api.service[138680]: request: GET /compute/v2.1/servers/7895aae7-b50f-48cf-a3e1-5b33dc6e16a3 HTTP/1.1 Jul 16 21:09:06.740267 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:09:06.740267 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:09:06.740267 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:09:06.740267 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 0 Jul 16 21:09:06.740267 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:09:06.740267 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:09:06.740267 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:09:06.740267 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:09:06.740267 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0yiwEDc6jCrIXaeq91y19seHWVuAydqowazCbU3Ptgxp4XZeouI5kZu6lb6qVjBtzG4PXc_eGUoHi4NASCxzS51pZnxcShhc_TTetcJdupN3R2A3t1jzrc5iKUJ5B24npTLiwYERA2Q_WmIZ5KSaFY0acD8RGBvhEitiUAFojMftS2o Jul 16 21:09:06.740267 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:09:06.740267 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:09:06.740267 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:09:06.740267 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:09:06.740267 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:09:06.740267 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:09:06.740267 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: e7c343c974bd4ec0a6a765f818b895be Jul 16 21:09:06.740267 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-AggregatesAdminTestJSON-412945753 Jul 16 21:09:06.740267 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:09:06.740267 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:09:06.741839 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/e7c343c974bd4ec0a6a765f818b895be"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/e7c343c974bd4ec0a6a765f818b895be"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_e7c343c974bd4ec0a6a765f818b895be"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/e7c343c974bd4ec0a6a765f818b895be"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:09:06.741839 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-AggregatesAdminTestJSON-412945753 Jul 16 21:09:06.741839 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: e7c343c974bd4ec0a6a765f818b895be Jul 16 21:09:06.741839 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-AggregatesAdminTestJSON-412945753 Jul 16 21:09:06.741839 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-AggregatesAdminTestJSON-412945753-project Jul 16 21:09:06.741839 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:09:06.741839 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:09:06.741839 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: db6c21ea09c54ea492d806f42d2f69a7 Jul 16 21:09:06.741839 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-AggregatesAdminTestJSON-412945753-project Jul 16 21:09:06.741839 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 21:09:06.741839 ncdv-master-1 devstack@n-api.service[138680]: exception: Instance 7895aae7-b50f-48cf-a3e1-5b33dc6e16a3 could not be found. Jul 16 21:09:06.741839 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': '7895aae7-b50f-48cf-a3e1-5b33dc6e16a3'} Jul 16 21:09:06.742650 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:09:06.742650 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:09:06.742650 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:09:06.742650 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:09:06.742650 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:09:06.742650 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:09:06.742650 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:09:06.742650 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:09:06.742650 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:09:06.742650 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:09:06.742650 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 7895aae7-b50f-48cf-a3e1-5b33dc6e16a3 could not be found. Jul 16 21:09:06.742650 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:09:06.742650 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:09:06.742650 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:09:06.742650 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:09:06.742650 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:09:06.742650 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:09:06.742650 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:09:06.742650 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:09:06.742650 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:09:06.744645 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:09:06.744645 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:09:06.744645 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:09:06.744645 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:09:06.744645 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:09:06.744645 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:09:06.744645 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:09:06.744645 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 7895aae7-b50f-48cf-a3e1-5b33dc6e16a3 could not be found. Jul 16 21:09:06.744645 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:09:06.744645 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:09:06.744645 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:09:06.744645 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:09:06.744645 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:09:06.744645 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:09:06.744645 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:09:06.744645 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:09:06.744645 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 7895aae7-b50f-48cf-a3e1-5b33dc6e16a3 could not be found. Jul 16 21:09:06.744645 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:09:06.749349 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-94308888-80a5-41ed-8744-250916de0e12 tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] Returning 404 to user: Instance 7895aae7-b50f-48cf-a3e1-5b33dc6e16a3 could not be found. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:09:06.749349 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-94308888-80a5-41ed-8744-250916de0e12 tempest-AggregatesAdminTestJSON-412945753 tempest-AggregatesAdminTestJSON-412945753-project] 10.222.0.16 "GET /compute/v2.1/servers/7895aae7-b50f-48cf-a3e1-5b33dc6e16a3" status: 404 len: 111 microversion: 2.1 time: 0.073232 Jul 16 21:09:06.749349 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 122/241] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:09:06 2022] GET /compute/v2.1/servers/7895aae7-b50f-48cf-a3e1-5b33dc6e16a3 => generated 111 bytes in 76 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:09:06.937498 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-85a8105f-2dfa-4f3a-afee-e067971dd48f tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:06.938402 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-85a8105f-2dfa-4f3a-afee-e067971dd48f tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] [instance: e506a6e0-fbcd-4e53-a6a0-d31761654c92] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:09:06.950103 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-85a8105f-2dfa-4f3a-afee-e067971dd48f tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:06.951603 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-85a8105f-2dfa-4f3a-afee-e067971dd48f tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:06.955239 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-4a841665-9702-45bd-a927-1c6fbfa72778 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:06.955857 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-4a841665-9702-45bd-a927-1c6fbfa72778 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: 1703f309-9e2e-4c5f-8588-eb4e43f21d91] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:09:06.957120 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-85a8105f-2dfa-4f3a-afee-e067971dd48f tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:06.958495 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-85a8105f-2dfa-4f3a-afee-e067971dd48f tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:06.962456 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4a841665-9702-45bd-a927-1c6fbfa72778 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:06.971107 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4a841665-9702-45bd-a927-1c6fbfa72778 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:06.976333 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4a841665-9702-45bd-a927-1c6fbfa72778 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:06.976864 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4a841665-9702-45bd-a927-1c6fbfa72778 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:07.614143 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-4a841665-9702-45bd-a927-1c6fbfa72778 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b68dacc4b8463ba07fae6372131117', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55a19b7a4682457a9132f5fc80110712', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:07.639601 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-4a841665-9702-45bd-a927-1c6fbfa72778 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "GET /compute/v2.1/servers/1703f309-9e2e-4c5f-8588-eb4e43f21d91" status: 200 len: 1461 microversion: 2.1 time: 0.689724 Jul 16 21:09:07.654718 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 123/242] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:09:06 2022] GET /compute/v2.1/servers/1703f309-9e2e-4c5f-8588-eb4e43f21d91 => generated 1461 bytes in 705 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:09:07.661293 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-f9bbce6a-42e4-4402-97f9-1762f05038a1 req-2b3afdf7-0459-47a2-95a6-792db7aac6e4 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "e506a6e0-fbcd-4e53-a6a0-d31761654c92", "tag": "fa66e5de-7c5f-4e0e-adc5-96ea3b90bd37"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:09:07.665838 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-85a8105f-2dfa-4f3a-afee-e067971dd48f tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ef9cbce2b2e64665b7f50767342dcb46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0ee02e4c8b448f2b77798ae504129b9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:07.680246 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-f9bbce6a-42e4-4402-97f9-1762f05038a1 req-2b3afdf7-0459-47a2-95a6-792db7aac6e4 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:07.681112 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-f9bbce6a-42e4-4402-97f9-1762f05038a1 req-2b3afdf7-0459-47a2-95a6-792db7aac6e4 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:07.682451 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-85a8105f-2dfa-4f3a-afee-e067971dd48f tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] 10.222.0.16 "GET /compute/v2.1/servers/e506a6e0-fbcd-4e53-a6a0-d31761654c92" status: 200 len: 1390 microversion: 2.1 time: 0.749193 Jul 16 21:09:07.685369 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 120/243] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:09:06 2022] GET /compute/v2.1/servers/e506a6e0-fbcd-4e53-a6a0-d31761654c92 => generated 1390 bytes in 753 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:09:07.695300 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-f9bbce6a-42e4-4402-97f9-1762f05038a1 req-2b3afdf7-0459-47a2-95a6-792db7aac6e4 service nova] Creating event network-changed:fa66e5de-7c5f-4e0e-adc5-96ea3b90bd37 for instance e506a6e0-fbcd-4e53-a6a0-d31761654c92 on nch1-master-1 Jul 16 21:09:07.702236 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-f9bbce6a-42e4-4402-97f9-1762f05038a1 req-2b3afdf7-0459-47a2-95a6-792db7aac6e4 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.045957 Jul 16 21:09:07.704668 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 124/244] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:09:07 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:09:07.820208 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell0:SELECT=10 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:09:08.475845 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-d7feca1c-1c8f-40b0-abfc-0430d708548e tempest-MigrationsAdminTest-399208122 tempest-MigrationsAdminTest-399208122-project-admin] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:08.479298 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-d7feca1c-1c8f-40b0-abfc-0430d708548e tempest-MigrationsAdminTest-399208122 tempest-MigrationsAdminTest-399208122-project-admin] [instance: 1703f309-9e2e-4c5f-8588-eb4e43f21d91] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:09:08.512131 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d7feca1c-1c8f-40b0-abfc-0430d708548e tempest-MigrationsAdminTest-399208122 tempest-MigrationsAdminTest-399208122-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:08.512722 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d7feca1c-1c8f-40b0-abfc-0430d708548e tempest-MigrationsAdminTest-399208122 tempest-MigrationsAdminTest-399208122-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:08.514118 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d7feca1c-1c8f-40b0-abfc-0430d708548e tempest-MigrationsAdminTest-399208122 tempest-MigrationsAdminTest-399208122-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:08.523870 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d7feca1c-1c8f-40b0-abfc-0430d708548e tempest-MigrationsAdminTest-399208122 tempest-MigrationsAdminTest-399208122-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:08.646761 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-74461de9-8874-4272-be59-ea255f3c7308 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServersOnMultiNodesTest-server-701383621", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452"}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:09:08.874567 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.network.neutron [None req-74461de9-8874-4272-be59-ea255f3c7308 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] validate_networks() for None {{(pid=138681) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 16 21:09:09.273087 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-74461de9-8874-4272-be59-ea255f3c7308 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Flavor limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 16 21:09:09.276623 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-74461de9-8874-4272-be59-ea255f3c7308 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Image limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 16 21:09:09.276623 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-74461de9-8874-4272-be59-ea255f3c7308 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Flavor pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 16 21:09:09.276623 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-74461de9-8874-4272-be59-ea255f3c7308 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Image pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 16 21:09:09.276623 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-74461de9-8874-4272-be59-ea255f3c7308 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 16 21:09:09.300640 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-74461de9-8874-4272-be59-ea255f3c7308 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:09.304641 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-74461de9-8874-4272-be59-ea255f3c7308 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:09.308212 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-74461de9-8874-4272-be59-ea255f3c7308 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:09.308789 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-74461de9-8874-4272-be59-ea255f3c7308 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:09.336630 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-74461de9-8874-4272-be59-ea255f3c7308 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Getting quotas for project 649babe54d104436af461f1ee4767001. Resources: {'ram', 'instances', 'cores'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:09:09.343616 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-74461de9-8874-4272-be59-ea255f3c7308 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Getting quotas for user abeb7020019d41a18de42956d706a011 and project 649babe54d104436af461f1ee4767001. Resources: {'ram', 'instances', 'cores'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:09:09.361847 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-74461de9-8874-4272-be59-ea255f3c7308 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Going to run 1 instances... {{(pid=138681) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 16 21:09:09.367794 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-74461de9-8874-4272-be59-ea255f3c7308 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] [instance: 148d5ad9-2900-4c2b-909b-aa446757e33d] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138681) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 16 21:09:09.437927 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-d7feca1c-1c8f-40b0-abfc-0430d708548e tempest-MigrationsAdminTest-399208122 tempest-MigrationsAdminTest-399208122-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/1703f309-9e2e-4c5f-8588-eb4e43f21d91" status: 200 len: 1612 microversion: 2.1 time: 1.731812 Jul 16 21:09:09.439171 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 125/245] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:09:07 2022] GET /compute/v2.1/servers/1703f309-9e2e-4c5f-8588-eb4e43f21d91 => generated 1612 bytes in 1734 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:09:09.467609 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-90df434b-ec75-43a5-9d6c-c41f8c9bb2a8 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:09.473498 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-90df434b-ec75-43a5-9d6c-c41f8c9bb2a8 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] [instance: e506a6e0-fbcd-4e53-a6a0-d31761654c92] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:09:09.493047 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-90df434b-ec75-43a5-9d6c-c41f8c9bb2a8 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:09.493576 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-90df434b-ec75-43a5-9d6c-c41f8c9bb2a8 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:09.499900 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-90df434b-ec75-43a5-9d6c-c41f8c9bb2a8 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:09.500362 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-90df434b-ec75-43a5-9d6c-c41f8c9bb2a8 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:09.537569 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-74461de9-8874-4272-be59-ea255f3c7308 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 1.851076 Jul 16 21:09:09.542174 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 121/246] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 21:09:07 2022] POST /compute/v2.1/servers => generated 380 bytes in 1856 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 16 21:09:10.005940 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-eda8b57f-3425-4f14-80eb-3b55de7fa22a tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-LiveMigrationNegativeTest-server-938218394", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452"}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:09:10.087148 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-90df434b-ec75-43a5-9d6c-c41f8c9bb2a8 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ef9cbce2b2e64665b7f50767342dcb46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0ee02e4c8b448f2b77798ae504129b9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:10.100719 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.network.neutron [None req-eda8b57f-3425-4f14-80eb-3b55de7fa22a tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] validate_networks() for None {{(pid=138681) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 16 21:09:10.108772 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-90df434b-ec75-43a5-9d6c-c41f8c9bb2a8 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] 10.222.0.16 "GET /compute/v2.1/servers/e506a6e0-fbcd-4e53-a6a0-d31761654c92" status: 200 len: 1390 microversion: 2.1 time: 0.663553 Jul 16 21:09:10.109371 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 126/247] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:09:09 2022] GET /compute/v2.1/servers/e506a6e0-fbcd-4e53-a6a0-d31761654c92 => generated 1390 bytes in 665 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:09:10.153458 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-c74223c5-88f1-4095-b044-c0667906b49f tempest-MigrationsAdminTest-399208122 tempest-MigrationsAdminTest-399208122-project-admin] Action: 'action', calling method: >, body: {"migrate": {}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:09:10.154287 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-c74223c5-88f1-4095-b044-c0667906b49f tempest-MigrationsAdminTest-399208122 tempest-MigrationsAdminTest-399208122-project-admin] [instance: 1703f309-9e2e-4c5f-8588-eb4e43f21d91] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:09:10.169270 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c74223c5-88f1-4095-b044-c0667906b49f tempest-MigrationsAdminTest-399208122 tempest-MigrationsAdminTest-399208122-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:10.169913 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c74223c5-88f1-4095-b044-c0667906b49f tempest-MigrationsAdminTest-399208122 tempest-MigrationsAdminTest-399208122-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:10.170871 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c74223c5-88f1-4095-b044-c0667906b49f tempest-MigrationsAdminTest-399208122 tempest-MigrationsAdminTest-399208122-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:10.171689 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c74223c5-88f1-4095-b044-c0667906b49f tempest-MigrationsAdminTest-399208122 tempest-MigrationsAdminTest-399208122-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:10.425961 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-c74223c5-88f1-4095-b044-c0667906b49f tempest-MigrationsAdminTest-399208122 tempest-MigrationsAdminTest-399208122-project-admin] Policy check for compute:servers:resize:cross_cell failed with credentials {'is_admin': True, 'user_id': '1ead1619fabc4552b11f712b28cc6187', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cbbed051cbaf450f92a79d71d58ba7a2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:10.426596 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-c74223c5-88f1-4095-b044-c0667906b49f tempest-MigrationsAdminTest-399208122 tempest-MigrationsAdminTest-399208122-project-admin] [instance: 1703f309-9e2e-4c5f-8588-eb4e43f21d91] flavor_id is None. Assuming migration. {{(pid=138680) resize /opt/stack/nova/nova/compute/api.py:4142}} Jul 16 21:09:10.427083 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-c74223c5-88f1-4095-b044-c0667906b49f tempest-MigrationsAdminTest-399208122 tempest-MigrationsAdminTest-399208122-project-admin] [instance: 1703f309-9e2e-4c5f-8588-eb4e43f21d91] Old instance type m1.heat2, new instance type m1.heat2 {{(pid=138680) resize /opt/stack/nova/nova/compute/api.py:4166}} Jul 16 21:09:10.721906 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-eda8b57f-3425-4f14-80eb-3b55de7fa22a tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Flavor limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 16 21:09:10.722768 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-eda8b57f-3425-4f14-80eb-3b55de7fa22a tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Image limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 16 21:09:10.723386 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-eda8b57f-3425-4f14-80eb-3b55de7fa22a tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Flavor pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 16 21:09:10.723937 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-eda8b57f-3425-4f14-80eb-3b55de7fa22a tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Image pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 16 21:09:10.724707 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-eda8b57f-3425-4f14-80eb-3b55de7fa22a tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 16 21:09:10.741814 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-eda8b57f-3425-4f14-80eb-3b55de7fa22a tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:10.742274 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-eda8b57f-3425-4f14-80eb-3b55de7fa22a tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:10.744119 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-eda8b57f-3425-4f14-80eb-3b55de7fa22a tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:10.744569 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-eda8b57f-3425-4f14-80eb-3b55de7fa22a tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:10.788955 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-eda8b57f-3425-4f14-80eb-3b55de7fa22a tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Getting quotas for project fc1c3463f9444119af9d18f85bdd703d. Resources: {'ram', 'instances', 'cores'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:09:10.795659 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-eda8b57f-3425-4f14-80eb-3b55de7fa22a tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Getting quotas for user 0d72626f465b4f37ad3b18bb945d4000 and project fc1c3463f9444119af9d18f85bdd703d. Resources: {'ram', 'instances', 'cores'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:09:10.812363 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-eda8b57f-3425-4f14-80eb-3b55de7fa22a tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Going to run 1 instances... {{(pid=138681) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 16 21:09:10.815148 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-eda8b57f-3425-4f14-80eb-3b55de7fa22a tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] [instance: 1631f0cc-de67-4c0d-abd0-3e0e11dfb4ed] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138681) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 16 21:09:11.102196 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-eda8b57f-3425-4f14-80eb-3b55de7fa22a tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 1.556448 Jul 16 21:09:11.104345 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 122/248] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 21:09:09 2022] POST /compute/v2.1/servers => generated 380 bytes in 1561 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 16 21:09:11.113503 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-132810d3-034c-4ac3-89f1-d52c3ad2b9d6 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:11.117394 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-132810d3-034c-4ac3-89f1-d52c3ad2b9d6 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] [instance: 148d5ad9-2900-4c2b-909b-aa446757e33d] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:09:11.336938 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-c74223c5-88f1-4095-b044-c0667906b49f tempest-MigrationsAdminTest-399208122 tempest-MigrationsAdminTest-399208122-project-admin] 10.222.0.16 "POST /compute/v2.1/servers/1703f309-9e2e-4c5f-8588-eb4e43f21d91/action" status: 202 len: 0 microversion: 2.1 time: 1.193961 Jul 16 21:09:11.340570 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 127/249] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 21:09:10 2022] POST /compute/v2.1/servers/1703f309-9e2e-4c5f-8588-eb4e43f21d91/action => generated 0 bytes in 1228 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 21:09:11.350050 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-abbddfc9-9fb5-4b06-8919-39a5b50cb185 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:11.353047 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-abbddfc9-9fb5-4b06-8919-39a5b50cb185 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] [instance: 1631f0cc-de67-4c0d-abd0-3e0e11dfb4ed] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:09:11.409977 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-132810d3-034c-4ac3-89f1-d52c3ad2b9d6 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'abeb7020019d41a18de42956d706a011', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '649babe54d104436af461f1ee4767001', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:11.422188 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-132810d3-034c-4ac3-89f1-d52c3ad2b9d6 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] 10.222.0.16 "GET /compute/v2.1/servers/148d5ad9-2900-4c2b-909b-aa446757e33d" status: 200 len: 1216 microversion: 2.1 time: 0.313767 Jul 16 21:09:11.432713 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 123/250] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:09:11 2022] GET /compute/v2.1/servers/148d5ad9-2900-4c2b-909b-aa446757e33d => generated 1216 bytes in 328 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:09:11.453397 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-db623ab7-f5eb-4b2c-b477-f78d889599eb tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:11.454922 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-db623ab7-f5eb-4b2c-b477-f78d889599eb tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] [instance: e506a6e0-fbcd-4e53-a6a0-d31761654c92] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:09:11.464543 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-db623ab7-f5eb-4b2c-b477-f78d889599eb tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:11.465003 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-db623ab7-f5eb-4b2c-b477-f78d889599eb tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:11.466552 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-db623ab7-f5eb-4b2c-b477-f78d889599eb tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:11.467031 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-db623ab7-f5eb-4b2c-b477-f78d889599eb tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:11.470406 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-abbddfc9-9fb5-4b06-8919-39a5b50cb185 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0d72626f465b4f37ad3b18bb945d4000', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fc1c3463f9444119af9d18f85bdd703d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:11.485902 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-abbddfc9-9fb5-4b06-8919-39a5b50cb185 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] 10.222.0.16 "GET /compute/v2.1/servers/1631f0cc-de67-4c0d-abd0-3e0e11dfb4ed" status: 200 len: 1218 microversion: 2.1 time: 0.145814 Jul 16 21:09:11.488580 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 128/251] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:09:11 2022] GET /compute/v2.1/servers/1631f0cc-de67-4c0d-abd0-3e0e11dfb4ed => generated 1218 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:09:11.493983 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-53084f6e-6d97-4f28-8767-e9176b9eab66 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:11.495455 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-53084f6e-6d97-4f28-8767-e9176b9eab66 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: 1703f309-9e2e-4c5f-8588-eb4e43f21d91] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:09:11.503438 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-53084f6e-6d97-4f28-8767-e9176b9eab66 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:11.506687 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-53084f6e-6d97-4f28-8767-e9176b9eab66 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:11.506687 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-53084f6e-6d97-4f28-8767-e9176b9eab66 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:11.506687 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-53084f6e-6d97-4f28-8767-e9176b9eab66 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:11.528260 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:09:11.834996 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-db623ab7-f5eb-4b2c-b477-f78d889599eb tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ef9cbce2b2e64665b7f50767342dcb46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0ee02e4c8b448f2b77798ae504129b9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:11.845918 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-db623ab7-f5eb-4b2c-b477-f78d889599eb tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] 10.222.0.16 "GET /compute/v2.1/servers/e506a6e0-fbcd-4e53-a6a0-d31761654c92" status: 200 len: 1390 microversion: 2.1 time: 0.411338 Jul 16 21:09:11.846124 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-53084f6e-6d97-4f28-8767-e9176b9eab66 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b68dacc4b8463ba07fae6372131117', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55a19b7a4682457a9132f5fc80110712', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:11.849764 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 124/252] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:09:11 2022] GET /compute/v2.1/servers/e506a6e0-fbcd-4e53-a6a0-d31761654c92 => generated 1390 bytes in 416 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:09:11.872298 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-53084f6e-6d97-4f28-8767-e9176b9eab66 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "GET /compute/v2.1/servers/1703f309-9e2e-4c5f-8588-eb4e43f21d91" status: 200 len: 1470 microversion: 2.1 time: 0.381925 Jul 16 21:09:11.873141 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 129/253] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:09:11 2022] GET /compute/v2.1/servers/1703f309-9e2e-4c5f-8588-eb4e43f21d91 => generated 1470 bytes in 384 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:09:12.439429 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-d48923dc-f5ce-4034-9dac-0f8bd6d83661 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:12.440635 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-d48923dc-f5ce-4034-9dac-0f8bd6d83661 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] [instance: 148d5ad9-2900-4c2b-909b-aa446757e33d] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:09:12.453553 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d48923dc-f5ce-4034-9dac-0f8bd6d83661 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:12.463295 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d48923dc-f5ce-4034-9dac-0f8bd6d83661 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:12.464362 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d48923dc-f5ce-4034-9dac-0f8bd6d83661 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:12.464826 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d48923dc-f5ce-4034-9dac-0f8bd6d83661 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:12.505687 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-c10e5203-1a76-433a-9a00-0dc2bac63d04 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:12.506878 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-c10e5203-1a76-433a-9a00-0dc2bac63d04 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] [instance: 1631f0cc-de67-4c0d-abd0-3e0e11dfb4ed] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:09:12.593814 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-d48923dc-f5ce-4034-9dac-0f8bd6d83661 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'abeb7020019d41a18de42956d706a011', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '649babe54d104436af461f1ee4767001', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:12.604069 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-d48923dc-f5ce-4034-9dac-0f8bd6d83661 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] 10.222.0.16 "GET /compute/v2.1/servers/148d5ad9-2900-4c2b-909b-aa446757e33d" status: 200 len: 1268 microversion: 2.1 time: 0.168878 Jul 16 21:09:12.619437 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 125/254] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:09:12 2022] GET /compute/v2.1/servers/148d5ad9-2900-4c2b-909b-aa446757e33d => generated 1268 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:09:12.641527 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-c10e5203-1a76-433a-9a00-0dc2bac63d04 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0d72626f465b4f37ad3b18bb945d4000', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fc1c3463f9444119af9d18f85bdd703d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:12.655751 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-c10e5203-1a76-433a-9a00-0dc2bac63d04 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] 10.222.0.16 "GET /compute/v2.1/servers/1631f0cc-de67-4c0d-abd0-3e0e11dfb4ed" status: 200 len: 1218 microversion: 2.1 time: 0.154969 Jul 16 21:09:12.656720 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 130/255] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:09:12 2022] GET /compute/v2.1/servers/1631f0cc-de67-4c0d-abd0-3e0e11dfb4ed => generated 1218 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:09:12.865347 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-adbe386d-5dc6-487c-b195-e1503f72bb4e tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:12.866169 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-adbe386d-5dc6-487c-b195-e1503f72bb4e tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] [instance: e506a6e0-fbcd-4e53-a6a0-d31761654c92] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:09:12.877873 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-adbe386d-5dc6-487c-b195-e1503f72bb4e tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:12.878433 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-adbe386d-5dc6-487c-b195-e1503f72bb4e tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:12.881445 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-adbe386d-5dc6-487c-b195-e1503f72bb4e tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:12.882548 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-adbe386d-5dc6-487c-b195-e1503f72bb4e tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:12.892507 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-e299c0de-d75a-46ec-a4a7-9f3f9c2c3336 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:12.894202 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-e299c0de-d75a-46ec-a4a7-9f3f9c2c3336 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: 1703f309-9e2e-4c5f-8588-eb4e43f21d91] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:09:12.906268 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e299c0de-d75a-46ec-a4a7-9f3f9c2c3336 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:12.907645 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e299c0de-d75a-46ec-a4a7-9f3f9c2c3336 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:12.908753 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e299c0de-d75a-46ec-a4a7-9f3f9c2c3336 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:12.909454 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e299c0de-d75a-46ec-a4a7-9f3f9c2c3336 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:13.407212 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-e299c0de-d75a-46ec-a4a7-9f3f9c2c3336 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b68dacc4b8463ba07fae6372131117', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55a19b7a4682457a9132f5fc80110712', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:13.416734 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-e299c0de-d75a-46ec-a4a7-9f3f9c2c3336 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "GET /compute/v2.1/servers/1703f309-9e2e-4c5f-8588-eb4e43f21d91" status: 200 len: 1470 microversion: 2.1 time: 0.529987 Jul 16 21:09:13.419193 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 131/256] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:09:12 2022] GET /compute/v2.1/servers/1703f309-9e2e-4c5f-8588-eb4e43f21d91 => generated 1470 bytes in 534 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:09:13.426496 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-adbe386d-5dc6-487c-b195-e1503f72bb4e tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ef9cbce2b2e64665b7f50767342dcb46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0ee02e4c8b448f2b77798ae504129b9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:13.437280 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-adbe386d-5dc6-487c-b195-e1503f72bb4e tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] 10.222.0.16 "GET /compute/v2.1/servers/e506a6e0-fbcd-4e53-a6a0-d31761654c92" status: 200 len: 1390 microversion: 2.1 time: 0.576055 Jul 16 21:09:13.440587 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 126/257] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:09:12 2022] GET /compute/v2.1/servers/e506a6e0-fbcd-4e53-a6a0-d31761654c92 => generated 1390 bytes in 581 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:09:13.627305 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-23c659e5-ae0d-424b-a087-bb0d9f35fb48 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:13.628846 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-23c659e5-ae0d-424b-a087-bb0d9f35fb48 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] [instance: 148d5ad9-2900-4c2b-909b-aa446757e33d] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:09:13.639724 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-23c659e5-ae0d-424b-a087-bb0d9f35fb48 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:13.642063 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-23c659e5-ae0d-424b-a087-bb0d9f35fb48 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:13.642785 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-23c659e5-ae0d-424b-a087-bb0d9f35fb48 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:13.644157 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-23c659e5-ae0d-424b-a087-bb0d9f35fb48 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:13.674618 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-56e52811-5c1f-483b-ab0d-a3da62341c12 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:13.675298 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-56e52811-5c1f-483b-ab0d-a3da62341c12 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] [instance: 1631f0cc-de67-4c0d-abd0-3e0e11dfb4ed] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:09:13.687654 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-56e52811-5c1f-483b-ab0d-a3da62341c12 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:13.689015 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-56e52811-5c1f-483b-ab0d-a3da62341c12 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:13.693249 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-56e52811-5c1f-483b-ab0d-a3da62341c12 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:13.694011 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-56e52811-5c1f-483b-ab0d-a3da62341c12 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:13.809154 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-23c659e5-ae0d-424b-a087-bb0d9f35fb48 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'abeb7020019d41a18de42956d706a011', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '649babe54d104436af461f1ee4767001', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:13.818952 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-23c659e5-ae0d-424b-a087-bb0d9f35fb48 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] 10.222.0.16 "GET /compute/v2.1/servers/148d5ad9-2900-4c2b-909b-aa446757e33d" status: 200 len: 1286 microversion: 2.1 time: 0.199639 Jul 16 21:09:13.831466 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 132/258] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:09:13 2022] GET /compute/v2.1/servers/148d5ad9-2900-4c2b-909b-aa446757e33d => generated 1286 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:09:13.850935 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-56e52811-5c1f-483b-ab0d-a3da62341c12 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0d72626f465b4f37ad3b18bb945d4000', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fc1c3463f9444119af9d18f85bdd703d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:13.874853 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-56e52811-5c1f-483b-ab0d-a3da62341c12 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] 10.222.0.16 "GET /compute/v2.1/servers/1631f0cc-de67-4c0d-abd0-3e0e11dfb4ed" status: 200 len: 1214 microversion: 2.1 time: 0.204696 Jul 16 21:09:13.887969 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 127/259] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:09:13 2022] GET /compute/v2.1/servers/1631f0cc-de67-4c0d-abd0-3e0e11dfb4ed => generated 1214 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:09:14.439148 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-e1adb1de-5166-475e-9bc7-fb0f2eec9724 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:14.440209 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-e1adb1de-5166-475e-9bc7-fb0f2eec9724 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: 1703f309-9e2e-4c5f-8588-eb4e43f21d91] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:09:14.442415 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell0:SELECT=4 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:09:14.446075 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_api:SELECT=85,nova_api:INSERT=4,nova_api:DELETE=9,nova_api:UPDATE=2 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:09:14.457575 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-5a254f79-266b-4750-ae6c-bb314b4eb6a5 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:14.458202 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-5a254f79-266b-4750-ae6c-bb314b4eb6a5 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] [instance: e506a6e0-fbcd-4e53-a6a0-d31761654c92] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:09:14.460410 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e1adb1de-5166-475e-9bc7-fb0f2eec9724 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:14.461925 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e1adb1de-5166-475e-9bc7-fb0f2eec9724 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:14.462860 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e1adb1de-5166-475e-9bc7-fb0f2eec9724 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:14.464635 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e1adb1de-5166-475e-9bc7-fb0f2eec9724 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:14.481298 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5a254f79-266b-4750-ae6c-bb314b4eb6a5 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:14.482720 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5a254f79-266b-4750-ae6c-bb314b4eb6a5 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:14.488255 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5a254f79-266b-4750-ae6c-bb314b4eb6a5 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:14.488934 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5a254f79-266b-4750-ae6c-bb314b4eb6a5 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:15.100850 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-5a254f79-266b-4750-ae6c-bb314b4eb6a5 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ef9cbce2b2e64665b7f50767342dcb46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0ee02e4c8b448f2b77798ae504129b9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:15.111870 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-5a254f79-266b-4750-ae6c-bb314b4eb6a5 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] 10.222.0.16 "GET /compute/v2.1/servers/e506a6e0-fbcd-4e53-a6a0-d31761654c92" status: 200 len: 1390 microversion: 2.1 time: 0.659043 Jul 16 21:09:15.113088 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 128/260] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:09:14 2022] GET /compute/v2.1/servers/e506a6e0-fbcd-4e53-a6a0-d31761654c92 => generated 1390 bytes in 661 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:09:15.126514 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-1d105de6-cad9-4b4a-ba2f-a09e8aa774ce req-6a7e8cca-3d2b-41e2-977b-a87ed2095085 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "e506a6e0-fbcd-4e53-a6a0-d31761654c92", "name": "network-vif-plugged", "status": "completed", "tag": "fa66e5de-7c5f-4e0e-adc5-96ea3b90bd37"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:09:15.142682 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-1d105de6-cad9-4b4a-ba2f-a09e8aa774ce req-6a7e8cca-3d2b-41e2-977b-a87ed2095085 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:15.145853 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-1d105de6-cad9-4b4a-ba2f-a09e8aa774ce req-6a7e8cca-3d2b-41e2-977b-a87ed2095085 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:15.166803 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-1d105de6-cad9-4b4a-ba2f-a09e8aa774ce req-6a7e8cca-3d2b-41e2-977b-a87ed2095085 service nova] Creating event network-vif-plugged:fa66e5de-7c5f-4e0e-adc5-96ea3b90bd37 for instance e506a6e0-fbcd-4e53-a6a0-d31761654c92 on nch1-master-1 Jul 16 21:09:15.179179 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-1d105de6-cad9-4b4a-ba2f-a09e8aa774ce req-6a7e8cca-3d2b-41e2-977b-a87ed2095085 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.059253 Jul 16 21:09:15.180733 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 129/261] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:09:15 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 67 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:09:15.200335 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-744bc993-0ad6-429e-be4b-fd20271d4bf9 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:15.201530 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-744bc993-0ad6-429e-be4b-fd20271d4bf9 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] [instance: 148d5ad9-2900-4c2b-909b-aa446757e33d] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:09:15.225862 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-744bc993-0ad6-429e-be4b-fd20271d4bf9 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:15.226826 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-744bc993-0ad6-429e-be4b-fd20271d4bf9 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:15.228748 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-744bc993-0ad6-429e-be4b-fd20271d4bf9 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:15.230203 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-744bc993-0ad6-429e-be4b-fd20271d4bf9 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:15.516510 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-e1adb1de-5166-475e-9bc7-fb0f2eec9724 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b68dacc4b8463ba07fae6372131117', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55a19b7a4682457a9132f5fc80110712', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:15.556994 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-e1adb1de-5166-475e-9bc7-fb0f2eec9724 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "GET /compute/v2.1/servers/1703f309-9e2e-4c5f-8588-eb4e43f21d91" status: 200 len: 1470 microversion: 2.1 time: 1.124585 Jul 16 21:09:15.558208 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 133/262] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:09:14 2022] GET /compute/v2.1/servers/1703f309-9e2e-4c5f-8588-eb4e43f21d91 => generated 1470 bytes in 1126 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:09:15.563868 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-156ebe39-0439-48bb-b446-659388adc01f tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:15.564700 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-156ebe39-0439-48bb-b446-659388adc01f tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] [instance: 1631f0cc-de67-4c0d-abd0-3e0e11dfb4ed] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:09:15.571254 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-156ebe39-0439-48bb-b446-659388adc01f tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:15.571950 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-156ebe39-0439-48bb-b446-659388adc01f tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:15.572945 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-156ebe39-0439-48bb-b446-659388adc01f tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:15.573517 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-156ebe39-0439-48bb-b446-659388adc01f tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:15.985339 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-744bc993-0ad6-429e-be4b-fd20271d4bf9 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'abeb7020019d41a18de42956d706a011', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '649babe54d104436af461f1ee4767001', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:15.995158 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-744bc993-0ad6-429e-be4b-fd20271d4bf9 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] 10.222.0.16 "GET /compute/v2.1/servers/148d5ad9-2900-4c2b-909b-aa446757e33d" status: 200 len: 1274 microversion: 2.1 time: 0.810296 Jul 16 21:09:16.001728 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 130/263] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:09:15 2022] GET /compute/v2.1/servers/148d5ad9-2900-4c2b-909b-aa446757e33d => generated 1274 bytes in 820 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:09:16.016762 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-156ebe39-0439-48bb-b446-659388adc01f tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0d72626f465b4f37ad3b18bb945d4000', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fc1c3463f9444119af9d18f85bdd703d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:16.027866 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-156ebe39-0439-48bb-b446-659388adc01f tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] 10.222.0.16 "GET /compute/v2.1/servers/1631f0cc-de67-4c0d-abd0-3e0e11dfb4ed" status: 200 len: 1288 microversion: 2.1 time: 0.468253 Jul 16 21:09:16.037143 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 134/264] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:09:15 2022] GET /compute/v2.1/servers/1631f0cc-de67-4c0d-abd0-3e0e11dfb4ed => generated 1288 bytes in 470 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:09:16.129614 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-dba16e07-2e99-430a-879b-f4657089d199 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:16.130863 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-dba16e07-2e99-430a-879b-f4657089d199 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] [instance: e506a6e0-fbcd-4e53-a6a0-d31761654c92] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:09:16.138309 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-dba16e07-2e99-430a-879b-f4657089d199 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:16.139162 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-dba16e07-2e99-430a-879b-f4657089d199 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:16.140862 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-dba16e07-2e99-430a-879b-f4657089d199 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:16.141407 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-dba16e07-2e99-430a-879b-f4657089d199 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:16.696255 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-c8c0e99e-6406-4b0a-aa8f-940271252425 tempest-FlavorsAdminTestJSON-758135383 tempest-FlavorsAdminTestJSON-758135383-project-admin] Action: 'create', calling method: >, body: {"flavor": {"ram": "1024", "vcpus": 1, "disk": 10, "name": "tempest-FlavorsAdminTestJSON-flavor-804223490", "id": 1717993016, "os-flavor-access:is_public": "True"}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:09:16.698324 ncdv-master-1 devstack@n-api.service[138680]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-flavor-manage:create": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:09:16.698324 ncdv-master-1 devstack@n-api.service[138680]: warnings.warn(msg) Jul 16 21:09:16.764906 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-dba16e07-2e99-430a-879b-f4657089d199 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ef9cbce2b2e64665b7f50767342dcb46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0ee02e4c8b448f2b77798ae504129b9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:16.775048 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-dba16e07-2e99-430a-879b-f4657089d199 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] 10.222.0.16 "GET /compute/v2.1/servers/e506a6e0-fbcd-4e53-a6a0-d31761654c92" status: 200 len: 1390 microversion: 2.1 time: 0.649613 Jul 16 21:09:16.777560 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 131/265] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:09:16 2022] GET /compute/v2.1/servers/e506a6e0-fbcd-4e53-a6a0-d31761654c92 => generated 1390 bytes in 653 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:09:16.792870 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-7a04e6e2-eafe-49f3-a43b-96b1f8ef6ca0 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:16.793923 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-7a04e6e2-eafe-49f3-a43b-96b1f8ef6ca0 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: 1703f309-9e2e-4c5f-8588-eb4e43f21d91] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:09:16.801019 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7a04e6e2-eafe-49f3-a43b-96b1f8ef6ca0 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:16.801562 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7a04e6e2-eafe-49f3-a43b-96b1f8ef6ca0 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:16.802591 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7a04e6e2-eafe-49f3-a43b-96b1f8ef6ca0 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:16.803523 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7a04e6e2-eafe-49f3-a43b-96b1f8ef6ca0 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:16.823126 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-c8c0e99e-6406-4b0a-aa8f-940271252425 tempest-FlavorsAdminTestJSON-758135383 tempest-FlavorsAdminTestJSON-758135383-project-admin] 10.222.0.16 "POST /compute/v2.1/flavors" status: 200 len: 429 microversion: 2.1 time: 0.373794 Jul 16 21:09:16.824133 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 135/266] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 21:09:16 2022] POST /compute/v2.1/flavors => generated 429 bytes in 376 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:09:16.835293 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-b3b988c8-18c7-4ff3-86bd-d23ac8df1c9f tempest-FlavorsAdminTestJSON-758135383 tempest-FlavorsAdminTestJSON-758135383-project-admin] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:16.836169 ncdv-master-1 devstack@n-api.service[138680]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-flavor-manage:delete": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:09:16.836169 ncdv-master-1 devstack@n-api.service[138680]: warnings.warn(msg) Jul 16 21:09:16.845809 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=100 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:09:16.965445 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-b3b988c8-18c7-4ff3-86bd-d23ac8df1c9f tempest-FlavorsAdminTestJSON-758135383 tempest-FlavorsAdminTestJSON-758135383-project-admin] 10.222.0.16 "DELETE /compute/v2.1/flavors/1717993016" status: 202 len: 0 microversion: 2.1 time: 0.133835 Jul 16 21:09:16.967505 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 136/267] 10.222.0.16 () {58 vars in 1160 bytes} [Sat Jul 16 21:09:16 2022] DELETE /compute/v2.1/flavors/1717993016 => generated 0 bytes in 137 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 21:09:17.235313 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-c0528e1c-7e2a-4859-bbe0-b820e83c1e75 tempest-FlavorsAccessNegativeTestJSON-1113799613 tempest-FlavorsAccessNegativeTestJSON-1113799613-project-admin] Action: 'create', calling method: >, body: {"flavor": {"ram": 512, "vcpus": 1, "disk": 10, "name": "tempest-FlavorsAccessNegativeTestJSON-flavor-1057537946", "id": 2134620316, "os-flavor-access:is_public": "False"}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:09:17.257441 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-7a04e6e2-eafe-49f3-a43b-96b1f8ef6ca0 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b68dacc4b8463ba07fae6372131117', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55a19b7a4682457a9132f5fc80110712', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:17.268860 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-7a04e6e2-eafe-49f3-a43b-96b1f8ef6ca0 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "GET /compute/v2.1/servers/1703f309-9e2e-4c5f-8588-eb4e43f21d91" status: 200 len: 1470 microversion: 2.1 time: 0.489913 Jul 16 21:09:17.281622 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 132/268] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:09:16 2022] GET /compute/v2.1/servers/1703f309-9e2e-4c5f-8588-eb4e43f21d91 => generated 1470 bytes in 495 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:09:17.283750 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-5bc47038-d2f3-4c7a-83ef-7ddd34e34854 tempest-FlavorsAdminTestJSON-758135383 tempest-FlavorsAdminTestJSON-758135383-project-admin] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:17.325782 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-5bc47038-d2f3-4c7a-83ef-7ddd34e34854 tempest-FlavorsAdminTestJSON-758135383 tempest-FlavorsAdminTestJSON-758135383-project-admin] 10.222.0.16 "GET /compute/v2.1/flavors/detail" status: 200 len: 5819 microversion: 2.1 time: 0.046235 Jul 16 21:09:17.340136 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 133/269] 10.222.0.16 () {58 vars in 1145 bytes} [Sat Jul 16 21:09:17 2022] GET /compute/v2.1/flavors/detail => generated 5819 bytes in 67 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:09:17.353015 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-7bd81521-c42f-43b4-a6bf-d68056e3c380 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:17.353923 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-7bd81521-c42f-43b4-a6bf-d68056e3c380 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] [instance: 148d5ad9-2900-4c2b-909b-aa446757e33d] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:09:17.362803 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7bd81521-c42f-43b4-a6bf-d68056e3c380 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:17.363476 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7bd81521-c42f-43b4-a6bf-d68056e3c380 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:17.364518 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7bd81521-c42f-43b4-a6bf-d68056e3c380 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:17.365039 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7bd81521-c42f-43b4-a6bf-d68056e3c380 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:17.409866 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-c0528e1c-7e2a-4859-bbe0-b820e83c1e75 tempest-FlavorsAccessNegativeTestJSON-1113799613 tempest-FlavorsAccessNegativeTestJSON-1113799613-project-admin] 10.222.0.16 "POST /compute/v2.1/flavors" status: 200 len: 439 microversion: 2.1 time: 0.435468 Jul 16 21:09:17.413602 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 137/270] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 21:09:16 2022] POST /compute/v2.1/flavors => generated 439 bytes in 446 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:09:17.432953 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-900845ca-ce24-4fa5-ace7-30867676a22e tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:17.436951 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-900845ca-ce24-4fa5-ace7-30867676a22e tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] [instance: 1631f0cc-de67-4c0d-abd0-3e0e11dfb4ed] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:09:17.447396 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-900845ca-ce24-4fa5-ace7-30867676a22e tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:17.448030 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-900845ca-ce24-4fa5-ace7-30867676a22e tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:17.450063 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-900845ca-ce24-4fa5-ace7-30867676a22e tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:17.450835 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-900845ca-ce24-4fa5-ace7-30867676a22e tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:17.678457 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-900845ca-ce24-4fa5-ace7-30867676a22e tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0d72626f465b4f37ad3b18bb945d4000', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fc1c3463f9444119af9d18f85bdd703d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:17.688787 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-900845ca-ce24-4fa5-ace7-30867676a22e tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] 10.222.0.16 "GET /compute/v2.1/servers/1631f0cc-de67-4c0d-abd0-3e0e11dfb4ed" status: 200 len: 1276 microversion: 2.1 time: 0.273513 Jul 16 21:09:17.692105 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 138/271] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:09:17 2022] GET /compute/v2.1/servers/1631f0cc-de67-4c0d-abd0-3e0e11dfb4ed => generated 1276 bytes in 278 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:09:17.702273 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-53e08242-0379-4b5e-ad72-a4c87f7dda44 tempest-FlavorsAdminTestJSON-758135383 tempest-FlavorsAdminTestJSON-758135383-project-admin] Action: 'create', calling method: >, body: {"flavor": {"ram": 512, "vcpus": 1, "disk": 10, "name": "tempest-test_flavor_-479074529", "id": 790087394, "swap": 1024, "rxtx_factor": 2, "OS-FLV-EXT-DATA:ephemeral": 10, "os-flavor-access:is_public": "True"}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:09:17.704059 ncdv-master-1 devstack@n-api.service[138680]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-flavor-manage:create": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:09:17.704059 ncdv-master-1 devstack@n-api.service[138680]: warnings.warn(msg) Jul 16 21:09:17.839982 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-53e08242-0379-4b5e-ad72-a4c87f7dda44 tempest-FlavorsAdminTestJSON-758135383 tempest-FlavorsAdminTestJSON-758135383-project-admin] 10.222.0.16 "POST /compute/v2.1/flavors" status: 200 len: 413 microversion: 2.1 time: 0.146735 Jul 16 21:09:17.840838 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 139/272] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 21:09:17 2022] POST /compute/v2.1/flavors => generated 413 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:09:17.858490 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-17ee37b5-d983-4711-847f-b24939e3a910 tempest-FlavorsAccessNegativeTestJSON-1113799613 tempest-FlavorsAccessNegativeTestJSON-1113799613-project-admin] Action: 'action', calling method: >, body: {"addTenantAccess": {"tenant": "6beaabf2d684478fac58e01a0d3a77fe"}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:09:17.866304 ncdv-master-1 devstack@n-api.service[138680]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-flavor-access:add_tenant_access": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:09:17.866304 ncdv-master-1 devstack@n-api.service[138680]: warnings.warn(msg) Jul 16 21:09:17.979108 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-7bd81521-c42f-43b4-a6bf-d68056e3c380 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'abeb7020019d41a18de42956d706a011', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '649babe54d104436af461f1ee4767001', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:17.990045 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-7bd81521-c42f-43b4-a6bf-d68056e3c380 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] 10.222.0.16 "GET /compute/v2.1/servers/148d5ad9-2900-4c2b-909b-aa446757e33d" status: 200 len: 1316 microversion: 2.1 time: 0.641452 Jul 16 21:09:17.990943 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 134/273] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:09:17 2022] GET /compute/v2.1/servers/148d5ad9-2900-4c2b-909b-aa446757e33d => generated 1316 bytes in 643 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:09:17.996558 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-13eb7c0d-1202-4f95-b376-81b5f25fe837 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:17.997464 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-13eb7c0d-1202-4f95-b376-81b5f25fe837 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] [instance: e506a6e0-fbcd-4e53-a6a0-d31761654c92] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:09:18.013766 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-13eb7c0d-1202-4f95-b376-81b5f25fe837 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:18.014481 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-13eb7c0d-1202-4f95-b376-81b5f25fe837 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:18.015581 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-13eb7c0d-1202-4f95-b376-81b5f25fe837 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:18.016751 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-13eb7c0d-1202-4f95-b376-81b5f25fe837 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:18.049363 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-17ee37b5-d983-4711-847f-b24939e3a910 tempest-FlavorsAccessNegativeTestJSON-1113799613 tempest-FlavorsAccessNegativeTestJSON-1113799613-project-admin] 10.222.0.16 "POST /compute/v2.1/flavors/2134620316/action" status: 200 len: 97 microversion: 2.1 time: 0.206867 Jul 16 21:09:18.050946 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 140/274] 10.222.0.16 () {60 vars in 1199 bytes} [Sat Jul 16 21:09:17 2022] POST /compute/v2.1/flavors/2134620316/action => generated 97 bytes in 209 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Jul 16 21:09:18.060420 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-6cd26f9e-43e4-4022-9e8b-6537d067d4d1 tempest-FlavorsAdminTestJSON-758135383 tempest-FlavorsAdminTestJSON-758135383-project-admin] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:18.098252 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-6cd26f9e-43e4-4022-9e8b-6537d067d4d1 tempest-FlavorsAdminTestJSON-758135383 tempest-FlavorsAdminTestJSON-758135383-project-admin] 10.222.0.16 "GET /compute/v2.1/flavors/detail" status: 200 len: 6222 microversion: 2.1 time: 0.045877 Jul 16 21:09:18.099252 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 141/275] 10.222.0.16 () {58 vars in 1145 bytes} [Sat Jul 16 21:09:18 2022] GET /compute/v2.1/flavors/detail => generated 6222 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:09:18.104893 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-08ade805-bda3-49b7-babe-3e311bc7982d tempest-FlavorsAccessNegativeTestJSON-1113799613 tempest-FlavorsAccessNegativeTestJSON-1113799613-project-admin] Action: 'action', calling method: >, body: {"addTenantAccess": {"tenant": "6beaabf2d684478fac58e01a0d3a77fe"}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:09:18.228586 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-08ade805-bda3-49b7-babe-3e311bc7982d tempest-FlavorsAccessNegativeTestJSON-1113799613 tempest-FlavorsAccessNegativeTestJSON-1113799613-project-admin] HTTP exception thrown: Flavor access already exists for flavor 23 and project 6beaabf2d684478fac58e01a0d3a77fe combination. Jul 16 21:09:18.257973 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-08ade805-bda3-49b7-babe-3e311bc7982d tempest-FlavorsAccessNegativeTestJSON-1113799613 tempest-FlavorsAccessNegativeTestJSON-1113799613-project-admin] Request method failure captured: Jul 16 21:09:18.257973 ncdv-master-1 devstack@n-api.service[138680]: request: POST /compute/v2.1/flavors/2134620316/action HTTP/1.1 Jul 16 21:09:18.257973 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:09:18.257973 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:09:18.257973 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:09:18.257973 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 67 Jul 16 21:09:18.257973 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:09:18.257973 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:09:18.257973 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:09:18.257973 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:09:18.257973 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0yj7TqhQMly9bDnDqOBZY1OHAWHNoSytoQcgL-V9TmjaQVAVWqaNvpeVrgxpHpBh8P29OcQ9zLIwJPxFZaJdC9FWMwFzpXueXcVMXCBTEwHMfizgMcOeg2IKk9VuXZnnHMfm4IQdXMisOAsjwoyoxjedCTDXTY7Kwsohcm0agHaBdtk Jul 16 21:09:18.257973 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:09:18.257973 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:09:18.257973 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:09:18.257973 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:09:18.257973 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:09:18.257973 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:09:18.257973 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: 054de79bd580451bafa72a8e903ffab5 Jul 16 21:09:18.257973 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-FlavorsAccessNegativeTestJSON-1113799613 Jul 16 21:09:18.257973 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,admin,reader Jul 16 21:09:18.257973 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,admin,reader Jul 16 21:09:18.259291 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/054de79bd580451bafa72a8e903ffab5"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/054de79bd580451bafa72a8e903ffab5"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_054de79bd580451bafa72a8e903ffab5"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/054de79bd580451bafa72a8e903ffab5"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:09:18.259291 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-FlavorsAccessNegativeTestJSON-1113799613 Jul 16 21:09:18.259291 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: 054de79bd580451bafa72a8e903ffab5 Jul 16 21:09:18.259291 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-FlavorsAccessNegativeTestJSON-1113799613 Jul 16 21:09:18.259291 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-FlavorsAccessNegativeTestJSON-1113799613-project-admin Jul 16 21:09:18.259291 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:09:18.259291 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:09:18.259291 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: 651cbc6596e24fcfa7232ddc273692ae Jul 16 21:09:18.259291 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-FlavorsAccessNegativeTestJSON-1113799613-project-admin Jul 16 21:09:18.259291 ncdv-master-1 devstack@n-api.service[138680]: Jul 16 21:09:18.259291 ncdv-master-1 devstack@n-api.service[138680]: {"addTenantAccess": {"tenant": "6beaabf2d684478fac58e01a0d3a77fe"}} Jul 16 21:09:18.259291 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 21:09:18.260011 ncdv-master-1 devstack@n-api.service[138680]: exception: Flavor access already exists for flavor 23 and project 6beaabf2d684478fac58e01a0d3a77fe combination. Jul 16 21:09:18.260011 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': '2134620316', 'body': {'addTenantAccess': {'tenant': '6beaabf2d684478fac58e01a0d3a77fe'}}} Jul 16 21:09:18.260011 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:09:18.260011 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:09:18.260011 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 16 21:09:18.260011 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi self.dialect.do_execute( Jul 16 21:09:18.260011 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 16 21:09:18.260011 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi cursor.execute(statement, parameters) Jul 16 21:09:18.260011 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 16 21:09:18.260011 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi result = self._query(query) Jul 16 21:09:18.260011 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 16 21:09:18.260011 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi conn.query(q) Jul 16 21:09:18.260011 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 16 21:09:18.260011 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 16 21:09:18.260011 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 16 21:09:18.260011 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi result.read() Jul 16 21:09:18.260011 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 16 21:09:18.260011 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi first_packet = self.connection._read_packet() Jul 16 21:09:18.261030 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 16 21:09:18.261030 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi packet.raise_for_error() Jul 16 21:09:18.261030 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 16 21:09:18.261030 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi err.raise_mysql_exception(self._data) Jul 16 21:09:18.261030 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 16 21:09:18.261030 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise errorclass(errno, errval) Jul 16 21:09:18.261030 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi pymysql.err.IntegrityError: (1062, "Duplicate entry '23-6beaabf2d684478fac58e01a0d3a77fe' for key 'flavor_projects.uniq_flavor_projects0flavor_id0project_id'") Jul 16 21:09:18.261030 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:09:18.261030 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi The above exception was the direct cause of the following exception: Jul 16 21:09:18.261030 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:09:18.261030 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:09:18.261030 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 73, in _flavor_add_project Jul 16 21:09:18.261030 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi project.save(context.session) Jul 16 21:09:18.261030 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/models.py", line 38, in save Jul 16 21:09:18.261030 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi session.flush() Jul 16 21:09:18.261030 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 16 21:09:18.261030 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi self._flush(objects) Jul 16 21:09:18.261030 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 16 21:09:18.261030 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi transaction.rollback(_capture_exception=True) Jul 16 21:09:18.263217 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 16 21:09:18.263217 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi compat.raise_( Jul 16 21:09:18.263217 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 16 21:09:18.263217 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception Jul 16 21:09:18.263217 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 16 21:09:18.263217 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi flush_context.execute() Jul 16 21:09:18.263217 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 16 21:09:18.263217 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi rec.execute(self) Jul 16 21:09:18.263217 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 16 21:09:18.263217 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi util.preloaded.orm_persistence.save_obj( Jul 16 21:09:18.263217 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 16 21:09:18.263217 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi _emit_insert_statements( Jul 16 21:09:18.263217 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jul 16 21:09:18.263217 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi result = connection._execute_20( Jul 16 21:09:18.263217 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 16 21:09:18.263217 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return meth(self, args_10style, kwargs_10style, execution_options) Jul 16 21:09:18.263217 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 16 21:09:18.263217 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return connection._execute_clauseelement( Jul 16 21:09:18.265691 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 16 21:09:18.265691 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi ret = self._execute_context( Jul 16 21:09:18.265691 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 16 21:09:18.265691 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi self._handle_dbapi_exception( Jul 16 21:09:18.265691 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 16 21:09:18.265691 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 16 21:09:18.265691 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 16 21:09:18.265691 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception Jul 16 21:09:18.265691 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 16 21:09:18.265691 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi self.dialect.do_execute( Jul 16 21:09:18.265691 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 16 21:09:18.265691 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi cursor.execute(statement, parameters) Jul 16 21:09:18.265691 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 16 21:09:18.265691 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi result = self._query(query) Jul 16 21:09:18.265691 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 16 21:09:18.265691 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi conn.query(q) Jul 16 21:09:18.265691 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 16 21:09:18.265691 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 16 21:09:18.267228 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 16 21:09:18.267228 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi result.read() Jul 16 21:09:18.267228 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 16 21:09:18.267228 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi first_packet = self.connection._read_packet() Jul 16 21:09:18.267228 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 16 21:09:18.267228 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi packet.raise_for_error() Jul 16 21:09:18.267228 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 16 21:09:18.267228 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi err.raise_mysql_exception(self._data) Jul 16 21:09:18.267228 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 16 21:09:18.267228 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise errorclass(errno, errval) Jul 16 21:09:18.267228 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry '23-6beaabf2d684478fac58e01a0d3a77fe' for key 'flavor_projects.uniq_flavor_projects0flavor_id0project_id'") Jul 16 21:09:18.267228 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi [SQL: INSERT INTO flavor_projects (created_at, updated_at, flavor_id, project_id) VALUES (%(created_at)s, %(updated_at)s, %(flavor_id)s, %(project_id)s)] Jul 16 21:09:18.267228 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi [parameters: {'created_at': datetime.datetime(2022, 7, 16, 21, 9, 18, 194812), 'updated_at': None, 'flavor_id': 23, 'project_id': '6beaabf2d684478fac58e01a0d3a77fe'}] Jul 16 21:09:18.267228 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 16 21:09:18.267228 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:09:18.267228 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:09:18.267228 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:09:18.268353 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:09:18.268353 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavor_access.py", line 79, in _add_tenant_access Jul 16 21:09:18.268353 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi flavor.add_access(tenant) Jul 16 21:09:18.268353 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 226, in wrapper Jul 16 21:09:18.268353 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return fn(self, *args, **kwargs) Jul 16 21:09:18.268353 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 411, in add_access Jul 16 21:09:18.268353 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi self._add_access(project_id) Jul 16 21:09:18.268353 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 404, in _add_access Jul 16 21:09:18.268353 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi self._flavor_add_project(self._context, self.id, project_id) Jul 16 21:09:18.268353 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 397, in _flavor_add_project Jul 16 21:09:18.268353 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return _flavor_add_project(context, flavor_id, project_id) Jul 16 21:09:18.268353 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1010, in wrapper Jul 16 21:09:18.268353 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return fn(*args, **kwargs) Jul 16 21:09:18.268353 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 75, in _flavor_add_project Jul 16 21:09:18.268353 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.FlavorAccessExists(flavor_id=flavor_id, Jul 16 21:09:18.268353 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.FlavorAccessExists: Flavor access already exists for flavor 23 and project 6beaabf2d684478fac58e01a0d3a77fe combination. Jul 16 21:09:18.268353 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:09:18.268353 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:09:18.268353 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:09:18.274421 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:09:18.274421 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:09:18.274421 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:09:18.274421 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:09:18.274421 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:09:18.274421 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:09:18.274421 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:09:18.274421 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 16 21:09:18.274421 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 16 21:09:18.274421 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavor_access.py", line 83, in _add_tenant_access Jul 16 21:09:18.274421 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPConflict(explanation=err.format_message()) Jul 16 21:09:18.274421 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPConflict: Flavor access already exists for flavor 23 and project 6beaabf2d684478fac58e01a0d3a77fe combination. Jul 16 21:09:18.274421 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:09:18.274421 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:09:18.274421 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:09:18.274421 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:09:18.274421 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:09:18.274421 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:09:18.274421 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:09:18.274421 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:09:18.275577 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor access already exists for flavor 23 and project 6beaabf2d684478fac58e01a0d3a77fe combination. Jul 16 21:09:18.275577 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:09:18.275577 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-08ade805-bda3-49b7-babe-3e311bc7982d tempest-FlavorsAccessNegativeTestJSON-1113799613 tempest-FlavorsAccessNegativeTestJSON-1113799613-project-admin] Returning 409 to user: Flavor access already exists for flavor 23 and project 6beaabf2d684478fac58e01a0d3a77fe combination. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:09:18.275577 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-08ade805-bda3-49b7-babe-3e311bc7982d tempest-FlavorsAccessNegativeTestJSON-1113799613 tempest-FlavorsAccessNegativeTestJSON-1113799613-project-admin] 10.222.0.16 "POST /compute/v2.1/flavors/2134620316/action" status: 409 len: 152 microversion: 2.1 time: 0.160084 Jul 16 21:09:18.275577 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 142/276] 10.222.0.16 () {60 vars in 1199 bytes} [Sat Jul 16 21:09:18 2022] POST /compute/v2.1/flavors/2134620316/action => generated 152 bytes in 165 msecs (HTTP/1.1 409) 9 headers in 378 bytes (1 switches on core 0) Jul 16 21:09:18.275577 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-a761f7f8-0273-4fab-9e45-97034246a12c tempest-FlavorsAdminTestJSON-758135383 tempest-FlavorsAdminTestJSON-758135383-project-admin] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:18.275930 ncdv-master-1 devstack@n-api.service[138680]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-flavor-manage:delete": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:09:18.275930 ncdv-master-1 devstack@n-api.service[138680]: warnings.warn(msg) Jul 16 21:09:18.354641 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-a761f7f8-0273-4fab-9e45-97034246a12c tempest-FlavorsAdminTestJSON-758135383 tempest-FlavorsAdminTestJSON-758135383-project-admin] 10.222.0.16 "DELETE /compute/v2.1/flavors/790087394" status: 202 len: 0 microversion: 2.1 time: 0.088121 Jul 16 21:09:18.355690 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 143/277] 10.222.0.16 () {58 vars in 1157 bytes} [Sat Jul 16 21:09:18 2022] DELETE /compute/v2.1/flavors/790087394 => generated 0 bytes in 90 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 21:09:18.363434 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-911aa14d-1dec-44ce-87db-1f15d96ddb48 tempest-FlavorsAccessNegativeTestJSON-1113799613 tempest-FlavorsAccessNegativeTestJSON-1113799613-project-admin] Action: 'action', calling method: >, body: {"removeTenantAccess": {"tenant": "6beaabf2d684478fac58e01a0d3a77fe"}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:09:18.365783 ncdv-master-1 devstack@n-api.service[138680]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-flavor-access:remove_tenant_access": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:09:18.365783 ncdv-master-1 devstack@n-api.service[138680]: warnings.warn(msg) Jul 16 21:09:18.693082 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-911aa14d-1dec-44ce-87db-1f15d96ddb48 tempest-FlavorsAccessNegativeTestJSON-1113799613 tempest-FlavorsAccessNegativeTestJSON-1113799613-project-admin] 10.222.0.16 "POST /compute/v2.1/flavors/2134620316/action" status: 200 len: 21 microversion: 2.1 time: 0.327510 Jul 16 21:09:18.693082 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 144/278] 10.222.0.16 () {60 vars in 1199 bytes} [Sat Jul 16 21:09:18 2022] POST /compute/v2.1/flavors/2134620316/action => generated 21 bytes in 330 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Jul 16 21:09:18.697766 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-e7524134-104d-4b31-bddf-428911f82e9c tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:18.700086 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-e7524134-104d-4b31-bddf-428911f82e9c tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: 1703f309-9e2e-4c5f-8588-eb4e43f21d91] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:09:18.723831 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e7524134-104d-4b31-bddf-428911f82e9c tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:18.730987 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e7524134-104d-4b31-bddf-428911f82e9c tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:18.731961 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e7524134-104d-4b31-bddf-428911f82e9c tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:18.741161 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e7524134-104d-4b31-bddf-428911f82e9c tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:18.847957 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-13eb7c0d-1202-4f95-b376-81b5f25fe837 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ef9cbce2b2e64665b7f50767342dcb46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0ee02e4c8b448f2b77798ae504129b9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:18.872040 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-13eb7c0d-1202-4f95-b376-81b5f25fe837 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] 10.222.0.16 "GET /compute/v2.1/servers/e506a6e0-fbcd-4e53-a6a0-d31761654c92" status: 200 len: 1590 microversion: 2.1 time: 0.878318 Jul 16 21:09:18.892790 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 135/279] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:09:17 2022] GET /compute/v2.1/servers/e506a6e0-fbcd-4e53-a6a0-d31761654c92 => generated 1590 bytes in 901 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:09:18.898388 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-e0b53124-c161-43fd-9068-afc5ea980026 tempest-FlavorsAdminTestJSON-758135383 tempest-FlavorsAdminTestJSON-758135383-project-admin] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:18.922744 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-e0b53124-c161-43fd-9068-afc5ea980026 tempest-FlavorsAdminTestJSON-758135383 tempest-FlavorsAdminTestJSON-758135383-project-admin] 10.222.0.16 "GET /compute/v2.1/flavors/detail" status: 200 len: 5819 microversion: 2.1 time: 0.028374 Jul 16 21:09:18.923513 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 136/280] 10.222.0.16 () {58 vars in 1145 bytes} [Sat Jul 16 21:09:18 2022] GET /compute/v2.1/flavors/detail => generated 5819 bytes in 30 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:09:18.938512 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-28130b83-672b-4b5d-abcc-58c8454bc8f7 tempest-FlavorsAccessNegativeTestJSON-1113799613 tempest-FlavorsAccessNegativeTestJSON-1113799613-project-admin] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:18.939653 ncdv-master-1 devstack@n-api.service[138681]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-flavor-manage:delete": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:09:18.939653 ncdv-master-1 devstack@n-api.service[138681]: warnings.warn(msg) Jul 16 21:09:19.040335 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-28130b83-672b-4b5d-abcc-58c8454bc8f7 tempest-FlavorsAccessNegativeTestJSON-1113799613 tempest-FlavorsAccessNegativeTestJSON-1113799613-project-admin] 10.222.0.16 "DELETE /compute/v2.1/flavors/2134620316" status: 202 len: 0 microversion: 2.1 time: 0.115302 Jul 16 21:09:19.040979 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 137/281] 10.222.0.16 () {58 vars in 1160 bytes} [Sat Jul 16 21:09:18 2022] DELETE /compute/v2.1/flavors/2134620316 => generated 0 bytes in 117 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 21:09:19.052595 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-09a8d313-a161-4814-945e-b03697713a9b tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:19.053311 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-09a8d313-a161-4814-945e-b03697713a9b tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] [instance: 1631f0cc-de67-4c0d-abd0-3e0e11dfb4ed] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:09:19.061031 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-09a8d313-a161-4814-945e-b03697713a9b tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:19.061751 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-09a8d313-a161-4814-945e-b03697713a9b tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:19.064471 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-09a8d313-a161-4814-945e-b03697713a9b tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:19.065715 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-09a8d313-a161-4814-945e-b03697713a9b tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:19.301710 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-e7524134-104d-4b31-bddf-428911f82e9c tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b68dacc4b8463ba07fae6372131117', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55a19b7a4682457a9132f5fc80110712', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:19.311802 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-e7524134-104d-4b31-bddf-428911f82e9c tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "GET /compute/v2.1/servers/1703f309-9e2e-4c5f-8588-eb4e43f21d91" status: 200 len: 1470 microversion: 2.1 time: 0.622142 Jul 16 21:09:19.312384 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 145/282] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:09:18 2022] GET /compute/v2.1/servers/1703f309-9e2e-4c5f-8588-eb4e43f21d91 => generated 1470 bytes in 625 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:09:19.317242 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-975cef46-bd5c-43be-9318-ceb02c610c8e tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:19.317743 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-975cef46-bd5c-43be-9318-ceb02c610c8e tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] [instance: e506a6e0-fbcd-4e53-a6a0-d31761654c92] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:09:19.324954 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-975cef46-bd5c-43be-9318-ceb02c610c8e tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:19.325377 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-975cef46-bd5c-43be-9318-ceb02c610c8e tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:19.326153 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-975cef46-bd5c-43be-9318-ceb02c610c8e tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:19.326594 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-975cef46-bd5c-43be-9318-ceb02c610c8e tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:19.553747 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-09a8d313-a161-4814-945e-b03697713a9b tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0d72626f465b4f37ad3b18bb945d4000', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fc1c3463f9444119af9d18f85bdd703d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:19.564092 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-09a8d313-a161-4814-945e-b03697713a9b tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] 10.222.0.16 "GET /compute/v2.1/servers/1631f0cc-de67-4c0d-abd0-3e0e11dfb4ed" status: 200 len: 1318 microversion: 2.1 time: 0.516359 Jul 16 21:09:19.564653 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-975cef46-bd5c-43be-9318-ceb02c610c8e tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] 10.222.0.16 "GET /compute/v2.1/servers/e506a6e0-fbcd-4e53-a6a0-d31761654c92/os-interface" status: 200 len: 285 microversion: 2.1 time: 0.251101 Jul 16 21:09:19.565554 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 146/283] 10.222.0.16 () {58 vars in 1274 bytes} [Sat Jul 16 21:09:19 2022] GET /compute/v2.1/servers/e506a6e0-fbcd-4e53-a6a0-d31761654c92/os-interface => generated 285 bytes in 252 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:09:19.572015 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-12132cd8-a7b5-4493-8950-a6b3ba5de623 tempest-FlavorsAdminTestJSON-758135383 tempest-FlavorsAdminTestJSON-758135383-project-admin] Action: 'create', calling method: >, body: {"flavor": {"ram": 512, "vcpus": 1, "disk": 10, "name": "tempest-FlavorsAdminTestJSON-flavor-959739463", "id": 870073143, "os-flavor-access:is_public": "True"}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:09:19.573823 ncdv-master-1 devstack@n-api.service[138680]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-flavor-manage:create": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:09:19.573823 ncdv-master-1 devstack@n-api.service[138680]: warnings.warn(msg) Jul 16 21:09:19.576920 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 138/284] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:09:19 2022] GET /compute/v2.1/servers/1631f0cc-de67-4c0d-abd0-3e0e11dfb4ed => generated 1318 bytes in 532 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:09:19.585754 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-7610ac16-b3ce-4b3e-94fa-32da561b3afe tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:19.587055 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-7610ac16-b3ce-4b3e-94fa-32da561b3afe tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] [instance: 148d5ad9-2900-4c2b-909b-aa446757e33d] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:09:19.595694 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7610ac16-b3ce-4b3e-94fa-32da561b3afe tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:19.596439 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7610ac16-b3ce-4b3e-94fa-32da561b3afe tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:19.597425 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7610ac16-b3ce-4b3e-94fa-32da561b3afe tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:19.598055 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7610ac16-b3ce-4b3e-94fa-32da561b3afe tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:19.709456 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-12132cd8-a7b5-4493-8950-a6b3ba5de623 tempest-FlavorsAdminTestJSON-758135383 tempest-FlavorsAdminTestJSON-758135383-project-admin] 10.222.0.16 "POST /compute/v2.1/flavors" status: 200 len: 425 microversion: 2.1 time: 0.142548 Jul 16 21:09:19.709979 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 147/285] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 21:09:19 2022] POST /compute/v2.1/flavors => generated 425 bytes in 144 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:09:19.714592 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-e02f4221-2439-4127-acb8-5bd872a5f033 tempest-FlavorsAccessNegativeTestJSON-1113799613 tempest-FlavorsAccessNegativeTestJSON-1113799613-project-admin] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:19.739500 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-e02f4221-2439-4127-acb8-5bd872a5f033 tempest-FlavorsAccessNegativeTestJSON-1113799613 tempest-FlavorsAccessNegativeTestJSON-1113799613-project-admin] 10.222.0.16 "GET /compute/v2.1/flavors/detail" status: 200 len: 6234 microversion: 2.1 time: 0.028115 Jul 16 21:09:19.741159 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 148/286] 10.222.0.16 () {58 vars in 1145 bytes} [Sat Jul 16 21:09:19 2022] GET /compute/v2.1/flavors/detail => generated 6234 bytes in 30 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:09:19.746042 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-5cf39856-4427-416a-b833-576a15e0749f tempest-FlavorsAdminTestJSON-758135383 tempest-FlavorsAdminTestJSON-758135383-project-admin] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:19.747073 ncdv-master-1 devstack@n-api.service[138680]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-flavor-manage:delete": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:09:19.747073 ncdv-master-1 devstack@n-api.service[138680]: warnings.warn(msg) Jul 16 21:09:19.840118 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-7610ac16-b3ce-4b3e-94fa-32da561b3afe tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'abeb7020019d41a18de42956d706a011', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '649babe54d104436af461f1ee4767001', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:19.851355 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-7610ac16-b3ce-4b3e-94fa-32da561b3afe tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] 10.222.0.16 "GET /compute/v2.1/servers/148d5ad9-2900-4c2b-909b-aa446757e33d" status: 200 len: 1316 microversion: 2.1 time: 0.273244 Jul 16 21:09:19.855799 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 139/287] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:09:19 2022] GET /compute/v2.1/servers/148d5ad9-2900-4c2b-909b-aa446757e33d => generated 1316 bytes in 278 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:09:19.865039 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-faf034de-02f4-4de2-af43-ec3bbfbd53ba tempest-FlavorsAccessNegativeTestJSON-1113799613 tempest-FlavorsAccessNegativeTestJSON-1113799613-project-admin] Action: 'create', calling method: >, body: {"flavor": {"ram": 512, "vcpus": 1, "disk": 10, "name": "tempest-FlavorsAccessNegativeTestJSON-flavor-65432360", "id": 1731160331, "os-flavor-access:is_public": "True"}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:09:19.868193 ncdv-master-1 devstack@n-api.service[138681]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-flavor-manage:create": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:09:19.868193 ncdv-master-1 devstack@n-api.service[138681]: warnings.warn(msg) Jul 16 21:09:19.885986 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-5cf39856-4427-416a-b833-576a15e0749f tempest-FlavorsAdminTestJSON-758135383 tempest-FlavorsAdminTestJSON-758135383-project-admin] 10.222.0.16 "DELETE /compute/v2.1/flavors/870073143" status: 202 len: 0 microversion: 2.1 time: 0.143247 Jul 16 21:09:19.891816 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 149/288] 10.222.0.16 () {58 vars in 1157 bytes} [Sat Jul 16 21:09:19 2022] DELETE /compute/v2.1/flavors/870073143 => generated 0 bytes in 145 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 21:09:19.895919 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-724bd3ab-cf58-42a0-b953-ed71a185c2c4 tempest-FlavorsAdminTestJSON-758135383 tempest-FlavorsAdminTestJSON-758135383-project-admin] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:19.929961 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-724bd3ab-cf58-42a0-b953-ed71a185c2c4 tempest-FlavorsAdminTestJSON-758135383 tempest-FlavorsAdminTestJSON-758135383-project-admin] 10.222.0.16 "GET /compute/v2.1/flavors/detail" status: 200 len: 5819 microversion: 2.1 time: 0.038699 Jul 16 21:09:19.944818 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 150/289] 10.222.0.16 () {58 vars in 1145 bytes} [Sat Jul 16 21:09:19 2022] GET /compute/v2.1/flavors/detail => generated 5819 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:09:19.964678 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-e4b89320-8ec0-40e8-9960-f16bc664d34f tempest-FlavorsAdminTestJSON-758135383 tempest-FlavorsAdminTestJSON-758135383-project-admin] Action: 'create', calling method: >, body: {"flavor": {"ram": 512, "vcpus": 1, "disk": 10, "name": "tempest-FlavorsAdminTestJSON-flavor-842239502", "id": null, "os-flavor-access:is_public": "True"}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:09:19.989429 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-faf034de-02f4-4de2-af43-ec3bbfbd53ba tempest-FlavorsAccessNegativeTestJSON-1113799613 tempest-FlavorsAccessNegativeTestJSON-1113799613-project-admin] 10.222.0.16 "POST /compute/v2.1/flavors" status: 200 len: 436 microversion: 2.1 time: 0.132203 Jul 16 21:09:19.990050 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 140/290] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 21:09:19 2022] POST /compute/v2.1/flavors => generated 436 bytes in 133 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:09:20.001967 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-3d3b68ec-47ff-4aad-aa6d-536373ba7a8b tempest-FlavorsAccessNegativeTestJSON-1113799613 tempest-FlavorsAccessNegativeTestJSON-1113799613-project-admin] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:20.003141 ncdv-master-1 devstack@n-api.service[138681]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-flavor-access": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:09:20.003141 ncdv-master-1 devstack@n-api.service[138681]: warnings.warn(msg) Jul 16 21:09:20.010792 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-3d3b68ec-47ff-4aad-aa6d-536373ba7a8b tempest-FlavorsAccessNegativeTestJSON-1113799613 tempest-FlavorsAccessNegativeTestJSON-1113799613-project-admin] HTTP exception thrown: Access list not available for public flavors. Jul 16 21:09:20.012997 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-3d3b68ec-47ff-4aad-aa6d-536373ba7a8b tempest-FlavorsAccessNegativeTestJSON-1113799613 tempest-FlavorsAccessNegativeTestJSON-1113799613-project-admin] Request method failure captured: Jul 16 21:09:20.012997 ncdv-master-1 devstack@n-api.service[138681]: request: GET /compute/v2.1/flavors/1731160331/os-flavor-access HTTP/1.1 Jul 16 21:09:20.012997 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:09:20.012997 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:09:20.012997 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:09:20.012997 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 21:09:20.012997 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:09:20.012997 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:09:20.012997 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:09:20.012997 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:09:20.012997 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0yj7TqhQMly9bDnDqOBZY1OHAWHNoSytoQcgL-V9TmjaQVAVWqaNvpeVrgxpHpBh8P29OcQ9zLIwJPxFZaJdC9FWMwFzpXueXcVMXCBTEwHMfizgMcOeg2IKk9VuXZnnHMfm4IQdXMisOAsjwoyoxjedCTDXTY7Kwsohcm0agHaBdtk Jul 16 21:09:20.012997 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:09:20.012997 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:09:20.012997 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:09:20.012997 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:09:20.012997 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:09:20.012997 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:09:20.012997 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: 054de79bd580451bafa72a8e903ffab5 Jul 16 21:09:20.012997 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-FlavorsAccessNegativeTestJSON-1113799613 Jul 16 21:09:20.012997 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,admin,reader Jul 16 21:09:20.012997 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,admin,reader Jul 16 21:09:20.014202 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/054de79bd580451bafa72a8e903ffab5"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/054de79bd580451bafa72a8e903ffab5"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_054de79bd580451bafa72a8e903ffab5"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/054de79bd580451bafa72a8e903ffab5"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:09:20.014202 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-FlavorsAccessNegativeTestJSON-1113799613 Jul 16 21:09:20.014202 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: 054de79bd580451bafa72a8e903ffab5 Jul 16 21:09:20.014202 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-FlavorsAccessNegativeTestJSON-1113799613 Jul 16 21:09:20.014202 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-FlavorsAccessNegativeTestJSON-1113799613-project-admin Jul 16 21:09:20.014202 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:09:20.014202 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:09:20.014202 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: 651cbc6596e24fcfa7232ddc273692ae Jul 16 21:09:20.014202 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-FlavorsAccessNegativeTestJSON-1113799613-project-admin Jul 16 21:09:20.014202 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:09:20.014202 ncdv-master-1 devstack@n-api.service[138681]: exception: Access list not available for public flavors. Jul 16 21:09:20.014202 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'flavor_id': '1731160331'} Jul 16 21:09:20.015613 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:09:20.015613 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:09:20.015613 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:09:20.015613 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:09:20.015613 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:09:20.015613 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:09:20.015613 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:09:20.015613 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:09:20.015613 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavor_access.py", line 52, in index Jul 16 21:09:20.015613 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=explanation) Jul 16 21:09:20.015613 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Access list not available for public flavors. Jul 16 21:09:20.015613 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:09:20.015613 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:09:20.015613 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:09:20.015613 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:09:20.015613 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:09:20.015613 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:09:20.015613 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:09:20.015613 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:09:20.015613 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Access list not available for public flavors. Jul 16 21:09:20.015613 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:09:20.019264 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-3d3b68ec-47ff-4aad-aa6d-536373ba7a8b tempest-FlavorsAccessNegativeTestJSON-1113799613 tempest-FlavorsAccessNegativeTestJSON-1113799613-project-admin] Returning 404 to user: Access list not available for public flavors. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:09:20.020693 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-3d3b68ec-47ff-4aad-aa6d-536373ba7a8b tempest-FlavorsAccessNegativeTestJSON-1113799613 tempest-FlavorsAccessNegativeTestJSON-1113799613-project-admin] 10.222.0.16 "GET /compute/v2.1/flavors/1731160331/os-flavor-access" status: 404 len: 91 microversion: 2.1 time: 0.023436 Jul 16 21:09:20.023909 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 141/291] 10.222.0.16 () {58 vars in 1208 bytes} [Sat Jul 16 21:09:19 2022] GET /compute/v2.1/flavors/1731160331/os-flavor-access => generated 91 bytes in 28 msecs (HTTP/1.1 404) 9 headers in 378 bytes (2 switches on core 0) Jul 16 21:09:20.046498 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-96d0dce3-fa6f-4150-9cbb-cc3093549cb0 tempest-FlavorsAccessNegativeTestJSON-1113799613 tempest-FlavorsAccessNegativeTestJSON-1113799613-project-admin] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:20.053232 ncdv-master-1 devstack@n-api.service[138681]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-flavor-manage:delete": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:09:20.053232 ncdv-master-1 devstack@n-api.service[138681]: warnings.warn(msg) Jul 16 21:09:20.123457 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-e4b89320-8ec0-40e8-9960-f16bc664d34f tempest-FlavorsAdminTestJSON-758135383 tempest-FlavorsAdminTestJSON-758135383-project-admin] 10.222.0.16 "POST /compute/v2.1/flavors" status: 200 len: 506 microversion: 2.1 time: 0.161853 Jul 16 21:09:20.131347 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 151/292] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 21:09:19 2022] POST /compute/v2.1/flavors => generated 506 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:09:20.139831 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-ff313cf4-9641-4ef3-9f07-d06c090d818a tempest-FlavorsAdminTestJSON-758135383 tempest-FlavorsAdminTestJSON-758135383-project-admin] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:20.228732 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-96d0dce3-fa6f-4150-9cbb-cc3093549cb0 tempest-FlavorsAccessNegativeTestJSON-1113799613 tempest-FlavorsAccessNegativeTestJSON-1113799613-project-admin] 10.222.0.16 "DELETE /compute/v2.1/flavors/1731160331" status: 202 len: 0 microversion: 2.1 time: 0.194172 Jul 16 21:09:20.229886 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 142/293] 10.222.0.16 () {58 vars in 1160 bytes} [Sat Jul 16 21:09:20 2022] DELETE /compute/v2.1/flavors/1731160331 => generated 0 bytes in 196 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 21:09:20.237496 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-27fdbcd2-3b38-44a2-8031-9753db99d523 tempest-FlavorsAccessNegativeTestJSON-1113799613 tempest-FlavorsAccessNegativeTestJSON-1113799613-project-admin] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:20.262594 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-ff313cf4-9641-4ef3-9f07-d06c090d818a tempest-FlavorsAdminTestJSON-758135383 tempest-FlavorsAdminTestJSON-758135383-project-admin] 10.222.0.16 "DELETE /compute/v2.1/flavors/0e8e18ec-fdbd-49b7-a4a5-fd0665401d82" status: 202 len: 0 microversion: 2.1 time: 0.128894 Jul 16 21:09:20.263226 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 152/294] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:09:20 2022] DELETE /compute/v2.1/flavors/0e8e18ec-fdbd-49b7-a4a5-fd0665401d82 => generated 0 bytes in 130 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 21:09:20.269425 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-27fdbcd2-3b38-44a2-8031-9753db99d523 tempest-FlavorsAccessNegativeTestJSON-1113799613 tempest-FlavorsAccessNegativeTestJSON-1113799613-project-admin] 10.222.0.16 "GET /compute/v2.1/flavors/detail" status: 200 len: 6315 microversion: 2.1 time: 0.035364 Jul 16 21:09:20.270860 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 143/295] 10.222.0.16 () {58 vars in 1145 bytes} [Sat Jul 16 21:09:20 2022] GET /compute/v2.1/flavors/detail => generated 6315 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:09:20.274588 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-32000a8d-15e7-4b08-8b20-937a12e48326 tempest-FlavorsAdminTestJSON-758135383 tempest-FlavorsAdminTestJSON-758135383-project-admin] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:20.292996 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-a0207681-469a-441b-a069-03c62223ab37 tempest-FlavorsAccessNegativeTestJSON-1113799613 tempest-FlavorsAccessNegativeTestJSON-1113799613-project-admin] Action: 'create', calling method: >, body: {"flavor": {"ram": 512, "vcpus": 1, "disk": 10, "name": "tempest-FlavorsAccessNegativeTestJSON-flavor-2090104035", "id": 1490462792, "os-flavor-access:is_public": "False"}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:09:20.316918 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-32000a8d-15e7-4b08-8b20-937a12e48326 tempest-FlavorsAdminTestJSON-758135383 tempest-FlavorsAdminTestJSON-758135383-project-admin] 10.222.0.16 "GET /compute/v2.1/flavors/detail" status: 200 len: 5819 microversion: 2.1 time: 0.046261 Jul 16 21:09:20.318372 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 153/296] 10.222.0.16 () {58 vars in 1145 bytes} [Sat Jul 16 21:09:20 2022] GET /compute/v2.1/flavors/detail => generated 5819 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:09:20.337046 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-92b1b5a7-a581-44f1-a9b7-e43949a539e7 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:20.338064 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-92b1b5a7-a581-44f1-a9b7-e43949a539e7 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: 1703f309-9e2e-4c5f-8588-eb4e43f21d91] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:09:20.346198 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-92b1b5a7-a581-44f1-a9b7-e43949a539e7 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:20.346650 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-92b1b5a7-a581-44f1-a9b7-e43949a539e7 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:20.347481 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-92b1b5a7-a581-44f1-a9b7-e43949a539e7 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:20.348140 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-92b1b5a7-a581-44f1-a9b7-e43949a539e7 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:20.428442 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-a0207681-469a-441b-a069-03c62223ab37 tempest-FlavorsAccessNegativeTestJSON-1113799613 tempest-FlavorsAccessNegativeTestJSON-1113799613-project-admin] 10.222.0.16 "POST /compute/v2.1/flavors" status: 200 len: 439 microversion: 2.1 time: 0.139923 Jul 16 21:09:20.429044 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 144/297] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 21:09:20 2022] POST /compute/v2.1/flavors => generated 439 bytes in 142 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 16 21:09:20.445243 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-adfd5ede-eff3-4e64-a05e-f901ba88b34e tempest-FlavorsAdminTestJSON-758135383 tempest-FlavorsAdminTestJSON-758135383-project-admin] Action: 'create', calling method: >, body: {"flavor": {"ram": 512, "vcpus": 1, "disk": 10, "name": "tempest-FlavorsAdminTestJSON-flavor-1397657933", "id": "596f010d-101c-45fc-8979-c481a1591a21", "os-flavor-access:is_public": "True"}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:09:20.566718 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_api:SELECT=85,nova_api:INSERT=9,nova_api:DELETE=6 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:09:20.573127 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-adfd5ede-eff3-4e64-a05e-f901ba88b34e tempest-FlavorsAdminTestJSON-758135383 tempest-FlavorsAdminTestJSON-758135383-project-admin] 10.222.0.16 "POST /compute/v2.1/flavors" status: 200 len: 507 microversion: 2.1 time: 0.142950 Jul 16 21:09:20.574026 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 145/298] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 21:09:20 2022] POST /compute/v2.1/flavors => generated 507 bytes in 144 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:09:20.775717 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell0:SELECT=4 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:09:20.809910 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-d50415fd-1b1d-4f4c-8b1c-11cd6b33268c tempest-FlavorsAccessNegativeTestJSON-1115678225 tempest-FlavorsAccessNegativeTestJSON-1115678225-project] Action: 'action', calling method: >, body: {"addTenantAccess": {"tenant": "6beaabf2d684478fac58e01a0d3a77fe"}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:09:20.811945 ncdv-master-1 devstack@n-api.service[138681]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-flavor-access:add_tenant_access": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:09:20.811945 ncdv-master-1 devstack@n-api.service[138681]: warnings.warn(msg) Jul 16 21:09:20.813812 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-d50415fd-1b1d-4f4c-8b1c-11cd6b33268c tempest-FlavorsAccessNegativeTestJSON-1115678225 tempest-FlavorsAccessNegativeTestJSON-1115678225-project] Policy check for os_compute_api:os-flavor-access:add_tenant_access failed with credentials {'is_admin': False, 'user_id': 'a0d51dd7cf834374b6a05298707b3ef1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6beaabf2d684478fac58e01a0d3a77fe', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:20.816756 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-d50415fd-1b1d-4f4c-8b1c-11cd6b33268c tempest-FlavorsAccessNegativeTestJSON-1115678225 tempest-FlavorsAccessNegativeTestJSON-1115678225-project] Request method failure captured: Jul 16 21:09:20.816756 ncdv-master-1 devstack@n-api.service[138681]: request: POST /compute/v2.1/flavors/1490462792/action HTTP/1.1 Jul 16 21:09:20.816756 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:09:20.816756 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:09:20.816756 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:09:20.816756 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 67 Jul 16 21:09:20.816756 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:09:20.816756 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:09:20.816756 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:09:20.816756 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:09:20.816756 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0yj6XZ2jYvptYwXXWJ01P3r4kZbdsLl5n626T72fQMATNGAKAOQHFLICs63eDQtSDP4Vv8vl200BYAEM2DLtg20ERbzUkDBRessW1d6-hoOQsGan0OBbNYoYDgoM3n0DCoixEa4cnCGRZGgIe-mD6dQ8chdAPzZ-LGafZOAimHWKYKk Jul 16 21:09:20.816756 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:09:20.816756 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:09:20.816756 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:09:20.816756 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:09:20.816756 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:09:20.816756 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:09:20.816756 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: 6beaabf2d684478fac58e01a0d3a77fe Jul 16 21:09:20.816756 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-FlavorsAccessNegativeTestJSON-1115678225 Jul 16 21:09:20.816756 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:09:20.816756 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:09:20.818151 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/6beaabf2d684478fac58e01a0d3a77fe"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/6beaabf2d684478fac58e01a0d3a77fe"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_6beaabf2d684478fac58e01a0d3a77fe"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/6beaabf2d684478fac58e01a0d3a77fe"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:09:20.818151 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-FlavorsAccessNegativeTestJSON-1115678225 Jul 16 21:09:20.818151 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: 6beaabf2d684478fac58e01a0d3a77fe Jul 16 21:09:20.818151 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-FlavorsAccessNegativeTestJSON-1115678225 Jul 16 21:09:20.818151 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-FlavorsAccessNegativeTestJSON-1115678225-project Jul 16 21:09:20.818151 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:09:20.818151 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:09:20.818151 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: a0d51dd7cf834374b6a05298707b3ef1 Jul 16 21:09:20.818151 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-FlavorsAccessNegativeTestJSON-1115678225-project Jul 16 21:09:20.818151 ncdv-master-1 devstack@n-api.service[138681]: Jul 16 21:09:20.818151 ncdv-master-1 devstack@n-api.service[138681]: {"addTenantAccess": {"tenant": "6beaabf2d684478fac58e01a0d3a77fe"}} Jul 16 21:09:20.818151 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:09:20.818805 ncdv-master-1 devstack@n-api.service[138681]: exception: Policy doesn't allow os_compute_api:os-flavor-access:add_tenant_access to be performed. Jul 16 21:09:20.818805 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': '1490462792', 'body': {'addTenantAccess': {'tenant': '6beaabf2d684478fac58e01a0d3a77fe'}}} Jul 16 21:09:20.818805 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:09:20.818805 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:09:20.818805 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:09:20.818805 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:09:20.818805 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:09:20.818805 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:09:20.818805 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:09:20.818805 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:09:20.818805 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 16 21:09:20.818805 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 16 21:09:20.818805 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavor_access.py", line 66, in _add_tenant_access Jul 16 21:09:20.818805 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi context.can(fa_policies.POLICY_ROOT % "add_tenant_access", target={}) Jul 16 21:09:20.818805 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/context.py", line 240, in can Jul 16 21:09:20.818805 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return policy.authorize(self, action, target) Jul 16 21:09:20.818805 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 199, in authorize Jul 16 21:09:20.818805 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi LOG.debug('Policy check for %(action)s failed with credentials ' Jul 16 21:09:20.818805 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 16 21:09:20.819876 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi self.force_reraise() Jul 16 21:09:20.819876 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 16 21:09:20.819876 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise self.value Jul 16 21:09:20.819876 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 186, in authorize Jul 16 21:09:20.819876 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi result = _ENFORCER.authorize(action, target, context, Jul 16 21:09:20.819876 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1184, in authorize Jul 16 21:09:20.819876 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return self.enforce( Jul 16 21:09:20.819876 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1088, in enforce Jul 16 21:09:20.819876 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc(*args, **kwargs) Jul 16 21:09:20.819876 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.PolicyNotAuthorized: Policy doesn't allow os_compute_api:os-flavor-access:add_tenant_access to be performed. Jul 16 21:09:20.819876 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:09:20.819876 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:09:20.819876 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:09:20.819876 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:09:20.819876 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:09:20.819876 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:09:20.819876 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 354, in __exit__ Jul 16 21:09:20.819876 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(webob.exc.HTTPForbidden( Jul 16 21:09:20.819876 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Policy doesn't allow os_compute_api:os-flavor-access:add_tenant_access to be performed. Jul 16 21:09:20.819876 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:09:20.821065 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-d50415fd-1b1d-4f4c-8b1c-11cd6b33268c tempest-FlavorsAccessNegativeTestJSON-1115678225 tempest-FlavorsAccessNegativeTestJSON-1115678225-project] Returning 403 to user: Policy doesn't allow os_compute_api:os-flavor-access:add_tenant_access to be performed. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:09:20.821854 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-d50415fd-1b1d-4f4c-8b1c-11cd6b33268c tempest-FlavorsAccessNegativeTestJSON-1115678225 tempest-FlavorsAccessNegativeTestJSON-1115678225-project] 10.222.0.16 "POST /compute/v2.1/flavors/1490462792/action" status: 403 len: 130 microversion: 2.1 time: 0.246180 Jul 16 21:09:20.831698 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 146/299] 10.222.0.16 () {60 vars in 1199 bytes} [Sat Jul 16 21:09:20 2022] POST /compute/v2.1/flavors/1490462792/action => generated 130 bytes in 257 msecs (HTTP/1.1 403) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:09:20.837246 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-3a1f55cc-b4a5-4317-8fa6-e50e4e9bf6ae req-0af7e1c6-6520-4bfa-9d39-691afe6354ad service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "148d5ad9-2900-4c2b-909b-aa446757e33d", "tag": "7ee252fe-6a69-4f90-9053-a7eabcf7750c"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:09:20.847996 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-3a1f55cc-b4a5-4317-8fa6-e50e4e9bf6ae req-0af7e1c6-6520-4bfa-9d39-691afe6354ad service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:20.848826 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-3a1f55cc-b4a5-4317-8fa6-e50e4e9bf6ae req-0af7e1c6-6520-4bfa-9d39-691afe6354ad service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:20.873182 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-3a1f55cc-b4a5-4317-8fa6-e50e4e9bf6ae req-0af7e1c6-6520-4bfa-9d39-691afe6354ad service nova] Creating event network-changed:7ee252fe-6a69-4f90-9053-a7eabcf7750c for instance 148d5ad9-2900-4c2b-909b-aa446757e33d on nch2-master-1 Jul 16 21:09:20.880392 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-3a1f55cc-b4a5-4317-8fa6-e50e4e9bf6ae req-0af7e1c6-6520-4bfa-9d39-691afe6354ad service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.046910 Jul 16 21:09:20.881640 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 147/300] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:09:20 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:09:20.887246 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-ea8fc55b-36a3-4b19-8e4f-a73062312bcc tempest-FlavorsAdminTestJSON-758135383 tempest-FlavorsAdminTestJSON-758135383-project-admin] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:20.888487 ncdv-master-1 devstack@n-api.service[138681]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-flavor-manage:delete": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:09:20.888487 ncdv-master-1 devstack@n-api.service[138681]: warnings.warn(msg) Jul 16 21:09:20.982904 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-ea8fc55b-36a3-4b19-8e4f-a73062312bcc tempest-FlavorsAdminTestJSON-758135383 tempest-FlavorsAdminTestJSON-758135383-project-admin] 10.222.0.16 "DELETE /compute/v2.1/flavors/596f010d-101c-45fc-8979-c481a1591a21" status: 202 len: 0 microversion: 2.1 time: 0.099739 Jul 16 21:09:20.988787 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 148/301] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:09:20 2022] DELETE /compute/v2.1/flavors/596f010d-101c-45fc-8979-c481a1591a21 => generated 0 bytes in 101 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 21:09:20.994990 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-b01a8ed6-bd00-4183-a394-846b468bb74c tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:20.996373 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-b01a8ed6-bd00-4183-a394-846b468bb74c tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] [instance: 1631f0cc-de67-4c0d-abd0-3e0e11dfb4ed] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:09:21.017524 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b01a8ed6-bd00-4183-a394-846b468bb74c tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:21.017524 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b01a8ed6-bd00-4183-a394-846b468bb74c tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:21.017524 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b01a8ed6-bd00-4183-a394-846b468bb74c tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:21.017524 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b01a8ed6-bd00-4183-a394-846b468bb74c tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:21.082047 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-92b1b5a7-a581-44f1-a9b7-e43949a539e7 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b68dacc4b8463ba07fae6372131117', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55a19b7a4682457a9132f5fc80110712', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:21.090731 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-92b1b5a7-a581-44f1-a9b7-e43949a539e7 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "GET /compute/v2.1/servers/1703f309-9e2e-4c5f-8588-eb4e43f21d91" status: 200 len: 1475 microversion: 2.1 time: 0.759579 Jul 16 21:09:21.091469 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 154/302] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:09:20 2022] GET /compute/v2.1/servers/1703f309-9e2e-4c5f-8588-eb4e43f21d91 => generated 1475 bytes in 761 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:09:21.097308 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-7e72be1f-ea2e-4157-a60a-b8fc2825204e tempest-FlavorsAccessNegativeTestJSON-1113799613 tempest-FlavorsAccessNegativeTestJSON-1113799613-project-admin] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:21.098421 ncdv-master-1 devstack@n-api.service[138680]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-flavor-manage:delete": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:09:21.098421 ncdv-master-1 devstack@n-api.service[138680]: warnings.warn(msg) Jul 16 21:09:21.235637 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-7e72be1f-ea2e-4157-a60a-b8fc2825204e tempest-FlavorsAccessNegativeTestJSON-1113799613 tempest-FlavorsAccessNegativeTestJSON-1113799613-project-admin] 10.222.0.16 "DELETE /compute/v2.1/flavors/1490462792" status: 202 len: 0 microversion: 2.1 time: 0.142356 Jul 16 21:09:21.236800 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 155/303] 10.222.0.16 () {58 vars in 1160 bytes} [Sat Jul 16 21:09:21 2022] DELETE /compute/v2.1/flavors/1490462792 => generated 0 bytes in 144 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 21:09:21.245061 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-d7afb695-c8a1-43c6-9639-3558be0eed25 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:21.245849 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-d7afb695-c8a1-43c6-9639-3558be0eed25 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] [instance: 148d5ad9-2900-4c2b-909b-aa446757e33d] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:09:21.254880 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d7afb695-c8a1-43c6-9639-3558be0eed25 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:21.255901 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d7afb695-c8a1-43c6-9639-3558be0eed25 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:21.257046 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d7afb695-c8a1-43c6-9639-3558be0eed25 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:21.257581 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d7afb695-c8a1-43c6-9639-3558be0eed25 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:21.416450 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell0:SELECT=14 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:09:21.471375 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-b01a8ed6-bd00-4183-a394-846b468bb74c tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0d72626f465b4f37ad3b18bb945d4000', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fc1c3463f9444119af9d18f85bdd703d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:21.484208 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-b01a8ed6-bd00-4183-a394-846b468bb74c tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] 10.222.0.16 "GET /compute/v2.1/servers/1631f0cc-de67-4c0d-abd0-3e0e11dfb4ed" status: 200 len: 1318 microversion: 2.1 time: 0.498471 Jul 16 21:09:21.485242 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 149/304] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:09:20 2022] GET /compute/v2.1/servers/1631f0cc-de67-4c0d-abd0-3e0e11dfb4ed => generated 1318 bytes in 501 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:09:21.497895 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-1cea35d5-e4bf-470b-b726-20802c44897f tempest-FlavorsAdminTestJSON-758135383 tempest-FlavorsAdminTestJSON-758135383-project-admin] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:21.537602 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-1cea35d5-e4bf-470b-b726-20802c44897f tempest-FlavorsAdminTestJSON-758135383 tempest-FlavorsAdminTestJSON-758135383-project-admin] 10.222.0.16 "GET /compute/v2.1/flavors/detail" status: 200 len: 5819 microversion: 2.1 time: 0.050591 Jul 16 21:09:21.539169 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 150/305] 10.222.0.16 () {58 vars in 1145 bytes} [Sat Jul 16 21:09:21 2022] GET /compute/v2.1/flavors/detail => generated 5819 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:09:21.558986 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-6c0f547e-adf0-4fa8-ab74-567c7afb3b64 tempest-FlavorsAccessNegativeTestJSON-1113799613 tempest-FlavorsAccessNegativeTestJSON-1113799613-project-admin] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:21.589513 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-6c0f547e-adf0-4fa8-ab74-567c7afb3b64 tempest-FlavorsAccessNegativeTestJSON-1113799613 tempest-FlavorsAccessNegativeTestJSON-1113799613-project-admin] 10.222.0.16 "GET /compute/v2.1/flavors/detail" status: 200 len: 5819 microversion: 2.1 time: 0.048779 Jul 16 21:09:21.590494 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 151/306] 10.222.0.16 () {58 vars in 1145 bytes} [Sat Jul 16 21:09:21 2022] GET /compute/v2.1/flavors/detail => generated 5819 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:09:21.597049 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-ff0520b0-9f7d-4b5d-b43d-4e07201a3b0b tempest-FlavorsAdminTestJSON-758135383 tempest-FlavorsAdminTestJSON-758135383-project-admin] Action: 'create', calling method: >, body: {"flavor": {"ram": 512, "vcpus": 1, "disk": 10, "name": "tempest-test_flavor_-776298514", "id": 1462132276, "os-flavor-access:is_public": "True"}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:09:21.598923 ncdv-master-1 devstack@n-api.service[138681]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-flavor-manage:create": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:09:21.598923 ncdv-master-1 devstack@n-api.service[138681]: warnings.warn(msg) Jul 16 21:09:21.712059 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-d7afb695-c8a1-43c6-9639-3558be0eed25 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'abeb7020019d41a18de42956d706a011', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '649babe54d104436af461f1ee4767001', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:21.720950 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-d7afb695-c8a1-43c6-9639-3558be0eed25 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] 10.222.0.16 "GET /compute/v2.1/servers/148d5ad9-2900-4c2b-909b-aa446757e33d" status: 200 len: 1316 microversion: 2.1 time: 0.482579 Jul 16 21:09:21.721857 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 156/307] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:09:21 2022] GET /compute/v2.1/servers/148d5ad9-2900-4c2b-909b-aa446757e33d => generated 1316 bytes in 485 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:09:21.727042 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-a6e33a6f-fcb3-4bf3-837e-75cb21605e32 tempest-FlavorsAccessNegativeTestJSON-1113799613 tempest-FlavorsAccessNegativeTestJSON-1113799613-project-admin] Action: 'create', calling method: >, body: {"flavor": {"ram": 512, "vcpus": 1, "disk": 10, "name": "tempest-FlavorsAccessNegativeTestJSON-flavor-1211230833", "id": 1952317055, "os-flavor-access:is_public": "False"}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:09:21.729147 ncdv-master-1 devstack@n-api.service[138680]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-flavor-manage:create": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:09:21.729147 ncdv-master-1 devstack@n-api.service[138680]: warnings.warn(msg) Jul 16 21:09:21.740172 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-ff0520b0-9f7d-4b5d-b43d-4e07201a3b0b tempest-FlavorsAdminTestJSON-758135383 tempest-FlavorsAdminTestJSON-758135383-project-admin] 10.222.0.16 "POST /compute/v2.1/flavors" status: 200 len: 413 microversion: 2.1 time: 0.148074 Jul 16 21:09:21.742951 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 152/308] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 21:09:21 2022] POST /compute/v2.1/flavors => generated 413 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:09:21.774245 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-06ceb14b-bb98-47ed-bdfb-007c23d16a7c tempest-FlavorsAdminTestJSON-758135383 tempest-FlavorsAdminTestJSON-758135383-project-admin] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:21.783329 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-06ceb14b-bb98-47ed-bdfb-007c23d16a7c tempest-FlavorsAdminTestJSON-758135383 tempest-FlavorsAdminTestJSON-758135383-project-admin] 10.222.0.16 "GET /compute/v2.1/flavors/1462132276" status: 200 len: 413 microversion: 2.1 time: 0.017927 Jul 16 21:09:21.802887 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 153/309] 10.222.0.16 () {58 vars in 1157 bytes} [Sat Jul 16 21:09:21 2022] GET /compute/v2.1/flavors/1462132276 => generated 413 bytes in 29 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:09:21.874077 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-a6e33a6f-fcb3-4bf3-837e-75cb21605e32 tempest-FlavorsAccessNegativeTestJSON-1113799613 tempest-FlavorsAccessNegativeTestJSON-1113799613-project-admin] 10.222.0.16 "POST /compute/v2.1/flavors" status: 200 len: 439 microversion: 2.1 time: 0.150809 Jul 16 21:09:21.874780 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 157/310] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 21:09:21 2022] POST /compute/v2.1/flavors => generated 439 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:09:21.892764 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-80cd4ad2-5955-47e2-ac28-973445dcb0bb tempest-FlavorsAccessNegativeTestJSON-1113799613 tempest-FlavorsAccessNegativeTestJSON-1113799613-project-admin] Action: 'action', calling method: >, body: {"addTenantAccess": {"tenant": "6beaabf2d684478fac58e01a0d3a77fe"}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:09:21.900489 ncdv-master-1 devstack@n-api.service[138680]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-flavor-access:add_tenant_access": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:09:21.900489 ncdv-master-1 devstack@n-api.service[138680]: warnings.warn(msg) Jul 16 21:09:21.967675 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-c7c72c33-670e-49da-87a7-2e4607296158 tempest-FlavorsAdminTestJSON-20925527 tempest-FlavorsAdminTestJSON-20925527-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:22.002946 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-c7c72c33-670e-49da-87a7-2e4607296158 tempest-FlavorsAdminTestJSON-20925527 tempest-FlavorsAdminTestJSON-20925527-project] 10.222.0.16 "GET /compute/v2.1/flavors/detail" status: 200 len: 6222 microversion: 2.1 time: 0.209054 Jul 16 21:09:22.003992 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 154/311] 10.222.0.16 () {58 vars in 1145 bytes} [Sat Jul 16 21:09:21 2022] GET /compute/v2.1/flavors/detail => generated 6222 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:09:22.022364 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-b6cbd457-08f4-45ee-824a-46dc6668f584 tempest-FlavorsAdminTestJSON-758135383 tempest-FlavorsAdminTestJSON-758135383-project-admin] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:22.023921 ncdv-master-1 devstack@n-api.service[138681]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-flavor-manage:delete": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:09:22.023921 ncdv-master-1 devstack@n-api.service[138681]: warnings.warn(msg) Jul 16 21:09:22.074176 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_api:SELECT=75,nova_api:INSERT=9,nova_api:DELETE=16 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:09:22.112659 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-b6cbd457-08f4-45ee-824a-46dc6668f584 tempest-FlavorsAdminTestJSON-758135383 tempest-FlavorsAdminTestJSON-758135383-project-admin] 10.222.0.16 "DELETE /compute/v2.1/flavors/1462132276" status: 202 len: 0 microversion: 2.1 time: 0.094914 Jul 16 21:09:22.113926 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 155/312] 10.222.0.16 () {58 vars in 1160 bytes} [Sat Jul 16 21:09:22 2022] DELETE /compute/v2.1/flavors/1462132276 => generated 0 bytes in 97 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 21:09:22.120531 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-dc2c1ee4-951d-4d6a-8641-877e9cbf9c25 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:22.121248 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-dc2c1ee4-951d-4d6a-8641-877e9cbf9c25 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: 1703f309-9e2e-4c5f-8588-eb4e43f21d91] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:09:22.129092 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-dc2c1ee4-951d-4d6a-8641-877e9cbf9c25 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:22.129644 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-dc2c1ee4-951d-4d6a-8641-877e9cbf9c25 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:22.130523 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-dc2c1ee4-951d-4d6a-8641-877e9cbf9c25 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:22.131046 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-dc2c1ee4-951d-4d6a-8641-877e9cbf9c25 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:22.190643 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-80cd4ad2-5955-47e2-ac28-973445dcb0bb tempest-FlavorsAccessNegativeTestJSON-1113799613 tempest-FlavorsAccessNegativeTestJSON-1113799613-project-admin] 10.222.0.16 "POST /compute/v2.1/flavors/1952317055/action" status: 200 len: 97 microversion: 2.1 time: 0.302291 Jul 16 21:09:22.197741 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 158/313] 10.222.0.16 () {60 vars in 1199 bytes} [Sat Jul 16 21:09:21 2022] POST /compute/v2.1/flavors/1952317055/action => generated 97 bytes in 310 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Jul 16 21:09:22.204682 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-6e54b5a0-1c47-4d5b-8a51-b87b82233e7b tempest-FlavorsAdminTestJSON-758135383 tempest-FlavorsAdminTestJSON-758135383-project-admin] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:22.240102 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-6e54b5a0-1c47-4d5b-8a51-b87b82233e7b tempest-FlavorsAdminTestJSON-758135383 tempest-FlavorsAdminTestJSON-758135383-project-admin] 10.222.0.16 "GET /compute/v2.1/flavors/detail" status: 200 len: 5819 microversion: 2.1 time: 0.040719 Jul 16 21:09:22.241702 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 159/314] 10.222.0.16 () {58 vars in 1145 bytes} [Sat Jul 16 21:09:22 2022] GET /compute/v2.1/flavors/detail => generated 5819 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:09:22.261302 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-dfca74a0-bb82-4822-b7f2-b790c96cc9bd tempest-FlavorsAccessNegativeTestJSON-1115678225 tempest-FlavorsAccessNegativeTestJSON-1115678225-project] Action: 'action', calling method: >, body: {"removeTenantAccess": {"tenant": "6beaabf2d684478fac58e01a0d3a77fe"}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:09:22.262840 ncdv-master-1 devstack@n-api.service[138680]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-flavor-access:remove_tenant_access": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:09:22.262840 ncdv-master-1 devstack@n-api.service[138680]: warnings.warn(msg) Jul 16 21:09:22.267401 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-dfca74a0-bb82-4822-b7f2-b790c96cc9bd tempest-FlavorsAccessNegativeTestJSON-1115678225 tempest-FlavorsAccessNegativeTestJSON-1115678225-project] Policy check for os_compute_api:os-flavor-access:remove_tenant_access failed with credentials {'is_admin': False, 'user_id': 'a0d51dd7cf834374b6a05298707b3ef1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6beaabf2d684478fac58e01a0d3a77fe', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:22.267401 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-dfca74a0-bb82-4822-b7f2-b790c96cc9bd tempest-FlavorsAccessNegativeTestJSON-1115678225 tempest-FlavorsAccessNegativeTestJSON-1115678225-project] Request method failure captured: Jul 16 21:09:22.267401 ncdv-master-1 devstack@n-api.service[138680]: request: POST /compute/v2.1/flavors/1952317055/action HTTP/1.1 Jul 16 21:09:22.267401 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:09:22.267401 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:09:22.267401 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:09:22.267401 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 70 Jul 16 21:09:22.267401 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:09:22.267401 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:09:22.267401 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:09:22.267401 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:09:22.267401 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0yj6XZ2jYvptYwXXWJ01P3r4kZbdsLl5n626T72fQMATNGAKAOQHFLICs63eDQtSDP4Vv8vl200BYAEM2DLtg20ERbzUkDBRessW1d6-hoOQsGan0OBbNYoYDgoM3n0DCoixEa4cnCGRZGgIe-mD6dQ8chdAPzZ-LGafZOAimHWKYKk Jul 16 21:09:22.267401 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:09:22.267401 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:09:22.267401 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:09:22.267401 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:09:22.267401 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:09:22.267401 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:09:22.267401 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: 6beaabf2d684478fac58e01a0d3a77fe Jul 16 21:09:22.267401 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-FlavorsAccessNegativeTestJSON-1115678225 Jul 16 21:09:22.267401 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:09:22.267401 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:09:22.268975 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/6beaabf2d684478fac58e01a0d3a77fe"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/6beaabf2d684478fac58e01a0d3a77fe"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_6beaabf2d684478fac58e01a0d3a77fe"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/6beaabf2d684478fac58e01a0d3a77fe"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:09:22.268975 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-FlavorsAccessNegativeTestJSON-1115678225 Jul 16 21:09:22.268975 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: 6beaabf2d684478fac58e01a0d3a77fe Jul 16 21:09:22.268975 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-FlavorsAccessNegativeTestJSON-1115678225 Jul 16 21:09:22.268975 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-FlavorsAccessNegativeTestJSON-1115678225-project Jul 16 21:09:22.268975 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:09:22.268975 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:09:22.268975 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: a0d51dd7cf834374b6a05298707b3ef1 Jul 16 21:09:22.268975 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-FlavorsAccessNegativeTestJSON-1115678225-project Jul 16 21:09:22.268975 ncdv-master-1 devstack@n-api.service[138680]: Jul 16 21:09:22.268975 ncdv-master-1 devstack@n-api.service[138680]: {"removeTenantAccess": {"tenant": "6beaabf2d684478fac58e01a0d3a77fe"}} Jul 16 21:09:22.268975 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 21:09:22.269753 ncdv-master-1 devstack@n-api.service[138680]: exception: Policy doesn't allow os_compute_api:os-flavor-access:remove_tenant_access to be performed. Jul 16 21:09:22.269753 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': '1952317055', 'body': {'removeTenantAccess': {'tenant': '6beaabf2d684478fac58e01a0d3a77fe'}}} Jul 16 21:09:22.269753 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:09:22.269753 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:09:22.269753 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:09:22.269753 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:09:22.269753 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:09:22.269753 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:09:22.269753 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:09:22.269753 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:09:22.269753 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 16 21:09:22.269753 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 16 21:09:22.269753 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavor_access.py", line 91, in _remove_tenant_access Jul 16 21:09:22.269753 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi context.can( Jul 16 21:09:22.269753 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/context.py", line 240, in can Jul 16 21:09:22.269753 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return policy.authorize(self, action, target) Jul 16 21:09:22.269753 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 199, in authorize Jul 16 21:09:22.269753 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi LOG.debug('Policy check for %(action)s failed with credentials ' Jul 16 21:09:22.269753 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 16 21:09:22.269753 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi self.force_reraise() Jul 16 21:09:22.271423 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 16 21:09:22.271423 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise self.value Jul 16 21:09:22.271423 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 186, in authorize Jul 16 21:09:22.271423 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi result = _ENFORCER.authorize(action, target, context, Jul 16 21:09:22.271423 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1184, in authorize Jul 16 21:09:22.271423 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return self.enforce( Jul 16 21:09:22.271423 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1088, in enforce Jul 16 21:09:22.271423 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc(*args, **kwargs) Jul 16 21:09:22.271423 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.PolicyNotAuthorized: Policy doesn't allow os_compute_api:os-flavor-access:remove_tenant_access to be performed. Jul 16 21:09:22.271423 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:09:22.271423 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:09:22.271423 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:09:22.271423 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:09:22.271423 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:09:22.271423 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:09:22.271423 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 354, in __exit__ Jul 16 21:09:22.271423 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(webob.exc.HTTPForbidden( Jul 16 21:09:22.271423 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Policy doesn't allow os_compute_api:os-flavor-access:remove_tenant_access to be performed. Jul 16 21:09:22.271423 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:09:22.272642 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-dfca74a0-bb82-4822-b7f2-b790c96cc9bd tempest-FlavorsAccessNegativeTestJSON-1115678225 tempest-FlavorsAccessNegativeTestJSON-1115678225-project] Returning 403 to user: Policy doesn't allow os_compute_api:os-flavor-access:remove_tenant_access to be performed. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:09:22.272642 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-dfca74a0-bb82-4822-b7f2-b790c96cc9bd tempest-FlavorsAccessNegativeTestJSON-1115678225 tempest-FlavorsAccessNegativeTestJSON-1115678225-project] 10.222.0.16 "POST /compute/v2.1/flavors/1952317055/action" status: 403 len: 133 microversion: 2.1 time: 0.024454 Jul 16 21:09:22.273096 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 160/315] 10.222.0.16 () {60 vars in 1199 bytes} [Sat Jul 16 21:09:22 2022] POST /compute/v2.1/flavors/1952317055/action => generated 133 bytes in 31 msecs (HTTP/1.1 403) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:09:22.284368 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-572b0c68-c05d-4a61-b63a-9e9436d7d93f tempest-FlavorsAdminTestJSON-758135383 tempest-FlavorsAdminTestJSON-758135383-project-admin] Action: 'create', calling method: >, body: {"flavor": {"ram": 512, "vcpus": 1, "disk": 10, "name": "tempest-FlavorsAdminTestJSON-flavor-2075263616", "id": 628570885, "os-flavor-access:is_public": "False"}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:09:22.347268 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-dc2c1ee4-951d-4d6a-8641-877e9cbf9c25 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b68dacc4b8463ba07fae6372131117', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55a19b7a4682457a9132f5fc80110712', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:22.355198 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-dc2c1ee4-951d-4d6a-8641-877e9cbf9c25 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "GET /compute/v2.1/servers/1703f309-9e2e-4c5f-8588-eb4e43f21d91" status: 200 len: 1475 microversion: 2.1 time: 0.240054 Jul 16 21:09:22.359087 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 156/316] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:09:22 2022] GET /compute/v2.1/servers/1703f309-9e2e-4c5f-8588-eb4e43f21d91 => generated 1475 bytes in 244 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:09:22.369263 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-e7941793-4176-45af-a488-f3252f611a84 tempest-FlavorsAccessNegativeTestJSON-1113799613 tempest-FlavorsAccessNegativeTestJSON-1113799613-project-admin] Action: 'action', calling method: >, body: {"removeTenantAccess": {"tenant": "6beaabf2d684478fac58e01a0d3a77fe"}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:09:22.373697 ncdv-master-1 devstack@n-api.service[138681]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-flavor-access:remove_tenant_access": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:09:22.373697 ncdv-master-1 devstack@n-api.service[138681]: warnings.warn(msg) Jul 16 21:09:22.403125 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-572b0c68-c05d-4a61-b63a-9e9436d7d93f tempest-FlavorsAdminTestJSON-758135383 tempest-FlavorsAdminTestJSON-758135383-project-admin] 10.222.0.16 "POST /compute/v2.1/flavors" status: 200 len: 427 microversion: 2.1 time: 0.124148 Jul 16 21:09:22.403834 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 161/317] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 21:09:22 2022] POST /compute/v2.1/flavors => generated 427 bytes in 126 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:09:22.420087 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-3eb6729a-bdbb-4691-8536-2b2daf7df35f tempest-FlavorsAdminTestJSON-20925527 tempest-FlavorsAdminTestJSON-20925527-project] Action: 'create', calling method: >, body: {"server": {"name": "test", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "628570885"}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:09:22.467171 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-3eb6729a-bdbb-4691-8536-2b2daf7df35f tempest-FlavorsAdminTestJSON-20925527 tempest-FlavorsAdminTestJSON-20925527-project] HTTP exception thrown: Flavor 628570885 could not be found. Jul 16 21:09:22.477098 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-3eb6729a-bdbb-4691-8536-2b2daf7df35f tempest-FlavorsAdminTestJSON-20925527 tempest-FlavorsAdminTestJSON-20925527-project] Request method failure captured: Jul 16 21:09:22.477098 ncdv-master-1 devstack@n-api.service[138680]: request: POST /compute/v2.1/servers HTTP/1.1 Jul 16 21:09:22.477098 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:09:22.477098 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:09:22.477098 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:09:22.477098 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 106 Jul 16 21:09:22.477098 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:09:22.477098 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:09:22.477098 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:09:22.477098 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:09:22.477098 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0yj6kubyVlZQa_xH5rgL2ds0C_4fLRztdTUZtUVD5Tq1M5GIU4iGY-PsocPudvaGq9N9lKOoQlQMb_Gah4ekUMWfUIahtrb83xvgoqHu9J0vC5YdD9Af2kqZ2P8ayyLb2VmN-3WmxgyuIa1otS-94WIEF_K9QB357qvM7_Fyqj9V1jE Jul 16 21:09:22.477098 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:09:22.477098 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:09:22.477098 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:09:22.477098 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:09:22.477098 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:09:22.477098 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:09:22.477098 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: f792890ae08b44fd90d0eed3a7a9877f Jul 16 21:09:22.477098 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-FlavorsAdminTestJSON-20925527 Jul 16 21:09:22.477098 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:09:22.477098 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:09:22.478543 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/f792890ae08b44fd90d0eed3a7a9877f"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/f792890ae08b44fd90d0eed3a7a9877f"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_f792890ae08b44fd90d0eed3a7a9877f"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/f792890ae08b44fd90d0eed3a7a9877f"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:09:22.478543 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-FlavorsAdminTestJSON-20925527 Jul 16 21:09:22.478543 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: f792890ae08b44fd90d0eed3a7a9877f Jul 16 21:09:22.478543 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-FlavorsAdminTestJSON-20925527 Jul 16 21:09:22.478543 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-FlavorsAdminTestJSON-20925527-project Jul 16 21:09:22.478543 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:09:22.478543 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:09:22.478543 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: 9b1dca63a7a94fe8b40a3967f7a293e1 Jul 16 21:09:22.478543 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-FlavorsAdminTestJSON-20925527-project Jul 16 21:09:22.478543 ncdv-master-1 devstack@n-api.service[138680]: Jul 16 21:09:22.478543 ncdv-master-1 devstack@n-api.service[138680]: {"server": {"name": "test", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "628570885"}} Jul 16 21:09:22.478543 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 21:09:22.478543 ncdv-master-1 devstack@n-api.service[138680]: exception: Flavor 628570885 could not be found. Jul 16 21:09:22.479593 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'body': {'server': {'name': 'test', 'imageRef': '1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9', 'flavorRef': '628570885'}}} Jul 16 21:09:22.479593 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:09:22.479593 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:09:22.479593 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 778, in create Jul 16 21:09:22.479593 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id( Jul 16 21:09:22.479593 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Jul 16 21:09:22.479593 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Jul 16 21:09:22.479593 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Jul 16 21:09:22.479593 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Jul 16 21:09:22.479593 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Jul 16 21:09:22.479593 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Jul 16 21:09:22.479593 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Jul 16 21:09:22.479593 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:09:22.479593 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Jul 16 21:09:22.479593 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Jul 16 21:09:22.479593 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 628570885 could not be found. Jul 16 21:09:22.479593 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:09:22.479593 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:09:22.479593 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:09:22.479593 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:09:22.481246 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:09:22.481246 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:09:22.481246 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:09:22.481246 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:09:22.481246 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:09:22.481246 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:09:22.481246 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 16 21:09:22.481246 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 16 21:09:22.481246 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 16 21:09:22.481246 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 16 21:09:22.481246 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 16 21:09:22.481246 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 16 21:09:22.481246 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi [Previous line repeated 10 more times] Jul 16 21:09:22.481246 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 866, in create Jul 16 21:09:22.481246 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=error.format_message()) Jul 16 21:09:22.481246 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Flavor 628570885 could not be found. Jul 16 21:09:22.481246 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:09:22.481246 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:09:22.481246 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:09:22.481246 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:09:22.481246 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:09:22.482857 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:09:22.482857 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:09:22.482857 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:09:22.482857 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 628570885 could not be found. Jul 16 21:09:22.482857 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:09:22.485620 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-3eb6729a-bdbb-4691-8536-2b2daf7df35f tempest-FlavorsAdminTestJSON-20925527 tempest-FlavorsAdminTestJSON-20925527-project] Returning 400 to user: Flavor 628570885 could not be found. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:09:22.488081 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-3eb6729a-bdbb-4691-8536-2b2daf7df35f tempest-FlavorsAdminTestJSON-20925527 tempest-FlavorsAdminTestJSON-20925527-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 400 len: 80 microversion: 2.1 time: 0.076593 Jul 16 21:09:22.490688 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 162/318] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 21:09:22 2022] POST /compute/v2.1/servers => generated 80 bytes in 80 msecs (HTTP/1.1 400) 9 headers in 380 bytes (1 switches on core 0) Jul 16 21:09:22.512797 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-d8fc4339-d19c-4e54-89f1-d15a09686a4a tempest-FlavorsAdminTestJSON-758135383 tempest-FlavorsAdminTestJSON-758135383-project-admin] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:22.513682 ncdv-master-1 devstack@n-api.service[138680]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-flavor-manage:delete": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:09:22.513682 ncdv-master-1 devstack@n-api.service[138680]: warnings.warn(msg) Jul 16 21:09:22.653854 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell0:SELECT=4 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:09:22.719682 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-e7941793-4176-45af-a488-f3252f611a84 tempest-FlavorsAccessNegativeTestJSON-1113799613 tempest-FlavorsAccessNegativeTestJSON-1113799613-project-admin] 10.222.0.16 "POST /compute/v2.1/flavors/1952317055/action" status: 200 len: 21 microversion: 2.1 time: 0.358757 Jul 16 21:09:22.720164 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-d8fc4339-d19c-4e54-89f1-d15a09686a4a tempest-FlavorsAdminTestJSON-758135383 tempest-FlavorsAdminTestJSON-758135383-project-admin] 10.222.0.16 "DELETE /compute/v2.1/flavors/628570885" status: 202 len: 0 microversion: 2.1 time: 0.211263 Jul 16 21:09:22.721278 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 157/319] 10.222.0.16 () {60 vars in 1199 bytes} [Sat Jul 16 21:09:22 2022] POST /compute/v2.1/flavors/1952317055/action => generated 21 bytes in 361 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Jul 16 21:09:22.722753 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 163/320] 10.222.0.16 () {58 vars in 1157 bytes} [Sat Jul 16 21:09:22 2022] DELETE /compute/v2.1/flavors/628570885 => generated 0 bytes in 219 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 21:09:22.728969 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-d1502b66-7e48-47b3-83fc-3dc73c98c19f tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:22.729622 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-d1502b66-7e48-47b3-83fc-3dc73c98c19f tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] [instance: 1631f0cc-de67-4c0d-abd0-3e0e11dfb4ed] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:09:22.739284 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d1502b66-7e48-47b3-83fc-3dc73c98c19f tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:22.739911 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d1502b66-7e48-47b3-83fc-3dc73c98c19f tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:22.740898 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d1502b66-7e48-47b3-83fc-3dc73c98c19f tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:22.741845 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d1502b66-7e48-47b3-83fc-3dc73c98c19f tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:22.835335 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-2d75ba85-f9b7-4a9e-846c-1f82241d5de5 req-cebe179e-b505-4904-b812-185d10cb64e6 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "1631f0cc-de67-4c0d-abd0-3e0e11dfb4ed", "tag": "e43e8868-6c5f-4776-ab31-fb29f6dcf4bf"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:09:22.846795 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-2d75ba85-f9b7-4a9e-846c-1f82241d5de5 req-cebe179e-b505-4904-b812-185d10cb64e6 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:22.847751 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-2d75ba85-f9b7-4a9e-846c-1f82241d5de5 req-cebe179e-b505-4904-b812-185d10cb64e6 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:22.859850 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-2d75ba85-f9b7-4a9e-846c-1f82241d5de5 req-cebe179e-b505-4904-b812-185d10cb64e6 service nova] Creating event network-changed:e43e8868-6c5f-4776-ab31-fb29f6dcf4bf for instance 1631f0cc-de67-4c0d-abd0-3e0e11dfb4ed on nch2-master-1 Jul 16 21:09:22.867251 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-2d75ba85-f9b7-4a9e-846c-1f82241d5de5 req-cebe179e-b505-4904-b812-185d10cb64e6 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.143017 Jul 16 21:09:22.868697 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 164/321] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:09:22 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 145 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:09:22.877723 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-7109fc1a-5dad-4ea9-8113-c5de81dbc08a tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:22.879800 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-7109fc1a-5dad-4ea9-8113-c5de81dbc08a tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] [instance: e506a6e0-fbcd-4e53-a6a0-d31761654c92] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:09:22.887071 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7109fc1a-5dad-4ea9-8113-c5de81dbc08a tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:22.887873 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7109fc1a-5dad-4ea9-8113-c5de81dbc08a tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:22.889221 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7109fc1a-5dad-4ea9-8113-c5de81dbc08a tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:22.889920 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7109fc1a-5dad-4ea9-8113-c5de81dbc08a tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:23.280870 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-d1502b66-7e48-47b3-83fc-3dc73c98c19f tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0d72626f465b4f37ad3b18bb945d4000', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fc1c3463f9444119af9d18f85bdd703d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:23.297256 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-d1502b66-7e48-47b3-83fc-3dc73c98c19f tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] 10.222.0.16 "GET /compute/v2.1/servers/1631f0cc-de67-4c0d-abd0-3e0e11dfb4ed" status: 200 len: 1318 microversion: 2.1 time: 0.574565 Jul 16 21:09:23.298521 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 158/322] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:09:22 2022] GET /compute/v2.1/servers/1631f0cc-de67-4c0d-abd0-3e0e11dfb4ed => generated 1318 bytes in 577 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:09:23.314429 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-6d59b9cf-2c5e-4c63-acf4-62d962c489a1 tempest-FlavorsAdminTestJSON-758135383 tempest-FlavorsAdminTestJSON-758135383-project-admin] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:23.362145 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-6d59b9cf-2c5e-4c63-acf4-62d962c489a1 tempest-FlavorsAdminTestJSON-758135383 tempest-FlavorsAdminTestJSON-758135383-project-admin] 10.222.0.16 "GET /compute/v2.1/flavors/detail" status: 200 len: 5819 microversion: 2.1 time: 0.062275 Jul 16 21:09:23.368741 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 159/323] 10.222.0.16 () {58 vars in 1145 bytes} [Sat Jul 16 21:09:23 2022] GET /compute/v2.1/flavors/detail => generated 5819 bytes in 69 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:09:23.374357 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-00ade250-a460-40bb-b973-2bc45dee370a tempest-FlavorsAccessNegativeTestJSON-1113799613 tempest-FlavorsAccessNegativeTestJSON-1113799613-project-admin] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:23.383264 ncdv-master-1 devstack@n-api.service[138681]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-flavor-manage:delete": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:09:23.383264 ncdv-master-1 devstack@n-api.service[138681]: warnings.warn(msg) Jul 16 21:09:23.509039 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-00ade250-a460-40bb-b973-2bc45dee370a tempest-FlavorsAccessNegativeTestJSON-1113799613 tempest-FlavorsAccessNegativeTestJSON-1113799613-project-admin] 10.222.0.16 "DELETE /compute/v2.1/flavors/1952317055" status: 202 len: 0 microversion: 2.1 time: 0.138224 Jul 16 21:09:23.510683 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 160/324] 10.222.0.16 () {58 vars in 1160 bytes} [Sat Jul 16 21:09:23 2022] DELETE /compute/v2.1/flavors/1952317055 => generated 0 bytes in 141 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 21:09:23.517665 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-0a2227b5-70a4-486f-aa85-0eed9a0c55a1 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:23.528919 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-0a2227b5-70a4-486f-aa85-0eed9a0c55a1 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] [instance: 148d5ad9-2900-4c2b-909b-aa446757e33d] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:09:23.547939 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0a2227b5-70a4-486f-aa85-0eed9a0c55a1 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:23.547939 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0a2227b5-70a4-486f-aa85-0eed9a0c55a1 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:23.547939 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0a2227b5-70a4-486f-aa85-0eed9a0c55a1 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:23.547939 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0a2227b5-70a4-486f-aa85-0eed9a0c55a1 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:23.636392 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-7109fc1a-5dad-4ea9-8113-c5de81dbc08a tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ef9cbce2b2e64665b7f50767342dcb46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0ee02e4c8b448f2b77798ae504129b9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:23.651443 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-7109fc1a-5dad-4ea9-8113-c5de81dbc08a tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] 10.222.0.16 "GET /compute/v2.1/servers/e506a6e0-fbcd-4e53-a6a0-d31761654c92" status: 200 len: 1590 microversion: 2.1 time: 0.781126 Jul 16 21:09:23.652440 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 165/325] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:09:22 2022] GET /compute/v2.1/servers/e506a6e0-fbcd-4e53-a6a0-d31761654c92 => generated 1590 bytes in 783 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:09:23.659661 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-9745a043-5efc-47b8-bdf3-db743d133e99 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:23.660642 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-9745a043-5efc-47b8-bdf3-db743d133e99 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: 1703f309-9e2e-4c5f-8588-eb4e43f21d91] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:09:23.672086 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9745a043-5efc-47b8-bdf3-db743d133e99 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:23.673040 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9745a043-5efc-47b8-bdf3-db743d133e99 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:23.674071 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9745a043-5efc-47b8-bdf3-db743d133e99 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:23.674679 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9745a043-5efc-47b8-bdf3-db743d133e99 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:24.259478 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-0a2227b5-70a4-486f-aa85-0eed9a0c55a1 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'abeb7020019d41a18de42956d706a011', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '649babe54d104436af461f1ee4767001', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:24.277761 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-9745a043-5efc-47b8-bdf3-db743d133e99 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b68dacc4b8463ba07fae6372131117', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55a19b7a4682457a9132f5fc80110712', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:24.280969 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-0a2227b5-70a4-486f-aa85-0eed9a0c55a1 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] 10.222.0.16 "GET /compute/v2.1/servers/148d5ad9-2900-4c2b-909b-aa446757e33d" status: 200 len: 1316 microversion: 2.1 time: 0.767560 Jul 16 21:09:24.284442 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 161/326] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:09:23 2022] GET /compute/v2.1/servers/148d5ad9-2900-4c2b-909b-aa446757e33d => generated 1316 bytes in 773 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:09:24.291693 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-a96eb3d6-ccb2-4f9b-9098-f4c5da2ee423 tempest-FlavorsAdminTestJSON-758135383 tempest-FlavorsAdminTestJSON-758135383-project-admin] Action: 'create', calling method: >, body: {"flavor": {"ram": 512, "vcpus": 1, "disk": 10, "name": "tempest-test_flavor_-491081266", "id": 699450632, "os-flavor-access:is_public": "False"}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:09:24.295124 ncdv-master-1 devstack@n-api.service[138681]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-flavor-manage:create": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:09:24.295124 ncdv-master-1 devstack@n-api.service[138681]: warnings.warn(msg) Jul 16 21:09:24.299609 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-9745a043-5efc-47b8-bdf3-db743d133e99 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "GET /compute/v2.1/servers/1703f309-9e2e-4c5f-8588-eb4e43f21d91" status: 200 len: 1475 microversion: 2.1 time: 0.643007 Jul 16 21:09:24.299609 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 166/327] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:09:23 2022] GET /compute/v2.1/servers/1703f309-9e2e-4c5f-8588-eb4e43f21d91 => generated 1475 bytes in 646 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:09:24.313274 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-3a2d988a-f4a8-4c29-a147-00dd0f1e2bc6 tempest-FlavorsAccessNegativeTestJSON-1113799613 tempest-FlavorsAccessNegativeTestJSON-1113799613-project-admin] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:24.337124 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-3a2d988a-f4a8-4c29-a147-00dd0f1e2bc6 tempest-FlavorsAccessNegativeTestJSON-1113799613 tempest-FlavorsAccessNegativeTestJSON-1113799613-project-admin] 10.222.0.16 "GET /compute/v2.1/flavors/detail" status: 200 len: 5819 microversion: 2.1 time: 0.037153 Jul 16 21:09:24.349732 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 167/328] 10.222.0.16 () {58 vars in 1145 bytes} [Sat Jul 16 21:09:24 2022] GET /compute/v2.1/flavors/detail => generated 5819 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:09:24.355131 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-aa9bedd9-c146-46a7-878b-cca703484f57 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:24.355991 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-aa9bedd9-c146-46a7-878b-cca703484f57 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] [instance: 1631f0cc-de67-4c0d-abd0-3e0e11dfb4ed] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:09:24.364324 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-aa9bedd9-c146-46a7-878b-cca703484f57 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:24.365017 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-aa9bedd9-c146-46a7-878b-cca703484f57 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:24.366200 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-aa9bedd9-c146-46a7-878b-cca703484f57 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:24.366797 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-aa9bedd9-c146-46a7-878b-cca703484f57 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:24.414032 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-a96eb3d6-ccb2-4f9b-9098-f4c5da2ee423 tempest-FlavorsAdminTestJSON-758135383 tempest-FlavorsAdminTestJSON-758135383-project-admin] 10.222.0.16 "POST /compute/v2.1/flavors" status: 200 len: 411 microversion: 2.1 time: 0.128069 Jul 16 21:09:24.421634 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 162/329] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 21:09:24 2022] POST /compute/v2.1/flavors => generated 411 bytes in 136 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:09:24.429609 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-0ab1618f-0961-4c0a-ba14-7888dcd985f5 tempest-FlavorsAccessNegativeTestJSON-1113799613 tempest-FlavorsAccessNegativeTestJSON-1113799613-project-admin] Action: 'create', calling method: >, body: {"flavor": {"ram": 512, "vcpus": 1, "disk": 10, "name": "tempest-FlavorsAccessNegativeTestJSON-flavor-945014493", "id": 286344921, "os-flavor-access:is_public": "False"}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:09:24.514077 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-0ab1618f-0961-4c0a-ba14-7888dcd985f5 tempest-FlavorsAccessNegativeTestJSON-1113799613 tempest-FlavorsAccessNegativeTestJSON-1113799613-project-admin] 10.222.0.16 "POST /compute/v2.1/flavors" status: 200 len: 435 microversion: 2.1 time: 0.089571 Jul 16 21:09:24.515068 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 163/330] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 21:09:24 2022] POST /compute/v2.1/flavors => generated 435 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:09:24.521631 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-e318c2d6-5657-48ec-800e-16e5f16f574a tempest-FlavorsAdminTestJSON-758135383 tempest-FlavorsAdminTestJSON-758135383-project-admin] Action: 'create', calling method: >, body: {"flavor": {"ram": 512, "vcpus": 1, "disk": 10, "name": "tempest-test_flavor_-1512493953", "id": 1993115830, "os-flavor-access:is_public": "True"}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:09:24.653860 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-e318c2d6-5657-48ec-800e-16e5f16f574a tempest-FlavorsAdminTestJSON-758135383 tempest-FlavorsAdminTestJSON-758135383-project-admin] 10.222.0.16 "POST /compute/v2.1/flavors" status: 200 len: 414 microversion: 2.1 time: 0.136885 Jul 16 21:09:24.655084 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 164/331] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 21:09:24 2022] POST /compute/v2.1/flavors => generated 414 bytes in 139 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:09:24.669517 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-68233a1f-6892-4563-84d1-45ea0819f685 tempest-FlavorsAccessNegativeTestJSON-1113799613 tempest-FlavorsAccessNegativeTestJSON-1113799613-project-admin] Action: 'action', calling method: >, body: {"removeTenantAccess": {"tenant": "4fcfaa54bdb445179a4e772933128170"}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:09:24.679094 ncdv-master-1 devstack@n-api.service[138681]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-flavor-access:remove_tenant_access": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:09:24.679094 ncdv-master-1 devstack@n-api.service[138681]: warnings.warn(msg) Jul 16 21:09:24.804505 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-aa9bedd9-c146-46a7-878b-cca703484f57 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0d72626f465b4f37ad3b18bb945d4000', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fc1c3463f9444119af9d18f85bdd703d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:24.814551 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-aa9bedd9-c146-46a7-878b-cca703484f57 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] 10.222.0.16 "GET /compute/v2.1/servers/1631f0cc-de67-4c0d-abd0-3e0e11dfb4ed" status: 200 len: 1318 microversion: 2.1 time: 0.463365 Jul 16 21:09:24.816172 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 168/332] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:09:24 2022] GET /compute/v2.1/servers/1631f0cc-de67-4c0d-abd0-3e0e11dfb4ed => generated 1318 bytes in 466 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:09:24.834637 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-8079f99f-10b9-4d2b-9c71-e4bd6d68abaf tempest-FlavorsAdminTestJSON-758135383 tempest-FlavorsAdminTestJSON-758135383-project-admin] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:24.873337 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-8079f99f-10b9-4d2b-9c71-e4bd6d68abaf tempest-FlavorsAdminTestJSON-758135383 tempest-FlavorsAdminTestJSON-758135383-project-admin] 10.222.0.16 "GET /compute/v2.1/flavors/detail?is_public=f" status: 200 len: 839 microversion: 2.1 time: 0.055778 Jul 16 21:09:24.875205 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 169/333] 10.222.0.16 () {58 vars in 1168 bytes} [Sat Jul 16 21:09:24 2022] GET /compute/v2.1/flavors/detail?is_public=f => generated 839 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:09:24.898668 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-589b82f4-2053-475b-98f0-0acaba408ae3 tempest-FlavorsAdminTestJSON-758135383 tempest-FlavorsAdminTestJSON-758135383-project-admin] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:24.915311 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-589b82f4-2053-475b-98f0-0acaba408ae3 tempest-FlavorsAdminTestJSON-758135383 tempest-FlavorsAdminTestJSON-758135383-project-admin] 10.222.0.16 "GET /compute/v2.1/flavors/detail?is_public=false" status: 200 len: 839 microversion: 2.1 time: 0.028438 Jul 16 21:09:24.916810 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 170/334] 10.222.0.16 () {58 vars in 1176 bytes} [Sat Jul 16 21:09:24 2022] GET /compute/v2.1/flavors/detail?is_public=false => generated 839 bytes in 31 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:09:24.926368 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-a92fa234-a7ee-4c64-8950-1f8e5947feb0 req-ffb2e2f5-7a43-44e6-ae1e-ead4023e3f09 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "e506a6e0-fbcd-4e53-a6a0-d31761654c92", "tag": "fa66e5de-7c5f-4e0e-adc5-96ea3b90bd37"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:09:24.933727 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-a92fa234-a7ee-4c64-8950-1f8e5947feb0 req-ffb2e2f5-7a43-44e6-ae1e-ead4023e3f09 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:24.934465 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-a92fa234-a7ee-4c64-8950-1f8e5947feb0 req-ffb2e2f5-7a43-44e6-ae1e-ead4023e3f09 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:24.947680 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-a92fa234-a7ee-4c64-8950-1f8e5947feb0 req-ffb2e2f5-7a43-44e6-ae1e-ead4023e3f09 service nova] Creating event network-changed:fa66e5de-7c5f-4e0e-adc5-96ea3b90bd37 for instance e506a6e0-fbcd-4e53-a6a0-d31761654c92 on nch1-master-1 Jul 16 21:09:24.954291 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-a92fa234-a7ee-4c64-8950-1f8e5947feb0 req-ffb2e2f5-7a43-44e6-ae1e-ead4023e3f09 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.035337 Jul 16 21:09:24.955552 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 171/335] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:09:24 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:09:24.964711 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-8252150b-6e4c-4c17-94dd-07daeebbb690 tempest-FlavorsAdminTestJSON-758135383 tempest-FlavorsAdminTestJSON-758135383-project-admin] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:24.976353 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-8252150b-6e4c-4c17-94dd-07daeebbb690 tempest-FlavorsAdminTestJSON-758135383 tempest-FlavorsAdminTestJSON-758135383-project-admin] 10.222.0.16 "GET /compute/v2.1/flavors/detail?is_public=no" status: 200 len: 839 microversion: 2.1 time: 0.019250 Jul 16 21:09:24.977809 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 172/336] 10.222.0.16 () {58 vars in 1170 bytes} [Sat Jul 16 21:09:24 2022] GET /compute/v2.1/flavors/detail?is_public=no => generated 839 bytes in 22 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:09:24.996701 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-d50c7773-861c-4841-8fb0-43b1540d9c38 tempest-FlavorsAdminTestJSON-758135383 tempest-FlavorsAdminTestJSON-758135383-project-admin] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:24.996872 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-68233a1f-6892-4563-84d1-45ea0819f685 tempest-FlavorsAccessNegativeTestJSON-1113799613 tempest-FlavorsAccessNegativeTestJSON-1113799613-project-admin] HTTP exception thrown: Flavor access not found for 34 / 4fcfaa54bdb445179a4e772933128170 combination. Jul 16 21:09:24.996872 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-68233a1f-6892-4563-84d1-45ea0819f685 tempest-FlavorsAccessNegativeTestJSON-1113799613 tempest-FlavorsAccessNegativeTestJSON-1113799613-project-admin] Request method failure captured: Jul 16 21:09:24.996872 ncdv-master-1 devstack@n-api.service[138681]: request: POST /compute/v2.1/flavors/286344921/action HTTP/1.1 Jul 16 21:09:24.996872 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:09:24.996872 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:09:24.996872 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:09:24.996872 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 70 Jul 16 21:09:24.996872 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:09:24.996872 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:09:24.996872 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:09:24.996872 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:09:24.996872 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0yj7TqhQMly9bDnDqOBZY1OHAWHNoSytoQcgL-V9TmjaQVAVWqaNvpeVrgxpHpBh8P29OcQ9zLIwJPxFZaJdC9FWMwFzpXueXcVMXCBTEwHMfizgMcOeg2IKk9VuXZnnHMfm4IQdXMisOAsjwoyoxjedCTDXTY7Kwsohcm0agHaBdtk Jul 16 21:09:24.996872 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:09:24.996872 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:09:24.996872 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:09:24.996872 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:09:24.996872 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:09:24.996872 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:09:24.996872 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: 054de79bd580451bafa72a8e903ffab5 Jul 16 21:09:24.996872 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-FlavorsAccessNegativeTestJSON-1113799613 Jul 16 21:09:24.996872 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,admin,reader Jul 16 21:09:24.996872 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,admin,reader Jul 16 21:09:24.998334 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/054de79bd580451bafa72a8e903ffab5"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/054de79bd580451bafa72a8e903ffab5"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_054de79bd580451bafa72a8e903ffab5"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/054de79bd580451bafa72a8e903ffab5"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:09:24.998334 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-FlavorsAccessNegativeTestJSON-1113799613 Jul 16 21:09:24.998334 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: 054de79bd580451bafa72a8e903ffab5 Jul 16 21:09:24.998334 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-FlavorsAccessNegativeTestJSON-1113799613 Jul 16 21:09:24.998334 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-FlavorsAccessNegativeTestJSON-1113799613-project-admin Jul 16 21:09:24.998334 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:09:24.998334 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:09:24.998334 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: 651cbc6596e24fcfa7232ddc273692ae Jul 16 21:09:24.998334 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-FlavorsAccessNegativeTestJSON-1113799613-project-admin Jul 16 21:09:24.998334 ncdv-master-1 devstack@n-api.service[138681]: Jul 16 21:09:24.998334 ncdv-master-1 devstack@n-api.service[138681]: {"removeTenantAccess": {"tenant": "4fcfaa54bdb445179a4e772933128170"}} Jul 16 21:09:24.998334 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:09:24.999173 ncdv-master-1 devstack@n-api.service[138681]: exception: Flavor access not found for 34 / 4fcfaa54bdb445179a4e772933128170 combination. Jul 16 21:09:24.999173 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': '286344921', 'body': {'removeTenantAccess': {'tenant': '4fcfaa54bdb445179a4e772933128170'}}} Jul 16 21:09:24.999173 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:09:24.999173 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:09:24.999173 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavor_access.py", line 104, in _remove_tenant_access Jul 16 21:09:24.999173 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi flavor.remove_access(tenant) Jul 16 21:09:24.999173 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 226, in wrapper Jul 16 21:09:24.999173 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return fn(self, *args, **kwargs) Jul 16 21:09:24.999173 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 423, in remove_access Jul 16 21:09:24.999173 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi self._remove_access(project_id) Jul 16 21:09:24.999173 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 416, in _remove_access Jul 16 21:09:24.999173 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi self._flavor_del_project(self._context, self.id, project_id) Jul 16 21:09:24.999173 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 401, in _flavor_del_project Jul 16 21:09:24.999173 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return _flavor_del_project(context, flavor_id, project_id) Jul 16 21:09:24.999173 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1010, in wrapper Jul 16 21:09:24.999173 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return fn(*args, **kwargs) Jul 16 21:09:24.999173 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 86, in _flavor_del_project Jul 16 21:09:24.999173 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.FlavorAccessNotFound(flavor_id=flavor_id, Jul 16 21:09:25.000936 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.FlavorAccessNotFound: Flavor access not found for 34 / 4fcfaa54bdb445179a4e772933128170 combination. Jul 16 21:09:25.000936 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:09:25.000936 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:09:25.000936 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:09:25.000936 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:09:25.000936 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:09:25.000936 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:09:25.000936 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:09:25.000936 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:09:25.000936 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:09:25.000936 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:09:25.000936 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 16 21:09:25.000936 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 16 21:09:25.000936 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavor_access.py", line 107, in _remove_tenant_access Jul 16 21:09:25.000936 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:09:25.000936 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor access not found for 34 / 4fcfaa54bdb445179a4e772933128170 combination. Jul 16 21:09:25.000936 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:09:25.000936 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:09:25.000936 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:09:25.000936 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:09:25.002214 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:09:25.002214 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:09:25.002214 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:09:25.002214 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:09:25.002214 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor access not found for 34 / 4fcfaa54bdb445179a4e772933128170 combination. Jul 16 21:09:25.002214 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:09:25.002214 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-68233a1f-6892-4563-84d1-45ea0819f685 tempest-FlavorsAccessNegativeTestJSON-1113799613 tempest-FlavorsAccessNegativeTestJSON-1113799613-project-admin] Returning 404 to user: Flavor access not found for 34 / 4fcfaa54bdb445179a4e772933128170 combination. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:09:25.002214 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-68233a1f-6892-4563-84d1-45ea0819f685 tempest-FlavorsAccessNegativeTestJSON-1113799613 tempest-FlavorsAccessNegativeTestJSON-1113799613-project-admin] 10.222.0.16 "POST /compute/v2.1/flavors/286344921/action" status: 404 len: 124 microversion: 2.1 time: 0.331144 Jul 16 21:09:25.007537 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 165/337] 10.222.0.16 () {60 vars in 1196 bytes} [Sat Jul 16 21:09:24 2022] POST /compute/v2.1/flavors/286344921/action => generated 124 bytes in 342 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:09:25.010040 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-d50c7773-861c-4841-8fb0-43b1540d9c38 tempest-FlavorsAdminTestJSON-758135383 tempest-FlavorsAdminTestJSON-758135383-project-admin] 10.222.0.16 "GET /compute/v2.1/flavors/detail?is_public=0" status: 200 len: 839 microversion: 2.1 time: 0.025664 Jul 16 21:09:25.013154 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-45deed08-443b-4d4b-bf7d-e5557977feb7 tempest-FlavorsAccessNegativeTestJSON-1113799613 tempest-FlavorsAccessNegativeTestJSON-1113799613-project-admin] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:25.014970 ncdv-master-1 devstack@n-api.service[138681]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-flavor-manage:delete": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:09:25.014970 ncdv-master-1 devstack@n-api.service[138681]: warnings.warn(msg) Jul 16 21:09:25.018585 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 173/338] 10.222.0.16 () {58 vars in 1168 bytes} [Sat Jul 16 21:09:24 2022] GET /compute/v2.1/flavors/detail?is_public=0 => generated 839 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:09:25.025195 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-5cb97659-0d31-49ff-baf6-93f675da7cfc tempest-FlavorsAdminTestJSON-758135383 tempest-FlavorsAdminTestJSON-758135383-project-admin] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:25.063810 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-5cb97659-0d31-49ff-baf6-93f675da7cfc tempest-FlavorsAdminTestJSON-758135383 tempest-FlavorsAdminTestJSON-758135383-project-admin] 10.222.0.16 "GET /compute/v2.1/flavors/detail?is_public=t" status: 200 len: 6223 microversion: 2.1 time: 0.043405 Jul 16 21:09:25.075089 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 174/339] 10.222.0.16 () {58 vars in 1168 bytes} [Sat Jul 16 21:09:25 2022] GET /compute/v2.1/flavors/detail?is_public=t => generated 6223 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:09:25.086260 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-195075de-c56a-4f96-b910-95a8bf4ab3c9 tempest-FlavorsAdminTestJSON-758135383 tempest-FlavorsAdminTestJSON-758135383-project-admin] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:25.123204 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-195075de-c56a-4f96-b910-95a8bf4ab3c9 tempest-FlavorsAdminTestJSON-758135383 tempest-FlavorsAdminTestJSON-758135383-project-admin] 10.222.0.16 "GET /compute/v2.1/flavors/detail?is_public=true" status: 200 len: 6223 microversion: 2.1 time: 0.046610 Jul 16 21:09:25.135165 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 175/340] 10.222.0.16 () {58 vars in 1174 bytes} [Sat Jul 16 21:09:25 2022] GET /compute/v2.1/flavors/detail?is_public=true => generated 6223 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:09:25.155482 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-9eb3b330-01af-4a1b-a243-8a78e735d695 tempest-FlavorsAdminTestJSON-758135383 tempest-FlavorsAdminTestJSON-758135383-project-admin] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:25.165461 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-45deed08-443b-4d4b-bf7d-e5557977feb7 tempest-FlavorsAccessNegativeTestJSON-1113799613 tempest-FlavorsAccessNegativeTestJSON-1113799613-project-admin] 10.222.0.16 "DELETE /compute/v2.1/flavors/286344921" status: 202 len: 0 microversion: 2.1 time: 0.156505 Jul 16 21:09:25.170534 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 166/341] 10.222.0.16 () {58 vars in 1157 bytes} [Sat Jul 16 21:09:25 2022] DELETE /compute/v2.1/flavors/286344921 => generated 0 bytes in 158 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 21:09:25.176094 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-89b06798-7331-4266-9742-b2ea46d6c981 tempest-FlavorsAccessNegativeTestJSON-1113799613 tempest-FlavorsAccessNegativeTestJSON-1113799613-project-admin] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:25.181557 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-9eb3b330-01af-4a1b-a243-8a78e735d695 tempest-FlavorsAdminTestJSON-758135383 tempest-FlavorsAdminTestJSON-758135383-project-admin] 10.222.0.16 "GET /compute/v2.1/flavors/detail?is_public=yes" status: 200 len: 6223 microversion: 2.1 time: 0.029538 Jul 16 21:09:25.201453 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 176/342] 10.222.0.16 () {58 vars in 1172 bytes} [Sat Jul 16 21:09:25 2022] GET /compute/v2.1/flavors/detail?is_public=yes => generated 6223 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:09:25.206777 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-f2a0c08c-1309-45eb-a1f2-69f401a589a2 tempest-FlavorsAdminTestJSON-758135383 tempest-FlavorsAdminTestJSON-758135383-project-admin] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:25.215355 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-89b06798-7331-4266-9742-b2ea46d6c981 tempest-FlavorsAccessNegativeTestJSON-1113799613 tempest-FlavorsAccessNegativeTestJSON-1113799613-project-admin] 10.222.0.16 "GET /compute/v2.1/flavors/detail" status: 200 len: 6223 microversion: 2.1 time: 0.043550 Jul 16 21:09:25.216777 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 167/343] 10.222.0.16 () {58 vars in 1145 bytes} [Sat Jul 16 21:09:25 2022] GET /compute/v2.1/flavors/detail => generated 6223 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:09:25.239826 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-f2a0c08c-1309-45eb-a1f2-69f401a589a2 tempest-FlavorsAdminTestJSON-758135383 tempest-FlavorsAdminTestJSON-758135383-project-admin] 10.222.0.16 "GET /compute/v2.1/flavors/detail?is_public=1" status: 200 len: 6223 microversion: 2.1 time: 0.037128 Jul 16 21:09:25.251511 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 177/344] 10.222.0.16 () {58 vars in 1168 bytes} [Sat Jul 16 21:09:25 2022] GET /compute/v2.1/flavors/detail?is_public=1 => generated 6223 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:09:25.266503 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-fe7c2f63-db54-44f6-bd45-e538edf71645 tempest-FlavorsAdminTestJSON-758135383 tempest-FlavorsAdminTestJSON-758135383-project-admin] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:25.304635 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-33bac4d4-2e6b-4689-8562-6a8b51dde46b tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:25.305503 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-33bac4d4-2e6b-4689-8562-6a8b51dde46b tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] [instance: 148d5ad9-2900-4c2b-909b-aa446757e33d] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:09:25.323324 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-33bac4d4-2e6b-4689-8562-6a8b51dde46b tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:25.324745 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-33bac4d4-2e6b-4689-8562-6a8b51dde46b tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:25.338310 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-33bac4d4-2e6b-4689-8562-6a8b51dde46b tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:25.338310 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-33bac4d4-2e6b-4689-8562-6a8b51dde46b tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:25.411176 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-fe7c2f63-db54-44f6-bd45-e538edf71645 tempest-FlavorsAdminTestJSON-758135383 tempest-FlavorsAdminTestJSON-758135383-project-admin] 10.222.0.16 "DELETE /compute/v2.1/flavors/1993115830" status: 202 len: 0 microversion: 2.1 time: 0.157450 Jul 16 21:09:25.418029 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 168/345] 10.222.0.16 () {58 vars in 1160 bytes} [Sat Jul 16 21:09:25 2022] DELETE /compute/v2.1/flavors/1993115830 => generated 0 bytes in 159 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 21:09:25.418029 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-cd20251d-df26-4371-8ee7-93c4388a7149 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:25.418219 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-cd20251d-df26-4371-8ee7-93c4388a7149 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: 1703f309-9e2e-4c5f-8588-eb4e43f21d91] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:09:25.440252 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cd20251d-df26-4371-8ee7-93c4388a7149 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:25.440724 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cd20251d-df26-4371-8ee7-93c4388a7149 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:25.441953 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cd20251d-df26-4371-8ee7-93c4388a7149 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:25.442585 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cd20251d-df26-4371-8ee7-93c4388a7149 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:25.759377 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-33bac4d4-2e6b-4689-8562-6a8b51dde46b tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'abeb7020019d41a18de42956d706a011', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '649babe54d104436af461f1ee4767001', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:25.772654 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-33bac4d4-2e6b-4689-8562-6a8b51dde46b tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] 10.222.0.16 "GET /compute/v2.1/servers/148d5ad9-2900-4c2b-909b-aa446757e33d" status: 200 len: 1316 microversion: 2.1 time: 0.474300 Jul 16 21:09:25.773608 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 178/346] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:09:25 2022] GET /compute/v2.1/servers/148d5ad9-2900-4c2b-909b-aa446757e33d => generated 1316 bytes in 477 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:09:25.791559 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-4ada8688-d8e2-4e9f-958f-f220d126fbff tempest-FlavorsAdminTestJSON-758135383 tempest-FlavorsAdminTestJSON-758135383-project-admin] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:25.815082 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-4ada8688-d8e2-4e9f-958f-f220d126fbff tempest-FlavorsAdminTestJSON-758135383 tempest-FlavorsAdminTestJSON-758135383-project-admin] 10.222.0.16 "GET /compute/v2.1/flavors/detail" status: 200 len: 5819 microversion: 2.1 time: 0.040132 Jul 16 21:09:25.816003 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 179/347] 10.222.0.16 () {58 vars in 1145 bytes} [Sat Jul 16 21:09:25 2022] GET /compute/v2.1/flavors/detail => generated 5819 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:09:25.825114 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-cd20251d-df26-4371-8ee7-93c4388a7149 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b68dacc4b8463ba07fae6372131117', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55a19b7a4682457a9132f5fc80110712', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:25.835991 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-798cba5c-adbe-461c-8a16-4a31ffc40b25 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:25.836839 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-798cba5c-adbe-461c-8a16-4a31ffc40b25 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] [instance: 1631f0cc-de67-4c0d-abd0-3e0e11dfb4ed] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:09:25.838425 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-cd20251d-df26-4371-8ee7-93c4388a7149 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "GET /compute/v2.1/servers/1703f309-9e2e-4c5f-8588-eb4e43f21d91" status: 200 len: 1475 microversion: 2.1 time: 0.424957 Jul 16 21:09:25.841688 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 169/348] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:09:25 2022] GET /compute/v2.1/servers/1703f309-9e2e-4c5f-8588-eb4e43f21d91 => generated 1475 bytes in 429 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:09:25.849207 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-8b5f5e8f-9b6f-463f-bccf-66b80dce3368 tempest-FlavorsAdminTestJSON-758135383 tempest-FlavorsAdminTestJSON-758135383-project-admin] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:25.849991 ncdv-master-1 devstack@n-api.service[138681]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-flavor-manage:delete": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:09:25.849991 ncdv-master-1 devstack@n-api.service[138681]: warnings.warn(msg) Jul 16 21:09:25.854749 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-798cba5c-adbe-461c-8a16-4a31ffc40b25 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:25.856984 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-798cba5c-adbe-461c-8a16-4a31ffc40b25 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:25.856984 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-798cba5c-adbe-461c-8a16-4a31ffc40b25 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:25.856984 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-798cba5c-adbe-461c-8a16-4a31ffc40b25 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:25.942480 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-8b5f5e8f-9b6f-463f-bccf-66b80dce3368 tempest-FlavorsAdminTestJSON-758135383 tempest-FlavorsAdminTestJSON-758135383-project-admin] 10.222.0.16 "DELETE /compute/v2.1/flavors/699450632" status: 202 len: 0 microversion: 2.1 time: 0.099418 Jul 16 21:09:25.943343 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 170/349] 10.222.0.16 () {58 vars in 1157 bytes} [Sat Jul 16 21:09:25 2022] DELETE /compute/v2.1/flavors/699450632 => generated 0 bytes in 101 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 21:09:25.951272 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-345b7252-6857-48dc-a492-b118c3c824fe tempest-FlavorsAdminTestJSON-758135383 tempest-FlavorsAdminTestJSON-758135383-project-admin] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:25.997262 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-345b7252-6857-48dc-a492-b118c3c824fe tempest-FlavorsAdminTestJSON-758135383 tempest-FlavorsAdminTestJSON-758135383-project-admin] 10.222.0.16 "GET /compute/v2.1/flavors/detail" status: 200 len: 5819 microversion: 2.1 time: 0.049473 Jul 16 21:09:25.998313 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 171/350] 10.222.0.16 () {58 vars in 1145 bytes} [Sat Jul 16 21:09:25 2022] GET /compute/v2.1/flavors/detail => generated 5819 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:09:26.019294 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-648c4020-cdeb-4139-ba34-52e23dd7f131 tempest-FlavorsAdminTestJSON-758135383 tempest-FlavorsAdminTestJSON-758135383-project-admin] Action: 'create', calling method: >, body: {"flavor": {"ram": 512, "vcpus": 1, "disk": 10, "name": "tempest-test_flavor_-1158467506", "id": 1946711514, "os-flavor-access:is_public": "False"}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:09:26.021350 ncdv-master-1 devstack@n-api.service[138681]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-flavor-manage:create": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:09:26.021350 ncdv-master-1 devstack@n-api.service[138681]: warnings.warn(msg) Jul 16 21:09:26.129406 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-648c4020-cdeb-4139-ba34-52e23dd7f131 tempest-FlavorsAdminTestJSON-758135383 tempest-FlavorsAdminTestJSON-758135383-project-admin] 10.222.0.16 "POST /compute/v2.1/flavors" status: 200 len: 415 microversion: 2.1 time: 0.109742 Jul 16 21:09:26.130096 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 172/351] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 21:09:26 2022] POST /compute/v2.1/flavors => generated 415 bytes in 117 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 16 21:09:26.134671 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-798cba5c-adbe-461c-8a16-4a31ffc40b25 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0d72626f465b4f37ad3b18bb945d4000', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fc1c3463f9444119af9d18f85bdd703d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:26.146841 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-798cba5c-adbe-461c-8a16-4a31ffc40b25 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] 10.222.0.16 "GET /compute/v2.1/servers/1631f0cc-de67-4c0d-abd0-3e0e11dfb4ed" status: 200 len: 1318 microversion: 2.1 time: 0.315836 Jul 16 21:09:26.149633 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 180/352] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:09:25 2022] GET /compute/v2.1/servers/1631f0cc-de67-4c0d-abd0-3e0e11dfb4ed => generated 1318 bytes in 319 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:09:26.154674 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-2099abd4-39ea-4126-9c67-f1fdd5f75b8c tempest-FlavorsAdminTestJSON-758135383 tempest-FlavorsAdminTestJSON-758135383-project-admin] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:26.192959 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-2099abd4-39ea-4126-9c67-f1fdd5f75b8c tempest-FlavorsAdminTestJSON-758135383 tempest-FlavorsAdminTestJSON-758135383-project-admin] 10.222.0.16 "GET /compute/v2.1/flavors/detail" status: 200 len: 5819 microversion: 2.1 time: 0.041674 Jul 16 21:09:26.194574 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 181/353] 10.222.0.16 () {58 vars in 1145 bytes} [Sat Jul 16 21:09:26 2022] GET /compute/v2.1/flavors/detail => generated 5819 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:09:26.218408 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-1c924796-e534-420b-ba78-d984dddddcbe tempest-FlavorsAdminTestJSON-20925527 tempest-FlavorsAdminTestJSON-20925527-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:26.246187 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-1c924796-e534-420b-ba78-d984dddddcbe tempest-FlavorsAdminTestJSON-20925527 tempest-FlavorsAdminTestJSON-20925527-project] 10.222.0.16 "GET /compute/v2.1/flavors/detail" status: 200 len: 5819 microversion: 2.1 time: 0.039167 Jul 16 21:09:26.257149 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 182/354] 10.222.0.16 () {58 vars in 1145 bytes} [Sat Jul 16 21:09:26 2022] GET /compute/v2.1/flavors/detail => generated 5819 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:09:26.264159 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-0f9a2806-f710-4ef9-b4e6-083761382290 tempest-FlavorsAdminTestJSON-758135383 tempest-FlavorsAdminTestJSON-758135383-project-admin] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:26.265479 ncdv-master-1 devstack@n-api.service[138680]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-flavor-manage:delete": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:09:26.265479 ncdv-master-1 devstack@n-api.service[138680]: warnings.warn(msg) Jul 16 21:09:26.351137 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-0f9a2806-f710-4ef9-b4e6-083761382290 tempest-FlavorsAdminTestJSON-758135383 tempest-FlavorsAdminTestJSON-758135383-project-admin] 10.222.0.16 "DELETE /compute/v2.1/flavors/1946711514" status: 202 len: 0 microversion: 2.1 time: 0.090767 Jul 16 21:09:26.353696 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 183/355] 10.222.0.16 () {58 vars in 1160 bytes} [Sat Jul 16 21:09:26 2022] DELETE /compute/v2.1/flavors/1946711514 => generated 0 bytes in 93 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Jul 16 21:09:26.361422 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-f75d4616-aa23-4038-8d16-8f6f87fcff31 tempest-FlavorsAdminTestJSON-758135383 tempest-FlavorsAdminTestJSON-758135383-project-admin] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:26.389816 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-f75d4616-aa23-4038-8d16-8f6f87fcff31 tempest-FlavorsAdminTestJSON-758135383 tempest-FlavorsAdminTestJSON-758135383-project-admin] 10.222.0.16 "GET /compute/v2.1/flavors/detail" status: 200 len: 5819 microversion: 2.1 time: 0.033083 Jul 16 21:09:26.391162 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 184/356] 10.222.0.16 () {58 vars in 1145 bytes} [Sat Jul 16 21:09:26 2022] GET /compute/v2.1/flavors/detail => generated 5819 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:09:26.404808 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-4bce5701-bb46-4fcc-b437-3da0d2d2a152 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-LiveMigrationTest-server-988289400", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452", "networks": [{"uuid": "26ad28eb-6622-4b58-ad05-2de54c33be8d"}]}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:09:26.408771 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-3e9a54b0-0501-41b1-924f-f156b7f1feb1 tempest-FlavorsAdminTestJSON-758135383 tempest-FlavorsAdminTestJSON-758135383-project-admin] Action: 'create', calling method: >, body: {"flavor": {"ram": 512, "vcpus": 1, "disk": 10, "name": "tempest-test_flavor_-1764160227", "id": 1449501417, "os-flavor-access:is_public": "True"}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:09:26.410295 ncdv-master-1 devstack@n-api.service[138680]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-flavor-manage:create": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:09:26.410295 ncdv-master-1 devstack@n-api.service[138680]: warnings.warn(msg) Jul 16 21:09:26.416567 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_api:SELECT=75,nova_api:DELETE=20,nova_api:INSERT=5 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:09:26.472436 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.network.neutron [None req-4bce5701-bb46-4fcc-b437-3da0d2d2a152 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] validate_networks() for [('26ad28eb-6622-4b58-ad05-2de54c33be8d', None, None, None, None, None)] {{(pid=138681) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 16 21:09:26.491335 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-3e9a54b0-0501-41b1-924f-f156b7f1feb1 tempest-FlavorsAdminTestJSON-758135383 tempest-FlavorsAdminTestJSON-758135383-project-admin] 10.222.0.16 "POST /compute/v2.1/flavors" status: 200 len: 414 microversion: 2.1 time: 0.086708 Jul 16 21:09:26.498463 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 185/357] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 21:09:26 2022] POST /compute/v2.1/flavors => generated 414 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:09:26.504730 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-b332915c-563c-4088-805e-55b77c42b60e tempest-FlavorsAdminTestJSON-20925527 tempest-FlavorsAdminTestJSON-20925527-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:26.534013 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-b332915c-563c-4088-805e-55b77c42b60e tempest-FlavorsAdminTestJSON-20925527 tempest-FlavorsAdminTestJSON-20925527-project] 10.222.0.16 "GET /compute/v2.1/flavors/detail" status: 200 len: 6223 microversion: 2.1 time: 0.034184 Jul 16 21:09:26.539972 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 186/358] 10.222.0.16 () {58 vars in 1145 bytes} [Sat Jul 16 21:09:26 2022] GET /compute/v2.1/flavors/detail => generated 6223 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:09:26.554137 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-e9d91bd8-97da-4403-81f0-dc5605a5888a tempest-FlavorsAdminTestJSON-758135383 tempest-FlavorsAdminTestJSON-758135383-project-admin] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:26.657697 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-e9d91bd8-97da-4403-81f0-dc5605a5888a tempest-FlavorsAdminTestJSON-758135383 tempest-FlavorsAdminTestJSON-758135383-project-admin] 10.222.0.16 "DELETE /compute/v2.1/flavors/1449501417" status: 202 len: 0 microversion: 2.1 time: 0.109292 Jul 16 21:09:26.658572 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 187/359] 10.222.0.16 () {58 vars in 1160 bytes} [Sat Jul 16 21:09:26 2022] DELETE /compute/v2.1/flavors/1449501417 => generated 0 bytes in 112 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Jul 16 21:09:26.668390 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-150df60c-60c5-44f7-840c-fe0e172a1ef8 tempest-FlavorsAdminTestJSON-758135383 tempest-FlavorsAdminTestJSON-758135383-project-admin] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:26.692374 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-150df60c-60c5-44f7-840c-fe0e172a1ef8 tempest-FlavorsAdminTestJSON-758135383 tempest-FlavorsAdminTestJSON-758135383-project-admin] 10.222.0.16 "GET /compute/v2.1/flavors/detail" status: 200 len: 5819 microversion: 2.1 time: 0.028466 Jul 16 21:09:26.693408 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 188/360] 10.222.0.16 () {58 vars in 1145 bytes} [Sat Jul 16 21:09:26 2022] GET /compute/v2.1/flavors/detail => generated 5819 bytes in 31 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:09:26.772295 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-f88cf271-ad51-4ddb-8a19-b3fcc4482d40 req-604b656d-18c2-456a-9883-4050f5b1070b service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "148d5ad9-2900-4c2b-909b-aa446757e33d", "name": "network-vif-plugged", "status": "completed", "tag": "7ee252fe-6a69-4f90-9053-a7eabcf7750c"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:09:26.778935 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-f88cf271-ad51-4ddb-8a19-b3fcc4482d40 req-604b656d-18c2-456a-9883-4050f5b1070b service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:26.779752 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-f88cf271-ad51-4ddb-8a19-b3fcc4482d40 req-604b656d-18c2-456a-9883-4050f5b1070b service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:26.791914 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-f88cf271-ad51-4ddb-8a19-b3fcc4482d40 req-604b656d-18c2-456a-9883-4050f5b1070b service nova] Creating event network-vif-plugged:7ee252fe-6a69-4f90-9053-a7eabcf7750c for instance 148d5ad9-2900-4c2b-909b-aa446757e33d on nch2-master-1 Jul 16 21:09:26.798368 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-f88cf271-ad51-4ddb-8a19-b3fcc4482d40 req-604b656d-18c2-456a-9883-4050f5b1070b service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.032396 Jul 16 21:09:26.798977 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 189/361] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:09:26 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 16 21:09:26.804551 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-963c450d-5dd2-48a4-8e0a-cca6d8e8210a tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:26.805255 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-963c450d-5dd2-48a4-8e0a-cca6d8e8210a tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] [instance: 148d5ad9-2900-4c2b-909b-aa446757e33d] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:09:26.811671 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-963c450d-5dd2-48a4-8e0a-cca6d8e8210a tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:26.812138 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-963c450d-5dd2-48a4-8e0a-cca6d8e8210a tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:26.813800 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-963c450d-5dd2-48a4-8e0a-cca6d8e8210a tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:26.814488 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-963c450d-5dd2-48a4-8e0a-cca6d8e8210a tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:27.026458 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-963c450d-5dd2-48a4-8e0a-cca6d8e8210a tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'abeb7020019d41a18de42956d706a011', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '649babe54d104436af461f1ee4767001', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:27.037090 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-963c450d-5dd2-48a4-8e0a-cca6d8e8210a tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] 10.222.0.16 "GET /compute/v2.1/servers/148d5ad9-2900-4c2b-909b-aa446757e33d" status: 200 len: 1316 microversion: 2.1 time: 0.236604 Jul 16 21:09:27.038493 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 190/362] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:09:26 2022] GET /compute/v2.1/servers/148d5ad9-2900-4c2b-909b-aa446757e33d => generated 1316 bytes in 239 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:09:27.050770 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-982ab7f9-67a8-4f6c-85a5-e47f158a6801 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:27.054810 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-982ab7f9-67a8-4f6c-85a5-e47f158a6801 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: 1703f309-9e2e-4c5f-8588-eb4e43f21d91] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:09:27.070924 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-982ab7f9-67a8-4f6c-85a5-e47f158a6801 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:27.072011 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-982ab7f9-67a8-4f6c-85a5-e47f158a6801 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:27.074199 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-982ab7f9-67a8-4f6c-85a5-e47f158a6801 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:27.074900 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-982ab7f9-67a8-4f6c-85a5-e47f158a6801 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:27.096258 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=100 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:09:27.256688 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-982ab7f9-67a8-4f6c-85a5-e47f158a6801 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b68dacc4b8463ba07fae6372131117', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55a19b7a4682457a9132f5fc80110712', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:27.265258 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-982ab7f9-67a8-4f6c-85a5-e47f158a6801 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "GET /compute/v2.1/servers/1703f309-9e2e-4c5f-8588-eb4e43f21d91" status: 200 len: 1475 microversion: 2.1 time: 0.225336 Jul 16 21:09:27.266096 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 191/363] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:09:27 2022] GET /compute/v2.1/servers/1703f309-9e2e-4c5f-8588-eb4e43f21d91 => generated 1475 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:09:27.274636 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-82995a1b-ae2e-4896-83b5-9103a4f8976e tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:27.275285 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-82995a1b-ae2e-4896-83b5-9103a4f8976e tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] [instance: 1631f0cc-de67-4c0d-abd0-3e0e11dfb4ed] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:09:27.282536 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-82995a1b-ae2e-4896-83b5-9103a4f8976e tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:27.284610 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-82995a1b-ae2e-4896-83b5-9103a4f8976e tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:27.284610 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-82995a1b-ae2e-4896-83b5-9103a4f8976e tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:27.284800 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-82995a1b-ae2e-4896-83b5-9103a4f8976e tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:27.529776 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-82995a1b-ae2e-4896-83b5-9103a4f8976e tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0d72626f465b4f37ad3b18bb945d4000', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fc1c3463f9444119af9d18f85bdd703d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:27.552916 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-82995a1b-ae2e-4896-83b5-9103a4f8976e tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] 10.222.0.16 "GET /compute/v2.1/servers/1631f0cc-de67-4c0d-abd0-3e0e11dfb4ed" status: 200 len: 1318 microversion: 2.1 time: 0.284099 Jul 16 21:09:27.553816 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 192/364] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:09:27 2022] GET /compute/v2.1/servers/1631f0cc-de67-4c0d-abd0-3e0e11dfb4ed => generated 1318 bytes in 287 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:09:27.677022 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-4bce5701-bb46-4fcc-b437-3da0d2d2a152 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Flavor limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 16 21:09:27.677863 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-4bce5701-bb46-4fcc-b437-3da0d2d2a152 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Image limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 16 21:09:27.677978 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-4bce5701-bb46-4fcc-b437-3da0d2d2a152 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Flavor pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 16 21:09:27.678381 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-4bce5701-bb46-4fcc-b437-3da0d2d2a152 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Image pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 16 21:09:27.678920 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-4bce5701-bb46-4fcc-b437-3da0d2d2a152 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 16 21:09:27.693988 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4bce5701-bb46-4fcc-b437-3da0d2d2a152 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:27.694671 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4bce5701-bb46-4fcc-b437-3da0d2d2a152 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:27.695898 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4bce5701-bb46-4fcc-b437-3da0d2d2a152 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:27.696523 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4bce5701-bb46-4fcc-b437-3da0d2d2a152 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:27.737685 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-4bce5701-bb46-4fcc-b437-3da0d2d2a152 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Getting quotas for project 9b769bf8f3ea4e368bf68de1df6538f9. Resources: {'ram', 'instances', 'cores'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:09:27.743168 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-4bce5701-bb46-4fcc-b437-3da0d2d2a152 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Getting quotas for user 6a69f9dee03d43beaafa966216b9e289 and project 9b769bf8f3ea4e368bf68de1df6538f9. Resources: {'ram', 'instances', 'cores'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:09:27.756966 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-4bce5701-bb46-4fcc-b437-3da0d2d2a152 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Going to run 1 instances... {{(pid=138681) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 16 21:09:27.759350 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-4bce5701-bb46-4fcc-b437-3da0d2d2a152 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] [instance: c40ba20d-eb88-4ee8-be8f-ecbbd3315c48] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138681) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 16 21:09:27.917265 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-4bce5701-bb46-4fcc-b437-3da0d2d2a152 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 1.780668 Jul 16 21:09:27.917930 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 173/365] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 21:09:26 2022] POST /compute/v2.1/servers => generated 380 bytes in 1782 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Jul 16 21:09:27.932829 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-0e9ea113-e71a-420f-8cbf-bc755c5c6420 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:27.934013 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-0e9ea113-e71a-420f-8cbf-bc755c5c6420 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] [instance: c40ba20d-eb88-4ee8-be8f-ecbbd3315c48] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:09:28.057059 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-ca58baae-1aa0-47f4-9d9c-c5e8c175605e tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:28.057688 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-ca58baae-1aa0-47f4-9d9c-c5e8c175605e tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] [instance: 148d5ad9-2900-4c2b-909b-aa446757e33d] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:09:28.064695 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-0e9ea113-e71a-420f-8cbf-bc755c5c6420 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a69f9dee03d43beaafa966216b9e289', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9b769bf8f3ea4e368bf68de1df6538f9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:28.075682 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-0e9ea113-e71a-420f-8cbf-bc755c5c6420 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] 10.222.0.16 "GET /compute/v2.1/servers/c40ba20d-eb88-4ee8-be8f-ecbbd3315c48" status: 200 len: 1210 microversion: 2.1 time: 0.150372 Jul 16 21:09:28.083078 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 193/366] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:09:27 2022] GET /compute/v2.1/servers/c40ba20d-eb88-4ee8-be8f-ecbbd3315c48 => generated 1210 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:09:28.083244 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ca58baae-1aa0-47f4-9d9c-c5e8c175605e tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:28.083244 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ca58baae-1aa0-47f4-9d9c-c5e8c175605e tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:28.097401 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ca58baae-1aa0-47f4-9d9c-c5e8c175605e tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:28.097886 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ca58baae-1aa0-47f4-9d9c-c5e8c175605e tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.016s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:28.295361 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-306031c0-1de2-457f-80e8-aba5c4bb33eb tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:28.295991 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-306031c0-1de2-457f-80e8-aba5c4bb33eb tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: 1703f309-9e2e-4c5f-8588-eb4e43f21d91] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:09:28.309764 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-306031c0-1de2-457f-80e8-aba5c4bb33eb tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:28.310237 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-306031c0-1de2-457f-80e8-aba5c4bb33eb tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:28.312346 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-306031c0-1de2-457f-80e8-aba5c4bb33eb tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:28.313172 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-306031c0-1de2-457f-80e8-aba5c4bb33eb tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:28.556891 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-ca58baae-1aa0-47f4-9d9c-c5e8c175605e tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'abeb7020019d41a18de42956d706a011', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '649babe54d104436af461f1ee4767001', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:28.585388 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-ca58baae-1aa0-47f4-9d9c-c5e8c175605e tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] 10.222.0.16 "GET /compute/v2.1/servers/148d5ad9-2900-4c2b-909b-aa446757e33d" status: 200 len: 1316 microversion: 2.1 time: 0.532391 Jul 16 21:09:28.590082 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 174/367] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:09:28 2022] GET /compute/v2.1/servers/148d5ad9-2900-4c2b-909b-aa446757e33d => generated 1316 bytes in 538 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:09:28.601391 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-efba3996-3c18-42f9-ae3b-300cd3f94916 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:28.604183 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-efba3996-3c18-42f9-ae3b-300cd3f94916 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] [instance: 1631f0cc-de67-4c0d-abd0-3e0e11dfb4ed] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:09:28.615207 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-efba3996-3c18-42f9-ae3b-300cd3f94916 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:28.615901 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-efba3996-3c18-42f9-ae3b-300cd3f94916 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:28.616938 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-efba3996-3c18-42f9-ae3b-300cd3f94916 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:28.617524 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-efba3996-3c18-42f9-ae3b-300cd3f94916 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:29.053571 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-306031c0-1de2-457f-80e8-aba5c4bb33eb tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b68dacc4b8463ba07fae6372131117', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55a19b7a4682457a9132f5fc80110712', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:29.063337 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-306031c0-1de2-457f-80e8-aba5c4bb33eb tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "GET /compute/v2.1/servers/1703f309-9e2e-4c5f-8588-eb4e43f21d91" status: 200 len: 1475 microversion: 2.1 time: 0.783717 Jul 16 21:09:29.074402 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 194/368] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:09:28 2022] GET /compute/v2.1/servers/1703f309-9e2e-4c5f-8588-eb4e43f21d91 => generated 1475 bytes in 793 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:09:29.094558 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-1fff40f1-1a23-4bc0-b430-0de7394bab67 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:29.095990 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-1fff40f1-1a23-4bc0-b430-0de7394bab67 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] [instance: c40ba20d-eb88-4ee8-be8f-ecbbd3315c48] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:09:29.176653 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-efba3996-3c18-42f9-ae3b-300cd3f94916 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0d72626f465b4f37ad3b18bb945d4000', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fc1c3463f9444119af9d18f85bdd703d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:29.187171 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-efba3996-3c18-42f9-ae3b-300cd3f94916 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] 10.222.0.16 "GET /compute/v2.1/servers/1631f0cc-de67-4c0d-abd0-3e0e11dfb4ed" status: 200 len: 1318 microversion: 2.1 time: 0.595620 Jul 16 21:09:29.203320 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 175/369] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:09:28 2022] GET /compute/v2.1/servers/1631f0cc-de67-4c0d-abd0-3e0e11dfb4ed => generated 1318 bytes in 612 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:09:29.254140 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-1fff40f1-1a23-4bc0-b430-0de7394bab67 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a69f9dee03d43beaafa966216b9e289', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9b769bf8f3ea4e368bf68de1df6538f9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:29.261647 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-1fff40f1-1a23-4bc0-b430-0de7394bab67 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] 10.222.0.16 "GET /compute/v2.1/servers/c40ba20d-eb88-4ee8-be8f-ecbbd3315c48" status: 200 len: 1210 microversion: 2.1 time: 0.171463 Jul 16 21:09:29.263646 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 195/370] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:09:29 2022] GET /compute/v2.1/servers/c40ba20d-eb88-4ee8-be8f-ecbbd3315c48 => generated 1210 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:09:29.605777 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-c70d9fce-316b-466f-9983-bd9fbf13920f tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:29.606396 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-c70d9fce-316b-466f-9983-bd9fbf13920f tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] [instance: 148d5ad9-2900-4c2b-909b-aa446757e33d] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:09:29.614798 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c70d9fce-316b-466f-9983-bd9fbf13920f tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:29.615353 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c70d9fce-316b-466f-9983-bd9fbf13920f tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:29.616197 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c70d9fce-316b-466f-9983-bd9fbf13920f tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:29.616647 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c70d9fce-316b-466f-9983-bd9fbf13920f tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:29.795057 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-c70d9fce-316b-466f-9983-bd9fbf13920f tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'abeb7020019d41a18de42956d706a011', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '649babe54d104436af461f1ee4767001', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:29.805760 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-c70d9fce-316b-466f-9983-bd9fbf13920f tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] 10.222.0.16 "GET /compute/v2.1/servers/148d5ad9-2900-4c2b-909b-aa446757e33d" status: 200 len: 1466 microversion: 2.1 time: 0.203830 Jul 16 21:09:29.806357 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 196/371] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:09:29 2022] GET /compute/v2.1/servers/148d5ad9-2900-4c2b-909b-aa446757e33d => generated 1466 bytes in 205 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:09:29.992300 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-0bedb7e1-9b54-4603-bdce-6c1995fc7553 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServersAdmin275Test-server-1828812776", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452", "networks": "none"}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:09:30.052808 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-0bedb7e1-9b54-4603-bdce-6c1995fc7553 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Flavor limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 16 21:09:30.053248 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-0bedb7e1-9b54-4603-bdce-6c1995fc7553 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Image limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 16 21:09:30.053726 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-0bedb7e1-9b54-4603-bdce-6c1995fc7553 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Flavor pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 16 21:09:30.054183 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-0bedb7e1-9b54-4603-bdce-6c1995fc7553 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Image pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 16 21:09:30.054833 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-0bedb7e1-9b54-4603-bdce-6c1995fc7553 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 16 21:09:30.065800 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0bedb7e1-9b54-4603-bdce-6c1995fc7553 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:30.066512 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0bedb7e1-9b54-4603-bdce-6c1995fc7553 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:30.072076 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0bedb7e1-9b54-4603-bdce-6c1995fc7553 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:30.072076 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0bedb7e1-9b54-4603-bdce-6c1995fc7553 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:30.089245 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-0bedb7e1-9b54-4603-bdce-6c1995fc7553 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Getting quotas for project fb6deae6b08e4ed8b15dc6edfe1d38d3. Resources: {'ram', 'instances', 'cores'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:09:30.093310 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-0bedb7e1-9b54-4603-bdce-6c1995fc7553 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Getting quotas for user ba60685a2001459cbfb531d95a6b296e and project fb6deae6b08e4ed8b15dc6edfe1d38d3. Resources: {'ram', 'instances', 'cores'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:09:30.107819 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-0bedb7e1-9b54-4603-bdce-6c1995fc7553 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Going to run 1 instances... {{(pid=138681) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 16 21:09:30.109694 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-0bedb7e1-9b54-4603-bdce-6c1995fc7553 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] [instance: cf53f7cd-deae-4b34-bf8e-3afef98b2ec6] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138681) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 16 21:09:30.292929 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-0bedb7e1-9b54-4603-bdce-6c1995fc7553 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.75 time: 1.046772 Jul 16 21:09:30.294866 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 176/372] 10.222.0.16 () {62 vars in 1187 bytes} [Sat Jul 16 21:09:29 2022] POST /compute/v2.1/servers => generated 380 bytes in 1050 msecs (HTTP/1.1 202) 10 headers in 453 bytes (1 switches on core 0) Jul 16 21:09:30.303075 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-1eab2893-14f0-4af8-86e2-9f8a1f5d609e tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:30.304950 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-1eab2893-14f0-4af8-86e2-9f8a1f5d609e tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: 1703f309-9e2e-4c5f-8588-eb4e43f21d91] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:09:30.329403 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1eab2893-14f0-4af8-86e2-9f8a1f5d609e tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:30.331129 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1eab2893-14f0-4af8-86e2-9f8a1f5d609e tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:30.332896 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1eab2893-14f0-4af8-86e2-9f8a1f5d609e tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:30.333839 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1eab2893-14f0-4af8-86e2-9f8a1f5d609e tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:30.535930 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-dec9990b-d1d9-440e-9b56-db7a4204e954 tempest-ServersOnMultiNodesTest-83775055 tempest-ServersOnMultiNodesTest-83775055-project-admin] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:30.538370 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-dec9990b-d1d9-440e-9b56-db7a4204e954 tempest-ServersOnMultiNodesTest-83775055 tempest-ServersOnMultiNodesTest-83775055-project-admin] [instance: 148d5ad9-2900-4c2b-909b-aa446757e33d] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:09:30.558906 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-1eab2893-14f0-4af8-86e2-9f8a1f5d609e tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b68dacc4b8463ba07fae6372131117', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55a19b7a4682457a9132f5fc80110712', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:30.570374 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_api:SELECT=89,nova_api:INSERT=2,nova_api:DELETE=9 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:09:30.573533 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-dec9990b-d1d9-440e-9b56-db7a4204e954 tempest-ServersOnMultiNodesTest-83775055 tempest-ServersOnMultiNodesTest-83775055-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:30.578289 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-dec9990b-d1d9-440e-9b56-db7a4204e954 tempest-ServersOnMultiNodesTest-83775055 tempest-ServersOnMultiNodesTest-83775055-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:30.604826 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-dec9990b-d1d9-440e-9b56-db7a4204e954 tempest-ServersOnMultiNodesTest-83775055 tempest-ServersOnMultiNodesTest-83775055-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:30.604986 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-1eab2893-14f0-4af8-86e2-9f8a1f5d609e tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "GET /compute/v2.1/servers/1703f309-9e2e-4c5f-8588-eb4e43f21d91" status: 200 len: 1475 microversion: 2.1 time: 0.290169 Jul 16 21:09:30.605149 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-dec9990b-d1d9-440e-9b56-db7a4204e954 tempest-ServersOnMultiNodesTest-83775055 tempest-ServersOnMultiNodesTest-83775055-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:30.605286 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 177/373] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:09:30 2022] GET /compute/v2.1/servers/1703f309-9e2e-4c5f-8588-eb4e43f21d91 => generated 1475 bytes in 297 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:09:30.619569 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-4c543c73-a9fd-4bb7-b70c-b00a60c22f33 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:30.621762 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-4c543c73-a9fd-4bb7-b70c-b00a60c22f33 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] [instance: 1631f0cc-de67-4c0d-abd0-3e0e11dfb4ed] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:09:30.638087 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4c543c73-a9fd-4bb7-b70c-b00a60c22f33 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:30.652293 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4c543c73-a9fd-4bb7-b70c-b00a60c22f33 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:30.652293 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4c543c73-a9fd-4bb7-b70c-b00a60c22f33 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:30.652293 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4c543c73-a9fd-4bb7-b70c-b00a60c22f33 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:30.919388 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-4c543c73-a9fd-4bb7-b70c-b00a60c22f33 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0d72626f465b4f37ad3b18bb945d4000', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fc1c3463f9444119af9d18f85bdd703d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:30.931795 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-4c543c73-a9fd-4bb7-b70c-b00a60c22f33 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] 10.222.0.16 "GET /compute/v2.1/servers/1631f0cc-de67-4c0d-abd0-3e0e11dfb4ed" status: 200 len: 1318 microversion: 2.1 time: 0.331455 Jul 16 21:09:30.933083 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 178/374] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:09:30 2022] GET /compute/v2.1/servers/1631f0cc-de67-4c0d-abd0-3e0e11dfb4ed => generated 1318 bytes in 334 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:09:30.941536 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-f1ad0b46-f491-451b-9f55-d8a69d40f878 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:30.943306 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-f1ad0b46-f491-451b-9f55-d8a69d40f878 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] [instance: c40ba20d-eb88-4ee8-be8f-ecbbd3315c48] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:09:30.950076 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f1ad0b46-f491-451b-9f55-d8a69d40f878 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:30.950524 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f1ad0b46-f491-451b-9f55-d8a69d40f878 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:30.951320 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f1ad0b46-f491-451b-9f55-d8a69d40f878 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:30.951930 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f1ad0b46-f491-451b-9f55-d8a69d40f878 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:31.037258 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-f1ad0b46-f491-451b-9f55-d8a69d40f878 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a69f9dee03d43beaafa966216b9e289', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9b769bf8f3ea4e368bf68de1df6538f9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:31.046379 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-f1ad0b46-f491-451b-9f55-d8a69d40f878 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] 10.222.0.16 "GET /compute/v2.1/servers/c40ba20d-eb88-4ee8-be8f-ecbbd3315c48" status: 200 len: 1262 microversion: 2.1 time: 0.111264 Jul 16 21:09:31.046881 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 179/375] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:09:30 2022] GET /compute/v2.1/servers/c40ba20d-eb88-4ee8-be8f-ecbbd3315c48 => generated 1262 bytes in 113 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:09:31.052184 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-ca3e1208-baff-4c5e-9192-90d6e54b0d60 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Action: 'update', calling method: >, body: {"server": {}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:09:31.053360 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-ca3e1208-baff-4c5e-9192-90d6e54b0d60 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] [instance: cf53f7cd-deae-4b34-bf8e-3afef98b2ec6] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:09:31.129196 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-ca3e1208-baff-4c5e-9192-90d6e54b0d60 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ba60685a2001459cbfb531d95a6b296e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb6deae6b08e4ed8b15dc6edfe1d38d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:31.137458 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-ca3e1208-baff-4c5e-9192-90d6e54b0d60 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ba60685a2001459cbfb531d95a6b296e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb6deae6b08e4ed8b15dc6edfe1d38d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:31.139028 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-ca3e1208-baff-4c5e-9192-90d6e54b0d60 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ba60685a2001459cbfb531d95a6b296e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb6deae6b08e4ed8b15dc6edfe1d38d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:31.147054 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-ca3e1208-baff-4c5e-9192-90d6e54b0d60 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] 10.222.0.16 "PUT /compute/v2.1/servers/cf53f7cd-deae-4b34-bf8e-3afef98b2ec6" status: 200 len: 1359 microversion: 2.75 time: 0.098903 Jul 16 21:09:31.159887 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 180/376] 10.222.0.16 () {62 vars in 1296 bytes} [Sat Jul 16 21:09:31 2022] PUT /compute/v2.1/servers/cf53f7cd-deae-4b34-bf8e-3afef98b2ec6 => generated 1359 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:09:31.159887 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-f6431bf7-1eaf-4c87-ac6f-b09811a6499e req-ad7cdd2c-359c-4423-ab65-0cdfad3c82e8 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "1631f0cc-de67-4c0d-abd0-3e0e11dfb4ed", "name": "network-vif-plugged", "status": "completed", "tag": "e43e8868-6c5f-4776-ab31-fb29f6dcf4bf"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:09:31.178868 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-f6431bf7-1eaf-4c87-ac6f-b09811a6499e req-ad7cdd2c-359c-4423-ab65-0cdfad3c82e8 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:31.178868 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-f6431bf7-1eaf-4c87-ac6f-b09811a6499e req-ad7cdd2c-359c-4423-ab65-0cdfad3c82e8 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:31.191458 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-f6431bf7-1eaf-4c87-ac6f-b09811a6499e req-ad7cdd2c-359c-4423-ab65-0cdfad3c82e8 service nova] Creating event network-vif-plugged:e43e8868-6c5f-4776-ab31-fb29f6dcf4bf for instance 1631f0cc-de67-4c0d-abd0-3e0e11dfb4ed on nch2-master-1 Jul 16 21:09:31.223149 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-f6431bf7-1eaf-4c87-ac6f-b09811a6499e req-ad7cdd2c-359c-4423-ab65-0cdfad3c82e8 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.073715 Jul 16 21:09:31.223806 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 181/377] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:09:31 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 75 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:09:31.246491 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-fe4b1dfd-a7d2-450b-9643-ea55bcc6966a tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:31.255965 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-fe4b1dfd-a7d2-450b-9643-ea55bcc6966a tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] [instance: cf53f7cd-deae-4b34-bf8e-3afef98b2ec6] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:09:31.297601 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-dec9990b-d1d9-440e-9b56-db7a4204e954 tempest-ServersOnMultiNodesTest-83775055 tempest-ServersOnMultiNodesTest-83775055-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/148d5ad9-2900-4c2b-909b-aa446757e33d" status: 200 len: 1617 microversion: 2.1 time: 1.476950 Jul 16 21:09:31.311738 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 197/378] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:09:29 2022] GET /compute/v2.1/servers/148d5ad9-2900-4c2b-909b-aa446757e33d => generated 1617 bytes in 1492 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:09:31.320595 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-d1900060-091d-426d-bcbb-823feaf84815 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Action: 'create', calling method: .version_select at 0x7fdd9d9cb1f0>, body: {"server_group": {"name": "tempest-ServersOnMultiNodesTest-Server-Group-1963440849", "policies": ["affinity"]}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:09:31.365204 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-d1900060-091d-426d-bcbb-823feaf84815 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Getting quotas for project 649babe54d104436af461f1ee4767001. Resources: {'server_groups'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:09:31.370584 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-d1900060-091d-426d-bcbb-823feaf84815 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Getting quotas for user abeb7020019d41a18de42956d706a011 and project 649babe54d104436af461f1ee4767001. Resources: {'server_groups'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:09:31.405622 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-fe4b1dfd-a7d2-450b-9643-ea55bcc6966a tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ba60685a2001459cbfb531d95a6b296e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb6deae6b08e4ed8b15dc6edfe1d38d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:31.417518 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-fe4b1dfd-a7d2-450b-9643-ea55bcc6966a tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ba60685a2001459cbfb531d95a6b296e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb6deae6b08e4ed8b15dc6edfe1d38d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:31.420845 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-fe4b1dfd-a7d2-450b-9643-ea55bcc6966a tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ba60685a2001459cbfb531d95a6b296e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb6deae6b08e4ed8b15dc6edfe1d38d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:31.428762 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-fe4b1dfd-a7d2-450b-9643-ea55bcc6966a tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] 10.222.0.16 "GET /compute/v2.1/servers/cf53f7cd-deae-4b34-bf8e-3afef98b2ec6" status: 200 len: 1359 microversion: 2.75 time: 0.195922 Jul 16 21:09:31.429788 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 182/379] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:09:31 2022] GET /compute/v2.1/servers/cf53f7cd-deae-4b34-bf8e-3afef98b2ec6 => generated 1359 bytes in 205 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:09:31.545286 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-d1900060-091d-426d-bcbb-823feaf84815 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Getting quotas for project 649babe54d104436af461f1ee4767001. Resources: {'server_groups'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:09:31.551615 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-d1900060-091d-426d-bcbb-823feaf84815 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Getting quotas for user abeb7020019d41a18de42956d706a011 and project 649babe54d104436af461f1ee4767001. Resources: {'server_groups'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:09:31.565088 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-d1900060-091d-426d-bcbb-823feaf84815 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] 10.222.0.16 "POST /compute/v2.1/os-server-groups" status: 200 len: 188 microversion: 2.1 time: 0.251760 Jul 16 21:09:31.565691 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 198/380] 10.222.0.16 () {60 vars in 1173 bytes} [Sat Jul 16 21:09:31 2022] POST /compute/v2.1/os-server-groups => generated 188 bytes in 253 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:09:31.606741 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-169db04d-6974-4f4d-8e4e-a5cd61912054 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:31.607526 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-169db04d-6974-4f4d-8e4e-a5cd61912054 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: 1703f309-9e2e-4c5f-8588-eb4e43f21d91] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:09:31.614787 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-169db04d-6974-4f4d-8e4e-a5cd61912054 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:31.615296 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-169db04d-6974-4f4d-8e4e-a5cd61912054 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:31.616355 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-169db04d-6974-4f4d-8e4e-a5cd61912054 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:31.616859 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-169db04d-6974-4f4d-8e4e-a5cd61912054 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:31.702300 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-57f86eb7-5a91-4648-868e-91a5a0109834 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServersOnMultiNodesTest-server-2009718450", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452", "min_count": 2, "return_reservation_id": true}, "os:scheduler_hints": {"group": "73f6bd9e-6a32-47fd-a7a1-a896ecc028e4"}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:09:31.763781 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.network.neutron [None req-57f86eb7-5a91-4648-868e-91a5a0109834 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] validate_networks() for None {{(pid=138680) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 16 21:09:31.815221 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-169db04d-6974-4f4d-8e4e-a5cd61912054 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b68dacc4b8463ba07fae6372131117', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55a19b7a4682457a9132f5fc80110712', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:31.823202 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=100 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:09:31.830962 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-169db04d-6974-4f4d-8e4e-a5cd61912054 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "GET /compute/v2.1/servers/1703f309-9e2e-4c5f-8588-eb4e43f21d91" status: 200 len: 1475 microversion: 2.1 time: 0.227683 Jul 16 21:09:31.842174 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 183/381] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:09:31 2022] GET /compute/v2.1/servers/1703f309-9e2e-4c5f-8588-eb4e43f21d91 => generated 1475 bytes in 231 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:09:31.948536 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-b661b713-8055-4915-8656-82032159de0f tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:31.949154 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-b661b713-8055-4915-8656-82032159de0f tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] [instance: 1631f0cc-de67-4c0d-abd0-3e0e11dfb4ed] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:09:31.961003 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b661b713-8055-4915-8656-82032159de0f tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:31.961469 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b661b713-8055-4915-8656-82032159de0f tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:31.962268 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b661b713-8055-4915-8656-82032159de0f tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:31.962710 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b661b713-8055-4915-8656-82032159de0f tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:32.181639 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-57f86eb7-5a91-4648-868e-91a5a0109834 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Flavor limits 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 16 21:09:32.182158 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-57f86eb7-5a91-4648-868e-91a5a0109834 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Image limits 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 16 21:09:32.182696 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-57f86eb7-5a91-4648-868e-91a5a0109834 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Flavor pref 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 16 21:09:32.183147 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-57f86eb7-5a91-4648-868e-91a5a0109834 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Image pref 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 16 21:09:32.183772 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-57f86eb7-5a91-4648-868e-91a5a0109834 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 16 21:09:32.213261 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-57f86eb7-5a91-4648-868e-91a5a0109834 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:32.213907 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-57f86eb7-5a91-4648-868e-91a5a0109834 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:32.214908 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-57f86eb7-5a91-4648-868e-91a5a0109834 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:32.215510 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-57f86eb7-5a91-4648-868e-91a5a0109834 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:32.239709 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-b661b713-8055-4915-8656-82032159de0f tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0d72626f465b4f37ad3b18bb945d4000', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fc1c3463f9444119af9d18f85bdd703d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:32.240347 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-57f86eb7-5a91-4648-868e-91a5a0109834 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Getting quotas for project 649babe54d104436af461f1ee4767001. Resources: {'ram', 'instances', 'cores'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:09:32.246770 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-57f86eb7-5a91-4648-868e-91a5a0109834 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Getting quotas for user abeb7020019d41a18de42956d706a011 and project 649babe54d104436af461f1ee4767001. Resources: {'ram', 'instances', 'cores'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:09:32.252245 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-b661b713-8055-4915-8656-82032159de0f tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] 10.222.0.16 "GET /compute/v2.1/servers/1631f0cc-de67-4c0d-abd0-3e0e11dfb4ed" status: 200 len: 1318 microversion: 2.1 time: 0.306682 Jul 16 21:09:32.254573 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 184/382] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:09:31 2022] GET /compute/v2.1/servers/1631f0cc-de67-4c0d-abd0-3e0e11dfb4ed => generated 1318 bytes in 310 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:09:32.260400 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-2270ad40-3291-4958-9d91-844026a1746f tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:32.262118 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-2270ad40-3291-4958-9d91-844026a1746f tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] [instance: c40ba20d-eb88-4ee8-be8f-ecbbd3315c48] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:09:32.272112 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2270ad40-3291-4958-9d91-844026a1746f tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:32.272939 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2270ad40-3291-4958-9d91-844026a1746f tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:32.274070 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2270ad40-3291-4958-9d91-844026a1746f tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:32.274864 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2270ad40-3291-4958-9d91-844026a1746f tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:32.275427 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-57f86eb7-5a91-4648-868e-91a5a0109834 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Going to run 2 instances... {{(pid=138680) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 16 21:09:32.277880 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-57f86eb7-5a91-4648-868e-91a5a0109834 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] [instance: 45164bce-7acf-4ef6-8afa-aaab6287ecf4] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138680) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 16 21:09:32.353219 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-2270ad40-3291-4958-9d91-844026a1746f tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a69f9dee03d43beaafa966216b9e289', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9b769bf8f3ea4e368bf68de1df6538f9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:32.362415 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-2270ad40-3291-4958-9d91-844026a1746f tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] 10.222.0.16 "GET /compute/v2.1/servers/c40ba20d-eb88-4ee8-be8f-ecbbd3315c48" status: 200 len: 1268 microversion: 2.1 time: 0.105654 Jul 16 21:09:32.362873 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 185/383] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:09:32 2022] GET /compute/v2.1/servers/c40ba20d-eb88-4ee8-be8f-ecbbd3315c48 => generated 1268 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:09:32.382894 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-57f86eb7-5a91-4648-868e-91a5a0109834 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Checking whether user_id and queued_for_delete are populated for all projects {{(pid=138680) is_qfd_populated /opt/stack/nova/nova/quota.py:1268}} Jul 16 21:09:32.437797 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-57f86eb7-5a91-4648-868e-91a5a0109834 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Getting quotas for project 649babe54d104436af461f1ee4767001. Resources: {'server_group_members'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:09:32.449764 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-4cfe3ea4-1fd1-4db9-b917-938dcf6e1825 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:32.459845 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-4cfe3ea4-1fd1-4db9-b917-938dcf6e1825 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] [instance: cf53f7cd-deae-4b34-bf8e-3afef98b2ec6] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:09:32.459845 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4cfe3ea4-1fd1-4db9-b917-938dcf6e1825 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:32.459845 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4cfe3ea4-1fd1-4db9-b917-938dcf6e1825 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:32.460285 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4cfe3ea4-1fd1-4db9-b917-938dcf6e1825 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:32.460540 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4cfe3ea4-1fd1-4db9-b917-938dcf6e1825 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:32.464452 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-57f86eb7-5a91-4648-868e-91a5a0109834 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Getting quotas for user abeb7020019d41a18de42956d706a011 and project 649babe54d104436af461f1ee4767001. Resources: {'server_group_members'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:09:32.584924 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-4cfe3ea4-1fd1-4db9-b917-938dcf6e1825 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ba60685a2001459cbfb531d95a6b296e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb6deae6b08e4ed8b15dc6edfe1d38d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:32.593555 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-4cfe3ea4-1fd1-4db9-b917-938dcf6e1825 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ba60685a2001459cbfb531d95a6b296e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb6deae6b08e4ed8b15dc6edfe1d38d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:32.595760 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-4cfe3ea4-1fd1-4db9-b917-938dcf6e1825 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ba60685a2001459cbfb531d95a6b296e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb6deae6b08e4ed8b15dc6edfe1d38d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:32.705663 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-4cfe3ea4-1fd1-4db9-b917-938dcf6e1825 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] 10.222.0.16 "GET /compute/v2.1/servers/cf53f7cd-deae-4b34-bf8e-3afef98b2ec6" status: 200 len: 1363 microversion: 2.75 time: 0.157425 Jul 16 21:09:32.705663 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 186/384] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:09:32 2022] GET /compute/v2.1/servers/cf53f7cd-deae-4b34-bf8e-3afef98b2ec6 => generated 1363 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 16 21:09:32.791520 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-57f86eb7-5a91-4648-868e-91a5a0109834 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Getting quotas for project 649babe54d104436af461f1ee4767001. Resources: {'server_group_members'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:09:32.797068 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-57f86eb7-5a91-4648-868e-91a5a0109834 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Getting quotas for user abeb7020019d41a18de42956d706a011 and project 649babe54d104436af461f1ee4767001. Resources: {'server_group_members'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:09:32.811801 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-57f86eb7-5a91-4648-868e-91a5a0109834 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] [instance: 944f68af-bf99-428f-9357-9ce9f807483b] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9',instance=,instance_uuid=45164bce-7acf-4ef6-8afa-aaab6287ecf4,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138680) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 16 21:09:32.850223 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-540bbb35-320f-471c-8cd0-396723e3db31 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:32.850945 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-540bbb35-320f-471c-8cd0-396723e3db31 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: 1703f309-9e2e-4c5f-8588-eb4e43f21d91] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:09:32.857240 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-540bbb35-320f-471c-8cd0-396723e3db31 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:32.858089 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-540bbb35-320f-471c-8cd0-396723e3db31 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:32.858874 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-540bbb35-320f-471c-8cd0-396723e3db31 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:32.859346 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-540bbb35-320f-471c-8cd0-396723e3db31 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:32.998809 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-540bbb35-320f-471c-8cd0-396723e3db31 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b68dacc4b8463ba07fae6372131117', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55a19b7a4682457a9132f5fc80110712', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:33.007101 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-540bbb35-320f-471c-8cd0-396723e3db31 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "GET /compute/v2.1/servers/1703f309-9e2e-4c5f-8588-eb4e43f21d91" status: 200 len: 1475 microversion: 2.1 time: 0.161687 Jul 16 21:09:33.008300 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 187/385] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:09:32 2022] GET /compute/v2.1/servers/1703f309-9e2e-4c5f-8588-eb4e43f21d91 => generated 1475 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:09:33.074203 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-57f86eb7-5a91-4648-868e-91a5a0109834 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Getting quotas for project 649babe54d104436af461f1ee4767001. Resources: {'server_group_members'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:09:33.082582 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-57f86eb7-5a91-4648-868e-91a5a0109834 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Getting quotas for user abeb7020019d41a18de42956d706a011 and project 649babe54d104436af461f1ee4767001. Resources: {'server_group_members'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:09:33.271557 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-35c77d2e-344b-42e0-bf13-f23eb83640d8 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:33.272545 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-35c77d2e-344b-42e0-bf13-f23eb83640d8 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] [instance: 1631f0cc-de67-4c0d-abd0-3e0e11dfb4ed] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:09:33.282794 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-35c77d2e-344b-42e0-bf13-f23eb83640d8 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:33.287184 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-57f86eb7-5a91-4648-868e-91a5a0109834 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Getting quotas for project 649babe54d104436af461f1ee4767001. Resources: {'server_group_members'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:09:33.287324 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-35c77d2e-344b-42e0-bf13-f23eb83640d8 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:33.287324 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-35c77d2e-344b-42e0-bf13-f23eb83640d8 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:33.287324 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-35c77d2e-344b-42e0-bf13-f23eb83640d8 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:33.292189 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-57f86eb7-5a91-4648-868e-91a5a0109834 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Getting quotas for user abeb7020019d41a18de42956d706a011 and project 649babe54d104436af461f1ee4767001. Resources: {'server_group_members'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:09:33.342180 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-57f86eb7-5a91-4648-868e-91a5a0109834 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 32 microversion: 2.1 time: 1.643737 Jul 16 21:09:33.343248 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 199/386] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 21:09:31 2022] POST /compute/v2.1/servers => generated 32 bytes in 1645 msecs (HTTP/1.1 202) 9 headers in 362 bytes (1 switches on core 0) Jul 16 21:09:33.354257 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-4271e804-9b59-4e4c-9335-9d4287bd9cc8 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:33.357943 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-4271e804-9b59-4e4c-9335-9d4287bd9cc8 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Policy check for os_compute_api:servers:allow_all_filters failed with credentials {'is_admin': False, 'user_id': 'abeb7020019d41a18de42956d706a011', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '649babe54d104436af461f1ee4767001', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:33.358736 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-4271e804-9b59-4e4c-9335-9d4287bd9cc8 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Searching by: {'deleted': False, 'project_id': '649babe54d104436af461f1ee4767001'} {{(pid=138680) get_all /opt/stack/nova/nova/compute/api.py:2962}} Jul 16 21:09:33.387386 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4271e804-9b59-4e4c-9335-9d4287bd9cc8 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:33.387386 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4271e804-9b59-4e4c-9335-9d4287bd9cc8 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:33.387386 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4271e804-9b59-4e4c-9335-9d4287bd9cc8 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:33.387386 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4271e804-9b59-4e4c-9335-9d4287bd9cc8 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:33.531657 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.multi_cell_list [None req-4271e804-9b59-4e4c-9335-9d4287bd9cc8 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Listed batch of 1 results from cell out of 998 limit. Returned 1 total so far. {{(pid=138680) do_query /opt/stack/nova/nova/compute/multi_cell_list.py:373}} Jul 16 21:09:33.565714 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-35c77d2e-344b-42e0-bf13-f23eb83640d8 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0d72626f465b4f37ad3b18bb945d4000', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fc1c3463f9444119af9d18f85bdd703d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:33.578470 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-35c77d2e-344b-42e0-bf13-f23eb83640d8 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] 10.222.0.16 "GET /compute/v2.1/servers/1631f0cc-de67-4c0d-abd0-3e0e11dfb4ed" status: 200 len: 1468 microversion: 2.1 time: 0.311545 Jul 16 21:09:33.587095 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 188/387] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:09:33 2022] GET /compute/v2.1/servers/1631f0cc-de67-4c0d-abd0-3e0e11dfb4ed => generated 1468 bytes in 313 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:09:33.591385 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-b0b1465b-7f02-4055-949b-ae7cffa33004 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:33.593493 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-b0b1465b-7f02-4055-949b-ae7cffa33004 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] [instance: c40ba20d-eb88-4ee8-be8f-ecbbd3315c48] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:09:33.601204 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b0b1465b-7f02-4055-949b-ae7cffa33004 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:33.603518 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b0b1465b-7f02-4055-949b-ae7cffa33004 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:33.603518 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b0b1465b-7f02-4055-949b-ae7cffa33004 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:33.603518 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b0b1465b-7f02-4055-949b-ae7cffa33004 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:33.714990 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-4271e804-9b59-4e4c-9335-9d4287bd9cc8 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] 10.222.0.16 "GET /compute/v2.1/servers" status: 200 len: 1003 microversion: 2.1 time: 0.364630 Jul 16 21:09:33.715976 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 200/388] 10.222.0.16 () {58 vars in 1124 bytes} [Sat Jul 16 21:09:33 2022] GET /compute/v2.1/servers => generated 1003 bytes in 366 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:09:33.785543 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-b0b1465b-7f02-4055-949b-ae7cffa33004 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a69f9dee03d43beaafa966216b9e289', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9b769bf8f3ea4e368bf68de1df6538f9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:33.793334 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-b0b1465b-7f02-4055-949b-ae7cffa33004 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] 10.222.0.16 "GET /compute/v2.1/servers/c40ba20d-eb88-4ee8-be8f-ecbbd3315c48" status: 200 len: 1310 microversion: 2.1 time: 0.208566 Jul 16 21:09:33.793900 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 189/389] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:09:33 2022] GET /compute/v2.1/servers/c40ba20d-eb88-4ee8-be8f-ecbbd3315c48 => generated 1310 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:09:33.799845 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-0961e340-449a-44ad-af6b-44c58bb67492 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:33.800463 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-0961e340-449a-44ad-af6b-44c58bb67492 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] [instance: cf53f7cd-deae-4b34-bf8e-3afef98b2ec6] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:09:33.806468 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0961e340-449a-44ad-af6b-44c58bb67492 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:33.806917 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0961e340-449a-44ad-af6b-44c58bb67492 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:33.807724 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0961e340-449a-44ad-af6b-44c58bb67492 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:33.808377 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0961e340-449a-44ad-af6b-44c58bb67492 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:33.949749 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-0961e340-449a-44ad-af6b-44c58bb67492 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ba60685a2001459cbfb531d95a6b296e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb6deae6b08e4ed8b15dc6edfe1d38d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:33.966695 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-0961e340-449a-44ad-af6b-44c58bb67492 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ba60685a2001459cbfb531d95a6b296e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb6deae6b08e4ed8b15dc6edfe1d38d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:33.972929 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-0961e340-449a-44ad-af6b-44c58bb67492 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ba60685a2001459cbfb531d95a6b296e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb6deae6b08e4ed8b15dc6edfe1d38d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:33.983610 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-0961e340-449a-44ad-af6b-44c58bb67492 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] 10.222.0.16 "GET /compute/v2.1/servers/cf53f7cd-deae-4b34-bf8e-3afef98b2ec6" status: 200 len: 1411 microversion: 2.75 time: 0.188586 Jul 16 21:09:33.986347 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 190/390] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:09:33 2022] GET /compute/v2.1/servers/cf53f7cd-deae-4b34-bf8e-3afef98b2ec6 => generated 1411 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:09:33.993919 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-962c835f-bf0c-47d0-aaf9-f4b871bb4c55 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:33.994900 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-962c835f-bf0c-47d0-aaf9-f4b871bb4c55 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] [instance: 944f68af-bf99-428f-9357-9ce9f807483b] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:09:34.075189 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-c2fc02b3-5ae0-40cf-b738-856f068b50c8 tempest-LiveMigrationNegativeTest-1294767845 tempest-LiveMigrationNegativeTest-1294767845-project-admin] Action: 'action', calling method: >, body: {"os-migrateLive": {"host": "tempest-host-1224354205", "block_migration": true, "disk_over_commit": false}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:09:34.076531 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-c2fc02b3-5ae0-40cf-b738-856f068b50c8 tempest-LiveMigrationNegativeTest-1294767845 tempest-LiveMigrationNegativeTest-1294767845-project-admin] [instance: 1631f0cc-de67-4c0d-abd0-3e0e11dfb4ed] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:09:34.083625 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c2fc02b3-5ae0-40cf-b738-856f068b50c8 tempest-LiveMigrationNegativeTest-1294767845 tempest-LiveMigrationNegativeTest-1294767845-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:34.084201 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c2fc02b3-5ae0-40cf-b738-856f068b50c8 tempest-LiveMigrationNegativeTest-1294767845 tempest-LiveMigrationNegativeTest-1294767845-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:34.085089 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c2fc02b3-5ae0-40cf-b738-856f068b50c8 tempest-LiveMigrationNegativeTest-1294767845 tempest-LiveMigrationNegativeTest-1294767845-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:34.085881 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c2fc02b3-5ae0-40cf-b738-856f068b50c8 tempest-LiveMigrationNegativeTest-1294767845 tempest-LiveMigrationNegativeTest-1294767845-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:34.161232 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-962c835f-bf0c-47d0-aaf9-f4b871bb4c55 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'abeb7020019d41a18de42956d706a011', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '649babe54d104436af461f1ee4767001', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:34.172372 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-962c835f-bf0c-47d0-aaf9-f4b871bb4c55 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] 10.222.0.16 "GET /compute/v2.1/servers/944f68af-bf99-428f-9357-9ce9f807483b" status: 200 len: 1219 microversion: 2.1 time: 0.182748 Jul 16 21:09:34.172959 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 191/391] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:09:33 2022] GET /compute/v2.1/servers/944f68af-bf99-428f-9357-9ce9f807483b => generated 1219 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:09:34.178383 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-82c21ff9-681c-4a37-8712-d532d46cc226 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:34.179073 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-82c21ff9-681c-4a37-8712-d532d46cc226 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: 1703f309-9e2e-4c5f-8588-eb4e43f21d91] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:09:34.198106 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-82c21ff9-681c-4a37-8712-d532d46cc226 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:34.198106 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-82c21ff9-681c-4a37-8712-d532d46cc226 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:34.198699 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-82c21ff9-681c-4a37-8712-d532d46cc226 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:34.199139 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-82c21ff9-681c-4a37-8712-d532d46cc226 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:34.201899 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.objects.instance [None req-c2fc02b3-5ae0-40cf-b738-856f068b50c8 tempest-LiveMigrationNegativeTest-1294767845 tempest-LiveMigrationNegativeTest-1294767845-project-admin] Lazy-loading 'flavor' on Instance uuid 1631f0cc-de67-4c0d-abd0-3e0e11dfb4ed {{(pid=138680) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:09:34.278294 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-c2fc02b3-5ae0-40cf-b738-856f068b50c8 tempest-LiveMigrationNegativeTest-1294767845 tempest-LiveMigrationNegativeTest-1294767845-project-admin] [instance: 1631f0cc-de67-4c0d-abd0-3e0e11dfb4ed] Going to try to live migrate instance to tempest-host-1224354205 {{(pid=138680) live_migrate /opt/stack/nova/nova/compute/api.py:5349}} Jul 16 21:09:34.302849 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-c2fc02b3-5ae0-40cf-b738-856f068b50c8 tempest-LiveMigrationNegativeTest-1294767845 tempest-LiveMigrationNegativeTest-1294767845-project-admin] HTTP exception thrown: Compute host tempest-host-1224354205 could not be found. Jul 16 21:09:34.313502 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-c2fc02b3-5ae0-40cf-b738-856f068b50c8 tempest-LiveMigrationNegativeTest-1294767845 tempest-LiveMigrationNegativeTest-1294767845-project-admin] Request method failure captured: Jul 16 21:09:34.313502 ncdv-master-1 devstack@n-api.service[138680]: request: POST /compute/v2.1/servers/1631f0cc-de67-4c0d-abd0-3e0e11dfb4ed/action HTTP/1.1 Jul 16 21:09:34.313502 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:09:34.313502 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:09:34.313502 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:09:34.313502 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 107 Jul 16 21:09:34.313502 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:09:34.313502 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:09:34.313502 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:09:34.313502 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:09:34.313502 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0yjz1qEdsJvcfI6TRG20vmhREjrkYHDJV3wmhkVZMKHLvvLRTksIbEDj0iJPdh4eC_fXyYNtKdv6xfnyupQMT733OepnnNCu6ae2ezn2GWnQUDiNOSUVTO58w51AATzxplYj67aGXbssyCsNcqcB8SJlN2xLMo4vfipSLLg_WlmIh50 Jul 16 21:09:34.313502 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:09:34.313502 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:09:34.313502 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:09:34.313502 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:09:34.313502 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:09:34.313502 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:09:34.313502 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: 7f515204606542ce9377e2c0b707e5bc Jul 16 21:09:34.313502 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-LiveMigrationNegativeTest-1294767845 Jul 16 21:09:34.313502 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,admin,reader Jul 16 21:09:34.313502 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,admin,reader Jul 16 21:09:34.316049 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/7f515204606542ce9377e2c0b707e5bc"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/7f515204606542ce9377e2c0b707e5bc"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_7f515204606542ce9377e2c0b707e5bc"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/7f515204606542ce9377e2c0b707e5bc"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:09:34.316049 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-LiveMigrationNegativeTest-1294767845 Jul 16 21:09:34.316049 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: 7f515204606542ce9377e2c0b707e5bc Jul 16 21:09:34.316049 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-LiveMigrationNegativeTest-1294767845 Jul 16 21:09:34.316049 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-LiveMigrationNegativeTest-1294767845-project-admin Jul 16 21:09:34.316049 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:09:34.316049 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:09:34.316049 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: 24d825f528ce4d6d9e4da4fa13a417d9 Jul 16 21:09:34.316049 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-LiveMigrationNegativeTest-1294767845-project-admin Jul 16 21:09:34.316049 ncdv-master-1 devstack@n-api.service[138680]: Jul 16 21:09:34.316049 ncdv-master-1 devstack@n-api.service[138680]: {"os-migrateLive": {"host": "tempest-host-1224354205", "block_migration": true, "disk_over_commit": false}} Jul 16 21:09:34.316049 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 21:09:34.316957 ncdv-master-1 devstack@n-api.service[138680]: exception: Compute host tempest-host-1224354205 could not be found. Jul 16 21:09:34.316957 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': '1631f0cc-de67-4c0d-abd0-3e0e11dfb4ed', 'body': {'os-migrateLive': {'host': 'tempest-host-1224354205', 'block_migration': True, 'disk_over_commit': False}}} Jul 16 21:09:34.316957 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:09:34.316957 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:09:34.316957 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/migrate_server.py", line 123, in _migrate_live Jul 16 21:09:34.316957 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi self.compute_api.live_migrate(context, instance, block_migration, Jul 16 21:09:34.316957 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 375, in inner Jul 16 21:09:34.316957 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Jul 16 21:09:34.316957 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 360, in wrapper Jul 16 21:09:34.316957 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return func(self, context, instance, *args, **kwargs) Jul 16 21:09:34.316957 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 296, in inner Jul 16 21:09:34.316957 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Jul 16 21:09:34.316957 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 343, in wrapper Jul 16 21:09:34.316957 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return func(self, context, instance, *args, **kwargs) Jul 16 21:09:34.316957 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 275, in inner Jul 16 21:09:34.316957 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Jul 16 21:09:34.316957 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 256, in inner Jul 16 21:09:34.316957 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Jul 16 21:09:34.316957 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner Jul 16 21:09:34.318473 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Jul 16 21:09:34.318473 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 164, in inner Jul 16 21:09:34.318473 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Jul 16 21:09:34.318473 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5355, in live_migrate Jul 16 21:09:34.318473 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nodes = objects.ComputeNodeList.get_all_by_host(context, host_name) Jul 16 21:09:34.318473 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Jul 16 21:09:34.318473 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Jul 16 21:09:34.318473 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/compute_node.py", line 476, in get_all_by_host Jul 16 21:09:34.318473 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi db_computes = cls._db_compute_node_get_all_by_host(context, host, Jul 16 21:09:34.318473 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/main/api.py", line 179, in wrapper Jul 16 21:09:34.318473 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:09:34.318473 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/compute_node.py", line 472, in _db_compute_node_get_all_by_host Jul 16 21:09:34.318473 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return db.compute_node_get_all_by_host(context, host) Jul 16 21:09:34.318473 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/main/api.py", line 241, in wrapper Jul 16 21:09:34.318473 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(context, *args, **kwargs) Jul 16 21:09:34.318473 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/main/api.py", line 740, in compute_node_get_all_by_host Jul 16 21:09:34.318473 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.ComputeHostNotFound(host=host) Jul 16 21:09:34.318473 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.ComputeHostNotFound: Compute host tempest-host-1224354205 could not be found. Jul 16 21:09:34.318473 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:09:34.330779 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:09:34.330779 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:09:34.330779 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:09:34.330779 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:09:34.330779 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:09:34.330779 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:09:34.330779 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:09:34.330779 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:09:34.330779 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:09:34.330779 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 16 21:09:34.330779 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 16 21:09:34.330779 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 16 21:09:34.330779 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 16 21:09:34.330779 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 16 21:09:34.330779 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 16 21:09:34.330779 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi [Previous line repeated 1 more time] Jul 16 21:09:34.330779 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/migrate_server.py", line 150, in _migrate_live Jul 16 21:09:34.330779 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Jul 16 21:09:34.330779 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Compute host tempest-host-1224354205 could not be found. Jul 16 21:09:34.330779 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:09:34.332409 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:09:34.332409 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:09:34.332409 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:09:34.332409 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:09:34.332409 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:09:34.332409 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:09:34.332409 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:09:34.332409 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Compute host tempest-host-1224354205 could not be found. Jul 16 21:09:34.332409 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:09:34.332409 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-c2fc02b3-5ae0-40cf-b738-856f068b50c8 tempest-LiveMigrationNegativeTest-1294767845 tempest-LiveMigrationNegativeTest-1294767845-project-admin] Returning 400 to user: Compute host tempest-host-1224354205 could not be found. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:09:34.332409 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-c2fc02b3-5ae0-40cf-b738-856f068b50c8 tempest-LiveMigrationNegativeTest-1294767845 tempest-LiveMigrationNegativeTest-1294767845-project-admin] 10.222.0.16 "POST /compute/v2.1/servers/1631f0cc-de67-4c0d-abd0-3e0e11dfb4ed/action" status: 400 len: 100 microversion: 2.1 time: 0.598910 Jul 16 21:09:34.332409 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 201/392] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Jul 16 21:09:33 2022] POST /compute/v2.1/servers/1631f0cc-de67-4c0d-abd0-3e0e11dfb4ed/action => generated 100 bytes in 603 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Jul 16 21:09:34.337977 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-343e02e1-c304-468c-b7c3-4f09da0b9e0a tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:34.338658 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-343e02e1-c304-468c-b7c3-4f09da0b9e0a tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] [instance: 1631f0cc-de67-4c0d-abd0-3e0e11dfb4ed] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:09:34.346951 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-343e02e1-c304-468c-b7c3-4f09da0b9e0a tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:34.356155 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-343e02e1-c304-468c-b7c3-4f09da0b9e0a tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:34.379066 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-343e02e1-c304-468c-b7c3-4f09da0b9e0a tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:34.380916 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-343e02e1-c304-468c-b7c3-4f09da0b9e0a tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:34.536556 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-82c21ff9-681c-4a37-8712-d532d46cc226 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b68dacc4b8463ba07fae6372131117', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55a19b7a4682457a9132f5fc80110712', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:34.544728 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-82c21ff9-681c-4a37-8712-d532d46cc226 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "GET /compute/v2.1/servers/1703f309-9e2e-4c5f-8588-eb4e43f21d91" status: 200 len: 1475 microversion: 2.1 time: 0.369782 Jul 16 21:09:34.545360 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 192/393] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:09:34 2022] GET /compute/v2.1/servers/1703f309-9e2e-4c5f-8588-eb4e43f21d91 => generated 1475 bytes in 372 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:09:34.573728 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-f9ad13d7-3475-45d0-99dc-de00022b93a8 req-989c862c-ede2-4755-a1ab-012278df0465 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "c40ba20d-eb88-4ee8-be8f-ecbbd3315c48", "tag": "a2fac50b-598b-43e0-8e39-546a5c1b4490"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:09:34.586253 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-f9ad13d7-3475-45d0-99dc-de00022b93a8 req-989c862c-ede2-4755-a1ab-012278df0465 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:34.587374 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-f9ad13d7-3475-45d0-99dc-de00022b93a8 req-989c862c-ede2-4755-a1ab-012278df0465 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:34.601827 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-f9ad13d7-3475-45d0-99dc-de00022b93a8 req-989c862c-ede2-4755-a1ab-012278df0465 service nova] Creating event network-changed:a2fac50b-598b-43e0-8e39-546a5c1b4490 for instance c40ba20d-eb88-4ee8-be8f-ecbbd3315c48 on nch1-master-1 Jul 16 21:09:34.616175 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-f9ad13d7-3475-45d0-99dc-de00022b93a8 req-989c862c-ede2-4755-a1ab-012278df0465 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.065446 Jul 16 21:09:34.617460 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 193/394] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:09:34 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 71 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:09:34.682518 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-343e02e1-c304-468c-b7c3-4f09da0b9e0a tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0d72626f465b4f37ad3b18bb945d4000', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fc1c3463f9444119af9d18f85bdd703d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:34.698557 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_api:SELECT=90,nova_api:INSERT=10 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:09:34.701243 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-343e02e1-c304-468c-b7c3-4f09da0b9e0a tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] 10.222.0.16 "GET /compute/v2.1/servers/1631f0cc-de67-4c0d-abd0-3e0e11dfb4ed" status: 200 len: 1468 microversion: 2.1 time: 0.368172 Jul 16 21:09:34.702345 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 202/395] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:09:34 2022] GET /compute/v2.1/servers/1631f0cc-de67-4c0d-abd0-3e0e11dfb4ed => generated 1468 bytes in 370 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:09:34.811731 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-1384f282-584b-4162-86fc-03d8f9efa0fd tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:34.812516 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-1384f282-584b-4162-86fc-03d8f9efa0fd tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] [instance: c40ba20d-eb88-4ee8-be8f-ecbbd3315c48] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:09:34.836652 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1384f282-584b-4162-86fc-03d8f9efa0fd tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:34.837303 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1384f282-584b-4162-86fc-03d8f9efa0fd tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:34.840244 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1384f282-584b-4162-86fc-03d8f9efa0fd tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:34.840840 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1384f282-584b-4162-86fc-03d8f9efa0fd tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:34.992679 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-1384f282-584b-4162-86fc-03d8f9efa0fd tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a69f9dee03d43beaafa966216b9e289', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9b769bf8f3ea4e368bf68de1df6538f9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:35.000834 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-1384f282-584b-4162-86fc-03d8f9efa0fd tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] 10.222.0.16 "GET /compute/v2.1/servers/c40ba20d-eb88-4ee8-be8f-ecbbd3315c48" status: 200 len: 1310 microversion: 2.1 time: 0.193799 Jul 16 21:09:35.002068 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 203/396] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:09:34 2022] GET /compute/v2.1/servers/c40ba20d-eb88-4ee8-be8f-ecbbd3315c48 => generated 1310 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:09:35.030759 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-0b30d52c-de66-4387-8af6-d1251e3b87c0 tempest-FlavorsExtraSpecsNegativeTestJSON-1647059018 tempest-FlavorsExtraSpecsNegativeTestJSON-1647059018-project-admin] Action: 'create', calling method: >, body: {"flavor": {"name": "tempest-test_flavor-603028716", "ram": 512, "vcpus": 1, "disk": 10, "id": 1272159723, "swap": 1024, "rxtx_factor": 1, "OS-FLV-EXT-DATA:ephemeral": 10}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:09:35.032783 ncdv-master-1 devstack@n-api.service[138681]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-flavor-manage:create": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:09:35.032783 ncdv-master-1 devstack@n-api.service[138681]: warnings.warn(msg) Jul 16 21:09:35.070918 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-e0445574-2113-4a10-b04f-e4bfa190873b tempest-FlavorMetadataValidation-817712997 tempest-FlavorMetadataValidation-817712997-project-admin] Action: 'create', calling method: >, body: {"flavor": {"ram": 512, "vcpus": 1, "disk": 10, "name": "tempest-test_flavor_validate_metadata_-531024907", "id": 729108947, "os-flavor-access:is_public": "True"}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:09:35.072566 ncdv-master-1 devstack@n-api.service[138680]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-flavor-manage:create": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:09:35.072566 ncdv-master-1 devstack@n-api.service[138680]: warnings.warn(msg) Jul 16 21:09:35.159874 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-0b30d52c-de66-4387-8af6-d1251e3b87c0 tempest-FlavorsExtraSpecsNegativeTestJSON-1647059018 tempest-FlavorsExtraSpecsNegativeTestJSON-1647059018-project-admin] 10.222.0.16 "POST /compute/v2.1/flavors" status: 200 len: 415 microversion: 2.1 time: 0.417275 Jul 16 21:09:35.161738 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 194/397] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 21:09:34 2022] POST /compute/v2.1/flavors => generated 415 bytes in 420 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:09:35.166408 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-a888bbf6-a83b-4857-8e5f-8001525449a5 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-LiveMigrationNegativeTest-server-1975690156", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452"}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:09:35.200489 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-e0445574-2113-4a10-b04f-e4bfa190873b tempest-FlavorMetadataValidation-817712997 tempest-FlavorMetadataValidation-817712997-project-admin] 10.222.0.16 "POST /compute/v2.1/flavors" status: 200 len: 467 microversion: 2.86 time: 0.197089 Jul 16 21:09:35.200489 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 204/398] 10.222.0.16 () {62 vars in 1187 bytes} [Sat Jul 16 21:09:35 2022] POST /compute/v2.1/flavors => generated 467 bytes in 198 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 16 21:09:35.205411 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-21671d2d-f51a-493e-8a3e-64ad673cb7a7 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:35.206011 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-21671d2d-f51a-493e-8a3e-64ad673cb7a7 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] [instance: cf53f7cd-deae-4b34-bf8e-3afef98b2ec6] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:09:35.212476 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-21671d2d-f51a-493e-8a3e-64ad673cb7a7 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:35.212924 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-21671d2d-f51a-493e-8a3e-64ad673cb7a7 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:35.213692 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-21671d2d-f51a-493e-8a3e-64ad673cb7a7 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:35.214122 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-21671d2d-f51a-493e-8a3e-64ad673cb7a7 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:35.221631 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.network.neutron [None req-a888bbf6-a83b-4857-8e5f-8001525449a5 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] validate_networks() for None {{(pid=138681) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 16 21:09:35.360254 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-21671d2d-f51a-493e-8a3e-64ad673cb7a7 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ba60685a2001459cbfb531d95a6b296e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb6deae6b08e4ed8b15dc6edfe1d38d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:35.369330 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-21671d2d-f51a-493e-8a3e-64ad673cb7a7 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ba60685a2001459cbfb531d95a6b296e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb6deae6b08e4ed8b15dc6edfe1d38d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:35.370999 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-21671d2d-f51a-493e-8a3e-64ad673cb7a7 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ba60685a2001459cbfb531d95a6b296e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb6deae6b08e4ed8b15dc6edfe1d38d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:35.378217 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-21671d2d-f51a-493e-8a3e-64ad673cb7a7 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] 10.222.0.16 "GET /compute/v2.1/servers/cf53f7cd-deae-4b34-bf8e-3afef98b2ec6" status: 200 len: 1417 microversion: 2.75 time: 0.176230 Jul 16 21:09:35.382587 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 205/399] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:09:35 2022] GET /compute/v2.1/servers/cf53f7cd-deae-4b34-bf8e-3afef98b2ec6 => generated 1417 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:09:35.503159 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-3a07c798-52a1-47a5-b38f-80e12a3db159 tempest-FlavorsExtraSpecsNegativeTestJSON-301486937 tempest-FlavorsExtraSpecsNegativeTestJSON-301486937-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:35.504714 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-a888bbf6-a83b-4857-8e5f-8001525449a5 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Flavor limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 16 21:09:35.505321 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-a888bbf6-a83b-4857-8e5f-8001525449a5 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Image limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 16 21:09:35.506101 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-a888bbf6-a83b-4857-8e5f-8001525449a5 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Flavor pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 16 21:09:35.506524 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-a888bbf6-a83b-4857-8e5f-8001525449a5 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Image pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 16 21:09:35.506992 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-a888bbf6-a83b-4857-8e5f-8001525449a5 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 16 21:09:35.512470 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-3a07c798-52a1-47a5-b38f-80e12a3db159 tempest-FlavorsExtraSpecsNegativeTestJSON-301486937 tempest-FlavorsExtraSpecsNegativeTestJSON-301486937-project] HTTP exception thrown: Flavor 1272159723 has no extra specs with key hw:cpu_thread_policy. Jul 16 21:09:35.517020 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a888bbf6-a83b-4857-8e5f-8001525449a5 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:35.517730 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a888bbf6-a83b-4857-8e5f-8001525449a5 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:35.518727 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a888bbf6-a83b-4857-8e5f-8001525449a5 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:35.519212 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a888bbf6-a83b-4857-8e5f-8001525449a5 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:35.520777 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-3a07c798-52a1-47a5-b38f-80e12a3db159 tempest-FlavorsExtraSpecsNegativeTestJSON-301486937 tempest-FlavorsExtraSpecsNegativeTestJSON-301486937-project] Request method failure captured: Jul 16 21:09:35.520777 ncdv-master-1 devstack@n-api.service[138680]: request: GET /compute/v2.1/flavors/1272159723/os-extra_specs/hw:cpu_thread_policy HTTP/1.1 Jul 16 21:09:35.520777 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:09:35.520777 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:09:35.520777 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:09:35.520777 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 0 Jul 16 21:09:35.520777 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:09:35.520777 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:09:35.520777 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:09:35.520777 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:09:35.520777 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0ykNjl2BLlfwqJqHy-bLyIa1JI3UDawqJORZRjbEeJOOW79ib0RKUD8OfjsWDrmmjOTEZYWfo6NDBGSPqV7YLaCgp7QbNESaTNSALt4WKjQ5kOqLdA7Pvxja97xvvf7arb_ZwJleWfcq73k5DcGx_zyha3U5Khgmyf8UX4DXsnbLh-4 Jul 16 21:09:35.520777 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:09:35.520777 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:09:35.520777 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:09:35.520777 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:09:35.520777 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:09:35.520777 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:09:35.520777 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: 5fcfd03d78284545bd934e58574f5f17 Jul 16 21:09:35.520777 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-FlavorsExtraSpecsNegativeTestJSON-301486937 Jul 16 21:09:35.520777 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:09:35.520777 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:09:35.522207 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/5fcfd03d78284545bd934e58574f5f17"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/5fcfd03d78284545bd934e58574f5f17"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_5fcfd03d78284545bd934e58574f5f17"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/5fcfd03d78284545bd934e58574f5f17"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:09:35.522207 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-FlavorsExtraSpecsNegativeTestJSON-301486937 Jul 16 21:09:35.522207 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: 5fcfd03d78284545bd934e58574f5f17 Jul 16 21:09:35.522207 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-FlavorsExtraSpecsNegativeTestJSON-301486937 Jul 16 21:09:35.522207 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-FlavorsExtraSpecsNegativeTestJSON-301486937-project Jul 16 21:09:35.522207 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:09:35.522207 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:09:35.522207 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: 95424d5b83cb45a8a291dc5c2e511ca4 Jul 16 21:09:35.522207 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-FlavorsExtraSpecsNegativeTestJSON-301486937-project Jul 16 21:09:35.522207 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 21:09:35.522207 ncdv-master-1 devstack@n-api.service[138680]: exception: Flavor 1272159723 has no extra specs with key hw:cpu_thread_policy. Jul 16 21:09:35.527260 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'flavor_id': '1272159723', 'id': 'hw:cpu_thread_policy'} Jul 16 21:09:35.527260 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:09:35.527260 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:09:35.527260 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors_extraspecs.py", line 115, in show Jul 16 21:09:35.527260 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return {id: flavor.extra_specs[id]} Jul 16 21:09:35.527260 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi KeyError: 'hw:cpu_thread_policy' Jul 16 21:09:35.527260 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:09:35.527260 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:09:35.527260 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:09:35.527260 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:09:35.527260 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:09:35.527260 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:09:35.527260 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:09:35.527260 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:09:35.527260 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:09:35.527260 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:09:35.527260 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors_extraspecs.py", line 120, in show Jul 16 21:09:35.527260 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=msg) Jul 16 21:09:35.527260 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 1272159723 has no extra specs with key hw:cpu_thread_policy. Jul 16 21:09:35.527260 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:09:35.527260 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:09:35.527260 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:09:35.529204 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:09:35.529204 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:09:35.529204 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:09:35.529204 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:09:35.529204 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:09:35.529204 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 1272159723 has no extra specs with key hw:cpu_thread_policy. Jul 16 21:09:35.529204 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:09:35.529204 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-3a07c798-52a1-47a5-b38f-80e12a3db159 tempest-FlavorsExtraSpecsNegativeTestJSON-301486937 tempest-FlavorsExtraSpecsNegativeTestJSON-301486937-project] Returning 404 to user: Flavor 1272159723 has no extra specs with key hw:cpu_thread_policy. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:09:35.529204 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-3a07c798-52a1-47a5-b38f-80e12a3db159 tempest-FlavorsExtraSpecsNegativeTestJSON-301486937 tempest-FlavorsExtraSpecsNegativeTestJSON-301486937-project] 10.222.0.16 "GET /compute/v2.1/flavors/1272159723/os-extra_specs/hw:cpu_thread_policy" status: 404 len: 113 microversion: 2.1 time: 0.142532 Jul 16 21:09:35.529204 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 206/400] 10.222.0.16 () {58 vars in 1265 bytes} [Sat Jul 16 21:09:35 2022] GET /compute/v2.1/flavors/1272159723/os-extra_specs/hw:cpu_thread_policy => generated 113 bytes in 144 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:09:35.534060 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-52d88202-3639-44b7-950a-51234511e002 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:35.534222 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-52d88202-3639-44b7-950a-51234511e002 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] [instance: 944f68af-bf99-428f-9357-9ce9f807483b] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:09:35.551422 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-a888bbf6-a83b-4857-8e5f-8001525449a5 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Getting quotas for project fc1c3463f9444119af9d18f85bdd703d. Resources: {'ram', 'instances', 'cores'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:09:35.559546 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-a888bbf6-a83b-4857-8e5f-8001525449a5 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Getting quotas for user 0d72626f465b4f37ad3b18bb945d4000 and project fc1c3463f9444119af9d18f85bdd703d. Resources: {'ram', 'instances', 'cores'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:09:35.562756 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_api:SELECT=93,nova_api:INSERT=7 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:09:35.574820 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-a888bbf6-a83b-4857-8e5f-8001525449a5 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Going to run 1 instances... {{(pid=138681) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 16 21:09:35.585924 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-a888bbf6-a83b-4857-8e5f-8001525449a5 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] [instance: 67266541-b43e-414c-a744-9c32c6f53975] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138681) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 16 21:09:35.618696 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-52d88202-3639-44b7-950a-51234511e002 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'abeb7020019d41a18de42956d706a011', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '649babe54d104436af461f1ee4767001', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:35.625851 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-52d88202-3639-44b7-950a-51234511e002 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] 10.222.0.16 "GET /compute/v2.1/servers/944f68af-bf99-428f-9357-9ce9f807483b" status: 200 len: 1219 microversion: 2.1 time: 0.101387 Jul 16 21:09:35.626503 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 207/401] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:09:35 2022] GET /compute/v2.1/servers/944f68af-bf99-428f-9357-9ce9f807483b => generated 1219 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:09:35.631001 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-e19900b0-3183-4fc4-9101-18b8f4d8b007 tempest-FlavorMetadataValidation-817712997 tempest-FlavorMetadataValidation-817712997-project-admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw:cpu_policy": "shared", "foo:bar": "baz"}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:09:35.632332 ncdv-master-1 devstack@n-api.service[138680]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-flavor-extra-specs:create": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:09:35.632332 ncdv-master-1 devstack@n-api.service[138680]: warnings.warn(msg) Jul 16 21:09:35.731966 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-a888bbf6-a83b-4857-8e5f-8001525449a5 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.569139 Jul 16 21:09:35.732612 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 195/402] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 21:09:35 2022] POST /compute/v2.1/servers => generated 380 bytes in 570 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 16 21:09:35.739053 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-d8100fe9-be19-4d81-a569-db0f15d48735 tempest-FlavorsExtraSpecsNegativeTestJSON-301486937 tempest-FlavorsExtraSpecsNegativeTestJSON-301486937-project] Action: 'create', calling method: >, body: {"extra_specs": {"hw:numa_nodes": "1", "hw:cpu_policy": "shared"}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:09:35.740595 ncdv-master-1 devstack@n-api.service[138681]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-flavor-extra-specs:create": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:09:35.740595 ncdv-master-1 devstack@n-api.service[138681]: warnings.warn(msg) Jul 16 21:09:35.741455 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-d8100fe9-be19-4d81-a569-db0f15d48735 tempest-FlavorsExtraSpecsNegativeTestJSON-301486937 tempest-FlavorsExtraSpecsNegativeTestJSON-301486937-project] Policy check for os_compute_api:os-flavor-extra-specs:create failed with credentials {'is_admin': False, 'user_id': '95424d5b83cb45a8a291dc5c2e511ca4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fcfd03d78284545bd934e58574f5f17', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:35.743393 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-d8100fe9-be19-4d81-a569-db0f15d48735 tempest-FlavorsExtraSpecsNegativeTestJSON-301486937 tempest-FlavorsExtraSpecsNegativeTestJSON-301486937-project] Request method failure captured: Jul 16 21:09:35.743393 ncdv-master-1 devstack@n-api.service[138681]: request: POST /compute/v2.1/flavors/1272159723/os-extra_specs HTTP/1.1 Jul 16 21:09:35.743393 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:09:35.743393 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:09:35.743393 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:09:35.743393 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 66 Jul 16 21:09:35.743393 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:09:35.743393 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:09:35.743393 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:09:35.743393 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:09:35.743393 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0ykNjl2BLlfwqJqHy-bLyIa1JI3UDawqJORZRjbEeJOOW79ib0RKUD8OfjsWDrmmjOTEZYWfo6NDBGSPqV7YLaCgp7QbNESaTNSALt4WKjQ5kOqLdA7Pvxja97xvvf7arb_ZwJleWfcq73k5DcGx_zyha3U5Khgmyf8UX4DXsnbLh-4 Jul 16 21:09:35.743393 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:09:35.743393 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:09:35.743393 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:09:35.743393 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:09:35.743393 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:09:35.743393 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:09:35.743393 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: 5fcfd03d78284545bd934e58574f5f17 Jul 16 21:09:35.743393 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-FlavorsExtraSpecsNegativeTestJSON-301486937 Jul 16 21:09:35.743393 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:09:35.743393 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:09:35.744730 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/5fcfd03d78284545bd934e58574f5f17"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/5fcfd03d78284545bd934e58574f5f17"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_5fcfd03d78284545bd934e58574f5f17"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/5fcfd03d78284545bd934e58574f5f17"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:09:35.744730 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-FlavorsExtraSpecsNegativeTestJSON-301486937 Jul 16 21:09:35.744730 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: 5fcfd03d78284545bd934e58574f5f17 Jul 16 21:09:35.744730 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-FlavorsExtraSpecsNegativeTestJSON-301486937 Jul 16 21:09:35.744730 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-FlavorsExtraSpecsNegativeTestJSON-301486937-project Jul 16 21:09:35.744730 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:09:35.744730 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:09:35.744730 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: 95424d5b83cb45a8a291dc5c2e511ca4 Jul 16 21:09:35.744730 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-FlavorsExtraSpecsNegativeTestJSON-301486937-project Jul 16 21:09:35.744730 ncdv-master-1 devstack@n-api.service[138681]: Jul 16 21:09:35.744730 ncdv-master-1 devstack@n-api.service[138681]: {"extra_specs": {"hw:numa_nodes": "1", "hw:cpu_policy": "shared"}} Jul 16 21:09:35.744730 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:09:35.745371 ncdv-master-1 devstack@n-api.service[138681]: exception: Policy doesn't allow os_compute_api:os-flavor-extra-specs:create to be performed. Jul 16 21:09:35.745371 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'flavor_id': '1272159723', 'body': {'extra_specs': {'hw:numa_nodes': '1', 'hw:cpu_policy': 'shared'}}} Jul 16 21:09:35.745371 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:09:35.745371 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:09:35.745371 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:09:35.745371 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:09:35.745371 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:09:35.745371 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:09:35.745371 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:09:35.745371 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:09:35.745371 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 16 21:09:35.745371 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 16 21:09:35.745371 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors_extraspecs.py", line 73, in create Jul 16 21:09:35.745371 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi context.can(fes_policies.POLICY_ROOT % 'create', target={}) Jul 16 21:09:35.745371 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/context.py", line 240, in can Jul 16 21:09:35.745371 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return policy.authorize(self, action, target) Jul 16 21:09:35.745371 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 199, in authorize Jul 16 21:09:35.745371 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi LOG.debug('Policy check for %(action)s failed with credentials ' Jul 16 21:09:35.745371 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 16 21:09:35.746419 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi self.force_reraise() Jul 16 21:09:35.746419 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 16 21:09:35.746419 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise self.value Jul 16 21:09:35.746419 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 186, in authorize Jul 16 21:09:35.746419 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi result = _ENFORCER.authorize(action, target, context, Jul 16 21:09:35.746419 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1184, in authorize Jul 16 21:09:35.746419 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return self.enforce( Jul 16 21:09:35.746419 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1088, in enforce Jul 16 21:09:35.746419 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc(*args, **kwargs) Jul 16 21:09:35.746419 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.PolicyNotAuthorized: Policy doesn't allow os_compute_api:os-flavor-extra-specs:create to be performed. Jul 16 21:09:35.746419 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:09:35.746419 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:09:35.746419 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:09:35.746419 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:09:35.746419 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:09:35.746419 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:09:35.746419 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 354, in __exit__ Jul 16 21:09:35.746419 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(webob.exc.HTTPForbidden( Jul 16 21:09:35.746419 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Policy doesn't allow os_compute_api:os-flavor-extra-specs:create to be performed. Jul 16 21:09:35.746419 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:09:35.748083 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-d8100fe9-be19-4d81-a569-db0f15d48735 tempest-FlavorsExtraSpecsNegativeTestJSON-301486937 tempest-FlavorsExtraSpecsNegativeTestJSON-301486937-project] Returning 403 to user: Policy doesn't allow os_compute_api:os-flavor-extra-specs:create to be performed. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:09:35.748083 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-d8100fe9-be19-4d81-a569-db0f15d48735 tempest-FlavorsExtraSpecsNegativeTestJSON-301486937 tempest-FlavorsExtraSpecsNegativeTestJSON-301486937-project] 10.222.0.16 "POST /compute/v2.1/flavors/1272159723/os-extra_specs" status: 403 len: 124 microversion: 2.1 time: 0.011053 Jul 16 21:09:35.748083 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 196/403] 10.222.0.16 () {60 vars in 1223 bytes} [Sat Jul 16 21:09:35 2022] POST /compute/v2.1/flavors/1272159723/os-extra_specs => generated 124 bytes in 12 msecs (HTTP/1.1 403) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:09:35.764947 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-9b79a294-43b3-498a-8e59-0d91c09fb73b tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:35.766619 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-9b79a294-43b3-498a-8e59-0d91c09fb73b tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: 1703f309-9e2e-4c5f-8588-eb4e43f21d91] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:09:35.785659 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9b79a294-43b3-498a-8e59-0d91c09fb73b tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:35.786118 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9b79a294-43b3-498a-8e59-0d91c09fb73b tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:35.786917 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9b79a294-43b3-498a-8e59-0d91c09fb73b tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:35.787400 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9b79a294-43b3-498a-8e59-0d91c09fb73b tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:35.936142 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-e19900b0-3183-4fc4-9101-18b8f4d8b007 tempest-FlavorMetadataValidation-817712997 tempest-FlavorMetadataValidation-817712997-project-admin] 10.222.0.16 "POST /compute/v2.1/flavors/729108947/os-extra_specs" status: 200 len: 62 microversion: 2.86 time: 0.308492 Jul 16 21:09:35.942250 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 208/404] 10.222.0.16 () {62 vars in 1261 bytes} [Sat Jul 16 21:09:35 2022] POST /compute/v2.1/flavors/729108947/os-extra_specs => generated 62 bytes in 310 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:09:35.946775 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-c6ff9c5c-7d8a-44ab-bb4f-e8ae55545055 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:35.947437 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-c6ff9c5c-7d8a-44ab-bb4f-e8ae55545055 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] [instance: 67266541-b43e-414c-a744-9c32c6f53975] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:09:36.254251 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-c6ff9c5c-7d8a-44ab-bb4f-e8ae55545055 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0d72626f465b4f37ad3b18bb945d4000', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fc1c3463f9444119af9d18f85bdd703d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:36.273092 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-c6ff9c5c-7d8a-44ab-bb4f-e8ae55545055 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] 10.222.0.16 "GET /compute/v2.1/servers/67266541-b43e-414c-a744-9c32c6f53975" status: 200 len: 1219 microversion: 2.1 time: 0.334315 Jul 16 21:09:36.277317 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 209/405] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:09:35 2022] GET /compute/v2.1/servers/67266541-b43e-414c-a744-9c32c6f53975 => generated 1219 bytes in 336 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:09:36.279168 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-22acf7da-fca9-4a1e-96e1-4cc328ad3ecc tempest-FlavorsExtraSpecsNegativeTestJSON-1647059018 tempest-FlavorsExtraSpecsNegativeTestJSON-1647059018-project-admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw:numa_nodes": "1", "hw:cpu_policy": "shared"}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:09:36.280880 ncdv-master-1 devstack@n-api.service[138680]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-flavor-extra-specs:create": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:09:36.280880 ncdv-master-1 devstack@n-api.service[138680]: warnings.warn(msg) Jul 16 21:09:36.328231 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-9b79a294-43b3-498a-8e59-0d91c09fb73b tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b68dacc4b8463ba07fae6372131117', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55a19b7a4682457a9132f5fc80110712', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:36.341599 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-9b79a294-43b3-498a-8e59-0d91c09fb73b tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "GET /compute/v2.1/servers/1703f309-9e2e-4c5f-8588-eb4e43f21d91" status: 200 len: 1475 microversion: 2.1 time: 0.591198 Jul 16 21:09:36.342169 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 197/406] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:09:35 2022] GET /compute/v2.1/servers/1703f309-9e2e-4c5f-8588-eb4e43f21d91 => generated 1475 bytes in 596 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:09:36.359261 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-577e26d4-6d34-46fa-b27c-7fbdf13e7107 tempest-FlavorMetadataValidation-817712997 tempest-FlavorMetadataValidation-817712997-project-admin] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:36.362264 ncdv-master-1 devstack@n-api.service[138681]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-flavor-manage:delete": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:09:36.362264 ncdv-master-1 devstack@n-api.service[138681]: warnings.warn(msg) Jul 16 21:09:36.388149 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-22acf7da-fca9-4a1e-96e1-4cc328ad3ecc tempest-FlavorsExtraSpecsNegativeTestJSON-1647059018 tempest-FlavorsExtraSpecsNegativeTestJSON-1647059018-project-admin] 10.222.0.16 "POST /compute/v2.1/flavors/1272159723/os-extra_specs" status: 200 len: 66 microversion: 2.1 time: 0.113051 Jul 16 21:09:36.392080 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 210/407] 10.222.0.16 () {60 vars in 1223 bytes} [Sat Jul 16 21:09:36 2022] POST /compute/v2.1/flavors/1272159723/os-extra_specs => generated 66 bytes in 114 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Jul 16 21:09:36.397518 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-c35b5098-0df0-44b9-b819-b430bdcbab6d tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:36.398146 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-c35b5098-0df0-44b9-b819-b430bdcbab6d tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] [instance: c40ba20d-eb88-4ee8-be8f-ecbbd3315c48] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:09:36.410832 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c35b5098-0df0-44b9-b819-b430bdcbab6d tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:36.411313 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c35b5098-0df0-44b9-b819-b430bdcbab6d tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:36.412090 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c35b5098-0df0-44b9-b819-b430bdcbab6d tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:36.412527 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c35b5098-0df0-44b9-b819-b430bdcbab6d tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:36.553413 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-577e26d4-6d34-46fa-b27c-7fbdf13e7107 tempest-FlavorMetadataValidation-817712997 tempest-FlavorMetadataValidation-817712997-project-admin] 10.222.0.16 "DELETE /compute/v2.1/flavors/729108947" status: 202 len: 0 microversion: 2.86 time: 0.210150 Jul 16 21:09:36.554004 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 198/408] 10.222.0.16 () {60 vars in 1198 bytes} [Sat Jul 16 21:09:36 2022] DELETE /compute/v2.1/flavors/729108947 => generated 0 bytes in 211 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Jul 16 21:09:36.559355 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-9ffe983b-dac8-485c-a8b4-a528d93d88e0 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:36.559971 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-9ffe983b-dac8-485c-a8b4-a528d93d88e0 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] [instance: cf53f7cd-deae-4b34-bf8e-3afef98b2ec6] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:09:36.566133 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9ffe983b-dac8-485c-a8b4-a528d93d88e0 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:36.566591 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9ffe983b-dac8-485c-a8b4-a528d93d88e0 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:36.567497 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9ffe983b-dac8-485c-a8b4-a528d93d88e0 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:36.568011 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9ffe983b-dac8-485c-a8b4-a528d93d88e0 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:36.608170 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-c35b5098-0df0-44b9-b819-b430bdcbab6d tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a69f9dee03d43beaafa966216b9e289', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9b769bf8f3ea4e368bf68de1df6538f9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:36.616889 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-c35b5098-0df0-44b9-b819-b430bdcbab6d tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] 10.222.0.16 "GET /compute/v2.1/servers/c40ba20d-eb88-4ee8-be8f-ecbbd3315c48" status: 200 len: 1310 microversion: 2.1 time: 0.227073 Jul 16 21:09:36.617470 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 211/409] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:09:36 2022] GET /compute/v2.1/servers/c40ba20d-eb88-4ee8-be8f-ecbbd3315c48 => generated 1310 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:09:36.628622 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-eceae0f0-f34a-48d6-a551-5c03353f3b98 tempest-FlavorsExtraSpecsNegativeTestJSON-301486937 tempest-FlavorsExtraSpecsNegativeTestJSON-301486937-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:36.629453 ncdv-master-1 devstack@n-api.service[138680]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-flavor-extra-specs:delete": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:09:36.629453 ncdv-master-1 devstack@n-api.service[138680]: warnings.warn(msg) Jul 16 21:09:36.630785 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-eceae0f0-f34a-48d6-a551-5c03353f3b98 tempest-FlavorsExtraSpecsNegativeTestJSON-301486937 tempest-FlavorsExtraSpecsNegativeTestJSON-301486937-project] Policy check for os_compute_api:os-flavor-extra-specs:delete failed with credentials {'is_admin': False, 'user_id': '95424d5b83cb45a8a291dc5c2e511ca4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fcfd03d78284545bd934e58574f5f17', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:36.632468 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-eceae0f0-f34a-48d6-a551-5c03353f3b98 tempest-FlavorsExtraSpecsNegativeTestJSON-301486937 tempest-FlavorsExtraSpecsNegativeTestJSON-301486937-project] Request method failure captured: Jul 16 21:09:36.632468 ncdv-master-1 devstack@n-api.service[138680]: request: DELETE /compute/v2.1/flavors/1272159723/os-extra_specs/hw:numa_nodes HTTP/1.1 Jul 16 21:09:36.632468 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:09:36.632468 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:09:36.632468 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:09:36.632468 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 0 Jul 16 21:09:36.632468 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:09:36.632468 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:09:36.632468 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:09:36.632468 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:09:36.632468 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0ykNjl2BLlfwqJqHy-bLyIa1JI3UDawqJORZRjbEeJOOW79ib0RKUD8OfjsWDrmmjOTEZYWfo6NDBGSPqV7YLaCgp7QbNESaTNSALt4WKjQ5kOqLdA7Pvxja97xvvf7arb_ZwJleWfcq73k5DcGx_zyha3U5Khgmyf8UX4DXsnbLh-4 Jul 16 21:09:36.632468 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:09:36.632468 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:09:36.632468 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:09:36.632468 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:09:36.632468 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:09:36.632468 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:09:36.632468 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: 5fcfd03d78284545bd934e58574f5f17 Jul 16 21:09:36.632468 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-FlavorsExtraSpecsNegativeTestJSON-301486937 Jul 16 21:09:36.632468 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:09:36.632468 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:09:36.639027 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/5fcfd03d78284545bd934e58574f5f17"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/5fcfd03d78284545bd934e58574f5f17"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_5fcfd03d78284545bd934e58574f5f17"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/5fcfd03d78284545bd934e58574f5f17"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:09:36.639027 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-FlavorsExtraSpecsNegativeTestJSON-301486937 Jul 16 21:09:36.639027 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: 5fcfd03d78284545bd934e58574f5f17 Jul 16 21:09:36.639027 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-FlavorsExtraSpecsNegativeTestJSON-301486937 Jul 16 21:09:36.639027 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-FlavorsExtraSpecsNegativeTestJSON-301486937-project Jul 16 21:09:36.639027 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:09:36.639027 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:09:36.639027 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: 95424d5b83cb45a8a291dc5c2e511ca4 Jul 16 21:09:36.639027 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-FlavorsExtraSpecsNegativeTestJSON-301486937-project Jul 16 21:09:36.639027 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 21:09:36.639027 ncdv-master-1 devstack@n-api.service[138680]: exception: Policy doesn't allow os_compute_api:os-flavor-extra-specs:delete to be performed. Jul 16 21:09:36.639625 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'flavor_id': '1272159723', 'id': 'hw:numa_nodes'} Jul 16 21:09:36.639625 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:09:36.639625 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:09:36.639625 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:09:36.639625 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:09:36.639625 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:09:36.639625 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:09:36.639625 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:09:36.639625 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:09:36.639625 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors_extraspecs.py", line 129, in delete Jul 16 21:09:36.639625 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi context.can(fes_policies.POLICY_ROOT % 'delete', target={}) Jul 16 21:09:36.639625 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/context.py", line 240, in can Jul 16 21:09:36.639625 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return policy.authorize(self, action, target) Jul 16 21:09:36.639625 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 199, in authorize Jul 16 21:09:36.639625 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi LOG.debug('Policy check for %(action)s failed with credentials ' Jul 16 21:09:36.639625 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 16 21:09:36.639625 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi self.force_reraise() Jul 16 21:09:36.639625 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 16 21:09:36.639625 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise self.value Jul 16 21:09:36.639625 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 186, in authorize Jul 16 21:09:36.640692 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi result = _ENFORCER.authorize(action, target, context, Jul 16 21:09:36.640692 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1184, in authorize Jul 16 21:09:36.640692 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return self.enforce( Jul 16 21:09:36.640692 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1088, in enforce Jul 16 21:09:36.640692 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc(*args, **kwargs) Jul 16 21:09:36.640692 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.PolicyNotAuthorized: Policy doesn't allow os_compute_api:os-flavor-extra-specs:delete to be performed. Jul 16 21:09:36.640692 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:09:36.640692 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:09:36.640692 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:09:36.640692 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:09:36.640692 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:09:36.640692 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:09:36.640692 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 354, in __exit__ Jul 16 21:09:36.640692 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(webob.exc.HTTPForbidden( Jul 16 21:09:36.640692 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Policy doesn't allow os_compute_api:os-flavor-extra-specs:delete to be performed. Jul 16 21:09:36.640692 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:09:36.640692 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-eceae0f0-f34a-48d6-a551-5c03353f3b98 tempest-FlavorsExtraSpecsNegativeTestJSON-301486937 tempest-FlavorsExtraSpecsNegativeTestJSON-301486937-project] Returning 403 to user: Policy doesn't allow os_compute_api:os-flavor-extra-specs:delete to be performed. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:09:36.641615 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-eceae0f0-f34a-48d6-a551-5c03353f3b98 tempest-FlavorsExtraSpecsNegativeTestJSON-301486937 tempest-FlavorsExtraSpecsNegativeTestJSON-301486937-project] 10.222.0.16 "DELETE /compute/v2.1/flavors/1272159723/os-extra_specs/hw:numa_nodes" status: 403 len: 124 microversion: 2.1 time: 0.015369 Jul 16 21:09:36.641615 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 212/410] 10.222.0.16 () {58 vars in 1247 bytes} [Sat Jul 16 21:09:36 2022] DELETE /compute/v2.1/flavors/1272159723/os-extra_specs/hw:numa_nodes => generated 124 bytes in 17 msecs (HTTP/1.1 403) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:09:36.645757 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-f15c995a-d92d-4979-a952-7dc544542635 tempest-FlavorMetadataValidation-817712997 tempest-FlavorMetadataValidation-817712997-project-admin] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:36.674011 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-f15c995a-d92d-4979-a952-7dc544542635 tempest-FlavorMetadataValidation-817712997 tempest-FlavorMetadataValidation-817712997-project-admin] 10.222.0.16 "GET /compute/v2.1/flavors/detail" status: 200 len: 7175 microversion: 2.86 time: 0.037812 Jul 16 21:09:36.675301 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 213/411] 10.222.0.16 () {60 vars in 1186 bytes} [Sat Jul 16 21:09:36 2022] GET /compute/v2.1/flavors/detail => generated 7175 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:09:36.677563 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-9ffe983b-dac8-485c-a8b4-a528d93d88e0 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ba60685a2001459cbfb531d95a6b296e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb6deae6b08e4ed8b15dc6edfe1d38d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:36.680267 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-61347e93-dffe-4604-bafe-f9946bb378c1 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:36.681132 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-61347e93-dffe-4604-bafe-f9946bb378c1 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] [instance: 944f68af-bf99-428f-9357-9ce9f807483b] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:09:36.688147 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-61347e93-dffe-4604-bafe-f9946bb378c1 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:36.692673 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-61347e93-dffe-4604-bafe-f9946bb378c1 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:36.694839 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-9ffe983b-dac8-485c-a8b4-a528d93d88e0 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ba60685a2001459cbfb531d95a6b296e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb6deae6b08e4ed8b15dc6edfe1d38d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:36.696622 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-9ffe983b-dac8-485c-a8b4-a528d93d88e0 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ba60685a2001459cbfb531d95a6b296e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb6deae6b08e4ed8b15dc6edfe1d38d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:36.699615 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-61347e93-dffe-4604-bafe-f9946bb378c1 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:36.700081 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-61347e93-dffe-4604-bafe-f9946bb378c1 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:36.706150 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-9ffe983b-dac8-485c-a8b4-a528d93d88e0 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] 10.222.0.16 "GET /compute/v2.1/servers/cf53f7cd-deae-4b34-bf8e-3afef98b2ec6" status: 200 len: 1417 microversion: 2.75 time: 0.150869 Jul 16 21:09:36.707058 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 199/412] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:09:36 2022] GET /compute/v2.1/servers/cf53f7cd-deae-4b34-bf8e-3afef98b2ec6 => generated 1417 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:09:36.711859 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-d0c561b5-15c4-4a9b-991d-446a4810b461 tempest-FlavorsExtraSpecsNegativeTestJSON-1647059018 tempest-FlavorsExtraSpecsNegativeTestJSON-1647059018-project-admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw:numa_nodes": "1", "hw:cpu_policy": "shared"}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:09:36.713433 ncdv-master-1 devstack@n-api.service[138681]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-flavor-extra-specs:create": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:09:36.713433 ncdv-master-1 devstack@n-api.service[138681]: warnings.warn(msg) Jul 16 21:09:36.735956 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-d0c561b5-15c4-4a9b-991d-446a4810b461 tempest-FlavorsExtraSpecsNegativeTestJSON-1647059018 tempest-FlavorsExtraSpecsNegativeTestJSON-1647059018-project-admin] 10.222.0.16 "POST /compute/v2.1/flavors/1272159723/os-extra_specs" status: 200 len: 66 microversion: 2.1 time: 0.027574 Jul 16 21:09:36.737284 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 200/413] 10.222.0.16 () {60 vars in 1223 bytes} [Sat Jul 16 21:09:36 2022] POST /compute/v2.1/flavors/1272159723/os-extra_specs => generated 66 bytes in 29 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Jul 16 21:09:36.747685 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-6d2ecf6a-89f1-4c01-8129-92a6f05250e5 tempest-FlavorsExtraSpecsNegativeTestJSON-301486937 tempest-FlavorsExtraSpecsNegativeTestJSON-301486937-project] Action: 'update', calling method: >, body: {"hw:numa_nodes": "1"} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:09:36.749588 ncdv-master-1 devstack@n-api.service[138681]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-flavor-extra-specs:update": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:09:36.749588 ncdv-master-1 devstack@n-api.service[138681]: warnings.warn(msg) Jul 16 21:09:36.751998 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-6d2ecf6a-89f1-4c01-8129-92a6f05250e5 tempest-FlavorsExtraSpecsNegativeTestJSON-301486937 tempest-FlavorsExtraSpecsNegativeTestJSON-301486937-project] Policy check for os_compute_api:os-flavor-extra-specs:update failed with credentials {'is_admin': False, 'user_id': '95424d5b83cb45a8a291dc5c2e511ca4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fcfd03d78284545bd934e58574f5f17', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:36.756301 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-6d2ecf6a-89f1-4c01-8129-92a6f05250e5 tempest-FlavorsExtraSpecsNegativeTestJSON-301486937 tempest-FlavorsExtraSpecsNegativeTestJSON-301486937-project] Request method failure captured: Jul 16 21:09:36.756301 ncdv-master-1 devstack@n-api.service[138681]: request: PUT /compute/v2.1/flavors/1272159723/os-extra_specs/hw:numa_nodes HTTP/1.1 Jul 16 21:09:36.756301 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:09:36.756301 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:09:36.756301 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:09:36.756301 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 22 Jul 16 21:09:36.756301 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:09:36.756301 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:09:36.756301 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:09:36.756301 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:09:36.756301 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0ykNjl2BLlfwqJqHy-bLyIa1JI3UDawqJORZRjbEeJOOW79ib0RKUD8OfjsWDrmmjOTEZYWfo6NDBGSPqV7YLaCgp7QbNESaTNSALt4WKjQ5kOqLdA7Pvxja97xvvf7arb_ZwJleWfcq73k5DcGx_zyha3U5Khgmyf8UX4DXsnbLh-4 Jul 16 21:09:36.756301 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:09:36.756301 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:09:36.756301 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:09:36.756301 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:09:36.756301 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:09:36.756301 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:09:36.756301 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: 5fcfd03d78284545bd934e58574f5f17 Jul 16 21:09:36.756301 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-FlavorsExtraSpecsNegativeTestJSON-301486937 Jul 16 21:09:36.756301 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:09:36.756301 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:09:36.757846 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/5fcfd03d78284545bd934e58574f5f17"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/5fcfd03d78284545bd934e58574f5f17"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_5fcfd03d78284545bd934e58574f5f17"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/5fcfd03d78284545bd934e58574f5f17"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:09:36.757846 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-FlavorsExtraSpecsNegativeTestJSON-301486937 Jul 16 21:09:36.757846 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: 5fcfd03d78284545bd934e58574f5f17 Jul 16 21:09:36.757846 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-FlavorsExtraSpecsNegativeTestJSON-301486937 Jul 16 21:09:36.757846 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-FlavorsExtraSpecsNegativeTestJSON-301486937-project Jul 16 21:09:36.757846 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:09:36.757846 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:09:36.757846 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: 95424d5b83cb45a8a291dc5c2e511ca4 Jul 16 21:09:36.757846 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-FlavorsExtraSpecsNegativeTestJSON-301486937-project Jul 16 21:09:36.757846 ncdv-master-1 devstack@n-api.service[138681]: Jul 16 21:09:36.757846 ncdv-master-1 devstack@n-api.service[138681]: {"hw:numa_nodes": "1"} Jul 16 21:09:36.757846 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:09:36.757846 ncdv-master-1 devstack@n-api.service[138681]: exception: Policy doesn't allow os_compute_api:os-flavor-extra-specs:update to be performed. Jul 16 21:09:36.758586 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'flavor_id': '1272159723', 'id': 'hw:numa_nodes', 'body': {'hw:numa_nodes': '1'}} Jul 16 21:09:36.758586 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:09:36.758586 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:09:36.758586 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:09:36.758586 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:09:36.758586 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:09:36.758586 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:09:36.758586 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:09:36.758586 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:09:36.758586 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 16 21:09:36.758586 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 16 21:09:36.758586 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors_extraspecs.py", line 91, in update Jul 16 21:09:36.758586 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi context.can(fes_policies.POLICY_ROOT % 'update', target={}) Jul 16 21:09:36.758586 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/context.py", line 240, in can Jul 16 21:09:36.758586 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return policy.authorize(self, action, target) Jul 16 21:09:36.758586 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 199, in authorize Jul 16 21:09:36.758586 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi LOG.debug('Policy check for %(action)s failed with credentials ' Jul 16 21:09:36.758586 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 16 21:09:36.758586 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi self.force_reraise() Jul 16 21:09:36.759671 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 16 21:09:36.759671 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise self.value Jul 16 21:09:36.759671 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 186, in authorize Jul 16 21:09:36.759671 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi result = _ENFORCER.authorize(action, target, context, Jul 16 21:09:36.759671 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1184, in authorize Jul 16 21:09:36.759671 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return self.enforce( Jul 16 21:09:36.759671 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1088, in enforce Jul 16 21:09:36.759671 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc(*args, **kwargs) Jul 16 21:09:36.759671 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.PolicyNotAuthorized: Policy doesn't allow os_compute_api:os-flavor-extra-specs:update to be performed. Jul 16 21:09:36.759671 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:09:36.759671 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:09:36.759671 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:09:36.759671 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:09:36.759671 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:09:36.759671 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:09:36.759671 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 354, in __exit__ Jul 16 21:09:36.759671 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(webob.exc.HTTPForbidden( Jul 16 21:09:36.759671 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Policy doesn't allow os_compute_api:os-flavor-extra-specs:update to be performed. Jul 16 21:09:36.759671 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:09:36.760782 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-6d2ecf6a-89f1-4c01-8129-92a6f05250e5 tempest-FlavorsExtraSpecsNegativeTestJSON-301486937 tempest-FlavorsExtraSpecsNegativeTestJSON-301486937-project] Returning 403 to user: Policy doesn't allow os_compute_api:os-flavor-extra-specs:update to be performed. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:09:36.760782 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-6d2ecf6a-89f1-4c01-8129-92a6f05250e5 tempest-FlavorsExtraSpecsNegativeTestJSON-301486937 tempest-FlavorsExtraSpecsNegativeTestJSON-301486937-project] 10.222.0.16 "PUT /compute/v2.1/flavors/1272159723/os-extra_specs/hw:numa_nodes" status: 403 len: 124 microversion: 2.1 time: 0.012926 Jul 16 21:09:36.760782 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 201/414] 10.222.0.16 () {60 vars in 1264 bytes} [Sat Jul 16 21:09:36 2022] PUT /compute/v2.1/flavors/1272159723/os-extra_specs/hw:numa_nodes => generated 124 bytes in 14 msecs (HTTP/1.1 403) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:09:36.770413 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-14ac8c93-0cda-4b24-8c11-b38cd210f5d9 tempest-FlavorsExtraSpecsNegativeTestJSON-1647059018 tempest-FlavorsExtraSpecsNegativeTestJSON-1647059018-project-admin] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:36.771415 ncdv-master-1 devstack@n-api.service[138681]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-flavor-extra-specs:delete": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:09:36.771415 ncdv-master-1 devstack@n-api.service[138681]: warnings.warn(msg) Jul 16 21:09:36.781254 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-14ac8c93-0cda-4b24-8c11-b38cd210f5d9 tempest-FlavorsExtraSpecsNegativeTestJSON-1647059018 tempest-FlavorsExtraSpecsNegativeTestJSON-1647059018-project-admin] HTTP exception thrown: Flavor 1272159723 has no extra specs with key hw:cpu_thread_policy. Jul 16 21:09:36.783382 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-14ac8c93-0cda-4b24-8c11-b38cd210f5d9 tempest-FlavorsExtraSpecsNegativeTestJSON-1647059018 tempest-FlavorsExtraSpecsNegativeTestJSON-1647059018-project-admin] Request method failure captured: Jul 16 21:09:36.783382 ncdv-master-1 devstack@n-api.service[138681]: request: DELETE /compute/v2.1/flavors/1272159723/os-extra_specs/hw:cpu_thread_policy HTTP/1.1 Jul 16 21:09:36.783382 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:09:36.783382 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:09:36.783382 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:09:36.783382 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 21:09:36.783382 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:09:36.783382 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:09:36.783382 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:09:36.783382 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:09:36.783382 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0ykOPmNhFIl8yxcbfGfrqgGUGO8GzLtEVntQMUQGihVpkG87xnDtQrmIGMZtXcbmz-Sk9bVRwRmjSfZmOZUNrbyUQREiFdVEAVr_C_QxSBJFq31AcfHlKeg5KRaSqxUrz3VjPX8qs8bOS8cijlmQ2Ekqrx1ZjNhfKFXLfYYHgQbGZrM Jul 16 21:09:36.783382 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:09:36.783382 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:09:36.783382 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:09:36.783382 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:09:36.783382 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:09:36.783382 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:09:36.783382 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: 792720ef974e4498b36e3faf082bb51b Jul 16 21:09:36.783382 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-FlavorsExtraSpecsNegativeTestJSON-1647059018 Jul 16 21:09:36.783382 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,admin,reader Jul 16 21:09:36.783382 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,admin,reader Jul 16 21:09:36.784567 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/792720ef974e4498b36e3faf082bb51b"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/792720ef974e4498b36e3faf082bb51b"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_792720ef974e4498b36e3faf082bb51b"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/792720ef974e4498b36e3faf082bb51b"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:09:36.784567 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-FlavorsExtraSpecsNegativeTestJSON-1647059018 Jul 16 21:09:36.784567 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: 792720ef974e4498b36e3faf082bb51b Jul 16 21:09:36.784567 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-FlavorsExtraSpecsNegativeTestJSON-1647059018 Jul 16 21:09:36.784567 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-FlavorsExtraSpecsNegativeTestJSON-1647059018-project-admin Jul 16 21:09:36.784567 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:09:36.784567 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:09:36.784567 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: 454d449792ca4f9e8b524fb38e7576ff Jul 16 21:09:36.784567 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-FlavorsExtraSpecsNegativeTestJSON-1647059018-project-admin Jul 16 21:09:36.784567 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:09:36.784567 ncdv-master-1 devstack@n-api.service[138681]: exception: Flavor 1272159723 has no extra specs with key hw:cpu_thread_policy. Jul 16 21:09:36.785211 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'flavor_id': '1272159723', 'id': 'hw:cpu_thread_policy'} Jul 16 21:09:36.785211 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:09:36.785211 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:09:36.785211 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors_extraspecs.py", line 132, in delete Jul 16 21:09:36.785211 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi del flavor.extra_specs[id] Jul 16 21:09:36.785211 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi KeyError: 'hw:cpu_thread_policy' Jul 16 21:09:36.785211 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:09:36.785211 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:09:36.785211 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:09:36.785211 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:09:36.785211 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:09:36.785211 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:09:36.785211 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:09:36.785211 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:09:36.785211 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:09:36.785211 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:09:36.785211 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors_extraspecs.py", line 141, in delete Jul 16 21:09:36.785211 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=msg) Jul 16 21:09:36.785211 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 1272159723 has no extra specs with key hw:cpu_thread_policy. Jul 16 21:09:36.785211 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:09:36.785211 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:09:36.785211 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:09:36.787052 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:09:36.787052 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:09:36.787052 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:09:36.787052 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:09:36.787052 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:09:36.787052 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 1272159723 has no extra specs with key hw:cpu_thread_policy. Jul 16 21:09:36.787052 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:09:36.787052 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-14ac8c93-0cda-4b24-8c11-b38cd210f5d9 tempest-FlavorsExtraSpecsNegativeTestJSON-1647059018 tempest-FlavorsExtraSpecsNegativeTestJSON-1647059018-project-admin] Returning 404 to user: Flavor 1272159723 has no extra specs with key hw:cpu_thread_policy. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:09:36.787052 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-14ac8c93-0cda-4b24-8c11-b38cd210f5d9 tempest-FlavorsExtraSpecsNegativeTestJSON-1647059018 tempest-FlavorsExtraSpecsNegativeTestJSON-1647059018-project-admin] 10.222.0.16 "DELETE /compute/v2.1/flavors/1272159723/os-extra_specs/hw:cpu_thread_policy" status: 404 len: 113 microversion: 2.1 time: 0.020760 Jul 16 21:09:36.790547 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 202/415] 10.222.0.16 () {58 vars in 1268 bytes} [Sat Jul 16 21:09:36 2022] DELETE /compute/v2.1/flavors/1272159723/os-extra_specs/hw:cpu_thread_policy => generated 113 bytes in 27 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:09:36.795290 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-220c53a7-ed16-4688-8cc2-fe5681277d49 tempest-FlavorsExtraSpecsNegativeTestJSON-1647059018 tempest-FlavorsExtraSpecsNegativeTestJSON-1647059018-project-admin] Action: 'update', calling method: >, body: {"hw:cpu_policy": "shared"} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:09:36.797259 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-220c53a7-ed16-4688-8cc2-fe5681277d49 tempest-FlavorsExtraSpecsNegativeTestJSON-1647059018 tempest-FlavorsExtraSpecsNegativeTestJSON-1647059018-project-admin] HTTP exception thrown: Request body and URI mismatch Jul 16 21:09:36.798948 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-220c53a7-ed16-4688-8cc2-fe5681277d49 tempest-FlavorsExtraSpecsNegativeTestJSON-1647059018 tempest-FlavorsExtraSpecsNegativeTestJSON-1647059018-project-admin] Request method failure captured: Jul 16 21:09:36.798948 ncdv-master-1 devstack@n-api.service[138681]: request: PUT /compute/v2.1/flavors/1272159723/os-extra_specs/hw:numa_nodes HTTP/1.1 Jul 16 21:09:36.798948 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:09:36.798948 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:09:36.798948 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:09:36.798948 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 27 Jul 16 21:09:36.798948 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:09:36.798948 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:09:36.798948 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:09:36.798948 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:09:36.798948 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0ykOPmNhFIl8yxcbfGfrqgGUGO8GzLtEVntQMUQGihVpkG87xnDtQrmIGMZtXcbmz-Sk9bVRwRmjSfZmOZUNrbyUQREiFdVEAVr_C_QxSBJFq31AcfHlKeg5KRaSqxUrz3VjPX8qs8bOS8cijlmQ2Ekqrx1ZjNhfKFXLfYYHgQbGZrM Jul 16 21:09:36.798948 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:09:36.798948 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:09:36.798948 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:09:36.798948 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:09:36.798948 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:09:36.798948 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:09:36.798948 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: 792720ef974e4498b36e3faf082bb51b Jul 16 21:09:36.798948 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-FlavorsExtraSpecsNegativeTestJSON-1647059018 Jul 16 21:09:36.798948 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,admin,reader Jul 16 21:09:36.798948 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,admin,reader Jul 16 21:09:36.800555 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/792720ef974e4498b36e3faf082bb51b"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/792720ef974e4498b36e3faf082bb51b"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_792720ef974e4498b36e3faf082bb51b"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/792720ef974e4498b36e3faf082bb51b"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:09:36.800555 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-FlavorsExtraSpecsNegativeTestJSON-1647059018 Jul 16 21:09:36.800555 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: 792720ef974e4498b36e3faf082bb51b Jul 16 21:09:36.800555 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-FlavorsExtraSpecsNegativeTestJSON-1647059018 Jul 16 21:09:36.800555 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-FlavorsExtraSpecsNegativeTestJSON-1647059018-project-admin Jul 16 21:09:36.800555 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:09:36.800555 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:09:36.800555 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: 454d449792ca4f9e8b524fb38e7576ff Jul 16 21:09:36.800555 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-FlavorsExtraSpecsNegativeTestJSON-1647059018-project-admin Jul 16 21:09:36.800555 ncdv-master-1 devstack@n-api.service[138681]: Jul 16 21:09:36.800555 ncdv-master-1 devstack@n-api.service[138681]: {"hw:cpu_policy": "shared"} Jul 16 21:09:36.800555 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:09:36.800555 ncdv-master-1 devstack@n-api.service[138681]: exception: Request body and URI mismatch Jul 16 21:09:36.801394 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'flavor_id': '1272159723', 'id': 'hw:numa_nodes', 'body': {'hw:cpu_policy': 'shared'}} Jul 16 21:09:36.801394 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:09:36.801394 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:09:36.801394 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:09:36.801394 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:09:36.801394 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:09:36.801394 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:09:36.801394 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:09:36.801394 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:09:36.801394 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 16 21:09:36.801394 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 16 21:09:36.801394 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors_extraspecs.py", line 96, in update Jul 16 21:09:36.801394 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPBadRequest(explanation=expl) Jul 16 21:09:36.801394 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Request body and URI mismatch Jul 16 21:09:36.801394 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:09:36.801394 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:09:36.801394 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:09:36.801394 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:09:36.801394 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:09:36.801394 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:09:36.802570 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:09:36.802570 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:09:36.802570 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Request body and URI mismatch Jul 16 21:09:36.802570 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:09:36.803855 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-220c53a7-ed16-4688-8cc2-fe5681277d49 tempest-FlavorsExtraSpecsNegativeTestJSON-1647059018 tempest-FlavorsExtraSpecsNegativeTestJSON-1647059018-project-admin] Returning 400 to user: Request body and URI mismatch {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:09:36.804686 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-220c53a7-ed16-4688-8cc2-fe5681277d49 tempest-FlavorsExtraSpecsNegativeTestJSON-1647059018 tempest-FlavorsExtraSpecsNegativeTestJSON-1647059018-project-admin] 10.222.0.16 "PUT /compute/v2.1/flavors/1272159723/os-extra_specs/hw:numa_nodes" status: 400 len: 73 microversion: 2.1 time: 0.013097 Jul 16 21:09:36.805321 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 203/416] 10.222.0.16 () {60 vars in 1264 bytes} [Sat Jul 16 21:09:36 2022] PUT /compute/v2.1/flavors/1272159723/os-extra_specs/hw:numa_nodes => generated 73 bytes in 14 msecs (HTTP/1.1 400) 9 headers in 380 bytes (1 switches on core 0) Jul 16 21:09:36.815936 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-ef18396c-e4e3-4eaf-9182-fff7c462dbdc tempest-FlavorsExtraSpecsNegativeTestJSON-1647059018 tempest-FlavorsExtraSpecsNegativeTestJSON-1647059018-project-admin] Action: 'update', calling method: >, body: {"hw:numa_nodes": "1", "hw:cpu_policy": "shared"} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:09:36.818699 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-ef18396c-e4e3-4eaf-9182-fff7c462dbdc tempest-FlavorsExtraSpecsNegativeTestJSON-1647059018 tempest-FlavorsExtraSpecsNegativeTestJSON-1647059018-project-admin] Request method failure captured: Jul 16 21:09:36.818699 ncdv-master-1 devstack@n-api.service[138681]: request: PUT /compute/v2.1/flavors/1272159723/os-extra_specs/hw:numa_nodes HTTP/1.1 Jul 16 21:09:36.818699 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:09:36.818699 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:09:36.818699 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:09:36.818699 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 49 Jul 16 21:09:36.818699 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:09:36.818699 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:09:36.818699 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:09:36.818699 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:09:36.818699 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0ykOPmNhFIl8yxcbfGfrqgGUGO8GzLtEVntQMUQGihVpkG87xnDtQrmIGMZtXcbmz-Sk9bVRwRmjSfZmOZUNrbyUQREiFdVEAVr_C_QxSBJFq31AcfHlKeg5KRaSqxUrz3VjPX8qs8bOS8cijlmQ2Ekqrx1ZjNhfKFXLfYYHgQbGZrM Jul 16 21:09:36.818699 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:09:36.818699 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:09:36.818699 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:09:36.818699 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:09:36.818699 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:09:36.818699 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:09:36.818699 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: 792720ef974e4498b36e3faf082bb51b Jul 16 21:09:36.818699 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-FlavorsExtraSpecsNegativeTestJSON-1647059018 Jul 16 21:09:36.818699 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,admin,reader Jul 16 21:09:36.818699 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,admin,reader Jul 16 21:09:36.819982 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/792720ef974e4498b36e3faf082bb51b"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/792720ef974e4498b36e3faf082bb51b"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_792720ef974e4498b36e3faf082bb51b"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/792720ef974e4498b36e3faf082bb51b"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:09:36.819982 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-FlavorsExtraSpecsNegativeTestJSON-1647059018 Jul 16 21:09:36.819982 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: 792720ef974e4498b36e3faf082bb51b Jul 16 21:09:36.819982 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-FlavorsExtraSpecsNegativeTestJSON-1647059018 Jul 16 21:09:36.819982 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-FlavorsExtraSpecsNegativeTestJSON-1647059018-project-admin Jul 16 21:09:36.819982 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:09:36.819982 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:09:36.819982 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: 454d449792ca4f9e8b524fb38e7576ff Jul 16 21:09:36.819982 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-FlavorsExtraSpecsNegativeTestJSON-1647059018-project-admin Jul 16 21:09:36.819982 ncdv-master-1 devstack@n-api.service[138681]: Jul 16 21:09:36.819982 ncdv-master-1 devstack@n-api.service[138681]: {"hw:numa_nodes": "1", "hw:cpu_policy": "shared"} Jul 16 21:09:36.819982 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:09:36.820629 ncdv-master-1 devstack@n-api.service[138681]: exception: {'hw:numa_nodes': '1', 'hw:cpu_policy': 'shared'} has too many properties Jul 16 21:09:36.820629 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'flavor_id': '1272159723', 'id': 'hw:numa_nodes', 'body': {'hw:numa_nodes': '1', 'hw:cpu_policy': 'shared'}} Jul 16 21:09:36.820629 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:09:36.820629 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:09:36.820629 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 279, in validate Jul 16 21:09:36.820629 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Jul 16 21:09:36.820629 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/lib/python3/dist-packages/jsonschema/validators.py", line 353, in validate Jul 16 21:09:36.820629 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise error Jul 16 21:09:36.820629 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: {'hw:numa_nodes': '1', 'hw:cpu_policy': 'shared'} has too many properties Jul 16 21:09:36.820629 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:09:36.820629 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Failed validating 'maxProperties' in schema: Jul 16 21:09:36.820629 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi {'additionalProperties': False, Jul 16 21:09:36.820629 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi 'maxProperties': 1, Jul 16 21:09:36.820629 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi 'minProperties': 1, Jul 16 21:09:36.820629 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi 'patternProperties': {'^[a-zA-Z0-9-_:. ]{1,255}$': {'maxLength': 255, Jul 16 21:09:36.820629 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi 'type': ['string', Jul 16 21:09:36.820629 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi 'number']}}, Jul 16 21:09:36.820629 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi 'type': 'object'} Jul 16 21:09:36.820629 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:09:36.820629 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi On instance: Jul 16 21:09:36.820629 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi {'hw:cpu_policy': 'shared', 'hw:numa_nodes': '1'} Jul 16 21:09:36.820629 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:09:36.826374 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:09:36.826374 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:09:36.826374 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:09:36.826374 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:09:36.826374 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:09:36.826374 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:09:36.826374 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:09:36.826374 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:09:36.826374 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:09:36.826374 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Jul 16 21:09:36.826374 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Jul 16 21:09:36.826374 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Jul 16 21:09:36.826374 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Jul 16 21:09:36.826374 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 308, in validate Jul 16 21:09:36.826374 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Jul 16 21:09:36.826374 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: {'hw:numa_nodes': '1', 'hw:cpu_policy': 'shared'} has too many properties Jul 16 21:09:36.826374 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:09:36.826374 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:09:36.826374 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:09:36.826374 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:09:36.827815 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:09:36.827815 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:09:36.827815 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Jul 16 21:09:36.827815 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Jul 16 21:09:36.827815 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: {'hw:numa_nodes': '1', 'hw:cpu_policy': 'shared'} has too many properties Jul 16 21:09:36.827815 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:09:36.827815 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-ef18396c-e4e3-4eaf-9182-fff7c462dbdc tempest-FlavorsExtraSpecsNegativeTestJSON-1647059018 tempest-FlavorsExtraSpecsNegativeTestJSON-1647059018-project-admin] Returning 400 to user: {'hw:numa_nodes': '1', 'hw:cpu_policy': 'shared'} has too many properties {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:09:36.827815 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-ef18396c-e4e3-4eaf-9182-fff7c462dbdc tempest-FlavorsExtraSpecsNegativeTestJSON-1647059018 tempest-FlavorsExtraSpecsNegativeTestJSON-1647059018-project-admin] 10.222.0.16 "PUT /compute/v2.1/flavors/1272159723/os-extra_specs/hw:numa_nodes" status: 400 len: 117 microversion: 2.1 time: 0.008134 Jul 16 21:09:36.827815 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 204/417] 10.222.0.16 () {60 vars in 1264 bytes} [Sat Jul 16 21:09:36 2022] PUT /compute/v2.1/flavors/1272159723/os-extra_specs/hw:numa_nodes => generated 117 bytes in 9 msecs (HTTP/1.1 400) 9 headers in 381 bytes (2 switches on core 0) Jul 16 21:09:36.834176 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-e81ecb24-30e9-4384-8081-e0726cb2daa7 tempest-FlavorsExtraSpecsNegativeTestJSON-1647059018 tempest-FlavorsExtraSpecsNegativeTestJSON-1647059018-project-admin] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:36.842565 ncdv-master-1 devstack@n-api.service[138681]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-flavor-manage:delete": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:09:36.842565 ncdv-master-1 devstack@n-api.service[138681]: warnings.warn(msg) Jul 16 21:09:36.843560 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-61347e93-dffe-4604-bafe-f9946bb378c1 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'abeb7020019d41a18de42956d706a011', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '649babe54d104436af461f1ee4767001', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:36.844802 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-61347e93-dffe-4604-bafe-f9946bb378c1 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] 10.222.0.16 "GET /compute/v2.1/servers/944f68af-bf99-428f-9357-9ce9f807483b" status: 200 len: 1271 microversion: 2.1 time: 0.168228 Jul 16 21:09:36.846909 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 214/418] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:09:36 2022] GET /compute/v2.1/servers/944f68af-bf99-428f-9357-9ce9f807483b => generated 1271 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:09:36.934079 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-e81ecb24-30e9-4384-8081-e0726cb2daa7 tempest-FlavorsExtraSpecsNegativeTestJSON-1647059018 tempest-FlavorsExtraSpecsNegativeTestJSON-1647059018-project-admin] 10.222.0.16 "DELETE /compute/v2.1/flavors/1272159723" status: 202 len: 0 microversion: 2.1 time: 0.104838 Jul 16 21:09:36.935189 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 205/419] 10.222.0.16 () {58 vars in 1160 bytes} [Sat Jul 16 21:09:36 2022] DELETE /compute/v2.1/flavors/1272159723 => generated 0 bytes in 107 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 21:09:36.945120 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-d2ad4549-f312-4646-a088-11e9d1515a0f tempest-FlavorsExtraSpecsNegativeTestJSON-1647059018 tempest-FlavorsExtraSpecsNegativeTestJSON-1647059018-project-admin] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:36.969979 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-d2ad4549-f312-4646-a088-11e9d1515a0f tempest-FlavorsExtraSpecsNegativeTestJSON-1647059018 tempest-FlavorsExtraSpecsNegativeTestJSON-1647059018-project-admin] 10.222.0.16 "GET /compute/v2.1/flavors/detail" status: 200 len: 5819 microversion: 2.1 time: 0.028329 Jul 16 21:09:36.970711 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 215/420] 10.222.0.16 () {58 vars in 1145 bytes} [Sat Jul 16 21:09:36 2022] GET /compute/v2.1/flavors/detail => generated 5819 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:09:37.291442 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-f4b80860-52e9-4c17-9c4f-380083fd5fbb tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:37.292330 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-f4b80860-52e9-4c17-9c4f-380083fd5fbb tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] [instance: 67266541-b43e-414c-a744-9c32c6f53975] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:09:37.300719 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f4b80860-52e9-4c17-9c4f-380083fd5fbb tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:37.301431 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f4b80860-52e9-4c17-9c4f-380083fd5fbb tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:37.302448 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f4b80860-52e9-4c17-9c4f-380083fd5fbb tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:37.303074 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f4b80860-52e9-4c17-9c4f-380083fd5fbb tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:37.358626 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-622b1a1c-fd12-421d-8413-bf7cb97b028f tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:37.359280 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-622b1a1c-fd12-421d-8413-bf7cb97b028f tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: 1703f309-9e2e-4c5f-8588-eb4e43f21d91] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:09:37.368721 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-622b1a1c-fd12-421d-8413-bf7cb97b028f tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:37.369225 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-622b1a1c-fd12-421d-8413-bf7cb97b028f tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:37.370882 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-622b1a1c-fd12-421d-8413-bf7cb97b028f tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:37.371031 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-622b1a1c-fd12-421d-8413-bf7cb97b028f tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:37.420682 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-f4b80860-52e9-4c17-9c4f-380083fd5fbb tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0d72626f465b4f37ad3b18bb945d4000', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fc1c3463f9444119af9d18f85bdd703d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:37.433269 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-f4b80860-52e9-4c17-9c4f-380083fd5fbb tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] 10.222.0.16 "GET /compute/v2.1/servers/67266541-b43e-414c-a744-9c32c6f53975" status: 200 len: 1223 microversion: 2.1 time: 0.145855 Jul 16 21:09:37.434214 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 206/421] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:09:37 2022] GET /compute/v2.1/servers/67266541-b43e-414c-a744-9c32c6f53975 => generated 1223 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:09:37.630860 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-622b1a1c-fd12-421d-8413-bf7cb97b028f tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b68dacc4b8463ba07fae6372131117', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55a19b7a4682457a9132f5fc80110712', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:37.636665 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-1210464d-480e-4149-b75c-e2d98b18d927 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:37.638611 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-1210464d-480e-4149-b75c-e2d98b18d927 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] [instance: c40ba20d-eb88-4ee8-be8f-ecbbd3315c48] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:09:37.648283 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-622b1a1c-fd12-421d-8413-bf7cb97b028f tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "GET /compute/v2.1/servers/1703f309-9e2e-4c5f-8588-eb4e43f21d91" status: 200 len: 1474 microversion: 2.1 time: 0.293666 Jul 16 21:09:37.649080 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 216/422] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:09:37 2022] GET /compute/v2.1/servers/1703f309-9e2e-4c5f-8588-eb4e43f21d91 => generated 1474 bytes in 295 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:09:37.658050 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1210464d-480e-4149-b75c-e2d98b18d927 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:37.658897 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1210464d-480e-4149-b75c-e2d98b18d927 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:37.660294 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-5189c437-0fba-4b10-a075-d09d25a8cf74 req-1f0dced3-d270-4e57-9b40-01f3144fd968 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "1703f309-9e2e-4c5f-8588-eb4e43f21d91", "name": "network-vif-unplugged", "status": "completed", "tag": "d97c4e64-34a5-4e3d-b81c-2020594e3e0b"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:09:37.660890 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1210464d-480e-4149-b75c-e2d98b18d927 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:37.661341 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1210464d-480e-4149-b75c-e2d98b18d927 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:37.670491 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-5189c437-0fba-4b10-a075-d09d25a8cf74 req-1f0dced3-d270-4e57-9b40-01f3144fd968 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:37.671376 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-5189c437-0fba-4b10-a075-d09d25a8cf74 req-1f0dced3-d270-4e57-9b40-01f3144fd968 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:37.688104 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-5189c437-0fba-4b10-a075-d09d25a8cf74 req-1f0dced3-d270-4e57-9b40-01f3144fd968 service nova] Creating event network-vif-unplugged:d97c4e64-34a5-4e3d-b81c-2020594e3e0b for instance 1703f309-9e2e-4c5f-8588-eb4e43f21d91 on nch1-master-1 Jul 16 21:09:37.709146 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [req-5189c437-0fba-4b10-a075-d09d25a8cf74 req-1f0dced3-d270-4e57-9b40-01f3144fd968 service nova] Instance 1703f309-9e2e-4c5f-8588-eb4e43f21d91 is migrating within the same cell, copying events to all relevant hosts: {'nch2-master-1', 'nch1-master-1'} {{(pid=138680) _get_relevant_hosts /opt/stack/nova/nova/compute/api.py:5807}} Jul 16 21:09:37.722698 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-5189c437-0fba-4b10-a075-d09d25a8cf74 req-1f0dced3-d270-4e57-9b40-01f3144fd968 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.066827 Jul 16 21:09:37.723598 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 217/423] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:09:37 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 68 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 16 21:09:37.731806 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-5dbf4c22-d895-4be2-b0b6-c6fde049b2c4 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:37.733146 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-5dbf4c22-d895-4be2-b0b6-c6fde049b2c4 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] [instance: cf53f7cd-deae-4b34-bf8e-3afef98b2ec6] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:09:37.741834 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5dbf4c22-d895-4be2-b0b6-c6fde049b2c4 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:37.742547 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5dbf4c22-d895-4be2-b0b6-c6fde049b2c4 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:37.743652 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5dbf4c22-d895-4be2-b0b6-c6fde049b2c4 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:37.744379 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5dbf4c22-d895-4be2-b0b6-c6fde049b2c4 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:37.878614 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-5dbf4c22-d895-4be2-b0b6-c6fde049b2c4 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ba60685a2001459cbfb531d95a6b296e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb6deae6b08e4ed8b15dc6edfe1d38d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:37.887269 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-5dbf4c22-d895-4be2-b0b6-c6fde049b2c4 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ba60685a2001459cbfb531d95a6b296e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb6deae6b08e4ed8b15dc6edfe1d38d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:37.888819 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-5dbf4c22-d895-4be2-b0b6-c6fde049b2c4 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ba60685a2001459cbfb531d95a6b296e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb6deae6b08e4ed8b15dc6edfe1d38d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:37.898019 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-5dbf4c22-d895-4be2-b0b6-c6fde049b2c4 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] 10.222.0.16 "GET /compute/v2.1/servers/cf53f7cd-deae-4b34-bf8e-3afef98b2ec6" status: 200 len: 1438 microversion: 2.75 time: 0.171513 Jul 16 21:09:37.899248 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 218/424] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:09:37 2022] GET /compute/v2.1/servers/cf53f7cd-deae-4b34-bf8e-3afef98b2ec6 => generated 1438 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 16 21:09:37.905283 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-934d067a-f6c8-431c-aac2-5322c291cc77 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:37.906099 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-934d067a-f6c8-431c-aac2-5322c291cc77 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] [instance: 944f68af-bf99-428f-9357-9ce9f807483b] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:09:37.914856 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-934d067a-f6c8-431c-aac2-5322c291cc77 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:37.915552 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-934d067a-f6c8-431c-aac2-5322c291cc77 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:37.917192 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-934d067a-f6c8-431c-aac2-5322c291cc77 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:37.917945 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-934d067a-f6c8-431c-aac2-5322c291cc77 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:38.047370 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-1210464d-480e-4149-b75c-e2d98b18d927 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a69f9dee03d43beaafa966216b9e289', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9b769bf8f3ea4e368bf68de1df6538f9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:38.056066 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-1210464d-480e-4149-b75c-e2d98b18d927 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] 10.222.0.16 "GET /compute/v2.1/servers/c40ba20d-eb88-4ee8-be8f-ecbbd3315c48" status: 200 len: 1310 microversion: 2.1 time: 0.426142 Jul 16 21:09:38.056651 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 207/425] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:09:37 2022] GET /compute/v2.1/servers/c40ba20d-eb88-4ee8-be8f-ecbbd3315c48 => generated 1310 bytes in 427 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:09:38.075071 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-f4b7c98b-23de-4751-81c9-56b9c8a5715b tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Action: 'action', calling method: >, body: {"rebuild": {"imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9"}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:09:38.076618 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-f4b7c98b-23de-4751-81c9-56b9c8a5715b tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] [instance: cf53f7cd-deae-4b34-bf8e-3afef98b2ec6] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:09:38.086834 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-934d067a-f6c8-431c-aac2-5322c291cc77 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'abeb7020019d41a18de42956d706a011', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '649babe54d104436af461f1ee4767001', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:38.097824 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-934d067a-f6c8-431c-aac2-5322c291cc77 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] 10.222.0.16 "GET /compute/v2.1/servers/944f68af-bf99-428f-9357-9ce9f807483b" status: 200 len: 1279 microversion: 2.1 time: 0.197307 Jul 16 21:09:38.102069 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f4b7c98b-23de-4751-81c9-56b9c8a5715b tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:38.102523 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f4b7c98b-23de-4751-81c9-56b9c8a5715b tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:38.103331 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f4b7c98b-23de-4751-81c9-56b9c8a5715b tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:38.103775 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f4b7c98b-23de-4751-81c9-56b9c8a5715b tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:38.107911 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 219/426] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:09:37 2022] GET /compute/v2.1/servers/944f68af-bf99-428f-9357-9ce9f807483b => generated 1279 bytes in 208 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:09:38.238470 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-f4b7c98b-23de-4751-81c9-56b9c8a5715b tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Flavor limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 16 21:09:38.239052 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-f4b7c98b-23de-4751-81c9-56b9c8a5715b tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Image limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 16 21:09:38.239632 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-f4b7c98b-23de-4751-81c9-56b9c8a5715b tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Flavor pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 16 21:09:38.240155 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-f4b7c98b-23de-4751-81c9-56b9c8a5715b tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Image pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 16 21:09:38.240646 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-f4b7c98b-23de-4751-81c9-56b9c8a5715b tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 16 21:09:38.451059 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-d7e76be8-6439-4e00-995a-3df0d6811214 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:38.451698 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-d7e76be8-6439-4e00-995a-3df0d6811214 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] [instance: 67266541-b43e-414c-a744-9c32c6f53975] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:09:38.462599 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d7e76be8-6439-4e00-995a-3df0d6811214 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:38.467511 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d7e76be8-6439-4e00-995a-3df0d6811214 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:38.468600 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d7e76be8-6439-4e00-995a-3df0d6811214 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:38.469192 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d7e76be8-6439-4e00-995a-3df0d6811214 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:38.502964 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=100 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:09:38.594202 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-d7e76be8-6439-4e00-995a-3df0d6811214 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0d72626f465b4f37ad3b18bb945d4000', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fc1c3463f9444119af9d18f85bdd703d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:38.597455 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-f4b7c98b-23de-4751-81c9-56b9c8a5715b tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] [instance: cf53f7cd-deae-4b34-bf8e-3afef98b2ec6] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:09:38.604720 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-d7e76be8-6439-4e00-995a-3df0d6811214 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] 10.222.0.16 "GET /compute/v2.1/servers/67266541-b43e-414c-a744-9c32c6f53975" status: 200 len: 1271 microversion: 2.1 time: 0.157342 Jul 16 21:09:38.604952 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f4b7c98b-23de-4751-81c9-56b9c8a5715b tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:38.605291 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f4b7c98b-23de-4751-81c9-56b9c8a5715b tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:38.606538 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 220/427] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:09:38 2022] GET /compute/v2.1/servers/67266541-b43e-414c-a744-9c32c6f53975 => generated 1271 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:09:38.606813 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f4b7c98b-23de-4751-81c9-56b9c8a5715b tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:38.607330 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f4b7c98b-23de-4751-81c9-56b9c8a5715b tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:38.667792 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-25e131bc-315c-488d-8754-094df7062368 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:38.668438 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-25e131bc-315c-488d-8754-094df7062368 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: 1703f309-9e2e-4c5f-8588-eb4e43f21d91] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:09:38.676459 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-25e131bc-315c-488d-8754-094df7062368 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:38.676916 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-25e131bc-315c-488d-8754-094df7062368 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:38.678037 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-25e131bc-315c-488d-8754-094df7062368 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:38.678622 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-25e131bc-315c-488d-8754-094df7062368 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:38.702333 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-f4b7c98b-23de-4751-81c9-56b9c8a5715b tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ba60685a2001459cbfb531d95a6b296e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb6deae6b08e4ed8b15dc6edfe1d38d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:38.715387 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-f4b7c98b-23de-4751-81c9-56b9c8a5715b tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ba60685a2001459cbfb531d95a6b296e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb6deae6b08e4ed8b15dc6edfe1d38d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:38.717364 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-f4b7c98b-23de-4751-81c9-56b9c8a5715b tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ba60685a2001459cbfb531d95a6b296e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb6deae6b08e4ed8b15dc6edfe1d38d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:38.724664 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-f4b7c98b-23de-4751-81c9-56b9c8a5715b tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] 10.222.0.16 "POST /compute/v2.1/servers/cf53f7cd-deae-4b34-bf8e-3afef98b2ec6/action" status: 202 len: 1495 microversion: 2.75 time: 0.663007 Jul 16 21:09:38.726123 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 208/428] 10.222.0.16 () {62 vars in 1318 bytes} [Sat Jul 16 21:09:38 2022] POST /compute/v2.1/servers/cf53f7cd-deae-4b34-bf8e-3afef98b2ec6/action => generated 1495 bytes in 665 msecs (HTTP/1.1 202) 10 headers in 454 bytes (1 switches on core 0) Jul 16 21:09:38.748922 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-6055ffbb-15a3-4402-9a9c-76642d0742e4 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:38.752468 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-6055ffbb-15a3-4402-9a9c-76642d0742e4 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] [instance: cf53f7cd-deae-4b34-bf8e-3afef98b2ec6] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:09:38.764753 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6055ffbb-15a3-4402-9a9c-76642d0742e4 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:38.765214 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6055ffbb-15a3-4402-9a9c-76642d0742e4 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:38.766028 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6055ffbb-15a3-4402-9a9c-76642d0742e4 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:38.766470 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6055ffbb-15a3-4402-9a9c-76642d0742e4 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:38.803181 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:09:38.977657 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-6055ffbb-15a3-4402-9a9c-76642d0742e4 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ba60685a2001459cbfb531d95a6b296e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb6deae6b08e4ed8b15dc6edfe1d38d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:38.988544 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-6055ffbb-15a3-4402-9a9c-76642d0742e4 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ba60685a2001459cbfb531d95a6b296e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb6deae6b08e4ed8b15dc6edfe1d38d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:38.990320 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-6055ffbb-15a3-4402-9a9c-76642d0742e4 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ba60685a2001459cbfb531d95a6b296e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb6deae6b08e4ed8b15dc6edfe1d38d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:38.996301 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-6055ffbb-15a3-4402-9a9c-76642d0742e4 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] 10.222.0.16 "GET /compute/v2.1/servers/cf53f7cd-deae-4b34-bf8e-3afef98b2ec6" status: 200 len: 1447 microversion: 2.75 time: 0.254635 Jul 16 21:09:38.997558 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 209/429] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:09:38 2022] GET /compute/v2.1/servers/cf53f7cd-deae-4b34-bf8e-3afef98b2ec6 => generated 1447 bytes in 258 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 16 21:09:39.086331 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-8b9bbadc-931b-4b12-a4a7-820a1a275867 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:39.087622 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-8b9bbadc-931b-4b12-a4a7-820a1a275867 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] [instance: c40ba20d-eb88-4ee8-be8f-ecbbd3315c48] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:09:39.124621 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8b9bbadc-931b-4b12-a4a7-820a1a275867 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:39.130792 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8b9bbadc-931b-4b12-a4a7-820a1a275867 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:39.134104 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8b9bbadc-931b-4b12-a4a7-820a1a275867 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:39.134957 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8b9bbadc-931b-4b12-a4a7-820a1a275867 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:39.234900 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-25e131bc-315c-488d-8754-094df7062368 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b68dacc4b8463ba07fae6372131117', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55a19b7a4682457a9132f5fc80110712', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:39.243174 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-25e131bc-315c-488d-8754-094df7062368 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "GET /compute/v2.1/servers/1703f309-9e2e-4c5f-8588-eb4e43f21d91" status: 200 len: 1474 microversion: 2.1 time: 0.579868 Jul 16 21:09:39.253152 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 221/430] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:09:38 2022] GET /compute/v2.1/servers/1703f309-9e2e-4c5f-8588-eb4e43f21d91 => generated 1474 bytes in 591 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:09:39.260543 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-f65ed39d-2e58-45d1-9402-e4938a8ddb90 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:39.260543 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-f65ed39d-2e58-45d1-9402-e4938a8ddb90 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] [instance: 944f68af-bf99-428f-9357-9ce9f807483b] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:09:39.264930 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f65ed39d-2e58-45d1-9402-e4938a8ddb90 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:39.265600 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f65ed39d-2e58-45d1-9402-e4938a8ddb90 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:39.266381 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f65ed39d-2e58-45d1-9402-e4938a8ddb90 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:39.267422 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f65ed39d-2e58-45d1-9402-e4938a8ddb90 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:39.366811 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-f65ed39d-2e58-45d1-9402-e4938a8ddb90 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'abeb7020019d41a18de42956d706a011', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '649babe54d104436af461f1ee4767001', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:39.374769 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-f65ed39d-2e58-45d1-9402-e4938a8ddb90 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] 10.222.0.16 "GET /compute/v2.1/servers/944f68af-bf99-428f-9357-9ce9f807483b" status: 200 len: 1277 microversion: 2.1 time: 0.120387 Jul 16 21:09:39.385977 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 222/431] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:09:39 2022] GET /compute/v2.1/servers/944f68af-bf99-428f-9357-9ce9f807483b => generated 1277 bytes in 132 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:09:39.571833 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-8b9bbadc-931b-4b12-a4a7-820a1a275867 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a69f9dee03d43beaafa966216b9e289', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9b769bf8f3ea4e368bf68de1df6538f9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:39.579489 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-8b9bbadc-931b-4b12-a4a7-820a1a275867 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] 10.222.0.16 "GET /compute/v2.1/servers/c40ba20d-eb88-4ee8-be8f-ecbbd3315c48" status: 200 len: 1310 microversion: 2.1 time: 0.500705 Jul 16 21:09:39.580069 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 210/432] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:09:39 2022] GET /compute/v2.1/servers/c40ba20d-eb88-4ee8-be8f-ecbbd3315c48 => generated 1310 bytes in 503 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:09:39.630151 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-d5a7f13b-852a-41e8-89c3-443a589973e5 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:39.630762 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-d5a7f13b-852a-41e8-89c3-443a589973e5 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] [instance: 67266541-b43e-414c-a744-9c32c6f53975] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:09:39.637690 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d5a7f13b-852a-41e8-89c3-443a589973e5 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:39.639308 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d5a7f13b-852a-41e8-89c3-443a589973e5 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:39.639308 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d5a7f13b-852a-41e8-89c3-443a589973e5 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:39.639308 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d5a7f13b-852a-41e8-89c3-443a589973e5 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:39.746952 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-5aeb79e0-111a-4e88-8814-dcd1332b7d13 req-5b5b74ae-2b47-437b-88eb-dc88837edd02 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "1703f309-9e2e-4c5f-8588-eb4e43f21d91", "tag": "d97c4e64-34a5-4e3d-b81c-2020594e3e0b"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:09:39.749017 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-d5a7f13b-852a-41e8-89c3-443a589973e5 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0d72626f465b4f37ad3b18bb945d4000', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fc1c3463f9444119af9d18f85bdd703d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:39.757219 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-5aeb79e0-111a-4e88-8814-dcd1332b7d13 req-5b5b74ae-2b47-437b-88eb-dc88837edd02 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:39.757447 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-d5a7f13b-852a-41e8-89c3-443a589973e5 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] 10.222.0.16 "GET /compute/v2.1/servers/67266541-b43e-414c-a744-9c32c6f53975" status: 200 len: 1277 microversion: 2.1 time: 0.131614 Jul 16 21:09:39.757862 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 223/433] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:09:39 2022] GET /compute/v2.1/servers/67266541-b43e-414c-a744-9c32c6f53975 => generated 1277 bytes in 133 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:09:39.758266 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-5aeb79e0-111a-4e88-8814-dcd1332b7d13 req-5b5b74ae-2b47-437b-88eb-dc88837edd02 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:39.780555 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-5aeb79e0-111a-4e88-8814-dcd1332b7d13 req-5b5b74ae-2b47-437b-88eb-dc88837edd02 service nova] Creating event network-changed:d97c4e64-34a5-4e3d-b81c-2020594e3e0b for instance 1703f309-9e2e-4c5f-8588-eb4e43f21d91 on nch1-master-1 Jul 16 21:09:39.792754 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [req-5aeb79e0-111a-4e88-8814-dcd1332b7d13 req-5b5b74ae-2b47-437b-88eb-dc88837edd02 service nova] Instance 1703f309-9e2e-4c5f-8588-eb4e43f21d91 is migrating within the same cell, copying events to all relevant hosts: {'nch2-master-1', 'nch1-master-1'} {{(pid=138681) _get_relevant_hosts /opt/stack/nova/nova/compute/api.py:5807}} Jul 16 21:09:39.805862 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-5aeb79e0-111a-4e88-8814-dcd1332b7d13 req-5b5b74ae-2b47-437b-88eb-dc88837edd02 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.062357 Jul 16 21:09:39.806482 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 211/434] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:09:39 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 64 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 16 21:09:40.022729 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-36adbdac-3a12-4105-9ca0-bf610b2657a6 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:40.025555 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-36adbdac-3a12-4105-9ca0-bf610b2657a6 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] [instance: cf53f7cd-deae-4b34-bf8e-3afef98b2ec6] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:09:40.050430 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-36adbdac-3a12-4105-9ca0-bf610b2657a6 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:40.056235 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-36adbdac-3a12-4105-9ca0-bf610b2657a6 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:40.058475 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-36adbdac-3a12-4105-9ca0-bf610b2657a6 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:40.058934 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-36adbdac-3a12-4105-9ca0-bf610b2657a6 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:40.203790 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-36adbdac-3a12-4105-9ca0-bf610b2657a6 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ba60685a2001459cbfb531d95a6b296e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb6deae6b08e4ed8b15dc6edfe1d38d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:40.230464 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-36adbdac-3a12-4105-9ca0-bf610b2657a6 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ba60685a2001459cbfb531d95a6b296e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb6deae6b08e4ed8b15dc6edfe1d38d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:40.232819 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-36adbdac-3a12-4105-9ca0-bf610b2657a6 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ba60685a2001459cbfb531d95a6b296e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb6deae6b08e4ed8b15dc6edfe1d38d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:40.240420 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-36adbdac-3a12-4105-9ca0-bf610b2657a6 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] 10.222.0.16 "GET /compute/v2.1/servers/cf53f7cd-deae-4b34-bf8e-3afef98b2ec6" status: 200 len: 1447 microversion: 2.75 time: 0.227017 Jul 16 21:09:40.241602 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 224/435] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:09:40 2022] GET /compute/v2.1/servers/cf53f7cd-deae-4b34-bf8e-3afef98b2ec6 => generated 1447 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:09:40.267618 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-33b3e816-3402-4052-aedf-8ecdbe06f5d5 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:40.268351 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-33b3e816-3402-4052-aedf-8ecdbe06f5d5 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: 1703f309-9e2e-4c5f-8588-eb4e43f21d91] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:09:40.291102 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-33b3e816-3402-4052-aedf-8ecdbe06f5d5 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:40.291844 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-33b3e816-3402-4052-aedf-8ecdbe06f5d5 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:40.294568 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-33b3e816-3402-4052-aedf-8ecdbe06f5d5 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:40.295521 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-33b3e816-3402-4052-aedf-8ecdbe06f5d5 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:40.402218 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-024ab202-daa0-4fa4-ba4b-ec4784824756 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:40.406233 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-024ab202-daa0-4fa4-ba4b-ec4784824756 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] [instance: 944f68af-bf99-428f-9357-9ce9f807483b] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:09:40.441772 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-024ab202-daa0-4fa4-ba4b-ec4784824756 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:40.443763 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-024ab202-daa0-4fa4-ba4b-ec4784824756 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:40.443763 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-024ab202-daa0-4fa4-ba4b-ec4784824756 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:40.445658 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-024ab202-daa0-4fa4-ba4b-ec4784824756 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:41.098958 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-33b3e816-3402-4052-aedf-8ecdbe06f5d5 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b68dacc4b8463ba07fae6372131117', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55a19b7a4682457a9132f5fc80110712', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:41.108596 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-33b3e816-3402-4052-aedf-8ecdbe06f5d5 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "GET /compute/v2.1/servers/1703f309-9e2e-4c5f-8588-eb4e43f21d91" status: 200 len: 1474 microversion: 2.1 time: 0.847342 Jul 16 21:09:41.110017 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 212/436] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:09:40 2022] GET /compute/v2.1/servers/1703f309-9e2e-4c5f-8588-eb4e43f21d91 => generated 1474 bytes in 850 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:09:41.118573 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-20d603f4-68db-423b-b6d9-52d6451f02c8 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:41.120174 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-20d603f4-68db-423b-b6d9-52d6451f02c8 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] [instance: c40ba20d-eb88-4ee8-be8f-ecbbd3315c48] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:09:41.121237 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-024ab202-daa0-4fa4-ba4b-ec4784824756 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'abeb7020019d41a18de42956d706a011', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '649babe54d104436af461f1ee4767001', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:41.130328 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-20d603f4-68db-423b-b6d9-52d6451f02c8 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:41.130328 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-20d603f4-68db-423b-b6d9-52d6451f02c8 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:41.130629 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-024ab202-daa0-4fa4-ba4b-ec4784824756 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] 10.222.0.16 "GET /compute/v2.1/servers/944f68af-bf99-428f-9357-9ce9f807483b" status: 200 len: 1319 microversion: 2.1 time: 0.740490 Jul 16 21:09:41.132257 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-20d603f4-68db-423b-b6d9-52d6451f02c8 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:41.132927 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-20d603f4-68db-423b-b6d9-52d6451f02c8 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:41.144058 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 225/437] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:09:40 2022] GET /compute/v2.1/servers/944f68af-bf99-428f-9357-9ce9f807483b => generated 1319 bytes in 755 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:09:41.156370 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-051cb127-f9b7-40ac-ab93-5e93c142ddfd tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:41.158392 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-051cb127-f9b7-40ac-ab93-5e93c142ddfd tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] [instance: 67266541-b43e-414c-a744-9c32c6f53975] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:09:41.167683 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-051cb127-f9b7-40ac-ab93-5e93c142ddfd tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:41.168598 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-051cb127-f9b7-40ac-ab93-5e93c142ddfd tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:41.170067 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-051cb127-f9b7-40ac-ab93-5e93c142ddfd tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:41.170680 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-051cb127-f9b7-40ac-ab93-5e93c142ddfd tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:41.270009 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-051cb127-f9b7-40ac-ab93-5e93c142ddfd tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0d72626f465b4f37ad3b18bb945d4000', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fc1c3463f9444119af9d18f85bdd703d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:41.281017 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-051cb127-f9b7-40ac-ab93-5e93c142ddfd tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] 10.222.0.16 "GET /compute/v2.1/servers/67266541-b43e-414c-a744-9c32c6f53975" status: 200 len: 1277 microversion: 2.1 time: 0.131624 Jul 16 21:09:41.281017 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 226/438] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:09:41 2022] GET /compute/v2.1/servers/67266541-b43e-414c-a744-9c32c6f53975 => generated 1277 bytes in 136 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:09:41.286312 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-5549a3ec-24da-4ac2-a2f6-f0845a583d09 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:41.287140 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-5549a3ec-24da-4ac2-a2f6-f0845a583d09 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] [instance: cf53f7cd-deae-4b34-bf8e-3afef98b2ec6] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:09:41.293477 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5549a3ec-24da-4ac2-a2f6-f0845a583d09 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:41.293926 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5549a3ec-24da-4ac2-a2f6-f0845a583d09 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:41.294710 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5549a3ec-24da-4ac2-a2f6-f0845a583d09 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:41.295198 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5549a3ec-24da-4ac2-a2f6-f0845a583d09 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:41.390262 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-5549a3ec-24da-4ac2-a2f6-f0845a583d09 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ba60685a2001459cbfb531d95a6b296e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb6deae6b08e4ed8b15dc6edfe1d38d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:41.410924 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-5549a3ec-24da-4ac2-a2f6-f0845a583d09 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ba60685a2001459cbfb531d95a6b296e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb6deae6b08e4ed8b15dc6edfe1d38d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:41.410924 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-5549a3ec-24da-4ac2-a2f6-f0845a583d09 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ba60685a2001459cbfb531d95a6b296e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb6deae6b08e4ed8b15dc6edfe1d38d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:41.411337 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-20d603f4-68db-423b-b6d9-52d6451f02c8 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a69f9dee03d43beaafa966216b9e289', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9b769bf8f3ea4e368bf68de1df6538f9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:41.416209 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-5549a3ec-24da-4ac2-a2f6-f0845a583d09 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] 10.222.0.16 "GET /compute/v2.1/servers/cf53f7cd-deae-4b34-bf8e-3afef98b2ec6" status: 200 len: 1447 microversion: 2.75 time: 0.133991 Jul 16 21:09:41.420248 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-20d603f4-68db-423b-b6d9-52d6451f02c8 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] 10.222.0.16 "GET /compute/v2.1/servers/c40ba20d-eb88-4ee8-be8f-ecbbd3315c48" status: 200 len: 1310 microversion: 2.1 time: 0.308317 Jul 16 21:09:41.425382 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 213/439] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:09:41 2022] GET /compute/v2.1/servers/c40ba20d-eb88-4ee8-be8f-ecbbd3315c48 => generated 1310 bytes in 315 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:09:41.429960 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 227/440] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:09:41 2022] GET /compute/v2.1/servers/cf53f7cd-deae-4b34-bf8e-3afef98b2ec6 => generated 1447 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:09:42.134212 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-69761ff1-58dd-491f-a5a8-07510ea3de7e tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:42.136139 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-69761ff1-58dd-491f-a5a8-07510ea3de7e tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: 1703f309-9e2e-4c5f-8588-eb4e43f21d91] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:09:42.148773 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-c19894aa-8a3d-4f49-889b-19a88dc6a40b tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:42.150075 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-c19894aa-8a3d-4f49-889b-19a88dc6a40b tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] [instance: 944f68af-bf99-428f-9357-9ce9f807483b] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:09:42.160474 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-69761ff1-58dd-491f-a5a8-07510ea3de7e tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:42.161305 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c19894aa-8a3d-4f49-889b-19a88dc6a40b tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:42.168114 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-69761ff1-58dd-491f-a5a8-07510ea3de7e tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:42.170941 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-69761ff1-58dd-491f-a5a8-07510ea3de7e tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:42.173659 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-69761ff1-58dd-491f-a5a8-07510ea3de7e tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:42.183448 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c19894aa-8a3d-4f49-889b-19a88dc6a40b tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:42.188315 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c19894aa-8a3d-4f49-889b-19a88dc6a40b tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:42.188653 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c19894aa-8a3d-4f49-889b-19a88dc6a40b tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:42.682669 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-69761ff1-58dd-491f-a5a8-07510ea3de7e tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b68dacc4b8463ba07fae6372131117', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55a19b7a4682457a9132f5fc80110712', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:42.690865 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-69761ff1-58dd-491f-a5a8-07510ea3de7e tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "GET /compute/v2.1/servers/1703f309-9e2e-4c5f-8588-eb4e43f21d91" status: 200 len: 1472 microversion: 2.1 time: 0.566217 Jul 16 21:09:42.706155 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 214/441] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:09:42 2022] GET /compute/v2.1/servers/1703f309-9e2e-4c5f-8588-eb4e43f21d91 => generated 1472 bytes in 578 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:09:42.711836 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-c6226585-fa4d-4ba6-8df0-a24d5dee80e0 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:42.712689 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-c6226585-fa4d-4ba6-8df0-a24d5dee80e0 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] [instance: 67266541-b43e-414c-a744-9c32c6f53975] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:09:42.724148 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c6226585-fa4d-4ba6-8df0-a24d5dee80e0 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:42.728790 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c6226585-fa4d-4ba6-8df0-a24d5dee80e0 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:42.728790 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c6226585-fa4d-4ba6-8df0-a24d5dee80e0 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:42.729803 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c6226585-fa4d-4ba6-8df0-a24d5dee80e0 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:42.740049 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-c19894aa-8a3d-4f49-889b-19a88dc6a40b tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'abeb7020019d41a18de42956d706a011', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '649babe54d104436af461f1ee4767001', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:42.750405 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-c19894aa-8a3d-4f49-889b-19a88dc6a40b tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] 10.222.0.16 "GET /compute/v2.1/servers/944f68af-bf99-428f-9357-9ce9f807483b" status: 200 len: 1319 microversion: 2.1 time: 0.606520 Jul 16 21:09:42.751585 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 228/442] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:09:42 2022] GET /compute/v2.1/servers/944f68af-bf99-428f-9357-9ce9f807483b => generated 1319 bytes in 609 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:09:42.756722 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-ce55c5c7-a4f2-4bdd-b8aa-7a36bb612954 req-5b3d9fae-d814-4d20-a3a9-59a1320a9bca service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "c40ba20d-eb88-4ee8-be8f-ecbbd3315c48", "name": "network-vif-plugged", "status": "completed", "tag": "a2fac50b-598b-43e0-8e39-546a5c1b4490"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:09:42.776558 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-ce55c5c7-a4f2-4bdd-b8aa-7a36bb612954 req-5b3d9fae-d814-4d20-a3a9-59a1320a9bca service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:42.777384 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-ce55c5c7-a4f2-4bdd-b8aa-7a36bb612954 req-5b3d9fae-d814-4d20-a3a9-59a1320a9bca service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:42.792205 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-ce55c5c7-a4f2-4bdd-b8aa-7a36bb612954 req-5b3d9fae-d814-4d20-a3a9-59a1320a9bca service nova] Creating event network-vif-plugged:a2fac50b-598b-43e0-8e39-546a5c1b4490 for instance c40ba20d-eb88-4ee8-be8f-ecbbd3315c48 on nch1-master-1 Jul 16 21:09:42.801724 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-ce55c5c7-a4f2-4bdd-b8aa-7a36bb612954 req-5b3d9fae-d814-4d20-a3a9-59a1320a9bca service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.048341 Jul 16 21:09:42.802955 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 229/443] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:09:42 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:09:42.810642 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-30a97476-a594-4939-aa73-eb680819438d tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:42.811683 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-30a97476-a594-4939-aa73-eb680819438d tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] [instance: cf53f7cd-deae-4b34-bf8e-3afef98b2ec6] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:09:42.819214 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-30a97476-a594-4939-aa73-eb680819438d tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:42.819897 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-30a97476-a594-4939-aa73-eb680819438d tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:42.820895 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-30a97476-a594-4939-aa73-eb680819438d tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:42.821551 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-30a97476-a594-4939-aa73-eb680819438d tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:43.032724 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-30a97476-a594-4939-aa73-eb680819438d tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ba60685a2001459cbfb531d95a6b296e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb6deae6b08e4ed8b15dc6edfe1d38d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:43.046904 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-30a97476-a594-4939-aa73-eb680819438d tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ba60685a2001459cbfb531d95a6b296e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb6deae6b08e4ed8b15dc6edfe1d38d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:43.049240 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-30a97476-a594-4939-aa73-eb680819438d tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ba60685a2001459cbfb531d95a6b296e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb6deae6b08e4ed8b15dc6edfe1d38d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:43.065489 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-30a97476-a594-4939-aa73-eb680819438d tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] 10.222.0.16 "GET /compute/v2.1/servers/cf53f7cd-deae-4b34-bf8e-3afef98b2ec6" status: 200 len: 1447 microversion: 2.75 time: 0.260938 Jul 16 21:09:43.084075 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 230/444] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:09:42 2022] GET /compute/v2.1/servers/cf53f7cd-deae-4b34-bf8e-3afef98b2ec6 => generated 1447 bytes in 280 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:09:43.090861 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-bbe29c86-f344-41f2-a532-d33b85952f51 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:43.094547 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-bbe29c86-f344-41f2-a532-d33b85952f51 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] [instance: c40ba20d-eb88-4ee8-be8f-ecbbd3315c48] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:09:43.113158 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-bbe29c86-f344-41f2-a532-d33b85952f51 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:43.113877 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-bbe29c86-f344-41f2-a532-d33b85952f51 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:43.114897 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-bbe29c86-f344-41f2-a532-d33b85952f51 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:43.115461 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-bbe29c86-f344-41f2-a532-d33b85952f51 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:43.340425 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-c6226585-fa4d-4ba6-8df0-a24d5dee80e0 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0d72626f465b4f37ad3b18bb945d4000', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fc1c3463f9444119af9d18f85bdd703d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:43.364388 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-c6226585-fa4d-4ba6-8df0-a24d5dee80e0 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] 10.222.0.16 "GET /compute/v2.1/servers/67266541-b43e-414c-a744-9c32c6f53975" status: 200 len: 1319 microversion: 2.1 time: 0.660877 Jul 16 21:09:43.369839 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 215/445] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:09:42 2022] GET /compute/v2.1/servers/67266541-b43e-414c-a744-9c32c6f53975 => generated 1319 bytes in 668 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:09:43.389230 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-4108e297-00bd-494c-a5a1-9681d96d1b5d req-677a9f88-e73a-4179-a9c7-1b4d6ee4cda3 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "45164bce-7acf-4ef6-8afa-aaab6287ecf4", "tag": "14e29168-1029-4ef2-8854-0a02dfa53951"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:09:43.421143 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell0:SELECT=4 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:09:43.434062 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-4108e297-00bd-494c-a5a1-9681d96d1b5d req-677a9f88-e73a-4179-a9c7-1b4d6ee4cda3 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:43.436185 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-4108e297-00bd-494c-a5a1-9681d96d1b5d req-677a9f88-e73a-4179-a9c7-1b4d6ee4cda3 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:43.468358 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-4108e297-00bd-494c-a5a1-9681d96d1b5d req-677a9f88-e73a-4179-a9c7-1b4d6ee4cda3 service nova] Creating event network-changed:14e29168-1029-4ef2-8854-0a02dfa53951 for instance 45164bce-7acf-4ef6-8afa-aaab6287ecf4 on nch2-master-1 Jul 16 21:09:43.481419 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-4108e297-00bd-494c-a5a1-9681d96d1b5d req-677a9f88-e73a-4179-a9c7-1b4d6ee4cda3 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.110143 Jul 16 21:09:43.484855 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 216/446] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:09:43 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 114 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:09:43.581015 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-bbe29c86-f344-41f2-a532-d33b85952f51 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a69f9dee03d43beaafa966216b9e289', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9b769bf8f3ea4e368bf68de1df6538f9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:43.597734 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-bbe29c86-f344-41f2-a532-d33b85952f51 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] 10.222.0.16 "GET /compute/v2.1/servers/c40ba20d-eb88-4ee8-be8f-ecbbd3315c48" status: 200 len: 1310 microversion: 2.1 time: 0.512223 Jul 16 21:09:43.602091 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 231/447] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:09:43 2022] GET /compute/v2.1/servers/c40ba20d-eb88-4ee8-be8f-ecbbd3315c48 => generated 1310 bytes in 517 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:09:43.715422 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-05bc6105-0378-48eb-8f87-86396726f378 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:43.716271 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-05bc6105-0378-48eb-8f87-86396726f378 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: 1703f309-9e2e-4c5f-8588-eb4e43f21d91] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:09:43.729441 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-05bc6105-0378-48eb-8f87-86396726f378 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:43.730152 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-05bc6105-0378-48eb-8f87-86396726f378 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:43.731168 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-05bc6105-0378-48eb-8f87-86396726f378 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:43.731764 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-05bc6105-0378-48eb-8f87-86396726f378 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:43.779628 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-5fbc4c08-8614-4985-a19e-a5e3bc32d198 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:43.781272 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-5fbc4c08-8614-4985-a19e-a5e3bc32d198 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] [instance: 944f68af-bf99-428f-9357-9ce9f807483b] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:09:43.784690 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_api:SELECT=87,nova_api:INSERT=5,nova_api:SAVEPOINT=4,nova_api:RELEASE=4 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:09:43.792285 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5fbc4c08-8614-4985-a19e-a5e3bc32d198 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:43.792285 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5fbc4c08-8614-4985-a19e-a5e3bc32d198 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:43.792577 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5fbc4c08-8614-4985-a19e-a5e3bc32d198 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:43.792780 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5fbc4c08-8614-4985-a19e-a5e3bc32d198 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:44.231078 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-5fbc4c08-8614-4985-a19e-a5e3bc32d198 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'abeb7020019d41a18de42956d706a011', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '649babe54d104436af461f1ee4767001', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:44.239631 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-05bc6105-0378-48eb-8f87-86396726f378 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b68dacc4b8463ba07fae6372131117', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55a19b7a4682457a9132f5fc80110712', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:44.241322 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-5fbc4c08-8614-4985-a19e-a5e3bc32d198 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] 10.222.0.16 "GET /compute/v2.1/servers/944f68af-bf99-428f-9357-9ce9f807483b" status: 200 len: 1319 microversion: 2.1 time: 0.472403 Jul 16 21:09:44.243070 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 232/448] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:09:43 2022] GET /compute/v2.1/servers/944f68af-bf99-428f-9357-9ce9f807483b => generated 1319 bytes in 479 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:09:44.248928 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-28735c64-4904-42e2-94e2-21d3cb51bc27 req-ea30fa84-7f99-40fd-b798-e274801c0f99 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "944f68af-bf99-428f-9357-9ce9f807483b", "tag": "5556f1e0-fe84-46bd-9585-73295be09188"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:09:44.249146 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-05bc6105-0378-48eb-8f87-86396726f378 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "GET /compute/v2.1/servers/1703f309-9e2e-4c5f-8588-eb4e43f21d91" status: 200 len: 1469 microversion: 2.1 time: 0.539037 Jul 16 21:09:44.262652 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-28735c64-4904-42e2-94e2-21d3cb51bc27 req-ea30fa84-7f99-40fd-b798-e274801c0f99 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:44.262845 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 217/449] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:09:43 2022] GET /compute/v2.1/servers/1703f309-9e2e-4c5f-8588-eb4e43f21d91 => generated 1469 bytes in 551 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:09:44.264816 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-28735c64-4904-42e2-94e2-21d3cb51bc27 req-ea30fa84-7f99-40fd-b798-e274801c0f99 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:44.267956 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-985fbfba-7b0d-4e8e-b858-7f6f667f6bee tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:44.268810 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-985fbfba-7b0d-4e8e-b858-7f6f667f6bee tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] [instance: cf53f7cd-deae-4b34-bf8e-3afef98b2ec6] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:09:44.277984 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-985fbfba-7b0d-4e8e-b858-7f6f667f6bee tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:44.278694 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-28735c64-4904-42e2-94e2-21d3cb51bc27 req-ea30fa84-7f99-40fd-b798-e274801c0f99 service nova] Creating event network-changed:5556f1e0-fe84-46bd-9585-73295be09188 for instance 944f68af-bf99-428f-9357-9ce9f807483b on nch2-master-1 Jul 16 21:09:44.278853 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-985fbfba-7b0d-4e8e-b858-7f6f667f6bee tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:44.279857 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-985fbfba-7b0d-4e8e-b858-7f6f667f6bee tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:44.280449 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-985fbfba-7b0d-4e8e-b858-7f6f667f6bee tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:44.285755 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-28735c64-4904-42e2-94e2-21d3cb51bc27 req-ea30fa84-7f99-40fd-b798-e274801c0f99 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.041135 Jul 16 21:09:44.288047 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 233/450] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:09:44 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:09:44.302182 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-6b47b3d2-6f4d-4bbe-9ff6-416448054b6f tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Action: 'action', calling method: >, body: {"confirmResize": {}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:09:44.305382 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-6b47b3d2-6f4d-4bbe-9ff6-416448054b6f tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: 1703f309-9e2e-4c5f-8588-eb4e43f21d91] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:09:44.318803 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6b47b3d2-6f4d-4bbe-9ff6-416448054b6f tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:44.320937 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6b47b3d2-6f4d-4bbe-9ff6-416448054b6f tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:44.324500 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6b47b3d2-6f4d-4bbe-9ff6-416448054b6f tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:44.325371 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6b47b3d2-6f4d-4bbe-9ff6-416448054b6f tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:44.398415 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-985fbfba-7b0d-4e8e-b858-7f6f667f6bee tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ba60685a2001459cbfb531d95a6b296e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb6deae6b08e4ed8b15dc6edfe1d38d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:44.411464 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-985fbfba-7b0d-4e8e-b858-7f6f667f6bee tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ba60685a2001459cbfb531d95a6b296e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb6deae6b08e4ed8b15dc6edfe1d38d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:44.414053 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-985fbfba-7b0d-4e8e-b858-7f6f667f6bee tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ba60685a2001459cbfb531d95a6b296e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb6deae6b08e4ed8b15dc6edfe1d38d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:44.422078 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-985fbfba-7b0d-4e8e-b858-7f6f667f6bee tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] 10.222.0.16 "GET /compute/v2.1/servers/cf53f7cd-deae-4b34-bf8e-3afef98b2ec6" status: 200 len: 1447 microversion: 2.75 time: 0.158142 Jul 16 21:09:44.423965 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 218/451] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:09:44 2022] GET /compute/v2.1/servers/cf53f7cd-deae-4b34-bf8e-3afef98b2ec6 => generated 1447 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:09:44.429450 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-efd69941-541f-4aca-9704-2f06d132c79f tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:44.430254 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-efd69941-541f-4aca-9704-2f06d132c79f tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] [instance: 67266541-b43e-414c-a744-9c32c6f53975] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:09:44.437942 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-efd69941-541f-4aca-9704-2f06d132c79f tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:44.438930 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-efd69941-541f-4aca-9704-2f06d132c79f tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:44.440978 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-efd69941-541f-4aca-9704-2f06d132c79f tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:44.441838 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-efd69941-541f-4aca-9704-2f06d132c79f tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:44.607401 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-efd69941-541f-4aca-9704-2f06d132c79f tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0d72626f465b4f37ad3b18bb945d4000', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fc1c3463f9444119af9d18f85bdd703d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:44.624916 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-efd69941-541f-4aca-9704-2f06d132c79f tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] 10.222.0.16 "GET /compute/v2.1/servers/67266541-b43e-414c-a744-9c32c6f53975" status: 200 len: 1319 microversion: 2.1 time: 0.199392 Jul 16 21:09:44.633786 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 219/452] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:09:44 2022] GET /compute/v2.1/servers/67266541-b43e-414c-a744-9c32c6f53975 => generated 1319 bytes in 204 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:09:44.638869 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-6984c9bd-bd18-4586-8d9b-289aa557d6c2 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:44.641251 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-6984c9bd-bd18-4586-8d9b-289aa557d6c2 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] [instance: c40ba20d-eb88-4ee8-be8f-ecbbd3315c48] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:09:44.660276 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6984c9bd-bd18-4586-8d9b-289aa557d6c2 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:44.661005 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6984c9bd-bd18-4586-8d9b-289aa557d6c2 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:44.662038 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6984c9bd-bd18-4586-8d9b-289aa557d6c2 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:44.662636 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6984c9bd-bd18-4586-8d9b-289aa557d6c2 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:44.685013 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-6b47b3d2-6f4d-4bbe-9ff6-416448054b6f tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "POST /compute/v2.1/servers/1703f309-9e2e-4c5f-8588-eb4e43f21d91/action" status: 204 len: 0 microversion: 2.1 time: 0.394642 Jul 16 21:09:44.687437 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 234/453] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 21:09:44 2022] POST /compute/v2.1/servers/1703f309-9e2e-4c5f-8588-eb4e43f21d91/action => generated 0 bytes in 397 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 16 21:09:44.696430 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-1b20cefa-ee14-42c1-841d-228acce7314b tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:44.699221 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-1b20cefa-ee14-42c1-841d-228acce7314b tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: 1703f309-9e2e-4c5f-8588-eb4e43f21d91] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:09:44.714729 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1b20cefa-ee14-42c1-841d-228acce7314b tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:44.715215 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1b20cefa-ee14-42c1-841d-228acce7314b tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:44.718619 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1b20cefa-ee14-42c1-841d-228acce7314b tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:44.719099 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1b20cefa-ee14-42c1-841d-228acce7314b tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:45.110653 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-6984c9bd-bd18-4586-8d9b-289aa557d6c2 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a69f9dee03d43beaafa966216b9e289', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9b769bf8f3ea4e368bf68de1df6538f9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:45.127304 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-6984c9bd-bd18-4586-8d9b-289aa557d6c2 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] 10.222.0.16 "GET /compute/v2.1/servers/c40ba20d-eb88-4ee8-be8f-ecbbd3315c48" status: 200 len: 1493 microversion: 2.1 time: 0.496173 Jul 16 21:09:45.129518 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 220/454] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:09:44 2022] GET /compute/v2.1/servers/c40ba20d-eb88-4ee8-be8f-ecbbd3315c48 => generated 1493 bytes in 500 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:09:45.155674 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-1b20cefa-ee14-42c1-841d-228acce7314b tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b68dacc4b8463ba07fae6372131117', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55a19b7a4682457a9132f5fc80110712', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:45.169845 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-1b20cefa-ee14-42c1-841d-228acce7314b tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "GET /compute/v2.1/servers/1703f309-9e2e-4c5f-8588-eb4e43f21d91" status: 200 len: 1469 microversion: 2.1 time: 0.478287 Jul 16 21:09:45.170928 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 235/455] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:09:44 2022] GET /compute/v2.1/servers/1703f309-9e2e-4c5f-8588-eb4e43f21d91 => generated 1469 bytes in 480 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:09:45.428035 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-41342812-55d2-4cc9-bbe5-986df8a552bc tempest-LiveMigrationTest-1131758637 tempest-LiveMigrationTest-1131758637-project-admin] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:45.429117 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-41342812-55d2-4cc9-bbe5-986df8a552bc tempest-LiveMigrationTest-1131758637 tempest-LiveMigrationTest-1131758637-project-admin] [instance: c40ba20d-eb88-4ee8-be8f-ecbbd3315c48] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:09:45.438945 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-41342812-55d2-4cc9-bbe5-986df8a552bc tempest-LiveMigrationTest-1131758637 tempest-LiveMigrationTest-1131758637-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:45.439751 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-41342812-55d2-4cc9-bbe5-986df8a552bc tempest-LiveMigrationTest-1131758637 tempest-LiveMigrationTest-1131758637-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:45.440565 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-41342812-55d2-4cc9-bbe5-986df8a552bc tempest-LiveMigrationTest-1131758637 tempest-LiveMigrationTest-1131758637-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:45.441376 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-41342812-55d2-4cc9-bbe5-986df8a552bc tempest-LiveMigrationTest-1131758637 tempest-LiveMigrationTest-1131758637-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:45.509650 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-ea43175e-f4cc-4e08-a080-2f8642fe9cd5 tempest-HypervisorAdminTestJSON-528937790 tempest-HypervisorAdminTestJSON-528937790-project-admin] Calling method '.version_select at 0x7fdd9d8d3c10>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:45.510616 ncdv-master-1 devstack@n-api.service[138681]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-hypervisors:list": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:09:45.510616 ncdv-master-1 devstack@n-api.service[138681]: warnings.warn(msg) Jul 16 21:09:45.517815 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ea43175e-f4cc-4e08-a080-2f8642fe9cd5 tempest-HypervisorAdminTestJSON-528937790 tempest-HypervisorAdminTestJSON-528937790-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:45.517815 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ea43175e-f4cc-4e08-a080-2f8642fe9cd5 tempest-HypervisorAdminTestJSON-528937790 tempest-HypervisorAdminTestJSON-528937790-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:45.530300 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ea43175e-f4cc-4e08-a080-2f8642fe9cd5 tempest-HypervisorAdminTestJSON-528937790 tempest-HypervisorAdminTestJSON-528937790-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:45.530855 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ea43175e-f4cc-4e08-a080-2f8642fe9cd5 tempest-HypervisorAdminTestJSON-528937790 tempest-HypervisorAdminTestJSON-528937790-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:45.542270 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ea43175e-f4cc-4e08-a080-2f8642fe9cd5 tempest-HypervisorAdminTestJSON-528937790 tempest-HypervisorAdminTestJSON-528937790-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:45.542728 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ea43175e-f4cc-4e08-a080-2f8642fe9cd5 tempest-HypervisorAdminTestJSON-528937790 tempest-HypervisorAdminTestJSON-528937790-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:45.545109 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell0:SELECT=6 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:09:45.551659 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-ea43175e-f4cc-4e08-a080-2f8642fe9cd5 tempest-HypervisorAdminTestJSON-528937790 tempest-HypervisorAdminTestJSON-528937790-project-admin] 10.222.0.16 "GET /compute/v2.1/os-hypervisors" status: 200 len: 191 microversion: 2.1 time: 0.420806 Jul 16 21:09:45.552536 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 221/456] 10.222.0.16 () {58 vars in 1145 bytes} [Sat Jul 16 21:09:45 2022] GET /compute/v2.1/os-hypervisors => generated 191 bytes in 422 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:09:45.553729 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-ea43175e-f4cc-4e08-a080-2f8642fe9cd5 tempest-HypervisorAdminTestJSON-528937790 tempest-HypervisorAdminTestJSON-528937790-project-admin] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:45.554819 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-ea43175e-f4cc-4e08-a080-2f8642fe9cd5 tempest-HypervisorAdminTestJSON-528937790 tempest-HypervisorAdminTestJSON-528937790-project-admin] 10.222.0.16 "GET /compute//" status: 200 len: 377 microversion: - time: 0.001305 Jul 16 21:09:45.555846 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 222/457] 10.222.0.16 () {56 vars in 885 bytes} [Sat Jul 16 21:09:45 2022] GET /compute/ => generated 377 bytes in 2 msecs (HTTP/1.1 200) 3 headers in 91 bytes (1 switches on core 0) Jul 16 21:09:45.568045 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-a5c8eeaa-b6a8-4a79-b756-8f8eb97795ab tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:45.568962 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-a5c8eeaa-b6a8-4a79-b756-8f8eb97795ab tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] [instance: 944f68af-bf99-428f-9357-9ce9f807483b] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:09:45.576212 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell0:SELECT=24 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:09:45.580000 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a5c8eeaa-b6a8-4a79-b756-8f8eb97795ab tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:45.580754 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a5c8eeaa-b6a8-4a79-b756-8f8eb97795ab tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:45.581859 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a5c8eeaa-b6a8-4a79-b756-8f8eb97795ab tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:45.582504 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a5c8eeaa-b6a8-4a79-b756-8f8eb97795ab tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:46.270672 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell0:SELECT=38 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:09:46.274145 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-41342812-55d2-4cc9-bbe5-986df8a552bc tempest-LiveMigrationTest-1131758637 tempest-LiveMigrationTest-1131758637-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/c40ba20d-eb88-4ee8-be8f-ecbbd3315c48" status: 200 len: 1644 microversion: 2.1 time: 1.101852 Jul 16 21:09:46.276511 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-a5c8eeaa-b6a8-4a79-b756-8f8eb97795ab tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'abeb7020019d41a18de42956d706a011', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '649babe54d104436af461f1ee4767001', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:46.285392 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 236/458] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:09:45 2022] GET /compute/v2.1/servers/c40ba20d-eb88-4ee8-be8f-ecbbd3315c48 => generated 1644 bytes in 1113 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:09:46.290340 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-a5c8eeaa-b6a8-4a79-b756-8f8eb97795ab tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] 10.222.0.16 "GET /compute/v2.1/servers/944f68af-bf99-428f-9357-9ce9f807483b" status: 200 len: 1319 microversion: 2.1 time: 0.733338 Jul 16 21:09:46.302172 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-ce35fc1e-4733-4759-9288-adfe6b749d17 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:46.302172 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-ce35fc1e-4733-4759-9288-adfe6b749d17 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] [instance: cf53f7cd-deae-4b34-bf8e-3afef98b2ec6] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:09:46.302414 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 223/459] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:09:45 2022] GET /compute/v2.1/servers/944f68af-bf99-428f-9357-9ce9f807483b => generated 1319 bytes in 738 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:09:46.302504 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ce35fc1e-4733-4759-9288-adfe6b749d17 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:46.302504 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ce35fc1e-4733-4759-9288-adfe6b749d17 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:46.304086 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ce35fc1e-4733-4759-9288-adfe6b749d17 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:46.306366 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ce35fc1e-4733-4759-9288-adfe6b749d17 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:46.306667 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-2927cf17-eb63-406c-856d-9e9e32344bba tempest-HypervisorAdminTestJSON-528937790 tempest-HypervisorAdminTestJSON-528937790-project-admin] Calling method '.version_select at 0x7fdd9d8b1430>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:46.307721 ncdv-master-1 devstack@n-api.service[138681]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-hypervisors:list-detail": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:09:46.307721 ncdv-master-1 devstack@n-api.service[138681]: warnings.warn(msg) Jul 16 21:09:46.309418 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2927cf17-eb63-406c-856d-9e9e32344bba tempest-HypervisorAdminTestJSON-528937790 tempest-HypervisorAdminTestJSON-528937790-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:46.311499 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2927cf17-eb63-406c-856d-9e9e32344bba tempest-HypervisorAdminTestJSON-528937790 tempest-HypervisorAdminTestJSON-528937790-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:46.330523 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:09:46.349489 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2927cf17-eb63-406c-856d-9e9e32344bba tempest-HypervisorAdminTestJSON-528937790 tempest-HypervisorAdminTestJSON-528937790-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:46.355469 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2927cf17-eb63-406c-856d-9e9e32344bba tempest-HypervisorAdminTestJSON-528937790 tempest-HypervisorAdminTestJSON-528937790-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:46.380662 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2927cf17-eb63-406c-856d-9e9e32344bba tempest-HypervisorAdminTestJSON-528937790 tempest-HypervisorAdminTestJSON-528937790-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:46.381214 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2927cf17-eb63-406c-856d-9e9e32344bba tempest-HypervisorAdminTestJSON-528937790 tempest-HypervisorAdminTestJSON-528937790-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:46.388651 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-2927cf17-eb63-406c-856d-9e9e32344bba tempest-HypervisorAdminTestJSON-528937790 tempest-HypervisorAdminTestJSON-528937790-project-admin] 10.222.0.16 "GET /compute/v2.1/os-hypervisors/detail" status: 200 len: 1504 microversion: 2.1 time: 0.093116 Jul 16 21:09:46.389742 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 224/460] 10.222.0.16 () {58 vars in 1166 bytes} [Sat Jul 16 21:09:46 2022] GET /compute/v2.1/os-hypervisors/detail => generated 1504 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:09:46.390033 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-ce35fc1e-4733-4759-9288-adfe6b749d17 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ba60685a2001459cbfb531d95a6b296e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb6deae6b08e4ed8b15dc6edfe1d38d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:46.396479 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-aee9def5-db88-49ef-8b62-6e17b9fc76b0 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:46.397200 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-aee9def5-db88-49ef-8b62-6e17b9fc76b0 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] [instance: 67266541-b43e-414c-a744-9c32c6f53975] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:09:46.403963 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-aee9def5-db88-49ef-8b62-6e17b9fc76b0 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:46.404550 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-aee9def5-db88-49ef-8b62-6e17b9fc76b0 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:46.405454 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-aee9def5-db88-49ef-8b62-6e17b9fc76b0 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:46.405997 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-aee9def5-db88-49ef-8b62-6e17b9fc76b0 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:46.406624 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-ce35fc1e-4733-4759-9288-adfe6b749d17 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ba60685a2001459cbfb531d95a6b296e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb6deae6b08e4ed8b15dc6edfe1d38d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:46.409549 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-ce35fc1e-4733-4759-9288-adfe6b749d17 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ba60685a2001459cbfb531d95a6b296e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb6deae6b08e4ed8b15dc6edfe1d38d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:46.416514 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-ce35fc1e-4733-4759-9288-adfe6b749d17 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] 10.222.0.16 "GET /compute/v2.1/servers/cf53f7cd-deae-4b34-bf8e-3afef98b2ec6" status: 200 len: 1447 microversion: 2.75 time: 0.126767 Jul 16 21:09:46.417311 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 237/461] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:09:46 2022] GET /compute/v2.1/servers/cf53f7cd-deae-4b34-bf8e-3afef98b2ec6 => generated 1447 bytes in 131 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:09:46.422940 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-845d7bef-9f6d-42e7-aaad-41ade17edcf9 req-0c8db49a-c69f-49ef-a34c-84dc4ea6bbd5 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "1703f309-9e2e-4c5f-8588-eb4e43f21d91", "name": "network-vif-plugged", "status": "completed", "tag": "d97c4e64-34a5-4e3d-b81c-2020594e3e0b"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:09:46.437394 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-845d7bef-9f6d-42e7-aaad-41ade17edcf9 req-0c8db49a-c69f-49ef-a34c-84dc4ea6bbd5 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:46.438048 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-845d7bef-9f6d-42e7-aaad-41ade17edcf9 req-0c8db49a-c69f-49ef-a34c-84dc4ea6bbd5 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:46.453636 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-845d7bef-9f6d-42e7-aaad-41ade17edcf9 req-0c8db49a-c69f-49ef-a34c-84dc4ea6bbd5 service nova] Creating event network-vif-plugged:d97c4e64-34a5-4e3d-b81c-2020594e3e0b for instance 1703f309-9e2e-4c5f-8588-eb4e43f21d91 on nch1-master-1 Jul 16 21:09:46.463053 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [req-845d7bef-9f6d-42e7-aaad-41ade17edcf9 req-0c8db49a-c69f-49ef-a34c-84dc4ea6bbd5 service nova] Instance 1703f309-9e2e-4c5f-8588-eb4e43f21d91 is migrating within the same cell, copying events to all relevant hosts: {'nch2-master-1', 'nch1-master-1'} {{(pid=138680) _get_relevant_hosts /opt/stack/nova/nova/compute/api.py:5807}} Jul 16 21:09:46.474813 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-845d7bef-9f6d-42e7-aaad-41ade17edcf9 req-0c8db49a-c69f-49ef-a34c-84dc4ea6bbd5 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.056131 Jul 16 21:09:46.475960 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 238/462] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:09:46 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:09:46.484275 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-84cdc6ea-8d5a-4bc0-813c-100498bc255e tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:46.485259 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-84cdc6ea-8d5a-4bc0-813c-100498bc255e tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: 1703f309-9e2e-4c5f-8588-eb4e43f21d91] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:09:46.492250 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-84cdc6ea-8d5a-4bc0-813c-100498bc255e tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:46.492888 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-84cdc6ea-8d5a-4bc0-813c-100498bc255e tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:46.493816 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-84cdc6ea-8d5a-4bc0-813c-100498bc255e tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:46.494349 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-84cdc6ea-8d5a-4bc0-813c-100498bc255e tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:46.659559 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-aee9def5-db88-49ef-8b62-6e17b9fc76b0 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0d72626f465b4f37ad3b18bb945d4000', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fc1c3463f9444119af9d18f85bdd703d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:46.667185 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-aee9def5-db88-49ef-8b62-6e17b9fc76b0 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] 10.222.0.16 "GET /compute/v2.1/servers/67266541-b43e-414c-a744-9c32c6f53975" status: 200 len: 1319 microversion: 2.1 time: 0.276106 Jul 16 21:09:46.667756 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 225/463] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:09:46 2022] GET /compute/v2.1/servers/67266541-b43e-414c-a744-9c32c6f53975 => generated 1319 bytes in 277 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:09:46.677254 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-480ff4df-1537-4c48-a36c-edbbf94c87df tempest-LiveMigrationTest-1131758637 tempest-LiveMigrationTest-1131758637-project-admin] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:46.677254 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-480ff4df-1537-4c48-a36c-edbbf94c87df tempest-LiveMigrationTest-1131758637 tempest-LiveMigrationTest-1131758637-project-admin] [instance: c40ba20d-eb88-4ee8-be8f-ecbbd3315c48] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:09:46.680115 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-480ff4df-1537-4c48-a36c-edbbf94c87df tempest-LiveMigrationTest-1131758637 tempest-LiveMigrationTest-1131758637-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:46.680564 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-480ff4df-1537-4c48-a36c-edbbf94c87df tempest-LiveMigrationTest-1131758637 tempest-LiveMigrationTest-1131758637-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:46.681340 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-480ff4df-1537-4c48-a36c-edbbf94c87df tempest-LiveMigrationTest-1131758637 tempest-LiveMigrationTest-1131758637-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:46.681776 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-480ff4df-1537-4c48-a36c-edbbf94c87df tempest-LiveMigrationTest-1131758637 tempest-LiveMigrationTest-1131758637-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:46.697057 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-84cdc6ea-8d5a-4bc0-813c-100498bc255e tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b68dacc4b8463ba07fae6372131117', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55a19b7a4682457a9132f5fc80110712', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:46.705245 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-84cdc6ea-8d5a-4bc0-813c-100498bc255e tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "GET /compute/v2.1/servers/1703f309-9e2e-4c5f-8588-eb4e43f21d91" status: 200 len: 1469 microversion: 2.1 time: 0.225536 Jul 16 21:09:46.706231 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 239/464] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:09:46 2022] GET /compute/v2.1/servers/1703f309-9e2e-4c5f-8588-eb4e43f21d91 => generated 1469 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:09:46.712354 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-7cd80369-549f-486f-a07b-0e5cbe41cba5 req-5491eb27-bfae-4447-a30e-4d4307e5afee service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "67266541-b43e-414c-a744-9c32c6f53975", "tag": "f15f3108-545f-4afa-9b93-b7dbd6384d3a"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:09:46.723668 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-7cd80369-549f-486f-a07b-0e5cbe41cba5 req-5491eb27-bfae-4447-a30e-4d4307e5afee service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:46.724492 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-7cd80369-549f-486f-a07b-0e5cbe41cba5 req-5491eb27-bfae-4447-a30e-4d4307e5afee service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:46.737899 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-7cd80369-549f-486f-a07b-0e5cbe41cba5 req-5491eb27-bfae-4447-a30e-4d4307e5afee service nova] Creating event network-changed:f15f3108-545f-4afa-9b93-b7dbd6384d3a for instance 67266541-b43e-414c-a744-9c32c6f53975 on nch2-master-1 Jul 16 21:09:46.743539 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-7cd80369-549f-486f-a07b-0e5cbe41cba5 req-5491eb27-bfae-4447-a30e-4d4307e5afee service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.035994 Jul 16 21:09:46.744338 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 240/465] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:09:46 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:09:46.750712 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-ce2afd7f-88a8-4027-9911-97744529566e tempest-HypervisorAdminTestJSON-528937790 tempest-HypervisorAdminTestJSON-528937790-project-admin] Calling method '.version_select at 0x7fdd9d434af0>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:46.751987 ncdv-master-1 devstack@n-api.service[138680]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-hypervisors:list": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:09:46.751987 ncdv-master-1 devstack@n-api.service[138680]: warnings.warn(msg) Jul 16 21:09:46.753423 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ce2afd7f-88a8-4027-9911-97744529566e tempest-HypervisorAdminTestJSON-528937790 tempest-HypervisorAdminTestJSON-528937790-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:46.753878 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ce2afd7f-88a8-4027-9911-97744529566e tempest-HypervisorAdminTestJSON-528937790 tempest-HypervisorAdminTestJSON-528937790-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:46.770024 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ce2afd7f-88a8-4027-9911-97744529566e tempest-HypervisorAdminTestJSON-528937790 tempest-HypervisorAdminTestJSON-528937790-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:46.770491 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ce2afd7f-88a8-4027-9911-97744529566e tempest-HypervisorAdminTestJSON-528937790 tempest-HypervisorAdminTestJSON-528937790-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:46.782436 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ce2afd7f-88a8-4027-9911-97744529566e tempest-HypervisorAdminTestJSON-528937790 tempest-HypervisorAdminTestJSON-528937790-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:46.782873 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ce2afd7f-88a8-4027-9911-97744529566e tempest-HypervisorAdminTestJSON-528937790 tempest-HypervisorAdminTestJSON-528937790-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:46.789258 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-ce2afd7f-88a8-4027-9911-97744529566e tempest-HypervisorAdminTestJSON-528937790 tempest-HypervisorAdminTestJSON-528937790-project-admin] 10.222.0.16 "GET /compute/v2.1/os-hypervisors" status: 200 len: 191 microversion: 2.1 time: 0.043622 Jul 16 21:09:46.790166 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 241/466] 10.222.0.16 () {58 vars in 1145 bytes} [Sat Jul 16 21:09:46 2022] GET /compute/v2.1/os-hypervisors => generated 191 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:09:46.800247 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-2a03e040-5b8c-49bc-a401-e23fe4b4dadd tempest-HypervisorAdminTestJSON-528937790 tempest-HypervisorAdminTestJSON-528937790-project-admin] Calling method '.version_select at 0x7fdd9d4345e0>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:46.801044 ncdv-master-1 devstack@n-api.service[138680]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-hypervisors:show": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:09:46.801044 ncdv-master-1 devstack@n-api.service[138680]: warnings.warn(msg) Jul 16 21:09:46.802293 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2a03e040-5b8c-49bc-a401-e23fe4b4dadd tempest-HypervisorAdminTestJSON-528937790 tempest-HypervisorAdminTestJSON-528937790-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:46.802748 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2a03e040-5b8c-49bc-a401-e23fe4b4dadd tempest-HypervisorAdminTestJSON-528937790 tempest-HypervisorAdminTestJSON-528937790-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:46.815986 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2a03e040-5b8c-49bc-a401-e23fe4b4dadd tempest-HypervisorAdminTestJSON-528937790 tempest-HypervisorAdminTestJSON-528937790-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:46.816442 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2a03e040-5b8c-49bc-a401-e23fe4b4dadd tempest-HypervisorAdminTestJSON-528937790 tempest-HypervisorAdminTestJSON-528937790-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:46.823610 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-2a03e040-5b8c-49bc-a401-e23fe4b4dadd tempest-HypervisorAdminTestJSON-528937790 tempest-HypervisorAdminTestJSON-528937790-project-admin] 10.222.0.16 "GET /compute/v2.1/os-hypervisors/1" status: 200 len: 758 microversion: 2.1 time: 0.027456 Jul 16 21:09:46.824982 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 242/467] 10.222.0.16 () {58 vars in 1151 bytes} [Sat Jul 16 21:09:46 2022] GET /compute/v2.1/os-hypervisors/1 => generated 758 bytes in 30 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 16 21:09:46.837372 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-52ada41c-d046-4f38-91e4-9ca5ef241199 tempest-HypervisorAdminTestJSON-528937790 tempest-HypervisorAdminTestJSON-528937790-project-admin] Calling method '.version_select at 0x7fdd9d405280>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:46.838157 ncdv-master-1 devstack@n-api.service[138680]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-hypervisors:statistics": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:09:46.838157 ncdv-master-1 devstack@n-api.service[138680]: warnings.warn(msg) Jul 16 21:09:46.839300 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-52ada41c-d046-4f38-91e4-9ca5ef241199 tempest-HypervisorAdminTestJSON-528937790 tempest-HypervisorAdminTestJSON-528937790-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:46.839756 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-52ada41c-d046-4f38-91e4-9ca5ef241199 tempest-HypervisorAdminTestJSON-528937790 tempest-HypervisorAdminTestJSON-528937790-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:46.857576 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-52ada41c-d046-4f38-91e4-9ca5ef241199 tempest-HypervisorAdminTestJSON-528937790 tempest-HypervisorAdminTestJSON-528937790-project-admin] 10.222.0.16 "GET /compute/v2.1/os-hypervisors/statistics" status: 200 len: 267 microversion: 2.1 time: 0.023927 Jul 16 21:09:46.858256 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 243/468] 10.222.0.16 () {58 vars in 1178 bytes} [Sat Jul 16 21:09:46 2022] GET /compute/v2.1/os-hypervisors/statistics => generated 267 bytes in 25 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 16 21:09:46.882181 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-aa7270e2-8aa9-44c1-b3db-16d281b00099 tempest-HypervisorAdminTestJSON-528937790 tempest-HypervisorAdminTestJSON-528937790-project-admin] Calling method '.version_select at 0x7fdd9d434b80>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:46.882181 ncdv-master-1 devstack@n-api.service[138680]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-hypervisors:list": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:09:46.882181 ncdv-master-1 devstack@n-api.service[138680]: warnings.warn(msg) Jul 16 21:09:46.882181 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-aa7270e2-8aa9-44c1-b3db-16d281b00099 tempest-HypervisorAdminTestJSON-528937790 tempest-HypervisorAdminTestJSON-528937790-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:46.882508 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-aa7270e2-8aa9-44c1-b3db-16d281b00099 tempest-HypervisorAdminTestJSON-528937790 tempest-HypervisorAdminTestJSON-528937790-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:46.897565 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-aa7270e2-8aa9-44c1-b3db-16d281b00099 tempest-HypervisorAdminTestJSON-528937790 tempest-HypervisorAdminTestJSON-528937790-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:46.898026 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-aa7270e2-8aa9-44c1-b3db-16d281b00099 tempest-HypervisorAdminTestJSON-528937790 tempest-HypervisorAdminTestJSON-528937790-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:46.910362 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-aa7270e2-8aa9-44c1-b3db-16d281b00099 tempest-HypervisorAdminTestJSON-528937790 tempest-HypervisorAdminTestJSON-528937790-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:46.910816 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-aa7270e2-8aa9-44c1-b3db-16d281b00099 tempest-HypervisorAdminTestJSON-528937790 tempest-HypervisorAdminTestJSON-528937790-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:46.913667 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-480ff4df-1537-4c48-a36c-edbbf94c87df tempest-LiveMigrationTest-1131758637 tempest-LiveMigrationTest-1131758637-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/c40ba20d-eb88-4ee8-be8f-ecbbd3315c48" status: 200 len: 1644 microversion: 2.1 time: 0.244689 Jul 16 21:09:46.914256 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 226/469] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:09:46 2022] GET /compute/v2.1/servers/c40ba20d-eb88-4ee8-be8f-ecbbd3315c48 => generated 1644 bytes in 246 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:09:46.918933 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-aa7270e2-8aa9-44c1-b3db-16d281b00099 tempest-HypervisorAdminTestJSON-528937790 tempest-HypervisorAdminTestJSON-528937790-project-admin] 10.222.0.16 "GET /compute/v2.1/os-hypervisors" status: 200 len: 191 microversion: 2.1 time: 0.042882 Jul 16 21:09:46.919772 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 244/470] 10.222.0.16 () {58 vars in 1145 bytes} [Sat Jul 16 21:09:46 2022] GET /compute/v2.1/os-hypervisors => generated 191 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:09:46.930433 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-3bf7f75d-b914-4247-a719-2db4abb5e0c7 tempest-HypervisorAdminTestJSON-528937790 tempest-HypervisorAdminTestJSON-528937790-project-admin] Calling method '.version_select at 0x7fdd9d8c59d0>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:46.931262 ncdv-master-1 devstack@n-api.service[138681]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-hypervisors:show": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:09:46.931262 ncdv-master-1 devstack@n-api.service[138681]: warnings.warn(msg) Jul 16 21:09:46.931623 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-c0c4fdb2-92de-43a3-85ad-9c3b02791193 tempest-LiveMigrationTest-1131758637 tempest-LiveMigrationTest-1131758637-project-admin] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:46.932451 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3bf7f75d-b914-4247-a719-2db4abb5e0c7 tempest-HypervisorAdminTestJSON-528937790 tempest-HypervisorAdminTestJSON-528937790-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:46.932972 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3bf7f75d-b914-4247-a719-2db4abb5e0c7 tempest-HypervisorAdminTestJSON-528937790 tempest-HypervisorAdminTestJSON-528937790-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:46.933399 ncdv-master-1 devstack@n-api.service[138680]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-services:list": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:09:46.933399 ncdv-master-1 devstack@n-api.service[138680]: warnings.warn(msg) Jul 16 21:09:46.935048 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c0c4fdb2-92de-43a3-85ad-9c3b02791193 tempest-LiveMigrationTest-1131758637 tempest-LiveMigrationTest-1131758637-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:46.935487 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c0c4fdb2-92de-43a3-85ad-9c3b02791193 tempest-LiveMigrationTest-1131758637 tempest-LiveMigrationTest-1131758637-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:46.936856 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c0c4fdb2-92de-43a3-85ad-9c3b02791193 tempest-LiveMigrationTest-1131758637 tempest-LiveMigrationTest-1131758637-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:46.937363 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c0c4fdb2-92de-43a3-85ad-9c3b02791193 tempest-LiveMigrationTest-1131758637 tempest-LiveMigrationTest-1131758637-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:46.951324 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3bf7f75d-b914-4247-a719-2db4abb5e0c7 tempest-HypervisorAdminTestJSON-528937790 tempest-HypervisorAdminTestJSON-528937790-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:46.951835 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3bf7f75d-b914-4247-a719-2db4abb5e0c7 tempest-HypervisorAdminTestJSON-528937790 tempest-HypervisorAdminTestJSON-528937790-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:46.960347 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-3bf7f75d-b914-4247-a719-2db4abb5e0c7 tempest-HypervisorAdminTestJSON-528937790 tempest-HypervisorAdminTestJSON-528937790-project-admin] 10.222.0.16 "GET /compute/v2.1/os-hypervisors/1" status: 200 len: 758 microversion: 2.1 time: 0.032665 Jul 16 21:09:46.960978 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 227/471] 10.222.0.16 () {58 vars in 1151 bytes} [Sat Jul 16 21:09:46 2022] GET /compute/v2.1/os-hypervisors/1 => generated 758 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:09:46.974933 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-a215c5a6-9c34-471b-91d9-fbf1bed0875c tempest-HypervisorAdminTestJSON-528937790 tempest-HypervisorAdminTestJSON-528937790-project-admin] Calling method '.version_select at 0x7fdd9d8c5820>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:46.976457 ncdv-master-1 devstack@n-api.service[138681]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-hypervisors:show": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:09:46.976457 ncdv-master-1 devstack@n-api.service[138681]: warnings.warn(msg) Jul 16 21:09:46.978018 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a215c5a6-9c34-471b-91d9-fbf1bed0875c tempest-HypervisorAdminTestJSON-528937790 tempest-HypervisorAdminTestJSON-528937790-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:46.978934 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a215c5a6-9c34-471b-91d9-fbf1bed0875c tempest-HypervisorAdminTestJSON-528937790 tempest-HypervisorAdminTestJSON-528937790-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:46.980279 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-c0c4fdb2-92de-43a3-85ad-9c3b02791193 tempest-LiveMigrationTest-1131758637 tempest-LiveMigrationTest-1131758637-project-admin] 10.222.0.16 "GET /compute/v2.1/os-services?binary=nova-compute" status: 200 len: 380 microversion: 2.1 time: 0.052414 Jul 16 21:09:46.980390 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 245/472] 10.222.0.16 () {58 vars in 1175 bytes} [Sat Jul 16 21:09:46 2022] GET /compute/v2.1/os-services?binary=nova-compute => generated 380 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:09:46.992727 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-ffa6b6ef-d1fd-4587-9b63-a6d2bfa0c508 tempest-LiveMigrationTest-1131758637 tempest-LiveMigrationTest-1131758637-project-admin] Action: 'action', calling method: >, body: {"os-migrateLive": {"host": "nch2-master-1", "block_migration": true, "disk_over_commit": false}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:09:46.994498 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-ffa6b6ef-d1fd-4587-9b63-a6d2bfa0c508 tempest-LiveMigrationTest-1131758637 tempest-LiveMigrationTest-1131758637-project-admin] [instance: c40ba20d-eb88-4ee8-be8f-ecbbd3315c48] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:09:46.995873 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a215c5a6-9c34-471b-91d9-fbf1bed0875c tempest-HypervisorAdminTestJSON-528937790 tempest-HypervisorAdminTestJSON-528937790-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:46.996440 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a215c5a6-9c34-471b-91d9-fbf1bed0875c tempest-HypervisorAdminTestJSON-528937790 tempest-HypervisorAdminTestJSON-528937790-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:47.001218 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=100 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:09:47.003895 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ffa6b6ef-d1fd-4587-9b63-a6d2bfa0c508 tempest-LiveMigrationTest-1131758637 tempest-LiveMigrationTest-1131758637-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:47.004511 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ffa6b6ef-d1fd-4587-9b63-a6d2bfa0c508 tempest-LiveMigrationTest-1131758637 tempest-LiveMigrationTest-1131758637-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:47.005602 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ffa6b6ef-d1fd-4587-9b63-a6d2bfa0c508 tempest-LiveMigrationTest-1131758637 tempest-LiveMigrationTest-1131758637-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:47.006486 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ffa6b6ef-d1fd-4587-9b63-a6d2bfa0c508 tempest-LiveMigrationTest-1131758637 tempest-LiveMigrationTest-1131758637-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:47.007112 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-a215c5a6-9c34-471b-91d9-fbf1bed0875c tempest-HypervisorAdminTestJSON-528937790 tempest-HypervisorAdminTestJSON-528937790-project-admin] 10.222.0.16 "GET /compute/v2.1/os-hypervisors/2" status: 200 len: 757 microversion: 2.1 time: 0.037465 Jul 16 21:09:47.014789 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 228/473] 10.222.0.16 () {58 vars in 1151 bytes} [Sat Jul 16 21:09:46 2022] GET /compute/v2.1/os-hypervisors/2 => generated 757 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 16 21:09:47.022321 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-f741cc78-9415-4369-90fc-b722dcb40812 tempest-HypervisorAdminTestJSON-528937790 tempest-HypervisorAdminTestJSON-528937790-project-admin] Calling method '.version_select at 0x7fdd9d8c53a0>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:47.023147 ncdv-master-1 devstack@n-api.service[138681]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-hypervisors:uptime": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:09:47.023147 ncdv-master-1 devstack@n-api.service[138681]: warnings.warn(msg) Jul 16 21:09:47.024375 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f741cc78-9415-4369-90fc-b722dcb40812 tempest-HypervisorAdminTestJSON-528937790 tempest-HypervisorAdminTestJSON-528937790-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:47.025237 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f741cc78-9415-4369-90fc-b722dcb40812 tempest-HypervisorAdminTestJSON-528937790 tempest-HypervisorAdminTestJSON-528937790-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:47.039321 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f741cc78-9415-4369-90fc-b722dcb40812 tempest-HypervisorAdminTestJSON-528937790 tempest-HypervisorAdminTestJSON-528937790-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:47.039921 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f741cc78-9415-4369-90fc-b722dcb40812 tempest-HypervisorAdminTestJSON-528937790 tempest-HypervisorAdminTestJSON-528937790-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:47.045474 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.objects.instance [None req-ffa6b6ef-d1fd-4587-9b63-a6d2bfa0c508 tempest-LiveMigrationTest-1131758637 tempest-LiveMigrationTest-1131758637-project-admin] Lazy-loading 'flavor' on Instance uuid c40ba20d-eb88-4ee8-be8f-ecbbd3315c48 {{(pid=138680) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:09:47.049207 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_api:SELECT=87,nova_api:INSERT=3,nova_api:DELETE=6,nova_api:SAVEPOINT=2,nova_api:RELEASE=2 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:09:47.053114 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f741cc78-9415-4369-90fc-b722dcb40812 tempest-HypervisorAdminTestJSON-528937790 tempest-HypervisorAdminTestJSON-528937790-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:47.053662 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f741cc78-9415-4369-90fc-b722dcb40812 tempest-HypervisorAdminTestJSON-528937790 tempest-HypervisorAdminTestJSON-528937790-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:47.061064 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-ffa6b6ef-d1fd-4587-9b63-a6d2bfa0c508 tempest-LiveMigrationTest-1131758637 tempest-LiveMigrationTest-1131758637-project-admin] [instance: c40ba20d-eb88-4ee8-be8f-ecbbd3315c48] Going to try to live migrate instance to nch2-master-1 {{(pid=138680) live_migrate /opt/stack/nova/nova/compute/api.py:5349}} Jul 16 21:09:47.082911 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-f741cc78-9415-4369-90fc-b722dcb40812 tempest-HypervisorAdminTestJSON-528937790 tempest-HypervisorAdminTestJSON-528937790-project-admin] 10.222.0.16 "GET /compute/v2.1/os-hypervisors/1/uptime" status: 200 len: 175 microversion: 2.1 time: 0.066478 Jul 16 21:09:47.083603 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 229/474] 10.222.0.16 () {58 vars in 1172 bytes} [Sat Jul 16 21:09:47 2022] GET /compute/v2.1/os-hypervisors/1/uptime => generated 175 bytes in 68 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:09:47.318364 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-9a957ebe-7594-45f6-a5ee-4a106d2f7750 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:47.319437 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-9a957ebe-7594-45f6-a5ee-4a106d2f7750 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] [instance: 944f68af-bf99-428f-9357-9ce9f807483b] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:09:47.328282 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9a957ebe-7594-45f6-a5ee-4a106d2f7750 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:47.330178 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9a957ebe-7594-45f6-a5ee-4a106d2f7750 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:47.331077 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9a957ebe-7594-45f6-a5ee-4a106d2f7750 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:47.331588 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9a957ebe-7594-45f6-a5ee-4a106d2f7750 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:47.634718 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-9a957ebe-7594-45f6-a5ee-4a106d2f7750 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'abeb7020019d41a18de42956d706a011', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '649babe54d104436af461f1ee4767001', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:47.645404 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-9a957ebe-7594-45f6-a5ee-4a106d2f7750 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] 10.222.0.16 "GET /compute/v2.1/servers/944f68af-bf99-428f-9357-9ce9f807483b" status: 200 len: 1319 microversion: 2.1 time: 0.331549 Jul 16 21:09:47.646011 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 230/475] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:09:47 2022] GET /compute/v2.1/servers/944f68af-bf99-428f-9357-9ce9f807483b => generated 1319 bytes in 333 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:09:47.663249 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-0b45b07d-fbb3-49b2-8634-80c5931c4047 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:47.663249 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-0b45b07d-fbb3-49b2-8634-80c5931c4047 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] [instance: cf53f7cd-deae-4b34-bf8e-3afef98b2ec6] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:09:47.668101 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0b45b07d-fbb3-49b2-8634-80c5931c4047 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:47.668793 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0b45b07d-fbb3-49b2-8634-80c5931c4047 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:47.669601 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0b45b07d-fbb3-49b2-8634-80c5931c4047 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:47.670045 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0b45b07d-fbb3-49b2-8634-80c5931c4047 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:47.743208 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-0b45b07d-fbb3-49b2-8634-80c5931c4047 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ba60685a2001459cbfb531d95a6b296e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb6deae6b08e4ed8b15dc6edfe1d38d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:47.752068 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-0b45b07d-fbb3-49b2-8634-80c5931c4047 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ba60685a2001459cbfb531d95a6b296e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb6deae6b08e4ed8b15dc6edfe1d38d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:47.753661 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-0b45b07d-fbb3-49b2-8634-80c5931c4047 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ba60685a2001459cbfb531d95a6b296e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb6deae6b08e4ed8b15dc6edfe1d38d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:47.759205 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-0b45b07d-fbb3-49b2-8634-80c5931c4047 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] 10.222.0.16 "GET /compute/v2.1/servers/cf53f7cd-deae-4b34-bf8e-3afef98b2ec6" status: 200 len: 1447 microversion: 2.75 time: 0.111518 Jul 16 21:09:47.759778 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 231/476] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:09:47 2022] GET /compute/v2.1/servers/cf53f7cd-deae-4b34-bf8e-3afef98b2ec6 => generated 1447 bytes in 113 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:09:47.764482 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-af0a6f00-c430-4850-b5ec-34d475509b04 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:47.764974 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-af0a6f00-c430-4850-b5ec-34d475509b04 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] [instance: 67266541-b43e-414c-a744-9c32c6f53975] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:09:47.772234 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-af0a6f00-c430-4850-b5ec-34d475509b04 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:47.772698 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-af0a6f00-c430-4850-b5ec-34d475509b04 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:47.773571 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-af0a6f00-c430-4850-b5ec-34d475509b04 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:47.774206 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-af0a6f00-c430-4850-b5ec-34d475509b04 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:47.927862 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-af0a6f00-c430-4850-b5ec-34d475509b04 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0d72626f465b4f37ad3b18bb945d4000', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fc1c3463f9444119af9d18f85bdd703d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:47.936875 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-af0a6f00-c430-4850-b5ec-34d475509b04 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] 10.222.0.16 "GET /compute/v2.1/servers/67266541-b43e-414c-a744-9c32c6f53975" status: 200 len: 1319 microversion: 2.1 time: 0.176055 Jul 16 21:09:47.937458 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 232/477] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:09:47 2022] GET /compute/v2.1/servers/67266541-b43e-414c-a744-9c32c6f53975 => generated 1319 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:09:47.944002 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-775d5368-3287-4470-a673-215102cbcf96 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:47.944677 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-775d5368-3287-4470-a673-215102cbcf96 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: 1703f309-9e2e-4c5f-8588-eb4e43f21d91] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:09:47.959506 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-775d5368-3287-4470-a673-215102cbcf96 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:47.962393 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-775d5368-3287-4470-a673-215102cbcf96 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:47.962393 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-775d5368-3287-4470-a673-215102cbcf96 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:47.962393 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-775d5368-3287-4470-a673-215102cbcf96 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:48.137545 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-775d5368-3287-4470-a673-215102cbcf96 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b68dacc4b8463ba07fae6372131117', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55a19b7a4682457a9132f5fc80110712', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:48.145680 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-775d5368-3287-4470-a673-215102cbcf96 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "GET /compute/v2.1/servers/1703f309-9e2e-4c5f-8588-eb4e43f21d91" status: 200 len: 1469 microversion: 2.1 time: 0.207201 Jul 16 21:09:48.146268 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 233/478] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:09:47 2022] GET /compute/v2.1/servers/1703f309-9e2e-4c5f-8588-eb4e43f21d91 => generated 1469 bytes in 208 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:09:48.663568 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-a12ab718-6b45-4431-b655-6b17eb6c1de7 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:48.664308 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-a12ab718-6b45-4431-b655-6b17eb6c1de7 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] [instance: 944f68af-bf99-428f-9357-9ce9f807483b] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:09:48.676987 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a12ab718-6b45-4431-b655-6b17eb6c1de7 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:48.677508 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a12ab718-6b45-4431-b655-6b17eb6c1de7 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:48.678307 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a12ab718-6b45-4431-b655-6b17eb6c1de7 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:48.678746 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a12ab718-6b45-4431-b655-6b17eb6c1de7 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:49.078492 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-a12ab718-6b45-4431-b655-6b17eb6c1de7 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'abeb7020019d41a18de42956d706a011', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '649babe54d104436af461f1ee4767001', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:49.088670 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-a12ab718-6b45-4431-b655-6b17eb6c1de7 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] 10.222.0.16 "GET /compute/v2.1/servers/944f68af-bf99-428f-9357-9ce9f807483b" status: 200 len: 1319 microversion: 2.1 time: 0.430170 Jul 16 21:09:49.089515 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 234/479] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:09:48 2022] GET /compute/v2.1/servers/944f68af-bf99-428f-9357-9ce9f807483b => generated 1319 bytes in 432 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:09:49.095621 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-164262e0-ec67-42ab-9f9a-6bccbaaf7fd7 req-07eb3eb1-f4a1-4718-9b82-810dceee044c service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "45164bce-7acf-4ef6-8afa-aaab6287ecf4", "name": "network-vif-plugged", "status": "completed", "tag": "14e29168-1029-4ef2-8854-0a02dfa53951"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:09:49.114874 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-164262e0-ec67-42ab-9f9a-6bccbaaf7fd7 req-07eb3eb1-f4a1-4718-9b82-810dceee044c service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:49.115368 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-164262e0-ec67-42ab-9f9a-6bccbaaf7fd7 req-07eb3eb1-f4a1-4718-9b82-810dceee044c service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:49.131815 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-164262e0-ec67-42ab-9f9a-6bccbaaf7fd7 req-07eb3eb1-f4a1-4718-9b82-810dceee044c service nova] Creating event network-vif-plugged:14e29168-1029-4ef2-8854-0a02dfa53951 for instance 45164bce-7acf-4ef6-8afa-aaab6287ecf4 on nch2-master-1 Jul 16 21:09:49.140780 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-164262e0-ec67-42ab-9f9a-6bccbaaf7fd7 req-07eb3eb1-f4a1-4718-9b82-810dceee044c service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.049091 Jul 16 21:09:49.142207 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 235/480] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:09:49 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:09:49.152103 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-0ba22bb0-b3df-46d4-a6e9-3e3a7921b33a tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:49.152103 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-0ba22bb0-b3df-46d4-a6e9-3e3a7921b33a tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] [instance: cf53f7cd-deae-4b34-bf8e-3afef98b2ec6] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:09:49.158242 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0ba22bb0-b3df-46d4-a6e9-3e3a7921b33a tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:49.158964 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0ba22bb0-b3df-46d4-a6e9-3e3a7921b33a tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:49.159986 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0ba22bb0-b3df-46d4-a6e9-3e3a7921b33a tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:49.160591 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0ba22bb0-b3df-46d4-a6e9-3e3a7921b33a tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:49.241382 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-0ba22bb0-b3df-46d4-a6e9-3e3a7921b33a tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ba60685a2001459cbfb531d95a6b296e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb6deae6b08e4ed8b15dc6edfe1d38d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:49.249500 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-0ba22bb0-b3df-46d4-a6e9-3e3a7921b33a tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ba60685a2001459cbfb531d95a6b296e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb6deae6b08e4ed8b15dc6edfe1d38d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:49.251612 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-0ba22bb0-b3df-46d4-a6e9-3e3a7921b33a tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ba60685a2001459cbfb531d95a6b296e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb6deae6b08e4ed8b15dc6edfe1d38d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:49.258305 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-0ba22bb0-b3df-46d4-a6e9-3e3a7921b33a tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] 10.222.0.16 "GET /compute/v2.1/servers/cf53f7cd-deae-4b34-bf8e-3afef98b2ec6" status: 200 len: 1447 microversion: 2.75 time: 0.114570 Jul 16 21:09:49.260646 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 236/481] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:09:49 2022] GET /compute/v2.1/servers/cf53f7cd-deae-4b34-bf8e-3afef98b2ec6 => generated 1447 bytes in 116 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:09:49.266005 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-28643d63-c5de-481f-b7b3-dd78d6d498f3 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:49.267895 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-28643d63-c5de-481f-b7b3-dd78d6d498f3 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] [instance: 67266541-b43e-414c-a744-9c32c6f53975] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:09:49.274928 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-28643d63-c5de-481f-b7b3-dd78d6d498f3 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:49.275736 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-28643d63-c5de-481f-b7b3-dd78d6d498f3 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:49.281211 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-28643d63-c5de-481f-b7b3-dd78d6d498f3 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:49.281821 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-28643d63-c5de-481f-b7b3-dd78d6d498f3 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:49.536396 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-28643d63-c5de-481f-b7b3-dd78d6d498f3 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0d72626f465b4f37ad3b18bb945d4000', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fc1c3463f9444119af9d18f85bdd703d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:49.544574 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-28643d63-c5de-481f-b7b3-dd78d6d498f3 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] 10.222.0.16 "GET /compute/v2.1/servers/67266541-b43e-414c-a744-9c32c6f53975" status: 200 len: 1319 microversion: 2.1 time: 0.283928 Jul 16 21:09:49.544772 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 237/482] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:09:49 2022] GET /compute/v2.1/servers/67266541-b43e-414c-a744-9c32c6f53975 => generated 1319 bytes in 285 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:09:49.575130 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-9ad29461-13cb-481c-a364-6adeda2ad240 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:49.577474 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-9ad29461-13cb-481c-a364-6adeda2ad240 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: 1703f309-9e2e-4c5f-8588-eb4e43f21d91] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:09:49.590758 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9ad29461-13cb-481c-a364-6adeda2ad240 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:49.591247 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9ad29461-13cb-481c-a364-6adeda2ad240 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:49.592109 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9ad29461-13cb-481c-a364-6adeda2ad240 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:49.592472 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9ad29461-13cb-481c-a364-6adeda2ad240 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:49.769194 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-9ad29461-13cb-481c-a364-6adeda2ad240 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b68dacc4b8463ba07fae6372131117', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55a19b7a4682457a9132f5fc80110712', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:49.778184 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-9ad29461-13cb-481c-a364-6adeda2ad240 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "GET /compute/v2.1/servers/1703f309-9e2e-4c5f-8588-eb4e43f21d91" status: 200 len: 1469 microversion: 2.1 time: 0.212186 Jul 16 21:09:49.778761 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 238/483] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:09:49 2022] GET /compute/v2.1/servers/1703f309-9e2e-4c5f-8588-eb4e43f21d91 => generated 1469 bytes in 234 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:09:50.107951 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-5a974f7f-65d8-4c6d-b683-fd00ed38bd9c tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:50.108822 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-5a974f7f-65d8-4c6d-b683-fd00ed38bd9c tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] [instance: 944f68af-bf99-428f-9357-9ce9f807483b] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:09:50.119163 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5a974f7f-65d8-4c6d-b683-fd00ed38bd9c tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:50.119617 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5a974f7f-65d8-4c6d-b683-fd00ed38bd9c tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:50.120402 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5a974f7f-65d8-4c6d-b683-fd00ed38bd9c tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:50.120877 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5a974f7f-65d8-4c6d-b683-fd00ed38bd9c tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:50.295206 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-5a974f7f-65d8-4c6d-b683-fd00ed38bd9c tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'abeb7020019d41a18de42956d706a011', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '649babe54d104436af461f1ee4767001', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:50.303545 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-5a974f7f-65d8-4c6d-b683-fd00ed38bd9c tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] 10.222.0.16 "GET /compute/v2.1/servers/944f68af-bf99-428f-9357-9ce9f807483b" status: 200 len: 1319 microversion: 2.1 time: 0.199637 Jul 16 21:09:50.304176 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 239/484] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:09:50 2022] GET /compute/v2.1/servers/944f68af-bf99-428f-9357-9ce9f807483b => generated 1319 bytes in 201 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:09:50.308933 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-a5952e73-2f74-40c9-abda-e890643012c7 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:50.309736 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-a5952e73-2f74-40c9-abda-e890643012c7 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] [instance: cf53f7cd-deae-4b34-bf8e-3afef98b2ec6] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:09:50.322143 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a5952e73-2f74-40c9-abda-e890643012c7 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:50.322594 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a5952e73-2f74-40c9-abda-e890643012c7 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:50.323412 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a5952e73-2f74-40c9-abda-e890643012c7 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:50.323849 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a5952e73-2f74-40c9-abda-e890643012c7 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:50.401610 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-a5952e73-2f74-40c9-abda-e890643012c7 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ba60685a2001459cbfb531d95a6b296e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb6deae6b08e4ed8b15dc6edfe1d38d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:50.412119 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-a5952e73-2f74-40c9-abda-e890643012c7 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ba60685a2001459cbfb531d95a6b296e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb6deae6b08e4ed8b15dc6edfe1d38d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:50.414020 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-a5952e73-2f74-40c9-abda-e890643012c7 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ba60685a2001459cbfb531d95a6b296e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb6deae6b08e4ed8b15dc6edfe1d38d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:50.419355 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-a5952e73-2f74-40c9-abda-e890643012c7 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] 10.222.0.16 "GET /compute/v2.1/servers/cf53f7cd-deae-4b34-bf8e-3afef98b2ec6" status: 200 len: 1447 microversion: 2.75 time: 0.113947 Jul 16 21:09:50.420150 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 240/485] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:09:50 2022] GET /compute/v2.1/servers/cf53f7cd-deae-4b34-bf8e-3afef98b2ec6 => generated 1447 bytes in 115 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:09:50.562873 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-ca9409ea-2967-411f-92df-581dcc5cb2f6 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:50.563852 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-ca9409ea-2967-411f-92df-581dcc5cb2f6 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] [instance: 67266541-b43e-414c-a744-9c32c6f53975] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:09:50.576509 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ca9409ea-2967-411f-92df-581dcc5cb2f6 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:50.576912 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ca9409ea-2967-411f-92df-581dcc5cb2f6 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:50.581577 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ca9409ea-2967-411f-92df-581dcc5cb2f6 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:50.581577 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ca9409ea-2967-411f-92df-581dcc5cb2f6 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:50.738986 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-ca9409ea-2967-411f-92df-581dcc5cb2f6 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0d72626f465b4f37ad3b18bb945d4000', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fc1c3463f9444119af9d18f85bdd703d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:50.749632 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-ca9409ea-2967-411f-92df-581dcc5cb2f6 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] 10.222.0.16 "GET /compute/v2.1/servers/67266541-b43e-414c-a744-9c32c6f53975" status: 200 len: 1319 microversion: 2.1 time: 0.191616 Jul 16 21:09:50.751745 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 241/486] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:09:50 2022] GET /compute/v2.1/servers/67266541-b43e-414c-a744-9c32c6f53975 => generated 1319 bytes in 195 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:09:50.803795 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-8b442b53-3de4-45a8-8cf6-d78830e1581d tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:50.804420 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-8b442b53-3de4-45a8-8cf6-d78830e1581d tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: 1703f309-9e2e-4c5f-8588-eb4e43f21d91] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:09:50.810625 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8b442b53-3de4-45a8-8cf6-d78830e1581d tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:50.811372 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8b442b53-3de4-45a8-8cf6-d78830e1581d tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:50.812318 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8b442b53-3de4-45a8-8cf6-d78830e1581d tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:50.819628 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8b442b53-3de4-45a8-8cf6-d78830e1581d tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:50.975756 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-8b442b53-3de4-45a8-8cf6-d78830e1581d tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b68dacc4b8463ba07fae6372131117', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55a19b7a4682457a9132f5fc80110712', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:50.984090 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-8b442b53-3de4-45a8-8cf6-d78830e1581d tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "GET /compute/v2.1/servers/1703f309-9e2e-4c5f-8588-eb4e43f21d91" status: 200 len: 1469 microversion: 2.1 time: 0.185228 Jul 16 21:09:50.985682 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 242/487] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:09:50 2022] GET /compute/v2.1/servers/1703f309-9e2e-4c5f-8588-eb4e43f21d91 => generated 1469 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:09:51.161629 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-fb5b1e79-b62f-430d-987a-cd3e1bed08d9 req-c9b26454-9a8a-4269-bfba-b427b832051d service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "944f68af-bf99-428f-9357-9ce9f807483b", "name": "network-vif-plugged", "status": "completed", "tag": "5556f1e0-fe84-46bd-9585-73295be09188"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:09:51.170495 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-fb5b1e79-b62f-430d-987a-cd3e1bed08d9 req-c9b26454-9a8a-4269-bfba-b427b832051d service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:51.170951 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-fb5b1e79-b62f-430d-987a-cd3e1bed08d9 req-c9b26454-9a8a-4269-bfba-b427b832051d service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:51.189092 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-fb5b1e79-b62f-430d-987a-cd3e1bed08d9 req-c9b26454-9a8a-4269-bfba-b427b832051d service nova] Creating event network-vif-plugged:5556f1e0-fe84-46bd-9585-73295be09188 for instance 944f68af-bf99-428f-9357-9ce9f807483b on nch2-master-1 Jul 16 21:09:51.194174 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-fb5b1e79-b62f-430d-987a-cd3e1bed08d9 req-c9b26454-9a8a-4269-bfba-b427b832051d service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.037062 Jul 16 21:09:51.195047 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 243/488] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:09:51 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 16 21:09:51.322271 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-8cce1da7-c451-405d-8d0a-80492d7ca736 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:51.322902 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-8cce1da7-c451-405d-8d0a-80492d7ca736 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] [instance: 944f68af-bf99-428f-9357-9ce9f807483b] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:09:51.330457 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8cce1da7-c451-405d-8d0a-80492d7ca736 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:51.330905 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8cce1da7-c451-405d-8d0a-80492d7ca736 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:51.331716 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8cce1da7-c451-405d-8d0a-80492d7ca736 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:51.332159 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8cce1da7-c451-405d-8d0a-80492d7ca736 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:51.560499 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-8cce1da7-c451-405d-8d0a-80492d7ca736 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'abeb7020019d41a18de42956d706a011', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '649babe54d104436af461f1ee4767001', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:51.571345 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-8cce1da7-c451-405d-8d0a-80492d7ca736 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] 10.222.0.16 "GET /compute/v2.1/servers/944f68af-bf99-428f-9357-9ce9f807483b" status: 200 len: 1319 microversion: 2.1 time: 0.253567 Jul 16 21:09:51.573642 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 244/489] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:09:51 2022] GET /compute/v2.1/servers/944f68af-bf99-428f-9357-9ce9f807483b => generated 1319 bytes in 257 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:09:51.582423 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-4d50f61f-ae98-4b8d-9345-0e38b6f7c620 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:51.583640 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-4d50f61f-ae98-4b8d-9345-0e38b6f7c620 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] [instance: cf53f7cd-deae-4b34-bf8e-3afef98b2ec6] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:09:51.597311 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4d50f61f-ae98-4b8d-9345-0e38b6f7c620 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:51.598680 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4d50f61f-ae98-4b8d-9345-0e38b6f7c620 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:51.599827 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4d50f61f-ae98-4b8d-9345-0e38b6f7c620 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:51.600453 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4d50f61f-ae98-4b8d-9345-0e38b6f7c620 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:51.684658 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-4d50f61f-ae98-4b8d-9345-0e38b6f7c620 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ba60685a2001459cbfb531d95a6b296e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb6deae6b08e4ed8b15dc6edfe1d38d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:51.694121 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-4d50f61f-ae98-4b8d-9345-0e38b6f7c620 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ba60685a2001459cbfb531d95a6b296e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb6deae6b08e4ed8b15dc6edfe1d38d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:51.695886 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-4d50f61f-ae98-4b8d-9345-0e38b6f7c620 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ba60685a2001459cbfb531d95a6b296e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb6deae6b08e4ed8b15dc6edfe1d38d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:51.701638 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-4d50f61f-ae98-4b8d-9345-0e38b6f7c620 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] 10.222.0.16 "GET /compute/v2.1/servers/cf53f7cd-deae-4b34-bf8e-3afef98b2ec6" status: 200 len: 1447 microversion: 2.75 time: 0.126636 Jul 16 21:09:51.702623 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 245/490] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:09:51 2022] GET /compute/v2.1/servers/cf53f7cd-deae-4b34-bf8e-3afef98b2ec6 => generated 1447 bytes in 128 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:09:51.768330 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-9b645f39-fcd6-40f1-8b2b-9af9326ed000 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:51.768957 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-9b645f39-fcd6-40f1-8b2b-9af9326ed000 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] [instance: 67266541-b43e-414c-a744-9c32c6f53975] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:09:51.780639 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9b645f39-fcd6-40f1-8b2b-9af9326ed000 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:51.781353 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9b645f39-fcd6-40f1-8b2b-9af9326ed000 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:51.782754 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9b645f39-fcd6-40f1-8b2b-9af9326ed000 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:51.783255 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9b645f39-fcd6-40f1-8b2b-9af9326ed000 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:51.961964 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-9b645f39-fcd6-40f1-8b2b-9af9326ed000 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0d72626f465b4f37ad3b18bb945d4000', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fc1c3463f9444119af9d18f85bdd703d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:51.976280 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=100 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:09:51.980811 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-9b645f39-fcd6-40f1-8b2b-9af9326ed000 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] 10.222.0.16 "GET /compute/v2.1/servers/67266541-b43e-414c-a744-9c32c6f53975" status: 200 len: 1319 microversion: 2.1 time: 0.216994 Jul 16 21:09:51.981403 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 246/491] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:09:51 2022] GET /compute/v2.1/servers/67266541-b43e-414c-a744-9c32c6f53975 => generated 1319 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:09:52.110915 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-59da06a3-e5bc-4dc0-82b5-dbb03bf3e236 tempest-HostsAdminTestJSON-1942136461 tempest-HostsAdminTestJSON-1942136461-project-admin] Calling method '.version_select at 0x7fdd9d8b1940>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:52.112443 ncdv-master-1 devstack@n-api.service[138681]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-hosts:list": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:09:52.112443 ncdv-master-1 devstack@n-api.service[138681]: warnings.warn(msg) Jul 16 21:09:52.113584 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-59da06a3-e5bc-4dc0-82b5-dbb03bf3e236 tempest-HostsAdminTestJSON-1942136461 tempest-HostsAdminTestJSON-1942136461-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:52.114028 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-59da06a3-e5bc-4dc0-82b5-dbb03bf3e236 tempest-HostsAdminTestJSON-1942136461 tempest-HostsAdminTestJSON-1942136461-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:52.114826 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-59da06a3-e5bc-4dc0-82b5-dbb03bf3e236 tempest-HostsAdminTestJSON-1942136461 tempest-HostsAdminTestJSON-1942136461-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:52.115279 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-59da06a3-e5bc-4dc0-82b5-dbb03bf3e236 tempest-HostsAdminTestJSON-1942136461 tempest-HostsAdminTestJSON-1942136461-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:52.140029 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-59da06a3-e5bc-4dc0-82b5-dbb03bf3e236 tempest-HostsAdminTestJSON-1942136461 tempest-HostsAdminTestJSON-1942136461-project-admin] 10.222.0.16 "GET /compute/v2.1/os-hosts" status: 200 len: 379 microversion: 2.1 time: 0.157434 Jul 16 21:09:52.140600 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 247/492] 10.222.0.16 () {58 vars in 1127 bytes} [Sat Jul 16 21:09:51 2022] GET /compute/v2.1/os-hosts => generated 379 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:09:52.145617 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-3a5f4576-99dd-4882-b6eb-55ae49946dd4 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:52.146410 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-3a5f4576-99dd-4882-b6eb-55ae49946dd4 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: 1703f309-9e2e-4c5f-8588-eb4e43f21d91] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:09:52.152472 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3a5f4576-99dd-4882-b6eb-55ae49946dd4 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:52.152922 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3a5f4576-99dd-4882-b6eb-55ae49946dd4 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:52.153689 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3a5f4576-99dd-4882-b6eb-55ae49946dd4 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:52.154127 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3a5f4576-99dd-4882-b6eb-55ae49946dd4 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:52.259158 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-ffa6b6ef-d1fd-4587-9b63-a6d2bfa0c508 tempest-LiveMigrationTest-1131758637 tempest-LiveMigrationTest-1131758637-project-admin] 10.222.0.16 "POST /compute/v2.1/servers/c40ba20d-eb88-4ee8-be8f-ecbbd3315c48/action" status: 202 len: 0 microversion: 2.1 time: 5.270622 Jul 16 21:09:52.259916 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 246/493] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 21:09:46 2022] POST /compute/v2.1/servers/c40ba20d-eb88-4ee8-be8f-ecbbd3315c48/action => generated 0 bytes in 5272 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 21:09:52.264581 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-253ec999-ae92-49ef-b063-8b3b375b8a3f tempest-HostsAdminTestJSON-1942136461 tempest-HostsAdminTestJSON-1942136461-project-admin] Calling method '.version_select at 0x7fdd9d3ade50>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:52.265835 ncdv-master-1 devstack@n-api.service[138680]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-hosts:list": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:09:52.265835 ncdv-master-1 devstack@n-api.service[138680]: warnings.warn(msg) Jul 16 21:09:52.267634 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-253ec999-ae92-49ef-b063-8b3b375b8a3f tempest-HostsAdminTestJSON-1942136461 tempest-HostsAdminTestJSON-1942136461-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:52.268092 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-253ec999-ae92-49ef-b063-8b3b375b8a3f tempest-HostsAdminTestJSON-1942136461 tempest-HostsAdminTestJSON-1942136461-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:52.268895 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-253ec999-ae92-49ef-b063-8b3b375b8a3f tempest-HostsAdminTestJSON-1942136461 tempest-HostsAdminTestJSON-1942136461-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:52.269586 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-253ec999-ae92-49ef-b063-8b3b375b8a3f tempest-HostsAdminTestJSON-1942136461 tempest-HostsAdminTestJSON-1942136461-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:52.298839 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-253ec999-ae92-49ef-b063-8b3b375b8a3f tempest-HostsAdminTestJSON-1942136461 tempest-HostsAdminTestJSON-1942136461-project-admin] 10.222.0.16 "GET /compute/v2.1/os-hosts?zone=" status: 200 len: 379 microversion: 2.1 time: 0.037715 Jul 16 21:09:52.299486 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 247/494] 10.222.0.16 () {58 vars in 1138 bytes} [Sat Jul 16 21:09:52 2022] GET /compute/v2.1/os-hosts?zone= => generated 379 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:09:52.299898 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-3a5f4576-99dd-4882-b6eb-55ae49946dd4 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b68dacc4b8463ba07fae6372131117', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55a19b7a4682457a9132f5fc80110712', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:52.304002 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-49a15c81-29da-45c4-ac09-22b32c3651a8 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:52.304956 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-49a15c81-29da-45c4-ac09-22b32c3651a8 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] [instance: c40ba20d-eb88-4ee8-be8f-ecbbd3315c48] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:09:52.309471 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-3a5f4576-99dd-4882-b6eb-55ae49946dd4 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "GET /compute/v2.1/servers/1703f309-9e2e-4c5f-8588-eb4e43f21d91" status: 200 len: 1461 microversion: 2.1 time: 0.167770 Jul 16 21:09:52.310512 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 248/495] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:09:52 2022] GET /compute/v2.1/servers/1703f309-9e2e-4c5f-8588-eb4e43f21d91 => generated 1461 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:09:52.315683 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-c47042d1-b245-46ef-8b08-b178fd124f96 tempest-HostsAdminTestJSON-1942136461 tempest-HostsAdminTestJSON-1942136461-project-admin] Calling method '.version_select at 0x7fdd9d8c58b0>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:52.317152 ncdv-master-1 devstack@n-api.service[138681]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-hosts:list": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:09:52.317152 ncdv-master-1 devstack@n-api.service[138681]: warnings.warn(msg) Jul 16 21:09:52.319537 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-49a15c81-29da-45c4-ac09-22b32c3651a8 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:52.320163 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-49a15c81-29da-45c4-ac09-22b32c3651a8 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:52.321225 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-49a15c81-29da-45c4-ac09-22b32c3651a8 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:52.321832 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-49a15c81-29da-45c4-ac09-22b32c3651a8 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:52.328015 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c47042d1-b245-46ef-8b08-b178fd124f96 tempest-HostsAdminTestJSON-1942136461 tempest-HostsAdminTestJSON-1942136461-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:52.329239 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c47042d1-b245-46ef-8b08-b178fd124f96 tempest-HostsAdminTestJSON-1942136461 tempest-HostsAdminTestJSON-1942136461-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.010s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:52.330158 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c47042d1-b245-46ef-8b08-b178fd124f96 tempest-HostsAdminTestJSON-1942136461 tempest-HostsAdminTestJSON-1942136461-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:52.330577 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c47042d1-b245-46ef-8b08-b178fd124f96 tempest-HostsAdminTestJSON-1942136461 tempest-HostsAdminTestJSON-1942136461-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:52.359398 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-c47042d1-b245-46ef-8b08-b178fd124f96 tempest-HostsAdminTestJSON-1942136461 tempest-HostsAdminTestJSON-1942136461-project-admin] 10.222.0.16 "GET /compute/v2.1/os-hosts?zone=xxx" status: 200 len: 13 microversion: 2.1 time: 0.047564 Jul 16 21:09:52.360699 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 249/496] 10.222.0.16 () {58 vars in 1144 bytes} [Sat Jul 16 21:09:52 2022] GET /compute/v2.1/os-hosts?zone=xxx => generated 13 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Jul 16 21:09:52.368505 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-f7451e93-d777-4f1b-bd35-fca6fd307d4d tempest-MigrationsAdminTest-399208122 tempest-MigrationsAdminTest-399208122-project-admin] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:52.369295 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-f7451e93-d777-4f1b-bd35-fca6fd307d4d tempest-MigrationsAdminTest-399208122 tempest-MigrationsAdminTest-399208122-project-admin] [instance: 1703f309-9e2e-4c5f-8588-eb4e43f21d91] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:09:52.376935 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f7451e93-d777-4f1b-bd35-fca6fd307d4d tempest-MigrationsAdminTest-399208122 tempest-MigrationsAdminTest-399208122-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:52.377580 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f7451e93-d777-4f1b-bd35-fca6fd307d4d tempest-MigrationsAdminTest-399208122 tempest-MigrationsAdminTest-399208122-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:52.378668 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f7451e93-d777-4f1b-bd35-fca6fd307d4d tempest-MigrationsAdminTest-399208122 tempest-MigrationsAdminTest-399208122-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:52.379292 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f7451e93-d777-4f1b-bd35-fca6fd307d4d tempest-MigrationsAdminTest-399208122 tempest-MigrationsAdminTest-399208122-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:52.545913 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-49a15c81-29da-45c4-ac09-22b32c3651a8 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a69f9dee03d43beaafa966216b9e289', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9b769bf8f3ea4e368bf68de1df6538f9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:52.555090 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-49a15c81-29da-45c4-ac09-22b32c3651a8 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] 10.222.0.16 "GET /compute/v2.1/servers/c40ba20d-eb88-4ee8-be8f-ecbbd3315c48" status: 200 len: 1503 microversion: 2.1 time: 0.254499 Jul 16 21:09:52.555674 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 248/497] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:09:52 2022] GET /compute/v2.1/servers/c40ba20d-eb88-4ee8-be8f-ecbbd3315c48 => generated 1503 bytes in 256 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:09:52.561969 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-f7451e93-d777-4f1b-bd35-fca6fd307d4d tempest-MigrationsAdminTest-399208122 tempest-MigrationsAdminTest-399208122-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/1703f309-9e2e-4c5f-8588-eb4e43f21d91" status: 200 len: 1612 microversion: 2.1 time: 0.199484 Jul 16 21:09:52.562709 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 250/498] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:09:52 2022] GET /compute/v2.1/servers/1703f309-9e2e-4c5f-8588-eb4e43f21d91 => generated 1612 bytes in 201 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:09:52.567148 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-2be96411-8559-429c-ac87-873d8c73908c tempest-HostsAdminTestJSON-1942136461 tempest-HostsAdminTestJSON-1942136461-project-admin] Calling method '.version_select at 0x7fdd9d3283a0>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:52.568374 ncdv-master-1 devstack@n-api.service[138680]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-hosts:list": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:09:52.568374 ncdv-master-1 devstack@n-api.service[138680]: warnings.warn(msg) Jul 16 21:09:52.569612 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2be96411-8559-429c-ac87-873d8c73908c tempest-HostsAdminTestJSON-1942136461 tempest-HostsAdminTestJSON-1942136461-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:52.569959 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2be96411-8559-429c-ac87-873d8c73908c tempest-HostsAdminTestJSON-1942136461 tempest-HostsAdminTestJSON-1942136461-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:52.570842 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2be96411-8559-429c-ac87-873d8c73908c tempest-HostsAdminTestJSON-1942136461 tempest-HostsAdminTestJSON-1942136461-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:52.571249 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2be96411-8559-429c-ac87-873d8c73908c tempest-HostsAdminTestJSON-1942136461 tempest-HostsAdminTestJSON-1942136461-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:52.593740 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-e428a736-d1ec-4eeb-b89b-ce4373ad1ca8 tempest-MigrationsAdminTest-399208122 tempest-MigrationsAdminTest-399208122-project-admin] Calling method '.version_select at 0x7fdd9d96adc0>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:52.605746 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-2be96411-8559-429c-ac87-873d8c73908c tempest-HostsAdminTestJSON-1942136461 tempest-HostsAdminTestJSON-1942136461-project-admin] 10.222.0.16 "GET /compute/v2.1/os-hosts" status: 200 len: 379 microversion: 2.1 time: 0.048721 Jul 16 21:09:52.606496 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 249/499] 10.222.0.16 () {58 vars in 1127 bytes} [Sat Jul 16 21:09:52 2022] GET /compute/v2.1/os-hosts => generated 379 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:09:52.609082 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e428a736-d1ec-4eeb-b89b-ce4373ad1ca8 tempest-MigrationsAdminTest-399208122 tempest-MigrationsAdminTest-399208122-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:52.609545 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e428a736-d1ec-4eeb-b89b-ce4373ad1ca8 tempest-MigrationsAdminTest-399208122 tempest-MigrationsAdminTest-399208122-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:52.616352 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-c9e767f6-cd48-4f53-9f52-9be147c61254 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:52.616881 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-c9e767f6-cd48-4f53-9f52-9be147c61254 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] [instance: 944f68af-bf99-428f-9357-9ce9f807483b] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:09:52.623873 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c9e767f6-cd48-4f53-9f52-9be147c61254 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:52.624578 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c9e767f6-cd48-4f53-9f52-9be147c61254 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:52.625664 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c9e767f6-cd48-4f53-9f52-9be147c61254 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:52.630847 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c9e767f6-cd48-4f53-9f52-9be147c61254 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:52.638452 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-e428a736-d1ec-4eeb-b89b-ce4373ad1ca8 tempest-MigrationsAdminTest-399208122 tempest-MigrationsAdminTest-399208122-project-admin] 10.222.0.16 "GET /compute/v2.1/os-migrations" status: 200 len: 793 microversion: 2.1 time: 0.048275 Jul 16 21:09:52.651130 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 251/500] 10.222.0.16 () {58 vars in 1142 bytes} [Sat Jul 16 21:09:52 2022] GET /compute/v2.1/os-migrations => generated 793 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:09:52.653435 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-39c82761-1d1b-4be5-904a-83be7479345c tempest-HostsAdminTestJSON-1942136461 tempest-HostsAdminTestJSON-1942136461-project-admin] Calling method '.version_select at 0x7fdd9d863af0>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:52.656028 ncdv-master-1 devstack@n-api.service[138681]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-hosts:list": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:09:52.656028 ncdv-master-1 devstack@n-api.service[138681]: warnings.warn(msg) Jul 16 21:09:52.659815 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-39c82761-1d1b-4be5-904a-83be7479345c tempest-HostsAdminTestJSON-1942136461 tempest-HostsAdminTestJSON-1942136461-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:52.660333 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-39c82761-1d1b-4be5-904a-83be7479345c tempest-HostsAdminTestJSON-1942136461 tempest-HostsAdminTestJSON-1942136461-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:52.661433 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-39c82761-1d1b-4be5-904a-83be7479345c tempest-HostsAdminTestJSON-1942136461 tempest-HostsAdminTestJSON-1942136461-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:52.661647 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-39c82761-1d1b-4be5-904a-83be7479345c tempest-HostsAdminTestJSON-1942136461 tempest-HostsAdminTestJSON-1942136461-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:52.693215 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-39c82761-1d1b-4be5-904a-83be7479345c tempest-HostsAdminTestJSON-1942136461 tempest-HostsAdminTestJSON-1942136461-project-admin] 10.222.0.16 "GET /compute/v2.1/os-hosts?zone=internal" status: 200 len: 239 microversion: 2.1 time: 0.047093 Jul 16 21:09:52.693861 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 252/501] 10.222.0.16 () {58 vars in 1154 bytes} [Sat Jul 16 21:09:52 2022] GET /compute/v2.1/os-hosts?zone=internal => generated 239 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:09:52.708886 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-f6565692-0229-4419-a9d2-caad9814f8a8 tempest-HostsAdminTestJSON-1942136461 tempest-HostsAdminTestJSON-1942136461-project-admin] Calling method '.version_select at 0x7fdd9da64700>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:52.710108 ncdv-master-1 devstack@n-api.service[138681]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-hosts:list": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:09:52.710108 ncdv-master-1 devstack@n-api.service[138681]: warnings.warn(msg) Jul 16 21:09:52.711300 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f6565692-0229-4419-a9d2-caad9814f8a8 tempest-HostsAdminTestJSON-1942136461 tempest-HostsAdminTestJSON-1942136461-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:52.711931 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f6565692-0229-4419-a9d2-caad9814f8a8 tempest-HostsAdminTestJSON-1942136461 tempest-HostsAdminTestJSON-1942136461-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:52.712887 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f6565692-0229-4419-a9d2-caad9814f8a8 tempest-HostsAdminTestJSON-1942136461 tempest-HostsAdminTestJSON-1942136461-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:52.713496 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f6565692-0229-4419-a9d2-caad9814f8a8 tempest-HostsAdminTestJSON-1942136461 tempest-HostsAdminTestJSON-1942136461-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:52.739813 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-f6565692-0229-4419-a9d2-caad9814f8a8 tempest-HostsAdminTestJSON-1942136461 tempest-HostsAdminTestJSON-1942136461-project-admin] 10.222.0.16 "GET /compute/v2.1/os-hosts" status: 200 len: 379 microversion: 2.1 time: 0.038518 Jul 16 21:09:52.740673 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 253/502] 10.222.0.16 () {58 vars in 1127 bytes} [Sat Jul 16 21:09:52 2022] GET /compute/v2.1/os-hosts => generated 379 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 16 21:09:52.755279 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-baead4b1-6cf8-4ccf-894b-1e5c48e93550 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:52.755905 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-baead4b1-6cf8-4ccf-894b-1e5c48e93550 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] [instance: cf53f7cd-deae-4b34-bf8e-3afef98b2ec6] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:09:52.769888 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-baead4b1-6cf8-4ccf-894b-1e5c48e93550 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:52.770396 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-baead4b1-6cf8-4ccf-894b-1e5c48e93550 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:52.771255 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-baead4b1-6cf8-4ccf-894b-1e5c48e93550 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:52.771706 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-baead4b1-6cf8-4ccf-894b-1e5c48e93550 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:52.867601 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-c9e767f6-cd48-4f53-9f52-9be147c61254 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'abeb7020019d41a18de42956d706a011', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '649babe54d104436af461f1ee4767001', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:52.881075 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-c9e767f6-cd48-4f53-9f52-9be147c61254 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] 10.222.0.16 "GET /compute/v2.1/servers/944f68af-bf99-428f-9357-9ce9f807483b" status: 200 len: 1319 microversion: 2.1 time: 0.268506 Jul 16 21:09:52.882390 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 250/503] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:09:52 2022] GET /compute/v2.1/servers/944f68af-bf99-428f-9357-9ce9f807483b => generated 1319 bytes in 275 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:09:52.887678 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-011fc3fa-fecb-4fd9-bb39-8c75a83f2966 tempest-HostsAdminTestJSON-1942136461 tempest-HostsAdminTestJSON-1942136461-project-admin] Calling method '.version_select at 0x7fdd9d434b80>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:52.888812 ncdv-master-1 devstack@n-api.service[138680]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-hosts:show": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:09:52.888812 ncdv-master-1 devstack@n-api.service[138680]: warnings.warn(msg) Jul 16 21:09:52.897499 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-baead4b1-6cf8-4ccf-894b-1e5c48e93550 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ba60685a2001459cbfb531d95a6b296e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb6deae6b08e4ed8b15dc6edfe1d38d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:52.898041 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-011fc3fa-fecb-4fd9-bb39-8c75a83f2966 tempest-HostsAdminTestJSON-1942136461 tempest-HostsAdminTestJSON-1942136461-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:52.898712 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-011fc3fa-fecb-4fd9-bb39-8c75a83f2966 tempest-HostsAdminTestJSON-1942136461 tempest-HostsAdminTestJSON-1942136461-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:52.907716 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-baead4b1-6cf8-4ccf-894b-1e5c48e93550 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ba60685a2001459cbfb531d95a6b296e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb6deae6b08e4ed8b15dc6edfe1d38d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:52.909490 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-baead4b1-6cf8-4ccf-894b-1e5c48e93550 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ba60685a2001459cbfb531d95a6b296e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb6deae6b08e4ed8b15dc6edfe1d38d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:52.914049 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-011fc3fa-fecb-4fd9-bb39-8c75a83f2966 tempest-HostsAdminTestJSON-1942136461 tempest-HostsAdminTestJSON-1942136461-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:52.914511 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-011fc3fa-fecb-4fd9-bb39-8c75a83f2966 tempest-HostsAdminTestJSON-1942136461 tempest-HostsAdminTestJSON-1942136461-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:52.917356 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-baead4b1-6cf8-4ccf-894b-1e5c48e93550 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] 10.222.0.16 "GET /compute/v2.1/servers/cf53f7cd-deae-4b34-bf8e-3afef98b2ec6" status: 200 len: 1447 microversion: 2.75 time: 0.175577 Jul 16 21:09:52.919244 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 254/504] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:09:52 2022] GET /compute/v2.1/servers/cf53f7cd-deae-4b34-bf8e-3afef98b2ec6 => generated 1447 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:09:52.924584 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-215ed8c7-4911-47de-87b7-c84e69aefcc2 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-MigrationsAdminTest-server-601854103", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452"}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:09:52.970530 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-011fc3fa-fecb-4fd9-bb39-8c75a83f2966 tempest-HostsAdminTestJSON-1942136461 tempest-HostsAdminTestJSON-1942136461-project-admin] 10.222.0.16 "GET /compute/v2.1/os-hosts/nch1-master-1" status: 200 len: 860 microversion: 2.1 time: 0.084613 Jul 16 21:09:52.977420 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 251/505] 10.222.0.16 () {58 vars in 1169 bytes} [Sat Jul 16 21:09:52 2022] GET /compute/v2.1/os-hosts/nch1-master-1 => generated 860 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:09:52.983718 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-799650a5-5351-49e3-942c-906773736929 tempest-HostsAdminTestJSON-1942136461 tempest-HostsAdminTestJSON-1942136461-project-admin] Calling method '.version_select at 0x7fdd9d4e0e50>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:52.984781 ncdv-master-1 devstack@n-api.service[138680]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-hosts:show": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:09:52.984781 ncdv-master-1 devstack@n-api.service[138680]: warnings.warn(msg) Jul 16 21:09:52.990791 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.network.neutron [None req-215ed8c7-4911-47de-87b7-c84e69aefcc2 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] validate_networks() for None {{(pid=138681) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 16 21:09:52.992795 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-799650a5-5351-49e3-942c-906773736929 tempest-HostsAdminTestJSON-1942136461 tempest-HostsAdminTestJSON-1942136461-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:52.993519 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-799650a5-5351-49e3-942c-906773736929 tempest-HostsAdminTestJSON-1942136461 tempest-HostsAdminTestJSON-1942136461-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:53.012143 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-799650a5-5351-49e3-942c-906773736929 tempest-HostsAdminTestJSON-1942136461 tempest-HostsAdminTestJSON-1942136461-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:53.012902 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-799650a5-5351-49e3-942c-906773736929 tempest-HostsAdminTestJSON-1942136461 tempest-HostsAdminTestJSON-1942136461-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:53.040927 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-799650a5-5351-49e3-942c-906773736929 tempest-HostsAdminTestJSON-1942136461 tempest-HostsAdminTestJSON-1942136461-project-admin] 10.222.0.16 "GET /compute/v2.1/os-hosts/nch2-master-1" status: 200 len: 598 microversion: 2.1 time: 0.061748 Jul 16 21:09:53.047066 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 252/506] 10.222.0.16 () {58 vars in 1169 bytes} [Sat Jul 16 21:09:52 2022] GET /compute/v2.1/os-hosts/nch2-master-1 => generated 598 bytes in 69 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:09:53.053559 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-200276ac-2daa-4b83-ad78-6b2e3485a0e1 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:53.054411 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-200276ac-2daa-4b83-ad78-6b2e3485a0e1 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] [instance: 67266541-b43e-414c-a744-9c32c6f53975] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:09:53.073996 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-200276ac-2daa-4b83-ad78-6b2e3485a0e1 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:53.079229 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-200276ac-2daa-4b83-ad78-6b2e3485a0e1 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:53.080292 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-200276ac-2daa-4b83-ad78-6b2e3485a0e1 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:53.084030 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-200276ac-2daa-4b83-ad78-6b2e3485a0e1 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:53.304911 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-200276ac-2daa-4b83-ad78-6b2e3485a0e1 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0d72626f465b4f37ad3b18bb945d4000', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fc1c3463f9444119af9d18f85bdd703d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:53.312997 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-200276ac-2daa-4b83-ad78-6b2e3485a0e1 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] 10.222.0.16 "GET /compute/v2.1/servers/67266541-b43e-414c-a744-9c32c6f53975" status: 200 len: 1319 microversion: 2.1 time: 0.264221 Jul 16 21:09:53.313677 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 253/507] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:09:53 2022] GET /compute/v2.1/servers/67266541-b43e-414c-a744-9c32c6f53975 => generated 1319 bytes in 266 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:09:53.345600 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-215ed8c7-4911-47de-87b7-c84e69aefcc2 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Flavor limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 16 21:09:53.345600 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-215ed8c7-4911-47de-87b7-c84e69aefcc2 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Image limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 16 21:09:53.345943 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-215ed8c7-4911-47de-87b7-c84e69aefcc2 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Flavor pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 16 21:09:53.346375 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-215ed8c7-4911-47de-87b7-c84e69aefcc2 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Image pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 16 21:09:53.347627 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-215ed8c7-4911-47de-87b7-c84e69aefcc2 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 16 21:09:53.358686 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-215ed8c7-4911-47de-87b7-c84e69aefcc2 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:53.359196 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-215ed8c7-4911-47de-87b7-c84e69aefcc2 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:53.360270 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-215ed8c7-4911-47de-87b7-c84e69aefcc2 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:53.360676 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-215ed8c7-4911-47de-87b7-c84e69aefcc2 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:53.383536 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-215ed8c7-4911-47de-87b7-c84e69aefcc2 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Getting quotas for project 55a19b7a4682457a9132f5fc80110712. Resources: {'ram', 'instances', 'cores'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:09:53.388672 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-215ed8c7-4911-47de-87b7-c84e69aefcc2 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Getting quotas for user a9b68dacc4b8463ba07fae6372131117 and project 55a19b7a4682457a9132f5fc80110712. Resources: {'ram', 'instances', 'cores'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:09:53.401933 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-215ed8c7-4911-47de-87b7-c84e69aefcc2 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Going to run 1 instances... {{(pid=138681) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 16 21:09:53.408492 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-215ed8c7-4911-47de-87b7-c84e69aefcc2 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: f98a9762-a67c-4e95-85b8-727c81bdb03a] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138681) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 16 21:09:53.444382 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-f801024b-7375-4ac8-928b-3dd404f7467a tempest-ServerDiagnosticsNegativeTest-1844467799 tempest-ServerDiagnosticsNegativeTest-1844467799-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerDiagnosticsNegativeTest-server-1760135536", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452"}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:09:53.521450 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.network.neutron [None req-f801024b-7375-4ac8-928b-3dd404f7467a tempest-ServerDiagnosticsNegativeTest-1844467799 tempest-ServerDiagnosticsNegativeTest-1844467799-project] validate_networks() for None {{(pid=138680) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 16 21:09:53.629164 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-215ed8c7-4911-47de-87b7-c84e69aefcc2 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.708678 Jul 16 21:09:53.629821 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 255/508] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 21:09:52 2022] POST /compute/v2.1/servers => generated 380 bytes in 710 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 16 21:09:53.634587 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-0b2983b9-0c9d-4db3-9d60-8c593f6414d0 req-eb67d32c-4c4e-4704-9136-77b4e8a49135 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "67266541-b43e-414c-a744-9c32c6f53975", "name": "network-vif-plugged", "status": "completed", "tag": "f15f3108-545f-4afa-9b93-b7dbd6384d3a"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:09:53.645942 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-0b2983b9-0c9d-4db3-9d60-8c593f6414d0 req-eb67d32c-4c4e-4704-9136-77b4e8a49135 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:53.646421 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-0b2983b9-0c9d-4db3-9d60-8c593f6414d0 req-eb67d32c-4c4e-4704-9136-77b4e8a49135 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:53.666490 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-0b2983b9-0c9d-4db3-9d60-8c593f6414d0 req-eb67d32c-4c4e-4704-9136-77b4e8a49135 service nova] Creating event network-vif-plugged:f15f3108-545f-4afa-9b93-b7dbd6384d3a for instance 67266541-b43e-414c-a744-9c32c6f53975 on nch2-master-1 Jul 16 21:09:53.671775 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-0b2983b9-0c9d-4db3-9d60-8c593f6414d0 req-eb67d32c-4c4e-4704-9136-77b4e8a49135 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.040514 Jul 16 21:09:53.672804 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 256/509] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:09:53 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:09:53.679037 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-3d704b4f-84b1-4002-b27f-ba72871a53b4 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:53.680014 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-3d704b4f-84b1-4002-b27f-ba72871a53b4 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] [instance: c40ba20d-eb88-4ee8-be8f-ecbbd3315c48] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:09:53.689180 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3d704b4f-84b1-4002-b27f-ba72871a53b4 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:53.689842 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3d704b4f-84b1-4002-b27f-ba72871a53b4 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:53.690618 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3d704b4f-84b1-4002-b27f-ba72871a53b4 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:53.691247 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3d704b4f-84b1-4002-b27f-ba72871a53b4 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:53.752623 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-f801024b-7375-4ac8-928b-3dd404f7467a tempest-ServerDiagnosticsNegativeTest-1844467799 tempest-ServerDiagnosticsNegativeTest-1844467799-project] Flavor limits 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 16 21:09:53.753127 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-f801024b-7375-4ac8-928b-3dd404f7467a tempest-ServerDiagnosticsNegativeTest-1844467799 tempest-ServerDiagnosticsNegativeTest-1844467799-project] Image limits 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 16 21:09:53.753615 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-f801024b-7375-4ac8-928b-3dd404f7467a tempest-ServerDiagnosticsNegativeTest-1844467799 tempest-ServerDiagnosticsNegativeTest-1844467799-project] Flavor pref 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 16 21:09:53.754030 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-f801024b-7375-4ac8-928b-3dd404f7467a tempest-ServerDiagnosticsNegativeTest-1844467799 tempest-ServerDiagnosticsNegativeTest-1844467799-project] Image pref 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 16 21:09:53.754462 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-f801024b-7375-4ac8-928b-3dd404f7467a tempest-ServerDiagnosticsNegativeTest-1844467799 tempest-ServerDiagnosticsNegativeTest-1844467799-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 16 21:09:53.768151 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f801024b-7375-4ac8-928b-3dd404f7467a tempest-ServerDiagnosticsNegativeTest-1844467799 tempest-ServerDiagnosticsNegativeTest-1844467799-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:53.768923 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f801024b-7375-4ac8-928b-3dd404f7467a tempest-ServerDiagnosticsNegativeTest-1844467799 tempest-ServerDiagnosticsNegativeTest-1844467799-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:53.771047 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f801024b-7375-4ac8-928b-3dd404f7467a tempest-ServerDiagnosticsNegativeTest-1844467799 tempest-ServerDiagnosticsNegativeTest-1844467799-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:53.771835 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f801024b-7375-4ac8-928b-3dd404f7467a tempest-ServerDiagnosticsNegativeTest-1844467799 tempest-ServerDiagnosticsNegativeTest-1844467799-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:53.800355 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-f801024b-7375-4ac8-928b-3dd404f7467a tempest-ServerDiagnosticsNegativeTest-1844467799 tempest-ServerDiagnosticsNegativeTest-1844467799-project] Getting quotas for project 5f9cdca4179c4875a4be57cf8a9f46a8. Resources: {'ram', 'instances', 'cores'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:09:53.806027 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-f801024b-7375-4ac8-928b-3dd404f7467a tempest-ServerDiagnosticsNegativeTest-1844467799 tempest-ServerDiagnosticsNegativeTest-1844467799-project] Getting quotas for user 9c428093bcf445a6aea6ddb209e6e5cf and project 5f9cdca4179c4875a4be57cf8a9f46a8. Resources: {'ram', 'instances', 'cores'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:09:53.828379 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-f801024b-7375-4ac8-928b-3dd404f7467a tempest-ServerDiagnosticsNegativeTest-1844467799 tempest-ServerDiagnosticsNegativeTest-1844467799-project] Going to run 1 instances... {{(pid=138680) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 16 21:09:53.834767 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-f801024b-7375-4ac8-928b-3dd404f7467a tempest-ServerDiagnosticsNegativeTest-1844467799 tempest-ServerDiagnosticsNegativeTest-1844467799-project] [instance: 1e4229cb-4ed9-48c3-b723-28619ae7045c] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138680) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 16 21:09:53.867715 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-3d704b4f-84b1-4002-b27f-ba72871a53b4 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a69f9dee03d43beaafa966216b9e289', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9b769bf8f3ea4e368bf68de1df6538f9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:53.876799 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-3d704b4f-84b1-4002-b27f-ba72871a53b4 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] 10.222.0.16 "GET /compute/v2.1/servers/c40ba20d-eb88-4ee8-be8f-ecbbd3315c48" status: 200 len: 1503 microversion: 2.1 time: 0.200938 Jul 16 21:09:53.876799 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 257/510] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:09:53 2022] GET /compute/v2.1/servers/c40ba20d-eb88-4ee8-be8f-ecbbd3315c48 => generated 1503 bytes in 203 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:09:53.882151 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-b24ba244-8372-44f8-b4e7-40a1a8d49d5f tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:53.882983 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-b24ba244-8372-44f8-b4e7-40a1a8d49d5f tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: f98a9762-a67c-4e95-85b8-727c81bdb03a] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:09:53.949523 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-b24ba244-8372-44f8-b4e7-40a1a8d49d5f tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b68dacc4b8463ba07fae6372131117', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55a19b7a4682457a9132f5fc80110712', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:53.958964 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-b24ba244-8372-44f8-b4e7-40a1a8d49d5f tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "GET /compute/v2.1/servers/f98a9762-a67c-4e95-85b8-727c81bdb03a" status: 200 len: 1212 microversion: 2.1 time: 0.080312 Jul 16 21:09:53.960042 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 258/511] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:09:53 2022] GET /compute/v2.1/servers/f98a9762-a67c-4e95-85b8-727c81bdb03a => generated 1212 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:09:53.969935 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-ea721e43-6a17-4fcd-9fa0-d4512b4aab3f tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:53.970718 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-ea721e43-6a17-4fcd-9fa0-d4512b4aab3f tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] [instance: 944f68af-bf99-428f-9357-9ce9f807483b] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:09:53.977703 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ea721e43-6a17-4fcd-9fa0-d4512b4aab3f tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:53.978208 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ea721e43-6a17-4fcd-9fa0-d4512b4aab3f tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:53.978987 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ea721e43-6a17-4fcd-9fa0-d4512b4aab3f tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:53.993467 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ea721e43-6a17-4fcd-9fa0-d4512b4aab3f tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:54.003727 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-f801024b-7375-4ac8-928b-3dd404f7467a tempest-ServerDiagnosticsNegativeTest-1844467799 tempest-ServerDiagnosticsNegativeTest-1844467799-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.688951 Jul 16 21:09:54.004924 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 254/512] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 21:09:53 2022] POST /compute/v2.1/servers => generated 380 bytes in 691 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 16 21:09:54.010624 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-56387df5-e759-4c38-90b5-02e53b052506 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:54.011489 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-56387df5-e759-4c38-90b5-02e53b052506 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] [instance: cf53f7cd-deae-4b34-bf8e-3afef98b2ec6] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:09:54.033607 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-56387df5-e759-4c38-90b5-02e53b052506 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:54.034073 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-56387df5-e759-4c38-90b5-02e53b052506 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:54.034883 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-56387df5-e759-4c38-90b5-02e53b052506 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:54.035353 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-56387df5-e759-4c38-90b5-02e53b052506 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:54.151414 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-56387df5-e759-4c38-90b5-02e53b052506 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ba60685a2001459cbfb531d95a6b296e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb6deae6b08e4ed8b15dc6edfe1d38d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:54.161900 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-56387df5-e759-4c38-90b5-02e53b052506 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ba60685a2001459cbfb531d95a6b296e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb6deae6b08e4ed8b15dc6edfe1d38d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:54.163697 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-56387df5-e759-4c38-90b5-02e53b052506 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ba60685a2001459cbfb531d95a6b296e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb6deae6b08e4ed8b15dc6edfe1d38d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:54.170729 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-56387df5-e759-4c38-90b5-02e53b052506 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] 10.222.0.16 "GET /compute/v2.1/servers/cf53f7cd-deae-4b34-bf8e-3afef98b2ec6" status: 200 len: 1447 microversion: 2.75 time: 0.164620 Jul 16 21:09:54.184168 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 255/513] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:09:54 2022] GET /compute/v2.1/servers/cf53f7cd-deae-4b34-bf8e-3afef98b2ec6 => generated 1447 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:09:54.193051 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-cd619d2f-953e-4852-94e2-08f428ae8a0f tempest-ServerDiagnosticsNegativeTest-1844467799 tempest-ServerDiagnosticsNegativeTest-1844467799-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:54.194450 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-cd619d2f-953e-4852-94e2-08f428ae8a0f tempest-ServerDiagnosticsNegativeTest-1844467799 tempest-ServerDiagnosticsNegativeTest-1844467799-project] [instance: 1e4229cb-4ed9-48c3-b723-28619ae7045c] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:09:54.216013 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-ea721e43-6a17-4fcd-9fa0-d4512b4aab3f tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'abeb7020019d41a18de42956d706a011', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '649babe54d104436af461f1ee4767001', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:54.230505 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-ea721e43-6a17-4fcd-9fa0-d4512b4aab3f tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] 10.222.0.16 "GET /compute/v2.1/servers/944f68af-bf99-428f-9357-9ce9f807483b" status: 200 len: 1468 microversion: 2.1 time: 0.264262 Jul 16 21:09:54.243548 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 259/514] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:09:53 2022] GET /compute/v2.1/servers/944f68af-bf99-428f-9357-9ce9f807483b => generated 1468 bytes in 266 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:09:54.265064 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-5b77c179-4a0e-464e-8f36-f0ed9004af80 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:54.265839 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-5b77c179-4a0e-464e-8f36-f0ed9004af80 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] [instance: 45164bce-7acf-4ef6-8afa-aaab6287ecf4] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:09:54.275840 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-cd619d2f-953e-4852-94e2-08f428ae8a0f tempest-ServerDiagnosticsNegativeTest-1844467799 tempest-ServerDiagnosticsNegativeTest-1844467799-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9c428093bcf445a6aea6ddb209e6e5cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5f9cdca4179c4875a4be57cf8a9f46a8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:54.287407 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-cd619d2f-953e-4852-94e2-08f428ae8a0f tempest-ServerDiagnosticsNegativeTest-1844467799 tempest-ServerDiagnosticsNegativeTest-1844467799-project] 10.222.0.16 "GET /compute/v2.1/servers/1e4229cb-4ed9-48c3-b723-28619ae7045c" status: 200 len: 1223 microversion: 2.1 time: 0.101478 Jul 16 21:09:54.289462 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5b77c179-4a0e-464e-8f36-f0ed9004af80 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:54.289890 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5b77c179-4a0e-464e-8f36-f0ed9004af80 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:54.290856 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5b77c179-4a0e-464e-8f36-f0ed9004af80 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:54.291339 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5b77c179-4a0e-464e-8f36-f0ed9004af80 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:54.295683 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 256/515] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:09:54 2022] GET /compute/v2.1/servers/1e4229cb-4ed9-48c3-b723-28619ae7045c => generated 1223 bytes in 111 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:09:54.331069 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-274a4f87-21e8-4f0b-aa1e-d1944cb0331c tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:54.331695 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-274a4f87-21e8-4f0b-aa1e-d1944cb0331c tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] [instance: 67266541-b43e-414c-a744-9c32c6f53975] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:09:54.342955 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-274a4f87-21e8-4f0b-aa1e-d1944cb0331c tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:54.343442 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-274a4f87-21e8-4f0b-aa1e-d1944cb0331c tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:54.347826 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-274a4f87-21e8-4f0b-aa1e-d1944cb0331c tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:54.348357 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-274a4f87-21e8-4f0b-aa1e-d1944cb0331c tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:54.515108 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-5b77c179-4a0e-464e-8f36-f0ed9004af80 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'abeb7020019d41a18de42956d706a011', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '649babe54d104436af461f1ee4767001', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:54.524526 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-5b77c179-4a0e-464e-8f36-f0ed9004af80 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] 10.222.0.16 "GET /compute/v2.1/servers/45164bce-7acf-4ef6-8afa-aaab6287ecf4" status: 200 len: 1469 microversion: 2.1 time: 0.270817 Jul 16 21:09:54.529802 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 260/516] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:09:54 2022] GET /compute/v2.1/servers/45164bce-7acf-4ef6-8afa-aaab6287ecf4 => generated 1469 bytes in 275 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:09:54.537398 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-77bfa846-a58b-44d7-96d1-ad1c884a1c11 req-de9b38af-0fb5-4051-81be-b3a8119a87c9 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "c40ba20d-eb88-4ee8-be8f-ecbbd3315c48", "name": "network-vif-unplugged", "status": "completed", "tag": "a2fac50b-598b-43e0-8e39-546a5c1b4490"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:09:54.545857 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-77bfa846-a58b-44d7-96d1-ad1c884a1c11 req-de9b38af-0fb5-4051-81be-b3a8119a87c9 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:54.546656 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-77bfa846-a58b-44d7-96d1-ad1c884a1c11 req-de9b38af-0fb5-4051-81be-b3a8119a87c9 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:54.556525 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-274a4f87-21e8-4f0b-aa1e-d1944cb0331c tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0d72626f465b4f37ad3b18bb945d4000', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fc1c3463f9444119af9d18f85bdd703d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:54.562713 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:09:54.567161 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-77bfa846-a58b-44d7-96d1-ad1c884a1c11 req-de9b38af-0fb5-4051-81be-b3a8119a87c9 service nova] Creating event network-vif-unplugged:a2fac50b-598b-43e0-8e39-546a5c1b4490 for instance c40ba20d-eb88-4ee8-be8f-ecbbd3315c48 on nch1-master-1 Jul 16 21:09:54.569563 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-274a4f87-21e8-4f0b-aa1e-d1944cb0331c tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] 10.222.0.16 "GET /compute/v2.1/servers/67266541-b43e-414c-a744-9c32c6f53975" status: 200 len: 1319 microversion: 2.1 time: 0.243661 Jul 16 21:09:54.579566 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-77bfa846-a58b-44d7-96d1-ad1c884a1c11 req-de9b38af-0fb5-4051-81be-b3a8119a87c9 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.042423 Jul 16 21:09:54.579566 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 261/517] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:09:54 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:09:54.580499 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-08fbd9aa-ac3b-4f79-aa4a-b696f4b2d22f tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:54.581139 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 257/518] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:09:54 2022] GET /compute/v2.1/servers/67266541-b43e-414c-a744-9c32c6f53975 => generated 1319 bytes in 256 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:09:54.584257 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-08fbd9aa-ac3b-4f79-aa4a-b696f4b2d22f tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Policy check for os_compute_api:servers:allow_all_filters failed with credentials {'is_admin': False, 'user_id': 'abeb7020019d41a18de42956d706a011', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '649babe54d104436af461f1ee4767001', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:54.585112 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-08fbd9aa-ac3b-4f79-aa4a-b696f4b2d22f tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Searching by: {'reservation_id': 'r-090jjfb0', 'deleted': False, 'project_id': '649babe54d104436af461f1ee4767001'} {{(pid=138681) get_all /opt/stack/nova/nova/compute/api.py:2962}} Jul 16 21:09:54.592823 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-08fbd9aa-ac3b-4f79-aa4a-b696f4b2d22f tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:54.593303 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-08fbd9aa-ac3b-4f79-aa4a-b696f4b2d22f tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:54.594137 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-08fbd9aa-ac3b-4f79-aa4a-b696f4b2d22f tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:54.594572 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-08fbd9aa-ac3b-4f79-aa4a-b696f4b2d22f tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:54.779686 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.multi_cell_list [None req-08fbd9aa-ac3b-4f79-aa4a-b696f4b2d22f tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Listed batch of 2 results from cell out of 1000 limit. Returned 2 total so far. {{(pid=138681) do_query /opt/stack/nova/nova/compute/multi_cell_list.py:373}} Jul 16 21:09:54.894174 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-1588fce2-0373-455c-a7d2-cd67c573b5b6 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:54.895067 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-1588fce2-0373-455c-a7d2-cd67c573b5b6 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] [instance: c40ba20d-eb88-4ee8-be8f-ecbbd3315c48] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:09:54.902808 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1588fce2-0373-455c-a7d2-cd67c573b5b6 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:54.903452 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1588fce2-0373-455c-a7d2-cd67c573b5b6 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:54.904455 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1588fce2-0373-455c-a7d2-cd67c573b5b6 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:54.910023 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1588fce2-0373-455c-a7d2-cd67c573b5b6 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:54.910696 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-08fbd9aa-ac3b-4f79-aa4a-b696f4b2d22f tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'abeb7020019d41a18de42956d706a011', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '649babe54d104436af461f1ee4767001', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:54.913736 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_api:SELECT=97,nova_api:INSERT=3 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:09:54.918457 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-08fbd9aa-ac3b-4f79-aa4a-b696f4b2d22f tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:54.918902 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-08fbd9aa-ac3b-4f79-aa4a-b696f4b2d22f tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:55.196964 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-1588fce2-0373-455c-a7d2-cd67c573b5b6 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a69f9dee03d43beaafa966216b9e289', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9b769bf8f3ea4e368bf68de1df6538f9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:55.200783 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-08fbd9aa-ac3b-4f79-aa4a-b696f4b2d22f tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] 10.222.0.16 "GET /compute/v2.1/servers/detail?reservation_id=r-090jjfb0" status: 200 len: 2930 microversion: 2.1 time: 0.625237 Jul 16 21:09:55.201387 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 262/519] 10.222.0.16 () {58 vars in 1196 bytes} [Sat Jul 16 21:09:54 2022] GET /compute/v2.1/servers/detail?reservation_id=r-090jjfb0 => generated 2930 bytes in 627 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:09:55.222524 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-34db2578-6b06-4b10-92e1-733793d1a401 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:55.222802 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-1588fce2-0373-455c-a7d2-cd67c573b5b6 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] 10.222.0.16 "GET /compute/v2.1/servers/c40ba20d-eb88-4ee8-be8f-ecbbd3315c48" status: 200 len: 1503 microversion: 2.1 time: 0.332985 Jul 16 21:09:55.223699 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-34db2578-6b06-4b10-92e1-733793d1a401 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: f98a9762-a67c-4e95-85b8-727c81bdb03a] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:09:55.224083 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 258/520] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:09:54 2022] GET /compute/v2.1/servers/c40ba20d-eb88-4ee8-be8f-ecbbd3315c48 => generated 1503 bytes in 335 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:09:55.230114 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-8b6c3424-3301-4615-81de-0b3e071bfb4a tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:55.231836 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-8b6c3424-3301-4615-81de-0b3e071bfb4a tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] [instance: cf53f7cd-deae-4b34-bf8e-3afef98b2ec6] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:09:55.239922 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8b6c3424-3301-4615-81de-0b3e071bfb4a tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:55.242585 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8b6c3424-3301-4615-81de-0b3e071bfb4a tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:55.242585 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8b6c3424-3301-4615-81de-0b3e071bfb4a tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:55.243121 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8b6c3424-3301-4615-81de-0b3e071bfb4a tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:55.244018 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-34db2578-6b06-4b10-92e1-733793d1a401 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:55.244644 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-34db2578-6b06-4b10-92e1-733793d1a401 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:55.245722 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-34db2578-6b06-4b10-92e1-733793d1a401 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:55.246164 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-34db2578-6b06-4b10-92e1-733793d1a401 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:55.377522 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-34db2578-6b06-4b10-92e1-733793d1a401 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b68dacc4b8463ba07fae6372131117', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55a19b7a4682457a9132f5fc80110712', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:55.387313 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-8b6c3424-3301-4615-81de-0b3e071bfb4a tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ba60685a2001459cbfb531d95a6b296e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb6deae6b08e4ed8b15dc6edfe1d38d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:55.397229 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-8b6c3424-3301-4615-81de-0b3e071bfb4a tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ba60685a2001459cbfb531d95a6b296e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb6deae6b08e4ed8b15dc6edfe1d38d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:55.398970 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-8b6c3424-3301-4615-81de-0b3e071bfb4a tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ba60685a2001459cbfb531d95a6b296e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb6deae6b08e4ed8b15dc6edfe1d38d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:55.401161 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-34db2578-6b06-4b10-92e1-733793d1a401 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "GET /compute/v2.1/servers/f98a9762-a67c-4e95-85b8-727c81bdb03a" status: 200 len: 1216 microversion: 2.1 time: 0.186391 Jul 16 21:09:55.404178 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 263/521] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:09:55 2022] GET /compute/v2.1/servers/f98a9762-a67c-4e95-85b8-727c81bdb03a => generated 1216 bytes in 201 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:09:55.414358 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-8b6c3424-3301-4615-81de-0b3e071bfb4a tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] 10.222.0.16 "GET /compute/v2.1/servers/cf53f7cd-deae-4b34-bf8e-3afef98b2ec6" status: 200 len: 1447 microversion: 2.75 time: 0.188884 Jul 16 21:09:55.416906 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 259/522] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:09:55 2022] GET /compute/v2.1/servers/cf53f7cd-deae-4b34-bf8e-3afef98b2ec6 => generated 1447 bytes in 191 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:09:55.420556 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-328159bb-5fd6-4293-847d-ef7c6bbf7652 tempest-ServerDiagnosticsNegativeTest-1844467799 tempest-ServerDiagnosticsNegativeTest-1844467799-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:55.421363 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-328159bb-5fd6-4293-847d-ef7c6bbf7652 tempest-ServerDiagnosticsNegativeTest-1844467799 tempest-ServerDiagnosticsNegativeTest-1844467799-project] [instance: 1e4229cb-4ed9-48c3-b723-28619ae7045c] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:09:55.425603 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-3d578a46-52e7-4173-bbd0-bdfbaded337a tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:55.428549 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-328159bb-5fd6-4293-847d-ef7c6bbf7652 tempest-ServerDiagnosticsNegativeTest-1844467799 tempest-ServerDiagnosticsNegativeTest-1844467799-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:55.429215 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-328159bb-5fd6-4293-847d-ef7c6bbf7652 tempest-ServerDiagnosticsNegativeTest-1844467799 tempest-ServerDiagnosticsNegativeTest-1844467799-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:55.430155 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-328159bb-5fd6-4293-847d-ef7c6bbf7652 tempest-ServerDiagnosticsNegativeTest-1844467799 tempest-ServerDiagnosticsNegativeTest-1844467799-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:55.430808 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-328159bb-5fd6-4293-847d-ef7c6bbf7652 tempest-ServerDiagnosticsNegativeTest-1844467799 tempest-ServerDiagnosticsNegativeTest-1844467799-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:55.468728 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.compute.server_groups [None req-3d578a46-52e7-4173-bbd0-bdfbaded337a tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Querying cell 327e8df1-39cc-4b36-9ea6-6a9b88ed54fc(cell1) for 2 instances {{(pid=138681) _get_not_deleted /opt/stack/nova/nova/api/openstack/compute/server_groups.py:69}} Jul 16 21:09:55.470039 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3d578a46-52e7-4173-bbd0-bdfbaded337a tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:55.483969 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3d578a46-52e7-4173-bbd0-bdfbaded337a tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.014s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:55.499425 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-328159bb-5fd6-4293-847d-ef7c6bbf7652 tempest-ServerDiagnosticsNegativeTest-1844467799 tempest-ServerDiagnosticsNegativeTest-1844467799-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9c428093bcf445a6aea6ddb209e6e5cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5f9cdca4179c4875a4be57cf8a9f46a8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:55.510500 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-328159bb-5fd6-4293-847d-ef7c6bbf7652 tempest-ServerDiagnosticsNegativeTest-1844467799 tempest-ServerDiagnosticsNegativeTest-1844467799-project] 10.222.0.16 "GET /compute/v2.1/servers/1e4229cb-4ed9-48c3-b723-28619ae7045c" status: 200 len: 1227 microversion: 2.1 time: 0.093995 Jul 16 21:09:55.511635 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 260/523] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:09:55 2022] GET /compute/v2.1/servers/1e4229cb-4ed9-48c3-b723-28619ae7045c => generated 1227 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:09:55.553764 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-3d578a46-52e7-4173-bbd0-bdfbaded337a tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] 10.222.0.16 "GET /compute/v2.1/os-server-groups/73f6bd9e-6a32-47fd-a7a1-a896ecc028e4" status: 200 len: 266 microversion: 2.1 time: 0.148202 Jul 16 21:09:55.554535 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 264/524] 10.222.0.16 () {58 vars in 1262 bytes} [Sat Jul 16 21:09:55 2022] GET /compute/v2.1/os-server-groups/73f6bd9e-6a32-47fd-a7a1-a896ecc028e4 => generated 266 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:09:55.564369 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-c5bf889c-a638-4272-b456-cdec4c4e9bf2 tempest-ServersOnMultiNodesTest-83775055 tempest-ServersOnMultiNodesTest-83775055-project-admin] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:55.564985 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-c5bf889c-a638-4272-b456-cdec4c4e9bf2 tempest-ServersOnMultiNodesTest-83775055 tempest-ServersOnMultiNodesTest-83775055-project-admin] [instance: 45164bce-7acf-4ef6-8afa-aaab6287ecf4] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:09:55.571345 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c5bf889c-a638-4272-b456-cdec4c4e9bf2 tempest-ServersOnMultiNodesTest-83775055 tempest-ServersOnMultiNodesTest-83775055-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:55.572012 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c5bf889c-a638-4272-b456-cdec4c4e9bf2 tempest-ServersOnMultiNodesTest-83775055 tempest-ServersOnMultiNodesTest-83775055-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:55.573064 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c5bf889c-a638-4272-b456-cdec4c4e9bf2 tempest-ServersOnMultiNodesTest-83775055 tempest-ServersOnMultiNodesTest-83775055-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:55.574068 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c5bf889c-a638-4272-b456-cdec4c4e9bf2 tempest-ServersOnMultiNodesTest-83775055 tempest-ServersOnMultiNodesTest-83775055-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:55.587085 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-0685c27e-a83b-49fd-a5c0-6cc3c5ba641e tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:55.587712 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-0685c27e-a83b-49fd-a5c0-6cc3c5ba641e tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] [instance: 67266541-b43e-414c-a744-9c32c6f53975] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:09:55.594175 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0685c27e-a83b-49fd-a5c0-6cc3c5ba641e tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:55.594839 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0685c27e-a83b-49fd-a5c0-6cc3c5ba641e tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:55.595823 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0685c27e-a83b-49fd-a5c0-6cc3c5ba641e tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:55.596274 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0685c27e-a83b-49fd-a5c0-6cc3c5ba641e tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:55.782362 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-0685c27e-a83b-49fd-a5c0-6cc3c5ba641e tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0d72626f465b4f37ad3b18bb945d4000', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fc1c3463f9444119af9d18f85bdd703d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:55.783633 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-c5bf889c-a638-4272-b456-cdec4c4e9bf2 tempest-ServersOnMultiNodesTest-83775055 tempest-ServersOnMultiNodesTest-83775055-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/45164bce-7acf-4ef6-8afa-aaab6287ecf4" status: 200 len: 1620 microversion: 2.1 time: 0.223130 Jul 16 21:09:55.784267 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 261/525] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:09:55 2022] GET /compute/v2.1/servers/45164bce-7acf-4ef6-8afa-aaab6287ecf4 => generated 1620 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:09:55.790879 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-0685c27e-a83b-49fd-a5c0-6cc3c5ba641e tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] 10.222.0.16 "GET /compute/v2.1/servers/67266541-b43e-414c-a744-9c32c6f53975" status: 200 len: 1319 microversion: 2.1 time: 0.207719 Jul 16 21:09:55.791556 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 265/526] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:09:55 2022] GET /compute/v2.1/servers/67266541-b43e-414c-a744-9c32c6f53975 => generated 1319 bytes in 209 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:09:55.800182 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-d71286cf-c146-41d5-8697-742ea02570db tempest-ServersOnMultiNodesTest-83775055 tempest-ServersOnMultiNodesTest-83775055-project-admin] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:55.800789 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-d71286cf-c146-41d5-8697-742ea02570db tempest-ServersOnMultiNodesTest-83775055 tempest-ServersOnMultiNodesTest-83775055-project-admin] [instance: 944f68af-bf99-428f-9357-9ce9f807483b] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:09:55.806854 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d71286cf-c146-41d5-8697-742ea02570db tempest-ServersOnMultiNodesTest-83775055 tempest-ServersOnMultiNodesTest-83775055-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:55.807347 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d71286cf-c146-41d5-8697-742ea02570db tempest-ServersOnMultiNodesTest-83775055 tempest-ServersOnMultiNodesTest-83775055-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:55.808199 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d71286cf-c146-41d5-8697-742ea02570db tempest-ServersOnMultiNodesTest-83775055 tempest-ServersOnMultiNodesTest-83775055-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:55.808632 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d71286cf-c146-41d5-8697-742ea02570db tempest-ServersOnMultiNodesTest-83775055 tempest-ServersOnMultiNodesTest-83775055-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:55.941420 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-d71286cf-c146-41d5-8697-742ea02570db tempest-ServersOnMultiNodesTest-83775055 tempest-ServersOnMultiNodesTest-83775055-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/944f68af-bf99-428f-9357-9ce9f807483b" status: 200 len: 1619 microversion: 2.1 time: 0.144948 Jul 16 21:09:55.941992 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 262/527] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:09:55 2022] GET /compute/v2.1/servers/944f68af-bf99-428f-9357-9ce9f807483b => generated 1619 bytes in 146 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:09:55.960099 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-f500266d-a0d5-46a4-a2b5-54a7be8c81dc tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Action: 'create', calling method: .version_select at 0x7fdd9d6dcd30>, body: {"server_group": {"name": "tempest-ServersOnMultiNodesTest-Server-Group-16712508", "policies": ["anti-affinity"]}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:09:55.981234 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-f500266d-a0d5-46a4-a2b5-54a7be8c81dc tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Getting quotas for project 649babe54d104436af461f1ee4767001. Resources: {'server_groups'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:09:55.986271 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-f500266d-a0d5-46a4-a2b5-54a7be8c81dc tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Getting quotas for user abeb7020019d41a18de42956d706a011 and project 649babe54d104436af461f1ee4767001. Resources: {'server_groups'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:09:56.122558 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-f500266d-a0d5-46a4-a2b5-54a7be8c81dc tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Getting quotas for project 649babe54d104436af461f1ee4767001. Resources: {'server_groups'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:09:56.127968 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-f500266d-a0d5-46a4-a2b5-54a7be8c81dc tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Getting quotas for user abeb7020019d41a18de42956d706a011 and project 649babe54d104436af461f1ee4767001. Resources: {'server_groups'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:09:56.140086 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-f500266d-a0d5-46a4-a2b5-54a7be8c81dc tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] 10.222.0.16 "POST /compute/v2.1/os-server-groups" status: 200 len: 191 microversion: 2.1 time: 0.183946 Jul 16 21:09:56.140691 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 266/528] 10.222.0.16 () {60 vars in 1173 bytes} [Sat Jul 16 21:09:55 2022] POST /compute/v2.1/os-server-groups => generated 191 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 16 21:09:56.241413 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-442172e4-ce74-4459-b18f-160e86b79e93 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:56.242400 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-442172e4-ce74-4459-b18f-160e86b79e93 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] [instance: c40ba20d-eb88-4ee8-be8f-ecbbd3315c48] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:09:56.246439 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-a5449a5c-dcec-4669-be4e-39cdfd3aacb8 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServersOnMultiNodesTest-server-529978616", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452", "min_count": 2, "return_reservation_id": true}, "os:scheduler_hints": {"group": "d6276585-4df3-40a1-817c-e7dcc6987bf4"}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:09:56.249710 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-442172e4-ce74-4459-b18f-160e86b79e93 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:56.251040 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-442172e4-ce74-4459-b18f-160e86b79e93 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:56.252697 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-442172e4-ce74-4459-b18f-160e86b79e93 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:56.253577 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-442172e4-ce74-4459-b18f-160e86b79e93 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:56.302315 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.network.neutron [None req-a5449a5c-dcec-4669-be4e-39cdfd3aacb8 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] validate_networks() for None {{(pid=138681) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 16 21:09:56.794686 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-442172e4-ce74-4459-b18f-160e86b79e93 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a69f9dee03d43beaafa966216b9e289', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9b769bf8f3ea4e368bf68de1df6538f9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:56.803794 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-442172e4-ce74-4459-b18f-160e86b79e93 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] 10.222.0.16 "GET /compute/v2.1/servers/c40ba20d-eb88-4ee8-be8f-ecbbd3315c48" status: 200 len: 1503 microversion: 2.1 time: 0.566309 Jul 16 21:09:56.804390 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 263/529] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:09:56 2022] GET /compute/v2.1/servers/c40ba20d-eb88-4ee8-be8f-ecbbd3315c48 => generated 1503 bytes in 568 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:09:56.809038 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-f4ebb6f2-4754-45c1-83dd-afa673e903a4 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:56.809719 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-f4ebb6f2-4754-45c1-83dd-afa673e903a4 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: f98a9762-a67c-4e95-85b8-727c81bdb03a] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:09:56.812834 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_api:SELECT=97,nova_api:INSERT=3 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:09:56.816759 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f4ebb6f2-4754-45c1-83dd-afa673e903a4 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:56.817209 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f4ebb6f2-4754-45c1-83dd-afa673e903a4 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:56.817988 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f4ebb6f2-4754-45c1-83dd-afa673e903a4 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:56.818433 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f4ebb6f2-4754-45c1-83dd-afa673e903a4 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:56.930181 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-f4ebb6f2-4754-45c1-83dd-afa673e903a4 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b68dacc4b8463ba07fae6372131117', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55a19b7a4682457a9132f5fc80110712', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:56.943467 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-a5449a5c-dcec-4669-be4e-39cdfd3aacb8 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Flavor limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 16 21:09:56.944468 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-a5449a5c-dcec-4669-be4e-39cdfd3aacb8 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Image limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 16 21:09:56.945435 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-a5449a5c-dcec-4669-be4e-39cdfd3aacb8 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Flavor pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 16 21:09:56.946165 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-a5449a5c-dcec-4669-be4e-39cdfd3aacb8 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Image pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 16 21:09:56.947366 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-a5449a5c-dcec-4669-be4e-39cdfd3aacb8 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 16 21:09:56.951095 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-f4ebb6f2-4754-45c1-83dd-afa673e903a4 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "GET /compute/v2.1/servers/f98a9762-a67c-4e95-85b8-727c81bdb03a" status: 200 len: 1272 microversion: 2.1 time: 0.141091 Jul 16 21:09:56.952575 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 264/530] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:09:56 2022] GET /compute/v2.1/servers/f98a9762-a67c-4e95-85b8-727c81bdb03a => generated 1272 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:09:56.960269 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-e89aa5ae-c507-4a71-ad61-a128815c9436 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:56.961359 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-e89aa5ae-c507-4a71-ad61-a128815c9436 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] [instance: cf53f7cd-deae-4b34-bf8e-3afef98b2ec6] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:09:56.968370 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e89aa5ae-c507-4a71-ad61-a128815c9436 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:56.969830 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e89aa5ae-c507-4a71-ad61-a128815c9436 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:56.970661 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e89aa5ae-c507-4a71-ad61-a128815c9436 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:56.971131 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e89aa5ae-c507-4a71-ad61-a128815c9436 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:56.974380 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a5449a5c-dcec-4669-be4e-39cdfd3aacb8 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:56.975443 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a5449a5c-dcec-4669-be4e-39cdfd3aacb8 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:56.976359 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a5449a5c-dcec-4669-be4e-39cdfd3aacb8 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:56.976852 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a5449a5c-dcec-4669-be4e-39cdfd3aacb8 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:57.010148 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-a5449a5c-dcec-4669-be4e-39cdfd3aacb8 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Getting quotas for project 649babe54d104436af461f1ee4767001. Resources: {'ram', 'instances', 'cores'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:09:57.018124 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-a5449a5c-dcec-4669-be4e-39cdfd3aacb8 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Getting quotas for user abeb7020019d41a18de42956d706a011 and project 649babe54d104436af461f1ee4767001. Resources: {'ram', 'instances', 'cores'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:09:57.033626 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-a5449a5c-dcec-4669-be4e-39cdfd3aacb8 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Going to run 2 instances... {{(pid=138681) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 16 21:09:57.039245 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-a5449a5c-dcec-4669-be4e-39cdfd3aacb8 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] [instance: 84ee8179-ff94-43b3-97f2-1867bc6420a8] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138681) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 16 21:09:57.068629 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-e89aa5ae-c507-4a71-ad61-a128815c9436 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ba60685a2001459cbfb531d95a6b296e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb6deae6b08e4ed8b15dc6edfe1d38d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:57.080649 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-e89aa5ae-c507-4a71-ad61-a128815c9436 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ba60685a2001459cbfb531d95a6b296e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb6deae6b08e4ed8b15dc6edfe1d38d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:57.086574 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-e89aa5ae-c507-4a71-ad61-a128815c9436 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ba60685a2001459cbfb531d95a6b296e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb6deae6b08e4ed8b15dc6edfe1d38d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:57.093333 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-e89aa5ae-c507-4a71-ad61-a128815c9436 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] 10.222.0.16 "GET /compute/v2.1/servers/cf53f7cd-deae-4b34-bf8e-3afef98b2ec6" status: 200 len: 1447 microversion: 2.75 time: 0.137294 Jul 16 21:09:57.109070 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 265/531] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:09:56 2022] GET /compute/v2.1/servers/cf53f7cd-deae-4b34-bf8e-3afef98b2ec6 => generated 1447 bytes in 143 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:09:57.109070 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-e5c27d8a-272a-4d13-83ba-ce2ccabb9691 tempest-ServerDiagnosticsNegativeTest-1844467799 tempest-ServerDiagnosticsNegativeTest-1844467799-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:57.109070 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-e5c27d8a-272a-4d13-83ba-ce2ccabb9691 tempest-ServerDiagnosticsNegativeTest-1844467799 tempest-ServerDiagnosticsNegativeTest-1844467799-project] [instance: 1e4229cb-4ed9-48c3-b723-28619ae7045c] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:09:57.117128 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e5c27d8a-272a-4d13-83ba-ce2ccabb9691 tempest-ServerDiagnosticsNegativeTest-1844467799 tempest-ServerDiagnosticsNegativeTest-1844467799-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:57.117667 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e5c27d8a-272a-4d13-83ba-ce2ccabb9691 tempest-ServerDiagnosticsNegativeTest-1844467799 tempest-ServerDiagnosticsNegativeTest-1844467799-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:57.118685 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e5c27d8a-272a-4d13-83ba-ce2ccabb9691 tempest-ServerDiagnosticsNegativeTest-1844467799 tempest-ServerDiagnosticsNegativeTest-1844467799-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:57.119451 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e5c27d8a-272a-4d13-83ba-ce2ccabb9691 tempest-ServerDiagnosticsNegativeTest-1844467799 tempest-ServerDiagnosticsNegativeTest-1844467799-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:57.190751 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-e5c27d8a-272a-4d13-83ba-ce2ccabb9691 tempest-ServerDiagnosticsNegativeTest-1844467799 tempest-ServerDiagnosticsNegativeTest-1844467799-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9c428093bcf445a6aea6ddb209e6e5cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5f9cdca4179c4875a4be57cf8a9f46a8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:57.198315 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-e5c27d8a-272a-4d13-83ba-ce2ccabb9691 tempest-ServerDiagnosticsNegativeTest-1844467799 tempest-ServerDiagnosticsNegativeTest-1844467799-project] 10.222.0.16 "GET /compute/v2.1/servers/1e4229cb-4ed9-48c3-b723-28619ae7045c" status: 200 len: 1275 microversion: 2.1 time: 0.099040 Jul 16 21:09:57.198897 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 266/532] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:09:57 2022] GET /compute/v2.1/servers/1e4229cb-4ed9-48c3-b723-28619ae7045c => generated 1275 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:09:57.203936 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-cba78ea8-1847-4d89-8e1b-ec5d4d27a2a5 req-35af3b6d-c69f-49b7-b102-564f0f894de4 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "c40ba20d-eb88-4ee8-be8f-ecbbd3315c48", "tag": "a2fac50b-598b-43e0-8e39-546a5c1b4490"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:09:57.213115 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-cba78ea8-1847-4d89-8e1b-ec5d4d27a2a5 req-35af3b6d-c69f-49b7-b102-564f0f894de4 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:57.213563 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-cba78ea8-1847-4d89-8e1b-ec5d4d27a2a5 req-35af3b6d-c69f-49b7-b102-564f0f894de4 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:57.226700 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-cba78ea8-1847-4d89-8e1b-ec5d4d27a2a5 req-35af3b6d-c69f-49b7-b102-564f0f894de4 service nova] Creating event network-changed:a2fac50b-598b-43e0-8e39-546a5c1b4490 for instance c40ba20d-eb88-4ee8-be8f-ecbbd3315c48 on nch1-master-1 Jul 16 21:09:57.231981 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-cba78ea8-1847-4d89-8e1b-ec5d4d27a2a5 req-35af3b6d-c69f-49b7-b102-564f0f894de4 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.031739 Jul 16 21:09:57.232603 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 267/533] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:09:57 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:09:57.237912 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-a5449a5c-dcec-4669-be4e-39cdfd3aacb8 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Checking whether user_id and queued_for_delete are populated for all projects {{(pid=138681) is_qfd_populated /opt/stack/nova/nova/quota.py:1268}} Jul 16 21:09:57.238807 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-0e7b48c9-0dbc-447d-8656-afaef538852e tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:57.239559 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-0e7b48c9-0dbc-447d-8656-afaef538852e tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] [instance: 67266541-b43e-414c-a744-9c32c6f53975] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:09:57.246659 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0e7b48c9-0dbc-447d-8656-afaef538852e tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:57.247210 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0e7b48c9-0dbc-447d-8656-afaef538852e tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:57.247995 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0e7b48c9-0dbc-447d-8656-afaef538852e tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:57.248524 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0e7b48c9-0dbc-447d-8656-afaef538852e tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:57.264310 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-a5449a5c-dcec-4669-be4e-39cdfd3aacb8 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Getting quotas for project 649babe54d104436af461f1ee4767001. Resources: {'server_group_members'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:09:57.270271 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-a5449a5c-dcec-4669-be4e-39cdfd3aacb8 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Getting quotas for user abeb7020019d41a18de42956d706a011 and project 649babe54d104436af461f1ee4767001. Resources: {'server_group_members'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:09:57.424510 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-a5449a5c-dcec-4669-be4e-39cdfd3aacb8 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Getting quotas for project 649babe54d104436af461f1ee4767001. Resources: {'server_group_members'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:09:57.431935 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-a5449a5c-dcec-4669-be4e-39cdfd3aacb8 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Getting quotas for user abeb7020019d41a18de42956d706a011 and project 649babe54d104436af461f1ee4767001. Resources: {'server_group_members'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:09:57.449598 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-a5449a5c-dcec-4669-be4e-39cdfd3aacb8 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] [instance: 7d624313-0c12-4da2-a95c-3903efae7608] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9',instance=,instance_uuid=84ee8179-ff94-43b3-97f2-1867bc6420a8,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138681) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 16 21:09:57.511903 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-0e7b48c9-0dbc-447d-8656-afaef538852e tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0d72626f465b4f37ad3b18bb945d4000', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fc1c3463f9444119af9d18f85bdd703d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:57.524341 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-0e7b48c9-0dbc-447d-8656-afaef538852e tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] 10.222.0.16 "GET /compute/v2.1/servers/67266541-b43e-414c-a744-9c32c6f53975" status: 200 len: 1468 microversion: 2.1 time: 0.290469 Jul 16 21:09:57.525270 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 268/534] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:09:57 2022] GET /compute/v2.1/servers/67266541-b43e-414c-a744-9c32c6f53975 => generated 1468 bytes in 292 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:09:57.531556 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-75a493af-8d62-4014-a3c9-9fd152231974 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:57.533272 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-75a493af-8d62-4014-a3c9-9fd152231974 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] [instance: e506a6e0-fbcd-4e53-a6a0-d31761654c92] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:09:57.543092 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-75a493af-8d62-4014-a3c9-9fd152231974 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:57.543556 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-75a493af-8d62-4014-a3c9-9fd152231974 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:57.544333 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-75a493af-8d62-4014-a3c9-9fd152231974 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:57.544764 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-75a493af-8d62-4014-a3c9-9fd152231974 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:57.591670 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-75a493af-8d62-4014-a3c9-9fd152231974 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] [instance: e506a6e0-fbcd-4e53-a6a0-d31761654c92] Going to try to terminate instance {{(pid=138680) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 16 21:09:57.654234 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-a5449a5c-dcec-4669-be4e-39cdfd3aacb8 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Getting quotas for project 649babe54d104436af461f1ee4767001. Resources: {'server_group_members'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:09:57.660276 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-a5449a5c-dcec-4669-be4e-39cdfd3aacb8 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Getting quotas for user abeb7020019d41a18de42956d706a011 and project 649babe54d104436af461f1ee4767001. Resources: {'server_group_members'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:09:57.808779 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_api:SELECT=90,nova_api:INSERT=10 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:09:57.816558 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-a5449a5c-dcec-4669-be4e-39cdfd3aacb8 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Getting quotas for project 649babe54d104436af461f1ee4767001. Resources: {'server_group_members'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:09:57.821630 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-a5449a5c-dcec-4669-be4e-39cdfd3aacb8 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Getting quotas for user abeb7020019d41a18de42956d706a011 and project 649babe54d104436af461f1ee4767001. Resources: {'server_group_members'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:09:57.865941 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-a5449a5c-dcec-4669-be4e-39cdfd3aacb8 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 32 microversion: 2.1 time: 1.623478 Jul 16 21:09:57.870745 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 267/535] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 21:09:56 2022] POST /compute/v2.1/servers => generated 32 bytes in 1629 msecs (HTTP/1.1 202) 9 headers in 362 bytes (2 switches on core 0) Jul 16 21:09:57.877087 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-063a6555-9c68-4c7c-9337-4dee42951241 tempest-LiveMigrationNegativeTest-1294767845 tempest-LiveMigrationNegativeTest-1294767845-project-admin] Action: 'action', calling method: >, body: {"suspend": {}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:09:57.877825 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-063a6555-9c68-4c7c-9337-4dee42951241 tempest-LiveMigrationNegativeTest-1294767845 tempest-LiveMigrationNegativeTest-1294767845-project-admin] [instance: 67266541-b43e-414c-a744-9c32c6f53975] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:09:57.886925 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-063a6555-9c68-4c7c-9337-4dee42951241 tempest-LiveMigrationNegativeTest-1294767845 tempest-LiveMigrationNegativeTest-1294767845-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:57.887826 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-063a6555-9c68-4c7c-9337-4dee42951241 tempest-LiveMigrationNegativeTest-1294767845 tempest-LiveMigrationNegativeTest-1294767845-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:57.888918 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-063a6555-9c68-4c7c-9337-4dee42951241 tempest-LiveMigrationNegativeTest-1294767845 tempest-LiveMigrationNegativeTest-1294767845-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:57.889541 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-063a6555-9c68-4c7c-9337-4dee42951241 tempest-LiveMigrationNegativeTest-1294767845 tempest-LiveMigrationNegativeTest-1294767845-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:58.007850 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-75a493af-8d62-4014-a3c9-9fd152231974 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] 10.222.0.16 "DELETE /compute/v2.1/servers/e506a6e0-fbcd-4e53-a6a0-d31761654c92" status: 204 len: 0 microversion: 2.1 time: 0.480328 Jul 16 21:09:58.018862 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 269/536] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:09:57 2022] DELETE /compute/v2.1/servers/e506a6e0-fbcd-4e53-a6a0-d31761654c92 => generated 0 bytes in 484 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 16 21:09:58.027255 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-fcaee8e8-6a3d-4181-b1a2-e8623b5e67ee tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:58.029265 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-fcaee8e8-6a3d-4181-b1a2-e8623b5e67ee tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] [instance: c40ba20d-eb88-4ee8-be8f-ecbbd3315c48] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:09:58.046194 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-fcaee8e8-6a3d-4181-b1a2-e8623b5e67ee tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:58.047492 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-fcaee8e8-6a3d-4181-b1a2-e8623b5e67ee tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:58.049489 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-fcaee8e8-6a3d-4181-b1a2-e8623b5e67ee tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:58.049630 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.objects.instance [None req-063a6555-9c68-4c7c-9337-4dee42951241 tempest-LiveMigrationNegativeTest-1294767845 tempest-LiveMigrationNegativeTest-1294767845-project-admin] Lazy-loading 'flavor' on Instance uuid 67266541-b43e-414c-a744-9c32c6f53975 {{(pid=138681) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:09:58.053985 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-fcaee8e8-6a3d-4181-b1a2-e8623b5e67ee tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:58.405891 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-fcaee8e8-6a3d-4181-b1a2-e8623b5e67ee tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a69f9dee03d43beaafa966216b9e289', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9b769bf8f3ea4e368bf68de1df6538f9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:58.417910 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-fcaee8e8-6a3d-4181-b1a2-e8623b5e67ee tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] 10.222.0.16 "GET /compute/v2.1/servers/c40ba20d-eb88-4ee8-be8f-ecbbd3315c48" status: 200 len: 1503 microversion: 2.1 time: 0.405758 Jul 16 21:09:58.429945 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 270/537] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:09:58 2022] GET /compute/v2.1/servers/c40ba20d-eb88-4ee8-be8f-ecbbd3315c48 => generated 1503 bytes in 419 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:09:58.438735 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-db2906e5-9f09-4744-9912-919a64f94191 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:58.444812 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-db2906e5-9f09-4744-9912-919a64f94191 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Policy check for os_compute_api:servers:allow_all_filters failed with credentials {'is_admin': False, 'user_id': 'abeb7020019d41a18de42956d706a011', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '649babe54d104436af461f1ee4767001', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:58.447035 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-db2906e5-9f09-4744-9912-919a64f94191 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Searching by: {'deleted': False, 'project_id': '649babe54d104436af461f1ee4767001'} {{(pid=138680) get_all /opt/stack/nova/nova/compute/api.py:2962}} Jul 16 21:09:58.475268 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-db2906e5-9f09-4744-9912-919a64f94191 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:58.475907 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-db2906e5-9f09-4744-9912-919a64f94191 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:58.476827 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-db2906e5-9f09-4744-9912-919a64f94191 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:58.477269 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-db2906e5-9f09-4744-9912-919a64f94191 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:58.524933 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.multi_cell_list [None req-db2906e5-9f09-4744-9912-919a64f94191 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Listed batch of 3 results from cell out of 998 limit. Returned 3 total so far. {{(pid=138680) do_query /opt/stack/nova/nova/compute/multi_cell_list.py:373}} Jul 16 21:09:58.546249 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-063a6555-9c68-4c7c-9337-4dee42951241 tempest-LiveMigrationNegativeTest-1294767845 tempest-LiveMigrationNegativeTest-1294767845-project-admin] 10.222.0.16 "POST /compute/v2.1/servers/67266541-b43e-414c-a744-9c32c6f53975/action" status: 202 len: 0 microversion: 2.1 time: 0.673657 Jul 16 21:09:58.550613 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 268/538] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 21:09:57 2022] POST /compute/v2.1/servers/67266541-b43e-414c-a744-9c32c6f53975/action => generated 0 bytes in 677 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 21:09:58.558097 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-f94e1d4b-9d13-4270-bac8-252c39af11bf tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:58.559170 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-f94e1d4b-9d13-4270-bac8-252c39af11bf tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: f98a9762-a67c-4e95-85b8-727c81bdb03a] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:09:58.568519 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-db2906e5-9f09-4744-9912-919a64f94191 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] 10.222.0.16 "GET /compute/v2.1/servers" status: 200 len: 1663 microversion: 2.1 time: 0.136994 Jul 16 21:09:58.569148 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f94e1d4b-9d13-4270-bac8-252c39af11bf tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:58.569544 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 271/539] 10.222.0.16 () {58 vars in 1124 bytes} [Sat Jul 16 21:09:58 2022] GET /compute/v2.1/servers => generated 1663 bytes in 139 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:09:58.571843 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f94e1d4b-9d13-4270-bac8-252c39af11bf tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:58.572852 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f94e1d4b-9d13-4270-bac8-252c39af11bf tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:58.573638 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f94e1d4b-9d13-4270-bac8-252c39af11bf tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:58.575306 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-41da4469-7d9d-40c0-a3cd-631e29910f43 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:58.576107 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-41da4469-7d9d-40c0-a3cd-631e29910f43 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] [instance: cf53f7cd-deae-4b34-bf8e-3afef98b2ec6] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:09:58.583405 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-41da4469-7d9d-40c0-a3cd-631e29910f43 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:58.583904 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-41da4469-7d9d-40c0-a3cd-631e29910f43 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:58.585019 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-41da4469-7d9d-40c0-a3cd-631e29910f43 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:58.585497 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-41da4469-7d9d-40c0-a3cd-631e29910f43 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:58.608773 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:09:58.674072 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-41da4469-7d9d-40c0-a3cd-631e29910f43 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ba60685a2001459cbfb531d95a6b296e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb6deae6b08e4ed8b15dc6edfe1d38d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:58.684830 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-41da4469-7d9d-40c0-a3cd-631e29910f43 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ba60685a2001459cbfb531d95a6b296e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb6deae6b08e4ed8b15dc6edfe1d38d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:58.686677 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-41da4469-7d9d-40c0-a3cd-631e29910f43 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ba60685a2001459cbfb531d95a6b296e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb6deae6b08e4ed8b15dc6edfe1d38d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:58.693950 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-41da4469-7d9d-40c0-a3cd-631e29910f43 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] 10.222.0.16 "GET /compute/v2.1/servers/cf53f7cd-deae-4b34-bf8e-3afef98b2ec6" status: 200 len: 1447 microversion: 2.75 time: 0.123156 Jul 16 21:09:58.694650 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 272/540] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:09:58 2022] GET /compute/v2.1/servers/cf53f7cd-deae-4b34-bf8e-3afef98b2ec6 => generated 1447 bytes in 125 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:09:58.708813 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-689452c4-4592-4824-aeab-36fd328608f9 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServersWithSpecificFlavorTestJSON-server-611763742", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "2089714046", "adminPass": "***", "networks": [{"uuid": "7ee2ff5a-751a-444f-a143-2ba55b7e64b7"}], "security_groups": [{"name": "tempest-securitygroup--1281138757"}], "key_name": "tempest-keypair-461056992", "user_data": "}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:09:58.917219 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.network.neutron [None req-689452c4-4592-4824-aeab-36fd328608f9 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] validate_networks() for [('7ee2ff5a-751a-444f-a143-2ba55b7e64b7', None, None, None, None, None)] {{(pid=138680) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 16 21:09:58.930500 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-f94e1d4b-9d13-4270-bac8-252c39af11bf tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b68dacc4b8463ba07fae6372131117', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55a19b7a4682457a9132f5fc80110712', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:58.939227 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-f94e1d4b-9d13-4270-bac8-252c39af11bf tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "GET /compute/v2.1/servers/f98a9762-a67c-4e95-85b8-727c81bdb03a" status: 200 len: 1312 microversion: 2.1 time: 0.387799 Jul 16 21:09:58.940866 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 269/541] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:09:58 2022] GET /compute/v2.1/servers/f98a9762-a67c-4e95-85b8-727c81bdb03a => generated 1312 bytes in 390 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:09:58.946073 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-fe258872-9aa0-46f6-b5e9-f7588af04803 tempest-ServerDiagnosticsNegativeTest-1844467799 tempest-ServerDiagnosticsNegativeTest-1844467799-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:58.946927 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-fe258872-9aa0-46f6-b5e9-f7588af04803 tempest-ServerDiagnosticsNegativeTest-1844467799 tempest-ServerDiagnosticsNegativeTest-1844467799-project] [instance: 1e4229cb-4ed9-48c3-b723-28619ae7045c] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:09:58.954304 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fe258872-9aa0-46f6-b5e9-f7588af04803 tempest-ServerDiagnosticsNegativeTest-1844467799 tempest-ServerDiagnosticsNegativeTest-1844467799-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:58.955126 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fe258872-9aa0-46f6-b5e9-f7588af04803 tempest-ServerDiagnosticsNegativeTest-1844467799 tempest-ServerDiagnosticsNegativeTest-1844467799-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:58.956189 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fe258872-9aa0-46f6-b5e9-f7588af04803 tempest-ServerDiagnosticsNegativeTest-1844467799 tempest-ServerDiagnosticsNegativeTest-1844467799-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:58.956700 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fe258872-9aa0-46f6-b5e9-f7588af04803 tempest-ServerDiagnosticsNegativeTest-1844467799 tempest-ServerDiagnosticsNegativeTest-1844467799-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:59.077741 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-fe258872-9aa0-46f6-b5e9-f7588af04803 tempest-ServerDiagnosticsNegativeTest-1844467799 tempest-ServerDiagnosticsNegativeTest-1844467799-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9c428093bcf445a6aea6ddb209e6e5cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5f9cdca4179c4875a4be57cf8a9f46a8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:59.086874 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-fe258872-9aa0-46f6-b5e9-f7588af04803 tempest-ServerDiagnosticsNegativeTest-1844467799 tempest-ServerDiagnosticsNegativeTest-1844467799-project] 10.222.0.16 "GET /compute/v2.1/servers/1e4229cb-4ed9-48c3-b723-28619ae7045c" status: 200 len: 1281 microversion: 2.1 time: 0.145662 Jul 16 21:09:59.095444 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 270/542] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:09:58 2022] GET /compute/v2.1/servers/1e4229cb-4ed9-48c3-b723-28619ae7045c => generated 1281 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:09:59.101723 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-89f6f323-b0d4-4d50-ac64-fd53bc294f8f tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:59.102334 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-89f6f323-b0d4-4d50-ac64-fd53bc294f8f tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] [instance: 67266541-b43e-414c-a744-9c32c6f53975] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:09:59.108622 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-89f6f323-b0d4-4d50-ac64-fd53bc294f8f tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:59.109077 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-89f6f323-b0d4-4d50-ac64-fd53bc294f8f tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:59.109989 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-89f6f323-b0d4-4d50-ac64-fd53bc294f8f tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:59.110403 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-89f6f323-b0d4-4d50-ac64-fd53bc294f8f tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:59.415564 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-89f6f323-b0d4-4d50-ac64-fd53bc294f8f tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0d72626f465b4f37ad3b18bb945d4000', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fc1c3463f9444119af9d18f85bdd703d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:59.423788 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-89f6f323-b0d4-4d50-ac64-fd53bc294f8f tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] 10.222.0.16 "GET /compute/v2.1/servers/67266541-b43e-414c-a744-9c32c6f53975" status: 200 len: 1476 microversion: 2.1 time: 0.325927 Jul 16 21:09:59.435218 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 271/543] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:09:59 2022] GET /compute/v2.1/servers/67266541-b43e-414c-a744-9c32c6f53975 => generated 1476 bytes in 338 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:09:59.440835 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-258ac1c5-2e99-458a-9c3d-ab39bb6cf271 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:59.441456 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-258ac1c5-2e99-458a-9c3d-ab39bb6cf271 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] [instance: 7d624313-0c12-4da2-a95c-3903efae7608] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:09:59.602372 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-258ac1c5-2e99-458a-9c3d-ab39bb6cf271 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'abeb7020019d41a18de42956d706a011', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '649babe54d104436af461f1ee4767001', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:09:59.614870 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-258ac1c5-2e99-458a-9c3d-ab39bb6cf271 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] 10.222.0.16 "GET /compute/v2.1/servers/7d624313-0c12-4da2-a95c-3903efae7608" status: 200 len: 1218 microversion: 2.1 time: 0.178238 Jul 16 21:09:59.624488 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 272/544] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:09:59 2022] GET /compute/v2.1/servers/7d624313-0c12-4da2-a95c-3903efae7608 => generated 1218 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:09:59.624488 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-81b97b94-dfe6-41d6-8055-a52661a3347a tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:09:59.625351 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-81b97b94-dfe6-41d6-8055-a52661a3347a tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] [instance: c40ba20d-eb88-4ee8-be8f-ecbbd3315c48] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:09:59.632938 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-81b97b94-dfe6-41d6-8055-a52661a3347a tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:59.634229 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-81b97b94-dfe6-41d6-8055-a52661a3347a tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:59.635192 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-81b97b94-dfe6-41d6-8055-a52661a3347a tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:59.635646 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-81b97b94-dfe6-41d6-8055-a52661a3347a tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:59.968653 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-689452c4-4592-4824-aeab-36fd328608f9 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Flavor limits 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 16 21:09:59.971774 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-689452c4-4592-4824-aeab-36fd328608f9 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Image limits 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 16 21:09:59.972565 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-689452c4-4592-4824-aeab-36fd328608f9 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Flavor pref 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 16 21:09:59.973221 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-689452c4-4592-4824-aeab-36fd328608f9 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Image pref 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 16 21:09:59.973830 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-689452c4-4592-4824-aeab-36fd328608f9 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 16 21:09:59.984108 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-689452c4-4592-4824-aeab-36fd328608f9 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:59.985881 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-689452c4-4592-4824-aeab-36fd328608f9 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:09:59.987078 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-689452c4-4592-4824-aeab-36fd328608f9 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:09:59.987729 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-689452c4-4592-4824-aeab-36fd328608f9 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:00.001451 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:10:00.017638 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-689452c4-4592-4824-aeab-36fd328608f9 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Getting quotas for project d0ee02e4c8b448f2b77798ae504129b9. Resources: {'ram', 'instances', 'cores'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:10:00.023985 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-689452c4-4592-4824-aeab-36fd328608f9 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Getting quotas for user ef9cbce2b2e64665b7f50767342dcb46 and project d0ee02e4c8b448f2b77798ae504129b9. Resources: {'ram', 'instances', 'cores'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:10:00.042275 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-689452c4-4592-4824-aeab-36fd328608f9 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Going to run 1 instances... {{(pid=138680) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 16 21:10:00.046492 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-689452c4-4592-4824-aeab-36fd328608f9 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] [instance: 8872e8cf-593e-4a7a-940d-e0c8159d4ad7] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None), BlockDeviceMapping(attachment_id=,boot_index=-1,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='blank',tag=None,updated_at=,uuid=,volume_id=None,volume_size=1,volume_type=None)] {{(pid=138680) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 16 21:10:00.046806 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-81b97b94-dfe6-41d6-8055-a52661a3347a tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a69f9dee03d43beaafa966216b9e289', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9b769bf8f3ea4e368bf68de1df6538f9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:00.058486 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-81b97b94-dfe6-41d6-8055-a52661a3347a tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] 10.222.0.16 "GET /compute/v2.1/servers/c40ba20d-eb88-4ee8-be8f-ecbbd3315c48" status: 200 len: 1503 microversion: 2.1 time: 0.440243 Jul 16 21:10:00.059695 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 273/545] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:09:59 2022] GET /compute/v2.1/servers/c40ba20d-eb88-4ee8-be8f-ecbbd3315c48 => generated 1503 bytes in 443 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:10:00.299701 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-689452c4-4592-4824-aeab-36fd328608f9 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 402 microversion: 2.1 time: 1.595418 Jul 16 21:10:00.307270 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 273/546] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 21:09:58 2022] POST /compute/v2.1/servers => generated 402 bytes in 1611 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 16 21:10:00.326944 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-b8be0a2b-19df-4511-aba8-ff3c5835782d tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:00.333816 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-b8be0a2b-19df-4511-aba8-ff3c5835782d tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] [instance: cf53f7cd-deae-4b34-bf8e-3afef98b2ec6] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:00.350580 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b8be0a2b-19df-4511-aba8-ff3c5835782d tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:00.351275 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b8be0a2b-19df-4511-aba8-ff3c5835782d tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:00.352461 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b8be0a2b-19df-4511-aba8-ff3c5835782d tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:00.353704 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b8be0a2b-19df-4511-aba8-ff3c5835782d tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:00.356842 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-3d24bb1b-ee41-4fe8-979c-53ffa8da4d51 tempest-KeyPairsV210TestJSON-621372744 tempest-KeyPairsV210TestJSON-621372744-project-admin] Action: 'create', calling method: .version_select at 0x7fdd9d5a9d30>, body: {"keypair": {"name": "tempest-keypair-151248390", "type": "ssh", "user_id": "587057ad0ece48d2a6ae419b7535ef64", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBP6JnL8YE5lb/fXY2LuBCI0gOYqP/l+75+9iQJv7Hfq8Koe46BQkeewLJNgrZYUmAep//3w8Lt0Qsc0241G6FaSXH9oi07WE88ZZAavPFL0wZTuK0RYqA798F67+65Bkrg=="}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:10:00.399936 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-3d24bb1b-ee41-4fe8-979c-53ffa8da4d51 tempest-KeyPairsV210TestJSON-621372744 tempest-KeyPairsV210TestJSON-621372744-project-admin] Getting quotas for project 8f20749824c24f70a981dad14a0c6b9a. Resources: {'key_pairs'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:10:00.405579 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-3d24bb1b-ee41-4fe8-979c-53ffa8da4d51 tempest-KeyPairsV210TestJSON-621372744 tempest-KeyPairsV210TestJSON-621372744-project-admin] Getting quotas for user 87ba7aa1510244a5a2543b78c9691a97 and project 8f20749824c24f70a981dad14a0c6b9a. Resources: {'key_pairs'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:10:00.515552 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-3d24bb1b-ee41-4fe8-979c-53ffa8da4d51 tempest-KeyPairsV210TestJSON-621372744 tempest-KeyPairsV210TestJSON-621372744-project-admin] 10.222.0.16 "POST /compute/v2.1/os-keypairs" status: 201 len: 400 microversion: 2.10 time: 0.454126 Jul 16 21:10:00.518791 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 274/547] 10.222.0.16 () {62 vars in 1199 bytes} [Sat Jul 16 21:10:00 2022] POST /compute/v2.1/os-keypairs => generated 400 bytes in 458 msecs (HTTP/1.1 201) 9 headers in 364 bytes (1 switches on core 0) Jul 16 21:10:00.530472 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-6006d1bd-f6b1-4ca4-bfd4-b1632daa8238 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:00.533027 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-6006d1bd-f6b1-4ca4-bfd4-b1632daa8238 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: f98a9762-a67c-4e95-85b8-727c81bdb03a] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:00.546418 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6006d1bd-f6b1-4ca4-bfd4-b1632daa8238 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:00.546888 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6006d1bd-f6b1-4ca4-bfd4-b1632daa8238 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:00.553525 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6006d1bd-f6b1-4ca4-bfd4-b1632daa8238 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:00.563845 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6006d1bd-f6b1-4ca4-bfd4-b1632daa8238 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:00.679229 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-b8be0a2b-19df-4511-aba8-ff3c5835782d tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ba60685a2001459cbfb531d95a6b296e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb6deae6b08e4ed8b15dc6edfe1d38d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:00.688620 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-b8be0a2b-19df-4511-aba8-ff3c5835782d tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ba60685a2001459cbfb531d95a6b296e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb6deae6b08e4ed8b15dc6edfe1d38d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:00.690510 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-b8be0a2b-19df-4511-aba8-ff3c5835782d tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ba60685a2001459cbfb531d95a6b296e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb6deae6b08e4ed8b15dc6edfe1d38d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:00.698653 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-b8be0a2b-19df-4511-aba8-ff3c5835782d tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] 10.222.0.16 "GET /compute/v2.1/servers/cf53f7cd-deae-4b34-bf8e-3afef98b2ec6" status: 200 len: 1447 microversion: 2.75 time: 0.387505 Jul 16 21:10:00.700105 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 274/548] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:10:00 2022] GET /compute/v2.1/servers/cf53f7cd-deae-4b34-bf8e-3afef98b2ec6 => generated 1447 bytes in 392 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:10:00.705907 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-b4b399ec-0258-4294-b473-83ad9ac71520 tempest-ServerDiagnosticsNegativeTest-1844467799 tempest-ServerDiagnosticsNegativeTest-1844467799-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:00.706525 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-b4b399ec-0258-4294-b473-83ad9ac71520 tempest-ServerDiagnosticsNegativeTest-1844467799 tempest-ServerDiagnosticsNegativeTest-1844467799-project] [instance: 1e4229cb-4ed9-48c3-b723-28619ae7045c] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:00.714363 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b4b399ec-0258-4294-b473-83ad9ac71520 tempest-ServerDiagnosticsNegativeTest-1844467799 tempest-ServerDiagnosticsNegativeTest-1844467799-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:00.714822 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b4b399ec-0258-4294-b473-83ad9ac71520 tempest-ServerDiagnosticsNegativeTest-1844467799 tempest-ServerDiagnosticsNegativeTest-1844467799-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:00.715699 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b4b399ec-0258-4294-b473-83ad9ac71520 tempest-ServerDiagnosticsNegativeTest-1844467799 tempest-ServerDiagnosticsNegativeTest-1844467799-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:00.716290 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b4b399ec-0258-4294-b473-83ad9ac71520 tempest-ServerDiagnosticsNegativeTest-1844467799 tempest-ServerDiagnosticsNegativeTest-1844467799-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:01.136393 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-6006d1bd-f6b1-4ca4-bfd4-b1632daa8238 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b68dacc4b8463ba07fae6372131117', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55a19b7a4682457a9132f5fc80110712', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:01.152638 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-6006d1bd-f6b1-4ca4-bfd4-b1632daa8238 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "GET /compute/v2.1/servers/f98a9762-a67c-4e95-85b8-727c81bdb03a" status: 200 len: 1312 microversion: 2.1 time: 0.631327 Jul 16 21:10:01.154810 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 275/549] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:10:00 2022] GET /compute/v2.1/servers/f98a9762-a67c-4e95-85b8-727c81bdb03a => generated 1312 bytes in 635 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:10:01.165227 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-78d021b9-3afc-4ced-bd8a-c40e8992ba0a tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:01.165919 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-78d021b9-3afc-4ced-bd8a-c40e8992ba0a tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] [instance: 8872e8cf-593e-4a7a-940d-e0c8159d4ad7] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:01.389343 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-b4b399ec-0258-4294-b473-83ad9ac71520 tempest-ServerDiagnosticsNegativeTest-1844467799 tempest-ServerDiagnosticsNegativeTest-1844467799-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9c428093bcf445a6aea6ddb209e6e5cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5f9cdca4179c4875a4be57cf8a9f46a8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:01.419411 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-b4b399ec-0258-4294-b473-83ad9ac71520 tempest-ServerDiagnosticsNegativeTest-1844467799 tempest-ServerDiagnosticsNegativeTest-1844467799-project] 10.222.0.16 "GET /compute/v2.1/servers/1e4229cb-4ed9-48c3-b723-28619ae7045c" status: 200 len: 1323 microversion: 2.1 time: 0.714460 Jul 16 21:10:01.420365 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 275/550] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:10:00 2022] GET /compute/v2.1/servers/1e4229cb-4ed9-48c3-b723-28619ae7045c => generated 1323 bytes in 720 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:10:01.432193 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-04551654-4e90-4b95-af60-3dff3bcf7307 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:01.432877 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-04551654-4e90-4b95-af60-3dff3bcf7307 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] [instance: 67266541-b43e-414c-a744-9c32c6f53975] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:01.436015 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-78d021b9-3afc-4ced-bd8a-c40e8992ba0a tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ef9cbce2b2e64665b7f50767342dcb46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0ee02e4c8b448f2b77798ae504129b9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:01.444373 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-04551654-4e90-4b95-af60-3dff3bcf7307 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:01.444824 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-04551654-4e90-4b95-af60-3dff3bcf7307 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:01.445620 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-04551654-4e90-4b95-af60-3dff3bcf7307 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:01.446055 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-04551654-4e90-4b95-af60-3dff3bcf7307 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:01.448312 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-78d021b9-3afc-4ced-bd8a-c40e8992ba0a tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] 10.222.0.16 "GET /compute/v2.1/servers/8872e8cf-593e-4a7a-940d-e0c8159d4ad7" status: 200 len: 1263 microversion: 2.1 time: 0.292179 Jul 16 21:10:01.448906 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 276/551] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:10:01 2022] GET /compute/v2.1/servers/8872e8cf-593e-4a7a-940d-e0c8159d4ad7 => generated 1263 bytes in 293 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:10:01.457092 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-ff0ed2c0-7192-4572-b7f8-57fc53f72f4a tempest-KeyPairsV210TestJSON-621372744 tempest-KeyPairsV210TestJSON-621372744-project-admin] Action: 'create', calling method: .version_select at 0x7fdd9d52ee50>, body: {"keypair": {"name": "tempest-keypair-292452046", "type": "ssh", "user_id": "587057ad0ece48d2a6ae419b7535ef64", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBIssK4ceBqZpbgtJHZJV5M66i17h2iAmNs4q6Z5VyVMOyi99XdeALQ0lSdId+mmppyRcO4M9aALjOUs1WMB7F2JG/FLVyue+bialuwJzIi6Bg7q45RBGbAvL9yVW1dpMjg=="}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:10:01.489868 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-ff0ed2c0-7192-4572-b7f8-57fc53f72f4a tempest-KeyPairsV210TestJSON-621372744 tempest-KeyPairsV210TestJSON-621372744-project-admin] Getting quotas for project 8f20749824c24f70a981dad14a0c6b9a. Resources: {'key_pairs'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:10:01.503185 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-ff0ed2c0-7192-4572-b7f8-57fc53f72f4a tempest-KeyPairsV210TestJSON-621372744 tempest-KeyPairsV210TestJSON-621372744-project-admin] Getting quotas for user 87ba7aa1510244a5a2543b78c9691a97 and project 8f20749824c24f70a981dad14a0c6b9a. Resources: {'key_pairs'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:10:01.624247 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-ff0ed2c0-7192-4572-b7f8-57fc53f72f4a tempest-KeyPairsV210TestJSON-621372744 tempest-KeyPairsV210TestJSON-621372744-project-admin] 10.222.0.16 "POST /compute/v2.1/os-keypairs" status: 201 len: 400 microversion: 2.10 time: 0.174031 Jul 16 21:10:01.628710 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 277/552] 10.222.0.16 () {62 vars in 1199 bytes} [Sat Jul 16 21:10:01 2022] POST /compute/v2.1/os-keypairs => generated 400 bytes in 179 msecs (HTTP/1.1 201) 9 headers in 364 bytes (1 switches on core 0) Jul 16 21:10:01.635082 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-72f283ac-78b3-4606-972f-8b9882a00a64 req-def2b3d6-ffba-4ad7-8d78-f25bd6c6a72e service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "f98a9762-a67c-4e95-85b8-727c81bdb03a", "tag": "b0aae229-268b-497f-8da0-36ccb43a67bd"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:10:01.649056 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-72f283ac-78b3-4606-972f-8b9882a00a64 req-def2b3d6-ffba-4ad7-8d78-f25bd6c6a72e service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:01.649670 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-72f283ac-78b3-4606-972f-8b9882a00a64 req-def2b3d6-ffba-4ad7-8d78-f25bd6c6a72e service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:01.662811 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-72f283ac-78b3-4606-972f-8b9882a00a64 req-def2b3d6-ffba-4ad7-8d78-f25bd6c6a72e service nova] Creating event network-changed:b0aae229-268b-497f-8da0-36ccb43a67bd for instance f98a9762-a67c-4e95-85b8-727c81bdb03a on nch1-master-1 Jul 16 21:10:01.668677 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-72f283ac-78b3-4606-972f-8b9882a00a64 req-def2b3d6-ffba-4ad7-8d78-f25bd6c6a72e service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.037600 Jul 16 21:10:01.668677 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 278/553] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:10:01 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:10:01.674931 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-84fdb8ea-5922-4897-88fd-fb2617ef5984 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:01.675718 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-84fdb8ea-5922-4897-88fd-fb2617ef5984 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] [instance: 7d624313-0c12-4da2-a95c-3903efae7608] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:01.682540 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-84fdb8ea-5922-4897-88fd-fb2617ef5984 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:01.682984 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-84fdb8ea-5922-4897-88fd-fb2617ef5984 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:01.684828 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-84fdb8ea-5922-4897-88fd-fb2617ef5984 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:01.685106 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-84fdb8ea-5922-4897-88fd-fb2617ef5984 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:01.837386 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-84fdb8ea-5922-4897-88fd-fb2617ef5984 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'abeb7020019d41a18de42956d706a011', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '649babe54d104436af461f1ee4767001', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:01.862556 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-84fdb8ea-5922-4897-88fd-fb2617ef5984 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] 10.222.0.16 "GET /compute/v2.1/servers/7d624313-0c12-4da2-a95c-3903efae7608" status: 200 len: 1270 microversion: 2.1 time: 0.192341 Jul 16 21:10:01.876976 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 279/554] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:10:01 2022] GET /compute/v2.1/servers/7d624313-0c12-4da2-a95c-3903efae7608 => generated 1270 bytes in 203 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:10:01.892063 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-3de67e69-5ba7-460a-bb23-8c8336e0be50 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:01.896472 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-3de67e69-5ba7-460a-bb23-8c8336e0be50 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] [instance: c40ba20d-eb88-4ee8-be8f-ecbbd3315c48] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:01.904750 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3de67e69-5ba7-460a-bb23-8c8336e0be50 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:01.905510 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3de67e69-5ba7-460a-bb23-8c8336e0be50 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:01.906126 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3de67e69-5ba7-460a-bb23-8c8336e0be50 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:01.906713 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3de67e69-5ba7-460a-bb23-8c8336e0be50 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:01.908735 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-04551654-4e90-4b95-af60-3dff3bcf7307 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0d72626f465b4f37ad3b18bb945d4000', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fc1c3463f9444119af9d18f85bdd703d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:01.921403 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-04551654-4e90-4b95-af60-3dff3bcf7307 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] 10.222.0.16 "GET /compute/v2.1/servers/67266541-b43e-414c-a744-9c32c6f53975" status: 200 len: 1459 microversion: 2.1 time: 0.499077 Jul 16 21:10:01.923160 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 276/555] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:10:01 2022] GET /compute/v2.1/servers/67266541-b43e-414c-a744-9c32c6f53975 => generated 1459 bytes in 502 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:10:01.928662 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-6b487c04-f3b1-41e3-9191-48695302d91a tempest-KeyPairsV210TestJSON-621372744 tempest-KeyPairsV210TestJSON-621372744-project-admin] Calling method '.version_select at 0x7fdd9d2a5160>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:01.943559 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-6b487c04-f3b1-41e3-9191-48695302d91a tempest-KeyPairsV210TestJSON-621372744 tempest-KeyPairsV210TestJSON-621372744-project-admin] 10.222.0.16 "GET /compute/v2.1/os-keypairs/tempest-keypair-151248390?user_id=587057ad0ece48d2a6ae419b7535ef64" status: 200 len: 511 microversion: 2.10 time: 0.019092 Jul 16 21:10:01.945087 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 277/556] 10.222.0.16 () {60 vars in 1336 bytes} [Sat Jul 16 21:10:01 2022] GET /compute/v2.1/os-keypairs/tempest-keypair-151248390?user_id=587057ad0ece48d2a6ae419b7535ef64 => generated 511 bytes in 21 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 16 21:10:01.952023 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-4d00c00a-af07-4bed-9888-394903c04400 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:01.954073 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-4d00c00a-af07-4bed-9888-394903c04400 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] [instance: cf53f7cd-deae-4b34-bf8e-3afef98b2ec6] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:01.960940 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4d00c00a-af07-4bed-9888-394903c04400 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:01.961391 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4d00c00a-af07-4bed-9888-394903c04400 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:01.962167 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4d00c00a-af07-4bed-9888-394903c04400 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:01.962796 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4d00c00a-af07-4bed-9888-394903c04400 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:02.149213 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-4d00c00a-af07-4bed-9888-394903c04400 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ba60685a2001459cbfb531d95a6b296e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb6deae6b08e4ed8b15dc6edfe1d38d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:02.158502 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-4d00c00a-af07-4bed-9888-394903c04400 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ba60685a2001459cbfb531d95a6b296e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb6deae6b08e4ed8b15dc6edfe1d38d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:02.160052 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-4d00c00a-af07-4bed-9888-394903c04400 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ba60685a2001459cbfb531d95a6b296e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb6deae6b08e4ed8b15dc6edfe1d38d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:02.173163 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-4d00c00a-af07-4bed-9888-394903c04400 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] 10.222.0.16 "GET /compute/v2.1/servers/cf53f7cd-deae-4b34-bf8e-3afef98b2ec6" status: 200 len: 1447 microversion: 2.75 time: 0.226002 Jul 16 21:10:02.173760 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 278/557] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:10:01 2022] GET /compute/v2.1/servers/cf53f7cd-deae-4b34-bf8e-3afef98b2ec6 => generated 1447 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:10:02.178366 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-f471b0c8-61d1-4054-b3e3-72a63b23561e tempest-LiveMigrationNegativeTest-1294767845 tempest-LiveMigrationNegativeTest-1294767845-project-admin] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:02.179283 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-f471b0c8-61d1-4054-b3e3-72a63b23561e tempest-LiveMigrationNegativeTest-1294767845 tempest-LiveMigrationNegativeTest-1294767845-project-admin] [instance: 67266541-b43e-414c-a744-9c32c6f53975] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:02.185949 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f471b0c8-61d1-4054-b3e3-72a63b23561e tempest-LiveMigrationNegativeTest-1294767845 tempest-LiveMigrationNegativeTest-1294767845-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:02.186405 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f471b0c8-61d1-4054-b3e3-72a63b23561e tempest-LiveMigrationNegativeTest-1294767845 tempest-LiveMigrationNegativeTest-1294767845-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:02.187230 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f471b0c8-61d1-4054-b3e3-72a63b23561e tempest-LiveMigrationNegativeTest-1294767845 tempest-LiveMigrationNegativeTest-1294767845-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:02.188346 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f471b0c8-61d1-4054-b3e3-72a63b23561e tempest-LiveMigrationNegativeTest-1294767845 tempest-LiveMigrationNegativeTest-1294767845-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:02.338896 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-3de67e69-5ba7-460a-bb23-8c8336e0be50 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a69f9dee03d43beaafa966216b9e289', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9b769bf8f3ea4e368bf68de1df6538f9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:02.349697 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-3de67e69-5ba7-460a-bb23-8c8336e0be50 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] 10.222.0.16 "GET /compute/v2.1/servers/c40ba20d-eb88-4ee8-be8f-ecbbd3315c48" status: 200 len: 1503 microversion: 2.1 time: 0.475259 Jul 16 21:10:02.359866 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 280/558] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:10:01 2022] GET /compute/v2.1/servers/c40ba20d-eb88-4ee8-be8f-ecbbd3315c48 => generated 1503 bytes in 486 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:10:02.364907 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-c5bc2fdf-7700-4071-b7af-1d25f7f41e93 tempest-KeyPairsV210TestJSON-621372744 tempest-KeyPairsV210TestJSON-621372744-project-admin] Action: 'create', calling method: .version_select at 0x7fdd9d52e4c0>, body: {"keypair": {"name": "tempest-KeyPairsV210TestJSON-keypair-1964472426", "type": "ssh", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBA2cMuQRBG0TmGtPzV7ebLa6F2wAj+QJix6+14fhamt3UsDTsiuzN2X5V7+UYw3M2Mb2In4jf4BuJ6Z/nr4R3nMcDh8b+9jmGIHItkX/wEU+LEdcfA5Iyo5CnZVqLKrCJg=="}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:10:02.383680 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-c5bc2fdf-7700-4071-b7af-1d25f7f41e93 tempest-KeyPairsV210TestJSON-621372744 tempest-KeyPairsV210TestJSON-621372744-project-admin] Getting quotas for project 8f20749824c24f70a981dad14a0c6b9a. Resources: {'key_pairs'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:10:02.400234 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-c5bc2fdf-7700-4071-b7af-1d25f7f41e93 tempest-KeyPairsV210TestJSON-621372744 tempest-KeyPairsV210TestJSON-621372744-project-admin] Getting quotas for user 87ba7aa1510244a5a2543b78c9691a97 and project 8f20749824c24f70a981dad14a0c6b9a. Resources: {'key_pairs'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:10:02.527861 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-c5bc2fdf-7700-4071-b7af-1d25f7f41e93 tempest-KeyPairsV210TestJSON-621372744 tempest-KeyPairsV210TestJSON-621372744-project-admin] 10.222.0.16 "POST /compute/v2.1/os-keypairs" status: 201 len: 422 microversion: 2.10 time: 0.166272 Jul 16 21:10:02.529647 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 281/559] 10.222.0.16 () {62 vars in 1199 bytes} [Sat Jul 16 21:10:02 2022] POST /compute/v2.1/os-keypairs => generated 422 bytes in 169 msecs (HTTP/1.1 201) 9 headers in 364 bytes (1 switches on core 0) Jul 16 21:10:02.539587 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-af69e67d-ea7a-4e92-b059-89c63d0a1004 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:02.540354 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-af69e67d-ea7a-4e92-b059-89c63d0a1004 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: f98a9762-a67c-4e95-85b8-727c81bdb03a] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:02.557802 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-af69e67d-ea7a-4e92-b059-89c63d0a1004 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:02.560456 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-af69e67d-ea7a-4e92-b059-89c63d0a1004 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:02.561448 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-af69e67d-ea7a-4e92-b059-89c63d0a1004 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:02.564687 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-af69e67d-ea7a-4e92-b059-89c63d0a1004 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:03.890086 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-af69e67d-ea7a-4e92-b059-89c63d0a1004 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b68dacc4b8463ba07fae6372131117', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55a19b7a4682457a9132f5fc80110712', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:03.901228 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-af69e67d-ea7a-4e92-b059-89c63d0a1004 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "GET /compute/v2.1/servers/f98a9762-a67c-4e95-85b8-727c81bdb03a" status: 200 len: 1312 microversion: 2.1 time: 1.370220 Jul 16 21:10:03.902472 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 282/560] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:10:02 2022] GET /compute/v2.1/servers/f98a9762-a67c-4e95-85b8-727c81bdb03a => generated 1312 bytes in 1372 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:10:03.911118 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-3e809fc0-455e-488a-8c36-2605ac7b09a7 tempest-ServerDiagnosticsNegativeTest-1844467799 tempest-ServerDiagnosticsNegativeTest-1844467799-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:03.912274 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-3e809fc0-455e-488a-8c36-2605ac7b09a7 tempest-ServerDiagnosticsNegativeTest-1844467799 tempest-ServerDiagnosticsNegativeTest-1844467799-project] [instance: 1e4229cb-4ed9-48c3-b723-28619ae7045c] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:03.922509 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3e809fc0-455e-488a-8c36-2605ac7b09a7 tempest-ServerDiagnosticsNegativeTest-1844467799 tempest-ServerDiagnosticsNegativeTest-1844467799-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:03.923666 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3e809fc0-455e-488a-8c36-2605ac7b09a7 tempest-ServerDiagnosticsNegativeTest-1844467799 tempest-ServerDiagnosticsNegativeTest-1844467799-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:03.925407 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3e809fc0-455e-488a-8c36-2605ac7b09a7 tempest-ServerDiagnosticsNegativeTest-1844467799 tempest-ServerDiagnosticsNegativeTest-1844467799-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:03.926246 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3e809fc0-455e-488a-8c36-2605ac7b09a7 tempest-ServerDiagnosticsNegativeTest-1844467799 tempest-ServerDiagnosticsNegativeTest-1844467799-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:04.501753 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-f471b0c8-61d1-4054-b3e3-72a63b23561e tempest-LiveMigrationNegativeTest-1294767845 tempest-LiveMigrationNegativeTest-1294767845-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/67266541-b43e-414c-a744-9c32c6f53975" status: 200 len: 1610 microversion: 2.1 time: 2.322989 Jul 16 21:10:04.515055 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 279/561] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:10:02 2022] GET /compute/v2.1/servers/67266541-b43e-414c-a744-9c32c6f53975 => generated 1610 bytes in 2340 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:10:04.532630 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-0620dc2a-f301-4ca7-8d8c-ae7bd57f0163 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:04.533923 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-0620dc2a-f301-4ca7-8d8c-ae7bd57f0163 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] [instance: 8872e8cf-593e-4a7a-940d-e0c8159d4ad7] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:04.551042 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0620dc2a-f301-4ca7-8d8c-ae7bd57f0163 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:04.552042 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0620dc2a-f301-4ca7-8d8c-ae7bd57f0163 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:04.555928 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0620dc2a-f301-4ca7-8d8c-ae7bd57f0163 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:04.557388 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0620dc2a-f301-4ca7-8d8c-ae7bd57f0163 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:04.984755 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-3e809fc0-455e-488a-8c36-2605ac7b09a7 tempest-ServerDiagnosticsNegativeTest-1844467799 tempest-ServerDiagnosticsNegativeTest-1844467799-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9c428093bcf445a6aea6ddb209e6e5cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5f9cdca4179c4875a4be57cf8a9f46a8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:04.991078 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-0620dc2a-f301-4ca7-8d8c-ae7bd57f0163 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ef9cbce2b2e64665b7f50767342dcb46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0ee02e4c8b448f2b77798ae504129b9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:04.994501 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-3e809fc0-455e-488a-8c36-2605ac7b09a7 tempest-ServerDiagnosticsNegativeTest-1844467799 tempest-ServerDiagnosticsNegativeTest-1844467799-project] 10.222.0.16 "GET /compute/v2.1/servers/1e4229cb-4ed9-48c3-b723-28619ae7045c" status: 200 len: 1323 microversion: 2.1 time: 1.088132 Jul 16 21:10:05.000628 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 283/562] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:10:03 2022] GET /compute/v2.1/servers/1e4229cb-4ed9-48c3-b723-28619ae7045c => generated 1323 bytes in 1094 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:10:05.014676 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-0620dc2a-f301-4ca7-8d8c-ae7bd57f0163 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] 10.222.0.16 "GET /compute/v2.1/servers/8872e8cf-593e-4a7a-940d-e0c8159d4ad7" status: 200 len: 1333 microversion: 2.1 time: 0.490613 Jul 16 21:10:05.020632 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 280/563] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:10:04 2022] GET /compute/v2.1/servers/8872e8cf-593e-4a7a-940d-e0c8159d4ad7 => generated 1333 bytes in 500 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:10:05.026164 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-1e55b115-af1f-4da5-89fd-6d7409ef9368 req-11bcabc3-9cee-42f3-ae22-094a18688889 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "c40ba20d-eb88-4ee8-be8f-ecbbd3315c48", "name": "network-vif-unplugged", "status": "completed", "tag": "a2fac50b-598b-43e0-8e39-546a5c1b4490"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:10:05.027205 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-baf5f59e-6ccd-4f57-80c2-e69e3b7c88c9 tempest-KeyPairsV210TestJSON-621372744 tempest-KeyPairsV210TestJSON-621372744-project-admin] Calling method '.version_select at 0x7fdd9d5eb700>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:05.034409 ncdv-master-1 devstack@n-api.service[138681]: WARNING oslo_db.sqlalchemy.utils [None req-baf5f59e-6ccd-4f57-80c2-e69e3b7c88c9 tempest-KeyPairsV210TestJSON-621372744 tempest-KeyPairsV210TestJSON-621372744-project-admin] Unique keys not in sort_keys. The sorting order may be unstable. Jul 16 21:10:05.058416 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-1e55b115-af1f-4da5-89fd-6d7409ef9368 req-11bcabc3-9cee-42f3-ae22-094a18688889 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:05.060787 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-baf5f59e-6ccd-4f57-80c2-e69e3b7c88c9 tempest-KeyPairsV210TestJSON-621372744 tempest-KeyPairsV210TestJSON-621372744-project-admin] 10.222.0.16 "GET /compute/v2.1/os-keypairs?user_id=587057ad0ece48d2a6ae419b7535ef64" status: 200 len: 724 microversion: 2.10 time: 0.058113 Jul 16 21:10:05.064654 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-1e55b115-af1f-4da5-89fd-6d7409ef9368 req-11bcabc3-9cee-42f3-ae22-094a18688889 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:05.078759 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 284/564] 10.222.0.16 () {60 vars in 1258 bytes} [Sat Jul 16 21:10:05 2022] GET /compute/v2.1/os-keypairs?user_id=587057ad0ece48d2a6ae419b7535ef64 => generated 724 bytes in 60 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 16 21:10:05.084351 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-8b69cc9b-0885-4833-bddf-be465725e0b9 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:05.084964 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-8b69cc9b-0885-4833-bddf-be465725e0b9 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] [instance: 7d624313-0c12-4da2-a95c-3903efae7608] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:05.101147 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-1e55b115-af1f-4da5-89fd-6d7409ef9368 req-11bcabc3-9cee-42f3-ae22-094a18688889 service nova] Creating event network-vif-unplugged:a2fac50b-598b-43e0-8e39-546a5c1b4490 for instance c40ba20d-eb88-4ee8-be8f-ecbbd3315c48 on nch1-master-1 Jul 16 21:10:05.103506 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8b69cc9b-0885-4833-bddf-be465725e0b9 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:05.103967 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8b69cc9b-0885-4833-bddf-be465725e0b9 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:05.104747 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8b69cc9b-0885-4833-bddf-be465725e0b9 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:05.105187 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8b69cc9b-0885-4833-bddf-be465725e0b9 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:05.117028 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-1e55b115-af1f-4da5-89fd-6d7409ef9368 req-11bcabc3-9cee-42f3-ae22-094a18688889 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.099635 Jul 16 21:10:05.119923 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 281/565] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:10:05 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:10:05.134062 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-9993a803-498d-4853-a68f-bfee29a34e26 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:05.137234 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-9993a803-498d-4853-a68f-bfee29a34e26 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] [instance: cf53f7cd-deae-4b34-bf8e-3afef98b2ec6] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:05.153967 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9993a803-498d-4853-a68f-bfee29a34e26 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:05.158560 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9993a803-498d-4853-a68f-bfee29a34e26 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:05.164374 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9993a803-498d-4853-a68f-bfee29a34e26 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:05.165070 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9993a803-498d-4853-a68f-bfee29a34e26 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:05.411467 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-8b69cc9b-0885-4833-bddf-be465725e0b9 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'abeb7020019d41a18de42956d706a011', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '649babe54d104436af461f1ee4767001', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:05.420961 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-8b69cc9b-0885-4833-bddf-be465725e0b9 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] 10.222.0.16 "GET /compute/v2.1/servers/7d624313-0c12-4da2-a95c-3903efae7608" status: 200 len: 1276 microversion: 2.1 time: 0.340495 Jul 16 21:10:05.421514 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 285/566] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:10:05 2022] GET /compute/v2.1/servers/7d624313-0c12-4da2-a95c-3903efae7608 => generated 1276 bytes in 342 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:10:05.444831 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-040b8fbf-b330-46cc-9858-ee70ba751b09 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:05.445467 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-040b8fbf-b330-46cc-9858-ee70ba751b09 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] [instance: c40ba20d-eb88-4ee8-be8f-ecbbd3315c48] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:05.449896 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-9993a803-498d-4853-a68f-bfee29a34e26 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ba60685a2001459cbfb531d95a6b296e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb6deae6b08e4ed8b15dc6edfe1d38d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:05.462165 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-040b8fbf-b330-46cc-9858-ee70ba751b09 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:05.462924 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-9993a803-498d-4853-a68f-bfee29a34e26 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ba60685a2001459cbfb531d95a6b296e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb6deae6b08e4ed8b15dc6edfe1d38d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:05.465382 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-040b8fbf-b330-46cc-9858-ee70ba751b09 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:05.466470 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-040b8fbf-b330-46cc-9858-ee70ba751b09 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:05.466919 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-040b8fbf-b330-46cc-9858-ee70ba751b09 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:05.467972 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-9993a803-498d-4853-a68f-bfee29a34e26 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ba60685a2001459cbfb531d95a6b296e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb6deae6b08e4ed8b15dc6edfe1d38d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:05.480849 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-9993a803-498d-4853-a68f-bfee29a34e26 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] 10.222.0.16 "GET /compute/v2.1/servers/cf53f7cd-deae-4b34-bf8e-3afef98b2ec6" status: 200 len: 1447 microversion: 2.75 time: 0.358159 Jul 16 21:10:05.486730 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 282/567] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:10:05 2022] GET /compute/v2.1/servers/cf53f7cd-deae-4b34-bf8e-3afef98b2ec6 => generated 1447 bytes in 366 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:10:05.503451 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-aff47ca1-72ef-410f-b26f-98d39517fe41 tempest-LiveMigrationNegativeTest-1294767845 tempest-LiveMigrationNegativeTest-1294767845-project-admin] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:05.503451 ncdv-master-1 devstack@n-api.service[138680]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-services:list": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:10:05.503451 ncdv-master-1 devstack@n-api.service[138680]: warnings.warn(msg) Jul 16 21:10:05.504705 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-aff47ca1-72ef-410f-b26f-98d39517fe41 tempest-LiveMigrationNegativeTest-1294767845 tempest-LiveMigrationNegativeTest-1294767845-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:05.504705 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-aff47ca1-72ef-410f-b26f-98d39517fe41 tempest-LiveMigrationNegativeTest-1294767845 tempest-LiveMigrationNegativeTest-1294767845-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:05.505448 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-aff47ca1-72ef-410f-b26f-98d39517fe41 tempest-LiveMigrationNegativeTest-1294767845 tempest-LiveMigrationNegativeTest-1294767845-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:05.505893 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-aff47ca1-72ef-410f-b26f-98d39517fe41 tempest-LiveMigrationNegativeTest-1294767845 tempest-LiveMigrationNegativeTest-1294767845-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:05.569543 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-aff47ca1-72ef-410f-b26f-98d39517fe41 tempest-LiveMigrationNegativeTest-1294767845 tempest-LiveMigrationNegativeTest-1294767845-project-admin] 10.222.0.16 "GET /compute/v2.1/os-services?binary=nova-compute" status: 200 len: 380 microversion: 2.1 time: 0.081403 Jul 16 21:10:05.576350 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 283/568] 10.222.0.16 () {58 vars in 1175 bytes} [Sat Jul 16 21:10:05 2022] GET /compute/v2.1/os-services?binary=nova-compute => generated 380 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:10:05.580147 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-f84797d6-7a30-44ee-84e3-f310906c8d45 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:05.580780 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-f84797d6-7a30-44ee-84e3-f310906c8d45 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: f98a9762-a67c-4e95-85b8-727c81bdb03a] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:05.601394 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f84797d6-7a30-44ee-84e3-f310906c8d45 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:05.601855 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f84797d6-7a30-44ee-84e3-f310906c8d45 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:05.603137 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f84797d6-7a30-44ee-84e3-f310906c8d45 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:05.603661 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f84797d6-7a30-44ee-84e3-f310906c8d45 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:06.144200 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-040b8fbf-b330-46cc-9858-ee70ba751b09 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a69f9dee03d43beaafa966216b9e289', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9b769bf8f3ea4e368bf68de1df6538f9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:06.165291 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-040b8fbf-b330-46cc-9858-ee70ba751b09 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] 10.222.0.16 "GET /compute/v2.1/servers/c40ba20d-eb88-4ee8-be8f-ecbbd3315c48" status: 200 len: 1503 microversion: 2.1 time: 0.723180 Jul 16 21:10:06.165291 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 286/569] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:10:05 2022] GET /compute/v2.1/servers/c40ba20d-eb88-4ee8-be8f-ecbbd3315c48 => generated 1503 bytes in 740 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:10:06.173937 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-131ba57c-fd3d-4bc4-8661-ae1647b8b54e tempest-KeyPairsV210TestJSON-621372744 tempest-KeyPairsV210TestJSON-621372744-project-admin] Calling method '.version_select at 0x7fdd9d52e790>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:06.301462 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-f84797d6-7a30-44ee-84e3-f310906c8d45 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b68dacc4b8463ba07fae6372131117', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55a19b7a4682457a9132f5fc80110712', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:06.310382 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-f84797d6-7a30-44ee-84e3-f310906c8d45 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "GET /compute/v2.1/servers/f98a9762-a67c-4e95-85b8-727c81bdb03a" status: 200 len: 1312 microversion: 2.1 time: 0.738760 Jul 16 21:10:06.314503 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 284/570] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:10:05 2022] GET /compute/v2.1/servers/f98a9762-a67c-4e95-85b8-727c81bdb03a => generated 1312 bytes in 744 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:10:06.331579 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-131ba57c-fd3d-4bc4-8661-ae1647b8b54e tempest-KeyPairsV210TestJSON-621372744 tempest-KeyPairsV210TestJSON-621372744-project-admin] 10.222.0.16 "DELETE /compute/v2.1/os-keypairs/tempest-KeyPairsV210TestJSON-keypair-1964472426" status: 204 len: 0 microversion: 2.10 time: 0.161178 Jul 16 21:10:06.331579 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 287/571] 10.222.0.16 () {60 vars in 1324 bytes} [Sat Jul 16 21:10:06 2022] DELETE /compute/v2.1/os-keypairs/tempest-KeyPairsV210TestJSON-keypair-1964472426 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Jul 16 21:10:06.334588 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-ed51d9ec-7fa0-422d-a1c3-179a6f78f222 tempest-LiveMigrationNegativeTest-1294767845 tempest-LiveMigrationNegativeTest-1294767845-project-admin] Action: 'action', calling method: >, body: {"os-migrateLive": {"host": "nch1-master-1", "block_migration": true, "disk_over_commit": false}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:10:06.335881 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-ed51d9ec-7fa0-422d-a1c3-179a6f78f222 tempest-LiveMigrationNegativeTest-1294767845 tempest-LiveMigrationNegativeTest-1294767845-project-admin] [instance: 67266541-b43e-414c-a744-9c32c6f53975] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:06.340148 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-e086c845-7c1f-402d-ab17-e52c310d6b95 tempest-ServerDiagnosticsNegativeTest-1844467799 tempest-ServerDiagnosticsNegativeTest-1844467799-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:06.340990 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-e086c845-7c1f-402d-ab17-e52c310d6b95 tempest-ServerDiagnosticsNegativeTest-1844467799 tempest-ServerDiagnosticsNegativeTest-1844467799-project] [instance: 1e4229cb-4ed9-48c3-b723-28619ae7045c] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:06.350426 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e086c845-7c1f-402d-ab17-e52c310d6b95 tempest-ServerDiagnosticsNegativeTest-1844467799 tempest-ServerDiagnosticsNegativeTest-1844467799-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:06.351233 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e086c845-7c1f-402d-ab17-e52c310d6b95 tempest-ServerDiagnosticsNegativeTest-1844467799 tempest-ServerDiagnosticsNegativeTest-1844467799-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:06.351375 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ed51d9ec-7fa0-422d-a1c3-179a6f78f222 tempest-LiveMigrationNegativeTest-1294767845 tempest-LiveMigrationNegativeTest-1294767845-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:06.351700 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ed51d9ec-7fa0-422d-a1c3-179a6f78f222 tempest-LiveMigrationNegativeTest-1294767845 tempest-LiveMigrationNegativeTest-1294767845-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:06.352516 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e086c845-7c1f-402d-ab17-e52c310d6b95 tempest-ServerDiagnosticsNegativeTest-1844467799 tempest-ServerDiagnosticsNegativeTest-1844467799-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:06.352647 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ed51d9ec-7fa0-422d-a1c3-179a6f78f222 tempest-LiveMigrationNegativeTest-1294767845 tempest-LiveMigrationNegativeTest-1294767845-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:06.352966 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ed51d9ec-7fa0-422d-a1c3-179a6f78f222 tempest-LiveMigrationNegativeTest-1294767845 tempest-LiveMigrationNegativeTest-1294767845-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:06.353386 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e086c845-7c1f-402d-ab17-e52c310d6b95 tempest-ServerDiagnosticsNegativeTest-1844467799 tempest-ServerDiagnosticsNegativeTest-1844467799-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:06.424351 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.objects.instance [None req-ed51d9ec-7fa0-422d-a1c3-179a6f78f222 tempest-LiveMigrationNegativeTest-1294767845 tempest-LiveMigrationNegativeTest-1294767845-project-admin] Lazy-loading 'flavor' on Instance uuid 67266541-b43e-414c-a744-9c32c6f53975 {{(pid=138680) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:10:06.461844 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-ed51d9ec-7fa0-422d-a1c3-179a6f78f222 tempest-LiveMigrationNegativeTest-1294767845 tempest-LiveMigrationNegativeTest-1294767845-project-admin] HTTP exception thrown: Cannot 'os-migrateLive' instance 67266541-b43e-414c-a744-9c32c6f53975 while it is in vm_state suspended Jul 16 21:10:06.464142 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-ed51d9ec-7fa0-422d-a1c3-179a6f78f222 tempest-LiveMigrationNegativeTest-1294767845 tempest-LiveMigrationNegativeTest-1294767845-project-admin] Request method failure captured: Jul 16 21:10:06.464142 ncdv-master-1 devstack@n-api.service[138680]: request: POST /compute/v2.1/servers/67266541-b43e-414c-a744-9c32c6f53975/action HTTP/1.1 Jul 16 21:10:06.464142 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:10:06.464142 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:10:06.464142 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:10:06.464142 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 97 Jul 16 21:10:06.464142 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:10:06.464142 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:10:06.464142 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:10:06.464142 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:10:06.464142 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0yjz1qEdsJvcfI6TRG20vmhREjrkYHDJV3wmhkVZMKHLvvLRTksIbEDj0iJPdh4eC_fXyYNtKdv6xfnyupQMT733OepnnNCu6ae2ezn2GWnQUDiNOSUVTO58w51AATzxplYj67aGXbssyCsNcqcB8SJlN2xLMo4vfipSLLg_WlmIh50 Jul 16 21:10:06.464142 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:10:06.464142 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:10:06.464142 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:10:06.464142 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:10:06.464142 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:10:06.464142 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:10:06.464142 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: 7f515204606542ce9377e2c0b707e5bc Jul 16 21:10:06.464142 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-LiveMigrationNegativeTest-1294767845 Jul 16 21:10:06.464142 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,admin,reader Jul 16 21:10:06.464142 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,admin,reader Jul 16 21:10:06.466005 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/7f515204606542ce9377e2c0b707e5bc"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/7f515204606542ce9377e2c0b707e5bc"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_7f515204606542ce9377e2c0b707e5bc"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/7f515204606542ce9377e2c0b707e5bc"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:10:06.466005 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-LiveMigrationNegativeTest-1294767845 Jul 16 21:10:06.466005 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: 7f515204606542ce9377e2c0b707e5bc Jul 16 21:10:06.466005 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-LiveMigrationNegativeTest-1294767845 Jul 16 21:10:06.466005 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-LiveMigrationNegativeTest-1294767845-project-admin Jul 16 21:10:06.466005 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:10:06.466005 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:10:06.466005 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: 24d825f528ce4d6d9e4da4fa13a417d9 Jul 16 21:10:06.466005 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-LiveMigrationNegativeTest-1294767845-project-admin Jul 16 21:10:06.466005 ncdv-master-1 devstack@n-api.service[138680]: Jul 16 21:10:06.466005 ncdv-master-1 devstack@n-api.service[138680]: {"os-migrateLive": {"host": "nch1-master-1", "block_migration": true, "disk_over_commit": false}} Jul 16 21:10:06.466005 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 21:10:06.466765 ncdv-master-1 devstack@n-api.service[138680]: exception: Cannot 'os-migrateLive' instance 67266541-b43e-414c-a744-9c32c6f53975 while it is in vm_state suspended Jul 16 21:10:06.466765 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': '67266541-b43e-414c-a744-9c32c6f53975', 'body': {'os-migrateLive': {'host': 'nch1-master-1', 'block_migration': True, 'disk_over_commit': False}}} Jul 16 21:10:06.466765 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:10:06.466765 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:10:06.466765 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/migrate_server.py", line 123, in _migrate_live Jul 16 21:10:06.466765 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi self.compute_api.live_migrate(context, instance, block_migration, Jul 16 21:10:06.466765 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 375, in inner Jul 16 21:10:06.466765 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Jul 16 21:10:06.466765 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 360, in wrapper Jul 16 21:10:06.466765 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return func(self, context, instance, *args, **kwargs) Jul 16 21:10:06.466765 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 296, in inner Jul 16 21:10:06.466765 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Jul 16 21:10:06.466765 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 343, in wrapper Jul 16 21:10:06.466765 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return func(self, context, instance, *args, **kwargs) Jul 16 21:10:06.466765 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 275, in inner Jul 16 21:10:06.466765 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Jul 16 21:10:06.466765 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 256, in inner Jul 16 21:10:06.466765 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Jul 16 21:10:06.467894 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner Jul 16 21:10:06.467894 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Jul 16 21:10:06.467894 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 145, in inner Jul 16 21:10:06.467894 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.InstanceInvalidState( Jul 16 21:10:06.467894 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.InstanceInvalidState: Instance 67266541-b43e-414c-a744-9c32c6f53975 in vm_state suspended. Cannot live_migrate while the instance is in this state. Jul 16 21:10:06.467894 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:10:06.467894 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:10:06.467894 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:10:06.467894 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:10:06.467894 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:10:06.467894 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:10:06.467894 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:10:06.467894 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:10:06.467894 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:10:06.467894 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:10:06.467894 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 16 21:10:06.467894 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 16 21:10:06.467894 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 16 21:10:06.467894 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 16 21:10:06.467894 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 16 21:10:06.469635 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 16 21:10:06.469635 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi [Previous line repeated 1 more time] Jul 16 21:10:06.469635 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/migrate_server.py", line 152, in _migrate_live Jul 16 21:10:06.469635 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi common.raise_http_conflict_for_instance_invalid_state(state_error, Jul 16 21:10:06.469635 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 347, in raise_http_conflict_for_instance_invalid_state Jul 16 21:10:06.469635 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPConflict(explanation=msg) Jul 16 21:10:06.469635 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPConflict: Cannot 'os-migrateLive' instance 67266541-b43e-414c-a744-9c32c6f53975 while it is in vm_state suspended Jul 16 21:10:06.469635 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:10:06.469635 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:10:06.469635 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:10:06.469635 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:10:06.469635 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:10:06.469635 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:10:06.469635 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:10:06.469635 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:10:06.469635 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Cannot 'os-migrateLive' instance 67266541-b43e-414c-a744-9c32c6f53975 while it is in vm_state suspended Jul 16 21:10:06.469635 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:10:06.470751 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-ed51d9ec-7fa0-422d-a1c3-179a6f78f222 tempest-LiveMigrationNegativeTest-1294767845 tempest-LiveMigrationNegativeTest-1294767845-project-admin] Returning 409 to user: Cannot 'os-migrateLive' instance 67266541-b43e-414c-a744-9c32c6f53975 while it is in vm_state suspended {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:10:06.470751 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-ed51d9ec-7fa0-422d-a1c3-179a6f78f222 tempest-LiveMigrationNegativeTest-1294767845 tempest-LiveMigrationNegativeTest-1294767845-project-admin] 10.222.0.16 "POST /compute/v2.1/servers/67266541-b43e-414c-a744-9c32c6f53975/action" status: 409 len: 155 microversion: 2.1 time: 0.149251 Jul 16 21:10:06.470751 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 285/572] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 21:10:06 2022] POST /compute/v2.1/servers/67266541-b43e-414c-a744-9c32c6f53975/action => generated 155 bytes in 151 msecs (HTTP/1.1 409) 9 headers in 378 bytes (1 switches on core 0) Jul 16 21:10:06.483438 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-a744bce6-84d8-4f04-b058-1d0e91a26ed8 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:06.485064 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-a744bce6-84d8-4f04-b058-1d0e91a26ed8 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] [instance: 8872e8cf-593e-4a7a-940d-e0c8159d4ad7] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:06.501905 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a744bce6-84d8-4f04-b058-1d0e91a26ed8 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:06.503325 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a744bce6-84d8-4f04-b058-1d0e91a26ed8 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:06.508780 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a744bce6-84d8-4f04-b058-1d0e91a26ed8 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:06.510299 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a744bce6-84d8-4f04-b058-1d0e91a26ed8 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:06.997171 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell0:SELECT=14 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:10:07.134328 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-e086c845-7c1f-402d-ab17-e52c310d6b95 tempest-ServerDiagnosticsNegativeTest-1844467799 tempest-ServerDiagnosticsNegativeTest-1844467799-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9c428093bcf445a6aea6ddb209e6e5cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5f9cdca4179c4875a4be57cf8a9f46a8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:07.145060 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-e086c845-7c1f-402d-ab17-e52c310d6b95 tempest-ServerDiagnosticsNegativeTest-1844467799 tempest-ServerDiagnosticsNegativeTest-1844467799-project] 10.222.0.16 "GET /compute/v2.1/servers/1e4229cb-4ed9-48c3-b723-28619ae7045c" status: 200 len: 1323 microversion: 2.1 time: 0.817586 Jul 16 21:10:07.155964 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 288/573] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:10:06 2022] GET /compute/v2.1/servers/1e4229cb-4ed9-48c3-b723-28619ae7045c => generated 1323 bytes in 829 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:10:07.161987 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-d45a267b-0570-4715-9619-41d8be438785 tempest-KeyPairsV210TestJSON-621372744 tempest-KeyPairsV210TestJSON-621372744-project-admin] Calling method '.version_select at 0x7fdd9d5eb1f0>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:07.238499 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-a744bce6-84d8-4f04-b058-1d0e91a26ed8 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ef9cbce2b2e64665b7f50767342dcb46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0ee02e4c8b448f2b77798ae504129b9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:07.255812 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-a744bce6-84d8-4f04-b058-1d0e91a26ed8 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] 10.222.0.16 "GET /compute/v2.1/servers/8872e8cf-593e-4a7a-940d-e0c8159d4ad7" status: 200 len: 1389 microversion: 2.1 time: 0.788502 Jul 16 21:10:07.258465 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 286/574] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:10:06 2022] GET /compute/v2.1/servers/8872e8cf-593e-4a7a-940d-e0c8159d4ad7 => generated 1389 bytes in 791 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:10:07.267299 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-4bf31286-7ec1-43a6-aa31-9bd4b1dd7b19 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:07.268467 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-4bf31286-7ec1-43a6-aa31-9bd4b1dd7b19 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] [instance: 7d624313-0c12-4da2-a95c-3903efae7608] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:07.277480 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4bf31286-7ec1-43a6-aa31-9bd4b1dd7b19 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:07.278553 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4bf31286-7ec1-43a6-aa31-9bd4b1dd7b19 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:07.280197 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4bf31286-7ec1-43a6-aa31-9bd4b1dd7b19 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:07.281329 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4bf31286-7ec1-43a6-aa31-9bd4b1dd7b19 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:07.309177 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-d45a267b-0570-4715-9619-41d8be438785 tempest-KeyPairsV210TestJSON-621372744 tempest-KeyPairsV210TestJSON-621372744-project-admin] 10.222.0.16 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-292452046?user_id=587057ad0ece48d2a6ae419b7535ef64" status: 204 len: 0 microversion: 2.10 time: 0.150882 Jul 16 21:10:07.309845 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 289/575] 10.222.0.16 () {60 vars in 1339 bytes} [Sat Jul 16 21:10:07 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-292452046?user_id=587057ad0ece48d2a6ae419b7535ef64 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Jul 16 21:10:07.319454 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-044a0463-69d4-4621-a949-71e7ce6d0471 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:07.320013 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-044a0463-69d4-4621-a949-71e7ce6d0471 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] [instance: 67266541-b43e-414c-a744-9c32c6f53975] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:07.332324 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-044a0463-69d4-4621-a949-71e7ce6d0471 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:07.332985 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-044a0463-69d4-4621-a949-71e7ce6d0471 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:07.337499 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-044a0463-69d4-4621-a949-71e7ce6d0471 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:07.337499 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-044a0463-69d4-4621-a949-71e7ce6d0471 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:07.374346 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-044a0463-69d4-4621-a949-71e7ce6d0471 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] [instance: 67266541-b43e-414c-a744-9c32c6f53975] Going to try to terminate instance {{(pid=138681) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 16 21:10:07.498414 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-4bf31286-7ec1-43a6-aa31-9bd4b1dd7b19 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'abeb7020019d41a18de42956d706a011', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '649babe54d104436af461f1ee4767001', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:07.508711 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-4bf31286-7ec1-43a6-aa31-9bd4b1dd7b19 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] 10.222.0.16 "GET /compute/v2.1/servers/7d624313-0c12-4da2-a95c-3903efae7608" status: 200 len: 1318 microversion: 2.1 time: 0.248424 Jul 16 21:10:07.509373 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 287/576] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:10:07 2022] GET /compute/v2.1/servers/7d624313-0c12-4da2-a95c-3903efae7608 => generated 1318 bytes in 250 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:10:07.525510 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-d94d02bc-6029-404f-bf5e-4afdcf00cdbf tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:07.525871 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-d94d02bc-6029-404f-bf5e-4afdcf00cdbf tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] [instance: cf53f7cd-deae-4b34-bf8e-3afef98b2ec6] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:07.533773 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d94d02bc-6029-404f-bf5e-4afdcf00cdbf tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:07.534345 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d94d02bc-6029-404f-bf5e-4afdcf00cdbf tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:07.535197 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d94d02bc-6029-404f-bf5e-4afdcf00cdbf tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:07.535623 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d94d02bc-6029-404f-bf5e-4afdcf00cdbf tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:07.811162 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-d94d02bc-6029-404f-bf5e-4afdcf00cdbf tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ba60685a2001459cbfb531d95a6b296e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb6deae6b08e4ed8b15dc6edfe1d38d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:07.820008 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-d94d02bc-6029-404f-bf5e-4afdcf00cdbf tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ba60685a2001459cbfb531d95a6b296e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb6deae6b08e4ed8b15dc6edfe1d38d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:07.822062 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-d94d02bc-6029-404f-bf5e-4afdcf00cdbf tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ba60685a2001459cbfb531d95a6b296e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb6deae6b08e4ed8b15dc6edfe1d38d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:07.829537 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-d94d02bc-6029-404f-bf5e-4afdcf00cdbf tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] 10.222.0.16 "GET /compute/v2.1/servers/cf53f7cd-deae-4b34-bf8e-3afef98b2ec6" status: 200 len: 1447 microversion: 2.75 time: 0.319099 Jul 16 21:10:07.831797 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 288/577] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:10:07 2022] GET /compute/v2.1/servers/cf53f7cd-deae-4b34-bf8e-3afef98b2ec6 => generated 1447 bytes in 322 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:10:07.841068 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-5826a112-e205-48ce-b4dc-e6ab21f9fb01 req-d00165a8-9438-40bb-9268-9a95762ffb06 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "84ee8179-ff94-43b3-97f2-1867bc6420a8", "tag": "48ddda29-7627-4cbb-adf0-0975571cbc33"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:10:07.851477 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-5826a112-e205-48ce-b4dc-e6ab21f9fb01 req-d00165a8-9438-40bb-9268-9a95762ffb06 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:07.851932 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-5826a112-e205-48ce-b4dc-e6ab21f9fb01 req-d00165a8-9438-40bb-9268-9a95762ffb06 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:07.858689 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-044a0463-69d4-4621-a949-71e7ce6d0471 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] 10.222.0.16 "DELETE /compute/v2.1/servers/67266541-b43e-414c-a744-9c32c6f53975" status: 204 len: 0 microversion: 2.1 time: 0.547121 Jul 16 21:10:07.864002 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 290/578] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:10:07 2022] DELETE /compute/v2.1/servers/67266541-b43e-414c-a744-9c32c6f53975 => generated 0 bytes in 553 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 16 21:10:07.872538 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-ea47c24b-9046-41e2-9310-3c5b191e1454 req-55695f67-093b-4f01-8976-e8f0cacdb8fc service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "f98a9762-a67c-4e95-85b8-727c81bdb03a", "name": "network-vif-plugged", "status": "completed", "tag": "b0aae229-268b-497f-8da0-36ccb43a67bd"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:10:07.874153 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-5826a112-e205-48ce-b4dc-e6ab21f9fb01 req-d00165a8-9438-40bb-9268-9a95762ffb06 service nova] Creating event network-changed:48ddda29-7627-4cbb-adf0-0975571cbc33 for instance 84ee8179-ff94-43b3-97f2-1867bc6420a8 on nch2-master-1 Jul 16 21:10:07.880768 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-5826a112-e205-48ce-b4dc-e6ab21f9fb01 req-d00165a8-9438-40bb-9268-9a95762ffb06 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.046871 Jul 16 21:10:07.881756 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 289/579] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:10:07 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:10:07.886327 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-ea47c24b-9046-41e2-9310-3c5b191e1454 req-55695f67-093b-4f01-8976-e8f0cacdb8fc service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:07.887477 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-ce339a2a-f6b3-44a3-be69-ed8c624c54ec tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:07.888195 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-ce339a2a-f6b3-44a3-be69-ed8c624c54ec tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] [instance: c40ba20d-eb88-4ee8-be8f-ecbbd3315c48] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:07.888412 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-ea47c24b-9046-41e2-9310-3c5b191e1454 req-55695f67-093b-4f01-8976-e8f0cacdb8fc service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:07.900283 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ce339a2a-f6b3-44a3-be69-ed8c624c54ec tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:07.900283 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ce339a2a-f6b3-44a3-be69-ed8c624c54ec tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:07.900283 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ce339a2a-f6b3-44a3-be69-ed8c624c54ec tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:07.900283 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ce339a2a-f6b3-44a3-be69-ed8c624c54ec tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:07.903289 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-ea47c24b-9046-41e2-9310-3c5b191e1454 req-55695f67-093b-4f01-8976-e8f0cacdb8fc service nova] Creating event network-vif-plugged:b0aae229-268b-497f-8da0-36ccb43a67bd for instance f98a9762-a67c-4e95-85b8-727c81bdb03a on nch1-master-1 Jul 16 21:10:07.910740 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-ea47c24b-9046-41e2-9310-3c5b191e1454 req-55695f67-093b-4f01-8976-e8f0cacdb8fc service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.044527 Jul 16 21:10:07.911752 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 291/580] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:10:07 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:10:07.918583 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-59a6bf8f-511b-4d65-b590-ebb4900a1ded req-24724dd2-f6eb-47e4-ba40-052930f2b41f service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "1e4229cb-4ed9-48c3-b723-28619ae7045c", "tag": "d94b5111-1b85-496e-b76b-7338bb0d2319"}, {"server_uuid": "e506a6e0-fbcd-4e53-a6a0-d31761654c92", "name": "network-vif-deleted", "tag": "fa66e5de-7c5f-4e0e-adc5-96ea3b90bd37"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:10:07.924243 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_api:SELECT=94,nova_api:INSERT=3,nova_api:DELETE=2,nova_api:UPDATE=1 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:10:07.932482 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-59a6bf8f-511b-4d65-b590-ebb4900a1ded req-24724dd2-f6eb-47e4-ba40-052930f2b41f service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:07.933250 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-59a6bf8f-511b-4d65-b590-ebb4900a1ded req-24724dd2-f6eb-47e4-ba40-052930f2b41f service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:07.951281 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-59a6bf8f-511b-4d65-b590-ebb4900a1ded req-24724dd2-f6eb-47e4-ba40-052930f2b41f service nova] Creating event network-changed:d94b5111-1b85-496e-b76b-7338bb0d2319 for instance 1e4229cb-4ed9-48c3-b723-28619ae7045c on nch1-master-1 Jul 16 21:10:07.952110 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-59a6bf8f-511b-4d65-b590-ebb4900a1ded req-24724dd2-f6eb-47e4-ba40-052930f2b41f service nova] Creating event network-vif-deleted:fa66e5de-7c5f-4e0e-adc5-96ea3b90bd37 for instance e506a6e0-fbcd-4e53-a6a0-d31761654c92 on nch1-master-1 Jul 16 21:10:07.961955 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-59a6bf8f-511b-4d65-b590-ebb4900a1ded req-24724dd2-f6eb-47e4-ba40-052930f2b41f service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 350 microversion: 2.1 time: 0.047748 Jul 16 21:10:07.962592 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 292/581] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:10:07 2022] POST /compute/v2.1/os-server-external-events => generated 350 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:10:07.968220 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-f1e1ec61-26bf-4ffe-bf07-22de1c1f1a39 tempest-KeyPairsV210TestJSON-621372744 tempest-KeyPairsV210TestJSON-621372744-project-admin] Calling method '.version_select at 0x7fdd9d5eb1f0>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:08.068917 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-f1e1ec61-26bf-4ffe-bf07-22de1c1f1a39 tempest-KeyPairsV210TestJSON-621372744 tempest-KeyPairsV210TestJSON-621372744-project-admin] 10.222.0.16 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-151248390?user_id=587057ad0ece48d2a6ae419b7535ef64" status: 204 len: 0 microversion: 2.10 time: 0.104859 Jul 16 21:10:08.076301 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 293/582] 10.222.0.16 () {60 vars in 1339 bytes} [Sat Jul 16 21:10:07 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-151248390?user_id=587057ad0ece48d2a6ae419b7535ef64 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Jul 16 21:10:08.082647 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-92ed1b1e-31b6-4715-929c-cb1efbe9b63a tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:08.083786 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-92ed1b1e-31b6-4715-929c-cb1efbe9b63a tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: f98a9762-a67c-4e95-85b8-727c81bdb03a] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:08.091334 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-92ed1b1e-31b6-4715-929c-cb1efbe9b63a tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:08.096537 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-92ed1b1e-31b6-4715-929c-cb1efbe9b63a tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:08.096537 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-92ed1b1e-31b6-4715-929c-cb1efbe9b63a tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:08.096537 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-92ed1b1e-31b6-4715-929c-cb1efbe9b63a tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:08.207696 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-ce339a2a-f6b3-44a3-be69-ed8c624c54ec tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a69f9dee03d43beaafa966216b9e289', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9b769bf8f3ea4e368bf68de1df6538f9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:08.215261 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-ce339a2a-f6b3-44a3-be69-ed8c624c54ec tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] 10.222.0.16 "GET /compute/v2.1/servers/c40ba20d-eb88-4ee8-be8f-ecbbd3315c48" status: 200 len: 1503 microversion: 2.1 time: 0.332305 Jul 16 21:10:08.217000 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 290/583] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:10:07 2022] GET /compute/v2.1/servers/c40ba20d-eb88-4ee8-be8f-ecbbd3315c48 => generated 1503 bytes in 335 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:10:08.225445 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-a949688b-d4ba-4f6c-a7a7-59021a32518d tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:08.227452 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-a949688b-d4ba-4f6c-a7a7-59021a32518d tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] [instance: 67266541-b43e-414c-a744-9c32c6f53975] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:08.240527 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a949688b-d4ba-4f6c-a7a7-59021a32518d tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:08.242316 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a949688b-d4ba-4f6c-a7a7-59021a32518d tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:08.243156 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a949688b-d4ba-4f6c-a7a7-59021a32518d tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:08.245094 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a949688b-d4ba-4f6c-a7a7-59021a32518d tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:08.482888 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-92ed1b1e-31b6-4715-929c-cb1efbe9b63a tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b68dacc4b8463ba07fae6372131117', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55a19b7a4682457a9132f5fc80110712', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:08.499329 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-92ed1b1e-31b6-4715-929c-cb1efbe9b63a tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "GET /compute/v2.1/servers/f98a9762-a67c-4e95-85b8-727c81bdb03a" status: 200 len: 1312 microversion: 2.1 time: 0.420914 Jul 16 21:10:08.509826 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 294/584] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:10:08 2022] GET /compute/v2.1/servers/f98a9762-a67c-4e95-85b8-727c81bdb03a => generated 1312 bytes in 433 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:10:08.542432 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-08696ffa-3117-48eb-9962-4b4eb9a75c8f tempest-ServerDiagnosticsNegativeTest-1844467799 tempest-ServerDiagnosticsNegativeTest-1844467799-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:08.543458 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-08696ffa-3117-48eb-9962-4b4eb9a75c8f tempest-ServerDiagnosticsNegativeTest-1844467799 tempest-ServerDiagnosticsNegativeTest-1844467799-project] [instance: 1e4229cb-4ed9-48c3-b723-28619ae7045c] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:08.553334 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-08696ffa-3117-48eb-9962-4b4eb9a75c8f tempest-ServerDiagnosticsNegativeTest-1844467799 tempest-ServerDiagnosticsNegativeTest-1844467799-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:08.554078 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-08696ffa-3117-48eb-9962-4b4eb9a75c8f tempest-ServerDiagnosticsNegativeTest-1844467799 tempest-ServerDiagnosticsNegativeTest-1844467799-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:08.555119 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-08696ffa-3117-48eb-9962-4b4eb9a75c8f tempest-ServerDiagnosticsNegativeTest-1844467799 tempest-ServerDiagnosticsNegativeTest-1844467799-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:08.555781 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-08696ffa-3117-48eb-9962-4b4eb9a75c8f tempest-ServerDiagnosticsNegativeTest-1844467799 tempest-ServerDiagnosticsNegativeTest-1844467799-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:08.586152 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:10:08.616671 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-a949688b-d4ba-4f6c-a7a7-59021a32518d tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0d72626f465b4f37ad3b18bb945d4000', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fc1c3463f9444119af9d18f85bdd703d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:08.627411 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-a949688b-d4ba-4f6c-a7a7-59021a32518d tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] 10.222.0.16 "GET /compute/v2.1/servers/67266541-b43e-414c-a744-9c32c6f53975" status: 200 len: 1465 microversion: 2.1 time: 0.409024 Jul 16 21:10:08.628544 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 291/585] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:10:08 2022] GET /compute/v2.1/servers/67266541-b43e-414c-a744-9c32c6f53975 => generated 1465 bytes in 411 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:10:08.649180 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-7f0ea12c-4a2e-4962-9708-6e1d21a9c51f tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:08.649180 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-7f0ea12c-4a2e-4962-9708-6e1d21a9c51f tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] [instance: 8872e8cf-593e-4a7a-940d-e0c8159d4ad7] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:08.653282 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7f0ea12c-4a2e-4962-9708-6e1d21a9c51f tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:08.653970 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7f0ea12c-4a2e-4962-9708-6e1d21a9c51f tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:08.655587 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7f0ea12c-4a2e-4962-9708-6e1d21a9c51f tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:08.656477 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7f0ea12c-4a2e-4962-9708-6e1d21a9c51f tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:08.872066 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-08696ffa-3117-48eb-9962-4b4eb9a75c8f tempest-ServerDiagnosticsNegativeTest-1844467799 tempest-ServerDiagnosticsNegativeTest-1844467799-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9c428093bcf445a6aea6ddb209e6e5cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5f9cdca4179c4875a4be57cf8a9f46a8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:08.884122 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-08696ffa-3117-48eb-9962-4b4eb9a75c8f tempest-ServerDiagnosticsNegativeTest-1844467799 tempest-ServerDiagnosticsNegativeTest-1844467799-project] 10.222.0.16 "GET /compute/v2.1/servers/1e4229cb-4ed9-48c3-b723-28619ae7045c" status: 200 len: 1323 microversion: 2.1 time: 0.369362 Jul 16 21:10:08.885037 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 295/586] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:10:08 2022] GET /compute/v2.1/servers/1e4229cb-4ed9-48c3-b723-28619ae7045c => generated 1323 bytes in 371 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:10:08.905986 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-cf888988-c6a2-4deb-9847-79ffd548a084 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:08.906621 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-cf888988-c6a2-4deb-9847-79ffd548a084 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] [instance: 7d624313-0c12-4da2-a95c-3903efae7608] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:08.926656 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cf888988-c6a2-4deb-9847-79ffd548a084 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:08.927296 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cf888988-c6a2-4deb-9847-79ffd548a084 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:08.949495 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cf888988-c6a2-4deb-9847-79ffd548a084 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:08.949495 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cf888988-c6a2-4deb-9847-79ffd548a084 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:08.950328 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-7f0ea12c-4a2e-4962-9708-6e1d21a9c51f tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ef9cbce2b2e64665b7f50767342dcb46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0ee02e4c8b448f2b77798ae504129b9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:08.954678 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-7f0ea12c-4a2e-4962-9708-6e1d21a9c51f tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] 10.222.0.16 "GET /compute/v2.1/servers/8872e8cf-593e-4a7a-940d-e0c8159d4ad7" status: 200 len: 1389 microversion: 2.1 time: 0.324489 Jul 16 21:10:08.966317 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 292/587] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:10:08 2022] GET /compute/v2.1/servers/8872e8cf-593e-4a7a-940d-e0c8159d4ad7 => generated 1389 bytes in 336 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:10:08.971768 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-c4c1daa9-fb86-475f-994b-2928acdc3263 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:08.972479 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-c4c1daa9-fb86-475f-994b-2928acdc3263 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] [instance: cf53f7cd-deae-4b34-bf8e-3afef98b2ec6] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:08.991209 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c4c1daa9-fb86-475f-994b-2928acdc3263 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:08.991914 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c4c1daa9-fb86-475f-994b-2928acdc3263 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:08.995605 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c4c1daa9-fb86-475f-994b-2928acdc3263 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:08.995605 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c4c1daa9-fb86-475f-994b-2928acdc3263 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:09.018660 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=100 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:10:09.099366 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-c4c1daa9-fb86-475f-994b-2928acdc3263 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ba60685a2001459cbfb531d95a6b296e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb6deae6b08e4ed8b15dc6edfe1d38d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:09.108857 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-c4c1daa9-fb86-475f-994b-2928acdc3263 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ba60685a2001459cbfb531d95a6b296e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb6deae6b08e4ed8b15dc6edfe1d38d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:09.113184 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-c4c1daa9-fb86-475f-994b-2928acdc3263 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ba60685a2001459cbfb531d95a6b296e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb6deae6b08e4ed8b15dc6edfe1d38d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:09.122464 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-c4c1daa9-fb86-475f-994b-2928acdc3263 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] 10.222.0.16 "GET /compute/v2.1/servers/cf53f7cd-deae-4b34-bf8e-3afef98b2ec6" status: 200 len: 1447 microversion: 2.75 time: 0.154911 Jul 16 21:10:09.137070 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 293/588] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:10:08 2022] GET /compute/v2.1/servers/cf53f7cd-deae-4b34-bf8e-3afef98b2ec6 => generated 1447 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:10:09.159836 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-cf888988-c6a2-4deb-9847-79ffd548a084 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'abeb7020019d41a18de42956d706a011', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '649babe54d104436af461f1ee4767001', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:09.170917 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-cf888988-c6a2-4deb-9847-79ffd548a084 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] 10.222.0.16 "GET /compute/v2.1/servers/7d624313-0c12-4da2-a95c-3903efae7608" status: 200 len: 1318 microversion: 2.1 time: 0.268851 Jul 16 21:10:09.173040 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 296/589] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:10:08 2022] GET /compute/v2.1/servers/7d624313-0c12-4da2-a95c-3903efae7608 => generated 1318 bytes in 272 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:10:09.233411 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-df24d72d-6827-4393-b27c-3880c2cbb794 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:09.234039 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-df24d72d-6827-4393-b27c-3880c2cbb794 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] [instance: c40ba20d-eb88-4ee8-be8f-ecbbd3315c48] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:09.240916 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-df24d72d-6827-4393-b27c-3880c2cbb794 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:09.241391 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-df24d72d-6827-4393-b27c-3880c2cbb794 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:09.242200 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-df24d72d-6827-4393-b27c-3880c2cbb794 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:09.242639 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-df24d72d-6827-4393-b27c-3880c2cbb794 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:09.467999 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-df24d72d-6827-4393-b27c-3880c2cbb794 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a69f9dee03d43beaafa966216b9e289', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9b769bf8f3ea4e368bf68de1df6538f9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:09.479948 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-df24d72d-6827-4393-b27c-3880c2cbb794 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] 10.222.0.16 "GET /compute/v2.1/servers/c40ba20d-eb88-4ee8-be8f-ecbbd3315c48" status: 200 len: 1493 microversion: 2.1 time: 0.249566 Jul 16 21:10:09.483578 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 294/590] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:10:09 2022] GET /compute/v2.1/servers/c40ba20d-eb88-4ee8-be8f-ecbbd3315c48 => generated 1493 bytes in 255 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:10:09.496282 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-f2364d82-5011-40f8-9108-df244c232370 tempest-LiveMigrationTest-1131758637 tempest-LiveMigrationTest-1131758637-project-admin] Calling method '.version_select at 0x7fdd9d64eb80>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:09.503919 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f2364d82-5011-40f8-9108-df244c232370 tempest-LiveMigrationTest-1131758637 tempest-LiveMigrationTest-1131758637-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:09.503919 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f2364d82-5011-40f8-9108-df244c232370 tempest-LiveMigrationTest-1131758637 tempest-LiveMigrationTest-1131758637-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:09.517834 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-f2364d82-5011-40f8-9108-df244c232370 tempest-LiveMigrationTest-1131758637 tempest-LiveMigrationTest-1131758637-project-admin] 10.222.0.16 "GET /compute/v2.1/os-migrations" status: 200 len: 791 microversion: 2.1 time: 0.025230 Jul 16 21:10:09.526267 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 297/591] 10.222.0.16 () {58 vars in 1142 bytes} [Sat Jul 16 21:10:09 2022] GET /compute/v2.1/os-migrations => generated 791 bytes in 27 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:10:09.528974 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-7de3b378-99d8-49f4-94b6-5b23c25c81ec tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:09.530095 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-7de3b378-99d8-49f4-94b6-5b23c25c81ec tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: f98a9762-a67c-4e95-85b8-727c81bdb03a] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:09.533644 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-61a77c6e-1803-4520-8b97-ff5c4ea6febe tempest-LiveMigrationTest-1131758637 tempest-LiveMigrationTest-1131758637-project-admin] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:09.534563 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-61a77c6e-1803-4520-8b97-ff5c4ea6febe tempest-LiveMigrationTest-1131758637 tempest-LiveMigrationTest-1131758637-project-admin] [instance: c40ba20d-eb88-4ee8-be8f-ecbbd3315c48] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:09.537068 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_api:SELECT=96,nova_api:UPDATE=1,nova_api:INSERT=3 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:10:09.539755 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7de3b378-99d8-49f4-94b6-5b23c25c81ec tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:09.540517 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7de3b378-99d8-49f4-94b6-5b23c25c81ec tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:09.541486 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7de3b378-99d8-49f4-94b6-5b23c25c81ec tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:09.542110 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7de3b378-99d8-49f4-94b6-5b23c25c81ec tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:09.567603 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-61a77c6e-1803-4520-8b97-ff5c4ea6febe tempest-LiveMigrationTest-1131758637 tempest-LiveMigrationTest-1131758637-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:09.569007 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-61a77c6e-1803-4520-8b97-ff5c4ea6febe tempest-LiveMigrationTest-1131758637 tempest-LiveMigrationTest-1131758637-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.012s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:09.571717 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-61a77c6e-1803-4520-8b97-ff5c4ea6febe tempest-LiveMigrationTest-1131758637 tempest-LiveMigrationTest-1131758637-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:09.571717 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-61a77c6e-1803-4520-8b97-ff5c4ea6febe tempest-LiveMigrationTest-1131758637 tempest-LiveMigrationTest-1131758637-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:09.834713 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-7de3b378-99d8-49f4-94b6-5b23c25c81ec tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b68dacc4b8463ba07fae6372131117', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55a19b7a4682457a9132f5fc80110712', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:09.836746 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-7de3b378-99d8-49f4-94b6-5b23c25c81ec tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "GET /compute/v2.1/servers/f98a9762-a67c-4e95-85b8-727c81bdb03a" status: 200 len: 1312 microversion: 2.1 time: 0.313515 Jul 16 21:10:09.837578 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 295/592] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:10:09 2022] GET /compute/v2.1/servers/f98a9762-a67c-4e95-85b8-727c81bdb03a => generated 1312 bytes in 315 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:10:09.854244 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-17080bfb-99ba-43fc-b311-99445846457f tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:09.855244 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-17080bfb-99ba-43fc-b311-99445846457f tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] [instance: 67266541-b43e-414c-a744-9c32c6f53975] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:09.863667 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-17080bfb-99ba-43fc-b311-99445846457f tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:09.864230 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-17080bfb-99ba-43fc-b311-99445846457f tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:09.869185 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-17080bfb-99ba-43fc-b311-99445846457f tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:09.869185 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-17080bfb-99ba-43fc-b311-99445846457f tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:09.917898 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-61a77c6e-1803-4520-8b97-ff5c4ea6febe tempest-LiveMigrationTest-1131758637 tempest-LiveMigrationTest-1131758637-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/c40ba20d-eb88-4ee8-be8f-ecbbd3315c48" status: 200 len: 1644 microversion: 2.1 time: 0.389501 Jul 16 21:10:09.918815 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 298/593] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:10:09 2022] GET /compute/v2.1/servers/c40ba20d-eb88-4ee8-be8f-ecbbd3315c48 => generated 1644 bytes in 391 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:10:09.933898 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-6acedb04-b763-45d8-b546-b77cdcc1b65e tempest-ServerDiagnosticsNegativeTest-1844467799 tempest-ServerDiagnosticsNegativeTest-1844467799-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:09.934916 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-6acedb04-b763-45d8-b546-b77cdcc1b65e tempest-ServerDiagnosticsNegativeTest-1844467799 tempest-ServerDiagnosticsNegativeTest-1844467799-project] [instance: 1e4229cb-4ed9-48c3-b723-28619ae7045c] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:09.945423 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6acedb04-b763-45d8-b546-b77cdcc1b65e tempest-ServerDiagnosticsNegativeTest-1844467799 tempest-ServerDiagnosticsNegativeTest-1844467799-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:09.946025 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6acedb04-b763-45d8-b546-b77cdcc1b65e tempest-ServerDiagnosticsNegativeTest-1844467799 tempest-ServerDiagnosticsNegativeTest-1844467799-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:09.946887 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6acedb04-b763-45d8-b546-b77cdcc1b65e tempest-ServerDiagnosticsNegativeTest-1844467799 tempest-ServerDiagnosticsNegativeTest-1844467799-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:09.947529 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6acedb04-b763-45d8-b546-b77cdcc1b65e tempest-ServerDiagnosticsNegativeTest-1844467799 tempest-ServerDiagnosticsNegativeTest-1844467799-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:10.035178 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell0:SELECT=14 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:10:10.104625 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-17080bfb-99ba-43fc-b311-99445846457f tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0d72626f465b4f37ad3b18bb945d4000', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fc1c3463f9444119af9d18f85bdd703d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:10.129856 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-17080bfb-99ba-43fc-b311-99445846457f tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] 10.222.0.16 "GET /compute/v2.1/servers/67266541-b43e-414c-a744-9c32c6f53975" status: 200 len: 1465 microversion: 2.1 time: 0.279893 Jul 16 21:10:10.139473 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 296/594] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:10:09 2022] GET /compute/v2.1/servers/67266541-b43e-414c-a744-9c32c6f53975 => generated 1465 bytes in 288 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:10:10.146625 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-07d09875-0e68-48d3-abbb-c34459770202 req-3dcb82ee-1b75-49af-a825-b0ce377c1e71 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "7d624313-0c12-4da2-a95c-3903efae7608", "tag": "186c161e-a1dc-4086-9efc-737d7b982c1b"}, {"name": "network-changed", "server_uuid": "8872e8cf-593e-4a7a-940d-e0c8159d4ad7", "tag": "a07c49a0-9773-4e57-963c-ebcc388b002a"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:10:10.159974 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-07d09875-0e68-48d3-abbb-c34459770202 req-3dcb82ee-1b75-49af-a825-b0ce377c1e71 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:10.160391 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-07d09875-0e68-48d3-abbb-c34459770202 req-3dcb82ee-1b75-49af-a825-b0ce377c1e71 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:10.177121 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-07d09875-0e68-48d3-abbb-c34459770202 req-3dcb82ee-1b75-49af-a825-b0ce377c1e71 service nova] Creating event network-changed:186c161e-a1dc-4086-9efc-737d7b982c1b for instance 7d624313-0c12-4da2-a95c-3903efae7608 on nch1-master-1 Jul 16 21:10:10.177792 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-07d09875-0e68-48d3-abbb-c34459770202 req-3dcb82ee-1b75-49af-a825-b0ce377c1e71 service nova] Creating event network-changed:a07c49a0-9773-4e57-963c-ebcc388b002a for instance 8872e8cf-593e-4a7a-940d-e0c8159d4ad7 on nch2-master-1 Jul 16 21:10:10.189522 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-6acedb04-b763-45d8-b546-b77cdcc1b65e tempest-ServerDiagnosticsNegativeTest-1844467799 tempest-ServerDiagnosticsNegativeTest-1844467799-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9c428093bcf445a6aea6ddb209e6e5cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5f9cdca4179c4875a4be57cf8a9f46a8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:10.191416 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-07d09875-0e68-48d3-abbb-c34459770202 req-3dcb82ee-1b75-49af-a825-b0ce377c1e71 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 346 microversion: 2.1 time: 0.051149 Jul 16 21:10:10.196633 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 297/595] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:10:10 2022] POST /compute/v2.1/os-server-external-events => generated 346 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:10:10.198735 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-6acedb04-b763-45d8-b546-b77cdcc1b65e tempest-ServerDiagnosticsNegativeTest-1844467799 tempest-ServerDiagnosticsNegativeTest-1844467799-project] 10.222.0.16 "GET /compute/v2.1/servers/1e4229cb-4ed9-48c3-b723-28619ae7045c" status: 200 len: 1323 microversion: 2.1 time: 0.277415 Jul 16 21:10:10.199676 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 299/596] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:10:09 2022] GET /compute/v2.1/servers/1e4229cb-4ed9-48c3-b723-28619ae7045c => generated 1323 bytes in 280 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:10:10.205775 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-6b6d19d8-5eb9-4021-9158-2f3ebe7a5323 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-LiveMigrationTest-server-1372522724", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452", "networks": [{"uuid": "26ad28eb-6622-4b58-ad05-2de54c33be8d"}]}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:10:10.211324 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-2ef01fcb-7232-4108-a94d-d443dfa2b4da tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:10.212039 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-2ef01fcb-7232-4108-a94d-d443dfa2b4da tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] [instance: 8872e8cf-593e-4a7a-940d-e0c8159d4ad7] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:10.218682 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2ef01fcb-7232-4108-a94d-d443dfa2b4da tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:10.219498 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2ef01fcb-7232-4108-a94d-d443dfa2b4da tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:10.223898 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2ef01fcb-7232-4108-a94d-d443dfa2b4da tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:10.223898 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2ef01fcb-7232-4108-a94d-d443dfa2b4da tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:10.272501 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.network.neutron [None req-6b6d19d8-5eb9-4021-9158-2f3ebe7a5323 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] validate_networks() for [('26ad28eb-6622-4b58-ad05-2de54c33be8d', None, None, None, None, None)] {{(pid=138681) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 16 21:10:10.454424 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-2ef01fcb-7232-4108-a94d-d443dfa2b4da tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ef9cbce2b2e64665b7f50767342dcb46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0ee02e4c8b448f2b77798ae504129b9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:10.469321 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-2ef01fcb-7232-4108-a94d-d443dfa2b4da tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] 10.222.0.16 "GET /compute/v2.1/servers/8872e8cf-593e-4a7a-940d-e0c8159d4ad7" status: 200 len: 1389 microversion: 2.1 time: 0.272560 Jul 16 21:10:10.470219 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 298/597] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:10:10 2022] GET /compute/v2.1/servers/8872e8cf-593e-4a7a-940d-e0c8159d4ad7 => generated 1389 bytes in 277 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:10:10.483468 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-4305184e-3fe4-4d09-90eb-e60f463e2c8c tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:10.493116 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-4305184e-3fe4-4d09-90eb-e60f463e2c8c tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] [instance: cf53f7cd-deae-4b34-bf8e-3afef98b2ec6] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:10.505811 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4305184e-3fe4-4d09-90eb-e60f463e2c8c tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:10.507077 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4305184e-3fe4-4d09-90eb-e60f463e2c8c tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:10.508568 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4305184e-3fe4-4d09-90eb-e60f463e2c8c tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:10.509471 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4305184e-3fe4-4d09-90eb-e60f463e2c8c tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:10.619513 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-4305184e-3fe4-4d09-90eb-e60f463e2c8c tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ba60685a2001459cbfb531d95a6b296e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb6deae6b08e4ed8b15dc6edfe1d38d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:10.630606 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-4305184e-3fe4-4d09-90eb-e60f463e2c8c tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ba60685a2001459cbfb531d95a6b296e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb6deae6b08e4ed8b15dc6edfe1d38d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:10.632249 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-4305184e-3fe4-4d09-90eb-e60f463e2c8c tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ba60685a2001459cbfb531d95a6b296e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb6deae6b08e4ed8b15dc6edfe1d38d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:10.645877 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-4305184e-3fe4-4d09-90eb-e60f463e2c8c tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] 10.222.0.16 "GET /compute/v2.1/servers/cf53f7cd-deae-4b34-bf8e-3afef98b2ec6" status: 200 len: 1447 microversion: 2.75 time: 0.158814 Jul 16 21:10:10.645877 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 299/598] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:10:10 2022] GET /compute/v2.1/servers/cf53f7cd-deae-4b34-bf8e-3afef98b2ec6 => generated 1447 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:10:10.649615 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-b4069c84-78c8-422f-af60-0d8fec3849af tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:10.650349 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-b4069c84-78c8-422f-af60-0d8fec3849af tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] [instance: 7d624313-0c12-4da2-a95c-3903efae7608] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:10.657183 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b4069c84-78c8-422f-af60-0d8fec3849af tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:10.658408 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b4069c84-78c8-422f-af60-0d8fec3849af tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:10.658537 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b4069c84-78c8-422f-af60-0d8fec3849af tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:10.658977 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b4069c84-78c8-422f-af60-0d8fec3849af tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:10.876956 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-6b6d19d8-5eb9-4021-9158-2f3ebe7a5323 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Flavor limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 16 21:10:10.877468 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-6b6d19d8-5eb9-4021-9158-2f3ebe7a5323 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Image limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 16 21:10:10.877895 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-6b6d19d8-5eb9-4021-9158-2f3ebe7a5323 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Flavor pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 16 21:10:10.878311 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-6b6d19d8-5eb9-4021-9158-2f3ebe7a5323 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Image pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 16 21:10:10.878723 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-6b6d19d8-5eb9-4021-9158-2f3ebe7a5323 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 16 21:10:10.892092 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6b6d19d8-5eb9-4021-9158-2f3ebe7a5323 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:10.892546 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6b6d19d8-5eb9-4021-9158-2f3ebe7a5323 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:10.893357 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6b6d19d8-5eb9-4021-9158-2f3ebe7a5323 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:10.894469 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6b6d19d8-5eb9-4021-9158-2f3ebe7a5323 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:10.906417 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-b4069c84-78c8-422f-af60-0d8fec3849af tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'abeb7020019d41a18de42956d706a011', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '649babe54d104436af461f1ee4767001', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:10.915102 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-6b6d19d8-5eb9-4021-9158-2f3ebe7a5323 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Getting quotas for project 9b769bf8f3ea4e368bf68de1df6538f9. Resources: {'ram', 'instances', 'cores'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:10:10.919035 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-b4069c84-78c8-422f-af60-0d8fec3849af tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] 10.222.0.16 "GET /compute/v2.1/servers/7d624313-0c12-4da2-a95c-3903efae7608" status: 200 len: 1318 microversion: 2.1 time: 0.278563 Jul 16 21:10:10.924737 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 300/599] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:10:10 2022] GET /compute/v2.1/servers/7d624313-0c12-4da2-a95c-3903efae7608 => generated 1318 bytes in 280 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:10:10.924938 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-6b6d19d8-5eb9-4021-9158-2f3ebe7a5323 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Getting quotas for user 6a69f9dee03d43beaafa966216b9e289 and project 9b769bf8f3ea4e368bf68de1df6538f9. Resources: {'ram', 'instances', 'cores'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:10:10.926685 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-2315a75c-f91e-4202-9068-6984ef179140 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:10.927405 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-2315a75c-f91e-4202-9068-6984ef179140 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: f98a9762-a67c-4e95-85b8-727c81bdb03a] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:10.934954 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2315a75c-f91e-4202-9068-6984ef179140 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:10.935446 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2315a75c-f91e-4202-9068-6984ef179140 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:10.936235 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2315a75c-f91e-4202-9068-6984ef179140 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:10.936906 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2315a75c-f91e-4202-9068-6984ef179140 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:10.944786 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-6b6d19d8-5eb9-4021-9158-2f3ebe7a5323 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Going to run 1 instances... {{(pid=138681) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 16 21:10:10.947272 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-6b6d19d8-5eb9-4021-9158-2f3ebe7a5323 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] [instance: 4ee3ffbe-c16a-4847-a343-469c645e4b25] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138681) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 16 21:10:11.081687 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-2315a75c-f91e-4202-9068-6984ef179140 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b68dacc4b8463ba07fae6372131117', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55a19b7a4682457a9132f5fc80110712', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:11.090854 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-2315a75c-f91e-4202-9068-6984ef179140 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "GET /compute/v2.1/servers/f98a9762-a67c-4e95-85b8-727c81bdb03a" status: 200 len: 1462 microversion: 2.1 time: 0.168760 Jul 16 21:10:11.098349 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-6b6d19d8-5eb9-4021-9158-2f3ebe7a5323 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.897027 Jul 16 21:10:11.100332 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 300/600] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 21:10:10 2022] POST /compute/v2.1/servers => generated 380 bytes in 900 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 16 21:10:11.102754 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 301/601] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:10:10 2022] GET /compute/v2.1/servers/f98a9762-a67c-4e95-85b8-727c81bdb03a => generated 1462 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:10:11.110561 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-79334ea7-3699-4cd7-83c0-9a65abc0af45 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Action: 'action', calling method: >, body: {"resize": {"flavorRef": "451"}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:10:11.112046 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-79334ea7-3699-4cd7-83c0-9a65abc0af45 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: f98a9762-a67c-4e95-85b8-727c81bdb03a] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:11.124177 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-79334ea7-3699-4cd7-83c0-9a65abc0af45 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:11.125498 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-31f7f903-c8b5-43b7-871b-5d0bcf114157 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:11.125706 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-79334ea7-3699-4cd7-83c0-9a65abc0af45 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:11.125985 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-31f7f903-c8b5-43b7-871b-5d0bcf114157 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] [instance: 4ee3ffbe-c16a-4847-a343-469c645e4b25] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:11.127213 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-79334ea7-3699-4cd7-83c0-9a65abc0af45 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:11.127916 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-79334ea7-3699-4cd7-83c0-9a65abc0af45 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:11.287893 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-31f7f903-c8b5-43b7-871b-5d0bcf114157 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a69f9dee03d43beaafa966216b9e289', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9b769bf8f3ea4e368bf68de1df6538f9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:11.302410 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-31f7f903-c8b5-43b7-871b-5d0bcf114157 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] 10.222.0.16 "GET /compute/v2.1/servers/4ee3ffbe-c16a-4847-a343-469c645e4b25" status: 200 len: 1211 microversion: 2.1 time: 0.194011 Jul 16 21:10:11.303125 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 302/602] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:10:11 2022] GET /compute/v2.1/servers/4ee3ffbe-c16a-4847-a343-469c645e4b25 => generated 1211 bytes in 195 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:10:11.307972 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-a448418d-1a05-4b79-a04f-abcd68411fa7 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:11.308600 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-a448418d-1a05-4b79-a04f-abcd68411fa7 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] [instance: 67266541-b43e-414c-a744-9c32c6f53975] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:11.323462 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a448418d-1a05-4b79-a04f-abcd68411fa7 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:11.324262 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a448418d-1a05-4b79-a04f-abcd68411fa7 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:11.325279 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a448418d-1a05-4b79-a04f-abcd68411fa7 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:11.325936 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a448418d-1a05-4b79-a04f-abcd68411fa7 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:11.481476 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-79334ea7-3699-4cd7-83c0-9a65abc0af45 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Policy check for compute:servers:resize:cross_cell failed with credentials {'is_admin': False, 'user_id': 'a9b68dacc4b8463ba07fae6372131117', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55a19b7a4682457a9132f5fc80110712', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:11.489447 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-79334ea7-3699-4cd7-83c0-9a65abc0af45 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: f98a9762-a67c-4e95-85b8-727c81bdb03a] Old instance type m1.heat2, new instance type m1.heat {{(pid=138681) resize /opt/stack/nova/nova/compute/api.py:4166}} Jul 16 21:10:11.498281 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-79334ea7-3699-4cd7-83c0-9a65abc0af45 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Flavor limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 16 21:10:11.498969 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-79334ea7-3699-4cd7-83c0-9a65abc0af45 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Image limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 16 21:10:11.501015 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-79334ea7-3699-4cd7-83c0-9a65abc0af45 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Flavor pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 16 21:10:11.501658 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-79334ea7-3699-4cd7-83c0-9a65abc0af45 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Image pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 16 21:10:11.502477 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-79334ea7-3699-4cd7-83c0-9a65abc0af45 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 16 21:10:11.663607 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-a448418d-1a05-4b79-a04f-abcd68411fa7 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0d72626f465b4f37ad3b18bb945d4000', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fc1c3463f9444119af9d18f85bdd703d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:11.674548 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-a448418d-1a05-4b79-a04f-abcd68411fa7 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] 10.222.0.16 "GET /compute/v2.1/servers/67266541-b43e-414c-a744-9c32c6f53975" status: 200 len: 1465 microversion: 2.1 time: 0.370437 Jul 16 21:10:11.675165 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 303/603] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:10:11 2022] GET /compute/v2.1/servers/67266541-b43e-414c-a744-9c32c6f53975 => generated 1465 bytes in 372 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:10:11.680035 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-cd684797-b7de-4000-8d37-4d449d9368ee tempest-ServerDiagnosticsNegativeTest-1844467799 tempest-ServerDiagnosticsNegativeTest-1844467799-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:11.680910 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-cd684797-b7de-4000-8d37-4d449d9368ee tempest-ServerDiagnosticsNegativeTest-1844467799 tempest-ServerDiagnosticsNegativeTest-1844467799-project] [instance: 1e4229cb-4ed9-48c3-b723-28619ae7045c] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:11.687528 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cd684797-b7de-4000-8d37-4d449d9368ee tempest-ServerDiagnosticsNegativeTest-1844467799 tempest-ServerDiagnosticsNegativeTest-1844467799-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:11.687943 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cd684797-b7de-4000-8d37-4d449d9368ee tempest-ServerDiagnosticsNegativeTest-1844467799 tempest-ServerDiagnosticsNegativeTest-1844467799-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:11.688806 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cd684797-b7de-4000-8d37-4d449d9368ee tempest-ServerDiagnosticsNegativeTest-1844467799 tempest-ServerDiagnosticsNegativeTest-1844467799-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:11.689315 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cd684797-b7de-4000-8d37-4d449d9368ee tempest-ServerDiagnosticsNegativeTest-1844467799 tempest-ServerDiagnosticsNegativeTest-1844467799-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:11.864812 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-cd684797-b7de-4000-8d37-4d449d9368ee tempest-ServerDiagnosticsNegativeTest-1844467799 tempest-ServerDiagnosticsNegativeTest-1844467799-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9c428093bcf445a6aea6ddb209e6e5cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5f9cdca4179c4875a4be57cf8a9f46a8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:11.877807 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-cd684797-b7de-4000-8d37-4d449d9368ee tempest-ServerDiagnosticsNegativeTest-1844467799 tempest-ServerDiagnosticsNegativeTest-1844467799-project] 10.222.0.16 "GET /compute/v2.1/servers/1e4229cb-4ed9-48c3-b723-28619ae7045c" status: 200 len: 1323 microversion: 2.1 time: 0.201259 Jul 16 21:10:11.888240 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 304/604] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:10:11 2022] GET /compute/v2.1/servers/1e4229cb-4ed9-48c3-b723-28619ae7045c => generated 1323 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:10:11.895943 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-54badba8-ab3a-45f5-808a-99ffecdfaf69 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:11.900915 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-79334ea7-3699-4cd7-83c0-9a65abc0af45 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "POST /compute/v2.1/servers/f98a9762-a67c-4e95-85b8-727c81bdb03a/action" status: 202 len: 0 microversion: 2.1 time: 0.793213 Jul 16 21:10:11.903969 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 301/605] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 21:10:11 2022] POST /compute/v2.1/servers/f98a9762-a67c-4e95-85b8-727c81bdb03a/action => generated 0 bytes in 801 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 21:10:11.913099 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-54badba8-ab3a-45f5-808a-99ffecdfaf69 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] [instance: 8872e8cf-593e-4a7a-940d-e0c8159d4ad7] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:11.917540 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-48ebb305-a705-4514-a0a8-4dc63e46d583 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:11.921147 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-48ebb305-a705-4514-a0a8-4dc63e46d583 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] [instance: cf53f7cd-deae-4b34-bf8e-3afef98b2ec6] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:11.928038 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-54badba8-ab3a-45f5-808a-99ffecdfaf69 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:11.928507 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-54badba8-ab3a-45f5-808a-99ffecdfaf69 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:11.929278 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-54badba8-ab3a-45f5-808a-99ffecdfaf69 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:11.931280 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-54badba8-ab3a-45f5-808a-99ffecdfaf69 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:11.937204 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-48ebb305-a705-4514-a0a8-4dc63e46d583 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:11.938880 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-48ebb305-a705-4514-a0a8-4dc63e46d583 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:11.941280 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-48ebb305-a705-4514-a0a8-4dc63e46d583 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:11.945863 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-48ebb305-a705-4514-a0a8-4dc63e46d583 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:12.051039 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-48ebb305-a705-4514-a0a8-4dc63e46d583 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ba60685a2001459cbfb531d95a6b296e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb6deae6b08e4ed8b15dc6edfe1d38d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:12.065192 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-48ebb305-a705-4514-a0a8-4dc63e46d583 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ba60685a2001459cbfb531d95a6b296e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb6deae6b08e4ed8b15dc6edfe1d38d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:12.067555 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-48ebb305-a705-4514-a0a8-4dc63e46d583 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ba60685a2001459cbfb531d95a6b296e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb6deae6b08e4ed8b15dc6edfe1d38d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:12.074742 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-48ebb305-a705-4514-a0a8-4dc63e46d583 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] 10.222.0.16 "GET /compute/v2.1/servers/cf53f7cd-deae-4b34-bf8e-3afef98b2ec6" status: 200 len: 1465 microversion: 2.75 time: 0.169140 Jul 16 21:10:12.087978 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 302/606] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:10:11 2022] GET /compute/v2.1/servers/cf53f7cd-deae-4b34-bf8e-3afef98b2ec6 => generated 1465 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:10:12.096000 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-78bc41d2-db80-4547-b4cd-3b14a86decf4 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:12.096317 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-78bc41d2-db80-4547-b4cd-3b14a86decf4 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: f98a9762-a67c-4e95-85b8-727c81bdb03a] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:12.103630 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-78bc41d2-db80-4547-b4cd-3b14a86decf4 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:12.104171 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-78bc41d2-db80-4547-b4cd-3b14a86decf4 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:12.104952 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-78bc41d2-db80-4547-b4cd-3b14a86decf4 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:12.105390 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-78bc41d2-db80-4547-b4cd-3b14a86decf4 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:12.129206 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-54badba8-ab3a-45f5-808a-99ffecdfaf69 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ef9cbce2b2e64665b7f50767342dcb46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0ee02e4c8b448f2b77798ae504129b9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:12.138121 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-54badba8-ab3a-45f5-808a-99ffecdfaf69 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] 10.222.0.16 "GET /compute/v2.1/servers/8872e8cf-593e-4a7a-940d-e0c8159d4ad7" status: 200 len: 1389 microversion: 2.1 time: 0.248310 Jul 16 21:10:12.146623 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 305/607] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:10:11 2022] GET /compute/v2.1/servers/8872e8cf-593e-4a7a-940d-e0c8159d4ad7 => generated 1389 bytes in 256 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:10:12.158061 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-69d14015-d23a-4c7e-8e43-fcc707df28d0 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:12.159731 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-69d14015-d23a-4c7e-8e43-fcc707df28d0 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] [instance: 7d624313-0c12-4da2-a95c-3903efae7608] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:12.167577 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-69d14015-d23a-4c7e-8e43-fcc707df28d0 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:12.168443 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-69d14015-d23a-4c7e-8e43-fcc707df28d0 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:12.172820 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-69d14015-d23a-4c7e-8e43-fcc707df28d0 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:12.173761 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-69d14015-d23a-4c7e-8e43-fcc707df28d0 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:12.498688 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-78bc41d2-db80-4547-b4cd-3b14a86decf4 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b68dacc4b8463ba07fae6372131117', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55a19b7a4682457a9132f5fc80110712', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:12.521575 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-78bc41d2-db80-4547-b4cd-3b14a86decf4 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "GET /compute/v2.1/servers/f98a9762-a67c-4e95-85b8-727c81bdb03a" status: 200 len: 1471 microversion: 2.1 time: 0.444210 Jul 16 21:10:12.522274 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 303/608] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:10:12 2022] GET /compute/v2.1/servers/f98a9762-a67c-4e95-85b8-727c81bdb03a => generated 1471 bytes in 446 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:10:12.535327 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-a712673a-6b11-4cd3-9512-da9c4fcfde05 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:12.536542 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-a712673a-6b11-4cd3-9512-da9c4fcfde05 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] [instance: 4ee3ffbe-c16a-4847-a343-469c645e4b25] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:12.554795 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a712673a-6b11-4cd3-9512-da9c4fcfde05 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:12.557309 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a712673a-6b11-4cd3-9512-da9c4fcfde05 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:12.558327 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a712673a-6b11-4cd3-9512-da9c4fcfde05 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:12.558774 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a712673a-6b11-4cd3-9512-da9c4fcfde05 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:12.574420 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-69d14015-d23a-4c7e-8e43-fcc707df28d0 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'abeb7020019d41a18de42956d706a011', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '649babe54d104436af461f1ee4767001', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:12.589814 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-69d14015-d23a-4c7e-8e43-fcc707df28d0 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] 10.222.0.16 "GET /compute/v2.1/servers/7d624313-0c12-4da2-a95c-3903efae7608" status: 200 len: 1318 microversion: 2.1 time: 0.441720 Jul 16 21:10:12.601757 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 306/609] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:10:12 2022] GET /compute/v2.1/servers/7d624313-0c12-4da2-a95c-3903efae7608 => generated 1318 bytes in 454 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:10:12.607834 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-d421a358-e6b7-4627-839d-52ab24164175 req-f11a05de-ea66-41c8-a342-ed91dcfe3b68 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "67266541-b43e-414c-a744-9c32c6f53975", "name": "network-vif-deleted", "tag": "f15f3108-545f-4afa-9b93-b7dbd6384d3a"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:10:12.617340 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-d421a358-e6b7-4627-839d-52ab24164175 req-f11a05de-ea66-41c8-a342-ed91dcfe3b68 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:12.618027 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-d421a358-e6b7-4627-839d-52ab24164175 req-f11a05de-ea66-41c8-a342-ed91dcfe3b68 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:12.649548 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-d421a358-e6b7-4627-839d-52ab24164175 req-f11a05de-ea66-41c8-a342-ed91dcfe3b68 service nova] Creating event network-vif-deleted:f15f3108-545f-4afa-9b93-b7dbd6384d3a for instance 67266541-b43e-414c-a744-9c32c6f53975 on nch2-master-1 Jul 16 21:10:12.662495 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-d421a358-e6b7-4627-839d-52ab24164175 req-f11a05de-ea66-41c8-a342-ed91dcfe3b68 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.059232 Jul 16 21:10:12.666654 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 307/610] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:10:12 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 64 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:10:12.688081 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-a712673a-6b11-4cd3-9512-da9c4fcfde05 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a69f9dee03d43beaafa966216b9e289', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9b769bf8f3ea4e368bf68de1df6538f9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:12.691175 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-e3d6f8d4-ac98-4e44-b577-a147232427fc req-dc43f0c9-86bd-461a-b609-91c5f2338102 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "1e4229cb-4ed9-48c3-b723-28619ae7045c", "name": "network-vif-plugged", "status": "completed", "tag": "d94b5111-1b85-496e-b76b-7338bb0d2319"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:10:12.692705 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-a712673a-6b11-4cd3-9512-da9c4fcfde05 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] 10.222.0.16 "GET /compute/v2.1/servers/4ee3ffbe-c16a-4847-a343-469c645e4b25" status: 200 len: 1215 microversion: 2.1 time: 0.168036 Jul 16 21:10:12.693293 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 304/611] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:10:12 2022] GET /compute/v2.1/servers/4ee3ffbe-c16a-4847-a343-469c645e4b25 => generated 1215 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:10:12.708658 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-608604ee-64ca-4f89-8e79-072437f8d2ca tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:12.709216 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-608604ee-64ca-4f89-8e79-072437f8d2ca tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] [instance: 67266541-b43e-414c-a744-9c32c6f53975] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:12.712920 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-e3d6f8d4-ac98-4e44-b577-a147232427fc req-dc43f0c9-86bd-461a-b609-91c5f2338102 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:12.714327 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-e3d6f8d4-ac98-4e44-b577-a147232427fc req-dc43f0c9-86bd-461a-b609-91c5f2338102 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:12.720670 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-608604ee-64ca-4f89-8e79-072437f8d2ca tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:12.728454 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-608604ee-64ca-4f89-8e79-072437f8d2ca tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:12.728902 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-608604ee-64ca-4f89-8e79-072437f8d2ca tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:12.729362 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-608604ee-64ca-4f89-8e79-072437f8d2ca tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:12.733163 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-e3d6f8d4-ac98-4e44-b577-a147232427fc req-dc43f0c9-86bd-461a-b609-91c5f2338102 service nova] Creating event network-vif-plugged:d94b5111-1b85-496e-b76b-7338bb0d2319 for instance 1e4229cb-4ed9-48c3-b723-28619ae7045c on nch1-master-1 Jul 16 21:10:12.741914 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-e3d6f8d4-ac98-4e44-b577-a147232427fc req-dc43f0c9-86bd-461a-b609-91c5f2338102 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.063445 Jul 16 21:10:12.743208 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 308/612] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:10:12 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 66 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:10:12.807180 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-608604ee-64ca-4f89-8e79-072437f8d2ca tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0d72626f465b4f37ad3b18bb945d4000', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fc1c3463f9444119af9d18f85bdd703d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:12.815306 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-608604ee-64ca-4f89-8e79-072437f8d2ca tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] 10.222.0.16 "GET /compute/v2.1/servers/67266541-b43e-414c-a744-9c32c6f53975" status: 200 len: 1423 microversion: 2.1 time: 0.120934 Jul 16 21:10:12.816082 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 305/613] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:10:12 2022] GET /compute/v2.1/servers/67266541-b43e-414c-a744-9c32c6f53975 => generated 1423 bytes in 122 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:10:12.899250 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-1d6d4e85-b1fa-44e5-8204-f042ef1ec0c4 tempest-ServerDiagnosticsNegativeTest-1844467799 tempest-ServerDiagnosticsNegativeTest-1844467799-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:12.899250 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-1d6d4e85-b1fa-44e5-8204-f042ef1ec0c4 tempest-ServerDiagnosticsNegativeTest-1844467799 tempest-ServerDiagnosticsNegativeTest-1844467799-project] [instance: 1e4229cb-4ed9-48c3-b723-28619ae7045c] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:12.903357 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1d6d4e85-b1fa-44e5-8204-f042ef1ec0c4 tempest-ServerDiagnosticsNegativeTest-1844467799 tempest-ServerDiagnosticsNegativeTest-1844467799-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:12.904042 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1d6d4e85-b1fa-44e5-8204-f042ef1ec0c4 tempest-ServerDiagnosticsNegativeTest-1844467799 tempest-ServerDiagnosticsNegativeTest-1844467799-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:12.904831 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1d6d4e85-b1fa-44e5-8204-f042ef1ec0c4 tempest-ServerDiagnosticsNegativeTest-1844467799 tempest-ServerDiagnosticsNegativeTest-1844467799-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:12.905272 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1d6d4e85-b1fa-44e5-8204-f042ef1ec0c4 tempest-ServerDiagnosticsNegativeTest-1844467799 tempest-ServerDiagnosticsNegativeTest-1844467799-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:13.043638 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-1d6d4e85-b1fa-44e5-8204-f042ef1ec0c4 tempest-ServerDiagnosticsNegativeTest-1844467799 tempest-ServerDiagnosticsNegativeTest-1844467799-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9c428093bcf445a6aea6ddb209e6e5cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5f9cdca4179c4875a4be57cf8a9f46a8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:13.051829 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-1d6d4e85-b1fa-44e5-8204-f042ef1ec0c4 tempest-ServerDiagnosticsNegativeTest-1844467799 tempest-ServerDiagnosticsNegativeTest-1844467799-project] 10.222.0.16 "GET /compute/v2.1/servers/1e4229cb-4ed9-48c3-b723-28619ae7045c" status: 200 len: 1323 microversion: 2.1 time: 0.160535 Jul 16 21:10:13.052516 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 309/614] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:10:12 2022] GET /compute/v2.1/servers/1e4229cb-4ed9-48c3-b723-28619ae7045c => generated 1323 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:10:13.104101 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-a2c6afdb-9dbd-49ad-8fa5-8cb3a020a866 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:13.105082 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-a2c6afdb-9dbd-49ad-8fa5-8cb3a020a866 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] [instance: cf53f7cd-deae-4b34-bf8e-3afef98b2ec6] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:13.112108 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a2c6afdb-9dbd-49ad-8fa5-8cb3a020a866 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:13.112586 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a2c6afdb-9dbd-49ad-8fa5-8cb3a020a866 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:13.113535 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a2c6afdb-9dbd-49ad-8fa5-8cb3a020a866 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:13.113980 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a2c6afdb-9dbd-49ad-8fa5-8cb3a020a866 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:13.160986 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-62b8ea90-b5db-4ce0-87f0-cd696ae45424 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:13.160986 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-62b8ea90-b5db-4ce0-87f0-cd696ae45424 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] [instance: 8872e8cf-593e-4a7a-940d-e0c8159d4ad7] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:13.177824 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-62b8ea90-b5db-4ce0-87f0-cd696ae45424 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:13.178289 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-62b8ea90-b5db-4ce0-87f0-cd696ae45424 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:13.179211 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-62b8ea90-b5db-4ce0-87f0-cd696ae45424 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:13.179718 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-62b8ea90-b5db-4ce0-87f0-cd696ae45424 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:13.268281 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-a2c6afdb-9dbd-49ad-8fa5-8cb3a020a866 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ba60685a2001459cbfb531d95a6b296e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb6deae6b08e4ed8b15dc6edfe1d38d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:13.279283 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-a2c6afdb-9dbd-49ad-8fa5-8cb3a020a866 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ba60685a2001459cbfb531d95a6b296e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb6deae6b08e4ed8b15dc6edfe1d38d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:13.288646 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-a2c6afdb-9dbd-49ad-8fa5-8cb3a020a866 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ba60685a2001459cbfb531d95a6b296e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb6deae6b08e4ed8b15dc6edfe1d38d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:13.296042 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-a2c6afdb-9dbd-49ad-8fa5-8cb3a020a866 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] 10.222.0.16 "GET /compute/v2.1/servers/cf53f7cd-deae-4b34-bf8e-3afef98b2ec6" status: 200 len: 1453 microversion: 2.75 time: 0.196169 Jul 16 21:10:13.309168 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 306/615] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:10:13 2022] GET /compute/v2.1/servers/cf53f7cd-deae-4b34-bf8e-3afef98b2ec6 => generated 1453 bytes in 198 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:10:13.493623 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-62b8ea90-b5db-4ce0-87f0-cd696ae45424 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ef9cbce2b2e64665b7f50767342dcb46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0ee02e4c8b448f2b77798ae504129b9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:13.503685 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-62b8ea90-b5db-4ce0-87f0-cd696ae45424 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] 10.222.0.16 "GET /compute/v2.1/servers/8872e8cf-593e-4a7a-940d-e0c8159d4ad7" status: 200 len: 1389 microversion: 2.1 time: 0.349531 Jul 16 21:10:13.504324 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 310/616] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:10:13 2022] GET /compute/v2.1/servers/8872e8cf-593e-4a7a-940d-e0c8159d4ad7 => generated 1389 bytes in 351 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:10:13.538965 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-7f703004-6db6-4f83-be72-8d4d3228b23b tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:13.539612 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-7f703004-6db6-4f83-be72-8d4d3228b23b tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: f98a9762-a67c-4e95-85b8-727c81bdb03a] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:13.553781 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7f703004-6db6-4f83-be72-8d4d3228b23b tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:13.554969 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7f703004-6db6-4f83-be72-8d4d3228b23b tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:13.555152 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7f703004-6db6-4f83-be72-8d4d3228b23b tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:13.555483 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7f703004-6db6-4f83-be72-8d4d3228b23b tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:13.611907 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-bc493242-8577-4450-af0b-7ac62c8c6e9d tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:13.613008 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-bc493242-8577-4450-af0b-7ac62c8c6e9d tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] [instance: 7d624313-0c12-4da2-a95c-3903efae7608] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:13.630539 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-bc493242-8577-4450-af0b-7ac62c8c6e9d tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:13.631327 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-bc493242-8577-4450-af0b-7ac62c8c6e9d tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:13.632766 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-bc493242-8577-4450-af0b-7ac62c8c6e9d tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:13.633507 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-bc493242-8577-4450-af0b-7ac62c8c6e9d tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:13.905026 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-bc493242-8577-4450-af0b-7ac62c8c6e9d tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'abeb7020019d41a18de42956d706a011', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '649babe54d104436af461f1ee4767001', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:13.911453 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-7f703004-6db6-4f83-be72-8d4d3228b23b tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b68dacc4b8463ba07fae6372131117', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55a19b7a4682457a9132f5fc80110712', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:13.914463 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-bc493242-8577-4450-af0b-7ac62c8c6e9d tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] 10.222.0.16 "GET /compute/v2.1/servers/7d624313-0c12-4da2-a95c-3903efae7608" status: 200 len: 1318 microversion: 2.1 time: 0.309687 Jul 16 21:10:13.915093 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 311/617] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:10:13 2022] GET /compute/v2.1/servers/7d624313-0c12-4da2-a95c-3903efae7608 => generated 1318 bytes in 311 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:10:13.919701 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-7f703004-6db6-4f83-be72-8d4d3228b23b tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "GET /compute/v2.1/servers/f98a9762-a67c-4e95-85b8-727c81bdb03a" status: 200 len: 1471 microversion: 2.1 time: 0.384651 Jul 16 21:10:13.920335 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 307/618] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:10:13 2022] GET /compute/v2.1/servers/f98a9762-a67c-4e95-85b8-727c81bdb03a => generated 1471 bytes in 386 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:10:13.926599 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-486da7fb-7dfc-4eca-8809-8981fa58aa0a tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:13.927643 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-486da7fb-7dfc-4eca-8809-8981fa58aa0a tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] [instance: 4ee3ffbe-c16a-4847-a343-469c645e4b25] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:13.935176 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-486da7fb-7dfc-4eca-8809-8981fa58aa0a tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:13.935858 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-486da7fb-7dfc-4eca-8809-8981fa58aa0a tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:13.937844 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-486da7fb-7dfc-4eca-8809-8981fa58aa0a tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:13.938630 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-486da7fb-7dfc-4eca-8809-8981fa58aa0a tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:13.940317 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-c50a4cd3-01b1-4240-8bc9-7cf44730bfaa tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:13.941057 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-c50a4cd3-01b1-4240-8bc9-7cf44730bfaa tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] [instance: 67266541-b43e-414c-a744-9c32c6f53975] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:13.948835 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c50a4cd3-01b1-4240-8bc9-7cf44730bfaa tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:13.949375 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c50a4cd3-01b1-4240-8bc9-7cf44730bfaa tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:13.950605 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c50a4cd3-01b1-4240-8bc9-7cf44730bfaa tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:13.951138 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c50a4cd3-01b1-4240-8bc9-7cf44730bfaa tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:14.032290 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-486da7fb-7dfc-4eca-8809-8981fa58aa0a tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a69f9dee03d43beaafa966216b9e289', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9b769bf8f3ea4e368bf68de1df6538f9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:14.040367 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-486da7fb-7dfc-4eca-8809-8981fa58aa0a tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] 10.222.0.16 "GET /compute/v2.1/servers/4ee3ffbe-c16a-4847-a343-469c645e4b25" status: 200 len: 1207 microversion: 2.1 time: 0.119641 Jul 16 21:10:14.051160 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 312/619] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:10:13 2022] GET /compute/v2.1/servers/4ee3ffbe-c16a-4847-a343-469c645e4b25 => generated 1207 bytes in 131 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:10:14.069799 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-f14f94b4-96d2-4294-8fad-5113162b57c1 tempest-ServerDiagnosticsNegativeTest-1844467799 tempest-ServerDiagnosticsNegativeTest-1844467799-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:14.070433 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-f14f94b4-96d2-4294-8fad-5113162b57c1 tempest-ServerDiagnosticsNegativeTest-1844467799 tempest-ServerDiagnosticsNegativeTest-1844467799-project] [instance: 1e4229cb-4ed9-48c3-b723-28619ae7045c] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:14.077010 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f14f94b4-96d2-4294-8fad-5113162b57c1 tempest-ServerDiagnosticsNegativeTest-1844467799 tempest-ServerDiagnosticsNegativeTest-1844467799-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:14.077769 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f14f94b4-96d2-4294-8fad-5113162b57c1 tempest-ServerDiagnosticsNegativeTest-1844467799 tempest-ServerDiagnosticsNegativeTest-1844467799-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:14.078726 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f14f94b4-96d2-4294-8fad-5113162b57c1 tempest-ServerDiagnosticsNegativeTest-1844467799 tempest-ServerDiagnosticsNegativeTest-1844467799-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:14.079366 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f14f94b4-96d2-4294-8fad-5113162b57c1 tempest-ServerDiagnosticsNegativeTest-1844467799 tempest-ServerDiagnosticsNegativeTest-1844467799-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:14.105011 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=100 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:10:14.149252 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-c50a4cd3-01b1-4240-8bc9-7cf44730bfaa tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0d72626f465b4f37ad3b18bb945d4000', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fc1c3463f9444119af9d18f85bdd703d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:14.165962 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-c50a4cd3-01b1-4240-8bc9-7cf44730bfaa tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] 10.222.0.16 "GET /compute/v2.1/servers/67266541-b43e-414c-a744-9c32c6f53975" status: 200 len: 1295 microversion: 2.1 time: 0.243347 Jul 16 21:10:14.177734 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 308/620] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:10:13 2022] GET /compute/v2.1/servers/67266541-b43e-414c-a744-9c32c6f53975 => generated 1295 bytes in 257 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:10:14.292463 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-f14f94b4-96d2-4294-8fad-5113162b57c1 tempest-ServerDiagnosticsNegativeTest-1844467799 tempest-ServerDiagnosticsNegativeTest-1844467799-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9c428093bcf445a6aea6ddb209e6e5cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5f9cdca4179c4875a4be57cf8a9f46a8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:14.301759 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-f14f94b4-96d2-4294-8fad-5113162b57c1 tempest-ServerDiagnosticsNegativeTest-1844467799 tempest-ServerDiagnosticsNegativeTest-1844467799-project] 10.222.0.16 "GET /compute/v2.1/servers/1e4229cb-4ed9-48c3-b723-28619ae7045c" status: 200 len: 1323 microversion: 2.1 time: 0.235920 Jul 16 21:10:14.311866 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 313/621] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:10:14 2022] GET /compute/v2.1/servers/1e4229cb-4ed9-48c3-b723-28619ae7045c => generated 1323 bytes in 247 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:10:14.317353 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-08f0db79-54fc-4972-8edb-5f19e1cb1084 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:14.318026 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-08f0db79-54fc-4972-8edb-5f19e1cb1084 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] [instance: cf53f7cd-deae-4b34-bf8e-3afef98b2ec6] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:14.324170 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-08f0db79-54fc-4972-8edb-5f19e1cb1084 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:14.324622 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-08f0db79-54fc-4972-8edb-5f19e1cb1084 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:14.325392 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-08f0db79-54fc-4972-8edb-5f19e1cb1084 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:14.325824 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-08f0db79-54fc-4972-8edb-5f19e1cb1084 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:14.415972 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-08f0db79-54fc-4972-8edb-5f19e1cb1084 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ba60685a2001459cbfb531d95a6b296e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb6deae6b08e4ed8b15dc6edfe1d38d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:14.428512 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-08f0db79-54fc-4972-8edb-5f19e1cb1084 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ba60685a2001459cbfb531d95a6b296e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb6deae6b08e4ed8b15dc6edfe1d38d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:14.431001 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-08f0db79-54fc-4972-8edb-5f19e1cb1084 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ba60685a2001459cbfb531d95a6b296e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb6deae6b08e4ed8b15dc6edfe1d38d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:14.436805 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-08f0db79-54fc-4972-8edb-5f19e1cb1084 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] 10.222.0.16 "GET /compute/v2.1/servers/cf53f7cd-deae-4b34-bf8e-3afef98b2ec6" status: 200 len: 1453 microversion: 2.75 time: 0.123673 Jul 16 21:10:14.438422 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 309/622] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:10:14 2022] GET /compute/v2.1/servers/cf53f7cd-deae-4b34-bf8e-3afef98b2ec6 => generated 1453 bytes in 126 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:10:14.522878 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-a38d0fae-3a1a-4890-8930-d42bd6c9e28e tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:14.523520 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-a38d0fae-3a1a-4890-8930-d42bd6c9e28e tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] [instance: 8872e8cf-593e-4a7a-940d-e0c8159d4ad7] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:14.542060 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a38d0fae-3a1a-4890-8930-d42bd6c9e28e tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:14.543337 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a38d0fae-3a1a-4890-8930-d42bd6c9e28e tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:14.544409 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a38d0fae-3a1a-4890-8930-d42bd6c9e28e tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:14.545546 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a38d0fae-3a1a-4890-8930-d42bd6c9e28e tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:14.740417 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-a38d0fae-3a1a-4890-8930-d42bd6c9e28e tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ef9cbce2b2e64665b7f50767342dcb46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0ee02e4c8b448f2b77798ae504129b9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:14.755130 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-a38d0fae-3a1a-4890-8930-d42bd6c9e28e tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] 10.222.0.16 "GET /compute/v2.1/servers/8872e8cf-593e-4a7a-940d-e0c8159d4ad7" status: 200 len: 1389 microversion: 2.1 time: 0.231383 Jul 16 21:10:14.755769 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 314/623] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:10:14 2022] GET /compute/v2.1/servers/8872e8cf-593e-4a7a-940d-e0c8159d4ad7 => generated 1389 bytes in 238 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:10:14.932871 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-7a5bc54f-5391-49db-8d10-ed25f93ec6d7 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:14.933513 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-7a5bc54f-5391-49db-8d10-ed25f93ec6d7 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] [instance: 7d624313-0c12-4da2-a95c-3903efae7608] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:14.941609 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7a5bc54f-5391-49db-8d10-ed25f93ec6d7 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:14.942060 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7a5bc54f-5391-49db-8d10-ed25f93ec6d7 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:14.942635 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-3aa5ab0b-8b25-49d5-a82e-492f6cd3ce5f tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:14.942831 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7a5bc54f-5391-49db-8d10-ed25f93ec6d7 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:14.943297 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7a5bc54f-5391-49db-8d10-ed25f93ec6d7 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:14.943802 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-3aa5ab0b-8b25-49d5-a82e-492f6cd3ce5f tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: f98a9762-a67c-4e95-85b8-727c81bdb03a] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:14.954493 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3aa5ab0b-8b25-49d5-a82e-492f6cd3ce5f tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:14.956991 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3aa5ab0b-8b25-49d5-a82e-492f6cd3ce5f tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:14.957425 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3aa5ab0b-8b25-49d5-a82e-492f6cd3ce5f tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:14.958020 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3aa5ab0b-8b25-49d5-a82e-492f6cd3ce5f tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:15.236973 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-7a5bc54f-5391-49db-8d10-ed25f93ec6d7 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'abeb7020019d41a18de42956d706a011', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '649babe54d104436af461f1ee4767001', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:15.245895 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-7a5bc54f-5391-49db-8d10-ed25f93ec6d7 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] 10.222.0.16 "GET /compute/v2.1/servers/7d624313-0c12-4da2-a95c-3903efae7608" status: 200 len: 1318 microversion: 2.1 time: 0.317001 Jul 16 21:10:15.246488 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 310/624] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:10:14 2022] GET /compute/v2.1/servers/7d624313-0c12-4da2-a95c-3903efae7608 => generated 1318 bytes in 318 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:10:15.250889 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-3aa5ab0b-8b25-49d5-a82e-492f6cd3ce5f tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b68dacc4b8463ba07fae6372131117', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55a19b7a4682457a9132f5fc80110712', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:15.261547 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-75af578f-fd0a-4c19-ac79-94c903c2787e tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:15.261873 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-3aa5ab0b-8b25-49d5-a82e-492f6cd3ce5f tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "GET /compute/v2.1/servers/f98a9762-a67c-4e95-85b8-727c81bdb03a" status: 200 len: 1471 microversion: 2.1 time: 0.324675 Jul 16 21:10:15.262637 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-75af578f-fd0a-4c19-ac79-94c903c2787e tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] [instance: 4ee3ffbe-c16a-4847-a343-469c645e4b25] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:15.264224 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 315/625] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:10:14 2022] GET /compute/v2.1/servers/f98a9762-a67c-4e95-85b8-727c81bdb03a => generated 1471 bytes in 328 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:10:15.272074 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-75af578f-fd0a-4c19-ac79-94c903c2787e tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:15.272765 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-75af578f-fd0a-4c19-ac79-94c903c2787e tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:15.273886 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-75af578f-fd0a-4c19-ac79-94c903c2787e tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:15.274466 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-75af578f-fd0a-4c19-ac79-94c903c2787e tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:15.280312 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-0b5cf3ed-fdfe-4bc6-b723-95af4b9ff016 req-a5b33f37-aebb-473d-a20c-7c45c85b7efd service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "84ee8179-ff94-43b3-97f2-1867bc6420a8", "name": "network-vif-plugged", "status": "completed", "tag": "48ddda29-7627-4cbb-adf0-0975571cbc33"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:10:15.291672 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-0b5cf3ed-fdfe-4bc6-b723-95af4b9ff016 req-a5b33f37-aebb-473d-a20c-7c45c85b7efd service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:15.292266 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-0b5cf3ed-fdfe-4bc6-b723-95af4b9ff016 req-a5b33f37-aebb-473d-a20c-7c45c85b7efd service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:15.308181 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-0b5cf3ed-fdfe-4bc6-b723-95af4b9ff016 req-a5b33f37-aebb-473d-a20c-7c45c85b7efd service nova] Creating event network-vif-plugged:48ddda29-7627-4cbb-adf0-0975571cbc33 for instance 84ee8179-ff94-43b3-97f2-1867bc6420a8 on nch2-master-1 Jul 16 21:10:15.323168 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-0b5cf3ed-fdfe-4bc6-b723-95af4b9ff016 req-a5b33f37-aebb-473d-a20c-7c45c85b7efd service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.057576 Jul 16 21:10:15.324070 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 316/626] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:10:15 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:10:15.352697 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-bac1bf77-b072-440d-9c0c-dca4b4e6f09a tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:15.353398 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-bac1bf77-b072-440d-9c0c-dca4b4e6f09a tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] [instance: 67266541-b43e-414c-a744-9c32c6f53975] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:15.360635 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-bac1bf77-b072-440d-9c0c-dca4b4e6f09a tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:15.361090 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-bac1bf77-b072-440d-9c0c-dca4b4e6f09a tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:15.361878 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-bac1bf77-b072-440d-9c0c-dca4b4e6f09a tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:15.362316 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-bac1bf77-b072-440d-9c0c-dca4b4e6f09a tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:15.389739 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-bac1bf77-b072-440d-9c0c-dca4b4e6f09a tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] HTTP exception thrown: Instance 67266541-b43e-414c-a744-9c32c6f53975 could not be found. Jul 16 21:10:15.392883 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-bac1bf77-b072-440d-9c0c-dca4b4e6f09a tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Request method failure captured: Jul 16 21:10:15.392883 ncdv-master-1 devstack@n-api.service[138680]: request: GET /compute/v2.1/servers/67266541-b43e-414c-a744-9c32c6f53975 HTTP/1.1 Jul 16 21:10:15.392883 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:10:15.392883 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:10:15.392883 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:10:15.392883 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 0 Jul 16 21:10:15.392883 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:10:15.392883 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:10:15.392883 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:10:15.392883 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:10:15.392883 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0yjxC34cYKsuWpx3tkgld3eV-A0-TjeQg_5ds20qHYp2Q6eY4sL39TrWBZ8MuWEkv-IWvSNQ1dzN43zHZHJVYwNvVGidmbekkFV44mT7Ab0o-XbW7DoqN-TrMfFTdGt5XtMhdkfAqddbANBDHWTKcPsdqIMuW-_kx0M-4rvUNYWtOkk Jul 16 21:10:15.392883 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:10:15.392883 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:10:15.392883 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:10:15.392883 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:10:15.392883 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:10:15.392883 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:10:15.392883 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: fc1c3463f9444119af9d18f85bdd703d Jul 16 21:10:15.392883 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-LiveMigrationNegativeTest-742579909 Jul 16 21:10:15.392883 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:10:15.392883 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:10:15.394740 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/fc1c3463f9444119af9d18f85bdd703d"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/fc1c3463f9444119af9d18f85bdd703d"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_fc1c3463f9444119af9d18f85bdd703d"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/fc1c3463f9444119af9d18f85bdd703d"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:10:15.394740 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-LiveMigrationNegativeTest-742579909 Jul 16 21:10:15.394740 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: fc1c3463f9444119af9d18f85bdd703d Jul 16 21:10:15.394740 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-LiveMigrationNegativeTest-742579909 Jul 16 21:10:15.394740 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-LiveMigrationNegativeTest-742579909-project Jul 16 21:10:15.394740 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:10:15.394740 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:10:15.394740 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: 0d72626f465b4f37ad3b18bb945d4000 Jul 16 21:10:15.394740 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-LiveMigrationNegativeTest-742579909-project Jul 16 21:10:15.394740 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 21:10:15.394740 ncdv-master-1 devstack@n-api.service[138680]: exception: Instance 67266541-b43e-414c-a744-9c32c6f53975 could not be found. Jul 16 21:10:15.394740 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': '67266541-b43e-414c-a744-9c32c6f53975'} Jul 16 21:10:15.395779 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:10:15.395779 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:10:15.395779 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:10:15.395779 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:10:15.395779 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:10:15.395779 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:10:15.395779 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:10:15.395779 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:10:15.395779 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:10:15.395779 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:10:15.395779 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 67266541-b43e-414c-a744-9c32c6f53975 could not be found. Jul 16 21:10:15.395779 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:10:15.395779 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:10:15.395779 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:10:15.395779 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:10:15.395779 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:10:15.395779 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:10:15.395779 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:10:15.395779 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:10:15.395779 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:10:15.397366 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:10:15.397366 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:10:15.397366 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:10:15.397366 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:10:15.397366 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:10:15.397366 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:10:15.397366 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:10:15.397366 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 67266541-b43e-414c-a744-9c32c6f53975 could not be found. Jul 16 21:10:15.397366 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:10:15.397366 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:10:15.397366 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:10:15.397366 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:10:15.397366 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:10:15.397366 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:10:15.397366 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:10:15.397366 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:10:15.397366 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 67266541-b43e-414c-a744-9c32c6f53975 could not be found. Jul 16 21:10:15.397366 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:10:15.398861 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-bac1bf77-b072-440d-9c0c-dca4b4e6f09a tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Returning 404 to user: Instance 67266541-b43e-414c-a744-9c32c6f53975 could not be found. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:10:15.398861 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-bac1bf77-b072-440d-9c0c-dca4b4e6f09a tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] 10.222.0.16 "GET /compute/v2.1/servers/67266541-b43e-414c-a744-9c32c6f53975" status: 404 len: 111 microversion: 2.1 time: 0.069404 Jul 16 21:10:15.398861 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 317/627] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:10:15 2022] GET /compute/v2.1/servers/67266541-b43e-414c-a744-9c32c6f53975 => generated 111 bytes in 71 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:10:15.400673 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-55f05742-b7eb-41d6-9c37-0e2e27888cdd tempest-ServerDiagnosticsNegativeTest-1844467799 tempest-ServerDiagnosticsNegativeTest-1844467799-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:15.401267 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-55f05742-b7eb-41d6-9c37-0e2e27888cdd tempest-ServerDiagnosticsNegativeTest-1844467799 tempest-ServerDiagnosticsNegativeTest-1844467799-project] [instance: 1e4229cb-4ed9-48c3-b723-28619ae7045c] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:15.409062 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-55f05742-b7eb-41d6-9c37-0e2e27888cdd tempest-ServerDiagnosticsNegativeTest-1844467799 tempest-ServerDiagnosticsNegativeTest-1844467799-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:15.413120 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-55f05742-b7eb-41d6-9c37-0e2e27888cdd tempest-ServerDiagnosticsNegativeTest-1844467799 tempest-ServerDiagnosticsNegativeTest-1844467799-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:15.413120 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-55f05742-b7eb-41d6-9c37-0e2e27888cdd tempest-ServerDiagnosticsNegativeTest-1844467799 tempest-ServerDiagnosticsNegativeTest-1844467799-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:15.413120 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-55f05742-b7eb-41d6-9c37-0e2e27888cdd tempest-ServerDiagnosticsNegativeTest-1844467799 tempest-ServerDiagnosticsNegativeTest-1844467799-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:15.424674 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-75af578f-fd0a-4c19-ac79-94c903c2787e tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a69f9dee03d43beaafa966216b9e289', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9b769bf8f3ea4e368bf68de1df6538f9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:15.435953 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-75af578f-fd0a-4c19-ac79-94c903c2787e tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] 10.222.0.16 "GET /compute/v2.1/servers/4ee3ffbe-c16a-4847-a343-469c645e4b25" status: 200 len: 1263 microversion: 2.1 time: 0.186553 Jul 16 21:10:15.436801 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 311/628] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:10:15 2022] GET /compute/v2.1/servers/4ee3ffbe-c16a-4847-a343-469c645e4b25 => generated 1263 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:10:15.443625 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-654a0069-e6ed-4c82-ae75-61488ee1a2d5 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:15.444297 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-654a0069-e6ed-4c82-ae75-61488ee1a2d5 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] [instance: 1631f0cc-de67-4c0d-abd0-3e0e11dfb4ed] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:15.450991 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-654a0069-e6ed-4c82-ae75-61488ee1a2d5 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:15.451564 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-654a0069-e6ed-4c82-ae75-61488ee1a2d5 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:15.452402 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-654a0069-e6ed-4c82-ae75-61488ee1a2d5 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:15.453501 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-654a0069-e6ed-4c82-ae75-61488ee1a2d5 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:15.458364 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:10:15.517639 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-654a0069-e6ed-4c82-ae75-61488ee1a2d5 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] [instance: 1631f0cc-de67-4c0d-abd0-3e0e11dfb4ed] Going to try to terminate instance {{(pid=138681) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 16 21:10:15.535774 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell0:SELECT=14 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:10:15.597781 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-55f05742-b7eb-41d6-9c37-0e2e27888cdd tempest-ServerDiagnosticsNegativeTest-1844467799 tempest-ServerDiagnosticsNegativeTest-1844467799-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9c428093bcf445a6aea6ddb209e6e5cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5f9cdca4179c4875a4be57cf8a9f46a8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:15.606362 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-55f05742-b7eb-41d6-9c37-0e2e27888cdd tempest-ServerDiagnosticsNegativeTest-1844467799 tempest-ServerDiagnosticsNegativeTest-1844467799-project] 10.222.0.16 "GET /compute/v2.1/servers/1e4229cb-4ed9-48c3-b723-28619ae7045c" status: 200 len: 1323 microversion: 2.1 time: 0.209821 Jul 16 21:10:15.607041 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 318/629] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:10:15 2022] GET /compute/v2.1/servers/1e4229cb-4ed9-48c3-b723-28619ae7045c => generated 1323 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:10:15.613462 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-64633e2f-066c-4018-b613-bd9bd8eaaefa tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:15.613462 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-64633e2f-066c-4018-b613-bd9bd8eaaefa tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] [instance: cf53f7cd-deae-4b34-bf8e-3afef98b2ec6] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:15.619692 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-64633e2f-066c-4018-b613-bd9bd8eaaefa tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:15.620150 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-64633e2f-066c-4018-b613-bd9bd8eaaefa tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:15.620931 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-64633e2f-066c-4018-b613-bd9bd8eaaefa tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:15.621377 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-64633e2f-066c-4018-b613-bd9bd8eaaefa tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:15.699539 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-64633e2f-066c-4018-b613-bd9bd8eaaefa tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ba60685a2001459cbfb531d95a6b296e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb6deae6b08e4ed8b15dc6edfe1d38d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:15.706051 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-64633e2f-066c-4018-b613-bd9bd8eaaefa tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ba60685a2001459cbfb531d95a6b296e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb6deae6b08e4ed8b15dc6edfe1d38d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:15.707714 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-64633e2f-066c-4018-b613-bd9bd8eaaefa tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ba60685a2001459cbfb531d95a6b296e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb6deae6b08e4ed8b15dc6edfe1d38d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:15.714555 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-64633e2f-066c-4018-b613-bd9bd8eaaefa tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] 10.222.0.16 "GET /compute/v2.1/servers/cf53f7cd-deae-4b34-bf8e-3afef98b2ec6" status: 200 len: 1453 microversion: 2.75 time: 0.106303 Jul 16 21:10:15.715175 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 319/630] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:10:15 2022] GET /compute/v2.1/servers/cf53f7cd-deae-4b34-bf8e-3afef98b2ec6 => generated 1453 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:10:15.773566 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-868e4b06-311d-4857-bfe3-0dca70d09e10 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:15.775461 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-868e4b06-311d-4857-bfe3-0dca70d09e10 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] [instance: 8872e8cf-593e-4a7a-940d-e0c8159d4ad7] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:15.781093 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-868e4b06-311d-4857-bfe3-0dca70d09e10 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:15.781546 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-868e4b06-311d-4857-bfe3-0dca70d09e10 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:15.782365 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-868e4b06-311d-4857-bfe3-0dca70d09e10 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:15.782837 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-868e4b06-311d-4857-bfe3-0dca70d09e10 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:16.001227 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-868e4b06-311d-4857-bfe3-0dca70d09e10 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ef9cbce2b2e64665b7f50767342dcb46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0ee02e4c8b448f2b77798ae504129b9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:16.012655 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-868e4b06-311d-4857-bfe3-0dca70d09e10 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] 10.222.0.16 "GET /compute/v2.1/servers/8872e8cf-593e-4a7a-940d-e0c8159d4ad7" status: 200 len: 1389 microversion: 2.1 time: 0.243533 Jul 16 21:10:16.013793 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 320/631] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:10:15 2022] GET /compute/v2.1/servers/8872e8cf-593e-4a7a-940d-e0c8159d4ad7 => generated 1389 bytes in 245 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:10:16.028664 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-654a0069-e6ed-4c82-ae75-61488ee1a2d5 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] 10.222.0.16 "DELETE /compute/v2.1/servers/1631f0cc-de67-4c0d-abd0-3e0e11dfb4ed" status: 204 len: 0 microversion: 2.1 time: 0.587349 Jul 16 21:10:16.032297 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 312/632] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:10:15 2022] DELETE /compute/v2.1/servers/1631f0cc-de67-4c0d-abd0-3e0e11dfb4ed => generated 0 bytes in 589 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 16 21:10:16.037325 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-7c7c57b0-befb-45eb-aad3-57bdd2ef60ec tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:16.038026 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-7c7c57b0-befb-45eb-aad3-57bdd2ef60ec tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] [instance: 1631f0cc-de67-4c0d-abd0-3e0e11dfb4ed] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:16.044691 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7c7c57b0-befb-45eb-aad3-57bdd2ef60ec tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:16.045506 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7c7c57b0-befb-45eb-aad3-57bdd2ef60ec tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:16.046146 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7c7c57b0-befb-45eb-aad3-57bdd2ef60ec tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:16.047969 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7c7c57b0-befb-45eb-aad3-57bdd2ef60ec tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:16.251427 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-7c7c57b0-befb-45eb-aad3-57bdd2ef60ec tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0d72626f465b4f37ad3b18bb945d4000', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fc1c3463f9444119af9d18f85bdd703d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:16.264910 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-34a1f453-f8be-4a86-81d6-da1f0809981b tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:16.266320 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-34a1f453-f8be-4a86-81d6-da1f0809981b tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] [instance: 7d624313-0c12-4da2-a95c-3903efae7608] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:16.271046 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-7c7c57b0-befb-45eb-aad3-57bdd2ef60ec tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] 10.222.0.16 "GET /compute/v2.1/servers/1631f0cc-de67-4c0d-abd0-3e0e11dfb4ed" status: 200 len: 1474 microversion: 2.1 time: 0.238354 Jul 16 21:10:16.271848 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 321/633] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:10:16 2022] GET /compute/v2.1/servers/1631f0cc-de67-4c0d-abd0-3e0e11dfb4ed => generated 1474 bytes in 240 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:10:16.288037 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-0dd0a5bd-daff-41ff-a239-200d9856b625 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:16.288573 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-0dd0a5bd-daff-41ff-a239-200d9856b625 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: f98a9762-a67c-4e95-85b8-727c81bdb03a] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:16.294873 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-34a1f453-f8be-4a86-81d6-da1f0809981b tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:16.295606 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-34a1f453-f8be-4a86-81d6-da1f0809981b tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:16.297096 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-34a1f453-f8be-4a86-81d6-da1f0809981b tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:16.297774 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-34a1f453-f8be-4a86-81d6-da1f0809981b tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:16.316523 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0dd0a5bd-daff-41ff-a239-200d9856b625 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:16.317109 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0dd0a5bd-daff-41ff-a239-200d9856b625 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:16.317923 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0dd0a5bd-daff-41ff-a239-200d9856b625 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:16.318554 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0dd0a5bd-daff-41ff-a239-200d9856b625 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:16.536032 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-0dd0a5bd-daff-41ff-a239-200d9856b625 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b68dacc4b8463ba07fae6372131117', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55a19b7a4682457a9132f5fc80110712', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:16.538158 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-34a1f453-f8be-4a86-81d6-da1f0809981b tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'abeb7020019d41a18de42956d706a011', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '649babe54d104436af461f1ee4767001', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:16.544843 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-0dd0a5bd-daff-41ff-a239-200d9856b625 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "GET /compute/v2.1/servers/f98a9762-a67c-4e95-85b8-727c81bdb03a" status: 200 len: 1471 microversion: 2.1 time: 0.260670 Jul 16 21:10:16.555064 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 322/634] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:10:16 2022] GET /compute/v2.1/servers/f98a9762-a67c-4e95-85b8-727c81bdb03a => generated 1471 bytes in 271 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:10:16.560285 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-ef31aaf0-bedc-4f02-a4c6-ecc3ce652fe5 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:16.561096 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-ef31aaf0-bedc-4f02-a4c6-ecc3ce652fe5 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] [instance: 4ee3ffbe-c16a-4847-a343-469c645e4b25] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:16.566197 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-34a1f453-f8be-4a86-81d6-da1f0809981b tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] 10.222.0.16 "GET /compute/v2.1/servers/7d624313-0c12-4da2-a95c-3903efae7608" status: 200 len: 1318 microversion: 2.1 time: 0.305583 Jul 16 21:10:16.568030 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ef31aaf0-bedc-4f02-a4c6-ecc3ce652fe5 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:16.568436 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ef31aaf0-bedc-4f02-a4c6-ecc3ce652fe5 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:16.569105 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 313/635] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:10:16 2022] GET /compute/v2.1/servers/7d624313-0c12-4da2-a95c-3903efae7608 => generated 1318 bytes in 309 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:10:16.569224 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ef31aaf0-bedc-4f02-a4c6-ecc3ce652fe5 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:16.569656 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ef31aaf0-bedc-4f02-a4c6-ecc3ce652fe5 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:16.626857 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-4caa4e22-7c66-4f3b-856c-120f89f80875 tempest-ServerDiagnosticsNegativeTest-1844467799 tempest-ServerDiagnosticsNegativeTest-1844467799-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:16.629335 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-4caa4e22-7c66-4f3b-856c-120f89f80875 tempest-ServerDiagnosticsNegativeTest-1844467799 tempest-ServerDiagnosticsNegativeTest-1844467799-project] [instance: 1e4229cb-4ed9-48c3-b723-28619ae7045c] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:16.638015 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4caa4e22-7c66-4f3b-856c-120f89f80875 tempest-ServerDiagnosticsNegativeTest-1844467799 tempest-ServerDiagnosticsNegativeTest-1844467799-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:16.638701 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4caa4e22-7c66-4f3b-856c-120f89f80875 tempest-ServerDiagnosticsNegativeTest-1844467799 tempest-ServerDiagnosticsNegativeTest-1844467799-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:16.642476 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4caa4e22-7c66-4f3b-856c-120f89f80875 tempest-ServerDiagnosticsNegativeTest-1844467799 tempest-ServerDiagnosticsNegativeTest-1844467799-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:16.643023 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4caa4e22-7c66-4f3b-856c-120f89f80875 tempest-ServerDiagnosticsNegativeTest-1844467799 tempest-ServerDiagnosticsNegativeTest-1844467799-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:16.652703 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-ef31aaf0-bedc-4f02-a4c6-ecc3ce652fe5 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a69f9dee03d43beaafa966216b9e289', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9b769bf8f3ea4e368bf68de1df6538f9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:16.670587 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-ef31aaf0-bedc-4f02-a4c6-ecc3ce652fe5 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] 10.222.0.16 "GET /compute/v2.1/servers/4ee3ffbe-c16a-4847-a343-469c645e4b25" status: 200 len: 1269 microversion: 2.1 time: 0.113981 Jul 16 21:10:16.681048 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 323/636] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:10:16 2022] GET /compute/v2.1/servers/4ee3ffbe-c16a-4847-a343-469c645e4b25 => generated 1269 bytes in 116 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:10:16.735581 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-7be638ce-4ff5-4212-a957-18e6315a3ed5 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:16.736243 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-7be638ce-4ff5-4212-a957-18e6315a3ed5 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] [instance: cf53f7cd-deae-4b34-bf8e-3afef98b2ec6] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:16.742997 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7be638ce-4ff5-4212-a957-18e6315a3ed5 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:16.743643 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7be638ce-4ff5-4212-a957-18e6315a3ed5 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:16.744563 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7be638ce-4ff5-4212-a957-18e6315a3ed5 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:16.745105 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7be638ce-4ff5-4212-a957-18e6315a3ed5 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:16.832600 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-7be638ce-4ff5-4212-a957-18e6315a3ed5 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ba60685a2001459cbfb531d95a6b296e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb6deae6b08e4ed8b15dc6edfe1d38d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:16.842636 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-7be638ce-4ff5-4212-a957-18e6315a3ed5 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ba60685a2001459cbfb531d95a6b296e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb6deae6b08e4ed8b15dc6edfe1d38d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:16.844893 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-7be638ce-4ff5-4212-a957-18e6315a3ed5 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ba60685a2001459cbfb531d95a6b296e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb6deae6b08e4ed8b15dc6edfe1d38d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:16.851798 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-7be638ce-4ff5-4212-a957-18e6315a3ed5 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] 10.222.0.16 "GET /compute/v2.1/servers/cf53f7cd-deae-4b34-bf8e-3afef98b2ec6" status: 200 len: 1438 microversion: 2.75 time: 0.120443 Jul 16 21:10:16.853839 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 324/637] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:10:16 2022] GET /compute/v2.1/servers/cf53f7cd-deae-4b34-bf8e-3afef98b2ec6 => generated 1438 bytes in 122 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 16 21:10:16.858211 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-4caa4e22-7c66-4f3b-856c-120f89f80875 tempest-ServerDiagnosticsNegativeTest-1844467799 tempest-ServerDiagnosticsNegativeTest-1844467799-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9c428093bcf445a6aea6ddb209e6e5cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5f9cdca4179c4875a4be57cf8a9f46a8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:16.884388 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-4caa4e22-7c66-4f3b-856c-120f89f80875 tempest-ServerDiagnosticsNegativeTest-1844467799 tempest-ServerDiagnosticsNegativeTest-1844467799-project] 10.222.0.16 "GET /compute/v2.1/servers/1e4229cb-4ed9-48c3-b723-28619ae7045c" status: 200 len: 1472 microversion: 2.1 time: 0.263107 Jul 16 21:10:16.887800 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 314/638] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:10:16 2022] GET /compute/v2.1/servers/1e4229cb-4ed9-48c3-b723-28619ae7045c => generated 1472 bytes in 265 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:10:16.919062 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-617f8f83-d977-4c23-88fe-b7656b6af465 tempest-ServerDiagnosticsNegativeTest-1844467799 tempest-ServerDiagnosticsNegativeTest-1844467799-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:16.919600 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-617f8f83-d977-4c23-88fe-b7656b6af465 tempest-ServerDiagnosticsNegativeTest-1844467799 tempest-ServerDiagnosticsNegativeTest-1844467799-project] [instance: 1e4229cb-4ed9-48c3-b723-28619ae7045c] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:16.931950 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-617f8f83-d977-4c23-88fe-b7656b6af465 tempest-ServerDiagnosticsNegativeTest-1844467799 tempest-ServerDiagnosticsNegativeTest-1844467799-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:16.932766 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-617f8f83-d977-4c23-88fe-b7656b6af465 tempest-ServerDiagnosticsNegativeTest-1844467799 tempest-ServerDiagnosticsNegativeTest-1844467799-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:16.934143 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-617f8f83-d977-4c23-88fe-b7656b6af465 tempest-ServerDiagnosticsNegativeTest-1844467799 tempest-ServerDiagnosticsNegativeTest-1844467799-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:16.934688 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-617f8f83-d977-4c23-88fe-b7656b6af465 tempest-ServerDiagnosticsNegativeTest-1844467799 tempest-ServerDiagnosticsNegativeTest-1844467799-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:16.996948 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-617f8f83-d977-4c23-88fe-b7656b6af465 tempest-ServerDiagnosticsNegativeTest-1844467799 tempest-ServerDiagnosticsNegativeTest-1844467799-project] Policy check for os_compute_api:os-server-diagnostics failed with credentials {'is_admin': False, 'user_id': '9c428093bcf445a6aea6ddb209e6e5cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5f9cdca4179c4875a4be57cf8a9f46a8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:16.999510 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-617f8f83-d977-4c23-88fe-b7656b6af465 tempest-ServerDiagnosticsNegativeTest-1844467799 tempest-ServerDiagnosticsNegativeTest-1844467799-project] Request method failure captured: Jul 16 21:10:16.999510 ncdv-master-1 devstack@n-api.service[138681]: request: GET /compute/v2.1/servers/1e4229cb-4ed9-48c3-b723-28619ae7045c/diagnostics HTTP/1.1 Jul 16 21:10:16.999510 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:10:16.999510 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:10:16.999510 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:10:16.999510 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 21:10:16.999510 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:10:16.999510 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:10:16.999510 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:10:16.999510 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:10:16.999510 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0ykfP999Jj_jyZ6OBjYyR9SL49JtWK62ZihwPNiXI5Yi1et43fvJNyoNSC7033r8QSoqpzzSVQA9-4iHQQzaBbmCSCKh2HYjy_dBpUIRYGIrPQz8x0mV49gXL-Nbi3yYEcmJ9PEAz83Nffjt3otj__V_Qej84Zgo99lmGAlv44iak0w Jul 16 21:10:16.999510 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:10:16.999510 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:10:16.999510 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:10:16.999510 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:10:16.999510 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:10:16.999510 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:10:16.999510 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: 5f9cdca4179c4875a4be57cf8a9f46a8 Jul 16 21:10:16.999510 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-ServerDiagnosticsNegativeTest-1844467799 Jul 16 21:10:16.999510 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:10:16.999510 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:10:17.000869 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/5f9cdca4179c4875a4be57cf8a9f46a8"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/5f9cdca4179c4875a4be57cf8a9f46a8"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_5f9cdca4179c4875a4be57cf8a9f46a8"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/5f9cdca4179c4875a4be57cf8a9f46a8"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:10:17.000869 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-ServerDiagnosticsNegativeTest-1844467799 Jul 16 21:10:17.000869 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: 5f9cdca4179c4875a4be57cf8a9f46a8 Jul 16 21:10:17.000869 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-ServerDiagnosticsNegativeTest-1844467799 Jul 16 21:10:17.000869 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-ServerDiagnosticsNegativeTest-1844467799-project Jul 16 21:10:17.000869 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:10:17.000869 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:10:17.000869 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: 9c428093bcf445a6aea6ddb209e6e5cf Jul 16 21:10:17.000869 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-ServerDiagnosticsNegativeTest-1844467799-project Jul 16 21:10:17.000869 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:10:17.000869 ncdv-master-1 devstack@n-api.service[138681]: exception: Policy doesn't allow os_compute_api:os-server-diagnostics to be performed. Jul 16 21:10:17.001669 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'server_id': '1e4229cb-4ed9-48c3-b723-28619ae7045c'} Jul 16 21:10:17.001669 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:10:17.001669 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:10:17.001669 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:10:17.001669 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:10:17.001669 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:10:17.001669 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:10:17.001669 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:10:17.001669 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:10:17.001669 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/server_diagnostics.py", line 38, in index Jul 16 21:10:17.001669 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi context.can(sd_policies.BASE_POLICY_NAME, Jul 16 21:10:17.001669 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/context.py", line 240, in can Jul 16 21:10:17.001669 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return policy.authorize(self, action, target) Jul 16 21:10:17.001669 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 199, in authorize Jul 16 21:10:17.001669 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi LOG.debug('Policy check for %(action)s failed with credentials ' Jul 16 21:10:17.001669 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 16 21:10:17.001669 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi self.force_reraise() Jul 16 21:10:17.001669 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 16 21:10:17.001669 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise self.value Jul 16 21:10:17.001669 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 186, in authorize Jul 16 21:10:17.002795 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi result = _ENFORCER.authorize(action, target, context, Jul 16 21:10:17.002795 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1184, in authorize Jul 16 21:10:17.002795 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return self.enforce( Jul 16 21:10:17.002795 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1088, in enforce Jul 16 21:10:17.002795 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc(*args, **kwargs) Jul 16 21:10:17.002795 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.PolicyNotAuthorized: Policy doesn't allow os_compute_api:os-server-diagnostics to be performed. Jul 16 21:10:17.002795 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:10:17.002795 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:10:17.002795 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:10:17.002795 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:10:17.002795 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:10:17.002795 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:10:17.002795 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 354, in __exit__ Jul 16 21:10:17.002795 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(webob.exc.HTTPForbidden( Jul 16 21:10:17.002795 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Policy doesn't allow os_compute_api:os-server-diagnostics to be performed. Jul 16 21:10:17.002795 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:10:17.002795 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-617f8f83-d977-4c23-88fe-b7656b6af465 tempest-ServerDiagnosticsNegativeTest-1844467799 tempest-ServerDiagnosticsNegativeTest-1844467799-project] Returning 403 to user: Policy doesn't allow os_compute_api:os-server-diagnostics to be performed. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:10:17.005238 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-617f8f83-d977-4c23-88fe-b7656b6af465 tempest-ServerDiagnosticsNegativeTest-1844467799 tempest-ServerDiagnosticsNegativeTest-1844467799-project] 10.222.0.16 "GET /compute/v2.1/servers/1e4229cb-4ed9-48c3-b723-28619ae7045c/diagnostics" status: 403 len: 117 microversion: 2.1 time: 0.101288 Jul 16 21:10:17.013496 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 315/639] 10.222.0.16 () {58 vars in 1271 bytes} [Sat Jul 16 21:10:16 2022] GET /compute/v2.1/servers/1e4229cb-4ed9-48c3-b723-28619ae7045c/diagnostics => generated 117 bytes in 113 msecs (HTTP/1.1 403) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:10:17.020959 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-397e658e-fc70-45cb-98ab-16f1830874f3 tempest-ServerDiagnosticsNegativeTest-1844467799 tempest-ServerDiagnosticsNegativeTest-1844467799-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:17.021551 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-397e658e-fc70-45cb-98ab-16f1830874f3 tempest-ServerDiagnosticsNegativeTest-1844467799 tempest-ServerDiagnosticsNegativeTest-1844467799-project] [instance: 1e4229cb-4ed9-48c3-b723-28619ae7045c] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:17.028831 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-397e658e-fc70-45cb-98ab-16f1830874f3 tempest-ServerDiagnosticsNegativeTest-1844467799 tempest-ServerDiagnosticsNegativeTest-1844467799-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:17.029380 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-397e658e-fc70-45cb-98ab-16f1830874f3 tempest-ServerDiagnosticsNegativeTest-1844467799 tempest-ServerDiagnosticsNegativeTest-1844467799-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:17.033472 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-397e658e-fc70-45cb-98ab-16f1830874f3 tempest-ServerDiagnosticsNegativeTest-1844467799 tempest-ServerDiagnosticsNegativeTest-1844467799-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:17.033472 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-397e658e-fc70-45cb-98ab-16f1830874f3 tempest-ServerDiagnosticsNegativeTest-1844467799 tempest-ServerDiagnosticsNegativeTest-1844467799-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:17.073201 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-397e658e-fc70-45cb-98ab-16f1830874f3 tempest-ServerDiagnosticsNegativeTest-1844467799 tempest-ServerDiagnosticsNegativeTest-1844467799-project] [instance: 1e4229cb-4ed9-48c3-b723-28619ae7045c] Going to try to terminate instance {{(pid=138681) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 16 21:10:17.212495 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-2df1772f-fe29-4d5c-a08e-76d985b5efa8 tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] Action: 'action', calling method: >, body: {"rebuild": {"imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9"}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:10:17.235870 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-2df1772f-fe29-4d5c-a08e-76d985b5efa8 tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] [instance: cf53f7cd-deae-4b34-bf8e-3afef98b2ec6] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:17.247418 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2df1772f-fe29-4d5c-a08e-76d985b5efa8 tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:17.247883 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2df1772f-fe29-4d5c-a08e-76d985b5efa8 tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:17.248675 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2df1772f-fe29-4d5c-a08e-76d985b5efa8 tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:17.250411 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2df1772f-fe29-4d5c-a08e-76d985b5efa8 tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:17.413428 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-2df1772f-fe29-4d5c-a08e-76d985b5efa8 tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] Flavor limits 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 16 21:10:17.419037 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-2df1772f-fe29-4d5c-a08e-76d985b5efa8 tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] Image limits 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 16 21:10:17.421580 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-2df1772f-fe29-4d5c-a08e-76d985b5efa8 tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] Flavor pref 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 16 21:10:17.422632 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-2df1772f-fe29-4d5c-a08e-76d985b5efa8 tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] Image pref 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 16 21:10:17.424447 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-2df1772f-fe29-4d5c-a08e-76d985b5efa8 tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 16 21:10:17.513175 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-397e658e-fc70-45cb-98ab-16f1830874f3 tempest-ServerDiagnosticsNegativeTest-1844467799 tempest-ServerDiagnosticsNegativeTest-1844467799-project] 10.222.0.16 "DELETE /compute/v2.1/servers/1e4229cb-4ed9-48c3-b723-28619ae7045c" status: 204 len: 0 microversion: 2.1 time: 0.497672 Jul 16 21:10:17.514952 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 316/640] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:10:17 2022] DELETE /compute/v2.1/servers/1e4229cb-4ed9-48c3-b723-28619ae7045c => generated 0 bytes in 500 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 16 21:10:17.526892 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-6dee5af9-6704-4f4c-9025-e2ac426e2c7c tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:17.529458 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-6dee5af9-6704-4f4c-9025-e2ac426e2c7c tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] [instance: 8872e8cf-593e-4a7a-940d-e0c8159d4ad7] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:17.547610 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6dee5af9-6704-4f4c-9025-e2ac426e2c7c tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:17.550650 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6dee5af9-6704-4f4c-9025-e2ac426e2c7c tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:17.554176 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6dee5af9-6704-4f4c-9025-e2ac426e2c7c tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:17.557107 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6dee5af9-6704-4f4c-9025-e2ac426e2c7c tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:17.676532 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-2df1772f-fe29-4d5c-a08e-76d985b5efa8 tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] [instance: cf53f7cd-deae-4b34-bf8e-3afef98b2ec6] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:17.694117 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2df1772f-fe29-4d5c-a08e-76d985b5efa8 tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:17.694645 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2df1772f-fe29-4d5c-a08e-76d985b5efa8 tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:17.698601 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2df1772f-fe29-4d5c-a08e-76d985b5efa8 tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:17.699460 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2df1772f-fe29-4d5c-a08e-76d985b5efa8 tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:17.826933 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-6dee5af9-6704-4f4c-9025-e2ac426e2c7c tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ef9cbce2b2e64665b7f50767342dcb46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0ee02e4c8b448f2b77798ae504129b9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:17.836888 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-6dee5af9-6704-4f4c-9025-e2ac426e2c7c tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] 10.222.0.16 "GET /compute/v2.1/servers/8872e8cf-593e-4a7a-940d-e0c8159d4ad7" status: 200 len: 1389 microversion: 2.1 time: 0.319340 Jul 16 21:10:17.861857 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 317/641] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:10:17 2022] GET /compute/v2.1/servers/8872e8cf-593e-4a7a-940d-e0c8159d4ad7 => generated 1389 bytes in 336 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:10:17.866262 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-800938fb-c84e-4a35-bb16-eca5b1321408 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:17.866885 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-800938fb-c84e-4a35-bb16-eca5b1321408 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] [instance: 1631f0cc-de67-4c0d-abd0-3e0e11dfb4ed] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:17.875122 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-800938fb-c84e-4a35-bb16-eca5b1321408 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:17.875914 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-800938fb-c84e-4a35-bb16-eca5b1321408 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:17.878109 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-800938fb-c84e-4a35-bb16-eca5b1321408 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:17.878109 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-800938fb-c84e-4a35-bb16-eca5b1321408 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:17.914214 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.objects.instance [None req-2df1772f-fe29-4d5c-a08e-76d985b5efa8 tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] Lazy-loading 'services' on Instance uuid cf53f7cd-deae-4b34-bf8e-3afef98b2ec6 {{(pid=138680) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:10:18.016244 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-2df1772f-fe29-4d5c-a08e-76d985b5efa8 tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] 10.222.0.16 "POST /compute/v2.1/servers/cf53f7cd-deae-4b34-bf8e-3afef98b2ec6/action" status: 202 len: 1942 microversion: 2.75 time: 1.144198 Jul 16 21:10:18.018931 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 325/642] 10.222.0.16 () {62 vars in 1318 bytes} [Sat Jul 16 21:10:16 2022] POST /compute/v2.1/servers/cf53f7cd-deae-4b34-bf8e-3afef98b2ec6/action => generated 1942 bytes in 1146 msecs (HTTP/1.1 202) 10 headers in 454 bytes (2 switches on core 0) Jul 16 21:10:18.041688 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-4155def1-c068-43e0-a22b-0bfd3df0dc05 req-7b1784ae-67cc-4ef3-8d0b-b0b995c670bb service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "7d624313-0c12-4da2-a95c-3903efae7608", "name": "network-vif-plugged", "status": "completed", "tag": "186c161e-a1dc-4086-9efc-737d7b982c1b"}, {"server_uuid": "8872e8cf-593e-4a7a-940d-e0c8159d4ad7", "name": "network-vif-plugged", "status": "completed", "tag": "a07c49a0-9773-4e57-963c-ebcc388b002a"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:10:18.041688 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-4155def1-c068-43e0-a22b-0bfd3df0dc05 req-7b1784ae-67cc-4ef3-8d0b-b0b995c670bb service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:18.041688 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-4155def1-c068-43e0-a22b-0bfd3df0dc05 req-7b1784ae-67cc-4ef3-8d0b-b0b995c670bb service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:18.048181 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-4155def1-c068-43e0-a22b-0bfd3df0dc05 req-7b1784ae-67cc-4ef3-8d0b-b0b995c670bb service nova] Creating event network-vif-plugged:186c161e-a1dc-4086-9efc-737d7b982c1b for instance 7d624313-0c12-4da2-a95c-3903efae7608 on nch1-master-1 Jul 16 21:10:18.048806 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-4155def1-c068-43e0-a22b-0bfd3df0dc05 req-7b1784ae-67cc-4ef3-8d0b-b0b995c670bb service nova] Creating event network-vif-plugged:a07c49a0-9773-4e57-963c-ebcc388b002a for instance 8872e8cf-593e-4a7a-940d-e0c8159d4ad7 on nch2-master-1 Jul 16 21:10:18.061262 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-4155def1-c068-43e0-a22b-0bfd3df0dc05 req-7b1784ae-67cc-4ef3-8d0b-b0b995c670bb service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 354 microversion: 2.1 time: 0.041936 Jul 16 21:10:18.065078 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 326/643] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:10:18 2022] POST /compute/v2.1/os-server-external-events => generated 354 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:10:18.076377 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-b916017e-1867-46db-aa0b-87ec4b45cd7d tempest-ServerDiagnosticsNegativeTest-1844467799 tempest-ServerDiagnosticsNegativeTest-1844467799-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:18.077055 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-b916017e-1867-46db-aa0b-87ec4b45cd7d tempest-ServerDiagnosticsNegativeTest-1844467799 tempest-ServerDiagnosticsNegativeTest-1844467799-project] [instance: 1e4229cb-4ed9-48c3-b723-28619ae7045c] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:18.085724 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b916017e-1867-46db-aa0b-87ec4b45cd7d tempest-ServerDiagnosticsNegativeTest-1844467799 tempest-ServerDiagnosticsNegativeTest-1844467799-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:18.086199 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b916017e-1867-46db-aa0b-87ec4b45cd7d tempest-ServerDiagnosticsNegativeTest-1844467799 tempest-ServerDiagnosticsNegativeTest-1844467799-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:18.087451 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b916017e-1867-46db-aa0b-87ec4b45cd7d tempest-ServerDiagnosticsNegativeTest-1844467799 tempest-ServerDiagnosticsNegativeTest-1844467799-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:18.088056 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b916017e-1867-46db-aa0b-87ec4b45cd7d tempest-ServerDiagnosticsNegativeTest-1844467799 tempest-ServerDiagnosticsNegativeTest-1844467799-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:18.139564 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-800938fb-c84e-4a35-bb16-eca5b1321408 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0d72626f465b4f37ad3b18bb945d4000', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fc1c3463f9444119af9d18f85bdd703d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:18.149736 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-800938fb-c84e-4a35-bb16-eca5b1321408 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] 10.222.0.16 "GET /compute/v2.1/servers/1631f0cc-de67-4c0d-abd0-3e0e11dfb4ed" status: 200 len: 1474 microversion: 2.1 time: 0.295554 Jul 16 21:10:18.160076 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 318/644] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:10:17 2022] GET /compute/v2.1/servers/1631f0cc-de67-4c0d-abd0-3e0e11dfb4ed => generated 1474 bytes in 297 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:10:18.164688 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-8227289a-47bb-4b7f-93ae-0b519f9fc73e tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:18.165353 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-8227289a-47bb-4b7f-93ae-0b519f9fc73e tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: f98a9762-a67c-4e95-85b8-727c81bdb03a] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:18.173620 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8227289a-47bb-4b7f-93ae-0b519f9fc73e tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:18.174081 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8227289a-47bb-4b7f-93ae-0b519f9fc73e tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:18.174884 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8227289a-47bb-4b7f-93ae-0b519f9fc73e tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:18.181730 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8227289a-47bb-4b7f-93ae-0b519f9fc73e tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:18.369552 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-b916017e-1867-46db-aa0b-87ec4b45cd7d tempest-ServerDiagnosticsNegativeTest-1844467799 tempest-ServerDiagnosticsNegativeTest-1844467799-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9c428093bcf445a6aea6ddb209e6e5cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5f9cdca4179c4875a4be57cf8a9f46a8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:18.387812 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-b916017e-1867-46db-aa0b-87ec4b45cd7d tempest-ServerDiagnosticsNegativeTest-1844467799 tempest-ServerDiagnosticsNegativeTest-1844467799-project] 10.222.0.16 "GET /compute/v2.1/servers/1e4229cb-4ed9-48c3-b723-28619ae7045c" status: 200 len: 1478 microversion: 2.1 time: 0.321202 Jul 16 21:10:18.398934 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 327/645] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:10:18 2022] GET /compute/v2.1/servers/1e4229cb-4ed9-48c3-b723-28619ae7045c => generated 1478 bytes in 325 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:10:18.398934 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-deaaac53-6831-4e70-95f0-afaa96671f83 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:18.398934 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-deaaac53-6831-4e70-95f0-afaa96671f83 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] [instance: 7d624313-0c12-4da2-a95c-3903efae7608] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:18.412303 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-deaaac53-6831-4e70-95f0-afaa96671f83 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:18.413303 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-deaaac53-6831-4e70-95f0-afaa96671f83 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:18.415184 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-deaaac53-6831-4e70-95f0-afaa96671f83 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:18.415810 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-deaaac53-6831-4e70-95f0-afaa96671f83 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:18.442540 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-8227289a-47bb-4b7f-93ae-0b519f9fc73e tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b68dacc4b8463ba07fae6372131117', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55a19b7a4682457a9132f5fc80110712', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:18.460747 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-8227289a-47bb-4b7f-93ae-0b519f9fc73e tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "GET /compute/v2.1/servers/f98a9762-a67c-4e95-85b8-727c81bdb03a" status: 200 len: 1471 microversion: 2.1 time: 0.308740 Jul 16 21:10:18.462413 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 319/646] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:10:18 2022] GET /compute/v2.1/servers/f98a9762-a67c-4e95-85b8-727c81bdb03a => generated 1471 bytes in 311 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:10:18.468737 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-684a784f-b54a-48fe-a9b0-21b814ad4197 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:18.469612 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-684a784f-b54a-48fe-a9b0-21b814ad4197 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] [instance: 4ee3ffbe-c16a-4847-a343-469c645e4b25] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:18.478526 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-684a784f-b54a-48fe-a9b0-21b814ad4197 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:18.479360 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-684a784f-b54a-48fe-a9b0-21b814ad4197 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:18.481391 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-684a784f-b54a-48fe-a9b0-21b814ad4197 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:18.483073 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-684a784f-b54a-48fe-a9b0-21b814ad4197 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:18.677102 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-deaaac53-6831-4e70-95f0-afaa96671f83 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'abeb7020019d41a18de42956d706a011', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '649babe54d104436af461f1ee4767001', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:18.720150 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-deaaac53-6831-4e70-95f0-afaa96671f83 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] 10.222.0.16 "GET /compute/v2.1/servers/7d624313-0c12-4da2-a95c-3903efae7608" status: 200 len: 1318 microversion: 2.1 time: 0.328816 Jul 16 21:10:18.730792 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-684a784f-b54a-48fe-a9b0-21b814ad4197 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a69f9dee03d43beaafa966216b9e289', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9b769bf8f3ea4e368bf68de1df6538f9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:18.735142 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 328/647] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:10:18 2022] GET /compute/v2.1/servers/7d624313-0c12-4da2-a95c-3903efae7608 => generated 1318 bytes in 344 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:10:18.740042 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-684a784f-b54a-48fe-a9b0-21b814ad4197 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] 10.222.0.16 "GET /compute/v2.1/servers/4ee3ffbe-c16a-4847-a343-469c645e4b25" status: 200 len: 1311 microversion: 2.1 time: 0.276246 Jul 16 21:10:18.741477 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-6cc16e3f-fcaf-4359-a075-3571337c639a tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:18.742328 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-6cc16e3f-fcaf-4359-a075-3571337c639a tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] [instance: cf53f7cd-deae-4b34-bf8e-3afef98b2ec6] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:18.744227 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 320/648] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:10:18 2022] GET /compute/v2.1/servers/4ee3ffbe-c16a-4847-a343-469c645e4b25 => generated 1311 bytes in 281 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:10:18.755712 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6cc16e3f-fcaf-4359-a075-3571337c639a tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:18.756764 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6cc16e3f-fcaf-4359-a075-3571337c639a tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:18.757838 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6cc16e3f-fcaf-4359-a075-3571337c639a tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:18.758512 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6cc16e3f-fcaf-4359-a075-3571337c639a tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:18.765889 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:10:18.865061 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.objects.instance [None req-6cc16e3f-fcaf-4359-a075-3571337c639a tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] Lazy-loading 'services' on Instance uuid cf53f7cd-deae-4b34-bf8e-3afef98b2ec6 {{(pid=138680) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:10:18.870210 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-a0e22b29-4004-4bf3-9424-c2c23862295d tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:18.870928 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-a0e22b29-4004-4bf3-9424-c2c23862295d tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] [instance: 8872e8cf-593e-4a7a-940d-e0c8159d4ad7] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:18.901150 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a0e22b29-4004-4bf3-9424-c2c23862295d tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:18.901633 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a0e22b29-4004-4bf3-9424-c2c23862295d tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:18.902511 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a0e22b29-4004-4bf3-9424-c2c23862295d tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:18.903046 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a0e22b29-4004-4bf3-9424-c2c23862295d tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:18.910024 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-6cc16e3f-fcaf-4359-a075-3571337c639a tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/cf53f7cd-deae-4b34-bf8e-3afef98b2ec6" status: 200 len: 1929 microversion: 2.75 time: 0.173259 Jul 16 21:10:18.927784 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 329/649] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:10:18 2022] GET /compute/v2.1/servers/cf53f7cd-deae-4b34-bf8e-3afef98b2ec6 => generated 1929 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:10:19.168152 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-5ff0598d-085b-482f-b683-52f9ca4e5479 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:19.169271 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-5ff0598d-085b-482f-b683-52f9ca4e5479 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] [instance: 1631f0cc-de67-4c0d-abd0-3e0e11dfb4ed] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:19.195583 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5ff0598d-085b-482f-b683-52f9ca4e5479 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:19.197555 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5ff0598d-085b-482f-b683-52f9ca4e5479 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:19.202812 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5ff0598d-085b-482f-b683-52f9ca4e5479 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:19.205386 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5ff0598d-085b-482f-b683-52f9ca4e5479 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:19.374522 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-a0e22b29-4004-4bf3-9424-c2c23862295d tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ef9cbce2b2e64665b7f50767342dcb46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0ee02e4c8b448f2b77798ae504129b9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:19.388502 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-a0e22b29-4004-4bf3-9424-c2c23862295d tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] 10.222.0.16 "GET /compute/v2.1/servers/8872e8cf-593e-4a7a-940d-e0c8159d4ad7" status: 200 len: 1389 microversion: 2.1 time: 0.522339 Jul 16 21:10:19.391727 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 321/650] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:10:18 2022] GET /compute/v2.1/servers/8872e8cf-593e-4a7a-940d-e0c8159d4ad7 => generated 1389 bytes in 526 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:10:19.406466 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-78e67cc4-2940-434b-9613-abcf946ca3ce tempest-ServerDiagnosticsNegativeTest-1844467799 tempest-ServerDiagnosticsNegativeTest-1844467799-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:19.407462 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-78e67cc4-2940-434b-9613-abcf946ca3ce tempest-ServerDiagnosticsNegativeTest-1844467799 tempest-ServerDiagnosticsNegativeTest-1844467799-project] [instance: 1e4229cb-4ed9-48c3-b723-28619ae7045c] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:19.414632 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-78e67cc4-2940-434b-9613-abcf946ca3ce tempest-ServerDiagnosticsNegativeTest-1844467799 tempest-ServerDiagnosticsNegativeTest-1844467799-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:19.415465 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-78e67cc4-2940-434b-9613-abcf946ca3ce tempest-ServerDiagnosticsNegativeTest-1844467799 tempest-ServerDiagnosticsNegativeTest-1844467799-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:19.416480 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-78e67cc4-2940-434b-9613-abcf946ca3ce tempest-ServerDiagnosticsNegativeTest-1844467799 tempest-ServerDiagnosticsNegativeTest-1844467799-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:19.416991 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-78e67cc4-2940-434b-9613-abcf946ca3ce tempest-ServerDiagnosticsNegativeTest-1844467799 tempest-ServerDiagnosticsNegativeTest-1844467799-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:19.590434 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-5ff0598d-085b-482f-b683-52f9ca4e5479 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0d72626f465b4f37ad3b18bb945d4000', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fc1c3463f9444119af9d18f85bdd703d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:19.612030 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-5ff0598d-085b-482f-b683-52f9ca4e5479 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] 10.222.0.16 "GET /compute/v2.1/servers/1631f0cc-de67-4c0d-abd0-3e0e11dfb4ed" status: 200 len: 1474 microversion: 2.1 time: 0.441127 Jul 16 21:10:19.612030 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 330/651] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:10:19 2022] GET /compute/v2.1/servers/1631f0cc-de67-4c0d-abd0-3e0e11dfb4ed => generated 1474 bytes in 443 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:10:19.612841 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-c1f4ed1c-1ccd-4a00-bd7d-61d0b77ba2ca tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:19.613767 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-c1f4ed1c-1ccd-4a00-bd7d-61d0b77ba2ca tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: f98a9762-a67c-4e95-85b8-727c81bdb03a] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:19.628737 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c1f4ed1c-1ccd-4a00-bd7d-61d0b77ba2ca tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:19.629776 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c1f4ed1c-1ccd-4a00-bd7d-61d0b77ba2ca tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:19.630840 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c1f4ed1c-1ccd-4a00-bd7d-61d0b77ba2ca tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:19.631586 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c1f4ed1c-1ccd-4a00-bd7d-61d0b77ba2ca tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:19.823282 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-78e67cc4-2940-434b-9613-abcf946ca3ce tempest-ServerDiagnosticsNegativeTest-1844467799 tempest-ServerDiagnosticsNegativeTest-1844467799-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9c428093bcf445a6aea6ddb209e6e5cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5f9cdca4179c4875a4be57cf8a9f46a8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:19.837737 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-78e67cc4-2940-434b-9613-abcf946ca3ce tempest-ServerDiagnosticsNegativeTest-1844467799 tempest-ServerDiagnosticsNegativeTest-1844467799-project] 10.222.0.16 "GET /compute/v2.1/servers/1e4229cb-4ed9-48c3-b723-28619ae7045c" status: 200 len: 1478 microversion: 2.1 time: 0.435729 Jul 16 21:10:19.839539 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 322/652] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:10:19 2022] GET /compute/v2.1/servers/1e4229cb-4ed9-48c3-b723-28619ae7045c => generated 1478 bytes in 438 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:10:19.853355 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-cadbb70f-f02f-4376-bc3a-8ad42b1dee27 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:19.854692 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-cadbb70f-f02f-4376-bc3a-8ad42b1dee27 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] [instance: 7d624313-0c12-4da2-a95c-3903efae7608] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:19.871492 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cadbb70f-f02f-4376-bc3a-8ad42b1dee27 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:19.872165 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cadbb70f-f02f-4376-bc3a-8ad42b1dee27 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:19.872992 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cadbb70f-f02f-4376-bc3a-8ad42b1dee27 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:19.873448 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cadbb70f-f02f-4376-bc3a-8ad42b1dee27 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:20.036845 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-c1f4ed1c-1ccd-4a00-bd7d-61d0b77ba2ca tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b68dacc4b8463ba07fae6372131117', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55a19b7a4682457a9132f5fc80110712', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:20.044649 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-c1f4ed1c-1ccd-4a00-bd7d-61d0b77ba2ca tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "GET /compute/v2.1/servers/f98a9762-a67c-4e95-85b8-727c81bdb03a" status: 200 len: 1471 microversion: 2.1 time: 0.437675 Jul 16 21:10:20.045616 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 331/653] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:10:19 2022] GET /compute/v2.1/servers/f98a9762-a67c-4e95-85b8-727c81bdb03a => generated 1471 bytes in 439 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:10:20.050696 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-7311292b-95b5-48ff-8487-3d581f094610 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:20.051338 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-7311292b-95b5-48ff-8487-3d581f094610 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] [instance: 4ee3ffbe-c16a-4847-a343-469c645e4b25] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:20.066054 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7311292b-95b5-48ff-8487-3d581f094610 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:20.066619 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7311292b-95b5-48ff-8487-3d581f094610 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:20.067680 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7311292b-95b5-48ff-8487-3d581f094610 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:20.068119 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7311292b-95b5-48ff-8487-3d581f094610 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:20.176331 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-cadbb70f-f02f-4376-bc3a-8ad42b1dee27 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'abeb7020019d41a18de42956d706a011', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '649babe54d104436af461f1ee4767001', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:20.189458 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-cadbb70f-f02f-4376-bc3a-8ad42b1dee27 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] 10.222.0.16 "GET /compute/v2.1/servers/7d624313-0c12-4da2-a95c-3903efae7608" status: 200 len: 1318 microversion: 2.1 time: 0.348235 Jul 16 21:10:20.206864 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 323/654] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:10:19 2022] GET /compute/v2.1/servers/7d624313-0c12-4da2-a95c-3903efae7608 => generated 1318 bytes in 366 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:10:20.225452 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-f7f73bc6-e453-496b-8bc0-57bad6f4539f tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:20.227125 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-f7f73bc6-e453-496b-8bc0-57bad6f4539f tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] [instance: cf53f7cd-deae-4b34-bf8e-3afef98b2ec6] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:20.236805 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_api:SELECT=94,nova_api:DELETE=1,nova_api:INSERT=3,nova_api:UPDATE=2 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:10:20.259517 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f7f73bc6-e453-496b-8bc0-57bad6f4539f tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:20.261230 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f7f73bc6-e453-496b-8bc0-57bad6f4539f tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:20.262182 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f7f73bc6-e453-496b-8bc0-57bad6f4539f tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:20.263822 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f7f73bc6-e453-496b-8bc0-57bad6f4539f tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:20.313145 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-7311292b-95b5-48ff-8487-3d581f094610 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a69f9dee03d43beaafa966216b9e289', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9b769bf8f3ea4e368bf68de1df6538f9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:20.337915 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-7311292b-95b5-48ff-8487-3d581f094610 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] 10.222.0.16 "GET /compute/v2.1/servers/4ee3ffbe-c16a-4847-a343-469c645e4b25" status: 200 len: 1311 microversion: 2.1 time: 0.286502 Jul 16 21:10:20.339194 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 332/655] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:10:20 2022] GET /compute/v2.1/servers/4ee3ffbe-c16a-4847-a343-469c645e4b25 => generated 1311 bytes in 293 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:10:20.430223 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-65fc83a8-1950-4d81-a9f4-1a81b38da22d tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:20.432775 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-65fc83a8-1950-4d81-a9f4-1a81b38da22d tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] [instance: 8872e8cf-593e-4a7a-940d-e0c8159d4ad7] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:20.457500 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-65fc83a8-1950-4d81-a9f4-1a81b38da22d tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:20.458361 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-65fc83a8-1950-4d81-a9f4-1a81b38da22d tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:20.470131 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-65fc83a8-1950-4d81-a9f4-1a81b38da22d tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:20.478621 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-65fc83a8-1950-4d81-a9f4-1a81b38da22d tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:20.493165 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.objects.instance [None req-f7f73bc6-e453-496b-8bc0-57bad6f4539f tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] Lazy-loading 'services' on Instance uuid cf53f7cd-deae-4b34-bf8e-3afef98b2ec6 {{(pid=138681) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:10:20.602047 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-f7f73bc6-e453-496b-8bc0-57bad6f4539f tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/cf53f7cd-deae-4b34-bf8e-3afef98b2ec6" status: 200 len: 1929 microversion: 2.75 time: 0.392815 Jul 16 21:10:20.602660 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 324/656] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:10:20 2022] GET /compute/v2.1/servers/cf53f7cd-deae-4b34-bf8e-3afef98b2ec6 => generated 1929 bytes in 394 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:10:20.612892 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-785534a2-6f95-4bf2-a67b-fe51367257ea req-76d63bd3-0dc0-4e42-8791-0ec1f937bec1 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "1631f0cc-de67-4c0d-abd0-3e0e11dfb4ed", "name": "network-vif-deleted", "tag": "e43e8868-6c5f-4776-ab31-fb29f6dcf4bf"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:10:20.638986 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-785534a2-6f95-4bf2-a67b-fe51367257ea req-76d63bd3-0dc0-4e42-8791-0ec1f937bec1 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:20.639458 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-785534a2-6f95-4bf2-a67b-fe51367257ea req-76d63bd3-0dc0-4e42-8791-0ec1f937bec1 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:20.652957 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-785534a2-6f95-4bf2-a67b-fe51367257ea req-76d63bd3-0dc0-4e42-8791-0ec1f937bec1 service nova] Creating event network-vif-deleted:e43e8868-6c5f-4776-ab31-fb29f6dcf4bf for instance 1631f0cc-de67-4c0d-abd0-3e0e11dfb4ed on nch2-master-1 Jul 16 21:10:20.659514 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-785534a2-6f95-4bf2-a67b-fe51367257ea req-76d63bd3-0dc0-4e42-8791-0ec1f937bec1 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.055515 Jul 16 21:10:20.660452 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 325/657] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:10:20 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:10:20.670003 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-dbfb34fc-e5f5-464e-82ec-42dd0cde51f1 req-7e4d320d-c6c9-4d7f-b49e-c679c781a95e service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "1e4229cb-4ed9-48c3-b723-28619ae7045c", "name": "network-vif-deleted", "tag": "d94b5111-1b85-496e-b76b-7338bb0d2319"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:10:20.687579 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-dbfb34fc-e5f5-464e-82ec-42dd0cde51f1 req-7e4d320d-c6c9-4d7f-b49e-c679c781a95e service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:20.688415 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-dbfb34fc-e5f5-464e-82ec-42dd0cde51f1 req-7e4d320d-c6c9-4d7f-b49e-c679c781a95e service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:20.711174 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=96,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:10:20.715275 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-dbfb34fc-e5f5-464e-82ec-42dd0cde51f1 req-7e4d320d-c6c9-4d7f-b49e-c679c781a95e service nova] Creating event network-vif-deleted:d94b5111-1b85-496e-b76b-7338bb0d2319 for instance 1e4229cb-4ed9-48c3-b723-28619ae7045c on nch1-master-1 Jul 16 21:10:20.725177 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-dbfb34fc-e5f5-464e-82ec-42dd0cde51f1 req-7e4d320d-c6c9-4d7f-b49e-c679c781a95e service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.063128 Jul 16 21:10:20.726061 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 326/658] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:10:20 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 65 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:10:20.738549 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-a8304a12-319e-4820-89fc-c30ff9a5c8a6 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:20.739893 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-a8304a12-319e-4820-89fc-c30ff9a5c8a6 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] [instance: 1631f0cc-de67-4c0d-abd0-3e0e11dfb4ed] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:20.752947 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a8304a12-319e-4820-89fc-c30ff9a5c8a6 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:20.754277 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a8304a12-319e-4820-89fc-c30ff9a5c8a6 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:20.755268 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a8304a12-319e-4820-89fc-c30ff9a5c8a6 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:20.756049 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a8304a12-319e-4820-89fc-c30ff9a5c8a6 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:20.908501 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell0:SELECT=2 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:10:20.937694 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell0:SELECT=62 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:10:20.964236 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-a8304a12-319e-4820-89fc-c30ff9a5c8a6 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0d72626f465b4f37ad3b18bb945d4000', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fc1c3463f9444119af9d18f85bdd703d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:20.968287 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-65fc83a8-1950-4d81-a9f4-1a81b38da22d tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ef9cbce2b2e64665b7f50767342dcb46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0ee02e4c8b448f2b77798ae504129b9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:20.976431 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-65fc83a8-1950-4d81-a9f4-1a81b38da22d tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] 10.222.0.16 "GET /compute/v2.1/servers/8872e8cf-593e-4a7a-940d-e0c8159d4ad7" status: 200 len: 1389 microversion: 2.1 time: 0.570548 Jul 16 21:10:20.983299 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-a8304a12-319e-4820-89fc-c30ff9a5c8a6 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] 10.222.0.16 "GET /compute/v2.1/servers/1631f0cc-de67-4c0d-abd0-3e0e11dfb4ed" status: 200 len: 1432 microversion: 2.1 time: 0.248996 Jul 16 21:10:20.986823 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 333/659] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:10:20 2022] GET /compute/v2.1/servers/8872e8cf-593e-4a7a-940d-e0c8159d4ad7 => generated 1389 bytes in 582 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:10:20.988915 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 327/660] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:10:20 2022] GET /compute/v2.1/servers/1631f0cc-de67-4c0d-abd0-3e0e11dfb4ed => generated 1432 bytes in 262 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:10:20.998738 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-ccc1c527-1bfe-4124-b600-653234cde8aa tempest-ServerDiagnosticsNegativeTest-1844467799 tempest-ServerDiagnosticsNegativeTest-1844467799-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:21.001652 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-ccc1c527-1bfe-4124-b600-653234cde8aa tempest-ServerDiagnosticsNegativeTest-1844467799 tempest-ServerDiagnosticsNegativeTest-1844467799-project] [instance: 1e4229cb-4ed9-48c3-b723-28619ae7045c] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:21.039444 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ccc1c527-1bfe-4124-b600-653234cde8aa tempest-ServerDiagnosticsNegativeTest-1844467799 tempest-ServerDiagnosticsNegativeTest-1844467799-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:21.039903 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ccc1c527-1bfe-4124-b600-653234cde8aa tempest-ServerDiagnosticsNegativeTest-1844467799 tempest-ServerDiagnosticsNegativeTest-1844467799-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:21.040700 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ccc1c527-1bfe-4124-b600-653234cde8aa tempest-ServerDiagnosticsNegativeTest-1844467799 tempest-ServerDiagnosticsNegativeTest-1844467799-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:21.041211 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ccc1c527-1bfe-4124-b600-653234cde8aa tempest-ServerDiagnosticsNegativeTest-1844467799 tempest-ServerDiagnosticsNegativeTest-1844467799-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:21.063734 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-ba955679-9797-40a8-8fe3-2252b05108a5 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:21.065697 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-ba955679-9797-40a8-8fe3-2252b05108a5 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: f98a9762-a67c-4e95-85b8-727c81bdb03a] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:21.081955 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ba955679-9797-40a8-8fe3-2252b05108a5 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:21.087335 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ba955679-9797-40a8-8fe3-2252b05108a5 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:21.094540 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ba955679-9797-40a8-8fe3-2252b05108a5 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:21.094540 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ba955679-9797-40a8-8fe3-2252b05108a5 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:21.222093 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-ccc1c527-1bfe-4124-b600-653234cde8aa tempest-ServerDiagnosticsNegativeTest-1844467799 tempest-ServerDiagnosticsNegativeTest-1844467799-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9c428093bcf445a6aea6ddb209e6e5cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5f9cdca4179c4875a4be57cf8a9f46a8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:21.239894 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-ccc1c527-1bfe-4124-b600-653234cde8aa tempest-ServerDiagnosticsNegativeTest-1844467799 tempest-ServerDiagnosticsNegativeTest-1844467799-project] 10.222.0.16 "GET /compute/v2.1/servers/1e4229cb-4ed9-48c3-b723-28619ae7045c" status: 200 len: 1308 microversion: 2.1 time: 0.240092 Jul 16 21:10:21.244082 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 334/661] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:10:20 2022] GET /compute/v2.1/servers/1e4229cb-4ed9-48c3-b723-28619ae7045c => generated 1308 bytes in 257 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:10:21.271946 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-4e5faf23-359a-42a3-b215-05ba661a47e1 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:21.274428 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-4e5faf23-359a-42a3-b215-05ba661a47e1 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] [instance: 7d624313-0c12-4da2-a95c-3903efae7608] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:21.292065 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4e5faf23-359a-42a3-b215-05ba661a47e1 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:21.294642 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4e5faf23-359a-42a3-b215-05ba661a47e1 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:21.296046 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4e5faf23-359a-42a3-b215-05ba661a47e1 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:21.297510 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4e5faf23-359a-42a3-b215-05ba661a47e1 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:21.297898 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell0:SELECT=2 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:10:21.561462 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-ba955679-9797-40a8-8fe3-2252b05108a5 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b68dacc4b8463ba07fae6372131117', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55a19b7a4682457a9132f5fc80110712', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:21.569515 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-ba955679-9797-40a8-8fe3-2252b05108a5 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "GET /compute/v2.1/servers/f98a9762-a67c-4e95-85b8-727c81bdb03a" status: 200 len: 1476 microversion: 2.1 time: 0.511474 Jul 16 21:10:21.570086 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 328/662] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:10:21 2022] GET /compute/v2.1/servers/f98a9762-a67c-4e95-85b8-727c81bdb03a => generated 1476 bytes in 513 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:10:21.574745 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-c4d214d2-abf0-4458-ac8a-69dd98c692cf tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:21.575379 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-c4d214d2-abf0-4458-ac8a-69dd98c692cf tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] [instance: 4ee3ffbe-c16a-4847-a343-469c645e4b25] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:21.585449 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c4d214d2-abf0-4458-ac8a-69dd98c692cf tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:21.585701 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c4d214d2-abf0-4458-ac8a-69dd98c692cf tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:21.586528 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c4d214d2-abf0-4458-ac8a-69dd98c692cf tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:21.586961 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c4d214d2-abf0-4458-ac8a-69dd98c692cf tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:21.641346 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-4e5faf23-359a-42a3-b215-05ba661a47e1 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'abeb7020019d41a18de42956d706a011', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '649babe54d104436af461f1ee4767001', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:21.678500 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-4e5faf23-359a-42a3-b215-05ba661a47e1 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] 10.222.0.16 "GET /compute/v2.1/servers/7d624313-0c12-4da2-a95c-3903efae7608" status: 200 len: 1468 microversion: 2.1 time: 0.430400 Jul 16 21:10:21.681490 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 335/663] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:10:21 2022] GET /compute/v2.1/servers/7d624313-0c12-4da2-a95c-3903efae7608 => generated 1468 bytes in 437 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:10:21.694572 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-4309b3db-ba25-4f89-8686-a74dbbacc580 tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:21.695332 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-4309b3db-ba25-4f89-8686-a74dbbacc580 tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] [instance: cf53f7cd-deae-4b34-bf8e-3afef98b2ec6] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:21.704767 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4309b3db-ba25-4f89-8686-a74dbbacc580 tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:21.705405 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4309b3db-ba25-4f89-8686-a74dbbacc580 tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:21.706375 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4309b3db-ba25-4f89-8686-a74dbbacc580 tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:21.706967 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4309b3db-ba25-4f89-8686-a74dbbacc580 tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:21.856117 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.objects.instance [None req-4309b3db-ba25-4f89-8686-a74dbbacc580 tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] Lazy-loading 'services' on Instance uuid cf53f7cd-deae-4b34-bf8e-3afef98b2ec6 {{(pid=138680) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:10:21.873780 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_api:SELECT=100 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:10:21.877777 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-4309b3db-ba25-4f89-8686-a74dbbacc580 tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/cf53f7cd-deae-4b34-bf8e-3afef98b2ec6" status: 200 len: 1929 microversion: 2.75 time: 0.195039 Jul 16 21:10:21.878368 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 336/664] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:10:21 2022] GET /compute/v2.1/servers/cf53f7cd-deae-4b34-bf8e-3afef98b2ec6 => generated 1929 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:10:21.882341 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-c4d214d2-abf0-4458-ac8a-69dd98c692cf tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a69f9dee03d43beaafa966216b9e289', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9b769bf8f3ea4e368bf68de1df6538f9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:21.883397 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-60419315-9c01-419d-954e-d42612808e31 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:21.884167 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-60419315-9c01-419d-954e-d42612808e31 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] [instance: 84ee8179-ff94-43b3-97f2-1867bc6420a8] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:21.892835 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-60419315-9c01-419d-954e-d42612808e31 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:21.893377 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-60419315-9c01-419d-954e-d42612808e31 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:21.894719 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-60419315-9c01-419d-954e-d42612808e31 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:21.895252 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-60419315-9c01-419d-954e-d42612808e31 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:21.896344 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-c4d214d2-abf0-4458-ac8a-69dd98c692cf tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] 10.222.0.16 "GET /compute/v2.1/servers/4ee3ffbe-c16a-4847-a343-469c645e4b25" status: 200 len: 1311 microversion: 2.1 time: 0.325218 Jul 16 21:10:21.896933 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 329/665] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:10:21 2022] GET /compute/v2.1/servers/4ee3ffbe-c16a-4847-a343-469c645e4b25 => generated 1311 bytes in 326 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:10:22.007185 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-f1109460-456b-447a-b0d9-818dc4de5c76 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:22.007824 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-f1109460-456b-447a-b0d9-818dc4de5c76 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] [instance: 8872e8cf-593e-4a7a-940d-e0c8159d4ad7] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:22.025654 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f1109460-456b-447a-b0d9-818dc4de5c76 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:22.026130 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f1109460-456b-447a-b0d9-818dc4de5c76 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:22.027508 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f1109460-456b-447a-b0d9-818dc4de5c76 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:22.027508 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f1109460-456b-447a-b0d9-818dc4de5c76 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:22.271024 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-60419315-9c01-419d-954e-d42612808e31 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'abeb7020019d41a18de42956d706a011', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '649babe54d104436af461f1ee4767001', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:22.281852 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-60419315-9c01-419d-954e-d42612808e31 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] 10.222.0.16 "GET /compute/v2.1/servers/84ee8179-ff94-43b3-97f2-1867bc6420a8" status: 200 len: 1468 microversion: 2.1 time: 0.402076 Jul 16 21:10:22.282620 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 337/666] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:10:21 2022] GET /compute/v2.1/servers/84ee8179-ff94-43b3-97f2-1867bc6420a8 => generated 1468 bytes in 404 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:10:22.287922 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-8afc1e5c-8091-42b4-8c58-ef17b9c8e434 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:22.288717 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-8afc1e5c-8091-42b4-8c58-ef17b9c8e434 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] [instance: 1631f0cc-de67-4c0d-abd0-3e0e11dfb4ed] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:22.295771 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8afc1e5c-8091-42b4-8c58-ef17b9c8e434 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:22.296150 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8afc1e5c-8091-42b4-8c58-ef17b9c8e434 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:22.297313 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8afc1e5c-8091-42b4-8c58-ef17b9c8e434 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:22.298087 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8afc1e5c-8091-42b4-8c58-ef17b9c8e434 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:22.314930 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-8afc1e5c-8091-42b4-8c58-ef17b9c8e434 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] HTTP exception thrown: Instance 1631f0cc-de67-4c0d-abd0-3e0e11dfb4ed could not be found. Jul 16 21:10:22.316852 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-8afc1e5c-8091-42b4-8c58-ef17b9c8e434 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Request method failure captured: Jul 16 21:10:22.316852 ncdv-master-1 devstack@n-api.service[138680]: request: GET /compute/v2.1/servers/1631f0cc-de67-4c0d-abd0-3e0e11dfb4ed HTTP/1.1 Jul 16 21:10:22.316852 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:10:22.316852 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:10:22.316852 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:10:22.316852 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 0 Jul 16 21:10:22.316852 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:10:22.316852 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:10:22.316852 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:10:22.316852 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:10:22.316852 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0yjxC34cYKsuWpx3tkgld3eV-A0-TjeQg_5ds20qHYp2Q6eY4sL39TrWBZ8MuWEkv-IWvSNQ1dzN43zHZHJVYwNvVGidmbekkFV44mT7Ab0o-XbW7DoqN-TrMfFTdGt5XtMhdkfAqddbANBDHWTKcPsdqIMuW-_kx0M-4rvUNYWtOkk Jul 16 21:10:22.316852 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:10:22.316852 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:10:22.316852 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:10:22.316852 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:10:22.316852 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:10:22.316852 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:10:22.316852 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: fc1c3463f9444119af9d18f85bdd703d Jul 16 21:10:22.316852 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-LiveMigrationNegativeTest-742579909 Jul 16 21:10:22.316852 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:10:22.316852 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:10:22.318156 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/fc1c3463f9444119af9d18f85bdd703d"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/fc1c3463f9444119af9d18f85bdd703d"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_fc1c3463f9444119af9d18f85bdd703d"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/fc1c3463f9444119af9d18f85bdd703d"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:10:22.318156 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-LiveMigrationNegativeTest-742579909 Jul 16 21:10:22.318156 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: fc1c3463f9444119af9d18f85bdd703d Jul 16 21:10:22.318156 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-LiveMigrationNegativeTest-742579909 Jul 16 21:10:22.318156 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-LiveMigrationNegativeTest-742579909-project Jul 16 21:10:22.318156 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:10:22.318156 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:10:22.318156 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: 0d72626f465b4f37ad3b18bb945d4000 Jul 16 21:10:22.318156 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-LiveMigrationNegativeTest-742579909-project Jul 16 21:10:22.318156 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 21:10:22.318156 ncdv-master-1 devstack@n-api.service[138680]: exception: Instance 1631f0cc-de67-4c0d-abd0-3e0e11dfb4ed could not be found. Jul 16 21:10:22.318156 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': '1631f0cc-de67-4c0d-abd0-3e0e11dfb4ed'} Jul 16 21:10:22.318837 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:10:22.318837 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:10:22.318837 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:10:22.318837 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:10:22.318837 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:10:22.318837 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:10:22.318837 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:10:22.318837 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:10:22.318837 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:10:22.318837 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:10:22.318837 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 1631f0cc-de67-4c0d-abd0-3e0e11dfb4ed could not be found. Jul 16 21:10:22.318837 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:10:22.318837 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:10:22.318837 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:10:22.318837 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:10:22.318837 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:10:22.318837 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:10:22.318837 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:10:22.318837 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:10:22.318837 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:10:22.319950 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:10:22.319950 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:10:22.319950 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:10:22.319950 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:10:22.319950 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:10:22.319950 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:10:22.319950 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:10:22.319950 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 1631f0cc-de67-4c0d-abd0-3e0e11dfb4ed could not be found. Jul 16 21:10:22.319950 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:10:22.319950 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:10:22.319950 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:10:22.319950 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:10:22.319950 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:10:22.319950 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:10:22.319950 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:10:22.319950 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:10:22.319950 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 1631f0cc-de67-4c0d-abd0-3e0e11dfb4ed could not be found. Jul 16 21:10:22.319950 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:10:22.325035 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-8afc1e5c-8091-42b4-8c58-ef17b9c8e434 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] Returning 404 to user: Instance 1631f0cc-de67-4c0d-abd0-3e0e11dfb4ed could not be found. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:10:22.325035 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-8afc1e5c-8091-42b4-8c58-ef17b9c8e434 tempest-LiveMigrationNegativeTest-742579909 tempest-LiveMigrationNegativeTest-742579909-project] 10.222.0.16 "GET /compute/v2.1/servers/1631f0cc-de67-4c0d-abd0-3e0e11dfb4ed" status: 404 len: 111 microversion: 2.1 time: 0.037224 Jul 16 21:10:22.325035 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 338/667] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:10:22 2022] GET /compute/v2.1/servers/1631f0cc-de67-4c0d-abd0-3e0e11dfb4ed => generated 111 bytes in 38 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:10:22.340422 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-db859e11-e67e-4543-b852-1713b5036dd1 tempest-ServerDiagnosticsNegativeTest-1844467799 tempest-ServerDiagnosticsNegativeTest-1844467799-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:22.341483 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-db859e11-e67e-4543-b852-1713b5036dd1 tempest-ServerDiagnosticsNegativeTest-1844467799 tempest-ServerDiagnosticsNegativeTest-1844467799-project] [instance: 1e4229cb-4ed9-48c3-b723-28619ae7045c] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:22.351896 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-db859e11-e67e-4543-b852-1713b5036dd1 tempest-ServerDiagnosticsNegativeTest-1844467799 tempest-ServerDiagnosticsNegativeTest-1844467799-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:22.352730 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-db859e11-e67e-4543-b852-1713b5036dd1 tempest-ServerDiagnosticsNegativeTest-1844467799 tempest-ServerDiagnosticsNegativeTest-1844467799-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:22.354290 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-db859e11-e67e-4543-b852-1713b5036dd1 tempest-ServerDiagnosticsNegativeTest-1844467799 tempest-ServerDiagnosticsNegativeTest-1844467799-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:22.354994 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-db859e11-e67e-4543-b852-1713b5036dd1 tempest-ServerDiagnosticsNegativeTest-1844467799 tempest-ServerDiagnosticsNegativeTest-1844467799-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:22.355348 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-f1109460-456b-447a-b0d9-818dc4de5c76 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ef9cbce2b2e64665b7f50767342dcb46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0ee02e4c8b448f2b77798ae504129b9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:22.366185 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-f1109460-456b-447a-b0d9-818dc4de5c76 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] 10.222.0.16 "GET /compute/v2.1/servers/8872e8cf-593e-4a7a-940d-e0c8159d4ad7" status: 200 len: 1588 microversion: 2.1 time: 0.362953 Jul 16 21:10:22.368260 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 330/668] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:10:21 2022] GET /compute/v2.1/servers/8872e8cf-593e-4a7a-940d-e0c8159d4ad7 => generated 1588 bytes in 369 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:10:22.380533 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-db859e11-e67e-4543-b852-1713b5036dd1 tempest-ServerDiagnosticsNegativeTest-1844467799 tempest-ServerDiagnosticsNegativeTest-1844467799-project] HTTP exception thrown: Instance 1e4229cb-4ed9-48c3-b723-28619ae7045c could not be found. Jul 16 21:10:22.382724 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-596fafe5-8d21-4170-901a-5f691c6f0797 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:22.383094 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-db859e11-e67e-4543-b852-1713b5036dd1 tempest-ServerDiagnosticsNegativeTest-1844467799 tempest-ServerDiagnosticsNegativeTest-1844467799-project] Request method failure captured: Jul 16 21:10:22.383094 ncdv-master-1 devstack@n-api.service[138680]: request: GET /compute/v2.1/servers/1e4229cb-4ed9-48c3-b723-28619ae7045c HTTP/1.1 Jul 16 21:10:22.383094 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:10:22.383094 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:10:22.383094 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:10:22.383094 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 0 Jul 16 21:10:22.383094 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:10:22.383094 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:10:22.383094 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:10:22.383094 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:10:22.383094 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0ykfP999Jj_jyZ6OBjYyR9SL49JtWK62ZihwPNiXI5Yi1et43fvJNyoNSC7033r8QSoqpzzSVQA9-4iHQQzaBbmCSCKh2HYjy_dBpUIRYGIrPQz8x0mV49gXL-Nbi3yYEcmJ9PEAz83Nffjt3otj__V_Qej84Zgo99lmGAlv44iak0w Jul 16 21:10:22.383094 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:10:22.383094 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:10:22.383094 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:10:22.383094 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:10:22.383094 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:10:22.383094 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:10:22.383094 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: 5f9cdca4179c4875a4be57cf8a9f46a8 Jul 16 21:10:22.383094 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-ServerDiagnosticsNegativeTest-1844467799 Jul 16 21:10:22.383094 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:10:22.383094 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:10:22.385976 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/5f9cdca4179c4875a4be57cf8a9f46a8"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/5f9cdca4179c4875a4be57cf8a9f46a8"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_5f9cdca4179c4875a4be57cf8a9f46a8"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/5f9cdca4179c4875a4be57cf8a9f46a8"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:10:22.385976 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-ServerDiagnosticsNegativeTest-1844467799 Jul 16 21:10:22.385976 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: 5f9cdca4179c4875a4be57cf8a9f46a8 Jul 16 21:10:22.385976 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-ServerDiagnosticsNegativeTest-1844467799 Jul 16 21:10:22.385976 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-ServerDiagnosticsNegativeTest-1844467799-project Jul 16 21:10:22.385976 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:10:22.385976 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:10:22.385976 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: 9c428093bcf445a6aea6ddb209e6e5cf Jul 16 21:10:22.385976 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-ServerDiagnosticsNegativeTest-1844467799-project Jul 16 21:10:22.385976 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 21:10:22.385976 ncdv-master-1 devstack@n-api.service[138680]: exception: Instance 1e4229cb-4ed9-48c3-b723-28619ae7045c could not be found. Jul 16 21:10:22.385976 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': '1e4229cb-4ed9-48c3-b723-28619ae7045c'} Jul 16 21:10:22.386661 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:10:22.386661 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:10:22.386661 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:10:22.386661 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:10:22.386661 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:10:22.386661 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:10:22.386661 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:10:22.386661 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:10:22.386661 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:10:22.386661 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:10:22.386661 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 1e4229cb-4ed9-48c3-b723-28619ae7045c could not be found. Jul 16 21:10:22.386661 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:10:22.386661 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:10:22.386661 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:10:22.386661 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:10:22.386661 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:10:22.386661 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:10:22.386661 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:10:22.386661 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:10:22.386661 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:10:22.388110 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:10:22.388110 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:10:22.388110 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:10:22.388110 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:10:22.388110 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:10:22.388110 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:10:22.388110 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:10:22.388110 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 1e4229cb-4ed9-48c3-b723-28619ae7045c could not be found. Jul 16 21:10:22.388110 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:10:22.388110 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:10:22.388110 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:10:22.388110 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:10:22.388110 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:10:22.388110 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:10:22.388110 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:10:22.388110 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:10:22.388110 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 1e4229cb-4ed9-48c3-b723-28619ae7045c could not be found. Jul 16 21:10:22.388110 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:10:22.389130 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-db859e11-e67e-4543-b852-1713b5036dd1 tempest-ServerDiagnosticsNegativeTest-1844467799 tempest-ServerDiagnosticsNegativeTest-1844467799-project] Returning 404 to user: Instance 1e4229cb-4ed9-48c3-b723-28619ae7045c could not be found. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:10:22.389130 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-db859e11-e67e-4543-b852-1713b5036dd1 tempest-ServerDiagnosticsNegativeTest-1844467799 tempest-ServerDiagnosticsNegativeTest-1844467799-project] 10.222.0.16 "GET /compute/v2.1/servers/1e4229cb-4ed9-48c3-b723-28619ae7045c" status: 404 len: 111 microversion: 2.1 time: 0.061764 Jul 16 21:10:22.389130 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 339/669] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:10:22 2022] GET /compute/v2.1/servers/1e4229cb-4ed9-48c3-b723-28619ae7045c => generated 111 bytes in 65 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:10:22.389329 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-596fafe5-8d21-4170-901a-5f691c6f0797 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Policy check for os_compute_api:servers:allow_all_filters failed with credentials {'is_admin': False, 'user_id': 'abeb7020019d41a18de42956d706a011', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '649babe54d104436af461f1ee4767001', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:22.393263 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-0d08cdcf-8396-4a98-8d7f-4a33d922d159 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:22.394021 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-0d08cdcf-8396-4a98-8d7f-4a33d922d159 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] [instance: 8872e8cf-593e-4a7a-940d-e0c8159d4ad7] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:22.394846 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-596fafe5-8d21-4170-901a-5f691c6f0797 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Searching by: {'reservation_id': 'r-tk0hg60y', 'deleted': False, 'project_id': '649babe54d104436af461f1ee4767001'} {{(pid=138681) get_all /opt/stack/nova/nova/compute/api.py:2962}} Jul 16 21:10:22.400550 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0d08cdcf-8396-4a98-8d7f-4a33d922d159 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:22.401256 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0d08cdcf-8396-4a98-8d7f-4a33d922d159 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:22.402202 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0d08cdcf-8396-4a98-8d7f-4a33d922d159 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:22.402796 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0d08cdcf-8396-4a98-8d7f-4a33d922d159 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:22.402919 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-596fafe5-8d21-4170-901a-5f691c6f0797 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:22.403351 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-596fafe5-8d21-4170-901a-5f691c6f0797 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:22.406543 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-596fafe5-8d21-4170-901a-5f691c6f0797 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:22.407047 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-596fafe5-8d21-4170-901a-5f691c6f0797 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:22.537666 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.multi_cell_list [None req-596fafe5-8d21-4170-901a-5f691c6f0797 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Listed batch of 2 results from cell out of 1000 limit. Returned 2 total so far. {{(pid=138681) do_query /opt/stack/nova/nova/compute/multi_cell_list.py:373}} Jul 16 21:10:22.625527 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-596fafe5-8d21-4170-901a-5f691c6f0797 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'abeb7020019d41a18de42956d706a011', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '649babe54d104436af461f1ee4767001', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:22.634403 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-596fafe5-8d21-4170-901a-5f691c6f0797 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:22.634858 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-596fafe5-8d21-4170-901a-5f691c6f0797 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:22.830486 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-0d08cdcf-8396-4a98-8d7f-4a33d922d159 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] 10.222.0.16 "GET /compute/v2.1/servers/8872e8cf-593e-4a7a-940d-e0c8159d4ad7/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.441106 Jul 16 21:10:22.831112 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 340/670] 10.222.0.16 () {58 vars in 1274 bytes} [Sat Jul 16 21:10:22 2022] GET /compute/v2.1/servers/8872e8cf-593e-4a7a-940d-e0c8159d4ad7/os-interface => generated 284 bytes in 443 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:10:22.836109 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-b69ce9cb-6591-4f52-9641-167057f94c87 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:22.836725 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-b69ce9cb-6591-4f52-9641-167057f94c87 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: f98a9762-a67c-4e95-85b8-727c81bdb03a] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:22.853308 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b69ce9cb-6591-4f52-9641-167057f94c87 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:22.854868 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b69ce9cb-6591-4f52-9641-167057f94c87 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:22.855879 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b69ce9cb-6591-4f52-9641-167057f94c87 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:22.856559 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b69ce9cb-6591-4f52-9641-167057f94c87 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:23.099893 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-596fafe5-8d21-4170-901a-5f691c6f0797 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] 10.222.0.16 "GET /compute/v2.1/servers/detail?reservation_id=r-tk0hg60y" status: 200 len: 2929 microversion: 2.1 time: 0.729793 Jul 16 21:10:23.101042 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 331/671] 10.222.0.16 () {58 vars in 1196 bytes} [Sat Jul 16 21:10:22 2022] GET /compute/v2.1/servers/detail?reservation_id=r-tk0hg60y => generated 2929 bytes in 732 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:10:23.106265 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-a2f01e19-b32e-4d20-93fe-9c71a4f18bb7 req-a42f63ec-8f9f-4074-9b94-3b77158609bc service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "4ee3ffbe-c16a-4847-a343-469c645e4b25", "tag": "f54ad45e-ef5d-4b3e-ae39-d46f5d43bc88"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:10:23.125987 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-a2f01e19-b32e-4d20-93fe-9c71a4f18bb7 req-a42f63ec-8f9f-4074-9b94-3b77158609bc service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:23.130248 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-a2f01e19-b32e-4d20-93fe-9c71a4f18bb7 req-a42f63ec-8f9f-4074-9b94-3b77158609bc service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:23.161347 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-a2f01e19-b32e-4d20-93fe-9c71a4f18bb7 req-a42f63ec-8f9f-4074-9b94-3b77158609bc service nova] Creating event network-changed:f54ad45e-ef5d-4b3e-ae39-d46f5d43bc88 for instance 4ee3ffbe-c16a-4847-a343-469c645e4b25 on nch1-master-1 Jul 16 21:10:23.170627 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-a2f01e19-b32e-4d20-93fe-9c71a4f18bb7 req-a42f63ec-8f9f-4074-9b94-3b77158609bc service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.068182 Jul 16 21:10:23.178502 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 332/672] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:10:23 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 71 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:10:23.180151 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-a7aeed80-e800-4bee-8797-bcea4e7ccead tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:23.182385 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-a7aeed80-e800-4bee-8797-bcea4e7ccead tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] [instance: cf53f7cd-deae-4b34-bf8e-3afef98b2ec6] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:23.192451 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-b69ce9cb-6591-4f52-9641-167057f94c87 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b68dacc4b8463ba07fae6372131117', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55a19b7a4682457a9132f5fc80110712', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:23.194773 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a7aeed80-e800-4bee-8797-bcea4e7ccead tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:23.196069 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a7aeed80-e800-4bee-8797-bcea4e7ccead tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:23.197060 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a7aeed80-e800-4bee-8797-bcea4e7ccead tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:23.197604 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a7aeed80-e800-4bee-8797-bcea4e7ccead tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:23.208271 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-b69ce9cb-6591-4f52-9641-167057f94c87 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "GET /compute/v2.1/servers/f98a9762-a67c-4e95-85b8-727c81bdb03a" status: 200 len: 1476 microversion: 2.1 time: 0.376009 Jul 16 21:10:23.222941 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 341/673] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:10:22 2022] GET /compute/v2.1/servers/f98a9762-a67c-4e95-85b8-727c81bdb03a => generated 1476 bytes in 391 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:10:23.237853 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-f5411614-4386-48c7-a3b5-3dd9350ff267 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:23.239136 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-f5411614-4386-48c7-a3b5-3dd9350ff267 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] [instance: 4ee3ffbe-c16a-4847-a343-469c645e4b25] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:23.254629 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f5411614-4386-48c7-a3b5-3dd9350ff267 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:23.258829 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f5411614-4386-48c7-a3b5-3dd9350ff267 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:23.260063 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f5411614-4386-48c7-a3b5-3dd9350ff267 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:23.263833 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f5411614-4386-48c7-a3b5-3dd9350ff267 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:23.391561 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.objects.instance [None req-a7aeed80-e800-4bee-8797-bcea4e7ccead tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] Lazy-loading 'services' on Instance uuid cf53f7cd-deae-4b34-bf8e-3afef98b2ec6 {{(pid=138681) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:10:23.433904 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-a7aeed80-e800-4bee-8797-bcea4e7ccead tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/cf53f7cd-deae-4b34-bf8e-3afef98b2ec6" status: 200 len: 1929 microversion: 2.75 time: 0.259821 Jul 16 21:10:23.447428 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 333/674] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:10:23 2022] GET /compute/v2.1/servers/cf53f7cd-deae-4b34-bf8e-3afef98b2ec6 => generated 1929 bytes in 274 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:10:23.452697 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-a54cbdd7-67e5-4e87-8fa5-65ee84059df5 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:23.493891 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.compute.server_groups [None req-a54cbdd7-67e5-4e87-8fa5-65ee84059df5 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Querying cell 327e8df1-39cc-4b36-9ea6-6a9b88ed54fc(cell1) for 2 instances {{(pid=138681) _get_not_deleted /opt/stack/nova/nova/api/openstack/compute/server_groups.py:69}} Jul 16 21:10:23.495934 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a54cbdd7-67e5-4e87-8fa5-65ee84059df5 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:23.497012 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a54cbdd7-67e5-4e87-8fa5-65ee84059df5 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:23.549573 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-a54cbdd7-67e5-4e87-8fa5-65ee84059df5 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] 10.222.0.16 "GET /compute/v2.1/os-server-groups/d6276585-4df3-40a1-817c-e7dcc6987bf4" status: 200 len: 269 microversion: 2.1 time: 0.100675 Jul 16 21:10:23.560207 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 334/675] 10.222.0.16 () {58 vars in 1262 bytes} [Sat Jul 16 21:10:23 2022] GET /compute/v2.1/os-server-groups/d6276585-4df3-40a1-817c-e7dcc6987bf4 => generated 269 bytes in 112 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:10:23.580403 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-d1059f48-f225-4415-8fc9-0d9fb04a2563 tempest-ServersOnMultiNodesTest-83775055 tempest-ServersOnMultiNodesTest-83775055-project-admin] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:23.581267 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-d1059f48-f225-4415-8fc9-0d9fb04a2563 tempest-ServersOnMultiNodesTest-83775055 tempest-ServersOnMultiNodesTest-83775055-project-admin] [instance: 7d624313-0c12-4da2-a95c-3903efae7608] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:23.588919 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d1059f48-f225-4415-8fc9-0d9fb04a2563 tempest-ServersOnMultiNodesTest-83775055 tempest-ServersOnMultiNodesTest-83775055-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:23.589426 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d1059f48-f225-4415-8fc9-0d9fb04a2563 tempest-ServersOnMultiNodesTest-83775055 tempest-ServersOnMultiNodesTest-83775055-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:23.590215 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d1059f48-f225-4415-8fc9-0d9fb04a2563 tempest-ServersOnMultiNodesTest-83775055 tempest-ServersOnMultiNodesTest-83775055-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:23.590657 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d1059f48-f225-4415-8fc9-0d9fb04a2563 tempest-ServersOnMultiNodesTest-83775055 tempest-ServersOnMultiNodesTest-83775055-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:23.669912 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-f5411614-4386-48c7-a3b5-3dd9350ff267 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a69f9dee03d43beaafa966216b9e289', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9b769bf8f3ea4e368bf68de1df6538f9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:23.685183 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-f5411614-4386-48c7-a3b5-3dd9350ff267 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] 10.222.0.16 "GET /compute/v2.1/servers/4ee3ffbe-c16a-4847-a343-469c645e4b25" status: 200 len: 1311 microversion: 2.1 time: 0.459204 Jul 16 21:10:23.686368 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 342/676] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:10:23 2022] GET /compute/v2.1/servers/4ee3ffbe-c16a-4847-a343-469c645e4b25 => generated 1311 bytes in 463 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:10:23.868390 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-d1059f48-f225-4415-8fc9-0d9fb04a2563 tempest-ServersOnMultiNodesTest-83775055 tempest-ServersOnMultiNodesTest-83775055-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/7d624313-0c12-4da2-a95c-3903efae7608" status: 200 len: 1619 microversion: 2.1 time: 0.296655 Jul 16 21:10:23.871981 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 335/677] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:10:23 2022] GET /compute/v2.1/servers/7d624313-0c12-4da2-a95c-3903efae7608 => generated 1619 bytes in 309 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:10:23.895544 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-39eeeafc-e3e9-4b69-8f99-1f047b7817f5 tempest-ServersOnMultiNodesTest-83775055 tempest-ServersOnMultiNodesTest-83775055-project-admin] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:23.901215 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-39eeeafc-e3e9-4b69-8f99-1f047b7817f5 tempest-ServersOnMultiNodesTest-83775055 tempest-ServersOnMultiNodesTest-83775055-project-admin] [instance: 84ee8179-ff94-43b3-97f2-1867bc6420a8] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:23.924722 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-39eeeafc-e3e9-4b69-8f99-1f047b7817f5 tempest-ServersOnMultiNodesTest-83775055 tempest-ServersOnMultiNodesTest-83775055-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:23.925523 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-39eeeafc-e3e9-4b69-8f99-1f047b7817f5 tempest-ServersOnMultiNodesTest-83775055 tempest-ServersOnMultiNodesTest-83775055-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:23.929145 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-39eeeafc-e3e9-4b69-8f99-1f047b7817f5 tempest-ServersOnMultiNodesTest-83775055 tempest-ServersOnMultiNodesTest-83775055-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:23.929919 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-39eeeafc-e3e9-4b69-8f99-1f047b7817f5 tempest-ServersOnMultiNodesTest-83775055 tempest-ServersOnMultiNodesTest-83775055-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:24.229300 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-35045b29-8a6c-4dad-9f88-4532c2859246 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:24.230167 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-35045b29-8a6c-4dad-9f88-4532c2859246 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: f98a9762-a67c-4e95-85b8-727c81bdb03a] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:24.240328 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-35045b29-8a6c-4dad-9f88-4532c2859246 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:24.241512 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-35045b29-8a6c-4dad-9f88-4532c2859246 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:24.242549 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-35045b29-8a6c-4dad-9f88-4532c2859246 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:24.243269 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-35045b29-8a6c-4dad-9f88-4532c2859246 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:24.249770 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-39eeeafc-e3e9-4b69-8f99-1f047b7817f5 tempest-ServersOnMultiNodesTest-83775055 tempest-ServersOnMultiNodesTest-83775055-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/84ee8179-ff94-43b3-97f2-1867bc6420a8" status: 200 len: 1619 microversion: 2.1 time: 0.360081 Jul 16 21:10:24.250937 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 343/678] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:10:23 2022] GET /compute/v2.1/servers/84ee8179-ff94-43b3-97f2-1867bc6420a8 => generated 1619 bytes in 362 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:10:24.412628 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-ea39fe49-e970-4ea0-9d4f-55739157f6f9 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServersOnMultiNodesTest-server-1921939260", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452"}, "os:scheduler_hints": {"different_host": "148d5ad9-2900-4c2b-909b-aa446757e33d"}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:10:24.527571 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.network.neutron [None req-ea39fe49-e970-4ea0-9d4f-55739157f6f9 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] validate_networks() for None {{(pid=138680) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 16 21:10:24.582082 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-35045b29-8a6c-4dad-9f88-4532c2859246 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b68dacc4b8463ba07fae6372131117', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55a19b7a4682457a9132f5fc80110712', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:24.590938 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-35045b29-8a6c-4dad-9f88-4532c2859246 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "GET /compute/v2.1/servers/f98a9762-a67c-4e95-85b8-727c81bdb03a" status: 200 len: 1476 microversion: 2.1 time: 0.367129 Jul 16 21:10:24.591751 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 336/679] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:10:24 2022] GET /compute/v2.1/servers/f98a9762-a67c-4e95-85b8-727c81bdb03a => generated 1476 bytes in 369 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:10:24.597128 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-50bae4a1-30c6-48e4-9602-0685f04117da tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:24.598201 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-50bae4a1-30c6-48e4-9602-0685f04117da tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] [instance: cf53f7cd-deae-4b34-bf8e-3afef98b2ec6] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:24.608627 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-50bae4a1-30c6-48e4-9602-0685f04117da tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:24.608627 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-50bae4a1-30c6-48e4-9602-0685f04117da tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:24.608627 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-50bae4a1-30c6-48e4-9602-0685f04117da tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:24.608627 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-50bae4a1-30c6-48e4-9602-0685f04117da tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:24.755387 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.objects.instance [None req-50bae4a1-30c6-48e4-9602-0685f04117da tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] Lazy-loading 'services' on Instance uuid cf53f7cd-deae-4b34-bf8e-3afef98b2ec6 {{(pid=138681) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:10:24.817636 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-50bae4a1-30c6-48e4-9602-0685f04117da tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/cf53f7cd-deae-4b34-bf8e-3afef98b2ec6" status: 200 len: 1929 microversion: 2.75 time: 0.224621 Jul 16 21:10:24.822733 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 337/680] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:10:24 2022] GET /compute/v2.1/servers/cf53f7cd-deae-4b34-bf8e-3afef98b2ec6 => generated 1929 bytes in 226 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:10:24.825202 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-580999d0-932c-4825-b002-dbf83fe96c4d tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:24.826254 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-580999d0-932c-4825-b002-dbf83fe96c4d tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] [instance: 4ee3ffbe-c16a-4847-a343-469c645e4b25] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:24.835641 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-580999d0-932c-4825-b002-dbf83fe96c4d tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:24.849007 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-580999d0-932c-4825-b002-dbf83fe96c4d tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:24.850370 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-580999d0-932c-4825-b002-dbf83fe96c4d tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:24.851072 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-580999d0-932c-4825-b002-dbf83fe96c4d tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:25.125301 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-ea39fe49-e970-4ea0-9d4f-55739157f6f9 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Flavor limits 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 16 21:10:25.125806 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-ea39fe49-e970-4ea0-9d4f-55739157f6f9 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Image limits 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 16 21:10:25.126338 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-ea39fe49-e970-4ea0-9d4f-55739157f6f9 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Flavor pref 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 16 21:10:25.126941 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-ea39fe49-e970-4ea0-9d4f-55739157f6f9 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Image pref 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 16 21:10:25.127405 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-ea39fe49-e970-4ea0-9d4f-55739157f6f9 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 16 21:10:25.140693 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ea39fe49-e970-4ea0-9d4f-55739157f6f9 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:25.141394 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ea39fe49-e970-4ea0-9d4f-55739157f6f9 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:25.142287 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ea39fe49-e970-4ea0-9d4f-55739157f6f9 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:25.142796 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ea39fe49-e970-4ea0-9d4f-55739157f6f9 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:25.162951 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-ea39fe49-e970-4ea0-9d4f-55739157f6f9 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Getting quotas for project 649babe54d104436af461f1ee4767001. Resources: {'ram', 'instances', 'cores'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:10:25.168588 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-ea39fe49-e970-4ea0-9d4f-55739157f6f9 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Getting quotas for user abeb7020019d41a18de42956d706a011 and project 649babe54d104436af461f1ee4767001. Resources: {'ram', 'instances', 'cores'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:10:25.173847 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-580999d0-932c-4825-b002-dbf83fe96c4d tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a69f9dee03d43beaafa966216b9e289', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9b769bf8f3ea4e368bf68de1df6538f9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:25.182644 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-ea39fe49-e970-4ea0-9d4f-55739157f6f9 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Going to run 1 instances... {{(pid=138680) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 16 21:10:25.184550 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-580999d0-932c-4825-b002-dbf83fe96c4d tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] 10.222.0.16 "GET /compute/v2.1/servers/4ee3ffbe-c16a-4847-a343-469c645e4b25" status: 200 len: 1311 microversion: 2.1 time: 0.364592 Jul 16 21:10:25.185145 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 338/681] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:10:24 2022] GET /compute/v2.1/servers/4ee3ffbe-c16a-4847-a343-469c645e4b25 => generated 1311 bytes in 366 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:10:25.187115 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-ea39fe49-e970-4ea0-9d4f-55739157f6f9 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] [instance: b941b324-a54d-446f-bf92-570a80ba2152] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138680) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 16 21:10:25.334281 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-4642af6d-8088-4731-88df-b8e1b76d6a70 req-204d8b88-167e-4a1e-87b5-31b327862f88 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "8872e8cf-593e-4a7a-940d-e0c8159d4ad7", "tag": "a07c49a0-9773-4e57-963c-ebcc388b002a"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:10:25.343560 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-4642af6d-8088-4731-88df-b8e1b76d6a70 req-204d8b88-167e-4a1e-87b5-31b327862f88 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:25.344051 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-4642af6d-8088-4731-88df-b8e1b76d6a70 req-204d8b88-167e-4a1e-87b5-31b327862f88 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:25.355663 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-4642af6d-8088-4731-88df-b8e1b76d6a70 req-204d8b88-167e-4a1e-87b5-31b327862f88 service nova] Creating event network-changed:a07c49a0-9773-4e57-963c-ebcc388b002a for instance 8872e8cf-593e-4a7a-940d-e0c8159d4ad7 on nch2-master-1 Jul 16 21:10:25.361523 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-4642af6d-8088-4731-88df-b8e1b76d6a70 req-204d8b88-167e-4a1e-87b5-31b327862f88 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.034988 Jul 16 21:10:25.362129 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 339/682] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:10:25 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 16 21:10:25.370713 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-5a4a8ead-5528-4450-bb8e-e8303fb83fe9 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:25.371379 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-5a4a8ead-5528-4450-bb8e-e8303fb83fe9 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] [instance: 8872e8cf-593e-4a7a-940d-e0c8159d4ad7] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:25.379553 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5a4a8ead-5528-4450-bb8e-e8303fb83fe9 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:25.380361 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5a4a8ead-5528-4450-bb8e-e8303fb83fe9 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:25.381920 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5a4a8ead-5528-4450-bb8e-e8303fb83fe9 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:25.382884 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5a4a8ead-5528-4450-bb8e-e8303fb83fe9 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:25.432865 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-ea39fe49-e970-4ea0-9d4f-55739157f6f9 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 1.025919 Jul 16 21:10:25.436467 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 344/683] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 21:10:24 2022] POST /compute/v2.1/servers => generated 380 bytes in 1034 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 16 21:10:25.456608 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-f584b5ff-633e-459d-8ba8-631f096b6ce1 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:25.457554 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-f584b5ff-633e-459d-8ba8-631f096b6ce1 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] [instance: b941b324-a54d-446f-bf92-570a80ba2152] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:25.700783 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-f584b5ff-633e-459d-8ba8-631f096b6ce1 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'abeb7020019d41a18de42956d706a011', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '649babe54d104436af461f1ee4767001', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:25.714014 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-f584b5ff-633e-459d-8ba8-631f096b6ce1 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] 10.222.0.16 "GET /compute/v2.1/servers/b941b324-a54d-446f-bf92-570a80ba2152" status: 200 len: 1217 microversion: 2.1 time: 0.270432 Jul 16 21:10:25.715104 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 345/684] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:10:25 2022] GET /compute/v2.1/servers/b941b324-a54d-446f-bf92-570a80ba2152 => generated 1217 bytes in 275 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:10:25.728559 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-9e8d99dd-2a35-4922-9720-4bcf06a42672 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:25.730331 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-5a4a8ead-5528-4450-bb8e-e8303fb83fe9 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ef9cbce2b2e64665b7f50767342dcb46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0ee02e4c8b448f2b77798ae504129b9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:25.731369 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-9e8d99dd-2a35-4922-9720-4bcf06a42672 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: f98a9762-a67c-4e95-85b8-727c81bdb03a] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:25.746207 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-5a4a8ead-5528-4450-bb8e-e8303fb83fe9 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] 10.222.0.16 "GET /compute/v2.1/servers/8872e8cf-593e-4a7a-940d-e0c8159d4ad7" status: 200 len: 1588 microversion: 2.1 time: 0.379594 Jul 16 21:10:25.756321 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9e8d99dd-2a35-4922-9720-4bcf06a42672 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:25.757188 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9e8d99dd-2a35-4922-9720-4bcf06a42672 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:25.758181 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9e8d99dd-2a35-4922-9720-4bcf06a42672 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:25.759588 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 340/685] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:10:25 2022] GET /compute/v2.1/servers/8872e8cf-593e-4a7a-940d-e0c8159d4ad7 => generated 1588 bytes in 394 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:10:25.775785 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9e8d99dd-2a35-4922-9720-4bcf06a42672 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:25.838964 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-aa1291d6-cb33-4f52-90ab-970cc269ec1a tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:25.839688 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-aa1291d6-cb33-4f52-90ab-970cc269ec1a tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] [instance: cf53f7cd-deae-4b34-bf8e-3afef98b2ec6] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:25.849977 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-aa1291d6-cb33-4f52-90ab-970cc269ec1a tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:25.850430 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-aa1291d6-cb33-4f52-90ab-970cc269ec1a tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:25.851339 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-aa1291d6-cb33-4f52-90ab-970cc269ec1a tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:25.851723 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-aa1291d6-cb33-4f52-90ab-970cc269ec1a tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:26.084371 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.objects.instance [None req-aa1291d6-cb33-4f52-90ab-970cc269ec1a tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] Lazy-loading 'services' on Instance uuid cf53f7cd-deae-4b34-bf8e-3afef98b2ec6 {{(pid=138681) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:10:26.121529 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-aa1291d6-cb33-4f52-90ab-970cc269ec1a tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/cf53f7cd-deae-4b34-bf8e-3afef98b2ec6" status: 200 len: 1929 microversion: 2.75 time: 0.286776 Jul 16 21:10:26.125121 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 341/686] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:10:25 2022] GET /compute/v2.1/servers/cf53f7cd-deae-4b34-bf8e-3afef98b2ec6 => generated 1929 bytes in 291 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 16 21:10:26.169897 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-9e8d99dd-2a35-4922-9720-4bcf06a42672 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b68dacc4b8463ba07fae6372131117', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55a19b7a4682457a9132f5fc80110712', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:26.182904 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-9e8d99dd-2a35-4922-9720-4bcf06a42672 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "GET /compute/v2.1/servers/f98a9762-a67c-4e95-85b8-727c81bdb03a" status: 200 len: 1476 microversion: 2.1 time: 0.466288 Jul 16 21:10:26.188703 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 346/687] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:10:25 2022] GET /compute/v2.1/servers/f98a9762-a67c-4e95-85b8-727c81bdb03a => generated 1476 bytes in 472 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:10:26.203888 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-22a56b49-e39d-4709-9596-6332b0f7dae8 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:26.204522 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-22a56b49-e39d-4709-9596-6332b0f7dae8 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] [instance: 4ee3ffbe-c16a-4847-a343-469c645e4b25] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:26.224642 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-22a56b49-e39d-4709-9596-6332b0f7dae8 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:26.227107 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-22a56b49-e39d-4709-9596-6332b0f7dae8 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:26.228151 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-22a56b49-e39d-4709-9596-6332b0f7dae8 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:26.231702 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-22a56b49-e39d-4709-9596-6332b0f7dae8 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:26.483601 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-22a56b49-e39d-4709-9596-6332b0f7dae8 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a69f9dee03d43beaafa966216b9e289', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9b769bf8f3ea4e368bf68de1df6538f9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:26.497653 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-22a56b49-e39d-4709-9596-6332b0f7dae8 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] 10.222.0.16 "GET /compute/v2.1/servers/4ee3ffbe-c16a-4847-a343-469c645e4b25" status: 200 len: 1311 microversion: 2.1 time: 0.298417 Jul 16 21:10:26.506025 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 342/688] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:10:26 2022] GET /compute/v2.1/servers/4ee3ffbe-c16a-4847-a343-469c645e4b25 => generated 1311 bytes in 307 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:10:26.732331 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-1486ada6-a257-410e-a94e-d608033ca9f7 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:26.733155 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-1486ada6-a257-410e-a94e-d608033ca9f7 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] [instance: b941b324-a54d-446f-bf92-570a80ba2152] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:26.838349 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-1486ada6-a257-410e-a94e-d608033ca9f7 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'abeb7020019d41a18de42956d706a011', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '649babe54d104436af461f1ee4767001', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:26.845782 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-1486ada6-a257-410e-a94e-d608033ca9f7 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] 10.222.0.16 "GET /compute/v2.1/servers/b941b324-a54d-446f-bf92-570a80ba2152" status: 200 len: 1217 microversion: 2.1 time: 0.117340 Jul 16 21:10:26.857111 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 347/689] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:10:26 2022] GET /compute/v2.1/servers/b941b324-a54d-446f-bf92-570a80ba2152 => generated 1217 bytes in 129 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:10:27.148608 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-6359450b-06d1-4ead-9f9a-36a61145ecf2 tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:27.149982 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-6359450b-06d1-4ead-9f9a-36a61145ecf2 tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] [instance: cf53f7cd-deae-4b34-bf8e-3afef98b2ec6] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:27.163637 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6359450b-06d1-4ead-9f9a-36a61145ecf2 tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:27.164562 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6359450b-06d1-4ead-9f9a-36a61145ecf2 tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:27.165773 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6359450b-06d1-4ead-9f9a-36a61145ecf2 tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:27.167001 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6359450b-06d1-4ead-9f9a-36a61145ecf2 tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:27.218010 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-d9b367bc-2c73-4924-a6e1-1088197181d7 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:27.219330 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-d9b367bc-2c73-4924-a6e1-1088197181d7 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: f98a9762-a67c-4e95-85b8-727c81bdb03a] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:27.224122 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=100 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:10:27.231800 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d9b367bc-2c73-4924-a6e1-1088197181d7 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:27.233591 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d9b367bc-2c73-4924-a6e1-1088197181d7 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:27.237726 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d9b367bc-2c73-4924-a6e1-1088197181d7 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:27.244332 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d9b367bc-2c73-4924-a6e1-1088197181d7 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:27.317317 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.objects.instance [None req-6359450b-06d1-4ead-9f9a-36a61145ecf2 tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] Lazy-loading 'services' on Instance uuid cf53f7cd-deae-4b34-bf8e-3afef98b2ec6 {{(pid=138681) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:10:27.362514 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-6359450b-06d1-4ead-9f9a-36a61145ecf2 tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/cf53f7cd-deae-4b34-bf8e-3afef98b2ec6" status: 200 len: 1929 microversion: 2.75 time: 0.221776 Jul 16 21:10:27.367166 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 343/690] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:10:27 2022] GET /compute/v2.1/servers/cf53f7cd-deae-4b34-bf8e-3afef98b2ec6 => generated 1929 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:10:27.523149 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-a3119237-b750-40ed-a7de-daaee51e4cb7 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:27.523881 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-a3119237-b750-40ed-a7de-daaee51e4cb7 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] [instance: 4ee3ffbe-c16a-4847-a343-469c645e4b25] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:27.540972 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a3119237-b750-40ed-a7de-daaee51e4cb7 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:27.547204 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a3119237-b750-40ed-a7de-daaee51e4cb7 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:27.549218 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a3119237-b750-40ed-a7de-daaee51e4cb7 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:27.552118 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a3119237-b750-40ed-a7de-daaee51e4cb7 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:27.595653 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-d9b367bc-2c73-4924-a6e1-1088197181d7 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b68dacc4b8463ba07fae6372131117', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55a19b7a4682457a9132f5fc80110712', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:27.604503 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-d9b367bc-2c73-4924-a6e1-1088197181d7 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "GET /compute/v2.1/servers/f98a9762-a67c-4e95-85b8-727c81bdb03a" status: 200 len: 1476 microversion: 2.1 time: 0.403267 Jul 16 21:10:27.607300 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 348/691] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:10:27 2022] GET /compute/v2.1/servers/f98a9762-a67c-4e95-85b8-727c81bdb03a => generated 1476 bytes in 406 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:10:27.809355 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-a3119237-b750-40ed-a7de-daaee51e4cb7 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a69f9dee03d43beaafa966216b9e289', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9b769bf8f3ea4e368bf68de1df6538f9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:27.814678 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-a3119237-b750-40ed-a7de-daaee51e4cb7 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] 10.222.0.16 "GET /compute/v2.1/servers/4ee3ffbe-c16a-4847-a343-469c645e4b25" status: 200 len: 1311 microversion: 2.1 time: 0.296600 Jul 16 21:10:27.825247 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 344/692] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:10:27 2022] GET /compute/v2.1/servers/4ee3ffbe-c16a-4847-a343-469c645e4b25 => generated 1311 bytes in 308 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:10:27.864910 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-eff9ca32-6bf8-46b3-9b5e-08071b358a98 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:27.865545 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-eff9ca32-6bf8-46b3-9b5e-08071b358a98 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] [instance: b941b324-a54d-446f-bf92-570a80ba2152] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:27.882535 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-eff9ca32-6bf8-46b3-9b5e-08071b358a98 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:27.883990 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-eff9ca32-6bf8-46b3-9b5e-08071b358a98 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:27.887622 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-eff9ca32-6bf8-46b3-9b5e-08071b358a98 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:27.888487 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-eff9ca32-6bf8-46b3-9b5e-08071b358a98 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:27.931834 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=100 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:10:28.011555 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-eff9ca32-6bf8-46b3-9b5e-08071b358a98 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'abeb7020019d41a18de42956d706a011', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '649babe54d104436af461f1ee4767001', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:28.022431 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-eff9ca32-6bf8-46b3-9b5e-08071b358a98 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] 10.222.0.16 "GET /compute/v2.1/servers/b941b324-a54d-446f-bf92-570a80ba2152" status: 200 len: 1269 microversion: 2.1 time: 0.162844 Jul 16 21:10:28.023464 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 349/693] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:10:27 2022] GET /compute/v2.1/servers/b941b324-a54d-446f-bf92-570a80ba2152 => generated 1269 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:10:28.393081 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-77298fda-f33c-4b1b-a187-b5aaf071caeb tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:28.394041 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-77298fda-f33c-4b1b-a187-b5aaf071caeb tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] [instance: cf53f7cd-deae-4b34-bf8e-3afef98b2ec6] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:28.407834 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-77298fda-f33c-4b1b-a187-b5aaf071caeb tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:28.410814 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-77298fda-f33c-4b1b-a187-b5aaf071caeb tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:28.411749 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-77298fda-f33c-4b1b-a187-b5aaf071caeb tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:28.412707 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-77298fda-f33c-4b1b-a187-b5aaf071caeb tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:28.627497 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-fc9f3a5e-9213-4b9f-bdd6-3044bb1cc8a3 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:28.628713 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-fc9f3a5e-9213-4b9f-bdd6-3044bb1cc8a3 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: f98a9762-a67c-4e95-85b8-727c81bdb03a] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:28.629252 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.objects.instance [None req-77298fda-f33c-4b1b-a187-b5aaf071caeb tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] Lazy-loading 'services' on Instance uuid cf53f7cd-deae-4b34-bf8e-3afef98b2ec6 {{(pid=138681) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:10:28.644788 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-fc9f3a5e-9213-4b9f-bdd6-3044bb1cc8a3 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:28.645103 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-fc9f3a5e-9213-4b9f-bdd6-3044bb1cc8a3 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:28.645902 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-fc9f3a5e-9213-4b9f-bdd6-3044bb1cc8a3 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:28.646353 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-fc9f3a5e-9213-4b9f-bdd6-3044bb1cc8a3 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:28.667720 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-77298fda-f33c-4b1b-a187-b5aaf071caeb tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/cf53f7cd-deae-4b34-bf8e-3afef98b2ec6" status: 200 len: 1929 microversion: 2.75 time: 0.279332 Jul 16 21:10:28.686856 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 345/694] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:10:28 2022] GET /compute/v2.1/servers/cf53f7cd-deae-4b34-bf8e-3afef98b2ec6 => generated 1929 bytes in 294 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:10:28.832952 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-9b038262-859c-4b0c-8966-9c139ea06858 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:28.833660 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-9b038262-859c-4b0c-8966-9c139ea06858 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] [instance: 4ee3ffbe-c16a-4847-a343-469c645e4b25] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:28.860519 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9b038262-859c-4b0c-8966-9c139ea06858 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:28.861385 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9b038262-859c-4b0c-8966-9c139ea06858 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:28.862261 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9b038262-859c-4b0c-8966-9c139ea06858 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:28.862952 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9b038262-859c-4b0c-8966-9c139ea06858 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:29.287984 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-fc9f3a5e-9213-4b9f-bdd6-3044bb1cc8a3 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b68dacc4b8463ba07fae6372131117', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55a19b7a4682457a9132f5fc80110712', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:29.294564 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-fc9f3a5e-9213-4b9f-bdd6-3044bb1cc8a3 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "GET /compute/v2.1/servers/f98a9762-a67c-4e95-85b8-727c81bdb03a" status: 200 len: 1476 microversion: 2.1 time: 0.671571 Jul 16 21:10:29.295570 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 350/695] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:10:28 2022] GET /compute/v2.1/servers/f98a9762-a67c-4e95-85b8-727c81bdb03a => generated 1476 bytes in 675 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:10:29.310689 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-f6715caa-384c-45bd-81fe-b895a0074ba6 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:29.312184 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-f6715caa-384c-45bd-81fe-b895a0074ba6 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] [instance: b941b324-a54d-446f-bf92-570a80ba2152] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:29.324629 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f6715caa-384c-45bd-81fe-b895a0074ba6 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:29.325363 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f6715caa-384c-45bd-81fe-b895a0074ba6 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:29.326457 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f6715caa-384c-45bd-81fe-b895a0074ba6 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:29.328068 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f6715caa-384c-45bd-81fe-b895a0074ba6 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:29.361318 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-9b038262-859c-4b0c-8966-9c139ea06858 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a69f9dee03d43beaafa966216b9e289', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9b769bf8f3ea4e368bf68de1df6538f9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:29.371523 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-9b038262-859c-4b0c-8966-9c139ea06858 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] 10.222.0.16 "GET /compute/v2.1/servers/4ee3ffbe-c16a-4847-a343-469c645e4b25" status: 200 len: 1311 microversion: 2.1 time: 0.542813 Jul 16 21:10:29.373609 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 346/696] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:10:28 2022] GET /compute/v2.1/servers/4ee3ffbe-c16a-4847-a343-469c645e4b25 => generated 1311 bytes in 546 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:10:29.479258 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-f6715caa-384c-45bd-81fe-b895a0074ba6 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'abeb7020019d41a18de42956d706a011', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '649babe54d104436af461f1ee4767001', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:29.488838 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-f6715caa-384c-45bd-81fe-b895a0074ba6 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] 10.222.0.16 "GET /compute/v2.1/servers/b941b324-a54d-446f-bf92-570a80ba2152" status: 200 len: 1275 microversion: 2.1 time: 0.192094 Jul 16 21:10:29.489722 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 351/697] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:10:29 2022] GET /compute/v2.1/servers/b941b324-a54d-446f-bf92-570a80ba2152 => generated 1275 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:10:29.690071 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-481365e8-6b77-4500-9cd3-b8322052b60c tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:29.690694 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-481365e8-6b77-4500-9cd3-b8322052b60c tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] [instance: cf53f7cd-deae-4b34-bf8e-3afef98b2ec6] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:29.697720 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-481365e8-6b77-4500-9cd3-b8322052b60c tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:29.698099 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-481365e8-6b77-4500-9cd3-b8322052b60c tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:29.698892 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-481365e8-6b77-4500-9cd3-b8322052b60c tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:29.699368 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-481365e8-6b77-4500-9cd3-b8322052b60c tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:29.842200 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.objects.instance [None req-481365e8-6b77-4500-9cd3-b8322052b60c tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] Lazy-loading 'services' on Instance uuid cf53f7cd-deae-4b34-bf8e-3afef98b2ec6 {{(pid=138681) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:10:29.876380 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-481365e8-6b77-4500-9cd3-b8322052b60c tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/cf53f7cd-deae-4b34-bf8e-3afef98b2ec6" status: 200 len: 1929 microversion: 2.75 time: 0.189902 Jul 16 21:10:29.878414 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 347/698] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:10:29 2022] GET /compute/v2.1/servers/cf53f7cd-deae-4b34-bf8e-3afef98b2ec6 => generated 1929 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:10:30.312597 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-e47df31b-4552-44cf-989f-690159c28d75 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:30.315305 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-e47df31b-4552-44cf-989f-690159c28d75 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: f98a9762-a67c-4e95-85b8-727c81bdb03a] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:30.321900 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e47df31b-4552-44cf-989f-690159c28d75 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:30.322845 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e47df31b-4552-44cf-989f-690159c28d75 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:30.323914 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e47df31b-4552-44cf-989f-690159c28d75 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:30.324594 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e47df31b-4552-44cf-989f-690159c28d75 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:30.440507 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-48917c87-d4cb-41d2-946f-81b9eab79ead tempest-QuotasAdminNegativeTest-760882802 tempest-QuotasAdminNegativeTest-760882802-project-admin] Calling method '.version_select at 0x7fdd9dd3c940>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:30.522684 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-48917c87-d4cb-41d2-946f-81b9eab79ead tempest-QuotasAdminNegativeTest-760882802 tempest-QuotasAdminNegativeTest-760882802-project-admin] 10.222.0.16 "GET /compute/v2.1/os-quota-sets/29f6db32114b458685269df31fae51c6" status: 200 len: 371 microversion: 2.1 time: 0.297077 Jul 16 21:10:30.524099 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 352/699] 10.222.0.16 () {58 vars in 1241 bytes} [Sat Jul 16 21:10:30 2022] GET /compute/v2.1/os-quota-sets/29f6db32114b458685269df31fae51c6 => generated 371 bytes in 299 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:10:30.536883 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-9a22dee0-9d4b-4e18-bac0-92891b32effa tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:30.537833 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-9a22dee0-9d4b-4e18-bac0-92891b32effa tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] [instance: 4ee3ffbe-c16a-4847-a343-469c645e4b25] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:30.547202 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9a22dee0-9d4b-4e18-bac0-92891b32effa tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:30.547887 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9a22dee0-9d4b-4e18-bac0-92891b32effa tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:30.548967 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9a22dee0-9d4b-4e18-bac0-92891b32effa tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:30.549638 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9a22dee0-9d4b-4e18-bac0-92891b32effa tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:30.850198 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-e47df31b-4552-44cf-989f-690159c28d75 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b68dacc4b8463ba07fae6372131117', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55a19b7a4682457a9132f5fc80110712', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:30.862834 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-e47df31b-4552-44cf-989f-690159c28d75 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "GET /compute/v2.1/servers/f98a9762-a67c-4e95-85b8-727c81bdb03a" status: 200 len: 1476 microversion: 2.1 time: 0.554379 Jul 16 21:10:30.864694 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 348/700] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:10:30 2022] GET /compute/v2.1/servers/f98a9762-a67c-4e95-85b8-727c81bdb03a => generated 1476 bytes in 557 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:10:30.872966 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-64a6739e-42d7-4224-9e7e-49352548e3ce tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:30.873463 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-64a6739e-42d7-4224-9e7e-49352548e3ce tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] [instance: b941b324-a54d-446f-bf92-570a80ba2152] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:30.881262 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-64a6739e-42d7-4224-9e7e-49352548e3ce tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:30.882345 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-64a6739e-42d7-4224-9e7e-49352548e3ce tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:30.883184 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-64a6739e-42d7-4224-9e7e-49352548e3ce tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:30.883832 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-64a6739e-42d7-4224-9e7e-49352548e3ce tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:30.989182 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-9a22dee0-9d4b-4e18-bac0-92891b32effa tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a69f9dee03d43beaafa966216b9e289', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9b769bf8f3ea4e368bf68de1df6538f9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:31.007817 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-9a22dee0-9d4b-4e18-bac0-92891b32effa tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] 10.222.0.16 "GET /compute/v2.1/servers/4ee3ffbe-c16a-4847-a343-469c645e4b25" status: 200 len: 1311 microversion: 2.1 time: 0.480979 Jul 16 21:10:31.014040 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 353/701] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:10:30 2022] GET /compute/v2.1/servers/4ee3ffbe-c16a-4847-a343-469c645e4b25 => generated 1311 bytes in 489 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:10:31.040178 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-fe7845d3-f6f0-436d-9435-ae11c65f1dcd tempest-QuotasAdminNegativeTest-760882802 tempest-QuotasAdminNegativeTest-760882802-project-admin] Action: 'update', calling method: .version_select at 0x7fdd9db928b0>, body: {"quota_set": {"force": true, "cores": 0}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:10:31.133266 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-fe7845d3-f6f0-436d-9435-ae11c65f1dcd tempest-QuotasAdminNegativeTest-760882802 tempest-QuotasAdminNegativeTest-760882802-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:31.138024 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-fe7845d3-f6f0-436d-9435-ae11c65f1dcd tempest-QuotasAdminNegativeTest-760882802 tempest-QuotasAdminNegativeTest-760882802-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:31.138024 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-fe7845d3-f6f0-436d-9435-ae11c65f1dcd tempest-QuotasAdminNegativeTest-760882802 tempest-QuotasAdminNegativeTest-760882802-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:31.138024 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-fe7845d3-f6f0-436d-9435-ae11c65f1dcd tempest-QuotasAdminNegativeTest-760882802 tempest-QuotasAdminNegativeTest-760882802-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:31.147206 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-64a6739e-42d7-4224-9e7e-49352548e3ce tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'abeb7020019d41a18de42956d706a011', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '649babe54d104436af461f1ee4767001', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:31.163090 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-64a6739e-42d7-4224-9e7e-49352548e3ce tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] 10.222.0.16 "GET /compute/v2.1/servers/b941b324-a54d-446f-bf92-570a80ba2152" status: 200 len: 1317 microversion: 2.1 time: 0.289478 Jul 16 21:10:31.164470 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 349/702] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:10:30 2022] GET /compute/v2.1/servers/b941b324-a54d-446f-bf92-570a80ba2152 => generated 1317 bytes in 299 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:10:31.169255 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-01e5548f-b263-42a2-8a5c-64bd9a1f5f89 tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:31.170053 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-01e5548f-b263-42a2-8a5c-64bd9a1f5f89 tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] [instance: cf53f7cd-deae-4b34-bf8e-3afef98b2ec6] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:31.181638 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-01e5548f-b263-42a2-8a5c-64bd9a1f5f89 tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:31.184054 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-01e5548f-b263-42a2-8a5c-64bd9a1f5f89 tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:31.187052 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-01e5548f-b263-42a2-8a5c-64bd9a1f5f89 tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:31.189696 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-01e5548f-b263-42a2-8a5c-64bd9a1f5f89 tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:31.355992 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-fe7845d3-f6f0-436d-9435-ae11c65f1dcd tempest-QuotasAdminNegativeTest-760882802 tempest-QuotasAdminNegativeTest-760882802-project-admin] 10.222.0.16 "PUT /compute/v2.1/os-quota-sets/29f6db32114b458685269df31fae51c6" status: 200 len: 328 microversion: 2.1 time: 0.328059 Jul 16 21:10:31.356654 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 354/703] 10.222.0.16 () {60 vars in 1261 bytes} [Sat Jul 16 21:10:31 2022] PUT /compute/v2.1/os-quota-sets/29f6db32114b458685269df31fae51c6 => generated 328 bytes in 342 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:10:31.373466 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.objects.instance [None req-01e5548f-b263-42a2-8a5c-64bd9a1f5f89 tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] Lazy-loading 'services' on Instance uuid cf53f7cd-deae-4b34-bf8e-3afef98b2ec6 {{(pid=138681) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:10:31.379887 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-94395d43-6c8a-4a4f-9761-2eab9e1d8396 req-871815bc-3f74-4e7e-aaaf-76e4084b45d0 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "b941b324-a54d-446f-bf92-570a80ba2152", "tag": "dc03a8a7-1bc4-4e8d-91b2-65102ff61630"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:10:31.407185 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-94395d43-6c8a-4a4f-9761-2eab9e1d8396 req-871815bc-3f74-4e7e-aaaf-76e4084b45d0 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:31.411539 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-94395d43-6c8a-4a4f-9761-2eab9e1d8396 req-871815bc-3f74-4e7e-aaaf-76e4084b45d0 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:31.420260 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-01e5548f-b263-42a2-8a5c-64bd9a1f5f89 tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/cf53f7cd-deae-4b34-bf8e-3afef98b2ec6" status: 200 len: 1929 microversion: 2.75 time: 0.251970 Jul 16 21:10:31.422688 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 350/704] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:10:31 2022] GET /compute/v2.1/servers/cf53f7cd-deae-4b34-bf8e-3afef98b2ec6 => generated 1929 bytes in 257 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:10:31.433416 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-94395d43-6c8a-4a4f-9761-2eab9e1d8396 req-871815bc-3f74-4e7e-aaaf-76e4084b45d0 service nova] Creating event network-changed:dc03a8a7-1bc4-4e8d-91b2-65102ff61630 for instance b941b324-a54d-446f-bf92-570a80ba2152 on nch1-master-1 Jul 16 21:10:31.443378 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-94395d43-6c8a-4a4f-9761-2eab9e1d8396 req-871815bc-3f74-4e7e-aaaf-76e4084b45d0 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.072318 Jul 16 21:10:31.445045 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 355/705] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:10:31 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 75 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:10:31.447966 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-37184f36-27c0-4845-b2dc-1918814109b5 req-7a7b07b3-9241-44d5-95e4-d211da1abd04 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "4ee3ffbe-c16a-4847-a343-469c645e4b25", "name": "network-vif-plugged", "status": "completed", "tag": "f54ad45e-ef5d-4b3e-ae39-d46f5d43bc88"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:10:31.462546 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-37184f36-27c0-4845-b2dc-1918814109b5 req-7a7b07b3-9241-44d5-95e4-d211da1abd04 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:31.463240 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-37184f36-27c0-4845-b2dc-1918814109b5 req-7a7b07b3-9241-44d5-95e4-d211da1abd04 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:31.480263 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-37184f36-27c0-4845-b2dc-1918814109b5 req-7a7b07b3-9241-44d5-95e4-d211da1abd04 service nova] Creating event network-vif-plugged:f54ad45e-ef5d-4b3e-ae39-d46f5d43bc88 for instance 4ee3ffbe-c16a-4847-a343-469c645e4b25 on nch1-master-1 Jul 16 21:10:31.487383 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-37184f36-27c0-4845-b2dc-1918814109b5 req-7a7b07b3-9241-44d5-95e4-d211da1abd04 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.047437 Jul 16 21:10:31.489781 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 351/706] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:10:31 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 66 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:10:31.615429 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-e259ed47-7282-4530-9905-9812f66eee25 tempest-QuotasAdminNegativeTest-1085886576 tempest-QuotasAdminNegativeTest-1085886576-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-QuotasAdminNegativeTest-server-162075714", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452"}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:10:31.684573 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.network.neutron [None req-e259ed47-7282-4530-9905-9812f66eee25 tempest-QuotasAdminNegativeTest-1085886576 tempest-QuotasAdminNegativeTest-1085886576-project] validate_networks() for None {{(pid=138680) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 16 21:10:31.882265 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-d3be3e02-9e71-4280-afbe-a562a8bf568a tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:31.883750 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-d3be3e02-9e71-4280-afbe-a562a8bf568a tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: f98a9762-a67c-4e95-85b8-727c81bdb03a] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:31.891734 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d3be3e02-9e71-4280-afbe-a562a8bf568a tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:31.892453 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d3be3e02-9e71-4280-afbe-a562a8bf568a tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:31.893689 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d3be3e02-9e71-4280-afbe-a562a8bf568a tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:31.894517 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d3be3e02-9e71-4280-afbe-a562a8bf568a tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:32.365411 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-d3be3e02-9e71-4280-afbe-a562a8bf568a tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b68dacc4b8463ba07fae6372131117', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55a19b7a4682457a9132f5fc80110712', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:32.381946 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-d3be3e02-9e71-4280-afbe-a562a8bf568a tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "GET /compute/v2.1/servers/f98a9762-a67c-4e95-85b8-727c81bdb03a" status: 200 len: 1476 microversion: 2.1 time: 0.504493 Jul 16 21:10:32.396920 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 352/707] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:10:31 2022] GET /compute/v2.1/servers/f98a9762-a67c-4e95-85b8-727c81bdb03a => generated 1476 bytes in 520 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:10:32.411279 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-f4b86a62-27f5-49c3-bc61-7ec262fe3ea0 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:32.411892 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-f4b86a62-27f5-49c3-bc61-7ec262fe3ea0 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] [instance: 4ee3ffbe-c16a-4847-a343-469c645e4b25] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:32.418554 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f4b86a62-27f5-49c3-bc61-7ec262fe3ea0 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:32.424816 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f4b86a62-27f5-49c3-bc61-7ec262fe3ea0 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:32.427326 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f4b86a62-27f5-49c3-bc61-7ec262fe3ea0 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:32.427949 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f4b86a62-27f5-49c3-bc61-7ec262fe3ea0 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:32.485792 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell0:SELECT=2 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:10:32.608270 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-e259ed47-7282-4530-9905-9812f66eee25 tempest-QuotasAdminNegativeTest-1085886576 tempest-QuotasAdminNegativeTest-1085886576-project] Flavor limits 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 16 21:10:32.610066 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-e259ed47-7282-4530-9905-9812f66eee25 tempest-QuotasAdminNegativeTest-1085886576 tempest-QuotasAdminNegativeTest-1085886576-project] Image limits 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 16 21:10:32.611216 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-e259ed47-7282-4530-9905-9812f66eee25 tempest-QuotasAdminNegativeTest-1085886576 tempest-QuotasAdminNegativeTest-1085886576-project] Flavor pref 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 16 21:10:32.612232 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-e259ed47-7282-4530-9905-9812f66eee25 tempest-QuotasAdminNegativeTest-1085886576 tempest-QuotasAdminNegativeTest-1085886576-project] Image pref 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 16 21:10:32.613180 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-e259ed47-7282-4530-9905-9812f66eee25 tempest-QuotasAdminNegativeTest-1085886576 tempest-QuotasAdminNegativeTest-1085886576-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 16 21:10:32.622388 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell0:SELECT=2 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:10:32.636082 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e259ed47-7282-4530-9905-9812f66eee25 tempest-QuotasAdminNegativeTest-1085886576 tempest-QuotasAdminNegativeTest-1085886576-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:32.640872 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e259ed47-7282-4530-9905-9812f66eee25 tempest-QuotasAdminNegativeTest-1085886576 tempest-QuotasAdminNegativeTest-1085886576-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:32.642025 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e259ed47-7282-4530-9905-9812f66eee25 tempest-QuotasAdminNegativeTest-1085886576 tempest-QuotasAdminNegativeTest-1085886576-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:32.642581 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e259ed47-7282-4530-9905-9812f66eee25 tempest-QuotasAdminNegativeTest-1085886576 tempest-QuotasAdminNegativeTest-1085886576-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:32.674332 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-e259ed47-7282-4530-9905-9812f66eee25 tempest-QuotasAdminNegativeTest-1085886576 tempest-QuotasAdminNegativeTest-1085886576-project] Getting quotas for project 29f6db32114b458685269df31fae51c6. Resources: {'ram', 'instances', 'cores'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:10:32.678243 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-f4b86a62-27f5-49c3-bc61-7ec262fe3ea0 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a69f9dee03d43beaafa966216b9e289', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9b769bf8f3ea4e368bf68de1df6538f9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:32.679960 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-e259ed47-7282-4530-9905-9812f66eee25 tempest-QuotasAdminNegativeTest-1085886576 tempest-QuotasAdminNegativeTest-1085886576-project] Getting quotas for user ca38f16f48494cc4aa71c7f5bd0916a4 and project 29f6db32114b458685269df31fae51c6. Resources: {'ram', 'instances', 'cores'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:10:32.690375 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-f4b86a62-27f5-49c3-bc61-7ec262fe3ea0 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] 10.222.0.16 "GET /compute/v2.1/servers/4ee3ffbe-c16a-4847-a343-469c645e4b25" status: 200 len: 1311 microversion: 2.1 time: 0.282891 Jul 16 21:10:32.690975 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 353/708] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:10:32 2022] GET /compute/v2.1/servers/4ee3ffbe-c16a-4847-a343-469c645e4b25 => generated 1311 bytes in 293 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:10:32.700529 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-b6cf4ca7-710f-486f-9ae1-737c0fc71903 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:32.701141 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-b6cf4ca7-710f-486f-9ae1-737c0fc71903 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] [instance: b941b324-a54d-446f-bf92-570a80ba2152] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:32.708961 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.utils [None req-e259ed47-7282-4530-9905-9812f66eee25 tempest-QuotasAdminNegativeTest-1085886576 tempest-QuotasAdminNegativeTest-1085886576-project] cores quota exceeded for 29f6db32114b458685269df31fae51c6, tried to run 1 instances. Cannot run any more instances of this type. {{(pid=138680) check_num_instances_quota /opt/stack/nova/nova/compute/utils.py:1173}} Jul 16 21:10:32.710444 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b6cf4ca7-710f-486f-9ae1-737c0fc71903 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:32.710649 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-e259ed47-7282-4530-9905-9812f66eee25 tempest-QuotasAdminNegativeTest-1085886576 tempest-QuotasAdminNegativeTest-1085886576-project] HTTP exception thrown: Quota exceeded for cores: Requested 1, but already used 0 of 0 cores Jul 16 21:10:32.711056 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b6cf4ca7-710f-486f-9ae1-737c0fc71903 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:32.711928 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b6cf4ca7-710f-486f-9ae1-737c0fc71903 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:32.712444 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b6cf4ca7-710f-486f-9ae1-737c0fc71903 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:32.715193 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-e259ed47-7282-4530-9905-9812f66eee25 tempest-QuotasAdminNegativeTest-1085886576 tempest-QuotasAdminNegativeTest-1085886576-project] Request method failure captured: Jul 16 21:10:32.715193 ncdv-master-1 devstack@n-api.service[138680]: request: POST /compute/v2.1/servers HTTP/1.1 Jul 16 21:10:32.715193 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:10:32.715193 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:10:32.715193 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:10:32.715193 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 144 Jul 16 21:10:32.715193 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:10:32.715193 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:10:32.715193 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:10:32.715193 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:10:32.715193 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0ylE3FeTCqHPw-nvznMcNwutvdVxrvxhHLMrl2DYHQWWrO_k5katXxSjE94qMCq_CcgcPCVdrJzaOJpXBYMMdRfP4KFW0LwWu9nJ1sGwPCoomWMu72aP9IflG0eb5aKzJ-qUYFJuSDp2eg3ZWjM-CDZZ_ESxIokox1kDJq2YBo2e11E Jul 16 21:10:32.715193 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:10:32.715193 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:10:32.715193 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:10:32.715193 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:10:32.715193 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:10:32.715193 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:10:32.715193 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: 29f6db32114b458685269df31fae51c6 Jul 16 21:10:32.715193 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-QuotasAdminNegativeTest-1085886576 Jul 16 21:10:32.715193 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:10:32.715193 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:10:32.716507 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/29f6db32114b458685269df31fae51c6"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/29f6db32114b458685269df31fae51c6"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_29f6db32114b458685269df31fae51c6"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/29f6db32114b458685269df31fae51c6"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:10:32.716507 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-QuotasAdminNegativeTest-1085886576 Jul 16 21:10:32.716507 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: 29f6db32114b458685269df31fae51c6 Jul 16 21:10:32.716507 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-QuotasAdminNegativeTest-1085886576 Jul 16 21:10:32.716507 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-QuotasAdminNegativeTest-1085886576-project Jul 16 21:10:32.716507 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:10:32.716507 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:10:32.716507 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: ca38f16f48494cc4aa71c7f5bd0916a4 Jul 16 21:10:32.716507 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-QuotasAdminNegativeTest-1085886576-project Jul 16 21:10:32.716507 ncdv-master-1 devstack@n-api.service[138680]: Jul 16 21:10:32.716507 ncdv-master-1 devstack@n-api.service[138680]: {"server": {"name": "tempest-QuotasAdminNegativeTest-server-162075714", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452"}} Jul 16 21:10:32.716507 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 21:10:32.717249 ncdv-master-1 devstack@n-api.service[138680]: exception: Quota exceeded for cores: Requested 1, but already used 0 of 0 cores Jul 16 21:10:32.717249 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'body': {'server': {'name': 'tempest-QuotasAdminNegativeTest-server-162075714', 'imageRef': '1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9', 'flavorRef': '452'}}} Jul 16 21:10:32.717249 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:10:32.717249 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:10:32.717249 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/utils.py", line 1115, in check_num_instances_quota Jul 16 21:10:32.717249 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi objects.Quotas.check_deltas(context, deltas, Jul 16 21:10:32.717249 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Jul 16 21:10:32.717249 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Jul 16 21:10:32.717249 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/quotas.py", line 370, in check_deltas Jul 16 21:10:32.717249 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc Jul 16 21:10:32.717249 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/quotas.py", line 365, in check_deltas Jul 16 21:10:32.717249 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi cls.limit_check_project_and_user(context, **check_kwargs) Jul 16 21:10:32.717249 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Jul 16 21:10:32.717249 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Jul 16 21:10:32.717249 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/quotas.py", line 298, in limit_check_project_and_user Jul 16 21:10:32.717249 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return quota.QUOTAS.limit_check_project_and_user(context, Jul 16 21:10:32.717249 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/quota.py", line 1150, in limit_check_project_and_user Jul 16 21:10:32.717249 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return self._driver.limit_check_project_and_user( Jul 16 21:10:32.718341 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/quota.py", line 588, in limit_check_project_and_user Jul 16 21:10:32.718341 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.OverQuota(overs=sorted(overs), Jul 16 21:10:32.718341 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.OverQuota: Quota exceeded for resources: ['cores'] Jul 16 21:10:32.718341 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:10:32.718341 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:10:32.718341 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:10:32.718341 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:10:32.718341 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 784, in create Jul 16 21:10:32.718341 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instances, resv_id = self.compute_api.create( Jul 16 21:10:32.718341 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2149, in create Jul 16 21:10:32.718341 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return self._create_instance( Jul 16 21:10:32.718341 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 1676, in _create_instance Jul 16 21:10:32.718341 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instances_to_build = self._provision_instances( Jul 16 21:10:32.718341 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 1348, in _provision_instances Jul 16 21:10:32.718341 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi num_instances = compute_utils.check_num_instances_quota( Jul 16 21:10:32.718341 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/utils.py", line 1181, in check_num_instances_quota Jul 16 21:10:32.718341 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.TooManyInstances(overs=overs, Jul 16 21:10:32.718341 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.TooManyInstances: Quota exceeded for cores: Requested 1, but already used 0 of 0 cores Jul 16 21:10:32.718341 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:10:32.718341 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:10:32.718341 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:10:32.719612 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:10:32.719612 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:10:32.719612 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:10:32.719612 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:10:32.719612 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:10:32.719612 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:10:32.719612 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:10:32.719612 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 16 21:10:32.719612 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 16 21:10:32.719612 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 16 21:10:32.719612 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 16 21:10:32.719612 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 16 21:10:32.719612 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 16 21:10:32.719612 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi [Previous line repeated 10 more times] Jul 16 21:10:32.719612 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 800, in create Jul 16 21:10:32.719612 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPForbidden( Jul 16 21:10:32.719612 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPForbidden: Quota exceeded for cores: Requested 1, but already used 0 of 0 cores Jul 16 21:10:32.719612 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:10:32.719612 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:10:32.719612 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:10:32.719612 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:10:32.720903 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:10:32.720903 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:10:32.720903 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:10:32.720903 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:10:32.720903 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Quota exceeded for cores: Requested 1, but already used 0 of 0 cores Jul 16 21:10:32.720903 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:10:32.723227 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-e259ed47-7282-4530-9905-9812f66eee25 tempest-QuotasAdminNegativeTest-1085886576 tempest-QuotasAdminNegativeTest-1085886576-project] Returning 403 to user: Quota exceeded for cores: Requested 1, but already used 0 of 0 cores {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:10:32.724947 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-e259ed47-7282-4530-9905-9812f66eee25 tempest-QuotasAdminNegativeTest-1085886576 tempest-QuotasAdminNegativeTest-1085886576-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 403 len: 111 microversion: 2.1 time: 1.177148 Jul 16 21:10:32.725727 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 356/709] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 21:10:31 2022] POST /compute/v2.1/servers => generated 111 bytes in 1180 msecs (HTTP/1.1 403) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:10:32.735186 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-5efe8eb2-7e56-49bf-896d-20347fc8519a tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:32.736433 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-5efe8eb2-7e56-49bf-896d-20347fc8519a tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] [instance: cf53f7cd-deae-4b34-bf8e-3afef98b2ec6] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:32.746736 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5efe8eb2-7e56-49bf-896d-20347fc8519a tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:32.747467 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5efe8eb2-7e56-49bf-896d-20347fc8519a tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:32.749735 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5efe8eb2-7e56-49bf-896d-20347fc8519a tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:32.750367 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5efe8eb2-7e56-49bf-896d-20347fc8519a tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:32.970260 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.objects.instance [None req-5efe8eb2-7e56-49bf-896d-20347fc8519a tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] Lazy-loading 'services' on Instance uuid cf53f7cd-deae-4b34-bf8e-3afef98b2ec6 {{(pid=138680) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:10:33.014202 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-5efe8eb2-7e56-49bf-896d-20347fc8519a tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/cf53f7cd-deae-4b34-bf8e-3afef98b2ec6" status: 200 len: 1929 microversion: 2.75 time: 0.286944 Jul 16 21:10:33.014964 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 357/710] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:10:32 2022] GET /compute/v2.1/servers/cf53f7cd-deae-4b34-bf8e-3afef98b2ec6 => generated 1929 bytes in 289 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:10:33.029958 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-5a302784-8bb0-4ce5-8365-3154e15bcfc8 tempest-QuotasAdminNegativeTest-760882802 tempest-QuotasAdminNegativeTest-760882802-project-admin] Action: 'update', calling method: .version_select at 0x7fdd9d34b550>, body: {"quota_set": {"cores": 20}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:10:33.062977 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-b6cf4ca7-710f-486f-9ae1-737c0fc71903 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'abeb7020019d41a18de42956d706a011', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '649babe54d104436af461f1ee4767001', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:33.087230 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-b6cf4ca7-710f-486f-9ae1-737c0fc71903 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] 10.222.0.16 "GET /compute/v2.1/servers/b941b324-a54d-446f-bf92-570a80ba2152" status: 200 len: 1317 microversion: 2.1 time: 0.390319 Jul 16 21:10:33.088686 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 354/711] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:10:32 2022] GET /compute/v2.1/servers/b941b324-a54d-446f-bf92-570a80ba2152 => generated 1317 bytes in 397 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:10:33.131120 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5a302784-8bb0-4ce5-8365-3154e15bcfc8 tempest-QuotasAdminNegativeTest-760882802 tempest-QuotasAdminNegativeTest-760882802-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:33.131896 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5a302784-8bb0-4ce5-8365-3154e15bcfc8 tempest-QuotasAdminNegativeTest-760882802 tempest-QuotasAdminNegativeTest-760882802-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:33.132708 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5a302784-8bb0-4ce5-8365-3154e15bcfc8 tempest-QuotasAdminNegativeTest-760882802 tempest-QuotasAdminNegativeTest-760882802-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:33.134548 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5a302784-8bb0-4ce5-8365-3154e15bcfc8 tempest-QuotasAdminNegativeTest-760882802 tempest-QuotasAdminNegativeTest-760882802-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:33.294311 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_api:SELECT=94,nova_api:INSERT=5,nova_api:UPDATE=1 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:10:33.315623 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-5a302784-8bb0-4ce5-8365-3154e15bcfc8 tempest-QuotasAdminNegativeTest-760882802 tempest-QuotasAdminNegativeTest-760882802-project-admin] 10.222.0.16 "PUT /compute/v2.1/os-quota-sets/29f6db32114b458685269df31fae51c6" status: 200 len: 329 microversion: 2.1 time: 0.299369 Jul 16 21:10:33.316438 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 358/712] 10.222.0.16 () {60 vars in 1261 bytes} [Sat Jul 16 21:10:33 2022] PUT /compute/v2.1/os-quota-sets/29f6db32114b458685269df31fae51c6 => generated 329 bytes in 301 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:10:33.331781 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-ec208a27-ea97-4854-b159-d41a794e1696 tempest-QuotasAdminNegativeTest-760882802 tempest-QuotasAdminNegativeTest-760882802-project-admin] Calling method '.version_select at 0x7fdd9d8d3dc0>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:33.413769 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-a5b97e19-8ed0-49ab-9973-4f1798c2b1ad tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:33.414394 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-a5b97e19-8ed0-49ab-9973-4f1798c2b1ad tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: f98a9762-a67c-4e95-85b8-727c81bdb03a] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:33.427655 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a5b97e19-8ed0-49ab-9973-4f1798c2b1ad tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:33.428134 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a5b97e19-8ed0-49ab-9973-4f1798c2b1ad tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:33.428916 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a5b97e19-8ed0-49ab-9973-4f1798c2b1ad tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:33.433937 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a5b97e19-8ed0-49ab-9973-4f1798c2b1ad tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:33.449312 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-ec208a27-ea97-4854-b159-d41a794e1696 tempest-QuotasAdminNegativeTest-760882802 tempest-QuotasAdminNegativeTest-760882802-project-admin] 10.222.0.16 "GET /compute/v2.1/os-quota-sets/29f6db32114b458685269df31fae51c6" status: 200 len: 371 microversion: 2.1 time: 0.123250 Jul 16 21:10:33.449915 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 355/713] 10.222.0.16 () {58 vars in 1241 bytes} [Sat Jul 16 21:10:33 2022] GET /compute/v2.1/os-quota-sets/29f6db32114b458685269df31fae51c6 => generated 371 bytes in 125 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:10:33.465100 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-3852b227-f193-4b2a-a9b8-14194d4b3ef9 tempest-QuotasAdminNegativeTest-760882802 tempest-QuotasAdminNegativeTest-760882802-project-admin] Action: 'update', calling method: .version_select at 0x7fdd9d29e670>, body: {"quota_set": {"force": true, "instances": 0}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:10:33.608731 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3852b227-f193-4b2a-a9b8-14194d4b3ef9 tempest-QuotasAdminNegativeTest-760882802 tempest-QuotasAdminNegativeTest-760882802-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:33.609520 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3852b227-f193-4b2a-a9b8-14194d4b3ef9 tempest-QuotasAdminNegativeTest-760882802 tempest-QuotasAdminNegativeTest-760882802-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:33.611226 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3852b227-f193-4b2a-a9b8-14194d4b3ef9 tempest-QuotasAdminNegativeTest-760882802 tempest-QuotasAdminNegativeTest-760882802-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:33.612016 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3852b227-f193-4b2a-a9b8-14194d4b3ef9 tempest-QuotasAdminNegativeTest-760882802 tempest-QuotasAdminNegativeTest-760882802-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:33.862661 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-3852b227-f193-4b2a-a9b8-14194d4b3ef9 tempest-QuotasAdminNegativeTest-760882802 tempest-QuotasAdminNegativeTest-760882802-project-admin] 10.222.0.16 "PUT /compute/v2.1/os-quota-sets/29f6db32114b458685269df31fae51c6" status: 200 len: 328 microversion: 2.1 time: 0.405794 Jul 16 21:10:33.863363 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 356/714] 10.222.0.16 () {60 vars in 1261 bytes} [Sat Jul 16 21:10:33 2022] PUT /compute/v2.1/os-quota-sets/29f6db32114b458685269df31fae51c6 => generated 328 bytes in 407 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:10:33.879330 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-d7742276-b137-4a14-a416-c55c70f4252a tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:33.880025 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-d7742276-b137-4a14-a416-c55c70f4252a tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] [instance: 4ee3ffbe-c16a-4847-a343-469c645e4b25] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:33.882470 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_api:SELECT=99,nova_api:INSERT=1 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:10:33.888512 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d7742276-b137-4a14-a416-c55c70f4252a tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:33.889835 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d7742276-b137-4a14-a416-c55c70f4252a tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:33.891703 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d7742276-b137-4a14-a416-c55c70f4252a tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:33.892374 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d7742276-b137-4a14-a416-c55c70f4252a tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:34.014237 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-a5b97e19-8ed0-49ab-9973-4f1798c2b1ad tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b68dacc4b8463ba07fae6372131117', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55a19b7a4682457a9132f5fc80110712', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:34.033962 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-a5b97e19-8ed0-49ab-9973-4f1798c2b1ad tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "GET /compute/v2.1/servers/f98a9762-a67c-4e95-85b8-727c81bdb03a" status: 200 len: 1476 microversion: 2.1 time: 0.623999 Jul 16 21:10:34.041027 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 359/715] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:10:33 2022] GET /compute/v2.1/servers/f98a9762-a67c-4e95-85b8-727c81bdb03a => generated 1476 bytes in 631 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:10:34.052517 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-d23acb4b-c7a9-4f2b-99e1-74587840b103 tempest-QuotasAdminNegativeTest-1085886576 tempest-QuotasAdminNegativeTest-1085886576-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-QuotasAdminNegativeTest-server-2033902712", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452"}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:10:34.126454 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.network.neutron [None req-d23acb4b-c7a9-4f2b-99e1-74587840b103 tempest-QuotasAdminNegativeTest-1085886576 tempest-QuotasAdminNegativeTest-1085886576-project] validate_networks() for None {{(pid=138680) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 16 21:10:34.532511 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-d7742276-b137-4a14-a416-c55c70f4252a tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a69f9dee03d43beaafa966216b9e289', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9b769bf8f3ea4e368bf68de1df6538f9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:34.556272 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-d7742276-b137-4a14-a416-c55c70f4252a tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] 10.222.0.16 "GET /compute/v2.1/servers/4ee3ffbe-c16a-4847-a343-469c645e4b25" status: 200 len: 1311 microversion: 2.1 time: 0.687464 Jul 16 21:10:34.557194 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 357/716] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:10:33 2022] GET /compute/v2.1/servers/4ee3ffbe-c16a-4847-a343-469c645e4b25 => generated 1311 bytes in 693 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:10:34.563081 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-2b07b3db-f65b-42c0-b0ae-1dd77e41c2af tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:34.564224 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-2b07b3db-f65b-42c0-b0ae-1dd77e41c2af tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] [instance: cf53f7cd-deae-4b34-bf8e-3afef98b2ec6] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:34.576301 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2b07b3db-f65b-42c0-b0ae-1dd77e41c2af tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:34.576856 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2b07b3db-f65b-42c0-b0ae-1dd77e41c2af tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:34.577657 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2b07b3db-f65b-42c0-b0ae-1dd77e41c2af tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:34.578099 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2b07b3db-f65b-42c0-b0ae-1dd77e41c2af tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:34.729945 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.objects.instance [None req-2b07b3db-f65b-42c0-b0ae-1dd77e41c2af tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] Lazy-loading 'services' on Instance uuid cf53f7cd-deae-4b34-bf8e-3afef98b2ec6 {{(pid=138681) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:10:34.760520 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-2b07b3db-f65b-42c0-b0ae-1dd77e41c2af tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/cf53f7cd-deae-4b34-bf8e-3afef98b2ec6" status: 200 len: 1929 microversion: 2.75 time: 0.201957 Jul 16 21:10:34.761360 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 358/717] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:10:34 2022] GET /compute/v2.1/servers/cf53f7cd-deae-4b34-bf8e-3afef98b2ec6 => generated 1929 bytes in 204 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:10:34.766251 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-fe60632b-76f7-4b44-bd61-bebab0b453f8 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:34.767446 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-fe60632b-76f7-4b44-bd61-bebab0b453f8 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] [instance: b941b324-a54d-446f-bf92-570a80ba2152] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:34.775796 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fe60632b-76f7-4b44-bd61-bebab0b453f8 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:34.782822 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fe60632b-76f7-4b44-bd61-bebab0b453f8 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:34.782822 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fe60632b-76f7-4b44-bd61-bebab0b453f8 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:34.782822 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fe60632b-76f7-4b44-bd61-bebab0b453f8 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:34.973570 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-d23acb4b-c7a9-4f2b-99e1-74587840b103 tempest-QuotasAdminNegativeTest-1085886576 tempest-QuotasAdminNegativeTest-1085886576-project] Flavor limits 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 16 21:10:34.973994 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-d23acb4b-c7a9-4f2b-99e1-74587840b103 tempest-QuotasAdminNegativeTest-1085886576 tempest-QuotasAdminNegativeTest-1085886576-project] Image limits 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 16 21:10:34.974466 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-d23acb4b-c7a9-4f2b-99e1-74587840b103 tempest-QuotasAdminNegativeTest-1085886576 tempest-QuotasAdminNegativeTest-1085886576-project] Flavor pref 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 16 21:10:34.974875 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-d23acb4b-c7a9-4f2b-99e1-74587840b103 tempest-QuotasAdminNegativeTest-1085886576 tempest-QuotasAdminNegativeTest-1085886576-project] Image pref 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 16 21:10:34.975338 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-d23acb4b-c7a9-4f2b-99e1-74587840b103 tempest-QuotasAdminNegativeTest-1085886576 tempest-QuotasAdminNegativeTest-1085886576-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 16 21:10:34.987716 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d23acb4b-c7a9-4f2b-99e1-74587840b103 tempest-QuotasAdminNegativeTest-1085886576 tempest-QuotasAdminNegativeTest-1085886576-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:34.988486 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d23acb4b-c7a9-4f2b-99e1-74587840b103 tempest-QuotasAdminNegativeTest-1085886576 tempest-QuotasAdminNegativeTest-1085886576-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:34.990097 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d23acb4b-c7a9-4f2b-99e1-74587840b103 tempest-QuotasAdminNegativeTest-1085886576 tempest-QuotasAdminNegativeTest-1085886576-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:34.990747 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d23acb4b-c7a9-4f2b-99e1-74587840b103 tempest-QuotasAdminNegativeTest-1085886576 tempest-QuotasAdminNegativeTest-1085886576-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:35.015069 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-d23acb4b-c7a9-4f2b-99e1-74587840b103 tempest-QuotasAdminNegativeTest-1085886576 tempest-QuotasAdminNegativeTest-1085886576-project] Getting quotas for project 29f6db32114b458685269df31fae51c6. Resources: {'ram', 'instances', 'cores'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:10:35.022882 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-d23acb4b-c7a9-4f2b-99e1-74587840b103 tempest-QuotasAdminNegativeTest-1085886576 tempest-QuotasAdminNegativeTest-1085886576-project] Getting quotas for user ca38f16f48494cc4aa71c7f5bd0916a4 and project 29f6db32114b458685269df31fae51c6. Resources: {'ram', 'instances', 'cores'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:10:35.038103 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.utils [None req-d23acb4b-c7a9-4f2b-99e1-74587840b103 tempest-QuotasAdminNegativeTest-1085886576 tempest-QuotasAdminNegativeTest-1085886576-project] instances quota exceeded for 29f6db32114b458685269df31fae51c6, tried to run 1 instances. Cannot run any more instances of this type. {{(pid=138680) check_num_instances_quota /opt/stack/nova/nova/compute/utils.py:1173}} Jul 16 21:10:35.038697 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-d23acb4b-c7a9-4f2b-99e1-74587840b103 tempest-QuotasAdminNegativeTest-1085886576 tempest-QuotasAdminNegativeTest-1085886576-project] HTTP exception thrown: Quota exceeded for instances: Requested 1, but already used 0 of 0 instances Jul 16 21:10:35.041959 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-d23acb4b-c7a9-4f2b-99e1-74587840b103 tempest-QuotasAdminNegativeTest-1085886576 tempest-QuotasAdminNegativeTest-1085886576-project] Request method failure captured: Jul 16 21:10:35.041959 ncdv-master-1 devstack@n-api.service[138680]: request: POST /compute/v2.1/servers HTTP/1.1 Jul 16 21:10:35.041959 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:10:35.041959 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:10:35.041959 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:10:35.041959 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 145 Jul 16 21:10:35.041959 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:10:35.041959 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:10:35.041959 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:10:35.041959 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:10:35.041959 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0ylE3FeTCqHPw-nvznMcNwutvdVxrvxhHLMrl2DYHQWWrO_k5katXxSjE94qMCq_CcgcPCVdrJzaOJpXBYMMdRfP4KFW0LwWu9nJ1sGwPCoomWMu72aP9IflG0eb5aKzJ-qUYFJuSDp2eg3ZWjM-CDZZ_ESxIokox1kDJq2YBo2e11E Jul 16 21:10:35.041959 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:10:35.041959 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:10:35.041959 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:10:35.041959 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:10:35.041959 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:10:35.041959 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:10:35.041959 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: 29f6db32114b458685269df31fae51c6 Jul 16 21:10:35.041959 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-QuotasAdminNegativeTest-1085886576 Jul 16 21:10:35.041959 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:10:35.041959 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:10:35.043384 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/29f6db32114b458685269df31fae51c6"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/29f6db32114b458685269df31fae51c6"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_29f6db32114b458685269df31fae51c6"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/29f6db32114b458685269df31fae51c6"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:10:35.043384 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-QuotasAdminNegativeTest-1085886576 Jul 16 21:10:35.043384 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: 29f6db32114b458685269df31fae51c6 Jul 16 21:10:35.043384 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-QuotasAdminNegativeTest-1085886576 Jul 16 21:10:35.043384 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-QuotasAdminNegativeTest-1085886576-project Jul 16 21:10:35.043384 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:10:35.043384 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:10:35.043384 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: ca38f16f48494cc4aa71c7f5bd0916a4 Jul 16 21:10:35.043384 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-QuotasAdminNegativeTest-1085886576-project Jul 16 21:10:35.043384 ncdv-master-1 devstack@n-api.service[138680]: Jul 16 21:10:35.043384 ncdv-master-1 devstack@n-api.service[138680]: {"server": {"name": "tempest-QuotasAdminNegativeTest-server-2033902712", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452"}} Jul 16 21:10:35.043384 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 21:10:35.044137 ncdv-master-1 devstack@n-api.service[138680]: exception: Quota exceeded for instances: Requested 1, but already used 0 of 0 instances Jul 16 21:10:35.044137 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'body': {'server': {'name': 'tempest-QuotasAdminNegativeTest-server-2033902712', 'imageRef': '1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9', 'flavorRef': '452'}}} Jul 16 21:10:35.044137 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:10:35.044137 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:10:35.044137 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/utils.py", line 1115, in check_num_instances_quota Jul 16 21:10:35.044137 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi objects.Quotas.check_deltas(context, deltas, Jul 16 21:10:35.044137 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Jul 16 21:10:35.044137 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Jul 16 21:10:35.044137 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/quotas.py", line 370, in check_deltas Jul 16 21:10:35.044137 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc Jul 16 21:10:35.044137 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/quotas.py", line 365, in check_deltas Jul 16 21:10:35.044137 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi cls.limit_check_project_and_user(context, **check_kwargs) Jul 16 21:10:35.044137 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Jul 16 21:10:35.044137 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Jul 16 21:10:35.044137 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/quotas.py", line 298, in limit_check_project_and_user Jul 16 21:10:35.044137 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return quota.QUOTAS.limit_check_project_and_user(context, Jul 16 21:10:35.044137 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/quota.py", line 1150, in limit_check_project_and_user Jul 16 21:10:35.044137 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return self._driver.limit_check_project_and_user( Jul 16 21:10:35.045300 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/quota.py", line 588, in limit_check_project_and_user Jul 16 21:10:35.045300 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.OverQuota(overs=sorted(overs), Jul 16 21:10:35.045300 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.OverQuota: Quota exceeded for resources: ['instances'] Jul 16 21:10:35.045300 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:10:35.045300 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:10:35.045300 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:10:35.045300 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:10:35.045300 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 784, in create Jul 16 21:10:35.045300 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instances, resv_id = self.compute_api.create( Jul 16 21:10:35.045300 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2149, in create Jul 16 21:10:35.045300 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return self._create_instance( Jul 16 21:10:35.045300 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 1676, in _create_instance Jul 16 21:10:35.045300 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instances_to_build = self._provision_instances( Jul 16 21:10:35.045300 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 1348, in _provision_instances Jul 16 21:10:35.045300 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi num_instances = compute_utils.check_num_instances_quota( Jul 16 21:10:35.045300 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/utils.py", line 1181, in check_num_instances_quota Jul 16 21:10:35.045300 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.TooManyInstances(overs=overs, Jul 16 21:10:35.045300 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.TooManyInstances: Quota exceeded for instances: Requested 1, but already used 0 of 0 instances Jul 16 21:10:35.045300 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:10:35.045300 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:10:35.045300 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:10:35.057360 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:10:35.057360 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:10:35.057360 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:10:35.057360 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:10:35.057360 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:10:35.057360 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:10:35.057360 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:10:35.057360 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 16 21:10:35.057360 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 16 21:10:35.057360 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 16 21:10:35.057360 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 16 21:10:35.057360 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 16 21:10:35.057360 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 16 21:10:35.057360 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi [Previous line repeated 10 more times] Jul 16 21:10:35.057360 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 800, in create Jul 16 21:10:35.057360 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPForbidden( Jul 16 21:10:35.057360 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPForbidden: Quota exceeded for instances: Requested 1, but already used 0 of 0 instances Jul 16 21:10:35.057360 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:10:35.057360 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:10:35.057360 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:10:35.057360 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:10:35.061692 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:10:35.061692 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:10:35.061692 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:10:35.061692 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:10:35.061692 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Quota exceeded for instances: Requested 1, but already used 0 of 0 instances Jul 16 21:10:35.061692 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:10:35.061692 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-d23acb4b-c7a9-4f2b-99e1-74587840b103 tempest-QuotasAdminNegativeTest-1085886576 tempest-QuotasAdminNegativeTest-1085886576-project] Returning 403 to user: Quota exceeded for instances: Requested 1, but already used 0 of 0 instances {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:10:35.061692 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-d23acb4b-c7a9-4f2b-99e1-74587840b103 tempest-QuotasAdminNegativeTest-1085886576 tempest-QuotasAdminNegativeTest-1085886576-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 403 len: 119 microversion: 2.1 time: 0.999882 Jul 16 21:10:35.061692 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 360/718] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 21:10:34 2022] POST /compute/v2.1/servers => generated 119 bytes in 1002 msecs (HTTP/1.1 403) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:10:35.061692 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-1d9a6f88-02c3-48ec-a7a5-f7f7851a2f0a tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:35.067648 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-1d9a6f88-02c3-48ec-a7a5-f7f7851a2f0a tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: f98a9762-a67c-4e95-85b8-727c81bdb03a] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:35.073973 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1d9a6f88-02c3-48ec-a7a5-f7f7851a2f0a tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:35.074426 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1d9a6f88-02c3-48ec-a7a5-f7f7851a2f0a tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:35.080806 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1d9a6f88-02c3-48ec-a7a5-f7f7851a2f0a tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:35.081525 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1d9a6f88-02c3-48ec-a7a5-f7f7851a2f0a tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:35.115106 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-fe60632b-76f7-4b44-bd61-bebab0b453f8 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'abeb7020019d41a18de42956d706a011', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '649babe54d104436af461f1ee4767001', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:35.129245 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-fe60632b-76f7-4b44-bd61-bebab0b453f8 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] 10.222.0.16 "GET /compute/v2.1/servers/b941b324-a54d-446f-bf92-570a80ba2152" status: 200 len: 1317 microversion: 2.1 time: 0.366524 Jul 16 21:10:35.129878 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 359/719] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:10:34 2022] GET /compute/v2.1/servers/b941b324-a54d-446f-bf92-570a80ba2152 => generated 1317 bytes in 368 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:10:35.141709 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-01b34949-19d9-4f0b-b55e-c0d01ec7bf85 tempest-QuotasAdminNegativeTest-760882802 tempest-QuotasAdminNegativeTest-760882802-project-admin] Action: 'update', calling method: .version_select at 0x7fdd9d8c5e50>, body: {"quota_set": {"instances": 10}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:10:35.220120 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-01b34949-19d9-4f0b-b55e-c0d01ec7bf85 tempest-QuotasAdminNegativeTest-760882802 tempest-QuotasAdminNegativeTest-760882802-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:35.220807 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-01b34949-19d9-4f0b-b55e-c0d01ec7bf85 tempest-QuotasAdminNegativeTest-760882802 tempest-QuotasAdminNegativeTest-760882802-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:35.221838 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-01b34949-19d9-4f0b-b55e-c0d01ec7bf85 tempest-QuotasAdminNegativeTest-760882802 tempest-QuotasAdminNegativeTest-760882802-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:35.222643 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-01b34949-19d9-4f0b-b55e-c0d01ec7bf85 tempest-QuotasAdminNegativeTest-760882802 tempest-QuotasAdminNegativeTest-760882802-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:35.362676 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-1d9a6f88-02c3-48ec-a7a5-f7f7851a2f0a tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b68dacc4b8463ba07fae6372131117', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55a19b7a4682457a9132f5fc80110712', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:35.374954 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-1d9a6f88-02c3-48ec-a7a5-f7f7851a2f0a tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "GET /compute/v2.1/servers/f98a9762-a67c-4e95-85b8-727c81bdb03a" status: 200 len: 1476 microversion: 2.1 time: 0.325526 Jul 16 21:10:35.375816 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 361/720] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:10:35 2022] GET /compute/v2.1/servers/f98a9762-a67c-4e95-85b8-727c81bdb03a => generated 1476 bytes in 327 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:10:35.531373 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-01b34949-19d9-4f0b-b55e-c0d01ec7bf85 tempest-QuotasAdminNegativeTest-760882802 tempest-QuotasAdminNegativeTest-760882802-project-admin] 10.222.0.16 "PUT /compute/v2.1/os-quota-sets/29f6db32114b458685269df31fae51c6" status: 200 len: 329 microversion: 2.1 time: 0.397346 Jul 16 21:10:35.531373 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 360/721] 10.222.0.16 () {60 vars in 1261 bytes} [Sat Jul 16 21:10:35 2022] PUT /compute/v2.1/os-quota-sets/29f6db32114b458685269df31fae51c6 => generated 329 bytes in 401 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:10:35.574437 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-132ecc81-b6c0-42b4-b8f4-edfd294fdf15 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:35.575397 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-132ecc81-b6c0-42b4-b8f4-edfd294fdf15 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] [instance: 4ee3ffbe-c16a-4847-a343-469c645e4b25] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:35.581927 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-11849159-6e14-496d-8e94-0c17e7f917de tempest-QuotasAdminNegativeTest-760882802 tempest-QuotasAdminNegativeTest-760882802-project-admin] Calling method '.version_select at 0x7fdd9d4bd790>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:35.589726 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-132ecc81-b6c0-42b4-b8f4-edfd294fdf15 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:35.602956 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-132ecc81-b6c0-42b4-b8f4-edfd294fdf15 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:35.607685 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-132ecc81-b6c0-42b4-b8f4-edfd294fdf15 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:35.609633 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-132ecc81-b6c0-42b4-b8f4-edfd294fdf15 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:35.691552 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=100 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:10:35.756381 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-11849159-6e14-496d-8e94-0c17e7f917de tempest-QuotasAdminNegativeTest-760882802 tempest-QuotasAdminNegativeTest-760882802-project-admin] 10.222.0.16 "GET /compute/v2.1/os-quota-sets/29f6db32114b458685269df31fae51c6" status: 200 len: 371 microversion: 2.1 time: 0.192739 Jul 16 21:10:35.762383 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 362/722] 10.222.0.16 () {58 vars in 1241 bytes} [Sat Jul 16 21:10:35 2022] GET /compute/v2.1/os-quota-sets/29f6db32114b458685269df31fae51c6 => generated 371 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:10:35.784705 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-35d20e8d-e6db-4301-90a7-b27ecacc3813 tempest-QuotasAdminNegativeTest-760882802 tempest-QuotasAdminNegativeTest-760882802-project-admin] Action: 'update', calling method: .version_select at 0x7fdd9d2678b0>, body: {"quota_set": {"force": true, "ram": 0}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:10:35.892516 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-35d20e8d-e6db-4301-90a7-b27ecacc3813 tempest-QuotasAdminNegativeTest-760882802 tempest-QuotasAdminNegativeTest-760882802-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:35.894070 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-35d20e8d-e6db-4301-90a7-b27ecacc3813 tempest-QuotasAdminNegativeTest-760882802 tempest-QuotasAdminNegativeTest-760882802-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:35.895749 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-35d20e8d-e6db-4301-90a7-b27ecacc3813 tempest-QuotasAdminNegativeTest-760882802 tempest-QuotasAdminNegativeTest-760882802-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:35.896582 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-35d20e8d-e6db-4301-90a7-b27ecacc3813 tempest-QuotasAdminNegativeTest-760882802 tempest-QuotasAdminNegativeTest-760882802-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:35.996271 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-132ecc81-b6c0-42b4-b8f4-edfd294fdf15 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a69f9dee03d43beaafa966216b9e289', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9b769bf8f3ea4e368bf68de1df6538f9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:36.005403 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-132ecc81-b6c0-42b4-b8f4-edfd294fdf15 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] 10.222.0.16 "GET /compute/v2.1/servers/4ee3ffbe-c16a-4847-a343-469c645e4b25" status: 200 len: 1494 microversion: 2.1 time: 0.435523 Jul 16 21:10:36.009832 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 361/723] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:10:35 2022] GET /compute/v2.1/servers/4ee3ffbe-c16a-4847-a343-469c645e4b25 => generated 1494 bytes in 441 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:10:36.021590 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-83f7d241-4634-4a44-81d9-4fa5b69c42c0 tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:36.026788 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-83f7d241-4634-4a44-81d9-4fa5b69c42c0 tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] [instance: cf53f7cd-deae-4b34-bf8e-3afef98b2ec6] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:36.033767 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-83f7d241-4634-4a44-81d9-4fa5b69c42c0 tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:36.035552 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-83f7d241-4634-4a44-81d9-4fa5b69c42c0 tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:36.041322 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-83f7d241-4634-4a44-81d9-4fa5b69c42c0 tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:36.041975 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-83f7d241-4634-4a44-81d9-4fa5b69c42c0 tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:36.097937 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-35d20e8d-e6db-4301-90a7-b27ecacc3813 tempest-QuotasAdminNegativeTest-760882802 tempest-QuotasAdminNegativeTest-760882802-project-admin] 10.222.0.16 "PUT /compute/v2.1/os-quota-sets/29f6db32114b458685269df31fae51c6" status: 200 len: 325 microversion: 2.1 time: 0.331825 Jul 16 21:10:36.111839 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 363/724] 10.222.0.16 () {60 vars in 1261 bytes} [Sat Jul 16 21:10:35 2022] PUT /compute/v2.1/os-quota-sets/29f6db32114b458685269df31fae51c6 => generated 325 bytes in 349 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:10:36.117667 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-826ed688-35fe-4873-a8b4-446a23f70946 tempest-LiveMigrationTest-1131758637 tempest-LiveMigrationTest-1131758637-project-admin] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:36.118372 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-826ed688-35fe-4873-a8b4-446a23f70946 tempest-LiveMigrationTest-1131758637 tempest-LiveMigrationTest-1131758637-project-admin] [instance: 4ee3ffbe-c16a-4847-a343-469c645e4b25] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:36.128087 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-826ed688-35fe-4873-a8b4-446a23f70946 tempest-LiveMigrationTest-1131758637 tempest-LiveMigrationTest-1131758637-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:36.128087 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-826ed688-35fe-4873-a8b4-446a23f70946 tempest-LiveMigrationTest-1131758637 tempest-LiveMigrationTest-1131758637-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:36.128827 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-826ed688-35fe-4873-a8b4-446a23f70946 tempest-LiveMigrationTest-1131758637 tempest-LiveMigrationTest-1131758637-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:36.129653 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-826ed688-35fe-4873-a8b4-446a23f70946 tempest-LiveMigrationTest-1131758637 tempest-LiveMigrationTest-1131758637-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:36.134515 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.objects.instance [None req-83f7d241-4634-4a44-81d9-4fa5b69c42c0 tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] Lazy-loading 'services' on Instance uuid cf53f7cd-deae-4b34-bf8e-3afef98b2ec6 {{(pid=138681) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:10:36.180850 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-83f7d241-4634-4a44-81d9-4fa5b69c42c0 tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/cf53f7cd-deae-4b34-bf8e-3afef98b2ec6" status: 200 len: 1929 microversion: 2.75 time: 0.169470 Jul 16 21:10:36.181465 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 362/725] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:10:36 2022] GET /compute/v2.1/servers/cf53f7cd-deae-4b34-bf8e-3afef98b2ec6 => generated 1929 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:10:36.186763 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-f165dc0e-9022-4b28-8516-1b32f6e8e6ce tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:36.187664 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-f165dc0e-9022-4b28-8516-1b32f6e8e6ce tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] [instance: b941b324-a54d-446f-bf92-570a80ba2152] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:36.205885 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f165dc0e-9022-4b28-8516-1b32f6e8e6ce tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:36.206319 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f165dc0e-9022-4b28-8516-1b32f6e8e6ce tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:36.207241 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f165dc0e-9022-4b28-8516-1b32f6e8e6ce tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:36.207865 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f165dc0e-9022-4b28-8516-1b32f6e8e6ce tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:36.862989 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-826ed688-35fe-4873-a8b4-446a23f70946 tempest-LiveMigrationTest-1131758637 tempest-LiveMigrationTest-1131758637-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/4ee3ffbe-c16a-4847-a343-469c645e4b25" status: 200 len: 1645 microversion: 2.1 time: 0.749573 Jul 16 21:10:36.867980 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 364/726] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:10:36 2022] GET /compute/v2.1/servers/4ee3ffbe-c16a-4847-a343-469c645e4b25 => generated 1645 bytes in 756 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:10:36.871229 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-f165dc0e-9022-4b28-8516-1b32f6e8e6ce tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'abeb7020019d41a18de42956d706a011', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '649babe54d104436af461f1ee4767001', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:36.876419 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-3b279d53-0dd8-4c72-a38e-2a49432c4dd1 tempest-QuotasAdminNegativeTest-1085886576 tempest-QuotasAdminNegativeTest-1085886576-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-QuotasAdminNegativeTest-server-1141800914", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452"}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:10:36.883863 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-f165dc0e-9022-4b28-8516-1b32f6e8e6ce tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] 10.222.0.16 "GET /compute/v2.1/servers/b941b324-a54d-446f-bf92-570a80ba2152" status: 200 len: 1317 microversion: 2.1 time: 0.701302 Jul 16 21:10:36.885062 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 363/727] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:10:36 2022] GET /compute/v2.1/servers/b941b324-a54d-446f-bf92-570a80ba2152 => generated 1317 bytes in 703 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:10:36.890705 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-c29aa5f4-83d7-4b8b-8df5-233d5ee4e14d tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:36.891170 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-c29aa5f4-83d7-4b8b-8df5-233d5ee4e14d tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: f98a9762-a67c-4e95-85b8-727c81bdb03a] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:36.902337 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c29aa5f4-83d7-4b8b-8df5-233d5ee4e14d tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:36.902872 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c29aa5f4-83d7-4b8b-8df5-233d5ee4e14d tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:36.903950 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c29aa5f4-83d7-4b8b-8df5-233d5ee4e14d tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:36.904463 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c29aa5f4-83d7-4b8b-8df5-233d5ee4e14d tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:36.953191 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.network.neutron [None req-3b279d53-0dd8-4c72-a38e-2a49432c4dd1 tempest-QuotasAdminNegativeTest-1085886576 tempest-QuotasAdminNegativeTest-1085886576-project] validate_networks() for None {{(pid=138680) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 16 21:10:37.303962 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-c29aa5f4-83d7-4b8b-8df5-233d5ee4e14d tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b68dacc4b8463ba07fae6372131117', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55a19b7a4682457a9132f5fc80110712', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:37.320077 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-c29aa5f4-83d7-4b8b-8df5-233d5ee4e14d tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "GET /compute/v2.1/servers/f98a9762-a67c-4e95-85b8-727c81bdb03a" status: 200 len: 1476 microversion: 2.1 time: 0.433290 Jul 16 21:10:37.331359 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 364/728] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:10:36 2022] GET /compute/v2.1/servers/f98a9762-a67c-4e95-85b8-727c81bdb03a => generated 1476 bytes in 446 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:10:37.344918 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-208c5437-f4eb-4e8d-910e-78e6aee32137 tempest-LiveMigrationTest-1131758637 tempest-LiveMigrationTest-1131758637-project-admin] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:37.347662 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-208c5437-f4eb-4e8d-910e-78e6aee32137 tempest-LiveMigrationTest-1131758637 tempest-LiveMigrationTest-1131758637-project-admin] [instance: 4ee3ffbe-c16a-4847-a343-469c645e4b25] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:37.372408 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-208c5437-f4eb-4e8d-910e-78e6aee32137 tempest-LiveMigrationTest-1131758637 tempest-LiveMigrationTest-1131758637-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:37.374161 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-208c5437-f4eb-4e8d-910e-78e6aee32137 tempest-LiveMigrationTest-1131758637 tempest-LiveMigrationTest-1131758637-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:37.375659 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-208c5437-f4eb-4e8d-910e-78e6aee32137 tempest-LiveMigrationTest-1131758637 tempest-LiveMigrationTest-1131758637-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:37.376325 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-208c5437-f4eb-4e8d-910e-78e6aee32137 tempest-LiveMigrationTest-1131758637 tempest-LiveMigrationTest-1131758637-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:37.448148 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-3b279d53-0dd8-4c72-a38e-2a49432c4dd1 tempest-QuotasAdminNegativeTest-1085886576 tempest-QuotasAdminNegativeTest-1085886576-project] Flavor limits 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 16 21:10:37.448621 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-3b279d53-0dd8-4c72-a38e-2a49432c4dd1 tempest-QuotasAdminNegativeTest-1085886576 tempest-QuotasAdminNegativeTest-1085886576-project] Image limits 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 16 21:10:37.449046 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-3b279d53-0dd8-4c72-a38e-2a49432c4dd1 tempest-QuotasAdminNegativeTest-1085886576 tempest-QuotasAdminNegativeTest-1085886576-project] Flavor pref 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 16 21:10:37.449466 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-3b279d53-0dd8-4c72-a38e-2a49432c4dd1 tempest-QuotasAdminNegativeTest-1085886576 tempest-QuotasAdminNegativeTest-1085886576-project] Image pref 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 16 21:10:37.449903 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-3b279d53-0dd8-4c72-a38e-2a49432c4dd1 tempest-QuotasAdminNegativeTest-1085886576 tempest-QuotasAdminNegativeTest-1085886576-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 16 21:10:37.481629 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3b279d53-0dd8-4c72-a38e-2a49432c4dd1 tempest-QuotasAdminNegativeTest-1085886576 tempest-QuotasAdminNegativeTest-1085886576-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:37.482343 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3b279d53-0dd8-4c72-a38e-2a49432c4dd1 tempest-QuotasAdminNegativeTest-1085886576 tempest-QuotasAdminNegativeTest-1085886576-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:37.484003 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3b279d53-0dd8-4c72-a38e-2a49432c4dd1 tempest-QuotasAdminNegativeTest-1085886576 tempest-QuotasAdminNegativeTest-1085886576-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:37.486487 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3b279d53-0dd8-4c72-a38e-2a49432c4dd1 tempest-QuotasAdminNegativeTest-1085886576 tempest-QuotasAdminNegativeTest-1085886576-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:37.532724 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-3b279d53-0dd8-4c72-a38e-2a49432c4dd1 tempest-QuotasAdminNegativeTest-1085886576 tempest-QuotasAdminNegativeTest-1085886576-project] Getting quotas for project 29f6db32114b458685269df31fae51c6. Resources: {'ram', 'instances', 'cores'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:10:37.567487 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-3b279d53-0dd8-4c72-a38e-2a49432c4dd1 tempest-QuotasAdminNegativeTest-1085886576 tempest-QuotasAdminNegativeTest-1085886576-project] Getting quotas for user ca38f16f48494cc4aa71c7f5bd0916a4 and project 29f6db32114b458685269df31fae51c6. Resources: {'ram', 'instances', 'cores'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:10:37.604038 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.utils [None req-3b279d53-0dd8-4c72-a38e-2a49432c4dd1 tempest-QuotasAdminNegativeTest-1085886576 tempest-QuotasAdminNegativeTest-1085886576-project] ram quota exceeded for 29f6db32114b458685269df31fae51c6, tried to run 1 instances. Cannot run any more instances of this type. {{(pid=138680) check_num_instances_quota /opt/stack/nova/nova/compute/utils.py:1173}} Jul 16 21:10:37.604595 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-3b279d53-0dd8-4c72-a38e-2a49432c4dd1 tempest-QuotasAdminNegativeTest-1085886576 tempest-QuotasAdminNegativeTest-1085886576-project] HTTP exception thrown: Quota exceeded for ram: Requested 1024, but already used 0 of 0 ram Jul 16 21:10:37.622389 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-3b279d53-0dd8-4c72-a38e-2a49432c4dd1 tempest-QuotasAdminNegativeTest-1085886576 tempest-QuotasAdminNegativeTest-1085886576-project] Request method failure captured: Jul 16 21:10:37.622389 ncdv-master-1 devstack@n-api.service[138680]: request: POST /compute/v2.1/servers HTTP/1.1 Jul 16 21:10:37.622389 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:10:37.622389 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:10:37.622389 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:10:37.622389 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 145 Jul 16 21:10:37.622389 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:10:37.622389 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:10:37.622389 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:10:37.622389 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:10:37.622389 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0ylE3FeTCqHPw-nvznMcNwutvdVxrvxhHLMrl2DYHQWWrO_k5katXxSjE94qMCq_CcgcPCVdrJzaOJpXBYMMdRfP4KFW0LwWu9nJ1sGwPCoomWMu72aP9IflG0eb5aKzJ-qUYFJuSDp2eg3ZWjM-CDZZ_ESxIokox1kDJq2YBo2e11E Jul 16 21:10:37.622389 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:10:37.622389 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:10:37.622389 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:10:37.622389 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:10:37.622389 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:10:37.622389 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:10:37.622389 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: 29f6db32114b458685269df31fae51c6 Jul 16 21:10:37.622389 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-QuotasAdminNegativeTest-1085886576 Jul 16 21:10:37.622389 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:10:37.622389 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:10:37.624457 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/29f6db32114b458685269df31fae51c6"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/29f6db32114b458685269df31fae51c6"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_29f6db32114b458685269df31fae51c6"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/29f6db32114b458685269df31fae51c6"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:10:37.624457 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-QuotasAdminNegativeTest-1085886576 Jul 16 21:10:37.624457 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: 29f6db32114b458685269df31fae51c6 Jul 16 21:10:37.624457 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-QuotasAdminNegativeTest-1085886576 Jul 16 21:10:37.624457 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-QuotasAdminNegativeTest-1085886576-project Jul 16 21:10:37.624457 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:10:37.624457 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:10:37.624457 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: ca38f16f48494cc4aa71c7f5bd0916a4 Jul 16 21:10:37.624457 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-QuotasAdminNegativeTest-1085886576-project Jul 16 21:10:37.624457 ncdv-master-1 devstack@n-api.service[138680]: Jul 16 21:10:37.624457 ncdv-master-1 devstack@n-api.service[138680]: {"server": {"name": "tempest-QuotasAdminNegativeTest-server-1141800914", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452"}} Jul 16 21:10:37.624457 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 21:10:37.625153 ncdv-master-1 devstack@n-api.service[138680]: exception: Quota exceeded for ram: Requested 1024, but already used 0 of 0 ram Jul 16 21:10:37.625153 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'body': {'server': {'name': 'tempest-QuotasAdminNegativeTest-server-1141800914', 'imageRef': '1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9', 'flavorRef': '452'}}} Jul 16 21:10:37.625153 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:10:37.625153 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:10:37.625153 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/utils.py", line 1115, in check_num_instances_quota Jul 16 21:10:37.625153 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi objects.Quotas.check_deltas(context, deltas, Jul 16 21:10:37.625153 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Jul 16 21:10:37.625153 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Jul 16 21:10:37.625153 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/quotas.py", line 370, in check_deltas Jul 16 21:10:37.625153 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc Jul 16 21:10:37.625153 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/quotas.py", line 365, in check_deltas Jul 16 21:10:37.625153 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi cls.limit_check_project_and_user(context, **check_kwargs) Jul 16 21:10:37.625153 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Jul 16 21:10:37.625153 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Jul 16 21:10:37.625153 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/quotas.py", line 298, in limit_check_project_and_user Jul 16 21:10:37.625153 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return quota.QUOTAS.limit_check_project_and_user(context, Jul 16 21:10:37.625153 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/quota.py", line 1150, in limit_check_project_and_user Jul 16 21:10:37.625153 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return self._driver.limit_check_project_and_user( Jul 16 21:10:37.626435 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/quota.py", line 588, in limit_check_project_and_user Jul 16 21:10:37.626435 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.OverQuota(overs=sorted(overs), Jul 16 21:10:37.626435 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.OverQuota: Quota exceeded for resources: ['ram'] Jul 16 21:10:37.626435 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:10:37.626435 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:10:37.626435 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:10:37.626435 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:10:37.626435 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 784, in create Jul 16 21:10:37.626435 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instances, resv_id = self.compute_api.create( Jul 16 21:10:37.626435 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2149, in create Jul 16 21:10:37.626435 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return self._create_instance( Jul 16 21:10:37.626435 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 1676, in _create_instance Jul 16 21:10:37.626435 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instances_to_build = self._provision_instances( Jul 16 21:10:37.626435 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 1348, in _provision_instances Jul 16 21:10:37.626435 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi num_instances = compute_utils.check_num_instances_quota( Jul 16 21:10:37.626435 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/utils.py", line 1181, in check_num_instances_quota Jul 16 21:10:37.626435 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.TooManyInstances(overs=overs, Jul 16 21:10:37.626435 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.TooManyInstances: Quota exceeded for ram: Requested 1024, but already used 0 of 0 ram Jul 16 21:10:37.626435 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:10:37.626435 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:10:37.626435 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:10:37.627625 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:10:37.627625 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:10:37.627625 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:10:37.627625 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:10:37.627625 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:10:37.627625 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:10:37.627625 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:10:37.627625 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 16 21:10:37.627625 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 16 21:10:37.627625 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 16 21:10:37.627625 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 16 21:10:37.627625 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 16 21:10:37.627625 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 16 21:10:37.627625 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi [Previous line repeated 10 more times] Jul 16 21:10:37.627625 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 800, in create Jul 16 21:10:37.627625 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPForbidden( Jul 16 21:10:37.627625 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPForbidden: Quota exceeded for ram: Requested 1024, but already used 0 of 0 ram Jul 16 21:10:37.627625 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:10:37.627625 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:10:37.627625 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:10:37.627625 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:10:37.629643 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:10:37.629643 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:10:37.629643 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:10:37.629643 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:10:37.629643 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Quota exceeded for ram: Requested 1024, but already used 0 of 0 ram Jul 16 21:10:37.629643 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:10:37.629643 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-3b279d53-0dd8-4c72-a38e-2a49432c4dd1 tempest-QuotasAdminNegativeTest-1085886576 tempest-QuotasAdminNegativeTest-1085886576-project] Returning 403 to user: Quota exceeded for ram: Requested 1024, but already used 0 of 0 ram {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:10:37.629643 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-3b279d53-0dd8-4c72-a38e-2a49432c4dd1 tempest-QuotasAdminNegativeTest-1085886576 tempest-QuotasAdminNegativeTest-1085886576-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 403 len: 110 microversion: 2.1 time: 0.755680 Jul 16 21:10:37.629643 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 365/729] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 21:10:36 2022] POST /compute/v2.1/servers => generated 110 bytes in 757 msecs (HTTP/1.1 403) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:10:37.636251 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-a11b2374-4c31-40fc-95a4-530b06d4e203 tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:37.640515 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-a11b2374-4c31-40fc-95a4-530b06d4e203 tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] [instance: cf53f7cd-deae-4b34-bf8e-3afef98b2ec6] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:37.654221 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a11b2374-4c31-40fc-95a4-530b06d4e203 tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:37.655753 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a11b2374-4c31-40fc-95a4-530b06d4e203 tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:37.665586 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a11b2374-4c31-40fc-95a4-530b06d4e203 tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:37.668114 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a11b2374-4c31-40fc-95a4-530b06d4e203 tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:37.722209 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-208c5437-f4eb-4e8d-910e-78e6aee32137 tempest-LiveMigrationTest-1131758637 tempest-LiveMigrationTest-1131758637-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/4ee3ffbe-c16a-4847-a343-469c645e4b25" status: 200 len: 1645 microversion: 2.1 time: 0.388196 Jul 16 21:10:37.743793 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 365/730] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:10:37 2022] GET /compute/v2.1/servers/4ee3ffbe-c16a-4847-a343-469c645e4b25 => generated 1645 bytes in 410 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:10:37.752255 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-1715af22-1e00-42e1-95ad-91ac4928d88d tempest-QuotasAdminNegativeTest-760882802 tempest-QuotasAdminNegativeTest-760882802-project-admin] Action: 'update', calling method: .version_select at 0x7fdd9da64790>, body: {"quota_set": {"ram": 51200}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:10:37.871576 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1715af22-1e00-42e1-95ad-91ac4928d88d tempest-QuotasAdminNegativeTest-760882802 tempest-QuotasAdminNegativeTest-760882802-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:37.877114 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1715af22-1e00-42e1-95ad-91ac4928d88d tempest-QuotasAdminNegativeTest-760882802 tempest-QuotasAdminNegativeTest-760882802-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:37.878459 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1715af22-1e00-42e1-95ad-91ac4928d88d tempest-QuotasAdminNegativeTest-760882802 tempest-QuotasAdminNegativeTest-760882802-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:37.879170 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1715af22-1e00-42e1-95ad-91ac4928d88d tempest-QuotasAdminNegativeTest-760882802 tempest-QuotasAdminNegativeTest-760882802-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:37.943367 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.objects.instance [None req-a11b2374-4c31-40fc-95a4-530b06d4e203 tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] Lazy-loading 'services' on Instance uuid cf53f7cd-deae-4b34-bf8e-3afef98b2ec6 {{(pid=138680) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:10:37.991864 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-a11b2374-4c31-40fc-95a4-530b06d4e203 tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/cf53f7cd-deae-4b34-bf8e-3afef98b2ec6" status: 200 len: 1929 microversion: 2.75 time: 0.365314 Jul 16 21:10:37.997424 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 366/731] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:10:37 2022] GET /compute/v2.1/servers/cf53f7cd-deae-4b34-bf8e-3afef98b2ec6 => generated 1929 bytes in 371 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:10:38.008710 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-327ce05c-2b00-4ea1-a3d8-8da83241e493 tempest-LiveMigrationTest-1131758637 tempest-LiveMigrationTest-1131758637-project-admin] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:38.020927 ncdv-master-1 devstack@n-api.service[138680]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-services:list": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:10:38.020927 ncdv-master-1 devstack@n-api.service[138680]: warnings.warn(msg) Jul 16 21:10:38.025934 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-327ce05c-2b00-4ea1-a3d8-8da83241e493 tempest-LiveMigrationTest-1131758637 tempest-LiveMigrationTest-1131758637-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:38.026402 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-327ce05c-2b00-4ea1-a3d8-8da83241e493 tempest-LiveMigrationTest-1131758637 tempest-LiveMigrationTest-1131758637-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:38.029354 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-327ce05c-2b00-4ea1-a3d8-8da83241e493 tempest-LiveMigrationTest-1131758637 tempest-LiveMigrationTest-1131758637-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:38.029831 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-327ce05c-2b00-4ea1-a3d8-8da83241e493 tempest-LiveMigrationTest-1131758637 tempest-LiveMigrationTest-1131758637-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:38.066901 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-327ce05c-2b00-4ea1-a3d8-8da83241e493 tempest-LiveMigrationTest-1131758637 tempest-LiveMigrationTest-1131758637-project-admin] 10.222.0.16 "GET /compute/v2.1/os-services?binary=nova-compute" status: 200 len: 380 microversion: 2.1 time: 0.066660 Jul 16 21:10:38.067581 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 367/732] 10.222.0.16 () {58 vars in 1175 bytes} [Sat Jul 16 21:10:37 2022] GET /compute/v2.1/os-services?binary=nova-compute => generated 380 bytes in 69 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:10:38.071163 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-1715af22-1e00-42e1-95ad-91ac4928d88d tempest-QuotasAdminNegativeTest-760882802 tempest-QuotasAdminNegativeTest-760882802-project-admin] 10.222.0.16 "PUT /compute/v2.1/os-quota-sets/29f6db32114b458685269df31fae51c6" status: 200 len: 329 microversion: 2.1 time: 0.325853 Jul 16 21:10:38.072422 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-767c911f-1216-474e-b706-0dc6656104eb tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:38.073407 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-767c911f-1216-474e-b706-0dc6656104eb tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] [instance: b941b324-a54d-446f-bf92-570a80ba2152] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:38.074347 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 366/733] 10.222.0.16 () {60 vars in 1261 bytes} [Sat Jul 16 21:10:37 2022] PUT /compute/v2.1/os-quota-sets/29f6db32114b458685269df31fae51c6 => generated 329 bytes in 330 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:10:38.080707 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-767c911f-1216-474e-b706-0dc6656104eb tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:38.081089 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-767c911f-1216-474e-b706-0dc6656104eb tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:38.081977 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-767c911f-1216-474e-b706-0dc6656104eb tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:38.082477 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-767c911f-1216-474e-b706-0dc6656104eb tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:38.090683 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-0eae0c1a-16ad-47cf-b758-6f35fbbcb6dc tempest-LiveMigrationTest-1131758637 tempest-LiveMigrationTest-1131758637-project-admin] Action: 'action', calling method: >, body: {"pause": {}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:10:38.091886 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-0eae0c1a-16ad-47cf-b758-6f35fbbcb6dc tempest-LiveMigrationTest-1131758637 tempest-LiveMigrationTest-1131758637-project-admin] [instance: 4ee3ffbe-c16a-4847-a343-469c645e4b25] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:38.102572 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0eae0c1a-16ad-47cf-b758-6f35fbbcb6dc tempest-LiveMigrationTest-1131758637 tempest-LiveMigrationTest-1131758637-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:38.110501 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0eae0c1a-16ad-47cf-b758-6f35fbbcb6dc tempest-LiveMigrationTest-1131758637 tempest-LiveMigrationTest-1131758637-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:38.111754 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0eae0c1a-16ad-47cf-b758-6f35fbbcb6dc tempest-LiveMigrationTest-1131758637 tempest-LiveMigrationTest-1131758637-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:38.114764 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0eae0c1a-16ad-47cf-b758-6f35fbbcb6dc tempest-LiveMigrationTest-1131758637 tempest-LiveMigrationTest-1131758637-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:38.299986 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-767c911f-1216-474e-b706-0dc6656104eb tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'abeb7020019d41a18de42956d706a011', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '649babe54d104436af461f1ee4767001', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:38.314480 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-767c911f-1216-474e-b706-0dc6656104eb tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] 10.222.0.16 "GET /compute/v2.1/servers/b941b324-a54d-446f-bf92-570a80ba2152" status: 200 len: 1317 microversion: 2.1 time: 0.241139 Jul 16 21:10:38.319356 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 368/734] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:10:38 2022] GET /compute/v2.1/servers/b941b324-a54d-446f-bf92-570a80ba2152 => generated 1317 bytes in 251 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:10:38.327102 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-6f0669fa-0d7c-4cf6-8da6-c475f071abea tempest-QuotasAdminNegativeTest-1085886576 tempest-QuotasAdminNegativeTest-1085886576-project] Action: 'update', calling method: .version_select at 0x7fdd9dd92a60>, body: {"quota_set": {"ram": 0}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:10:38.333968 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-6f0669fa-0d7c-4cf6-8da6-c475f071abea tempest-QuotasAdminNegativeTest-1085886576 tempest-QuotasAdminNegativeTest-1085886576-project] Policy check for os_compute_api:os-quota-sets:update failed with credentials {'is_admin': False, 'user_id': 'ca38f16f48494cc4aa71c7f5bd0916a4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '29f6db32114b458685269df31fae51c6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:38.337064 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-6f0669fa-0d7c-4cf6-8da6-c475f071abea tempest-QuotasAdminNegativeTest-1085886576 tempest-QuotasAdminNegativeTest-1085886576-project] Request method failure captured: Jul 16 21:10:38.337064 ncdv-master-1 devstack@n-api.service[138680]: request: PUT /compute/v2.1/os-quota-sets/29f6db32114b458685269df31fae51c6 HTTP/1.1 Jul 16 21:10:38.337064 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:10:38.337064 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:10:38.337064 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:10:38.337064 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 25 Jul 16 21:10:38.337064 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:10:38.337064 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:10:38.337064 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:10:38.337064 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:10:38.337064 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0ylE3FeTCqHPw-nvznMcNwutvdVxrvxhHLMrl2DYHQWWrO_k5katXxSjE94qMCq_CcgcPCVdrJzaOJpXBYMMdRfP4KFW0LwWu9nJ1sGwPCoomWMu72aP9IflG0eb5aKzJ-qUYFJuSDp2eg3ZWjM-CDZZ_ESxIokox1kDJq2YBo2e11E Jul 16 21:10:38.337064 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:10:38.337064 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:10:38.337064 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:10:38.337064 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:10:38.337064 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:10:38.337064 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:10:38.337064 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: 29f6db32114b458685269df31fae51c6 Jul 16 21:10:38.337064 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-QuotasAdminNegativeTest-1085886576 Jul 16 21:10:38.337064 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:10:38.337064 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:10:38.338441 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/29f6db32114b458685269df31fae51c6"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/29f6db32114b458685269df31fae51c6"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_29f6db32114b458685269df31fae51c6"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/29f6db32114b458685269df31fae51c6"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:10:38.338441 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-QuotasAdminNegativeTest-1085886576 Jul 16 21:10:38.338441 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: 29f6db32114b458685269df31fae51c6 Jul 16 21:10:38.338441 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-QuotasAdminNegativeTest-1085886576 Jul 16 21:10:38.338441 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-QuotasAdminNegativeTest-1085886576-project Jul 16 21:10:38.338441 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:10:38.338441 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:10:38.338441 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: ca38f16f48494cc4aa71c7f5bd0916a4 Jul 16 21:10:38.338441 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-QuotasAdminNegativeTest-1085886576-project Jul 16 21:10:38.338441 ncdv-master-1 devstack@n-api.service[138680]: Jul 16 21:10:38.338441 ncdv-master-1 devstack@n-api.service[138680]: {"quota_set": {"ram": 0}} Jul 16 21:10:38.338441 ncdv-master-1 devstack@n-api.service[138680]: method: Jul 16 21:10:38.338441 ncdv-master-1 devstack@n-api.service[138680]: exception: Policy doesn't allow os_compute_api:os-quota-sets:update to be performed. Jul 16 21:10:38.338441 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': '29f6db32114b458685269df31fae51c6', 'body': {'quota_set': {'ram': 0}}} Jul 16 21:10:38.339313 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:10:38.339313 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:10:38.339313 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:10:38.339313 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:10:38.339313 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:10:38.339313 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:10:38.339313 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Jul 16 21:10:38.339313 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Jul 16 21:10:38.339313 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:10:38.339313 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:10:38.339313 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 16 21:10:38.339313 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 16 21:10:38.339313 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/quota_sets.py", line 170, in update Jul 16 21:10:38.339313 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return self._update(req, id, body, []) Jul 16 21:10:38.339313 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper Jul 16 21:10:38.339313 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 16 21:10:38.339313 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper Jul 16 21:10:38.339313 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 16 21:10:38.339313 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/quota_sets.py", line 189, in _update Jul 16 21:10:38.341124 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi context.can(qs_policies.POLICY_ROOT % 'update', {'project_id': id}) Jul 16 21:10:38.341124 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/context.py", line 240, in can Jul 16 21:10:38.341124 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return policy.authorize(self, action, target) Jul 16 21:10:38.341124 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 199, in authorize Jul 16 21:10:38.341124 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi LOG.debug('Policy check for %(action)s failed with credentials ' Jul 16 21:10:38.341124 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 16 21:10:38.341124 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi self.force_reraise() Jul 16 21:10:38.341124 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 16 21:10:38.341124 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise self.value Jul 16 21:10:38.341124 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 186, in authorize Jul 16 21:10:38.341124 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi result = _ENFORCER.authorize(action, target, context, Jul 16 21:10:38.341124 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1184, in authorize Jul 16 21:10:38.341124 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return self.enforce( Jul 16 21:10:38.341124 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1088, in enforce Jul 16 21:10:38.341124 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc(*args, **kwargs) Jul 16 21:10:38.341124 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.PolicyNotAuthorized: Policy doesn't allow os_compute_api:os-quota-sets:update to be performed. Jul 16 21:10:38.341124 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:10:38.341124 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:10:38.341124 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:10:38.341124 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:10:38.343988 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:10:38.343988 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:10:38.343988 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 354, in __exit__ Jul 16 21:10:38.343988 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(webob.exc.HTTPForbidden( Jul 16 21:10:38.343988 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Policy doesn't allow os_compute_api:os-quota-sets:update to be performed. Jul 16 21:10:38.343988 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:10:38.343988 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-6f0669fa-0d7c-4cf6-8da6-c475f071abea tempest-QuotasAdminNegativeTest-1085886576 tempest-QuotasAdminNegativeTest-1085886576-project] Returning 403 to user: Policy doesn't allow os_compute_api:os-quota-sets:update to be performed. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:10:38.343988 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-6f0669fa-0d7c-4cf6-8da6-c475f071abea tempest-QuotasAdminNegativeTest-1085886576 tempest-QuotasAdminNegativeTest-1085886576-project] 10.222.0.16 "PUT /compute/v2.1/os-quota-sets/29f6db32114b458685269df31fae51c6" status: 403 len: 116 microversion: 2.1 time: 0.018526 Jul 16 21:10:38.343988 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 369/735] 10.222.0.16 () {60 vars in 1261 bytes} [Sat Jul 16 21:10:38 2022] PUT /compute/v2.1/os-quota-sets/29f6db32114b458685269df31fae51c6 => generated 116 bytes in 20 msecs (HTTP/1.1 403) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:10:38.345645 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-1f168eda-f5b1-4807-a4a6-cf664d68efdb tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:38.346262 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-1f168eda-f5b1-4807-a4a6-cf664d68efdb tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: f98a9762-a67c-4e95-85b8-727c81bdb03a] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:38.353947 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1f168eda-f5b1-4807-a4a6-cf664d68efdb tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:38.354614 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1f168eda-f5b1-4807-a4a6-cf664d68efdb tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:38.355399 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1f168eda-f5b1-4807-a4a6-cf664d68efdb tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:38.355846 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1f168eda-f5b1-4807-a4a6-cf664d68efdb tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:38.454952 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-0eae0c1a-16ad-47cf-b758-6f35fbbcb6dc tempest-LiveMigrationTest-1131758637 tempest-LiveMigrationTest-1131758637-project-admin] 10.222.0.16 "POST /compute/v2.1/servers/4ee3ffbe-c16a-4847-a343-469c645e4b25/action" status: 202 len: 0 microversion: 2.1 time: 0.375722 Jul 16 21:10:38.458845 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 367/736] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 21:10:38 2022] POST /compute/v2.1/servers/4ee3ffbe-c16a-4847-a343-469c645e4b25/action => generated 0 bytes in 381 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 21:10:38.577209 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-6f966151-f6fa-44e3-8910-e4fc02674998 tempest-SecurityGroupRulesTestJSON-1769504197 tempest-SecurityGroupRulesTestJSON-1769504197-project] Action: 'create', calling method: .version_select at 0x7fdd9d267790>, body: {"security_group": {"name": "tempest-SecurityGroupRulesTestJSON-securitygroup-1074575969", "description": "tempest-description-1216761633"}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:10:38.743147 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-1f168eda-f5b1-4807-a4a6-cf664d68efdb tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b68dacc4b8463ba07fae6372131117', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55a19b7a4682457a9132f5fc80110712', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:38.754145 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-1f168eda-f5b1-4807-a4a6-cf664d68efdb tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "GET /compute/v2.1/servers/f98a9762-a67c-4e95-85b8-727c81bdb03a" status: 200 len: 1476 microversion: 2.1 time: 0.413381 Jul 16 21:10:38.754734 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 370/737] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:10:38 2022] GET /compute/v2.1/servers/f98a9762-a67c-4e95-85b8-727c81bdb03a => generated 1476 bytes in 415 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:10:38.759491 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-60857ad3-99a9-4407-9544-780897e17002 tempest-LiveMigrationTest-1131758637 tempest-LiveMigrationTest-1131758637-project-admin] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:38.760473 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-60857ad3-99a9-4407-9544-780897e17002 tempest-LiveMigrationTest-1131758637 tempest-LiveMigrationTest-1131758637-project-admin] [instance: 4ee3ffbe-c16a-4847-a343-469c645e4b25] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:38.769555 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-60857ad3-99a9-4407-9544-780897e17002 tempest-LiveMigrationTest-1131758637 tempest-LiveMigrationTest-1131758637-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:38.770020 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-60857ad3-99a9-4407-9544-780897e17002 tempest-LiveMigrationTest-1131758637 tempest-LiveMigrationTest-1131758637-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:38.776775 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-60857ad3-99a9-4407-9544-780897e17002 tempest-LiveMigrationTest-1131758637 tempest-LiveMigrationTest-1131758637-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:38.776775 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-60857ad3-99a9-4407-9544-780897e17002 tempest-LiveMigrationTest-1131758637 tempest-LiveMigrationTest-1131758637-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:39.067058 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-60857ad3-99a9-4407-9544-780897e17002 tempest-LiveMigrationTest-1131758637 tempest-LiveMigrationTest-1131758637-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/4ee3ffbe-c16a-4847-a343-469c645e4b25" status: 200 len: 1650 microversion: 2.1 time: 0.310949 Jul 16 21:10:39.071472 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 371/738] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:10:38 2022] GET /compute/v2.1/servers/4ee3ffbe-c16a-4847-a343-469c645e4b25 => generated 1650 bytes in 313 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:10:39.075996 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-f27aad55-f8f0-4175-9af8-9a93520aefa3 tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:39.077463 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-f27aad55-f8f0-4175-9af8-9a93520aefa3 tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] [instance: cf53f7cd-deae-4b34-bf8e-3afef98b2ec6] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:39.084980 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f27aad55-f8f0-4175-9af8-9a93520aefa3 tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:39.085800 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f27aad55-f8f0-4175-9af8-9a93520aefa3 tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:39.087277 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f27aad55-f8f0-4175-9af8-9a93520aefa3 tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:39.087957 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f27aad55-f8f0-4175-9af8-9a93520aefa3 tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:39.199291 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-6f966151-f6fa-44e3-8910-e4fc02674998 tempest-SecurityGroupRulesTestJSON-1769504197 tempest-SecurityGroupRulesTestJSON-1769504197-project] 10.222.0.16 "POST /compute/v2.1/os-security-groups" status: 200 len: 248 microversion: 2.1 time: 0.741877 Jul 16 21:10:39.200216 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 368/739] 10.222.0.16 () {60 vars in 1179 bytes} [Sat Jul 16 21:10:38 2022] POST /compute/v2.1/os-security-groups => generated 248 bytes in 743 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:10:39.211724 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-b61ccb26-b1f8-4af5-9a34-b34e749dcc6f tempest-SecurityGroupRulesTestJSON-1769504197 tempest-SecurityGroupRulesTestJSON-1769504197-project] Action: 'create', calling method: .version_select at 0x7fdd9d4cc0d0>, body: {"security_group_rule": {"parent_group_id": "24258b25-667d-4e38-a209-c64d74832751", "ip_protocol": "tcp", "from_port": 22, "to_port": 22}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:10:39.220600 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.objects.instance [None req-f27aad55-f8f0-4175-9af8-9a93520aefa3 tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] Lazy-loading 'services' on Instance uuid cf53f7cd-deae-4b34-bf8e-3afef98b2ec6 {{(pid=138680) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:10:39.242707 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-f27aad55-f8f0-4175-9af8-9a93520aefa3 tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/cf53f7cd-deae-4b34-bf8e-3afef98b2ec6" status: 200 len: 1929 microversion: 2.75 time: 0.173646 Jul 16 21:10:39.254785 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 372/740] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:10:39 2022] GET /compute/v2.1/servers/cf53f7cd-deae-4b34-bf8e-3afef98b2ec6 => generated 1929 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:10:39.334975 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-6b7afe66-79ae-459e-9ad4-d94884b4de9c tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:39.336006 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-6b7afe66-79ae-459e-9ad4-d94884b4de9c tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] [instance: b941b324-a54d-446f-bf92-570a80ba2152] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:39.351656 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6b7afe66-79ae-459e-9ad4-d94884b4de9c tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:39.352119 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6b7afe66-79ae-459e-9ad4-d94884b4de9c tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:39.353296 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6b7afe66-79ae-459e-9ad4-d94884b4de9c tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:39.354039 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6b7afe66-79ae-459e-9ad4-d94884b4de9c tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:39.374486 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=100 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:10:39.700446 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-6b7afe66-79ae-459e-9ad4-d94884b4de9c tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'abeb7020019d41a18de42956d706a011', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '649babe54d104436af461f1ee4767001', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:39.710089 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-6b7afe66-79ae-459e-9ad4-d94884b4de9c tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] 10.222.0.16 "GET /compute/v2.1/servers/b941b324-a54d-446f-bf92-570a80ba2152" status: 200 len: 1317 microversion: 2.1 time: 0.380546 Jul 16 21:10:39.711484 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 373/741] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:10:39 2022] GET /compute/v2.1/servers/b941b324-a54d-446f-bf92-570a80ba2152 => generated 1317 bytes in 384 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:10:39.773742 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-17645012-3632-4717-91fe-bb23fc5c76ae tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:39.775247 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-17645012-3632-4717-91fe-bb23fc5c76ae tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: f98a9762-a67c-4e95-85b8-727c81bdb03a] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:39.808270 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-17645012-3632-4717-91fe-bb23fc5c76ae tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:39.809368 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-17645012-3632-4717-91fe-bb23fc5c76ae tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:39.810259 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-17645012-3632-4717-91fe-bb23fc5c76ae tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:39.812767 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-17645012-3632-4717-91fe-bb23fc5c76ae tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:40.182038 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-17645012-3632-4717-91fe-bb23fc5c76ae tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b68dacc4b8463ba07fae6372131117', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55a19b7a4682457a9132f5fc80110712', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:40.191131 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-17645012-3632-4717-91fe-bb23fc5c76ae tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "GET /compute/v2.1/servers/f98a9762-a67c-4e95-85b8-727c81bdb03a" status: 200 len: 1476 microversion: 2.1 time: 0.422777 Jul 16 21:10:40.194384 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 374/742] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:10:39 2022] GET /compute/v2.1/servers/f98a9762-a67c-4e95-85b8-727c81bdb03a => generated 1476 bytes in 425 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:10:40.226331 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-4868c1fe-9b46-478c-894f-4d81aa0a6fe3 tempest-LiveMigrationTest-1131758637 tempest-LiveMigrationTest-1131758637-project-admin] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:40.234269 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-4868c1fe-9b46-478c-894f-4d81aa0a6fe3 tempest-LiveMigrationTest-1131758637 tempest-LiveMigrationTest-1131758637-project-admin] [instance: 4ee3ffbe-c16a-4847-a343-469c645e4b25] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:40.252857 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4868c1fe-9b46-478c-894f-4d81aa0a6fe3 tempest-LiveMigrationTest-1131758637 tempest-LiveMigrationTest-1131758637-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:40.254023 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4868c1fe-9b46-478c-894f-4d81aa0a6fe3 tempest-LiveMigrationTest-1131758637 tempest-LiveMigrationTest-1131758637-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:40.255486 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4868c1fe-9b46-478c-894f-4d81aa0a6fe3 tempest-LiveMigrationTest-1131758637 tempest-LiveMigrationTest-1131758637-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:40.256700 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4868c1fe-9b46-478c-894f-4d81aa0a6fe3 tempest-LiveMigrationTest-1131758637 tempest-LiveMigrationTest-1131758637-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:40.275052 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-b61ccb26-b1f8-4af5-9a34-b34e749dcc6f tempest-SecurityGroupRulesTestJSON-1769504197 tempest-SecurityGroupRulesTestJSON-1769504197-project] 10.222.0.16 "POST /compute/v2.1/os-security-group-rules" status: 200 len: 232 microversion: 2.1 time: 1.066562 Jul 16 21:10:40.275494 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 369/743] 10.222.0.16 () {60 vars in 1194 bytes} [Sat Jul 16 21:10:39 2022] POST /compute/v2.1/os-security-group-rules => generated 232 bytes in 1068 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:10:40.282641 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-ac3e4132-9414-48cf-90cb-b69a87ffee17 tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:40.283439 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-ac3e4132-9414-48cf-90cb-b69a87ffee17 tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] [instance: cf53f7cd-deae-4b34-bf8e-3afef98b2ec6] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:40.297711 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ac3e4132-9414-48cf-90cb-b69a87ffee17 tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:40.298534 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ac3e4132-9414-48cf-90cb-b69a87ffee17 tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:40.300428 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ac3e4132-9414-48cf-90cb-b69a87ffee17 tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:40.301148 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ac3e4132-9414-48cf-90cb-b69a87ffee17 tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:40.476841 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.objects.instance [None req-ac3e4132-9414-48cf-90cb-b69a87ffee17 tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] Lazy-loading 'services' on Instance uuid cf53f7cd-deae-4b34-bf8e-3afef98b2ec6 {{(pid=138681) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:10:40.496618 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-ac3e4132-9414-48cf-90cb-b69a87ffee17 tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/cf53f7cd-deae-4b34-bf8e-3afef98b2ec6" status: 200 len: 1929 microversion: 2.75 time: 0.219769 Jul 16 21:10:40.497761 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 370/744] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:10:40 2022] GET /compute/v2.1/servers/cf53f7cd-deae-4b34-bf8e-3afef98b2ec6 => generated 1929 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:10:40.504356 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-fc212f60-d2b9-4765-ab6c-82aa560077b6 tempest-SecurityGroupRulesTestJSON-1769504197 tempest-SecurityGroupRulesTestJSON-1769504197-project] Action: 'create', calling method: .version_select at 0x7fdd9d4cc0d0>, body: {"security_group": {"name": "tempest-SecurityGroupRulesTestJSON-securitygroup-766808467", "description": "tempest-description-1048721586"}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:10:40.667467 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-4868c1fe-9b46-478c-894f-4d81aa0a6fe3 tempest-LiveMigrationTest-1131758637 tempest-LiveMigrationTest-1131758637-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/4ee3ffbe-c16a-4847-a343-469c645e4b25" status: 200 len: 1630 microversion: 2.1 time: 0.473583 Jul 16 21:10:40.687217 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 375/745] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:10:40 2022] GET /compute/v2.1/servers/4ee3ffbe-c16a-4847-a343-469c645e4b25 => generated 1630 bytes in 492 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:10:40.709039 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-a6bf0c92-2907-4f4e-9560-5d5f6b92d7db tempest-LiveMigrationTest-1131758637 tempest-LiveMigrationTest-1131758637-project-admin] Action: 'action', calling method: >, body: {"os-migrateLive": {"host": "nch2-master-1", "block_migration": true, "disk_over_commit": false}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:10:40.711533 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-a6bf0c92-2907-4f4e-9560-5d5f6b92d7db tempest-LiveMigrationTest-1131758637 tempest-LiveMigrationTest-1131758637-project-admin] [instance: 4ee3ffbe-c16a-4847-a343-469c645e4b25] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:40.722035 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a6bf0c92-2907-4f4e-9560-5d5f6b92d7db tempest-LiveMigrationTest-1131758637 tempest-LiveMigrationTest-1131758637-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:40.723157 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a6bf0c92-2907-4f4e-9560-5d5f6b92d7db tempest-LiveMigrationTest-1131758637 tempest-LiveMigrationTest-1131758637-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:40.724394 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a6bf0c92-2907-4f4e-9560-5d5f6b92d7db tempest-LiveMigrationTest-1131758637 tempest-LiveMigrationTest-1131758637-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:40.725799 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a6bf0c92-2907-4f4e-9560-5d5f6b92d7db tempest-LiveMigrationTest-1131758637 tempest-LiveMigrationTest-1131758637-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:40.757935 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.objects.instance [None req-a6bf0c92-2907-4f4e-9560-5d5f6b92d7db tempest-LiveMigrationTest-1131758637 tempest-LiveMigrationTest-1131758637-project-admin] Lazy-loading 'flavor' on Instance uuid 4ee3ffbe-c16a-4847-a343-469c645e4b25 {{(pid=138680) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:10:40.774064 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-a6bf0c92-2907-4f4e-9560-5d5f6b92d7db tempest-LiveMigrationTest-1131758637 tempest-LiveMigrationTest-1131758637-project-admin] [instance: 4ee3ffbe-c16a-4847-a343-469c645e4b25] Going to try to live migrate instance to nch2-master-1 {{(pid=138680) live_migrate /opt/stack/nova/nova/compute/api.py:5349}} Jul 16 21:10:41.096298 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-fc212f60-d2b9-4765-ab6c-82aa560077b6 tempest-SecurityGroupRulesTestJSON-1769504197 tempest-SecurityGroupRulesTestJSON-1769504197-project] 10.222.0.16 "POST /compute/v2.1/os-security-groups" status: 200 len: 247 microversion: 2.1 time: 0.593804 Jul 16 21:10:41.096298 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 371/746] 10.222.0.16 () {60 vars in 1179 bytes} [Sat Jul 16 21:10:40 2022] POST /compute/v2.1/os-security-groups => generated 247 bytes in 596 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:10:41.099542 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-ca0a7ab1-beb2-4cfc-aba7-54fcb925358d req-e468b8b5-b401-40f9-b98b-5b7ed89f8fc5 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "b941b324-a54d-446f-bf92-570a80ba2152", "name": "network-vif-plugged", "status": "completed", "tag": "dc03a8a7-1bc4-4e8d-91b2-65102ff61630"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:10:41.110168 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-ca0a7ab1-beb2-4cfc-aba7-54fcb925358d req-e468b8b5-b401-40f9-b98b-5b7ed89f8fc5 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:41.110877 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-ca0a7ab1-beb2-4cfc-aba7-54fcb925358d req-e468b8b5-b401-40f9-b98b-5b7ed89f8fc5 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:41.135709 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-ca0a7ab1-beb2-4cfc-aba7-54fcb925358d req-e468b8b5-b401-40f9-b98b-5b7ed89f8fc5 service nova] Creating event network-vif-plugged:dc03a8a7-1bc4-4e8d-91b2-65102ff61630 for instance b941b324-a54d-446f-bf92-570a80ba2152 on nch1-master-1 Jul 16 21:10:41.148352 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-ca0a7ab1-beb2-4cfc-aba7-54fcb925358d req-e468b8b5-b401-40f9-b98b-5b7ed89f8fc5 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.047216 Jul 16 21:10:41.152814 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 372/747] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:10:41 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:10:41.159836 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-0ea035f4-562c-4339-a871-1e1b94fabf3c tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:41.160549 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-0ea035f4-562c-4339-a871-1e1b94fabf3c tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] [instance: b941b324-a54d-446f-bf92-570a80ba2152] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:41.167600 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0ea035f4-562c-4339-a871-1e1b94fabf3c tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:41.176577 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0ea035f4-562c-4339-a871-1e1b94fabf3c tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:41.176577 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0ea035f4-562c-4339-a871-1e1b94fabf3c tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:41.176577 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0ea035f4-562c-4339-a871-1e1b94fabf3c tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:41.325321 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-0ea035f4-562c-4339-a871-1e1b94fabf3c tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'abeb7020019d41a18de42956d706a011', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '649babe54d104436af461f1ee4767001', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:41.334470 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-0ea035f4-562c-4339-a871-1e1b94fabf3c tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] 10.222.0.16 "GET /compute/v2.1/servers/b941b324-a54d-446f-bf92-570a80ba2152" status: 200 len: 1317 microversion: 2.1 time: 0.179695 Jul 16 21:10:41.335637 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 373/748] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:10:41 2022] GET /compute/v2.1/servers/b941b324-a54d-446f-bf92-570a80ba2152 => generated 1317 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:10:41.785215 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-ad46f899-d715-409e-9ef0-dcd620a84c1a tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-LiveAutoBlockMigrationV225Test-server-1307137206", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452", "networks": [{"uuid": "b3fd4d7b-96dc-4cc4-8a4c-4e1986f1bc15"}]}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:10:41.891248 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.network.neutron [None req-ad46f899-d715-409e-9ef0-dcd620a84c1a tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] validate_networks() for [('b3fd4d7b-96dc-4cc4-8a4c-4e1986f1bc15', None, None, None, None, None)] {{(pid=138681) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 16 21:10:42.566089 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-ad46f899-d715-409e-9ef0-dcd620a84c1a tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Flavor limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 16 21:10:42.566687 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-ad46f899-d715-409e-9ef0-dcd620a84c1a tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Image limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 16 21:10:42.568407 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-ad46f899-d715-409e-9ef0-dcd620a84c1a tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Flavor pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 16 21:10:42.569519 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-ad46f899-d715-409e-9ef0-dcd620a84c1a tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Image pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 16 21:10:42.570096 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-ad46f899-d715-409e-9ef0-dcd620a84c1a tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 16 21:10:42.585493 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ad46f899-d715-409e-9ef0-dcd620a84c1a tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:42.586341 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ad46f899-d715-409e-9ef0-dcd620a84c1a tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:42.586776 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ad46f899-d715-409e-9ef0-dcd620a84c1a tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:42.587281 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ad46f899-d715-409e-9ef0-dcd620a84c1a tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:42.608338 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-ad46f899-d715-409e-9ef0-dcd620a84c1a tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Getting quotas for project d9b92fc544c544f686ac2ab7ff5123b6. Resources: {'ram', 'instances', 'cores'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:10:42.614826 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-ad46f899-d715-409e-9ef0-dcd620a84c1a tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Getting quotas for user 578b87f863b34f57a571f7e56d44f118 and project d9b92fc544c544f686ac2ab7ff5123b6. Resources: {'ram', 'instances', 'cores'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:10:42.635941 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-ad46f899-d715-409e-9ef0-dcd620a84c1a tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Going to run 1 instances... {{(pid=138681) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 16 21:10:42.635941 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-ad46f899-d715-409e-9ef0-dcd620a84c1a tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] [instance: de210dbd-1398-413f-9a0f-63cae300bd03] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138681) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 16 21:10:42.836791 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-ad46f899-d715-409e-9ef0-dcd620a84c1a tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.25 time: 1.500012 Jul 16 21:10:42.843182 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 374/749] 10.222.0.16 () {62 vars in 1187 bytes} [Sat Jul 16 21:10:41 2022] POST /compute/v2.1/servers => generated 380 bytes in 1507 msecs (HTTP/1.1 202) 10 headers in 453 bytes (1 switches on core 0) Jul 16 21:10:42.853562 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-350eb40e-e456-4012-8bf5-45c6c64a552c tempest-SecurityGroupRulesTestJSON-1769504197 tempest-SecurityGroupRulesTestJSON-1769504197-project] Action: 'create', calling method: .version_select at 0x7fdd9d540af0>, body: {"security_group_rule": {"parent_group_id": "7e73aa4a-33ad-4464-97d6-98250316ce9c", "ip_protocol": "tcp", "from_port": 22, "to_port": 22, "cidr": "10.2.3.124/24"}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:10:43.479532 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-350eb40e-e456-4012-8bf5-45c6c64a552c tempest-SecurityGroupRulesTestJSON-1769504197 tempest-SecurityGroupRulesTestJSON-1769504197-project] 10.222.0.16 "POST /compute/v2.1/os-security-group-rules" status: 200 len: 236 microversion: 2.1 time: 0.632908 Jul 16 21:10:43.487705 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 375/750] 10.222.0.16 () {60 vars in 1194 bytes} [Sat Jul 16 21:10:42 2022] POST /compute/v2.1/os-security-group-rules => generated 236 bytes in 643 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:10:43.494600 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-25f5b169-864c-4019-ae09-a5a879bd9ceb tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:43.495513 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-25f5b169-864c-4019-ae09-a5a879bd9ceb tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: f98a9762-a67c-4e95-85b8-727c81bdb03a] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:43.506602 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-25f5b169-864c-4019-ae09-a5a879bd9ceb tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:43.507257 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-25f5b169-864c-4019-ae09-a5a879bd9ceb tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:43.508614 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-25f5b169-864c-4019-ae09-a5a879bd9ceb tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:43.509709 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-25f5b169-864c-4019-ae09-a5a879bd9ceb tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:43.889091 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-25f5b169-864c-4019-ae09-a5a879bd9ceb tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b68dacc4b8463ba07fae6372131117', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55a19b7a4682457a9132f5fc80110712', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:43.914159 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-25f5b169-864c-4019-ae09-a5a879bd9ceb tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "GET /compute/v2.1/servers/f98a9762-a67c-4e95-85b8-727c81bdb03a" status: 200 len: 1475 microversion: 2.1 time: 0.424299 Jul 16 21:10:43.915945 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 376/751] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:10:43 2022] GET /compute/v2.1/servers/f98a9762-a67c-4e95-85b8-727c81bdb03a => generated 1475 bytes in 427 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:10:43.947511 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-323c74c4-974a-4224-b7dd-60cfb2bdcc04 tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:43.949175 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-323c74c4-974a-4224-b7dd-60cfb2bdcc04 tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] [instance: cf53f7cd-deae-4b34-bf8e-3afef98b2ec6] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:43.970094 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-323c74c4-974a-4224-b7dd-60cfb2bdcc04 tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:43.972125 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-323c74c4-974a-4224-b7dd-60cfb2bdcc04 tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:43.973328 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-323c74c4-974a-4224-b7dd-60cfb2bdcc04 tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:43.976298 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-323c74c4-974a-4224-b7dd-60cfb2bdcc04 tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:44.227934 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.objects.instance [None req-323c74c4-974a-4224-b7dd-60cfb2bdcc04 tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] Lazy-loading 'services' on Instance uuid cf53f7cd-deae-4b34-bf8e-3afef98b2ec6 {{(pid=138681) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:10:44.265973 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-323c74c4-974a-4224-b7dd-60cfb2bdcc04 tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/cf53f7cd-deae-4b34-bf8e-3afef98b2ec6" status: 200 len: 1929 microversion: 2.75 time: 0.347008 Jul 16 21:10:44.267134 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 377/752] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:10:43 2022] GET /compute/v2.1/servers/cf53f7cd-deae-4b34-bf8e-3afef98b2ec6 => generated 1929 bytes in 349 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:10:44.272571 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-f931a7cb-b9c2-4575-b2bf-77460ce6e267 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:44.273516 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-f931a7cb-b9c2-4575-b2bf-77460ce6e267 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] [instance: b941b324-a54d-446f-bf92-570a80ba2152] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:44.283205 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f931a7cb-b9c2-4575-b2bf-77460ce6e267 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:44.283764 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f931a7cb-b9c2-4575-b2bf-77460ce6e267 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:44.284824 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f931a7cb-b9c2-4575-b2bf-77460ce6e267 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:44.285750 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f931a7cb-b9c2-4575-b2bf-77460ce6e267 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:44.576961 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-f931a7cb-b9c2-4575-b2bf-77460ce6e267 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'abeb7020019d41a18de42956d706a011', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '649babe54d104436af461f1ee4767001', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:44.588355 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-f931a7cb-b9c2-4575-b2bf-77460ce6e267 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] 10.222.0.16 "GET /compute/v2.1/servers/b941b324-a54d-446f-bf92-570a80ba2152" status: 200 len: 1466 microversion: 2.1 time: 0.319425 Jul 16 21:10:44.592454 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 378/753] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:10:44 2022] GET /compute/v2.1/servers/b941b324-a54d-446f-bf92-570a80ba2152 => generated 1466 bytes in 324 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:10:44.611133 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-e2268bd6-a2dc-4961-a322-d831e2d32b01 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:44.612136 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-e2268bd6-a2dc-4961-a322-d831e2d32b01 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] [instance: de210dbd-1398-413f-9a0f-63cae300bd03] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:44.622465 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e2268bd6-a2dc-4961-a322-d831e2d32b01 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:44.622926 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e2268bd6-a2dc-4961-a322-d831e2d32b01 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:44.632046 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e2268bd6-a2dc-4961-a322-d831e2d32b01 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:44.632525 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e2268bd6-a2dc-4961-a322-d831e2d32b01 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:44.739621 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-e2268bd6-a2dc-4961-a322-d831e2d32b01 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:44.749964 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-e2268bd6-a2dc-4961-a322-d831e2d32b01 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:44.753021 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-e2268bd6-a2dc-4961-a322-d831e2d32b01 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:44.754516 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-e2268bd6-a2dc-4961-a322-d831e2d32b01 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] 10.222.0.16 "GET /compute/v2.1/servers/de210dbd-1398-413f-9a0f-63cae300bd03" status: 200 len: 1266 microversion: 2.25 time: 0.160899 Jul 16 21:10:44.755375 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 379/754] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:10:44 2022] GET /compute/v2.1/servers/de210dbd-1398-413f-9a0f-63cae300bd03 => generated 1266 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:10:44.760706 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-cf6f8759-88ae-4da6-8f46-3d99ad07e2f1 tempest-SecurityGroupRulesTestJSON-1769504197 tempest-SecurityGroupRulesTestJSON-1769504197-project] Action: 'create', calling method: .version_select at 0x7fdd9d29e8b0>, body: {"security_group": {"name": "tempest-SecurityGroupRulesTestJSON-securitygroup-1913136377", "description": "tempest-description-1678214739"}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:10:45.564653 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-a6bf0c92-2907-4f4e-9560-5d5f6b92d7db tempest-LiveMigrationTest-1131758637 tempest-LiveMigrationTest-1131758637-project-admin] 10.222.0.16 "POST /compute/v2.1/servers/4ee3ffbe-c16a-4847-a343-469c645e4b25/action" status: 202 len: 0 microversion: 2.1 time: 4.875855 Jul 16 21:10:45.567321 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 376/755] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 21:10:40 2022] POST /compute/v2.1/servers/4ee3ffbe-c16a-4847-a343-469c645e4b25/action => generated 0 bytes in 4879 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 21:10:45.580186 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-57b6bb1f-463e-4399-9888-8763ea6722ec tempest-ServersOnMultiNodesTest-83775055 tempest-ServersOnMultiNodesTest-83775055-project-admin] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:45.600661 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-57b6bb1f-463e-4399-9888-8763ea6722ec tempest-ServersOnMultiNodesTest-83775055 tempest-ServersOnMultiNodesTest-83775055-project-admin] [instance: b941b324-a54d-446f-bf92-570a80ba2152] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:45.618750 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-cf6f8759-88ae-4da6-8f46-3d99ad07e2f1 tempest-SecurityGroupRulesTestJSON-1769504197 tempest-SecurityGroupRulesTestJSON-1769504197-project] 10.222.0.16 "POST /compute/v2.1/os-security-groups" status: 200 len: 248 microversion: 2.1 time: 0.855526 Jul 16 21:10:45.618750 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 380/756] 10.222.0.16 () {60 vars in 1179 bytes} [Sat Jul 16 21:10:44 2022] POST /compute/v2.1/os-security-groups => generated 248 bytes in 857 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:10:45.620580 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-57b6bb1f-463e-4399-9888-8763ea6722ec tempest-ServersOnMultiNodesTest-83775055 tempest-ServersOnMultiNodesTest-83775055-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:45.621087 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-57b6bb1f-463e-4399-9888-8763ea6722ec tempest-ServersOnMultiNodesTest-83775055 tempest-ServersOnMultiNodesTest-83775055-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:45.621902 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-57b6bb1f-463e-4399-9888-8763ea6722ec tempest-ServersOnMultiNodesTest-83775055 tempest-ServersOnMultiNodesTest-83775055-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:45.622545 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-57b6bb1f-463e-4399-9888-8763ea6722ec tempest-ServersOnMultiNodesTest-83775055 tempest-ServersOnMultiNodesTest-83775055-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:45.626392 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-8fbcc037-0ba0-4f2a-919c-baa341be5332 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:45.627495 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-8fbcc037-0ba0-4f2a-919c-baa341be5332 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: f98a9762-a67c-4e95-85b8-727c81bdb03a] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:45.636956 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8fbcc037-0ba0-4f2a-919c-baa341be5332 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:45.640266 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8fbcc037-0ba0-4f2a-919c-baa341be5332 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:45.642081 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8fbcc037-0ba0-4f2a-919c-baa341be5332 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:45.643200 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8fbcc037-0ba0-4f2a-919c-baa341be5332 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:46.250093 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-57b6bb1f-463e-4399-9888-8763ea6722ec tempest-ServersOnMultiNodesTest-83775055 tempest-ServersOnMultiNodesTest-83775055-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/b941b324-a54d-446f-bf92-570a80ba2152" status: 200 len: 1617 microversion: 2.1 time: 0.681282 Jul 16 21:10:46.258324 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 377/757] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:10:45 2022] GET /compute/v2.1/servers/b941b324-a54d-446f-bf92-570a80ba2152 => generated 1617 bytes in 690 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:10:46.267244 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-1c714fd3-4a8d-4ae7-a1cb-f5265ed06538 tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:46.272932 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-1c714fd3-4a8d-4ae7-a1cb-f5265ed06538 tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] [instance: cf53f7cd-deae-4b34-bf8e-3afef98b2ec6] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:46.286951 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1c714fd3-4a8d-4ae7-a1cb-f5265ed06538 tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:46.287755 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1c714fd3-4a8d-4ae7-a1cb-f5265ed06538 tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:46.288783 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1c714fd3-4a8d-4ae7-a1cb-f5265ed06538 tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:46.289416 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1c714fd3-4a8d-4ae7-a1cb-f5265ed06538 tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:46.391713 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-8fbcc037-0ba0-4f2a-919c-baa341be5332 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b68dacc4b8463ba07fae6372131117', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55a19b7a4682457a9132f5fc80110712', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:46.402722 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-8fbcc037-0ba0-4f2a-919c-baa341be5332 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "GET /compute/v2.1/servers/f98a9762-a67c-4e95-85b8-727c81bdb03a" status: 200 len: 1473 microversion: 2.1 time: 0.788588 Jul 16 21:10:46.407972 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 381/758] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:10:45 2022] GET /compute/v2.1/servers/f98a9762-a67c-4e95-85b8-727c81bdb03a => generated 1473 bytes in 794 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:10:46.428360 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-65a073b4-b2c0-4b96-b91d-29560e60007d tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:46.434078 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-65a073b4-b2c0-4b96-b91d-29560e60007d tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] [instance: 4ee3ffbe-c16a-4847-a343-469c645e4b25] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:46.447420 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-65a073b4-b2c0-4b96-b91d-29560e60007d tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:46.448291 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-65a073b4-b2c0-4b96-b91d-29560e60007d tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:46.449349 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-65a073b4-b2c0-4b96-b91d-29560e60007d tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:46.454112 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-65a073b4-b2c0-4b96-b91d-29560e60007d tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:46.494949 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.objects.instance [None req-1c714fd3-4a8d-4ae7-a1cb-f5265ed06538 tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] Lazy-loading 'services' on Instance uuid cf53f7cd-deae-4b34-bf8e-3afef98b2ec6 {{(pid=138680) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:10:46.574244 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-1c714fd3-4a8d-4ae7-a1cb-f5265ed06538 tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/cf53f7cd-deae-4b34-bf8e-3afef98b2ec6" status: 200 len: 1929 microversion: 2.75 time: 0.311548 Jul 16 21:10:46.574244 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 378/759] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:10:46 2022] GET /compute/v2.1/servers/cf53f7cd-deae-4b34-bf8e-3afef98b2ec6 => generated 1929 bytes in 314 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:10:46.597269 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-462b97f7-29b6-47c0-bee8-42392dae519f tempest-SecurityGroupRulesTestJSON-1769504197 tempest-SecurityGroupRulesTestJSON-1769504197-project] Action: 'create', calling method: .version_select at 0x7fdd9d267550>, body: {"security_group": {"name": "tempest-SecurityGroupRulesTestJSON-securitygroup-593360985", "description": "tempest-description-1772652568"}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:10:46.783172 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-65a073b4-b2c0-4b96-b91d-29560e60007d tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a69f9dee03d43beaafa966216b9e289', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9b769bf8f3ea4e368bf68de1df6538f9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:46.794243 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-65a073b4-b2c0-4b96-b91d-29560e60007d tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] 10.222.0.16 "GET /compute/v2.1/servers/4ee3ffbe-c16a-4847-a343-469c645e4b25" status: 200 len: 1504 microversion: 2.1 time: 0.384588 Jul 16 21:10:46.795338 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 382/760] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:10:46 2022] GET /compute/v2.1/servers/4ee3ffbe-c16a-4847-a343-469c645e4b25 => generated 1504 bytes in 386 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:10:46.812954 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-c98481f5-6aa0-4152-b50e-4d8dca2207bc tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:46.813951 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-c98481f5-6aa0-4152-b50e-4d8dca2207bc tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] [instance: de210dbd-1398-413f-9a0f-63cae300bd03] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:46.822404 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c98481f5-6aa0-4152-b50e-4d8dca2207bc tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:46.823317 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c98481f5-6aa0-4152-b50e-4d8dca2207bc tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:46.824356 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c98481f5-6aa0-4152-b50e-4d8dca2207bc tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:46.825463 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c98481f5-6aa0-4152-b50e-4d8dca2207bc tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:46.835141 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:10:46.924645 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-c98481f5-6aa0-4152-b50e-4d8dca2207bc tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:46.942916 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-c98481f5-6aa0-4152-b50e-4d8dca2207bc tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:46.944640 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-c98481f5-6aa0-4152-b50e-4d8dca2207bc tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:46.945848 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-c98481f5-6aa0-4152-b50e-4d8dca2207bc tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] 10.222.0.16 "GET /compute/v2.1/servers/de210dbd-1398-413f-9a0f-63cae300bd03" status: 200 len: 1332 microversion: 2.25 time: 0.138305 Jul 16 21:10:46.946446 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 383/761] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:10:46 2022] GET /compute/v2.1/servers/de210dbd-1398-413f-9a0f-63cae300bd03 => generated 1332 bytes in 140 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:10:46.952000 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-5d93270f-8059-4cc4-a57e-753e6539efcf tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServersOnMultiNodesTest-server-810182439", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452"}, "os:scheduler_hints": {"different_host": ["148d5ad9-2900-4c2b-909b-aa446757e33d"]}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:10:47.017532 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.network.neutron [None req-5d93270f-8059-4cc4-a57e-753e6539efcf tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] validate_networks() for None {{(pid=138681) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 16 21:10:47.301574 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-462b97f7-29b6-47c0-bee8-42392dae519f tempest-SecurityGroupRulesTestJSON-1769504197 tempest-SecurityGroupRulesTestJSON-1769504197-project] 10.222.0.16 "POST /compute/v2.1/os-security-groups" status: 200 len: 247 microversion: 2.1 time: 0.726797 Jul 16 21:10:47.302748 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 379/762] 10.222.0.16 () {60 vars in 1179 bytes} [Sat Jul 16 21:10:46 2022] POST /compute/v2.1/os-security-groups => generated 247 bytes in 729 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:10:47.312096 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-abcf2976-4ec5-4651-83da-154084e8eba7 req-7a201a67-a914-4729-b564-71dd0d41ef73 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "f98a9762-a67c-4e95-85b8-727c81bdb03a", "name": "network-vif-unplugged", "status": "completed", "tag": "b0aae229-268b-497f-8da0-36ccb43a67bd"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:10:47.323112 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-abcf2976-4ec5-4651-83da-154084e8eba7 req-7a201a67-a914-4729-b564-71dd0d41ef73 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:47.325159 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-abcf2976-4ec5-4651-83da-154084e8eba7 req-7a201a67-a914-4729-b564-71dd0d41ef73 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:47.348362 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-abcf2976-4ec5-4651-83da-154084e8eba7 req-7a201a67-a914-4729-b564-71dd0d41ef73 service nova] Creating event network-vif-unplugged:b0aae229-268b-497f-8da0-36ccb43a67bd for instance f98a9762-a67c-4e95-85b8-727c81bdb03a on nch1-master-1 Jul 16 21:10:47.356089 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [req-abcf2976-4ec5-4651-83da-154084e8eba7 req-7a201a67-a914-4729-b564-71dd0d41ef73 service nova] Instance f98a9762-a67c-4e95-85b8-727c81bdb03a is migrating within the same cell, copying events to all relevant hosts: {'nch1-master-1'} {{(pid=138680) _get_relevant_hosts /opt/stack/nova/nova/compute/api.py:5807}} Jul 16 21:10:47.361839 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-5d93270f-8059-4cc4-a57e-753e6539efcf tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Flavor limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 16 21:10:47.362377 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-5d93270f-8059-4cc4-a57e-753e6539efcf tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Image limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 16 21:10:47.362819 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-abcf2976-4ec5-4651-83da-154084e8eba7 req-7a201a67-a914-4729-b564-71dd0d41ef73 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.058676 Jul 16 21:10:47.362925 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-5d93270f-8059-4cc4-a57e-753e6539efcf tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Flavor pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 16 21:10:47.363718 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 380/763] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:10:47 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 60 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:10:47.367300 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-5d93270f-8059-4cc4-a57e-753e6539efcf tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Image pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 16 21:10:47.368155 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-5d93270f-8059-4cc4-a57e-753e6539efcf tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 16 21:10:47.371346 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-1885c6d7-f740-468f-977a-c23512ed96a9 tempest-SecurityGroupRulesTestJSON-1769504197 tempest-SecurityGroupRulesTestJSON-1769504197-project] Action: 'create', calling method: .version_select at 0x7fdd9d242940>, body: {"security_group_rule": {"parent_group_id": "57736166-7c94-4eb2-abd1-a804b9128fc1", "ip_protocol": "tcp", "from_port": 22, "to_port": 22, "group_id": "6e6c37e2-9637-49bf-93cf-b4c44d8669d8"}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:10:47.381674 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5d93270f-8059-4cc4-a57e-753e6539efcf tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:47.382500 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5d93270f-8059-4cc4-a57e-753e6539efcf tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:47.384059 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5d93270f-8059-4cc4-a57e-753e6539efcf tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:47.384877 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5d93270f-8059-4cc4-a57e-753e6539efcf tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:47.408832 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-5d93270f-8059-4cc4-a57e-753e6539efcf tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Getting quotas for project 649babe54d104436af461f1ee4767001. Resources: {'ram', 'instances', 'cores'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:10:47.415269 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_api:SELECT=93,nova_api:INSERT=5,nova_api:UPDATE=2 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:10:47.416601 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-5d93270f-8059-4cc4-a57e-753e6539efcf tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Getting quotas for user abeb7020019d41a18de42956d706a011 and project 649babe54d104436af461f1ee4767001. Resources: {'ram', 'instances', 'cores'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:10:47.436695 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-5d93270f-8059-4cc4-a57e-753e6539efcf tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Going to run 1 instances... {{(pid=138681) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 16 21:10:47.441831 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-5d93270f-8059-4cc4-a57e-753e6539efcf tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] [instance: a4a0aaec-dc64-400c-b3f0-ec61fa740b8d] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138681) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 16 21:10:47.592474 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell0:SELECT=56 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:10:47.773895 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-5d93270f-8059-4cc4-a57e-753e6539efcf tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.825806 Jul 16 21:10:47.778446 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 384/764] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 21:10:46 2022] POST /compute/v2.1/servers => generated 380 bytes in 830 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 16 21:10:47.798789 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-5fdd4c49-758b-4baf-8fd0-517574706423 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:47.799619 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-5fdd4c49-758b-4baf-8fd0-517574706423 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: f98a9762-a67c-4e95-85b8-727c81bdb03a] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:47.817177 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5fdd4c49-758b-4baf-8fd0-517574706423 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:47.817626 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5fdd4c49-758b-4baf-8fd0-517574706423 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:47.823283 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5fdd4c49-758b-4baf-8fd0-517574706423 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:47.825928 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5fdd4c49-758b-4baf-8fd0-517574706423 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:48.048014 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell0:SELECT=16 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:10:48.260824 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-5fdd4c49-758b-4baf-8fd0-517574706423 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b68dacc4b8463ba07fae6372131117', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55a19b7a4682457a9132f5fc80110712', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:48.271035 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-5fdd4c49-758b-4baf-8fd0-517574706423 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "GET /compute/v2.1/servers/f98a9762-a67c-4e95-85b8-727c81bdb03a" status: 200 len: 1473 microversion: 2.1 time: 0.487088 Jul 16 21:10:48.272650 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 385/765] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:10:47 2022] GET /compute/v2.1/servers/f98a9762-a67c-4e95-85b8-727c81bdb03a => generated 1473 bytes in 494 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:10:48.285003 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-dac73a46-2dbc-4695-b259-8c0cf07287b1 tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:48.287252 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-dac73a46-2dbc-4695-b259-8c0cf07287b1 tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] [instance: cf53f7cd-deae-4b34-bf8e-3afef98b2ec6] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:48.295628 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-dac73a46-2dbc-4695-b259-8c0cf07287b1 tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:48.296797 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-dac73a46-2dbc-4695-b259-8c0cf07287b1 tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:48.298036 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-dac73a46-2dbc-4695-b259-8c0cf07287b1 tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:48.298642 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-dac73a46-2dbc-4695-b259-8c0cf07287b1 tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:48.317688 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-1885c6d7-f740-468f-977a-c23512ed96a9 tempest-SecurityGroupRulesTestJSON-1769504197 tempest-SecurityGroupRulesTestJSON-1769504197-project] 10.222.0.16 "POST /compute/v2.1/os-security-group-rules" status: 200 len: 330 microversion: 2.1 time: 0.952443 Jul 16 21:10:48.321490 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 381/766] 10.222.0.16 () {60 vars in 1194 bytes} [Sat Jul 16 21:10:47 2022] POST /compute/v2.1/os-security-group-rules => generated 330 bytes in 954 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:10:48.323486 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-c75be00c-1684-4036-b7a4-0877b244e37b tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:48.324986 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-c75be00c-1684-4036-b7a4-0877b244e37b tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] [instance: a4a0aaec-dc64-400c-b3f0-ec61fa740b8d] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:48.447895 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.objects.instance [None req-dac73a46-2dbc-4695-b259-8c0cf07287b1 tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] Lazy-loading 'services' on Instance uuid cf53f7cd-deae-4b34-bf8e-3afef98b2ec6 {{(pid=138681) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:10:48.477438 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-c75be00c-1684-4036-b7a4-0877b244e37b tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'abeb7020019d41a18de42956d706a011', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '649babe54d104436af461f1ee4767001', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:48.478824 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-dac73a46-2dbc-4695-b259-8c0cf07287b1 tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/cf53f7cd-deae-4b34-bf8e-3afef98b2ec6" status: 200 len: 1929 microversion: 2.75 time: 0.204024 Jul 16 21:10:48.479594 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 386/767] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:10:48 2022] GET /compute/v2.1/servers/cf53f7cd-deae-4b34-bf8e-3afef98b2ec6 => generated 1929 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:10:48.487491 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-c75be00c-1684-4036-b7a4-0877b244e37b tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] 10.222.0.16 "GET /compute/v2.1/servers/a4a0aaec-dc64-400c-b3f0-ec61fa740b8d" status: 200 len: 1216 microversion: 2.1 time: 0.167749 Jul 16 21:10:48.488077 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 382/768] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:10:48 2022] GET /compute/v2.1/servers/a4a0aaec-dc64-400c-b3f0-ec61fa740b8d => generated 1216 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:10:48.498934 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-10afc17d-d2d1-4cbc-abda-b32166d0ac19 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:48.498934 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-10afc17d-d2d1-4cbc-abda-b32166d0ac19 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] [instance: 4ee3ffbe-c16a-4847-a343-469c645e4b25] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:48.504009 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-50687158-ee01-4f41-b699-7a8e8bc81722 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:48.505037 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-50687158-ee01-4f41-b699-7a8e8bc81722 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] [instance: de210dbd-1398-413f-9a0f-63cae300bd03] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:48.509427 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-10afc17d-d2d1-4cbc-abda-b32166d0ac19 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:48.510086 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-10afc17d-d2d1-4cbc-abda-b32166d0ac19 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:48.512755 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-10afc17d-d2d1-4cbc-abda-b32166d0ac19 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:48.513295 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-50687158-ee01-4f41-b699-7a8e8bc81722 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:48.514946 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-50687158-ee01-4f41-b699-7a8e8bc81722 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:48.515971 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-50687158-ee01-4f41-b699-7a8e8bc81722 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:48.516600 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-50687158-ee01-4f41-b699-7a8e8bc81722 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:48.525057 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-10afc17d-d2d1-4cbc-abda-b32166d0ac19 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.013s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:48.805995 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-50687158-ee01-4f41-b699-7a8e8bc81722 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:48.817355 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-50687158-ee01-4f41-b699-7a8e8bc81722 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:48.819074 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-50687158-ee01-4f41-b699-7a8e8bc81722 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:48.820473 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-50687158-ee01-4f41-b699-7a8e8bc81722 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] 10.222.0.16 "GET /compute/v2.1/servers/de210dbd-1398-413f-9a0f-63cae300bd03" status: 200 len: 1362 microversion: 2.25 time: 0.321203 Jul 16 21:10:48.836708 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 383/769] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:10:48 2022] GET /compute/v2.1/servers/de210dbd-1398-413f-9a0f-63cae300bd03 => generated 1362 bytes in 333 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:10:48.840012 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-10afc17d-d2d1-4cbc-abda-b32166d0ac19 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a69f9dee03d43beaafa966216b9e289', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9b769bf8f3ea4e368bf68de1df6538f9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:48.841964 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-a7d3c689-551e-4a1d-bbd7-009040bd6fb2 tempest-SecurityGroupRulesTestJSON-1769504197 tempest-SecurityGroupRulesTestJSON-1769504197-project] Action: 'create', calling method: .version_select at 0x7fdd9d1edd30>, body: {"security_group": {"name": "tempest-SecurityGroupRulesTestJSON-securitygroup-1176322968", "description": "tempest-description-1655921802"}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:10:48.866656 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-10afc17d-d2d1-4cbc-abda-b32166d0ac19 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] 10.222.0.16 "GET /compute/v2.1/servers/4ee3ffbe-c16a-4847-a343-469c645e4b25" status: 200 len: 1504 microversion: 2.1 time: 0.370815 Jul 16 21:10:48.866656 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 387/770] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:10:48 2022] GET /compute/v2.1/servers/4ee3ffbe-c16a-4847-a343-469c645e4b25 => generated 1504 bytes in 376 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:10:49.303032 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-480c7639-7064-4106-b5bd-0d628d0f7c0b tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:49.303661 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-480c7639-7064-4106-b5bd-0d628d0f7c0b tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: f98a9762-a67c-4e95-85b8-727c81bdb03a] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:49.323102 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-480c7639-7064-4106-b5bd-0d628d0f7c0b tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:49.323921 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-480c7639-7064-4106-b5bd-0d628d0f7c0b tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:49.324957 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-480c7639-7064-4106-b5bd-0d628d0f7c0b tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:49.330183 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-480c7639-7064-4106-b5bd-0d628d0f7c0b tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:49.370409 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-a7d3c689-551e-4a1d-bbd7-009040bd6fb2 tempest-SecurityGroupRulesTestJSON-1769504197 tempest-SecurityGroupRulesTestJSON-1769504197-project] 10.222.0.16 "POST /compute/v2.1/os-security-groups" status: 200 len: 248 microversion: 2.1 time: 0.523887 Jul 16 21:10:49.370409 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 384/771] 10.222.0.16 () {60 vars in 1179 bytes} [Sat Jul 16 21:10:48 2022] POST /compute/v2.1/os-security-groups => generated 248 bytes in 526 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:10:49.376606 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-d4ba4bfa-13c4-4bbb-a29e-99d363127363 tempest-SecurityGroupRulesTestJSON-1769504197 tempest-SecurityGroupRulesTestJSON-1769504197-project] Action: 'create', calling method: .version_select at 0x7fdd9d242a60>, body: {"security_group": {"name": "tempest-SecurityGroupRulesTestJSON-securitygroup-137031006", "description": "tempest-description-507396756"}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:10:49.537975 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-480c7639-7064-4106-b5bd-0d628d0f7c0b tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b68dacc4b8463ba07fae6372131117', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55a19b7a4682457a9132f5fc80110712', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:49.549470 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-480c7639-7064-4106-b5bd-0d628d0f7c0b tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "GET /compute/v2.1/servers/f98a9762-a67c-4e95-85b8-727c81bdb03a" status: 200 len: 1470 microversion: 2.1 time: 0.264361 Jul 16 21:10:49.550156 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 388/772] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:10:49 2022] GET /compute/v2.1/servers/f98a9762-a67c-4e95-85b8-727c81bdb03a => generated 1470 bytes in 266 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:10:49.577660 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-7e857e74-f178-4ce0-8148-9280eb801d18 tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:49.578516 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-7e857e74-f178-4ce0-8148-9280eb801d18 tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] [instance: cf53f7cd-deae-4b34-bf8e-3afef98b2ec6] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:49.599817 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7e857e74-f178-4ce0-8148-9280eb801d18 tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:49.600706 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7e857e74-f178-4ce0-8148-9280eb801d18 tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:49.601934 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7e857e74-f178-4ce0-8148-9280eb801d18 tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:49.602500 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7e857e74-f178-4ce0-8148-9280eb801d18 tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:49.773170 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.objects.instance [None req-7e857e74-f178-4ce0-8148-9280eb801d18 tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] Lazy-loading 'services' on Instance uuid cf53f7cd-deae-4b34-bf8e-3afef98b2ec6 {{(pid=138681) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:10:49.805192 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-7e857e74-f178-4ce0-8148-9280eb801d18 tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/cf53f7cd-deae-4b34-bf8e-3afef98b2ec6" status: 200 len: 1929 microversion: 2.75 time: 0.242289 Jul 16 21:10:49.806247 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 389/773] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:10:49 2022] GET /compute/v2.1/servers/cf53f7cd-deae-4b34-bf8e-3afef98b2ec6 => generated 1929 bytes in 253 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:10:49.814516 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-6727e94f-636c-4eac-8a68-15d060530db4 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:49.815293 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-6727e94f-636c-4eac-8a68-15d060530db4 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] [instance: a4a0aaec-dc64-400c-b3f0-ec61fa740b8d] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:49.825679 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6727e94f-636c-4eac-8a68-15d060530db4 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:49.826845 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6727e94f-636c-4eac-8a68-15d060530db4 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:49.828883 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6727e94f-636c-4eac-8a68-15d060530db4 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:49.843662 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6727e94f-636c-4eac-8a68-15d060530db4 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.015s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:49.975238 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-6727e94f-636c-4eac-8a68-15d060530db4 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'abeb7020019d41a18de42956d706a011', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '649babe54d104436af461f1ee4767001', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:49.988606 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-6727e94f-636c-4eac-8a68-15d060530db4 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] 10.222.0.16 "GET /compute/v2.1/servers/a4a0aaec-dc64-400c-b3f0-ec61fa740b8d" status: 200 len: 1268 microversion: 2.1 time: 0.179854 Jul 16 21:10:49.989522 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 390/774] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:10:49 2022] GET /compute/v2.1/servers/a4a0aaec-dc64-400c-b3f0-ec61fa740b8d => generated 1268 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:10:49.996914 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-d9e9968d-eaad-4416-859c-4de15247c604 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Action: 'action', calling method: >, body: {"confirmResize": {}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:10:49.998366 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-d9e9968d-eaad-4416-859c-4de15247c604 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: f98a9762-a67c-4e95-85b8-727c81bdb03a] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:50.006749 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d9e9968d-eaad-4416-859c-4de15247c604 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:50.007665 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d9e9968d-eaad-4416-859c-4de15247c604 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:50.008838 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d9e9968d-eaad-4416-859c-4de15247c604 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:50.009569 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d9e9968d-eaad-4416-859c-4de15247c604 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:50.021760 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-d4ba4bfa-13c4-4bbb-a29e-99d363127363 tempest-SecurityGroupRulesTestJSON-1769504197 tempest-SecurityGroupRulesTestJSON-1769504197-project] 10.222.0.16 "POST /compute/v2.1/os-security-groups" status: 200 len: 246 microversion: 2.1 time: 0.650004 Jul 16 21:10:50.024289 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 385/775] 10.222.0.16 () {60 vars in 1179 bytes} [Sat Jul 16 21:10:49 2022] POST /compute/v2.1/os-security-groups => generated 246 bytes in 653 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:10:50.030829 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-116b40c5-eb86-4738-b50c-664f11f84580 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:50.031942 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-116b40c5-eb86-4738-b50c-664f11f84580 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] [instance: de210dbd-1398-413f-9a0f-63cae300bd03] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:50.043960 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-116b40c5-eb86-4738-b50c-664f11f84580 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:50.045352 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-116b40c5-eb86-4738-b50c-664f11f84580 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:50.051298 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-116b40c5-eb86-4738-b50c-664f11f84580 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:50.052288 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-116b40c5-eb86-4738-b50c-664f11f84580 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:50.268796 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-116b40c5-eb86-4738-b50c-664f11f84580 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:50.286043 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-116b40c5-eb86-4738-b50c-664f11f84580 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:50.290322 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-116b40c5-eb86-4738-b50c-664f11f84580 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:50.290322 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-116b40c5-eb86-4738-b50c-664f11f84580 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] 10.222.0.16 "GET /compute/v2.1/servers/de210dbd-1398-413f-9a0f-63cae300bd03" status: 200 len: 1362 microversion: 2.25 time: 0.264367 Jul 16 21:10:50.291575 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 386/776] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:10:50 2022] GET /compute/v2.1/servers/de210dbd-1398-413f-9a0f-63cae300bd03 => generated 1362 bytes in 266 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:10:50.299514 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-efceba40-f39d-4b02-af78-d71e73a9d09e tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:50.300978 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-efceba40-f39d-4b02-af78-d71e73a9d09e tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] [instance: 4ee3ffbe-c16a-4847-a343-469c645e4b25] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:50.306180 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-d9e9968d-eaad-4416-859c-4de15247c604 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "POST /compute/v2.1/servers/f98a9762-a67c-4e95-85b8-727c81bdb03a/action" status: 204 len: 0 microversion: 2.1 time: 0.313571 Jul 16 21:10:50.310340 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 391/777] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 21:10:49 2022] POST /compute/v2.1/servers/f98a9762-a67c-4e95-85b8-727c81bdb03a/action => generated 0 bytes in 317 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 16 21:10:50.315531 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-83f520ae-d2f9-4212-9387-ae3384d945e1 tempest-SecurityGroupRulesTestJSON-1769504197 tempest-SecurityGroupRulesTestJSON-1769504197-project] Action: 'create', calling method: .version_select at 0x7fdd9d267160>, body: {"security_group_rule": {"parent_group_id": "3e52cc49-94d1-4f54-91ea-99e2789ec1ca", "ip_protocol": "tcp", "from_port": 22, "to_port": 22, "group_id": "91acad9f-4045-46df-87c9-831806306000"}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:10:50.317533 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-efceba40-f39d-4b02-af78-d71e73a9d09e tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:50.319044 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-efceba40-f39d-4b02-af78-d71e73a9d09e tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:50.320206 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-efceba40-f39d-4b02-af78-d71e73a9d09e tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:50.321245 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-efceba40-f39d-4b02-af78-d71e73a9d09e tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:50.544719 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-efceba40-f39d-4b02-af78-d71e73a9d09e tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a69f9dee03d43beaafa966216b9e289', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9b769bf8f3ea4e368bf68de1df6538f9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:50.554150 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-efceba40-f39d-4b02-af78-d71e73a9d09e tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] 10.222.0.16 "GET /compute/v2.1/servers/4ee3ffbe-c16a-4847-a343-469c645e4b25" status: 200 len: 1504 microversion: 2.1 time: 0.260684 Jul 16 21:10:50.554722 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 387/778] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:10:50 2022] GET /compute/v2.1/servers/4ee3ffbe-c16a-4847-a343-469c645e4b25 => generated 1504 bytes in 262 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:10:50.560523 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-344773d4-83e5-48b0-9e33-665ecc5750f8 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:50.562283 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-344773d4-83e5-48b0-9e33-665ecc5750f8 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: f98a9762-a67c-4e95-85b8-727c81bdb03a] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:50.569394 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-344773d4-83e5-48b0-9e33-665ecc5750f8 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:50.570140 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-344773d4-83e5-48b0-9e33-665ecc5750f8 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:50.571002 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-344773d4-83e5-48b0-9e33-665ecc5750f8 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:50.571840 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-344773d4-83e5-48b0-9e33-665ecc5750f8 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:50.770437 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-344773d4-83e5-48b0-9e33-665ecc5750f8 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b68dacc4b8463ba07fae6372131117', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55a19b7a4682457a9132f5fc80110712', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:50.781974 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-344773d4-83e5-48b0-9e33-665ecc5750f8 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "GET /compute/v2.1/servers/f98a9762-a67c-4e95-85b8-727c81bdb03a" status: 200 len: 1470 microversion: 2.1 time: 0.225648 Jul 16 21:10:50.797807 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 388/779] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:10:50 2022] GET /compute/v2.1/servers/f98a9762-a67c-4e95-85b8-727c81bdb03a => generated 1470 bytes in 242 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:10:50.811136 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-a138d3d5-7516-4d7e-84f7-74aa43705159 req-70bbec67-47fb-4cfe-af6c-d40f94332da1 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "f98a9762-a67c-4e95-85b8-727c81bdb03a", "name": "network-vif-plugged", "status": "completed", "tag": "b0aae229-268b-497f-8da0-36ccb43a67bd"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:10:50.823221 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_api:SELECT=99,nova_api:INSERT=1 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:10:50.829187 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-a138d3d5-7516-4d7e-84f7-74aa43705159 req-70bbec67-47fb-4cfe-af6c-d40f94332da1 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:50.830003 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-a138d3d5-7516-4d7e-84f7-74aa43705159 req-70bbec67-47fb-4cfe-af6c-d40f94332da1 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:50.845272 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-a138d3d5-7516-4d7e-84f7-74aa43705159 req-70bbec67-47fb-4cfe-af6c-d40f94332da1 service nova] Creating event network-vif-plugged:b0aae229-268b-497f-8da0-36ccb43a67bd for instance f98a9762-a67c-4e95-85b8-727c81bdb03a on nch1-master-1 Jul 16 21:10:50.852832 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [req-a138d3d5-7516-4d7e-84f7-74aa43705159 req-70bbec67-47fb-4cfe-af6c-d40f94332da1 service nova] Instance f98a9762-a67c-4e95-85b8-727c81bdb03a is migrating within the same cell, copying events to all relevant hosts: {'nch1-master-1'} {{(pid=138680) _get_relevant_hosts /opt/stack/nova/nova/compute/api.py:5807}} Jul 16 21:10:50.860598 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-a138d3d5-7516-4d7e-84f7-74aa43705159 req-70bbec67-47fb-4cfe-af6c-d40f94332da1 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.061118 Jul 16 21:10:50.865626 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 389/780] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:10:50 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 67 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:10:50.874695 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-ddaeef13-8756-411b-955c-faac4450cdb8 req-568ec9b2-0375-48b3-83fb-fc312a094525 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "4ee3ffbe-c16a-4847-a343-469c645e4b25", "name": "network-vif-unplugged", "status": "completed", "tag": "f54ad45e-ef5d-4b3e-ae39-d46f5d43bc88"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:10:50.888175 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-ddaeef13-8756-411b-955c-faac4450cdb8 req-568ec9b2-0375-48b3-83fb-fc312a094525 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:50.888175 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-ddaeef13-8756-411b-955c-faac4450cdb8 req-568ec9b2-0375-48b3-83fb-fc312a094525 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:50.917794 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-ddaeef13-8756-411b-955c-faac4450cdb8 req-568ec9b2-0375-48b3-83fb-fc312a094525 service nova] Creating event network-vif-unplugged:f54ad45e-ef5d-4b3e-ae39-d46f5d43bc88 for instance 4ee3ffbe-c16a-4847-a343-469c645e4b25 on nch1-master-1 Jul 16 21:10:50.924363 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-ddaeef13-8756-411b-955c-faac4450cdb8 req-568ec9b2-0375-48b3-83fb-fc312a094525 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.054282 Jul 16 21:10:50.924971 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 390/781] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:10:50 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:10:50.930931 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-53b5feba-cb77-409c-8472-efdf808341e4 tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:50.931919 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-53b5feba-cb77-409c-8472-efdf808341e4 tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] [instance: cf53f7cd-deae-4b34-bf8e-3afef98b2ec6] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:50.938953 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-53b5feba-cb77-409c-8472-efdf808341e4 tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:50.939529 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-53b5feba-cb77-409c-8472-efdf808341e4 tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:50.940701 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-53b5feba-cb77-409c-8472-efdf808341e4 tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:50.946768 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-53b5feba-cb77-409c-8472-efdf808341e4 tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:51.080274 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.objects.instance [None req-53b5feba-cb77-409c-8472-efdf808341e4 tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] Lazy-loading 'services' on Instance uuid cf53f7cd-deae-4b34-bf8e-3afef98b2ec6 {{(pid=138680) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:10:51.087248 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-83f520ae-d2f9-4212-9387-ae3384d945e1 tempest-SecurityGroupRulesTestJSON-1769504197 tempest-SecurityGroupRulesTestJSON-1769504197-project] 10.222.0.16 "POST /compute/v2.1/os-security-group-rules" status: 200 len: 330 microversion: 2.1 time: 0.778458 Jul 16 21:10:51.092878 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 392/782] 10.222.0.16 () {60 vars in 1194 bytes} [Sat Jul 16 21:10:50 2022] POST /compute/v2.1/os-security-group-rules => generated 330 bytes in 785 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:10:51.100074 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-c2e8931b-8079-4c06-9fa9-bb6ca0e3a790 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:51.101058 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-c2e8931b-8079-4c06-9fa9-bb6ca0e3a790 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] [instance: a4a0aaec-dc64-400c-b3f0-ec61fa740b8d] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:51.104746 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-53b5feba-cb77-409c-8472-efdf808341e4 tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/cf53f7cd-deae-4b34-bf8e-3afef98b2ec6" status: 200 len: 1929 microversion: 2.75 time: 0.178189 Jul 16 21:10:51.105432 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 391/783] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:10:50 2022] GET /compute/v2.1/servers/cf53f7cd-deae-4b34-bf8e-3afef98b2ec6 => generated 1929 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:10:51.111290 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c2e8931b-8079-4c06-9fa9-bb6ca0e3a790 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:51.111604 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-40aaf6de-d6de-4ecb-8c96-ef72bcf71f22 tempest-SecurityGroupRulesTestJSON-1769504197 tempest-SecurityGroupRulesTestJSON-1769504197-project] Calling method '.version_select at 0x7fdd9d4059d0>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:51.114655 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c2e8931b-8079-4c06-9fa9-bb6ca0e3a790 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:51.114655 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c2e8931b-8079-4c06-9fa9-bb6ca0e3a790 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:51.114655 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c2e8931b-8079-4c06-9fa9-bb6ca0e3a790 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:51.213032 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-c2e8931b-8079-4c06-9fa9-bb6ca0e3a790 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'abeb7020019d41a18de42956d706a011', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '649babe54d104436af461f1ee4767001', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:51.224859 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-c2e8931b-8079-4c06-9fa9-bb6ca0e3a790 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] 10.222.0.16 "GET /compute/v2.1/servers/a4a0aaec-dc64-400c-b3f0-ec61fa740b8d" status: 200 len: 1268 microversion: 2.1 time: 0.131538 Jul 16 21:10:51.225829 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 393/784] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:10:51 2022] GET /compute/v2.1/servers/a4a0aaec-dc64-400c-b3f0-ec61fa740b8d => generated 1268 bytes in 133 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:10:51.315341 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-ced4e688-2095-4f63-8eae-d6d93af92acf tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:51.316115 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-ced4e688-2095-4f63-8eae-d6d93af92acf tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] [instance: de210dbd-1398-413f-9a0f-63cae300bd03] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:51.331191 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ced4e688-2095-4f63-8eae-d6d93af92acf tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:51.333045 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ced4e688-2095-4f63-8eae-d6d93af92acf tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:51.334486 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ced4e688-2095-4f63-8eae-d6d93af92acf tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:51.334725 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ced4e688-2095-4f63-8eae-d6d93af92acf tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:51.546725 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-ced4e688-2095-4f63-8eae-d6d93af92acf tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:51.559505 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-ced4e688-2095-4f63-8eae-d6d93af92acf tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:51.562185 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-ced4e688-2095-4f63-8eae-d6d93af92acf tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:51.563559 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-ced4e688-2095-4f63-8eae-d6d93af92acf tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] 10.222.0.16 "GET /compute/v2.1/servers/de210dbd-1398-413f-9a0f-63cae300bd03" status: 200 len: 1362 microversion: 2.25 time: 0.256034 Jul 16 21:10:51.574596 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 394/785] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:10:51 2022] GET /compute/v2.1/servers/de210dbd-1398-413f-9a0f-63cae300bd03 => generated 1362 bytes in 258 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 16 21:10:51.581244 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-338c4799-faea-422b-993b-58b802185bbe tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:51.582136 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-338c4799-faea-422b-993b-58b802185bbe tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] [instance: 4ee3ffbe-c16a-4847-a343-469c645e4b25] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:51.590727 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-338c4799-faea-422b-993b-58b802185bbe tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:51.591409 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-338c4799-faea-422b-993b-58b802185bbe tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:51.593379 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-338c4799-faea-422b-993b-58b802185bbe tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:51.594101 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-338c4799-faea-422b-993b-58b802185bbe tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:51.605189 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-40aaf6de-d6de-4ecb-8c96-ef72bcf71f22 tempest-SecurityGroupRulesTestJSON-1769504197 tempest-SecurityGroupRulesTestJSON-1769504197-project] 10.222.0.16 "DELETE /compute/v2.1/os-security-groups/91acad9f-4045-46df-87c9-831806306000" status: 202 len: 0 microversion: 2.1 time: 0.498326 Jul 16 21:10:51.609056 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 392/786] 10.222.0.16 () {58 vars in 1271 bytes} [Sat Jul 16 21:10:51 2022] DELETE /compute/v2.1/os-security-groups/91acad9f-4045-46df-87c9-831806306000 => generated 0 bytes in 503 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 21:10:51.614805 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-f8523d2e-72c6-4e62-b9df-64468913625e tempest-SecurityGroupRulesTestJSON-1769504197 tempest-SecurityGroupRulesTestJSON-1769504197-project] Calling method '.version_select at 0x7fdd9d1ec160>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:51.908612 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-f8523d2e-72c6-4e62-b9df-64468913625e tempest-SecurityGroupRulesTestJSON-1769504197 tempest-SecurityGroupRulesTestJSON-1769504197-project] 10.222.0.16 "GET /compute/v2.1/os-security-groups/3e52cc49-94d1-4f54-91ea-99e2789ec1ca" status: 200 len: 248 microversion: 2.1 time: 0.298187 Jul 16 21:10:51.919897 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 393/787] 10.222.0.16 () {58 vars in 1268 bytes} [Sat Jul 16 21:10:51 2022] GET /compute/v2.1/os-security-groups/3e52cc49-94d1-4f54-91ea-99e2789ec1ca => generated 248 bytes in 310 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:10:51.925345 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-c4e3b2e8-efe4-42f6-88ee-34be8d46d12f tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:51.926229 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-c4e3b2e8-efe4-42f6-88ee-34be8d46d12f tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: f98a9762-a67c-4e95-85b8-727c81bdb03a] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:51.940279 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c4e3b2e8-efe4-42f6-88ee-34be8d46d12f tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:51.941010 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c4e3b2e8-efe4-42f6-88ee-34be8d46d12f tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:51.941962 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c4e3b2e8-efe4-42f6-88ee-34be8d46d12f tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:51.949571 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c4e3b2e8-efe4-42f6-88ee-34be8d46d12f tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:52.093549 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-338c4799-faea-422b-993b-58b802185bbe tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a69f9dee03d43beaafa966216b9e289', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9b769bf8f3ea4e368bf68de1df6538f9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:52.106242 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-338c4799-faea-422b-993b-58b802185bbe tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] 10.222.0.16 "GET /compute/v2.1/servers/4ee3ffbe-c16a-4847-a343-469c645e4b25" status: 200 len: 1504 microversion: 2.1 time: 0.529132 Jul 16 21:10:52.109401 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 395/788] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:10:51 2022] GET /compute/v2.1/servers/4ee3ffbe-c16a-4847-a343-469c645e4b25 => generated 1504 bytes in 534 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:10:52.125498 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-ae583635-d3f9-4d0e-a713-a0389366eee6 tempest-SecurityGroupRulesTestJSON-1769504197 tempest-SecurityGroupRulesTestJSON-1769504197-project] Action: 'create', calling method: .version_select at 0x7fdd9d257d30>, body: {"security_group": {"name": "tempest-SecurityGroupRulesTestJSON-securitygroup-1863721842", "description": "tempest-description-832614260"}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:10:52.244344 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-c4e3b2e8-efe4-42f6-88ee-34be8d46d12f tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b68dacc4b8463ba07fae6372131117', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55a19b7a4682457a9132f5fc80110712', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:52.257903 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-c4e3b2e8-efe4-42f6-88ee-34be8d46d12f tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "GET /compute/v2.1/servers/f98a9762-a67c-4e95-85b8-727c81bdb03a" status: 200 len: 1470 microversion: 2.1 time: 0.336080 Jul 16 21:10:52.263124 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 394/789] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:10:51 2022] GET /compute/v2.1/servers/f98a9762-a67c-4e95-85b8-727c81bdb03a => generated 1470 bytes in 339 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:10:52.270185 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-74c04532-4743-42dd-8a1e-790b15d936c3 tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:52.270816 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-74c04532-4743-42dd-8a1e-790b15d936c3 tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] [instance: cf53f7cd-deae-4b34-bf8e-3afef98b2ec6] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:52.295248 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-74c04532-4743-42dd-8a1e-790b15d936c3 tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:52.295355 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-74c04532-4743-42dd-8a1e-790b15d936c3 tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:52.296231 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-74c04532-4743-42dd-8a1e-790b15d936c3 tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:52.296686 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-74c04532-4743-42dd-8a1e-790b15d936c3 tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:52.502998 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:10:52.561516 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.objects.instance [None req-74c04532-4743-42dd-8a1e-790b15d936c3 tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] Lazy-loading 'services' on Instance uuid cf53f7cd-deae-4b34-bf8e-3afef98b2ec6 {{(pid=138680) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:10:52.663999 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-74c04532-4743-42dd-8a1e-790b15d936c3 tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/cf53f7cd-deae-4b34-bf8e-3afef98b2ec6" status: 200 len: 1935 microversion: 2.75 time: 0.396252 Jul 16 21:10:52.675440 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 395/790] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:10:52 2022] GET /compute/v2.1/servers/cf53f7cd-deae-4b34-bf8e-3afef98b2ec6 => generated 1935 bytes in 402 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:10:52.675440 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-f4161f80-8e8b-489a-9f27-2dde28ebf903 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:52.685416 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-f4161f80-8e8b-489a-9f27-2dde28ebf903 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] [instance: a4a0aaec-dc64-400c-b3f0-ec61fa740b8d] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:52.698473 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f4161f80-8e8b-489a-9f27-2dde28ebf903 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:52.699493 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f4161f80-8e8b-489a-9f27-2dde28ebf903 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:52.701112 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f4161f80-8e8b-489a-9f27-2dde28ebf903 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:52.702021 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f4161f80-8e8b-489a-9f27-2dde28ebf903 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:52.868230 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-f4161f80-8e8b-489a-9f27-2dde28ebf903 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'abeb7020019d41a18de42956d706a011', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '649babe54d104436af461f1ee4767001', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:52.884615 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-f4161f80-8e8b-489a-9f27-2dde28ebf903 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] 10.222.0.16 "GET /compute/v2.1/servers/a4a0aaec-dc64-400c-b3f0-ec61fa740b8d" status: 200 len: 1276 microversion: 2.1 time: 0.217414 Jul 16 21:10:52.893566 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 396/791] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:10:52 2022] GET /compute/v2.1/servers/a4a0aaec-dc64-400c-b3f0-ec61fa740b8d => generated 1276 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:10:52.905096 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-d21cc4fd-c184-4bf1-8829-a9f5ae7a3bd3 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:52.906635 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-d21cc4fd-c184-4bf1-8829-a9f5ae7a3bd3 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] [instance: de210dbd-1398-413f-9a0f-63cae300bd03] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:52.919317 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d21cc4fd-c184-4bf1-8829-a9f5ae7a3bd3 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:52.920455 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d21cc4fd-c184-4bf1-8829-a9f5ae7a3bd3 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:52.922391 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d21cc4fd-c184-4bf1-8829-a9f5ae7a3bd3 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:52.923516 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d21cc4fd-c184-4bf1-8829-a9f5ae7a3bd3 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:53.138048 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-ae583635-d3f9-4d0e-a713-a0389366eee6 tempest-SecurityGroupRulesTestJSON-1769504197 tempest-SecurityGroupRulesTestJSON-1769504197-project] 10.222.0.16 "POST /compute/v2.1/os-security-groups" status: 200 len: 247 microversion: 2.1 time: 1.026018 Jul 16 21:10:53.146458 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 396/792] 10.222.0.16 () {60 vars in 1179 bytes} [Sat Jul 16 21:10:52 2022] POST /compute/v2.1/os-security-groups => generated 247 bytes in 1027 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:10:53.146458 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-8ee42a9d-fe90-4211-bc48-29eef218da24 req-19834520-26c2-4351-b5c1-f9e5267d2268 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "de210dbd-1398-413f-9a0f-63cae300bd03", "tag": "176a187b-2abc-455b-bbcd-a736dec0c33e"}, {"name": "network-changed", "server_uuid": "4ee3ffbe-c16a-4847-a343-469c645e4b25", "tag": "f54ad45e-ef5d-4b3e-ae39-d46f5d43bc88"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:10:53.160177 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-8ee42a9d-fe90-4211-bc48-29eef218da24 req-19834520-26c2-4351-b5c1-f9e5267d2268 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:53.160628 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-8ee42a9d-fe90-4211-bc48-29eef218da24 req-19834520-26c2-4351-b5c1-f9e5267d2268 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:53.203830 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-8ee42a9d-fe90-4211-bc48-29eef218da24 req-19834520-26c2-4351-b5c1-f9e5267d2268 service nova] Creating event network-changed:176a187b-2abc-455b-bbcd-a736dec0c33e for instance de210dbd-1398-413f-9a0f-63cae300bd03 on nch2-master-1 Jul 16 21:10:53.204417 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-8ee42a9d-fe90-4211-bc48-29eef218da24 req-19834520-26c2-4351-b5c1-f9e5267d2268 service nova] Creating event network-changed:f54ad45e-ef5d-4b3e-ae39-d46f5d43bc88 for instance 4ee3ffbe-c16a-4847-a343-469c645e4b25 on nch1-master-1 Jul 16 21:10:53.214249 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-8ee42a9d-fe90-4211-bc48-29eef218da24 req-19834520-26c2-4351-b5c1-f9e5267d2268 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 346 microversion: 2.1 time: 0.074239 Jul 16 21:10:53.215851 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 397/793] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:10:53 2022] POST /compute/v2.1/os-server-external-events => generated 346 bytes in 76 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:10:53.223955 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-d21cc4fd-c184-4bf1-8829-a9f5ae7a3bd3 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:53.224160 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-db542662-6452-432f-985d-a66089fa952b tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:53.224789 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-db542662-6452-432f-985d-a66089fa952b tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] [instance: 4ee3ffbe-c16a-4847-a343-469c645e4b25] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:53.233235 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-db542662-6452-432f-985d-a66089fa952b tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:53.235436 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-db542662-6452-432f-985d-a66089fa952b tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:53.235436 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-db542662-6452-432f-985d-a66089fa952b tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:53.235436 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-db542662-6452-432f-985d-a66089fa952b tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:53.238295 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-d21cc4fd-c184-4bf1-8829-a9f5ae7a3bd3 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:53.241206 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-d21cc4fd-c184-4bf1-8829-a9f5ae7a3bd3 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:53.242713 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-d21cc4fd-c184-4bf1-8829-a9f5ae7a3bd3 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] 10.222.0.16 "GET /compute/v2.1/servers/de210dbd-1398-413f-9a0f-63cae300bd03" status: 200 len: 1362 microversion: 2.25 time: 0.345239 Jul 16 21:10:53.244202 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 397/794] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:10:52 2022] GET /compute/v2.1/servers/de210dbd-1398-413f-9a0f-63cae300bd03 => generated 1362 bytes in 347 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:10:53.249348 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-6d95ff09-019d-406b-9660-fc9886110218 tempest-SecurityGroupRulesTestJSON-1769504197 tempest-SecurityGroupRulesTestJSON-1769504197-project] Action: 'create', calling method: .version_select at 0x7fdd9d1ecdc0>, body: {"security_group_rule": {"parent_group_id": "a69b87bd-016b-430c-9f7e-92bfdc2493f2", "ip_protocol": "tcp", "from_port": 22, "to_port": 22}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:10:54.037284 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-db542662-6452-432f-985d-a66089fa952b tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a69f9dee03d43beaafa966216b9e289', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9b769bf8f3ea4e368bf68de1df6538f9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:54.046106 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-db542662-6452-432f-985d-a66089fa952b tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] 10.222.0.16 "GET /compute/v2.1/servers/4ee3ffbe-c16a-4847-a343-469c645e4b25" status: 200 len: 1504 microversion: 2.1 time: 0.827719 Jul 16 21:10:54.051113 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 398/795] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:10:53 2022] GET /compute/v2.1/servers/4ee3ffbe-c16a-4847-a343-469c645e4b25 => generated 1504 bytes in 830 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:10:54.052678 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-ba8eb122-980c-4885-9c05-fd00cf80427b tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:54.053305 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-ba8eb122-980c-4885-9c05-fd00cf80427b tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: f98a9762-a67c-4e95-85b8-727c81bdb03a] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:54.061131 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ba8eb122-980c-4885-9c05-fd00cf80427b tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:54.066159 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ba8eb122-980c-4885-9c05-fd00cf80427b tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:54.071306 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ba8eb122-980c-4885-9c05-fd00cf80427b tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:54.072241 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ba8eb122-980c-4885-9c05-fd00cf80427b tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:54.549060 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-ba8eb122-980c-4885-9c05-fd00cf80427b tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b68dacc4b8463ba07fae6372131117', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55a19b7a4682457a9132f5fc80110712', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:54.563349 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-ba8eb122-980c-4885-9c05-fd00cf80427b tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "GET /compute/v2.1/servers/f98a9762-a67c-4e95-85b8-727c81bdb03a" status: 200 len: 1470 microversion: 2.1 time: 0.515381 Jul 16 21:10:54.574124 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 399/796] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:10:54 2022] GET /compute/v2.1/servers/f98a9762-a67c-4e95-85b8-727c81bdb03a => generated 1470 bytes in 526 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:10:54.580527 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-d8fab6a7-f70e-43e8-a7a9-c1b4ad412b5f tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:54.581821 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-d8fab6a7-f70e-43e8-a7a9-c1b4ad412b5f tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] [instance: cf53f7cd-deae-4b34-bf8e-3afef98b2ec6] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:54.590060 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d8fab6a7-f70e-43e8-a7a9-c1b4ad412b5f tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:54.590525 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d8fab6a7-f70e-43e8-a7a9-c1b4ad412b5f tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:54.591428 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d8fab6a7-f70e-43e8-a7a9-c1b4ad412b5f tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:54.591872 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d8fab6a7-f70e-43e8-a7a9-c1b4ad412b5f tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:54.675648 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.objects.instance [None req-d8fab6a7-f70e-43e8-a7a9-c1b4ad412b5f tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] Lazy-loading 'services' on Instance uuid cf53f7cd-deae-4b34-bf8e-3afef98b2ec6 {{(pid=138681) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:10:54.694981 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-d8fab6a7-f70e-43e8-a7a9-c1b4ad412b5f tempest-ServersAdmin275Test-1451915950 tempest-ServersAdmin275Test-1451915950-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/cf53f7cd-deae-4b34-bf8e-3afef98b2ec6" status: 200 len: 1920 microversion: 2.75 time: 0.119365 Jul 16 21:10:54.696183 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 400/797] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:10:54 2022] GET /compute/v2.1/servers/cf53f7cd-deae-4b34-bf8e-3afef98b2ec6 => generated 1920 bytes in 121 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:10:54.701253 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-ebe10f33-884f-409e-acd9-36d1f55bf403 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:54.701908 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-ebe10f33-884f-409e-acd9-36d1f55bf403 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] [instance: a4a0aaec-dc64-400c-b3f0-ec61fa740b8d] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:54.708110 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ebe10f33-884f-409e-acd9-36d1f55bf403 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:54.708848 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ebe10f33-884f-409e-acd9-36d1f55bf403 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:54.709334 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-6d95ff09-019d-406b-9660-fc9886110218 tempest-SecurityGroupRulesTestJSON-1769504197 tempest-SecurityGroupRulesTestJSON-1769504197-project] 10.222.0.16 "POST /compute/v2.1/os-security-group-rules" status: 200 len: 232 microversion: 2.1 time: 1.450432 Jul 16 21:10:54.710040 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 398/798] 10.222.0.16 () {60 vars in 1194 bytes} [Sat Jul 16 21:10:53 2022] POST /compute/v2.1/os-security-group-rules => generated 232 bytes in 1465 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:10:54.710638 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ebe10f33-884f-409e-acd9-36d1f55bf403 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:54.711329 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ebe10f33-884f-409e-acd9-36d1f55bf403 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:54.714605 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:10:54.753797 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-6442e546-b1b4-47ec-bf40-0e46d848a66f tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:54.758044 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-6442e546-b1b4-47ec-bf40-0e46d848a66f tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] [instance: de210dbd-1398-413f-9a0f-63cae300bd03] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:54.776894 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6442e546-b1b4-47ec-bf40-0e46d848a66f tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:54.780353 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6442e546-b1b4-47ec-bf40-0e46d848a66f tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:54.782674 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6442e546-b1b4-47ec-bf40-0e46d848a66f tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:54.783432 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6442e546-b1b4-47ec-bf40-0e46d848a66f tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:55.465293 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-ebe10f33-884f-409e-acd9-36d1f55bf403 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'abeb7020019d41a18de42956d706a011', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '649babe54d104436af461f1ee4767001', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:55.491456 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-ebe10f33-884f-409e-acd9-36d1f55bf403 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] 10.222.0.16 "GET /compute/v2.1/servers/a4a0aaec-dc64-400c-b3f0-ec61fa740b8d" status: 200 len: 1316 microversion: 2.1 time: 0.793821 Jul 16 21:10:55.496761 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 401/799] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:10:54 2022] GET /compute/v2.1/servers/a4a0aaec-dc64-400c-b3f0-ec61fa740b8d => generated 1316 bytes in 800 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:10:55.503528 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-39ec3ac0-dde7-4d9f-ba06-faa091c05c90 tempest-SecurityGroupRulesTestJSON-1769504197 tempest-SecurityGroupRulesTestJSON-1769504197-project] Action: 'create', calling method: .version_select at 0x7fdd9d29e820>, body: {"security_group_rule": {"parent_group_id": "a69b87bd-016b-430c-9f7e-92bfdc2493f2", "ip_protocol": "icmp", "from_port": -1, "to_port": -1}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:10:55.513866 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-6442e546-b1b4-47ec-bf40-0e46d848a66f tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:55.526101 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-6442e546-b1b4-47ec-bf40-0e46d848a66f tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:55.527813 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-6442e546-b1b4-47ec-bf40-0e46d848a66f tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:55.529596 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-6442e546-b1b4-47ec-bf40-0e46d848a66f tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] 10.222.0.16 "GET /compute/v2.1/servers/de210dbd-1398-413f-9a0f-63cae300bd03" status: 200 len: 1362 microversion: 2.25 time: 0.805861 Jul 16 21:10:55.530160 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 399/800] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:10:54 2022] GET /compute/v2.1/servers/de210dbd-1398-413f-9a0f-63cae300bd03 => generated 1362 bytes in 820 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:10:55.536424 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-8117a5bc-ff8b-4bb3-b922-ce65f2f6b811 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:55.537149 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-8117a5bc-ff8b-4bb3-b922-ce65f2f6b811 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] [instance: cf53f7cd-deae-4b34-bf8e-3afef98b2ec6] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:55.544423 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8117a5bc-ff8b-4bb3-b922-ce65f2f6b811 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:55.548797 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8117a5bc-ff8b-4bb3-b922-ce65f2f6b811 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:55.548797 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8117a5bc-ff8b-4bb3-b922-ce65f2f6b811 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:55.548797 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8117a5bc-ff8b-4bb3-b922-ce65f2f6b811 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:55.596810 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-8117a5bc-ff8b-4bb3-b922-ce65f2f6b811 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] [instance: cf53f7cd-deae-4b34-bf8e-3afef98b2ec6] Going to try to terminate instance {{(pid=138680) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 16 21:10:55.974062 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-8117a5bc-ff8b-4bb3-b922-ce65f2f6b811 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] 10.222.0.16 "DELETE /compute/v2.1/servers/cf53f7cd-deae-4b34-bf8e-3afef98b2ec6" status: 204 len: 0 microversion: 2.75 time: 0.436446 Jul 16 21:10:55.974062 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 400/801] 10.222.0.16 () {60 vars in 1279 bytes} [Sat Jul 16 21:10:55 2022] DELETE /compute/v2.1/servers/cf53f7cd-deae-4b34-bf8e-3afef98b2ec6 => generated 0 bytes in 438 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Jul 16 21:10:55.981522 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-0940b38a-bf2a-4fac-a15c-79a0eee61471 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:55.984133 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-0940b38a-bf2a-4fac-a15c-79a0eee61471 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] [instance: 4ee3ffbe-c16a-4847-a343-469c645e4b25] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:56.007709 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0940b38a-bf2a-4fac-a15c-79a0eee61471 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:56.017088 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0940b38a-bf2a-4fac-a15c-79a0eee61471 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:56.017088 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0940b38a-bf2a-4fac-a15c-79a0eee61471 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:56.017088 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0940b38a-bf2a-4fac-a15c-79a0eee61471 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:56.360156 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-0940b38a-bf2a-4fac-a15c-79a0eee61471 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a69f9dee03d43beaafa966216b9e289', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9b769bf8f3ea4e368bf68de1df6538f9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:56.367931 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-0940b38a-bf2a-4fac-a15c-79a0eee61471 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] 10.222.0.16 "GET /compute/v2.1/servers/4ee3ffbe-c16a-4847-a343-469c645e4b25" status: 200 len: 1504 microversion: 2.1 time: 0.391230 Jul 16 21:10:56.368554 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 401/802] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:10:55 2022] GET /compute/v2.1/servers/4ee3ffbe-c16a-4847-a343-469c645e4b25 => generated 1504 bytes in 399 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:10:56.374204 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-6044a254-2ebd-4dd3-8dfb-dae65b0e82d4 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:56.374772 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-6044a254-2ebd-4dd3-8dfb-dae65b0e82d4 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: f98a9762-a67c-4e95-85b8-727c81bdb03a] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:56.381396 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6044a254-2ebd-4dd3-8dfb-dae65b0e82d4 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:56.382450 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6044a254-2ebd-4dd3-8dfb-dae65b0e82d4 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:56.382843 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6044a254-2ebd-4dd3-8dfb-dae65b0e82d4 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:56.383327 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6044a254-2ebd-4dd3-8dfb-dae65b0e82d4 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:56.620825 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-6044a254-2ebd-4dd3-8dfb-dae65b0e82d4 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b68dacc4b8463ba07fae6372131117', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55a19b7a4682457a9132f5fc80110712', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:56.629356 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-6044a254-2ebd-4dd3-8dfb-dae65b0e82d4 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "GET /compute/v2.1/servers/f98a9762-a67c-4e95-85b8-727c81bdb03a" status: 200 len: 1462 microversion: 2.1 time: 0.259764 Jul 16 21:10:56.630732 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 402/803] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:10:56 2022] GET /compute/v2.1/servers/f98a9762-a67c-4e95-85b8-727c81bdb03a => generated 1462 bytes in 262 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:10:56.641782 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-305471f8-bdf5-4105-9bcc-abaa01d41fa3 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:56.642996 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-305471f8-bdf5-4105-9bcc-abaa01d41fa3 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] [instance: cf53f7cd-deae-4b34-bf8e-3afef98b2ec6] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:56.651077 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-305471f8-bdf5-4105-9bcc-abaa01d41fa3 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:56.651518 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-305471f8-bdf5-4105-9bcc-abaa01d41fa3 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:56.652294 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-305471f8-bdf5-4105-9bcc-abaa01d41fa3 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:56.652738 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-305471f8-bdf5-4105-9bcc-abaa01d41fa3 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:56.750745 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-305471f8-bdf5-4105-9bcc-abaa01d41fa3 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ba60685a2001459cbfb531d95a6b296e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb6deae6b08e4ed8b15dc6edfe1d38d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:56.753855 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-39ec3ac0-dde7-4d9f-ba06-faa091c05c90 tempest-SecurityGroupRulesTestJSON-1769504197 tempest-SecurityGroupRulesTestJSON-1769504197-project] 10.222.0.16 "POST /compute/v2.1/os-security-group-rules" status: 200 len: 233 microversion: 2.1 time: 1.254057 Jul 16 21:10:56.756523 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 402/804] 10.222.0.16 () {60 vars in 1194 bytes} [Sat Jul 16 21:10:55 2022] POST /compute/v2.1/os-security-group-rules => generated 233 bytes in 1259 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:10:56.774133 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-1b433760-5c09-4ebc-a8bd-8e32db89da5b tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:56.775955 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-1b433760-5c09-4ebc-a8bd-8e32db89da5b tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] [instance: a4a0aaec-dc64-400c-b3f0-ec61fa740b8d] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:56.776211 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-305471f8-bdf5-4105-9bcc-abaa01d41fa3 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ba60685a2001459cbfb531d95a6b296e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb6deae6b08e4ed8b15dc6edfe1d38d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:56.778072 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-305471f8-bdf5-4105-9bcc-abaa01d41fa3 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ba60685a2001459cbfb531d95a6b296e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb6deae6b08e4ed8b15dc6edfe1d38d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:56.786071 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-305471f8-bdf5-4105-9bcc-abaa01d41fa3 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] 10.222.0.16 "GET /compute/v2.1/servers/cf53f7cd-deae-4b34-bf8e-3afef98b2ec6" status: 200 len: 1444 microversion: 2.75 time: 0.153633 Jul 16 21:10:56.787068 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 403/805] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:10:56 2022] GET /compute/v2.1/servers/cf53f7cd-deae-4b34-bf8e-3afef98b2ec6 => generated 1444 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:10:56.789045 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1b433760-5c09-4ebc-a8bd-8e32db89da5b tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:56.789045 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1b433760-5c09-4ebc-a8bd-8e32db89da5b tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:56.790303 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1b433760-5c09-4ebc-a8bd-8e32db89da5b tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:56.790820 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1b433760-5c09-4ebc-a8bd-8e32db89da5b tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:56.792822 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-a3eb8bf8-9215-4ca3-ba65-37406b84a154 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:56.793765 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-a3eb8bf8-9215-4ca3-ba65-37406b84a154 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] [instance: de210dbd-1398-413f-9a0f-63cae300bd03] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:56.801842 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a3eb8bf8-9215-4ca3-ba65-37406b84a154 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:56.802584 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a3eb8bf8-9215-4ca3-ba65-37406b84a154 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:56.803732 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a3eb8bf8-9215-4ca3-ba65-37406b84a154 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:56.804327 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a3eb8bf8-9215-4ca3-ba65-37406b84a154 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:57.398233 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell0:SELECT=10 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:10:57.425392 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell0:SELECT=38 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:10:57.429233 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-1b433760-5c09-4ebc-a8bd-8e32db89da5b tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'abeb7020019d41a18de42956d706a011', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '649babe54d104436af461f1ee4767001', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:57.437259 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-a3eb8bf8-9215-4ca3-ba65-37406b84a154 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:57.437776 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-1b433760-5c09-4ebc-a8bd-8e32db89da5b tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] 10.222.0.16 "GET /compute/v2.1/servers/a4a0aaec-dc64-400c-b3f0-ec61fa740b8d" status: 200 len: 1316 microversion: 2.1 time: 0.679367 Jul 16 21:10:57.438249 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 403/806] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:10:56 2022] GET /compute/v2.1/servers/a4a0aaec-dc64-400c-b3f0-ec61fa740b8d => generated 1316 bytes in 681 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:10:57.444301 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-7efcd2e2-525c-47c8-b9a3-e4f923f19b7b tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:57.445204 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-7efcd2e2-525c-47c8-b9a3-e4f923f19b7b tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: f98a9762-a67c-4e95-85b8-727c81bdb03a] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:57.446788 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-a3eb8bf8-9215-4ca3-ba65-37406b84a154 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:57.449122 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-a3eb8bf8-9215-4ca3-ba65-37406b84a154 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:57.450336 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-a3eb8bf8-9215-4ca3-ba65-37406b84a154 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] 10.222.0.16 "GET /compute/v2.1/servers/de210dbd-1398-413f-9a0f-63cae300bd03" status: 200 len: 1362 microversion: 2.25 time: 0.662227 Jul 16 21:10:57.452337 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 404/807] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:10:56 2022] GET /compute/v2.1/servers/de210dbd-1398-413f-9a0f-63cae300bd03 => generated 1362 bytes in 665 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:10:57.452868 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7efcd2e2-525c-47c8-b9a3-e4f923f19b7b tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:57.453338 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7efcd2e2-525c-47c8-b9a3-e4f923f19b7b tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:57.454127 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7efcd2e2-525c-47c8-b9a3-e4f923f19b7b tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:57.454570 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7efcd2e2-525c-47c8-b9a3-e4f923f19b7b tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:57.459618 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-5601ddd7-c52a-46ca-bfad-c7e9e9caf36c tempest-SecurityGroupRulesTestJSON-1769504197 tempest-SecurityGroupRulesTestJSON-1769504197-project] Calling method '.version_select at 0x7fdd9d1ec4c0>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:57.620503 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-5601ddd7-c52a-46ca-bfad-c7e9e9caf36c tempest-SecurityGroupRulesTestJSON-1769504197 tempest-SecurityGroupRulesTestJSON-1769504197-project] 10.222.0.16 "GET /compute/v2.1/os-security-groups/a69b87bd-016b-430c-9f7e-92bfdc2493f2" status: 200 len: 664 microversion: 2.1 time: 0.164896 Jul 16 21:10:57.620503 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 405/808] 10.222.0.16 () {58 vars in 1268 bytes} [Sat Jul 16 21:10:57 2022] GET /compute/v2.1/os-security-groups/a69b87bd-016b-430c-9f7e-92bfdc2493f2 => generated 664 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:10:57.627905 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-362191ba-9755-4ac0-a847-7186792ce4ac tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:57.629097 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-362191ba-9755-4ac0-a847-7186792ce4ac tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] [instance: 4ee3ffbe-c16a-4847-a343-469c645e4b25] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:57.650120 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-362191ba-9755-4ac0-a847-7186792ce4ac tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:57.650120 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-362191ba-9755-4ac0-a847-7186792ce4ac tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:57.650120 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-362191ba-9755-4ac0-a847-7186792ce4ac tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:57.650120 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-362191ba-9755-4ac0-a847-7186792ce4ac tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:57.979298 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-7efcd2e2-525c-47c8-b9a3-e4f923f19b7b tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b68dacc4b8463ba07fae6372131117', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55a19b7a4682457a9132f5fc80110712', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:58.020158 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-7efcd2e2-525c-47c8-b9a3-e4f923f19b7b tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "GET /compute/v2.1/servers/f98a9762-a67c-4e95-85b8-727c81bdb03a" status: 200 len: 1462 microversion: 2.1 time: 0.578263 Jul 16 21:10:58.027540 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 404/809] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:10:57 2022] GET /compute/v2.1/servers/f98a9762-a67c-4e95-85b8-727c81bdb03a => generated 1462 bytes in 588 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:10:58.046588 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-7e7db112-f635-49d4-bf29-021c6c18d8fd tempest-SecurityGroupRulesTestJSON-1769504197 tempest-SecurityGroupRulesTestJSON-1769504197-project] Calling method '.version_select at 0x7fdd9d914430>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:58.191353 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-362191ba-9755-4ac0-a847-7186792ce4ac tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a69f9dee03d43beaafa966216b9e289', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9b769bf8f3ea4e368bf68de1df6538f9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:58.201496 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-362191ba-9755-4ac0-a847-7186792ce4ac tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] 10.222.0.16 "GET /compute/v2.1/servers/4ee3ffbe-c16a-4847-a343-469c645e4b25" status: 200 len: 1504 microversion: 2.1 time: 0.580108 Jul 16 21:10:58.202111 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 406/810] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:10:57 2022] GET /compute/v2.1/servers/4ee3ffbe-c16a-4847-a343-469c645e4b25 => generated 1504 bytes in 582 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:10:58.211412 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-eb9f8f3c-5ad0-40b2-ac63-12bd64afb9d9 req-efe41463-acc3-40b6-98cd-432aa302e206 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "a4a0aaec-dc64-400c-b3f0-ec61fa740b8d", "tag": "34a8af9d-6c47-4062-aba8-98d7061acf17"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:10:58.221171 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-eb9f8f3c-5ad0-40b2-ac63-12bd64afb9d9 req-efe41463-acc3-40b6-98cd-432aa302e206 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:58.221417 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-eb9f8f3c-5ad0-40b2-ac63-12bd64afb9d9 req-efe41463-acc3-40b6-98cd-432aa302e206 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:58.243466 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-eb9f8f3c-5ad0-40b2-ac63-12bd64afb9d9 req-efe41463-acc3-40b6-98cd-432aa302e206 service nova] Creating event network-changed:34a8af9d-6c47-4062-aba8-98d7061acf17 for instance a4a0aaec-dc64-400c-b3f0-ec61fa740b8d on nch1-master-1 Jul 16 21:10:58.256594 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-eb9f8f3c-5ad0-40b2-ac63-12bd64afb9d9 req-efe41463-acc3-40b6-98cd-432aa302e206 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.052992 Jul 16 21:10:58.257487 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 407/811] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:10:58 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:10:58.269196 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-aee1c6fc-0742-40ef-a593-6e0e98257a85 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:58.280595 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-aee1c6fc-0742-40ef-a593-6e0e98257a85 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] [instance: cf53f7cd-deae-4b34-bf8e-3afef98b2ec6] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:58.286999 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-aee1c6fc-0742-40ef-a593-6e0e98257a85 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:58.291777 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-aee1c6fc-0742-40ef-a593-6e0e98257a85 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:58.293904 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-aee1c6fc-0742-40ef-a593-6e0e98257a85 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:58.296539 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-aee1c6fc-0742-40ef-a593-6e0e98257a85 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:58.389808 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-aee1c6fc-0742-40ef-a593-6e0e98257a85 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:58.389808 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-aee1c6fc-0742-40ef-a593-6e0e98257a85 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:58.389808 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.objects.instance [None req-aee1c6fc-0742-40ef-a593-6e0e98257a85 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Lazy-loading 'fault' on Instance uuid cf53f7cd-deae-4b34-bf8e-3afef98b2ec6 {{(pid=138680) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:10:58.459805 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-aee1c6fc-0742-40ef-a593-6e0e98257a85 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ba60685a2001459cbfb531d95a6b296e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb6deae6b08e4ed8b15dc6edfe1d38d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:58.474456 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-aee1c6fc-0742-40ef-a593-6e0e98257a85 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ba60685a2001459cbfb531d95a6b296e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb6deae6b08e4ed8b15dc6edfe1d38d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:58.479896 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-aee1c6fc-0742-40ef-a593-6e0e98257a85 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ba60685a2001459cbfb531d95a6b296e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb6deae6b08e4ed8b15dc6edfe1d38d3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:58.490983 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-aee1c6fc-0742-40ef-a593-6e0e98257a85 tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] 10.222.0.16 "GET /compute/v2.1/servers/cf53f7cd-deae-4b34-bf8e-3afef98b2ec6" status: 200 len: 1449 microversion: 2.75 time: 0.231909 Jul 16 21:10:58.493899 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 408/812] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:10:58 2022] GET /compute/v2.1/servers/cf53f7cd-deae-4b34-bf8e-3afef98b2ec6 => generated 1449 bytes in 235 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:10:58.510675 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-d1979517-80be-466a-bc49-2b3d4b561c6e req-b2c27383-8275-4931-8066-017c7cd59c85 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "4ee3ffbe-c16a-4847-a343-469c645e4b25", "name": "network-vif-unplugged", "status": "completed", "tag": "f54ad45e-ef5d-4b3e-ae39-d46f5d43bc88"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:10:58.526730 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-d1979517-80be-466a-bc49-2b3d4b561c6e req-b2c27383-8275-4931-8066-017c7cd59c85 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:58.527260 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-d1979517-80be-466a-bc49-2b3d4b561c6e req-b2c27383-8275-4931-8066-017c7cd59c85 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:58.546365 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-d1979517-80be-466a-bc49-2b3d4b561c6e req-b2c27383-8275-4931-8066-017c7cd59c85 service nova] Creating event network-vif-unplugged:f54ad45e-ef5d-4b3e-ae39-d46f5d43bc88 for instance 4ee3ffbe-c16a-4847-a343-469c645e4b25 on nch1-master-1 Jul 16 21:10:58.568043 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-d1979517-80be-466a-bc49-2b3d4b561c6e req-b2c27383-8275-4931-8066-017c7cd59c85 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.071731 Jul 16 21:10:58.568782 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 409/813] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:10:58 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 74 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:10:58.585958 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-04dcc267-7296-4ae8-9ae4-8e80edc3c2c9 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Action: 'create', calling method: .version_select at 0x7fdd9d242e50>, body: {"keypair": {"name": "tempest-keypair-180937062", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBEeuuoLDlEemQngSxytGMaAbghgS3bXZtWA6y9EsyObyNtnp2Ihm5mZnUJWwTry1IS5GJwfRh54oJyhkuyVy6T8DsOmVpEuxlpxegqfemsuvNaK7Qp5Xf+aXKb3s4nEZxQ=="}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:10:58.602616 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-04dcc267-7296-4ae8-9ae4-8e80edc3c2c9 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Getting quotas for project 55a19b7a4682457a9132f5fc80110712. Resources: {'key_pairs'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:10:58.609353 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-04dcc267-7296-4ae8-9ae4-8e80edc3c2c9 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Getting quotas for user a9b68dacc4b8463ba07fae6372131117 and project 55a19b7a4682457a9132f5fc80110712. Resources: {'key_pairs'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:10:58.728817 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-7e7db112-f635-49d4-bf29-021c6c18d8fd tempest-SecurityGroupRulesTestJSON-1769504197 tempest-SecurityGroupRulesTestJSON-1769504197-project] 10.222.0.16 "DELETE /compute/v2.1/os-security-group-rules/ca789dd4-4b41-4970-a147-3f70e0c15683" status: 202 len: 0 microversion: 2.1 time: 0.697965 Jul 16 21:10:58.729897 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 405/814] 10.222.0.16 () {58 vars in 1286 bytes} [Sat Jul 16 21:10:58 2022] DELETE /compute/v2.1/os-security-group-rules/ca789dd4-4b41-4970-a147-3f70e0c15683 => generated 0 bytes in 701 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 21:10:58.734807 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-3bfee718-f2da-4409-8606-31e5a56b98ec tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:58.735629 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-3bfee718-f2da-4409-8606-31e5a56b98ec tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] [instance: a4a0aaec-dc64-400c-b3f0-ec61fa740b8d] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:58.742710 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3bfee718-f2da-4409-8606-31e5a56b98ec tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:58.743226 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3bfee718-f2da-4409-8606-31e5a56b98ec tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:58.744199 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3bfee718-f2da-4409-8606-31e5a56b98ec tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:58.744891 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3bfee718-f2da-4409-8606-31e5a56b98ec tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:58.775966 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-04dcc267-7296-4ae8-9ae4-8e80edc3c2c9 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "POST /compute/v2.1/os-keypairs" status: 200 len: 385 microversion: 2.1 time: 0.205899 Jul 16 21:10:58.778459 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 410/815] 10.222.0.16 () {60 vars in 1158 bytes} [Sat Jul 16 21:10:58 2022] POST /compute/v2.1/os-keypairs => generated 385 bytes in 209 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:10:58.785142 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-fa0399f2-e5c4-4a5f-a58d-29404ec207b6 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:58.789336 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-fa0399f2-e5c4-4a5f-a58d-29404ec207b6 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] [instance: de210dbd-1398-413f-9a0f-63cae300bd03] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:58.811637 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-fa0399f2-e5c4-4a5f-a58d-29404ec207b6 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:58.821581 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-fa0399f2-e5c4-4a5f-a58d-29404ec207b6 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:58.827060 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-fa0399f2-e5c4-4a5f-a58d-29404ec207b6 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:58.827854 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-fa0399f2-e5c4-4a5f-a58d-29404ec207b6 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:59.092816 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-3bfee718-f2da-4409-8606-31e5a56b98ec tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'abeb7020019d41a18de42956d706a011', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '649babe54d104436af461f1ee4767001', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:59.105975 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-3bfee718-f2da-4409-8606-31e5a56b98ec tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] 10.222.0.16 "GET /compute/v2.1/servers/a4a0aaec-dc64-400c-b3f0-ec61fa740b8d" status: 200 len: 1316 microversion: 2.1 time: 0.375224 Jul 16 21:10:59.108694 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 406/816] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:10:58 2022] GET /compute/v2.1/servers/a4a0aaec-dc64-400c-b3f0-ec61fa740b8d => generated 1316 bytes in 378 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:10:59.112010 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-fa0399f2-e5c4-4a5f-a58d-29404ec207b6 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:59.114429 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-e95dec74-134f-45ae-ad54-4ac43cb50a44 tempest-SecurityGroupRulesTestJSON-1769504197 tempest-SecurityGroupRulesTestJSON-1769504197-project] Calling method '.version_select at 0x7fdd9d8d3e50>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:59.123393 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-fa0399f2-e5c4-4a5f-a58d-29404ec207b6 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:59.127933 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-fa0399f2-e5c4-4a5f-a58d-29404ec207b6 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:59.133892 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-fa0399f2-e5c4-4a5f-a58d-29404ec207b6 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] 10.222.0.16 "GET /compute/v2.1/servers/de210dbd-1398-413f-9a0f-63cae300bd03" status: 200 len: 1362 microversion: 2.25 time: 0.354260 Jul 16 21:10:59.134892 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 411/817] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:10:58 2022] GET /compute/v2.1/servers/de210dbd-1398-413f-9a0f-63cae300bd03 => generated 1362 bytes in 356 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:10:59.224103 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-617c635a-7f87-4231-ab9b-f59056ff6df7 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:59.225217 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-617c635a-7f87-4231-ab9b-f59056ff6df7 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] [instance: 4ee3ffbe-c16a-4847-a343-469c645e4b25] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:59.232834 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-617c635a-7f87-4231-ab9b-f59056ff6df7 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:59.233566 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-617c635a-7f87-4231-ab9b-f59056ff6df7 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:59.234627 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-617c635a-7f87-4231-ab9b-f59056ff6df7 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:59.235281 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-617c635a-7f87-4231-ab9b-f59056ff6df7 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:59.430584 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-617c635a-7f87-4231-ab9b-f59056ff6df7 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a69f9dee03d43beaafa966216b9e289', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9b769bf8f3ea4e368bf68de1df6538f9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:10:59.440992 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-617c635a-7f87-4231-ab9b-f59056ff6df7 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] 10.222.0.16 "GET /compute/v2.1/servers/4ee3ffbe-c16a-4847-a343-469c645e4b25" status: 200 len: 1504 microversion: 2.1 time: 0.221784 Jul 16 21:10:59.442241 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 412/818] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:10:59 2022] GET /compute/v2.1/servers/4ee3ffbe-c16a-4847-a343-469c645e4b25 => generated 1504 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:10:59.516341 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-08d795a2-c761-4527-b8ff-115da40a63ad tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:59.521388 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-08d795a2-c761-4527-b8ff-115da40a63ad tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] [instance: cf53f7cd-deae-4b34-bf8e-3afef98b2ec6] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:10:59.525812 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-08d795a2-c761-4527-b8ff-115da40a63ad tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:59.526421 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-08d795a2-c761-4527-b8ff-115da40a63ad tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:59.528314 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-08d795a2-c761-4527-b8ff-115da40a63ad tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:10:59.529270 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-08d795a2-c761-4527-b8ff-115da40a63ad tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:10:59.529494 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-e95dec74-134f-45ae-ad54-4ac43cb50a44 tempest-SecurityGroupRulesTestJSON-1769504197 tempest-SecurityGroupRulesTestJSON-1769504197-project] 10.222.0.16 "DELETE /compute/v2.1/os-security-groups/a69b87bd-016b-430c-9f7e-92bfdc2493f2" status: 202 len: 0 microversion: 2.1 time: 0.419077 Jul 16 21:10:59.530978 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 407/819] 10.222.0.16 () {58 vars in 1271 bytes} [Sat Jul 16 21:10:59 2022] DELETE /compute/v2.1/os-security-groups/a69b87bd-016b-430c-9f7e-92bfdc2493f2 => generated 0 bytes in 421 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 21:10:59.541574 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-62d4dfb1-9f62-4591-9678-88033035801b tempest-SecurityGroupRulesTestJSON-1769504197 tempest-SecurityGroupRulesTestJSON-1769504197-project] Calling method '.version_select at 0x7fdd9d2571f0>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:10:59.547203 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-08d795a2-c761-4527-b8ff-115da40a63ad tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] HTTP exception thrown: Instance cf53f7cd-deae-4b34-bf8e-3afef98b2ec6 could not be found. Jul 16 21:10:59.549402 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-08d795a2-c761-4527-b8ff-115da40a63ad tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Request method failure captured: Jul 16 21:10:59.549402 ncdv-master-1 devstack@n-api.service[138680]: request: GET /compute/v2.1/servers/cf53f7cd-deae-4b34-bf8e-3afef98b2ec6 HTTP/1.1 Jul 16 21:10:59.549402 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:10:59.549402 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:10:59.549402 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:10:59.549402 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 0 Jul 16 21:10:59.549402 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:10:59.549402 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:10:59.549402 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:10:59.549402 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:10:59.549402 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0ykH6ruyDPAASF9Gf8gmhHl48DuEFe65zelFwp7PIrenL2C3qaKOCo7aou-ZYXIQwHzDWQhoygppfjQ36nJrpgiMUPI82lLGYWF7jOz8-EGniMq4NmlFvNGO9j1y1pnTFcqX9VNbEieMTbeKIkFR74cr_64NAGVhQLC1GtZHy_Of99M Jul 16 21:10:59.549402 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:10:59.549402 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:10:59.549402 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:10:59.549402 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:10:59.549402 ncdv-master-1 devstack@n-api.service[138680]: X-Openstack-Nova-Api-Version: 2.75 Jul 16 21:10:59.549402 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:10:59.549402 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:10:59.549402 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: fb6deae6b08e4ed8b15dc6edfe1d38d3 Jul 16 21:10:59.549402 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-ServersAdmin275Test-319972257 Jul 16 21:10:59.549402 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:10:59.549402 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:10:59.551054 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/fb6deae6b08e4ed8b15dc6edfe1d38d3"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/fb6deae6b08e4ed8b15dc6edfe1d38d3"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_fb6deae6b08e4ed8b15dc6edfe1d38d3"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/fb6deae6b08e4ed8b15dc6edfe1d38d3"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:10:59.551054 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-ServersAdmin275Test-319972257 Jul 16 21:10:59.551054 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: fb6deae6b08e4ed8b15dc6edfe1d38d3 Jul 16 21:10:59.551054 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-ServersAdmin275Test-319972257 Jul 16 21:10:59.551054 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-ServersAdmin275Test-319972257-project Jul 16 21:10:59.551054 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:10:59.551054 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:10:59.551054 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: ba60685a2001459cbfb531d95a6b296e Jul 16 21:10:59.551054 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-ServersAdmin275Test-319972257-project Jul 16 21:10:59.551054 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 21:10:59.551054 ncdv-master-1 devstack@n-api.service[138680]: exception: Instance cf53f7cd-deae-4b34-bf8e-3afef98b2ec6 could not be found. Jul 16 21:10:59.551054 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': 'cf53f7cd-deae-4b34-bf8e-3afef98b2ec6'} Jul 16 21:10:59.557837 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:10:59.557837 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:10:59.557837 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:10:59.557837 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:10:59.557837 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:10:59.557837 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:10:59.557837 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:10:59.557837 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:10:59.557837 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:10:59.557837 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:10:59.557837 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance cf53f7cd-deae-4b34-bf8e-3afef98b2ec6 could not be found. Jul 16 21:10:59.557837 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:10:59.557837 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:10:59.557837 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:10:59.557837 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:10:59.557837 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:10:59.557837 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:10:59.557837 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:10:59.557837 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:10:59.557837 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:10:59.559716 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:10:59.559716 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:10:59.559716 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:10:59.559716 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:10:59.559716 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:10:59.559716 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:10:59.559716 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:10:59.559716 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance cf53f7cd-deae-4b34-bf8e-3afef98b2ec6 could not be found. Jul 16 21:10:59.559716 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:10:59.559716 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:10:59.559716 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:10:59.559716 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:10:59.559716 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:10:59.559716 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:10:59.559716 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:10:59.559716 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:10:59.559716 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance cf53f7cd-deae-4b34-bf8e-3afef98b2ec6 could not be found. Jul 16 21:10:59.559716 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:10:59.561623 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-08d795a2-c761-4527-b8ff-115da40a63ad tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] Returning 404 to user: Instance cf53f7cd-deae-4b34-bf8e-3afef98b2ec6 could not be found. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:10:59.561623 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-08d795a2-c761-4527-b8ff-115da40a63ad tempest-ServersAdmin275Test-319972257 tempest-ServersAdmin275Test-319972257-project] 10.222.0.16 "GET /compute/v2.1/servers/cf53f7cd-deae-4b34-bf8e-3afef98b2ec6" status: 404 len: 111 microversion: 2.75 time: 0.042392 Jul 16 21:10:59.561623 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 413/820] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:10:59 2022] GET /compute/v2.1/servers/cf53f7cd-deae-4b34-bf8e-3afef98b2ec6 => generated 111 bytes in 44 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Jul 16 21:10:59.579340 ncdv-master-1 devstack@n-api.service[138681]: DEBUG neutronclient.v2_0.client [None req-62d4dfb1-9f62-4591-9678-88033035801b tempest-SecurityGroupRulesTestJSON-1769504197 tempest-SecurityGroupRulesTestJSON-1769504197-project] Error message: {"NeutronError": {"type": "SecurityGroupNotFound", "message": "Security group 91acad9f-4045-46df-87c9-831806306000 does not exist", "detail": ""}} {{(pid=138681) _handle_fault_response /usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py:262}} Jul 16 21:10:59.580103 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.network.security_group_api [None req-62d4dfb1-9f62-4591-9678-88033035801b tempest-SecurityGroupRulesTestJSON-1769504197 tempest-SecurityGroupRulesTestJSON-1769504197-project] Neutron security group 91acad9f-4045-46df-87c9-831806306000 not found {{(pid=138681) get /opt/stack/nova/nova/network/security_group_api.py:313}} Jul 16 21:10:59.580914 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-62d4dfb1-9f62-4591-9678-88033035801b tempest-SecurityGroupRulesTestJSON-1769504197 tempest-SecurityGroupRulesTestJSON-1769504197-project] HTTP exception thrown: Security group 91acad9f-4045-46df-87c9-831806306000 does not exist Jul 16 21:10:59.580914 ncdv-master-1 devstack@n-api.service[138681]: Neutron server returns request_ids: ['req-c8ef3348-0971-4917-9885-d0902dd572ef'] Jul 16 21:10:59.589675 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-62d4dfb1-9f62-4591-9678-88033035801b tempest-SecurityGroupRulesTestJSON-1769504197 tempest-SecurityGroupRulesTestJSON-1769504197-project] Request method failure captured: Jul 16 21:10:59.589675 ncdv-master-1 devstack@n-api.service[138681]: request: DELETE /compute/v2.1/os-security-groups/91acad9f-4045-46df-87c9-831806306000 HTTP/1.1 Jul 16 21:10:59.589675 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:10:59.589675 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:10:59.589675 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:10:59.589675 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 21:10:59.589675 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:10:59.589675 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:10:59.589675 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:10:59.589675 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:10:59.589675 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0ylOALUYF4KLhmdCP9g4BhguKQzITjC6EjhJxL14XPfVOm8q-UrXfGhYb6pu622UYgqKeWws_os7Rhdi0KWHAnlVkWMpmyUhxTcpwZWfmR9CGwlPv82i4GS94R00lveUInDTWutTy-x_8Wzy3LsHCFHkhTBR-TMNbkZeoVGdqEZVAnA Jul 16 21:10:59.589675 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:10:59.589675 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:10:59.589675 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:10:59.589675 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:10:59.589675 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:10:59.589675 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:10:59.589675 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: e62d0d1cfd6b44bf8a0c0df282e585ad Jul 16 21:10:59.589675 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-SecurityGroupRulesTestJSON-1769504197 Jul 16 21:10:59.589675 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:10:59.589675 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:10:59.591219 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/e62d0d1cfd6b44bf8a0c0df282e585ad"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/e62d0d1cfd6b44bf8a0c0df282e585ad"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_e62d0d1cfd6b44bf8a0c0df282e585ad"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/e62d0d1cfd6b44bf8a0c0df282e585ad"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:10:59.591219 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-SecurityGroupRulesTestJSON-1769504197 Jul 16 21:10:59.591219 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: e62d0d1cfd6b44bf8a0c0df282e585ad Jul 16 21:10:59.591219 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-SecurityGroupRulesTestJSON-1769504197 Jul 16 21:10:59.591219 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-SecurityGroupRulesTestJSON-1769504197-project Jul 16 21:10:59.591219 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:10:59.591219 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:10:59.591219 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: 31756abdfccf43e0b33e33300a527f29 Jul 16 21:10:59.591219 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-SecurityGroupRulesTestJSON-1769504197-project Jul 16 21:10:59.591219 ncdv-master-1 devstack@n-api.service[138681]: method: Jul 16 21:10:59.591219 ncdv-master-1 devstack@n-api.service[138681]: exception: Security group 91acad9f-4045-46df-87c9-831806306000 does not exist Jul 16 21:10:59.591219 ncdv-master-1 devstack@n-api.service[138681]: Neutron server returns request_ids: ['req-c8ef3348-0971-4917-9885-d0902dd572ef'] Jul 16 21:10:59.591219 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': '91acad9f-4045-46df-87c9-831806306000'} Jul 16 21:10:59.592444 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:10:59.592444 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:10:59.592444 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/security_group_api.py", line 309, in get Jul 16 21:10:59.592444 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi group = neutron.show_security_group(id).get('security_group') Jul 16 21:10:59.592444 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 197, in wrapper Jul 16 21:10:59.592444 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Jul 16 21:10:59.592444 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 1040, in show_security_group Jul 16 21:10:59.592444 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return self.get(self.security_group_path % (security_group), Jul 16 21:10:59.592444 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 197, in wrapper Jul 16 21:10:59.592444 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Jul 16 21:10:59.592444 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 356, in get Jul 16 21:10:59.592444 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return self.retry_request("GET", action, body=body, Jul 16 21:10:59.592444 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 197, in wrapper Jul 16 21:10:59.592444 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Jul 16 21:10:59.592444 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 333, in retry_request Jul 16 21:10:59.592444 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return self.do_request(method, action, body=body, Jul 16 21:10:59.592444 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 197, in wrapper Jul 16 21:10:59.592444 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Jul 16 21:10:59.592444 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 297, in do_request Jul 16 21:10:59.593780 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi self._handle_fault_response(status_code, replybody, resp) Jul 16 21:10:59.593780 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 197, in wrapper Jul 16 21:10:59.593780 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Jul 16 21:10:59.593780 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 272, in _handle_fault_response Jul 16 21:10:59.593780 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi exception_handler_v20(status_code, error_body) Jul 16 21:10:59.593780 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 90, in exception_handler_v20 Jul 16 21:10:59.593780 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise client_exc(message=error_message, Jul 16 21:10:59.593780 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi neutronclient.common.exceptions.NotFound: Security group 91acad9f-4045-46df-87c9-831806306000 does not exist Jul 16 21:10:59.593780 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-c8ef3348-0971-4917-9885-d0902dd572ef'] Jul 16 21:10:59.593780 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:10:59.593780 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:10:59.593780 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:10:59.593780 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:10:59.593780 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/security_groups.py", line 178, in delete Jul 16 21:10:59.593780 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi security_group = security_group_api.get(context, id) Jul 16 21:10:59.593780 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/security_group_api.py", line 314, in get Jul 16 21:10:59.593780 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.SecurityGroupNotFound(str(e)) Jul 16 21:10:59.593780 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.SecurityGroupNotFound: Security group 91acad9f-4045-46df-87c9-831806306000 does not exist Jul 16 21:10:59.600603 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-c8ef3348-0971-4917-9885-d0902dd572ef'] Jul 16 21:10:59.600603 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:10:59.600603 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:10:59.600603 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:10:59.600603 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:10:59.600603 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:10:59.600603 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:10:59.600603 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:10:59.600603 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:10:59.600603 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Jul 16 21:10:59.600603 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Jul 16 21:10:59.600603 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:10:59.600603 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:10:59.600603 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/security_groups.py", line 181, in delete Jul 16 21:10:59.600603 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=exp.format_message()) Jul 16 21:10:59.600603 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Security group 91acad9f-4045-46df-87c9-831806306000 does not exist Jul 16 21:10:59.600603 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-c8ef3348-0971-4917-9885-d0902dd572ef'] Jul 16 21:10:59.600603 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:10:59.600603 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:10:59.600603 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:10:59.600603 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:10:59.603109 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:10:59.603109 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:10:59.603109 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:10:59.603109 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:10:59.603109 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Security group 91acad9f-4045-46df-87c9-831806306000 does not exist Jul 16 21:10:59.603109 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-c8ef3348-0971-4917-9885-d0902dd572ef'] Jul 16 21:10:59.603109 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:10:59.603109 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-62d4dfb1-9f62-4591-9678-88033035801b tempest-SecurityGroupRulesTestJSON-1769504197 tempest-SecurityGroupRulesTestJSON-1769504197-project] Returning 404 to user: Security group 91acad9f-4045-46df-87c9-831806306000 does not exist Jul 16 21:10:59.603109 ncdv-master-1 devstack@n-api.service[138681]: Neutron server returns request_ids: ['req-c8ef3348-0971-4917-9885-d0902dd572ef'] {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:10:59.603109 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-62d4dfb1-9f62-4591-9678-88033035801b tempest-SecurityGroupRulesTestJSON-1769504197 tempest-SecurityGroupRulesTestJSON-1769504197-project] 10.222.0.16 "DELETE /compute/v2.1/os-security-groups/91acad9f-4045-46df-87c9-831806306000" status: 404 len: 194 microversion: 2.1 time: 0.057345 Jul 16 21:10:59.603109 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 408/821] 10.222.0.16 () {58 vars in 1271 bytes} [Sat Jul 16 21:10:59 2022] DELETE /compute/v2.1/os-security-groups/91acad9f-4045-46df-87c9-831806306000 => generated 194 bytes in 61 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:10:59.605240 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-3a38bf64-69df-46c0-b899-8bcd70dd1e5e tempest-SecurityGroupRulesTestJSON-1769504197 tempest-SecurityGroupRulesTestJSON-1769504197-project] Calling method '.version_select at 0x7fdd9d2a5430>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:11:00.031933 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-3a38bf64-69df-46c0-b899-8bcd70dd1e5e tempest-SecurityGroupRulesTestJSON-1769504197 tempest-SecurityGroupRulesTestJSON-1769504197-project] 10.222.0.16 "DELETE /compute/v2.1/os-security-groups/3e52cc49-94d1-4f54-91ea-99e2789ec1ca" status: 202 len: 0 microversion: 2.1 time: 0.432848 Jul 16 21:11:00.034102 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 414/822] 10.222.0.16 () {58 vars in 1271 bytes} [Sat Jul 16 21:10:59 2022] DELETE /compute/v2.1/os-security-groups/3e52cc49-94d1-4f54-91ea-99e2789ec1ca => generated 0 bytes in 436 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 21:11:00.048039 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-2f88a826-c729-44fd-a841-d8c6fb3e0d2c tempest-SecurityGroupRulesTestJSON-1769504197 tempest-SecurityGroupRulesTestJSON-1769504197-project] Calling method '.version_select at 0x7fdd9d186160>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:11:00.108459 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-4feb86e3-7152-4c98-a7d2-18e6ff7ad67f req-24a924e8-361d-4527-a901-1baf355d5a14 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "de210dbd-1398-413f-9a0f-63cae300bd03", "name": "network-vif-plugged", "status": "completed", "tag": "176a187b-2abc-455b-bbcd-a736dec0c33e"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:11:00.135666 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-4feb86e3-7152-4c98-a7d2-18e6ff7ad67f req-24a924e8-361d-4527-a901-1baf355d5a14 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:00.138488 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-4feb86e3-7152-4c98-a7d2-18e6ff7ad67f req-24a924e8-361d-4527-a901-1baf355d5a14 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:00.169089 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-4feb86e3-7152-4c98-a7d2-18e6ff7ad67f req-24a924e8-361d-4527-a901-1baf355d5a14 service nova] Creating event network-vif-plugged:176a187b-2abc-455b-bbcd-a736dec0c33e for instance de210dbd-1398-413f-9a0f-63cae300bd03 on nch2-master-1 Jul 16 21:11:00.178407 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-4feb86e3-7152-4c98-a7d2-18e6ff7ad67f req-24a924e8-361d-4527-a901-1baf355d5a14 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.085778 Jul 16 21:11:00.179369 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 415/823] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:11:00 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 16 21:11:00.185472 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-1c49416a-10a4-476d-945c-8fe58f044524 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:11:00.186201 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-1c49416a-10a4-476d-945c-8fe58f044524 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] [instance: a4a0aaec-dc64-400c-b3f0-ec61fa740b8d] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:11:00.194282 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1c49416a-10a4-476d-945c-8fe58f044524 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:00.194764 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1c49416a-10a4-476d-945c-8fe58f044524 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:00.195561 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1c49416a-10a4-476d-945c-8fe58f044524 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:00.196009 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1c49416a-10a4-476d-945c-8fe58f044524 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:00.969347 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-1c49416a-10a4-476d-945c-8fe58f044524 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'abeb7020019d41a18de42956d706a011', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '649babe54d104436af461f1ee4767001', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:11:00.972029 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:11:00.978315 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-1c49416a-10a4-476d-945c-8fe58f044524 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] 10.222.0.16 "GET /compute/v2.1/servers/a4a0aaec-dc64-400c-b3f0-ec61fa740b8d" status: 200 len: 1316 microversion: 2.1 time: 0.797545 Jul 16 21:11:00.979055 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 416/824] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:11:00 2022] GET /compute/v2.1/servers/a4a0aaec-dc64-400c-b3f0-ec61fa740b8d => generated 1316 bytes in 799 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:11:00.984594 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-5e78f2f5-97e9-4dd6-a4c8-2315295d0e7c tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:11:00.985621 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-5e78f2f5-97e9-4dd6-a4c8-2315295d0e7c tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] [instance: de210dbd-1398-413f-9a0f-63cae300bd03] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:11:00.992093 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5e78f2f5-97e9-4dd6-a4c8-2315295d0e7c tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:00.993034 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5e78f2f5-97e9-4dd6-a4c8-2315295d0e7c tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:00.993835 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5e78f2f5-97e9-4dd6-a4c8-2315295d0e7c tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:00.994345 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5e78f2f5-97e9-4dd6-a4c8-2315295d0e7c tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:01.492079 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-5e78f2f5-97e9-4dd6-a4c8-2315295d0e7c tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:11:01.503776 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-5e78f2f5-97e9-4dd6-a4c8-2315295d0e7c tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:11:01.507203 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-5e78f2f5-97e9-4dd6-a4c8-2315295d0e7c tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:11:01.508504 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-5e78f2f5-97e9-4dd6-a4c8-2315295d0e7c tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] 10.222.0.16 "GET /compute/v2.1/servers/de210dbd-1398-413f-9a0f-63cae300bd03" status: 200 len: 1362 microversion: 2.25 time: 0.528028 Jul 16 21:11:01.521033 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 417/825] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:11:00 2022] GET /compute/v2.1/servers/de210dbd-1398-413f-9a0f-63cae300bd03 => generated 1362 bytes in 541 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:11:01.529598 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-546e2fe0-d62f-4aee-a698-f3ee21873dec tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:11:01.530529 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-546e2fe0-d62f-4aee-a698-f3ee21873dec tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] [instance: 4ee3ffbe-c16a-4847-a343-469c645e4b25] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:11:01.537517 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-546e2fe0-d62f-4aee-a698-f3ee21873dec tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:01.538054 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-546e2fe0-d62f-4aee-a698-f3ee21873dec tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:01.539091 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-546e2fe0-d62f-4aee-a698-f3ee21873dec tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:01.539815 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-546e2fe0-d62f-4aee-a698-f3ee21873dec tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:01.554596 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-2f88a826-c729-44fd-a841-d8c6fb3e0d2c tempest-SecurityGroupRulesTestJSON-1769504197 tempest-SecurityGroupRulesTestJSON-1769504197-project] 10.222.0.16 "DELETE /compute/v2.1/os-security-groups/6e6c37e2-9637-49bf-93cf-b4c44d8669d8" status: 202 len: 0 microversion: 2.1 time: 1.516710 Jul 16 21:11:01.555455 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 409/826] 10.222.0.16 () {58 vars in 1271 bytes} [Sat Jul 16 21:11:00 2022] DELETE /compute/v2.1/os-security-groups/6e6c37e2-9637-49bf-93cf-b4c44d8669d8 => generated 0 bytes in 1518 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 21:11:01.579394 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-faa18de2-dd82-4315-acdf-6b25b925621f tempest-SecurityGroupRulesTestJSON-1769504197 tempest-SecurityGroupRulesTestJSON-1769504197-project] Calling method '.version_select at 0x7fdd9dd3bca0>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:11:01.909899 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-546e2fe0-d62f-4aee-a698-f3ee21873dec tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a69f9dee03d43beaafa966216b9e289', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9b769bf8f3ea4e368bf68de1df6538f9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:11:01.932455 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-546e2fe0-d62f-4aee-a698-f3ee21873dec tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] 10.222.0.16 "GET /compute/v2.1/servers/4ee3ffbe-c16a-4847-a343-469c645e4b25" status: 200 len: 1479 microversion: 2.1 time: 0.402231 Jul 16 21:11:01.935113 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 418/827] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:11:01 2022] GET /compute/v2.1/servers/4ee3ffbe-c16a-4847-a343-469c645e4b25 => generated 1479 bytes in 413 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:11:01.956297 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-c63e4648-0892-4cd1-9da0-cb6786a3e279 tempest-LiveMigrationTest-1131758637 tempest-LiveMigrationTest-1131758637-project-admin] Calling method '.version_select at 0x7fdd9d1ec9d0>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:11:01.959208 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c63e4648-0892-4cd1-9da0-cb6786a3e279 tempest-LiveMigrationTest-1131758637 tempest-LiveMigrationTest-1131758637-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:01.959668 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c63e4648-0892-4cd1-9da0-cb6786a3e279 tempest-LiveMigrationTest-1131758637 tempest-LiveMigrationTest-1131758637-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:01.980569 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-c63e4648-0892-4cd1-9da0-cb6786a3e279 tempest-LiveMigrationTest-1131758637 tempest-LiveMigrationTest-1131758637-project-admin] 10.222.0.16 "GET /compute/v2.1/os-migrations" status: 200 len: 1568 microversion: 2.1 time: 0.028052 Jul 16 21:11:01.990987 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 419/828] 10.222.0.16 () {58 vars in 1142 bytes} [Sat Jul 16 21:11:01 2022] GET /compute/v2.1/os-migrations => generated 1568 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:11:02.014831 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-c22306b2-e15a-4baa-8ec1-c7fc48012d19 tempest-LiveMigrationTest-1131758637 tempest-LiveMigrationTest-1131758637-project-admin] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:11:02.014831 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-c22306b2-e15a-4baa-8ec1-c7fc48012d19 tempest-LiveMigrationTest-1131758637 tempest-LiveMigrationTest-1131758637-project-admin] [instance: 4ee3ffbe-c16a-4847-a343-469c645e4b25] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:11:02.026406 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c22306b2-e15a-4baa-8ec1-c7fc48012d19 tempest-LiveMigrationTest-1131758637 tempest-LiveMigrationTest-1131758637-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:02.026893 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c22306b2-e15a-4baa-8ec1-c7fc48012d19 tempest-LiveMigrationTest-1131758637 tempest-LiveMigrationTest-1131758637-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:02.027720 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c22306b2-e15a-4baa-8ec1-c7fc48012d19 tempest-LiveMigrationTest-1131758637 tempest-LiveMigrationTest-1131758637-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:02.028217 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c22306b2-e15a-4baa-8ec1-c7fc48012d19 tempest-LiveMigrationTest-1131758637 tempest-LiveMigrationTest-1131758637-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:02.321986 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-c22306b2-e15a-4baa-8ec1-c7fc48012d19 tempest-LiveMigrationTest-1131758637 tempest-LiveMigrationTest-1131758637-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/4ee3ffbe-c16a-4847-a343-469c645e4b25" status: 200 len: 1630 microversion: 2.1 time: 0.329136 Jul 16 21:11:02.327243 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 420/829] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:11:01 2022] GET /compute/v2.1/servers/4ee3ffbe-c16a-4847-a343-469c645e4b25 => generated 1630 bytes in 331 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:11:02.337204 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-86d4d82c-d3cb-4fa0-ad84-e9f2274eb67b tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:11:02.339864 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-86d4d82c-d3cb-4fa0-ad84-e9f2274eb67b tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] [instance: a4a0aaec-dc64-400c-b3f0-ec61fa740b8d] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:11:02.349422 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-86d4d82c-d3cb-4fa0-ad84-e9f2274eb67b tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:02.349887 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-86d4d82c-d3cb-4fa0-ad84-e9f2274eb67b tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:02.350689 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-86d4d82c-d3cb-4fa0-ad84-e9f2274eb67b tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:02.351171 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-86d4d82c-d3cb-4fa0-ad84-e9f2274eb67b tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:02.397927 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-faa18de2-dd82-4315-acdf-6b25b925621f tempest-SecurityGroupRulesTestJSON-1769504197 tempest-SecurityGroupRulesTestJSON-1769504197-project] 10.222.0.16 "DELETE /compute/v2.1/os-security-groups/57736166-7c94-4eb2-abd1-a804b9128fc1" status: 202 len: 0 microversion: 2.1 time: 0.838555 Jul 16 21:11:02.398636 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 410/830] 10.222.0.16 () {58 vars in 1271 bytes} [Sat Jul 16 21:11:01 2022] DELETE /compute/v2.1/os-security-groups/57736166-7c94-4eb2-abd1-a804b9128fc1 => generated 0 bytes in 840 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Jul 16 21:11:02.407912 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-5919f1d8-54a2-4c58-81b5-d39c246652ef tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Action: 'create', calling method: .version_select at 0x7fdd9d233e50>, body: {"keypair": {"name": "tempest-keypair-1244080767", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBEuOMeVIS7Hh0nVEL5TOpJuR2ZL0T9b1+wUasGb4yGUPBF/HQk2cDyQG93jInpFmZKcfdJl3JWbg8A51yaOIRBqx9t/YdoFNMimbZuIq1t9quxdrIMf1Il78BsNIKL6Pjg=="}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:11:02.430834 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-5919f1d8-54a2-4c58-81b5-d39c246652ef tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Getting quotas for project 9b769bf8f3ea4e368bf68de1df6538f9. Resources: {'key_pairs'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:11:02.438231 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-5919f1d8-54a2-4c58-81b5-d39c246652ef tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Getting quotas for user 6a69f9dee03d43beaafa966216b9e289 and project 9b769bf8f3ea4e368bf68de1df6538f9. Resources: {'key_pairs'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:11:02.537594 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-5919f1d8-54a2-4c58-81b5-d39c246652ef tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] 10.222.0.16 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.137239 Jul 16 21:11:02.538646 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 411/831] 10.222.0.16 () {60 vars in 1158 bytes} [Sat Jul 16 21:11:02 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 139 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:11:02.546819 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-77050519-e47d-4fb9-957d-013802b98a1e tempest-SecurityGroupRulesTestJSON-1769504197 tempest-SecurityGroupRulesTestJSON-1769504197-project] Calling method '.version_select at 0x7fdd9d2768b0>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:11:02.699821 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-86d4d82c-d3cb-4fa0-ad84-e9f2274eb67b tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'abeb7020019d41a18de42956d706a011', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '649babe54d104436af461f1ee4767001', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:11:02.716965 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-86d4d82c-d3cb-4fa0-ad84-e9f2274eb67b tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] 10.222.0.16 "GET /compute/v2.1/servers/a4a0aaec-dc64-400c-b3f0-ec61fa740b8d" status: 200 len: 1316 microversion: 2.1 time: 0.392481 Jul 16 21:11:02.729301 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 421/832] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:11:02 2022] GET /compute/v2.1/servers/a4a0aaec-dc64-400c-b3f0-ec61fa740b8d => generated 1316 bytes in 405 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:11:02.741837 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-fd34598a-502f-4a57-bdd5-a9f142d92a6a tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:11:02.742756 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-fd34598a-502f-4a57-bdd5-a9f142d92a6a tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] [instance: de210dbd-1398-413f-9a0f-63cae300bd03] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:11:02.757321 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-fd34598a-502f-4a57-bdd5-a9f142d92a6a tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:02.757988 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-fd34598a-502f-4a57-bdd5-a9f142d92a6a tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:02.758962 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-fd34598a-502f-4a57-bdd5-a9f142d92a6a tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:02.759759 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-fd34598a-502f-4a57-bdd5-a9f142d92a6a tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:03.083867 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-fd34598a-502f-4a57-bdd5-a9f142d92a6a tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:11:03.094010 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-fd34598a-502f-4a57-bdd5-a9f142d92a6a tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:11:03.095933 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-fd34598a-502f-4a57-bdd5-a9f142d92a6a tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:11:03.097418 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-fd34598a-502f-4a57-bdd5-a9f142d92a6a tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] 10.222.0.16 "GET /compute/v2.1/servers/de210dbd-1398-413f-9a0f-63cae300bd03" status: 200 len: 1557 microversion: 2.25 time: 0.366849 Jul 16 21:11:03.109771 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 422/833] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:11:02 2022] GET /compute/v2.1/servers/de210dbd-1398-413f-9a0f-63cae300bd03 => generated 1557 bytes in 380 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:11:03.297575 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-77050519-e47d-4fb9-957d-013802b98a1e tempest-SecurityGroupRulesTestJSON-1769504197 tempest-SecurityGroupRulesTestJSON-1769504197-project] 10.222.0.16 "DELETE /compute/v2.1/os-security-groups/7e73aa4a-33ad-4464-97d6-98250316ce9c" status: 202 len: 0 microversion: 2.1 time: 0.757546 Jul 16 21:11:03.298522 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 412/834] 10.222.0.16 () {58 vars in 1271 bytes} [Sat Jul 16 21:11:02 2022] DELETE /compute/v2.1/os-security-groups/7e73aa4a-33ad-4464-97d6-98250316ce9c => generated 0 bytes in 759 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 21:11:03.307609 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-4818d5db-fb5c-419d-ba0f-f80a77c57c8c tempest-SecurityGroupRulesTestJSON-1769504197 tempest-SecurityGroupRulesTestJSON-1769504197-project] Calling method '.version_select at 0x7fdd9d276ee0>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:11:03.533459 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-e457664d-4302-422a-9258-06bdb9794e93 tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:11:03.534243 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-e457664d-4302-422a-9258-06bdb9794e93 tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] [instance: de210dbd-1398-413f-9a0f-63cae300bd03] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:11:03.543119 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e457664d-4302-422a-9258-06bdb9794e93 tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:03.544594 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e457664d-4302-422a-9258-06bdb9794e93 tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:03.545983 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e457664d-4302-422a-9258-06bdb9794e93 tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:03.546850 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e457664d-4302-422a-9258-06bdb9794e93 tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:03.843425 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.objects.instance [None req-e457664d-4302-422a-9258-06bdb9794e93 tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] Lazy-loading 'services' on Instance uuid de210dbd-1398-413f-9a0f-63cae300bd03 {{(pid=138680) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:11:03.860167 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-e457664d-4302-422a-9258-06bdb9794e93 tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/de210dbd-1398-413f-9a0f-63cae300bd03" status: 200 len: 2050 microversion: 2.25 time: 0.745457 Jul 16 21:11:03.862036 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 423/835] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:11:03 2022] GET /compute/v2.1/servers/de210dbd-1398-413f-9a0f-63cae300bd03 => generated 2050 bytes in 748 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:11:03.867410 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-f7443018-0404-41e2-a7be-4a03700e5c4f tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:11:03.868094 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-f7443018-0404-41e2-a7be-4a03700e5c4f tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] [instance: a4a0aaec-dc64-400c-b3f0-ec61fa740b8d] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:11:03.876202 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f7443018-0404-41e2-a7be-4a03700e5c4f tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:03.876665 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f7443018-0404-41e2-a7be-4a03700e5c4f tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:03.877469 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f7443018-0404-41e2-a7be-4a03700e5c4f tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:03.877914 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f7443018-0404-41e2-a7be-4a03700e5c4f tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:03.911669 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-4818d5db-fb5c-419d-ba0f-f80a77c57c8c tempest-SecurityGroupRulesTestJSON-1769504197 tempest-SecurityGroupRulesTestJSON-1769504197-project] 10.222.0.16 "DELETE /compute/v2.1/os-security-groups/24258b25-667d-4e38-a209-c64d74832751" status: 202 len: 0 microversion: 2.1 time: 0.608450 Jul 16 21:11:03.912902 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 413/836] 10.222.0.16 () {58 vars in 1271 bytes} [Sat Jul 16 21:11:03 2022] DELETE /compute/v2.1/os-security-groups/24258b25-667d-4e38-a209-c64d74832751 => generated 0 bytes in 610 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Jul 16 21:11:03.920579 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-4fa76a07-5c03-406e-a538-930465650cfc tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:11:03.921674 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-4fa76a07-5c03-406e-a538-930465650cfc tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] [instance: de210dbd-1398-413f-9a0f-63cae300bd03] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:11:03.930794 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4fa76a07-5c03-406e-a538-930465650cfc tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:03.931520 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4fa76a07-5c03-406e-a538-930465650cfc tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:03.933358 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4fa76a07-5c03-406e-a538-930465650cfc tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:03.934596 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4fa76a07-5c03-406e-a538-930465650cfc tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:04.146761 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-f7443018-0404-41e2-a7be-4a03700e5c4f tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'abeb7020019d41a18de42956d706a011', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '649babe54d104436af461f1ee4767001', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:11:04.163886 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-f7443018-0404-41e2-a7be-4a03700e5c4f tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] 10.222.0.16 "GET /compute/v2.1/servers/a4a0aaec-dc64-400c-b3f0-ec61fa740b8d" status: 200 len: 1316 microversion: 2.1 time: 0.300292 Jul 16 21:11:04.165072 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 424/837] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:11:03 2022] GET /compute/v2.1/servers/a4a0aaec-dc64-400c-b3f0-ec61fa740b8d => generated 1316 bytes in 302 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:11:04.222885 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.objects.instance [None req-4fa76a07-5c03-406e-a538-930465650cfc tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] Lazy-loading 'services' on Instance uuid de210dbd-1398-413f-9a0f-63cae300bd03 {{(pid=138681) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:11:04.245096 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-4fa76a07-5c03-406e-a538-930465650cfc tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/de210dbd-1398-413f-9a0f-63cae300bd03" status: 200 len: 2050 microversion: 2.25 time: 0.330601 Jul 16 21:11:04.246387 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 414/838] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:11:03 2022] GET /compute/v2.1/servers/de210dbd-1398-413f-9a0f-63cae300bd03 => generated 2050 bytes in 333 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:11:04.263565 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-a87ec568-2799-4708-8df1-d7a333fb46ad tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:11:04.265442 ncdv-master-1 devstack@n-api.service[138680]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-services:list": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:11:04.265442 ncdv-master-1 devstack@n-api.service[138680]: warnings.warn(msg) Jul 16 21:11:04.266752 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a87ec568-2799-4708-8df1-d7a333fb46ad tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:04.267491 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a87ec568-2799-4708-8df1-d7a333fb46ad tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:04.268744 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a87ec568-2799-4708-8df1-d7a333fb46ad tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:04.269354 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a87ec568-2799-4708-8df1-d7a333fb46ad tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:04.333443 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-a87ec568-2799-4708-8df1-d7a333fb46ad tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] 10.222.0.16 "GET /compute/v2.1/os-services?binary=nova-compute" status: 200 len: 424 microversion: 2.25 time: 0.074158 Jul 16 21:11:04.338454 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 425/839] 10.222.0.16 () {60 vars in 1216 bytes} [Sat Jul 16 21:11:04 2022] GET /compute/v2.1/os-services?binary=nova-compute => generated 424 bytes in 76 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 16 21:11:04.346719 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-1460fa34-cee5-46bc-bd33-8ff913362ad5 tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] Action: 'action', calling method: >, body: {"os-migrateLive": {"host": "nch1-master-1", "block_migration": "auto"}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:11:04.347975 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-1460fa34-cee5-46bc-bd33-8ff913362ad5 tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] [instance: de210dbd-1398-413f-9a0f-63cae300bd03] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:11:04.369170 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1460fa34-cee5-46bc-bd33-8ff913362ad5 tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:04.369876 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1460fa34-cee5-46bc-bd33-8ff913362ad5 tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:04.370876 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1460fa34-cee5-46bc-bd33-8ff913362ad5 tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:04.371611 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1460fa34-cee5-46bc-bd33-8ff913362ad5 tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:04.414405 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-1fdcd14e-51f7-460e-89ee-390cbcbadca1 tempest-MigrationsAdminTest-399208122 tempest-MigrationsAdminTest-399208122-project-admin] Calling method '.version_select at 0x7fdd9d2429d0>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:11:04.421034 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1fdcd14e-51f7-460e-89ee-390cbcbadca1 tempest-MigrationsAdminTest-399208122 tempest-MigrationsAdminTest-399208122-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:04.423558 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1fdcd14e-51f7-460e-89ee-390cbcbadca1 tempest-MigrationsAdminTest-399208122 tempest-MigrationsAdminTest-399208122-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:04.440173 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-1fdcd14e-51f7-460e-89ee-390cbcbadca1 tempest-MigrationsAdminTest-399208122 tempest-MigrationsAdminTest-399208122-project-admin] 10.222.0.16 "GET /compute/v2.1/os-migrations" status: 200 len: 1570 microversion: 2.1 time: 0.030529 Jul 16 21:11:04.443084 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 426/840] 10.222.0.16 () {58 vars in 1142 bytes} [Sat Jul 16 21:11:04 2022] GET /compute/v2.1/os-migrations => generated 1570 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:11:04.472464 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-4cff9d37-0c1c-4e40-b737-80018b7478b8 tempest-MigrationsAdminTest-399208122 tempest-MigrationsAdminTest-399208122-project-admin] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:11:04.481110 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-4cff9d37-0c1c-4e40-b737-80018b7478b8 tempest-MigrationsAdminTest-399208122 tempest-MigrationsAdminTest-399208122-project-admin] 10.222.0.16 "GET /compute/v2.1/flavors/452" status: 200 len: 370 microversion: 2.1 time: 0.014909 Jul 16 21:11:04.482383 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 427/841] 10.222.0.16 () {58 vars in 1136 bytes} [Sat Jul 16 21:11:04 2022] GET /compute/v2.1/flavors/452 => generated 370 bytes in 17 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:11:04.494913 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-4d675b6e-9097-4211-b569-b4bfe63dd894 tempest-MigrationsAdminTest-399208122 tempest-MigrationsAdminTest-399208122-project-admin] Action: 'create', calling method: >, body: {"flavor": {"name": "tempest-test_resize_flavor_-1563016479", "ram": 1024, "disk": 1, "vcpus": 1}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:11:04.503912 ncdv-master-1 devstack@n-api.service[138680]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-flavor-manage:create": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:11:04.503912 ncdv-master-1 devstack@n-api.service[138680]: warnings.warn(msg) Jul 16 21:11:04.566823 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.objects.instance [None req-1460fa34-cee5-46bc-bd33-8ff913362ad5 tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] Lazy-loading 'flavor' on Instance uuid de210dbd-1398-413f-9a0f-63cae300bd03 {{(pid=138681) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:11:04.595927 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-4d675b6e-9097-4211-b569-b4bfe63dd894 tempest-MigrationsAdminTest-399208122 tempest-MigrationsAdminTest-399208122-project-admin] 10.222.0.16 "POST /compute/v2.1/flavors" status: 200 len: 499 microversion: 2.1 time: 0.105882 Jul 16 21:11:04.597206 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 428/842] 10.222.0.16 () {60 vars in 1145 bytes} [Sat Jul 16 21:11:04 2022] POST /compute/v2.1/flavors => generated 499 bytes in 108 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:11:04.608634 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-bdea54db-ac2d-4724-ae01-e63f1a81f465 tempest-MigrationsAdminTest-399208122 tempest-MigrationsAdminTest-399208122-project-admin] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:11:04.634004 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-1460fa34-cee5-46bc-bd33-8ff913362ad5 tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] [instance: de210dbd-1398-413f-9a0f-63cae300bd03] Going to try to live migrate instance to nch1-master-1 {{(pid=138681) live_migrate /opt/stack/nova/nova/compute/api.py:5349}} Jul 16 21:11:04.634629 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-bdea54db-ac2d-4724-ae01-e63f1a81f465 tempest-MigrationsAdminTest-399208122 tempest-MigrationsAdminTest-399208122-project-admin] 10.222.0.16 "GET /compute/v2.1/flavors/452/os-extra_specs" status: 200 len: 19 microversion: 2.1 time: 0.017994 Jul 16 21:11:04.634629 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 429/843] 10.222.0.16 () {58 vars in 1181 bytes} [Sat Jul 16 21:11:04 2022] GET /compute/v2.1/flavors/452/os-extra_specs => generated 19 bytes in 20 msecs (HTTP/1.1 200) 9 headers in 356 bytes (2 switches on core 0) Jul 16 21:11:04.743279 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-b797902d-805b-4acf-b637-51745e351450 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-MigrationsAdminTest-server-807100546", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "228f3e42-8394-4732-ac78-3c79d14592dd"}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:11:04.893688 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.network.neutron [None req-b797902d-805b-4acf-b637-51745e351450 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] validate_networks() for None {{(pid=138680) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 16 21:11:05.924100 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-b797902d-805b-4acf-b637-51745e351450 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Flavor limits 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 16 21:11:05.925494 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-b797902d-805b-4acf-b637-51745e351450 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Image limits 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 16 21:11:05.926156 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-b797902d-805b-4acf-b637-51745e351450 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Flavor pref 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 16 21:11:05.926578 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-b797902d-805b-4acf-b637-51745e351450 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Image pref 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 16 21:11:05.926999 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-b797902d-805b-4acf-b637-51745e351450 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 16 21:11:05.930267 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_api:SELECT=97,nova_api:UPDATE=1,nova_api:INSERT=2 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:11:05.953427 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b797902d-805b-4acf-b637-51745e351450 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:05.954625 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b797902d-805b-4acf-b637-51745e351450 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:05.956269 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b797902d-805b-4acf-b637-51745e351450 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:05.958298 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b797902d-805b-4acf-b637-51745e351450 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:06.004344 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-b797902d-805b-4acf-b637-51745e351450 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Getting quotas for project 55a19b7a4682457a9132f5fc80110712. Resources: {'ram', 'instances', 'cores'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:11:06.019615 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-b797902d-805b-4acf-b637-51745e351450 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Getting quotas for user a9b68dacc4b8463ba07fae6372131117 and project 55a19b7a4682457a9132f5fc80110712. Resources: {'ram', 'instances', 'cores'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:11:06.036210 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-b797902d-805b-4acf-b637-51745e351450 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Going to run 1 instances... {{(pid=138680) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 16 21:11:06.038114 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-b797902d-805b-4acf-b637-51745e351450 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138680) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 16 21:11:06.206313 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-b797902d-805b-4acf-b637-51745e351450 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 1.465675 Jul 16 21:11:06.216990 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 430/844] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 21:11:04 2022] POST /compute/v2.1/servers => generated 380 bytes in 1479 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Jul 16 21:11:06.256801 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-bbd309f9-e60a-4158-a1ce-c58c1370e9d1 req-d393f569-7758-4ffe-abfd-43db683df3ae service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "a4a0aaec-dc64-400c-b3f0-ec61fa740b8d", "name": "network-vif-plugged", "status": "completed", "tag": "34a8af9d-6c47-4062-aba8-98d7061acf17"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:11:06.290070 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-bbd309f9-e60a-4158-a1ce-c58c1370e9d1 req-d393f569-7758-4ffe-abfd-43db683df3ae service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:06.292958 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-bbd309f9-e60a-4158-a1ce-c58c1370e9d1 req-d393f569-7758-4ffe-abfd-43db683df3ae service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:06.330096 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-bbd309f9-e60a-4158-a1ce-c58c1370e9d1 req-d393f569-7758-4ffe-abfd-43db683df3ae service nova] Creating event network-vif-plugged:34a8af9d-6c47-4062-aba8-98d7061acf17 for instance a4a0aaec-dc64-400c-b3f0-ec61fa740b8d on nch1-master-1 Jul 16 21:11:06.348262 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-bbd309f9-e60a-4158-a1ce-c58c1370e9d1 req-d393f569-7758-4ffe-abfd-43db683df3ae service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.123138 Jul 16 21:11:06.350489 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 431/845] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:11:06 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 133 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:11:06.396141 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-1661fdc1-fb74-432f-a87f-0b6338a7db32 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:11:06.400746 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-1661fdc1-fb74-432f-a87f-0b6338a7db32 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] [instance: a4a0aaec-dc64-400c-b3f0-ec61fa740b8d] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:11:06.455771 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1661fdc1-fb74-432f-a87f-0b6338a7db32 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:06.456789 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1661fdc1-fb74-432f-a87f-0b6338a7db32 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:06.462382 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1661fdc1-fb74-432f-a87f-0b6338a7db32 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:06.474340 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1661fdc1-fb74-432f-a87f-0b6338a7db32 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:07.025384 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-1661fdc1-fb74-432f-a87f-0b6338a7db32 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'abeb7020019d41a18de42956d706a011', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '649babe54d104436af461f1ee4767001', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:11:07.051619 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-1661fdc1-fb74-432f-a87f-0b6338a7db32 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] 10.222.0.16 "GET /compute/v2.1/servers/a4a0aaec-dc64-400c-b3f0-ec61fa740b8d" status: 200 len: 1316 microversion: 2.1 time: 0.699295 Jul 16 21:11:07.053040 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 432/846] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:11:06 2022] GET /compute/v2.1/servers/a4a0aaec-dc64-400c-b3f0-ec61fa740b8d => generated 1316 bytes in 702 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:11:07.074126 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-ea8c390b-e68b-413e-97d6-fd89efcb1acf tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:11:07.075864 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-ea8c390b-e68b-413e-97d6-fd89efcb1acf tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:11:07.161994 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-ea8c390b-e68b-413e-97d6-fd89efcb1acf tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b68dacc4b8463ba07fae6372131117', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55a19b7a4682457a9132f5fc80110712', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:11:07.170301 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-ea8c390b-e68b-413e-97d6-fd89efcb1acf tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "GET /compute/v2.1/servers/a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2" status: 200 len: 1278 microversion: 2.1 time: 0.115695 Jul 16 21:11:07.170885 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 433/847] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:11:07 2022] GET /compute/v2.1/servers/a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2 => generated 1278 bytes in 117 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:11:07.529255 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-a70561ba-1ebb-46dd-8a43-c5b6f1fb2411 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-LiveMigrationTest-server-1712991119", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452", "networks": [{"uuid": "26ad28eb-6622-4b58-ad05-2de54c33be8d"}], "security_groups": [{"name": "tempest-securitygroup--1147806589"}], "key_name": "tempest-keypair-1244080767", "user_data": "IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIHVidW50dSB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH51YnVudHUvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo="}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:11:07.695959 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.network.neutron [None req-a70561ba-1ebb-46dd-8a43-c5b6f1fb2411 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] validate_networks() for [('26ad28eb-6622-4b58-ad05-2de54c33be8d', None, None, None, None, None)] {{(pid=138680) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 16 21:11:08.970815 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-a70561ba-1ebb-46dd-8a43-c5b6f1fb2411 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Flavor limits 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 16 21:11:08.971515 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-a70561ba-1ebb-46dd-8a43-c5b6f1fb2411 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Image limits 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 16 21:11:08.971724 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-a70561ba-1ebb-46dd-8a43-c5b6f1fb2411 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Flavor pref 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 16 21:11:08.972313 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-a70561ba-1ebb-46dd-8a43-c5b6f1fb2411 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Image pref 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 16 21:11:08.972915 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-a70561ba-1ebb-46dd-8a43-c5b6f1fb2411 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 16 21:11:08.990878 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a70561ba-1ebb-46dd-8a43-c5b6f1fb2411 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:08.991588 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a70561ba-1ebb-46dd-8a43-c5b6f1fb2411 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:08.992599 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a70561ba-1ebb-46dd-8a43-c5b6f1fb2411 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:08.993040 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a70561ba-1ebb-46dd-8a43-c5b6f1fb2411 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:09.050752 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-a70561ba-1ebb-46dd-8a43-c5b6f1fb2411 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Getting quotas for project 9b769bf8f3ea4e368bf68de1df6538f9. Resources: {'ram', 'instances', 'cores'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:11:09.062309 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-a70561ba-1ebb-46dd-8a43-c5b6f1fb2411 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Getting quotas for user 6a69f9dee03d43beaafa966216b9e289 and project 9b769bf8f3ea4e368bf68de1df6538f9. Resources: {'ram', 'instances', 'cores'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:11:09.082895 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-a70561ba-1ebb-46dd-8a43-c5b6f1fb2411 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Going to run 1 instances... {{(pid=138680) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 16 21:11:09.085602 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-a70561ba-1ebb-46dd-8a43-c5b6f1fb2411 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] [instance: 6a391156-ffce-49c0-a79d-47b4544264f5] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138680) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 16 21:11:09.367138 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-a70561ba-1ebb-46dd-8a43-c5b6f1fb2411 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 406 microversion: 2.1 time: 1.846120 Jul 16 21:11:09.374136 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 434/848] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 21:11:07 2022] POST /compute/v2.1/servers => generated 406 bytes in 1854 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Jul 16 21:11:09.379661 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-317209b4-d5d5-4f4c-850f-b5a117fa121f tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:11:09.397498 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-317209b4-d5d5-4f4c-850f-b5a117fa121f tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] [instance: a4a0aaec-dc64-400c-b3f0-ec61fa740b8d] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:11:09.424030 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-317209b4-d5d5-4f4c-850f-b5a117fa121f tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:09.424901 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-317209b4-d5d5-4f4c-850f-b5a117fa121f tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:09.425956 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-317209b4-d5d5-4f4c-850f-b5a117fa121f tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:09.427786 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-317209b4-d5d5-4f4c-850f-b5a117fa121f tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:09.439324 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-1460fa34-cee5-46bc-bd33-8ff913362ad5 tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] 10.222.0.16 "POST /compute/v2.1/servers/de210dbd-1398-413f-9a0f-63cae300bd03/action" status: 202 len: 0 microversion: 2.25 time: 5.097013 Jul 16 21:11:09.439970 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 415/849] 10.222.0.16 () {62 vars in 1318 bytes} [Sat Jul 16 21:11:04 2022] POST /compute/v2.1/servers/de210dbd-1398-413f-9a0f-63cae300bd03/action => generated 0 bytes in 5099 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Jul 16 21:11:09.933200 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-7f81b47c-090d-487b-ab20-476b2f22a6d5 tempest-ServicesAdminTestJSON-1357983847 tempest-ServicesAdminTestJSON-1357983847-project-admin] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:11:09.935967 ncdv-master-1 devstack@n-api.service[138681]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-services:list": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:11:09.935967 ncdv-master-1 devstack@n-api.service[138681]: warnings.warn(msg) Jul 16 21:11:09.937605 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7f81b47c-090d-487b-ab20-476b2f22a6d5 tempest-ServicesAdminTestJSON-1357983847 tempest-ServicesAdminTestJSON-1357983847-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:09.943294 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7f81b47c-090d-487b-ab20-476b2f22a6d5 tempest-ServicesAdminTestJSON-1357983847 tempest-ServicesAdminTestJSON-1357983847-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:09.948031 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7f81b47c-090d-487b-ab20-476b2f22a6d5 tempest-ServicesAdminTestJSON-1357983847 tempest-ServicesAdminTestJSON-1357983847-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:09.951160 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7f81b47c-090d-487b-ab20-476b2f22a6d5 tempest-ServicesAdminTestJSON-1357983847 tempest-ServicesAdminTestJSON-1357983847-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:10.043792 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-7f81b47c-090d-487b-ab20-476b2f22a6d5 tempest-ServicesAdminTestJSON-1357983847 tempest-ServicesAdminTestJSON-1357983847-project-admin] 10.222.0.16 "GET /compute/v2.1/os-services" status: 200 len: 947 microversion: 2.1 time: 0.601487 Jul 16 21:11:10.045953 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 416/850] 10.222.0.16 () {58 vars in 1136 bytes} [Sat Jul 16 21:11:09 2022] GET /compute/v2.1/os-services => generated 947 bytes in 605 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:11:10.056996 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-23de8739-ebff-4bfd-8513-329ddfc3ac2a tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:11:10.058737 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-23de8739-ebff-4bfd-8513-329ddfc3ac2a tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:11:10.079108 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-23de8739-ebff-4bfd-8513-329ddfc3ac2a tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:10.079596 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-23de8739-ebff-4bfd-8513-329ddfc3ac2a tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:10.080424 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-23de8739-ebff-4bfd-8513-329ddfc3ac2a tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:10.080884 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-23de8739-ebff-4bfd-8513-329ddfc3ac2a tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:10.236910 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-317209b4-d5d5-4f4c-850f-b5a117fa121f tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'abeb7020019d41a18de42956d706a011', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '649babe54d104436af461f1ee4767001', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:11:10.248856 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-317209b4-d5d5-4f4c-850f-b5a117fa121f tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] 10.222.0.16 "GET /compute/v2.1/servers/a4a0aaec-dc64-400c-b3f0-ec61fa740b8d" status: 200 len: 1466 microversion: 2.1 time: 0.873078 Jul 16 21:11:10.250692 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 435/851] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:11:09 2022] GET /compute/v2.1/servers/a4a0aaec-dc64-400c-b3f0-ec61fa740b8d => generated 1466 bytes in 875 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:11:10.256734 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-e898a2c5-2b8d-44d7-aeba-97d96680203f tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:11:10.264760 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-e898a2c5-2b8d-44d7-aeba-97d96680203f tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] [instance: 8872e8cf-593e-4a7a-940d-e0c8159d4ad7] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:11:10.281517 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-23de8739-ebff-4bfd-8513-329ddfc3ac2a tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b68dacc4b8463ba07fae6372131117', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55a19b7a4682457a9132f5fc80110712', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:11:10.285509 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e898a2c5-2b8d-44d7-aeba-97d96680203f tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:10.285509 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e898a2c5-2b8d-44d7-aeba-97d96680203f tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:10.285509 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e898a2c5-2b8d-44d7-aeba-97d96680203f tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:10.287625 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e898a2c5-2b8d-44d7-aeba-97d96680203f tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:10.293743 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-23de8739-ebff-4bfd-8513-329ddfc3ac2a tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "GET /compute/v2.1/servers/a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2" status: 200 len: 1336 microversion: 2.1 time: 0.245594 Jul 16 21:11:10.298817 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 417/852] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:11:10 2022] GET /compute/v2.1/servers/a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2 => generated 1336 bytes in 252 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:11:10.303637 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-9dd6175d-f368-4645-ba3b-18ed7c154b81 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:11:10.308476 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-9dd6175d-f368-4645-ba3b-18ed7c154b81 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] [instance: 6a391156-ffce-49c0-a79d-47b4544264f5] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:11:10.339262 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-e898a2c5-2b8d-44d7-aeba-97d96680203f tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] [instance: 8872e8cf-593e-4a7a-940d-e0c8159d4ad7] Going to try to terminate instance {{(pid=138680) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 16 21:11:10.428152 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-9dd6175d-f368-4645-ba3b-18ed7c154b81 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a69f9dee03d43beaafa966216b9e289', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9b769bf8f3ea4e368bf68de1df6538f9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:11:10.437390 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-9dd6175d-f368-4645-ba3b-18ed7c154b81 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] 10.222.0.16 "GET /compute/v2.1/servers/6a391156-ffce-49c0-a79d-47b4544264f5" status: 200 len: 1235 microversion: 2.1 time: 0.136510 Jul 16 21:11:10.438528 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 418/853] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:11:10 2022] GET /compute/v2.1/servers/6a391156-ffce-49c0-a79d-47b4544264f5 => generated 1235 bytes in 139 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:11:10.446496 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-272d24d9-de89-4dd5-a6f2-1251adb92946 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:11:10.446496 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-272d24d9-de89-4dd5-a6f2-1251adb92946 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] [instance: de210dbd-1398-413f-9a0f-63cae300bd03] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:11:10.452934 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-272d24d9-de89-4dd5-a6f2-1251adb92946 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:10.453667 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-272d24d9-de89-4dd5-a6f2-1251adb92946 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:10.454690 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-272d24d9-de89-4dd5-a6f2-1251adb92946 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:10.455353 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-272d24d9-de89-4dd5-a6f2-1251adb92946 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:10.722891 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-272d24d9-de89-4dd5-a6f2-1251adb92946 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:11:10.735041 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-272d24d9-de89-4dd5-a6f2-1251adb92946 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:11:10.737069 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-272d24d9-de89-4dd5-a6f2-1251adb92946 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:11:10.738706 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-272d24d9-de89-4dd5-a6f2-1251adb92946 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] 10.222.0.16 "GET /compute/v2.1/servers/de210dbd-1398-413f-9a0f-63cae300bd03" status: 200 len: 1567 microversion: 2.25 time: 0.298908 Jul 16 21:11:10.753106 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 419/854] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:11:10 2022] GET /compute/v2.1/servers/de210dbd-1398-413f-9a0f-63cae300bd03 => generated 1567 bytes in 314 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:11:10.758467 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-e3720129-b880-4774-8af1-17bb6ab39921 tempest-ServicesAdminTestJSON-1357983847 tempest-ServicesAdminTestJSON-1357983847-project-admin] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:11:10.760304 ncdv-master-1 devstack@n-api.service[138681]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-services:list": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:11:10.760304 ncdv-master-1 devstack@n-api.service[138681]: warnings.warn(msg) Jul 16 21:11:10.761960 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e3720129-b880-4774-8af1-17bb6ab39921 tempest-ServicesAdminTestJSON-1357983847 tempest-ServicesAdminTestJSON-1357983847-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:10.763532 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e3720129-b880-4774-8af1-17bb6ab39921 tempest-ServicesAdminTestJSON-1357983847 tempest-ServicesAdminTestJSON-1357983847-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:10.765085 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e3720129-b880-4774-8af1-17bb6ab39921 tempest-ServicesAdminTestJSON-1357983847 tempest-ServicesAdminTestJSON-1357983847-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:10.765827 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e3720129-b880-4774-8af1-17bb6ab39921 tempest-ServicesAdminTestJSON-1357983847 tempest-ServicesAdminTestJSON-1357983847-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:10.777839 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-e898a2c5-2b8d-44d7-aeba-97d96680203f tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] 10.222.0.16 "DELETE /compute/v2.1/servers/8872e8cf-593e-4a7a-940d-e0c8159d4ad7" status: 204 len: 0 microversion: 2.1 time: 0.525690 Jul 16 21:11:10.783986 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 436/855] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:11:10 2022] DELETE /compute/v2.1/servers/8872e8cf-593e-4a7a-940d-e0c8159d4ad7 => generated 0 bytes in 528 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 16 21:11:10.789007 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-aca53418-9257-4ff2-82bd-d46326acd9e3 tempest-ServersOnMultiNodesTest-83775055 tempest-ServersOnMultiNodesTest-83775055-project-admin] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:11:10.790071 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-aca53418-9257-4ff2-82bd-d46326acd9e3 tempest-ServersOnMultiNodesTest-83775055 tempest-ServersOnMultiNodesTest-83775055-project-admin] [instance: a4a0aaec-dc64-400c-b3f0-ec61fa740b8d] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:11:10.802175 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-e3720129-b880-4774-8af1-17bb6ab39921 tempest-ServicesAdminTestJSON-1357983847 tempest-ServicesAdminTestJSON-1357983847-project-admin] 10.222.0.16 "GET /compute/v2.1/os-services?host=ncdv-master-1" status: 200 len: 581 microversion: 2.1 time: 0.047590 Jul 16 21:11:10.803288 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 420/856] 10.222.0.16 () {58 vars in 1173 bytes} [Sat Jul 16 21:11:10 2022] GET /compute/v2.1/os-services?host=ncdv-master-1 => generated 581 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:11:10.815184 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-aca53418-9257-4ff2-82bd-d46326acd9e3 tempest-ServersOnMultiNodesTest-83775055 tempest-ServersOnMultiNodesTest-83775055-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:10.815704 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-aca53418-9257-4ff2-82bd-d46326acd9e3 tempest-ServersOnMultiNodesTest-83775055 tempest-ServersOnMultiNodesTest-83775055-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:10.817014 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-2049e419-24c6-40e4-ba4d-0b7e2bec5a87 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:11:10.820348 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-aca53418-9257-4ff2-82bd-d46326acd9e3 tempest-ServersOnMultiNodesTest-83775055 tempest-ServersOnMultiNodesTest-83775055-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:10.820474 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-2049e419-24c6-40e4-ba4d-0b7e2bec5a87 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] [instance: 8872e8cf-593e-4a7a-940d-e0c8159d4ad7] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:11:10.820556 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-aca53418-9257-4ff2-82bd-d46326acd9e3 tempest-ServersOnMultiNodesTest-83775055 tempest-ServersOnMultiNodesTest-83775055-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:10.826901 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2049e419-24c6-40e4-ba4d-0b7e2bec5a87 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:10.827499 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2049e419-24c6-40e4-ba4d-0b7e2bec5a87 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:10.828514 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2049e419-24c6-40e4-ba4d-0b7e2bec5a87 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:10.829072 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2049e419-24c6-40e4-ba4d-0b7e2bec5a87 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:11.196061 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-aca53418-9257-4ff2-82bd-d46326acd9e3 tempest-ServersOnMultiNodesTest-83775055 tempest-ServersOnMultiNodesTest-83775055-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/a4a0aaec-dc64-400c-b3f0-ec61fa740b8d" status: 200 len: 1617 microversion: 2.1 time: 0.413597 Jul 16 21:11:11.197071 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 437/857] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:11:10 2022] GET /compute/v2.1/servers/a4a0aaec-dc64-400c-b3f0-ec61fa740b8d => generated 1617 bytes in 417 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:11:11.207459 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-f1308ab3-74bb-4f78-b908-a4d2d3b76086 tempest-ServicesAdminTestJSON-1357983847 tempest-ServicesAdminTestJSON-1357983847-project-admin] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:11:11.209255 ncdv-master-1 devstack@n-api.service[138680]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-services:list": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:11:11.209255 ncdv-master-1 devstack@n-api.service[138680]: warnings.warn(msg) Jul 16 21:11:11.211630 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f1308ab3-74bb-4f78-b908-a4d2d3b76086 tempest-ServicesAdminTestJSON-1357983847 tempest-ServicesAdminTestJSON-1357983847-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:11.212431 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f1308ab3-74bb-4f78-b908-a4d2d3b76086 tempest-ServicesAdminTestJSON-1357983847 tempest-ServicesAdminTestJSON-1357983847-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:11.220196 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f1308ab3-74bb-4f78-b908-a4d2d3b76086 tempest-ServicesAdminTestJSON-1357983847 tempest-ServicesAdminTestJSON-1357983847-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:11.221969 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f1308ab3-74bb-4f78-b908-a4d2d3b76086 tempest-ServicesAdminTestJSON-1357983847 tempest-ServicesAdminTestJSON-1357983847-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:11.283076 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-f1308ab3-74bb-4f78-b908-a4d2d3b76086 tempest-ServicesAdminTestJSON-1357983847 tempest-ServicesAdminTestJSON-1357983847-project-admin] 10.222.0.16 "GET /compute/v2.1/os-services?binary=nova-compute" status: 200 len: 380 microversion: 2.1 time: 0.082582 Jul 16 21:11:11.284988 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 438/858] 10.222.0.16 () {58 vars in 1175 bytes} [Sat Jul 16 21:11:11 2022] GET /compute/v2.1/os-services?binary=nova-compute => generated 380 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:11:11.303712 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-911e06ca-9edb-45ce-a11c-f49bffb1f70f tempest-ServicesAdminTestJSON-1357983847 tempest-ServicesAdminTestJSON-1357983847-project-admin] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:11:11.305184 ncdv-master-1 devstack@n-api.service[138680]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-services:list": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:11:11.305184 ncdv-master-1 devstack@n-api.service[138680]: warnings.warn(msg) Jul 16 21:11:11.306645 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-911e06ca-9edb-45ce-a11c-f49bffb1f70f tempest-ServicesAdminTestJSON-1357983847 tempest-ServicesAdminTestJSON-1357983847-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:11.307229 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-911e06ca-9edb-45ce-a11c-f49bffb1f70f tempest-ServicesAdminTestJSON-1357983847 tempest-ServicesAdminTestJSON-1357983847-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:11.308121 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-911e06ca-9edb-45ce-a11c-f49bffb1f70f tempest-ServicesAdminTestJSON-1357983847 tempest-ServicesAdminTestJSON-1357983847-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:11.308748 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-911e06ca-9edb-45ce-a11c-f49bffb1f70f tempest-ServicesAdminTestJSON-1357983847 tempest-ServicesAdminTestJSON-1357983847-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:11.330186 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-2049e419-24c6-40e4-ba4d-0b7e2bec5a87 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ef9cbce2b2e64665b7f50767342dcb46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0ee02e4c8b448f2b77798ae504129b9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:11:11.343126 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-2049e419-24c6-40e4-ba4d-0b7e2bec5a87 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] 10.222.0.16 "GET /compute/v2.1/servers/8872e8cf-593e-4a7a-940d-e0c8159d4ad7" status: 200 len: 1713 microversion: 2.1 time: 0.537747 Jul 16 21:11:11.346527 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 421/859] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:11:10 2022] GET /compute/v2.1/servers/8872e8cf-593e-4a7a-940d-e0c8159d4ad7 => generated 1713 bytes in 542 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:11:11.351960 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-2530a80d-d1fb-4b77-8569-325ef2d33c1a tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:11:11.354506 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-2530a80d-d1fb-4b77-8569-325ef2d33c1a tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:11:11.366651 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-911e06ca-9edb-45ce-a11c-f49bffb1f70f tempest-ServicesAdminTestJSON-1357983847 tempest-ServicesAdminTestJSON-1357983847-project-admin] 10.222.0.16 "GET /compute/v2.1/os-services" status: 200 len: 947 microversion: 2.1 time: 0.068315 Jul 16 21:11:11.367325 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 439/860] 10.222.0.16 () {58 vars in 1136 bytes} [Sat Jul 16 21:11:11 2022] GET /compute/v2.1/os-services => generated 947 bytes in 70 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 16 21:11:11.375346 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2530a80d-d1fb-4b77-8569-325ef2d33c1a tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:11.375346 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2530a80d-d1fb-4b77-8569-325ef2d33c1a tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:11.375346 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2530a80d-d1fb-4b77-8569-325ef2d33c1a tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:11.375346 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2530a80d-d1fb-4b77-8569-325ef2d33c1a tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:11.411032 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-02791a17-aaf2-442d-b226-de65ffa96eac tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServersOnMultiNodesTest-server-1033668018", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452"}, "os:scheduler_hints": {"same_host": "148d5ad9-2900-4c2b-909b-aa446757e33d"}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:11:11.478679 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.network.neutron [None req-02791a17-aaf2-442d-b226-de65ffa96eac tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] validate_networks() for None {{(pid=138680) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 16 21:11:11.517265 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-2530a80d-d1fb-4b77-8569-325ef2d33c1a tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b68dacc4b8463ba07fae6372131117', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55a19b7a4682457a9132f5fc80110712', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:11:11.530962 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-2530a80d-d1fb-4b77-8569-325ef2d33c1a tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "GET /compute/v2.1/servers/a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2" status: 200 len: 1336 microversion: 2.1 time: 0.183151 Jul 16 21:11:11.532052 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 422/861] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:11:11 2022] GET /compute/v2.1/servers/a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2 => generated 1336 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:11:11.545507 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-c3281779-f752-425e-92dd-08560bbb9117 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:11:11.546689 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-c3281779-f752-425e-92dd-08560bbb9117 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] [instance: 6a391156-ffce-49c0-a79d-47b4544264f5] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:11:11.555864 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c3281779-f752-425e-92dd-08560bbb9117 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:11.557187 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c3281779-f752-425e-92dd-08560bbb9117 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:11.560298 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c3281779-f752-425e-92dd-08560bbb9117 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:11.561099 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c3281779-f752-425e-92dd-08560bbb9117 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:11.691802 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-c3281779-f752-425e-92dd-08560bbb9117 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a69f9dee03d43beaafa966216b9e289', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9b769bf8f3ea4e368bf68de1df6538f9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:11:11.702782 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-c3281779-f752-425e-92dd-08560bbb9117 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] 10.222.0.16 "GET /compute/v2.1/servers/6a391156-ffce-49c0-a79d-47b4544264f5" status: 200 len: 1239 microversion: 2.1 time: 0.169725 Jul 16 21:11:11.719231 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 423/862] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:11:11 2022] GET /compute/v2.1/servers/6a391156-ffce-49c0-a79d-47b4544264f5 => generated 1239 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:11:11.760446 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-2be865b2-63f4-450c-85b8-48b847d8e367 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:11:11.761178 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-2be865b2-63f4-450c-85b8-48b847d8e367 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] [instance: de210dbd-1398-413f-9a0f-63cae300bd03] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:11:11.773394 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2be865b2-63f4-450c-85b8-48b847d8e367 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:11.793090 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2be865b2-63f4-450c-85b8-48b847d8e367 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.020s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:11.794419 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2be865b2-63f4-450c-85b8-48b847d8e367 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:11.804255 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2be865b2-63f4-450c-85b8-48b847d8e367 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.009s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:12.171166 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-02791a17-aaf2-442d-b226-de65ffa96eac tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Flavor limits 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 16 21:11:12.171828 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-02791a17-aaf2-442d-b226-de65ffa96eac tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Image limits 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 16 21:11:12.172559 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-02791a17-aaf2-442d-b226-de65ffa96eac tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Flavor pref 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 16 21:11:12.173228 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-02791a17-aaf2-442d-b226-de65ffa96eac tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Image pref 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 16 21:11:12.173892 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-02791a17-aaf2-442d-b226-de65ffa96eac tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 16 21:11:12.196773 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-02791a17-aaf2-442d-b226-de65ffa96eac tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:12.199644 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-02791a17-aaf2-442d-b226-de65ffa96eac tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:12.202168 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-02791a17-aaf2-442d-b226-de65ffa96eac tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:12.204994 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-02791a17-aaf2-442d-b226-de65ffa96eac tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:12.238102 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-02791a17-aaf2-442d-b226-de65ffa96eac tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Getting quotas for project 649babe54d104436af461f1ee4767001. Resources: {'ram', 'instances', 'cores'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:11:12.245674 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-02791a17-aaf2-442d-b226-de65ffa96eac tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Getting quotas for user abeb7020019d41a18de42956d706a011 and project 649babe54d104436af461f1ee4767001. Resources: {'ram', 'instances', 'cores'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:11:12.287300 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-02791a17-aaf2-442d-b226-de65ffa96eac tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Going to run 1 instances... {{(pid=138680) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 16 21:11:12.290936 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-02791a17-aaf2-442d-b226-de65ffa96eac tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] [instance: 65a70cca-b978-41ab-b890-6052f653c5c3] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138680) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 16 21:11:12.448665 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-2be865b2-63f4-450c-85b8-48b847d8e367 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:11:12.465065 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-02791a17-aaf2-442d-b226-de65ffa96eac tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 1.059476 Jul 16 21:11:12.467689 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-2be865b2-63f4-450c-85b8-48b847d8e367 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:11:12.471157 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 440/863] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 21:11:11 2022] POST /compute/v2.1/servers => generated 380 bytes in 1066 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 16 21:11:12.473708 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-2be865b2-63f4-450c-85b8-48b847d8e367 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:11:12.482066 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-2be865b2-63f4-450c-85b8-48b847d8e367 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] 10.222.0.16 "GET /compute/v2.1/servers/de210dbd-1398-413f-9a0f-63cae300bd03" status: 200 len: 1567 microversion: 2.25 time: 0.725472 Jul 16 21:11:12.485433 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 424/864] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:11:11 2022] GET /compute/v2.1/servers/de210dbd-1398-413f-9a0f-63cae300bd03 => generated 1567 bytes in 730 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:11:12.495705 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-1ca794f2-0407-4d6e-8d80-3ca8228754be tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:11:12.502864 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-1ca794f2-0407-4d6e-8d80-3ca8228754be tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] [instance: 8872e8cf-593e-4a7a-940d-e0c8159d4ad7] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:11:12.521368 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-d492200d-a365-4250-b4eb-1f498ae9a253 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:11:12.526109 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-d492200d-a365-4250-b4eb-1f498ae9a253 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] [instance: 65a70cca-b978-41ab-b890-6052f653c5c3] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:11:12.549190 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1ca794f2-0407-4d6e-8d80-3ca8228754be tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:12.549190 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1ca794f2-0407-4d6e-8d80-3ca8228754be tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:12.557280 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1ca794f2-0407-4d6e-8d80-3ca8228754be tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:12.560416 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1ca794f2-0407-4d6e-8d80-3ca8228754be tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.010s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:12.600278 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:11:12.686559 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-d492200d-a365-4250-b4eb-1f498ae9a253 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'abeb7020019d41a18de42956d706a011', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '649babe54d104436af461f1ee4767001', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:11:12.695494 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-d492200d-a365-4250-b4eb-1f498ae9a253 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] 10.222.0.16 "GET /compute/v2.1/servers/65a70cca-b978-41ab-b890-6052f653c5c3" status: 200 len: 1217 microversion: 2.1 time: 0.208761 Jul 16 21:11:12.696370 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 425/865] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:11:12 2022] GET /compute/v2.1/servers/65a70cca-b978-41ab-b890-6052f653c5c3 => generated 1217 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:11:12.709731 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-321148f7-6f5e-479e-a8cb-26e1e20b7470 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:11:12.711886 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-321148f7-6f5e-479e-a8cb-26e1e20b7470 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:11:12.724619 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-321148f7-6f5e-479e-a8cb-26e1e20b7470 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:12.725244 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-321148f7-6f5e-479e-a8cb-26e1e20b7470 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:12.732407 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-321148f7-6f5e-479e-a8cb-26e1e20b7470 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:12.732407 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-321148f7-6f5e-479e-a8cb-26e1e20b7470 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:13.237586 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-321148f7-6f5e-479e-a8cb-26e1e20b7470 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b68dacc4b8463ba07fae6372131117', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55a19b7a4682457a9132f5fc80110712', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:11:13.250183 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-321148f7-6f5e-479e-a8cb-26e1e20b7470 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "GET /compute/v2.1/servers/a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2" status: 200 len: 1378 microversion: 2.1 time: 0.551026 Jul 16 21:11:13.261039 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 426/866] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:11:12 2022] GET /compute/v2.1/servers/a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2 => generated 1378 bytes in 561 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:11:13.271680 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-b43e8ebb-a7f9-4be3-bf2e-d411e045003d tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:11:13.272222 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-b43e8ebb-a7f9-4be3-bf2e-d411e045003d tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] [instance: 6a391156-ffce-49c0-a79d-47b4544264f5] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:11:13.274139 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_api:SELECT=96,nova_api:INSERT=4 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:11:13.284143 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b43e8ebb-a7f9-4be3-bf2e-d411e045003d tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:13.284956 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b43e8ebb-a7f9-4be3-bf2e-d411e045003d tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:13.285958 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b43e8ebb-a7f9-4be3-bf2e-d411e045003d tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:13.289839 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b43e8ebb-a7f9-4be3-bf2e-d411e045003d tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:13.297467 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-1ca794f2-0407-4d6e-8d80-3ca8228754be tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ef9cbce2b2e64665b7f50767342dcb46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0ee02e4c8b448f2b77798ae504129b9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:11:13.305996 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-1ca794f2-0407-4d6e-8d80-3ca8228754be tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] 10.222.0.16 "GET /compute/v2.1/servers/8872e8cf-593e-4a7a-940d-e0c8159d4ad7" status: 200 len: 1713 microversion: 2.1 time: 0.832741 Jul 16 21:11:13.306600 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 441/867] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:11:12 2022] GET /compute/v2.1/servers/8872e8cf-593e-4a7a-940d-e0c8159d4ad7 => generated 1713 bytes in 835 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:11:13.377101 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-b43e8ebb-a7f9-4be3-bf2e-d411e045003d tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a69f9dee03d43beaafa966216b9e289', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9b769bf8f3ea4e368bf68de1df6538f9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:11:13.379738 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:11:13.388635 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-b43e8ebb-a7f9-4be3-bf2e-d411e045003d tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] 10.222.0.16 "GET /compute/v2.1/servers/6a391156-ffce-49c0-a79d-47b4544264f5" status: 200 len: 1287 microversion: 2.1 time: 0.127282 Jul 16 21:11:13.389242 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 427/868] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:11:13 2022] GET /compute/v2.1/servers/6a391156-ffce-49c0-a79d-47b4544264f5 => generated 1287 bytes in 129 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:11:13.502119 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-5d0aa62c-9830-46cc-95d2-54e29ede55eb tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:11:13.503374 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-5d0aa62c-9830-46cc-95d2-54e29ede55eb tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] [instance: de210dbd-1398-413f-9a0f-63cae300bd03] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:11:13.511377 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5d0aa62c-9830-46cc-95d2-54e29ede55eb tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:13.511955 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5d0aa62c-9830-46cc-95d2-54e29ede55eb tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:13.513672 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5d0aa62c-9830-46cc-95d2-54e29ede55eb tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:13.514447 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5d0aa62c-9830-46cc-95d2-54e29ede55eb tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:13.714236 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-b55b45b4-a8d5-4add-94a9-9b6e1f4be85a tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:11:13.714874 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-b55b45b4-a8d5-4add-94a9-9b6e1f4be85a tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] [instance: 65a70cca-b978-41ab-b890-6052f653c5c3] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:11:13.809814 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-b55b45b4-a8d5-4add-94a9-9b6e1f4be85a tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'abeb7020019d41a18de42956d706a011', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '649babe54d104436af461f1ee4767001', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:11:13.822905 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-b55b45b4-a8d5-4add-94a9-9b6e1f4be85a tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] 10.222.0.16 "GET /compute/v2.1/servers/65a70cca-b978-41ab-b890-6052f653c5c3" status: 200 len: 1217 microversion: 2.1 time: 0.110379 Jul 16 21:11:13.827067 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 428/869] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:11:13 2022] GET /compute/v2.1/servers/65a70cca-b978-41ab-b890-6052f653c5c3 => generated 1217 bytes in 117 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:11:13.880460 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-5d0aa62c-9830-46cc-95d2-54e29ede55eb tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:11:13.893339 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-5d0aa62c-9830-46cc-95d2-54e29ede55eb tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:11:13.898272 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-5d0aa62c-9830-46cc-95d2-54e29ede55eb tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:11:13.902866 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-5d0aa62c-9830-46cc-95d2-54e29ede55eb tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] 10.222.0.16 "GET /compute/v2.1/servers/de210dbd-1398-413f-9a0f-63cae300bd03" status: 200 len: 1567 microversion: 2.25 time: 0.405540 Jul 16 21:11:13.908802 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 442/870] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:11:13 2022] GET /compute/v2.1/servers/de210dbd-1398-413f-9a0f-63cae300bd03 => generated 1567 bytes in 412 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:11:14.144133 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-f800914c-df12-46a4-869a-bed9f5ebb659 req-46e69b94-d22a-443a-93ae-66fadb009b33 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2", "tag": "bf9d54d7-5718-4678-bc2f-5ba8241bd855"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:11:14.158656 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-f800914c-df12-46a4-869a-bed9f5ebb659 req-46e69b94-d22a-443a-93ae-66fadb009b33 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:14.160822 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-f800914c-df12-46a4-869a-bed9f5ebb659 req-46e69b94-d22a-443a-93ae-66fadb009b33 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:14.190434 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-f800914c-df12-46a4-869a-bed9f5ebb659 req-46e69b94-d22a-443a-93ae-66fadb009b33 service nova] Creating event network-changed:bf9d54d7-5718-4678-bc2f-5ba8241bd855 for instance a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2 on nch1-master-1 Jul 16 21:11:14.202038 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-f800914c-df12-46a4-869a-bed9f5ebb659 req-46e69b94-d22a-443a-93ae-66fadb009b33 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.063797 Jul 16 21:11:14.204704 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 429/871] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:11:14 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 68 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:11:14.269230 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-a423f3e6-0fa3-44ed-b4cb-8e97a23c06be tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:11:14.270069 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-a423f3e6-0fa3-44ed-b4cb-8e97a23c06be tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:11:14.285308 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a423f3e6-0fa3-44ed-b4cb-8e97a23c06be tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:14.286106 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a423f3e6-0fa3-44ed-b4cb-8e97a23c06be tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:14.287144 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a423f3e6-0fa3-44ed-b4cb-8e97a23c06be tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:14.287754 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a423f3e6-0fa3-44ed-b4cb-8e97a23c06be tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:14.330311 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-2843d0d2-2eae-4c28-9871-943a0b72c0c5 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:11:14.332030 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-2843d0d2-2eae-4c28-9871-943a0b72c0c5 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] [instance: 8872e8cf-593e-4a7a-940d-e0c8159d4ad7] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:11:14.342594 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2843d0d2-2eae-4c28-9871-943a0b72c0c5 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:14.342976 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2843d0d2-2eae-4c28-9871-943a0b72c0c5 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:14.343827 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2843d0d2-2eae-4c28-9871-943a0b72c0c5 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:14.344586 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2843d0d2-2eae-4c28-9871-943a0b72c0c5 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:14.698505 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-a423f3e6-0fa3-44ed-b4cb-8e97a23c06be tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b68dacc4b8463ba07fae6372131117', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55a19b7a4682457a9132f5fc80110712', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:11:14.710624 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-a423f3e6-0fa3-44ed-b4cb-8e97a23c06be tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "GET /compute/v2.1/servers/a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2" status: 200 len: 1378 microversion: 2.1 time: 0.446676 Jul 16 21:11:14.711912 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 443/872] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:11:14 2022] GET /compute/v2.1/servers/a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2 => generated 1378 bytes in 449 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:11:14.718889 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-1a8fce07-dca8-4f97-8350-7783753c546e tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:11:14.720095 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-1a8fce07-dca8-4f97-8350-7783753c546e tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] [instance: 6a391156-ffce-49c0-a79d-47b4544264f5] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:11:14.720240 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-2843d0d2-2eae-4c28-9871-943a0b72c0c5 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ef9cbce2b2e64665b7f50767342dcb46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0ee02e4c8b448f2b77798ae504129b9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:11:14.729585 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1a8fce07-dca8-4f97-8350-7783753c546e tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:14.730530 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-2843d0d2-2eae-4c28-9871-943a0b72c0c5 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] 10.222.0.16 "GET /compute/v2.1/servers/8872e8cf-593e-4a7a-940d-e0c8159d4ad7" status: 200 len: 1713 microversion: 2.1 time: 0.408181 Jul 16 21:11:14.730849 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1a8fce07-dca8-4f97-8350-7783753c546e tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:14.732378 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 430/873] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:11:14 2022] GET /compute/v2.1/servers/8872e8cf-593e-4a7a-940d-e0c8159d4ad7 => generated 1713 bytes in 411 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:11:14.732937 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1a8fce07-dca8-4f97-8350-7783753c546e tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:14.733610 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1a8fce07-dca8-4f97-8350-7783753c546e tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:14.737923 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-6ec5a60a-d704-4729-aa3a-2d4439facbf5 req-898c8c53-e1c0-417b-a311-e5e1e1476d4f service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "de210dbd-1398-413f-9a0f-63cae300bd03", "name": "network-vif-unplugged", "status": "completed", "tag": "176a187b-2abc-455b-bbcd-a736dec0c33e"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:11:14.748194 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-6ec5a60a-d704-4729-aa3a-2d4439facbf5 req-898c8c53-e1c0-417b-a311-e5e1e1476d4f service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:14.749405 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-6ec5a60a-d704-4729-aa3a-2d4439facbf5 req-898c8c53-e1c0-417b-a311-e5e1e1476d4f service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:14.773579 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-6ec5a60a-d704-4729-aa3a-2d4439facbf5 req-898c8c53-e1c0-417b-a311-e5e1e1476d4f service nova] Creating event network-vif-unplugged:176a187b-2abc-455b-bbcd-a736dec0c33e for instance de210dbd-1398-413f-9a0f-63cae300bd03 on nch2-master-1 Jul 16 21:11:14.789064 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-6ec5a60a-d704-4729-aa3a-2d4439facbf5 req-898c8c53-e1c0-417b-a311-e5e1e1476d4f service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.054720 Jul 16 21:11:14.794340 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 431/874] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:11:14 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 61 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:11:14.859745 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-a9f4db63-d6bf-4d9b-9371-a3866099c081 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:11:14.861166 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-a9f4db63-d6bf-4d9b-9371-a3866099c081 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] [instance: 65a70cca-b978-41ab-b890-6052f653c5c3] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:11:14.871288 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-1a8fce07-dca8-4f97-8350-7783753c546e tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a69f9dee03d43beaafa966216b9e289', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9b769bf8f3ea4e368bf68de1df6538f9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:11:14.878986 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a9f4db63-d6bf-4d9b-9371-a3866099c081 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:14.879470 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a9f4db63-d6bf-4d9b-9371-a3866099c081 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:14.887071 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a9f4db63-d6bf-4d9b-9371-a3866099c081 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:14.888410 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a9f4db63-d6bf-4d9b-9371-a3866099c081 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:14.904018 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-1a8fce07-dca8-4f97-8350-7783753c546e tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] 10.222.0.16 "GET /compute/v2.1/servers/6a391156-ffce-49c0-a79d-47b4544264f5" status: 200 len: 1293 microversion: 2.1 time: 0.189810 Jul 16 21:11:14.906511 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 444/875] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:11:14 2022] GET /compute/v2.1/servers/6a391156-ffce-49c0-a79d-47b4544264f5 => generated 1293 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:11:14.926899 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-630d5e98-1da2-446f-a65f-5fb84d057834 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:11:14.927707 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-630d5e98-1da2-446f-a65f-5fb84d057834 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] [instance: de210dbd-1398-413f-9a0f-63cae300bd03] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:11:14.951966 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-630d5e98-1da2-446f-a65f-5fb84d057834 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:14.952819 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-630d5e98-1da2-446f-a65f-5fb84d057834 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:14.954094 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-630d5e98-1da2-446f-a65f-5fb84d057834 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:14.954826 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-630d5e98-1da2-446f-a65f-5fb84d057834 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:15.042373 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-a9f4db63-d6bf-4d9b-9371-a3866099c081 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'abeb7020019d41a18de42956d706a011', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '649babe54d104436af461f1ee4767001', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:11:15.053796 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-a9f4db63-d6bf-4d9b-9371-a3866099c081 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] 10.222.0.16 "GET /compute/v2.1/servers/65a70cca-b978-41ab-b890-6052f653c5c3" status: 200 len: 1269 microversion: 2.1 time: 0.209494 Jul 16 21:11:15.066432 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 432/876] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:11:14 2022] GET /compute/v2.1/servers/65a70cca-b978-41ab-b890-6052f653c5c3 => generated 1269 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:11:15.317271 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-630d5e98-1da2-446f-a65f-5fb84d057834 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:11:15.333705 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-630d5e98-1da2-446f-a65f-5fb84d057834 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:11:15.338772 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-630d5e98-1da2-446f-a65f-5fb84d057834 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:11:15.350434 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-630d5e98-1da2-446f-a65f-5fb84d057834 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] 10.222.0.16 "GET /compute/v2.1/servers/de210dbd-1398-413f-9a0f-63cae300bd03" status: 200 len: 1567 microversion: 2.25 time: 0.426916 Jul 16 21:11:15.361282 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 445/877] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:11:14 2022] GET /compute/v2.1/servers/de210dbd-1398-413f-9a0f-63cae300bd03 => generated 1567 bytes in 440 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 16 21:11:15.627182 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-6ab8128c-ca8a-4c4d-b6d0-cc693c658c05 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Action: 'create', calling method: .version_select at 0x7fdd9d0e2c10>, body: {"keypair": {"name": "tempest-keypair-963257303", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBMqOKr0KEGls0DgSst82+3s+hJeGwbYXOOgL0Kwgm1gWx/guOA3JwpLtGyFDLAtWLy4yIZfdyZUdG+jTH+B9vE43g8mNsypg6NDCHkiRODq21Au3xTWFGckm8KIDKgOL+Q=="}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:11:15.668252 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-6ab8128c-ca8a-4c4d-b6d0-cc693c658c05 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Getting quotas for project 30d028fa9b854c8fb71c8cbed2985c27. Resources: {'key_pairs'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:11:15.674587 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-6ab8128c-ca8a-4c4d-b6d0-cc693c658c05 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Getting quotas for user 52281b3989a04ca2a50b0bc93bf783f0 and project 30d028fa9b854c8fb71c8cbed2985c27. Resources: {'key_pairs'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:11:15.729351 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-0dc01e94-b6bd-45ed-86ee-ab4c8ebfb388 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:11:15.731950 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-0dc01e94-b6bd-45ed-86ee-ab4c8ebfb388 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:11:15.741604 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0dc01e94-b6bd-45ed-86ee-ab4c8ebfb388 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:15.748622 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0dc01e94-b6bd-45ed-86ee-ab4c8ebfb388 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:15.749733 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0dc01e94-b6bd-45ed-86ee-ab4c8ebfb388 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:15.750363 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0dc01e94-b6bd-45ed-86ee-ab4c8ebfb388 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:15.775441 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-6ab8128c-ca8a-4c4d-b6d0-cc693c658c05 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] 10.222.0.16 "POST /compute/v2.1/os-keypairs" status: 200 len: 385 microversion: 2.1 time: 0.581061 Jul 16 21:11:15.776721 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 433/878] 10.222.0.16 () {60 vars in 1158 bytes} [Sat Jul 16 21:11:15 2022] POST /compute/v2.1/os-keypairs => generated 385 bytes in 583 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 16 21:11:15.786660 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-08482aeb-c153-4565-a2bd-5af8bfd2a00a tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:11:15.787597 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-08482aeb-c153-4565-a2bd-5af8bfd2a00a tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] [instance: 8872e8cf-593e-4a7a-940d-e0c8159d4ad7] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:11:15.812701 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-08482aeb-c153-4565-a2bd-5af8bfd2a00a tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:15.813556 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-08482aeb-c153-4565-a2bd-5af8bfd2a00a tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:15.815391 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-08482aeb-c153-4565-a2bd-5af8bfd2a00a tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:15.815391 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-08482aeb-c153-4565-a2bd-5af8bfd2a00a tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:16.235036 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-08482aeb-c153-4565-a2bd-5af8bfd2a00a tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ef9cbce2b2e64665b7f50767342dcb46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0ee02e4c8b448f2b77798ae504129b9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:11:16.249478 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-08482aeb-c153-4565-a2bd-5af8bfd2a00a tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] 10.222.0.16 "GET /compute/v2.1/servers/8872e8cf-593e-4a7a-940d-e0c8159d4ad7" status: 200 len: 1645 microversion: 2.1 time: 0.471385 Jul 16 21:11:16.266751 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 434/879] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:11:15 2022] GET /compute/v2.1/servers/8872e8cf-593e-4a7a-940d-e0c8159d4ad7 => generated 1645 bytes in 488 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:11:16.273430 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-c5c6bd1a-e28a-4240-a2cc-98301e803206 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:11:16.274489 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-c5c6bd1a-e28a-4240-a2cc-98301e803206 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] [instance: 6a391156-ffce-49c0-a79d-47b4544264f5] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:11:16.283562 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c5c6bd1a-e28a-4240-a2cc-98301e803206 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:16.284301 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c5c6bd1a-e28a-4240-a2cc-98301e803206 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:16.285296 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c5c6bd1a-e28a-4240-a2cc-98301e803206 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:16.286032 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c5c6bd1a-e28a-4240-a2cc-98301e803206 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:16.547491 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-0dc01e94-b6bd-45ed-86ee-ab4c8ebfb388 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b68dacc4b8463ba07fae6372131117', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55a19b7a4682457a9132f5fc80110712', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:11:16.562402 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-0dc01e94-b6bd-45ed-86ee-ab4c8ebfb388 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "GET /compute/v2.1/servers/a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2" status: 200 len: 1378 microversion: 2.1 time: 0.835199 Jul 16 21:11:16.563949 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 446/880] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:11:15 2022] GET /compute/v2.1/servers/a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2 => generated 1378 bytes in 838 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:11:16.576309 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-de51bde5-1975-4fa0-baa6-496f12b32b96 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:11:16.578585 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-de51bde5-1975-4fa0-baa6-496f12b32b96 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] [instance: 65a70cca-b978-41ab-b890-6052f653c5c3] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:11:16.599050 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-de51bde5-1975-4fa0-baa6-496f12b32b96 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:16.602037 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-de51bde5-1975-4fa0-baa6-496f12b32b96 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:16.603241 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-de51bde5-1975-4fa0-baa6-496f12b32b96 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:16.604204 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-de51bde5-1975-4fa0-baa6-496f12b32b96 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:16.625060 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-c5c6bd1a-e28a-4240-a2cc-98301e803206 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a69f9dee03d43beaafa966216b9e289', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9b769bf8f3ea4e368bf68de1df6538f9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:11:16.639375 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-c5c6bd1a-e28a-4240-a2cc-98301e803206 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] 10.222.0.16 "GET /compute/v2.1/servers/6a391156-ffce-49c0-a79d-47b4544264f5" status: 200 len: 1293 microversion: 2.1 time: 0.368798 Jul 16 21:11:16.657159 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 435/881] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:11:16 2022] GET /compute/v2.1/servers/6a391156-ffce-49c0-a79d-47b4544264f5 => generated 1293 bytes in 389 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:11:16.668765 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-f893e8ba-2bf8-431d-a612-bc6890ec60c2 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:11:16.673029 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-f893e8ba-2bf8-431d-a612-bc6890ec60c2 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] [instance: de210dbd-1398-413f-9a0f-63cae300bd03] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:11:16.688328 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f893e8ba-2bf8-431d-a612-bc6890ec60c2 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:16.689053 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f893e8ba-2bf8-431d-a612-bc6890ec60c2 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:16.693907 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f893e8ba-2bf8-431d-a612-bc6890ec60c2 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:16.694673 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f893e8ba-2bf8-431d-a612-bc6890ec60c2 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:16.794959 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-de51bde5-1975-4fa0-baa6-496f12b32b96 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'abeb7020019d41a18de42956d706a011', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '649babe54d104436af461f1ee4767001', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:11:16.806262 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-de51bde5-1975-4fa0-baa6-496f12b32b96 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] 10.222.0.16 "GET /compute/v2.1/servers/65a70cca-b978-41ab-b890-6052f653c5c3" status: 200 len: 1275 microversion: 2.1 time: 0.240852 Jul 16 21:11:16.812198 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 447/882] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:11:16 2022] GET /compute/v2.1/servers/65a70cca-b978-41ab-b890-6052f653c5c3 => generated 1275 bytes in 247 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:11:16.940469 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-439dc168-e391-43d9-8b06-4f9535c8c820 req-857b7a5a-0419-4974-910e-2804462c647b service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "de210dbd-1398-413f-9a0f-63cae300bd03", "tag": "176a187b-2abc-455b-bbcd-a736dec0c33e"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:11:16.956981 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-439dc168-e391-43d9-8b06-4f9535c8c820 req-857b7a5a-0419-4974-910e-2804462c647b service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:16.957437 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-439dc168-e391-43d9-8b06-4f9535c8c820 req-857b7a5a-0419-4974-910e-2804462c647b service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:16.990182 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-439dc168-e391-43d9-8b06-4f9535c8c820 req-857b7a5a-0419-4974-910e-2804462c647b service nova] Creating event network-changed:176a187b-2abc-455b-bbcd-a736dec0c33e for instance de210dbd-1398-413f-9a0f-63cae300bd03 on nch2-master-1 Jul 16 21:11:17.000885 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-439dc168-e391-43d9-8b06-4f9535c8c820 req-857b7a5a-0419-4974-910e-2804462c647b service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.072333 Jul 16 21:11:17.006022 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 448/883] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:11:16 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 78 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:11:17.288623 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-e8fb2f5c-44be-4045-abef-f86e43c248a7 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:11:17.290311 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-e8fb2f5c-44be-4045-abef-f86e43c248a7 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] [instance: 8872e8cf-593e-4a7a-940d-e0c8159d4ad7] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:11:17.306961 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e8fb2f5c-44be-4045-abef-f86e43c248a7 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:17.311065 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e8fb2f5c-44be-4045-abef-f86e43c248a7 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:17.312244 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e8fb2f5c-44be-4045-abef-f86e43c248a7 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:17.312815 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e8fb2f5c-44be-4045-abef-f86e43c248a7 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:17.427549 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-f893e8ba-2bf8-431d-a612-bc6890ec60c2 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:11:17.447251 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-f893e8ba-2bf8-431d-a612-bc6890ec60c2 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:11:17.449308 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-f893e8ba-2bf8-431d-a612-bc6890ec60c2 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:11:17.455951 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-f893e8ba-2bf8-431d-a612-bc6890ec60c2 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] 10.222.0.16 "GET /compute/v2.1/servers/de210dbd-1398-413f-9a0f-63cae300bd03" status: 200 len: 1567 microversion: 2.25 time: 0.797575 Jul 16 21:11:17.456566 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 436/884] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:11:16 2022] GET /compute/v2.1/servers/de210dbd-1398-413f-9a0f-63cae300bd03 => generated 1567 bytes in 799 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:11:17.553018 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-e8fb2f5c-44be-4045-abef-f86e43c248a7 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ef9cbce2b2e64665b7f50767342dcb46', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd0ee02e4c8b448f2b77798ae504129b9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:11:17.574589 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-e8fb2f5c-44be-4045-abef-f86e43c248a7 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] 10.222.0.16 "GET /compute/v2.1/servers/8872e8cf-593e-4a7a-940d-e0c8159d4ad7" status: 200 len: 1645 microversion: 2.1 time: 0.295190 Jul 16 21:11:17.580342 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-19c8e501-cd62-4254-9b76-ca16b90cc3f7 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:11:17.580984 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-19c8e501-cd62-4254-9b76-ca16b90cc3f7 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:11:17.587449 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 449/885] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:11:17 2022] GET /compute/v2.1/servers/8872e8cf-593e-4a7a-940d-e0c8159d4ad7 => generated 1645 bytes in 323 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:11:17.589005 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-19c8e501-cd62-4254-9b76-ca16b90cc3f7 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:17.591482 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-19c8e501-cd62-4254-9b76-ca16b90cc3f7 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:17.592539 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-19c8e501-cd62-4254-9b76-ca16b90cc3f7 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:17.594816 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-19c8e501-cd62-4254-9b76-ca16b90cc3f7 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:17.656167 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-86bc724f-9de0-44cf-899a-03cabd5c7852 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:11:17.656870 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-86bc724f-9de0-44cf-899a-03cabd5c7852 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] [instance: 6a391156-ffce-49c0-a79d-47b4544264f5] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:11:17.674790 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-86bc724f-9de0-44cf-899a-03cabd5c7852 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:17.675379 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-86bc724f-9de0-44cf-899a-03cabd5c7852 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.010s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:17.676237 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-86bc724f-9de0-44cf-899a-03cabd5c7852 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:17.676708 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-86bc724f-9de0-44cf-899a-03cabd5c7852 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:18.585272 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-86bc724f-9de0-44cf-899a-03cabd5c7852 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a69f9dee03d43beaafa966216b9e289', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9b769bf8f3ea4e368bf68de1df6538f9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:11:18.604056 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-86bc724f-9de0-44cf-899a-03cabd5c7852 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] 10.222.0.16 "GET /compute/v2.1/servers/6a391156-ffce-49c0-a79d-47b4544264f5" status: 200 len: 1361 microversion: 2.1 time: 0.948845 Jul 16 21:11:18.604951 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 450/886] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:11:17 2022] GET /compute/v2.1/servers/6a391156-ffce-49c0-a79d-47b4544264f5 => generated 1361 bytes in 953 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:11:18.623113 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-ee43e619-e7cb-4a1c-9d13-5c6e03e16aa7 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:11:18.629884 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-ee43e619-e7cb-4a1c-9d13-5c6e03e16aa7 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] [instance: 65a70cca-b978-41ab-b890-6052f653c5c3] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:11:18.661479 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ee43e619-e7cb-4a1c-9d13-5c6e03e16aa7 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:18.662000 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ee43e619-e7cb-4a1c-9d13-5c6e03e16aa7 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:18.662924 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ee43e619-e7cb-4a1c-9d13-5c6e03e16aa7 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:18.663428 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ee43e619-e7cb-4a1c-9d13-5c6e03e16aa7 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:18.669847 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-19c8e501-cd62-4254-9b76-ca16b90cc3f7 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b68dacc4b8463ba07fae6372131117', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55a19b7a4682457a9132f5fc80110712', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:11:18.683922 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-19c8e501-cd62-4254-9b76-ca16b90cc3f7 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "GET /compute/v2.1/servers/a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2" status: 200 len: 1378 microversion: 2.1 time: 1.107212 Jul 16 21:11:18.687419 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 437/887] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:11:17 2022] GET /compute/v2.1/servers/a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2 => generated 1378 bytes in 1112 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:11:18.703093 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-1f561d06-e59d-4014-ad89-15bfa566bf7a tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:11:18.707275 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-1f561d06-e59d-4014-ad89-15bfa566bf7a tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] [instance: de210dbd-1398-413f-9a0f-63cae300bd03] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:11:18.722758 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1f561d06-e59d-4014-ad89-15bfa566bf7a tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:18.723259 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1f561d06-e59d-4014-ad89-15bfa566bf7a tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:18.724077 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1f561d06-e59d-4014-ad89-15bfa566bf7a tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:18.727571 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1f561d06-e59d-4014-ad89-15bfa566bf7a tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:19.284516 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-ee43e619-e7cb-4a1c-9d13-5c6e03e16aa7 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'abeb7020019d41a18de42956d706a011', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '649babe54d104436af461f1ee4767001', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:11:19.295154 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-ee43e619-e7cb-4a1c-9d13-5c6e03e16aa7 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] 10.222.0.16 "GET /compute/v2.1/servers/65a70cca-b978-41ab-b890-6052f653c5c3" status: 200 len: 1317 microversion: 2.1 time: 0.685678 Jul 16 21:11:19.295753 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 451/888] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:11:18 2022] GET /compute/v2.1/servers/65a70cca-b978-41ab-b890-6052f653c5c3 => generated 1317 bytes in 690 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:11:19.310565 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-66d92691-912a-4836-b0fa-eb20f75ff13f tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:11:19.311255 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-66d92691-912a-4836-b0fa-eb20f75ff13f tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] [instance: 8872e8cf-593e-4a7a-940d-e0c8159d4ad7] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:11:19.326424 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-66d92691-912a-4836-b0fa-eb20f75ff13f tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:19.329102 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-66d92691-912a-4836-b0fa-eb20f75ff13f tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:19.332622 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-66d92691-912a-4836-b0fa-eb20f75ff13f tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:19.332622 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-66d92691-912a-4836-b0fa-eb20f75ff13f tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:19.369592 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-66d92691-912a-4836-b0fa-eb20f75ff13f tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] HTTP exception thrown: Instance 8872e8cf-593e-4a7a-940d-e0c8159d4ad7 could not be found. Jul 16 21:11:19.375026 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-66d92691-912a-4836-b0fa-eb20f75ff13f tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Request method failure captured: Jul 16 21:11:19.375026 ncdv-master-1 devstack@n-api.service[138680]: request: GET /compute/v2.1/servers/8872e8cf-593e-4a7a-940d-e0c8159d4ad7 HTTP/1.1 Jul 16 21:11:19.375026 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:11:19.375026 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:11:19.375026 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:11:19.375026 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 0 Jul 16 21:11:19.375026 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:11:19.375026 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:11:19.375026 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:11:19.375026 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:11:19.375026 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0yjHRLOyvkW4FtfKJ0aopiZmEAMH7-zISqbih_qd9IN9V6rg6ySi183S78Jj7U1G1bZhLF_06FgKgnd6qfUgBYrxxNzIlkQaM2RTs_72Xwmzh_mbrOgpxdZycKk72lEJy3fGUTvXNUG7g578TevOCjVVpUrtiC3IxL1PtBbk7YGZvrA Jul 16 21:11:19.375026 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:11:19.375026 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:11:19.375026 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:11:19.375026 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:11:19.375026 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:11:19.375026 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:11:19.375026 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: d0ee02e4c8b448f2b77798ae504129b9 Jul 16 21:11:19.375026 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-ServersWithSpecificFlavorTestJSON-1854487373 Jul 16 21:11:19.375026 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:11:19.375026 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:11:19.376520 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/d0ee02e4c8b448f2b77798ae504129b9"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/d0ee02e4c8b448f2b77798ae504129b9"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_d0ee02e4c8b448f2b77798ae504129b9"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/d0ee02e4c8b448f2b77798ae504129b9"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:11:19.376520 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-ServersWithSpecificFlavorTestJSON-1854487373 Jul 16 21:11:19.376520 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: d0ee02e4c8b448f2b77798ae504129b9 Jul 16 21:11:19.376520 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-ServersWithSpecificFlavorTestJSON-1854487373 Jul 16 21:11:19.376520 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-ServersWithSpecificFlavorTestJSON-1854487373-project Jul 16 21:11:19.376520 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:11:19.376520 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:11:19.376520 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: ef9cbce2b2e64665b7f50767342dcb46 Jul 16 21:11:19.376520 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-ServersWithSpecificFlavorTestJSON-1854487373-project Jul 16 21:11:19.376520 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 21:11:19.376520 ncdv-master-1 devstack@n-api.service[138680]: exception: Instance 8872e8cf-593e-4a7a-940d-e0c8159d4ad7 could not be found. Jul 16 21:11:19.376520 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': '8872e8cf-593e-4a7a-940d-e0c8159d4ad7'} Jul 16 21:11:19.377320 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:11:19.377320 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:11:19.377320 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:11:19.377320 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:11:19.377320 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:11:19.377320 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:11:19.377320 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:11:19.377320 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:11:19.377320 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:11:19.377320 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:11:19.377320 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 8872e8cf-593e-4a7a-940d-e0c8159d4ad7 could not be found. Jul 16 21:11:19.377320 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:11:19.377320 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:11:19.377320 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:11:19.377320 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:11:19.377320 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:11:19.377320 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:11:19.377320 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:11:19.377320 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:11:19.377320 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:11:19.378469 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:11:19.378469 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:11:19.378469 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:11:19.378469 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:11:19.378469 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:11:19.378469 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:11:19.378469 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:11:19.378469 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 8872e8cf-593e-4a7a-940d-e0c8159d4ad7 could not be found. Jul 16 21:11:19.378469 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:11:19.378469 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:11:19.378469 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:11:19.378469 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:11:19.378469 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:11:19.378469 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:11:19.378469 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:11:19.378469 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:11:19.378469 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 8872e8cf-593e-4a7a-940d-e0c8159d4ad7 could not be found. Jul 16 21:11:19.378469 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:11:19.393049 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-66d92691-912a-4836-b0fa-eb20f75ff13f tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Returning 404 to user: Instance 8872e8cf-593e-4a7a-940d-e0c8159d4ad7 could not be found. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:11:19.398680 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-66d92691-912a-4836-b0fa-eb20f75ff13f tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] 10.222.0.16 "GET /compute/v2.1/servers/8872e8cf-593e-4a7a-940d-e0c8159d4ad7" status: 404 len: 111 microversion: 2.1 time: 0.099283 Jul 16 21:11:19.410970 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 452/889] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:11:19 2022] GET /compute/v2.1/servers/8872e8cf-593e-4a7a-940d-e0c8159d4ad7 => generated 111 bytes in 112 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:11:19.422523 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-4044922e-51f3-4d48-a4c1-991fa7cb1a07 req-1bbf2b0b-999b-48c4-aa76-be2549ce8dc0 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "8872e8cf-593e-4a7a-940d-e0c8159d4ad7", "name": "network-vif-deleted", "tag": "a07c49a0-9773-4e57-963c-ebcc388b002a"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:11:19.443484 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_api:SELECT=90,nova_api:INSERT=9,nova_api:UPDATE=1 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:11:19.456013 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-4044922e-51f3-4d48-a4c1-991fa7cb1a07 req-1bbf2b0b-999b-48c4-aa76-be2549ce8dc0 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:19.463598 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-4044922e-51f3-4d48-a4c1-991fa7cb1a07 req-1bbf2b0b-999b-48c4-aa76-be2549ce8dc0 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:19.494243 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-4044922e-51f3-4d48-a4c1-991fa7cb1a07 req-1bbf2b0b-999b-48c4-aa76-be2549ce8dc0 service nova] Creating event network-vif-deleted:a07c49a0-9773-4e57-963c-ebcc388b002a for instance 8872e8cf-593e-4a7a-940d-e0c8159d4ad7 on nch2-master-1 Jul 16 21:11:19.509238 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-4044922e-51f3-4d48-a4c1-991fa7cb1a07 req-1bbf2b0b-999b-48c4-aa76-be2549ce8dc0 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.096422 Jul 16 21:11:19.510519 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 453/890] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:11:19 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:11:19.522779 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-8033fcd8-f432-434a-801f-6114ab4e8d31 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:11:19.524985 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-8033fcd8-f432-434a-801f-6114ab4e8d31 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] [instance: e506a6e0-fbcd-4e53-a6a0-d31761654c92] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:11:19.541710 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8033fcd8-f432-434a-801f-6114ab4e8d31 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:19.546439 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8033fcd8-f432-434a-801f-6114ab4e8d31 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:19.547680 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8033fcd8-f432-434a-801f-6114ab4e8d31 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:19.549393 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8033fcd8-f432-434a-801f-6114ab4e8d31 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:19.587047 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-8033fcd8-f432-434a-801f-6114ab4e8d31 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] HTTP exception thrown: Instance e506a6e0-fbcd-4e53-a6a0-d31761654c92 could not be found. Jul 16 21:11:19.596452 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-8033fcd8-f432-434a-801f-6114ab4e8d31 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Request method failure captured: Jul 16 21:11:19.596452 ncdv-master-1 devstack@n-api.service[138680]: request: DELETE /compute/v2.1/servers/e506a6e0-fbcd-4e53-a6a0-d31761654c92 HTTP/1.1 Jul 16 21:11:19.596452 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:11:19.596452 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:11:19.596452 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:11:19.596452 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 0 Jul 16 21:11:19.596452 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:11:19.596452 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:11:19.596452 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:11:19.596452 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:11:19.596452 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0yjHRLOyvkW4FtfKJ0aopiZmEAMH7-zISqbih_qd9IN9V6rg6ySi183S78Jj7U1G1bZhLF_06FgKgnd6qfUgBYrxxNzIlkQaM2RTs_72Xwmzh_mbrOgpxdZycKk72lEJy3fGUTvXNUG7g578TevOCjVVpUrtiC3IxL1PtBbk7YGZvrA Jul 16 21:11:19.596452 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:11:19.596452 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:11:19.596452 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:11:19.596452 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:11:19.596452 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:11:19.596452 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:11:19.596452 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: d0ee02e4c8b448f2b77798ae504129b9 Jul 16 21:11:19.596452 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-ServersWithSpecificFlavorTestJSON-1854487373 Jul 16 21:11:19.596452 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:11:19.596452 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:11:19.597786 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/d0ee02e4c8b448f2b77798ae504129b9"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/d0ee02e4c8b448f2b77798ae504129b9"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_d0ee02e4c8b448f2b77798ae504129b9"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/d0ee02e4c8b448f2b77798ae504129b9"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:11:19.597786 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-ServersWithSpecificFlavorTestJSON-1854487373 Jul 16 21:11:19.597786 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: d0ee02e4c8b448f2b77798ae504129b9 Jul 16 21:11:19.597786 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-ServersWithSpecificFlavorTestJSON-1854487373 Jul 16 21:11:19.597786 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-ServersWithSpecificFlavorTestJSON-1854487373-project Jul 16 21:11:19.597786 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:11:19.597786 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:11:19.597786 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: ef9cbce2b2e64665b7f50767342dcb46 Jul 16 21:11:19.597786 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-ServersWithSpecificFlavorTestJSON-1854487373-project Jul 16 21:11:19.597786 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 21:11:19.597786 ncdv-master-1 devstack@n-api.service[138680]: exception: Instance e506a6e0-fbcd-4e53-a6a0-d31761654c92 could not be found. Jul 16 21:11:19.597786 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': 'e506a6e0-fbcd-4e53-a6a0-d31761654c92'} Jul 16 21:11:19.598460 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:11:19.598460 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:11:19.598460 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:11:19.598460 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:11:19.598460 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:11:19.598460 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:11:19.598460 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:11:19.598460 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:11:19.598460 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:11:19.598460 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:11:19.598460 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance e506a6e0-fbcd-4e53-a6a0-d31761654c92 could not be found. Jul 16 21:11:19.598460 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:11:19.598460 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:11:19.598460 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:11:19.598460 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:11:19.598460 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:11:19.598460 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:11:19.598460 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:11:19.598460 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:11:19.598460 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:11:19.600132 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:11:19.600132 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Jul 16 21:11:19.600132 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Jul 16 21:11:19.600132 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Jul 16 21:11:19.600132 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Jul 16 21:11:19.600132 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:11:19.600132 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:11:19.600132 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:11:19.600132 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:11:19.600132 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance e506a6e0-fbcd-4e53-a6a0-d31761654c92 could not be found. Jul 16 21:11:19.600132 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:11:19.600132 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:11:19.600132 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:11:19.600132 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:11:19.600132 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:11:19.600132 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:11:19.600132 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:11:19.600132 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:11:19.600132 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance e506a6e0-fbcd-4e53-a6a0-d31761654c92 could not be found. Jul 16 21:11:19.600132 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:11:19.601262 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-8033fcd8-f432-434a-801f-6114ab4e8d31 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Returning 404 to user: Instance e506a6e0-fbcd-4e53-a6a0-d31761654c92 could not be found. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:11:19.601262 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-8033fcd8-f432-434a-801f-6114ab4e8d31 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] 10.222.0.16 "DELETE /compute/v2.1/servers/e506a6e0-fbcd-4e53-a6a0-d31761654c92" status: 404 len: 111 microversion: 2.1 time: 0.086201 Jul 16 21:11:19.601262 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 454/891] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:11:19 2022] DELETE /compute/v2.1/servers/e506a6e0-fbcd-4e53-a6a0-d31761654c92 => generated 111 bytes in 88 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:11:19.611220 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-bd95a112-9339-4c70-95a5-8796df6b89ef tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:11:19.613108 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-bd95a112-9339-4c70-95a5-8796df6b89ef tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] [instance: e506a6e0-fbcd-4e53-a6a0-d31761654c92] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:11:19.633763 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-bd95a112-9339-4c70-95a5-8796df6b89ef tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:19.633763 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-bd95a112-9339-4c70-95a5-8796df6b89ef tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:19.633763 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-bd95a112-9339-4c70-95a5-8796df6b89ef tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:19.634003 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-bd95a112-9339-4c70-95a5-8796df6b89ef tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:19.665932 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-bd95a112-9339-4c70-95a5-8796df6b89ef tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] HTTP exception thrown: Instance e506a6e0-fbcd-4e53-a6a0-d31761654c92 could not be found. Jul 16 21:11:19.673195 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-bd95a112-9339-4c70-95a5-8796df6b89ef tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Request method failure captured: Jul 16 21:11:19.673195 ncdv-master-1 devstack@n-api.service[138680]: request: GET /compute/v2.1/servers/e506a6e0-fbcd-4e53-a6a0-d31761654c92 HTTP/1.1 Jul 16 21:11:19.673195 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:11:19.673195 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:11:19.673195 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:11:19.673195 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 0 Jul 16 21:11:19.673195 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:11:19.673195 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:11:19.673195 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:11:19.673195 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:11:19.673195 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0yjHRLOyvkW4FtfKJ0aopiZmEAMH7-zISqbih_qd9IN9V6rg6ySi183S78Jj7U1G1bZhLF_06FgKgnd6qfUgBYrxxNzIlkQaM2RTs_72Xwmzh_mbrOgpxdZycKk72lEJy3fGUTvXNUG7g578TevOCjVVpUrtiC3IxL1PtBbk7YGZvrA Jul 16 21:11:19.673195 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:11:19.673195 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:11:19.673195 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:11:19.673195 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:11:19.673195 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:11:19.673195 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:11:19.673195 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: d0ee02e4c8b448f2b77798ae504129b9 Jul 16 21:11:19.673195 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-ServersWithSpecificFlavorTestJSON-1854487373 Jul 16 21:11:19.673195 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:11:19.673195 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:11:19.674747 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/d0ee02e4c8b448f2b77798ae504129b9"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/d0ee02e4c8b448f2b77798ae504129b9"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_d0ee02e4c8b448f2b77798ae504129b9"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/d0ee02e4c8b448f2b77798ae504129b9"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:11:19.674747 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-ServersWithSpecificFlavorTestJSON-1854487373 Jul 16 21:11:19.674747 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: d0ee02e4c8b448f2b77798ae504129b9 Jul 16 21:11:19.674747 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-ServersWithSpecificFlavorTestJSON-1854487373 Jul 16 21:11:19.674747 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-ServersWithSpecificFlavorTestJSON-1854487373-project Jul 16 21:11:19.674747 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:11:19.674747 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:11:19.674747 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: ef9cbce2b2e64665b7f50767342dcb46 Jul 16 21:11:19.674747 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-ServersWithSpecificFlavorTestJSON-1854487373-project Jul 16 21:11:19.674747 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 21:11:19.674747 ncdv-master-1 devstack@n-api.service[138680]: exception: Instance e506a6e0-fbcd-4e53-a6a0-d31761654c92 could not be found. Jul 16 21:11:19.674747 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': 'e506a6e0-fbcd-4e53-a6a0-d31761654c92'} Jul 16 21:11:19.675603 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:11:19.675603 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:11:19.675603 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:11:19.675603 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:11:19.675603 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:11:19.675603 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:11:19.675603 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:11:19.675603 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:11:19.675603 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:11:19.675603 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:11:19.675603 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance e506a6e0-fbcd-4e53-a6a0-d31761654c92 could not be found. Jul 16 21:11:19.675603 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:11:19.675603 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:11:19.675603 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:11:19.675603 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:11:19.675603 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:11:19.675603 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:11:19.675603 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:11:19.675603 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:11:19.675603 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:11:19.676941 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:11:19.676941 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:11:19.676941 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:11:19.676941 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:11:19.676941 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:11:19.676941 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:11:19.676941 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:11:19.676941 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance e506a6e0-fbcd-4e53-a6a0-d31761654c92 could not be found. Jul 16 21:11:19.676941 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:11:19.676941 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:11:19.676941 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:11:19.676941 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:11:19.676941 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:11:19.676941 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:11:19.676941 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:11:19.676941 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:11:19.676941 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance e506a6e0-fbcd-4e53-a6a0-d31761654c92 could not be found. Jul 16 21:11:19.676941 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:11:19.680761 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-bd95a112-9339-4c70-95a5-8796df6b89ef tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Returning 404 to user: Instance e506a6e0-fbcd-4e53-a6a0-d31761654c92 could not be found. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:11:19.689248 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-bd95a112-9339-4c70-95a5-8796df6b89ef tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] 10.222.0.16 "GET /compute/v2.1/servers/e506a6e0-fbcd-4e53-a6a0-d31761654c92" status: 404 len: 111 microversion: 2.1 time: 0.083018 Jul 16 21:11:19.696625 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 455/892] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:11:19 2022] GET /compute/v2.1/servers/e506a6e0-fbcd-4e53-a6a0-d31761654c92 => generated 111 bytes in 94 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:11:19.715810 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-47eba36b-b7f6-4274-9e6a-5f5397a79df1 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:11:19.716931 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-47eba36b-b7f6-4274-9e6a-5f5397a79df1 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] [instance: 6a391156-ffce-49c0-a79d-47b4544264f5] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:11:19.731056 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-47eba36b-b7f6-4274-9e6a-5f5397a79df1 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:19.733274 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-47eba36b-b7f6-4274-9e6a-5f5397a79df1 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:19.737785 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-47eba36b-b7f6-4274-9e6a-5f5397a79df1 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:19.738246 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-47eba36b-b7f6-4274-9e6a-5f5397a79df1 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:19.843919 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-1f561d06-e59d-4014-ad89-15bfa566bf7a tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:11:19.899450 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-1f561d06-e59d-4014-ad89-15bfa566bf7a tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:11:19.903470 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-1f561d06-e59d-4014-ad89-15bfa566bf7a tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:11:19.915404 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-1f561d06-e59d-4014-ad89-15bfa566bf7a tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] 10.222.0.16 "GET /compute/v2.1/servers/de210dbd-1398-413f-9a0f-63cae300bd03" status: 200 len: 1567 microversion: 2.25 time: 1.222105 Jul 16 21:11:19.927292 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 438/893] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:11:18 2022] GET /compute/v2.1/servers/de210dbd-1398-413f-9a0f-63cae300bd03 => generated 1567 bytes in 1235 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:11:19.936356 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-377e6952-4bb2-413c-9467-fe61be949493 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Calling method '.version_select at 0x7fdd9d0e2040>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:11:20.073068 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-377e6952-4bb2-413c-9467-fe61be949493 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] 10.222.0.16 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-461056992" status: 202 len: 0 microversion: 2.1 time: 0.149177 Jul 16 21:11:20.074458 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 439/894] 10.222.0.16 () {58 vars in 1217 bytes} [Sat Jul 16 21:11:19 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-461056992 => generated 0 bytes in 151 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 21:11:20.081979 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-e2687dae-78a1-4e1a-84d0-0666c792d839 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:11:20.085727 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-e2687dae-78a1-4e1a-84d0-0666c792d839 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:11:20.115992 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e2687dae-78a1-4e1a-84d0-0666c792d839 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:20.118035 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e2687dae-78a1-4e1a-84d0-0666c792d839 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:20.122319 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e2687dae-78a1-4e1a-84d0-0666c792d839 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:20.122799 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e2687dae-78a1-4e1a-84d0-0666c792d839 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:20.315225 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-47eba36b-b7f6-4274-9e6a-5f5397a79df1 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a69f9dee03d43beaafa966216b9e289', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9b769bf8f3ea4e368bf68de1df6538f9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:11:20.338202 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-47eba36b-b7f6-4274-9e6a-5f5397a79df1 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] 10.222.0.16 "GET /compute/v2.1/servers/6a391156-ffce-49c0-a79d-47b4544264f5" status: 200 len: 1361 microversion: 2.1 time: 0.638969 Jul 16 21:11:20.342690 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 456/895] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:11:19 2022] GET /compute/v2.1/servers/6a391156-ffce-49c0-a79d-47b4544264f5 => generated 1361 bytes in 644 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:11:20.360703 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-62a9d80f-ad23-44ee-af1b-ba9810c0ae91 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:11:20.361870 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-62a9d80f-ad23-44ee-af1b-ba9810c0ae91 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] [instance: 65a70cca-b978-41ab-b890-6052f653c5c3] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:11:20.383284 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-62a9d80f-ad23-44ee-af1b-ba9810c0ae91 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:20.386801 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-62a9d80f-ad23-44ee-af1b-ba9810c0ae91 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:20.391558 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-62a9d80f-ad23-44ee-af1b-ba9810c0ae91 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:20.393427 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-62a9d80f-ad23-44ee-af1b-ba9810c0ae91 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:20.773923 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-e2687dae-78a1-4e1a-84d0-0666c792d839 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b68dacc4b8463ba07fae6372131117', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55a19b7a4682457a9132f5fc80110712', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:11:20.784666 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell0:SELECT=4 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:11:20.788343 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-e2687dae-78a1-4e1a-84d0-0666c792d839 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "GET /compute/v2.1/servers/a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2" status: 200 len: 1378 microversion: 2.1 time: 0.712111 Jul 16 21:11:20.790015 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 440/896] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:11:20 2022] GET /compute/v2.1/servers/a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2 => generated 1378 bytes in 715 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:11:20.938000 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-21864a3a-a2f8-42ec-b5fe-3731f9d1b9bf tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:11:20.943566 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-21864a3a-a2f8-42ec-b5fe-3731f9d1b9bf tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] [instance: de210dbd-1398-413f-9a0f-63cae300bd03] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:11:20.953734 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-21864a3a-a2f8-42ec-b5fe-3731f9d1b9bf tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:20.954314 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-21864a3a-a2f8-42ec-b5fe-3731f9d1b9bf tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:20.955402 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-21864a3a-a2f8-42ec-b5fe-3731f9d1b9bf tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:20.956926 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-21864a3a-a2f8-42ec-b5fe-3731f9d1b9bf tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:20.986741 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-62a9d80f-ad23-44ee-af1b-ba9810c0ae91 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'abeb7020019d41a18de42956d706a011', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '649babe54d104436af461f1ee4767001', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:11:21.021992 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-62a9d80f-ad23-44ee-af1b-ba9810c0ae91 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] 10.222.0.16 "GET /compute/v2.1/servers/65a70cca-b978-41ab-b890-6052f653c5c3" status: 200 len: 1317 microversion: 2.1 time: 0.675076 Jul 16 21:11:21.037234 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 457/897] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:11:20 2022] GET /compute/v2.1/servers/65a70cca-b978-41ab-b890-6052f653c5c3 => generated 1317 bytes in 683 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:11:21.373008 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-bf3a9b22-68b7-4f90-97df-0c62219082a8 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:11:21.374831 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-bf3a9b22-68b7-4f90-97df-0c62219082a8 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] [instance: 6a391156-ffce-49c0-a79d-47b4544264f5] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:11:21.411492 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-bf3a9b22-68b7-4f90-97df-0c62219082a8 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:21.414630 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-bf3a9b22-68b7-4f90-97df-0c62219082a8 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:21.416153 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-bf3a9b22-68b7-4f90-97df-0c62219082a8 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:21.419951 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-bf3a9b22-68b7-4f90-97df-0c62219082a8 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:21.818433 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-21864a3a-a2f8-42ec-b5fe-3731f9d1b9bf tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:11:21.832986 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-21864a3a-a2f8-42ec-b5fe-3731f9d1b9bf tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:11:21.837748 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-21864a3a-a2f8-42ec-b5fe-3731f9d1b9bf tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:11:21.841830 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-21864a3a-a2f8-42ec-b5fe-3731f9d1b9bf tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] 10.222.0.16 "GET /compute/v2.1/servers/de210dbd-1398-413f-9a0f-63cae300bd03" status: 200 len: 1567 microversion: 2.25 time: 0.910819 Jul 16 21:11:21.844446 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 441/898] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:11:20 2022] GET /compute/v2.1/servers/de210dbd-1398-413f-9a0f-63cae300bd03 => generated 1567 bytes in 914 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 16 21:11:21.856397 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-4842b8e2-3e63-4068-a166-640853d241ef tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:11:21.858983 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-4842b8e2-3e63-4068-a166-640853d241ef tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:11:21.879475 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4842b8e2-3e63-4068-a166-640853d241ef tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:21.881064 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4842b8e2-3e63-4068-a166-640853d241ef tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:21.881931 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4842b8e2-3e63-4068-a166-640853d241ef tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:21.882441 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4842b8e2-3e63-4068-a166-640853d241ef tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:22.226170 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell0:SELECT=12 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:11:22.252771 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-bf3a9b22-68b7-4f90-97df-0c62219082a8 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a69f9dee03d43beaafa966216b9e289', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9b769bf8f3ea4e368bf68de1df6538f9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:11:22.261238 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-bf3a9b22-68b7-4f90-97df-0c62219082a8 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] 10.222.0.16 "GET /compute/v2.1/servers/6a391156-ffce-49c0-a79d-47b4544264f5" status: 200 len: 1361 microversion: 2.1 time: 0.905094 Jul 16 21:11:22.263318 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 458/899] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:11:21 2022] GET /compute/v2.1/servers/6a391156-ffce-49c0-a79d-47b4544264f5 => generated 1361 bytes in 910 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:11:22.271472 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-6bb11152-cba8-424a-bf9e-c5ff75d8366a req-64e238d6-af85-4b47-942e-bbb83a405f7a service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "6a391156-ffce-49c0-a79d-47b4544264f5", "tag": "a8de1bba-f28a-49ad-92e5-8a10d67f1e75"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:11:22.281809 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell0:SELECT=26 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:11:22.284249 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-6bb11152-cba8-424a-bf9e-c5ff75d8366a req-64e238d6-af85-4b47-942e-bbb83a405f7a service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:22.285672 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-6bb11152-cba8-424a-bf9e-c5ff75d8366a req-64e238d6-af85-4b47-942e-bbb83a405f7a service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:22.303757 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-6bb11152-cba8-424a-bf9e-c5ff75d8366a req-64e238d6-af85-4b47-942e-bbb83a405f7a service nova] Creating event network-changed:a8de1bba-f28a-49ad-92e5-8a10d67f1e75 for instance 6a391156-ffce-49c0-a79d-47b4544264f5 on nch2-master-1 Jul 16 21:11:22.311306 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-6bb11152-cba8-424a-bf9e-c5ff75d8366a req-64e238d6-af85-4b47-942e-bbb83a405f7a service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.045334 Jul 16 21:11:22.312311 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 459/900] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:11:22 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:11:22.320398 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-7f716a19-6268-40fe-8828-89f5a5b749eb tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:11:22.321426 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-7f716a19-6268-40fe-8828-89f5a5b749eb tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] [instance: 65a70cca-b978-41ab-b890-6052f653c5c3] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:11:22.330939 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7f716a19-6268-40fe-8828-89f5a5b749eb tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:22.332385 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7f716a19-6268-40fe-8828-89f5a5b749eb tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:22.333257 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7f716a19-6268-40fe-8828-89f5a5b749eb tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:22.333711 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7f716a19-6268-40fe-8828-89f5a5b749eb tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:22.404074 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-4842b8e2-3e63-4068-a166-640853d241ef tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b68dacc4b8463ba07fae6372131117', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55a19b7a4682457a9132f5fc80110712', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:11:22.414001 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-4842b8e2-3e63-4068-a166-640853d241ef tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "GET /compute/v2.1/servers/a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2" status: 200 len: 1378 microversion: 2.1 time: 0.566783 Jul 16 21:11:22.415126 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 442/901] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:11:21 2022] GET /compute/v2.1/servers/a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2 => generated 1378 bytes in 568 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:11:22.838638 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-7f716a19-6268-40fe-8828-89f5a5b749eb tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'abeb7020019d41a18de42956d706a011', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '649babe54d104436af461f1ee4767001', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:11:22.848569 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-7f716a19-6268-40fe-8828-89f5a5b749eb tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] 10.222.0.16 "GET /compute/v2.1/servers/65a70cca-b978-41ab-b890-6052f653c5c3" status: 200 len: 1317 microversion: 2.1 time: 0.534045 Jul 16 21:11:22.849879 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 460/902] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:11:22 2022] GET /compute/v2.1/servers/65a70cca-b978-41ab-b890-6052f653c5c3 => generated 1317 bytes in 537 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:11:22.867280 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-d711525b-dfee-4044-94e7-0a6a96fd24ef tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:11:22.868297 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-d711525b-dfee-4044-94e7-0a6a96fd24ef tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] [instance: de210dbd-1398-413f-9a0f-63cae300bd03] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:11:22.878832 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d711525b-dfee-4044-94e7-0a6a96fd24ef tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:22.878832 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d711525b-dfee-4044-94e7-0a6a96fd24ef tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:22.879554 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d711525b-dfee-4044-94e7-0a6a96fd24ef tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:22.879908 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d711525b-dfee-4044-94e7-0a6a96fd24ef tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:23.151340 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-d711525b-dfee-4044-94e7-0a6a96fd24ef tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:11:23.163089 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-d711525b-dfee-4044-94e7-0a6a96fd24ef tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:11:23.165725 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-d711525b-dfee-4044-94e7-0a6a96fd24ef tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:11:23.167721 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-d711525b-dfee-4044-94e7-0a6a96fd24ef tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] 10.222.0.16 "GET /compute/v2.1/servers/de210dbd-1398-413f-9a0f-63cae300bd03" status: 200 len: 1567 microversion: 2.25 time: 0.308340 Jul 16 21:11:23.167873 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-03e5247b-62de-4026-ae2d-fdf46dcdfaa9 req-4fb88cc5-4e46-4bed-a36d-c5b457b442b6 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "de210dbd-1398-413f-9a0f-63cae300bd03", "name": "network-vif-unplugged", "status": "completed", "tag": "176a187b-2abc-455b-bbcd-a736dec0c33e"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:11:23.177911 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-03e5247b-62de-4026-ae2d-fdf46dcdfaa9 req-4fb88cc5-4e46-4bed-a36d-c5b457b442b6 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:23.178400 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-03e5247b-62de-4026-ae2d-fdf46dcdfaa9 req-4fb88cc5-4e46-4bed-a36d-c5b457b442b6 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:23.181786 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 443/903] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:11:22 2022] GET /compute/v2.1/servers/de210dbd-1398-413f-9a0f-63cae300bd03 => generated 1567 bytes in 324 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:11:23.192506 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-03e5247b-62de-4026-ae2d-fdf46dcdfaa9 req-4fb88cc5-4e46-4bed-a36d-c5b457b442b6 service nova] Creating event network-vif-unplugged:176a187b-2abc-455b-bbcd-a736dec0c33e for instance de210dbd-1398-413f-9a0f-63cae300bd03 on nch2-master-1 Jul 16 21:11:23.201859 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-03e5247b-62de-4026-ae2d-fdf46dcdfaa9 req-4fb88cc5-4e46-4bed-a36d-c5b457b442b6 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.041676 Jul 16 21:11:23.202623 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 461/904] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:11:23 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:11:23.289978 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-efc22d20-d37b-4944-a4ba-f53879ca7993 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:11:23.290604 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-efc22d20-d37b-4944-a4ba-f53879ca7993 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] [instance: 6a391156-ffce-49c0-a79d-47b4544264f5] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:11:23.297320 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-efc22d20-d37b-4944-a4ba-f53879ca7993 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:23.297786 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-efc22d20-d37b-4944-a4ba-f53879ca7993 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:23.298569 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-efc22d20-d37b-4944-a4ba-f53879ca7993 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:23.299046 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-efc22d20-d37b-4944-a4ba-f53879ca7993 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:23.433090 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-eb614040-7f4f-44f0-8522-93ac7b2d8797 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:11:23.438698 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-eb614040-7f4f-44f0-8522-93ac7b2d8797 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:11:23.447740 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-eb614040-7f4f-44f0-8522-93ac7b2d8797 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:23.452991 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-eb614040-7f4f-44f0-8522-93ac7b2d8797 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:23.454042 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-eb614040-7f4f-44f0-8522-93ac7b2d8797 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:23.454646 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-eb614040-7f4f-44f0-8522-93ac7b2d8797 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:23.761120 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-efc22d20-d37b-4944-a4ba-f53879ca7993 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a69f9dee03d43beaafa966216b9e289', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9b769bf8f3ea4e368bf68de1df6538f9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:11:23.777473 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-efc22d20-d37b-4944-a4ba-f53879ca7993 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] 10.222.0.16 "GET /compute/v2.1/servers/6a391156-ffce-49c0-a79d-47b4544264f5" status: 200 len: 1361 microversion: 2.1 time: 0.491305 Jul 16 21:11:23.780264 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 444/905] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:11:23 2022] GET /compute/v2.1/servers/6a391156-ffce-49c0-a79d-47b4544264f5 => generated 1361 bytes in 494 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:11:23.792806 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-faab9c84-a9cb-42ed-a0ea-afc9bfb965bc tempest-ServersWithSpecificFlavorTestJSON-2008354030 tempest-ServersWithSpecificFlavorTestJSON-2008354030-project-admin] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:11:23.794070 ncdv-master-1 devstack@n-api.service[138681]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-flavor-manage:delete": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:11:23.794070 ncdv-master-1 devstack@n-api.service[138681]: warnings.warn(msg) Jul 16 21:11:23.912828 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-eb614040-7f4f-44f0-8522-93ac7b2d8797 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b68dacc4b8463ba07fae6372131117', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55a19b7a4682457a9132f5fc80110712', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:11:23.935276 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=100 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:11:23.945207 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-faab9c84-a9cb-42ed-a0ea-afc9bfb965bc tempest-ServersWithSpecificFlavorTestJSON-2008354030 tempest-ServersWithSpecificFlavorTestJSON-2008354030-project-admin] 10.222.0.16 "DELETE /compute/v2.1/flavors/1856122662" status: 202 len: 0 microversion: 2.1 time: 0.156068 Jul 16 21:11:23.945484 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-eb614040-7f4f-44f0-8522-93ac7b2d8797 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "GET /compute/v2.1/servers/a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2" status: 200 len: 1378 microversion: 2.1 time: 0.516302 Jul 16 21:11:23.946576 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 445/906] 10.222.0.16 () {58 vars in 1160 bytes} [Sat Jul 16 21:11:23 2022] DELETE /compute/v2.1/flavors/1856122662 => generated 0 bytes in 167 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 21:11:23.947559 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 462/907] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:11:23 2022] GET /compute/v2.1/servers/a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2 => generated 1378 bytes in 520 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:11:23.953961 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-1c0d2890-8ff7-4cb0-99a9-685d268a5efa tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesAdminNegativeTest-server-339711966", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452", "networks": [{"uuid": "9303ba55-8c09-4f8a-9781-a3f09b24a194"}], "security_groups": [{"name": "tempest-securitygroup--2028390458"}], "key_name": "tempest-keypair-963257303", "user_data": "IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIHVidW50dSB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH51YnVudHUvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo="}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:11:23.959857 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-76aa7796-96dd-4634-9224-06c83e9f34f9 req-b7b850c2-e5dc-418b-b942-1c7cc52e163c service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2", "name": "network-vif-plugged", "status": "completed", "tag": "bf9d54d7-5718-4678-bc2f-5ba8241bd855"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:11:23.986460 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-76aa7796-96dd-4634-9224-06c83e9f34f9 req-b7b850c2-e5dc-418b-b942-1c7cc52e163c service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:23.990703 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-76aa7796-96dd-4634-9224-06c83e9f34f9 req-b7b850c2-e5dc-418b-b942-1c7cc52e163c service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:24.040884 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-76aa7796-96dd-4634-9224-06c83e9f34f9 req-b7b850c2-e5dc-418b-b942-1c7cc52e163c service nova] Creating event network-vif-plugged:bf9d54d7-5718-4678-bc2f-5ba8241bd855 for instance a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2 on nch1-master-1 Jul 16 21:11:24.054558 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-76aa7796-96dd-4634-9224-06c83e9f34f9 req-b7b850c2-e5dc-418b-b942-1c7cc52e163c service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.102021 Jul 16 21:11:24.058413 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 463/908] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:11:23 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 110 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:11:24.071715 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-c98ddca2-2bb6-4a8f-8f83-733579871c2d tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:11:24.080558 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-c98ddca2-2bb6-4a8f-8f83-733579871c2d tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] [instance: 65a70cca-b978-41ab-b890-6052f653c5c3] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:11:24.094163 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c98ddca2-2bb6-4a8f-8f83-733579871c2d tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:24.094640 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c98ddca2-2bb6-4a8f-8f83-733579871c2d tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:24.102884 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c98ddca2-2bb6-4a8f-8f83-733579871c2d tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:24.102884 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c98ddca2-2bb6-4a8f-8f83-733579871c2d tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:24.561592 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.network.neutron [None req-1c0d2890-8ff7-4cb0-99a9-685d268a5efa tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] validate_networks() for [('9303ba55-8c09-4f8a-9781-a3f09b24a194', None, None, None, None, None)] {{(pid=138681) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 16 21:11:24.730556 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-c98ddca2-2bb6-4a8f-8f83-733579871c2d tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'abeb7020019d41a18de42956d706a011', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '649babe54d104436af461f1ee4767001', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:11:24.744176 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-c98ddca2-2bb6-4a8f-8f83-733579871c2d tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] 10.222.0.16 "GET /compute/v2.1/servers/65a70cca-b978-41ab-b890-6052f653c5c3" status: 200 len: 1317 microversion: 2.1 time: 0.683474 Jul 16 21:11:24.745363 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 464/909] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:11:24 2022] GET /compute/v2.1/servers/65a70cca-b978-41ab-b890-6052f653c5c3 => generated 1317 bytes in 686 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:11:24.758878 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-be1552db-6187-4853-8542-52dbd46855b9 tempest-ServersWithSpecificFlavorTestJSON-2008354030 tempest-ServersWithSpecificFlavorTestJSON-2008354030-project-admin] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:11:24.781882 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-be1552db-6187-4853-8542-52dbd46855b9 tempest-ServersWithSpecificFlavorTestJSON-2008354030 tempest-ServersWithSpecificFlavorTestJSON-2008354030-project-admin] 10.222.0.16 "GET /compute/v2.1/flavors/detail" status: 200 len: 5895 microversion: 2.1 time: 0.033920 Jul 16 21:11:24.784581 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 465/910] 10.222.0.16 () {58 vars in 1145 bytes} [Sat Jul 16 21:11:24 2022] GET /compute/v2.1/flavors/detail => generated 5895 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:11:24.800713 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-c939002e-32e9-4b51-a753-e86f4ac490fe tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:11:24.801533 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-c939002e-32e9-4b51-a753-e86f4ac490fe tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] [instance: de210dbd-1398-413f-9a0f-63cae300bd03] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:11:24.811153 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c939002e-32e9-4b51-a753-e86f4ac490fe tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:24.816937 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c939002e-32e9-4b51-a753-e86f4ac490fe tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:24.816937 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c939002e-32e9-4b51-a753-e86f4ac490fe tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:24.816937 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c939002e-32e9-4b51-a753-e86f4ac490fe tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:25.144884 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-c939002e-32e9-4b51-a753-e86f4ac490fe tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:11:25.161753 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-c939002e-32e9-4b51-a753-e86f4ac490fe tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:11:25.163550 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-c939002e-32e9-4b51-a753-e86f4ac490fe tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:11:25.164980 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-c939002e-32e9-4b51-a753-e86f4ac490fe tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] 10.222.0.16 "GET /compute/v2.1/servers/de210dbd-1398-413f-9a0f-63cae300bd03" status: 200 len: 1567 microversion: 2.25 time: 0.368809 Jul 16 21:11:25.165551 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 466/911] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:11:24 2022] GET /compute/v2.1/servers/de210dbd-1398-413f-9a0f-63cae300bd03 => generated 1567 bytes in 370 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:11:25.171444 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-0cdccc05-c694-4096-b288-4f18e419df2e tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:11:25.172071 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-0cdccc05-c694-4096-b288-4f18e419df2e tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] [instance: 6a391156-ffce-49c0-a79d-47b4544264f5] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:11:25.178412 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0cdccc05-c694-4096-b288-4f18e419df2e tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:25.178863 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0cdccc05-c694-4096-b288-4f18e419df2e tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:25.179706 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0cdccc05-c694-4096-b288-4f18e419df2e tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:25.180145 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0cdccc05-c694-4096-b288-4f18e419df2e tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:25.338020 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-1c0d2890-8ff7-4cb0-99a9-685d268a5efa tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Flavor limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 16 21:11:25.339527 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-1c0d2890-8ff7-4cb0-99a9-685d268a5efa tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Image limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 16 21:11:25.340264 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-1c0d2890-8ff7-4cb0-99a9-685d268a5efa tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Flavor pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 16 21:11:25.340843 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-1c0d2890-8ff7-4cb0-99a9-685d268a5efa tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Image pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 16 21:11:25.342220 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-1c0d2890-8ff7-4cb0-99a9-685d268a5efa tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 16 21:11:25.358372 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1c0d2890-8ff7-4cb0-99a9-685d268a5efa tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:25.359169 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1c0d2890-8ff7-4cb0-99a9-685d268a5efa tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:25.360384 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1c0d2890-8ff7-4cb0-99a9-685d268a5efa tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:25.361474 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1c0d2890-8ff7-4cb0-99a9-685d268a5efa tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:25.383998 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-0cdccc05-c694-4096-b288-4f18e419df2e tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a69f9dee03d43beaafa966216b9e289', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9b769bf8f3ea4e368bf68de1df6538f9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:11:25.393929 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-0cdccc05-c694-4096-b288-4f18e419df2e tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] 10.222.0.16 "GET /compute/v2.1/servers/6a391156-ffce-49c0-a79d-47b4544264f5" status: 200 len: 1361 microversion: 2.1 time: 0.226911 Jul 16 21:11:25.394677 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-1c0d2890-8ff7-4cb0-99a9-685d268a5efa tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Getting quotas for project 30d028fa9b854c8fb71c8cbed2985c27. Resources: {'ram', 'instances', 'cores'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:11:25.395745 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 467/912] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:11:25 2022] GET /compute/v2.1/servers/6a391156-ffce-49c0-a79d-47b4544264f5 => generated 1361 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:11:25.415356 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-83791c1c-e5b3-4795-aa3e-844aeab97e94 tempest-ServersWithSpecificFlavorTestJSON-2008354030 tempest-ServersWithSpecificFlavorTestJSON-2008354030-project-admin] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:11:25.416217 ncdv-master-1 devstack@n-api.service[138680]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-flavor-manage:delete": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:11:25.416217 ncdv-master-1 devstack@n-api.service[138680]: warnings.warn(msg) Jul 16 21:11:25.419850 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-1c0d2890-8ff7-4cb0-99a9-685d268a5efa tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Getting quotas for user 52281b3989a04ca2a50b0bc93bf783f0 and project 30d028fa9b854c8fb71c8cbed2985c27. Resources: {'ram', 'instances', 'cores'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:11:25.437271 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-1c0d2890-8ff7-4cb0-99a9-685d268a5efa tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Going to run 1 instances... {{(pid=138681) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 16 21:11:25.440263 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-1c0d2890-8ff7-4cb0-99a9-685d268a5efa tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] [instance: b07f0d53-2cd4-440f-be47-077703aa66e6] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138681) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 16 21:11:25.518965 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-83791c1c-e5b3-4795-aa3e-844aeab97e94 tempest-ServersWithSpecificFlavorTestJSON-2008354030 tempest-ServersWithSpecificFlavorTestJSON-2008354030-project-admin] 10.222.0.16 "DELETE /compute/v2.1/flavors/2089714046" status: 202 len: 0 microversion: 2.1 time: 0.122265 Jul 16 21:11:25.520323 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 468/913] 10.222.0.16 () {58 vars in 1160 bytes} [Sat Jul 16 21:11:25 2022] DELETE /compute/v2.1/flavors/2089714046 => generated 0 bytes in 124 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 21:11:25.533380 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-617453d2-9fca-40b3-a8bf-bc360ddd1f5f tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:11:25.534947 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-617453d2-9fca-40b3-a8bf-bc360ddd1f5f tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:11:25.550489 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-617453d2-9fca-40b3-a8bf-bc360ddd1f5f tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:25.551352 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-617453d2-9fca-40b3-a8bf-bc360ddd1f5f tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:25.552832 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-617453d2-9fca-40b3-a8bf-bc360ddd1f5f tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:25.554513 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-617453d2-9fca-40b3-a8bf-bc360ddd1f5f tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:25.620198 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-1c0d2890-8ff7-4cb0-99a9-685d268a5efa tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 406 microversion: 2.1 time: 1.671114 Jul 16 21:11:25.628566 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 446/914] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 21:11:23 2022] POST /compute/v2.1/servers => generated 406 bytes in 1681 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 16 21:11:25.642905 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-7042636b-d6bd-423d-a4f1-7a88ad7b74a6 req-9d22419c-62fa-4fda-a5bc-88a003698dbb service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "65a70cca-b978-41ab-b890-6052f653c5c3", "tag": "7782780c-06ee-4b9b-b252-ba55ad0b48d8"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:11:25.660471 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-7042636b-d6bd-423d-a4f1-7a88ad7b74a6 req-9d22419c-62fa-4fda-a5bc-88a003698dbb service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:25.661077 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-7042636b-d6bd-423d-a4f1-7a88ad7b74a6 req-9d22419c-62fa-4fda-a5bc-88a003698dbb service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:25.676842 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-7042636b-d6bd-423d-a4f1-7a88ad7b74a6 req-9d22419c-62fa-4fda-a5bc-88a003698dbb service nova] Creating event network-changed:7782780c-06ee-4b9b-b252-ba55ad0b48d8 for instance 65a70cca-b978-41ab-b890-6052f653c5c3 on nch2-master-1 Jul 16 21:11:25.682800 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-7042636b-d6bd-423d-a4f1-7a88ad7b74a6 req-9d22419c-62fa-4fda-a5bc-88a003698dbb service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.051846 Jul 16 21:11:25.684051 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 447/915] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:11:25 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:11:25.689740 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-77c2138f-cb59-4c46-8d1f-33964b616527 tempest-ServersWithSpecificFlavorTestJSON-2008354030 tempest-ServersWithSpecificFlavorTestJSON-2008354030-project-admin] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:11:25.725376 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-77c2138f-cb59-4c46-8d1f-33964b616527 tempest-ServersWithSpecificFlavorTestJSON-2008354030 tempest-ServersWithSpecificFlavorTestJSON-2008354030-project-admin] 10.222.0.16 "GET /compute/v2.1/flavors/detail" status: 200 len: 5480 microversion: 2.1 time: 0.039413 Jul 16 21:11:25.736684 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 448/916] 10.222.0.16 () {58 vars in 1145 bytes} [Sat Jul 16 21:11:25 2022] GET /compute/v2.1/flavors/detail => generated 5480 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:11:25.736684 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-c26d4b2c-99d8-47c4-8913-74faa8a9250e tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:11:25.740988 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-c26d4b2c-99d8-47c4-8913-74faa8a9250e tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] [instance: b07f0d53-2cd4-440f-be47-077703aa66e6] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:11:25.924729 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-c26d4b2c-99d8-47c4-8913-74faa8a9250e tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '52281b3989a04ca2a50b0bc93bf783f0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '30d028fa9b854c8fb71c8cbed2985c27', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:11:25.939386 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-c26d4b2c-99d8-47c4-8913-74faa8a9250e tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] 10.222.0.16 "GET /compute/v2.1/servers/b07f0d53-2cd4-440f-be47-077703aa66e6" status: 200 len: 1240 microversion: 2.1 time: 0.211211 Jul 16 21:11:25.939988 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 449/917] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:11:25 2022] GET /compute/v2.1/servers/b07f0d53-2cd4-440f-be47-077703aa66e6 => generated 1240 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:11:25.955348 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-74e70af8-b5d6-4c62-a87a-f7aae8014df5 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:11:25.958201 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-74e70af8-b5d6-4c62-a87a-f7aae8014df5 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] [instance: 8872e8cf-593e-4a7a-940d-e0c8159d4ad7] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:11:25.964518 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-74e70af8-b5d6-4c62-a87a-f7aae8014df5 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:25.966356 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-74e70af8-b5d6-4c62-a87a-f7aae8014df5 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:25.967585 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-74e70af8-b5d6-4c62-a87a-f7aae8014df5 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:25.968185 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-74e70af8-b5d6-4c62-a87a-f7aae8014df5 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:26.012189 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-74e70af8-b5d6-4c62-a87a-f7aae8014df5 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] HTTP exception thrown: Instance 8872e8cf-593e-4a7a-940d-e0c8159d4ad7 could not be found. Jul 16 21:11:26.018229 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-74e70af8-b5d6-4c62-a87a-f7aae8014df5 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Request method failure captured: Jul 16 21:11:26.018229 ncdv-master-1 devstack@n-api.service[138681]: request: DELETE /compute/v2.1/servers/8872e8cf-593e-4a7a-940d-e0c8159d4ad7 HTTP/1.1 Jul 16 21:11:26.018229 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:11:26.018229 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:11:26.018229 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:11:26.018229 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 21:11:26.018229 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:11:26.018229 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:11:26.018229 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:11:26.018229 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:11:26.018229 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0yjHRLOyvkW4FtfKJ0aopiZmEAMH7-zISqbih_qd9IN9V6rg6ySi183S78Jj7U1G1bZhLF_06FgKgnd6qfUgBYrxxNzIlkQaM2RTs_72Xwmzh_mbrOgpxdZycKk72lEJy3fGUTvXNUG7g578TevOCjVVpUrtiC3IxL1PtBbk7YGZvrA Jul 16 21:11:26.018229 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:11:26.018229 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:11:26.018229 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:11:26.018229 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:11:26.018229 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:11:26.018229 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:11:26.018229 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: d0ee02e4c8b448f2b77798ae504129b9 Jul 16 21:11:26.018229 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-ServersWithSpecificFlavorTestJSON-1854487373 Jul 16 21:11:26.018229 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:11:26.018229 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:11:26.019761 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/d0ee02e4c8b448f2b77798ae504129b9"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/d0ee02e4c8b448f2b77798ae504129b9"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_d0ee02e4c8b448f2b77798ae504129b9"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/d0ee02e4c8b448f2b77798ae504129b9"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:11:26.019761 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-ServersWithSpecificFlavorTestJSON-1854487373 Jul 16 21:11:26.019761 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: d0ee02e4c8b448f2b77798ae504129b9 Jul 16 21:11:26.019761 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-ServersWithSpecificFlavorTestJSON-1854487373 Jul 16 21:11:26.019761 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-ServersWithSpecificFlavorTestJSON-1854487373-project Jul 16 21:11:26.019761 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:11:26.019761 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:11:26.019761 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: ef9cbce2b2e64665b7f50767342dcb46 Jul 16 21:11:26.019761 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-ServersWithSpecificFlavorTestJSON-1854487373-project Jul 16 21:11:26.019761 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:11:26.019761 ncdv-master-1 devstack@n-api.service[138681]: exception: Instance 8872e8cf-593e-4a7a-940d-e0c8159d4ad7 could not be found. Jul 16 21:11:26.019761 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': '8872e8cf-593e-4a7a-940d-e0c8159d4ad7'} Jul 16 21:11:26.020724 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:11:26.020724 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:11:26.020724 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:11:26.020724 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:11:26.020724 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:11:26.020724 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:11:26.020724 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:11:26.020724 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:11:26.020724 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:11:26.020724 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:11:26.020724 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 8872e8cf-593e-4a7a-940d-e0c8159d4ad7 could not be found. Jul 16 21:11:26.020724 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:11:26.020724 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:11:26.020724 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:11:26.020724 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:11:26.020724 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:11:26.020724 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:11:26.020724 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:11:26.020724 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:11:26.020724 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:11:26.022416 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:11:26.022416 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Jul 16 21:11:26.022416 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Jul 16 21:11:26.022416 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Jul 16 21:11:26.022416 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Jul 16 21:11:26.022416 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:11:26.022416 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:11:26.022416 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:11:26.022416 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:11:26.022416 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 8872e8cf-593e-4a7a-940d-e0c8159d4ad7 could not be found. Jul 16 21:11:26.022416 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:11:26.022416 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:11:26.022416 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:11:26.022416 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:11:26.022416 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:11:26.022416 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:11:26.022416 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:11:26.022416 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:11:26.022416 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 8872e8cf-593e-4a7a-940d-e0c8159d4ad7 could not be found. Jul 16 21:11:26.022416 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:11:26.024129 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-74e70af8-b5d6-4c62-a87a-f7aae8014df5 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Returning 404 to user: Instance 8872e8cf-593e-4a7a-940d-e0c8159d4ad7 could not be found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:11:26.024129 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-74e70af8-b5d6-4c62-a87a-f7aae8014df5 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] 10.222.0.16 "DELETE /compute/v2.1/servers/8872e8cf-593e-4a7a-940d-e0c8159d4ad7" status: 404 len: 111 microversion: 2.1 time: 0.068988 Jul 16 21:11:26.024129 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 450/918] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:11:25 2022] DELETE /compute/v2.1/servers/8872e8cf-593e-4a7a-940d-e0c8159d4ad7 => generated 111 bytes in 80 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:11:26.044592 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-6a11d2fc-17c5-46e7-b3db-a664c6e5a3a3 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:11:26.045700 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-6a11d2fc-17c5-46e7-b3db-a664c6e5a3a3 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] [instance: 65a70cca-b978-41ab-b890-6052f653c5c3] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:11:26.100117 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6a11d2fc-17c5-46e7-b3db-a664c6e5a3a3 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:26.100579 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6a11d2fc-17c5-46e7-b3db-a664c6e5a3a3 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:26.104983 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6a11d2fc-17c5-46e7-b3db-a664c6e5a3a3 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:26.105435 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6a11d2fc-17c5-46e7-b3db-a664c6e5a3a3 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:26.812441 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-617453d2-9fca-40b3-a8bf-bc360ddd1f5f tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b68dacc4b8463ba07fae6372131117', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55a19b7a4682457a9132f5fc80110712', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:11:26.831648 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-617453d2-9fca-40b3-a8bf-bc360ddd1f5f tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "GET /compute/v2.1/servers/a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2" status: 200 len: 1378 microversion: 2.1 time: 1.305890 Jul 16 21:11:26.832852 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 469/919] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:11:25 2022] GET /compute/v2.1/servers/a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2 => generated 1378 bytes in 1308 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:11:26.861486 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-56834807-98e0-468d-a849-996028681b00 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:11:26.868825 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-56834807-98e0-468d-a849-996028681b00 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] [instance: 8872e8cf-593e-4a7a-940d-e0c8159d4ad7] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:11:26.878619 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-56834807-98e0-468d-a849-996028681b00 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:26.879187 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-56834807-98e0-468d-a849-996028681b00 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:26.880200 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-56834807-98e0-468d-a849-996028681b00 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:26.880837 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-56834807-98e0-468d-a849-996028681b00 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:26.918616 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-56834807-98e0-468d-a849-996028681b00 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] HTTP exception thrown: Instance 8872e8cf-593e-4a7a-940d-e0c8159d4ad7 could not be found. Jul 16 21:11:26.922191 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-56834807-98e0-468d-a849-996028681b00 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Request method failure captured: Jul 16 21:11:26.922191 ncdv-master-1 devstack@n-api.service[138680]: request: GET /compute/v2.1/servers/8872e8cf-593e-4a7a-940d-e0c8159d4ad7 HTTP/1.1 Jul 16 21:11:26.922191 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:11:26.922191 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:11:26.922191 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:11:26.922191 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 0 Jul 16 21:11:26.922191 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:11:26.922191 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:11:26.922191 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:11:26.922191 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:11:26.922191 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0yjHRLOyvkW4FtfKJ0aopiZmEAMH7-zISqbih_qd9IN9V6rg6ySi183S78Jj7U1G1bZhLF_06FgKgnd6qfUgBYrxxNzIlkQaM2RTs_72Xwmzh_mbrOgpxdZycKk72lEJy3fGUTvXNUG7g578TevOCjVVpUrtiC3IxL1PtBbk7YGZvrA Jul 16 21:11:26.922191 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:11:26.922191 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:11:26.922191 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:11:26.922191 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:11:26.922191 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:11:26.922191 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:11:26.922191 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: d0ee02e4c8b448f2b77798ae504129b9 Jul 16 21:11:26.922191 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-ServersWithSpecificFlavorTestJSON-1854487373 Jul 16 21:11:26.922191 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:11:26.922191 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:11:26.923616 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/d0ee02e4c8b448f2b77798ae504129b9"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/d0ee02e4c8b448f2b77798ae504129b9"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_d0ee02e4c8b448f2b77798ae504129b9"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/d0ee02e4c8b448f2b77798ae504129b9"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:11:26.923616 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-ServersWithSpecificFlavorTestJSON-1854487373 Jul 16 21:11:26.923616 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: d0ee02e4c8b448f2b77798ae504129b9 Jul 16 21:11:26.923616 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-ServersWithSpecificFlavorTestJSON-1854487373 Jul 16 21:11:26.923616 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-ServersWithSpecificFlavorTestJSON-1854487373-project Jul 16 21:11:26.923616 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:11:26.923616 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:11:26.923616 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: ef9cbce2b2e64665b7f50767342dcb46 Jul 16 21:11:26.923616 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-ServersWithSpecificFlavorTestJSON-1854487373-project Jul 16 21:11:26.923616 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 21:11:26.923616 ncdv-master-1 devstack@n-api.service[138680]: exception: Instance 8872e8cf-593e-4a7a-940d-e0c8159d4ad7 could not be found. Jul 16 21:11:26.923616 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': '8872e8cf-593e-4a7a-940d-e0c8159d4ad7'} Jul 16 21:11:26.924446 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:11:26.924446 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:11:26.924446 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:11:26.924446 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:11:26.924446 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:11:26.924446 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:11:26.924446 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:11:26.924446 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:11:26.924446 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:11:26.924446 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:11:26.924446 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 8872e8cf-593e-4a7a-940d-e0c8159d4ad7 could not be found. Jul 16 21:11:26.924446 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:11:26.924446 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:11:26.924446 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:11:26.924446 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:11:26.924446 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:11:26.924446 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:11:26.924446 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:11:26.924446 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:11:26.924446 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:11:26.925963 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:11:26.925963 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:11:26.925963 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:11:26.925963 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:11:26.925963 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:11:26.925963 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:11:26.925963 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:11:26.925963 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 8872e8cf-593e-4a7a-940d-e0c8159d4ad7 could not be found. Jul 16 21:11:26.925963 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:11:26.925963 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:11:26.925963 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:11:26.925963 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:11:26.925963 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:11:26.925963 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:11:26.925963 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:11:26.925963 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:11:26.925963 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 8872e8cf-593e-4a7a-940d-e0c8159d4ad7 could not be found. Jul 16 21:11:26.925963 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:11:26.928193 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-56834807-98e0-468d-a849-996028681b00 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Returning 404 to user: Instance 8872e8cf-593e-4a7a-940d-e0c8159d4ad7 could not be found. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:11:26.929229 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-56834807-98e0-468d-a849-996028681b00 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] 10.222.0.16 "GET /compute/v2.1/servers/8872e8cf-593e-4a7a-940d-e0c8159d4ad7" status: 404 len: 111 microversion: 2.1 time: 0.076651 Jul 16 21:11:26.929952 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 470/920] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:11:26 2022] GET /compute/v2.1/servers/8872e8cf-593e-4a7a-940d-e0c8159d4ad7 => generated 111 bytes in 85 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:11:26.936066 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-178e7421-88e5-4b53-9064-91cb6ff2be46 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:11:26.936963 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-178e7421-88e5-4b53-9064-91cb6ff2be46 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] [instance: de210dbd-1398-413f-9a0f-63cae300bd03] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:11:26.946960 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-178e7421-88e5-4b53-9064-91cb6ff2be46 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:26.947976 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-178e7421-88e5-4b53-9064-91cb6ff2be46 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:26.948975 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-178e7421-88e5-4b53-9064-91cb6ff2be46 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:26.950269 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-178e7421-88e5-4b53-9064-91cb6ff2be46 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:27.058514 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-6a11d2fc-17c5-46e7-b3db-a664c6e5a3a3 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'abeb7020019d41a18de42956d706a011', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '649babe54d104436af461f1ee4767001', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:11:27.080087 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-6a11d2fc-17c5-46e7-b3db-a664c6e5a3a3 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] 10.222.0.16 "GET /compute/v2.1/servers/65a70cca-b978-41ab-b890-6052f653c5c3" status: 200 len: 1317 microversion: 2.1 time: 1.052887 Jul 16 21:11:27.112538 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 451/921] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:11:26 2022] GET /compute/v2.1/servers/65a70cca-b978-41ab-b890-6052f653c5c3 => generated 1317 bytes in 1091 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:11:27.123504 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-25d7372e-a71d-4a51-9b02-20d27aa35c4e tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:11:27.136925 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-25d7372e-a71d-4a51-9b02-20d27aa35c4e tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] [instance: 6a391156-ffce-49c0-a79d-47b4544264f5] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:11:27.154592 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-25d7372e-a71d-4a51-9b02-20d27aa35c4e tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:27.156529 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-25d7372e-a71d-4a51-9b02-20d27aa35c4e tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:27.158270 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-25d7372e-a71d-4a51-9b02-20d27aa35c4e tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:27.159442 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-25d7372e-a71d-4a51-9b02-20d27aa35c4e tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:27.616327 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-178e7421-88e5-4b53-9064-91cb6ff2be46 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:11:27.633519 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-178e7421-88e5-4b53-9064-91cb6ff2be46 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:11:27.644446 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-178e7421-88e5-4b53-9064-91cb6ff2be46 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:11:27.646180 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-178e7421-88e5-4b53-9064-91cb6ff2be46 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] 10.222.0.16 "GET /compute/v2.1/servers/de210dbd-1398-413f-9a0f-63cae300bd03" status: 200 len: 1567 microversion: 2.25 time: 0.714946 Jul 16 21:11:27.662689 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 471/922] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:11:26 2022] GET /compute/v2.1/servers/de210dbd-1398-413f-9a0f-63cae300bd03 => generated 1567 bytes in 729 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:11:27.672222 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-d1dc89a8-71dd-4b2f-a8d3-3a120fa8cd61 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:11:27.672741 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-d1dc89a8-71dd-4b2f-a8d3-3a120fa8cd61 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] [instance: e506a6e0-fbcd-4e53-a6a0-d31761654c92] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:11:27.693660 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d1dc89a8-71dd-4b2f-a8d3-3a120fa8cd61 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:27.703162 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d1dc89a8-71dd-4b2f-a8d3-3a120fa8cd61 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:27.703162 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d1dc89a8-71dd-4b2f-a8d3-3a120fa8cd61 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:27.703162 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d1dc89a8-71dd-4b2f-a8d3-3a120fa8cd61 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:27.743297 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-d1dc89a8-71dd-4b2f-a8d3-3a120fa8cd61 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] HTTP exception thrown: Instance e506a6e0-fbcd-4e53-a6a0-d31761654c92 could not be found. Jul 16 21:11:27.744815 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-d1dc89a8-71dd-4b2f-a8d3-3a120fa8cd61 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Request method failure captured: Jul 16 21:11:27.744815 ncdv-master-1 devstack@n-api.service[138680]: request: DELETE /compute/v2.1/servers/e506a6e0-fbcd-4e53-a6a0-d31761654c92 HTTP/1.1 Jul 16 21:11:27.744815 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:11:27.744815 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:11:27.744815 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:11:27.744815 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 0 Jul 16 21:11:27.744815 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:11:27.744815 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:11:27.744815 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:11:27.744815 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:11:27.744815 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0yjHRLOyvkW4FtfKJ0aopiZmEAMH7-zISqbih_qd9IN9V6rg6ySi183S78Jj7U1G1bZhLF_06FgKgnd6qfUgBYrxxNzIlkQaM2RTs_72Xwmzh_mbrOgpxdZycKk72lEJy3fGUTvXNUG7g578TevOCjVVpUrtiC3IxL1PtBbk7YGZvrA Jul 16 21:11:27.744815 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:11:27.744815 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:11:27.744815 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:11:27.744815 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:11:27.744815 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:11:27.744815 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:11:27.744815 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: d0ee02e4c8b448f2b77798ae504129b9 Jul 16 21:11:27.744815 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-ServersWithSpecificFlavorTestJSON-1854487373 Jul 16 21:11:27.744815 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:11:27.744815 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:11:27.746639 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/d0ee02e4c8b448f2b77798ae504129b9"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/d0ee02e4c8b448f2b77798ae504129b9"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_d0ee02e4c8b448f2b77798ae504129b9"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/d0ee02e4c8b448f2b77798ae504129b9"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:11:27.746639 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-ServersWithSpecificFlavorTestJSON-1854487373 Jul 16 21:11:27.746639 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: d0ee02e4c8b448f2b77798ae504129b9 Jul 16 21:11:27.746639 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-ServersWithSpecificFlavorTestJSON-1854487373 Jul 16 21:11:27.746639 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-ServersWithSpecificFlavorTestJSON-1854487373-project Jul 16 21:11:27.746639 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:11:27.746639 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:11:27.746639 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: ef9cbce2b2e64665b7f50767342dcb46 Jul 16 21:11:27.746639 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-ServersWithSpecificFlavorTestJSON-1854487373-project Jul 16 21:11:27.746639 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 21:11:27.746639 ncdv-master-1 devstack@n-api.service[138680]: exception: Instance e506a6e0-fbcd-4e53-a6a0-d31761654c92 could not be found. Jul 16 21:11:27.746639 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': 'e506a6e0-fbcd-4e53-a6a0-d31761654c92'} Jul 16 21:11:27.747409 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:11:27.747409 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:11:27.747409 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:11:27.747409 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:11:27.747409 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:11:27.747409 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:11:27.747409 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:11:27.747409 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:11:27.747409 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:11:27.747409 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:11:27.747409 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance e506a6e0-fbcd-4e53-a6a0-d31761654c92 could not be found. Jul 16 21:11:27.747409 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:11:27.747409 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:11:27.747409 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:11:27.747409 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:11:27.747409 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:11:27.747409 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:11:27.747409 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:11:27.747409 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:11:27.747409 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:11:27.752437 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:11:27.752437 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Jul 16 21:11:27.752437 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Jul 16 21:11:27.752437 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Jul 16 21:11:27.752437 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Jul 16 21:11:27.752437 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:11:27.752437 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:11:27.752437 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:11:27.752437 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:11:27.752437 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance e506a6e0-fbcd-4e53-a6a0-d31761654c92 could not be found. Jul 16 21:11:27.752437 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:11:27.752437 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:11:27.752437 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:11:27.752437 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:11:27.752437 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:11:27.752437 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:11:27.752437 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:11:27.752437 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:11:27.752437 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance e506a6e0-fbcd-4e53-a6a0-d31761654c92 could not be found. Jul 16 21:11:27.752437 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:11:27.755118 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-d1dc89a8-71dd-4b2f-a8d3-3a120fa8cd61 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Returning 404 to user: Instance e506a6e0-fbcd-4e53-a6a0-d31761654c92 could not be found. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:11:27.755118 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-d1dc89a8-71dd-4b2f-a8d3-3a120fa8cd61 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] 10.222.0.16 "DELETE /compute/v2.1/servers/e506a6e0-fbcd-4e53-a6a0-d31761654c92" status: 404 len: 111 microversion: 2.1 time: 0.087281 Jul 16 21:11:27.755118 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 472/923] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:11:27 2022] DELETE /compute/v2.1/servers/e506a6e0-fbcd-4e53-a6a0-d31761654c92 => generated 111 bytes in 88 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:11:27.767839 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-7cd2134b-3ba6-418f-b5cf-d34b6402ac9e tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:11:27.770282 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-7cd2134b-3ba6-418f-b5cf-d34b6402ac9e tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] [instance: b07f0d53-2cd4-440f-be47-077703aa66e6] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:11:27.786800 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7cd2134b-3ba6-418f-b5cf-d34b6402ac9e tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:27.787287 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7cd2134b-3ba6-418f-b5cf-d34b6402ac9e tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:27.793237 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7cd2134b-3ba6-418f-b5cf-d34b6402ac9e tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:27.794087 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7cd2134b-3ba6-418f-b5cf-d34b6402ac9e tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:27.947860 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-7cd2134b-3ba6-418f-b5cf-d34b6402ac9e tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '52281b3989a04ca2a50b0bc93bf783f0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '30d028fa9b854c8fb71c8cbed2985c27', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:11:27.960219 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-25d7372e-a71d-4a51-9b02-20d27aa35c4e tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a69f9dee03d43beaafa966216b9e289', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9b769bf8f3ea4e368bf68de1df6538f9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:11:27.975301 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-7cd2134b-3ba6-418f-b5cf-d34b6402ac9e tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] 10.222.0.16 "GET /compute/v2.1/servers/b07f0d53-2cd4-440f-be47-077703aa66e6" status: 200 len: 1244 microversion: 2.1 time: 0.220545 Jul 16 21:11:27.977308 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-25d7372e-a71d-4a51-9b02-20d27aa35c4e tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] 10.222.0.16 "GET /compute/v2.1/servers/6a391156-ffce-49c0-a79d-47b4544264f5" status: 200 len: 1361 microversion: 2.1 time: 0.862602 Jul 16 21:11:27.983432 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 473/924] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:11:27 2022] GET /compute/v2.1/servers/b07f0d53-2cd4-440f-be47-077703aa66e6 => generated 1244 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:11:27.989942 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 452/925] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:11:27 2022] GET /compute/v2.1/servers/6a391156-ffce-49c0-a79d-47b4544264f5 => generated 1361 bytes in 876 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:11:27.991067 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-55e5f173-f1db-4943-8f4b-30f5fdd817f0 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:11:27.994310 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-55e5f173-f1db-4943-8f4b-30f5fdd817f0 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] [instance: e506a6e0-fbcd-4e53-a6a0-d31761654c92] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:11:27.994845 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-696cf062-21b7-4cc5-9392-169ff61bbe49 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:11:27.995566 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-696cf062-21b7-4cc5-9392-169ff61bbe49 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:11:28.004752 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-55e5f173-f1db-4943-8f4b-30f5fdd817f0 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:28.005251 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-55e5f173-f1db-4943-8f4b-30f5fdd817f0 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:28.006232 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-55e5f173-f1db-4943-8f4b-30f5fdd817f0 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:28.007331 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-55e5f173-f1db-4943-8f4b-30f5fdd817f0 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:28.021665 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-696cf062-21b7-4cc5-9392-169ff61bbe49 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:28.022682 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-696cf062-21b7-4cc5-9392-169ff61bbe49 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:28.025120 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-696cf062-21b7-4cc5-9392-169ff61bbe49 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:28.026362 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-696cf062-21b7-4cc5-9392-169ff61bbe49 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:28.028582 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-55e5f173-f1db-4943-8f4b-30f5fdd817f0 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] HTTP exception thrown: Instance e506a6e0-fbcd-4e53-a6a0-d31761654c92 could not be found. Jul 16 21:11:28.030441 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-55e5f173-f1db-4943-8f4b-30f5fdd817f0 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Request method failure captured: Jul 16 21:11:28.030441 ncdv-master-1 devstack@n-api.service[138680]: request: GET /compute/v2.1/servers/e506a6e0-fbcd-4e53-a6a0-d31761654c92 HTTP/1.1 Jul 16 21:11:28.030441 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:11:28.030441 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:11:28.030441 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:11:28.030441 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 0 Jul 16 21:11:28.030441 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:11:28.030441 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:11:28.030441 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:11:28.030441 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:11:28.030441 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0yjHRLOyvkW4FtfKJ0aopiZmEAMH7-zISqbih_qd9IN9V6rg6ySi183S78Jj7U1G1bZhLF_06FgKgnd6qfUgBYrxxNzIlkQaM2RTs_72Xwmzh_mbrOgpxdZycKk72lEJy3fGUTvXNUG7g578TevOCjVVpUrtiC3IxL1PtBbk7YGZvrA Jul 16 21:11:28.030441 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:11:28.030441 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:11:28.030441 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:11:28.030441 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:11:28.030441 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:11:28.030441 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:11:28.030441 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: d0ee02e4c8b448f2b77798ae504129b9 Jul 16 21:11:28.030441 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-ServersWithSpecificFlavorTestJSON-1854487373 Jul 16 21:11:28.030441 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:11:28.030441 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:11:28.032050 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/d0ee02e4c8b448f2b77798ae504129b9"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/d0ee02e4c8b448f2b77798ae504129b9"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_d0ee02e4c8b448f2b77798ae504129b9"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/d0ee02e4c8b448f2b77798ae504129b9"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:11:28.032050 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-ServersWithSpecificFlavorTestJSON-1854487373 Jul 16 21:11:28.032050 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: d0ee02e4c8b448f2b77798ae504129b9 Jul 16 21:11:28.032050 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-ServersWithSpecificFlavorTestJSON-1854487373 Jul 16 21:11:28.032050 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-ServersWithSpecificFlavorTestJSON-1854487373-project Jul 16 21:11:28.032050 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:11:28.032050 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:11:28.032050 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: ef9cbce2b2e64665b7f50767342dcb46 Jul 16 21:11:28.032050 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-ServersWithSpecificFlavorTestJSON-1854487373-project Jul 16 21:11:28.032050 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 21:11:28.032050 ncdv-master-1 devstack@n-api.service[138680]: exception: Instance e506a6e0-fbcd-4e53-a6a0-d31761654c92 could not be found. Jul 16 21:11:28.032050 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': 'e506a6e0-fbcd-4e53-a6a0-d31761654c92'} Jul 16 21:11:28.032874 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:11:28.032874 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:11:28.032874 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:11:28.032874 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:11:28.032874 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:11:28.032874 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:11:28.032874 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:11:28.032874 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:11:28.032874 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:11:28.032874 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:11:28.032874 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance e506a6e0-fbcd-4e53-a6a0-d31761654c92 could not be found. Jul 16 21:11:28.032874 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:11:28.032874 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:11:28.032874 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:11:28.032874 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:11:28.032874 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:11:28.032874 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:11:28.032874 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:11:28.032874 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:11:28.032874 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:11:28.034245 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:11:28.034245 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:11:28.034245 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:11:28.034245 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:11:28.034245 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:11:28.034245 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:11:28.034245 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:11:28.034245 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance e506a6e0-fbcd-4e53-a6a0-d31761654c92 could not be found. Jul 16 21:11:28.034245 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:11:28.034245 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:11:28.034245 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:11:28.034245 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:11:28.034245 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:11:28.034245 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:11:28.034245 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:11:28.034245 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:11:28.034245 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance e506a6e0-fbcd-4e53-a6a0-d31761654c92 could not be found. Jul 16 21:11:28.034245 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:11:28.042379 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-55e5f173-f1db-4943-8f4b-30f5fdd817f0 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] Returning 404 to user: Instance e506a6e0-fbcd-4e53-a6a0-d31761654c92 could not be found. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:11:28.042379 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-55e5f173-f1db-4943-8f4b-30f5fdd817f0 tempest-ServersWithSpecificFlavorTestJSON-1854487373 tempest-ServersWithSpecificFlavorTestJSON-1854487373-project] 10.222.0.16 "GET /compute/v2.1/servers/e506a6e0-fbcd-4e53-a6a0-d31761654c92" status: 404 len: 111 microversion: 2.1 time: 0.056994 Jul 16 21:11:28.042379 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 474/926] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:11:27 2022] GET /compute/v2.1/servers/e506a6e0-fbcd-4e53-a6a0-d31761654c92 => generated 111 bytes in 59 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:11:28.117313 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-6f316580-2821-43c4-8187-103f703be655 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:11:28.118089 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-6f316580-2821-43c4-8187-103f703be655 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] [instance: 65a70cca-b978-41ab-b890-6052f653c5c3] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:11:28.126436 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6f316580-2821-43c4-8187-103f703be655 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:28.129682 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6f316580-2821-43c4-8187-103f703be655 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:28.131909 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6f316580-2821-43c4-8187-103f703be655 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:28.134325 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6f316580-2821-43c4-8187-103f703be655 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:28.499421 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-696cf062-21b7-4cc5-9392-169ff61bbe49 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b68dacc4b8463ba07fae6372131117', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55a19b7a4682457a9132f5fc80110712', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:11:28.501805 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=100 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:11:28.515055 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-696cf062-21b7-4cc5-9392-169ff61bbe49 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "GET /compute/v2.1/servers/a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2" status: 200 len: 1528 microversion: 2.1 time: 0.523785 Jul 16 21:11:28.515559 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 453/927] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:11:27 2022] GET /compute/v2.1/servers/a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2 => generated 1528 bytes in 525 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:11:28.539686 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-6f316580-2821-43c4-8187-103f703be655 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'abeb7020019d41a18de42956d706a011', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '649babe54d104436af461f1ee4767001', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:11:28.547033 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-fbc12a02-0873-4a28-869b-90eb4e8421b9 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:11:28.547453 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-fbc12a02-0873-4a28-869b-90eb4e8421b9 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:11:28.562833 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-6f316580-2821-43c4-8187-103f703be655 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] 10.222.0.16 "GET /compute/v2.1/servers/65a70cca-b978-41ab-b890-6052f653c5c3" status: 200 len: 1317 microversion: 2.1 time: 0.446837 Jul 16 21:11:28.563499 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 475/928] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:11:28 2022] GET /compute/v2.1/servers/65a70cca-b978-41ab-b890-6052f653c5c3 => generated 1317 bytes in 452 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:11:28.578283 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fbc12a02-0873-4a28-869b-90eb4e8421b9 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:28.582787 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fbc12a02-0873-4a28-869b-90eb4e8421b9 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:28.582787 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fbc12a02-0873-4a28-869b-90eb4e8421b9 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:28.584243 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fbc12a02-0873-4a28-869b-90eb4e8421b9 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:28.667776 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-16118f31-2ef6-438a-895a-ce074c9e4ab6 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:11:28.668860 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-16118f31-2ef6-438a-895a-ce074c9e4ab6 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] [instance: de210dbd-1398-413f-9a0f-63cae300bd03] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:11:28.686563 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-16118f31-2ef6-438a-895a-ce074c9e4ab6 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:28.688383 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-16118f31-2ef6-438a-895a-ce074c9e4ab6 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:28.691683 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-16118f31-2ef6-438a-895a-ce074c9e4ab6 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:28.692855 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-16118f31-2ef6-438a-895a-ce074c9e4ab6 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:29.094788 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-fbc12a02-0873-4a28-869b-90eb4e8421b9 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b68dacc4b8463ba07fae6372131117', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55a19b7a4682457a9132f5fc80110712', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:11:29.102595 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-fbc12a02-0873-4a28-869b-90eb4e8421b9 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "GET /compute/v2.1/servers/a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2" status: 200 len: 1528 microversion: 2.1 time: 0.562980 Jul 16 21:11:29.105030 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 454/929] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:11:28 2022] GET /compute/v2.1/servers/a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2 => generated 1528 bytes in 566 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:11:29.116155 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-1b39ef79-705e-4c75-a1b5-1a451a3ef792 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:11:29.127989 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-1b39ef79-705e-4c75-a1b5-1a451a3ef792 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] [instance: b07f0d53-2cd4-440f-be47-077703aa66e6] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:11:29.134623 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1b39ef79-705e-4c75-a1b5-1a451a3ef792 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:29.138550 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1b39ef79-705e-4c75-a1b5-1a451a3ef792 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:29.138550 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1b39ef79-705e-4c75-a1b5-1a451a3ef792 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:29.138550 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1b39ef79-705e-4c75-a1b5-1a451a3ef792 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:29.283459 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-1b39ef79-705e-4c75-a1b5-1a451a3ef792 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '52281b3989a04ca2a50b0bc93bf783f0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '30d028fa9b854c8fb71c8cbed2985c27', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:11:29.291903 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-1b39ef79-705e-4c75-a1b5-1a451a3ef792 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] 10.222.0.16 "GET /compute/v2.1/servers/b07f0d53-2cd4-440f-be47-077703aa66e6" status: 200 len: 1300 microversion: 2.1 time: 0.184333 Jul 16 21:11:29.299147 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 455/930] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:11:29 2022] GET /compute/v2.1/servers/b07f0d53-2cd4-440f-be47-077703aa66e6 => generated 1300 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:11:29.301973 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-2fb52ec6-67be-4efd-98cc-bf5644e92ccf tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:11:29.302674 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-2fb52ec6-67be-4efd-98cc-bf5644e92ccf tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] [instance: 6a391156-ffce-49c0-a79d-47b4544264f5] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:11:29.321660 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2fb52ec6-67be-4efd-98cc-bf5644e92ccf tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:29.322144 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2fb52ec6-67be-4efd-98cc-bf5644e92ccf tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:29.322966 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2fb52ec6-67be-4efd-98cc-bf5644e92ccf tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:29.323432 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2fb52ec6-67be-4efd-98cc-bf5644e92ccf tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:29.336126 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-16118f31-2ef6-438a-895a-ce074c9e4ab6 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:11:29.350057 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-16118f31-2ef6-438a-895a-ce074c9e4ab6 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:11:29.352614 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-16118f31-2ef6-438a-895a-ce074c9e4ab6 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:11:29.355475 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-16118f31-2ef6-438a-895a-ce074c9e4ab6 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] 10.222.0.16 "GET /compute/v2.1/servers/de210dbd-1398-413f-9a0f-63cae300bd03" status: 200 len: 1557 microversion: 2.25 time: 0.693515 Jul 16 21:11:29.372886 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 476/931] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:11:28 2022] GET /compute/v2.1/servers/de210dbd-1398-413f-9a0f-63cae300bd03 => generated 1557 bytes in 709 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 16 21:11:29.379551 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-619ca63e-4f33-484f-b81d-d3bb61e6c3c6 req-00852302-0090-4f48-b83e-b852d0418c31 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "6a391156-ffce-49c0-a79d-47b4544264f5", "name": "network-vif-plugged", "status": "completed", "tag": "a8de1bba-f28a-49ad-92e5-8a10d67f1e75"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:11:29.392619 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-619ca63e-4f33-484f-b81d-d3bb61e6c3c6 req-00852302-0090-4f48-b83e-b852d0418c31 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:29.393712 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-619ca63e-4f33-484f-b81d-d3bb61e6c3c6 req-00852302-0090-4f48-b83e-b852d0418c31 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:29.409160 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-619ca63e-4f33-484f-b81d-d3bb61e6c3c6 req-00852302-0090-4f48-b83e-b852d0418c31 service nova] Creating event network-vif-plugged:a8de1bba-f28a-49ad-92e5-8a10d67f1e75 for instance 6a391156-ffce-49c0-a79d-47b4544264f5 on nch2-master-1 Jul 16 21:11:29.415680 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-619ca63e-4f33-484f-b81d-d3bb61e6c3c6 req-00852302-0090-4f48-b83e-b852d0418c31 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.040573 Jul 16 21:11:29.417097 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 477/932] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:11:29 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:11:29.433494 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-db967ae5-1741-4c35-8fe2-fe843250cde9 tempest-MigrationsAdminTest-399208122 tempest-MigrationsAdminTest-399208122-project-admin] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:11:29.433494 ncdv-master-1 devstack@n-api.service[138680]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-flavor-manage:delete": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:11:29.433494 ncdv-master-1 devstack@n-api.service[138680]: warnings.warn(msg) Jul 16 21:11:29.551219 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-db967ae5-1741-4c35-8fe2-fe843250cde9 tempest-MigrationsAdminTest-399208122 tempest-MigrationsAdminTest-399208122-project-admin] 10.222.0.16 "DELETE /compute/v2.1/flavors/228f3e42-8394-4732-ac78-3c79d14592dd" status: 202 len: 0 microversion: 2.1 time: 0.131930 Jul 16 21:11:29.555360 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 478/933] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:11:29 2022] DELETE /compute/v2.1/flavors/228f3e42-8394-4732-ac78-3c79d14592dd => generated 0 bytes in 134 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 21:11:29.566426 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-2b54dd56-36b4-4470-a337-5d8ceb415ea4 tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] Calling method '.version_select at 0x7fdd9d1a9670>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:11:29.566426 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2b54dd56-36b4-4470-a337-5d8ceb415ea4 tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:29.566426 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2b54dd56-36b4-4470-a337-5d8ceb415ea4 tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:29.582442 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-2b54dd56-36b4-4470-a337-5d8ceb415ea4 tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] 10.222.0.16 "GET /compute/v2.1/os-migrations" status: 200 len: 2365 microversion: 2.25 time: 0.029383 Jul 16 21:11:29.595858 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 479/934] 10.222.0.16 () {60 vars in 1183 bytes} [Sat Jul 16 21:11:29 2022] GET /compute/v2.1/os-migrations => generated 2365 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:11:29.600785 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-0e0f0a4b-ee13-48b5-aa7f-bf21462cb35f tempest-MigrationsAdminTest-399208122 tempest-MigrationsAdminTest-399208122-project-admin] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:11:29.637005 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-0e0f0a4b-ee13-48b5-aa7f-bf21462cb35f tempest-MigrationsAdminTest-399208122 tempest-MigrationsAdminTest-399208122-project-admin] 10.222.0.16 "GET /compute/v2.1/flavors/detail" status: 200 len: 4991 microversion: 2.1 time: 0.039556 Jul 16 21:11:29.655183 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 480/935] 10.222.0.16 () {58 vars in 1145 bytes} [Sat Jul 16 21:11:29 2022] GET /compute/v2.1/flavors/detail => generated 4991 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:11:29.655575 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-2fb52ec6-67be-4efd-98cc-bf5644e92ccf tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a69f9dee03d43beaafa966216b9e289', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9b769bf8f3ea4e368bf68de1df6538f9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:11:29.656030 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-68e54124-1535-4d58-80ff-6945c28c9233 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:11:29.656030 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-68e54124-1535-4d58-80ff-6945c28c9233 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] [instance: 65a70cca-b978-41ab-b890-6052f653c5c3] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:11:29.662767 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-2fb52ec6-67be-4efd-98cc-bf5644e92ccf tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] 10.222.0.16 "GET /compute/v2.1/servers/6a391156-ffce-49c0-a79d-47b4544264f5" status: 200 len: 1361 microversion: 2.1 time: 0.364546 Jul 16 21:11:29.667414 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-68e54124-1535-4d58-80ff-6945c28c9233 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:29.669157 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-68e54124-1535-4d58-80ff-6945c28c9233 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:29.673648 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-68e54124-1535-4d58-80ff-6945c28c9233 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:29.674519 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-68e54124-1535-4d58-80ff-6945c28c9233 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:29.675770 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 456/936] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:11:29 2022] GET /compute/v2.1/servers/6a391156-ffce-49c0-a79d-47b4544264f5 => generated 1361 bytes in 378 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:11:29.680667 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-edfcfa77-2443-4300-b1a5-c5dd4a240f38 tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:11:29.700399 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-edfcfa77-2443-4300-b1a5-c5dd4a240f38 tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] [instance: de210dbd-1398-413f-9a0f-63cae300bd03] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:11:29.706656 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-edfcfa77-2443-4300-b1a5-c5dd4a240f38 tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:29.708537 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-edfcfa77-2443-4300-b1a5-c5dd4a240f38 tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:29.709603 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-edfcfa77-2443-4300-b1a5-c5dd4a240f38 tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:29.715673 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-edfcfa77-2443-4300-b1a5-c5dd4a240f38 tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:30.052476 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-68e54124-1535-4d58-80ff-6945c28c9233 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'abeb7020019d41a18de42956d706a011', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '649babe54d104436af461f1ee4767001', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:11:30.071460 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-68e54124-1535-4d58-80ff-6945c28c9233 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] 10.222.0.16 "GET /compute/v2.1/servers/65a70cca-b978-41ab-b890-6052f653c5c3" status: 200 len: 1317 microversion: 2.1 time: 0.418608 Jul 16 21:11:30.071460 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 481/937] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:11:29 2022] GET /compute/v2.1/servers/65a70cca-b978-41ab-b890-6052f653c5c3 => generated 1317 bytes in 424 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:11:30.071460 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-5dbbdeaa-24e3-4f91-923e-a8e4da172ccb tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Action: 'action', calling method: >, body: {"resize": {"flavorRef": "451"}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:11:30.071460 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-5dbbdeaa-24e3-4f91-923e-a8e4da172ccb tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:11:30.076074 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.objects.instance [None req-edfcfa77-2443-4300-b1a5-c5dd4a240f38 tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] Lazy-loading 'services' on Instance uuid de210dbd-1398-413f-9a0f-63cae300bd03 {{(pid=138681) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:11:30.084793 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5dbbdeaa-24e3-4f91-923e-a8e4da172ccb tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:30.088693 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5dbbdeaa-24e3-4f91-923e-a8e4da172ccb tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:30.094951 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5dbbdeaa-24e3-4f91-923e-a8e4da172ccb tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:30.100116 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-edfcfa77-2443-4300-b1a5-c5dd4a240f38 tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/de210dbd-1398-413f-9a0f-63cae300bd03" status: 200 len: 2050 microversion: 2.25 time: 0.422905 Jul 16 21:11:30.101386 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5dbbdeaa-24e3-4f91-923e-a8e4da172ccb tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.011s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:30.115514 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 457/938] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:11:29 2022] GET /compute/v2.1/servers/de210dbd-1398-413f-9a0f-63cae300bd03 => generated 2050 bytes in 439 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:11:30.283043 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-4fd8ee23-7f4c-4780-acd0-0237d56fe0c6 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-LiveAutoBlockMigrationV225Test-server-1301038122", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452", "networks": [{"uuid": "b3fd4d7b-96dc-4cc4-8a4c-4e1986f1bc15"}]}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:11:30.300582 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-5dbbdeaa-24e3-4f91-923e-a8e4da172ccb tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Policy check for compute:servers:resize:cross_cell failed with credentials {'is_admin': False, 'user_id': 'a9b68dacc4b8463ba07fae6372131117', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55a19b7a4682457a9132f5fc80110712', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:11:30.304181 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_api:SELECT=92,nova_api:INSERT=4,nova_api:DELETE=4 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:11:30.308643 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-5dbbdeaa-24e3-4f91-923e-a8e4da172ccb tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2] Old instance type tempest-test_resize_flavor_-1563016479, new instance type m1.heat {{(pid=138680) resize /opt/stack/nova/nova/compute/api.py:4166}} Jul 16 21:11:30.323739 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-5dbbdeaa-24e3-4f91-923e-a8e4da172ccb tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Flavor limits 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 16 21:11:30.325085 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-5dbbdeaa-24e3-4f91-923e-a8e4da172ccb tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Image limits 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 16 21:11:30.329988 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-5dbbdeaa-24e3-4f91-923e-a8e4da172ccb tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Flavor pref 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 16 21:11:30.330614 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-5dbbdeaa-24e3-4f91-923e-a8e4da172ccb tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Image pref 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 16 21:11:30.331186 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-5dbbdeaa-24e3-4f91-923e-a8e4da172ccb tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 16 21:11:30.379041 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.network.neutron [None req-4fd8ee23-7f4c-4780-acd0-0237d56fe0c6 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] validate_networks() for [('b3fd4d7b-96dc-4cc4-8a4c-4e1986f1bc15', None, None, None, None, None)] {{(pid=138681) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 16 21:11:30.648305 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-5dbbdeaa-24e3-4f91-923e-a8e4da172ccb tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "POST /compute/v2.1/servers/a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2/action" status: 202 len: 0 microversion: 2.1 time: 0.581493 Jul 16 21:11:30.653214 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 482/939] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 21:11:30 2022] POST /compute/v2.1/servers/a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2/action => generated 0 bytes in 590 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 21:11:30.670334 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-b0b5bd9a-c885-4865-95bf-6ebb944f71cc tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:11:30.672367 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-b0b5bd9a-c885-4865-95bf-6ebb944f71cc tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] [instance: b07f0d53-2cd4-440f-be47-077703aa66e6] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:11:30.706072 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b0b5bd9a-c885-4865-95bf-6ebb944f71cc tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:30.711300 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b0b5bd9a-c885-4865-95bf-6ebb944f71cc tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:30.715594 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b0b5bd9a-c885-4865-95bf-6ebb944f71cc tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:30.717143 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b0b5bd9a-c885-4865-95bf-6ebb944f71cc tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:31.219996 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-b0b5bd9a-c885-4865-95bf-6ebb944f71cc tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '52281b3989a04ca2a50b0bc93bf783f0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '30d028fa9b854c8fb71c8cbed2985c27', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:11:31.231327 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-b0b5bd9a-c885-4865-95bf-6ebb944f71cc tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] 10.222.0.16 "GET /compute/v2.1/servers/b07f0d53-2cd4-440f-be47-077703aa66e6" status: 200 len: 1298 microversion: 2.1 time: 0.573411 Jul 16 21:11:31.232071 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 483/940] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:11:30 2022] GET /compute/v2.1/servers/b07f0d53-2cd4-440f-be47-077703aa66e6 => generated 1298 bytes in 578 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:11:31.237499 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-06f05d50-7225-4a27-be00-12d0a4080b58 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:11:31.238419 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-06f05d50-7225-4a27-be00-12d0a4080b58 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:11:31.247108 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-06f05d50-7225-4a27-be00-12d0a4080b58 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:31.247806 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-06f05d50-7225-4a27-be00-12d0a4080b58 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:31.251457 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-06f05d50-7225-4a27-be00-12d0a4080b58 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:31.251457 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-06f05d50-7225-4a27-be00-12d0a4080b58 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:32.004655 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-4fd8ee23-7f4c-4780-acd0-0237d56fe0c6 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Flavor limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 16 21:11:32.006196 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-4fd8ee23-7f4c-4780-acd0-0237d56fe0c6 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Image limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 16 21:11:32.007553 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-4fd8ee23-7f4c-4780-acd0-0237d56fe0c6 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Flavor pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 16 21:11:32.008278 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-4fd8ee23-7f4c-4780-acd0-0237d56fe0c6 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Image pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 16 21:11:32.008918 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-4fd8ee23-7f4c-4780-acd0-0237d56fe0c6 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 16 21:11:32.028861 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell0:SELECT=23 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:11:32.039352 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4fd8ee23-7f4c-4780-acd0-0237d56fe0c6 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:32.040102 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4fd8ee23-7f4c-4780-acd0-0237d56fe0c6 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:32.041314 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-06f05d50-7225-4a27-be00-12d0a4080b58 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b68dacc4b8463ba07fae6372131117', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55a19b7a4682457a9132f5fc80110712', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:11:32.044746 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4fd8ee23-7f4c-4780-acd0-0237d56fe0c6 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:32.047647 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4fd8ee23-7f4c-4780-acd0-0237d56fe0c6 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:32.050697 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-06f05d50-7225-4a27-be00-12d0a4080b58 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "GET /compute/v2.1/servers/a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2" status: 200 len: 1537 microversion: 2.1 time: 0.816812 Jul 16 21:11:32.051413 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 484/941] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:11:31 2022] GET /compute/v2.1/servers/a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2 => generated 1537 bytes in 818 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:11:32.062929 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-df673515-d9cd-4efb-acc5-e1a273740c62 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:11:32.063564 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-df673515-d9cd-4efb-acc5-e1a273740c62 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] [instance: 6a391156-ffce-49c0-a79d-47b4544264f5] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:11:32.086718 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-df673515-d9cd-4efb-acc5-e1a273740c62 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:32.087399 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-df673515-d9cd-4efb-acc5-e1a273740c62 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:32.088250 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-df673515-d9cd-4efb-acc5-e1a273740c62 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:32.088789 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-df673515-d9cd-4efb-acc5-e1a273740c62 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:32.098494 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-4fd8ee23-7f4c-4780-acd0-0237d56fe0c6 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Getting quotas for project d9b92fc544c544f686ac2ab7ff5123b6. Resources: {'ram', 'instances', 'cores'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:11:32.107509 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-4fd8ee23-7f4c-4780-acd0-0237d56fe0c6 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Getting quotas for user 578b87f863b34f57a571f7e56d44f118 and project d9b92fc544c544f686ac2ab7ff5123b6. Resources: {'ram', 'instances', 'cores'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:11:32.125785 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-4fd8ee23-7f4c-4780-acd0-0237d56fe0c6 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Going to run 1 instances... {{(pid=138681) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 16 21:11:32.129596 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-4fd8ee23-7f4c-4780-acd0-0237d56fe0c6 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] [instance: 0fb5b5b6-1796-4406-b276-326e1d7dd451] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138681) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 16 21:11:32.316711 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-4fd8ee23-7f4c-4780-acd0-0237d56fe0c6 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.25 time: 2.038957 Jul 16 21:11:32.317953 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 458/942] 10.222.0.16 () {62 vars in 1187 bytes} [Sat Jul 16 21:11:30 2022] POST /compute/v2.1/servers => generated 380 bytes in 2042 msecs (HTTP/1.1 202) 10 headers in 453 bytes (1 switches on core 0) Jul 16 21:11:32.331109 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-f226e5ac-16e2-47e2-8b31-8d8bd9e1561f tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:11:32.331109 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-f226e5ac-16e2-47e2-8b31-8d8bd9e1561f tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] [instance: 65a70cca-b978-41ab-b890-6052f653c5c3] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:11:32.344130 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f226e5ac-16e2-47e2-8b31-8d8bd9e1561f tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:32.344869 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f226e5ac-16e2-47e2-8b31-8d8bd9e1561f tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:32.345845 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f226e5ac-16e2-47e2-8b31-8d8bd9e1561f tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:32.346514 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f226e5ac-16e2-47e2-8b31-8d8bd9e1561f tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:32.512903 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-df673515-d9cd-4efb-acc5-e1a273740c62 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a69f9dee03d43beaafa966216b9e289', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9b769bf8f3ea4e368bf68de1df6538f9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:11:32.523212 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-df673515-d9cd-4efb-acc5-e1a273740c62 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] 10.222.0.16 "GET /compute/v2.1/servers/6a391156-ffce-49c0-a79d-47b4544264f5" status: 200 len: 1361 microversion: 2.1 time: 0.466225 Jul 16 21:11:32.523801 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 485/943] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:11:32 2022] GET /compute/v2.1/servers/6a391156-ffce-49c0-a79d-47b4544264f5 => generated 1361 bytes in 472 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:11:32.552728 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-16b3f90f-3e56-4bac-afc1-82538d26ff64 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:11:32.553787 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-16b3f90f-3e56-4bac-afc1-82538d26ff64 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] [instance: b07f0d53-2cd4-440f-be47-077703aa66e6] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:11:32.580505 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-16b3f90f-3e56-4bac-afc1-82538d26ff64 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:32.581535 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-16b3f90f-3e56-4bac-afc1-82538d26ff64 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:32.584965 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-16b3f90f-3e56-4bac-afc1-82538d26ff64 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:32.585812 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-16b3f90f-3e56-4bac-afc1-82538d26ff64 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:32.609281 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:11:32.940361 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-f226e5ac-16e2-47e2-8b31-8d8bd9e1561f tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'abeb7020019d41a18de42956d706a011', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '649babe54d104436af461f1ee4767001', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:11:32.969889 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-f226e5ac-16e2-47e2-8b31-8d8bd9e1561f tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] 10.222.0.16 "GET /compute/v2.1/servers/65a70cca-b978-41ab-b890-6052f653c5c3" status: 200 len: 1317 microversion: 2.1 time: 0.650051 Jul 16 21:11:32.994062 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 459/944] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:11:32 2022] GET /compute/v2.1/servers/65a70cca-b978-41ab-b890-6052f653c5c3 => generated 1317 bytes in 675 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:11:33.031239 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-5f0a9185-2a6d-4e2b-979a-71fa871509a2 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:11:33.037100 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-5f0a9185-2a6d-4e2b-979a-71fa871509a2 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] [instance: 0fb5b5b6-1796-4406-b276-326e1d7dd451] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:11:33.046876 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-16b3f90f-3e56-4bac-afc1-82538d26ff64 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '52281b3989a04ca2a50b0bc93bf783f0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '30d028fa9b854c8fb71c8cbed2985c27', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:11:33.067214 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-16b3f90f-3e56-4bac-afc1-82538d26ff64 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] 10.222.0.16 "GET /compute/v2.1/servers/b07f0d53-2cd4-440f-be47-077703aa66e6" status: 200 len: 1366 microversion: 2.1 time: 0.526379 Jul 16 21:11:33.068757 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 486/945] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:11:32 2022] GET /compute/v2.1/servers/b07f0d53-2cd4-440f-be47-077703aa66e6 => generated 1366 bytes in 535 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:11:33.088940 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-baa56895-786b-4a11-b323-727b9f7ed9f4 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:11:33.092084 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-baa56895-786b-4a11-b323-727b9f7ed9f4 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:11:33.113033 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-baa56895-786b-4a11-b323-727b9f7ed9f4 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:33.120754 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-baa56895-786b-4a11-b323-727b9f7ed9f4 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.010s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:33.128606 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-baa56895-786b-4a11-b323-727b9f7ed9f4 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:33.129171 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-baa56895-786b-4a11-b323-727b9f7ed9f4 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:33.398685 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-5f0a9185-2a6d-4e2b-979a-71fa871509a2 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:11:33.409210 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-5f0a9185-2a6d-4e2b-979a-71fa871509a2 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:11:33.412054 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-5f0a9185-2a6d-4e2b-979a-71fa871509a2 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:11:33.420677 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-5f0a9185-2a6d-4e2b-979a-71fa871509a2 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] 10.222.0.16 "GET /compute/v2.1/servers/0fb5b5b6-1796-4406-b276-326e1d7dd451" status: 200 len: 1262 microversion: 2.25 time: 0.421184 Jul 16 21:11:33.421958 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 460/946] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:11:32 2022] GET /compute/v2.1/servers/0fb5b5b6-1796-4406-b276-326e1d7dd451 => generated 1262 bytes in 427 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:11:33.521049 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-27b6a499-4f08-4240-a01f-dabc60720c48 req-8449188c-fc7b-4d11-8b15-3a58e498c142 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "65a70cca-b978-41ab-b890-6052f653c5c3", "name": "network-vif-plugged", "status": "completed", "tag": "7782780c-06ee-4b9b-b252-ba55ad0b48d8"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:11:33.544172 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-27b6a499-4f08-4240-a01f-dabc60720c48 req-8449188c-fc7b-4d11-8b15-3a58e498c142 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:33.545549 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-27b6a499-4f08-4240-a01f-dabc60720c48 req-8449188c-fc7b-4d11-8b15-3a58e498c142 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:33.574084 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-27b6a499-4f08-4240-a01f-dabc60720c48 req-8449188c-fc7b-4d11-8b15-3a58e498c142 service nova] Creating event network-vif-plugged:7782780c-06ee-4b9b-b252-ba55ad0b48d8 for instance 65a70cca-b978-41ab-b890-6052f653c5c3 on nch2-master-1 Jul 16 21:11:33.585192 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-27b6a499-4f08-4240-a01f-dabc60720c48 req-8449188c-fc7b-4d11-8b15-3a58e498c142 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.074990 Jul 16 21:11:33.585192 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 461/947] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:11:33 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 77 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 16 21:11:33.599620 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-cc619043-a371-40aa-8739-ca74edcfb89f tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:11:33.600496 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-cc619043-a371-40aa-8739-ca74edcfb89f tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] [instance: 6a391156-ffce-49c0-a79d-47b4544264f5] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:11:33.618415 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cc619043-a371-40aa-8739-ca74edcfb89f tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:33.619236 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cc619043-a371-40aa-8739-ca74edcfb89f tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:33.620238 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cc619043-a371-40aa-8739-ca74edcfb89f tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:33.621393 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cc619043-a371-40aa-8739-ca74edcfb89f tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:33.866976 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-baa56895-786b-4a11-b323-727b9f7ed9f4 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b68dacc4b8463ba07fae6372131117', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55a19b7a4682457a9132f5fc80110712', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:11:33.889116 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-baa56895-786b-4a11-b323-727b9f7ed9f4 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "GET /compute/v2.1/servers/a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2" status: 200 len: 1537 microversion: 2.1 time: 0.814543 Jul 16 21:11:33.890012 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 487/948] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:11:33 2022] GET /compute/v2.1/servers/a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2 => generated 1537 bytes in 821 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:11:34.001176 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-6a66e51d-2703-434f-bc6f-ba39de47562b tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:11:34.002155 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-6a66e51d-2703-434f-bc6f-ba39de47562b tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] [instance: 65a70cca-b978-41ab-b890-6052f653c5c3] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:11:34.027426 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6a66e51d-2703-434f-bc6f-ba39de47562b tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:34.028650 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6a66e51d-2703-434f-bc6f-ba39de47562b tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:34.033757 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6a66e51d-2703-434f-bc6f-ba39de47562b tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:34.034410 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6a66e51d-2703-434f-bc6f-ba39de47562b tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:34.055159 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-cc619043-a371-40aa-8739-ca74edcfb89f tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a69f9dee03d43beaafa966216b9e289', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9b769bf8f3ea4e368bf68de1df6538f9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:11:34.069601 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-cc619043-a371-40aa-8739-ca74edcfb89f tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] 10.222.0.16 "GET /compute/v2.1/servers/6a391156-ffce-49c0-a79d-47b4544264f5" status: 200 len: 1543 microversion: 2.1 time: 0.484662 Jul 16 21:11:34.070206 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 462/949] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:11:33 2022] GET /compute/v2.1/servers/6a391156-ffce-49c0-a79d-47b4544264f5 => generated 1543 bytes in 486 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:11:34.090031 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-17188fa7-1d22-4838-9755-d4c9e74fb9a9 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:11:34.090789 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-17188fa7-1d22-4838-9755-d4c9e74fb9a9 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] [instance: b07f0d53-2cd4-440f-be47-077703aa66e6] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:11:34.110625 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-17188fa7-1d22-4838-9755-d4c9e74fb9a9 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:34.110915 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-17188fa7-1d22-4838-9755-d4c9e74fb9a9 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:34.114307 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-17188fa7-1d22-4838-9755-d4c9e74fb9a9 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:34.114772 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-17188fa7-1d22-4838-9755-d4c9e74fb9a9 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:34.612308 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-6a66e51d-2703-434f-bc6f-ba39de47562b tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'abeb7020019d41a18de42956d706a011', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '649babe54d104436af461f1ee4767001', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:11:34.633880 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-6a66e51d-2703-434f-bc6f-ba39de47562b tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] 10.222.0.16 "GET /compute/v2.1/servers/65a70cca-b978-41ab-b890-6052f653c5c3" status: 200 len: 1317 microversion: 2.1 time: 0.640126 Jul 16 21:11:34.635173 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 488/950] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:11:33 2022] GET /compute/v2.1/servers/65a70cca-b978-41ab-b890-6052f653c5c3 => generated 1317 bytes in 642 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:11:34.651147 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-f767e12a-346b-41ba-bd63-e665f0fa7bb4 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:11:34.651147 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-f767e12a-346b-41ba-bd63-e665f0fa7bb4 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] [instance: 6a391156-ffce-49c0-a79d-47b4544264f5] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:11:34.666739 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f767e12a-346b-41ba-bd63-e665f0fa7bb4 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:34.667403 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f767e12a-346b-41ba-bd63-e665f0fa7bb4 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:34.668938 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f767e12a-346b-41ba-bd63-e665f0fa7bb4 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:34.669653 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f767e12a-346b-41ba-bd63-e665f0fa7bb4 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:34.688437 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-17188fa7-1d22-4838-9755-d4c9e74fb9a9 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '52281b3989a04ca2a50b0bc93bf783f0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '30d028fa9b854c8fb71c8cbed2985c27', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:11:34.709040 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-17188fa7-1d22-4838-9755-d4c9e74fb9a9 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] 10.222.0.16 "GET /compute/v2.1/servers/b07f0d53-2cd4-440f-be47-077703aa66e6" status: 200 len: 1366 microversion: 2.1 time: 0.627829 Jul 16 21:11:34.714345 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 463/951] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:11:34 2022] GET /compute/v2.1/servers/b07f0d53-2cd4-440f-be47-077703aa66e6 => generated 1366 bytes in 634 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:11:34.726082 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-47c485fa-752a-475c-b807-9534a03d069a tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:11:34.726715 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-47c485fa-752a-475c-b807-9534a03d069a tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] [instance: 0fb5b5b6-1796-4406-b276-326e1d7dd451] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:11:34.736902 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-47c485fa-752a-475c-b807-9534a03d069a tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:34.753572 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-47c485fa-752a-475c-b807-9534a03d069a tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.017s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:34.754587 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-47c485fa-752a-475c-b807-9534a03d069a tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:34.767973 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-47c485fa-752a-475c-b807-9534a03d069a tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.013s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:34.892156 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-f767e12a-346b-41ba-bd63-e665f0fa7bb4 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] 10.222.0.16 "GET /compute/v2.1/servers/6a391156-ffce-49c0-a79d-47b4544264f5/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.255363 Jul 16 21:11:34.893710 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 489/952] 10.222.0.16 () {58 vars in 1274 bytes} [Sat Jul 16 21:11:34 2022] GET /compute/v2.1/servers/6a391156-ffce-49c0-a79d-47b4544264f5/os-interface => generated 284 bytes in 258 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:11:34.908071 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-59d9ed9a-94e5-4372-8412-46871394b31e tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:11:34.910437 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-59d9ed9a-94e5-4372-8412-46871394b31e tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:11:34.926204 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-59d9ed9a-94e5-4372-8412-46871394b31e tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:34.928431 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-59d9ed9a-94e5-4372-8412-46871394b31e tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:34.931291 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-59d9ed9a-94e5-4372-8412-46871394b31e tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:34.931989 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-59d9ed9a-94e5-4372-8412-46871394b31e tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:35.120797 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-47c485fa-752a-475c-b807-9534a03d069a tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:11:35.134901 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-47c485fa-752a-475c-b807-9534a03d069a tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:11:35.136739 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-47c485fa-752a-475c-b807-9534a03d069a tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:11:35.137830 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-47c485fa-752a-475c-b807-9534a03d069a tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] 10.222.0.16 "GET /compute/v2.1/servers/0fb5b5b6-1796-4406-b276-326e1d7dd451" status: 200 len: 1258 microversion: 2.25 time: 0.422179 Jul 16 21:11:35.155569 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 464/953] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:11:34 2022] GET /compute/v2.1/servers/0fb5b5b6-1796-4406-b276-326e1d7dd451 => generated 1258 bytes in 440 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:11:35.544995 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-59d9ed9a-94e5-4372-8412-46871394b31e tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b68dacc4b8463ba07fae6372131117', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55a19b7a4682457a9132f5fc80110712', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:11:35.555986 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-59d9ed9a-94e5-4372-8412-46871394b31e tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "GET /compute/v2.1/servers/a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2" status: 200 len: 1537 microversion: 2.1 time: 0.653261 Jul 16 21:11:35.560245 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 490/954] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:11:34 2022] GET /compute/v2.1/servers/a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2 => generated 1537 bytes in 658 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:11:35.659406 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-2b022105-1c96-4009-b89c-823c130e3b21 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:11:35.660210 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-2b022105-1c96-4009-b89c-823c130e3b21 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] [instance: 65a70cca-b978-41ab-b890-6052f653c5c3] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:11:35.683551 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2b022105-1c96-4009-b89c-823c130e3b21 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:35.692178 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2b022105-1c96-4009-b89c-823c130e3b21 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:35.693285 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2b022105-1c96-4009-b89c-823c130e3b21 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:35.698367 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2b022105-1c96-4009-b89c-823c130e3b21 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:35.727426 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-9a0adc2a-e1f5-4147-87d9-c43299a1c914 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:11:35.728073 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-9a0adc2a-e1f5-4147-87d9-c43299a1c914 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] [instance: b07f0d53-2cd4-440f-be47-077703aa66e6] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:11:35.738072 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9a0adc2a-e1f5-4147-87d9-c43299a1c914 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:35.748134 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9a0adc2a-e1f5-4147-87d9-c43299a1c914 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:35.749289 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9a0adc2a-e1f5-4147-87d9-c43299a1c914 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:35.749903 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9a0adc2a-e1f5-4147-87d9-c43299a1c914 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:36.879447 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-9a0adc2a-e1f5-4147-87d9-c43299a1c914 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '52281b3989a04ca2a50b0bc93bf783f0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '30d028fa9b854c8fb71c8cbed2985c27', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:11:36.890054 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-9a0adc2a-e1f5-4147-87d9-c43299a1c914 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] 10.222.0.16 "GET /compute/v2.1/servers/b07f0d53-2cd4-440f-be47-077703aa66e6" status: 200 len: 1366 microversion: 2.1 time: 1.166642 Jul 16 21:11:36.891211 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 491/955] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:11:35 2022] GET /compute/v2.1/servers/b07f0d53-2cd4-440f-be47-077703aa66e6 => generated 1366 bytes in 1169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:11:36.896972 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-6b105680-9bc4-490c-a829-16958eb0b54b tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:11:36.898003 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-6b105680-9bc4-490c-a829-16958eb0b54b tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] [instance: 0fb5b5b6-1796-4406-b276-326e1d7dd451] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:11:36.916028 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6b105680-9bc4-490c-a829-16958eb0b54b tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:36.917085 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6b105680-9bc4-490c-a829-16958eb0b54b tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:36.919181 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6b105680-9bc4-490c-a829-16958eb0b54b tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:36.920830 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6b105680-9bc4-490c-a829-16958eb0b54b tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:37.074813 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-2b022105-1c96-4009-b89c-823c130e3b21 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'abeb7020019d41a18de42956d706a011', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '649babe54d104436af461f1ee4767001', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:11:37.085874 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-2b022105-1c96-4009-b89c-823c130e3b21 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] 10.222.0.16 "GET /compute/v2.1/servers/65a70cca-b978-41ab-b890-6052f653c5c3" status: 200 len: 1317 microversion: 2.1 time: 1.437764 Jul 16 21:11:37.087313 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 465/956] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:11:35 2022] GET /compute/v2.1/servers/65a70cca-b978-41ab-b890-6052f653c5c3 => generated 1317 bytes in 1440 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:11:37.096548 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-bea5bd36-7ed4-4b53-8785-956ee600843b req-d979e1b4-bfe2-4537-903a-8b22a66d7b10 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "b07f0d53-2cd4-440f-be47-077703aa66e6", "tag": "9b0f580d-ea65-4f1a-9836-c0f8d10895e9"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:11:37.124245 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-bea5bd36-7ed4-4b53-8785-956ee600843b req-d979e1b4-bfe2-4537-903a-8b22a66d7b10 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:37.125127 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-bea5bd36-7ed4-4b53-8785-956ee600843b req-d979e1b4-bfe2-4537-903a-8b22a66d7b10 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:37.139919 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-bea5bd36-7ed4-4b53-8785-956ee600843b req-d979e1b4-bfe2-4537-903a-8b22a66d7b10 service nova] Creating event network-changed:9b0f580d-ea65-4f1a-9836-c0f8d10895e9 for instance b07f0d53-2cd4-440f-be47-077703aa66e6 on nch1-master-1 Jul 16 21:11:37.145771 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-bea5bd36-7ed4-4b53-8785-956ee600843b req-d979e1b4-bfe2-4537-903a-8b22a66d7b10 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.056483 Jul 16 21:11:37.146785 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 466/957] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:11:37 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:11:37.155937 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-149b83d4-f070-44ae-a487-71a41d1b8c65 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:11:37.158852 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-149b83d4-f070-44ae-a487-71a41d1b8c65 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:11:37.165685 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-149b83d4-f070-44ae-a487-71a41d1b8c65 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:37.166499 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-149b83d4-f070-44ae-a487-71a41d1b8c65 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:37.168085 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-149b83d4-f070-44ae-a487-71a41d1b8c65 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:37.168992 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-149b83d4-f070-44ae-a487-71a41d1b8c65 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:37.285316 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-6b105680-9bc4-490c-a829-16958eb0b54b tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:11:37.295875 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-6b105680-9bc4-490c-a829-16958eb0b54b tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:11:37.298089 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-6b105680-9bc4-490c-a829-16958eb0b54b tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:11:37.299461 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-6b105680-9bc4-490c-a829-16958eb0b54b tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] 10.222.0.16 "GET /compute/v2.1/servers/0fb5b5b6-1796-4406-b276-326e1d7dd451" status: 200 len: 1320 microversion: 2.25 time: 0.406630 Jul 16 21:11:37.300554 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 492/958] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:11:36 2022] GET /compute/v2.1/servers/0fb5b5b6-1796-4406-b276-326e1d7dd451 => generated 1320 bytes in 409 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:11:37.741967 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-149b83d4-f070-44ae-a487-71a41d1b8c65 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b68dacc4b8463ba07fae6372131117', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55a19b7a4682457a9132f5fc80110712', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:11:37.757984 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-149b83d4-f070-44ae-a487-71a41d1b8c65 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "GET /compute/v2.1/servers/a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2" status: 200 len: 1537 microversion: 2.1 time: 0.608488 Jul 16 21:11:37.759346 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 467/959] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:11:37 2022] GET /compute/v2.1/servers/a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2 => generated 1537 bytes in 612 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:11:37.907955 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-0cd43fac-c393-4198-9008-725668523168 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:11:37.908740 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-0cd43fac-c393-4198-9008-725668523168 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] [instance: b07f0d53-2cd4-440f-be47-077703aa66e6] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:11:37.924786 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0cd43fac-c393-4198-9008-725668523168 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:37.928200 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0cd43fac-c393-4198-9008-725668523168 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:37.932447 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0cd43fac-c393-4198-9008-725668523168 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:37.934418 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0cd43fac-c393-4198-9008-725668523168 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:38.106569 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-29bbfb0f-1f8a-437f-9d1b-eab5589fe129 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:11:38.108618 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-29bbfb0f-1f8a-437f-9d1b-eab5589fe129 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] [instance: 65a70cca-b978-41ab-b890-6052f653c5c3] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:11:38.132278 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-29bbfb0f-1f8a-437f-9d1b-eab5589fe129 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:38.133630 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-29bbfb0f-1f8a-437f-9d1b-eab5589fe129 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:38.136034 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-29bbfb0f-1f8a-437f-9d1b-eab5589fe129 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:38.143867 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-29bbfb0f-1f8a-437f-9d1b-eab5589fe129 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:38.822921 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-0cd43fac-c393-4198-9008-725668523168 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '52281b3989a04ca2a50b0bc93bf783f0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '30d028fa9b854c8fb71c8cbed2985c27', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:11:38.831842 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-0cd43fac-c393-4198-9008-725668523168 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] 10.222.0.16 "GET /compute/v2.1/servers/b07f0d53-2cd4-440f-be47-077703aa66e6" status: 200 len: 1366 microversion: 2.1 time: 0.928284 Jul 16 21:11:38.841687 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 493/960] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:11:37 2022] GET /compute/v2.1/servers/b07f0d53-2cd4-440f-be47-077703aa66e6 => generated 1366 bytes in 939 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:11:38.847284 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-ea1703fd-dbc1-47a3-979e-b42012148cb1 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:11:38.848233 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-ea1703fd-dbc1-47a3-979e-b42012148cb1 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] [instance: 0fb5b5b6-1796-4406-b276-326e1d7dd451] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:11:38.854913 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ea1703fd-dbc1-47a3-979e-b42012148cb1 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:38.855791 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ea1703fd-dbc1-47a3-979e-b42012148cb1 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:38.856861 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ea1703fd-dbc1-47a3-979e-b42012148cb1 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:38.857588 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ea1703fd-dbc1-47a3-979e-b42012148cb1 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:39.207251 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-ea1703fd-dbc1-47a3-979e-b42012148cb1 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:11:39.207251 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-ea1703fd-dbc1-47a3-979e-b42012148cb1 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:11:39.209222 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-ea1703fd-dbc1-47a3-979e-b42012148cb1 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:11:39.209222 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-ea1703fd-dbc1-47a3-979e-b42012148cb1 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] 10.222.0.16 "GET /compute/v2.1/servers/0fb5b5b6-1796-4406-b276-326e1d7dd451" status: 200 len: 1320 microversion: 2.25 time: 0.128964 Jul 16 21:11:39.209222 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 494/961] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:11:38 2022] GET /compute/v2.1/servers/0fb5b5b6-1796-4406-b276-326e1d7dd451 => generated 1320 bytes in 131 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:11:39.209950 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-29bbfb0f-1f8a-437f-9d1b-eab5589fe129 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'abeb7020019d41a18de42956d706a011', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '649babe54d104436af461f1ee4767001', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:11:39.209950 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-29bbfb0f-1f8a-437f-9d1b-eab5589fe129 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] 10.222.0.16 "GET /compute/v2.1/servers/65a70cca-b978-41ab-b890-6052f653c5c3" status: 200 len: 1466 microversion: 2.1 time: 0.924379 Jul 16 21:11:39.209950 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 468/962] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:11:38 2022] GET /compute/v2.1/servers/65a70cca-b978-41ab-b890-6052f653c5c3 => generated 1466 bytes in 927 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:11:39.209950 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-fe62eac7-c47d-406b-9559-0e26a0e0f3e2 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:11:39.211753 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-fe62eac7-c47d-406b-9559-0e26a0e0f3e2 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:11:39.211753 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fe62eac7-c47d-406b-9559-0e26a0e0f3e2 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:39.211753 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fe62eac7-c47d-406b-9559-0e26a0e0f3e2 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:39.211753 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fe62eac7-c47d-406b-9559-0e26a0e0f3e2 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:39.212436 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fe62eac7-c47d-406b-9559-0e26a0e0f3e2 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:39.212626 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-73dd7b55-afc6-48f5-82a0-67fafd996bd9 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-FloatingIPsAssociationTestJSON-server-144754939", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452", "networks": [{"uuid": "9b8d8e4f-14e5-4e94-bfa5-2387fb1adcf8"}]}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:11:39.249965 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.network.neutron [None req-73dd7b55-afc6-48f5-82a0-67fafd996bd9 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] validate_networks() for [('9b8d8e4f-14e5-4e94-bfa5-2387fb1adcf8', None, None, None, None, None)] {{(pid=138680) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 16 21:11:39.635139 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-fe62eac7-c47d-406b-9559-0e26a0e0f3e2 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b68dacc4b8463ba07fae6372131117', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55a19b7a4682457a9132f5fc80110712', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:11:39.682456 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-fe62eac7-c47d-406b-9559-0e26a0e0f3e2 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "GET /compute/v2.1/servers/a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2" status: 200 len: 1542 microversion: 2.1 time: 0.652568 Jul 16 21:11:39.687636 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 469/963] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:11:39 2022] GET /compute/v2.1/servers/a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2 => generated 1542 bytes in 659 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:11:39.710996 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-77b44d2c-fa77-4e0f-930d-536c7cbfbcee tempest-ServersOnMultiNodesTest-83775055 tempest-ServersOnMultiNodesTest-83775055-project-admin] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:11:39.713887 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-77b44d2c-fa77-4e0f-930d-536c7cbfbcee tempest-ServersOnMultiNodesTest-83775055 tempest-ServersOnMultiNodesTest-83775055-project-admin] [instance: 65a70cca-b978-41ab-b890-6052f653c5c3] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:11:39.737573 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-77b44d2c-fa77-4e0f-930d-536c7cbfbcee tempest-ServersOnMultiNodesTest-83775055 tempest-ServersOnMultiNodesTest-83775055-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:39.740730 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-77b44d2c-fa77-4e0f-930d-536c7cbfbcee tempest-ServersOnMultiNodesTest-83775055 tempest-ServersOnMultiNodesTest-83775055-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:39.742045 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-77b44d2c-fa77-4e0f-930d-536c7cbfbcee tempest-ServersOnMultiNodesTest-83775055 tempest-ServersOnMultiNodesTest-83775055-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:39.744605 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-77b44d2c-fa77-4e0f-930d-536c7cbfbcee tempest-ServersOnMultiNodesTest-83775055 tempest-ServersOnMultiNodesTest-83775055-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:40.288291 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-73dd7b55-afc6-48f5-82a0-67fafd996bd9 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Flavor limits 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 16 21:11:40.288291 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-73dd7b55-afc6-48f5-82a0-67fafd996bd9 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Image limits 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 16 21:11:40.289208 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-73dd7b55-afc6-48f5-82a0-67fafd996bd9 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Flavor pref 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 16 21:11:40.289208 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-73dd7b55-afc6-48f5-82a0-67fafd996bd9 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Image pref 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 16 21:11:40.289582 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-73dd7b55-afc6-48f5-82a0-67fafd996bd9 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 16 21:11:40.312866 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-73dd7b55-afc6-48f5-82a0-67fafd996bd9 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:40.312866 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-73dd7b55-afc6-48f5-82a0-67fafd996bd9 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:40.327392 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-73dd7b55-afc6-48f5-82a0-67fafd996bd9 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:40.334749 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-77b44d2c-fa77-4e0f-930d-536c7cbfbcee tempest-ServersOnMultiNodesTest-83775055 tempest-ServersOnMultiNodesTest-83775055-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/65a70cca-b978-41ab-b890-6052f653c5c3" status: 200 len: 1617 microversion: 2.1 time: 0.645242 Jul 16 21:11:40.335314 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-73dd7b55-afc6-48f5-82a0-67fafd996bd9 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:40.336620 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 470/964] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:11:39 2022] GET /compute/v2.1/servers/65a70cca-b978-41ab-b890-6052f653c5c3 => generated 1617 bytes in 648 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:11:40.373762 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_api:SELECT=94,nova_api:DELETE=6 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:11:40.391083 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-73dd7b55-afc6-48f5-82a0-67fafd996bd9 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Getting quotas for project cda86413e74b4c1e8e09bd4df4926664. Resources: {'ram', 'instances', 'cores'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:11:40.404618 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-73dd7b55-afc6-48f5-82a0-67fafd996bd9 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Getting quotas for user 2dfcad8d2b57481bb33aa8f5d7483c51 and project cda86413e74b4c1e8e09bd4df4926664. Resources: {'ram', 'instances', 'cores'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:11:40.441928 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-73dd7b55-afc6-48f5-82a0-67fafd996bd9 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Going to run 1 instances... {{(pid=138680) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 16 21:11:40.444412 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-73dd7b55-afc6-48f5-82a0-67fafd996bd9 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] [instance: 8020f553-6025-4f19-835c-c00e2fb4f764] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138680) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 16 21:11:40.466855 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-3561c751-09b3-47f5-9e18-91c635ad4055 tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] Action: 'create', calling method: .version_select at 0x7fdd9d233280>, body: {"keypair": {"name": "tempest-keypair-280247600", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBEhc1PmmaTsAhcN/6vw0mOHVUcEzhCvW8WyRn6J5QTClQneP+57w9po/KwfoqqDvF4/5kZw44KwQzadWRpEwZeJTjq8os9Dn0iRC2dv5kUQfuQZuCS9YgmqZoP0x9kSyAw=="}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:11:40.508277 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-3561c751-09b3-47f5-9e18-91c635ad4055 tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] Getting quotas for project fb4ff5d7e42940cdb7be71dadd895ee4. Resources: {'key_pairs'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:11:40.517948 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-3561c751-09b3-47f5-9e18-91c635ad4055 tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] Getting quotas for user 034c11cc3f174c6fb44c8d64e55cf8b9 and project fb4ff5d7e42940cdb7be71dadd895ee4. Resources: {'key_pairs'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:11:40.618836 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-73dd7b55-afc6-48f5-82a0-67fafd996bd9 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 1.643663 Jul 16 21:11:40.621867 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 495/965] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 21:11:38 2022] POST /compute/v2.1/servers => generated 380 bytes in 1647 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 16 21:11:40.629977 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-fe9fcf3a-53d9-428a-b658-f05d719163be req-aa776dbc-bee7-4fda-a05a-6588e7d4295d service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "6a391156-ffce-49c0-a79d-47b4544264f5", "tag": "a8de1bba-f28a-49ad-92e5-8a10d67f1e75"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:11:40.644560 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-3561c751-09b3-47f5-9e18-91c635ad4055 tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] 10.222.0.16 "POST /compute/v2.1/os-keypairs" status: 200 len: 385 microversion: 2.1 time: 0.302531 Jul 16 21:11:40.645710 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 471/966] 10.222.0.16 () {60 vars in 1158 bytes} [Sat Jul 16 21:11:40 2022] POST /compute/v2.1/os-keypairs => generated 385 bytes in 308 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:11:40.656885 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-caa4f872-163b-4252-9587-4263a31c8226 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:11:40.658358 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-caa4f872-163b-4252-9587-4263a31c8226 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] [instance: b07f0d53-2cd4-440f-be47-077703aa66e6] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:11:40.668323 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-fe9fcf3a-53d9-428a-b658-f05d719163be req-aa776dbc-bee7-4fda-a05a-6588e7d4295d service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:40.672185 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-fe9fcf3a-53d9-428a-b658-f05d719163be req-aa776dbc-bee7-4fda-a05a-6588e7d4295d service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:40.672328 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-caa4f872-163b-4252-9587-4263a31c8226 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:40.672328 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-caa4f872-163b-4252-9587-4263a31c8226 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:40.672328 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-caa4f872-163b-4252-9587-4263a31c8226 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:40.672328 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-caa4f872-163b-4252-9587-4263a31c8226 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:40.691470 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-fe9fcf3a-53d9-428a-b658-f05d719163be req-aa776dbc-bee7-4fda-a05a-6588e7d4295d service nova] Creating event network-changed:a8de1bba-f28a-49ad-92e5-8a10d67f1e75 for instance 6a391156-ffce-49c0-a79d-47b4544264f5 on nch2-master-1 Jul 16 21:11:40.705609 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-fe9fcf3a-53d9-428a-b658-f05d719163be req-aa776dbc-bee7-4fda-a05a-6588e7d4295d service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.081261 Jul 16 21:11:40.707549 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 496/967] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:11:40 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:11:40.719875 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-d17476cc-f5b4-4a8f-8c85-868b89444f34 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:11:40.721750 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-d17476cc-f5b4-4a8f-8c85-868b89444f34 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] [instance: 0fb5b5b6-1796-4406-b276-326e1d7dd451] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:11:40.740418 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d17476cc-f5b4-4a8f-8c85-868b89444f34 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:40.741210 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d17476cc-f5b4-4a8f-8c85-868b89444f34 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:40.742663 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d17476cc-f5b4-4a8f-8c85-868b89444f34 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:40.745378 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d17476cc-f5b4-4a8f-8c85-868b89444f34 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:41.332357 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-caa4f872-163b-4252-9587-4263a31c8226 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '52281b3989a04ca2a50b0bc93bf783f0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '30d028fa9b854c8fb71c8cbed2985c27', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:11:41.341809 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-caa4f872-163b-4252-9587-4263a31c8226 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] 10.222.0.16 "GET /compute/v2.1/servers/b07f0d53-2cd4-440f-be47-077703aa66e6" status: 200 len: 1366 microversion: 2.1 time: 0.694071 Jul 16 21:11:41.347788 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 472/968] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:11:40 2022] GET /compute/v2.1/servers/b07f0d53-2cd4-440f-be47-077703aa66e6 => generated 1366 bytes in 701 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:11:41.362596 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-08ab83e4-966e-455f-a99d-3113be19c00c tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:11:41.365884 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-08ab83e4-966e-455f-a99d-3113be19c00c tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] [instance: 65a70cca-b978-41ab-b890-6052f653c5c3] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:11:41.396700 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-08ab83e4-966e-455f-a99d-3113be19c00c tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:41.398215 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-08ab83e4-966e-455f-a99d-3113be19c00c tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:41.405191 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-08ab83e4-966e-455f-a99d-3113be19c00c tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:41.405860 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-08ab83e4-966e-455f-a99d-3113be19c00c tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:41.486133 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-08ab83e4-966e-455f-a99d-3113be19c00c tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] [instance: 65a70cca-b978-41ab-b890-6052f653c5c3] Going to try to terminate instance {{(pid=138681) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 16 21:11:41.514214 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-d17476cc-f5b4-4a8f-8c85-868b89444f34 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:11:41.515508 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=100 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:11:41.528116 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-d17476cc-f5b4-4a8f-8c85-868b89444f34 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:11:41.530172 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-d17476cc-f5b4-4a8f-8c85-868b89444f34 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:11:41.532031 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-d17476cc-f5b4-4a8f-8c85-868b89444f34 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] 10.222.0.16 "GET /compute/v2.1/servers/0fb5b5b6-1796-4406-b276-326e1d7dd451" status: 200 len: 1362 microversion: 2.25 time: 0.823144 Jul 16 21:11:41.535381 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 497/969] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:11:40 2022] GET /compute/v2.1/servers/0fb5b5b6-1796-4406-b276-326e1d7dd451 => generated 1362 bytes in 827 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:11:41.562267 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-b4108c9b-ec5a-4b14-aeb6-54716a772d40 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:11:41.562894 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-b4108c9b-ec5a-4b14-aeb6-54716a772d40 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] [instance: 8020f553-6025-4f19-835c-c00e2fb4f764] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:11:41.728851 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-b4108c9b-ec5a-4b14-aeb6-54716a772d40 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2dfcad8d2b57481bb33aa8f5d7483c51', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cda86413e74b4c1e8e09bd4df4926664', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:11:41.739359 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-b4108c9b-ec5a-4b14-aeb6-54716a772d40 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] 10.222.0.16 "GET /compute/v2.1/servers/8020f553-6025-4f19-835c-c00e2fb4f764" status: 200 len: 1223 microversion: 2.1 time: 0.181445 Jul 16 21:11:41.742588 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 498/970] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:11:41 2022] GET /compute/v2.1/servers/8020f553-6025-4f19-835c-c00e2fb4f764 => generated 1223 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:11:41.755804 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-9e5c20c6-48e1-4d21-81ef-d99a79463d81 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:11:41.759378 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-9e5c20c6-48e1-4d21-81ef-d99a79463d81 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:11:41.791621 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9e5c20c6-48e1-4d21-81ef-d99a79463d81 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:41.792321 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9e5c20c6-48e1-4d21-81ef-d99a79463d81 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:41.799059 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9e5c20c6-48e1-4d21-81ef-d99a79463d81 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:41.799758 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9e5c20c6-48e1-4d21-81ef-d99a79463d81 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:41.893962 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-08ab83e4-966e-455f-a99d-3113be19c00c tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] 10.222.0.16 "DELETE /compute/v2.1/servers/65a70cca-b978-41ab-b890-6052f653c5c3" status: 204 len: 0 microversion: 2.1 time: 0.543249 Jul 16 21:11:41.894920 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 473/971] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:11:41 2022] DELETE /compute/v2.1/servers/65a70cca-b978-41ab-b890-6052f653c5c3 => generated 0 bytes in 545 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 16 21:11:41.908539 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-7bd5d73b-8992-4158-916a-70d7bc11e7d4 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:11:41.909465 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-7bd5d73b-8992-4158-916a-70d7bc11e7d4 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] [instance: 65a70cca-b978-41ab-b890-6052f653c5c3] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:11:41.922541 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7bd5d73b-8992-4158-916a-70d7bc11e7d4 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:41.923091 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7bd5d73b-8992-4158-916a-70d7bc11e7d4 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:41.924031 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7bd5d73b-8992-4158-916a-70d7bc11e7d4 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:41.929506 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7bd5d73b-8992-4158-916a-70d7bc11e7d4 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:42.083678 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell0:SELECT=4 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:11:42.606759 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-9e5c20c6-48e1-4d21-81ef-d99a79463d81 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b68dacc4b8463ba07fae6372131117', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55a19b7a4682457a9132f5fc80110712', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:11:42.628031 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-9e5c20c6-48e1-4d21-81ef-d99a79463d81 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "GET /compute/v2.1/servers/a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2" status: 200 len: 1542 microversion: 2.1 time: 0.882376 Jul 16 21:11:42.629909 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 499/972] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:11:41 2022] GET /compute/v2.1/servers/a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2 => generated 1542 bytes in 887 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:11:42.661843 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-09d62487-6282-4246-ad46-43fb4a36f394 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:11:42.668740 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-09d62487-6282-4246-ad46-43fb4a36f394 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] [instance: b07f0d53-2cd4-440f-be47-077703aa66e6] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:11:42.679724 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-09d62487-6282-4246-ad46-43fb4a36f394 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:42.679724 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-09d62487-6282-4246-ad46-43fb4a36f394 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:42.679724 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-09d62487-6282-4246-ad46-43fb4a36f394 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:42.680619 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-09d62487-6282-4246-ad46-43fb4a36f394 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:42.916177 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-7bd5d73b-8992-4158-916a-70d7bc11e7d4 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'abeb7020019d41a18de42956d706a011', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '649babe54d104436af461f1ee4767001', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:11:42.934858 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-7bd5d73b-8992-4158-916a-70d7bc11e7d4 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] 10.222.0.16 "GET /compute/v2.1/servers/65a70cca-b978-41ab-b890-6052f653c5c3" status: 200 len: 1472 microversion: 2.1 time: 1.032657 Jul 16 21:11:42.945809 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 474/973] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:11:41 2022] GET /compute/v2.1/servers/65a70cca-b978-41ab-b890-6052f653c5c3 => generated 1472 bytes in 1035 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:11:42.948920 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-852cebb7-57b9-4ae8-8feb-949acb3eb1d9 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:11:42.949813 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-852cebb7-57b9-4ae8-8feb-949acb3eb1d9 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] [instance: 0fb5b5b6-1796-4406-b276-326e1d7dd451] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:11:42.974959 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-852cebb7-57b9-4ae8-8feb-949acb3eb1d9 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:42.978224 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-852cebb7-57b9-4ae8-8feb-949acb3eb1d9 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:42.980900 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-852cebb7-57b9-4ae8-8feb-949acb3eb1d9 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:42.983859 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-852cebb7-57b9-4ae8-8feb-949acb3eb1d9 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:43.672234 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-09d62487-6282-4246-ad46-43fb4a36f394 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '52281b3989a04ca2a50b0bc93bf783f0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '30d028fa9b854c8fb71c8cbed2985c27', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:11:43.681349 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-09d62487-6282-4246-ad46-43fb4a36f394 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] 10.222.0.16 "GET /compute/v2.1/servers/b07f0d53-2cd4-440f-be47-077703aa66e6" status: 200 len: 1366 microversion: 2.1 time: 1.047074 Jul 16 21:11:43.683959 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 500/974] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:11:42 2022] GET /compute/v2.1/servers/b07f0d53-2cd4-440f-be47-077703aa66e6 => generated 1366 bytes in 1053 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:11:43.700916 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-341804c2-b0f3-4fb9-b9e8-8b5654e77d6e tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:11:43.701657 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-341804c2-b0f3-4fb9-b9e8-8b5654e77d6e tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] [instance: 8020f553-6025-4f19-835c-c00e2fb4f764] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:11:43.713376 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-341804c2-b0f3-4fb9-b9e8-8b5654e77d6e tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:43.713833 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-341804c2-b0f3-4fb9-b9e8-8b5654e77d6e tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:43.715509 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-341804c2-b0f3-4fb9-b9e8-8b5654e77d6e tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:43.716051 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-341804c2-b0f3-4fb9-b9e8-8b5654e77d6e tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:44.000693 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-341804c2-b0f3-4fb9-b9e8-8b5654e77d6e tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2dfcad8d2b57481bb33aa8f5d7483c51', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cda86413e74b4c1e8e09bd4df4926664', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:11:44.009347 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-341804c2-b0f3-4fb9-b9e8-8b5654e77d6e tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] 10.222.0.16 "GET /compute/v2.1/servers/8020f553-6025-4f19-835c-c00e2fb4f764" status: 200 len: 1275 microversion: 2.1 time: 0.313196 Jul 16 21:11:44.010151 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 501/975] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:11:43 2022] GET /compute/v2.1/servers/8020f553-6025-4f19-835c-c00e2fb4f764 => generated 1275 bytes in 326 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:11:44.031725 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-7ec77672-70b3-4085-84b9-2049cb6ac8c6 req-bf2bee67-c82f-445e-a61f-b62bbf0e9ea2 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "0fb5b5b6-1796-4406-b276-326e1d7dd451", "tag": "19f08e09-ff8e-43eb-b5f2-421d0eadc41c"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:11:44.043912 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-7ec77672-70b3-4085-84b9-2049cb6ac8c6 req-bf2bee67-c82f-445e-a61f-b62bbf0e9ea2 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:44.044878 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-7ec77672-70b3-4085-84b9-2049cb6ac8c6 req-bf2bee67-c82f-445e-a61f-b62bbf0e9ea2 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:44.061663 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-7ec77672-70b3-4085-84b9-2049cb6ac8c6 req-bf2bee67-c82f-445e-a61f-b62bbf0e9ea2 service nova] Creating event network-changed:19f08e09-ff8e-43eb-b5f2-421d0eadc41c for instance 0fb5b5b6-1796-4406-b276-326e1d7dd451 on nch2-master-1 Jul 16 21:11:44.069658 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-7ec77672-70b3-4085-84b9-2049cb6ac8c6 req-bf2bee67-c82f-445e-a61f-b62bbf0e9ea2 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.045049 Jul 16 21:11:44.070433 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 502/976] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:11:44 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:11:44.077557 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-f56510f2-8da6-4d2e-ac61-c0934bfb3e10 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:11:44.078538 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-f56510f2-8da6-4d2e-ac61-c0934bfb3e10 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:11:44.086618 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f56510f2-8da6-4d2e-ac61-c0934bfb3e10 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:44.089994 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f56510f2-8da6-4d2e-ac61-c0934bfb3e10 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:44.089994 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f56510f2-8da6-4d2e-ac61-c0934bfb3e10 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:44.089994 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f56510f2-8da6-4d2e-ac61-c0934bfb3e10 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:44.102647 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-852cebb7-57b9-4ae8-8feb-949acb3eb1d9 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:11:44.123117 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-852cebb7-57b9-4ae8-8feb-949acb3eb1d9 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:11:44.131250 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-852cebb7-57b9-4ae8-8feb-949acb3eb1d9 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:11:44.132770 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-852cebb7-57b9-4ae8-8feb-949acb3eb1d9 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] 10.222.0.16 "GET /compute/v2.1/servers/0fb5b5b6-1796-4406-b276-326e1d7dd451" status: 200 len: 1362 microversion: 2.25 time: 1.195381 Jul 16 21:11:44.135788 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 475/977] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:11:42 2022] GET /compute/v2.1/servers/0fb5b5b6-1796-4406-b276-326e1d7dd451 => generated 1362 bytes in 1199 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:11:44.158818 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-4b46d238-bbec-4fea-9000-e2ec790546cc tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:11:44.159945 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-4b46d238-bbec-4fea-9000-e2ec790546cc tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] [instance: 65a70cca-b978-41ab-b890-6052f653c5c3] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:11:44.180439 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4b46d238-bbec-4fea-9000-e2ec790546cc tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:44.181155 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4b46d238-bbec-4fea-9000-e2ec790546cc tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:44.187573 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4b46d238-bbec-4fea-9000-e2ec790546cc tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:44.187573 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4b46d238-bbec-4fea-9000-e2ec790546cc tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:45.278340 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-4b46d238-bbec-4fea-9000-e2ec790546cc tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'abeb7020019d41a18de42956d706a011', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '649babe54d104436af461f1ee4767001', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:11:45.286583 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-f56510f2-8da6-4d2e-ac61-c0934bfb3e10 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b68dacc4b8463ba07fae6372131117', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55a19b7a4682457a9132f5fc80110712', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:11:45.289069 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-4b46d238-bbec-4fea-9000-e2ec790546cc tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] 10.222.0.16 "GET /compute/v2.1/servers/65a70cca-b978-41ab-b890-6052f653c5c3" status: 200 len: 1472 microversion: 2.1 time: 1.151269 Jul 16 21:11:45.290026 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 476/978] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:11:44 2022] GET /compute/v2.1/servers/65a70cca-b978-41ab-b890-6052f653c5c3 => generated 1472 bytes in 1153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:11:45.304888 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-f56510f2-8da6-4d2e-ac61-c0934bfb3e10 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "GET /compute/v2.1/servers/a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2" status: 200 len: 1542 microversion: 2.1 time: 1.232778 Jul 16 21:11:45.306533 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-fcab8e0d-c707-43de-adb8-5c30ec56ee61 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:11:45.306662 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 503/979] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:11:44 2022] GET /compute/v2.1/servers/a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2 => generated 1542 bytes in 1235 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:11:45.316707 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-fcab8e0d-c707-43de-adb8-5c30ec56ee61 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] [instance: b07f0d53-2cd4-440f-be47-077703aa66e6] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:11:45.320200 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-2bf546a5-f0fc-4295-8561-db3061f34914 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:11:45.321176 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-2bf546a5-f0fc-4295-8561-db3061f34914 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] [instance: 8020f553-6025-4f19-835c-c00e2fb4f764] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:11:45.333320 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2bf546a5-f0fc-4295-8561-db3061f34914 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:45.334071 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2bf546a5-f0fc-4295-8561-db3061f34914 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:45.335414 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2bf546a5-f0fc-4295-8561-db3061f34914 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:45.336070 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2bf546a5-f0fc-4295-8561-db3061f34914 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:45.341889 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fcab8e0d-c707-43de-adb8-5c30ec56ee61 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:45.342906 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fcab8e0d-c707-43de-adb8-5c30ec56ee61 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:45.343986 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fcab8e0d-c707-43de-adb8-5c30ec56ee61 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:45.344593 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fcab8e0d-c707-43de-adb8-5c30ec56ee61 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:45.560322 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-2bf546a5-f0fc-4295-8561-db3061f34914 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2dfcad8d2b57481bb33aa8f5d7483c51', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cda86413e74b4c1e8e09bd4df4926664', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:11:45.569173 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-2bf546a5-f0fc-4295-8561-db3061f34914 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] 10.222.0.16 "GET /compute/v2.1/servers/8020f553-6025-4f19-835c-c00e2fb4f764" status: 200 len: 1281 microversion: 2.1 time: 0.262322 Jul 16 21:11:45.570571 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 504/980] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:11:45 2022] GET /compute/v2.1/servers/8020f553-6025-4f19-835c-c00e2fb4f764 => generated 1281 bytes in 264 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:11:45.576154 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-138537db-a135-4eb5-8f49-11df294a36d2 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:11:45.577123 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-138537db-a135-4eb5-8f49-11df294a36d2 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] [instance: 0fb5b5b6-1796-4406-b276-326e1d7dd451] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:11:45.583989 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-138537db-a135-4eb5-8f49-11df294a36d2 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:45.584844 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-138537db-a135-4eb5-8f49-11df294a36d2 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:45.585926 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-138537db-a135-4eb5-8f49-11df294a36d2 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:45.586787 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-138537db-a135-4eb5-8f49-11df294a36d2 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:45.969800 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-138537db-a135-4eb5-8f49-11df294a36d2 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:11:45.986426 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-138537db-a135-4eb5-8f49-11df294a36d2 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:11:45.988616 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-138537db-a135-4eb5-8f49-11df294a36d2 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:11:45.999893 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-138537db-a135-4eb5-8f49-11df294a36d2 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] 10.222.0.16 "GET /compute/v2.1/servers/0fb5b5b6-1796-4406-b276-326e1d7dd451" status: 200 len: 1362 microversion: 2.25 time: 0.427513 Jul 16 21:11:46.013769 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 505/981] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:11:45 2022] GET /compute/v2.1/servers/0fb5b5b6-1796-4406-b276-326e1d7dd451 => generated 1362 bytes in 442 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:11:46.020094 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-fb2455eb-7551-4855-b755-4ecb2963289a req-38ce2874-8bfc-4e79-a55f-67aff928554c service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "b07f0d53-2cd4-440f-be47-077703aa66e6", "name": "network-vif-plugged", "status": "completed", "tag": "9b0f580d-ea65-4f1a-9836-c0f8d10895e9"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:11:46.053340 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-fb2455eb-7551-4855-b755-4ecb2963289a req-38ce2874-8bfc-4e79-a55f-67aff928554c service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:46.054352 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-fcab8e0d-c707-43de-adb8-5c30ec56ee61 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '52281b3989a04ca2a50b0bc93bf783f0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '30d028fa9b854c8fb71c8cbed2985c27', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:11:46.054615 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-fb2455eb-7551-4855-b755-4ecb2963289a req-38ce2874-8bfc-4e79-a55f-67aff928554c service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:46.065159 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-fcab8e0d-c707-43de-adb8-5c30ec56ee61 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] 10.222.0.16 "GET /compute/v2.1/servers/b07f0d53-2cd4-440f-be47-077703aa66e6" status: 200 len: 1366 microversion: 2.1 time: 0.773885 Jul 16 21:11:46.075308 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 477/982] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:11:45 2022] GET /compute/v2.1/servers/b07f0d53-2cd4-440f-be47-077703aa66e6 => generated 1366 bytes in 784 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:11:46.084962 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=100 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:11:46.091373 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-fb2455eb-7551-4855-b755-4ecb2963289a req-38ce2874-8bfc-4e79-a55f-67aff928554c service nova] Creating event network-vif-plugged:9b0f580d-ea65-4f1a-9836-c0f8d10895e9 for instance b07f0d53-2cd4-440f-be47-077703aa66e6 on nch1-master-1 Jul 16 21:11:46.099153 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-fb2455eb-7551-4855-b755-4ecb2963289a req-38ce2874-8bfc-4e79-a55f-67aff928554c service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.083444 Jul 16 21:11:46.100142 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 506/983] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:11:46 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:11:46.306945 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-a8177801-7295-40b6-9777-627125308c9c tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:11:46.308433 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-a8177801-7295-40b6-9777-627125308c9c tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] [instance: 65a70cca-b978-41ab-b890-6052f653c5c3] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:11:46.324348 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-b8308140-eb5a-4caf-98d1-069df7c2a0f0 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:11:46.325069 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-b8308140-eb5a-4caf-98d1-069df7c2a0f0 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:11:46.331091 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a8177801-7295-40b6-9777-627125308c9c tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:46.335302 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b8308140-eb5a-4caf-98d1-069df7c2a0f0 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:46.336281 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a8177801-7295-40b6-9777-627125308c9c tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:46.337465 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a8177801-7295-40b6-9777-627125308c9c tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:46.338240 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b8308140-eb5a-4caf-98d1-069df7c2a0f0 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:46.339834 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b8308140-eb5a-4caf-98d1-069df7c2a0f0 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:46.340602 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a8177801-7295-40b6-9777-627125308c9c tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:46.343667 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b8308140-eb5a-4caf-98d1-069df7c2a0f0 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:46.557358 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-a8177801-7295-40b6-9777-627125308c9c tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'abeb7020019d41a18de42956d706a011', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '649babe54d104436af461f1ee4767001', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:11:46.565481 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-a8177801-7295-40b6-9777-627125308c9c tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] 10.222.0.16 "GET /compute/v2.1/servers/65a70cca-b978-41ab-b890-6052f653c5c3" status: 200 len: 1430 microversion: 2.1 time: 0.262372 Jul 16 21:11:46.579311 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 478/984] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:11:46 2022] GET /compute/v2.1/servers/65a70cca-b978-41ab-b890-6052f653c5c3 => generated 1430 bytes in 274 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:11:46.585516 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-65e95104-aa9b-4f61-94ad-1c45d73a7773 req-36a39bcb-bba2-42be-9a38-fdf393dbb889 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "65a70cca-b978-41ab-b890-6052f653c5c3", "name": "network-vif-deleted", "tag": "7782780c-06ee-4b9b-b252-ba55ad0b48d8"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:11:46.599723 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-65e95104-aa9b-4f61-94ad-1c45d73a7773 req-36a39bcb-bba2-42be-9a38-fdf393dbb889 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:46.600559 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-65e95104-aa9b-4f61-94ad-1c45d73a7773 req-36a39bcb-bba2-42be-9a38-fdf393dbb889 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:46.613419 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-65e95104-aa9b-4f61-94ad-1c45d73a7773 req-36a39bcb-bba2-42be-9a38-fdf393dbb889 service nova] Creating event network-vif-deleted:7782780c-06ee-4b9b-b252-ba55ad0b48d8 for instance 65a70cca-b978-41ab-b890-6052f653c5c3 on nch2-master-1 Jul 16 21:11:46.620115 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-65e95104-aa9b-4f61-94ad-1c45d73a7773 req-36a39bcb-bba2-42be-9a38-fdf393dbb889 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.041441 Jul 16 21:11:46.621447 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 479/985] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:11:46 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:11:46.627032 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-ca35037f-52af-4c77-9268-4a0a939cf54b tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:11:46.633639 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-ca35037f-52af-4c77-9268-4a0a939cf54b tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] [instance: 8020f553-6025-4f19-835c-c00e2fb4f764] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:11:46.637626 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ca35037f-52af-4c77-9268-4a0a939cf54b tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:46.638309 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ca35037f-52af-4c77-9268-4a0a939cf54b tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:46.639393 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ca35037f-52af-4c77-9268-4a0a939cf54b tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:46.640047 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ca35037f-52af-4c77-9268-4a0a939cf54b tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:46.738323 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-ca35037f-52af-4c77-9268-4a0a939cf54b tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2dfcad8d2b57481bb33aa8f5d7483c51', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cda86413e74b4c1e8e09bd4df4926664', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:11:46.749941 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-ca35037f-52af-4c77-9268-4a0a939cf54b tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] 10.222.0.16 "GET /compute/v2.1/servers/8020f553-6025-4f19-835c-c00e2fb4f764" status: 200 len: 1281 microversion: 2.1 time: 0.126721 Jul 16 21:11:46.765027 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 480/986] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:11:46 2022] GET /compute/v2.1/servers/8020f553-6025-4f19-835c-c00e2fb4f764 => generated 1281 bytes in 142 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:11:46.986817 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-b8308140-eb5a-4caf-98d1-069df7c2a0f0 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b68dacc4b8463ba07fae6372131117', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55a19b7a4682457a9132f5fc80110712', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:11:46.997192 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-b8308140-eb5a-4caf-98d1-069df7c2a0f0 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "GET /compute/v2.1/servers/a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2" status: 200 len: 1542 microversion: 2.1 time: 0.677324 Jul 16 21:11:46.998241 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 507/987] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:11:46 2022] GET /compute/v2.1/servers/a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2 => generated 1542 bytes in 679 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:11:47.020041 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-ad4aeb74-273f-4e3f-818a-f67c3e31a791 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:11:47.020709 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-ad4aeb74-273f-4e3f-818a-f67c3e31a791 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] [instance: 0fb5b5b6-1796-4406-b276-326e1d7dd451] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:11:47.029702 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ad4aeb74-273f-4e3f-818a-f67c3e31a791 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:47.030151 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ad4aeb74-273f-4e3f-818a-f67c3e31a791 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:47.031164 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ad4aeb74-273f-4e3f-818a-f67c3e31a791 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:47.031609 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ad4aeb74-273f-4e3f-818a-f67c3e31a791 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:47.091256 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-e9ed4329-6b68-42eb-9f59-62a1c3abf25f tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:11:47.091920 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-e9ed4329-6b68-42eb-9f59-62a1c3abf25f tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] [instance: b07f0d53-2cd4-440f-be47-077703aa66e6] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:11:47.098176 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e9ed4329-6b68-42eb-9f59-62a1c3abf25f tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:47.098659 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e9ed4329-6b68-42eb-9f59-62a1c3abf25f tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:47.099646 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e9ed4329-6b68-42eb-9f59-62a1c3abf25f tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:47.100114 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e9ed4329-6b68-42eb-9f59-62a1c3abf25f tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:47.436435 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-ad4aeb74-273f-4e3f-818a-f67c3e31a791 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:11:47.445002 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-ad4aeb74-273f-4e3f-818a-f67c3e31a791 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:11:47.446511 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-ad4aeb74-273f-4e3f-818a-f67c3e31a791 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:11:47.447674 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-ad4aeb74-273f-4e3f-818a-f67c3e31a791 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] 10.222.0.16 "GET /compute/v2.1/servers/0fb5b5b6-1796-4406-b276-326e1d7dd451" status: 200 len: 1362 microversion: 2.25 time: 0.432731 Jul 16 21:11:47.459033 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 481/988] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:11:47 2022] GET /compute/v2.1/servers/0fb5b5b6-1796-4406-b276-326e1d7dd451 => generated 1362 bytes in 434 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:11:47.461283 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-e9ed4329-6b68-42eb-9f59-62a1c3abf25f tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '52281b3989a04ca2a50b0bc93bf783f0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '30d028fa9b854c8fb71c8cbed2985c27', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:11:47.471460 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-e9ed4329-6b68-42eb-9f59-62a1c3abf25f tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] 10.222.0.16 "GET /compute/v2.1/servers/b07f0d53-2cd4-440f-be47-077703aa66e6" status: 200 len: 1366 microversion: 2.1 time: 0.384961 Jul 16 21:11:47.472289 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 508/989] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:11:47 2022] GET /compute/v2.1/servers/b07f0d53-2cd4-440f-be47-077703aa66e6 => generated 1366 bytes in 386 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:11:47.583920 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-2e7aa908-156c-4e59-9a46-1f92eaaa28e3 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:11:47.584723 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-2e7aa908-156c-4e59-9a46-1f92eaaa28e3 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] [instance: 65a70cca-b978-41ab-b890-6052f653c5c3] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:11:47.592155 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2e7aa908-156c-4e59-9a46-1f92eaaa28e3 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:47.592615 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2e7aa908-156c-4e59-9a46-1f92eaaa28e3 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:47.593512 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2e7aa908-156c-4e59-9a46-1f92eaaa28e3 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:47.593962 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2e7aa908-156c-4e59-9a46-1f92eaaa28e3 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:47.630601 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2e7aa908-156c-4e59-9a46-1f92eaaa28e3 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:47.631096 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2e7aa908-156c-4e59-9a46-1f92eaaa28e3 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:47.631562 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.objects.instance [None req-2e7aa908-156c-4e59-9a46-1f92eaaa28e3 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lazy-loading 'fault' on Instance uuid 65a70cca-b978-41ab-b890-6052f653c5c3 {{(pid=138681) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:11:47.770540 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-e91b7fda-6ba4-405a-879e-765d0c7005a5 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:11:47.771189 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-e91b7fda-6ba4-405a-879e-765d0c7005a5 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] [instance: 8020f553-6025-4f19-835c-c00e2fb4f764] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:11:47.779315 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e91b7fda-6ba4-405a-879e-765d0c7005a5 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:47.779769 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e91b7fda-6ba4-405a-879e-765d0c7005a5 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:47.780542 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e91b7fda-6ba4-405a-879e-765d0c7005a5 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:47.781345 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e91b7fda-6ba4-405a-879e-765d0c7005a5 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:47.901811 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-2e7aa908-156c-4e59-9a46-1f92eaaa28e3 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'abeb7020019d41a18de42956d706a011', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '649babe54d104436af461f1ee4767001', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:11:47.914787 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-2e7aa908-156c-4e59-9a46-1f92eaaa28e3 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] 10.222.0.16 "GET /compute/v2.1/servers/65a70cca-b978-41ab-b890-6052f653c5c3" status: 200 len: 1307 microversion: 2.1 time: 0.335066 Jul 16 21:11:47.918929 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 482/990] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:11:47 2022] GET /compute/v2.1/servers/65a70cca-b978-41ab-b890-6052f653c5c3 => generated 1307 bytes in 340 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:11:48.018273 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-d67d94c1-9f92-4b1f-a498-19c057bc33a7 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:11:48.019201 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-d67d94c1-9f92-4b1f-a498-19c057bc33a7 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:11:48.029360 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d67d94c1-9f92-4b1f-a498-19c057bc33a7 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:48.035960 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d67d94c1-9f92-4b1f-a498-19c057bc33a7 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:48.039127 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d67d94c1-9f92-4b1f-a498-19c057bc33a7 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:48.039756 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d67d94c1-9f92-4b1f-a498-19c057bc33a7 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:48.603623 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-e91b7fda-6ba4-405a-879e-765d0c7005a5 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2dfcad8d2b57481bb33aa8f5d7483c51', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cda86413e74b4c1e8e09bd4df4926664', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:11:48.621448 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-e91b7fda-6ba4-405a-879e-765d0c7005a5 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] 10.222.0.16 "GET /compute/v2.1/servers/8020f553-6025-4f19-835c-c00e2fb4f764" status: 200 len: 1323 microversion: 2.1 time: 0.853752 Jul 16 21:11:48.639750 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 509/991] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:11:47 2022] GET /compute/v2.1/servers/8020f553-6025-4f19-835c-c00e2fb4f764 => generated 1323 bytes in 874 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:11:48.647899 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-e96915c4-f449-4bd2-b2e6-43585a2679ad tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:11:48.648518 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-e96915c4-f449-4bd2-b2e6-43585a2679ad tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] [instance: 0fb5b5b6-1796-4406-b276-326e1d7dd451] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:11:48.655165 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e96915c4-f449-4bd2-b2e6-43585a2679ad tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:48.656189 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e96915c4-f449-4bd2-b2e6-43585a2679ad tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:48.658926 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e96915c4-f449-4bd2-b2e6-43585a2679ad tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:48.658926 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e96915c4-f449-4bd2-b2e6-43585a2679ad tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:48.832576 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-d67d94c1-9f92-4b1f-a498-19c057bc33a7 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b68dacc4b8463ba07fae6372131117', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55a19b7a4682457a9132f5fc80110712', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:11:48.842887 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-d67d94c1-9f92-4b1f-a498-19c057bc33a7 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "GET /compute/v2.1/servers/a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2" status: 200 len: 1542 microversion: 2.1 time: 0.831607 Jul 16 21:11:48.843703 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 483/992] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:11:48 2022] GET /compute/v2.1/servers/a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2 => generated 1542 bytes in 833 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:11:48.852106 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-c4bf0f8e-ca03-47ec-b0c5-85e5353d6b08 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:11:48.853079 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-c4bf0f8e-ca03-47ec-b0c5-85e5353d6b08 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] [instance: b07f0d53-2cd4-440f-be47-077703aa66e6] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:11:48.860825 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c4bf0f8e-ca03-47ec-b0c5-85e5353d6b08 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:48.862051 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c4bf0f8e-ca03-47ec-b0c5-85e5353d6b08 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:48.863097 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c4bf0f8e-ca03-47ec-b0c5-85e5353d6b08 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:48.863751 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c4bf0f8e-ca03-47ec-b0c5-85e5353d6b08 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:49.239320 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-e96915c4-f449-4bd2-b2e6-43585a2679ad tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:11:49.251505 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-e96915c4-f449-4bd2-b2e6-43585a2679ad tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:11:49.252998 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-e96915c4-f449-4bd2-b2e6-43585a2679ad tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:11:49.254458 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-e96915c4-f449-4bd2-b2e6-43585a2679ad tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] 10.222.0.16 "GET /compute/v2.1/servers/0fb5b5b6-1796-4406-b276-326e1d7dd451" status: 200 len: 1362 microversion: 2.25 time: 0.612386 Jul 16 21:11:49.263180 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 510/993] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:11:48 2022] GET /compute/v2.1/servers/0fb5b5b6-1796-4406-b276-326e1d7dd451 => generated 1362 bytes in 614 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:11:49.263180 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-157f6ad1-6384-4901-bd65-8cc0517067c4 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:11:49.263180 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-157f6ad1-6384-4901-bd65-8cc0517067c4 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] [instance: 65a70cca-b978-41ab-b890-6052f653c5c3] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:11:49.268641 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-157f6ad1-6384-4901-bd65-8cc0517067c4 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:49.269091 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-157f6ad1-6384-4901-bd65-8cc0517067c4 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:49.269865 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-157f6ad1-6384-4901-bd65-8cc0517067c4 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:49.270305 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-157f6ad1-6384-4901-bd65-8cc0517067c4 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:49.288913 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-157f6ad1-6384-4901-bd65-8cc0517067c4 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] HTTP exception thrown: Instance 65a70cca-b978-41ab-b890-6052f653c5c3 could not be found. Jul 16 21:11:49.290614 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-157f6ad1-6384-4901-bd65-8cc0517067c4 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Request method failure captured: Jul 16 21:11:49.290614 ncdv-master-1 devstack@n-api.service[138680]: request: GET /compute/v2.1/servers/65a70cca-b978-41ab-b890-6052f653c5c3 HTTP/1.1 Jul 16 21:11:49.290614 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:11:49.290614 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:11:49.290614 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:11:49.290614 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 0 Jul 16 21:11:49.290614 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:11:49.290614 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:11:49.290614 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:11:49.290614 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:11:49.290614 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0yjvVgP0jq68r-ny5Eh1zEbXMYJhEkVpBaJLnFwKzfSzzxAGgQPy49cpkkoW8Rc5r0epWFa_INrkbRxHc5yb1M5XgG50--wLBrLRBZrPuxqH4ynry56OcrGI4F8X9HPf6z5uwbMiaMLyGnsa6t3q-Npn98NzTN2KKNq9Fp4f3zZbeyY Jul 16 21:11:49.290614 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:11:49.290614 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:11:49.290614 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:11:49.290614 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:11:49.290614 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:11:49.290614 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:11:49.290614 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: 649babe54d104436af461f1ee4767001 Jul 16 21:11:49.290614 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-ServersOnMultiNodesTest-1258903790 Jul 16 21:11:49.290614 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:11:49.290614 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:11:49.292851 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/649babe54d104436af461f1ee4767001"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/649babe54d104436af461f1ee4767001"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_649babe54d104436af461f1ee4767001"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/649babe54d104436af461f1ee4767001"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:11:49.292851 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-ServersOnMultiNodesTest-1258903790 Jul 16 21:11:49.292851 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: 649babe54d104436af461f1ee4767001 Jul 16 21:11:49.292851 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-ServersOnMultiNodesTest-1258903790 Jul 16 21:11:49.292851 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-ServersOnMultiNodesTest-1258903790-project Jul 16 21:11:49.292851 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:11:49.292851 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:11:49.292851 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: abeb7020019d41a18de42956d706a011 Jul 16 21:11:49.292851 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-ServersOnMultiNodesTest-1258903790-project Jul 16 21:11:49.292851 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 21:11:49.292851 ncdv-master-1 devstack@n-api.service[138680]: exception: Instance 65a70cca-b978-41ab-b890-6052f653c5c3 could not be found. Jul 16 21:11:49.292851 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': '65a70cca-b978-41ab-b890-6052f653c5c3'} Jul 16 21:11:49.293812 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:11:49.293812 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:11:49.293812 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:11:49.293812 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:11:49.293812 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:11:49.293812 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:11:49.293812 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:11:49.293812 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:11:49.293812 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:11:49.293812 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:11:49.293812 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 65a70cca-b978-41ab-b890-6052f653c5c3 could not be found. Jul 16 21:11:49.293812 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:11:49.293812 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:11:49.293812 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:11:49.293812 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:11:49.293812 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:11:49.293812 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:11:49.293812 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:11:49.293812 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:11:49.293812 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:11:49.295395 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:11:49.295395 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:11:49.295395 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:11:49.295395 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:11:49.295395 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:11:49.295395 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:11:49.295395 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:11:49.295395 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 65a70cca-b978-41ab-b890-6052f653c5c3 could not be found. Jul 16 21:11:49.295395 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:11:49.295395 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:11:49.295395 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:11:49.295395 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:11:49.295395 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:11:49.295395 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:11:49.295395 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:11:49.295395 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:11:49.295395 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 65a70cca-b978-41ab-b890-6052f653c5c3 could not be found. Jul 16 21:11:49.295395 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:11:49.296907 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-157f6ad1-6384-4901-bd65-8cc0517067c4 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Returning 404 to user: Instance 65a70cca-b978-41ab-b890-6052f653c5c3 could not be found. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:11:49.296907 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-157f6ad1-6384-4901-bd65-8cc0517067c4 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] 10.222.0.16 "GET /compute/v2.1/servers/65a70cca-b978-41ab-b890-6052f653c5c3" status: 404 len: 111 microversion: 2.1 time: 0.035977 Jul 16 21:11:49.296907 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 511/994] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:11:49 2022] GET /compute/v2.1/servers/65a70cca-b978-41ab-b890-6052f653c5c3 => generated 111 bytes in 38 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:11:49.310148 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-5292a4e2-1af2-467b-aa7a-39a190dd8901 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:11:49.311077 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-5292a4e2-1af2-467b-aa7a-39a190dd8901 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] [instance: a4a0aaec-dc64-400c-b3f0-ec61fa740b8d] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:11:49.317610 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5292a4e2-1af2-467b-aa7a-39a190dd8901 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:49.318198 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5292a4e2-1af2-467b-aa7a-39a190dd8901 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:49.319302 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5292a4e2-1af2-467b-aa7a-39a190dd8901 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:49.320049 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5292a4e2-1af2-467b-aa7a-39a190dd8901 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:49.351215 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-5292a4e2-1af2-467b-aa7a-39a190dd8901 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] [instance: a4a0aaec-dc64-400c-b3f0-ec61fa740b8d] Going to try to terminate instance {{(pid=138680) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 16 21:11:49.365112 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-c4bf0f8e-ca03-47ec-b0c5-85e5353d6b08 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '52281b3989a04ca2a50b0bc93bf783f0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '30d028fa9b854c8fb71c8cbed2985c27', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:11:49.374045 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-c4bf0f8e-ca03-47ec-b0c5-85e5353d6b08 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] 10.222.0.16 "GET /compute/v2.1/servers/b07f0d53-2cd4-440f-be47-077703aa66e6" status: 200 len: 1555 microversion: 2.1 time: 0.529006 Jul 16 21:11:49.374662 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 484/995] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:11:48 2022] GET /compute/v2.1/servers/b07f0d53-2cd4-440f-be47-077703aa66e6 => generated 1555 bytes in 530 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:11:49.380624 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-7a5306f5-59cc-4354-bf2d-c4b779b8b7ba tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServersTestJSON-server-1656058047", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452", "metadata": {"hello": "world"}, "accessIPv4": "1.1.1.1", "accessIPv6": "0000:0000:0000:0000:0000:babe:220.12.22.2", "adminPass": "***", "networks": [{"uuid": "14535844-911c-4d37-8f0c-c6bfb0452ce6"}], "security_groups": [{"name": "tempest-securitygroup--1198870479"}], "key_name": "tempest-keypair-280247600", "user_data": "IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIHVidW50dSB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH51YnVudHUvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=", "OS-DCF:diskConfig": "}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:11:49.588197 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.network.neutron [None req-7a5306f5-59cc-4354-bf2d-c4b779b8b7ba tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] validate_networks() for [('14535844-911c-4d37-8f0c-c6bfb0452ce6', None, None, None, None, None)] {{(pid=138681) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 16 21:11:49.690320 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-5292a4e2-1af2-467b-aa7a-39a190dd8901 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] 10.222.0.16 "DELETE /compute/v2.1/servers/a4a0aaec-dc64-400c-b3f0-ec61fa740b8d" status: 204 len: 0 microversion: 2.1 time: 0.385165 Jul 16 21:11:49.694047 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 512/996] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:11:49 2022] DELETE /compute/v2.1/servers/a4a0aaec-dc64-400c-b3f0-ec61fa740b8d => generated 0 bytes in 387 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 16 21:11:49.697439 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-c86423be-9323-4cb1-af28-04ea6e33c11c tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:11:49.704251 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-c86423be-9323-4cb1-af28-04ea6e33c11c tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] [instance: b07f0d53-2cd4-440f-be47-077703aa66e6] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:11:49.711267 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c86423be-9323-4cb1-af28-04ea6e33c11c tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:49.712073 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c86423be-9323-4cb1-af28-04ea6e33c11c tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:49.713240 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c86423be-9323-4cb1-af28-04ea6e33c11c tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:49.714043 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c86423be-9323-4cb1-af28-04ea6e33c11c tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:49.909490 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-c86423be-9323-4cb1-af28-04ea6e33c11c tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] 10.222.0.16 "GET /compute/v2.1/servers/b07f0d53-2cd4-440f-be47-077703aa66e6/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.216700 Jul 16 21:11:49.910905 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 513/997] 10.222.0.16 () {58 vars in 1274 bytes} [Sat Jul 16 21:11:49 2022] GET /compute/v2.1/servers/b07f0d53-2cd4-440f-be47-077703aa66e6/os-interface => generated 284 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:11:49.916282 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-83b07e52-878a-4c33-a6c6-44ba312b59a2 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:11:49.917550 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-83b07e52-878a-4c33-a6c6-44ba312b59a2 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] [instance: 8020f553-6025-4f19-835c-c00e2fb4f764] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:11:49.925909 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-83b07e52-878a-4c33-a6c6-44ba312b59a2 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:49.926553 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-83b07e52-878a-4c33-a6c6-44ba312b59a2 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:49.927818 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-83b07e52-878a-4c33-a6c6-44ba312b59a2 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:49.928423 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-83b07e52-878a-4c33-a6c6-44ba312b59a2 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:50.330650 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-83b07e52-878a-4c33-a6c6-44ba312b59a2 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2dfcad8d2b57481bb33aa8f5d7483c51', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cda86413e74b4c1e8e09bd4df4926664', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:11:50.344372 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-83b07e52-878a-4c33-a6c6-44ba312b59a2 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] 10.222.0.16 "GET /compute/v2.1/servers/8020f553-6025-4f19-835c-c00e2fb4f764" status: 200 len: 1323 microversion: 2.1 time: 0.431816 Jul 16 21:11:50.345435 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 514/998] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:11:49 2022] GET /compute/v2.1/servers/8020f553-6025-4f19-835c-c00e2fb4f764 => generated 1323 bytes in 434 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:11:50.351940 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-606ec010-eb4e-4cc1-9ac9-e38a61d38c5d tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:11:50.353062 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-606ec010-eb4e-4cc1-9ac9-e38a61d38c5d tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] [instance: a4a0aaec-dc64-400c-b3f0-ec61fa740b8d] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:11:50.361310 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell0:SELECT=2 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:11:50.363785 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-606ec010-eb4e-4cc1-9ac9-e38a61d38c5d tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:50.364587 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-606ec010-eb4e-4cc1-9ac9-e38a61d38c5d tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:50.365741 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-606ec010-eb4e-4cc1-9ac9-e38a61d38c5d tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:50.366557 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-606ec010-eb4e-4cc1-9ac9-e38a61d38c5d tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:50.530457 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell0:SELECT=11 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:11:51.066101 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-606ec010-eb4e-4cc1-9ac9-e38a61d38c5d tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'abeb7020019d41a18de42956d706a011', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '649babe54d104436af461f1ee4767001', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:11:51.077960 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-606ec010-eb4e-4cc1-9ac9-e38a61d38c5d tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] 10.222.0.16 "GET /compute/v2.1/servers/a4a0aaec-dc64-400c-b3f0-ec61fa740b8d" status: 200 len: 1472 microversion: 2.1 time: 0.731094 Jul 16 21:11:51.087496 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 515/999] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:11:50 2022] GET /compute/v2.1/servers/a4a0aaec-dc64-400c-b3f0-ec61fa740b8d => generated 1472 bytes in 733 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:11:51.087496 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-ee67ce14-8607-4df5-9dfc-f321b4797763 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:11:51.087496 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-ee67ce14-8607-4df5-9dfc-f321b4797763 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:11:51.101162 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ee67ce14-8607-4df5-9dfc-f321b4797763 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:51.102028 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ee67ce14-8607-4df5-9dfc-f321b4797763 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:51.102951 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ee67ce14-8607-4df5-9dfc-f321b4797763 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:51.105238 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ee67ce14-8607-4df5-9dfc-f321b4797763 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:51.221647 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-7a5306f5-59cc-4354-bf2d-c4b779b8b7ba tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] Getting quotas for project fb4ff5d7e42940cdb7be71dadd895ee4. Resources: dict_keys(['metadata_items']) {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:11:51.224934 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-7a5306f5-59cc-4354-bf2d-c4b779b8b7ba tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] Getting quotas for user 034c11cc3f174c6fb44c8d64e55cf8b9 and project fb4ff5d7e42940cdb7be71dadd895ee4. Resources: dict_keys(['metadata_items']) {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:11:51.246042 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-7a5306f5-59cc-4354-bf2d-c4b779b8b7ba tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] Flavor limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 16 21:11:51.246742 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-7a5306f5-59cc-4354-bf2d-c4b779b8b7ba tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] Image limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 16 21:11:51.247531 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-7a5306f5-59cc-4354-bf2d-c4b779b8b7ba tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] Flavor pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 16 21:11:51.248119 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-7a5306f5-59cc-4354-bf2d-c4b779b8b7ba tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] Image pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 16 21:11:51.248540 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-7a5306f5-59cc-4354-bf2d-c4b779b8b7ba tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 16 21:11:51.253235 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_api:SELECT=95,nova_api:INSERT=4,nova_api:UPDATE=1 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:11:51.260312 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7a5306f5-59cc-4354-bf2d-c4b779b8b7ba tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:51.261091 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7a5306f5-59cc-4354-bf2d-c4b779b8b7ba tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:51.261960 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7a5306f5-59cc-4354-bf2d-c4b779b8b7ba tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:51.262411 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7a5306f5-59cc-4354-bf2d-c4b779b8b7ba tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:51.288401 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-7a5306f5-59cc-4354-bf2d-c4b779b8b7ba tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] Getting quotas for project fb4ff5d7e42940cdb7be71dadd895ee4. Resources: {'ram', 'instances', 'cores'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:11:51.295071 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-7a5306f5-59cc-4354-bf2d-c4b779b8b7ba tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] Getting quotas for user 034c11cc3f174c6fb44c8d64e55cf8b9 and project fb4ff5d7e42940cdb7be71dadd895ee4. Resources: {'ram', 'instances', 'cores'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:11:51.309363 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-7a5306f5-59cc-4354-bf2d-c4b779b8b7ba tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] Going to run 1 instances... {{(pid=138681) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 16 21:11:51.312268 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-7a5306f5-59cc-4354-bf2d-c4b779b8b7ba tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] [instance: 2b26717b-355f-417e-8a7c-72c41e1974e1] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138681) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 16 21:11:51.470132 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-ee67ce14-8607-4df5-9dfc-f321b4797763 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b68dacc4b8463ba07fae6372131117', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55a19b7a4682457a9132f5fc80110712', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:11:51.474173 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-7a5306f5-59cc-4354-bf2d-c4b779b8b7ba tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 407 microversion: 2.1 time: 2.090583 Jul 16 21:11:51.481399 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 485/1000] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 21:11:49 2022] POST /compute/v2.1/servers => generated 407 bytes in 2106 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 16 21:11:51.482940 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-ee67ce14-8607-4df5-9dfc-f321b4797763 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "GET /compute/v2.1/servers/a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2" status: 200 len: 1542 microversion: 2.1 time: 0.401370 Jul 16 21:11:51.485001 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 516/1001] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:11:51 2022] GET /compute/v2.1/servers/a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2 => generated 1542 bytes in 404 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:11:51.494635 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-832ab319-856a-40dd-b14c-aa6afc01189a tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:11:51.497801 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-832ab319-856a-40dd-b14c-aa6afc01189a tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] [instance: 0fb5b5b6-1796-4406-b276-326e1d7dd451] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:11:51.505084 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-70439d20-4079-4cb3-81c3-e2689e838e20 req-c0178d0e-a391-402a-9d07-05e293b12eeb service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "0fb5b5b6-1796-4406-b276-326e1d7dd451", "name": "network-vif-plugged", "status": "completed", "tag": "19f08e09-ff8e-43eb-b5f2-421d0eadc41c"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:11:51.523426 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-832ab319-856a-40dd-b14c-aa6afc01189a tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:51.523426 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-832ab319-856a-40dd-b14c-aa6afc01189a tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:51.524355 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-832ab319-856a-40dd-b14c-aa6afc01189a tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:51.525149 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-832ab319-856a-40dd-b14c-aa6afc01189a tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:51.531355 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-70439d20-4079-4cb3-81c3-e2689e838e20 req-c0178d0e-a391-402a-9d07-05e293b12eeb service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:51.532192 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-70439d20-4079-4cb3-81c3-e2689e838e20 req-c0178d0e-a391-402a-9d07-05e293b12eeb service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:51.561577 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-70439d20-4079-4cb3-81c3-e2689e838e20 req-c0178d0e-a391-402a-9d07-05e293b12eeb service nova] Creating event network-vif-plugged:19f08e09-ff8e-43eb-b5f2-421d0eadc41c for instance 0fb5b5b6-1796-4406-b276-326e1d7dd451 on nch2-master-1 Jul 16 21:11:51.575505 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-70439d20-4079-4cb3-81c3-e2689e838e20 req-c0178d0e-a391-402a-9d07-05e293b12eeb service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.086208 Jul 16 21:11:51.577170 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 517/1002] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:11:51 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:11:51.591515 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-be43b3a4-8219-46c0-8a02-7b565b379151 req-05a62af2-1ea9-47f8-bf26-32da099e683a service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "8020f553-6025-4f19-835c-c00e2fb4f764", "tag": "05294e88-e71a-4faa-8acc-79137ca6ee05"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:11:51.621502 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-be43b3a4-8219-46c0-8a02-7b565b379151 req-05a62af2-1ea9-47f8-bf26-32da099e683a service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:51.622227 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-be43b3a4-8219-46c0-8a02-7b565b379151 req-05a62af2-1ea9-47f8-bf26-32da099e683a service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:51.639637 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-be43b3a4-8219-46c0-8a02-7b565b379151 req-05a62af2-1ea9-47f8-bf26-32da099e683a service nova] Creating event network-changed:05294e88-e71a-4faa-8acc-79137ca6ee05 for instance 8020f553-6025-4f19-835c-c00e2fb4f764 on nch1-master-1 Jul 16 21:11:51.646204 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-be43b3a4-8219-46c0-8a02-7b565b379151 req-05a62af2-1ea9-47f8-bf26-32da099e683a service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.065473 Jul 16 21:11:51.647596 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 518/1003] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:11:51 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 70 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:11:51.658570 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-07721715-5247-4c5b-a78c-94e7546f6ffe tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:11:51.659786 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-07721715-5247-4c5b-a78c-94e7546f6ffe tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] [instance: 8020f553-6025-4f19-835c-c00e2fb4f764] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:11:51.668823 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-07721715-5247-4c5b-a78c-94e7546f6ffe tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:51.676210 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-07721715-5247-4c5b-a78c-94e7546f6ffe tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:51.677711 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-07721715-5247-4c5b-a78c-94e7546f6ffe tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:51.678414 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-07721715-5247-4c5b-a78c-94e7546f6ffe tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:51.742376 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell0:SELECT=8 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:11:52.025240 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-832ab319-856a-40dd-b14c-aa6afc01189a tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:11:52.040856 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-832ab319-856a-40dd-b14c-aa6afc01189a tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:11:52.045457 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-832ab319-856a-40dd-b14c-aa6afc01189a tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:11:52.046919 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-832ab319-856a-40dd-b14c-aa6afc01189a tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] 10.222.0.16 "GET /compute/v2.1/servers/0fb5b5b6-1796-4406-b276-326e1d7dd451" status: 200 len: 1362 microversion: 2.25 time: 0.563171 Jul 16 21:11:52.051478 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 486/1004] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:11:51 2022] GET /compute/v2.1/servers/0fb5b5b6-1796-4406-b276-326e1d7dd451 => generated 1362 bytes in 569 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:11:52.052122 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-07721715-5247-4c5b-a78c-94e7546f6ffe tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2dfcad8d2b57481bb33aa8f5d7483c51', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cda86413e74b4c1e8e09bd4df4926664', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:11:52.056687 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-df3c8855-46d3-4fee-95fe-977dfe6b6963 tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:11:52.058106 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-df3c8855-46d3-4fee-95fe-977dfe6b6963 tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] [instance: 2b26717b-355f-417e-8a7c-72c41e1974e1] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:11:52.061519 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-07721715-5247-4c5b-a78c-94e7546f6ffe tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] 10.222.0.16 "GET /compute/v2.1/servers/8020f553-6025-4f19-835c-c00e2fb4f764" status: 200 len: 1323 microversion: 2.1 time: 0.411319 Jul 16 21:11:52.062604 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 519/1005] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:11:51 2022] GET /compute/v2.1/servers/8020f553-6025-4f19-835c-c00e2fb4f764 => generated 1323 bytes in 414 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:11:52.067651 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-5c5daf1c-c9ef-4741-9962-4da816f763e0 req-4451afbd-5617-45d7-aba5-c5d512ec907c service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "b07f0d53-2cd4-440f-be47-077703aa66e6", "tag": "9b0f580d-ea65-4f1a-9836-c0f8d10895e9"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:11:52.077494 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-5c5daf1c-c9ef-4741-9962-4da816f763e0 req-4451afbd-5617-45d7-aba5-c5d512ec907c service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:52.077954 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-5c5daf1c-c9ef-4741-9962-4da816f763e0 req-4451afbd-5617-45d7-aba5-c5d512ec907c service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:52.091823 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-5c5daf1c-c9ef-4741-9962-4da816f763e0 req-4451afbd-5617-45d7-aba5-c5d512ec907c service nova] Creating event network-changed:9b0f580d-ea65-4f1a-9836-c0f8d10895e9 for instance b07f0d53-2cd4-440f-be47-077703aa66e6 on nch1-master-1 Jul 16 21:11:52.097531 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-5c5daf1c-c9ef-4741-9962-4da816f763e0 req-4451afbd-5617-45d7-aba5-c5d512ec907c service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.033617 Jul 16 21:11:52.098495 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 520/1006] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:11:52 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:11:52.105838 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-79f32911-3ffc-4eb0-bb52-136e09099639 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:11:52.105838 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-79f32911-3ffc-4eb0-bb52-136e09099639 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] [instance: a4a0aaec-dc64-400c-b3f0-ec61fa740b8d] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:11:52.112724 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-79f32911-3ffc-4eb0-bb52-136e09099639 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:52.113837 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-79f32911-3ffc-4eb0-bb52-136e09099639 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:52.125048 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-79f32911-3ffc-4eb0-bb52-136e09099639 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:52.125661 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-79f32911-3ffc-4eb0-bb52-136e09099639 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.012s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:52.229612 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-df3c8855-46d3-4fee-95fe-977dfe6b6963 tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '034c11cc3f174c6fb44c8d64e55cf8b9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb4ff5d7e42940cdb7be71dadd895ee4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:11:52.253130 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-df3c8855-46d3-4fee-95fe-977dfe6b6963 tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] 10.222.0.16 "GET /compute/v2.1/servers/2b26717b-355f-417e-8a7c-72c41e1974e1" status: 200 len: 1269 microversion: 2.1 time: 0.200336 Jul 16 21:11:52.254824 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 487/1007] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:11:52 2022] GET /compute/v2.1/servers/2b26717b-355f-417e-8a7c-72c41e1974e1 => generated 1269 bytes in 203 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:11:52.505975 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-8f9e2605-aa4c-447b-ab0a-12d0236ec3a9 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:11:52.507216 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-8f9e2605-aa4c-447b-ab0a-12d0236ec3a9 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:11:52.524832 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8f9e2605-aa4c-447b-ab0a-12d0236ec3a9 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:52.525872 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8f9e2605-aa4c-447b-ab0a-12d0236ec3a9 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:52.527948 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8f9e2605-aa4c-447b-ab0a-12d0236ec3a9 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:52.528551 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8f9e2605-aa4c-447b-ab0a-12d0236ec3a9 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:52.585678 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-79f32911-3ffc-4eb0-bb52-136e09099639 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'abeb7020019d41a18de42956d706a011', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '649babe54d104436af461f1ee4767001', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:11:52.594467 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-79f32911-3ffc-4eb0-bb52-136e09099639 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] 10.222.0.16 "GET /compute/v2.1/servers/a4a0aaec-dc64-400c-b3f0-ec61fa740b8d" status: 200 len: 1472 microversion: 2.1 time: 0.494717 Jul 16 21:11:52.595120 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 521/1008] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:11:52 2022] GET /compute/v2.1/servers/a4a0aaec-dc64-400c-b3f0-ec61fa740b8d => generated 1472 bytes in 496 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:11:52.696953 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-8f9e2605-aa4c-447b-ab0a-12d0236ec3a9 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b68dacc4b8463ba07fae6372131117', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55a19b7a4682457a9132f5fc80110712', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:11:52.707662 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-8f9e2605-aa4c-447b-ab0a-12d0236ec3a9 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "GET /compute/v2.1/servers/a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2" status: 200 len: 1542 microversion: 2.1 time: 0.206137 Jul 16 21:11:52.718660 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 488/1009] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:11:52 2022] GET /compute/v2.1/servers/a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2 => generated 1542 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:11:53.068783 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-5622b8bc-7f7d-48bb-a590-e61ac6be439b tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:11:53.069400 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-5622b8bc-7f7d-48bb-a590-e61ac6be439b tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] [instance: 0fb5b5b6-1796-4406-b276-326e1d7dd451] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:11:53.076878 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5622b8bc-7f7d-48bb-a590-e61ac6be439b tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:53.077569 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5622b8bc-7f7d-48bb-a590-e61ac6be439b tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:53.078099 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-f82c363c-c552-4ae4-a6f5-060dfdcd9476 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:11:53.078649 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5622b8bc-7f7d-48bb-a590-e61ac6be439b tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:53.078744 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-f82c363c-c552-4ae4-a6f5-060dfdcd9476 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] [instance: 8020f553-6025-4f19-835c-c00e2fb4f764] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:11:53.079211 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5622b8bc-7f7d-48bb-a590-e61ac6be439b tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:53.086691 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f82c363c-c552-4ae4-a6f5-060dfdcd9476 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:53.087184 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f82c363c-c552-4ae4-a6f5-060dfdcd9476 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:53.091474 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f82c363c-c552-4ae4-a6f5-060dfdcd9476 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:53.091474 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f82c363c-c552-4ae4-a6f5-060dfdcd9476 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:53.469542 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-5622b8bc-7f7d-48bb-a590-e61ac6be439b tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:11:53.499192 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-5622b8bc-7f7d-48bb-a590-e61ac6be439b tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:11:53.505450 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-5622b8bc-7f7d-48bb-a590-e61ac6be439b tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:11:53.506663 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-5622b8bc-7f7d-48bb-a590-e61ac6be439b tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] 10.222.0.16 "GET /compute/v2.1/servers/0fb5b5b6-1796-4406-b276-326e1d7dd451" status: 200 len: 1362 microversion: 2.25 time: 0.441938 Jul 16 21:11:53.532420 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 522/1010] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:11:53 2022] GET /compute/v2.1/servers/0fb5b5b6-1796-4406-b276-326e1d7dd451 => generated 1362 bytes in 467 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:11:53.545184 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-ee36ab92-fd15-4a1b-8d6d-c6f7f6b6c8ef tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:11:53.548020 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-ee36ab92-fd15-4a1b-8d6d-c6f7f6b6c8ef tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] [instance: 2b26717b-355f-417e-8a7c-72c41e1974e1] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:11:53.578113 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-f82c363c-c552-4ae4-a6f5-060dfdcd9476 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2dfcad8d2b57481bb33aa8f5d7483c51', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cda86413e74b4c1e8e09bd4df4926664', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:11:53.580541 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ee36ab92-fd15-4a1b-8d6d-c6f7f6b6c8ef tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:53.585341 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ee36ab92-fd15-4a1b-8d6d-c6f7f6b6c8ef tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:53.587546 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ee36ab92-fd15-4a1b-8d6d-c6f7f6b6c8ef tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:53.587957 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-f82c363c-c552-4ae4-a6f5-060dfdcd9476 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] 10.222.0.16 "GET /compute/v2.1/servers/8020f553-6025-4f19-835c-c00e2fb4f764" status: 200 len: 1323 microversion: 2.1 time: 0.512982 Jul 16 21:11:53.588522 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ee36ab92-fd15-4a1b-8d6d-c6f7f6b6c8ef tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:53.598908 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 489/1011] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:11:53 2022] GET /compute/v2.1/servers/8020f553-6025-4f19-835c-c00e2fb4f764 => generated 1323 bytes in 517 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:11:53.613510 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-b0a2fd1a-784a-4e3e-b065-d20f6cbe990a tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:11:53.615060 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-b0a2fd1a-784a-4e3e-b065-d20f6cbe990a tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] [instance: a4a0aaec-dc64-400c-b3f0-ec61fa740b8d] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:11:53.634050 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b0a2fd1a-784a-4e3e-b065-d20f6cbe990a tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:53.639397 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b0a2fd1a-784a-4e3e-b065-d20f6cbe990a tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:53.639397 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b0a2fd1a-784a-4e3e-b065-d20f6cbe990a tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:53.639397 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b0a2fd1a-784a-4e3e-b065-d20f6cbe990a tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:53.769323 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-ee36ab92-fd15-4a1b-8d6d-c6f7f6b6c8ef tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '034c11cc3f174c6fb44c8d64e55cf8b9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb4ff5d7e42940cdb7be71dadd895ee4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:11:53.800439 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-ee36ab92-fd15-4a1b-8d6d-c6f7f6b6c8ef tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] 10.222.0.16 "GET /compute/v2.1/servers/2b26717b-355f-417e-8a7c-72c41e1974e1" status: 200 len: 1273 microversion: 2.1 time: 0.251120 Jul 16 21:11:53.800439 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 523/1012] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:11:53 2022] GET /compute/v2.1/servers/2b26717b-355f-417e-8a7c-72c41e1974e1 => generated 1273 bytes in 254 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:11:53.800939 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-d9605a8d-120e-4e91-bc91-a8f7a5383ee6 req-a6cf327a-8295-4ea7-adbb-404150d00659 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "a4a0aaec-dc64-400c-b3f0-ec61fa740b8d", "name": "network-vif-deleted", "tag": "34a8af9d-6c47-4062-aba8-98d7061acf17"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:11:53.824812 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-d9605a8d-120e-4e91-bc91-a8f7a5383ee6 req-a6cf327a-8295-4ea7-adbb-404150d00659 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:53.827248 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-d9605a8d-120e-4e91-bc91-a8f7a5383ee6 req-a6cf327a-8295-4ea7-adbb-404150d00659 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:53.860414 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-d9605a8d-120e-4e91-bc91-a8f7a5383ee6 req-a6cf327a-8295-4ea7-adbb-404150d00659 service nova] Creating event network-vif-deleted:34a8af9d-6c47-4062-aba8-98d7061acf17 for instance a4a0aaec-dc64-400c-b3f0-ec61fa740b8d on nch1-master-1 Jul 16 21:11:53.871576 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-d9605a8d-120e-4e91-bc91-a8f7a5383ee6 req-a6cf327a-8295-4ea7-adbb-404150d00659 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.080154 Jul 16 21:11:53.872196 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 524/1013] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:11:53 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:11:53.881046 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-45c65432-4b21-4a93-ad81-ba0693892367 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:11:53.882055 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-45c65432-4b21-4a93-ad81-ba0693892367 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:11:53.889721 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-45c65432-4b21-4a93-ad81-ba0693892367 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:53.890741 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-45c65432-4b21-4a93-ad81-ba0693892367 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:53.892025 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-45c65432-4b21-4a93-ad81-ba0693892367 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:53.892660 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-45c65432-4b21-4a93-ad81-ba0693892367 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:53.917648 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:11:53.924249 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-b0a2fd1a-784a-4e3e-b065-d20f6cbe990a tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'abeb7020019d41a18de42956d706a011', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '649babe54d104436af461f1ee4767001', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:11:53.941901 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-b0a2fd1a-784a-4e3e-b065-d20f6cbe990a tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] 10.222.0.16 "GET /compute/v2.1/servers/a4a0aaec-dc64-400c-b3f0-ec61fa740b8d" status: 200 len: 1301 microversion: 2.1 time: 0.325169 Jul 16 21:11:53.942890 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 490/1014] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:11:53 2022] GET /compute/v2.1/servers/a4a0aaec-dc64-400c-b3f0-ec61fa740b8d => generated 1301 bytes in 336 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:11:54.336436 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-45c65432-4b21-4a93-ad81-ba0693892367 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b68dacc4b8463ba07fae6372131117', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55a19b7a4682457a9132f5fc80110712', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:11:54.348864 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-45c65432-4b21-4a93-ad81-ba0693892367 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "GET /compute/v2.1/servers/a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2" status: 200 len: 1542 microversion: 2.1 time: 0.468632 Jul 16 21:11:54.349503 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 525/1015] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:11:53 2022] GET /compute/v2.1/servers/a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2 => generated 1542 bytes in 474 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:11:54.527133 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-53982ed6-ce95-44e8-8d05-287a19ae8d19 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:11:54.528006 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-53982ed6-ce95-44e8-8d05-287a19ae8d19 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] [instance: 0fb5b5b6-1796-4406-b276-326e1d7dd451] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:11:54.536886 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-53982ed6-ce95-44e8-8d05-287a19ae8d19 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:54.537355 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-53982ed6-ce95-44e8-8d05-287a19ae8d19 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:54.538147 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-53982ed6-ce95-44e8-8d05-287a19ae8d19 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:54.538575 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-53982ed6-ce95-44e8-8d05-287a19ae8d19 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:54.543121 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:11:54.607580 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-b6523a47-c622-4498-b5bd-da1306463235 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:11:54.608292 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-b6523a47-c622-4498-b5bd-da1306463235 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] [instance: 8020f553-6025-4f19-835c-c00e2fb4f764] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:11:54.614886 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b6523a47-c622-4498-b5bd-da1306463235 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:54.615681 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b6523a47-c622-4498-b5bd-da1306463235 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:54.616512 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b6523a47-c622-4498-b5bd-da1306463235 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:54.617490 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b6523a47-c622-4498-b5bd-da1306463235 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:54.748797 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-53982ed6-ce95-44e8-8d05-287a19ae8d19 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:11:54.756853 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-53982ed6-ce95-44e8-8d05-287a19ae8d19 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:11:54.758845 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-53982ed6-ce95-44e8-8d05-287a19ae8d19 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:11:54.759776 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-53982ed6-ce95-44e8-8d05-287a19ae8d19 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] 10.222.0.16 "GET /compute/v2.1/servers/0fb5b5b6-1796-4406-b276-326e1d7dd451" status: 200 len: 1557 microversion: 2.25 time: 0.237667 Jul 16 21:11:54.760333 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 491/1016] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:11:54 2022] GET /compute/v2.1/servers/0fb5b5b6-1796-4406-b276-326e1d7dd451 => generated 1557 bytes in 239 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:11:54.777976 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-5a98cd57-3668-4313-a4ba-3b7d16fb6f12 tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:11:54.778621 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-5a98cd57-3668-4313-a4ba-3b7d16fb6f12 tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] [instance: 0fb5b5b6-1796-4406-b276-326e1d7dd451] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:11:54.784655 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5a98cd57-3668-4313-a4ba-3b7d16fb6f12 tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:54.787590 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5a98cd57-3668-4313-a4ba-3b7d16fb6f12 tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:54.787590 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5a98cd57-3668-4313-a4ba-3b7d16fb6f12 tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:54.787590 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5a98cd57-3668-4313-a4ba-3b7d16fb6f12 tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:54.805607 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-b6523a47-c622-4498-b5bd-da1306463235 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2dfcad8d2b57481bb33aa8f5d7483c51', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cda86413e74b4c1e8e09bd4df4926664', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:11:54.814357 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-b6523a47-c622-4498-b5bd-da1306463235 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] 10.222.0.16 "GET /compute/v2.1/servers/8020f553-6025-4f19-835c-c00e2fb4f764" status: 200 len: 1323 microversion: 2.1 time: 0.210909 Jul 16 21:11:54.824755 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 526/1017] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:11:54 2022] GET /compute/v2.1/servers/8020f553-6025-4f19-835c-c00e2fb4f764 => generated 1323 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:11:54.824755 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-64b5c6c5-e5a9-4166-a014-e92798c7e2b4 tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:11:54.824755 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-64b5c6c5-e5a9-4166-a014-e92798c7e2b4 tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] [instance: 2b26717b-355f-417e-8a7c-72c41e1974e1] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:11:54.827333 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-64b5c6c5-e5a9-4166-a014-e92798c7e2b4 tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:54.827789 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-64b5c6c5-e5a9-4166-a014-e92798c7e2b4 tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:54.828555 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-64b5c6c5-e5a9-4166-a014-e92798c7e2b4 tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:54.828997 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-64b5c6c5-e5a9-4166-a014-e92798c7e2b4 tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:54.905405 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-64b5c6c5-e5a9-4166-a014-e92798c7e2b4 tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '034c11cc3f174c6fb44c8d64e55cf8b9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb4ff5d7e42940cdb7be71dadd895ee4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:11:54.917542 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-64b5c6c5-e5a9-4166-a014-e92798c7e2b4 tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] 10.222.0.16 "GET /compute/v2.1/servers/2b26717b-355f-417e-8a7c-72c41e1974e1" status: 200 len: 1321 microversion: 2.1 time: 0.101140 Jul 16 21:11:54.918354 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 527/1018] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:11:54 2022] GET /compute/v2.1/servers/2b26717b-355f-417e-8a7c-72c41e1974e1 => generated 1321 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:11:54.959867 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-b57ff08e-d024-40de-9005-a035f34e5134 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:11:54.961233 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-b57ff08e-d024-40de-9005-a035f34e5134 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] [instance: a4a0aaec-dc64-400c-b3f0-ec61fa740b8d] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:11:54.967757 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b57ff08e-d024-40de-9005-a035f34e5134 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:54.968210 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b57ff08e-d024-40de-9005-a035f34e5134 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:54.969006 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b57ff08e-d024-40de-9005-a035f34e5134 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:54.969452 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b57ff08e-d024-40de-9005-a035f34e5134 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:54.974635 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.objects.instance [None req-5a98cd57-3668-4313-a4ba-3b7d16fb6f12 tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] Lazy-loading 'services' on Instance uuid 0fb5b5b6-1796-4406-b276-326e1d7dd451 {{(pid=138681) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:11:54.985328 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-b57ff08e-d024-40de-9005-a035f34e5134 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] HTTP exception thrown: Instance a4a0aaec-dc64-400c-b3f0-ec61fa740b8d could not be found. Jul 16 21:11:54.987525 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-b57ff08e-d024-40de-9005-a035f34e5134 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Request method failure captured: Jul 16 21:11:54.987525 ncdv-master-1 devstack@n-api.service[138680]: request: GET /compute/v2.1/servers/a4a0aaec-dc64-400c-b3f0-ec61fa740b8d HTTP/1.1 Jul 16 21:11:54.987525 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:11:54.987525 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:11:54.987525 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:11:54.987525 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 0 Jul 16 21:11:54.987525 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:11:54.987525 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:11:54.987525 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:11:54.987525 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:11:54.987525 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0yjvVgP0jq68r-ny5Eh1zEbXMYJhEkVpBaJLnFwKzfSzzxAGgQPy49cpkkoW8Rc5r0epWFa_INrkbRxHc5yb1M5XgG50--wLBrLRBZrPuxqH4ynry56OcrGI4F8X9HPf6z5uwbMiaMLyGnsa6t3q-Npn98NzTN2KKNq9Fp4f3zZbeyY Jul 16 21:11:54.987525 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:11:54.987525 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:11:54.987525 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:11:54.987525 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:11:54.987525 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:11:54.987525 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:11:54.987525 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: 649babe54d104436af461f1ee4767001 Jul 16 21:11:54.987525 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-ServersOnMultiNodesTest-1258903790 Jul 16 21:11:54.987525 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:11:54.987525 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:11:54.989126 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/649babe54d104436af461f1ee4767001"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/649babe54d104436af461f1ee4767001"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_649babe54d104436af461f1ee4767001"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/649babe54d104436af461f1ee4767001"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:11:54.989126 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-ServersOnMultiNodesTest-1258903790 Jul 16 21:11:54.989126 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: 649babe54d104436af461f1ee4767001 Jul 16 21:11:54.989126 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-ServersOnMultiNodesTest-1258903790 Jul 16 21:11:54.989126 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-ServersOnMultiNodesTest-1258903790-project Jul 16 21:11:54.989126 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:11:54.989126 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:11:54.989126 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: abeb7020019d41a18de42956d706a011 Jul 16 21:11:54.989126 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-ServersOnMultiNodesTest-1258903790-project Jul 16 21:11:54.989126 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 21:11:54.989126 ncdv-master-1 devstack@n-api.service[138680]: exception: Instance a4a0aaec-dc64-400c-b3f0-ec61fa740b8d could not be found. Jul 16 21:11:54.989126 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': 'a4a0aaec-dc64-400c-b3f0-ec61fa740b8d'} Jul 16 21:11:54.990017 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:11:54.990017 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:11:54.990017 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:11:54.990017 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:11:54.990017 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:11:54.990017 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:11:54.990017 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:11:54.990017 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:11:54.990017 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:11:54.990017 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:11:54.990017 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance a4a0aaec-dc64-400c-b3f0-ec61fa740b8d could not be found. Jul 16 21:11:54.990017 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:11:54.990017 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:11:54.990017 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:11:54.990017 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:11:54.990017 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:11:54.990017 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:11:54.990017 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:11:54.990017 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:11:54.990017 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:11:54.991586 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:11:54.991586 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:11:54.991586 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:11:54.991586 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:11:54.991586 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:11:54.991586 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:11:54.991586 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:11:54.991586 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance a4a0aaec-dc64-400c-b3f0-ec61fa740b8d could not be found. Jul 16 21:11:54.991586 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:11:54.991586 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:11:54.991586 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:11:54.991586 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:11:54.991586 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:11:54.991586 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:11:54.991586 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:11:54.991586 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:11:54.991586 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance a4a0aaec-dc64-400c-b3f0-ec61fa740b8d could not be found. Jul 16 21:11:54.991586 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:11:54.992969 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-5a98cd57-3668-4313-a4ba-3b7d16fb6f12 tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/0fb5b5b6-1796-4406-b276-326e1d7dd451" status: 200 len: 2050 microversion: 2.25 time: 0.216028 Jul 16 21:11:54.992969 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 492/1019] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:11:54 2022] GET /compute/v2.1/servers/0fb5b5b6-1796-4406-b276-326e1d7dd451 => generated 2050 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 16 21:11:54.993948 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-b57ff08e-d024-40de-9005-a035f34e5134 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Returning 404 to user: Instance a4a0aaec-dc64-400c-b3f0-ec61fa740b8d could not be found. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:11:54.994821 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-b57ff08e-d024-40de-9005-a035f34e5134 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] 10.222.0.16 "GET /compute/v2.1/servers/a4a0aaec-dc64-400c-b3f0-ec61fa740b8d" status: 404 len: 111 microversion: 2.1 time: 0.038860 Jul 16 21:11:54.995995 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 528/1020] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:11:54 2022] GET /compute/v2.1/servers/a4a0aaec-dc64-400c-b3f0-ec61fa740b8d => generated 111 bytes in 41 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Jul 16 21:11:55.004325 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-32dd770d-aabf-426f-8879-307290981ab5 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:11:55.005069 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-32dd770d-aabf-426f-8879-307290981ab5 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] [instance: b941b324-a54d-446f-bf92-570a80ba2152] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:11:55.011565 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-32dd770d-aabf-426f-8879-307290981ab5 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:55.011890 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-6fb4b554-9962-4e52-89c4-73dc10400275 tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:11:55.012303 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-32dd770d-aabf-426f-8879-307290981ab5 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:55.012651 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-6fb4b554-9962-4e52-89c4-73dc10400275 tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] [instance: 0fb5b5b6-1796-4406-b276-326e1d7dd451] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:11:55.013218 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-32dd770d-aabf-426f-8879-307290981ab5 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:55.013867 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-32dd770d-aabf-426f-8879-307290981ab5 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:55.020300 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6fb4b554-9962-4e52-89c4-73dc10400275 tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:55.020756 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6fb4b554-9962-4e52-89c4-73dc10400275 tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:55.021533 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6fb4b554-9962-4e52-89c4-73dc10400275 tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:55.021969 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6fb4b554-9962-4e52-89c4-73dc10400275 tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:55.050071 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-32dd770d-aabf-426f-8879-307290981ab5 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] [instance: b941b324-a54d-446f-bf92-570a80ba2152] Going to try to terminate instance {{(pid=138681) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 16 21:11:55.181706 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.objects.instance [None req-6fb4b554-9962-4e52-89c4-73dc10400275 tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] Lazy-loading 'services' on Instance uuid 0fb5b5b6-1796-4406-b276-326e1d7dd451 {{(pid=138680) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:11:55.197464 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-6fb4b554-9962-4e52-89c4-73dc10400275 tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/0fb5b5b6-1796-4406-b276-326e1d7dd451" status: 200 len: 2050 microversion: 2.25 time: 0.191657 Jul 16 21:11:55.203241 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 529/1021] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:11:55 2022] GET /compute/v2.1/servers/0fb5b5b6-1796-4406-b276-326e1d7dd451 => generated 2050 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:11:55.216902 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-69506b12-23e3-44bf-8159-8e069cf187e3 tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:11:55.219287 ncdv-master-1 devstack@n-api.service[138680]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-services:list": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:11:55.219287 ncdv-master-1 devstack@n-api.service[138680]: warnings.warn(msg) Jul 16 21:11:55.220577 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-69506b12-23e3-44bf-8159-8e069cf187e3 tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:55.221093 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-69506b12-23e3-44bf-8159-8e069cf187e3 tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:55.221975 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-69506b12-23e3-44bf-8159-8e069cf187e3 tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:55.222408 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-69506b12-23e3-44bf-8159-8e069cf187e3 tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:55.250191 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-69506b12-23e3-44bf-8159-8e069cf187e3 tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] 10.222.0.16 "GET /compute/v2.1/os-services?binary=nova-compute" status: 200 len: 424 microversion: 2.25 time: 0.037814 Jul 16 21:11:55.250952 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 530/1022] 10.222.0.16 () {60 vars in 1216 bytes} [Sat Jul 16 21:11:55 2022] GET /compute/v2.1/os-services?binary=nova-compute => generated 424 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 16 21:11:55.266406 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-5e3c8025-3e30-40d1-9fee-80b0e1ea8be8 tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] Action: 'action', calling method: >, body: {"pause": {}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:11:55.266984 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-5e3c8025-3e30-40d1-9fee-80b0e1ea8be8 tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] [instance: 0fb5b5b6-1796-4406-b276-326e1d7dd451] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:11:55.273533 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5e3c8025-3e30-40d1-9fee-80b0e1ea8be8 tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:55.274339 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5e3c8025-3e30-40d1-9fee-80b0e1ea8be8 tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:55.275257 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5e3c8025-3e30-40d1-9fee-80b0e1ea8be8 tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:55.275799 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5e3c8025-3e30-40d1-9fee-80b0e1ea8be8 tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:55.362361 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-32dd770d-aabf-426f-8879-307290981ab5 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] 10.222.0.16 "DELETE /compute/v2.1/servers/b941b324-a54d-446f-bf92-570a80ba2152" status: 204 len: 0 microversion: 2.1 time: 0.361490 Jul 16 21:11:55.364478 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 493/1023] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:11:54 2022] DELETE /compute/v2.1/servers/b941b324-a54d-446f-bf92-570a80ba2152 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Jul 16 21:11:55.371378 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-7e24755f-9096-44c8-8d62-33b94056f749 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:11:55.372188 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-7e24755f-9096-44c8-8d62-33b94056f749 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:11:55.378413 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7e24755f-9096-44c8-8d62-33b94056f749 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:55.378872 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7e24755f-9096-44c8-8d62-33b94056f749 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:55.379673 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7e24755f-9096-44c8-8d62-33b94056f749 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:55.380115 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7e24755f-9096-44c8-8d62-33b94056f749 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:55.439783 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-5e3c8025-3e30-40d1-9fee-80b0e1ea8be8 tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] 10.222.0.16 "POST /compute/v2.1/servers/0fb5b5b6-1796-4406-b276-326e1d7dd451/action" status: 202 len: 0 microversion: 2.25 time: 0.177794 Jul 16 21:11:55.443396 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 531/1024] 10.222.0.16 () {62 vars in 1318 bytes} [Sat Jul 16 21:11:55 2022] POST /compute/v2.1/servers/0fb5b5b6-1796-4406-b276-326e1d7dd451/action => generated 0 bytes in 183 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Jul 16 21:11:55.449594 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-6916b8d5-84aa-472b-9fbd-c2071e6060bd tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:11:55.450310 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-6916b8d5-84aa-472b-9fbd-c2071e6060bd tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] [instance: b941b324-a54d-446f-bf92-570a80ba2152] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:11:55.457643 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6916b8d5-84aa-472b-9fbd-c2071e6060bd tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:55.459160 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6916b8d5-84aa-472b-9fbd-c2071e6060bd tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:55.459275 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6916b8d5-84aa-472b-9fbd-c2071e6060bd tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:55.460093 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6916b8d5-84aa-472b-9fbd-c2071e6060bd tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:55.558167 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-7e24755f-9096-44c8-8d62-33b94056f749 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b68dacc4b8463ba07fae6372131117', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55a19b7a4682457a9132f5fc80110712', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:11:55.570431 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-7e24755f-9096-44c8-8d62-33b94056f749 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "GET /compute/v2.1/servers/a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2" status: 200 len: 1542 microversion: 2.1 time: 0.204051 Jul 16 21:11:55.571301 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 494/1025] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:11:55 2022] GET /compute/v2.1/servers/a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2 => generated 1542 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:11:55.588074 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-337c1b42-8aa7-41e4-95fd-f205d361aacf tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:11:55.588074 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-337c1b42-8aa7-41e4-95fd-f205d361aacf tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] [instance: 0fb5b5b6-1796-4406-b276-326e1d7dd451] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:11:55.593833 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-337c1b42-8aa7-41e4-95fd-f205d361aacf tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:55.594566 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-337c1b42-8aa7-41e4-95fd-f205d361aacf tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:55.607948 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-337c1b42-8aa7-41e4-95fd-f205d361aacf tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:55.608688 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-337c1b42-8aa7-41e4-95fd-f205d361aacf tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:55.634237 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-6916b8d5-84aa-472b-9fbd-c2071e6060bd tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'abeb7020019d41a18de42956d706a011', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '649babe54d104436af461f1ee4767001', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:11:55.644757 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-6916b8d5-84aa-472b-9fbd-c2071e6060bd tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] 10.222.0.16 "GET /compute/v2.1/servers/b941b324-a54d-446f-bf92-570a80ba2152" status: 200 len: 1472 microversion: 2.1 time: 0.199906 Jul 16 21:11:55.645432 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 532/1026] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:11:55 2022] GET /compute/v2.1/servers/b941b324-a54d-446f-bf92-570a80ba2152 => generated 1472 bytes in 202 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:11:55.753517 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.objects.instance [None req-337c1b42-8aa7-41e4-95fd-f205d361aacf tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] Lazy-loading 'services' on Instance uuid 0fb5b5b6-1796-4406-b276-326e1d7dd451 {{(pid=138681) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:11:55.768582 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-337c1b42-8aa7-41e4-95fd-f205d361aacf tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/0fb5b5b6-1796-4406-b276-326e1d7dd451" status: 200 len: 2055 microversion: 2.25 time: 0.191254 Jul 16 21:11:55.769184 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 495/1027] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:11:55 2022] GET /compute/v2.1/servers/0fb5b5b6-1796-4406-b276-326e1d7dd451 => generated 2055 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:11:55.832315 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-bf94f87f-5dd9-46d3-ada8-8105e7cf3e51 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:11:55.832973 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-bf94f87f-5dd9-46d3-ada8-8105e7cf3e51 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] [instance: 8020f553-6025-4f19-835c-c00e2fb4f764] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:11:55.839935 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-bf94f87f-5dd9-46d3-ada8-8105e7cf3e51 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:55.843355 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-bf94f87f-5dd9-46d3-ada8-8105e7cf3e51 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:55.843355 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-bf94f87f-5dd9-46d3-ada8-8105e7cf3e51 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:55.843355 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-bf94f87f-5dd9-46d3-ada8-8105e7cf3e51 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:55.938094 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-93763630-e89b-4699-87c7-dbaaa0f238ad tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:11:55.938372 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-93763630-e89b-4699-87c7-dbaaa0f238ad tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] [instance: 2b26717b-355f-417e-8a7c-72c41e1974e1] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:11:55.944894 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-93763630-e89b-4699-87c7-dbaaa0f238ad tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:55.945345 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-93763630-e89b-4699-87c7-dbaaa0f238ad tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:55.946134 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-93763630-e89b-4699-87c7-dbaaa0f238ad tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:55.946585 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-93763630-e89b-4699-87c7-dbaaa0f238ad tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:56.005829 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-bf94f87f-5dd9-46d3-ada8-8105e7cf3e51 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2dfcad8d2b57481bb33aa8f5d7483c51', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cda86413e74b4c1e8e09bd4df4926664', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:11:56.018435 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-bf94f87f-5dd9-46d3-ada8-8105e7cf3e51 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] 10.222.0.16 "GET /compute/v2.1/servers/8020f553-6025-4f19-835c-c00e2fb4f764" status: 200 len: 1323 microversion: 2.1 time: 0.189587 Jul 16 21:11:56.031148 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 533/1028] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:11:55 2022] GET /compute/v2.1/servers/8020f553-6025-4f19-835c-c00e2fb4f764 => generated 1323 bytes in 203 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:11:56.153647 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-93763630-e89b-4699-87c7-dbaaa0f238ad tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '034c11cc3f174c6fb44c8d64e55cf8b9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb4ff5d7e42940cdb7be71dadd895ee4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:11:56.153647 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-93763630-e89b-4699-87c7-dbaaa0f238ad tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] 10.222.0.16 "GET /compute/v2.1/servers/2b26717b-355f-417e-8a7c-72c41e1974e1" status: 200 len: 1395 microversion: 2.1 time: 0.218797 Jul 16 21:11:56.153647 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 496/1029] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:11:55 2022] GET /compute/v2.1/servers/2b26717b-355f-417e-8a7c-72c41e1974e1 => generated 1395 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:11:56.588442 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-2bdc19a0-2606-4de0-b948-cc6b388c5388 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:11:56.589997 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-2bdc19a0-2606-4de0-b948-cc6b388c5388 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:11:56.597680 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2bdc19a0-2606-4de0-b948-cc6b388c5388 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:56.599267 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2bdc19a0-2606-4de0-b948-cc6b388c5388 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:56.599267 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2bdc19a0-2606-4de0-b948-cc6b388c5388 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:56.599526 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2bdc19a0-2606-4de0-b948-cc6b388c5388 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:56.663566 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-42176561-013c-4fb7-a524-e933c9788b9a tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:11:56.664318 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-42176561-013c-4fb7-a524-e933c9788b9a tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] [instance: b941b324-a54d-446f-bf92-570a80ba2152] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:11:56.672816 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-42176561-013c-4fb7-a524-e933c9788b9a tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:56.674620 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-42176561-013c-4fb7-a524-e933c9788b9a tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:56.675546 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-42176561-013c-4fb7-a524-e933c9788b9a tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:56.676065 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-42176561-013c-4fb7-a524-e933c9788b9a tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:56.933809 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-2bdc19a0-2606-4de0-b948-cc6b388c5388 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b68dacc4b8463ba07fae6372131117', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55a19b7a4682457a9132f5fc80110712', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:11:56.944279 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-2bdc19a0-2606-4de0-b948-cc6b388c5388 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "GET /compute/v2.1/servers/a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2" status: 200 len: 1542 microversion: 2.1 time: 0.359653 Jul 16 21:11:56.950262 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 534/1030] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:11:56 2022] GET /compute/v2.1/servers/a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2 => generated 1542 bytes in 366 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:11:56.956674 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-e6fd39ae-743d-4e6e-82ce-0704b7c85feb tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:11:56.957360 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-e6fd39ae-743d-4e6e-82ce-0704b7c85feb tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] [instance: 0fb5b5b6-1796-4406-b276-326e1d7dd451] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:11:56.960537 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_api:SELECT=96,nova_api:INSERT=3,nova_api:UPDATE=1 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:11:56.976473 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e6fd39ae-743d-4e6e-82ce-0704b7c85feb tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:56.977182 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e6fd39ae-743d-4e6e-82ce-0704b7c85feb tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:56.978152 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e6fd39ae-743d-4e6e-82ce-0704b7c85feb tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:56.978768 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e6fd39ae-743d-4e6e-82ce-0704b7c85feb tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:57.101821 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-42176561-013c-4fb7-a524-e933c9788b9a tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'abeb7020019d41a18de42956d706a011', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '649babe54d104436af461f1ee4767001', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:11:57.110976 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-42176561-013c-4fb7-a524-e933c9788b9a tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] 10.222.0.16 "GET /compute/v2.1/servers/b941b324-a54d-446f-bf92-570a80ba2152" status: 200 len: 1472 microversion: 2.1 time: 0.452118 Jul 16 21:11:57.112002 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 497/1031] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:11:56 2022] GET /compute/v2.1/servers/b941b324-a54d-446f-bf92-570a80ba2152 => generated 1472 bytes in 454 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:11:57.117464 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-28ee13b3-36d9-493d-a870-3c636658bcb7 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:11:57.118534 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-28ee13b3-36d9-493d-a870-3c636658bcb7 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] [instance: 8020f553-6025-4f19-835c-c00e2fb4f764] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:11:57.127679 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-28ee13b3-36d9-493d-a870-3c636658bcb7 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:57.128422 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-28ee13b3-36d9-493d-a870-3c636658bcb7 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:57.134824 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-28ee13b3-36d9-493d-a870-3c636658bcb7 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:57.134824 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-28ee13b3-36d9-493d-a870-3c636658bcb7 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:57.346978 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.objects.instance [None req-e6fd39ae-743d-4e6e-82ce-0704b7c85feb tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] Lazy-loading 'services' on Instance uuid 0fb5b5b6-1796-4406-b276-326e1d7dd451 {{(pid=138680) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:11:57.364713 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-e6fd39ae-743d-4e6e-82ce-0704b7c85feb tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/0fb5b5b6-1796-4406-b276-326e1d7dd451" status: 200 len: 2035 microversion: 2.25 time: 0.412961 Jul 16 21:11:57.366103 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 535/1032] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:11:56 2022] GET /compute/v2.1/servers/0fb5b5b6-1796-4406-b276-326e1d7dd451 => generated 2035 bytes in 415 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:11:57.374769 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-0eec755c-f11f-4b8f-b73f-8d36cf8fc998 tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:11:57.375783 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-0eec755c-f11f-4b8f-b73f-8d36cf8fc998 tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] [instance: 2b26717b-355f-417e-8a7c-72c41e1974e1] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:11:57.376844 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-28ee13b3-36d9-493d-a870-3c636658bcb7 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2dfcad8d2b57481bb33aa8f5d7483c51', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cda86413e74b4c1e8e09bd4df4926664', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:11:57.386933 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-28ee13b3-36d9-493d-a870-3c636658bcb7 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] 10.222.0.16 "GET /compute/v2.1/servers/8020f553-6025-4f19-835c-c00e2fb4f764" status: 200 len: 1323 microversion: 2.1 time: 0.273479 Jul 16 21:11:57.391938 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0eec755c-f11f-4b8f-b73f-8d36cf8fc998 tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:57.392074 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 498/1033] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:11:57 2022] GET /compute/v2.1/servers/8020f553-6025-4f19-835c-c00e2fb4f764 => generated 1323 bytes in 276 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:11:57.392164 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0eec755c-f11f-4b8f-b73f-8d36cf8fc998 tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:57.392164 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0eec755c-f11f-4b8f-b73f-8d36cf8fc998 tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:57.393719 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0eec755c-f11f-4b8f-b73f-8d36cf8fc998 tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:57.401910 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-d4be49b0-3f0f-4bd9-8322-ddb3009f8a78 tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] Action: 'action', calling method: >, body: {"os-migrateLive": {"host": "nch1-master-1", "block_migration": "auto"}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:11:57.403439 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-d4be49b0-3f0f-4bd9-8322-ddb3009f8a78 tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] [instance: 0fb5b5b6-1796-4406-b276-326e1d7dd451] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:11:57.413467 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d4be49b0-3f0f-4bd9-8322-ddb3009f8a78 tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:57.422930 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d4be49b0-3f0f-4bd9-8322-ddb3009f8a78 tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:57.422930 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d4be49b0-3f0f-4bd9-8322-ddb3009f8a78 tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:57.422930 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d4be49b0-3f0f-4bd9-8322-ddb3009f8a78 tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:57.449499 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.objects.instance [None req-d4be49b0-3f0f-4bd9-8322-ddb3009f8a78 tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] Lazy-loading 'flavor' on Instance uuid 0fb5b5b6-1796-4406-b276-326e1d7dd451 {{(pid=138681) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:11:57.462460 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-d4be49b0-3f0f-4bd9-8322-ddb3009f8a78 tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] [instance: 0fb5b5b6-1796-4406-b276-326e1d7dd451] Going to try to live migrate instance to nch1-master-1 {{(pid=138681) live_migrate /opt/stack/nova/nova/compute/api.py:5349}} Jul 16 21:11:57.634028 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-0eec755c-f11f-4b8f-b73f-8d36cf8fc998 tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '034c11cc3f174c6fb44c8d64e55cf8b9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb4ff5d7e42940cdb7be71dadd895ee4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:11:57.644656 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-0eec755c-f11f-4b8f-b73f-8d36cf8fc998 tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] 10.222.0.16 "GET /compute/v2.1/servers/2b26717b-355f-417e-8a7c-72c41e1974e1" status: 200 len: 1395 microversion: 2.1 time: 0.276936 Jul 16 21:11:57.645641 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 536/1034] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:11:57 2022] GET /compute/v2.1/servers/2b26717b-355f-417e-8a7c-72c41e1974e1 => generated 1395 bytes in 279 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:11:57.650862 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-8f7a2584-b393-485a-b725-b010b789195a req-f21bf57b-7866-459f-9ffd-e60edf7e0ff8 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "2b26717b-355f-417e-8a7c-72c41e1974e1", "tag": "c64dc016-0b59-43fd-8f3b-dcfc4298e2bd"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:11:57.664077 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-8f7a2584-b393-485a-b725-b010b789195a req-f21bf57b-7866-459f-9ffd-e60edf7e0ff8 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:57.664529 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-8f7a2584-b393-485a-b725-b010b789195a req-f21bf57b-7866-459f-9ffd-e60edf7e0ff8 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:57.676955 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-8f7a2584-b393-485a-b725-b010b789195a req-f21bf57b-7866-459f-9ffd-e60edf7e0ff8 service nova] Creating event network-changed:c64dc016-0b59-43fd-8f3b-dcfc4298e2bd for instance 2b26717b-355f-417e-8a7c-72c41e1974e1 on nch2-master-1 Jul 16 21:11:57.681916 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-8f7a2584-b393-485a-b725-b010b789195a req-f21bf57b-7866-459f-9ffd-e60edf7e0ff8 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.034768 Jul 16 21:11:57.683230 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 537/1035] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:11:57 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:11:57.926263 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-4f9d8854-5934-4a4e-8185-e2dd3cc551f1 req-e670a532-2a23-49fd-9a08-b515bf863170 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "b941b324-a54d-446f-bf92-570a80ba2152", "name": "network-vif-deleted", "tag": "dc03a8a7-1bc4-4e8d-91b2-65102ff61630"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:11:57.940172 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-4f9d8854-5934-4a4e-8185-e2dd3cc551f1 req-e670a532-2a23-49fd-9a08-b515bf863170 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:57.941104 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-4f9d8854-5934-4a4e-8185-e2dd3cc551f1 req-e670a532-2a23-49fd-9a08-b515bf863170 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:57.953244 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-4f9d8854-5934-4a4e-8185-e2dd3cc551f1 req-e670a532-2a23-49fd-9a08-b515bf863170 service nova] Creating event network-vif-deleted:dc03a8a7-1bc4-4e8d-91b2-65102ff61630 for instance b941b324-a54d-446f-bf92-570a80ba2152 on nch1-master-1 Jul 16 21:11:57.958666 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-4f9d8854-5934-4a4e-8185-e2dd3cc551f1 req-e670a532-2a23-49fd-9a08-b515bf863170 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.036436 Jul 16 21:11:57.959949 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 538/1036] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:11:57 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:11:57.966605 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-f0bf606b-cfd7-4124-89ff-63448d769443 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:11:57.967595 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-f0bf606b-cfd7-4124-89ff-63448d769443 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:11:57.989217 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f0bf606b-cfd7-4124-89ff-63448d769443 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:57.989758 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f0bf606b-cfd7-4124-89ff-63448d769443 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.015s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:57.990606 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f0bf606b-cfd7-4124-89ff-63448d769443 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:57.991091 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f0bf606b-cfd7-4124-89ff-63448d769443 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:58.204252 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-f0bf606b-cfd7-4124-89ff-63448d769443 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b68dacc4b8463ba07fae6372131117', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55a19b7a4682457a9132f5fc80110712', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:11:58.212892 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-f0bf606b-cfd7-4124-89ff-63448d769443 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "GET /compute/v2.1/servers/a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2" status: 200 len: 1542 microversion: 2.1 time: 0.251078 Jul 16 21:11:58.213524 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 539/1037] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:11:57 2022] GET /compute/v2.1/servers/a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2 => generated 1542 bytes in 253 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:11:58.231225 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-ca85951a-2a8a-41d5-b5d2-bbe61ec4a737 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:11:58.231920 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-ca85951a-2a8a-41d5-b5d2-bbe61ec4a737 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] [instance: b941b324-a54d-446f-bf92-570a80ba2152] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:11:58.238448 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ca85951a-2a8a-41d5-b5d2-bbe61ec4a737 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:58.239744 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ca85951a-2a8a-41d5-b5d2-bbe61ec4a737 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:58.240879 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ca85951a-2a8a-41d5-b5d2-bbe61ec4a737 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:58.241351 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ca85951a-2a8a-41d5-b5d2-bbe61ec4a737 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:58.372238 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-ca85951a-2a8a-41d5-b5d2-bbe61ec4a737 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'abeb7020019d41a18de42956d706a011', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '649babe54d104436af461f1ee4767001', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:11:58.384184 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-ca85951a-2a8a-41d5-b5d2-bbe61ec4a737 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] 10.222.0.16 "GET /compute/v2.1/servers/b941b324-a54d-446f-bf92-570a80ba2152" status: 200 len: 1302 microversion: 2.1 time: 0.162338 Jul 16 21:11:58.387234 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 540/1038] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:11:58 2022] GET /compute/v2.1/servers/b941b324-a54d-446f-bf92-570a80ba2152 => generated 1302 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:11:58.407767 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-10e4df0e-c804-4d2d-aed1-bf70fcecae21 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:11:58.408959 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-10e4df0e-c804-4d2d-aed1-bf70fcecae21 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] [instance: 8020f553-6025-4f19-835c-c00e2fb4f764] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:11:58.418579 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-10e4df0e-c804-4d2d-aed1-bf70fcecae21 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:58.419268 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-10e4df0e-c804-4d2d-aed1-bf70fcecae21 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:58.421099 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-10e4df0e-c804-4d2d-aed1-bf70fcecae21 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:58.421686 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-10e4df0e-c804-4d2d-aed1-bf70fcecae21 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:58.674784 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-10e4df0e-c804-4d2d-aed1-bf70fcecae21 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2dfcad8d2b57481bb33aa8f5d7483c51', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cda86413e74b4c1e8e09bd4df4926664', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:11:58.685759 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-10e4df0e-c804-4d2d-aed1-bf70fcecae21 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] 10.222.0.16 "GET /compute/v2.1/servers/8020f553-6025-4f19-835c-c00e2fb4f764" status: 200 len: 1323 microversion: 2.1 time: 0.282809 Jul 16 21:11:58.688455 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 541/1039] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:11:58 2022] GET /compute/v2.1/servers/8020f553-6025-4f19-835c-c00e2fb4f764 => generated 1323 bytes in 286 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:11:58.701303 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-4a7cac19-d171-48a6-9ba3-f1e3272771fc req-afe78c8c-32e3-454f-9b7d-4d74c0036369 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "8020f553-6025-4f19-835c-c00e2fb4f764", "name": "network-vif-plugged", "status": "completed", "tag": "05294e88-e71a-4faa-8acc-79137ca6ee05"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:11:58.712928 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-4a7cac19-d171-48a6-9ba3-f1e3272771fc req-afe78c8c-32e3-454f-9b7d-4d74c0036369 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:58.712928 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-4a7cac19-d171-48a6-9ba3-f1e3272771fc req-afe78c8c-32e3-454f-9b7d-4d74c0036369 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:58.728068 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-4a7cac19-d171-48a6-9ba3-f1e3272771fc req-afe78c8c-32e3-454f-9b7d-4d74c0036369 service nova] Creating event network-vif-plugged:05294e88-e71a-4faa-8acc-79137ca6ee05 for instance 8020f553-6025-4f19-835c-c00e2fb4f764 on nch1-master-1 Jul 16 21:11:58.733908 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-4a7cac19-d171-48a6-9ba3-f1e3272771fc req-afe78c8c-32e3-454f-9b7d-4d74c0036369 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.044167 Jul 16 21:11:58.734679 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 542/1040] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:11:58 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:11:58.740139 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-22ee0e76-94e0-4093-86d2-4ee7411119e6 tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:11:58.740842 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-22ee0e76-94e0-4093-86d2-4ee7411119e6 tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] [instance: 2b26717b-355f-417e-8a7c-72c41e1974e1] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:11:58.748091 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-22ee0e76-94e0-4093-86d2-4ee7411119e6 tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:58.748547 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-22ee0e76-94e0-4093-86d2-4ee7411119e6 tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:58.749327 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-22ee0e76-94e0-4093-86d2-4ee7411119e6 tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:58.749770 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-22ee0e76-94e0-4093-86d2-4ee7411119e6 tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:58.943283 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-22ee0e76-94e0-4093-86d2-4ee7411119e6 tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '034c11cc3f174c6fb44c8d64e55cf8b9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb4ff5d7e42940cdb7be71dadd895ee4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:11:58.945720 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:11:58.960700 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-22ee0e76-94e0-4093-86d2-4ee7411119e6 tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] 10.222.0.16 "GET /compute/v2.1/servers/2b26717b-355f-417e-8a7c-72c41e1974e1" status: 200 len: 1395 microversion: 2.1 time: 0.216864 Jul 16 21:11:58.960700 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 543/1041] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:11:58 2022] GET /compute/v2.1/servers/2b26717b-355f-417e-8a7c-72c41e1974e1 => generated 1395 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:11:59.230114 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-c2008db6-262b-491d-906a-461fbb37773c tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:11:59.230766 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-c2008db6-262b-491d-906a-461fbb37773c tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:11:59.236934 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c2008db6-262b-491d-906a-461fbb37773c tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:59.237386 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c2008db6-262b-491d-906a-461fbb37773c tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:59.238151 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c2008db6-262b-491d-906a-461fbb37773c tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:59.238638 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c2008db6-262b-491d-906a-461fbb37773c tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:59.393207 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-c2008db6-262b-491d-906a-461fbb37773c tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b68dacc4b8463ba07fae6372131117', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55a19b7a4682457a9132f5fc80110712', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:11:59.400019 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-c2008db6-262b-491d-906a-461fbb37773c tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "GET /compute/v2.1/servers/a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2" status: 200 len: 1542 microversion: 2.1 time: 0.174116 Jul 16 21:11:59.400636 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 544/1042] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:11:59 2022] GET /compute/v2.1/servers/a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2 => generated 1542 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:11:59.405425 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-d874b0f7-a7ac-4033-8272-ac62872e7ea6 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:11:59.406032 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-d874b0f7-a7ac-4033-8272-ac62872e7ea6 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] [instance: b941b324-a54d-446f-bf92-570a80ba2152] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:11:59.411914 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d874b0f7-a7ac-4033-8272-ac62872e7ea6 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:59.412366 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d874b0f7-a7ac-4033-8272-ac62872e7ea6 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:59.413135 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d874b0f7-a7ac-4033-8272-ac62872e7ea6 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:59.413575 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d874b0f7-a7ac-4033-8272-ac62872e7ea6 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:59.447353 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d874b0f7-a7ac-4033-8272-ac62872e7ea6 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:11:59.447866 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d874b0f7-a7ac-4033-8272-ac62872e7ea6 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:11:59.449134 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.objects.instance [None req-d874b0f7-a7ac-4033-8272-ac62872e7ea6 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lazy-loading 'fault' on Instance uuid b941b324-a54d-446f-bf92-570a80ba2152 {{(pid=138680) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:11:59.501844 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-d874b0f7-a7ac-4033-8272-ac62872e7ea6 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'abeb7020019d41a18de42956d706a011', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '649babe54d104436af461f1ee4767001', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:11:59.511091 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-d874b0f7-a7ac-4033-8272-ac62872e7ea6 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] 10.222.0.16 "GET /compute/v2.1/servers/b941b324-a54d-446f-bf92-570a80ba2152" status: 200 len: 1307 microversion: 2.1 time: 0.108688 Jul 16 21:11:59.511685 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 545/1043] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:11:59 2022] GET /compute/v2.1/servers/b941b324-a54d-446f-bf92-570a80ba2152 => generated 1307 bytes in 111 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:11:59.787764 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-ab27db11-0825-4b9e-8011-fa5ef9692050 tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-DeleteServersAdminTestJSON-server-150724934", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452"}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:11:59.846376 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.network.neutron [None req-ab27db11-0825-4b9e-8011-fa5ef9692050 tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] validate_networks() for None {{(pid=138680) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 16 21:12:00.425612 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-ab27db11-0825-4b9e-8011-fa5ef9692050 tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Flavor limits 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 16 21:12:00.426038 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-ab27db11-0825-4b9e-8011-fa5ef9692050 tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Image limits 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 16 21:12:00.426546 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-ab27db11-0825-4b9e-8011-fa5ef9692050 tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Flavor pref 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 16 21:12:00.427002 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-ab27db11-0825-4b9e-8011-fa5ef9692050 tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Image pref 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 16 21:12:00.427402 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-ab27db11-0825-4b9e-8011-fa5ef9692050 tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 16 21:12:00.437622 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ab27db11-0825-4b9e-8011-fa5ef9692050 tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:00.438054 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ab27db11-0825-4b9e-8011-fa5ef9692050 tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:00.438901 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ab27db11-0825-4b9e-8011-fa5ef9692050 tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:00.439368 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ab27db11-0825-4b9e-8011-fa5ef9692050 tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:00.458319 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-ab27db11-0825-4b9e-8011-fa5ef9692050 tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Getting quotas for project 7d2b583c48254e17a8082f6467f2ae6d. Resources: {'ram', 'instances', 'cores'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:12:00.464043 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-ab27db11-0825-4b9e-8011-fa5ef9692050 tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Getting quotas for user 56d4bbcdebc24a34a814d7f6e72fbcfd and project 7d2b583c48254e17a8082f6467f2ae6d. Resources: {'ram', 'instances', 'cores'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:12:00.476976 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-ab27db11-0825-4b9e-8011-fa5ef9692050 tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Going to run 1 instances... {{(pid=138680) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 16 21:12:00.479492 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-ab27db11-0825-4b9e-8011-fa5ef9692050 tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] [instance: 83b1fe77-ef6d-408d-b255-aa3b60aad8f3] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138680) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 16 21:12:00.687543 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-ab27db11-0825-4b9e-8011-fa5ef9692050 tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 1.033833 Jul 16 21:12:00.688229 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 546/1044] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 21:11:59 2022] POST /compute/v2.1/servers => generated 380 bytes in 1035 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Jul 16 21:12:00.693327 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-b4042dd9-15f2-491b-b380-97f8249575a8 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:00.694010 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-b4042dd9-15f2-491b-b380-97f8249575a8 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] [instance: 8020f553-6025-4f19-835c-c00e2fb4f764] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:12:00.701106 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b4042dd9-15f2-491b-b380-97f8249575a8 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:00.701562 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b4042dd9-15f2-491b-b380-97f8249575a8 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:00.702344 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b4042dd9-15f2-491b-b380-97f8249575a8 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:00.702807 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b4042dd9-15f2-491b-b380-97f8249575a8 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:00.928162 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-b4042dd9-15f2-491b-b380-97f8249575a8 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2dfcad8d2b57481bb33aa8f5d7483c51', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cda86413e74b4c1e8e09bd4df4926664', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:00.943057 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-b4042dd9-15f2-491b-b380-97f8249575a8 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] 10.222.0.16 "GET /compute/v2.1/servers/8020f553-6025-4f19-835c-c00e2fb4f764" status: 200 len: 1323 microversion: 2.1 time: 0.253695 Jul 16 21:12:00.955312 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 547/1045] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:12:00 2022] GET /compute/v2.1/servers/8020f553-6025-4f19-835c-c00e2fb4f764 => generated 1323 bytes in 265 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:12:00.960148 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-72234a11-caa4-4b34-b19f-4cf4f0c1daa0 tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:00.960767 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-72234a11-caa4-4b34-b19f-4cf4f0c1daa0 tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] [instance: 2b26717b-355f-417e-8a7c-72c41e1974e1] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:12:00.972466 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-72234a11-caa4-4b34-b19f-4cf4f0c1daa0 tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:00.972841 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-72234a11-caa4-4b34-b19f-4cf4f0c1daa0 tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:00.973776 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-72234a11-caa4-4b34-b19f-4cf4f0c1daa0 tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:00.974220 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-72234a11-caa4-4b34-b19f-4cf4f0c1daa0 tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:01.172762 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-72234a11-caa4-4b34-b19f-4cf4f0c1daa0 tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '034c11cc3f174c6fb44c8d64e55cf8b9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb4ff5d7e42940cdb7be71dadd895ee4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:01.183354 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-72234a11-caa4-4b34-b19f-4cf4f0c1daa0 tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] 10.222.0.16 "GET /compute/v2.1/servers/2b26717b-355f-417e-8a7c-72c41e1974e1" status: 200 len: 1395 microversion: 2.1 time: 0.226875 Jul 16 21:12:01.184231 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 548/1046] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:12:00 2022] GET /compute/v2.1/servers/2b26717b-355f-417e-8a7c-72c41e1974e1 => generated 1395 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:12:01.191469 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-b3646b3f-3f67-480c-9f12-e829c9de4e35 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:01.192260 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-b3646b3f-3f67-480c-9f12-e829c9de4e35 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:12:01.200276 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b3646b3f-3f67-480c-9f12-e829c9de4e35 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:01.200739 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b3646b3f-3f67-480c-9f12-e829c9de4e35 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:01.201534 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b3646b3f-3f67-480c-9f12-e829c9de4e35 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:01.201977 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b3646b3f-3f67-480c-9f12-e829c9de4e35 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:01.281286 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell0:SELECT=2 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:12:01.352168 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-b3646b3f-3f67-480c-9f12-e829c9de4e35 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b68dacc4b8463ba07fae6372131117', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55a19b7a4682457a9132f5fc80110712', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:01.359944 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-b3646b3f-3f67-480c-9f12-e829c9de4e35 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "GET /compute/v2.1/servers/a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2" status: 200 len: 1541 microversion: 2.1 time: 0.174623 Jul 16 21:12:01.360524 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 549/1047] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:12:01 2022] GET /compute/v2.1/servers/a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2 => generated 1541 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:12:01.365184 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-670db2d6-82f9-49ee-9af5-71fe7d66912e tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:01.365897 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-670db2d6-82f9-49ee-9af5-71fe7d66912e tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] [instance: b941b324-a54d-446f-bf92-570a80ba2152] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:12:01.373012 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-670db2d6-82f9-49ee-9af5-71fe7d66912e tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:01.373471 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-670db2d6-82f9-49ee-9af5-71fe7d66912e tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:01.374251 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-670db2d6-82f9-49ee-9af5-71fe7d66912e tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:01.374707 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-670db2d6-82f9-49ee-9af5-71fe7d66912e tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:01.387903 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-d4be49b0-3f0f-4bd9-8322-ddb3009f8a78 tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] 10.222.0.16 "POST /compute/v2.1/servers/0fb5b5b6-1796-4406-b276-326e1d7dd451/action" status: 202 len: 0 microversion: 2.25 time: 3.997737 Jul 16 21:12:01.392926 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-670db2d6-82f9-49ee-9af5-71fe7d66912e tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] HTTP exception thrown: Instance b941b324-a54d-446f-bf92-570a80ba2152 could not be found. Jul 16 21:12:01.396002 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 499/1048] 10.222.0.16 () {62 vars in 1318 bytes} [Sat Jul 16 21:11:57 2022] POST /compute/v2.1/servers/0fb5b5b6-1796-4406-b276-326e1d7dd451/action => generated 0 bytes in 4003 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Jul 16 21:12:01.396142 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-670db2d6-82f9-49ee-9af5-71fe7d66912e tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Request method failure captured: Jul 16 21:12:01.396142 ncdv-master-1 devstack@n-api.service[138680]: request: GET /compute/v2.1/servers/b941b324-a54d-446f-bf92-570a80ba2152 HTTP/1.1 Jul 16 21:12:01.396142 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:12:01.396142 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:12:01.396142 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:12:01.396142 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 0 Jul 16 21:12:01.396142 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:12:01.396142 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:12:01.396142 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:12:01.396142 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:12:01.396142 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0yjvVgP0jq68r-ny5Eh1zEbXMYJhEkVpBaJLnFwKzfSzzxAGgQPy49cpkkoW8Rc5r0epWFa_INrkbRxHc5yb1M5XgG50--wLBrLRBZrPuxqH4ynry56OcrGI4F8X9HPf6z5uwbMiaMLyGnsa6t3q-Npn98NzTN2KKNq9Fp4f3zZbeyY Jul 16 21:12:01.396142 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:12:01.396142 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:12:01.396142 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:12:01.396142 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:12:01.396142 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:12:01.396142 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:12:01.396142 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: 649babe54d104436af461f1ee4767001 Jul 16 21:12:01.396142 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-ServersOnMultiNodesTest-1258903790 Jul 16 21:12:01.396142 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:12:01.396142 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:12:01.397645 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/649babe54d104436af461f1ee4767001"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/649babe54d104436af461f1ee4767001"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_649babe54d104436af461f1ee4767001"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/649babe54d104436af461f1ee4767001"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:12:01.397645 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-ServersOnMultiNodesTest-1258903790 Jul 16 21:12:01.397645 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: 649babe54d104436af461f1ee4767001 Jul 16 21:12:01.397645 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-ServersOnMultiNodesTest-1258903790 Jul 16 21:12:01.397645 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-ServersOnMultiNodesTest-1258903790-project Jul 16 21:12:01.397645 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:12:01.397645 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:12:01.397645 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: abeb7020019d41a18de42956d706a011 Jul 16 21:12:01.397645 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-ServersOnMultiNodesTest-1258903790-project Jul 16 21:12:01.397645 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 21:12:01.397645 ncdv-master-1 devstack@n-api.service[138680]: exception: Instance b941b324-a54d-446f-bf92-570a80ba2152 could not be found. Jul 16 21:12:01.397645 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': 'b941b324-a54d-446f-bf92-570a80ba2152'} Jul 16 21:12:01.399129 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:12:01.399129 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:12:01.399129 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:12:01.399129 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:12:01.399129 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:12:01.399129 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:12:01.399129 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:12:01.399129 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:12:01.399129 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:12:01.399129 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:12:01.399129 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance b941b324-a54d-446f-bf92-570a80ba2152 could not be found. Jul 16 21:12:01.399129 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:12:01.399129 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:12:01.399129 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:12:01.399129 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:12:01.399129 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:12:01.399129 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:12:01.399129 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:12:01.399129 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:12:01.399129 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:12:01.400578 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:12:01.400578 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:12:01.400578 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:12:01.400578 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:12:01.400578 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:12:01.400578 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:12:01.400578 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:12:01.400578 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance b941b324-a54d-446f-bf92-570a80ba2152 could not be found. Jul 16 21:12:01.400578 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:12:01.400578 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:12:01.400578 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:12:01.400578 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:12:01.400578 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:12:01.400578 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:12:01.400578 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:12:01.400578 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:12:01.400578 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance b941b324-a54d-446f-bf92-570a80ba2152 could not be found. Jul 16 21:12:01.400578 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:12:01.402530 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-670db2d6-82f9-49ee-9af5-71fe7d66912e tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Returning 404 to user: Instance b941b324-a54d-446f-bf92-570a80ba2152 could not be found. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:12:01.402530 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-670db2d6-82f9-49ee-9af5-71fe7d66912e tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] 10.222.0.16 "GET /compute/v2.1/servers/b941b324-a54d-446f-bf92-570a80ba2152" status: 404 len: 111 microversion: 2.1 time: 0.033343 Jul 16 21:12:01.402530 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 550/1049] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:12:01 2022] GET /compute/v2.1/servers/b941b324-a54d-446f-bf92-570a80ba2152 => generated 111 bytes in 35 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:12:01.404252 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-98fd2935-3003-4d48-a90d-123f2fba14ab tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:01.405281 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-3fc31ea7-9b91-42a3-9bb8-e37bf45e9cb7 req-9ef5532e-cab0-4a34-b698-395f6b307d4d service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2", "name": "network-vif-unplugged", "status": "completed", "tag": "bf9d54d7-5718-4678-bc2f-5ba8241bd855"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:12:01.414461 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-3fc31ea7-9b91-42a3-9bb8-e37bf45e9cb7 req-9ef5532e-cab0-4a34-b698-395f6b307d4d service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:01.415059 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-3fc31ea7-9b91-42a3-9bb8-e37bf45e9cb7 req-9ef5532e-cab0-4a34-b698-395f6b307d4d service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:01.420028 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-98fd2935-3003-4d48-a90d-123f2fba14ab tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] [instance: 83b1fe77-ef6d-408d-b255-aa3b60aad8f3] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:12:01.428838 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-3fc31ea7-9b91-42a3-9bb8-e37bf45e9cb7 req-9ef5532e-cab0-4a34-b698-395f6b307d4d service nova] Creating event network-vif-unplugged:bf9d54d7-5718-4678-bc2f-5ba8241bd855 for instance a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2 on nch2-master-1 Jul 16 21:12:01.435328 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [req-3fc31ea7-9b91-42a3-9bb8-e37bf45e9cb7 req-9ef5532e-cab0-4a34-b698-395f6b307d4d service nova] Instance a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2 is migrating within the same cell, copying events to all relevant hosts: {'nch2-master-1', 'nch1-master-1'} {{(pid=138680) _get_relevant_hosts /opt/stack/nova/nova/compute/api.py:5807}} Jul 16 21:12:01.446274 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-3fc31ea7-9b91-42a3-9bb8-e37bf45e9cb7 req-9ef5532e-cab0-4a34-b698-395f6b307d4d service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.048969 Jul 16 21:12:01.448755 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 551/1050] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:12:01 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:12:01.452726 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-d190c6ee-34d9-4652-bcee-45c29e5f82c8 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:01.453430 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-d190c6ee-34d9-4652-bcee-45c29e5f82c8 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] [instance: 0fb5b5b6-1796-4406-b276-326e1d7dd451] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:12:01.460452 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d190c6ee-34d9-4652-bcee-45c29e5f82c8 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:01.460911 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d190c6ee-34d9-4652-bcee-45c29e5f82c8 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:01.461696 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d190c6ee-34d9-4652-bcee-45c29e5f82c8 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:01.462147 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d190c6ee-34d9-4652-bcee-45c29e5f82c8 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:01.480553 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-98fd2935-3003-4d48-a90d-123f2fba14ab tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56d4bbcdebc24a34a814d7f6e72fbcfd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7d2b583c48254e17a8082f6467f2ae6d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:01.488340 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-98fd2935-3003-4d48-a90d-123f2fba14ab tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] 10.222.0.16 "GET /compute/v2.1/servers/83b1fe77-ef6d-408d-b255-aa3b60aad8f3" status: 200 len: 1219 microversion: 2.1 time: 0.094015 Jul 16 21:12:01.488853 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 500/1051] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:12:01 2022] GET /compute/v2.1/servers/83b1fe77-ef6d-408d-b255-aa3b60aad8f3 => generated 1219 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:12:01.493623 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-9af8a715-5a8e-44f3-9729-863cd71518f2 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:01.494133 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-9af8a715-5a8e-44f3-9729-863cd71518f2 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] [instance: 84ee8179-ff94-43b3-97f2-1867bc6420a8] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:12:01.502746 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9af8a715-5a8e-44f3-9729-863cd71518f2 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:01.503478 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9af8a715-5a8e-44f3-9729-863cd71518f2 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:01.504459 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9af8a715-5a8e-44f3-9729-863cd71518f2 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:01.505177 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9af8a715-5a8e-44f3-9729-863cd71518f2 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:01.534836 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-9af8a715-5a8e-44f3-9729-863cd71518f2 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] [instance: 84ee8179-ff94-43b3-97f2-1867bc6420a8] Going to try to terminate instance {{(pid=138681) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 16 21:12:01.651950 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-d190c6ee-34d9-4652-bcee-45c29e5f82c8 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:01.660172 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-d190c6ee-34d9-4652-bcee-45c29e5f82c8 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:01.662440 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-d190c6ee-34d9-4652-bcee-45c29e5f82c8 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:01.664865 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-d190c6ee-34d9-4652-bcee-45c29e5f82c8 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] 10.222.0.16 "GET /compute/v2.1/servers/0fb5b5b6-1796-4406-b276-326e1d7dd451" status: 200 len: 1567 microversion: 2.25 time: 0.216845 Jul 16 21:12:01.666077 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 552/1052] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:12:01 2022] GET /compute/v2.1/servers/0fb5b5b6-1796-4406-b276-326e1d7dd451 => generated 1567 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:12:01.934834 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-9af8a715-5a8e-44f3-9729-863cd71518f2 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] 10.222.0.16 "DELETE /compute/v2.1/servers/84ee8179-ff94-43b3-97f2-1867bc6420a8" status: 204 len: 0 microversion: 2.1 time: 0.444134 Jul 16 21:12:01.935997 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 501/1053] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:12:01 2022] DELETE /compute/v2.1/servers/84ee8179-ff94-43b3-97f2-1867bc6420a8 => generated 0 bytes in 446 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 16 21:12:01.944249 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-f570af96-a6a1-4b0d-862e-3e82b6565fdc tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:01.944842 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-f570af96-a6a1-4b0d-862e-3e82b6565fdc tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] [instance: 7d624313-0c12-4da2-a95c-3903efae7608] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:12:01.953804 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f570af96-a6a1-4b0d-862e-3e82b6565fdc tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:01.954565 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f570af96-a6a1-4b0d-862e-3e82b6565fdc tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:01.955690 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f570af96-a6a1-4b0d-862e-3e82b6565fdc tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:01.956067 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f570af96-a6a1-4b0d-862e-3e82b6565fdc tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:01.971900 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-d9b2a7d8-3bfb-4bcd-af19-d3c0ecb151eb tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:01.972603 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-d9b2a7d8-3bfb-4bcd-af19-d3c0ecb151eb tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] [instance: 8020f553-6025-4f19-835c-c00e2fb4f764] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:12:01.993238 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d9b2a7d8-3bfb-4bcd-af19-d3c0ecb151eb tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:01.993694 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d9b2a7d8-3bfb-4bcd-af19-d3c0ecb151eb tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:01.995375 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-f570af96-a6a1-4b0d-862e-3e82b6565fdc tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] [instance: 7d624313-0c12-4da2-a95c-3903efae7608] Going to try to terminate instance {{(pid=138680) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 16 21:12:01.996044 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d9b2a7d8-3bfb-4bcd-af19-d3c0ecb151eb tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:01.996704 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d9b2a7d8-3bfb-4bcd-af19-d3c0ecb151eb tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:02.268161 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-d9b2a7d8-3bfb-4bcd-af19-d3c0ecb151eb tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2dfcad8d2b57481bb33aa8f5d7483c51', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cda86413e74b4c1e8e09bd4df4926664', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:02.270609 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=93,nova_cell1:UPDATE=4,nova_cell1:INSERT=3 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:12:02.277741 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-d9b2a7d8-3bfb-4bcd-af19-d3c0ecb151eb tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] 10.222.0.16 "GET /compute/v2.1/servers/8020f553-6025-4f19-835c-c00e2fb4f764" status: 200 len: 1519 microversion: 2.1 time: 0.310513 Jul 16 21:12:02.283567 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 502/1054] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:12:01 2022] GET /compute/v2.1/servers/8020f553-6025-4f19-835c-c00e2fb4f764 => generated 1519 bytes in 318 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:12:02.290959 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-c4e53656-b3f1-432f-b4d1-bba5728525c7 tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:02.292566 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-c4e53656-b3f1-432f-b4d1-bba5728525c7 tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] [instance: 2b26717b-355f-417e-8a7c-72c41e1974e1] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:12:02.300938 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c4e53656-b3f1-432f-b4d1-bba5728525c7 tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:02.301395 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c4e53656-b3f1-432f-b4d1-bba5728525c7 tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:02.302813 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c4e53656-b3f1-432f-b4d1-bba5728525c7 tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:02.304220 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c4e53656-b3f1-432f-b4d1-bba5728525c7 tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:02.311384 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-f570af96-a6a1-4b0d-862e-3e82b6565fdc tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] 10.222.0.16 "DELETE /compute/v2.1/servers/7d624313-0c12-4da2-a95c-3903efae7608" status: 204 len: 0 microversion: 2.1 time: 0.370857 Jul 16 21:12:02.316988 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 553/1055] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:12:01 2022] DELETE /compute/v2.1/servers/7d624313-0c12-4da2-a95c-3903efae7608 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 16 21:12:02.318148 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-ed6f16c2-5d5d-4319-b64c-b88c740f80f5 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Action: 'create', calling method: .version_select at 0x7fdd9d1598b0>, body: {"pool": null} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:12:02.626573 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-c4e53656-b3f1-432f-b4d1-bba5728525c7 tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '034c11cc3f174c6fb44c8d64e55cf8b9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb4ff5d7e42940cdb7be71dadd895ee4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:02.633666 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-c4e53656-b3f1-432f-b4d1-bba5728525c7 tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] 10.222.0.16 "GET /compute/v2.1/servers/2b26717b-355f-417e-8a7c-72c41e1974e1" status: 200 len: 1395 microversion: 2.1 time: 0.348066 Jul 16 21:12:02.634237 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 503/1056] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:12:02 2022] GET /compute/v2.1/servers/2b26717b-355f-417e-8a7c-72c41e1974e1 => generated 1395 bytes in 350 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:12:02.644002 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-4f151364-4dda-4147-9c59-489b4cc3b54d tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:02.644002 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-4f151364-4dda-4147-9c59-489b4cc3b54d tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] [instance: 84ee8179-ff94-43b3-97f2-1867bc6420a8] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:12:02.646835 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4f151364-4dda-4147-9c59-489b4cc3b54d tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:02.647346 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4f151364-4dda-4147-9c59-489b4cc3b54d tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:02.654376 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4f151364-4dda-4147-9c59-489b4cc3b54d tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:02.654376 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4f151364-4dda-4147-9c59-489b4cc3b54d tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:02.841025 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-4f151364-4dda-4147-9c59-489b4cc3b54d tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'abeb7020019d41a18de42956d706a011', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '649babe54d104436af461f1ee4767001', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:02.848798 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-4f151364-4dda-4147-9c59-489b4cc3b54d tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] 10.222.0.16 "GET /compute/v2.1/servers/84ee8179-ff94-43b3-97f2-1867bc6420a8" status: 200 len: 1474 microversion: 2.1 time: 0.213523 Jul 16 21:12:02.849382 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 504/1057] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:12:02 2022] GET /compute/v2.1/servers/84ee8179-ff94-43b3-97f2-1867bc6420a8 => generated 1474 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:12:02.860160 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-f92fde05-813c-433c-adc9-66cf97d27db1 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:02.861463 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-f92fde05-813c-433c-adc9-66cf97d27db1 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:12:02.867856 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f92fde05-813c-433c-adc9-66cf97d27db1 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:02.868315 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f92fde05-813c-433c-adc9-66cf97d27db1 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:02.869097 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f92fde05-813c-433c-adc9-66cf97d27db1 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:02.870126 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f92fde05-813c-433c-adc9-66cf97d27db1 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:03.095552 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-f92fde05-813c-433c-adc9-66cf97d27db1 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b68dacc4b8463ba07fae6372131117', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55a19b7a4682457a9132f5fc80110712', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:03.105774 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-f92fde05-813c-433c-adc9-66cf97d27db1 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "GET /compute/v2.1/servers/a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2" status: 200 len: 1541 microversion: 2.1 time: 0.255094 Jul 16 21:12:03.106501 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 505/1058] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:12:02 2022] GET /compute/v2.1/servers/a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2 => generated 1541 bytes in 257 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:12:03.111475 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-c1e2c6a6-5f13-4983-92dc-91114f3defad tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:03.112233 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-c1e2c6a6-5f13-4983-92dc-91114f3defad tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] [instance: 83b1fe77-ef6d-408d-b255-aa3b60aad8f3] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:12:03.119300 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c1e2c6a6-5f13-4983-92dc-91114f3defad tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:03.120388 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c1e2c6a6-5f13-4983-92dc-91114f3defad tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:03.121779 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c1e2c6a6-5f13-4983-92dc-91114f3defad tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:03.122223 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c1e2c6a6-5f13-4983-92dc-91114f3defad tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:03.199107 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-c1e2c6a6-5f13-4983-92dc-91114f3defad tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56d4bbcdebc24a34a814d7f6e72fbcfd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7d2b583c48254e17a8082f6467f2ae6d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:03.206372 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-c1e2c6a6-5f13-4983-92dc-91114f3defad tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] 10.222.0.16 "GET /compute/v2.1/servers/83b1fe77-ef6d-408d-b255-aa3b60aad8f3" status: 200 len: 1215 microversion: 2.1 time: 0.098639 Jul 16 21:12:03.217524 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 506/1059] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:12:03 2022] GET /compute/v2.1/servers/83b1fe77-ef6d-408d-b255-aa3b60aad8f3 => generated 1215 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:12:03.217524 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-42798812-8e1c-4cb6-996d-39a16ff8fdcb tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:03.217524 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-42798812-8e1c-4cb6-996d-39a16ff8fdcb tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] [instance: 0fb5b5b6-1796-4406-b276-326e1d7dd451] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:12:03.223733 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-42798812-8e1c-4cb6-996d-39a16ff8fdcb tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:03.224218 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-42798812-8e1c-4cb6-996d-39a16ff8fdcb tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:03.224996 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-42798812-8e1c-4cb6-996d-39a16ff8fdcb tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:03.225427 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-42798812-8e1c-4cb6-996d-39a16ff8fdcb tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:03.482335 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-42798812-8e1c-4cb6-996d-39a16ff8fdcb tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:03.492752 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-42798812-8e1c-4cb6-996d-39a16ff8fdcb tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:03.494242 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-42798812-8e1c-4cb6-996d-39a16ff8fdcb tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:03.496143 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-42798812-8e1c-4cb6-996d-39a16ff8fdcb tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] 10.222.0.16 "GET /compute/v2.1/servers/0fb5b5b6-1796-4406-b276-326e1d7dd451" status: 200 len: 1567 microversion: 2.25 time: 0.288075 Jul 16 21:12:03.496695 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 507/1060] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:12:03 2022] GET /compute/v2.1/servers/0fb5b5b6-1796-4406-b276-326e1d7dd451 => generated 1567 bytes in 289 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:12:03.501890 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-6d853296-b86a-46bd-8b63-e0f667e7e791 req-ce467cac-8ef0-432d-aa7c-bb2726d99413 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2", "tag": "bf9d54d7-5718-4678-bc2f-5ba8241bd855"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:12:03.513752 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-6d853296-b86a-46bd-8b63-e0f667e7e791 req-ce467cac-8ef0-432d-aa7c-bb2726d99413 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:03.514509 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-6d853296-b86a-46bd-8b63-e0f667e7e791 req-ce467cac-8ef0-432d-aa7c-bb2726d99413 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:03.528450 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-6d853296-b86a-46bd-8b63-e0f667e7e791 req-ce467cac-8ef0-432d-aa7c-bb2726d99413 service nova] Creating event network-changed:bf9d54d7-5718-4678-bc2f-5ba8241bd855 for instance a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2 on nch2-master-1 Jul 16 21:12:03.535805 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [req-6d853296-b86a-46bd-8b63-e0f667e7e791 req-ce467cac-8ef0-432d-aa7c-bb2726d99413 service nova] Instance a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2 is migrating within the same cell, copying events to all relevant hosts: {'nch2-master-1', 'nch1-master-1'} {{(pid=138681) _get_relevant_hosts /opt/stack/nova/nova/compute/api.py:5807}} Jul 16 21:12:03.547649 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-6d853296-b86a-46bd-8b63-e0f667e7e791 req-ce467cac-8ef0-432d-aa7c-bb2726d99413 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.049583 Jul 16 21:12:03.548639 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 508/1061] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:12:03 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:12:03.655203 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-4e846877-a397-41bd-a5f9-f053f6d7e6ca tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:03.656474 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-4e846877-a397-41bd-a5f9-f053f6d7e6ca tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] [instance: 2b26717b-355f-417e-8a7c-72c41e1974e1] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:12:03.689922 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4e846877-a397-41bd-a5f9-f053f6d7e6ca tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:03.690872 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4e846877-a397-41bd-a5f9-f053f6d7e6ca tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:03.692426 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4e846877-a397-41bd-a5f9-f053f6d7e6ca tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:03.693432 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4e846877-a397-41bd-a5f9-f053f6d7e6ca tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:04.203421 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-4e846877-a397-41bd-a5f9-f053f6d7e6ca tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '034c11cc3f174c6fb44c8d64e55cf8b9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb4ff5d7e42940cdb7be71dadd895ee4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:04.216661 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-4e846877-a397-41bd-a5f9-f053f6d7e6ca tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] 10.222.0.16 "GET /compute/v2.1/servers/2b26717b-355f-417e-8a7c-72c41e1974e1" status: 200 len: 1395 microversion: 2.1 time: 0.568660 Jul 16 21:12:04.218508 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 509/1062] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:12:03 2022] GET /compute/v2.1/servers/2b26717b-355f-417e-8a7c-72c41e1974e1 => generated 1395 bytes in 571 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:12:04.224977 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-f29e2f69-6ab5-4e45-847f-c0011ad982aa tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:04.226748 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-f29e2f69-6ab5-4e45-847f-c0011ad982aa tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] [instance: 84ee8179-ff94-43b3-97f2-1867bc6420a8] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:12:04.235294 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f29e2f69-6ab5-4e45-847f-c0011ad982aa tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:04.236778 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f29e2f69-6ab5-4e45-847f-c0011ad982aa tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:04.237943 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f29e2f69-6ab5-4e45-847f-c0011ad982aa tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:04.238815 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f29e2f69-6ab5-4e45-847f-c0011ad982aa tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:05.133927 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-f29e2f69-6ab5-4e45-847f-c0011ad982aa tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'abeb7020019d41a18de42956d706a011', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '649babe54d104436af461f1ee4767001', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:05.142604 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-f29e2f69-6ab5-4e45-847f-c0011ad982aa tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] 10.222.0.16 "GET /compute/v2.1/servers/84ee8179-ff94-43b3-97f2-1867bc6420a8" status: 200 len: 1474 microversion: 2.1 time: 0.922747 Jul 16 21:12:05.143301 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 510/1063] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:12:04 2022] GET /compute/v2.1/servers/84ee8179-ff94-43b3-97f2-1867bc6420a8 => generated 1474 bytes in 924 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:12:05.156806 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-878be9c7-7fa3-46c7-8a18-c66d87b60eca tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:05.157416 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-878be9c7-7fa3-46c7-8a18-c66d87b60eca tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:12:05.163424 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-878be9c7-7fa3-46c7-8a18-c66d87b60eca tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:05.163906 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-878be9c7-7fa3-46c7-8a18-c66d87b60eca tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:05.164747 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-878be9c7-7fa3-46c7-8a18-c66d87b60eca tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:05.165260 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-878be9c7-7fa3-46c7-8a18-c66d87b60eca tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:05.769756 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-878be9c7-7fa3-46c7-8a18-c66d87b60eca tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b68dacc4b8463ba07fae6372131117', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55a19b7a4682457a9132f5fc80110712', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:05.780550 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-878be9c7-7fa3-46c7-8a18-c66d87b60eca tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "GET /compute/v2.1/servers/a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2" status: 200 len: 1473 microversion: 2.1 time: 0.627360 Jul 16 21:12:05.784145 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 511/1064] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:12:05 2022] GET /compute/v2.1/servers/a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2 => generated 1473 bytes in 637 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:12:05.787374 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-5813deb7-9a46-4abe-a645-a3fac8995a1f tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:05.788193 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-5813deb7-9a46-4abe-a645-a3fac8995a1f tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] [instance: 83b1fe77-ef6d-408d-b255-aa3b60aad8f3] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:12:05.795988 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5813deb7-9a46-4abe-a645-a3fac8995a1f tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:05.796730 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5813deb7-9a46-4abe-a645-a3fac8995a1f tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:05.797806 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5813deb7-9a46-4abe-a645-a3fac8995a1f tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:05.798530 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5813deb7-9a46-4abe-a645-a3fac8995a1f tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:05.836266 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-ed6f16c2-5d5d-4319-b64c-b88c740f80f5 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] 10.222.0.16 "POST /compute/v2.1/os-floating-ips" status: 200 len: 142 microversion: 2.1 time: 3.522112 Jul 16 21:12:05.851923 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 554/1065] 10.222.0.16 () {60 vars in 1169 bytes} [Sat Jul 16 21:12:02 2022] POST /compute/v2.1/os-floating-ips => generated 142 bytes in 3527 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:12:05.851923 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-81b6d1b8-3c69-4132-8389-571211077dad req-f20af3f4-c142-46b2-b1a4-f3d555267cec service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "7d624313-0c12-4da2-a95c-3903efae7608", "name": "network-vif-unplugged", "status": "completed", "tag": "186c161e-a1dc-4086-9efc-737d7b982c1b"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:12:05.873136 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-81b6d1b8-3c69-4132-8389-571211077dad req-f20af3f4-c142-46b2-b1a4-f3d555267cec service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:05.874023 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-81b6d1b8-3c69-4132-8389-571211077dad req-f20af3f4-c142-46b2-b1a4-f3d555267cec service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:05.887156 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-81b6d1b8-3c69-4132-8389-571211077dad req-f20af3f4-c142-46b2-b1a4-f3d555267cec service nova] Creating event network-vif-unplugged:186c161e-a1dc-4086-9efc-737d7b982c1b for instance 7d624313-0c12-4da2-a95c-3903efae7608 on nch1-master-1 Jul 16 21:12:05.895549 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-81b6d1b8-3c69-4132-8389-571211077dad req-f20af3f4-c142-46b2-b1a4-f3d555267cec service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.054049 Jul 16 21:12:05.896295 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 555/1066] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:12:05 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:12:05.902770 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-6b1253b0-3358-4d40-9a3b-859e8cff51b0 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:05.903866 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-6b1253b0-3358-4d40-9a3b-859e8cff51b0 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] [instance: 0fb5b5b6-1796-4406-b276-326e1d7dd451] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:12:05.915625 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6b1253b0-3358-4d40-9a3b-859e8cff51b0 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:05.916962 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6b1253b0-3358-4d40-9a3b-859e8cff51b0 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:05.918320 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6b1253b0-3358-4d40-9a3b-859e8cff51b0 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:05.918971 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6b1253b0-3358-4d40-9a3b-859e8cff51b0 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:05.961500 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-5813deb7-9a46-4abe-a645-a3fac8995a1f tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56d4bbcdebc24a34a814d7f6e72fbcfd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7d2b583c48254e17a8082f6467f2ae6d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:05.973807 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-5813deb7-9a46-4abe-a645-a3fac8995a1f tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] 10.222.0.16 "GET /compute/v2.1/servers/83b1fe77-ef6d-408d-b255-aa3b60aad8f3" status: 200 len: 1277 microversion: 2.1 time: 0.190999 Jul 16 21:12:05.974585 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 512/1067] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:12:05 2022] GET /compute/v2.1/servers/83b1fe77-ef6d-408d-b255-aa3b60aad8f3 => generated 1277 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:12:05.979904 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-33301589-0e97-4496-804e-19c0d083a0ee tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:05.980633 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-33301589-0e97-4496-804e-19c0d083a0ee tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] [instance: 2b26717b-355f-417e-8a7c-72c41e1974e1] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:12:05.992119 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-33301589-0e97-4496-804e-19c0d083a0ee tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:05.992776 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-33301589-0e97-4496-804e-19c0d083a0ee tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:05.993740 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-33301589-0e97-4496-804e-19c0d083a0ee tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:05.995136 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-33301589-0e97-4496-804e-19c0d083a0ee tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:06.465814 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-6b1253b0-3358-4d40-9a3b-859e8cff51b0 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:06.478902 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-6b1253b0-3358-4d40-9a3b-859e8cff51b0 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:06.483685 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-6b1253b0-3358-4d40-9a3b-859e8cff51b0 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:06.485032 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-6b1253b0-3358-4d40-9a3b-859e8cff51b0 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] 10.222.0.16 "GET /compute/v2.1/servers/0fb5b5b6-1796-4406-b276-326e1d7dd451" status: 200 len: 1567 microversion: 2.25 time: 0.587391 Jul 16 21:12:06.488432 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 556/1068] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:12:05 2022] GET /compute/v2.1/servers/0fb5b5b6-1796-4406-b276-326e1d7dd451 => generated 1567 bytes in 591 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:12:06.498443 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-af4bcb2b-8cd9-4de0-a7d2-440c4b7a5664 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-FloatingIPsAssociationTestJSON-server-432233479", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452", "networks": [{"uuid": "9b8d8e4f-14e5-4e94-bfa5-2387fb1adcf8"}]}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:12:06.530002 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-33301589-0e97-4496-804e-19c0d083a0ee tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '034c11cc3f174c6fb44c8d64e55cf8b9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb4ff5d7e42940cdb7be71dadd895ee4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:06.540260 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-33301589-0e97-4496-804e-19c0d083a0ee tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] 10.222.0.16 "GET /compute/v2.1/servers/2b26717b-355f-417e-8a7c-72c41e1974e1" status: 200 len: 1395 microversion: 2.1 time: 0.564725 Jul 16 21:12:06.550176 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 513/1069] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:12:05 2022] GET /compute/v2.1/servers/2b26717b-355f-417e-8a7c-72c41e1974e1 => generated 1395 bytes in 575 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:12:06.563493 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-1ab1d1c1-a830-480e-9409-eae466c96793 req-4c4eff42-dd20-4153-a12c-eac5028dc808 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "84ee8179-ff94-43b3-97f2-1867bc6420a8", "name": "network-vif-deleted", "tag": "48ddda29-7627-4cbb-adf0-0975571cbc33"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:12:06.567744 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.network.neutron [None req-af4bcb2b-8cd9-4de0-a7d2-440c4b7a5664 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] validate_networks() for [('9b8d8e4f-14e5-4e94-bfa5-2387fb1adcf8', None, None, None, None, None)] {{(pid=138680) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 16 21:12:06.573391 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-1ab1d1c1-a830-480e-9409-eae466c96793 req-4c4eff42-dd20-4153-a12c-eac5028dc808 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:06.574188 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-1ab1d1c1-a830-480e-9409-eae466c96793 req-4c4eff42-dd20-4153-a12c-eac5028dc808 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:06.592415 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-1ab1d1c1-a830-480e-9409-eae466c96793 req-4c4eff42-dd20-4153-a12c-eac5028dc808 service nova] Creating event network-vif-deleted:48ddda29-7627-4cbb-adf0-0975571cbc33 for instance 84ee8179-ff94-43b3-97f2-1867bc6420a8 on nch2-master-1 Jul 16 21:12:06.605577 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-1ab1d1c1-a830-480e-9409-eae466c96793 req-4c4eff42-dd20-4153-a12c-eac5028dc808 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.053967 Jul 16 21:12:06.608060 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 514/1070] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:12:06 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:12:06.620596 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-475930f1-fa9b-40e6-9775-535162354457 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:06.623488 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-475930f1-fa9b-40e6-9775-535162354457 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] [instance: 84ee8179-ff94-43b3-97f2-1867bc6420a8] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:12:06.637823 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-475930f1-fa9b-40e6-9775-535162354457 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:06.638720 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-475930f1-fa9b-40e6-9775-535162354457 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:06.639868 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-475930f1-fa9b-40e6-9775-535162354457 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:06.643730 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-475930f1-fa9b-40e6-9775-535162354457 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:06.691642 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-475930f1-fa9b-40e6-9775-535162354457 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:06.692071 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-475930f1-fa9b-40e6-9775-535162354457 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:06.692541 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.objects.instance [None req-475930f1-fa9b-40e6-9775-535162354457 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lazy-loading 'fault' on Instance uuid 84ee8179-ff94-43b3-97f2-1867bc6420a8 {{(pid=138681) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:12:06.742142 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-475930f1-fa9b-40e6-9775-535162354457 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'abeb7020019d41a18de42956d706a011', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '649babe54d104436af461f1ee4767001', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:06.753645 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-475930f1-fa9b-40e6-9775-535162354457 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] 10.222.0.16 "GET /compute/v2.1/servers/84ee8179-ff94-43b3-97f2-1867bc6420a8" status: 200 len: 1308 microversion: 2.1 time: 0.141379 Jul 16 21:12:06.754960 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 515/1071] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:12:06 2022] GET /compute/v2.1/servers/84ee8179-ff94-43b3-97f2-1867bc6420a8 => generated 1308 bytes in 144 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:12:06.800278 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-79040175-7c52-461a-8c77-1e4726c01423 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:06.800918 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-79040175-7c52-461a-8c77-1e4726c01423 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:12:06.808135 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-79040175-7c52-461a-8c77-1e4726c01423 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:06.808977 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-79040175-7c52-461a-8c77-1e4726c01423 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:06.810708 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-79040175-7c52-461a-8c77-1e4726c01423 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:06.812980 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-79040175-7c52-461a-8c77-1e4726c01423 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:07.294319 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-79040175-7c52-461a-8c77-1e4726c01423 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b68dacc4b8463ba07fae6372131117', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55a19b7a4682457a9132f5fc80110712', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:07.302434 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-79040175-7c52-461a-8c77-1e4726c01423 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "GET /compute/v2.1/servers/a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2" status: 200 len: 1470 microversion: 2.1 time: 0.508006 Jul 16 21:12:07.303235 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 516/1072] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:12:06 2022] GET /compute/v2.1/servers/a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2 => generated 1470 bytes in 510 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:12:07.309115 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-55aab7bf-3d1e-4986-848a-fb57ca9c15b7 tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:07.309941 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-55aab7bf-3d1e-4986-848a-fb57ca9c15b7 tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] [instance: 83b1fe77-ef6d-408d-b255-aa3b60aad8f3] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:12:07.318894 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-55aab7bf-3d1e-4986-848a-fb57ca9c15b7 tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:07.319715 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-55aab7bf-3d1e-4986-848a-fb57ca9c15b7 tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:07.320480 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-55aab7bf-3d1e-4986-848a-fb57ca9c15b7 tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:07.320920 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-55aab7bf-3d1e-4986-848a-fb57ca9c15b7 tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:07.429157 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-55aab7bf-3d1e-4986-848a-fb57ca9c15b7 tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56d4bbcdebc24a34a814d7f6e72fbcfd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7d2b583c48254e17a8082f6467f2ae6d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:07.437812 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-55aab7bf-3d1e-4986-848a-fb57ca9c15b7 tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] 10.222.0.16 "GET /compute/v2.1/servers/83b1fe77-ef6d-408d-b255-aa3b60aad8f3" status: 200 len: 1277 microversion: 2.1 time: 0.133381 Jul 16 21:12:07.439048 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 517/1073] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:12:07 2022] GET /compute/v2.1/servers/83b1fe77-ef6d-408d-b255-aa3b60aad8f3 => generated 1277 bytes in 135 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:12:07.444084 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-2751c3d4-a6c7-4a9f-968f-31132f3c1706 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Action: 'action', calling method: >, body: {"revertResize": {}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:12:07.444791 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-2751c3d4-a6c7-4a9f-968f-31132f3c1706 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:12:07.451651 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2751c3d4-a6c7-4a9f-968f-31132f3c1706 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:07.455171 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2751c3d4-a6c7-4a9f-968f-31132f3c1706 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:07.455171 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2751c3d4-a6c7-4a9f-968f-31132f3c1706 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:07.455171 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2751c3d4-a6c7-4a9f-968f-31132f3c1706 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:07.490531 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_api:SELECT=95,nova_api:INSERT=3,nova_api:UPDATE=2 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:12:07.495320 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2751c3d4-a6c7-4a9f-968f-31132f3c1706 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:07.495690 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2751c3d4-a6c7-4a9f-968f-31132f3c1706 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:07.496697 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2751c3d4-a6c7-4a9f-968f-31132f3c1706 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:07.497121 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2751c3d4-a6c7-4a9f-968f-31132f3c1706 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:07.521043 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-2751c3d4-a6c7-4a9f-968f-31132f3c1706 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Getting quotas for project 55a19b7a4682457a9132f5fc80110712. Resources: {'ram', 'cores'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:12:07.526272 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-2751c3d4-a6c7-4a9f-968f-31132f3c1706 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Getting quotas for user a9b68dacc4b8463ba07fae6372131117 and project 55a19b7a4682457a9132f5fc80110712. Resources: {'ram', 'cores'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:12:07.534183 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=100 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:12:07.640996 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-af4bcb2b-8cd9-4de0-a7d2-440c4b7a5664 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Flavor limits 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 16 21:12:07.641433 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-af4bcb2b-8cd9-4de0-a7d2-440c4b7a5664 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Image limits 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 16 21:12:07.641914 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-af4bcb2b-8cd9-4de0-a7d2-440c4b7a5664 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Flavor pref 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 16 21:12:07.642328 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-af4bcb2b-8cd9-4de0-a7d2-440c4b7a5664 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Image pref 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 16 21:12:07.642770 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-af4bcb2b-8cd9-4de0-a7d2-440c4b7a5664 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 16 21:12:07.654012 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-af4bcb2b-8cd9-4de0-a7d2-440c4b7a5664 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:07.654469 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-af4bcb2b-8cd9-4de0-a7d2-440c4b7a5664 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:07.655485 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-af4bcb2b-8cd9-4de0-a7d2-440c4b7a5664 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:07.655952 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-af4bcb2b-8cd9-4de0-a7d2-440c4b7a5664 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:07.676427 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-af4bcb2b-8cd9-4de0-a7d2-440c4b7a5664 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Getting quotas for project cda86413e74b4c1e8e09bd4df4926664. Resources: {'ram', 'instances', 'cores'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:12:07.682719 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-af4bcb2b-8cd9-4de0-a7d2-440c4b7a5664 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Getting quotas for user 2dfcad8d2b57481bb33aa8f5d7483c51 and project cda86413e74b4c1e8e09bd4df4926664. Resources: {'ram', 'instances', 'cores'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:12:07.696124 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-af4bcb2b-8cd9-4de0-a7d2-440c4b7a5664 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Going to run 1 instances... {{(pid=138680) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 16 21:12:07.699339 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-af4bcb2b-8cd9-4de0-a7d2-440c4b7a5664 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] [instance: 4ae3e944-b450-40e5-bb71-c02df03212e3] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138680) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 16 21:12:07.840641 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-af4bcb2b-8cd9-4de0-a7d2-440c4b7a5664 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 1.350905 Jul 16 21:12:07.841775 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 557/1074] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 21:12:06 2022] POST /compute/v2.1/servers => generated 380 bytes in 1353 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 16 21:12:07.846701 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-4020e81d-4fc9-46c6-975e-ebba0fcb135a tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:07.847623 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-4020e81d-4fc9-46c6-975e-ebba0fcb135a tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] [instance: 0fb5b5b6-1796-4406-b276-326e1d7dd451] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:12:07.854772 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4020e81d-4fc9-46c6-975e-ebba0fcb135a tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:07.855371 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4020e81d-4fc9-46c6-975e-ebba0fcb135a tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:07.856519 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4020e81d-4fc9-46c6-975e-ebba0fcb135a tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:07.857041 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4020e81d-4fc9-46c6-975e-ebba0fcb135a tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:08.072561 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-2751c3d4-a6c7-4a9f-968f-31132f3c1706 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "POST /compute/v2.1/servers/a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2/action" status: 202 len: 0 microversion: 2.1 time: 0.631999 Jul 16 21:12:08.073555 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 518/1075] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 21:12:07 2022] POST /compute/v2.1/servers/a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2/action => generated 0 bytes in 634 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 21:12:08.081141 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-a419f068-2f3e-4cff-9079-cd607cb6dae2 tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:08.082213 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-a419f068-2f3e-4cff-9079-cd607cb6dae2 tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] [instance: 2b26717b-355f-417e-8a7c-72c41e1974e1] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:12:08.091140 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a419f068-2f3e-4cff-9079-cd607cb6dae2 tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:08.091843 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a419f068-2f3e-4cff-9079-cd607cb6dae2 tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:08.093754 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a419f068-2f3e-4cff-9079-cd607cb6dae2 tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:08.093754 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a419f068-2f3e-4cff-9079-cd607cb6dae2 tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:08.133784 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-4020e81d-4fc9-46c6-975e-ebba0fcb135a tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:08.147696 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-4020e81d-4fc9-46c6-975e-ebba0fcb135a tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:08.158957 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-4020e81d-4fc9-46c6-975e-ebba0fcb135a tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:08.160379 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-4020e81d-4fc9-46c6-975e-ebba0fcb135a tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] 10.222.0.16 "GET /compute/v2.1/servers/0fb5b5b6-1796-4406-b276-326e1d7dd451" status: 200 len: 1567 microversion: 2.25 time: 0.317626 Jul 16 21:12:08.161003 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 558/1076] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:12:07 2022] GET /compute/v2.1/servers/0fb5b5b6-1796-4406-b276-326e1d7dd451 => generated 1567 bytes in 319 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:12:08.165974 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-cae3ef85-0d5d-4296-b130-1c6bcc2fe401 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:08.167891 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-cae3ef85-0d5d-4296-b130-1c6bcc2fe401 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] [instance: 84ee8179-ff94-43b3-97f2-1867bc6420a8] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:12:08.180680 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cae3ef85-0d5d-4296-b130-1c6bcc2fe401 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:08.181233 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cae3ef85-0d5d-4296-b130-1c6bcc2fe401 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:08.181976 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cae3ef85-0d5d-4296-b130-1c6bcc2fe401 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:08.183727 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cae3ef85-0d5d-4296-b130-1c6bcc2fe401 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:08.203906 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-cae3ef85-0d5d-4296-b130-1c6bcc2fe401 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] HTTP exception thrown: Instance 84ee8179-ff94-43b3-97f2-1867bc6420a8 could not be found. Jul 16 21:12:08.206105 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-cae3ef85-0d5d-4296-b130-1c6bcc2fe401 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Request method failure captured: Jul 16 21:12:08.206105 ncdv-master-1 devstack@n-api.service[138680]: request: GET /compute/v2.1/servers/84ee8179-ff94-43b3-97f2-1867bc6420a8 HTTP/1.1 Jul 16 21:12:08.206105 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:12:08.206105 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:12:08.206105 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:12:08.206105 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 0 Jul 16 21:12:08.206105 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:12:08.206105 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:12:08.206105 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:12:08.206105 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:12:08.206105 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0yjvVgP0jq68r-ny5Eh1zEbXMYJhEkVpBaJLnFwKzfSzzxAGgQPy49cpkkoW8Rc5r0epWFa_INrkbRxHc5yb1M5XgG50--wLBrLRBZrPuxqH4ynry56OcrGI4F8X9HPf6z5uwbMiaMLyGnsa6t3q-Npn98NzTN2KKNq9Fp4f3zZbeyY Jul 16 21:12:08.206105 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:12:08.206105 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:12:08.206105 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:12:08.206105 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:12:08.206105 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:12:08.206105 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:12:08.206105 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: 649babe54d104436af461f1ee4767001 Jul 16 21:12:08.206105 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-ServersOnMultiNodesTest-1258903790 Jul 16 21:12:08.206105 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:12:08.206105 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:12:08.207449 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/649babe54d104436af461f1ee4767001"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/649babe54d104436af461f1ee4767001"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_649babe54d104436af461f1ee4767001"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/649babe54d104436af461f1ee4767001"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:12:08.207449 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-ServersOnMultiNodesTest-1258903790 Jul 16 21:12:08.207449 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: 649babe54d104436af461f1ee4767001 Jul 16 21:12:08.207449 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-ServersOnMultiNodesTest-1258903790 Jul 16 21:12:08.207449 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-ServersOnMultiNodesTest-1258903790-project Jul 16 21:12:08.207449 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:12:08.207449 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:12:08.207449 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: abeb7020019d41a18de42956d706a011 Jul 16 21:12:08.207449 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-ServersOnMultiNodesTest-1258903790-project Jul 16 21:12:08.207449 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 21:12:08.207449 ncdv-master-1 devstack@n-api.service[138680]: exception: Instance 84ee8179-ff94-43b3-97f2-1867bc6420a8 could not be found. Jul 16 21:12:08.207449 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': '84ee8179-ff94-43b3-97f2-1867bc6420a8'} Jul 16 21:12:08.208171 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:12:08.208171 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:12:08.208171 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:12:08.208171 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:12:08.208171 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:12:08.208171 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:12:08.208171 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:12:08.208171 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:12:08.208171 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:12:08.208171 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:12:08.208171 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 84ee8179-ff94-43b3-97f2-1867bc6420a8 could not be found. Jul 16 21:12:08.208171 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:12:08.208171 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:12:08.208171 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:12:08.208171 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:12:08.208171 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:12:08.208171 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:12:08.208171 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:12:08.208171 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:12:08.208171 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:12:08.209457 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:12:08.209457 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:12:08.209457 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:12:08.209457 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:12:08.209457 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:12:08.209457 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:12:08.209457 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:12:08.209457 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 84ee8179-ff94-43b3-97f2-1867bc6420a8 could not be found. Jul 16 21:12:08.209457 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:12:08.209457 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:12:08.209457 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:12:08.209457 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:12:08.209457 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:12:08.209457 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:12:08.209457 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:12:08.209457 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:12:08.209457 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 84ee8179-ff94-43b3-97f2-1867bc6420a8 could not be found. Jul 16 21:12:08.209457 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:12:08.210919 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-cae3ef85-0d5d-4296-b130-1c6bcc2fe401 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Returning 404 to user: Instance 84ee8179-ff94-43b3-97f2-1867bc6420a8 could not be found. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:12:08.210919 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-cae3ef85-0d5d-4296-b130-1c6bcc2fe401 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] 10.222.0.16 "GET /compute/v2.1/servers/84ee8179-ff94-43b3-97f2-1867bc6420a8" status: 404 len: 111 microversion: 2.1 time: 0.045609 Jul 16 21:12:08.212292 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 559/1077] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:12:08 2022] GET /compute/v2.1/servers/84ee8179-ff94-43b3-97f2-1867bc6420a8 => generated 111 bytes in 50 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:12:08.222941 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-f681ae00-4d96-4330-b37f-b3699a5a4c8f tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:08.223542 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-f681ae00-4d96-4330-b37f-b3699a5a4c8f tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] [instance: 4ae3e944-b450-40e5-bb71-c02df03212e3] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:12:08.302786 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-f681ae00-4d96-4330-b37f-b3699a5a4c8f tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2dfcad8d2b57481bb33aa8f5d7483c51', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cda86413e74b4c1e8e09bd4df4926664', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:08.308504 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-f681ae00-4d96-4330-b37f-b3699a5a4c8f tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] 10.222.0.16 "GET /compute/v2.1/servers/4ae3e944-b450-40e5-bb71-c02df03212e3" status: 200 len: 1223 microversion: 2.1 time: 0.095848 Jul 16 21:12:08.309177 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 560/1078] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:12:08 2022] GET /compute/v2.1/servers/4ae3e944-b450-40e5-bb71-c02df03212e3 => generated 1223 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:12:08.313903 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-6133e1d6-3596-4d66-a98e-2b248014a727 req-4fc841cf-7ac4-47cd-9e0d-c2e1cdae71fd service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "2b26717b-355f-417e-8a7c-72c41e1974e1", "name": "network-vif-plugged", "status": "completed", "tag": "c64dc016-0b59-43fd-8f3b-dcfc4298e2bd"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:12:08.323339 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-6133e1d6-3596-4d66-a98e-2b248014a727 req-4fc841cf-7ac4-47cd-9e0d-c2e1cdae71fd service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:08.324013 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-6133e1d6-3596-4d66-a98e-2b248014a727 req-4fc841cf-7ac4-47cd-9e0d-c2e1cdae71fd service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:08.334867 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-6133e1d6-3596-4d66-a98e-2b248014a727 req-4fc841cf-7ac4-47cd-9e0d-c2e1cdae71fd service nova] Creating event network-vif-plugged:c64dc016-0b59-43fd-8f3b-dcfc4298e2bd for instance 2b26717b-355f-417e-8a7c-72c41e1974e1 on nch2-master-1 Jul 16 21:12:08.340964 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-6133e1d6-3596-4d66-a98e-2b248014a727 req-4fc841cf-7ac4-47cd-9e0d-c2e1cdae71fd service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.030390 Jul 16 21:12:08.341629 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 561/1079] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:12:08 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:12:08.348314 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-4e237b3c-ad40-48f1-bebb-04bb9edb608d tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:08.349193 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-4e237b3c-ad40-48f1-bebb-04bb9edb608d tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:12:08.359408 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4e237b3c-ad40-48f1-bebb-04bb9edb608d tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:08.360172 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4e237b3c-ad40-48f1-bebb-04bb9edb608d tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:08.361627 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4e237b3c-ad40-48f1-bebb-04bb9edb608d tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:08.364920 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4e237b3c-ad40-48f1-bebb-04bb9edb608d tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:08.415135 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-a419f068-2f3e-4cff-9079-cd607cb6dae2 tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '034c11cc3f174c6fb44c8d64e55cf8b9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb4ff5d7e42940cdb7be71dadd895ee4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:08.445125 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-a419f068-2f3e-4cff-9079-cd607cb6dae2 tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] 10.222.0.16 "GET /compute/v2.1/servers/2b26717b-355f-417e-8a7c-72c41e1974e1" status: 200 len: 1395 microversion: 2.1 time: 0.369953 Jul 16 21:12:08.446090 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 519/1080] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:12:08 2022] GET /compute/v2.1/servers/2b26717b-355f-417e-8a7c-72c41e1974e1 => generated 1395 bytes in 372 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:12:08.455290 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-9d8bcb63-3d96-4047-80eb-c87edf048dc9 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:08.460484 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-9d8bcb63-3d96-4047-80eb-c87edf048dc9 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] [instance: 7d624313-0c12-4da2-a95c-3903efae7608] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:12:08.478908 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9d8bcb63-3d96-4047-80eb-c87edf048dc9 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:08.479986 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9d8bcb63-3d96-4047-80eb-c87edf048dc9 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:08.480788 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9d8bcb63-3d96-4047-80eb-c87edf048dc9 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:08.483709 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9d8bcb63-3d96-4047-80eb-c87edf048dc9 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:08.500093 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-9d8bcb63-3d96-4047-80eb-c87edf048dc9 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] HTTP exception thrown: Instance 7d624313-0c12-4da2-a95c-3903efae7608 could not be found. Jul 16 21:12:08.502185 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-9d8bcb63-3d96-4047-80eb-c87edf048dc9 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Request method failure captured: Jul 16 21:12:08.502185 ncdv-master-1 devstack@n-api.service[138681]: request: GET /compute/v2.1/servers/7d624313-0c12-4da2-a95c-3903efae7608 HTTP/1.1 Jul 16 21:12:08.502185 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:12:08.502185 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:12:08.502185 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:12:08.502185 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 21:12:08.502185 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:12:08.502185 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:12:08.502185 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:12:08.502185 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:12:08.502185 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0yjvVgP0jq68r-ny5Eh1zEbXMYJhEkVpBaJLnFwKzfSzzxAGgQPy49cpkkoW8Rc5r0epWFa_INrkbRxHc5yb1M5XgG50--wLBrLRBZrPuxqH4ynry56OcrGI4F8X9HPf6z5uwbMiaMLyGnsa6t3q-Npn98NzTN2KKNq9Fp4f3zZbeyY Jul 16 21:12:08.502185 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:12:08.502185 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:12:08.502185 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:12:08.502185 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:12:08.502185 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:12:08.502185 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:12:08.502185 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: 649babe54d104436af461f1ee4767001 Jul 16 21:12:08.502185 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-ServersOnMultiNodesTest-1258903790 Jul 16 21:12:08.502185 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:12:08.502185 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:12:08.503543 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/649babe54d104436af461f1ee4767001"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/649babe54d104436af461f1ee4767001"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_649babe54d104436af461f1ee4767001"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/649babe54d104436af461f1ee4767001"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:12:08.503543 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-ServersOnMultiNodesTest-1258903790 Jul 16 21:12:08.503543 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: 649babe54d104436af461f1ee4767001 Jul 16 21:12:08.503543 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-ServersOnMultiNodesTest-1258903790 Jul 16 21:12:08.503543 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-ServersOnMultiNodesTest-1258903790-project Jul 16 21:12:08.503543 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:12:08.503543 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:12:08.503543 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: abeb7020019d41a18de42956d706a011 Jul 16 21:12:08.503543 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-ServersOnMultiNodesTest-1258903790-project Jul 16 21:12:08.503543 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:12:08.503543 ncdv-master-1 devstack@n-api.service[138681]: exception: Instance 7d624313-0c12-4da2-a95c-3903efae7608 could not be found. Jul 16 21:12:08.503543 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': '7d624313-0c12-4da2-a95c-3903efae7608'} Jul 16 21:12:08.504275 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:12:08.504275 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:12:08.504275 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:12:08.504275 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:12:08.504275 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:12:08.504275 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:12:08.504275 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:12:08.504275 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:12:08.504275 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:12:08.504275 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:12:08.504275 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 7d624313-0c12-4da2-a95c-3903efae7608 could not be found. Jul 16 21:12:08.504275 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:12:08.504275 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:12:08.504275 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:12:08.504275 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:12:08.504275 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:12:08.504275 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:12:08.504275 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:12:08.504275 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:12:08.504275 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:12:08.508974 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:12:08.508974 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:12:08.508974 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:12:08.508974 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:12:08.508974 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:12:08.508974 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:12:08.508974 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:12:08.508974 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 7d624313-0c12-4da2-a95c-3903efae7608 could not be found. Jul 16 21:12:08.508974 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:12:08.508974 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:12:08.508974 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:12:08.508974 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:12:08.508974 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:12:08.508974 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:12:08.508974 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:12:08.508974 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:12:08.508974 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 7d624313-0c12-4da2-a95c-3903efae7608 could not be found. Jul 16 21:12:08.508974 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:12:08.510808 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-9d8bcb63-3d96-4047-80eb-c87edf048dc9 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Returning 404 to user: Instance 7d624313-0c12-4da2-a95c-3903efae7608 could not be found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:12:08.510808 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-9d8bcb63-3d96-4047-80eb-c87edf048dc9 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] 10.222.0.16 "GET /compute/v2.1/servers/7d624313-0c12-4da2-a95c-3903efae7608" status: 404 len: 111 microversion: 2.1 time: 0.056897 Jul 16 21:12:08.510808 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 520/1081] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:12:08 2022] GET /compute/v2.1/servers/7d624313-0c12-4da2-a95c-3903efae7608 => generated 111 bytes in 58 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:12:08.513497 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-2f42ca97-90aa-4a11-beaa-0f6284130711 tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:08.514116 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-2f42ca97-90aa-4a11-beaa-0f6284130711 tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] [instance: 83b1fe77-ef6d-408d-b255-aa3b60aad8f3] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:12:08.521675 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2f42ca97-90aa-4a11-beaa-0f6284130711 tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:08.521675 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2f42ca97-90aa-4a11-beaa-0f6284130711 tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:08.522471 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2f42ca97-90aa-4a11-beaa-0f6284130711 tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:08.522913 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2f42ca97-90aa-4a11-beaa-0f6284130711 tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:08.737279 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-4e237b3c-ad40-48f1-bebb-04bb9edb608d tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b68dacc4b8463ba07fae6372131117', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55a19b7a4682457a9132f5fc80110712', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:08.748435 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-4e237b3c-ad40-48f1-bebb-04bb9edb608d tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "GET /compute/v2.1/servers/a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2" status: 200 len: 1469 microversion: 2.1 time: 0.404644 Jul 16 21:12:08.760535 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 562/1082] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:12:08 2022] GET /compute/v2.1/servers/a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2 => generated 1469 bytes in 418 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:12:08.766618 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-12ed6bc0-fc50-433c-83bd-da19ca6bcba3 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:08.860517 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-12ed6bc0-fc50-433c-83bd-da19ca6bcba3 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] 10.222.0.16 "DELETE /compute/v2.1/os-server-groups/d6276585-4df3-40a1-817c-e7dcc6987bf4" status: 204 len: 0 microversion: 2.1 time: 0.098256 Jul 16 21:12:08.861765 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 563/1083] 10.222.0.16 () {58 vars in 1265 bytes} [Sat Jul 16 21:12:08 2022] DELETE /compute/v2.1/os-server-groups/d6276585-4df3-40a1-817c-e7dcc6987bf4 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 16 21:12:08.866890 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-2de8f323-ba21-422b-9989-baefbfedf912 req-1c8307be-6fdb-496d-a655-efc8f4b8913c service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "7d624313-0c12-4da2-a95c-3903efae7608", "name": "network-vif-deleted", "tag": "186c161e-a1dc-4086-9efc-737d7b982c1b"}, {"server_uuid": "0fb5b5b6-1796-4406-b276-326e1d7dd451", "name": "network-vif-unplugged", "status": "completed", "tag": "19f08e09-ff8e-43eb-b5f2-421d0eadc41c"}, {"name": "network-changed", "server_uuid": "0fb5b5b6-1796-4406-b276-326e1d7dd451", "tag": "19f08e09-ff8e-43eb-b5f2-421d0eadc41c"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:12:08.877693 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_api:SELECT=90,nova_api:INSERT=6,nova_api:UPDATE=1,nova_api:DELETE=3 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:12:08.882321 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-2de8f323-ba21-422b-9989-baefbfedf912 req-1c8307be-6fdb-496d-a655-efc8f4b8913c service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:08.882774 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-2de8f323-ba21-422b-9989-baefbfedf912 req-1c8307be-6fdb-496d-a655-efc8f4b8913c service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:08.897527 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-2de8f323-ba21-422b-9989-baefbfedf912 req-1c8307be-6fdb-496d-a655-efc8f4b8913c service nova] Creating event network-vif-deleted:186c161e-a1dc-4086-9efc-737d7b982c1b for instance 7d624313-0c12-4da2-a95c-3903efae7608 on nch1-master-1 Jul 16 21:12:08.898421 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-2de8f323-ba21-422b-9989-baefbfedf912 req-1c8307be-6fdb-496d-a655-efc8f4b8913c service nova] Creating event network-vif-unplugged:19f08e09-ff8e-43eb-b5f2-421d0eadc41c for instance 0fb5b5b6-1796-4406-b276-326e1d7dd451 on nch2-master-1 Jul 16 21:12:08.899127 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-2de8f323-ba21-422b-9989-baefbfedf912 req-1c8307be-6fdb-496d-a655-efc8f4b8913c service nova] Creating event network-changed:19f08e09-ff8e-43eb-b5f2-421d0eadc41c for instance 0fb5b5b6-1796-4406-b276-326e1d7dd451 on nch2-master-1 Jul 16 21:12:08.910619 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-2de8f323-ba21-422b-9989-baefbfedf912 req-1c8307be-6fdb-496d-a655-efc8f4b8913c service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 523 microversion: 2.1 time: 0.047118 Jul 16 21:12:08.911962 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 564/1084] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:12:08 2022] POST /compute/v2.1/os-server-external-events => generated 523 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:12:08.918068 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-e538cd2c-423a-459a-9171-8b98ffdd4d1e tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:08.918667 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-e538cd2c-423a-459a-9171-8b98ffdd4d1e tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] [instance: 45164bce-7acf-4ef6-8afa-aaab6287ecf4] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:12:08.925352 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e538cd2c-423a-459a-9171-8b98ffdd4d1e tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:08.925805 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e538cd2c-423a-459a-9171-8b98ffdd4d1e tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:08.926588 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e538cd2c-423a-459a-9171-8b98ffdd4d1e tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:08.927051 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e538cd2c-423a-459a-9171-8b98ffdd4d1e tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:08.956636 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-e538cd2c-423a-459a-9171-8b98ffdd4d1e tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] [instance: 45164bce-7acf-4ef6-8afa-aaab6287ecf4] Going to try to terminate instance {{(pid=138680) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 16 21:12:08.988155 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:12:09.193823 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-2f42ca97-90aa-4a11-beaa-0f6284130711 tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56d4bbcdebc24a34a814d7f6e72fbcfd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7d2b583c48254e17a8082f6467f2ae6d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:09.211582 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-2f42ca97-90aa-4a11-beaa-0f6284130711 tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] 10.222.0.16 "GET /compute/v2.1/servers/83b1fe77-ef6d-408d-b255-aa3b60aad8f3" status: 200 len: 1319 microversion: 2.1 time: 0.705285 Jul 16 21:12:09.212746 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 521/1085] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:12:08 2022] GET /compute/v2.1/servers/83b1fe77-ef6d-408d-b255-aa3b60aad8f3 => generated 1319 bytes in 707 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:12:09.219187 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-dafd9457-0058-4714-aee7-caf34e28b722 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:09.220444 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-dafd9457-0058-4714-aee7-caf34e28b722 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] [instance: 0fb5b5b6-1796-4406-b276-326e1d7dd451] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:12:09.228301 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-dafd9457-0058-4714-aee7-caf34e28b722 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:09.228857 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-dafd9457-0058-4714-aee7-caf34e28b722 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:09.229661 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-dafd9457-0058-4714-aee7-caf34e28b722 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:09.230093 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-dafd9457-0058-4714-aee7-caf34e28b722 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:09.284357 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-e538cd2c-423a-459a-9171-8b98ffdd4d1e tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] 10.222.0.16 "DELETE /compute/v2.1/servers/45164bce-7acf-4ef6-8afa-aaab6287ecf4" status: 204 len: 0 microversion: 2.1 time: 0.370819 Jul 16 21:12:09.284961 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 565/1086] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:12:08 2022] DELETE /compute/v2.1/servers/45164bce-7acf-4ef6-8afa-aaab6287ecf4 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 16 21:12:09.295395 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-ce3c5d4d-7c23-4501-80d8-0cfc75d8aefc tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:09.298939 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-ce3c5d4d-7c23-4501-80d8-0cfc75d8aefc tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] [instance: 944f68af-bf99-428f-9357-9ce9f807483b] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:12:09.306798 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ce3c5d4d-7c23-4501-80d8-0cfc75d8aefc tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:09.312765 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ce3c5d4d-7c23-4501-80d8-0cfc75d8aefc tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:09.312765 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ce3c5d4d-7c23-4501-80d8-0cfc75d8aefc tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:09.312765 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ce3c5d4d-7c23-4501-80d8-0cfc75d8aefc tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:09.346982 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-ce3c5d4d-7c23-4501-80d8-0cfc75d8aefc tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] [instance: 944f68af-bf99-428f-9357-9ce9f807483b] Going to try to terminate instance {{(pid=138680) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 16 21:12:09.413232 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-dafd9457-0058-4714-aee7-caf34e28b722 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:09.422767 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-dafd9457-0058-4714-aee7-caf34e28b722 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:09.424582 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-dafd9457-0058-4714-aee7-caf34e28b722 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:09.425752 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-dafd9457-0058-4714-aee7-caf34e28b722 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] 10.222.0.16 "GET /compute/v2.1/servers/0fb5b5b6-1796-4406-b276-326e1d7dd451" status: 200 len: 1567 microversion: 2.25 time: 0.210999 Jul 16 21:12:09.426311 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 522/1087] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:12:09 2022] GET /compute/v2.1/servers/0fb5b5b6-1796-4406-b276-326e1d7dd451 => generated 1567 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:12:09.431421 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-f3c9d839-8914-4741-adfb-62cc0b63bffd tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:09.435663 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-f3c9d839-8914-4741-adfb-62cc0b63bffd tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] [instance: 4ae3e944-b450-40e5-bb71-c02df03212e3] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:12:09.443774 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f3c9d839-8914-4741-adfb-62cc0b63bffd tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:09.444475 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f3c9d839-8914-4741-adfb-62cc0b63bffd tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:09.445290 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f3c9d839-8914-4741-adfb-62cc0b63bffd tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:09.445730 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f3c9d839-8914-4741-adfb-62cc0b63bffd tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:09.570556 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-f3c9d839-8914-4741-adfb-62cc0b63bffd tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2dfcad8d2b57481bb33aa8f5d7483c51', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cda86413e74b4c1e8e09bd4df4926664', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:09.581951 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-f3c9d839-8914-4741-adfb-62cc0b63bffd tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] 10.222.0.16 "GET /compute/v2.1/servers/4ae3e944-b450-40e5-bb71-c02df03212e3" status: 200 len: 1219 microversion: 2.1 time: 0.154573 Jul 16 21:12:09.593063 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 523/1088] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:12:09 2022] GET /compute/v2.1/servers/4ae3e944-b450-40e5-bb71-c02df03212e3 => generated 1219 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:12:09.598403 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-d15f162a-fc61-4c00-906b-60a1b5383510 tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:09.599336 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-d15f162a-fc61-4c00-906b-60a1b5383510 tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] [instance: 2b26717b-355f-417e-8a7c-72c41e1974e1] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:12:09.606812 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d15f162a-fc61-4c00-906b-60a1b5383510 tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:09.607588 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d15f162a-fc61-4c00-906b-60a1b5383510 tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:09.608617 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d15f162a-fc61-4c00-906b-60a1b5383510 tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:09.609254 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d15f162a-fc61-4c00-906b-60a1b5383510 tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:09.638311 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-ce3c5d4d-7c23-4501-80d8-0cfc75d8aefc tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] 10.222.0.16 "DELETE /compute/v2.1/servers/944f68af-bf99-428f-9357-9ce9f807483b" status: 204 len: 0 microversion: 2.1 time: 0.347411 Jul 16 21:12:09.643541 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 566/1089] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:12:09 2022] DELETE /compute/v2.1/servers/944f68af-bf99-428f-9357-9ce9f807483b => generated 0 bytes in 351 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 16 21:12:09.649385 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-19f1dd12-9ca2-470a-bd4c-d6797d825ac0 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:09.652150 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-19f1dd12-9ca2-470a-bd4c-d6797d825ac0 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] [instance: 45164bce-7acf-4ef6-8afa-aaab6287ecf4] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:12:09.661440 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-19f1dd12-9ca2-470a-bd4c-d6797d825ac0 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:09.662396 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-19f1dd12-9ca2-470a-bd4c-d6797d825ac0 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:09.667641 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-19f1dd12-9ca2-470a-bd4c-d6797d825ac0 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:09.668164 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-19f1dd12-9ca2-470a-bd4c-d6797d825ac0 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:09.895265 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-d15f162a-fc61-4c00-906b-60a1b5383510 tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '034c11cc3f174c6fb44c8d64e55cf8b9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb4ff5d7e42940cdb7be71dadd895ee4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:09.903035 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-d15f162a-fc61-4c00-906b-60a1b5383510 tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] 10.222.0.16 "GET /compute/v2.1/servers/2b26717b-355f-417e-8a7c-72c41e1974e1" status: 200 len: 1395 microversion: 2.1 time: 0.308501 Jul 16 21:12:09.903803 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 524/1090] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:12:09 2022] GET /compute/v2.1/servers/2b26717b-355f-417e-8a7c-72c41e1974e1 => generated 1395 bytes in 310 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:12:09.908819 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-62a6f70c-0e2c-4796-829d-eb4d85736b7d tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:09.909609 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-62a6f70c-0e2c-4796-829d-eb4d85736b7d tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:12:09.915934 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-62a6f70c-0e2c-4796-829d-eb4d85736b7d tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:09.916610 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-62a6f70c-0e2c-4796-829d-eb4d85736b7d tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:09.917486 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-62a6f70c-0e2c-4796-829d-eb4d85736b7d tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:09.917957 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-62a6f70c-0e2c-4796-829d-eb4d85736b7d tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:09.930507 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-19f1dd12-9ca2-470a-bd4c-d6797d825ac0 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'abeb7020019d41a18de42956d706a011', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '649babe54d104436af461f1ee4767001', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:09.938910 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-19f1dd12-9ca2-470a-bd4c-d6797d825ac0 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] 10.222.0.16 "GET /compute/v2.1/servers/45164bce-7acf-4ef6-8afa-aaab6287ecf4" status: 200 len: 1475 microversion: 2.1 time: 0.293873 Jul 16 21:12:09.939521 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 567/1091] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:12:09 2022] GET /compute/v2.1/servers/45164bce-7acf-4ef6-8afa-aaab6287ecf4 => generated 1475 bytes in 295 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:12:10.080747 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-62a6f70c-0e2c-4796-829d-eb4d85736b7d tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b68dacc4b8463ba07fae6372131117', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55a19b7a4682457a9132f5fc80110712', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:10.095893 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-62a6f70c-0e2c-4796-829d-eb4d85736b7d tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "GET /compute/v2.1/servers/a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2" status: 200 len: 1469 microversion: 2.1 time: 0.183970 Jul 16 21:12:10.095893 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 525/1092] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:12:09 2022] GET /compute/v2.1/servers/a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2 => generated 1469 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:12:10.231180 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-a4ef6975-3fe3-45c5-a932-336ff2d02bb7 tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:10.232100 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-a4ef6975-3fe3-45c5-a932-336ff2d02bb7 tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] [instance: 83b1fe77-ef6d-408d-b255-aa3b60aad8f3] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:12:10.241189 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a4ef6975-3fe3-45c5-a932-336ff2d02bb7 tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:10.242703 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a4ef6975-3fe3-45c5-a932-336ff2d02bb7 tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:10.244121 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a4ef6975-3fe3-45c5-a932-336ff2d02bb7 tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:10.245219 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a4ef6975-3fe3-45c5-a932-336ff2d02bb7 tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:10.444997 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-c6b1e203-7f33-4dec-bfa0-99c62d04b725 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:10.445622 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-c6b1e203-7f33-4dec-bfa0-99c62d04b725 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] [instance: 0fb5b5b6-1796-4406-b276-326e1d7dd451] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:12:10.452249 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c6b1e203-7f33-4dec-bfa0-99c62d04b725 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:10.452707 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c6b1e203-7f33-4dec-bfa0-99c62d04b725 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:10.453486 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c6b1e203-7f33-4dec-bfa0-99c62d04b725 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:10.453928 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c6b1e203-7f33-4dec-bfa0-99c62d04b725 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:10.492703 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-a4ef6975-3fe3-45c5-a932-336ff2d02bb7 tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56d4bbcdebc24a34a814d7f6e72fbcfd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7d2b583c48254e17a8082f6467f2ae6d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:10.505988 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-a4ef6975-3fe3-45c5-a932-336ff2d02bb7 tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] 10.222.0.16 "GET /compute/v2.1/servers/83b1fe77-ef6d-408d-b255-aa3b60aad8f3" status: 200 len: 1319 microversion: 2.1 time: 0.279658 Jul 16 21:12:10.507254 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 568/1093] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:12:10 2022] GET /compute/v2.1/servers/83b1fe77-ef6d-408d-b255-aa3b60aad8f3 => generated 1319 bytes in 281 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:12:10.578548 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-fa47b3e1-b920-4518-abb4-450398ef4ee4 req-f21c315f-a983-4a97-a597-f1499da61080 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2", "name": "network-vif-plugged", "status": "completed", "tag": "bf9d54d7-5718-4678-bc2f-5ba8241bd855"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:12:10.593764 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-fa47b3e1-b920-4518-abb4-450398ef4ee4 req-f21c315f-a983-4a97-a597-f1499da61080 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:10.594210 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-fa47b3e1-b920-4518-abb4-450398ef4ee4 req-f21c315f-a983-4a97-a597-f1499da61080 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:10.610324 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-fa47b3e1-b920-4518-abb4-450398ef4ee4 req-f21c315f-a983-4a97-a597-f1499da61080 service nova] Creating event network-vif-plugged:bf9d54d7-5718-4678-bc2f-5ba8241bd855 for instance a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2 on nch2-master-1 Jul 16 21:12:10.616050 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [req-fa47b3e1-b920-4518-abb4-450398ef4ee4 req-f21c315f-a983-4a97-a597-f1499da61080 service nova] Instance a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2 is migrating within the same cell, copying events to all relevant hosts: {'nch2-master-1', 'nch1-master-1'} {{(pid=138680) _get_relevant_hosts /opt/stack/nova/nova/compute/api.py:5807}} Jul 16 21:12:10.627343 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-fa47b3e1-b920-4518-abb4-450398ef4ee4 req-f21c315f-a983-4a97-a597-f1499da61080 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.053467 Jul 16 21:12:10.628389 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 569/1094] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:12:10 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 16 21:12:10.633522 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-8b00b65c-48ab-4d0a-a67e-18d60f75d238 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:10.634257 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-8b00b65c-48ab-4d0a-a67e-18d60f75d238 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] [instance: 4ae3e944-b450-40e5-bb71-c02df03212e3] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:12:10.641344 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8b00b65c-48ab-4d0a-a67e-18d60f75d238 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:10.641793 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8b00b65c-48ab-4d0a-a67e-18d60f75d238 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:10.642567 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8b00b65c-48ab-4d0a-a67e-18d60f75d238 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:10.642998 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8b00b65c-48ab-4d0a-a67e-18d60f75d238 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:10.673074 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-c6b1e203-7f33-4dec-bfa0-99c62d04b725 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:10.681308 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-c6b1e203-7f33-4dec-bfa0-99c62d04b725 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:10.682947 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-c6b1e203-7f33-4dec-bfa0-99c62d04b725 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:10.684284 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-c6b1e203-7f33-4dec-bfa0-99c62d04b725 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] 10.222.0.16 "GET /compute/v2.1/servers/0fb5b5b6-1796-4406-b276-326e1d7dd451" status: 200 len: 1567 microversion: 2.25 time: 0.243280 Jul 16 21:12:10.684852 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 526/1095] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:12:10 2022] GET /compute/v2.1/servers/0fb5b5b6-1796-4406-b276-326e1d7dd451 => generated 1567 bytes in 245 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 16 21:12:10.745965 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-8b00b65c-48ab-4d0a-a67e-18d60f75d238 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2dfcad8d2b57481bb33aa8f5d7483c51', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cda86413e74b4c1e8e09bd4df4926664', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:10.767101 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-8b00b65c-48ab-4d0a-a67e-18d60f75d238 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] 10.222.0.16 "GET /compute/v2.1/servers/4ae3e944-b450-40e5-bb71-c02df03212e3" status: 200 len: 1275 microversion: 2.1 time: 0.137414 Jul 16 21:12:10.767934 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 570/1096] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:12:10 2022] GET /compute/v2.1/servers/4ae3e944-b450-40e5-bb71-c02df03212e3 => generated 1275 bytes in 139 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:12:10.920507 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-488fec59-35ab-4c49-9eab-fe8d4ab9c173 tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:10.921119 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-488fec59-35ab-4c49-9eab-fe8d4ab9c173 tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] [instance: 2b26717b-355f-417e-8a7c-72c41e1974e1] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:12:10.927936 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-488fec59-35ab-4c49-9eab-fe8d4ab9c173 tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:10.928386 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-488fec59-35ab-4c49-9eab-fe8d4ab9c173 tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:10.929168 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-488fec59-35ab-4c49-9eab-fe8d4ab9c173 tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:10.929602 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-488fec59-35ab-4c49-9eab-fe8d4ab9c173 tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:10.956252 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-b3cff1c3-056e-46ee-96f2-548dbd5a49f9 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:10.957020 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-b3cff1c3-056e-46ee-96f2-548dbd5a49f9 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] [instance: 45164bce-7acf-4ef6-8afa-aaab6287ecf4] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:12:10.964102 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b3cff1c3-056e-46ee-96f2-548dbd5a49f9 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:10.964801 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b3cff1c3-056e-46ee-96f2-548dbd5a49f9 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:10.965580 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b3cff1c3-056e-46ee-96f2-548dbd5a49f9 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:10.966019 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b3cff1c3-056e-46ee-96f2-548dbd5a49f9 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:11.127048 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-488fec59-35ab-4c49-9eab-fe8d4ab9c173 tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '034c11cc3f174c6fb44c8d64e55cf8b9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb4ff5d7e42940cdb7be71dadd895ee4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:11.134752 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-488fec59-35ab-4c49-9eab-fe8d4ab9c173 tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] 10.222.0.16 "GET /compute/v2.1/servers/2b26717b-355f-417e-8a7c-72c41e1974e1" status: 200 len: 1395 microversion: 2.1 time: 0.218528 Jul 16 21:12:11.135910 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 527/1097] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:12:10 2022] GET /compute/v2.1/servers/2b26717b-355f-417e-8a7c-72c41e1974e1 => generated 1395 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:12:11.140858 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-fed98601-6a00-4652-81b7-1ebcbb438b86 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:11.141668 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-fed98601-6a00-4652-81b7-1ebcbb438b86 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:12:11.147688 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fed98601-6a00-4652-81b7-1ebcbb438b86 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:11.148159 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fed98601-6a00-4652-81b7-1ebcbb438b86 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:11.148942 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fed98601-6a00-4652-81b7-1ebcbb438b86 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:11.149383 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fed98601-6a00-4652-81b7-1ebcbb438b86 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:11.156890 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-b3cff1c3-056e-46ee-96f2-548dbd5a49f9 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'abeb7020019d41a18de42956d706a011', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '649babe54d104436af461f1ee4767001', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:11.164702 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-b3cff1c3-056e-46ee-96f2-548dbd5a49f9 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] 10.222.0.16 "GET /compute/v2.1/servers/45164bce-7acf-4ef6-8afa-aaab6287ecf4" status: 200 len: 1475 microversion: 2.1 time: 0.212244 Jul 16 21:12:11.165271 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 571/1098] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:12:10 2022] GET /compute/v2.1/servers/45164bce-7acf-4ef6-8afa-aaab6287ecf4 => generated 1475 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:12:11.170331 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-4ce52592-d913-4907-a19e-67643914eb5b req-d312ac60-f30c-433f-b253-d89e734bdf61 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "83b1fe77-ef6d-408d-b255-aa3b60aad8f3", "tag": "fef3d648-18d7-4308-9bcb-4555c790a3c7"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:12:11.178155 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-4ce52592-d913-4907-a19e-67643914eb5b req-d312ac60-f30c-433f-b253-d89e734bdf61 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:11.178595 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-4ce52592-d913-4907-a19e-67643914eb5b req-d312ac60-f30c-433f-b253-d89e734bdf61 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:11.190717 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-4ce52592-d913-4907-a19e-67643914eb5b req-d312ac60-f30c-433f-b253-d89e734bdf61 service nova] Creating event network-changed:fef3d648-18d7-4308-9bcb-4555c790a3c7 for instance 83b1fe77-ef6d-408d-b255-aa3b60aad8f3 on nch1-master-1 Jul 16 21:12:11.195603 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-4ce52592-d913-4907-a19e-67643914eb5b req-d312ac60-f30c-433f-b253-d89e734bdf61 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.029181 Jul 16 21:12:11.196387 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 572/1099] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:12:11 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 30 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:12:11.287029 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-fed98601-6a00-4652-81b7-1ebcbb438b86 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b68dacc4b8463ba07fae6372131117', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55a19b7a4682457a9132f5fc80110712', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:11.294561 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-fed98601-6a00-4652-81b7-1ebcbb438b86 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "GET /compute/v2.1/servers/a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2" status: 200 len: 1469 microversion: 2.1 time: 0.157653 Jul 16 21:12:11.295157 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 528/1100] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:12:11 2022] GET /compute/v2.1/servers/a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2 => generated 1469 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:12:11.525098 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-d7c1bef3-8f2d-4a52-8852-398fb9197df9 tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:11.526541 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-d7c1bef3-8f2d-4a52-8852-398fb9197df9 tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] [instance: 83b1fe77-ef6d-408d-b255-aa3b60aad8f3] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:12:11.532870 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d7c1bef3-8f2d-4a52-8852-398fb9197df9 tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:11.533622 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d7c1bef3-8f2d-4a52-8852-398fb9197df9 tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:11.534067 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d7c1bef3-8f2d-4a52-8852-398fb9197df9 tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:11.534505 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d7c1bef3-8f2d-4a52-8852-398fb9197df9 tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:11.671598 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-d7c1bef3-8f2d-4a52-8852-398fb9197df9 tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56d4bbcdebc24a34a814d7f6e72fbcfd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7d2b583c48254e17a8082f6467f2ae6d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:11.679373 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-d7c1bef3-8f2d-4a52-8852-398fb9197df9 tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] 10.222.0.16 "GET /compute/v2.1/servers/83b1fe77-ef6d-408d-b255-aa3b60aad8f3" status: 200 len: 1319 microversion: 2.1 time: 0.158919 Jul 16 21:12:11.679946 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 573/1101] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:12:11 2022] GET /compute/v2.1/servers/83b1fe77-ef6d-408d-b255-aa3b60aad8f3 => generated 1319 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:12:11.702181 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-d826995c-41f8-4dc3-8a59-f91d2955573c tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:11.702751 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-d826995c-41f8-4dc3-8a59-f91d2955573c tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] [instance: 0fb5b5b6-1796-4406-b276-326e1d7dd451] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:12:11.704798 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell0:SELECT=14 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:12:11.710248 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d826995c-41f8-4dc3-8a59-f91d2955573c tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:11.710705 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d826995c-41f8-4dc3-8a59-f91d2955573c tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:11.711498 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d826995c-41f8-4dc3-8a59-f91d2955573c tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:11.711988 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d826995c-41f8-4dc3-8a59-f91d2955573c tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:11.785394 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-8772d488-42b6-4572-af19-df79a6c4c69a tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:11.786004 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-8772d488-42b6-4572-af19-df79a6c4c69a tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] [instance: 4ae3e944-b450-40e5-bb71-c02df03212e3] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:12:11.794686 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8772d488-42b6-4572-af19-df79a6c4c69a tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:11.795167 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8772d488-42b6-4572-af19-df79a6c4c69a tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:11.796219 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8772d488-42b6-4572-af19-df79a6c4c69a tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:11.796654 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8772d488-42b6-4572-af19-df79a6c4c69a tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:11.870695 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-8772d488-42b6-4572-af19-df79a6c4c69a tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2dfcad8d2b57481bb33aa8f5d7483c51', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cda86413e74b4c1e8e09bd4df4926664', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:11.875707 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-d826995c-41f8-4dc3-8a59-f91d2955573c tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:11.878443 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-8772d488-42b6-4572-af19-df79a6c4c69a tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] 10.222.0.16 "GET /compute/v2.1/servers/4ae3e944-b450-40e5-bb71-c02df03212e3" status: 200 len: 1275 microversion: 2.1 time: 0.096942 Jul 16 21:12:11.879050 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 574/1102] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:12:11 2022] GET /compute/v2.1/servers/4ae3e944-b450-40e5-bb71-c02df03212e3 => generated 1275 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:12:11.882367 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-d826995c-41f8-4dc3-8a59-f91d2955573c tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:11.884168 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-d826995c-41f8-4dc3-8a59-f91d2955573c tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:11.885439 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-d826995c-41f8-4dc3-8a59-f91d2955573c tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] 10.222.0.16 "GET /compute/v2.1/servers/0fb5b5b6-1796-4406-b276-326e1d7dd451" status: 200 len: 1567 microversion: 2.25 time: 0.187549 Jul 16 21:12:11.886137 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 529/1103] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:12:11 2022] GET /compute/v2.1/servers/0fb5b5b6-1796-4406-b276-326e1d7dd451 => generated 1567 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:12:12.152841 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-b1818860-8bb7-4934-bcf1-e9d3a69506f3 tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:12.153516 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-b1818860-8bb7-4934-bcf1-e9d3a69506f3 tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] [instance: 2b26717b-355f-417e-8a7c-72c41e1974e1] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:12:12.159651 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b1818860-8bb7-4934-bcf1-e9d3a69506f3 tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:12.160111 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b1818860-8bb7-4934-bcf1-e9d3a69506f3 tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:12.160887 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b1818860-8bb7-4934-bcf1-e9d3a69506f3 tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:12.161335 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b1818860-8bb7-4934-bcf1-e9d3a69506f3 tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:12.171838 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-0f70c2b5-5cd5-43ff-a0f8-d917fd570e09 tempest-LiveMigrationTest-1131758637 tempest-LiveMigrationTest-1131758637-project-admin] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:12.172456 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-0f70c2b5-5cd5-43ff-a0f8-d917fd570e09 tempest-LiveMigrationTest-1131758637 tempest-LiveMigrationTest-1131758637-project-admin] [instance: 6a391156-ffce-49c0-a79d-47b4544264f5] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:12:12.188853 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0f70c2b5-5cd5-43ff-a0f8-d917fd570e09 tempest-LiveMigrationTest-1131758637 tempest-LiveMigrationTest-1131758637-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:12.189339 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0f70c2b5-5cd5-43ff-a0f8-d917fd570e09 tempest-LiveMigrationTest-1131758637 tempest-LiveMigrationTest-1131758637-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:12.190189 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0f70c2b5-5cd5-43ff-a0f8-d917fd570e09 tempest-LiveMigrationTest-1131758637 tempest-LiveMigrationTest-1131758637-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:12.190626 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0f70c2b5-5cd5-43ff-a0f8-d917fd570e09 tempest-LiveMigrationTest-1131758637 tempest-LiveMigrationTest-1131758637-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:12.499875 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-b1818860-8bb7-4934-bcf1-e9d3a69506f3 tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '034c11cc3f174c6fb44c8d64e55cf8b9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb4ff5d7e42940cdb7be71dadd895ee4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:12.508053 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-b1818860-8bb7-4934-bcf1-e9d3a69506f3 tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] 10.222.0.16 "GET /compute/v2.1/servers/2b26717b-355f-417e-8a7c-72c41e1974e1" status: 200 len: 1395 microversion: 2.1 time: 0.359747 Jul 16 21:12:12.518531 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 575/1104] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:12:12 2022] GET /compute/v2.1/servers/2b26717b-355f-417e-8a7c-72c41e1974e1 => generated 1395 bytes in 361 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:12:12.518718 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-0f70c2b5-5cd5-43ff-a0f8-d917fd570e09 tempest-LiveMigrationTest-1131758637 tempest-LiveMigrationTest-1131758637-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/6a391156-ffce-49c0-a79d-47b4544264f5" status: 200 len: 1812 microversion: 2.1 time: 0.343553 Jul 16 21:12:12.522456 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-5f679855-1ce9-4f2c-8300-a22da53f459d tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:12.522456 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-5f679855-1ce9-4f2c-8300-a22da53f459d tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] [instance: 45164bce-7acf-4ef6-8afa-aaab6287ecf4] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:12:12.522456 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5f679855-1ce9-4f2c-8300-a22da53f459d tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:12.522456 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5f679855-1ce9-4f2c-8300-a22da53f459d tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:12.522881 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5f679855-1ce9-4f2c-8300-a22da53f459d tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:12.523415 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5f679855-1ce9-4f2c-8300-a22da53f459d tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:12.523693 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 530/1105] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:12:12 2022] GET /compute/v2.1/servers/6a391156-ffce-49c0-a79d-47b4544264f5 => generated 1812 bytes in 357 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:12:12.530150 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-06b8b046-a7e2-4d34-a951-8229d2954a62 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:12.530502 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-06b8b046-a7e2-4d34-a951-8229d2954a62 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:12:12.536929 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-06b8b046-a7e2-4d34-a951-8229d2954a62 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:12.537458 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-06b8b046-a7e2-4d34-a951-8229d2954a62 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:12.538326 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-06b8b046-a7e2-4d34-a951-8229d2954a62 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:12.538873 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-06b8b046-a7e2-4d34-a951-8229d2954a62 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:12.814328 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-5f679855-1ce9-4f2c-8300-a22da53f459d tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'abeb7020019d41a18de42956d706a011', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '649babe54d104436af461f1ee4767001', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:12.821812 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-5f679855-1ce9-4f2c-8300-a22da53f459d tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] 10.222.0.16 "GET /compute/v2.1/servers/45164bce-7acf-4ef6-8afa-aaab6287ecf4" status: 200 len: 1475 microversion: 2.1 time: 0.312132 Jul 16 21:12:12.831967 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 576/1106] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:12:12 2022] GET /compute/v2.1/servers/45164bce-7acf-4ef6-8afa-aaab6287ecf4 => generated 1475 bytes in 314 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:12:12.832173 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-06b8b046-a7e2-4d34-a951-8229d2954a62 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b68dacc4b8463ba07fae6372131117', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55a19b7a4682457a9132f5fc80110712', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:12.834441 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-e2168b8d-f3be-4f6b-83b8-cfaaffc441e8 tempest-LiveMigrationTest-1131758637 tempest-LiveMigrationTest-1131758637-project-admin] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:12.836088 ncdv-master-1 devstack@n-api.service[138680]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-services:list": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:12:12.836088 ncdv-master-1 devstack@n-api.service[138680]: warnings.warn(msg) Jul 16 21:12:12.837355 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e2168b8d-f3be-4f6b-83b8-cfaaffc441e8 tempest-LiveMigrationTest-1131758637 tempest-LiveMigrationTest-1131758637-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:12.837799 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e2168b8d-f3be-4f6b-83b8-cfaaffc441e8 tempest-LiveMigrationTest-1131758637 tempest-LiveMigrationTest-1131758637-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:12.838266 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-06b8b046-a7e2-4d34-a951-8229d2954a62 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "GET /compute/v2.1/servers/a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2" status: 200 len: 1469 microversion: 2.1 time: 0.313420 Jul 16 21:12:12.838873 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 531/1107] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:12:12 2022] GET /compute/v2.1/servers/a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2 => generated 1469 bytes in 315 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:12:12.843272 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e2168b8d-f3be-4f6b-83b8-cfaaffc441e8 tempest-LiveMigrationTest-1131758637 tempest-LiveMigrationTest-1131758637-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:12.843700 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-abd28cec-74ad-4a5e-9c7a-701e45ab2e99 tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:12.844369 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-abd28cec-74ad-4a5e-9c7a-701e45ab2e99 tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] [instance: 83b1fe77-ef6d-408d-b255-aa3b60aad8f3] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:12:12.846886 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e2168b8d-f3be-4f6b-83b8-cfaaffc441e8 tempest-LiveMigrationTest-1131758637 tempest-LiveMigrationTest-1131758637-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:12.850476 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-abd28cec-74ad-4a5e-9c7a-701e45ab2e99 tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:12.850928 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-abd28cec-74ad-4a5e-9c7a-701e45ab2e99 tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:12.851770 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-abd28cec-74ad-4a5e-9c7a-701e45ab2e99 tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:12.852208 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-abd28cec-74ad-4a5e-9c7a-701e45ab2e99 tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:12.873167 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-e2168b8d-f3be-4f6b-83b8-cfaaffc441e8 tempest-LiveMigrationTest-1131758637 tempest-LiveMigrationTest-1131758637-project-admin] 10.222.0.16 "GET /compute/v2.1/os-services?binary=nova-compute" status: 200 len: 380 microversion: 2.1 time: 0.048616 Jul 16 21:12:12.873784 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 577/1108] 10.222.0.16 () {58 vars in 1175 bytes} [Sat Jul 16 21:12:12 2022] GET /compute/v2.1/os-services?binary=nova-compute => generated 380 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:12:12.896280 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-73a55599-8af6-4408-bb33-e4f10dc21be0 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:12.898230 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-73a55599-8af6-4408-bb33-e4f10dc21be0 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] [instance: 4ae3e944-b450-40e5-bb71-c02df03212e3] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:12:12.904288 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-73a55599-8af6-4408-bb33-e4f10dc21be0 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:12.904739 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-73a55599-8af6-4408-bb33-e4f10dc21be0 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:12.905516 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-73a55599-8af6-4408-bb33-e4f10dc21be0 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:12.905948 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-73a55599-8af6-4408-bb33-e4f10dc21be0 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:12.982781 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-73a55599-8af6-4408-bb33-e4f10dc21be0 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2dfcad8d2b57481bb33aa8f5d7483c51', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cda86413e74b4c1e8e09bd4df4926664', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:12.991343 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-73a55599-8af6-4408-bb33-e4f10dc21be0 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] 10.222.0.16 "GET /compute/v2.1/servers/4ae3e944-b450-40e5-bb71-c02df03212e3" status: 200 len: 1275 microversion: 2.1 time: 0.099025 Jul 16 21:12:12.992162 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 578/1109] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:12:12 2022] GET /compute/v2.1/servers/4ae3e944-b450-40e5-bb71-c02df03212e3 => generated 1275 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:12:13.008552 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-ea376e8a-7ac5-4299-a4a6-8c2ecfb1ac13 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:13.010278 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-ea376e8a-7ac5-4299-a4a6-8c2ecfb1ac13 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] [instance: 0fb5b5b6-1796-4406-b276-326e1d7dd451] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:12:13.016194 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ea376e8a-7ac5-4299-a4a6-8c2ecfb1ac13 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:13.016653 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ea376e8a-7ac5-4299-a4a6-8c2ecfb1ac13 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:13.017438 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ea376e8a-7ac5-4299-a4a6-8c2ecfb1ac13 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:13.017879 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ea376e8a-7ac5-4299-a4a6-8c2ecfb1ac13 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:13.021856 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-abd28cec-74ad-4a5e-9c7a-701e45ab2e99 tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56d4bbcdebc24a34a814d7f6e72fbcfd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7d2b583c48254e17a8082f6467f2ae6d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:13.030757 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-abd28cec-74ad-4a5e-9c7a-701e45ab2e99 tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] 10.222.0.16 "GET /compute/v2.1/servers/83b1fe77-ef6d-408d-b255-aa3b60aad8f3" status: 200 len: 1319 microversion: 2.1 time: 0.190728 Jul 16 21:12:13.031565 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 532/1110] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:12:12 2022] GET /compute/v2.1/servers/83b1fe77-ef6d-408d-b255-aa3b60aad8f3 => generated 1319 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:12:13.169866 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-ea376e8a-7ac5-4299-a4a6-8c2ecfb1ac13 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:13.178123 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-ea376e8a-7ac5-4299-a4a6-8c2ecfb1ac13 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:13.179715 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-ea376e8a-7ac5-4299-a4a6-8c2ecfb1ac13 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:13.180906 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-ea376e8a-7ac5-4299-a4a6-8c2ecfb1ac13 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] 10.222.0.16 "GET /compute/v2.1/servers/0fb5b5b6-1796-4406-b276-326e1d7dd451" status: 200 len: 1567 microversion: 2.25 time: 0.176358 Jul 16 21:12:13.182208 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 579/1111] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:12:12 2022] GET /compute/v2.1/servers/0fb5b5b6-1796-4406-b276-326e1d7dd451 => generated 1567 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:12:13.526033 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-deebf9b2-aa99-4cb1-8434-ebdfb52e723a tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:13.526645 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-deebf9b2-aa99-4cb1-8434-ebdfb52e723a tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] [instance: 2b26717b-355f-417e-8a7c-72c41e1974e1] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:12:13.533304 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-deebf9b2-aa99-4cb1-8434-ebdfb52e723a tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:13.533751 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-deebf9b2-aa99-4cb1-8434-ebdfb52e723a tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:13.534505 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-deebf9b2-aa99-4cb1-8434-ebdfb52e723a tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:13.534942 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-deebf9b2-aa99-4cb1-8434-ebdfb52e723a tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:13.672278 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-deebf9b2-aa99-4cb1-8434-ebdfb52e723a tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '034c11cc3f174c6fb44c8d64e55cf8b9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb4ff5d7e42940cdb7be71dadd895ee4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:13.680467 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-deebf9b2-aa99-4cb1-8434-ebdfb52e723a tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] 10.222.0.16 "GET /compute/v2.1/servers/2b26717b-355f-417e-8a7c-72c41e1974e1" status: 200 len: 1395 microversion: 2.1 time: 0.158897 Jul 16 21:12:13.681040 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 533/1112] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:12:13 2022] GET /compute/v2.1/servers/2b26717b-355f-417e-8a7c-72c41e1974e1 => generated 1395 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:12:13.840359 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-43d85e59-56a0-498c-bdd7-bd5cdcd5a516 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:13.840983 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-43d85e59-56a0-498c-bdd7-bd5cdcd5a516 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] [instance: 45164bce-7acf-4ef6-8afa-aaab6287ecf4] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:12:13.847964 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-43d85e59-56a0-498c-bdd7-bd5cdcd5a516 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:13.848425 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-43d85e59-56a0-498c-bdd7-bd5cdcd5a516 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:13.849209 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-43d85e59-56a0-498c-bdd7-bd5cdcd5a516 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:13.854517 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-43d85e59-56a0-498c-bdd7-bd5cdcd5a516 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:13.856244 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-4b1b5b31-1183-4ce9-9b7b-eacf810bf760 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:13.861745 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-4b1b5b31-1183-4ce9-9b7b-eacf810bf760 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:12:13.872609 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4b1b5b31-1183-4ce9-9b7b-eacf810bf760 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:13.873577 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4b1b5b31-1183-4ce9-9b7b-eacf810bf760 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:13.874821 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4b1b5b31-1183-4ce9-9b7b-eacf810bf760 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:13.875540 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4b1b5b31-1183-4ce9-9b7b-eacf810bf760 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:13.922137 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:12:14.138417 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-43d85e59-56a0-498c-bdd7-bd5cdcd5a516 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'abeb7020019d41a18de42956d706a011', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '649babe54d104436af461f1ee4767001', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:14.146381 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-43d85e59-56a0-498c-bdd7-bd5cdcd5a516 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] 10.222.0.16 "GET /compute/v2.1/servers/45164bce-7acf-4ef6-8afa-aaab6287ecf4" status: 200 len: 1475 microversion: 2.1 time: 0.310588 Jul 16 21:12:14.146931 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 580/1113] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:12:13 2022] GET /compute/v2.1/servers/45164bce-7acf-4ef6-8afa-aaab6287ecf4 => generated 1475 bytes in 312 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:12:14.152100 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-7ee31c6c-bcd3-49da-af07-695a852c4f47 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:14.153474 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-7ee31c6c-bcd3-49da-af07-695a852c4f47 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] [instance: 4ae3e944-b450-40e5-bb71-c02df03212e3] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:12:14.159834 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-4b1b5b31-1183-4ce9-9b7b-eacf810bf760 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b68dacc4b8463ba07fae6372131117', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55a19b7a4682457a9132f5fc80110712', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:14.160273 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7ee31c6c-bcd3-49da-af07-695a852c4f47 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:14.160736 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7ee31c6c-bcd3-49da-af07-695a852c4f47 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:14.161524 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7ee31c6c-bcd3-49da-af07-695a852c4f47 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:14.161967 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7ee31c6c-bcd3-49da-af07-695a852c4f47 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:14.169763 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-4b1b5b31-1183-4ce9-9b7b-eacf810bf760 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "GET /compute/v2.1/servers/a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2" status: 200 len: 1469 microversion: 2.1 time: 0.318056 Jul 16 21:12:14.170559 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 534/1114] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:12:13 2022] GET /compute/v2.1/servers/a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2 => generated 1469 bytes in 320 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:12:14.175867 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-8219acd5-252a-4350-9d60-2d4b04c6d9c4 tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:14.176686 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-8219acd5-252a-4350-9d60-2d4b04c6d9c4 tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] [instance: 83b1fe77-ef6d-408d-b255-aa3b60aad8f3] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:12:14.179392 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=96,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:12:14.185049 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8219acd5-252a-4350-9d60-2d4b04c6d9c4 tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:14.185975 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8219acd5-252a-4350-9d60-2d4b04c6d9c4 tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:14.186837 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8219acd5-252a-4350-9d60-2d4b04c6d9c4 tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:14.187368 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8219acd5-252a-4350-9d60-2d4b04c6d9c4 tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:14.263811 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-7ee31c6c-bcd3-49da-af07-695a852c4f47 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2dfcad8d2b57481bb33aa8f5d7483c51', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cda86413e74b4c1e8e09bd4df4926664', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:14.276423 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-7ee31c6c-bcd3-49da-af07-695a852c4f47 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] 10.222.0.16 "GET /compute/v2.1/servers/4ae3e944-b450-40e5-bb71-c02df03212e3" status: 200 len: 1275 microversion: 2.1 time: 0.127937 Jul 16 21:12:14.277641 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 581/1115] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:12:14 2022] GET /compute/v2.1/servers/4ae3e944-b450-40e5-bb71-c02df03212e3 => generated 1275 bytes in 130 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:12:14.310736 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-eb502e87-62f5-4889-b733-906cf2a031e8 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:14.311407 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-eb502e87-62f5-4889-b733-906cf2a031e8 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] [instance: 0fb5b5b6-1796-4406-b276-326e1d7dd451] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:12:14.317931 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-eb502e87-62f5-4889-b733-906cf2a031e8 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:14.320711 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-eb502e87-62f5-4889-b733-906cf2a031e8 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:14.320711 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-eb502e87-62f5-4889-b733-906cf2a031e8 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:14.320711 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-eb502e87-62f5-4889-b733-906cf2a031e8 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:14.582692 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-8219acd5-252a-4350-9d60-2d4b04c6d9c4 tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56d4bbcdebc24a34a814d7f6e72fbcfd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7d2b583c48254e17a8082f6467f2ae6d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:14.597748 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-8219acd5-252a-4350-9d60-2d4b04c6d9c4 tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] 10.222.0.16 "GET /compute/v2.1/servers/83b1fe77-ef6d-408d-b255-aa3b60aad8f3" status: 200 len: 1319 microversion: 2.1 time: 0.422664 Jul 16 21:12:14.597748 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 535/1116] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:12:14 2022] GET /compute/v2.1/servers/83b1fe77-ef6d-408d-b255-aa3b60aad8f3 => generated 1319 bytes in 424 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:12:14.624984 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-eb502e87-62f5-4889-b733-906cf2a031e8 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:14.633171 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-eb502e87-62f5-4889-b733-906cf2a031e8 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:14.635359 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-eb502e87-62f5-4889-b733-906cf2a031e8 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:14.636774 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-eb502e87-62f5-4889-b733-906cf2a031e8 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] 10.222.0.16 "GET /compute/v2.1/servers/0fb5b5b6-1796-4406-b276-326e1d7dd451" status: 200 len: 1567 microversion: 2.25 time: 0.357896 Jul 16 21:12:14.648160 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 582/1117] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:12:14 2022] GET /compute/v2.1/servers/0fb5b5b6-1796-4406-b276-326e1d7dd451 => generated 1567 bytes in 369 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:12:14.758748 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-b5911a87-3003-4dac-b78b-96bfef1b110a tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:14.760850 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-b5911a87-3003-4dac-b78b-96bfef1b110a tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] [instance: 2b26717b-355f-417e-8a7c-72c41e1974e1] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:12:14.769458 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b5911a87-3003-4dac-b78b-96bfef1b110a tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:14.770181 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b5911a87-3003-4dac-b78b-96bfef1b110a tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:14.771586 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b5911a87-3003-4dac-b78b-96bfef1b110a tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:14.772259 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b5911a87-3003-4dac-b78b-96bfef1b110a tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:15.164402 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-b5911a87-3003-4dac-b78b-96bfef1b110a tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '034c11cc3f174c6fb44c8d64e55cf8b9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb4ff5d7e42940cdb7be71dadd895ee4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:15.165147 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-e3cdaff4-50fd-4e22-b265-f28853f63a05 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:15.172507 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-e3cdaff4-50fd-4e22-b265-f28853f63a05 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] [instance: 45164bce-7acf-4ef6-8afa-aaab6287ecf4] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:12:15.175674 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-b5911a87-3003-4dac-b78b-96bfef1b110a tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] 10.222.0.16 "GET /compute/v2.1/servers/2b26717b-355f-417e-8a7c-72c41e1974e1" status: 200 len: 1576 microversion: 2.1 time: 0.421169 Jul 16 21:12:15.177046 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 536/1118] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:12:14 2022] GET /compute/v2.1/servers/2b26717b-355f-417e-8a7c-72c41e1974e1 => generated 1576 bytes in 424 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:12:15.192325 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e3cdaff4-50fd-4e22-b265-f28853f63a05 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:15.194550 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e3cdaff4-50fd-4e22-b265-f28853f63a05 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:15.195455 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-e8fa4153-8d8d-43d6-873e-0fbbd17ccc40 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:15.196089 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-e8fa4153-8d8d-43d6-873e-0fbbd17ccc40 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:12:15.202242 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e3cdaff4-50fd-4e22-b265-f28853f63a05 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:15.202934 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e3cdaff4-50fd-4e22-b265-f28853f63a05 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:15.214323 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e8fa4153-8d8d-43d6-873e-0fbbd17ccc40 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:15.215273 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e8fa4153-8d8d-43d6-873e-0fbbd17ccc40 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:15.224017 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e8fa4153-8d8d-43d6-873e-0fbbd17ccc40 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:15.224017 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e8fa4153-8d8d-43d6-873e-0fbbd17ccc40 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:15.326050 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-e3cdaff4-50fd-4e22-b265-f28853f63a05 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'abeb7020019d41a18de42956d706a011', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '649babe54d104436af461f1ee4767001', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:15.334146 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-e3cdaff4-50fd-4e22-b265-f28853f63a05 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] 10.222.0.16 "GET /compute/v2.1/servers/45164bce-7acf-4ef6-8afa-aaab6287ecf4" status: 200 len: 1433 microversion: 2.1 time: 0.174324 Jul 16 21:12:15.344543 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 583/1119] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:12:15 2022] GET /compute/v2.1/servers/45164bce-7acf-4ef6-8afa-aaab6287ecf4 => generated 1433 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:12:15.344543 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-f0966f17-e421-4921-8dd1-a9eb08cf7c5e tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:15.344543 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-f0966f17-e421-4921-8dd1-a9eb08cf7c5e tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] [instance: 2b26717b-355f-417e-8a7c-72c41e1974e1] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:12:15.349951 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f0966f17-e421-4921-8dd1-a9eb08cf7c5e tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:15.350416 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f0966f17-e421-4921-8dd1-a9eb08cf7c5e tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:15.352864 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f0966f17-e421-4921-8dd1-a9eb08cf7c5e tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:15.354431 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f0966f17-e421-4921-8dd1-a9eb08cf7c5e tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:15.539765 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-f0966f17-e421-4921-8dd1-a9eb08cf7c5e tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] 10.222.0.16 "GET /compute/v2.1/servers/2b26717b-355f-417e-8a7c-72c41e1974e1/os-interface" status: 200 len: 285 microversion: 2.1 time: 0.203471 Jul 16 21:12:15.544882 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 584/1120] 10.222.0.16 () {58 vars in 1274 bytes} [Sat Jul 16 21:12:15 2022] GET /compute/v2.1/servers/2b26717b-355f-417e-8a7c-72c41e1974e1/os-interface => generated 285 bytes in 209 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:12:15.555307 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-e8fa4153-8d8d-43d6-873e-0fbbd17ccc40 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b68dacc4b8463ba07fae6372131117', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55a19b7a4682457a9132f5fc80110712', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:15.561966 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-259de6a2-611b-43db-813f-41c618424c1a tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:15.568632 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-259de6a2-611b-43db-813f-41c618424c1a tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] [instance: 4ae3e944-b450-40e5-bb71-c02df03212e3] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:12:15.604011 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-e8fa4153-8d8d-43d6-873e-0fbbd17ccc40 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "GET /compute/v2.1/servers/a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2" status: 200 len: 1469 microversion: 2.1 time: 0.416718 Jul 16 21:12:15.611968 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 537/1121] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:12:15 2022] GET /compute/v2.1/servers/a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2 => generated 1469 bytes in 426 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:12:15.617212 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-0022cf4e-6a62-4721-9bf1-d4d367629eec tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:15.630435 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-259de6a2-611b-43db-813f-41c618424c1a tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:15.630997 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-259de6a2-611b-43db-813f-41c618424c1a tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:15.635366 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-0022cf4e-6a62-4721-9bf1-d4d367629eec tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] [instance: 83b1fe77-ef6d-408d-b255-aa3b60aad8f3] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:12:15.635505 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-259de6a2-611b-43db-813f-41c618424c1a tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:15.636662 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-259de6a2-611b-43db-813f-41c618424c1a tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:15.654810 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0022cf4e-6a62-4721-9bf1-d4d367629eec tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:15.659213 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0022cf4e-6a62-4721-9bf1-d4d367629eec tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:15.661178 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0022cf4e-6a62-4721-9bf1-d4d367629eec tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:15.664632 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0022cf4e-6a62-4721-9bf1-d4d367629eec tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:15.827311 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-259de6a2-611b-43db-813f-41c618424c1a tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2dfcad8d2b57481bb33aa8f5d7483c51', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cda86413e74b4c1e8e09bd4df4926664', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:15.839579 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-259de6a2-611b-43db-813f-41c618424c1a tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] 10.222.0.16 "GET /compute/v2.1/servers/4ae3e944-b450-40e5-bb71-c02df03212e3" status: 200 len: 1281 microversion: 2.1 time: 0.293151 Jul 16 21:12:15.841480 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 585/1122] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:12:15 2022] GET /compute/v2.1/servers/4ae3e944-b450-40e5-bb71-c02df03212e3 => generated 1281 bytes in 296 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:12:15.847909 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-c9a79c53-bacc-45e6-932f-d545af6fccf6 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:15.848709 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-c9a79c53-bacc-45e6-932f-d545af6fccf6 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] [instance: 0fb5b5b6-1796-4406-b276-326e1d7dd451] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:12:15.875261 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c9a79c53-bacc-45e6-932f-d545af6fccf6 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:15.875746 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c9a79c53-bacc-45e6-932f-d545af6fccf6 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:15.879416 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c9a79c53-bacc-45e6-932f-d545af6fccf6 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:15.879882 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c9a79c53-bacc-45e6-932f-d545af6fccf6 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:16.636952 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-0022cf4e-6a62-4721-9bf1-d4d367629eec tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56d4bbcdebc24a34a814d7f6e72fbcfd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7d2b583c48254e17a8082f6467f2ae6d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:16.650256 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-0022cf4e-6a62-4721-9bf1-d4d367629eec tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] 10.222.0.16 "GET /compute/v2.1/servers/83b1fe77-ef6d-408d-b255-aa3b60aad8f3" status: 200 len: 1319 microversion: 2.1 time: 1.036784 Jul 16 21:12:16.655200 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 538/1123] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:12:15 2022] GET /compute/v2.1/servers/83b1fe77-ef6d-408d-b255-aa3b60aad8f3 => generated 1319 bytes in 1042 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:12:16.660090 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-870cf244-04b0-4bce-b33e-a4563e1edef0 req-33885cfc-4614-4f3e-871d-fa351393dea5 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "0fb5b5b6-1796-4406-b276-326e1d7dd451", "name": "network-vif-unplugged", "status": "completed", "tag": "19f08e09-ff8e-43eb-b5f2-421d0eadc41c"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:12:16.691293 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-870cf244-04b0-4bce-b33e-a4563e1edef0 req-33885cfc-4614-4f3e-871d-fa351393dea5 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:16.702578 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-870cf244-04b0-4bce-b33e-a4563e1edef0 req-33885cfc-4614-4f3e-871d-fa351393dea5 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.011s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:16.727756 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-870cf244-04b0-4bce-b33e-a4563e1edef0 req-33885cfc-4614-4f3e-871d-fa351393dea5 service nova] Creating event network-vif-unplugged:19f08e09-ff8e-43eb-b5f2-421d0eadc41c for instance 0fb5b5b6-1796-4406-b276-326e1d7dd451 on nch2-master-1 Jul 16 21:12:16.747433 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-870cf244-04b0-4bce-b33e-a4563e1edef0 req-33885cfc-4614-4f3e-871d-fa351393dea5 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.090780 Jul 16 21:12:16.750235 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 539/1124] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:12:16 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:12:16.767529 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-28934a7d-9a8d-4435-ab36-8f497f4ad3ce req-f5624a16-3b31-42d9-8c70-2e3a162cbc0c service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "45164bce-7acf-4ef6-8afa-aaab6287ecf4", "name": "network-vif-deleted", "tag": "14e29168-1029-4ef2-8854-0a02dfa53951"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:12:16.786960 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-28934a7d-9a8d-4435-ab36-8f497f4ad3ce req-f5624a16-3b31-42d9-8c70-2e3a162cbc0c service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:16.790974 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-28934a7d-9a8d-4435-ab36-8f497f4ad3ce req-f5624a16-3b31-42d9-8c70-2e3a162cbc0c service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:16.807658 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-28934a7d-9a8d-4435-ab36-8f497f4ad3ce req-f5624a16-3b31-42d9-8c70-2e3a162cbc0c service nova] Creating event network-vif-deleted:14e29168-1029-4ef2-8854-0a02dfa53951 for instance 45164bce-7acf-4ef6-8afa-aaab6287ecf4 on nch2-master-1 Jul 16 21:12:16.810246 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-28934a7d-9a8d-4435-ab36-8f497f4ad3ce req-f5624a16-3b31-42d9-8c70-2e3a162cbc0c service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.058202 Jul 16 21:12:16.815071 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 540/1125] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:12:16 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 64 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:12:16.821756 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-54d07af3-066e-466f-aa4d-b5c99beb2557 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "ce0ae7fc-6256-4387-a72f-dd24801e38f0", "device": "/dev/xvdb"}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:12:16.828436 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-54d07af3-066e-466f-aa4d-b5c99beb2557 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] [instance: 6a391156-ffce-49c0-a79d-47b4544264f5] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:12:16.832584 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-54d07af3-066e-466f-aa4d-b5c99beb2557 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:16.833424 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-54d07af3-066e-466f-aa4d-b5c99beb2557 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:16.835039 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-54d07af3-066e-466f-aa4d-b5c99beb2557 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:16.835682 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-54d07af3-066e-466f-aa4d-b5c99beb2557 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:16.846477 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-c9a79c53-bacc-45e6-932f-d545af6fccf6 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:16.860713 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-c9a79c53-bacc-45e6-932f-d545af6fccf6 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:16.863470 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-c9a79c53-bacc-45e6-932f-d545af6fccf6 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:16.865364 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-c9a79c53-bacc-45e6-932f-d545af6fccf6 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] 10.222.0.16 "GET /compute/v2.1/servers/0fb5b5b6-1796-4406-b276-326e1d7dd451" status: 200 len: 1567 microversion: 2.25 time: 1.021523 Jul 16 21:12:16.867103 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 586/1126] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:12:15 2022] GET /compute/v2.1/servers/0fb5b5b6-1796-4406-b276-326e1d7dd451 => generated 1567 bytes in 1024 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:12:16.873364 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-2d4453fe-dab8-4404-b767-915556550f26 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:16.874202 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-2d4453fe-dab8-4404-b767-915556550f26 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] [instance: 45164bce-7acf-4ef6-8afa-aaab6287ecf4] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:12:16.881977 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2d4453fe-dab8-4404-b767-915556550f26 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:16.882672 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2d4453fe-dab8-4404-b767-915556550f26 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:16.883671 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2d4453fe-dab8-4404-b767-915556550f26 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:16.884301 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2d4453fe-dab8-4404-b767-915556550f26 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:16.907629 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-2d4453fe-dab8-4404-b767-915556550f26 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] HTTP exception thrown: Instance 45164bce-7acf-4ef6-8afa-aaab6287ecf4 could not be found. Jul 16 21:12:16.910599 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-2d4453fe-dab8-4404-b767-915556550f26 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Request method failure captured: Jul 16 21:12:16.910599 ncdv-master-1 devstack@n-api.service[138680]: request: GET /compute/v2.1/servers/45164bce-7acf-4ef6-8afa-aaab6287ecf4 HTTP/1.1 Jul 16 21:12:16.910599 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:12:16.910599 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:12:16.910599 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:12:16.910599 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 0 Jul 16 21:12:16.910599 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:12:16.910599 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:12:16.910599 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:12:16.910599 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:12:16.910599 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0yjvVgP0jq68r-ny5Eh1zEbXMYJhEkVpBaJLnFwKzfSzzxAGgQPy49cpkkoW8Rc5r0epWFa_INrkbRxHc5yb1M5XgG50--wLBrLRBZrPuxqH4ynry56OcrGI4F8X9HPf6z5uwbMiaMLyGnsa6t3q-Npn98NzTN2KKNq9Fp4f3zZbeyY Jul 16 21:12:16.910599 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:12:16.910599 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:12:16.910599 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:12:16.910599 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:12:16.910599 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:12:16.910599 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:12:16.910599 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: 649babe54d104436af461f1ee4767001 Jul 16 21:12:16.910599 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-ServersOnMultiNodesTest-1258903790 Jul 16 21:12:16.910599 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:12:16.910599 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:12:16.912166 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/649babe54d104436af461f1ee4767001"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/649babe54d104436af461f1ee4767001"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_649babe54d104436af461f1ee4767001"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/649babe54d104436af461f1ee4767001"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:12:16.912166 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-ServersOnMultiNodesTest-1258903790 Jul 16 21:12:16.912166 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: 649babe54d104436af461f1ee4767001 Jul 16 21:12:16.912166 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-ServersOnMultiNodesTest-1258903790 Jul 16 21:12:16.912166 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-ServersOnMultiNodesTest-1258903790-project Jul 16 21:12:16.912166 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:12:16.912166 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:12:16.912166 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: abeb7020019d41a18de42956d706a011 Jul 16 21:12:16.912166 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-ServersOnMultiNodesTest-1258903790-project Jul 16 21:12:16.912166 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 21:12:16.912166 ncdv-master-1 devstack@n-api.service[138680]: exception: Instance 45164bce-7acf-4ef6-8afa-aaab6287ecf4 could not be found. Jul 16 21:12:16.912166 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': '45164bce-7acf-4ef6-8afa-aaab6287ecf4'} Jul 16 21:12:16.913123 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:12:16.913123 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:12:16.913123 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:12:16.913123 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:12:16.913123 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:12:16.913123 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:12:16.913123 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:12:16.913123 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:12:16.913123 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:12:16.913123 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:12:16.913123 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 45164bce-7acf-4ef6-8afa-aaab6287ecf4 could not be found. Jul 16 21:12:16.913123 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:12:16.913123 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:12:16.913123 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:12:16.913123 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:12:16.913123 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:12:16.913123 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:12:16.913123 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:12:16.913123 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:12:16.913123 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:12:16.914520 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:12:16.914520 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:12:16.914520 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:12:16.914520 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:12:16.914520 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:12:16.914520 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:12:16.914520 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:12:16.914520 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 45164bce-7acf-4ef6-8afa-aaab6287ecf4 could not be found. Jul 16 21:12:16.914520 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:12:16.914520 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:12:16.914520 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:12:16.914520 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:12:16.914520 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:12:16.914520 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:12:16.914520 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:12:16.914520 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:12:16.914520 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 45164bce-7acf-4ef6-8afa-aaab6287ecf4 could not be found. Jul 16 21:12:16.914520 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:12:16.920095 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-2d4453fe-dab8-4404-b767-915556550f26 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Returning 404 to user: Instance 45164bce-7acf-4ef6-8afa-aaab6287ecf4 could not be found. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:12:16.921000 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-2d4453fe-dab8-4404-b767-915556550f26 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] 10.222.0.16 "GET /compute/v2.1/servers/45164bce-7acf-4ef6-8afa-aaab6287ecf4" status: 404 len: 111 microversion: 2.1 time: 0.052626 Jul 16 21:12:16.924375 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 587/1127] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:12:16 2022] GET /compute/v2.1/servers/45164bce-7acf-4ef6-8afa-aaab6287ecf4 => generated 111 bytes in 56 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:12:16.954223 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-f420fe42-3d33-4867-bc4b-defa4197f46a req-67b69dbe-6d53-4876-8c8f-9386e14d66d2 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "83b1fe77-ef6d-408d-b255-aa3b60aad8f3", "name": "network-vif-plugged", "status": "completed", "tag": "fef3d648-18d7-4308-9bcb-4555c790a3c7"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:12:16.969253 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-f420fe42-3d33-4867-bc4b-defa4197f46a req-67b69dbe-6d53-4876-8c8f-9386e14d66d2 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:16.969731 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-f420fe42-3d33-4867-bc4b-defa4197f46a req-67b69dbe-6d53-4876-8c8f-9386e14d66d2 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:16.997697 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-f420fe42-3d33-4867-bc4b-defa4197f46a req-67b69dbe-6d53-4876-8c8f-9386e14d66d2 service nova] Creating event network-vif-plugged:fef3d648-18d7-4308-9bcb-4555c790a3c7 for instance 83b1fe77-ef6d-408d-b255-aa3b60aad8f3 on nch1-master-1 Jul 16 21:12:17.003536 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-f420fe42-3d33-4867-bc4b-defa4197f46a req-67b69dbe-6d53-4876-8c8f-9386e14d66d2 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.075490 Jul 16 21:12:17.004756 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 588/1128] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:12:16 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 80 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:12:17.011218 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-0024f901-5997-4539-8709-b35614acee08 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:17.012263 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-0024f901-5997-4539-8709-b35614acee08 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:12:17.021266 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0024f901-5997-4539-8709-b35614acee08 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:17.022605 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0024f901-5997-4539-8709-b35614acee08 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:17.023658 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0024f901-5997-4539-8709-b35614acee08 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:17.024283 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0024f901-5997-4539-8709-b35614acee08 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:17.510590 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell0:SELECT=3 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:12:17.713378 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-0024f901-5997-4539-8709-b35614acee08 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b68dacc4b8463ba07fae6372131117', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55a19b7a4682457a9132f5fc80110712', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:17.722723 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-0024f901-5997-4539-8709-b35614acee08 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "GET /compute/v2.1/servers/a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2" status: 200 len: 1535 microversion: 2.1 time: 0.716450 Jul 16 21:12:17.726431 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 589/1129] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:12:17 2022] GET /compute/v2.1/servers/a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2 => generated 1535 bytes in 721 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:12:17.733196 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-8b86008a-2f4b-4d3d-8644-17b5bbdbe763 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:17.733833 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-8b86008a-2f4b-4d3d-8644-17b5bbdbe763 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] [instance: 4ae3e944-b450-40e5-bb71-c02df03212e3] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:12:17.741141 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8b86008a-2f4b-4d3d-8644-17b5bbdbe763 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:17.746495 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8b86008a-2f4b-4d3d-8644-17b5bbdbe763 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:17.747884 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8b86008a-2f4b-4d3d-8644-17b5bbdbe763 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:17.748729 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8b86008a-2f4b-4d3d-8644-17b5bbdbe763 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:18.107823 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-54d07af3-066e-466f-aa4d-b5c99beb2557 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] 10.222.0.16 "POST /compute/v2.1/servers/6a391156-ffce-49c0-a79d-47b4544264f5/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 1.282079 Jul 16 21:12:18.107823 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 541/1130] 10.222.0.16 () {60 vars in 1322 bytes} [Sat Jul 16 21:12:16 2022] POST /compute/v2.1/servers/6a391156-ffce-49c0-a79d-47b4544264f5/os-volume_attachments => generated 194 bytes in 1285 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:12:18.121230 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-d244f8b9-0186-4c52-ace4-9ea94f8214ee tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:18.122272 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-d244f8b9-0186-4c52-ace4-9ea94f8214ee tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] [instance: 944f68af-bf99-428f-9357-9ce9f807483b] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:12:18.137206 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d244f8b9-0186-4c52-ace4-9ea94f8214ee tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:18.141104 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d244f8b9-0186-4c52-ace4-9ea94f8214ee tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:18.142024 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d244f8b9-0186-4c52-ace4-9ea94f8214ee tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:18.142646 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d244f8b9-0186-4c52-ace4-9ea94f8214ee tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:18.180223 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-8b86008a-2f4b-4d3d-8644-17b5bbdbe763 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2dfcad8d2b57481bb33aa8f5d7483c51', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cda86413e74b4c1e8e09bd4df4926664', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:18.185576 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-d244f8b9-0186-4c52-ace4-9ea94f8214ee tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] HTTP exception thrown: Instance 944f68af-bf99-428f-9357-9ce9f807483b could not be found. Jul 16 21:12:18.188100 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-d244f8b9-0186-4c52-ace4-9ea94f8214ee tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Request method failure captured: Jul 16 21:12:18.188100 ncdv-master-1 devstack@n-api.service[138681]: request: GET /compute/v2.1/servers/944f68af-bf99-428f-9357-9ce9f807483b HTTP/1.1 Jul 16 21:12:18.188100 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:12:18.188100 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:12:18.188100 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:12:18.188100 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 21:12:18.188100 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:12:18.188100 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:12:18.188100 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:12:18.188100 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:12:18.188100 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0yjvVgP0jq68r-ny5Eh1zEbXMYJhEkVpBaJLnFwKzfSzzxAGgQPy49cpkkoW8Rc5r0epWFa_INrkbRxHc5yb1M5XgG50--wLBrLRBZrPuxqH4ynry56OcrGI4F8X9HPf6z5uwbMiaMLyGnsa6t3q-Npn98NzTN2KKNq9Fp4f3zZbeyY Jul 16 21:12:18.188100 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:12:18.188100 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:12:18.188100 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:12:18.188100 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:12:18.188100 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:12:18.188100 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:12:18.188100 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: 649babe54d104436af461f1ee4767001 Jul 16 21:12:18.188100 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-ServersOnMultiNodesTest-1258903790 Jul 16 21:12:18.188100 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:12:18.188100 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:12:18.189646 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/649babe54d104436af461f1ee4767001"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/649babe54d104436af461f1ee4767001"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_649babe54d104436af461f1ee4767001"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/649babe54d104436af461f1ee4767001"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:12:18.189646 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-ServersOnMultiNodesTest-1258903790 Jul 16 21:12:18.189646 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: 649babe54d104436af461f1ee4767001 Jul 16 21:12:18.189646 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-ServersOnMultiNodesTest-1258903790 Jul 16 21:12:18.189646 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-ServersOnMultiNodesTest-1258903790-project Jul 16 21:12:18.189646 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:12:18.189646 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:12:18.189646 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: abeb7020019d41a18de42956d706a011 Jul 16 21:12:18.189646 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-ServersOnMultiNodesTest-1258903790-project Jul 16 21:12:18.189646 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:12:18.189646 ncdv-master-1 devstack@n-api.service[138681]: exception: Instance 944f68af-bf99-428f-9357-9ce9f807483b could not be found. Jul 16 21:12:18.189646 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': '944f68af-bf99-428f-9357-9ce9f807483b'} Jul 16 21:12:18.190433 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:12:18.190433 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:12:18.190433 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:12:18.190433 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:12:18.190433 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:12:18.190433 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:12:18.190433 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:12:18.190433 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:12:18.190433 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:12:18.190433 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:12:18.190433 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 944f68af-bf99-428f-9357-9ce9f807483b could not be found. Jul 16 21:12:18.190433 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:12:18.190433 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:12:18.190433 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:12:18.190433 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:12:18.190433 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:12:18.190433 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:12:18.190433 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:12:18.190433 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:12:18.190433 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:12:18.194947 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:12:18.194947 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:12:18.194947 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:12:18.194947 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:12:18.194947 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:12:18.194947 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:12:18.194947 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:12:18.194947 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 944f68af-bf99-428f-9357-9ce9f807483b could not be found. Jul 16 21:12:18.194947 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:12:18.194947 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:12:18.194947 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:12:18.194947 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:12:18.194947 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:12:18.194947 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:12:18.194947 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:12:18.194947 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:12:18.194947 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 944f68af-bf99-428f-9357-9ce9f807483b could not be found. Jul 16 21:12:18.194947 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:12:18.196371 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-8b86008a-2f4b-4d3d-8644-17b5bbdbe763 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] 10.222.0.16 "GET /compute/v2.1/servers/4ae3e944-b450-40e5-bb71-c02df03212e3" status: 200 len: 1323 microversion: 2.1 time: 0.464159 Jul 16 21:12:18.196371 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 590/1131] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:12:17 2022] GET /compute/v2.1/servers/4ae3e944-b450-40e5-bb71-c02df03212e3 => generated 1323 bytes in 466 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:12:18.203453 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-d244f8b9-0186-4c52-ace4-9ea94f8214ee tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Returning 404 to user: Instance 944f68af-bf99-428f-9357-9ce9f807483b could not be found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:12:18.204786 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-d244f8b9-0186-4c52-ace4-9ea94f8214ee tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] 10.222.0.16 "GET /compute/v2.1/servers/944f68af-bf99-428f-9357-9ce9f807483b" status: 404 len: 111 microversion: 2.1 time: 0.102283 Jul 16 21:12:18.208214 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 542/1132] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:12:18 2022] GET /compute/v2.1/servers/944f68af-bf99-428f-9357-9ce9f807483b => generated 111 bytes in 106 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:12:18.213685 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-609c4d08-f169-4f2e-b69f-e4486c64467e tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:18.214324 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-609c4d08-f169-4f2e-b69f-e4486c64467e tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] [instance: 83b1fe77-ef6d-408d-b255-aa3b60aad8f3] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:12:18.215938 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-651f39d7-cdb4-4a4c-a8ed-fdd62bc48725 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:18.216750 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-651f39d7-cdb4-4a4c-a8ed-fdd62bc48725 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] [instance: 0fb5b5b6-1796-4406-b276-326e1d7dd451] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:12:18.227129 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-651f39d7-cdb4-4a4c-a8ed-fdd62bc48725 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:18.227778 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-651f39d7-cdb4-4a4c-a8ed-fdd62bc48725 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:18.229431 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-651f39d7-cdb4-4a4c-a8ed-fdd62bc48725 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:18.239294 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-651f39d7-cdb4-4a4c-a8ed-fdd62bc48725 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:18.245524 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-609c4d08-f169-4f2e-b69f-e4486c64467e tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:18.245988 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-609c4d08-f169-4f2e-b69f-e4486c64467e tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:18.251272 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-609c4d08-f169-4f2e-b69f-e4486c64467e tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:18.251743 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-609c4d08-f169-4f2e-b69f-e4486c64467e tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:18.310616 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell0:SELECT=14 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:12:18.741395 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-651f39d7-cdb4-4a4c-a8ed-fdd62bc48725 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:18.752202 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-651f39d7-cdb4-4a4c-a8ed-fdd62bc48725 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:18.753981 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-651f39d7-cdb4-4a4c-a8ed-fdd62bc48725 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:18.755454 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-651f39d7-cdb4-4a4c-a8ed-fdd62bc48725 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] 10.222.0.16 "GET /compute/v2.1/servers/0fb5b5b6-1796-4406-b276-326e1d7dd451" status: 200 len: 1567 microversion: 2.25 time: 0.545786 Jul 16 21:12:18.767512 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 543/1133] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:12:18 2022] GET /compute/v2.1/servers/0fb5b5b6-1796-4406-b276-326e1d7dd451 => generated 1567 bytes in 558 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:12:18.776630 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-65ec82d2-4024-4ff9-9246-a4a20cca1914 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:18.838947 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-609c4d08-f169-4f2e-b69f-e4486c64467e tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56d4bbcdebc24a34a814d7f6e72fbcfd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7d2b583c48254e17a8082f6467f2ae6d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:18.847915 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-609c4d08-f169-4f2e-b69f-e4486c64467e tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] 10.222.0.16 "GET /compute/v2.1/servers/83b1fe77-ef6d-408d-b255-aa3b60aad8f3" status: 200 len: 1319 microversion: 2.1 time: 0.654039 Jul 16 21:12:18.848674 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 591/1134] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:12:18 2022] GET /compute/v2.1/servers/83b1fe77-ef6d-408d-b255-aa3b60aad8f3 => generated 1319 bytes in 655 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:12:18.854146 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-7b3d5b5e-05f2-46d3-aafb-d99d4ef81b77 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:18.854795 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-7b3d5b5e-05f2-46d3-aafb-d99d4ef81b77 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:12:18.861285 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7b3d5b5e-05f2-46d3-aafb-d99d4ef81b77 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:18.861989 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7b3d5b5e-05f2-46d3-aafb-d99d4ef81b77 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:18.862776 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7b3d5b5e-05f2-46d3-aafb-d99d4ef81b77 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:18.863330 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7b3d5b5e-05f2-46d3-aafb-d99d4ef81b77 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:18.911327 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-65ec82d2-4024-4ff9-9246-a4a20cca1914 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] 10.222.0.16 "DELETE /compute/v2.1/os-server-groups/73f6bd9e-6a32-47fd-a7a1-a896ecc028e4" status: 204 len: 0 microversion: 2.1 time: 0.141598 Jul 16 21:12:18.911907 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 544/1135] 10.222.0.16 () {58 vars in 1265 bytes} [Sat Jul 16 21:12:18 2022] DELETE /compute/v2.1/os-server-groups/73f6bd9e-6a32-47fd-a7a1-a896ecc028e4 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 16 21:12:18.920411 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-f73b9c4d-0163-40e0-859f-bbd88b335f2f req-e09c8f8e-d3cf-416f-83f0-0d83230dd9e8 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2", "name": "network-vif-unplugged", "status": "completed", "tag": "bf9d54d7-5718-4678-bc2f-5ba8241bd855"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:12:18.929047 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-f73b9c4d-0163-40e0-859f-bbd88b335f2f req-e09c8f8e-d3cf-416f-83f0-0d83230dd9e8 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:18.929748 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-f73b9c4d-0163-40e0-859f-bbd88b335f2f req-e09c8f8e-d3cf-416f-83f0-0d83230dd9e8 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:18.945341 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-f73b9c4d-0163-40e0-859f-bbd88b335f2f req-e09c8f8e-d3cf-416f-83f0-0d83230dd9e8 service nova] Creating event network-vif-unplugged:bf9d54d7-5718-4678-bc2f-5ba8241bd855 for instance a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2 on nch1-master-1 Jul 16 21:12:18.953048 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [req-f73b9c4d-0163-40e0-859f-bbd88b335f2f req-e09c8f8e-d3cf-416f-83f0-0d83230dd9e8 service nova] Instance a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2 is migrating within the same cell, copying events to all relevant hosts: {'nch2-master-1', 'nch1-master-1'} {{(pid=138681) _get_relevant_hosts /opt/stack/nova/nova/compute/api.py:5807}} Jul 16 21:12:18.979279 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-f73b9c4d-0163-40e0-859f-bbd88b335f2f req-e09c8f8e-d3cf-416f-83f0-0d83230dd9e8 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.062577 Jul 16 21:12:18.980058 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 545/1136] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:12:18 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 67 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:12:18.987542 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-368b9ec0-3c38-4228-bb25-24a3bf83f448 req-2e61955c-a260-4bdc-bd50-048f39c7914f service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "944f68af-bf99-428f-9357-9ce9f807483b", "name": "network-vif-deleted", "tag": "5556f1e0-fe84-46bd-9585-73295be09188"}, {"name": "network-changed", "server_uuid": "4ae3e944-b450-40e5-bb71-c02df03212e3", "tag": "578b9284-5794-41e6-80d7-b5c49283c6f9"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:12:19.005341 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-368b9ec0-3c38-4228-bb25-24a3bf83f448 req-2e61955c-a260-4bdc-bd50-048f39c7914f service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:19.006026 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-368b9ec0-3c38-4228-bb25-24a3bf83f448 req-2e61955c-a260-4bdc-bd50-048f39c7914f service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:19.021118 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-368b9ec0-3c38-4228-bb25-24a3bf83f448 req-2e61955c-a260-4bdc-bd50-048f39c7914f service nova] Creating event network-vif-deleted:5556f1e0-fe84-46bd-9585-73295be09188 for instance 944f68af-bf99-428f-9357-9ce9f807483b on nch2-master-1 Jul 16 21:12:19.022032 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-368b9ec0-3c38-4228-bb25-24a3bf83f448 req-2e61955c-a260-4bdc-bd50-048f39c7914f service nova] Creating event network-changed:578b9284-5794-41e6-80d7-b5c49283c6f9 for instance 4ae3e944-b450-40e5-bb71-c02df03212e3 on nch2-master-1 Jul 16 21:12:19.029084 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-368b9ec0-3c38-4228-bb25-24a3bf83f448 req-2e61955c-a260-4bdc-bd50-048f39c7914f service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 350 microversion: 2.1 time: 0.047244 Jul 16 21:12:19.029704 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 546/1137] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:12:18 2022] POST /compute/v2.1/os-server-external-events => generated 350 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:12:19.038599 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-666f9acc-5f0b-44a1-bbb8-ad9ed0852ac9 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:19.038599 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-666f9acc-5f0b-44a1-bbb8-ad9ed0852ac9 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] [instance: 148d5ad9-2900-4c2b-909b-aa446757e33d] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:12:19.045575 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-666f9acc-5f0b-44a1-bbb8-ad9ed0852ac9 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:19.046073 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-666f9acc-5f0b-44a1-bbb8-ad9ed0852ac9 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:19.046875 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-666f9acc-5f0b-44a1-bbb8-ad9ed0852ac9 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:19.047347 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-666f9acc-5f0b-44a1-bbb8-ad9ed0852ac9 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:19.077967 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-666f9acc-5f0b-44a1-bbb8-ad9ed0852ac9 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] [instance: 148d5ad9-2900-4c2b-909b-aa446757e33d] Going to try to terminate instance {{(pid=138681) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 16 21:12:19.268053 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-7b3d5b5e-05f2-46d3-aafb-d99d4ef81b77 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b68dacc4b8463ba07fae6372131117', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55a19b7a4682457a9132f5fc80110712', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:19.277930 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-7b3d5b5e-05f2-46d3-aafb-d99d4ef81b77 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "GET /compute/v2.1/servers/a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2" status: 200 len: 1535 microversion: 2.1 time: 0.427947 Jul 16 21:12:19.292087 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 592/1138] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:12:18 2022] GET /compute/v2.1/servers/a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2 => generated 1535 bytes in 443 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:12:19.297041 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-15e237c5-0752-44c4-ae7d-5f32ce0502f2 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:19.297884 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-15e237c5-0752-44c4-ae7d-5f32ce0502f2 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] [instance: 4ae3e944-b450-40e5-bb71-c02df03212e3] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:12:19.304450 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-15e237c5-0752-44c4-ae7d-5f32ce0502f2 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:19.304900 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-15e237c5-0752-44c4-ae7d-5f32ce0502f2 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:19.305682 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-15e237c5-0752-44c4-ae7d-5f32ce0502f2 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:19.306114 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-15e237c5-0752-44c4-ae7d-5f32ce0502f2 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:19.401216 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-666f9acc-5f0b-44a1-bbb8-ad9ed0852ac9 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] 10.222.0.16 "DELETE /compute/v2.1/servers/148d5ad9-2900-4c2b-909b-aa446757e33d" status: 204 len: 0 microversion: 2.1 time: 0.369721 Jul 16 21:12:19.403267 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 547/1139] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:12:19 2022] DELETE /compute/v2.1/servers/148d5ad9-2900-4c2b-909b-aa446757e33d => generated 0 bytes in 372 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 16 21:12:19.412385 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-48249590-96ea-4805-8e3f-b1f046f2fab3 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:19.413488 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-48249590-96ea-4805-8e3f-b1f046f2fab3 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] [instance: 148d5ad9-2900-4c2b-909b-aa446757e33d] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:12:19.422472 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-48249590-96ea-4805-8e3f-b1f046f2fab3 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:19.423344 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-48249590-96ea-4805-8e3f-b1f046f2fab3 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:19.424412 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-48249590-96ea-4805-8e3f-b1f046f2fab3 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:19.425111 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-48249590-96ea-4805-8e3f-b1f046f2fab3 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:20.176293 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-15e237c5-0752-44c4-ae7d-5f32ce0502f2 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2dfcad8d2b57481bb33aa8f5d7483c51', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cda86413e74b4c1e8e09bd4df4926664', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:20.185590 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-15e237c5-0752-44c4-ae7d-5f32ce0502f2 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] 10.222.0.16 "GET /compute/v2.1/servers/4ae3e944-b450-40e5-bb71-c02df03212e3" status: 200 len: 1323 microversion: 2.1 time: 0.892162 Jul 16 21:12:20.195376 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 593/1140] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:12:19 2022] GET /compute/v2.1/servers/4ae3e944-b450-40e5-bb71-c02df03212e3 => generated 1323 bytes in 902 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:12:20.201119 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-5d47b516-d7f9-459e-a3d6-bc5929e0f485 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:20.201931 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-5d47b516-d7f9-459e-a3d6-bc5929e0f485 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] [instance: 0fb5b5b6-1796-4406-b276-326e1d7dd451] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:12:20.208411 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5d47b516-d7f9-459e-a3d6-bc5929e0f485 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:20.209084 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5d47b516-d7f9-459e-a3d6-bc5929e0f485 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:20.211053 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5d47b516-d7f9-459e-a3d6-bc5929e0f485 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:20.211661 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5d47b516-d7f9-459e-a3d6-bc5929e0f485 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:20.267931 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-48249590-96ea-4805-8e3f-b1f046f2fab3 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'abeb7020019d41a18de42956d706a011', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '649babe54d104436af461f1ee4767001', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:20.276408 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-48249590-96ea-4805-8e3f-b1f046f2fab3 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] 10.222.0.16 "GET /compute/v2.1/servers/148d5ad9-2900-4c2b-909b-aa446757e33d" status: 200 len: 1472 microversion: 2.1 time: 0.869145 Jul 16 21:12:20.286854 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 548/1141] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:12:19 2022] GET /compute/v2.1/servers/148d5ad9-2900-4c2b-909b-aa446757e33d => generated 1472 bytes in 880 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:12:20.296058 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-d39a5cca-8593-4c25-8d45-f4846b799600 tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:20.297032 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-d39a5cca-8593-4c25-8d45-f4846b799600 tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] [instance: 83b1fe77-ef6d-408d-b255-aa3b60aad8f3] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:12:20.309678 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d39a5cca-8593-4c25-8d45-f4846b799600 tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:20.310751 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d39a5cca-8593-4c25-8d45-f4846b799600 tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:20.311884 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d39a5cca-8593-4c25-8d45-f4846b799600 tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:20.312562 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d39a5cca-8593-4c25-8d45-f4846b799600 tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:20.645273 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-5d47b516-d7f9-459e-a3d6-bc5929e0f485 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:20.656374 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-5d47b516-d7f9-459e-a3d6-bc5929e0f485 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:20.658140 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-5d47b516-d7f9-459e-a3d6-bc5929e0f485 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:20.664595 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-5d47b516-d7f9-459e-a3d6-bc5929e0f485 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] 10.222.0.16 "GET /compute/v2.1/servers/0fb5b5b6-1796-4406-b276-326e1d7dd451" status: 200 len: 1567 microversion: 2.25 time: 0.463028 Jul 16 21:12:20.664595 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 594/1142] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:12:20 2022] GET /compute/v2.1/servers/0fb5b5b6-1796-4406-b276-326e1d7dd451 => generated 1567 bytes in 464 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:12:20.664901 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-870739b9-7726-4fb1-9b8e-0d13c3b2394b tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:20.665678 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-870739b9-7726-4fb1-9b8e-0d13c3b2394b tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] [instance: 2b26717b-355f-417e-8a7c-72c41e1974e1] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:12:20.672753 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-870739b9-7726-4fb1-9b8e-0d13c3b2394b tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:20.673201 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-870739b9-7726-4fb1-9b8e-0d13c3b2394b tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:20.673980 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-870739b9-7726-4fb1-9b8e-0d13c3b2394b tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:20.674544 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-870739b9-7726-4fb1-9b8e-0d13c3b2394b tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:20.702789 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-d39a5cca-8593-4c25-8d45-f4846b799600 tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56d4bbcdebc24a34a814d7f6e72fbcfd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7d2b583c48254e17a8082f6467f2ae6d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:20.710798 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-d39a5cca-8593-4c25-8d45-f4846b799600 tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] 10.222.0.16 "GET /compute/v2.1/servers/83b1fe77-ef6d-408d-b255-aa3b60aad8f3" status: 200 len: 1467 microversion: 2.1 time: 0.422453 Jul 16 21:12:20.723138 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 549/1143] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:12:20 2022] GET /compute/v2.1/servers/83b1fe77-ef6d-408d-b255-aa3b60aad8f3 => generated 1467 bytes in 434 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:12:20.730333 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-192352a4-2c26-4000-bed4-091bf3fe655a tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:20.730955 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-192352a4-2c26-4000-bed4-091bf3fe655a tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:12:20.738309 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-192352a4-2c26-4000-bed4-091bf3fe655a tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:20.738758 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-192352a4-2c26-4000-bed4-091bf3fe655a tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:20.739570 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-192352a4-2c26-4000-bed4-091bf3fe655a tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:20.742937 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-192352a4-2c26-4000-bed4-091bf3fe655a tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:20.959746 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-870739b9-7726-4fb1-9b8e-0d13c3b2394b tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '034c11cc3f174c6fb44c8d64e55cf8b9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb4ff5d7e42940cdb7be71dadd895ee4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:20.967573 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-870739b9-7726-4fb1-9b8e-0d13c3b2394b tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] 10.222.0.16 "GET /compute/v2.1/servers/2b26717b-355f-417e-8a7c-72c41e1974e1" status: 200 len: 1576 microversion: 2.1 time: 0.306286 Jul 16 21:12:20.971464 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-192352a4-2c26-4000-bed4-091bf3fe655a tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b68dacc4b8463ba07fae6372131117', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55a19b7a4682457a9132f5fc80110712', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:20.979181 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-192352a4-2c26-4000-bed4-091bf3fe655a tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "GET /compute/v2.1/servers/a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2" status: 200 len: 1535 microversion: 2.1 time: 0.254903 Jul 16 21:12:20.980001 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 595/1144] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:12:20 2022] GET /compute/v2.1/servers/2b26717b-355f-417e-8a7c-72c41e1974e1 => generated 1576 bytes in 319 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:12:20.994292 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 550/1145] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:12:20 2022] GET /compute/v2.1/servers/a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2 => generated 1535 bytes in 267 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:12:21.001952 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-20302538-2582-424e-bffb-3283d3ae1f18 req-ce76d0ea-fb2d-4181-95c2-19a6dcce2a9e service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2", "tag": "bf9d54d7-5718-4678-bc2f-5ba8241bd855"}, {"name": "network-changed", "server_uuid": "2b26717b-355f-417e-8a7c-72c41e1974e1", "tag": "c64dc016-0b59-43fd-8f3b-dcfc4298e2bd"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:12:21.012963 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-20302538-2582-424e-bffb-3283d3ae1f18 req-ce76d0ea-fb2d-4181-95c2-19a6dcce2a9e service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:21.013743 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-20302538-2582-424e-bffb-3283d3ae1f18 req-ce76d0ea-fb2d-4181-95c2-19a6dcce2a9e service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:21.033823 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-20302538-2582-424e-bffb-3283d3ae1f18 req-ce76d0ea-fb2d-4181-95c2-19a6dcce2a9e service nova] Creating event network-changed:bf9d54d7-5718-4678-bc2f-5ba8241bd855 for instance a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2 on nch1-master-1 Jul 16 21:12:21.033823 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-20302538-2582-424e-bffb-3283d3ae1f18 req-ce76d0ea-fb2d-4181-95c2-19a6dcce2a9e service nova] Creating event network-changed:c64dc016-0b59-43fd-8f3b-dcfc4298e2bd for instance 2b26717b-355f-417e-8a7c-72c41e1974e1 on nch2-master-1 Jul 16 21:12:21.038442 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [req-20302538-2582-424e-bffb-3283d3ae1f18 req-ce76d0ea-fb2d-4181-95c2-19a6dcce2a9e service nova] Instance a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2 is migrating within the same cell, copying events to all relevant hosts: {'nch2-master-1', 'nch1-master-1'} {{(pid=138681) _get_relevant_hosts /opt/stack/nova/nova/compute/api.py:5807}} Jul 16 21:12:21.062799 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-20302538-2582-424e-bffb-3283d3ae1f18 req-ce76d0ea-fb2d-4181-95c2-19a6dcce2a9e service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 346 microversion: 2.1 time: 0.056838 Jul 16 21:12:21.062799 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 551/1146] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:12:20 2022] POST /compute/v2.1/os-server-external-events => generated 346 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:12:21.081113 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-c9d4e681-45f5-4f31-8fba-3326382ace39 tempest-DeleteServersAdminTestJSON-2093208747 tempest-DeleteServersAdminTestJSON-2093208747-project-admin] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:21.081616 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-c9d4e681-45f5-4f31-8fba-3326382ace39 tempest-DeleteServersAdminTestJSON-2093208747 tempest-DeleteServersAdminTestJSON-2093208747-project-admin] [instance: 83b1fe77-ef6d-408d-b255-aa3b60aad8f3] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:12:21.087995 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c9d4e681-45f5-4f31-8fba-3326382ace39 tempest-DeleteServersAdminTestJSON-2093208747 tempest-DeleteServersAdminTestJSON-2093208747-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:21.088451 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c9d4e681-45f5-4f31-8fba-3326382ace39 tempest-DeleteServersAdminTestJSON-2093208747 tempest-DeleteServersAdminTestJSON-2093208747-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:21.089266 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c9d4e681-45f5-4f31-8fba-3326382ace39 tempest-DeleteServersAdminTestJSON-2093208747 tempest-DeleteServersAdminTestJSON-2093208747-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:21.089745 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c9d4e681-45f5-4f31-8fba-3326382ace39 tempest-DeleteServersAdminTestJSON-2093208747 tempest-DeleteServersAdminTestJSON-2093208747-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:21.115125 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-c9d4e681-45f5-4f31-8fba-3326382ace39 tempest-DeleteServersAdminTestJSON-2093208747 tempest-DeleteServersAdminTestJSON-2093208747-project-admin] [instance: 83b1fe77-ef6d-408d-b255-aa3b60aad8f3] Going to try to terminate instance {{(pid=138680) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 16 21:12:21.203131 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-cb3e6a92-e61a-48d3-9a5e-398abdc45ca1 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:21.203889 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-cb3e6a92-e61a-48d3-9a5e-398abdc45ca1 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] [instance: 4ae3e944-b450-40e5-bb71-c02df03212e3] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:12:21.210138 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cb3e6a92-e61a-48d3-9a5e-398abdc45ca1 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:21.210616 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cb3e6a92-e61a-48d3-9a5e-398abdc45ca1 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:21.211418 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cb3e6a92-e61a-48d3-9a5e-398abdc45ca1 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:21.211894 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cb3e6a92-e61a-48d3-9a5e-398abdc45ca1 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:21.229567 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=95,nova_cell1:UPDATE=3,nova_cell1:INSERT=2 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:12:21.357139 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-cb3e6a92-e61a-48d3-9a5e-398abdc45ca1 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2dfcad8d2b57481bb33aa8f5d7483c51', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cda86413e74b4c1e8e09bd4df4926664', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:21.365224 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-cb3e6a92-e61a-48d3-9a5e-398abdc45ca1 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] 10.222.0.16 "GET /compute/v2.1/servers/4ae3e944-b450-40e5-bb71-c02df03212e3" status: 200 len: 1323 microversion: 2.1 time: 0.166394 Jul 16 21:12:21.365818 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 552/1147] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:12:21 2022] GET /compute/v2.1/servers/4ae3e944-b450-40e5-bb71-c02df03212e3 => generated 1323 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:12:21.371725 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-3146641f-5689-48f6-84a1-e5e36a32dd1b tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:21.372423 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-3146641f-5689-48f6-84a1-e5e36a32dd1b tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] [instance: 148d5ad9-2900-4c2b-909b-aa446757e33d] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:12:21.379093 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3146641f-5689-48f6-84a1-e5e36a32dd1b tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:21.379544 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3146641f-5689-48f6-84a1-e5e36a32dd1b tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:21.380318 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3146641f-5689-48f6-84a1-e5e36a32dd1b tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:21.380758 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3146641f-5689-48f6-84a1-e5e36a32dd1b tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:21.418066 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-c9d4e681-45f5-4f31-8fba-3326382ace39 tempest-DeleteServersAdminTestJSON-2093208747 tempest-DeleteServersAdminTestJSON-2093208747-project-admin] 10.222.0.16 "DELETE /compute/v2.1/servers/83b1fe77-ef6d-408d-b255-aa3b60aad8f3" status: 204 len: 0 microversion: 2.1 time: 0.436495 Jul 16 21:12:21.418765 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 596/1148] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:12:20 2022] DELETE /compute/v2.1/servers/83b1fe77-ef6d-408d-b255-aa3b60aad8f3 => generated 0 bytes in 438 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 16 21:12:21.428960 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-57b90aa1-60e7-4f59-a082-f54393522a50 tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:21.429674 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-57b90aa1-60e7-4f59-a082-f54393522a50 tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] [instance: 83b1fe77-ef6d-408d-b255-aa3b60aad8f3] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:12:21.436352 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-57b90aa1-60e7-4f59-a082-f54393522a50 tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:21.437033 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-57b90aa1-60e7-4f59-a082-f54393522a50 tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:21.443857 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-57b90aa1-60e7-4f59-a082-f54393522a50 tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:21.443857 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-57b90aa1-60e7-4f59-a082-f54393522a50 tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:21.700802 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-3146641f-5689-48f6-84a1-e5e36a32dd1b tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'abeb7020019d41a18de42956d706a011', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '649babe54d104436af461f1ee4767001', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:21.708351 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-3146641f-5689-48f6-84a1-e5e36a32dd1b tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] 10.222.0.16 "GET /compute/v2.1/servers/148d5ad9-2900-4c2b-909b-aa446757e33d" status: 200 len: 1472 microversion: 2.1 time: 0.341380 Jul 16 21:12:21.708962 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 553/1149] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:12:21 2022] GET /compute/v2.1/servers/148d5ad9-2900-4c2b-909b-aa446757e33d => generated 1472 bytes in 342 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:12:21.713874 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-4805962b-b99e-4e10-983c-6b7d772f4922 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:21.714635 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-4805962b-b99e-4e10-983c-6b7d772f4922 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] [instance: 0fb5b5b6-1796-4406-b276-326e1d7dd451] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:12:21.715253 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-57b90aa1-60e7-4f59-a082-f54393522a50 tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56d4bbcdebc24a34a814d7f6e72fbcfd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7d2b583c48254e17a8082f6467f2ae6d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:21.721064 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4805962b-b99e-4e10-983c-6b7d772f4922 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:21.721518 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4805962b-b99e-4e10-983c-6b7d772f4922 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:21.722297 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4805962b-b99e-4e10-983c-6b7d772f4922 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:21.722739 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4805962b-b99e-4e10-983c-6b7d772f4922 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:21.723857 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-57b90aa1-60e7-4f59-a082-f54393522a50 tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] 10.222.0.16 "GET /compute/v2.1/servers/83b1fe77-ef6d-408d-b255-aa3b60aad8f3" status: 200 len: 1473 microversion: 2.1 time: 0.298949 Jul 16 21:12:21.724657 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 597/1150] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:12:21 2022] GET /compute/v2.1/servers/83b1fe77-ef6d-408d-b255-aa3b60aad8f3 => generated 1473 bytes in 301 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:12:21.950350 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-4805962b-b99e-4e10-983c-6b7d772f4922 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:21.959117 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-4805962b-b99e-4e10-983c-6b7d772f4922 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:21.960933 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-4805962b-b99e-4e10-983c-6b7d772f4922 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:21.962102 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-4805962b-b99e-4e10-983c-6b7d772f4922 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] 10.222.0.16 "GET /compute/v2.1/servers/0fb5b5b6-1796-4406-b276-326e1d7dd451" status: 200 len: 1542 microversion: 2.25 time: 0.252269 Jul 16 21:12:21.962975 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 554/1151] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:12:21 2022] GET /compute/v2.1/servers/0fb5b5b6-1796-4406-b276-326e1d7dd451 => generated 1542 bytes in 253 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:12:21.981690 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-fc44ed23-f938-4368-923c-bb798b8f0ecd tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] Calling method '.version_select at 0x7fdd9d1598b0>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:21.984881 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-fc44ed23-f938-4368-923c-bb798b8f0ecd tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:21.985440 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-fc44ed23-f938-4368-923c-bb798b8f0ecd tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:21.987675 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:12:21.997411 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-7addab96-9441-44e3-8cd6-46b1e0f2292d tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:21.998358 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-7addab96-9441-44e3-8cd6-46b1e0f2292d tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:12:22.000910 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-fc44ed23-f938-4368-923c-bb798b8f0ecd tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] 10.222.0.16 "GET /compute/v2.1/os-migrations" status: 200 len: 3205 microversion: 2.25 time: 0.024247 Jul 16 21:12:22.001961 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 598/1152] 10.222.0.16 () {60 vars in 1183 bytes} [Sat Jul 16 21:12:21 2022] GET /compute/v2.1/os-migrations => generated 3205 bytes in 26 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:12:22.006647 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7addab96-9441-44e3-8cd6-46b1e0f2292d tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:22.007143 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7addab96-9441-44e3-8cd6-46b1e0f2292d tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:22.008759 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7addab96-9441-44e3-8cd6-46b1e0f2292d tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:22.011530 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7addab96-9441-44e3-8cd6-46b1e0f2292d tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:22.019826 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-743282d1-316a-4317-b4ec-a914334bf4e4 tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:22.020331 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-743282d1-316a-4317-b4ec-a914334bf4e4 tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] [instance: 0fb5b5b6-1796-4406-b276-326e1d7dd451] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:12:22.027558 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-743282d1-316a-4317-b4ec-a914334bf4e4 tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:22.028300 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-743282d1-316a-4317-b4ec-a914334bf4e4 tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:22.029243 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-743282d1-316a-4317-b4ec-a914334bf4e4 tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:22.029861 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-743282d1-316a-4317-b4ec-a914334bf4e4 tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:22.340867 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-7addab96-9441-44e3-8cd6-46b1e0f2292d tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b68dacc4b8463ba07fae6372131117', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55a19b7a4682457a9132f5fc80110712', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:22.355826 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-7addab96-9441-44e3-8cd6-46b1e0f2292d tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "GET /compute/v2.1/servers/a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2" status: 200 len: 1535 microversion: 2.1 time: 0.361747 Jul 16 21:12:22.357307 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 555/1153] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:12:21 2022] GET /compute/v2.1/servers/a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2 => generated 1535 bytes in 366 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:12:22.359178 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.objects.instance [None req-743282d1-316a-4317-b4ec-a914334bf4e4 tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] Lazy-loading 'services' on Instance uuid 0fb5b5b6-1796-4406-b276-326e1d7dd451 {{(pid=138680) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:12:22.381494 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-743282d1-316a-4317-b4ec-a914334bf4e4 tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/0fb5b5b6-1796-4406-b276-326e1d7dd451" status: 200 len: 2035 microversion: 2.25 time: 0.370774 Jul 16 21:12:22.382507 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 599/1154] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:12:22 2022] GET /compute/v2.1/servers/0fb5b5b6-1796-4406-b276-326e1d7dd451 => generated 2035 bytes in 372 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:12:22.382626 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-ba883fbf-8b33-4557-9a8c-d948575296c5 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:22.383315 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-ba883fbf-8b33-4557-9a8c-d948575296c5 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] [instance: 4ae3e944-b450-40e5-bb71-c02df03212e3] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:12:22.388754 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_api:SELECT=95,nova_api:UPDATE=2,nova_api:DELETE=3 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:12:22.402470 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ba883fbf-8b33-4557-9a8c-d948575296c5 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:22.402975 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ba883fbf-8b33-4557-9a8c-d948575296c5 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:22.404548 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ba883fbf-8b33-4557-9a8c-d948575296c5 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:22.405006 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ba883fbf-8b33-4557-9a8c-d948575296c5 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:22.409157 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-958fff30-6782-4d8e-804f-5cd811c490e4 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Action: 'create', calling method: .version_select at 0x7fdd9d1594c0>, body: {"keypair": {"name": "tempest-keypair-108238508", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBFzZAxMtxBG1kvWHT3ihHOx61h69S49LZci+UAWMJImHlWoZrkKtmE1vYTE6Tjn/CnkgdQzRJbm7wjpvAx4BtXtOE3ToxZ+afMGeeL80s0xkOYuO263eVvvYVDdsv4nPWQ=="}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:12:22.432913 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-958fff30-6782-4d8e-804f-5cd811c490e4 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Getting quotas for project d9b92fc544c544f686ac2ab7ff5123b6. Resources: {'key_pairs'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:12:22.438404 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-958fff30-6782-4d8e-804f-5cd811c490e4 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Getting quotas for user 578b87f863b34f57a571f7e56d44f118 and project d9b92fc544c544f686ac2ab7ff5123b6. Resources: {'key_pairs'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:12:22.457964 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_api:SELECT=96,nova_api:UPDATE=3,nova_api:INSERT=1 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:12:22.592422 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-958fff30-6782-4d8e-804f-5cd811c490e4 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] 10.222.0.16 "POST /compute/v2.1/os-keypairs" status: 201 len: 400 microversion: 2.25 time: 0.187138 Jul 16 21:12:22.593350 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 600/1155] 10.222.0.16 () {62 vars in 1199 bytes} [Sat Jul 16 21:12:22 2022] POST /compute/v2.1/os-keypairs => generated 400 bytes in 189 msecs (HTTP/1.1 201) 9 headers in 364 bytes (2 switches on core 0) Jul 16 21:12:22.598814 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-43680f36-ddf2-4af5-b993-5cc1389d411a req-858a92e7-1a80-4855-93e8-96d828016a5a service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "148d5ad9-2900-4c2b-909b-aa446757e33d", "name": "network-vif-deleted", "tag": "7ee252fe-6a69-4f90-9053-a7eabcf7750c"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:12:22.611366 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-43680f36-ddf2-4af5-b993-5cc1389d411a req-858a92e7-1a80-4855-93e8-96d828016a5a service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:22.615296 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-43680f36-ddf2-4af5-b993-5cc1389d411a req-858a92e7-1a80-4855-93e8-96d828016a5a service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:22.625042 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-43680f36-ddf2-4af5-b993-5cc1389d411a req-858a92e7-1a80-4855-93e8-96d828016a5a service nova] Creating event network-vif-deleted:7ee252fe-6a69-4f90-9053-a7eabcf7750c for instance 148d5ad9-2900-4c2b-909b-aa446757e33d on nch2-master-1 Jul 16 21:12:22.638221 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-43680f36-ddf2-4af5-b993-5cc1389d411a req-858a92e7-1a80-4855-93e8-96d828016a5a service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.043002 Jul 16 21:12:22.640681 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 601/1156] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:12:22 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:12:22.726246 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-2706cb1c-56b9-4ed2-bbd6-c4f962bf8802 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:22.726994 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-2706cb1c-56b9-4ed2-bbd6-c4f962bf8802 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] [instance: 148d5ad9-2900-4c2b-909b-aa446757e33d] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:12:22.744764 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-ba883fbf-8b33-4557-9a8c-d948575296c5 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2dfcad8d2b57481bb33aa8f5d7483c51', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cda86413e74b4c1e8e09bd4df4926664', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:22.754135 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2706cb1c-56b9-4ed2-bbd6-c4f962bf8802 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:22.754783 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-ba883fbf-8b33-4557-9a8c-d948575296c5 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] 10.222.0.16 "GET /compute/v2.1/servers/4ae3e944-b450-40e5-bb71-c02df03212e3" status: 200 len: 1323 microversion: 2.1 time: 0.375115 Jul 16 21:12:22.755351 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2706cb1c-56b9-4ed2-bbd6-c4f962bf8802 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:22.757106 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2706cb1c-56b9-4ed2-bbd6-c4f962bf8802 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:22.757641 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 556/1157] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:12:22 2022] GET /compute/v2.1/servers/4ae3e944-b450-40e5-bb71-c02df03212e3 => generated 1323 bytes in 380 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:12:22.759205 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2706cb1c-56b9-4ed2-bbd6-c4f962bf8802 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:22.768673 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-d28e8323-778a-40ce-9809-265da062fd7b tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:22.770662 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-d28e8323-778a-40ce-9809-265da062fd7b tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] [instance: 83b1fe77-ef6d-408d-b255-aa3b60aad8f3] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:12:22.785657 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d28e8323-778a-40ce-9809-265da062fd7b tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:22.786117 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d28e8323-778a-40ce-9809-265da062fd7b tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:22.797130 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d28e8323-778a-40ce-9809-265da062fd7b tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:22.797755 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d28e8323-778a-40ce-9809-265da062fd7b tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:22.857369 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell0:SELECT=8 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:12:22.883365 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-2706cb1c-56b9-4ed2-bbd6-c4f962bf8802 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'abeb7020019d41a18de42956d706a011', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '649babe54d104436af461f1ee4767001', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:22.893706 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-2706cb1c-56b9-4ed2-bbd6-c4f962bf8802 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] 10.222.0.16 "GET /compute/v2.1/servers/148d5ad9-2900-4c2b-909b-aa446757e33d" status: 200 len: 1430 microversion: 2.1 time: 0.170675 Jul 16 21:12:22.902839 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 602/1158] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:12:22 2022] GET /compute/v2.1/servers/148d5ad9-2900-4c2b-909b-aa446757e33d => generated 1430 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:12:23.000710 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-d28e8323-778a-40ce-9809-265da062fd7b tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56d4bbcdebc24a34a814d7f6e72fbcfd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7d2b583c48254e17a8082f6467f2ae6d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:23.008436 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-d28e8323-778a-40ce-9809-265da062fd7b tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] 10.222.0.16 "GET /compute/v2.1/servers/83b1fe77-ef6d-408d-b255-aa3b60aad8f3" status: 200 len: 1473 microversion: 2.1 time: 0.249455 Jul 16 21:12:23.009011 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 557/1159] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:12:22 2022] GET /compute/v2.1/servers/83b1fe77-ef6d-408d-b255-aa3b60aad8f3 => generated 1473 bytes in 251 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:12:23.374588 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-09dc0721-290d-4dbe-88b4-ba30e0f1255a tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:23.375243 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-09dc0721-290d-4dbe-88b4-ba30e0f1255a tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:12:23.382215 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-09dc0721-290d-4dbe-88b4-ba30e0f1255a tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:23.382668 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-09dc0721-290d-4dbe-88b4-ba30e0f1255a tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:23.383462 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-09dc0721-290d-4dbe-88b4-ba30e0f1255a tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:23.383901 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-09dc0721-290d-4dbe-88b4-ba30e0f1255a tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:23.522337 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-09dc0721-290d-4dbe-88b4-ba30e0f1255a tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b68dacc4b8463ba07fae6372131117', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55a19b7a4682457a9132f5fc80110712', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:23.530849 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-09dc0721-290d-4dbe-88b4-ba30e0f1255a tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "GET /compute/v2.1/servers/a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2" status: 200 len: 1535 microversion: 2.1 time: 0.160493 Jul 16 21:12:23.531460 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 603/1160] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:12:23 2022] GET /compute/v2.1/servers/a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2 => generated 1535 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:12:23.772865 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-f63876a2-7f18-4890-a6ec-3110496254ce tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:23.773476 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-f63876a2-7f18-4890-a6ec-3110496254ce tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] [instance: 4ae3e944-b450-40e5-bb71-c02df03212e3] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:12:23.781784 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f63876a2-7f18-4890-a6ec-3110496254ce tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:23.782248 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f63876a2-7f18-4890-a6ec-3110496254ce tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:23.783024 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f63876a2-7f18-4890-a6ec-3110496254ce tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:23.783509 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f63876a2-7f18-4890-a6ec-3110496254ce tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:23.910304 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-39f3e2cd-9700-4201-9b49-b9ad90a0a253 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:23.911702 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-39f3e2cd-9700-4201-9b49-b9ad90a0a253 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] [instance: 148d5ad9-2900-4c2b-909b-aa446757e33d] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:12:23.918723 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-39f3e2cd-9700-4201-9b49-b9ad90a0a253 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:23.919229 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-39f3e2cd-9700-4201-9b49-b9ad90a0a253 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:23.920099 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-39f3e2cd-9700-4201-9b49-b9ad90a0a253 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:23.920784 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-39f3e2cd-9700-4201-9b49-b9ad90a0a253 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:23.945085 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-f63876a2-7f18-4890-a6ec-3110496254ce tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2dfcad8d2b57481bb33aa8f5d7483c51', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cda86413e74b4c1e8e09bd4df4926664', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:23.961405 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-f63876a2-7f18-4890-a6ec-3110496254ce tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] 10.222.0.16 "GET /compute/v2.1/servers/4ae3e944-b450-40e5-bb71-c02df03212e3" status: 200 len: 1323 microversion: 2.1 time: 0.192432 Jul 16 21:12:23.967736 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 558/1161] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:12:23 2022] GET /compute/v2.1/servers/4ae3e944-b450-40e5-bb71-c02df03212e3 => generated 1323 bytes in 199 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:12:24.020107 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-39f3e2cd-9700-4201-9b49-b9ad90a0a253 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'abeb7020019d41a18de42956d706a011', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '649babe54d104436af461f1ee4767001', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:24.029588 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-39f3e2cd-9700-4201-9b49-b9ad90a0a253 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] 10.222.0.16 "GET /compute/v2.1/servers/148d5ad9-2900-4c2b-909b-aa446757e33d" status: 200 len: 1430 microversion: 2.1 time: 0.123307 Jul 16 21:12:24.032773 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 604/1162] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:12:23 2022] GET /compute/v2.1/servers/148d5ad9-2900-4c2b-909b-aa446757e33d => generated 1430 bytes in 125 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:12:24.034565 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-70ddc1b4-97ac-43d0-9896-c628ee91bf8a tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:24.035230 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-70ddc1b4-97ac-43d0-9896-c628ee91bf8a tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] [instance: 83b1fe77-ef6d-408d-b255-aa3b60aad8f3] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:12:24.046068 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-70ddc1b4-97ac-43d0-9896-c628ee91bf8a tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:24.046732 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-70ddc1b4-97ac-43d0-9896-c628ee91bf8a tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:24.047691 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-70ddc1b4-97ac-43d0-9896-c628ee91bf8a tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:24.048293 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-70ddc1b4-97ac-43d0-9896-c628ee91bf8a tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:24.188112 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-70ddc1b4-97ac-43d0-9896-c628ee91bf8a tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56d4bbcdebc24a34a814d7f6e72fbcfd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7d2b583c48254e17a8082f6467f2ae6d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:24.195993 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-70ddc1b4-97ac-43d0-9896-c628ee91bf8a tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] 10.222.0.16 "GET /compute/v2.1/servers/83b1fe77-ef6d-408d-b255-aa3b60aad8f3" status: 200 len: 1473 microversion: 2.1 time: 0.174156 Jul 16 21:12:24.197283 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 559/1163] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:12:24 2022] GET /compute/v2.1/servers/83b1fe77-ef6d-408d-b255-aa3b60aad8f3 => generated 1473 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:12:24.517193 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-8f587256-63ae-4bbb-9734-8eae3045286b req-f751dc0d-10d4-41f3-b769-1cbdbd7ca805 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2", "name": "network-vif-plugged", "status": "completed", "tag": "bf9d54d7-5718-4678-bc2f-5ba8241bd855"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:12:24.526635 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-8f587256-63ae-4bbb-9734-8eae3045286b req-f751dc0d-10d4-41f3-b769-1cbdbd7ca805 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:24.527400 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-8f587256-63ae-4bbb-9734-8eae3045286b req-f751dc0d-10d4-41f3-b769-1cbdbd7ca805 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:24.539996 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-8f587256-63ae-4bbb-9734-8eae3045286b req-f751dc0d-10d4-41f3-b769-1cbdbd7ca805 service nova] Creating event network-vif-plugged:bf9d54d7-5718-4678-bc2f-5ba8241bd855 for instance a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2 on nch1-master-1 Jul 16 21:12:24.548595 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-ee841988-11bd-4438-978b-81d8706bd43e tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:24.549599 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-ee841988-11bd-4438-978b-81d8706bd43e tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:12:24.558553 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ee841988-11bd-4438-978b-81d8706bd43e tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:24.559526 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ee841988-11bd-4438-978b-81d8706bd43e tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:24.561053 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ee841988-11bd-4438-978b-81d8706bd43e tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:24.562451 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-8f587256-63ae-4bbb-9734-8eae3045286b req-f751dc0d-10d4-41f3-b769-1cbdbd7ca805 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.049522 Jul 16 21:12:24.563609 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ee841988-11bd-4438-978b-81d8706bd43e tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:24.567569 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 605/1164] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:12:24 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:12:24.992446 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-d3726c83-76d3-444d-8d66-954565a68e6d tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:24.996307 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-d3726c83-76d3-444d-8d66-954565a68e6d tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] [instance: 4ae3e944-b450-40e5-bb71-c02df03212e3] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:12:25.019308 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d3726c83-76d3-444d-8d66-954565a68e6d tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:25.019994 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d3726c83-76d3-444d-8d66-954565a68e6d tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:25.020878 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d3726c83-76d3-444d-8d66-954565a68e6d tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:25.021336 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d3726c83-76d3-444d-8d66-954565a68e6d tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:25.272733 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-ee841988-11bd-4438-978b-81d8706bd43e tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b68dacc4b8463ba07fae6372131117', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55a19b7a4682457a9132f5fc80110712', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:25.280117 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-ee841988-11bd-4438-978b-81d8706bd43e tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "GET /compute/v2.1/servers/a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2" status: 200 len: 1535 microversion: 2.1 time: 0.735720 Jul 16 21:12:25.280903 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 560/1165] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:12:24 2022] GET /compute/v2.1/servers/a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2 => generated 1535 bytes in 737 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:12:25.285717 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-abd1a12a-c70d-4a71-a8f2-7cd767e0e6d0 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:25.286332 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-abd1a12a-c70d-4a71-a8f2-7cd767e0e6d0 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] [instance: 148d5ad9-2900-4c2b-909b-aa446757e33d] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:12:25.294800 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-abd1a12a-c70d-4a71-a8f2-7cd767e0e6d0 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:25.302852 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-abd1a12a-c70d-4a71-a8f2-7cd767e0e6d0 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:25.302852 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-abd1a12a-c70d-4a71-a8f2-7cd767e0e6d0 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:25.302852 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-abd1a12a-c70d-4a71-a8f2-7cd767e0e6d0 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:25.333278 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-abd1a12a-c70d-4a71-a8f2-7cd767e0e6d0 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:25.333734 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-abd1a12a-c70d-4a71-a8f2-7cd767e0e6d0 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:25.334184 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.objects.instance [None req-abd1a12a-c70d-4a71-a8f2-7cd767e0e6d0 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lazy-loading 'fault' on Instance uuid 148d5ad9-2900-4c2b-909b-aa446757e33d {{(pid=138681) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:12:25.465118 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-abd1a12a-c70d-4a71-a8f2-7cd767e0e6d0 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'abeb7020019d41a18de42956d706a011', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '649babe54d104436af461f1ee4767001', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:25.475305 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-abd1a12a-c70d-4a71-a8f2-7cd767e0e6d0 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] 10.222.0.16 "GET /compute/v2.1/servers/148d5ad9-2900-4c2b-909b-aa446757e33d" status: 200 len: 1306 microversion: 2.1 time: 0.193309 Jul 16 21:12:25.481752 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 561/1166] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:12:25 2022] GET /compute/v2.1/servers/148d5ad9-2900-4c2b-909b-aa446757e33d => generated 1306 bytes in 200 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:12:25.505595 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-98f73c5c-c0bc-402d-a4ea-6d58dfa65f50 tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:25.505595 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-98f73c5c-c0bc-402d-a4ea-6d58dfa65f50 tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] [instance: 83b1fe77-ef6d-408d-b255-aa3b60aad8f3] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:12:25.505595 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-98f73c5c-c0bc-402d-a4ea-6d58dfa65f50 tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:25.505595 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-98f73c5c-c0bc-402d-a4ea-6d58dfa65f50 tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:25.506669 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-98f73c5c-c0bc-402d-a4ea-6d58dfa65f50 tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:25.506669 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-98f73c5c-c0bc-402d-a4ea-6d58dfa65f50 tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:25.525673 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-d3726c83-76d3-444d-8d66-954565a68e6d tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2dfcad8d2b57481bb33aa8f5d7483c51', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cda86413e74b4c1e8e09bd4df4926664', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:25.537690 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-d3726c83-76d3-444d-8d66-954565a68e6d tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] 10.222.0.16 "GET /compute/v2.1/servers/4ae3e944-b450-40e5-bb71-c02df03212e3" status: 200 len: 1323 microversion: 2.1 time: 0.561357 Jul 16 21:12:25.538518 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 606/1167] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:12:24 2022] GET /compute/v2.1/servers/4ae3e944-b450-40e5-bb71-c02df03212e3 => generated 1323 bytes in 563 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:12:25.543483 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-749e1792-6c7e-44f8-8aa7-f09ff990f68c tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-LiveAutoBlockMigrationV225Test-server-341081406", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452", "networks": [{"uuid": "b3fd4d7b-96dc-4cc4-8a4c-4e1986f1bc15"}], "security_groups": [{"name": "tempest-securitygroup--83991345"}], "key_name": "tempest-keypair-108238508", "user_data": "IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIHVidW50dSB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH51YnVudHUvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo="}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:12:25.812901 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.network.neutron [None req-749e1792-6c7e-44f8-8aa7-f09ff990f68c tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] validate_networks() for [('b3fd4d7b-96dc-4cc4-8a4c-4e1986f1bc15', None, None, None, None, None)] {{(pid=138680) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 16 21:12:25.929297 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-98f73c5c-c0bc-402d-a4ea-6d58dfa65f50 tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56d4bbcdebc24a34a814d7f6e72fbcfd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7d2b583c48254e17a8082f6467f2ae6d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:25.938055 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-98f73c5c-c0bc-402d-a4ea-6d58dfa65f50 tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] 10.222.0.16 "GET /compute/v2.1/servers/83b1fe77-ef6d-408d-b255-aa3b60aad8f3" status: 200 len: 1473 microversion: 2.1 time: 0.454891 Jul 16 21:12:25.940337 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 562/1168] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:12:25 2022] GET /compute/v2.1/servers/83b1fe77-ef6d-408d-b255-aa3b60aad8f3 => generated 1473 bytes in 456 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:12:25.943116 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-4b166569-ec3e-4c85-838a-09ebc3dd467b tempest-LiveMigrationTest-1131758637 tempest-LiveMigrationTest-1131758637-project-admin] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:25.943782 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-4b166569-ec3e-4c85-838a-09ebc3dd467b tempest-LiveMigrationTest-1131758637 tempest-LiveMigrationTest-1131758637-project-admin] [instance: 6a391156-ffce-49c0-a79d-47b4544264f5] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:12:25.951459 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4b166569-ec3e-4c85-838a-09ebc3dd467b tempest-LiveMigrationTest-1131758637 tempest-LiveMigrationTest-1131758637-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:25.951988 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4b166569-ec3e-4c85-838a-09ebc3dd467b tempest-LiveMigrationTest-1131758637 tempest-LiveMigrationTest-1131758637-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:25.952822 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4b166569-ec3e-4c85-838a-09ebc3dd467b tempest-LiveMigrationTest-1131758637 tempest-LiveMigrationTest-1131758637-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:25.953343 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4b166569-ec3e-4c85-838a-09ebc3dd467b tempest-LiveMigrationTest-1131758637 tempest-LiveMigrationTest-1131758637-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:26.353873 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-4b166569-ec3e-4c85-838a-09ebc3dd467b tempest-LiveMigrationTest-1131758637 tempest-LiveMigrationTest-1131758637-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/6a391156-ffce-49c0-a79d-47b4544264f5" status: 200 len: 1858 microversion: 2.1 time: 0.413279 Jul 16 21:12:26.360706 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 563/1169] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:12:25 2022] GET /compute/v2.1/servers/6a391156-ffce-49c0-a79d-47b4544264f5 => generated 1858 bytes in 416 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:12:26.360986 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-7b53343d-ebaf-4a11-83cb-5eed13cd6a95 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:26.361952 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-7b53343d-ebaf-4a11-83cb-5eed13cd6a95 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:12:26.371594 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7b53343d-ebaf-4a11-83cb-5eed13cd6a95 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:26.372119 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7b53343d-ebaf-4a11-83cb-5eed13cd6a95 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:26.377816 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7b53343d-ebaf-4a11-83cb-5eed13cd6a95 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:26.377816 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7b53343d-ebaf-4a11-83cb-5eed13cd6a95 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:26.596852 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-7b53343d-ebaf-4a11-83cb-5eed13cd6a95 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b68dacc4b8463ba07fae6372131117', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55a19b7a4682457a9132f5fc80110712', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:26.605488 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-7b53343d-ebaf-4a11-83cb-5eed13cd6a95 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "GET /compute/v2.1/servers/a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2" status: 200 len: 1528 microversion: 2.1 time: 0.249192 Jul 16 21:12:26.606428 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 564/1170] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:12:26 2022] GET /compute/v2.1/servers/a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2 => generated 1528 bytes in 251 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:12:26.611292 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-dd8b39d5-ee55-4be6-8393-31ee4ff77da5 req-60940401-de76-4c1b-86bb-5c5aec8ff246 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "83b1fe77-ef6d-408d-b255-aa3b60aad8f3", "name": "network-vif-deleted", "tag": "fef3d648-18d7-4308-9bcb-4555c790a3c7"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:12:26.620831 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-dd8b39d5-ee55-4be6-8393-31ee4ff77da5 req-60940401-de76-4c1b-86bb-5c5aec8ff246 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:26.621350 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-dd8b39d5-ee55-4be6-8393-31ee4ff77da5 req-60940401-de76-4c1b-86bb-5c5aec8ff246 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:26.635067 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-dd8b39d5-ee55-4be6-8393-31ee4ff77da5 req-60940401-de76-4c1b-86bb-5c5aec8ff246 service nova] Creating event network-vif-deleted:fef3d648-18d7-4308-9bcb-4555c790a3c7 for instance 83b1fe77-ef6d-408d-b255-aa3b60aad8f3 on nch1-master-1 Jul 16 21:12:26.635244 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-749e1792-6c7e-44f8-8aa7-f09ff990f68c tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Flavor limits 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 16 21:12:26.635932 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-749e1792-6c7e-44f8-8aa7-f09ff990f68c tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Image limits 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 16 21:12:26.636419 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-749e1792-6c7e-44f8-8aa7-f09ff990f68c tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Flavor pref 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 16 21:12:26.636857 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-749e1792-6c7e-44f8-8aa7-f09ff990f68c tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Image pref 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 16 21:12:26.637263 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-749e1792-6c7e-44f8-8aa7-f09ff990f68c tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 16 21:12:26.641311 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-dd8b39d5-ee55-4be6-8393-31ee4ff77da5 req-60940401-de76-4c1b-86bb-5c5aec8ff246 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.033454 Jul 16 21:12:26.649827 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 565/1171] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:12:26 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:12:26.649827 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-12f0ea29-8e29-4687-9c8b-86694dd73ade tempest-LiveMigrationTest-1131758637 tempest-LiveMigrationTest-1131758637-project-admin] Action: 'action', calling method: >, body: {"os-migrateLive": {"host": "nch1-master-1", "block_migration": true, "disk_over_commit": false}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:12:26.649827 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-12f0ea29-8e29-4687-9c8b-86694dd73ade tempest-LiveMigrationTest-1131758637 tempest-LiveMigrationTest-1131758637-project-admin] [instance: 6a391156-ffce-49c0-a79d-47b4544264f5] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:12:26.650984 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-749e1792-6c7e-44f8-8aa7-f09ff990f68c tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:26.651469 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-749e1792-6c7e-44f8-8aa7-f09ff990f68c tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:26.652273 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-749e1792-6c7e-44f8-8aa7-f09ff990f68c tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:26.652711 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-749e1792-6c7e-44f8-8aa7-f09ff990f68c tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:26.655785 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-12f0ea29-8e29-4687-9c8b-86694dd73ade tempest-LiveMigrationTest-1131758637 tempest-LiveMigrationTest-1131758637-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:26.656246 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-12f0ea29-8e29-4687-9c8b-86694dd73ade tempest-LiveMigrationTest-1131758637 tempest-LiveMigrationTest-1131758637-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:26.657025 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-12f0ea29-8e29-4687-9c8b-86694dd73ade tempest-LiveMigrationTest-1131758637 tempest-LiveMigrationTest-1131758637-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:26.657469 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-12f0ea29-8e29-4687-9c8b-86694dd73ade tempest-LiveMigrationTest-1131758637 tempest-LiveMigrationTest-1131758637-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:26.670103 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-749e1792-6c7e-44f8-8aa7-f09ff990f68c tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Getting quotas for project d9b92fc544c544f686ac2ab7ff5123b6. Resources: {'ram', 'instances', 'cores'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:12:26.676134 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-749e1792-6c7e-44f8-8aa7-f09ff990f68c tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Getting quotas for user 578b87f863b34f57a571f7e56d44f118 and project d9b92fc544c544f686ac2ab7ff5123b6. Resources: {'ram', 'instances', 'cores'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:12:26.688368 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.objects.instance [None req-12f0ea29-8e29-4687-9c8b-86694dd73ade tempest-LiveMigrationTest-1131758637 tempest-LiveMigrationTest-1131758637-project-admin] Lazy-loading 'flavor' on Instance uuid 6a391156-ffce-49c0-a79d-47b4544264f5 {{(pid=138681) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:12:26.689284 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-749e1792-6c7e-44f8-8aa7-f09ff990f68c tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Going to run 1 instances... {{(pid=138680) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 16 21:12:26.691744 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-749e1792-6c7e-44f8-8aa7-f09ff990f68c tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] [instance: 3c5ab09e-0a97-4cfc-b249-a11ab221b5d2] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138680) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 16 21:12:26.704623 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-12f0ea29-8e29-4687-9c8b-86694dd73ade tempest-LiveMigrationTest-1131758637 tempest-LiveMigrationTest-1131758637-project-admin] [instance: 6a391156-ffce-49c0-a79d-47b4544264f5] Going to try to live migrate instance to nch1-master-1 {{(pid=138681) live_migrate /opt/stack/nova/nova/compute/api.py:5349}} Jul 16 21:12:26.827313 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-749e1792-6c7e-44f8-8aa7-f09ff990f68c tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 404 microversion: 2.25 time: 1.287344 Jul 16 21:12:26.832684 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 607/1172] 10.222.0.16 () {62 vars in 1187 bytes} [Sat Jul 16 21:12:25 2022] POST /compute/v2.1/servers => generated 404 bytes in 1289 msecs (HTTP/1.1 202) 10 headers in 453 bytes (1 switches on core 0) Jul 16 21:12:26.833308 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-1153f614-9eac-472d-b33b-a401f63f0ef4 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:26.833938 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-1153f614-9eac-472d-b33b-a401f63f0ef4 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] [instance: 148d5ad9-2900-4c2b-909b-aa446757e33d] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:12:26.843480 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1153f614-9eac-472d-b33b-a401f63f0ef4 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:26.843480 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1153f614-9eac-472d-b33b-a401f63f0ef4 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:26.843480 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1153f614-9eac-472d-b33b-a401f63f0ef4 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:26.843886 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1153f614-9eac-472d-b33b-a401f63f0ef4 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:26.861668 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-1153f614-9eac-472d-b33b-a401f63f0ef4 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] HTTP exception thrown: Instance 148d5ad9-2900-4c2b-909b-aa446757e33d could not be found. Jul 16 21:12:26.863560 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-1153f614-9eac-472d-b33b-a401f63f0ef4 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Request method failure captured: Jul 16 21:12:26.863560 ncdv-master-1 devstack@n-api.service[138680]: request: GET /compute/v2.1/servers/148d5ad9-2900-4c2b-909b-aa446757e33d HTTP/1.1 Jul 16 21:12:26.863560 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:12:26.863560 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:12:26.863560 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:12:26.863560 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 0 Jul 16 21:12:26.863560 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:12:26.863560 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:12:26.863560 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:12:26.863560 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:12:26.863560 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0yjvVgP0jq68r-ny5Eh1zEbXMYJhEkVpBaJLnFwKzfSzzxAGgQPy49cpkkoW8Rc5r0epWFa_INrkbRxHc5yb1M5XgG50--wLBrLRBZrPuxqH4ynry56OcrGI4F8X9HPf6z5uwbMiaMLyGnsa6t3q-Npn98NzTN2KKNq9Fp4f3zZbeyY Jul 16 21:12:26.863560 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:12:26.863560 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:12:26.863560 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:12:26.863560 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:12:26.863560 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:12:26.863560 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:12:26.863560 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: 649babe54d104436af461f1ee4767001 Jul 16 21:12:26.863560 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-ServersOnMultiNodesTest-1258903790 Jul 16 21:12:26.863560 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:12:26.863560 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:12:26.865196 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/649babe54d104436af461f1ee4767001"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/649babe54d104436af461f1ee4767001"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_649babe54d104436af461f1ee4767001"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/649babe54d104436af461f1ee4767001"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:12:26.865196 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-ServersOnMultiNodesTest-1258903790 Jul 16 21:12:26.865196 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: 649babe54d104436af461f1ee4767001 Jul 16 21:12:26.865196 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-ServersOnMultiNodesTest-1258903790 Jul 16 21:12:26.865196 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-ServersOnMultiNodesTest-1258903790-project Jul 16 21:12:26.865196 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:12:26.865196 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:12:26.865196 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: abeb7020019d41a18de42956d706a011 Jul 16 21:12:26.865196 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-ServersOnMultiNodesTest-1258903790-project Jul 16 21:12:26.865196 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 21:12:26.865196 ncdv-master-1 devstack@n-api.service[138680]: exception: Instance 148d5ad9-2900-4c2b-909b-aa446757e33d could not be found. Jul 16 21:12:26.865196 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': '148d5ad9-2900-4c2b-909b-aa446757e33d'} Jul 16 21:12:26.866113 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:12:26.866113 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:12:26.866113 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:12:26.866113 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:12:26.866113 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:12:26.866113 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:12:26.866113 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:12:26.866113 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:12:26.866113 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:12:26.866113 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:12:26.866113 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 148d5ad9-2900-4c2b-909b-aa446757e33d could not be found. Jul 16 21:12:26.866113 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:12:26.866113 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:12:26.866113 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:12:26.866113 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:12:26.866113 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:12:26.866113 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:12:26.866113 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:12:26.866113 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:12:26.866113 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:12:26.872820 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:12:26.872820 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:12:26.872820 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:12:26.872820 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:12:26.872820 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:12:26.872820 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:12:26.872820 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:12:26.872820 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 148d5ad9-2900-4c2b-909b-aa446757e33d could not be found. Jul 16 21:12:26.872820 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:12:26.872820 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:12:26.872820 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:12:26.872820 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:12:26.872820 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:12:26.872820 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:12:26.872820 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:12:26.872820 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:12:26.872820 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 148d5ad9-2900-4c2b-909b-aa446757e33d could not be found. Jul 16 21:12:26.872820 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:12:26.874698 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-1153f614-9eac-472d-b33b-a401f63f0ef4 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] Returning 404 to user: Instance 148d5ad9-2900-4c2b-909b-aa446757e33d could not be found. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:12:26.874698 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-1153f614-9eac-472d-b33b-a401f63f0ef4 tempest-ServersOnMultiNodesTest-1258903790 tempest-ServersOnMultiNodesTest-1258903790-project] 10.222.0.16 "GET /compute/v2.1/servers/148d5ad9-2900-4c2b-909b-aa446757e33d" status: 404 len: 111 microversion: 2.1 time: 0.039714 Jul 16 21:12:26.874698 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 608/1173] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:12:26 2022] GET /compute/v2.1/servers/148d5ad9-2900-4c2b-909b-aa446757e33d => generated 111 bytes in 42 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:12:26.878918 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-397a7244-e55a-4b7c-b946-9b0b258b31fc tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:26.879572 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-397a7244-e55a-4b7c-b946-9b0b258b31fc tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] [instance: 4ae3e944-b450-40e5-bb71-c02df03212e3] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:12:26.886769 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-397a7244-e55a-4b7c-b946-9b0b258b31fc tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:26.887411 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-397a7244-e55a-4b7c-b946-9b0b258b31fc tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:26.888305 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-397a7244-e55a-4b7c-b946-9b0b258b31fc tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:26.888868 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-397a7244-e55a-4b7c-b946-9b0b258b31fc tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:27.073106 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-397a7244-e55a-4b7c-b946-9b0b258b31fc tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2dfcad8d2b57481bb33aa8f5d7483c51', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cda86413e74b4c1e8e09bd4df4926664', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:27.081561 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-397a7244-e55a-4b7c-b946-9b0b258b31fc tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] 10.222.0.16 "GET /compute/v2.1/servers/4ae3e944-b450-40e5-bb71-c02df03212e3" status: 200 len: 1323 microversion: 2.1 time: 0.210046 Jul 16 21:12:27.082892 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 609/1174] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:12:26 2022] GET /compute/v2.1/servers/4ae3e944-b450-40e5-bb71-c02df03212e3 => generated 1323 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:12:27.088677 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-3ab41fb3-3171-4328-af70-c517a9cb0784 req-c913b241-2854-476e-b6ef-6e3fa7990c93 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "4ae3e944-b450-40e5-bb71-c02df03212e3", "name": "network-vif-plugged", "status": "completed", "tag": "578b9284-5794-41e6-80d7-b5c49283c6f9"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:12:27.098938 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-3ab41fb3-3171-4328-af70-c517a9cb0784 req-c913b241-2854-476e-b6ef-6e3fa7990c93 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:27.099618 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-3ab41fb3-3171-4328-af70-c517a9cb0784 req-c913b241-2854-476e-b6ef-6e3fa7990c93 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:27.112690 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-3ab41fb3-3171-4328-af70-c517a9cb0784 req-c913b241-2854-476e-b6ef-6e3fa7990c93 service nova] Creating event network-vif-plugged:578b9284-5794-41e6-80d7-b5c49283c6f9 for instance 4ae3e944-b450-40e5-bb71-c02df03212e3 on nch2-master-1 Jul 16 21:12:27.117739 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-3ab41fb3-3171-4328-af70-c517a9cb0784 req-c913b241-2854-476e-b6ef-6e3fa7990c93 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.033223 Jul 16 21:12:27.118966 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 610/1175] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:12:27 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:12:27.124538 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-d9cacae5-b6dd-4d05-bf20-b9b2d7369fcc tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:27.125309 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-d9cacae5-b6dd-4d05-bf20-b9b2d7369fcc tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:12:27.132489 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d9cacae5-b6dd-4d05-bf20-b9b2d7369fcc tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:27.133127 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d9cacae5-b6dd-4d05-bf20-b9b2d7369fcc tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:27.133969 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d9cacae5-b6dd-4d05-bf20-b9b2d7369fcc tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:27.134445 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d9cacae5-b6dd-4d05-bf20-b9b2d7369fcc tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:27.355120 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-d9cacae5-b6dd-4d05-bf20-b9b2d7369fcc tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b68dacc4b8463ba07fae6372131117', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55a19b7a4682457a9132f5fc80110712', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:27.363214 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-d9cacae5-b6dd-4d05-bf20-b9b2d7369fcc tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "GET /compute/v2.1/servers/a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2" status: 200 len: 1528 microversion: 2.1 time: 0.242820 Jul 16 21:12:27.363961 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 611/1176] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:12:27 2022] GET /compute/v2.1/servers/a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2 => generated 1528 bytes in 244 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:12:27.368832 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-79a76e5e-32f2-46d7-ac37-24d61bcd4922 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:27.370119 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-79a76e5e-32f2-46d7-ac37-24d61bcd4922 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] [instance: 3c5ab09e-0a97-4cfc-b249-a11ab221b5d2] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:12:27.437932 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-79a76e5e-32f2-46d7-ac37-24d61bcd4922 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:27.448812 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-79a76e5e-32f2-46d7-ac37-24d61bcd4922 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:27.456319 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-79a76e5e-32f2-46d7-ac37-24d61bcd4922 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:27.456319 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-79a76e5e-32f2-46d7-ac37-24d61bcd4922 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] 10.222.0.16 "GET /compute/v2.1/servers/3c5ab09e-0a97-4cfc-b249-a11ab221b5d2" status: 200 len: 1284 microversion: 2.25 time: 0.086541 Jul 16 21:12:27.456319 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 612/1177] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:12:27 2022] GET /compute/v2.1/servers/3c5ab09e-0a97-4cfc-b249-a11ab221b5d2 => generated 1284 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:12:27.461269 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-2f8e62e8-8a2d-4477-a371-b87d2447ee8b tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:27.461978 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-2f8e62e8-8a2d-4477-a371-b87d2447ee8b tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] [instance: 83b1fe77-ef6d-408d-b255-aa3b60aad8f3] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:12:27.468855 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2f8e62e8-8a2d-4477-a371-b87d2447ee8b tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:27.469315 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2f8e62e8-8a2d-4477-a371-b87d2447ee8b tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:27.470095 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2f8e62e8-8a2d-4477-a371-b87d2447ee8b tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:27.471022 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2f8e62e8-8a2d-4477-a371-b87d2447ee8b tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:27.486147 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-2f8e62e8-8a2d-4477-a371-b87d2447ee8b tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] HTTP exception thrown: Instance 83b1fe77-ef6d-408d-b255-aa3b60aad8f3 could not be found. Jul 16 21:12:27.487863 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-2f8e62e8-8a2d-4477-a371-b87d2447ee8b tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Request method failure captured: Jul 16 21:12:27.487863 ncdv-master-1 devstack@n-api.service[138680]: request: GET /compute/v2.1/servers/83b1fe77-ef6d-408d-b255-aa3b60aad8f3 HTTP/1.1 Jul 16 21:12:27.487863 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:12:27.487863 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:12:27.487863 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:12:27.487863 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 0 Jul 16 21:12:27.487863 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:12:27.487863 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:12:27.487863 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:12:27.487863 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:12:27.487863 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0yme2OmgPmghOZxTYv1prM8I30jqdYxV66dKB7kNvJ87HPySbthZ6En_3VLO5oCbOf2UuM1ARwLRB17-5AINJPTiqPrtco0L2oHyrrsd2HBbkXlr_mLFUNTutX_YwODzB_43ujJc9IK-SLZSWvs3yKGjofUk2kkT8-qMrEcv7y0wtas Jul 16 21:12:27.487863 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:12:27.487863 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:12:27.487863 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:12:27.487863 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:12:27.487863 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:12:27.487863 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:12:27.487863 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: 7d2b583c48254e17a8082f6467f2ae6d Jul 16 21:12:27.487863 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-DeleteServersAdminTestJSON-69305867 Jul 16 21:12:27.487863 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:12:27.487863 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:12:27.489468 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/7d2b583c48254e17a8082f6467f2ae6d"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/7d2b583c48254e17a8082f6467f2ae6d"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_7d2b583c48254e17a8082f6467f2ae6d"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/7d2b583c48254e17a8082f6467f2ae6d"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:12:27.489468 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-DeleteServersAdminTestJSON-69305867 Jul 16 21:12:27.489468 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: 7d2b583c48254e17a8082f6467f2ae6d Jul 16 21:12:27.489468 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-DeleteServersAdminTestJSON-69305867 Jul 16 21:12:27.489468 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-DeleteServersAdminTestJSON-69305867-project Jul 16 21:12:27.489468 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:12:27.489468 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:12:27.489468 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: 56d4bbcdebc24a34a814d7f6e72fbcfd Jul 16 21:12:27.489468 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-DeleteServersAdminTestJSON-69305867-project Jul 16 21:12:27.489468 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 21:12:27.489468 ncdv-master-1 devstack@n-api.service[138680]: exception: Instance 83b1fe77-ef6d-408d-b255-aa3b60aad8f3 could not be found. Jul 16 21:12:27.489468 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': '83b1fe77-ef6d-408d-b255-aa3b60aad8f3'} Jul 16 21:12:27.490384 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:12:27.490384 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:12:27.490384 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:12:27.490384 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:12:27.490384 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:12:27.490384 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:12:27.490384 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:12:27.490384 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:12:27.490384 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:12:27.490384 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:12:27.490384 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 83b1fe77-ef6d-408d-b255-aa3b60aad8f3 could not be found. Jul 16 21:12:27.490384 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:12:27.490384 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:12:27.490384 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:12:27.490384 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:12:27.490384 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:12:27.490384 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:12:27.490384 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:12:27.490384 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:12:27.490384 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:12:27.491839 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:12:27.491839 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:12:27.491839 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:12:27.491839 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:12:27.491839 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:12:27.491839 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:12:27.491839 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:12:27.491839 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 83b1fe77-ef6d-408d-b255-aa3b60aad8f3 could not be found. Jul 16 21:12:27.491839 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:12:27.491839 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:12:27.491839 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:12:27.491839 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:12:27.491839 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:12:27.491839 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:12:27.491839 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:12:27.491839 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:12:27.491839 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 83b1fe77-ef6d-408d-b255-aa3b60aad8f3 could not be found. Jul 16 21:12:27.491839 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:12:27.492921 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-2f8e62e8-8a2d-4477-a371-b87d2447ee8b tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Returning 404 to user: Instance 83b1fe77-ef6d-408d-b255-aa3b60aad8f3 could not be found. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:12:27.492921 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-2f8e62e8-8a2d-4477-a371-b87d2447ee8b tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] 10.222.0.16 "GET /compute/v2.1/servers/83b1fe77-ef6d-408d-b255-aa3b60aad8f3" status: 404 len: 111 microversion: 2.1 time: 0.032963 Jul 16 21:12:27.492921 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 613/1178] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:12:27 2022] GET /compute/v2.1/servers/83b1fe77-ef6d-408d-b255-aa3b60aad8f3 => generated 111 bytes in 34 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:12:27.505303 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-3e694243-0b1d-4d49-91d3-7fe345af9e16 tempest-MigrationsAdminTest-399208122 tempest-MigrationsAdminTest-399208122-project-admin] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:27.506255 ncdv-master-1 devstack@n-api.service[138680]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-flavor-manage:delete": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:12:27.506255 ncdv-master-1 devstack@n-api.service[138680]: warnings.warn(msg) Jul 16 21:12:27.512510 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-3e694243-0b1d-4d49-91d3-7fe345af9e16 tempest-MigrationsAdminTest-399208122 tempest-MigrationsAdminTest-399208122-project-admin] HTTP exception thrown: Flavor 228f3e42-8394-4732-ac78-3c79d14592dd could not be found. Jul 16 21:12:27.518132 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-3e694243-0b1d-4d49-91d3-7fe345af9e16 tempest-MigrationsAdminTest-399208122 tempest-MigrationsAdminTest-399208122-project-admin] Request method failure captured: Jul 16 21:12:27.518132 ncdv-master-1 devstack@n-api.service[138680]: request: DELETE /compute/v2.1/flavors/228f3e42-8394-4732-ac78-3c79d14592dd HTTP/1.1 Jul 16 21:12:27.518132 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:12:27.518132 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:12:27.518132 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:12:27.518132 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 0 Jul 16 21:12:27.518132 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:12:27.518132 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:12:27.518132 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:12:27.518132 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:12:27.518132 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0yjUJaNYu-8h_h5OB8ZHvb09YNvOoDzv937FFi_d8kxJgKcWNzcB6odv58LngoScajIKw4iAkqhMoBm-HD39fvnZae67X3tn3Yf0BSVTKmRTxSE17vATp0zbtM6ftJljW18hEx0fMLl2bQrPY04Mmih1oASa_I8gO8VmDDw54jtJLE8 Jul 16 21:12:27.518132 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:12:27.518132 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:12:27.518132 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:12:27.518132 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:12:27.518132 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:12:27.518132 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:12:27.518132 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: cbbed051cbaf450f92a79d71d58ba7a2 Jul 16 21:12:27.518132 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-MigrationsAdminTest-399208122 Jul 16 21:12:27.518132 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,admin,reader Jul 16 21:12:27.518132 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,admin,reader Jul 16 21:12:27.519818 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/cbbed051cbaf450f92a79d71d58ba7a2"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/cbbed051cbaf450f92a79d71d58ba7a2"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_cbbed051cbaf450f92a79d71d58ba7a2"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/cbbed051cbaf450f92a79d71d58ba7a2"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:12:27.519818 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-MigrationsAdminTest-399208122 Jul 16 21:12:27.519818 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: cbbed051cbaf450f92a79d71d58ba7a2 Jul 16 21:12:27.519818 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-MigrationsAdminTest-399208122 Jul 16 21:12:27.519818 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-MigrationsAdminTest-399208122-project-admin Jul 16 21:12:27.519818 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:12:27.519818 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:12:27.519818 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: 1ead1619fabc4552b11f712b28cc6187 Jul 16 21:12:27.519818 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-MigrationsAdminTest-399208122-project-admin Jul 16 21:12:27.519818 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 21:12:27.519818 ncdv-master-1 devstack@n-api.service[138680]: exception: Flavor 228f3e42-8394-4732-ac78-3c79d14592dd could not be found. Jul 16 21:12:27.519818 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': '228f3e42-8394-4732-ac78-3c79d14592dd'} Jul 16 21:12:27.520746 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:12:27.520746 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:12:27.520746 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavor_manage.py", line 43, in _delete Jul 16 21:12:27.520746 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi flavor.destroy() Jul 16 21:12:27.520746 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 226, in wrapper Jul 16 21:12:27.520746 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return fn(self, *args, **kwargs) Jul 16 21:12:27.520746 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 558, in destroy Jul 16 21:12:27.520746 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi db_flavor = self._flavor_destroy(self._context, Jul 16 21:12:27.520746 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 544, in _flavor_destroy Jul 16 21:12:27.520746 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return _flavor_destroy(context, flavor_id=flavor_id, flavorid=flavorid) Jul 16 21:12:27.520746 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1010, in wrapper Jul 16 21:12:27.520746 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return fn(*args, **kwargs) Jul 16 21:12:27.520746 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 185, in _flavor_destroy Jul 16 21:12:27.520746 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=(flavor_id or flavorid)) Jul 16 21:12:27.520746 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 228f3e42-8394-4732-ac78-3c79d14592dd could not be found. Jul 16 21:12:27.520746 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:12:27.520746 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:12:27.520746 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:12:27.520746 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:12:27.520746 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:12:27.522581 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:12:27.522581 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:12:27.522581 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:12:27.522581 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:12:27.522581 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:12:27.522581 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavor_manage.py", line 45, in _delete Jul 16 21:12:27.522581 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:12:27.522581 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 228f3e42-8394-4732-ac78-3c79d14592dd could not be found. Jul 16 21:12:27.522581 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:12:27.522581 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:12:27.522581 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:12:27.522581 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:12:27.522581 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:12:27.522581 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:12:27.522581 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:12:27.522581 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:12:27.522581 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 228f3e42-8394-4732-ac78-3c79d14592dd could not be found. Jul 16 21:12:27.522581 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:12:27.529232 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-3e694243-0b1d-4d49-91d3-7fe345af9e16 tempest-MigrationsAdminTest-399208122 tempest-MigrationsAdminTest-399208122-project-admin] Returning 404 to user: Flavor 228f3e42-8394-4732-ac78-3c79d14592dd could not be found. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:12:27.529232 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-3e694243-0b1d-4d49-91d3-7fe345af9e16 tempest-MigrationsAdminTest-399208122 tempest-MigrationsAdminTest-399208122-project-admin] 10.222.0.16 "DELETE /compute/v2.1/flavors/228f3e42-8394-4732-ac78-3c79d14592dd" status: 404 len: 109 microversion: 2.1 time: 0.023746 Jul 16 21:12:27.529232 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 614/1179] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:12:27 2022] DELETE /compute/v2.1/flavors/228f3e42-8394-4732-ac78-3c79d14592dd => generated 109 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:12:27.654418 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-e980b67f-b20b-4860-8dd4-f1ca64147108 tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-DeleteServersAdminTestJSON-server-1573835858", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452"}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:12:27.759622 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.network.neutron [None req-e980b67f-b20b-4860-8dd4-f1ca64147108 tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] validate_networks() for None {{(pid=138680) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 16 21:12:28.015350 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-e980b67f-b20b-4860-8dd4-f1ca64147108 tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Flavor limits 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 16 21:12:28.016049 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-e980b67f-b20b-4860-8dd4-f1ca64147108 tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Image limits 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 16 21:12:28.016529 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-e980b67f-b20b-4860-8dd4-f1ca64147108 tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Flavor pref 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 16 21:12:28.016944 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-e980b67f-b20b-4860-8dd4-f1ca64147108 tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Image pref 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 16 21:12:28.017361 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-e980b67f-b20b-4860-8dd4-f1ca64147108 tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 16 21:12:28.027933 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e980b67f-b20b-4860-8dd4-f1ca64147108 tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:28.033343 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e980b67f-b20b-4860-8dd4-f1ca64147108 tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:28.033343 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e980b67f-b20b-4860-8dd4-f1ca64147108 tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:28.033343 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e980b67f-b20b-4860-8dd4-f1ca64147108 tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:28.047777 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-e980b67f-b20b-4860-8dd4-f1ca64147108 tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Getting quotas for project 7d2b583c48254e17a8082f6467f2ae6d. Resources: {'ram', 'instances', 'cores'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:12:28.052563 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-e980b67f-b20b-4860-8dd4-f1ca64147108 tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Getting quotas for user 56d4bbcdebc24a34a814d7f6e72fbcfd and project 7d2b583c48254e17a8082f6467f2ae6d. Resources: {'ram', 'instances', 'cores'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:12:28.066071 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-e980b67f-b20b-4860-8dd4-f1ca64147108 tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Going to run 1 instances... {{(pid=138680) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 16 21:12:28.068368 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-e980b67f-b20b-4860-8dd4-f1ca64147108 tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] [instance: 74a19091-819b-4cf0-8109-fbb0de65313c] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138680) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 16 21:12:28.226520 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-e980b67f-b20b-4860-8dd4-f1ca64147108 tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.578587 Jul 16 21:12:28.227813 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 615/1180] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 21:12:27 2022] POST /compute/v2.1/servers => generated 380 bytes in 581 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 16 21:12:28.232769 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-deb6d2e9-5302-4c66-93a5-462a2b8ecf40 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-MigrationsAdminTest-server-1825556160", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452"}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:12:28.291755 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.network.neutron [None req-deb6d2e9-5302-4c66-93a5-462a2b8ecf40 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] validate_networks() for None {{(pid=138680) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 16 21:12:28.551574 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-deb6d2e9-5302-4c66-93a5-462a2b8ecf40 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Flavor limits 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 16 21:12:28.552016 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-deb6d2e9-5302-4c66-93a5-462a2b8ecf40 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Image limits 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 16 21:12:28.554033 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-deb6d2e9-5302-4c66-93a5-462a2b8ecf40 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Flavor pref 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 16 21:12:28.554033 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-deb6d2e9-5302-4c66-93a5-462a2b8ecf40 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Image pref 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 16 21:12:28.554033 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-deb6d2e9-5302-4c66-93a5-462a2b8ecf40 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 16 21:12:28.562884 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-deb6d2e9-5302-4c66-93a5-462a2b8ecf40 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:28.563366 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-deb6d2e9-5302-4c66-93a5-462a2b8ecf40 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:28.564340 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-deb6d2e9-5302-4c66-93a5-462a2b8ecf40 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:28.564790 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-deb6d2e9-5302-4c66-93a5-462a2b8ecf40 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:28.584622 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-deb6d2e9-5302-4c66-93a5-462a2b8ecf40 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Getting quotas for project 55a19b7a4682457a9132f5fc80110712. Resources: {'ram', 'instances', 'cores'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:12:28.592750 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-deb6d2e9-5302-4c66-93a5-462a2b8ecf40 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Getting quotas for user a9b68dacc4b8463ba07fae6372131117 and project 55a19b7a4682457a9132f5fc80110712. Resources: {'ram', 'instances', 'cores'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:12:28.607744 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-deb6d2e9-5302-4c66-93a5-462a2b8ecf40 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Going to run 1 instances... {{(pid=138680) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 16 21:12:28.610105 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-deb6d2e9-5302-4c66-93a5-462a2b8ecf40 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: 126f561c-163f-4f6d-8be1-98a2e29b0909] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138680) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 16 21:12:28.939779 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-deb6d2e9-5302-4c66-93a5-462a2b8ecf40 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.710065 Jul 16 21:12:28.940766 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 616/1181] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 21:12:28 2022] POST /compute/v2.1/servers => generated 380 bytes in 712 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 16 21:12:28.945939 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-5e79546e-eab5-44a9-aa69-c7a4bf3098ca tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:28.947135 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-5e79546e-eab5-44a9-aa69-c7a4bf3098ca tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] [instance: 4ae3e944-b450-40e5-bb71-c02df03212e3] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:12:28.954145 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5e79546e-eab5-44a9-aa69-c7a4bf3098ca tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:28.955268 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5e79546e-eab5-44a9-aa69-c7a4bf3098ca tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:28.956118 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5e79546e-eab5-44a9-aa69-c7a4bf3098ca tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:28.956640 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5e79546e-eab5-44a9-aa69-c7a4bf3098ca tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:29.174903 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-5e79546e-eab5-44a9-aa69-c7a4bf3098ca tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2dfcad8d2b57481bb33aa8f5d7483c51', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cda86413e74b4c1e8e09bd4df4926664', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:29.198967 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-5e79546e-eab5-44a9-aa69-c7a4bf3098ca tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] 10.222.0.16 "GET /compute/v2.1/servers/4ae3e944-b450-40e5-bb71-c02df03212e3" status: 200 len: 1518 microversion: 2.1 time: 0.251775 Jul 16 21:12:29.202944 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 617/1182] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:12:28 2022] GET /compute/v2.1/servers/4ae3e944-b450-40e5-bb71-c02df03212e3 => generated 1518 bytes in 261 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:12:29.217400 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-0f8e81ef-4666-4f87-8db3-1f4df6b7ac42 tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:29.218136 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-0f8e81ef-4666-4f87-8db3-1f4df6b7ac42 tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] [instance: 74a19091-819b-4cf0-8109-fbb0de65313c] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:12:29.349371 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-0f8e81ef-4666-4f87-8db3-1f4df6b7ac42 tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56d4bbcdebc24a34a814d7f6e72fbcfd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7d2b583c48254e17a8082f6467f2ae6d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:29.357286 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-0f8e81ef-4666-4f87-8db3-1f4df6b7ac42 tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] 10.222.0.16 "GET /compute/v2.1/servers/74a19091-819b-4cf0-8109-fbb0de65313c" status: 200 len: 1220 microversion: 2.1 time: 0.152769 Jul 16 21:12:29.358635 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 618/1183] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:12:29 2022] GET /compute/v2.1/servers/74a19091-819b-4cf0-8109-fbb0de65313c => generated 1220 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:12:29.372751 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-a9297aa1-1450-4f78-a8de-9a3525274a6f tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:29.373367 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-a9297aa1-1450-4f78-a8de-9a3525274a6f tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] [instance: 3c5ab09e-0a97-4cfc-b249-a11ab221b5d2] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:12:29.379951 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a9297aa1-1450-4f78-a8de-9a3525274a6f tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:29.380408 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a9297aa1-1450-4f78-a8de-9a3525274a6f tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:29.381448 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a9297aa1-1450-4f78-a8de-9a3525274a6f tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:29.382066 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a9297aa1-1450-4f78-a8de-9a3525274a6f tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:29.519230 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-a9297aa1-1450-4f78-a8de-9a3525274a6f tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:29.529451 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-a9297aa1-1450-4f78-a8de-9a3525274a6f tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:29.531262 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-a9297aa1-1450-4f78-a8de-9a3525274a6f tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:29.532402 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-a9297aa1-1450-4f78-a8de-9a3525274a6f tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] 10.222.0.16 "GET /compute/v2.1/servers/3c5ab09e-0a97-4cfc-b249-a11ab221b5d2" status: 200 len: 1336 microversion: 2.25 time: 0.163766 Jul 16 21:12:29.532961 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 619/1184] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:12:29 2022] GET /compute/v2.1/servers/3c5ab09e-0a97-4cfc-b249-a11ab221b5d2 => generated 1336 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:12:29.537769 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-a38ed8d8-ef58-4ba8-b71c-4d959b1178e4 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:29.538531 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-a38ed8d8-ef58-4ba8-b71c-4d959b1178e4 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: 126f561c-163f-4f6d-8be1-98a2e29b0909] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:12:29.611321 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-a38ed8d8-ef58-4ba8-b71c-4d959b1178e4 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b68dacc4b8463ba07fae6372131117', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55a19b7a4682457a9132f5fc80110712', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:29.619759 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-a38ed8d8-ef58-4ba8-b71c-4d959b1178e4 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "GET /compute/v2.1/servers/126f561c-163f-4f6d-8be1-98a2e29b0909" status: 200 len: 1213 microversion: 2.1 time: 0.084137 Jul 16 21:12:29.619759 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 620/1185] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:12:29 2022] GET /compute/v2.1/servers/126f561c-163f-4f6d-8be1-98a2e29b0909 => generated 1213 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:12:29.624205 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-c8476a41-2197-41a9-8d65-7f51e4a2c31b tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Action: 'action', calling method: .version_select at 0x7fdd9fd0f310>, body: {"addFloatingIp": {"address": "172.35.1.237"}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:12:29.625292 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-c8476a41-2197-41a9-8d65-7f51e4a2c31b tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] [instance: 8020f553-6025-4f19-835c-c00e2fb4f764] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:12:29.635439 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c8476a41-2197-41a9-8d65-7f51e4a2c31b tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:29.635899 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c8476a41-2197-41a9-8d65-7f51e4a2c31b tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:29.636681 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c8476a41-2197-41a9-8d65-7f51e4a2c31b tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:29.637121 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c8476a41-2197-41a9-8d65-7f51e4a2c31b tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:29.707768 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c8476a41-2197-41a9-8d65-7f51e4a2c31b tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Acquired lock "refresh_cache-8020f553-6025-4f19-835c-c00e2fb4f764" {{(pid=138680) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:294}} Jul 16 21:12:30.157470 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-12f0ea29-8e29-4687-9c8b-86694dd73ade tempest-LiveMigrationTest-1131758637 tempest-LiveMigrationTest-1131758637-project-admin] 10.222.0.16 "POST /compute/v2.1/servers/6a391156-ffce-49c0-a79d-47b4544264f5/action" status: 202 len: 0 microversion: 2.1 time: 3.513402 Jul 16 21:12:30.158186 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 566/1186] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 21:12:26 2022] POST /compute/v2.1/servers/6a391156-ffce-49c0-a79d-47b4544264f5/action => generated 0 bytes in 3515 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 21:12:30.163349 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-9280269d-4616-4184-a44d-52201dfaa3d0 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "b61a51e0-885e-44b7-8fc0-f32fe665e112"}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:12:30.164475 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-9280269d-4616-4184-a44d-52201dfaa3d0 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] [instance: b07f0d53-2cd4-440f-be47-077703aa66e6] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:12:30.172201 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9280269d-4616-4184-a44d-52201dfaa3d0 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:30.172654 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9280269d-4616-4184-a44d-52201dfaa3d0 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:30.173423 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9280269d-4616-4184-a44d-52201dfaa3d0 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:30.173858 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9280269d-4616-4184-a44d-52201dfaa3d0 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:30.205153 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:12:31.206562 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-9280269d-4616-4184-a44d-52201dfaa3d0 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] 10.222.0.16 "POST /compute/v2.1/servers/b07f0d53-2cd4-440f-be47-077703aa66e6/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 1.046731 Jul 16 21:12:31.215244 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 567/1187] 10.222.0.16 () {60 vars in 1322 bytes} [Sat Jul 16 21:12:30 2022] POST /compute/v2.1/servers/b07f0d53-2cd4-440f-be47-077703aa66e6/os-volume_attachments => generated 194 bytes in 1056 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:12:31.221670 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-277000c6-b390-4228-b87d-9a91091f5083 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:31.222924 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-277000c6-b390-4228-b87d-9a91091f5083 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] [instance: 6a391156-ffce-49c0-a79d-47b4544264f5] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:12:31.231591 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-277000c6-b390-4228-b87d-9a91091f5083 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:31.232434 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-277000c6-b390-4228-b87d-9a91091f5083 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:31.233642 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-277000c6-b390-4228-b87d-9a91091f5083 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:31.234404 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-277000c6-b390-4228-b87d-9a91091f5083 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:31.710876 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-277000c6-b390-4228-b87d-9a91091f5083 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a69f9dee03d43beaafa966216b9e289', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9b769bf8f3ea4e368bf68de1df6538f9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:31.725008 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-277000c6-b390-4228-b87d-9a91091f5083 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] 10.222.0.16 "GET /compute/v2.1/servers/6a391156-ffce-49c0-a79d-47b4544264f5" status: 200 len: 1717 microversion: 2.1 time: 0.506507 Jul 16 21:12:31.728448 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 568/1188] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:12:31 2022] GET /compute/v2.1/servers/6a391156-ffce-49c0-a79d-47b4544264f5 => generated 1717 bytes in 512 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:12:31.748970 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-b0e86b01-7550-4376-bc82-d49419cc8466 tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:31.749764 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-b0e86b01-7550-4376-bc82-d49419cc8466 tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] [instance: 74a19091-819b-4cf0-8109-fbb0de65313c] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:12:31.763393 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b0e86b01-7550-4376-bc82-d49419cc8466 tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:31.768270 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b0e86b01-7550-4376-bc82-d49419cc8466 tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:31.769423 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b0e86b01-7550-4376-bc82-d49419cc8466 tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:31.771303 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b0e86b01-7550-4376-bc82-d49419cc8466 tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:31.968253 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-b0e86b01-7550-4376-bc82-d49419cc8466 tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56d4bbcdebc24a34a814d7f6e72fbcfd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7d2b583c48254e17a8082f6467f2ae6d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:31.994082 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-b0e86b01-7550-4376-bc82-d49419cc8466 tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] 10.222.0.16 "GET /compute/v2.1/servers/74a19091-819b-4cf0-8109-fbb0de65313c" status: 200 len: 1278 microversion: 2.1 time: 0.262027 Jul 16 21:12:31.997271 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 569/1189] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:12:31 2022] GET /compute/v2.1/servers/74a19091-819b-4cf0-8109-fbb0de65313c => generated 1278 bytes in 268 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:12:32.015156 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-acdb1e31-41eb-4974-9cab-2f3f4a0712b0 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:32.015987 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-acdb1e31-41eb-4974-9cab-2f3f4a0712b0 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] [instance: 3c5ab09e-0a97-4cfc-b249-a11ab221b5d2] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:12:32.023059 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-acdb1e31-41eb-4974-9cab-2f3f4a0712b0 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:32.023767 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-acdb1e31-41eb-4974-9cab-2f3f4a0712b0 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:32.026452 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-acdb1e31-41eb-4974-9cab-2f3f4a0712b0 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:32.027265 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-acdb1e31-41eb-4974-9cab-2f3f4a0712b0 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:32.213707 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-acdb1e31-41eb-4974-9cab-2f3f4a0712b0 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:32.288969 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-acdb1e31-41eb-4974-9cab-2f3f4a0712b0 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:32.299555 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-acdb1e31-41eb-4974-9cab-2f3f4a0712b0 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:32.304649 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-acdb1e31-41eb-4974-9cab-2f3f4a0712b0 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] 10.222.0.16 "GET /compute/v2.1/servers/3c5ab09e-0a97-4cfc-b249-a11ab221b5d2" status: 200 len: 1342 microversion: 2.25 time: 0.306008 Jul 16 21:12:32.309218 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 570/1190] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:12:31 2022] GET /compute/v2.1/servers/3c5ab09e-0a97-4cfc-b249-a11ab221b5d2 => generated 1342 bytes in 311 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:12:32.332964 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.network.neutron [None req-c8476a41-2197-41a9-8d65-7f51e4a2c31b tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] [instance: 8020f553-6025-4f19-835c-c00e2fb4f764] Building network info cache for instance {{(pid=138680) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1990}} Jul 16 21:12:32.347163 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-edaa6295-15f4-4803-95fa-0880d88cd1f2 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:32.347806 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-edaa6295-15f4-4803-95fa-0880d88cd1f2 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: 126f561c-163f-4f6d-8be1-98a2e29b0909] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:12:32.354710 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-edaa6295-15f4-4803-95fa-0880d88cd1f2 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:32.387370 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-edaa6295-15f4-4803-95fa-0880d88cd1f2 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:32.388381 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-edaa6295-15f4-4803-95fa-0880d88cd1f2 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:32.388837 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-edaa6295-15f4-4803-95fa-0880d88cd1f2 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:32.866877 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-edaa6295-15f4-4803-95fa-0880d88cd1f2 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b68dacc4b8463ba07fae6372131117', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55a19b7a4682457a9132f5fc80110712', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:32.878119 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-edaa6295-15f4-4803-95fa-0880d88cd1f2 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "GET /compute/v2.1/servers/126f561c-163f-4f6d-8be1-98a2e29b0909" status: 200 len: 1283 microversion: 2.1 time: 0.564278 Jul 16 21:12:32.878740 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 571/1191] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:12:32 2022] GET /compute/v2.1/servers/126f561c-163f-4f6d-8be1-98a2e29b0909 => generated 1283 bytes in 566 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:12:32.892643 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-d4a7ac27-0123-4071-b0ac-76bf036f6630 req-ee518e58-3236-4d1d-8009-863197ea7669 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "8020f553-6025-4f19-835c-c00e2fb4f764", "tag": "05294e88-e71a-4faa-8acc-79137ca6ee05"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:12:32.912665 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-d4a7ac27-0123-4071-b0ac-76bf036f6630 req-ee518e58-3236-4d1d-8009-863197ea7669 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:32.916590 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-d4a7ac27-0123-4071-b0ac-76bf036f6630 req-ee518e58-3236-4d1d-8009-863197ea7669 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:32.943346 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-d4a7ac27-0123-4071-b0ac-76bf036f6630 req-ee518e58-3236-4d1d-8009-863197ea7669 service nova] Creating event network-changed:05294e88-e71a-4faa-8acc-79137ca6ee05 for instance 8020f553-6025-4f19-835c-c00e2fb4f764 on nch1-master-1 Jul 16 21:12:32.954541 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-d4a7ac27-0123-4071-b0ac-76bf036f6630 req-ee518e58-3236-4d1d-8009-863197ea7669 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.071059 Jul 16 21:12:32.958985 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 572/1192] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:12:32 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 80 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:12:32.964601 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-4a614c7c-50ed-476a-a85c-474b2b167532 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:32.965321 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-4a614c7c-50ed-476a-a85c-474b2b167532 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] [instance: 6a391156-ffce-49c0-a79d-47b4544264f5] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:12:32.991106 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4a614c7c-50ed-476a-a85c-474b2b167532 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:32.991571 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4a614c7c-50ed-476a-a85c-474b2b167532 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:32.992395 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4a614c7c-50ed-476a-a85c-474b2b167532 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:32.992829 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4a614c7c-50ed-476a-a85c-474b2b167532 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:33.585961 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-4a614c7c-50ed-476a-a85c-474b2b167532 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a69f9dee03d43beaafa966216b9e289', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9b769bf8f3ea4e368bf68de1df6538f9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:33.599555 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-4a614c7c-50ed-476a-a85c-474b2b167532 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] 10.222.0.16 "GET /compute/v2.1/servers/6a391156-ffce-49c0-a79d-47b4544264f5" status: 200 len: 1717 microversion: 2.1 time: 0.639032 Jul 16 21:12:33.603432 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 573/1193] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:12:32 2022] GET /compute/v2.1/servers/6a391156-ffce-49c0-a79d-47b4544264f5 => generated 1717 bytes in 642 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:12:33.826577 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-4f660e96-a6e5-4a20-8c26-ec5a3fc765b0 tempest-ServicesAdminNegativeV253TestJSON-1167242690 tempest-ServicesAdminNegativeV253TestJSON-1167242690-project-admin] Action: 'update', calling method: .version_select at 0x7fdd9d4cc700>, body: {"status": "disabled", "disabled_reason": "maintenance"} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:12:33.834041 ncdv-master-1 devstack@n-api.service[138681]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-services:update": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:12:33.834041 ncdv-master-1 devstack@n-api.service[138681]: warnings.warn(msg) Jul 16 21:12:33.834041 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4f660e96-a6e5-4a20-8c26-ec5a3fc765b0 tempest-ServicesAdminNegativeV253TestJSON-1167242690 tempest-ServicesAdminNegativeV253TestJSON-1167242690-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:33.834041 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4f660e96-a6e5-4a20-8c26-ec5a3fc765b0 tempest-ServicesAdminNegativeV253TestJSON-1167242690 tempest-ServicesAdminNegativeV253TestJSON-1167242690-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:33.859917 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4f660e96-a6e5-4a20-8c26-ec5a3fc765b0 tempest-ServicesAdminNegativeV253TestJSON-1167242690 tempest-ServicesAdminNegativeV253TestJSON-1167242690-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:33.860281 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4f660e96-a6e5-4a20-8c26-ec5a3fc765b0 tempest-ServicesAdminNegativeV253TestJSON-1167242690 tempest-ServicesAdminNegativeV253TestJSON-1167242690-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:33.886810 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-4f660e96-a6e5-4a20-8c26-ec5a3fc765b0 tempest-ServicesAdminNegativeV253TestJSON-1167242690 tempest-ServicesAdminNegativeV253TestJSON-1167242690-project-admin] HTTP exception thrown: Service d8b59866-61ec-4a37-8936-57e4ac10aa2a could not be found. Jul 16 21:12:33.893712 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-4f660e96-a6e5-4a20-8c26-ec5a3fc765b0 tempest-ServicesAdminNegativeV253TestJSON-1167242690 tempest-ServicesAdminNegativeV253TestJSON-1167242690-project-admin] Request method failure captured: Jul 16 21:12:33.893712 ncdv-master-1 devstack@n-api.service[138681]: request: PUT /compute/v2.1/os-services/d8b59866-61ec-4a37-8936-57e4ac10aa2a HTTP/1.1 Jul 16 21:12:33.893712 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:12:33.893712 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:12:33.893712 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:12:33.893712 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 56 Jul 16 21:12:33.893712 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:12:33.893712 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:12:33.893712 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:12:33.893712 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:12:33.893712 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0ynA2SDCeGm5x6JfPucncAKvQmAzPBsZ3UCl4vZWzwD5cROZ5R5UyG6IpXYY8YmCJSTip954_0uz21joUyE0kJ_GxIw0fcgSfgp0FMHnpUSfCAEN62-I9WqYHJX-uqYr63OWBafGNaPPRzw11frERbmUeaeM8AR3QYOEpSSC5_Ms1hY Jul 16 21:12:33.893712 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:12:33.893712 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:12:33.893712 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:12:33.893712 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:12:33.893712 ncdv-master-1 devstack@n-api.service[138681]: X-Openstack-Nova-Api-Version: 2.53 Jul 16 21:12:33.893712 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:12:33.893712 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:12:33.893712 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: dec88ce6b54d4046afa64c2e4d116c11 Jul 16 21:12:33.893712 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-ServicesAdminNegativeV253TestJSON-1167242690 Jul 16 21:12:33.893712 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,admin,reader Jul 16 21:12:33.893712 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,admin,reader Jul 16 21:12:33.895190 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/dec88ce6b54d4046afa64c2e4d116c11"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/dec88ce6b54d4046afa64c2e4d116c11"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_dec88ce6b54d4046afa64c2e4d116c11"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/dec88ce6b54d4046afa64c2e4d116c11"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:12:33.895190 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-ServicesAdminNegativeV253TestJSON-1167242690 Jul 16 21:12:33.895190 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: dec88ce6b54d4046afa64c2e4d116c11 Jul 16 21:12:33.895190 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-ServicesAdminNegativeV253TestJSON-1167242690 Jul 16 21:12:33.895190 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-ServicesAdminNegativeV253TestJSON-1167242690-project-admin Jul 16 21:12:33.895190 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:12:33.895190 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:12:33.895190 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: 1044a67f4df44c07b3a62100f9cc1534 Jul 16 21:12:33.895190 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-ServicesAdminNegativeV253TestJSON-1167242690-project-admin Jul 16 21:12:33.895190 ncdv-master-1 devstack@n-api.service[138681]: Jul 16 21:12:33.895190 ncdv-master-1 devstack@n-api.service[138681]: {"status": "disabled", "disabled_reason": "maintenance"} Jul 16 21:12:33.895190 ncdv-master-1 devstack@n-api.service[138681]: method: Jul 16 21:12:33.895190 ncdv-master-1 devstack@n-api.service[138681]: exception: Service d8b59866-61ec-4a37-8936-57e4ac10aa2a could not be found. Jul 16 21:12:33.895993 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': 'd8b59866-61ec-4a37-8936-57e4ac10aa2a', 'body': {'status': 'disabled', 'disabled_reason': 'maintenance'}} Jul 16 21:12:33.895993 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:12:33.895993 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:12:33.895993 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 6076, in service_get_by_id Jul 16 21:12:33.895993 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return _find_service_in_cell(context, service_id=service_id) Jul 16 21:12:33.895993 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5936, in _find_service_in_cell Jul 16 21:12:33.895993 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.NotFound() Jul 16 21:12:33.895993 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.NotFound: Resource could not be found. Jul 16 21:12:33.895993 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:12:33.895993 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:12:33.895993 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:12:33.895993 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:12:33.895993 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/services.py", line 439, in update Jul 16 21:12:33.895993 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi service = self.host_api.service_get_by_id(context, service_id) Jul 16 21:12:33.895993 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 6078, in service_get_by_id Jul 16 21:12:33.895993 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.ServiceNotFound(service_id=service_id) Jul 16 21:12:33.895993 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.ServiceNotFound: Service d8b59866-61ec-4a37-8936-57e4ac10aa2a could not be found. Jul 16 21:12:33.895993 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:12:33.895993 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:12:33.895993 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:12:33.895993 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:12:33.897321 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:12:33.897321 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:12:33.897321 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:12:33.897321 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:12:33.897321 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Jul 16 21:12:33.897321 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Jul 16 21:12:33.897321 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:12:33.897321 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:12:33.897321 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 16 21:12:33.897321 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 16 21:12:33.897321 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/services.py", line 444, in update Jul 16 21:12:33.897321 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:12:33.897321 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Service d8b59866-61ec-4a37-8936-57e4ac10aa2a could not be found. Jul 16 21:12:33.897321 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:12:33.897321 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:12:33.897321 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:12:33.897321 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:12:33.897321 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:12:33.897321 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:12:33.897321 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:12:33.902259 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:12:33.902259 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Service d8b59866-61ec-4a37-8936-57e4ac10aa2a could not be found. Jul 16 21:12:33.902259 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:12:33.902259 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-4f660e96-a6e5-4a20-8c26-ec5a3fc765b0 tempest-ServicesAdminNegativeV253TestJSON-1167242690 tempest-ServicesAdminNegativeV253TestJSON-1167242690-project-admin] Returning 404 to user: Service d8b59866-61ec-4a37-8936-57e4ac10aa2a could not be found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:12:33.902259 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-4f660e96-a6e5-4a20-8c26-ec5a3fc765b0 tempest-ServicesAdminNegativeV253TestJSON-1167242690 tempest-ServicesAdminNegativeV253TestJSON-1167242690-project-admin] 10.222.0.16 "PUT /compute/v2.1/os-services/d8b59866-61ec-4a37-8936-57e4ac10aa2a" status: 404 len: 110 microversion: 2.53 time: 0.292951 Jul 16 21:12:33.902259 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 574/1194] 10.222.0.16 () {62 vars in 1308 bytes} [Sat Jul 16 21:12:33 2022] PUT /compute/v2.1/os-services/d8b59866-61ec-4a37-8936-57e4ac10aa2a => generated 110 bytes in 294 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Jul 16 21:12:33.905120 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-a2842db3-be64-480a-a07c-71d054d7e8fa tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:33.910418 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-a2842db3-be64-480a-a07c-71d054d7e8fa tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] [instance: 74a19091-819b-4cf0-8109-fbb0de65313c] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:12:33.927733 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a2842db3-be64-480a-a07c-71d054d7e8fa tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:33.928278 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a2842db3-be64-480a-a07c-71d054d7e8fa tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:33.929125 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a2842db3-be64-480a-a07c-71d054d7e8fa tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:33.929565 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a2842db3-be64-480a-a07c-71d054d7e8fa tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:34.830153 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.network.neutron [None req-c8476a41-2197-41a9-8d65-7f51e4a2c31b tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] [instance: 8020f553-6025-4f19-835c-c00e2fb4f764] Updating instance_info_cache with network_info: [{"id": "05294e88-e71a-4faa-8acc-79137ca6ee05", "address": "fa:16:3e:30:6c:72", "network": {"id": "9b8d8e4f-14e5-4e94-bfa5-2387fb1adcf8", "bridge": null, "label": "tempest-FloatingIPsAssociationTestJSON-379677821-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.35.1.237", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "172.20.0.2"}}], "meta": {"injected": false, "tenant_id": "cda86413e74b4c1e8e09bd4df4926664", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false, "bound_drivers": {"0": "hyperv"}}, "devname": "tap05294e88-e7", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=138680) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:117}} Jul 16 21:12:34.922078 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c8476a41-2197-41a9-8d65-7f51e4a2c31b tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Releasing lock "refresh_cache-8020f553-6025-4f19-835c-c00e2fb4f764" {{(pid=138680) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:312}} Jul 16 21:12:34.923193 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-c8476a41-2197-41a9-8d65-7f51e4a2c31b tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] 10.222.0.16 "POST /compute/v2.1/servers/8020f553-6025-4f19-835c-c00e2fb4f764/action" status: 202 len: 0 microversion: 2.1 time: 5.302527 Jul 16 21:12:34.927153 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 621/1195] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 21:12:29 2022] POST /compute/v2.1/servers/8020f553-6025-4f19-835c-c00e2fb4f764/action => generated 0 bytes in 5304 msecs (HTTP/1.1 202) 9 headers in 369 bytes (1 switches on core 0) Jul 16 21:12:34.929242 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-6650d376-b35a-47a7-b19c-0a7220a40655 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:34.931179 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-6650d376-b35a-47a7-b19c-0a7220a40655 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] [instance: 3c5ab09e-0a97-4cfc-b249-a11ab221b5d2] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:12:34.952251 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6650d376-b35a-47a7-b19c-0a7220a40655 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:34.952807 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6650d376-b35a-47a7-b19c-0a7220a40655 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:34.953737 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6650d376-b35a-47a7-b19c-0a7220a40655 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:34.954380 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6650d376-b35a-47a7-b19c-0a7220a40655 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:35.077145 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-a2842db3-be64-480a-a07c-71d054d7e8fa tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56d4bbcdebc24a34a814d7f6e72fbcfd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7d2b583c48254e17a8082f6467f2ae6d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:35.087882 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-a2842db3-be64-480a-a07c-71d054d7e8fa tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] 10.222.0.16 "GET /compute/v2.1/servers/74a19091-819b-4cf0-8109-fbb0de65313c" status: 200 len: 1320 microversion: 2.1 time: 1.190352 Jul 16 21:12:35.088476 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 575/1196] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:12:33 2022] GET /compute/v2.1/servers/74a19091-819b-4cf0-8109-fbb0de65313c => generated 1320 bytes in 1192 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:12:35.095239 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-4a4e6cab-cb46-44d2-89e0-98792a46f263 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:35.095955 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-4a4e6cab-cb46-44d2-89e0-98792a46f263 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: 126f561c-163f-4f6d-8be1-98a2e29b0909] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:12:35.114716 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4a4e6cab-cb46-44d2-89e0-98792a46f263 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:35.115732 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4a4e6cab-cb46-44d2-89e0-98792a46f263 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:35.126647 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4a4e6cab-cb46-44d2-89e0-98792a46f263 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:35.126647 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4a4e6cab-cb46-44d2-89e0-98792a46f263 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:35.363696 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-6650d376-b35a-47a7-b19c-0a7220a40655 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:35.380592 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-6650d376-b35a-47a7-b19c-0a7220a40655 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:35.382648 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-6650d376-b35a-47a7-b19c-0a7220a40655 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:35.386163 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-6650d376-b35a-47a7-b19c-0a7220a40655 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] 10.222.0.16 "GET /compute/v2.1/servers/3c5ab09e-0a97-4cfc-b249-a11ab221b5d2" status: 200 len: 1408 microversion: 2.25 time: 0.460930 Jul 16 21:12:35.387672 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 622/1197] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:12:34 2022] GET /compute/v2.1/servers/3c5ab09e-0a97-4cfc-b249-a11ab221b5d2 => generated 1408 bytes in 462 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:12:35.395186 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-53811423-212b-4ccf-8e81-ec41cf48d13d tempest-ServicesAdminNegativeV253TestJSON-1167242690 tempest-ServicesAdminNegativeV253TestJSON-1167242690-project-admin] Action: 'update', calling method: .version_select at 0x7fdd9cfee8b0>, body: {"status": "disabled"} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:12:35.398129 ncdv-master-1 devstack@n-api.service[138680]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-services:update": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:12:35.398129 ncdv-master-1 devstack@n-api.service[138680]: warnings.warn(msg) Jul 16 21:12:35.400872 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-53811423-212b-4ccf-8e81-ec41cf48d13d tempest-ServicesAdminNegativeV253TestJSON-1167242690 tempest-ServicesAdminNegativeV253TestJSON-1167242690-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:35.401614 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-53811423-212b-4ccf-8e81-ec41cf48d13d tempest-ServicesAdminNegativeV253TestJSON-1167242690 tempest-ServicesAdminNegativeV253TestJSON-1167242690-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:35.414135 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-53811423-212b-4ccf-8e81-ec41cf48d13d tempest-ServicesAdminNegativeV253TestJSON-1167242690 tempest-ServicesAdminNegativeV253TestJSON-1167242690-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:35.415243 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-53811423-212b-4ccf-8e81-ec41cf48d13d tempest-ServicesAdminNegativeV253TestJSON-1167242690 tempest-ServicesAdminNegativeV253TestJSON-1167242690-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:35.437200 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-53811423-212b-4ccf-8e81-ec41cf48d13d tempest-ServicesAdminNegativeV253TestJSON-1167242690 tempest-ServicesAdminNegativeV253TestJSON-1167242690-project-admin] HTTP exception thrown: Service d8b59866-61ec-4a37-8936-57e4ac10aa2a could not be found. Jul 16 21:12:35.440634 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-53811423-212b-4ccf-8e81-ec41cf48d13d tempest-ServicesAdminNegativeV253TestJSON-1167242690 tempest-ServicesAdminNegativeV253TestJSON-1167242690-project-admin] Request method failure captured: Jul 16 21:12:35.440634 ncdv-master-1 devstack@n-api.service[138680]: request: PUT /compute/v2.1/os-services/d8b59866-61ec-4a37-8936-57e4ac10aa2a HTTP/1.1 Jul 16 21:12:35.440634 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:12:35.440634 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:12:35.440634 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:12:35.440634 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 22 Jul 16 21:12:35.440634 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:12:35.440634 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:12:35.440634 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:12:35.440634 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:12:35.440634 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0ynA2SDCeGm5x6JfPucncAKvQmAzPBsZ3UCl4vZWzwD5cROZ5R5UyG6IpXYY8YmCJSTip954_0uz21joUyE0kJ_GxIw0fcgSfgp0FMHnpUSfCAEN62-I9WqYHJX-uqYr63OWBafGNaPPRzw11frERbmUeaeM8AR3QYOEpSSC5_Ms1hY Jul 16 21:12:35.440634 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:12:35.440634 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:12:35.440634 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:12:35.440634 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:12:35.440634 ncdv-master-1 devstack@n-api.service[138680]: X-Openstack-Nova-Api-Version: 2.53 Jul 16 21:12:35.440634 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:12:35.440634 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:12:35.440634 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: dec88ce6b54d4046afa64c2e4d116c11 Jul 16 21:12:35.440634 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-ServicesAdminNegativeV253TestJSON-1167242690 Jul 16 21:12:35.440634 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,admin,reader Jul 16 21:12:35.440634 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,admin,reader Jul 16 21:12:35.442049 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/dec88ce6b54d4046afa64c2e4d116c11"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/dec88ce6b54d4046afa64c2e4d116c11"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_dec88ce6b54d4046afa64c2e4d116c11"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/dec88ce6b54d4046afa64c2e4d116c11"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:12:35.442049 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-ServicesAdminNegativeV253TestJSON-1167242690 Jul 16 21:12:35.442049 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: dec88ce6b54d4046afa64c2e4d116c11 Jul 16 21:12:35.442049 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-ServicesAdminNegativeV253TestJSON-1167242690 Jul 16 21:12:35.442049 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-ServicesAdminNegativeV253TestJSON-1167242690-project-admin Jul 16 21:12:35.442049 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:12:35.442049 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:12:35.442049 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: 1044a67f4df44c07b3a62100f9cc1534 Jul 16 21:12:35.442049 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-ServicesAdminNegativeV253TestJSON-1167242690-project-admin Jul 16 21:12:35.442049 ncdv-master-1 devstack@n-api.service[138680]: Jul 16 21:12:35.442049 ncdv-master-1 devstack@n-api.service[138680]: {"status": "disabled"} Jul 16 21:12:35.442049 ncdv-master-1 devstack@n-api.service[138680]: method: Jul 16 21:12:35.442049 ncdv-master-1 devstack@n-api.service[138680]: exception: Service d8b59866-61ec-4a37-8936-57e4ac10aa2a could not be found. Jul 16 21:12:35.442049 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': 'd8b59866-61ec-4a37-8936-57e4ac10aa2a', 'body': {'status': 'disabled'}} Jul 16 21:12:35.442893 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:12:35.442893 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:12:35.442893 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 6076, in service_get_by_id Jul 16 21:12:35.442893 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return _find_service_in_cell(context, service_id=service_id) Jul 16 21:12:35.442893 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5936, in _find_service_in_cell Jul 16 21:12:35.442893 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.NotFound() Jul 16 21:12:35.442893 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.NotFound: Resource could not be found. Jul 16 21:12:35.442893 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:12:35.442893 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:12:35.442893 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:12:35.442893 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:12:35.442893 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/services.py", line 439, in update Jul 16 21:12:35.442893 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi service = self.host_api.service_get_by_id(context, service_id) Jul 16 21:12:35.442893 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 6078, in service_get_by_id Jul 16 21:12:35.442893 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.ServiceNotFound(service_id=service_id) Jul 16 21:12:35.442893 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.ServiceNotFound: Service d8b59866-61ec-4a37-8936-57e4ac10aa2a could not be found. Jul 16 21:12:35.442893 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:12:35.442893 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:12:35.442893 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:12:35.442893 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:12:35.442893 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:12:35.444436 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:12:35.444436 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:12:35.444436 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:12:35.444436 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Jul 16 21:12:35.444436 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Jul 16 21:12:35.444436 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:12:35.444436 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:12:35.444436 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 16 21:12:35.444436 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 16 21:12:35.444436 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/services.py", line 444, in update Jul 16 21:12:35.444436 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:12:35.444436 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Service d8b59866-61ec-4a37-8936-57e4ac10aa2a could not be found. Jul 16 21:12:35.444436 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:12:35.444436 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:12:35.444436 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:12:35.444436 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:12:35.444436 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:12:35.444436 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:12:35.444436 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:12:35.444436 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:12:35.445954 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Service d8b59866-61ec-4a37-8936-57e4ac10aa2a could not be found. Jul 16 21:12:35.445954 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:12:35.447761 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-53811423-212b-4ccf-8e81-ec41cf48d13d tempest-ServicesAdminNegativeV253TestJSON-1167242690 tempest-ServicesAdminNegativeV253TestJSON-1167242690-project-admin] Returning 404 to user: Service d8b59866-61ec-4a37-8936-57e4ac10aa2a could not be found. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:12:35.450404 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-53811423-212b-4ccf-8e81-ec41cf48d13d tempest-ServicesAdminNegativeV253TestJSON-1167242690 tempest-ServicesAdminNegativeV253TestJSON-1167242690-project-admin] 10.222.0.16 "PUT /compute/v2.1/os-services/d8b59866-61ec-4a37-8936-57e4ac10aa2a" status: 404 len: 110 microversion: 2.53 time: 0.062273 Jul 16 21:12:35.455938 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 623/1198] 10.222.0.16 () {62 vars in 1308 bytes} [Sat Jul 16 21:12:35 2022] PUT /compute/v2.1/os-services/d8b59866-61ec-4a37-8936-57e4ac10aa2a => generated 110 bytes in 68 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Jul 16 21:12:35.462942 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-32bbef2a-fff1-4c07-92be-bac0f937fe02 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:35.463921 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-32bbef2a-fff1-4c07-92be-bac0f937fe02 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] [instance: 6a391156-ffce-49c0-a79d-47b4544264f5] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:12:35.472278 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-32bbef2a-fff1-4c07-92be-bac0f937fe02 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:35.472841 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-32bbef2a-fff1-4c07-92be-bac0f937fe02 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:35.473790 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-32bbef2a-fff1-4c07-92be-bac0f937fe02 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:35.474309 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-32bbef2a-fff1-4c07-92be-bac0f937fe02 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:35.517489 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-4a4e6cab-cb46-44d2-89e0-98792a46f263 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b68dacc4b8463ba07fae6372131117', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55a19b7a4682457a9132f5fc80110712', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:35.532528 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-4a4e6cab-cb46-44d2-89e0-98792a46f263 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "GET /compute/v2.1/servers/126f561c-163f-4f6d-8be1-98a2e29b0909" status: 200 len: 1313 microversion: 2.1 time: 0.441042 Jul 16 21:12:35.533879 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 576/1199] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:12:35 2022] GET /compute/v2.1/servers/126f561c-163f-4f6d-8be1-98a2e29b0909 => generated 1313 bytes in 443 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:12:35.553946 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-854fb16e-dcb8-4c80-a5bb-9985f9417d81 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Action: 'action', calling method: .version_select at 0x7fdd9fd8d310>, body: {"addFloatingIp": {"address": "172.35.1.237"}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:12:35.560615 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-854fb16e-dcb8-4c80-a5bb-9985f9417d81 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] [instance: 4ae3e944-b450-40e5-bb71-c02df03212e3] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:12:35.578528 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-854fb16e-dcb8-4c80-a5bb-9985f9417d81 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:35.578987 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-854fb16e-dcb8-4c80-a5bb-9985f9417d81 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:35.579807 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-854fb16e-dcb8-4c80-a5bb-9985f9417d81 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:35.582582 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-854fb16e-dcb8-4c80-a5bb-9985f9417d81 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:35.760417 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-854fb16e-dcb8-4c80-a5bb-9985f9417d81 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Acquired lock "refresh_cache-4ae3e944-b450-40e5-bb71-c02df03212e3" {{(pid=138681) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:294}} Jul 16 21:12:35.882988 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-32bbef2a-fff1-4c07-92be-bac0f937fe02 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a69f9dee03d43beaafa966216b9e289', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9b769bf8f3ea4e368bf68de1df6538f9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:35.894236 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-32bbef2a-fff1-4c07-92be-bac0f937fe02 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] 10.222.0.16 "GET /compute/v2.1/servers/6a391156-ffce-49c0-a79d-47b4544264f5" status: 200 len: 1717 microversion: 2.1 time: 0.436911 Jul 16 21:12:35.895333 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 624/1200] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:12:35 2022] GET /compute/v2.1/servers/6a391156-ffce-49c0-a79d-47b4544264f5 => generated 1717 bytes in 439 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:12:35.900979 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-30f0fd75-17b9-4753-9f3c-d12c0da34b30 req-3e55d167-4690-4e0c-bef3-39416a9671b8 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "74a19091-819b-4cf0-8109-fbb0de65313c", "tag": "7500284f-1b91-4c4b-bf7d-d5f21190a8ed"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:12:35.912925 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-30f0fd75-17b9-4753-9f3c-d12c0da34b30 req-3e55d167-4690-4e0c-bef3-39416a9671b8 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:35.913487 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-30f0fd75-17b9-4753-9f3c-d12c0da34b30 req-3e55d167-4690-4e0c-bef3-39416a9671b8 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:35.922239 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=97,nova_cell1:SAVEPOINT=1,nova_cell1:UPDATE=1,nova_cell1:RELEASE=1 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:12:35.933331 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-30f0fd75-17b9-4753-9f3c-d12c0da34b30 req-3e55d167-4690-4e0c-bef3-39416a9671b8 service nova] Creating event network-changed:7500284f-1b91-4c4b-bf7d-d5f21190a8ed for instance 74a19091-819b-4cf0-8109-fbb0de65313c on nch2-master-1 Jul 16 21:12:35.938805 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-30f0fd75-17b9-4753-9f3c-d12c0da34b30 req-3e55d167-4690-4e0c-bef3-39416a9671b8 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.041832 Jul 16 21:12:35.939481 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 625/1201] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:12:35 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:12:35.944886 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-6148c9f7-38e0-41b7-8476-a71a6614a386 tempest-ServicesAdminNegativeV253TestJSON-1167242690 tempest-ServicesAdminNegativeV253TestJSON-1167242690-project-admin] Action: 'update', calling method: .version_select at 0x7fdd9d0b41f0>, body: {"status": "enabled"} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:12:35.947647 ncdv-master-1 devstack@n-api.service[138680]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-services:update": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:12:35.947647 ncdv-master-1 devstack@n-api.service[138680]: warnings.warn(msg) Jul 16 21:12:35.949324 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6148c9f7-38e0-41b7-8476-a71a6614a386 tempest-ServicesAdminNegativeV253TestJSON-1167242690 tempest-ServicesAdminNegativeV253TestJSON-1167242690-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:35.950028 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6148c9f7-38e0-41b7-8476-a71a6614a386 tempest-ServicesAdminNegativeV253TestJSON-1167242690 tempest-ServicesAdminNegativeV253TestJSON-1167242690-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:35.960903 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6148c9f7-38e0-41b7-8476-a71a6614a386 tempest-ServicesAdminNegativeV253TestJSON-1167242690 tempest-ServicesAdminNegativeV253TestJSON-1167242690-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:35.961571 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6148c9f7-38e0-41b7-8476-a71a6614a386 tempest-ServicesAdminNegativeV253TestJSON-1167242690 tempest-ServicesAdminNegativeV253TestJSON-1167242690-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:35.969420 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-6148c9f7-38e0-41b7-8476-a71a6614a386 tempest-ServicesAdminNegativeV253TestJSON-1167242690 tempest-ServicesAdminNegativeV253TestJSON-1167242690-project-admin] HTTP exception thrown: Service d8b59866-61ec-4a37-8936-57e4ac10aa2a could not be found. Jul 16 21:12:35.971730 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-6148c9f7-38e0-41b7-8476-a71a6614a386 tempest-ServicesAdminNegativeV253TestJSON-1167242690 tempest-ServicesAdminNegativeV253TestJSON-1167242690-project-admin] Request method failure captured: Jul 16 21:12:35.971730 ncdv-master-1 devstack@n-api.service[138680]: request: PUT /compute/v2.1/os-services/d8b59866-61ec-4a37-8936-57e4ac10aa2a HTTP/1.1 Jul 16 21:12:35.971730 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:12:35.971730 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:12:35.971730 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:12:35.971730 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 21 Jul 16 21:12:35.971730 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:12:35.971730 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:12:35.971730 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:12:35.971730 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:12:35.971730 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0ynA2SDCeGm5x6JfPucncAKvQmAzPBsZ3UCl4vZWzwD5cROZ5R5UyG6IpXYY8YmCJSTip954_0uz21joUyE0kJ_GxIw0fcgSfgp0FMHnpUSfCAEN62-I9WqYHJX-uqYr63OWBafGNaPPRzw11frERbmUeaeM8AR3QYOEpSSC5_Ms1hY Jul 16 21:12:35.971730 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:12:35.971730 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:12:35.971730 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:12:35.971730 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:12:35.971730 ncdv-master-1 devstack@n-api.service[138680]: X-Openstack-Nova-Api-Version: 2.53 Jul 16 21:12:35.971730 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:12:35.971730 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:12:35.971730 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: dec88ce6b54d4046afa64c2e4d116c11 Jul 16 21:12:35.971730 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-ServicesAdminNegativeV253TestJSON-1167242690 Jul 16 21:12:35.971730 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,admin,reader Jul 16 21:12:35.971730 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,admin,reader Jul 16 21:12:35.973162 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/dec88ce6b54d4046afa64c2e4d116c11"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/dec88ce6b54d4046afa64c2e4d116c11"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_dec88ce6b54d4046afa64c2e4d116c11"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/dec88ce6b54d4046afa64c2e4d116c11"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:12:35.973162 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-ServicesAdminNegativeV253TestJSON-1167242690 Jul 16 21:12:35.973162 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: dec88ce6b54d4046afa64c2e4d116c11 Jul 16 21:12:35.973162 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-ServicesAdminNegativeV253TestJSON-1167242690 Jul 16 21:12:35.973162 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-ServicesAdminNegativeV253TestJSON-1167242690-project-admin Jul 16 21:12:35.973162 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:12:35.973162 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:12:35.973162 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: 1044a67f4df44c07b3a62100f9cc1534 Jul 16 21:12:35.973162 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-ServicesAdminNegativeV253TestJSON-1167242690-project-admin Jul 16 21:12:35.973162 ncdv-master-1 devstack@n-api.service[138680]: Jul 16 21:12:35.973162 ncdv-master-1 devstack@n-api.service[138680]: {"status": "enabled"} Jul 16 21:12:35.973162 ncdv-master-1 devstack@n-api.service[138680]: method: Jul 16 21:12:35.973162 ncdv-master-1 devstack@n-api.service[138680]: exception: Service d8b59866-61ec-4a37-8936-57e4ac10aa2a could not be found. Jul 16 21:12:35.973162 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': 'd8b59866-61ec-4a37-8936-57e4ac10aa2a', 'body': {'status': 'enabled'}} Jul 16 21:12:35.974000 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:12:35.974000 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:12:35.974000 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 6076, in service_get_by_id Jul 16 21:12:35.974000 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return _find_service_in_cell(context, service_id=service_id) Jul 16 21:12:35.974000 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5936, in _find_service_in_cell Jul 16 21:12:35.974000 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.NotFound() Jul 16 21:12:35.974000 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.NotFound: Resource could not be found. Jul 16 21:12:35.974000 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:12:35.974000 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:12:35.974000 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:12:35.974000 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:12:35.974000 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/services.py", line 439, in update Jul 16 21:12:35.974000 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi service = self.host_api.service_get_by_id(context, service_id) Jul 16 21:12:35.974000 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 6078, in service_get_by_id Jul 16 21:12:35.974000 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.ServiceNotFound(service_id=service_id) Jul 16 21:12:35.974000 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.ServiceNotFound: Service d8b59866-61ec-4a37-8936-57e4ac10aa2a could not be found. Jul 16 21:12:35.974000 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:12:35.974000 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:12:35.974000 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:12:35.974000 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:12:35.974000 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:12:35.981189 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:12:35.981189 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:12:35.981189 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:12:35.981189 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Jul 16 21:12:35.981189 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Jul 16 21:12:35.981189 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:12:35.981189 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:12:35.981189 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 16 21:12:35.981189 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 16 21:12:35.981189 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/services.py", line 444, in update Jul 16 21:12:35.981189 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:12:35.981189 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Service d8b59866-61ec-4a37-8936-57e4ac10aa2a could not be found. Jul 16 21:12:35.981189 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:12:35.981189 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:12:35.981189 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:12:35.981189 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:12:35.981189 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:12:35.981189 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:12:35.981189 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:12:35.981189 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:12:35.982453 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Service d8b59866-61ec-4a37-8936-57e4ac10aa2a could not be found. Jul 16 21:12:35.982453 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:12:35.982453 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-6148c9f7-38e0-41b7-8476-a71a6614a386 tempest-ServicesAdminNegativeV253TestJSON-1167242690 tempest-ServicesAdminNegativeV253TestJSON-1167242690-project-admin] Returning 404 to user: Service d8b59866-61ec-4a37-8936-57e4ac10aa2a could not be found. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:12:35.982453 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-6148c9f7-38e0-41b7-8476-a71a6614a386 tempest-ServicesAdminNegativeV253TestJSON-1167242690 tempest-ServicesAdminNegativeV253TestJSON-1167242690-project-admin] 10.222.0.16 "PUT /compute/v2.1/os-services/d8b59866-61ec-4a37-8936-57e4ac10aa2a" status: 404 len: 110 microversion: 2.53 time: 0.037687 Jul 16 21:12:35.982453 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 626/1202] 10.222.0.16 () {62 vars in 1308 bytes} [Sat Jul 16 21:12:35 2022] PUT /compute/v2.1/os-services/d8b59866-61ec-4a37-8936-57e4ac10aa2a => generated 110 bytes in 39 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Jul 16 21:12:35.990729 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-17445526-a9dd-4a48-8786-e24becf9ac9d tempest-ServicesAdminNegativeV253TestJSON-1167242690 tempest-ServicesAdminNegativeV253TestJSON-1167242690-project-admin] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:35.991944 ncdv-master-1 devstack@n-api.service[138680]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-services:list": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:12:35.991944 ncdv-master-1 devstack@n-api.service[138680]: warnings.warn(msg) Jul 16 21:12:35.994044 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-17445526-a9dd-4a48-8786-e24becf9ac9d tempest-ServicesAdminNegativeV253TestJSON-1167242690 tempest-ServicesAdminNegativeV253TestJSON-1167242690-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:35.994505 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-17445526-a9dd-4a48-8786-e24becf9ac9d tempest-ServicesAdminNegativeV253TestJSON-1167242690 tempest-ServicesAdminNegativeV253TestJSON-1167242690-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:35.995352 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-17445526-a9dd-4a48-8786-e24becf9ac9d tempest-ServicesAdminNegativeV253TestJSON-1167242690 tempest-ServicesAdminNegativeV253TestJSON-1167242690-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:35.995794 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-17445526-a9dd-4a48-8786-e24becf9ac9d tempest-ServicesAdminNegativeV253TestJSON-1167242690 tempest-ServicesAdminNegativeV253TestJSON-1167242690-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:36.008626 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_api:SELECT=91,nova_api:INSERT=9 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:12:36.025550 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-17445526-a9dd-4a48-8786-e24becf9ac9d tempest-ServicesAdminNegativeV253TestJSON-1167242690 tempest-ServicesAdminNegativeV253TestJSON-1167242690-project-admin] 10.222.0.16 "GET /compute/v2.1/os-services" status: 200 len: 1242 microversion: 2.53 time: 0.038285 Jul 16 21:12:36.032370 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 627/1203] 10.222.0.16 () {60 vars in 1177 bytes} [Sat Jul 16 21:12:35 2022] GET /compute/v2.1/os-services => generated 1242 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:12:36.038995 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-2617e4c4-73df-4f0f-beed-5ae2ce9b5269 tempest-ServicesAdminNegativeV253TestJSON-1167242690 tempest-ServicesAdminNegativeV253TestJSON-1167242690-project-admin] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:36.042114 ncdv-master-1 devstack@n-api.service[138680]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-services:list": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:12:36.042114 ncdv-master-1 devstack@n-api.service[138680]: warnings.warn(msg) Jul 16 21:12:36.043695 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2617e4c4-73df-4f0f-beed-5ae2ce9b5269 tempest-ServicesAdminNegativeV253TestJSON-1167242690 tempest-ServicesAdminNegativeV253TestJSON-1167242690-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:36.044166 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2617e4c4-73df-4f0f-beed-5ae2ce9b5269 tempest-ServicesAdminNegativeV253TestJSON-1167242690 tempest-ServicesAdminNegativeV253TestJSON-1167242690-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:36.045690 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2617e4c4-73df-4f0f-beed-5ae2ce9b5269 tempest-ServicesAdminNegativeV253TestJSON-1167242690 tempest-ServicesAdminNegativeV253TestJSON-1167242690-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:36.046150 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2617e4c4-73df-4f0f-beed-5ae2ce9b5269 tempest-ServicesAdminNegativeV253TestJSON-1167242690 tempest-ServicesAdminNegativeV253TestJSON-1167242690-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:36.080324 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-2617e4c4-73df-4f0f-beed-5ae2ce9b5269 tempest-ServicesAdminNegativeV253TestJSON-1167242690 tempest-ServicesAdminNegativeV253TestJSON-1167242690-project-admin] 10.222.0.16 "GET /compute/v2.1/os-services" status: 200 len: 1242 microversion: 2.53 time: 0.045805 Jul 16 21:12:36.081266 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 628/1204] 10.222.0.16 () {60 vars in 1210 bytes} [Sat Jul 16 21:12:36 2022] GET /compute/v2.1/os-services?xxx=nova-compute => generated 1242 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 16 21:12:36.094891 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-b817a276-38c1-4d5f-8b54-46fb14777d5c tempest-ServicesAdminNegativeV253TestJSON-1167242690 tempest-ServicesAdminNegativeV253TestJSON-1167242690-project-admin] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:36.097875 ncdv-master-1 devstack@n-api.service[138680]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-services:list": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:12:36.097875 ncdv-master-1 devstack@n-api.service[138680]: warnings.warn(msg) Jul 16 21:12:36.113932 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b817a276-38c1-4d5f-8b54-46fb14777d5c tempest-ServicesAdminNegativeV253TestJSON-1167242690 tempest-ServicesAdminNegativeV253TestJSON-1167242690-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:36.116226 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b817a276-38c1-4d5f-8b54-46fb14777d5c tempest-ServicesAdminNegativeV253TestJSON-1167242690 tempest-ServicesAdminNegativeV253TestJSON-1167242690-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.018s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:36.117546 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b817a276-38c1-4d5f-8b54-46fb14777d5c tempest-ServicesAdminNegativeV253TestJSON-1167242690 tempest-ServicesAdminNegativeV253TestJSON-1167242690-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:36.118315 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b817a276-38c1-4d5f-8b54-46fb14777d5c tempest-ServicesAdminNegativeV253TestJSON-1167242690 tempest-ServicesAdminNegativeV253TestJSON-1167242690-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:36.160027 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-b817a276-38c1-4d5f-8b54-46fb14777d5c tempest-ServicesAdminNegativeV253TestJSON-1167242690 tempest-ServicesAdminNegativeV253TestJSON-1167242690-project-admin] 10.222.0.16 "GET /compute/v2.1/os-services" status: 200 len: 1242 microversion: 2.53 time: 0.069235 Jul 16 21:12:36.160949 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 629/1205] 10.222.0.16 () {60 vars in 1177 bytes} [Sat Jul 16 21:12:36 2022] GET /compute/v2.1/os-services => generated 1242 bytes in 71 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:12:36.168670 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-e2b06caa-b504-4585-820d-2b7d4b34cada tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:36.170127 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-e2b06caa-b504-4585-820d-2b7d4b34cada tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] [instance: 74a19091-819b-4cf0-8109-fbb0de65313c] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:12:36.184586 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e2b06caa-b504-4585-820d-2b7d4b34cada tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:36.185051 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e2b06caa-b504-4585-820d-2b7d4b34cada tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:36.185845 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e2b06caa-b504-4585-820d-2b7d4b34cada tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:36.186289 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e2b06caa-b504-4585-820d-2b7d4b34cada tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:36.554304 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-e2b06caa-b504-4585-820d-2b7d4b34cada tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56d4bbcdebc24a34a814d7f6e72fbcfd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7d2b583c48254e17a8082f6467f2ae6d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:36.562936 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-e2b06caa-b504-4585-820d-2b7d4b34cada tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] 10.222.0.16 "GET /compute/v2.1/servers/74a19091-819b-4cf0-8109-fbb0de65313c" status: 200 len: 1320 microversion: 2.1 time: 0.400905 Jul 16 21:12:36.564142 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 630/1206] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:12:36 2022] GET /compute/v2.1/servers/74a19091-819b-4cf0-8109-fbb0de65313c => generated 1320 bytes in 403 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:12:36.569179 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-1c063665-20d2-4c47-89cb-5bcb4331c554 tempest-ServicesAdminNegativeV253TestJSON-1167242690 tempest-ServicesAdminNegativeV253TestJSON-1167242690-project-admin] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:36.570930 ncdv-master-1 devstack@n-api.service[138680]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-services:list": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:12:36.570930 ncdv-master-1 devstack@n-api.service[138680]: warnings.warn(msg) Jul 16 21:12:36.572412 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1c063665-20d2-4c47-89cb-5bcb4331c554 tempest-ServicesAdminNegativeV253TestJSON-1167242690 tempest-ServicesAdminNegativeV253TestJSON-1167242690-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:36.573040 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1c063665-20d2-4c47-89cb-5bcb4331c554 tempest-ServicesAdminNegativeV253TestJSON-1167242690 tempest-ServicesAdminNegativeV253TestJSON-1167242690-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:36.574567 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1c063665-20d2-4c47-89cb-5bcb4331c554 tempest-ServicesAdminNegativeV253TestJSON-1167242690 tempest-ServicesAdminNegativeV253TestJSON-1167242690-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:36.574567 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1c063665-20d2-4c47-89cb-5bcb4331c554 tempest-ServicesAdminNegativeV253TestJSON-1167242690 tempest-ServicesAdminNegativeV253TestJSON-1167242690-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:36.599586 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-1c063665-20d2-4c47-89cb-5bcb4331c554 tempest-ServicesAdminNegativeV253TestJSON-1167242690 tempest-ServicesAdminNegativeV253TestJSON-1167242690-project-admin] 10.222.0.16 "GET /compute/v2.1/os-services?host=ncdv-master-1&binary=xxx" status: 200 len: 16 microversion: 2.53 time: 0.033937 Jul 16 21:12:36.600506 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 631/1207] 10.222.0.16 () {60 vars in 1236 bytes} [Sat Jul 16 21:12:36 2022] GET /compute/v2.1/os-services?host=ncdv-master-1&binary=xxx => generated 16 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:12:36.605917 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-6d687149-7f3d-4b52-92ef-d7f9098fecf8 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:36.606617 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-6d687149-7f3d-4b52-92ef-d7f9098fecf8 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] [instance: 3c5ab09e-0a97-4cfc-b249-a11ab221b5d2] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:12:36.613132 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6d687149-7f3d-4b52-92ef-d7f9098fecf8 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:36.613711 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6d687149-7f3d-4b52-92ef-d7f9098fecf8 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:36.614499 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6d687149-7f3d-4b52-92ef-d7f9098fecf8 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:36.614943 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6d687149-7f3d-4b52-92ef-d7f9098fecf8 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:36.848781 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-6d687149-7f3d-4b52-92ef-d7f9098fecf8 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:36.857135 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-6d687149-7f3d-4b52-92ef-d7f9098fecf8 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:36.858656 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-6d687149-7f3d-4b52-92ef-d7f9098fecf8 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:36.859826 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-6d687149-7f3d-4b52-92ef-d7f9098fecf8 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] 10.222.0.16 "GET /compute/v2.1/servers/3c5ab09e-0a97-4cfc-b249-a11ab221b5d2" status: 200 len: 1408 microversion: 2.25 time: 0.258300 Jul 16 21:12:36.870877 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 632/1208] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:12:36 2022] GET /compute/v2.1/servers/3c5ab09e-0a97-4cfc-b249-a11ab221b5d2 => generated 1408 bytes in 259 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:12:36.874627 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-281e79b4-5dd4-49d5-bd18-07437359bc32 req-db2d94f9-82c0-4420-8526-ac7633c87773 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "3c5ab09e-0a97-4cfc-b249-a11ab221b5d2", "tag": "4f9dfe87-5bed-49a6-8b89-464c4930e746"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:12:36.882804 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-281e79b4-5dd4-49d5-bd18-07437359bc32 req-db2d94f9-82c0-4420-8526-ac7633c87773 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:36.883365 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-281e79b4-5dd4-49d5-bd18-07437359bc32 req-db2d94f9-82c0-4420-8526-ac7633c87773 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:36.897220 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-281e79b4-5dd4-49d5-bd18-07437359bc32 req-db2d94f9-82c0-4420-8526-ac7633c87773 service nova] Creating event network-changed:4f9dfe87-5bed-49a6-8b89-464c4930e746 for instance 3c5ab09e-0a97-4cfc-b249-a11ab221b5d2 on nch2-master-1 Jul 16 21:12:36.903111 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-281e79b4-5dd4-49d5-bd18-07437359bc32 req-db2d94f9-82c0-4420-8526-ac7633c87773 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.041539 Jul 16 21:12:36.903719 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 633/1209] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:12:36 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:12:36.908783 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-9f7b09ca-bab3-4589-a1fb-8ddc1111bf5c tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:36.909490 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-9f7b09ca-bab3-4589-a1fb-8ddc1111bf5c tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: 126f561c-163f-4f6d-8be1-98a2e29b0909] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:12:36.916287 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9f7b09ca-bab3-4589-a1fb-8ddc1111bf5c tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:36.916754 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9f7b09ca-bab3-4589-a1fb-8ddc1111bf5c tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:36.917535 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9f7b09ca-bab3-4589-a1fb-8ddc1111bf5c tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:36.917985 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9f7b09ca-bab3-4589-a1fb-8ddc1111bf5c tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:37.130325 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-9f7b09ca-bab3-4589-a1fb-8ddc1111bf5c tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b68dacc4b8463ba07fae6372131117', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55a19b7a4682457a9132f5fc80110712', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:37.139305 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-9f7b09ca-bab3-4589-a1fb-8ddc1111bf5c tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "GET /compute/v2.1/servers/126f561c-163f-4f6d-8be1-98a2e29b0909" status: 200 len: 1313 microversion: 2.1 time: 0.234370 Jul 16 21:12:37.140231 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 634/1210] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:12:36 2022] GET /compute/v2.1/servers/126f561c-163f-4f6d-8be1-98a2e29b0909 => generated 1313 bytes in 236 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:12:37.144637 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-5ea3f50f-8288-45e8-a2c2-349108bbbaef tempest-ServicesAdminNegativeV253TestJSON-1167242690 tempest-ServicesAdminNegativeV253TestJSON-1167242690-project-admin] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:37.145959 ncdv-master-1 devstack@n-api.service[138680]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-services:list": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:12:37.145959 ncdv-master-1 devstack@n-api.service[138680]: warnings.warn(msg) Jul 16 21:12:37.147328 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5ea3f50f-8288-45e8-a2c2-349108bbbaef tempest-ServicesAdminNegativeV253TestJSON-1167242690 tempest-ServicesAdminNegativeV253TestJSON-1167242690-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:37.147790 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5ea3f50f-8288-45e8-a2c2-349108bbbaef tempest-ServicesAdminNegativeV253TestJSON-1167242690 tempest-ServicesAdminNegativeV253TestJSON-1167242690-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:37.148663 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5ea3f50f-8288-45e8-a2c2-349108bbbaef tempest-ServicesAdminNegativeV253TestJSON-1167242690 tempest-ServicesAdminNegativeV253TestJSON-1167242690-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:37.149272 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5ea3f50f-8288-45e8-a2c2-349108bbbaef tempest-ServicesAdminNegativeV253TestJSON-1167242690 tempest-ServicesAdminNegativeV253TestJSON-1167242690-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:37.173992 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-5ea3f50f-8288-45e8-a2c2-349108bbbaef tempest-ServicesAdminNegativeV253TestJSON-1167242690 tempest-ServicesAdminNegativeV253TestJSON-1167242690-project-admin] 10.222.0.16 "GET /compute/v2.1/os-services" status: 200 len: 1242 microversion: 2.53 time: 0.032642 Jul 16 21:12:37.174760 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 635/1211] 10.222.0.16 () {60 vars in 1177 bytes} [Sat Jul 16 21:12:37 2022] GET /compute/v2.1/os-services => generated 1242 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:12:37.179485 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-a23c3819-8a68-4a0e-8990-b73e6684a532 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:37.181136 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-a23c3819-8a68-4a0e-8990-b73e6684a532 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] [instance: 6a391156-ffce-49c0-a79d-47b4544264f5] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:12:37.187796 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a23c3819-8a68-4a0e-8990-b73e6684a532 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:37.188250 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a23c3819-8a68-4a0e-8990-b73e6684a532 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:37.189021 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a23c3819-8a68-4a0e-8990-b73e6684a532 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:37.189458 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a23c3819-8a68-4a0e-8990-b73e6684a532 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:37.423674 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-a23c3819-8a68-4a0e-8990-b73e6684a532 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a69f9dee03d43beaafa966216b9e289', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9b769bf8f3ea4e368bf68de1df6538f9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:37.435333 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-a23c3819-8a68-4a0e-8990-b73e6684a532 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] 10.222.0.16 "GET /compute/v2.1/servers/6a391156-ffce-49c0-a79d-47b4544264f5" status: 200 len: 1717 microversion: 2.1 time: 0.259153 Jul 16 21:12:37.436175 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 636/1212] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:12:37 2022] GET /compute/v2.1/servers/6a391156-ffce-49c0-a79d-47b4544264f5 => generated 1717 bytes in 261 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:12:37.440947 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-e8d20f3c-0b8e-40d4-b652-d022f8870214 tempest-ServicesAdminNegativeV253TestJSON-1167242690 tempest-ServicesAdminNegativeV253TestJSON-1167242690-project-admin] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:37.442437 ncdv-master-1 devstack@n-api.service[138680]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-services:list": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:12:37.442437 ncdv-master-1 devstack@n-api.service[138680]: warnings.warn(msg) Jul 16 21:12:37.443831 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e8d20f3c-0b8e-40d4-b652-d022f8870214 tempest-ServicesAdminNegativeV253TestJSON-1167242690 tempest-ServicesAdminNegativeV253TestJSON-1167242690-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:37.444384 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e8d20f3c-0b8e-40d4-b652-d022f8870214 tempest-ServicesAdminNegativeV253TestJSON-1167242690 tempest-ServicesAdminNegativeV253TestJSON-1167242690-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:37.445425 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e8d20f3c-0b8e-40d4-b652-d022f8870214 tempest-ServicesAdminNegativeV253TestJSON-1167242690 tempest-ServicesAdminNegativeV253TestJSON-1167242690-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:37.445964 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e8d20f3c-0b8e-40d4-b652-d022f8870214 tempest-ServicesAdminNegativeV253TestJSON-1167242690 tempest-ServicesAdminNegativeV253TestJSON-1167242690-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:37.469596 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-e8d20f3c-0b8e-40d4-b652-d022f8870214 tempest-ServicesAdminNegativeV253TestJSON-1167242690 tempest-ServicesAdminNegativeV253TestJSON-1167242690-project-admin] 10.222.0.16 "GET /compute/v2.1/os-services?host=xxx&binary=nova-scheduler" status: 200 len: 16 microversion: 2.53 time: 0.032045 Jul 16 21:12:37.476594 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 637/1213] 10.222.0.16 () {60 vars in 1238 bytes} [Sat Jul 16 21:12:37 2022] GET /compute/v2.1/os-services?host=xxx&binary=nova-scheduler => generated 16 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:12:37.623376 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-25c6a502-ee5d-49b0-bb20-43828b40f5be tempest-ServicesAdminNegativeV253TestJSON-1083131952 tempest-ServicesAdminNegativeV253TestJSON-1083131952-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:37.624885 ncdv-master-1 devstack@n-api.service[138680]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-services:list": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:12:37.624885 ncdv-master-1 devstack@n-api.service[138680]: warnings.warn(msg) Jul 16 21:12:37.626197 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-25c6a502-ee5d-49b0-bb20-43828b40f5be tempest-ServicesAdminNegativeV253TestJSON-1083131952 tempest-ServicesAdminNegativeV253TestJSON-1083131952-project] Policy check for os_compute_api:os-services:list failed with credentials {'is_admin': False, 'user_id': '732d3205d99740469700a324f0054a22', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '590e71ab4bc34f53a63735fa625d7c2e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:37.628149 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-25c6a502-ee5d-49b0-bb20-43828b40f5be tempest-ServicesAdminNegativeV253TestJSON-1083131952 tempest-ServicesAdminNegativeV253TestJSON-1083131952-project] Request method failure captured: Jul 16 21:12:37.628149 ncdv-master-1 devstack@n-api.service[138680]: request: GET /compute/v2.1/os-services HTTP/1.1 Jul 16 21:12:37.628149 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:12:37.628149 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:12:37.628149 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:12:37.628149 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 0 Jul 16 21:12:37.628149 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:12:37.628149 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:12:37.628149 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:12:37.628149 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:12:37.628149 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0ym_7W1GPwNJAYvE_yIaRVaUfSd0kZufIAltlbzVeaYQtRDc9-3aD8uNN65VH50JMH6ADfSeRMRS7-6AUILpD3HGI7-CLp69EkBXMu8rFvX3jGougMsoqXQHAgxc00HCXNx8GZfJlK6lK7jSKKX7JmCN6auUxbXIK4UYxURVFS1jNAs Jul 16 21:12:37.628149 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:12:37.628149 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:12:37.628149 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:12:37.628149 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:12:37.628149 ncdv-master-1 devstack@n-api.service[138680]: X-Openstack-Nova-Api-Version: 2.53 Jul 16 21:12:37.628149 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:12:37.628149 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:12:37.628149 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: 590e71ab4bc34f53a63735fa625d7c2e Jul 16 21:12:37.628149 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-ServicesAdminNegativeV253TestJSON-1083131952 Jul 16 21:12:37.628149 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:12:37.628149 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:12:37.629754 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/590e71ab4bc34f53a63735fa625d7c2e"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/590e71ab4bc34f53a63735fa625d7c2e"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_590e71ab4bc34f53a63735fa625d7c2e"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/590e71ab4bc34f53a63735fa625d7c2e"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:12:37.629754 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-ServicesAdminNegativeV253TestJSON-1083131952 Jul 16 21:12:37.629754 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: 590e71ab4bc34f53a63735fa625d7c2e Jul 16 21:12:37.629754 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-ServicesAdminNegativeV253TestJSON-1083131952 Jul 16 21:12:37.629754 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-ServicesAdminNegativeV253TestJSON-1083131952-project Jul 16 21:12:37.629754 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:12:37.629754 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:12:37.629754 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: 732d3205d99740469700a324f0054a22 Jul 16 21:12:37.629754 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-ServicesAdminNegativeV253TestJSON-1083131952-project Jul 16 21:12:37.629754 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 21:12:37.629754 ncdv-master-1 devstack@n-api.service[138680]: exception: Policy doesn't allow os_compute_api:os-services:list to be performed. Jul 16 21:12:37.629754 ncdv-master-1 devstack@n-api.service[138680]: action_args: {} Jul 16 21:12:37.630644 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:12:37.630644 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:12:37.630644 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:12:37.630644 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:12:37.630644 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:12:37.630644 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:12:37.630644 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper Jul 16 21:12:37.630644 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 16 21:12:37.630644 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper Jul 16 21:12:37.630644 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 16 21:12:37.630644 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:12:37.630644 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:12:37.630644 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/services.py", line 387, in index Jul 16 21:12:37.630644 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi context.can(services_policies.BASE_POLICY_NAME % 'list', target={}) Jul 16 21:12:37.630644 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/context.py", line 240, in can Jul 16 21:12:37.630644 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return policy.authorize(self, action, target) Jul 16 21:12:37.630644 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 199, in authorize Jul 16 21:12:37.630644 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi LOG.debug('Policy check for %(action)s failed with credentials ' Jul 16 21:12:37.630644 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 16 21:12:37.631848 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi self.force_reraise() Jul 16 21:12:37.631848 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 16 21:12:37.631848 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise self.value Jul 16 21:12:37.631848 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 186, in authorize Jul 16 21:12:37.631848 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi result = _ENFORCER.authorize(action, target, context, Jul 16 21:12:37.631848 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1184, in authorize Jul 16 21:12:37.631848 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return self.enforce( Jul 16 21:12:37.631848 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1088, in enforce Jul 16 21:12:37.631848 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc(*args, **kwargs) Jul 16 21:12:37.631848 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.PolicyNotAuthorized: Policy doesn't allow os_compute_api:os-services:list to be performed. Jul 16 21:12:37.631848 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:12:37.631848 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:12:37.631848 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:12:37.631848 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:12:37.631848 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:12:37.631848 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:12:37.631848 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 354, in __exit__ Jul 16 21:12:37.631848 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(webob.exc.HTTPForbidden( Jul 16 21:12:37.631848 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Policy doesn't allow os_compute_api:os-services:list to be performed. Jul 16 21:12:37.631848 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:12:37.633020 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-25c6a502-ee5d-49b0-bb20-43828b40f5be tempest-ServicesAdminNegativeV253TestJSON-1083131952 tempest-ServicesAdminNegativeV253TestJSON-1083131952-project] Returning 403 to user: Policy doesn't allow os_compute_api:os-services:list to be performed. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:12:37.633020 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-25c6a502-ee5d-49b0-bb20-43828b40f5be tempest-ServicesAdminNegativeV253TestJSON-1083131952 tempest-ServicesAdminNegativeV253TestJSON-1083131952-project] 10.222.0.16 "GET /compute/v2.1/os-services" status: 403 len: 112 microversion: 2.53 time: 0.151325 Jul 16 21:12:37.633020 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 638/1214] 10.222.0.16 () {60 vars in 1177 bytes} [Sat Jul 16 21:12:37 2022] GET /compute/v2.1/os-services => generated 112 bytes in 153 msecs (HTTP/1.1 403) 9 headers in 381 bytes (1 switches on core 0) Jul 16 21:12:37.637673 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-859a6bae-ffac-43f7-9768-752a5db95b89 tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:37.638363 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-859a6bae-ffac-43f7-9768-752a5db95b89 tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] [instance: 74a19091-819b-4cf0-8109-fbb0de65313c] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:12:37.646937 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-859a6bae-ffac-43f7-9768-752a5db95b89 tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:37.650779 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-859a6bae-ffac-43f7-9768-752a5db95b89 tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:37.650779 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-859a6bae-ffac-43f7-9768-752a5db95b89 tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:37.650779 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-859a6bae-ffac-43f7-9768-752a5db95b89 tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:38.042242 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-859a6bae-ffac-43f7-9768-752a5db95b89 tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56d4bbcdebc24a34a814d7f6e72fbcfd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7d2b583c48254e17a8082f6467f2ae6d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:38.053202 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-859a6bae-ffac-43f7-9768-752a5db95b89 tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] 10.222.0.16 "GET /compute/v2.1/servers/74a19091-819b-4cf0-8109-fbb0de65313c" status: 200 len: 1320 microversion: 2.1 time: 0.421282 Jul 16 21:12:38.065350 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 639/1215] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:12:37 2022] GET /compute/v2.1/servers/74a19091-819b-4cf0-8109-fbb0de65313c => generated 1320 bytes in 434 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:12:38.081588 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-9f751e51-3316-44b5-a589-882ba2ff78fa tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:38.083529 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-9f751e51-3316-44b5-a589-882ba2ff78fa tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] [instance: 3c5ab09e-0a97-4cfc-b249-a11ab221b5d2] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:12:38.093435 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9f751e51-3316-44b5-a589-882ba2ff78fa tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:38.094931 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9f751e51-3316-44b5-a589-882ba2ff78fa tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:38.096668 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9f751e51-3316-44b5-a589-882ba2ff78fa tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:38.097358 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9f751e51-3316-44b5-a589-882ba2ff78fa tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:38.778481 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-9f751e51-3316-44b5-a589-882ba2ff78fa tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:38.789082 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-9f751e51-3316-44b5-a589-882ba2ff78fa tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:38.803520 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-9f751e51-3316-44b5-a589-882ba2ff78fa tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:38.803520 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-9f751e51-3316-44b5-a589-882ba2ff78fa tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] 10.222.0.16 "GET /compute/v2.1/servers/3c5ab09e-0a97-4cfc-b249-a11ab221b5d2" status: 200 len: 1408 microversion: 2.25 time: 0.718489 Jul 16 21:12:38.803520 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 640/1216] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:12:38 2022] GET /compute/v2.1/servers/3c5ab09e-0a97-4cfc-b249-a11ab221b5d2 => generated 1408 bytes in 727 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:12:38.810072 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-dd8f5cb1-20ec-49c5-8d01-9cd28697751e req-702e9d7c-26c9-408e-959e-71232b0b193a service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "126f561c-163f-4f6d-8be1-98a2e29b0909", "tag": "3b739133-7af5-4df3-a04d-fff7e0a81a3d"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:12:38.819222 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-dd8f5cb1-20ec-49c5-8d01-9cd28697751e req-702e9d7c-26c9-408e-959e-71232b0b193a service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:38.819906 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-dd8f5cb1-20ec-49c5-8d01-9cd28697751e req-702e9d7c-26c9-408e-959e-71232b0b193a service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:38.833235 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-dd8f5cb1-20ec-49c5-8d01-9cd28697751e req-702e9d7c-26c9-408e-959e-71232b0b193a service nova] Creating event network-changed:3b739133-7af5-4df3-a04d-fff7e0a81a3d for instance 126f561c-163f-4f6d-8be1-98a2e29b0909 on nch2-master-1 Jul 16 21:12:38.843193 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-dd8f5cb1-20ec-49c5-8d01-9cd28697751e req-702e9d7c-26c9-408e-959e-71232b0b193a service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.037259 Jul 16 21:12:38.843820 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 641/1217] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:12:38 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:12:38.849598 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-ae839e5a-9d3e-470f-a3c4-48d8e956acd1 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:38.850318 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-ae839e5a-9d3e-470f-a3c4-48d8e956acd1 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: 126f561c-163f-4f6d-8be1-98a2e29b0909] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:12:38.857419 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ae839e5a-9d3e-470f-a3c4-48d8e956acd1 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:38.857879 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ae839e5a-9d3e-470f-a3c4-48d8e956acd1 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:38.858712 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ae839e5a-9d3e-470f-a3c4-48d8e956acd1 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:38.859192 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ae839e5a-9d3e-470f-a3c4-48d8e956acd1 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:39.088584 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-ae839e5a-9d3e-470f-a3c4-48d8e956acd1 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b68dacc4b8463ba07fae6372131117', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55a19b7a4682457a9132f5fc80110712', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:39.097089 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-ae839e5a-9d3e-470f-a3c4-48d8e956acd1 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "GET /compute/v2.1/servers/126f561c-163f-4f6d-8be1-98a2e29b0909" status: 200 len: 1313 microversion: 2.1 time: 0.251887 Jul 16 21:12:39.098481 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 642/1218] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:12:38 2022] GET /compute/v2.1/servers/126f561c-163f-4f6d-8be1-98a2e29b0909 => generated 1313 bytes in 254 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:12:39.103284 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.network.neutron [None req-854fb16e-dcb8-4c80-a5bb-9985f9417d81 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] [instance: 4ae3e944-b450-40e5-bb71-c02df03212e3] re-assign floating IP 172.35.1.237 from instance 8020f553-6025-4f19-835c-c00e2fb4f764 Jul 16 21:12:39.160708 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.network.neutron [None req-854fb16e-dcb8-4c80-a5bb-9985f9417d81 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] [instance: 8020f553-6025-4f19-835c-c00e2fb4f764] Building network info cache for instance {{(pid=138681) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1990}} Jul 16 21:12:39.417949 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-c5ff4843-49c3-420c-ad12-2ef08825910f tempest-VolumesAdminNegativeTest-1185523518 tempest-VolumesAdminNegativeTest-1185523518-project-admin] Action: 'update', calling method: >, body: {"volumeAttachment": {"volumeId": "9bbb8809-60e0-47f2-b183-285c0e3edc3f"}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:12:39.419174 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-c5ff4843-49c3-420c-ad12-2ef08825910f tempest-VolumesAdminNegativeTest-1185523518 tempest-VolumesAdminNegativeTest-1185523518-project-admin] [instance: b07f0d53-2cd4-440f-be47-077703aa66e6] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:12:39.430106 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c5ff4843-49c3-420c-ad12-2ef08825910f tempest-VolumesAdminNegativeTest-1185523518 tempest-VolumesAdminNegativeTest-1185523518-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:39.432024 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c5ff4843-49c3-420c-ad12-2ef08825910f tempest-VolumesAdminNegativeTest-1185523518 tempest-VolumesAdminNegativeTest-1185523518-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:39.433123 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c5ff4843-49c3-420c-ad12-2ef08825910f tempest-VolumesAdminNegativeTest-1185523518 tempest-VolumesAdminNegativeTest-1185523518-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:39.434043 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c5ff4843-49c3-420c-ad12-2ef08825910f tempest-VolumesAdminNegativeTest-1185523518 tempest-VolumesAdminNegativeTest-1185523518-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:39.617356 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell0:SELECT=28 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:12:39.633946 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-c5ff4843-49c3-420c-ad12-2ef08825910f tempest-VolumesAdminNegativeTest-1185523518 tempest-VolumesAdminNegativeTest-1185523518-project-admin] HTTP exception thrown: Volume 9bbb8809-60e0-47f2-b183-285c0e3edc3f could not be found. Jul 16 21:12:39.642794 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-c5ff4843-49c3-420c-ad12-2ef08825910f tempest-VolumesAdminNegativeTest-1185523518 tempest-VolumesAdminNegativeTest-1185523518-project-admin] Request method failure captured: Jul 16 21:12:39.642794 ncdv-master-1 devstack@n-api.service[138680]: request: PUT /compute/v2.1/servers/b07f0d53-2cd4-440f-be47-077703aa66e6/os-volume_attachments/b61a51e0-885e-44b7-8fc0-f32fe665e112 HTTP/1.1 Jul 16 21:12:39.642794 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:12:39.642794 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:12:39.642794 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:12:39.642794 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 74 Jul 16 21:12:39.642794 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:12:39.642794 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:12:39.642794 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:12:39.642794 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:12:39.642794 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0ylyPrDr_ELiD-cVGkDdxIwIy9yztHbKzORPMKyk_Z-quXpZuUroAQ7B6aRJ_xMnhc7Mw9L85FOxhXg0BkOCB3k_jmKYEXDG1DCKKyD2w4H3kaQh0PNoOYkTf-uFALfgRbkzuyZ4NwMUzNG81KX7XtzeBIVNZDY0W8riazzimeQdxUE Jul 16 21:12:39.642794 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:12:39.642794 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:12:39.642794 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:12:39.642794 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:12:39.642794 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:12:39.642794 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:12:39.642794 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: f27ffd13e54d41ebb9e79a74725cb3bf Jul 16 21:12:39.642794 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-VolumesAdminNegativeTest-1185523518 Jul 16 21:12:39.642794 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,admin,reader Jul 16 21:12:39.642794 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,admin,reader Jul 16 21:12:39.646158 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/f27ffd13e54d41ebb9e79a74725cb3bf"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/f27ffd13e54d41ebb9e79a74725cb3bf"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_f27ffd13e54d41ebb9e79a74725cb3bf"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/f27ffd13e54d41ebb9e79a74725cb3bf"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:12:39.646158 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-VolumesAdminNegativeTest-1185523518 Jul 16 21:12:39.646158 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: f27ffd13e54d41ebb9e79a74725cb3bf Jul 16 21:12:39.646158 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-VolumesAdminNegativeTest-1185523518 Jul 16 21:12:39.646158 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-VolumesAdminNegativeTest-1185523518-project-admin Jul 16 21:12:39.646158 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:12:39.646158 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:12:39.646158 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: 6dfd80a71b204625b5401e5249a0bbed Jul 16 21:12:39.646158 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-VolumesAdminNegativeTest-1185523518-project-admin Jul 16 21:12:39.646158 ncdv-master-1 devstack@n-api.service[138680]: Jul 16 21:12:39.646158 ncdv-master-1 devstack@n-api.service[138680]: {"volumeAttachment": {"volumeId": "9bbb8809-60e0-47f2-b183-285c0e3edc3f"}} Jul 16 21:12:39.646158 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 21:12:39.646158 ncdv-master-1 devstack@n-api.service[138680]: exception: Volume 9bbb8809-60e0-47f2-b183-285c0e3edc3f could not be found. Jul 16 21:12:39.647178 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'server_id': 'b07f0d53-2cd4-440f-be47-077703aa66e6', 'id': 'b61a51e0-885e-44b7-8fc0-f32fe665e112', 'body': {'volumeAttachment': {'volumeId': '9bbb8809-60e0-47f2-b183-285c0e3edc3f'}}} Jul 16 21:12:39.647178 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:12:39.647178 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:12:39.647178 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jul 16 21:12:39.647178 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jul 16 21:12:39.647178 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Jul 16 21:12:39.647178 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi item = cinderclient( Jul 16 21:12:39.647178 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Jul 16 21:12:39.647178 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Jul 16 21:12:39.647178 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Jul 16 21:12:39.647178 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Jul 16 21:12:39.647178 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Jul 16 21:12:39.647178 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Jul 16 21:12:39.647178 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jul 16 21:12:39.647178 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jul 16 21:12:39.647178 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jul 16 21:12:39.647178 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jul 16 21:12:39.648406 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume 9bbb8809-60e0-47f2-b183-285c0e3edc3f could not be found. (HTTP 404) (Request-ID: req-cfe2fa0e-971a-499f-b102-99170dd8b647) Jul 16 21:12:39.648406 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:12:39.648406 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:12:39.648406 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:12:39.648406 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:12:39.648406 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 436, in _update_volume_swap Jul 16 21:12:39.648406 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi new_volume = self.volume_api.get(context, new_volume_id) Jul 16 21:12:39.648406 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Jul 16 21:12:39.648406 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Jul 16 21:12:39.648406 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Jul 16 21:12:39.648406 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Jul 16 21:12:39.648406 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Jul 16 21:12:39.648406 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Jul 16 21:12:39.648406 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jul 16 21:12:39.648406 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jul 16 21:12:39.648406 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Jul 16 21:12:39.648406 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi item = cinderclient( Jul 16 21:12:39.648406 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Jul 16 21:12:39.648406 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Jul 16 21:12:39.650885 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Jul 16 21:12:39.650885 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Jul 16 21:12:39.650885 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Jul 16 21:12:39.650885 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Jul 16 21:12:39.650885 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jul 16 21:12:39.650885 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jul 16 21:12:39.650885 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jul 16 21:12:39.650885 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jul 16 21:12:39.650885 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume 9bbb8809-60e0-47f2-b183-285c0e3edc3f could not be found. Jul 16 21:12:39.650885 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:12:39.650885 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:12:39.650885 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:12:39.650885 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:12:39.650885 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:12:39.650885 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:12:39.650885 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:12:39.650885 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:12:39.650885 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:12:39.650885 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:12:39.652409 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 16 21:12:39.652409 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 16 21:12:39.652409 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 16 21:12:39.652409 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 16 21:12:39.652409 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 522, in update Jul 16 21:12:39.652409 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi self._update_volume_swap(req, instance, id, body) Jul 16 21:12:39.652409 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 445, in _update_volume_swap Jul 16 21:12:39.652409 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Jul 16 21:12:39.652409 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Volume 9bbb8809-60e0-47f2-b183-285c0e3edc3f could not be found. Jul 16 21:12:39.652409 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:12:39.652409 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:12:39.652409 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:12:39.652409 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:12:39.652409 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:12:39.652409 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:12:39.652409 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:12:39.652409 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:12:39.652409 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume 9bbb8809-60e0-47f2-b183-285c0e3edc3f could not be found. Jul 16 21:12:39.652409 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:12:39.653802 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-c5ff4843-49c3-420c-ad12-2ef08825910f tempest-VolumesAdminNegativeTest-1185523518 tempest-VolumesAdminNegativeTest-1185523518-project-admin] Returning 400 to user: Volume 9bbb8809-60e0-47f2-b183-285c0e3edc3f could not be found. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:12:39.653802 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-c5ff4843-49c3-420c-ad12-2ef08825910f tempest-VolumesAdminNegativeTest-1185523518 tempest-VolumesAdminNegativeTest-1185523518-project-admin] 10.222.0.16 "PUT /compute/v2.1/servers/b07f0d53-2cd4-440f-be47-077703aa66e6/os-volume_attachments/b61a51e0-885e-44b7-8fc0-f32fe665e112" status: 400 len: 107 microversion: 2.1 time: 0.550906 Jul 16 21:12:39.656244 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 643/1219] 10.222.0.16 () {60 vars in 1432 bytes} [Sat Jul 16 21:12:39 2022] PUT /compute/v2.1/servers/b07f0d53-2cd4-440f-be47-077703aa66e6/os-volume_attachments/b61a51e0-885e-44b7-8fc0-f32fe665e112 => generated 107 bytes in 557 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Jul 16 21:12:39.679725 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-3b146fa3-bac0-4965-b051-53dcdce90909 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:39.680816 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-3b146fa3-bac0-4965-b051-53dcdce90909 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] [instance: 6a391156-ffce-49c0-a79d-47b4544264f5] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:12:39.696746 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3b146fa3-bac0-4965-b051-53dcdce90909 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:39.697703 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3b146fa3-bac0-4965-b051-53dcdce90909 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:39.698894 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3b146fa3-bac0-4965-b051-53dcdce90909 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:39.699544 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3b146fa3-bac0-4965-b051-53dcdce90909 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:40.099863 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-3b146fa3-bac0-4965-b051-53dcdce90909 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a69f9dee03d43beaafa966216b9e289', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9b769bf8f3ea4e368bf68de1df6538f9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:40.113651 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-3b146fa3-bac0-4965-b051-53dcdce90909 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] 10.222.0.16 "GET /compute/v2.1/servers/6a391156-ffce-49c0-a79d-47b4544264f5" status: 200 len: 1717 microversion: 2.1 time: 0.455904 Jul 16 21:12:40.118654 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 644/1220] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:12:39 2022] GET /compute/v2.1/servers/6a391156-ffce-49c0-a79d-47b4544264f5 => generated 1717 bytes in 462 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:12:40.131078 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-3c57a0ef-c794-4ed2-afb8-f056099c212b req-a652e8f5-1807-4ae8-b806-ecc1532aa044 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "8020f553-6025-4f19-835c-c00e2fb4f764", "tag": "05294e88-e71a-4faa-8acc-79137ca6ee05"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:12:40.142614 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-3c57a0ef-c794-4ed2-afb8-f056099c212b req-a652e8f5-1807-4ae8-b806-ecc1532aa044 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:40.143473 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-3c57a0ef-c794-4ed2-afb8-f056099c212b req-a652e8f5-1807-4ae8-b806-ecc1532aa044 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:40.158809 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-3c57a0ef-c794-4ed2-afb8-f056099c212b req-a652e8f5-1807-4ae8-b806-ecc1532aa044 service nova] Creating event network-changed:05294e88-e71a-4faa-8acc-79137ca6ee05 for instance 8020f553-6025-4f19-835c-c00e2fb4f764 on nch1-master-1 Jul 16 21:12:40.166012 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-3c57a0ef-c794-4ed2-afb8-f056099c212b req-a652e8f5-1807-4ae8-b806-ecc1532aa044 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.044947 Jul 16 21:12:40.167174 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 645/1221] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:12:40 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:12:40.175573 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-2bfaddb4-0a0f-473b-8100-3acaff4a116a tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:40.176685 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-2bfaddb4-0a0f-473b-8100-3acaff4a116a tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] [instance: 74a19091-819b-4cf0-8109-fbb0de65313c] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:12:40.184545 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2bfaddb4-0a0f-473b-8100-3acaff4a116a tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:40.185266 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2bfaddb4-0a0f-473b-8100-3acaff4a116a tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:40.186683 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2bfaddb4-0a0f-473b-8100-3acaff4a116a tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:40.187340 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2bfaddb4-0a0f-473b-8100-3acaff4a116a tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:40.189746 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.network.neutron [None req-854fb16e-dcb8-4c80-a5bb-9985f9417d81 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] [instance: 8020f553-6025-4f19-835c-c00e2fb4f764] Updating instance_info_cache with network_info: [{"id": "05294e88-e71a-4faa-8acc-79137ca6ee05", "address": "fa:16:3e:30:6c:72", "network": {"id": "9b8d8e4f-14e5-4e94-bfa5-2387fb1adcf8", "bridge": null, "label": "tempest-FloatingIPsAssociationTestJSON-379677821-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "172.20.0.2"}}], "meta": {"injected": false, "tenant_id": "cda86413e74b4c1e8e09bd4df4926664", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false, "bound_drivers": {"0": "hyperv"}}, "devname": "tap05294e88-e7", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=138681) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:117}} Jul 16 21:12:40.507404 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.network.neutron [None req-854fb16e-dcb8-4c80-a5bb-9985f9417d81 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] [instance: 4ae3e944-b450-40e5-bb71-c02df03212e3] Building network info cache for instance {{(pid=138681) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1990}} Jul 16 21:12:40.584905 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-2bfaddb4-0a0f-473b-8100-3acaff4a116a tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56d4bbcdebc24a34a814d7f6e72fbcfd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7d2b583c48254e17a8082f6467f2ae6d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:40.599384 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-2bfaddb4-0a0f-473b-8100-3acaff4a116a tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] 10.222.0.16 "GET /compute/v2.1/servers/74a19091-819b-4cf0-8109-fbb0de65313c" status: 200 len: 1320 microversion: 2.1 time: 0.430749 Jul 16 21:12:40.600281 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 646/1222] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:12:40 2022] GET /compute/v2.1/servers/74a19091-819b-4cf0-8109-fbb0de65313c => generated 1320 bytes in 432 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:12:40.617160 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-f4f0db99-37b1-489f-882c-61dbc48c2e9c tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:40.618030 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-f4f0db99-37b1-489f-882c-61dbc48c2e9c tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] [instance: b07f0d53-2cd4-440f-be47-077703aa66e6] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:12:40.629894 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f4f0db99-37b1-489f-882c-61dbc48c2e9c tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:40.630691 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f4f0db99-37b1-489f-882c-61dbc48c2e9c tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:40.631676 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f4f0db99-37b1-489f-882c-61dbc48c2e9c tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:40.633057 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f4f0db99-37b1-489f-882c-61dbc48c2e9c tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:40.805403 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.objects.instance [None req-f4f0db99-37b1-489f-882c-61dbc48c2e9c tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Lazy-loading 'services' on Instance uuid b07f0d53-2cd4-440f-be47-077703aa66e6 {{(pid=138680) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:12:41.299804 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-f4f0db99-37b1-489f-882c-61dbc48c2e9c tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] 10.222.0.16 "DELETE /compute/v2.1/servers/b07f0d53-2cd4-440f-be47-077703aa66e6/os-volume_attachments/b61a51e0-885e-44b7-8fc0-f32fe665e112" status: 202 len: 0 microversion: 2.1 time: 0.697345 Jul 16 21:12:41.302082 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 647/1223] 10.222.0.16 () {58 vars in 1415 bytes} [Sat Jul 16 21:12:40 2022] DELETE /compute/v2.1/servers/b07f0d53-2cd4-440f-be47-077703aa66e6/os-volume_attachments/b61a51e0-885e-44b7-8fc0-f32fe665e112 => generated 0 bytes in 701 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 21:12:41.310669 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-70398e9d-4e8e-4960-9e19-3acce85356ea tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:41.312448 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-70398e9d-4e8e-4960-9e19-3acce85356ea tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] [instance: 3c5ab09e-0a97-4cfc-b249-a11ab221b5d2] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:12:41.323494 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-70398e9d-4e8e-4960-9e19-3acce85356ea tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:41.323494 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-70398e9d-4e8e-4960-9e19-3acce85356ea tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:41.323494 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-70398e9d-4e8e-4960-9e19-3acce85356ea tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:41.323494 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-70398e9d-4e8e-4960-9e19-3acce85356ea tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:41.686094 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-70398e9d-4e8e-4960-9e19-3acce85356ea tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:41.694945 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-70398e9d-4e8e-4960-9e19-3acce85356ea tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:41.697687 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-70398e9d-4e8e-4960-9e19-3acce85356ea tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:41.699050 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-70398e9d-4e8e-4960-9e19-3acce85356ea tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] 10.222.0.16 "GET /compute/v2.1/servers/3c5ab09e-0a97-4cfc-b249-a11ab221b5d2" status: 200 len: 1408 microversion: 2.25 time: 0.395439 Jul 16 21:12:41.699605 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 648/1224] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:12:41 2022] GET /compute/v2.1/servers/3c5ab09e-0a97-4cfc-b249-a11ab221b5d2 => generated 1408 bytes in 397 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:12:41.704741 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-5ca91758-1774-409e-a9d4-58ccd5bb8e2e tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:41.705349 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-5ca91758-1774-409e-a9d4-58ccd5bb8e2e tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: 126f561c-163f-4f6d-8be1-98a2e29b0909] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:12:41.715672 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5ca91758-1774-409e-a9d4-58ccd5bb8e2e tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:41.716075 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5ca91758-1774-409e-a9d4-58ccd5bb8e2e tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:41.716858 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5ca91758-1774-409e-a9d4-58ccd5bb8e2e tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:41.718025 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5ca91758-1774-409e-a9d4-58ccd5bb8e2e tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:42.388268 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-5ca91758-1774-409e-a9d4-58ccd5bb8e2e tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b68dacc4b8463ba07fae6372131117', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55a19b7a4682457a9132f5fc80110712', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:42.396859 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=99,nova_cell1:INSERT=1 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:12:42.405489 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-5ca91758-1774-409e-a9d4-58ccd5bb8e2e tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "GET /compute/v2.1/servers/126f561c-163f-4f6d-8be1-98a2e29b0909" status: 200 len: 1313 microversion: 2.1 time: 0.704451 Jul 16 21:12:42.407889 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 649/1225] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:12:41 2022] GET /compute/v2.1/servers/126f561c-163f-4f6d-8be1-98a2e29b0909 => generated 1313 bytes in 707 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:12:42.414874 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-52b06e17-3370-4db4-a59a-f89da0259739 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:42.416165 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-52b06e17-3370-4db4-a59a-f89da0259739 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] [instance: 6a391156-ffce-49c0-a79d-47b4544264f5] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:12:42.425476 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.network.neutron [None req-854fb16e-dcb8-4c80-a5bb-9985f9417d81 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] [instance: 4ae3e944-b450-40e5-bb71-c02df03212e3] Updating instance_info_cache with network_info: [{"id": "578b9284-5794-41e6-80d7-b5c49283c6f9", "address": "fa:16:3e:8e:8d:32", "network": {"id": "9b8d8e4f-14e5-4e94-bfa5-2387fb1adcf8", "bridge": null, "label": "tempest-FloatingIPsAssociationTestJSON-379677821-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.35.1.237", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "172.20.0.2"}}], "meta": {"injected": false, "tenant_id": "cda86413e74b4c1e8e09bd4df4926664", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false, "bound_drivers": {"0": "hyperv"}}, "devname": "tap578b9284-57", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=138681) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:117}} Jul 16 21:12:42.441888 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-52b06e17-3370-4db4-a59a-f89da0259739 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:42.442354 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-52b06e17-3370-4db4-a59a-f89da0259739 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:42.443230 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-52b06e17-3370-4db4-a59a-f89da0259739 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:42.444506 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-52b06e17-3370-4db4-a59a-f89da0259739 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:42.535755 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-854fb16e-dcb8-4c80-a5bb-9985f9417d81 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Releasing lock "refresh_cache-4ae3e944-b450-40e5-bb71-c02df03212e3" {{(pid=138681) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:312}} Jul 16 21:12:42.536637 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-854fb16e-dcb8-4c80-a5bb-9985f9417d81 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] 10.222.0.16 "POST /compute/v2.1/servers/4ae3e944-b450-40e5-bb71-c02df03212e3/action" status: 202 len: 0 microversion: 2.1 time: 7.001471 Jul 16 21:12:42.537254 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 577/1226] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 21:12:35 2022] POST /compute/v2.1/servers/4ae3e944-b450-40e5-bb71-c02df03212e3/action => generated 0 bytes in 7003 msecs (HTTP/1.1 202) 9 headers in 369 bytes (1 switches on core 0) Jul 16 21:12:42.546824 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-6c19d0aa-ff48-4690-ba50-bc064ab9243b tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:42.547855 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-6c19d0aa-ff48-4690-ba50-bc064ab9243b tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] [instance: b07f0d53-2cd4-440f-be47-077703aa66e6] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:12:42.556169 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6c19d0aa-ff48-4690-ba50-bc064ab9243b tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:42.556169 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6c19d0aa-ff48-4690-ba50-bc064ab9243b tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:42.556874 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6c19d0aa-ff48-4690-ba50-bc064ab9243b tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:42.557293 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6c19d0aa-ff48-4690-ba50-bc064ab9243b tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:42.594349 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-6c19d0aa-ff48-4690-ba50-bc064ab9243b tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] 10.222.0.16 "GET /compute/v2.1/servers/b07f0d53-2cd4-440f-be47-077703aa66e6/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.052164 Jul 16 21:12:42.595272 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 578/1227] 10.222.0.16 () {58 vars in 1301 bytes} [Sat Jul 16 21:12:42 2022] GET /compute/v2.1/servers/b07f0d53-2cd4-440f-be47-077703aa66e6/os-volume_attachments => generated 25 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Jul 16 21:12:42.612190 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-ac373d2f-3cd8-4a9c-868c-9e3b97bd36f1 tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:42.612945 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-ac373d2f-3cd8-4a9c-868c-9e3b97bd36f1 tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] [instance: 74a19091-819b-4cf0-8109-fbb0de65313c] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:12:42.627980 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ac373d2f-3cd8-4a9c-868c-9e3b97bd36f1 tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:42.629312 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ac373d2f-3cd8-4a9c-868c-9e3b97bd36f1 tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:42.630435 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ac373d2f-3cd8-4a9c-868c-9e3b97bd36f1 tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:42.632198 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ac373d2f-3cd8-4a9c-868c-9e3b97bd36f1 tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:42.652052 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-52b06e17-3370-4db4-a59a-f89da0259739 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a69f9dee03d43beaafa966216b9e289', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9b769bf8f3ea4e368bf68de1df6538f9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:42.665300 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-52b06e17-3370-4db4-a59a-f89da0259739 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] 10.222.0.16 "GET /compute/v2.1/servers/6a391156-ffce-49c0-a79d-47b4544264f5" status: 200 len: 1717 microversion: 2.1 time: 0.255652 Jul 16 21:12:42.665903 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 650/1228] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:12:42 2022] GET /compute/v2.1/servers/6a391156-ffce-49c0-a79d-47b4544264f5 => generated 1717 bytes in 257 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:12:42.683761 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-8842b3dd-a438-41a5-828d-c8b6b1389f87 req-693a3b61-26b7-4645-8d11-92e7127f7d29 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "4ae3e944-b450-40e5-bb71-c02df03212e3", "tag": "578b9284-5794-41e6-80d7-b5c49283c6f9"}, {"server_uuid": "6a391156-ffce-49c0-a79d-47b4544264f5", "name": "network-vif-unplugged", "status": "completed", "tag": "a8de1bba-f28a-49ad-92e5-8a10d67f1e75"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:12:42.693431 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-8842b3dd-a438-41a5-828d-c8b6b1389f87 req-693a3b61-26b7-4645-8d11-92e7127f7d29 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:42.694160 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-8842b3dd-a438-41a5-828d-c8b6b1389f87 req-693a3b61-26b7-4645-8d11-92e7127f7d29 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:42.736838 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-8842b3dd-a438-41a5-828d-c8b6b1389f87 req-693a3b61-26b7-4645-8d11-92e7127f7d29 service nova] Creating event network-changed:578b9284-5794-41e6-80d7-b5c49283c6f9 for instance 4ae3e944-b450-40e5-bb71-c02df03212e3 on nch2-master-1 Jul 16 21:12:42.737626 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-8842b3dd-a438-41a5-828d-c8b6b1389f87 req-693a3b61-26b7-4645-8d11-92e7127f7d29 service nova] Creating event network-vif-unplugged:a8de1bba-f28a-49ad-92e5-8a10d67f1e75 for instance 6a391156-ffce-49c0-a79d-47b4544264f5 on nch2-master-1 Jul 16 21:12:42.754040 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-8842b3dd-a438-41a5-828d-c8b6b1389f87 req-693a3b61-26b7-4645-8d11-92e7127f7d29 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 352 microversion: 2.1 time: 0.074879 Jul 16 21:12:42.757374 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 651/1229] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:12:42 2022] POST /compute/v2.1/os-server-external-events => generated 352 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:12:42.776541 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-7599a3e0-e84d-432d-989d-21bb0ad8fec0 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Action: 'action', calling method: .version_select at 0x7fdd9fd0f280>, body: {"removeFloatingIp": {"address": "172.35.1.237"}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:12:42.998353 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-ac373d2f-3cd8-4a9c-868c-9e3b97bd36f1 tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56d4bbcdebc24a34a814d7f6e72fbcfd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7d2b583c48254e17a8082f6467f2ae6d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:43.011777 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-ac373d2f-3cd8-4a9c-868c-9e3b97bd36f1 tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] 10.222.0.16 "GET /compute/v2.1/servers/74a19091-819b-4cf0-8109-fbb0de65313c" status: 200 len: 1320 microversion: 2.1 time: 0.414756 Jul 16 21:12:43.023254 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-7599a3e0-e84d-432d-989d-21bb0ad8fec0 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] [instance: 4ae3e944-b450-40e5-bb71-c02df03212e3] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:12:43.024737 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 579/1230] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:12:42 2022] GET /compute/v2.1/servers/74a19091-819b-4cf0-8109-fbb0de65313c => generated 1320 bytes in 428 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:12:43.031570 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-d8cb89b3-d340-444f-b1d7-320303c663b9 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:43.032201 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-d8cb89b3-d340-444f-b1d7-320303c663b9 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] [instance: 3c5ab09e-0a97-4cfc-b249-a11ab221b5d2] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:12:43.037255 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7599a3e0-e84d-432d-989d-21bb0ad8fec0 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:43.037747 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7599a3e0-e84d-432d-989d-21bb0ad8fec0 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:43.041550 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7599a3e0-e84d-432d-989d-21bb0ad8fec0 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:43.042288 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7599a3e0-e84d-432d-989d-21bb0ad8fec0 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:43.046702 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d8cb89b3-d340-444f-b1d7-320303c663b9 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:43.047228 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d8cb89b3-d340-444f-b1d7-320303c663b9 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:43.048047 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d8cb89b3-d340-444f-b1d7-320303c663b9 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:43.048542 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d8cb89b3-d340-444f-b1d7-320303c663b9 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:43.392256 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-7599a3e0-e84d-432d-989d-21bb0ad8fec0 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] HTTP exception thrown: Floating IP 172.35.1.237 is not associated with instance 8020f553-6025-4f19-835c-c00e2fb4f764. Jul 16 21:12:43.399538 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-d8cb89b3-d340-444f-b1d7-320303c663b9 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:43.400999 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-7599a3e0-e84d-432d-989d-21bb0ad8fec0 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Request method failure captured: Jul 16 21:12:43.400999 ncdv-master-1 devstack@n-api.service[138680]: request: POST /compute/v2.1/servers/8020f553-6025-4f19-835c-c00e2fb4f764/action HTTP/1.1 Jul 16 21:12:43.400999 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:12:43.400999 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:12:43.400999 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:12:43.400999 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 49 Jul 16 21:12:43.400999 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:12:43.400999 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:12:43.400999 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:12:43.400999 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:12:43.400999 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0ymKM_ECvjVkk6e7K1WKEy-tkQQ7EHRMB-Urg2_4RVouK1u8YOWROAshpDCUoNKJCmCo4JxDMnUkTw8ca8yVeRAtm7eAKieqo3ZQzYMt8n410aWYlQFenytzNBNAOHZnquy4qrFWM3_CaMN-nzEw34w9P4vt5Jr8OZ2O_l_ChXunM28 Jul 16 21:12:43.400999 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:12:43.400999 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:12:43.400999 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:12:43.400999 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:12:43.400999 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:12:43.400999 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:12:43.400999 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: cda86413e74b4c1e8e09bd4df4926664 Jul 16 21:12:43.400999 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-FloatingIPsAssociationTestJSON-1179742205 Jul 16 21:12:43.400999 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:12:43.400999 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:12:43.402613 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/cda86413e74b4c1e8e09bd4df4926664"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/cda86413e74b4c1e8e09bd4df4926664"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_cda86413e74b4c1e8e09bd4df4926664"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/cda86413e74b4c1e8e09bd4df4926664"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:12:43.402613 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-FloatingIPsAssociationTestJSON-1179742205 Jul 16 21:12:43.402613 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: cda86413e74b4c1e8e09bd4df4926664 Jul 16 21:12:43.402613 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-FloatingIPsAssociationTestJSON-1179742205 Jul 16 21:12:43.402613 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-FloatingIPsAssociationTestJSON-1179742205-project Jul 16 21:12:43.402613 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:12:43.402613 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:12:43.402613 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: 2dfcad8d2b57481bb33aa8f5d7483c51 Jul 16 21:12:43.402613 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-FloatingIPsAssociationTestJSON-1179742205-project Jul 16 21:12:43.402613 ncdv-master-1 devstack@n-api.service[138680]: Jul 16 21:12:43.402613 ncdv-master-1 devstack@n-api.service[138680]: {"removeFloatingIp": {"address": "172.35.1.237"}} Jul 16 21:12:43.402613 ncdv-master-1 devstack@n-api.service[138680]: method: Jul 16 21:12:43.402613 ncdv-master-1 devstack@n-api.service[138680]: exception: Floating IP 172.35.1.237 is not associated with instance 8020f553-6025-4f19-835c-c00e2fb4f764. Jul 16 21:12:43.403649 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': '8020f553-6025-4f19-835c-c00e2fb4f764', 'body': {'removeFloatingIp': {'address': '172.35.1.237'}}} Jul 16 21:12:43.403649 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:12:43.403649 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:12:43.403649 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:12:43.403649 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:12:43.403649 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:12:43.403649 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:12:43.403649 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Jul 16 21:12:43.403649 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Jul 16 21:12:43.403649 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:12:43.403649 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:12:43.403649 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 16 21:12:43.403649 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 16 21:12:43.403649 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/floating_ips.py", line 300, in _remove_floating_ip Jul 16 21:12:43.403649 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPConflict(explanation=msg) Jul 16 21:12:43.403649 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPConflict: Floating IP 172.35.1.237 is not associated with instance 8020f553-6025-4f19-835c-c00e2fb4f764. Jul 16 21:12:43.403649 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:12:43.403649 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:12:43.403649 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:12:43.403649 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:12:43.405149 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:12:43.405149 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:12:43.405149 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:12:43.405149 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:12:43.405149 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Floating IP 172.35.1.237 is not associated with instance 8020f553-6025-4f19-835c-c00e2fb4f764. Jul 16 21:12:43.405149 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:12:43.405621 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-7599a3e0-e84d-432d-989d-21bb0ad8fec0 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Returning 409 to user: Floating IP 172.35.1.237 is not associated with instance 8020f553-6025-4f19-835c-c00e2fb4f764. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:12:43.406904 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-7599a3e0-e84d-432d-989d-21bb0ad8fec0 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] 10.222.0.16 "POST /compute/v2.1/servers/8020f553-6025-4f19-835c-c00e2fb4f764/action" status: 409 len: 146 microversion: 2.1 time: 0.645619 Jul 16 21:12:43.407921 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 652/1231] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 21:12:42 2022] POST /compute/v2.1/servers/8020f553-6025-4f19-835c-c00e2fb4f764/action => generated 146 bytes in 650 msecs (HTTP/1.1 409) 9 headers in 378 bytes (1 switches on core 0) Jul 16 21:12:43.413457 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-efb26acc-e646-4d66-bad5-e0b773928ef0 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesAdminNegativeTest-server-2110617616", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452", "networks": [{"uuid": "9303ba55-8c09-4f8a-9781-a3f09b24a194"}]}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:12:43.417959 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-d8cb89b3-d340-444f-b1d7-320303c663b9 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:43.421737 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-d8cb89b3-d340-444f-b1d7-320303c663b9 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:43.426792 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-d8cb89b3-d340-444f-b1d7-320303c663b9 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] 10.222.0.16 "GET /compute/v2.1/servers/3c5ab09e-0a97-4cfc-b249-a11ab221b5d2" status: 200 len: 1408 microversion: 2.25 time: 0.398500 Jul 16 21:12:43.430809 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 580/1232] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:12:43 2022] GET /compute/v2.1/servers/3c5ab09e-0a97-4cfc-b249-a11ab221b5d2 => generated 1408 bytes in 405 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:12:43.435859 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-aa56f75b-2f7a-494c-8ba5-8a8d1333350b req-06d1c4cd-f28a-4544-8ffb-2018eb9ece16 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "74a19091-819b-4cf0-8109-fbb0de65313c", "name": "network-vif-plugged", "status": "completed", "tag": "7500284f-1b91-4c4b-bf7d-d5f21190a8ed"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:12:43.449042 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-aa56f75b-2f7a-494c-8ba5-8a8d1333350b req-06d1c4cd-f28a-4544-8ffb-2018eb9ece16 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:43.449675 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-aa56f75b-2f7a-494c-8ba5-8a8d1333350b req-06d1c4cd-f28a-4544-8ffb-2018eb9ece16 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:43.461843 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-aa56f75b-2f7a-494c-8ba5-8a8d1333350b req-06d1c4cd-f28a-4544-8ffb-2018eb9ece16 service nova] Creating event network-vif-plugged:7500284f-1b91-4c4b-bf7d-d5f21190a8ed for instance 74a19091-819b-4cf0-8109-fbb0de65313c on nch2-master-1 Jul 16 21:12:43.467651 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-aa56f75b-2f7a-494c-8ba5-8a8d1333350b req-06d1c4cd-f28a-4544-8ffb-2018eb9ece16 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.035241 Jul 16 21:12:43.470785 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 581/1233] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:12:43 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:12:43.474940 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-1aa3f075-27dd-4cb4-b2e7-16bf081da204 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Action: 'action', calling method: .version_select at 0x7fdd9fd8d280>, body: {"removeFloatingIp": {"address": "172.35.1.237"}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:12:43.509278 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.network.neutron [None req-efb26acc-e646-4d66-bad5-e0b773928ef0 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] validate_networks() for [('9303ba55-8c09-4f8a-9781-a3f09b24a194', None, None, None, None, None)] {{(pid=138680) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 16 21:12:43.692888 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-1aa3f075-27dd-4cb4-b2e7-16bf081da204 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] [instance: 4ae3e944-b450-40e5-bb71-c02df03212e3] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:12:43.700496 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1aa3f075-27dd-4cb4-b2e7-16bf081da204 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:43.700882 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1aa3f075-27dd-4cb4-b2e7-16bf081da204 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:43.701752 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1aa3f075-27dd-4cb4-b2e7-16bf081da204 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:43.702199 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1aa3f075-27dd-4cb4-b2e7-16bf081da204 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:43.718618 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=91,nova_cell1:UPDATE=4,nova_cell1:INSERT=1,nova_cell1:SAVEPOINT=2,nova_cell1:RELEASE=2 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:12:43.849708 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell0:SELECT=2 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:12:44.054584 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1aa3f075-27dd-4cb4-b2e7-16bf081da204 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Acquired lock "refresh_cache-4ae3e944-b450-40e5-bb71-c02df03212e3" {{(pid=138681) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:294}} Jul 16 21:12:44.285356 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-efb26acc-e646-4d66-bad5-e0b773928ef0 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Flavor limits 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 16 21:12:44.285784 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-efb26acc-e646-4d66-bad5-e0b773928ef0 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Image limits 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 16 21:12:44.286259 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-efb26acc-e646-4d66-bad5-e0b773928ef0 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Flavor pref 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 16 21:12:44.286669 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-efb26acc-e646-4d66-bad5-e0b773928ef0 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Image pref 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 16 21:12:44.287138 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-efb26acc-e646-4d66-bad5-e0b773928ef0 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 16 21:12:44.297557 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-efb26acc-e646-4d66-bad5-e0b773928ef0 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:44.298226 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-efb26acc-e646-4d66-bad5-e0b773928ef0 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:44.299084 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-efb26acc-e646-4d66-bad5-e0b773928ef0 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:44.299535 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-efb26acc-e646-4d66-bad5-e0b773928ef0 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:44.322910 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-efb26acc-e646-4d66-bad5-e0b773928ef0 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Getting quotas for project 30d028fa9b854c8fb71c8cbed2985c27. Resources: {'ram', 'instances', 'cores'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:12:44.331663 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-efb26acc-e646-4d66-bad5-e0b773928ef0 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Getting quotas for user 52281b3989a04ca2a50b0bc93bf783f0 and project 30d028fa9b854c8fb71c8cbed2985c27. Resources: {'ram', 'instances', 'cores'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:12:44.345828 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-efb26acc-e646-4d66-bad5-e0b773928ef0 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Going to run 1 instances... {{(pid=138680) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 16 21:12:44.348273 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-efb26acc-e646-4d66-bad5-e0b773928ef0 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] [instance: 08e7e1f9-8b5c-446c-a05f-d8043473dd35] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138680) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 16 21:12:44.478771 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-efb26acc-e646-4d66-bad5-e0b773928ef0 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 1.069509 Jul 16 21:12:44.484376 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 653/1234] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 21:12:43 2022] POST /compute/v2.1/servers => generated 380 bytes in 1076 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 16 21:12:44.489456 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-9d0c14bc-b990-454e-84e6-96563aab23b7 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:44.490069 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-9d0c14bc-b990-454e-84e6-96563aab23b7 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: 126f561c-163f-4f6d-8be1-98a2e29b0909] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:12:44.511955 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9d0c14bc-b990-454e-84e6-96563aab23b7 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:44.512737 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9d0c14bc-b990-454e-84e6-96563aab23b7 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:44.513532 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9d0c14bc-b990-454e-84e6-96563aab23b7 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:44.514129 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9d0c14bc-b990-454e-84e6-96563aab23b7 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:44.915300 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-9d0c14bc-b990-454e-84e6-96563aab23b7 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b68dacc4b8463ba07fae6372131117', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55a19b7a4682457a9132f5fc80110712', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:44.924156 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-9d0c14bc-b990-454e-84e6-96563aab23b7 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "GET /compute/v2.1/servers/126f561c-163f-4f6d-8be1-98a2e29b0909" status: 200 len: 1313 microversion: 2.1 time: 0.438412 Jul 16 21:12:44.934396 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 654/1235] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:12:44 2022] GET /compute/v2.1/servers/126f561c-163f-4f6d-8be1-98a2e29b0909 => generated 1313 bytes in 449 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:12:44.941138 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-0332df4d-c559-44fa-bdee-f075043f0c1e tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:44.941994 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-0332df4d-c559-44fa-bdee-f075043f0c1e tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] [instance: 6a391156-ffce-49c0-a79d-47b4544264f5] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:12:44.957268 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0332df4d-c559-44fa-bdee-f075043f0c1e tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:44.957969 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0332df4d-c559-44fa-bdee-f075043f0c1e tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:44.958749 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0332df4d-c559-44fa-bdee-f075043f0c1e tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:44.959413 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0332df4d-c559-44fa-bdee-f075043f0c1e tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:45.223381 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-0332df4d-c559-44fa-bdee-f075043f0c1e tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a69f9dee03d43beaafa966216b9e289', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9b769bf8f3ea4e368bf68de1df6538f9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:45.232321 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-0332df4d-c559-44fa-bdee-f075043f0c1e tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] 10.222.0.16 "GET /compute/v2.1/servers/6a391156-ffce-49c0-a79d-47b4544264f5" status: 200 len: 1717 microversion: 2.1 time: 0.296340 Jul 16 21:12:45.241912 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 655/1236] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:12:44 2022] GET /compute/v2.1/servers/6a391156-ffce-49c0-a79d-47b4544264f5 => generated 1717 bytes in 298 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:12:45.468084 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.network.neutron [None req-1aa3f075-27dd-4cb4-b2e7-16bf081da204 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] [instance: 4ae3e944-b450-40e5-bb71-c02df03212e3] Building network info cache for instance {{(pid=138681) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1990}} Jul 16 21:12:45.564070 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-a5e13eba-0afe-4391-99f7-0ee7a4e991c2 tempest-TenantUsagesTestJSON-260059616 tempest-TenantUsagesTestJSON-260059616-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TenantUsagesTestJSON-server-1182151361", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452"}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:12:45.641583 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.network.neutron [None req-a5e13eba-0afe-4391-99f7-0ee7a4e991c2 tempest-TenantUsagesTestJSON-260059616 tempest-TenantUsagesTestJSON-260059616-project] validate_networks() for None {{(pid=138680) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 16 21:12:46.037206 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-a5e13eba-0afe-4391-99f7-0ee7a4e991c2 tempest-TenantUsagesTestJSON-260059616 tempest-TenantUsagesTestJSON-260059616-project] Flavor limits 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 16 21:12:46.037636 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-a5e13eba-0afe-4391-99f7-0ee7a4e991c2 tempest-TenantUsagesTestJSON-260059616 tempest-TenantUsagesTestJSON-260059616-project] Image limits 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 16 21:12:46.038113 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-a5e13eba-0afe-4391-99f7-0ee7a4e991c2 tempest-TenantUsagesTestJSON-260059616 tempest-TenantUsagesTestJSON-260059616-project] Flavor pref 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 16 21:12:46.038527 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-a5e13eba-0afe-4391-99f7-0ee7a4e991c2 tempest-TenantUsagesTestJSON-260059616 tempest-TenantUsagesTestJSON-260059616-project] Image pref 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 16 21:12:46.038940 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-a5e13eba-0afe-4391-99f7-0ee7a4e991c2 tempest-TenantUsagesTestJSON-260059616 tempest-TenantUsagesTestJSON-260059616-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 16 21:12:46.054348 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a5e13eba-0afe-4391-99f7-0ee7a4e991c2 tempest-TenantUsagesTestJSON-260059616 tempest-TenantUsagesTestJSON-260059616-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:46.055096 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a5e13eba-0afe-4391-99f7-0ee7a4e991c2 tempest-TenantUsagesTestJSON-260059616 tempest-TenantUsagesTestJSON-260059616-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:46.056541 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a5e13eba-0afe-4391-99f7-0ee7a4e991c2 tempest-TenantUsagesTestJSON-260059616 tempest-TenantUsagesTestJSON-260059616-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:46.057136 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a5e13eba-0afe-4391-99f7-0ee7a4e991c2 tempest-TenantUsagesTestJSON-260059616 tempest-TenantUsagesTestJSON-260059616-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:46.082442 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-a5e13eba-0afe-4391-99f7-0ee7a4e991c2 tempest-TenantUsagesTestJSON-260059616 tempest-TenantUsagesTestJSON-260059616-project] Getting quotas for project cad2f07400d54ef9825022d5b3489db0. Resources: {'ram', 'instances', 'cores'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:12:46.088969 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-a5e13eba-0afe-4391-99f7-0ee7a4e991c2 tempest-TenantUsagesTestJSON-260059616 tempest-TenantUsagesTestJSON-260059616-project] Getting quotas for user 2db871d6aa9746d2b660efbfbdb1dc75 and project cad2f07400d54ef9825022d5b3489db0. Resources: {'ram', 'instances', 'cores'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:12:46.103929 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-a5e13eba-0afe-4391-99f7-0ee7a4e991c2 tempest-TenantUsagesTestJSON-260059616 tempest-TenantUsagesTestJSON-260059616-project] Going to run 1 instances... {{(pid=138680) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 16 21:12:46.107162 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-a5e13eba-0afe-4391-99f7-0ee7a4e991c2 tempest-TenantUsagesTestJSON-260059616 tempest-TenantUsagesTestJSON-260059616-project] [instance: 6e68dbd6-1d5c-471e-ad69-cf1c4aa4004a] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138680) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 16 21:12:46.214312 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.network.neutron [None req-1aa3f075-27dd-4cb4-b2e7-16bf081da204 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] [instance: 4ae3e944-b450-40e5-bb71-c02df03212e3] Updating instance_info_cache with network_info: [{"id": "578b9284-5794-41e6-80d7-b5c49283c6f9", "address": "fa:16:3e:8e:8d:32", "network": {"id": "9b8d8e4f-14e5-4e94-bfa5-2387fb1adcf8", "bridge": null, "label": "tempest-FloatingIPsAssociationTestJSON-379677821-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "172.20.0.2"}}], "meta": {"injected": false, "tenant_id": "cda86413e74b4c1e8e09bd4df4926664", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false, "bound_drivers": {"0": "hyperv"}}, "devname": "tap578b9284-57", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=138681) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:117}} Jul 16 21:12:46.228430 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-a5e13eba-0afe-4391-99f7-0ee7a4e991c2 tempest-TenantUsagesTestJSON-260059616 tempest-TenantUsagesTestJSON-260059616-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.994483 Jul 16 21:12:46.229427 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 656/1237] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 21:12:45 2022] POST /compute/v2.1/servers => generated 380 bytes in 996 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 16 21:12:46.233762 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-223441f6-8267-4dcf-9000-3f041b843a6b tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:46.234599 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-223441f6-8267-4dcf-9000-3f041b843a6b tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] [instance: 74a19091-819b-4cf0-8109-fbb0de65313c] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:12:46.241073 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-223441f6-8267-4dcf-9000-3f041b843a6b tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:46.241524 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-223441f6-8267-4dcf-9000-3f041b843a6b tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:46.242356 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-223441f6-8267-4dcf-9000-3f041b843a6b tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:46.243108 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-223441f6-8267-4dcf-9000-3f041b843a6b tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:46.302357 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1aa3f075-27dd-4cb4-b2e7-16bf081da204 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Releasing lock "refresh_cache-4ae3e944-b450-40e5-bb71-c02df03212e3" {{(pid=138681) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:312}} Jul 16 21:12:46.303326 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-1aa3f075-27dd-4cb4-b2e7-16bf081da204 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] 10.222.0.16 "POST /compute/v2.1/servers/4ae3e944-b450-40e5-bb71-c02df03212e3/action" status: 202 len: 0 microversion: 2.1 time: 2.833731 Jul 16 21:12:46.304455 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 582/1238] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 21:12:43 2022] POST /compute/v2.1/servers/4ae3e944-b450-40e5-bb71-c02df03212e3/action => generated 0 bytes in 2835 msecs (HTTP/1.1 202) 9 headers in 369 bytes (1 switches on core 0) Jul 16 21:12:46.312053 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-888fc76f-563c-4048-bf3c-d2ed3af70e47 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:46.312677 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-888fc76f-563c-4048-bf3c-d2ed3af70e47 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] [instance: 3c5ab09e-0a97-4cfc-b249-a11ab221b5d2] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:12:46.325485 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-888fc76f-563c-4048-bf3c-d2ed3af70e47 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:46.326182 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-888fc76f-563c-4048-bf3c-d2ed3af70e47 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:46.327337 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-888fc76f-563c-4048-bf3c-d2ed3af70e47 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:46.327946 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-888fc76f-563c-4048-bf3c-d2ed3af70e47 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:46.464681 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-223441f6-8267-4dcf-9000-3f041b843a6b tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56d4bbcdebc24a34a814d7f6e72fbcfd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7d2b583c48254e17a8082f6467f2ae6d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:46.473374 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-223441f6-8267-4dcf-9000-3f041b843a6b tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] 10.222.0.16 "GET /compute/v2.1/servers/74a19091-819b-4cf0-8109-fbb0de65313c" status: 200 len: 1470 microversion: 2.1 time: 0.243127 Jul 16 21:12:46.474097 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 657/1239] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:12:46 2022] GET /compute/v2.1/servers/74a19091-819b-4cf0-8109-fbb0de65313c => generated 1470 bytes in 244 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:12:46.479284 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-f7e9d128-666b-4295-9edf-8db5679fa813 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:46.480295 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-f7e9d128-666b-4295-9edf-8db5679fa813 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] [instance: 08e7e1f9-8b5c-446c-a05f-d8043473dd35] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:12:46.486964 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_api:SELECT=94,nova_api:INSERT=6 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:12:46.491865 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f7e9d128-666b-4295-9edf-8db5679fa813 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:46.493118 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f7e9d128-666b-4295-9edf-8db5679fa813 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:46.494006 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f7e9d128-666b-4295-9edf-8db5679fa813 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:46.494536 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f7e9d128-666b-4295-9edf-8db5679fa813 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:46.522586 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-888fc76f-563c-4048-bf3c-d2ed3af70e47 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:46.536074 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-888fc76f-563c-4048-bf3c-d2ed3af70e47 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:46.537737 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-888fc76f-563c-4048-bf3c-d2ed3af70e47 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:46.544452 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-888fc76f-563c-4048-bf3c-d2ed3af70e47 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] 10.222.0.16 "GET /compute/v2.1/servers/3c5ab09e-0a97-4cfc-b249-a11ab221b5d2" status: 200 len: 1408 microversion: 2.25 time: 0.238751 Jul 16 21:12:46.545499 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 583/1240] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:12:46 2022] GET /compute/v2.1/servers/3c5ab09e-0a97-4cfc-b249-a11ab221b5d2 => generated 1408 bytes in 240 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:12:46.562258 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-5a955aa8-9ea1-414c-b29f-bd21e88f047f req-4ae4966c-1f39-4086-9aca-621ba5f6315d service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "6a391156-ffce-49c0-a79d-47b4544264f5", "tag": "a8de1bba-f28a-49ad-92e5-8a10d67f1e75"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:12:46.573421 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-5a955aa8-9ea1-414c-b29f-bd21e88f047f req-4ae4966c-1f39-4086-9aca-621ba5f6315d service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:46.573965 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-5a955aa8-9ea1-414c-b29f-bd21e88f047f req-4ae4966c-1f39-4086-9aca-621ba5f6315d service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:46.589680 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-f7e9d128-666b-4295-9edf-8db5679fa813 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '52281b3989a04ca2a50b0bc93bf783f0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '30d028fa9b854c8fb71c8cbed2985c27', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:46.599397 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-f7e9d128-666b-4295-9edf-8db5679fa813 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] 10.222.0.16 "GET /compute/v2.1/servers/08e7e1f9-8b5c-446c-a05f-d8043473dd35" status: 200 len: 1270 microversion: 2.1 time: 0.123759 Jul 16 21:12:46.600102 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 658/1241] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:12:46 2022] GET /compute/v2.1/servers/08e7e1f9-8b5c-446c-a05f-d8043473dd35 => generated 1270 bytes in 125 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:12:46.606421 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-5a955aa8-9ea1-414c-b29f-bd21e88f047f req-4ae4966c-1f39-4086-9aca-621ba5f6315d service nova] Creating event network-changed:a8de1bba-f28a-49ad-92e5-8a10d67f1e75 for instance 6a391156-ffce-49c0-a79d-47b4544264f5 on nch2-master-1 Jul 16 21:12:46.607872 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-9f9c6e19-21bb-43a1-a31a-4b24004daffd req-2b921351-c703-403a-96e6-59b2584c95cb service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "126f561c-163f-4f6d-8be1-98a2e29b0909", "name": "network-vif-plugged", "status": "completed", "tag": "3b739133-7af5-4df3-a04d-fff7e0a81a3d"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:12:46.618747 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-9f9c6e19-21bb-43a1-a31a-4b24004daffd req-2b921351-c703-403a-96e6-59b2584c95cb service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:46.620012 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-9f9c6e19-21bb-43a1-a31a-4b24004daffd req-2b921351-c703-403a-96e6-59b2584c95cb service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:46.632614 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-5a955aa8-9ea1-414c-b29f-bd21e88f047f req-4ae4966c-1f39-4086-9aca-621ba5f6315d service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.083300 Jul 16 21:12:46.633323 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 584/1242] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:12:46 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:12:46.637355 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-9f9c6e19-21bb-43a1-a31a-4b24004daffd req-2b921351-c703-403a-96e6-59b2584c95cb service nova] Creating event network-vif-plugged:3b739133-7af5-4df3-a04d-fff7e0a81a3d for instance 126f561c-163f-4f6d-8be1-98a2e29b0909 on nch2-master-1 Jul 16 21:12:46.642924 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-9f9c6e19-21bb-43a1-a31a-4b24004daffd req-2b921351-c703-403a-96e6-59b2584c95cb service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.041360 Jul 16 21:12:46.643727 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 659/1243] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:12:46 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:12:46.650253 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-765434af-fc59-4381-8525-cb71058da07a tempest-TenantUsagesTestJSON-260059616 tempest-TenantUsagesTestJSON-260059616-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:46.651328 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-765434af-fc59-4381-8525-cb71058da07a tempest-TenantUsagesTestJSON-260059616 tempest-TenantUsagesTestJSON-260059616-project] [instance: 6e68dbd6-1d5c-471e-ad69-cf1c4aa4004a] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:12:46.655387 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-bf427b15-c853-4fac-83e2-cb16d9cf9e68 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:46.655387 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-bf427b15-c853-4fac-83e2-cb16d9cf9e68 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: 126f561c-163f-4f6d-8be1-98a2e29b0909] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:12:46.663619 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-bf427b15-c853-4fac-83e2-cb16d9cf9e68 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:46.672711 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-bf427b15-c853-4fac-83e2-cb16d9cf9e68 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:46.673793 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-bf427b15-c853-4fac-83e2-cb16d9cf9e68 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:46.674459 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-bf427b15-c853-4fac-83e2-cb16d9cf9e68 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:46.780183 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-765434af-fc59-4381-8525-cb71058da07a tempest-TenantUsagesTestJSON-260059616 tempest-TenantUsagesTestJSON-260059616-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2db871d6aa9746d2b660efbfbdb1dc75', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cad2f07400d54ef9825022d5b3489db0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:46.792997 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-765434af-fc59-4381-8525-cb71058da07a tempest-TenantUsagesTestJSON-260059616 tempest-TenantUsagesTestJSON-260059616-project] 10.222.0.16 "GET /compute/v2.1/servers/6e68dbd6-1d5c-471e-ad69-cf1c4aa4004a" status: 200 len: 1214 microversion: 2.1 time: 0.145628 Jul 16 21:12:46.792997 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 660/1244] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:12:46 2022] GET /compute/v2.1/servers/6e68dbd6-1d5c-471e-ad69-cf1c4aa4004a => generated 1214 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:12:46.801393 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-8480022c-5dd0-4d37-942a-1918dd37dad3 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:46.803050 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-8480022c-5dd0-4d37-942a-1918dd37dad3 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] [instance: 6a391156-ffce-49c0-a79d-47b4544264f5] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:12:46.813217 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8480022c-5dd0-4d37-942a-1918dd37dad3 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:46.814419 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8480022c-5dd0-4d37-942a-1918dd37dad3 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:46.815613 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8480022c-5dd0-4d37-942a-1918dd37dad3 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:46.819225 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8480022c-5dd0-4d37-942a-1918dd37dad3 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:47.007837 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-bf427b15-c853-4fac-83e2-cb16d9cf9e68 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b68dacc4b8463ba07fae6372131117', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55a19b7a4682457a9132f5fc80110712', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:47.025142 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-bf427b15-c853-4fac-83e2-cb16d9cf9e68 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "GET /compute/v2.1/servers/126f561c-163f-4f6d-8be1-98a2e29b0909" status: 200 len: 1313 microversion: 2.1 time: 0.388042 Jul 16 21:12:47.032051 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 585/1245] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:12:46 2022] GET /compute/v2.1/servers/126f561c-163f-4f6d-8be1-98a2e29b0909 => generated 1313 bytes in 393 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:12:47.037279 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-3785911e-a2af-4ca7-a3d2-9fa3cae515a5 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:47.045705 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-3785911e-a2af-4ca7-a3d2-9fa3cae515a5 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] [instance: 4ae3e944-b450-40e5-bb71-c02df03212e3] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:12:47.052470 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3785911e-a2af-4ca7-a3d2-9fa3cae515a5 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:47.054450 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3785911e-a2af-4ca7-a3d2-9fa3cae515a5 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:47.058930 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3785911e-a2af-4ca7-a3d2-9fa3cae515a5 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:47.059522 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3785911e-a2af-4ca7-a3d2-9fa3cae515a5 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:47.097674 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-3785911e-a2af-4ca7-a3d2-9fa3cae515a5 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] [instance: 4ae3e944-b450-40e5-bb71-c02df03212e3] Going to try to terminate instance {{(pid=138681) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 16 21:12:47.457859 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-3785911e-a2af-4ca7-a3d2-9fa3cae515a5 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] 10.222.0.16 "DELETE /compute/v2.1/servers/4ae3e944-b450-40e5-bb71-c02df03212e3" status: 204 len: 0 microversion: 2.1 time: 0.424076 Jul 16 21:12:47.459453 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 586/1246] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:12:47 2022] DELETE /compute/v2.1/servers/4ae3e944-b450-40e5-bb71-c02df03212e3 => generated 0 bytes in 426 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 16 21:12:47.484616 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-82dbb688-2ae2-4257-9199-91ba546f14ca tempest-DeleteServersAdminTestJSON-2093208747 tempest-DeleteServersAdminTestJSON-2093208747-project-admin] Action: 'action', calling method: >, body: {"os-resetState": {"state": "error"}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:12:47.484616 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-82dbb688-2ae2-4257-9199-91ba546f14ca tempest-DeleteServersAdminTestJSON-2093208747 tempest-DeleteServersAdminTestJSON-2093208747-project-admin] [instance: 74a19091-819b-4cf0-8109-fbb0de65313c] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:12:47.484616 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-82dbb688-2ae2-4257-9199-91ba546f14ca tempest-DeleteServersAdminTestJSON-2093208747 tempest-DeleteServersAdminTestJSON-2093208747-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:47.484616 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-82dbb688-2ae2-4257-9199-91ba546f14ca tempest-DeleteServersAdminTestJSON-2093208747 tempest-DeleteServersAdminTestJSON-2093208747-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:47.485270 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-82dbb688-2ae2-4257-9199-91ba546f14ca tempest-DeleteServersAdminTestJSON-2093208747 tempest-DeleteServersAdminTestJSON-2093208747-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:47.485869 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-82dbb688-2ae2-4257-9199-91ba546f14ca tempest-DeleteServersAdminTestJSON-2093208747 tempest-DeleteServersAdminTestJSON-2093208747-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:47.558190 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-8480022c-5dd0-4d37-942a-1918dd37dad3 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a69f9dee03d43beaafa966216b9e289', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9b769bf8f3ea4e368bf68de1df6538f9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:47.566303 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-8480022c-5dd0-4d37-942a-1918dd37dad3 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] 10.222.0.16 "GET /compute/v2.1/servers/6a391156-ffce-49c0-a79d-47b4544264f5" status: 200 len: 1717 microversion: 2.1 time: 0.773383 Jul 16 21:12:47.576872 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 661/1247] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:12:46 2022] GET /compute/v2.1/servers/6a391156-ffce-49c0-a79d-47b4544264f5 => generated 1717 bytes in 775 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:12:47.576872 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-511b02bb-912c-4f2d-a9ab-11f04df96fa3 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Action: 'action', calling method: , body: {"addFloatingIp": {"address": "172.35.1.237"}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:12:47.578062 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-511b02bb-912c-4f2d-a9ab-11f04df96fa3 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] [instance: 8020f553-6025-4f19-835c-c00e2fb4f764] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:12:47.585333 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-511b02bb-912c-4f2d-a9ab-11f04df96fa3 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:47.585784 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-511b02bb-912c-4f2d-a9ab-11f04df96fa3 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:47.591441 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-511b02bb-912c-4f2d-a9ab-11f04df96fa3 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:47.592121 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-511b02bb-912c-4f2d-a9ab-11f04df96fa3 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:47.621672 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-511b02bb-912c-4f2d-a9ab-11f04df96fa3 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Acquired lock "refresh_cache-8020f553-6025-4f19-835c-c00e2fb4f764" {{(pid=138680) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:294}} Jul 16 21:12:47.902811 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-82dbb688-2ae2-4257-9199-91ba546f14ca tempest-DeleteServersAdminTestJSON-2093208747 tempest-DeleteServersAdminTestJSON-2093208747-project-admin] 10.222.0.16 "POST /compute/v2.1/servers/74a19091-819b-4cf0-8109-fbb0de65313c/action" status: 202 len: 0 microversion: 2.1 time: 0.441775 Jul 16 21:12:47.903660 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 587/1248] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 21:12:47 2022] POST /compute/v2.1/servers/74a19091-819b-4cf0-8109-fbb0de65313c/action => generated 0 bytes in 443 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 21:12:47.908996 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-743a1bd3-7fd3-4262-92cb-3a3fc1747649 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:47.910522 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-743a1bd3-7fd3-4262-92cb-3a3fc1747649 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] [instance: 3c5ab09e-0a97-4cfc-b249-a11ab221b5d2] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:12:47.917457 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-743a1bd3-7fd3-4262-92cb-3a3fc1747649 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:47.917918 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-743a1bd3-7fd3-4262-92cb-3a3fc1747649 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:47.918920 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-743a1bd3-7fd3-4262-92cb-3a3fc1747649 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:47.919391 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-743a1bd3-7fd3-4262-92cb-3a3fc1747649 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:48.249474 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-743a1bd3-7fd3-4262-92cb-3a3fc1747649 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:48.258705 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-743a1bd3-7fd3-4262-92cb-3a3fc1747649 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:48.260445 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-743a1bd3-7fd3-4262-92cb-3a3fc1747649 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:48.261586 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-743a1bd3-7fd3-4262-92cb-3a3fc1747649 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] 10.222.0.16 "GET /compute/v2.1/servers/3c5ab09e-0a97-4cfc-b249-a11ab221b5d2" status: 200 len: 1408 microversion: 2.25 time: 0.356716 Jul 16 21:12:48.262151 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 588/1249] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:12:47 2022] GET /compute/v2.1/servers/3c5ab09e-0a97-4cfc-b249-a11ab221b5d2 => generated 1408 bytes in 358 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:12:48.267149 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-fc5d8cfa-b2e7-4bca-932d-3b8b43386700 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:48.267766 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-fc5d8cfa-b2e7-4bca-932d-3b8b43386700 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] [instance: 08e7e1f9-8b5c-446c-a05f-d8043473dd35] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:12:48.274171 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fc5d8cfa-b2e7-4bca-932d-3b8b43386700 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:48.274724 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fc5d8cfa-b2e7-4bca-932d-3b8b43386700 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:48.275536 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fc5d8cfa-b2e7-4bca-932d-3b8b43386700 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:48.276001 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fc5d8cfa-b2e7-4bca-932d-3b8b43386700 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:48.402359 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-fc5d8cfa-b2e7-4bca-932d-3b8b43386700 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '52281b3989a04ca2a50b0bc93bf783f0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '30d028fa9b854c8fb71c8cbed2985c27', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:48.410359 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-fc5d8cfa-b2e7-4bca-932d-3b8b43386700 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] 10.222.0.16 "GET /compute/v2.1/servers/08e7e1f9-8b5c-446c-a05f-d8043473dd35" status: 200 len: 1276 microversion: 2.1 time: 0.147164 Jul 16 21:12:48.410948 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 589/1250] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:12:48 2022] GET /compute/v2.1/servers/08e7e1f9-8b5c-446c-a05f-d8043473dd35 => generated 1276 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:12:48.416230 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-3e082e37-17e7-4631-9be6-c2046b80b3d4 tempest-TenantUsagesTestJSON-260059616 tempest-TenantUsagesTestJSON-260059616-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:48.416838 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-3e082e37-17e7-4631-9be6-c2046b80b3d4 tempest-TenantUsagesTestJSON-260059616 tempest-TenantUsagesTestJSON-260059616-project] [instance: 6e68dbd6-1d5c-471e-ad69-cf1c4aa4004a] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:12:48.422962 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3e082e37-17e7-4631-9be6-c2046b80b3d4 tempest-TenantUsagesTestJSON-260059616 tempest-TenantUsagesTestJSON-260059616-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:48.423443 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3e082e37-17e7-4631-9be6-c2046b80b3d4 tempest-TenantUsagesTestJSON-260059616 tempest-TenantUsagesTestJSON-260059616-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:48.424219 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3e082e37-17e7-4631-9be6-c2046b80b3d4 tempest-TenantUsagesTestJSON-260059616 tempest-TenantUsagesTestJSON-260059616-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:48.424651 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3e082e37-17e7-4631-9be6-c2046b80b3d4 tempest-TenantUsagesTestJSON-260059616 tempest-TenantUsagesTestJSON-260059616-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:48.529878 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-3e082e37-17e7-4631-9be6-c2046b80b3d4 tempest-TenantUsagesTestJSON-260059616 tempest-TenantUsagesTestJSON-260059616-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2db871d6aa9746d2b660efbfbdb1dc75', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cad2f07400d54ef9825022d5b3489db0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:48.537962 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-3e082e37-17e7-4631-9be6-c2046b80b3d4 tempest-TenantUsagesTestJSON-260059616 tempest-TenantUsagesTestJSON-260059616-project] 10.222.0.16 "GET /compute/v2.1/servers/6e68dbd6-1d5c-471e-ad69-cf1c4aa4004a" status: 200 len: 1218 microversion: 2.1 time: 0.125966 Jul 16 21:12:48.538568 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 590/1251] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:12:48 2022] GET /compute/v2.1/servers/6e68dbd6-1d5c-471e-ad69-cf1c4aa4004a => generated 1218 bytes in 127 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:12:48.548380 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-706cc0e9-ec21-4c76-9655-1d3b79ca47ba tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:48.548380 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-706cc0e9-ec21-4c76-9655-1d3b79ca47ba tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] [instance: 74a19091-819b-4cf0-8109-fbb0de65313c] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:12:48.550654 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-706cc0e9-ec21-4c76-9655-1d3b79ca47ba tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:48.551142 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-706cc0e9-ec21-4c76-9655-1d3b79ca47ba tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:48.551949 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-706cc0e9-ec21-4c76-9655-1d3b79ca47ba tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:48.552386 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-706cc0e9-ec21-4c76-9655-1d3b79ca47ba tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:48.586005 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-706cc0e9-ec21-4c76-9655-1d3b79ca47ba tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:48.586574 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-706cc0e9-ec21-4c76-9655-1d3b79ca47ba tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:48.587171 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.objects.instance [None req-706cc0e9-ec21-4c76-9655-1d3b79ca47ba tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Lazy-loading 'fault' on Instance uuid 74a19091-819b-4cf0-8109-fbb0de65313c {{(pid=138681) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:12:48.841272 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-706cc0e9-ec21-4c76-9655-1d3b79ca47ba tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56d4bbcdebc24a34a814d7f6e72fbcfd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7d2b583c48254e17a8082f6467f2ae6d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:48.853337 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-706cc0e9-ec21-4c76-9655-1d3b79ca47ba tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] 10.222.0.16 "GET /compute/v2.1/servers/74a19091-819b-4cf0-8109-fbb0de65313c" status: 200 len: 1453 microversion: 2.1 time: 0.311557 Jul 16 21:12:48.853337 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 591/1252] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:12:48 2022] GET /compute/v2.1/servers/74a19091-819b-4cf0-8109-fbb0de65313c => generated 1453 bytes in 313 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:12:48.858315 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-a7ffc61c-e5ad-4a70-95fc-165642fc4970 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:48.858924 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-a7ffc61c-e5ad-4a70-95fc-165642fc4970 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: 126f561c-163f-4f6d-8be1-98a2e29b0909] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:12:48.876585 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a7ffc61c-e5ad-4a70-95fc-165642fc4970 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:48.876585 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a7ffc61c-e5ad-4a70-95fc-165642fc4970 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:48.876585 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a7ffc61c-e5ad-4a70-95fc-165642fc4970 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:48.876585 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a7ffc61c-e5ad-4a70-95fc-165642fc4970 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:49.215945 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-a7ffc61c-e5ad-4a70-95fc-165642fc4970 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b68dacc4b8463ba07fae6372131117', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55a19b7a4682457a9132f5fc80110712', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:49.224397 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-a7ffc61c-e5ad-4a70-95fc-165642fc4970 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "GET /compute/v2.1/servers/126f561c-163f-4f6d-8be1-98a2e29b0909" status: 200 len: 1313 microversion: 2.1 time: 0.369862 Jul 16 21:12:49.234268 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 592/1253] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:12:48 2022] GET /compute/v2.1/servers/126f561c-163f-4f6d-8be1-98a2e29b0909 => generated 1313 bytes in 372 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:12:49.234268 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-031f9b4a-7f22-49d3-9ed4-25641f1ba688 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:49.234268 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-031f9b4a-7f22-49d3-9ed4-25641f1ba688 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] [instance: 6a391156-ffce-49c0-a79d-47b4544264f5] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:12:49.237061 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-031f9b4a-7f22-49d3-9ed4-25641f1ba688 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:49.237516 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-031f9b4a-7f22-49d3-9ed4-25641f1ba688 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:49.238298 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-031f9b4a-7f22-49d3-9ed4-25641f1ba688 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:49.238734 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-031f9b4a-7f22-49d3-9ed4-25641f1ba688 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:49.579094 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-031f9b4a-7f22-49d3-9ed4-25641f1ba688 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a69f9dee03d43beaafa966216b9e289', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9b769bf8f3ea4e368bf68de1df6538f9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:49.588190 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-031f9b4a-7f22-49d3-9ed4-25641f1ba688 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] 10.222.0.16 "GET /compute/v2.1/servers/6a391156-ffce-49c0-a79d-47b4544264f5" status: 200 len: 1717 microversion: 2.1 time: 0.362039 Jul 16 21:12:49.588858 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 593/1254] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:12:49 2022] GET /compute/v2.1/servers/6a391156-ffce-49c0-a79d-47b4544264f5 => generated 1717 bytes in 363 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:12:49.593375 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-ff9d18bc-cbe6-4cb5-a8a0-dd6b588a7a96 req-a370c8c9-af4b-4d62-9971-082bc94d3503 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "4ae3e944-b450-40e5-bb71-c02df03212e3", "tag": "578b9284-5794-41e6-80d7-b5c49283c6f9"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:12:49.602566 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-ff9d18bc-cbe6-4cb5-a8a0-dd6b588a7a96 req-a370c8c9-af4b-4d62-9971-082bc94d3503 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:49.603462 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-ff9d18bc-cbe6-4cb5-a8a0-dd6b588a7a96 req-a370c8c9-af4b-4d62-9971-082bc94d3503 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:49.615714 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-ff9d18bc-cbe6-4cb5-a8a0-dd6b588a7a96 req-a370c8c9-af4b-4d62-9971-082bc94d3503 service nova] Creating event network-changed:578b9284-5794-41e6-80d7-b5c49283c6f9 for instance 4ae3e944-b450-40e5-bb71-c02df03212e3 on nch2-master-1 Jul 16 21:12:49.621446 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-ff9d18bc-cbe6-4cb5-a8a0-dd6b588a7a96 req-a370c8c9-af4b-4d62-9971-082bc94d3503 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.031267 Jul 16 21:12:49.633220 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 594/1255] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:12:49 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:12:49.644210 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-606872af-febe-4f15-984e-73beec717b9a req-ccd0d44d-e3e3-4533-a0ee-f82b00da63f4 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "3c5ab09e-0a97-4cfc-b249-a11ab221b5d2", "name": "network-vif-plugged", "status": "completed", "tag": "4f9dfe87-5bed-49a6-8b89-464c4930e746"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:12:49.657756 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-606872af-febe-4f15-984e-73beec717b9a req-ccd0d44d-e3e3-4533-a0ee-f82b00da63f4 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:49.659529 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-606872af-febe-4f15-984e-73beec717b9a req-ccd0d44d-e3e3-4533-a0ee-f82b00da63f4 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:49.678548 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-606872af-febe-4f15-984e-73beec717b9a req-ccd0d44d-e3e3-4533-a0ee-f82b00da63f4 service nova] Creating event network-vif-plugged:4f9dfe87-5bed-49a6-8b89-464c4930e746 for instance 3c5ab09e-0a97-4cfc-b249-a11ab221b5d2 on nch2-master-1 Jul 16 21:12:49.686613 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-606872af-febe-4f15-984e-73beec717b9a req-ccd0d44d-e3e3-4533-a0ee-f82b00da63f4 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.049724 Jul 16 21:12:49.687441 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 595/1256] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:12:49 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:12:49.695629 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-1ebbe6ac-364b-40de-b8e9-bd73af54e8fa tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:49.696415 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-1ebbe6ac-364b-40de-b8e9-bd73af54e8fa tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] [instance: 74a19091-819b-4cf0-8109-fbb0de65313c] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:12:49.725276 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1ebbe6ac-364b-40de-b8e9-bd73af54e8fa tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:49.725741 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1ebbe6ac-364b-40de-b8e9-bd73af54e8fa tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:49.726732 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1ebbe6ac-364b-40de-b8e9-bd73af54e8fa tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:49.727812 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1ebbe6ac-364b-40de-b8e9-bd73af54e8fa tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:49.770603 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.network.neutron [None req-511b02bb-912c-4f2d-a9ab-11f04df96fa3 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] [instance: 8020f553-6025-4f19-835c-c00e2fb4f764] Building network info cache for instance {{(pid=138680) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1990}} Jul 16 21:12:49.788177 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-1ebbe6ac-364b-40de-b8e9-bd73af54e8fa tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] [instance: 74a19091-819b-4cf0-8109-fbb0de65313c] Going to try to terminate instance {{(pid=138681) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 16 21:12:49.800530 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=92,nova_cell1:SAVEPOINT=1,nova_cell1:UPDATE=4,nova_cell1:RELEASE=1,nova_cell1:INSERT=2 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:12:50.265251 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-1ebbe6ac-364b-40de-b8e9-bd73af54e8fa tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] 10.222.0.16 "DELETE /compute/v2.1/servers/74a19091-819b-4cf0-8109-fbb0de65313c" status: 204 len: 0 microversion: 2.1 time: 0.575716 Jul 16 21:12:50.265883 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 596/1257] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:12:49 2022] DELETE /compute/v2.1/servers/74a19091-819b-4cf0-8109-fbb0de65313c => generated 0 bytes in 578 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 16 21:12:50.272367 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-1273256b-9c33-4511-b1a5-0c399212b51f tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:50.273302 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-1273256b-9c33-4511-b1a5-0c399212b51f tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] [instance: 3c5ab09e-0a97-4cfc-b249-a11ab221b5d2] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:12:50.284638 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1273256b-9c33-4511-b1a5-0c399212b51f tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:50.285212 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1273256b-9c33-4511-b1a5-0c399212b51f tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:50.286100 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1273256b-9c33-4511-b1a5-0c399212b51f tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:50.286621 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1273256b-9c33-4511-b1a5-0c399212b51f tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:50.764825 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-1273256b-9c33-4511-b1a5-0c399212b51f tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:50.783354 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-1273256b-9c33-4511-b1a5-0c399212b51f tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:50.784940 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-1273256b-9c33-4511-b1a5-0c399212b51f tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:50.786992 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-1273256b-9c33-4511-b1a5-0c399212b51f tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] 10.222.0.16 "GET /compute/v2.1/servers/3c5ab09e-0a97-4cfc-b249-a11ab221b5d2" status: 200 len: 1408 microversion: 2.25 time: 0.520133 Jul 16 21:12:50.791563 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 597/1258] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:12:50 2022] GET /compute/v2.1/servers/3c5ab09e-0a97-4cfc-b249-a11ab221b5d2 => generated 1408 bytes in 525 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:12:50.799106 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-294ef168-3c9c-44d8-9230-979f3bfc80f6 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:50.803390 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-294ef168-3c9c-44d8-9230-979f3bfc80f6 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] [instance: 08e7e1f9-8b5c-446c-a05f-d8043473dd35] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:12:50.805536 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_api:SELECT=98,nova_api:UPDATE=2 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:12:50.822047 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-294ef168-3c9c-44d8-9230-979f3bfc80f6 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:50.822503 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-294ef168-3c9c-44d8-9230-979f3bfc80f6 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:50.826546 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-294ef168-3c9c-44d8-9230-979f3bfc80f6 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:50.827335 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-294ef168-3c9c-44d8-9230-979f3bfc80f6 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:51.171903 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-294ef168-3c9c-44d8-9230-979f3bfc80f6 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '52281b3989a04ca2a50b0bc93bf783f0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '30d028fa9b854c8fb71c8cbed2985c27', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:51.182487 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-294ef168-3c9c-44d8-9230-979f3bfc80f6 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] 10.222.0.16 "GET /compute/v2.1/servers/08e7e1f9-8b5c-446c-a05f-d8043473dd35" status: 200 len: 1318 microversion: 2.1 time: 0.389492 Jul 16 21:12:51.192806 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 598/1259] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:12:50 2022] GET /compute/v2.1/servers/08e7e1f9-8b5c-446c-a05f-d8043473dd35 => generated 1318 bytes in 400 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:12:51.198188 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-c1dd6446-db07-4c2a-8924-d53183922024 tempest-TenantUsagesTestJSON-260059616 tempest-TenantUsagesTestJSON-260059616-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:51.199743 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-c1dd6446-db07-4c2a-8924-d53183922024 tempest-TenantUsagesTestJSON-260059616 tempest-TenantUsagesTestJSON-260059616-project] [instance: 6e68dbd6-1d5c-471e-ad69-cf1c4aa4004a] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:12:51.207333 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c1dd6446-db07-4c2a-8924-d53183922024 tempest-TenantUsagesTestJSON-260059616 tempest-TenantUsagesTestJSON-260059616-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:51.208049 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c1dd6446-db07-4c2a-8924-d53183922024 tempest-TenantUsagesTestJSON-260059616 tempest-TenantUsagesTestJSON-260059616-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:51.209082 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c1dd6446-db07-4c2a-8924-d53183922024 tempest-TenantUsagesTestJSON-260059616 tempest-TenantUsagesTestJSON-260059616-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:51.209695 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c1dd6446-db07-4c2a-8924-d53183922024 tempest-TenantUsagesTestJSON-260059616 tempest-TenantUsagesTestJSON-260059616-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:51.361206 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-c1dd6446-db07-4c2a-8924-d53183922024 tempest-TenantUsagesTestJSON-260059616 tempest-TenantUsagesTestJSON-260059616-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2db871d6aa9746d2b660efbfbdb1dc75', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cad2f07400d54ef9825022d5b3489db0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:51.369112 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-c1dd6446-db07-4c2a-8924-d53183922024 tempest-TenantUsagesTestJSON-260059616 tempest-TenantUsagesTestJSON-260059616-project] 10.222.0.16 "GET /compute/v2.1/servers/6e68dbd6-1d5c-471e-ad69-cf1c4aa4004a" status: 200 len: 1266 microversion: 2.1 time: 0.174820 Jul 16 21:12:51.386565 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 599/1260] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:12:51 2022] GET /compute/v2.1/servers/6e68dbd6-1d5c-471e-ad69-cf1c4aa4004a => generated 1266 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:12:51.386565 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-dbcfaa5c-f33c-4a2a-be5d-9dc70c5a2448 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:51.386565 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-dbcfaa5c-f33c-4a2a-be5d-9dc70c5a2448 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: 126f561c-163f-4f6d-8be1-98a2e29b0909] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:12:51.388210 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-dbcfaa5c-f33c-4a2a-be5d-9dc70c5a2448 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:51.388388 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-dbcfaa5c-f33c-4a2a-be5d-9dc70c5a2448 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:51.389067 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-dbcfaa5c-f33c-4a2a-be5d-9dc70c5a2448 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:51.389767 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-dbcfaa5c-f33c-4a2a-be5d-9dc70c5a2448 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:51.469460 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.network.neutron [None req-511b02bb-912c-4f2d-a9ab-11f04df96fa3 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] [instance: 8020f553-6025-4f19-835c-c00e2fb4f764] Updating instance_info_cache with network_info: [{"id": "05294e88-e71a-4faa-8acc-79137ca6ee05", "address": "fa:16:3e:30:6c:72", "network": {"id": "9b8d8e4f-14e5-4e94-bfa5-2387fb1adcf8", "bridge": null, "label": "tempest-FloatingIPsAssociationTestJSON-379677821-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.35.1.237", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "172.20.0.2"}}], "meta": {"injected": false, "tenant_id": "cda86413e74b4c1e8e09bd4df4926664", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false, "bound_drivers": {"0": "hyperv"}}, "devname": "tap05294e88-e7", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=138680) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:117}} Jul 16 21:12:51.576362 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-511b02bb-912c-4f2d-a9ab-11f04df96fa3 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Releasing lock "refresh_cache-8020f553-6025-4f19-835c-c00e2fb4f764" {{(pid=138680) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:312}} Jul 16 21:12:51.577464 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-511b02bb-912c-4f2d-a9ab-11f04df96fa3 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] 10.222.0.16 "POST /compute/v2.1/servers/8020f553-6025-4f19-835c-c00e2fb4f764/action" status: 202 len: 0 microversion: 2.1 time: 4.009305 Jul 16 21:12:51.578294 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 662/1261] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 21:12:47 2022] POST /compute/v2.1/servers/8020f553-6025-4f19-835c-c00e2fb4f764/action => generated 0 bytes in 4011 msecs (HTTP/1.1 202) 9 headers in 369 bytes (1 switches on core 0) Jul 16 21:12:51.583170 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-5ab84067-61bc-4212-babc-e41bb96b1738 tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:51.583912 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-5ab84067-61bc-4212-babc-e41bb96b1738 tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] [instance: 74a19091-819b-4cf0-8109-fbb0de65313c] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:12:51.595226 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5ab84067-61bc-4212-babc-e41bb96b1738 tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:51.595226 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5ab84067-61bc-4212-babc-e41bb96b1738 tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:51.595226 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5ab84067-61bc-4212-babc-e41bb96b1738 tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:51.595226 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5ab84067-61bc-4212-babc-e41bb96b1738 tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:51.624707 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-dbcfaa5c-f33c-4a2a-be5d-9dc70c5a2448 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b68dacc4b8463ba07fae6372131117', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55a19b7a4682457a9132f5fc80110712', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:51.627531 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5ab84067-61bc-4212-babc-e41bb96b1738 tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:51.627980 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5ab84067-61bc-4212-babc-e41bb96b1738 tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:51.628430 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.objects.instance [None req-5ab84067-61bc-4212-babc-e41bb96b1738 tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Lazy-loading 'fault' on Instance uuid 74a19091-819b-4cf0-8109-fbb0de65313c {{(pid=138680) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:12:51.635119 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-dbcfaa5c-f33c-4a2a-be5d-9dc70c5a2448 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "GET /compute/v2.1/servers/126f561c-163f-4f6d-8be1-98a2e29b0909" status: 200 len: 1463 microversion: 2.1 time: 0.264344 Jul 16 21:12:51.636033 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 600/1262] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:12:51 2022] GET /compute/v2.1/servers/126f561c-163f-4f6d-8be1-98a2e29b0909 => generated 1463 bytes in 266 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:12:51.652619 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-f1d86303-7326-47f2-bcde-336ec62c581b tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:51.658533 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-f1d86303-7326-47f2-bcde-336ec62c581b tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] [instance: 6a391156-ffce-49c0-a79d-47b4544264f5] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:12:51.660215 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f1d86303-7326-47f2-bcde-336ec62c581b tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:51.660713 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f1d86303-7326-47f2-bcde-336ec62c581b tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:51.661510 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f1d86303-7326-47f2-bcde-336ec62c581b tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:51.661957 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f1d86303-7326-47f2-bcde-336ec62c581b tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:51.958436 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-5ab84067-61bc-4212-babc-e41bb96b1738 tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56d4bbcdebc24a34a814d7f6e72fbcfd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7d2b583c48254e17a8082f6467f2ae6d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:51.963152 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-f1d86303-7326-47f2-bcde-336ec62c581b tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a69f9dee03d43beaafa966216b9e289', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9b769bf8f3ea4e368bf68de1df6538f9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:51.968046 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-5ab84067-61bc-4212-babc-e41bb96b1738 tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] 10.222.0.16 "GET /compute/v2.1/servers/74a19091-819b-4cf0-8109-fbb0de65313c" status: 200 len: 1459 microversion: 2.1 time: 0.388667 Jul 16 21:12:51.968965 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 663/1263] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:12:51 2022] GET /compute/v2.1/servers/74a19091-819b-4cf0-8109-fbb0de65313c => generated 1459 bytes in 390 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:12:51.971415 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-f1d86303-7326-47f2-bcde-336ec62c581b tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] 10.222.0.16 "GET /compute/v2.1/servers/6a391156-ffce-49c0-a79d-47b4544264f5" status: 200 len: 1717 microversion: 2.1 time: 0.323203 Jul 16 21:12:51.971994 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 601/1264] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:12:51 2022] GET /compute/v2.1/servers/6a391156-ffce-49c0-a79d-47b4544264f5 => generated 1717 bytes in 336 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:12:51.976475 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-47036db2-ef0f-4b31-ba89-492571643cee tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Calling method '.version_select at 0x7fdd9cf0f550>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:51.983261 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-b96ab8d0-df14-49f5-af30-b979cc86577e req-d6dbd9f3-96eb-4d3d-9cbd-d0330c5c56c5 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "4ae3e944-b450-40e5-bb71-c02df03212e3", "name": "network-vif-deleted", "tag": "578b9284-5794-41e6-80d7-b5c49283c6f9"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:12:51.991744 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-b96ab8d0-df14-49f5-af30-b979cc86577e req-d6dbd9f3-96eb-4d3d-9cbd-d0330c5c56c5 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:51.992842 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-b96ab8d0-df14-49f5-af30-b979cc86577e req-d6dbd9f3-96eb-4d3d-9cbd-d0330c5c56c5 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:52.004081 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-b96ab8d0-df14-49f5-af30-b979cc86577e req-d6dbd9f3-96eb-4d3d-9cbd-d0330c5c56c5 service nova] Creating event network-vif-deleted:578b9284-5794-41e6-80d7-b5c49283c6f9 for instance 4ae3e944-b450-40e5-bb71-c02df03212e3 on nch2-master-1 Jul 16 21:12:52.009747 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-b96ab8d0-df14-49f5-af30-b979cc86577e req-d6dbd9f3-96eb-4d3d-9cbd-d0330c5c56c5 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.029763 Jul 16 21:12:52.010265 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 664/1265] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:12:51 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:12:52.016885 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-61cf450e-b8cb-44c1-8b0f-ed240ef345d5 tempest-MigrationsAdminTest-399208122 tempest-MigrationsAdminTest-399208122-project-admin] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:52.017593 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-61cf450e-b8cb-44c1-8b0f-ed240ef345d5 tempest-MigrationsAdminTest-399208122 tempest-MigrationsAdminTest-399208122-project-admin] [instance: 126f561c-163f-4f6d-8be1-98a2e29b0909] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:12:52.024278 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-61cf450e-b8cb-44c1-8b0f-ed240ef345d5 tempest-MigrationsAdminTest-399208122 tempest-MigrationsAdminTest-399208122-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:52.024750 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-61cf450e-b8cb-44c1-8b0f-ed240ef345d5 tempest-MigrationsAdminTest-399208122 tempest-MigrationsAdminTest-399208122-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:52.025551 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-61cf450e-b8cb-44c1-8b0f-ed240ef345d5 tempest-MigrationsAdminTest-399208122 tempest-MigrationsAdminTest-399208122-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:52.026045 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-61cf450e-b8cb-44c1-8b0f-ed240ef345d5 tempest-MigrationsAdminTest-399208122 tempest-MigrationsAdminTest-399208122-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:52.244930 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-47036db2-ef0f-4b31-ba89-492571643cee tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] 10.222.0.16 "GET /compute/v2.1/os-floating-ips/727dc9c6-603c-4649-98a7-3759801fbfef" status: 200 len: 185 microversion: 2.1 time: 0.271885 Jul 16 21:12:52.245712 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 602/1266] 10.222.0.16 () {58 vars in 1259 bytes} [Sat Jul 16 21:12:51 2022] GET /compute/v2.1/os-floating-ips/727dc9c6-603c-4649-98a7-3759801fbfef => generated 185 bytes in 273 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:12:52.250093 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-ef88dc13-f1dc-4464-ba6f-48d946d44aeb req-ec1d1c7d-40aa-4166-a127-cf5b0f94ce8b service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "8020f553-6025-4f19-835c-c00e2fb4f764", "tag": "05294e88-e71a-4faa-8acc-79137ca6ee05"}, {"name": "network-changed", "server_uuid": "08e7e1f9-8b5c-446c-a05f-d8043473dd35", "tag": "b21352ed-bfe1-4d3f-93ff-6f5027326093"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:12:52.259870 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-ef88dc13-f1dc-4464-ba6f-48d946d44aeb req-ec1d1c7d-40aa-4166-a127-cf5b0f94ce8b service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:52.260320 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-ef88dc13-f1dc-4464-ba6f-48d946d44aeb req-ec1d1c7d-40aa-4166-a127-cf5b0f94ce8b service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:52.262540 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-61cf450e-b8cb-44c1-8b0f-ed240ef345d5 tempest-MigrationsAdminTest-399208122 tempest-MigrationsAdminTest-399208122-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/126f561c-163f-4f6d-8be1-98a2e29b0909" status: 200 len: 1614 microversion: 2.1 time: 0.250147 Jul 16 21:12:52.263411 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 665/1267] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:12:52 2022] GET /compute/v2.1/servers/126f561c-163f-4f6d-8be1-98a2e29b0909 => generated 1614 bytes in 252 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:12:52.268086 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-405238b2-1755-416d-ae8b-6d4068333800 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:52.268691 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-405238b2-1755-416d-ae8b-6d4068333800 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] [instance: 3c5ab09e-0a97-4cfc-b249-a11ab221b5d2] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:12:52.274860 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-ef88dc13-f1dc-4464-ba6f-48d946d44aeb req-ec1d1c7d-40aa-4166-a127-cf5b0f94ce8b service nova] Creating event network-changed:05294e88-e71a-4faa-8acc-79137ca6ee05 for instance 8020f553-6025-4f19-835c-c00e2fb4f764 on nch1-master-1 Jul 16 21:12:52.275513 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-ef88dc13-f1dc-4464-ba6f-48d946d44aeb req-ec1d1c7d-40aa-4166-a127-cf5b0f94ce8b service nova] Creating event network-changed:b21352ed-bfe1-4d3f-93ff-6f5027326093 for instance 08e7e1f9-8b5c-446c-a05f-d8043473dd35 on nch1-master-1 Jul 16 21:12:52.277038 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-405238b2-1755-416d-ae8b-6d4068333800 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:52.277499 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-405238b2-1755-416d-ae8b-6d4068333800 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:52.278284 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-405238b2-1755-416d-ae8b-6d4068333800 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:52.278755 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-405238b2-1755-416d-ae8b-6d4068333800 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:52.282852 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-ef88dc13-f1dc-4464-ba6f-48d946d44aeb req-ec1d1c7d-40aa-4166-a127-cf5b0f94ce8b service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 346 microversion: 2.1 time: 0.036122 Jul 16 21:12:52.291062 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 603/1268] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:12:52 2022] POST /compute/v2.1/os-server-external-events => generated 346 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:12:52.297918 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-6e5ade6e-7bfe-4cea-9420-f573174768e7 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:52.299066 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-6e5ade6e-7bfe-4cea-9420-f573174768e7 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] [instance: 08e7e1f9-8b5c-446c-a05f-d8043473dd35] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:12:52.310583 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6e5ade6e-7bfe-4cea-9420-f573174768e7 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:52.312982 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6e5ade6e-7bfe-4cea-9420-f573174768e7 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:52.314217 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6e5ade6e-7bfe-4cea-9420-f573174768e7 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:52.314957 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6e5ade6e-7bfe-4cea-9420-f573174768e7 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:52.725305 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-405238b2-1755-416d-ae8b-6d4068333800 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:52.725817 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-6e5ade6e-7bfe-4cea-9420-f573174768e7 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '52281b3989a04ca2a50b0bc93bf783f0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '30d028fa9b854c8fb71c8cbed2985c27', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:52.737182 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-6e5ade6e-7bfe-4cea-9420-f573174768e7 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] 10.222.0.16 "GET /compute/v2.1/servers/08e7e1f9-8b5c-446c-a05f-d8043473dd35" status: 200 len: 1318 microversion: 2.1 time: 0.444746 Jul 16 21:12:52.738142 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 604/1269] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:12:52 2022] GET /compute/v2.1/servers/08e7e1f9-8b5c-446c-a05f-d8043473dd35 => generated 1318 bytes in 446 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:12:52.743836 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-65893770-0ddf-4193-9977-099101afec10 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Action: 'action', calling method: , body: {"removeFloatingIp": {"address": "172.35.1.237"}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:12:52.753917 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-405238b2-1755-416d-ae8b-6d4068333800 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:52.756073 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-405238b2-1755-416d-ae8b-6d4068333800 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:52.757526 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-405238b2-1755-416d-ae8b-6d4068333800 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] 10.222.0.16 "GET /compute/v2.1/servers/3c5ab09e-0a97-4cfc-b249-a11ab221b5d2" status: 200 len: 1408 microversion: 2.25 time: 0.493296 Jul 16 21:12:52.769549 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 666/1270] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:12:52 2022] GET /compute/v2.1/servers/3c5ab09e-0a97-4cfc-b249-a11ab221b5d2 => generated 1408 bytes in 497 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:12:52.769549 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-bdfaf651-8520-430e-a989-078c2b52b221 tempest-MigrationsAdminTest-399208122 tempest-MigrationsAdminTest-399208122-project-admin] Action: 'action', calling method: >, body: {"migrate": {}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:12:52.769549 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-bdfaf651-8520-430e-a989-078c2b52b221 tempest-MigrationsAdminTest-399208122 tempest-MigrationsAdminTest-399208122-project-admin] [instance: 126f561c-163f-4f6d-8be1-98a2e29b0909] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:12:52.784756 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-bdfaf651-8520-430e-a989-078c2b52b221 tempest-MigrationsAdminTest-399208122 tempest-MigrationsAdminTest-399208122-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:52.785369 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-bdfaf651-8520-430e-a989-078c2b52b221 tempest-MigrationsAdminTest-399208122 tempest-MigrationsAdminTest-399208122-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:52.786168 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-bdfaf651-8520-430e-a989-078c2b52b221 tempest-MigrationsAdminTest-399208122 tempest-MigrationsAdminTest-399208122-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:52.786611 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-bdfaf651-8520-430e-a989-078c2b52b221 tempest-MigrationsAdminTest-399208122 tempest-MigrationsAdminTest-399208122-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:52.826254 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-bdfaf651-8520-430e-a989-078c2b52b221 tempest-MigrationsAdminTest-399208122 tempest-MigrationsAdminTest-399208122-project-admin] Policy check for compute:servers:resize:cross_cell failed with credentials {'is_admin': True, 'user_id': '1ead1619fabc4552b11f712b28cc6187', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cbbed051cbaf450f92a79d71d58ba7a2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:52.828223 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-bdfaf651-8520-430e-a989-078c2b52b221 tempest-MigrationsAdminTest-399208122 tempest-MigrationsAdminTest-399208122-project-admin] [instance: 126f561c-163f-4f6d-8be1-98a2e29b0909] flavor_id is None. Assuming migration. {{(pid=138680) resize /opt/stack/nova/nova/compute/api.py:4142}} Jul 16 21:12:52.828881 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-bdfaf651-8520-430e-a989-078c2b52b221 tempest-MigrationsAdminTest-399208122 tempest-MigrationsAdminTest-399208122-project-admin] [instance: 126f561c-163f-4f6d-8be1-98a2e29b0909] Old instance type m1.heat2, new instance type m1.heat2 {{(pid=138680) resize /opt/stack/nova/nova/compute/api.py:4166}} Jul 16 21:12:52.988091 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-65893770-0ddf-4193-9977-099101afec10 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] [instance: 8020f553-6025-4f19-835c-c00e2fb4f764] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:12:52.995697 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-65893770-0ddf-4193-9977-099101afec10 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:52.995988 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-65893770-0ddf-4193-9977-099101afec10 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:52.997052 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-65893770-0ddf-4193-9977-099101afec10 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:52.997748 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-65893770-0ddf-4193-9977-099101afec10 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:53.195590 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-bdfaf651-8520-430e-a989-078c2b52b221 tempest-MigrationsAdminTest-399208122 tempest-MigrationsAdminTest-399208122-project-admin] 10.222.0.16 "POST /compute/v2.1/servers/126f561c-163f-4f6d-8be1-98a2e29b0909/action" status: 202 len: 0 microversion: 2.1 time: 0.432724 Jul 16 21:12:53.199229 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 667/1271] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 21:12:52 2022] POST /compute/v2.1/servers/126f561c-163f-4f6d-8be1-98a2e29b0909/action => generated 0 bytes in 437 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 21:12:53.205510 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-c759848b-3121-49ee-8a77-0bb15dfcc308 tempest-TenantUsagesTestJSON-260059616 tempest-TenantUsagesTestJSON-260059616-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:53.206790 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-c759848b-3121-49ee-8a77-0bb15dfcc308 tempest-TenantUsagesTestJSON-260059616 tempest-TenantUsagesTestJSON-260059616-project] [instance: 6e68dbd6-1d5c-471e-ad69-cf1c4aa4004a] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:12:53.226300 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c759848b-3121-49ee-8a77-0bb15dfcc308 tempest-TenantUsagesTestJSON-260059616 tempest-TenantUsagesTestJSON-260059616-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:53.227793 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c759848b-3121-49ee-8a77-0bb15dfcc308 tempest-TenantUsagesTestJSON-260059616 tempest-TenantUsagesTestJSON-260059616-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:53.228879 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c759848b-3121-49ee-8a77-0bb15dfcc308 tempest-TenantUsagesTestJSON-260059616 tempest-TenantUsagesTestJSON-260059616-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:53.229578 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c759848b-3121-49ee-8a77-0bb15dfcc308 tempest-TenantUsagesTestJSON-260059616 tempest-TenantUsagesTestJSON-260059616-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:53.260028 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=95,nova_cell1:SAVEPOINT=1,nova_cell1:UPDATE=2,nova_cell1:RELEASE=1,nova_cell1:INSERT=1 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:12:53.371077 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-c759848b-3121-49ee-8a77-0bb15dfcc308 tempest-TenantUsagesTestJSON-260059616 tempest-TenantUsagesTestJSON-260059616-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2db871d6aa9746d2b660efbfbdb1dc75', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cad2f07400d54ef9825022d5b3489db0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:53.380585 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-c759848b-3121-49ee-8a77-0bb15dfcc308 tempest-TenantUsagesTestJSON-260059616 tempest-TenantUsagesTestJSON-260059616-project] 10.222.0.16 "GET /compute/v2.1/servers/6e68dbd6-1d5c-471e-ad69-cf1c4aa4004a" status: 200 len: 1272 microversion: 2.1 time: 0.179829 Jul 16 21:12:53.383657 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 668/1272] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:12:53 2022] GET /compute/v2.1/servers/6e68dbd6-1d5c-471e-ad69-cf1c4aa4004a => generated 1272 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:12:53.398658 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-a15f981b-0ccd-4c1a-bd81-e7d16fe2c7d8 req-7e7efeb4-30a8-45c7-9e3c-c1311062f262 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "6a391156-ffce-49c0-a79d-47b4544264f5", "name": "network-vif-unplugged", "status": "completed", "tag": "a8de1bba-f28a-49ad-92e5-8a10d67f1e75"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:12:53.407511 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-a15f981b-0ccd-4c1a-bd81-e7d16fe2c7d8 req-7e7efeb4-30a8-45c7-9e3c-c1311062f262 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:53.408530 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-a15f981b-0ccd-4c1a-bd81-e7d16fe2c7d8 req-7e7efeb4-30a8-45c7-9e3c-c1311062f262 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:53.420169 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-a15f981b-0ccd-4c1a-bd81-e7d16fe2c7d8 req-7e7efeb4-30a8-45c7-9e3c-c1311062f262 service nova] Creating event network-vif-unplugged:a8de1bba-f28a-49ad-92e5-8a10d67f1e75 for instance 6a391156-ffce-49c0-a79d-47b4544264f5 on nch2-master-1 Jul 16 21:12:53.425076 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-a15f981b-0ccd-4c1a-bd81-e7d16fe2c7d8 req-7e7efeb4-30a8-45c7-9e3c-c1311062f262 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.039169 Jul 16 21:12:53.432821 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 669/1273] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:12:53 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:12:53.435801 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-da538298-1184-48c2-b301-b7efad5e323b tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:53.436502 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-da538298-1184-48c2-b301-b7efad5e323b tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] [instance: 74a19091-819b-4cf0-8109-fbb0de65313c] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:12:53.443159 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-da538298-1184-48c2-b301-b7efad5e323b tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:53.443920 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-da538298-1184-48c2-b301-b7efad5e323b tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:53.444686 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-da538298-1184-48c2-b301-b7efad5e323b tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:53.445126 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-da538298-1184-48c2-b301-b7efad5e323b tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:53.489002 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-da538298-1184-48c2-b301-b7efad5e323b tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:53.489305 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-da538298-1184-48c2-b301-b7efad5e323b tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:53.489761 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.objects.instance [None req-da538298-1184-48c2-b301-b7efad5e323b tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Lazy-loading 'fault' on Instance uuid 74a19091-819b-4cf0-8109-fbb0de65313c {{(pid=138680) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:12:53.604201 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-da538298-1184-48c2-b301-b7efad5e323b tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '56d4bbcdebc24a34a814d7f6e72fbcfd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7d2b583c48254e17a8082f6467f2ae6d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:53.614482 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-da538298-1184-48c2-b301-b7efad5e323b tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] 10.222.0.16 "GET /compute/v2.1/servers/74a19091-819b-4cf0-8109-fbb0de65313c" status: 200 len: 1417 microversion: 2.1 time: 0.182736 Jul 16 21:12:53.617762 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 670/1274] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:12:53 2022] GET /compute/v2.1/servers/74a19091-819b-4cf0-8109-fbb0de65313c => generated 1417 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:12:53.621561 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-e80803ac-414c-4ddc-9e48-c4c115b0912a tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:53.622189 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-e80803ac-414c-4ddc-9e48-c4c115b0912a tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] [instance: 6a391156-ffce-49c0-a79d-47b4544264f5] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:12:53.634879 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e80803ac-414c-4ddc-9e48-c4c115b0912a tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:53.635272 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e80803ac-414c-4ddc-9e48-c4c115b0912a tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:53.636139 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e80803ac-414c-4ddc-9e48-c4c115b0912a tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:53.636731 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e80803ac-414c-4ddc-9e48-c4c115b0912a tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:53.637600 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-65893770-0ddf-4193-9977-099101afec10 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Acquired lock "refresh_cache-8020f553-6025-4f19-835c-c00e2fb4f764" {{(pid=138681) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:294}} Jul 16 21:12:54.424668 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-e80803ac-414c-4ddc-9e48-c4c115b0912a tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a69f9dee03d43beaafa966216b9e289', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9b769bf8f3ea4e368bf68de1df6538f9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:54.446536 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-e80803ac-414c-4ddc-9e48-c4c115b0912a tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] 10.222.0.16 "GET /compute/v2.1/servers/6a391156-ffce-49c0-a79d-47b4544264f5" status: 200 len: 1717 microversion: 2.1 time: 0.818537 Jul 16 21:12:54.446536 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 671/1275] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:12:53 2022] GET /compute/v2.1/servers/6a391156-ffce-49c0-a79d-47b4544264f5 => generated 1717 bytes in 820 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:12:54.451350 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-89fea7a1-70e3-4dbc-b619-96d7d8906fe6 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:54.452487 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-89fea7a1-70e3-4dbc-b619-96d7d8906fe6 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: 126f561c-163f-4f6d-8be1-98a2e29b0909] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:12:54.459818 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-89fea7a1-70e3-4dbc-b619-96d7d8906fe6 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:54.460282 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-89fea7a1-70e3-4dbc-b619-96d7d8906fe6 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:54.462893 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-89fea7a1-70e3-4dbc-b619-96d7d8906fe6 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:54.463388 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-89fea7a1-70e3-4dbc-b619-96d7d8906fe6 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:55.156610 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-89fea7a1-70e3-4dbc-b619-96d7d8906fe6 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b68dacc4b8463ba07fae6372131117', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55a19b7a4682457a9132f5fc80110712', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:55.166201 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-89fea7a1-70e3-4dbc-b619-96d7d8906fe6 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "GET /compute/v2.1/servers/126f561c-163f-4f6d-8be1-98a2e29b0909" status: 200 len: 1472 microversion: 2.1 time: 0.728803 Jul 16 21:12:55.176329 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 672/1276] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:12:54 2022] GET /compute/v2.1/servers/126f561c-163f-4f6d-8be1-98a2e29b0909 => generated 1472 bytes in 730 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:12:55.176329 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-2014b645-1170-458a-b33e-de115b6a85d2 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:55.176329 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-2014b645-1170-458a-b33e-de115b6a85d2 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] [instance: 08e7e1f9-8b5c-446c-a05f-d8043473dd35] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:12:55.179057 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2014b645-1170-458a-b33e-de115b6a85d2 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:55.180512 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2014b645-1170-458a-b33e-de115b6a85d2 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:55.185505 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2014b645-1170-458a-b33e-de115b6a85d2 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:55.185980 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2014b645-1170-458a-b33e-de115b6a85d2 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:55.706231 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-2014b645-1170-458a-b33e-de115b6a85d2 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '52281b3989a04ca2a50b0bc93bf783f0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '30d028fa9b854c8fb71c8cbed2985c27', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:55.706231 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-2014b645-1170-458a-b33e-de115b6a85d2 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] 10.222.0.16 "GET /compute/v2.1/servers/08e7e1f9-8b5c-446c-a05f-d8043473dd35" status: 200 len: 1318 microversion: 2.1 time: 0.520115 Jul 16 21:12:55.706231 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 673/1277] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:12:55 2022] GET /compute/v2.1/servers/08e7e1f9-8b5c-446c-a05f-d8043473dd35 => generated 1318 bytes in 521 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:12:55.706231 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-3ff10216-db2b-43bc-8445-7868cca267d4 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:55.707376 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-3ff10216-db2b-43bc-8445-7868cca267d4 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] [instance: 3c5ab09e-0a97-4cfc-b249-a11ab221b5d2] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:12:55.707376 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3ff10216-db2b-43bc-8445-7868cca267d4 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:55.707376 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3ff10216-db2b-43bc-8445-7868cca267d4 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:55.707376 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3ff10216-db2b-43bc-8445-7868cca267d4 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:55.707735 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3ff10216-db2b-43bc-8445-7868cca267d4 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:56.065170 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell0:SELECT=26 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:12:56.077267 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-3ff10216-db2b-43bc-8445-7868cca267d4 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:56.085427 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-3ff10216-db2b-43bc-8445-7868cca267d4 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:56.087581 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-3ff10216-db2b-43bc-8445-7868cca267d4 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:56.088813 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-3ff10216-db2b-43bc-8445-7868cca267d4 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] 10.222.0.16 "GET /compute/v2.1/servers/3c5ab09e-0a97-4cfc-b249-a11ab221b5d2" status: 200 len: 1603 microversion: 2.25 time: 0.399053 Jul 16 21:12:56.089377 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 674/1278] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:12:55 2022] GET /compute/v2.1/servers/3c5ab09e-0a97-4cfc-b249-a11ab221b5d2 => generated 1603 bytes in 400 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:12:56.093866 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-ada4f4b3-aa6b-48b2-9a35-c9a6ce460513 req-bfb1e78e-f6cc-4802-befc-d3478b60f67a service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "74a19091-819b-4cf0-8109-fbb0de65313c", "name": "network-vif-deleted", "tag": "7500284f-1b91-4c4b-bf7d-d5f21190a8ed"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:12:56.102880 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-ada4f4b3-aa6b-48b2-9a35-c9a6ce460513 req-bfb1e78e-f6cc-4802-befc-d3478b60f67a service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:56.108489 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-ada4f4b3-aa6b-48b2-9a35-c9a6ce460513 req-bfb1e78e-f6cc-4802-befc-d3478b60f67a service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:56.116127 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-ada4f4b3-aa6b-48b2-9a35-c9a6ce460513 req-bfb1e78e-f6cc-4802-befc-d3478b60f67a service nova] Creating event network-vif-deleted:7500284f-1b91-4c4b-bf7d-d5f21190a8ed for instance 74a19091-819b-4cf0-8109-fbb0de65313c on nch2-master-1 Jul 16 21:12:56.122179 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-ada4f4b3-aa6b-48b2-9a35-c9a6ce460513 req-bfb1e78e-f6cc-4802-befc-d3478b60f67a service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.031623 Jul 16 21:12:56.122854 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 675/1279] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:12:56 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:12:56.128797 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-5a7844de-7704-4d61-bcc4-9dfafbc80f6e req-92f34207-7b00-4b77-a57f-2176cf152e26 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "6a391156-ffce-49c0-a79d-47b4544264f5", "name": "network-vif-unplugged", "status": "completed", "tag": "a8de1bba-f28a-49ad-92e5-8a10d67f1e75"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:12:56.139234 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-5a7844de-7704-4d61-bcc4-9dfafbc80f6e req-92f34207-7b00-4b77-a57f-2176cf152e26 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:56.139629 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-5a7844de-7704-4d61-bcc4-9dfafbc80f6e req-92f34207-7b00-4b77-a57f-2176cf152e26 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:56.151770 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-5a7844de-7704-4d61-bcc4-9dfafbc80f6e req-92f34207-7b00-4b77-a57f-2176cf152e26 service nova] Creating event network-vif-unplugged:a8de1bba-f28a-49ad-92e5-8a10d67f1e75 for instance 6a391156-ffce-49c0-a79d-47b4544264f5 on nch2-master-1 Jul 16 21:12:56.157041 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-5a7844de-7704-4d61-bcc4-9dfafbc80f6e req-92f34207-7b00-4b77-a57f-2176cf152e26 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.032506 Jul 16 21:12:56.158295 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 676/1280] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:12:56 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:12:56.163733 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-c1a0c450-5c0d-4d66-8072-6736427612be tempest-TenantUsagesTestJSON-260059616 tempest-TenantUsagesTestJSON-260059616-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:56.164746 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-c1a0c450-5c0d-4d66-8072-6736427612be tempest-TenantUsagesTestJSON-260059616 tempest-TenantUsagesTestJSON-260059616-project] [instance: 6e68dbd6-1d5c-471e-ad69-cf1c4aa4004a] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:12:56.171669 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c1a0c450-5c0d-4d66-8072-6736427612be tempest-TenantUsagesTestJSON-260059616 tempest-TenantUsagesTestJSON-260059616-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:56.174214 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c1a0c450-5c0d-4d66-8072-6736427612be tempest-TenantUsagesTestJSON-260059616 tempest-TenantUsagesTestJSON-260059616-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:56.174214 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c1a0c450-5c0d-4d66-8072-6736427612be tempest-TenantUsagesTestJSON-260059616 tempest-TenantUsagesTestJSON-260059616-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:56.174214 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c1a0c450-5c0d-4d66-8072-6736427612be tempest-TenantUsagesTestJSON-260059616 tempest-TenantUsagesTestJSON-260059616-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:56.230669 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.network.neutron [None req-65893770-0ddf-4193-9977-099101afec10 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] [instance: 8020f553-6025-4f19-835c-c00e2fb4f764] Building network info cache for instance {{(pid=138681) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1990}} Jul 16 21:12:56.786890 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell0:SELECT=14 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:12:57.024992 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-c1a0c450-5c0d-4d66-8072-6736427612be tempest-TenantUsagesTestJSON-260059616 tempest-TenantUsagesTestJSON-260059616-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2db871d6aa9746d2b660efbfbdb1dc75', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cad2f07400d54ef9825022d5b3489db0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:57.036414 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-c1a0c450-5c0d-4d66-8072-6736427612be tempest-TenantUsagesTestJSON-260059616 tempest-TenantUsagesTestJSON-260059616-project] 10.222.0.16 "GET /compute/v2.1/servers/6e68dbd6-1d5c-471e-ad69-cf1c4aa4004a" status: 200 len: 1314 microversion: 2.1 time: 0.876585 Jul 16 21:12:57.038018 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 677/1281] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:12:56 2022] GET /compute/v2.1/servers/6e68dbd6-1d5c-471e-ad69-cf1c4aa4004a => generated 1314 bytes in 879 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:12:57.049541 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-a6e1bba6-1826-4684-af92-6c7dd52240b5 tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:57.052300 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-a6e1bba6-1826-4684-af92-6c7dd52240b5 tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] [instance: 74a19091-819b-4cf0-8109-fbb0de65313c] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:12:57.061394 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a6e1bba6-1826-4684-af92-6c7dd52240b5 tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:57.067728 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a6e1bba6-1826-4684-af92-6c7dd52240b5 tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:57.067728 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a6e1bba6-1826-4684-af92-6c7dd52240b5 tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:57.067728 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a6e1bba6-1826-4684-af92-6c7dd52240b5 tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:57.085881 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-a6e1bba6-1826-4684-af92-6c7dd52240b5 tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] HTTP exception thrown: Instance 74a19091-819b-4cf0-8109-fbb0de65313c could not be found. Jul 16 21:12:57.091365 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-a6e1bba6-1826-4684-af92-6c7dd52240b5 tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Request method failure captured: Jul 16 21:12:57.091365 ncdv-master-1 devstack@n-api.service[138680]: request: GET /compute/v2.1/servers/74a19091-819b-4cf0-8109-fbb0de65313c HTTP/1.1 Jul 16 21:12:57.091365 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:12:57.091365 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:12:57.091365 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:12:57.091365 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 0 Jul 16 21:12:57.091365 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:12:57.091365 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:12:57.091365 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:12:57.091365 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:12:57.091365 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0yme2OmgPmghOZxTYv1prM8I30jqdYxV66dKB7kNvJ87HPySbthZ6En_3VLO5oCbOf2UuM1ARwLRB17-5AINJPTiqPrtco0L2oHyrrsd2HBbkXlr_mLFUNTutX_YwODzB_43ujJc9IK-SLZSWvs3yKGjofUk2kkT8-qMrEcv7y0wtas Jul 16 21:12:57.091365 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:12:57.091365 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:12:57.091365 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:12:57.091365 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:12:57.091365 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:12:57.091365 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:12:57.091365 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: 7d2b583c48254e17a8082f6467f2ae6d Jul 16 21:12:57.091365 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-DeleteServersAdminTestJSON-69305867 Jul 16 21:12:57.091365 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:12:57.091365 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:12:57.102602 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/7d2b583c48254e17a8082f6467f2ae6d"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/7d2b583c48254e17a8082f6467f2ae6d"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_7d2b583c48254e17a8082f6467f2ae6d"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/7d2b583c48254e17a8082f6467f2ae6d"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:12:57.102602 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-DeleteServersAdminTestJSON-69305867 Jul 16 21:12:57.102602 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: 7d2b583c48254e17a8082f6467f2ae6d Jul 16 21:12:57.102602 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-DeleteServersAdminTestJSON-69305867 Jul 16 21:12:57.102602 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-DeleteServersAdminTestJSON-69305867-project Jul 16 21:12:57.102602 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:12:57.102602 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:12:57.102602 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: 56d4bbcdebc24a34a814d7f6e72fbcfd Jul 16 21:12:57.102602 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-DeleteServersAdminTestJSON-69305867-project Jul 16 21:12:57.102602 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 21:12:57.102602 ncdv-master-1 devstack@n-api.service[138680]: exception: Instance 74a19091-819b-4cf0-8109-fbb0de65313c could not be found. Jul 16 21:12:57.102602 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': '74a19091-819b-4cf0-8109-fbb0de65313c'} Jul 16 21:12:57.103468 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:12:57.103468 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:12:57.103468 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:12:57.103468 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:12:57.103468 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:12:57.103468 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:12:57.103468 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:12:57.103468 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:12:57.103468 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:12:57.103468 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:12:57.103468 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 74a19091-819b-4cf0-8109-fbb0de65313c could not be found. Jul 16 21:12:57.103468 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:12:57.103468 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:12:57.103468 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:12:57.103468 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:12:57.103468 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:12:57.103468 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:12:57.103468 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:12:57.103468 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:12:57.103468 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:12:57.104724 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:12:57.104724 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:12:57.104724 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:12:57.104724 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:12:57.104724 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:12:57.104724 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:12:57.104724 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:12:57.104724 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 74a19091-819b-4cf0-8109-fbb0de65313c could not be found. Jul 16 21:12:57.104724 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:12:57.104724 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:12:57.104724 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:12:57.104724 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:12:57.104724 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:12:57.104724 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:12:57.104724 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:12:57.104724 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:12:57.104724 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 74a19091-819b-4cf0-8109-fbb0de65313c could not be found. Jul 16 21:12:57.104724 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:12:57.105816 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-a6e1bba6-1826-4684-af92-6c7dd52240b5 tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Returning 404 to user: Instance 74a19091-819b-4cf0-8109-fbb0de65313c could not be found. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:12:57.105816 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-a6e1bba6-1826-4684-af92-6c7dd52240b5 tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] 10.222.0.16 "GET /compute/v2.1/servers/74a19091-819b-4cf0-8109-fbb0de65313c" status: 404 len: 111 microversion: 2.1 time: 0.053977 Jul 16 21:12:57.105816 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 678/1282] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:12:57 2022] GET /compute/v2.1/servers/74a19091-819b-4cf0-8109-fbb0de65313c => generated 111 bytes in 63 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:12:57.107601 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-3ab2e85f-0c36-4cf9-b5ae-cb2e398903b3 tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:57.113882 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-3ab2e85f-0c36-4cf9-b5ae-cb2e398903b3 tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] Policy check for os_compute_api:servers:allow_all_filters failed with credentials {'is_admin': False, 'user_id': '034c11cc3f174c6fb44c8d64e55cf8b9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb4ff5d7e42940cdb7be71dadd895ee4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:57.114778 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-3ab2e85f-0c36-4cf9-b5ae-cb2e398903b3 tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] Searching by: {'deleted': False, 'project_id': 'fb4ff5d7e42940cdb7be71dadd895ee4'} {{(pid=138680) get_all /opt/stack/nova/nova/compute/api.py:2962}} Jul 16 21:12:57.133989 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3ab2e85f-0c36-4cf9-b5ae-cb2e398903b3 tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:57.134883 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3ab2e85f-0c36-4cf9-b5ae-cb2e398903b3 tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:57.136916 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3ab2e85f-0c36-4cf9-b5ae-cb2e398903b3 tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:57.139192 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3ab2e85f-0c36-4cf9-b5ae-cb2e398903b3 tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:57.175476 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.multi_cell_list [None req-3ab2e85f-0c36-4cf9-b5ae-cb2e398903b3 tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] Listed batch of 1 results from cell out of 1000 limit. Returned 1 total so far. {{(pid=138680) do_query /opt/stack/nova/nova/compute/multi_cell_list.py:373}} Jul 16 21:12:57.209915 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-3ab2e85f-0c36-4cf9-b5ae-cb2e398903b3 tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] 10.222.0.16 "GET /compute/v2.1/servers" status: 200 len: 334 microversion: 2.1 time: 0.106275 Jul 16 21:12:57.210496 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 679/1283] 10.222.0.16 () {58 vars in 1124 bytes} [Sat Jul 16 21:12:57 2022] GET /compute/v2.1/servers => generated 334 bytes in 108 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:12:57.216507 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-46a7ab20-c50a-4eed-804c-24717fccc685 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:57.218835 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-46a7ab20-c50a-4eed-804c-24717fccc685 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] [instance: 6a391156-ffce-49c0-a79d-47b4544264f5] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:12:57.225843 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-46a7ab20-c50a-4eed-804c-24717fccc685 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:57.228417 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-46a7ab20-c50a-4eed-804c-24717fccc685 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:57.228417 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-46a7ab20-c50a-4eed-804c-24717fccc685 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:57.228417 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-46a7ab20-c50a-4eed-804c-24717fccc685 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:57.887075 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-46a7ab20-c50a-4eed-804c-24717fccc685 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a69f9dee03d43beaafa966216b9e289', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9b769bf8f3ea4e368bf68de1df6538f9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:57.895527 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-46a7ab20-c50a-4eed-804c-24717fccc685 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] 10.222.0.16 "GET /compute/v2.1/servers/6a391156-ffce-49c0-a79d-47b4544264f5" status: 200 len: 1717 microversion: 2.1 time: 0.683483 Jul 16 21:12:57.896033 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 680/1284] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:12:57 2022] GET /compute/v2.1/servers/6a391156-ffce-49c0-a79d-47b4544264f5 => generated 1717 bytes in 685 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:12:57.910471 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-30f9e057-b26b-44b4-b3fb-0cb6165c858a tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:57.910973 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-30f9e057-b26b-44b4-b3fb-0cb6165c858a tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] [instance: 3c5ab09e-0a97-4cfc-b249-a11ab221b5d2] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:12:57.917655 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-30f9e057-b26b-44b4-b3fb-0cb6165c858a tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:57.918106 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-30f9e057-b26b-44b4-b3fb-0cb6165c858a tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:57.918878 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-30f9e057-b26b-44b4-b3fb-0cb6165c858a tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:57.919361 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-30f9e057-b26b-44b4-b3fb-0cb6165c858a tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:58.171479 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-30f9e057-b26b-44b4-b3fb-0cb6165c858a tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] 10.222.0.16 "GET /compute/v2.1/servers/3c5ab09e-0a97-4cfc-b249-a11ab221b5d2/os-interface" status: 200 len: 284 microversion: 2.25 time: 0.264698 Jul 16 21:12:58.172155 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 681/1285] 10.222.0.16 () {60 vars in 1315 bytes} [Sat Jul 16 21:12:57 2022] GET /compute/v2.1/servers/3c5ab09e-0a97-4cfc-b249-a11ab221b5d2/os-interface => generated 284 bytes in 276 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 16 21:12:58.177282 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-ea717b7b-25bd-48b5-91a1-fc7fc8e96cd8 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:58.177892 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-ea717b7b-25bd-48b5-91a1-fc7fc8e96cd8 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: 126f561c-163f-4f6d-8be1-98a2e29b0909] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:12:58.185399 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ea717b7b-25bd-48b5-91a1-fc7fc8e96cd8 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:58.185855 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ea717b7b-25bd-48b5-91a1-fc7fc8e96cd8 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:58.186903 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ea717b7b-25bd-48b5-91a1-fc7fc8e96cd8 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:58.187364 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ea717b7b-25bd-48b5-91a1-fc7fc8e96cd8 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:58.562610 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.network.neutron [None req-65893770-0ddf-4193-9977-099101afec10 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] [instance: 8020f553-6025-4f19-835c-c00e2fb4f764] Updating instance_info_cache with network_info: [{"id": "05294e88-e71a-4faa-8acc-79137ca6ee05", "address": "fa:16:3e:30:6c:72", "network": {"id": "9b8d8e4f-14e5-4e94-bfa5-2387fb1adcf8", "bridge": null, "label": "tempest-FloatingIPsAssociationTestJSON-379677821-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "172.20.0.2"}}], "meta": {"injected": false, "tenant_id": "cda86413e74b4c1e8e09bd4df4926664", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false, "bound_drivers": {"0": "hyperv"}}, "devname": "tap05294e88-e7", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=138681) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:117}} Jul 16 21:12:58.662479 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-65893770-0ddf-4193-9977-099101afec10 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Releasing lock "refresh_cache-8020f553-6025-4f19-835c-c00e2fb4f764" {{(pid=138681) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:312}} Jul 16 21:12:58.663726 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-65893770-0ddf-4193-9977-099101afec10 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] 10.222.0.16 "POST /compute/v2.1/servers/8020f553-6025-4f19-835c-c00e2fb4f764/action" status: 202 len: 0 microversion: 2.1 time: 5.924236 Jul 16 21:12:58.665116 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 605/1286] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 21:12:52 2022] POST /compute/v2.1/servers/8020f553-6025-4f19-835c-c00e2fb4f764/action => generated 0 bytes in 5926 msecs (HTTP/1.1 202) 9 headers in 369 bytes (1 switches on core 0) Jul 16 21:12:58.675371 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-fe345820-d257-44a2-873d-0e564813ff55 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:58.675371 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-fe345820-d257-44a2-873d-0e564813ff55 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] [instance: 08e7e1f9-8b5c-446c-a05f-d8043473dd35] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:12:58.685147 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fe345820-d257-44a2-873d-0e564813ff55 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:58.687484 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fe345820-d257-44a2-873d-0e564813ff55 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:58.688766 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fe345820-d257-44a2-873d-0e564813ff55 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:58.689393 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fe345820-d257-44a2-873d-0e564813ff55 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:59.026347 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-ea717b7b-25bd-48b5-91a1-fc7fc8e96cd8 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b68dacc4b8463ba07fae6372131117', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55a19b7a4682457a9132f5fc80110712', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:59.034827 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-ea717b7b-25bd-48b5-91a1-fc7fc8e96cd8 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "GET /compute/v2.1/servers/126f561c-163f-4f6d-8be1-98a2e29b0909" status: 200 len: 1472 microversion: 2.1 time: 0.861378 Jul 16 21:12:59.042398 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 682/1287] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:12:58 2022] GET /compute/v2.1/servers/126f561c-163f-4f6d-8be1-98a2e29b0909 => generated 1472 bytes in 863 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:12:59.042398 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-88ae04cd-8fd5-4381-a9a1-9412b377d3ca req-168aa588-ecca-417e-9b50-2a25e74e7990 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "08e7e1f9-8b5c-446c-a05f-d8043473dd35", "name": "network-vif-plugged", "status": "completed", "tag": "b21352ed-bfe1-4d3f-93ff-6f5027326093"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:12:59.052469 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-88ae04cd-8fd5-4381-a9a1-9412b377d3ca req-168aa588-ecca-417e-9b50-2a25e74e7990 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:59.053351 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-88ae04cd-8fd5-4381-a9a1-9412b377d3ca req-168aa588-ecca-417e-9b50-2a25e74e7990 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:59.071141 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-88ae04cd-8fd5-4381-a9a1-9412b377d3ca req-168aa588-ecca-417e-9b50-2a25e74e7990 service nova] Creating event network-vif-plugged:b21352ed-bfe1-4d3f-93ff-6f5027326093 for instance 08e7e1f9-8b5c-446c-a05f-d8043473dd35 on nch1-master-1 Jul 16 21:12:59.076455 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-88ae04cd-8fd5-4381-a9a1-9412b377d3ca req-168aa588-ecca-417e-9b50-2a25e74e7990 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.039613 Jul 16 21:12:59.077067 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 683/1288] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:12:59 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:12:59.083423 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-df84ee79-06ea-4987-8f03-9f85c6f2ecda tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:59.083592 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-df84ee79-06ea-4987-8f03-9f85c6f2ecda tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] [instance: 74a19091-819b-4cf0-8109-fbb0de65313c] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:12:59.090440 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-df84ee79-06ea-4987-8f03-9f85c6f2ecda tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:59.091253 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-df84ee79-06ea-4987-8f03-9f85c6f2ecda tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:59.092257 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-df84ee79-06ea-4987-8f03-9f85c6f2ecda tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:59.092909 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-df84ee79-06ea-4987-8f03-9f85c6f2ecda tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:59.109538 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-df84ee79-06ea-4987-8f03-9f85c6f2ecda tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] HTTP exception thrown: Instance 74a19091-819b-4cf0-8109-fbb0de65313c could not be found. Jul 16 21:12:59.111841 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-df84ee79-06ea-4987-8f03-9f85c6f2ecda tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Request method failure captured: Jul 16 21:12:59.111841 ncdv-master-1 devstack@n-api.service[138680]: request: DELETE /compute/v2.1/servers/74a19091-819b-4cf0-8109-fbb0de65313c HTTP/1.1 Jul 16 21:12:59.111841 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:12:59.111841 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:12:59.111841 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:12:59.111841 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 0 Jul 16 21:12:59.111841 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:12:59.111841 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:12:59.111841 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:12:59.111841 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:12:59.111841 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0yme2OmgPmghOZxTYv1prM8I30jqdYxV66dKB7kNvJ87HPySbthZ6En_3VLO5oCbOf2UuM1ARwLRB17-5AINJPTiqPrtco0L2oHyrrsd2HBbkXlr_mLFUNTutX_YwODzB_43ujJc9IK-SLZSWvs3yKGjofUk2kkT8-qMrEcv7y0wtas Jul 16 21:12:59.111841 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:12:59.111841 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:12:59.111841 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:12:59.111841 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:12:59.111841 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:12:59.111841 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:12:59.111841 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: 7d2b583c48254e17a8082f6467f2ae6d Jul 16 21:12:59.111841 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-DeleteServersAdminTestJSON-69305867 Jul 16 21:12:59.111841 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:12:59.111841 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:12:59.113172 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/7d2b583c48254e17a8082f6467f2ae6d"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/7d2b583c48254e17a8082f6467f2ae6d"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_7d2b583c48254e17a8082f6467f2ae6d"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/7d2b583c48254e17a8082f6467f2ae6d"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:12:59.113172 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-DeleteServersAdminTestJSON-69305867 Jul 16 21:12:59.113172 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: 7d2b583c48254e17a8082f6467f2ae6d Jul 16 21:12:59.113172 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-DeleteServersAdminTestJSON-69305867 Jul 16 21:12:59.113172 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-DeleteServersAdminTestJSON-69305867-project Jul 16 21:12:59.113172 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:12:59.113172 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:12:59.113172 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: 56d4bbcdebc24a34a814d7f6e72fbcfd Jul 16 21:12:59.113172 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-DeleteServersAdminTestJSON-69305867-project Jul 16 21:12:59.113172 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 21:12:59.113172 ncdv-master-1 devstack@n-api.service[138680]: exception: Instance 74a19091-819b-4cf0-8109-fbb0de65313c could not be found. Jul 16 21:12:59.113172 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': '74a19091-819b-4cf0-8109-fbb0de65313c'} Jul 16 21:12:59.113892 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:12:59.113892 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:12:59.113892 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:12:59.113892 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:12:59.113892 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:12:59.113892 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:12:59.113892 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:12:59.113892 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:12:59.113892 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:12:59.113892 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:12:59.113892 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 74a19091-819b-4cf0-8109-fbb0de65313c could not be found. Jul 16 21:12:59.113892 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:12:59.113892 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:12:59.113892 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:12:59.113892 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:12:59.113892 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:12:59.113892 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:12:59.113892 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:12:59.113892 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:12:59.113892 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:12:59.115072 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:12:59.115072 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Jul 16 21:12:59.115072 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Jul 16 21:12:59.115072 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Jul 16 21:12:59.115072 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Jul 16 21:12:59.115072 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:12:59.115072 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:12:59.115072 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:12:59.115072 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:12:59.115072 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 74a19091-819b-4cf0-8109-fbb0de65313c could not be found. Jul 16 21:12:59.115072 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:12:59.115072 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:12:59.115072 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:12:59.115072 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:12:59.115072 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:12:59.115072 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:12:59.115072 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:12:59.115072 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:12:59.115072 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 74a19091-819b-4cf0-8109-fbb0de65313c could not be found. Jul 16 21:12:59.115072 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:12:59.116286 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-df84ee79-06ea-4987-8f03-9f85c6f2ecda tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Returning 404 to user: Instance 74a19091-819b-4cf0-8109-fbb0de65313c could not be found. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:12:59.117777 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-df84ee79-06ea-4987-8f03-9f85c6f2ecda tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] 10.222.0.16 "DELETE /compute/v2.1/servers/74a19091-819b-4cf0-8109-fbb0de65313c" status: 404 len: 111 microversion: 2.1 time: 0.038553 Jul 16 21:12:59.119080 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 684/1289] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:12:59 2022] DELETE /compute/v2.1/servers/74a19091-819b-4cf0-8109-fbb0de65313c => generated 111 bytes in 40 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:12:59.126158 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-00a87659-47da-4173-844a-bb85b8c43c4b tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:59.131111 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-00a87659-47da-4173-844a-bb85b8c43c4b tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] Policy check for os_compute_api:servers:allow_all_filters failed with credentials {'is_admin': False, 'user_id': '034c11cc3f174c6fb44c8d64e55cf8b9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb4ff5d7e42940cdb7be71dadd895ee4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:59.132294 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-00a87659-47da-4173-844a-bb85b8c43c4b tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] Searching by: {'deleted': False, 'project_id': 'fb4ff5d7e42940cdb7be71dadd895ee4'} {{(pid=138680) get_all /opt/stack/nova/nova/compute/api.py:2962}} Jul 16 21:12:59.140750 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-00a87659-47da-4173-844a-bb85b8c43c4b tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:59.143377 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-00a87659-47da-4173-844a-bb85b8c43c4b tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:59.144259 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-00a87659-47da-4173-844a-bb85b8c43c4b tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:59.144770 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-00a87659-47da-4173-844a-bb85b8c43c4b tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:59.243541 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.multi_cell_list [None req-00a87659-47da-4173-844a-bb85b8c43c4b tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] Listed batch of 1 results from cell out of 1000 limit. Returned 1 total so far. {{(pid=138680) do_query /opt/stack/nova/nova/compute/multi_cell_list.py:373}} Jul 16 21:12:59.330621 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-00a87659-47da-4173-844a-bb85b8c43c4b tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '034c11cc3f174c6fb44c8d64e55cf8b9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb4ff5d7e42940cdb7be71dadd895ee4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:59.338735 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-00a87659-47da-4173-844a-bb85b8c43c4b tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:59.339211 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-00a87659-47da-4173-844a-bb85b8c43c4b tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:59.419378 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-fe345820-d257-44a2-873d-0e564813ff55 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '52281b3989a04ca2a50b0bc93bf783f0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '30d028fa9b854c8fb71c8cbed2985c27', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:59.424250 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-fe345820-d257-44a2-873d-0e564813ff55 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] 10.222.0.16 "GET /compute/v2.1/servers/08e7e1f9-8b5c-446c-a05f-d8043473dd35" status: 200 len: 1318 microversion: 2.1 time: 0.757765 Jul 16 21:12:59.425327 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 606/1290] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:12:58 2022] GET /compute/v2.1/servers/08e7e1f9-8b5c-446c-a05f-d8043473dd35 => generated 1318 bytes in 759 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:12:59.431140 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-1730289a-8d8b-48c9-beba-23f87d4e718f tempest-TenantUsagesTestJSON-260059616 tempest-TenantUsagesTestJSON-260059616-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:59.431985 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-1730289a-8d8b-48c9-beba-23f87d4e718f tempest-TenantUsagesTestJSON-260059616 tempest-TenantUsagesTestJSON-260059616-project] [instance: 6e68dbd6-1d5c-471e-ad69-cf1c4aa4004a] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:12:59.438694 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1730289a-8d8b-48c9-beba-23f87d4e718f tempest-TenantUsagesTestJSON-260059616 tempest-TenantUsagesTestJSON-260059616-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:59.440093 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1730289a-8d8b-48c9-beba-23f87d4e718f tempest-TenantUsagesTestJSON-260059616 tempest-TenantUsagesTestJSON-260059616-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:59.440882 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1730289a-8d8b-48c9-beba-23f87d4e718f tempest-TenantUsagesTestJSON-260059616 tempest-TenantUsagesTestJSON-260059616-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:59.441324 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1730289a-8d8b-48c9-beba-23f87d4e718f tempest-TenantUsagesTestJSON-260059616 tempest-TenantUsagesTestJSON-260059616-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:59.634406 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-00a87659-47da-4173-844a-bb85b8c43c4b tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] 10.222.0.16 "GET /compute/v2.1/servers/detail" status: 200 len: 1697 microversion: 2.1 time: 0.513964 Jul 16 21:12:59.635527 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 685/1291] 10.222.0.16 () {58 vars in 1145 bytes} [Sat Jul 16 21:12:59 2022] GET /compute/v2.1/servers/detail => generated 1697 bytes in 516 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:12:59.640864 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-3a3f7adf-d25a-4592-8733-d65931d5f2f5 req-5b6135c2-529e-431b-b70a-e716dc7baae5 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "8020f553-6025-4f19-835c-c00e2fb4f764", "tag": "05294e88-e71a-4faa-8acc-79137ca6ee05"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:12:59.679627 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-1730289a-8d8b-48c9-beba-23f87d4e718f tempest-TenantUsagesTestJSON-260059616 tempest-TenantUsagesTestJSON-260059616-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2db871d6aa9746d2b660efbfbdb1dc75', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cad2f07400d54ef9825022d5b3489db0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:12:59.693081 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-1730289a-8d8b-48c9-beba-23f87d4e718f tempest-TenantUsagesTestJSON-260059616 tempest-TenantUsagesTestJSON-260059616-project] 10.222.0.16 "GET /compute/v2.1/servers/6e68dbd6-1d5c-471e-ad69-cf1c4aa4004a" status: 200 len: 1314 microversion: 2.1 time: 0.266111 Jul 16 21:12:59.694063 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 607/1292] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:12:59 2022] GET /compute/v2.1/servers/6e68dbd6-1d5c-471e-ad69-cf1c4aa4004a => generated 1314 bytes in 268 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:12:59.703553 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-1186aa42-e8cb-48ba-b2c4-8bdc2a18c866 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:59.704474 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-1186aa42-e8cb-48ba-b2c4-8bdc2a18c866 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] [instance: 4ae3e944-b450-40e5-bb71-c02df03212e3] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:12:59.716347 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1186aa42-e8cb-48ba-b2c4-8bdc2a18c866 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:59.716835 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1186aa42-e8cb-48ba-b2c4-8bdc2a18c866 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:59.717686 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1186aa42-e8cb-48ba-b2c4-8bdc2a18c866 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:59.718112 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1186aa42-e8cb-48ba-b2c4-8bdc2a18c866 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:59.734903 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-1186aa42-e8cb-48ba-b2c4-8bdc2a18c866 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] HTTP exception thrown: Instance 4ae3e944-b450-40e5-bb71-c02df03212e3 could not be found. Jul 16 21:12:59.736596 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-1186aa42-e8cb-48ba-b2c4-8bdc2a18c866 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Request method failure captured: Jul 16 21:12:59.736596 ncdv-master-1 devstack@n-api.service[138681]: request: DELETE /compute/v2.1/servers/4ae3e944-b450-40e5-bb71-c02df03212e3 HTTP/1.1 Jul 16 21:12:59.736596 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:12:59.736596 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:12:59.736596 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:12:59.736596 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 21:12:59.736596 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:12:59.736596 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:12:59.736596 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:12:59.736596 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:12:59.736596 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0ymKM_ECvjVkk6e7K1WKEy-tkQQ7EHRMB-Urg2_4RVouK1u8YOWROAshpDCUoNKJCmCo4JxDMnUkTw8ca8yVeRAtm7eAKieqo3ZQzYMt8n410aWYlQFenytzNBNAOHZnquy4qrFWM3_CaMN-nzEw34w9P4vt5Jr8OZ2O_l_ChXunM28 Jul 16 21:12:59.736596 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:12:59.736596 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:12:59.736596 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:12:59.736596 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:12:59.736596 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:12:59.736596 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:12:59.736596 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: cda86413e74b4c1e8e09bd4df4926664 Jul 16 21:12:59.736596 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-FloatingIPsAssociationTestJSON-1179742205 Jul 16 21:12:59.736596 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:12:59.736596 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:12:59.738183 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/cda86413e74b4c1e8e09bd4df4926664"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/cda86413e74b4c1e8e09bd4df4926664"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_cda86413e74b4c1e8e09bd4df4926664"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/cda86413e74b4c1e8e09bd4df4926664"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:12:59.738183 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-FloatingIPsAssociationTestJSON-1179742205 Jul 16 21:12:59.738183 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: cda86413e74b4c1e8e09bd4df4926664 Jul 16 21:12:59.738183 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-FloatingIPsAssociationTestJSON-1179742205 Jul 16 21:12:59.738183 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-FloatingIPsAssociationTestJSON-1179742205-project Jul 16 21:12:59.738183 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:12:59.738183 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:12:59.738183 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: 2dfcad8d2b57481bb33aa8f5d7483c51 Jul 16 21:12:59.738183 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-FloatingIPsAssociationTestJSON-1179742205-project Jul 16 21:12:59.738183 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:12:59.738183 ncdv-master-1 devstack@n-api.service[138681]: exception: Instance 4ae3e944-b450-40e5-bb71-c02df03212e3 could not be found. Jul 16 21:12:59.738183 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': '4ae3e944-b450-40e5-bb71-c02df03212e3'} Jul 16 21:12:59.739248 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:12:59.739248 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:12:59.739248 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:12:59.739248 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:12:59.739248 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:12:59.739248 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:12:59.739248 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:12:59.739248 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:12:59.739248 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:12:59.739248 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:12:59.739248 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 4ae3e944-b450-40e5-bb71-c02df03212e3 could not be found. Jul 16 21:12:59.739248 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:12:59.739248 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:12:59.739248 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:12:59.739248 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:12:59.739248 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:12:59.739248 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:12:59.739248 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:12:59.739248 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:12:59.739248 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:12:59.740472 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:12:59.740472 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Jul 16 21:12:59.740472 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Jul 16 21:12:59.740472 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Jul 16 21:12:59.740472 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Jul 16 21:12:59.740472 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:12:59.740472 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:12:59.740472 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:12:59.740472 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:12:59.740472 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 4ae3e944-b450-40e5-bb71-c02df03212e3 could not be found. Jul 16 21:12:59.740472 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:12:59.740472 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:12:59.740472 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:12:59.740472 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:12:59.740472 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:12:59.740472 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:12:59.740472 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:12:59.740472 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:12:59.740472 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 4ae3e944-b450-40e5-bb71-c02df03212e3 could not be found. Jul 16 21:12:59.740472 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:12:59.741628 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-1186aa42-e8cb-48ba-b2c4-8bdc2a18c866 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Returning 404 to user: Instance 4ae3e944-b450-40e5-bb71-c02df03212e3 could not be found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:12:59.741628 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-1186aa42-e8cb-48ba-b2c4-8bdc2a18c866 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] 10.222.0.16 "DELETE /compute/v2.1/servers/4ae3e944-b450-40e5-bb71-c02df03212e3" status: 404 len: 111 microversion: 2.1 time: 0.042081 Jul 16 21:12:59.741628 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 608/1293] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:12:59 2022] DELETE /compute/v2.1/servers/4ae3e944-b450-40e5-bb71-c02df03212e3 => generated 111 bytes in 44 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:12:59.744690 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-24c1a256-e718-48f6-943b-7d22ac80da95 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:59.745601 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-24c1a256-e718-48f6-943b-7d22ac80da95 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] [instance: 6a391156-ffce-49c0-a79d-47b4544264f5] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:12:59.760539 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-24c1a256-e718-48f6-943b-7d22ac80da95 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:59.765359 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-24c1a256-e718-48f6-943b-7d22ac80da95 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:59.766505 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-24c1a256-e718-48f6-943b-7d22ac80da95 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:59.770074 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-24c1a256-e718-48f6-943b-7d22ac80da95 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:59.873264 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-3a3f7adf-d25a-4592-8733-d65931d5f2f5 req-5b6135c2-529e-431b-b70a-e716dc7baae5 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:59.873920 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-3a3f7adf-d25a-4592-8733-d65931d5f2f5 req-5b6135c2-529e-431b-b70a-e716dc7baae5 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:59.901158 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-3a3f7adf-d25a-4592-8733-d65931d5f2f5 req-5b6135c2-529e-431b-b70a-e716dc7baae5 service nova] Creating event network-changed:05294e88-e71a-4faa-8acc-79137ca6ee05 for instance 8020f553-6025-4f19-835c-c00e2fb4f764 on nch1-master-1 Jul 16 21:12:59.909300 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-3a3f7adf-d25a-4592-8733-d65931d5f2f5 req-5b6135c2-529e-431b-b70a-e716dc7baae5 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.272213 Jul 16 21:12:59.911417 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 686/1294] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:12:59 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 274 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:12:59.918816 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-8210106e-ebeb-4a83-b20f-a9e9db318693 tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:12:59.919811 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-8210106e-ebeb-4a83-b20f-a9e9db318693 tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] [instance: 74a19091-819b-4cf0-8109-fbb0de65313c] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:12:59.927176 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8210106e-ebeb-4a83-b20f-a9e9db318693 tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:59.928530 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8210106e-ebeb-4a83-b20f-a9e9db318693 tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:59.931042 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8210106e-ebeb-4a83-b20f-a9e9db318693 tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:59.932024 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8210106e-ebeb-4a83-b20f-a9e9db318693 tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:12:59.962841 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-8210106e-ebeb-4a83-b20f-a9e9db318693 tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] HTTP exception thrown: Instance 74a19091-819b-4cf0-8109-fbb0de65313c could not be found. Jul 16 21:12:59.964663 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-8210106e-ebeb-4a83-b20f-a9e9db318693 tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Request method failure captured: Jul 16 21:12:59.964663 ncdv-master-1 devstack@n-api.service[138680]: request: GET /compute/v2.1/servers/74a19091-819b-4cf0-8109-fbb0de65313c HTTP/1.1 Jul 16 21:12:59.964663 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:12:59.964663 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:12:59.964663 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:12:59.964663 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 0 Jul 16 21:12:59.964663 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:12:59.964663 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:12:59.964663 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:12:59.964663 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:12:59.964663 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0yme2OmgPmghOZxTYv1prM8I30jqdYxV66dKB7kNvJ87HPySbthZ6En_3VLO5oCbOf2UuM1ARwLRB17-5AINJPTiqPrtco0L2oHyrrsd2HBbkXlr_mLFUNTutX_YwODzB_43ujJc9IK-SLZSWvs3yKGjofUk2kkT8-qMrEcv7y0wtas Jul 16 21:12:59.964663 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:12:59.964663 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:12:59.964663 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:12:59.964663 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:12:59.964663 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:12:59.964663 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:12:59.964663 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: 7d2b583c48254e17a8082f6467f2ae6d Jul 16 21:12:59.964663 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-DeleteServersAdminTestJSON-69305867 Jul 16 21:12:59.964663 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:12:59.964663 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:12:59.966055 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/7d2b583c48254e17a8082f6467f2ae6d"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/7d2b583c48254e17a8082f6467f2ae6d"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_7d2b583c48254e17a8082f6467f2ae6d"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/7d2b583c48254e17a8082f6467f2ae6d"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:12:59.966055 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-DeleteServersAdminTestJSON-69305867 Jul 16 21:12:59.966055 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: 7d2b583c48254e17a8082f6467f2ae6d Jul 16 21:12:59.966055 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-DeleteServersAdminTestJSON-69305867 Jul 16 21:12:59.966055 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-DeleteServersAdminTestJSON-69305867-project Jul 16 21:12:59.966055 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:12:59.966055 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:12:59.966055 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: 56d4bbcdebc24a34a814d7f6e72fbcfd Jul 16 21:12:59.966055 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-DeleteServersAdminTestJSON-69305867-project Jul 16 21:12:59.966055 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 21:12:59.966055 ncdv-master-1 devstack@n-api.service[138680]: exception: Instance 74a19091-819b-4cf0-8109-fbb0de65313c could not be found. Jul 16 21:12:59.966055 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': '74a19091-819b-4cf0-8109-fbb0de65313c'} Jul 16 21:12:59.967079 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:12:59.967079 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:12:59.967079 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:12:59.967079 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:12:59.967079 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:12:59.967079 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:12:59.967079 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:12:59.967079 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:12:59.967079 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:12:59.967079 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:12:59.967079 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 74a19091-819b-4cf0-8109-fbb0de65313c could not be found. Jul 16 21:12:59.967079 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:12:59.967079 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:12:59.967079 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:12:59.967079 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:12:59.967079 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:12:59.967079 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:12:59.967079 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:12:59.967079 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:12:59.967079 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:12:59.968364 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:12:59.968364 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:12:59.968364 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:12:59.968364 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:12:59.968364 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:12:59.968364 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:12:59.968364 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:12:59.968364 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 74a19091-819b-4cf0-8109-fbb0de65313c could not be found. Jul 16 21:12:59.968364 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:12:59.968364 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:12:59.968364 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:12:59.968364 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:12:59.968364 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:12:59.968364 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:12:59.968364 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:12:59.968364 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:12:59.968364 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 74a19091-819b-4cf0-8109-fbb0de65313c could not be found. Jul 16 21:12:59.968364 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:12:59.971742 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-8210106e-ebeb-4a83-b20f-a9e9db318693 tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Returning 404 to user: Instance 74a19091-819b-4cf0-8109-fbb0de65313c could not be found. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:12:59.972626 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-8210106e-ebeb-4a83-b20f-a9e9db318693 tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] 10.222.0.16 "GET /compute/v2.1/servers/74a19091-819b-4cf0-8109-fbb0de65313c" status: 404 len: 111 microversion: 2.1 time: 0.060011 Jul 16 21:12:59.976253 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 687/1295] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:12:59 2022] GET /compute/v2.1/servers/74a19091-819b-4cf0-8109-fbb0de65313c => generated 111 bytes in 61 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:12:59.977914 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-c4102e37-5683-4dd5-8ce0-25f76d3e0e5d req-e07e91f6-927f-47fa-87d3-82ed92d20eb3 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "3c5ab09e-0a97-4cfc-b249-a11ab221b5d2", "tag": "4f9dfe87-5bed-49a6-8b89-464c4930e746"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:12:59.994227 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-c4102e37-5683-4dd5-8ce0-25f76d3e0e5d req-e07e91f6-927f-47fa-87d3-82ed92d20eb3 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:12:59.994588 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-c4102e37-5683-4dd5-8ce0-25f76d3e0e5d req-e07e91f6-927f-47fa-87d3-82ed92d20eb3 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:00.022744 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-c4102e37-5683-4dd5-8ce0-25f76d3e0e5d req-e07e91f6-927f-47fa-87d3-82ed92d20eb3 service nova] Creating event network-changed:4f9dfe87-5bed-49a6-8b89-464c4930e746 for instance 3c5ab09e-0a97-4cfc-b249-a11ab221b5d2 on nch2-master-1 Jul 16 21:13:00.035879 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-c4102e37-5683-4dd5-8ce0-25f76d3e0e5d req-e07e91f6-927f-47fa-87d3-82ed92d20eb3 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.060254 Jul 16 21:13:00.038678 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 688/1296] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:12:59 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 65 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:13:00.051136 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-5f8aa2c4-e6f4-4d6e-a564-a7e0a1fe9495 tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:13:00.082335 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-5f8aa2c4-e6f4-4d6e-a564-a7e0a1fe9495 tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] 10.222.0.16 "GET /compute/v2.1/flavors/452" status: 200 len: 370 microversion: 2.1 time: 0.021873 Jul 16 21:13:00.082335 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 689/1297] 10.222.0.16 () {58 vars in 1136 bytes} [Sat Jul 16 21:13:00 2022] GET /compute/v2.1/flavors/452 => generated 370 bytes in 24 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:13:00.082335 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-55de063b-d7a8-4b15-9b4f-7d824c583e43 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:13:00.082335 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-55de063b-d7a8-4b15-9b4f-7d824c583e43 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] [instance: 4ae3e944-b450-40e5-bb71-c02df03212e3] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:13:00.082707 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-24c1a256-e718-48f6-943b-7d22ac80da95 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a69f9dee03d43beaafa966216b9e289', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9b769bf8f3ea4e368bf68de1df6538f9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:13:00.084683 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-55de063b-d7a8-4b15-9b4f-7d824c583e43 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:00.087860 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-55de063b-d7a8-4b15-9b4f-7d824c583e43 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:00.088828 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-55de063b-d7a8-4b15-9b4f-7d824c583e43 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:00.089277 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-55de063b-d7a8-4b15-9b4f-7d824c583e43 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:00.091267 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-24c1a256-e718-48f6-943b-7d22ac80da95 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] 10.222.0.16 "GET /compute/v2.1/servers/6a391156-ffce-49c0-a79d-47b4544264f5" status: 200 len: 1707 microversion: 2.1 time: 0.350854 Jul 16 21:13:00.100304 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 609/1298] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:12:59 2022] GET /compute/v2.1/servers/6a391156-ffce-49c0-a79d-47b4544264f5 => generated 1707 bytes in 360 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:13:00.105090 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-55de063b-d7a8-4b15-9b4f-7d824c583e43 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] HTTP exception thrown: Instance 4ae3e944-b450-40e5-bb71-c02df03212e3 could not be found. Jul 16 21:13:00.106925 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-55de063b-d7a8-4b15-9b4f-7d824c583e43 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Request method failure captured: Jul 16 21:13:00.106925 ncdv-master-1 devstack@n-api.service[138680]: request: GET /compute/v2.1/servers/4ae3e944-b450-40e5-bb71-c02df03212e3 HTTP/1.1 Jul 16 21:13:00.106925 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:13:00.106925 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:13:00.106925 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:13:00.106925 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 0 Jul 16 21:13:00.106925 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:13:00.106925 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:13:00.106925 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:13:00.106925 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:13:00.106925 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0ymKM_ECvjVkk6e7K1WKEy-tkQQ7EHRMB-Urg2_4RVouK1u8YOWROAshpDCUoNKJCmCo4JxDMnUkTw8ca8yVeRAtm7eAKieqo3ZQzYMt8n410aWYlQFenytzNBNAOHZnquy4qrFWM3_CaMN-nzEw34w9P4vt5Jr8OZ2O_l_ChXunM28 Jul 16 21:13:00.106925 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:13:00.106925 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:13:00.106925 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:13:00.106925 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:13:00.106925 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:13:00.106925 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:13:00.106925 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: cda86413e74b4c1e8e09bd4df4926664 Jul 16 21:13:00.106925 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-FloatingIPsAssociationTestJSON-1179742205 Jul 16 21:13:00.106925 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:13:00.106925 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:13:00.108502 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/cda86413e74b4c1e8e09bd4df4926664"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/cda86413e74b4c1e8e09bd4df4926664"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_cda86413e74b4c1e8e09bd4df4926664"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/cda86413e74b4c1e8e09bd4df4926664"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:13:00.108502 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-FloatingIPsAssociationTestJSON-1179742205 Jul 16 21:13:00.108502 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: cda86413e74b4c1e8e09bd4df4926664 Jul 16 21:13:00.108502 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-FloatingIPsAssociationTestJSON-1179742205 Jul 16 21:13:00.108502 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-FloatingIPsAssociationTestJSON-1179742205-project Jul 16 21:13:00.108502 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:13:00.108502 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:13:00.108502 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: 2dfcad8d2b57481bb33aa8f5d7483c51 Jul 16 21:13:00.108502 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-FloatingIPsAssociationTestJSON-1179742205-project Jul 16 21:13:00.108502 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 21:13:00.108502 ncdv-master-1 devstack@n-api.service[138680]: exception: Instance 4ae3e944-b450-40e5-bb71-c02df03212e3 could not be found. Jul 16 21:13:00.108502 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': '4ae3e944-b450-40e5-bb71-c02df03212e3'} Jul 16 21:13:00.109343 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:13:00.109343 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:13:00.109343 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:13:00.109343 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:13:00.109343 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:13:00.109343 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:13:00.109343 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:13:00.109343 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:13:00.109343 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:13:00.109343 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:13:00.109343 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 4ae3e944-b450-40e5-bb71-c02df03212e3 could not be found. Jul 16 21:13:00.109343 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:13:00.109343 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:13:00.109343 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:13:00.109343 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:13:00.109343 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:13:00.109343 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:13:00.109343 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:13:00.109343 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:13:00.109343 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:13:00.110763 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:13:00.110763 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:13:00.110763 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:13:00.110763 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:13:00.110763 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:13:00.110763 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:13:00.110763 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:13:00.110763 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 4ae3e944-b450-40e5-bb71-c02df03212e3 could not be found. Jul 16 21:13:00.110763 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:13:00.110763 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:13:00.110763 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:13:00.110763 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:13:00.110763 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:13:00.110763 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:13:00.110763 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:13:00.110763 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:13:00.110763 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 4ae3e944-b450-40e5-bb71-c02df03212e3 could not be found. Jul 16 21:13:00.110763 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:13:00.112131 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-55de063b-d7a8-4b15-9b4f-7d824c583e43 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Returning 404 to user: Instance 4ae3e944-b450-40e5-bb71-c02df03212e3 could not be found. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:13:00.112131 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-55de063b-d7a8-4b15-9b4f-7d824c583e43 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] 10.222.0.16 "GET /compute/v2.1/servers/4ae3e944-b450-40e5-bb71-c02df03212e3" status: 404 len: 111 microversion: 2.1 time: 0.044582 Jul 16 21:13:00.112276 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 690/1299] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:13:00 2022] GET /compute/v2.1/servers/4ae3e944-b450-40e5-bb71-c02df03212e3 => generated 111 bytes in 49 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:13:00.113662 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-ace787d2-204b-4855-b3ca-4ff08b74d0a5 tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:13:00.115202 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-ace787d2-204b-4855-b3ca-4ff08b74d0a5 tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] [instance: 83b1fe77-ef6d-408d-b255-aa3b60aad8f3] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:13:00.120762 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-329b1cd8-0cff-4b4b-ae0f-3f8767a4d1fc tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:13:00.121325 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-329b1cd8-0cff-4b4b-ae0f-3f8767a4d1fc tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: 126f561c-163f-4f6d-8be1-98a2e29b0909] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:13:00.125392 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ace787d2-204b-4855-b3ca-4ff08b74d0a5 tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:00.125950 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ace787d2-204b-4855-b3ca-4ff08b74d0a5 tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:00.126823 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ace787d2-204b-4855-b3ca-4ff08b74d0a5 tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:00.127382 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ace787d2-204b-4855-b3ca-4ff08b74d0a5 tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:00.138000 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-329b1cd8-0cff-4b4b-ae0f-3f8767a4d1fc tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:00.138247 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-329b1cd8-0cff-4b4b-ae0f-3f8767a4d1fc tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:00.139258 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-329b1cd8-0cff-4b4b-ae0f-3f8767a4d1fc tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:00.139700 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-329b1cd8-0cff-4b4b-ae0f-3f8767a4d1fc tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:00.147415 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-ace787d2-204b-4855-b3ca-4ff08b74d0a5 tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] HTTP exception thrown: Instance 83b1fe77-ef6d-408d-b255-aa3b60aad8f3 could not be found. Jul 16 21:13:00.149618 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-ace787d2-204b-4855-b3ca-4ff08b74d0a5 tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Request method failure captured: Jul 16 21:13:00.149618 ncdv-master-1 devstack@n-api.service[138681]: request: DELETE /compute/v2.1/servers/83b1fe77-ef6d-408d-b255-aa3b60aad8f3 HTTP/1.1 Jul 16 21:13:00.149618 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:13:00.149618 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:13:00.149618 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:13:00.149618 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 21:13:00.149618 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:13:00.149618 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:13:00.149618 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:13:00.149618 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:13:00.149618 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0yme2OmgPmghOZxTYv1prM8I30jqdYxV66dKB7kNvJ87HPySbthZ6En_3VLO5oCbOf2UuM1ARwLRB17-5AINJPTiqPrtco0L2oHyrrsd2HBbkXlr_mLFUNTutX_YwODzB_43ujJc9IK-SLZSWvs3yKGjofUk2kkT8-qMrEcv7y0wtas Jul 16 21:13:00.149618 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:13:00.149618 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:13:00.149618 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:13:00.149618 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:13:00.149618 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:13:00.149618 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:13:00.149618 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: 7d2b583c48254e17a8082f6467f2ae6d Jul 16 21:13:00.149618 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-DeleteServersAdminTestJSON-69305867 Jul 16 21:13:00.149618 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:13:00.149618 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:13:00.151419 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/7d2b583c48254e17a8082f6467f2ae6d"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/7d2b583c48254e17a8082f6467f2ae6d"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_7d2b583c48254e17a8082f6467f2ae6d"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/7d2b583c48254e17a8082f6467f2ae6d"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:13:00.151419 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-DeleteServersAdminTestJSON-69305867 Jul 16 21:13:00.151419 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: 7d2b583c48254e17a8082f6467f2ae6d Jul 16 21:13:00.151419 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-DeleteServersAdminTestJSON-69305867 Jul 16 21:13:00.151419 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-DeleteServersAdminTestJSON-69305867-project Jul 16 21:13:00.151419 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:13:00.151419 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:13:00.151419 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: 56d4bbcdebc24a34a814d7f6e72fbcfd Jul 16 21:13:00.151419 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-DeleteServersAdminTestJSON-69305867-project Jul 16 21:13:00.151419 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:13:00.151419 ncdv-master-1 devstack@n-api.service[138681]: exception: Instance 83b1fe77-ef6d-408d-b255-aa3b60aad8f3 could not be found. Jul 16 21:13:00.151419 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': '83b1fe77-ef6d-408d-b255-aa3b60aad8f3'} Jul 16 21:13:00.152243 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:13:00.152243 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:13:00.152243 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:13:00.152243 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:13:00.152243 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:13:00.152243 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:13:00.152243 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:13:00.152243 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:13:00.152243 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:13:00.152243 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:13:00.152243 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 83b1fe77-ef6d-408d-b255-aa3b60aad8f3 could not be found. Jul 16 21:13:00.152243 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:13:00.152243 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:13:00.152243 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:13:00.152243 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:13:00.152243 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:13:00.152243 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:13:00.152243 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:13:00.152243 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:13:00.152243 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:13:00.153671 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:13:00.153671 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Jul 16 21:13:00.153671 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Jul 16 21:13:00.153671 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Jul 16 21:13:00.153671 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Jul 16 21:13:00.153671 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:13:00.153671 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:13:00.153671 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:13:00.153671 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:13:00.153671 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 83b1fe77-ef6d-408d-b255-aa3b60aad8f3 could not be found. Jul 16 21:13:00.153671 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:13:00.153671 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:13:00.153671 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:13:00.153671 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:13:00.153671 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:13:00.153671 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:13:00.153671 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:13:00.153671 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:13:00.153671 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 83b1fe77-ef6d-408d-b255-aa3b60aad8f3 could not be found. Jul 16 21:13:00.153671 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:13:00.157232 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-ace787d2-204b-4855-b3ca-4ff08b74d0a5 tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Returning 404 to user: Instance 83b1fe77-ef6d-408d-b255-aa3b60aad8f3 could not be found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:13:00.158356 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-ace787d2-204b-4855-b3ca-4ff08b74d0a5 tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] 10.222.0.16 "DELETE /compute/v2.1/servers/83b1fe77-ef6d-408d-b255-aa3b60aad8f3" status: 404 len: 111 microversion: 2.1 time: 0.056553 Jul 16 21:13:00.159476 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 610/1300] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:13:00 2022] DELETE /compute/v2.1/servers/83b1fe77-ef6d-408d-b255-aa3b60aad8f3 => generated 111 bytes in 58 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:13:00.164480 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-bfc20a94-17a8-474b-91a9-bd458d8744cc tempest-LiveMigrationTest-1131758637 tempest-LiveMigrationTest-1131758637-project-admin] Calling method '.version_select at 0x7fdd9d0f3e50>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:13:00.167972 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-bfc20a94-17a8-474b-91a9-bd458d8744cc tempest-LiveMigrationTest-1131758637 tempest-LiveMigrationTest-1131758637-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:00.168718 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-bfc20a94-17a8-474b-91a9-bd458d8744cc tempest-LiveMigrationTest-1131758637 tempest-LiveMigrationTest-1131758637-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:00.186498 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-bfc20a94-17a8-474b-91a9-bd458d8744cc tempest-LiveMigrationTest-1131758637 tempest-LiveMigrationTest-1131758637-project-admin] 10.222.0.16 "GET /compute/v2.1/os-migrations" status: 200 len: 3505 microversion: 2.1 time: 0.025728 Jul 16 21:13:00.188594 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 611/1301] 10.222.0.16 () {58 vars in 1142 bytes} [Sat Jul 16 21:13:00 2022] GET /compute/v2.1/os-migrations => generated 3505 bytes in 28 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:13:00.195397 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-9d0732ae-e8e9-4d01-aade-db11fc28273b tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Calling method '.version_select at 0x7fdd9d0b5820>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:13:00.913455 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-329b1cd8-0cff-4b4b-ae0f-3f8767a4d1fc tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b68dacc4b8463ba07fae6372131117', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55a19b7a4682457a9132f5fc80110712', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:13:00.915879 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=100 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:13:00.923359 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-329b1cd8-0cff-4b4b-ae0f-3f8767a4d1fc tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "GET /compute/v2.1/servers/126f561c-163f-4f6d-8be1-98a2e29b0909" status: 200 len: 1477 microversion: 2.1 time: 0.808748 Jul 16 21:13:00.923359 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 691/1302] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:13:00 2022] GET /compute/v2.1/servers/126f561c-163f-4f6d-8be1-98a2e29b0909 => generated 1477 bytes in 810 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:13:00.927704 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-901f9c7f-914a-41c2-b42d-3730d0a33b77 tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:13:00.930280 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-901f9c7f-914a-41c2-b42d-3730d0a33b77 tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] [instance: 83b1fe77-ef6d-408d-b255-aa3b60aad8f3] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:13:00.936086 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-901f9c7f-914a-41c2-b42d-3730d0a33b77 tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:00.936654 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-901f9c7f-914a-41c2-b42d-3730d0a33b77 tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:00.937527 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-901f9c7f-914a-41c2-b42d-3730d0a33b77 tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:00.938227 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-901f9c7f-914a-41c2-b42d-3730d0a33b77 tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:00.961604 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-901f9c7f-914a-41c2-b42d-3730d0a33b77 tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] HTTP exception thrown: Instance 83b1fe77-ef6d-408d-b255-aa3b60aad8f3 could not be found. Jul 16 21:13:00.963692 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-901f9c7f-914a-41c2-b42d-3730d0a33b77 tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Request method failure captured: Jul 16 21:13:00.963692 ncdv-master-1 devstack@n-api.service[138680]: request: GET /compute/v2.1/servers/83b1fe77-ef6d-408d-b255-aa3b60aad8f3 HTTP/1.1 Jul 16 21:13:00.963692 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:13:00.963692 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:13:00.963692 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:13:00.963692 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 0 Jul 16 21:13:00.963692 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:13:00.963692 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:13:00.963692 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:13:00.963692 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:13:00.963692 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0yme2OmgPmghOZxTYv1prM8I30jqdYxV66dKB7kNvJ87HPySbthZ6En_3VLO5oCbOf2UuM1ARwLRB17-5AINJPTiqPrtco0L2oHyrrsd2HBbkXlr_mLFUNTutX_YwODzB_43ujJc9IK-SLZSWvs3yKGjofUk2kkT8-qMrEcv7y0wtas Jul 16 21:13:00.963692 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:13:00.963692 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:13:00.963692 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:13:00.963692 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:13:00.963692 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:13:00.963692 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:13:00.963692 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: 7d2b583c48254e17a8082f6467f2ae6d Jul 16 21:13:00.963692 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-DeleteServersAdminTestJSON-69305867 Jul 16 21:13:00.963692 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:13:00.963692 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:13:00.966594 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/7d2b583c48254e17a8082f6467f2ae6d"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/7d2b583c48254e17a8082f6467f2ae6d"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_7d2b583c48254e17a8082f6467f2ae6d"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/7d2b583c48254e17a8082f6467f2ae6d"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:13:00.966594 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-DeleteServersAdminTestJSON-69305867 Jul 16 21:13:00.966594 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: 7d2b583c48254e17a8082f6467f2ae6d Jul 16 21:13:00.966594 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-DeleteServersAdminTestJSON-69305867 Jul 16 21:13:00.966594 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-DeleteServersAdminTestJSON-69305867-project Jul 16 21:13:00.966594 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:13:00.966594 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:13:00.966594 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: 56d4bbcdebc24a34a814d7f6e72fbcfd Jul 16 21:13:00.966594 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-DeleteServersAdminTestJSON-69305867-project Jul 16 21:13:00.966594 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 21:13:00.966594 ncdv-master-1 devstack@n-api.service[138680]: exception: Instance 83b1fe77-ef6d-408d-b255-aa3b60aad8f3 could not be found. Jul 16 21:13:00.966594 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': '83b1fe77-ef6d-408d-b255-aa3b60aad8f3'} Jul 16 21:13:00.968681 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:13:00.968681 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:13:00.968681 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:13:00.968681 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:13:00.968681 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:13:00.968681 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:13:00.968681 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:13:00.968681 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:13:00.968681 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:13:00.968681 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:13:00.968681 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 83b1fe77-ef6d-408d-b255-aa3b60aad8f3 could not be found. Jul 16 21:13:00.968681 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:13:00.968681 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:13:00.968681 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:13:00.968681 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:13:00.968681 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:13:00.968681 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:13:00.968681 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:13:00.968681 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:13:00.968681 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:13:00.976644 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:13:00.976644 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:13:00.976644 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:13:00.976644 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:13:00.976644 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:13:00.976644 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:13:00.976644 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:13:00.976644 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 83b1fe77-ef6d-408d-b255-aa3b60aad8f3 could not be found. Jul 16 21:13:00.976644 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:13:00.976644 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:13:00.976644 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:13:00.976644 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:13:00.976644 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:13:00.976644 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:13:00.976644 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:13:00.976644 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:13:00.976644 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 83b1fe77-ef6d-408d-b255-aa3b60aad8f3 could not be found. Jul 16 21:13:00.976644 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:13:00.980062 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-901f9c7f-914a-41c2-b42d-3730d0a33b77 tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] Returning 404 to user: Instance 83b1fe77-ef6d-408d-b255-aa3b60aad8f3 could not be found. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:13:00.980062 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-901f9c7f-914a-41c2-b42d-3730d0a33b77 tempest-DeleteServersAdminTestJSON-69305867 tempest-DeleteServersAdminTestJSON-69305867-project] 10.222.0.16 "GET /compute/v2.1/servers/83b1fe77-ef6d-408d-b255-aa3b60aad8f3" status: 404 len: 111 microversion: 2.1 time: 0.043506 Jul 16 21:13:00.980062 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 692/1303] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:13:00 2022] GET /compute/v2.1/servers/83b1fe77-ef6d-408d-b255-aa3b60aad8f3 => generated 111 bytes in 45 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:13:00.980062 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-d0c13d15-5898-4c14-b0b5-3dc53354474c tempest-LiveMigrationTest-1131758637 tempest-LiveMigrationTest-1131758637-project-admin] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:13:00.980062 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-d0c13d15-5898-4c14-b0b5-3dc53354474c tempest-LiveMigrationTest-1131758637 tempest-LiveMigrationTest-1131758637-project-admin] [instance: 6a391156-ffce-49c0-a79d-47b4544264f5] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:13:00.988978 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d0c13d15-5898-4c14-b0b5-3dc53354474c tempest-LiveMigrationTest-1131758637 tempest-LiveMigrationTest-1131758637-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:00.989716 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d0c13d15-5898-4c14-b0b5-3dc53354474c tempest-LiveMigrationTest-1131758637 tempest-LiveMigrationTest-1131758637-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:00.990519 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d0c13d15-5898-4c14-b0b5-3dc53354474c tempest-LiveMigrationTest-1131758637 tempest-LiveMigrationTest-1131758637-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:00.991537 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d0c13d15-5898-4c14-b0b5-3dc53354474c tempest-LiveMigrationTest-1131758637 tempest-LiveMigrationTest-1131758637-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:01.567876 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-d0c13d15-5898-4c14-b0b5-3dc53354474c tempest-LiveMigrationTest-1131758637 tempest-LiveMigrationTest-1131758637-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/6a391156-ffce-49c0-a79d-47b4544264f5" status: 200 len: 1858 microversion: 2.1 time: 0.596072 Jul 16 21:13:01.580375 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 693/1304] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:13:00 2022] GET /compute/v2.1/servers/6a391156-ffce-49c0-a79d-47b4544264f5 => generated 1858 bytes in 600 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:13:01.581088 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-f65d071a-4c72-46b6-a180-6aa353e93a64 tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:13:01.582631 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-f65d071a-4c72-46b6-a180-6aa353e93a64 tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] [instance: 2b26717b-355f-417e-8a7c-72c41e1974e1] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:13:01.595511 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f65d071a-4c72-46b6-a180-6aa353e93a64 tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:01.596242 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f65d071a-4c72-46b6-a180-6aa353e93a64 tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:01.597286 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f65d071a-4c72-46b6-a180-6aa353e93a64 tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:01.597854 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f65d071a-4c72-46b6-a180-6aa353e93a64 tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:01.635590 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-f65d071a-4c72-46b6-a180-6aa353e93a64 tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] [instance: 2b26717b-355f-417e-8a7c-72c41e1974e1] Going to try to terminate instance {{(pid=138680) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 16 21:13:01.970625 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-f65d071a-4c72-46b6-a180-6aa353e93a64 tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] 10.222.0.16 "DELETE /compute/v2.1/servers/2b26717b-355f-417e-8a7c-72c41e1974e1" status: 204 len: 0 microversion: 2.1 time: 0.399950 Jul 16 21:13:01.971253 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 694/1305] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:13:01 2022] DELETE /compute/v2.1/servers/2b26717b-355f-417e-8a7c-72c41e1974e1 => generated 0 bytes in 402 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 16 21:13:01.981206 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-21205b06-06e6-46ca-8357-94848ff6cae4 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:13:01.981939 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-21205b06-06e6-46ca-8357-94848ff6cae4 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] [instance: 08e7e1f9-8b5c-446c-a05f-d8043473dd35] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:13:01.998675 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-21205b06-06e6-46ca-8357-94848ff6cae4 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:01.999108 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-21205b06-06e6-46ca-8357-94848ff6cae4 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:02.008555 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-21205b06-06e6-46ca-8357-94848ff6cae4 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:02.008555 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-21205b06-06e6-46ca-8357-94848ff6cae4 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:02.482294 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-21205b06-06e6-46ca-8357-94848ff6cae4 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '52281b3989a04ca2a50b0bc93bf783f0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '30d028fa9b854c8fb71c8cbed2985c27', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:13:02.490093 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-21205b06-06e6-46ca-8357-94848ff6cae4 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] 10.222.0.16 "GET /compute/v2.1/servers/08e7e1f9-8b5c-446c-a05f-d8043473dd35" status: 200 len: 1508 microversion: 2.1 time: 0.517576 Jul 16 21:13:02.490755 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 695/1306] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:13:01 2022] GET /compute/v2.1/servers/08e7e1f9-8b5c-446c-a05f-d8043473dd35 => generated 1508 bytes in 519 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:13:02.495882 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-4a3f8dba-23e7-48ef-a3c0-0a1a22295103 tempest-TenantUsagesTestJSON-260059616 tempest-TenantUsagesTestJSON-260059616-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:13:02.496572 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-4a3f8dba-23e7-48ef-a3c0-0a1a22295103 tempest-TenantUsagesTestJSON-260059616 tempest-TenantUsagesTestJSON-260059616-project] [instance: 6e68dbd6-1d5c-471e-ad69-cf1c4aa4004a] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:13:02.506895 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4a3f8dba-23e7-48ef-a3c0-0a1a22295103 tempest-TenantUsagesTestJSON-260059616 tempest-TenantUsagesTestJSON-260059616-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:02.508048 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4a3f8dba-23e7-48ef-a3c0-0a1a22295103 tempest-TenantUsagesTestJSON-260059616 tempest-TenantUsagesTestJSON-260059616-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:02.511426 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4a3f8dba-23e7-48ef-a3c0-0a1a22295103 tempest-TenantUsagesTestJSON-260059616 tempest-TenantUsagesTestJSON-260059616-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:02.511426 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4a3f8dba-23e7-48ef-a3c0-0a1a22295103 tempest-TenantUsagesTestJSON-260059616 tempest-TenantUsagesTestJSON-260059616-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:02.731850 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-4a3f8dba-23e7-48ef-a3c0-0a1a22295103 tempest-TenantUsagesTestJSON-260059616 tempest-TenantUsagesTestJSON-260059616-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2db871d6aa9746d2b660efbfbdb1dc75', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cad2f07400d54ef9825022d5b3489db0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:13:02.739697 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-4a3f8dba-23e7-48ef-a3c0-0a1a22295103 tempest-TenantUsagesTestJSON-260059616 tempest-TenantUsagesTestJSON-260059616-project] 10.222.0.16 "GET /compute/v2.1/servers/6e68dbd6-1d5c-471e-ad69-cf1c4aa4004a" status: 200 len: 1314 microversion: 2.1 time: 0.247696 Jul 16 21:13:02.740276 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 696/1307] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:13:02 2022] GET /compute/v2.1/servers/6e68dbd6-1d5c-471e-ad69-cf1c4aa4004a => generated 1314 bytes in 249 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:13:02.745580 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-dcded689-c8b3-4286-8498-edac132a0dc0 tempest-LiveMigrationTest-1131758637 tempest-LiveMigrationTest-1131758637-project-admin] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:13:02.746199 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-dcded689-c8b3-4286-8498-edac132a0dc0 tempest-LiveMigrationTest-1131758637 tempest-LiveMigrationTest-1131758637-project-admin] [instance: 6a391156-ffce-49c0-a79d-47b4544264f5] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:13:02.752781 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-dcded689-c8b3-4286-8498-edac132a0dc0 tempest-LiveMigrationTest-1131758637 tempest-LiveMigrationTest-1131758637-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:02.753246 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-dcded689-c8b3-4286-8498-edac132a0dc0 tempest-LiveMigrationTest-1131758637 tempest-LiveMigrationTest-1131758637-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:02.754120 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-dcded689-c8b3-4286-8498-edac132a0dc0 tempest-LiveMigrationTest-1131758637 tempest-LiveMigrationTest-1131758637-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:02.754502 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-dcded689-c8b3-4286-8498-edac132a0dc0 tempest-LiveMigrationTest-1131758637 tempest-LiveMigrationTest-1131758637-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:02.928887 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-dcded689-c8b3-4286-8498-edac132a0dc0 tempest-LiveMigrationTest-1131758637 tempest-LiveMigrationTest-1131758637-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/6a391156-ffce-49c0-a79d-47b4544264f5" status: 200 len: 1858 microversion: 2.1 time: 0.187521 Jul 16 21:13:02.930765 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 697/1308] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:13:02 2022] GET /compute/v2.1/servers/6a391156-ffce-49c0-a79d-47b4544264f5 => generated 1858 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:13:02.936993 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-ca56f923-5e17-4d99-b2b3-ad2c687efa7c tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:13:02.936993 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-ca56f923-5e17-4d99-b2b3-ad2c687efa7c tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: 126f561c-163f-4f6d-8be1-98a2e29b0909] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:13:02.937340 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-9d0732ae-e8e9-4d01-aade-db11fc28273b tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] 10.222.0.16 "DELETE /compute/v2.1/os-floating-ips/727dc9c6-603c-4649-98a7-3759801fbfef" status: 202 len: 0 microversion: 2.1 time: 2.747267 Jul 16 21:13:02.937782 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 612/1309] 10.222.0.16 () {58 vars in 1262 bytes} [Sat Jul 16 21:13:00 2022] DELETE /compute/v2.1/os-floating-ips/727dc9c6-603c-4649-98a7-3759801fbfef => generated 0 bytes in 2748 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 21:13:02.944540 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-b0dd5ad5-6b22-4f87-bef7-5362281be10c tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:13:02.946442 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-b0dd5ad5-6b22-4f87-bef7-5362281be10c tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] [instance: 2b26717b-355f-417e-8a7c-72c41e1974e1] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:13:02.948506 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ca56f923-5e17-4d99-b2b3-ad2c687efa7c tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:02.948962 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ca56f923-5e17-4d99-b2b3-ad2c687efa7c tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:02.949742 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ca56f923-5e17-4d99-b2b3-ad2c687efa7c tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:02.950180 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ca56f923-5e17-4d99-b2b3-ad2c687efa7c tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:02.954285 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b0dd5ad5-6b22-4f87-bef7-5362281be10c tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:02.954742 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b0dd5ad5-6b22-4f87-bef7-5362281be10c tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:02.955653 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b0dd5ad5-6b22-4f87-bef7-5362281be10c tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:02.956892 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b0dd5ad5-6b22-4f87-bef7-5362281be10c tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:03.240594 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-ca56f923-5e17-4d99-b2b3-ad2c687efa7c tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b68dacc4b8463ba07fae6372131117', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55a19b7a4682457a9132f5fc80110712', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:13:03.249882 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-b0dd5ad5-6b22-4f87-bef7-5362281be10c tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '034c11cc3f174c6fb44c8d64e55cf8b9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb4ff5d7e42940cdb7be71dadd895ee4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:13:03.259789 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-ca56f923-5e17-4d99-b2b3-ad2c687efa7c tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "GET /compute/v2.1/servers/126f561c-163f-4f6d-8be1-98a2e29b0909" status: 200 len: 1477 microversion: 2.1 time: 0.319911 Jul 16 21:13:03.262928 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 698/1310] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:13:02 2022] GET /compute/v2.1/servers/126f561c-163f-4f6d-8be1-98a2e29b0909 => generated 1477 bytes in 332 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:13:03.268116 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-b0dd5ad5-6b22-4f87-bef7-5362281be10c tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] 10.222.0.16 "GET /compute/v2.1/servers/2b26717b-355f-417e-8a7c-72c41e1974e1" status: 200 len: 1700 microversion: 2.1 time: 0.329433 Jul 16 21:13:03.268737 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 613/1311] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:13:02 2022] GET /compute/v2.1/servers/2b26717b-355f-417e-8a7c-72c41e1974e1 => generated 1700 bytes in 331 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:13:03.273885 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-1b85246c-bd0c-4e47-abe3-310bc663e727 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:13:03.274384 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-1b85246c-bd0c-4e47-abe3-310bc663e727 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] [instance: 8020f553-6025-4f19-835c-c00e2fb4f764] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:13:03.278424 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-89804b6a-8233-490d-a9bb-149e951998e5 req-37aea7e2-3ca7-4223-b968-bc8a3ab41b7a service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "6e68dbd6-1d5c-471e-ad69-cf1c4aa4004a", "tag": "ca8395d6-64dd-4e53-92a0-b2a79f375684"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:13:03.289436 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1b85246c-bd0c-4e47-abe3-310bc663e727 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:03.289436 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1b85246c-bd0c-4e47-abe3-310bc663e727 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:03.289436 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1b85246c-bd0c-4e47-abe3-310bc663e727 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:03.289436 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1b85246c-bd0c-4e47-abe3-310bc663e727 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:03.290000 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-89804b6a-8233-490d-a9bb-149e951998e5 req-37aea7e2-3ca7-4223-b968-bc8a3ab41b7a service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:03.290000 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-89804b6a-8233-490d-a9bb-149e951998e5 req-37aea7e2-3ca7-4223-b968-bc8a3ab41b7a service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:03.300145 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-89804b6a-8233-490d-a9bb-149e951998e5 req-37aea7e2-3ca7-4223-b968-bc8a3ab41b7a service nova] Creating event network-changed:ca8395d6-64dd-4e53-92a0-b2a79f375684 for instance 6e68dbd6-1d5c-471e-ad69-cf1c4aa4004a on nch2-master-1 Jul 16 21:13:03.307170 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-89804b6a-8233-490d-a9bb-149e951998e5 req-37aea7e2-3ca7-4223-b968-bc8a3ab41b7a service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.042428 Jul 16 21:13:03.307795 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 699/1312] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:13:03 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:13:03.313847 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-a535b703-ec4a-47c0-a764-849da1ee02b0 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:13:03.314444 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-a535b703-ec4a-47c0-a764-849da1ee02b0 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] [instance: 6a391156-ffce-49c0-a79d-47b4544264f5] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:13:03.314784 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-1b85246c-bd0c-4e47-abe3-310bc663e727 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] [instance: 8020f553-6025-4f19-835c-c00e2fb4f764] Going to try to terminate instance {{(pid=138681) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 16 21:13:03.317803 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_api:SELECT=99,nova_api:UPDATE=1 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:13:03.322876 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a535b703-ec4a-47c0-a764-849da1ee02b0 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:03.323385 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a535b703-ec4a-47c0-a764-849da1ee02b0 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:03.324221 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a535b703-ec4a-47c0-a764-849da1ee02b0 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:03.324652 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a535b703-ec4a-47c0-a764-849da1ee02b0 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:03.346489 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=95,nova_cell1:UPDATE=2,nova_cell1:INSERT=1,nova_cell1:SAVEPOINT=1,nova_cell1:RELEASE=1 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:13:03.416613 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.objects.instance [None req-a535b703-ec4a-47c0-a764-849da1ee02b0 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lazy-loading 'services' on Instance uuid 6a391156-ffce-49c0-a79d-47b4544264f5 {{(pid=138680) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:13:03.666875 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-a535b703-ec4a-47c0-a764-849da1ee02b0 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] 10.222.0.16 "DELETE /compute/v2.1/servers/6a391156-ffce-49c0-a79d-47b4544264f5/os-volume_attachments/ce0ae7fc-6256-4387-a72f-dd24801e38f0" status: 202 len: 0 microversion: 2.1 time: 0.357535 Jul 16 21:13:03.667518 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 700/1313] 10.222.0.16 () {58 vars in 1415 bytes} [Sat Jul 16 21:13:03 2022] DELETE /compute/v2.1/servers/6a391156-ffce-49c0-a79d-47b4544264f5/os-volume_attachments/ce0ae7fc-6256-4387-a72f-dd24801e38f0 => generated 0 bytes in 359 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 21:13:03.676385 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-ef3b64b2-6562-4449-b41f-2ad7bc6ad0a1 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:13:03.677497 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-ef3b64b2-6562-4449-b41f-2ad7bc6ad0a1 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] [instance: 6a391156-ffce-49c0-a79d-47b4544264f5] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:13:03.685158 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ef3b64b2-6562-4449-b41f-2ad7bc6ad0a1 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:03.685825 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ef3b64b2-6562-4449-b41f-2ad7bc6ad0a1 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:03.687935 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ef3b64b2-6562-4449-b41f-2ad7bc6ad0a1 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:03.687935 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ef3b64b2-6562-4449-b41f-2ad7bc6ad0a1 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:03.714028 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-1b85246c-bd0c-4e47-abe3-310bc663e727 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] 10.222.0.16 "DELETE /compute/v2.1/servers/8020f553-6025-4f19-835c-c00e2fb4f764" status: 204 len: 0 microversion: 2.1 time: 0.438860 Jul 16 21:13:03.714028 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 614/1314] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:13:03 2022] DELETE /compute/v2.1/servers/8020f553-6025-4f19-835c-c00e2fb4f764 => generated 0 bytes in 440 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 16 21:13:03.719733 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-0fbcc4b6-68b6-42b2-b594-6ee4d0bb2f34 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:13:03.720445 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-0fbcc4b6-68b6-42b2-b594-6ee4d0bb2f34 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] [instance: 8020f553-6025-4f19-835c-c00e2fb4f764] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:13:03.723032 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-ef3b64b2-6562-4449-b41f-2ad7bc6ad0a1 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] 10.222.0.16 "GET /compute/v2.1/servers/6a391156-ffce-49c0-a79d-47b4544264f5/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050705 Jul 16 21:13:03.723611 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 701/1315] 10.222.0.16 () {58 vars in 1301 bytes} [Sat Jul 16 21:13:03 2022] GET /compute/v2.1/servers/6a391156-ffce-49c0-a79d-47b4544264f5/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 16 21:13:03.729206 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0fbcc4b6-68b6-42b2-b594-6ee4d0bb2f34 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:03.729668 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0fbcc4b6-68b6-42b2-b594-6ee4d0bb2f34 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:03.730462 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0fbcc4b6-68b6-42b2-b594-6ee4d0bb2f34 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:03.730908 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0fbcc4b6-68b6-42b2-b594-6ee4d0bb2f34 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:03.756884 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-e90052c4-4ec9-4250-838a-0833f4eabb49 tempest-TenantUsagesTestJSON-260059616 tempest-TenantUsagesTestJSON-260059616-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:13:03.758218 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-e90052c4-4ec9-4250-838a-0833f4eabb49 tempest-TenantUsagesTestJSON-260059616 tempest-TenantUsagesTestJSON-260059616-project] [instance: 6e68dbd6-1d5c-471e-ad69-cf1c4aa4004a] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:13:03.765385 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e90052c4-4ec9-4250-838a-0833f4eabb49 tempest-TenantUsagesTestJSON-260059616 tempest-TenantUsagesTestJSON-260059616-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:03.765846 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e90052c4-4ec9-4250-838a-0833f4eabb49 tempest-TenantUsagesTestJSON-260059616 tempest-TenantUsagesTestJSON-260059616-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:03.766646 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e90052c4-4ec9-4250-838a-0833f4eabb49 tempest-TenantUsagesTestJSON-260059616 tempest-TenantUsagesTestJSON-260059616-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:03.767124 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e90052c4-4ec9-4250-838a-0833f4eabb49 tempest-TenantUsagesTestJSON-260059616 tempest-TenantUsagesTestJSON-260059616-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:03.938517 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-0fbcc4b6-68b6-42b2-b594-6ee4d0bb2f34 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2dfcad8d2b57481bb33aa8f5d7483c51', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cda86413e74b4c1e8e09bd4df4926664', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:13:03.947435 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-0fbcc4b6-68b6-42b2-b594-6ee4d0bb2f34 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] 10.222.0.16 "GET /compute/v2.1/servers/8020f553-6025-4f19-835c-c00e2fb4f764" status: 200 len: 1525 microversion: 2.1 time: 0.232216 Jul 16 21:13:03.948357 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 615/1316] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:13:03 2022] GET /compute/v2.1/servers/8020f553-6025-4f19-835c-c00e2fb4f764 => generated 1525 bytes in 234 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:13:03.964736 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-e90052c4-4ec9-4250-838a-0833f4eabb49 tempest-TenantUsagesTestJSON-260059616 tempest-TenantUsagesTestJSON-260059616-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2db871d6aa9746d2b660efbfbdb1dc75', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cad2f07400d54ef9825022d5b3489db0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:13:03.972645 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-e90052c4-4ec9-4250-838a-0833f4eabb49 tempest-TenantUsagesTestJSON-260059616 tempest-TenantUsagesTestJSON-260059616-project] 10.222.0.16 "GET /compute/v2.1/servers/6e68dbd6-1d5c-471e-ad69-cf1c4aa4004a" status: 200 len: 1314 microversion: 2.1 time: 0.219704 Jul 16 21:13:03.973296 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 702/1317] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:13:03 2022] GET /compute/v2.1/servers/6e68dbd6-1d5c-471e-ad69-cf1c4aa4004a => generated 1314 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:13:04.270554 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-29e3db6f-69c8-463c-a9c3-4faf54ecd073 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:13:04.271451 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-29e3db6f-69c8-463c-a9c3-4faf54ecd073 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: 126f561c-163f-4f6d-8be1-98a2e29b0909] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:13:04.279090 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-29e3db6f-69c8-463c-a9c3-4faf54ecd073 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:04.279549 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-29e3db6f-69c8-463c-a9c3-4faf54ecd073 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:04.280327 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-29e3db6f-69c8-463c-a9c3-4faf54ecd073 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:04.280774 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-29e3db6f-69c8-463c-a9c3-4faf54ecd073 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:04.285707 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-b01bbf57-2acc-42df-aad9-c15f38b26873 tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:13:04.286328 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-b01bbf57-2acc-42df-aad9-c15f38b26873 tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] [instance: 2b26717b-355f-417e-8a7c-72c41e1974e1] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:13:04.293171 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b01bbf57-2acc-42df-aad9-c15f38b26873 tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:04.294062 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b01bbf57-2acc-42df-aad9-c15f38b26873 tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:04.294534 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b01bbf57-2acc-42df-aad9-c15f38b26873 tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:04.294972 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b01bbf57-2acc-42df-aad9-c15f38b26873 tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:04.499121 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-29e3db6f-69c8-463c-a9c3-4faf54ecd073 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b68dacc4b8463ba07fae6372131117', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55a19b7a4682457a9132f5fc80110712', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:13:04.506284 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-b01bbf57-2acc-42df-aad9-c15f38b26873 tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '034c11cc3f174c6fb44c8d64e55cf8b9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb4ff5d7e42940cdb7be71dadd895ee4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:13:04.506919 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-29e3db6f-69c8-463c-a9c3-4faf54ecd073 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "GET /compute/v2.1/servers/126f561c-163f-4f6d-8be1-98a2e29b0909" status: 200 len: 1477 microversion: 2.1 time: 0.240582 Jul 16 21:13:04.508296 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 616/1318] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:13:04 2022] GET /compute/v2.1/servers/126f561c-163f-4f6d-8be1-98a2e29b0909 => generated 1477 bytes in 243 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:13:04.515453 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-b01bbf57-2acc-42df-aad9-c15f38b26873 tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] 10.222.0.16 "GET /compute/v2.1/servers/2b26717b-355f-417e-8a7c-72c41e1974e1" status: 200 len: 1700 microversion: 2.1 time: 0.233592 Jul 16 21:13:04.516227 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 703/1319] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:13:04 2022] GET /compute/v2.1/servers/2b26717b-355f-417e-8a7c-72c41e1974e1 => generated 1700 bytes in 235 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:13:04.736114 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-a706e911-cb82-4bf9-887d-c1c1e0e6a959 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:13:04.737391 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-a706e911-cb82-4bf9-887d-c1c1e0e6a959 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] [instance: 6a391156-ffce-49c0-a79d-47b4544264f5] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:13:04.754718 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a706e911-cb82-4bf9-887d-c1c1e0e6a959 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:04.756551 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a706e911-cb82-4bf9-887d-c1c1e0e6a959 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:04.760095 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a706e911-cb82-4bf9-887d-c1c1e0e6a959 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:04.760536 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a706e911-cb82-4bf9-887d-c1c1e0e6a959 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:04.802573 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-a706e911-cb82-4bf9-887d-c1c1e0e6a959 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] 10.222.0.16 "GET /compute/v2.1/servers/6a391156-ffce-49c0-a79d-47b4544264f5/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.071303 Jul 16 21:13:04.803941 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 617/1320] 10.222.0.16 () {58 vars in 1301 bytes} [Sat Jul 16 21:13:04 2022] GET /compute/v2.1/servers/6a391156-ffce-49c0-a79d-47b4544264f5/os-volume_attachments => generated 197 bytes in 73 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:13:04.966085 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-33ea8939-693e-4234-ae4d-ae08610ed910 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:13:04.966693 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-33ea8939-693e-4234-ae4d-ae08610ed910 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] [instance: 8020f553-6025-4f19-835c-c00e2fb4f764] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:13:04.980774 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-33ea8939-693e-4234-ae4d-ae08610ed910 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:04.981210 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-33ea8939-693e-4234-ae4d-ae08610ed910 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:04.981997 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-33ea8939-693e-4234-ae4d-ae08610ed910 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:04.982439 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-33ea8939-693e-4234-ae4d-ae08610ed910 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:04.991417 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-6d2c38cb-b70e-46a7-9700-b9fa915c4557 tempest-TenantUsagesTestJSON-260059616 tempest-TenantUsagesTestJSON-260059616-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:13:04.993807 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-6d2c38cb-b70e-46a7-9700-b9fa915c4557 tempest-TenantUsagesTestJSON-260059616 tempest-TenantUsagesTestJSON-260059616-project] [instance: 6e68dbd6-1d5c-471e-ad69-cf1c4aa4004a] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:13:05.001807 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6d2c38cb-b70e-46a7-9700-b9fa915c4557 tempest-TenantUsagesTestJSON-260059616 tempest-TenantUsagesTestJSON-260059616-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:05.003042 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6d2c38cb-b70e-46a7-9700-b9fa915c4557 tempest-TenantUsagesTestJSON-260059616 tempest-TenantUsagesTestJSON-260059616-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:05.004301 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6d2c38cb-b70e-46a7-9700-b9fa915c4557 tempest-TenantUsagesTestJSON-260059616 tempest-TenantUsagesTestJSON-260059616-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:05.005255 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6d2c38cb-b70e-46a7-9700-b9fa915c4557 tempest-TenantUsagesTestJSON-260059616 tempest-TenantUsagesTestJSON-260059616-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:05.431406 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-33ea8939-693e-4234-ae4d-ae08610ed910 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2dfcad8d2b57481bb33aa8f5d7483c51', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cda86413e74b4c1e8e09bd4df4926664', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:13:05.441231 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-33ea8939-693e-4234-ae4d-ae08610ed910 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] 10.222.0.16 "GET /compute/v2.1/servers/8020f553-6025-4f19-835c-c00e2fb4f764" status: 200 len: 1525 microversion: 2.1 time: 0.479504 Jul 16 21:13:05.452945 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 704/1321] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:13:04 2022] GET /compute/v2.1/servers/8020f553-6025-4f19-835c-c00e2fb4f764 => generated 1525 bytes in 492 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:13:05.458041 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-6324d95d-5e53-4534-8a5f-4c09aba6148c tempest-VolumesAdminNegativeTest-1185523518 tempest-VolumesAdminNegativeTest-1185523518-project-admin] Action: 'update', calling method: >, body: {"volumeAttachment": {"volumeId": "4f03adb7-f52d-4f10-8f7c-107b8e84874d"}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:13:05.459374 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-6324d95d-5e53-4534-8a5f-4c09aba6148c tempest-VolumesAdminNegativeTest-1185523518 tempest-VolumesAdminNegativeTest-1185523518-project-admin] [instance: 08e7e1f9-8b5c-446c-a05f-d8043473dd35] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:13:05.465954 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6324d95d-5e53-4534-8a5f-4c09aba6148c tempest-VolumesAdminNegativeTest-1185523518 tempest-VolumesAdminNegativeTest-1185523518-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:05.466529 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6324d95d-5e53-4534-8a5f-4c09aba6148c tempest-VolumesAdminNegativeTest-1185523518 tempest-VolumesAdminNegativeTest-1185523518-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:05.467609 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6324d95d-5e53-4534-8a5f-4c09aba6148c tempest-VolumesAdminNegativeTest-1185523518 tempest-VolumesAdminNegativeTest-1185523518-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:05.468121 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6324d95d-5e53-4534-8a5f-4c09aba6148c tempest-VolumesAdminNegativeTest-1185523518 tempest-VolumesAdminNegativeTest-1185523518-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:05.513018 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-6d2c38cb-b70e-46a7-9700-b9fa915c4557 tempest-TenantUsagesTestJSON-260059616 tempest-TenantUsagesTestJSON-260059616-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2db871d6aa9746d2b660efbfbdb1dc75', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cad2f07400d54ef9825022d5b3489db0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:13:05.521542 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-6d2c38cb-b70e-46a7-9700-b9fa915c4557 tempest-TenantUsagesTestJSON-260059616 tempest-TenantUsagesTestJSON-260059616-project] 10.222.0.16 "GET /compute/v2.1/servers/6e68dbd6-1d5c-471e-ad69-cf1c4aa4004a" status: 200 len: 1314 microversion: 2.1 time: 0.535245 Jul 16 21:13:05.528222 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-6324d95d-5e53-4534-8a5f-4c09aba6148c tempest-VolumesAdminNegativeTest-1185523518 tempest-VolumesAdminNegativeTest-1185523518-project-admin] HTTP exception thrown: Volume 4c0d3168-2fa3-4d5a-aabe-3783e306d44e could not be found. Jul 16 21:13:05.530722 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-6324d95d-5e53-4534-8a5f-4c09aba6148c tempest-VolumesAdminNegativeTest-1185523518 tempest-VolumesAdminNegativeTest-1185523518-project-admin] Request method failure captured: Jul 16 21:13:05.530722 ncdv-master-1 devstack@n-api.service[138680]: request: PUT /compute/v2.1/servers/08e7e1f9-8b5c-446c-a05f-d8043473dd35/os-volume_attachments/4c0d3168-2fa3-4d5a-aabe-3783e306d44e HTTP/1.1 Jul 16 21:13:05.530722 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:13:05.530722 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:13:05.530722 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:13:05.530722 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 74 Jul 16 21:13:05.530722 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:13:05.530722 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:13:05.530722 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:13:05.530722 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:13:05.530722 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0ylyPrDr_ELiD-cVGkDdxIwIy9yztHbKzORPMKyk_Z-quXpZuUroAQ7B6aRJ_xMnhc7Mw9L85FOxhXg0BkOCB3k_jmKYEXDG1DCKKyD2w4H3kaQh0PNoOYkTf-uFALfgRbkzuyZ4NwMUzNG81KX7XtzeBIVNZDY0W8riazzimeQdxUE Jul 16 21:13:05.530722 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:13:05.530722 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:13:05.530722 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:13:05.530722 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:13:05.530722 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:13:05.530722 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:13:05.530722 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: f27ffd13e54d41ebb9e79a74725cb3bf Jul 16 21:13:05.530722 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-VolumesAdminNegativeTest-1185523518 Jul 16 21:13:05.530722 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,admin,reader Jul 16 21:13:05.530722 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,admin,reader Jul 16 21:13:05.532229 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/f27ffd13e54d41ebb9e79a74725cb3bf"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/f27ffd13e54d41ebb9e79a74725cb3bf"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_f27ffd13e54d41ebb9e79a74725cb3bf"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/f27ffd13e54d41ebb9e79a74725cb3bf"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:13:05.532229 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-VolumesAdminNegativeTest-1185523518 Jul 16 21:13:05.532229 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: f27ffd13e54d41ebb9e79a74725cb3bf Jul 16 21:13:05.532229 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-VolumesAdminNegativeTest-1185523518 Jul 16 21:13:05.532229 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-VolumesAdminNegativeTest-1185523518-project-admin Jul 16 21:13:05.532229 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:13:05.532229 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:13:05.532229 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: 6dfd80a71b204625b5401e5249a0bbed Jul 16 21:13:05.532229 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-VolumesAdminNegativeTest-1185523518-project-admin Jul 16 21:13:05.532229 ncdv-master-1 devstack@n-api.service[138680]: Jul 16 21:13:05.532229 ncdv-master-1 devstack@n-api.service[138680]: {"volumeAttachment": {"volumeId": "4f03adb7-f52d-4f10-8f7c-107b8e84874d"}} Jul 16 21:13:05.532229 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 21:13:05.532229 ncdv-master-1 devstack@n-api.service[138680]: exception: Volume 4c0d3168-2fa3-4d5a-aabe-3783e306d44e could not be found. Jul 16 21:13:05.533017 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'server_id': '08e7e1f9-8b5c-446c-a05f-d8043473dd35', 'id': '4c0d3168-2fa3-4d5a-aabe-3783e306d44e', 'body': {'volumeAttachment': {'volumeId': '4f03adb7-f52d-4f10-8f7c-107b8e84874d'}}} Jul 16 21:13:05.533017 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:13:05.533017 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:13:05.533017 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jul 16 21:13:05.533017 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jul 16 21:13:05.533017 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Jul 16 21:13:05.533017 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi item = cinderclient( Jul 16 21:13:05.533017 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Jul 16 21:13:05.533017 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Jul 16 21:13:05.533017 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Jul 16 21:13:05.533017 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Jul 16 21:13:05.533017 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Jul 16 21:13:05.533017 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Jul 16 21:13:05.533017 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jul 16 21:13:05.533017 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jul 16 21:13:05.533017 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jul 16 21:13:05.533017 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jul 16 21:13:05.534035 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume 4c0d3168-2fa3-4d5a-aabe-3783e306d44e could not be found. (HTTP 404) (Request-ID: req-a883ee49-b84f-495a-8a4b-1a0bfb8bf605) Jul 16 21:13:05.534035 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:13:05.534035 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:13:05.534035 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:13:05.534035 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:13:05.534035 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 430, in _update_volume_swap Jul 16 21:13:05.534035 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi old_volume = self.volume_api.get(context, old_volume_id) Jul 16 21:13:05.534035 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Jul 16 21:13:05.534035 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Jul 16 21:13:05.534035 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Jul 16 21:13:05.534035 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Jul 16 21:13:05.534035 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Jul 16 21:13:05.534035 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Jul 16 21:13:05.534035 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jul 16 21:13:05.534035 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jul 16 21:13:05.534035 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Jul 16 21:13:05.534035 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi item = cinderclient( Jul 16 21:13:05.534035 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Jul 16 21:13:05.534035 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Jul 16 21:13:05.543324 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Jul 16 21:13:05.543324 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Jul 16 21:13:05.543324 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Jul 16 21:13:05.543324 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Jul 16 21:13:05.543324 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jul 16 21:13:05.543324 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jul 16 21:13:05.543324 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jul 16 21:13:05.543324 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jul 16 21:13:05.543324 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume 4c0d3168-2fa3-4d5a-aabe-3783e306d44e could not be found. Jul 16 21:13:05.543324 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:13:05.543324 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:13:05.543324 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:13:05.543324 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:13:05.543324 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:13:05.543324 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:13:05.543324 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:13:05.543324 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:13:05.543324 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:13:05.543324 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:13:05.544647 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 16 21:13:05.544647 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 16 21:13:05.544647 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 16 21:13:05.544647 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 16 21:13:05.544647 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 522, in update Jul 16 21:13:05.544647 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi self._update_volume_swap(req, instance, id, body) Jul 16 21:13:05.544647 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 432, in _update_volume_swap Jul 16 21:13:05.544647 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:13:05.544647 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume 4c0d3168-2fa3-4d5a-aabe-3783e306d44e could not be found. Jul 16 21:13:05.544647 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:13:05.544647 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:13:05.544647 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:13:05.544647 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:13:05.544647 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:13:05.544647 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:13:05.544647 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:13:05.544647 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:13:05.544647 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume 4c0d3168-2fa3-4d5a-aabe-3783e306d44e could not be found. Jul 16 21:13:05.544647 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:13:05.545762 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-6324d95d-5e53-4534-8a5f-4c09aba6148c tempest-VolumesAdminNegativeTest-1185523518 tempest-VolumesAdminNegativeTest-1185523518-project-admin] Returning 404 to user: Volume 4c0d3168-2fa3-4d5a-aabe-3783e306d44e could not be found. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:13:05.545762 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-6324d95d-5e53-4534-8a5f-4c09aba6148c tempest-VolumesAdminNegativeTest-1185523518 tempest-VolumesAdminNegativeTest-1185523518-project-admin] 10.222.0.16 "PUT /compute/v2.1/servers/08e7e1f9-8b5c-446c-a05f-d8043473dd35/os-volume_attachments/4c0d3168-2fa3-4d5a-aabe-3783e306d44e" status: 404 len: 109 microversion: 2.1 time: 0.078642 Jul 16 21:13:05.545903 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 618/1322] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:13:04 2022] GET /compute/v2.1/servers/6e68dbd6-1d5c-471e-ad69-cf1c4aa4004a => generated 1314 bytes in 548 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:13:05.545989 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 705/1323] 10.222.0.16 () {60 vars in 1432 bytes} [Sat Jul 16 21:13:05 2022] PUT /compute/v2.1/servers/08e7e1f9-8b5c-446c-a05f-d8043473dd35/os-volume_attachments/4c0d3168-2fa3-4d5a-aabe-3783e306d44e => generated 109 bytes in 81 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:13:05.545989 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-495255e4-9c2a-40df-ad4a-9c91c5c66e2f tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:13:05.546134 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-1c67d24d-856a-4cdf-b6f3-ed3c8403185d tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:13:05.546207 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-495255e4-9c2a-40df-ad4a-9c91c5c66e2f tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] [instance: 2b26717b-355f-417e-8a7c-72c41e1974e1] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:13:05.546287 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-1c67d24d-856a-4cdf-b6f3-ed3c8403185d tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: 126f561c-163f-4f6d-8be1-98a2e29b0909] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:13:05.551892 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1c67d24d-856a-4cdf-b6f3-ed3c8403185d tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:05.552358 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1c67d24d-856a-4cdf-b6f3-ed3c8403185d tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:05.553702 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1c67d24d-856a-4cdf-b6f3-ed3c8403185d tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:05.554794 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1c67d24d-856a-4cdf-b6f3-ed3c8403185d tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:05.561681 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-495255e4-9c2a-40df-ad4a-9c91c5c66e2f tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:05.563432 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-495255e4-9c2a-40df-ad4a-9c91c5c66e2f tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:05.568817 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-495255e4-9c2a-40df-ad4a-9c91c5c66e2f tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:05.568817 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-495255e4-9c2a-40df-ad4a-9c91c5c66e2f tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:06.135359 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-495255e4-9c2a-40df-ad4a-9c91c5c66e2f tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '034c11cc3f174c6fb44c8d64e55cf8b9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb4ff5d7e42940cdb7be71dadd895ee4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:13:06.145630 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-495255e4-9c2a-40df-ad4a-9c91c5c66e2f tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] 10.222.0.16 "GET /compute/v2.1/servers/2b26717b-355f-417e-8a7c-72c41e1974e1" status: 200 len: 1700 microversion: 2.1 time: 0.609499 Jul 16 21:13:06.146325 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 706/1324] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:13:05 2022] GET /compute/v2.1/servers/2b26717b-355f-417e-8a7c-72c41e1974e1 => generated 1700 bytes in 611 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:13:06.159039 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-1c67d24d-856a-4cdf-b6f3-ed3c8403185d tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b68dacc4b8463ba07fae6372131117', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55a19b7a4682457a9132f5fc80110712', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:13:06.167978 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-1c67d24d-856a-4cdf-b6f3-ed3c8403185d tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "GET /compute/v2.1/servers/126f561c-163f-4f6d-8be1-98a2e29b0909" status: 200 len: 1477 microversion: 2.1 time: 0.632690 Jul 16 21:13:06.168702 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 619/1325] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:13:05 2022] GET /compute/v2.1/servers/126f561c-163f-4f6d-8be1-98a2e29b0909 => generated 1477 bytes in 634 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:13:06.173818 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-17e697d1-50a1-4afb-ad15-876e182cbdd2 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:13:06.175028 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-17e697d1-50a1-4afb-ad15-876e182cbdd2 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] [instance: 6a391156-ffce-49c0-a79d-47b4544264f5] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:13:06.181439 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-17e697d1-50a1-4afb-ad15-876e182cbdd2 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:06.181897 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-17e697d1-50a1-4afb-ad15-876e182cbdd2 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:06.182683 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-17e697d1-50a1-4afb-ad15-876e182cbdd2 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:06.183148 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-17e697d1-50a1-4afb-ad15-876e182cbdd2 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:06.215600 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-17e697d1-50a1-4afb-ad15-876e182cbdd2 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] 10.222.0.16 "GET /compute/v2.1/servers/6a391156-ffce-49c0-a79d-47b4544264f5/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.045516 Jul 16 21:13:06.216474 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 620/1326] 10.222.0.16 () {58 vars in 1301 bytes} [Sat Jul 16 21:13:06 2022] GET /compute/v2.1/servers/6a391156-ffce-49c0-a79d-47b4544264f5/os-volume_attachments => generated 25 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Jul 16 21:13:06.283926 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-bf6a898f-a8c6-4729-b79e-4693ad0f2a3c tempest-HypervisorAdminNegativeUnderV252Test-1239772270 tempest-HypervisorAdminNegativeUnderV252Test-1239772270-project-admin] Calling method '.version_select at 0x7fdd9cdf5160>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:13:06.290399 ncdv-master-1 devstack@n-api.service[138680]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-hypervisors:list": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:13:06.290399 ncdv-master-1 devstack@n-api.service[138680]: warnings.warn(msg) Jul 16 21:13:06.291909 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-bf6a898f-a8c6-4729-b79e-4693ad0f2a3c tempest-HypervisorAdminNegativeUnderV252Test-1239772270 tempest-HypervisorAdminNegativeUnderV252Test-1239772270-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:06.292610 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-bf6a898f-a8c6-4729-b79e-4693ad0f2a3c tempest-HypervisorAdminNegativeUnderV252Test-1239772270 tempest-HypervisorAdminNegativeUnderV252Test-1239772270-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:06.308106 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-bf6a898f-a8c6-4729-b79e-4693ad0f2a3c tempest-HypervisorAdminNegativeUnderV252Test-1239772270 tempest-HypervisorAdminNegativeUnderV252Test-1239772270-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:06.308661 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-bf6a898f-a8c6-4729-b79e-4693ad0f2a3c tempest-HypervisorAdminNegativeUnderV252Test-1239772270 tempest-HypervisorAdminNegativeUnderV252Test-1239772270-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:06.321292 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-bf6a898f-a8c6-4729-b79e-4693ad0f2a3c tempest-HypervisorAdminNegativeUnderV252Test-1239772270 tempest-HypervisorAdminNegativeUnderV252Test-1239772270-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:06.321805 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-bf6a898f-a8c6-4729-b79e-4693ad0f2a3c tempest-HypervisorAdminNegativeUnderV252Test-1239772270 tempest-HypervisorAdminNegativeUnderV252Test-1239772270-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:06.325327 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=1,nova_cell1:INSERT=2 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:13:06.329073 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-bf6a898f-a8c6-4729-b79e-4693ad0f2a3c tempest-HypervisorAdminNegativeUnderV252Test-1239772270 tempest-HypervisorAdminNegativeUnderV252Test-1239772270-project-admin] 10.222.0.16 "GET /compute/v2.1/os-hypervisors" status: 200 len: 191 microversion: 2.1 time: 0.181643 Jul 16 21:13:06.329768 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 707/1327] 10.222.0.16 () {58 vars in 1145 bytes} [Sat Jul 16 21:13:06 2022] GET /compute/v2.1/os-hypervisors => generated 191 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:13:06.459902 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-b3c192c1-0848-4182-acb6-6c90dded9b5e tempest-HypervisorAdminNegativeUnderV252Test-509999991 tempest-HypervisorAdminNegativeUnderV252Test-509999991-project] Calling method '.version_select at 0x7fdd9d0e2790>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:13:06.462421 ncdv-master-1 devstack@n-api.service[138681]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-hypervisors:search": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:13:06.462421 ncdv-master-1 devstack@n-api.service[138681]: warnings.warn(msg) Jul 16 21:13:06.464018 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-b3c192c1-0848-4182-acb6-6c90dded9b5e tempest-HypervisorAdminNegativeUnderV252Test-509999991 tempest-HypervisorAdminNegativeUnderV252Test-509999991-project] Policy check for os_compute_api:os-hypervisors:search failed with credentials {'is_admin': False, 'user_id': '702b9008184d401c855087785ce319ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '96d95e580d4543b58779272cd43f48eb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:13:06.466204 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-b3c192c1-0848-4182-acb6-6c90dded9b5e tempest-HypervisorAdminNegativeUnderV252Test-509999991 tempest-HypervisorAdminNegativeUnderV252Test-509999991-project] Request method failure captured: Jul 16 21:13:06.466204 ncdv-master-1 devstack@n-api.service[138681]: request: GET /compute/v2.1/os-hypervisors/nch1-master-1/search HTTP/1.1 Jul 16 21:13:06.466204 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:13:06.466204 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:13:06.466204 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:13:06.466204 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 21:13:06.466204 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:13:06.466204 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:13:06.466204 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:13:06.466204 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:13:06.466204 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0yng3N8LseaF8QTOqw7bADgwIIN16C6wGxW2q1z8LdvHf_nzD9kLeR5MWmYkaiBs73ALXLHhIl3KvvjmHwG4SoKWAzCIyBsPi3j_rxHhKtqDTgshBufcfHvp0T_3QYRBDCl23XLL1Tw910efcV-_PnUIABuMRBsg7o2-91bwCWN14ho Jul 16 21:13:06.466204 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:13:06.466204 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:13:06.466204 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:13:06.466204 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:13:06.466204 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:13:06.466204 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:13:06.466204 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: 96d95e580d4543b58779272cd43f48eb Jul 16 21:13:06.466204 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-HypervisorAdminNegativeUnderV252Test-509999991 Jul 16 21:13:06.466204 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:13:06.466204 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:13:06.467770 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/96d95e580d4543b58779272cd43f48eb"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/96d95e580d4543b58779272cd43f48eb"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_96d95e580d4543b58779272cd43f48eb"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/96d95e580d4543b58779272cd43f48eb"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:13:06.467770 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-HypervisorAdminNegativeUnderV252Test-509999991 Jul 16 21:13:06.467770 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: 96d95e580d4543b58779272cd43f48eb Jul 16 21:13:06.467770 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-HypervisorAdminNegativeUnderV252Test-509999991 Jul 16 21:13:06.467770 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-HypervisorAdminNegativeUnderV252Test-509999991-project Jul 16 21:13:06.467770 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:13:06.467770 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:13:06.467770 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: 702b9008184d401c855087785ce319ad Jul 16 21:13:06.467770 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-HypervisorAdminNegativeUnderV252Test-509999991-project Jul 16 21:13:06.467770 ncdv-master-1 devstack@n-api.service[138681]: method: Jul 16 21:13:06.467770 ncdv-master-1 devstack@n-api.service[138681]: exception: Policy doesn't allow os_compute_api:os-hypervisors:search to be performed. Jul 16 21:13:06.467770 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': 'nch1-master-1'} Jul 16 21:13:06.468664 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:13:06.468664 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:13:06.468664 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:13:06.468664 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:13:06.468664 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:13:06.468664 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:13:06.468664 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Jul 16 21:13:06.468664 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Jul 16 21:13:06.468664 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:13:06.468664 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:13:06.468664 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/hypervisors.py", line 439, in search Jul 16 21:13:06.468664 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi context.can(hv_policies.BASE_POLICY_NAME % 'search', target={}) Jul 16 21:13:06.468664 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/context.py", line 240, in can Jul 16 21:13:06.468664 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return policy.authorize(self, action, target) Jul 16 21:13:06.468664 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 199, in authorize Jul 16 21:13:06.468664 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi LOG.debug('Policy check for %(action)s failed with credentials ' Jul 16 21:13:06.468664 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 16 21:13:06.468664 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi self.force_reraise() Jul 16 21:13:06.468664 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 16 21:13:06.470021 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise self.value Jul 16 21:13:06.470021 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 186, in authorize Jul 16 21:13:06.470021 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi result = _ENFORCER.authorize(action, target, context, Jul 16 21:13:06.470021 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1184, in authorize Jul 16 21:13:06.470021 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return self.enforce( Jul 16 21:13:06.470021 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1088, in enforce Jul 16 21:13:06.470021 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc(*args, **kwargs) Jul 16 21:13:06.470021 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.PolicyNotAuthorized: Policy doesn't allow os_compute_api:os-hypervisors:search to be performed. Jul 16 21:13:06.470021 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:13:06.470021 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:13:06.470021 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:13:06.470021 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:13:06.470021 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:13:06.470021 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:13:06.470021 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 354, in __exit__ Jul 16 21:13:06.470021 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(webob.exc.HTTPForbidden( Jul 16 21:13:06.470021 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Policy doesn't allow os_compute_api:os-hypervisors:search to be performed. Jul 16 21:13:06.470021 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:13:06.471483 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-cc0bf703-5919-4cf7-a6f6-bf67df56583e tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:13:06.472228 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-cc0bf703-5919-4cf7-a6f6-bf67df56583e tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] [instance: 8020f553-6025-4f19-835c-c00e2fb4f764] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:13:06.473643 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-b3c192c1-0848-4182-acb6-6c90dded9b5e tempest-HypervisorAdminNegativeUnderV252Test-509999991 tempest-HypervisorAdminNegativeUnderV252Test-509999991-project] Returning 403 to user: Policy doesn't allow os_compute_api:os-hypervisors:search to be performed. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:13:06.474705 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-b3c192c1-0848-4182-acb6-6c90dded9b5e tempest-HypervisorAdminNegativeUnderV252Test-509999991 tempest-HypervisorAdminNegativeUnderV252Test-509999991-project] 10.222.0.16 "GET /compute/v2.1/os-hypervisors/nch1-master-1/search" status: 403 len: 117 microversion: 2.1 time: 0.137786 Jul 16 21:13:06.476462 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 621/1328] 10.222.0.16 () {58 vars in 1208 bytes} [Sat Jul 16 21:13:06 2022] GET /compute/v2.1/os-hypervisors/nch1-master-1/search => generated 117 bytes in 140 msecs (HTTP/1.1 403) 9 headers in 379 bytes (2 switches on core 0) Jul 16 21:13:06.489246 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-2e0267cd-2638-43bc-8a62-39a73126f81b tempest-HypervisorAdminNegativeUnderV252Test-1239772270 tempest-HypervisorAdminNegativeUnderV252Test-1239772270-project-admin] Calling method '.version_select at 0x7fdd9d090a60>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:13:06.489572 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cc0bf703-5919-4cf7-a6f6-bf67df56583e tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:06.490050 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cc0bf703-5919-4cf7-a6f6-bf67df56583e tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:06.493129 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2e0267cd-2638-43bc-8a62-39a73126f81b tempest-HypervisorAdminNegativeUnderV252Test-1239772270 tempest-HypervisorAdminNegativeUnderV252Test-1239772270-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:06.493348 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cc0bf703-5919-4cf7-a6f6-bf67df56583e tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:06.494031 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2e0267cd-2638-43bc-8a62-39a73126f81b tempest-HypervisorAdminNegativeUnderV252Test-1239772270 tempest-HypervisorAdminNegativeUnderV252Test-1239772270-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:06.494532 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cc0bf703-5919-4cf7-a6f6-bf67df56583e tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:06.513373 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-2e0267cd-2638-43bc-8a62-39a73126f81b tempest-HypervisorAdminNegativeUnderV252Test-1239772270 tempest-HypervisorAdminNegativeUnderV252Test-1239772270-project-admin] HTTP exception thrown: No hypervisor matching 'nonexistent_hypervisor_name' could be found. Jul 16 21:13:06.516809 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-2e0267cd-2638-43bc-8a62-39a73126f81b tempest-HypervisorAdminNegativeUnderV252Test-1239772270 tempest-HypervisorAdminNegativeUnderV252Test-1239772270-project-admin] Request method failure captured: Jul 16 21:13:06.516809 ncdv-master-1 devstack@n-api.service[138681]: request: GET /compute/v2.1/os-hypervisors/nonexistent_hypervisor_name/search HTTP/1.1 Jul 16 21:13:06.516809 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:13:06.516809 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:13:06.516809 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:13:06.516809 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 21:13:06.516809 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:13:06.516809 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:13:06.516809 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:13:06.516809 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:13:06.516809 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0ynhtRxF4kMHk2WTrlkQURniCuoejuZRer0cJL9shKZpfy8iFIE1MdWDHUUURsAGkkRXSRavlR9DeYM11sVqamWbA_7MZJLijjZnQBZL8dmmX1LFNzW6eTW6y0kL5WUI8lEo_h_04IIu8DdcWPDJYauP_KehvKhG5mo89RD36oRVKw8 Jul 16 21:13:06.516809 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:13:06.516809 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:13:06.516809 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:13:06.516809 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:13:06.516809 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:13:06.516809 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:13:06.516809 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: 61f05fc0cab24be5885bd585883e2053 Jul 16 21:13:06.516809 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-HypervisorAdminNegativeUnderV252Test-1239772270 Jul 16 21:13:06.516809 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,admin,reader Jul 16 21:13:06.516809 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,admin,reader Jul 16 21:13:06.518220 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/61f05fc0cab24be5885bd585883e2053"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/61f05fc0cab24be5885bd585883e2053"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_61f05fc0cab24be5885bd585883e2053"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/61f05fc0cab24be5885bd585883e2053"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:13:06.518220 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-HypervisorAdminNegativeUnderV252Test-1239772270 Jul 16 21:13:06.518220 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: 61f05fc0cab24be5885bd585883e2053 Jul 16 21:13:06.518220 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-HypervisorAdminNegativeUnderV252Test-1239772270 Jul 16 21:13:06.518220 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-HypervisorAdminNegativeUnderV252Test-1239772270-project-admin Jul 16 21:13:06.518220 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:13:06.518220 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:13:06.518220 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: 7c815676ecac4f0f89546cbaaa3c447f Jul 16 21:13:06.518220 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-HypervisorAdminNegativeUnderV252Test-1239772270-project-admin Jul 16 21:13:06.518220 ncdv-master-1 devstack@n-api.service[138681]: method: Jul 16 21:13:06.518220 ncdv-master-1 devstack@n-api.service[138681]: exception: No hypervisor matching 'nonexistent_hypervisor_name' could be found. Jul 16 21:13:06.518220 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': 'nonexistent_hypervisor_name'} Jul 16 21:13:06.519151 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:13:06.519151 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:13:06.519151 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:13:06.519151 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:13:06.519151 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:13:06.519151 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:13:06.519151 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Jul 16 21:13:06.519151 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Jul 16 21:13:06.519151 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:13:06.519151 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:13:06.519151 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/hypervisors.py", line 443, in search Jul 16 21:13:06.519151 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi compute_nodes = self._get_compute_nodes_by_name_pattern(context, id) Jul 16 21:13:06.519151 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/hypervisors.py", line 139, in _get_compute_nodes_by_name_pattern Jul 16 21:13:06.519151 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=msg) Jul 16 21:13:06.519151 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: No hypervisor matching 'nonexistent_hypervisor_name' could be found. Jul 16 21:13:06.519151 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:13:06.519151 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:13:06.519151 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:13:06.519151 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:13:06.519151 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:13:06.520341 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:13:06.520341 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:13:06.520341 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:13:06.520341 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: No hypervisor matching 'nonexistent_hypervisor_name' could be found. Jul 16 21:13:06.520341 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:13:06.523847 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-2e0267cd-2638-43bc-8a62-39a73126f81b tempest-HypervisorAdminNegativeUnderV252Test-1239772270 tempest-HypervisorAdminNegativeUnderV252Test-1239772270-project-admin] Returning 404 to user: No hypervisor matching 'nonexistent_hypervisor_name' could be found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:13:06.525043 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-2e0267cd-2638-43bc-8a62-39a73126f81b tempest-HypervisorAdminNegativeUnderV252Test-1239772270 tempest-HypervisorAdminNegativeUnderV252Test-1239772270-project-admin] 10.222.0.16 "GET /compute/v2.1/os-hypervisors/nonexistent_hypervisor_name/search" status: 404 len: 114 microversion: 2.1 time: 0.041772 Jul 16 21:13:06.529812 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 622/1329] 10.222.0.16 () {58 vars in 1250 bytes} [Sat Jul 16 21:13:06 2022] GET /compute/v2.1/os-hypervisors/nonexistent_hypervisor_name/search => generated 114 bytes in 43 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:13:06.536096 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-208a05ff-d5d0-4d84-a338-b4fc56df21a1 tempest-HypervisorAdminNegativeUnderV252Test-1239772270 tempest-HypervisorAdminNegativeUnderV252Test-1239772270-project-admin] Calling method '.version_select at 0x7fdd9cf97310>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:13:06.536996 ncdv-master-1 devstack@n-api.service[138681]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-hypervisors:list": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:13:06.536996 ncdv-master-1 devstack@n-api.service[138681]: warnings.warn(msg) Jul 16 21:13:06.538282 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-208a05ff-d5d0-4d84-a338-b4fc56df21a1 tempest-HypervisorAdminNegativeUnderV252Test-1239772270 tempest-HypervisorAdminNegativeUnderV252Test-1239772270-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:06.539361 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-208a05ff-d5d0-4d84-a338-b4fc56df21a1 tempest-HypervisorAdminNegativeUnderV252Test-1239772270 tempest-HypervisorAdminNegativeUnderV252Test-1239772270-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:06.554788 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-208a05ff-d5d0-4d84-a338-b4fc56df21a1 tempest-HypervisorAdminNegativeUnderV252Test-1239772270 tempest-HypervisorAdminNegativeUnderV252Test-1239772270-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:06.555833 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-208a05ff-d5d0-4d84-a338-b4fc56df21a1 tempest-HypervisorAdminNegativeUnderV252Test-1239772270 tempest-HypervisorAdminNegativeUnderV252Test-1239772270-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:06.568088 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-208a05ff-d5d0-4d84-a338-b4fc56df21a1 tempest-HypervisorAdminNegativeUnderV252Test-1239772270 tempest-HypervisorAdminNegativeUnderV252Test-1239772270-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:06.568712 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-208a05ff-d5d0-4d84-a338-b4fc56df21a1 tempest-HypervisorAdminNegativeUnderV252Test-1239772270 tempest-HypervisorAdminNegativeUnderV252Test-1239772270-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:06.576343 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-208a05ff-d5d0-4d84-a338-b4fc56df21a1 tempest-HypervisorAdminNegativeUnderV252Test-1239772270 tempest-HypervisorAdminNegativeUnderV252Test-1239772270-project-admin] 10.222.0.16 "GET /compute/v2.1/os-hypervisors" status: 200 len: 191 microversion: 2.1 time: 0.044959 Jul 16 21:13:06.577034 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 623/1330] 10.222.0.16 () {58 vars in 1145 bytes} [Sat Jul 16 21:13:06 2022] GET /compute/v2.1/os-hypervisors => generated 191 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:13:06.587745 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-8463b60f-2bb7-4fed-a0da-57022e67bb3b tempest-TenantUsagesTestJSON-260059616 tempest-TenantUsagesTestJSON-260059616-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:13:06.590493 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-8463b60f-2bb7-4fed-a0da-57022e67bb3b tempest-TenantUsagesTestJSON-260059616 tempest-TenantUsagesTestJSON-260059616-project] [instance: 6e68dbd6-1d5c-471e-ad69-cf1c4aa4004a] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:13:06.594695 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8463b60f-2bb7-4fed-a0da-57022e67bb3b tempest-TenantUsagesTestJSON-260059616 tempest-TenantUsagesTestJSON-260059616-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:06.595397 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8463b60f-2bb7-4fed-a0da-57022e67bb3b tempest-TenantUsagesTestJSON-260059616 tempest-TenantUsagesTestJSON-260059616-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:06.596425 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8463b60f-2bb7-4fed-a0da-57022e67bb3b tempest-TenantUsagesTestJSON-260059616 tempest-TenantUsagesTestJSON-260059616-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:06.596942 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8463b60f-2bb7-4fed-a0da-57022e67bb3b tempest-TenantUsagesTestJSON-260059616 tempest-TenantUsagesTestJSON-260059616-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:06.687756 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-cc0bf703-5919-4cf7-a6f6-bf67df56583e tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2dfcad8d2b57481bb33aa8f5d7483c51', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cda86413e74b4c1e8e09bd4df4926664', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:13:06.713811 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-cc0bf703-5919-4cf7-a6f6-bf67df56583e tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] 10.222.0.16 "GET /compute/v2.1/servers/8020f553-6025-4f19-835c-c00e2fb4f764" status: 200 len: 1483 microversion: 2.1 time: 0.256442 Jul 16 21:13:06.714938 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 708/1331] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:13:06 2022] GET /compute/v2.1/servers/8020f553-6025-4f19-835c-c00e2fb4f764 => generated 1483 bytes in 260 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:13:06.723611 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-59097859-d29d-4fdc-b57e-e96fa0801197 tempest-HypervisorAdminNegativeUnderV252Test-509999991 tempest-HypervisorAdminNegativeUnderV252Test-509999991-project] Calling method '.version_select at 0x7fdd9d03ea60>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:13:06.725013 ncdv-master-1 devstack@n-api.service[138680]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-hypervisors:servers": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:13:06.725013 ncdv-master-1 devstack@n-api.service[138680]: warnings.warn(msg) Jul 16 21:13:06.727921 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-59097859-d29d-4fdc-b57e-e96fa0801197 tempest-HypervisorAdminNegativeUnderV252Test-509999991 tempest-HypervisorAdminNegativeUnderV252Test-509999991-project] Policy check for os_compute_api:os-hypervisors:servers failed with credentials {'is_admin': False, 'user_id': '702b9008184d401c855087785ce319ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '96d95e580d4543b58779272cd43f48eb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:13:06.729887 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-59097859-d29d-4fdc-b57e-e96fa0801197 tempest-HypervisorAdminNegativeUnderV252Test-509999991 tempest-HypervisorAdminNegativeUnderV252Test-509999991-project] Request method failure captured: Jul 16 21:13:06.729887 ncdv-master-1 devstack@n-api.service[138680]: request: GET /compute/v2.1/os-hypervisors/1/servers HTTP/1.1 Jul 16 21:13:06.729887 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:13:06.729887 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:13:06.729887 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:13:06.729887 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 0 Jul 16 21:13:06.729887 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:13:06.729887 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:13:06.729887 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:13:06.729887 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:13:06.729887 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0yng3N8LseaF8QTOqw7bADgwIIN16C6wGxW2q1z8LdvHf_nzD9kLeR5MWmYkaiBs73ALXLHhIl3KvvjmHwG4SoKWAzCIyBsPi3j_rxHhKtqDTgshBufcfHvp0T_3QYRBDCl23XLL1Tw910efcV-_PnUIABuMRBsg7o2-91bwCWN14ho Jul 16 21:13:06.729887 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:13:06.729887 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:13:06.729887 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:13:06.729887 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:13:06.729887 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:13:06.729887 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:13:06.729887 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: 96d95e580d4543b58779272cd43f48eb Jul 16 21:13:06.729887 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-HypervisorAdminNegativeUnderV252Test-509999991 Jul 16 21:13:06.729887 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:13:06.729887 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:13:06.731468 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/96d95e580d4543b58779272cd43f48eb"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/96d95e580d4543b58779272cd43f48eb"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_96d95e580d4543b58779272cd43f48eb"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/96d95e580d4543b58779272cd43f48eb"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:13:06.731468 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-HypervisorAdminNegativeUnderV252Test-509999991 Jul 16 21:13:06.731468 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: 96d95e580d4543b58779272cd43f48eb Jul 16 21:13:06.731468 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-HypervisorAdminNegativeUnderV252Test-509999991 Jul 16 21:13:06.731468 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-HypervisorAdminNegativeUnderV252Test-509999991-project Jul 16 21:13:06.731468 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:13:06.731468 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:13:06.731468 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: 702b9008184d401c855087785ce319ad Jul 16 21:13:06.731468 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-HypervisorAdminNegativeUnderV252Test-509999991-project Jul 16 21:13:06.731468 ncdv-master-1 devstack@n-api.service[138680]: method: Jul 16 21:13:06.731468 ncdv-master-1 devstack@n-api.service[138680]: exception: Policy doesn't allow os_compute_api:os-hypervisors:servers to be performed. Jul 16 21:13:06.731468 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': '1'} Jul 16 21:13:06.731468 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:13:06.732248 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:13:06.732248 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:13:06.732248 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:13:06.732248 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:13:06.732248 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:13:06.732248 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Jul 16 21:13:06.732248 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Jul 16 21:13:06.732248 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:13:06.732248 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:13:06.732248 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/hypervisors.py", line 478, in servers Jul 16 21:13:06.732248 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi context.can(hv_policies.BASE_POLICY_NAME % 'servers', target={}) Jul 16 21:13:06.732248 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/context.py", line 240, in can Jul 16 21:13:06.732248 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return policy.authorize(self, action, target) Jul 16 21:13:06.732248 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 199, in authorize Jul 16 21:13:06.732248 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi LOG.debug('Policy check for %(action)s failed with credentials ' Jul 16 21:13:06.732248 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 16 21:13:06.732248 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi self.force_reraise() Jul 16 21:13:06.732248 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 16 21:13:06.732248 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise self.value Jul 16 21:13:06.733365 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 186, in authorize Jul 16 21:13:06.733365 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi result = _ENFORCER.authorize(action, target, context, Jul 16 21:13:06.733365 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1184, in authorize Jul 16 21:13:06.733365 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return self.enforce( Jul 16 21:13:06.733365 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1088, in enforce Jul 16 21:13:06.733365 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc(*args, **kwargs) Jul 16 21:13:06.733365 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.PolicyNotAuthorized: Policy doesn't allow os_compute_api:os-hypervisors:servers to be performed. Jul 16 21:13:06.733365 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:13:06.733365 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:13:06.733365 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:13:06.733365 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:13:06.733365 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:13:06.733365 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:13:06.733365 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 354, in __exit__ Jul 16 21:13:06.733365 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(webob.exc.HTTPForbidden( Jul 16 21:13:06.733365 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Policy doesn't allow os_compute_api:os-hypervisors:servers to be performed. Jul 16 21:13:06.733365 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:13:06.734479 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-59097859-d29d-4fdc-b57e-e96fa0801197 tempest-HypervisorAdminNegativeUnderV252Test-509999991 tempest-HypervisorAdminNegativeUnderV252Test-509999991-project] Returning 403 to user: Policy doesn't allow os_compute_api:os-hypervisors:servers to be performed. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:13:06.734479 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-59097859-d29d-4fdc-b57e-e96fa0801197 tempest-HypervisorAdminNegativeUnderV252Test-509999991 tempest-HypervisorAdminNegativeUnderV252Test-509999991-project] 10.222.0.16 "GET /compute/v2.1/os-hypervisors/1/servers" status: 403 len: 118 microversion: 2.1 time: 0.016901 Jul 16 21:13:06.738682 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 709/1332] 10.222.0.16 () {58 vars in 1175 bytes} [Sat Jul 16 21:13:06 2022] GET /compute/v2.1/os-hypervisors/1/servers => generated 118 bytes in 23 msecs (HTTP/1.1 403) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:13:06.744335 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-500cc9cc-ff77-46d2-a61c-a23ac8897ed0 tempest-HypervisorAdminNegativeUnderV252Test-1239772270 tempest-HypervisorAdminNegativeUnderV252Test-1239772270-project-admin] Calling method '.version_select at 0x7fdd9d4d98b0>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:13:06.746568 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-500cc9cc-ff77-46d2-a61c-a23ac8897ed0 tempest-HypervisorAdminNegativeUnderV252Test-1239772270 tempest-HypervisorAdminNegativeUnderV252Test-1239772270-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:06.747143 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-500cc9cc-ff77-46d2-a61c-a23ac8897ed0 tempest-HypervisorAdminNegativeUnderV252Test-1239772270 tempest-HypervisorAdminNegativeUnderV252Test-1239772270-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:06.754406 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-500cc9cc-ff77-46d2-a61c-a23ac8897ed0 tempest-HypervisorAdminNegativeUnderV252Test-1239772270 tempest-HypervisorAdminNegativeUnderV252Test-1239772270-project-admin] HTTP exception thrown: No hypervisor matching 'd8f6dc99-4061-49f2-b71f-29b6102010cd' could be found. Jul 16 21:13:06.755986 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-500cc9cc-ff77-46d2-a61c-a23ac8897ed0 tempest-HypervisorAdminNegativeUnderV252Test-1239772270 tempest-HypervisorAdminNegativeUnderV252Test-1239772270-project-admin] Request method failure captured: Jul 16 21:13:06.755986 ncdv-master-1 devstack@n-api.service[138680]: request: GET /compute/v2.1/os-hypervisors/d8f6dc99-4061-49f2-b71f-29b6102010cd/servers HTTP/1.1 Jul 16 21:13:06.755986 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:13:06.755986 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:13:06.755986 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:13:06.755986 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 0 Jul 16 21:13:06.755986 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:13:06.755986 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:13:06.755986 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:13:06.755986 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:13:06.755986 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0ynhtRxF4kMHk2WTrlkQURniCuoejuZRer0cJL9shKZpfy8iFIE1MdWDHUUURsAGkkRXSRavlR9DeYM11sVqamWbA_7MZJLijjZnQBZL8dmmX1LFNzW6eTW6y0kL5WUI8lEo_h_04IIu8DdcWPDJYauP_KehvKhG5mo89RD36oRVKw8 Jul 16 21:13:06.755986 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:13:06.755986 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:13:06.755986 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:13:06.755986 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:13:06.755986 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:13:06.755986 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:13:06.755986 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: 61f05fc0cab24be5885bd585883e2053 Jul 16 21:13:06.755986 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-HypervisorAdminNegativeUnderV252Test-1239772270 Jul 16 21:13:06.755986 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,admin,reader Jul 16 21:13:06.755986 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,admin,reader Jul 16 21:13:06.758075 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/61f05fc0cab24be5885bd585883e2053"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/61f05fc0cab24be5885bd585883e2053"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_61f05fc0cab24be5885bd585883e2053"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/61f05fc0cab24be5885bd585883e2053"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:13:06.758075 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-HypervisorAdminNegativeUnderV252Test-1239772270 Jul 16 21:13:06.758075 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: 61f05fc0cab24be5885bd585883e2053 Jul 16 21:13:06.758075 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-HypervisorAdminNegativeUnderV252Test-1239772270 Jul 16 21:13:06.758075 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-HypervisorAdminNegativeUnderV252Test-1239772270-project-admin Jul 16 21:13:06.758075 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:13:06.758075 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:13:06.758075 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: 7c815676ecac4f0f89546cbaaa3c447f Jul 16 21:13:06.758075 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-HypervisorAdminNegativeUnderV252Test-1239772270-project-admin Jul 16 21:13:06.758075 ncdv-master-1 devstack@n-api.service[138680]: method: Jul 16 21:13:06.758075 ncdv-master-1 devstack@n-api.service[138680]: exception: No hypervisor matching 'd8f6dc99-4061-49f2-b71f-29b6102010cd' could be found. Jul 16 21:13:06.758075 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': 'd8f6dc99-4061-49f2-b71f-29b6102010cd'} Jul 16 21:13:06.759680 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:13:06.759680 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:13:06.759680 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:13:06.759680 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:13:06.759680 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:13:06.759680 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:13:06.759680 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Jul 16 21:13:06.759680 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Jul 16 21:13:06.759680 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:13:06.759680 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:13:06.759680 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/hypervisors.py", line 482, in servers Jul 16 21:13:06.759680 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi compute_nodes = self._get_compute_nodes_by_name_pattern(context, id) Jul 16 21:13:06.759680 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/hypervisors.py", line 139, in _get_compute_nodes_by_name_pattern Jul 16 21:13:06.759680 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=msg) Jul 16 21:13:06.759680 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: No hypervisor matching 'd8f6dc99-4061-49f2-b71f-29b6102010cd' could be found. Jul 16 21:13:06.759680 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:13:06.759680 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:13:06.759680 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:13:06.759680 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:13:06.761633 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:13:06.761633 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:13:06.761633 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:13:06.761633 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:13:06.761633 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: No hypervisor matching 'd8f6dc99-4061-49f2-b71f-29b6102010cd' could be found. Jul 16 21:13:06.761633 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:13:06.761633 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-500cc9cc-ff77-46d2-a61c-a23ac8897ed0 tempest-HypervisorAdminNegativeUnderV252Test-1239772270 tempest-HypervisorAdminNegativeUnderV252Test-1239772270-project-admin] Returning 404 to user: No hypervisor matching 'd8f6dc99-4061-49f2-b71f-29b6102010cd' could be found. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:13:06.761633 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-500cc9cc-ff77-46d2-a61c-a23ac8897ed0 tempest-HypervisorAdminNegativeUnderV252Test-1239772270 tempest-HypervisorAdminNegativeUnderV252Test-1239772270-project-admin] 10.222.0.16 "GET /compute/v2.1/os-hypervisors/d8f6dc99-4061-49f2-b71f-29b6102010cd/servers" status: 404 len: 123 microversion: 2.1 time: 0.017522 Jul 16 21:13:06.763252 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 710/1333] 10.222.0.16 () {58 vars in 1280 bytes} [Sat Jul 16 21:13:06 2022] GET /compute/v2.1/os-hypervisors/d8f6dc99-4061-49f2-b71f-29b6102010cd/servers => generated 123 bytes in 24 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:13:07.164903 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-35fcfc45-e113-4be4-9cdd-3537a9558b5c tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:13:07.165669 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-35fcfc45-e113-4be4-9cdd-3537a9558b5c tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] [instance: 2b26717b-355f-417e-8a7c-72c41e1974e1] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:13:07.173625 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-35fcfc45-e113-4be4-9cdd-3537a9558b5c tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:07.174157 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-35fcfc45-e113-4be4-9cdd-3537a9558b5c tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:07.174844 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-35fcfc45-e113-4be4-9cdd-3537a9558b5c tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:07.175320 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-35fcfc45-e113-4be4-9cdd-3537a9558b5c tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:07.269240 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-35fcfc45-e113-4be4-9cdd-3537a9558b5c tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '034c11cc3f174c6fb44c8d64e55cf8b9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb4ff5d7e42940cdb7be71dadd895ee4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:13:07.281393 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-35fcfc45-e113-4be4-9cdd-3537a9558b5c tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] 10.222.0.16 "GET /compute/v2.1/servers/2b26717b-355f-417e-8a7c-72c41e1974e1" status: 200 len: 1632 microversion: 2.1 time: 0.121394 Jul 16 21:13:07.283274 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 711/1334] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:13:07 2022] GET /compute/v2.1/servers/2b26717b-355f-417e-8a7c-72c41e1974e1 => generated 1632 bytes in 124 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:13:07.291113 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-d33f7aa2-c885-4283-99d3-8cfe1deaab95 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:13:07.293675 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-d33f7aa2-c885-4283-99d3-8cfe1deaab95 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: 126f561c-163f-4f6d-8be1-98a2e29b0909] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:13:07.299242 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-8463b60f-2bb7-4fed-a0da-57022e67bb3b tempest-TenantUsagesTestJSON-260059616 tempest-TenantUsagesTestJSON-260059616-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2db871d6aa9746d2b660efbfbdb1dc75', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cad2f07400d54ef9825022d5b3489db0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:13:07.300941 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d33f7aa2-c885-4283-99d3-8cfe1deaab95 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:07.301770 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d33f7aa2-c885-4283-99d3-8cfe1deaab95 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:07.303317 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d33f7aa2-c885-4283-99d3-8cfe1deaab95 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:07.304033 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d33f7aa2-c885-4283-99d3-8cfe1deaab95 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:07.314890 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-8463b60f-2bb7-4fed-a0da-57022e67bb3b tempest-TenantUsagesTestJSON-260059616 tempest-TenantUsagesTestJSON-260059616-project] 10.222.0.16 "GET /compute/v2.1/servers/6e68dbd6-1d5c-471e-ad69-cf1c4aa4004a" status: 200 len: 1314 microversion: 2.1 time: 0.731706 Jul 16 21:13:07.315667 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 624/1335] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:13:06 2022] GET /compute/v2.1/servers/6e68dbd6-1d5c-471e-ad69-cf1c4aa4004a => generated 1314 bytes in 733 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:13:07.323117 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-d5166066-9a12-41a2-99c9-e0d33ded471f req-563db29f-bf7d-4cb1-ab7b-db7f151f1a15 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "8020f553-6025-4f19-835c-c00e2fb4f764", "name": "network-vif-deleted", "tag": "05294e88-e71a-4faa-8acc-79137ca6ee05"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:13:07.346178 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-d5166066-9a12-41a2-99c9-e0d33ded471f req-563db29f-bf7d-4cb1-ab7b-db7f151f1a15 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:07.346635 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-d5166066-9a12-41a2-99c9-e0d33ded471f req-563db29f-bf7d-4cb1-ab7b-db7f151f1a15 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:07.377718 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-d5166066-9a12-41a2-99c9-e0d33ded471f req-563db29f-bf7d-4cb1-ab7b-db7f151f1a15 service nova] Creating event network-vif-deleted:05294e88-e71a-4faa-8acc-79137ca6ee05 for instance 8020f553-6025-4f19-835c-c00e2fb4f764 on nch1-master-1 Jul 16 21:13:07.405690 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-d5166066-9a12-41a2-99c9-e0d33ded471f req-563db29f-bf7d-4cb1-ab7b-db7f151f1a15 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.085018 Jul 16 21:13:07.406359 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 625/1336] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:13:07 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:13:07.620373 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-d33f7aa2-c885-4283-99d3-8cfe1deaab95 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b68dacc4b8463ba07fae6372131117', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55a19b7a4682457a9132f5fc80110712', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:13:07.629335 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-d33f7aa2-c885-4283-99d3-8cfe1deaab95 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "GET /compute/v2.1/servers/126f561c-163f-4f6d-8be1-98a2e29b0909" status: 200 len: 1477 microversion: 2.1 time: 0.344785 Jul 16 21:13:07.633556 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 712/1337] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:13:07 2022] GET /compute/v2.1/servers/126f561c-163f-4f6d-8be1-98a2e29b0909 => generated 1477 bytes in 350 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:13:07.734316 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-5ab0fc45-183d-4978-8734-6d90b1e44100 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:13:07.734316 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-5ab0fc45-183d-4978-8734-6d90b1e44100 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] [instance: 8020f553-6025-4f19-835c-c00e2fb4f764] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:13:07.740290 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5ab0fc45-183d-4978-8734-6d90b1e44100 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:07.741014 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5ab0fc45-183d-4978-8734-6d90b1e44100 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:07.742577 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5ab0fc45-183d-4978-8734-6d90b1e44100 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:07.743242 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5ab0fc45-183d-4978-8734-6d90b1e44100 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:07.779324 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5ab0fc45-183d-4978-8734-6d90b1e44100 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:07.779805 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5ab0fc45-183d-4978-8734-6d90b1e44100 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:07.780272 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.objects.instance [None req-5ab0fc45-183d-4978-8734-6d90b1e44100 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Lazy-loading 'fault' on Instance uuid 8020f553-6025-4f19-835c-c00e2fb4f764 {{(pid=138681) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:13:07.830807 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-5ab0fc45-183d-4978-8734-6d90b1e44100 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2dfcad8d2b57481bb33aa8f5d7483c51', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cda86413e74b4c1e8e09bd4df4926664', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:13:07.838962 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-5ab0fc45-183d-4978-8734-6d90b1e44100 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] 10.222.0.16 "GET /compute/v2.1/servers/8020f553-6025-4f19-835c-c00e2fb4f764" status: 200 len: 1313 microversion: 2.1 time: 0.110619 Jul 16 21:13:07.839574 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 626/1338] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:13:07 2022] GET /compute/v2.1/servers/8020f553-6025-4f19-835c-c00e2fb4f764 => generated 1313 bytes in 112 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:13:08.305774 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-29b18f41-31a2-4e7d-af1f-6b19105de003 tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:13:08.312230 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-29b18f41-31a2-4e7d-af1f-6b19105de003 tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] [instance: 2b26717b-355f-417e-8a7c-72c41e1974e1] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:13:08.320884 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-29b18f41-31a2-4e7d-af1f-6b19105de003 tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:08.321344 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-29b18f41-31a2-4e7d-af1f-6b19105de003 tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:08.322154 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-29b18f41-31a2-4e7d-af1f-6b19105de003 tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:08.322655 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-29b18f41-31a2-4e7d-af1f-6b19105de003 tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:08.343493 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-de9356a4-cb0a-4e58-ac69-117968f492a4 tempest-TenantUsagesTestJSON-260059616 tempest-TenantUsagesTestJSON-260059616-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:13:08.345055 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-de9356a4-cb0a-4e58-ac69-117968f492a4 tempest-TenantUsagesTestJSON-260059616 tempest-TenantUsagesTestJSON-260059616-project] [instance: 6e68dbd6-1d5c-471e-ad69-cf1c4aa4004a] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:13:08.357208 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-de9356a4-cb0a-4e58-ac69-117968f492a4 tempest-TenantUsagesTestJSON-260059616 tempest-TenantUsagesTestJSON-260059616-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:08.357936 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-de9356a4-cb0a-4e58-ac69-117968f492a4 tempest-TenantUsagesTestJSON-260059616 tempest-TenantUsagesTestJSON-260059616-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:08.361162 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-de9356a4-cb0a-4e58-ac69-117968f492a4 tempest-TenantUsagesTestJSON-260059616 tempest-TenantUsagesTestJSON-260059616-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:08.361808 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-de9356a4-cb0a-4e58-ac69-117968f492a4 tempest-TenantUsagesTestJSON-260059616 tempest-TenantUsagesTestJSON-260059616-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:08.378359 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-29b18f41-31a2-4e7d-af1f-6b19105de003 tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:08.378820 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-29b18f41-31a2-4e7d-af1f-6b19105de003 tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:08.379307 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.objects.instance [None req-29b18f41-31a2-4e7d-af1f-6b19105de003 tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] Lazy-loading 'fault' on Instance uuid 2b26717b-355f-417e-8a7c-72c41e1974e1 {{(pid=138680) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:13:08.478591 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-29b18f41-31a2-4e7d-af1f-6b19105de003 tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '034c11cc3f174c6fb44c8d64e55cf8b9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fb4ff5d7e42940cdb7be71dadd895ee4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:13:08.494042 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-29b18f41-31a2-4e7d-af1f-6b19105de003 tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] 10.222.0.16 "GET /compute/v2.1/servers/2b26717b-355f-417e-8a7c-72c41e1974e1" status: 200 len: 1359 microversion: 2.1 time: 0.189459 Jul 16 21:13:08.495395 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 713/1339] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:13:08 2022] GET /compute/v2.1/servers/2b26717b-355f-417e-8a7c-72c41e1974e1 => generated 1359 bytes in 195 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:13:08.604770 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-de9356a4-cb0a-4e58-ac69-117968f492a4 tempest-TenantUsagesTestJSON-260059616 tempest-TenantUsagesTestJSON-260059616-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2db871d6aa9746d2b660efbfbdb1dc75', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cad2f07400d54ef9825022d5b3489db0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:13:08.613571 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-de9356a4-cb0a-4e58-ac69-117968f492a4 tempest-TenantUsagesTestJSON-260059616 tempest-TenantUsagesTestJSON-260059616-project] 10.222.0.16 "GET /compute/v2.1/servers/6e68dbd6-1d5c-471e-ad69-cf1c4aa4004a" status: 200 len: 1314 microversion: 2.1 time: 0.284924 Jul 16 21:13:08.624612 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 627/1340] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:13:08 2022] GET /compute/v2.1/servers/6e68dbd6-1d5c-471e-ad69-cf1c4aa4004a => generated 1314 bytes in 288 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:13:08.654020 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-58206603-4e6d-4c85-b80c-b63abf769a65 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:13:08.654020 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-58206603-4e6d-4c85-b80c-b63abf769a65 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: 126f561c-163f-4f6d-8be1-98a2e29b0909] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:13:08.665200 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-58206603-4e6d-4c85-b80c-b63abf769a65 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:08.665674 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-58206603-4e6d-4c85-b80c-b63abf769a65 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:08.667703 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-58206603-4e6d-4c85-b80c-b63abf769a65 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:08.668166 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-58206603-4e6d-4c85-b80c-b63abf769a65 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:08.847115 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-58206603-4e6d-4c85-b80c-b63abf769a65 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b68dacc4b8463ba07fae6372131117', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55a19b7a4682457a9132f5fc80110712', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:13:08.859524 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-58206603-4e6d-4c85-b80c-b63abf769a65 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "GET /compute/v2.1/servers/126f561c-163f-4f6d-8be1-98a2e29b0909" status: 200 len: 1477 microversion: 2.1 time: 0.212378 Jul 16 21:13:08.860146 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 714/1341] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:13:08 2022] GET /compute/v2.1/servers/126f561c-163f-4f6d-8be1-98a2e29b0909 => generated 1477 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:13:08.865000 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-af5012ef-8147-45d9-9fba-bbe8ce470bb3 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:13:08.866073 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-af5012ef-8147-45d9-9fba-bbe8ce470bb3 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] [instance: 8020f553-6025-4f19-835c-c00e2fb4f764] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:13:08.883073 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-af5012ef-8147-45d9-9fba-bbe8ce470bb3 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:08.883779 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-af5012ef-8147-45d9-9fba-bbe8ce470bb3 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:08.884586 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-af5012ef-8147-45d9-9fba-bbe8ce470bb3 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:08.885030 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-af5012ef-8147-45d9-9fba-bbe8ce470bb3 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:08.902784 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-af5012ef-8147-45d9-9fba-bbe8ce470bb3 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] HTTP exception thrown: Instance 8020f553-6025-4f19-835c-c00e2fb4f764 could not be found. Jul 16 21:13:08.904528 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-af5012ef-8147-45d9-9fba-bbe8ce470bb3 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Request method failure captured: Jul 16 21:13:08.904528 ncdv-master-1 devstack@n-api.service[138681]: request: GET /compute/v2.1/servers/8020f553-6025-4f19-835c-c00e2fb4f764 HTTP/1.1 Jul 16 21:13:08.904528 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:13:08.904528 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:13:08.904528 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:13:08.904528 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 21:13:08.904528 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:13:08.904528 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:13:08.904528 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:13:08.904528 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:13:08.904528 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0ymKM_ECvjVkk6e7K1WKEy-tkQQ7EHRMB-Urg2_4RVouK1u8YOWROAshpDCUoNKJCmCo4JxDMnUkTw8ca8yVeRAtm7eAKieqo3ZQzYMt8n410aWYlQFenytzNBNAOHZnquy4qrFWM3_CaMN-nzEw34w9P4vt5Jr8OZ2O_l_ChXunM28 Jul 16 21:13:08.904528 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:13:08.904528 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:13:08.904528 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:13:08.904528 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:13:08.904528 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:13:08.904528 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:13:08.904528 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: cda86413e74b4c1e8e09bd4df4926664 Jul 16 21:13:08.904528 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-FloatingIPsAssociationTestJSON-1179742205 Jul 16 21:13:08.904528 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:13:08.904528 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:13:08.906185 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/cda86413e74b4c1e8e09bd4df4926664"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/cda86413e74b4c1e8e09bd4df4926664"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_cda86413e74b4c1e8e09bd4df4926664"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/cda86413e74b4c1e8e09bd4df4926664"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:13:08.906185 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-FloatingIPsAssociationTestJSON-1179742205 Jul 16 21:13:08.906185 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: cda86413e74b4c1e8e09bd4df4926664 Jul 16 21:13:08.906185 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-FloatingIPsAssociationTestJSON-1179742205 Jul 16 21:13:08.906185 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-FloatingIPsAssociationTestJSON-1179742205-project Jul 16 21:13:08.906185 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:13:08.906185 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:13:08.906185 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: 2dfcad8d2b57481bb33aa8f5d7483c51 Jul 16 21:13:08.906185 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-FloatingIPsAssociationTestJSON-1179742205-project Jul 16 21:13:08.906185 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:13:08.906185 ncdv-master-1 devstack@n-api.service[138681]: exception: Instance 8020f553-6025-4f19-835c-c00e2fb4f764 could not be found. Jul 16 21:13:08.906185 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': '8020f553-6025-4f19-835c-c00e2fb4f764'} Jul 16 21:13:08.911290 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:13:08.911290 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:13:08.911290 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:13:08.911290 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:13:08.911290 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:13:08.911290 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:13:08.911290 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:13:08.911290 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:13:08.911290 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:13:08.911290 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:13:08.911290 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 8020f553-6025-4f19-835c-c00e2fb4f764 could not be found. Jul 16 21:13:08.911290 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:13:08.911290 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:13:08.911290 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:13:08.911290 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:13:08.911290 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:13:08.911290 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:13:08.911290 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:13:08.911290 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:13:08.911290 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:13:08.915406 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:13:08.915406 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:13:08.915406 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:13:08.915406 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:13:08.915406 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:13:08.915406 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:13:08.915406 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:13:08.915406 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 8020f553-6025-4f19-835c-c00e2fb4f764 could not be found. Jul 16 21:13:08.915406 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:13:08.915406 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:13:08.915406 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:13:08.915406 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:13:08.915406 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:13:08.915406 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:13:08.915406 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:13:08.915406 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:13:08.915406 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 8020f553-6025-4f19-835c-c00e2fb4f764 could not be found. Jul 16 21:13:08.915406 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:13:08.918255 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-af5012ef-8147-45d9-9fba-bbe8ce470bb3 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] Returning 404 to user: Instance 8020f553-6025-4f19-835c-c00e2fb4f764 could not be found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:13:08.918255 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-af5012ef-8147-45d9-9fba-bbe8ce470bb3 tempest-FloatingIPsAssociationTestJSON-1179742205 tempest-FloatingIPsAssociationTestJSON-1179742205-project] 10.222.0.16 "GET /compute/v2.1/servers/8020f553-6025-4f19-835c-c00e2fb4f764" status: 404 len: 111 microversion: 2.1 time: 0.045932 Jul 16 21:13:08.918255 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 628/1342] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:13:08 2022] GET /compute/v2.1/servers/8020f553-6025-4f19-835c-c00e2fb4f764 => generated 111 bytes in 47 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:13:09.085194 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-31c156e0-c933-42cd-bd5e-8f15739173de req-83721898-c0a5-480f-8a51-292f4fd7594f service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "6e68dbd6-1d5c-471e-ad69-cf1c4aa4004a", "name": "network-vif-plugged", "status": "completed", "tag": "ca8395d6-64dd-4e53-92a0-b2a79f375684"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:13:09.098789 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-31c156e0-c933-42cd-bd5e-8f15739173de req-83721898-c0a5-480f-8a51-292f4fd7594f service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:09.099372 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-31c156e0-c933-42cd-bd5e-8f15739173de req-83721898-c0a5-480f-8a51-292f4fd7594f service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:09.113342 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-31c156e0-c933-42cd-bd5e-8f15739173de req-83721898-c0a5-480f-8a51-292f4fd7594f service nova] Creating event network-vif-plugged:ca8395d6-64dd-4e53-92a0-b2a79f375684 for instance 6e68dbd6-1d5c-471e-ad69-cf1c4aa4004a on nch2-master-1 Jul 16 21:13:09.121389 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-31c156e0-c933-42cd-bd5e-8f15739173de req-83721898-c0a5-480f-8a51-292f4fd7594f service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.040321 Jul 16 21:13:09.122493 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 715/1343] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:13:09 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:13:09.437778 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-798c04a2-7c12-4a12-9297-f9283e9db6bc req-147feeeb-8681-4cb4-889e-518b0f677557 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "2b26717b-355f-417e-8a7c-72c41e1974e1", "name": "network-vif-deleted", "tag": "c64dc016-0b59-43fd-8f3b-dcfc4298e2bd"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:13:09.447808 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-798c04a2-7c12-4a12-9297-f9283e9db6bc req-147feeeb-8681-4cb4-889e-518b0f677557 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:09.448323 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-798c04a2-7c12-4a12-9297-f9283e9db6bc req-147feeeb-8681-4cb4-889e-518b0f677557 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:09.465662 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-798c04a2-7c12-4a12-9297-f9283e9db6bc req-147feeeb-8681-4cb4-889e-518b0f677557 service nova] Creating event network-vif-deleted:c64dc016-0b59-43fd-8f3b-dcfc4298e2bd for instance 2b26717b-355f-417e-8a7c-72c41e1974e1 on nch2-master-1 Jul 16 21:13:09.472625 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-798c04a2-7c12-4a12-9297-f9283e9db6bc req-147feeeb-8681-4cb4-889e-518b0f677557 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.038378 Jul 16 21:13:09.473763 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 629/1344] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:13:09 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:13:09.513118 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-e4c4376a-b48f-4e79-a894-f251724224f4 tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:13:09.514090 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-e4c4376a-b48f-4e79-a894-f251724224f4 tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] [instance: 2b26717b-355f-417e-8a7c-72c41e1974e1] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:13:09.524210 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell0:SELECT=4 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:13:09.527728 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell0:SELECT=2 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:13:09.537991 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e4c4376a-b48f-4e79-a894-f251724224f4 tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:09.538490 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e4c4376a-b48f-4e79-a894-f251724224f4 tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:09.539355 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e4c4376a-b48f-4e79-a894-f251724224f4 tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:09.540050 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e4c4376a-b48f-4e79-a894-f251724224f4 tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:09.558620 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-e4c4376a-b48f-4e79-a894-f251724224f4 tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] HTTP exception thrown: Instance 2b26717b-355f-417e-8a7c-72c41e1974e1 could not be found. Jul 16 21:13:09.560620 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-e4c4376a-b48f-4e79-a894-f251724224f4 tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] Request method failure captured: Jul 16 21:13:09.560620 ncdv-master-1 devstack@n-api.service[138680]: request: GET /compute/v2.1/servers/2b26717b-355f-417e-8a7c-72c41e1974e1 HTTP/1.1 Jul 16 21:13:09.560620 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:13:09.560620 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:13:09.560620 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:13:09.560620 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 0 Jul 16 21:13:09.560620 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:13:09.560620 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:13:09.560620 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:13:09.560620 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:13:09.560620 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0ymL--epueWWGpSjHZmJWp5xAkpoWHpumAaxKSXe5M7Zd0q69_C1yohOZRGVLUkKlP8on8JXwhrwjhXzR8jwRKRbS19xOxwyJmqhkesgCGpj-4oKAejvbgaPKH66egSzaaATfV2tcQtkNYoNONJO992_-BNCdqKJRIzjrDLg5b0HcM0 Jul 16 21:13:09.560620 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:13:09.560620 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:13:09.560620 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:13:09.560620 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:13:09.560620 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:13:09.560620 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:13:09.560620 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: fb4ff5d7e42940cdb7be71dadd895ee4 Jul 16 21:13:09.560620 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-ServersTestJSON-27104667 Jul 16 21:13:09.560620 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:13:09.560620 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:13:09.562226 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/fb4ff5d7e42940cdb7be71dadd895ee4"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/fb4ff5d7e42940cdb7be71dadd895ee4"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_fb4ff5d7e42940cdb7be71dadd895ee4"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/fb4ff5d7e42940cdb7be71dadd895ee4"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:13:09.562226 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-ServersTestJSON-27104667 Jul 16 21:13:09.562226 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: fb4ff5d7e42940cdb7be71dadd895ee4 Jul 16 21:13:09.562226 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-ServersTestJSON-27104667 Jul 16 21:13:09.562226 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-ServersTestJSON-27104667-project Jul 16 21:13:09.562226 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:13:09.562226 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:13:09.562226 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: 034c11cc3f174c6fb44c8d64e55cf8b9 Jul 16 21:13:09.562226 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-ServersTestJSON-27104667-project Jul 16 21:13:09.562226 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 21:13:09.562226 ncdv-master-1 devstack@n-api.service[138680]: exception: Instance 2b26717b-355f-417e-8a7c-72c41e1974e1 could not be found. Jul 16 21:13:09.562226 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': '2b26717b-355f-417e-8a7c-72c41e1974e1'} Jul 16 21:13:09.562226 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:13:09.569264 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:13:09.569264 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:13:09.569264 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:13:09.569264 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:13:09.569264 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:13:09.569264 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:13:09.569264 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:13:09.569264 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:13:09.569264 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:13:09.569264 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 2b26717b-355f-417e-8a7c-72c41e1974e1 could not be found. Jul 16 21:13:09.569264 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:13:09.569264 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:13:09.569264 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:13:09.569264 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:13:09.569264 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:13:09.569264 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:13:09.569264 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:13:09.569264 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:13:09.569264 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:13:09.569264 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:13:09.571421 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:13:09.571421 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:13:09.571421 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:13:09.571421 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:13:09.571421 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:13:09.571421 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:13:09.571421 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 2b26717b-355f-417e-8a7c-72c41e1974e1 could not be found. Jul 16 21:13:09.571421 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:13:09.571421 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:13:09.571421 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:13:09.571421 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:13:09.571421 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:13:09.571421 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:13:09.571421 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:13:09.571421 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:13:09.571421 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 2b26717b-355f-417e-8a7c-72c41e1974e1 could not be found. Jul 16 21:13:09.571421 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:13:09.572652 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-e4c4376a-b48f-4e79-a894-f251724224f4 tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] Returning 404 to user: Instance 2b26717b-355f-417e-8a7c-72c41e1974e1 could not be found. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:13:09.572652 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-e4c4376a-b48f-4e79-a894-f251724224f4 tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] 10.222.0.16 "GET /compute/v2.1/servers/2b26717b-355f-417e-8a7c-72c41e1974e1" status: 404 len: 111 microversion: 2.1 time: 0.055904 Jul 16 21:13:09.572652 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 716/1345] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:13:09 2022] GET /compute/v2.1/servers/2b26717b-355f-417e-8a7c-72c41e1974e1 => generated 111 bytes in 58 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:13:09.576027 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-36a8e212-f4da-4ff0-9ff1-54eecb290f02 tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] Calling method '.version_select at 0x7fdd9d0a05e0>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:13:09.634464 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-4ec699e2-b0f4-4b77-9a51-c7de27d80c20 tempest-TenantUsagesTestJSON-260059616 tempest-TenantUsagesTestJSON-260059616-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:13:09.635682 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-4ec699e2-b0f4-4b77-9a51-c7de27d80c20 tempest-TenantUsagesTestJSON-260059616 tempest-TenantUsagesTestJSON-260059616-project] [instance: 6e68dbd6-1d5c-471e-ad69-cf1c4aa4004a] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:13:09.643266 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4ec699e2-b0f4-4b77-9a51-c7de27d80c20 tempest-TenantUsagesTestJSON-260059616 tempest-TenantUsagesTestJSON-260059616-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:09.648369 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4ec699e2-b0f4-4b77-9a51-c7de27d80c20 tempest-TenantUsagesTestJSON-260059616 tempest-TenantUsagesTestJSON-260059616-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:09.648369 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4ec699e2-b0f4-4b77-9a51-c7de27d80c20 tempest-TenantUsagesTestJSON-260059616 tempest-TenantUsagesTestJSON-260059616-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:09.648369 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4ec699e2-b0f4-4b77-9a51-c7de27d80c20 tempest-TenantUsagesTestJSON-260059616 tempest-TenantUsagesTestJSON-260059616-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:09.703997 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-36a8e212-f4da-4ff0-9ff1-54eecb290f02 tempest-ServersTestJSON-27104667 tempest-ServersTestJSON-27104667-project] 10.222.0.16 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-280247600" status: 202 len: 0 microversion: 2.1 time: 0.133248 Jul 16 21:13:09.708948 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 630/1346] 10.222.0.16 () {58 vars in 1217 bytes} [Sat Jul 16 21:13:09 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-280247600 => generated 0 bytes in 139 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 21:13:09.876662 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-4ec699e2-b0f4-4b77-9a51-c7de27d80c20 tempest-TenantUsagesTestJSON-260059616 tempest-TenantUsagesTestJSON-260059616-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2db871d6aa9746d2b660efbfbdb1dc75', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cad2f07400d54ef9825022d5b3489db0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:13:09.880451 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-0554323f-0fe1-416c-81dd-1ac16f5cf9dc tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:13:09.881246 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-0554323f-0fe1-416c-81dd-1ac16f5cf9dc tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: 126f561c-163f-4f6d-8be1-98a2e29b0909] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:13:09.886376 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-4ec699e2-b0f4-4b77-9a51-c7de27d80c20 tempest-TenantUsagesTestJSON-260059616 tempest-TenantUsagesTestJSON-260059616-project] 10.222.0.16 "GET /compute/v2.1/servers/6e68dbd6-1d5c-471e-ad69-cf1c4aa4004a" status: 200 len: 1314 microversion: 2.1 time: 0.257274 Jul 16 21:13:09.889680 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 717/1347] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:13:09 2022] GET /compute/v2.1/servers/6e68dbd6-1d5c-471e-ad69-cf1c4aa4004a => generated 1314 bytes in 261 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:13:09.900997 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0554323f-0fe1-416c-81dd-1ac16f5cf9dc tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:09.901861 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0554323f-0fe1-416c-81dd-1ac16f5cf9dc tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:09.903268 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0554323f-0fe1-416c-81dd-1ac16f5cf9dc tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:09.904623 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0554323f-0fe1-416c-81dd-1ac16f5cf9dc tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:10.131468 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-0554323f-0fe1-416c-81dd-1ac16f5cf9dc tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b68dacc4b8463ba07fae6372131117', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55a19b7a4682457a9132f5fc80110712', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:13:10.140652 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-0554323f-0fe1-416c-81dd-1ac16f5cf9dc tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "GET /compute/v2.1/servers/126f561c-163f-4f6d-8be1-98a2e29b0909" status: 200 len: 1477 microversion: 2.1 time: 0.266651 Jul 16 21:13:10.152247 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 631/1348] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:13:09 2022] GET /compute/v2.1/servers/126f561c-163f-4f6d-8be1-98a2e29b0909 => generated 1477 bytes in 279 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:13:10.910948 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-84c15569-f2e5-4cb8-9646-035213d691a2 tempest-TenantUsagesTestJSON-260059616 tempest-TenantUsagesTestJSON-260059616-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:13:10.911681 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-84c15569-f2e5-4cb8-9646-035213d691a2 tempest-TenantUsagesTestJSON-260059616 tempest-TenantUsagesTestJSON-260059616-project] [instance: 6e68dbd6-1d5c-471e-ad69-cf1c4aa4004a] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:13:10.925075 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-84c15569-f2e5-4cb8-9646-035213d691a2 tempest-TenantUsagesTestJSON-260059616 tempest-TenantUsagesTestJSON-260059616-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:10.925542 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-84c15569-f2e5-4cb8-9646-035213d691a2 tempest-TenantUsagesTestJSON-260059616 tempest-TenantUsagesTestJSON-260059616-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:10.926359 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-84c15569-f2e5-4cb8-9646-035213d691a2 tempest-TenantUsagesTestJSON-260059616 tempest-TenantUsagesTestJSON-260059616-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:10.926803 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-84c15569-f2e5-4cb8-9646-035213d691a2 tempest-TenantUsagesTestJSON-260059616 tempest-TenantUsagesTestJSON-260059616-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:11.157745 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-7be50540-6bc0-4d62-b2b2-701972fdc167 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:13:11.158429 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-7be50540-6bc0-4d62-b2b2-701972fdc167 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: 126f561c-163f-4f6d-8be1-98a2e29b0909] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:13:11.174394 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7be50540-6bc0-4d62-b2b2-701972fdc167 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:11.177201 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7be50540-6bc0-4d62-b2b2-701972fdc167 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:11.180012 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7be50540-6bc0-4d62-b2b2-701972fdc167 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:11.180685 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7be50540-6bc0-4d62-b2b2-701972fdc167 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:11.240830 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-84c15569-f2e5-4cb8-9646-035213d691a2 tempest-TenantUsagesTestJSON-260059616 tempest-TenantUsagesTestJSON-260059616-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2db871d6aa9746d2b660efbfbdb1dc75', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cad2f07400d54ef9825022d5b3489db0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:13:11.271400 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-84c15569-f2e5-4cb8-9646-035213d691a2 tempest-TenantUsagesTestJSON-260059616 tempest-TenantUsagesTestJSON-260059616-project] 10.222.0.16 "GET /compute/v2.1/servers/6e68dbd6-1d5c-471e-ad69-cf1c4aa4004a" status: 200 len: 1464 microversion: 2.1 time: 0.361478 Jul 16 21:13:11.288322 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 718/1349] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:13:10 2022] GET /compute/v2.1/servers/6e68dbd6-1d5c-471e-ad69-cf1c4aa4004a => generated 1464 bytes in 383 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:13:11.634340 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-7be50540-6bc0-4d62-b2b2-701972fdc167 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b68dacc4b8463ba07fae6372131117', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55a19b7a4682457a9132f5fc80110712', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:13:11.648235 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-7be50540-6bc0-4d62-b2b2-701972fdc167 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "GET /compute/v2.1/servers/126f561c-163f-4f6d-8be1-98a2e29b0909" status: 200 len: 1477 microversion: 2.1 time: 0.493104 Jul 16 21:13:11.662901 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 632/1350] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:13:11 2022] GET /compute/v2.1/servers/126f561c-163f-4f6d-8be1-98a2e29b0909 => generated 1477 bytes in 506 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:13:11.672222 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-edc2fdc6-440a-47d9-b20d-bddc75107bea tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:13:11.675418 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-edc2fdc6-440a-47d9-b20d-bddc75107bea tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] [instance: 08e7e1f9-8b5c-446c-a05f-d8043473dd35] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:13:11.678811 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-c23ff992-5b7c-4d9f-a2af-ff502c2596b0 tempest-TenantUsagesTestJSON-1328999348 tempest-TenantUsagesTestJSON-1328999348-project-admin] Calling method '.version_select at 0x7fdd9cdf5430>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:13:11.697447 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c23ff992-5b7c-4d9f-a2af-ff502c2596b0 tempest-TenantUsagesTestJSON-1328999348 tempest-TenantUsagesTestJSON-1328999348-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:11.698630 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c23ff992-5b7c-4d9f-a2af-ff502c2596b0 tempest-TenantUsagesTestJSON-1328999348 tempest-TenantUsagesTestJSON-1328999348-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:11.699771 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-edc2fdc6-440a-47d9-b20d-bddc75107bea tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:11.701034 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-edc2fdc6-440a-47d9-b20d-bddc75107bea tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:11.705569 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-edc2fdc6-440a-47d9-b20d-bddc75107bea tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:11.713860 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-edc2fdc6-440a-47d9-b20d-bddc75107bea tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:11.747548 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c23ff992-5b7c-4d9f-a2af-ff502c2596b0 tempest-TenantUsagesTestJSON-1328999348 tempest-TenantUsagesTestJSON-1328999348-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:11.748624 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c23ff992-5b7c-4d9f-a2af-ff502c2596b0 tempest-TenantUsagesTestJSON-1328999348 tempest-TenantUsagesTestJSON-1328999348-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:11.771266 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-edc2fdc6-440a-47d9-b20d-bddc75107bea tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] [instance: 08e7e1f9-8b5c-446c-a05f-d8043473dd35] Going to try to terminate instance {{(pid=138681) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 16 21:13:11.785401 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-c23ff992-5b7c-4d9f-a2af-ff502c2596b0 tempest-TenantUsagesTestJSON-1328999348 tempest-TenantUsagesTestJSON-1328999348-project-admin] 10.222.0.16 "GET /compute/v2.1/os-simple-tenant-usage/cad2f07400d54ef9825022d5b3489db0?start=2022-07-15T21%3A13%3A11.284966&end=2022-07-17T21%3A13%3A11.284966" status: 200 len: 700 microversion: 2.1 time: 0.494716 Jul 16 21:13:11.793082 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 719/1351] 10.222.0.16 () {58 vars in 1411 bytes} [Sat Jul 16 21:13:11 2022] GET /compute/v2.1/os-simple-tenant-usage/cad2f07400d54ef9825022d5b3489db0?start=2022-07-15T21%3A13%3A11.284966&end=2022-07-17T21%3A13%3A11.284966 => generated 700 bytes in 501 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:13:11.804842 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-a256da45-5b33-4186-a7f8-a1e655b8b39f tempest-TenantUsagesTestJSON-260059616 tempest-TenantUsagesTestJSON-260059616-project] Calling method '.version_select at 0x7fdd9cdd64c0>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:13:11.829561 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a256da45-5b33-4186-a7f8-a1e655b8b39f tempest-TenantUsagesTestJSON-260059616 tempest-TenantUsagesTestJSON-260059616-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:11.830469 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a256da45-5b33-4186-a7f8-a1e655b8b39f tempest-TenantUsagesTestJSON-260059616 tempest-TenantUsagesTestJSON-260059616-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:11.862913 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:UPDATE=2,nova_cell1:SELECT=97,nova_cell1:INSERT=1 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:13:11.863505 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a256da45-5b33-4186-a7f8-a1e655b8b39f tempest-TenantUsagesTestJSON-260059616 tempest-TenantUsagesTestJSON-260059616-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:11.868968 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a256da45-5b33-4186-a7f8-a1e655b8b39f tempest-TenantUsagesTestJSON-260059616 tempest-TenantUsagesTestJSON-260059616-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:11.900795 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-a256da45-5b33-4186-a7f8-a1e655b8b39f tempest-TenantUsagesTestJSON-260059616 tempest-TenantUsagesTestJSON-260059616-project] 10.222.0.16 "GET /compute/v2.1/os-simple-tenant-usage/cad2f07400d54ef9825022d5b3489db0?start=2022-07-15T21%3A13%3A11.284966&end=2022-07-17T21%3A13%3A11.284966" status: 200 len: 695 microversion: 2.1 time: 0.098006 Jul 16 21:13:11.904303 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 720/1352] 10.222.0.16 () {58 vars in 1411 bytes} [Sat Jul 16 21:13:11 2022] GET /compute/v2.1/os-simple-tenant-usage/cad2f07400d54ef9825022d5b3489db0?start=2022-07-15T21%3A13%3A11.284966&end=2022-07-17T21%3A13%3A11.284966 => generated 695 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 16 21:13:11.920514 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-39bf738d-936c-4c85-8c3d-ca9e8e66948e tempest-TenantUsagesTestJSON-1328999348 tempest-TenantUsagesTestJSON-1328999348-project-admin] Calling method '.version_select at 0x7fdd9cdf5ee0>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:13:11.939630 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-39bf738d-936c-4c85-8c3d-ca9e8e66948e tempest-TenantUsagesTestJSON-1328999348 tempest-TenantUsagesTestJSON-1328999348-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:11.940383 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-39bf738d-936c-4c85-8c3d-ca9e8e66948e tempest-TenantUsagesTestJSON-1328999348 tempest-TenantUsagesTestJSON-1328999348-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:11.965635 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-39bf738d-936c-4c85-8c3d-ca9e8e66948e tempest-TenantUsagesTestJSON-1328999348 tempest-TenantUsagesTestJSON-1328999348-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:11.966137 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-39bf738d-936c-4c85-8c3d-ca9e8e66948e tempest-TenantUsagesTestJSON-1328999348 tempest-TenantUsagesTestJSON-1328999348-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:12.053027 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-39bf738d-936c-4c85-8c3d-ca9e8e66948e tempest-TenantUsagesTestJSON-1328999348 tempest-TenantUsagesTestJSON-1328999348-project-admin] 10.222.0.16 "GET /compute/v2.1/os-simple-tenant-usage?start=2022-07-15T21%3A13%3A11.284966&end=2022-07-17T21%3A13%3A11.284966&detailed=1" status: 200 len: 16878 microversion: 2.1 time: 0.138478 Jul 16 21:13:12.055062 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 721/1353] 10.222.0.16 () {58 vars in 1334 bytes} [Sat Jul 16 21:13:11 2022] GET /compute/v2.1/os-simple-tenant-usage?start=2022-07-15T21%3A13%3A11.284966&end=2022-07-17T21%3A13%3A11.284966&detailed=1 => generated 16878 bytes in 142 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 16 21:13:12.083261 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-705228bf-ad28-47d2-bc1e-fedf44d5dba3 tempest-TenantUsagesTestJSON-260059616 tempest-TenantUsagesTestJSON-260059616-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:13:12.085503 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-705228bf-ad28-47d2-bc1e-fedf44d5dba3 tempest-TenantUsagesTestJSON-260059616 tempest-TenantUsagesTestJSON-260059616-project] [instance: 6e68dbd6-1d5c-471e-ad69-cf1c4aa4004a] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:13:12.105521 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-705228bf-ad28-47d2-bc1e-fedf44d5dba3 tempest-TenantUsagesTestJSON-260059616 tempest-TenantUsagesTestJSON-260059616-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:12.106364 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-705228bf-ad28-47d2-bc1e-fedf44d5dba3 tempest-TenantUsagesTestJSON-260059616 tempest-TenantUsagesTestJSON-260059616-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:12.107794 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-705228bf-ad28-47d2-bc1e-fedf44d5dba3 tempest-TenantUsagesTestJSON-260059616 tempest-TenantUsagesTestJSON-260059616-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:12.110010 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-705228bf-ad28-47d2-bc1e-fedf44d5dba3 tempest-TenantUsagesTestJSON-260059616 tempest-TenantUsagesTestJSON-260059616-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:12.162458 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-705228bf-ad28-47d2-bc1e-fedf44d5dba3 tempest-TenantUsagesTestJSON-260059616 tempest-TenantUsagesTestJSON-260059616-project] [instance: 6e68dbd6-1d5c-471e-ad69-cf1c4aa4004a] Going to try to terminate instance {{(pid=138680) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 16 21:13:12.189696 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-edc2fdc6-440a-47d9-b20d-bddc75107bea tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] 10.222.0.16 "DELETE /compute/v2.1/servers/08e7e1f9-8b5c-446c-a05f-d8043473dd35" status: 204 len: 0 microversion: 2.1 time: 0.521719 Jul 16 21:13:12.189696 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 633/1354] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:13:11 2022] DELETE /compute/v2.1/servers/08e7e1f9-8b5c-446c-a05f-d8043473dd35 => generated 0 bytes in 523 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 16 21:13:12.201907 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-b6a74530-368d-46ba-8f97-0d12f5434cd4 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:13:12.203745 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-b6a74530-368d-46ba-8f97-0d12f5434cd4 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] [instance: 08e7e1f9-8b5c-446c-a05f-d8043473dd35] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:13:12.211824 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b6a74530-368d-46ba-8f97-0d12f5434cd4 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:12.212579 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b6a74530-368d-46ba-8f97-0d12f5434cd4 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:12.214270 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b6a74530-368d-46ba-8f97-0d12f5434cd4 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:12.214873 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b6a74530-368d-46ba-8f97-0d12f5434cd4 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:12.494575 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-b6a74530-368d-46ba-8f97-0d12f5434cd4 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '52281b3989a04ca2a50b0bc93bf783f0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '30d028fa9b854c8fb71c8cbed2985c27', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:13:12.506733 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-b6a74530-368d-46ba-8f97-0d12f5434cd4 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] 10.222.0.16 "GET /compute/v2.1/servers/08e7e1f9-8b5c-446c-a05f-d8043473dd35" status: 200 len: 1514 microversion: 2.1 time: 0.313558 Jul 16 21:13:12.508413 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 634/1355] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:13:12 2022] GET /compute/v2.1/servers/08e7e1f9-8b5c-446c-a05f-d8043473dd35 => generated 1514 bytes in 316 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:13:12.542776 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-705228bf-ad28-47d2-bc1e-fedf44d5dba3 tempest-TenantUsagesTestJSON-260059616 tempest-TenantUsagesTestJSON-260059616-project] 10.222.0.16 "DELETE /compute/v2.1/servers/6e68dbd6-1d5c-471e-ad69-cf1c4aa4004a" status: 204 len: 0 microversion: 2.1 time: 0.464564 Jul 16 21:13:12.543413 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 722/1356] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:13:12 2022] DELETE /compute/v2.1/servers/6e68dbd6-1d5c-471e-ad69-cf1c4aa4004a => generated 0 bytes in 467 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Jul 16 21:13:12.551819 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-b23ea305-d90b-4be6-94da-235e2c5f6a30 tempest-TenantUsagesTestJSON-260059616 tempest-TenantUsagesTestJSON-260059616-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:13:12.552530 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-b23ea305-d90b-4be6-94da-235e2c5f6a30 tempest-TenantUsagesTestJSON-260059616 tempest-TenantUsagesTestJSON-260059616-project] [instance: 6e68dbd6-1d5c-471e-ad69-cf1c4aa4004a] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:13:12.560370 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b23ea305-d90b-4be6-94da-235e2c5f6a30 tempest-TenantUsagesTestJSON-260059616 tempest-TenantUsagesTestJSON-260059616-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:12.560794 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b23ea305-d90b-4be6-94da-235e2c5f6a30 tempest-TenantUsagesTestJSON-260059616 tempest-TenantUsagesTestJSON-260059616-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:12.561702 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b23ea305-d90b-4be6-94da-235e2c5f6a30 tempest-TenantUsagesTestJSON-260059616 tempest-TenantUsagesTestJSON-260059616-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:12.562183 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b23ea305-d90b-4be6-94da-235e2c5f6a30 tempest-TenantUsagesTestJSON-260059616 tempest-TenantUsagesTestJSON-260059616-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:12.676321 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-86e44c40-0a7e-4434-912c-48b9359c0838 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:13:12.676321 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-86e44c40-0a7e-4434-912c-48b9359c0838 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: 126f561c-163f-4f6d-8be1-98a2e29b0909] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:13:12.686125 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-86e44c40-0a7e-4434-912c-48b9359c0838 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:12.686588 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-86e44c40-0a7e-4434-912c-48b9359c0838 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:12.687425 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-86e44c40-0a7e-4434-912c-48b9359c0838 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:12.687867 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-86e44c40-0a7e-4434-912c-48b9359c0838 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:12.954351 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-b23ea305-d90b-4be6-94da-235e2c5f6a30 tempest-TenantUsagesTestJSON-260059616 tempest-TenantUsagesTestJSON-260059616-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2db871d6aa9746d2b660efbfbdb1dc75', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cad2f07400d54ef9825022d5b3489db0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:13:12.967504 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-b23ea305-d90b-4be6-94da-235e2c5f6a30 tempest-TenantUsagesTestJSON-260059616 tempest-TenantUsagesTestJSON-260059616-project] 10.222.0.16 "GET /compute/v2.1/servers/6e68dbd6-1d5c-471e-ad69-cf1c4aa4004a" status: 200 len: 1470 microversion: 2.1 time: 0.419813 Jul 16 21:13:12.980198 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 635/1357] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:13:12 2022] GET /compute/v2.1/servers/6e68dbd6-1d5c-471e-ad69-cf1c4aa4004a => generated 1470 bytes in 433 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:13:13.181126 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-e999c6b6-43d9-4687-8969-23fddf60cf11 tempest-InstanceUsageAuditLogTestJSON-47970016 tempest-InstanceUsageAuditLogTestJSON-47970016-project-admin] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:13:13.184951 ncdv-master-1 devstack@n-api.service[138681]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-instance-usage-audit-log:show": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:13:13.184951 ncdv-master-1 devstack@n-api.service[138681]: warnings.warn(msg) Jul 16 21:13:13.207767 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-e999c6b6-43d9-4687-8969-23fddf60cf11 tempest-InstanceUsageAuditLogTestJSON-47970016 tempest-InstanceUsageAuditLogTestJSON-47970016-project-admin] 10.222.0.16 "GET /compute/v2.1/os-instance_usage_audit_log/2022-07-16 21:13:12" status: 200 len: 318 microversion: 2.1 time: 0.226049 Jul 16 21:13:13.218664 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 636/1358] 10.222.0.16 () {58 vars in 1250 bytes} [Sat Jul 16 21:13:12 2022] GET /compute/v2.1/os-instance_usage_audit_log/2022-07-16%2021%3A13%3A12 => generated 318 bytes in 235 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:13:13.221824 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-c3dda913-f483-41ba-adf8-bee86bc2e65a tempest-InstanceUsageAuditLogTestJSON-47970016 tempest-InstanceUsageAuditLogTestJSON-47970016-project-admin] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:13:13.222576 ncdv-master-1 devstack@n-api.service[138681]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-instance-usage-audit-log:list": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:13:13.222576 ncdv-master-1 devstack@n-api.service[138681]: warnings.warn(msg) Jul 16 21:13:13.240479 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-c3dda913-f483-41ba-adf8-bee86bc2e65a tempest-InstanceUsageAuditLogTestJSON-47970016 tempest-InstanceUsageAuditLogTestJSON-47970016-project-admin] 10.222.0.16 "GET /compute/v2.1/os-instance_usage_audit_log" status: 200 len: 319 microversion: 2.1 time: 0.022710 Jul 16 21:13:13.250934 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 637/1359] 10.222.0.16 () {58 vars in 1184 bytes} [Sat Jul 16 21:13:13 2022] GET /compute/v2.1/os-instance_usage_audit_log => generated 319 bytes in 30 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:13:13.527685 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-01495828-f78a-4d24-ad3e-705bd94b2a9b tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:13:13.528752 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-01495828-f78a-4d24-ad3e-705bd94b2a9b tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] [instance: 08e7e1f9-8b5c-446c-a05f-d8043473dd35] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:13:13.536634 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-01495828-f78a-4d24-ad3e-705bd94b2a9b tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:13.537092 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-01495828-f78a-4d24-ad3e-705bd94b2a9b tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:13.541343 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-01495828-f78a-4d24-ad3e-705bd94b2a9b tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:13.542912 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-01495828-f78a-4d24-ad3e-705bd94b2a9b tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:13.547900 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-86e44c40-0a7e-4434-912c-48b9359c0838 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b68dacc4b8463ba07fae6372131117', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55a19b7a4682457a9132f5fc80110712', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:13:13.567140 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-86e44c40-0a7e-4434-912c-48b9359c0838 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "GET /compute/v2.1/servers/126f561c-163f-4f6d-8be1-98a2e29b0909" status: 200 len: 1477 microversion: 2.1 time: 0.898381 Jul 16 21:13:13.569568 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 723/1360] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:13:12 2022] GET /compute/v2.1/servers/126f561c-163f-4f6d-8be1-98a2e29b0909 => generated 1477 bytes in 904 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:13:13.986001 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-623b6b92-930b-48d0-bf59-f4e8b0696327 tempest-TenantUsagesTestJSON-260059616 tempest-TenantUsagesTestJSON-260059616-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:13:13.986526 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-623b6b92-930b-48d0-bf59-f4e8b0696327 tempest-TenantUsagesTestJSON-260059616 tempest-TenantUsagesTestJSON-260059616-project] [instance: 6e68dbd6-1d5c-471e-ad69-cf1c4aa4004a] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:13:13.994809 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-623b6b92-930b-48d0-bf59-f4e8b0696327 tempest-TenantUsagesTestJSON-260059616 tempest-TenantUsagesTestJSON-260059616-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:13.995604 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-623b6b92-930b-48d0-bf59-f4e8b0696327 tempest-TenantUsagesTestJSON-260059616 tempest-TenantUsagesTestJSON-260059616-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:13.996394 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-623b6b92-930b-48d0-bf59-f4e8b0696327 tempest-TenantUsagesTestJSON-260059616 tempest-TenantUsagesTestJSON-260059616-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:13.997006 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-623b6b92-930b-48d0-bf59-f4e8b0696327 tempest-TenantUsagesTestJSON-260059616 tempest-TenantUsagesTestJSON-260059616-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:14.680824 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-01495828-f78a-4d24-ad3e-705bd94b2a9b tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '52281b3989a04ca2a50b0bc93bf783f0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '30d028fa9b854c8fb71c8cbed2985c27', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:13:14.693015 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-01495828-f78a-4d24-ad3e-705bd94b2a9b tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] 10.222.0.16 "GET /compute/v2.1/servers/08e7e1f9-8b5c-446c-a05f-d8043473dd35" status: 200 len: 1514 microversion: 2.1 time: 1.167491 Jul 16 21:13:14.693015 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 638/1361] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:13:13 2022] GET /compute/v2.1/servers/08e7e1f9-8b5c-446c-a05f-d8043473dd35 => generated 1514 bytes in 1169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:13:14.710113 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-dd7d43d2-3c68-4b48-b68b-5e823903a371 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:13:14.712167 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-dd7d43d2-3c68-4b48-b68b-5e823903a371 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: 126f561c-163f-4f6d-8be1-98a2e29b0909] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:13:14.725356 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-dd7d43d2-3c68-4b48-b68b-5e823903a371 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:14.727723 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-dd7d43d2-3c68-4b48-b68b-5e823903a371 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:14.729045 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-dd7d43d2-3c68-4b48-b68b-5e823903a371 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:14.729498 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-dd7d43d2-3c68-4b48-b68b-5e823903a371 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:15.057019 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-623b6b92-930b-48d0-bf59-f4e8b0696327 tempest-TenantUsagesTestJSON-260059616 tempest-TenantUsagesTestJSON-260059616-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2db871d6aa9746d2b660efbfbdb1dc75', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cad2f07400d54ef9825022d5b3489db0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:13:15.080005 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-623b6b92-930b-48d0-bf59-f4e8b0696327 tempest-TenantUsagesTestJSON-260059616 tempest-TenantUsagesTestJSON-260059616-project] 10.222.0.16 "GET /compute/v2.1/servers/6e68dbd6-1d5c-471e-ad69-cf1c4aa4004a" status: 200 len: 1470 microversion: 2.1 time: 1.095878 Jul 16 21:13:15.085110 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 724/1362] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:13:13 2022] GET /compute/v2.1/servers/6e68dbd6-1d5c-471e-ad69-cf1c4aa4004a => generated 1470 bytes in 1104 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:13:15.567053 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-71faa5df-728f-45d7-86c5-e92c4a881885 req-e24489cc-d65d-451e-a70e-94019d524f17 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "08e7e1f9-8b5c-446c-a05f-d8043473dd35", "name": "network-vif-deleted", "tag": "b21352ed-bfe1-4d3f-93ff-6f5027326093"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:13:15.609172 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-71faa5df-728f-45d7-86c5-e92c4a881885 req-e24489cc-d65d-451e-a70e-94019d524f17 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:15.609638 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-71faa5df-728f-45d7-86c5-e92c4a881885 req-e24489cc-d65d-451e-a70e-94019d524f17 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:15.630535 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-71faa5df-728f-45d7-86c5-e92c4a881885 req-e24489cc-d65d-451e-a70e-94019d524f17 service nova] Creating event network-vif-deleted:b21352ed-bfe1-4d3f-93ff-6f5027326093 for instance 08e7e1f9-8b5c-446c-a05f-d8043473dd35 on nch1-master-1 Jul 16 21:13:15.656910 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-71faa5df-728f-45d7-86c5-e92c4a881885 req-e24489cc-d65d-451e-a70e-94019d524f17 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.093902 Jul 16 21:13:15.671111 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 725/1363] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:13:15 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 108 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 16 21:13:15.717305 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-db2798bf-a2fb-4002-bc2e-e3c78dd4d962 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:13:15.718401 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-db2798bf-a2fb-4002-bc2e-e3c78dd4d962 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] [instance: 08e7e1f9-8b5c-446c-a05f-d8043473dd35] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:13:15.726078 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-db2798bf-a2fb-4002-bc2e-e3c78dd4d962 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:15.726638 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-db2798bf-a2fb-4002-bc2e-e3c78dd4d962 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:15.728422 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-db2798bf-a2fb-4002-bc2e-e3c78dd4d962 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:15.729372 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-db2798bf-a2fb-4002-bc2e-e3c78dd4d962 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:15.784225 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-dd7d43d2-3c68-4b48-b68b-5e823903a371 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b68dacc4b8463ba07fae6372131117', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55a19b7a4682457a9132f5fc80110712', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:13:15.801273 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-dd7d43d2-3c68-4b48-b68b-5e823903a371 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "GET /compute/v2.1/servers/126f561c-163f-4f6d-8be1-98a2e29b0909" status: 200 len: 1477 microversion: 2.1 time: 1.110146 Jul 16 21:13:15.802211 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 639/1364] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:13:14 2022] GET /compute/v2.1/servers/126f561c-163f-4f6d-8be1-98a2e29b0909 => generated 1477 bytes in 1112 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:13:15.857449 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-db2798bf-a2fb-4002-bc2e-e3c78dd4d962 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '52281b3989a04ca2a50b0bc93bf783f0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '30d028fa9b854c8fb71c8cbed2985c27', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:13:15.909444 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-db2798bf-a2fb-4002-bc2e-e3c78dd4d962 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] 10.222.0.16 "GET /compute/v2.1/servers/08e7e1f9-8b5c-446c-a05f-d8043473dd35" status: 200 len: 1472 microversion: 2.1 time: 0.205300 Jul 16 21:13:15.910942 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 726/1365] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:13:15 2022] GET /compute/v2.1/servers/08e7e1f9-8b5c-446c-a05f-d8043473dd35 => generated 1472 bytes in 208 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:13:16.107812 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-29a93b79-834e-4e37-acfa-14bc4cd0e238 tempest-TenantUsagesTestJSON-260059616 tempest-TenantUsagesTestJSON-260059616-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:13:16.109980 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-29a93b79-834e-4e37-acfa-14bc4cd0e238 tempest-TenantUsagesTestJSON-260059616 tempest-TenantUsagesTestJSON-260059616-project] [instance: 6e68dbd6-1d5c-471e-ad69-cf1c4aa4004a] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:13:16.122373 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-29a93b79-834e-4e37-acfa-14bc4cd0e238 tempest-TenantUsagesTestJSON-260059616 tempest-TenantUsagesTestJSON-260059616-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:16.124070 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-29a93b79-834e-4e37-acfa-14bc4cd0e238 tempest-TenantUsagesTestJSON-260059616 tempest-TenantUsagesTestJSON-260059616-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:16.125078 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-29a93b79-834e-4e37-acfa-14bc4cd0e238 tempest-TenantUsagesTestJSON-260059616 tempest-TenantUsagesTestJSON-260059616-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:16.126506 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-29a93b79-834e-4e37-acfa-14bc4cd0e238 tempest-TenantUsagesTestJSON-260059616 tempest-TenantUsagesTestJSON-260059616-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:16.275700 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-29a93b79-834e-4e37-acfa-14bc4cd0e238 tempest-TenantUsagesTestJSON-260059616 tempest-TenantUsagesTestJSON-260059616-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2db871d6aa9746d2b660efbfbdb1dc75', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cad2f07400d54ef9825022d5b3489db0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:13:16.289646 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-29a93b79-834e-4e37-acfa-14bc4cd0e238 tempest-TenantUsagesTestJSON-260059616 tempest-TenantUsagesTestJSON-260059616-project] 10.222.0.16 "GET /compute/v2.1/servers/6e68dbd6-1d5c-471e-ad69-cf1c4aa4004a" status: 200 len: 1428 microversion: 2.1 time: 0.191665 Jul 16 21:13:16.299148 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 640/1366] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:13:16 2022] GET /compute/v2.1/servers/6e68dbd6-1d5c-471e-ad69-cf1c4aa4004a => generated 1428 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:13:16.820896 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-ac70a87b-0ccd-4653-912e-a0740f46e40a tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:13:16.821451 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-ac70a87b-0ccd-4653-912e-a0740f46e40a tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: 126f561c-163f-4f6d-8be1-98a2e29b0909] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:13:16.831195 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ac70a87b-0ccd-4653-912e-a0740f46e40a tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:16.831655 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ac70a87b-0ccd-4653-912e-a0740f46e40a tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:16.832674 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ac70a87b-0ccd-4653-912e-a0740f46e40a tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:16.833259 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ac70a87b-0ccd-4653-912e-a0740f46e40a tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:16.929436 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-3253761d-bcb7-4033-86c9-6552eb64da11 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:13:16.930125 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-3253761d-bcb7-4033-86c9-6552eb64da11 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] [instance: 08e7e1f9-8b5c-446c-a05f-d8043473dd35] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:13:16.937208 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3253761d-bcb7-4033-86c9-6552eb64da11 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:16.937978 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3253761d-bcb7-4033-86c9-6552eb64da11 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:16.938869 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3253761d-bcb7-4033-86c9-6552eb64da11 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:16.939401 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3253761d-bcb7-4033-86c9-6552eb64da11 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:16.960035 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-3253761d-bcb7-4033-86c9-6552eb64da11 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] HTTP exception thrown: Instance 08e7e1f9-8b5c-446c-a05f-d8043473dd35 could not be found. Jul 16 21:13:16.962187 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-3253761d-bcb7-4033-86c9-6552eb64da11 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Request method failure captured: Jul 16 21:13:16.962187 ncdv-master-1 devstack@n-api.service[138681]: request: GET /compute/v2.1/servers/08e7e1f9-8b5c-446c-a05f-d8043473dd35 HTTP/1.1 Jul 16 21:13:16.962187 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:13:16.962187 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:13:16.962187 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:13:16.962187 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 21:13:16.962187 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:13:16.962187 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:13:16.962187 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:13:16.962187 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:13:16.962187 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0ylhXkEcUNUjx8XwHedkkbbEFGLbl1ipHzW2-01XQZFgiFO2FFNqfHwU4TEzUeINSl3sRZJc9IyRArkygeU15fbG9AQsbVjzJj1gOlYNe-wBjmbrSABbJAVNm59rIc-j8wjTwTaxRQjqaoZdreNlp7_Oz3QV5ZoHvzQiRIpBEzcndF8 Jul 16 21:13:16.962187 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:13:16.962187 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:13:16.962187 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:13:16.962187 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:13:16.962187 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:13:16.962187 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:13:16.962187 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: 30d028fa9b854c8fb71c8cbed2985c27 Jul 16 21:13:16.962187 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-VolumesAdminNegativeTest-1468695956 Jul 16 21:13:16.962187 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:13:16.962187 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:13:16.963857 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/30d028fa9b854c8fb71c8cbed2985c27"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/30d028fa9b854c8fb71c8cbed2985c27"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_30d028fa9b854c8fb71c8cbed2985c27"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/30d028fa9b854c8fb71c8cbed2985c27"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:13:16.963857 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-VolumesAdminNegativeTest-1468695956 Jul 16 21:13:16.963857 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: 30d028fa9b854c8fb71c8cbed2985c27 Jul 16 21:13:16.963857 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-VolumesAdminNegativeTest-1468695956 Jul 16 21:13:16.963857 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-VolumesAdminNegativeTest-1468695956-project Jul 16 21:13:16.963857 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:13:16.963857 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:13:16.963857 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: 52281b3989a04ca2a50b0bc93bf783f0 Jul 16 21:13:16.963857 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-VolumesAdminNegativeTest-1468695956-project Jul 16 21:13:16.963857 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:13:16.963857 ncdv-master-1 devstack@n-api.service[138681]: exception: Instance 08e7e1f9-8b5c-446c-a05f-d8043473dd35 could not be found. Jul 16 21:13:16.963857 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': '08e7e1f9-8b5c-446c-a05f-d8043473dd35'} Jul 16 21:13:16.964799 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:13:16.964799 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:13:16.964799 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:13:16.964799 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:13:16.964799 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:13:16.964799 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:13:16.964799 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:13:16.964799 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:13:16.964799 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:13:16.964799 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:13:16.964799 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 08e7e1f9-8b5c-446c-a05f-d8043473dd35 could not be found. Jul 16 21:13:16.964799 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:13:16.964799 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:13:16.964799 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:13:16.964799 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:13:16.964799 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:13:16.964799 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:13:16.964799 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:13:16.964799 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:13:16.964799 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:13:16.966138 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:13:16.966138 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:13:16.966138 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:13:16.966138 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:13:16.966138 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:13:16.966138 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:13:16.966138 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:13:16.966138 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 08e7e1f9-8b5c-446c-a05f-d8043473dd35 could not be found. Jul 16 21:13:16.966138 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:13:16.966138 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:13:16.966138 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:13:16.966138 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:13:16.966138 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:13:16.966138 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:13:16.966138 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:13:16.966138 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:13:16.966138 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 08e7e1f9-8b5c-446c-a05f-d8043473dd35 could not be found. Jul 16 21:13:16.966138 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:13:16.967477 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-3253761d-bcb7-4033-86c9-6552eb64da11 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Returning 404 to user: Instance 08e7e1f9-8b5c-446c-a05f-d8043473dd35 could not be found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:13:16.967477 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-3253761d-bcb7-4033-86c9-6552eb64da11 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] 10.222.0.16 "GET /compute/v2.1/servers/08e7e1f9-8b5c-446c-a05f-d8043473dd35" status: 404 len: 111 microversion: 2.1 time: 0.041232 Jul 16 21:13:16.967477 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 641/1367] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:13:16 2022] GET /compute/v2.1/servers/08e7e1f9-8b5c-446c-a05f-d8043473dd35 => generated 111 bytes in 43 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:13:17.063249 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-ac70a87b-0ccd-4653-912e-a0740f46e40a tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b68dacc4b8463ba07fae6372131117', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55a19b7a4682457a9132f5fc80110712', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:13:17.075272 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-ac70a87b-0ccd-4653-912e-a0740f46e40a tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "GET /compute/v2.1/servers/126f561c-163f-4f6d-8be1-98a2e29b0909" status: 200 len: 1477 microversion: 2.1 time: 0.258649 Jul 16 21:13:17.076091 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 727/1368] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:13:16 2022] GET /compute/v2.1/servers/126f561c-163f-4f6d-8be1-98a2e29b0909 => generated 1477 bytes in 260 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:13:17.309680 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-e42aff1c-0f34-4383-8b4b-6d6585a7aa12 tempest-TenantUsagesTestJSON-260059616 tempest-TenantUsagesTestJSON-260059616-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:13:17.310641 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-e42aff1c-0f34-4383-8b4b-6d6585a7aa12 tempest-TenantUsagesTestJSON-260059616 tempest-TenantUsagesTestJSON-260059616-project] [instance: 6e68dbd6-1d5c-471e-ad69-cf1c4aa4004a] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:13:17.325818 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e42aff1c-0f34-4383-8b4b-6d6585a7aa12 tempest-TenantUsagesTestJSON-260059616 tempest-TenantUsagesTestJSON-260059616-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:17.326455 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e42aff1c-0f34-4383-8b4b-6d6585a7aa12 tempest-TenantUsagesTestJSON-260059616 tempest-TenantUsagesTestJSON-260059616-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:17.327410 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e42aff1c-0f34-4383-8b4b-6d6585a7aa12 tempest-TenantUsagesTestJSON-260059616 tempest-TenantUsagesTestJSON-260059616-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:17.329035 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e42aff1c-0f34-4383-8b4b-6d6585a7aa12 tempest-TenantUsagesTestJSON-260059616 tempest-TenantUsagesTestJSON-260059616-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:17.439500 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e42aff1c-0f34-4383-8b4b-6d6585a7aa12 tempest-TenantUsagesTestJSON-260059616 tempest-TenantUsagesTestJSON-260059616-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:17.442408 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e42aff1c-0f34-4383-8b4b-6d6585a7aa12 tempest-TenantUsagesTestJSON-260059616 tempest-TenantUsagesTestJSON-260059616-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:17.443136 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.objects.instance [None req-e42aff1c-0f34-4383-8b4b-6d6585a7aa12 tempest-TenantUsagesTestJSON-260059616 tempest-TenantUsagesTestJSON-260059616-project] Lazy-loading 'fault' on Instance uuid 6e68dbd6-1d5c-471e-ad69-cf1c4aa4004a {{(pid=138681) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:13:17.529691 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-e42aff1c-0f34-4383-8b4b-6d6585a7aa12 tempest-TenantUsagesTestJSON-260059616 tempest-TenantUsagesTestJSON-260059616-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2db871d6aa9746d2b660efbfbdb1dc75', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cad2f07400d54ef9825022d5b3489db0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:13:17.539114 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-e42aff1c-0f34-4383-8b4b-6d6585a7aa12 tempest-TenantUsagesTestJSON-260059616 tempest-TenantUsagesTestJSON-260059616-project] 10.222.0.16 "GET /compute/v2.1/servers/6e68dbd6-1d5c-471e-ad69-cf1c4aa4004a" status: 200 len: 1304 microversion: 2.1 time: 0.236759 Jul 16 21:13:17.539935 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 642/1369] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:13:17 2022] GET /compute/v2.1/servers/6e68dbd6-1d5c-471e-ad69-cf1c4aa4004a => generated 1304 bytes in 239 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:13:17.675036 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-287eac0f-b89f-436d-a765-0192f83af686 req-7c6e223d-634c-4874-8521-2e020970dff1 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "6e68dbd6-1d5c-471e-ad69-cf1c4aa4004a", "name": "network-vif-deleted", "tag": "ca8395d6-64dd-4e53-92a0-b2a79f375684"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:13:17.683866 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-287eac0f-b89f-436d-a765-0192f83af686 req-7c6e223d-634c-4874-8521-2e020970dff1 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:17.684553 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-287eac0f-b89f-436d-a765-0192f83af686 req-7c6e223d-634c-4874-8521-2e020970dff1 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:17.689925 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:13:17.700464 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-287eac0f-b89f-436d-a765-0192f83af686 req-7c6e223d-634c-4874-8521-2e020970dff1 service nova] Creating event network-vif-deleted:ca8395d6-64dd-4e53-92a0-b2a79f375684 for instance 6e68dbd6-1d5c-471e-ad69-cf1c4aa4004a on nch2-master-1 Jul 16 21:13:17.708114 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-287eac0f-b89f-436d-a765-0192f83af686 req-7c6e223d-634c-4874-8521-2e020970dff1 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.036808 Jul 16 21:13:17.709315 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 728/1370] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:13:17 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:13:18.092715 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-ec8d8c1f-7bf2-4660-a25d-280f60334d43 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:13:18.094032 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-ec8d8c1f-7bf2-4660-a25d-280f60334d43 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: 126f561c-163f-4f6d-8be1-98a2e29b0909] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:13:18.102758 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ec8d8c1f-7bf2-4660-a25d-280f60334d43 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:18.106026 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ec8d8c1f-7bf2-4660-a25d-280f60334d43 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:18.106026 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ec8d8c1f-7bf2-4660-a25d-280f60334d43 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:18.106026 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ec8d8c1f-7bf2-4660-a25d-280f60334d43 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:18.296647 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-ec8d8c1f-7bf2-4660-a25d-280f60334d43 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b68dacc4b8463ba07fae6372131117', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55a19b7a4682457a9132f5fc80110712', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:13:18.325497 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-ec8d8c1f-7bf2-4660-a25d-280f60334d43 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "GET /compute/v2.1/servers/126f561c-163f-4f6d-8be1-98a2e29b0909" status: 200 len: 1477 microversion: 2.1 time: 0.237073 Jul 16 21:13:18.326107 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 643/1371] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:13:18 2022] GET /compute/v2.1/servers/126f561c-163f-4f6d-8be1-98a2e29b0909 => generated 1477 bytes in 238 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:13:18.558185 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-f90698b5-0d9f-4625-a72d-7b68070bc28e tempest-TenantUsagesTestJSON-260059616 tempest-TenantUsagesTestJSON-260059616-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:13:18.559100 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-f90698b5-0d9f-4625-a72d-7b68070bc28e tempest-TenantUsagesTestJSON-260059616 tempest-TenantUsagesTestJSON-260059616-project] [instance: 6e68dbd6-1d5c-471e-ad69-cf1c4aa4004a] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:13:18.565992 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f90698b5-0d9f-4625-a72d-7b68070bc28e tempest-TenantUsagesTestJSON-260059616 tempest-TenantUsagesTestJSON-260059616-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:18.567354 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f90698b5-0d9f-4625-a72d-7b68070bc28e tempest-TenantUsagesTestJSON-260059616 tempest-TenantUsagesTestJSON-260059616-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:18.567354 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f90698b5-0d9f-4625-a72d-7b68070bc28e tempest-TenantUsagesTestJSON-260059616 tempest-TenantUsagesTestJSON-260059616-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:18.567673 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f90698b5-0d9f-4625-a72d-7b68070bc28e tempest-TenantUsagesTestJSON-260059616 tempest-TenantUsagesTestJSON-260059616-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:18.581731 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-f90698b5-0d9f-4625-a72d-7b68070bc28e tempest-TenantUsagesTestJSON-260059616 tempest-TenantUsagesTestJSON-260059616-project] HTTP exception thrown: Instance 6e68dbd6-1d5c-471e-ad69-cf1c4aa4004a could not be found. Jul 16 21:13:18.583441 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-f90698b5-0d9f-4625-a72d-7b68070bc28e tempest-TenantUsagesTestJSON-260059616 tempest-TenantUsagesTestJSON-260059616-project] Request method failure captured: Jul 16 21:13:18.583441 ncdv-master-1 devstack@n-api.service[138680]: request: GET /compute/v2.1/servers/6e68dbd6-1d5c-471e-ad69-cf1c4aa4004a HTTP/1.1 Jul 16 21:13:18.583441 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:13:18.583441 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:13:18.583441 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:13:18.583441 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 0 Jul 16 21:13:18.583441 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:13:18.583441 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:13:18.583441 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:13:18.583441 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:13:18.583441 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0ynKmfeLGNQEEVCS5ebp--7PBBjU-XM8_VSQ5yJ9ioOzIUAd-c1rh9ivGKrqt-Djp4WfPjQJjYqlPL69dhSSKxwN3wsgo2ukCCwFUwQ1V4mHKFqCyzAAorwXm0TCFW5bWXcO-lsRt_xFrnM7VqSMAzQxpZnyaw74FhPvFBsv26NPZd0 Jul 16 21:13:18.583441 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:13:18.583441 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:13:18.583441 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:13:18.583441 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:13:18.583441 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:13:18.583441 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:13:18.583441 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: cad2f07400d54ef9825022d5b3489db0 Jul 16 21:13:18.583441 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-TenantUsagesTestJSON-260059616 Jul 16 21:13:18.583441 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:13:18.583441 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:13:18.585729 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/cad2f07400d54ef9825022d5b3489db0"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/cad2f07400d54ef9825022d5b3489db0"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_cad2f07400d54ef9825022d5b3489db0"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/cad2f07400d54ef9825022d5b3489db0"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:13:18.585729 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-TenantUsagesTestJSON-260059616 Jul 16 21:13:18.585729 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: cad2f07400d54ef9825022d5b3489db0 Jul 16 21:13:18.585729 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-TenantUsagesTestJSON-260059616 Jul 16 21:13:18.585729 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-TenantUsagesTestJSON-260059616-project Jul 16 21:13:18.585729 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:13:18.585729 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:13:18.585729 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: 2db871d6aa9746d2b660efbfbdb1dc75 Jul 16 21:13:18.585729 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-TenantUsagesTestJSON-260059616-project Jul 16 21:13:18.585729 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 21:13:18.585729 ncdv-master-1 devstack@n-api.service[138680]: exception: Instance 6e68dbd6-1d5c-471e-ad69-cf1c4aa4004a could not be found. Jul 16 21:13:18.585729 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': '6e68dbd6-1d5c-471e-ad69-cf1c4aa4004a'} Jul 16 21:13:18.601810 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:13:18.601810 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:13:18.601810 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:13:18.601810 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:13:18.601810 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:13:18.601810 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:13:18.601810 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:13:18.601810 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:13:18.601810 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:13:18.601810 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:13:18.601810 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 6e68dbd6-1d5c-471e-ad69-cf1c4aa4004a could not be found. Jul 16 21:13:18.601810 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:13:18.601810 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:13:18.601810 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:13:18.601810 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:13:18.601810 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:13:18.601810 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:13:18.601810 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:13:18.601810 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:13:18.601810 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:13:18.603432 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:13:18.603432 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:13:18.603432 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:13:18.603432 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:13:18.603432 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:13:18.603432 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:13:18.603432 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:13:18.603432 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 6e68dbd6-1d5c-471e-ad69-cf1c4aa4004a could not be found. Jul 16 21:13:18.603432 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:13:18.603432 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:13:18.603432 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:13:18.603432 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:13:18.603432 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:13:18.603432 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:13:18.603432 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:13:18.603432 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:13:18.603432 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 6e68dbd6-1d5c-471e-ad69-cf1c4aa4004a could not be found. Jul 16 21:13:18.603432 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:13:18.604783 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-f90698b5-0d9f-4625-a72d-7b68070bc28e tempest-TenantUsagesTestJSON-260059616 tempest-TenantUsagesTestJSON-260059616-project] Returning 404 to user: Instance 6e68dbd6-1d5c-471e-ad69-cf1c4aa4004a could not be found. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:13:18.604783 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-f90698b5-0d9f-4625-a72d-7b68070bc28e tempest-TenantUsagesTestJSON-260059616 tempest-TenantUsagesTestJSON-260059616-project] 10.222.0.16 "GET /compute/v2.1/servers/6e68dbd6-1d5c-471e-ad69-cf1c4aa4004a" status: 404 len: 111 microversion: 2.1 time: 0.032774 Jul 16 21:13:18.604783 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 729/1372] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:13:18 2022] GET /compute/v2.1/servers/6e68dbd6-1d5c-471e-ad69-cf1c4aa4004a => generated 111 bytes in 35 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:13:19.346224 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-75b00139-8dad-4ad6-99ce-554a5c79cb8f tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:13:19.347744 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-75b00139-8dad-4ad6-99ce-554a5c79cb8f tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: 126f561c-163f-4f6d-8be1-98a2e29b0909] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:13:19.355374 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-75b00139-8dad-4ad6-99ce-554a5c79cb8f tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:19.356102 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-75b00139-8dad-4ad6-99ce-554a5c79cb8f tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:19.357118 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-75b00139-8dad-4ad6-99ce-554a5c79cb8f tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:19.357743 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-75b00139-8dad-4ad6-99ce-554a5c79cb8f tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:19.551382 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-75b00139-8dad-4ad6-99ce-554a5c79cb8f tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b68dacc4b8463ba07fae6372131117', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55a19b7a4682457a9132f5fc80110712', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:13:19.559753 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-75b00139-8dad-4ad6-99ce-554a5c79cb8f tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "GET /compute/v2.1/servers/126f561c-163f-4f6d-8be1-98a2e29b0909" status: 200 len: 1477 microversion: 2.1 time: 0.221616 Jul 16 21:13:19.560680 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 644/1373] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:13:19 2022] GET /compute/v2.1/servers/126f561c-163f-4f6d-8be1-98a2e29b0909 => generated 1477 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:13:20.577436 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-2fe96bc7-7d7f-44bd-9dbd-432cb9c72e54 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:13:20.578307 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-2fe96bc7-7d7f-44bd-9dbd-432cb9c72e54 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: 126f561c-163f-4f6d-8be1-98a2e29b0909] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:13:20.588879 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2fe96bc7-7d7f-44bd-9dbd-432cb9c72e54 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:20.589465 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2fe96bc7-7d7f-44bd-9dbd-432cb9c72e54 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:20.590767 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2fe96bc7-7d7f-44bd-9dbd-432cb9c72e54 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:20.591453 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2fe96bc7-7d7f-44bd-9dbd-432cb9c72e54 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:20.791336 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-2fe96bc7-7d7f-44bd-9dbd-432cb9c72e54 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b68dacc4b8463ba07fae6372131117', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55a19b7a4682457a9132f5fc80110712', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:13:20.799798 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-2fe96bc7-7d7f-44bd-9dbd-432cb9c72e54 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "GET /compute/v2.1/servers/126f561c-163f-4f6d-8be1-98a2e29b0909" status: 200 len: 1477 microversion: 2.1 time: 0.226347 Jul 16 21:13:20.800402 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 730/1374] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:13:20 2022] GET /compute/v2.1/servers/126f561c-163f-4f6d-8be1-98a2e29b0909 => generated 1477 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:13:21.886727 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-235ed388-0129-4e77-a27a-8d5984090cf8 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:13:21.889160 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-235ed388-0129-4e77-a27a-8d5984090cf8 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: 126f561c-163f-4f6d-8be1-98a2e29b0909] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:13:21.899963 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_api:SELECT=97,nova_api:UPDATE=2,nova_api:DELETE=1 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:13:21.911941 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-235ed388-0129-4e77-a27a-8d5984090cf8 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:21.915345 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-235ed388-0129-4e77-a27a-8d5984090cf8 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:21.916680 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-235ed388-0129-4e77-a27a-8d5984090cf8 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:21.918101 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-235ed388-0129-4e77-a27a-8d5984090cf8 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:22.356108 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-235ed388-0129-4e77-a27a-8d5984090cf8 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b68dacc4b8463ba07fae6372131117', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55a19b7a4682457a9132f5fc80110712', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:13:22.363375 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-235ed388-0129-4e77-a27a-8d5984090cf8 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "GET /compute/v2.1/servers/126f561c-163f-4f6d-8be1-98a2e29b0909" status: 200 len: 1477 microversion: 2.1 time: 0.488169 Jul 16 21:13:22.368203 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 645/1375] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:13:21 2022] GET /compute/v2.1/servers/126f561c-163f-4f6d-8be1-98a2e29b0909 => generated 1477 bytes in 494 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:13:23.385615 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-3ff91e42-8a66-47e6-85c8-73413e5fc2aa tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:13:23.386284 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-3ff91e42-8a66-47e6-85c8-73413e5fc2aa tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: 126f561c-163f-4f6d-8be1-98a2e29b0909] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:13:23.395078 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell0:SELECT=6 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:13:23.397906 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3ff91e42-8a66-47e6-85c8-73413e5fc2aa tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:23.398713 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3ff91e42-8a66-47e6-85c8-73413e5fc2aa tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:23.399758 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3ff91e42-8a66-47e6-85c8-73413e5fc2aa tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:23.400364 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3ff91e42-8a66-47e6-85c8-73413e5fc2aa tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:23.711578 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-66a24159-ee25-4b9b-aded-90a2cfdc8d27 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:13:23.712433 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-66a24159-ee25-4b9b-aded-90a2cfdc8d27 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] [instance: b07f0d53-2cd4-440f-be47-077703aa66e6] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:13:23.728434 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell0:SELECT=8 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:13:23.732111 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-66a24159-ee25-4b9b-aded-90a2cfdc8d27 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:23.733747 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-66a24159-ee25-4b9b-aded-90a2cfdc8d27 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:23.734788 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-66a24159-ee25-4b9b-aded-90a2cfdc8d27 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:23.737005 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-66a24159-ee25-4b9b-aded-90a2cfdc8d27 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:23.816171 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-66a24159-ee25-4b9b-aded-90a2cfdc8d27 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] [instance: b07f0d53-2cd4-440f-be47-077703aa66e6] Going to try to terminate instance {{(pid=138681) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 16 21:13:24.035238 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-3ff91e42-8a66-47e6-85c8-73413e5fc2aa tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b68dacc4b8463ba07fae6372131117', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55a19b7a4682457a9132f5fc80110712', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:13:24.067268 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-3ff91e42-8a66-47e6-85c8-73413e5fc2aa tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "GET /compute/v2.1/servers/126f561c-163f-4f6d-8be1-98a2e29b0909" status: 200 len: 1476 microversion: 2.1 time: 0.685152 Jul 16 21:13:24.067883 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 731/1376] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:13:23 2022] GET /compute/v2.1/servers/126f561c-163f-4f6d-8be1-98a2e29b0909 => generated 1476 bytes in 688 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:13:24.365000 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-66a24159-ee25-4b9b-aded-90a2cfdc8d27 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] 10.222.0.16 "DELETE /compute/v2.1/servers/b07f0d53-2cd4-440f-be47-077703aa66e6" status: 204 len: 0 microversion: 2.1 time: 0.663188 Jul 16 21:13:24.366494 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 646/1377] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:13:23 2022] DELETE /compute/v2.1/servers/b07f0d53-2cd4-440f-be47-077703aa66e6 => generated 0 bytes in 666 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 16 21:13:24.377598 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-843471a0-9ef2-44b9-8dd3-f4a2eeeb4022 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:13:24.378348 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-843471a0-9ef2-44b9-8dd3-f4a2eeeb4022 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] [instance: b07f0d53-2cd4-440f-be47-077703aa66e6] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:13:24.399994 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-843471a0-9ef2-44b9-8dd3-f4a2eeeb4022 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:24.401062 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-843471a0-9ef2-44b9-8dd3-f4a2eeeb4022 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:24.403319 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-843471a0-9ef2-44b9-8dd3-f4a2eeeb4022 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:24.403319 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-843471a0-9ef2-44b9-8dd3-f4a2eeeb4022 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:24.672611 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-4169c9d7-8550-4e82-b5d5-6a48d6b3da85 req-9794fc76-0201-475a-a924-e838f9caf7bc service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "126f561c-163f-4f6d-8be1-98a2e29b0909", "name": "network-vif-unplugged", "status": "completed", "tag": "3b739133-7af5-4df3-a04d-fff7e0a81a3d"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:13:24.685798 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-4169c9d7-8550-4e82-b5d5-6a48d6b3da85 req-9794fc76-0201-475a-a924-e838f9caf7bc service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:24.686426 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-4169c9d7-8550-4e82-b5d5-6a48d6b3da85 req-9794fc76-0201-475a-a924-e838f9caf7bc service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:24.723686 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-4169c9d7-8550-4e82-b5d5-6a48d6b3da85 req-9794fc76-0201-475a-a924-e838f9caf7bc service nova] Creating event network-vif-unplugged:3b739133-7af5-4df3-a04d-fff7e0a81a3d for instance 126f561c-163f-4f6d-8be1-98a2e29b0909 on nch1-master-1 Jul 16 21:13:24.733102 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [req-4169c9d7-8550-4e82-b5d5-6a48d6b3da85 req-9794fc76-0201-475a-a924-e838f9caf7bc service nova] Instance 126f561c-163f-4f6d-8be1-98a2e29b0909 is migrating within the same cell, copying events to all relevant hosts: {'nch2-master-1', 'nch1-master-1'} {{(pid=138681) _get_relevant_hosts /opt/stack/nova/nova/compute/api.py:5807}} Jul 16 21:13:24.748710 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-4169c9d7-8550-4e82-b5d5-6a48d6b3da85 req-9794fc76-0201-475a-a924-e838f9caf7bc service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.080494 Jul 16 21:13:24.752817 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 647/1378] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:13:24 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:13:24.848330 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-843471a0-9ef2-44b9-8dd3-f4a2eeeb4022 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '52281b3989a04ca2a50b0bc93bf783f0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '30d028fa9b854c8fb71c8cbed2985c27', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:13:24.856686 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-843471a0-9ef2-44b9-8dd3-f4a2eeeb4022 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] 10.222.0.16 "GET /compute/v2.1/servers/b07f0d53-2cd4-440f-be47-077703aa66e6" status: 200 len: 1680 microversion: 2.1 time: 0.483851 Jul 16 21:13:24.857663 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 732/1379] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:13:24 2022] GET /compute/v2.1/servers/b07f0d53-2cd4-440f-be47-077703aa66e6 => generated 1680 bytes in 487 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:13:25.088129 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-134e8368-6035-487b-9b19-8c5f2ced12c0 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:13:25.089182 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-134e8368-6035-487b-9b19-8c5f2ced12c0 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: 126f561c-163f-4f6d-8be1-98a2e29b0909] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:13:25.112693 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-134e8368-6035-487b-9b19-8c5f2ced12c0 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:25.113422 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-134e8368-6035-487b-9b19-8c5f2ced12c0 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:25.115380 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-134e8368-6035-487b-9b19-8c5f2ced12c0 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:25.116593 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-134e8368-6035-487b-9b19-8c5f2ced12c0 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:25.882781 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-0e1161d1-15e9-443b-a9e7-a601bf96a8e4 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:13:25.884235 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-0e1161d1-15e9-443b-a9e7-a601bf96a8e4 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] [instance: b07f0d53-2cd4-440f-be47-077703aa66e6] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:13:25.891312 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0e1161d1-15e9-443b-a9e7-a601bf96a8e4 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:25.898730 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0e1161d1-15e9-443b-a9e7-a601bf96a8e4 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:25.899973 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0e1161d1-15e9-443b-a9e7-a601bf96a8e4 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:25.900424 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0e1161d1-15e9-443b-a9e7-a601bf96a8e4 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:26.127292 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-134e8368-6035-487b-9b19-8c5f2ced12c0 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b68dacc4b8463ba07fae6372131117', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55a19b7a4682457a9132f5fc80110712', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:13:26.140630 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-134e8368-6035-487b-9b19-8c5f2ced12c0 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "GET /compute/v2.1/servers/126f561c-163f-4f6d-8be1-98a2e29b0909" status: 200 len: 1476 microversion: 2.1 time: 1.057659 Jul 16 21:13:26.162147 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 648/1380] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:13:25 2022] GET /compute/v2.1/servers/126f561c-163f-4f6d-8be1-98a2e29b0909 => generated 1476 bytes in 1081 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:13:26.739573 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-0e1161d1-15e9-443b-a9e7-a601bf96a8e4 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '52281b3989a04ca2a50b0bc93bf783f0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '30d028fa9b854c8fb71c8cbed2985c27', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:13:26.755682 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-0e1161d1-15e9-443b-a9e7-a601bf96a8e4 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] 10.222.0.16 "GET /compute/v2.1/servers/b07f0d53-2cd4-440f-be47-077703aa66e6" status: 200 len: 1680 microversion: 2.1 time: 0.882098 Jul 16 21:13:26.756363 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 733/1381] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:13:25 2022] GET /compute/v2.1/servers/b07f0d53-2cd4-440f-be47-077703aa66e6 => generated 1680 bytes in 883 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:13:26.771497 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-49308ba4-99cb-4bd1-8647-053b987e65cc req-3679eabd-62d0-4c68-a221-3e047dfe21ca service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "126f561c-163f-4f6d-8be1-98a2e29b0909", "tag": "3b739133-7af5-4df3-a04d-fff7e0a81a3d"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:13:26.790187 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-49308ba4-99cb-4bd1-8647-053b987e65cc req-3679eabd-62d0-4c68-a221-3e047dfe21ca service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:26.790650 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-49308ba4-99cb-4bd1-8647-053b987e65cc req-3679eabd-62d0-4c68-a221-3e047dfe21ca service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:26.830141 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-49308ba4-99cb-4bd1-8647-053b987e65cc req-3679eabd-62d0-4c68-a221-3e047dfe21ca service nova] Creating event network-changed:3b739133-7af5-4df3-a04d-fff7e0a81a3d for instance 126f561c-163f-4f6d-8be1-98a2e29b0909 on nch1-master-1 Jul 16 21:13:26.848922 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [req-49308ba4-99cb-4bd1-8647-053b987e65cc req-3679eabd-62d0-4c68-a221-3e047dfe21ca service nova] Instance 126f561c-163f-4f6d-8be1-98a2e29b0909 is migrating within the same cell, copying events to all relevant hosts: {'nch2-master-1', 'nch1-master-1'} {{(pid=138681) _get_relevant_hosts /opt/stack/nova/nova/compute/api.py:5807}} Jul 16 21:13:26.885889 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-49308ba4-99cb-4bd1-8647-053b987e65cc req-3679eabd-62d0-4c68-a221-3e047dfe21ca service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.116969 Jul 16 21:13:26.885889 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 649/1382] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:13:26 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 119 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:13:27.163465 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-b01ea049-fb28-4af6-8895-1e01d6512833 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:13:27.165301 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-b01ea049-fb28-4af6-8895-1e01d6512833 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: 126f561c-163f-4f6d-8be1-98a2e29b0909] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:13:27.174895 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b01ea049-fb28-4af6-8895-1e01d6512833 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:27.175711 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b01ea049-fb28-4af6-8895-1e01d6512833 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:27.177210 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b01ea049-fb28-4af6-8895-1e01d6512833 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:27.177786 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b01ea049-fb28-4af6-8895-1e01d6512833 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:27.776060 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-181fa4a7-9fff-472e-acb1-487312717d00 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:13:27.776802 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-181fa4a7-9fff-472e-acb1-487312717d00 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] [instance: b07f0d53-2cd4-440f-be47-077703aa66e6] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:13:27.799448 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-181fa4a7-9fff-472e-acb1-487312717d00 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:27.801181 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-181fa4a7-9fff-472e-acb1-487312717d00 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:27.802332 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-181fa4a7-9fff-472e-acb1-487312717d00 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:27.802789 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-181fa4a7-9fff-472e-acb1-487312717d00 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:28.304912 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-b01ea049-fb28-4af6-8895-1e01d6512833 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b68dacc4b8463ba07fae6372131117', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55a19b7a4682457a9132f5fc80110712', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:13:28.314857 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-b01ea049-fb28-4af6-8895-1e01d6512833 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "GET /compute/v2.1/servers/126f561c-163f-4f6d-8be1-98a2e29b0909" status: 200 len: 1476 microversion: 2.1 time: 1.157133 Jul 16 21:13:28.315502 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 734/1383] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:13:27 2022] GET /compute/v2.1/servers/126f561c-163f-4f6d-8be1-98a2e29b0909 => generated 1476 bytes in 1159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:13:28.752274 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-181fa4a7-9fff-472e-acb1-487312717d00 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '52281b3989a04ca2a50b0bc93bf783f0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '30d028fa9b854c8fb71c8cbed2985c27', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:13:28.761407 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-181fa4a7-9fff-472e-acb1-487312717d00 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] 10.222.0.16 "GET /compute/v2.1/servers/b07f0d53-2cd4-440f-be47-077703aa66e6" status: 200 len: 1680 microversion: 2.1 time: 0.989664 Jul 16 21:13:28.765302 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 650/1384] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:13:27 2022] GET /compute/v2.1/servers/b07f0d53-2cd4-440f-be47-077703aa66e6 => generated 1680 bytes in 994 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:13:29.333184 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-effe0d64-53f2-402b-877f-c3484f3059a1 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:13:29.333915 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-effe0d64-53f2-402b-877f-c3484f3059a1 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: 126f561c-163f-4f6d-8be1-98a2e29b0909] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:13:29.362580 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-effe0d64-53f2-402b-877f-c3484f3059a1 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:29.364031 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-effe0d64-53f2-402b-877f-c3484f3059a1 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.013s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:29.367538 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-effe0d64-53f2-402b-877f-c3484f3059a1 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:29.368179 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-effe0d64-53f2-402b-877f-c3484f3059a1 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:29.779197 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-529579f6-3fa0-495c-9e85-0998eff28160 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:13:29.779850 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-529579f6-3fa0-495c-9e85-0998eff28160 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] [instance: b07f0d53-2cd4-440f-be47-077703aa66e6] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:13:29.796950 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-529579f6-3fa0-495c-9e85-0998eff28160 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:29.798230 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-529579f6-3fa0-495c-9e85-0998eff28160 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:29.803545 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-529579f6-3fa0-495c-9e85-0998eff28160 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:29.805289 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-529579f6-3fa0-495c-9e85-0998eff28160 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:29.816271 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=97,nova_cell1:INSERT=2,nova_cell1:UPDATE=1 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:13:30.462633 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-529579f6-3fa0-495c-9e85-0998eff28160 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '52281b3989a04ca2a50b0bc93bf783f0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '30d028fa9b854c8fb71c8cbed2985c27', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:13:30.476724 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-529579f6-3fa0-495c-9e85-0998eff28160 tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] 10.222.0.16 "GET /compute/v2.1/servers/b07f0d53-2cd4-440f-be47-077703aa66e6" status: 200 len: 1612 microversion: 2.1 time: 0.701526 Jul 16 21:13:30.486267 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 651/1385] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:13:29 2022] GET /compute/v2.1/servers/b07f0d53-2cd4-440f-be47-077703aa66e6 => generated 1612 bytes in 703 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:13:30.926943 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-effe0d64-53f2-402b-877f-c3484f3059a1 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b68dacc4b8463ba07fae6372131117', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55a19b7a4682457a9132f5fc80110712', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:13:30.936230 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-effe0d64-53f2-402b-877f-c3484f3059a1 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "GET /compute/v2.1/servers/126f561c-163f-4f6d-8be1-98a2e29b0909" status: 200 len: 1476 microversion: 2.1 time: 1.607762 Jul 16 21:13:30.936744 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 735/1386] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:13:29 2022] GET /compute/v2.1/servers/126f561c-163f-4f6d-8be1-98a2e29b0909 => generated 1476 bytes in 1609 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:13:31.304159 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-42e2b302-0272-4723-813a-d43f203ad801 req-bef7ec2f-7f57-404b-95c5-7ae2af7198b5 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "b07f0d53-2cd4-440f-be47-077703aa66e6", "name": "network-vif-deleted", "tag": "9b0f580d-ea65-4f1a-9836-c0f8d10895e9"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:13:31.335890 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-42e2b302-0272-4723-813a-d43f203ad801 req-bef7ec2f-7f57-404b-95c5-7ae2af7198b5 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:31.337187 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-42e2b302-0272-4723-813a-d43f203ad801 req-bef7ec2f-7f57-404b-95c5-7ae2af7198b5 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:31.394252 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-42e2b302-0272-4723-813a-d43f203ad801 req-bef7ec2f-7f57-404b-95c5-7ae2af7198b5 service nova] Creating event network-vif-deleted:9b0f580d-ea65-4f1a-9836-c0f8d10895e9 for instance b07f0d53-2cd4-440f-be47-077703aa66e6 on nch1-master-1 Jul 16 21:13:31.411296 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-42e2b302-0272-4723-813a-d43f203ad801 req-bef7ec2f-7f57-404b-95c5-7ae2af7198b5 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.414928 Jul 16 21:13:31.411942 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 652/1387] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:13:30 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 417 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:13:31.496191 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-aaa73328-aa7e-4641-b817-3f4980e855db tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:13:31.496828 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-aaa73328-aa7e-4641-b817-3f4980e855db tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] [instance: b07f0d53-2cd4-440f-be47-077703aa66e6] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:13:31.519709 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-aaa73328-aa7e-4641-b817-3f4980e855db tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:31.523994 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-aaa73328-aa7e-4641-b817-3f4980e855db tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:31.530904 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-aaa73328-aa7e-4641-b817-3f4980e855db tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:31.533181 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-aaa73328-aa7e-4641-b817-3f4980e855db tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:31.634831 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-aaa73328-aa7e-4641-b817-3f4980e855db tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:31.636613 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-aaa73328-aa7e-4641-b817-3f4980e855db tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:31.638306 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.objects.instance [None req-aaa73328-aa7e-4641-b817-3f4980e855db tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Lazy-loading 'fault' on Instance uuid b07f0d53-2cd4-440f-be47-077703aa66e6 {{(pid=138680) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:13:31.816423 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-aaa73328-aa7e-4641-b817-3f4980e855db tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '52281b3989a04ca2a50b0bc93bf783f0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '30d028fa9b854c8fb71c8cbed2985c27', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:13:31.852080 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-aaa73328-aa7e-4641-b817-3f4980e855db tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] 10.222.0.16 "GET /compute/v2.1/servers/b07f0d53-2cd4-440f-be47-077703aa66e6" status: 200 len: 1330 microversion: 2.1 time: 0.361897 Jul 16 21:13:31.855497 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 736/1388] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:13:31 2022] GET /compute/v2.1/servers/b07f0d53-2cd4-440f-be47-077703aa66e6 => generated 1330 bytes in 366 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:13:31.954132 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-7c3850e3-4d94-4ef5-a758-81011fe80be2 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:13:31.955108 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-7c3850e3-4d94-4ef5-a758-81011fe80be2 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: 126f561c-163f-4f6d-8be1-98a2e29b0909] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:13:31.986171 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7c3850e3-4d94-4ef5-a758-81011fe80be2 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:31.991798 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7c3850e3-4d94-4ef5-a758-81011fe80be2 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:31.997905 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7c3850e3-4d94-4ef5-a758-81011fe80be2 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:32.003526 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7c3850e3-4d94-4ef5-a758-81011fe80be2 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:32.480323 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-7c3850e3-4d94-4ef5-a758-81011fe80be2 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b68dacc4b8463ba07fae6372131117', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55a19b7a4682457a9132f5fc80110712', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:13:32.492071 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-7c3850e3-4d94-4ef5-a758-81011fe80be2 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "GET /compute/v2.1/servers/126f561c-163f-4f6d-8be1-98a2e29b0909" status: 200 len: 1474 microversion: 2.1 time: 0.542196 Jul 16 21:13:32.496499 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 653/1389] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:13:31 2022] GET /compute/v2.1/servers/126f561c-163f-4f6d-8be1-98a2e29b0909 => generated 1474 bytes in 547 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:13:32.672032 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-5ac6ffc4-0e53-44d4-86e7-e60c9c111793 tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServersAdminNegativeTestJSON-server-349190430", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452"}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:13:32.725361 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-43b4f8c7-d1a8-41c8-a682-52bdf8e11659 tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:13:32.727997 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-43b4f8c7-d1a8-41c8-a682-52bdf8e11659 tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] [instance: 3c5ab09e-0a97-4cfc-b249-a11ab221b5d2] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:13:32.758692 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-43b4f8c7-d1a8-41c8-a682-52bdf8e11659 tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:32.763953 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-43b4f8c7-d1a8-41c8-a682-52bdf8e11659 tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:32.781018 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-43b4f8c7-d1a8-41c8-a682-52bdf8e11659 tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:32.781018 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-43b4f8c7-d1a8-41c8-a682-52bdf8e11659 tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.009s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:32.807715 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.network.neutron [None req-5ac6ffc4-0e53-44d4-86e7-e60c9c111793 tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] validate_networks() for None {{(pid=138680) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 16 21:13:33.765142 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-5ac6ffc4-0e53-44d4-86e7-e60c9c111793 tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Flavor limits 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 16 21:13:33.765689 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-5ac6ffc4-0e53-44d4-86e7-e60c9c111793 tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Image limits 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 16 21:13:33.766382 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-5ac6ffc4-0e53-44d4-86e7-e60c9c111793 tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Flavor pref 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 16 21:13:33.766897 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-5ac6ffc4-0e53-44d4-86e7-e60c9c111793 tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Image pref 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 16 21:13:33.767613 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-5ac6ffc4-0e53-44d4-86e7-e60c9c111793 tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 16 21:13:33.794643 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5ac6ffc4-0e53-44d4-86e7-e60c9c111793 tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:33.796969 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5ac6ffc4-0e53-44d4-86e7-e60c9c111793 tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:33.800385 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5ac6ffc4-0e53-44d4-86e7-e60c9c111793 tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:33.800385 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5ac6ffc4-0e53-44d4-86e7-e60c9c111793 tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:33.862960 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-5ac6ffc4-0e53-44d4-86e7-e60c9c111793 tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Getting quotas for project 45d9edd2fe854483a6f0d676efbd6622. Resources: {'ram', 'instances', 'cores'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:13:33.883216 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-5ac6ffc4-0e53-44d4-86e7-e60c9c111793 tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Getting quotas for user c1aea87154224f1c81428ddef4438801 and project 45d9edd2fe854483a6f0d676efbd6622. Resources: {'ram', 'instances', 'cores'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:13:33.949946 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-5ac6ffc4-0e53-44d4-86e7-e60c9c111793 tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Going to run 1 instances... {{(pid=138680) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 16 21:13:33.960284 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-5ac6ffc4-0e53-44d4-86e7-e60c9c111793 tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] [instance: 4602867b-88ed-43b1-9ee6-bf661e2d7d99] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138680) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 16 21:13:34.118199 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.objects.instance [None req-43b4f8c7-d1a8-41c8-a682-52bdf8e11659 tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] Lazy-loading 'services' on Instance uuid 3c5ab09e-0a97-4cfc-b249-a11ab221b5d2 {{(pid=138681) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:13:34.145865 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-43b4f8c7-d1a8-41c8-a682-52bdf8e11659 tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/3c5ab09e-0a97-4cfc-b249-a11ab221b5d2" status: 200 len: 2339 microversion: 2.25 time: 1.435149 Jul 16 21:13:34.149554 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 654/1390] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:13:32 2022] GET /compute/v2.1/servers/3c5ab09e-0a97-4cfc-b249-a11ab221b5d2 => generated 2339 bytes in 1440 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:13:34.190394 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-991ee8c7-c441-4cd0-a976-437592fd67dd tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:13:34.191492 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-991ee8c7-c441-4cd0-a976-437592fd67dd tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] [instance: b07f0d53-2cd4-440f-be47-077703aa66e6] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:13:34.202255 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-991ee8c7-c441-4cd0-a976-437592fd67dd tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:34.203269 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-5ac6ffc4-0e53-44d4-86e7-e60c9c111793 tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 1.888761 Jul 16 21:13:34.203943 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 737/1391] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 21:13:32 2022] POST /compute/v2.1/servers => generated 380 bytes in 1890 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 16 21:13:34.204122 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-991ee8c7-c441-4cd0-a976-437592fd67dd tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:34.205279 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-991ee8c7-c441-4cd0-a976-437592fd67dd tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:34.205924 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-991ee8c7-c441-4cd0-a976-437592fd67dd tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:34.216232 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-0b9bc88e-ec5d-4077-bcca-1c4b8c79cc6a tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:13:34.218982 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-0b9bc88e-ec5d-4077-bcca-1c4b8c79cc6a tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: 126f561c-163f-4f6d-8be1-98a2e29b0909] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:13:34.225422 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0b9bc88e-ec5d-4077-bcca-1c4b8c79cc6a tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:34.227163 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0b9bc88e-ec5d-4077-bcca-1c4b8c79cc6a tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:34.228458 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0b9bc88e-ec5d-4077-bcca-1c4b8c79cc6a tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:34.229104 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0b9bc88e-ec5d-4077-bcca-1c4b8c79cc6a tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:34.257980 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-991ee8c7-c441-4cd0-a976-437592fd67dd tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] HTTP exception thrown: Instance b07f0d53-2cd4-440f-be47-077703aa66e6 could not be found. Jul 16 21:13:34.260319 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-991ee8c7-c441-4cd0-a976-437592fd67dd tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Request method failure captured: Jul 16 21:13:34.260319 ncdv-master-1 devstack@n-api.service[138681]: request: GET /compute/v2.1/servers/b07f0d53-2cd4-440f-be47-077703aa66e6 HTTP/1.1 Jul 16 21:13:34.260319 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:13:34.260319 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:13:34.260319 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:13:34.260319 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 21:13:34.260319 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:13:34.260319 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:13:34.260319 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:13:34.260319 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:13:34.260319 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0ylhXkEcUNUjx8XwHedkkbbEFGLbl1ipHzW2-01XQZFgiFO2FFNqfHwU4TEzUeINSl3sRZJc9IyRArkygeU15fbG9AQsbVjzJj1gOlYNe-wBjmbrSABbJAVNm59rIc-j8wjTwTaxRQjqaoZdreNlp7_Oz3QV5ZoHvzQiRIpBEzcndF8 Jul 16 21:13:34.260319 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:13:34.260319 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:13:34.260319 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:13:34.260319 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:13:34.260319 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:13:34.260319 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:13:34.260319 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: 30d028fa9b854c8fb71c8cbed2985c27 Jul 16 21:13:34.260319 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-VolumesAdminNegativeTest-1468695956 Jul 16 21:13:34.260319 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:13:34.260319 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:13:34.261898 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/30d028fa9b854c8fb71c8cbed2985c27"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/30d028fa9b854c8fb71c8cbed2985c27"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_30d028fa9b854c8fb71c8cbed2985c27"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/30d028fa9b854c8fb71c8cbed2985c27"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:13:34.261898 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-VolumesAdminNegativeTest-1468695956 Jul 16 21:13:34.261898 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: 30d028fa9b854c8fb71c8cbed2985c27 Jul 16 21:13:34.261898 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-VolumesAdminNegativeTest-1468695956 Jul 16 21:13:34.261898 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-VolumesAdminNegativeTest-1468695956-project Jul 16 21:13:34.261898 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:13:34.261898 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:13:34.261898 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: 52281b3989a04ca2a50b0bc93bf783f0 Jul 16 21:13:34.261898 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-VolumesAdminNegativeTest-1468695956-project Jul 16 21:13:34.261898 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:13:34.261898 ncdv-master-1 devstack@n-api.service[138681]: exception: Instance b07f0d53-2cd4-440f-be47-077703aa66e6 could not be found. Jul 16 21:13:34.261898 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': 'b07f0d53-2cd4-440f-be47-077703aa66e6'} Jul 16 21:13:34.263114 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:13:34.263114 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:13:34.263114 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:13:34.263114 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:13:34.263114 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:13:34.263114 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:13:34.263114 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:13:34.263114 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:13:34.263114 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:13:34.263114 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:13:34.263114 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance b07f0d53-2cd4-440f-be47-077703aa66e6 could not be found. Jul 16 21:13:34.263114 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:13:34.263114 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:13:34.263114 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:13:34.263114 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:13:34.263114 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:13:34.263114 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:13:34.263114 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:13:34.263114 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:13:34.263114 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:13:34.264646 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:13:34.264646 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:13:34.264646 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:13:34.264646 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:13:34.264646 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:13:34.264646 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:13:34.264646 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:13:34.264646 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance b07f0d53-2cd4-440f-be47-077703aa66e6 could not be found. Jul 16 21:13:34.264646 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:13:34.264646 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:13:34.264646 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:13:34.264646 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:13:34.264646 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:13:34.264646 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:13:34.264646 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:13:34.264646 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:13:34.264646 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance b07f0d53-2cd4-440f-be47-077703aa66e6 could not be found. Jul 16 21:13:34.264646 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:13:34.267881 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-991ee8c7-c441-4cd0-a976-437592fd67dd tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Returning 404 to user: Instance b07f0d53-2cd4-440f-be47-077703aa66e6 could not be found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:13:34.268950 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-991ee8c7-c441-4cd0-a976-437592fd67dd tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] 10.222.0.16 "GET /compute/v2.1/servers/b07f0d53-2cd4-440f-be47-077703aa66e6" status: 404 len: 111 microversion: 2.1 time: 0.083749 Jul 16 21:13:34.269763 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 655/1392] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:13:34 2022] GET /compute/v2.1/servers/b07f0d53-2cd4-440f-be47-077703aa66e6 => generated 111 bytes in 99 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:13:34.280179 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-a2b083ef-b518-4943-8507-156b1f21c33b tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:13:34.282741 ncdv-master-1 devstack@n-api.service[138681]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-services:list": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:13:34.282741 ncdv-master-1 devstack@n-api.service[138681]: warnings.warn(msg) Jul 16 21:13:34.284428 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a2b083ef-b518-4943-8507-156b1f21c33b tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:34.298433 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a2b083ef-b518-4943-8507-156b1f21c33b tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:34.299810 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a2b083ef-b518-4943-8507-156b1f21c33b tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:34.303604 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a2b083ef-b518-4943-8507-156b1f21c33b tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:34.377894 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-a2b083ef-b518-4943-8507-156b1f21c33b tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] 10.222.0.16 "GET /compute/v2.1/os-services?binary=nova-compute" status: 200 len: 424 microversion: 2.25 time: 0.102366 Jul 16 21:13:34.379208 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 656/1393] 10.222.0.16 () {60 vars in 1216 bytes} [Sat Jul 16 21:13:34 2022] GET /compute/v2.1/os-services?binary=nova-compute => generated 424 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 16 21:13:34.405311 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-f32f5c2c-c268-46b0-8c93-7bed10454a18 tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:13:34.405311 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-f32f5c2c-c268-46b0-8c93-7bed10454a18 tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] [instance: 4602867b-88ed-43b1-9ee6-bf661e2d7d99] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:13:34.644184 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-f32f5c2c-c268-46b0-8c93-7bed10454a18 tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c1aea87154224f1c81428ddef4438801', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45d9edd2fe854483a6f0d676efbd6622', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:13:34.657579 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-f32f5c2c-c268-46b0-8c93-7bed10454a18 tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] 10.222.0.16 "GET /compute/v2.1/servers/4602867b-88ed-43b1-9ee6-bf661e2d7d99" status: 200 len: 1221 microversion: 2.1 time: 0.274751 Jul 16 21:13:34.658207 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 657/1394] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:13:34 2022] GET /compute/v2.1/servers/4602867b-88ed-43b1-9ee6-bf661e2d7d99 => generated 1221 bytes in 278 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:13:34.677299 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-41c84d04-8152-45d4-8967-df31fbad05fb tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] Calling method '.version_select at 0x7fdd9d879280>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:13:34.812023 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-41c84d04-8152-45d4-8967-df31fbad05fb tempest-VolumesAdminNegativeTest-1468695956 tempest-VolumesAdminNegativeTest-1468695956-project] 10.222.0.16 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-963257303" status: 202 len: 0 microversion: 2.1 time: 0.144242 Jul 16 21:13:34.812630 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 658/1395] 10.222.0.16 () {58 vars in 1217 bytes} [Sat Jul 16 21:13:34 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-963257303 => generated 0 bytes in 153 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 21:13:34.922102 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-0b9bc88e-ec5d-4077-bcca-1c4b8c79cc6a tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b68dacc4b8463ba07fae6372131117', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55a19b7a4682457a9132f5fc80110712', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:13:34.938562 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-0b9bc88e-ec5d-4077-bcca-1c4b8c79cc6a tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "GET /compute/v2.1/servers/126f561c-163f-4f6d-8be1-98a2e29b0909" status: 200 len: 1471 microversion: 2.1 time: 0.727226 Jul 16 21:13:34.938562 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 738/1396] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:13:34 2022] GET /compute/v2.1/servers/126f561c-163f-4f6d-8be1-98a2e29b0909 => generated 1471 bytes in 732 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:13:34.965674 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-fdeea901-791a-4533-924b-c58e4e1058d8 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Action: 'action', calling method: >, body: {"revertResize": {}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:13:34.967184 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-fdeea901-791a-4533-924b-c58e4e1058d8 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: 126f561c-163f-4f6d-8be1-98a2e29b0909] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:13:34.985556 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fdeea901-791a-4533-924b-c58e4e1058d8 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:34.988138 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fdeea901-791a-4533-924b-c58e4e1058d8 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:34.989755 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fdeea901-791a-4533-924b-c58e4e1058d8 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:34.993831 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fdeea901-791a-4533-924b-c58e4e1058d8 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:35.676422 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-76746e22-d881-4ae2-af6b-967f966ba0c8 tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:13:35.678614 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-76746e22-d881-4ae2-af6b-967f966ba0c8 tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] [instance: 4602867b-88ed-43b1-9ee6-bf661e2d7d99] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:13:35.682799 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-fdeea901-791a-4533-924b-c58e4e1058d8 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "POST /compute/v2.1/servers/126f561c-163f-4f6d-8be1-98a2e29b0909/action" status: 202 len: 0 microversion: 2.1 time: 0.732289 Jul 16 21:13:35.683445 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 659/1397] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 21:13:34 2022] POST /compute/v2.1/servers/126f561c-163f-4f6d-8be1-98a2e29b0909/action => generated 0 bytes in 734 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 21:13:35.694395 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_api:SELECT=95,nova_api:UPDATE=1,nova_api:INSERT=3 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:13:35.701046 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-8b6af8f1-077e-42d5-9927-94c2364fb54e tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:13:35.701939 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-8b6af8f1-077e-42d5-9927-94c2364fb54e tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: 126f561c-163f-4f6d-8be1-98a2e29b0909] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:13:35.733859 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8b6af8f1-077e-42d5-9927-94c2364fb54e tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:35.734344 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8b6af8f1-077e-42d5-9927-94c2364fb54e tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:35.735168 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8b6af8f1-077e-42d5-9927-94c2364fb54e tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:35.735616 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8b6af8f1-077e-42d5-9927-94c2364fb54e tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:36.067590 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-76746e22-d881-4ae2-af6b-967f966ba0c8 tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c1aea87154224f1c81428ddef4438801', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45d9edd2fe854483a6f0d676efbd6622', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:13:36.076686 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-76746e22-d881-4ae2-af6b-967f966ba0c8 tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] 10.222.0.16 "GET /compute/v2.1/servers/4602867b-88ed-43b1-9ee6-bf661e2d7d99" status: 200 len: 1221 microversion: 2.1 time: 0.403867 Jul 16 21:13:36.076686 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 739/1398] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:13:35 2022] GET /compute/v2.1/servers/4602867b-88ed-43b1-9ee6-bf661e2d7d99 => generated 1221 bytes in 405 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:13:36.893503 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-8b6af8f1-077e-42d5-9927-94c2364fb54e tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b68dacc4b8463ba07fae6372131117', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55a19b7a4682457a9132f5fc80110712', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:13:36.903351 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-8b6af8f1-077e-42d5-9927-94c2364fb54e tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "GET /compute/v2.1/servers/126f561c-163f-4f6d-8be1-98a2e29b0909" status: 200 len: 1470 microversion: 2.1 time: 1.206499 Jul 16 21:13:36.904007 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 660/1399] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:13:35 2022] GET /compute/v2.1/servers/126f561c-163f-4f6d-8be1-98a2e29b0909 => generated 1470 bytes in 1208 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:13:37.095862 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-9e6d81b9-e8ec-4549-888b-2aca9e5e266c tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:13:37.097964 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-9e6d81b9-e8ec-4549-888b-2aca9e5e266c tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] [instance: 4602867b-88ed-43b1-9ee6-bf661e2d7d99] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:13:37.113732 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9e6d81b9-e8ec-4549-888b-2aca9e5e266c tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:37.114610 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9e6d81b9-e8ec-4549-888b-2aca9e5e266c tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:37.116765 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9e6d81b9-e8ec-4549-888b-2aca9e5e266c tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:37.118107 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9e6d81b9-e8ec-4549-888b-2aca9e5e266c tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:37.405696 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-9e6d81b9-e8ec-4549-888b-2aca9e5e266c tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c1aea87154224f1c81428ddef4438801', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45d9edd2fe854483a6f0d676efbd6622', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:13:37.419393 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-9e6d81b9-e8ec-4549-888b-2aca9e5e266c tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] 10.222.0.16 "GET /compute/v2.1/servers/4602867b-88ed-43b1-9ee6-bf661e2d7d99" status: 200 len: 1217 microversion: 2.1 time: 0.325788 Jul 16 21:13:37.419860 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 740/1400] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:13:37 2022] GET /compute/v2.1/servers/4602867b-88ed-43b1-9ee6-bf661e2d7d99 => generated 1217 bytes in 329 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:13:37.507632 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-6d801fba-b6b5-4302-b847-a1e7948f3525 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-FloatingIPsAssociationNegativeTestJSON-server-584206483", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452", "networks": [{"uuid": "a27fab11-02d4-4a1c-99a4-dc86a33836ba"}]}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:13:37.594943 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.network.neutron [None req-6d801fba-b6b5-4302-b847-a1e7948f3525 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] validate_networks() for [('a27fab11-02d4-4a1c-99a4-dc86a33836ba', None, None, None, None, None)] {{(pid=138681) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 16 21:13:37.846803 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-c6629054-99e4-49d1-b1b3-a798d7593305 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "cc3f1f89-b013-47d9-982f-2e3f355e5391", "device": "/dev/xvdb"}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:13:37.849145 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-c6629054-99e4-49d1-b1b3-a798d7593305 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] [instance: 3c5ab09e-0a97-4cfc-b249-a11ab221b5d2] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:13:37.857951 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c6629054-99e4-49d1-b1b3-a798d7593305 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:37.858781 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c6629054-99e4-49d1-b1b3-a798d7593305 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:37.868451 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c6629054-99e4-49d1-b1b3-a798d7593305 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:37.870076 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c6629054-99e4-49d1-b1b3-a798d7593305 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.009s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:39.021509 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-c6629054-99e4-49d1-b1b3-a798d7593305 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] 10.222.0.16 "POST /compute/v2.1/servers/3c5ab09e-0a97-4cfc-b249-a11ab221b5d2/os-volume_attachments" status: 200 len: 194 microversion: 2.25 time: 1.189878 Jul 16 21:13:39.025991 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 741/1401] 10.222.0.16 () {62 vars in 1363 bytes} [Sat Jul 16 21:13:37 2022] POST /compute/v2.1/servers/3c5ab09e-0a97-4cfc-b249-a11ab221b5d2/os-volume_attachments => generated 194 bytes in 1196 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 16 21:13:39.401108 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-03030304-2e25-4890-aeb6-ddeae13cbde1 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:13:39.402011 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-03030304-2e25-4890-aeb6-ddeae13cbde1 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: 126f561c-163f-4f6d-8be1-98a2e29b0909] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:13:39.422801 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-03030304-2e25-4890-aeb6-ddeae13cbde1 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:39.430751 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-03030304-2e25-4890-aeb6-ddeae13cbde1 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:39.441880 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-03030304-2e25-4890-aeb6-ddeae13cbde1 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:39.442736 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-03030304-2e25-4890-aeb6-ddeae13cbde1 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:40.362938 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-03030304-2e25-4890-aeb6-ddeae13cbde1 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b68dacc4b8463ba07fae6372131117', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55a19b7a4682457a9132f5fc80110712', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:13:40.374053 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-03030304-2e25-4890-aeb6-ddeae13cbde1 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "GET /compute/v2.1/servers/126f561c-163f-4f6d-8be1-98a2e29b0909" status: 200 len: 1470 microversion: 2.1 time: 1.346283 Jul 16 21:13:40.381978 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 742/1402] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:13:39 2022] GET /compute/v2.1/servers/126f561c-163f-4f6d-8be1-98a2e29b0909 => generated 1470 bytes in 1355 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:13:40.404113 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-833648d8-f2a3-44dc-b306-49c931310e8d tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:13:40.407120 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-833648d8-f2a3-44dc-b306-49c931310e8d tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] [instance: 4602867b-88ed-43b1-9ee6-bf661e2d7d99] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:13:40.427290 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-833648d8-f2a3-44dc-b306-49c931310e8d tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:40.432440 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-833648d8-f2a3-44dc-b306-49c931310e8d tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:40.434865 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-833648d8-f2a3-44dc-b306-49c931310e8d tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:40.436549 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-833648d8-f2a3-44dc-b306-49c931310e8d tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:41.374463 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-833648d8-f2a3-44dc-b306-49c931310e8d tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c1aea87154224f1c81428ddef4438801', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45d9edd2fe854483a6f0d676efbd6622', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:13:41.404460 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-833648d8-f2a3-44dc-b306-49c931310e8d tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] 10.222.0.16 "GET /compute/v2.1/servers/4602867b-88ed-43b1-9ee6-bf661e2d7d99" status: 200 len: 1279 microversion: 2.1 time: 1.020766 Jul 16 21:13:41.405073 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 743/1403] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:13:40 2022] GET /compute/v2.1/servers/4602867b-88ed-43b1-9ee6-bf661e2d7d99 => generated 1279 bytes in 1023 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:13:41.718353 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-b385b4f6-3f08-4e07-8cb1-21e03d3ace34 req-d01ab903-03cd-4b0f-af89-3dfeea711753 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "126f561c-163f-4f6d-8be1-98a2e29b0909", "name": "network-vif-plugged", "status": "completed", "tag": "3b739133-7af5-4df3-a04d-fff7e0a81a3d"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:13:41.738504 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-b385b4f6-3f08-4e07-8cb1-21e03d3ace34 req-d01ab903-03cd-4b0f-af89-3dfeea711753 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:41.739335 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-b385b4f6-3f08-4e07-8cb1-21e03d3ace34 req-d01ab903-03cd-4b0f-af89-3dfeea711753 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:41.740981 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-6d801fba-b6b5-4302-b847-a1e7948f3525 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] Flavor limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 16 21:13:41.741670 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-6d801fba-b6b5-4302-b847-a1e7948f3525 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] Image limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 16 21:13:41.742184 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-6d801fba-b6b5-4302-b847-a1e7948f3525 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] Flavor pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 16 21:13:41.742607 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-6d801fba-b6b5-4302-b847-a1e7948f3525 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] Image pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 16 21:13:41.743047 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-6d801fba-b6b5-4302-b847-a1e7948f3525 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 16 21:13:41.768854 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6d801fba-b6b5-4302-b847-a1e7948f3525 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:41.772375 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6d801fba-b6b5-4302-b847-a1e7948f3525 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:41.776123 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6d801fba-b6b5-4302-b847-a1e7948f3525 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:41.778506 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6d801fba-b6b5-4302-b847-a1e7948f3525 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:41.797614 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-b385b4f6-3f08-4e07-8cb1-21e03d3ace34 req-d01ab903-03cd-4b0f-af89-3dfeea711753 service nova] Creating event network-vif-plugged:3b739133-7af5-4df3-a04d-fff7e0a81a3d for instance 126f561c-163f-4f6d-8be1-98a2e29b0909 on nch1-master-1 Jul 16 21:13:41.813753 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [req-b385b4f6-3f08-4e07-8cb1-21e03d3ace34 req-d01ab903-03cd-4b0f-af89-3dfeea711753 service nova] Instance 126f561c-163f-4f6d-8be1-98a2e29b0909 is migrating within the same cell, copying events to all relevant hosts: {'nch2-master-1', 'nch1-master-1'} {{(pid=138680) _get_relevant_hosts /opt/stack/nova/nova/compute/api.py:5807}} Jul 16 21:13:41.863343 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-b385b4f6-3f08-4e07-8cb1-21e03d3ace34 req-d01ab903-03cd-4b0f-af89-3dfeea711753 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.456901 Jul 16 21:13:41.882017 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 744/1404] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:13:41 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 476 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:13:41.902044 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-4b16c044-4fae-41eb-bb8d-b4fc5d505657 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:13:41.910784 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-4b16c044-4fae-41eb-bb8d-b4fc5d505657 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: 126f561c-163f-4f6d-8be1-98a2e29b0909] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:13:41.932416 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4b16c044-4fae-41eb-bb8d-b4fc5d505657 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:41.933326 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4b16c044-4fae-41eb-bb8d-b4fc5d505657 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.009s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:41.934473 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4b16c044-4fae-41eb-bb8d-b4fc5d505657 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:41.935167 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4b16c044-4fae-41eb-bb8d-b4fc5d505657 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:41.937657 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-6d801fba-b6b5-4302-b847-a1e7948f3525 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] Getting quotas for project 6c2095f96f8d45bb9e62c517678c3dcd. Resources: {'ram', 'instances', 'cores'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:13:41.967851 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-6d801fba-b6b5-4302-b847-a1e7948f3525 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] Getting quotas for user 2eba2ac21c344394a7d3f605775b8713 and project 6c2095f96f8d45bb9e62c517678c3dcd. Resources: {'ram', 'instances', 'cores'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:13:42.021023 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-6d801fba-b6b5-4302-b847-a1e7948f3525 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] Going to run 1 instances... {{(pid=138681) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 16 21:13:42.023971 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-6d801fba-b6b5-4302-b847-a1e7948f3525 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] [instance: dada2ea0-2ef4-4a30-8e2d-40177d0bafce] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138681) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 16 21:13:42.240718 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-6d801fba-b6b5-4302-b847-a1e7948f3525 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 4.819649 Jul 16 21:13:42.244696 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 661/1405] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 21:13:37 2022] POST /compute/v2.1/servers => generated 380 bytes in 4825 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 16 21:13:42.283785 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-7a75bd35-10e9-4e57-ba2e-7a247f314793 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:13:42.293412 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-7a75bd35-10e9-4e57-ba2e-7a247f314793 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] [instance: dada2ea0-2ef4-4a30-8e2d-40177d0bafce] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:13:42.354739 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-4b16c044-4fae-41eb-bb8d-b4fc5d505657 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b68dacc4b8463ba07fae6372131117', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55a19b7a4682457a9132f5fc80110712', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:13:42.363689 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-4b16c044-4fae-41eb-bb8d-b4fc5d505657 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "GET /compute/v2.1/servers/126f561c-163f-4f6d-8be1-98a2e29b0909" status: 200 len: 1470 microversion: 2.1 time: 0.480294 Jul 16 21:13:42.373757 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 745/1406] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:13:41 2022] GET /compute/v2.1/servers/126f561c-163f-4f6d-8be1-98a2e29b0909 => generated 1470 bytes in 491 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:13:42.422997 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-5636601a-a6b4-4405-a7a4-7250de1cb3d0 tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:13:42.431402 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-5636601a-a6b4-4405-a7a4-7250de1cb3d0 tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] [instance: 4602867b-88ed-43b1-9ee6-bf661e2d7d99] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:13:42.447694 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5636601a-a6b4-4405-a7a4-7250de1cb3d0 tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:42.447899 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5636601a-a6b4-4405-a7a4-7250de1cb3d0 tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:42.448655 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5636601a-a6b4-4405-a7a4-7250de1cb3d0 tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:42.449103 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5636601a-a6b4-4405-a7a4-7250de1cb3d0 tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:42.512241 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:13:42.573825 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-7a75bd35-10e9-4e57-ba2e-7a247f314793 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2eba2ac21c344394a7d3f605775b8713', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c2095f96f8d45bb9e62c517678c3dcd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:13:42.586905 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-7a75bd35-10e9-4e57-ba2e-7a247f314793 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] 10.222.0.16 "GET /compute/v2.1/servers/dada2ea0-2ef4-4a30-8e2d-40177d0bafce" status: 200 len: 1231 microversion: 2.1 time: 0.331652 Jul 16 21:13:42.590398 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 662/1407] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:13:42 2022] GET /compute/v2.1/servers/dada2ea0-2ef4-4a30-8e2d-40177d0bafce => generated 1231 bytes in 340 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:13:42.932197 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-5636601a-a6b4-4405-a7a4-7250de1cb3d0 tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c1aea87154224f1c81428ddef4438801', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45d9edd2fe854483a6f0d676efbd6622', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:13:42.948172 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-5636601a-a6b4-4405-a7a4-7250de1cb3d0 tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] 10.222.0.16 "GET /compute/v2.1/servers/4602867b-88ed-43b1-9ee6-bf661e2d7d99" status: 200 len: 1321 microversion: 2.1 time: 0.528534 Jul 16 21:13:42.952617 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 746/1408] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:13:42 2022] GET /compute/v2.1/servers/4602867b-88ed-43b1-9ee6-bf661e2d7d99 => generated 1321 bytes in 534 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:13:43.401585 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-cfd6d685-153e-4274-93b3-8ff7fe62b3d6 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:13:43.404213 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-cfd6d685-153e-4274-93b3-8ff7fe62b3d6 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: 126f561c-163f-4f6d-8be1-98a2e29b0909] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:13:43.470790 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cfd6d685-153e-4274-93b3-8ff7fe62b3d6 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:43.471639 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cfd6d685-153e-4274-93b3-8ff7fe62b3d6 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:43.474351 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cfd6d685-153e-4274-93b3-8ff7fe62b3d6 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:43.491762 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cfd6d685-153e-4274-93b3-8ff7fe62b3d6 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:43.610973 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-da6b0460-89db-4427-ad60-04ef69aa2b75 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:13:43.627848 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-da6b0460-89db-4427-ad60-04ef69aa2b75 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] [instance: dada2ea0-2ef4-4a30-8e2d-40177d0bafce] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:13:43.833254 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell0:SELECT=2 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:13:43.951103 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-da6b0460-89db-4427-ad60-04ef69aa2b75 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2eba2ac21c344394a7d3f605775b8713', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c2095f96f8d45bb9e62c517678c3dcd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:13:43.998795 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-da6b0460-89db-4427-ad60-04ef69aa2b75 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] 10.222.0.16 "GET /compute/v2.1/servers/dada2ea0-2ef4-4a30-8e2d-40177d0bafce" status: 200 len: 1231 microversion: 2.1 time: 0.397563 Jul 16 21:13:44.000463 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 747/1409] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:13:43 2022] GET /compute/v2.1/servers/dada2ea0-2ef4-4a30-8e2d-40177d0bafce => generated 1231 bytes in 400 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:13:44.033368 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-04900593-55be-4034-8b53-2b4105c2f82c tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:13:44.039898 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-04900593-55be-4034-8b53-2b4105c2f82c tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] [instance: 4602867b-88ed-43b1-9ee6-bf661e2d7d99] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:13:44.056631 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-04900593-55be-4034-8b53-2b4105c2f82c tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:44.061912 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-04900593-55be-4034-8b53-2b4105c2f82c tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:44.061912 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-04900593-55be-4034-8b53-2b4105c2f82c tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:44.061912 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-04900593-55be-4034-8b53-2b4105c2f82c tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:44.165055 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-cfd6d685-153e-4274-93b3-8ff7fe62b3d6 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b68dacc4b8463ba07fae6372131117', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55a19b7a4682457a9132f5fc80110712', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:13:44.181940 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-cfd6d685-153e-4274-93b3-8ff7fe62b3d6 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "GET /compute/v2.1/servers/126f561c-163f-4f6d-8be1-98a2e29b0909" status: 200 len: 1470 microversion: 2.1 time: 0.804202 Jul 16 21:13:44.201125 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 663/1410] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:13:43 2022] GET /compute/v2.1/servers/126f561c-163f-4f6d-8be1-98a2e29b0909 => generated 1470 bytes in 824 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:13:45.024311 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-914946ff-ade9-4bb3-a158-d05f0d3e5933 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:13:45.025608 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-914946ff-ade9-4bb3-a158-d05f0d3e5933 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] [instance: dada2ea0-2ef4-4a30-8e2d-40177d0bafce] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:13:45.057026 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-914946ff-ade9-4bb3-a158-d05f0d3e5933 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:45.058225 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-914946ff-ade9-4bb3-a158-d05f0d3e5933 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.011s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:45.060369 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-914946ff-ade9-4bb3-a158-d05f0d3e5933 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:45.061030 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-914946ff-ade9-4bb3-a158-d05f0d3e5933 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:45.185527 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-04900593-55be-4034-8b53-2b4105c2f82c tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c1aea87154224f1c81428ddef4438801', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45d9edd2fe854483a6f0d676efbd6622', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:13:45.202841 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-04900593-55be-4034-8b53-2b4105c2f82c tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] 10.222.0.16 "GET /compute/v2.1/servers/4602867b-88ed-43b1-9ee6-bf661e2d7d99" status: 200 len: 1321 microversion: 2.1 time: 1.187193 Jul 16 21:13:45.210847 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 748/1411] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:13:44 2022] GET /compute/v2.1/servers/4602867b-88ed-43b1-9ee6-bf661e2d7d99 => generated 1321 bytes in 1207 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:13:45.234134 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-bd76753b-72c0-47ff-be89-dae21574a43e tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:13:45.246157 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-bd76753b-72c0-47ff-be89-dae21574a43e tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] [instance: 3c5ab09e-0a97-4cfc-b249-a11ab221b5d2] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:13:45.265254 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-bd76753b-72c0-47ff-be89-dae21574a43e tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:45.266096 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-bd76753b-72c0-47ff-be89-dae21574a43e tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:45.267149 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-bd76753b-72c0-47ff-be89-dae21574a43e tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:45.275677 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-bd76753b-72c0-47ff-be89-dae21574a43e tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:45.484207 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-914946ff-ade9-4bb3-a158-d05f0d3e5933 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2eba2ac21c344394a7d3f605775b8713', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c2095f96f8d45bb9e62c517678c3dcd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:13:45.508918 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-914946ff-ade9-4bb3-a158-d05f0d3e5933 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] 10.222.0.16 "GET /compute/v2.1/servers/dada2ea0-2ef4-4a30-8e2d-40177d0bafce" status: 200 len: 1235 microversion: 2.1 time: 0.496626 Jul 16 21:13:45.513242 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 664/1412] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:13:45 2022] GET /compute/v2.1/servers/dada2ea0-2ef4-4a30-8e2d-40177d0bafce => generated 1235 bytes in 502 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:13:45.530450 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-4593be17-beed-40f0-ac0a-57bc6a4bfffa tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:13:45.531320 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-4593be17-beed-40f0-ac0a-57bc6a4bfffa tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: 126f561c-163f-4f6d-8be1-98a2e29b0909] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:13:45.541947 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4593be17-beed-40f0-ac0a-57bc6a4bfffa tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:45.542893 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4593be17-beed-40f0-ac0a-57bc6a4bfffa tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:45.543995 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4593be17-beed-40f0-ac0a-57bc6a4bfffa tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:45.544910 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4593be17-beed-40f0-ac0a-57bc6a4bfffa tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:46.653806 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.objects.instance [None req-bd76753b-72c0-47ff-be89-dae21574a43e tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] Lazy-loading 'services' on Instance uuid 3c5ab09e-0a97-4cfc-b249-a11ab221b5d2 {{(pid=138680) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:13:46.716262 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-bd76753b-72c0-47ff-be89-dae21574a43e tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/3c5ab09e-0a97-4cfc-b249-a11ab221b5d2" status: 200 len: 2417 microversion: 2.25 time: 1.503849 Jul 16 21:13:46.721112 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 749/1413] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:13:45 2022] GET /compute/v2.1/servers/3c5ab09e-0a97-4cfc-b249-a11ab221b5d2 => generated 2417 bytes in 1509 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:13:46.740495 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-0e87bb94-2aaf-4ae2-8e5f-ca6e6f1d655f req-5f5221e0-8f96-4929-885e-07bca3cac690 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "4602867b-88ed-43b1-9ee6-bf661e2d7d99", "tag": "081d646d-03f3-4a3f-8409-9106695a46bb"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:13:46.777192 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-0e87bb94-2aaf-4ae2-8e5f-ca6e6f1d655f req-5f5221e0-8f96-4929-885e-07bca3cac690 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:46.777661 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-0e87bb94-2aaf-4ae2-8e5f-ca6e6f1d655f req-5f5221e0-8f96-4929-885e-07bca3cac690 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:46.784588 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-4593be17-beed-40f0-ac0a-57bc6a4bfffa tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b68dacc4b8463ba07fae6372131117', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55a19b7a4682457a9132f5fc80110712', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:13:46.807378 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-4593be17-beed-40f0-ac0a-57bc6a4bfffa tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "GET /compute/v2.1/servers/126f561c-163f-4f6d-8be1-98a2e29b0909" status: 200 len: 1470 microversion: 2.1 time: 1.292050 Jul 16 21:13:46.814462 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 665/1414] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:13:45 2022] GET /compute/v2.1/servers/126f561c-163f-4f6d-8be1-98a2e29b0909 => generated 1470 bytes in 1296 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:13:46.822268 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-0e87bb94-2aaf-4ae2-8e5f-ca6e6f1d655f req-5f5221e0-8f96-4929-885e-07bca3cac690 service nova] Creating event network-changed:081d646d-03f3-4a3f-8409-9106695a46bb for instance 4602867b-88ed-43b1-9ee6-bf661e2d7d99 on nch1-master-1 Jul 16 21:13:46.843974 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-0e87bb94-2aaf-4ae2-8e5f-ca6e6f1d655f req-5f5221e0-8f96-4929-885e-07bca3cac690 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.121270 Jul 16 21:13:46.845894 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 750/1415] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:13:46 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 124 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:13:46.861485 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-03c59856-23cc-4606-a0c3-6ead7ccfb74b tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:13:46.863861 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-03c59856-23cc-4606-a0c3-6ead7ccfb74b tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] [instance: 4602867b-88ed-43b1-9ee6-bf661e2d7d99] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:13:46.891042 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-03c59856-23cc-4606-a0c3-6ead7ccfb74b tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:46.891549 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-03c59856-23cc-4606-a0c3-6ead7ccfb74b tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:46.892358 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-03c59856-23cc-4606-a0c3-6ead7ccfb74b tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:46.895273 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-03c59856-23cc-4606-a0c3-6ead7ccfb74b tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:46.992899 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-f2e73bf7-9902-4ccd-a7b1-bcf2d3c7f713 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ImagesOneServerNegativeTestJSON-server-756228293", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452", "networks": [{"uuid": "45344261-e9f3-4a7b-95e1-5380175d73fa"}]}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:13:47.111527 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.network.neutron [None req-f2e73bf7-9902-4ccd-a7b1-bcf2d3c7f713 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] validate_networks() for [('45344261-e9f3-4a7b-95e1-5380175d73fa', None, None, None, None, None)] {{(pid=138681) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 16 21:13:48.770493 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-03c59856-23cc-4606-a0c3-6ead7ccfb74b tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c1aea87154224f1c81428ddef4438801', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45d9edd2fe854483a6f0d676efbd6622', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:13:48.781542 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-03c59856-23cc-4606-a0c3-6ead7ccfb74b tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] 10.222.0.16 "GET /compute/v2.1/servers/4602867b-88ed-43b1-9ee6-bf661e2d7d99" status: 200 len: 1321 microversion: 2.1 time: 1.932962 Jul 16 21:13:48.786587 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 751/1416] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:13:46 2022] GET /compute/v2.1/servers/4602867b-88ed-43b1-9ee6-bf661e2d7d99 => generated 1321 bytes in 1940 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:13:48.793924 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-fe7a5ff8-a53a-4983-819b-66e60758efc2 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:13:48.799644 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-fe7a5ff8-a53a-4983-819b-66e60758efc2 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] [instance: dada2ea0-2ef4-4a30-8e2d-40177d0bafce] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:13:48.822605 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-fe7a5ff8-a53a-4983-819b-66e60758efc2 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:48.823102 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-fe7a5ff8-a53a-4983-819b-66e60758efc2 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:48.823896 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-fe7a5ff8-a53a-4983-819b-66e60758efc2 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:48.827953 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-fe7a5ff8-a53a-4983-819b-66e60758efc2 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:48.986592 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-fe7a5ff8-a53a-4983-819b-66e60758efc2 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2eba2ac21c344394a7d3f605775b8713', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c2095f96f8d45bb9e62c517678c3dcd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:13:48.995339 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-fe7a5ff8-a53a-4983-819b-66e60758efc2 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] 10.222.0.16 "GET /compute/v2.1/servers/dada2ea0-2ef4-4a30-8e2d-40177d0bafce" status: 200 len: 1289 microversion: 2.1 time: 0.207296 Jul 16 21:13:49.002214 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 752/1417] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:13:48 2022] GET /compute/v2.1/servers/dada2ea0-2ef4-4a30-8e2d-40177d0bafce => generated 1289 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:13:49.010112 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-c29ae4ae-e4ef-42f8-8928-cc66ca2cbfd8 tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] Action: 'action', calling method: >, body: {"os-migrateLive": {"host": "nch1-master-1", "block_migration": "auto"}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:13:49.013286 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-c29ae4ae-e4ef-42f8-8928-cc66ca2cbfd8 tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] [instance: 3c5ab09e-0a97-4cfc-b249-a11ab221b5d2] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:13:49.027846 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c29ae4ae-e4ef-42f8-8928-cc66ca2cbfd8 tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:49.028305 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c29ae4ae-e4ef-42f8-8928-cc66ca2cbfd8 tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:49.032125 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c29ae4ae-e4ef-42f8-8928-cc66ca2cbfd8 tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:49.032799 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c29ae4ae-e4ef-42f8-8928-cc66ca2cbfd8 tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:49.082128 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.objects.instance [None req-c29ae4ae-e4ef-42f8-8928-cc66ca2cbfd8 tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] Lazy-loading 'flavor' on Instance uuid 3c5ab09e-0a97-4cfc-b249-a11ab221b5d2 {{(pid=138680) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:13:49.107991 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-c29ae4ae-e4ef-42f8-8928-cc66ca2cbfd8 tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] [instance: 3c5ab09e-0a97-4cfc-b249-a11ab221b5d2] Going to try to live migrate instance to nch1-master-1 {{(pid=138680) live_migrate /opt/stack/nova/nova/compute/api.py:5349}} Jul 16 21:13:50.047851 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-f2e73bf7-9902-4ccd-a7b1-bcf2d3c7f713 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Flavor limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 16 21:13:50.047851 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-f2e73bf7-9902-4ccd-a7b1-bcf2d3c7f713 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Image limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 16 21:13:50.047851 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-f2e73bf7-9902-4ccd-a7b1-bcf2d3c7f713 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Flavor pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 16 21:13:50.047851 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-f2e73bf7-9902-4ccd-a7b1-bcf2d3c7f713 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Image pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 16 21:13:50.047851 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-f2e73bf7-9902-4ccd-a7b1-bcf2d3c7f713 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 16 21:13:50.066474 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f2e73bf7-9902-4ccd-a7b1-bcf2d3c7f713 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:50.069401 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f2e73bf7-9902-4ccd-a7b1-bcf2d3c7f713 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:50.073452 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f2e73bf7-9902-4ccd-a7b1-bcf2d3c7f713 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:50.073971 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f2e73bf7-9902-4ccd-a7b1-bcf2d3c7f713 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:50.125255 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-f2e73bf7-9902-4ccd-a7b1-bcf2d3c7f713 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Getting quotas for project 4337575622d04a26942905c9dc044ba7. Resources: {'ram', 'instances', 'cores'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:13:50.142490 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-f2e73bf7-9902-4ccd-a7b1-bcf2d3c7f713 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Getting quotas for user 4e9df82f95ac4779859972403869096a and project 4337575622d04a26942905c9dc044ba7. Resources: {'ram', 'instances', 'cores'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:13:50.163611 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-f2e73bf7-9902-4ccd-a7b1-bcf2d3c7f713 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Going to run 1 instances... {{(pid=138681) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 16 21:13:50.175252 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-f2e73bf7-9902-4ccd-a7b1-bcf2d3c7f713 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] [instance: e66baae9-d488-46a4-8347-623030fc8b69] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138681) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 16 21:13:52.749033 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-f2e73bf7-9902-4ccd-a7b1-bcf2d3c7f713 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 5.934038 Jul 16 21:13:52.749033 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 666/1418] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 21:13:46 2022] POST /compute/v2.1/servers => generated 380 bytes in 5936 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 16 21:13:52.752325 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-d4b2df32-ff31-459e-aaff-7fb906318a0d tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:13:52.753177 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-d4b2df32-ff31-459e-aaff-7fb906318a0d tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: 126f561c-163f-4f6d-8be1-98a2e29b0909] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:13:52.760450 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d4b2df32-ff31-459e-aaff-7fb906318a0d tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:52.761698 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d4b2df32-ff31-459e-aaff-7fb906318a0d tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:52.763245 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d4b2df32-ff31-459e-aaff-7fb906318a0d tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:52.763860 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d4b2df32-ff31-459e-aaff-7fb906318a0d tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:53.361385 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-d4b2df32-ff31-459e-aaff-7fb906318a0d tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b68dacc4b8463ba07fae6372131117', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55a19b7a4682457a9132f5fc80110712', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:13:53.372855 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-d4b2df32-ff31-459e-aaff-7fb906318a0d tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "GET /compute/v2.1/servers/126f561c-163f-4f6d-8be1-98a2e29b0909" status: 200 len: 1470 microversion: 2.1 time: 0.624834 Jul 16 21:13:53.376094 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 667/1419] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:13:52 2022] GET /compute/v2.1/servers/126f561c-163f-4f6d-8be1-98a2e29b0909 => generated 1470 bytes in 629 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:13:53.402285 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-8dd26120-3058-4dca-ada1-34f47cecc8d6 req-814991d3-fdb4-4f6f-997d-9e73f37f6327 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "126f561c-163f-4f6d-8be1-98a2e29b0909", "name": "network-vif-unplugged", "status": "completed", "tag": "3b739133-7af5-4df3-a04d-fff7e0a81a3d"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:13:53.428095 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-8dd26120-3058-4dca-ada1-34f47cecc8d6 req-814991d3-fdb4-4f6f-997d-9e73f37f6327 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:53.430504 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-8dd26120-3058-4dca-ada1-34f47cecc8d6 req-814991d3-fdb4-4f6f-997d-9e73f37f6327 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:53.449821 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-8dd26120-3058-4dca-ada1-34f47cecc8d6 req-814991d3-fdb4-4f6f-997d-9e73f37f6327 service nova] Creating event network-vif-unplugged:3b739133-7af5-4df3-a04d-fff7e0a81a3d for instance 126f561c-163f-4f6d-8be1-98a2e29b0909 on nch2-master-1 Jul 16 21:13:53.455836 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-8dd26120-3058-4dca-ada1-34f47cecc8d6 req-814991d3-fdb4-4f6f-997d-9e73f37f6327 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.078176 Jul 16 21:13:53.471182 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 668/1420] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:13:53 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:13:53.475179 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-3a67c9b9-7261-44f6-a895-e800b68c4c04 tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:13:53.476166 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-3a67c9b9-7261-44f6-a895-e800b68c4c04 tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] [instance: 4602867b-88ed-43b1-9ee6-bf661e2d7d99] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:13:53.499138 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3a67c9b9-7261-44f6-a895-e800b68c4c04 tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:53.499620 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3a67c9b9-7261-44f6-a895-e800b68c4c04 tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:53.500406 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3a67c9b9-7261-44f6-a895-e800b68c4c04 tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:53.500849 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3a67c9b9-7261-44f6-a895-e800b68c4c04 tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:53.803156 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-3a67c9b9-7261-44f6-a895-e800b68c4c04 tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c1aea87154224f1c81428ddef4438801', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45d9edd2fe854483a6f0d676efbd6622', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:13:53.812699 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-3a67c9b9-7261-44f6-a895-e800b68c4c04 tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] 10.222.0.16 "GET /compute/v2.1/servers/4602867b-88ed-43b1-9ee6-bf661e2d7d99" status: 200 len: 1321 microversion: 2.1 time: 0.352998 Jul 16 21:13:53.817929 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 669/1421] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:13:53 2022] GET /compute/v2.1/servers/4602867b-88ed-43b1-9ee6-bf661e2d7d99 => generated 1321 bytes in 358 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:13:53.834024 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-ecc5126d-3e38-46ea-8e4b-09e360990a36 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServersTestMultiNic-server-305433575", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452", "networks": [{"uuid": "3dfb8009-2bdd-4def-bb34-608ada61f925"}, {"uuid": "56e2ee19-cccf-4b93-ba8b-ced25d2a6e45"}, {"uuid": "3dfb8009-2bdd-4def-bb34-608ada61f925"}]}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:13:53.918255 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.network.neutron [None req-ecc5126d-3e38-46ea-8e4b-09e360990a36 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] validate_networks() for [('3dfb8009-2bdd-4def-bb34-608ada61f925', None, None, None, None, None), ('56e2ee19-cccf-4b93-ba8b-ced25d2a6e45', None, None, None, None, None), ('3dfb8009-2bdd-4def-bb34-608ada61f925', None, None, None, None, None)] {{(pid=138681) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 16 21:13:53.978905 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell0:SELECT=10 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:13:56.870984 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-c29ae4ae-e4ef-42f8-8928-cc66ca2cbfd8 tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] 10.222.0.16 "POST /compute/v2.1/servers/3c5ab09e-0a97-4cfc-b249-a11ab221b5d2/action" status: 202 len: 0 microversion: 2.25 time: 7.862540 Jul 16 21:13:56.870984 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 753/1422] 10.222.0.16 () {62 vars in 1318 bytes} [Sat Jul 16 21:13:49 2022] POST /compute/v2.1/servers/3c5ab09e-0a97-4cfc-b249-a11ab221b5d2/action => generated 0 bytes in 7866 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Jul 16 21:13:56.877472 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-10f1e18a-0396-4c93-a0fe-8828ab05a4d7 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:13:56.878684 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-10f1e18a-0396-4c93-a0fe-8828ab05a4d7 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] [instance: dada2ea0-2ef4-4a30-8e2d-40177d0bafce] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:13:56.899112 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-10f1e18a-0396-4c93-a0fe-8828ab05a4d7 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:56.899999 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-10f1e18a-0396-4c93-a0fe-8828ab05a4d7 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:56.918094 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-10f1e18a-0396-4c93-a0fe-8828ab05a4d7 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:56.919727 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-10f1e18a-0396-4c93-a0fe-8828ab05a4d7 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:56.982860 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-ecc5126d-3e38-46ea-8e4b-09e360990a36 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Flavor limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 16 21:13:56.984288 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-ecc5126d-3e38-46ea-8e4b-09e360990a36 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Image limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 16 21:13:56.984997 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-ecc5126d-3e38-46ea-8e4b-09e360990a36 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Flavor pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 16 21:13:56.985827 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-ecc5126d-3e38-46ea-8e4b-09e360990a36 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Image pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 16 21:13:56.986521 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-ecc5126d-3e38-46ea-8e4b-09e360990a36 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 16 21:13:57.010689 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_api:SELECT=90,nova_api:UPDATE=1,nova_api:DELETE=1,nova_api:INSERT=6 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:13:57.062746 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell0:SELECT=17 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:13:57.084339 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ecc5126d-3e38-46ea-8e4b-09e360990a36 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:57.091494 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ecc5126d-3e38-46ea-8e4b-09e360990a36 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:57.096732 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ecc5126d-3e38-46ea-8e4b-09e360990a36 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:57.097614 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ecc5126d-3e38-46ea-8e4b-09e360990a36 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:57.161008 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-ecc5126d-3e38-46ea-8e4b-09e360990a36 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Getting quotas for project bb1ede0d0b6941cc97278e6c617f9b01. Resources: {'ram', 'instances', 'cores'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:13:57.176889 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-ecc5126d-3e38-46ea-8e4b-09e360990a36 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Getting quotas for user 53af488f3b0242a797f539284b78d59e and project bb1ede0d0b6941cc97278e6c617f9b01. Resources: {'ram', 'instances', 'cores'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:13:57.222370 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-ecc5126d-3e38-46ea-8e4b-09e360990a36 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Going to run 1 instances... {{(pid=138681) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 16 21:13:57.230009 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-ecc5126d-3e38-46ea-8e4b-09e360990a36 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] [instance: 69cabe57-da71-4a02-b4ee-c0d84b3d32a2] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138681) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 16 21:13:57.370189 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-10f1e18a-0396-4c93-a0fe-8828ab05a4d7 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2eba2ac21c344394a7d3f605775b8713', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c2095f96f8d45bb9e62c517678c3dcd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:13:57.381610 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-10f1e18a-0396-4c93-a0fe-8828ab05a4d7 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] 10.222.0.16 "GET /compute/v2.1/servers/dada2ea0-2ef4-4a30-8e2d-40177d0bafce" status: 200 len: 1331 microversion: 2.1 time: 0.511473 Jul 16 21:13:57.384539 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 754/1423] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:13:56 2022] GET /compute/v2.1/servers/dada2ea0-2ef4-4a30-8e2d-40177d0bafce => generated 1331 bytes in 513 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:13:57.397055 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-8419b475-e72e-4e71-90d0-bc314723ef12 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:13:57.402417 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-8419b475-e72e-4e71-90d0-bc314723ef12 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] [instance: e66baae9-d488-46a4-8347-623030fc8b69] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:13:57.419988 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8419b475-e72e-4e71-90d0-bc314723ef12 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:57.420733 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8419b475-e72e-4e71-90d0-bc314723ef12 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:57.423539 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8419b475-e72e-4e71-90d0-bc314723ef12 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:57.424358 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8419b475-e72e-4e71-90d0-bc314723ef12 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:57.483266 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-ecc5126d-3e38-46ea-8e4b-09e360990a36 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 3.653118 Jul 16 21:13:57.484975 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 670/1424] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 21:13:53 2022] POST /compute/v2.1/servers => generated 380 bytes in 3666 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 16 21:13:57.494650 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-b812953d-bb22-4180-9c94-0eeb7eb3957e tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:13:57.495385 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-b812953d-bb22-4180-9c94-0eeb7eb3957e tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: 126f561c-163f-4f6d-8be1-98a2e29b0909] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:13:57.503884 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b812953d-bb22-4180-9c94-0eeb7eb3957e tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:57.504293 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b812953d-bb22-4180-9c94-0eeb7eb3957e tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:57.505410 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b812953d-bb22-4180-9c94-0eeb7eb3957e tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:57.506047 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b812953d-bb22-4180-9c94-0eeb7eb3957e tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:57.709292 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-8419b475-e72e-4e71-90d0-bc314723ef12 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4e9df82f95ac4779859972403869096a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4337575622d04a26942905c9dc044ba7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:13:57.720169 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-8419b475-e72e-4e71-90d0-bc314723ef12 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] 10.222.0.16 "GET /compute/v2.1/servers/e66baae9-d488-46a4-8347-623030fc8b69" status: 200 len: 1282 microversion: 2.1 time: 0.334465 Jul 16 21:13:57.720791 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 755/1425] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:13:57 2022] GET /compute/v2.1/servers/e66baae9-d488-46a4-8347-623030fc8b69 => generated 1282 bytes in 336 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:13:57.744495 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-435057ef-588a-4e3c-8530-2d948bb1ea46 tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:13:57.745156 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-435057ef-588a-4e3c-8530-2d948bb1ea46 tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] [instance: 4602867b-88ed-43b1-9ee6-bf661e2d7d99] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:13:57.758538 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-435057ef-588a-4e3c-8530-2d948bb1ea46 tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:57.759037 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-435057ef-588a-4e3c-8530-2d948bb1ea46 tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:57.759918 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-435057ef-588a-4e3c-8530-2d948bb1ea46 tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:57.760380 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-435057ef-588a-4e3c-8530-2d948bb1ea46 tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:57.982569 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-435057ef-588a-4e3c-8530-2d948bb1ea46 tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c1aea87154224f1c81428ddef4438801', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45d9edd2fe854483a6f0d676efbd6622', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:13:57.992676 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-435057ef-588a-4e3c-8530-2d948bb1ea46 tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] 10.222.0.16 "GET /compute/v2.1/servers/4602867b-88ed-43b1-9ee6-bf661e2d7d99" status: 200 len: 1321 microversion: 2.1 time: 0.255817 Jul 16 21:13:57.995928 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 756/1426] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:13:57 2022] GET /compute/v2.1/servers/4602867b-88ed-43b1-9ee6-bf661e2d7d99 => generated 1321 bytes in 272 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:13:57.999375 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-b0e18eae-9975-4c62-a7bc-0a67b4537917 req-f957c9e5-c196-4414-953b-eb3a3fc826c0 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "126f561c-163f-4f6d-8be1-98a2e29b0909", "tag": "3b739133-7af5-4df3-a04d-fff7e0a81a3d"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:13:58.010418 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-b0e18eae-9975-4c62-a7bc-0a67b4537917 req-f957c9e5-c196-4414-953b-eb3a3fc826c0 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:58.012348 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-b0e18eae-9975-4c62-a7bc-0a67b4537917 req-f957c9e5-c196-4414-953b-eb3a3fc826c0 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:58.026654 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-b0e18eae-9975-4c62-a7bc-0a67b4537917 req-f957c9e5-c196-4414-953b-eb3a3fc826c0 service nova] Creating event network-changed:3b739133-7af5-4df3-a04d-fff7e0a81a3d for instance 126f561c-163f-4f6d-8be1-98a2e29b0909 on nch2-master-1 Jul 16 21:13:58.032661 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-b0e18eae-9975-4c62-a7bc-0a67b4537917 req-f957c9e5-c196-4414-953b-eb3a3fc826c0 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.036683 Jul 16 21:13:58.033987 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 757/1427] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:13:57 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:13:58.041451 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-0fbbb33b-60c7-4ff7-9546-57df29dd3163 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:13:58.042818 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-0fbbb33b-60c7-4ff7-9546-57df29dd3163 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] [instance: 3c5ab09e-0a97-4cfc-b249-a11ab221b5d2] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:13:58.050155 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0fbbb33b-60c7-4ff7-9546-57df29dd3163 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:58.051114 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0fbbb33b-60c7-4ff7-9546-57df29dd3163 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:58.052647 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0fbbb33b-60c7-4ff7-9546-57df29dd3163 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:58.053335 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0fbbb33b-60c7-4ff7-9546-57df29dd3163 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:58.299869 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-b812953d-bb22-4180-9c94-0eeb7eb3957e tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b68dacc4b8463ba07fae6372131117', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55a19b7a4682457a9132f5fc80110712', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:13:58.312250 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-b812953d-bb22-4180-9c94-0eeb7eb3957e tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "GET /compute/v2.1/servers/126f561c-163f-4f6d-8be1-98a2e29b0909" status: 200 len: 1463 microversion: 2.1 time: 0.823738 Jul 16 21:13:58.314738 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 671/1428] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:13:57 2022] GET /compute/v2.1/servers/126f561c-163f-4f6d-8be1-98a2e29b0909 => generated 1463 bytes in 828 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:13:58.336140 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-8a2c608a-3a9d-4644-92b3-f1575c6f5489 req-3cc76e8e-9a2c-4155-8437-7c69e75aeaf2 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "4602867b-88ed-43b1-9ee6-bf661e2d7d99", "name": "network-vif-plugged", "status": "completed", "tag": "081d646d-03f3-4a3f-8409-9106695a46bb"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:13:58.348163 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-8a2c608a-3a9d-4644-92b3-f1575c6f5489 req-3cc76e8e-9a2c-4155-8437-7c69e75aeaf2 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:58.349637 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-8a2c608a-3a9d-4644-92b3-f1575c6f5489 req-3cc76e8e-9a2c-4155-8437-7c69e75aeaf2 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:58.385262 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-8a2c608a-3a9d-4644-92b3-f1575c6f5489 req-3cc76e8e-9a2c-4155-8437-7c69e75aeaf2 service nova] Creating event network-vif-plugged:081d646d-03f3-4a3f-8409-9106695a46bb for instance 4602867b-88ed-43b1-9ee6-bf661e2d7d99 on nch1-master-1 Jul 16 21:13:58.398238 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-8a2c608a-3a9d-4644-92b3-f1575c6f5489 req-3cc76e8e-9a2c-4155-8437-7c69e75aeaf2 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.065880 Jul 16 21:13:58.400341 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 672/1429] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:13:58 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 68 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:13:58.413081 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-4893e6ab-a957-4288-a9ce-44d2a005566c tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:13:58.414345 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-4893e6ab-a957-4288-a9ce-44d2a005566c tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] [instance: 69cabe57-da71-4a02-b4ee-c0d84b3d32a2] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:13:58.616048 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-4893e6ab-a957-4288-a9ce-44d2a005566c tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '53af488f3b0242a797f539284b78d59e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bb1ede0d0b6941cc97278e6c617f9b01', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:13:58.643241 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-4893e6ab-a957-4288-a9ce-44d2a005566c tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] 10.222.0.16 "GET /compute/v2.1/servers/69cabe57-da71-4a02-b4ee-c0d84b3d32a2" status: 200 len: 1212 microversion: 2.1 time: 0.241160 Jul 16 21:13:58.654674 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 673/1430] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:13:58 2022] GET /compute/v2.1/servers/69cabe57-da71-4a02-b4ee-c0d84b3d32a2 => generated 1212 bytes in 253 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:13:58.659390 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-2d29f5ec-01aa-4bd4-bcc7-2a402aedb45f tempest-MigrationsAdminTest-399208122 tempest-MigrationsAdminTest-399208122-project-admin] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:13:58.659921 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-2d29f5ec-01aa-4bd4-bcc7-2a402aedb45f tempest-MigrationsAdminTest-399208122 tempest-MigrationsAdminTest-399208122-project-admin] [instance: 126f561c-163f-4f6d-8be1-98a2e29b0909] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:13:58.679902 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2d29f5ec-01aa-4bd4-bcc7-2a402aedb45f tempest-MigrationsAdminTest-399208122 tempest-MigrationsAdminTest-399208122-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:58.680364 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2d29f5ec-01aa-4bd4-bcc7-2a402aedb45f tempest-MigrationsAdminTest-399208122 tempest-MigrationsAdminTest-399208122-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:58.681162 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2d29f5ec-01aa-4bd4-bcc7-2a402aedb45f tempest-MigrationsAdminTest-399208122 tempest-MigrationsAdminTest-399208122-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:58.681690 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2d29f5ec-01aa-4bd4-bcc7-2a402aedb45f tempest-MigrationsAdminTest-399208122 tempest-MigrationsAdminTest-399208122-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:58.771865 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-0fbbb33b-60c7-4ff7-9546-57df29dd3163 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:13:58.781811 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-0fbbb33b-60c7-4ff7-9546-57df29dd3163 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:13:58.786241 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-0fbbb33b-60c7-4ff7-9546-57df29dd3163 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:13:58.791130 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-0fbbb33b-60c7-4ff7-9546-57df29dd3163 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] 10.222.0.16 "GET /compute/v2.1/servers/3c5ab09e-0a97-4cfc-b249-a11ab221b5d2" status: 200 len: 1809 microversion: 2.25 time: 0.754434 Jul 16 21:13:58.792477 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 758/1431] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:13:58 2022] GET /compute/v2.1/servers/3c5ab09e-0a97-4cfc-b249-a11ab221b5d2 => generated 1809 bytes in 758 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:13:58.805131 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-9b12fff6-6943-45cc-ad29-583dfdd7c747 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:13:58.806320 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-9b12fff6-6943-45cc-ad29-583dfdd7c747 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] [instance: dada2ea0-2ef4-4a30-8e2d-40177d0bafce] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:13:58.830550 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9b12fff6-6943-45cc-ad29-583dfdd7c747 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:58.830550 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9b12fff6-6943-45cc-ad29-583dfdd7c747 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:58.830550 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9b12fff6-6943-45cc-ad29-583dfdd7c747 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:58.831537 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9b12fff6-6943-45cc-ad29-583dfdd7c747 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:59.151128 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-2d29f5ec-01aa-4bd4-bcc7-2a402aedb45f tempest-MigrationsAdminTest-399208122 tempest-MigrationsAdminTest-399208122-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/126f561c-163f-4f6d-8be1-98a2e29b0909" status: 200 len: 1614 microversion: 2.1 time: 0.495500 Jul 16 21:13:59.152076 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 674/1432] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:13:58 2022] GET /compute/v2.1/servers/126f561c-163f-4f6d-8be1-98a2e29b0909 => generated 1614 bytes in 497 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:13:59.152980 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-9b12fff6-6943-45cc-ad29-583dfdd7c747 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2eba2ac21c344394a7d3f605775b8713', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c2095f96f8d45bb9e62c517678c3dcd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:13:59.176508 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-9b12fff6-6943-45cc-ad29-583dfdd7c747 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] 10.222.0.16 "GET /compute/v2.1/servers/dada2ea0-2ef4-4a30-8e2d-40177d0bafce" status: 200 len: 1331 microversion: 2.1 time: 0.381139 Jul 16 21:13:59.177303 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 759/1433] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:13:58 2022] GET /compute/v2.1/servers/dada2ea0-2ef4-4a30-8e2d-40177d0bafce => generated 1331 bytes in 382 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:13:59.177603 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-c9c29dd2-2aa1-4ae1-b24f-664fa1fd2dff tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:13:59.178683 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-c9c29dd2-2aa1-4ae1-b24f-664fa1fd2dff tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] [instance: e66baae9-d488-46a4-8347-623030fc8b69] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:13:59.200894 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-26e9bb6b-5557-455f-9bd4-db361f6d875e tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:13:59.200894 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-26e9bb6b-5557-455f-9bd4-db361f6d875e tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] [instance: 4602867b-88ed-43b1-9ee6-bf661e2d7d99] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:13:59.202822 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c9c29dd2-2aa1-4ae1-b24f-664fa1fd2dff tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:59.205150 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c9c29dd2-2aa1-4ae1-b24f-664fa1fd2dff tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:59.207336 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c9c29dd2-2aa1-4ae1-b24f-664fa1fd2dff tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:59.209029 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c9c29dd2-2aa1-4ae1-b24f-664fa1fd2dff tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:59.217509 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-26e9bb6b-5557-455f-9bd4-db361f6d875e tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:59.227046 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:13:59.227444 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-26e9bb6b-5557-455f-9bd4-db361f6d875e tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:59.228407 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-26e9bb6b-5557-455f-9bd4-db361f6d875e tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:59.228866 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-26e9bb6b-5557-455f-9bd4-db361f6d875e tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:59.531504 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-26e9bb6b-5557-455f-9bd4-db361f6d875e tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c1aea87154224f1c81428ddef4438801', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45d9edd2fe854483a6f0d676efbd6622', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:13:59.535129 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-c9c29dd2-2aa1-4ae1-b24f-664fa1fd2dff tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4e9df82f95ac4779859972403869096a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4337575622d04a26942905c9dc044ba7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:13:59.543419 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-26e9bb6b-5557-455f-9bd4-db361f6d875e tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] 10.222.0.16 "GET /compute/v2.1/servers/4602867b-88ed-43b1-9ee6-bf661e2d7d99" status: 200 len: 1321 microversion: 2.1 time: 0.349520 Jul 16 21:13:59.553294 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-c9c29dd2-2aa1-4ae1-b24f-664fa1fd2dff tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] 10.222.0.16 "GET /compute/v2.1/servers/e66baae9-d488-46a4-8347-623030fc8b69" status: 200 len: 1324 microversion: 2.1 time: 0.391524 Jul 16 21:13:59.553294 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 675/1434] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:13:59 2022] GET /compute/v2.1/servers/e66baae9-d488-46a4-8347-623030fc8b69 => generated 1324 bytes in 393 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:13:59.560351 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-53ef9dee-1f31-4906-9ccb-c0d78ed233bf tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:13:59.560869 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-53ef9dee-1f31-4906-9ccb-c0d78ed233bf tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: 126f561c-163f-4f6d-8be1-98a2e29b0909] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:13:59.567887 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-53ef9dee-1f31-4906-9ccb-c0d78ed233bf tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:59.568535 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-53ef9dee-1f31-4906-9ccb-c0d78ed233bf tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:59.568766 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 760/1435] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:13:59 2022] GET /compute/v2.1/servers/4602867b-88ed-43b1-9ee6-bf661e2d7d99 => generated 1321 bytes in 391 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:13:59.569153 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-53ef9dee-1f31-4906-9ccb-c0d78ed233bf tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:59.569597 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-53ef9dee-1f31-4906-9ccb-c0d78ed233bf tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:59.601241 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-53ef9dee-1f31-4906-9ccb-c0d78ed233bf tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: 126f561c-163f-4f6d-8be1-98a2e29b0909] Going to try to terminate instance {{(pid=138681) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 16 21:13:59.661600 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-b4778ce3-130a-432f-8444-3de3477cb00b tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:13:59.664293 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-b4778ce3-130a-432f-8444-3de3477cb00b tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] [instance: 69cabe57-da71-4a02-b4ee-c0d84b3d32a2] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:13:59.671466 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b4778ce3-130a-432f-8444-3de3477cb00b tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:59.671984 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b4778ce3-130a-432f-8444-3de3477cb00b tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:59.672899 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b4778ce3-130a-432f-8444-3de3477cb00b tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:59.673252 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b4778ce3-130a-432f-8444-3de3477cb00b tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:59.772286 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-b4778ce3-130a-432f-8444-3de3477cb00b tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '53af488f3b0242a797f539284b78d59e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bb1ede0d0b6941cc97278e6c617f9b01', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:13:59.782546 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-b4778ce3-130a-432f-8444-3de3477cb00b tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] 10.222.0.16 "GET /compute/v2.1/servers/69cabe57-da71-4a02-b4ee-c0d84b3d32a2" status: 200 len: 1216 microversion: 2.1 time: 0.125564 Jul 16 21:13:59.783823 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 761/1436] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:13:59 2022] GET /compute/v2.1/servers/69cabe57-da71-4a02-b4ee-c0d84b3d32a2 => generated 1216 bytes in 128 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:13:59.810619 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-a248889d-5e8e-44a2-b203-a73778cf6af9 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:13:59.811271 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-a248889d-5e8e-44a2-b203-a73778cf6af9 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] [instance: 3c5ab09e-0a97-4cfc-b249-a11ab221b5d2] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:13:59.819354 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a248889d-5e8e-44a2-b203-a73778cf6af9 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:59.820862 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a248889d-5e8e-44a2-b203-a73778cf6af9 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:13:59.822387 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a248889d-5e8e-44a2-b203-a73778cf6af9 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:13:59.823111 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a248889d-5e8e-44a2-b203-a73778cf6af9 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:00.145171 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-53ef9dee-1f31-4906-9ccb-c0d78ed233bf tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "DELETE /compute/v2.1/servers/126f561c-163f-4f6d-8be1-98a2e29b0909" status: 204 len: 0 microversion: 2.1 time: 0.588521 Jul 16 21:14:00.146527 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 676/1437] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:13:59 2022] DELETE /compute/v2.1/servers/126f561c-163f-4f6d-8be1-98a2e29b0909 => generated 0 bytes in 600 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 16 21:14:00.157130 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-295f909f-f0e7-4aaa-b6d0-a56ebb7309fd req-30f2f561-cfc5-4c7a-affa-128af236ab98 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "dada2ea0-2ef4-4a30-8e2d-40177d0bafce", "tag": "b63bf933-a84e-4e18-98ed-0e772979bec5"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:14:00.192391 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-295f909f-f0e7-4aaa-b6d0-a56ebb7309fd req-30f2f561-cfc5-4c7a-affa-128af236ab98 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:00.195103 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-295f909f-f0e7-4aaa-b6d0-a56ebb7309fd req-30f2f561-cfc5-4c7a-affa-128af236ab98 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:00.209217 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-295f909f-f0e7-4aaa-b6d0-a56ebb7309fd req-30f2f561-cfc5-4c7a-affa-128af236ab98 service nova] Creating event network-changed:b63bf933-a84e-4e18-98ed-0e772979bec5 for instance dada2ea0-2ef4-4a30-8e2d-40177d0bafce on nch2-master-1 Jul 16 21:14:00.216785 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-295f909f-f0e7-4aaa-b6d0-a56ebb7309fd req-30f2f561-cfc5-4c7a-affa-128af236ab98 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.068781 Jul 16 21:14:00.218001 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 677/1438] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:14:00 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 71 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:14:00.223712 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-d996cb5b-0e1b-4945-a1bd-7314394f24c3 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:14:00.224441 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-d996cb5b-0e1b-4945-a1bd-7314394f24c3 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: 126f561c-163f-4f6d-8be1-98a2e29b0909] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:14:00.232916 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d996cb5b-0e1b-4945-a1bd-7314394f24c3 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:00.233380 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d996cb5b-0e1b-4945-a1bd-7314394f24c3 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:00.234184 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d996cb5b-0e1b-4945-a1bd-7314394f24c3 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:00.234626 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d996cb5b-0e1b-4945-a1bd-7314394f24c3 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:00.244992 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-a248889d-5e8e-44a2-b203-a73778cf6af9 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:14:00.261492 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-a248889d-5e8e-44a2-b203-a73778cf6af9 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:14:00.267824 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-a248889d-5e8e-44a2-b203-a73778cf6af9 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:14:00.270867 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-a248889d-5e8e-44a2-b203-a73778cf6af9 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] 10.222.0.16 "GET /compute/v2.1/servers/3c5ab09e-0a97-4cfc-b249-a11ab221b5d2" status: 200 len: 1809 microversion: 2.25 time: 0.464276 Jul 16 21:14:00.274345 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 762/1439] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:13:59 2022] GET /compute/v2.1/servers/3c5ab09e-0a97-4cfc-b249-a11ab221b5d2 => generated 1809 bytes in 468 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 16 21:14:00.286424 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-d00f707c-287a-4475-b581-58517a3caead tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:14:00.298934 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-d00f707c-287a-4475-b581-58517a3caead tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] [instance: dada2ea0-2ef4-4a30-8e2d-40177d0bafce] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:14:00.308965 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d00f707c-287a-4475-b581-58517a3caead tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:00.310045 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d00f707c-287a-4475-b581-58517a3caead tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:00.312882 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d00f707c-287a-4475-b581-58517a3caead tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:00.313363 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d00f707c-287a-4475-b581-58517a3caead tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:00.353603 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:14:00.592207 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-d996cb5b-0e1b-4945-a1bd-7314394f24c3 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b68dacc4b8463ba07fae6372131117', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55a19b7a4682457a9132f5fc80110712', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:14:00.624735 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-d996cb5b-0e1b-4945-a1bd-7314394f24c3 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "GET /compute/v2.1/servers/126f561c-163f-4f6d-8be1-98a2e29b0909" status: 200 len: 1469 microversion: 2.1 time: 0.399555 Jul 16 21:14:00.625750 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 678/1440] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:14:00 2022] GET /compute/v2.1/servers/126f561c-163f-4f6d-8be1-98a2e29b0909 => generated 1469 bytes in 407 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:14:00.635278 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-54259b1c-8e09-46d0-a6ea-a51d548c313a req-42063078-b0f1-4438-b796-f7c334addfaf service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "126f561c-163f-4f6d-8be1-98a2e29b0909", "name": "network-vif-plugged", "status": "completed", "tag": "3b739133-7af5-4df3-a04d-fff7e0a81a3d"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:14:00.635536 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-d00f707c-287a-4475-b581-58517a3caead tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2eba2ac21c344394a7d3f605775b8713', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c2095f96f8d45bb9e62c517678c3dcd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:14:00.645199 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-d00f707c-287a-4475-b581-58517a3caead tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] 10.222.0.16 "GET /compute/v2.1/servers/dada2ea0-2ef4-4a30-8e2d-40177d0bafce" status: 200 len: 1331 microversion: 2.1 time: 0.369427 Jul 16 21:14:00.651102 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 763/1441] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:14:00 2022] GET /compute/v2.1/servers/dada2ea0-2ef4-4a30-8e2d-40177d0bafce => generated 1331 bytes in 371 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:14:00.657421 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-54259b1c-8e09-46d0-a6ea-a51d548c313a req-42063078-b0f1-4438-b796-f7c334addfaf service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:00.657421 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-54259b1c-8e09-46d0-a6ea-a51d548c313a req-42063078-b0f1-4438-b796-f7c334addfaf service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:00.657660 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-32b664fc-1f9d-4d59-bd39-8e57edf21b3e tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:14:00.657660 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-32b664fc-1f9d-4d59-bd39-8e57edf21b3e tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] [instance: 4602867b-88ed-43b1-9ee6-bf661e2d7d99] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:14:00.669828 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-32b664fc-1f9d-4d59-bd39-8e57edf21b3e tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:00.670701 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-32b664fc-1f9d-4d59-bd39-8e57edf21b3e tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:00.672228 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-32b664fc-1f9d-4d59-bd39-8e57edf21b3e tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:00.672756 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-54259b1c-8e09-46d0-a6ea-a51d548c313a req-42063078-b0f1-4438-b796-f7c334addfaf service nova] Creating event network-vif-plugged:3b739133-7af5-4df3-a04d-fff7e0a81a3d for instance 126f561c-163f-4f6d-8be1-98a2e29b0909 on nch2-master-1 Jul 16 21:14:00.673532 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-32b664fc-1f9d-4d59-bd39-8e57edf21b3e tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:00.689262 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-54259b1c-8e09-46d0-a6ea-a51d548c313a req-42063078-b0f1-4438-b796-f7c334addfaf service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.059211 Jul 16 21:14:00.697178 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 679/1442] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:14:00 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 64 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:14:00.702397 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-81cb0e40-9eab-41ea-8b7f-9d412e9f7f5c tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:14:00.704418 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-81cb0e40-9eab-41ea-8b7f-9d412e9f7f5c tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] [instance: e66baae9-d488-46a4-8347-623030fc8b69] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:14:00.714573 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-81cb0e40-9eab-41ea-8b7f-9d412e9f7f5c tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:00.715463 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-81cb0e40-9eab-41ea-8b7f-9d412e9f7f5c tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:00.722778 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-81cb0e40-9eab-41ea-8b7f-9d412e9f7f5c tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:00.723426 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-81cb0e40-9eab-41ea-8b7f-9d412e9f7f5c tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:01.846280 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-32b664fc-1f9d-4d59-bd39-8e57edf21b3e tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c1aea87154224f1c81428ddef4438801', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45d9edd2fe854483a6f0d676efbd6622', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:14:01.854775 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-32b664fc-1f9d-4d59-bd39-8e57edf21b3e tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] 10.222.0.16 "GET /compute/v2.1/servers/4602867b-88ed-43b1-9ee6-bf661e2d7d99" status: 200 len: 1471 microversion: 2.1 time: 1.207329 Jul 16 21:14:01.860421 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 764/1443] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:14:00 2022] GET /compute/v2.1/servers/4602867b-88ed-43b1-9ee6-bf661e2d7d99 => generated 1471 bytes in 1213 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:14:01.874189 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-642e0788-f2b6-43d7-b01a-b5ebbed1deea tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:14:01.875312 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-642e0788-f2b6-43d7-b01a-b5ebbed1deea tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] [instance: 69cabe57-da71-4a02-b4ee-c0d84b3d32a2] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:14:01.882455 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-642e0788-f2b6-43d7-b01a-b5ebbed1deea tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:01.895800 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-642e0788-f2b6-43d7-b01a-b5ebbed1deea tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:01.896996 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-642e0788-f2b6-43d7-b01a-b5ebbed1deea tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:01.897579 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-642e0788-f2b6-43d7-b01a-b5ebbed1deea tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:01.906349 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-81cb0e40-9eab-41ea-8b7f-9d412e9f7f5c tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4e9df82f95ac4779859972403869096a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4337575622d04a26942905c9dc044ba7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:14:01.918187 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-81cb0e40-9eab-41ea-8b7f-9d412e9f7f5c tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] 10.222.0.16 "GET /compute/v2.1/servers/e66baae9-d488-46a4-8347-623030fc8b69" status: 200 len: 1324 microversion: 2.1 time: 1.226878 Jul 16 21:14:01.931329 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 680/1444] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:14:00 2022] GET /compute/v2.1/servers/e66baae9-d488-46a4-8347-623030fc8b69 => generated 1324 bytes in 1241 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:14:01.954354 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-db47cd72-7eb6-4588-8492-e0c487ac2ef7 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:14:01.955484 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-db47cd72-7eb6-4588-8492-e0c487ac2ef7 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] [instance: 3c5ab09e-0a97-4cfc-b249-a11ab221b5d2] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:14:01.973873 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-db47cd72-7eb6-4588-8492-e0c487ac2ef7 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:01.974642 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-db47cd72-7eb6-4588-8492-e0c487ac2ef7 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:01.975775 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-db47cd72-7eb6-4588-8492-e0c487ac2ef7 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:01.976627 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-db47cd72-7eb6-4588-8492-e0c487ac2ef7 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:02.169726 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-642e0788-f2b6-43d7-b01a-b5ebbed1deea tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '53af488f3b0242a797f539284b78d59e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bb1ede0d0b6941cc97278e6c617f9b01', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:14:02.182065 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-642e0788-f2b6-43d7-b01a-b5ebbed1deea tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] 10.222.0.16 "GET /compute/v2.1/servers/69cabe57-da71-4a02-b4ee-c0d84b3d32a2" status: 200 len: 1282 microversion: 2.1 time: 0.319878 Jul 16 21:14:02.183548 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 765/1445] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:14:01 2022] GET /compute/v2.1/servers/69cabe57-da71-4a02-b4ee-c0d84b3d32a2 => generated 1282 bytes in 322 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:14:02.188613 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-0a299b87-cda1-4b9b-b7cf-b211cb433eff req-87c1a45a-aff6-4358-9ef2-1b6d9c7e5027 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "e66baae9-d488-46a4-8347-623030fc8b69", "tag": "ed0a7ee7-b878-4680-b977-42bcc7c86cb8"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:14:02.202026 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-0a299b87-cda1-4b9b-b7cf-b211cb433eff req-87c1a45a-aff6-4358-9ef2-1b6d9c7e5027 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:02.205092 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-0a299b87-cda1-4b9b-b7cf-b211cb433eff req-87c1a45a-aff6-4358-9ef2-1b6d9c7e5027 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:02.228502 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-0a299b87-cda1-4b9b-b7cf-b211cb433eff req-87c1a45a-aff6-4358-9ef2-1b6d9c7e5027 service nova] Creating event network-changed:ed0a7ee7-b878-4680-b977-42bcc7c86cb8 for instance e66baae9-d488-46a4-8347-623030fc8b69 on nch2-master-1 Jul 16 21:14:02.237829 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-0a299b87-cda1-4b9b-b7cf-b211cb433eff req-87c1a45a-aff6-4358-9ef2-1b6d9c7e5027 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.052811 Jul 16 21:14:02.238956 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 766/1446] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:14:02 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:14:02.247552 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-feda4e43-8252-4ed6-b75d-9ae8ea6b9450 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:14:02.248267 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-feda4e43-8252-4ed6-b75d-9ae8ea6b9450 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: 126f561c-163f-4f6d-8be1-98a2e29b0909] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:14:02.268481 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-feda4e43-8252-4ed6-b75d-9ae8ea6b9450 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:02.270418 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-feda4e43-8252-4ed6-b75d-9ae8ea6b9450 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:02.273883 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-feda4e43-8252-4ed6-b75d-9ae8ea6b9450 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:02.275067 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-feda4e43-8252-4ed6-b75d-9ae8ea6b9450 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:02.517770 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-db47cd72-7eb6-4588-8492-e0c487ac2ef7 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:14:02.531192 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-db47cd72-7eb6-4588-8492-e0c487ac2ef7 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:14:02.533408 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-db47cd72-7eb6-4588-8492-e0c487ac2ef7 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:14:02.535109 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-db47cd72-7eb6-4588-8492-e0c487ac2ef7 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] 10.222.0.16 "GET /compute/v2.1/servers/3c5ab09e-0a97-4cfc-b249-a11ab221b5d2" status: 200 len: 1809 microversion: 2.25 time: 0.602638 Jul 16 21:14:02.536065 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 681/1447] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:14:01 2022] GET /compute/v2.1/servers/3c5ab09e-0a97-4cfc-b249-a11ab221b5d2 => generated 1809 bytes in 604 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:14:02.542300 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-a017211e-b025-4ec9-b53f-1e70480b75b4 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:14:02.543155 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-a017211e-b025-4ec9-b53f-1e70480b75b4 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] [instance: dada2ea0-2ef4-4a30-8e2d-40177d0bafce] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:14:02.567470 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a017211e-b025-4ec9-b53f-1e70480b75b4 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:02.568529 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a017211e-b025-4ec9-b53f-1e70480b75b4 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:02.570280 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a017211e-b025-4ec9-b53f-1e70480b75b4 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:02.570957 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a017211e-b025-4ec9-b53f-1e70480b75b4 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:02.805872 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-feda4e43-8252-4ed6-b75d-9ae8ea6b9450 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b68dacc4b8463ba07fae6372131117', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55a19b7a4682457a9132f5fc80110712', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:14:02.822086 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-feda4e43-8252-4ed6-b75d-9ae8ea6b9450 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "GET /compute/v2.1/servers/126f561c-163f-4f6d-8be1-98a2e29b0909" status: 200 len: 1469 microversion: 2.1 time: 0.581872 Jul 16 21:14:02.822954 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 767/1448] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:14:02 2022] GET /compute/v2.1/servers/126f561c-163f-4f6d-8be1-98a2e29b0909 => generated 1469 bytes in 583 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:14:02.922036 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-213292de-25af-4048-8f0e-68effd19a0fb tempest-ServersAdminNegativeTestJSON-1884193353 tempest-ServersAdminNegativeTestJSON-1884193353-project-admin] Action: 'action', calling method: >, body: {"migrate": {}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:14:02.922645 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-213292de-25af-4048-8f0e-68effd19a0fb tempest-ServersAdminNegativeTestJSON-1884193353 tempest-ServersAdminNegativeTestJSON-1884193353-project-admin] [instance: bf6beb8d-9005-471d-a651-765a0fa9f581] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:14:02.929931 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-213292de-25af-4048-8f0e-68effd19a0fb tempest-ServersAdminNegativeTestJSON-1884193353 tempest-ServersAdminNegativeTestJSON-1884193353-project-admin] HTTP exception thrown: Instance bf6beb8d-9005-471d-a651-765a0fa9f581 could not be found. Jul 16 21:14:02.932615 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-213292de-25af-4048-8f0e-68effd19a0fb tempest-ServersAdminNegativeTestJSON-1884193353 tempest-ServersAdminNegativeTestJSON-1884193353-project-admin] Request method failure captured: Jul 16 21:14:02.932615 ncdv-master-1 devstack@n-api.service[138680]: request: POST /compute/v2.1/servers/bf6beb8d-9005-471d-a651-765a0fa9f581/action HTTP/1.1 Jul 16 21:14:02.932615 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:14:02.932615 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:14:02.932615 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:14:02.932615 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 15 Jul 16 21:14:02.932615 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:14:02.932615 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:14:02.932615 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:14:02.932615 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:14:02.932615 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0yn7QG_tnLYMptx-bTEUJY8ylzBGFOwnadoDojC2tCzwJeRalDhG_GURBfSS7Jpd1amigWw5Nbi2c_3Jq_jqZUD7WjqZe8pZKa1SRAQVw-5Rq-fp6JQvZsa6miiKLCZoWSEIKgl6wCp7cEj6e1JS1FO8OqdXGKZOwT9NqIcGygKtteg Jul 16 21:14:02.932615 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:14:02.932615 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:14:02.932615 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:14:02.932615 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:14:02.932615 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:14:02.932615 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:14:02.932615 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: 38594b6bc3354178a22f0abb698e98a6 Jul 16 21:14:02.932615 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-ServersAdminNegativeTestJSON-1884193353 Jul 16 21:14:02.932615 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,admin,reader Jul 16 21:14:02.932615 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,admin,reader Jul 16 21:14:02.934163 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/38594b6bc3354178a22f0abb698e98a6"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/38594b6bc3354178a22f0abb698e98a6"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_38594b6bc3354178a22f0abb698e98a6"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/38594b6bc3354178a22f0abb698e98a6"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:14:02.934163 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-ServersAdminNegativeTestJSON-1884193353 Jul 16 21:14:02.934163 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: 38594b6bc3354178a22f0abb698e98a6 Jul 16 21:14:02.934163 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-ServersAdminNegativeTestJSON-1884193353 Jul 16 21:14:02.934163 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-ServersAdminNegativeTestJSON-1884193353-project-admin Jul 16 21:14:02.934163 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:14:02.934163 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:14:02.934163 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: 27aeaef0a671464493a8cac846186af7 Jul 16 21:14:02.934163 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-ServersAdminNegativeTestJSON-1884193353-project-admin Jul 16 21:14:02.934163 ncdv-master-1 devstack@n-api.service[138680]: Jul 16 21:14:02.934163 ncdv-master-1 devstack@n-api.service[138680]: {"migrate": {}} Jul 16 21:14:02.934163 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 21:14:02.934163 ncdv-master-1 devstack@n-api.service[138680]: exception: Instance bf6beb8d-9005-471d-a651-765a0fa9f581 could not be found. Jul 16 21:14:02.935118 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': 'bf6beb8d-9005-471d-a651-765a0fa9f581', 'body': {'migrate': {}}} Jul 16 21:14:02.935118 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:14:02.935118 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:14:02.935118 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:14:02.935118 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:14:02.935118 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:14:02.935118 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:14:02.935118 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2898, in _get_instance Jul 16 21:14:02.935118 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=instance_uuid) Jul 16 21:14:02.935118 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance bf6beb8d-9005-471d-a651-765a0fa9f581 could not be found. Jul 16 21:14:02.935118 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:14:02.935118 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:14:02.935118 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:14:02.935118 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:14:02.935118 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:14:02.935118 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:14:02.935118 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:14:02.935118 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:14:02.935118 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:14:02.935118 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:14:02.936553 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 16 21:14:02.936553 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 16 21:14:02.936553 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/migrate_server.py", line 47, in _migrate Jul 16 21:14:02.936553 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, id, Jul 16 21:14:02.936553 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:14:02.936553 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:14:02.936553 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance bf6beb8d-9005-471d-a651-765a0fa9f581 could not be found. Jul 16 21:14:02.936553 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:14:02.936553 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:14:02.936553 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:14:02.936553 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:14:02.936553 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:14:02.936553 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:14:02.936553 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:14:02.936553 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:14:02.936553 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance bf6beb8d-9005-471d-a651-765a0fa9f581 could not be found. Jul 16 21:14:02.936553 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:14:02.937747 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-213292de-25af-4048-8f0e-68effd19a0fb tempest-ServersAdminNegativeTestJSON-1884193353 tempest-ServersAdminNegativeTestJSON-1884193353-project-admin] Returning 404 to user: Instance bf6beb8d-9005-471d-a651-765a0fa9f581 could not be found. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:14:02.937747 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-213292de-25af-4048-8f0e-68effd19a0fb tempest-ServersAdminNegativeTestJSON-1884193353 tempest-ServersAdminNegativeTestJSON-1884193353-project-admin] 10.222.0.16 "POST /compute/v2.1/servers/bf6beb8d-9005-471d-a651-765a0fa9f581/action" status: 404 len: 111 microversion: 2.1 time: 0.109121 Jul 16 21:14:02.937747 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 768/1449] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 21:14:02 2022] POST /compute/v2.1/servers/bf6beb8d-9005-471d-a651-765a0fa9f581/action => generated 111 bytes in 110 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:14:02.938919 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-fe45707a-b334-4e2c-8466-4b58456ea3c2 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:14:02.943639 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-fe45707a-b334-4e2c-8466-4b58456ea3c2 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] [instance: e66baae9-d488-46a4-8347-623030fc8b69] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:14:02.950630 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-fe45707a-b334-4e2c-8466-4b58456ea3c2 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:02.951116 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-fe45707a-b334-4e2c-8466-4b58456ea3c2 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:02.951965 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-fe45707a-b334-4e2c-8466-4b58456ea3c2 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:02.952512 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-fe45707a-b334-4e2c-8466-4b58456ea3c2 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:03.121894 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-a017211e-b025-4ec9-b53f-1e70480b75b4 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2eba2ac21c344394a7d3f605775b8713', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c2095f96f8d45bb9e62c517678c3dcd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:14:03.129318 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-a017211e-b025-4ec9-b53f-1e70480b75b4 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] 10.222.0.16 "GET /compute/v2.1/servers/dada2ea0-2ef4-4a30-8e2d-40177d0bafce" status: 200 len: 1331 microversion: 2.1 time: 0.590422 Jul 16 21:14:03.130552 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 682/1450] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:14:02 2022] GET /compute/v2.1/servers/dada2ea0-2ef4-4a30-8e2d-40177d0bafce => generated 1331 bytes in 593 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:14:03.138325 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-c766f625-f884-4ecd-aee1-7a1b9d706b68 tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServersAdminNegativeTestJSON-server-1954229250", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452"}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:14:03.213778 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.network.neutron [None req-c766f625-f884-4ecd-aee1-7a1b9d706b68 tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] validate_networks() for None {{(pid=138681) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 16 21:14:03.307478 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-fe45707a-b334-4e2c-8466-4b58456ea3c2 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4e9df82f95ac4779859972403869096a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4337575622d04a26942905c9dc044ba7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:14:03.315797 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-fe45707a-b334-4e2c-8466-4b58456ea3c2 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] 10.222.0.16 "GET /compute/v2.1/servers/e66baae9-d488-46a4-8347-623030fc8b69" status: 200 len: 1324 microversion: 2.1 time: 0.380867 Jul 16 21:14:03.316378 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 769/1451] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:14:02 2022] GET /compute/v2.1/servers/e66baae9-d488-46a4-8347-623030fc8b69 => generated 1324 bytes in 382 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:14:03.321577 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-1d460de5-50e2-4279-81fa-bcdfc733a2af tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:14:03.322335 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-1d460de5-50e2-4279-81fa-bcdfc733a2af tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] [instance: 69cabe57-da71-4a02-b4ee-c0d84b3d32a2] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:14:03.330431 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1d460de5-50e2-4279-81fa-bcdfc733a2af tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:03.330887 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1d460de5-50e2-4279-81fa-bcdfc733a2af tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:03.336847 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1d460de5-50e2-4279-81fa-bcdfc733a2af tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:03.337638 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1d460de5-50e2-4279-81fa-bcdfc733a2af tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:03.461083 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-1d460de5-50e2-4279-81fa-bcdfc733a2af tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '53af488f3b0242a797f539284b78d59e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bb1ede0d0b6941cc97278e6c617f9b01', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:14:03.468753 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-1d460de5-50e2-4279-81fa-bcdfc733a2af tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] 10.222.0.16 "GET /compute/v2.1/servers/69cabe57-da71-4a02-b4ee-c0d84b3d32a2" status: 200 len: 1270 microversion: 2.1 time: 0.151362 Jul 16 21:14:03.469336 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 770/1452] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:14:03 2022] GET /compute/v2.1/servers/69cabe57-da71-4a02-b4ee-c0d84b3d32a2 => generated 1270 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:14:03.475437 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-d34d1e4b-a6c1-413a-8237-ef222758726a req-f08efae0-e43d-4552-b142-c433791afc47 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "dada2ea0-2ef4-4a30-8e2d-40177d0bafce", "name": "network-vif-plugged", "status": "completed", "tag": "b63bf933-a84e-4e18-98ed-0e772979bec5"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:14:03.485467 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-d34d1e4b-a6c1-413a-8237-ef222758726a req-f08efae0-e43d-4552-b142-c433791afc47 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:03.486187 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-d34d1e4b-a6c1-413a-8237-ef222758726a req-f08efae0-e43d-4552-b142-c433791afc47 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:03.505546 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-d34d1e4b-a6c1-413a-8237-ef222758726a req-f08efae0-e43d-4552-b142-c433791afc47 service nova] Creating event network-vif-plugged:b63bf933-a84e-4e18-98ed-0e772979bec5 for instance dada2ea0-2ef4-4a30-8e2d-40177d0bafce on nch2-master-1 Jul 16 21:14:03.511501 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-d34d1e4b-a6c1-413a-8237-ef222758726a req-f08efae0-e43d-4552-b142-c433791afc47 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.040761 Jul 16 21:14:03.512632 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 771/1453] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:14:03 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:14:03.565783 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-2937f932-9c01-4917-8d46-a478de6e3d6e tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:14:03.566617 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-2937f932-9c01-4917-8d46-a478de6e3d6e tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] [instance: 3c5ab09e-0a97-4cfc-b249-a11ab221b5d2] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:14:03.590774 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2937f932-9c01-4917-8d46-a478de6e3d6e tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:03.591519 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2937f932-9c01-4917-8d46-a478de6e3d6e tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:03.593285 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2937f932-9c01-4917-8d46-a478de6e3d6e tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:03.594903 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2937f932-9c01-4917-8d46-a478de6e3d6e tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:03.848879 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-c766f625-f884-4ecd-aee1-7a1b9d706b68 tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Flavor limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 16 21:14:03.849473 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-c766f625-f884-4ecd-aee1-7a1b9d706b68 tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Image limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 16 21:14:03.851326 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-c766f625-f884-4ecd-aee1-7a1b9d706b68 tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Flavor pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 16 21:14:03.851326 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-c766f625-f884-4ecd-aee1-7a1b9d706b68 tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Image pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 16 21:14:03.851846 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-c766f625-f884-4ecd-aee1-7a1b9d706b68 tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 16 21:14:03.864365 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c766f625-f884-4ecd-aee1-7a1b9d706b68 tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:03.865343 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c766f625-f884-4ecd-aee1-7a1b9d706b68 tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:03.867126 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c766f625-f884-4ecd-aee1-7a1b9d706b68 tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:03.867835 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c766f625-f884-4ecd-aee1-7a1b9d706b68 tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:03.915653 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-c766f625-f884-4ecd-aee1-7a1b9d706b68 tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Getting quotas for project 45d9edd2fe854483a6f0d676efbd6622. Resources: {'ram', 'instances', 'cores'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:14:03.920722 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-2937f932-9c01-4917-8d46-a478de6e3d6e tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:14:03.922634 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-c766f625-f884-4ecd-aee1-7a1b9d706b68 tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Getting quotas for user c1aea87154224f1c81428ddef4438801 and project 45d9edd2fe854483a6f0d676efbd6622. Resources: {'ram', 'instances', 'cores'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:14:03.927030 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-2937f932-9c01-4917-8d46-a478de6e3d6e tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:14:03.929875 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-2937f932-9c01-4917-8d46-a478de6e3d6e tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:14:03.931428 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-2937f932-9c01-4917-8d46-a478de6e3d6e tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] 10.222.0.16 "GET /compute/v2.1/servers/3c5ab09e-0a97-4cfc-b249-a11ab221b5d2" status: 200 len: 1809 microversion: 2.25 time: 0.380997 Jul 16 21:14:03.932490 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 772/1454] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:14:03 2022] GET /compute/v2.1/servers/3c5ab09e-0a97-4cfc-b249-a11ab221b5d2 => generated 1809 bytes in 383 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:14:03.937419 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-62189678-6402-4801-917f-62657e87c03d req-33c7ac83-9e6a-4590-9642-1bfd1d9b51d5 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "126f561c-163f-4f6d-8be1-98a2e29b0909", "name": "network-vif-deleted", "tag": "3b739133-7af5-4df3-a04d-fff7e0a81a3d"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:14:03.946421 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-62189678-6402-4801-917f-62657e87c03d req-33c7ac83-9e6a-4590-9642-1bfd1d9b51d5 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:03.949421 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-62189678-6402-4801-917f-62657e87c03d req-33c7ac83-9e6a-4590-9642-1bfd1d9b51d5 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:03.953779 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-c766f625-f884-4ecd-aee1-7a1b9d706b68 tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Going to run 1 instances... {{(pid=138681) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 16 21:14:03.957809 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-c766f625-f884-4ecd-aee1-7a1b9d706b68 tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] [instance: 76ee9d21-1f17-4b64-8f95-95a33e70e907] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138681) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 16 21:14:03.964555 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-62189678-6402-4801-917f-62657e87c03d req-33c7ac83-9e6a-4590-9642-1bfd1d9b51d5 service nova] Creating event network-vif-deleted:3b739133-7af5-4df3-a04d-fff7e0a81a3d for instance 126f561c-163f-4f6d-8be1-98a2e29b0909 on nch2-master-1 Jul 16 21:14:03.975029 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-62189678-6402-4801-917f-62657e87c03d req-33c7ac83-9e6a-4590-9642-1bfd1d9b51d5 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.040822 Jul 16 21:14:03.977993 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 773/1455] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:14:03 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:14:03.986345 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-c3e6396e-9964-4e72-b644-21943f7f1c9a tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:14:03.987474 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-c3e6396e-9964-4e72-b644-21943f7f1c9a tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: 126f561c-163f-4f6d-8be1-98a2e29b0909] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:14:03.996731 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c3e6396e-9964-4e72-b644-21943f7f1c9a tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:03.997369 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c3e6396e-9964-4e72-b644-21943f7f1c9a tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:03.998755 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c3e6396e-9964-4e72-b644-21943f7f1c9a tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:03.999978 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c3e6396e-9964-4e72-b644-21943f7f1c9a tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:04.157890 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-c3e6396e-9964-4e72-b644-21943f7f1c9a tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b68dacc4b8463ba07fae6372131117', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55a19b7a4682457a9132f5fc80110712', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:14:04.170205 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-c3e6396e-9964-4e72-b644-21943f7f1c9a tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "GET /compute/v2.1/servers/126f561c-163f-4f6d-8be1-98a2e29b0909" status: 200 len: 1427 microversion: 2.1 time: 0.192462 Jul 16 21:14:04.170825 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 774/1456] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:14:03 2022] GET /compute/v2.1/servers/126f561c-163f-4f6d-8be1-98a2e29b0909 => generated 1427 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:14:04.176615 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-2418921f-5a98-4271-bc39-4498034cc96c tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:14:04.177253 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-2418921f-5a98-4271-bc39-4498034cc96c tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] [instance: dada2ea0-2ef4-4a30-8e2d-40177d0bafce] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:14:04.185662 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2418921f-5a98-4271-bc39-4498034cc96c tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:04.186432 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2418921f-5a98-4271-bc39-4498034cc96c tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:04.187284 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2418921f-5a98-4271-bc39-4498034cc96c tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:04.188077 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2418921f-5a98-4271-bc39-4498034cc96c tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:04.285449 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-c766f625-f884-4ecd-aee1-7a1b9d706b68 tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 1.153426 Jul 16 21:14:04.286591 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 683/1457] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 21:14:03 2022] POST /compute/v2.1/servers => generated 380 bytes in 1155 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 16 21:14:04.299640 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-b7378883-abf4-4f0b-9c3d-34e2e38b1183 tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:14:04.301558 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-b7378883-abf4-4f0b-9c3d-34e2e38b1183 tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] [instance: 76ee9d21-1f17-4b64-8f95-95a33e70e907] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:14:04.436405 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-b7378883-abf4-4f0b-9c3d-34e2e38b1183 tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c1aea87154224f1c81428ddef4438801', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45d9edd2fe854483a6f0d676efbd6622', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:14:04.450725 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-b7378883-abf4-4f0b-9c3d-34e2e38b1183 tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] 10.222.0.16 "GET /compute/v2.1/servers/76ee9d21-1f17-4b64-8f95-95a33e70e907" status: 200 len: 1222 microversion: 2.1 time: 0.155424 Jul 16 21:14:04.456207 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 684/1458] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:14:04 2022] GET /compute/v2.1/servers/76ee9d21-1f17-4b64-8f95-95a33e70e907 => generated 1222 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:14:04.464755 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-a41ffbdc-4678-41b7-a3e1-953e7fdb9a29 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:14:04.464755 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-a41ffbdc-4678-41b7-a3e1-953e7fdb9a29 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] [instance: e66baae9-d488-46a4-8347-623030fc8b69] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:14:04.474723 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a41ffbdc-4678-41b7-a3e1-953e7fdb9a29 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:04.475225 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a41ffbdc-4678-41b7-a3e1-953e7fdb9a29 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:04.476056 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a41ffbdc-4678-41b7-a3e1-953e7fdb9a29 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:04.476497 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a41ffbdc-4678-41b7-a3e1-953e7fdb9a29 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:04.681547 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-2418921f-5a98-4271-bc39-4498034cc96c tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2eba2ac21c344394a7d3f605775b8713', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c2095f96f8d45bb9e62c517678c3dcd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:14:04.690242 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-2418921f-5a98-4271-bc39-4498034cc96c tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] 10.222.0.16 "GET /compute/v2.1/servers/dada2ea0-2ef4-4a30-8e2d-40177d0bafce" status: 200 len: 1331 microversion: 2.1 time: 0.517234 Jul 16 21:14:04.698502 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 775/1459] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:14:04 2022] GET /compute/v2.1/servers/dada2ea0-2ef4-4a30-8e2d-40177d0bafce => generated 1331 bytes in 527 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:14:04.703326 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-794197ab-48fa-40fa-821c-a00e08107517 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:14:04.703951 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-794197ab-48fa-40fa-821c-a00e08107517 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] [instance: 69cabe57-da71-4a02-b4ee-c0d84b3d32a2] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:14:04.718930 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-794197ab-48fa-40fa-821c-a00e08107517 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:04.719498 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-794197ab-48fa-40fa-821c-a00e08107517 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:04.720316 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-794197ab-48fa-40fa-821c-a00e08107517 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:04.720764 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-794197ab-48fa-40fa-821c-a00e08107517 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:04.818970 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-a41ffbdc-4678-41b7-a3e1-953e7fdb9a29 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4e9df82f95ac4779859972403869096a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4337575622d04a26942905c9dc044ba7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:14:04.833677 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-a41ffbdc-4678-41b7-a3e1-953e7fdb9a29 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] 10.222.0.16 "GET /compute/v2.1/servers/e66baae9-d488-46a4-8347-623030fc8b69" status: 200 len: 1324 microversion: 2.1 time: 0.376619 Jul 16 21:14:04.834852 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 685/1460] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:14:04 2022] GET /compute/v2.1/servers/e66baae9-d488-46a4-8347-623030fc8b69 => generated 1324 bytes in 378 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:14:04.927137 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-794197ab-48fa-40fa-821c-a00e08107517 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '53af488f3b0242a797f539284b78d59e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bb1ede0d0b6941cc97278e6c617f9b01', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:14:04.935833 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-794197ab-48fa-40fa-821c-a00e08107517 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] 10.222.0.16 "GET /compute/v2.1/servers/69cabe57-da71-4a02-b4ee-c0d84b3d32a2" status: 200 len: 1312 microversion: 2.1 time: 0.236192 Jul 16 21:14:04.937316 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 776/1461] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:14:04 2022] GET /compute/v2.1/servers/69cabe57-da71-4a02-b4ee-c0d84b3d32a2 => generated 1312 bytes in 238 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:14:04.952340 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-5049ad9a-8748-4838-8e4b-0dd3dcac41ac tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:14:04.952882 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-5049ad9a-8748-4838-8e4b-0dd3dcac41ac tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] [instance: 3c5ab09e-0a97-4cfc-b249-a11ab221b5d2] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:14:04.959863 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5049ad9a-8748-4838-8e4b-0dd3dcac41ac tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:04.965578 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5049ad9a-8748-4838-8e4b-0dd3dcac41ac tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:04.965578 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5049ad9a-8748-4838-8e4b-0dd3dcac41ac tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:04.965578 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5049ad9a-8748-4838-8e4b-0dd3dcac41ac tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:05.187928 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-67896b24-8b54-437f-9305-ecc280ac79e6 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:14:05.188672 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-67896b24-8b54-437f-9305-ecc280ac79e6 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: 126f561c-163f-4f6d-8be1-98a2e29b0909] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:14:05.196555 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-67896b24-8b54-437f-9305-ecc280ac79e6 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:05.197023 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-67896b24-8b54-437f-9305-ecc280ac79e6 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:05.197823 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-67896b24-8b54-437f-9305-ecc280ac79e6 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:05.198268 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-67896b24-8b54-437f-9305-ecc280ac79e6 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:05.236109 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-67896b24-8b54-437f-9305-ecc280ac79e6 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:05.236583 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-67896b24-8b54-437f-9305-ecc280ac79e6 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:05.237062 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.objects.instance [None req-67896b24-8b54-437f-9305-ecc280ac79e6 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lazy-loading 'fault' on Instance uuid 126f561c-163f-4f6d-8be1-98a2e29b0909 {{(pid=138680) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:14:05.511350 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-67896b24-8b54-437f-9305-ecc280ac79e6 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b68dacc4b8463ba07fae6372131117', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55a19b7a4682457a9132f5fc80110712', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:14:05.523304 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-67896b24-8b54-437f-9305-ecc280ac79e6 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "GET /compute/v2.1/servers/126f561c-163f-4f6d-8be1-98a2e29b0909" status: 200 len: 1303 microversion: 2.1 time: 0.338794 Jul 16 21:14:05.525348 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 777/1462] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:14:05 2022] GET /compute/v2.1/servers/126f561c-163f-4f6d-8be1-98a2e29b0909 => generated 1303 bytes in 342 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:14:05.540983 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-4429e297-a44d-419a-8754-d8851344c218 tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:14:05.541609 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-4429e297-a44d-419a-8754-d8851344c218 tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] [instance: 76ee9d21-1f17-4b64-8f95-95a33e70e907] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:14:05.673251 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-5049ad9a-8748-4838-8e4b-0dd3dcac41ac tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:14:05.701983 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-5049ad9a-8748-4838-8e4b-0dd3dcac41ac tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:14:05.703761 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-5049ad9a-8748-4838-8e4b-0dd3dcac41ac tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:14:05.705059 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-5049ad9a-8748-4838-8e4b-0dd3dcac41ac tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] 10.222.0.16 "GET /compute/v2.1/servers/3c5ab09e-0a97-4cfc-b249-a11ab221b5d2" status: 200 len: 1809 microversion: 2.25 time: 0.756863 Jul 16 21:14:05.707280 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 686/1463] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:14:04 2022] GET /compute/v2.1/servers/3c5ab09e-0a97-4cfc-b249-a11ab221b5d2 => generated 1809 bytes in 760 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:14:05.718639 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-4429e297-a44d-419a-8754-d8851344c218 tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c1aea87154224f1c81428ddef4438801', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45d9edd2fe854483a6f0d676efbd6622', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:14:05.729170 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-3183bc24-921b-4428-91d9-d9f149524713 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:14:05.729805 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-3183bc24-921b-4428-91d9-d9f149524713 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] [instance: dada2ea0-2ef4-4a30-8e2d-40177d0bafce] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:14:05.731774 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-4429e297-a44d-419a-8754-d8851344c218 tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] 10.222.0.16 "GET /compute/v2.1/servers/76ee9d21-1f17-4b64-8f95-95a33e70e907" status: 200 len: 1222 microversion: 2.1 time: 0.190640 Jul 16 21:14:05.743634 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 778/1464] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:14:05 2022] GET /compute/v2.1/servers/76ee9d21-1f17-4b64-8f95-95a33e70e907 => generated 1222 bytes in 208 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:14:05.745008 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3183bc24-921b-4428-91d9-d9f149524713 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:05.745663 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3183bc24-921b-4428-91d9-d9f149524713 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:05.746729 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3183bc24-921b-4428-91d9-d9f149524713 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:05.747730 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3183bc24-921b-4428-91d9-d9f149524713 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:05.852067 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-628bdf98-6ea6-409c-b477-e334d46d7147 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:14:05.852706 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-628bdf98-6ea6-409c-b477-e334d46d7147 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] [instance: e66baae9-d488-46a4-8347-623030fc8b69] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:14:05.859693 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-628bdf98-6ea6-409c-b477-e334d46d7147 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:05.860148 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-628bdf98-6ea6-409c-b477-e334d46d7147 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:05.861128 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-628bdf98-6ea6-409c-b477-e334d46d7147 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:05.862714 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-628bdf98-6ea6-409c-b477-e334d46d7147 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:06.248133 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-3183bc24-921b-4428-91d9-d9f149524713 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2eba2ac21c344394a7d3f605775b8713', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c2095f96f8d45bb9e62c517678c3dcd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:14:06.266168 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-3183bc24-921b-4428-91d9-d9f149524713 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] 10.222.0.16 "GET /compute/v2.1/servers/dada2ea0-2ef4-4a30-8e2d-40177d0bafce" status: 200 len: 1535 microversion: 2.1 time: 0.542706 Jul 16 21:14:06.267429 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 687/1465] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:14:05 2022] GET /compute/v2.1/servers/dada2ea0-2ef4-4a30-8e2d-40177d0bafce => generated 1535 bytes in 545 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:14:06.275069 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-a24f216e-9510-450d-8b5c-0b3a485ac2a4 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:14:06.275681 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-a24f216e-9510-450d-8b5c-0b3a485ac2a4 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] [instance: 69cabe57-da71-4a02-b4ee-c0d84b3d32a2] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:14:06.291474 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a24f216e-9510-450d-8b5c-0b3a485ac2a4 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:06.292442 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a24f216e-9510-450d-8b5c-0b3a485ac2a4 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:06.293609 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a24f216e-9510-450d-8b5c-0b3a485ac2a4 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:06.294260 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a24f216e-9510-450d-8b5c-0b3a485ac2a4 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:06.406459 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-628bdf98-6ea6-409c-b477-e334d46d7147 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4e9df82f95ac4779859972403869096a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4337575622d04a26942905c9dc044ba7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:14:06.417191 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-628bdf98-6ea6-409c-b477-e334d46d7147 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] 10.222.0.16 "GET /compute/v2.1/servers/e66baae9-d488-46a4-8347-623030fc8b69" status: 200 len: 1324 microversion: 2.1 time: 0.569553 Jul 16 21:14:06.428040 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 779/1466] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:14:05 2022] GET /compute/v2.1/servers/e66baae9-d488-46a4-8347-623030fc8b69 => generated 1324 bytes in 571 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:14:06.428040 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-5aa73b6f-0e13-4f1b-be23-fc3b2fc2f8a4 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] Action: 'create', calling method: .version_select at 0x7fdd9ce91c10>, body: {"pool": "f39f47fa-8d3d-4a2f-9486-cc28d1bbfe66"} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:14:06.855837 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-a24f216e-9510-450d-8b5c-0b3a485ac2a4 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '53af488f3b0242a797f539284b78d59e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bb1ede0d0b6941cc97278e6c617f9b01', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:14:06.869872 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-a24f216e-9510-450d-8b5c-0b3a485ac2a4 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] 10.222.0.16 "GET /compute/v2.1/servers/69cabe57-da71-4a02-b4ee-c0d84b3d32a2" status: 200 len: 1312 microversion: 2.1 time: 0.601097 Jul 16 21:14:06.892296 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 688/1467] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:14:06 2022] GET /compute/v2.1/servers/69cabe57-da71-4a02-b4ee-c0d84b3d32a2 => generated 1312 bytes in 624 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:14:06.897681 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-0b788c52-3099-487a-9120-597a0518639e tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:14:06.898583 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-0b788c52-3099-487a-9120-597a0518639e tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: 126f561c-163f-4f6d-8be1-98a2e29b0909] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:14:06.919601 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0b788c52-3099-487a-9120-597a0518639e tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:06.924091 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0b788c52-3099-487a-9120-597a0518639e tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:06.924960 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0b788c52-3099-487a-9120-597a0518639e tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:06.925474 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0b788c52-3099-487a-9120-597a0518639e tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:06.942074 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-0b788c52-3099-487a-9120-597a0518639e tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] HTTP exception thrown: Instance 126f561c-163f-4f6d-8be1-98a2e29b0909 could not be found. Jul 16 21:14:06.953230 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-0b788c52-3099-487a-9120-597a0518639e tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Request method failure captured: Jul 16 21:14:06.953230 ncdv-master-1 devstack@n-api.service[138681]: request: GET /compute/v2.1/servers/126f561c-163f-4f6d-8be1-98a2e29b0909 HTTP/1.1 Jul 16 21:14:06.953230 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:14:06.953230 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:14:06.953230 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:14:06.953230 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 21:14:06.953230 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:14:06.953230 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:14:06.953230 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:14:06.953230 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:14:06.953230 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0yjS0OwquTdxFvFwnqD6oXhLiVMC2H2T0PTTAPBs5CbYhBauxUQKqaIu70UDpOvDtiTqvXU4-VL9hTewWDu4Gh7O9lNbGbnPrucyL4QpaGXDfvsW3rocl0tQjubLkrzEuiaQ0dP8qaKxkSk27mrRw2WAWCg3PheG3BhIZr0IEKSfAB4 Jul 16 21:14:06.953230 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:14:06.953230 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:14:06.953230 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:14:06.953230 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:14:06.953230 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:14:06.953230 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:14:06.953230 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: 55a19b7a4682457a9132f5fc80110712 Jul 16 21:14:06.953230 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-MigrationsAdminTest-116332175 Jul 16 21:14:06.953230 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:14:06.953230 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:14:06.954926 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/55a19b7a4682457a9132f5fc80110712"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/55a19b7a4682457a9132f5fc80110712"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_55a19b7a4682457a9132f5fc80110712"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/55a19b7a4682457a9132f5fc80110712"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:14:06.954926 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-MigrationsAdminTest-116332175 Jul 16 21:14:06.954926 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: 55a19b7a4682457a9132f5fc80110712 Jul 16 21:14:06.954926 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-MigrationsAdminTest-116332175 Jul 16 21:14:06.954926 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-MigrationsAdminTest-116332175-project Jul 16 21:14:06.954926 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:14:06.954926 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:14:06.954926 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: a9b68dacc4b8463ba07fae6372131117 Jul 16 21:14:06.954926 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-MigrationsAdminTest-116332175-project Jul 16 21:14:06.954926 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:14:06.954926 ncdv-master-1 devstack@n-api.service[138681]: exception: Instance 126f561c-163f-4f6d-8be1-98a2e29b0909 could not be found. Jul 16 21:14:06.954926 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': '126f561c-163f-4f6d-8be1-98a2e29b0909'} Jul 16 21:14:06.956713 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:14:06.956713 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:14:06.956713 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:14:06.956713 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:14:06.956713 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:14:06.956713 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:14:06.956713 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:14:06.956713 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:14:06.956713 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:14:06.956713 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:14:06.956713 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 126f561c-163f-4f6d-8be1-98a2e29b0909 could not be found. Jul 16 21:14:06.956713 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:14:06.956713 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:14:06.956713 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:14:06.956713 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:14:06.956713 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:14:06.956713 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:14:06.956713 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:14:06.956713 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:14:06.956713 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:14:06.958289 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:14:06.958289 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:14:06.958289 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:14:06.958289 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:14:06.958289 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:14:06.958289 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:14:06.958289 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:14:06.958289 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 126f561c-163f-4f6d-8be1-98a2e29b0909 could not be found. Jul 16 21:14:06.958289 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:14:06.958289 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:14:06.958289 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:14:06.958289 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:14:06.958289 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:14:06.958289 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:14:06.958289 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:14:06.958289 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:14:06.958289 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 126f561c-163f-4f6d-8be1-98a2e29b0909 could not be found. Jul 16 21:14:06.958289 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:14:06.959878 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-0b788c52-3099-487a-9120-597a0518639e tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Returning 404 to user: Instance 126f561c-163f-4f6d-8be1-98a2e29b0909 could not be found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:14:06.959878 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-0b788c52-3099-487a-9120-597a0518639e tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "GET /compute/v2.1/servers/126f561c-163f-4f6d-8be1-98a2e29b0909" status: 404 len: 111 microversion: 2.1 time: 0.051882 Jul 16 21:14:06.959878 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 689/1468] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:14:06 2022] GET /compute/v2.1/servers/126f561c-163f-4f6d-8be1-98a2e29b0909 => generated 111 bytes in 53 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:14:06.959878 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-f56ab9f4-9e27-4615-bdc9-ce2774d1a04c tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:14:06.959878 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-f56ab9f4-9e27-4615-bdc9-ce2774d1a04c tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] [instance: 3c5ab09e-0a97-4cfc-b249-a11ab221b5d2] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:14:06.962789 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f56ab9f4-9e27-4615-bdc9-ce2774d1a04c tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:06.963250 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f56ab9f4-9e27-4615-bdc9-ce2774d1a04c tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:06.964040 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f56ab9f4-9e27-4615-bdc9-ce2774d1a04c tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:06.964503 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f56ab9f4-9e27-4615-bdc9-ce2774d1a04c tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:07.254503 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-f56ab9f4-9e27-4615-bdc9-ce2774d1a04c tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:14:07.271155 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-f56ab9f4-9e27-4615-bdc9-ce2774d1a04c tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:14:07.287618 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-f56ab9f4-9e27-4615-bdc9-ce2774d1a04c tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:14:07.291705 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-f56ab9f4-9e27-4615-bdc9-ce2774d1a04c tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] 10.222.0.16 "GET /compute/v2.1/servers/3c5ab09e-0a97-4cfc-b249-a11ab221b5d2" status: 200 len: 1809 microversion: 2.25 time: 0.343509 Jul 16 21:14:07.294720 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 690/1469] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:14:06 2022] GET /compute/v2.1/servers/3c5ab09e-0a97-4cfc-b249-a11ab221b5d2 => generated 1809 bytes in 348 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:14:07.309912 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-afa93839-c9b2-48ae-be10-8f3fb221a861 tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:14:07.314475 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-afa93839-c9b2-48ae-be10-8f3fb221a861 tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] [instance: 76ee9d21-1f17-4b64-8f95-95a33e70e907] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:14:07.319162 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-afa93839-c9b2-48ae-be10-8f3fb221a861 tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:07.321748 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-afa93839-c9b2-48ae-be10-8f3fb221a861 tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:07.322824 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-afa93839-c9b2-48ae-be10-8f3fb221a861 tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:07.323528 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-afa93839-c9b2-48ae-be10-8f3fb221a861 tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:07.518979 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-afa93839-c9b2-48ae-be10-8f3fb221a861 tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c1aea87154224f1c81428ddef4438801', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45d9edd2fe854483a6f0d676efbd6622', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:14:07.538547 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-afa93839-c9b2-48ae-be10-8f3fb221a861 tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] 10.222.0.16 "GET /compute/v2.1/servers/76ee9d21-1f17-4b64-8f95-95a33e70e907" status: 200 len: 1282 microversion: 2.1 time: 0.240210 Jul 16 21:14:07.540550 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 691/1470] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:14:07 2022] GET /compute/v2.1/servers/76ee9d21-1f17-4b64-8f95-95a33e70e907 => generated 1282 bytes in 245 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:14:07.545892 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-22ca70ad-28a4-47e9-98b3-929904c98aab req-56e09b3c-f32b-440e-aaa5-f0faf87254b7 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "3c5ab09e-0a97-4cfc-b249-a11ab221b5d2", "name": "network-vif-unplugged", "status": "completed", "tag": "4f9dfe87-5bed-49a6-8b89-464c4930e746"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:14:07.564834 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-22ca70ad-28a4-47e9-98b3-929904c98aab req-56e09b3c-f32b-440e-aaa5-f0faf87254b7 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:07.565292 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-22ca70ad-28a4-47e9-98b3-929904c98aab req-56e09b3c-f32b-440e-aaa5-f0faf87254b7 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:07.579541 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-22ca70ad-28a4-47e9-98b3-929904c98aab req-56e09b3c-f32b-440e-aaa5-f0faf87254b7 service nova] Creating event network-vif-unplugged:4f9dfe87-5bed-49a6-8b89-464c4930e746 for instance 3c5ab09e-0a97-4cfc-b249-a11ab221b5d2 on nch2-master-1 Jul 16 21:14:07.586948 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-22ca70ad-28a4-47e9-98b3-929904c98aab req-56e09b3c-f32b-440e-aaa5-f0faf87254b7 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.044912 Jul 16 21:14:07.588014 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 692/1471] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:14:07 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:14:07.593822 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-cf9e03b0-5efc-4169-9134-7fbaffcad515 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:14:07.594406 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-cf9e03b0-5efc-4169-9134-7fbaffcad515 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:14:07.603162 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cf9e03b0-5efc-4169-9134-7fbaffcad515 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:07.604446 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cf9e03b0-5efc-4169-9134-7fbaffcad515 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:07.610236 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cf9e03b0-5efc-4169-9134-7fbaffcad515 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:07.611538 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cf9e03b0-5efc-4169-9134-7fbaffcad515 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:07.642141 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-cf9e03b0-5efc-4169-9134-7fbaffcad515 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2] Going to try to terminate instance {{(pid=138681) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 16 21:14:07.670408 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:14:08.033702 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-cf9e03b0-5efc-4169-9134-7fbaffcad515 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "DELETE /compute/v2.1/servers/a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2" status: 204 len: 0 microversion: 2.1 time: 0.443663 Jul 16 21:14:08.040532 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 693/1472] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:14:07 2022] DELETE /compute/v2.1/servers/a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2 => generated 0 bytes in 446 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 16 21:14:08.043814 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-6a9704f3-d36d-4f2c-9805-765325eb1c8e req-7122b8c3-1736-4f70-9d33-da92b54e3bf5 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "e66baae9-d488-46a4-8347-623030fc8b69", "name": "network-vif-plugged", "status": "completed", "tag": "ed0a7ee7-b878-4680-b977-42bcc7c86cb8"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:14:08.055222 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-6a9704f3-d36d-4f2c-9805-765325eb1c8e req-7122b8c3-1736-4f70-9d33-da92b54e3bf5 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:08.055681 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-6a9704f3-d36d-4f2c-9805-765325eb1c8e req-7122b8c3-1736-4f70-9d33-da92b54e3bf5 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:08.071882 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-6a9704f3-d36d-4f2c-9805-765325eb1c8e req-7122b8c3-1736-4f70-9d33-da92b54e3bf5 service nova] Creating event network-vif-plugged:ed0a7ee7-b878-4680-b977-42bcc7c86cb8 for instance e66baae9-d488-46a4-8347-623030fc8b69 on nch2-master-1 Jul 16 21:14:08.076854 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-6a9704f3-d36d-4f2c-9805-765325eb1c8e req-7122b8c3-1736-4f70-9d33-da92b54e3bf5 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.040233 Jul 16 21:14:08.080570 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 694/1473] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:14:08 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:14:08.086998 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-3c345f89-1068-47c6-aa03-6e4b69b72343 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:14:08.087750 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-3c345f89-1068-47c6-aa03-6e4b69b72343 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] [instance: e66baae9-d488-46a4-8347-623030fc8b69] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:14:08.094600 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3c345f89-1068-47c6-aa03-6e4b69b72343 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:08.095078 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3c345f89-1068-47c6-aa03-6e4b69b72343 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:08.096874 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3c345f89-1068-47c6-aa03-6e4b69b72343 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:08.096874 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3c345f89-1068-47c6-aa03-6e4b69b72343 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:08.840430 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-3c345f89-1068-47c6-aa03-6e4b69b72343 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4e9df82f95ac4779859972403869096a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4337575622d04a26942905c9dc044ba7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:14:08.848560 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-3c345f89-1068-47c6-aa03-6e4b69b72343 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] 10.222.0.16 "GET /compute/v2.1/servers/e66baae9-d488-46a4-8347-623030fc8b69" status: 200 len: 1324 microversion: 2.1 time: 0.766764 Jul 16 21:14:08.849147 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 695/1474] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:14:08 2022] GET /compute/v2.1/servers/e66baae9-d488-46a4-8347-623030fc8b69 => generated 1324 bytes in 768 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:14:08.855208 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-6b77ac2b-c85d-4c96-88ea-bbda231fb40c tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:14:08.855829 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-6b77ac2b-c85d-4c96-88ea-bbda231fb40c tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] [instance: 69cabe57-da71-4a02-b4ee-c0d84b3d32a2] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:14:08.865202 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6b77ac2b-c85d-4c96-88ea-bbda231fb40c tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:08.866012 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6b77ac2b-c85d-4c96-88ea-bbda231fb40c tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:08.867993 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6b77ac2b-c85d-4c96-88ea-bbda231fb40c tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:08.868512 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6b77ac2b-c85d-4c96-88ea-bbda231fb40c tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:09.295998 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-6b77ac2b-c85d-4c96-88ea-bbda231fb40c tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '53af488f3b0242a797f539284b78d59e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bb1ede0d0b6941cc97278e6c617f9b01', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:14:09.307770 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-6b77ac2b-c85d-4c96-88ea-bbda231fb40c tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] 10.222.0.16 "GET /compute/v2.1/servers/69cabe57-da71-4a02-b4ee-c0d84b3d32a2" status: 200 len: 1333 microversion: 2.1 time: 0.456052 Jul 16 21:14:09.308973 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 696/1475] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:14:08 2022] GET /compute/v2.1/servers/69cabe57-da71-4a02-b4ee-c0d84b3d32a2 => generated 1333 bytes in 459 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:14:09.316756 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-8871c760-81a4-40f3-ba96-e6c17cad2a28 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:14:09.317906 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-8871c760-81a4-40f3-ba96-e6c17cad2a28 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:14:09.328290 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8871c760-81a4-40f3-ba96-e6c17cad2a28 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:09.328872 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8871c760-81a4-40f3-ba96-e6c17cad2a28 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:09.329764 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8871c760-81a4-40f3-ba96-e6c17cad2a28 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:09.330273 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8871c760-81a4-40f3-ba96-e6c17cad2a28 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:09.885909 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-8871c760-81a4-40f3-ba96-e6c17cad2a28 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b68dacc4b8463ba07fae6372131117', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55a19b7a4682457a9132f5fc80110712', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:14:09.900579 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-8871c760-81a4-40f3-ba96-e6c17cad2a28 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "GET /compute/v2.1/servers/a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2" status: 200 len: 1534 microversion: 2.1 time: 0.588758 Jul 16 21:14:09.904744 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 697/1476] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:14:09 2022] GET /compute/v2.1/servers/a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2 => generated 1534 bytes in 594 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:14:09.915599 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-f5f68ccc-1c7b-4690-b0d7-71697c86c650 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:14:09.943756 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-f5f68ccc-1c7b-4690-b0d7-71697c86c650 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] [instance: 3c5ab09e-0a97-4cfc-b249-a11ab221b5d2] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:14:09.973456 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f5f68ccc-1c7b-4690-b0d7-71697c86c650 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:09.975290 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f5f68ccc-1c7b-4690-b0d7-71697c86c650 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:09.976731 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f5f68ccc-1c7b-4690-b0d7-71697c86c650 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:09.980037 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f5f68ccc-1c7b-4690-b0d7-71697c86c650 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:10.307776 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-f5f68ccc-1c7b-4690-b0d7-71697c86c650 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:14:10.321677 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-f5f68ccc-1c7b-4690-b0d7-71697c86c650 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:14:10.337957 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-f5f68ccc-1c7b-4690-b0d7-71697c86c650 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:14:10.337957 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-f5f68ccc-1c7b-4690-b0d7-71697c86c650 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] 10.222.0.16 "GET /compute/v2.1/servers/3c5ab09e-0a97-4cfc-b249-a11ab221b5d2" status: 200 len: 1809 microversion: 2.25 time: 0.417653 Jul 16 21:14:10.337957 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 698/1477] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:14:09 2022] GET /compute/v2.1/servers/3c5ab09e-0a97-4cfc-b249-a11ab221b5d2 => generated 1809 bytes in 420 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:14:10.342882 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-5e8683d2-5536-476a-afe0-7f63f15e1e9c tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:14:10.348809 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-5e8683d2-5536-476a-afe0-7f63f15e1e9c tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] [instance: 76ee9d21-1f17-4b64-8f95-95a33e70e907] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:14:10.354272 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_api:SELECT=92,nova_api:INSERT=6,nova_api:UPDATE=2 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:14:10.358803 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5e8683d2-5536-476a-afe0-7f63f15e1e9c tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:10.360745 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5e8683d2-5536-476a-afe0-7f63f15e1e9c tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:10.360745 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5e8683d2-5536-476a-afe0-7f63f15e1e9c tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:10.360745 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5e8683d2-5536-476a-afe0-7f63f15e1e9c tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:10.440394 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-5e8683d2-5536-476a-afe0-7f63f15e1e9c tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c1aea87154224f1c81428ddef4438801', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45d9edd2fe854483a6f0d676efbd6622', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:14:10.448508 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-5e8683d2-5536-476a-afe0-7f63f15e1e9c tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] 10.222.0.16 "GET /compute/v2.1/servers/76ee9d21-1f17-4b64-8f95-95a33e70e907" status: 200 len: 1280 microversion: 2.1 time: 0.121250 Jul 16 21:14:10.449089 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 699/1478] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:14:10 2022] GET /compute/v2.1/servers/76ee9d21-1f17-4b64-8f95-95a33e70e907 => generated 1280 bytes in 122 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:14:10.454384 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-2eba5aba-53d3-402c-8e5d-837a7ab86995 req-4a26d1c8-f508-44f7-86d0-4b457e2438d5 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "3c5ab09e-0a97-4cfc-b249-a11ab221b5d2", "tag": "4f9dfe87-5bed-49a6-8b89-464c4930e746"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:14:10.468212 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-2eba5aba-53d3-402c-8e5d-837a7ab86995 req-4a26d1c8-f508-44f7-86d0-4b457e2438d5 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:10.469581 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-2eba5aba-53d3-402c-8e5d-837a7ab86995 req-4a26d1c8-f508-44f7-86d0-4b457e2438d5 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:10.485348 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-2eba5aba-53d3-402c-8e5d-837a7ab86995 req-4a26d1c8-f508-44f7-86d0-4b457e2438d5 service nova] Creating event network-changed:4f9dfe87-5bed-49a6-8b89-464c4930e746 for instance 3c5ab09e-0a97-4cfc-b249-a11ab221b5d2 on nch2-master-1 Jul 16 21:14:10.490948 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-2eba5aba-53d3-402c-8e5d-837a7ab86995 req-4a26d1c8-f508-44f7-86d0-4b457e2438d5 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.040434 Jul 16 21:14:10.492322 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 700/1479] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:14:10 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:14:10.497396 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-0d03fcf3-8853-4b6e-b5ef-e8cf90727ae2 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:14:10.498107 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-0d03fcf3-8853-4b6e-b5ef-e8cf90727ae2 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] [instance: e66baae9-d488-46a4-8347-623030fc8b69] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:14:10.504471 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0d03fcf3-8853-4b6e-b5ef-e8cf90727ae2 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:10.504930 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0d03fcf3-8853-4b6e-b5ef-e8cf90727ae2 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:10.505714 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0d03fcf3-8853-4b6e-b5ef-e8cf90727ae2 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:10.506166 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0d03fcf3-8853-4b6e-b5ef-e8cf90727ae2 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:11.506231 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-0d03fcf3-8853-4b6e-b5ef-e8cf90727ae2 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4e9df82f95ac4779859972403869096a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4337575622d04a26942905c9dc044ba7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:14:11.517225 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-0d03fcf3-8853-4b6e-b5ef-e8cf90727ae2 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] 10.222.0.16 "GET /compute/v2.1/servers/e66baae9-d488-46a4-8347-623030fc8b69" status: 200 len: 1520 microversion: 2.1 time: 1.023618 Jul 16 21:14:11.518487 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 701/1480] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:14:10 2022] GET /compute/v2.1/servers/e66baae9-d488-46a4-8347-623030fc8b69 => generated 1520 bytes in 1025 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:14:11.524696 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-79f7fd44-24d0-4539-aa81-1b66f359645c tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:14:11.525606 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-79f7fd44-24d0-4539-aa81-1b66f359645c tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] [instance: 69cabe57-da71-4a02-b4ee-c0d84b3d32a2] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:14:11.537312 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-79f7fd44-24d0-4539-aa81-1b66f359645c tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:11.538094 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-79f7fd44-24d0-4539-aa81-1b66f359645c tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:11.539892 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-79f7fd44-24d0-4539-aa81-1b66f359645c tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:11.540700 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-79f7fd44-24d0-4539-aa81-1b66f359645c tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:12.248174 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-5aa73b6f-0e13-4f1b-be23-fc3b2fc2f8a4 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] 10.222.0.16 "POST /compute/v2.1/os-floating-ips" status: 200 len: 142 microversion: 2.1 time: 5.828852 Jul 16 21:14:12.249805 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 780/1481] 10.222.0.16 () {60 vars in 1169 bytes} [Sat Jul 16 21:14:06 2022] POST /compute/v2.1/os-floating-ips => generated 142 bytes in 5831 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:14:12.254735 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-2d0dca9a-a08d-481f-bb1b-777aa4a06b51 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:14:12.255589 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-2d0dca9a-a08d-481f-bb1b-777aa4a06b51 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:14:12.259397 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_api:SELECT=92 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:14:12.276712 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2d0dca9a-a08d-481f-bb1b-777aa4a06b51 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:12.276712 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2d0dca9a-a08d-481f-bb1b-777aa4a06b51 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:12.276712 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2d0dca9a-a08d-481f-bb1b-777aa4a06b51 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:12.276712 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2d0dca9a-a08d-481f-bb1b-777aa4a06b51 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:12.335362 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2d0dca9a-a08d-481f-bb1b-777aa4a06b51 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:12.336068 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2d0dca9a-a08d-481f-bb1b-777aa4a06b51 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:12.336643 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.objects.instance [None req-2d0dca9a-a08d-481f-bb1b-777aa4a06b51 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lazy-loading 'fault' on Instance uuid a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2 {{(pid=138680) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:14:12.471519 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-2d0dca9a-a08d-481f-bb1b-777aa4a06b51 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b68dacc4b8463ba07fae6372131117', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55a19b7a4682457a9132f5fc80110712', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:14:12.491788 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-2d0dca9a-a08d-481f-bb1b-777aa4a06b51 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "GET /compute/v2.1/servers/a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2" status: 200 len: 1368 microversion: 2.1 time: 0.235617 Jul 16 21:14:12.491788 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 781/1482] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:14:12 2022] GET /compute/v2.1/servers/a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2 => generated 1368 bytes in 240 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:14:12.508282 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-1197cf72-09b4-431a-bbb0-682b080138e1 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:14:12.509280 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-1197cf72-09b4-431a-bbb0-682b080138e1 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] [instance: 3c5ab09e-0a97-4cfc-b249-a11ab221b5d2] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:14:12.520750 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1197cf72-09b4-431a-bbb0-682b080138e1 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:12.521471 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1197cf72-09b4-431a-bbb0-682b080138e1 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:12.523844 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1197cf72-09b4-431a-bbb0-682b080138e1 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:12.524590 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1197cf72-09b4-431a-bbb0-682b080138e1 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:12.591989 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-79f7fd44-24d0-4539-aa81-1b66f359645c tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '53af488f3b0242a797f539284b78d59e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bb1ede0d0b6941cc97278e6c617f9b01', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:14:12.604358 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-79f7fd44-24d0-4539-aa81-1b66f359645c tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] 10.222.0.16 "GET /compute/v2.1/servers/69cabe57-da71-4a02-b4ee-c0d84b3d32a2" status: 200 len: 1354 microversion: 2.1 time: 1.084526 Jul 16 21:14:12.605759 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 702/1483] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:14:11 2022] GET /compute/v2.1/servers/69cabe57-da71-4a02-b4ee-c0d84b3d32a2 => generated 1354 bytes in 1086 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:14:12.620911 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-a8b955b3-0135-4aa8-a100-bd2a9f682d30 tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:14:12.622632 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-a8b955b3-0135-4aa8-a100-bd2a9f682d30 tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] [instance: 76ee9d21-1f17-4b64-8f95-95a33e70e907] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:14:12.632829 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a8b955b3-0135-4aa8-a100-bd2a9f682d30 tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:12.638128 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a8b955b3-0135-4aa8-a100-bd2a9f682d30 tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:12.639376 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a8b955b3-0135-4aa8-a100-bd2a9f682d30 tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:12.640279 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a8b955b3-0135-4aa8-a100-bd2a9f682d30 tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:13.338967 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-a8b955b3-0135-4aa8-a100-bd2a9f682d30 tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c1aea87154224f1c81428ddef4438801', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45d9edd2fe854483a6f0d676efbd6622', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:14:13.350820 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-a8b955b3-0135-4aa8-a100-bd2a9f682d30 tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] 10.222.0.16 "GET /compute/v2.1/servers/76ee9d21-1f17-4b64-8f95-95a33e70e907" status: 200 len: 1322 microversion: 2.1 time: 0.743155 Jul 16 21:14:13.367930 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 703/1484] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:14:12 2022] GET /compute/v2.1/servers/76ee9d21-1f17-4b64-8f95-95a33e70e907 => generated 1322 bytes in 761 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:14:13.370093 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-1197cf72-09b4-431a-bbb0-682b080138e1 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:14:13.390732 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-1197cf72-09b4-431a-bbb0-682b080138e1 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:14:13.392309 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-1197cf72-09b4-431a-bbb0-682b080138e1 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:14:13.393783 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-1197cf72-09b4-431a-bbb0-682b080138e1 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] 10.222.0.16 "GET /compute/v2.1/servers/3c5ab09e-0a97-4cfc-b249-a11ab221b5d2" status: 200 len: 1809 microversion: 2.25 time: 0.889682 Jul 16 21:14:13.394866 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 782/1485] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:14:12 2022] GET /compute/v2.1/servers/3c5ab09e-0a97-4cfc-b249-a11ab221b5d2 => generated 1809 bytes in 904 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:14:13.396691 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-b3b6b801-4002-4d54-9346-59ee89f7876c tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:14:13.397992 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-b3b6b801-4002-4d54-9346-59ee89f7876c tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] [instance: e66baae9-d488-46a4-8347-623030fc8b69] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:14:13.402804 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-886c6cd6-fa79-452d-87d7-a87aced40369 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] Action: 'action', calling method: , body: {"addFloatingIp": {"address": "172.35.1.110"}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:14:13.404797 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-886c6cd6-fa79-452d-87d7-a87aced40369 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] [instance: dada2ea0-2ef4-4a30-8e2d-40177d0bafce] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:14:13.414442 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b3b6b801-4002-4d54-9346-59ee89f7876c tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:13.418896 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b3b6b801-4002-4d54-9346-59ee89f7876c tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:13.418896 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b3b6b801-4002-4d54-9346-59ee89f7876c tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:13.419435 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-886c6cd6-fa79-452d-87d7-a87aced40369 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:13.419435 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-886c6cd6-fa79-452d-87d7-a87aced40369 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:13.419642 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b3b6b801-4002-4d54-9346-59ee89f7876c tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:13.420592 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-886c6cd6-fa79-452d-87d7-a87aced40369 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:13.421141 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-886c6cd6-fa79-452d-87d7-a87aced40369 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:13.462278 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-886c6cd6-fa79-452d-87d7-a87aced40369 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] Acquired lock "refresh_cache-dada2ea0-2ef4-4a30-8e2d-40177d0bafce" {{(pid=138680) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:294}} Jul 16 21:14:13.898184 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell0:SELECT=10 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:14:13.988314 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-b3b6b801-4002-4d54-9346-59ee89f7876c tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4e9df82f95ac4779859972403869096a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4337575622d04a26942905c9dc044ba7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:14:14.005930 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-b3b6b801-4002-4d54-9346-59ee89f7876c tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] 10.222.0.16 "GET /compute/v2.1/servers/e66baae9-d488-46a4-8347-623030fc8b69" status: 200 len: 1520 microversion: 2.1 time: 0.632795 Jul 16 21:14:14.006668 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 704/1486] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:14:13 2022] GET /compute/v2.1/servers/e66baae9-d488-46a4-8347-623030fc8b69 => generated 1520 bytes in 638 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:14:14.025227 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-a394c370-48ff-4d05-ab28-986ceecc6c90 req-56ebfa8f-2f20-4023-ac75-4bdf7f536174 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2", "name": "network-vif-deleted", "tag": "bf9d54d7-5718-4678-bc2f-5ba8241bd855"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:14:14.036371 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-a394c370-48ff-4d05-ab28-986ceecc6c90 req-56ebfa8f-2f20-4023-ac75-4bdf7f536174 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:14.036891 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-a394c370-48ff-4d05-ab28-986ceecc6c90 req-56ebfa8f-2f20-4023-ac75-4bdf7f536174 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:14.051219 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-a394c370-48ff-4d05-ab28-986ceecc6c90 req-56ebfa8f-2f20-4023-ac75-4bdf7f536174 service nova] Creating event network-vif-deleted:bf9d54d7-5718-4678-bc2f-5ba8241bd855 for instance a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2 on nch1-master-1 Jul 16 21:14:14.062041 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-a394c370-48ff-4d05-ab28-986ceecc6c90 req-56ebfa8f-2f20-4023-ac75-4bdf7f536174 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.045286 Jul 16 21:14:14.066963 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 705/1487] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:14:14 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:14:14.073070 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-79b0f731-b305-4e00-b41f-1d37a6fdea2a tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:14:14.074434 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-79b0f731-b305-4e00-b41f-1d37a6fdea2a tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:14:14.084279 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-79b0f731-b305-4e00-b41f-1d37a6fdea2a tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:14.086968 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-79b0f731-b305-4e00-b41f-1d37a6fdea2a tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:14.088297 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-79b0f731-b305-4e00-b41f-1d37a6fdea2a tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:14.088955 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-79b0f731-b305-4e00-b41f-1d37a6fdea2a tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:14.114558 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-79b0f731-b305-4e00-b41f-1d37a6fdea2a tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] HTTP exception thrown: Instance a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2 could not be found. Jul 16 21:14:14.117852 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-79b0f731-b305-4e00-b41f-1d37a6fdea2a tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Request method failure captured: Jul 16 21:14:14.117852 ncdv-master-1 devstack@n-api.service[138681]: request: GET /compute/v2.1/servers/a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2 HTTP/1.1 Jul 16 21:14:14.117852 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:14:14.117852 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:14:14.117852 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:14:14.117852 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 21:14:14.117852 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:14:14.117852 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:14:14.117852 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:14:14.117852 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:14:14.117852 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0yjS0OwquTdxFvFwnqD6oXhLiVMC2H2T0PTTAPBs5CbYhBauxUQKqaIu70UDpOvDtiTqvXU4-VL9hTewWDu4Gh7O9lNbGbnPrucyL4QpaGXDfvsW3rocl0tQjubLkrzEuiaQ0dP8qaKxkSk27mrRw2WAWCg3PheG3BhIZr0IEKSfAB4 Jul 16 21:14:14.117852 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:14:14.117852 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:14:14.117852 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:14:14.117852 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:14:14.117852 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:14:14.117852 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:14:14.117852 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: 55a19b7a4682457a9132f5fc80110712 Jul 16 21:14:14.117852 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-MigrationsAdminTest-116332175 Jul 16 21:14:14.117852 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:14:14.117852 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:14:14.119518 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/55a19b7a4682457a9132f5fc80110712"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/55a19b7a4682457a9132f5fc80110712"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_55a19b7a4682457a9132f5fc80110712"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/55a19b7a4682457a9132f5fc80110712"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:14:14.119518 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-MigrationsAdminTest-116332175 Jul 16 21:14:14.119518 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: 55a19b7a4682457a9132f5fc80110712 Jul 16 21:14:14.119518 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-MigrationsAdminTest-116332175 Jul 16 21:14:14.119518 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-MigrationsAdminTest-116332175-project Jul 16 21:14:14.119518 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:14:14.119518 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:14:14.119518 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: a9b68dacc4b8463ba07fae6372131117 Jul 16 21:14:14.119518 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-MigrationsAdminTest-116332175-project Jul 16 21:14:14.119518 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:14:14.119518 ncdv-master-1 devstack@n-api.service[138681]: exception: Instance a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2 could not be found. Jul 16 21:14:14.119518 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': 'a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2'} Jul 16 21:14:14.120409 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:14:14.120409 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:14:14.120409 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:14:14.120409 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:14:14.120409 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:14:14.120409 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:14:14.120409 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:14:14.120409 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:14:14.120409 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:14:14.120409 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:14:14.120409 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2 could not be found. Jul 16 21:14:14.120409 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:14:14.120409 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:14:14.120409 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:14:14.120409 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:14:14.120409 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:14:14.120409 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:14:14.120409 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:14:14.120409 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:14:14.120409 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:14:14.121911 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:14:14.121911 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:14:14.121911 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:14:14.121911 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:14:14.121911 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:14:14.121911 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:14:14.121911 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:14:14.121911 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2 could not be found. Jul 16 21:14:14.121911 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:14:14.121911 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:14:14.121911 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:14:14.121911 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:14:14.121911 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:14:14.121911 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:14:14.121911 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:14:14.121911 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:14:14.121911 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2 could not be found. Jul 16 21:14:14.121911 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:14:14.138609 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-79b0f731-b305-4e00-b41f-1d37a6fdea2a tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Returning 404 to user: Instance a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2 could not be found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:14:14.140946 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-79b0f731-b305-4e00-b41f-1d37a6fdea2a tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "GET /compute/v2.1/servers/a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2" status: 404 len: 111 microversion: 2.1 time: 0.076763 Jul 16 21:14:14.144583 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 706/1488] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:14:14 2022] GET /compute/v2.1/servers/a7a6fe88-6ffc-40f8-9fae-f00f91a91ac2 => generated 111 bytes in 81 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:14:14.155149 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-ad45a81c-7d4a-4859-af9b-441cdc3160e1 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:14:14.155149 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-ad45a81c-7d4a-4859-af9b-441cdc3160e1 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] [instance: 69cabe57-da71-4a02-b4ee-c0d84b3d32a2] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:14:14.162886 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ad45a81c-7d4a-4859-af9b-441cdc3160e1 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:14.163790 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ad45a81c-7d4a-4859-af9b-441cdc3160e1 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:14.164580 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ad45a81c-7d4a-4859-af9b-441cdc3160e1 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:14.165082 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ad45a81c-7d4a-4859-af9b-441cdc3160e1 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:14.440951 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell0:SELECT=15 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:14:14.585268 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-ad45a81c-7d4a-4859-af9b-441cdc3160e1 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '53af488f3b0242a797f539284b78d59e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bb1ede0d0b6941cc97278e6c617f9b01', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:14:14.602320 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-ad45a81c-7d4a-4859-af9b-441cdc3160e1 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] 10.222.0.16 "GET /compute/v2.1/servers/69cabe57-da71-4a02-b4ee-c0d84b3d32a2" status: 200 len: 1354 microversion: 2.1 time: 0.454912 Jul 16 21:14:14.603803 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 707/1489] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:14:14 2022] GET /compute/v2.1/servers/69cabe57-da71-4a02-b4ee-c0d84b3d32a2 => generated 1354 bytes in 458 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:14:14.611332 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-3dad023d-b261-4c39-a81c-8c431bb2688c tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Action: 'action', calling method: >, body: {"createImage": {"name": "tempest-ImagesOneServerNegativeTestJSON-image-107384522", "metadata": {"": ""}}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:14:14.616600 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-3dad023d-b261-4c39-a81c-8c431bb2688c tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Request method failure captured: Jul 16 21:14:14.616600 ncdv-master-1 devstack@n-api.service[138681]: request: POST /compute/v2.1/servers/e66baae9-d488-46a4-8347-623030fc8b69/action HTTP/1.1 Jul 16 21:14:14.616600 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:14:14.616600 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:14:14.616600 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:14:14.616600 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 106 Jul 16 21:14:14.616600 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:14:14.616600 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:14:14.616600 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:14:14.616600 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:14:14.616600 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0yoJFUAf_8Wtq8fI-wZdJwoKyg8rUEOC_eBMeqcRE2hk6JM7_ssDplugQwmk2S9xH7Oys7Ho2tw0kc9hmlY-jX_rR-I-9WVwQYFbyogU7ki99siEGGR9aVA2GKsp5nNcVHXBtjM3bI_zfa21XSWedvaSIVwZV8kEGixKYsBf2GMYKVI Jul 16 21:14:14.616600 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:14:14.616600 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:14:14.616600 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:14:14.616600 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:14:14.616600 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:14:14.616600 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:14:14.616600 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: 4337575622d04a26942905c9dc044ba7 Jul 16 21:14:14.616600 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-ImagesOneServerNegativeTestJSON-526509748 Jul 16 21:14:14.616600 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:14:14.616600 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:14:14.618457 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/4337575622d04a26942905c9dc044ba7"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/4337575622d04a26942905c9dc044ba7"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_4337575622d04a26942905c9dc044ba7"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/4337575622d04a26942905c9dc044ba7"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:14:14.618457 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-ImagesOneServerNegativeTestJSON-526509748 Jul 16 21:14:14.618457 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: 4337575622d04a26942905c9dc044ba7 Jul 16 21:14:14.618457 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-ImagesOneServerNegativeTestJSON-526509748 Jul 16 21:14:14.618457 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-ImagesOneServerNegativeTestJSON-526509748-project Jul 16 21:14:14.618457 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:14:14.618457 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:14:14.618457 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: 4e9df82f95ac4779859972403869096a Jul 16 21:14:14.618457 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-ImagesOneServerNegativeTestJSON-526509748-project Jul 16 21:14:14.618457 ncdv-master-1 devstack@n-api.service[138681]: Jul 16 21:14:14.618457 ncdv-master-1 devstack@n-api.service[138681]: {"createImage": {"name": "tempest-ImagesOneServerNegativeTestJSON-image-107384522", "metadata": {"": ""}}} Jul 16 21:14:14.618457 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:14:14.619402 ncdv-master-1 devstack@n-api.service[138681]: exception: Invalid input for field/attribute metadata. Value: {'': ''}. '' does not match any of the regexes: '^[a-zA-Z0-9-_:. ]{1,255}$' Jul 16 21:14:14.619402 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': 'e66baae9-d488-46a4-8347-623030fc8b69', 'body': {'createImage': {'name': 'tempest-ImagesOneServerNegativeTestJSON-image-107384522', 'metadata': {'': ''}}}} Jul 16 21:14:14.619402 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:14:14.619402 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:14:14.619402 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 279, in validate Jul 16 21:14:14.619402 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Jul 16 21:14:14.619402 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/lib/python3/dist-packages/jsonschema/validators.py", line 353, in validate Jul 16 21:14:14.619402 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise error Jul 16 21:14:14.619402 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: '' does not match any of the regexes: '^[a-zA-Z0-9-_:. ]{1,255}$' Jul 16 21:14:14.619402 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:14:14.619402 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Failed validating 'additionalProperties' in schema['properties']['createImage']['properties']['metadata']: Jul 16 21:14:14.619402 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi {'additionalProperties': False, Jul 16 21:14:14.619402 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi 'patternProperties': {'^[a-zA-Z0-9-_:. ]{1,255}$': {'maxLength': 255, Jul 16 21:14:14.619402 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi 'type': 'string'}}, Jul 16 21:14:14.619402 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi 'type': 'object'} Jul 16 21:14:14.619402 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:14:14.619402 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi On instance['createImage']: Jul 16 21:14:14.619402 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi {'': ''} Jul 16 21:14:14.619402 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:14:14.619402 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:14:14.619402 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:14:14.620965 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:14:14.620965 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:14:14.620965 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:14:14.620965 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:14:14.620965 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:14:14.620965 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:14:14.620965 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:14:14.620965 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 16 21:14:14.620965 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 16 21:14:14.620965 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Jul 16 21:14:14.620965 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Jul 16 21:14:14.620965 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Jul 16 21:14:14.620965 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Jul 16 21:14:14.620965 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 308, in validate Jul 16 21:14:14.620965 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Jul 16 21:14:14.620965 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute metadata. Value: {'': ''}. '' does not match any of the regexes: '^[a-zA-Z0-9-_:. ]{1,255}$' Jul 16 21:14:14.620965 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:14:14.620965 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:14:14.620965 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:14:14.620965 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:14:14.622420 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:14:14.622420 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:14:14.622420 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Jul 16 21:14:14.622420 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Jul 16 21:14:14.622420 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute metadata. Value: {'': ''}. '' does not match any of the regexes: '^[a-zA-Z0-9-_:. ]{1,255}$' Jul 16 21:14:14.622420 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:14:14.629994 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-3dad023d-b261-4c39-a81c-8c431bb2688c tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Returning 400 to user: Invalid input for field/attribute metadata. Value: {'': ''}. '' does not match any of the regexes: '^[a-zA-Z0-9-_:. ]{1,255}$' {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:14:14.629994 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-3dad023d-b261-4c39-a81c-8c431bb2688c tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] 10.222.0.16 "POST /compute/v2.1/servers/e66baae9-d488-46a4-8347-623030fc8b69/action" status: 400 len: 170 microversion: 2.1 time: 0.021026 Jul 16 21:14:14.629994 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 708/1490] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Jul 16 21:14:14 2022] POST /compute/v2.1/servers/e66baae9-d488-46a4-8347-623030fc8b69/action => generated 170 bytes in 23 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Jul 16 21:14:14.649751 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-fa2125d1-2dcd-4fce-be2f-e9747bfe0856 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Calling method '.version_select at 0x7fdd9d5c3f70>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:14:14.749060 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-fa2125d1-2dcd-4fce-be2f-e9747bfe0856 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-180937062" status: 202 len: 0 microversion: 2.1 time: 0.120517 Jul 16 21:14:14.751842 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 709/1491] 10.222.0.16 () {58 vars in 1217 bytes} [Sat Jul 16 21:14:14 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-180937062 => generated 0 bytes in 122 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 21:14:14.754733 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-8e153262-6d88-4d1d-adc2-a8208c30d19f tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:14:14.755613 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-8e153262-6d88-4d1d-adc2-a8208c30d19f tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] [instance: 76ee9d21-1f17-4b64-8f95-95a33e70e907] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:14:14.780378 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8e153262-6d88-4d1d-adc2-a8208c30d19f tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:14.781146 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8e153262-6d88-4d1d-adc2-a8208c30d19f tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:14.782432 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8e153262-6d88-4d1d-adc2-a8208c30d19f tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:14.783104 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8e153262-6d88-4d1d-adc2-a8208c30d19f tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:15.457604 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-8e153262-6d88-4d1d-adc2-a8208c30d19f tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c1aea87154224f1c81428ddef4438801', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45d9edd2fe854483a6f0d676efbd6622', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:14:15.484809 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-8e153262-6d88-4d1d-adc2-a8208c30d19f tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] 10.222.0.16 "GET /compute/v2.1/servers/76ee9d21-1f17-4b64-8f95-95a33e70e907" status: 200 len: 1322 microversion: 2.1 time: 0.733991 Jul 16 21:14:15.496784 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 710/1492] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:14:14 2022] GET /compute/v2.1/servers/76ee9d21-1f17-4b64-8f95-95a33e70e907 => generated 1322 bytes in 746 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:14:15.509779 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-07eaf6f8-52d7-4b7b-8714-b5b3b6c42cd2 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:14:15.510885 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-07eaf6f8-52d7-4b7b-8714-b5b3b6c42cd2 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] [instance: 3c5ab09e-0a97-4cfc-b249-a11ab221b5d2] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:14:15.526712 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-07eaf6f8-52d7-4b7b-8714-b5b3b6c42cd2 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:15.526712 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-07eaf6f8-52d7-4b7b-8714-b5b3b6c42cd2 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:15.526712 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-07eaf6f8-52d7-4b7b-8714-b5b3b6c42cd2 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:15.526712 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-07eaf6f8-52d7-4b7b-8714-b5b3b6c42cd2 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:15.724333 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell0:SELECT=2 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:14:16.329199 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-07eaf6f8-52d7-4b7b-8714-b5b3b6c42cd2 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:14:16.338119 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-07eaf6f8-52d7-4b7b-8714-b5b3b6c42cd2 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:14:16.339861 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-07eaf6f8-52d7-4b7b-8714-b5b3b6c42cd2 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:14:16.341092 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-07eaf6f8-52d7-4b7b-8714-b5b3b6c42cd2 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] 10.222.0.16 "GET /compute/v2.1/servers/3c5ab09e-0a97-4cfc-b249-a11ab221b5d2" status: 200 len: 1809 microversion: 2.25 time: 0.842986 Jul 16 21:14:16.341848 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 711/1493] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:14:15 2022] GET /compute/v2.1/servers/3c5ab09e-0a97-4cfc-b249-a11ab221b5d2 => generated 1809 bytes in 844 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:14:16.346911 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-1e128743-c89f-4bdd-8163-426b1ea02b14 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:14:16.347767 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-1e128743-c89f-4bdd-8163-426b1ea02b14 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] [instance: e66baae9-d488-46a4-8347-623030fc8b69] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:14:16.359397 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1e128743-c89f-4bdd-8163-426b1ea02b14 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:16.359898 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1e128743-c89f-4bdd-8163-426b1ea02b14 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:16.360790 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1e128743-c89f-4bdd-8163-426b1ea02b14 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:16.361299 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1e128743-c89f-4bdd-8163-426b1ea02b14 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:16.838190 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-1e128743-c89f-4bdd-8163-426b1ea02b14 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4e9df82f95ac4779859972403869096a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4337575622d04a26942905c9dc044ba7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:14:16.852352 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-1e128743-c89f-4bdd-8163-426b1ea02b14 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] 10.222.0.16 "GET /compute/v2.1/servers/e66baae9-d488-46a4-8347-623030fc8b69" status: 200 len: 1520 microversion: 2.1 time: 0.508289 Jul 16 21:14:16.854299 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 712/1494] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:14:16 2022] GET /compute/v2.1/servers/e66baae9-d488-46a4-8347-623030fc8b69 => generated 1520 bytes in 512 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:14:17.036016 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-ef0af307-37bf-45b5-bbae-5e45d7f5ab55 tempest-FloatingIPsTestJSON-1267052927 tempest-FloatingIPsTestJSON-1267052927-project] Action: 'create', calling method: .version_select at 0x7fdd9d540ee0>, body: {"pool": null} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:14:17.369728 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.network.neutron [None req-886c6cd6-fa79-452d-87d7-a87aced40369 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] [instance: dada2ea0-2ef4-4a30-8e2d-40177d0bafce] Building network info cache for instance {{(pid=138680) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1990}} Jul 16 21:14:18.920786 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.network.neutron [None req-886c6cd6-fa79-452d-87d7-a87aced40369 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] [instance: dada2ea0-2ef4-4a30-8e2d-40177d0bafce] Updating instance_info_cache with network_info: [{"id": "b63bf933-a84e-4e18-98ed-0e772979bec5", "address": "fa:16:3e:68:6e:75", "network": {"id": "a27fab11-02d4-4a1c-99a4-dc86a33836ba", "bridge": null, "label": "tempest-FloatingIPsAssociationNegativeTestJSON-255901221-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.35.1.110", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "172.20.0.2"}}], "meta": {"injected": false, "tenant_id": "6c2095f96f8d45bb9e62c517678c3dcd", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false, "bound_drivers": {"0": "hyperv"}}, "devname": "tapb63bf933-a8", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=138680) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:117}} Jul 16 21:14:18.946203 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=100 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:14:19.116132 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-886c6cd6-fa79-452d-87d7-a87aced40369 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] Releasing lock "refresh_cache-dada2ea0-2ef4-4a30-8e2d-40177d0bafce" {{(pid=138680) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:312}} Jul 16 21:14:19.117489 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-886c6cd6-fa79-452d-87d7-a87aced40369 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] 10.222.0.16 "POST /compute/v2.1/servers/dada2ea0-2ef4-4a30-8e2d-40177d0bafce/action" status: 202 len: 0 microversion: 2.1 time: 5.721421 Jul 16 21:14:19.118282 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 783/1495] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 21:14:13 2022] POST /compute/v2.1/servers/dada2ea0-2ef4-4a30-8e2d-40177d0bafce/action => generated 0 bytes in 5723 msecs (HTTP/1.1 202) 9 headers in 369 bytes (1 switches on core 0) Jul 16 21:14:19.125231 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-9540d4df-c785-404e-8dfb-ffb0e82550f7 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:14:19.126075 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-9540d4df-c785-404e-8dfb-ffb0e82550f7 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] [instance: 69cabe57-da71-4a02-b4ee-c0d84b3d32a2] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:14:19.133952 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9540d4df-c785-404e-8dfb-ffb0e82550f7 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:19.136103 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9540d4df-c785-404e-8dfb-ffb0e82550f7 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:19.137203 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9540d4df-c785-404e-8dfb-ffb0e82550f7 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:19.137648 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9540d4df-c785-404e-8dfb-ffb0e82550f7 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:19.958249 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-9540d4df-c785-404e-8dfb-ffb0e82550f7 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '53af488f3b0242a797f539284b78d59e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bb1ede0d0b6941cc97278e6c617f9b01', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:14:19.973250 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-9540d4df-c785-404e-8dfb-ffb0e82550f7 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] 10.222.0.16 "GET /compute/v2.1/servers/69cabe57-da71-4a02-b4ee-c0d84b3d32a2" status: 200 len: 1354 microversion: 2.1 time: 0.848146 Jul 16 21:14:19.984020 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 784/1496] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:14:19 2022] GET /compute/v2.1/servers/69cabe57-da71-4a02-b4ee-c0d84b3d32a2 => generated 1354 bytes in 855 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:14:19.984020 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-9b2a1720-cf09-42e9-917c-5b5df81702bd req-de5de4d4-8683-4c59-be63-da30e5eacd4e service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "76ee9d21-1f17-4b64-8f95-95a33e70e907", "tag": "65068e4e-def5-46ec-95ec-de961e8c1e0e"}, {"name": "network-changed", "server_uuid": "69cabe57-da71-4a02-b4ee-c0d84b3d32a2", "tag": "9e27e304-0422-4ee7-8ca0-2d670e12f5ee"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:14:19.993208 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-9b2a1720-cf09-42e9-917c-5b5df81702bd req-de5de4d4-8683-4c59-be63-da30e5eacd4e service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:19.993801 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-9b2a1720-cf09-42e9-917c-5b5df81702bd req-de5de4d4-8683-4c59-be63-da30e5eacd4e service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:20.008871 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-9b2a1720-cf09-42e9-917c-5b5df81702bd req-de5de4d4-8683-4c59-be63-da30e5eacd4e service nova] Creating event network-changed:65068e4e-def5-46ec-95ec-de961e8c1e0e for instance 76ee9d21-1f17-4b64-8f95-95a33e70e907 on nch2-master-1 Jul 16 21:14:20.009562 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-9b2a1720-cf09-42e9-917c-5b5df81702bd req-de5de4d4-8683-4c59-be63-da30e5eacd4e service nova] Creating event network-changed:9e27e304-0422-4ee7-8ca0-2d670e12f5ee for instance 69cabe57-da71-4a02-b4ee-c0d84b3d32a2 on nch1-master-1 Jul 16 21:14:20.026201 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-9b2a1720-cf09-42e9-917c-5b5df81702bd req-de5de4d4-8683-4c59-be63-da30e5eacd4e service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 346 microversion: 2.1 time: 0.050429 Jul 16 21:14:20.027603 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 785/1497] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:14:19 2022] POST /compute/v2.1/os-server-external-events => generated 346 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:14:20.033813 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-e9379b1b-219b-402d-94b4-1ee80b961fdd tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:14:20.035095 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-e9379b1b-219b-402d-94b4-1ee80b961fdd tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] [instance: 76ee9d21-1f17-4b64-8f95-95a33e70e907] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:14:20.045412 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e9379b1b-219b-402d-94b4-1ee80b961fdd tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:20.046216 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e9379b1b-219b-402d-94b4-1ee80b961fdd tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:20.049397 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e9379b1b-219b-402d-94b4-1ee80b961fdd tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:20.049915 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e9379b1b-219b-402d-94b4-1ee80b961fdd tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:20.702229 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-e9379b1b-219b-402d-94b4-1ee80b961fdd tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c1aea87154224f1c81428ddef4438801', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45d9edd2fe854483a6f0d676efbd6622', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:14:20.710116 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-e9379b1b-219b-402d-94b4-1ee80b961fdd tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] 10.222.0.16 "GET /compute/v2.1/servers/76ee9d21-1f17-4b64-8f95-95a33e70e907" status: 200 len: 1322 microversion: 2.1 time: 0.681137 Jul 16 21:14:20.711023 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 786/1498] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:14:20 2022] GET /compute/v2.1/servers/76ee9d21-1f17-4b64-8f95-95a33e70e907 => generated 1322 bytes in 683 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:14:20.716078 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-48a39b37-c1a5-40a4-abf3-2dc82b312210 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:14:20.716579 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-48a39b37-c1a5-40a4-abf3-2dc82b312210 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: f98a9762-a67c-4e95-85b8-727c81bdb03a] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:14:20.723269 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-48a39b37-c1a5-40a4-abf3-2dc82b312210 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:20.725060 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-48a39b37-c1a5-40a4-abf3-2dc82b312210 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:20.725060 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-48a39b37-c1a5-40a4-abf3-2dc82b312210 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:20.725060 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-48a39b37-c1a5-40a4-abf3-2dc82b312210 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:20.758299 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-48a39b37-c1a5-40a4-abf3-2dc82b312210 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: f98a9762-a67c-4e95-85b8-727c81bdb03a] Going to try to terminate instance {{(pid=138680) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 16 21:14:21.345388 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-48a39b37-c1a5-40a4-abf3-2dc82b312210 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "DELETE /compute/v2.1/servers/f98a9762-a67c-4e95-85b8-727c81bdb03a" status: 204 len: 0 microversion: 2.1 time: 0.632872 Jul 16 21:14:21.354445 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 787/1499] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:14:20 2022] DELETE /compute/v2.1/servers/f98a9762-a67c-4e95-85b8-727c81bdb03a => generated 0 bytes in 636 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 16 21:14:21.359536 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-f1e607b7-5972-481c-86a2-324cd711d9d0 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:14:21.362008 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-f1e607b7-5972-481c-86a2-324cd711d9d0 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] [instance: e66baae9-d488-46a4-8347-623030fc8b69] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:14:21.370740 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f1e607b7-5972-481c-86a2-324cd711d9d0 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:21.371489 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f1e607b7-5972-481c-86a2-324cd711d9d0 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:21.372313 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f1e607b7-5972-481c-86a2-324cd711d9d0 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:21.372781 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f1e607b7-5972-481c-86a2-324cd711d9d0 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:21.519435 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-ef0af307-37bf-45b5-bbae-5e45d7f5ab55 tempest-FloatingIPsTestJSON-1267052927 tempest-FloatingIPsTestJSON-1267052927-project] 10.222.0.16 "POST /compute/v2.1/os-floating-ips" status: 200 len: 142 microversion: 2.1 time: 4.661102 Jul 16 21:14:21.519435 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 713/1500] 10.222.0.16 () {60 vars in 1169 bytes} [Sat Jul 16 21:14:16 2022] POST /compute/v2.1/os-floating-ips => generated 142 bytes in 4663 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:14:21.525408 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-cb3d8a04-5d60-41e5-bdfb-09a0bb2feb00 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:14:21.526130 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-cb3d8a04-5d60-41e5-bdfb-09a0bb2feb00 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] [instance: 3c5ab09e-0a97-4cfc-b249-a11ab221b5d2] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:14:21.532731 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cb3d8a04-5d60-41e5-bdfb-09a0bb2feb00 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:21.533267 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cb3d8a04-5d60-41e5-bdfb-09a0bb2feb00 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:21.534188 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cb3d8a04-5d60-41e5-bdfb-09a0bb2feb00 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:21.534638 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cb3d8a04-5d60-41e5-bdfb-09a0bb2feb00 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:21.770880 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-f1e607b7-5972-481c-86a2-324cd711d9d0 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4e9df82f95ac4779859972403869096a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4337575622d04a26942905c9dc044ba7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:14:21.779778 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-f1e607b7-5972-481c-86a2-324cd711d9d0 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] 10.222.0.16 "GET /compute/v2.1/servers/e66baae9-d488-46a4-8347-623030fc8b69" status: 200 len: 1520 microversion: 2.1 time: 0.427935 Jul 16 21:14:21.780698 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 788/1501] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:14:21 2022] GET /compute/v2.1/servers/e66baae9-d488-46a4-8347-623030fc8b69 => generated 1520 bytes in 430 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:14:21.785721 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-817d7a73-ce2c-4d1a-bcff-f34e79c42e62 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] Action: 'create', calling method: .version_select at 0x7fdd9d07a8b0>, body: {"pool": "f39f47fa-8d3d-4a2f-9486-cc28d1bbfe66"} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:14:21.939585 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-cb3d8a04-5d60-41e5-bdfb-09a0bb2feb00 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:14:21.952243 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-cb3d8a04-5d60-41e5-bdfb-09a0bb2feb00 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:14:21.954200 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-cb3d8a04-5d60-41e5-bdfb-09a0bb2feb00 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:14:21.955819 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-cb3d8a04-5d60-41e5-bdfb-09a0bb2feb00 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] 10.222.0.16 "GET /compute/v2.1/servers/3c5ab09e-0a97-4cfc-b249-a11ab221b5d2" status: 200 len: 1809 microversion: 2.25 time: 0.436680 Jul 16 21:14:21.956485 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 714/1502] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:14:21 2022] GET /compute/v2.1/servers/3c5ab09e-0a97-4cfc-b249-a11ab221b5d2 => generated 1809 bytes in 438 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:14:21.964271 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-70fd85df-9476-41ef-b752-0437b5316ddf req-17204dd9-7155-4e18-a1eb-bf912553d6e7 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "3c5ab09e-0a97-4cfc-b249-a11ab221b5d2", "name": "network-vif-unplugged", "status": "completed", "tag": "4f9dfe87-5bed-49a6-8b89-464c4930e746"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:14:21.973097 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-70fd85df-9476-41ef-b752-0437b5316ddf req-17204dd9-7155-4e18-a1eb-bf912553d6e7 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:21.973920 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-70fd85df-9476-41ef-b752-0437b5316ddf req-17204dd9-7155-4e18-a1eb-bf912553d6e7 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:21.978815 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:14:21.986090 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-70fd85df-9476-41ef-b752-0437b5316ddf req-17204dd9-7155-4e18-a1eb-bf912553d6e7 service nova] Creating event network-vif-unplugged:4f9dfe87-5bed-49a6-8b89-464c4930e746 for instance 3c5ab09e-0a97-4cfc-b249-a11ab221b5d2 on nch2-master-1 Jul 16 21:14:21.992561 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-70fd85df-9476-41ef-b752-0437b5316ddf req-17204dd9-7155-4e18-a1eb-bf912553d6e7 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.034715 Jul 16 21:14:21.993308 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 715/1503] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:14:21 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:14:21.999359 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-84ccbb47-2a81-49b7-aa7e-4e9eb5a53603 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:14:22.000063 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-84ccbb47-2a81-49b7-aa7e-4e9eb5a53603 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] [instance: 69cabe57-da71-4a02-b4ee-c0d84b3d32a2] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:14:22.007639 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-84ccbb47-2a81-49b7-aa7e-4e9eb5a53603 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:22.008017 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-84ccbb47-2a81-49b7-aa7e-4e9eb5a53603 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:22.008795 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-84ccbb47-2a81-49b7-aa7e-4e9eb5a53603 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:22.009237 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-84ccbb47-2a81-49b7-aa7e-4e9eb5a53603 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:22.345748 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-84ccbb47-2a81-49b7-aa7e-4e9eb5a53603 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '53af488f3b0242a797f539284b78d59e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bb1ede0d0b6941cc97278e6c617f9b01', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:14:22.355553 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-84ccbb47-2a81-49b7-aa7e-4e9eb5a53603 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] 10.222.0.16 "GET /compute/v2.1/servers/69cabe57-da71-4a02-b4ee-c0d84b3d32a2" status: 200 len: 1354 microversion: 2.1 time: 0.360601 Jul 16 21:14:22.356145 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 716/1504] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:14:21 2022] GET /compute/v2.1/servers/69cabe57-da71-4a02-b4ee-c0d84b3d32a2 => generated 1354 bytes in 362 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:14:22.369953 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-46599c26-c9ee-4850-b3bf-6a925ae144a8 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:14:22.371912 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-46599c26-c9ee-4850-b3bf-6a925ae144a8 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: f98a9762-a67c-4e95-85b8-727c81bdb03a] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:14:22.378769 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-46599c26-c9ee-4850-b3bf-6a925ae144a8 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:22.379773 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-46599c26-c9ee-4850-b3bf-6a925ae144a8 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:22.380790 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-46599c26-c9ee-4850-b3bf-6a925ae144a8 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:22.381862 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-46599c26-c9ee-4850-b3bf-6a925ae144a8 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:23.130140 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-46599c26-c9ee-4850-b3bf-6a925ae144a8 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b68dacc4b8463ba07fae6372131117', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55a19b7a4682457a9132f5fc80110712', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:14:23.139430 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-46599c26-c9ee-4850-b3bf-6a925ae144a8 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "GET /compute/v2.1/servers/f98a9762-a67c-4e95-85b8-727c81bdb03a" status: 200 len: 1468 microversion: 2.1 time: 0.782035 Jul 16 21:14:23.140013 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 717/1505] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:14:22 2022] GET /compute/v2.1/servers/f98a9762-a67c-4e95-85b8-727c81bdb03a => generated 1468 bytes in 783 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:14:23.144541 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-90fcac63-0957-4112-a4bc-8e0f5e133a13 tempest-FloatingIPsTestJSON-1267052927 tempest-FloatingIPsTestJSON-1267052927-project] Calling method '.version_select at 0x7fdd9d5c3c10>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:14:23.691568 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-90fcac63-0957-4112-a4bc-8e0f5e133a13 tempest-FloatingIPsTestJSON-1267052927 tempest-FloatingIPsTestJSON-1267052927-project] 10.222.0.16 "GET /compute/v2.1/os-floating-ips/837b239a-219d-4f51-8490-ee6de4391d9f" status: 200 len: 142 microversion: 2.1 time: 0.550399 Jul 16 21:14:23.692160 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 718/1506] 10.222.0.16 () {58 vars in 1259 bytes} [Sat Jul 16 21:14:23 2022] GET /compute/v2.1/os-floating-ips/837b239a-219d-4f51-8490-ee6de4391d9f => generated 142 bytes in 552 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:14:23.697771 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-a271c190-b956-4277-8c86-4aa4af9f7601 tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:14:23.699169 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-a271c190-b956-4277-8c86-4aa4af9f7601 tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] [instance: 76ee9d21-1f17-4b64-8f95-95a33e70e907] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:14:23.705818 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a271c190-b956-4277-8c86-4aa4af9f7601 tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:23.706422 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a271c190-b956-4277-8c86-4aa4af9f7601 tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:23.707235 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a271c190-b956-4277-8c86-4aa4af9f7601 tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:23.707675 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a271c190-b956-4277-8c86-4aa4af9f7601 tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:24.234370 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-a271c190-b956-4277-8c86-4aa4af9f7601 tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c1aea87154224f1c81428ddef4438801', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45d9edd2fe854483a6f0d676efbd6622', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:14:24.243381 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-a271c190-b956-4277-8c86-4aa4af9f7601 tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] 10.222.0.16 "GET /compute/v2.1/servers/76ee9d21-1f17-4b64-8f95-95a33e70e907" status: 200 len: 1322 microversion: 2.1 time: 0.550120 Jul 16 21:14:24.244602 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 719/1507] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:14:23 2022] GET /compute/v2.1/servers/76ee9d21-1f17-4b64-8f95-95a33e70e907 => generated 1322 bytes in 552 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:14:24.257225 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-20aa2f63-984b-43d2-bab9-8fd9f0bbd3f7 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Action: 'action', calling method: >, body: {"createImage": {"name": "tempest-ImagesOneServerNegativeTestJSON-image-1344451433", "metadata": {"aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa": "bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb"}}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:14:24.262964 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-20aa2f63-984b-43d2-bab9-8fd9f0bbd3f7 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Request method failure captured: Jul 16 21:14:24.262964 ncdv-master-1 devstack@n-api.service[138681]: request: POST /compute/v2.1/servers/e66baae9-d488-46a4-8347-623030fc8b69/action HTTP/1.1 Jul 16 21:14:24.262964 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:14:24.262964 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:14:24.262964 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:14:24.262964 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 619 Jul 16 21:14:24.262964 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:14:24.262964 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:14:24.262964 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:14:24.262964 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:14:24.262964 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0yoJFUAf_8Wtq8fI-wZdJwoKyg8rUEOC_eBMeqcRE2hk6JM7_ssDplugQwmk2S9xH7Oys7Ho2tw0kc9hmlY-jX_rR-I-9WVwQYFbyogU7ki99siEGGR9aVA2GKsp5nNcVHXBtjM3bI_zfa21XSWedvaSIVwZV8kEGixKYsBf2GMYKVI Jul 16 21:14:24.262964 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:14:24.262964 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:14:24.262964 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:14:24.262964 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:14:24.262964 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:14:24.262964 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:14:24.262964 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: 4337575622d04a26942905c9dc044ba7 Jul 16 21:14:24.262964 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-ImagesOneServerNegativeTestJSON-526509748 Jul 16 21:14:24.262964 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:14:24.262964 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:14:24.264660 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/4337575622d04a26942905c9dc044ba7"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/4337575622d04a26942905c9dc044ba7"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_4337575622d04a26942905c9dc044ba7"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/4337575622d04a26942905c9dc044ba7"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:14:24.264660 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-ImagesOneServerNegativeTestJSON-526509748 Jul 16 21:14:24.264660 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: 4337575622d04a26942905c9dc044ba7 Jul 16 21:14:24.264660 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-ImagesOneServerNegativeTestJSON-526509748 Jul 16 21:14:24.264660 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-ImagesOneServerNegativeTestJSON-526509748-project Jul 16 21:14:24.264660 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:14:24.264660 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:14:24.264660 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: 4e9df82f95ac4779859972403869096a Jul 16 21:14:24.264660 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-ImagesOneServerNegativeTestJSON-526509748-project Jul 16 21:14:24.264660 ncdv-master-1 devstack@n-api.service[138681]: Jul 16 21:14:24.265380 ncdv-master-1 devstack@n-api.service[138681]: {"createImage": {"name": "tempest-ImagesOneServerNegativeTestJSON-image-1344451433", "metadata": {"aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa": "bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb"}}} Jul 16 21:14:24.265380 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:14:24.265380 ncdv-master-1 devstack@n-api.service[138681]: exception: Invalid input for field/attribute metadata. Value: {'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa': 'bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb'}. 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' does not match any of the regexes: '^[a-zA-Z0-9-_:. ]{1,255}$' Jul 16 21:14:24.265731 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': 'e66baae9-d488-46a4-8347-623030fc8b69', 'body': {'createImage': {'name': 'tempest-ImagesOneServerNegativeTestJSON-image-1344451433', 'metadata': {'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa': 'bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb'}}}} Jul 16 21:14:24.265731 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:14:24.265731 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:14:24.265731 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 279, in validate Jul 16 21:14:24.265731 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Jul 16 21:14:24.265731 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/lib/python3/dist-packages/jsonschema/validators.py", line 353, in validate Jul 16 21:14:24.265731 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise error Jul 16 21:14:24.265731 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' does not match any of the regexes: '^[a-zA-Z0-9-_:. ]{1,255}$' Jul 16 21:14:24.265731 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:14:24.265731 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Failed validating 'additionalProperties' in schema['properties']['createImage']['properties']['metadata']: Jul 16 21:14:24.265731 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi {'additionalProperties': False, Jul 16 21:14:24.267339 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi 'patternProperties': {'^[a-zA-Z0-9-_:. ]{1,255}$': {'maxLength': 255, Jul 16 21:14:24.267339 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi 'type': 'string'}}, Jul 16 21:14:24.267339 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi 'type': 'object'} Jul 16 21:14:24.267339 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:14:24.267339 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi On instance['createImage']: Jul 16 21:14:24.267339 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi {'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa': 'bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb'} Jul 16 21:14:24.267339 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:14:24.267339 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:14:24.267339 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:14:24.267339 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:14:24.267339 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:14:24.267339 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:14:24.267339 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:14:24.267339 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:14:24.267339 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:14:24.267339 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:14:24.267339 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 16 21:14:24.270018 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 16 21:14:24.270018 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Jul 16 21:14:24.270018 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Jul 16 21:14:24.270018 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Jul 16 21:14:24.270018 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Jul 16 21:14:24.270018 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 308, in validate Jul 16 21:14:24.270018 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Jul 16 21:14:24.270018 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute metadata. Value: {'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa': 'bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb'}. 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' does not match any of the regexes: '^[a-zA-Z0-9-_:. ]{1,255}$' Jul 16 21:14:24.270018 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:14:24.270018 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:14:24.270018 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:14:24.270018 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:14:24.270874 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:14:24.270874 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:14:24.270874 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Jul 16 21:14:24.270874 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Jul 16 21:14:24.270874 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute metadata. Value: {'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa': 'bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb'}. 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' does not match any of the regexes: '^[a-zA-Z0-9-_:. ]{1,255}$' Jul 16 21:14:24.270874 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:14:24.271354 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-20aa2f63-984b-43d2-bab9-8fd9f0bbd3f7 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Returning 400 to user: Invalid input for field/attribute metadata. Value: {'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa': 'bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb'}. 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' does not match any of the regexes: '^[a-zA-Z0-9-_:. ]{1,255}$' {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:14:24.271354 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-20aa2f63-984b-43d2-bab9-8fd9f0bbd3f7 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] 10.222.0.16 "POST /compute/v2.1/servers/e66baae9-d488-46a4-8347-623030fc8b69/action" status: 400 len: 938 microversion: 2.1 time: 0.018981 Jul 16 21:14:24.275343 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 720/1508] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Jul 16 21:14:24 2022] POST /compute/v2.1/servers/e66baae9-d488-46a4-8347-623030fc8b69/action => generated 938 bytes in 30 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Jul 16 21:14:24.280588 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-9534c9f7-f440-40b5-8cb9-c27fd999a237 req-57df89fd-8930-4b1b-a93a-55d2aea5c4b9 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "dada2ea0-2ef4-4a30-8e2d-40177d0bafce", "tag": "b63bf933-a84e-4e18-98ed-0e772979bec5"}, {"name": "network-changed", "server_uuid": "69cabe57-da71-4a02-b4ee-c0d84b3d32a2", "tag": "da98933d-c6aa-4341-b47f-1483a9e3aa36"}, {"name": "network-changed", "server_uuid": "69cabe57-da71-4a02-b4ee-c0d84b3d32a2", "tag": "1f354c18-a81d-412a-97e0-43d35608c82c"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:14:24.290961 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-9534c9f7-f440-40b5-8cb9-c27fd999a237 req-57df89fd-8930-4b1b-a93a-55d2aea5c4b9 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:24.291439 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-9534c9f7-f440-40b5-8cb9-c27fd999a237 req-57df89fd-8930-4b1b-a93a-55d2aea5c4b9 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:24.305168 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-9534c9f7-f440-40b5-8cb9-c27fd999a237 req-57df89fd-8930-4b1b-a93a-55d2aea5c4b9 service nova] Creating event network-changed:b63bf933-a84e-4e18-98ed-0e772979bec5 for instance dada2ea0-2ef4-4a30-8e2d-40177d0bafce on nch2-master-1 Jul 16 21:14:24.305693 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-9534c9f7-f440-40b5-8cb9-c27fd999a237 req-57df89fd-8930-4b1b-a93a-55d2aea5c4b9 service nova] Creating event network-changed:da98933d-c6aa-4341-b47f-1483a9e3aa36 for instance 69cabe57-da71-4a02-b4ee-c0d84b3d32a2 on nch1-master-1 Jul 16 21:14:24.306344 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-9534c9f7-f440-40b5-8cb9-c27fd999a237 req-57df89fd-8930-4b1b-a93a-55d2aea5c4b9 service nova] Creating event network-changed:1f354c18-a81d-412a-97e0-43d35608c82c for instance 69cabe57-da71-4a02-b4ee-c0d84b3d32a2 on nch1-master-1 Jul 16 21:14:24.314770 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-9534c9f7-f440-40b5-8cb9-c27fd999a237 req-57df89fd-8930-4b1b-a93a-55d2aea5c4b9 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 513 microversion: 2.1 time: 0.038082 Jul 16 21:14:24.315472 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 721/1509] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:14:24 2022] POST /compute/v2.1/os-server-external-events => generated 513 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:14:24.322319 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-0b47c78f-6ba7-40d4-bdd7-b6b72c8f4e17 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:14:24.323090 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-0b47c78f-6ba7-40d4-bdd7-b6b72c8f4e17 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] [instance: 3c5ab09e-0a97-4cfc-b249-a11ab221b5d2] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:14:24.330404 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0b47c78f-6ba7-40d4-bdd7-b6b72c8f4e17 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:24.330860 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0b47c78f-6ba7-40d4-bdd7-b6b72c8f4e17 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:24.332220 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0b47c78f-6ba7-40d4-bdd7-b6b72c8f4e17 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:24.334269 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0b47c78f-6ba7-40d4-bdd7-b6b72c8f4e17 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:24.977931 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-0b47c78f-6ba7-40d4-bdd7-b6b72c8f4e17 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:14:24.987598 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-0b47c78f-6ba7-40d4-bdd7-b6b72c8f4e17 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:14:24.990782 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-0b47c78f-6ba7-40d4-bdd7-b6b72c8f4e17 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:14:24.992326 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-0b47c78f-6ba7-40d4-bdd7-b6b72c8f4e17 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] 10.222.0.16 "GET /compute/v2.1/servers/3c5ab09e-0a97-4cfc-b249-a11ab221b5d2" status: 200 len: 1809 microversion: 2.25 time: 0.675736 Jul 16 21:14:25.002874 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 722/1510] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:14:24 2022] GET /compute/v2.1/servers/3c5ab09e-0a97-4cfc-b249-a11ab221b5d2 => generated 1809 bytes in 679 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:14:25.002874 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-eeffa1f5-f37a-436b-9092-3c20c77bec34 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:14:25.002874 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-eeffa1f5-f37a-436b-9092-3c20c77bec34 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] [instance: 69cabe57-da71-4a02-b4ee-c0d84b3d32a2] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:14:25.008059 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-eeffa1f5-f37a-436b-9092-3c20c77bec34 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:25.008512 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-eeffa1f5-f37a-436b-9092-3c20c77bec34 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:25.009281 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-eeffa1f5-f37a-436b-9092-3c20c77bec34 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:25.009717 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-eeffa1f5-f37a-436b-9092-3c20c77bec34 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:25.719302 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-eeffa1f5-f37a-436b-9092-3c20c77bec34 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '53af488f3b0242a797f539284b78d59e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bb1ede0d0b6941cc97278e6c617f9b01', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:14:25.727447 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-eeffa1f5-f37a-436b-9092-3c20c77bec34 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] 10.222.0.16 "GET /compute/v2.1/servers/69cabe57-da71-4a02-b4ee-c0d84b3d32a2" status: 200 len: 1354 microversion: 2.1 time: 0.731725 Jul 16 21:14:25.729109 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 723/1511] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:14:24 2022] GET /compute/v2.1/servers/69cabe57-da71-4a02-b4ee-c0d84b3d32a2 => generated 1354 bytes in 734 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:14:25.744811 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-fcbc9def-968a-4e99-8ba4-492f4246337a tempest-FloatingIPsTestJSON-1267052927 tempest-FloatingIPsTestJSON-1267052927-project] Calling method '.version_select at 0x7fdd9d9ffee0>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:14:26.390935 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-fcbc9def-968a-4e99-8ba4-492f4246337a tempest-FloatingIPsTestJSON-1267052927 tempest-FloatingIPsTestJSON-1267052927-project] 10.222.0.16 "GET /compute/v2.1/os-floating-ips" status: 200 len: 145 microversion: 2.1 time: 0.660418 Jul 16 21:14:26.391633 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 724/1512] 10.222.0.16 () {58 vars in 1148 bytes} [Sat Jul 16 21:14:25 2022] GET /compute/v2.1/os-floating-ips => generated 145 bytes in 662 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:14:26.396358 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-8ae97fe0-b074-4163-8e0f-07ac58189751 req-9f18d8bb-534f-4453-b762-60ad64c5ecf3 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "76ee9d21-1f17-4b64-8f95-95a33e70e907", "name": "network-vif-plugged", "status": "completed", "tag": "65068e4e-def5-46ec-95ec-de961e8c1e0e"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:14:26.407423 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-8ae97fe0-b074-4163-8e0f-07ac58189751 req-9f18d8bb-534f-4453-b762-60ad64c5ecf3 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:26.407878 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-8ae97fe0-b074-4163-8e0f-07ac58189751 req-9f18d8bb-534f-4453-b762-60ad64c5ecf3 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:26.420140 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-8ae97fe0-b074-4163-8e0f-07ac58189751 req-9f18d8bb-534f-4453-b762-60ad64c5ecf3 service nova] Creating event network-vif-plugged:65068e4e-def5-46ec-95ec-de961e8c1e0e for instance 76ee9d21-1f17-4b64-8f95-95a33e70e907 on nch2-master-1 Jul 16 21:14:26.425375 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-8ae97fe0-b074-4163-8e0f-07ac58189751 req-9f18d8bb-534f-4453-b762-60ad64c5ecf3 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.032565 Jul 16 21:14:26.425983 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 725/1513] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:14:26 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:14:26.432883 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-e464623e-b08e-41c7-92b9-3a3834a4efb8 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:14:26.434687 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-e464623e-b08e-41c7-92b9-3a3834a4efb8 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: f98a9762-a67c-4e95-85b8-727c81bdb03a] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:14:26.452323 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e464623e-b08e-41c7-92b9-3a3834a4efb8 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:26.452775 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e464623e-b08e-41c7-92b9-3a3834a4efb8 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:26.454200 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e464623e-b08e-41c7-92b9-3a3834a4efb8 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:26.459762 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e464623e-b08e-41c7-92b9-3a3834a4efb8 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:26.486862 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-e464623e-b08e-41c7-92b9-3a3834a4efb8 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] HTTP exception thrown: Instance f98a9762-a67c-4e95-85b8-727c81bdb03a could not be found. Jul 16 21:14:26.488971 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-e464623e-b08e-41c7-92b9-3a3834a4efb8 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Request method failure captured: Jul 16 21:14:26.488971 ncdv-master-1 devstack@n-api.service[138681]: request: GET /compute/v2.1/servers/f98a9762-a67c-4e95-85b8-727c81bdb03a HTTP/1.1 Jul 16 21:14:26.488971 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:14:26.488971 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:14:26.488971 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:14:26.488971 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 21:14:26.488971 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:14:26.488971 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:14:26.488971 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:14:26.488971 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:14:26.488971 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0yjS0OwquTdxFvFwnqD6oXhLiVMC2H2T0PTTAPBs5CbYhBauxUQKqaIu70UDpOvDtiTqvXU4-VL9hTewWDu4Gh7O9lNbGbnPrucyL4QpaGXDfvsW3rocl0tQjubLkrzEuiaQ0dP8qaKxkSk27mrRw2WAWCg3PheG3BhIZr0IEKSfAB4 Jul 16 21:14:26.488971 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:14:26.488971 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:14:26.488971 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:14:26.488971 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:14:26.488971 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:14:26.488971 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:14:26.488971 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: 55a19b7a4682457a9132f5fc80110712 Jul 16 21:14:26.488971 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-MigrationsAdminTest-116332175 Jul 16 21:14:26.488971 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:14:26.488971 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:14:26.491618 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/55a19b7a4682457a9132f5fc80110712"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/55a19b7a4682457a9132f5fc80110712"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_55a19b7a4682457a9132f5fc80110712"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/55a19b7a4682457a9132f5fc80110712"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:14:26.491618 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-MigrationsAdminTest-116332175 Jul 16 21:14:26.491618 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: 55a19b7a4682457a9132f5fc80110712 Jul 16 21:14:26.491618 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-MigrationsAdminTest-116332175 Jul 16 21:14:26.491618 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-MigrationsAdminTest-116332175-project Jul 16 21:14:26.491618 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:14:26.491618 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:14:26.491618 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: a9b68dacc4b8463ba07fae6372131117 Jul 16 21:14:26.491618 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-MigrationsAdminTest-116332175-project Jul 16 21:14:26.491618 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:14:26.491618 ncdv-master-1 devstack@n-api.service[138681]: exception: Instance f98a9762-a67c-4e95-85b8-727c81bdb03a could not be found. Jul 16 21:14:26.491618 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': 'f98a9762-a67c-4e95-85b8-727c81bdb03a'} Jul 16 21:14:26.492678 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:14:26.492678 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:14:26.492678 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:14:26.492678 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:14:26.492678 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:14:26.492678 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:14:26.492678 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:14:26.492678 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:14:26.492678 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:14:26.492678 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:14:26.492678 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance f98a9762-a67c-4e95-85b8-727c81bdb03a could not be found. Jul 16 21:14:26.492678 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:14:26.492678 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:14:26.492678 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:14:26.492678 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:14:26.492678 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:14:26.492678 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:14:26.492678 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:14:26.492678 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:14:26.492678 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:14:26.494152 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:14:26.494152 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:14:26.494152 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:14:26.494152 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:14:26.494152 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:14:26.494152 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:14:26.494152 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:14:26.494152 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance f98a9762-a67c-4e95-85b8-727c81bdb03a could not be found. Jul 16 21:14:26.494152 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:14:26.494152 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:14:26.494152 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:14:26.494152 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:14:26.494152 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:14:26.494152 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:14:26.494152 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:14:26.494152 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:14:26.494152 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance f98a9762-a67c-4e95-85b8-727c81bdb03a could not be found. Jul 16 21:14:26.494152 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:14:26.495474 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-e464623e-b08e-41c7-92b9-3a3834a4efb8 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Returning 404 to user: Instance f98a9762-a67c-4e95-85b8-727c81bdb03a could not be found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:14:26.497294 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-e464623e-b08e-41c7-92b9-3a3834a4efb8 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "GET /compute/v2.1/servers/f98a9762-a67c-4e95-85b8-727c81bdb03a" status: 404 len: 111 microversion: 2.1 time: 0.069728 Jul 16 21:14:26.502847 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 726/1514] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:14:26 2022] GET /compute/v2.1/servers/f98a9762-a67c-4e95-85b8-727c81bdb03a => generated 111 bytes in 71 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:14:26.506205 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-c210c8b3-a168-4a00-8747-7e2a28026af0 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:14:26.507276 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-c210c8b3-a168-4a00-8747-7e2a28026af0 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] [instance: e66baae9-d488-46a4-8347-623030fc8b69] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:14:26.515168 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c210c8b3-a168-4a00-8747-7e2a28026af0 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:26.515627 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c210c8b3-a168-4a00-8747-7e2a28026af0 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:26.516416 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c210c8b3-a168-4a00-8747-7e2a28026af0 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:26.516860 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c210c8b3-a168-4a00-8747-7e2a28026af0 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:26.813127 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-817d7a73-ce2c-4d1a-bcff-f34e79c42e62 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] 10.222.0.16 "POST /compute/v2.1/os-floating-ips" status: 200 len: 141 microversion: 2.1 time: 5.031518 Jul 16 21:14:26.824851 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 789/1515] 10.222.0.16 () {60 vars in 1169 bytes} [Sat Jul 16 21:14:21 2022] POST /compute/v2.1/os-floating-ips => generated 141 bytes in 5040 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:14:26.826678 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-35096d54-dd02-4b2e-a4f0-97990b2f79ab tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:14:26.827499 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-35096d54-dd02-4b2e-a4f0-97990b2f79ab tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] [instance: 76ee9d21-1f17-4b64-8f95-95a33e70e907] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:14:26.835282 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-35096d54-dd02-4b2e-a4f0-97990b2f79ab tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:26.835781 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-35096d54-dd02-4b2e-a4f0-97990b2f79ab tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:26.836712 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-35096d54-dd02-4b2e-a4f0-97990b2f79ab tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:26.837554 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-35096d54-dd02-4b2e-a4f0-97990b2f79ab tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:26.957418 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-c210c8b3-a168-4a00-8747-7e2a28026af0 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4e9df82f95ac4779859972403869096a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4337575622d04a26942905c9dc044ba7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:14:26.966890 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-c210c8b3-a168-4a00-8747-7e2a28026af0 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] 10.222.0.16 "GET /compute/v2.1/servers/e66baae9-d488-46a4-8347-623030fc8b69" status: 200 len: 1520 microversion: 2.1 time: 0.467649 Jul 16 21:14:26.967518 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 727/1516] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:14:26 2022] GET /compute/v2.1/servers/e66baae9-d488-46a4-8347-623030fc8b69 => generated 1520 bytes in 469 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:14:26.972418 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-3cbb55d5-1462-45ce-ba3e-a035c8affa85 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:14:26.973031 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-3cbb55d5-1462-45ce-ba3e-a035c8affa85 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] [instance: 3c5ab09e-0a97-4cfc-b249-a11ab221b5d2] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:14:26.981345 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3cbb55d5-1462-45ce-ba3e-a035c8affa85 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:26.981851 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3cbb55d5-1462-45ce-ba3e-a035c8affa85 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:26.982628 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3cbb55d5-1462-45ce-ba3e-a035c8affa85 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:26.983095 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3cbb55d5-1462-45ce-ba3e-a035c8affa85 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:27.213757 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-35096d54-dd02-4b2e-a4f0-97990b2f79ab tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c1aea87154224f1c81428ddef4438801', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45d9edd2fe854483a6f0d676efbd6622', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:14:27.223414 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-35096d54-dd02-4b2e-a4f0-97990b2f79ab tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] 10.222.0.16 "GET /compute/v2.1/servers/76ee9d21-1f17-4b64-8f95-95a33e70e907" status: 200 len: 1322 microversion: 2.1 time: 0.401060 Jul 16 21:14:27.224280 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 790/1517] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:14:26 2022] GET /compute/v2.1/servers/76ee9d21-1f17-4b64-8f95-95a33e70e907 => generated 1322 bytes in 403 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:14:27.238799 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-701adbf9-9ac0-49bd-a470-7d1649f3c122 req-7a0eb6fc-8d53-4650-980d-a7f5e0b3413b service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "f98a9762-a67c-4e95-85b8-727c81bdb03a", "name": "network-vif-deleted", "tag": "b0aae229-268b-497f-8da0-36ccb43a67bd"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:14:27.248707 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-701adbf9-9ac0-49bd-a470-7d1649f3c122 req-7a0eb6fc-8d53-4650-980d-a7f5e0b3413b service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:27.249161 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-701adbf9-9ac0-49bd-a470-7d1649f3c122 req-7a0eb6fc-8d53-4650-980d-a7f5e0b3413b service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:27.264876 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-3cbb55d5-1462-45ce-ba3e-a035c8affa85 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:14:27.270336 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-701adbf9-9ac0-49bd-a470-7d1649f3c122 req-7a0eb6fc-8d53-4650-980d-a7f5e0b3413b service nova] Creating event network-vif-deleted:b0aae229-268b-497f-8da0-36ccb43a67bd for instance f98a9762-a67c-4e95-85b8-727c81bdb03a on nch1-master-1 Jul 16 21:14:27.276527 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-3cbb55d5-1462-45ce-ba3e-a035c8affa85 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:14:27.282492 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-3cbb55d5-1462-45ce-ba3e-a035c8affa85 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:14:27.288191 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-3cbb55d5-1462-45ce-ba3e-a035c8affa85 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] 10.222.0.16 "GET /compute/v2.1/servers/3c5ab09e-0a97-4cfc-b249-a11ab221b5d2" status: 200 len: 1799 microversion: 2.25 time: 0.315317 Jul 16 21:14:27.288191 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 728/1518] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:14:26 2022] GET /compute/v2.1/servers/3c5ab09e-0a97-4cfc-b249-a11ab221b5d2 => generated 1799 bytes in 316 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:14:27.289418 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-701adbf9-9ac0-49bd-a470-7d1649f3c122 req-7a0eb6fc-8d53-4650-980d-a7f5e0b3413b service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.054071 Jul 16 21:14:27.290192 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 791/1519] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:14:27 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:14:27.295385 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-e5b96e09-3dda-42d1-98ac-588c4b1340b0 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:14:27.295979 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-e5b96e09-3dda-42d1-98ac-588c4b1340b0 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: 1703f309-9e2e-4c5f-8588-eb4e43f21d91] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:14:27.301729 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-a04f8efe-9523-4727-ab0b-f9fc3547a204 tempest-FloatingIPsTestJSON-1267052927 tempest-FloatingIPsTestJSON-1267052927-project] Calling method '.version_select at 0x7fdd9d233430>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:14:27.302490 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e5b96e09-3dda-42d1-98ac-588c4b1340b0 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:27.303712 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e5b96e09-3dda-42d1-98ac-588c4b1340b0 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:27.304503 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e5b96e09-3dda-42d1-98ac-588c4b1340b0 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:27.305225 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e5b96e09-3dda-42d1-98ac-588c4b1340b0 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:27.333678 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-e5b96e09-3dda-42d1-98ac-588c4b1340b0 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: 1703f309-9e2e-4c5f-8588-eb4e43f21d91] Going to try to terminate instance {{(pid=138680) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 16 21:14:27.786548 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-e5b96e09-3dda-42d1-98ac-588c4b1340b0 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "DELETE /compute/v2.1/servers/1703f309-9e2e-4c5f-8588-eb4e43f21d91" status: 204 len: 0 microversion: 2.1 time: 0.494777 Jul 16 21:14:27.787833 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 792/1520] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:14:27 2022] DELETE /compute/v2.1/servers/1703f309-9e2e-4c5f-8588-eb4e43f21d91 => generated 0 bytes in 497 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 16 21:14:27.794507 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-427821f7-0f5e-4785-ab17-1b2818ed211b tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:14:27.795417 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-427821f7-0f5e-4785-ab17-1b2818ed211b tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] [instance: 69cabe57-da71-4a02-b4ee-c0d84b3d32a2] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:14:27.801619 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-427821f7-0f5e-4785-ab17-1b2818ed211b tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:27.802276 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-427821f7-0f5e-4785-ab17-1b2818ed211b tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:27.827247 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-427821f7-0f5e-4785-ab17-1b2818ed211b tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:27.827247 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-427821f7-0f5e-4785-ab17-1b2818ed211b tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:28.025691 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-427821f7-0f5e-4785-ab17-1b2818ed211b tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '53af488f3b0242a797f539284b78d59e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bb1ede0d0b6941cc97278e6c617f9b01', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:14:28.033282 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-427821f7-0f5e-4785-ab17-1b2818ed211b tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] 10.222.0.16 "GET /compute/v2.1/servers/69cabe57-da71-4a02-b4ee-c0d84b3d32a2" status: 200 len: 1354 microversion: 2.1 time: 0.243371 Jul 16 21:14:28.033872 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 793/1521] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:14:27 2022] GET /compute/v2.1/servers/69cabe57-da71-4a02-b4ee-c0d84b3d32a2 => generated 1354 bytes in 245 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:14:28.039395 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-adf3e9b9-ca08-495f-9fdb-6aad9d91391b tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] Action: 'action', calling method: , body: {"addFloatingIp": {"address": "172.35.1.90"}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:14:28.040550 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-adf3e9b9-ca08-495f-9fdb-6aad9d91391b tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] [instance: dada2ea0-2ef4-4a30-8e2d-40177d0bafce] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:14:28.046524 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-adf3e9b9-ca08-495f-9fdb-6aad9d91391b tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:28.055338 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-adf3e9b9-ca08-495f-9fdb-6aad9d91391b tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:28.056248 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-adf3e9b9-ca08-495f-9fdb-6aad9d91391b tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:28.056710 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-adf3e9b9-ca08-495f-9fdb-6aad9d91391b tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:28.081924 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-adf3e9b9-ca08-495f-9fdb-6aad9d91391b tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] Acquired lock "refresh_cache-dada2ea0-2ef4-4a30-8e2d-40177d0bafce" {{(pid=138680) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:294}} Jul 16 21:14:28.282158 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-a04f8efe-9523-4727-ab0b-f9fc3547a204 tempest-FloatingIPsTestJSON-1267052927 tempest-FloatingIPsTestJSON-1267052927-project] 10.222.0.16 "DELETE /compute/v2.1/os-floating-ips/837b239a-219d-4f51-8490-ee6de4391d9f" status: 202 len: 0 microversion: 2.1 time: 0.984199 Jul 16 21:14:28.288391 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 729/1522] 10.222.0.16 () {58 vars in 1262 bytes} [Sat Jul 16 21:14:27 2022] DELETE /compute/v2.1/os-floating-ips/837b239a-219d-4f51-8490-ee6de4391d9f => generated 0 bytes in 998 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 21:14:28.292322 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-2e21b41e-ad74-4ab7-a3cd-f69ee1c0a3e9 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:14:28.293511 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-2e21b41e-ad74-4ab7-a3cd-f69ee1c0a3e9 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] [instance: e66baae9-d488-46a4-8347-623030fc8b69] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:14:28.304803 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2e21b41e-ad74-4ab7-a3cd-f69ee1c0a3e9 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:28.305323 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2e21b41e-ad74-4ab7-a3cd-f69ee1c0a3e9 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:28.306279 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2e21b41e-ad74-4ab7-a3cd-f69ee1c0a3e9 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:28.306733 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2e21b41e-ad74-4ab7-a3cd-f69ee1c0a3e9 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:28.948308 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-2e21b41e-ad74-4ab7-a3cd-f69ee1c0a3e9 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4e9df82f95ac4779859972403869096a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4337575622d04a26942905c9dc044ba7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:14:28.996226 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-2e21b41e-ad74-4ab7-a3cd-f69ee1c0a3e9 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] 10.222.0.16 "GET /compute/v2.1/servers/e66baae9-d488-46a4-8347-623030fc8b69" status: 200 len: 1520 microversion: 2.1 time: 0.674634 Jul 16 21:14:28.996226 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 730/1523] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:14:28 2022] GET /compute/v2.1/servers/e66baae9-d488-46a4-8347-623030fc8b69 => generated 1520 bytes in 688 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:14:28.996981 ncdv-master-1 devstack@n-api.service[138680]: DEBUG neutronclient.v2_0.client [None req-adf3e9b9-ca08-495f-9fdb-6aad9d91391b tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] Error message: {"NeutronError": {"type": "FloatingIPPortAlreadyAssociated", "message": "Cannot associate floating IP 172.35.1.90 (8ed275be-2d70-481f-bb5c-b19cb27bdb30) with port b63bf933-a84e-4e18-98ed-0e772979bec5 using fixed IP 172.20.0.13, as that fixed IP already has a floating IP on external network f39f47fa-8d3d-4a2f-9486-cc28d1bbfe66.", "detail": ""}} {{(pid=138680) _handle_fault_response /usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py:262}} Jul 16 21:14:28.996981 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-adf3e9b9-ca08-495f-9fdb-6aad9d91391b tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] Releasing lock "refresh_cache-dada2ea0-2ef4-4a30-8e2d-40177d0bafce" {{(pid=138680) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:312}} Jul 16 21:14:28.996981 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-adf3e9b9-ca08-495f-9fdb-6aad9d91391b tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] HTTP exception thrown: Cannot associate floating IP 172.35.1.90 (8ed275be-2d70-481f-bb5c-b19cb27bdb30) with port b63bf933-a84e-4e18-98ed-0e772979bec5 using fixed IP 172.20.0.13, as that fixed IP already has a floating IP on external network f39f47fa-8d3d-4a2f-9486-cc28d1bbfe66. Jul 16 21:14:28.996981 ncdv-master-1 devstack@n-api.service[138680]: Neutron server returns request_ids: ['req-1df156b5-8f51-4b90-a551-c6c8708615a5'] Jul 16 21:14:28.997749 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-adf3e9b9-ca08-495f-9fdb-6aad9d91391b tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] Request method failure captured: Jul 16 21:14:28.997749 ncdv-master-1 devstack@n-api.service[138680]: request: POST /compute/v2.1/servers/dada2ea0-2ef4-4a30-8e2d-40177d0bafce/action HTTP/1.1 Jul 16 21:14:28.997749 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:14:28.997749 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:14:28.997749 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:14:28.997749 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 45 Jul 16 21:14:28.997749 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:14:28.997749 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:14:28.997749 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:14:28.997749 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:14:28.997749 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0yoB0XYghSUBtF84YOYGfB4_xYzUA_F4WrRobF-Rt4qMvE82h1iXAkIM12_RAx3rpkycxMLJn3NNCBruBkg08dFc8vTeMt2laJatIrAsrctW_jXBNb9rR0ETfgDp7wIjwtKtViFNgoI7i99au4p9qB9nlk-sgKFBAUCsSzgT232xMd8 Jul 16 21:14:28.997749 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:14:28.997749 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:14:28.997749 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:14:28.997749 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:14:28.997749 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:14:28.997749 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:14:28.997749 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: 6c2095f96f8d45bb9e62c517678c3dcd Jul 16 21:14:28.997749 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 Jul 16 21:14:28.997749 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:14:28.997749 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:14:28.999655 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/6c2095f96f8d45bb9e62c517678c3dcd"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/6c2095f96f8d45bb9e62c517678c3dcd"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_6c2095f96f8d45bb9e62c517678c3dcd"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/6c2095f96f8d45bb9e62c517678c3dcd"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:14:28.999655 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 Jul 16 21:14:28.999655 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: 6c2095f96f8d45bb9e62c517678c3dcd Jul 16 21:14:28.999655 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 Jul 16 21:14:28.999655 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project Jul 16 21:14:28.999655 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:14:28.999655 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:14:28.999655 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: 2eba2ac21c344394a7d3f605775b8713 Jul 16 21:14:28.999655 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project Jul 16 21:14:28.999655 ncdv-master-1 devstack@n-api.service[138680]: Jul 16 21:14:28.999655 ncdv-master-1 devstack@n-api.service[138680]: {"addFloatingIp": {"address": "172.35.1.90"}} Jul 16 21:14:28.999655 ncdv-master-1 devstack@n-api.service[138680]: method: Jul 16 21:14:29.000656 ncdv-master-1 devstack@n-api.service[138680]: exception: Cannot associate floating IP 172.35.1.90 (8ed275be-2d70-481f-bb5c-b19cb27bdb30) with port b63bf933-a84e-4e18-98ed-0e772979bec5 using fixed IP 172.20.0.13, as that fixed IP already has a floating IP on external network f39f47fa-8d3d-4a2f-9486-cc28d1bbfe66. Jul 16 21:14:29.000656 ncdv-master-1 devstack@n-api.service[138680]: Neutron server returns request_ids: ['req-1df156b5-8f51-4b90-a551-c6c8708615a5'] Jul 16 21:14:29.000656 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': 'dada2ea0-2ef4-4a30-8e2d-40177d0bafce', 'body': {'addFloatingIp': {'address': '172.35.1.90'}}} Jul 16 21:14:29.000656 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:14:29.000656 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:14:29.000656 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 2706, in associate_floating_ip Jul 16 21:14:29.000656 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi client.update_floatingip(fip['id'], {'floatingip': param}) Jul 16 21:14:29.000656 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 197, in wrapper Jul 16 21:14:29.000656 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Jul 16 21:14:29.000656 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 1014, in update_floatingip Jul 16 21:14:29.000656 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return self._update_resource(self.floatingip_path % (floatingip), Jul 16 21:14:29.000656 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 197, in wrapper Jul 16 21:14:29.000656 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Jul 16 21:14:29.000656 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 2520, in _update_resource Jul 16 21:14:29.000656 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return self.put(path, **kwargs) Jul 16 21:14:29.000656 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 197, in wrapper Jul 16 21:14:29.000656 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Jul 16 21:14:29.002027 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 365, in put Jul 16 21:14:29.002027 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return self.retry_request("PUT", action, body=body, Jul 16 21:14:29.002027 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 197, in wrapper Jul 16 21:14:29.002027 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Jul 16 21:14:29.002027 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 333, in retry_request Jul 16 21:14:29.002027 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return self.do_request(method, action, body=body, Jul 16 21:14:29.002027 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 197, in wrapper Jul 16 21:14:29.002027 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Jul 16 21:14:29.002027 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 297, in do_request Jul 16 21:14:29.002027 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi self._handle_fault_response(status_code, replybody, resp) Jul 16 21:14:29.002027 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 197, in wrapper Jul 16 21:14:29.002027 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Jul 16 21:14:29.002027 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 272, in _handle_fault_response Jul 16 21:14:29.002027 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi exception_handler_v20(status_code, error_body) Jul 16 21:14:29.002027 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 90, in exception_handler_v20 Jul 16 21:14:29.002027 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise client_exc(message=error_message, Jul 16 21:14:29.003525 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi neutronclient.common.exceptions.Conflict: Cannot associate floating IP 172.35.1.90 (8ed275be-2d70-481f-bb5c-b19cb27bdb30) with port b63bf933-a84e-4e18-98ed-0e772979bec5 using fixed IP 172.20.0.13, as that fixed IP already has a floating IP on external network f39f47fa-8d3d-4a2f-9486-cc28d1bbfe66. Jul 16 21:14:29.003525 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-1df156b5-8f51-4b90-a551-c6c8708615a5'] Jul 16 21:14:29.003525 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:14:29.003525 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:14:29.003525 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:14:29.003525 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:14:29.003525 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/floating_ips.py", line 239, in _add_floating_ip Jul 16 21:14:29.003525 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi self.network_api.associate_floating_ip(context, instance, Jul 16 21:14:29.003525 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 166, in wrapper Jul 16 21:14:29.003525 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi res = f(self, context, *args, **kwargs) Jul 16 21:14:29.003525 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 2708, in associate_floating_ip Jul 16 21:14:29.003525 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.FloatingIpAssociateFailed(str(e)) Jul 16 21:14:29.003525 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.FloatingIpAssociateFailed: Cannot associate floating IP 172.35.1.90 (8ed275be-2d70-481f-bb5c-b19cb27bdb30) with port b63bf933-a84e-4e18-98ed-0e772979bec5 using fixed IP 172.20.0.13, as that fixed IP already has a floating IP on external network f39f47fa-8d3d-4a2f-9486-cc28d1bbfe66. Jul 16 21:14:29.003525 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-1df156b5-8f51-4b90-a551-c6c8708615a5'] Jul 16 21:14:29.003525 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:14:29.003525 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:14:29.005446 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:14:29.005446 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:14:29.005446 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:14:29.005446 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:14:29.005446 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:14:29.005446 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:14:29.005446 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Jul 16 21:14:29.005446 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Jul 16 21:14:29.005446 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:14:29.005446 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:14:29.005446 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 16 21:14:29.005446 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 16 21:14:29.005446 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/floating_ips.py", line 246, in _add_floating_ip Jul 16 21:14:29.005446 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPBadRequest(explanation=e.format_message()) Jul 16 21:14:29.005446 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Cannot associate floating IP 172.35.1.90 (8ed275be-2d70-481f-bb5c-b19cb27bdb30) with port b63bf933-a84e-4e18-98ed-0e772979bec5 using fixed IP 172.20.0.13, as that fixed IP already has a floating IP on external network f39f47fa-8d3d-4a2f-9486-cc28d1bbfe66. Jul 16 21:14:29.005446 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-1df156b5-8f51-4b90-a551-c6c8708615a5'] Jul 16 21:14:29.005446 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:14:29.005446 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:14:29.007093 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:14:29.007093 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:14:29.007093 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:14:29.007093 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:14:29.007093 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:14:29.007093 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:14:29.007093 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Cannot associate floating IP 172.35.1.90 (8ed275be-2d70-481f-bb5c-b19cb27bdb30) with port b63bf933-a84e-4e18-98ed-0e772979bec5 using fixed IP 172.20.0.13, as that fixed IP already has a floating IP on external network f39f47fa-8d3d-4a2f-9486-cc28d1bbfe66. Jul 16 21:14:29.007093 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-1df156b5-8f51-4b90-a551-c6c8708615a5'] Jul 16 21:14:29.007093 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:14:29.007093 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-adf3e9b9-ca08-495f-9fdb-6aad9d91391b tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] Returning 400 to user: Cannot associate floating IP 172.35.1.90 (8ed275be-2d70-481f-bb5c-b19cb27bdb30) with port b63bf933-a84e-4e18-98ed-0e772979bec5 using fixed IP 172.20.0.13, as that fixed IP already has a floating IP on external network f39f47fa-8d3d-4a2f-9486-cc28d1bbfe66. Jul 16 21:14:29.007093 ncdv-master-1 devstack@n-api.service[138680]: Neutron server returns request_ids: ['req-1df156b5-8f51-4b90-a551-c6c8708615a5'] {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:14:29.011085 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-adf3e9b9-ca08-495f-9fdb-6aad9d91391b tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] 10.222.0.16 "POST /compute/v2.1/servers/dada2ea0-2ef4-4a30-8e2d-40177d0bafce/action" status: 400 len: 381 microversion: 2.1 time: 0.953092 Jul 16 21:14:29.011085 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 794/1524] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 21:14:28 2022] POST /compute/v2.1/servers/dada2ea0-2ef4-4a30-8e2d-40177d0bafce/action => generated 381 bytes in 955 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Jul 16 21:14:29.012326 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-aac4ffe6-e277-4d66-b5fe-b327dc8ff36b tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] Calling method '.version_select at 0x7fdd9d4f58b0>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:14:29.012326 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-aac4ffe6-e277-4d66-b5fe-b327dc8ff36b tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:29.012539 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-9a9902a0-7045-4312-8b86-783a0ebce24d tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:14:29.012641 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-aac4ffe6-e277-4d66-b5fe-b327dc8ff36b tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:29.012741 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-9a9902a0-7045-4312-8b86-783a0ebce24d tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: 1703f309-9e2e-4c5f-8588-eb4e43f21d91] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:14:29.013573 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9a9902a0-7045-4312-8b86-783a0ebce24d tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:29.014034 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9a9902a0-7045-4312-8b86-783a0ebce24d tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:29.014995 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9a9902a0-7045-4312-8b86-783a0ebce24d tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:29.015469 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9a9902a0-7045-4312-8b86-783a0ebce24d tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:29.018646 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-aac4ffe6-e277-4d66-b5fe-b327dc8ff36b tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] 10.222.0.16 "GET /compute/v2.1/os-migrations" status: 200 len: 3207 microversion: 2.25 time: 0.045969 Jul 16 21:14:29.019742 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 731/1525] 10.222.0.16 () {60 vars in 1183 bytes} [Sat Jul 16 21:14:28 2022] GET /compute/v2.1/os-migrations => generated 3207 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:14:29.025452 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-89c7c23d-d9b0-4d76-8ac2-e55c1f7756a0 tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:14:29.026191 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-89c7c23d-d9b0-4d76-8ac2-e55c1f7756a0 tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] [instance: 76ee9d21-1f17-4b64-8f95-95a33e70e907] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:14:29.032897 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-89c7c23d-d9b0-4d76-8ac2-e55c1f7756a0 tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:29.034978 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-89c7c23d-d9b0-4d76-8ac2-e55c1f7756a0 tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:29.034978 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-89c7c23d-d9b0-4d76-8ac2-e55c1f7756a0 tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:29.034978 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-89c7c23d-d9b0-4d76-8ac2-e55c1f7756a0 tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:29.224365 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-9a9902a0-7045-4312-8b86-783a0ebce24d tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b68dacc4b8463ba07fae6372131117', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55a19b7a4682457a9132f5fc80110712', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:14:29.232459 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-9a9902a0-7045-4312-8b86-783a0ebce24d tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "GET /compute/v2.1/servers/1703f309-9e2e-4c5f-8588-eb4e43f21d91" status: 200 len: 1467 microversion: 2.1 time: 0.242059 Jul 16 21:14:29.233224 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 795/1526] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:14:28 2022] GET /compute/v2.1/servers/1703f309-9e2e-4c5f-8588-eb4e43f21d91 => generated 1467 bytes in 243 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:14:29.237904 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-fcf34f38-9973-4334-98d4-251cf712f030 tempest-FloatingIPsTestJSON-1267052927 tempest-FloatingIPsTestJSON-1267052927-project] Action: 'create', calling method: .version_select at 0x7fdd9d434790>, body: {"pool": null} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:14:29.238169 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-89c7c23d-d9b0-4d76-8ac2-e55c1f7756a0 tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c1aea87154224f1c81428ddef4438801', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45d9edd2fe854483a6f0d676efbd6622', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:14:29.249656 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-89c7c23d-d9b0-4d76-8ac2-e55c1f7756a0 tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] 10.222.0.16 "GET /compute/v2.1/servers/76ee9d21-1f17-4b64-8f95-95a33e70e907" status: 200 len: 1471 microversion: 2.1 time: 0.228661 Jul 16 21:14:29.250322 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 732/1527] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:14:29 2022] GET /compute/v2.1/servers/76ee9d21-1f17-4b64-8f95-95a33e70e907 => generated 1471 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:14:29.257122 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-42f0b186-ad7d-477b-b764-832b21c61bb3 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Action: 'action', calling method: >, body: {"createImage": {"name": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:14:29.260390 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-42f0b186-ad7d-477b-b764-832b21c61bb3 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Request method failure captured: Jul 16 21:14:29.260390 ncdv-master-1 devstack@n-api.service[138681]: request: POST /compute/v2.1/servers/e66baae9-d488-46a4-8347-623030fc8b69/action HTTP/1.1 Jul 16 21:14:29.260390 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:14:29.260390 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:14:29.260390 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:14:29.260390 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 285 Jul 16 21:14:29.260390 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:14:29.260390 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:14:29.260390 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:14:29.260390 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:14:29.260390 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0yoJFUAf_8Wtq8fI-wZdJwoKyg8rUEOC_eBMeqcRE2hk6JM7_ssDplugQwmk2S9xH7Oys7Ho2tw0kc9hmlY-jX_rR-I-9WVwQYFbyogU7ki99siEGGR9aVA2GKsp5nNcVHXBtjM3bI_zfa21XSWedvaSIVwZV8kEGixKYsBf2GMYKVI Jul 16 21:14:29.260390 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:14:29.260390 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:14:29.260390 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:14:29.260390 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:14:29.260390 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:14:29.260390 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:14:29.260390 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: 4337575622d04a26942905c9dc044ba7 Jul 16 21:14:29.260390 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-ImagesOneServerNegativeTestJSON-526509748 Jul 16 21:14:29.260390 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:14:29.260390 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:14:29.262321 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/4337575622d04a26942905c9dc044ba7"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/4337575622d04a26942905c9dc044ba7"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_4337575622d04a26942905c9dc044ba7"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/4337575622d04a26942905c9dc044ba7"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:14:29.262321 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-ImagesOneServerNegativeTestJSON-526509748 Jul 16 21:14:29.262321 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: 4337575622d04a26942905c9dc044ba7 Jul 16 21:14:29.262321 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-ImagesOneServerNegativeTestJSON-526509748 Jul 16 21:14:29.262321 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-ImagesOneServerNegativeTestJSON-526509748-project Jul 16 21:14:29.262321 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:14:29.262321 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:14:29.262321 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: 4e9df82f95ac4779859972403869096a Jul 16 21:14:29.262321 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-ImagesOneServerNegativeTestJSON-526509748-project Jul 16 21:14:29.262321 ncdv-master-1 devstack@n-api.service[138681]: Jul 16 21:14:29.262321 ncdv-master-1 devstack@n-api.service[138681]: {"createImage": {"name": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"}} Jul 16 21:14:29.263150 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:14:29.263150 ncdv-master-1 devstack@n-api.service[138681]: exception: Invalid input for field/attribute name. Value: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa. 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' is too long Jul 16 21:14:29.263150 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': 'e66baae9-d488-46a4-8347-623030fc8b69', 'body': {'createImage': {'name': 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa'}}} Jul 16 21:14:29.263150 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:14:29.263150 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:14:29.263150 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 279, in validate Jul 16 21:14:29.263150 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Jul 16 21:14:29.263150 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/lib/python3/dist-packages/jsonschema/validators.py", line 353, in validate Jul 16 21:14:29.263150 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise error Jul 16 21:14:29.263150 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' is too long Jul 16 21:14:29.263837 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:14:29.263837 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Failed validating 'maxLength' in schema['properties']['createImage']['properties']['name']: Jul 16 21:14:29.263837 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi {'format': 'name', 'maxLength': 255, 'minLength': 1, 'type': 'string'} Jul 16 21:14:29.263837 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:14:29.263837 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi On instance['createImage']: Jul 16 21:14:29.263837 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' Jul 16 21:14:29.263837 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:14:29.263837 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:14:29.263837 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:14:29.263837 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:14:29.263837 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:14:29.263837 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:14:29.263837 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:14:29.263837 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:14:29.263837 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:14:29.263837 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:14:29.263837 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 16 21:14:29.263837 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 16 21:14:29.263837 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Jul 16 21:14:29.263837 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Jul 16 21:14:29.265159 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Jul 16 21:14:29.265159 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Jul 16 21:14:29.265159 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 308, in validate Jul 16 21:14:29.265159 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Jul 16 21:14:29.265159 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute name. Value: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa. 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' is too long Jul 16 21:14:29.265159 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:14:29.265159 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:14:29.265159 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:14:29.265159 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:14:29.265159 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:14:29.265159 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:14:29.265159 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Jul 16 21:14:29.265159 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Jul 16 21:14:29.265960 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute name. Value: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa. 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' is too long Jul 16 21:14:29.265960 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:14:29.265960 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-42f0b186-ad7d-477b-b764-832b21c61bb3 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Returning 400 to user: Invalid input for field/attribute name. Value: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa. 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' is too long {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:14:29.265960 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-42f0b186-ad7d-477b-b764-832b21c61bb3 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] 10.222.0.16 "POST /compute/v2.1/servers/e66baae9-d488-46a4-8347-623030fc8b69/action" status: 400 len: 619 microversion: 2.1 time: 0.011635 Jul 16 21:14:29.266268 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 733/1528] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Jul 16 21:14:29 2022] POST /compute/v2.1/servers/e66baae9-d488-46a4-8347-623030fc8b69/action => generated 619 bytes in 13 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Jul 16 21:14:29.269457 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-c6ff0487-55c3-4021-a949-1f9506f0bd1e tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] Calling method '.version_select at 0x7fdd9d7a3940>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:14:30.993098 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-c6ff0487-55c3-4021-a949-1f9506f0bd1e tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] 10.222.0.16 "DELETE /compute/v2.1/os-floating-ips/8ed275be-2d70-481f-bb5c-b19cb27bdb30" status: 202 len: 0 microversion: 2.1 time: 1.727117 Jul 16 21:14:30.993726 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 734/1529] 10.222.0.16 () {58 vars in 1262 bytes} [Sat Jul 16 21:14:29 2022] DELETE /compute/v2.1/os-floating-ips/8ed275be-2d70-481f-bb5c-b19cb27bdb30 => generated 0 bytes in 1729 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 21:14:31.000375 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-820af6ac-ef83-4fea-abaf-386f53d8ec77 tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:14:31.000589 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-820af6ac-ef83-4fea-abaf-386f53d8ec77 tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] [instance: 3c5ab09e-0a97-4cfc-b249-a11ab221b5d2] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:14:31.010523 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-820af6ac-ef83-4fea-abaf-386f53d8ec77 tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:31.011037 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-820af6ac-ef83-4fea-abaf-386f53d8ec77 tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:31.013623 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-820af6ac-ef83-4fea-abaf-386f53d8ec77 tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:31.014240 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-820af6ac-ef83-4fea-abaf-386f53d8ec77 tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:31.422603 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.objects.instance [None req-820af6ac-ef83-4fea-abaf-386f53d8ec77 tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] Lazy-loading 'services' on Instance uuid 3c5ab09e-0a97-4cfc-b249-a11ab221b5d2 {{(pid=138681) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:14:31.449799 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-820af6ac-ef83-4fea-abaf-386f53d8ec77 tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/3c5ab09e-0a97-4cfc-b249-a11ab221b5d2" status: 200 len: 2417 microversion: 2.25 time: 0.454856 Jul 16 21:14:31.465617 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 735/1530] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:14:30 2022] GET /compute/v2.1/servers/3c5ab09e-0a97-4cfc-b249-a11ab221b5d2 => generated 2417 bytes in 471 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:14:31.477588 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-7507835a-957d-487a-aa83-1d0fb4342863 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:14:31.481251 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-7507835a-957d-487a-aa83-1d0fb4342863 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] [instance: 69cabe57-da71-4a02-b4ee-c0d84b3d32a2] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:14:31.485325 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7507835a-957d-487a-aa83-1d0fb4342863 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:31.485980 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7507835a-957d-487a-aa83-1d0fb4342863 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:31.486822 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7507835a-957d-487a-aa83-1d0fb4342863 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:31.487317 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7507835a-957d-487a-aa83-1d0fb4342863 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:31.884309 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-7507835a-957d-487a-aa83-1d0fb4342863 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '53af488f3b0242a797f539284b78d59e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bb1ede0d0b6941cc97278e6c617f9b01', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:14:31.895216 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-7507835a-957d-487a-aa83-1d0fb4342863 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] 10.222.0.16 "GET /compute/v2.1/servers/69cabe57-da71-4a02-b4ee-c0d84b3d32a2" status: 200 len: 1354 microversion: 2.1 time: 0.421293 Jul 16 21:14:31.897230 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 736/1531] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:14:31 2022] GET /compute/v2.1/servers/69cabe57-da71-4a02-b4ee-c0d84b3d32a2 => generated 1354 bytes in 430 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:14:31.903102 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-0a8f70fa-98c0-44ea-a3a5-b717f7270e3f tempest-ServersAdminNegativeTestJSON-1884193353 tempest-ServersAdminNegativeTestJSON-1884193353-project-admin] Action: 'action', calling method: >, body: {"suspend": {}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:14:31.903967 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-0a8f70fa-98c0-44ea-a3a5-b717f7270e3f tempest-ServersAdminNegativeTestJSON-1884193353 tempest-ServersAdminNegativeTestJSON-1884193353-project-admin] [instance: 76ee9d21-1f17-4b64-8f95-95a33e70e907] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:14:31.918702 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0a8f70fa-98c0-44ea-a3a5-b717f7270e3f tempest-ServersAdminNegativeTestJSON-1884193353 tempest-ServersAdminNegativeTestJSON-1884193353-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:31.919666 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0a8f70fa-98c0-44ea-a3a5-b717f7270e3f tempest-ServersAdminNegativeTestJSON-1884193353 tempest-ServersAdminNegativeTestJSON-1884193353-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:31.920821 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0a8f70fa-98c0-44ea-a3a5-b717f7270e3f tempest-ServersAdminNegativeTestJSON-1884193353 tempest-ServersAdminNegativeTestJSON-1884193353-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:31.921494 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0a8f70fa-98c0-44ea-a3a5-b717f7270e3f tempest-ServersAdminNegativeTestJSON-1884193353 tempest-ServersAdminNegativeTestJSON-1884193353-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:31.950421 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.objects.instance [None req-0a8f70fa-98c0-44ea-a3a5-b717f7270e3f tempest-ServersAdminNegativeTestJSON-1884193353 tempest-ServersAdminNegativeTestJSON-1884193353-project-admin] Lazy-loading 'flavor' on Instance uuid 76ee9d21-1f17-4b64-8f95-95a33e70e907 {{(pid=138681) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:14:32.378929 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-0a8f70fa-98c0-44ea-a3a5-b717f7270e3f tempest-ServersAdminNegativeTestJSON-1884193353 tempest-ServersAdminNegativeTestJSON-1884193353-project-admin] 10.222.0.16 "POST /compute/v2.1/servers/76ee9d21-1f17-4b64-8f95-95a33e70e907/action" status: 202 len: 0 microversion: 2.1 time: 0.479642 Jul 16 21:14:32.383034 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 737/1532] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 21:14:31 2022] POST /compute/v2.1/servers/76ee9d21-1f17-4b64-8f95-95a33e70e907/action => generated 0 bytes in 485 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 21:14:32.388839 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-520b509c-8b2e-4ffd-968c-65f2cdf70499 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:14:32.389563 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-520b509c-8b2e-4ffd-968c-65f2cdf70499 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] [instance: e66baae9-d488-46a4-8347-623030fc8b69] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:14:32.400277 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-520b509c-8b2e-4ffd-968c-65f2cdf70499 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:32.400731 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-520b509c-8b2e-4ffd-968c-65f2cdf70499 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:32.406863 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-520b509c-8b2e-4ffd-968c-65f2cdf70499 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:32.406863 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-520b509c-8b2e-4ffd-968c-65f2cdf70499 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:32.665647 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-520b509c-8b2e-4ffd-968c-65f2cdf70499 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4e9df82f95ac4779859972403869096a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4337575622d04a26942905c9dc044ba7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:14:32.673401 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-520b509c-8b2e-4ffd-968c-65f2cdf70499 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] 10.222.0.16 "GET /compute/v2.1/servers/e66baae9-d488-46a4-8347-623030fc8b69" status: 200 len: 1520 microversion: 2.1 time: 0.289142 Jul 16 21:14:32.678489 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 738/1533] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:14:32 2022] GET /compute/v2.1/servers/e66baae9-d488-46a4-8347-623030fc8b69 => generated 1520 bytes in 292 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:14:32.679732 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-542416db-31d7-45c6-93fd-28cb4857a806 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:14:32.680337 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-542416db-31d7-45c6-93fd-28cb4857a806 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: 1703f309-9e2e-4c5f-8588-eb4e43f21d91] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:14:32.686295 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-542416db-31d7-45c6-93fd-28cb4857a806 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:32.686743 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-542416db-31d7-45c6-93fd-28cb4857a806 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:32.687563 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-542416db-31d7-45c6-93fd-28cb4857a806 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:32.687999 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-542416db-31d7-45c6-93fd-28cb4857a806 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:32.693406 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:14:32.728442 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-542416db-31d7-45c6-93fd-28cb4857a806 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:32.728442 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-542416db-31d7-45c6-93fd-28cb4857a806 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:32.728442 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.objects.instance [None req-542416db-31d7-45c6-93fd-28cb4857a806 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lazy-loading 'fault' on Instance uuid 1703f309-9e2e-4c5f-8588-eb4e43f21d91 {{(pid=138681) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:14:32.827337 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-542416db-31d7-45c6-93fd-28cb4857a806 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a9b68dacc4b8463ba07fae6372131117', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55a19b7a4682457a9132f5fc80110712', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:14:32.835087 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-542416db-31d7-45c6-93fd-28cb4857a806 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "GET /compute/v2.1/servers/1703f309-9e2e-4c5f-8588-eb4e43f21d91" status: 200 len: 1302 microversion: 2.1 time: 0.158620 Jul 16 21:14:32.835675 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 739/1534] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:14:32 2022] GET /compute/v2.1/servers/1703f309-9e2e-4c5f-8588-eb4e43f21d91 => generated 1302 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:14:32.840578 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-85d24c79-d5ff-43a1-8193-383df7151cc6 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] Action: 'action', calling method: , body: {"removeFloatingIp": {"address": "172.35.1.110"}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:14:32.987441 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-fcf34f38-9973-4334-98d4-251cf712f030 tempest-FloatingIPsTestJSON-1267052927 tempest-FloatingIPsTestJSON-1267052927-project] 10.222.0.16 "POST /compute/v2.1/os-floating-ips" status: 200 len: 142 microversion: 2.1 time: 3.753314 Jul 16 21:14:32.988398 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 796/1535] 10.222.0.16 () {60 vars in 1169 bytes} [Sat Jul 16 21:14:29 2022] POST /compute/v2.1/os-floating-ips => generated 142 bytes in 3755 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:14:32.993402 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-14e4dbd1-794d-4eba-85d6-95cab4075ad7 req-f7056ad3-e90e-4e2c-af01-0ae22bfaaab3 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "1703f309-9e2e-4c5f-8588-eb4e43f21d91", "name": "network-vif-deleted", "tag": "d97c4e64-34a5-4e3d-b81c-2020594e3e0b"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:14:33.003093 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-14e4dbd1-794d-4eba-85d6-95cab4075ad7 req-f7056ad3-e90e-4e2c-af01-0ae22bfaaab3 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:33.003553 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-14e4dbd1-794d-4eba-85d6-95cab4075ad7 req-f7056ad3-e90e-4e2c-af01-0ae22bfaaab3 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:33.018462 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-14e4dbd1-794d-4eba-85d6-95cab4075ad7 req-f7056ad3-e90e-4e2c-af01-0ae22bfaaab3 service nova] Creating event network-vif-deleted:d97c4e64-34a5-4e3d-b81c-2020594e3e0b for instance 1703f309-9e2e-4c5f-8588-eb4e43f21d91 on nch1-master-1 Jul 16 21:14:33.024632 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-14e4dbd1-794d-4eba-85d6-95cab4075ad7 req-f7056ad3-e90e-4e2c-af01-0ae22bfaaab3 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.034747 Jul 16 21:14:33.025376 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 797/1536] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:14:32 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:14:33.029137 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-85d24c79-d5ff-43a1-8193-383df7151cc6 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] [instance: dada2ea0-2ef4-4a30-8e2d-40177d0bafce] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:14:33.030752 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-ec89fffd-1296-4529-8569-f8b1879ad169 tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:14:33.031490 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-ec89fffd-1296-4529-8569-f8b1879ad169 tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] [instance: 3c5ab09e-0a97-4cfc-b249-a11ab221b5d2] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:14:33.035813 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-85d24c79-d5ff-43a1-8193-383df7151cc6 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:33.036328 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-85d24c79-d5ff-43a1-8193-383df7151cc6 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:33.037158 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-85d24c79-d5ff-43a1-8193-383df7151cc6 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:33.037594 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-85d24c79-d5ff-43a1-8193-383df7151cc6 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:33.037876 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ec89fffd-1296-4529-8569-f8b1879ad169 tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:33.038340 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ec89fffd-1296-4529-8569-f8b1879ad169 tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:33.039205 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ec89fffd-1296-4529-8569-f8b1879ad169 tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:33.039675 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ec89fffd-1296-4529-8569-f8b1879ad169 tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:33.286877 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.objects.instance [None req-ec89fffd-1296-4529-8569-f8b1879ad169 tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] Lazy-loading 'services' on Instance uuid 3c5ab09e-0a97-4cfc-b249-a11ab221b5d2 {{(pid=138680) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:14:33.301117 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-ec89fffd-1296-4529-8569-f8b1879ad169 tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/3c5ab09e-0a97-4cfc-b249-a11ab221b5d2" status: 200 len: 2417 microversion: 2.25 time: 0.274412 Jul 16 21:14:33.301714 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 798/1537] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:14:33 2022] GET /compute/v2.1/servers/3c5ab09e-0a97-4cfc-b249-a11ab221b5d2 => generated 2417 bytes in 276 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:14:33.307237 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-9c7e279d-8289-4d79-b7ac-92a9fffe765a tempest-ServersAdminNegativeTestJSON-1884193353 tempest-ServersAdminNegativeTestJSON-1884193353-project-admin] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:14:33.307841 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-9c7e279d-8289-4d79-b7ac-92a9fffe765a tempest-ServersAdminNegativeTestJSON-1884193353 tempest-ServersAdminNegativeTestJSON-1884193353-project-admin] [instance: 76ee9d21-1f17-4b64-8f95-95a33e70e907] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:14:33.312266 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-85d24c79-d5ff-43a1-8193-383df7151cc6 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] Acquired lock "refresh_cache-dada2ea0-2ef4-4a30-8e2d-40177d0bafce" {{(pid=138681) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:294}} Jul 16 21:14:33.314391 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9c7e279d-8289-4d79-b7ac-92a9fffe765a tempest-ServersAdminNegativeTestJSON-1884193353 tempest-ServersAdminNegativeTestJSON-1884193353-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:33.314749 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9c7e279d-8289-4d79-b7ac-92a9fffe765a tempest-ServersAdminNegativeTestJSON-1884193353 tempest-ServersAdminNegativeTestJSON-1884193353-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:33.315546 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9c7e279d-8289-4d79-b7ac-92a9fffe765a tempest-ServersAdminNegativeTestJSON-1884193353 tempest-ServersAdminNegativeTestJSON-1884193353-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:33.315985 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9c7e279d-8289-4d79-b7ac-92a9fffe765a tempest-ServersAdminNegativeTestJSON-1884193353 tempest-ServersAdminNegativeTestJSON-1884193353-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:33.835886 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-9c7e279d-8289-4d79-b7ac-92a9fffe765a tempest-ServersAdminNegativeTestJSON-1884193353 tempest-ServersAdminNegativeTestJSON-1884193353-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/76ee9d21-1f17-4b64-8f95-95a33e70e907" status: 200 len: 1613 microversion: 2.1 time: 0.532886 Jul 16 21:14:33.851559 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 799/1538] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:14:33 2022] GET /compute/v2.1/servers/76ee9d21-1f17-4b64-8f95-95a33e70e907 => generated 1613 bytes in 549 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:14:33.861896 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-cbcf2e16-149d-46a1-8478-833f829d1090 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:14:33.862909 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-cbcf2e16-149d-46a1-8478-833f829d1090 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] [instance: e66baae9-d488-46a4-8347-623030fc8b69] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:14:33.873917 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cbcf2e16-149d-46a1-8478-833f829d1090 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:33.874432 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cbcf2e16-149d-46a1-8478-833f829d1090 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:33.875318 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cbcf2e16-149d-46a1-8478-833f829d1090 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:33.875760 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cbcf2e16-149d-46a1-8478-833f829d1090 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:33.897715 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=93,nova_cell1:SAVEPOINT=1,nova_cell1:UPDATE=3,nova_cell1:RELEASE=1,nova_cell1:INSERT=2 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:14:34.105364 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-cbcf2e16-149d-46a1-8478-833f829d1090 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4e9df82f95ac4779859972403869096a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4337575622d04a26942905c9dc044ba7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:14:34.113112 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-cbcf2e16-149d-46a1-8478-833f829d1090 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] 10.222.0.16 "GET /compute/v2.1/servers/e66baae9-d488-46a4-8347-623030fc8b69" status: 200 len: 1520 microversion: 2.1 time: 0.260364 Jul 16 21:14:34.113687 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 800/1539] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:14:33 2022] GET /compute/v2.1/servers/e66baae9-d488-46a4-8347-623030fc8b69 => generated 1520 bytes in 262 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:14:34.120155 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-36f7c538-75e3-4198-b88a-f589714a9907 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:14:34.121190 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-36f7c538-75e3-4198-b88a-f589714a9907 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] [instance: 69cabe57-da71-4a02-b4ee-c0d84b3d32a2] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:14:34.127607 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-36f7c538-75e3-4198-b88a-f589714a9907 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:34.128065 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-36f7c538-75e3-4198-b88a-f589714a9907 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:34.128846 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-36f7c538-75e3-4198-b88a-f589714a9907 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:34.129288 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-36f7c538-75e3-4198-b88a-f589714a9907 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:34.360566 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.network.neutron [None req-85d24c79-d5ff-43a1-8193-383df7151cc6 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] [instance: dada2ea0-2ef4-4a30-8e2d-40177d0bafce] Building network info cache for instance {{(pid=138681) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1990}} Jul 16 21:14:34.364772 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-36f7c538-75e3-4198-b88a-f589714a9907 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '53af488f3b0242a797f539284b78d59e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bb1ede0d0b6941cc97278e6c617f9b01', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:14:34.376099 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-36f7c538-75e3-4198-b88a-f589714a9907 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] 10.222.0.16 "GET /compute/v2.1/servers/69cabe57-da71-4a02-b4ee-c0d84b3d32a2" status: 200 len: 1354 microversion: 2.1 time: 0.259606 Jul 16 21:14:34.386725 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 801/1540] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:14:34 2022] GET /compute/v2.1/servers/69cabe57-da71-4a02-b4ee-c0d84b3d32a2 => generated 1354 bytes in 264 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:14:34.386725 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-28e5ce92-d7f9-475f-93b1-ad091601a66a tempest-FloatingIPsTestJSON-1267052927 tempest-FloatingIPsTestJSON-1267052927-project] Calling method '.version_select at 0x7fdd9d34bee0>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:14:35.448815 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.network.neutron [None req-85d24c79-d5ff-43a1-8193-383df7151cc6 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] [instance: dada2ea0-2ef4-4a30-8e2d-40177d0bafce] Updating instance_info_cache with network_info: [{"id": "b63bf933-a84e-4e18-98ed-0e772979bec5", "address": "fa:16:3e:68:6e:75", "network": {"id": "a27fab11-02d4-4a1c-99a4-dc86a33836ba", "bridge": null, "label": "tempest-FloatingIPsAssociationNegativeTestJSON-255901221-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "172.20.0.2"}}], "meta": {"injected": false, "tenant_id": "6c2095f96f8d45bb9e62c517678c3dcd", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false, "bound_drivers": {"0": "hyperv"}}, "devname": "tapb63bf933-a8", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=138681) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:117}} Jul 16 21:14:35.580237 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-85d24c79-d5ff-43a1-8193-383df7151cc6 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] Releasing lock "refresh_cache-dada2ea0-2ef4-4a30-8e2d-40177d0bafce" {{(pid=138681) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:312}} Jul 16 21:14:35.581799 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-85d24c79-d5ff-43a1-8193-383df7151cc6 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] 10.222.0.16 "POST /compute/v2.1/servers/dada2ea0-2ef4-4a30-8e2d-40177d0bafce/action" status: 202 len: 0 microversion: 2.1 time: 2.744543 Jul 16 21:14:35.582425 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 740/1541] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 21:14:32 2022] POST /compute/v2.1/servers/dada2ea0-2ef4-4a30-8e2d-40177d0bafce/action => generated 0 bytes in 2746 msecs (HTTP/1.1 202) 9 headers in 369 bytes (1 switches on core 0) Jul 16 21:14:35.587408 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-d86653b7-dbc3-4c7c-a47c-28258ad7d896 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:14:35.587902 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-d86653b7-dbc3-4c7c-a47c-28258ad7d896 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] [instance: 3c5ab09e-0a97-4cfc-b249-a11ab221b5d2] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:14:35.594303 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d86653b7-dbc3-4c7c-a47c-28258ad7d896 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:35.594752 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d86653b7-dbc3-4c7c-a47c-28258ad7d896 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:35.595542 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d86653b7-dbc3-4c7c-a47c-28258ad7d896 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:35.595977 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d86653b7-dbc3-4c7c-a47c-28258ad7d896 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:35.688829 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-28e5ce92-d7f9-475f-93b1-ad091601a66a tempest-FloatingIPsTestJSON-1267052927 tempest-FloatingIPsTestJSON-1267052927-project] 10.222.0.16 "DELETE /compute/v2.1/os-floating-ips/cd24ccc6-9821-4496-8e51-56fe56ed84a0" status: 202 len: 0 microversion: 2.1 time: 1.308280 Jul 16 21:14:35.689631 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 802/1542] 10.222.0.16 () {58 vars in 1262 bytes} [Sat Jul 16 21:14:34 2022] DELETE /compute/v2.1/os-floating-ips/cd24ccc6-9821-4496-8e51-56fe56ed84a0 => generated 0 bytes in 1310 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 21:14:35.699191 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-718a5110-9513-416e-8c24-15ced228b681 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:14:35.699891 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-718a5110-9513-416e-8c24-15ced228b681 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] [instance: 1703f309-9e2e-4c5f-8588-eb4e43f21d91] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:14:35.708692 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-718a5110-9513-416e-8c24-15ced228b681 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:35.709256 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-718a5110-9513-416e-8c24-15ced228b681 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:35.710530 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-718a5110-9513-416e-8c24-15ced228b681 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:35.711098 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-718a5110-9513-416e-8c24-15ced228b681 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:35.728025 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-718a5110-9513-416e-8c24-15ced228b681 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] HTTP exception thrown: Instance 1703f309-9e2e-4c5f-8588-eb4e43f21d91 could not be found. Jul 16 21:14:35.729572 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-718a5110-9513-416e-8c24-15ced228b681 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Request method failure captured: Jul 16 21:14:35.729572 ncdv-master-1 devstack@n-api.service[138680]: request: GET /compute/v2.1/servers/1703f309-9e2e-4c5f-8588-eb4e43f21d91 HTTP/1.1 Jul 16 21:14:35.729572 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:14:35.729572 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:14:35.729572 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:14:35.729572 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 0 Jul 16 21:14:35.729572 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:14:35.729572 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:14:35.729572 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:14:35.729572 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:14:35.729572 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0yjS0OwquTdxFvFwnqD6oXhLiVMC2H2T0PTTAPBs5CbYhBauxUQKqaIu70UDpOvDtiTqvXU4-VL9hTewWDu4Gh7O9lNbGbnPrucyL4QpaGXDfvsW3rocl0tQjubLkrzEuiaQ0dP8qaKxkSk27mrRw2WAWCg3PheG3BhIZr0IEKSfAB4 Jul 16 21:14:35.729572 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:14:35.729572 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:14:35.729572 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:14:35.729572 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:14:35.729572 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:14:35.729572 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:14:35.729572 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: 55a19b7a4682457a9132f5fc80110712 Jul 16 21:14:35.729572 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-MigrationsAdminTest-116332175 Jul 16 21:14:35.729572 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:14:35.729572 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:14:35.731456 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/55a19b7a4682457a9132f5fc80110712"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/55a19b7a4682457a9132f5fc80110712"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_55a19b7a4682457a9132f5fc80110712"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/55a19b7a4682457a9132f5fc80110712"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:14:35.731456 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-MigrationsAdminTest-116332175 Jul 16 21:14:35.731456 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: 55a19b7a4682457a9132f5fc80110712 Jul 16 21:14:35.731456 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-MigrationsAdminTest-116332175 Jul 16 21:14:35.731456 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-MigrationsAdminTest-116332175-project Jul 16 21:14:35.731456 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:14:35.731456 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:14:35.731456 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: a9b68dacc4b8463ba07fae6372131117 Jul 16 21:14:35.731456 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-MigrationsAdminTest-116332175-project Jul 16 21:14:35.731456 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 21:14:35.731456 ncdv-master-1 devstack@n-api.service[138680]: exception: Instance 1703f309-9e2e-4c5f-8588-eb4e43f21d91 could not be found. Jul 16 21:14:35.731456 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': '1703f309-9e2e-4c5f-8588-eb4e43f21d91'} Jul 16 21:14:35.732329 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:14:35.732329 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:14:35.732329 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:14:35.732329 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:14:35.732329 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:14:35.732329 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:14:35.732329 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:14:35.732329 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:14:35.732329 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:14:35.732329 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:14:35.732329 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 1703f309-9e2e-4c5f-8588-eb4e43f21d91 could not be found. Jul 16 21:14:35.732329 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:14:35.732329 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:14:35.732329 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:14:35.732329 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:14:35.732329 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:14:35.732329 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:14:35.732329 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:14:35.732329 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:14:35.732329 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:14:35.733868 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:14:35.733868 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:14:35.733868 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:14:35.733868 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:14:35.733868 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:14:35.733868 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:14:35.733868 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:14:35.733868 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 1703f309-9e2e-4c5f-8588-eb4e43f21d91 could not be found. Jul 16 21:14:35.733868 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:14:35.733868 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:14:35.733868 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:14:35.733868 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:14:35.733868 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:14:35.733868 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:14:35.733868 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:14:35.733868 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:14:35.733868 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 1703f309-9e2e-4c5f-8588-eb4e43f21d91 could not be found. Jul 16 21:14:35.733868 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:14:35.735469 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-718a5110-9513-416e-8c24-15ced228b681 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] Returning 404 to user: Instance 1703f309-9e2e-4c5f-8588-eb4e43f21d91 could not be found. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:14:35.736397 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-718a5110-9513-416e-8c24-15ced228b681 tempest-MigrationsAdminTest-116332175 tempest-MigrationsAdminTest-116332175-project] 10.222.0.16 "GET /compute/v2.1/servers/1703f309-9e2e-4c5f-8588-eb4e43f21d91" status: 404 len: 111 microversion: 2.1 time: 0.045215 Jul 16 21:14:35.737157 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 803/1543] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:14:35 2022] GET /compute/v2.1/servers/1703f309-9e2e-4c5f-8588-eb4e43f21d91 => generated 111 bytes in 47 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:14:35.739626 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.objects.instance [None req-d86653b7-dbc3-4c7c-a47c-28258ad7d896 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lazy-loading 'services' on Instance uuid 3c5ab09e-0a97-4cfc-b249-a11ab221b5d2 {{(pid=138681) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:14:35.750100 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-59fc5d54-cbed-47d2-883f-e254e8e32ca1 tempest-ServersAdminNegativeTestJSON-1884193353 tempest-ServersAdminNegativeTestJSON-1884193353-project-admin] Action: 'action', calling method: >, body: {"migrate": {}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:14:35.750100 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-59fc5d54-cbed-47d2-883f-e254e8e32ca1 tempest-ServersAdminNegativeTestJSON-1884193353 tempest-ServersAdminNegativeTestJSON-1884193353-project-admin] [instance: 76ee9d21-1f17-4b64-8f95-95a33e70e907] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:14:35.757771 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-59fc5d54-cbed-47d2-883f-e254e8e32ca1 tempest-ServersAdminNegativeTestJSON-1884193353 tempest-ServersAdminNegativeTestJSON-1884193353-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:35.758604 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-59fc5d54-cbed-47d2-883f-e254e8e32ca1 tempest-ServersAdminNegativeTestJSON-1884193353 tempest-ServersAdminNegativeTestJSON-1884193353-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:35.760500 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-59fc5d54-cbed-47d2-883f-e254e8e32ca1 tempest-ServersAdminNegativeTestJSON-1884193353 tempest-ServersAdminNegativeTestJSON-1884193353-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:35.761965 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-59fc5d54-cbed-47d2-883f-e254e8e32ca1 tempest-ServersAdminNegativeTestJSON-1884193353 tempest-ServersAdminNegativeTestJSON-1884193353-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:35.842901 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-59fc5d54-cbed-47d2-883f-e254e8e32ca1 tempest-ServersAdminNegativeTestJSON-1884193353 tempest-ServersAdminNegativeTestJSON-1884193353-project-admin] HTTP exception thrown: Cannot 'migrate' instance 76ee9d21-1f17-4b64-8f95-95a33e70e907 while it is in vm_state suspended Jul 16 21:14:35.842901 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-59fc5d54-cbed-47d2-883f-e254e8e32ca1 tempest-ServersAdminNegativeTestJSON-1884193353 tempest-ServersAdminNegativeTestJSON-1884193353-project-admin] Request method failure captured: Jul 16 21:14:35.842901 ncdv-master-1 devstack@n-api.service[138680]: request: POST /compute/v2.1/servers/76ee9d21-1f17-4b64-8f95-95a33e70e907/action HTTP/1.1 Jul 16 21:14:35.842901 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:14:35.842901 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:14:35.842901 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:14:35.842901 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 15 Jul 16 21:14:35.842901 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:14:35.842901 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:14:35.842901 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:14:35.842901 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:14:35.842901 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0yn7QG_tnLYMptx-bTEUJY8ylzBGFOwnadoDojC2tCzwJeRalDhG_GURBfSS7Jpd1amigWw5Nbi2c_3Jq_jqZUD7WjqZe8pZKa1SRAQVw-5Rq-fp6JQvZsa6miiKLCZoWSEIKgl6wCp7cEj6e1JS1FO8OqdXGKZOwT9NqIcGygKtteg Jul 16 21:14:35.842901 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:14:35.842901 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:14:35.842901 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:14:35.842901 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:14:35.842901 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:14:35.842901 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:14:35.842901 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: 38594b6bc3354178a22f0abb698e98a6 Jul 16 21:14:35.842901 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-ServersAdminNegativeTestJSON-1884193353 Jul 16 21:14:35.842901 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,admin,reader Jul 16 21:14:35.842901 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,admin,reader Jul 16 21:14:35.844489 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/38594b6bc3354178a22f0abb698e98a6"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/38594b6bc3354178a22f0abb698e98a6"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_38594b6bc3354178a22f0abb698e98a6"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/38594b6bc3354178a22f0abb698e98a6"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:14:35.844489 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-ServersAdminNegativeTestJSON-1884193353 Jul 16 21:14:35.844489 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: 38594b6bc3354178a22f0abb698e98a6 Jul 16 21:14:35.844489 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-ServersAdminNegativeTestJSON-1884193353 Jul 16 21:14:35.844489 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-ServersAdminNegativeTestJSON-1884193353-project-admin Jul 16 21:14:35.844489 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:14:35.844489 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:14:35.844489 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: 27aeaef0a671464493a8cac846186af7 Jul 16 21:14:35.844489 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-ServersAdminNegativeTestJSON-1884193353-project-admin Jul 16 21:14:35.844489 ncdv-master-1 devstack@n-api.service[138680]: Jul 16 21:14:35.844489 ncdv-master-1 devstack@n-api.service[138680]: {"migrate": {}} Jul 16 21:14:35.844489 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 21:14:35.844489 ncdv-master-1 devstack@n-api.service[138680]: exception: Cannot 'migrate' instance 76ee9d21-1f17-4b64-8f95-95a33e70e907 while it is in vm_state suspended Jul 16 21:14:35.845487 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': '76ee9d21-1f17-4b64-8f95-95a33e70e907', 'body': {'migrate': {}}} Jul 16 21:14:35.845487 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:14:35.845487 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:14:35.845487 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/migrate_server.py", line 58, in _migrate Jul 16 21:14:35.845487 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi self.compute_api.resize(req.environ['nova.context'], instance, Jul 16 21:14:35.845487 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 375, in inner Jul 16 21:14:35.845487 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Jul 16 21:14:35.845487 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 360, in wrapper Jul 16 21:14:35.845487 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return func(self, context, instance, *args, **kwargs) Jul 16 21:14:35.845487 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 296, in inner Jul 16 21:14:35.845487 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Jul 16 21:14:35.845487 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 343, in wrapper Jul 16 21:14:35.845487 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return func(self, context, instance, *args, **kwargs) Jul 16 21:14:35.845487 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner Jul 16 21:14:35.845487 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Jul 16 21:14:35.845487 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 145, in inner Jul 16 21:14:35.845487 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.InstanceInvalidState( Jul 16 21:14:35.845487 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.InstanceInvalidState: Instance 76ee9d21-1f17-4b64-8f95-95a33e70e907 in vm_state suspended. Cannot resize while the instance is in this state. Jul 16 21:14:35.845487 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:14:35.846790 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:14:35.846790 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:14:35.846790 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:14:35.846790 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:14:35.846790 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:14:35.846790 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:14:35.846790 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:14:35.846790 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:14:35.846790 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:14:35.846790 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 16 21:14:35.846790 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 16 21:14:35.846790 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/migrate_server.py", line 69, in _migrate Jul 16 21:14:35.846790 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi common.raise_http_conflict_for_instance_invalid_state(state_error, Jul 16 21:14:35.846790 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 347, in raise_http_conflict_for_instance_invalid_state Jul 16 21:14:35.846790 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPConflict(explanation=msg) Jul 16 21:14:35.846790 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPConflict: Cannot 'migrate' instance 76ee9d21-1f17-4b64-8f95-95a33e70e907 while it is in vm_state suspended Jul 16 21:14:35.846790 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:14:35.846790 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:14:35.846790 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:14:35.846790 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:14:35.850580 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:14:35.850580 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:14:35.850580 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:14:35.850580 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:14:35.850580 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Cannot 'migrate' instance 76ee9d21-1f17-4b64-8f95-95a33e70e907 while it is in vm_state suspended Jul 16 21:14:35.850580 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:14:35.850580 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-59fc5d54-cbed-47d2-883f-e254e8e32ca1 tempest-ServersAdminNegativeTestJSON-1884193353 tempest-ServersAdminNegativeTestJSON-1884193353-project-admin] Returning 409 to user: Cannot 'migrate' instance 76ee9d21-1f17-4b64-8f95-95a33e70e907 while it is in vm_state suspended {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:14:35.850580 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-59fc5d54-cbed-47d2-883f-e254e8e32ca1 tempest-ServersAdminNegativeTestJSON-1884193353 tempest-ServersAdminNegativeTestJSON-1884193353-project-admin] 10.222.0.16 "POST /compute/v2.1/servers/76ee9d21-1f17-4b64-8f95-95a33e70e907/action" status: 409 len: 148 microversion: 2.1 time: 0.101641 Jul 16 21:14:35.850580 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 804/1544] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 21:14:35 2022] POST /compute/v2.1/servers/76ee9d21-1f17-4b64-8f95-95a33e70e907/action => generated 148 bytes in 103 msecs (HTTP/1.1 409) 9 headers in 378 bytes (1 switches on core 0) Jul 16 21:14:35.854570 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-69f6bd5c-0f85-4c00-8330-be35eba58003 req-d6517dbb-4513-4ea5-9426-b92c5500914e service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "69cabe57-da71-4a02-b4ee-c0d84b3d32a2", "name": "network-vif-plugged", "status": "completed", "tag": "9e27e304-0422-4ee7-8ca0-2d670e12f5ee"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:14:35.864325 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-69f6bd5c-0f85-4c00-8330-be35eba58003 req-d6517dbb-4513-4ea5-9426-b92c5500914e service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:35.866276 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-69f6bd5c-0f85-4c00-8330-be35eba58003 req-d6517dbb-4513-4ea5-9426-b92c5500914e service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:35.878945 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-69f6bd5c-0f85-4c00-8330-be35eba58003 req-d6517dbb-4513-4ea5-9426-b92c5500914e service nova] Creating event network-vif-plugged:9e27e304-0422-4ee7-8ca0-2d670e12f5ee for instance 69cabe57-da71-4a02-b4ee-c0d84b3d32a2 on nch1-master-1 Jul 16 21:14:35.897579 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-69f6bd5c-0f85-4c00-8330-be35eba58003 req-d6517dbb-4513-4ea5-9426-b92c5500914e service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.055847 Jul 16 21:14:35.898378 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 805/1545] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:14:35 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:14:35.904402 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-eb07232b-88a4-4bb2-9a78-d4c2b23eb821 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Action: 'action', calling method: >, body: {"createImage": {"name": "tempest-ImagesOneServerNegativeTestJSON-image-1894186500"}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:14:35.905534 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-eb07232b-88a4-4bb2-9a78-d4c2b23eb821 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] [instance: e66baae9-d488-46a4-8347-623030fc8b69] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:14:35.912480 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-eb07232b-88a4-4bb2-9a78-d4c2b23eb821 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:35.912936 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-eb07232b-88a4-4bb2-9a78-d4c2b23eb821 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:35.913986 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-eb07232b-88a4-4bb2-9a78-d4c2b23eb821 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:35.914434 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-eb07232b-88a4-4bb2-9a78-d4c2b23eb821 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:36.235948 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-d86653b7-dbc3-4c7c-a47c-28258ad7d896 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] 10.222.0.16 "DELETE /compute/v2.1/servers/3c5ab09e-0a97-4cfc-b249-a11ab221b5d2/os-volume_attachments/cc3f1f89-b013-47d9-982f-2e3f355e5391" status: 202 len: 0 microversion: 2.25 time: 0.652194 Jul 16 21:14:36.236897 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 741/1546] 10.222.0.16 () {60 vars in 1456 bytes} [Sat Jul 16 21:14:35 2022] DELETE /compute/v2.1/servers/3c5ab09e-0a97-4cfc-b249-a11ab221b5d2/os-volume_attachments/cc3f1f89-b013-47d9-982f-2e3f355e5391 => generated 0 bytes in 654 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Jul 16 21:14:36.243466 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-ce1b6a0a-23ee-4e07-9fb1-e832176f673b req-56d7a1e6-12cd-4f34-a5a0-b7bb23cce4ee service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "dada2ea0-2ef4-4a30-8e2d-40177d0bafce", "tag": "b63bf933-a84e-4e18-98ed-0e772979bec5"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:14:36.252341 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-ce1b6a0a-23ee-4e07-9fb1-e832176f673b req-56d7a1e6-12cd-4f34-a5a0-b7bb23cce4ee service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:36.252789 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-ce1b6a0a-23ee-4e07-9fb1-e832176f673b req-56d7a1e6-12cd-4f34-a5a0-b7bb23cce4ee service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:36.267047 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-ce1b6a0a-23ee-4e07-9fb1-e832176f673b req-56d7a1e6-12cd-4f34-a5a0-b7bb23cce4ee service nova] Creating event network-changed:b63bf933-a84e-4e18-98ed-0e772979bec5 for instance dada2ea0-2ef4-4a30-8e2d-40177d0bafce on nch2-master-1 Jul 16 21:14:36.272452 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-ce1b6a0a-23ee-4e07-9fb1-e832176f673b req-56d7a1e6-12cd-4f34-a5a0-b7bb23cce4ee service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.034219 Jul 16 21:14:36.273061 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 742/1547] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:14:36 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:14:36.278409 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-34463792-c616-4323-865c-53979be57f7f tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:14:36.279145 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-34463792-c616-4323-865c-53979be57f7f tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] [instance: 69cabe57-da71-4a02-b4ee-c0d84b3d32a2] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:14:36.285911 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-34463792-c616-4323-865c-53979be57f7f tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:36.287538 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-34463792-c616-4323-865c-53979be57f7f tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:36.288225 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-34463792-c616-4323-865c-53979be57f7f tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:36.288664 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-34463792-c616-4323-865c-53979be57f7f tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:36.531028 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-eb07232b-88a4-4bb2-9a78-d4c2b23eb821 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] 10.222.0.16 "POST /compute/v2.1/servers/e66baae9-d488-46a4-8347-623030fc8b69/action" status: 202 len: 0 microversion: 2.1 time: 0.630819 Jul 16 21:14:36.532751 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 806/1548] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 21:14:35 2022] POST /compute/v2.1/servers/e66baae9-d488-46a4-8347-623030fc8b69/action => generated 0 bytes in 633 msecs (HTTP/1.1 202) 10 headers in 450 bytes (1 switches on core 0) Jul 16 21:14:36.539960 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-927b4c00-2cc1-49ef-b01d-5cbde5a4aa66 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] Calling method '.version_select at 0x7fdd9db920d0>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:14:36.543582 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-34463792-c616-4323-865c-53979be57f7f tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '53af488f3b0242a797f539284b78d59e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bb1ede0d0b6941cc97278e6c617f9b01', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:14:36.557406 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-34463792-c616-4323-865c-53979be57f7f tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] 10.222.0.16 "GET /compute/v2.1/servers/69cabe57-da71-4a02-b4ee-c0d84b3d32a2" status: 200 len: 1354 microversion: 2.1 time: 0.282690 Jul 16 21:14:36.558375 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 743/1549] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:14:36 2022] GET /compute/v2.1/servers/69cabe57-da71-4a02-b4ee-c0d84b3d32a2 => generated 1354 bytes in 285 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:14:36.566011 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-8d8a1f8a-2b4f-4870-81ce-6183a480f724 tempest-FloatingIPsTestJSON-1267052927 tempest-FloatingIPsTestJSON-1267052927-project] Calling method '.version_select at 0x7fdd9cf973a0>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:14:36.648102 ncdv-master-1 devstack@n-api.service[138681]: DEBUG neutronclient.v2_0.client [None req-8d8a1f8a-2b4f-4870-81ce-6183a480f724 tempest-FloatingIPsTestJSON-1267052927 tempest-FloatingIPsTestJSON-1267052927-project] Error message: {"NeutronError": {"type": "FloatingIPNotFound", "message": "Floating IP cd24ccc6-9821-4496-8e51-56fe56ed84a0 could not be found", "detail": ""}} {{(pid=138681) _handle_fault_response /usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py:262}} Jul 16 21:14:36.648797 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-8d8a1f8a-2b4f-4870-81ce-6183a480f724 tempest-FloatingIPsTestJSON-1267052927 tempest-FloatingIPsTestJSON-1267052927-project] HTTP exception thrown: Floating IP not found for ID cd24ccc6-9821-4496-8e51-56fe56ed84a0 Jul 16 21:14:36.651142 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-8d8a1f8a-2b4f-4870-81ce-6183a480f724 tempest-FloatingIPsTestJSON-1267052927 tempest-FloatingIPsTestJSON-1267052927-project] Request method failure captured: Jul 16 21:14:36.651142 ncdv-master-1 devstack@n-api.service[138681]: request: GET /compute/v2.1/os-floating-ips/cd24ccc6-9821-4496-8e51-56fe56ed84a0 HTTP/1.1 Jul 16 21:14:36.651142 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:14:36.651142 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:14:36.651142 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:14:36.651142 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 21:14:36.651142 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:14:36.651142 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:14:36.651142 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:14:36.651142 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:14:36.651142 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0yonFOGKRLELAsCe8m1X2Z3-TG2zYxKtr9a40p9bP_XTJyZ895vPVspCAmVCXESKGrpfzA-4iGrX0S47x_B3iAm7yeJaZlwEE4Ev19Sf5YFOTlTWU4Iu7ahUjou3cpmgdeGX5piTC7pTxWIwLFLVDQeUmyk_ZQ9pfN7sipULqaLhdYo Jul 16 21:14:36.651142 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:14:36.651142 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:14:36.651142 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:14:36.651142 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:14:36.651142 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:14:36.651142 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:14:36.651142 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: 538d051aac0f4b66a7f4738f367a1ba8 Jul 16 21:14:36.651142 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-FloatingIPsTestJSON-1267052927 Jul 16 21:14:36.651142 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:14:36.651142 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:14:36.652647 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/538d051aac0f4b66a7f4738f367a1ba8"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/538d051aac0f4b66a7f4738f367a1ba8"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_538d051aac0f4b66a7f4738f367a1ba8"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/538d051aac0f4b66a7f4738f367a1ba8"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:14:36.652647 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-FloatingIPsTestJSON-1267052927 Jul 16 21:14:36.652647 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: 538d051aac0f4b66a7f4738f367a1ba8 Jul 16 21:14:36.652647 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-FloatingIPsTestJSON-1267052927 Jul 16 21:14:36.652647 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-FloatingIPsTestJSON-1267052927-project Jul 16 21:14:36.652647 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:14:36.652647 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:14:36.652647 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: 348961e36710431387915e2220ee3431 Jul 16 21:14:36.652647 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-FloatingIPsTestJSON-1267052927-project Jul 16 21:14:36.652647 ncdv-master-1 devstack@n-api.service[138681]: method: Jul 16 21:14:36.652647 ncdv-master-1 devstack@n-api.service[138681]: exception: Floating IP not found for ID cd24ccc6-9821-4496-8e51-56fe56ed84a0 Jul 16 21:14:36.652647 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': 'cd24ccc6-9821-4496-8e51-56fe56ed84a0'} Jul 16 21:14:36.652647 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:14:36.653576 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:14:36.653576 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 2827, in get_floating_ip Jul 16 21:14:36.653576 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi fip = client.show_floatingip(id)['floatingip'] Jul 16 21:14:36.653576 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 197, in wrapper Jul 16 21:14:36.653576 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Jul 16 21:14:36.653576 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 1006, in show_floatingip Jul 16 21:14:36.653576 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return self.get(self.floatingip_path % (floatingip), params=_params) Jul 16 21:14:36.653576 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 197, in wrapper Jul 16 21:14:36.653576 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Jul 16 21:14:36.653576 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 356, in get Jul 16 21:14:36.653576 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return self.retry_request("GET", action, body=body, Jul 16 21:14:36.653576 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 197, in wrapper Jul 16 21:14:36.653576 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Jul 16 21:14:36.653576 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 333, in retry_request Jul 16 21:14:36.653576 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return self.do_request(method, action, body=body, Jul 16 21:14:36.653576 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 197, in wrapper Jul 16 21:14:36.653576 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Jul 16 21:14:36.653576 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 297, in do_request Jul 16 21:14:36.653576 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi self._handle_fault_response(status_code, replybody, resp) Jul 16 21:14:36.654800 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 197, in wrapper Jul 16 21:14:36.654800 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Jul 16 21:14:36.654800 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 272, in _handle_fault_response Jul 16 21:14:36.654800 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi exception_handler_v20(status_code, error_body) Jul 16 21:14:36.654800 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 90, in exception_handler_v20 Jul 16 21:14:36.654800 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise client_exc(message=error_message, Jul 16 21:14:36.654800 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi neutronclient.common.exceptions.NotFound: Floating IP cd24ccc6-9821-4496-8e51-56fe56ed84a0 could not be found Jul 16 21:14:36.654800 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-ad41dbe0-d1d5-49d4-a081-cc8bdbd34c59'] Jul 16 21:14:36.654800 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:14:36.654800 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:14:36.654800 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:14:36.654800 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:14:36.654800 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/floating_ips.py", line 94, in show Jul 16 21:14:36.654800 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi floating_ip = self.network_api.get_floating_ip(context, id) Jul 16 21:14:36.654800 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 2830, in get_floating_ip Jul 16 21:14:36.654800 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.FloatingIpNotFound(id=id) Jul 16 21:14:36.654800 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.FloatingIpNotFound: Floating IP not found for ID cd24ccc6-9821-4496-8e51-56fe56ed84a0. Jul 16 21:14:36.654800 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:14:36.654800 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:14:36.654800 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:14:36.658636 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:14:36.658636 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:14:36.658636 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:14:36.658636 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:14:36.658636 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:14:36.658636 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Jul 16 21:14:36.658636 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Jul 16 21:14:36.658636 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:14:36.658636 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:14:36.658636 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/floating_ips.py", line 97, in show Jul 16 21:14:36.658636 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=msg) Jul 16 21:14:36.658636 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Floating IP not found for ID cd24ccc6-9821-4496-8e51-56fe56ed84a0 Jul 16 21:14:36.658636 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:14:36.658636 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:14:36.658636 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:14:36.658636 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:14:36.658636 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:14:36.658636 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:14:36.658636 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:14:36.658636 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:14:36.661399 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Floating IP not found for ID cd24ccc6-9821-4496-8e51-56fe56ed84a0 Jul 16 21:14:36.661399 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:14:36.661399 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-8d8a1f8a-2b4f-4870-81ce-6183a480f724 tempest-FloatingIPsTestJSON-1267052927 tempest-FloatingIPsTestJSON-1267052927-project] Returning 404 to user: Floating IP not found for ID cd24ccc6-9821-4496-8e51-56fe56ed84a0 {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:14:36.661399 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-8d8a1f8a-2b4f-4870-81ce-6183a480f724 tempest-FloatingIPsTestJSON-1267052927 tempest-FloatingIPsTestJSON-1267052927-project] 10.222.0.16 "GET /compute/v2.1/os-floating-ips/cd24ccc6-9821-4496-8e51-56fe56ed84a0" status: 404 len: 111 microversion: 2.1 time: 0.093163 Jul 16 21:14:36.661399 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 744/1550] 10.222.0.16 () {58 vars in 1259 bytes} [Sat Jul 16 21:14:36 2022] GET /compute/v2.1/os-floating-ips/cd24ccc6-9821-4496-8e51-56fe56ed84a0 => generated 111 bytes in 98 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:14:36.661399 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-09758311-acd2-4be7-a24e-fd250c498b52 tempest-ServersAdminNegativeTestJSON-1884193353 tempest-ServersAdminNegativeTestJSON-1884193353-project-admin] Action: 'action', calling method: >, body: {"os-resetState": {"state": "invalid"}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:14:36.664276 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-09758311-acd2-4be7-a24e-fd250c498b52 tempest-ServersAdminNegativeTestJSON-1884193353 tempest-ServersAdminNegativeTestJSON-1884193353-project-admin] Request method failure captured: Jul 16 21:14:36.664276 ncdv-master-1 devstack@n-api.service[138681]: request: POST /compute/v2.1/servers/4602867b-88ed-43b1-9ee6-bf661e2d7d99/action HTTP/1.1 Jul 16 21:14:36.664276 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:14:36.664276 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:14:36.664276 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:14:36.664276 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 39 Jul 16 21:14:36.664276 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:14:36.664276 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:14:36.664276 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:14:36.664276 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:14:36.664276 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0yn7QG_tnLYMptx-bTEUJY8ylzBGFOwnadoDojC2tCzwJeRalDhG_GURBfSS7Jpd1amigWw5Nbi2c_3Jq_jqZUD7WjqZe8pZKa1SRAQVw-5Rq-fp6JQvZsa6miiKLCZoWSEIKgl6wCp7cEj6e1JS1FO8OqdXGKZOwT9NqIcGygKtteg Jul 16 21:14:36.664276 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:14:36.664276 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:14:36.664276 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:14:36.664276 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:14:36.664276 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:14:36.664276 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:14:36.664276 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: 38594b6bc3354178a22f0abb698e98a6 Jul 16 21:14:36.664276 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-ServersAdminNegativeTestJSON-1884193353 Jul 16 21:14:36.664276 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,admin,reader Jul 16 21:14:36.664276 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,admin,reader Jul 16 21:14:36.667435 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/38594b6bc3354178a22f0abb698e98a6"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/38594b6bc3354178a22f0abb698e98a6"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_38594b6bc3354178a22f0abb698e98a6"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/38594b6bc3354178a22f0abb698e98a6"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:14:36.667435 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-ServersAdminNegativeTestJSON-1884193353 Jul 16 21:14:36.667435 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: 38594b6bc3354178a22f0abb698e98a6 Jul 16 21:14:36.667435 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-ServersAdminNegativeTestJSON-1884193353 Jul 16 21:14:36.667435 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-ServersAdminNegativeTestJSON-1884193353-project-admin Jul 16 21:14:36.667435 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:14:36.667435 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:14:36.667435 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: 27aeaef0a671464493a8cac846186af7 Jul 16 21:14:36.667435 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-ServersAdminNegativeTestJSON-1884193353-project-admin Jul 16 21:14:36.667435 ncdv-master-1 devstack@n-api.service[138681]: Jul 16 21:14:36.667435 ncdv-master-1 devstack@n-api.service[138681]: {"os-resetState": {"state": "invalid"}} Jul 16 21:14:36.667435 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:14:36.669205 ncdv-master-1 devstack@n-api.service[138681]: exception: Invalid input for field/attribute state. Value: invalid. 'invalid' is not one of ['active', 'error'] Jul 16 21:14:36.669205 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': '4602867b-88ed-43b1-9ee6-bf661e2d7d99', 'body': {'os-resetState': {'state': 'invalid'}}} Jul 16 21:14:36.669205 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:14:36.669205 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:14:36.669205 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 279, in validate Jul 16 21:14:36.669205 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Jul 16 21:14:36.669205 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/lib/python3/dist-packages/jsonschema/validators.py", line 353, in validate Jul 16 21:14:36.669205 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise error Jul 16 21:14:36.669205 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: 'invalid' is not one of ['active', 'error'] Jul 16 21:14:36.669205 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:14:36.669205 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Failed validating 'enum' in schema['properties']['os-resetState']['properties']['state']: Jul 16 21:14:36.669205 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi {'enum': ['active', 'error'], 'type': 'string'} Jul 16 21:14:36.669205 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:14:36.669205 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi On instance['os-resetState']: Jul 16 21:14:36.669205 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi 'invalid' Jul 16 21:14:36.669205 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:14:36.669205 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:14:36.669205 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:14:36.669205 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:14:36.669205 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:14:36.669205 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:14:36.669205 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:14:36.672921 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:14:36.672921 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:14:36.672921 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:14:36.672921 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Jul 16 21:14:36.672921 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Jul 16 21:14:36.672921 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Jul 16 21:14:36.672921 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Jul 16 21:14:36.672921 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 308, in validate Jul 16 21:14:36.672921 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Jul 16 21:14:36.672921 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute state. Value: invalid. 'invalid' is not one of ['active', 'error'] Jul 16 21:14:36.672921 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:14:36.672921 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:14:36.672921 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:14:36.672921 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:14:36.672921 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:14:36.672921 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:14:36.672921 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Jul 16 21:14:36.672921 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Jul 16 21:14:36.672921 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute state. Value: invalid. 'invalid' is not one of ['active', 'error'] Jul 16 21:14:36.676197 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:14:36.676197 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-09758311-acd2-4be7-a24e-fd250c498b52 tempest-ServersAdminNegativeTestJSON-1884193353 tempest-ServersAdminNegativeTestJSON-1884193353-project-admin] Returning 400 to user: Invalid input for field/attribute state. Value: invalid. 'invalid' is not one of ['active', 'error'] {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:14:36.676197 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-09758311-acd2-4be7-a24e-fd250c498b52 tempest-ServersAdminNegativeTestJSON-1884193353 tempest-ServersAdminNegativeTestJSON-1884193353-project-admin] 10.222.0.16 "POST /compute/v2.1/servers/4602867b-88ed-43b1-9ee6-bf661e2d7d99/action" status: 400 len: 144 microversion: 2.1 time: 0.007874 Jul 16 21:14:36.676197 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 745/1551] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 21:14:36 2022] POST /compute/v2.1/servers/4602867b-88ed-43b1-9ee6-bf661e2d7d99/action => generated 144 bytes in 9 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Jul 16 21:14:36.676197 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-291bdd70-0c20-4842-85d5-f795bf7f4c6f tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:14:36.676197 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-291bdd70-0c20-4842-85d5-f795bf7f4c6f tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] [instance: 3c5ab09e-0a97-4cfc-b249-a11ab221b5d2] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:14:36.686336 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-291bdd70-0c20-4842-85d5-f795bf7f4c6f tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:36.686336 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-291bdd70-0c20-4842-85d5-f795bf7f4c6f tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:36.686954 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-291bdd70-0c20-4842-85d5-f795bf7f4c6f tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:36.692457 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-291bdd70-0c20-4842-85d5-f795bf7f4c6f tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:36.739550 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-291bdd70-0c20-4842-85d5-f795bf7f4c6f tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] 10.222.0.16 "GET /compute/v2.1/servers/3c5ab09e-0a97-4cfc-b249-a11ab221b5d2/os-volume_attachments" status: 200 len: 197 microversion: 2.25 time: 0.069649 Jul 16 21:14:36.739550 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 746/1552] 10.222.0.16 () {60 vars in 1342 bytes} [Sat Jul 16 21:14:36 2022] GET /compute/v2.1/servers/3c5ab09e-0a97-4cfc-b249-a11ab221b5d2/os-volume_attachments => generated 197 bytes in 72 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 16 21:14:36.748086 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-73da82de-d383-42dd-b72a-a43aa990575c tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Action: 'action', calling method: >, body: {"createImage": {"name": "tempest-ImagesOneServerNegativeTestJSON-image-1282562837"}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:14:36.749336 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-73da82de-d383-42dd-b72a-a43aa990575c tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] [instance: e66baae9-d488-46a4-8347-623030fc8b69] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:14:36.756110 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-73da82de-d383-42dd-b72a-a43aa990575c tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:36.757358 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-73da82de-d383-42dd-b72a-a43aa990575c tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:36.758344 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-73da82de-d383-42dd-b72a-a43aa990575c tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:36.758876 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-73da82de-d383-42dd-b72a-a43aa990575c tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:36.808259 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-73da82de-d383-42dd-b72a-a43aa990575c tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] HTTP exception thrown: Cannot 'createImage' instance e66baae9-d488-46a4-8347-623030fc8b69 while it is in task_state image_snapshot_pending Jul 16 21:14:36.810018 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-73da82de-d383-42dd-b72a-a43aa990575c tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Request method failure captured: Jul 16 21:14:36.810018 ncdv-master-1 devstack@n-api.service[138681]: request: POST /compute/v2.1/servers/e66baae9-d488-46a4-8347-623030fc8b69/action HTTP/1.1 Jul 16 21:14:36.810018 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:14:36.810018 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:14:36.810018 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:14:36.810018 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 85 Jul 16 21:14:36.810018 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:14:36.810018 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:14:36.810018 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:14:36.810018 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:14:36.810018 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0yoJFUAf_8Wtq8fI-wZdJwoKyg8rUEOC_eBMeqcRE2hk6JM7_ssDplugQwmk2S9xH7Oys7Ho2tw0kc9hmlY-jX_rR-I-9WVwQYFbyogU7ki99siEGGR9aVA2GKsp5nNcVHXBtjM3bI_zfa21XSWedvaSIVwZV8kEGixKYsBf2GMYKVI Jul 16 21:14:36.810018 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:14:36.810018 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:14:36.810018 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:14:36.810018 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:14:36.810018 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:14:36.810018 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:14:36.810018 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: 4337575622d04a26942905c9dc044ba7 Jul 16 21:14:36.810018 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-ImagesOneServerNegativeTestJSON-526509748 Jul 16 21:14:36.810018 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:14:36.810018 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:14:36.812004 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/4337575622d04a26942905c9dc044ba7"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/4337575622d04a26942905c9dc044ba7"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_4337575622d04a26942905c9dc044ba7"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/4337575622d04a26942905c9dc044ba7"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:14:36.812004 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-ImagesOneServerNegativeTestJSON-526509748 Jul 16 21:14:36.812004 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: 4337575622d04a26942905c9dc044ba7 Jul 16 21:14:36.812004 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-ImagesOneServerNegativeTestJSON-526509748 Jul 16 21:14:36.812004 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-ImagesOneServerNegativeTestJSON-526509748-project Jul 16 21:14:36.812004 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:14:36.812004 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:14:36.812004 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: 4e9df82f95ac4779859972403869096a Jul 16 21:14:36.812004 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-ImagesOneServerNegativeTestJSON-526509748-project Jul 16 21:14:36.812004 ncdv-master-1 devstack@n-api.service[138681]: Jul 16 21:14:36.812004 ncdv-master-1 devstack@n-api.service[138681]: {"createImage": {"name": "tempest-ImagesOneServerNegativeTestJSON-image-1282562837"}} Jul 16 21:14:36.812004 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:14:36.812913 ncdv-master-1 devstack@n-api.service[138681]: exception: Cannot 'createImage' instance e66baae9-d488-46a4-8347-623030fc8b69 while it is in task_state image_snapshot_pending Jul 16 21:14:36.812913 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': 'e66baae9-d488-46a4-8347-623030fc8b69', 'body': {'createImage': {'name': 'tempest-ImagesOneServerNegativeTestJSON-image-1282562837'}}} Jul 16 21:14:36.812913 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:14:36.812913 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:14:36.812913 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1346, in _action_create_image Jul 16 21:14:36.812913 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi image = self.compute_api.snapshot(context, Jul 16 21:14:36.812913 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 152, in inner Jul 16 21:14:36.812913 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.InstanceInvalidState( Jul 16 21:14:36.812913 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.InstanceInvalidState: Instance e66baae9-d488-46a4-8347-623030fc8b69 in task_state image_snapshot_pending. Cannot snapshot while the instance is in this state. Jul 16 21:14:36.812913 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:14:36.812913 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:14:36.812913 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:14:36.812913 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:14:36.812913 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:14:36.812913 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:14:36.812913 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:14:36.812913 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:14:36.812913 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:14:36.812913 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:14:36.814727 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 16 21:14:36.814727 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 16 21:14:36.814727 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 16 21:14:36.814727 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 16 21:14:36.814727 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1351, in _action_create_image Jul 16 21:14:36.814727 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi common.raise_http_conflict_for_instance_invalid_state(state_error, Jul 16 21:14:36.814727 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 347, in raise_http_conflict_for_instance_invalid_state Jul 16 21:14:36.814727 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPConflict(explanation=msg) Jul 16 21:14:36.814727 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPConflict: Cannot 'createImage' instance e66baae9-d488-46a4-8347-623030fc8b69 while it is in task_state image_snapshot_pending Jul 16 21:14:36.814727 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:14:36.814727 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:14:36.814727 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:14:36.814727 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:14:36.814727 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:14:36.814727 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:14:36.814727 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:14:36.814727 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:14:36.814727 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Cannot 'createImage' instance e66baae9-d488-46a4-8347-623030fc8b69 while it is in task_state image_snapshot_pending Jul 16 21:14:36.814727 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:14:36.816801 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-73da82de-d383-42dd-b72a-a43aa990575c tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Returning 409 to user: Cannot 'createImage' instance e66baae9-d488-46a4-8347-623030fc8b69 while it is in task_state image_snapshot_pending {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:14:36.825497 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-73da82de-d383-42dd-b72a-a43aa990575c tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] 10.222.0.16 "POST /compute/v2.1/servers/e66baae9-d488-46a4-8347-623030fc8b69/action" status: 409 len: 167 microversion: 2.1 time: 0.083903 Jul 16 21:14:36.826124 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 747/1553] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 21:14:36 2022] POST /compute/v2.1/servers/e66baae9-d488-46a4-8347-623030fc8b69/action => generated 167 bytes in 86 msecs (HTTP/1.1 409) 9 headers in 378 bytes (1 switches on core 0) Jul 16 21:14:36.848720 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-d89209ef-4e3c-49c6-9f9e-4742566bf5a3 tempest-FloatingIPsTestJSON-1267052927 tempest-FloatingIPsTestJSON-1267052927-project] Calling method '.version_select at 0x7fdd9d8d33a0>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:14:37.250197 ncdv-master-1 devstack@n-api.service[138681]: DEBUG neutronclient.v2_0.client [None req-d89209ef-4e3c-49c6-9f9e-4742566bf5a3 tempest-FloatingIPsTestJSON-1267052927 tempest-FloatingIPsTestJSON-1267052927-project] Error message: {"NeutronError": {"type": "FloatingIPNotFound", "message": "Floating IP cd24ccc6-9821-4496-8e51-56fe56ed84a0 could not be found", "detail": ""}} {{(pid=138681) _handle_fault_response /usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py:262}} Jul 16 21:14:37.251848 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-d89209ef-4e3c-49c6-9f9e-4742566bf5a3 tempest-FloatingIPsTestJSON-1267052927 tempest-FloatingIPsTestJSON-1267052927-project] HTTP exception thrown: Floating IP not found for ID cd24ccc6-9821-4496-8e51-56fe56ed84a0 Jul 16 21:14:37.291659 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-d89209ef-4e3c-49c6-9f9e-4742566bf5a3 tempest-FloatingIPsTestJSON-1267052927 tempest-FloatingIPsTestJSON-1267052927-project] Request method failure captured: Jul 16 21:14:37.291659 ncdv-master-1 devstack@n-api.service[138681]: request: DELETE /compute/v2.1/os-floating-ips/cd24ccc6-9821-4496-8e51-56fe56ed84a0 HTTP/1.1 Jul 16 21:14:37.291659 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:14:37.291659 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:14:37.291659 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:14:37.291659 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 21:14:37.291659 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:14:37.291659 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:14:37.291659 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:14:37.291659 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:14:37.291659 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0yonFOGKRLELAsCe8m1X2Z3-TG2zYxKtr9a40p9bP_XTJyZ895vPVspCAmVCXESKGrpfzA-4iGrX0S47x_B3iAm7yeJaZlwEE4Ev19Sf5YFOTlTWU4Iu7ahUjou3cpmgdeGX5piTC7pTxWIwLFLVDQeUmyk_ZQ9pfN7sipULqaLhdYo Jul 16 21:14:37.291659 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:14:37.291659 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:14:37.291659 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:14:37.291659 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:14:37.291659 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:14:37.291659 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:14:37.291659 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: 538d051aac0f4b66a7f4738f367a1ba8 Jul 16 21:14:37.291659 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-FloatingIPsTestJSON-1267052927 Jul 16 21:14:37.291659 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:14:37.291659 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:14:37.295067 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/538d051aac0f4b66a7f4738f367a1ba8"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/538d051aac0f4b66a7f4738f367a1ba8"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_538d051aac0f4b66a7f4738f367a1ba8"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/538d051aac0f4b66a7f4738f367a1ba8"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:14:37.295067 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-FloatingIPsTestJSON-1267052927 Jul 16 21:14:37.295067 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: 538d051aac0f4b66a7f4738f367a1ba8 Jul 16 21:14:37.295067 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-FloatingIPsTestJSON-1267052927 Jul 16 21:14:37.295067 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-FloatingIPsTestJSON-1267052927-project Jul 16 21:14:37.295067 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:14:37.295067 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:14:37.295067 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: 348961e36710431387915e2220ee3431 Jul 16 21:14:37.295067 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-FloatingIPsTestJSON-1267052927-project Jul 16 21:14:37.295067 ncdv-master-1 devstack@n-api.service[138681]: method: Jul 16 21:14:37.295067 ncdv-master-1 devstack@n-api.service[138681]: exception: Floating IP not found for ID cd24ccc6-9821-4496-8e51-56fe56ed84a0 Jul 16 21:14:37.295067 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': 'cd24ccc6-9821-4496-8e51-56fe56ed84a0'} Jul 16 21:14:37.295067 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:14:37.296109 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:14:37.296109 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 2827, in get_floating_ip Jul 16 21:14:37.296109 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi fip = client.show_floatingip(id)['floatingip'] Jul 16 21:14:37.296109 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 197, in wrapper Jul 16 21:14:37.296109 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Jul 16 21:14:37.296109 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 1006, in show_floatingip Jul 16 21:14:37.296109 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return self.get(self.floatingip_path % (floatingip), params=_params) Jul 16 21:14:37.296109 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 197, in wrapper Jul 16 21:14:37.296109 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Jul 16 21:14:37.296109 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 356, in get Jul 16 21:14:37.296109 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return self.retry_request("GET", action, body=body, Jul 16 21:14:37.296109 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 197, in wrapper Jul 16 21:14:37.296109 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Jul 16 21:14:37.296109 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 333, in retry_request Jul 16 21:14:37.296109 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return self.do_request(method, action, body=body, Jul 16 21:14:37.296109 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 197, in wrapper Jul 16 21:14:37.296109 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Jul 16 21:14:37.296109 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 297, in do_request Jul 16 21:14:37.296109 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi self._handle_fault_response(status_code, replybody, resp) Jul 16 21:14:37.297492 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 197, in wrapper Jul 16 21:14:37.297492 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Jul 16 21:14:37.297492 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 272, in _handle_fault_response Jul 16 21:14:37.297492 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi exception_handler_v20(status_code, error_body) Jul 16 21:14:37.297492 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 90, in exception_handler_v20 Jul 16 21:14:37.297492 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise client_exc(message=error_message, Jul 16 21:14:37.297492 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi neutronclient.common.exceptions.NotFound: Floating IP cd24ccc6-9821-4496-8e51-56fe56ed84a0 could not be found Jul 16 21:14:37.297492 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-c597f980-c906-43b6-888e-316a3aef8e85'] Jul 16 21:14:37.297492 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:14:37.297492 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:14:37.297492 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:14:37.297492 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:14:37.297492 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/floating_ips.py", line 158, in delete Jul 16 21:14:37.297492 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi floating_ip = self.network_api.get_floating_ip(context, id) Jul 16 21:14:37.297492 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 2830, in get_floating_ip Jul 16 21:14:37.297492 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.FloatingIpNotFound(id=id) Jul 16 21:14:37.297492 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.FloatingIpNotFound: Floating IP not found for ID cd24ccc6-9821-4496-8e51-56fe56ed84a0. Jul 16 21:14:37.297492 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:14:37.297492 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:14:37.297492 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:14:37.298946 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:14:37.298946 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:14:37.298946 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:14:37.298946 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:14:37.298946 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:14:37.298946 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Jul 16 21:14:37.298946 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Jul 16 21:14:37.298946 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:14:37.298946 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:14:37.298946 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/floating_ips.py", line 161, in delete Jul 16 21:14:37.298946 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=msg) Jul 16 21:14:37.298946 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Floating IP not found for ID cd24ccc6-9821-4496-8e51-56fe56ed84a0 Jul 16 21:14:37.298946 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:14:37.298946 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:14:37.298946 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:14:37.298946 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:14:37.298946 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:14:37.298946 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:14:37.298946 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:14:37.298946 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:14:37.300402 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Floating IP not found for ID cd24ccc6-9821-4496-8e51-56fe56ed84a0 Jul 16 21:14:37.300402 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:14:37.300402 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-d89209ef-4e3c-49c6-9f9e-4742566bf5a3 tempest-FloatingIPsTestJSON-1267052927 tempest-FloatingIPsTestJSON-1267052927-project] Returning 404 to user: Floating IP not found for ID cd24ccc6-9821-4496-8e51-56fe56ed84a0 {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:14:37.300402 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-d89209ef-4e3c-49c6-9f9e-4742566bf5a3 tempest-FloatingIPsTestJSON-1267052927 tempest-FloatingIPsTestJSON-1267052927-project] 10.222.0.16 "DELETE /compute/v2.1/os-floating-ips/cd24ccc6-9821-4496-8e51-56fe56ed84a0" status: 404 len: 111 microversion: 2.1 time: 0.429942 Jul 16 21:14:37.300402 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 748/1554] 10.222.0.16 () {58 vars in 1262 bytes} [Sat Jul 16 21:14:36 2022] DELETE /compute/v2.1/os-floating-ips/cd24ccc6-9821-4496-8e51-56fe56ed84a0 => generated 111 bytes in 431 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:14:37.300402 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-44f9b636-2f31-4c25-98da-76a3e4797618 tempest-ServersAdminNegativeTestJSON-1884193353 tempest-ServersAdminNegativeTestJSON-1884193353-project-admin] Action: 'action', calling method: >, body: {"os-resetState": {"state": 1}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:14:37.300402 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-44f9b636-2f31-4c25-98da-76a3e4797618 tempest-ServersAdminNegativeTestJSON-1884193353 tempest-ServersAdminNegativeTestJSON-1884193353-project-admin] Request method failure captured: Jul 16 21:14:37.301009 ncdv-master-1 devstack@n-api.service[138681]: request: POST /compute/v2.1/servers/4602867b-88ed-43b1-9ee6-bf661e2d7d99/action HTTP/1.1 Jul 16 21:14:37.301009 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:14:37.301009 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:14:37.301009 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:14:37.301009 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 31 Jul 16 21:14:37.301009 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:14:37.301009 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:14:37.301009 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:14:37.301009 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:14:37.301009 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0yn7QG_tnLYMptx-bTEUJY8ylzBGFOwnadoDojC2tCzwJeRalDhG_GURBfSS7Jpd1amigWw5Nbi2c_3Jq_jqZUD7WjqZe8pZKa1SRAQVw-5Rq-fp6JQvZsa6miiKLCZoWSEIKgl6wCp7cEj6e1JS1FO8OqdXGKZOwT9NqIcGygKtteg Jul 16 21:14:37.301009 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:14:37.301009 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:14:37.301009 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:14:37.301009 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:14:37.301009 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:14:37.301009 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:14:37.301009 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: 38594b6bc3354178a22f0abb698e98a6 Jul 16 21:14:37.301009 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-ServersAdminNegativeTestJSON-1884193353 Jul 16 21:14:37.301009 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,admin,reader Jul 16 21:14:37.301009 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,admin,reader Jul 16 21:14:37.302936 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/38594b6bc3354178a22f0abb698e98a6"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/38594b6bc3354178a22f0abb698e98a6"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_38594b6bc3354178a22f0abb698e98a6"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/38594b6bc3354178a22f0abb698e98a6"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:14:37.302936 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-ServersAdminNegativeTestJSON-1884193353 Jul 16 21:14:37.302936 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: 38594b6bc3354178a22f0abb698e98a6 Jul 16 21:14:37.302936 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-ServersAdminNegativeTestJSON-1884193353 Jul 16 21:14:37.302936 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-ServersAdminNegativeTestJSON-1884193353-project-admin Jul 16 21:14:37.302936 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:14:37.302936 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:14:37.302936 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: 27aeaef0a671464493a8cac846186af7 Jul 16 21:14:37.302936 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-ServersAdminNegativeTestJSON-1884193353-project-admin Jul 16 21:14:37.302936 ncdv-master-1 devstack@n-api.service[138681]: Jul 16 21:14:37.302936 ncdv-master-1 devstack@n-api.service[138681]: {"os-resetState": {"state": 1}} Jul 16 21:14:37.302936 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:14:37.302936 ncdv-master-1 devstack@n-api.service[138681]: exception: Invalid input for field/attribute state. Value: 1. 1 is not of type 'string' Jul 16 21:14:37.307734 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': '4602867b-88ed-43b1-9ee6-bf661e2d7d99', 'body': {'os-resetState': {'state': 1}}} Jul 16 21:14:37.307734 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:14:37.307734 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:14:37.307734 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 279, in validate Jul 16 21:14:37.307734 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Jul 16 21:14:37.307734 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/lib/python3/dist-packages/jsonschema/validators.py", line 353, in validate Jul 16 21:14:37.307734 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise error Jul 16 21:14:37.307734 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: 1 is not of type 'string' Jul 16 21:14:37.307734 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:14:37.307734 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Failed validating 'type' in schema['properties']['os-resetState']['properties']['state']: Jul 16 21:14:37.307734 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi {'enum': ['active', 'error'], 'type': 'string'} Jul 16 21:14:37.307734 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:14:37.307734 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi On instance['os-resetState']: Jul 16 21:14:37.307734 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi 1 Jul 16 21:14:37.307734 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:14:37.307734 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:14:37.307734 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:14:37.307734 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:14:37.307734 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:14:37.307734 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:14:37.307734 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:14:37.307734 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:14:37.310537 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:14:37.310537 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:14:37.310537 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Jul 16 21:14:37.310537 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Jul 16 21:14:37.310537 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Jul 16 21:14:37.310537 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Jul 16 21:14:37.310537 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 308, in validate Jul 16 21:14:37.310537 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Jul 16 21:14:37.310537 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute state. Value: 1. 1 is not of type 'string' Jul 16 21:14:37.310537 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:14:37.310537 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:14:37.310537 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:14:37.310537 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:14:37.310537 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:14:37.310537 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:14:37.310537 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Jul 16 21:14:37.310537 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Jul 16 21:14:37.310537 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute state. Value: 1. 1 is not of type 'string' Jul 16 21:14:37.310537 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:14:37.312899 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-44f9b636-2f31-4c25-98da-76a3e4797618 tempest-ServersAdminNegativeTestJSON-1884193353 tempest-ServersAdminNegativeTestJSON-1884193353-project-admin] Returning 400 to user: Invalid input for field/attribute state. Value: 1. 1 is not of type 'string' {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:14:37.312899 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-44f9b636-2f31-4c25-98da-76a3e4797618 tempest-ServersAdminNegativeTestJSON-1884193353 tempest-ServersAdminNegativeTestJSON-1884193353-project-admin] 10.222.0.16 "POST /compute/v2.1/servers/4602867b-88ed-43b1-9ee6-bf661e2d7d99/action" status: 400 len: 120 microversion: 2.1 time: 0.011126 Jul 16 21:14:37.312899 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 749/1555] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 21:14:37 2022] POST /compute/v2.1/servers/4602867b-88ed-43b1-9ee6-bf661e2d7d99/action => generated 120 bytes in 13 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Jul 16 21:14:37.312899 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-2e84f5fa-9a58-44e4-beba-8a1af328b69f tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Calling method '.version_select at 0x7fdd9d008430>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:14:37.705041 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-2e84f5fa-9a58-44e4-beba-8a1af328b69f tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] 10.222.0.16 "DELETE /compute/v2.1/images/55ce9fb1-ab88-4209-bcf9-89b9a3541595" status: 204 len: 0 microversion: 2.1 time: 0.432391 Jul 16 21:14:37.709432 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 750/1556] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:14:37 2022] DELETE /compute/v2.1/images/55ce9fb1-ab88-4209-bcf9-89b9a3541595 => generated 0 bytes in 434 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 16 21:14:37.712622 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-f53109aa-d48c-4bb4-9b52-645c6dcca94d tempest-ServersAdminNegativeTestJSON-1884193353 tempest-ServersAdminNegativeTestJSON-1884193353-project-admin] Action: 'action', calling method: >, body: {"os-resetState": {"state": "error"}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:14:37.714250 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-f53109aa-d48c-4bb4-9b52-645c6dcca94d tempest-ServersAdminNegativeTestJSON-1884193353 tempest-ServersAdminNegativeTestJSON-1884193353-project-admin] Failed to fetch instance by id 999 {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2926}} Jul 16 21:14:37.715124 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-f53109aa-d48c-4bb4-9b52-645c6dcca94d tempest-ServersAdminNegativeTestJSON-1884193353 tempest-ServersAdminNegativeTestJSON-1884193353-project-admin] HTTP exception thrown: Instance 999 could not be found. Jul 16 21:14:37.717378 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-f53109aa-d48c-4bb4-9b52-645c6dcca94d tempest-ServersAdminNegativeTestJSON-1884193353 tempest-ServersAdminNegativeTestJSON-1884193353-project-admin] Request method failure captured: Jul 16 21:14:37.717378 ncdv-master-1 devstack@n-api.service[138681]: request: POST /compute/v2.1/servers/999/action HTTP/1.1 Jul 16 21:14:37.717378 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:14:37.717378 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:14:37.717378 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:14:37.717378 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 37 Jul 16 21:14:37.717378 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:14:37.717378 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:14:37.717378 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:14:37.717378 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:14:37.717378 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0yn7QG_tnLYMptx-bTEUJY8ylzBGFOwnadoDojC2tCzwJeRalDhG_GURBfSS7Jpd1amigWw5Nbi2c_3Jq_jqZUD7WjqZe8pZKa1SRAQVw-5Rq-fp6JQvZsa6miiKLCZoWSEIKgl6wCp7cEj6e1JS1FO8OqdXGKZOwT9NqIcGygKtteg Jul 16 21:14:37.717378 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:14:37.717378 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:14:37.717378 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:14:37.717378 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:14:37.717378 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:14:37.717378 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:14:37.717378 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: 38594b6bc3354178a22f0abb698e98a6 Jul 16 21:14:37.717378 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-ServersAdminNegativeTestJSON-1884193353 Jul 16 21:14:37.717378 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,admin,reader Jul 16 21:14:37.717378 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,admin,reader Jul 16 21:14:37.718736 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/38594b6bc3354178a22f0abb698e98a6"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/38594b6bc3354178a22f0abb698e98a6"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_38594b6bc3354178a22f0abb698e98a6"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/38594b6bc3354178a22f0abb698e98a6"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:14:37.718736 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-ServersAdminNegativeTestJSON-1884193353 Jul 16 21:14:37.718736 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: 38594b6bc3354178a22f0abb698e98a6 Jul 16 21:14:37.718736 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-ServersAdminNegativeTestJSON-1884193353 Jul 16 21:14:37.718736 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-ServersAdminNegativeTestJSON-1884193353-project-admin Jul 16 21:14:37.718736 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:14:37.718736 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:14:37.718736 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: 27aeaef0a671464493a8cac846186af7 Jul 16 21:14:37.718736 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-ServersAdminNegativeTestJSON-1884193353-project-admin Jul 16 21:14:37.718736 ncdv-master-1 devstack@n-api.service[138681]: Jul 16 21:14:37.718736 ncdv-master-1 devstack@n-api.service[138681]: {"os-resetState": {"state": "error"}} Jul 16 21:14:37.718736 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:14:37.718736 ncdv-master-1 devstack@n-api.service[138681]: exception: Instance 999 could not be found. Jul 16 21:14:37.719634 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': '999', 'body': {'os-resetState': {'state': 'error'}}} Jul 16 21:14:37.719634 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:14:37.719634 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:14:37.719634 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:14:37.719634 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:14:37.719634 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2927, in get Jul 16 21:14:37.719634 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=instance_id) Jul 16 21:14:37.719634 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 999 could not be found. Jul 16 21:14:37.719634 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:14:37.719634 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:14:37.719634 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:14:37.719634 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:14:37.719634 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:14:37.719634 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:14:37.719634 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:14:37.719634 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:14:37.719634 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:14:37.719634 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:14:37.719634 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 16 21:14:37.719634 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 16 21:14:37.719634 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/admin_actions.py", line 66, in _reset_state Jul 16 21:14:37.721248 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, id) Jul 16 21:14:37.721248 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:14:37.721248 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:14:37.721248 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 999 could not be found. Jul 16 21:14:37.721248 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:14:37.721248 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:14:37.721248 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:14:37.721248 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:14:37.721248 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:14:37.721248 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:14:37.721248 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:14:37.721248 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:14:37.721248 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 999 could not be found. Jul 16 21:14:37.721248 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:14:37.724765 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-f53109aa-d48c-4bb4-9b52-645c6dcca94d tempest-ServersAdminNegativeTestJSON-1884193353 tempest-ServersAdminNegativeTestJSON-1884193353-project-admin] Returning 404 to user: Instance 999 could not be found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:14:37.726162 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-f53109aa-d48c-4bb4-9b52-645c6dcca94d tempest-ServersAdminNegativeTestJSON-1884193353 tempest-ServersAdminNegativeTestJSON-1884193353-project-admin] 10.222.0.16 "POST /compute/v2.1/servers/999/action" status: 404 len: 78 microversion: 2.1 time: 0.018616 Jul 16 21:14:37.730036 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 751/1557] 10.222.0.16 () {60 vars in 1178 bytes} [Sat Jul 16 21:14:37 2022] POST /compute/v2.1/servers/999/action => generated 78 bytes in 23 msecs (HTTP/1.1 404) 9 headers in 378 bytes (1 switches on core 0) Jul 16 21:14:37.736824 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-5f18eb1e-6eb8-4147-9401-05af7b130520 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:14:37.736824 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-5f18eb1e-6eb8-4147-9401-05af7b130520 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] [instance: 69cabe57-da71-4a02-b4ee-c0d84b3d32a2] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:14:37.747600 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5f18eb1e-6eb8-4147-9401-05af7b130520 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:37.748064 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5f18eb1e-6eb8-4147-9401-05af7b130520 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:37.748911 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5f18eb1e-6eb8-4147-9401-05af7b130520 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:37.749566 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5f18eb1e-6eb8-4147-9401-05af7b130520 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:38.036016 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-5f18eb1e-6eb8-4147-9401-05af7b130520 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '53af488f3b0242a797f539284b78d59e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bb1ede0d0b6941cc97278e6c617f9b01', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:14:38.048605 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-5f18eb1e-6eb8-4147-9401-05af7b130520 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] 10.222.0.16 "GET /compute/v2.1/servers/69cabe57-da71-4a02-b4ee-c0d84b3d32a2" status: 200 len: 1354 microversion: 2.1 time: 0.316328 Jul 16 21:14:38.049688 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 752/1558] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:14:37 2022] GET /compute/v2.1/servers/69cabe57-da71-4a02-b4ee-c0d84b3d32a2 => generated 1354 bytes in 319 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:14:38.059025 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-eeeca617-bdc5-477d-abaf-1d0ee2648925 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:14:38.059656 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-eeeca617-bdc5-477d-abaf-1d0ee2648925 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] [instance: e66baae9-d488-46a4-8347-623030fc8b69] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:14:38.067573 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-eeeca617-bdc5-477d-abaf-1d0ee2648925 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:38.068203 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-eeeca617-bdc5-477d-abaf-1d0ee2648925 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:38.069054 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-eeeca617-bdc5-477d-abaf-1d0ee2648925 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:38.069502 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-eeeca617-bdc5-477d-abaf-1d0ee2648925 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:38.275489 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-eeeca617-bdc5-477d-abaf-1d0ee2648925 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4e9df82f95ac4779859972403869096a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4337575622d04a26942905c9dc044ba7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:14:38.278851 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-927b4c00-2cc1-49ef-b01d-5cbde5a4aa66 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] 10.222.0.16 "DELETE /compute/v2.1/os-floating-ips/793f33e8-b858-468e-999d-b178a1b1313b" status: 202 len: 0 microversion: 2.1 time: 1.745084 Jul 16 21:14:38.279555 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 807/1559] 10.222.0.16 () {58 vars in 1262 bytes} [Sat Jul 16 21:14:36 2022] DELETE /compute/v2.1/os-floating-ips/793f33e8-b858-468e-999d-b178a1b1313b => generated 0 bytes in 1747 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 21:14:38.286037 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-eeeca617-bdc5-477d-abaf-1d0ee2648925 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] 10.222.0.16 "GET /compute/v2.1/servers/e66baae9-d488-46a4-8347-623030fc8b69" status: 200 len: 1538 microversion: 2.1 time: 0.234931 Jul 16 21:14:38.287487 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 753/1560] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:14:38 2022] GET /compute/v2.1/servers/e66baae9-d488-46a4-8347-623030fc8b69 => generated 1538 bytes in 237 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:14:38.287863 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-5ad84ea5-5c51-47a3-88e1-fdf35c7ae2d1 tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:14:38.288847 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-5ad84ea5-5c51-47a3-88e1-fdf35c7ae2d1 tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] [instance: 76ee9d21-1f17-4b64-8f95-95a33e70e907] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:14:38.294689 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-475dc8e6-d16a-441b-8fbe-2b75b6a8cef8 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:14:38.295999 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-475dc8e6-d16a-441b-8fbe-2b75b6a8cef8 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] [instance: 3c5ab09e-0a97-4cfc-b249-a11ab221b5d2] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:14:38.303514 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5ad84ea5-5c51-47a3-88e1-fdf35c7ae2d1 tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:38.305082 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5ad84ea5-5c51-47a3-88e1-fdf35c7ae2d1 tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:38.305942 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5ad84ea5-5c51-47a3-88e1-fdf35c7ae2d1 tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:38.306403 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5ad84ea5-5c51-47a3-88e1-fdf35c7ae2d1 tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:38.306957 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-475dc8e6-d16a-441b-8fbe-2b75b6a8cef8 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:38.307803 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-475dc8e6-d16a-441b-8fbe-2b75b6a8cef8 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:38.308782 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-475dc8e6-d16a-441b-8fbe-2b75b6a8cef8 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:38.309376 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-475dc8e6-d16a-441b-8fbe-2b75b6a8cef8 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:38.333121 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-5ad84ea5-5c51-47a3-88e1-fdf35c7ae2d1 tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] [instance: 76ee9d21-1f17-4b64-8f95-95a33e70e907] Going to try to terminate instance {{(pid=138680) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 16 21:14:38.345167 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-475dc8e6-d16a-441b-8fbe-2b75b6a8cef8 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] 10.222.0.16 "GET /compute/v2.1/servers/3c5ab09e-0a97-4cfc-b249-a11ab221b5d2/os-volume_attachments" status: 200 len: 25 microversion: 2.25 time: 0.055790 Jul 16 21:14:38.349837 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 754/1561] 10.222.0.16 () {60 vars in 1342 bytes} [Sat Jul 16 21:14:38 2022] GET /compute/v2.1/servers/3c5ab09e-0a97-4cfc-b249-a11ab221b5d2/os-volume_attachments => generated 25 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:14:38.355437 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-51bacc96-fc6b-44ee-bcce-3281acab7025 req-5f710bd2-5254-4dc8-a6fd-0b85d38d5c3a service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "69cabe57-da71-4a02-b4ee-c0d84b3d32a2", "name": "network-vif-plugged", "status": "completed", "tag": "1f354c18-a81d-412a-97e0-43d35608c82c"}, {"server_uuid": "69cabe57-da71-4a02-b4ee-c0d84b3d32a2", "name": "network-vif-plugged", "status": "completed", "tag": "da98933d-c6aa-4341-b47f-1483a9e3aa36"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:14:38.365634 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-51bacc96-fc6b-44ee-bcce-3281acab7025 req-5f710bd2-5254-4dc8-a6fd-0b85d38d5c3a service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:38.366440 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-51bacc96-fc6b-44ee-bcce-3281acab7025 req-5f710bd2-5254-4dc8-a6fd-0b85d38d5c3a service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:38.379506 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-51bacc96-fc6b-44ee-bcce-3281acab7025 req-5f710bd2-5254-4dc8-a6fd-0b85d38d5c3a service nova] Creating event network-vif-plugged:1f354c18-a81d-412a-97e0-43d35608c82c for instance 69cabe57-da71-4a02-b4ee-c0d84b3d32a2 on nch1-master-1 Jul 16 21:14:38.380023 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-51bacc96-fc6b-44ee-bcce-3281acab7025 req-5f710bd2-5254-4dc8-a6fd-0b85d38d5c3a service nova] Creating event network-vif-plugged:da98933d-c6aa-4341-b47f-1483a9e3aa36 for instance 69cabe57-da71-4a02-b4ee-c0d84b3d32a2 on nch1-master-1 Jul 16 21:14:38.385434 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-51bacc96-fc6b-44ee-bcce-3281acab7025 req-5f710bd2-5254-4dc8-a6fd-0b85d38d5c3a service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 354 microversion: 2.1 time: 0.038248 Jul 16 21:14:38.386607 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 755/1562] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:14:38 2022] POST /compute/v2.1/os-server-external-events => generated 354 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:14:38.395635 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-aee2bb87-df7a-4d9c-8338-9515cb48956c tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] Action: 'action', calling method: , body: {"addFloatingIp": {"address": ""}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:14:38.398145 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-aee2bb87-df7a-4d9c-8338-9515cb48956c tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] Request method failure captured: Jul 16 21:14:38.398145 ncdv-master-1 devstack@n-api.service[138681]: request: POST /compute/v2.1/servers/dada2ea0-2ef4-4a30-8e2d-40177d0bafce/action HTTP/1.1 Jul 16 21:14:38.398145 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:14:38.398145 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:14:38.398145 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:14:38.398145 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 34 Jul 16 21:14:38.398145 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:14:38.398145 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:14:38.398145 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:14:38.398145 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:14:38.398145 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0yoB0XYghSUBtF84YOYGfB4_xYzUA_F4WrRobF-Rt4qMvE82h1iXAkIM12_RAx3rpkycxMLJn3NNCBruBkg08dFc8vTeMt2laJatIrAsrctW_jXBNb9rR0ETfgDp7wIjwtKtViFNgoI7i99au4p9qB9nlk-sgKFBAUCsSzgT232xMd8 Jul 16 21:14:38.398145 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:14:38.398145 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:14:38.398145 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:14:38.398145 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:14:38.398145 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:14:38.398145 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:14:38.398145 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: 6c2095f96f8d45bb9e62c517678c3dcd Jul 16 21:14:38.398145 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 Jul 16 21:14:38.398145 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:14:38.398145 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:14:38.399861 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/6c2095f96f8d45bb9e62c517678c3dcd"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/6c2095f96f8d45bb9e62c517678c3dcd"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_6c2095f96f8d45bb9e62c517678c3dcd"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/6c2095f96f8d45bb9e62c517678c3dcd"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:14:38.399861 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 Jul 16 21:14:38.399861 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: 6c2095f96f8d45bb9e62c517678c3dcd Jul 16 21:14:38.399861 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 Jul 16 21:14:38.399861 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project Jul 16 21:14:38.399861 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:14:38.399861 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:14:38.399861 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: 2eba2ac21c344394a7d3f605775b8713 Jul 16 21:14:38.399861 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project Jul 16 21:14:38.399861 ncdv-master-1 devstack@n-api.service[138681]: Jul 16 21:14:38.399861 ncdv-master-1 devstack@n-api.service[138681]: {"addFloatingIp": {"address": ""}} Jul 16 21:14:38.399861 ncdv-master-1 devstack@n-api.service[138681]: method: Jul 16 21:14:38.399861 ncdv-master-1 devstack@n-api.service[138681]: exception: Invalid input for field/attribute address. Value: . '' is not valid under any of the given schemas Jul 16 21:14:38.404900 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': 'dada2ea0-2ef4-4a30-8e2d-40177d0bafce', 'body': {'addFloatingIp': {'address': ''}}} Jul 16 21:14:38.404900 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:14:38.404900 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:14:38.404900 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 279, in validate Jul 16 21:14:38.404900 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Jul 16 21:14:38.404900 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/lib/python3/dist-packages/jsonschema/validators.py", line 353, in validate Jul 16 21:14:38.404900 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise error Jul 16 21:14:38.404900 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: '' is not valid under any of the given schemas Jul 16 21:14:38.404900 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:14:38.404900 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Failed validating 'oneOf' in schema['properties']['addFloatingIp']['properties']['address']: Jul 16 21:14:38.404900 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi {'oneOf': [{'format': 'ipv4'}, {'format': 'ipv6'}], 'type': 'string'} Jul 16 21:14:38.404900 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:14:38.404900 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi On instance['addFloatingIp']: Jul 16 21:14:38.404900 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi '' Jul 16 21:14:38.404900 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:14:38.404900 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:14:38.404900 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:14:38.404900 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:14:38.404900 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:14:38.404900 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:14:38.404900 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:14:38.404900 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:14:38.406429 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Jul 16 21:14:38.406429 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Jul 16 21:14:38.406429 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:14:38.406429 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:14:38.406429 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Jul 16 21:14:38.406429 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Jul 16 21:14:38.406429 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Jul 16 21:14:38.406429 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Jul 16 21:14:38.406429 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 308, in validate Jul 16 21:14:38.406429 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Jul 16 21:14:38.406429 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute address. Value: . '' is not valid under any of the given schemas Jul 16 21:14:38.406429 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:14:38.406429 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:14:38.406429 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:14:38.406429 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:14:38.406429 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:14:38.406429 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:14:38.406429 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Jul 16 21:14:38.406429 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Jul 16 21:14:38.407631 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute address. Value: . '' is not valid under any of the given schemas Jul 16 21:14:38.407631 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:14:38.407631 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-aee2bb87-df7a-4d9c-8338-9515cb48956c tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] Returning 400 to user: Invalid input for field/attribute address. Value: . '' is not valid under any of the given schemas {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:14:38.407631 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-aee2bb87-df7a-4d9c-8338-9515cb48956c tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] 10.222.0.16 "POST /compute/v2.1/servers/dada2ea0-2ef4-4a30-8e2d-40177d0bafce/action" status: 400 len: 142 microversion: 2.1 time: 0.008244 Jul 16 21:14:38.407631 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 756/1563] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 21:14:38 2022] POST /compute/v2.1/servers/dada2ea0-2ef4-4a30-8e2d-40177d0bafce/action => generated 142 bytes in 10 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Jul 16 21:14:38.409012 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-2282fbc4-337f-492c-bd85-e34b7d5ce25c tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] Action: 'action', calling method: , body: {"addFloatingIp": {"address": "0.0.0.0"}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:14:38.410191 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-2282fbc4-337f-492c-bd85-e34b7d5ce25c tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] [instance: dada2ea0-2ef4-4a30-8e2d-40177d0bafce] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:14:38.418685 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2282fbc4-337f-492c-bd85-e34b7d5ce25c tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:38.419241 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2282fbc4-337f-492c-bd85-e34b7d5ce25c tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:38.420201 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2282fbc4-337f-492c-bd85-e34b7d5ce25c tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:38.420648 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2282fbc4-337f-492c-bd85-e34b7d5ce25c tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:38.460638 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2282fbc4-337f-492c-bd85-e34b7d5ce25c tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] Acquired lock "refresh_cache-dada2ea0-2ef4-4a30-8e2d-40177d0bafce" {{(pid=138681) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:294}} Jul 16 21:14:38.688579 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2282fbc4-337f-492c-bd85-e34b7d5ce25c tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] Releasing lock "refresh_cache-dada2ea0-2ef4-4a30-8e2d-40177d0bafce" {{(pid=138681) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:312}} Jul 16 21:14:38.689489 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-2282fbc4-337f-492c-bd85-e34b7d5ce25c tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] HTTP exception thrown: floating IP not found Jul 16 21:14:38.691474 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-2282fbc4-337f-492c-bd85-e34b7d5ce25c tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] Request method failure captured: Jul 16 21:14:38.691474 ncdv-master-1 devstack@n-api.service[138681]: request: POST /compute/v2.1/servers/dada2ea0-2ef4-4a30-8e2d-40177d0bafce/action HTTP/1.1 Jul 16 21:14:38.691474 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:14:38.691474 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:14:38.691474 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:14:38.691474 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 41 Jul 16 21:14:38.691474 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:14:38.691474 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:14:38.691474 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:14:38.691474 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:14:38.691474 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0yoB0XYghSUBtF84YOYGfB4_xYzUA_F4WrRobF-Rt4qMvE82h1iXAkIM12_RAx3rpkycxMLJn3NNCBruBkg08dFc8vTeMt2laJatIrAsrctW_jXBNb9rR0ETfgDp7wIjwtKtViFNgoI7i99au4p9qB9nlk-sgKFBAUCsSzgT232xMd8 Jul 16 21:14:38.691474 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:14:38.691474 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:14:38.691474 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:14:38.691474 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:14:38.691474 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:14:38.691474 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:14:38.691474 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: 6c2095f96f8d45bb9e62c517678c3dcd Jul 16 21:14:38.691474 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 Jul 16 21:14:38.691474 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:14:38.691474 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:14:38.692856 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/6c2095f96f8d45bb9e62c517678c3dcd"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/6c2095f96f8d45bb9e62c517678c3dcd"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_6c2095f96f8d45bb9e62c517678c3dcd"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/6c2095f96f8d45bb9e62c517678c3dcd"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:14:38.692856 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 Jul 16 21:14:38.692856 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: 6c2095f96f8d45bb9e62c517678c3dcd Jul 16 21:14:38.692856 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 Jul 16 21:14:38.692856 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project Jul 16 21:14:38.692856 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:14:38.692856 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:14:38.692856 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: 2eba2ac21c344394a7d3f605775b8713 Jul 16 21:14:38.692856 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project Jul 16 21:14:38.692856 ncdv-master-1 devstack@n-api.service[138681]: Jul 16 21:14:38.692856 ncdv-master-1 devstack@n-api.service[138681]: {"addFloatingIp": {"address": "0.0.0.0"}} Jul 16 21:14:38.692856 ncdv-master-1 devstack@n-api.service[138681]: method: Jul 16 21:14:38.692856 ncdv-master-1 devstack@n-api.service[138681]: exception: floating IP not found Jul 16 21:14:38.694868 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': 'dada2ea0-2ef4-4a30-8e2d-40177d0bafce', 'body': {'addFloatingIp': {'address': '0.0.0.0'}}} Jul 16 21:14:38.694868 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:14:38.694868 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:14:38.694868 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/floating_ips.py", line 239, in _add_floating_ip Jul 16 21:14:38.694868 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi self.network_api.associate_floating_ip(context, instance, Jul 16 21:14:38.694868 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 166, in wrapper Jul 16 21:14:38.694868 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi res = f(self, context, *args, **kwargs) Jul 16 21:14:38.694868 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 2702, in associate_floating_ip Jul 16 21:14:38.694868 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi fip = self._get_floating_ip_by_address(client, floating_address) Jul 16 21:14:38.694868 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 3014, in _get_floating_ip_by_address Jul 16 21:14:38.694868 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.FloatingIpNotFoundForAddress(address=address) Jul 16 21:14:38.694868 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.FloatingIpNotFoundForAddress: Floating IP not found for address 0.0.0.0. Jul 16 21:14:38.694868 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:14:38.694868 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:14:38.694868 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:14:38.694868 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:14:38.694868 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:14:38.694868 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:14:38.694868 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:14:38.696202 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:14:38.696202 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Jul 16 21:14:38.696202 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Jul 16 21:14:38.696202 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:14:38.696202 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:14:38.696202 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 16 21:14:38.696202 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 16 21:14:38.696202 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/floating_ips.py", line 252, in _add_floating_ip Jul 16 21:14:38.696202 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=msg) Jul 16 21:14:38.696202 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: floating IP not found Jul 16 21:14:38.696202 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:14:38.696202 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:14:38.696202 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:14:38.696202 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:14:38.696202 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:14:38.696202 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:14:38.696202 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:14:38.696202 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:14:38.696202 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: floating IP not found Jul 16 21:14:38.696202 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:14:38.706394 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-2282fbc4-337f-492c-bd85-e34b7d5ce25c tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] Returning 404 to user: floating IP not found {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:14:38.708072 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-2282fbc4-337f-492c-bd85-e34b7d5ce25c tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] 10.222.0.16 "POST /compute/v2.1/servers/dada2ea0-2ef4-4a30-8e2d-40177d0bafce/action" status: 404 len: 67 microversion: 2.1 time: 0.302915 Jul 16 21:14:38.715150 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 757/1564] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 21:14:38 2022] POST /compute/v2.1/servers/dada2ea0-2ef4-4a30-8e2d-40177d0bafce/action => generated 67 bytes in 310 msecs (HTTP/1.1 404) 9 headers in 378 bytes (1 switches on core 0) Jul 16 21:14:38.727632 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-9a89759c-60c0-4267-8dde-476d6e0200a4 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] Action: 'action', calling method: , body: {"removeFloatingIp": {"address": "0.0.0.0"}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:14:38.807124 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-9a89759c-60c0-4267-8dde-476d6e0200a4 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] HTTP exception thrown: Floating IP not found for address 0.0.0.0. Jul 16 21:14:38.809197 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-9a89759c-60c0-4267-8dde-476d6e0200a4 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] Request method failure captured: Jul 16 21:14:38.809197 ncdv-master-1 devstack@n-api.service[138681]: request: POST /compute/v2.1/servers/dada2ea0-2ef4-4a30-8e2d-40177d0bafce/action HTTP/1.1 Jul 16 21:14:38.809197 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:14:38.809197 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:14:38.809197 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:14:38.809197 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 44 Jul 16 21:14:38.809197 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:14:38.809197 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:14:38.809197 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:14:38.809197 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:14:38.809197 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0yoB0XYghSUBtF84YOYGfB4_xYzUA_F4WrRobF-Rt4qMvE82h1iXAkIM12_RAx3rpkycxMLJn3NNCBruBkg08dFc8vTeMt2laJatIrAsrctW_jXBNb9rR0ETfgDp7wIjwtKtViFNgoI7i99au4p9qB9nlk-sgKFBAUCsSzgT232xMd8 Jul 16 21:14:38.809197 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:14:38.809197 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:14:38.809197 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:14:38.809197 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:14:38.809197 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:14:38.809197 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:14:38.809197 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: 6c2095f96f8d45bb9e62c517678c3dcd Jul 16 21:14:38.809197 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 Jul 16 21:14:38.809197 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:14:38.809197 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:14:38.810564 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/6c2095f96f8d45bb9e62c517678c3dcd"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/6c2095f96f8d45bb9e62c517678c3dcd"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_6c2095f96f8d45bb9e62c517678c3dcd"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/6c2095f96f8d45bb9e62c517678c3dcd"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:14:38.810564 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 Jul 16 21:14:38.810564 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: 6c2095f96f8d45bb9e62c517678c3dcd Jul 16 21:14:38.810564 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 Jul 16 21:14:38.810564 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project Jul 16 21:14:38.810564 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:14:38.810564 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:14:38.810564 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: 2eba2ac21c344394a7d3f605775b8713 Jul 16 21:14:38.810564 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project Jul 16 21:14:38.810564 ncdv-master-1 devstack@n-api.service[138681]: Jul 16 21:14:38.810564 ncdv-master-1 devstack@n-api.service[138681]: {"removeFloatingIp": {"address": "0.0.0.0"}} Jul 16 21:14:38.810564 ncdv-master-1 devstack@n-api.service[138681]: method: Jul 16 21:14:38.810564 ncdv-master-1 devstack@n-api.service[138681]: exception: Floating IP not found for address 0.0.0.0. Jul 16 21:14:38.811502 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': 'dada2ea0-2ef4-4a30-8e2d-40177d0bafce', 'body': {'removeFloatingIp': {'address': '0.0.0.0'}}} Jul 16 21:14:38.811502 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:14:38.811502 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:14:38.811502 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/floating_ips.py", line 58, in get_instance_by_floating_ip_addr Jul 16 21:14:38.811502 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi self.network_api.get_instance_id_by_floating_address( Jul 16 21:14:38.811502 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 2930, in get_instance_id_by_floating_address Jul 16 21:14:38.811502 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi fip = self._get_floating_ip_by_address(client, address) Jul 16 21:14:38.811502 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 3014, in _get_floating_ip_by_address Jul 16 21:14:38.811502 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.FloatingIpNotFoundForAddress(address=address) Jul 16 21:14:38.811502 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.FloatingIpNotFoundForAddress: Floating IP not found for address 0.0.0.0. Jul 16 21:14:38.811502 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:14:38.811502 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:14:38.811502 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:14:38.811502 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:14:38.811502 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:14:38.811502 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:14:38.811502 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:14:38.811502 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:14:38.811502 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Jul 16 21:14:38.812748 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Jul 16 21:14:38.812748 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:14:38.812748 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:14:38.812748 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 16 21:14:38.812748 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 16 21:14:38.812748 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/floating_ips.py", line 277, in _remove_floating_ip Jul 16 21:14:38.812748 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = get_instance_by_floating_ip_addr(self, context, address) Jul 16 21:14:38.812748 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/floating_ips.py", line 61, in get_instance_by_floating_ip_addr Jul 16 21:14:38.812748 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=ex.format_message()) Jul 16 21:14:38.812748 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Floating IP not found for address 0.0.0.0. Jul 16 21:14:38.812748 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:14:38.812748 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:14:38.812748 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:14:38.812748 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:14:38.812748 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:14:38.812748 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:14:38.812748 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:14:38.812748 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:14:38.812748 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Floating IP not found for address 0.0.0.0. Jul 16 21:14:38.812748 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:14:38.824479 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-9a89759c-60c0-4267-8dde-476d6e0200a4 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] Returning 404 to user: Floating IP not found for address 0.0.0.0. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:14:38.837376 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-9a89759c-60c0-4267-8dde-476d6e0200a4 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] 10.222.0.16 "POST /compute/v2.1/servers/dada2ea0-2ef4-4a30-8e2d-40177d0bafce/action" status: 404 len: 88 microversion: 2.1 time: 0.120686 Jul 16 21:14:38.845391 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 758/1565] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 21:14:38 2022] POST /compute/v2.1/servers/dada2ea0-2ef4-4a30-8e2d-40177d0bafce/action => generated 88 bytes in 129 msecs (HTTP/1.1 404) 9 headers in 378 bytes (1 switches on core 0) Jul 16 21:14:38.851157 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-d89bf209-c251-4d66-8e9c-3f8e15471040 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:14:38.858218 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-d89bf209-c251-4d66-8e9c-3f8e15471040 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] [instance: dada2ea0-2ef4-4a30-8e2d-40177d0bafce] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:14:38.880327 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d89bf209-c251-4d66-8e9c-3f8e15471040 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:38.888642 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d89bf209-c251-4d66-8e9c-3f8e15471040 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:38.888642 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d89bf209-c251-4d66-8e9c-3f8e15471040 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:38.888642 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d89bf209-c251-4d66-8e9c-3f8e15471040 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:38.920201 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-d89bf209-c251-4d66-8e9c-3f8e15471040 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] [instance: dada2ea0-2ef4-4a30-8e2d-40177d0bafce] Going to try to terminate instance {{(pid=138681) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 16 21:14:38.939706 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-5ad84ea5-5c51-47a3-88e1-fdf35c7ae2d1 tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] 10.222.0.16 "DELETE /compute/v2.1/servers/76ee9d21-1f17-4b64-8f95-95a33e70e907" status: 204 len: 0 microversion: 2.1 time: 0.655492 Jul 16 21:14:38.940383 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 808/1566] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:14:38 2022] DELETE /compute/v2.1/servers/76ee9d21-1f17-4b64-8f95-95a33e70e907 => generated 0 bytes in 659 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 16 21:14:38.951288 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-151934a7-6ce7-49e5-b000-497ff1d5165f tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:14:38.953127 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-151934a7-6ce7-49e5-b000-497ff1d5165f tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] [instance: 76ee9d21-1f17-4b64-8f95-95a33e70e907] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:14:38.961696 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-151934a7-6ce7-49e5-b000-497ff1d5165f tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:38.963211 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-151934a7-6ce7-49e5-b000-497ff1d5165f tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:38.964256 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-151934a7-6ce7-49e5-b000-497ff1d5165f tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:38.965011 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-151934a7-6ce7-49e5-b000-497ff1d5165f tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:39.149715 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-151934a7-6ce7-49e5-b000-497ff1d5165f tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c1aea87154224f1c81428ddef4438801', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45d9edd2fe854483a6f0d676efbd6622', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:14:39.167424 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-151934a7-6ce7-49e5-b000-497ff1d5165f tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] 10.222.0.16 "GET /compute/v2.1/servers/76ee9d21-1f17-4b64-8f95-95a33e70e907" status: 200 len: 1468 microversion: 2.1 time: 0.221481 Jul 16 21:14:39.179881 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 809/1567] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:14:38 2022] GET /compute/v2.1/servers/76ee9d21-1f17-4b64-8f95-95a33e70e907 => generated 1468 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:14:39.179881 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-1b899935-35ed-4a6b-afd6-b4e79e8dbc30 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:14:39.179881 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-1b899935-35ed-4a6b-afd6-b4e79e8dbc30 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] [instance: 69cabe57-da71-4a02-b4ee-c0d84b3d32a2] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:14:39.185149 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1b899935-35ed-4a6b-afd6-b4e79e8dbc30 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:39.185602 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1b899935-35ed-4a6b-afd6-b4e79e8dbc30 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:39.186914 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1b899935-35ed-4a6b-afd6-b4e79e8dbc30 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:39.187409 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1b899935-35ed-4a6b-afd6-b4e79e8dbc30 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:39.436195 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-1b899935-35ed-4a6b-afd6-b4e79e8dbc30 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '53af488f3b0242a797f539284b78d59e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bb1ede0d0b6941cc97278e6c617f9b01', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:14:39.455404 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-1b899935-35ed-4a6b-afd6-b4e79e8dbc30 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] 10.222.0.16 "GET /compute/v2.1/servers/69cabe57-da71-4a02-b4ee-c0d84b3d32a2" status: 200 len: 1354 microversion: 2.1 time: 0.285911 Jul 16 21:14:39.456099 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 810/1568] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:14:39 2022] GET /compute/v2.1/servers/69cabe57-da71-4a02-b4ee-c0d84b3d32a2 => generated 1354 bytes in 287 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:14:39.462568 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-46430b39-d46e-47c6-91a3-425c353f3eb0 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:14:39.463525 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-46430b39-d46e-47c6-91a3-425c353f3eb0 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] [instance: e66baae9-d488-46a4-8347-623030fc8b69] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:14:39.470656 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-46430b39-d46e-47c6-91a3-425c353f3eb0 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:39.471409 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-46430b39-d46e-47c6-91a3-425c353f3eb0 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:39.473218 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-46430b39-d46e-47c6-91a3-425c353f3eb0 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:39.473865 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-46430b39-d46e-47c6-91a3-425c353f3eb0 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:39.561637 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-d89bf209-c251-4d66-8e9c-3f8e15471040 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] 10.222.0.16 "DELETE /compute/v2.1/servers/dada2ea0-2ef4-4a30-8e2d-40177d0bafce" status: 204 len: 0 microversion: 2.1 time: 0.714758 Jul 16 21:14:39.562224 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 759/1569] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:14:38 2022] DELETE /compute/v2.1/servers/dada2ea0-2ef4-4a30-8e2d-40177d0bafce => generated 0 bytes in 716 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 16 21:14:39.585391 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-15c46d32-fcbf-4556-97e2-e6cac6ef8144 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:14:39.587126 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-15c46d32-fcbf-4556-97e2-e6cac6ef8144 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] [instance: dada2ea0-2ef4-4a30-8e2d-40177d0bafce] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:14:39.628253 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-15c46d32-fcbf-4556-97e2-e6cac6ef8144 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:39.630126 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-15c46d32-fcbf-4556-97e2-e6cac6ef8144 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:39.632153 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-15c46d32-fcbf-4556-97e2-e6cac6ef8144 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:39.634328 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-15c46d32-fcbf-4556-97e2-e6cac6ef8144 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:39.816693 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-46430b39-d46e-47c6-91a3-425c353f3eb0 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4e9df82f95ac4779859972403869096a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4337575622d04a26942905c9dc044ba7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:14:39.826642 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-46430b39-d46e-47c6-91a3-425c353f3eb0 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] 10.222.0.16 "GET /compute/v2.1/servers/e66baae9-d488-46a4-8347-623030fc8b69" status: 200 len: 1538 microversion: 2.1 time: 0.368813 Jul 16 21:14:39.827635 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 811/1570] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:14:39 2022] GET /compute/v2.1/servers/e66baae9-d488-46a4-8347-623030fc8b69 => generated 1538 bytes in 371 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:14:39.968571 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-15c46d32-fcbf-4556-97e2-e6cac6ef8144 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2eba2ac21c344394a7d3f605775b8713', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c2095f96f8d45bb9e62c517678c3dcd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:14:39.994292 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-15c46d32-fcbf-4556-97e2-e6cac6ef8144 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] 10.222.0.16 "GET /compute/v2.1/servers/dada2ea0-2ef4-4a30-8e2d-40177d0bafce" status: 200 len: 1541 microversion: 2.1 time: 0.426970 Jul 16 21:14:39.996666 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 760/1571] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:14:39 2022] GET /compute/v2.1/servers/dada2ea0-2ef4-4a30-8e2d-40177d0bafce => generated 1541 bytes in 431 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:14:40.190950 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-32798e14-918b-4b13-a46d-03874cc11e22 tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:14:40.190950 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-32798e14-918b-4b13-a46d-03874cc11e22 tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] [instance: 76ee9d21-1f17-4b64-8f95-95a33e70e907] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:14:40.209415 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-32798e14-918b-4b13-a46d-03874cc11e22 tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:40.210136 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-32798e14-918b-4b13-a46d-03874cc11e22 tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:40.211236 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-32798e14-918b-4b13-a46d-03874cc11e22 tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:40.213145 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-32798e14-918b-4b13-a46d-03874cc11e22 tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:40.487356 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-d4fd891d-5891-4ad0-8041-bce4f3965f21 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:14:40.488261 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-d4fd891d-5891-4ad0-8041-bce4f3965f21 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] [instance: 69cabe57-da71-4a02-b4ee-c0d84b3d32a2] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:14:40.493787 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-32798e14-918b-4b13-a46d-03874cc11e22 tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c1aea87154224f1c81428ddef4438801', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45d9edd2fe854483a6f0d676efbd6622', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:14:40.494284 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_api:SELECT=98,nova_api:DELETE=1,nova_api:UPDATE=1 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:14:40.498492 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d4fd891d-5891-4ad0-8041-bce4f3965f21 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:40.499081 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d4fd891d-5891-4ad0-8041-bce4f3965f21 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:40.500205 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d4fd891d-5891-4ad0-8041-bce4f3965f21 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:40.500746 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d4fd891d-5891-4ad0-8041-bce4f3965f21 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:40.504404 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-32798e14-918b-4b13-a46d-03874cc11e22 tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] 10.222.0.16 "GET /compute/v2.1/servers/76ee9d21-1f17-4b64-8f95-95a33e70e907" status: 200 len: 1468 microversion: 2.1 time: 0.320703 Jul 16 21:14:40.515649 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 812/1572] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:14:40 2022] GET /compute/v2.1/servers/76ee9d21-1f17-4b64-8f95-95a33e70e907 => generated 1468 bytes in 322 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:14:40.742868 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-d4fd891d-5891-4ad0-8041-bce4f3965f21 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '53af488f3b0242a797f539284b78d59e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bb1ede0d0b6941cc97278e6c617f9b01', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:14:40.746212 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=94,nova_cell1:SAVEPOINT=1,nova_cell1:UPDATE=2,nova_cell1:RELEASE=1,nova_cell1:INSERT=2 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:14:40.751073 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-d4fd891d-5891-4ad0-8041-bce4f3965f21 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] 10.222.0.16 "GET /compute/v2.1/servers/69cabe57-da71-4a02-b4ee-c0d84b3d32a2" status: 200 len: 1803 microversion: 2.1 time: 0.276654 Jul 16 21:14:40.751662 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 761/1573] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:14:40 2022] GET /compute/v2.1/servers/69cabe57-da71-4a02-b4ee-c0d84b3d32a2 => generated 1803 bytes in 278 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:14:40.769493 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-97019f07-d901-4b77-a847-bc56e9352224 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:14:40.770303 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-97019f07-d901-4b77-a847-bc56e9352224 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] [instance: 69cabe57-da71-4a02-b4ee-c0d84b3d32a2] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:14:40.776411 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-97019f07-d901-4b77-a847-bc56e9352224 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:40.776883 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-97019f07-d901-4b77-a847-bc56e9352224 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:40.777674 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-97019f07-d901-4b77-a847-bc56e9352224 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:40.778117 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-97019f07-d901-4b77-a847-bc56e9352224 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:40.809550 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-97019f07-d901-4b77-a847-bc56e9352224 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] 10.222.0.16 "GET /compute/v2.1/servers/69cabe57-da71-4a02-b4ee-c0d84b3d32a2/ips" status: 200 len: 217 microversion: 2.1 time: 0.044124 Jul 16 21:14:40.816888 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 813/1574] 10.222.0.16 () {58 vars in 1247 bytes} [Sat Jul 16 21:14:40 2022] GET /compute/v2.1/servers/69cabe57-da71-4a02-b4ee-c0d84b3d32a2/ips => generated 217 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:14:40.829581 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-0af40dc3-b5d5-458a-ac08-11f8c1bb4513 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:14:40.830162 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-0af40dc3-b5d5-458a-ac08-11f8c1bb4513 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] [instance: 69cabe57-da71-4a02-b4ee-c0d84b3d32a2] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:14:40.837642 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0af40dc3-b5d5-458a-ac08-11f8c1bb4513 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:40.838233 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0af40dc3-b5d5-458a-ac08-11f8c1bb4513 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:40.841065 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0af40dc3-b5d5-458a-ac08-11f8c1bb4513 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:40.842133 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0af40dc3-b5d5-458a-ac08-11f8c1bb4513 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:40.850853 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-429d9ae0-9fa7-4152-b3d7-4f301e253169 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:14:40.851437 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-429d9ae0-9fa7-4152-b3d7-4f301e253169 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] [instance: e66baae9-d488-46a4-8347-623030fc8b69] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:14:40.858466 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-429d9ae0-9fa7-4152-b3d7-4f301e253169 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:40.858923 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-429d9ae0-9fa7-4152-b3d7-4f301e253169 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:40.860279 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-429d9ae0-9fa7-4152-b3d7-4f301e253169 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:40.860858 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-429d9ae0-9fa7-4152-b3d7-4f301e253169 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:40.891099 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-0af40dc3-b5d5-458a-ac08-11f8c1bb4513 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] [instance: 69cabe57-da71-4a02-b4ee-c0d84b3d32a2] Going to try to terminate instance {{(pid=138681) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 16 21:14:41.237144 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-429d9ae0-9fa7-4152-b3d7-4f301e253169 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4e9df82f95ac4779859972403869096a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4337575622d04a26942905c9dc044ba7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:14:41.247735 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-429d9ae0-9fa7-4152-b3d7-4f301e253169 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] 10.222.0.16 "GET /compute/v2.1/servers/e66baae9-d488-46a4-8347-623030fc8b69" status: 200 len: 1538 microversion: 2.1 time: 0.403204 Jul 16 21:14:41.258023 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 814/1575] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:14:40 2022] GET /compute/v2.1/servers/e66baae9-d488-46a4-8347-623030fc8b69 => generated 1538 bytes in 408 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:14:41.263986 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-d85c68d3-27c5-4bdd-b242-545a92e47666 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:14:41.264816 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-d85c68d3-27c5-4bdd-b242-545a92e47666 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] [instance: dada2ea0-2ef4-4a30-8e2d-40177d0bafce] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:14:41.278617 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d85c68d3-27c5-4bdd-b242-545a92e47666 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:41.279163 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d85c68d3-27c5-4bdd-b242-545a92e47666 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:41.279850 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d85c68d3-27c5-4bdd-b242-545a92e47666 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:41.280294 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d85c68d3-27c5-4bdd-b242-545a92e47666 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:41.308266 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-0af40dc3-b5d5-458a-ac08-11f8c1bb4513 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] 10.222.0.16 "DELETE /compute/v2.1/servers/69cabe57-da71-4a02-b4ee-c0d84b3d32a2" status: 204 len: 0 microversion: 2.1 time: 0.487845 Jul 16 21:14:41.308857 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 762/1576] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:14:40 2022] DELETE /compute/v2.1/servers/69cabe57-da71-4a02-b4ee-c0d84b3d32a2 => generated 0 bytes in 492 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Jul 16 21:14:41.317131 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-8929f0ba-fedf-42cc-a2da-4058242916e4 req-197ab15e-ac25-4b86-94c0-77821d1ae6e4 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "e66baae9-d488-46a4-8347-623030fc8b69", "name": "network-vif-plugged", "status": "completed", "tag": "ed0a7ee7-b878-4680-b977-42bcc7c86cb8"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:14:41.326582 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-8929f0ba-fedf-42cc-a2da-4058242916e4 req-197ab15e-ac25-4b86-94c0-77821d1ae6e4 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:41.334211 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-8929f0ba-fedf-42cc-a2da-4058242916e4 req-197ab15e-ac25-4b86-94c0-77821d1ae6e4 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:41.341663 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-8929f0ba-fedf-42cc-a2da-4058242916e4 req-197ab15e-ac25-4b86-94c0-77821d1ae6e4 service nova] Creating event network-vif-plugged:ed0a7ee7-b878-4680-b977-42bcc7c86cb8 for instance e66baae9-d488-46a4-8347-623030fc8b69 on nch2-master-1 Jul 16 21:14:41.347129 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-8929f0ba-fedf-42cc-a2da-4058242916e4 req-197ab15e-ac25-4b86-94c0-77821d1ae6e4 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.036577 Jul 16 21:14:41.348184 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 763/1577] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:14:41 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:14:41.353560 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-3a9761c0-0e6f-4cc2-b01c-040e01b4b933 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:14:41.354402 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-3a9761c0-0e6f-4cc2-b01c-040e01b4b933 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] [instance: 69cabe57-da71-4a02-b4ee-c0d84b3d32a2] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:14:41.370411 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3a9761c0-0e6f-4cc2-b01c-040e01b4b933 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:41.402814 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3a9761c0-0e6f-4cc2-b01c-040e01b4b933 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:41.402814 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3a9761c0-0e6f-4cc2-b01c-040e01b4b933 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:41.402814 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3a9761c0-0e6f-4cc2-b01c-040e01b4b933 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:41.476455 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-d85c68d3-27c5-4bdd-b242-545a92e47666 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2eba2ac21c344394a7d3f605775b8713', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c2095f96f8d45bb9e62c517678c3dcd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:14:41.484353 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-d85c68d3-27c5-4bdd-b242-545a92e47666 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] 10.222.0.16 "GET /compute/v2.1/servers/dada2ea0-2ef4-4a30-8e2d-40177d0bafce" status: 200 len: 1541 microversion: 2.1 time: 0.230018 Jul 16 21:14:41.485178 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 815/1578] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:14:41 2022] GET /compute/v2.1/servers/dada2ea0-2ef4-4a30-8e2d-40177d0bafce => generated 1541 bytes in 231 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:14:41.606346 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-3a9761c0-0e6f-4cc2-b01c-040e01b4b933 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '53af488f3b0242a797f539284b78d59e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bb1ede0d0b6941cc97278e6c617f9b01', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:14:41.623297 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-3a9761c0-0e6f-4cc2-b01c-040e01b4b933 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] 10.222.0.16 "GET /compute/v2.1/servers/69cabe57-da71-4a02-b4ee-c0d84b3d32a2" status: 200 len: 1809 microversion: 2.1 time: 0.273849 Jul 16 21:14:41.637900 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 764/1579] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:14:41 2022] GET /compute/v2.1/servers/69cabe57-da71-4a02-b4ee-c0d84b3d32a2 => generated 1809 bytes in 286 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:14:41.640149 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-489edd36-3910-4a72-ab1f-cd0cd0d1a6c2 tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:14:41.641202 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-489edd36-3910-4a72-ab1f-cd0cd0d1a6c2 tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] [instance: 76ee9d21-1f17-4b64-8f95-95a33e70e907] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:14:41.649081 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-489edd36-3910-4a72-ab1f-cd0cd0d1a6c2 tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:41.649845 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-489edd36-3910-4a72-ab1f-cd0cd0d1a6c2 tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:41.650639 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-489edd36-3910-4a72-ab1f-cd0cd0d1a6c2 tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:41.651273 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-489edd36-3910-4a72-ab1f-cd0cd0d1a6c2 tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:41.662670 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-f7aa2d4a-aacd-4f2b-a4bd-e1fba18af98f tempest-NetworksTest-476331121 tempest-NetworksTest-476331121-project-admin] Calling method '.version_select at 0x7fdd9d242550>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:14:41.894350 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-f7aa2d4a-aacd-4f2b-a4bd-e1fba18af98f tempest-NetworksTest-476331121 tempest-NetworksTest-476331121-project-admin] 10.222.0.16 "GET /compute/v2.1/os-networks" status: 200 len: 8773 microversion: 2.1 time: 0.407674 Jul 16 21:14:41.901109 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 816/1580] 10.222.0.16 () {58 vars in 1136 bytes} [Sat Jul 16 21:14:41 2022] GET /compute/v2.1/os-networks => generated 8773 bytes in 415 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:14:41.906561 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-11cbb1cf-3e41-4257-ae27-fed077d3b9a0 tempest-NetworksTest-476331121 tempest-NetworksTest-476331121-project-admin] Calling method '.version_select at 0x7fdd9d2a50d0>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:14:41.976815 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-489edd36-3910-4a72-ab1f-cd0cd0d1a6c2 tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c1aea87154224f1c81428ddef4438801', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45d9edd2fe854483a6f0d676efbd6622', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:14:41.986068 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-489edd36-3910-4a72-ab1f-cd0cd0d1a6c2 tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] 10.222.0.16 "GET /compute/v2.1/servers/76ee9d21-1f17-4b64-8f95-95a33e70e907" status: 200 len: 1468 microversion: 2.1 time: 0.350355 Jul 16 21:14:41.987515 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 765/1581] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:14:41 2022] GET /compute/v2.1/servers/76ee9d21-1f17-4b64-8f95-95a33e70e907 => generated 1468 bytes in 352 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:14:42.063611 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-11cbb1cf-3e41-4257-ae27-fed077d3b9a0 tempest-NetworksTest-476331121 tempest-NetworksTest-476331121-project-admin] 10.222.0.16 "GET /compute/v2.1/os-networks/f39f47fa-8d3d-4a2f-9486-cc28d1bbfe66" status: 200 len: 652 microversion: 2.1 time: 0.160439 Jul 16 21:14:42.064209 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 817/1582] 10.222.0.16 () {58 vars in 1247 bytes} [Sat Jul 16 21:14:41 2022] GET /compute/v2.1/os-networks/f39f47fa-8d3d-4a2f-9486-cc28d1bbfe66 => generated 652 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:14:42.073983 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-38d0589d-b844-4420-a51d-a06044f30660 tempest-NetworksTest-476331121 tempest-NetworksTest-476331121-project-admin] Calling method '.version_select at 0x7fdd9d5c3ee0>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:14:42.267244 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-aa371ac5-f7e7-43e3-9c0f-8c034d6955af tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:14:42.268185 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-aa371ac5-f7e7-43e3-9c0f-8c034d6955af tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] [instance: e66baae9-d488-46a4-8347-623030fc8b69] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:14:42.277301 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-aa371ac5-f7e7-43e3-9c0f-8c034d6955af tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:42.278023 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-aa371ac5-f7e7-43e3-9c0f-8c034d6955af tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:42.278800 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-aa371ac5-f7e7-43e3-9c0f-8c034d6955af tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:42.279429 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-aa371ac5-f7e7-43e3-9c0f-8c034d6955af tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:42.315282 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-38d0589d-b844-4420-a51d-a06044f30660 tempest-NetworksTest-476331121 tempest-NetworksTest-476331121-project-admin] 10.222.0.16 "GET /compute/v2.1/os-networks" status: 200 len: 8773 microversion: 2.1 time: 0.245511 Jul 16 21:14:42.324350 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 766/1583] 10.222.0.16 () {58 vars in 1136 bytes} [Sat Jul 16 21:14:42 2022] GET /compute/v2.1/os-networks => generated 8773 bytes in 247 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:14:42.502693 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-79ede8e8-2c58-4eed-bfc6-819befbf804d tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:14:42.503652 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-79ede8e8-2c58-4eed-bfc6-819befbf804d tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] [instance: dada2ea0-2ef4-4a30-8e2d-40177d0bafce] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:14:42.512521 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-79ede8e8-2c58-4eed-bfc6-819befbf804d tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:42.513344 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-79ede8e8-2c58-4eed-bfc6-819befbf804d tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:42.514348 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-79ede8e8-2c58-4eed-bfc6-819befbf804d tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:42.515056 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-79ede8e8-2c58-4eed-bfc6-819befbf804d tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:42.691750 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-aa371ac5-f7e7-43e3-9c0f-8c034d6955af tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4e9df82f95ac4779859972403869096a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4337575622d04a26942905c9dc044ba7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:14:42.704185 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-aa371ac5-f7e7-43e3-9c0f-8c034d6955af tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] 10.222.0.16 "GET /compute/v2.1/servers/e66baae9-d488-46a4-8347-623030fc8b69" status: 200 len: 1538 microversion: 2.1 time: 0.439387 Jul 16 21:14:42.711234 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 818/1584] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:14:42 2022] GET /compute/v2.1/servers/e66baae9-d488-46a4-8347-623030fc8b69 => generated 1538 bytes in 443 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:14:42.720824 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-dc51c2b3-83f5-4471-9d02-63968e794a68 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:14:42.737169 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-dc51c2b3-83f5-4471-9d02-63968e794a68 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] [instance: 69cabe57-da71-4a02-b4ee-c0d84b3d32a2] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:14:42.743776 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-dc51c2b3-83f5-4471-9d02-63968e794a68 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:42.744230 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-dc51c2b3-83f5-4471-9d02-63968e794a68 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:42.745440 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-dc51c2b3-83f5-4471-9d02-63968e794a68 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:42.746046 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-dc51c2b3-83f5-4471-9d02-63968e794a68 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:42.879492 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-79ede8e8-2c58-4eed-bfc6-819befbf804d tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2eba2ac21c344394a7d3f605775b8713', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c2095f96f8d45bb9e62c517678c3dcd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:14:42.889572 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-79ede8e8-2c58-4eed-bfc6-819befbf804d tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] 10.222.0.16 "GET /compute/v2.1/servers/dada2ea0-2ef4-4a30-8e2d-40177d0bafce" status: 200 len: 1541 microversion: 2.1 time: 0.390725 Jul 16 21:14:42.890808 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 767/1585] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:14:42 2022] GET /compute/v2.1/servers/dada2ea0-2ef4-4a30-8e2d-40177d0bafce => generated 1541 bytes in 393 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:14:43.006002 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-98919b16-5280-4cab-886f-370d16e76fe1 tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:14:43.007212 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-98919b16-5280-4cab-886f-370d16e76fe1 tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] [instance: 76ee9d21-1f17-4b64-8f95-95a33e70e907] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:14:43.014900 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-98919b16-5280-4cab-886f-370d16e76fe1 tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:43.015912 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-98919b16-5280-4cab-886f-370d16e76fe1 tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:43.016913 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-98919b16-5280-4cab-886f-370d16e76fe1 tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:43.017573 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-98919b16-5280-4cab-886f-370d16e76fe1 tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:43.295689 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-dc51c2b3-83f5-4471-9d02-63968e794a68 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '53af488f3b0242a797f539284b78d59e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bb1ede0d0b6941cc97278e6c617f9b01', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:14:43.305182 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-dc51c2b3-83f5-4471-9d02-63968e794a68 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] 10.222.0.16 "GET /compute/v2.1/servers/69cabe57-da71-4a02-b4ee-c0d84b3d32a2" status: 200 len: 1809 microversion: 2.1 time: 0.596281 Jul 16 21:14:43.305771 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 819/1586] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:14:42 2022] GET /compute/v2.1/servers/69cabe57-da71-4a02-b4ee-c0d84b3d32a2 => generated 1809 bytes in 600 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:14:43.455618 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-98919b16-5280-4cab-886f-370d16e76fe1 tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c1aea87154224f1c81428ddef4438801', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45d9edd2fe854483a6f0d676efbd6622', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:14:43.468782 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-98919b16-5280-4cab-886f-370d16e76fe1 tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] 10.222.0.16 "GET /compute/v2.1/servers/76ee9d21-1f17-4b64-8f95-95a33e70e907" status: 200 len: 1468 microversion: 2.1 time: 0.468223 Jul 16 21:14:43.470390 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 768/1587] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:14:42 2022] GET /compute/v2.1/servers/76ee9d21-1f17-4b64-8f95-95a33e70e907 => generated 1468 bytes in 470 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:14:43.725376 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-9b8757b5-7a1b-451e-9eba-87e2872ac548 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:14:43.730556 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-9b8757b5-7a1b-451e-9eba-87e2872ac548 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] [instance: e66baae9-d488-46a4-8347-623030fc8b69] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:14:43.749442 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_api:SELECT=83,nova_api:UPDATE=3 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:14:43.776580 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9b8757b5-7a1b-451e-9eba-87e2872ac548 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:43.777146 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9b8757b5-7a1b-451e-9eba-87e2872ac548 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:43.778047 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9b8757b5-7a1b-451e-9eba-87e2872ac548 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:43.778569 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9b8757b5-7a1b-451e-9eba-87e2872ac548 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:43.788559 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=96,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:14:43.837493 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-a6ebf2ce-b7c8-4d64-b87c-8a07a754055f req-b48bf584-6386-4181-bca2-d62f1aca5571 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "76ee9d21-1f17-4b64-8f95-95a33e70e907", "name": "network-vif-deleted", "tag": "65068e4e-def5-46ec-95ec-de961e8c1e0e"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:14:43.851337 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-a6ebf2ce-b7c8-4d64-b87c-8a07a754055f req-b48bf584-6386-4181-bca2-d62f1aca5571 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:43.857019 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-a6ebf2ce-b7c8-4d64-b87c-8a07a754055f req-b48bf584-6386-4181-bca2-d62f1aca5571 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:43.869236 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-a6ebf2ce-b7c8-4d64-b87c-8a07a754055f req-b48bf584-6386-4181-bca2-d62f1aca5571 service nova] Creating event network-vif-deleted:65068e4e-def5-46ec-95ec-de961e8c1e0e for instance 76ee9d21-1f17-4b64-8f95-95a33e70e907 on nch2-master-1 Jul 16 21:14:43.875369 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-a6ebf2ce-b7c8-4d64-b87c-8a07a754055f req-b48bf584-6386-4181-bca2-d62f1aca5571 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.043816 Jul 16 21:14:43.878389 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 769/1588] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:14:43 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:14:43.909412 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-f961da2f-8446-4eb3-a856-90728685d20d tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:14:43.910166 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-f961da2f-8446-4eb3-a856-90728685d20d tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] [instance: dada2ea0-2ef4-4a30-8e2d-40177d0bafce] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:14:43.917284 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f961da2f-8446-4eb3-a856-90728685d20d tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:43.917805 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f961da2f-8446-4eb3-a856-90728685d20d tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:43.918812 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f961da2f-8446-4eb3-a856-90728685d20d tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:43.919296 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f961da2f-8446-4eb3-a856-90728685d20d tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:44.513553 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-f961da2f-8446-4eb3-a856-90728685d20d tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2eba2ac21c344394a7d3f605775b8713', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c2095f96f8d45bb9e62c517678c3dcd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:14:44.526906 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-f961da2f-8446-4eb3-a856-90728685d20d tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] 10.222.0.16 "GET /compute/v2.1/servers/dada2ea0-2ef4-4a30-8e2d-40177d0bafce" status: 200 len: 1499 microversion: 2.1 time: 0.621857 Jul 16 21:14:44.527555 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 770/1589] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:14:43 2022] GET /compute/v2.1/servers/dada2ea0-2ef4-4a30-8e2d-40177d0bafce => generated 1499 bytes in 625 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:14:44.535616 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-626d7780-b8b0-46bd-a035-6848b460965e tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:14:44.536981 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-626d7780-b8b0-46bd-a035-6848b460965e tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] [instance: 69cabe57-da71-4a02-b4ee-c0d84b3d32a2] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:14:44.550927 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-626d7780-b8b0-46bd-a035-6848b460965e tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:44.553835 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-626d7780-b8b0-46bd-a035-6848b460965e tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:44.554679 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-626d7780-b8b0-46bd-a035-6848b460965e tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:44.555148 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-626d7780-b8b0-46bd-a035-6848b460965e tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:44.599567 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-9b8757b5-7a1b-451e-9eba-87e2872ac548 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4e9df82f95ac4779859972403869096a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4337575622d04a26942905c9dc044ba7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:14:44.609110 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-9b8757b5-7a1b-451e-9eba-87e2872ac548 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] 10.222.0.16 "GET /compute/v2.1/servers/e66baae9-d488-46a4-8347-623030fc8b69" status: 200 len: 1520 microversion: 2.1 time: 0.890701 Jul 16 21:14:44.621673 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 820/1590] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:14:43 2022] GET /compute/v2.1/servers/e66baae9-d488-46a4-8347-623030fc8b69 => generated 1520 bytes in 903 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:14:44.628359 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-57a0cb72-1d58-4818-8857-a215435e1dca tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:14:44.629189 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-57a0cb72-1d58-4818-8857-a215435e1dca tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] [instance: 76ee9d21-1f17-4b64-8f95-95a33e70e907] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:14:44.637914 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-57a0cb72-1d58-4818-8857-a215435e1dca tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:44.638363 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-57a0cb72-1d58-4818-8857-a215435e1dca tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:44.639496 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-57a0cb72-1d58-4818-8857-a215435e1dca tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:44.640276 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-57a0cb72-1d58-4818-8857-a215435e1dca tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:44.682449 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-57a0cb72-1d58-4818-8857-a215435e1dca tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:44.683223 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-57a0cb72-1d58-4818-8857-a215435e1dca tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:44.683848 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.objects.instance [None req-57a0cb72-1d58-4818-8857-a215435e1dca tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Lazy-loading 'fault' on Instance uuid 76ee9d21-1f17-4b64-8f95-95a33e70e907 {{(pid=138680) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:14:44.750620 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-57a0cb72-1d58-4818-8857-a215435e1dca tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c1aea87154224f1c81428ddef4438801', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45d9edd2fe854483a6f0d676efbd6622', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:14:44.758078 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-57a0cb72-1d58-4818-8857-a215435e1dca tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] 10.222.0.16 "GET /compute/v2.1/servers/76ee9d21-1f17-4b64-8f95-95a33e70e907" status: 200 len: 1312 microversion: 2.1 time: 0.133766 Jul 16 21:14:44.758912 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 821/1591] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:14:44 2022] GET /compute/v2.1/servers/76ee9d21-1f17-4b64-8f95-95a33e70e907 => generated 1312 bytes in 135 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:14:44.764416 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-193eddd1-3710-4b15-96e2-55fd4c6fd334 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:14:44.765241 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-193eddd1-3710-4b15-96e2-55fd4c6fd334 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] [instance: e66baae9-d488-46a4-8347-623030fc8b69] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:14:44.772303 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-193eddd1-3710-4b15-96e2-55fd4c6fd334 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:44.773060 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-193eddd1-3710-4b15-96e2-55fd4c6fd334 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:44.773856 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-193eddd1-3710-4b15-96e2-55fd4c6fd334 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:44.774467 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-193eddd1-3710-4b15-96e2-55fd4c6fd334 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:44.801232 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-626d7780-b8b0-46bd-a035-6848b460965e tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '53af488f3b0242a797f539284b78d59e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bb1ede0d0b6941cc97278e6c617f9b01', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:14:44.805199 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-193eddd1-3710-4b15-96e2-55fd4c6fd334 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] [instance: e66baae9-d488-46a4-8347-623030fc8b69] Going to try to terminate instance {{(pid=138680) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 16 21:14:44.816264 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-626d7780-b8b0-46bd-a035-6848b460965e tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] 10.222.0.16 "GET /compute/v2.1/servers/69cabe57-da71-4a02-b4ee-c0d84b3d32a2" status: 200 len: 1809 microversion: 2.1 time: 0.287434 Jul 16 21:14:44.819214 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 771/1592] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:14:44 2022] GET /compute/v2.1/servers/69cabe57-da71-4a02-b4ee-c0d84b3d32a2 => generated 1809 bytes in 291 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:14:45.345689 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-193eddd1-3710-4b15-96e2-55fd4c6fd334 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] 10.222.0.16 "DELETE /compute/v2.1/servers/e66baae9-d488-46a4-8347-623030fc8b69" status: 204 len: 0 microversion: 2.1 time: 0.584766 Jul 16 21:14:45.346501 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 822/1593] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:14:44 2022] DELETE /compute/v2.1/servers/e66baae9-d488-46a4-8347-623030fc8b69 => generated 0 bytes in 587 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 16 21:14:45.355410 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-a1336b6f-05a9-447f-9f15-26a154763d06 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:14:45.356024 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-a1336b6f-05a9-447f-9f15-26a154763d06 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] [instance: e66baae9-d488-46a4-8347-623030fc8b69] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:14:45.363798 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a1336b6f-05a9-447f-9f15-26a154763d06 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:45.364246 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a1336b6f-05a9-447f-9f15-26a154763d06 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:45.365834 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a1336b6f-05a9-447f-9f15-26a154763d06 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:45.366351 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a1336b6f-05a9-447f-9f15-26a154763d06 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:45.507813 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-a1336b6f-05a9-447f-9f15-26a154763d06 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4e9df82f95ac4779859972403869096a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4337575622d04a26942905c9dc044ba7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:14:45.515872 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-a1336b6f-05a9-447f-9f15-26a154763d06 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] 10.222.0.16 "GET /compute/v2.1/servers/e66baae9-d488-46a4-8347-623030fc8b69" status: 200 len: 1526 microversion: 2.1 time: 0.164313 Jul 16 21:14:45.528192 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 772/1594] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:14:45 2022] GET /compute/v2.1/servers/e66baae9-d488-46a4-8347-623030fc8b69 => generated 1526 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:14:45.546881 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-02923a67-d6a4-4811-9365-42cb6d4395e4 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:14:45.548150 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-02923a67-d6a4-4811-9365-42cb6d4395e4 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] [instance: dada2ea0-2ef4-4a30-8e2d-40177d0bafce] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:14:45.558494 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-02923a67-d6a4-4811-9365-42cb6d4395e4 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:45.558835 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-02923a67-d6a4-4811-9365-42cb6d4395e4 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:45.559882 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-02923a67-d6a4-4811-9365-42cb6d4395e4 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:45.560448 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-02923a67-d6a4-4811-9365-42cb6d4395e4 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:45.610596 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-02923a67-d6a4-4811-9365-42cb6d4395e4 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:45.611459 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-02923a67-d6a4-4811-9365-42cb6d4395e4 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:45.612125 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.objects.instance [None req-02923a67-d6a4-4811-9365-42cb6d4395e4 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] Lazy-loading 'fault' on Instance uuid dada2ea0-2ef4-4a30-8e2d-40177d0bafce {{(pid=138680) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:14:45.695417 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-02923a67-d6a4-4811-9365-42cb6d4395e4 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2eba2ac21c344394a7d3f605775b8713', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c2095f96f8d45bb9e62c517678c3dcd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:14:45.706975 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-02923a67-d6a4-4811-9365-42cb6d4395e4 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] 10.222.0.16 "GET /compute/v2.1/servers/dada2ea0-2ef4-4a30-8e2d-40177d0bafce" status: 200 len: 1321 microversion: 2.1 time: 0.166122 Jul 16 21:14:45.707634 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 823/1595] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:14:45 2022] GET /compute/v2.1/servers/dada2ea0-2ef4-4a30-8e2d-40177d0bafce => generated 1321 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:14:45.778748 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-74a7afd3-7a17-43e4-bef1-c0be50252be8 tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:14:45.779837 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-74a7afd3-7a17-43e4-bef1-c0be50252be8 tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] [instance: 76ee9d21-1f17-4b64-8f95-95a33e70e907] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:14:45.786641 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-74a7afd3-7a17-43e4-bef1-c0be50252be8 tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:45.786641 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-74a7afd3-7a17-43e4-bef1-c0be50252be8 tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:45.787915 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-74a7afd3-7a17-43e4-bef1-c0be50252be8 tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:45.788375 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-74a7afd3-7a17-43e4-bef1-c0be50252be8 tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:45.807044 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-74a7afd3-7a17-43e4-bef1-c0be50252be8 tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] HTTP exception thrown: Instance 76ee9d21-1f17-4b64-8f95-95a33e70e907 could not be found. Jul 16 21:14:45.808818 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-74a7afd3-7a17-43e4-bef1-c0be50252be8 tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Request method failure captured: Jul 16 21:14:45.808818 ncdv-master-1 devstack@n-api.service[138681]: request: GET /compute/v2.1/servers/76ee9d21-1f17-4b64-8f95-95a33e70e907 HTTP/1.1 Jul 16 21:14:45.808818 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:14:45.808818 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:14:45.808818 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:14:45.808818 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 21:14:45.808818 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:14:45.808818 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:14:45.808818 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:14:45.808818 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:14:45.808818 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0yn6K6QftoPpkEysn4UVt0CZkaXupfIonEWl3L75o2URhqIcdNiz9cIehYJ1L-ssxF-OWGGDzSzhCaPaoZNk55CU3ZOG4LNQVyuBHPyTObFqRRv53RHcWu8kd8Q8R3EopbISfq78hx_LODvbVr1VYmgBwBkt5FFXCv7w6P_-coX-Pnc Jul 16 21:14:45.808818 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:14:45.808818 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:14:45.808818 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:14:45.808818 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:14:45.808818 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:14:45.808818 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:14:45.808818 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: 45d9edd2fe854483a6f0d676efbd6622 Jul 16 21:14:45.808818 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-ServersAdminNegativeTestJSON-1083411175 Jul 16 21:14:45.808818 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:14:45.808818 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:14:45.810573 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/45d9edd2fe854483a6f0d676efbd6622"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/45d9edd2fe854483a6f0d676efbd6622"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_45d9edd2fe854483a6f0d676efbd6622"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/45d9edd2fe854483a6f0d676efbd6622"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:14:45.810573 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-ServersAdminNegativeTestJSON-1083411175 Jul 16 21:14:45.810573 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: 45d9edd2fe854483a6f0d676efbd6622 Jul 16 21:14:45.810573 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-ServersAdminNegativeTestJSON-1083411175 Jul 16 21:14:45.810573 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-ServersAdminNegativeTestJSON-1083411175-project Jul 16 21:14:45.810573 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:14:45.810573 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:14:45.810573 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: c1aea87154224f1c81428ddef4438801 Jul 16 21:14:45.810573 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-ServersAdminNegativeTestJSON-1083411175-project Jul 16 21:14:45.810573 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:14:45.810573 ncdv-master-1 devstack@n-api.service[138681]: exception: Instance 76ee9d21-1f17-4b64-8f95-95a33e70e907 could not be found. Jul 16 21:14:45.810573 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': '76ee9d21-1f17-4b64-8f95-95a33e70e907'} Jul 16 21:14:45.811544 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:14:45.811544 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:14:45.811544 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:14:45.811544 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:14:45.811544 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:14:45.811544 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:14:45.811544 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:14:45.811544 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:14:45.811544 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:14:45.811544 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:14:45.811544 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 76ee9d21-1f17-4b64-8f95-95a33e70e907 could not be found. Jul 16 21:14:45.811544 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:14:45.811544 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:14:45.811544 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:14:45.811544 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:14:45.811544 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:14:45.811544 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:14:45.811544 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:14:45.811544 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:14:45.811544 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:14:45.813187 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:14:45.813187 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:14:45.813187 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:14:45.813187 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:14:45.813187 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:14:45.813187 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:14:45.813187 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:14:45.813187 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 76ee9d21-1f17-4b64-8f95-95a33e70e907 could not be found. Jul 16 21:14:45.813187 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:14:45.813187 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:14:45.813187 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:14:45.813187 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:14:45.813187 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:14:45.813187 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:14:45.813187 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:14:45.813187 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:14:45.813187 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 76ee9d21-1f17-4b64-8f95-95a33e70e907 could not be found. Jul 16 21:14:45.813187 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:14:45.815570 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-74a7afd3-7a17-43e4-bef1-c0be50252be8 tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Returning 404 to user: Instance 76ee9d21-1f17-4b64-8f95-95a33e70e907 could not be found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:14:45.817258 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-74a7afd3-7a17-43e4-bef1-c0be50252be8 tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] 10.222.0.16 "GET /compute/v2.1/servers/76ee9d21-1f17-4b64-8f95-95a33e70e907" status: 404 len: 111 microversion: 2.1 time: 0.043279 Jul 16 21:14:45.824927 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 773/1596] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:14:45 2022] GET /compute/v2.1/servers/76ee9d21-1f17-4b64-8f95-95a33e70e907 => generated 111 bytes in 53 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Jul 16 21:14:45.829216 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-e2cf7506-d68c-4199-8704-e710a71e454c tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:14:45.835927 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-ec564eee-0491-4beb-ad51-62577cde988e tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:14:45.843810 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-e2cf7506-d68c-4199-8704-e710a71e454c tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] [instance: 4602867b-88ed-43b1-9ee6-bf661e2d7d99] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:14:45.844855 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-ec564eee-0491-4beb-ad51-62577cde988e tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] [instance: 69cabe57-da71-4a02-b4ee-c0d84b3d32a2] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:14:45.857340 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ec564eee-0491-4beb-ad51-62577cde988e tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:45.857715 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e2cf7506-d68c-4199-8704-e710a71e454c tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:45.858131 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ec564eee-0491-4beb-ad51-62577cde988e tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:45.860023 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ec564eee-0491-4beb-ad51-62577cde988e tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:45.860810 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ec564eee-0491-4beb-ad51-62577cde988e tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:45.861059 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e2cf7506-d68c-4199-8704-e710a71e454c tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:45.862321 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e2cf7506-d68c-4199-8704-e710a71e454c tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:45.862775 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e2cf7506-d68c-4199-8704-e710a71e454c tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:45.916812 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-e2cf7506-d68c-4199-8704-e710a71e454c tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] [instance: 4602867b-88ed-43b1-9ee6-bf661e2d7d99] Going to try to terminate instance {{(pid=138680) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 16 21:14:46.168206 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-ec564eee-0491-4beb-ad51-62577cde988e tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '53af488f3b0242a797f539284b78d59e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bb1ede0d0b6941cc97278e6c617f9b01', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:14:46.178322 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-ec564eee-0491-4beb-ad51-62577cde988e tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] 10.222.0.16 "GET /compute/v2.1/servers/69cabe57-da71-4a02-b4ee-c0d84b3d32a2" status: 200 len: 1767 microversion: 2.1 time: 0.346469 Jul 16 21:14:46.188678 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 774/1597] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:14:45 2022] GET /compute/v2.1/servers/69cabe57-da71-4a02-b4ee-c0d84b3d32a2 => generated 1767 bytes in 358 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:14:46.195159 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-5aa77c67-4898-4eb0-b839-ffcf1e5df2c3 req-86ae5f72-d226-48f6-bb39-c9db3c0c35ba service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "dada2ea0-2ef4-4a30-8e2d-40177d0bafce", "name": "network-vif-deleted", "tag": "b63bf933-a84e-4e18-98ed-0e772979bec5"}, {"server_uuid": "69cabe57-da71-4a02-b4ee-c0d84b3d32a2", "name": "network-vif-deleted", "tag": "da98933d-c6aa-4341-b47f-1483a9e3aa36"}, {"server_uuid": "69cabe57-da71-4a02-b4ee-c0d84b3d32a2", "name": "network-vif-deleted", "tag": "1f354c18-a81d-412a-97e0-43d35608c82c"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:14:46.204758 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-5aa77c67-4898-4eb0-b839-ffcf1e5df2c3 req-86ae5f72-d226-48f6-bb39-c9db3c0c35ba service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:46.205281 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-5aa77c67-4898-4eb0-b839-ffcf1e5df2c3 req-86ae5f72-d226-48f6-bb39-c9db3c0c35ba service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:46.250850 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-5aa77c67-4898-4eb0-b839-ffcf1e5df2c3 req-86ae5f72-d226-48f6-bb39-c9db3c0c35ba service nova] Creating event network-vif-deleted:b63bf933-a84e-4e18-98ed-0e772979bec5 for instance dada2ea0-2ef4-4a30-8e2d-40177d0bafce on nch2-master-1 Jul 16 21:14:46.254672 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-5aa77c67-4898-4eb0-b839-ffcf1e5df2c3 req-86ae5f72-d226-48f6-bb39-c9db3c0c35ba service nova] Creating event network-vif-deleted:da98933d-c6aa-4341-b47f-1483a9e3aa36 for instance 69cabe57-da71-4a02-b4ee-c0d84b3d32a2 on nch1-master-1 Jul 16 21:14:46.255298 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-5aa77c67-4898-4eb0-b839-ffcf1e5df2c3 req-86ae5f72-d226-48f6-bb39-c9db3c0c35ba service nova] Creating event network-vif-deleted:1f354c18-a81d-412a-97e0-43d35608c82c for instance 69cabe57-da71-4a02-b4ee-c0d84b3d32a2 on nch1-master-1 Jul 16 21:14:46.265126 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-5aa77c67-4898-4eb0-b839-ffcf1e5df2c3 req-86ae5f72-d226-48f6-bb39-c9db3c0c35ba service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 525 microversion: 2.1 time: 0.075054 Jul 16 21:14:46.266377 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 775/1598] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:14:46 2022] POST /compute/v2.1/os-server-external-events => generated 525 bytes in 77 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:14:46.274267 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-4829acb7-8c65-4e24-b342-9939778f9832 req-d594bd47-9919-450a-a5c8-8c6c42ee14fd service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "69cabe57-da71-4a02-b4ee-c0d84b3d32a2", "name": "network-vif-unplugged", "status": "completed", "tag": "9e27e304-0422-4ee7-8ca0-2d670e12f5ee"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:14:46.289381 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-4829acb7-8c65-4e24-b342-9939778f9832 req-d594bd47-9919-450a-a5c8-8c6c42ee14fd service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:46.289381 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-4829acb7-8c65-4e24-b342-9939778f9832 req-d594bd47-9919-450a-a5c8-8c6c42ee14fd service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:46.300553 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-4829acb7-8c65-4e24-b342-9939778f9832 req-d594bd47-9919-450a-a5c8-8c6c42ee14fd service nova] Creating event network-vif-unplugged:9e27e304-0422-4ee7-8ca0-2d670e12f5ee for instance 69cabe57-da71-4a02-b4ee-c0d84b3d32a2 on nch1-master-1 Jul 16 21:14:46.306949 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-4829acb7-8c65-4e24-b342-9939778f9832 req-d594bd47-9919-450a-a5c8-8c6c42ee14fd service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.038724 Jul 16 21:14:46.308500 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 776/1599] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:14:46 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:14:46.384353 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-e2cf7506-d68c-4199-8704-e710a71e454c tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] 10.222.0.16 "DELETE /compute/v2.1/servers/4602867b-88ed-43b1-9ee6-bf661e2d7d99" status: 204 len: 0 microversion: 2.1 time: 0.559894 Jul 16 21:14:46.384984 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 824/1600] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:14:45 2022] DELETE /compute/v2.1/servers/4602867b-88ed-43b1-9ee6-bf661e2d7d99 => generated 0 bytes in 562 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 16 21:14:46.394803 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-f42fac3a-d7a1-4de5-a9de-a2a68aa0551f tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:14:46.395568 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-f42fac3a-d7a1-4de5-a9de-a2a68aa0551f tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] [instance: 4602867b-88ed-43b1-9ee6-bf661e2d7d99] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:14:46.402789 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f42fac3a-d7a1-4de5-a9de-a2a68aa0551f tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:46.403278 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f42fac3a-d7a1-4de5-a9de-a2a68aa0551f tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:46.404078 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f42fac3a-d7a1-4de5-a9de-a2a68aa0551f tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:46.404515 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f42fac3a-d7a1-4de5-a9de-a2a68aa0551f tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:46.538685 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-539f90db-4e76-4b91-b020-0c60a9c420b2 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:14:46.539827 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-539f90db-4e76-4b91-b020-0c60a9c420b2 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] [instance: e66baae9-d488-46a4-8347-623030fc8b69] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:14:46.555848 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-539f90db-4e76-4b91-b020-0c60a9c420b2 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:46.557189 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-539f90db-4e76-4b91-b020-0c60a9c420b2 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:46.559878 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-539f90db-4e76-4b91-b020-0c60a9c420b2 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:46.563123 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-539f90db-4e76-4b91-b020-0c60a9c420b2 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:46.728136 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-f42fac3a-d7a1-4de5-a9de-a2a68aa0551f tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c1aea87154224f1c81428ddef4438801', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45d9edd2fe854483a6f0d676efbd6622', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:14:46.736442 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-f42fac3a-d7a1-4de5-a9de-a2a68aa0551f tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] 10.222.0.16 "GET /compute/v2.1/servers/4602867b-88ed-43b1-9ee6-bf661e2d7d99" status: 200 len: 1477 microversion: 2.1 time: 0.346645 Jul 16 21:14:46.737008 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 777/1601] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:14:46 2022] GET /compute/v2.1/servers/4602867b-88ed-43b1-9ee6-bf661e2d7d99 => generated 1477 bytes in 348 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:14:46.748844 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-a4620014-7904-402d-a9f5-98030564a156 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:14:46.749618 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-a4620014-7904-402d-a9f5-98030564a156 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] [instance: dada2ea0-2ef4-4a30-8e2d-40177d0bafce] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:14:46.756092 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a4620014-7904-402d-a9f5-98030564a156 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:46.756463 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a4620014-7904-402d-a9f5-98030564a156 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:46.757243 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a4620014-7904-402d-a9f5-98030564a156 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:46.757682 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a4620014-7904-402d-a9f5-98030564a156 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:46.781095 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-539f90db-4e76-4b91-b020-0c60a9c420b2 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4e9df82f95ac4779859972403869096a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4337575622d04a26942905c9dc044ba7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:14:46.790311 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-539f90db-4e76-4b91-b020-0c60a9c420b2 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] 10.222.0.16 "GET /compute/v2.1/servers/e66baae9-d488-46a4-8347-623030fc8b69" status: 200 len: 1526 microversion: 2.1 time: 0.259209 Jul 16 21:14:46.790968 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 825/1602] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:14:46 2022] GET /compute/v2.1/servers/e66baae9-d488-46a4-8347-623030fc8b69 => generated 1526 bytes in 261 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:14:46.792707 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a4620014-7904-402d-a9f5-98030564a156 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:46.793175 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a4620014-7904-402d-a9f5-98030564a156 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:46.793633 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.objects.instance [None req-a4620014-7904-402d-a9f5-98030564a156 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] Lazy-loading 'fault' on Instance uuid dada2ea0-2ef4-4a30-8e2d-40177d0bafce {{(pid=138681) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:14:46.843473 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-a4620014-7904-402d-a9f5-98030564a156 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2eba2ac21c344394a7d3f605775b8713', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c2095f96f8d45bb9e62c517678c3dcd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:14:46.854206 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-a4620014-7904-402d-a9f5-98030564a156 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] 10.222.0.16 "GET /compute/v2.1/servers/dada2ea0-2ef4-4a30-8e2d-40177d0bafce" status: 200 len: 1321 microversion: 2.1 time: 0.109173 Jul 16 21:14:46.854797 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 778/1603] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:14:46 2022] GET /compute/v2.1/servers/dada2ea0-2ef4-4a30-8e2d-40177d0bafce => generated 1321 bytes in 117 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:14:47.196654 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-9998bcb6-9071-4726-90a6-a8e1e8a56bdf tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:14:47.198545 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-9998bcb6-9071-4726-90a6-a8e1e8a56bdf tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] [instance: 69cabe57-da71-4a02-b4ee-c0d84b3d32a2] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:14:47.205610 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9998bcb6-9071-4726-90a6-a8e1e8a56bdf tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:47.206121 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9998bcb6-9071-4726-90a6-a8e1e8a56bdf tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:47.206921 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9998bcb6-9071-4726-90a6-a8e1e8a56bdf tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:47.207407 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9998bcb6-9071-4726-90a6-a8e1e8a56bdf tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:47.307523 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-9998bcb6-9071-4726-90a6-a8e1e8a56bdf tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '53af488f3b0242a797f539284b78d59e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bb1ede0d0b6941cc97278e6c617f9b01', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:14:47.316196 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-9998bcb6-9071-4726-90a6-a8e1e8a56bdf tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] 10.222.0.16 "GET /compute/v2.1/servers/69cabe57-da71-4a02-b4ee-c0d84b3d32a2" status: 200 len: 1453 microversion: 2.1 time: 0.124116 Jul 16 21:14:47.317641 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 826/1604] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:14:47 2022] GET /compute/v2.1/servers/69cabe57-da71-4a02-b4ee-c0d84b3d32a2 => generated 1453 bytes in 126 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:14:47.753444 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-d88644ac-39ab-476a-bdc8-4cd875adee4a tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:14:47.760011 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-d88644ac-39ab-476a-bdc8-4cd875adee4a tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] [instance: 4602867b-88ed-43b1-9ee6-bf661e2d7d99] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:14:47.766554 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d88644ac-39ab-476a-bdc8-4cd875adee4a tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:47.767278 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d88644ac-39ab-476a-bdc8-4cd875adee4a tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:47.768241 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d88644ac-39ab-476a-bdc8-4cd875adee4a tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:47.768879 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d88644ac-39ab-476a-bdc8-4cd875adee4a tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:47.809104 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-73be7eda-a357-49dc-afc4-561a72805119 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:14:47.811229 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-73be7eda-a357-49dc-afc4-561a72805119 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] [instance: e66baae9-d488-46a4-8347-623030fc8b69] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:14:47.815918 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-73be7eda-a357-49dc-afc4-561a72805119 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:47.816374 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-73be7eda-a357-49dc-afc4-561a72805119 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:47.817160 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-73be7eda-a357-49dc-afc4-561a72805119 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:47.818164 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-73be7eda-a357-49dc-afc4-561a72805119 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:48.236335 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-d88644ac-39ab-476a-bdc8-4cd875adee4a tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c1aea87154224f1c81428ddef4438801', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45d9edd2fe854483a6f0d676efbd6622', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:14:48.261021 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-d88644ac-39ab-476a-bdc8-4cd875adee4a tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] 10.222.0.16 "GET /compute/v2.1/servers/4602867b-88ed-43b1-9ee6-bf661e2d7d99" status: 200 len: 1477 microversion: 2.1 time: 0.508885 Jul 16 21:14:48.264704 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 779/1605] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:14:47 2022] GET /compute/v2.1/servers/4602867b-88ed-43b1-9ee6-bf661e2d7d99 => generated 1477 bytes in 515 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:14:48.277718 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-73be7eda-a357-49dc-afc4-561a72805119 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4e9df82f95ac4779859972403869096a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4337575622d04a26942905c9dc044ba7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:14:48.280070 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-e5d3a117-d04c-47e4-8f83-bcfea6e0ff8a tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:14:48.283882 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-e5d3a117-d04c-47e4-8f83-bcfea6e0ff8a tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] [instance: dada2ea0-2ef4-4a30-8e2d-40177d0bafce] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:14:48.287867 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-73be7eda-a357-49dc-afc4-561a72805119 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] 10.222.0.16 "GET /compute/v2.1/servers/e66baae9-d488-46a4-8347-623030fc8b69" status: 200 len: 1526 microversion: 2.1 time: 0.482479 Jul 16 21:14:48.303446 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 827/1606] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:14:47 2022] GET /compute/v2.1/servers/e66baae9-d488-46a4-8347-623030fc8b69 => generated 1526 bytes in 484 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:14:48.304998 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e5d3a117-d04c-47e4-8f83-bcfea6e0ff8a tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:48.304998 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e5d3a117-d04c-47e4-8f83-bcfea6e0ff8a tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:48.304998 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e5d3a117-d04c-47e4-8f83-bcfea6e0ff8a tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:48.309430 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e5d3a117-d04c-47e4-8f83-bcfea6e0ff8a tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:48.350484 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-e5d3a117-d04c-47e4-8f83-bcfea6e0ff8a tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] HTTP exception thrown: Instance dada2ea0-2ef4-4a30-8e2d-40177d0bafce could not be found. Jul 16 21:14:48.354209 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-42fe12ee-25f1-4b28-b8ef-66903b82fc97 req-b701d45b-89cf-485e-8d3d-9a443eedfb56 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "e66baae9-d488-46a4-8347-623030fc8b69", "name": "network-vif-unplugged", "status": "completed", "tag": "ed0a7ee7-b878-4680-b977-42bcc7c86cb8"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:14:48.357162 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-e5d3a117-d04c-47e4-8f83-bcfea6e0ff8a tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] Request method failure captured: Jul 16 21:14:48.357162 ncdv-master-1 devstack@n-api.service[138681]: request: GET /compute/v2.1/servers/dada2ea0-2ef4-4a30-8e2d-40177d0bafce HTTP/1.1 Jul 16 21:14:48.357162 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:14:48.357162 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:14:48.357162 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:14:48.357162 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 21:14:48.357162 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:14:48.357162 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:14:48.357162 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:14:48.357162 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:14:48.357162 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0yoB0XYghSUBtF84YOYGfB4_xYzUA_F4WrRobF-Rt4qMvE82h1iXAkIM12_RAx3rpkycxMLJn3NNCBruBkg08dFc8vTeMt2laJatIrAsrctW_jXBNb9rR0ETfgDp7wIjwtKtViFNgoI7i99au4p9qB9nlk-sgKFBAUCsSzgT232xMd8 Jul 16 21:14:48.357162 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:14:48.357162 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:14:48.357162 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:14:48.357162 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:14:48.357162 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:14:48.357162 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:14:48.357162 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: 6c2095f96f8d45bb9e62c517678c3dcd Jul 16 21:14:48.357162 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 Jul 16 21:14:48.357162 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:14:48.357162 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:14:48.358921 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/6c2095f96f8d45bb9e62c517678c3dcd"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/6c2095f96f8d45bb9e62c517678c3dcd"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_6c2095f96f8d45bb9e62c517678c3dcd"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/6c2095f96f8d45bb9e62c517678c3dcd"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:14:48.358921 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 Jul 16 21:14:48.358921 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: 6c2095f96f8d45bb9e62c517678c3dcd Jul 16 21:14:48.358921 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 Jul 16 21:14:48.358921 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project Jul 16 21:14:48.358921 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:14:48.358921 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:14:48.358921 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: 2eba2ac21c344394a7d3f605775b8713 Jul 16 21:14:48.358921 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project Jul 16 21:14:48.358921 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:14:48.358921 ncdv-master-1 devstack@n-api.service[138681]: exception: Instance dada2ea0-2ef4-4a30-8e2d-40177d0bafce could not be found. Jul 16 21:14:48.358921 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': 'dada2ea0-2ef4-4a30-8e2d-40177d0bafce'} Jul 16 21:14:48.359848 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:14:48.359848 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:14:48.359848 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:14:48.359848 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:14:48.359848 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:14:48.359848 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:14:48.359848 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:14:48.359848 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:14:48.359848 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:14:48.359848 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:14:48.359848 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance dada2ea0-2ef4-4a30-8e2d-40177d0bafce could not be found. Jul 16 21:14:48.359848 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:14:48.359848 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:14:48.359848 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:14:48.359848 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:14:48.359848 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:14:48.359848 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:14:48.359848 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:14:48.359848 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:14:48.359848 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:14:48.361119 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:14:48.361119 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:14:48.361119 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:14:48.361119 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:14:48.361119 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:14:48.361119 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:14:48.361119 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:14:48.361119 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance dada2ea0-2ef4-4a30-8e2d-40177d0bafce could not be found. Jul 16 21:14:48.361119 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:14:48.361119 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:14:48.361119 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:14:48.361119 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:14:48.361119 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:14:48.361119 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:14:48.361119 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:14:48.361119 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:14:48.361119 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance dada2ea0-2ef4-4a30-8e2d-40177d0bafce could not be found. Jul 16 21:14:48.361119 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:14:48.362681 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-e5d3a117-d04c-47e4-8f83-bcfea6e0ff8a tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] Returning 404 to user: Instance dada2ea0-2ef4-4a30-8e2d-40177d0bafce could not be found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:14:48.362681 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-e5d3a117-d04c-47e4-8f83-bcfea6e0ff8a tempest-FloatingIPsAssociationNegativeTestJSON-1411074507 tempest-FloatingIPsAssociationNegativeTestJSON-1411074507-project] 10.222.0.16 "GET /compute/v2.1/servers/dada2ea0-2ef4-4a30-8e2d-40177d0bafce" status: 404 len: 111 microversion: 2.1 time: 0.092610 Jul 16 21:14:48.362681 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 780/1607] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:14:48 2022] GET /compute/v2.1/servers/dada2ea0-2ef4-4a30-8e2d-40177d0bafce => generated 111 bytes in 94 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:14:48.381380 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-06de9228-b85d-4d55-b60a-6bcae5eba243 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:14:48.382013 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-06de9228-b85d-4d55-b60a-6bcae5eba243 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] [instance: 69cabe57-da71-4a02-b4ee-c0d84b3d32a2] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:14:48.401097 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-06de9228-b85d-4d55-b60a-6bcae5eba243 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:48.401558 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-06de9228-b85d-4d55-b60a-6bcae5eba243 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:48.403865 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-06de9228-b85d-4d55-b60a-6bcae5eba243 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:48.406945 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-06de9228-b85d-4d55-b60a-6bcae5eba243 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:48.407134 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-42fe12ee-25f1-4b28-b8ef-66903b82fc97 req-b701d45b-89cf-485e-8d3d-9a443eedfb56 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:48.410296 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-42fe12ee-25f1-4b28-b8ef-66903b82fc97 req-b701d45b-89cf-485e-8d3d-9a443eedfb56 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:48.457701 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:14:48.501028 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-06de9228-b85d-4d55-b60a-6bcae5eba243 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:48.501897 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-06de9228-b85d-4d55-b60a-6bcae5eba243 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:48.502537 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.objects.instance [None req-06de9228-b85d-4d55-b60a-6bcae5eba243 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lazy-loading 'fault' on Instance uuid 69cabe57-da71-4a02-b4ee-c0d84b3d32a2 {{(pid=138681) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:14:48.546770 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-42fe12ee-25f1-4b28-b8ef-66903b82fc97 req-b701d45b-89cf-485e-8d3d-9a443eedfb56 service nova] Creating event network-vif-unplugged:ed0a7ee7-b878-4680-b977-42bcc7c86cb8 for instance e66baae9-d488-46a4-8347-623030fc8b69 on nch2-master-1 Jul 16 21:14:48.576625 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-42fe12ee-25f1-4b28-b8ef-66903b82fc97 req-b701d45b-89cf-485e-8d3d-9a443eedfb56 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.246468 Jul 16 21:14:48.592743 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 828/1608] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:14:48 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 269 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:14:48.628118 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-33908bf4-f32a-4e8c-bd2f-2002c2cd51ea req-e94dfe55-7896-4262-9b29-7b9ebafc1ab5 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "69cabe57-da71-4a02-b4ee-c0d84b3d32a2", "name": "network-vif-deleted", "tag": "9e27e304-0422-4ee7-8ca0-2d670e12f5ee"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:14:48.633336 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-06de9228-b85d-4d55-b60a-6bcae5eba243 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '53af488f3b0242a797f539284b78d59e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bb1ede0d0b6941cc97278e6c617f9b01', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:14:48.642948 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-06de9228-b85d-4d55-b60a-6bcae5eba243 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] 10.222.0.16 "GET /compute/v2.1/servers/69cabe57-da71-4a02-b4ee-c0d84b3d32a2" status: 200 len: 1302 microversion: 2.1 time: 0.282628 Jul 16 21:14:48.643567 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 781/1609] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:14:48 2022] GET /compute/v2.1/servers/69cabe57-da71-4a02-b4ee-c0d84b3d32a2 => generated 1302 bytes in 284 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:14:48.667235 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-33908bf4-f32a-4e8c-bd2f-2002c2cd51ea req-e94dfe55-7896-4262-9b29-7b9ebafc1ab5 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:48.667990 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-33908bf4-f32a-4e8c-bd2f-2002c2cd51ea req-e94dfe55-7896-4262-9b29-7b9ebafc1ab5 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:48.694986 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-33908bf4-f32a-4e8c-bd2f-2002c2cd51ea req-e94dfe55-7896-4262-9b29-7b9ebafc1ab5 service nova] Creating event network-vif-deleted:9e27e304-0422-4ee7-8ca0-2d670e12f5ee for instance 69cabe57-da71-4a02-b4ee-c0d84b3d32a2 on nch1-master-1 Jul 16 21:14:48.708290 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-33908bf4-f32a-4e8c-bd2f-2002c2cd51ea req-e94dfe55-7896-4262-9b29-7b9ebafc1ab5 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.113324 Jul 16 21:14:48.709500 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 829/1610] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:14:48 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 116 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:14:48.873984 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-f6ff4472-04f0-4def-bd61-86f457822c85 tempest-TenantUsagesNegativeTestJSON-2047252272 tempest-TenantUsagesNegativeTestJSON-2047252272-project-admin] 10.222.0.16 "GET /compute/v2.1/os-simple-tenant-usage/?start=2022-07-15T21%3A14%3A48.618741&end=2022-07-17T21%3A14%3A48.618741" status: 404 len: 112 microversion: - time: 0.229358 Jul 16 21:14:48.888679 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-e1d82fb0-765f-41a2-a695-84887cbc19d3 tempest-TenantUsagesNegativeTestJSON-2047252272 tempest-TenantUsagesNegativeTestJSON-2047252272-project-admin] Calling method '.version_select at 0x7fdd9d2a58b0>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:14:48.890664 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 782/1611] 10.222.0.16 () {58 vars in 1315 bytes} [Sat Jul 16 21:14:48 2022] GET /compute/v2.1/os-simple-tenant-usage/?start=2022-07-15T21%3A14%3A48.618741&end=2022-07-17T21%3A14%3A48.618741 => generated 112 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 228 bytes (1 switches on core 0) Jul 16 21:14:48.899607 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-e1d82fb0-765f-41a2-a695-84887cbc19d3 tempest-TenantUsagesNegativeTestJSON-2047252272 tempest-TenantUsagesNegativeTestJSON-2047252272-project-admin] HTTP exception thrown: Invalid start time. The start time cannot occur after the end time. Jul 16 21:14:48.902000 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-e1d82fb0-765f-41a2-a695-84887cbc19d3 tempest-TenantUsagesNegativeTestJSON-2047252272 tempest-TenantUsagesNegativeTestJSON-2047252272-project-admin] Request method failure captured: Jul 16 21:14:48.902000 ncdv-master-1 devstack@n-api.service[138680]: request: GET /compute/v2.1/os-simple-tenant-usage/9b260142a0b749f29fef8f948a33ac69?start=2022-07-17T21%3A14%3A48.618741&end=2022-07-15T21%3A14%3A48.618741 HTTP/1.1 Jul 16 21:14:48.902000 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:14:48.902000 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:14:48.902000 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:14:48.902000 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 0 Jul 16 21:14:48.902000 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:14:48.902000 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:14:48.902000 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:14:48.902000 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:14:48.902000 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0ypI6w83B3NrtB6nnV1ByoX5VUVs0a0oxXz-_-lOuqkp0nwgfSHMakIpiT5ww6n7V8kQAZEhXrkw1KZuKG1_NFUfv3oHeeMjMBijHGp0NPXVPyEIMpuM260qJ1TGFKjKrFewtjlWDpHPkInfGYlhLeyeSmM7M8KZhmgsiCIxkLvlERE Jul 16 21:14:48.902000 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:14:48.902000 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:14:48.902000 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:14:48.902000 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:14:48.902000 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:14:48.902000 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:14:48.902000 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: db035c9138754b57b83510fca1331059 Jul 16 21:14:48.902000 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-TenantUsagesNegativeTestJSON-2047252272 Jul 16 21:14:48.902000 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,admin,reader Jul 16 21:14:48.902000 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,admin,reader Jul 16 21:14:48.905134 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/db035c9138754b57b83510fca1331059"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/db035c9138754b57b83510fca1331059"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_db035c9138754b57b83510fca1331059"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/db035c9138754b57b83510fca1331059"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:14:48.905134 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-TenantUsagesNegativeTestJSON-2047252272 Jul 16 21:14:48.905134 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: db035c9138754b57b83510fca1331059 Jul 16 21:14:48.905134 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-TenantUsagesNegativeTestJSON-2047252272 Jul 16 21:14:48.905134 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-TenantUsagesNegativeTestJSON-2047252272-project-admin Jul 16 21:14:48.905134 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:14:48.905134 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:14:48.905134 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: c900ca5debac4b64a2da9b3abdc431e1 Jul 16 21:14:48.905134 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-TenantUsagesNegativeTestJSON-2047252272-project-admin Jul 16 21:14:48.905134 ncdv-master-1 devstack@n-api.service[138680]: method: Jul 16 21:14:48.905134 ncdv-master-1 devstack@n-api.service[138680]: exception: Invalid start time. The start time cannot occur after the end time. Jul 16 21:14:48.905134 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': '9b260142a0b749f29fef8f948a33ac69'} Jul 16 21:14:48.907200 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:14:48.907200 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:14:48.907200 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:14:48.907200 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:14:48.907200 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:14:48.907200 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:14:48.907200 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Jul 16 21:14:48.907200 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Jul 16 21:14:48.907200 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper Jul 16 21:14:48.907200 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 16 21:14:48.907200 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:14:48.907200 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:14:48.907200 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/simple_tenant_usage.py", line 292, in show Jul 16 21:14:48.907200 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return self._show(req, id) Jul 16 21:14:48.907200 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/simple_tenant_usage.py", line 338, in _show Jul 16 21:14:48.907200 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi (period_start, period_stop, ignore) = self._get_datetime_range( Jul 16 21:14:48.907200 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/simple_tenant_usage.py", line 259, in _get_datetime_range Jul 16 21:14:48.907200 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=msg) Jul 16 21:14:48.907200 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid start time. The start time cannot occur after the end time. Jul 16 21:14:48.910131 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:14:48.910131 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:14:48.910131 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:14:48.910131 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:14:48.910131 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:14:48.910131 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:14:48.910131 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:14:48.910131 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:14:48.910131 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid start time. The start time cannot occur after the end time. Jul 16 21:14:48.910131 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:14:48.910131 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-e1d82fb0-765f-41a2-a695-84887cbc19d3 tempest-TenantUsagesNegativeTestJSON-2047252272 tempest-TenantUsagesNegativeTestJSON-2047252272-project-admin] Returning 400 to user: Invalid start time. The start time cannot occur after the end time. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:14:48.910131 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-e1d82fb0-765f-41a2-a695-84887cbc19d3 tempest-TenantUsagesNegativeTestJSON-2047252272 tempest-TenantUsagesNegativeTestJSON-2047252272-project-admin] 10.222.0.16 "GET /compute/v2.1/os-simple-tenant-usage/9b260142a0b749f29fef8f948a33ac69?start=2022-07-17T21%3A14%3A48.618741&end=2022-07-15T21%3A14%3A48.618741" status: 400 len: 111 microversion: 2.1 time: 0.019872 Jul 16 21:14:48.932732 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 830/1612] 10.222.0.16 () {58 vars in 1411 bytes} [Sat Jul 16 21:14:48 2022] GET /compute/v2.1/os-simple-tenant-usage/9b260142a0b749f29fef8f948a33ac69?start=2022-07-17T21%3A14%3A48.618741&end=2022-07-15T21%3A14%3A48.618741 => generated 111 bytes in 50 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Jul 16 21:14:49.069690 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-cb44b4ba-1901-4561-bd24-fc49f0f68483 tempest-TenantUsagesNegativeTestJSON-2071156129 tempest-TenantUsagesNegativeTestJSON-2071156129-project] Calling method '.version_select at 0x7fdd9d492700>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:14:49.072135 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-cb44b4ba-1901-4561-bd24-fc49f0f68483 tempest-TenantUsagesNegativeTestJSON-2071156129 tempest-TenantUsagesNegativeTestJSON-2071156129-project] Policy check for os_compute_api:os-simple-tenant-usage:list failed with credentials {'is_admin': False, 'user_id': '99b41c7e1ab3428495058e583156f0a0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9b260142a0b749f29fef8f948a33ac69', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:14:49.074069 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-cb44b4ba-1901-4561-bd24-fc49f0f68483 tempest-TenantUsagesNegativeTestJSON-2071156129 tempest-TenantUsagesNegativeTestJSON-2071156129-project] Request method failure captured: Jul 16 21:14:49.074069 ncdv-master-1 devstack@n-api.service[138681]: request: GET /compute/v2.1/os-simple-tenant-usage?start=2022-07-15T21%3A14%3A48.618741&end=2022-07-17T21%3A14%3A48.618741&detailed=1 HTTP/1.1 Jul 16 21:14:49.074069 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:14:49.074069 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:14:49.074069 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:14:49.074069 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 21:14:49.074069 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:14:49.074069 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:14:49.074069 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:14:49.074069 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:14:49.074069 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0ypHMIDA_Z8roS1bEsc5EFiCBC9YZHVNqFMH3Pw8ujS8Vf1Ed1mxFiVruxGDHeisOA7mPwhAHhCxlplC1uixNr2FyUNlAEygB-41DeN_eFhwrkWCkoZwiip0lKY0ixyAw0Fpd4Eumm8TyI-qcsABmevdvwmQwPECEneyOnSBFzOe-1M Jul 16 21:14:49.074069 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:14:49.074069 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:14:49.074069 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:14:49.074069 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:14:49.074069 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:14:49.074069 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:14:49.074069 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: 9b260142a0b749f29fef8f948a33ac69 Jul 16 21:14:49.074069 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-TenantUsagesNegativeTestJSON-2071156129 Jul 16 21:14:49.074069 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:14:49.074069 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:14:49.091451 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/9b260142a0b749f29fef8f948a33ac69"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/9b260142a0b749f29fef8f948a33ac69"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_9b260142a0b749f29fef8f948a33ac69"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/9b260142a0b749f29fef8f948a33ac69"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:14:49.091451 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-TenantUsagesNegativeTestJSON-2071156129 Jul 16 21:14:49.091451 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: 9b260142a0b749f29fef8f948a33ac69 Jul 16 21:14:49.091451 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-TenantUsagesNegativeTestJSON-2071156129 Jul 16 21:14:49.091451 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-TenantUsagesNegativeTestJSON-2071156129-project Jul 16 21:14:49.091451 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:14:49.091451 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:14:49.091451 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: 99b41c7e1ab3428495058e583156f0a0 Jul 16 21:14:49.091451 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-TenantUsagesNegativeTestJSON-2071156129-project Jul 16 21:14:49.091451 ncdv-master-1 devstack@n-api.service[138681]: method: Jul 16 21:14:49.091451 ncdv-master-1 devstack@n-api.service[138681]: exception: Policy doesn't allow os_compute_api:os-simple-tenant-usage:list to be performed. Jul 16 21:14:49.091451 ncdv-master-1 devstack@n-api.service[138681]: action_args: {} Jul 16 21:14:49.091451 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:14:49.093484 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:14:49.093484 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:14:49.093484 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:14:49.093484 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:14:49.093484 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:14:49.093484 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Jul 16 21:14:49.093484 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Jul 16 21:14:49.093484 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper Jul 16 21:14:49.093484 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 16 21:14:49.093484 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:14:49.093484 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:14:49.093484 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/simple_tenant_usage.py", line 277, in index Jul 16 21:14:49.093484 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return self._index(req) Jul 16 21:14:49.093484 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/simple_tenant_usage.py", line 297, in _index Jul 16 21:14:49.093484 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi context.can(stu_policies.POLICY_ROOT % 'list') Jul 16 21:14:49.093484 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/context.py", line 240, in can Jul 16 21:14:49.093484 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return policy.authorize(self, action, target) Jul 16 21:14:49.093484 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 199, in authorize Jul 16 21:14:49.093484 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi LOG.debug('Policy check for %(action)s failed with credentials ' Jul 16 21:14:49.095078 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 16 21:14:49.095078 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi self.force_reraise() Jul 16 21:14:49.095078 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 16 21:14:49.095078 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise self.value Jul 16 21:14:49.095078 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 186, in authorize Jul 16 21:14:49.095078 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi result = _ENFORCER.authorize(action, target, context, Jul 16 21:14:49.095078 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1184, in authorize Jul 16 21:14:49.095078 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return self.enforce( Jul 16 21:14:49.095078 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1088, in enforce Jul 16 21:14:49.095078 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc(*args, **kwargs) Jul 16 21:14:49.095078 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.PolicyNotAuthorized: Policy doesn't allow os_compute_api:os-simple-tenant-usage:list to be performed. Jul 16 21:14:49.095078 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:14:49.095078 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:14:49.095078 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:14:49.095078 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:14:49.095078 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:14:49.095078 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:14:49.095078 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 354, in __exit__ Jul 16 21:14:49.095078 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(webob.exc.HTTPForbidden( Jul 16 21:14:49.096726 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Policy doesn't allow os_compute_api:os-simple-tenant-usage:list to be performed. Jul 16 21:14:49.096726 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:14:49.096726 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-cb44b4ba-1901-4561-bd24-fc49f0f68483 tempest-TenantUsagesNegativeTestJSON-2071156129 tempest-TenantUsagesNegativeTestJSON-2071156129-project] Returning 403 to user: Policy doesn't allow os_compute_api:os-simple-tenant-usage:list to be performed. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:14:49.096726 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-cb44b4ba-1901-4561-bd24-fc49f0f68483 tempest-TenantUsagesNegativeTestJSON-2071156129 tempest-TenantUsagesNegativeTestJSON-2071156129-project] 10.222.0.16 "GET /compute/v2.1/os-simple-tenant-usage?start=2022-07-15T21%3A14%3A48.618741&end=2022-07-17T21%3A14%3A48.618741&detailed=1" status: 403 len: 123 microversion: 2.1 time: 0.153513 Jul 16 21:14:49.096726 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 783/1613] 10.222.0.16 () {58 vars in 1334 bytes} [Sat Jul 16 21:14:48 2022] GET /compute/v2.1/os-simple-tenant-usage?start=2022-07-15T21%3A14%3A48.618741&end=2022-07-17T21%3A14%3A48.618741&detailed=1 => generated 123 bytes in 155 msecs (HTTP/1.1 403) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:14:49.282296 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-3664b5d0-18e3-4891-a4dd-f31f78a0ea82 tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:14:49.283672 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-3664b5d0-18e3-4891-a4dd-f31f78a0ea82 tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] [instance: 4602867b-88ed-43b1-9ee6-bf661e2d7d99] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:14:49.311203 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3664b5d0-18e3-4891-a4dd-f31f78a0ea82 tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:49.313201 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3664b5d0-18e3-4891-a4dd-f31f78a0ea82 tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:49.313201 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3664b5d0-18e3-4891-a4dd-f31f78a0ea82 tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:49.313201 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3664b5d0-18e3-4891-a4dd-f31f78a0ea82 tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:49.315621 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-22777c29-1454-42b4-8c20-c690d8062a63 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:14:49.316237 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-22777c29-1454-42b4-8c20-c690d8062a63 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] [instance: e66baae9-d488-46a4-8347-623030fc8b69] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:14:49.341033 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-22777c29-1454-42b4-8c20-c690d8062a63 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:49.342084 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-22777c29-1454-42b4-8c20-c690d8062a63 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:49.343367 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-22777c29-1454-42b4-8c20-c690d8062a63 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:49.343989 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-22777c29-1454-42b4-8c20-c690d8062a63 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:49.454070 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-22777c29-1454-42b4-8c20-c690d8062a63 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4e9df82f95ac4779859972403869096a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4337575622d04a26942905c9dc044ba7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:14:49.468971 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-22777c29-1454-42b4-8c20-c690d8062a63 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] 10.222.0.16 "GET /compute/v2.1/servers/e66baae9-d488-46a4-8347-623030fc8b69" status: 200 len: 1309 microversion: 2.1 time: 0.157420 Jul 16 21:14:49.471328 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 784/1614] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:14:49 2022] GET /compute/v2.1/servers/e66baae9-d488-46a4-8347-623030fc8b69 => generated 1309 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:14:49.525799 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-3664b5d0-18e3-4891-a4dd-f31f78a0ea82 tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c1aea87154224f1c81428ddef4438801', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45d9edd2fe854483a6f0d676efbd6622', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:14:49.545765 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-3664b5d0-18e3-4891-a4dd-f31f78a0ea82 tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] 10.222.0.16 "GET /compute/v2.1/servers/4602867b-88ed-43b1-9ee6-bf661e2d7d99" status: 200 len: 1435 microversion: 2.1 time: 0.268723 Jul 16 21:14:49.558942 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 831/1615] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:14:49 2022] GET /compute/v2.1/servers/4602867b-88ed-43b1-9ee6-bf661e2d7d99 => generated 1435 bytes in 274 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:14:49.665186 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-162bae8a-2c3d-475e-aa88-4a920cbed64b tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:14:49.665814 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-162bae8a-2c3d-475e-aa88-4a920cbed64b tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] [instance: 69cabe57-da71-4a02-b4ee-c0d84b3d32a2] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:14:49.672453 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-162bae8a-2c3d-475e-aa88-4a920cbed64b tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:49.672962 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-162bae8a-2c3d-475e-aa88-4a920cbed64b tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:49.674025 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-162bae8a-2c3d-475e-aa88-4a920cbed64b tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:49.674479 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-162bae8a-2c3d-475e-aa88-4a920cbed64b tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:49.689320 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-162bae8a-2c3d-475e-aa88-4a920cbed64b tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] HTTP exception thrown: Instance 69cabe57-da71-4a02-b4ee-c0d84b3d32a2 could not be found. Jul 16 21:14:49.690905 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-162bae8a-2c3d-475e-aa88-4a920cbed64b tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Request method failure captured: Jul 16 21:14:49.690905 ncdv-master-1 devstack@n-api.service[138681]: request: GET /compute/v2.1/servers/69cabe57-da71-4a02-b4ee-c0d84b3d32a2 HTTP/1.1 Jul 16 21:14:49.690905 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:14:49.690905 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:14:49.690905 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:14:49.690905 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 21:14:49.690905 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:14:49.690905 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:14:49.690905 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:14:49.690905 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:14:49.690905 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0yoCD7IX15brFQUHVPKPIJrpEu20fxENHCEz31UnXrkVZbNHxjQ0ElqFOE7MXQnga3bWs8iQFWXWTDFYHn6RqK2SreYxppaBRq4w7eCzu-cpcuRq-bR3warXbUzkwx17fxxMPjZ200EVRXuxxzJd_xS8-ssDDdZBxRLNj6zLIJYowVk Jul 16 21:14:49.690905 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:14:49.690905 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:14:49.690905 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:14:49.690905 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:14:49.690905 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:14:49.690905 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:14:49.690905 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: bb1ede0d0b6941cc97278e6c617f9b01 Jul 16 21:14:49.690905 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-ServersTestMultiNic-262233723 Jul 16 21:14:49.690905 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:14:49.690905 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:14:49.694735 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/bb1ede0d0b6941cc97278e6c617f9b01"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/bb1ede0d0b6941cc97278e6c617f9b01"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_bb1ede0d0b6941cc97278e6c617f9b01"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/bb1ede0d0b6941cc97278e6c617f9b01"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:14:49.694735 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-ServersTestMultiNic-262233723 Jul 16 21:14:49.694735 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: bb1ede0d0b6941cc97278e6c617f9b01 Jul 16 21:14:49.694735 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-ServersTestMultiNic-262233723 Jul 16 21:14:49.694735 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-ServersTestMultiNic-262233723-project Jul 16 21:14:49.694735 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:14:49.694735 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:14:49.694735 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: 53af488f3b0242a797f539284b78d59e Jul 16 21:14:49.694735 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-ServersTestMultiNic-262233723-project Jul 16 21:14:49.694735 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:14:49.694735 ncdv-master-1 devstack@n-api.service[138681]: exception: Instance 69cabe57-da71-4a02-b4ee-c0d84b3d32a2 could not be found. Jul 16 21:14:49.694735 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': '69cabe57-da71-4a02-b4ee-c0d84b3d32a2'} Jul 16 21:14:49.695808 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:14:49.695808 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:14:49.695808 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:14:49.695808 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:14:49.695808 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:14:49.695808 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:14:49.695808 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:14:49.695808 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:14:49.695808 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:14:49.695808 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:14:49.695808 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 69cabe57-da71-4a02-b4ee-c0d84b3d32a2 could not be found. Jul 16 21:14:49.695808 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:14:49.695808 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:14:49.695808 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:14:49.695808 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:14:49.695808 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:14:49.695808 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:14:49.695808 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:14:49.695808 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:14:49.695808 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:14:49.697347 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:14:49.697347 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:14:49.697347 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:14:49.697347 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:14:49.697347 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:14:49.697347 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:14:49.697347 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:14:49.697347 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 69cabe57-da71-4a02-b4ee-c0d84b3d32a2 could not be found. Jul 16 21:14:49.697347 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:14:49.697347 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:14:49.697347 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:14:49.697347 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:14:49.697347 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:14:49.697347 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:14:49.697347 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:14:49.697347 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:14:49.697347 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 69cabe57-da71-4a02-b4ee-c0d84b3d32a2 could not be found. Jul 16 21:14:49.697347 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:14:49.698795 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-162bae8a-2c3d-475e-aa88-4a920cbed64b tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Returning 404 to user: Instance 69cabe57-da71-4a02-b4ee-c0d84b3d32a2 could not be found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:14:49.698795 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-162bae8a-2c3d-475e-aa88-4a920cbed64b tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] 10.222.0.16 "GET /compute/v2.1/servers/69cabe57-da71-4a02-b4ee-c0d84b3d32a2" status: 404 len: 111 microversion: 2.1 time: 0.034117 Jul 16 21:14:49.698795 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 785/1616] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:14:49 2022] GET /compute/v2.1/servers/69cabe57-da71-4a02-b4ee-c0d84b3d32a2 => generated 111 bytes in 39 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:14:50.490751 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-64fca2ab-82fe-41aa-885e-4bc4687667b7 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:14:50.491757 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-64fca2ab-82fe-41aa-885e-4bc4687667b7 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] [instance: e66baae9-d488-46a4-8347-623030fc8b69] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:14:50.519184 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-64fca2ab-82fe-41aa-885e-4bc4687667b7 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:50.519877 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-64fca2ab-82fe-41aa-885e-4bc4687667b7 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:50.524519 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-64fca2ab-82fe-41aa-885e-4bc4687667b7 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:50.525244 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-64fca2ab-82fe-41aa-885e-4bc4687667b7 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:50.567900 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-a54fa499-de5a-4b23-8ed7-4c08c543e3f0 tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:14:50.569011 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-a54fa499-de5a-4b23-8ed7-4c08c543e3f0 tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] [instance: 4602867b-88ed-43b1-9ee6-bf661e2d7d99] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:14:50.591122 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a54fa499-de5a-4b23-8ed7-4c08c543e3f0 tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:50.594991 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a54fa499-de5a-4b23-8ed7-4c08c543e3f0 tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:50.597512 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a54fa499-de5a-4b23-8ed7-4c08c543e3f0 tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:50.601593 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a54fa499-de5a-4b23-8ed7-4c08c543e3f0 tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:50.611585 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-64fca2ab-82fe-41aa-885e-4bc4687667b7 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:50.612317 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-64fca2ab-82fe-41aa-885e-4bc4687667b7 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:50.619508 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.objects.instance [None req-64fca2ab-82fe-41aa-885e-4bc4687667b7 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lazy-loading 'fault' on Instance uuid e66baae9-d488-46a4-8347-623030fc8b69 {{(pid=138680) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:14:50.675928 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a54fa499-de5a-4b23-8ed7-4c08c543e3f0 tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:50.683492 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a54fa499-de5a-4b23-8ed7-4c08c543e3f0 tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:50.684095 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.objects.instance [None req-a54fa499-de5a-4b23-8ed7-4c08c543e3f0 tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Lazy-loading 'fault' on Instance uuid 4602867b-88ed-43b1-9ee6-bf661e2d7d99 {{(pid=138681) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:14:50.830781 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-64fca2ab-82fe-41aa-885e-4bc4687667b7 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4e9df82f95ac4779859972403869096a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4337575622d04a26942905c9dc044ba7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:14:50.871519 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-a54fa499-de5a-4b23-8ed7-4c08c543e3f0 tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c1aea87154224f1c81428ddef4438801', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45d9edd2fe854483a6f0d676efbd6622', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:14:50.876567 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-64fca2ab-82fe-41aa-885e-4bc4687667b7 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] 10.222.0.16 "GET /compute/v2.1/servers/e66baae9-d488-46a4-8347-623030fc8b69" status: 200 len: 1314 microversion: 2.1 time: 0.388784 Jul 16 21:14:50.879137 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-a54fa499-de5a-4b23-8ed7-4c08c543e3f0 tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] 10.222.0.16 "GET /compute/v2.1/servers/4602867b-88ed-43b1-9ee6-bf661e2d7d99" status: 200 len: 1311 microversion: 2.1 time: 0.317225 Jul 16 21:14:50.881654 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 786/1617] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:14:50 2022] GET /compute/v2.1/servers/4602867b-88ed-43b1-9ee6-bf661e2d7d99 => generated 1311 bytes in 320 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:14:50.893630 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 832/1618] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:14:50 2022] GET /compute/v2.1/servers/e66baae9-d488-46a4-8347-623030fc8b69 => generated 1314 bytes in 409 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:14:50.910044 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-4eeff334-3015-4085-8d1c-d411c67f4ff1 req-445382c3-f7e0-4b15-81ed-160a1600e022 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "e66baae9-d488-46a4-8347-623030fc8b69", "name": "network-vif-deleted", "tag": "ed0a7ee7-b878-4680-b977-42bcc7c86cb8"}, {"server_uuid": "4602867b-88ed-43b1-9ee6-bf661e2d7d99", "name": "network-vif-deleted", "tag": "081d646d-03f3-4a3f-8409-9106695a46bb"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:14:50.937114 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-4eeff334-3015-4085-8d1c-d411c67f4ff1 req-445382c3-f7e0-4b15-81ed-160a1600e022 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:50.938750 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-4eeff334-3015-4085-8d1c-d411c67f4ff1 req-445382c3-f7e0-4b15-81ed-160a1600e022 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:50.982016 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-4eeff334-3015-4085-8d1c-d411c67f4ff1 req-445382c3-f7e0-4b15-81ed-160a1600e022 service nova] Creating event network-vif-deleted:ed0a7ee7-b878-4680-b977-42bcc7c86cb8 for instance e66baae9-d488-46a4-8347-623030fc8b69 on nch2-master-1 Jul 16 21:14:50.982553 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-4eeff334-3015-4085-8d1c-d411c67f4ff1 req-445382c3-f7e0-4b15-81ed-160a1600e022 service nova] Creating event network-vif-deleted:081d646d-03f3-4a3f-8409-9106695a46bb for instance 4602867b-88ed-43b1-9ee6-bf661e2d7d99 on nch1-master-1 Jul 16 21:14:51.023281 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-4eeff334-3015-4085-8d1c-d411c67f4ff1 req-445382c3-f7e0-4b15-81ed-160a1600e022 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 354 microversion: 2.1 time: 0.136182 Jul 16 21:14:51.025422 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 787/1619] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:14:50 2022] POST /compute/v2.1/os-server-external-events => generated 354 bytes in 138 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:14:51.898950 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-ecd4598f-a79e-46f3-be11-ec0ca6e5ed94 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:14:51.899607 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-ecd4598f-a79e-46f3-be11-ec0ca6e5ed94 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] [instance: e66baae9-d488-46a4-8347-623030fc8b69] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:14:51.903197 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-3444729b-f442-4fae-a4e1-bed65ed9ff3b tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:14:51.903960 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-3444729b-f442-4fae-a4e1-bed65ed9ff3b tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] [instance: 4602867b-88ed-43b1-9ee6-bf661e2d7d99] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:14:51.919156 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ecd4598f-a79e-46f3-be11-ec0ca6e5ed94 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:51.920335 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ecd4598f-a79e-46f3-be11-ec0ca6e5ed94 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:51.921529 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ecd4598f-a79e-46f3-be11-ec0ca6e5ed94 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:51.922154 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ecd4598f-a79e-46f3-be11-ec0ca6e5ed94 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:51.927606 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3444729b-f442-4fae-a4e1-bed65ed9ff3b tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:51.928075 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3444729b-f442-4fae-a4e1-bed65ed9ff3b tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:51.928896 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3444729b-f442-4fae-a4e1-bed65ed9ff3b tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:51.935057 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3444729b-f442-4fae-a4e1-bed65ed9ff3b tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:51.963659 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-ecd4598f-a79e-46f3-be11-ec0ca6e5ed94 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] HTTP exception thrown: Instance e66baae9-d488-46a4-8347-623030fc8b69 could not be found. Jul 16 21:14:51.965573 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-ecd4598f-a79e-46f3-be11-ec0ca6e5ed94 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Request method failure captured: Jul 16 21:14:51.965573 ncdv-master-1 devstack@n-api.service[138680]: request: GET /compute/v2.1/servers/e66baae9-d488-46a4-8347-623030fc8b69 HTTP/1.1 Jul 16 21:14:51.965573 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:14:51.965573 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:14:51.965573 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:14:51.965573 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 0 Jul 16 21:14:51.965573 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:14:51.965573 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:14:51.965573 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:14:51.965573 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:14:51.965573 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0yoJFUAf_8Wtq8fI-wZdJwoKyg8rUEOC_eBMeqcRE2hk6JM7_ssDplugQwmk2S9xH7Oys7Ho2tw0kc9hmlY-jX_rR-I-9WVwQYFbyogU7ki99siEGGR9aVA2GKsp5nNcVHXBtjM3bI_zfa21XSWedvaSIVwZV8kEGixKYsBf2GMYKVI Jul 16 21:14:51.965573 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:14:51.965573 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:14:51.965573 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:14:51.965573 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:14:51.965573 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:14:51.965573 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:14:51.965573 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: 4337575622d04a26942905c9dc044ba7 Jul 16 21:14:51.965573 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-ImagesOneServerNegativeTestJSON-526509748 Jul 16 21:14:51.965573 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:14:51.965573 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:14:51.967592 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/4337575622d04a26942905c9dc044ba7"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/4337575622d04a26942905c9dc044ba7"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_4337575622d04a26942905c9dc044ba7"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/4337575622d04a26942905c9dc044ba7"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:14:51.967592 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-ImagesOneServerNegativeTestJSON-526509748 Jul 16 21:14:51.967592 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: 4337575622d04a26942905c9dc044ba7 Jul 16 21:14:51.967592 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-ImagesOneServerNegativeTestJSON-526509748 Jul 16 21:14:51.967592 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-ImagesOneServerNegativeTestJSON-526509748-project Jul 16 21:14:51.967592 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:14:51.967592 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:14:51.967592 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: 4e9df82f95ac4779859972403869096a Jul 16 21:14:51.967592 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-ImagesOneServerNegativeTestJSON-526509748-project Jul 16 21:14:51.967592 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 21:14:51.967592 ncdv-master-1 devstack@n-api.service[138680]: exception: Instance e66baae9-d488-46a4-8347-623030fc8b69 could not be found. Jul 16 21:14:51.967592 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': 'e66baae9-d488-46a4-8347-623030fc8b69'} Jul 16 21:14:51.968571 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:14:51.968571 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:14:51.968571 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:14:51.968571 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:14:51.968571 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:14:51.968571 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:14:51.968571 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:14:51.968571 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:14:51.968571 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:14:51.968571 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:14:51.968571 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance e66baae9-d488-46a4-8347-623030fc8b69 could not be found. Jul 16 21:14:51.968571 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:14:51.968571 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:14:51.968571 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:14:51.968571 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:14:51.968571 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:14:51.968571 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:14:51.968571 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:14:51.968571 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:14:51.968571 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:14:51.970408 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:14:51.970408 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:14:51.970408 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:14:51.970408 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:14:51.970408 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:14:51.970408 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:14:51.970408 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:14:51.970408 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance e66baae9-d488-46a4-8347-623030fc8b69 could not be found. Jul 16 21:14:51.970408 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:14:51.970408 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:14:51.970408 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:14:51.970408 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:14:51.970408 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:14:51.970408 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:14:51.970408 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:14:51.970408 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:14:51.970408 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance e66baae9-d488-46a4-8347-623030fc8b69 could not be found. Jul 16 21:14:51.970408 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:14:51.974446 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-3444729b-f442-4fae-a4e1-bed65ed9ff3b tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] HTTP exception thrown: Instance 4602867b-88ed-43b1-9ee6-bf661e2d7d99 could not be found. Jul 16 21:14:51.976266 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-3444729b-f442-4fae-a4e1-bed65ed9ff3b tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Request method failure captured: Jul 16 21:14:51.976266 ncdv-master-1 devstack@n-api.service[138681]: request: GET /compute/v2.1/servers/4602867b-88ed-43b1-9ee6-bf661e2d7d99 HTTP/1.1 Jul 16 21:14:51.976266 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:14:51.976266 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:14:51.976266 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:14:51.976266 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 21:14:51.976266 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:14:51.976266 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:14:51.976266 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:14:51.976266 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:14:51.976266 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0yn6K6QftoPpkEysn4UVt0CZkaXupfIonEWl3L75o2URhqIcdNiz9cIehYJ1L-ssxF-OWGGDzSzhCaPaoZNk55CU3ZOG4LNQVyuBHPyTObFqRRv53RHcWu8kd8Q8R3EopbISfq78hx_LODvbVr1VYmgBwBkt5FFXCv7w6P_-coX-Pnc Jul 16 21:14:51.976266 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:14:51.976266 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:14:51.976266 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:14:51.976266 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:14:51.976266 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:14:51.976266 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:14:51.976266 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: 45d9edd2fe854483a6f0d676efbd6622 Jul 16 21:14:51.976266 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-ServersAdminNegativeTestJSON-1083411175 Jul 16 21:14:51.976266 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:14:51.976266 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:14:51.977720 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/45d9edd2fe854483a6f0d676efbd6622"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/45d9edd2fe854483a6f0d676efbd6622"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_45d9edd2fe854483a6f0d676efbd6622"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/45d9edd2fe854483a6f0d676efbd6622"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:14:51.977720 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-ServersAdminNegativeTestJSON-1083411175 Jul 16 21:14:51.977720 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: 45d9edd2fe854483a6f0d676efbd6622 Jul 16 21:14:51.977720 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-ServersAdminNegativeTestJSON-1083411175 Jul 16 21:14:51.977720 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-ServersAdminNegativeTestJSON-1083411175-project Jul 16 21:14:51.977720 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:14:51.977720 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:14:51.977720 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: c1aea87154224f1c81428ddef4438801 Jul 16 21:14:51.977720 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-ServersAdminNegativeTestJSON-1083411175-project Jul 16 21:14:51.977720 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:14:51.977720 ncdv-master-1 devstack@n-api.service[138681]: exception: Instance 4602867b-88ed-43b1-9ee6-bf661e2d7d99 could not be found. Jul 16 21:14:51.977720 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': '4602867b-88ed-43b1-9ee6-bf661e2d7d99'} Jul 16 21:14:51.978601 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:14:51.978601 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:14:51.978601 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:14:51.978601 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:14:51.978601 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:14:51.978601 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:14:51.978601 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:14:51.978601 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:14:51.978601 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:14:51.978601 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:14:51.978601 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 4602867b-88ed-43b1-9ee6-bf661e2d7d99 could not be found. Jul 16 21:14:51.978601 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:14:51.978601 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:14:51.978601 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:14:51.978601 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:14:51.978601 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:14:51.978601 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:14:51.978601 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:14:51.978601 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:14:51.978601 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:14:51.981077 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:14:51.981077 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:14:51.981077 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:14:51.981077 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:14:51.981077 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:14:51.981077 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:14:51.981077 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:14:51.981077 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 4602867b-88ed-43b1-9ee6-bf661e2d7d99 could not be found. Jul 16 21:14:51.981077 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:14:51.981077 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:14:51.981077 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:14:51.981077 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:14:51.981077 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:14:51.981077 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:14:51.981077 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:14:51.981077 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:14:51.981077 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 4602867b-88ed-43b1-9ee6-bf661e2d7d99 could not be found. Jul 16 21:14:51.981077 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:14:51.984030 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-3444729b-f442-4fae-a4e1-bed65ed9ff3b tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] Returning 404 to user: Instance 4602867b-88ed-43b1-9ee6-bf661e2d7d99 could not be found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:14:51.984030 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-3444729b-f442-4fae-a4e1-bed65ed9ff3b tempest-ServersAdminNegativeTestJSON-1083411175 tempest-ServersAdminNegativeTestJSON-1083411175-project] 10.222.0.16 "GET /compute/v2.1/servers/4602867b-88ed-43b1-9ee6-bf661e2d7d99" status: 404 len: 111 microversion: 2.1 time: 0.083182 Jul 16 21:14:51.984030 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 788/1620] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:14:51 2022] GET /compute/v2.1/servers/4602867b-88ed-43b1-9ee6-bf661e2d7d99 => generated 111 bytes in 85 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:14:51.984548 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-ecd4598f-a79e-46f3-be11-ec0ca6e5ed94 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Returning 404 to user: Instance e66baae9-d488-46a4-8347-623030fc8b69 could not be found. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:14:51.984548 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-ecd4598f-a79e-46f3-be11-ec0ca6e5ed94 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] 10.222.0.16 "GET /compute/v2.1/servers/e66baae9-d488-46a4-8347-623030fc8b69" status: 404 len: 111 microversion: 2.1 time: 0.090186 Jul 16 21:14:51.994590 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 833/1621] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:14:51 2022] GET /compute/v2.1/servers/e66baae9-d488-46a4-8347-623030fc8b69 => generated 111 bytes in 91 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:14:52.002802 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-759cbf18-55fa-4e19-a308-eb7b85489450 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Action: 'create', calling method: .version_select at 0x7fdd9cdf9e50>, body: {"keypair": {"name": "tempest-keypair-660085568", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBA+b6mUu5olHYLiBxVJGFGw8RaiLPdx6j8k3bJ2FOpNZ1OwHHfLWUOVktSLu5ZsH3G/d3AGaLdVnlj3KMAY4IddwWPEF3bQnEuu/BM9001jU2Z1cHYcizh+jbU5blvTK1A=="}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:14:52.063314 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-759cbf18-55fa-4e19-a308-eb7b85489450 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Getting quotas for project 4337575622d04a26942905c9dc044ba7. Resources: {'key_pairs'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:14:52.071857 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-759cbf18-55fa-4e19-a308-eb7b85489450 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Getting quotas for user 4e9df82f95ac4779859972403869096a and project 4337575622d04a26942905c9dc044ba7. Resources: {'key_pairs'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:14:52.200260 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-759cbf18-55fa-4e19-a308-eb7b85489450 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] 10.222.0.16 "POST /compute/v2.1/os-keypairs" status: 200 len: 385 microversion: 2.1 time: 0.202289 Jul 16 21:14:52.201197 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 789/1622] 10.222.0.16 () {60 vars in 1158 bytes} [Sat Jul 16 21:14:51 2022] POST /compute/v2.1/os-keypairs => generated 385 bytes in 204 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:14:56.605960 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-a0ca88ee-bd0d-40e3-ab56-c73f1b4d6f1d tempest-ImagesMetadataTestJSON-1264454256 tempest-ImagesMetadataTestJSON-1264454256-project] Calling method '.version_select at 0x7fdd9d328790>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:14:56.731984 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-a0ca88ee-bd0d-40e3-ab56-c73f1b4d6f1d tempest-ImagesMetadataTestJSON-1264454256 tempest-ImagesMetadataTestJSON-1264454256-project] 10.222.0.16 "GET /compute/v2.1/images/9ab4ec44-a79d-4f52-93c5-c44ab4e3a79f" status: 200 len: 636 microversion: 2.1 time: 0.141217 Jul 16 21:14:56.739636 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 834/1623] 10.222.0.16 () {58 vars in 1232 bytes} [Sat Jul 16 21:14:56 2022] GET /compute/v2.1/images/9ab4ec44-a79d-4f52-93c5-c44ab4e3a79f => generated 636 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:14:56.748712 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-3ad22dce-05b5-48c0-a285-5a476e3b82d4 tempest-ImagesMetadataTestJSON-1264454256 tempest-ImagesMetadataTestJSON-1264454256-project] Action: 'update_all', calling method: .version_select at 0x7fdd9cdb5310>, body: {"metadata": {"os_version": "value1", "os_distro": "value2"}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:14:56.872400 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-3ad22dce-05b5-48c0-a285-5a476e3b82d4 tempest-ImagesMetadataTestJSON-1264454256 tempest-ImagesMetadataTestJSON-1264454256-project] Getting quotas for project a30b6074cef24df6ba020b01fc85b3e3. Resources: dict_keys(['metadata_items']) {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:14:56.881761 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-3ad22dce-05b5-48c0-a285-5a476e3b82d4 tempest-ImagesMetadataTestJSON-1264454256 tempest-ImagesMetadataTestJSON-1264454256-project] Getting quotas for user bd9e72b27f304a9383d7f3fdcd718972 and project a30b6074cef24df6ba020b01fc85b3e3. Resources: dict_keys(['metadata_items']) {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:14:57.651184 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-3ad22dce-05b5-48c0-a285-5a476e3b82d4 tempest-ImagesMetadataTestJSON-1264454256 tempest-ImagesMetadataTestJSON-1264454256-project] 10.222.0.16 "PUT /compute/v2.1/images/9ab4ec44-a79d-4f52-93c5-c44ab4e3a79f/metadata" status: 200 len: 61 microversion: 2.1 time: 0.908994 Jul 16 21:14:57.652340 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 790/1624] 10.222.0.16 () {60 vars in 1279 bytes} [Sat Jul 16 21:14:56 2022] PUT /compute/v2.1/images/9ab4ec44-a79d-4f52-93c5-c44ab4e3a79f/metadata => generated 61 bytes in 911 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Jul 16 21:14:57.662459 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-092ba48b-3c29-4f47-8722-620d9f248ab1 tempest-ImagesMetadataTestJSON-1264454256 tempest-ImagesMetadataTestJSON-1264454256-project] Calling method '.version_select at 0x7fdd9cd91a60>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:14:58.431997 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-092ba48b-3c29-4f47-8722-620d9f248ab1 tempest-ImagesMetadataTestJSON-1264454256 tempest-ImagesMetadataTestJSON-1264454256-project] 10.222.0.16 "DELETE /compute/v2.1/images/9ab4ec44-a79d-4f52-93c5-c44ab4e3a79f/metadata/os_version" status: 204 len: 0 microversion: 2.1 time: 0.774507 Jul 16 21:14:58.433686 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 835/1625] 10.222.0.16 () {58 vars in 1295 bytes} [Sat Jul 16 21:14:57 2022] DELETE /compute/v2.1/images/9ab4ec44-a79d-4f52-93c5-c44ab4e3a79f/metadata/os_version => generated 0 bytes in 777 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 16 21:14:58.454699 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-3f270a9d-4d58-4b9b-8de0-f5b224273b79 tempest-ImagesMetadataTestJSON-1264454256 tempest-ImagesMetadataTestJSON-1264454256-project] Calling method '.version_select at 0x7fdd9d025ca0>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:14:58.544063 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-3f270a9d-4d58-4b9b-8de0-f5b224273b79 tempest-ImagesMetadataTestJSON-1264454256 tempest-ImagesMetadataTestJSON-1264454256-project] 10.222.0.16 "GET /compute/v2.1/images/9ab4ec44-a79d-4f52-93c5-c44ab4e3a79f/metadata" status: 200 len: 37 microversion: 2.1 time: 0.105562 Jul 16 21:14:58.545279 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 836/1626] 10.222.0.16 () {58 vars in 1259 bytes} [Sat Jul 16 21:14:58 2022] GET /compute/v2.1/images/9ab4ec44-a79d-4f52-93c5-c44ab4e3a79f/metadata => generated 37 bytes in 108 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Jul 16 21:14:58.560311 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-08e5c783-a4b3-4883-bde7-4e8bf02d05fa tempest-ImagesMetadataTestJSON-1264454256 tempest-ImagesMetadataTestJSON-1264454256-project] Action: 'update_all', calling method: .version_select at 0x7fdd9d2a58b0>, body: {"metadata": {"os_version": "value1", "os_distro": "value2"}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:14:58.669009 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-08e5c783-a4b3-4883-bde7-4e8bf02d05fa tempest-ImagesMetadataTestJSON-1264454256 tempest-ImagesMetadataTestJSON-1264454256-project] Getting quotas for project a30b6074cef24df6ba020b01fc85b3e3. Resources: dict_keys(['metadata_items']) {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:14:58.683384 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-08e5c783-a4b3-4883-bde7-4e8bf02d05fa tempest-ImagesMetadataTestJSON-1264454256 tempest-ImagesMetadataTestJSON-1264454256-project] Getting quotas for user bd9e72b27f304a9383d7f3fdcd718972 and project a30b6074cef24df6ba020b01fc85b3e3. Resources: dict_keys(['metadata_items']) {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:14:58.697273 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-9c69c989-1cf4-46af-98b3-af3ed529908b tempest-FloatingIPDetailsNegativeTestJSON-580292985 tempest-FloatingIPDetailsNegativeTestJSON-580292985-project] Calling method '.version_select at 0x7fdd9ce39b80>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:14:58.793535 ncdv-master-1 devstack@n-api.service[138681]: DEBUG neutronclient.v2_0.client [None req-9c69c989-1cf4-46af-98b3-af3ed529908b tempest-FloatingIPDetailsNegativeTestJSON-580292985 tempest-FloatingIPDetailsNegativeTestJSON-580292985-project] Error message: {"NeutronError": {"type": "FloatingIPNotFound", "message": "Floating IP ce1d33e7-8d17-4e88-bbec-ce1513cf93e9 could not be found", "detail": ""}} {{(pid=138681) _handle_fault_response /usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py:262}} Jul 16 21:14:58.794587 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-9c69c989-1cf4-46af-98b3-af3ed529908b tempest-FloatingIPDetailsNegativeTestJSON-580292985 tempest-FloatingIPDetailsNegativeTestJSON-580292985-project] HTTP exception thrown: Floating IP not found for ID ce1d33e7-8d17-4e88-bbec-ce1513cf93e9 Jul 16 21:14:58.796284 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-9c69c989-1cf4-46af-98b3-af3ed529908b tempest-FloatingIPDetailsNegativeTestJSON-580292985 tempest-FloatingIPDetailsNegativeTestJSON-580292985-project] Request method failure captured: Jul 16 21:14:58.796284 ncdv-master-1 devstack@n-api.service[138681]: request: GET /compute/v2.1/os-floating-ips/ce1d33e7-8d17-4e88-bbec-ce1513cf93e9 HTTP/1.1 Jul 16 21:14:58.796284 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:14:58.796284 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:14:58.796284 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:14:58.796284 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 21:14:58.796284 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:14:58.796284 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:14:58.796284 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:14:58.796284 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:14:58.796284 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0ypS_TEt23LJDJDPi5CCJQykwcd-xBVZ-wWEUQGTn-N41M-rpWTrGAop6n7sOvNZ3RmuLEozC0Rvf0C8W1KX98F6FWs5JKjam9yZ-3rokyLszW0urrPgXLzxYXJ_R9Rqy2rUy-BMIAnCEwhFx4s3sGyLS1-dzMbqn32gCG1QYG9CoI4 Jul 16 21:14:58.796284 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:14:58.796284 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:14:58.796284 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:14:58.796284 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:14:58.796284 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:14:58.796284 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:14:58.796284 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: 31216bba3e1148f2b00f59a6913ce35b Jul 16 21:14:58.796284 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-FloatingIPDetailsNegativeTestJSON-580292985 Jul 16 21:14:58.796284 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:14:58.796284 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:14:58.797777 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/31216bba3e1148f2b00f59a6913ce35b"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/31216bba3e1148f2b00f59a6913ce35b"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_31216bba3e1148f2b00f59a6913ce35b"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/31216bba3e1148f2b00f59a6913ce35b"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:14:58.797777 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-FloatingIPDetailsNegativeTestJSON-580292985 Jul 16 21:14:58.797777 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: 31216bba3e1148f2b00f59a6913ce35b Jul 16 21:14:58.797777 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-FloatingIPDetailsNegativeTestJSON-580292985 Jul 16 21:14:58.797777 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-FloatingIPDetailsNegativeTestJSON-580292985-project Jul 16 21:14:58.797777 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:14:58.797777 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:14:58.797777 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: 94649762be79464785b4412c9b9a61fa Jul 16 21:14:58.797777 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-FloatingIPDetailsNegativeTestJSON-580292985-project Jul 16 21:14:58.797777 ncdv-master-1 devstack@n-api.service[138681]: method: Jul 16 21:14:58.797777 ncdv-master-1 devstack@n-api.service[138681]: exception: Floating IP not found for ID ce1d33e7-8d17-4e88-bbec-ce1513cf93e9 Jul 16 21:14:58.797777 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': 'ce1d33e7-8d17-4e88-bbec-ce1513cf93e9'} Jul 16 21:14:58.798632 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:14:58.798632 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:14:58.798632 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 2827, in get_floating_ip Jul 16 21:14:58.798632 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi fip = client.show_floatingip(id)['floatingip'] Jul 16 21:14:58.798632 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 197, in wrapper Jul 16 21:14:58.798632 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Jul 16 21:14:58.798632 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 1006, in show_floatingip Jul 16 21:14:58.798632 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return self.get(self.floatingip_path % (floatingip), params=_params) Jul 16 21:14:58.798632 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 197, in wrapper Jul 16 21:14:58.798632 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Jul 16 21:14:58.798632 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 356, in get Jul 16 21:14:58.798632 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return self.retry_request("GET", action, body=body, Jul 16 21:14:58.798632 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 197, in wrapper Jul 16 21:14:58.798632 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Jul 16 21:14:58.798632 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 333, in retry_request Jul 16 21:14:58.798632 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return self.do_request(method, action, body=body, Jul 16 21:14:58.798632 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 197, in wrapper Jul 16 21:14:58.798632 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Jul 16 21:14:58.798632 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 297, in do_request Jul 16 21:14:58.800395 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi self._handle_fault_response(status_code, replybody, resp) Jul 16 21:14:58.800395 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 197, in wrapper Jul 16 21:14:58.800395 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Jul 16 21:14:58.800395 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 272, in _handle_fault_response Jul 16 21:14:58.800395 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi exception_handler_v20(status_code, error_body) Jul 16 21:14:58.800395 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 90, in exception_handler_v20 Jul 16 21:14:58.800395 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise client_exc(message=error_message, Jul 16 21:14:58.800395 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi neutronclient.common.exceptions.NotFound: Floating IP ce1d33e7-8d17-4e88-bbec-ce1513cf93e9 could not be found Jul 16 21:14:58.800395 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-e79383be-8ef4-4625-883c-bd1b12ed70d1'] Jul 16 21:14:58.800395 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:14:58.800395 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:14:58.800395 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:14:58.800395 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:14:58.800395 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/floating_ips.py", line 94, in show Jul 16 21:14:58.800395 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi floating_ip = self.network_api.get_floating_ip(context, id) Jul 16 21:14:58.800395 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 2830, in get_floating_ip Jul 16 21:14:58.800395 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.FloatingIpNotFound(id=id) Jul 16 21:14:58.800395 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.FloatingIpNotFound: Floating IP not found for ID ce1d33e7-8d17-4e88-bbec-ce1513cf93e9. Jul 16 21:14:58.800395 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:14:58.802229 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:14:58.802229 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:14:58.802229 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:14:58.802229 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:14:58.802229 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:14:58.802229 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:14:58.802229 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:14:58.802229 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Jul 16 21:14:58.802229 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Jul 16 21:14:58.802229 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:14:58.802229 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:14:58.802229 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/floating_ips.py", line 97, in show Jul 16 21:14:58.802229 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=msg) Jul 16 21:14:58.802229 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Floating IP not found for ID ce1d33e7-8d17-4e88-bbec-ce1513cf93e9 Jul 16 21:14:58.802229 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:14:58.802229 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:14:58.802229 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:14:58.802229 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:14:58.802229 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:14:58.802229 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:14:58.802229 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:14:58.807728 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:14:58.807728 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Floating IP not found for ID ce1d33e7-8d17-4e88-bbec-ce1513cf93e9 Jul 16 21:14:58.807728 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:14:58.807728 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-9c69c989-1cf4-46af-98b3-af3ed529908b tempest-FloatingIPDetailsNegativeTestJSON-580292985 tempest-FloatingIPDetailsNegativeTestJSON-580292985-project] Returning 404 to user: Floating IP not found for ID ce1d33e7-8d17-4e88-bbec-ce1513cf93e9 {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:14:58.811400 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-9c69c989-1cf4-46af-98b3-af3ed529908b tempest-FloatingIPDetailsNegativeTestJSON-580292985 tempest-FloatingIPDetailsNegativeTestJSON-580292985-project] 10.222.0.16 "GET /compute/v2.1/os-floating-ips/ce1d33e7-8d17-4e88-bbec-ce1513cf93e9" status: 404 len: 111 microversion: 2.1 time: 0.544529 Jul 16 21:14:58.812853 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 791/1627] 10.222.0.16 () {58 vars in 1259 bytes} [Sat Jul 16 21:14:58 2022] GET /compute/v2.1/os-floating-ips/ce1d33e7-8d17-4e88-bbec-ce1513cf93e9 => generated 111 bytes in 548 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:14:58.906157 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-8f25d619-09e1-4071-8910-4b32863bbd42 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-LiveMigrationTest-server-7144524", "imageRef": "", "flavorRef": "452", "networks": [{"uuid": "26ad28eb-6622-4b58-ad05-2de54c33be8d"}], "block_device_mapping_v2": [{"uuid": "eb5651c9-06af-46c0-905c-8ecec06e7945", "source_type": "volume", "destination_type": "volume", "boot_index": 0, "delete_on_termination": true}]}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:14:59.104943 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.network.neutron [None req-8f25d619-09e1-4071-8910-4b32863bbd42 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] validate_networks() for [('26ad28eb-6622-4b58-ad05-2de54c33be8d', None, None, None, None, None)] {{(pid=138681) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 16 21:14:59.461523 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-08e5c783-a4b3-4883-bde7-4e8bf02d05fa tempest-ImagesMetadataTestJSON-1264454256 tempest-ImagesMetadataTestJSON-1264454256-project] 10.222.0.16 "PUT /compute/v2.1/images/9ab4ec44-a79d-4f52-93c5-c44ab4e3a79f/metadata" status: 200 len: 61 microversion: 2.1 time: 0.907206 Jul 16 21:14:59.466125 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 837/1628] 10.222.0.16 () {60 vars in 1279 bytes} [Sat Jul 16 21:14:58 2022] PUT /compute/v2.1/images/9ab4ec44-a79d-4f52-93c5-c44ab4e3a79f/metadata => generated 61 bytes in 910 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Jul 16 21:14:59.468692 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-8ece112b-56ee-4f77-bd72-e063425b5d7f tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ImagesOneServerNegativeTestJSON-server-1465186379", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "}]}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:14:59.532614 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.network.neutron [None req-8ece112b-56ee-4f77-bd72-e063425b5d7f tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] validate_networks() for [('45344261-e9f3-4a7b-95e1-5380175d73fa', None, None, None, None, None)] {{(pid=138680) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 16 21:14:59.850616 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-8f25d619-09e1-4071-8910-4b32863bbd42 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Flavor limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 16 21:14:59.851904 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-8f25d619-09e1-4071-8910-4b32863bbd42 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Image limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 16 21:14:59.852547 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-8f25d619-09e1-4071-8910-4b32863bbd42 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Flavor pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 16 21:14:59.853234 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-8f25d619-09e1-4071-8910-4b32863bbd42 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Image pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 16 21:14:59.854062 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-8f25d619-09e1-4071-8910-4b32863bbd42 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 16 21:14:59.875334 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8f25d619-09e1-4071-8910-4b32863bbd42 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:59.875802 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8f25d619-09e1-4071-8910-4b32863bbd42 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:59.876652 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8f25d619-09e1-4071-8910-4b32863bbd42 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:14:59.877096 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8f25d619-09e1-4071-8910-4b32863bbd42 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:14:59.916325 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-8f25d619-09e1-4071-8910-4b32863bbd42 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Getting quotas for project 9b769bf8f3ea4e368bf68de1df6538f9. Resources: {'ram', 'instances', 'cores'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:14:59.922458 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-8f25d619-09e1-4071-8910-4b32863bbd42 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Getting quotas for user 6a69f9dee03d43beaafa966216b9e289 and project 9b769bf8f3ea4e368bf68de1df6538f9. Resources: {'ram', 'instances', 'cores'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:15:00.024579 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-8f25d619-09e1-4071-8910-4b32863bbd42 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Going to run 1 instances... {{(pid=138681) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 16 21:15:00.026686 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-8f25d619-09e1-4071-8910-4b32863bbd42 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] [instance: 23df6e0a-3637-4f59-93f9-b02dd614b5c3] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag=None,updated_at=,uuid=,volume_id='eb5651c9-06af-46c0-905c-8ecec06e7945',volume_size=None,volume_type=None)] {{(pid=138681) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 16 21:15:00.217353 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-8ece112b-56ee-4f77-bd72-e063425b5d7f tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Flavor limits 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 16 21:15:00.217936 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-8ece112b-56ee-4f77-bd72-e063425b5d7f tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Image limits 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 16 21:15:00.218381 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-8ece112b-56ee-4f77-bd72-e063425b5d7f tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Flavor pref 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 16 21:15:00.218803 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-8ece112b-56ee-4f77-bd72-e063425b5d7f tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Image pref 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 16 21:15:00.219253 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-8ece112b-56ee-4f77-bd72-e063425b5d7f tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 16 21:15:00.230234 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8ece112b-56ee-4f77-bd72-e063425b5d7f tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:00.230958 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8ece112b-56ee-4f77-bd72-e063425b5d7f tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:00.232010 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8ece112b-56ee-4f77-bd72-e063425b5d7f tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:00.232634 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8ece112b-56ee-4f77-bd72-e063425b5d7f tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:00.259254 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-8ece112b-56ee-4f77-bd72-e063425b5d7f tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Getting quotas for project 4337575622d04a26942905c9dc044ba7. Resources: {'ram', 'instances', 'cores'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:15:00.270839 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-8ece112b-56ee-4f77-bd72-e063425b5d7f tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Getting quotas for user 4e9df82f95ac4779859972403869096a and project 4337575622d04a26942905c9dc044ba7. Resources: {'ram', 'instances', 'cores'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:15:00.292183 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-8ece112b-56ee-4f77-bd72-e063425b5d7f tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Going to run 1 instances... {{(pid=138680) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 16 21:15:00.295317 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-8ece112b-56ee-4f77-bd72-e063425b5d7f tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] [instance: 39e51d01-21e0-4dd5-bfd4-16b0f8b0f827] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138680) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 16 21:15:00.515415 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-8ece112b-56ee-4f77-bd72-e063425b5d7f tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 383 microversion: 2.1 time: 1.051507 Jul 16 21:15:00.522749 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 838/1629] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 21:14:59 2022] POST /compute/v2.1/servers => generated 383 bytes in 1060 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 16 21:15:00.707060 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-8f25d619-09e1-4071-8910-4b32863bbd42 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 1.805818 Jul 16 21:15:00.713093 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 792/1630] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 21:14:58 2022] POST /compute/v2.1/servers => generated 380 bytes in 1813 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Jul 16 21:15:00.717974 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-2f955c9a-e736-48ea-b842-88d738b471c4 tempest-ImagesMetadataTestJSON-1264454256 tempest-ImagesMetadataTestJSON-1264454256-project] Calling method '.version_select at 0x7fdd9cf2a790>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:15:00.863034 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-2f955c9a-e736-48ea-b842-88d738b471c4 tempest-ImagesMetadataTestJSON-1264454256 tempest-ImagesMetadataTestJSON-1264454256-project] 10.222.0.16 "GET /compute/v2.1/images/9ab4ec44-a79d-4f52-93c5-c44ab4e3a79f/metadata/os_distro" status: 200 len: 33 microversion: 2.1 time: 0.145820 Jul 16 21:15:00.872949 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 793/1631] 10.222.0.16 () {58 vars in 1289 bytes} [Sat Jul 16 21:15:00 2022] GET /compute/v2.1/images/9ab4ec44-a79d-4f52-93c5-c44ab4e3a79f/metadata/os_distro => generated 33 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Jul 16 21:15:00.881696 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-a4c2d7b9-9f27-4564-b664-1133f762af46 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServersTestMultiNic-server-1592282188", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452", "networks": [{"uuid": "f5d9ab31-7c1d-4411-adf3-8fdb4647b24c"}, {"uuid": "6ad9b08b-b075-4501-86f6-4142c2070904"}]}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:15:01.019622 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-b4742a5a-8415-4eb1-b951-7c7c37e0f050 tempest-FlavorsV2NegativeTest-1032641942 tempest-FlavorsV2NegativeTest-1032641942-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:15:01.036611 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-b4742a5a-8415-4eb1-b951-7c7c37e0f050 tempest-FlavorsV2NegativeTest-1032641942 tempest-FlavorsV2NegativeTest-1032641942-project] 10.222.0.16 "GET /compute/v2.1/flavors/452" status: 200 len: 370 microversion: 2.1 time: 0.512268 Jul 16 21:15:01.037787 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 839/1632] 10.222.0.16 () {58 vars in 1136 bytes} [Sat Jul 16 21:15:00 2022] GET /compute/v2.1/flavors/452 => generated 370 bytes in 514 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:15:01.045094 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-63fe4a58-a7af-4aa1-8131-5aaef313cf4c tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:15:01.046383 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-63fe4a58-a7af-4aa1-8131-5aaef313cf4c tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] [instance: 39e51d01-21e0-4dd5-bfd4-16b0f8b0f827] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:15:01.132144 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-63fe4a58-a7af-4aa1-8131-5aaef313cf4c tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4e9df82f95ac4779859972403869096a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4337575622d04a26942905c9dc044ba7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:15:01.138538 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.network.neutron [None req-a4c2d7b9-9f27-4564-b664-1133f762af46 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] validate_networks() for [('f5d9ab31-7c1d-4411-adf3-8fdb4647b24c', None, None, None, None, None), ('6ad9b08b-b075-4501-86f6-4142c2070904', None, None, None, None, None)] {{(pid=138681) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 16 21:15:01.143504 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-63fe4a58-a7af-4aa1-8131-5aaef313cf4c tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] 10.222.0.16 "GET /compute/v2.1/servers/39e51d01-21e0-4dd5-bfd4-16b0f8b0f827" status: 200 len: 1225 microversion: 2.1 time: 0.104178 Jul 16 21:15:01.146797 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 840/1633] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:15:01 2022] GET /compute/v2.1/servers/39e51d01-21e0-4dd5-bfd4-16b0f8b0f827 => generated 1225 bytes in 108 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:15:01.153806 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-e6b1703f-5514-48ff-a57d-11c83080d0c6 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:15:01.163906 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-e6b1703f-5514-48ff-a57d-11c83080d0c6 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] [instance: 23df6e0a-3637-4f59-93f9-b02dd614b5c3] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:15:01.502306 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-e6b1703f-5514-48ff-a57d-11c83080d0c6 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a69f9dee03d43beaafa966216b9e289', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9b769bf8f3ea4e368bf68de1df6538f9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:15:01.512247 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-e6b1703f-5514-48ff-a57d-11c83080d0c6 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] 10.222.0.16 "GET /compute/v2.1/servers/23df6e0a-3637-4f59-93f9-b02dd614b5c3" status: 200 len: 1050 microversion: 2.1 time: 0.362202 Jul 16 21:15:01.531367 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 841/1634] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:15:01 2022] GET /compute/v2.1/servers/23df6e0a-3637-4f59-93f9-b02dd614b5c3 => generated 1050 bytes in 382 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:15:01.550498 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-a33271d9-1499-4e41-90c9-2c851afbce28 tempest-ImagesMetadataTestJSON-1264454256 tempest-ImagesMetadataTestJSON-1264454256-project] Action: 'update_all', calling method: .version_select at 0x7fdd9cd915e0>, body: {"metadata": {"os_version": "value1", "os_distro": "value2"}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:15:01.695414 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-a33271d9-1499-4e41-90c9-2c851afbce28 tempest-ImagesMetadataTestJSON-1264454256 tempest-ImagesMetadataTestJSON-1264454256-project] Getting quotas for project a30b6074cef24df6ba020b01fc85b3e3. Resources: dict_keys(['metadata_items']) {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:15:01.701120 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-a33271d9-1499-4e41-90c9-2c851afbce28 tempest-ImagesMetadataTestJSON-1264454256 tempest-ImagesMetadataTestJSON-1264454256-project] Getting quotas for user bd9e72b27f304a9383d7f3fdcd718972 and project a30b6074cef24df6ba020b01fc85b3e3. Resources: dict_keys(['metadata_items']) {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:15:02.121325 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-a33271d9-1499-4e41-90c9-2c851afbce28 tempest-ImagesMetadataTestJSON-1264454256 tempest-ImagesMetadataTestJSON-1264454256-project] 10.222.0.16 "PUT /compute/v2.1/images/9ab4ec44-a79d-4f52-93c5-c44ab4e3a79f/metadata" status: 200 len: 61 microversion: 2.1 time: 0.588376 Jul 16 21:15:02.124547 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 842/1635] 10.222.0.16 () {60 vars in 1279 bytes} [Sat Jul 16 21:15:01 2022] PUT /compute/v2.1/images/9ab4ec44-a79d-4f52-93c5-c44ab4e3a79f/metadata => generated 61 bytes in 590 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Jul 16 21:15:02.132516 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-a667cb67-1f13-479d-be52-4f9b0da02cd1 tempest-ImagesMetadataTestJSON-1264454256 tempest-ImagesMetadataTestJSON-1264454256-project] Calling method '.version_select at 0x7fdd9cbe1a60>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:15:02.184527 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-a667cb67-1f13-479d-be52-4f9b0da02cd1 tempest-ImagesMetadataTestJSON-1264454256 tempest-ImagesMetadataTestJSON-1264454256-project] 10.222.0.16 "GET /compute/v2.1/images/9ab4ec44-a79d-4f52-93c5-c44ab4e3a79f/metadata" status: 200 len: 61 microversion: 2.1 time: 0.056590 Jul 16 21:15:02.193336 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 843/1636] 10.222.0.16 () {58 vars in 1259 bytes} [Sat Jul 16 21:15:02 2022] GET /compute/v2.1/images/9ab4ec44-a79d-4f52-93c5-c44ab4e3a79f/metadata => generated 61 bytes in 66 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Jul 16 21:15:02.199659 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-4c28d85d-1033-4324-8dd2-da6ddc9c41e8 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:15:02.200496 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-4c28d85d-1033-4324-8dd2-da6ddc9c41e8 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] [instance: 39e51d01-21e0-4dd5-bfd4-16b0f8b0f827] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:15:02.207992 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4c28d85d-1033-4324-8dd2-da6ddc9c41e8 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:02.208707 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4c28d85d-1033-4324-8dd2-da6ddc9c41e8 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:02.210883 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4c28d85d-1033-4324-8dd2-da6ddc9c41e8 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:02.210883 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4c28d85d-1033-4324-8dd2-da6ddc9c41e8 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:02.330868 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-4c28d85d-1033-4324-8dd2-da6ddc9c41e8 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4e9df82f95ac4779859972403869096a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4337575622d04a26942905c9dc044ba7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:15:02.347587 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-4c28d85d-1033-4324-8dd2-da6ddc9c41e8 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] 10.222.0.16 "GET /compute/v2.1/servers/39e51d01-21e0-4dd5-bfd4-16b0f8b0f827" status: 200 len: 1229 microversion: 2.1 time: 0.151119 Jul 16 21:15:02.352295 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 844/1637] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:15:02 2022] GET /compute/v2.1/servers/39e51d01-21e0-4dd5-bfd4-16b0f8b0f827 => generated 1229 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:15:02.358289 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-5346b8f7-0038-4a6c-bc92-21bbd3aa01ba tempest-ImagesMetadataTestJSON-1264454256 tempest-ImagesMetadataTestJSON-1264454256-project] Action: 'update_all', calling method: .version_select at 0x7fdd9cc3fa60>, body: {"metadata": {"os_version": "value1", "os_distro": "value2"}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:15:02.446623 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-5346b8f7-0038-4a6c-bc92-21bbd3aa01ba tempest-ImagesMetadataTestJSON-1264454256 tempest-ImagesMetadataTestJSON-1264454256-project] Getting quotas for project a30b6074cef24df6ba020b01fc85b3e3. Resources: dict_keys(['metadata_items']) {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:15:02.453033 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-5346b8f7-0038-4a6c-bc92-21bbd3aa01ba tempest-ImagesMetadataTestJSON-1264454256 tempest-ImagesMetadataTestJSON-1264454256-project] Getting quotas for user bd9e72b27f304a9383d7f3fdcd718972 and project a30b6074cef24df6ba020b01fc85b3e3. Resources: dict_keys(['metadata_items']) {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:15:02.548204 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-a4c2d7b9-9f27-4564-b664-1133f762af46 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Flavor limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 16 21:15:02.548743 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-a4c2d7b9-9f27-4564-b664-1133f762af46 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Image limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 16 21:15:02.549330 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-a4c2d7b9-9f27-4564-b664-1133f762af46 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Flavor pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 16 21:15:02.549845 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-a4c2d7b9-9f27-4564-b664-1133f762af46 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Image pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 16 21:15:02.550365 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-a4c2d7b9-9f27-4564-b664-1133f762af46 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 16 21:15:02.554185 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_api:SELECT=95,nova_api:UPDATE=1,nova_api:INSERT=4 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:15:02.594348 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a4c2d7b9-9f27-4564-b664-1133f762af46 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:02.596802 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a4c2d7b9-9f27-4564-b664-1133f762af46 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:02.597900 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a4c2d7b9-9f27-4564-b664-1133f762af46 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:02.598562 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a4c2d7b9-9f27-4564-b664-1133f762af46 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:02.620063 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-a4c2d7b9-9f27-4564-b664-1133f762af46 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Getting quotas for project bb1ede0d0b6941cc97278e6c617f9b01. Resources: {'ram', 'instances', 'cores'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:15:02.625767 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-a4c2d7b9-9f27-4564-b664-1133f762af46 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Getting quotas for user 53af488f3b0242a797f539284b78d59e and project bb1ede0d0b6941cc97278e6c617f9b01. Resources: {'ram', 'instances', 'cores'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:15:02.647614 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-a4c2d7b9-9f27-4564-b664-1133f762af46 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Going to run 1 instances... {{(pid=138681) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 16 21:15:02.654729 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-a4c2d7b9-9f27-4564-b664-1133f762af46 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] [instance: 0079e567-30df-4cac-a0ad-4379bcbbcf48] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138681) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 16 21:15:02.822054 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-a4c2d7b9-9f27-4564-b664-1133f762af46 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 1.943551 Jul 16 21:15:02.824705 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 794/1638] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 21:15:00 2022] POST /compute/v2.1/servers => generated 380 bytes in 1952 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 16 21:15:02.841980 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-b99a5c48-c814-4367-af38-00447e84db87 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:15:02.845220 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-b99a5c48-c814-4367-af38-00447e84db87 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] [instance: 23df6e0a-3637-4f59-93f9-b02dd614b5c3] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:15:02.868568 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b99a5c48-c814-4367-af38-00447e84db87 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:02.871676 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b99a5c48-c814-4367-af38-00447e84db87 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:02.878560 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b99a5c48-c814-4367-af38-00447e84db87 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:02.882329 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b99a5c48-c814-4367-af38-00447e84db87 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:02.983490 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-5346b8f7-0038-4a6c-bc92-21bbd3aa01ba tempest-ImagesMetadataTestJSON-1264454256 tempest-ImagesMetadataTestJSON-1264454256-project] 10.222.0.16 "PUT /compute/v2.1/images/9ab4ec44-a79d-4f52-93c5-c44ab4e3a79f/metadata" status: 200 len: 61 microversion: 2.1 time: 0.631037 Jul 16 21:15:02.984835 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 845/1639] 10.222.0.16 () {60 vars in 1279 bytes} [Sat Jul 16 21:15:02 2022] PUT /compute/v2.1/images/9ab4ec44-a79d-4f52-93c5-c44ab4e3a79f/metadata => generated 61 bytes in 633 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Jul 16 21:15:02.998936 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-97ae323a-a75d-4c72-879b-f422284d37de tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:15:03.001051 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-97ae323a-a75d-4c72-879b-f422284d37de tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] [instance: 0079e567-30df-4cac-a0ad-4379bcbbcf48] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:15:03.118250 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-b99a5c48-c814-4367-af38-00447e84db87 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a69f9dee03d43beaafa966216b9e289', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9b769bf8f3ea4e368bf68de1df6538f9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:15:03.136619 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-b99a5c48-c814-4367-af38-00447e84db87 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] 10.222.0.16 "GET /compute/v2.1/servers/23df6e0a-3637-4f59-93f9-b02dd614b5c3" status: 200 len: 1100 microversion: 2.1 time: 0.308598 Jul 16 21:15:03.142331 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 795/1640] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:15:02 2022] GET /compute/v2.1/servers/23df6e0a-3637-4f59-93f9-b02dd614b5c3 => generated 1100 bytes in 315 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:15:03.154870 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-2be1c962-1055-4e90-b3ab-f2952b1092be tempest-ImagesMetadataTestJSON-1264454256 tempest-ImagesMetadataTestJSON-1264454256-project] Action: 'update_all', calling method: .version_select at 0x7fdd9cf2af70>, body: {"metadata": {"os_version": "value2", "architecture": "value3"}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:15:03.192459 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-97ae323a-a75d-4c72-879b-f422284d37de tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '53af488f3b0242a797f539284b78d59e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bb1ede0d0b6941cc97278e6c617f9b01', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:15:03.202100 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-97ae323a-a75d-4c72-879b-f422284d37de tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] 10.222.0.16 "GET /compute/v2.1/servers/0079e567-30df-4cac-a0ad-4379bcbbcf48" status: 200 len: 1213 microversion: 2.1 time: 0.206028 Jul 16 21:15:03.202729 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 846/1641] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:15:02 2022] GET /compute/v2.1/servers/0079e567-30df-4cac-a0ad-4379bcbbcf48 => generated 1213 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:15:03.252644 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-d9673510-f870-4ce5-b805-a61fa4563efe tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-LiveAutoBlockMigrationV225Test-server-1064412383", "imageRef": "", "flavorRef": "452", "networks": [{"uuid": "b3fd4d7b-96dc-4cc4-8a4c-4e1986f1bc15"}], "block_device_mapping_v2": [{"uuid": "b73aa13f-ef6f-4a53-a7a2-c8ef03eab5ab", "source_type": "volume", "destination_type": "volume", "boot_index": 0, "delete_on_termination": true}]}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:15:03.296118 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_api:SELECT=95,nova_api:UPDATE=2,nova_api:INSERT=3 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:15:03.307382 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-2be1c962-1055-4e90-b3ab-f2952b1092be tempest-ImagesMetadataTestJSON-1264454256 tempest-ImagesMetadataTestJSON-1264454256-project] Getting quotas for project a30b6074cef24df6ba020b01fc85b3e3. Resources: dict_keys(['metadata_items']) {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:15:03.328625 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-2be1c962-1055-4e90-b3ab-f2952b1092be tempest-ImagesMetadataTestJSON-1264454256 tempest-ImagesMetadataTestJSON-1264454256-project] Getting quotas for user bd9e72b27f304a9383d7f3fdcd718972 and project a30b6074cef24df6ba020b01fc85b3e3. Resources: dict_keys(['metadata_items']) {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:15:03.392418 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.network.neutron [None req-d9673510-f870-4ce5-b805-a61fa4563efe tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] validate_networks() for [('b3fd4d7b-96dc-4cc4-8a4c-4e1986f1bc15', None, None, None, None, None)] {{(pid=138680) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 16 21:15:04.001636 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-2be1c962-1055-4e90-b3ab-f2952b1092be tempest-ImagesMetadataTestJSON-1264454256 tempest-ImagesMetadataTestJSON-1264454256-project] 10.222.0.16 "PUT /compute/v2.1/images/9ab4ec44-a79d-4f52-93c5-c44ab4e3a79f/metadata" status: 200 len: 64 microversion: 2.1 time: 0.855190 Jul 16 21:15:04.001636 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 796/1642] 10.222.0.16 () {60 vars in 1279 bytes} [Sat Jul 16 21:15:03 2022] PUT /compute/v2.1/images/9ab4ec44-a79d-4f52-93c5-c44ab4e3a79f/metadata => generated 64 bytes in 857 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Jul 16 21:15:04.005541 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-9b23a4e0-6ebb-40ef-b15f-300842cc6c71 tempest-FlavorsV2NegativeTest-1032641942 tempest-FlavorsV2NegativeTest-1032641942-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-FlavorsV2NegativeTest-server-1721974142", "imageRef": "e997d011-f3ad-4544-b02f-f0a8e5971665", "flavorRef": "452"}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:15:04.147505 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.network.neutron [None req-9b23a4e0-6ebb-40ef-b15f-300842cc6c71 tempest-FlavorsV2NegativeTest-1032641942 tempest-FlavorsV2NegativeTest-1032641942-project] validate_networks() for None {{(pid=138681) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 16 21:15:04.150788 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-d9673510-f870-4ce5-b805-a61fa4563efe tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Flavor limits 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 16 21:15:04.151347 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-d9673510-f870-4ce5-b805-a61fa4563efe tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Image limits 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 16 21:15:04.151839 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-d9673510-f870-4ce5-b805-a61fa4563efe tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Flavor pref 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 16 21:15:04.152944 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-d9673510-f870-4ce5-b805-a61fa4563efe tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Image pref 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 16 21:15:04.153378 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-d9673510-f870-4ce5-b805-a61fa4563efe tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 16 21:15:04.174173 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d9673510-f870-4ce5-b805-a61fa4563efe tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:04.175324 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d9673510-f870-4ce5-b805-a61fa4563efe tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:04.176998 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d9673510-f870-4ce5-b805-a61fa4563efe tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:04.177466 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d9673510-f870-4ce5-b805-a61fa4563efe tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:04.209697 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=96,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:15:04.243523 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-d9673510-f870-4ce5-b805-a61fa4563efe tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Getting quotas for project d9b92fc544c544f686ac2ab7ff5123b6. Resources: {'ram', 'instances', 'cores'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:15:04.250334 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-d9673510-f870-4ce5-b805-a61fa4563efe tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Getting quotas for user 578b87f863b34f57a571f7e56d44f118 and project d9b92fc544c544f686ac2ab7ff5123b6. Resources: {'ram', 'instances', 'cores'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:15:04.373940 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-d9673510-f870-4ce5-b805-a61fa4563efe tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Going to run 1 instances... {{(pid=138680) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 16 21:15:04.378888 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-d9673510-f870-4ce5-b805-a61fa4563efe tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] [instance: 84ab7e65-1bf0-4b8b-baea-6d1d4ca13dd7] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag=None,updated_at=,uuid=,volume_id='b73aa13f-ef6f-4a53-a7a2-c8ef03eab5ab',volume_size=None,volume_type=None)] {{(pid=138680) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 16 21:15:05.147847 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-d9673510-f870-4ce5-b805-a61fa4563efe tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.25 time: 1.921320 Jul 16 21:15:05.149474 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 847/1643] 10.222.0.16 () {62 vars in 1187 bytes} [Sat Jul 16 21:15:03 2022] POST /compute/v2.1/servers => generated 380 bytes in 1946 msecs (HTTP/1.1 202) 10 headers in 453 bytes (1 switches on core 0) Jul 16 21:15:05.163781 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-f30a39c7-8bed-40dd-8459-207fa8593c1e tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:15:05.166404 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-f30a39c7-8bed-40dd-8459-207fa8593c1e tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] [instance: 39e51d01-21e0-4dd5-bfd4-16b0f8b0f827] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:15:05.177294 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f30a39c7-8bed-40dd-8459-207fa8593c1e tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:05.178152 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f30a39c7-8bed-40dd-8459-207fa8593c1e tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:05.179895 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f30a39c7-8bed-40dd-8459-207fa8593c1e tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:05.180619 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f30a39c7-8bed-40dd-8459-207fa8593c1e tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:05.192464 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-9b23a4e0-6ebb-40ef-b15f-300842cc6c71 tempest-FlavorsV2NegativeTest-1032641942 tempest-FlavorsV2NegativeTest-1032641942-project] HTTP exception thrown: Flavor's memory is too small for requested image. Jul 16 21:15:05.195056 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-9b23a4e0-6ebb-40ef-b15f-300842cc6c71 tempest-FlavorsV2NegativeTest-1032641942 tempest-FlavorsV2NegativeTest-1032641942-project] Request method failure captured: Jul 16 21:15:05.195056 ncdv-master-1 devstack@n-api.service[138681]: request: POST /compute/v2.1/servers HTTP/1.1 Jul 16 21:15:05.195056 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:15:05.195056 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:15:05.195056 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:15:05.195056 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 143 Jul 16 21:15:05.195056 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:15:05.195056 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:15:05.195056 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:15:05.195056 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:15:05.195056 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0ypSRWTzK86SBaB3qm6aNRdec8B8xGkFmv6C4qO_YzaXMroExJW517BhjxNzgl1iKYjTfnFwUoxRTwc1NF24rPAEDi3c66CQxXMeGFTWx-McPI_Xpd6ijHg7bw0PZ1X7dMHdyZGMUULF_Z4t6XU2_kJ5yq9YlC3ijjVWiLY026eKAik Jul 16 21:15:05.195056 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:15:05.195056 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:15:05.195056 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:15:05.195056 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:15:05.195056 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:15:05.195056 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:15:05.195056 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: bf3b90d53acb44479975ab2f5217d5a6 Jul 16 21:15:05.195056 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-FlavorsV2NegativeTest-1032641942 Jul 16 21:15:05.195056 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:15:05.195056 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:15:05.196653 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/bf3b90d53acb44479975ab2f5217d5a6"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/bf3b90d53acb44479975ab2f5217d5a6"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_bf3b90d53acb44479975ab2f5217d5a6"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/bf3b90d53acb44479975ab2f5217d5a6"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:15:05.196653 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-FlavorsV2NegativeTest-1032641942 Jul 16 21:15:05.196653 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: bf3b90d53acb44479975ab2f5217d5a6 Jul 16 21:15:05.196653 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-FlavorsV2NegativeTest-1032641942 Jul 16 21:15:05.196653 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-FlavorsV2NegativeTest-1032641942-project Jul 16 21:15:05.196653 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:15:05.196653 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:15:05.196653 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: 730f67eb5f1f41e9ab9da831340f4f49 Jul 16 21:15:05.196653 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-FlavorsV2NegativeTest-1032641942-project Jul 16 21:15:05.196653 ncdv-master-1 devstack@n-api.service[138681]: Jul 16 21:15:05.196653 ncdv-master-1 devstack@n-api.service[138681]: {"server": {"name": "tempest-FlavorsV2NegativeTest-server-1721974142", "imageRef": "e997d011-f3ad-4544-b02f-f0a8e5971665", "flavorRef": "452"}} Jul 16 21:15:05.196653 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:15:05.197484 ncdv-master-1 devstack@n-api.service[138681]: exception: Flavor's memory is too small for requested image. Jul 16 21:15:05.197484 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'body': {'server': {'name': 'tempest-FlavorsV2NegativeTest-server-1721974142', 'imageRef': 'e997d011-f3ad-4544-b02f-f0a8e5971665', 'flavorRef': '452'}}} Jul 16 21:15:05.197484 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:15:05.197484 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:15:05.197484 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 784, in create Jul 16 21:15:05.197484 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instances, resv_id = self.compute_api.create( Jul 16 21:15:05.197484 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2149, in create Jul 16 21:15:05.197484 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return self._create_instance( Jul 16 21:15:05.197484 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 1667, in _create_instance Jul 16 21:15:05.197484 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi self._checks_for_create_and_rebuild(context, image_id, boot_meta, Jul 16 21:15:05.197484 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 1005, in _checks_for_create_and_rebuild Jul 16 21:15:05.197484 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi self._validate_flavor_image(context, image_id, image, Jul 16 21:15:05.197484 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 669, in _validate_flavor_image Jul 16 21:15:05.197484 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi self._validate_flavor_image_nostatus( Jul 16 21:15:05.197484 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 737, in _validate_flavor_image_nostatus Jul 16 21:15:05.197484 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.FlavorMemoryTooSmall() Jul 16 21:15:05.197484 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.FlavorMemoryTooSmall: Flavor's memory is too small for requested image. Jul 16 21:15:05.197484 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:15:05.197484 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:15:05.197484 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:15:05.198859 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:15:05.198859 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:15:05.198859 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:15:05.198859 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:15:05.198859 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:15:05.198859 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:15:05.198859 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:15:05.198859 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 16 21:15:05.198859 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 16 21:15:05.198859 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 16 21:15:05.198859 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 16 21:15:05.198859 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 16 21:15:05.198859 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 16 21:15:05.198859 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi [Previous line repeated 10 more times] Jul 16 21:15:05.198859 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 868, in create Jul 16 21:15:05.198859 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=error.format_message()) Jul 16 21:15:05.198859 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Flavor's memory is too small for requested image. Jul 16 21:15:05.198859 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:15:05.198859 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:15:05.198859 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:15:05.198859 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:15:05.203689 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:15:05.203689 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:15:05.203689 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:15:05.203689 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:15:05.203689 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor's memory is too small for requested image. Jul 16 21:15:05.203689 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:15:05.203689 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-9b23a4e0-6ebb-40ef-b15f-300842cc6c71 tempest-FlavorsV2NegativeTest-1032641942 tempest-FlavorsV2NegativeTest-1032641942-project] Returning 400 to user: Flavor's memory is too small for requested image. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:15:05.203689 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-9b23a4e0-6ebb-40ef-b15f-300842cc6c71 tempest-FlavorsV2NegativeTest-1032641942 tempest-FlavorsV2NegativeTest-1032641942-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 400 len: 93 microversion: 2.1 time: 1.196304 Jul 16 21:15:05.210647 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 797/1644] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 21:15:03 2022] POST /compute/v2.1/servers => generated 93 bytes in 1211 msecs (HTTP/1.1 400) 9 headers in 380 bytes (1 switches on core 0) Jul 16 21:15:05.231770 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-df529948-b549-4124-b078-f9403ec2de1e tempest-ImagesMetadataTestJSON-1264454256 tempest-ImagesMetadataTestJSON-1264454256-project] Calling method '.version_select at 0x7fdd9cdb58b0>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:15:05.328233 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-df529948-b549-4124-b078-f9403ec2de1e tempest-ImagesMetadataTestJSON-1264454256 tempest-ImagesMetadataTestJSON-1264454256-project] 10.222.0.16 "GET /compute/v2.1/images/9ab4ec44-a79d-4f52-93c5-c44ab4e3a79f/metadata" status: 200 len: 64 microversion: 2.1 time: 0.116201 Jul 16 21:15:05.335190 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 798/1645] 10.222.0.16 () {58 vars in 1259 bytes} [Sat Jul 16 21:15:05 2022] GET /compute/v2.1/images/9ab4ec44-a79d-4f52-93c5-c44ab4e3a79f/metadata => generated 64 bytes in 124 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Jul 16 21:15:05.347061 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-4a08ad44-d14b-48da-9baa-2267b0e5fa5c tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:15:05.347784 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-4a08ad44-d14b-48da-9baa-2267b0e5fa5c tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] [instance: 23df6e0a-3637-4f59-93f9-b02dd614b5c3] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:15:05.370407 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4a08ad44-d14b-48da-9baa-2267b0e5fa5c tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:05.370918 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4a08ad44-d14b-48da-9baa-2267b0e5fa5c tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:05.371780 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4a08ad44-d14b-48da-9baa-2267b0e5fa5c tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:05.372235 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4a08ad44-d14b-48da-9baa-2267b0e5fa5c tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:05.450328 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-f30a39c7-8bed-40dd-8459-207fa8593c1e tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4e9df82f95ac4779859972403869096a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4337575622d04a26942905c9dc044ba7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:15:05.468185 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-f30a39c7-8bed-40dd-8459-207fa8593c1e tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] 10.222.0.16 "GET /compute/v2.1/servers/39e51d01-21e0-4dd5-bfd4-16b0f8b0f827" status: 200 len: 1283 microversion: 2.1 time: 0.316008 Jul 16 21:15:05.470028 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 848/1646] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:15:05 2022] GET /compute/v2.1/servers/39e51d01-21e0-4dd5-bfd4-16b0f8b0f827 => generated 1283 bytes in 318 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:15:05.486527 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-ff031424-7032-4e07-9aed-0eb74918058a tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:15:05.487812 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-ff031424-7032-4e07-9aed-0eb74918058a tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] [instance: 0079e567-30df-4cac-a0ad-4379bcbbcf48] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:15:05.528664 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ff031424-7032-4e07-9aed-0eb74918058a tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:05.528664 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ff031424-7032-4e07-9aed-0eb74918058a tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:05.534163 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ff031424-7032-4e07-9aed-0eb74918058a tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:05.535426 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ff031424-7032-4e07-9aed-0eb74918058a tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:05.626637 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-4a08ad44-d14b-48da-9baa-2267b0e5fa5c tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a69f9dee03d43beaafa966216b9e289', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9b769bf8f3ea4e368bf68de1df6538f9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:15:05.637856 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-4a08ad44-d14b-48da-9baa-2267b0e5fa5c tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] 10.222.0.16 "GET /compute/v2.1/servers/23df6e0a-3637-4f59-93f9-b02dd614b5c3" status: 200 len: 1166 microversion: 2.1 time: 0.301464 Jul 16 21:15:05.638453 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 799/1647] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:15:05 2022] GET /compute/v2.1/servers/23df6e0a-3637-4f59-93f9-b02dd614b5c3 => generated 1166 bytes in 303 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:15:05.652211 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-2855c69d-364e-4377-96be-75e6de1bb18a tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:15:05.652850 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-2855c69d-364e-4377-96be-75e6de1bb18a tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] [instance: 84ab7e65-1bf0-4b8b-baea-6d1d4ca13dd7] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:15:05.756354 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-ff031424-7032-4e07-9aed-0eb74918058a tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '53af488f3b0242a797f539284b78d59e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bb1ede0d0b6941cc97278e6c617f9b01', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:15:05.766788 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-ff031424-7032-4e07-9aed-0eb74918058a tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] 10.222.0.16 "GET /compute/v2.1/servers/0079e567-30df-4cac-a0ad-4379bcbbcf48" status: 200 len: 1265 microversion: 2.1 time: 0.295503 Jul 16 21:15:05.771949 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 849/1648] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:15:05 2022] GET /compute/v2.1/servers/0079e567-30df-4cac-a0ad-4379bcbbcf48 => generated 1265 bytes in 301 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:15:05.799205 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-8e92df4f-0d25-4c8a-a857-2317858e868f tempest-ImagesMetadataTestJSON-1264454256 tempest-ImagesMetadataTestJSON-1264454256-project] Action: 'update_all', calling method: .version_select at 0x7fdd9d4349d0>, body: {"metadata": {"os_version": "value1", "os_distro": "value2"}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:15:05.905425 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-8e92df4f-0d25-4c8a-a857-2317858e868f tempest-ImagesMetadataTestJSON-1264454256 tempest-ImagesMetadataTestJSON-1264454256-project] Getting quotas for project a30b6074cef24df6ba020b01fc85b3e3. Resources: dict_keys(['metadata_items']) {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:15:05.912789 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-8e92df4f-0d25-4c8a-a857-2317858e868f tempest-ImagesMetadataTestJSON-1264454256 tempest-ImagesMetadataTestJSON-1264454256-project] Getting quotas for user bd9e72b27f304a9383d7f3fdcd718972 and project a30b6074cef24df6ba020b01fc85b3e3. Resources: dict_keys(['metadata_items']) {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:15:06.215120 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-2855c69d-364e-4377-96be-75e6de1bb18a tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:15:06.226617 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-2855c69d-364e-4377-96be-75e6de1bb18a tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:15:06.236966 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-2855c69d-364e-4377-96be-75e6de1bb18a tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:15:06.239001 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-2855c69d-364e-4377-96be-75e6de1bb18a tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] 10.222.0.16 "GET /compute/v2.1/servers/84ab7e65-1bf0-4b8b-baea-6d1d4ca13dd7" status: 200 len: 1104 microversion: 2.25 time: 0.599072 Jul 16 21:15:06.250357 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 800/1649] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:15:05 2022] GET /compute/v2.1/servers/84ab7e65-1bf0-4b8b-baea-6d1d4ca13dd7 => generated 1104 bytes in 611 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:15:06.492616 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-caa14d83-d8ee-4dbb-b2b4-889e0bc1f0fe tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:15:06.493430 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-caa14d83-d8ee-4dbb-b2b4-889e0bc1f0fe tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] [instance: 39e51d01-21e0-4dd5-bfd4-16b0f8b0f827] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:15:06.506506 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-caa14d83-d8ee-4dbb-b2b4-889e0bc1f0fe tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:06.522203 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-caa14d83-d8ee-4dbb-b2b4-889e0bc1f0fe tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:06.523424 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-caa14d83-d8ee-4dbb-b2b4-889e0bc1f0fe tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:06.525528 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-caa14d83-d8ee-4dbb-b2b4-889e0bc1f0fe tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:06.751601 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-8e92df4f-0d25-4c8a-a857-2317858e868f tempest-ImagesMetadataTestJSON-1264454256 tempest-ImagesMetadataTestJSON-1264454256-project] 10.222.0.16 "PUT /compute/v2.1/images/9ab4ec44-a79d-4f52-93c5-c44ab4e3a79f/metadata" status: 200 len: 61 microversion: 2.1 time: 0.978392 Jul 16 21:15:06.753045 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 850/1650] 10.222.0.16 () {60 vars in 1279 bytes} [Sat Jul 16 21:15:05 2022] PUT /compute/v2.1/images/9ab4ec44-a79d-4f52-93c5-c44ab4e3a79f/metadata => generated 61 bytes in 980 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Jul 16 21:15:06.759830 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-5f0b15c7-b087-4e61-9bd7-882bfe96a760 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:15:06.765177 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-5f0b15c7-b087-4e61-9bd7-882bfe96a760 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] [instance: 23df6e0a-3637-4f59-93f9-b02dd614b5c3] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:15:06.779154 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5f0b15c7-b087-4e61-9bd7-882bfe96a760 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:06.779675 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5f0b15c7-b087-4e61-9bd7-882bfe96a760 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:06.780662 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5f0b15c7-b087-4e61-9bd7-882bfe96a760 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:06.780897 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5f0b15c7-b087-4e61-9bd7-882bfe96a760 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:07.152097 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-caa14d83-d8ee-4dbb-b2b4-889e0bc1f0fe tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4e9df82f95ac4779859972403869096a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4337575622d04a26942905c9dc044ba7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:15:07.169457 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-caa14d83-d8ee-4dbb-b2b4-889e0bc1f0fe tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] 10.222.0.16 "GET /compute/v2.1/servers/39e51d01-21e0-4dd5-bfd4-16b0f8b0f827" status: 200 len: 1283 microversion: 2.1 time: 0.682076 Jul 16 21:15:07.173162 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 801/1651] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:15:06 2022] GET /compute/v2.1/servers/39e51d01-21e0-4dd5-bfd4-16b0f8b0f827 => generated 1283 bytes in 688 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:15:07.190331 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-cffe4916-8048-445b-a56d-6db42be28b31 tempest-ImagesMetadataTestJSON-1264454256 tempest-ImagesMetadataTestJSON-1264454256-project] Action: 'update', calling method: .version_select at 0x7fdd9cdc1790>, body: {"meta": {"os_version": "alt"}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:15:07.259913 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-5f0b15c7-b087-4e61-9bd7-882bfe96a760 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a69f9dee03d43beaafa966216b9e289', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9b769bf8f3ea4e368bf68de1df6538f9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:15:07.275133 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-5f0b15c7-b087-4e61-9bd7-882bfe96a760 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] 10.222.0.16 "GET /compute/v2.1/servers/23df6e0a-3637-4f59-93f9-b02dd614b5c3" status: 200 len: 1166 microversion: 2.1 time: 0.518846 Jul 16 21:15:07.281147 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 851/1652] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:15:06 2022] GET /compute/v2.1/servers/23df6e0a-3637-4f59-93f9-b02dd614b5c3 => generated 1166 bytes in 527 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:15:07.286698 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-e3786636-e8df-473b-b0b9-e183895039ec tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:15:07.287595 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-e3786636-e8df-473b-b0b9-e183895039ec tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] [instance: 0079e567-30df-4cac-a0ad-4379bcbbcf48] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:15:07.298335 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e3786636-e8df-473b-b0b9-e183895039ec tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:07.301411 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e3786636-e8df-473b-b0b9-e183895039ec tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:07.301411 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e3786636-e8df-473b-b0b9-e183895039ec tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:07.301411 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e3786636-e8df-473b-b0b9-e183895039ec tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:07.361356 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-cffe4916-8048-445b-a56d-6db42be28b31 tempest-ImagesMetadataTestJSON-1264454256 tempest-ImagesMetadataTestJSON-1264454256-project] Getting quotas for project a30b6074cef24df6ba020b01fc85b3e3. Resources: dict_keys(['metadata_items']) {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:15:07.367275 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-cffe4916-8048-445b-a56d-6db42be28b31 tempest-ImagesMetadataTestJSON-1264454256 tempest-ImagesMetadataTestJSON-1264454256-project] Getting quotas for user bd9e72b27f304a9383d7f3fdcd718972 and project a30b6074cef24df6ba020b01fc85b3e3. Resources: dict_keys(['metadata_items']) {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:15:07.476648 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-e3786636-e8df-473b-b0b9-e183895039ec tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '53af488f3b0242a797f539284b78d59e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bb1ede0d0b6941cc97278e6c617f9b01', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:15:07.490618 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-e3786636-e8df-473b-b0b9-e183895039ec tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] 10.222.0.16 "GET /compute/v2.1/servers/0079e567-30df-4cac-a0ad-4379bcbbcf48" status: 200 len: 1271 microversion: 2.1 time: 0.207959 Jul 16 21:15:07.517652 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 852/1653] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:15:07 2022] GET /compute/v2.1/servers/0079e567-30df-4cac-a0ad-4379bcbbcf48 => generated 1271 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:15:07.528290 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-244e914d-618b-4301-becb-7273d6ecbe61 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:15:07.529062 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-244e914d-618b-4301-becb-7273d6ecbe61 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] [instance: 84ab7e65-1bf0-4b8b-baea-6d1d4ca13dd7] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:15:07.548325 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-244e914d-618b-4301-becb-7273d6ecbe61 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:07.548803 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-244e914d-618b-4301-becb-7273d6ecbe61 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:07.549616 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-244e914d-618b-4301-becb-7273d6ecbe61 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:07.550178 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-244e914d-618b-4301-becb-7273d6ecbe61 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:07.772640 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-244e914d-618b-4301-becb-7273d6ecbe61 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:15:07.797688 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-244e914d-618b-4301-becb-7273d6ecbe61 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:15:07.799254 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-244e914d-618b-4301-becb-7273d6ecbe61 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:15:07.805236 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-244e914d-618b-4301-becb-7273d6ecbe61 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] 10.222.0.16 "GET /compute/v2.1/servers/84ab7e65-1bf0-4b8b-baea-6d1d4ca13dd7" status: 200 len: 1185 microversion: 2.25 time: 0.293828 Jul 16 21:15:07.805833 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 853/1654] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:15:07 2022] GET /compute/v2.1/servers/84ab7e65-1bf0-4b8b-baea-6d1d4ca13dd7 => generated 1185 bytes in 297 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:15:08.120151 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-cffe4916-8048-445b-a56d-6db42be28b31 tempest-ImagesMetadataTestJSON-1264454256 tempest-ImagesMetadataTestJSON-1264454256-project] 10.222.0.16 "PUT /compute/v2.1/images/9ab4ec44-a79d-4f52-93c5-c44ab4e3a79f/metadata/os_version" status: 200 len: 31 microversion: 2.1 time: 0.944426 Jul 16 21:15:08.127487 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 802/1655] 10.222.0.16 () {60 vars in 1312 bytes} [Sat Jul 16 21:15:07 2022] PUT /compute/v2.1/images/9ab4ec44-a79d-4f52-93c5-c44ab4e3a79f/metadata/os_version => generated 31 bytes in 953 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Jul 16 21:15:08.131639 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-fc0055ad-a22c-4514-95f1-408cf1251992 tempest-ImagesMetadataTestJSON-1264454256 tempest-ImagesMetadataTestJSON-1264454256-project] Calling method '.version_select at 0x7fdd9d4e0790>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:15:08.191119 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-bacdbff3-7974-4a05-b7b8-aac6ebf40dc4 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:15:08.192527 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-bacdbff3-7974-4a05-b7b8-aac6ebf40dc4 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] [instance: 39e51d01-21e0-4dd5-bfd4-16b0f8b0f827] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:15:08.207390 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-fc0055ad-a22c-4514-95f1-408cf1251992 tempest-ImagesMetadataTestJSON-1264454256 tempest-ImagesMetadataTestJSON-1264454256-project] 10.222.0.16 "GET /compute/v2.1/images/9ab4ec44-a79d-4f52-93c5-c44ab4e3a79f/metadata" status: 200 len: 58 microversion: 2.1 time: 0.080753 Jul 16 21:15:08.208826 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 854/1656] 10.222.0.16 () {58 vars in 1259 bytes} [Sat Jul 16 21:15:08 2022] GET /compute/v2.1/images/9ab4ec44-a79d-4f52-93c5-c44ab4e3a79f/metadata => generated 58 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Jul 16 21:15:08.213254 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-bacdbff3-7974-4a05-b7b8-aac6ebf40dc4 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:08.213254 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-bacdbff3-7974-4a05-b7b8-aac6ebf40dc4 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:08.213254 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-bacdbff3-7974-4a05-b7b8-aac6ebf40dc4 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:08.220248 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-bacdbff3-7974-4a05-b7b8-aac6ebf40dc4 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:08.229385 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-589f24c5-3e3c-4286-a88d-e85a4353496f tempest-ImagesMetadataTestJSON-1264454256 tempest-ImagesMetadataTestJSON-1264454256-project] Action: 'update_all', calling method: .version_select at 0x7fdd9cc3de50>, body: {"metadata": {"os_version": "value1", "os_distro": "value2"}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:15:08.310018 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-589f24c5-3e3c-4286-a88d-e85a4353496f tempest-ImagesMetadataTestJSON-1264454256 tempest-ImagesMetadataTestJSON-1264454256-project] Getting quotas for project a30b6074cef24df6ba020b01fc85b3e3. Resources: dict_keys(['metadata_items']) {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:15:08.318055 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-589f24c5-3e3c-4286-a88d-e85a4353496f tempest-ImagesMetadataTestJSON-1264454256 tempest-ImagesMetadataTestJSON-1264454256-project] Getting quotas for user bd9e72b27f304a9383d7f3fdcd718972 and project a30b6074cef24df6ba020b01fc85b3e3. Resources: dict_keys(['metadata_items']) {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:15:08.981230 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-589f24c5-3e3c-4286-a88d-e85a4353496f tempest-ImagesMetadataTestJSON-1264454256 tempest-ImagesMetadataTestJSON-1264454256-project] 10.222.0.16 "PUT /compute/v2.1/images/9ab4ec44-a79d-4f52-93c5-c44ab4e3a79f/metadata" status: 200 len: 61 microversion: 2.1 time: 0.765381 Jul 16 21:15:08.982593 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 855/1657] 10.222.0.16 () {60 vars in 1279 bytes} [Sat Jul 16 21:15:08 2022] PUT /compute/v2.1/images/9ab4ec44-a79d-4f52-93c5-c44ab4e3a79f/metadata => generated 61 bytes in 768 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Jul 16 21:15:08.995437 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-dc3c7bc0-ba2b-439f-a6c8-bad245931f55 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:15:08.996402 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-dc3c7bc0-ba2b-439f-a6c8-bad245931f55 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] [instance: 23df6e0a-3637-4f59-93f9-b02dd614b5c3] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:15:09.008287 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-dc3c7bc0-ba2b-439f-a6c8-bad245931f55 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:09.010392 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-dc3c7bc0-ba2b-439f-a6c8-bad245931f55 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:09.012320 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-dc3c7bc0-ba2b-439f-a6c8-bad245931f55 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:09.014597 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-dc3c7bc0-ba2b-439f-a6c8-bad245931f55 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:09.105729 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-bacdbff3-7974-4a05-b7b8-aac6ebf40dc4 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4e9df82f95ac4779859972403869096a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4337575622d04a26942905c9dc044ba7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:15:09.119888 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-bacdbff3-7974-4a05-b7b8-aac6ebf40dc4 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] 10.222.0.16 "GET /compute/v2.1/servers/39e51d01-21e0-4dd5-bfd4-16b0f8b0f827" status: 200 len: 1325 microversion: 2.1 time: 0.933540 Jul 16 21:15:09.129984 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 803/1658] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:15:08 2022] GET /compute/v2.1/servers/39e51d01-21e0-4dd5-bfd4-16b0f8b0f827 => generated 1325 bytes in 936 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:15:09.129984 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-be554c89-ec64-4349-914b-bfac26d02b38 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:15:09.129984 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-be554c89-ec64-4349-914b-bfac26d02b38 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] [instance: 0079e567-30df-4cac-a0ad-4379bcbbcf48] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:15:09.152701 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-be554c89-ec64-4349-914b-bfac26d02b38 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:09.153307 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-be554c89-ec64-4349-914b-bfac26d02b38 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:09.154335 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-be554c89-ec64-4349-914b-bfac26d02b38 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:09.154976 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-be554c89-ec64-4349-914b-bfac26d02b38 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:10.193318 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-be554c89-ec64-4349-914b-bfac26d02b38 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '53af488f3b0242a797f539284b78d59e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bb1ede0d0b6941cc97278e6c617f9b01', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:15:10.193318 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-be554c89-ec64-4349-914b-bfac26d02b38 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] 10.222.0.16 "GET /compute/v2.1/servers/0079e567-30df-4cac-a0ad-4379bcbbcf48" status: 200 len: 1313 microversion: 2.1 time: 0.944284 Jul 16 21:15:10.193318 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 804/1659] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:15:09 2022] GET /compute/v2.1/servers/0079e567-30df-4cac-a0ad-4379bcbbcf48 => generated 1313 bytes in 946 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:15:10.193318 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-f79cd070-47e4-4cc0-8df7-7593c3ecd401 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:15:10.196753 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-f79cd070-47e4-4cc0-8df7-7593c3ecd401 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] [instance: 84ab7e65-1bf0-4b8b-baea-6d1d4ca13dd7] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:15:10.196753 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f79cd070-47e4-4cc0-8df7-7593c3ecd401 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:10.196753 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f79cd070-47e4-4cc0-8df7-7593c3ecd401 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:10.196753 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f79cd070-47e4-4cc0-8df7-7593c3ecd401 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:10.197361 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f79cd070-47e4-4cc0-8df7-7593c3ecd401 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:10.197551 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-dc3c7bc0-ba2b-439f-a6c8-bad245931f55 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a69f9dee03d43beaafa966216b9e289', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9b769bf8f3ea4e368bf68de1df6538f9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:15:10.197551 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-dc3c7bc0-ba2b-439f-a6c8-bad245931f55 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] 10.222.0.16 "GET /compute/v2.1/servers/23df6e0a-3637-4f59-93f9-b02dd614b5c3" status: 200 len: 1196 microversion: 2.1 time: 1.130154 Jul 16 21:15:10.197551 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 856/1660] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:15:08 2022] GET /compute/v2.1/servers/23df6e0a-3637-4f59-93f9-b02dd614b5c3 => generated 1196 bytes in 1132 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:15:10.197551 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-1d9fab2a-9dfc-4aef-93e4-6db4c03e62d3 tempest-ImagesMetadataTestJSON-1264454256 tempest-ImagesMetadataTestJSON-1264454256-project] Action: 'create', calling method: .version_select at 0x7fdd9d3288b0>, body: {"metadata": {"os_version": "alt1", "architecture": "value3"}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:15:10.222391 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-1d9fab2a-9dfc-4aef-93e4-6db4c03e62d3 tempest-ImagesMetadataTestJSON-1264454256 tempest-ImagesMetadataTestJSON-1264454256-project] Getting quotas for project a30b6074cef24df6ba020b01fc85b3e3. Resources: dict_keys(['metadata_items']) {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:15:10.233750 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-1d9fab2a-9dfc-4aef-93e4-6db4c03e62d3 tempest-ImagesMetadataTestJSON-1264454256 tempest-ImagesMetadataTestJSON-1264454256-project] Getting quotas for user bd9e72b27f304a9383d7f3fdcd718972 and project a30b6074cef24df6ba020b01fc85b3e3. Resources: dict_keys(['metadata_items']) {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:15:10.289676 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-f79cd070-47e4-4cc0-8df7-7593c3ecd401 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:15:10.304987 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-f79cd070-47e4-4cc0-8df7-7593c3ecd401 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:15:10.312168 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-f79cd070-47e4-4cc0-8df7-7593c3ecd401 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:15:10.316888 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-f79cd070-47e4-4cc0-8df7-7593c3ecd401 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] 10.222.0.16 "GET /compute/v2.1/servers/84ab7e65-1bf0-4b8b-baea-6d1d4ca13dd7" status: 200 len: 1251 microversion: 2.25 time: 0.247156 Jul 16 21:15:10.321002 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 805/1661] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:15:10 2022] GET /compute/v2.1/servers/84ab7e65-1bf0-4b8b-baea-6d1d4ca13dd7 => generated 1251 bytes in 253 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:15:10.338204 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-bf6c15b9-3285-4154-9ebb-bbb803d4e5a8 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:15:10.340316 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-bf6c15b9-3285-4154-9ebb-bbb803d4e5a8 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] [instance: 39e51d01-21e0-4dd5-bfd4-16b0f8b0f827] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:15:10.358704 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-bf6c15b9-3285-4154-9ebb-bbb803d4e5a8 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:10.359474 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-bf6c15b9-3285-4154-9ebb-bbb803d4e5a8 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:10.382704 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-bf6c15b9-3285-4154-9ebb-bbb803d4e5a8 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:10.383215 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-bf6c15b9-3285-4154-9ebb-bbb803d4e5a8 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:10.971629 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-1d9fab2a-9dfc-4aef-93e4-6db4c03e62d3 tempest-ImagesMetadataTestJSON-1264454256 tempest-ImagesMetadataTestJSON-1264454256-project] 10.222.0.16 "POST /compute/v2.1/images/9ab4ec44-a79d-4f52-93c5-c44ab4e3a79f/metadata" status: 200 len: 85 microversion: 2.1 time: 0.854542 Jul 16 21:15:10.972204 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 857/1662] 10.222.0.16 () {60 vars in 1280 bytes} [Sat Jul 16 21:15:10 2022] POST /compute/v2.1/images/9ab4ec44-a79d-4f52-93c5-c44ab4e3a79f/metadata => generated 85 bytes in 856 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Jul 16 21:15:11.045637 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-9ef5aacc-7e14-49c8-8ac9-ee0083f7dbc7 tempest-ImagesMetadataTestJSON-1264454256 tempest-ImagesMetadataTestJSON-1264454256-project] Calling method '.version_select at 0x7fdd9cc3f430>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:15:11.120180 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-9ef5aacc-7e14-49c8-8ac9-ee0083f7dbc7 tempest-ImagesMetadataTestJSON-1264454256 tempest-ImagesMetadataTestJSON-1264454256-project] 10.222.0.16 "GET /compute/v2.1/images/9ab4ec44-a79d-4f52-93c5-c44ab4e3a79f/metadata" status: 200 len: 85 microversion: 2.1 time: 0.078299 Jul 16 21:15:11.129436 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 858/1663] 10.222.0.16 () {58 vars in 1259 bytes} [Sat Jul 16 21:15:11 2022] GET /compute/v2.1/images/9ab4ec44-a79d-4f52-93c5-c44ab4e3a79f/metadata => generated 85 bytes in 80 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Jul 16 21:15:11.203235 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-421b11e5-eee3-4cd0-a7b9-20bf080458f5 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:15:11.203864 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-421b11e5-eee3-4cd0-a7b9-20bf080458f5 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] [instance: 23df6e0a-3637-4f59-93f9-b02dd614b5c3] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:15:11.219606 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-421b11e5-eee3-4cd0-a7b9-20bf080458f5 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:11.224241 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-421b11e5-eee3-4cd0-a7b9-20bf080458f5 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:11.224443 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-421b11e5-eee3-4cd0-a7b9-20bf080458f5 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:11.225016 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-421b11e5-eee3-4cd0-a7b9-20bf080458f5 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:12.076665 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-421b11e5-eee3-4cd0-a7b9-20bf080458f5 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a69f9dee03d43beaafa966216b9e289', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9b769bf8f3ea4e368bf68de1df6538f9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:15:12.098636 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-421b11e5-eee3-4cd0-a7b9-20bf080458f5 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] 10.222.0.16 "GET /compute/v2.1/servers/23df6e0a-3637-4f59-93f9-b02dd614b5c3" status: 200 len: 1196 microversion: 2.1 time: 0.899363 Jul 16 21:15:12.106156 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-bf6c15b9-3285-4154-9ebb-bbb803d4e5a8 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4e9df82f95ac4779859972403869096a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4337575622d04a26942905c9dc044ba7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:15:12.107413 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 859/1664] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:15:11 2022] GET /compute/v2.1/servers/23df6e0a-3637-4f59-93f9-b02dd614b5c3 => generated 1196 bytes in 909 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:15:12.113905 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-c5c53ee1-4c55-4cf7-a2c9-483dbfa9b0af tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:15:12.120890 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-c5c53ee1-4c55-4cf7-a2c9-483dbfa9b0af tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] [instance: 0079e567-30df-4cac-a0ad-4379bcbbcf48] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:15:12.126199 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-bf6c15b9-3285-4154-9ebb-bbb803d4e5a8 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] 10.222.0.16 "GET /compute/v2.1/servers/39e51d01-21e0-4dd5-bfd4-16b0f8b0f827" status: 200 len: 1325 microversion: 2.1 time: 1.803090 Jul 16 21:15:12.132442 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c5c53ee1-4c55-4cf7-a2c9-483dbfa9b0af tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:12.137073 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c5c53ee1-4c55-4cf7-a2c9-483dbfa9b0af tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:12.137073 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c5c53ee1-4c55-4cf7-a2c9-483dbfa9b0af tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:12.137073 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c5c53ee1-4c55-4cf7-a2c9-483dbfa9b0af tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:12.137437 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 806/1665] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:15:10 2022] GET /compute/v2.1/servers/39e51d01-21e0-4dd5-bfd4-16b0f8b0f827 => generated 1325 bytes in 1813 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:15:12.148765 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-300a1ae8-8100-472a-ab4d-c6ccf90936af tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:15:12.159997 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-300a1ae8-8100-472a-ab4d-c6ccf90936af tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] [instance: 84ab7e65-1bf0-4b8b-baea-6d1d4ca13dd7] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:15:12.183153 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-300a1ae8-8100-472a-ab4d-c6ccf90936af tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:12.184288 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-300a1ae8-8100-472a-ab4d-c6ccf90936af tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:12.185921 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-300a1ae8-8100-472a-ab4d-c6ccf90936af tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:12.186542 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-300a1ae8-8100-472a-ab4d-c6ccf90936af tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:12.609269 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell0:SELECT=4 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:15:13.229069 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-c5c53ee1-4c55-4cf7-a2c9-483dbfa9b0af tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '53af488f3b0242a797f539284b78d59e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bb1ede0d0b6941cc97278e6c617f9b01', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:15:13.270599 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-c5c53ee1-4c55-4cf7-a2c9-483dbfa9b0af tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] 10.222.0.16 "GET /compute/v2.1/servers/0079e567-30df-4cac-a0ad-4379bcbbcf48" status: 200 len: 1334 microversion: 2.1 time: 1.161602 Jul 16 21:15:13.293849 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 860/1666] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:15:12 2022] GET /compute/v2.1/servers/0079e567-30df-4cac-a0ad-4379bcbbcf48 => generated 1334 bytes in 1171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:15:13.310131 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-2ae5d0c5-0910-47d4-96ed-52a776c49a58 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:15:13.310800 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-2ae5d0c5-0910-47d4-96ed-52a776c49a58 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] [instance: 23df6e0a-3637-4f59-93f9-b02dd614b5c3] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:15:13.337487 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2ae5d0c5-0910-47d4-96ed-52a776c49a58 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:13.341045 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2ae5d0c5-0910-47d4-96ed-52a776c49a58 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:13.346804 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2ae5d0c5-0910-47d4-96ed-52a776c49a58 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:13.348695 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2ae5d0c5-0910-47d4-96ed-52a776c49a58 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:13.441981 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-300a1ae8-8100-472a-ab4d-c6ccf90936af tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:15:13.454372 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-300a1ae8-8100-472a-ab4d-c6ccf90936af tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:15:13.459212 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-300a1ae8-8100-472a-ab4d-c6ccf90936af tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:15:13.463170 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-300a1ae8-8100-472a-ab4d-c6ccf90936af tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] 10.222.0.16 "GET /compute/v2.1/servers/84ab7e65-1bf0-4b8b-baea-6d1d4ca13dd7" status: 200 len: 1293 microversion: 2.25 time: 1.326737 Jul 16 21:15:13.472099 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 807/1667] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:15:12 2022] GET /compute/v2.1/servers/84ab7e65-1bf0-4b8b-baea-6d1d4ca13dd7 => generated 1293 bytes in 1337 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:15:13.501579 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-afa1e061-24cd-4599-8c57-3f2842e626bf tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:15:13.501579 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-afa1e061-24cd-4599-8c57-3f2842e626bf tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] [instance: 39e51d01-21e0-4dd5-bfd4-16b0f8b0f827] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:15:13.514762 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-afa1e061-24cd-4599-8c57-3f2842e626bf tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:13.515494 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-afa1e061-24cd-4599-8c57-3f2842e626bf tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:13.518138 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-afa1e061-24cd-4599-8c57-3f2842e626bf tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:13.518821 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-afa1e061-24cd-4599-8c57-3f2842e626bf tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:14.175339 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-afa1e061-24cd-4599-8c57-3f2842e626bf tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4e9df82f95ac4779859972403869096a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4337575622d04a26942905c9dc044ba7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:15:14.189395 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-afa1e061-24cd-4599-8c57-3f2842e626bf tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] 10.222.0.16 "GET /compute/v2.1/servers/39e51d01-21e0-4dd5-bfd4-16b0f8b0f827" status: 200 len: 1325 microversion: 2.1 time: 0.715290 Jul 16 21:15:14.190762 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 808/1668] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:15:13 2022] GET /compute/v2.1/servers/39e51d01-21e0-4dd5-bfd4-16b0f8b0f827 => generated 1325 bytes in 718 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:15:14.206485 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell0:SELECT=4 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:15:14.301450 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-2ae5d0c5-0910-47d4-96ed-52a776c49a58 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a69f9dee03d43beaafa966216b9e289', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9b769bf8f3ea4e368bf68de1df6538f9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:15:14.309313 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-2ae5d0c5-0910-47d4-96ed-52a776c49a58 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] 10.222.0.16 "GET /compute/v2.1/servers/23df6e0a-3637-4f59-93f9-b02dd614b5c3" status: 200 len: 1196 microversion: 2.1 time: 1.028361 Jul 16 21:15:14.310142 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 861/1669] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:15:13 2022] GET /compute/v2.1/servers/23df6e0a-3637-4f59-93f9-b02dd614b5c3 => generated 1196 bytes in 1030 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:15:14.322181 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-0d432e51-5456-4a07-a3ac-d3a1c5f910e5 req-2f7130ea-e705-4cfe-b134-10d11753f79f service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "39e51d01-21e0-4dd5-bfd4-16b0f8b0f827", "tag": "55c3f33b-16c2-42c1-94d7-42da613add6d"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:15:14.340988 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-0d432e51-5456-4a07-a3ac-d3a1c5f910e5 req-2f7130ea-e705-4cfe-b134-10d11753f79f service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:14.343918 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-0d432e51-5456-4a07-a3ac-d3a1c5f910e5 req-2f7130ea-e705-4cfe-b134-10d11753f79f service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:14.386674 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-0d432e51-5456-4a07-a3ac-d3a1c5f910e5 req-2f7130ea-e705-4cfe-b134-10d11753f79f service nova] Creating event network-changed:55c3f33b-16c2-42c1-94d7-42da613add6d for instance 39e51d01-21e0-4dd5-bfd4-16b0f8b0f827 on nch2-master-1 Jul 16 21:15:14.422743 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-07c8301b-c74b-467f-b698-150c88451637 req-ce61ce1d-aebb-41de-9ee9-10256ab7c04f service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "0079e567-30df-4cac-a0ad-4379bcbbcf48", "tag": "1079a73b-5553-45b8-bff2-e4be618fbceb"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:15:14.435174 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-0d432e51-5456-4a07-a3ac-d3a1c5f910e5 req-2f7130ea-e705-4cfe-b134-10d11753f79f service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.118881 Jul 16 21:15:14.439473 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 862/1670] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:15:14 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 124 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:15:14.452586 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-07c8301b-c74b-467f-b698-150c88451637 req-ce61ce1d-aebb-41de-9ee9-10256ab7c04f service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:14.456149 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-07c8301b-c74b-467f-b698-150c88451637 req-ce61ce1d-aebb-41de-9ee9-10256ab7c04f service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:14.473267 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-c815c8e0-928a-40dd-b456-641556db4c3d tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:15:14.474422 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-c815c8e0-928a-40dd-b456-641556db4c3d tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] [instance: 0079e567-30df-4cac-a0ad-4379bcbbcf48] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:15:14.483358 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c815c8e0-928a-40dd-b456-641556db4c3d tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:14.487227 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c815c8e0-928a-40dd-b456-641556db4c3d tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:14.487227 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c815c8e0-928a-40dd-b456-641556db4c3d tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:14.487227 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c815c8e0-928a-40dd-b456-641556db4c3d tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:14.512699 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-07c8301b-c74b-467f-b698-150c88451637 req-ce61ce1d-aebb-41de-9ee9-10256ab7c04f service nova] Creating event network-changed:1079a73b-5553-45b8-bff2-e4be618fbceb for instance 0079e567-30df-4cac-a0ad-4379bcbbcf48 on nch2-master-1 Jul 16 21:15:14.530944 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-07c8301b-c74b-467f-b698-150c88451637 req-ce61ce1d-aebb-41de-9ee9-10256ab7c04f service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.338209 Jul 16 21:15:14.531900 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 809/1671] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:15:14 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 340 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:15:14.540342 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-fd33cfb1-f184-4557-b2b7-f51d5aec4999 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:15:14.545310 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-fd33cfb1-f184-4557-b2b7-f51d5aec4999 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] [instance: 84ab7e65-1bf0-4b8b-baea-6d1d4ca13dd7] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:15:14.562186 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fd33cfb1-f184-4557-b2b7-f51d5aec4999 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:14.566129 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fd33cfb1-f184-4557-b2b7-f51d5aec4999 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:14.567789 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fd33cfb1-f184-4557-b2b7-f51d5aec4999 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:14.568444 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fd33cfb1-f184-4557-b2b7-f51d5aec4999 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:15.525829 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-c815c8e0-928a-40dd-b456-641556db4c3d tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '53af488f3b0242a797f539284b78d59e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bb1ede0d0b6941cc97278e6c617f9b01', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:15:15.544806 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-c815c8e0-928a-40dd-b456-641556db4c3d tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] 10.222.0.16 "GET /compute/v2.1/servers/0079e567-30df-4cac-a0ad-4379bcbbcf48" status: 200 len: 1334 microversion: 2.1 time: 1.093977 Jul 16 21:15:15.550966 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 863/1672] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:15:14 2022] GET /compute/v2.1/servers/0079e567-30df-4cac-a0ad-4379bcbbcf48 => generated 1334 bytes in 1106 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:15:15.711326 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-fd33cfb1-f184-4557-b2b7-f51d5aec4999 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:15:15.727479 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-fd33cfb1-f184-4557-b2b7-f51d5aec4999 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:15:15.733495 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-fd33cfb1-f184-4557-b2b7-f51d5aec4999 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:15:15.741176 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-fd33cfb1-f184-4557-b2b7-f51d5aec4999 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] 10.222.0.16 "GET /compute/v2.1/servers/84ab7e65-1bf0-4b8b-baea-6d1d4ca13dd7" status: 200 len: 1281 microversion: 2.25 time: 1.207926 Jul 16 21:15:15.743124 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 810/1673] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:15:14 2022] GET /compute/v2.1/servers/84ab7e65-1bf0-4b8b-baea-6d1d4ca13dd7 => generated 1281 bytes in 1210 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:15:15.758148 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-a1861347-5661-4348-8270-7db6a3ca38af tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:15:15.759251 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-a1861347-5661-4348-8270-7db6a3ca38af tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] [instance: 39e51d01-21e0-4dd5-bfd4-16b0f8b0f827] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:15:15.779318 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a1861347-5661-4348-8270-7db6a3ca38af tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:15.779725 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a1861347-5661-4348-8270-7db6a3ca38af tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:15.780729 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a1861347-5661-4348-8270-7db6a3ca38af tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:15.784926 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a1861347-5661-4348-8270-7db6a3ca38af tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:16.018506 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-da22345b-0460-4f5f-9298-33df3216dc05 tempest-ImagesDeleteNegativeTestJSON-956360194 tempest-ImagesDeleteNegativeTestJSON-956360194-project] 10.222.0.16 "DELETE /compute/v2.1/images/" status: 404 len: 112 microversion: - time: 0.471389 Jul 16 21:15:16.019380 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 864/1674] 10.222.0.16 () {58 vars in 1127 bytes} [Sat Jul 16 21:15:15 2022] DELETE /compute/v2.1/images/ => generated 112 bytes in 473 msecs (HTTP/1.1 404) 5 headers in 228 bytes (1 switches on core 0) Jul 16 21:15:16.028665 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-de6fcc93-3404-42d7-baf8-680480f22537 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:15:16.030802 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-de6fcc93-3404-42d7-baf8-680480f22537 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] [instance: 23df6e0a-3637-4f59-93f9-b02dd614b5c3] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:15:16.050043 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-de6fcc93-3404-42d7-baf8-680480f22537 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:16.051708 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-de6fcc93-3404-42d7-baf8-680480f22537 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:16.052608 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-de6fcc93-3404-42d7-baf8-680480f22537 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:16.053245 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-de6fcc93-3404-42d7-baf8-680480f22537 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:16.331610 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-a1861347-5661-4348-8270-7db6a3ca38af tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4e9df82f95ac4779859972403869096a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4337575622d04a26942905c9dc044ba7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:15:16.344784 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-a1861347-5661-4348-8270-7db6a3ca38af tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] 10.222.0.16 "GET /compute/v2.1/servers/39e51d01-21e0-4dd5-bfd4-16b0f8b0f827" status: 200 len: 1325 microversion: 2.1 time: 0.597517 Jul 16 21:15:16.345534 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 811/1675] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:15:15 2022] GET /compute/v2.1/servers/39e51d01-21e0-4dd5-bfd4-16b0f8b0f827 => generated 1325 bytes in 601 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:15:16.361187 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-0695072b-23dc-4a32-95fe-48341dd76cbe tempest-ImagesDeleteNegativeTestJSON-956360194 tempest-ImagesDeleteNegativeTestJSON-956360194-project] Calling method '.version_select at 0x7fdd9d52e160>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:15:16.443265 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-0695072b-23dc-4a32-95fe-48341dd76cbe tempest-ImagesDeleteNegativeTestJSON-956360194 tempest-ImagesDeleteNegativeTestJSON-956360194-project] HTTP exception thrown: Image not found. Jul 16 21:15:16.494073 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-0695072b-23dc-4a32-95fe-48341dd76cbe tempest-ImagesDeleteNegativeTestJSON-956360194 tempest-ImagesDeleteNegativeTestJSON-956360194-project] Request method failure captured: Jul 16 21:15:16.494073 ncdv-master-1 devstack@n-api.service[138681]: request: DELETE /compute/v2.1/images/-1 HTTP/1.1 Jul 16 21:15:16.494073 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:15:16.494073 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:15:16.494073 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:15:16.494073 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 21:15:16.494073 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:15:16.494073 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:15:16.494073 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:15:16.494073 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:15:16.494073 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0ypiy5pOlkK7-D_CSkb9ZozS5v9QeDR2dv8PgJfbWR-ih6gM6SuerUpWfWvQBucqG0xGxZqRipWwdjblc0QTBpIuu2zp5DuDXrHp-OviNTgIaNRN2NSh3CG24aNSOxPo7UzmLlXHMkUm5fbBx_Pkqs5uXfgT1XjRKDmmwJ_B4hz-LVg Jul 16 21:15:16.494073 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:15:16.494073 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:15:16.494073 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:15:16.494073 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:15:16.494073 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:15:16.494073 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:15:16.494073 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: a01b0a9bea9f4e09bcebcb9bbc0e5678 Jul 16 21:15:16.494073 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-ImagesDeleteNegativeTestJSON-956360194 Jul 16 21:15:16.494073 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:15:16.494073 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:15:16.495864 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/a01b0a9bea9f4e09bcebcb9bbc0e5678"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/a01b0a9bea9f4e09bcebcb9bbc0e5678"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_a01b0a9bea9f4e09bcebcb9bbc0e5678"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/a01b0a9bea9f4e09bcebcb9bbc0e5678"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:15:16.495864 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-ImagesDeleteNegativeTestJSON-956360194 Jul 16 21:15:16.495864 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: a01b0a9bea9f4e09bcebcb9bbc0e5678 Jul 16 21:15:16.495864 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-ImagesDeleteNegativeTestJSON-956360194 Jul 16 21:15:16.495864 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-ImagesDeleteNegativeTestJSON-956360194-project Jul 16 21:15:16.495864 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:15:16.495864 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:15:16.495864 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: cd263c351cf044fabf45e0f4f2c0ce98 Jul 16 21:15:16.495864 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-ImagesDeleteNegativeTestJSON-956360194-project Jul 16 21:15:16.495864 ncdv-master-1 devstack@n-api.service[138681]: method: Jul 16 21:15:16.495864 ncdv-master-1 devstack@n-api.service[138681]: exception: Image not found. Jul 16 21:15:16.495864 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': '-1'} Jul 16 21:15:16.495864 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:15:16.495864 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:15:16.497321 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 724, in delete Jul 16 21:15:16.497321 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi self._client.call(context, 2, 'delete', args=(image_id,)) Jul 16 21:15:16.497321 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 191, in call Jul 16 21:15:16.497321 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi result = getattr(controller, method)(*args, **kwargs) Jul 16 21:15:16.497321 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/utils.py", line 670, in inner Jul 16 21:15:16.497321 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return RequestIdProxy(wrapped(*args, **kwargs)) Jul 16 21:15:16.497321 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/v2/images.py", line 386, in delete Jul 16 21:15:16.497321 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi resp, body = self.http_client.delete(url) Jul 16 21:15:16.497321 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/adapter.py", line 410, in delete Jul 16 21:15:16.497321 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return self.request(url, 'DELETE', **kwargs) Jul 16 21:15:16.497321 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 380, in request Jul 16 21:15:16.497321 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return self._handle_response(resp) Jul 16 21:15:16.497321 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 120, in _handle_response Jul 16 21:15:16.497321 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.from_response(resp, resp.content) Jul 16 21:15:16.497321 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi glanceclient.exc.HTTPNotFound: HTTP 404 Not Found: Failed to find image -1 to delete Jul 16 21:15:16.497321 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:15:16.497321 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:15:16.497321 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:15:16.497321 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:15:16.498950 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/images.py", line 104, in delete Jul 16 21:15:16.498950 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi self._image_api.delete(context, id) Jul 16 21:15:16.498950 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 1255, in delete Jul 16 21:15:16.498950 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return session.delete(context, image_id) Jul 16 21:15:16.498950 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 726, in delete Jul 16 21:15:16.498950 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.ImageNotFound(image_id=image_id) Jul 16 21:15:16.498950 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.ImageNotFound: Image -1 could not be found. Jul 16 21:15:16.498950 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:15:16.498950 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:15:16.498950 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:15:16.498950 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:15:16.498950 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:15:16.498950 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:15:16.498950 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:15:16.498950 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:15:16.498950 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Jul 16 21:15:16.498950 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Jul 16 21:15:16.498950 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:15:16.498950 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:15:16.498950 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/images.py", line 107, in delete Jul 16 21:15:16.500683 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=explanation) Jul 16 21:15:16.500683 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Image not found. Jul 16 21:15:16.500683 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:15:16.500683 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:15:16.500683 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:15:16.500683 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:15:16.500683 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:15:16.500683 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:15:16.500683 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:15:16.500683 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:15:16.500683 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Image not found. Jul 16 21:15:16.500683 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:15:16.507477 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-0695072b-23dc-4a32-95fe-48341dd76cbe tempest-ImagesDeleteNegativeTestJSON-956360194 tempest-ImagesDeleteNegativeTestJSON-956360194-project] Returning 404 to user: Image not found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:15:16.510717 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-0695072b-23dc-4a32-95fe-48341dd76cbe tempest-ImagesDeleteNegativeTestJSON-956360194 tempest-ImagesDeleteNegativeTestJSON-956360194-project] 10.222.0.16 "DELETE /compute/v2.1/images/-1" status: 404 len: 62 microversion: 2.1 time: 0.151234 Jul 16 21:15:16.511747 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 812/1676] 10.222.0.16 () {58 vars in 1133 bytes} [Sat Jul 16 21:15:16 2022] DELETE /compute/v2.1/images/-1 => generated 62 bytes in 166 msecs (HTTP/1.1 404) 9 headers in 378 bytes (1 switches on core 0) Jul 16 21:15:16.531160 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-2ca66f57-acea-401b-8afa-1c23c1c92b0c tempest-ImagesDeleteNegativeTestJSON-956360194 tempest-ImagesDeleteNegativeTestJSON-956360194-project] Calling method '.version_select at 0x7fdda2b9c940>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:15:16.624844 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-2ca66f57-acea-401b-8afa-1c23c1c92b0c tempest-ImagesDeleteNegativeTestJSON-956360194 tempest-ImagesDeleteNegativeTestJSON-956360194-project] HTTP exception thrown: Image not found. Jul 16 21:15:16.626932 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-2ca66f57-acea-401b-8afa-1c23c1c92b0c tempest-ImagesDeleteNegativeTestJSON-956360194 tempest-ImagesDeleteNegativeTestJSON-956360194-project] Request method failure captured: Jul 16 21:15:16.626932 ncdv-master-1 devstack@n-api.service[138681]: request: DELETE /compute/v2.1/images/2c1a2d49-8f34-42f8-a3fa-1e1bbd07bcdj HTTP/1.1 Jul 16 21:15:16.626932 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:15:16.626932 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:15:16.626932 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:15:16.626932 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 21:15:16.626932 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:15:16.626932 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:15:16.626932 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:15:16.626932 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:15:16.626932 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0ypiy5pOlkK7-D_CSkb9ZozS5v9QeDR2dv8PgJfbWR-ih6gM6SuerUpWfWvQBucqG0xGxZqRipWwdjblc0QTBpIuu2zp5DuDXrHp-OviNTgIaNRN2NSh3CG24aNSOxPo7UzmLlXHMkUm5fbBx_Pkqs5uXfgT1XjRKDmmwJ_B4hz-LVg Jul 16 21:15:16.626932 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:15:16.626932 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:15:16.626932 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:15:16.626932 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:15:16.626932 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:15:16.626932 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:15:16.626932 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: a01b0a9bea9f4e09bcebcb9bbc0e5678 Jul 16 21:15:16.626932 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-ImagesDeleteNegativeTestJSON-956360194 Jul 16 21:15:16.626932 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:15:16.626932 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:15:16.629215 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/a01b0a9bea9f4e09bcebcb9bbc0e5678"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/a01b0a9bea9f4e09bcebcb9bbc0e5678"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_a01b0a9bea9f4e09bcebcb9bbc0e5678"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/a01b0a9bea9f4e09bcebcb9bbc0e5678"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:15:16.629215 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-ImagesDeleteNegativeTestJSON-956360194 Jul 16 21:15:16.629215 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: a01b0a9bea9f4e09bcebcb9bbc0e5678 Jul 16 21:15:16.629215 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-ImagesDeleteNegativeTestJSON-956360194 Jul 16 21:15:16.629215 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-ImagesDeleteNegativeTestJSON-956360194-project Jul 16 21:15:16.629215 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:15:16.629215 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:15:16.629215 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: cd263c351cf044fabf45e0f4f2c0ce98 Jul 16 21:15:16.629215 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-ImagesDeleteNegativeTestJSON-956360194-project Jul 16 21:15:16.629215 ncdv-master-1 devstack@n-api.service[138681]: method: Jul 16 21:15:16.629215 ncdv-master-1 devstack@n-api.service[138681]: exception: Image not found. Jul 16 21:15:16.629215 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': '2c1a2d49-8f34-42f8-a3fa-1e1bbd07bcdj'} Jul 16 21:15:16.629215 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:15:16.630330 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:15:16.630330 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 724, in delete Jul 16 21:15:16.630330 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi self._client.call(context, 2, 'delete', args=(image_id,)) Jul 16 21:15:16.630330 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 191, in call Jul 16 21:15:16.630330 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi result = getattr(controller, method)(*args, **kwargs) Jul 16 21:15:16.630330 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/utils.py", line 670, in inner Jul 16 21:15:16.630330 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return RequestIdProxy(wrapped(*args, **kwargs)) Jul 16 21:15:16.630330 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/v2/images.py", line 386, in delete Jul 16 21:15:16.630330 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi resp, body = self.http_client.delete(url) Jul 16 21:15:16.630330 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/adapter.py", line 410, in delete Jul 16 21:15:16.630330 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return self.request(url, 'DELETE', **kwargs) Jul 16 21:15:16.630330 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 380, in request Jul 16 21:15:16.630330 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return self._handle_response(resp) Jul 16 21:15:16.630330 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 120, in _handle_response Jul 16 21:15:16.630330 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.from_response(resp, resp.content) Jul 16 21:15:16.630330 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi glanceclient.exc.HTTPNotFound: HTTP 404 Not Found: Failed to find image 2c1a2d49-8f34-42f8-a3fa-1e1bbd07bcdj to delete Jul 16 21:15:16.630330 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:15:16.630330 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:15:16.630330 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:15:16.631684 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:15:16.631684 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/images.py", line 104, in delete Jul 16 21:15:16.631684 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi self._image_api.delete(context, id) Jul 16 21:15:16.631684 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 1255, in delete Jul 16 21:15:16.631684 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return session.delete(context, image_id) Jul 16 21:15:16.631684 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 726, in delete Jul 16 21:15:16.631684 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.ImageNotFound(image_id=image_id) Jul 16 21:15:16.631684 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.ImageNotFound: Image 2c1a2d49-8f34-42f8-a3fa-1e1bbd07bcdj could not be found. Jul 16 21:15:16.631684 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:15:16.631684 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:15:16.631684 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:15:16.631684 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:15:16.631684 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:15:16.631684 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:15:16.631684 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:15:16.631684 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:15:16.631684 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Jul 16 21:15:16.631684 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Jul 16 21:15:16.631684 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:15:16.631684 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:15:16.632999 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/images.py", line 107, in delete Jul 16 21:15:16.632999 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=explanation) Jul 16 21:15:16.632999 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Image not found. Jul 16 21:15:16.632999 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:15:16.632999 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:15:16.632999 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:15:16.632999 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:15:16.632999 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:15:16.632999 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:15:16.632999 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:15:16.632999 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:15:16.632999 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Image not found. Jul 16 21:15:16.632999 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:15:16.640963 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-2ca66f57-acea-401b-8afa-1c23c1c92b0c tempest-ImagesDeleteNegativeTestJSON-956360194 tempest-ImagesDeleteNegativeTestJSON-956360194-project] Returning 404 to user: Image not found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:15:16.646059 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-2ca66f57-acea-401b-8afa-1c23c1c92b0c tempest-ImagesDeleteNegativeTestJSON-956360194 tempest-ImagesDeleteNegativeTestJSON-956360194-project] 10.222.0.16 "DELETE /compute/v2.1/images/2c1a2d49-8f34-42f8-a3fa-1e1bbd07bcdj" status: 404 len: 62 microversion: 2.1 time: 0.122960 Jul 16 21:15:16.646881 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 813/1677] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:15:16 2022] DELETE /compute/v2.1/images/2c1a2d49-8f34-42f8-a3fa-1e1bbd07bcdj => generated 62 bytes in 130 msecs (HTTP/1.1 404) 9 headers in 378 bytes (2 switches on core 0) Jul 16 21:15:16.665246 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-02977670-c965-4b7c-92d9-b20bcebe2aa9 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:15:16.668646 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-02977670-c965-4b7c-92d9-b20bcebe2aa9 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] [instance: 0079e567-30df-4cac-a0ad-4379bcbbcf48] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:15:16.699549 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-02977670-c965-4b7c-92d9-b20bcebe2aa9 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:16.701527 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-02977670-c965-4b7c-92d9-b20bcebe2aa9 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:16.704982 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-02977670-c965-4b7c-92d9-b20bcebe2aa9 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:16.708163 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-02977670-c965-4b7c-92d9-b20bcebe2aa9 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:16.772037 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=100 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:15:16.913364 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-de6fcc93-3404-42d7-baf8-680480f22537 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a69f9dee03d43beaafa966216b9e289', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9b769bf8f3ea4e368bf68de1df6538f9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:15:16.922815 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-de6fcc93-3404-42d7-baf8-680480f22537 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] 10.222.0.16 "GET /compute/v2.1/servers/23df6e0a-3637-4f59-93f9-b02dd614b5c3" status: 200 len: 1196 microversion: 2.1 time: 0.901977 Jul 16 21:15:16.933108 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 865/1678] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:15:16 2022] GET /compute/v2.1/servers/23df6e0a-3637-4f59-93f9-b02dd614b5c3 => generated 1196 bytes in 913 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:15:16.942956 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-dae1a80b-dd31-4508-9fa2-3553a56c1892 tempest-ImagesDeleteNegativeTestJSON-956360194 tempest-ImagesDeleteNegativeTestJSON-956360194-project] Calling method '.version_select at 0x7fdd9cc3f5e0>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:15:16.967878 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-dae1a80b-dd31-4508-9fa2-3553a56c1892 tempest-ImagesDeleteNegativeTestJSON-956360194 tempest-ImagesDeleteNegativeTestJSON-956360194-project] HTTP exception thrown: Image not found. Jul 16 21:15:16.973204 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-dae1a80b-dd31-4508-9fa2-3553a56c1892 tempest-ImagesDeleteNegativeTestJSON-956360194 tempest-ImagesDeleteNegativeTestJSON-956360194-project] Request method failure captured: Jul 16 21:15:16.973204 ncdv-master-1 devstack@n-api.service[138680]: request: DELETE /compute/v2.1/images/35bbc231-b33e-4acb-86bc-9c1625a6b0d61 HTTP/1.1 Jul 16 21:15:16.973204 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:15:16.973204 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:15:16.973204 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:15:16.973204 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 0 Jul 16 21:15:16.973204 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:15:16.973204 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:15:16.973204 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:15:16.973204 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:15:16.973204 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0ypiy5pOlkK7-D_CSkb9ZozS5v9QeDR2dv8PgJfbWR-ih6gM6SuerUpWfWvQBucqG0xGxZqRipWwdjblc0QTBpIuu2zp5DuDXrHp-OviNTgIaNRN2NSh3CG24aNSOxPo7UzmLlXHMkUm5fbBx_Pkqs5uXfgT1XjRKDmmwJ_B4hz-LVg Jul 16 21:15:16.973204 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:15:16.973204 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:15:16.973204 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:15:16.973204 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:15:16.973204 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:15:16.973204 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:15:16.973204 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: a01b0a9bea9f4e09bcebcb9bbc0e5678 Jul 16 21:15:16.973204 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-ImagesDeleteNegativeTestJSON-956360194 Jul 16 21:15:16.973204 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:15:16.973204 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:15:16.978259 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/a01b0a9bea9f4e09bcebcb9bbc0e5678"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/a01b0a9bea9f4e09bcebcb9bbc0e5678"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_a01b0a9bea9f4e09bcebcb9bbc0e5678"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/a01b0a9bea9f4e09bcebcb9bbc0e5678"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:15:16.978259 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-ImagesDeleteNegativeTestJSON-956360194 Jul 16 21:15:16.978259 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: a01b0a9bea9f4e09bcebcb9bbc0e5678 Jul 16 21:15:16.978259 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-ImagesDeleteNegativeTestJSON-956360194 Jul 16 21:15:16.978259 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-ImagesDeleteNegativeTestJSON-956360194-project Jul 16 21:15:16.978259 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:15:16.978259 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:15:16.978259 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: cd263c351cf044fabf45e0f4f2c0ce98 Jul 16 21:15:16.978259 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-ImagesDeleteNegativeTestJSON-956360194-project Jul 16 21:15:16.978259 ncdv-master-1 devstack@n-api.service[138680]: method: Jul 16 21:15:16.978259 ncdv-master-1 devstack@n-api.service[138680]: exception: Image not found. Jul 16 21:15:16.978259 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': '35bbc231-b33e-4acb-86bc-9c1625a6b0d61'} Jul 16 21:15:16.978259 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:15:16.979750 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:15:16.979750 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 724, in delete Jul 16 21:15:16.979750 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi self._client.call(context, 2, 'delete', args=(image_id,)) Jul 16 21:15:16.979750 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 191, in call Jul 16 21:15:16.979750 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi result = getattr(controller, method)(*args, **kwargs) Jul 16 21:15:16.979750 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/utils.py", line 670, in inner Jul 16 21:15:16.979750 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return RequestIdProxy(wrapped(*args, **kwargs)) Jul 16 21:15:16.979750 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/v2/images.py", line 386, in delete Jul 16 21:15:16.979750 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi resp, body = self.http_client.delete(url) Jul 16 21:15:16.979750 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/adapter.py", line 410, in delete Jul 16 21:15:16.979750 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return self.request(url, 'DELETE', **kwargs) Jul 16 21:15:16.979750 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 380, in request Jul 16 21:15:16.979750 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return self._handle_response(resp) Jul 16 21:15:16.979750 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 120, in _handle_response Jul 16 21:15:16.979750 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.from_response(resp, resp.content) Jul 16 21:15:16.979750 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi glanceclient.exc.HTTPNotFound: HTTP 404 Not Found: Failed to find image 35bbc231-b33e-4acb-86bc-9c1625a6b0d61 to delete Jul 16 21:15:16.979750 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:15:16.979750 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:15:16.979750 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:15:16.981410 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:15:16.981410 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/images.py", line 104, in delete Jul 16 21:15:16.981410 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi self._image_api.delete(context, id) Jul 16 21:15:16.981410 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 1255, in delete Jul 16 21:15:16.981410 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return session.delete(context, image_id) Jul 16 21:15:16.981410 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 726, in delete Jul 16 21:15:16.981410 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.ImageNotFound(image_id=image_id) Jul 16 21:15:16.981410 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.ImageNotFound: Image 35bbc231-b33e-4acb-86bc-9c1625a6b0d61 could not be found. Jul 16 21:15:16.981410 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:15:16.981410 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:15:16.981410 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:15:16.981410 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:15:16.981410 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:15:16.981410 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:15:16.981410 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:15:16.981410 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:15:16.981410 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Jul 16 21:15:16.981410 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Jul 16 21:15:16.981410 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:15:16.981410 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:15:16.983111 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/images.py", line 107, in delete Jul 16 21:15:16.983111 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=explanation) Jul 16 21:15:16.983111 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Image not found. Jul 16 21:15:16.983111 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:15:16.983111 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:15:16.983111 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:15:16.983111 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:15:16.983111 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:15:16.983111 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:15:16.983111 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:15:16.983111 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:15:16.983111 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Image not found. Jul 16 21:15:16.983111 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:15:16.993502 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-dae1a80b-dd31-4508-9fa2-3553a56c1892 tempest-ImagesDeleteNegativeTestJSON-956360194 tempest-ImagesDeleteNegativeTestJSON-956360194-project] Returning 404 to user: Image not found. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:15:16.994563 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-dae1a80b-dd31-4508-9fa2-3553a56c1892 tempest-ImagesDeleteNegativeTestJSON-956360194 tempest-ImagesDeleteNegativeTestJSON-956360194-project] 10.222.0.16 "DELETE /compute/v2.1/images/35bbc231-b33e-4acb-86bc-9c1625a6b0d61" status: 404 len: 62 microversion: 2.1 time: 0.059698 Jul 16 21:15:16.996822 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 866/1679] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:15:16 2022] DELETE /compute/v2.1/images/35bbc231-b33e-4acb-86bc-9c1625a6b0d61 => generated 62 bytes in 63 msecs (HTTP/1.1 404) 9 headers in 378 bytes (1 switches on core 0) Jul 16 21:15:17.003427 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-05dc8e00-c307-4b45-859f-b3aca8217a6d tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:15:17.014195 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-05dc8e00-c307-4b45-859f-b3aca8217a6d tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] [instance: 84ab7e65-1bf0-4b8b-baea-6d1d4ca13dd7] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:15:17.026052 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-05dc8e00-c307-4b45-859f-b3aca8217a6d tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:17.033507 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-05dc8e00-c307-4b45-859f-b3aca8217a6d tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:17.033507 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-05dc8e00-c307-4b45-859f-b3aca8217a6d tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:17.033507 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-05dc8e00-c307-4b45-859f-b3aca8217a6d tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:17.330894 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-02977670-c965-4b7c-92d9-b20bcebe2aa9 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '53af488f3b0242a797f539284b78d59e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bb1ede0d0b6941cc97278e6c617f9b01', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:15:17.350336 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-02977670-c965-4b7c-92d9-b20bcebe2aa9 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] 10.222.0.16 "GET /compute/v2.1/servers/0079e567-30df-4cac-a0ad-4379bcbbcf48" status: 200 len: 1334 microversion: 2.1 time: 0.699167 Jul 16 21:15:17.352313 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 814/1680] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:15:16 2022] GET /compute/v2.1/servers/0079e567-30df-4cac-a0ad-4379bcbbcf48 => generated 1334 bytes in 705 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:15:17.360656 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-cb5a2aff-80b4-465c-83cc-4fba5c9837be req-25494730-c942-45fb-9c9a-7750163e411a service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "23df6e0a-3637-4f59-93f9-b02dd614b5c3", "tag": "60e499ad-afcb-4fe7-80ef-058f63c4fc21"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:15:17.382093 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-cb5a2aff-80b4-465c-83cc-4fba5c9837be req-25494730-c942-45fb-9c9a-7750163e411a service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:17.382986 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-cb5a2aff-80b4-465c-83cc-4fba5c9837be req-25494730-c942-45fb-9c9a-7750163e411a service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:17.390796 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell0:SELECT=30 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:15:17.405825 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-cb5a2aff-80b4-465c-83cc-4fba5c9837be req-25494730-c942-45fb-9c9a-7750163e411a service nova] Creating event network-changed:60e499ad-afcb-4fe7-80ef-058f63c4fc21 for instance 23df6e0a-3637-4f59-93f9-b02dd614b5c3 on nch2-master-1 Jul 16 21:15:17.412724 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-cb5a2aff-80b4-465c-83cc-4fba5c9837be req-25494730-c942-45fb-9c9a-7750163e411a service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.058510 Jul 16 21:15:17.414237 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 815/1681] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:15:17 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 61 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:15:17.421500 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-b23ee036-744d-464e-849d-82098908691c tempest-ImagesDeleteNegativeTestJSON-956360194 tempest-ImagesDeleteNegativeTestJSON-956360194-project] Calling method '.version_select at 0x7fdda2b7ee50>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:15:17.486623 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-05dc8e00-c307-4b45-859f-b3aca8217a6d tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:15:17.497061 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-b23ee036-744d-464e-849d-82098908691c tempest-ImagesDeleteNegativeTestJSON-956360194 tempest-ImagesDeleteNegativeTestJSON-956360194-project] HTTP exception thrown: Image not found. Jul 16 21:15:17.500430 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-b23ee036-744d-464e-849d-82098908691c tempest-ImagesDeleteNegativeTestJSON-956360194 tempest-ImagesDeleteNegativeTestJSON-956360194-project] Request method failure captured: Jul 16 21:15:17.500430 ncdv-master-1 devstack@n-api.service[138681]: request: DELETE /compute/v2.1/images/tempest-invalid-1590103085 HTTP/1.1 Jul 16 21:15:17.500430 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:15:17.500430 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:15:17.500430 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:15:17.500430 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 21:15:17.500430 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:15:17.500430 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:15:17.500430 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:15:17.500430 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:15:17.500430 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0ypiy5pOlkK7-D_CSkb9ZozS5v9QeDR2dv8PgJfbWR-ih6gM6SuerUpWfWvQBucqG0xGxZqRipWwdjblc0QTBpIuu2zp5DuDXrHp-OviNTgIaNRN2NSh3CG24aNSOxPo7UzmLlXHMkUm5fbBx_Pkqs5uXfgT1XjRKDmmwJ_B4hz-LVg Jul 16 21:15:17.500430 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:15:17.500430 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:15:17.500430 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:15:17.500430 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:15:17.500430 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:15:17.500430 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:15:17.500430 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: a01b0a9bea9f4e09bcebcb9bbc0e5678 Jul 16 21:15:17.500430 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-ImagesDeleteNegativeTestJSON-956360194 Jul 16 21:15:17.500430 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:15:17.500430 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:15:17.502192 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/a01b0a9bea9f4e09bcebcb9bbc0e5678"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/a01b0a9bea9f4e09bcebcb9bbc0e5678"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_a01b0a9bea9f4e09bcebcb9bbc0e5678"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/a01b0a9bea9f4e09bcebcb9bbc0e5678"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:15:17.502192 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-ImagesDeleteNegativeTestJSON-956360194 Jul 16 21:15:17.502192 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: a01b0a9bea9f4e09bcebcb9bbc0e5678 Jul 16 21:15:17.502192 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-ImagesDeleteNegativeTestJSON-956360194 Jul 16 21:15:17.502192 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-ImagesDeleteNegativeTestJSON-956360194-project Jul 16 21:15:17.502192 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:15:17.502192 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:15:17.502192 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: cd263c351cf044fabf45e0f4f2c0ce98 Jul 16 21:15:17.502192 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-ImagesDeleteNegativeTestJSON-956360194-project Jul 16 21:15:17.502192 ncdv-master-1 devstack@n-api.service[138681]: method: Jul 16 21:15:17.502192 ncdv-master-1 devstack@n-api.service[138681]: exception: Image not found. Jul 16 21:15:17.502192 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': 'tempest-invalid-1590103085'} Jul 16 21:15:17.502192 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:15:17.503430 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:15:17.503430 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 724, in delete Jul 16 21:15:17.503430 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi self._client.call(context, 2, 'delete', args=(image_id,)) Jul 16 21:15:17.503430 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 191, in call Jul 16 21:15:17.503430 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi result = getattr(controller, method)(*args, **kwargs) Jul 16 21:15:17.503430 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/utils.py", line 670, in inner Jul 16 21:15:17.503430 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return RequestIdProxy(wrapped(*args, **kwargs)) Jul 16 21:15:17.503430 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/v2/images.py", line 386, in delete Jul 16 21:15:17.503430 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi resp, body = self.http_client.delete(url) Jul 16 21:15:17.503430 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/adapter.py", line 410, in delete Jul 16 21:15:17.503430 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return self.request(url, 'DELETE', **kwargs) Jul 16 21:15:17.503430 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 380, in request Jul 16 21:15:17.503430 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return self._handle_response(resp) Jul 16 21:15:17.503430 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 120, in _handle_response Jul 16 21:15:17.503430 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.from_response(resp, resp.content) Jul 16 21:15:17.503430 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi glanceclient.exc.HTTPNotFound: HTTP 404 Not Found: Failed to find image tempest-invalid-1590103085 to delete Jul 16 21:15:17.503430 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:15:17.503430 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:15:17.503430 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:15:17.506786 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:15:17.506786 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/images.py", line 104, in delete Jul 16 21:15:17.506786 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi self._image_api.delete(context, id) Jul 16 21:15:17.506786 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 1255, in delete Jul 16 21:15:17.506786 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return session.delete(context, image_id) Jul 16 21:15:17.506786 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 726, in delete Jul 16 21:15:17.506786 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.ImageNotFound(image_id=image_id) Jul 16 21:15:17.506786 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.ImageNotFound: Image tempest-invalid-1590103085 could not be found. Jul 16 21:15:17.506786 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:15:17.506786 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:15:17.506786 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:15:17.506786 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:15:17.506786 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:15:17.506786 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:15:17.506786 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:15:17.506786 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:15:17.506786 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Jul 16 21:15:17.506786 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Jul 16 21:15:17.506786 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:15:17.506786 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:15:17.509293 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/images.py", line 107, in delete Jul 16 21:15:17.509293 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=explanation) Jul 16 21:15:17.509293 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Image not found. Jul 16 21:15:17.509293 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:15:17.509293 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:15:17.509293 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:15:17.509293 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:15:17.509293 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:15:17.509293 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:15:17.509293 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:15:17.509293 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:15:17.509293 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Image not found. Jul 16 21:15:17.509293 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:15:17.509293 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-b23ee036-744d-464e-849d-82098908691c tempest-ImagesDeleteNegativeTestJSON-956360194 tempest-ImagesDeleteNegativeTestJSON-956360194-project] Returning 404 to user: Image not found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:15:17.511949 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-05dc8e00-c307-4b45-859f-b3aca8217a6d tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:15:17.512108 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-b23ee036-744d-464e-849d-82098908691c tempest-ImagesDeleteNegativeTestJSON-956360194 tempest-ImagesDeleteNegativeTestJSON-956360194-project] 10.222.0.16 "DELETE /compute/v2.1/images/tempest-invalid-1590103085" status: 404 len: 62 microversion: 2.1 time: 0.089832 Jul 16 21:15:17.513395 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-05dc8e00-c307-4b45-859f-b3aca8217a6d tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:15:17.513395 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-05dc8e00-c307-4b45-859f-b3aca8217a6d tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] 10.222.0.16 "GET /compute/v2.1/servers/84ab7e65-1bf0-4b8b-baea-6d1d4ca13dd7" status: 200 len: 1281 microversion: 2.25 time: 0.511029 Jul 16 21:15:17.514460 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 816/1682] 10.222.0.16 () {58 vars in 1205 bytes} [Sat Jul 16 21:15:17 2022] DELETE /compute/v2.1/images/tempest-invalid-1590103085 => generated 62 bytes in 99 msecs (HTTP/1.1 404) 9 headers in 378 bytes (1 switches on core 0) Jul 16 21:15:17.526776 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 867/1683] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:15:16 2022] GET /compute/v2.1/servers/84ab7e65-1bf0-4b8b-baea-6d1d4ca13dd7 => generated 1281 bytes in 529 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:15:17.538203 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-b26e5b67-2b2f-47a8-a3d5-1ac58779d377 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:15:17.540026 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-b26e5b67-2b2f-47a8-a3d5-1ac58779d377 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] [instance: 39e51d01-21e0-4dd5-bfd4-16b0f8b0f827] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:15:17.559160 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b26e5b67-2b2f-47a8-a3d5-1ac58779d377 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:17.559623 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b26e5b67-2b2f-47a8-a3d5-1ac58779d377 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:17.560411 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b26e5b67-2b2f-47a8-a3d5-1ac58779d377 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:17.560866 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b26e5b67-2b2f-47a8-a3d5-1ac58779d377 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:17.807210 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-2d96d64b-2a9a-4b95-8fe6-88fe6c4c9b71 tempest-AbsoluteLimitsNegativeTestJSON-1995367712 tempest-AbsoluteLimitsNegativeTestJSON-1995367712-project] Calling method '.version_select at 0x7fdda2b9c9d0>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:15:17.845965 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2d96d64b-2a9a-4b95-8fe6-88fe6c4c9b71 tempest-AbsoluteLimitsNegativeTestJSON-1995367712 tempest-AbsoluteLimitsNegativeTestJSON-1995367712-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:17.846425 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2d96d64b-2a9a-4b95-8fe6-88fe6c4c9b71 tempest-AbsoluteLimitsNegativeTestJSON-1995367712 tempest-AbsoluteLimitsNegativeTestJSON-1995367712-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:17.863897 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2d96d64b-2a9a-4b95-8fe6-88fe6c4c9b71 tempest-AbsoluteLimitsNegativeTestJSON-1995367712 tempest-AbsoluteLimitsNegativeTestJSON-1995367712-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:17.863897 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2d96d64b-2a9a-4b95-8fe6-88fe6c4c9b71 tempest-AbsoluteLimitsNegativeTestJSON-1995367712 tempest-AbsoluteLimitsNegativeTestJSON-1995367712-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.013s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:17.903378 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-2d96d64b-2a9a-4b95-8fe6-88fe6c4c9b71 tempest-AbsoluteLimitsNegativeTestJSON-1995367712 tempest-AbsoluteLimitsNegativeTestJSON-1995367712-project] 10.222.0.16 "GET /compute/v2.1/limits" status: 200 len: 511 microversion: 2.1 time: 0.388031 Jul 16 21:15:17.903975 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 817/1684] 10.222.0.16 () {58 vars in 1121 bytes} [Sat Jul 16 21:15:17 2022] GET /compute/v2.1/limits => generated 511 bytes in 389 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:15:17.929203 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-73fd5de4-dccf-49d5-a0ef-f49f708049af tempest-ImagesDeleteNegativeTestJSON-956360194 tempest-ImagesDeleteNegativeTestJSON-956360194-project] Calling method '.version_select at 0x7fdda2b7e160>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:15:17.969338 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-73fd5de4-dccf-49d5-a0ef-f49f708049af tempest-ImagesDeleteNegativeTestJSON-956360194 tempest-ImagesDeleteNegativeTestJSON-956360194-project] HTTP exception thrown: Image not found. Jul 16 21:15:17.990041 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-73fd5de4-dccf-49d5-a0ef-f49f708049af tempest-ImagesDeleteNegativeTestJSON-956360194 tempest-ImagesDeleteNegativeTestJSON-956360194-project] Request method failure captured: Jul 16 21:15:17.990041 ncdv-master-1 devstack@n-api.service[138681]: request: DELETE /compute/v2.1/images/ad5525ff-b5d9-408a-be30-46e10ce26bf6 HTTP/1.1 Jul 16 21:15:17.990041 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:15:17.990041 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:15:17.990041 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:15:17.990041 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 21:15:17.990041 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:15:17.990041 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:15:17.990041 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:15:17.990041 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:15:17.990041 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0ypiy5pOlkK7-D_CSkb9ZozS5v9QeDR2dv8PgJfbWR-ih6gM6SuerUpWfWvQBucqG0xGxZqRipWwdjblc0QTBpIuu2zp5DuDXrHp-OviNTgIaNRN2NSh3CG24aNSOxPo7UzmLlXHMkUm5fbBx_Pkqs5uXfgT1XjRKDmmwJ_B4hz-LVg Jul 16 21:15:17.990041 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:15:17.990041 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:15:17.990041 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:15:17.990041 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:15:17.990041 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:15:17.990041 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:15:17.990041 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: a01b0a9bea9f4e09bcebcb9bbc0e5678 Jul 16 21:15:17.990041 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-ImagesDeleteNegativeTestJSON-956360194 Jul 16 21:15:17.990041 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:15:17.990041 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:15:17.991896 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/a01b0a9bea9f4e09bcebcb9bbc0e5678"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/a01b0a9bea9f4e09bcebcb9bbc0e5678"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_a01b0a9bea9f4e09bcebcb9bbc0e5678"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/a01b0a9bea9f4e09bcebcb9bbc0e5678"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:15:17.991896 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-ImagesDeleteNegativeTestJSON-956360194 Jul 16 21:15:17.991896 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: a01b0a9bea9f4e09bcebcb9bbc0e5678 Jul 16 21:15:17.991896 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-ImagesDeleteNegativeTestJSON-956360194 Jul 16 21:15:17.991896 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-ImagesDeleteNegativeTestJSON-956360194-project Jul 16 21:15:17.991896 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:15:17.991896 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:15:17.991896 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: cd263c351cf044fabf45e0f4f2c0ce98 Jul 16 21:15:17.991896 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-ImagesDeleteNegativeTestJSON-956360194-project Jul 16 21:15:17.991896 ncdv-master-1 devstack@n-api.service[138681]: method: Jul 16 21:15:17.991896 ncdv-master-1 devstack@n-api.service[138681]: exception: Image not found. Jul 16 21:15:17.991896 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': 'ad5525ff-b5d9-408a-be30-46e10ce26bf6'} Jul 16 21:15:17.991896 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:15:17.993172 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:15:17.993172 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 724, in delete Jul 16 21:15:17.993172 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi self._client.call(context, 2, 'delete', args=(image_id,)) Jul 16 21:15:17.993172 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 191, in call Jul 16 21:15:17.993172 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi result = getattr(controller, method)(*args, **kwargs) Jul 16 21:15:17.993172 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/utils.py", line 670, in inner Jul 16 21:15:17.993172 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return RequestIdProxy(wrapped(*args, **kwargs)) Jul 16 21:15:17.993172 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/v2/images.py", line 386, in delete Jul 16 21:15:17.993172 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi resp, body = self.http_client.delete(url) Jul 16 21:15:17.993172 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/adapter.py", line 410, in delete Jul 16 21:15:17.993172 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return self.request(url, 'DELETE', **kwargs) Jul 16 21:15:17.993172 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 380, in request Jul 16 21:15:17.993172 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return self._handle_response(resp) Jul 16 21:15:17.993172 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 120, in _handle_response Jul 16 21:15:17.993172 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.from_response(resp, resp.content) Jul 16 21:15:17.993172 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi glanceclient.exc.HTTPNotFound: HTTP 404 Not Found: Failed to find image ad5525ff-b5d9-408a-be30-46e10ce26bf6 to delete Jul 16 21:15:17.993172 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:15:17.993172 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:15:17.993172 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:15:17.994946 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:15:17.994946 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/images.py", line 104, in delete Jul 16 21:15:17.994946 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi self._image_api.delete(context, id) Jul 16 21:15:17.994946 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 1255, in delete Jul 16 21:15:17.994946 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return session.delete(context, image_id) Jul 16 21:15:17.994946 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 726, in delete Jul 16 21:15:17.994946 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.ImageNotFound(image_id=image_id) Jul 16 21:15:17.994946 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.ImageNotFound: Image ad5525ff-b5d9-408a-be30-46e10ce26bf6 could not be found. Jul 16 21:15:17.994946 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:15:17.994946 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:15:17.994946 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:15:17.994946 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:15:17.994946 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:15:17.994946 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:15:17.994946 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:15:17.994946 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:15:17.994946 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Jul 16 21:15:17.994946 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Jul 16 21:15:17.994946 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:15:17.994946 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:15:17.996569 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/images.py", line 107, in delete Jul 16 21:15:17.996569 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=explanation) Jul 16 21:15:17.996569 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Image not found. Jul 16 21:15:17.996569 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:15:17.996569 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:15:17.996569 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:15:17.996569 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:15:17.996569 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:15:17.996569 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:15:17.996569 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:15:17.996569 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:15:17.996569 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Image not found. Jul 16 21:15:17.996569 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:15:17.999182 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-73fd5de4-dccf-49d5-a0ef-f49f708049af tempest-ImagesDeleteNegativeTestJSON-956360194 tempest-ImagesDeleteNegativeTestJSON-956360194-project] Returning 404 to user: Image not found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:15:17.999182 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-73fd5de4-dccf-49d5-a0ef-f49f708049af tempest-ImagesDeleteNegativeTestJSON-956360194 tempest-ImagesDeleteNegativeTestJSON-956360194-project] 10.222.0.16 "DELETE /compute/v2.1/images/ad5525ff-b5d9-408a-be30-46e10ce26bf6" status: 404 len: 62 microversion: 2.1 time: 0.093531 Jul 16 21:15:18.015140 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 818/1685] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:15:17 2022] DELETE /compute/v2.1/images/ad5525ff-b5d9-408a-be30-46e10ce26bf6 => generated 62 bytes in 101 msecs (HTTP/1.1 404) 9 headers in 378 bytes (1 switches on core 0) Jul 16 21:15:18.015140 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-ad3620a8-c939-439e-9b37-c2566f4d5fba tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:15:18.015140 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-ad3620a8-c939-439e-9b37-c2566f4d5fba tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] [instance: 23df6e0a-3637-4f59-93f9-b02dd614b5c3] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:15:18.029849 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ad3620a8-c939-439e-9b37-c2566f4d5fba tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:18.030967 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ad3620a8-c939-439e-9b37-c2566f4d5fba tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:18.031894 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ad3620a8-c939-439e-9b37-c2566f4d5fba tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:18.032688 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ad3620a8-c939-439e-9b37-c2566f4d5fba tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:18.453631 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-b26e5b67-2b2f-47a8-a3d5-1ac58779d377 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4e9df82f95ac4779859972403869096a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4337575622d04a26942905c9dc044ba7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:15:18.465843 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-b26e5b67-2b2f-47a8-a3d5-1ac58779d377 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] 10.222.0.16 "GET /compute/v2.1/servers/39e51d01-21e0-4dd5-bfd4-16b0f8b0f827" status: 200 len: 1325 microversion: 2.1 time: 0.936090 Jul 16 21:15:18.467458 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 868/1686] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:15:17 2022] GET /compute/v2.1/servers/39e51d01-21e0-4dd5-bfd4-16b0f8b0f827 => generated 1325 bytes in 940 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:15:18.475071 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-ef4b62d7-582c-47ad-995c-7f6d73a7f299 tempest-AbsoluteLimitsNegativeTestJSON-1995367712 tempest-AbsoluteLimitsNegativeTestJSON-1995367712-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AbsoluteLimitsNegativeTestJSON-server-930810617", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452", "metadata": {"0": "0", "1": "1", "2": "2", "3": "3", "4": "4", "5": "5", "6": "6", "7": "7", "8": "8", "9": "9", "10": "10", "11": "11", "12": "12", "13": "13", "14": "14", "15": "15", "16": "16", "17": "17", "18": "18", "19": "19", "20": "20", "21": "21", "22": "22", "23": "23", "24": "24", "25": "25", "26": "26", "27": "27", "28": "28", "29": "29", "30": "30", "31": "31", "32": "32", "33": "33", "34": "34", "35": "35", "36": "36", "37": "37", "38": "38", "39": "39", "40": "40", "41": "41", "42": "42", "43": "43", "44": "44", "45": "45", "46": "46", "47": "47", "48": "48", "49": "49", "50": "50", "51": "51", "52": "52", "53": "53", "54": "54", "55": "55", "56": "56", "57": "57", "58": "58", "59": "59", "60": "60", "61": "61", "62": "62", "63": "63", "64": "64", "65": "65", "66": "66", "67": "67", "68": "68", "69": "69", "70": "70", "71": "71", "72": "72", "73": "73", "74": "74", "75": "75", "76": "76", "77": "77", "78": "78", "79": "79", "80": "80", "81": "81", "82": "82", "83": "83", "84": "84", "85": "85", "86": "86", "87": "87", "88": "88", "89": "89", "90": "90", "91": "91", "92": "92", "93": "93", "94": "94", "95": "95", "96": "96", "97": "97", "98": "98", "99": "99", "100": "100", "101": "101", "102": "102", "103": "103", "104": "104", "105": "105", "106": "106", "107": "107", "108": "108", "109": "109", "110": "110", "111": "111", "112": "112", "113": "113", "114": "114", "115": "115", "116": "116", "117": "117", "118": "118", "119": "119", "120": "120", "121": "121", "122": "122", "123": "123", "124": "124", "125": "125", "126": "126", "127": "127", "128": "128"}}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:15:18.485722 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-ad3620a8-c939-439e-9b37-c2566f4d5fba tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a69f9dee03d43beaafa966216b9e289', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9b769bf8f3ea4e368bf68de1df6538f9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:15:18.518674 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-ad3620a8-c939-439e-9b37-c2566f4d5fba tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] 10.222.0.16 "GET /compute/v2.1/servers/23df6e0a-3637-4f59-93f9-b02dd614b5c3" status: 200 len: 1196 microversion: 2.1 time: 0.509327 Jul 16 21:15:18.524801 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 819/1687] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:15:18 2022] GET /compute/v2.1/servers/23df6e0a-3637-4f59-93f9-b02dd614b5c3 => generated 1196 bytes in 518 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:15:18.542878 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-2f732b2f-10b8-404c-be67-28af2dac4364 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:15:18.542878 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-2f732b2f-10b8-404c-be67-28af2dac4364 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] [instance: 0079e567-30df-4cac-a0ad-4379bcbbcf48] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:15:18.549867 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2f732b2f-10b8-404c-be67-28af2dac4364 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:18.550831 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2f732b2f-10b8-404c-be67-28af2dac4364 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:18.553768 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2f732b2f-10b8-404c-be67-28af2dac4364 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:18.554537 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2f732b2f-10b8-404c-be67-28af2dac4364 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:18.593636 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.network.neutron [None req-ef4b62d7-582c-47ad-995c-7f6d73a7f299 tempest-AbsoluteLimitsNegativeTestJSON-1995367712 tempest-AbsoluteLimitsNegativeTestJSON-1995367712-project] validate_networks() for None {{(pid=138680) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 16 21:15:19.576972 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-2f732b2f-10b8-404c-be67-28af2dac4364 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '53af488f3b0242a797f539284b78d59e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bb1ede0d0b6941cc97278e6c617f9b01', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:15:19.585404 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-2f732b2f-10b8-404c-be67-28af2dac4364 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] 10.222.0.16 "GET /compute/v2.1/servers/0079e567-30df-4cac-a0ad-4379bcbbcf48" status: 200 len: 1334 microversion: 2.1 time: 1.059176 Jul 16 21:15:19.586210 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 820/1688] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:15:18 2022] GET /compute/v2.1/servers/0079e567-30df-4cac-a0ad-4379bcbbcf48 => generated 1334 bytes in 1061 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:15:19.600394 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-3802e2f5-cd22-46f8-8286-34ae38d26008 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:15:19.601010 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-3802e2f5-cd22-46f8-8286-34ae38d26008 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] [instance: 84ab7e65-1bf0-4b8b-baea-6d1d4ca13dd7] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:15:19.607798 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3802e2f5-cd22-46f8-8286-34ae38d26008 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:19.608396 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3802e2f5-cd22-46f8-8286-34ae38d26008 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:19.609436 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3802e2f5-cd22-46f8-8286-34ae38d26008 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:19.609879 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3802e2f5-cd22-46f8-8286-34ae38d26008 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:19.844296 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-ef4b62d7-582c-47ad-995c-7f6d73a7f299 tempest-AbsoluteLimitsNegativeTestJSON-1995367712 tempest-AbsoluteLimitsNegativeTestJSON-1995367712-project] Getting quotas for project 160fd40061294ac49b70ecaba1be9432. Resources: dict_keys(['metadata_items']) {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:15:19.863763 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-ef4b62d7-582c-47ad-995c-7f6d73a7f299 tempest-AbsoluteLimitsNegativeTestJSON-1995367712 tempest-AbsoluteLimitsNegativeTestJSON-1995367712-project] Getting quotas for user bb2272990663498a8de9e5af940b0b54 and project 160fd40061294ac49b70ecaba1be9432. Resources: dict_keys(['metadata_items']) {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:15:19.914390 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-ef4b62d7-582c-47ad-995c-7f6d73a7f299 tempest-AbsoluteLimitsNegativeTestJSON-1995367712 tempest-AbsoluteLimitsNegativeTestJSON-1995367712-project] HTTP exception thrown: Maximum number of metadata items exceeds 128 Jul 16 21:15:19.917491 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-ef4b62d7-582c-47ad-995c-7f6d73a7f299 tempest-AbsoluteLimitsNegativeTestJSON-1995367712 tempest-AbsoluteLimitsNegativeTestJSON-1995367712-project] Request method failure captured: Jul 16 21:15:19.917491 ncdv-master-1 devstack@n-api.service[138680]: request: POST /compute/v2.1/servers HTTP/1.1 Jul 16 21:15:19.917491 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:15:19.917491 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:15:19.917491 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:15:19.917491 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 1751 Jul 16 21:15:19.917491 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:15:19.917491 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:15:19.917491 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:15:19.917491 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:15:19.917491 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0ypkNB0b4tf0gwY4s-LoJEoxVLAIgS6EWGTC5blN1l1URK_5ywDIBhcrXpfsrx1xbdk6QNMZtoPrvAPPznMrDnWtsXnx9kKOv-0m78ahn2jH5hSjw2ftQz0NiRqlmUjZuFfxaNJa4GRJKU1eb05Nqagc35HaptUPHUOROBnMX5u50ME Jul 16 21:15:19.917491 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:15:19.917491 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:15:19.917491 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:15:19.917491 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:15:19.917491 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:15:19.917491 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:15:19.917491 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: 160fd40061294ac49b70ecaba1be9432 Jul 16 21:15:19.917491 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-AbsoluteLimitsNegativeTestJSON-1995367712 Jul 16 21:15:19.917491 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:15:19.917491 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:15:19.917491 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/160fd40061294ac49b70ecaba1be9432"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/160fd40061294ac49b70ecaba1be9432"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_160fd40061294ac49b70ecaba1be9432"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/160fd40061294ac49b70ecaba1be9432"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:15:19.917491 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-AbsoluteLimitsNegativeTestJSON-1995367712 Jul 16 21:15:19.917491 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: 160fd40061294ac49b70ecaba1be9432 Jul 16 21:15:19.917491 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-AbsoluteLimitsNegativeTestJSON-1995367712 Jul 16 21:15:19.917491 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-AbsoluteLimitsNegativeTestJSON-1995367712-project Jul 16 21:15:19.917491 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:15:19.917491 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:15:19.917491 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: bb2272990663498a8de9e5af940b0b54 Jul 16 21:15:19.917491 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-AbsoluteLimitsNegativeTestJSON-1995367712-project Jul 16 21:15:19.917491 ncdv-master-1 devstack@n-api.service[138680]: Jul 16 21:15:19.917491 ncdv-master-1 devstack@n-api.service[138680]: {"server": {"name": "tempest-AbsoluteLimitsNegativeTestJSON-server-930810617", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452", "metadata": {"0": "0", "1": "1", "2": "2", "3": "3", "4": "4", "5": "5", "6": "6", "7": "7", "8": "8", "9": "9", "10": "10", "11": "11", "12": "12", "13": "13", "14": "14", "15": "15", "16": "16", "17": "17", "18": "18", "19": "19", "20": "20", "21": "21", "22": "22", "23": "23", "24": "24", "25": "25", "26": "26", "27": "27", "28": "28", "29": "29", "30": "30", "31": "31", "32": "32", "33": "33", "34": "34", "35": "35", "36": "36", "37": "37", "38": "38", "39": "39", "40": "40", "41": "41", "42": "42", "43": "43", "44": "44", "45": "45", "46": "46", "47": "47", "48": "48", "49": "49", "50": "50", "51": "51", "52": "52", "53": "53", "54": "54", "55": "55", "56": "56", "57": "57", "58": "58", "59": "59", "60": "60", "61": "61", "62": "62", "63": "63", "64": "64", "65": "65", "66": "66", "67": "67", "68": "68", "69": "69", "70": "70", "71": "71", "72": "72", "73": "73", "74": "74", "75": "75", "76": "76", "77": "77", "78": "78", "79": "79", "80": "80", "81": "81", "82": "82", "83": "83", "84": "84", "85": "85", "86": "86", "87": "87", "88": "88", "89": "89", "90": "90", "91": "91", "92": "92", "93": "93", "94": "94", "95": "95", "96": "96", "97": "97", "98": "98", "99": "99", "100": "100", "101": "101", "102": "102", "103": "103", "104": "104", "105": "105", "106": "106", "107": "107", "108": "108", "109": "109", "110": "110", "111": "111", "112": "112", "113": "113", "114": "114", "115": "115", "116": "116", "117": "117", "118": "118", "119": "119", "120": "120", "121": "121", "122": "122", "123": "123", "124": "124", "125": "125", "126": "126", "127": "127", "128": "128"}}} Jul 16 21:15:19.917491 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 21:15:19.917491 ncdv-master-1 devstack@n-api.service[138680]: exception: Maximum number of metadata items exceeds 128 Jul 16 21:15:19.920430 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'body': {'server': {'name': 'tempest-AbsoluteLimitsNegativeTestJSON-server-930810617', 'imageRef': '1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9', 'flavorRef': '452', 'metadata': {'0': '0', '1': '1', '2': '2', '3': '3', '4': '4', '5': '5', '6': '6', '7': '7', '8': '8', '9': '9', '10': '10', '11': '11', '12': '12', '13': '13', '14': '14', '15': '15', '16': '16', '17': '17', '18': '18', '19': '19', '20': '20', '21': '21', '22': '22', '23': '23', '24': '24', '25': '25', '26': '26', '27': '27', '28': '28', '29': '29', '30': '30', '31': '31', '32': '32', '33': '33', '34': '34', '35': '35', '36': '36', '37': '37', '38': '38', '39': '39', '40': '40', '41': '41', '42': '42', '43': '43', '44': '44', '45': '45', '46': '46', '47': '47', '48': '48', '49': '49', '50': '50', '51': '51', '52': '52', '53': '53', '54': '54', '55': '55', '56': '56', '57': '57', '58': '58', '59': '59', '60': '60', '61': '61', '62': '62', '63': '63', '64': '64', '65': '65', '66': '66', '67': '67', '68': '68', '69': '69', '70': '70', '71': '71', '72': '72', '73': '73', '74': '74', '75': '75', '76': '76', '77': '77', '78': '78', '79': '79', '80': '80', '81': '81', '82': '82', '83': '83', '84': '84', '85': '85', '86': '86', '87': '87', '88': '88', '89': '89', '90': '90', '91': '91', '92': '92', '93': '93', '94': '94', '95': '95', '96': '96', '97': '97', '98': '98', '99': '99', '100': '100', '101': '101', '102': '102', '103': '103', '104': '104', '105': '105', '106': '106', '107': '107', '108': '108', '109': '109', '110': '110', '111': '111', '112': '112', '113': '113', '114': '114', '115': '115', '116': '116', '117': '117', '118': '118', '119': '119', '120': '120', '121': '121', '122': '122', '123': '123', '124': '124', '125': '125', '126': '126', '127': '127', '128': '128'}}}} Jul 16 21:15:19.920430 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:15:19.920430 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:15:19.920430 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 462, in _check_metadata_properties_quota Jul 16 21:15:19.920430 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi objects.Quotas.limit_check(context, metadata_items=num_metadata) Jul 16 21:15:19.920430 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Jul 16 21:15:19.920430 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Jul 16 21:15:19.920430 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/quotas.py", line 290, in limit_check Jul 16 21:15:19.920430 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return quota.QUOTAS.limit_check( Jul 16 21:15:19.920430 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/quota.py", line 1118, in limit_check Jul 16 21:15:19.920430 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return self._driver.limit_check(context, self._resources, values, Jul 16 21:15:19.920430 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/quota.py", line 468, in limit_check Jul 16 21:15:19.920430 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.OverQuota(overs=sorted(overs), quotas=quotas, Jul 16 21:15:19.920430 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.OverQuota: Quota exceeded for resources: ['metadata_items'] Jul 16 21:15:19.920430 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:15:19.920430 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:15:19.920430 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:15:19.920430 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:15:19.920430 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 784, in create Jul 16 21:15:19.920430 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instances, resv_id = self.compute_api.create( Jul 16 21:15:19.920430 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2149, in create Jul 16 21:15:19.920430 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return self._create_instance( Jul 16 21:15:19.920430 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 1667, in _create_instance Jul 16 21:15:19.920430 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi self._checks_for_create_and_rebuild(context, image_id, boot_meta, Jul 16 21:15:19.920430 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 1002, in _checks_for_create_and_rebuild Jul 16 21:15:19.920430 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi self._check_metadata_properties_quota(context, metadata) Jul 16 21:15:19.920430 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 469, in _check_metadata_properties_quota Jul 16 21:15:19.920430 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.MetadataLimitExceeded(allowed=quota_metadata) Jul 16 21:15:19.920430 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.MetadataLimitExceeded: Maximum number of metadata items exceeds 128 Jul 16 21:15:19.920430 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:15:19.920430 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:15:19.920430 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:15:19.920430 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:15:19.920430 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:15:19.920430 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:15:19.920430 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:15:19.920430 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:15:19.920430 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:15:19.920430 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:15:19.920430 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 16 21:15:19.920430 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 16 21:15:19.920430 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 16 21:15:19.920430 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 16 21:15:19.920430 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 16 21:15:19.924778 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 16 21:15:19.924778 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi [Previous line repeated 10 more times] Jul 16 21:15:19.924778 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 800, in create Jul 16 21:15:19.924778 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPForbidden( Jul 16 21:15:19.924778 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPForbidden: Maximum number of metadata items exceeds 128 Jul 16 21:15:19.924778 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:15:19.924778 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:15:19.924778 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:15:19.924778 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:15:19.924778 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:15:19.924778 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:15:19.924778 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:15:19.924778 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:15:19.924778 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Maximum number of metadata items exceeds 128 Jul 16 21:15:19.924778 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:15:19.927973 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-ef4b62d7-582c-47ad-995c-7f6d73a7f299 tempest-AbsoluteLimitsNegativeTestJSON-1995367712 tempest-AbsoluteLimitsNegativeTestJSON-1995367712-project] Returning 403 to user: Maximum number of metadata items exceeds 128 {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:15:19.927973 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-ef4b62d7-582c-47ad-995c-7f6d73a7f299 tempest-AbsoluteLimitsNegativeTestJSON-1995367712 tempest-AbsoluteLimitsNegativeTestJSON-1995367712-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 403 len: 87 microversion: 2.1 time: 1.458279 Jul 16 21:15:19.928844 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 869/1689] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Jul 16 21:15:18 2022] POST /compute/v2.1/servers => generated 87 bytes in 1460 msecs (HTTP/1.1 403) 9 headers in 378 bytes (1 switches on core 0) Jul 16 21:15:20.409115 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-3802e2f5-cd22-46f8-8286-34ae38d26008 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:15:20.421992 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-3802e2f5-cd22-46f8-8286-34ae38d26008 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:15:20.428822 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-3802e2f5-cd22-46f8-8286-34ae38d26008 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:15:20.430335 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-3802e2f5-cd22-46f8-8286-34ae38d26008 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] 10.222.0.16 "GET /compute/v2.1/servers/84ab7e65-1bf0-4b8b-baea-6d1d4ca13dd7" status: 200 len: 1281 microversion: 2.25 time: 0.834313 Jul 16 21:15:20.433876 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 821/1690] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:15:19 2022] GET /compute/v2.1/servers/84ab7e65-1bf0-4b8b-baea-6d1d4ca13dd7 => generated 1281 bytes in 847 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:15:20.454723 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-9c8b8d72-af43-47d7-8bf6-0f7723099d3c req-c80c6ee5-7755-4c82-a0bb-75426ce5ffa2 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "0079e567-30df-4cac-a0ad-4379bcbbcf48", "tag": "08381e59-62d7-471e-a05a-7de705ec2dae"}, {"name": "network-changed", "server_uuid": "84ab7e65-1bf0-4b8b-baea-6d1d4ca13dd7", "tag": "42dfc89b-90fd-4214-983c-f02029faca17"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:15:20.476140 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-9c8b8d72-af43-47d7-8bf6-0f7723099d3c req-c80c6ee5-7755-4c82-a0bb-75426ce5ffa2 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:20.479889 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-9c8b8d72-af43-47d7-8bf6-0f7723099d3c req-c80c6ee5-7755-4c82-a0bb-75426ce5ffa2 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:20.507525 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-9c8b8d72-af43-47d7-8bf6-0f7723099d3c req-c80c6ee5-7755-4c82-a0bb-75426ce5ffa2 service nova] Creating event network-changed:08381e59-62d7-471e-a05a-7de705ec2dae for instance 0079e567-30df-4cac-a0ad-4379bcbbcf48 on nch2-master-1 Jul 16 21:15:20.508333 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-9c8b8d72-af43-47d7-8bf6-0f7723099d3c req-c80c6ee5-7755-4c82-a0bb-75426ce5ffa2 service nova] Creating event network-changed:42dfc89b-90fd-4214-983c-f02029faca17 for instance 84ab7e65-1bf0-4b8b-baea-6d1d4ca13dd7 on nch1-master-1 Jul 16 21:15:20.520783 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-9c8b8d72-af43-47d7-8bf6-0f7723099d3c req-c80c6ee5-7755-4c82-a0bb-75426ce5ffa2 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 346 microversion: 2.1 time: 0.084822 Jul 16 21:15:20.521268 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 822/1691] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:15:20 2022] POST /compute/v2.1/os-server-external-events => generated 346 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:15:20.560784 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-8de026b4-7b8e-4404-87e7-1cd0c1189da5 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:15:20.562255 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-8de026b4-7b8e-4404-87e7-1cd0c1189da5 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] [instance: 39e51d01-21e0-4dd5-bfd4-16b0f8b0f827] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:15:20.570878 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8de026b4-7b8e-4404-87e7-1cd0c1189da5 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:20.571948 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8de026b4-7b8e-4404-87e7-1cd0c1189da5 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:20.573445 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8de026b4-7b8e-4404-87e7-1cd0c1189da5 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:20.574303 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8de026b4-7b8e-4404-87e7-1cd0c1189da5 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:21.038877 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-647c8fbc-ec6c-4420-90ca-1c9e9f79a102 tempest-FloatingIPDetailsTestJSON-1149902298 tempest-FloatingIPDetailsTestJSON-1149902298-project] Action: 'create', calling method: .version_select at 0x7fdda2bc18b0>, body: {"pool": null} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:15:21.651381 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-8de026b4-7b8e-4404-87e7-1cd0c1189da5 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4e9df82f95ac4779859972403869096a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4337575622d04a26942905c9dc044ba7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:15:21.659547 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-8de026b4-7b8e-4404-87e7-1cd0c1189da5 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] 10.222.0.16 "GET /compute/v2.1/servers/39e51d01-21e0-4dd5-bfd4-16b0f8b0f827" status: 200 len: 1325 microversion: 2.1 time: 1.124939 Jul 16 21:15:21.672158 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 823/1692] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:15:20 2022] GET /compute/v2.1/servers/39e51d01-21e0-4dd5-bfd4-16b0f8b0f827 => generated 1325 bytes in 1150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:15:21.678217 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-591b4c02-38ab-4e32-96fb-7af4e049ee28 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:15:21.679124 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-591b4c02-38ab-4e32-96fb-7af4e049ee28 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] [instance: 23df6e0a-3637-4f59-93f9-b02dd614b5c3] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:15:21.687472 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-591b4c02-38ab-4e32-96fb-7af4e049ee28 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:21.688234 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-591b4c02-38ab-4e32-96fb-7af4e049ee28 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:21.689274 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-591b4c02-38ab-4e32-96fb-7af4e049ee28 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:21.689873 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-591b4c02-38ab-4e32-96fb-7af4e049ee28 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:22.692415 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-591b4c02-38ab-4e32-96fb-7af4e049ee28 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a69f9dee03d43beaafa966216b9e289', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9b769bf8f3ea4e368bf68de1df6538f9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:15:22.700475 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-591b4c02-38ab-4e32-96fb-7af4e049ee28 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] 10.222.0.16 "GET /compute/v2.1/servers/23df6e0a-3637-4f59-93f9-b02dd614b5c3" status: 200 len: 1196 microversion: 2.1 time: 1.026733 Jul 16 21:15:22.701069 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 824/1693] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:15:21 2022] GET /compute/v2.1/servers/23df6e0a-3637-4f59-93f9-b02dd614b5c3 => generated 1196 bytes in 1028 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:15:22.719835 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-3bde030d-d4c1-49ac-9943-ce6a02a8d7ab req-be3e02ce-073b-4905-af9f-1c83a87d08d4 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "39e51d01-21e0-4dd5-bfd4-16b0f8b0f827", "name": "network-vif-plugged", "status": "completed", "tag": "55c3f33b-16c2-42c1-94d7-42da613add6d"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:15:22.745984 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-3bde030d-d4c1-49ac-9943-ce6a02a8d7ab req-be3e02ce-073b-4905-af9f-1c83a87d08d4 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:22.746830 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-3bde030d-d4c1-49ac-9943-ce6a02a8d7ab req-be3e02ce-073b-4905-af9f-1c83a87d08d4 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:22.763532 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-3bde030d-d4c1-49ac-9943-ce6a02a8d7ab req-be3e02ce-073b-4905-af9f-1c83a87d08d4 service nova] Creating event network-vif-plugged:55c3f33b-16c2-42c1-94d7-42da613add6d for instance 39e51d01-21e0-4dd5-bfd4-16b0f8b0f827 on nch2-master-1 Jul 16 21:15:22.771598 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-3bde030d-d4c1-49ac-9943-ce6a02a8d7ab req-be3e02ce-073b-4905-af9f-1c83a87d08d4 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.062914 Jul 16 21:15:22.799116 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 825/1694] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:15:22 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 65 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:15:22.799116 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-6f3efc53-176a-4ea9-9af5-afad7450a095 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:15:22.799116 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-6f3efc53-176a-4ea9-9af5-afad7450a095 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] [instance: 0079e567-30df-4cac-a0ad-4379bcbbcf48] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:15:22.799116 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6f3efc53-176a-4ea9-9af5-afad7450a095 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:22.799116 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6f3efc53-176a-4ea9-9af5-afad7450a095 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:22.799116 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6f3efc53-176a-4ea9-9af5-afad7450a095 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:22.799116 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6f3efc53-176a-4ea9-9af5-afad7450a095 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:23.846812 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-6f3efc53-176a-4ea9-9af5-afad7450a095 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '53af488f3b0242a797f539284b78d59e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bb1ede0d0b6941cc97278e6c617f9b01', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:15:23.864781 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-6f3efc53-176a-4ea9-9af5-afad7450a095 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] 10.222.0.16 "GET /compute/v2.1/servers/0079e567-30df-4cac-a0ad-4379bcbbcf48" status: 200 len: 1334 microversion: 2.1 time: 1.089824 Jul 16 21:15:23.879574 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 826/1695] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:15:22 2022] GET /compute/v2.1/servers/0079e567-30df-4cac-a0ad-4379bcbbcf48 => generated 1334 bytes in 1106 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:15:23.888429 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-5d0b3514-d507-4d6e-bdc8-724db58e5908 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:15:23.889091 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-5d0b3514-d507-4d6e-bdc8-724db58e5908 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] [instance: 84ab7e65-1bf0-4b8b-baea-6d1d4ca13dd7] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:15:23.900514 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5d0b3514-d507-4d6e-bdc8-724db58e5908 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:23.901394 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5d0b3514-d507-4d6e-bdc8-724db58e5908 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:23.902939 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5d0b3514-d507-4d6e-bdc8-724db58e5908 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:23.903619 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5d0b3514-d507-4d6e-bdc8-724db58e5908 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:24.231267 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-5d0b3514-d507-4d6e-bdc8-724db58e5908 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:15:24.241769 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-5d0b3514-d507-4d6e-bdc8-724db58e5908 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:15:24.243515 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-5d0b3514-d507-4d6e-bdc8-724db58e5908 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:15:24.244949 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-5d0b3514-d507-4d6e-bdc8-724db58e5908 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] 10.222.0.16 "GET /compute/v2.1/servers/84ab7e65-1bf0-4b8b-baea-6d1d4ca13dd7" status: 200 len: 1281 microversion: 2.25 time: 0.364113 Jul 16 21:15:24.245627 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 827/1696] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:15:23 2022] GET /compute/v2.1/servers/84ab7e65-1bf0-4b8b-baea-6d1d4ca13dd7 => generated 1281 bytes in 365 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:15:24.399208 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-feeb441f-f4ce-4263-bd5c-b64c65dd7902 tempest-KeyPairsNegativeTestJSON-2037449827 tempest-KeyPairsNegativeTestJSON-2037449827-project] Action: 'create', calling method: .version_select at 0x7fdd9cdf95e0>, body: {"keypair": {"name": "key_/.\\@:", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBCN3QDhmza3/+K74KmgVt8/1VETJQSy7E74i8W++eHDf/wPuu+q/6r+frRkjkArwguL21YHFxt1ASa3OtbLsTd/qnVtUHF5OlLtr7Fd9r69vx2EyGfCCmpB2ccwX50yHIA=="}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:15:24.410514 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-feeb441f-f4ce-4263-bd5c-b64c65dd7902 tempest-KeyPairsNegativeTestJSON-2037449827 tempest-KeyPairsNegativeTestJSON-2037449827-project] HTTP exception thrown: Keypair data is invalid: Keypair name contains unsafe characters Jul 16 21:15:24.418134 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-feeb441f-f4ce-4263-bd5c-b64c65dd7902 tempest-KeyPairsNegativeTestJSON-2037449827 tempest-KeyPairsNegativeTestJSON-2037449827-project] Request method failure captured: Jul 16 21:15:24.418134 ncdv-master-1 devstack@n-api.service[138681]: request: POST /compute/v2.1/os-keypairs HTTP/1.1 Jul 16 21:15:24.418134 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:15:24.418134 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:15:24.418134 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:15:24.418134 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 257 Jul 16 21:15:24.418134 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:15:24.418134 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:15:24.418134 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:15:24.418134 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:15:24.418134 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0ypqWdrcrtbXaSq2GEpWPon25FMLnHNO2qMo-SvtJJgkVPuxEq-ypCmZ__i9RWDig4ZDfMaeDQTVbW5PHkz1pceOF1CtLJFW3A3oLh4lnXqlF8rU0ZGZuPihQd9gGmqkk_eaWH1_Ok_SeKFxd3pRS6aDEwNoV2B4X0519r5BRpQwhSI Jul 16 21:15:24.418134 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:15:24.418134 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:15:24.418134 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:15:24.418134 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:15:24.418134 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:15:24.418134 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:15:24.418134 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: 71aa49f99a7647329b40abfd7e10a606 Jul 16 21:15:24.418134 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-KeyPairsNegativeTestJSON-2037449827 Jul 16 21:15:24.418134 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:15:24.418134 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:15:24.418134 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/71aa49f99a7647329b40abfd7e10a606"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/71aa49f99a7647329b40abfd7e10a606"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_71aa49f99a7647329b40abfd7e10a606"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/71aa49f99a7647329b40abfd7e10a606"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:15:24.418134 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-KeyPairsNegativeTestJSON-2037449827 Jul 16 21:15:24.418134 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: 71aa49f99a7647329b40abfd7e10a606 Jul 16 21:15:24.418134 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-KeyPairsNegativeTestJSON-2037449827 Jul 16 21:15:24.418134 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-KeyPairsNegativeTestJSON-2037449827-project Jul 16 21:15:24.418134 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:15:24.418134 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:15:24.418134 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: 29baab42d8c64f5480701a768f793d5e Jul 16 21:15:24.418134 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-KeyPairsNegativeTestJSON-2037449827-project Jul 16 21:15:24.418134 ncdv-master-1 devstack@n-api.service[138681]: Jul 16 21:15:24.418134 ncdv-master-1 devstack@n-api.service[138681]: {"keypair": {"name": "key_/.\\@:", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBCN3QDhmza3/+K74KmgVt8/1VETJQSy7E74i8W++eHDf/wPuu+q/6r+frRkjkArwguL21YHFxt1ASa3OtbLsTd/qnVtUHF5OlLtr7Fd9r69vx2EyGfCCmpB2ccwX50yHIA=="}} Jul 16 21:15:24.418134 ncdv-master-1 devstack@n-api.service[138681]: method: Jul 16 21:15:24.418134 ncdv-master-1 devstack@n-api.service[138681]: exception: Keypair data is invalid: Keypair name contains unsafe characters Jul 16 21:15:24.418134 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'body': {'keypair': {'name': 'key_/.\\@:', 'public_key': 'ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBCN3QDhmza3/+K74KmgVt8/1VETJQSy7E74i8W++eHDf/wPuu+q/6r+frRkjkArwguL21YHFxt1ASa3OtbLsTd/qnVtUHF5OlLtr7Fd9r69vx2EyGfCCmpB2ccwX50yHIA=='}}} Jul 16 21:15:24.418134 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:15:24.418134 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:15:24.418134 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/keypairs.py", line 113, in _create Jul 16 21:15:24.418134 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi keypair = self.api.import_key_pair( Jul 16 21:15:24.418134 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/exception_wrapper.py", line 71, in wrapped Jul 16 21:15:24.418134 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi _emit_versioned_exception_notification( Jul 16 21:15:24.418134 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 16 21:15:24.418134 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi self.force_reraise() Jul 16 21:15:24.418134 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 16 21:15:24.418134 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise self.value Jul 16 21:15:24.418134 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/exception_wrapper.py", line 63, in wrapped Jul 16 21:15:24.418134 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(self, context, *args, **kw) Jul 16 21:15:24.418134 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 6641, in import_key_pair Jul 16 21:15:24.418134 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi self._validate_new_key_pair(context, user_id, key_name, key_type) Jul 16 21:15:24.418134 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 6619, in _validate_new_key_pair Jul 16 21:15:24.418134 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.InvalidKeypair( Jul 16 21:15:24.418134 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.InvalidKeypair: Keypair data is invalid: Keypair name contains unsafe characters Jul 16 21:15:24.418134 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:15:24.418134 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:15:24.418134 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:15:24.418134 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:15:24.418134 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:15:24.418134 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:15:24.418134 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:15:24.418134 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:15:24.418134 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Jul 16 21:15:24.418134 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Jul 16 21:15:24.423478 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:15:24.423478 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:15:24.423478 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 16 21:15:24.423478 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 16 21:15:24.423478 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 16 21:15:24.423478 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 16 21:15:24.423478 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/keypairs.py", line 99, in create Jul 16 21:15:24.423478 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return self._create(req, body) Jul 16 21:15:24.423478 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/keypairs.py", line 124, in _create Jul 16 21:15:24.423478 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPBadRequest(explanation=exc.format_message()) Jul 16 21:15:24.423478 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Keypair data is invalid: Keypair name contains unsafe characters Jul 16 21:15:24.423478 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:15:24.423478 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:15:24.423478 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:15:24.423478 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:15:24.423478 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:15:24.423478 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:15:24.423478 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:15:24.423478 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:15:24.423478 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Keypair data is invalid: Keypair name contains unsafe characters Jul 16 21:15:24.423478 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:15:24.433915 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-feeb441f-f4ce-4263-bd5c-b64c65dd7902 tempest-KeyPairsNegativeTestJSON-2037449827 tempest-KeyPairsNegativeTestJSON-2037449827-project] Returning 400 to user: Keypair data is invalid: Keypair name contains unsafe characters {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:15:24.433915 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-feeb441f-f4ce-4263-bd5c-b64c65dd7902 tempest-KeyPairsNegativeTestJSON-2037449827 tempest-KeyPairsNegativeTestJSON-2037449827-project] 10.222.0.16 "POST /compute/v2.1/os-keypairs" status: 400 len: 108 microversion: 2.1 time: 0.181044 Jul 16 21:15:24.433915 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 828/1697] 10.222.0.16 () {60 vars in 1158 bytes} [Sat Jul 16 21:15:24 2022] POST /compute/v2.1/os-keypairs => generated 108 bytes in 182 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Jul 16 21:15:24.438354 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-8f5af20d-e671-43e5-ae01-5406d30720f2 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:15:24.439424 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-8f5af20d-e671-43e5-ae01-5406d30720f2 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] [instance: 39e51d01-21e0-4dd5-bfd4-16b0f8b0f827] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:15:24.453800 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8f5af20d-e671-43e5-ae01-5406d30720f2 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:24.454516 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8f5af20d-e671-43e5-ae01-5406d30720f2 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:24.456150 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8f5af20d-e671-43e5-ae01-5406d30720f2 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:24.461222 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8f5af20d-e671-43e5-ae01-5406d30720f2 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:24.664971 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-8f5af20d-e671-43e5-ae01-5406d30720f2 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4e9df82f95ac4779859972403869096a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4337575622d04a26942905c9dc044ba7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:15:24.679630 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-8f5af20d-e671-43e5-ae01-5406d30720f2 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] 10.222.0.16 "GET /compute/v2.1/servers/39e51d01-21e0-4dd5-bfd4-16b0f8b0f827" status: 200 len: 1325 microversion: 2.1 time: 0.249441 Jul 16 21:15:24.680226 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 829/1698] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:15:24 2022] GET /compute/v2.1/servers/39e51d01-21e0-4dd5-bfd4-16b0f8b0f827 => generated 1325 bytes in 251 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:15:24.701421 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-f81040f7-7cc3-43e6-aa90-beac0ec81611 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:15:24.706975 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-f81040f7-7cc3-43e6-aa90-beac0ec81611 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] [instance: 23df6e0a-3637-4f59-93f9-b02dd614b5c3] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:15:24.721112 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f81040f7-7cc3-43e6-aa90-beac0ec81611 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:24.721684 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f81040f7-7cc3-43e6-aa90-beac0ec81611 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:24.724565 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f81040f7-7cc3-43e6-aa90-beac0ec81611 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:24.732357 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f81040f7-7cc3-43e6-aa90-beac0ec81611 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:25.092892 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-f81040f7-7cc3-43e6-aa90-beac0ec81611 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a69f9dee03d43beaafa966216b9e289', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9b769bf8f3ea4e368bf68de1df6538f9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:15:25.105761 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-f81040f7-7cc3-43e6-aa90-beac0ec81611 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] 10.222.0.16 "GET /compute/v2.1/servers/23df6e0a-3637-4f59-93f9-b02dd614b5c3" status: 200 len: 1196 microversion: 2.1 time: 0.411076 Jul 16 21:15:25.108199 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 830/1699] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:15:24 2022] GET /compute/v2.1/servers/23df6e0a-3637-4f59-93f9-b02dd614b5c3 => generated 1196 bytes in 427 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:15:25.132468 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-9a992690-e7f3-4122-ba7e-8eea6161f6f0 tempest-KeyPairsNegativeTestJSON-2037449827 tempest-KeyPairsNegativeTestJSON-2037449827-project] Action: 'create', calling method: .version_select at 0x7fdd9cdf9670>, body: {"keypair": {"name": "tempest-KeyPairsNegativeTestJSON-keypair-1346059116", "public_key": "ssh-rsa AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA openstack@ubuntu"}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:15:25.153382 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-9a992690-e7f3-4122-ba7e-8eea6161f6f0 tempest-KeyPairsNegativeTestJSON-2037449827 tempest-KeyPairsNegativeTestJSON-2037449827-project] Getting quotas for project 71aa49f99a7647329b40abfd7e10a606. Resources: {'key_pairs'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:15:25.159505 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-9a992690-e7f3-4122-ba7e-8eea6161f6f0 tempest-KeyPairsNegativeTestJSON-2037449827 tempest-KeyPairsNegativeTestJSON-2037449827-project] Getting quotas for user 29baab42d8c64f5480701a768f793d5e and project 71aa49f99a7647329b40abfd7e10a606. Resources: {'key_pairs'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:15:25.182506 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_api:SELECT=97,nova_api:INSERT=3 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:15:25.188384 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-9a992690-e7f3-4122-ba7e-8eea6161f6f0 tempest-KeyPairsNegativeTestJSON-2037449827 tempest-KeyPairsNegativeTestJSON-2037449827-project] HTTP exception thrown: Keypair data is invalid: failed to generate fingerprint Jul 16 21:15:25.202629 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-9a992690-e7f3-4122-ba7e-8eea6161f6f0 tempest-KeyPairsNegativeTestJSON-2037449827 tempest-KeyPairsNegativeTestJSON-2037449827-project] Request method failure captured: Jul 16 21:15:25.202629 ncdv-master-1 devstack@n-api.service[138681]: request: POST /compute/v2.1/os-keypairs HTTP/1.1 Jul 16 21:15:25.202629 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:15:25.202629 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:15:25.202629 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:15:25.202629 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 2167 Jul 16 21:15:25.202629 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:15:25.202629 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:15:25.202629 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:15:25.202629 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:15:25.202629 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0ypqWdrcrtbXaSq2GEpWPon25FMLnHNO2qMo-SvtJJgkVPuxEq-ypCmZ__i9RWDig4ZDfMaeDQTVbW5PHkz1pceOF1CtLJFW3A3oLh4lnXqlF8rU0ZGZuPihQd9gGmqkk_eaWH1_Ok_SeKFxd3pRS6aDEwNoV2B4X0519r5BRpQwhSI Jul 16 21:15:25.202629 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:15:25.202629 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:15:25.202629 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:15:25.202629 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:15:25.202629 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:15:25.202629 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:15:25.202629 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: 71aa49f99a7647329b40abfd7e10a606 Jul 16 21:15:25.202629 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-KeyPairsNegativeTestJSON-2037449827 Jul 16 21:15:25.202629 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:15:25.202629 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:15:25.202629 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/71aa49f99a7647329b40abfd7e10a606"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/71aa49f99a7647329b40abfd7e10a606"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_71aa49f99a7647329b40abfd7e10a606"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/71aa49f99a7647329b40abfd7e10a606"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:15:25.202629 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-KeyPairsNegativeTestJSON-2037449827 Jul 16 21:15:25.202629 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: 71aa49f99a7647329b40abfd7e10a606 Jul 16 21:15:25.202629 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-KeyPairsNegativeTestJSON-2037449827 Jul 16 21:15:25.202629 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-KeyPairsNegativeTestJSON-2037449827-project Jul 16 21:15:25.202629 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:15:25.202629 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:15:25.202629 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: 29baab42d8c64f5480701a768f793d5e Jul 16 21:15:25.202629 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-KeyPairsNegativeTestJSON-2037449827-project Jul 16 21:15:25.202629 ncdv-master-1 devstack@n-api.service[138681]: Jul 16 21:15:25.202629 ncdv-master-1 devstack@n-api.service[138681]: {"keypair": {"name": "tempest-KeyPairsNegativeTestJSON-keypair-1346059116", "public_key": "ssh-rsa AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA openstack@ubuntu"}} Jul 16 21:15:25.202629 ncdv-master-1 devstack@n-api.service[138681]: method: Jul 16 21:15:25.202629 ncdv-master-1 devstack@n-api.service[138681]: exception: Keypair data is invalid: failed to generate fingerprint Jul 16 21:15:25.207451 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'body': {'keypair': {'name': 'tempest-KeyPairsNegativeTestJSON-keypair-1346059116', 'public_key': 'ssh-rsa AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA openstack@ubuntu'}}} Jul 16 21:15:25.207451 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:15:25.207451 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:15:25.207451 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/crypto.py", line 71, in generate_fingerprint Jul 16 21:15:25.207451 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi serialization.load_ssh_public_key( Jul 16 21:15:25.207451 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cryptography/hazmat/primitives/serialization/ssh.py", line 666, in load_ssh_public_key Jul 16 21:15:25.207451 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise ValueError("Invalid key format") Jul 16 21:15:25.207451 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi ValueError: Invalid key format Jul 16 21:15:25.207451 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:15:25.207451 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:15:25.207451 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:15:25.207451 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:15:25.207451 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/keypairs.py", line 113, in _create Jul 16 21:15:25.207451 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi keypair = self.api.import_key_pair( Jul 16 21:15:25.207451 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/exception_wrapper.py", line 71, in wrapped Jul 16 21:15:25.207451 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi _emit_versioned_exception_notification( Jul 16 21:15:25.207451 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 16 21:15:25.207451 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi self.force_reraise() Jul 16 21:15:25.207451 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 16 21:15:25.207451 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise self.value Jul 16 21:15:25.207451 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/exception_wrapper.py", line 63, in wrapped Jul 16 21:15:25.207451 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(self, context, *args, **kw) Jul 16 21:15:25.207451 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 6658, in import_key_pair Jul 16 21:15:25.207451 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi fingerprint = self._generate_fingerprint(public_key, key_type) Jul 16 21:15:25.207451 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 6731, in _generate_fingerprint Jul 16 21:15:25.207451 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return crypto.generate_fingerprint(public_key) Jul 16 21:15:25.207451 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/crypto.py", line 77, in generate_fingerprint Jul 16 21:15:25.207451 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.InvalidKeypair( Jul 16 21:15:25.207451 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.InvalidKeypair: Keypair data is invalid: failed to generate fingerprint Jul 16 21:15:25.207451 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:15:25.207451 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:15:25.207451 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:15:25.207451 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:15:25.207451 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:15:25.207451 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:15:25.207451 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:15:25.207451 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:15:25.207451 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Jul 16 21:15:25.207451 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Jul 16 21:15:25.207451 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:15:25.207451 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:15:25.211231 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 16 21:15:25.211231 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 16 21:15:25.211231 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 16 21:15:25.211231 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 16 21:15:25.211231 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/keypairs.py", line 99, in create Jul 16 21:15:25.211231 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return self._create(req, body) Jul 16 21:15:25.211231 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/keypairs.py", line 124, in _create Jul 16 21:15:25.211231 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPBadRequest(explanation=exc.format_message()) Jul 16 21:15:25.211231 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Keypair data is invalid: failed to generate fingerprint Jul 16 21:15:25.211231 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:15:25.211231 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:15:25.211231 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:15:25.211231 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:15:25.211231 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:15:25.211231 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:15:25.211231 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:15:25.211231 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:15:25.211231 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Keypair data is invalid: failed to generate fingerprint Jul 16 21:15:25.211231 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:15:25.211231 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-9a992690-e7f3-4122-ba7e-8eea6161f6f0 tempest-KeyPairsNegativeTestJSON-2037449827 tempest-KeyPairsNegativeTestJSON-2037449827-project] Returning 400 to user: Keypair data is invalid: failed to generate fingerprint {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:15:25.218484 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-9a992690-e7f3-4122-ba7e-8eea6161f6f0 tempest-KeyPairsNegativeTestJSON-2037449827 tempest-KeyPairsNegativeTestJSON-2037449827-project] 10.222.0.16 "POST /compute/v2.1/os-keypairs" status: 400 len: 99 microversion: 2.1 time: 0.108945 Jul 16 21:15:25.228207 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 831/1700] 10.222.0.16 () {60 vars in 1159 bytes} [Sat Jul 16 21:15:25 2022] POST /compute/v2.1/os-keypairs => generated 99 bytes in 119 msecs (HTTP/1.1 400) 9 headers in 380 bytes (1 switches on core 0) Jul 16 21:15:25.235328 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-3b17adbd-d38c-470d-9177-03ca9aa48547 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:15:25.236274 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-3b17adbd-d38c-470d-9177-03ca9aa48547 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] [instance: 0079e567-30df-4cac-a0ad-4379bcbbcf48] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:15:25.247205 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3b17adbd-d38c-470d-9177-03ca9aa48547 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:25.248146 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3b17adbd-d38c-470d-9177-03ca9aa48547 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:25.249124 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3b17adbd-d38c-470d-9177-03ca9aa48547 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:25.249908 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3b17adbd-d38c-470d-9177-03ca9aa48547 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:25.655929 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-3b17adbd-d38c-470d-9177-03ca9aa48547 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '53af488f3b0242a797f539284b78d59e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bb1ede0d0b6941cc97278e6c617f9b01', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:15:25.670904 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-3b17adbd-d38c-470d-9177-03ca9aa48547 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] 10.222.0.16 "GET /compute/v2.1/servers/0079e567-30df-4cac-a0ad-4379bcbbcf48" status: 200 len: 1334 microversion: 2.1 time: 0.441170 Jul 16 21:15:25.673168 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 832/1701] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:15:25 2022] GET /compute/v2.1/servers/0079e567-30df-4cac-a0ad-4379bcbbcf48 => generated 1334 bytes in 444 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:15:25.679885 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-7a6530e1-603e-4010-a541-7f08e3812ca8 tempest-KeyPairsNegativeTestJSON-2037449827 tempest-KeyPairsNegativeTestJSON-2037449827-project] Action: 'create', calling method: .version_select at 0x7fdda2b89670>, body: {"keypair": {"name": "tempest-keypair-1649610775", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBGTCjYuGkYyDjEyYNGkdbYy5bGRLlZ9lra42DDZqbcSqyIIBwrPx7bCy/9YIoPoXUCz69mEPj68xA2gpOGgZcwH6OYD1UNwZD1B0xef4GIt4quTTatWkZUTNM8JSuecpNQ=="}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:15:25.701995 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-7a6530e1-603e-4010-a541-7f08e3812ca8 tempest-KeyPairsNegativeTestJSON-2037449827 tempest-KeyPairsNegativeTestJSON-2037449827-project] Getting quotas for project 71aa49f99a7647329b40abfd7e10a606. Resources: {'key_pairs'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:15:25.709003 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-7a6530e1-603e-4010-a541-7f08e3812ca8 tempest-KeyPairsNegativeTestJSON-2037449827 tempest-KeyPairsNegativeTestJSON-2037449827-project] Getting quotas for user 29baab42d8c64f5480701a768f793d5e and project 71aa49f99a7647329b40abfd7e10a606. Resources: {'key_pairs'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:15:25.858401 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-7a6530e1-603e-4010-a541-7f08e3812ca8 tempest-KeyPairsNegativeTestJSON-2037449827 tempest-KeyPairsNegativeTestJSON-2037449827-project] 10.222.0.16 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.178340 Jul 16 21:15:25.858401 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 833/1702] 10.222.0.16 () {60 vars in 1158 bytes} [Sat Jul 16 21:15:25 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:15:25.871952 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-4d4f5d17-d33f-43dd-9803-c492f6049bd5 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:15:25.872580 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-4d4f5d17-d33f-43dd-9803-c492f6049bd5 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] [instance: 84ab7e65-1bf0-4b8b-baea-6d1d4ca13dd7] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:15:25.882158 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4d4f5d17-d33f-43dd-9803-c492f6049bd5 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:25.883159 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4d4f5d17-d33f-43dd-9803-c492f6049bd5 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:25.883527 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4d4f5d17-d33f-43dd-9803-c492f6049bd5 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:25.883965 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4d4f5d17-d33f-43dd-9803-c492f6049bd5 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:26.131187 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-647c8fbc-ec6c-4420-90ca-1c9e9f79a102 tempest-FloatingIPDetailsTestJSON-1149902298 tempest-FloatingIPDetailsTestJSON-1149902298-project] 10.222.0.16 "POST /compute/v2.1/os-floating-ips" status: 200 len: 141 microversion: 2.1 time: 6.200868 Jul 16 21:15:26.133608 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 870/1703] 10.222.0.16 () {60 vars in 1169 bytes} [Sat Jul 16 21:15:19 2022] POST /compute/v2.1/os-floating-ips => generated 141 bytes in 6205 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:15:26.156062 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-4d4f5d17-d33f-43dd-9803-c492f6049bd5 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:15:26.166191 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-4d4f5d17-d33f-43dd-9803-c492f6049bd5 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:15:26.168371 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-4d4f5d17-d33f-43dd-9803-c492f6049bd5 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:15:26.169724 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-4d4f5d17-d33f-43dd-9803-c492f6049bd5 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] 10.222.0.16 "GET /compute/v2.1/servers/84ab7e65-1bf0-4b8b-baea-6d1d4ca13dd7" status: 200 len: 1281 microversion: 2.25 time: 0.311780 Jul 16 21:15:26.170814 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 834/1704] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:15:25 2022] GET /compute/v2.1/servers/84ab7e65-1bf0-4b8b-baea-6d1d4ca13dd7 => generated 1281 bytes in 313 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:15:26.176254 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-11db32f5-0960-4b33-bfbd-9c09c687aaf8 req-ec101da1-aee5-4bb5-a593-7fb6f8328ab0 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "23df6e0a-3637-4f59-93f9-b02dd614b5c3", "name": "network-vif-plugged", "status": "completed", "tag": "60e499ad-afcb-4fe7-80ef-058f63c4fc21"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:15:26.184746 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-11db32f5-0960-4b33-bfbd-9c09c687aaf8 req-ec101da1-aee5-4bb5-a593-7fb6f8328ab0 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:26.185196 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-11db32f5-0960-4b33-bfbd-9c09c687aaf8 req-ec101da1-aee5-4bb5-a593-7fb6f8328ab0 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:26.197198 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-11db32f5-0960-4b33-bfbd-9c09c687aaf8 req-ec101da1-aee5-4bb5-a593-7fb6f8328ab0 service nova] Creating event network-vif-plugged:60e499ad-afcb-4fe7-80ef-058f63c4fc21 for instance 23df6e0a-3637-4f59-93f9-b02dd614b5c3 on nch2-master-1 Jul 16 21:15:26.207308 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-11db32f5-0960-4b33-bfbd-9c09c687aaf8 req-ec101da1-aee5-4bb5-a593-7fb6f8328ab0 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.034868 Jul 16 21:15:26.208189 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 835/1705] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:15:26 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:15:26.213648 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-ce6371f0-927e-4ee8-aaef-0a4d3ab8314d tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:15:26.214381 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-ce6371f0-927e-4ee8-aaef-0a4d3ab8314d tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] [instance: 39e51d01-21e0-4dd5-bfd4-16b0f8b0f827] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:15:26.220856 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ce6371f0-927e-4ee8-aaef-0a4d3ab8314d tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:26.221310 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ce6371f0-927e-4ee8-aaef-0a4d3ab8314d tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:26.222635 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ce6371f0-927e-4ee8-aaef-0a4d3ab8314d tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:26.223270 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ce6371f0-927e-4ee8-aaef-0a4d3ab8314d tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:26.458018 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-3c5ed771-3073-402c-8c07-edc43d3a5239 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ImagesTestJSON-server-2047721313", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452", "networks": [{"uuid": "e0956c94-52de-403b-bf7c-aa26a7179ba6"}]}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:15:26.532393 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.network.neutron [None req-3c5ed771-3073-402c-8c07-edc43d3a5239 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] validate_networks() for [('e0956c94-52de-403b-bf7c-aa26a7179ba6', None, None, None, None, None)] {{(pid=138680) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 16 21:15:26.779137 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-ce6371f0-927e-4ee8-aaef-0a4d3ab8314d tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4e9df82f95ac4779859972403869096a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4337575622d04a26942905c9dc044ba7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:15:26.798721 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-ce6371f0-927e-4ee8-aaef-0a4d3ab8314d tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] 10.222.0.16 "GET /compute/v2.1/servers/39e51d01-21e0-4dd5-bfd4-16b0f8b0f827" status: 200 len: 1325 microversion: 2.1 time: 0.586950 Jul 16 21:15:26.810441 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 836/1706] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:15:26 2022] GET /compute/v2.1/servers/39e51d01-21e0-4dd5-bfd4-16b0f8b0f827 => generated 1325 bytes in 601 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:15:26.818759 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-b8696703-8cf0-450a-972c-7d07d84b56b5 tempest-KeyPairsNegativeTestJSON-2037449827 tempest-KeyPairsNegativeTestJSON-2037449827-project] Action: 'create', calling method: .version_select at 0x7fdd9d5c3f70>, body: {"keypair": {"name": "tempest-keypair-1649610775", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBBdt6pb8E5p0sWzQGb/KtEdVYd8+QF7ko6IYezbtB3SnSFxxa++xpdY8cdbNmXXTLNb7QgRFIY5gYCf6rgDWxGlforJbNWky62VXCdqpFIi5G25DbG2Rjskkiyt57pnCrA=="}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:15:26.839327 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-b8696703-8cf0-450a-972c-7d07d84b56b5 tempest-KeyPairsNegativeTestJSON-2037449827 tempest-KeyPairsNegativeTestJSON-2037449827-project] Getting quotas for project 71aa49f99a7647329b40abfd7e10a606. Resources: {'key_pairs'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:15:26.847207 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-b8696703-8cf0-450a-972c-7d07d84b56b5 tempest-KeyPairsNegativeTestJSON-2037449827 tempest-KeyPairsNegativeTestJSON-2037449827-project] Getting quotas for user 29baab42d8c64f5480701a768f793d5e and project 71aa49f99a7647329b40abfd7e10a606. Resources: {'key_pairs'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:15:26.913100 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-b8696703-8cf0-450a-972c-7d07d84b56b5 tempest-KeyPairsNegativeTestJSON-2037449827 tempest-KeyPairsNegativeTestJSON-2037449827-project] HTTP exception thrown: Key pair 'tempest-keypair-1649610775' already exists. Jul 16 21:15:26.918189 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-b8696703-8cf0-450a-972c-7d07d84b56b5 tempest-KeyPairsNegativeTestJSON-2037449827 tempest-KeyPairsNegativeTestJSON-2037449827-project] Request method failure captured: Jul 16 21:15:26.918189 ncdv-master-1 devstack@n-api.service[138681]: request: POST /compute/v2.1/os-keypairs HTTP/1.1 Jul 16 21:15:26.918189 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:15:26.918189 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:15:26.918189 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:15:26.918189 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 273 Jul 16 21:15:26.918189 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:15:26.918189 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:15:26.918189 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:15:26.918189 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:15:26.918189 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0ypqWdrcrtbXaSq2GEpWPon25FMLnHNO2qMo-SvtJJgkVPuxEq-ypCmZ__i9RWDig4ZDfMaeDQTVbW5PHkz1pceOF1CtLJFW3A3oLh4lnXqlF8rU0ZGZuPihQd9gGmqkk_eaWH1_Ok_SeKFxd3pRS6aDEwNoV2B4X0519r5BRpQwhSI Jul 16 21:15:26.918189 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:15:26.918189 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:15:26.918189 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:15:26.918189 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:15:26.918189 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:15:26.918189 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:15:26.918189 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: 71aa49f99a7647329b40abfd7e10a606 Jul 16 21:15:26.918189 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-KeyPairsNegativeTestJSON-2037449827 Jul 16 21:15:26.918189 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:15:26.918189 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:15:26.918189 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/71aa49f99a7647329b40abfd7e10a606"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/71aa49f99a7647329b40abfd7e10a606"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_71aa49f99a7647329b40abfd7e10a606"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/71aa49f99a7647329b40abfd7e10a606"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:15:26.918189 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-KeyPairsNegativeTestJSON-2037449827 Jul 16 21:15:26.918189 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: 71aa49f99a7647329b40abfd7e10a606 Jul 16 21:15:26.918189 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-KeyPairsNegativeTestJSON-2037449827 Jul 16 21:15:26.918189 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-KeyPairsNegativeTestJSON-2037449827-project Jul 16 21:15:26.918189 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:15:26.918189 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:15:26.918189 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: 29baab42d8c64f5480701a768f793d5e Jul 16 21:15:26.918189 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-KeyPairsNegativeTestJSON-2037449827-project Jul 16 21:15:26.918189 ncdv-master-1 devstack@n-api.service[138681]: Jul 16 21:15:26.918189 ncdv-master-1 devstack@n-api.service[138681]: {"keypair": {"name": "tempest-keypair-1649610775", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBBdt6pb8E5p0sWzQGb/KtEdVYd8+QF7ko6IYezbtB3SnSFxxa++xpdY8cdbNmXXTLNb7QgRFIY5gYCf6rgDWxGlforJbNWky62VXCdqpFIi5G25DbG2Rjskkiyt57pnCrA=="}} Jul 16 21:15:26.918189 ncdv-master-1 devstack@n-api.service[138681]: method: Jul 16 21:15:26.918189 ncdv-master-1 devstack@n-api.service[138681]: exception: Key pair 'tempest-keypair-1649610775' already exists. Jul 16 21:15:26.918189 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'body': {'keypair': {'name': 'tempest-keypair-1649610775', 'public_key': 'ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBBdt6pb8E5p0sWzQGb/KtEdVYd8+QF7ko6IYezbtB3SnSFxxa++xpdY8cdbNmXXTLNb7QgRFIY5gYCf6rgDWxGlforJbNWky62VXCdqpFIi5G25DbG2Rjskkiyt57pnCrA=='}}} Jul 16 21:15:26.918189 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:15:26.918189 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:15:26.918189 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 16 21:15:26.918189 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi self.dialect.do_execute( Jul 16 21:15:26.918189 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 16 21:15:26.918189 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi cursor.execute(statement, parameters) Jul 16 21:15:26.918189 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 16 21:15:26.918189 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi result = self._query(query) Jul 16 21:15:26.918189 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 16 21:15:26.918189 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi conn.query(q) Jul 16 21:15:26.918189 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 16 21:15:26.918189 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 16 21:15:26.918189 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 16 21:15:26.918189 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi result.read() Jul 16 21:15:26.918189 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 16 21:15:26.918189 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi first_packet = self.connection._read_packet() Jul 16 21:15:26.918189 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 16 21:15:26.918189 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi packet.raise_for_error() Jul 16 21:15:26.918189 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 16 21:15:26.918189 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi err.raise_mysql_exception(self._data) Jul 16 21:15:26.918189 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 16 21:15:26.918189 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise errorclass(errno, errval) Jul 16 21:15:26.918189 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi pymysql.err.IntegrityError: (1062, "Duplicate entry '29baab42d8c64f5480701a768f793d5e-tempest-keypair-1649610775' for key 'key_pairs.uniq_key_pairs0user_id0name'") Jul 16 21:15:26.918189 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:15:26.924932 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi The above exception was the direct cause of the following exception: Jul 16 21:15:26.924932 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:15:26.924932 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:15:26.924932 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/keypair.py", line 69, in _create_in_db Jul 16 21:15:26.924932 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi kp.save(context.session) Jul 16 21:15:26.924932 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/models.py", line 38, in save Jul 16 21:15:26.924932 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi session.flush() Jul 16 21:15:26.924932 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 16 21:15:26.924932 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi self._flush(objects) Jul 16 21:15:26.924932 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 16 21:15:26.924932 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi transaction.rollback(_capture_exception=True) Jul 16 21:15:26.924932 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 16 21:15:26.924932 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi compat.raise_( Jul 16 21:15:26.924932 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 16 21:15:26.924932 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception Jul 16 21:15:26.924932 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 16 21:15:26.924932 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi flush_context.execute() Jul 16 21:15:26.924932 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 16 21:15:26.924932 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi rec.execute(self) Jul 16 21:15:26.924932 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 16 21:15:26.924932 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi util.preloaded.orm_persistence.save_obj( Jul 16 21:15:26.924932 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 16 21:15:26.924932 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi _emit_insert_statements( Jul 16 21:15:26.924932 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jul 16 21:15:26.924932 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi result = connection._execute_20( Jul 16 21:15:26.924932 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 16 21:15:26.924932 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return meth(self, args_10style, kwargs_10style, execution_options) Jul 16 21:15:26.924932 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 16 21:15:26.924932 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return connection._execute_clauseelement( Jul 16 21:15:26.924932 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 16 21:15:26.924932 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi ret = self._execute_context( Jul 16 21:15:26.924932 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 16 21:15:26.924932 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi self._handle_dbapi_exception( Jul 16 21:15:26.924932 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 16 21:15:26.924932 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 16 21:15:26.924932 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 16 21:15:26.924932 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception Jul 16 21:15:26.924932 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 16 21:15:26.924932 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi self.dialect.do_execute( Jul 16 21:15:26.924932 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 16 21:15:26.924932 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi cursor.execute(statement, parameters) Jul 16 21:15:26.924932 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 16 21:15:26.924932 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi result = self._query(query) Jul 16 21:15:26.924932 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 16 21:15:26.924932 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi conn.query(q) Jul 16 21:15:26.924932 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 16 21:15:26.924932 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 16 21:15:26.924932 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 16 21:15:26.924932 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi result.read() Jul 16 21:15:26.924932 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 16 21:15:26.924932 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi first_packet = self.connection._read_packet() Jul 16 21:15:26.924932 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 16 21:15:26.924932 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi packet.raise_for_error() Jul 16 21:15:26.924932 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 16 21:15:26.924932 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi err.raise_mysql_exception(self._data) Jul 16 21:15:26.924932 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 16 21:15:26.933620 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise errorclass(errno, errval) Jul 16 21:15:26.933620 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry '29baab42d8c64f5480701a768f793d5e-tempest-keypair-1649610775' for key 'key_pairs.uniq_key_pairs0user_id0name'") Jul 16 21:15:26.933620 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi [SQL: INSERT INTO key_pairs (created_at, updated_at, name, user_id, fingerprint, public_key, type) VALUES (%(created_at)s, %(updated_at)s, %(name)s, %(user_id)s, %(fingerprint)s, %(public_key)s, %(type)s)] Jul 16 21:15:26.933620 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi [parameters: {'created_at': datetime.datetime(2022, 7, 16, 21, 15, 26, 866274), 'updated_at': None, 'name': 'tempest-keypair-1649610775', 'user_id': '29baab42d8c64f5480701a768f793d5e', 'fingerprint': '2e:07:34:31:d6:03:6f:46:a4:b3:17:b4:93:60:de:73', 'public_key': 'ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBBdt6pb8E5p0sWzQGb/KtEdVYd8+QF7ko6IYezbtB3SnSFxxa++xpdY8cdbNmXXTLNb7QgRFIY5gYCf6rgDWxGlforJbNWky62VXCdqpFIi5G25DbG2Rjskkiyt57pnCrA==', 'type': 'ssh'}] Jul 16 21:15:26.933620 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 16 21:15:26.933620 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:15:26.933620 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:15:26.933620 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:15:26.933620 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:15:26.933620 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/keypairs.py", line 113, in _create Jul 16 21:15:26.933620 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi keypair = self.api.import_key_pair( Jul 16 21:15:26.933620 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/exception_wrapper.py", line 71, in wrapped Jul 16 21:15:26.933620 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi _emit_versioned_exception_notification( Jul 16 21:15:26.933620 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 16 21:15:26.933620 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi self.force_reraise() Jul 16 21:15:26.933620 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 16 21:15:26.933620 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise self.value Jul 16 21:15:26.933620 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/exception_wrapper.py", line 63, in wrapped Jul 16 21:15:26.933620 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(self, context, *args, **kw) Jul 16 21:15:26.933620 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 6661, in import_key_pair Jul 16 21:15:26.933620 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi keypair.create() Jul 16 21:15:26.933620 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 226, in wrapper Jul 16 21:15:26.933620 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return fn(self, *args, **kwargs) Jul 16 21:15:26.933620 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/keypair.py", line 158, in create Jul 16 21:15:26.933620 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi self._create() Jul 16 21:15:26.933620 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/keypair.py", line 162, in _create Jul 16 21:15:26.933620 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi db_keypair = self._create_in_db(self._context, updates) Jul 16 21:15:26.933620 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/keypair.py", line 134, in _create_in_db Jul 16 21:15:26.933620 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return _create_in_db(context, values) Jul 16 21:15:26.933620 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1010, in wrapper Jul 16 21:15:26.933620 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return fn(*args, **kwargs) Jul 16 21:15:26.933620 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/keypair.py", line 71, in _create_in_db Jul 16 21:15:26.933620 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.KeyPairExists(key_name=values['name']) Jul 16 21:15:26.933620 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.KeyPairExists: Key pair 'tempest-keypair-1649610775' already exists. Jul 16 21:15:26.933620 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:15:26.933620 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:15:26.933620 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:15:26.933620 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:15:26.933620 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:15:26.933620 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:15:26.933620 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:15:26.933620 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:15:26.933620 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Jul 16 21:15:26.933620 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Jul 16 21:15:26.933620 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:15:26.933620 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:15:26.933620 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 16 21:15:26.933620 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 16 21:15:26.933620 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 16 21:15:26.933620 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 16 21:15:26.933620 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/keypairs.py", line 99, in create Jul 16 21:15:26.933620 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return self._create(req, body) Jul 16 21:15:26.933620 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/keypairs.py", line 126, in _create Jul 16 21:15:26.933620 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPConflict(explanation=exc.format_message()) Jul 16 21:15:26.933620 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPConflict: Key pair 'tempest-keypair-1649610775' already exists. Jul 16 21:15:26.942037 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:15:26.942037 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:15:26.942037 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:15:26.942037 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:15:26.942037 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:15:26.942037 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:15:26.942037 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:15:26.942037 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:15:26.942037 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Key pair 'tempest-keypair-1649610775' already exists. Jul 16 21:15:26.942037 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:15:26.942037 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-b8696703-8cf0-450a-972c-7d07d84b56b5 tempest-KeyPairsNegativeTestJSON-2037449827 tempest-KeyPairsNegativeTestJSON-2037449827-project] Returning 409 to user: Key pair 'tempest-keypair-1649610775' already exists. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:15:26.942037 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-b8696703-8cf0-450a-972c-7d07d84b56b5 tempest-KeyPairsNegativeTestJSON-2037449827 tempest-KeyPairsNegativeTestJSON-2037449827-project] 10.222.0.16 "POST /compute/v2.1/os-keypairs" status: 409 len: 105 microversion: 2.1 time: 0.109845 Jul 16 21:15:26.942037 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 837/1707] 10.222.0.16 () {60 vars in 1158 bytes} [Sat Jul 16 21:15:26 2022] POST /compute/v2.1/os-keypairs => generated 105 bytes in 111 msecs (HTTP/1.1 409) 9 headers in 378 bytes (1 switches on core 0) Jul 16 21:15:26.942037 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-a7c94f7f-380d-4ba2-8b14-a69715c2189d tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:15:26.942037 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-a7c94f7f-380d-4ba2-8b14-a69715c2189d tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] [instance: 23df6e0a-3637-4f59-93f9-b02dd614b5c3] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:15:26.946885 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a7c94f7f-380d-4ba2-8b14-a69715c2189d tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:26.946885 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a7c94f7f-380d-4ba2-8b14-a69715c2189d tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:26.946885 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a7c94f7f-380d-4ba2-8b14-a69715c2189d tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:26.946885 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a7c94f7f-380d-4ba2-8b14-a69715c2189d tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:27.176779 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-a7c94f7f-380d-4ba2-8b14-a69715c2189d tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a69f9dee03d43beaafa966216b9e289', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9b769bf8f3ea4e368bf68de1df6538f9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:15:27.186000 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-a7c94f7f-380d-4ba2-8b14-a69715c2189d tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] 10.222.0.16 "GET /compute/v2.1/servers/23df6e0a-3637-4f59-93f9-b02dd614b5c3" status: 200 len: 1196 microversion: 2.1 time: 0.261910 Jul 16 21:15:27.186844 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 838/1708] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:15:26 2022] GET /compute/v2.1/servers/23df6e0a-3637-4f59-93f9-b02dd614b5c3 => generated 1196 bytes in 264 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:15:27.196537 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-1bf10d3d-3272-4649-b51f-d5cbd1b76988 tempest-FloatingIPDetailsTestJSON-1149902298 tempest-FloatingIPDetailsTestJSON-1149902298-project] Action: 'create', calling method: .version_select at 0x7fdd9cdb5310>, body: {"pool": null} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:15:27.317208 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-3c5ed771-3073-402c-8c07-edc43d3a5239 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Flavor limits 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 16 21:15:27.317955 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-3c5ed771-3073-402c-8c07-edc43d3a5239 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Image limits 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 16 21:15:27.318421 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-3c5ed771-3073-402c-8c07-edc43d3a5239 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Flavor pref 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 16 21:15:27.318832 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-3c5ed771-3073-402c-8c07-edc43d3a5239 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Image pref 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 16 21:15:27.319297 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-3c5ed771-3073-402c-8c07-edc43d3a5239 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 16 21:15:27.329001 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell0:SELECT=47 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:15:27.331146 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3c5ed771-3073-402c-8c07-edc43d3a5239 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:27.331695 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3c5ed771-3073-402c-8c07-edc43d3a5239 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:27.332745 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3c5ed771-3073-402c-8c07-edc43d3a5239 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:27.333426 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3c5ed771-3073-402c-8c07-edc43d3a5239 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:27.354610 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-3c5ed771-3073-402c-8c07-edc43d3a5239 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Getting quotas for project 24c64db8082648c8837c16352252f96d. Resources: {'ram', 'instances', 'cores'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:15:27.361453 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-3c5ed771-3073-402c-8c07-edc43d3a5239 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Getting quotas for user daf3c9e6634e4ae1bfb492eae1f44c93 and project 24c64db8082648c8837c16352252f96d. Resources: {'ram', 'instances', 'cores'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:15:27.377065 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-3c5ed771-3073-402c-8c07-edc43d3a5239 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Going to run 1 instances... {{(pid=138680) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 16 21:15:27.385276 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-3c5ed771-3073-402c-8c07-edc43d3a5239 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] [instance: 93e327c3-c63d-439a-9b25-1a805f785949] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138680) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 16 21:15:27.520256 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-3c5ed771-3073-402c-8c07-edc43d3a5239 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 1.384952 Jul 16 21:15:27.522044 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 871/1709] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 21:15:26 2022] POST /compute/v2.1/servers => generated 380 bytes in 1388 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 16 21:15:27.529105 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-c5998f7d-20aa-47bb-9b37-c436344b6feb tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:15:27.530175 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-c5998f7d-20aa-47bb-9b37-c436344b6feb tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] [instance: 0079e567-30df-4cac-a0ad-4379bcbbcf48] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:15:27.541107 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c5998f7d-20aa-47bb-9b37-c436344b6feb tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:27.541787 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c5998f7d-20aa-47bb-9b37-c436344b6feb tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:27.560649 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c5998f7d-20aa-47bb-9b37-c436344b6feb tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:27.561115 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c5998f7d-20aa-47bb-9b37-c436344b6feb tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:27.797785 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-c5998f7d-20aa-47bb-9b37-c436344b6feb tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '53af488f3b0242a797f539284b78d59e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bb1ede0d0b6941cc97278e6c617f9b01', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:15:27.806472 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-c5998f7d-20aa-47bb-9b37-c436344b6feb tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] 10.222.0.16 "GET /compute/v2.1/servers/0079e567-30df-4cac-a0ad-4379bcbbcf48" status: 200 len: 1334 microversion: 2.1 time: 0.281657 Jul 16 21:15:27.807127 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 872/1710] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:15:27 2022] GET /compute/v2.1/servers/0079e567-30df-4cac-a0ad-4379bcbbcf48 => generated 1334 bytes in 284 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:15:27.812274 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-d499d6aa-2dc6-4a49-a68f-00967f6cad5d tempest-KeyPairsNegativeTestJSON-2037449827 tempest-KeyPairsNegativeTestJSON-2037449827-project] Calling method '.version_select at 0x7fdd9d4e0280>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:15:27.889979 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell0:SELECT=2 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:15:27.957925 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-d499d6aa-2dc6-4a49-a68f-00967f6cad5d tempest-KeyPairsNegativeTestJSON-2037449827 tempest-KeyPairsNegativeTestJSON-2037449827-project] 10.222.0.16 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1649610775" status: 202 len: 0 microversion: 2.1 time: 0.149521 Jul 16 21:15:27.958749 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 873/1711] 10.222.0.16 () {58 vars in 1220 bytes} [Sat Jul 16 21:15:27 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1649610775 => generated 0 bytes in 151 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 21:15:27.970095 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-4488c909-d0ee-4906-826f-d95a9fb96598 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:15:27.971078 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-4488c909-d0ee-4906-826f-d95a9fb96598 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] [instance: 84ab7e65-1bf0-4b8b-baea-6d1d4ca13dd7] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:15:27.973210 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_api:SELECT=93,nova_api:INSERT=6,nova_api:DELETE=1 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:15:27.979394 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4488c909-d0ee-4906-826f-d95a9fb96598 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:27.979874 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4488c909-d0ee-4906-826f-d95a9fb96598 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:27.980674 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4488c909-d0ee-4906-826f-d95a9fb96598 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:27.981121 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4488c909-d0ee-4906-826f-d95a9fb96598 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:28.210788 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-4488c909-d0ee-4906-826f-d95a9fb96598 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:15:28.232534 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-4488c909-d0ee-4906-826f-d95a9fb96598 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:15:28.234340 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-4488c909-d0ee-4906-826f-d95a9fb96598 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:15:28.243902 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-4488c909-d0ee-4906-826f-d95a9fb96598 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] 10.222.0.16 "GET /compute/v2.1/servers/84ab7e65-1bf0-4b8b-baea-6d1d4ca13dd7" status: 200 len: 1281 microversion: 2.25 time: 0.276784 Jul 16 21:15:28.257555 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 874/1712] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:15:27 2022] GET /compute/v2.1/servers/84ab7e65-1bf0-4b8b-baea-6d1d4ca13dd7 => generated 1281 bytes in 298 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:15:28.264541 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-ae5bcb20-79b3-400c-995c-db4f7b558f8d tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:15:28.265468 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-ae5bcb20-79b3-400c-995c-db4f7b558f8d tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] [instance: 93e327c3-c63d-439a-9b25-1a805f785949] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:15:28.367318 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-ae5bcb20-79b3-400c-995c-db4f7b558f8d tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'daf3c9e6634e4ae1bfb492eae1f44c93', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '24c64db8082648c8837c16352252f96d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:15:28.375710 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-ae5bcb20-79b3-400c-995c-db4f7b558f8d tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] 10.222.0.16 "GET /compute/v2.1/servers/93e327c3-c63d-439a-9b25-1a805f785949" status: 200 len: 1208 microversion: 2.1 time: 0.116706 Jul 16 21:15:28.376303 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 875/1713] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:15:28 2022] GET /compute/v2.1/servers/93e327c3-c63d-439a-9b25-1a805f785949 => generated 1208 bytes in 118 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:15:28.402966 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-0450b884-8480-4f95-b242-b5e2bd58f114 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:15:28.404476 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-0450b884-8480-4f95-b242-b5e2bd58f114 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] [instance: 39e51d01-21e0-4dd5-bfd4-16b0f8b0f827] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:15:28.432528 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0450b884-8480-4f95-b242-b5e2bd58f114 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:28.437585 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0450b884-8480-4f95-b242-b5e2bd58f114 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:28.439454 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0450b884-8480-4f95-b242-b5e2bd58f114 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:28.440257 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0450b884-8480-4f95-b242-b5e2bd58f114 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:28.875883 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-0450b884-8480-4f95-b242-b5e2bd58f114 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4e9df82f95ac4779859972403869096a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4337575622d04a26942905c9dc044ba7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:15:28.885027 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=100 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:15:28.888319 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-0450b884-8480-4f95-b242-b5e2bd58f114 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] 10.222.0.16 "GET /compute/v2.1/servers/39e51d01-21e0-4dd5-bfd4-16b0f8b0f827" status: 200 len: 1521 microversion: 2.1 time: 0.500151 Jul 16 21:15:28.890849 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 876/1714] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:15:28 2022] GET /compute/v2.1/servers/39e51d01-21e0-4dd5-bfd4-16b0f8b0f827 => generated 1521 bytes in 505 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:15:28.900593 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-84e37959-12d0-49b8-bacc-5a6e538621f8 tempest-KeyPairsNegativeTestJSON-2037449827 tempest-KeyPairsNegativeTestJSON-2037449827-project] Action: 'create', calling method: .version_select at 0x7fdd9d025dc0>, body: {"keypair": {"name": "", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBFbQWHCzNveMVllR14yCqiqhUB7qlbxp2AxeOl4mFu7YxcKgjFn4R4c1hmQ0tOQUMNfKrz21ja5GAm3EratLNwUZMObddz4d6SFo8joouAm7FvJQ50kBrgF+Tja8wBaOkg=="}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:15:28.904542 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-84e37959-12d0-49b8-bacc-5a6e538621f8 tempest-KeyPairsNegativeTestJSON-2037449827 tempest-KeyPairsNegativeTestJSON-2037449827-project] Request method failure captured: Jul 16 21:15:28.904542 ncdv-master-1 devstack@n-api.service[138680]: request: POST /compute/v2.1/os-keypairs HTTP/1.1 Jul 16 21:15:28.904542 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:15:28.904542 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:15:28.904542 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:15:28.904542 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 247 Jul 16 21:15:28.904542 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:15:28.904542 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:15:28.904542 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:15:28.904542 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:15:28.904542 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0ypqWdrcrtbXaSq2GEpWPon25FMLnHNO2qMo-SvtJJgkVPuxEq-ypCmZ__i9RWDig4ZDfMaeDQTVbW5PHkz1pceOF1CtLJFW3A3oLh4lnXqlF8rU0ZGZuPihQd9gGmqkk_eaWH1_Ok_SeKFxd3pRS6aDEwNoV2B4X0519r5BRpQwhSI Jul 16 21:15:28.904542 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:15:28.904542 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:15:28.904542 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:15:28.904542 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:15:28.904542 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:15:28.904542 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:15:28.904542 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: 71aa49f99a7647329b40abfd7e10a606 Jul 16 21:15:28.904542 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-KeyPairsNegativeTestJSON-2037449827 Jul 16 21:15:28.904542 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:15:28.904542 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:15:28.904542 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/71aa49f99a7647329b40abfd7e10a606"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/71aa49f99a7647329b40abfd7e10a606"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_71aa49f99a7647329b40abfd7e10a606"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/71aa49f99a7647329b40abfd7e10a606"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:15:28.904542 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-KeyPairsNegativeTestJSON-2037449827 Jul 16 21:15:28.904542 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: 71aa49f99a7647329b40abfd7e10a606 Jul 16 21:15:28.904542 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-KeyPairsNegativeTestJSON-2037449827 Jul 16 21:15:28.904542 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-KeyPairsNegativeTestJSON-2037449827-project Jul 16 21:15:28.904542 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:15:28.904542 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:15:28.904542 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: 29baab42d8c64f5480701a768f793d5e Jul 16 21:15:28.904542 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-KeyPairsNegativeTestJSON-2037449827-project Jul 16 21:15:28.904542 ncdv-master-1 devstack@n-api.service[138680]: Jul 16 21:15:28.904542 ncdv-master-1 devstack@n-api.service[138680]: {"keypair": {"name": "", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBFbQWHCzNveMVllR14yCqiqhUB7qlbxp2AxeOl4mFu7YxcKgjFn4R4c1hmQ0tOQUMNfKrz21ja5GAm3EratLNwUZMObddz4d6SFo8joouAm7FvJQ50kBrgF+Tja8wBaOkg=="}} Jul 16 21:15:28.904542 ncdv-master-1 devstack@n-api.service[138680]: method: Jul 16 21:15:28.904542 ncdv-master-1 devstack@n-api.service[138680]: exception: Invalid input for field/attribute name. Value: . '' is too short Jul 16 21:15:28.904542 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'body': {'keypair': {'name': '', 'public_key': 'ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBFbQWHCzNveMVllR14yCqiqhUB7qlbxp2AxeOl4mFu7YxcKgjFn4R4c1hmQ0tOQUMNfKrz21ja5GAm3EratLNwUZMObddz4d6SFo8joouAm7FvJQ50kBrgF+Tja8wBaOkg=='}}} Jul 16 21:15:28.904542 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:15:28.904542 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:15:28.904542 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 279, in validate Jul 16 21:15:28.904542 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Jul 16 21:15:28.904542 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/lib/python3/dist-packages/jsonschema/validators.py", line 353, in validate Jul 16 21:15:28.904542 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise error Jul 16 21:15:28.904542 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: '' is too short Jul 16 21:15:28.904542 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:15:28.904542 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Failed validating 'minLength' in schema['properties']['keypair']['properties']['name']: Jul 16 21:15:28.904542 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi {'format': 'name', 'maxLength': 255, 'minLength': 1, 'type': 'string'} Jul 16 21:15:28.904542 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:15:28.904542 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi On instance['keypair']: Jul 16 21:15:28.904542 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi '' Jul 16 21:15:28.904542 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:15:28.904542 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:15:28.904542 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:15:28.904542 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:15:28.904542 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:15:28.904542 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:15:28.904542 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:15:28.904542 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:15:28.904542 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Jul 16 21:15:28.904542 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Jul 16 21:15:28.904542 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:15:28.904542 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:15:28.904542 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 16 21:15:28.904542 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 16 21:15:28.904542 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Jul 16 21:15:28.904542 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Jul 16 21:15:28.909126 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Jul 16 21:15:28.909126 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Jul 16 21:15:28.909126 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 308, in validate Jul 16 21:15:28.909126 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Jul 16 21:15:28.909126 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute name. Value: . '' is too short Jul 16 21:15:28.909126 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:15:28.909126 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:15:28.909126 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:15:28.909126 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:15:28.909126 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:15:28.909126 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:15:28.909126 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Jul 16 21:15:28.909126 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Jul 16 21:15:28.909126 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute name. Value: . '' is too short Jul 16 21:15:28.909126 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:15:28.912724 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-84e37959-12d0-49b8-bacc-5a6e538621f8 tempest-KeyPairsNegativeTestJSON-2037449827 tempest-KeyPairsNegativeTestJSON-2037449827-project] Returning 400 to user: Invalid input for field/attribute name. Value: . '' is too short {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:15:28.918005 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-84e37959-12d0-49b8-bacc-5a6e538621f8 tempest-KeyPairsNegativeTestJSON-2037449827 tempest-KeyPairsNegativeTestJSON-2037449827-project] 10.222.0.16 "POST /compute/v2.1/os-keypairs" status: 400 len: 108 microversion: 2.1 time: 0.023224 Jul 16 21:15:28.918005 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 877/1715] 10.222.0.16 () {60 vars in 1158 bytes} [Sat Jul 16 21:15:28 2022] POST /compute/v2.1/os-keypairs => generated 108 bytes in 25 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Jul 16 21:15:28.923224 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-5b38f12b-1c0d-4923-b9dc-611478253a97 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:15:28.924174 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-5b38f12b-1c0d-4923-b9dc-611478253a97 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] [instance: 23df6e0a-3637-4f59-93f9-b02dd614b5c3] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:15:28.933519 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5b38f12b-1c0d-4923-b9dc-611478253a97 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:28.934585 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5b38f12b-1c0d-4923-b9dc-611478253a97 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:28.936693 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5b38f12b-1c0d-4923-b9dc-611478253a97 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:28.938008 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5b38f12b-1c0d-4923-b9dc-611478253a97 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:29.168898 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-5b38f12b-1c0d-4923-b9dc-611478253a97 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a69f9dee03d43beaafa966216b9e289', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9b769bf8f3ea4e368bf68de1df6538f9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:15:29.177388 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-5b38f12b-1c0d-4923-b9dc-611478253a97 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] 10.222.0.16 "GET /compute/v2.1/servers/23df6e0a-3637-4f59-93f9-b02dd614b5c3" status: 200 len: 1379 microversion: 2.1 time: 0.260047 Jul 16 21:15:29.189048 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 878/1716] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:15:28 2022] GET /compute/v2.1/servers/23df6e0a-3637-4f59-93f9-b02dd614b5c3 => generated 1379 bytes in 272 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:15:29.195179 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-d850f635-13d7-4654-b9d5-83eae0c6d8f8 req-c49232ae-333c-4405-87ae-655e619d7f7c service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "84ab7e65-1bf0-4b8b-baea-6d1d4ca13dd7", "name": "network-vif-plugged", "status": "completed", "tag": "42dfc89b-90fd-4214-983c-f02029faca17"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:15:29.203756 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-d850f635-13d7-4654-b9d5-83eae0c6d8f8 req-c49232ae-333c-4405-87ae-655e619d7f7c service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:29.204470 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-d850f635-13d7-4654-b9d5-83eae0c6d8f8 req-c49232ae-333c-4405-87ae-655e619d7f7c service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:29.224045 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-d850f635-13d7-4654-b9d5-83eae0c6d8f8 req-c49232ae-333c-4405-87ae-655e619d7f7c service nova] Creating event network-vif-plugged:42dfc89b-90fd-4214-983c-f02029faca17 for instance 84ab7e65-1bf0-4b8b-baea-6d1d4ca13dd7 on nch1-master-1 Jul 16 21:15:29.230739 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-d850f635-13d7-4654-b9d5-83eae0c6d8f8 req-c49232ae-333c-4405-87ae-655e619d7f7c service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.039902 Jul 16 21:15:29.231773 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 879/1717] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:15:29 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:15:29.239343 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-9937c3a6-19ee-4837-beba-704ad9495837 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:15:29.240322 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-9937c3a6-19ee-4837-beba-704ad9495837 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] [instance: 0079e567-30df-4cac-a0ad-4379bcbbcf48] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:15:29.274283 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9937c3a6-19ee-4837-beba-704ad9495837 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:29.275198 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9937c3a6-19ee-4837-beba-704ad9495837 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:29.276234 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9937c3a6-19ee-4837-beba-704ad9495837 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:29.276904 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9937c3a6-19ee-4837-beba-704ad9495837 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:29.678554 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-9937c3a6-19ee-4837-beba-704ad9495837 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '53af488f3b0242a797f539284b78d59e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bb1ede0d0b6941cc97278e6c617f9b01', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:15:29.692230 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-9937c3a6-19ee-4837-beba-704ad9495837 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] 10.222.0.16 "GET /compute/v2.1/servers/0079e567-30df-4cac-a0ad-4379bcbbcf48" status: 200 len: 1334 microversion: 2.1 time: 0.456750 Jul 16 21:15:29.695537 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 880/1718] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:15:29 2022] GET /compute/v2.1/servers/0079e567-30df-4cac-a0ad-4379bcbbcf48 => generated 1334 bytes in 463 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:15:29.706504 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-1bf10d3d-3272-4649-b51f-d5cbd1b76988 tempest-FloatingIPDetailsTestJSON-1149902298 tempest-FloatingIPDetailsTestJSON-1149902298-project] 10.222.0.16 "POST /compute/v2.1/os-floating-ips" status: 200 len: 142 microversion: 2.1 time: 2.518061 Jul 16 21:15:29.712147 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 839/1719] 10.222.0.16 () {60 vars in 1169 bytes} [Sat Jul 16 21:15:27 2022] POST /compute/v2.1/os-floating-ips => generated 142 bytes in 2521 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:15:29.715915 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-907ab50b-b9fd-4f9f-a582-0032f6238e9d tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:15:29.716966 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-907ab50b-b9fd-4f9f-a582-0032f6238e9d tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] [instance: 39e51d01-21e0-4dd5-bfd4-16b0f8b0f827] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:15:29.719884 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-d1d20db1-9f6d-4c7a-8776-ef1e383f4291 tempest-KeyPairsNegativeTestJSON-2037449827 tempest-KeyPairsNegativeTestJSON-2037449827-project] Action: 'create', calling method: .version_select at 0x7fdd9d107d30>, body: {"keypair": {"name": "tempest-KeyPairsNegativeTestJSON-keypair-1333129704", "public_key": " "}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:15:29.740231 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-907ab50b-b9fd-4f9f-a582-0032f6238e9d tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:29.740930 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-907ab50b-b9fd-4f9f-a582-0032f6238e9d tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:29.742192 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-907ab50b-b9fd-4f9f-a582-0032f6238e9d tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:29.746588 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-907ab50b-b9fd-4f9f-a582-0032f6238e9d tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:29.755909 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-d1d20db1-9f6d-4c7a-8776-ef1e383f4291 tempest-KeyPairsNegativeTestJSON-2037449827 tempest-KeyPairsNegativeTestJSON-2037449827-project] Getting quotas for project 71aa49f99a7647329b40abfd7e10a606. Resources: {'key_pairs'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:15:29.768446 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-d1d20db1-9f6d-4c7a-8776-ef1e383f4291 tempest-KeyPairsNegativeTestJSON-2037449827 tempest-KeyPairsNegativeTestJSON-2037449827-project] Getting quotas for user 29baab42d8c64f5480701a768f793d5e and project 71aa49f99a7647329b40abfd7e10a606. Resources: {'key_pairs'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:15:29.821465 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-d1d20db1-9f6d-4c7a-8776-ef1e383f4291 tempest-KeyPairsNegativeTestJSON-2037449827 tempest-KeyPairsNegativeTestJSON-2037449827-project] HTTP exception thrown: Keypair data is invalid: failed to generate fingerprint Jul 16 21:15:29.824120 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-d1d20db1-9f6d-4c7a-8776-ef1e383f4291 tempest-KeyPairsNegativeTestJSON-2037449827 tempest-KeyPairsNegativeTestJSON-2037449827-project] Request method failure captured: Jul 16 21:15:29.824120 ncdv-master-1 devstack@n-api.service[138681]: request: POST /compute/v2.1/os-keypairs HTTP/1.1 Jul 16 21:15:29.824120 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:15:29.824120 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:15:29.824120 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:15:29.824120 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 95 Jul 16 21:15:29.824120 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:15:29.824120 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:15:29.824120 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:15:29.824120 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:15:29.824120 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0ypqWdrcrtbXaSq2GEpWPon25FMLnHNO2qMo-SvtJJgkVPuxEq-ypCmZ__i9RWDig4ZDfMaeDQTVbW5PHkz1pceOF1CtLJFW3A3oLh4lnXqlF8rU0ZGZuPihQd9gGmqkk_eaWH1_Ok_SeKFxd3pRS6aDEwNoV2B4X0519r5BRpQwhSI Jul 16 21:15:29.824120 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:15:29.824120 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:15:29.824120 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:15:29.824120 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:15:29.824120 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:15:29.824120 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:15:29.824120 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: 71aa49f99a7647329b40abfd7e10a606 Jul 16 21:15:29.824120 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-KeyPairsNegativeTestJSON-2037449827 Jul 16 21:15:29.824120 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:15:29.824120 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:15:29.824120 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/71aa49f99a7647329b40abfd7e10a606"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/71aa49f99a7647329b40abfd7e10a606"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_71aa49f99a7647329b40abfd7e10a606"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/71aa49f99a7647329b40abfd7e10a606"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:15:29.824120 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-KeyPairsNegativeTestJSON-2037449827 Jul 16 21:15:29.824120 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: 71aa49f99a7647329b40abfd7e10a606 Jul 16 21:15:29.824120 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-KeyPairsNegativeTestJSON-2037449827 Jul 16 21:15:29.824120 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-KeyPairsNegativeTestJSON-2037449827-project Jul 16 21:15:29.824120 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:15:29.824120 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:15:29.824120 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: 29baab42d8c64f5480701a768f793d5e Jul 16 21:15:29.824120 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-KeyPairsNegativeTestJSON-2037449827-project Jul 16 21:15:29.824120 ncdv-master-1 devstack@n-api.service[138681]: Jul 16 21:15:29.824120 ncdv-master-1 devstack@n-api.service[138681]: {"keypair": {"name": "tempest-KeyPairsNegativeTestJSON-keypair-1333129704", "public_key": " "}} Jul 16 21:15:29.824120 ncdv-master-1 devstack@n-api.service[138681]: method: Jul 16 21:15:29.824120 ncdv-master-1 devstack@n-api.service[138681]: exception: Keypair data is invalid: failed to generate fingerprint Jul 16 21:15:29.824120 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'body': {'keypair': {'name': 'tempest-KeyPairsNegativeTestJSON-keypair-1333129704', 'public_key': ' '}}} Jul 16 21:15:29.824120 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:15:29.824120 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:15:29.824120 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/crypto.py", line 71, in generate_fingerprint Jul 16 21:15:29.824120 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi serialization.load_ssh_public_key( Jul 16 21:15:29.824120 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cryptography/hazmat/primitives/serialization/ssh.py", line 650, in load_ssh_public_key Jul 16 21:15:29.824120 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise ValueError("Invalid line format") Jul 16 21:15:29.824120 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi ValueError: Invalid line format Jul 16 21:15:29.824120 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:15:29.824120 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:15:29.824120 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:15:29.824120 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:15:29.824120 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/keypairs.py", line 113, in _create Jul 16 21:15:29.824120 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi keypair = self.api.import_key_pair( Jul 16 21:15:29.824120 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/exception_wrapper.py", line 71, in wrapped Jul 16 21:15:29.824120 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi _emit_versioned_exception_notification( Jul 16 21:15:29.824120 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 16 21:15:29.824120 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi self.force_reraise() Jul 16 21:15:29.824120 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 16 21:15:29.824120 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise self.value Jul 16 21:15:29.824120 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/exception_wrapper.py", line 63, in wrapped Jul 16 21:15:29.824120 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(self, context, *args, **kw) Jul 16 21:15:29.824120 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 6658, in import_key_pair Jul 16 21:15:29.824120 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi fingerprint = self._generate_fingerprint(public_key, key_type) Jul 16 21:15:29.824120 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 6731, in _generate_fingerprint Jul 16 21:15:29.824120 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return crypto.generate_fingerprint(public_key) Jul 16 21:15:29.824120 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/crypto.py", line 77, in generate_fingerprint Jul 16 21:15:29.824120 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.InvalidKeypair( Jul 16 21:15:29.824120 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.InvalidKeypair: Keypair data is invalid: failed to generate fingerprint Jul 16 21:15:29.824120 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:15:29.824120 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:15:29.824120 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:15:29.833919 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:15:29.833919 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:15:29.833919 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:15:29.833919 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:15:29.833919 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:15:29.833919 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Jul 16 21:15:29.833919 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Jul 16 21:15:29.833919 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:15:29.833919 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:15:29.833919 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 16 21:15:29.833919 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 16 21:15:29.833919 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 16 21:15:29.833919 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 16 21:15:29.833919 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/keypairs.py", line 99, in create Jul 16 21:15:29.833919 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return self._create(req, body) Jul 16 21:15:29.833919 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/keypairs.py", line 124, in _create Jul 16 21:15:29.833919 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPBadRequest(explanation=exc.format_message()) Jul 16 21:15:29.833919 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Keypair data is invalid: failed to generate fingerprint Jul 16 21:15:29.833919 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:15:29.833919 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:15:29.833919 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:15:29.833919 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:15:29.833919 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:15:29.833919 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:15:29.833919 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:15:29.833919 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:15:29.833919 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Keypair data is invalid: failed to generate fingerprint Jul 16 21:15:29.833919 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:15:29.833919 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-d1d20db1-9f6d-4c7a-8776-ef1e383f4291 tempest-KeyPairsNegativeTestJSON-2037449827 tempest-KeyPairsNegativeTestJSON-2037449827-project] Returning 400 to user: Keypair data is invalid: failed to generate fingerprint {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:15:29.833919 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-d1d20db1-9f6d-4c7a-8776-ef1e383f4291 tempest-KeyPairsNegativeTestJSON-2037449827 tempest-KeyPairsNegativeTestJSON-2037449827-project] 10.222.0.16 "POST /compute/v2.1/os-keypairs" status: 400 len: 99 microversion: 2.1 time: 0.120264 Jul 16 21:15:29.836924 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 840/1720] 10.222.0.16 () {60 vars in 1157 bytes} [Sat Jul 16 21:15:29 2022] POST /compute/v2.1/os-keypairs => generated 99 bytes in 128 msecs (HTTP/1.1 400) 9 headers in 380 bytes (1 switches on core 0) Jul 16 21:15:29.991903 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-907ab50b-b9fd-4f9f-a582-0032f6238e9d tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4e9df82f95ac4779859972403869096a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4337575622d04a26942905c9dc044ba7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:15:30.000403 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-907ab50b-b9fd-4f9f-a582-0032f6238e9d tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] 10.222.0.16 "GET /compute/v2.1/servers/39e51d01-21e0-4dd5-bfd4-16b0f8b0f827" status: 200 len: 1521 microversion: 2.1 time: 0.303755 Jul 16 21:15:30.001677 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 881/1721] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:15:29 2022] GET /compute/v2.1/servers/39e51d01-21e0-4dd5-bfd4-16b0f8b0f827 => generated 1521 bytes in 305 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:15:30.008419 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-f013b276-a872-4034-b72e-56c4a3bda5c6 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:15:30.009602 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-f013b276-a872-4034-b72e-56c4a3bda5c6 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] [instance: 84ab7e65-1bf0-4b8b-baea-6d1d4ca13dd7] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:15:30.029762 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f013b276-a872-4034-b72e-56c4a3bda5c6 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:30.030279 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f013b276-a872-4034-b72e-56c4a3bda5c6 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:30.031138 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f013b276-a872-4034-b72e-56c4a3bda5c6 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:30.032740 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f013b276-a872-4034-b72e-56c4a3bda5c6 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:30.100039 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-e9072509-24b4-44b3-91e9-d5d9b869dddb tempest-LiveMigrationTest-1131758637 tempest-LiveMigrationTest-1131758637-project-admin] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:15:30.100662 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-e9072509-24b4-44b3-91e9-d5d9b869dddb tempest-LiveMigrationTest-1131758637 tempest-LiveMigrationTest-1131758637-project-admin] [instance: 23df6e0a-3637-4f59-93f9-b02dd614b5c3] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:15:30.107637 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e9072509-24b4-44b3-91e9-d5d9b869dddb tempest-LiveMigrationTest-1131758637 tempest-LiveMigrationTest-1131758637-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:30.108737 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e9072509-24b4-44b3-91e9-d5d9b869dddb tempest-LiveMigrationTest-1131758637 tempest-LiveMigrationTest-1131758637-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:30.109851 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e9072509-24b4-44b3-91e9-d5d9b869dddb tempest-LiveMigrationTest-1131758637 tempest-LiveMigrationTest-1131758637-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:30.110750 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e9072509-24b4-44b3-91e9-d5d9b869dddb tempest-LiveMigrationTest-1131758637 tempest-LiveMigrationTest-1131758637-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:30.475975 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-f013b276-a872-4034-b72e-56c4a3bda5c6 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:15:30.486259 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-f013b276-a872-4034-b72e-56c4a3bda5c6 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:15:30.492313 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-f013b276-a872-4034-b72e-56c4a3bda5c6 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:15:30.495977 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-f013b276-a872-4034-b72e-56c4a3bda5c6 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] 10.222.0.16 "GET /compute/v2.1/servers/84ab7e65-1bf0-4b8b-baea-6d1d4ca13dd7" status: 200 len: 1281 microversion: 2.25 time: 0.492988 Jul 16 21:15:30.498087 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 882/1722] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:15:30 2022] GET /compute/v2.1/servers/84ab7e65-1bf0-4b8b-baea-6d1d4ca13dd7 => generated 1281 bytes in 495 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:15:30.505616 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-a7f07eba-0239-4233-ab25-ef033e7e6826 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:15:30.514484 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-a7f07eba-0239-4233-ab25-ef033e7e6826 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] [instance: 93e327c3-c63d-439a-9b25-1a805f785949] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:15:30.529135 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a7f07eba-0239-4233-ab25-ef033e7e6826 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:30.530857 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a7f07eba-0239-4233-ab25-ef033e7e6826 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:30.533353 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a7f07eba-0239-4233-ab25-ef033e7e6826 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:30.534068 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a7f07eba-0239-4233-ab25-ef033e7e6826 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:30.545530 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-e9072509-24b4-44b3-91e9-d5d9b869dddb tempest-LiveMigrationTest-1131758637 tempest-LiveMigrationTest-1131758637-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/23df6e0a-3637-4f59-93f9-b02dd614b5c3" status: 200 len: 1530 microversion: 2.1 time: 0.704748 Jul 16 21:15:30.558598 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 841/1723] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:15:29 2022] GET /compute/v2.1/servers/23df6e0a-3637-4f59-93f9-b02dd614b5c3 => generated 1530 bytes in 721 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:15:30.572266 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-ff7a328a-4c30-45cf-902f-89ea65f03cae tempest-FloatingIPDetailsTestJSON-1149902298 tempest-FloatingIPDetailsTestJSON-1149902298-project] Action: 'create', calling method: .version_select at 0x7fdd9d5c3940>, body: {"pool": null} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:15:30.639431 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-a7f07eba-0239-4233-ab25-ef033e7e6826 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'daf3c9e6634e4ae1bfb492eae1f44c93', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '24c64db8082648c8837c16352252f96d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:15:30.654223 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-a7f07eba-0239-4233-ab25-ef033e7e6826 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] 10.222.0.16 "GET /compute/v2.1/servers/93e327c3-c63d-439a-9b25-1a805f785949" status: 200 len: 1260 microversion: 2.1 time: 0.149798 Jul 16 21:15:30.667306 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 883/1724] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:15:30 2022] GET /compute/v2.1/servers/93e327c3-c63d-439a-9b25-1a805f785949 => generated 1260 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:15:30.679815 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-eb5806e2-9e31-4b7a-850d-387c6e6f4b67 tempest-KeyPairsNegativeTestJSON-2037449827 tempest-KeyPairsNegativeTestJSON-2037449827-project] Action: 'create', calling method: .version_select at 0x7fdd9d434940>, body: {"keypair": {"name": "keypair-000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBBn1X8J0ULd3oAF7FSNpfRtb2YMFX9Pbt9GVOP++OrPhMJhib30thXhNVkop+EqBPdYYPcxlW3cgCM/z6jKsTnfPHIXKD7wVjFyCv8O2W5DbbVt5mg4LdDHl+CbGPZ7sPg=="}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:15:30.683732 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-eb5806e2-9e31-4b7a-850d-387c6e6f4b67 tempest-KeyPairsNegativeTestJSON-2037449827 tempest-KeyPairsNegativeTestJSON-2037449827-project] Request method failure captured: Jul 16 21:15:30.683732 ncdv-master-1 devstack@n-api.service[138680]: request: POST /compute/v2.1/os-keypairs HTTP/1.1 Jul 16 21:15:30.683732 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:15:30.683732 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:15:30.683732 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:15:30.683732 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 507 Jul 16 21:15:30.683732 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:15:30.683732 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:15:30.683732 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:15:30.683732 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:15:30.683732 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0ypqWdrcrtbXaSq2GEpWPon25FMLnHNO2qMo-SvtJJgkVPuxEq-ypCmZ__i9RWDig4ZDfMaeDQTVbW5PHkz1pceOF1CtLJFW3A3oLh4lnXqlF8rU0ZGZuPihQd9gGmqkk_eaWH1_Ok_SeKFxd3pRS6aDEwNoV2B4X0519r5BRpQwhSI Jul 16 21:15:30.683732 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:15:30.683732 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:15:30.683732 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:15:30.683732 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:15:30.683732 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:15:30.683732 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:15:30.683732 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: 71aa49f99a7647329b40abfd7e10a606 Jul 16 21:15:30.683732 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-KeyPairsNegativeTestJSON-2037449827 Jul 16 21:15:30.683732 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:15:30.683732 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:15:30.683732 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/71aa49f99a7647329b40abfd7e10a606"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/71aa49f99a7647329b40abfd7e10a606"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_71aa49f99a7647329b40abfd7e10a606"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/71aa49f99a7647329b40abfd7e10a606"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:15:30.683732 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-KeyPairsNegativeTestJSON-2037449827 Jul 16 21:15:30.683732 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: 71aa49f99a7647329b40abfd7e10a606 Jul 16 21:15:30.683732 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-KeyPairsNegativeTestJSON-2037449827 Jul 16 21:15:30.683732 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-KeyPairsNegativeTestJSON-2037449827-project Jul 16 21:15:30.683732 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:15:30.683732 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:15:30.683732 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: 29baab42d8c64f5480701a768f793d5e Jul 16 21:15:30.683732 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-KeyPairsNegativeTestJSON-2037449827-project Jul 16 21:15:30.683732 ncdv-master-1 devstack@n-api.service[138680]: Jul 16 21:15:30.683732 ncdv-master-1 devstack@n-api.service[138680]: {"keypair": {"name": "keypair-000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBBn1X8J0ULd3oAF7FSNpfRtb2YMFX9Pbt9GVOP++OrPhMJhib30thXhNVkop+EqBPdYYPcxlW3cgCM/z6jKsTnfPHIXKD7wVjFyCv8O2W5DbbVt5mg4LdDHl+CbGPZ7sPg=="}} Jul 16 21:15:30.683732 ncdv-master-1 devstack@n-api.service[138680]: method: Jul 16 21:15:30.683732 ncdv-master-1 devstack@n-api.service[138680]: exception: Invalid input for field/attribute name. Value: keypair-000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000. 'keypair-000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000' is too long Jul 16 21:15:30.683732 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'body': {'keypair': {'name': 'keypair-000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000', 'public_key': 'ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBBn1X8J0ULd3oAF7FSNpfRtb2YMFX9Pbt9GVOP++OrPhMJhib30thXhNVkop+EqBPdYYPcxlW3cgCM/z6jKsTnfPHIXKD7wVjFyCv8O2W5DbbVt5mg4LdDHl+CbGPZ7sPg=='}}} Jul 16 21:15:30.683732 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:15:30.683732 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:15:30.683732 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 279, in validate Jul 16 21:15:30.683732 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Jul 16 21:15:30.683732 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/lib/python3/dist-packages/jsonschema/validators.py", line 353, in validate Jul 16 21:15:30.683732 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise error Jul 16 21:15:30.683732 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: 'keypair-000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000' is too long Jul 16 21:15:30.683732 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:15:30.683732 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Failed validating 'maxLength' in schema['properties']['keypair']['properties']['name']: Jul 16 21:15:30.683732 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi {'format': 'name', 'maxLength': 255, 'minLength': 1, 'type': 'string'} Jul 16 21:15:30.683732 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:15:30.683732 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi On instance['keypair']: Jul 16 21:15:30.683732 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi 'keypair-000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000' Jul 16 21:15:30.683732 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:15:30.687297 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:15:30.687297 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:15:30.687297 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:15:30.687297 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:15:30.687297 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:15:30.687297 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:15:30.687297 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:15:30.687297 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Jul 16 21:15:30.687297 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Jul 16 21:15:30.687297 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:15:30.687297 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:15:30.687297 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 16 21:15:30.687297 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 16 21:15:30.687297 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Jul 16 21:15:30.687297 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Jul 16 21:15:30.687297 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Jul 16 21:15:30.687297 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Jul 16 21:15:30.687297 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 308, in validate Jul 16 21:15:30.687297 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Jul 16 21:15:30.687297 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute name. Value: keypair-000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000. 'keypair-000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000' is too long Jul 16 21:15:30.687297 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:15:30.687297 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:15:30.687297 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:15:30.687297 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:15:30.687297 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:15:30.687297 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:15:30.687297 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Jul 16 21:15:30.687297 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Jul 16 21:15:30.687297 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute name. Value: keypair-000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000. 'keypair-000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000' is too long Jul 16 21:15:30.687297 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:15:30.707380 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-eb5806e2-9e31-4b7a-850d-387c6e6f4b67 tempest-KeyPairsNegativeTestJSON-2037449827 tempest-KeyPairsNegativeTestJSON-2037449827-project] Returning 400 to user: Invalid input for field/attribute name. Value: keypair-000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000. 'keypair-000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000' is too long {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:15:30.712621 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-eb5806e2-9e31-4b7a-850d-387c6e6f4b67 tempest-KeyPairsNegativeTestJSON-2037449827 tempest-KeyPairsNegativeTestJSON-2037449827-project] 10.222.0.16 "POST /compute/v2.1/os-keypairs" status: 400 len: 627 microversion: 2.1 time: 0.043260 Jul 16 21:15:30.713795 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 884/1725] 10.222.0.16 () {60 vars in 1158 bytes} [Sat Jul 16 21:15:30 2022] POST /compute/v2.1/os-keypairs => generated 627 bytes in 46 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Jul 16 21:15:30.735944 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-5bcc4087-ace9-452c-93a4-d12ae924d09b tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Action: 'action', calling method: >, body: {"createImage": {"name": "tempest-ImagesOneServerNegativeTestJSON-image-1234106104"}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:15:30.737583 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-5bcc4087-ace9-452c-93a4-d12ae924d09b tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] [instance: 39e51d01-21e0-4dd5-bfd4-16b0f8b0f827] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:15:30.751498 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5bcc4087-ace9-452c-93a4-d12ae924d09b tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:30.752142 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5bcc4087-ace9-452c-93a4-d12ae924d09b tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:30.753811 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5bcc4087-ace9-452c-93a4-d12ae924d09b tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:30.754435 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5bcc4087-ace9-452c-93a4-d12ae924d09b tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:31.640362 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-5bcc4087-ace9-452c-93a4-d12ae924d09b tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] 10.222.0.16 "POST /compute/v2.1/servers/39e51d01-21e0-4dd5-bfd4-16b0f8b0f827/action" status: 202 len: 0 microversion: 2.1 time: 0.917761 Jul 16 21:15:31.645998 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 885/1726] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 21:15:30 2022] POST /compute/v2.1/servers/39e51d01-21e0-4dd5-bfd4-16b0f8b0f827/action => generated 0 bytes in 925 msecs (HTTP/1.1 202) 10 headers in 450 bytes (1 switches on core 0) Jul 16 21:15:31.666561 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-6633fecd-713f-4b79-901a-eafa442ba01f tempest-LiveMigrationTest-1131758637 tempest-LiveMigrationTest-1131758637-project-admin] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:15:31.667820 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-6633fecd-713f-4b79-901a-eafa442ba01f tempest-LiveMigrationTest-1131758637 tempest-LiveMigrationTest-1131758637-project-admin] [instance: 23df6e0a-3637-4f59-93f9-b02dd614b5c3] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:15:31.679653 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6633fecd-713f-4b79-901a-eafa442ba01f tempest-LiveMigrationTest-1131758637 tempest-LiveMigrationTest-1131758637-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:31.680212 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6633fecd-713f-4b79-901a-eafa442ba01f tempest-LiveMigrationTest-1131758637 tempest-LiveMigrationTest-1131758637-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:31.684287 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6633fecd-713f-4b79-901a-eafa442ba01f tempest-LiveMigrationTest-1131758637 tempest-LiveMigrationTest-1131758637-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:31.684287 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6633fecd-713f-4b79-901a-eafa442ba01f tempest-LiveMigrationTest-1131758637 tempest-LiveMigrationTest-1131758637-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:32.052357 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-6633fecd-713f-4b79-901a-eafa442ba01f tempest-LiveMigrationTest-1131758637 tempest-LiveMigrationTest-1131758637-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/23df6e0a-3637-4f59-93f9-b02dd614b5c3" status: 200 len: 1530 microversion: 2.1 time: 0.404495 Jul 16 21:15:32.053225 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 886/1727] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:15:31 2022] GET /compute/v2.1/servers/23df6e0a-3637-4f59-93f9-b02dd614b5c3 => generated 1530 bytes in 406 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:15:32.071297 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-eb25386e-d473-4372-a272-1d5109316bac tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:15:32.072282 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-eb25386e-d473-4372-a272-1d5109316bac tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] [instance: 0079e567-30df-4cac-a0ad-4379bcbbcf48] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:15:32.078942 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-eb25386e-d473-4372-a272-1d5109316bac tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:32.079712 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-eb25386e-d473-4372-a272-1d5109316bac tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:32.080707 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-eb25386e-d473-4372-a272-1d5109316bac tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:32.082116 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-eb25386e-d473-4372-a272-1d5109316bac tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:32.605039 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-eb25386e-d473-4372-a272-1d5109316bac tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '53af488f3b0242a797f539284b78d59e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bb1ede0d0b6941cc97278e6c617f9b01', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:15:32.624258 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-eb25386e-d473-4372-a272-1d5109316bac tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] 10.222.0.16 "GET /compute/v2.1/servers/0079e567-30df-4cac-a0ad-4379bcbbcf48" status: 200 len: 1334 microversion: 2.1 time: 0.556693 Jul 16 21:15:32.625238 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 887/1728] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:15:32 2022] GET /compute/v2.1/servers/0079e567-30df-4cac-a0ad-4379bcbbcf48 => generated 1334 bytes in 559 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:15:32.631293 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-f266797e-cd08-4a4f-9779-7977c89f8849 tempest-KeyPairsNegativeTestJSON-2037449827 tempest-KeyPairsNegativeTestJSON-2037449827-project] Action: 'create', calling method: .version_select at 0x7fdd9cc433a0>, body: {"keypair": {"name": "tempest-KeyPairsNegativeTestJSON-keypair-496362552", "public_key": "ssh-rsa JUNK nova@ubuntu"}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:15:32.649614 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-f266797e-cd08-4a4f-9779-7977c89f8849 tempest-KeyPairsNegativeTestJSON-2037449827 tempest-KeyPairsNegativeTestJSON-2037449827-project] Getting quotas for project 71aa49f99a7647329b40abfd7e10a606. Resources: {'key_pairs'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:15:32.661386 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-f266797e-cd08-4a4f-9779-7977c89f8849 tempest-KeyPairsNegativeTestJSON-2037449827 tempest-KeyPairsNegativeTestJSON-2037449827-project] Getting quotas for user 29baab42d8c64f5480701a768f793d5e and project 71aa49f99a7647329b40abfd7e10a606. Resources: {'key_pairs'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:15:32.692333 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-f266797e-cd08-4a4f-9779-7977c89f8849 tempest-KeyPairsNegativeTestJSON-2037449827 tempest-KeyPairsNegativeTestJSON-2037449827-project] HTTP exception thrown: Keypair data is invalid: failed to generate fingerprint Jul 16 21:15:32.697082 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-f266797e-cd08-4a4f-9779-7977c89f8849 tempest-KeyPairsNegativeTestJSON-2037449827 tempest-KeyPairsNegativeTestJSON-2037449827-project] Request method failure captured: Jul 16 21:15:32.697082 ncdv-master-1 devstack@n-api.service[138680]: request: POST /compute/v2.1/os-keypairs HTTP/1.1 Jul 16 21:15:32.697082 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:15:32.697082 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:15:32.697082 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:15:32.697082 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 117 Jul 16 21:15:32.697082 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:15:32.697082 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:15:32.697082 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:15:32.697082 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:15:32.697082 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0ypqWdrcrtbXaSq2GEpWPon25FMLnHNO2qMo-SvtJJgkVPuxEq-ypCmZ__i9RWDig4ZDfMaeDQTVbW5PHkz1pceOF1CtLJFW3A3oLh4lnXqlF8rU0ZGZuPihQd9gGmqkk_eaWH1_Ok_SeKFxd3pRS6aDEwNoV2B4X0519r5BRpQwhSI Jul 16 21:15:32.697082 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:15:32.697082 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:15:32.697082 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:15:32.697082 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:15:32.697082 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:15:32.697082 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:15:32.697082 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: 71aa49f99a7647329b40abfd7e10a606 Jul 16 21:15:32.697082 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-KeyPairsNegativeTestJSON-2037449827 Jul 16 21:15:32.697082 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:15:32.697082 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:15:32.697082 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/71aa49f99a7647329b40abfd7e10a606"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/71aa49f99a7647329b40abfd7e10a606"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_71aa49f99a7647329b40abfd7e10a606"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/71aa49f99a7647329b40abfd7e10a606"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:15:32.697082 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-KeyPairsNegativeTestJSON-2037449827 Jul 16 21:15:32.697082 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: 71aa49f99a7647329b40abfd7e10a606 Jul 16 21:15:32.697082 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-KeyPairsNegativeTestJSON-2037449827 Jul 16 21:15:32.697082 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-KeyPairsNegativeTestJSON-2037449827-project Jul 16 21:15:32.697082 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:15:32.697082 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:15:32.697082 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: 29baab42d8c64f5480701a768f793d5e Jul 16 21:15:32.697082 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-KeyPairsNegativeTestJSON-2037449827-project Jul 16 21:15:32.697082 ncdv-master-1 devstack@n-api.service[138680]: Jul 16 21:15:32.697082 ncdv-master-1 devstack@n-api.service[138680]: {"keypair": {"name": "tempest-KeyPairsNegativeTestJSON-keypair-496362552", "public_key": "ssh-rsa JUNK nova@ubuntu"}} Jul 16 21:15:32.697082 ncdv-master-1 devstack@n-api.service[138680]: method: Jul 16 21:15:32.697082 ncdv-master-1 devstack@n-api.service[138680]: exception: Keypair data is invalid: failed to generate fingerprint Jul 16 21:15:32.697082 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'body': {'keypair': {'name': 'tempest-KeyPairsNegativeTestJSON-keypair-496362552', 'public_key': 'ssh-rsa JUNK nova@ubuntu'}}} Jul 16 21:15:32.697082 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:15:32.697082 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:15:32.697082 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/crypto.py", line 71, in generate_fingerprint Jul 16 21:15:32.697082 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi serialization.load_ssh_public_key( Jul 16 21:15:32.697082 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cryptography/hazmat/primitives/serialization/ssh.py", line 664, in load_ssh_public_key Jul 16 21:15:32.697082 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi inner_key_type, data = _get_sshstr(data) Jul 16 21:15:32.697082 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cryptography/hazmat/primitives/serialization/ssh.py", line 134, in _get_sshstr Jul 16 21:15:32.697082 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi n, data = _get_u32(data) Jul 16 21:15:32.697082 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cryptography/hazmat/primitives/serialization/ssh.py", line 121, in _get_u32 Jul 16 21:15:32.697082 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise ValueError("Invalid data") Jul 16 21:15:32.697082 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi ValueError: Invalid data Jul 16 21:15:32.697082 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:15:32.697082 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:15:32.697082 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:15:32.697082 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:15:32.697082 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/keypairs.py", line 113, in _create Jul 16 21:15:32.697082 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi keypair = self.api.import_key_pair( Jul 16 21:15:32.697082 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/exception_wrapper.py", line 71, in wrapped Jul 16 21:15:32.697082 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi _emit_versioned_exception_notification( Jul 16 21:15:32.697082 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 16 21:15:32.697082 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi self.force_reraise() Jul 16 21:15:32.697082 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 16 21:15:32.697082 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise self.value Jul 16 21:15:32.697082 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/exception_wrapper.py", line 63, in wrapped Jul 16 21:15:32.697082 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(self, context, *args, **kw) Jul 16 21:15:32.697082 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 6658, in import_key_pair Jul 16 21:15:32.697082 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi fingerprint = self._generate_fingerprint(public_key, key_type) Jul 16 21:15:32.697082 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 6731, in _generate_fingerprint Jul 16 21:15:32.697082 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return crypto.generate_fingerprint(public_key) Jul 16 21:15:32.710597 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/crypto.py", line 77, in generate_fingerprint Jul 16 21:15:32.710597 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.InvalidKeypair( Jul 16 21:15:32.710597 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.InvalidKeypair: Keypair data is invalid: failed to generate fingerprint Jul 16 21:15:32.710597 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:15:32.710597 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:15:32.710597 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:15:32.710597 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:15:32.710597 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:15:32.710597 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:15:32.710597 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:15:32.710597 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:15:32.710597 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Jul 16 21:15:32.710597 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Jul 16 21:15:32.710597 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:15:32.710597 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:15:32.710597 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 16 21:15:32.710597 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 16 21:15:32.710597 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 16 21:15:32.710597 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 16 21:15:32.710597 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/keypairs.py", line 99, in create Jul 16 21:15:32.710597 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return self._create(req, body) Jul 16 21:15:32.710597 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/keypairs.py", line 124, in _create Jul 16 21:15:32.710597 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPBadRequest(explanation=exc.format_message()) Jul 16 21:15:32.710597 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Keypair data is invalid: failed to generate fingerprint Jul 16 21:15:32.710597 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:15:32.710597 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:15:32.710597 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:15:32.710597 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:15:32.710597 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:15:32.710597 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:15:32.710597 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:15:32.710597 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:15:32.710597 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Keypair data is invalid: failed to generate fingerprint Jul 16 21:15:32.710597 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:15:32.710597 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-f266797e-cd08-4a4f-9779-7977c89f8849 tempest-KeyPairsNegativeTestJSON-2037449827 tempest-KeyPairsNegativeTestJSON-2037449827-project] Returning 400 to user: Keypair data is invalid: failed to generate fingerprint {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:15:32.710597 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-f266797e-cd08-4a4f-9779-7977c89f8849 tempest-KeyPairsNegativeTestJSON-2037449827 tempest-KeyPairsNegativeTestJSON-2037449827-project] 10.222.0.16 "POST /compute/v2.1/os-keypairs" status: 400 len: 99 microversion: 2.1 time: 0.077068 Jul 16 21:15:32.710597 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 888/1729] 10.222.0.16 () {60 vars in 1158 bytes} [Sat Jul 16 21:15:32 2022] POST /compute/v2.1/os-keypairs => generated 99 bytes in 78 msecs (HTTP/1.1 400) 9 headers in 380 bytes (1 switches on core 0) Jul 16 21:15:32.722879 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-227b749d-df42-4af6-89bf-b312644dbceb tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:15:32.723947 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-227b749d-df42-4af6-89bf-b312644dbceb tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] [instance: 84ab7e65-1bf0-4b8b-baea-6d1d4ca13dd7] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:15:32.745241 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-227b749d-df42-4af6-89bf-b312644dbceb tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:32.763804 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-227b749d-df42-4af6-89bf-b312644dbceb tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.018s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:32.764774 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-227b749d-df42-4af6-89bf-b312644dbceb tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:32.765239 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-227b749d-df42-4af6-89bf-b312644dbceb tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:33.171499 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-227b749d-df42-4af6-89bf-b312644dbceb tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:15:33.186645 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-227b749d-df42-4af6-89bf-b312644dbceb tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:15:33.189322 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-227b749d-df42-4af6-89bf-b312644dbceb tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:15:33.192622 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-227b749d-df42-4af6-89bf-b312644dbceb tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] 10.222.0.16 "GET /compute/v2.1/servers/84ab7e65-1bf0-4b8b-baea-6d1d4ca13dd7" status: 200 len: 1477 microversion: 2.25 time: 0.487166 Jul 16 21:15:33.205319 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 889/1730] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:15:32 2022] GET /compute/v2.1/servers/84ab7e65-1bf0-4b8b-baea-6d1d4ca13dd7 => generated 1477 bytes in 496 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:15:33.213463 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-51469e8e-2d4b-4647-be95-55956858f023 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Calling method '.version_select at 0x7fdda2bc5700>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:15:33.509612 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-51469e8e-2d4b-4647-be95-55956858f023 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] 10.222.0.16 "DELETE /compute/v2.1/images/23300f39-e85b-4c2f-ad31-2d2a76ad6f94" status: 204 len: 0 microversion: 2.1 time: 0.306887 Jul 16 21:15:33.510968 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 890/1731] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:15:33 2022] DELETE /compute/v2.1/images/23300f39-e85b-4c2f-ad31-2d2a76ad6f94 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 16 21:15:33.515836 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-1aa1f4ee-27c5-46c3-8c9f-752f513e9d88 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:15:33.516672 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-1aa1f4ee-27c5-46c3-8c9f-752f513e9d88 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] [instance: 93e327c3-c63d-439a-9b25-1a805f785949] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:15:33.525039 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1aa1f4ee-27c5-46c3-8c9f-752f513e9d88 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:33.525731 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1aa1f4ee-27c5-46c3-8c9f-752f513e9d88 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:33.526683 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1aa1f4ee-27c5-46c3-8c9f-752f513e9d88 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:33.527325 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1aa1f4ee-27c5-46c3-8c9f-752f513e9d88 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:33.949232 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-1aa1f4ee-27c5-46c3-8c9f-752f513e9d88 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'daf3c9e6634e4ae1bfb492eae1f44c93', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '24c64db8082648c8837c16352252f96d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:15:33.962013 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-1aa1f4ee-27c5-46c3-8c9f-752f513e9d88 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] 10.222.0.16 "GET /compute/v2.1/servers/93e327c3-c63d-439a-9b25-1a805f785949" status: 200 len: 1308 microversion: 2.1 time: 0.450742 Jul 16 21:15:33.963209 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 891/1732] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:15:33 2022] GET /compute/v2.1/servers/93e327c3-c63d-439a-9b25-1a805f785949 => generated 1308 bytes in 452 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:15:33.968017 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-b2779716-d1a9-41c0-b933-e76675a769a4 tempest-LiveMigrationTest-1131758637 tempest-LiveMigrationTest-1131758637-project-admin] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:15:33.975303 ncdv-master-1 devstack@n-api.service[138680]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-services:list": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:15:33.975303 ncdv-master-1 devstack@n-api.service[138680]: warnings.warn(msg) Jul 16 21:15:33.976696 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b2779716-d1a9-41c0-b933-e76675a769a4 tempest-LiveMigrationTest-1131758637 tempest-LiveMigrationTest-1131758637-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:33.977718 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b2779716-d1a9-41c0-b933-e76675a769a4 tempest-LiveMigrationTest-1131758637 tempest-LiveMigrationTest-1131758637-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:33.978540 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b2779716-d1a9-41c0-b933-e76675a769a4 tempest-LiveMigrationTest-1131758637 tempest-LiveMigrationTest-1131758637-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:33.978979 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b2779716-d1a9-41c0-b933-e76675a769a4 tempest-LiveMigrationTest-1131758637 tempest-LiveMigrationTest-1131758637-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:34.043143 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-b2779716-d1a9-41c0-b933-e76675a769a4 tempest-LiveMigrationTest-1131758637 tempest-LiveMigrationTest-1131758637-project-admin] 10.222.0.16 "GET /compute/v2.1/os-services?binary=nova-compute" status: 200 len: 380 microversion: 2.1 time: 0.078560 Jul 16 21:15:34.043789 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 892/1733] 10.222.0.16 () {58 vars in 1175 bytes} [Sat Jul 16 21:15:33 2022] GET /compute/v2.1/os-services?binary=nova-compute => generated 380 bytes in 80 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:15:34.057215 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-e96ac42a-e7b1-4d3a-b695-fb21e6b057ef req-4b77103e-3d5e-4fde-88f6-41fd8da77808 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "0079e567-30df-4cac-a0ad-4379bcbbcf48", "name": "network-vif-plugged", "status": "completed", "tag": "08381e59-62d7-471e-a05a-7de705ec2dae"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:15:34.073303 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-e96ac42a-e7b1-4d3a-b695-fb21e6b057ef req-4b77103e-3d5e-4fde-88f6-41fd8da77808 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:34.074148 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-e96ac42a-e7b1-4d3a-b695-fb21e6b057ef req-4b77103e-3d5e-4fde-88f6-41fd8da77808 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:34.092669 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-e96ac42a-e7b1-4d3a-b695-fb21e6b057ef req-4b77103e-3d5e-4fde-88f6-41fd8da77808 service nova] Creating event network-vif-plugged:08381e59-62d7-471e-a05a-7de705ec2dae for instance 0079e567-30df-4cac-a0ad-4379bcbbcf48 on nch2-master-1 Jul 16 21:15:34.102364 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-e96ac42a-e7b1-4d3a-b695-fb21e6b057ef req-4b77103e-3d5e-4fde-88f6-41fd8da77808 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.053170 Jul 16 21:15:34.102364 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 893/1734] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:15:34 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:15:34.108629 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-15fe3e6b-cb50-4a8f-b596-2a092a47c148 tempest-KeyPairsNegativeTestJSON-2037449827 tempest-KeyPairsNegativeTestJSON-2037449827-project] Calling method '.version_select at 0x7fdd9d025550>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:15:34.131177 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-15fe3e6b-cb50-4a8f-b596-2a092a47c148 tempest-KeyPairsNegativeTestJSON-2037449827 tempest-KeyPairsNegativeTestJSON-2037449827-project] HTTP exception thrown: Keypair tempest-keypair-non-existent-2126067516 not found for user 29baab42d8c64f5480701a768f793d5e Jul 16 21:15:34.131177 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-15fe3e6b-cb50-4a8f-b596-2a092a47c148 tempest-KeyPairsNegativeTestJSON-2037449827 tempest-KeyPairsNegativeTestJSON-2037449827-project] Request method failure captured: Jul 16 21:15:34.131177 ncdv-master-1 devstack@n-api.service[138680]: request: DELETE /compute/v2.1/os-keypairs/tempest-keypair-non-existent-2126067516 HTTP/1.1 Jul 16 21:15:34.131177 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:15:34.131177 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:15:34.131177 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:15:34.131177 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 0 Jul 16 21:15:34.131177 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:15:34.131177 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:15:34.131177 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:15:34.131177 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:15:34.131177 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0ypqWdrcrtbXaSq2GEpWPon25FMLnHNO2qMo-SvtJJgkVPuxEq-ypCmZ__i9RWDig4ZDfMaeDQTVbW5PHkz1pceOF1CtLJFW3A3oLh4lnXqlF8rU0ZGZuPihQd9gGmqkk_eaWH1_Ok_SeKFxd3pRS6aDEwNoV2B4X0519r5BRpQwhSI Jul 16 21:15:34.131177 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:15:34.131177 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:15:34.131177 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:15:34.131177 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:15:34.131177 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:15:34.131177 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:15:34.131177 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: 71aa49f99a7647329b40abfd7e10a606 Jul 16 21:15:34.131177 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-KeyPairsNegativeTestJSON-2037449827 Jul 16 21:15:34.131177 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:15:34.131177 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:15:34.131177 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/71aa49f99a7647329b40abfd7e10a606"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/71aa49f99a7647329b40abfd7e10a606"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_71aa49f99a7647329b40abfd7e10a606"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/71aa49f99a7647329b40abfd7e10a606"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:15:34.131177 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-KeyPairsNegativeTestJSON-2037449827 Jul 16 21:15:34.131177 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: 71aa49f99a7647329b40abfd7e10a606 Jul 16 21:15:34.131177 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-KeyPairsNegativeTestJSON-2037449827 Jul 16 21:15:34.131177 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-KeyPairsNegativeTestJSON-2037449827-project Jul 16 21:15:34.131177 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:15:34.131177 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:15:34.131177 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: 29baab42d8c64f5480701a768f793d5e Jul 16 21:15:34.131177 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-KeyPairsNegativeTestJSON-2037449827-project Jul 16 21:15:34.131177 ncdv-master-1 devstack@n-api.service[138680]: method: Jul 16 21:15:34.131177 ncdv-master-1 devstack@n-api.service[138680]: exception: Keypair tempest-keypair-non-existent-2126067516 not found for user 29baab42d8c64f5480701a768f793d5e Jul 16 21:15:34.131177 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': 'tempest-keypair-non-existent-2126067516'} Jul 16 21:15:34.131177 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:15:34.131177 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:15:34.131177 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/keypairs.py", line 164, in _delete Jul 16 21:15:34.131177 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi self.api.delete_key_pair(context, user_id, id) Jul 16 21:15:34.131177 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/exception_wrapper.py", line 71, in wrapped Jul 16 21:15:34.131177 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi _emit_versioned_exception_notification( Jul 16 21:15:34.131177 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 16 21:15:34.131177 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi self.force_reraise() Jul 16 21:15:34.131177 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 16 21:15:34.131177 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise self.value Jul 16 21:15:34.131177 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/exception_wrapper.py", line 63, in wrapped Jul 16 21:15:34.131177 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(self, context, *args, **kw) Jul 16 21:15:34.131177 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 6745, in delete_key_pair Jul 16 21:15:34.131177 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi keypair = self.get_key_pair(context, user_id, key_name) Jul 16 21:15:34.131177 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 6766, in get_key_pair Jul 16 21:15:34.131177 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return objects.KeyPair.get_by_name(context, user_id, key_name) Jul 16 21:15:34.131177 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Jul 16 21:15:34.131177 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Jul 16 21:15:34.131177 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/keypair.py", line 144, in get_by_name Jul 16 21:15:34.131177 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi db_keypair = cls._get_from_db(context, user_id, name) Jul 16 21:15:34.131177 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/keypair.py", line 126, in _get_from_db Jul 16 21:15:34.131177 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return _get_from_db(context, user_id, name=name) Jul 16 21:15:34.131177 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1010, in wrapper Jul 16 21:15:34.131177 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return fn(*args, **kwargs) Jul 16 21:15:34.131177 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/keypair.py", line 40, in _get_from_db Jul 16 21:15:34.131177 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.KeypairNotFound(user_id=user_id, name=name) Jul 16 21:15:34.135997 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.KeypairNotFound: Keypair tempest-keypair-non-existent-2126067516 not found for user 29baab42d8c64f5480701a768f793d5e Jul 16 21:15:34.135997 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:15:34.135997 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:15:34.135997 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:15:34.135997 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:15:34.135997 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:15:34.135997 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:15:34.135997 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:15:34.135997 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:15:34.135997 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Jul 16 21:15:34.135997 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Jul 16 21:15:34.135997 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper Jul 16 21:15:34.135997 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 16 21:15:34.135997 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:15:34.135997 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:15:34.135997 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/keypairs.py", line 137, in delete Jul 16 21:15:34.135997 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi self._delete(req, id) Jul 16 21:15:34.135997 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/keypairs.py", line 166, in _delete Jul 16 21:15:34.135997 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=exc.format_message()) Jul 16 21:15:34.135997 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Keypair tempest-keypair-non-existent-2126067516 not found for user 29baab42d8c64f5480701a768f793d5e Jul 16 21:15:34.135997 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:15:34.135997 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:15:34.135997 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:15:34.135997 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:15:34.135997 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:15:34.135997 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:15:34.135997 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:15:34.135997 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:15:34.135997 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Keypair tempest-keypair-non-existent-2126067516 not found for user 29baab42d8c64f5480701a768f793d5e Jul 16 21:15:34.135997 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:15:34.135997 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-15fe3e6b-cb50-4a8f-b596-2a092a47c148 tempest-KeyPairsNegativeTestJSON-2037449827 tempest-KeyPairsNegativeTestJSON-2037449827-project] Returning 404 to user: Keypair tempest-keypair-non-existent-2126067516 not found for user 29baab42d8c64f5480701a768f793d5e {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:15:34.135997 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-15fe3e6b-cb50-4a8f-b596-2a092a47c148 tempest-KeyPairsNegativeTestJSON-2037449827 tempest-KeyPairsNegativeTestJSON-2037449827-project] 10.222.0.16 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-non-existent-2126067516" status: 404 len: 145 microversion: 2.1 time: 0.027172 Jul 16 21:15:34.135997 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 894/1735] 10.222.0.16 () {58 vars in 1259 bytes} [Sat Jul 16 21:15:34 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-non-existent-2126067516 => generated 145 bytes in 28 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:15:34.144322 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-03f332d7-bdb3-4463-87f2-ffeec28171ec tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:15:34.145204 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-03f332d7-bdb3-4463-87f2-ffeec28171ec tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] [instance: 84ab7e65-1bf0-4b8b-baea-6d1d4ca13dd7] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:15:34.153232 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-03f332d7-bdb3-4463-87f2-ffeec28171ec tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:34.153818 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-03f332d7-bdb3-4463-87f2-ffeec28171ec tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:34.154708 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-03f332d7-bdb3-4463-87f2-ffeec28171ec tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:34.155251 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-03f332d7-bdb3-4463-87f2-ffeec28171ec tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:34.531585 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.objects.instance [None req-03f332d7-bdb3-4463-87f2-ffeec28171ec tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] Lazy-loading 'services' on Instance uuid 84ab7e65-1bf0-4b8b-baea-6d1d4ca13dd7 {{(pid=138680) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:15:34.532580 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-ff7a328a-4c30-45cf-902f-89ea65f03cae tempest-FloatingIPDetailsTestJSON-1149902298 tempest-FloatingIPDetailsTestJSON-1149902298-project] 10.222.0.16 "POST /compute/v2.1/os-floating-ips" status: 200 len: 141 microversion: 2.1 time: 3.972136 Jul 16 21:15:34.541552 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 842/1736] 10.222.0.16 () {60 vars in 1169 bytes} [Sat Jul 16 21:15:30 2022] POST /compute/v2.1/os-floating-ips => generated 141 bytes in 3982 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:15:34.549574 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-c3fb9817-67bf-4a4e-8f59-e2f7dada041a tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Calling method '.version_select at 0x7fdd9cdca940>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:15:34.557139 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-03f332d7-bdb3-4463-87f2-ffeec28171ec tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/84ab7e65-1bf0-4b8b-baea-6d1d4ca13dd7" status: 200 len: 1970 microversion: 2.25 time: 0.425545 Jul 16 21:15:34.559099 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 895/1737] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:15:34 2022] GET /compute/v2.1/servers/84ab7e65-1bf0-4b8b-baea-6d1d4ca13dd7 => generated 1970 bytes in 428 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:15:34.567449 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-97da3760-d226-4c1e-addd-064a036877fc tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:15:34.568319 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-97da3760-d226-4c1e-addd-064a036877fc tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] [instance: 0079e567-30df-4cac-a0ad-4379bcbbcf48] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:15:34.578337 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-97da3760-d226-4c1e-addd-064a036877fc tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:34.578930 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-97da3760-d226-4c1e-addd-064a036877fc tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:34.582011 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-97da3760-d226-4c1e-addd-064a036877fc tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:34.582011 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-97da3760-d226-4c1e-addd-064a036877fc tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:34.592963 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-c3fb9817-67bf-4a4e-8f59-e2f7dada041a tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] HTTP exception thrown: Image not found. Jul 16 21:15:34.596792 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-c3fb9817-67bf-4a4e-8f59-e2f7dada041a tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Request method failure captured: Jul 16 21:15:34.596792 ncdv-master-1 devstack@n-api.service[138681]: request: GET /compute/v2.1/images/23300f39-e85b-4c2f-ad31-2d2a76ad6f94 HTTP/1.1 Jul 16 21:15:34.596792 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:15:34.596792 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:15:34.596792 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:15:34.596792 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 21:15:34.596792 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:15:34.596792 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:15:34.596792 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:15:34.596792 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:15:34.596792 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0yoJFUAf_8Wtq8fI-wZdJwoKyg8rUEOC_eBMeqcRE2hk6JM7_ssDplugQwmk2S9xH7Oys7Ho2tw0kc9hmlY-jX_rR-I-9WVwQYFbyogU7ki99siEGGR9aVA2GKsp5nNcVHXBtjM3bI_zfa21XSWedvaSIVwZV8kEGixKYsBf2GMYKVI Jul 16 21:15:34.596792 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:15:34.596792 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:15:34.596792 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:15:34.596792 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:15:34.596792 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:15:34.596792 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:15:34.596792 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: 4337575622d04a26942905c9dc044ba7 Jul 16 21:15:34.596792 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-ImagesOneServerNegativeTestJSON-526509748 Jul 16 21:15:34.596792 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:15:34.596792 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:15:34.596792 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/4337575622d04a26942905c9dc044ba7"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/4337575622d04a26942905c9dc044ba7"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_4337575622d04a26942905c9dc044ba7"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/4337575622d04a26942905c9dc044ba7"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:15:34.596792 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-ImagesOneServerNegativeTestJSON-526509748 Jul 16 21:15:34.596792 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: 4337575622d04a26942905c9dc044ba7 Jul 16 21:15:34.596792 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-ImagesOneServerNegativeTestJSON-526509748 Jul 16 21:15:34.596792 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-ImagesOneServerNegativeTestJSON-526509748-project Jul 16 21:15:34.596792 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:15:34.596792 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:15:34.596792 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: 4e9df82f95ac4779859972403869096a Jul 16 21:15:34.596792 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-ImagesOneServerNegativeTestJSON-526509748-project Jul 16 21:15:34.596792 ncdv-master-1 devstack@n-api.service[138681]: method: Jul 16 21:15:34.596792 ncdv-master-1 devstack@n-api.service[138681]: exception: Image not found. Jul 16 21:15:34.596792 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': '23300f39-e85b-4c2f-ad31-2d2a76ad6f94'} Jul 16 21:15:34.596792 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:15:34.596792 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:15:34.596792 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 285, in show Jul 16 21:15:34.596792 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi image = self._client.call(context, 2, 'get', args=(image_id,)) Jul 16 21:15:34.596792 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 191, in call Jul 16 21:15:34.596792 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi result = getattr(controller, method)(*args, **kwargs) Jul 16 21:15:34.596792 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/v2/images.py", line 197, in get Jul 16 21:15:34.596792 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return self._get(image_id) Jul 16 21:15:34.596792 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/utils.py", line 670, in inner Jul 16 21:15:34.596792 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return RequestIdProxy(wrapped(*args, **kwargs)) Jul 16 21:15:34.596792 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/v2/images.py", line 190, in _get Jul 16 21:15:34.596792 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi resp, body = self.http_client.get(url, headers=header) Jul 16 21:15:34.596792 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/adapter.py", line 395, in get Jul 16 21:15:34.596792 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return self.request(url, 'GET', **kwargs) Jul 16 21:15:34.596792 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 380, in request Jul 16 21:15:34.596792 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return self._handle_response(resp) Jul 16 21:15:34.596792 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 120, in _handle_response Jul 16 21:15:34.596792 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.from_response(resp, resp.content) Jul 16 21:15:34.596792 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi glanceclient.exc.HTTPNotFound: HTTP 404 Not Found: No image found with ID 23300f39-e85b-4c2f-ad31-2d2a76ad6f94 Jul 16 21:15:34.596792 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:15:34.596792 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:15:34.596792 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:15:34.596792 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:15:34.596792 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/images.py", line 86, in show Jul 16 21:15:34.596792 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi image = self._image_api.get(context, id) Jul 16 21:15:34.596792 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 1205, in get Jul 16 21:15:34.596792 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return session.show(context, image_id, Jul 16 21:15:34.596792 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 287, in show Jul 16 21:15:34.596792 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi _reraise_translated_image_exception(image_id) Jul 16 21:15:34.596792 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 1031, in _reraise_translated_image_exception Jul 16 21:15:34.610293 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise new_exc.with_traceback(exc_trace) Jul 16 21:15:34.610293 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 285, in show Jul 16 21:15:34.610293 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi image = self._client.call(context, 2, 'get', args=(image_id,)) Jul 16 21:15:34.610293 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 191, in call Jul 16 21:15:34.610293 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi result = getattr(controller, method)(*args, **kwargs) Jul 16 21:15:34.610293 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/v2/images.py", line 197, in get Jul 16 21:15:34.610293 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return self._get(image_id) Jul 16 21:15:34.610293 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/utils.py", line 670, in inner Jul 16 21:15:34.610293 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return RequestIdProxy(wrapped(*args, **kwargs)) Jul 16 21:15:34.610293 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/v2/images.py", line 190, in _get Jul 16 21:15:34.610293 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi resp, body = self.http_client.get(url, headers=header) Jul 16 21:15:34.610293 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/adapter.py", line 395, in get Jul 16 21:15:34.610293 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return self.request(url, 'GET', **kwargs) Jul 16 21:15:34.610293 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 380, in request Jul 16 21:15:34.610293 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return self._handle_response(resp) Jul 16 21:15:34.610293 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 120, in _handle_response Jul 16 21:15:34.610293 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.from_response(resp, resp.content) Jul 16 21:15:34.610293 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.ImageNotFound: Image 23300f39-e85b-4c2f-ad31-2d2a76ad6f94 could not be found. Jul 16 21:15:34.610293 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:15:34.610293 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:15:34.610293 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:15:34.610293 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:15:34.610293 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:15:34.610293 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:15:34.610293 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:15:34.610293 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:15:34.610293 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Jul 16 21:15:34.610293 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Jul 16 21:15:34.610293 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:15:34.610293 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:15:34.610293 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/images.py", line 89, in show Jul 16 21:15:34.610293 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=explanation) Jul 16 21:15:34.610293 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Image not found. Jul 16 21:15:34.610293 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:15:34.610293 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:15:34.610293 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:15:34.610293 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:15:34.610293 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:15:34.610293 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:15:34.610293 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:15:34.610293 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:15:34.610293 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Image not found. Jul 16 21:15:34.610293 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:15:34.610293 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-c3fb9817-67bf-4a4e-8f59-e2f7dada041a tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Returning 404 to user: Image not found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:15:34.610293 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-c3fb9817-67bf-4a4e-8f59-e2f7dada041a tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] 10.222.0.16 "GET /compute/v2.1/images/23300f39-e85b-4c2f-ad31-2d2a76ad6f94" status: 404 len: 62 microversion: 2.1 time: 0.058719 Jul 16 21:15:34.610293 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 843/1738] 10.222.0.16 () {58 vars in 1232 bytes} [Sat Jul 16 21:15:34 2022] GET /compute/v2.1/images/23300f39-e85b-4c2f-ad31-2d2a76ad6f94 => generated 62 bytes in 60 msecs (HTTP/1.1 404) 9 headers in 378 bytes (1 switches on core 0) Jul 16 21:15:34.617532 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-04a95fcb-5164-4efb-833f-82c94fd8a734 tempest-LiveMigrationTest-1131758637 tempest-LiveMigrationTest-1131758637-project-admin] Action: 'action', calling method: >, body: {"os-migrateLive": {"host": "nch1-master-1", "block_migration": false, "disk_over_commit": false}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:15:34.619707 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-04a95fcb-5164-4efb-833f-82c94fd8a734 tempest-LiveMigrationTest-1131758637 tempest-LiveMigrationTest-1131758637-project-admin] [instance: 23df6e0a-3637-4f59-93f9-b02dd614b5c3] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:15:34.632097 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-04a95fcb-5164-4efb-833f-82c94fd8a734 tempest-LiveMigrationTest-1131758637 tempest-LiveMigrationTest-1131758637-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:34.632908 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-04a95fcb-5164-4efb-833f-82c94fd8a734 tempest-LiveMigrationTest-1131758637 tempest-LiveMigrationTest-1131758637-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:34.634040 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-04a95fcb-5164-4efb-833f-82c94fd8a734 tempest-LiveMigrationTest-1131758637 tempest-LiveMigrationTest-1131758637-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:34.635177 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-04a95fcb-5164-4efb-833f-82c94fd8a734 tempest-LiveMigrationTest-1131758637 tempest-LiveMigrationTest-1131758637-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:34.657603 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=100 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:15:34.667234 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.objects.instance [None req-04a95fcb-5164-4efb-833f-82c94fd8a734 tempest-LiveMigrationTest-1131758637 tempest-LiveMigrationTest-1131758637-project-admin] Lazy-loading 'flavor' on Instance uuid 23df6e0a-3637-4f59-93f9-b02dd614b5c3 {{(pid=138681) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:15:34.683433 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-04a95fcb-5164-4efb-833f-82c94fd8a734 tempest-LiveMigrationTest-1131758637 tempest-LiveMigrationTest-1131758637-project-admin] [instance: 23df6e0a-3637-4f59-93f9-b02dd614b5c3] Going to try to live migrate instance to nch1-master-1 {{(pid=138681) live_migrate /opt/stack/nova/nova/compute/api.py:5349}} Jul 16 21:15:34.824828 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-97da3760-d226-4c1e-addd-064a036877fc tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '53af488f3b0242a797f539284b78d59e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bb1ede0d0b6941cc97278e6c617f9b01', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:15:34.837089 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-97da3760-d226-4c1e-addd-064a036877fc tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] 10.222.0.16 "GET /compute/v2.1/servers/0079e567-30df-4cac-a0ad-4379bcbbcf48" status: 200 len: 1334 microversion: 2.1 time: 0.275526 Jul 16 21:15:34.846972 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 896/1739] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:15:34 2022] GET /compute/v2.1/servers/0079e567-30df-4cac-a0ad-4379bcbbcf48 => generated 1334 bytes in 278 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:15:35.246378 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-c0055d7d-e60e-4fcf-be81-91b995886647 tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] Action: 'create', calling method: .version_select at 0x7fdd9cc3f8b0>, body: {"keypair": {"name": "tempest-keypair-1395237950", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBFRHh5fMMq8/VQ3s1X0YtGogS/eeyOhyfeWVvKFAk/eCGNBYgBFpopohN6vgKfifGNBmbsKnRejJ2TE5Q5BiCe2vucOXIaOe++qjMJqbpG9yIltLvLAQ8D+S0xhqevdNtA=="}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:15:35.273707 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-c0055d7d-e60e-4fcf-be81-91b995886647 tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] Getting quotas for project d82c2d92c67f469e863eb70bb7f9af84. Resources: {'key_pairs'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:15:35.280032 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-c0055d7d-e60e-4fcf-be81-91b995886647 tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] Getting quotas for user 1aa99be675e946fd995d71465b4cbec6 and project d82c2d92c67f469e863eb70bb7f9af84. Resources: {'key_pairs'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:15:35.403689 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-c0055d7d-e60e-4fcf-be81-91b995886647 tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] 10.222.0.16 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.564324 Jul 16 21:15:35.404418 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 897/1740] 10.222.0.16 () {60 vars in 1158 bytes} [Sat Jul 16 21:15:34 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 566 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:15:35.412118 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-1b813c0b-08a3-480b-ba69-fa10545790ac tempest-FloatingIPDetailsTestJSON-1149902298 tempest-FloatingIPDetailsTestJSON-1149902298-project] Action: 'create', calling method: .version_select at 0x7fdda2bc1040>, body: {"pool": null} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:15:38.275197 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-1b813c0b-08a3-480b-ba69-fa10545790ac tempest-FloatingIPDetailsTestJSON-1149902298 tempest-FloatingIPDetailsTestJSON-1149902298-project] 10.222.0.16 "POST /compute/v2.1/os-floating-ips" status: 200 len: 142 microversion: 2.1 time: 2.869405 Jul 16 21:15:38.278683 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 898/1741] 10.222.0.16 () {60 vars in 1169 bytes} [Sat Jul 16 21:15:35 2022] POST /compute/v2.1/os-floating-ips => generated 142 bytes in 2872 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:15:38.282117 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-97e83391-25d8-4b56-a186-c4b7dc15b6b8 tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:15:38.282811 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-97e83391-25d8-4b56-a186-c4b7dc15b6b8 tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] [instance: 84ab7e65-1bf0-4b8b-baea-6d1d4ca13dd7] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:15:38.294648 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-97e83391-25d8-4b56-a186-c4b7dc15b6b8 tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:38.298587 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-97e83391-25d8-4b56-a186-c4b7dc15b6b8 tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:38.299851 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-97e83391-25d8-4b56-a186-c4b7dc15b6b8 tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:38.300504 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-97e83391-25d8-4b56-a186-c4b7dc15b6b8 tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:38.579510 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.objects.instance [None req-97e83391-25d8-4b56-a186-c4b7dc15b6b8 tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] Lazy-loading 'services' on Instance uuid 84ab7e65-1bf0-4b8b-baea-6d1d4ca13dd7 {{(pid=138680) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:15:38.609014 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-97e83391-25d8-4b56-a186-c4b7dc15b6b8 tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/84ab7e65-1bf0-4b8b-baea-6d1d4ca13dd7" status: 200 len: 1970 microversion: 2.25 time: 0.330661 Jul 16 21:15:38.609942 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 899/1742] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:15:38 2022] GET /compute/v2.1/servers/84ab7e65-1bf0-4b8b-baea-6d1d4ca13dd7 => generated 1970 bytes in 332 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:15:38.628384 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-d7105c2d-4933-49f9-92d1-6adca8c7bf51 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:15:38.629074 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-d7105c2d-4933-49f9-92d1-6adca8c7bf51 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] [instance: 39e51d01-21e0-4dd5-bfd4-16b0f8b0f827] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:15:38.648510 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d7105c2d-4933-49f9-92d1-6adca8c7bf51 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:38.649097 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d7105c2d-4933-49f9-92d1-6adca8c7bf51 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:38.650006 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d7105c2d-4933-49f9-92d1-6adca8c7bf51 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:38.650527 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d7105c2d-4933-49f9-92d1-6adca8c7bf51 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:38.687876 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:15:38.917170 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-d7105c2d-4933-49f9-92d1-6adca8c7bf51 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4e9df82f95ac4779859972403869096a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4337575622d04a26942905c9dc044ba7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:15:38.945018 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-d7105c2d-4933-49f9-92d1-6adca8c7bf51 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] 10.222.0.16 "GET /compute/v2.1/servers/39e51d01-21e0-4dd5-bfd4-16b0f8b0f827" status: 200 len: 1521 microversion: 2.1 time: 0.321922 Jul 16 21:15:38.945018 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 900/1743] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:15:38 2022] GET /compute/v2.1/servers/39e51d01-21e0-4dd5-bfd4-16b0f8b0f827 => generated 1521 bytes in 323 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:15:38.948588 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-aaef3810-808c-40c0-9610-41c3aa2055ce tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:15:38.949286 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-aaef3810-808c-40c0-9610-41c3aa2055ce tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] [instance: 93e327c3-c63d-439a-9b25-1a805f785949] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:15:38.956823 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-aaef3810-808c-40c0-9610-41c3aa2055ce tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:38.960156 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-aaef3810-808c-40c0-9610-41c3aa2055ce tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:38.961865 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-aaef3810-808c-40c0-9610-41c3aa2055ce tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:38.962666 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-aaef3810-808c-40c0-9610-41c3aa2055ce tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:39.142612 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-aaef3810-808c-40c0-9610-41c3aa2055ce tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'daf3c9e6634e4ae1bfb492eae1f44c93', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '24c64db8082648c8837c16352252f96d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:15:39.149759 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-aaef3810-808c-40c0-9610-41c3aa2055ce tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] 10.222.0.16 "GET /compute/v2.1/servers/93e327c3-c63d-439a-9b25-1a805f785949" status: 200 len: 1308 microversion: 2.1 time: 0.214800 Jul 16 21:15:39.150351 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 901/1744] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:15:38 2022] GET /compute/v2.1/servers/93e327c3-c63d-439a-9b25-1a805f785949 => generated 1308 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:15:39.154822 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-85a76217-d3b8-4db1-a2d1-19cffa0148d1 req-1d3728b5-f928-4b91-ab52-3eaa5fbb9fa8 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "93e327c3-c63d-439a-9b25-1a805f785949", "tag": "4a074ea9-9de4-4dc1-aba4-632bf2d90076"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:15:39.164617 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-85a76217-d3b8-4db1-a2d1-19cffa0148d1 req-1d3728b5-f928-4b91-ab52-3eaa5fbb9fa8 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:39.165250 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-85a76217-d3b8-4db1-a2d1-19cffa0148d1 req-1d3728b5-f928-4b91-ab52-3eaa5fbb9fa8 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:39.187697 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-85a76217-d3b8-4db1-a2d1-19cffa0148d1 req-1d3728b5-f928-4b91-ab52-3eaa5fbb9fa8 service nova] Creating event network-changed:4a074ea9-9de4-4dc1-aba4-632bf2d90076 for instance 93e327c3-c63d-439a-9b25-1a805f785949 on nch1-master-1 Jul 16 21:15:39.196470 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-85a76217-d3b8-4db1-a2d1-19cffa0148d1 req-1d3728b5-f928-4b91-ab52-3eaa5fbb9fa8 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.043564 Jul 16 21:15:39.201345 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 902/1745] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:15:39 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:15:39.221454 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-38c0af77-0591-4856-a777-9670b7cfcb36 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:15:39.221454 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-38c0af77-0591-4856-a777-9670b7cfcb36 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] [instance: 0079e567-30df-4cac-a0ad-4379bcbbcf48] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:15:39.239428 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-38c0af77-0591-4856-a777-9670b7cfcb36 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:39.240325 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-38c0af77-0591-4856-a777-9670b7cfcb36 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:39.241381 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-38c0af77-0591-4856-a777-9670b7cfcb36 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:39.242003 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-38c0af77-0591-4856-a777-9670b7cfcb36 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:39.488573 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-04a95fcb-5164-4efb-833f-82c94fd8a734 tempest-LiveMigrationTest-1131758637 tempest-LiveMigrationTest-1131758637-project-admin] 10.222.0.16 "POST /compute/v2.1/servers/23df6e0a-3637-4f59-93f9-b02dd614b5c3/action" status: 202 len: 0 microversion: 2.1 time: 4.883516 Jul 16 21:15:39.489792 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 844/1746] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 21:15:34 2022] POST /compute/v2.1/servers/23df6e0a-3637-4f59-93f9-b02dd614b5c3/action => generated 0 bytes in 4887 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 21:15:39.496950 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-614f3bbe-2b7d-4075-95ad-7c4a609aaaa6 req-aef4dab2-2a9e-49bc-96ad-bb5f8286404e service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "0079e567-30df-4cac-a0ad-4379bcbbcf48", "name": "network-vif-plugged", "status": "completed", "tag": "1079a73b-5553-45b8-bff2-e4be618fbceb"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:15:39.520083 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-614f3bbe-2b7d-4075-95ad-7c4a609aaaa6 req-aef4dab2-2a9e-49bc-96ad-bb5f8286404e service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:39.521074 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-614f3bbe-2b7d-4075-95ad-7c4a609aaaa6 req-aef4dab2-2a9e-49bc-96ad-bb5f8286404e service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:39.547824 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-614f3bbe-2b7d-4075-95ad-7c4a609aaaa6 req-aef4dab2-2a9e-49bc-96ad-bb5f8286404e service nova] Creating event network-vif-plugged:1079a73b-5553-45b8-bff2-e4be618fbceb for instance 0079e567-30df-4cac-a0ad-4379bcbbcf48 on nch2-master-1 Jul 16 21:15:39.561274 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-614f3bbe-2b7d-4075-95ad-7c4a609aaaa6 req-aef4dab2-2a9e-49bc-96ad-bb5f8286404e service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.067818 Jul 16 21:15:39.563707 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 845/1747] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:15:39 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 74 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:15:39.574223 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-38c0af77-0591-4856-a777-9670b7cfcb36 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '53af488f3b0242a797f539284b78d59e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bb1ede0d0b6941cc97278e6c617f9b01', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:15:39.575347 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-d9884abf-5dd7-4d6f-9643-5399a8d14b6a tempest-FloatingIPDetailsTestJSON-1149902298 tempest-FloatingIPDetailsTestJSON-1149902298-project] Calling method '.version_select at 0x7fdda2b35670>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:15:39.588942 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-38c0af77-0591-4856-a777-9670b7cfcb36 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] 10.222.0.16 "GET /compute/v2.1/servers/0079e567-30df-4cac-a0ad-4379bcbbcf48" status: 200 len: 1334 microversion: 2.1 time: 0.384128 Jul 16 21:15:39.590196 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 903/1748] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:15:39 2022] GET /compute/v2.1/servers/0079e567-30df-4cac-a0ad-4379bcbbcf48 => generated 1334 bytes in 388 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:15:39.618779 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-84fc7dc4-56f7-44b3-9059-78458d9a842c tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:15:39.624733 ncdv-master-1 devstack@n-api.service[138680]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-services:list": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:15:39.624733 ncdv-master-1 devstack@n-api.service[138680]: warnings.warn(msg) Jul 16 21:15:39.629556 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-84fc7dc4-56f7-44b3-9059-78458d9a842c tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:39.631723 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-84fc7dc4-56f7-44b3-9059-78458d9a842c tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:39.632840 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-84fc7dc4-56f7-44b3-9059-78458d9a842c tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:39.637043 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-84fc7dc4-56f7-44b3-9059-78458d9a842c tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:39.667886 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-84fc7dc4-56f7-44b3-9059-78458d9a842c tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] 10.222.0.16 "GET /compute/v2.1/os-services?binary=nova-compute" status: 200 len: 424 microversion: 2.25 time: 0.053234 Jul 16 21:15:39.674536 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 904/1749] 10.222.0.16 () {60 vars in 1216 bytes} [Sat Jul 16 21:15:39 2022] GET /compute/v2.1/os-services?binary=nova-compute => generated 424 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 16 21:15:39.679868 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-6f0ad113-2651-4f87-a1d8-6a1314a4e094 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:15:39.680381 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-6f0ad113-2651-4f87-a1d8-6a1314a4e094 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] [instance: 39e51d01-21e0-4dd5-bfd4-16b0f8b0f827] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:15:39.686585 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6f0ad113-2651-4f87-a1d8-6a1314a4e094 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:39.687109 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6f0ad113-2651-4f87-a1d8-6a1314a4e094 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:39.688362 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6f0ad113-2651-4f87-a1d8-6a1314a4e094 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:39.688649 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6f0ad113-2651-4f87-a1d8-6a1314a4e094 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:39.727690 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-6f0ad113-2651-4f87-a1d8-6a1314a4e094 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] [instance: 39e51d01-21e0-4dd5-bfd4-16b0f8b0f827] Going to try to terminate instance {{(pid=138680) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 16 21:15:39.793195 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell0:SELECT=18 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:15:40.211840 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-6f0ad113-2651-4f87-a1d8-6a1314a4e094 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] 10.222.0.16 "DELETE /compute/v2.1/servers/39e51d01-21e0-4dd5-bfd4-16b0f8b0f827" status: 204 len: 0 microversion: 2.1 time: 0.535423 Jul 16 21:15:40.213952 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 905/1750] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:15:39 2022] DELETE /compute/v2.1/servers/39e51d01-21e0-4dd5-bfd4-16b0f8b0f827 => generated 0 bytes in 538 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 16 21:15:40.220078 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-af2824e5-2aa3-4ca8-8d2f-8f8bac7ecea6 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:15:40.220804 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-af2824e5-2aa3-4ca8-8d2f-8f8bac7ecea6 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] [instance: 23df6e0a-3637-4f59-93f9-b02dd614b5c3] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:15:40.239468 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-af2824e5-2aa3-4ca8-8d2f-8f8bac7ecea6 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:40.240086 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-af2824e5-2aa3-4ca8-8d2f-8f8bac7ecea6 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:40.240908 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-af2824e5-2aa3-4ca8-8d2f-8f8bac7ecea6 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:40.241359 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-af2824e5-2aa3-4ca8-8d2f-8f8bac7ecea6 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:40.351658 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-d9884abf-5dd7-4d6f-9643-5399a8d14b6a tempest-FloatingIPDetailsTestJSON-1149902298 tempest-FloatingIPDetailsTestJSON-1149902298-project] 10.222.0.16 "GET /compute/v2.1/os-floating-ips/ec94a932-da6d-4f9c-9ebe-eeec2637bda5" status: 200 len: 142 microversion: 2.1 time: 0.786501 Jul 16 21:15:40.354095 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 846/1751] 10.222.0.16 () {58 vars in 1259 bytes} [Sat Jul 16 21:15:39 2022] GET /compute/v2.1/os-floating-ips/ec94a932-da6d-4f9c-9ebe-eeec2637bda5 => generated 142 bytes in 790 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:15:40.359938 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-3b2a7d41-310c-48ce-b408-bf9f38c73a24 tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] Action: 'action', calling method: >, body: {"os-migrateLive": {"host": "nch2-master-1", "block_migration": "auto"}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:15:40.395484 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-3b2a7d41-310c-48ce-b408-bf9f38c73a24 tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] [instance: 84ab7e65-1bf0-4b8b-baea-6d1d4ca13dd7] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:15:40.452884 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3b2a7d41-310c-48ce-b408-bf9f38c73a24 tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:40.468528 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3b2a7d41-310c-48ce-b408-bf9f38c73a24 tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.015s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:40.472466 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3b2a7d41-310c-48ce-b408-bf9f38c73a24 tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:40.473398 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3b2a7d41-310c-48ce-b408-bf9f38c73a24 tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:40.538458 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.objects.instance [None req-3b2a7d41-310c-48ce-b408-bf9f38c73a24 tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] Lazy-loading 'flavor' on Instance uuid 84ab7e65-1bf0-4b8b-baea-6d1d4ca13dd7 {{(pid=138681) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:15:40.553935 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-3b2a7d41-310c-48ce-b408-bf9f38c73a24 tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] [instance: 84ab7e65-1bf0-4b8b-baea-6d1d4ca13dd7] Going to try to live migrate instance to nch2-master-1 {{(pid=138681) live_migrate /opt/stack/nova/nova/compute/api.py:5349}} Jul 16 21:15:40.931137 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-af2824e5-2aa3-4ca8-8d2f-8f8bac7ecea6 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a69f9dee03d43beaafa966216b9e289', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9b769bf8f3ea4e368bf68de1df6538f9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:15:40.942603 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-af2824e5-2aa3-4ca8-8d2f-8f8bac7ecea6 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] 10.222.0.16 "GET /compute/v2.1/servers/23df6e0a-3637-4f59-93f9-b02dd614b5c3" status: 200 len: 1389 microversion: 2.1 time: 0.727132 Jul 16 21:15:40.953325 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 906/1752] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:15:40 2022] GET /compute/v2.1/servers/23df6e0a-3637-4f59-93f9-b02dd614b5c3 => generated 1389 bytes in 735 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:15:40.965308 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-824116a6-e111-4fac-8884-1c1a6051c338 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:15:40.965963 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-824116a6-e111-4fac-8884-1c1a6051c338 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] [instance: 93e327c3-c63d-439a-9b25-1a805f785949] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:15:40.983874 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-824116a6-e111-4fac-8884-1c1a6051c338 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:40.986412 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-824116a6-e111-4fac-8884-1c1a6051c338 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:40.988627 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-824116a6-e111-4fac-8884-1c1a6051c338 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:40.990908 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-824116a6-e111-4fac-8884-1c1a6051c338 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:41.429042 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-824116a6-e111-4fac-8884-1c1a6051c338 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'daf3c9e6634e4ae1bfb492eae1f44c93', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '24c64db8082648c8837c16352252f96d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:15:41.437485 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-824116a6-e111-4fac-8884-1c1a6051c338 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] 10.222.0.16 "GET /compute/v2.1/servers/93e327c3-c63d-439a-9b25-1a805f785949" status: 200 len: 1308 microversion: 2.1 time: 0.486326 Jul 16 21:15:41.438154 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 907/1753] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:15:40 2022] GET /compute/v2.1/servers/93e327c3-c63d-439a-9b25-1a805f785949 => generated 1308 bytes in 488 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:15:41.451779 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-530c8f2d-7e23-4364-a14d-3006ade68fd1 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:15:41.452771 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-530c8f2d-7e23-4364-a14d-3006ade68fd1 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] [instance: 39e51d01-21e0-4dd5-bfd4-16b0f8b0f827] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:15:41.461455 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-530c8f2d-7e23-4364-a14d-3006ade68fd1 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:41.461964 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-530c8f2d-7e23-4364-a14d-3006ade68fd1 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:41.462762 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-530c8f2d-7e23-4364-a14d-3006ade68fd1 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:41.463435 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-530c8f2d-7e23-4364-a14d-3006ade68fd1 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:41.681462 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-530c8f2d-7e23-4364-a14d-3006ade68fd1 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4e9df82f95ac4779859972403869096a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4337575622d04a26942905c9dc044ba7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:15:41.693914 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-530c8f2d-7e23-4364-a14d-3006ade68fd1 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] 10.222.0.16 "GET /compute/v2.1/servers/39e51d01-21e0-4dd5-bfd4-16b0f8b0f827" status: 200 len: 1527 microversion: 2.1 time: 0.253803 Jul 16 21:15:41.695411 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 908/1754] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:15:41 2022] GET /compute/v2.1/servers/39e51d01-21e0-4dd5-bfd4-16b0f8b0f827 => generated 1527 bytes in 256 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:15:41.705640 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-f54b5001-c720-4ad4-8477-579ba66b199f tempest-FloatingIPDetailsTestJSON-1149902298 tempest-FloatingIPDetailsTestJSON-1149902298-project] Calling method '.version_select at 0x7fdd9d3284c0>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:15:43.199176 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-f54b5001-c720-4ad4-8477-579ba66b199f tempest-FloatingIPDetailsTestJSON-1149902298 tempest-FloatingIPDetailsTestJSON-1149902298-project] 10.222.0.16 "DELETE /compute/v2.1/os-floating-ips/ec94a932-da6d-4f9c-9ebe-eeec2637bda5" status: 202 len: 0 microversion: 2.1 time: 1.502378 Jul 16 21:15:43.200555 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 909/1755] 10.222.0.16 () {58 vars in 1262 bytes} [Sat Jul 16 21:15:41 2022] DELETE /compute/v2.1/os-floating-ips/ec94a932-da6d-4f9c-9ebe-eeec2637bda5 => generated 0 bytes in 1504 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 21:15:43.214267 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-efd77d50-8e1e-47be-b2a8-f938b60493fa tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:15:43.215296 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-efd77d50-8e1e-47be-b2a8-f938b60493fa tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] [instance: 0079e567-30df-4cac-a0ad-4379bcbbcf48] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:15:43.224336 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-efd77d50-8e1e-47be-b2a8-f938b60493fa tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:43.224841 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-efd77d50-8e1e-47be-b2a8-f938b60493fa tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:43.225649 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-efd77d50-8e1e-47be-b2a8-f938b60493fa tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:43.226093 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-efd77d50-8e1e-47be-b2a8-f938b60493fa tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:43.545965 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-efd77d50-8e1e-47be-b2a8-f938b60493fa tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '53af488f3b0242a797f539284b78d59e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bb1ede0d0b6941cc97278e6c617f9b01', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:15:43.565251 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-efd77d50-8e1e-47be-b2a8-f938b60493fa tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] 10.222.0.16 "GET /compute/v2.1/servers/0079e567-30df-4cac-a0ad-4379bcbbcf48" status: 200 len: 1670 microversion: 2.1 time: 0.362605 Jul 16 21:15:43.566263 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 910/1756] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:15:43 2022] GET /compute/v2.1/servers/0079e567-30df-4cac-a0ad-4379bcbbcf48 => generated 1670 bytes in 365 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:15:43.573380 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-aacd7816-9b48-4934-868a-02fe35bc2add req-770bc645-cc91-425f-8ecd-ff6c8051d02d service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "93e327c3-c63d-439a-9b25-1a805f785949", "name": "network-vif-plugged", "status": "completed", "tag": "4a074ea9-9de4-4dc1-aba4-632bf2d90076"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:15:43.585303 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-aacd7816-9b48-4934-868a-02fe35bc2add req-770bc645-cc91-425f-8ecd-ff6c8051d02d service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:43.586333 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-aacd7816-9b48-4934-868a-02fe35bc2add req-770bc645-cc91-425f-8ecd-ff6c8051d02d service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:43.608009 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-aacd7816-9b48-4934-868a-02fe35bc2add req-770bc645-cc91-425f-8ecd-ff6c8051d02d service nova] Creating event network-vif-plugged:4a074ea9-9de4-4dc1-aba4-632bf2d90076 for instance 93e327c3-c63d-439a-9b25-1a805f785949 on nch1-master-1 Jul 16 21:15:43.641276 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-aacd7816-9b48-4934-868a-02fe35bc2add req-770bc645-cc91-425f-8ecd-ff6c8051d02d service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.072885 Jul 16 21:15:43.643930 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 911/1757] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:15:43 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 77 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:15:43.654400 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-0724d06b-ac06-45bf-b63b-10426e63fa11 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:15:43.655585 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-0724d06b-ac06-45bf-b63b-10426e63fa11 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] [instance: 23df6e0a-3637-4f59-93f9-b02dd614b5c3] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:15:43.666592 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0724d06b-ac06-45bf-b63b-10426e63fa11 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:43.672944 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0724d06b-ac06-45bf-b63b-10426e63fa11 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:43.672944 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0724d06b-ac06-45bf-b63b-10426e63fa11 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:43.672944 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0724d06b-ac06-45bf-b63b-10426e63fa11 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:43.961113 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-0724d06b-ac06-45bf-b63b-10426e63fa11 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a69f9dee03d43beaafa966216b9e289', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9b769bf8f3ea4e368bf68de1df6538f9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:15:43.970788 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-0724d06b-ac06-45bf-b63b-10426e63fa11 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] 10.222.0.16 "GET /compute/v2.1/servers/23df6e0a-3637-4f59-93f9-b02dd614b5c3" status: 200 len: 1389 microversion: 2.1 time: 0.325171 Jul 16 21:15:43.974749 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 912/1758] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:15:43 2022] GET /compute/v2.1/servers/23df6e0a-3637-4f59-93f9-b02dd614b5c3 => generated 1389 bytes in 330 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:15:43.988372 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-5c7e39a7-96ed-4ee4-944e-a81804424464 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:15:43.989261 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-5c7e39a7-96ed-4ee4-944e-a81804424464 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] [instance: 93e327c3-c63d-439a-9b25-1a805f785949] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:15:44.004841 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5c7e39a7-96ed-4ee4-944e-a81804424464 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:44.006542 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5c7e39a7-96ed-4ee4-944e-a81804424464 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:44.006542 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5c7e39a7-96ed-4ee4-944e-a81804424464 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:44.006542 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5c7e39a7-96ed-4ee4-944e-a81804424464 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:44.240770 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-5c7e39a7-96ed-4ee4-944e-a81804424464 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'daf3c9e6634e4ae1bfb492eae1f44c93', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '24c64db8082648c8837c16352252f96d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:15:44.257499 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-5c7e39a7-96ed-4ee4-944e-a81804424464 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] 10.222.0.16 "GET /compute/v2.1/servers/93e327c3-c63d-439a-9b25-1a805f785949" status: 200 len: 1308 microversion: 2.1 time: 0.280562 Jul 16 21:15:44.261263 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 913/1759] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:15:43 2022] GET /compute/v2.1/servers/93e327c3-c63d-439a-9b25-1a805f785949 => generated 1308 bytes in 286 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:15:44.269118 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-8e2e746f-c1ee-4f3c-bf11-6cd7b02f42a4 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:15:44.272267 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-8e2e746f-c1ee-4f3c-bf11-6cd7b02f42a4 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] [instance: 39e51d01-21e0-4dd5-bfd4-16b0f8b0f827] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:15:44.288506 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_api:SELECT=98,nova_api:INSERT=1,nova_api:UPDATE=1 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:15:44.290699 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8e2e746f-c1ee-4f3c-bf11-6cd7b02f42a4 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:44.294153 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8e2e746f-c1ee-4f3c-bf11-6cd7b02f42a4 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:44.302972 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8e2e746f-c1ee-4f3c-bf11-6cd7b02f42a4 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:44.304249 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8e2e746f-c1ee-4f3c-bf11-6cd7b02f42a4 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:44.385315 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8e2e746f-c1ee-4f3c-bf11-6cd7b02f42a4 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:44.386152 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8e2e746f-c1ee-4f3c-bf11-6cd7b02f42a4 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:44.386891 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.objects.instance [None req-8e2e746f-c1ee-4f3c-bf11-6cd7b02f42a4 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lazy-loading 'fault' on Instance uuid 39e51d01-21e0-4dd5-bfd4-16b0f8b0f827 {{(pid=138680) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:15:44.502008 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-8e2e746f-c1ee-4f3c-bf11-6cd7b02f42a4 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4e9df82f95ac4779859972403869096a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4337575622d04a26942905c9dc044ba7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:15:44.531675 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-8e2e746f-c1ee-4f3c-bf11-6cd7b02f42a4 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] 10.222.0.16 "GET /compute/v2.1/servers/39e51d01-21e0-4dd5-bfd4-16b0f8b0f827" status: 200 len: 1315 microversion: 2.1 time: 0.268878 Jul 16 21:15:44.551570 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 914/1760] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:15:44 2022] GET /compute/v2.1/servers/39e51d01-21e0-4dd5-bfd4-16b0f8b0f827 => generated 1315 bytes in 284 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:15:44.573917 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-50444680-dfd3-402c-9dff-26db3d14c133 tempest-FloatingIPDetailsTestJSON-1149902298 tempest-FloatingIPDetailsTestJSON-1149902298-project] Calling method '.version_select at 0x7fdda2b7faf0>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:15:45.290345 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell0:SELECT=15 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:15:45.510566 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-50444680-dfd3-402c-9dff-26db3d14c133 tempest-FloatingIPDetailsTestJSON-1149902298 tempest-FloatingIPDetailsTestJSON-1149902298-project] 10.222.0.16 "GET /compute/v2.1/os-floating-ip-pools" status: 200 len: 43 microversion: 2.1 time: 0.950269 Jul 16 21:15:45.511115 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 915/1761] 10.222.0.16 () {58 vars in 1163 bytes} [Sat Jul 16 21:15:44 2022] GET /compute/v2.1/os-floating-ip-pools => generated 43 bytes in 964 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Jul 16 21:15:45.516252 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-97260a55-8b10-43de-8f37-4667b0847b5c req-101ac501-726b-4cde-8dda-df76806dee18 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "39e51d01-21e0-4dd5-bfd4-16b0f8b0f827", "name": "network-vif-deleted", "tag": "55c3f33b-16c2-42c1-94d7-42da613add6d"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:15:45.526924 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-97260a55-8b10-43de-8f37-4667b0847b5c req-101ac501-726b-4cde-8dda-df76806dee18 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:45.527668 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-97260a55-8b10-43de-8f37-4667b0847b5c req-101ac501-726b-4cde-8dda-df76806dee18 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:45.543024 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-97260a55-8b10-43de-8f37-4667b0847b5c req-101ac501-726b-4cde-8dda-df76806dee18 service nova] Creating event network-vif-deleted:55c3f33b-16c2-42c1-94d7-42da613add6d for instance 39e51d01-21e0-4dd5-bfd4-16b0f8b0f827 on nch2-master-1 Jul 16 21:15:45.552242 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-97260a55-8b10-43de-8f37-4667b0847b5c req-101ac501-726b-4cde-8dda-df76806dee18 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.039664 Jul 16 21:15:45.552861 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 916/1762] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:15:45 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:15:45.558427 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-2ac51f9f-41cf-4e58-8e52-f4e752ca1ce7 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:15:45.559055 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-2ac51f9f-41cf-4e58-8e52-f4e752ca1ce7 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] [instance: 0079e567-30df-4cac-a0ad-4379bcbbcf48] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:15:45.566087 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2ac51f9f-41cf-4e58-8e52-f4e752ca1ce7 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:45.566747 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2ac51f9f-41cf-4e58-8e52-f4e752ca1ce7 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:45.567935 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2ac51f9f-41cf-4e58-8e52-f4e752ca1ce7 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:45.568548 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2ac51f9f-41cf-4e58-8e52-f4e752ca1ce7 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:45.604063 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-2ac51f9f-41cf-4e58-8e52-f4e752ca1ce7 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] 10.222.0.16 "GET /compute/v2.1/servers/0079e567-30df-4cac-a0ad-4379bcbbcf48/ips" status: 200 len: 180 microversion: 2.1 time: 0.049037 Jul 16 21:15:45.611657 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 917/1763] 10.222.0.16 () {58 vars in 1247 bytes} [Sat Jul 16 21:15:45 2022] GET /compute/v2.1/servers/0079e567-30df-4cac-a0ad-4379bcbbcf48/ips => generated 180 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:15:45.619381 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-050b56af-6ea4-44e7-9dc2-bc9e4f51cf0b tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:15:45.620283 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-050b56af-6ea4-44e7-9dc2-bc9e4f51cf0b tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] [instance: 23df6e0a-3637-4f59-93f9-b02dd614b5c3] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:15:45.630659 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-050b56af-6ea4-44e7-9dc2-bc9e4f51cf0b tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:45.639068 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-050b56af-6ea4-44e7-9dc2-bc9e4f51cf0b tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:45.639068 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-050b56af-6ea4-44e7-9dc2-bc9e4f51cf0b tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:45.639068 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-050b56af-6ea4-44e7-9dc2-bc9e4f51cf0b tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:45.891737 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-3b2a7d41-310c-48ce-b408-bf9f38c73a24 tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] 10.222.0.16 "POST /compute/v2.1/servers/84ab7e65-1bf0-4b8b-baea-6d1d4ca13dd7/action" status: 202 len: 0 microversion: 2.25 time: 5.525265 Jul 16 21:15:45.891737 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 847/1764] 10.222.0.16 () {62 vars in 1318 bytes} [Sat Jul 16 21:15:40 2022] POST /compute/v2.1/servers/84ab7e65-1bf0-4b8b-baea-6d1d4ca13dd7/action => generated 0 bytes in 5527 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Jul 16 21:15:45.892223 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-050b56af-6ea4-44e7-9dc2-bc9e4f51cf0b tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a69f9dee03d43beaafa966216b9e289', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9b769bf8f3ea4e368bf68de1df6538f9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:15:45.897202 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-050b56af-6ea4-44e7-9dc2-bc9e4f51cf0b tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] 10.222.0.16 "GET /compute/v2.1/servers/23df6e0a-3637-4f59-93f9-b02dd614b5c3" status: 200 len: 1389 microversion: 2.1 time: 0.283962 Jul 16 21:15:45.899397 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-8633f64e-39af-4227-b866-e115d7e2b533 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:15:45.899931 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-8633f64e-39af-4227-b866-e115d7e2b533 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] [instance: 93e327c3-c63d-439a-9b25-1a805f785949] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:15:45.908968 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 918/1765] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:15:45 2022] GET /compute/v2.1/servers/23df6e0a-3637-4f59-93f9-b02dd614b5c3 => generated 1389 bytes in 297 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:15:45.924377 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-470ccd91-1ed2-4b62-8b6d-8cf6bd4f7a18 tempest-FloatingIPDetailsTestJSON-1149902298 tempest-FloatingIPDetailsTestJSON-1149902298-project] Calling method '.version_select at 0x7fdd9cd91dc0>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:15:45.931149 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8633f64e-39af-4227-b866-e115d7e2b533 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:45.933124 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8633f64e-39af-4227-b866-e115d7e2b533 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:45.935849 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8633f64e-39af-4227-b866-e115d7e2b533 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:45.937387 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8633f64e-39af-4227-b866-e115d7e2b533 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:46.426888 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-470ccd91-1ed2-4b62-8b6d-8cf6bd4f7a18 tempest-FloatingIPDetailsTestJSON-1149902298 tempest-FloatingIPDetailsTestJSON-1149902298-project] 10.222.0.16 "GET /compute/v2.1/os-floating-ips" status: 200 len: 397 microversion: 2.1 time: 0.507155 Jul 16 21:15:46.426888 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 919/1766] 10.222.0.16 () {58 vars in 1148 bytes} [Sat Jul 16 21:15:45 2022] GET /compute/v2.1/os-floating-ips => generated 397 bytes in 517 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:15:46.434026 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-9ee3cfea-2ecd-4231-9416-3998588c47ae tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:15:46.435340 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-9ee3cfea-2ecd-4231-9416-3998588c47ae tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] [instance: 39e51d01-21e0-4dd5-bfd4-16b0f8b0f827] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:15:46.451130 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9ee3cfea-2ecd-4231-9416-3998588c47ae tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:46.451974 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9ee3cfea-2ecd-4231-9416-3998588c47ae tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:46.453072 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9ee3cfea-2ecd-4231-9416-3998588c47ae tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:46.453838 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9ee3cfea-2ecd-4231-9416-3998588c47ae tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:46.466753 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-8633f64e-39af-4227-b866-e115d7e2b533 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'daf3c9e6634e4ae1bfb492eae1f44c93', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '24c64db8082648c8837c16352252f96d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:15:46.478545 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-9ee3cfea-2ecd-4231-9416-3998588c47ae tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] HTTP exception thrown: Instance 39e51d01-21e0-4dd5-bfd4-16b0f8b0f827 could not be found. Jul 16 21:15:46.480709 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-9ee3cfea-2ecd-4231-9416-3998588c47ae tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Request method failure captured: Jul 16 21:15:46.480709 ncdv-master-1 devstack@n-api.service[138680]: request: GET /compute/v2.1/servers/39e51d01-21e0-4dd5-bfd4-16b0f8b0f827 HTTP/1.1 Jul 16 21:15:46.480709 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:15:46.480709 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:15:46.480709 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:15:46.480709 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 0 Jul 16 21:15:46.480709 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:15:46.480709 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:15:46.480709 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:15:46.480709 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:15:46.480709 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0yoJFUAf_8Wtq8fI-wZdJwoKyg8rUEOC_eBMeqcRE2hk6JM7_ssDplugQwmk2S9xH7Oys7Ho2tw0kc9hmlY-jX_rR-I-9WVwQYFbyogU7ki99siEGGR9aVA2GKsp5nNcVHXBtjM3bI_zfa21XSWedvaSIVwZV8kEGixKYsBf2GMYKVI Jul 16 21:15:46.480709 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:15:46.480709 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:15:46.480709 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:15:46.480709 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:15:46.480709 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:15:46.480709 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:15:46.480709 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: 4337575622d04a26942905c9dc044ba7 Jul 16 21:15:46.480709 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-ImagesOneServerNegativeTestJSON-526509748 Jul 16 21:15:46.480709 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:15:46.480709 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:15:46.480709 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/4337575622d04a26942905c9dc044ba7"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/4337575622d04a26942905c9dc044ba7"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_4337575622d04a26942905c9dc044ba7"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/4337575622d04a26942905c9dc044ba7"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:15:46.480709 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-ImagesOneServerNegativeTestJSON-526509748 Jul 16 21:15:46.480709 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: 4337575622d04a26942905c9dc044ba7 Jul 16 21:15:46.480709 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-ImagesOneServerNegativeTestJSON-526509748 Jul 16 21:15:46.480709 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-ImagesOneServerNegativeTestJSON-526509748-project Jul 16 21:15:46.480709 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:15:46.480709 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:15:46.480709 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: 4e9df82f95ac4779859972403869096a Jul 16 21:15:46.480709 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-ImagesOneServerNegativeTestJSON-526509748-project Jul 16 21:15:46.480709 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 21:15:46.480709 ncdv-master-1 devstack@n-api.service[138680]: exception: Instance 39e51d01-21e0-4dd5-bfd4-16b0f8b0f827 could not be found. Jul 16 21:15:46.480709 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': '39e51d01-21e0-4dd5-bfd4-16b0f8b0f827'} Jul 16 21:15:46.480709 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:15:46.480709 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:15:46.480709 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:15:46.480709 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:15:46.480709 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:15:46.480709 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:15:46.480709 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:15:46.480709 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:15:46.480709 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:15:46.480709 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:15:46.480709 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 39e51d01-21e0-4dd5-bfd4-16b0f8b0f827 could not be found. Jul 16 21:15:46.480709 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:15:46.480709 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:15:46.480709 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:15:46.480709 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:15:46.480709 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:15:46.480709 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:15:46.480709 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:15:46.480709 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:15:46.480709 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:15:46.480709 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:15:46.480709 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:15:46.480709 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:15:46.480709 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:15:46.480709 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:15:46.480709 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:15:46.480709 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:15:46.480709 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 39e51d01-21e0-4dd5-bfd4-16b0f8b0f827 could not be found. Jul 16 21:15:46.480709 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:15:46.480709 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:15:46.485527 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:15:46.485527 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:15:46.485527 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:15:46.485527 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:15:46.485527 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:15:46.485527 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:15:46.485527 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 39e51d01-21e0-4dd5-bfd4-16b0f8b0f827 could not be found. Jul 16 21:15:46.485527 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:15:46.486149 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-8633f64e-39af-4227-b866-e115d7e2b533 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] 10.222.0.16 "GET /compute/v2.1/servers/93e327c3-c63d-439a-9b25-1a805f785949" status: 200 len: 1487 microversion: 2.1 time: 0.596691 Jul 16 21:15:46.491379 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 848/1767] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:15:45 2022] GET /compute/v2.1/servers/93e327c3-c63d-439a-9b25-1a805f785949 => generated 1487 bytes in 609 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:15:46.494019 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-9ee3cfea-2ecd-4231-9416-3998588c47ae tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Returning 404 to user: Instance 39e51d01-21e0-4dd5-bfd4-16b0f8b0f827 could not be found. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:15:46.495313 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-9ee3cfea-2ecd-4231-9416-3998588c47ae tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] 10.222.0.16 "GET /compute/v2.1/servers/39e51d01-21e0-4dd5-bfd4-16b0f8b0f827" status: 404 len: 111 microversion: 2.1 time: 0.067482 Jul 16 21:15:46.496455 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 920/1768] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:15:46 2022] GET /compute/v2.1/servers/39e51d01-21e0-4dd5-bfd4-16b0f8b0f827 => generated 111 bytes in 69 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:15:46.517033 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-8c0543e7-37aa-42c7-bb91-9f46de3d6b9d tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:15:46.519907 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-8c0543e7-37aa-42c7-bb91-9f46de3d6b9d tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] [instance: 0079e567-30df-4cac-a0ad-4379bcbbcf48] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:15:46.538309 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8c0543e7-37aa-42c7-bb91-9f46de3d6b9d tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:46.540614 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8c0543e7-37aa-42c7-bb91-9f46de3d6b9d tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:46.542723 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8c0543e7-37aa-42c7-bb91-9f46de3d6b9d tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:46.545228 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8c0543e7-37aa-42c7-bb91-9f46de3d6b9d tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:46.625117 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-8c0543e7-37aa-42c7-bb91-9f46de3d6b9d tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] [instance: 0079e567-30df-4cac-a0ad-4379bcbbcf48] Going to try to terminate instance {{(pid=138681) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 16 21:15:46.753131 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-a40a4c43-5f14-44e8-bbac-595e94670e52 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:15:46.754150 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-a40a4c43-5f14-44e8-bbac-595e94670e52 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] [instance: 84ab7e65-1bf0-4b8b-baea-6d1d4ca13dd7] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:15:46.764006 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a40a4c43-5f14-44e8-bbac-595e94670e52 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:46.765108 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a40a4c43-5f14-44e8-bbac-595e94670e52 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:46.767281 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a40a4c43-5f14-44e8-bbac-595e94670e52 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:46.768072 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a40a4c43-5f14-44e8-bbac-595e94670e52 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:47.101990 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-a40a4c43-5f14-44e8-bbac-595e94670e52 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:15:47.107225 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:15:47.116350 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-8c0543e7-37aa-42c7-bb91-9f46de3d6b9d tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] 10.222.0.16 "DELETE /compute/v2.1/servers/0079e567-30df-4cac-a0ad-4379bcbbcf48" status: 204 len: 0 microversion: 2.1 time: 0.615641 Jul 16 21:15:47.118274 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 849/1769] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:15:46 2022] DELETE /compute/v2.1/servers/0079e567-30df-4cac-a0ad-4379bcbbcf48 => generated 0 bytes in 624 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 16 21:15:47.126405 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-827020c3-b1da-40ae-8aae-d3c2ea617930 tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServersTestBootFromVolume-server-532589203", "imageRef": "", "flavorRef": "452", "metadata": {"hello": "world"}, "accessIPv4": "1.1.1.1", "accessIPv6": "0000:0000:0000:0000:0000:babe:220.12.22.2", "adminPass": "***", "networks": [{"uuid": "9ee099db-5e6c-4039-bdc5-bd35a40f0a05"}], "security_groups": [{"name": "tempest-securitygroup--2140786008"}], "key_name": "tempest-keypair-1395237950", "user_data": "IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIHVidW50dSB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH51YnVudHUvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=", "block_device_mapping_v2": [{"uuid": "7fa5082b-070c-4f64-9f0b-f24264ffca4a", "source_type": "volume", "destination_type": "volume", "boot_index": 0, "delete_on_termination": true}], "OS-DCF:diskConfig": "}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:15:47.128724 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-a40a4c43-5f14-44e8-bbac-595e94670e52 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:15:47.136998 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-a40a4c43-5f14-44e8-bbac-595e94670e52 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:15:47.139409 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-a40a4c43-5f14-44e8-bbac-595e94670e52 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] 10.222.0.16 "GET /compute/v2.1/servers/84ab7e65-1bf0-4b8b-baea-6d1d4ca13dd7" status: 200 len: 1487 microversion: 2.25 time: 0.641603 Jul 16 21:15:47.140682 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 921/1770] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:15:46 2022] GET /compute/v2.1/servers/84ab7e65-1bf0-4b8b-baea-6d1d4ca13dd7 => generated 1487 bytes in 643 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:15:47.152616 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-ae26d02c-a874-4c3a-9f08-7e799ffd0c25 tempest-FloatingIPDetailsTestJSON-1149902298 tempest-FloatingIPDetailsTestJSON-1149902298-project] Calling method '.version_select at 0x7fdda2bc0e50>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:15:47.405888 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.network.neutron [None req-827020c3-b1da-40ae-8aae-d3c2ea617930 tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] validate_networks() for [('9ee099db-5e6c-4039-bdc5-bd35a40f0a05', None, None, None, None, None)] {{(pid=138681) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 16 21:15:48.197816 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-827020c3-b1da-40ae-8aae-d3c2ea617930 tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] Getting quotas for project d82c2d92c67f469e863eb70bb7f9af84. Resources: dict_keys(['metadata_items']) {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:15:48.205323 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-827020c3-b1da-40ae-8aae-d3c2ea617930 tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] Getting quotas for user 1aa99be675e946fd995d71465b4cbec6 and project d82c2d92c67f469e863eb70bb7f9af84. Resources: dict_keys(['metadata_items']) {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:15:48.220074 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-827020c3-b1da-40ae-8aae-d3c2ea617930 tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] Flavor limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 16 21:15:48.220614 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-827020c3-b1da-40ae-8aae-d3c2ea617930 tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] Image limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 16 21:15:48.221558 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-827020c3-b1da-40ae-8aae-d3c2ea617930 tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] Flavor pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 16 21:15:48.222342 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-827020c3-b1da-40ae-8aae-d3c2ea617930 tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] Image pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 16 21:15:48.222988 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-827020c3-b1da-40ae-8aae-d3c2ea617930 tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 16 21:15:48.233340 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-827020c3-b1da-40ae-8aae-d3c2ea617930 tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:48.234006 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-827020c3-b1da-40ae-8aae-d3c2ea617930 tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:48.234986 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-827020c3-b1da-40ae-8aae-d3c2ea617930 tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:48.235466 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-827020c3-b1da-40ae-8aae-d3c2ea617930 tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:48.254858 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-827020c3-b1da-40ae-8aae-d3c2ea617930 tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] Getting quotas for project d82c2d92c67f469e863eb70bb7f9af84. Resources: {'ram', 'instances', 'cores'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:15:48.260145 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-827020c3-b1da-40ae-8aae-d3c2ea617930 tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] Getting quotas for user 1aa99be675e946fd995d71465b4cbec6 and project d82c2d92c67f469e863eb70bb7f9af84. Resources: {'ram', 'instances', 'cores'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:15:48.323891 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-827020c3-b1da-40ae-8aae-d3c2ea617930 tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] Going to run 1 instances... {{(pid=138681) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 16 21:15:48.326430 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-827020c3-b1da-40ae-8aae-d3c2ea617930 tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] [instance: b9f2c527-6346-4403-841e-b7bd876a5fb7] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag=None,updated_at=,uuid=,volume_id='7fa5082b-070c-4f64-9f0b-f24264ffca4a',volume_size=None,volume_type=None)] {{(pid=138681) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 16 21:15:48.628615 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-ae26d02c-a874-4c3a-9f08-7e799ffd0c25 tempest-FloatingIPDetailsTestJSON-1149902298 tempest-FloatingIPDetailsTestJSON-1149902298-project] 10.222.0.16 "DELETE /compute/v2.1/os-floating-ips/db2e7d2f-fe00-42a0-8159-2a656b4d8077" status: 202 len: 0 microversion: 2.1 time: 1.486440 Jul 16 21:15:48.630594 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 922/1771] 10.222.0.16 () {58 vars in 1262 bytes} [Sat Jul 16 21:15:47 2022] DELETE /compute/v2.1/os-floating-ips/db2e7d2f-fe00-42a0-8159-2a656b4d8077 => generated 0 bytes in 1488 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 21:15:48.638968 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-cc2c38b5-bf19-4ad8-b04c-f031d2dc5072 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Action: 'action', calling method: >, body: {"pause": {}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:15:48.639933 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-cc2c38b5-bf19-4ad8-b04c-f031d2dc5072 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] [instance: 93e327c3-c63d-439a-9b25-1a805f785949] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:15:48.650376 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cc2c38b5-bf19-4ad8-b04c-f031d2dc5072 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:48.651670 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cc2c38b5-bf19-4ad8-b04c-f031d2dc5072 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:48.652739 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cc2c38b5-bf19-4ad8-b04c-f031d2dc5072 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:48.653499 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cc2c38b5-bf19-4ad8-b04c-f031d2dc5072 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:48.894836 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-827020c3-b1da-40ae-8aae-d3c2ea617930 tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 407 microversion: 2.1 time: 1.775106 Jul 16 21:15:48.904711 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 850/1772] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 21:15:47 2022] POST /compute/v2.1/servers => generated 407 bytes in 1784 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 16 21:15:48.913552 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-0cb056f2-b0c3-4448-b82e-16f898fc625a tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ImagesOneServerNegativeTestJSON-server-1665832383", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "}]}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:15:48.984792 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.network.neutron [None req-0cb056f2-b0c3-4448-b82e-16f898fc625a tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] validate_networks() for [('45344261-e9f3-4a7b-95e1-5380175d73fa', None, None, None, None, None)] {{(pid=138681) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 16 21:15:49.016774 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-cc2c38b5-bf19-4ad8-b04c-f031d2dc5072 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] 10.222.0.16 "POST /compute/v2.1/servers/93e327c3-c63d-439a-9b25-1a805f785949/action" status: 202 len: 0 microversion: 2.1 time: 0.383991 Jul 16 21:15:49.017686 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 923/1773] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 21:15:48 2022] POST /compute/v2.1/servers/93e327c3-c63d-439a-9b25-1a805f785949/action => generated 0 bytes in 386 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 21:15:49.039694 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-319699c7-bed2-47b2-8dbb-6268e8cd40c8 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:15:49.040614 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-319699c7-bed2-47b2-8dbb-6268e8cd40c8 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] [instance: 23df6e0a-3637-4f59-93f9-b02dd614b5c3] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:15:49.048214 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-319699c7-bed2-47b2-8dbb-6268e8cd40c8 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:49.048827 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-319699c7-bed2-47b2-8dbb-6268e8cd40c8 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:49.050305 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-319699c7-bed2-47b2-8dbb-6268e8cd40c8 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:49.051316 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-319699c7-bed2-47b2-8dbb-6268e8cd40c8 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:49.652898 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell0:SELECT=6 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:15:50.017068 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-319699c7-bed2-47b2-8dbb-6268e8cd40c8 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a69f9dee03d43beaafa966216b9e289', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9b769bf8f3ea4e368bf68de1df6538f9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:15:50.027164 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-319699c7-bed2-47b2-8dbb-6268e8cd40c8 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] 10.222.0.16 "GET /compute/v2.1/servers/23df6e0a-3637-4f59-93f9-b02dd614b5c3" status: 200 len: 1389 microversion: 2.1 time: 1.007652 Jul 16 21:15:50.029115 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 924/1774] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:15:49 2022] GET /compute/v2.1/servers/23df6e0a-3637-4f59-93f9-b02dd614b5c3 => generated 1389 bytes in 1011 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:15:50.035522 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-75d801ea-c3ae-4704-93a5-1d9f56b6e286 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:15:50.036394 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-75d801ea-c3ae-4704-93a5-1d9f56b6e286 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] [instance: 0079e567-30df-4cac-a0ad-4379bcbbcf48] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:15:50.044579 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-75d801ea-c3ae-4704-93a5-1d9f56b6e286 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:50.045329 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-75d801ea-c3ae-4704-93a5-1d9f56b6e286 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:50.046340 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-75d801ea-c3ae-4704-93a5-1d9f56b6e286 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:50.047302 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-75d801ea-c3ae-4704-93a5-1d9f56b6e286 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:50.633309 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-75d801ea-c3ae-4704-93a5-1d9f56b6e286 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '53af488f3b0242a797f539284b78d59e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bb1ede0d0b6941cc97278e6c617f9b01', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:15:50.661839 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-75d801ea-c3ae-4704-93a5-1d9f56b6e286 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] 10.222.0.16 "GET /compute/v2.1/servers/0079e567-30df-4cac-a0ad-4379bcbbcf48" status: 200 len: 1676 microversion: 2.1 time: 0.614675 Jul 16 21:15:50.679914 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 925/1775] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:15:50 2022] GET /compute/v2.1/servers/0079e567-30df-4cac-a0ad-4379bcbbcf48 => generated 1676 bytes in 650 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:15:50.854568 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-7037a7f0-e564-4b98-9696-f12b38443f15 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerDiskConfigTestJSON-server-820217111", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452", "networks": [{"uuid": "b137fafe-f0b3-4d44-9dfc-add7ee4d76f5"}]}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:15:50.995735 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.network.neutron [None req-7037a7f0-e564-4b98-9696-f12b38443f15 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] validate_networks() for [('b137fafe-f0b3-4d44-9dfc-add7ee4d76f5', None, None, None, None, None)] {{(pid=138680) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 16 21:15:51.505373 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-0cb056f2-b0c3-4448-b82e-16f898fc625a tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Flavor limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 16 21:15:51.505875 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-0cb056f2-b0c3-4448-b82e-16f898fc625a tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Image limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 16 21:15:51.506262 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-0cb056f2-b0c3-4448-b82e-16f898fc625a tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Flavor pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 16 21:15:51.506672 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-0cb056f2-b0c3-4448-b82e-16f898fc625a tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Image pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 16 21:15:51.507548 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-0cb056f2-b0c3-4448-b82e-16f898fc625a tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 16 21:15:51.528583 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0cb056f2-b0c3-4448-b82e-16f898fc625a tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:51.529045 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0cb056f2-b0c3-4448-b82e-16f898fc625a tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:51.534270 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0cb056f2-b0c3-4448-b82e-16f898fc625a tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:51.536383 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0cb056f2-b0c3-4448-b82e-16f898fc625a tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:51.597142 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-0cb056f2-b0c3-4448-b82e-16f898fc625a tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Getting quotas for project 4337575622d04a26942905c9dc044ba7. Resources: {'ram', 'instances', 'cores'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:15:51.602120 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_api:SELECT=94,nova_api:INSERT=5,nova_api:UPDATE=1 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:15:51.603229 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-0cb056f2-b0c3-4448-b82e-16f898fc625a tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Getting quotas for user 4e9df82f95ac4779859972403869096a and project 4337575622d04a26942905c9dc044ba7. Resources: {'ram', 'instances', 'cores'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:15:51.639580 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-0cb056f2-b0c3-4448-b82e-16f898fc625a tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Going to run 1 instances... {{(pid=138681) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 16 21:15:51.649677 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-0cb056f2-b0c3-4448-b82e-16f898fc625a tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] [instance: f6c350c4-b1b2-42d3-ba02-65cbbacf0f58] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138681) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 16 21:15:51.841647 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-0cb056f2-b0c3-4448-b82e-16f898fc625a tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 383 microversion: 2.1 time: 2.937103 Jul 16 21:15:51.844590 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 851/1776] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 21:15:48 2022] POST /compute/v2.1/servers => generated 383 bytes in 2940 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 16 21:15:51.860626 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-e7a9b41c-90d0-4fda-9b9a-53bf875ed82e tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:15:51.861690 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-e7a9b41c-90d0-4fda-9b9a-53bf875ed82e tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] [instance: 84ab7e65-1bf0-4b8b-baea-6d1d4ca13dd7] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:15:51.876719 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e7a9b41c-90d0-4fda-9b9a-53bf875ed82e tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:51.879228 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e7a9b41c-90d0-4fda-9b9a-53bf875ed82e tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:51.884546 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e7a9b41c-90d0-4fda-9b9a-53bf875ed82e tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:51.884546 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e7a9b41c-90d0-4fda-9b9a-53bf875ed82e tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:52.446220 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-e7a9b41c-90d0-4fda-9b9a-53bf875ed82e tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:15:52.460304 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-e7a9b41c-90d0-4fda-9b9a-53bf875ed82e tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:15:52.469550 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-e7a9b41c-90d0-4fda-9b9a-53bf875ed82e tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:15:52.475357 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-e7a9b41c-90d0-4fda-9b9a-53bf875ed82e tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] 10.222.0.16 "GET /compute/v2.1/servers/84ab7e65-1bf0-4b8b-baea-6d1d4ca13dd7" status: 200 len: 1487 microversion: 2.25 time: 0.629290 Jul 16 21:15:52.476546 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 852/1777] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:15:51 2022] GET /compute/v2.1/servers/84ab7e65-1bf0-4b8b-baea-6d1d4ca13dd7 => generated 1487 bytes in 631 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:15:52.488360 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-769cb3d9-c84e-4182-bb40-d92de2213bdb tempest-FloatingIPDetailsTestJSON-1149902298 tempest-FloatingIPDetailsTestJSON-1149902298-project] Calling method '.version_select at 0x7fdda2b7e040>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:15:52.936985 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-7037a7f0-e564-4b98-9696-f12b38443f15 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Flavor limits 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 16 21:15:52.941191 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-7037a7f0-e564-4b98-9696-f12b38443f15 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Image limits 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 16 21:15:52.945362 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-7037a7f0-e564-4b98-9696-f12b38443f15 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Flavor pref 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 16 21:15:52.951220 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-7037a7f0-e564-4b98-9696-f12b38443f15 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Image pref 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 16 21:15:52.952064 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-7037a7f0-e564-4b98-9696-f12b38443f15 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 16 21:15:52.973944 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7037a7f0-e564-4b98-9696-f12b38443f15 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:52.974912 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7037a7f0-e564-4b98-9696-f12b38443f15 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:52.975850 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7037a7f0-e564-4b98-9696-f12b38443f15 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:52.976509 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7037a7f0-e564-4b98-9696-f12b38443f15 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:53.007193 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-7037a7f0-e564-4b98-9696-f12b38443f15 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Getting quotas for project a90d4bcd25ff4bf68bdc194180fbbe09. Resources: {'ram', 'instances', 'cores'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:15:53.014282 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-7037a7f0-e564-4b98-9696-f12b38443f15 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Getting quotas for user dd4144b5388b46a989a41a2f556bb508 and project a90d4bcd25ff4bf68bdc194180fbbe09. Resources: {'ram', 'instances', 'cores'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:15:53.029510 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-7037a7f0-e564-4b98-9696-f12b38443f15 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Going to run 1 instances... {{(pid=138680) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 16 21:15:53.034104 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-7037a7f0-e564-4b98-9696-f12b38443f15 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] [instance: 11194822-5d42-45f4-99d0-88c70c808ddd] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138680) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 16 21:15:53.221662 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-7037a7f0-e564-4b98-9696-f12b38443f15 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 2.533962 Jul 16 21:15:53.222832 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 926/1778] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 21:15:50 2022] POST /compute/v2.1/servers => generated 380 bytes in 2542 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 16 21:15:53.237913 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-85e043b7-e870-4f65-a676-0ff4e04217eb req-f6d28473-4dc8-45ae-b79e-f428418f7240 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "23df6e0a-3637-4f59-93f9-b02dd614b5c3", "name": "network-vif-unplugged", "status": "completed", "tag": "60e499ad-afcb-4fe7-80ef-058f63c4fc21"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:15:53.248787 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-85e043b7-e870-4f65-a676-0ff4e04217eb req-f6d28473-4dc8-45ae-b79e-f428418f7240 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:53.249432 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-85e043b7-e870-4f65-a676-0ff4e04217eb req-f6d28473-4dc8-45ae-b79e-f428418f7240 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:53.263704 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-85e043b7-e870-4f65-a676-0ff4e04217eb req-f6d28473-4dc8-45ae-b79e-f428418f7240 service nova] Creating event network-vif-unplugged:60e499ad-afcb-4fe7-80ef-058f63c4fc21 for instance 23df6e0a-3637-4f59-93f9-b02dd614b5c3 on nch2-master-1 Jul 16 21:15:53.269822 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-85e043b7-e870-4f65-a676-0ff4e04217eb req-f6d28473-4dc8-45ae-b79e-f428418f7240 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.039796 Jul 16 21:15:53.271042 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 927/1779] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:15:53 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:15:53.276196 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-6e884d7a-828f-4e32-9501-c1bb5c831efe tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:15:53.276903 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-6e884d7a-828f-4e32-9501-c1bb5c831efe tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] [instance: b9f2c527-6346-4403-841e-b7bd876a5fb7] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:15:53.284088 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6e884d7a-828f-4e32-9501-c1bb5c831efe tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:53.284803 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6e884d7a-828f-4e32-9501-c1bb5c831efe tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:53.285595 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6e884d7a-828f-4e32-9501-c1bb5c831efe tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:53.286045 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6e884d7a-828f-4e32-9501-c1bb5c831efe tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:53.369984 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-6e884d7a-828f-4e32-9501-c1bb5c831efe tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aa99be675e946fd995d71465b4cbec6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd82c2d92c67f469e863eb70bb7f9af84', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:15:53.382354 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-6e884d7a-828f-4e32-9501-c1bb5c831efe tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] 10.222.0.16 "GET /compute/v2.1/servers/b9f2c527-6346-4403-841e-b7bd876a5fb7" status: 200 len: 1237 microversion: 2.1 time: 0.110076 Jul 16 21:15:53.384269 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 928/1780] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:15:53 2022] GET /compute/v2.1/servers/b9f2c527-6346-4403-841e-b7bd876a5fb7 => generated 1237 bytes in 113 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:15:53.390732 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-0721dee8-bca1-4062-8992-c069109185c8 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:15:53.391436 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-0721dee8-bca1-4062-8992-c069109185c8 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] [instance: 93e327c3-c63d-439a-9b25-1a805f785949] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:15:53.398885 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0721dee8-bca1-4062-8992-c069109185c8 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:53.399658 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0721dee8-bca1-4062-8992-c069109185c8 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:53.400673 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0721dee8-bca1-4062-8992-c069109185c8 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:53.400976 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0721dee8-bca1-4062-8992-c069109185c8 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:53.761671 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-0721dee8-bca1-4062-8992-c069109185c8 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'daf3c9e6634e4ae1bfb492eae1f44c93', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '24c64db8082648c8837c16352252f96d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:15:53.774818 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-0721dee8-bca1-4062-8992-c069109185c8 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] 10.222.0.16 "GET /compute/v2.1/servers/93e327c3-c63d-439a-9b25-1a805f785949" status: 200 len: 1472 microversion: 2.1 time: 0.388966 Jul 16 21:15:53.778657 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 929/1781] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:15:53 2022] GET /compute/v2.1/servers/93e327c3-c63d-439a-9b25-1a805f785949 => generated 1472 bytes in 394 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:15:53.788494 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-e4c5f254-2b50-46a2-bbdf-9dd09b25af2d tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:15:53.791423 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-e4c5f254-2b50-46a2-bbdf-9dd09b25af2d tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] [instance: 23df6e0a-3637-4f59-93f9-b02dd614b5c3] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:15:53.803871 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e4c5f254-2b50-46a2-bbdf-9dd09b25af2d tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:53.804512 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e4c5f254-2b50-46a2-bbdf-9dd09b25af2d tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:53.805792 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e4c5f254-2b50-46a2-bbdf-9dd09b25af2d tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:53.806242 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e4c5f254-2b50-46a2-bbdf-9dd09b25af2d tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:54.181959 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-e4c5f254-2b50-46a2-bbdf-9dd09b25af2d tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a69f9dee03d43beaafa966216b9e289', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9b769bf8f3ea4e368bf68de1df6538f9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:15:54.195170 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-e4c5f254-2b50-46a2-bbdf-9dd09b25af2d tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] 10.222.0.16 "GET /compute/v2.1/servers/23df6e0a-3637-4f59-93f9-b02dd614b5c3" status: 200 len: 1389 microversion: 2.1 time: 0.414928 Jul 16 21:15:54.196545 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 930/1782] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:15:53 2022] GET /compute/v2.1/servers/23df6e0a-3637-4f59-93f9-b02dd614b5c3 => generated 1389 bytes in 417 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:15:54.201934 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-efc7ea83-52c7-4b1a-b92a-5285edfb1999 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:15:54.202881 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-efc7ea83-52c7-4b1a-b92a-5285edfb1999 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] [instance: 0079e567-30df-4cac-a0ad-4379bcbbcf48] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:15:54.223324 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-efc7ea83-52c7-4b1a-b92a-5285edfb1999 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:54.224613 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-efc7ea83-52c7-4b1a-b92a-5285edfb1999 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:54.226190 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-efc7ea83-52c7-4b1a-b92a-5285edfb1999 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:54.227323 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-efc7ea83-52c7-4b1a-b92a-5285edfb1999 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:54.251305 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-efc7ea83-52c7-4b1a-b92a-5285edfb1999 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] HTTP exception thrown: Instance 0079e567-30df-4cac-a0ad-4379bcbbcf48 could not be found. Jul 16 21:15:54.253915 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-efc7ea83-52c7-4b1a-b92a-5285edfb1999 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Request method failure captured: Jul 16 21:15:54.253915 ncdv-master-1 devstack@n-api.service[138680]: request: GET /compute/v2.1/servers/0079e567-30df-4cac-a0ad-4379bcbbcf48 HTTP/1.1 Jul 16 21:15:54.253915 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:15:54.253915 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:15:54.253915 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:15:54.253915 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 0 Jul 16 21:15:54.253915 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:15:54.253915 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:15:54.253915 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:15:54.253915 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:15:54.253915 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0yoCD7IX15brFQUHVPKPIJrpEu20fxENHCEz31UnXrkVZbNHxjQ0ElqFOE7MXQnga3bWs8iQFWXWTDFYHn6RqK2SreYxppaBRq4w7eCzu-cpcuRq-bR3warXbUzkwx17fxxMPjZ200EVRXuxxzJd_xS8-ssDDdZBxRLNj6zLIJYowVk Jul 16 21:15:54.253915 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:15:54.253915 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:15:54.253915 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:15:54.253915 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:15:54.253915 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:15:54.253915 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:15:54.253915 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: bb1ede0d0b6941cc97278e6c617f9b01 Jul 16 21:15:54.253915 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-ServersTestMultiNic-262233723 Jul 16 21:15:54.253915 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:15:54.253915 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:15:54.253915 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/bb1ede0d0b6941cc97278e6c617f9b01"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/bb1ede0d0b6941cc97278e6c617f9b01"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_bb1ede0d0b6941cc97278e6c617f9b01"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/bb1ede0d0b6941cc97278e6c617f9b01"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:15:54.253915 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-ServersTestMultiNic-262233723 Jul 16 21:15:54.253915 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: bb1ede0d0b6941cc97278e6c617f9b01 Jul 16 21:15:54.253915 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-ServersTestMultiNic-262233723 Jul 16 21:15:54.253915 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-ServersTestMultiNic-262233723-project Jul 16 21:15:54.253915 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:15:54.253915 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:15:54.253915 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: 53af488f3b0242a797f539284b78d59e Jul 16 21:15:54.253915 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-ServersTestMultiNic-262233723-project Jul 16 21:15:54.253915 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 21:15:54.253915 ncdv-master-1 devstack@n-api.service[138680]: exception: Instance 0079e567-30df-4cac-a0ad-4379bcbbcf48 could not be found. Jul 16 21:15:54.253915 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': '0079e567-30df-4cac-a0ad-4379bcbbcf48'} Jul 16 21:15:54.253915 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:15:54.253915 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:15:54.253915 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:15:54.253915 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:15:54.253915 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:15:54.253915 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:15:54.253915 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:15:54.253915 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:15:54.253915 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:15:54.253915 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:15:54.253915 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 0079e567-30df-4cac-a0ad-4379bcbbcf48 could not be found. Jul 16 21:15:54.253915 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:15:54.253915 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:15:54.253915 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:15:54.253915 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:15:54.253915 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:15:54.253915 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:15:54.253915 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:15:54.253915 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:15:54.253915 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:15:54.253915 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:15:54.253915 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:15:54.253915 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:15:54.253915 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:15:54.253915 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:15:54.253915 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:15:54.253915 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:15:54.253915 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 0079e567-30df-4cac-a0ad-4379bcbbcf48 could not be found. Jul 16 21:15:54.253915 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:15:54.253915 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:15:54.253915 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:15:54.273379 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:15:54.273379 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:15:54.273379 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:15:54.273379 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:15:54.273379 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:15:54.273379 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 0079e567-30df-4cac-a0ad-4379bcbbcf48 could not be found. Jul 16 21:15:54.273379 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:15:54.273379 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-efc7ea83-52c7-4b1a-b92a-5285edfb1999 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Returning 404 to user: Instance 0079e567-30df-4cac-a0ad-4379bcbbcf48 could not be found. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:15:54.273379 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-efc7ea83-52c7-4b1a-b92a-5285edfb1999 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] 10.222.0.16 "GET /compute/v2.1/servers/0079e567-30df-4cac-a0ad-4379bcbbcf48" status: 404 len: 111 microversion: 2.1 time: 0.058767 Jul 16 21:15:54.273379 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 931/1783] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:15:54 2022] GET /compute/v2.1/servers/0079e567-30df-4cac-a0ad-4379bcbbcf48 => generated 111 bytes in 60 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:15:54.273379 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-a15121a4-968d-4d82-a0a8-56de6ce57a01 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:15:54.273379 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-a15121a4-968d-4d82-a0a8-56de6ce57a01 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] [instance: f6c350c4-b1b2-42d3-ba02-65cbbacf0f58] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:15:54.273379 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a15121a4-968d-4d82-a0a8-56de6ce57a01 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:54.273379 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a15121a4-968d-4d82-a0a8-56de6ce57a01 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:54.275873 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a15121a4-968d-4d82-a0a8-56de6ce57a01 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:54.275873 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a15121a4-968d-4d82-a0a8-56de6ce57a01 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:54.458864 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-a15121a4-968d-4d82-a0a8-56de6ce57a01 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4e9df82f95ac4779859972403869096a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4337575622d04a26942905c9dc044ba7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:15:54.474448 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-a15121a4-968d-4d82-a0a8-56de6ce57a01 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] 10.222.0.16 "GET /compute/v2.1/servers/f6c350c4-b1b2-42d3-ba02-65cbbacf0f58" status: 200 len: 1277 microversion: 2.1 time: 0.215579 Jul 16 21:15:54.475688 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 932/1784] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:15:54 2022] GET /compute/v2.1/servers/f6c350c4-b1b2-42d3-ba02-65cbbacf0f58 => generated 1277 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:15:54.493564 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-a3e4b604-78bd-4ef0-8a83-fcad039fd763 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:15:54.494511 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-a3e4b604-78bd-4ef0-8a83-fcad039fd763 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] [instance: 11194822-5d42-45f4-99d0-88c70c808ddd] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:15:54.601913 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-769cb3d9-c84e-4182-bb40-d92de2213bdb tempest-FloatingIPDetailsTestJSON-1149902298 tempest-FloatingIPDetailsTestJSON-1149902298-project] 10.222.0.16 "DELETE /compute/v2.1/os-floating-ips/2b304a01-e907-458c-89d1-bb580de2eb05" status: 202 len: 0 microversion: 2.1 time: 2.124088 Jul 16 21:15:54.602632 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 853/1785] 10.222.0.16 () {58 vars in 1262 bytes} [Sat Jul 16 21:15:52 2022] DELETE /compute/v2.1/os-floating-ips/2b304a01-e907-458c-89d1-bb580de2eb05 => generated 0 bytes in 2125 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 21:15:54.610780 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-26dabf06-3872-4acc-a724-b10cbcb401d4 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:15:54.612020 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-26dabf06-3872-4acc-a724-b10cbcb401d4 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] [instance: 84ab7e65-1bf0-4b8b-baea-6d1d4ca13dd7] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:15:54.619726 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-26dabf06-3872-4acc-a724-b10cbcb401d4 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:54.620435 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-26dabf06-3872-4acc-a724-b10cbcb401d4 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:54.621404 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-26dabf06-3872-4acc-a724-b10cbcb401d4 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:54.621874 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-26dabf06-3872-4acc-a724-b10cbcb401d4 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:54.742060 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-a3e4b604-78bd-4ef0-8a83-fcad039fd763 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd4144b5388b46a989a41a2f556bb508', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a90d4bcd25ff4bf68bdc194180fbbe09', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:15:54.750540 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-a3e4b604-78bd-4ef0-8a83-fcad039fd763 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] 10.222.0.16 "GET /compute/v2.1/servers/11194822-5d42-45f4-99d0-88c70c808ddd" status: 200 len: 1217 microversion: 2.1 time: 0.271829 Jul 16 21:15:54.751153 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 933/1786] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:15:54 2022] GET /compute/v2.1/servers/11194822-5d42-45f4-99d0-88c70c808ddd => generated 1217 bytes in 275 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:15:54.757515 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-00bfe28b-11d1-4d9f-af15-f6c8a58a9bb6 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Action: 'action', calling method: >, body: {"createImage": {"name": "tempest-test-snap-393449576"}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:15:54.759611 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-00bfe28b-11d1-4d9f-af15-f6c8a58a9bb6 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] [instance: 93e327c3-c63d-439a-9b25-1a805f785949] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:15:54.768789 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-00bfe28b-11d1-4d9f-af15-f6c8a58a9bb6 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:54.769617 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-00bfe28b-11d1-4d9f-af15-f6c8a58a9bb6 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:54.770649 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-00bfe28b-11d1-4d9f-af15-f6c8a58a9bb6 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:54.771309 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-00bfe28b-11d1-4d9f-af15-f6c8a58a9bb6 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:55.033186 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-26dabf06-3872-4acc-a724-b10cbcb401d4 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:15:55.042355 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-26dabf06-3872-4acc-a724-b10cbcb401d4 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:15:55.046221 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-26dabf06-3872-4acc-a724-b10cbcb401d4 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:15:55.048151 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-26dabf06-3872-4acc-a724-b10cbcb401d4 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] 10.222.0.16 "GET /compute/v2.1/servers/84ab7e65-1bf0-4b8b-baea-6d1d4ca13dd7" status: 200 len: 1487 microversion: 2.25 time: 0.441230 Jul 16 21:15:55.061149 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 854/1787] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:15:54 2022] GET /compute/v2.1/servers/84ab7e65-1bf0-4b8b-baea-6d1d4ca13dd7 => generated 1487 bytes in 457 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:15:55.065929 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-b53efae4-a2a4-4018-8744-a260eb738edb tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:15:55.066551 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-b53efae4-a2a4-4018-8744-a260eb738edb tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] [instance: b9f2c527-6346-4403-841e-b7bd876a5fb7] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:15:55.073357 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b53efae4-a2a4-4018-8744-a260eb738edb tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:55.073775 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b53efae4-a2a4-4018-8744-a260eb738edb tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:55.074554 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b53efae4-a2a4-4018-8744-a260eb738edb tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:55.074997 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b53efae4-a2a4-4018-8744-a260eb738edb tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:55.149381 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-b53efae4-a2a4-4018-8744-a260eb738edb tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aa99be675e946fd995d71465b4cbec6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd82c2d92c67f469e863eb70bb7f9af84', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:15:55.158752 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-b53efae4-a2a4-4018-8744-a260eb738edb tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] 10.222.0.16 "GET /compute/v2.1/servers/b9f2c527-6346-4403-841e-b7bd876a5fb7" status: 200 len: 1237 microversion: 2.1 time: 0.097124 Jul 16 21:15:55.159790 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 855/1788] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:15:55 2022] GET /compute/v2.1/servers/b9f2c527-6346-4403-841e-b7bd876a5fb7 => generated 1237 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:15:55.164560 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-3ece46d1-d51c-4777-a2e7-3fd3402720e2 tempest-FloatingIPDetailsTestJSON-1149902298 tempest-FloatingIPDetailsTestJSON-1149902298-project] Calling method '.version_select at 0x7fdda2b89700>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:15:55.454273 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-00bfe28b-11d1-4d9f-af15-f6c8a58a9bb6 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] 10.222.0.16 "POST /compute/v2.1/servers/93e327c3-c63d-439a-9b25-1a805f785949/action" status: 202 len: 0 microversion: 2.1 time: 0.700935 Jul 16 21:15:55.461699 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 934/1789] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 21:15:54 2022] POST /compute/v2.1/servers/93e327c3-c63d-439a-9b25-1a805f785949/action => generated 0 bytes in 706 msecs (HTTP/1.1 202) 10 headers in 450 bytes (1 switches on core 0) Jul 16 21:15:55.465879 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-ba955d9c-ce32-46e7-9429-7c2433ce72d6 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:15:55.466596 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-ba955d9c-ce32-46e7-9429-7c2433ce72d6 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] [instance: 23df6e0a-3637-4f59-93f9-b02dd614b5c3] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:15:55.483293 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ba955d9c-ce32-46e7-9429-7c2433ce72d6 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:55.483747 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ba955d9c-ce32-46e7-9429-7c2433ce72d6 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:55.484559 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ba955d9c-ce32-46e7-9429-7c2433ce72d6 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:55.488476 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ba955d9c-ce32-46e7-9429-7c2433ce72d6 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:56.301975 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-ba955d9c-ce32-46e7-9429-7c2433ce72d6 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a69f9dee03d43beaafa966216b9e289', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9b769bf8f3ea4e368bf68de1df6538f9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:15:56.310699 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-ba955d9c-ce32-46e7-9429-7c2433ce72d6 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] 10.222.0.16 "GET /compute/v2.1/servers/23df6e0a-3637-4f59-93f9-b02dd614b5c3" status: 200 len: 1389 microversion: 2.1 time: 0.851712 Jul 16 21:15:56.319645 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 935/1790] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:15:55 2022] GET /compute/v2.1/servers/23df6e0a-3637-4f59-93f9-b02dd614b5c3 => generated 1389 bytes in 853 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:15:56.319645 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-e9fbd8be-d7f5-4814-8668-8520cebb51aa req-ee7a374f-856c-4f78-86e7-466525a9e242 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "23df6e0a-3637-4f59-93f9-b02dd614b5c3", "tag": "60e499ad-afcb-4fe7-80ef-058f63c4fc21"}, {"server_uuid": "0079e567-30df-4cac-a0ad-4379bcbbcf48", "name": "network-vif-deleted", "tag": "08381e59-62d7-471e-a05a-7de705ec2dae"}, {"server_uuid": "0079e567-30df-4cac-a0ad-4379bcbbcf48", "name": "network-vif-deleted", "tag": "1079a73b-5553-45b8-bff2-e4be618fbceb"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:15:56.335685 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-e9fbd8be-d7f5-4814-8668-8520cebb51aa req-ee7a374f-856c-4f78-86e7-466525a9e242 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:56.336230 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-e9fbd8be-d7f5-4814-8668-8520cebb51aa req-ee7a374f-856c-4f78-86e7-466525a9e242 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:56.353508 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-e9fbd8be-d7f5-4814-8668-8520cebb51aa req-ee7a374f-856c-4f78-86e7-466525a9e242 service nova] Creating event network-changed:60e499ad-afcb-4fe7-80ef-058f63c4fc21 for instance 23df6e0a-3637-4f59-93f9-b02dd614b5c3 on nch2-master-1 Jul 16 21:15:56.354382 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-e9fbd8be-d7f5-4814-8668-8520cebb51aa req-ee7a374f-856c-4f78-86e7-466525a9e242 service nova] Creating event network-vif-deleted:08381e59-62d7-471e-a05a-7de705ec2dae for instance 0079e567-30df-4cac-a0ad-4379bcbbcf48 on nch2-master-1 Jul 16 21:15:56.355242 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-e9fbd8be-d7f5-4814-8668-8520cebb51aa req-ee7a374f-856c-4f78-86e7-466525a9e242 service nova] Creating event network-vif-deleted:1079a73b-5553-45b8-bff2-e4be618fbceb for instance 0079e567-30df-4cac-a0ad-4379bcbbcf48 on nch2-master-1 Jul 16 21:15:56.364364 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-e9fbd8be-d7f5-4814-8668-8520cebb51aa req-ee7a374f-856c-4f78-86e7-466525a9e242 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 521 microversion: 2.1 time: 0.051093 Jul 16 21:15:56.365753 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 936/1791] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:15:56 2022] POST /compute/v2.1/os-server-external-events => generated 521 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:15:56.375190 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-17e8217c-6998-4fe6-9b36-af67ad886e0b tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Calling method '.version_select at 0x7fdda2b9a0d0>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:15:56.470682 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-17e8217c-6998-4fe6-9b36-af67ad886e0b tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] 10.222.0.16 "GET /compute/v2.1/images/5af5e2eb-8145-46b4-abad-4d2a1e3630fc" status: 200 len: 1413 microversion: 2.1 time: 0.102819 Jul 16 21:15:56.479847 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 937/1792] 10.222.0.16 () {58 vars in 1232 bytes} [Sat Jul 16 21:15:56 2022] GET /compute/v2.1/images/5af5e2eb-8145-46b4-abad-4d2a1e3630fc => generated 1413 bytes in 112 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:15:56.484799 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-b4de1228-27e5-4acc-a57c-c1937a6e4861 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:15:56.485699 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-b4de1228-27e5-4acc-a57c-c1937a6e4861 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] [instance: f6c350c4-b1b2-42d3-ba02-65cbbacf0f58] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:15:56.494450 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b4de1228-27e5-4acc-a57c-c1937a6e4861 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:56.495189 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b4de1228-27e5-4acc-a57c-c1937a6e4861 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:56.496967 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b4de1228-27e5-4acc-a57c-c1937a6e4861 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:56.498760 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b4de1228-27e5-4acc-a57c-c1937a6e4861 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:56.737194 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-b4de1228-27e5-4acc-a57c-c1937a6e4861 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4e9df82f95ac4779859972403869096a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4337575622d04a26942905c9dc044ba7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:15:56.753931 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-b4de1228-27e5-4acc-a57c-c1937a6e4861 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] 10.222.0.16 "GET /compute/v2.1/servers/f6c350c4-b1b2-42d3-ba02-65cbbacf0f58" status: 200 len: 1283 microversion: 2.1 time: 0.270995 Jul 16 21:15:56.755873 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 938/1793] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:15:56 2022] GET /compute/v2.1/servers/f6c350c4-b1b2-42d3-ba02-65cbbacf0f58 => generated 1283 bytes in 276 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:15:56.775565 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-e4f60cc6-a961-431a-9e2d-b68ac2b911fb tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:15:56.776190 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-e4f60cc6-a961-431a-9e2d-b68ac2b911fb tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] [instance: 11194822-5d42-45f4-99d0-88c70c808ddd] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:15:56.794462 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e4f60cc6-a961-431a-9e2d-b68ac2b911fb tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:56.795288 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e4f60cc6-a961-431a-9e2d-b68ac2b911fb tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:56.800079 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e4f60cc6-a961-431a-9e2d-b68ac2b911fb tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:56.800932 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e4f60cc6-a961-431a-9e2d-b68ac2b911fb tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:57.006160 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-e4f60cc6-a961-431a-9e2d-b68ac2b911fb tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd4144b5388b46a989a41a2f556bb508', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a90d4bcd25ff4bf68bdc194180fbbe09', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:15:57.014870 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-e4f60cc6-a961-431a-9e2d-b68ac2b911fb tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] 10.222.0.16 "GET /compute/v2.1/servers/11194822-5d42-45f4-99d0-88c70c808ddd" status: 200 len: 1277 microversion: 2.1 time: 0.257453 Jul 16 21:15:57.015640 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 939/1794] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:15:56 2022] GET /compute/v2.1/servers/11194822-5d42-45f4-99d0-88c70c808ddd => generated 1277 bytes in 259 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:15:57.039299 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-5b8aa730-3842-4e2c-8dca-1c734b5fae31 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:15:57.043101 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-5b8aa730-3842-4e2c-8dca-1c734b5fae31 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] [instance: 84ab7e65-1bf0-4b8b-baea-6d1d4ca13dd7] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:15:57.057350 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5b8aa730-3842-4e2c-8dca-1c734b5fae31 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:57.058130 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5b8aa730-3842-4e2c-8dca-1c734b5fae31 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:57.061141 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5b8aa730-3842-4e2c-8dca-1c734b5fae31 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:57.062394 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5b8aa730-3842-4e2c-8dca-1c734b5fae31 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:57.733398 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-5b8aa730-3842-4e2c-8dca-1c734b5fae31 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:15:57.786699 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-5b8aa730-3842-4e2c-8dca-1c734b5fae31 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:15:57.803703 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-5b8aa730-3842-4e2c-8dca-1c734b5fae31 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:15:57.807584 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-5b8aa730-3842-4e2c-8dca-1c734b5fae31 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] 10.222.0.16 "GET /compute/v2.1/servers/84ab7e65-1bf0-4b8b-baea-6d1d4ca13dd7" status: 200 len: 1487 microversion: 2.25 time: 0.770233 Jul 16 21:15:57.808579 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 940/1795] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:15:57 2022] GET /compute/v2.1/servers/84ab7e65-1bf0-4b8b-baea-6d1d4ca13dd7 => generated 1487 bytes in 774 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:15:57.834338 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-556ff280-2b99-4580-9715-a3fcb66e81f2 tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:15:57.836971 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-556ff280-2b99-4580-9715-a3fcb66e81f2 tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] [instance: b9f2c527-6346-4403-841e-b7bd876a5fb7] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:15:57.855128 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-556ff280-2b99-4580-9715-a3fcb66e81f2 tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:57.855128 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-556ff280-2b99-4580-9715-a3fcb66e81f2 tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:57.856128 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-556ff280-2b99-4580-9715-a3fcb66e81f2 tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:57.856754 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-556ff280-2b99-4580-9715-a3fcb66e81f2 tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:58.920500 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-556ff280-2b99-4580-9715-a3fcb66e81f2 tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aa99be675e946fd995d71465b4cbec6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd82c2d92c67f469e863eb70bb7f9af84', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:15:58.926077 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=96,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:15:58.930943 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-556ff280-2b99-4580-9715-a3fcb66e81f2 tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] 10.222.0.16 "GET /compute/v2.1/servers/b9f2c527-6346-4403-841e-b7bd876a5fb7" status: 200 len: 1293 microversion: 2.1 time: 1.116645 Jul 16 21:15:58.932422 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 941/1796] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:15:57 2022] GET /compute/v2.1/servers/b9f2c527-6346-4403-841e-b7bd876a5fb7 => generated 1293 bytes in 1119 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:15:58.937250 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-c7a99559-cd37-4fab-aba2-54639526acf2 req-e434884e-237b-4518-9e5b-f42ee37f41c7 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "23df6e0a-3637-4f59-93f9-b02dd614b5c3", "name": "network-vif-unplugged", "status": "completed", "tag": "60e499ad-afcb-4fe7-80ef-058f63c4fc21"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:15:58.948923 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-c7a99559-cd37-4fab-aba2-54639526acf2 req-e434884e-237b-4518-9e5b-f42ee37f41c7 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:58.949388 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-c7a99559-cd37-4fab-aba2-54639526acf2 req-e434884e-237b-4518-9e5b-f42ee37f41c7 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:58.964548 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-c7a99559-cd37-4fab-aba2-54639526acf2 req-e434884e-237b-4518-9e5b-f42ee37f41c7 service nova] Creating event network-vif-unplugged:60e499ad-afcb-4fe7-80ef-058f63c4fc21 for instance 23df6e0a-3637-4f59-93f9-b02dd614b5c3 on nch2-master-1 Jul 16 21:15:58.970371 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-c7a99559-cd37-4fab-aba2-54639526acf2 req-e434884e-237b-4518-9e5b-f42ee37f41c7 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.036435 Jul 16 21:15:58.975606 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 942/1797] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:15:58 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:15:58.981801 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-988873b0-1cf0-47b6-a0b1-613019b3388c tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:15:58.985134 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-988873b0-1cf0-47b6-a0b1-613019b3388c tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] [instance: 23df6e0a-3637-4f59-93f9-b02dd614b5c3] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:15:58.991481 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-988873b0-1cf0-47b6-a0b1-613019b3388c tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:58.992016 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-988873b0-1cf0-47b6-a0b1-613019b3388c tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:58.992834 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-988873b0-1cf0-47b6-a0b1-613019b3388c tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:58.993269 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-988873b0-1cf0-47b6-a0b1-613019b3388c tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:59.539289 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-3ece46d1-d51c-4777-a2e7-3fd3402720e2 tempest-FloatingIPDetailsTestJSON-1149902298 tempest-FloatingIPDetailsTestJSON-1149902298-project] 10.222.0.16 "DELETE /compute/v2.1/os-floating-ips/e0a728db-955c-4f05-93b0-022f8c499503" status: 202 len: 0 microversion: 2.1 time: 4.378435 Jul 16 21:15:59.539890 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 856/1798] 10.222.0.16 () {58 vars in 1262 bytes} [Sat Jul 16 21:15:55 2022] DELETE /compute/v2.1/os-floating-ips/e0a728db-955c-4f05-93b0-022f8c499503 => generated 0 bytes in 4380 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 21:15:59.544521 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-caad92fb-c925-49c9-9b5c-eeb2894073ac tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Calling method '.version_select at 0x7fdda2b36310>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:15:59.608835 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-caad92fb-c925-49c9-9b5c-eeb2894073ac tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] 10.222.0.16 "GET /compute/v2.1/images/5af5e2eb-8145-46b4-abad-4d2a1e3630fc" status: 200 len: 1413 microversion: 2.1 time: 0.067892 Jul 16 21:15:59.609420 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 857/1799] 10.222.0.16 () {58 vars in 1232 bytes} [Sat Jul 16 21:15:59 2022] GET /compute/v2.1/images/5af5e2eb-8145-46b4-abad-4d2a1e3630fc => generated 1413 bytes in 69 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:15:59.615579 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-4cc0017e-e107-445c-bbb9-857d3ec0c8a2 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:15:59.621504 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-4cc0017e-e107-445c-bbb9-857d3ec0c8a2 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] [instance: f6c350c4-b1b2-42d3-ba02-65cbbacf0f58] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:15:59.635018 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4cc0017e-e107-445c-bbb9-857d3ec0c8a2 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:59.635485 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4cc0017e-e107-445c-bbb9-857d3ec0c8a2 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:59.636271 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4cc0017e-e107-445c-bbb9-857d3ec0c8a2 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:59.638788 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4cc0017e-e107-445c-bbb9-857d3ec0c8a2 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:59.728894 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-988873b0-1cf0-47b6-a0b1-613019b3388c tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a69f9dee03d43beaafa966216b9e289', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9b769bf8f3ea4e368bf68de1df6538f9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:15:59.738212 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-988873b0-1cf0-47b6-a0b1-613019b3388c tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] 10.222.0.16 "GET /compute/v2.1/servers/23df6e0a-3637-4f59-93f9-b02dd614b5c3" status: 200 len: 1389 microversion: 2.1 time: 0.761079 Jul 16 21:15:59.740406 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 943/1800] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:15:58 2022] GET /compute/v2.1/servers/23df6e0a-3637-4f59-93f9-b02dd614b5c3 => generated 1389 bytes in 764 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:15:59.746322 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-a90347bf-e9a9-49fc-be86-a5fa4a3d474c tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:15:59.747986 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-a90347bf-e9a9-49fc-be86-a5fa4a3d474c tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] [instance: 11194822-5d42-45f4-99d0-88c70c808ddd] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:15:59.757649 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a90347bf-e9a9-49fc-be86-a5fa4a3d474c tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:59.758101 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a90347bf-e9a9-49fc-be86-a5fa4a3d474c tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:59.759175 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a90347bf-e9a9-49fc-be86-a5fa4a3d474c tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:59.759616 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a90347bf-e9a9-49fc-be86-a5fa4a3d474c tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:59.824729 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-4cc0017e-e107-445c-bbb9-857d3ec0c8a2 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4e9df82f95ac4779859972403869096a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4337575622d04a26942905c9dc044ba7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:15:59.834844 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-4cc0017e-e107-445c-bbb9-857d3ec0c8a2 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] 10.222.0.16 "GET /compute/v2.1/servers/f6c350c4-b1b2-42d3-ba02-65cbbacf0f58" status: 200 len: 1283 microversion: 2.1 time: 0.224054 Jul 16 21:15:59.836619 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 858/1801] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:15:59 2022] GET /compute/v2.1/servers/f6c350c4-b1b2-42d3-ba02-65cbbacf0f58 => generated 1283 bytes in 226 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:15:59.843133 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-aaeaf0fd-0735-4cf2-8a72-2c9767202046 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:15:59.843540 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-aaeaf0fd-0735-4cf2-8a72-2c9767202046 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] [instance: 0079e567-30df-4cac-a0ad-4379bcbbcf48] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:15:59.852220 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-aaeaf0fd-0735-4cf2-8a72-2c9767202046 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:59.853948 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-aaeaf0fd-0735-4cf2-8a72-2c9767202046 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:59.863716 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-aaeaf0fd-0735-4cf2-8a72-2c9767202046 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:59.864738 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-aaeaf0fd-0735-4cf2-8a72-2c9767202046 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:59.890402 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-aaeaf0fd-0735-4cf2-8a72-2c9767202046 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] HTTP exception thrown: Instance 0079e567-30df-4cac-a0ad-4379bcbbcf48 could not be found. Jul 16 21:15:59.892848 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-aaeaf0fd-0735-4cf2-8a72-2c9767202046 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Request method failure captured: Jul 16 21:15:59.892848 ncdv-master-1 devstack@n-api.service[138681]: request: DELETE /compute/v2.1/servers/0079e567-30df-4cac-a0ad-4379bcbbcf48 HTTP/1.1 Jul 16 21:15:59.892848 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:15:59.892848 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:15:59.892848 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:15:59.892848 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 21:15:59.892848 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:15:59.892848 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:15:59.892848 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:15:59.892848 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:15:59.892848 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0yoCD7IX15brFQUHVPKPIJrpEu20fxENHCEz31UnXrkVZbNHxjQ0ElqFOE7MXQnga3bWs8iQFWXWTDFYHn6RqK2SreYxppaBRq4w7eCzu-cpcuRq-bR3warXbUzkwx17fxxMPjZ200EVRXuxxzJd_xS8-ssDDdZBxRLNj6zLIJYowVk Jul 16 21:15:59.892848 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:15:59.892848 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:15:59.892848 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:15:59.892848 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:15:59.892848 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:15:59.892848 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:15:59.892848 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: bb1ede0d0b6941cc97278e6c617f9b01 Jul 16 21:15:59.892848 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-ServersTestMultiNic-262233723 Jul 16 21:15:59.892848 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:15:59.892848 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:15:59.892848 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/bb1ede0d0b6941cc97278e6c617f9b01"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/bb1ede0d0b6941cc97278e6c617f9b01"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_bb1ede0d0b6941cc97278e6c617f9b01"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/bb1ede0d0b6941cc97278e6c617f9b01"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:15:59.892848 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-ServersTestMultiNic-262233723 Jul 16 21:15:59.892848 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: bb1ede0d0b6941cc97278e6c617f9b01 Jul 16 21:15:59.892848 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-ServersTestMultiNic-262233723 Jul 16 21:15:59.892848 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-ServersTestMultiNic-262233723-project Jul 16 21:15:59.892848 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:15:59.892848 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:15:59.892848 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: 53af488f3b0242a797f539284b78d59e Jul 16 21:15:59.892848 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-ServersTestMultiNic-262233723-project Jul 16 21:15:59.892848 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:15:59.892848 ncdv-master-1 devstack@n-api.service[138681]: exception: Instance 0079e567-30df-4cac-a0ad-4379bcbbcf48 could not be found. Jul 16 21:15:59.892848 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': '0079e567-30df-4cac-a0ad-4379bcbbcf48'} Jul 16 21:15:59.892848 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:15:59.892848 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:15:59.892848 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:15:59.892848 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:15:59.892848 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:15:59.892848 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:15:59.892848 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:15:59.892848 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:15:59.892848 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:15:59.892848 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:15:59.892848 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 0079e567-30df-4cac-a0ad-4379bcbbcf48 could not be found. Jul 16 21:15:59.892848 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:15:59.892848 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:15:59.892848 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:15:59.892848 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:15:59.892848 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:15:59.892848 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:15:59.892848 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:15:59.892848 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:15:59.892848 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:15:59.892848 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:15:59.892848 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Jul 16 21:15:59.892848 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Jul 16 21:15:59.892848 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Jul 16 21:15:59.892848 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Jul 16 21:15:59.892848 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:15:59.892848 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:15:59.892848 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:15:59.892848 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:15:59.892848 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 0079e567-30df-4cac-a0ad-4379bcbbcf48 could not be found. Jul 16 21:15:59.902366 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:15:59.902366 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:15:59.902366 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:15:59.902366 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:15:59.902366 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:15:59.902366 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:15:59.902366 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:15:59.902366 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:15:59.902366 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 0079e567-30df-4cac-a0ad-4379bcbbcf48 could not be found. Jul 16 21:15:59.902366 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:15:59.902366 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-aaeaf0fd-0735-4cf2-8a72-2c9767202046 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Returning 404 to user: Instance 0079e567-30df-4cac-a0ad-4379bcbbcf48 could not be found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:15:59.902366 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-aaeaf0fd-0735-4cf2-8a72-2c9767202046 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] 10.222.0.16 "DELETE /compute/v2.1/servers/0079e567-30df-4cac-a0ad-4379bcbbcf48" status: 404 len: 111 microversion: 2.1 time: 0.056949 Jul 16 21:15:59.902366 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 859/1802] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:15:59 2022] DELETE /compute/v2.1/servers/0079e567-30df-4cac-a0ad-4379bcbbcf48 => generated 111 bytes in 58 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:15:59.902366 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-fc2732b9-1efb-43ce-a8fb-97f90f7c9d7a req-d6a5fcc5-c40b-4c6d-b5db-2fe95497cb28 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "84ab7e65-1bf0-4b8b-baea-6d1d4ca13dd7", "name": "network-vif-unplugged", "status": "completed", "tag": "42dfc89b-90fd-4214-983c-f02029faca17"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:15:59.912473 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-fc2732b9-1efb-43ce-a8fb-97f90f7c9d7a req-d6a5fcc5-c40b-4c6d-b5db-2fe95497cb28 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:59.913153 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-fc2732b9-1efb-43ce-a8fb-97f90f7c9d7a req-d6a5fcc5-c40b-4c6d-b5db-2fe95497cb28 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:59.939857 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-fc2732b9-1efb-43ce-a8fb-97f90f7c9d7a req-d6a5fcc5-c40b-4c6d-b5db-2fe95497cb28 service nova] Creating event network-vif-unplugged:42dfc89b-90fd-4214-983c-f02029faca17 for instance 84ab7e65-1bf0-4b8b-baea-6d1d4ca13dd7 on nch1-master-1 Jul 16 21:15:59.947070 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-fc2732b9-1efb-43ce-a8fb-97f90f7c9d7a req-d6a5fcc5-c40b-4c6d-b5db-2fe95497cb28 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.049230 Jul 16 21:15:59.948116 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 860/1803] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:15:59 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:15:59.955416 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-74bc1e61-9920-4bd4-b57c-8031d47f1cf6 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:15:59.956145 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-74bc1e61-9920-4bd4-b57c-8031d47f1cf6 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] [instance: 84ab7e65-1bf0-4b8b-baea-6d1d4ca13dd7] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:15:59.965933 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-74bc1e61-9920-4bd4-b57c-8031d47f1cf6 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:59.969839 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-74bc1e61-9920-4bd4-b57c-8031d47f1cf6 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:15:59.969839 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-74bc1e61-9920-4bd4-b57c-8031d47f1cf6 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:15:59.969839 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-74bc1e61-9920-4bd4-b57c-8031d47f1cf6 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:00.005072 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-a90347bf-e9a9-49fc-be86-a5fa4a3d474c tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd4144b5388b46a989a41a2f556bb508', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a90d4bcd25ff4bf68bdc194180fbbe09', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:00.043666 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-a90347bf-e9a9-49fc-be86-a5fa4a3d474c tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] 10.222.0.16 "GET /compute/v2.1/servers/11194822-5d42-45f4-99d0-88c70c808ddd" status: 200 len: 1275 microversion: 2.1 time: 0.296109 Jul 16 21:16:00.055733 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 944/1804] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:15:59 2022] GET /compute/v2.1/servers/11194822-5d42-45f4-99d0-88c70c808ddd => generated 1275 bytes in 314 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:16:00.067913 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-b4988d3e-3fbf-4fe5-90ef-3f67d3b82245 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:00.071871 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-b4988d3e-3fbf-4fe5-90ef-3f67d3b82245 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] [instance: 0079e567-30df-4cac-a0ad-4379bcbbcf48] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:00.080310 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b4988d3e-3fbf-4fe5-90ef-3f67d3b82245 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:00.081233 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b4988d3e-3fbf-4fe5-90ef-3f67d3b82245 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:00.082860 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b4988d3e-3fbf-4fe5-90ef-3f67d3b82245 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:00.083552 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b4988d3e-3fbf-4fe5-90ef-3f67d3b82245 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:00.103970 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-b4988d3e-3fbf-4fe5-90ef-3f67d3b82245 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] HTTP exception thrown: Instance 0079e567-30df-4cac-a0ad-4379bcbbcf48 could not be found. Jul 16 21:16:00.107210 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-b4988d3e-3fbf-4fe5-90ef-3f67d3b82245 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Request method failure captured: Jul 16 21:16:00.107210 ncdv-master-1 devstack@n-api.service[138680]: request: GET /compute/v2.1/servers/0079e567-30df-4cac-a0ad-4379bcbbcf48 HTTP/1.1 Jul 16 21:16:00.107210 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:16:00.107210 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:16:00.107210 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:16:00.107210 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 0 Jul 16 21:16:00.107210 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:16:00.107210 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:16:00.107210 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:16:00.107210 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:16:00.107210 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0yoCD7IX15brFQUHVPKPIJrpEu20fxENHCEz31UnXrkVZbNHxjQ0ElqFOE7MXQnga3bWs8iQFWXWTDFYHn6RqK2SreYxppaBRq4w7eCzu-cpcuRq-bR3warXbUzkwx17fxxMPjZ200EVRXuxxzJd_xS8-ssDDdZBxRLNj6zLIJYowVk Jul 16 21:16:00.107210 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:16:00.107210 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:16:00.107210 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:16:00.107210 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:16:00.107210 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:16:00.107210 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:16:00.107210 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: bb1ede0d0b6941cc97278e6c617f9b01 Jul 16 21:16:00.107210 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-ServersTestMultiNic-262233723 Jul 16 21:16:00.107210 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:16:00.107210 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:16:00.107210 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/bb1ede0d0b6941cc97278e6c617f9b01"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/bb1ede0d0b6941cc97278e6c617f9b01"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_bb1ede0d0b6941cc97278e6c617f9b01"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/bb1ede0d0b6941cc97278e6c617f9b01"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:16:00.107210 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-ServersTestMultiNic-262233723 Jul 16 21:16:00.107210 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: bb1ede0d0b6941cc97278e6c617f9b01 Jul 16 21:16:00.107210 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-ServersTestMultiNic-262233723 Jul 16 21:16:00.107210 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-ServersTestMultiNic-262233723-project Jul 16 21:16:00.107210 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:16:00.107210 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:16:00.107210 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: 53af488f3b0242a797f539284b78d59e Jul 16 21:16:00.107210 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-ServersTestMultiNic-262233723-project Jul 16 21:16:00.107210 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 21:16:00.107210 ncdv-master-1 devstack@n-api.service[138680]: exception: Instance 0079e567-30df-4cac-a0ad-4379bcbbcf48 could not be found. Jul 16 21:16:00.107210 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': '0079e567-30df-4cac-a0ad-4379bcbbcf48'} Jul 16 21:16:00.107210 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:16:00.107210 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:16:00.107210 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:16:00.107210 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:16:00.107210 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:16:00.107210 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:16:00.107210 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:16:00.107210 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:16:00.107210 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:16:00.107210 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:16:00.107210 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 0079e567-30df-4cac-a0ad-4379bcbbcf48 could not be found. Jul 16 21:16:00.107210 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:16:00.107210 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:16:00.107210 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:16:00.107210 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:16:00.107210 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:16:00.107210 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:16:00.107210 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:16:00.107210 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:16:00.107210 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:16:00.107210 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:16:00.107210 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:16:00.107210 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:16:00.107210 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:16:00.107210 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:16:00.107210 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:16:00.107210 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:16:00.107210 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 0079e567-30df-4cac-a0ad-4379bcbbcf48 could not be found. Jul 16 21:16:00.107210 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:16:00.107210 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:16:00.107210 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:16:00.116473 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:16:00.116473 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:16:00.116473 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:16:00.116473 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:16:00.116473 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:16:00.116473 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 0079e567-30df-4cac-a0ad-4379bcbbcf48 could not be found. Jul 16 21:16:00.116473 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:16:00.121647 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-b4988d3e-3fbf-4fe5-90ef-3f67d3b82245 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Returning 404 to user: Instance 0079e567-30df-4cac-a0ad-4379bcbbcf48 could not be found. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:16:00.123592 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-b4988d3e-3fbf-4fe5-90ef-3f67d3b82245 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] 10.222.0.16 "GET /compute/v2.1/servers/0079e567-30df-4cac-a0ad-4379bcbbcf48" status: 404 len: 111 microversion: 2.1 time: 0.066181 Jul 16 21:16:00.128558 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 945/1805] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:16:00 2022] GET /compute/v2.1/servers/0079e567-30df-4cac-a0ad-4379bcbbcf48 => generated 111 bytes in 72 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:16:00.136483 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-b747b1b2-f5a6-48c0-9492-0b67a7ad9b2f tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:00.137749 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-b747b1b2-f5a6-48c0-9492-0b67a7ad9b2f tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] [instance: b9f2c527-6346-4403-841e-b7bd876a5fb7] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:00.146374 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b747b1b2-f5a6-48c0-9492-0b67a7ad9b2f tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:00.147360 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b747b1b2-f5a6-48c0-9492-0b67a7ad9b2f tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:00.154183 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b747b1b2-f5a6-48c0-9492-0b67a7ad9b2f tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:00.155841 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b747b1b2-f5a6-48c0-9492-0b67a7ad9b2f tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:00.861994 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-74bc1e61-9920-4bd4-b57c-8031d47f1cf6 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:00.889766 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-74bc1e61-9920-4bd4-b57c-8031d47f1cf6 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:00.894496 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-74bc1e61-9920-4bd4-b57c-8031d47f1cf6 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:00.898891 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-74bc1e61-9920-4bd4-b57c-8031d47f1cf6 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] 10.222.0.16 "GET /compute/v2.1/servers/84ab7e65-1bf0-4b8b-baea-6d1d4ca13dd7" status: 200 len: 1487 microversion: 2.25 time: 0.949473 Jul 16 21:16:00.903675 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 861/1806] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:15:59 2022] GET /compute/v2.1/servers/84ab7e65-1bf0-4b8b-baea-6d1d4ca13dd7 => generated 1487 bytes in 955 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:16:00.914624 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-35cf8ae6-3513-4e02-9862-9e6d97329347 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:00.916730 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-35cf8ae6-3513-4e02-9862-9e6d97329347 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] [instance: 69cabe57-da71-4a02-b4ee-c0d84b3d32a2] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:00.932370 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-35cf8ae6-3513-4e02-9862-9e6d97329347 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:00.937098 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-35cf8ae6-3513-4e02-9862-9e6d97329347 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:00.938071 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-35cf8ae6-3513-4e02-9862-9e6d97329347 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:00.938605 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-35cf8ae6-3513-4e02-9862-9e6d97329347 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:00.979342 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-35cf8ae6-3513-4e02-9862-9e6d97329347 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] HTTP exception thrown: Instance 69cabe57-da71-4a02-b4ee-c0d84b3d32a2 could not be found. Jul 16 21:16:00.981844 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-35cf8ae6-3513-4e02-9862-9e6d97329347 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Request method failure captured: Jul 16 21:16:00.981844 ncdv-master-1 devstack@n-api.service[138681]: request: DELETE /compute/v2.1/servers/69cabe57-da71-4a02-b4ee-c0d84b3d32a2 HTTP/1.1 Jul 16 21:16:00.981844 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:16:00.981844 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:16:00.981844 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:16:00.981844 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 21:16:00.981844 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:16:00.981844 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:16:00.981844 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:16:00.981844 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:16:00.981844 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0yoCD7IX15brFQUHVPKPIJrpEu20fxENHCEz31UnXrkVZbNHxjQ0ElqFOE7MXQnga3bWs8iQFWXWTDFYHn6RqK2SreYxppaBRq4w7eCzu-cpcuRq-bR3warXbUzkwx17fxxMPjZ200EVRXuxxzJd_xS8-ssDDdZBxRLNj6zLIJYowVk Jul 16 21:16:00.981844 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:16:00.981844 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:16:00.981844 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:16:00.981844 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:16:00.981844 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:16:00.981844 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:16:00.981844 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: bb1ede0d0b6941cc97278e6c617f9b01 Jul 16 21:16:00.981844 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-ServersTestMultiNic-262233723 Jul 16 21:16:00.981844 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:16:00.981844 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:16:00.981844 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/bb1ede0d0b6941cc97278e6c617f9b01"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/bb1ede0d0b6941cc97278e6c617f9b01"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_bb1ede0d0b6941cc97278e6c617f9b01"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/bb1ede0d0b6941cc97278e6c617f9b01"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:16:00.981844 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-ServersTestMultiNic-262233723 Jul 16 21:16:00.981844 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: bb1ede0d0b6941cc97278e6c617f9b01 Jul 16 21:16:00.981844 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-ServersTestMultiNic-262233723 Jul 16 21:16:00.981844 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-ServersTestMultiNic-262233723-project Jul 16 21:16:00.981844 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:16:00.981844 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:16:00.981844 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: 53af488f3b0242a797f539284b78d59e Jul 16 21:16:00.981844 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-ServersTestMultiNic-262233723-project Jul 16 21:16:00.981844 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:16:00.981844 ncdv-master-1 devstack@n-api.service[138681]: exception: Instance 69cabe57-da71-4a02-b4ee-c0d84b3d32a2 could not be found. Jul 16 21:16:00.981844 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': '69cabe57-da71-4a02-b4ee-c0d84b3d32a2'} Jul 16 21:16:00.981844 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:16:00.981844 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:16:00.981844 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:16:00.981844 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:16:00.981844 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:16:00.981844 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:16:00.981844 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:16:00.981844 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:16:00.981844 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:16:00.981844 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:16:00.981844 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 69cabe57-da71-4a02-b4ee-c0d84b3d32a2 could not be found. Jul 16 21:16:00.981844 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:16:00.981844 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:16:00.981844 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:16:00.981844 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:16:00.981844 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:16:00.981844 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:16:00.981844 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:16:00.981844 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:16:00.981844 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:16:00.981844 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:16:00.981844 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Jul 16 21:16:00.981844 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Jul 16 21:16:00.981844 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Jul 16 21:16:00.981844 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Jul 16 21:16:00.981844 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:16:00.981844 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:16:00.981844 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:16:00.981844 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:16:00.981844 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 69cabe57-da71-4a02-b4ee-c0d84b3d32a2 could not be found. Jul 16 21:16:00.993745 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:16:00.993745 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:16:00.993745 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:16:00.993745 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:16:00.993745 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:16:00.993745 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:16:00.993745 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:16:00.993745 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:16:00.993745 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 69cabe57-da71-4a02-b4ee-c0d84b3d32a2 could not be found. Jul 16 21:16:00.993745 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:16:00.993745 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-35cf8ae6-3513-4e02-9862-9e6d97329347 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Returning 404 to user: Instance 69cabe57-da71-4a02-b4ee-c0d84b3d32a2 could not be found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:16:00.993745 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-35cf8ae6-3513-4e02-9862-9e6d97329347 tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] 10.222.0.16 "DELETE /compute/v2.1/servers/69cabe57-da71-4a02-b4ee-c0d84b3d32a2" status: 404 len: 111 microversion: 2.1 time: 0.078268 Jul 16 21:16:00.993745 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 862/1807] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:16:00 2022] DELETE /compute/v2.1/servers/69cabe57-da71-4a02-b4ee-c0d84b3d32a2 => generated 111 bytes in 81 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:16:00.999220 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-d4bf5efe-0e63-49ca-8e67-609f5a7b6e16 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Calling method '.version_select at 0x7fdda2b86040>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:01.104297 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-d4bf5efe-0e63-49ca-8e67-609f5a7b6e16 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] 10.222.0.16 "GET /compute/v2.1/images/5af5e2eb-8145-46b4-abad-4d2a1e3630fc" status: 200 len: 1413 microversion: 2.1 time: 0.111453 Jul 16 21:16:01.106610 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 863/1808] 10.222.0.16 () {58 vars in 1232 bytes} [Sat Jul 16 21:16:00 2022] GET /compute/v2.1/images/5af5e2eb-8145-46b4-abad-4d2a1e3630fc => generated 1413 bytes in 120 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:16:01.116659 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-c000d549-ae36-457f-bc91-070dd50f3a12 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:01.119565 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-c000d549-ae36-457f-bc91-070dd50f3a12 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] [instance: 23df6e0a-3637-4f59-93f9-b02dd614b5c3] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:01.139420 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c000d549-ae36-457f-bc91-070dd50f3a12 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:01.140053 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c000d549-ae36-457f-bc91-070dd50f3a12 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:01.142630 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c000d549-ae36-457f-bc91-070dd50f3a12 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:01.145675 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c000d549-ae36-457f-bc91-070dd50f3a12 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:01.165137 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-b747b1b2-f5a6-48c0-9492-0b67a7ad9b2f tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aa99be675e946fd995d71465b4cbec6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd82c2d92c67f469e863eb70bb7f9af84', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:01.184896 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=92,nova_cell1:UPDATE=5,nova_cell1:INSERT=3 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:16:01.211348 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-b747b1b2-f5a6-48c0-9492-0b67a7ad9b2f tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] 10.222.0.16 "GET /compute/v2.1/servers/b9f2c527-6346-4403-841e-b7bd876a5fb7" status: 200 len: 1293 microversion: 2.1 time: 1.080265 Jul 16 21:16:01.212327 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 946/1809] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:16:00 2022] GET /compute/v2.1/servers/b9f2c527-6346-4403-841e-b7bd876a5fb7 => generated 1293 bytes in 1083 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:16:01.226682 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-c3415179-24bd-4cf7-ab8a-422b3c171f56 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:01.235094 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-c3415179-24bd-4cf7-ab8a-422b3c171f56 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] [instance: f6c350c4-b1b2-42d3-ba02-65cbbacf0f58] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:01.259281 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c3415179-24bd-4cf7-ab8a-422b3c171f56 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:01.274047 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c3415179-24bd-4cf7-ab8a-422b3c171f56 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.015s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:01.275805 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c3415179-24bd-4cf7-ab8a-422b3c171f56 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:01.276974 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c3415179-24bd-4cf7-ab8a-422b3c171f56 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:01.562005 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell0:SELECT=4 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:16:01.619669 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell0:SELECT=16 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:16:02.061502 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-c000d549-ae36-457f-bc91-070dd50f3a12 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a69f9dee03d43beaafa966216b9e289', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9b769bf8f3ea4e368bf68de1df6538f9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:02.077524 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-c000d549-ae36-457f-bc91-070dd50f3a12 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] 10.222.0.16 "GET /compute/v2.1/servers/23df6e0a-3637-4f59-93f9-b02dd614b5c3" status: 200 len: 1389 microversion: 2.1 time: 0.965343 Jul 16 21:16:02.078172 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 864/1810] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:16:01 2022] GET /compute/v2.1/servers/23df6e0a-3637-4f59-93f9-b02dd614b5c3 => generated 1389 bytes in 971 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:16:02.091200 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-1d686e7b-5b47-438d-8747-e88a1585fcba tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:02.092020 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-1d686e7b-5b47-438d-8747-e88a1585fcba tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] [instance: 69cabe57-da71-4a02-b4ee-c0d84b3d32a2] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:02.102586 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1d686e7b-5b47-438d-8747-e88a1585fcba tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:02.104195 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1d686e7b-5b47-438d-8747-e88a1585fcba tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:02.105289 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1d686e7b-5b47-438d-8747-e88a1585fcba tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:02.105917 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1d686e7b-5b47-438d-8747-e88a1585fcba tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:02.146124 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-1d686e7b-5b47-438d-8747-e88a1585fcba tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] HTTP exception thrown: Instance 69cabe57-da71-4a02-b4ee-c0d84b3d32a2 could not be found. Jul 16 21:16:02.151460 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-1d686e7b-5b47-438d-8747-e88a1585fcba tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Request method failure captured: Jul 16 21:16:02.151460 ncdv-master-1 devstack@n-api.service[138681]: request: GET /compute/v2.1/servers/69cabe57-da71-4a02-b4ee-c0d84b3d32a2 HTTP/1.1 Jul 16 21:16:02.151460 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:16:02.151460 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:16:02.151460 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:16:02.151460 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 21:16:02.151460 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:16:02.151460 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:16:02.151460 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:16:02.151460 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:16:02.151460 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0yoCD7IX15brFQUHVPKPIJrpEu20fxENHCEz31UnXrkVZbNHxjQ0ElqFOE7MXQnga3bWs8iQFWXWTDFYHn6RqK2SreYxppaBRq4w7eCzu-cpcuRq-bR3warXbUzkwx17fxxMPjZ200EVRXuxxzJd_xS8-ssDDdZBxRLNj6zLIJYowVk Jul 16 21:16:02.151460 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:16:02.151460 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:16:02.151460 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:16:02.151460 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:16:02.151460 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:16:02.151460 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:16:02.151460 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: bb1ede0d0b6941cc97278e6c617f9b01 Jul 16 21:16:02.151460 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-ServersTestMultiNic-262233723 Jul 16 21:16:02.151460 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:16:02.151460 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:16:02.151460 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/bb1ede0d0b6941cc97278e6c617f9b01"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/bb1ede0d0b6941cc97278e6c617f9b01"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_bb1ede0d0b6941cc97278e6c617f9b01"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/bb1ede0d0b6941cc97278e6c617f9b01"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:16:02.151460 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-ServersTestMultiNic-262233723 Jul 16 21:16:02.151460 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: bb1ede0d0b6941cc97278e6c617f9b01 Jul 16 21:16:02.151460 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-ServersTestMultiNic-262233723 Jul 16 21:16:02.151460 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-ServersTestMultiNic-262233723-project Jul 16 21:16:02.151460 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:16:02.151460 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:16:02.151460 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: 53af488f3b0242a797f539284b78d59e Jul 16 21:16:02.151460 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-ServersTestMultiNic-262233723-project Jul 16 21:16:02.151460 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:16:02.151460 ncdv-master-1 devstack@n-api.service[138681]: exception: Instance 69cabe57-da71-4a02-b4ee-c0d84b3d32a2 could not be found. Jul 16 21:16:02.151460 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': '69cabe57-da71-4a02-b4ee-c0d84b3d32a2'} Jul 16 21:16:02.151460 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:16:02.151460 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:16:02.151460 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:16:02.151460 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:16:02.151460 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:16:02.151460 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:16:02.151460 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:16:02.151460 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:16:02.151460 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:16:02.151460 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:16:02.151460 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 69cabe57-da71-4a02-b4ee-c0d84b3d32a2 could not be found. Jul 16 21:16:02.151460 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:16:02.151460 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:16:02.151460 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:16:02.151460 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:16:02.151460 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:16:02.151460 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:16:02.151460 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:16:02.151460 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:16:02.151460 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:16:02.151460 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:16:02.151460 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:16:02.151460 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:16:02.151460 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:16:02.151460 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:16:02.151460 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:16:02.151460 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:16:02.151460 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 69cabe57-da71-4a02-b4ee-c0d84b3d32a2 could not be found. Jul 16 21:16:02.151460 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:16:02.151460 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:16:02.151460 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:16:02.156920 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:16:02.156920 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:16:02.156920 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:16:02.156920 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:16:02.156920 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:16:02.156920 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 69cabe57-da71-4a02-b4ee-c0d84b3d32a2 could not be found. Jul 16 21:16:02.156920 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:16:02.160297 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-1d686e7b-5b47-438d-8747-e88a1585fcba tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] Returning 404 to user: Instance 69cabe57-da71-4a02-b4ee-c0d84b3d32a2 could not be found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:16:02.164860 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-1d686e7b-5b47-438d-8747-e88a1585fcba tempest-ServersTestMultiNic-262233723 tempest-ServersTestMultiNic-262233723-project] 10.222.0.16 "GET /compute/v2.1/servers/69cabe57-da71-4a02-b4ee-c0d84b3d32a2" status: 404 len: 111 microversion: 2.1 time: 0.080874 Jul 16 21:16:02.164860 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 865/1811] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:16:02 2022] GET /compute/v2.1/servers/69cabe57-da71-4a02-b4ee-c0d84b3d32a2 => generated 111 bytes in 83 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:16:02.173987 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-9f4af980-fc7c-42dc-9309-9ace753106d3 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:02.175827 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-9f4af980-fc7c-42dc-9309-9ace753106d3 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] [instance: 11194822-5d42-45f4-99d0-88c70c808ddd] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:02.185413 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9f4af980-fc7c-42dc-9309-9ace753106d3 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:02.186170 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9f4af980-fc7c-42dc-9309-9ace753106d3 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:02.187244 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9f4af980-fc7c-42dc-9309-9ace753106d3 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:02.187865 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9f4af980-fc7c-42dc-9309-9ace753106d3 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:02.993866 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell0:SELECT=2 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:16:03.144006 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-c3415179-24bd-4cf7-ab8a-422b3c171f56 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4e9df82f95ac4779859972403869096a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4337575622d04a26942905c9dc044ba7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:03.162680 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-c3415179-24bd-4cf7-ab8a-422b3c171f56 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] 10.222.0.16 "GET /compute/v2.1/servers/f6c350c4-b1b2-42d3-ba02-65cbbacf0f58" status: 200 len: 1325 microversion: 2.1 time: 1.948566 Jul 16 21:16:03.164381 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 947/1812] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:16:01 2022] GET /compute/v2.1/servers/f6c350c4-b1b2-42d3-ba02-65cbbacf0f58 => generated 1325 bytes in 1951 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:16:03.178455 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-9d3a6a70-ed50-4770-ac1d-e6580c051eeb req-4f16f111-4668-459b-a9af-90995197d69d service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "93e327c3-c63d-439a-9b25-1a805f785949", "name": "network-vif-plugged", "status": "completed", "tag": "4a074ea9-9de4-4dc1-aba4-632bf2d90076"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:16:03.188669 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-9d3a6a70-ed50-4770-ac1d-e6580c051eeb req-4f16f111-4668-459b-a9af-90995197d69d service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:03.190058 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-9d3a6a70-ed50-4770-ac1d-e6580c051eeb req-4f16f111-4668-459b-a9af-90995197d69d service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:03.210842 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-9d3a6a70-ed50-4770-ac1d-e6580c051eeb req-4f16f111-4668-459b-a9af-90995197d69d service nova] Creating event network-vif-plugged:4a074ea9-9de4-4dc1-aba4-632bf2d90076 for instance 93e327c3-c63d-439a-9b25-1a805f785949 on nch1-master-1 Jul 16 21:16:03.224108 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-9d3a6a70-ed50-4770-ac1d-e6580c051eeb req-4f16f111-4668-459b-a9af-90995197d69d service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.058167 Jul 16 21:16:03.227894 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 948/1813] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:16:03 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 63 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:16:03.259035 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-aaadc498-7272-4131-8bea-0683bc57b21e tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:03.261791 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-aaadc498-7272-4131-8bea-0683bc57b21e tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] [instance: 84ab7e65-1bf0-4b8b-baea-6d1d4ca13dd7] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:03.288974 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-aaadc498-7272-4131-8bea-0683bc57b21e tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:03.290952 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-aaadc498-7272-4131-8bea-0683bc57b21e tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:03.292010 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-aaadc498-7272-4131-8bea-0683bc57b21e tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:03.292638 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-aaadc498-7272-4131-8bea-0683bc57b21e tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:04.246693 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-9f4af980-fc7c-42dc-9309-9ace753106d3 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd4144b5388b46a989a41a2f556bb508', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a90d4bcd25ff4bf68bdc194180fbbe09', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:04.274736 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-9f4af980-fc7c-42dc-9309-9ace753106d3 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] 10.222.0.16 "GET /compute/v2.1/servers/11194822-5d42-45f4-99d0-88c70c808ddd" status: 200 len: 1317 microversion: 2.1 time: 2.106490 Jul 16 21:16:04.277835 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 866/1814] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:16:02 2022] GET /compute/v2.1/servers/11194822-5d42-45f4-99d0-88c70c808ddd => generated 1317 bytes in 2114 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:16:04.302462 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-45e9aae9-1454-41fc-a5c8-1f82c84fcf07 req-d853dbdd-9420-4a6b-a47a-28d9e5957eff service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "84ab7e65-1bf0-4b8b-baea-6d1d4ca13dd7", "tag": "42dfc89b-90fd-4214-983c-f02029faca17"}, {"name": "network-changed", "server_uuid": "b9f2c527-6346-4403-841e-b7bd876a5fb7", "tag": "edbdb017-2f7f-440d-9d46-aacbd858ccb6"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:16:04.336627 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-45e9aae9-1454-41fc-a5c8-1f82c84fcf07 req-d853dbdd-9420-4a6b-a47a-28d9e5957eff service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:04.339060 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-45e9aae9-1454-41fc-a5c8-1f82c84fcf07 req-d853dbdd-9420-4a6b-a47a-28d9e5957eff service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:04.370587 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-45e9aae9-1454-41fc-a5c8-1f82c84fcf07 req-d853dbdd-9420-4a6b-a47a-28d9e5957eff service nova] Creating event network-changed:42dfc89b-90fd-4214-983c-f02029faca17 for instance 84ab7e65-1bf0-4b8b-baea-6d1d4ca13dd7 on nch1-master-1 Jul 16 21:16:04.371631 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-45e9aae9-1454-41fc-a5c8-1f82c84fcf07 req-d853dbdd-9420-4a6b-a47a-28d9e5957eff service nova] Creating event network-changed:edbdb017-2f7f-440d-9d46-aacbd858ccb6 for instance b9f2c527-6346-4403-841e-b7bd876a5fb7 on nch2-master-1 Jul 16 21:16:04.384718 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-45e9aae9-1454-41fc-a5c8-1f82c84fcf07 req-d853dbdd-9420-4a6b-a47a-28d9e5957eff service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 346 microversion: 2.1 time: 0.104213 Jul 16 21:16:04.385337 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 867/1815] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:16:04 2022] POST /compute/v2.1/os-server-external-events => generated 346 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:16:04.394452 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-8068673f-2ff1-4614-99ff-1c9b7a01b621 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Calling method '.version_select at 0x7fdda2b9c9d0>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:04.451726 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-8068673f-2ff1-4614-99ff-1c9b7a01b621 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] 10.222.0.16 "GET /compute/v2.1/images/5af5e2eb-8145-46b4-abad-4d2a1e3630fc" status: 200 len: 1413 microversion: 2.1 time: 0.064302 Jul 16 21:16:04.459631 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 868/1816] 10.222.0.16 () {58 vars in 1232 bytes} [Sat Jul 16 21:16:04 2022] GET /compute/v2.1/images/5af5e2eb-8145-46b4-abad-4d2a1e3630fc => generated 1413 bytes in 73 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:16:04.468180 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-5a8e00f9-9ad8-4f12-80d7-a78600612a19 tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:04.470736 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-5a8e00f9-9ad8-4f12-80d7-a78600612a19 tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] [instance: b9f2c527-6346-4403-841e-b7bd876a5fb7] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:04.478038 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5a8e00f9-9ad8-4f12-80d7-a78600612a19 tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:04.478858 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5a8e00f9-9ad8-4f12-80d7-a78600612a19 tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:04.479874 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5a8e00f9-9ad8-4f12-80d7-a78600612a19 tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:04.480330 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5a8e00f9-9ad8-4f12-80d7-a78600612a19 tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:04.579919 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-aaadc498-7272-4131-8bea-0683bc57b21e tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:04.606525 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-aaadc498-7272-4131-8bea-0683bc57b21e tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:04.615826 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-aaadc498-7272-4131-8bea-0683bc57b21e tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:04.620433 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-aaadc498-7272-4131-8bea-0683bc57b21e tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] 10.222.0.16 "GET /compute/v2.1/servers/84ab7e65-1bf0-4b8b-baea-6d1d4ca13dd7" status: 200 len: 1487 microversion: 2.25 time: 1.387265 Jul 16 21:16:04.629805 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 949/1817] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:16:03 2022] GET /compute/v2.1/servers/84ab7e65-1bf0-4b8b-baea-6d1d4ca13dd7 => generated 1487 bytes in 1401 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:16:04.637348 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-879a8978-4dc9-4245-b9ee-7ea9604fc30d tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:04.638283 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-879a8978-4dc9-4245-b9ee-7ea9604fc30d tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] [instance: 23df6e0a-3637-4f59-93f9-b02dd614b5c3] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:04.650735 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-879a8978-4dc9-4245-b9ee-7ea9604fc30d tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:04.654925 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-879a8978-4dc9-4245-b9ee-7ea9604fc30d tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:04.654925 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-879a8978-4dc9-4245-b9ee-7ea9604fc30d tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:04.654925 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-879a8978-4dc9-4245-b9ee-7ea9604fc30d tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:04.747616 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell0:SELECT=8 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:16:05.563510 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-879a8978-4dc9-4245-b9ee-7ea9604fc30d tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a69f9dee03d43beaafa966216b9e289', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9b769bf8f3ea4e368bf68de1df6538f9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:05.571214 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-5a8e00f9-9ad8-4f12-80d7-a78600612a19 tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aa99be675e946fd995d71465b4cbec6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd82c2d92c67f469e863eb70bb7f9af84', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:05.578263 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-879a8978-4dc9-4245-b9ee-7ea9604fc30d tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] 10.222.0.16 "GET /compute/v2.1/servers/23df6e0a-3637-4f59-93f9-b02dd614b5c3" status: 200 len: 1379 microversion: 2.1 time: 0.944919 Jul 16 21:16:05.581292 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 950/1818] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:16:04 2022] GET /compute/v2.1/servers/23df6e0a-3637-4f59-93f9-b02dd614b5c3 => generated 1379 bytes in 951 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:16:05.586607 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-5a8e00f9-9ad8-4f12-80d7-a78600612a19 tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] 10.222.0.16 "GET /compute/v2.1/servers/b9f2c527-6346-4403-841e-b7bd876a5fb7" status: 200 len: 1293 microversion: 2.1 time: 1.121164 Jul 16 21:16:05.600414 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 869/1819] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:16:04 2022] GET /compute/v2.1/servers/b9f2c527-6346-4403-841e-b7bd876a5fb7 => generated 1293 bytes in 1129 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:16:05.600761 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-b612c629-6da0-42a0-afcb-c746ac9f6596 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:05.600871 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-f31cc312-2eaf-4207-9151-235b03f56199 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:05.600871 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-f31cc312-2eaf-4207-9151-235b03f56199 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] [instance: 11194822-5d42-45f4-99d0-88c70c808ddd] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:05.604705 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-b612c629-6da0-42a0-afcb-c746ac9f6596 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] [instance: f6c350c4-b1b2-42d3-ba02-65cbbacf0f58] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:05.624288 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f31cc312-2eaf-4207-9151-235b03f56199 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:05.624753 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f31cc312-2eaf-4207-9151-235b03f56199 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:05.625553 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f31cc312-2eaf-4207-9151-235b03f56199 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:05.635111 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f31cc312-2eaf-4207-9151-235b03f56199 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:05.653587 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b612c629-6da0-42a0-afcb-c746ac9f6596 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:05.654773 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b612c629-6da0-42a0-afcb-c746ac9f6596 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.009s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:05.657298 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b612c629-6da0-42a0-afcb-c746ac9f6596 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:05.661277 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b612c629-6da0-42a0-afcb-c746ac9f6596 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:06.243210 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-f31cc312-2eaf-4207-9151-235b03f56199 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd4144b5388b46a989a41a2f556bb508', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a90d4bcd25ff4bf68bdc194180fbbe09', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:06.259920 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-f31cc312-2eaf-4207-9151-235b03f56199 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] 10.222.0.16 "GET /compute/v2.1/servers/11194822-5d42-45f4-99d0-88c70c808ddd" status: 200 len: 1317 microversion: 2.1 time: 0.668437 Jul 16 21:16:06.261134 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 870/1820] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:16:05 2022] GET /compute/v2.1/servers/11194822-5d42-45f4-99d0-88c70c808ddd => generated 1317 bytes in 670 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:16:06.266429 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-f8ab91a3-a28b-400a-9980-c44d4c2e6ab3 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Calling method '.version_select at 0x7fdd9d0a0d30>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:06.361018 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-f8ab91a3-a28b-400a-9980-c44d4c2e6ab3 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] 10.222.0.16 "GET /compute/v2.1/images/5af5e2eb-8145-46b4-abad-4d2a1e3630fc" status: 200 len: 1413 microversion: 2.1 time: 0.095278 Jul 16 21:16:06.362004 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 871/1821] 10.222.0.16 () {58 vars in 1232 bytes} [Sat Jul 16 21:16:06 2022] GET /compute/v2.1/images/5af5e2eb-8145-46b4-abad-4d2a1e3630fc => generated 1413 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:16:06.367628 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-1e37a92d-e845-4f41-af3f-12c9da0128f1 tempest-LiveMigrationTest-1131758637 tempest-LiveMigrationTest-1131758637-project-admin] Calling method '.version_select at 0x7fdda2b384c0>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:06.371296 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1e37a92d-e845-4f41-af3f-12c9da0128f1 tempest-LiveMigrationTest-1131758637 tempest-LiveMigrationTest-1131758637-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:06.372027 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1e37a92d-e845-4f41-af3f-12c9da0128f1 tempest-LiveMigrationTest-1131758637 tempest-LiveMigrationTest-1131758637-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:06.391381 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-1e37a92d-e845-4f41-af3f-12c9da0128f1 tempest-LiveMigrationTest-1131758637 tempest-LiveMigrationTest-1131758637-project-admin] 10.222.0.16 "GET /compute/v2.1/os-migrations" status: 200 len: 3087 microversion: 2.1 time: 0.027884 Jul 16 21:16:06.394504 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 872/1822] 10.222.0.16 () {58 vars in 1142 bytes} [Sat Jul 16 21:16:06 2022] GET /compute/v2.1/os-migrations => generated 3087 bytes in 31 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:16:06.400357 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-4a1ec621-aafb-4065-a88f-a3b8ab7e1199 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:06.401400 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-4a1ec621-aafb-4065-a88f-a3b8ab7e1199 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] [instance: 84ab7e65-1bf0-4b8b-baea-6d1d4ca13dd7] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:06.410215 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4a1ec621-aafb-4065-a88f-a3b8ab7e1199 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:06.412437 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4a1ec621-aafb-4065-a88f-a3b8ab7e1199 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:06.412895 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4a1ec621-aafb-4065-a88f-a3b8ab7e1199 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:06.413513 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4a1ec621-aafb-4065-a88f-a3b8ab7e1199 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:07.155422 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-b612c629-6da0-42a0-afcb-c746ac9f6596 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4e9df82f95ac4779859972403869096a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4337575622d04a26942905c9dc044ba7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:07.164184 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-b612c629-6da0-42a0-afcb-c746ac9f6596 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] 10.222.0.16 "GET /compute/v2.1/servers/f6c350c4-b1b2-42d3-ba02-65cbbacf0f58" status: 200 len: 1325 microversion: 2.1 time: 1.577621 Jul 16 21:16:07.164848 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 951/1823] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:16:05 2022] GET /compute/v2.1/servers/f6c350c4-b1b2-42d3-ba02-65cbbacf0f58 => generated 1325 bytes in 1583 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:16:07.169461 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-82892ed4-30ec-463e-b4a2-8fcd6b4fb97f tempest-LiveMigrationTest-1131758637 tempest-LiveMigrationTest-1131758637-project-admin] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:07.170072 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-82892ed4-30ec-463e-b4a2-8fcd6b4fb97f tempest-LiveMigrationTest-1131758637 tempest-LiveMigrationTest-1131758637-project-admin] [instance: 23df6e0a-3637-4f59-93f9-b02dd614b5c3] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:07.177169 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-82892ed4-30ec-463e-b4a2-8fcd6b4fb97f tempest-LiveMigrationTest-1131758637 tempest-LiveMigrationTest-1131758637-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:07.177648 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-82892ed4-30ec-463e-b4a2-8fcd6b4fb97f tempest-LiveMigrationTest-1131758637 tempest-LiveMigrationTest-1131758637-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:07.178422 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-82892ed4-30ec-463e-b4a2-8fcd6b4fb97f tempest-LiveMigrationTest-1131758637 tempest-LiveMigrationTest-1131758637-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:07.178867 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-82892ed4-30ec-463e-b4a2-8fcd6b4fb97f tempest-LiveMigrationTest-1131758637 tempest-LiveMigrationTest-1131758637-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:07.631805 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-4a1ec621-aafb-4065-a88f-a3b8ab7e1199 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:07.650365 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-4a1ec621-aafb-4065-a88f-a3b8ab7e1199 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:07.651909 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-4a1ec621-aafb-4065-a88f-a3b8ab7e1199 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:07.653090 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-4a1ec621-aafb-4065-a88f-a3b8ab7e1199 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] 10.222.0.16 "GET /compute/v2.1/servers/84ab7e65-1bf0-4b8b-baea-6d1d4ca13dd7" status: 200 len: 1487 microversion: 2.25 time: 1.257257 Jul 16 21:16:07.667411 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 873/1824] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:16:06 2022] GET /compute/v2.1/servers/84ab7e65-1bf0-4b8b-baea-6d1d4ca13dd7 => generated 1487 bytes in 1272 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:16:07.672445 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-3b98fd38-b66a-4f57-88f9-1c3c3b5a65c1 req-7308e3a8-c4b4-462f-a265-699d83cb7ea3 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "f6c350c4-b1b2-42d3-ba02-65cbbacf0f58", "tag": "d0843d7d-ff0d-4603-aeeb-d26f6ebde7db"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:16:07.681794 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-3b98fd38-b66a-4f57-88f9-1c3c3b5a65c1 req-7308e3a8-c4b4-462f-a265-699d83cb7ea3 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:07.682277 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-3b98fd38-b66a-4f57-88f9-1c3c3b5a65c1 req-7308e3a8-c4b4-462f-a265-699d83cb7ea3 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:07.697754 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-3b98fd38-b66a-4f57-88f9-1c3c3b5a65c1 req-7308e3a8-c4b4-462f-a265-699d83cb7ea3 service nova] Creating event network-changed:d0843d7d-ff0d-4603-aeeb-d26f6ebde7db for instance f6c350c4-b1b2-42d3-ba02-65cbbacf0f58 on nch1-master-1 Jul 16 21:16:07.706028 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-3b98fd38-b66a-4f57-88f9-1c3c3b5a65c1 req-7308e3a8-c4b4-462f-a265-699d83cb7ea3 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.035995 Jul 16 21:16:07.707152 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 874/1825] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:16:07 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:16:07.713409 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-2d1dd936-29f5-4c7e-a055-92173cb3a268 tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:07.714428 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-2d1dd936-29f5-4c7e-a055-92173cb3a268 tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] [instance: b9f2c527-6346-4403-841e-b7bd876a5fb7] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:07.722061 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2d1dd936-29f5-4c7e-a055-92173cb3a268 tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:07.722822 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2d1dd936-29f5-4c7e-a055-92173cb3a268 tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:07.723842 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2d1dd936-29f5-4c7e-a055-92173cb3a268 tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:07.724458 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2d1dd936-29f5-4c7e-a055-92173cb3a268 tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:08.266672 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-82892ed4-30ec-463e-b4a2-8fcd6b4fb97f tempest-LiveMigrationTest-1131758637 tempest-LiveMigrationTest-1131758637-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/23df6e0a-3637-4f59-93f9-b02dd614b5c3" status: 200 len: 1530 microversion: 2.1 time: 1.100730 Jul 16 21:16:08.270024 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 952/1826] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:16:07 2022] GET /compute/v2.1/servers/23df6e0a-3637-4f59-93f9-b02dd614b5c3 => generated 1530 bytes in 1104 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:16:08.277052 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-d1d1f09a-214f-46ed-8a9d-83eadee01c55 req-ba4ee54b-aae4-4a79-86db-d6a948508c27 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "84ab7e65-1bf0-4b8b-baea-6d1d4ca13dd7", "name": "network-vif-unplugged", "status": "completed", "tag": "42dfc89b-90fd-4214-983c-f02029faca17"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:16:08.308224 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-d1d1f09a-214f-46ed-8a9d-83eadee01c55 req-ba4ee54b-aae4-4a79-86db-d6a948508c27 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:08.309350 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-d1d1f09a-214f-46ed-8a9d-83eadee01c55 req-ba4ee54b-aae4-4a79-86db-d6a948508c27 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:08.326599 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-d1d1f09a-214f-46ed-8a9d-83eadee01c55 req-ba4ee54b-aae4-4a79-86db-d6a948508c27 service nova] Creating event network-vif-unplugged:42dfc89b-90fd-4214-983c-f02029faca17 for instance 84ab7e65-1bf0-4b8b-baea-6d1d4ca13dd7 on nch1-master-1 Jul 16 21:16:08.332781 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-d1d1f09a-214f-46ed-8a9d-83eadee01c55 req-ba4ee54b-aae4-4a79-86db-d6a948508c27 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.061476 Jul 16 21:16:08.333402 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 953/1827] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:16:08 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 63 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:16:08.344428 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-f18426d5-cdbd-45b6-b713-fb98c65a9519 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:08.367631 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-f18426d5-cdbd-45b6-b713-fb98c65a9519 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] [instance: 11194822-5d42-45f4-99d0-88c70c808ddd] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:08.387226 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f18426d5-cdbd-45b6-b713-fb98c65a9519 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:08.390719 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f18426d5-cdbd-45b6-b713-fb98c65a9519 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:08.393234 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f18426d5-cdbd-45b6-b713-fb98c65a9519 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:08.394067 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f18426d5-cdbd-45b6-b713-fb98c65a9519 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:08.449143 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-2d1dd936-29f5-4c7e-a055-92173cb3a268 tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aa99be675e946fd995d71465b4cbec6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd82c2d92c67f469e863eb70bb7f9af84', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:08.457733 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-2d1dd936-29f5-4c7e-a055-92173cb3a268 tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] 10.222.0.16 "GET /compute/v2.1/servers/b9f2c527-6346-4403-841e-b7bd876a5fb7" status: 200 len: 1293 microversion: 2.1 time: 0.749135 Jul 16 21:16:08.471410 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 875/1828] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:16:07 2022] GET /compute/v2.1/servers/b9f2c527-6346-4403-841e-b7bd876a5fb7 => generated 1293 bytes in 759 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:16:08.474269 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-d70a0294-27c1-4e20-8f0e-7630edac3d39 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Calling method '.version_select at 0x7fdda2b50790>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:08.535977 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-d70a0294-27c1-4e20-8f0e-7630edac3d39 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] 10.222.0.16 "GET /compute/v2.1/images/5af5e2eb-8145-46b4-abad-4d2a1e3630fc" status: 200 len: 1413 microversion: 2.1 time: 0.067682 Jul 16 21:16:08.541886 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 876/1829] 10.222.0.16 () {58 vars in 1232 bytes} [Sat Jul 16 21:16:08 2022] GET /compute/v2.1/images/5af5e2eb-8145-46b4-abad-4d2a1e3630fc => generated 1413 bytes in 69 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:16:08.541886 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-7d8e5e36-b91f-430f-8dd9-ff7767c26d53 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:08.545274 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-7d8e5e36-b91f-430f-8dd9-ff7767c26d53 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] [instance: f6c350c4-b1b2-42d3-ba02-65cbbacf0f58] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:08.550876 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7d8e5e36-b91f-430f-8dd9-ff7767c26d53 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:08.551710 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7d8e5e36-b91f-430f-8dd9-ff7767c26d53 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:08.553432 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7d8e5e36-b91f-430f-8dd9-ff7767c26d53 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:08.554797 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7d8e5e36-b91f-430f-8dd9-ff7767c26d53 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:09.257535 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-f18426d5-cdbd-45b6-b713-fb98c65a9519 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd4144b5388b46a989a41a2f556bb508', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a90d4bcd25ff4bf68bdc194180fbbe09', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:09.266357 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-f18426d5-cdbd-45b6-b713-fb98c65a9519 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] 10.222.0.16 "GET /compute/v2.1/servers/11194822-5d42-45f4-99d0-88c70c808ddd" status: 200 len: 1317 microversion: 2.1 time: 0.926696 Jul 16 21:16:09.278166 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 954/1830] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:16:08 2022] GET /compute/v2.1/servers/11194822-5d42-45f4-99d0-88c70c808ddd => generated 1317 bytes in 944 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:16:09.285271 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-4cf1d522-e5ef-47d2-baf0-514cc1ae1142 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:09.286308 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-4cf1d522-e5ef-47d2-baf0-514cc1ae1142 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] [instance: 23df6e0a-3637-4f59-93f9-b02dd614b5c3] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:09.293557 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4cf1d522-e5ef-47d2-baf0-514cc1ae1142 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:09.295108 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4cf1d522-e5ef-47d2-baf0-514cc1ae1142 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:09.296088 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4cf1d522-e5ef-47d2-baf0-514cc1ae1142 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:09.296632 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4cf1d522-e5ef-47d2-baf0-514cc1ae1142 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:09.305684 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-7d8e5e36-b91f-430f-8dd9-ff7767c26d53 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4e9df82f95ac4779859972403869096a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4337575622d04a26942905c9dc044ba7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:09.318022 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-7d8e5e36-b91f-430f-8dd9-ff7767c26d53 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] 10.222.0.16 "GET /compute/v2.1/servers/f6c350c4-b1b2-42d3-ba02-65cbbacf0f58" status: 200 len: 1325 microversion: 2.1 time: 0.775835 Jul 16 21:16:09.318683 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 877/1831] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:16:08 2022] GET /compute/v2.1/servers/f6c350c4-b1b2-42d3-ba02-65cbbacf0f58 => generated 1325 bytes in 781 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:16:09.340412 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-38eb0ce7-0496-4e78-b81e-35462673f119 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:09.342809 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-4cf1d522-e5ef-47d2-baf0-514cc1ae1142 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] [instance: 23df6e0a-3637-4f59-93f9-b02dd614b5c3] Going to try to terminate instance {{(pid=138680) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 16 21:16:09.346926 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-38eb0ce7-0496-4e78-b81e-35462673f119 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] [instance: 84ab7e65-1bf0-4b8b-baea-6d1d4ca13dd7] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:09.359852 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-38eb0ce7-0496-4e78-b81e-35462673f119 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:09.360339 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-38eb0ce7-0496-4e78-b81e-35462673f119 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:09.366451 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-38eb0ce7-0496-4e78-b81e-35462673f119 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:09.366909 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-38eb0ce7-0496-4e78-b81e-35462673f119 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:09.788037 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-4cf1d522-e5ef-47d2-baf0-514cc1ae1142 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] 10.222.0.16 "DELETE /compute/v2.1/servers/23df6e0a-3637-4f59-93f9-b02dd614b5c3" status: 204 len: 0 microversion: 2.1 time: 0.507258 Jul 16 21:16:09.792696 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 955/1832] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:16:09 2022] DELETE /compute/v2.1/servers/23df6e0a-3637-4f59-93f9-b02dd614b5c3 => generated 0 bytes in 509 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 16 21:16:09.799321 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-33c4af97-a85f-44c4-ad16-70920c328411 tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:09.800353 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-33c4af97-a85f-44c4-ad16-70920c328411 tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] [instance: b9f2c527-6346-4403-841e-b7bd876a5fb7] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:09.810262 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_api:SELECT=96,nova_api:INSERT=3,nova_api:UPDATE=1 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:16:09.812309 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-33c4af97-a85f-44c4-ad16-70920c328411 tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:09.813285 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-33c4af97-a85f-44c4-ad16-70920c328411 tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:09.815245 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-33c4af97-a85f-44c4-ad16-70920c328411 tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:09.815922 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-33c4af97-a85f-44c4-ad16-70920c328411 tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:10.041659 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-38eb0ce7-0496-4e78-b81e-35462673f119 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:10.060285 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-38eb0ce7-0496-4e78-b81e-35462673f119 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:10.062381 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-38eb0ce7-0496-4e78-b81e-35462673f119 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:10.064014 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-38eb0ce7-0496-4e78-b81e-35462673f119 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] 10.222.0.16 "GET /compute/v2.1/servers/84ab7e65-1bf0-4b8b-baea-6d1d4ca13dd7" status: 200 len: 1487 microversion: 2.25 time: 0.741599 Jul 16 21:16:10.076947 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 878/1833] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:16:09 2022] GET /compute/v2.1/servers/84ab7e65-1bf0-4b8b-baea-6d1d4ca13dd7 => generated 1487 bytes in 755 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:16:10.083498 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-2cd8c66e-2558-4c66-b65f-60b99d44aa73 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Calling method '.version_select at 0x7fdd9cf2a790>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:10.145517 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-2cd8c66e-2558-4c66-b65f-60b99d44aa73 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] 10.222.0.16 "GET /compute/v2.1/images/5af5e2eb-8145-46b4-abad-4d2a1e3630fc" status: 200 len: 1413 microversion: 2.1 time: 0.066981 Jul 16 21:16:10.151885 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 879/1834] 10.222.0.16 () {58 vars in 1232 bytes} [Sat Jul 16 21:16:10 2022] GET /compute/v2.1/images/5af5e2eb-8145-46b4-abad-4d2a1e3630fc => generated 1413 bytes in 71 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:16:10.159365 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-80ac45b2-504c-4599-af11-d5c2a1218ad3 req-fe8b6080-3941-4cd6-a931-8b2c0edac610 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "11194822-5d42-45f4-99d0-88c70c808ddd", "tag": "1e6a5dec-046c-4c31-bb3f-1b8d00690085"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:16:10.171476 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-80ac45b2-504c-4599-af11-d5c2a1218ad3 req-fe8b6080-3941-4cd6-a931-8b2c0edac610 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:10.172248 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-80ac45b2-504c-4599-af11-d5c2a1218ad3 req-fe8b6080-3941-4cd6-a931-8b2c0edac610 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:10.188699 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-80ac45b2-504c-4599-af11-d5c2a1218ad3 req-fe8b6080-3941-4cd6-a931-8b2c0edac610 service nova] Creating event network-changed:1e6a5dec-046c-4c31-bb3f-1b8d00690085 for instance 11194822-5d42-45f4-99d0-88c70c808ddd on nch1-master-1 Jul 16 21:16:10.194058 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-80ac45b2-504c-4599-af11-d5c2a1218ad3 req-fe8b6080-3941-4cd6-a931-8b2c0edac610 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.041768 Jul 16 21:16:10.195141 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 880/1835] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:16:10 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:16:10.200908 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-9e7545f4-35a3-4d86-a794-1339665ae2ed tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:10.202298 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-9e7545f4-35a3-4d86-a794-1339665ae2ed tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] [instance: 23df6e0a-3637-4f59-93f9-b02dd614b5c3] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:10.215585 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9e7545f4-35a3-4d86-a794-1339665ae2ed tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:10.218315 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9e7545f4-35a3-4d86-a794-1339665ae2ed tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:10.218315 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9e7545f4-35a3-4d86-a794-1339665ae2ed tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:10.218315 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9e7545f4-35a3-4d86-a794-1339665ae2ed tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:10.461321 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-33c4af97-a85f-44c4-ad16-70920c328411 tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aa99be675e946fd995d71465b4cbec6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd82c2d92c67f469e863eb70bb7f9af84', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:10.484146 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-33c4af97-a85f-44c4-ad16-70920c328411 tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] 10.222.0.16 "GET /compute/v2.1/servers/b9f2c527-6346-4403-841e-b7bd876a5fb7" status: 200 len: 1293 microversion: 2.1 time: 0.689859 Jul 16 21:16:10.488334 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 956/1836] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:16:09 2022] GET /compute/v2.1/servers/b9f2c527-6346-4403-841e-b7bd876a5fb7 => generated 1293 bytes in 695 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:16:10.697462 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-8a6088a1-1c89-478f-9a94-cc90676a8c92 tempest-ImagesMetadataNegativeTestJSON-511989606 tempest-ImagesMetadataNegativeTestJSON-511989606-project] Calling method '.version_select at 0x7fdda2b7fc10>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:10.727555 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-8a6088a1-1c89-478f-9a94-cc90676a8c92 tempest-ImagesMetadataNegativeTestJSON-511989606 tempest-ImagesMetadataNegativeTestJSON-511989606-project] HTTP exception thrown: Image not found. Jul 16 21:16:10.741093 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-8a6088a1-1c89-478f-9a94-cc90676a8c92 tempest-ImagesMetadataNegativeTestJSON-511989606 tempest-ImagesMetadataNegativeTestJSON-511989606-project] Request method failure captured: Jul 16 21:16:10.741093 ncdv-master-1 devstack@n-api.service[138680]: request: DELETE /compute/v2.1/images/14cc265e-0147-4450-b166-783db3119c1e/metadata/os_distro HTTP/1.1 Jul 16 21:16:10.741093 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:16:10.741093 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:16:10.741093 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:16:10.741093 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 0 Jul 16 21:16:10.741093 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:16:10.741093 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:16:10.741093 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:16:10.741093 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:16:10.741093 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0yqZkT1PUPUnzET9ZAX-gVvwOnLYP0XB16rL3CiSft_4-2yik_0nM1TjnA8QuhU9TPl18FOAH6hNk2W83R2uXPXb9LBAVwu5sPf5tx8634HEmK9BrlhvdwDryqW3j2z1mBjVeiFYH4_OXVjNdPS3YVrbp1SvI0sDi3Yiu6r0NJNZJWo Jul 16 21:16:10.741093 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:16:10.741093 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:16:10.741093 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:16:10.741093 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:16:10.741093 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:16:10.741093 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:16:10.741093 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: 5163e91b263949d2b4ac6cf1e110eb62 Jul 16 21:16:10.741093 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-ImagesMetadataNegativeTestJSON-511989606 Jul 16 21:16:10.741093 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:16:10.741093 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:16:10.741093 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/5163e91b263949d2b4ac6cf1e110eb62"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/5163e91b263949d2b4ac6cf1e110eb62"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_5163e91b263949d2b4ac6cf1e110eb62"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/5163e91b263949d2b4ac6cf1e110eb62"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:16:10.741093 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-ImagesMetadataNegativeTestJSON-511989606 Jul 16 21:16:10.741093 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: 5163e91b263949d2b4ac6cf1e110eb62 Jul 16 21:16:10.741093 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-ImagesMetadataNegativeTestJSON-511989606 Jul 16 21:16:10.741093 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-ImagesMetadataNegativeTestJSON-511989606-project Jul 16 21:16:10.741093 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:16:10.741093 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:16:10.741093 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: b3b5f2d7a35b44649de7c41e5fdebaac Jul 16 21:16:10.741093 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-ImagesMetadataNegativeTestJSON-511989606-project Jul 16 21:16:10.741093 ncdv-master-1 devstack@n-api.service[138680]: method: Jul 16 21:16:10.741093 ncdv-master-1 devstack@n-api.service[138680]: exception: Image not found. Jul 16 21:16:10.741093 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'image_id': '14cc265e-0147-4450-b166-783db3119c1e', 'id': 'os_distro'} Jul 16 21:16:10.741093 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:16:10.741093 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:16:10.741093 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 285, in show Jul 16 21:16:10.741093 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi image = self._client.call(context, 2, 'get', args=(image_id,)) Jul 16 21:16:10.741093 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 191, in call Jul 16 21:16:10.741093 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi result = getattr(controller, method)(*args, **kwargs) Jul 16 21:16:10.741093 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/v2/images.py", line 197, in get Jul 16 21:16:10.741093 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return self._get(image_id) Jul 16 21:16:10.741093 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/utils.py", line 670, in inner Jul 16 21:16:10.741093 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return RequestIdProxy(wrapped(*args, **kwargs)) Jul 16 21:16:10.741093 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/v2/images.py", line 190, in _get Jul 16 21:16:10.741093 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi resp, body = self.http_client.get(url, headers=header) Jul 16 21:16:10.741093 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/adapter.py", line 395, in get Jul 16 21:16:10.741093 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return self.request(url, 'GET', **kwargs) Jul 16 21:16:10.741093 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 380, in request Jul 16 21:16:10.741093 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return self._handle_response(resp) Jul 16 21:16:10.741093 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 120, in _handle_response Jul 16 21:16:10.741093 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.from_response(resp, resp.content) Jul 16 21:16:10.741093 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi glanceclient.exc.HTTPNotFound: HTTP 404 Not Found: No image found with ID 14cc265e-0147-4450-b166-783db3119c1e Jul 16 21:16:10.741093 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:16:10.741093 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:16:10.741093 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:16:10.741093 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:16:10.741093 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/image_metadata.py", line 39, in _get_image Jul 16 21:16:10.741093 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return self.image_api.get(context, image_id) Jul 16 21:16:10.741093 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 1205, in get Jul 16 21:16:10.741093 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return session.show(context, image_id, Jul 16 21:16:10.741093 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 287, in show Jul 16 21:16:10.741093 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi _reraise_translated_image_exception(image_id) Jul 16 21:16:10.749432 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 1031, in _reraise_translated_image_exception Jul 16 21:16:10.749432 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise new_exc.with_traceback(exc_trace) Jul 16 21:16:10.749432 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 285, in show Jul 16 21:16:10.749432 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi image = self._client.call(context, 2, 'get', args=(image_id,)) Jul 16 21:16:10.749432 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 191, in call Jul 16 21:16:10.749432 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi result = getattr(controller, method)(*args, **kwargs) Jul 16 21:16:10.749432 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/v2/images.py", line 197, in get Jul 16 21:16:10.749432 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return self._get(image_id) Jul 16 21:16:10.749432 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/utils.py", line 670, in inner Jul 16 21:16:10.749432 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return RequestIdProxy(wrapped(*args, **kwargs)) Jul 16 21:16:10.749432 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/v2/images.py", line 190, in _get Jul 16 21:16:10.749432 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi resp, body = self.http_client.get(url, headers=header) Jul 16 21:16:10.749432 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/adapter.py", line 395, in get Jul 16 21:16:10.749432 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return self.request(url, 'GET', **kwargs) Jul 16 21:16:10.749432 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 380, in request Jul 16 21:16:10.749432 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return self._handle_response(resp) Jul 16 21:16:10.749432 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 120, in _handle_response Jul 16 21:16:10.749432 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.from_response(resp, resp.content) Jul 16 21:16:10.749432 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.ImageNotFound: Image 14cc265e-0147-4450-b166-783db3119c1e could not be found. Jul 16 21:16:10.749432 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:16:10.749432 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:16:10.749432 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:16:10.749432 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:16:10.749432 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:16:10.749432 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:16:10.749432 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:16:10.749432 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:16:10.749432 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Jul 16 21:16:10.749432 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Jul 16 21:16:10.749432 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:16:10.749432 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:16:10.749432 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/image_metadata.py", line 125, in delete Jul 16 21:16:10.749432 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi image = self._get_image(context, image_id) Jul 16 21:16:10.749432 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/image_metadata.py", line 44, in _get_image Jul 16 21:16:10.749432 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=msg) Jul 16 21:16:10.749432 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Image not found. Jul 16 21:16:10.749432 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:16:10.749432 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:16:10.749432 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:16:10.749432 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:16:10.749432 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:16:10.749432 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:16:10.749432 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:16:10.749432 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:16:10.749432 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Image not found. Jul 16 21:16:10.749432 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:16:10.749432 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-8a6088a1-1c89-478f-9a94-cc90676a8c92 tempest-ImagesMetadataNegativeTestJSON-511989606 tempest-ImagesMetadataNegativeTestJSON-511989606-project] Returning 404 to user: Image not found. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:16:10.757288 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-8a6088a1-1c89-478f-9a94-cc90676a8c92 tempest-ImagesMetadataNegativeTestJSON-511989606 tempest-ImagesMetadataNegativeTestJSON-511989606-project] 10.222.0.16 "DELETE /compute/v2.1/images/14cc265e-0147-4450-b166-783db3119c1e/metadata/os_distro" status: 404 len: 62 microversion: 2.1 time: 0.255993 Jul 16 21:16:10.757288 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 957/1837] 10.222.0.16 () {58 vars in 1292 bytes} [Sat Jul 16 21:16:10 2022] DELETE /compute/v2.1/images/14cc265e-0147-4450-b166-783db3119c1e/metadata/os_distro => generated 62 bytes in 268 msecs (HTTP/1.1 404) 9 headers in 378 bytes (1 switches on core 0) Jul 16 21:16:10.772772 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-7be58701-ca7e-401d-968d-44e0cf42a8d4 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:10.773692 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-7be58701-ca7e-401d-968d-44e0cf42a8d4 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] [instance: 11194822-5d42-45f4-99d0-88c70c808ddd] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:10.783179 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7be58701-ca7e-401d-968d-44e0cf42a8d4 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:10.783969 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7be58701-ca7e-401d-968d-44e0cf42a8d4 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:10.790490 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7be58701-ca7e-401d-968d-44e0cf42a8d4 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:10.791516 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7be58701-ca7e-401d-968d-44e0cf42a8d4 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:11.311406 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-9e7545f4-35a3-4d86-a794-1339665ae2ed tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a69f9dee03d43beaafa966216b9e289', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9b769bf8f3ea4e368bf68de1df6538f9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:11.314190 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-9e7545f4-35a3-4d86-a794-1339665ae2ed tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] 10.222.0.16 "GET /compute/v2.1/servers/23df6e0a-3637-4f59-93f9-b02dd614b5c3" status: 200 len: 1385 microversion: 2.1 time: 1.114812 Jul 16 21:16:11.314965 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 881/1838] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:16:10 2022] GET /compute/v2.1/servers/23df6e0a-3637-4f59-93f9-b02dd614b5c3 => generated 1385 bytes in 1119 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:16:11.322567 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-0473551b-b525-416f-99d8-6e7075862100 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:11.326200 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-0473551b-b525-416f-99d8-6e7075862100 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] [instance: f6c350c4-b1b2-42d3-ba02-65cbbacf0f58] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:11.331715 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0473551b-b525-416f-99d8-6e7075862100 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:11.332330 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0473551b-b525-416f-99d8-6e7075862100 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:11.333321 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0473551b-b525-416f-99d8-6e7075862100 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:11.334034 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0473551b-b525-416f-99d8-6e7075862100 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:12.008413 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-7be58701-ca7e-401d-968d-44e0cf42a8d4 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd4144b5388b46a989a41a2f556bb508', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a90d4bcd25ff4bf68bdc194180fbbe09', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:12.025658 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-7be58701-ca7e-401d-968d-44e0cf42a8d4 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] 10.222.0.16 "GET /compute/v2.1/servers/11194822-5d42-45f4-99d0-88c70c808ddd" status: 200 len: 1317 microversion: 2.1 time: 1.264301 Jul 16 21:16:12.027714 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 958/1839] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:16:10 2022] GET /compute/v2.1/servers/11194822-5d42-45f4-99d0-88c70c808ddd => generated 1317 bytes in 1270 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:16:12.057020 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-322fb87f-c849-4a7c-a5f0-2ae8c87d9bb4 tempest-ImagesMetadataNegativeTestJSON-511989606 tempest-ImagesMetadataNegativeTestJSON-511989606-project] Calling method '.version_select at 0x7fdda2bc0e50>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:12.118582 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-322fb87f-c849-4a7c-a5f0-2ae8c87d9bb4 tempest-ImagesMetadataNegativeTestJSON-511989606 tempest-ImagesMetadataNegativeTestJSON-511989606-project] HTTP exception thrown: Image not found. Jul 16 21:16:12.118582 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-322fb87f-c849-4a7c-a5f0-2ae8c87d9bb4 tempest-ImagesMetadataNegativeTestJSON-511989606 tempest-ImagesMetadataNegativeTestJSON-511989606-project] Request method failure captured: Jul 16 21:16:12.118582 ncdv-master-1 devstack@n-api.service[138680]: request: GET /compute/v2.1/images/f6f961c2-9858-4ac2-bff5-39f98a9800f8/metadata/os_version HTTP/1.1 Jul 16 21:16:12.118582 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:16:12.118582 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:16:12.118582 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:16:12.118582 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 0 Jul 16 21:16:12.118582 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:16:12.118582 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:16:12.118582 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:16:12.118582 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:16:12.118582 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0yqZkT1PUPUnzET9ZAX-gVvwOnLYP0XB16rL3CiSft_4-2yik_0nM1TjnA8QuhU9TPl18FOAH6hNk2W83R2uXPXb9LBAVwu5sPf5tx8634HEmK9BrlhvdwDryqW3j2z1mBjVeiFYH4_OXVjNdPS3YVrbp1SvI0sDi3Yiu6r0NJNZJWo Jul 16 21:16:12.118582 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:16:12.118582 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:16:12.118582 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:16:12.118582 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:16:12.118582 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:16:12.118582 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:16:12.118582 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: 5163e91b263949d2b4ac6cf1e110eb62 Jul 16 21:16:12.118582 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-ImagesMetadataNegativeTestJSON-511989606 Jul 16 21:16:12.118582 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:16:12.118582 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:16:12.118582 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/5163e91b263949d2b4ac6cf1e110eb62"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/5163e91b263949d2b4ac6cf1e110eb62"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_5163e91b263949d2b4ac6cf1e110eb62"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/5163e91b263949d2b4ac6cf1e110eb62"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:16:12.118582 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-ImagesMetadataNegativeTestJSON-511989606 Jul 16 21:16:12.118582 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: 5163e91b263949d2b4ac6cf1e110eb62 Jul 16 21:16:12.118582 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-ImagesMetadataNegativeTestJSON-511989606 Jul 16 21:16:12.118582 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-ImagesMetadataNegativeTestJSON-511989606-project Jul 16 21:16:12.118582 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:16:12.118582 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:16:12.118582 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: b3b5f2d7a35b44649de7c41e5fdebaac Jul 16 21:16:12.118582 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-ImagesMetadataNegativeTestJSON-511989606-project Jul 16 21:16:12.118582 ncdv-master-1 devstack@n-api.service[138680]: method: Jul 16 21:16:12.118582 ncdv-master-1 devstack@n-api.service[138680]: exception: Image not found. Jul 16 21:16:12.118582 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'image_id': 'f6f961c2-9858-4ac2-bff5-39f98a9800f8', 'id': 'os_version'} Jul 16 21:16:12.118582 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:16:12.118582 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:16:12.118582 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 285, in show Jul 16 21:16:12.118582 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi image = self._client.call(context, 2, 'get', args=(image_id,)) Jul 16 21:16:12.118582 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 191, in call Jul 16 21:16:12.118582 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi result = getattr(controller, method)(*args, **kwargs) Jul 16 21:16:12.118582 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/v2/images.py", line 197, in get Jul 16 21:16:12.118582 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return self._get(image_id) Jul 16 21:16:12.118582 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/utils.py", line 670, in inner Jul 16 21:16:12.118582 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return RequestIdProxy(wrapped(*args, **kwargs)) Jul 16 21:16:12.118582 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/v2/images.py", line 190, in _get Jul 16 21:16:12.118582 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi resp, body = self.http_client.get(url, headers=header) Jul 16 21:16:12.118582 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/adapter.py", line 395, in get Jul 16 21:16:12.118582 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return self.request(url, 'GET', **kwargs) Jul 16 21:16:12.118582 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 380, in request Jul 16 21:16:12.118582 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return self._handle_response(resp) Jul 16 21:16:12.118582 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 120, in _handle_response Jul 16 21:16:12.118582 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.from_response(resp, resp.content) Jul 16 21:16:12.118582 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi glanceclient.exc.HTTPNotFound: HTTP 404 Not Found: No image found with ID f6f961c2-9858-4ac2-bff5-39f98a9800f8 Jul 16 21:16:12.118582 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:16:12.118582 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:16:12.118582 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:16:12.118582 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:16:12.118582 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/image_metadata.py", line 39, in _get_image Jul 16 21:16:12.118582 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return self.image_api.get(context, image_id) Jul 16 21:16:12.118582 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 1205, in get Jul 16 21:16:12.118582 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return session.show(context, image_id, Jul 16 21:16:12.123905 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 287, in show Jul 16 21:16:12.123905 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi _reraise_translated_image_exception(image_id) Jul 16 21:16:12.123905 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 1031, in _reraise_translated_image_exception Jul 16 21:16:12.123905 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise new_exc.with_traceback(exc_trace) Jul 16 21:16:12.123905 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 285, in show Jul 16 21:16:12.123905 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi image = self._client.call(context, 2, 'get', args=(image_id,)) Jul 16 21:16:12.123905 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 191, in call Jul 16 21:16:12.123905 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi result = getattr(controller, method)(*args, **kwargs) Jul 16 21:16:12.123905 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/v2/images.py", line 197, in get Jul 16 21:16:12.123905 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return self._get(image_id) Jul 16 21:16:12.123905 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/utils.py", line 670, in inner Jul 16 21:16:12.123905 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return RequestIdProxy(wrapped(*args, **kwargs)) Jul 16 21:16:12.123905 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/v2/images.py", line 190, in _get Jul 16 21:16:12.123905 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi resp, body = self.http_client.get(url, headers=header) Jul 16 21:16:12.123905 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/adapter.py", line 395, in get Jul 16 21:16:12.123905 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return self.request(url, 'GET', **kwargs) Jul 16 21:16:12.123905 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 380, in request Jul 16 21:16:12.123905 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return self._handle_response(resp) Jul 16 21:16:12.123905 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 120, in _handle_response Jul 16 21:16:12.123905 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.from_response(resp, resp.content) Jul 16 21:16:12.123905 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.ImageNotFound: Image f6f961c2-9858-4ac2-bff5-39f98a9800f8 could not be found. Jul 16 21:16:12.123905 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:16:12.123905 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:16:12.123905 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:16:12.123905 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:16:12.123905 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:16:12.123905 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:16:12.123905 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:16:12.123905 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:16:12.123905 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Jul 16 21:16:12.123905 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Jul 16 21:16:12.123905 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:16:12.123905 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:16:12.123905 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/image_metadata.py", line 58, in show Jul 16 21:16:12.123905 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi metadata = self._get_image(context, image_id)['properties'] Jul 16 21:16:12.123905 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/image_metadata.py", line 44, in _get_image Jul 16 21:16:12.123905 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=msg) Jul 16 21:16:12.123905 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Image not found. Jul 16 21:16:12.123905 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:16:12.123905 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:16:12.123905 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:16:12.123905 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:16:12.123905 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:16:12.123905 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:16:12.123905 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:16:12.123905 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:16:12.123905 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Image not found. Jul 16 21:16:12.123905 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:16:12.123905 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-322fb87f-c849-4a7c-a5f0-2ae8c87d9bb4 tempest-ImagesMetadataNegativeTestJSON-511989606 tempest-ImagesMetadataNegativeTestJSON-511989606-project] Returning 404 to user: Image not found. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:16:12.123905 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-322fb87f-c849-4a7c-a5f0-2ae8c87d9bb4 tempest-ImagesMetadataNegativeTestJSON-511989606 tempest-ImagesMetadataNegativeTestJSON-511989606-project] 10.222.0.16 "GET /compute/v2.1/images/f6f961c2-9858-4ac2-bff5-39f98a9800f8/metadata/os_version" status: 404 len: 62 microversion: 2.1 time: 0.086605 Jul 16 21:16:12.123905 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 959/1840] 10.222.0.16 () {58 vars in 1292 bytes} [Sat Jul 16 21:16:12 2022] GET /compute/v2.1/images/f6f961c2-9858-4ac2-bff5-39f98a9800f8/metadata/os_version => generated 62 bytes in 89 msecs (HTTP/1.1 404) 9 headers in 378 bytes (1 switches on core 0) Jul 16 21:16:12.136930 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-a8e86198-94f9-4735-aa7b-f0fc47260727 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:12.145937 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-a8e86198-94f9-4735-aa7b-f0fc47260727 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] [instance: 84ab7e65-1bf0-4b8b-baea-6d1d4ca13dd7] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:12.151309 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-0473551b-b525-416f-99d8-6e7075862100 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4e9df82f95ac4779859972403869096a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4337575622d04a26942905c9dc044ba7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:12.166932 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-0473551b-b525-416f-99d8-6e7075862100 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] 10.222.0.16 "GET /compute/v2.1/servers/f6c350c4-b1b2-42d3-ba02-65cbbacf0f58" status: 200 len: 1325 microversion: 2.1 time: 0.843980 Jul 16 21:16:12.171357 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 882/1841] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:16:11 2022] GET /compute/v2.1/servers/f6c350c4-b1b2-42d3-ba02-65cbbacf0f58 => generated 1325 bytes in 856 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:16:12.189557 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a8e86198-94f9-4735-aa7b-f0fc47260727 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:12.190194 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a8e86198-94f9-4735-aa7b-f0fc47260727 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:12.192119 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a8e86198-94f9-4735-aa7b-f0fc47260727 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:12.194388 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a8e86198-94f9-4735-aa7b-f0fc47260727 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:12.200691 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-50c4c9ae-ed69-4867-8545-773d71b731e9 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Calling method '.version_select at 0x7fdda2b38430>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:12.341434 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-50c4c9ae-ed69-4867-8545-773d71b731e9 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] 10.222.0.16 "GET /compute/v2.1/images/5af5e2eb-8145-46b4-abad-4d2a1e3630fc" status: 200 len: 1413 microversion: 2.1 time: 0.160675 Jul 16 21:16:12.343388 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 883/1842] 10.222.0.16 () {58 vars in 1232 bytes} [Sat Jul 16 21:16:12 2022] GET /compute/v2.1/images/5af5e2eb-8145-46b4-abad-4d2a1e3630fc => generated 1413 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:16:12.355884 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-69573f98-2778-4474-b165-debffb10ce99 tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:12.356511 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-69573f98-2778-4474-b165-debffb10ce99 tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] [instance: b9f2c527-6346-4403-841e-b7bd876a5fb7] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:12.378899 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-69573f98-2778-4474-b165-debffb10ce99 tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:12.381861 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-69573f98-2778-4474-b165-debffb10ce99 tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:12.384180 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-69573f98-2778-4474-b165-debffb10ce99 tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:12.386483 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-69573f98-2778-4474-b165-debffb10ce99 tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:12.913525 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-a8e86198-94f9-4735-aa7b-f0fc47260727 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:12.931147 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-69573f98-2778-4474-b165-debffb10ce99 tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aa99be675e946fd995d71465b4cbec6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd82c2d92c67f469e863eb70bb7f9af84', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:12.935491 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-a8e86198-94f9-4735-aa7b-f0fc47260727 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:12.940319 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-a8e86198-94f9-4735-aa7b-f0fc47260727 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:12.947150 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-a8e86198-94f9-4735-aa7b-f0fc47260727 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] 10.222.0.16 "GET /compute/v2.1/servers/84ab7e65-1bf0-4b8b-baea-6d1d4ca13dd7" status: 200 len: 1487 microversion: 2.25 time: 0.815617 Jul 16 21:16:12.947357 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-69573f98-2778-4474-b165-debffb10ce99 tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] 10.222.0.16 "GET /compute/v2.1/servers/b9f2c527-6346-4403-841e-b7bd876a5fb7" status: 200 len: 1293 microversion: 2.1 time: 0.602349 Jul 16 21:16:12.947808 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 884/1843] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:16:12 2022] GET /compute/v2.1/servers/b9f2c527-6346-4403-841e-b7bd876a5fb7 => generated 1293 bytes in 604 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:16:12.961140 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-8e316772-feaf-4fb2-9765-f9684ff5ee8b req-4f944208-8844-44b9-8358-f0713d96d1aa service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "23df6e0a-3637-4f59-93f9-b02dd614b5c3", "name": "network-vif-deleted", "tag": "60e499ad-afcb-4fe7-80ef-058f63c4fc21"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:16:12.963458 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 960/1844] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:16:12 2022] GET /compute/v2.1/servers/84ab7e65-1bf0-4b8b-baea-6d1d4ca13dd7 => generated 1487 bytes in 844 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:16:12.969147 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-c933ce6f-e274-4361-8c54-e16fbe0177ac tempest-ImagesMetadataNegativeTestJSON-511989606 tempest-ImagesMetadataNegativeTestJSON-511989606-project] Calling method '.version_select at 0x7fdda2b3f040>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:13.000240 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-8e316772-feaf-4fb2-9765-f9684ff5ee8b req-4f944208-8844-44b9-8358-f0713d96d1aa service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:13.000694 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-8e316772-feaf-4fb2-9765-f9684ff5ee8b req-4f944208-8844-44b9-8358-f0713d96d1aa service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:13.005605 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-c933ce6f-e274-4361-8c54-e16fbe0177ac tempest-ImagesMetadataNegativeTestJSON-511989606 tempest-ImagesMetadataNegativeTestJSON-511989606-project] HTTP exception thrown: Image not found. Jul 16 21:16:13.012132 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-c933ce6f-e274-4361-8c54-e16fbe0177ac tempest-ImagesMetadataNegativeTestJSON-511989606 tempest-ImagesMetadataNegativeTestJSON-511989606-project] Request method failure captured: Jul 16 21:16:13.012132 ncdv-master-1 devstack@n-api.service[138680]: request: GET /compute/v2.1/images/00658ce3-3a60-4009-80eb-6f33f75fabde/metadata HTTP/1.1 Jul 16 21:16:13.012132 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:16:13.012132 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:16:13.012132 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:16:13.012132 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 0 Jul 16 21:16:13.012132 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:16:13.012132 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:16:13.012132 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:16:13.012132 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:16:13.012132 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0yqZkT1PUPUnzET9ZAX-gVvwOnLYP0XB16rL3CiSft_4-2yik_0nM1TjnA8QuhU9TPl18FOAH6hNk2W83R2uXPXb9LBAVwu5sPf5tx8634HEmK9BrlhvdwDryqW3j2z1mBjVeiFYH4_OXVjNdPS3YVrbp1SvI0sDi3Yiu6r0NJNZJWo Jul 16 21:16:13.012132 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:16:13.012132 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:16:13.012132 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:16:13.012132 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:16:13.012132 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:16:13.012132 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:16:13.012132 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: 5163e91b263949d2b4ac6cf1e110eb62 Jul 16 21:16:13.012132 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-ImagesMetadataNegativeTestJSON-511989606 Jul 16 21:16:13.012132 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:16:13.012132 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:16:13.012132 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/5163e91b263949d2b4ac6cf1e110eb62"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/5163e91b263949d2b4ac6cf1e110eb62"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_5163e91b263949d2b4ac6cf1e110eb62"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/5163e91b263949d2b4ac6cf1e110eb62"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:16:13.012132 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-ImagesMetadataNegativeTestJSON-511989606 Jul 16 21:16:13.012132 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: 5163e91b263949d2b4ac6cf1e110eb62 Jul 16 21:16:13.012132 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-ImagesMetadataNegativeTestJSON-511989606 Jul 16 21:16:13.012132 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-ImagesMetadataNegativeTestJSON-511989606-project Jul 16 21:16:13.012132 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:16:13.012132 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:16:13.012132 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: b3b5f2d7a35b44649de7c41e5fdebaac Jul 16 21:16:13.012132 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-ImagesMetadataNegativeTestJSON-511989606-project Jul 16 21:16:13.012132 ncdv-master-1 devstack@n-api.service[138680]: method: Jul 16 21:16:13.012132 ncdv-master-1 devstack@n-api.service[138680]: exception: Image not found. Jul 16 21:16:13.012132 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'image_id': '00658ce3-3a60-4009-80eb-6f33f75fabde'} Jul 16 21:16:13.012132 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:16:13.012132 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:16:13.012132 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 285, in show Jul 16 21:16:13.012132 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi image = self._client.call(context, 2, 'get', args=(image_id,)) Jul 16 21:16:13.012132 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 191, in call Jul 16 21:16:13.012132 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi result = getattr(controller, method)(*args, **kwargs) Jul 16 21:16:13.012132 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/v2/images.py", line 197, in get Jul 16 21:16:13.012132 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return self._get(image_id) Jul 16 21:16:13.012132 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/utils.py", line 670, in inner Jul 16 21:16:13.012132 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return RequestIdProxy(wrapped(*args, **kwargs)) Jul 16 21:16:13.012132 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/v2/images.py", line 190, in _get Jul 16 21:16:13.012132 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi resp, body = self.http_client.get(url, headers=header) Jul 16 21:16:13.012132 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/adapter.py", line 395, in get Jul 16 21:16:13.012132 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return self.request(url, 'GET', **kwargs) Jul 16 21:16:13.012132 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 380, in request Jul 16 21:16:13.012132 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return self._handle_response(resp) Jul 16 21:16:13.012132 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 120, in _handle_response Jul 16 21:16:13.012132 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.from_response(resp, resp.content) Jul 16 21:16:13.012132 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi glanceclient.exc.HTTPNotFound: HTTP 404 Not Found: No image found with ID 00658ce3-3a60-4009-80eb-6f33f75fabde Jul 16 21:16:13.012132 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:16:13.012132 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:16:13.012132 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:16:13.012132 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:16:13.012132 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/image_metadata.py", line 39, in _get_image Jul 16 21:16:13.012132 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return self.image_api.get(context, image_id) Jul 16 21:16:13.012132 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 1205, in get Jul 16 21:16:13.012132 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return session.show(context, image_id, Jul 16 21:16:13.012132 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 287, in show Jul 16 21:16:13.012132 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi _reraise_translated_image_exception(image_id) Jul 16 21:16:13.012132 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 1031, in _reraise_translated_image_exception Jul 16 21:16:13.024037 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise new_exc.with_traceback(exc_trace) Jul 16 21:16:13.024037 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 285, in show Jul 16 21:16:13.024037 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi image = self._client.call(context, 2, 'get', args=(image_id,)) Jul 16 21:16:13.024037 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 191, in call Jul 16 21:16:13.024037 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi result = getattr(controller, method)(*args, **kwargs) Jul 16 21:16:13.024037 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/v2/images.py", line 197, in get Jul 16 21:16:13.024037 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return self._get(image_id) Jul 16 21:16:13.024037 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/utils.py", line 670, in inner Jul 16 21:16:13.024037 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return RequestIdProxy(wrapped(*args, **kwargs)) Jul 16 21:16:13.024037 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/v2/images.py", line 190, in _get Jul 16 21:16:13.024037 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi resp, body = self.http_client.get(url, headers=header) Jul 16 21:16:13.024037 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/adapter.py", line 395, in get Jul 16 21:16:13.024037 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return self.request(url, 'GET', **kwargs) Jul 16 21:16:13.024037 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 380, in request Jul 16 21:16:13.024037 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return self._handle_response(resp) Jul 16 21:16:13.024037 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 120, in _handle_response Jul 16 21:16:13.024037 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.from_response(resp, resp.content) Jul 16 21:16:13.024037 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.ImageNotFound: Image 00658ce3-3a60-4009-80eb-6f33f75fabde could not be found. Jul 16 21:16:13.024037 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:16:13.024037 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:16:13.024037 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:16:13.024037 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:16:13.024037 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:16:13.024037 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:16:13.024037 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:16:13.024037 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:16:13.024037 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Jul 16 21:16:13.024037 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Jul 16 21:16:13.024037 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:16:13.024037 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:16:13.024037 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/image_metadata.py", line 51, in index Jul 16 21:16:13.024037 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi metadata = self._get_image(context, image_id)['properties'] Jul 16 21:16:13.024037 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/image_metadata.py", line 44, in _get_image Jul 16 21:16:13.024037 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=msg) Jul 16 21:16:13.024037 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Image not found. Jul 16 21:16:13.024037 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:16:13.024037 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:16:13.024037 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:16:13.024037 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:16:13.024037 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:16:13.024037 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:16:13.024037 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:16:13.024037 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:16:13.024037 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Image not found. Jul 16 21:16:13.024037 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:16:13.024037 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-c933ce6f-e274-4361-8c54-e16fbe0177ac tempest-ImagesMetadataNegativeTestJSON-511989606 tempest-ImagesMetadataNegativeTestJSON-511989606-project] Returning 404 to user: Image not found. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:16:13.024037 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-c933ce6f-e274-4361-8c54-e16fbe0177ac tempest-ImagesMetadataNegativeTestJSON-511989606 tempest-ImagesMetadataNegativeTestJSON-511989606-project] 10.222.0.16 "GET /compute/v2.1/images/00658ce3-3a60-4009-80eb-6f33f75fabde/metadata" status: 404 len: 62 microversion: 2.1 time: 0.055537 Jul 16 21:16:13.024037 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 961/1845] 10.222.0.16 () {58 vars in 1259 bytes} [Sat Jul 16 21:16:12 2022] GET /compute/v2.1/images/00658ce3-3a60-4009-80eb-6f33f75fabde/metadata => generated 62 bytes in 57 msecs (HTTP/1.1 404) 9 headers in 378 bytes (1 switches on core 0) Jul 16 21:16:13.036515 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-8e316772-feaf-4fb2-9765-f9684ff5ee8b req-4f944208-8844-44b9-8358-f0713d96d1aa service nova] Creating event network-vif-deleted:60e499ad-afcb-4fe7-80ef-058f63c4fc21 for instance 23df6e0a-3637-4f59-93f9-b02dd614b5c3 on nch1-master-1 Jul 16 21:16:13.043886 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-bc6db9fb-3cff-4a24-9a4a-8b9607003e32 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:13.044914 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-bc6db9fb-3cff-4a24-9a4a-8b9607003e32 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] [instance: 23df6e0a-3637-4f59-93f9-b02dd614b5c3] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:13.050167 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-8e316772-feaf-4fb2-9765-f9684ff5ee8b req-4f944208-8844-44b9-8358-f0713d96d1aa service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.101050 Jul 16 21:16:13.055475 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-bc6db9fb-3cff-4a24-9a4a-8b9607003e32 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:13.055475 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-bc6db9fb-3cff-4a24-9a4a-8b9607003e32 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:13.055986 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 885/1846] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:16:12 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:16:13.056448 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-bc6db9fb-3cff-4a24-9a4a-8b9607003e32 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:13.057207 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-bc6db9fb-3cff-4a24-9a4a-8b9607003e32 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:13.068057 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-d360129d-8fe4-4308-a827-234d5ce1f7ef tempest-ImagesMetadataNegativeTestJSON-511989606 tempest-ImagesMetadataNegativeTestJSON-511989606-project] Action: 'update_all', calling method: .version_select at 0x7fdd9d0a0ca0>, body: {"metadata": {"os_distro": "alt1", "os_version": "alt2"}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:16:13.082108 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:16:13.108090 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-d360129d-8fe4-4308-a827-234d5ce1f7ef tempest-ImagesMetadataNegativeTestJSON-511989606 tempest-ImagesMetadataNegativeTestJSON-511989606-project] HTTP exception thrown: Image not found. Jul 16 21:16:13.111701 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-d360129d-8fe4-4308-a827-234d5ce1f7ef tempest-ImagesMetadataNegativeTestJSON-511989606 tempest-ImagesMetadataNegativeTestJSON-511989606-project] Request method failure captured: Jul 16 21:16:13.111701 ncdv-master-1 devstack@n-api.service[138681]: request: PUT /compute/v2.1/images/22eff938-ec35-4b0e-bb04-069cb6537f2f/metadata HTTP/1.1 Jul 16 21:16:13.111701 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:16:13.111701 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:16:13.111701 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:16:13.111701 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 57 Jul 16 21:16:13.111701 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:16:13.111701 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:16:13.111701 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:16:13.111701 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:16:13.111701 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0yqZkT1PUPUnzET9ZAX-gVvwOnLYP0XB16rL3CiSft_4-2yik_0nM1TjnA8QuhU9TPl18FOAH6hNk2W83R2uXPXb9LBAVwu5sPf5tx8634HEmK9BrlhvdwDryqW3j2z1mBjVeiFYH4_OXVjNdPS3YVrbp1SvI0sDi3Yiu6r0NJNZJWo Jul 16 21:16:13.111701 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:16:13.111701 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:16:13.111701 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:16:13.111701 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:16:13.111701 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:16:13.111701 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:16:13.111701 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: 5163e91b263949d2b4ac6cf1e110eb62 Jul 16 21:16:13.111701 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-ImagesMetadataNegativeTestJSON-511989606 Jul 16 21:16:13.111701 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:16:13.111701 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:16:13.111701 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/5163e91b263949d2b4ac6cf1e110eb62"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/5163e91b263949d2b4ac6cf1e110eb62"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_5163e91b263949d2b4ac6cf1e110eb62"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/5163e91b263949d2b4ac6cf1e110eb62"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:16:13.111701 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-ImagesMetadataNegativeTestJSON-511989606 Jul 16 21:16:13.111701 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: 5163e91b263949d2b4ac6cf1e110eb62 Jul 16 21:16:13.111701 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-ImagesMetadataNegativeTestJSON-511989606 Jul 16 21:16:13.111701 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-ImagesMetadataNegativeTestJSON-511989606-project Jul 16 21:16:13.111701 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:16:13.111701 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:16:13.111701 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: b3b5f2d7a35b44649de7c41e5fdebaac Jul 16 21:16:13.111701 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-ImagesMetadataNegativeTestJSON-511989606-project Jul 16 21:16:13.111701 ncdv-master-1 devstack@n-api.service[138681]: Jul 16 21:16:13.111701 ncdv-master-1 devstack@n-api.service[138681]: {"metadata": {"os_distro": "alt1", "os_version": "alt2"}} Jul 16 21:16:13.111701 ncdv-master-1 devstack@n-api.service[138681]: method: Jul 16 21:16:13.111701 ncdv-master-1 devstack@n-api.service[138681]: exception: Image not found. Jul 16 21:16:13.111701 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'image_id': '22eff938-ec35-4b0e-bb04-069cb6537f2f', 'body': {'metadata': {'os_distro': 'alt1', 'os_version': 'alt2'}}} Jul 16 21:16:13.111701 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:16:13.111701 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:16:13.111701 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 285, in show Jul 16 21:16:13.111701 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi image = self._client.call(context, 2, 'get', args=(image_id,)) Jul 16 21:16:13.111701 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 191, in call Jul 16 21:16:13.111701 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi result = getattr(controller, method)(*args, **kwargs) Jul 16 21:16:13.111701 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/v2/images.py", line 197, in get Jul 16 21:16:13.111701 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return self._get(image_id) Jul 16 21:16:13.111701 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/utils.py", line 670, in inner Jul 16 21:16:13.111701 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return RequestIdProxy(wrapped(*args, **kwargs)) Jul 16 21:16:13.111701 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/v2/images.py", line 190, in _get Jul 16 21:16:13.111701 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi resp, body = self.http_client.get(url, headers=header) Jul 16 21:16:13.111701 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/adapter.py", line 395, in get Jul 16 21:16:13.111701 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return self.request(url, 'GET', **kwargs) Jul 16 21:16:13.111701 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 380, in request Jul 16 21:16:13.111701 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return self._handle_response(resp) Jul 16 21:16:13.111701 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 120, in _handle_response Jul 16 21:16:13.111701 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.from_response(resp, resp.content) Jul 16 21:16:13.111701 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi glanceclient.exc.HTTPNotFound: HTTP 404 Not Found: No image found with ID 22eff938-ec35-4b0e-bb04-069cb6537f2f Jul 16 21:16:13.111701 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:16:13.111701 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:16:13.111701 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:16:13.111701 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:16:13.111701 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/image_metadata.py", line 39, in _get_image Jul 16 21:16:13.111701 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return self.image_api.get(context, image_id) Jul 16 21:16:13.111701 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 1205, in get Jul 16 21:16:13.111701 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return session.show(context, image_id, Jul 16 21:16:13.111701 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 287, in show Jul 16 21:16:13.111701 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi _reraise_translated_image_exception(image_id) Jul 16 21:16:13.131741 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 1031, in _reraise_translated_image_exception Jul 16 21:16:13.131741 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise new_exc.with_traceback(exc_trace) Jul 16 21:16:13.131741 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 285, in show Jul 16 21:16:13.131741 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi image = self._client.call(context, 2, 'get', args=(image_id,)) Jul 16 21:16:13.131741 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 191, in call Jul 16 21:16:13.131741 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi result = getattr(controller, method)(*args, **kwargs) Jul 16 21:16:13.131741 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/v2/images.py", line 197, in get Jul 16 21:16:13.131741 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return self._get(image_id) Jul 16 21:16:13.131741 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/utils.py", line 670, in inner Jul 16 21:16:13.131741 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return RequestIdProxy(wrapped(*args, **kwargs)) Jul 16 21:16:13.131741 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/v2/images.py", line 190, in _get Jul 16 21:16:13.131741 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi resp, body = self.http_client.get(url, headers=header) Jul 16 21:16:13.131741 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/adapter.py", line 395, in get Jul 16 21:16:13.131741 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return self.request(url, 'GET', **kwargs) Jul 16 21:16:13.131741 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 380, in request Jul 16 21:16:13.131741 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return self._handle_response(resp) Jul 16 21:16:13.131741 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 120, in _handle_response Jul 16 21:16:13.131741 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.from_response(resp, resp.content) Jul 16 21:16:13.131741 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.ImageNotFound: Image 22eff938-ec35-4b0e-bb04-069cb6537f2f could not be found. Jul 16 21:16:13.131741 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:16:13.131741 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:16:13.131741 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:16:13.131741 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:16:13.131741 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:16:13.131741 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:16:13.131741 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:16:13.131741 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:16:13.131741 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Jul 16 21:16:13.131741 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Jul 16 21:16:13.131741 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:16:13.131741 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:16:13.131741 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 16 21:16:13.131741 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 16 21:16:13.131741 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/image_metadata.py", line 109, in update_all Jul 16 21:16:13.131741 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi image = self._get_image(context, image_id) Jul 16 21:16:13.131741 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/image_metadata.py", line 44, in _get_image Jul 16 21:16:13.131741 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=msg) Jul 16 21:16:13.131741 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Image not found. Jul 16 21:16:13.131741 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:16:13.131741 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:16:13.131741 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:16:13.131741 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:16:13.131741 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:16:13.131741 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:16:13.131741 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:16:13.131741 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:16:13.131741 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Image not found. Jul 16 21:16:13.131741 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:16:13.131741 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-d360129d-8fe4-4308-a827-234d5ce1f7ef tempest-ImagesMetadataNegativeTestJSON-511989606 tempest-ImagesMetadataNegativeTestJSON-511989606-project] Returning 404 to user: Image not found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:16:13.131741 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-d360129d-8fe4-4308-a827-234d5ce1f7ef tempest-ImagesMetadataNegativeTestJSON-511989606 tempest-ImagesMetadataNegativeTestJSON-511989606-project] 10.222.0.16 "PUT /compute/v2.1/images/22eff938-ec35-4b0e-bb04-069cb6537f2f/metadata" status: 404 len: 62 microversion: 2.1 time: 0.056475 Jul 16 21:16:13.131741 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 886/1847] 10.222.0.16 () {60 vars in 1279 bytes} [Sat Jul 16 21:16:13 2022] PUT /compute/v2.1/images/22eff938-ec35-4b0e-bb04-069cb6537f2f/metadata => generated 62 bytes in 58 msecs (HTTP/1.1 404) 9 headers in 378 bytes (1 switches on core 0) Jul 16 21:16:13.137133 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-59fd5a0a-f1b4-4d8a-a0fe-3680ed2e9f81 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:13.137133 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-59fd5a0a-f1b4-4d8a-a0fe-3680ed2e9f81 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] [instance: 11194822-5d42-45f4-99d0-88c70c808ddd] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:13.151469 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-59fd5a0a-f1b4-4d8a-a0fe-3680ed2e9f81 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:13.158858 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-59fd5a0a-f1b4-4d8a-a0fe-3680ed2e9f81 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:13.160335 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-59fd5a0a-f1b4-4d8a-a0fe-3680ed2e9f81 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:13.164073 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-59fd5a0a-f1b4-4d8a-a0fe-3680ed2e9f81 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:13.248185 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-bc6db9fb-3cff-4a24-9a4a-8b9607003e32 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a69f9dee03d43beaafa966216b9e289', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9b769bf8f3ea4e368bf68de1df6538f9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:13.260086 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-bc6db9fb-3cff-4a24-9a4a-8b9607003e32 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] 10.222.0.16 "GET /compute/v2.1/servers/23df6e0a-3637-4f59-93f9-b02dd614b5c3" status: 200 len: 1343 microversion: 2.1 time: 0.229803 Jul 16 21:16:13.262873 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 962/1848] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:16:13 2022] GET /compute/v2.1/servers/23df6e0a-3637-4f59-93f9-b02dd614b5c3 => generated 1343 bytes in 239 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:16:13.277891 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-bfa5c5b6-8de9-4471-a2b7-e16ba45255dd tempest-ImagesMetadataNegativeTestJSON-511989606 tempest-ImagesMetadataNegativeTestJSON-511989606-project] Action: 'update', calling method: .version_select at 0x7fdd9d34baf0>, body: {"meta": {"os_distro": "alt"}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:16:13.315535 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-bfa5c5b6-8de9-4471-a2b7-e16ba45255dd tempest-ImagesMetadataNegativeTestJSON-511989606 tempest-ImagesMetadataNegativeTestJSON-511989606-project] HTTP exception thrown: Image not found. Jul 16 21:16:13.317636 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-bfa5c5b6-8de9-4471-a2b7-e16ba45255dd tempest-ImagesMetadataNegativeTestJSON-511989606 tempest-ImagesMetadataNegativeTestJSON-511989606-project] Request method failure captured: Jul 16 21:16:13.317636 ncdv-master-1 devstack@n-api.service[138680]: request: PUT /compute/v2.1/images/1e9bd18f-fcf8-49c2-8404-18e6f41b239e/metadata/os_distro HTTP/1.1 Jul 16 21:16:13.317636 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:16:13.317636 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:16:13.317636 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:16:13.317636 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 30 Jul 16 21:16:13.317636 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:16:13.317636 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:16:13.317636 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:16:13.317636 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:16:13.317636 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0yqZkT1PUPUnzET9ZAX-gVvwOnLYP0XB16rL3CiSft_4-2yik_0nM1TjnA8QuhU9TPl18FOAH6hNk2W83R2uXPXb9LBAVwu5sPf5tx8634HEmK9BrlhvdwDryqW3j2z1mBjVeiFYH4_OXVjNdPS3YVrbp1SvI0sDi3Yiu6r0NJNZJWo Jul 16 21:16:13.317636 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:16:13.317636 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:16:13.317636 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:16:13.317636 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:16:13.317636 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:16:13.317636 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:16:13.317636 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: 5163e91b263949d2b4ac6cf1e110eb62 Jul 16 21:16:13.317636 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-ImagesMetadataNegativeTestJSON-511989606 Jul 16 21:16:13.317636 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:16:13.317636 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:16:13.317636 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/5163e91b263949d2b4ac6cf1e110eb62"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/5163e91b263949d2b4ac6cf1e110eb62"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_5163e91b263949d2b4ac6cf1e110eb62"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/5163e91b263949d2b4ac6cf1e110eb62"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:16:13.317636 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-ImagesMetadataNegativeTestJSON-511989606 Jul 16 21:16:13.317636 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: 5163e91b263949d2b4ac6cf1e110eb62 Jul 16 21:16:13.317636 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-ImagesMetadataNegativeTestJSON-511989606 Jul 16 21:16:13.317636 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-ImagesMetadataNegativeTestJSON-511989606-project Jul 16 21:16:13.317636 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:16:13.317636 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:16:13.317636 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: b3b5f2d7a35b44649de7c41e5fdebaac Jul 16 21:16:13.317636 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-ImagesMetadataNegativeTestJSON-511989606-project Jul 16 21:16:13.317636 ncdv-master-1 devstack@n-api.service[138680]: Jul 16 21:16:13.317636 ncdv-master-1 devstack@n-api.service[138680]: {"meta": {"os_distro": "alt"}} Jul 16 21:16:13.317636 ncdv-master-1 devstack@n-api.service[138680]: method: Jul 16 21:16:13.317636 ncdv-master-1 devstack@n-api.service[138680]: exception: Image not found. Jul 16 21:16:13.317636 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'image_id': '1e9bd18f-fcf8-49c2-8404-18e6f41b239e', 'id': 'os_distro', 'body': {'meta': {'os_distro': 'alt'}}} Jul 16 21:16:13.317636 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:16:13.317636 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:16:13.317636 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 285, in show Jul 16 21:16:13.317636 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi image = self._client.call(context, 2, 'get', args=(image_id,)) Jul 16 21:16:13.317636 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 191, in call Jul 16 21:16:13.317636 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi result = getattr(controller, method)(*args, **kwargs) Jul 16 21:16:13.317636 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/v2/images.py", line 197, in get Jul 16 21:16:13.317636 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return self._get(image_id) Jul 16 21:16:13.317636 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/utils.py", line 670, in inner Jul 16 21:16:13.317636 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return RequestIdProxy(wrapped(*args, **kwargs)) Jul 16 21:16:13.317636 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/v2/images.py", line 190, in _get Jul 16 21:16:13.317636 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi resp, body = self.http_client.get(url, headers=header) Jul 16 21:16:13.317636 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/adapter.py", line 395, in get Jul 16 21:16:13.317636 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return self.request(url, 'GET', **kwargs) Jul 16 21:16:13.317636 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 380, in request Jul 16 21:16:13.317636 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return self._handle_response(resp) Jul 16 21:16:13.317636 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 120, in _handle_response Jul 16 21:16:13.317636 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.from_response(resp, resp.content) Jul 16 21:16:13.317636 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi glanceclient.exc.HTTPNotFound: HTTP 404 Not Found: No image found with ID 1e9bd18f-fcf8-49c2-8404-18e6f41b239e Jul 16 21:16:13.317636 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:16:13.317636 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:16:13.317636 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:16:13.317636 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:16:13.317636 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/image_metadata.py", line 39, in _get_image Jul 16 21:16:13.317636 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return self.image_api.get(context, image_id) Jul 16 21:16:13.317636 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 1205, in get Jul 16 21:16:13.317636 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return session.show(context, image_id, Jul 16 21:16:13.317636 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 287, in show Jul 16 21:16:13.317636 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi _reraise_translated_image_exception(image_id) Jul 16 21:16:13.323158 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 1031, in _reraise_translated_image_exception Jul 16 21:16:13.323158 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise new_exc.with_traceback(exc_trace) Jul 16 21:16:13.323158 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 285, in show Jul 16 21:16:13.323158 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi image = self._client.call(context, 2, 'get', args=(image_id,)) Jul 16 21:16:13.323158 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 191, in call Jul 16 21:16:13.323158 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi result = getattr(controller, method)(*args, **kwargs) Jul 16 21:16:13.323158 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/v2/images.py", line 197, in get Jul 16 21:16:13.323158 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return self._get(image_id) Jul 16 21:16:13.323158 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/utils.py", line 670, in inner Jul 16 21:16:13.323158 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return RequestIdProxy(wrapped(*args, **kwargs)) Jul 16 21:16:13.323158 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/v2/images.py", line 190, in _get Jul 16 21:16:13.323158 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi resp, body = self.http_client.get(url, headers=header) Jul 16 21:16:13.323158 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/adapter.py", line 395, in get Jul 16 21:16:13.323158 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return self.request(url, 'GET', **kwargs) Jul 16 21:16:13.323158 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 380, in request Jul 16 21:16:13.323158 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return self._handle_response(resp) Jul 16 21:16:13.323158 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 120, in _handle_response Jul 16 21:16:13.323158 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.from_response(resp, resp.content) Jul 16 21:16:13.323158 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.ImageNotFound: Image 1e9bd18f-fcf8-49c2-8404-18e6f41b239e could not be found. Jul 16 21:16:13.323158 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:16:13.323158 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:16:13.323158 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:16:13.323158 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:16:13.323158 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:16:13.323158 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:16:13.323158 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:16:13.323158 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:16:13.323158 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Jul 16 21:16:13.323158 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Jul 16 21:16:13.323158 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:16:13.323158 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:16:13.323158 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 16 21:16:13.323158 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 16 21:16:13.323158 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/image_metadata.py", line 93, in update Jul 16 21:16:13.323158 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi image = self._get_image(context, image_id) Jul 16 21:16:13.323158 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/image_metadata.py", line 44, in _get_image Jul 16 21:16:13.323158 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=msg) Jul 16 21:16:13.323158 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Image not found. Jul 16 21:16:13.323158 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:16:13.323158 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:16:13.323158 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:16:13.323158 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:16:13.323158 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:16:13.323158 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:16:13.323158 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:16:13.323158 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:16:13.323158 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Image not found. Jul 16 21:16:13.323158 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:16:13.323158 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-bfa5c5b6-8de9-4471-a2b7-e16ba45255dd tempest-ImagesMetadataNegativeTestJSON-511989606 tempest-ImagesMetadataNegativeTestJSON-511989606-project] Returning 404 to user: Image not found. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:16:13.323158 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-bfa5c5b6-8de9-4471-a2b7-e16ba45255dd tempest-ImagesMetadataNegativeTestJSON-511989606 tempest-ImagesMetadataNegativeTestJSON-511989606-project] 10.222.0.16 "PUT /compute/v2.1/images/1e9bd18f-fcf8-49c2-8404-18e6f41b239e/metadata/os_distro" status: 404 len: 62 microversion: 2.1 time: 0.053042 Jul 16 21:16:13.328441 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 963/1849] 10.222.0.16 () {60 vars in 1309 bytes} [Sat Jul 16 21:16:13 2022] PUT /compute/v2.1/images/1e9bd18f-fcf8-49c2-8404-18e6f41b239e/metadata/os_distro => generated 62 bytes in 63 msecs (HTTP/1.1 404) 9 headers in 378 bytes (1 switches on core 0) Jul 16 21:16:13.338712 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-641e80a2-3062-47b5-8170-0127bb77797b tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:13.338712 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-641e80a2-3062-47b5-8170-0127bb77797b tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] [instance: f6c350c4-b1b2-42d3-ba02-65cbbacf0f58] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:13.353222 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-641e80a2-3062-47b5-8170-0127bb77797b tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:13.353682 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-641e80a2-3062-47b5-8170-0127bb77797b tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:13.354469 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-641e80a2-3062-47b5-8170-0127bb77797b tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:13.354906 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-641e80a2-3062-47b5-8170-0127bb77797b tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:13.806794 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-641e80a2-3062-47b5-8170-0127bb77797b tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4e9df82f95ac4779859972403869096a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4337575622d04a26942905c9dc044ba7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:13.815302 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-641e80a2-3062-47b5-8170-0127bb77797b tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] 10.222.0.16 "GET /compute/v2.1/servers/f6c350c4-b1b2-42d3-ba02-65cbbacf0f58" status: 200 len: 1325 microversion: 2.1 time: 0.486772 Jul 16 21:16:13.818172 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 964/1850] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:16:13 2022] GET /compute/v2.1/servers/f6c350c4-b1b2-42d3-ba02-65cbbacf0f58 => generated 1325 bytes in 491 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:16:13.833149 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-c08a6133-d51f-4105-b794-de1ad57454bd tempest-ImagesMetadataNegativeTestJSON-511989606 tempest-ImagesMetadataNegativeTestJSON-511989606-project] Action: 'create', calling method: .version_select at 0x7fdda2b90700>, body: {"metadata": {"os_distro": "alt1", "os_version": "alt2"}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:16:13.865824 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-c08a6133-d51f-4105-b794-de1ad57454bd tempest-ImagesMetadataNegativeTestJSON-511989606 tempest-ImagesMetadataNegativeTestJSON-511989606-project] HTTP exception thrown: Image not found. Jul 16 21:16:13.868018 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-c08a6133-d51f-4105-b794-de1ad57454bd tempest-ImagesMetadataNegativeTestJSON-511989606 tempest-ImagesMetadataNegativeTestJSON-511989606-project] Request method failure captured: Jul 16 21:16:13.868018 ncdv-master-1 devstack@n-api.service[138680]: request: POST /compute/v2.1/images/abac1470-fcd7-4a92-bde9-c2a41c735543/metadata HTTP/1.1 Jul 16 21:16:13.868018 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:16:13.868018 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:16:13.868018 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:16:13.868018 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 57 Jul 16 21:16:13.868018 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:16:13.868018 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:16:13.868018 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:16:13.868018 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:16:13.868018 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0yqZkT1PUPUnzET9ZAX-gVvwOnLYP0XB16rL3CiSft_4-2yik_0nM1TjnA8QuhU9TPl18FOAH6hNk2W83R2uXPXb9LBAVwu5sPf5tx8634HEmK9BrlhvdwDryqW3j2z1mBjVeiFYH4_OXVjNdPS3YVrbp1SvI0sDi3Yiu6r0NJNZJWo Jul 16 21:16:13.868018 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:16:13.868018 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:16:13.868018 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:16:13.868018 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:16:13.868018 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:16:13.868018 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:16:13.868018 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: 5163e91b263949d2b4ac6cf1e110eb62 Jul 16 21:16:13.868018 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-ImagesMetadataNegativeTestJSON-511989606 Jul 16 21:16:13.868018 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:16:13.868018 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:16:13.868018 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/5163e91b263949d2b4ac6cf1e110eb62"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/5163e91b263949d2b4ac6cf1e110eb62"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_5163e91b263949d2b4ac6cf1e110eb62"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/5163e91b263949d2b4ac6cf1e110eb62"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:16:13.868018 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-ImagesMetadataNegativeTestJSON-511989606 Jul 16 21:16:13.868018 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: 5163e91b263949d2b4ac6cf1e110eb62 Jul 16 21:16:13.868018 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-ImagesMetadataNegativeTestJSON-511989606 Jul 16 21:16:13.868018 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-ImagesMetadataNegativeTestJSON-511989606-project Jul 16 21:16:13.868018 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:16:13.868018 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:16:13.868018 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: b3b5f2d7a35b44649de7c41e5fdebaac Jul 16 21:16:13.868018 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-ImagesMetadataNegativeTestJSON-511989606-project Jul 16 21:16:13.868018 ncdv-master-1 devstack@n-api.service[138680]: Jul 16 21:16:13.868018 ncdv-master-1 devstack@n-api.service[138680]: {"metadata": {"os_distro": "alt1", "os_version": "alt2"}} Jul 16 21:16:13.868018 ncdv-master-1 devstack@n-api.service[138680]: method: Jul 16 21:16:13.868018 ncdv-master-1 devstack@n-api.service[138680]: exception: Image not found. Jul 16 21:16:13.868018 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'image_id': 'abac1470-fcd7-4a92-bde9-c2a41c735543', 'body': {'metadata': {'os_distro': 'alt1', 'os_version': 'alt2'}}} Jul 16 21:16:13.868018 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:16:13.868018 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:16:13.868018 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 285, in show Jul 16 21:16:13.868018 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi image = self._client.call(context, 2, 'get', args=(image_id,)) Jul 16 21:16:13.868018 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 191, in call Jul 16 21:16:13.868018 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi result = getattr(controller, method)(*args, **kwargs) Jul 16 21:16:13.868018 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/v2/images.py", line 197, in get Jul 16 21:16:13.868018 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return self._get(image_id) Jul 16 21:16:13.868018 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/utils.py", line 670, in inner Jul 16 21:16:13.868018 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return RequestIdProxy(wrapped(*args, **kwargs)) Jul 16 21:16:13.868018 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/v2/images.py", line 190, in _get Jul 16 21:16:13.868018 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi resp, body = self.http_client.get(url, headers=header) Jul 16 21:16:13.868018 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/adapter.py", line 395, in get Jul 16 21:16:13.868018 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return self.request(url, 'GET', **kwargs) Jul 16 21:16:13.868018 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 380, in request Jul 16 21:16:13.868018 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return self._handle_response(resp) Jul 16 21:16:13.868018 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 120, in _handle_response Jul 16 21:16:13.868018 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.from_response(resp, resp.content) Jul 16 21:16:13.868018 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi glanceclient.exc.HTTPNotFound: HTTP 404 Not Found: No image found with ID abac1470-fcd7-4a92-bde9-c2a41c735543 Jul 16 21:16:13.868018 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:16:13.868018 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:16:13.868018 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:16:13.868018 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:16:13.868018 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/image_metadata.py", line 39, in _get_image Jul 16 21:16:13.868018 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return self.image_api.get(context, image_id) Jul 16 21:16:13.868018 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 1205, in get Jul 16 21:16:13.868018 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return session.show(context, image_id, Jul 16 21:16:13.868018 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 287, in show Jul 16 21:16:13.868018 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi _reraise_translated_image_exception(image_id) Jul 16 21:16:13.873282 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 1031, in _reraise_translated_image_exception Jul 16 21:16:13.873282 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise new_exc.with_traceback(exc_trace) Jul 16 21:16:13.873282 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 285, in show Jul 16 21:16:13.873282 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi image = self._client.call(context, 2, 'get', args=(image_id,)) Jul 16 21:16:13.873282 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 191, in call Jul 16 21:16:13.873282 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi result = getattr(controller, method)(*args, **kwargs) Jul 16 21:16:13.873282 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/v2/images.py", line 197, in get Jul 16 21:16:13.873282 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return self._get(image_id) Jul 16 21:16:13.873282 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/utils.py", line 670, in inner Jul 16 21:16:13.873282 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return RequestIdProxy(wrapped(*args, **kwargs)) Jul 16 21:16:13.873282 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/v2/images.py", line 190, in _get Jul 16 21:16:13.873282 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi resp, body = self.http_client.get(url, headers=header) Jul 16 21:16:13.873282 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/adapter.py", line 395, in get Jul 16 21:16:13.873282 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return self.request(url, 'GET', **kwargs) Jul 16 21:16:13.873282 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 380, in request Jul 16 21:16:13.873282 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return self._handle_response(resp) Jul 16 21:16:13.873282 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 120, in _handle_response Jul 16 21:16:13.873282 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.from_response(resp, resp.content) Jul 16 21:16:13.873282 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.ImageNotFound: Image abac1470-fcd7-4a92-bde9-c2a41c735543 could not be found. Jul 16 21:16:13.873282 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:16:13.873282 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:16:13.873282 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:16:13.873282 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:16:13.873282 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:16:13.873282 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:16:13.873282 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:16:13.873282 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:16:13.873282 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Jul 16 21:16:13.873282 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Jul 16 21:16:13.873282 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:16:13.873282 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:16:13.873282 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 16 21:16:13.873282 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 16 21:16:13.873282 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/image_metadata.py", line 69, in create Jul 16 21:16:13.873282 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi image = self._get_image(context, image_id) Jul 16 21:16:13.873282 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/image_metadata.py", line 44, in _get_image Jul 16 21:16:13.873282 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=msg) Jul 16 21:16:13.873282 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Image not found. Jul 16 21:16:13.873282 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:16:13.873282 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:16:13.873282 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:16:13.873282 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:16:13.873282 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:16:13.873282 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:16:13.873282 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:16:13.873282 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:16:13.873282 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Image not found. Jul 16 21:16:13.873282 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:16:13.873282 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-c08a6133-d51f-4105-b794-de1ad57454bd tempest-ImagesMetadataNegativeTestJSON-511989606 tempest-ImagesMetadataNegativeTestJSON-511989606-project] Returning 404 to user: Image not found. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:16:13.873282 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-c08a6133-d51f-4105-b794-de1ad57454bd tempest-ImagesMetadataNegativeTestJSON-511989606 tempest-ImagesMetadataNegativeTestJSON-511989606-project] 10.222.0.16 "POST /compute/v2.1/images/abac1470-fcd7-4a92-bde9-c2a41c735543/metadata" status: 404 len: 62 microversion: 2.1 time: 0.049371 Jul 16 21:16:13.873282 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 965/1851] 10.222.0.16 () {60 vars in 1280 bytes} [Sat Jul 16 21:16:13 2022] POST /compute/v2.1/images/abac1470-fcd7-4a92-bde9-c2a41c735543/metadata => generated 62 bytes in 51 msecs (HTTP/1.1 404) 9 headers in 378 bytes (1 switches on core 0) Jul 16 21:16:13.880618 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-d663eb6c-2a71-4f1b-beba-0c88c04618e6 req-08725cc6-79e3-4f63-bab6-b31845a576e5 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "f6c350c4-b1b2-42d3-ba02-65cbbacf0f58", "name": "network-vif-plugged", "status": "completed", "tag": "d0843d7d-ff0d-4603-aeeb-d26f6ebde7db"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:16:13.903454 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-d663eb6c-2a71-4f1b-beba-0c88c04618e6 req-08725cc6-79e3-4f63-bab6-b31845a576e5 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:13.903909 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-d663eb6c-2a71-4f1b-beba-0c88c04618e6 req-08725cc6-79e3-4f63-bab6-b31845a576e5 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:13.936015 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-d663eb6c-2a71-4f1b-beba-0c88c04618e6 req-08725cc6-79e3-4f63-bab6-b31845a576e5 service nova] Creating event network-vif-plugged:d0843d7d-ff0d-4603-aeeb-d26f6ebde7db for instance f6c350c4-b1b2-42d3-ba02-65cbbacf0f58 on nch1-master-1 Jul 16 21:16:13.942888 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-d663eb6c-2a71-4f1b-beba-0c88c04618e6 req-08725cc6-79e3-4f63-bab6-b31845a576e5 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.071334 Jul 16 21:16:13.944255 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 966/1852] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:16:13 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 73 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:16:13.951590 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-6fa3cf90-6d13-44be-bfb9-855f2d3a3c45 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Calling method '.version_select at 0x7fdda2b90160>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:14.033966 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-6fa3cf90-6d13-44be-bfb9-855f2d3a3c45 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] 10.222.0.16 "GET /compute/v2.1/images/5af5e2eb-8145-46b4-abad-4d2a1e3630fc" status: 200 len: 1413 microversion: 2.1 time: 0.088222 Jul 16 21:16:14.036248 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 967/1853] 10.222.0.16 () {58 vars in 1232 bytes} [Sat Jul 16 21:16:13 2022] GET /compute/v2.1/images/5af5e2eb-8145-46b4-abad-4d2a1e3630fc => generated 1413 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:16:14.041938 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-b534f4bf-bd82-4349-8014-3008b4bf6e9f tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:14.048683 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-b534f4bf-bd82-4349-8014-3008b4bf6e9f tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] [instance: b9f2c527-6346-4403-841e-b7bd876a5fb7] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:14.061677 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b534f4bf-bd82-4349-8014-3008b4bf6e9f tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:14.062572 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b534f4bf-bd82-4349-8014-3008b4bf6e9f tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:14.066329 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b534f4bf-bd82-4349-8014-3008b4bf6e9f tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:14.067118 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b534f4bf-bd82-4349-8014-3008b4bf6e9f tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:14.228583 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-59fd5a0a-f1b4-4d8a-a0fe-3680ed2e9f81 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd4144b5388b46a989a41a2f556bb508', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a90d4bcd25ff4bf68bdc194180fbbe09', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:14.250374 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-59fd5a0a-f1b4-4d8a-a0fe-3680ed2e9f81 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] 10.222.0.16 "GET /compute/v2.1/servers/11194822-5d42-45f4-99d0-88c70c808ddd" status: 200 len: 1317 microversion: 2.1 time: 1.134041 Jul 16 21:16:14.257850 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 887/1854] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:16:13 2022] GET /compute/v2.1/servers/11194822-5d42-45f4-99d0-88c70c808ddd => generated 1317 bytes in 1140 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:16:14.277615 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-3154ea8b-a14e-44d7-a70e-cb3b890e8adf tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:14.279840 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-3154ea8b-a14e-44d7-a70e-cb3b890e8adf tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] [instance: 84ab7e65-1bf0-4b8b-baea-6d1d4ca13dd7] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:14.314725 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3154ea8b-a14e-44d7-a70e-cb3b890e8adf tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:14.325447 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3154ea8b-a14e-44d7-a70e-cb3b890e8adf tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:14.326813 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3154ea8b-a14e-44d7-a70e-cb3b890e8adf tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:14.328089 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3154ea8b-a14e-44d7-a70e-cb3b890e8adf tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:14.559480 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-b534f4bf-bd82-4349-8014-3008b4bf6e9f tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aa99be675e946fd995d71465b4cbec6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd82c2d92c67f469e863eb70bb7f9af84', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:14.579533 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-b534f4bf-bd82-4349-8014-3008b4bf6e9f tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] 10.222.0.16 "GET /compute/v2.1/servers/b9f2c527-6346-4403-841e-b7bd876a5fb7" status: 200 len: 1293 microversion: 2.1 time: 0.533245 Jul 16 21:16:14.580429 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 968/1855] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:16:14 2022] GET /compute/v2.1/servers/b9f2c527-6346-4403-841e-b7bd876a5fb7 => generated 1293 bytes in 543 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:16:14.586163 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-68acc64e-28c5-4e13-86c2-d25cf78007ba tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:14.588531 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-68acc64e-28c5-4e13-86c2-d25cf78007ba tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] [instance: 23df6e0a-3637-4f59-93f9-b02dd614b5c3] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:14.616673 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-68acc64e-28c5-4e13-86c2-d25cf78007ba tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:14.620295 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-68acc64e-28c5-4e13-86c2-d25cf78007ba tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:14.621611 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-68acc64e-28c5-4e13-86c2-d25cf78007ba tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:14.622249 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-68acc64e-28c5-4e13-86c2-d25cf78007ba tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:14.667953 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-3154ea8b-a14e-44d7-a70e-cb3b890e8adf tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:14.679356 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-3154ea8b-a14e-44d7-a70e-cb3b890e8adf tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:14.693628 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-3154ea8b-a14e-44d7-a70e-cb3b890e8adf tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:14.695414 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-3154ea8b-a14e-44d7-a70e-cb3b890e8adf tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] 10.222.0.16 "GET /compute/v2.1/servers/84ab7e65-1bf0-4b8b-baea-6d1d4ca13dd7" status: 200 len: 1487 microversion: 2.25 time: 0.435525 Jul 16 21:16:14.703291 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 888/1856] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:16:14 2022] GET /compute/v2.1/servers/84ab7e65-1bf0-4b8b-baea-6d1d4ca13dd7 => generated 1487 bytes in 438 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:16:14.726184 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-68acc64e-28c5-4e13-86c2-d25cf78007ba tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:14.726930 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-68acc64e-28c5-4e13-86c2-d25cf78007ba tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:14.727590 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.objects.instance [None req-68acc64e-28c5-4e13-86c2-d25cf78007ba tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lazy-loading 'fault' on Instance uuid 23df6e0a-3637-4f59-93f9-b02dd614b5c3 {{(pid=138680) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:16:14.811559 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-68acc64e-28c5-4e13-86c2-d25cf78007ba tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a69f9dee03d43beaafa966216b9e289', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9b769bf8f3ea4e368bf68de1df6538f9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:14.821380 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-68acc64e-28c5-4e13-86c2-d25cf78007ba tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] 10.222.0.16 "GET /compute/v2.1/servers/23df6e0a-3637-4f59-93f9-b02dd614b5c3" status: 200 len: 1186 microversion: 2.1 time: 0.239029 Jul 16 21:16:14.838111 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 969/1857] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:16:14 2022] GET /compute/v2.1/servers/23df6e0a-3637-4f59-93f9-b02dd614b5c3 => generated 1186 bytes in 256 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:16:14.841543 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-fcb1735e-0467-43f4-90c0-dc768600d23e tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:14.842557 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-fcb1735e-0467-43f4-90c0-dc768600d23e tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] [instance: f6c350c4-b1b2-42d3-ba02-65cbbacf0f58] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:14.856199 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fcb1735e-0467-43f4-90c0-dc768600d23e tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:14.857302 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fcb1735e-0467-43f4-90c0-dc768600d23e tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:14.858370 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fcb1735e-0467-43f4-90c0-dc768600d23e tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:14.858979 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fcb1735e-0467-43f4-90c0-dc768600d23e tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:15.048270 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-c65e43fa-c651-4332-b8c0-91c4bd6e9c20 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Calling method '.version_select at 0x7fdda2ae0550>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:15.137672 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-c65e43fa-c651-4332-b8c0-91c4bd6e9c20 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] 10.222.0.16 "GET /compute/v2.1/images/5af5e2eb-8145-46b4-abad-4d2a1e3630fc" status: 200 len: 1413 microversion: 2.1 time: 0.093928 Jul 16 21:16:15.138636 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 970/1858] 10.222.0.16 () {58 vars in 1232 bytes} [Sat Jul 16 21:16:15 2022] GET /compute/v2.1/images/5af5e2eb-8145-46b4-abad-4d2a1e3630fc => generated 1413 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:16:15.264167 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-fcb1735e-0467-43f4-90c0-dc768600d23e tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4e9df82f95ac4779859972403869096a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4337575622d04a26942905c9dc044ba7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:15.272485 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-138758eb-3f28-441f-8ba8-3dcf5fd5a83d tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:15.275726 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-138758eb-3f28-441f-8ba8-3dcf5fd5a83d tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] [instance: 11194822-5d42-45f4-99d0-88c70c808ddd] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:15.285531 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-138758eb-3f28-441f-8ba8-3dcf5fd5a83d tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:15.286283 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-138758eb-3f28-441f-8ba8-3dcf5fd5a83d tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:15.290927 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-fcb1735e-0467-43f4-90c0-dc768600d23e tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] 10.222.0.16 "GET /compute/v2.1/servers/f6c350c4-b1b2-42d3-ba02-65cbbacf0f58" status: 200 len: 1325 microversion: 2.1 time: 0.455081 Jul 16 21:16:15.291633 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-138758eb-3f28-441f-8ba8-3dcf5fd5a83d tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:15.292331 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-138758eb-3f28-441f-8ba8-3dcf5fd5a83d tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:15.297470 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 889/1859] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:16:14 2022] GET /compute/v2.1/servers/f6c350c4-b1b2-42d3-ba02-65cbbacf0f58 => generated 1325 bytes in 462 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:16:15.601358 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-2d36a9d7-d67c-421a-b823-c379065ffe3b tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:15.602220 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-2d36a9d7-d67c-421a-b823-c379065ffe3b tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] [instance: b9f2c527-6346-4403-841e-b7bd876a5fb7] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:15.626809 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2d36a9d7-d67c-421a-b823-c379065ffe3b tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:15.629503 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2d36a9d7-d67c-421a-b823-c379065ffe3b tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:15.631214 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2d36a9d7-d67c-421a-b823-c379065ffe3b tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:15.632405 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2d36a9d7-d67c-421a-b823-c379065ffe3b tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:15.785224 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-138758eb-3f28-441f-8ba8-3dcf5fd5a83d tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd4144b5388b46a989a41a2f556bb508', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a90d4bcd25ff4bf68bdc194180fbbe09', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:15.794145 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-138758eb-3f28-441f-8ba8-3dcf5fd5a83d tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] 10.222.0.16 "GET /compute/v2.1/servers/11194822-5d42-45f4-99d0-88c70c808ddd" status: 200 len: 1317 microversion: 2.1 time: 0.525581 Jul 16 21:16:15.794741 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 971/1860] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:16:15 2022] GET /compute/v2.1/servers/11194822-5d42-45f4-99d0-88c70c808ddd => generated 1317 bytes in 527 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:16:15.808092 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-20291f0f-42ec-459d-a3ba-2ee6b0068af0 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:15.812727 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-20291f0f-42ec-459d-a3ba-2ee6b0068af0 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] [instance: 84ab7e65-1bf0-4b8b-baea-6d1d4ca13dd7] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:15.825576 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-20291f0f-42ec-459d-a3ba-2ee6b0068af0 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:15.826375 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-20291f0f-42ec-459d-a3ba-2ee6b0068af0 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:15.827458 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-20291f0f-42ec-459d-a3ba-2ee6b0068af0 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:15.827904 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-20291f0f-42ec-459d-a3ba-2ee6b0068af0 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:16.042875 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-2d36a9d7-d67c-421a-b823-c379065ffe3b tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aa99be675e946fd995d71465b4cbec6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd82c2d92c67f469e863eb70bb7f9af84', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:16.057224 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-2d36a9d7-d67c-421a-b823-c379065ffe3b tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] 10.222.0.16 "GET /compute/v2.1/servers/b9f2c527-6346-4403-841e-b7bd876a5fb7" status: 200 len: 1293 microversion: 2.1 time: 0.463870 Jul 16 21:16:16.057864 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 890/1861] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:16:15 2022] GET /compute/v2.1/servers/b9f2c527-6346-4403-841e-b7bd876a5fb7 => generated 1293 bytes in 466 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:16:16.072352 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-31c5ce01-1f4b-4b07-bbc0-bdbe768e8e58 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:16.073622 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-31c5ce01-1f4b-4b07-bbc0-bdbe768e8e58 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] [instance: 23df6e0a-3637-4f59-93f9-b02dd614b5c3] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:16.087147 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-31c5ce01-1f4b-4b07-bbc0-bdbe768e8e58 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:16.095441 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-31c5ce01-1f4b-4b07-bbc0-bdbe768e8e58 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:16.099049 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-31c5ce01-1f4b-4b07-bbc0-bdbe768e8e58 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:16.099049 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-31c5ce01-1f4b-4b07-bbc0-bdbe768e8e58 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:16.114864 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=100 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:16:16.116860 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-31c5ce01-1f4b-4b07-bbc0-bdbe768e8e58 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] HTTP exception thrown: Instance 23df6e0a-3637-4f59-93f9-b02dd614b5c3 could not be found. Jul 16 21:16:16.119220 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-31c5ce01-1f4b-4b07-bbc0-bdbe768e8e58 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Request method failure captured: Jul 16 21:16:16.119220 ncdv-master-1 devstack@n-api.service[138681]: request: GET /compute/v2.1/servers/23df6e0a-3637-4f59-93f9-b02dd614b5c3 HTTP/1.1 Jul 16 21:16:16.119220 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:16:16.119220 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:16:16.119220 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:16:16.119220 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 21:16:16.119220 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:16:16.119220 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:16:16.119220 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:16:16.119220 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:16:16.119220 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0yjys5urr_ei5Seo_QKDPCqmR0EFBibpmCG26yjPCPljKUNzADf-WX077qzJM5iZPEhFd-9J9wlncEYo9S_mA0fOh-Gvf9LxFXEvyCYlVLvbQiN1HZ19aSAOzuz2ad1WkBxSsEap9oMo0DE5_h7t-ONjtOk159u5RHwmU9Cr_ibnvE8 Jul 16 21:16:16.119220 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:16:16.119220 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:16:16.119220 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:16:16.119220 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:16:16.119220 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:16:16.119220 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:16:16.119220 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: 9b769bf8f3ea4e368bf68de1df6538f9 Jul 16 21:16:16.119220 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-LiveMigrationTest-1001758322 Jul 16 21:16:16.119220 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:16:16.119220 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:16:16.119220 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/9b769bf8f3ea4e368bf68de1df6538f9"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/9b769bf8f3ea4e368bf68de1df6538f9"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_9b769bf8f3ea4e368bf68de1df6538f9"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/9b769bf8f3ea4e368bf68de1df6538f9"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:16:16.119220 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-LiveMigrationTest-1001758322 Jul 16 21:16:16.119220 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: 9b769bf8f3ea4e368bf68de1df6538f9 Jul 16 21:16:16.119220 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-LiveMigrationTest-1001758322 Jul 16 21:16:16.119220 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-LiveMigrationTest-1001758322-project Jul 16 21:16:16.119220 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:16:16.119220 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:16:16.119220 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: 6a69f9dee03d43beaafa966216b9e289 Jul 16 21:16:16.119220 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-LiveMigrationTest-1001758322-project Jul 16 21:16:16.119220 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:16:16.119220 ncdv-master-1 devstack@n-api.service[138681]: exception: Instance 23df6e0a-3637-4f59-93f9-b02dd614b5c3 could not be found. Jul 16 21:16:16.119220 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': '23df6e0a-3637-4f59-93f9-b02dd614b5c3'} Jul 16 21:16:16.119220 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:16:16.119220 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:16:16.119220 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:16:16.119220 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:16:16.119220 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:16:16.119220 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:16:16.119220 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:16:16.119220 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:16:16.119220 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:16:16.119220 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:16:16.119220 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 23df6e0a-3637-4f59-93f9-b02dd614b5c3 could not be found. Jul 16 21:16:16.119220 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:16:16.119220 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:16:16.119220 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:16:16.119220 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:16:16.119220 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:16:16.119220 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:16:16.119220 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:16:16.119220 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:16:16.119220 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:16:16.119220 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:16:16.119220 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:16:16.119220 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:16:16.119220 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:16:16.119220 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:16:16.119220 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:16:16.119220 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:16:16.119220 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 23df6e0a-3637-4f59-93f9-b02dd614b5c3 could not be found. Jul 16 21:16:16.119220 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:16:16.119220 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:16:16.119220 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:16:16.125932 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:16:16.125932 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:16:16.125932 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:16:16.125932 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:16:16.125932 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:16:16.125932 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 23df6e0a-3637-4f59-93f9-b02dd614b5c3 could not be found. Jul 16 21:16:16.125932 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:16:16.128390 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-31c5ce01-1f4b-4b07-bbc0-bdbe768e8e58 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Returning 404 to user: Instance 23df6e0a-3637-4f59-93f9-b02dd614b5c3 could not be found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:16:16.130165 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-31c5ce01-1f4b-4b07-bbc0-bdbe768e8e58 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] 10.222.0.16 "GET /compute/v2.1/servers/23df6e0a-3637-4f59-93f9-b02dd614b5c3" status: 404 len: 111 microversion: 2.1 time: 0.070863 Jul 16 21:16:16.131310 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 891/1862] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:16:16 2022] GET /compute/v2.1/servers/23df6e0a-3637-4f59-93f9-b02dd614b5c3 => generated 111 bytes in 73 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:16:16.138422 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-cd35950d-9a68-42be-ae61-585b9b23651c req-515323e4-d147-4dfd-bc05-de2742022423 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "b9f2c527-6346-4403-841e-b7bd876a5fb7", "name": "network-vif-plugged", "status": "completed", "tag": "edbdb017-2f7f-440d-9d46-aacbd858ccb6"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:16:16.152808 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-cd35950d-9a68-42be-ae61-585b9b23651c req-515323e4-d147-4dfd-bc05-de2742022423 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:16.156529 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-cd35950d-9a68-42be-ae61-585b9b23651c req-515323e4-d147-4dfd-bc05-de2742022423 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:16.171242 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-cd35950d-9a68-42be-ae61-585b9b23651c req-515323e4-d147-4dfd-bc05-de2742022423 service nova] Creating event network-vif-plugged:edbdb017-2f7f-440d-9d46-aacbd858ccb6 for instance b9f2c527-6346-4403-841e-b7bd876a5fb7 on nch2-master-1 Jul 16 21:16:16.178227 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-cd35950d-9a68-42be-ae61-585b9b23651c req-515323e4-d147-4dfd-bc05-de2742022423 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.045520 Jul 16 21:16:16.179603 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 892/1863] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:16:16 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:16:16.185032 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-d0cf7870-03d4-4514-979e-88f3989a3b8e tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Calling method '.version_select at 0x7fdd9cdc4dc0>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:16.245239 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-d0cf7870-03d4-4514-979e-88f3989a3b8e tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] 10.222.0.16 "GET /compute/v2.1/images/5af5e2eb-8145-46b4-abad-4d2a1e3630fc" status: 200 len: 1413 microversion: 2.1 time: 0.063809 Jul 16 21:16:16.247087 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 893/1864] 10.222.0.16 () {58 vars in 1232 bytes} [Sat Jul 16 21:16:16 2022] GET /compute/v2.1/images/5af5e2eb-8145-46b4-abad-4d2a1e3630fc => generated 1413 bytes in 67 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:16:16.277370 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-20291f0f-42ec-459d-a3ba-2ee6b0068af0 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:16.293477 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-20291f0f-42ec-459d-a3ba-2ee6b0068af0 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:16.296444 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-20291f0f-42ec-459d-a3ba-2ee6b0068af0 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:16.298871 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-20291f0f-42ec-459d-a3ba-2ee6b0068af0 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] 10.222.0.16 "GET /compute/v2.1/servers/84ab7e65-1bf0-4b8b-baea-6d1d4ca13dd7" status: 200 len: 1477 microversion: 2.25 time: 0.494011 Jul 16 21:16:16.304229 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 972/1865] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:16:15 2022] GET /compute/v2.1/servers/84ab7e65-1bf0-4b8b-baea-6d1d4ca13dd7 => generated 1477 bytes in 509 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:16:16.324260 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-1ef24c6c-68ea-48c6-aad1-e6318ad79a97 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:16.325778 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-1ef24c6c-68ea-48c6-aad1-e6318ad79a97 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] [instance: f6c350c4-b1b2-42d3-ba02-65cbbacf0f58] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:16.340822 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1ef24c6c-68ea-48c6-aad1-e6318ad79a97 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:16.343641 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-09056381-368d-4061-9e8f-f915d7a3aa75 tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] Calling method '.version_select at 0x7fdda2b3da60>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:16.345401 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1ef24c6c-68ea-48c6-aad1-e6318ad79a97 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:16.350293 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1ef24c6c-68ea-48c6-aad1-e6318ad79a97 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:16.351614 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1ef24c6c-68ea-48c6-aad1-e6318ad79a97 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:16.353157 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-09056381-368d-4061-9e8f-f915d7a3aa75 tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:16.357577 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-09056381-368d-4061-9e8f-f915d7a3aa75 tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:16.387089 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-09056381-368d-4061-9e8f-f915d7a3aa75 tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] 10.222.0.16 "GET /compute/v2.1/os-migrations" status: 200 len: 3204 microversion: 2.25 time: 0.053473 Jul 16 21:16:16.388236 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 973/1866] 10.222.0.16 () {60 vars in 1183 bytes} [Sat Jul 16 21:16:16 2022] GET /compute/v2.1/os-migrations => generated 3204 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:16:16.403483 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-35a0873d-a715-4668-a061-a23c15df8a75 tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:16.412033 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-35a0873d-a715-4668-a061-a23c15df8a75 tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] [instance: 84ab7e65-1bf0-4b8b-baea-6d1d4ca13dd7] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:16.420969 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-35a0873d-a715-4668-a061-a23c15df8a75 tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:16.420969 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-35a0873d-a715-4668-a061-a23c15df8a75 tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:16.420969 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-35a0873d-a715-4668-a061-a23c15df8a75 tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:16.421589 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-35a0873d-a715-4668-a061-a23c15df8a75 tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:16.817236 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.objects.instance [None req-35a0873d-a715-4668-a061-a23c15df8a75 tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] Lazy-loading 'services' on Instance uuid 84ab7e65-1bf0-4b8b-baea-6d1d4ca13dd7 {{(pid=138680) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:16:16.846905 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-35a0873d-a715-4668-a061-a23c15df8a75 tempest-LiveAutoBlockMigrationV225Test-1179203521 tempest-LiveAutoBlockMigrationV225Test-1179203521-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/84ab7e65-1bf0-4b8b-baea-6d1d4ca13dd7" status: 200 len: 1970 microversion: 2.25 time: 0.448917 Jul 16 21:16:16.864138 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 974/1867] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:16:16 2022] GET /compute/v2.1/servers/84ab7e65-1bf0-4b8b-baea-6d1d4ca13dd7 => generated 1970 bytes in 467 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:16:16.865994 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-1ef24c6c-68ea-48c6-aad1-e6318ad79a97 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4e9df82f95ac4779859972403869096a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4337575622d04a26942905c9dc044ba7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:16.881391 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-cb27f250-edc8-4ed8-a67d-58c3c5701264 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:16.882497 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-cb27f250-edc8-4ed8-a67d-58c3c5701264 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] [instance: 11194822-5d42-45f4-99d0-88c70c808ddd] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:16.885540 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-1ef24c6c-68ea-48c6-aad1-e6318ad79a97 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] 10.222.0.16 "GET /compute/v2.1/servers/f6c350c4-b1b2-42d3-ba02-65cbbacf0f58" status: 200 len: 1521 microversion: 2.1 time: 0.574731 Jul 16 21:16:16.887445 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 894/1868] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:16:16 2022] GET /compute/v2.1/servers/f6c350c4-b1b2-42d3-ba02-65cbbacf0f58 => generated 1521 bytes in 578 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:16:16.899981 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-f4bb3cae-5d2a-4ba4-8874-7c929cf735ea tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:16.900764 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-f4bb3cae-5d2a-4ba4-8874-7c929cf735ea tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] [instance: 84ab7e65-1bf0-4b8b-baea-6d1d4ca13dd7] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:16.911050 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cb27f250-edc8-4ed8-a67d-58c3c5701264 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:16.914937 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cb27f250-edc8-4ed8-a67d-58c3c5701264 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:16.919815 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cb27f250-edc8-4ed8-a67d-58c3c5701264 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:16.919815 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cb27f250-edc8-4ed8-a67d-58c3c5701264 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:16.920039 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f4bb3cae-5d2a-4ba4-8874-7c929cf735ea tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:16.927502 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f4bb3cae-5d2a-4ba4-8874-7c929cf735ea tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.009s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:16.928701 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f4bb3cae-5d2a-4ba4-8874-7c929cf735ea tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:16.929157 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f4bb3cae-5d2a-4ba4-8874-7c929cf735ea tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:16.998753 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-f4bb3cae-5d2a-4ba4-8874-7c929cf735ea tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] [instance: 84ab7e65-1bf0-4b8b-baea-6d1d4ca13dd7] Going to try to terminate instance {{(pid=138681) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 16 21:16:17.162360 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-cb27f250-edc8-4ed8-a67d-58c3c5701264 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd4144b5388b46a989a41a2f556bb508', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a90d4bcd25ff4bf68bdc194180fbbe09', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:17.168443 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-cb27f250-edc8-4ed8-a67d-58c3c5701264 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] 10.222.0.16 "GET /compute/v2.1/servers/11194822-5d42-45f4-99d0-88c70c808ddd" status: 200 len: 1317 microversion: 2.1 time: 0.302438 Jul 16 21:16:17.169638 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 975/1869] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:16:16 2022] GET /compute/v2.1/servers/11194822-5d42-45f4-99d0-88c70c808ddd => generated 1317 bytes in 304 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:16:17.177058 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-ab27e8bf-be69-476d-a7c6-10055d382440 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:17.178111 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-ab27e8bf-be69-476d-a7c6-10055d382440 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] [instance: f6c350c4-b1b2-42d3-ba02-65cbbacf0f58] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:17.202424 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ab27e8bf-be69-476d-a7c6-10055d382440 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:17.202900 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ab27e8bf-be69-476d-a7c6-10055d382440 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:17.208330 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ab27e8bf-be69-476d-a7c6-10055d382440 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:17.232602 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ab27e8bf-be69-476d-a7c6-10055d382440 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:17.318187 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-ab27e8bf-be69-476d-a7c6-10055d382440 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] [instance: f6c350c4-b1b2-42d3-ba02-65cbbacf0f58] Going to try to terminate instance {{(pid=138680) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 16 21:16:17.553700 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-f4bb3cae-5d2a-4ba4-8874-7c929cf735ea tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] 10.222.0.16 "DELETE /compute/v2.1/servers/84ab7e65-1bf0-4b8b-baea-6d1d4ca13dd7" status: 204 len: 0 microversion: 2.25 time: 0.664023 Jul 16 21:16:17.556780 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 895/1870] 10.222.0.16 () {60 vars in 1279 bytes} [Sat Jul 16 21:16:16 2022] DELETE /compute/v2.1/servers/84ab7e65-1bf0-4b8b-baea-6d1d4ca13dd7 => generated 0 bytes in 665 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Jul 16 21:16:17.568814 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-1d64d7e6-bc76-498f-8fd4-cd7f3bc87b2b tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:17.569826 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-1d64d7e6-bc76-498f-8fd4-cd7f3bc87b2b tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] [instance: b9f2c527-6346-4403-841e-b7bd876a5fb7] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:17.576559 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1d64d7e6-bc76-498f-8fd4-cd7f3bc87b2b tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:17.578045 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1d64d7e6-bc76-498f-8fd4-cd7f3bc87b2b tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:17.579420 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1d64d7e6-bc76-498f-8fd4-cd7f3bc87b2b tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:17.580326 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1d64d7e6-bc76-498f-8fd4-cd7f3bc87b2b tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:17.810912 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-1d64d7e6-bc76-498f-8fd4-cd7f3bc87b2b tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aa99be675e946fd995d71465b4cbec6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd82c2d92c67f469e863eb70bb7f9af84', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:17.831451 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-1d64d7e6-bc76-498f-8fd4-cd7f3bc87b2b tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] 10.222.0.16 "GET /compute/v2.1/servers/b9f2c527-6346-4403-841e-b7bd876a5fb7" status: 200 len: 1293 microversion: 2.1 time: 0.272688 Jul 16 21:16:17.834568 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 896/1871] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:16:17 2022] GET /compute/v2.1/servers/b9f2c527-6346-4403-841e-b7bd876a5fb7 => generated 1293 bytes in 276 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:16:17.843209 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-6e62c725-41ea-4fd8-a648-2108151c00ea tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Calling method '.version_select at 0x7fdda2b8fd30>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:17.927042 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-6e62c725-41ea-4fd8-a648-2108151c00ea tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] 10.222.0.16 "GET /compute/v2.1/images/5af5e2eb-8145-46b4-abad-4d2a1e3630fc" status: 200 len: 1413 microversion: 2.1 time: 0.089447 Jul 16 21:16:17.927896 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 897/1872] 10.222.0.16 () {58 vars in 1232 bytes} [Sat Jul 16 21:16:17 2022] GET /compute/v2.1/images/5af5e2eb-8145-46b4-abad-4d2a1e3630fc => generated 1413 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:16:17.933140 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-81f5ffea-7a5d-40d3-8d10-a98765663caa tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:17.939040 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-81f5ffea-7a5d-40d3-8d10-a98765663caa tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] [instance: 84ab7e65-1bf0-4b8b-baea-6d1d4ca13dd7] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:17.939764 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-ab27e8bf-be69-476d-a7c6-10055d382440 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] 10.222.0.16 "DELETE /compute/v2.1/servers/f6c350c4-b1b2-42d3-ba02-65cbbacf0f58" status: 204 len: 0 microversion: 2.1 time: 0.766026 Jul 16 21:16:17.944667 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 976/1873] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:16:17 2022] DELETE /compute/v2.1/servers/f6c350c4-b1b2-42d3-ba02-65cbbacf0f58 => generated 0 bytes in 771 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 16 21:16:17.947463 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-81f5ffea-7a5d-40d3-8d10-a98765663caa tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:17.948166 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-81f5ffea-7a5d-40d3-8d10-a98765663caa tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:17.949168 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-81f5ffea-7a5d-40d3-8d10-a98765663caa tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:17.949777 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-81f5ffea-7a5d-40d3-8d10-a98765663caa tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:17.952784 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-8791388a-0a73-4c56-91cf-54da31e85a15 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:17.957579 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-8791388a-0a73-4c56-91cf-54da31e85a15 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] [instance: f6c350c4-b1b2-42d3-ba02-65cbbacf0f58] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:17.972804 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8791388a-0a73-4c56-91cf-54da31e85a15 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:17.973364 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8791388a-0a73-4c56-91cf-54da31e85a15 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:17.974160 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8791388a-0a73-4c56-91cf-54da31e85a15 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:17.974607 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8791388a-0a73-4c56-91cf-54da31e85a15 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:18.270887 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-81f5ffea-7a5d-40d3-8d10-a98765663caa tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:18.280714 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-8791388a-0a73-4c56-91cf-54da31e85a15 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4e9df82f95ac4779859972403869096a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4337575622d04a26942905c9dc044ba7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:18.292942 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-81f5ffea-7a5d-40d3-8d10-a98765663caa tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:18.293579 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-8791388a-0a73-4c56-91cf-54da31e85a15 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] 10.222.0.16 "GET /compute/v2.1/servers/f6c350c4-b1b2-42d3-ba02-65cbbacf0f58" status: 200 len: 1527 microversion: 2.1 time: 0.344167 Jul 16 21:16:18.294805 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-81f5ffea-7a5d-40d3-8d10-a98765663caa tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:18.296316 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-81f5ffea-7a5d-40d3-8d10-a98765663caa tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] 10.222.0.16 "GET /compute/v2.1/servers/84ab7e65-1bf0-4b8b-baea-6d1d4ca13dd7" status: 200 len: 1483 microversion: 2.25 time: 0.367345 Jul 16 21:16:18.296932 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 898/1874] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:16:17 2022] GET /compute/v2.1/servers/84ab7e65-1bf0-4b8b-baea-6d1d4ca13dd7 => generated 1483 bytes in 368 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:16:18.304645 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-f34bf751-7b5c-499c-a71d-baabe8a8b9ca tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:18.305378 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-f34bf751-7b5c-499c-a71d-baabe8a8b9ca tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] [instance: 11194822-5d42-45f4-99d0-88c70c808ddd] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:18.306155 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 977/1875] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:16:17 2022] GET /compute/v2.1/servers/f6c350c4-b1b2-42d3-ba02-65cbbacf0f58 => generated 1527 bytes in 360 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:16:18.311625 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f34bf751-7b5c-499c-a71d-baabe8a8b9ca tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:18.312072 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f34bf751-7b5c-499c-a71d-baabe8a8b9ca tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:18.312839 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f34bf751-7b5c-499c-a71d-baabe8a8b9ca tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:18.314049 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f34bf751-7b5c-499c-a71d-baabe8a8b9ca tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:18.461562 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-f34bf751-7b5c-499c-a71d-baabe8a8b9ca tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd4144b5388b46a989a41a2f556bb508', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a90d4bcd25ff4bf68bdc194180fbbe09', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:18.470335 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-f34bf751-7b5c-499c-a71d-baabe8a8b9ca tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] 10.222.0.16 "GET /compute/v2.1/servers/11194822-5d42-45f4-99d0-88c70c808ddd" status: 200 len: 1317 microversion: 2.1 time: 0.170413 Jul 16 21:16:18.470335 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 899/1876] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:16:18 2022] GET /compute/v2.1/servers/11194822-5d42-45f4-99d0-88c70c808ddd => generated 1317 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:16:18.611861 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-18c85cea-baef-4c2c-ab31-3102bcce3156 req-5be00b5d-7f61-4690-a01b-f5edad699cd8 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "11194822-5d42-45f4-99d0-88c70c808ddd", "name": "network-vif-plugged", "status": "completed", "tag": "1e6a5dec-046c-4c31-bb3f-1b8d00690085"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:16:18.621038 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-18c85cea-baef-4c2c-ab31-3102bcce3156 req-5be00b5d-7f61-4690-a01b-f5edad699cd8 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:18.621480 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-18c85cea-baef-4c2c-ab31-3102bcce3156 req-5be00b5d-7f61-4690-a01b-f5edad699cd8 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:18.632665 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-18c85cea-baef-4c2c-ab31-3102bcce3156 req-5be00b5d-7f61-4690-a01b-f5edad699cd8 service nova] Creating event network-vif-plugged:1e6a5dec-046c-4c31-bb3f-1b8d00690085 for instance 11194822-5d42-45f4-99d0-88c70c808ddd on nch1-master-1 Jul 16 21:16:18.637842 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-18c85cea-baef-4c2c-ab31-3102bcce3156 req-5be00b5d-7f61-4690-a01b-f5edad699cd8 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.030277 Jul 16 21:16:18.638453 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 978/1877] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:16:18 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:16:18.850053 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-adf9115c-7ec7-42bd-80df-2a1b0dee9a3d tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:18.851026 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-adf9115c-7ec7-42bd-80df-2a1b0dee9a3d tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] [instance: b9f2c527-6346-4403-841e-b7bd876a5fb7] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:18.870409 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-adf9115c-7ec7-42bd-80df-2a1b0dee9a3d tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:18.874493 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-adf9115c-7ec7-42bd-80df-2a1b0dee9a3d tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:18.875736 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-adf9115c-7ec7-42bd-80df-2a1b0dee9a3d tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:18.876293 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-adf9115c-7ec7-42bd-80df-2a1b0dee9a3d tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:18.950361 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-b8300934-f034-4255-b15b-23ee3d3df56b tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Calling method '.version_select at 0x7fdda2bc1790>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:19.073399 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-b8300934-f034-4255-b15b-23ee3d3df56b tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] 10.222.0.16 "GET /compute/v2.1/images/5af5e2eb-8145-46b4-abad-4d2a1e3630fc" status: 200 len: 1413 microversion: 2.1 time: 0.135813 Jul 16 21:16:19.089135 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 979/1878] 10.222.0.16 () {58 vars in 1232 bytes} [Sat Jul 16 21:16:18 2022] GET /compute/v2.1/images/5af5e2eb-8145-46b4-abad-4d2a1e3630fc => generated 1413 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:16:19.314608 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-e2cbb98d-12c7-46a6-b47a-61ef028b5007 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:19.316227 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-e2cbb98d-12c7-46a6-b47a-61ef028b5007 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] [instance: f6c350c4-b1b2-42d3-ba02-65cbbacf0f58] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:19.356565 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e2cbb98d-12c7-46a6-b47a-61ef028b5007 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:19.356945 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e2cbb98d-12c7-46a6-b47a-61ef028b5007 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:19.360894 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e2cbb98d-12c7-46a6-b47a-61ef028b5007 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:19.361554 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e2cbb98d-12c7-46a6-b47a-61ef028b5007 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:19.385828 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-adf9115c-7ec7-42bd-80df-2a1b0dee9a3d tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aa99be675e946fd995d71465b4cbec6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd82c2d92c67f469e863eb70bb7f9af84', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:19.404137 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-adf9115c-7ec7-42bd-80df-2a1b0dee9a3d tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] 10.222.0.16 "GET /compute/v2.1/servers/b9f2c527-6346-4403-841e-b7bd876a5fb7" status: 200 len: 1293 microversion: 2.1 time: 0.556907 Jul 16 21:16:19.408357 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 900/1879] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:16:18 2022] GET /compute/v2.1/servers/b9f2c527-6346-4403-841e-b7bd876a5fb7 => generated 1293 bytes in 561 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:16:19.414866 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-3470094b-e926-40b6-ba0a-cc7ff91b188c tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:19.418868 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-3470094b-e926-40b6-ba0a-cc7ff91b188c tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] [instance: 84ab7e65-1bf0-4b8b-baea-6d1d4ca13dd7] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:19.433212 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3470094b-e926-40b6-ba0a-cc7ff91b188c tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:19.435150 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3470094b-e926-40b6-ba0a-cc7ff91b188c tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:19.437269 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3470094b-e926-40b6-ba0a-cc7ff91b188c tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:19.441091 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3470094b-e926-40b6-ba0a-cc7ff91b188c tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:20.200715 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-e2cbb98d-12c7-46a6-b47a-61ef028b5007 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4e9df82f95ac4779859972403869096a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4337575622d04a26942905c9dc044ba7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:20.219747 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-e2cbb98d-12c7-46a6-b47a-61ef028b5007 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] 10.222.0.16 "GET /compute/v2.1/servers/f6c350c4-b1b2-42d3-ba02-65cbbacf0f58" status: 200 len: 1527 microversion: 2.1 time: 0.907117 Jul 16 21:16:20.219747 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 980/1880] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:16:19 2022] GET /compute/v2.1/servers/f6c350c4-b1b2-42d3-ba02-65cbbacf0f58 => generated 1527 bytes in 910 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:16:20.234437 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-32aa1ea4-6903-4dad-b3db-0630e7c7dd6a tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:20.235683 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-32aa1ea4-6903-4dad-b3db-0630e7c7dd6a tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] [instance: 11194822-5d42-45f4-99d0-88c70c808ddd] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:20.247318 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-32aa1ea4-6903-4dad-b3db-0630e7c7dd6a tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:20.255931 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-32aa1ea4-6903-4dad-b3db-0630e7c7dd6a tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:20.255931 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-32aa1ea4-6903-4dad-b3db-0630e7c7dd6a tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:20.255931 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-32aa1ea4-6903-4dad-b3db-0630e7c7dd6a tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:20.298621 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-3470094b-e926-40b6-ba0a-cc7ff91b188c tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:20.315746 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-3470094b-e926-40b6-ba0a-cc7ff91b188c tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:20.317594 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-3470094b-e926-40b6-ba0a-cc7ff91b188c tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:20.320164 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-3470094b-e926-40b6-ba0a-cc7ff91b188c tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] 10.222.0.16 "GET /compute/v2.1/servers/84ab7e65-1bf0-4b8b-baea-6d1d4ca13dd7" status: 200 len: 1483 microversion: 2.25 time: 0.911635 Jul 16 21:16:20.320768 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 901/1881] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:16:19 2022] GET /compute/v2.1/servers/84ab7e65-1bf0-4b8b-baea-6d1d4ca13dd7 => generated 1483 bytes in 915 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:16:20.329590 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-003119ba-b339-4faa-a201-6d8d938b8702 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Calling method '.version_select at 0x7fdda2ae6940>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:20.430786 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-003119ba-b339-4faa-a201-6d8d938b8702 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] 10.222.0.16 "GET /compute/v2.1/images/5af5e2eb-8145-46b4-abad-4d2a1e3630fc" status: 200 len: 1422 microversion: 2.1 time: 0.098295 Jul 16 21:16:20.433544 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 902/1882] 10.222.0.16 () {58 vars in 1232 bytes} [Sat Jul 16 21:16:20 2022] GET /compute/v2.1/images/5af5e2eb-8145-46b4-abad-4d2a1e3630fc => generated 1422 bytes in 112 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:16:20.446848 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-cc251b3f-d3ee-42d7-b9e0-297d4e89e3fc req-2660b66a-1407-497e-97a3-16a5f47b44c6 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "84ab7e65-1bf0-4b8b-baea-6d1d4ca13dd7", "name": "network-vif-deleted", "tag": "42dfc89b-90fd-4214-983c-f02029faca17"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:16:20.467345 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-cc251b3f-d3ee-42d7-b9e0-297d4e89e3fc req-2660b66a-1407-497e-97a3-16a5f47b44c6 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:20.468094 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-cc251b3f-d3ee-42d7-b9e0-297d4e89e3fc req-2660b66a-1407-497e-97a3-16a5f47b44c6 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:20.484757 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-cc251b3f-d3ee-42d7-b9e0-297d4e89e3fc req-2660b66a-1407-497e-97a3-16a5f47b44c6 service nova] Creating event network-vif-deleted:42dfc89b-90fd-4214-983c-f02029faca17 for instance 84ab7e65-1bf0-4b8b-baea-6d1d4ca13dd7 on nch2-master-1 Jul 16 21:16:20.506411 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-cc251b3f-d3ee-42d7-b9e0-297d4e89e3fc req-2660b66a-1407-497e-97a3-16a5f47b44c6 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.064066 Jul 16 21:16:20.507129 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 903/1883] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:16:20 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 73 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:16:20.515387 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-fb89d51b-03cb-4ccd-bb08-8d3a1b09b222 tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:20.516489 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-fb89d51b-03cb-4ccd-bb08-8d3a1b09b222 tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] [instance: b9f2c527-6346-4403-841e-b7bd876a5fb7] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:20.535526 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fb89d51b-03cb-4ccd-bb08-8d3a1b09b222 tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:20.536006 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fb89d51b-03cb-4ccd-bb08-8d3a1b09b222 tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:20.539925 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fb89d51b-03cb-4ccd-bb08-8d3a1b09b222 tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:20.541097 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fb89d51b-03cb-4ccd-bb08-8d3a1b09b222 tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:20.690169 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-32aa1ea4-6903-4dad-b3db-0630e7c7dd6a tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd4144b5388b46a989a41a2f556bb508', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a90d4bcd25ff4bf68bdc194180fbbe09', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:20.699154 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-32aa1ea4-6903-4dad-b3db-0630e7c7dd6a tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] 10.222.0.16 "GET /compute/v2.1/servers/11194822-5d42-45f4-99d0-88c70c808ddd" status: 200 len: 1317 microversion: 2.1 time: 0.480355 Jul 16 21:16:20.701797 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 981/1884] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:16:20 2022] GET /compute/v2.1/servers/11194822-5d42-45f4-99d0-88c70c808ddd => generated 1317 bytes in 482 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:16:20.707319 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-d3598c86-c373-4ea1-b8ba-1d6d3f83c0fa tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Calling method '.version_select at 0x7fdda2ae0f70>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:20.771519 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-d3598c86-c373-4ea1-b8ba-1d6d3f83c0fa tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] 10.222.0.16 "GET /compute/v2.1/images/5af5e2eb-8145-46b4-abad-4d2a1e3630fc" status: 200 len: 1422 microversion: 2.1 time: 0.069866 Jul 16 21:16:20.773023 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 982/1885] 10.222.0.16 () {58 vars in 1232 bytes} [Sat Jul 16 21:16:20 2022] GET /compute/v2.1/images/5af5e2eb-8145-46b4-abad-4d2a1e3630fc => generated 1422 bytes in 72 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:16:20.812761 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-fb89d51b-03cb-4ccd-bb08-8d3a1b09b222 tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aa99be675e946fd995d71465b4cbec6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd82c2d92c67f469e863eb70bb7f9af84', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:20.823245 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-fb89d51b-03cb-4ccd-bb08-8d3a1b09b222 tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] 10.222.0.16 "GET /compute/v2.1/servers/b9f2c527-6346-4403-841e-b7bd876a5fb7" status: 200 len: 1484 microversion: 2.1 time: 0.314407 Jul 16 21:16:20.823839 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 904/1886] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:16:20 2022] GET /compute/v2.1/servers/b9f2c527-6346-4403-841e-b7bd876a5fb7 => generated 1484 bytes in 316 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:16:20.831931 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-05b50000-ce55-4ac4-8014-b3f705cdf245 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Calling method '.version_select at 0x7fdda2b8fe50>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:21.011715 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-5474d3a2-0463-4f41-b925-ffd7d3da8850 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerShowV247Test-server-260710589", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452", "networks": "none"}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:16:21.097981 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-5474d3a2-0463-4f41-b925-ffd7d3da8850 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Flavor limits 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 16 21:16:21.098412 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-5474d3a2-0463-4f41-b925-ffd7d3da8850 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Image limits 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 16 21:16:21.098883 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-5474d3a2-0463-4f41-b925-ffd7d3da8850 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Flavor pref 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 16 21:16:21.099334 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-5474d3a2-0463-4f41-b925-ffd7d3da8850 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Image pref 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 16 21:16:21.099748 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-5474d3a2-0463-4f41-b925-ffd7d3da8850 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 16 21:16:21.109654 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5474d3a2-0463-4f41-b925-ffd7d3da8850 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:21.110143 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5474d3a2-0463-4f41-b925-ffd7d3da8850 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:21.111230 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5474d3a2-0463-4f41-b925-ffd7d3da8850 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:21.111674 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5474d3a2-0463-4f41-b925-ffd7d3da8850 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:21.169755 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-5474d3a2-0463-4f41-b925-ffd7d3da8850 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Getting quotas for project 9f5f03e09936489fa1505188cefe3445. Resources: {'ram', 'instances', 'cores'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:16:21.177423 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-5474d3a2-0463-4f41-b925-ffd7d3da8850 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Getting quotas for user 015ac1bbb2df487e99e5bddc60ee6923 and project 9f5f03e09936489fa1505188cefe3445. Resources: {'ram', 'instances', 'cores'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:16:21.194729 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-5474d3a2-0463-4f41-b925-ffd7d3da8850 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Going to run 1 instances... {{(pid=138680) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 16 21:16:21.204363 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-5474d3a2-0463-4f41-b925-ffd7d3da8850 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] [instance: 0bcfc499-2e0f-48b6-994a-e0cb74ac7b4f] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138680) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 16 21:16:21.387079 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-5474d3a2-0463-4f41-b925-ffd7d3da8850 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.47 time: 0.611261 Jul 16 21:16:21.388279 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 983/1887] 10.222.0.16 () {62 vars in 1187 bytes} [Sat Jul 16 21:16:20 2022] POST /compute/v2.1/servers => generated 380 bytes in 613 msecs (HTTP/1.1 202) 10 headers in 453 bytes (1 switches on core 0) Jul 16 21:16:21.394039 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-7d0a67da-04ee-41fe-aa84-9db91c117105 tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:21.395833 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-7d0a67da-04ee-41fe-aa84-9db91c117105 tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] [instance: b9f2c527-6346-4403-841e-b7bd876a5fb7] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:21.414363 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7d0a67da-04ee-41fe-aa84-9db91c117105 tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:21.415134 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7d0a67da-04ee-41fe-aa84-9db91c117105 tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:21.415671 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7d0a67da-04ee-41fe-aa84-9db91c117105 tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:21.416118 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7d0a67da-04ee-41fe-aa84-9db91c117105 tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:21.510520 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-7d0a67da-04ee-41fe-aa84-9db91c117105 tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] 10.222.0.16 "GET /compute/v2.1/servers/b9f2c527-6346-4403-841e-b7bd876a5fb7/os-interface" status: 200 len: 285 microversion: 2.1 time: 0.120733 Jul 16 21:16:21.511158 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 984/1888] 10.222.0.16 () {58 vars in 1274 bytes} [Sat Jul 16 21:16:21 2022] GET /compute/v2.1/servers/b9f2c527-6346-4403-841e-b7bd876a5fb7/os-interface => generated 285 bytes in 122 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:16:21.523674 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-46229c38-3d5e-45a6-988c-26f677b43e9d tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:21.523674 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-46229c38-3d5e-45a6-988c-26f677b43e9d tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] [instance: f6c350c4-b1b2-42d3-ba02-65cbbacf0f58] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:21.529926 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-46229c38-3d5e-45a6-988c-26f677b43e9d tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:21.530373 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-46229c38-3d5e-45a6-988c-26f677b43e9d tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:21.533424 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-46229c38-3d5e-45a6-988c-26f677b43e9d tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:21.533956 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-46229c38-3d5e-45a6-988c-26f677b43e9d tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:21.588224 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-46229c38-3d5e-45a6-988c-26f677b43e9d tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:21.588662 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-46229c38-3d5e-45a6-988c-26f677b43e9d tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:21.588976 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.objects.instance [None req-46229c38-3d5e-45a6-988c-26f677b43e9d tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lazy-loading 'fault' on Instance uuid f6c350c4-b1b2-42d3-ba02-65cbbacf0f58 {{(pid=138680) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:16:21.603431 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:16:21.647667 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-46229c38-3d5e-45a6-988c-26f677b43e9d tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4e9df82f95ac4779859972403869096a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4337575622d04a26942905c9dc044ba7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:21.655883 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-46229c38-3d5e-45a6-988c-26f677b43e9d tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] 10.222.0.16 "GET /compute/v2.1/servers/f6c350c4-b1b2-42d3-ba02-65cbbacf0f58" status: 200 len: 1315 microversion: 2.1 time: 0.142234 Jul 16 21:16:21.657490 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 985/1889] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:16:21 2022] GET /compute/v2.1/servers/f6c350c4-b1b2-42d3-ba02-65cbbacf0f58 => generated 1315 bytes in 144 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:16:21.662341 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-dea2ebc4-06d9-4836-b749-ece08011fbde tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:21.663888 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-dea2ebc4-06d9-4836-b749-ece08011fbde tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] [instance: 84ab7e65-1bf0-4b8b-baea-6d1d4ca13dd7] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:21.670839 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-dea2ebc4-06d9-4836-b749-ece08011fbde tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:21.671329 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-dea2ebc4-06d9-4836-b749-ece08011fbde tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:21.672422 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-dea2ebc4-06d9-4836-b749-ece08011fbde tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:21.673493 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-dea2ebc4-06d9-4836-b749-ece08011fbde tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:21.716848 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-dea2ebc4-06d9-4836-b749-ece08011fbde tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:21.717290 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-dea2ebc4-06d9-4836-b749-ece08011fbde tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:21.717752 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.objects.instance [None req-dea2ebc4-06d9-4836-b749-ece08011fbde tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lazy-loading 'fault' on Instance uuid 84ab7e65-1bf0-4b8b-baea-6d1d4ca13dd7 {{(pid=138680) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:16:21.764533 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-dea2ebc4-06d9-4836-b749-ece08011fbde tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:21.773063 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-dea2ebc4-06d9-4836-b749-ece08011fbde tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:21.774577 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-dea2ebc4-06d9-4836-b749-ece08011fbde tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:21.775787 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-dea2ebc4-06d9-4836-b749-ece08011fbde tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] 10.222.0.16 "GET /compute/v2.1/servers/84ab7e65-1bf0-4b8b-baea-6d1d4ca13dd7" status: 200 len: 1271 microversion: 2.25 time: 0.117190 Jul 16 21:16:21.776352 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 986/1890] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:16:21 2022] GET /compute/v2.1/servers/84ab7e65-1bf0-4b8b-baea-6d1d4ca13dd7 => generated 1271 bytes in 118 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:16:21.781324 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-525ca4b9-fea8-4ba6-a085-64064d33a0c3 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:21.782544 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-525ca4b9-fea8-4ba6-a085-64064d33a0c3 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] [instance: 0bcfc499-2e0f-48b6-994a-e0cb74ac7b4f] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:21.846916 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-525ca4b9-fea8-4ba6-a085-64064d33a0c3 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '015ac1bbb2df487e99e5bddc60ee6923', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9f5f03e09936489fa1505188cefe3445', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:21.854613 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-525ca4b9-fea8-4ba6-a085-64064d33a0c3 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '015ac1bbb2df487e99e5bddc60ee6923', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9f5f03e09936489fa1505188cefe3445', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:21.856711 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-525ca4b9-fea8-4ba6-a085-64064d33a0c3 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '015ac1bbb2df487e99e5bddc60ee6923', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9f5f03e09936489fa1505188cefe3445', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:21.857889 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-525ca4b9-fea8-4ba6-a085-64064d33a0c3 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] 10.222.0.16 "GET /compute/v2.1/servers/0bcfc499-2e0f-48b6-994a-e0cb74ac7b4f" status: 200 len: 1277 microversion: 2.47 time: 0.080325 Jul 16 21:16:21.858723 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 987/1891] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:16:21 2022] GET /compute/v2.1/servers/0bcfc499-2e0f-48b6-994a-e0cb74ac7b4f => generated 1277 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:16:21.871913 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-a4992942-9191-4b01-879c-74da092af1a9 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:21.878758 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-a4992942-9191-4b01-879c-74da092af1a9 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] [instance: 11194822-5d42-45f4-99d0-88c70c808ddd] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:21.919395 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a4992942-9191-4b01-879c-74da092af1a9 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:21.921899 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a4992942-9191-4b01-879c-74da092af1a9 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:21.928708 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a4992942-9191-4b01-879c-74da092af1a9 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:21.935116 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a4992942-9191-4b01-879c-74da092af1a9 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:22.159883 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-a4992942-9191-4b01-879c-74da092af1a9 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd4144b5388b46a989a41a2f556bb508', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a90d4bcd25ff4bf68bdc194180fbbe09', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:22.170661 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-a4992942-9191-4b01-879c-74da092af1a9 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] 10.222.0.16 "GET /compute/v2.1/servers/11194822-5d42-45f4-99d0-88c70c808ddd" status: 200 len: 1506 microversion: 2.1 time: 0.309292 Jul 16 21:16:22.171483 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 988/1892] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:16:21 2022] GET /compute/v2.1/servers/11194822-5d42-45f4-99d0-88c70c808ddd => generated 1506 bytes in 312 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:16:22.180492 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-23d8b976-5108-4178-a9a8-11f83a88c05d tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerShowV247Test-server-864622990", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452", "networks": "none"}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:16:22.269785 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-23d8b976-5108-4178-a9a8-11f83a88c05d tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Flavor limits 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 16 21:16:22.270999 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-23d8b976-5108-4178-a9a8-11f83a88c05d tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Image limits 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 16 21:16:22.271892 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-23d8b976-5108-4178-a9a8-11f83a88c05d tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Flavor pref 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 16 21:16:22.272928 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-23d8b976-5108-4178-a9a8-11f83a88c05d tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Image pref 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 16 21:16:22.273995 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-23d8b976-5108-4178-a9a8-11f83a88c05d tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 16 21:16:22.296690 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-23d8b976-5108-4178-a9a8-11f83a88c05d tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:22.297565 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-23d8b976-5108-4178-a9a8-11f83a88c05d tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.011s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:22.299751 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-23d8b976-5108-4178-a9a8-11f83a88c05d tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:22.300466 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-23d8b976-5108-4178-a9a8-11f83a88c05d tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:22.324809 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-23d8b976-5108-4178-a9a8-11f83a88c05d tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Getting quotas for project 9f5f03e09936489fa1505188cefe3445. Resources: {'ram', 'instances', 'cores'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:16:22.332073 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-23d8b976-5108-4178-a9a8-11f83a88c05d tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Getting quotas for user 015ac1bbb2df487e99e5bddc60ee6923 and project 9f5f03e09936489fa1505188cefe3445. Resources: {'ram', 'instances', 'cores'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:16:22.350606 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-23d8b976-5108-4178-a9a8-11f83a88c05d tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Going to run 1 instances... {{(pid=138680) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 16 21:16:22.353643 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-23d8b976-5108-4178-a9a8-11f83a88c05d tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] [instance: 9657dddc-2416-4c95-aab3-d6b6f9673369] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138680) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 16 21:16:22.548225 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-23d8b976-5108-4178-a9a8-11f83a88c05d tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.47 time: 0.374372 Jul 16 21:16:22.549076 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 989/1893] 10.222.0.16 () {62 vars in 1187 bytes} [Sat Jul 16 21:16:22 2022] POST /compute/v2.1/servers => generated 380 bytes in 376 msecs (HTTP/1.1 202) 10 headers in 453 bytes (1 switches on core 0) Jul 16 21:16:22.561551 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-dd5b92df-08a1-461b-99ba-f4aa82325e8c tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:22.563632 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-dd5b92df-08a1-461b-99ba-f4aa82325e8c tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] [instance: 11194822-5d42-45f4-99d0-88c70c808ddd] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:22.571496 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-dd5b92df-08a1-461b-99ba-f4aa82325e8c tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:22.571790 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-dd5b92df-08a1-461b-99ba-f4aa82325e8c tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:22.573024 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-dd5b92df-08a1-461b-99ba-f4aa82325e8c tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:22.573562 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-dd5b92df-08a1-461b-99ba-f4aa82325e8c tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:22.683327 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-05b50000-ce55-4ac4-8014-b3f705cdf245 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] 10.222.0.16 "DELETE /compute/v2.1/images/5af5e2eb-8145-46b4-abad-4d2a1e3630fc" status: 204 len: 0 microversion: 2.1 time: 1.854850 Jul 16 21:16:22.683327 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 905/1894] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:16:20 2022] DELETE /compute/v2.1/images/5af5e2eb-8145-46b4-abad-4d2a1e3630fc => generated 0 bytes in 1857 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 16 21:16:22.687722 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-6762f6b7-49c3-4a04-aca2-46e5c71800f3 req-ec3c64a4-a8c0-4a7c-988d-b08352c9b64f service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "f6c350c4-b1b2-42d3-ba02-65cbbacf0f58", "name": "network-vif-deleted", "tag": "d0843d7d-ff0d-4603-aeeb-d26f6ebde7db"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:16:22.715092 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-6762f6b7-49c3-4a04-aca2-46e5c71800f3 req-ec3c64a4-a8c0-4a7c-988d-b08352c9b64f service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:22.715486 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-6762f6b7-49c3-4a04-aca2-46e5c71800f3 req-ec3c64a4-a8c0-4a7c-988d-b08352c9b64f service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:22.747903 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-6762f6b7-49c3-4a04-aca2-46e5c71800f3 req-ec3c64a4-a8c0-4a7c-988d-b08352c9b64f service nova] Creating event network-vif-deleted:d0843d7d-ff0d-4603-aeeb-d26f6ebde7db for instance f6c350c4-b1b2-42d3-ba02-65cbbacf0f58 on nch1-master-1 Jul 16 21:16:22.772483 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-6762f6b7-49c3-4a04-aca2-46e5c71800f3 req-ec3c64a4-a8c0-4a7c-988d-b08352c9b64f service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.079633 Jul 16 21:16:22.772483 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 906/1895] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:16:22 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:16:22.775815 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-183cc04a-7097-441b-b344-b5af4f748f46 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Action: 'update', calling method: >, body: {"server": {}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:16:22.778317 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-183cc04a-7097-441b-b344-b5af4f748f46 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] [instance: 9657dddc-2416-4c95-aab3-d6b6f9673369] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:22.826011 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_api:SELECT=96,nova_api:INSERT=3,nova_api:UPDATE=1 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:16:22.855415 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-183cc04a-7097-441b-b344-b5af4f748f46 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] 10.222.0.16 "PUT /compute/v2.1/servers/9657dddc-2416-4c95-aab3-d6b6f9673369" status: 200 len: 991 microversion: 2.47 time: 0.086204 Jul 16 21:16:22.856381 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 907/1896] 10.222.0.16 () {62 vars in 1296 bytes} [Sat Jul 16 21:16:22 2022] PUT /compute/v2.1/servers/9657dddc-2416-4c95-aab3-d6b6f9673369 => generated 991 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 16 21:16:22.897654 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-8fafda5d-0f96-41b6-8934-930b95f564f6 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:22.898320 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-8fafda5d-0f96-41b6-8934-930b95f564f6 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] [instance: f6c350c4-b1b2-42d3-ba02-65cbbacf0f58] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:22.908244 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8fafda5d-0f96-41b6-8934-930b95f564f6 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:22.909049 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8fafda5d-0f96-41b6-8934-930b95f564f6 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:22.910200 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8fafda5d-0f96-41b6-8934-930b95f564f6 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:22.910962 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8fafda5d-0f96-41b6-8934-930b95f564f6 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:22.938798 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-dd5b92df-08a1-461b-99ba-f4aa82325e8c tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd4144b5388b46a989a41a2f556bb508', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a90d4bcd25ff4bf68bdc194180fbbe09', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:22.947767 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-dd5b92df-08a1-461b-99ba-f4aa82325e8c tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] 10.222.0.16 "GET /compute/v2.1/servers/11194822-5d42-45f4-99d0-88c70c808ddd" status: 200 len: 1506 microversion: 2.1 time: 0.391094 Jul 16 21:16:22.948615 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 990/1897] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:16:22 2022] GET /compute/v2.1/servers/11194822-5d42-45f4-99d0-88c70c808ddd => generated 1506 bytes in 399 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:16:22.959731 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-8fafda5d-0f96-41b6-8934-930b95f564f6 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] HTTP exception thrown: Instance f6c350c4-b1b2-42d3-ba02-65cbbacf0f58 could not be found. Jul 16 21:16:22.962137 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-8fafda5d-0f96-41b6-8934-930b95f564f6 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Request method failure captured: Jul 16 21:16:22.962137 ncdv-master-1 devstack@n-api.service[138681]: request: GET /compute/v2.1/servers/f6c350c4-b1b2-42d3-ba02-65cbbacf0f58 HTTP/1.1 Jul 16 21:16:22.962137 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:16:22.962137 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:16:22.962137 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:16:22.962137 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 21:16:22.962137 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:16:22.962137 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:16:22.962137 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:16:22.962137 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:16:22.962137 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0yoJFUAf_8Wtq8fI-wZdJwoKyg8rUEOC_eBMeqcRE2hk6JM7_ssDplugQwmk2S9xH7Oys7Ho2tw0kc9hmlY-jX_rR-I-9WVwQYFbyogU7ki99siEGGR9aVA2GKsp5nNcVHXBtjM3bI_zfa21XSWedvaSIVwZV8kEGixKYsBf2GMYKVI Jul 16 21:16:22.962137 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:16:22.962137 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:16:22.962137 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:16:22.962137 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:16:22.962137 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:16:22.962137 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:16:22.962137 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: 4337575622d04a26942905c9dc044ba7 Jul 16 21:16:22.962137 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-ImagesOneServerNegativeTestJSON-526509748 Jul 16 21:16:22.962137 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:16:22.962137 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:16:22.962137 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/4337575622d04a26942905c9dc044ba7"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/4337575622d04a26942905c9dc044ba7"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_4337575622d04a26942905c9dc044ba7"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/4337575622d04a26942905c9dc044ba7"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:16:22.962137 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-ImagesOneServerNegativeTestJSON-526509748 Jul 16 21:16:22.962137 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: 4337575622d04a26942905c9dc044ba7 Jul 16 21:16:22.962137 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-ImagesOneServerNegativeTestJSON-526509748 Jul 16 21:16:22.962137 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-ImagesOneServerNegativeTestJSON-526509748-project Jul 16 21:16:22.962137 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:16:22.962137 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:16:22.962137 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: 4e9df82f95ac4779859972403869096a Jul 16 21:16:22.962137 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-ImagesOneServerNegativeTestJSON-526509748-project Jul 16 21:16:22.962137 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:16:22.962137 ncdv-master-1 devstack@n-api.service[138681]: exception: Instance f6c350c4-b1b2-42d3-ba02-65cbbacf0f58 could not be found. Jul 16 21:16:22.962137 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': 'f6c350c4-b1b2-42d3-ba02-65cbbacf0f58'} Jul 16 21:16:22.962137 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:16:22.962137 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:16:22.962137 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:16:22.962137 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:16:22.962137 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:16:22.962137 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:16:22.962137 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:16:22.962137 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:16:22.962137 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:16:22.962137 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:16:22.962137 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance f6c350c4-b1b2-42d3-ba02-65cbbacf0f58 could not be found. Jul 16 21:16:22.962137 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:16:22.962137 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:16:22.962137 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:16:22.962137 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:16:22.962137 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:16:22.962137 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:16:22.962137 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:16:22.962137 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:16:22.962137 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:16:22.962137 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:16:22.962137 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:16:22.962137 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:16:22.962137 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:16:22.962137 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:16:22.962137 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:16:22.962137 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:16:22.962137 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance f6c350c4-b1b2-42d3-ba02-65cbbacf0f58 could not be found. Jul 16 21:16:22.962137 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:16:22.962137 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:16:22.969353 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:16:22.969353 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:16:22.969353 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:16:22.969353 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:16:22.969353 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:16:22.969353 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:16:22.969353 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance f6c350c4-b1b2-42d3-ba02-65cbbacf0f58 could not be found. Jul 16 21:16:22.969353 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:16:22.969353 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-8fafda5d-0f96-41b6-8934-930b95f564f6 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Returning 404 to user: Instance f6c350c4-b1b2-42d3-ba02-65cbbacf0f58 could not be found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:16:22.969353 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-8fafda5d-0f96-41b6-8934-930b95f564f6 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] 10.222.0.16 "GET /compute/v2.1/servers/f6c350c4-b1b2-42d3-ba02-65cbbacf0f58" status: 404 len: 111 microversion: 2.1 time: 0.107144 Jul 16 21:16:22.969353 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 908/1898] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:16:22 2022] GET /compute/v2.1/servers/f6c350c4-b1b2-42d3-ba02-65cbbacf0f58 => generated 111 bytes in 109 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:16:22.972462 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-78432787-2fee-4fa8-ad28-6be094e6a345 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:22.973112 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-78432787-2fee-4fa8-ad28-6be094e6a345 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] [instance: 84ab7e65-1bf0-4b8b-baea-6d1d4ca13dd7] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:22.974640 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-aedfa170-264d-4ca5-bf01-917979577073 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:22.975479 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-aedfa170-264d-4ca5-bf01-917979577073 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] [instance: 93e327c3-c63d-439a-9b25-1a805f785949] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:22.981729 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-78432787-2fee-4fa8-ad28-6be094e6a345 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:22.982247 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-78432787-2fee-4fa8-ad28-6be094e6a345 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:22.983086 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-78432787-2fee-4fa8-ad28-6be094e6a345 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:22.983539 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-78432787-2fee-4fa8-ad28-6be094e6a345 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:22.983852 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-aedfa170-264d-4ca5-bf01-917979577073 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:22.984505 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-aedfa170-264d-4ca5-bf01-917979577073 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:22.986175 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-aedfa170-264d-4ca5-bf01-917979577073 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:22.987313 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-aedfa170-264d-4ca5-bf01-917979577073 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:23.026661 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-78432787-2fee-4fa8-ad28-6be094e6a345 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] HTTP exception thrown: Instance 84ab7e65-1bf0-4b8b-baea-6d1d4ca13dd7 could not be found. Jul 16 21:16:23.031305 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-78432787-2fee-4fa8-ad28-6be094e6a345 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Request method failure captured: Jul 16 21:16:23.031305 ncdv-master-1 devstack@n-api.service[138681]: request: GET /compute/v2.1/servers/84ab7e65-1bf0-4b8b-baea-6d1d4ca13dd7 HTTP/1.1 Jul 16 21:16:23.031305 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:16:23.031305 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:16:23.031305 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:16:23.031305 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 21:16:23.031305 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:16:23.031305 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:16:23.031305 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:16:23.031305 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:16:23.031305 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0ylBOHnbwqerAHhqONpFSBMhwk4nWfHPAHc-OTgMvb17y3jQh2-2K7JnAKTp7DvWJ_uO-bYr8xJfvvOUSkF97J9lR0DMJJ8o4htp9CLMoM61BjIIA0xJtMBeTA8oUgOHWMLAymuEaOlZbY4tC3l04tVeZqVEgsWsWeWmyuuhOz7GDcE Jul 16 21:16:23.031305 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:16:23.031305 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:16:23.031305 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:16:23.031305 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:16:23.031305 ncdv-master-1 devstack@n-api.service[138681]: X-Openstack-Nova-Api-Version: 2.25 Jul 16 21:16:23.031305 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:16:23.031305 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:16:23.031305 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: d9b92fc544c544f686ac2ab7ff5123b6 Jul 16 21:16:23.031305 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-LiveAutoBlockMigrationV225Test-1461646193 Jul 16 21:16:23.031305 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:16:23.031305 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:16:23.031305 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/d9b92fc544c544f686ac2ab7ff5123b6"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/d9b92fc544c544f686ac2ab7ff5123b6"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_d9b92fc544c544f686ac2ab7ff5123b6"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/d9b92fc544c544f686ac2ab7ff5123b6"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:16:23.031305 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-LiveAutoBlockMigrationV225Test-1461646193 Jul 16 21:16:23.031305 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: d9b92fc544c544f686ac2ab7ff5123b6 Jul 16 21:16:23.031305 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-LiveAutoBlockMigrationV225Test-1461646193 Jul 16 21:16:23.031305 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-LiveAutoBlockMigrationV225Test-1461646193-project Jul 16 21:16:23.031305 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:16:23.031305 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:16:23.031305 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: 578b87f863b34f57a571f7e56d44f118 Jul 16 21:16:23.031305 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-LiveAutoBlockMigrationV225Test-1461646193-project Jul 16 21:16:23.031305 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:16:23.031305 ncdv-master-1 devstack@n-api.service[138681]: exception: Instance 84ab7e65-1bf0-4b8b-baea-6d1d4ca13dd7 could not be found. Jul 16 21:16:23.031305 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': '84ab7e65-1bf0-4b8b-baea-6d1d4ca13dd7'} Jul 16 21:16:23.031305 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:16:23.031305 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:16:23.031305 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:16:23.031305 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:16:23.031305 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:16:23.031305 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:16:23.031305 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:16:23.031305 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:16:23.031305 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:16:23.031305 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:16:23.031305 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 84ab7e65-1bf0-4b8b-baea-6d1d4ca13dd7 could not be found. Jul 16 21:16:23.031305 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:16:23.031305 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:16:23.031305 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:16:23.031305 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:16:23.031305 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:16:23.031305 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:16:23.031305 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:16:23.031305 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:16:23.031305 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:16:23.031305 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:16:23.031305 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:16:23.031305 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:16:23.031305 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:16:23.031305 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:16:23.031305 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:16:23.031305 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:16:23.031305 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 84ab7e65-1bf0-4b8b-baea-6d1d4ca13dd7 could not be found. Jul 16 21:16:23.031305 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:16:23.040009 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:16:23.040009 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:16:23.040009 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:16:23.040009 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:16:23.040009 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:16:23.040009 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:16:23.040009 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:16:23.040009 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 84ab7e65-1bf0-4b8b-baea-6d1d4ca13dd7 could not be found. Jul 16 21:16:23.040009 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:16:23.040009 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-78432787-2fee-4fa8-ad28-6be094e6a345 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Returning 404 to user: Instance 84ab7e65-1bf0-4b8b-baea-6d1d4ca13dd7 could not be found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:16:23.040009 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-78432787-2fee-4fa8-ad28-6be094e6a345 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] 10.222.0.16 "GET /compute/v2.1/servers/84ab7e65-1bf0-4b8b-baea-6d1d4ca13dd7" status: 404 len: 111 microversion: 2.25 time: 0.068924 Jul 16 21:16:23.040009 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 909/1899] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:16:22 2022] GET /compute/v2.1/servers/84ab7e65-1bf0-4b8b-baea-6d1d4ca13dd7 => generated 111 bytes in 72 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Jul 16 21:16:23.054016 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-aedfa170-264d-4ca5-bf01-917979577073 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] [instance: 93e327c3-c63d-439a-9b25-1a805f785949] Going to try to terminate instance {{(pid=138680) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 16 21:16:23.054815 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-6495fdea-1c8a-4e87-bff9-ef6f6a81427a tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:23.061105 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-6495fdea-1c8a-4e87-bff9-ef6f6a81427a tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] [instance: 9657dddc-2416-4c95-aab3-d6b6f9673369] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:23.140353 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-6495fdea-1c8a-4e87-bff9-ef6f6a81427a tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '015ac1bbb2df487e99e5bddc60ee6923', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9f5f03e09936489fa1505188cefe3445', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:23.150717 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-6495fdea-1c8a-4e87-bff9-ef6f6a81427a tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '015ac1bbb2df487e99e5bddc60ee6923', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9f5f03e09936489fa1505188cefe3445', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:23.152409 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-6495fdea-1c8a-4e87-bff9-ef6f6a81427a tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '015ac1bbb2df487e99e5bddc60ee6923', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9f5f03e09936489fa1505188cefe3445', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:23.154110 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-6495fdea-1c8a-4e87-bff9-ef6f6a81427a tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] 10.222.0.16 "GET /compute/v2.1/servers/9657dddc-2416-4c95-aab3-d6b6f9673369" status: 200 len: 1277 microversion: 2.47 time: 0.112280 Jul 16 21:16:23.155248 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 910/1900] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:16:23 2022] GET /compute/v2.1/servers/9657dddc-2416-4c95-aab3-d6b6f9673369 => generated 1277 bytes in 115 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:16:23.164630 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-1df90ee3-57e5-4406-9124-1d9cfc97c87c tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Action: 'action', calling method: >, body: {"rebuild": {"imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "OS-DCF:diskConfig": "AUTO"}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:16:23.166429 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-1df90ee3-57e5-4406-9124-1d9cfc97c87c tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] [instance: 11194822-5d42-45f4-99d0-88c70c808ddd] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:23.175000 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1df90ee3-57e5-4406-9124-1d9cfc97c87c tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:23.175573 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1df90ee3-57e5-4406-9124-1d9cfc97c87c tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:23.176433 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1df90ee3-57e5-4406-9124-1d9cfc97c87c tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:23.176942 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1df90ee3-57e5-4406-9124-1d9cfc97c87c tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:23.321700 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-1df90ee3-57e5-4406-9124-1d9cfc97c87c tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Flavor limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 16 21:16:23.321700 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-1df90ee3-57e5-4406-9124-1d9cfc97c87c tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Image limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 16 21:16:23.321700 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-1df90ee3-57e5-4406-9124-1d9cfc97c87c tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Flavor pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 16 21:16:23.321700 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-1df90ee3-57e5-4406-9124-1d9cfc97c87c tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Image pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 16 21:16:23.321700 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-1df90ee3-57e5-4406-9124-1d9cfc97c87c tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 16 21:16:23.486802 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-aedfa170-264d-4ca5-bf01-917979577073 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] 10.222.0.16 "DELETE /compute/v2.1/servers/93e327c3-c63d-439a-9b25-1a805f785949" status: 204 len: 0 microversion: 2.1 time: 0.518338 Jul 16 21:16:23.487901 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 991/1901] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:16:22 2022] DELETE /compute/v2.1/servers/93e327c3-c63d-439a-9b25-1a805f785949 => generated 0 bytes in 526 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 16 21:16:23.504248 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-19be8796-3ffd-48fe-8b87-c9de51084cd4 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Calling method '.version_select at 0x7fdda2b3fee0>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:23.543666 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-19be8796-3ffd-48fe-8b87-c9de51084cd4 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] HTTP exception thrown: Image not found. Jul 16 21:16:23.546039 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-19be8796-3ffd-48fe-8b87-c9de51084cd4 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Request method failure captured: Jul 16 21:16:23.546039 ncdv-master-1 devstack@n-api.service[138680]: request: DELETE /compute/v2.1/images/23300f39-e85b-4c2f-ad31-2d2a76ad6f94 HTTP/1.1 Jul 16 21:16:23.546039 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:16:23.546039 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:16:23.546039 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:16:23.546039 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 0 Jul 16 21:16:23.546039 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:16:23.546039 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:16:23.546039 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:16:23.546039 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:16:23.546039 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0yoJFUAf_8Wtq8fI-wZdJwoKyg8rUEOC_eBMeqcRE2hk6JM7_ssDplugQwmk2S9xH7Oys7Ho2tw0kc9hmlY-jX_rR-I-9WVwQYFbyogU7ki99siEGGR9aVA2GKsp5nNcVHXBtjM3bI_zfa21XSWedvaSIVwZV8kEGixKYsBf2GMYKVI Jul 16 21:16:23.546039 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:16:23.546039 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:16:23.546039 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:16:23.546039 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:16:23.546039 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:16:23.546039 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:16:23.546039 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: 4337575622d04a26942905c9dc044ba7 Jul 16 21:16:23.546039 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-ImagesOneServerNegativeTestJSON-526509748 Jul 16 21:16:23.546039 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:16:23.546039 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:16:23.546039 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/4337575622d04a26942905c9dc044ba7"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/4337575622d04a26942905c9dc044ba7"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_4337575622d04a26942905c9dc044ba7"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/4337575622d04a26942905c9dc044ba7"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:16:23.546039 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-ImagesOneServerNegativeTestJSON-526509748 Jul 16 21:16:23.546039 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: 4337575622d04a26942905c9dc044ba7 Jul 16 21:16:23.546039 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-ImagesOneServerNegativeTestJSON-526509748 Jul 16 21:16:23.546039 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-ImagesOneServerNegativeTestJSON-526509748-project Jul 16 21:16:23.546039 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:16:23.546039 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:16:23.546039 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: 4e9df82f95ac4779859972403869096a Jul 16 21:16:23.546039 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-ImagesOneServerNegativeTestJSON-526509748-project Jul 16 21:16:23.546039 ncdv-master-1 devstack@n-api.service[138680]: method: Jul 16 21:16:23.546039 ncdv-master-1 devstack@n-api.service[138680]: exception: Image not found. Jul 16 21:16:23.546039 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': '23300f39-e85b-4c2f-ad31-2d2a76ad6f94'} Jul 16 21:16:23.546039 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:16:23.546039 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:16:23.546039 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 724, in delete Jul 16 21:16:23.546039 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi self._client.call(context, 2, 'delete', args=(image_id,)) Jul 16 21:16:23.546039 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 191, in call Jul 16 21:16:23.546039 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi result = getattr(controller, method)(*args, **kwargs) Jul 16 21:16:23.546039 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/utils.py", line 670, in inner Jul 16 21:16:23.546039 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return RequestIdProxy(wrapped(*args, **kwargs)) Jul 16 21:16:23.546039 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/v2/images.py", line 386, in delete Jul 16 21:16:23.546039 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi resp, body = self.http_client.delete(url) Jul 16 21:16:23.546039 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/adapter.py", line 410, in delete Jul 16 21:16:23.546039 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return self.request(url, 'DELETE', **kwargs) Jul 16 21:16:23.546039 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 380, in request Jul 16 21:16:23.546039 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return self._handle_response(resp) Jul 16 21:16:23.546039 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 120, in _handle_response Jul 16 21:16:23.546039 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.from_response(resp, resp.content) Jul 16 21:16:23.546039 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi glanceclient.exc.HTTPNotFound: HTTP 404 Not Found: Failed to find image 23300f39-e85b-4c2f-ad31-2d2a76ad6f94 to delete Jul 16 21:16:23.546039 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:16:23.546039 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:16:23.546039 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:16:23.546039 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:16:23.546039 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/images.py", line 104, in delete Jul 16 21:16:23.546039 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi self._image_api.delete(context, id) Jul 16 21:16:23.546039 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 1255, in delete Jul 16 21:16:23.546039 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return session.delete(context, image_id) Jul 16 21:16:23.546039 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 726, in delete Jul 16 21:16:23.546039 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.ImageNotFound(image_id=image_id) Jul 16 21:16:23.546039 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.ImageNotFound: Image 23300f39-e85b-4c2f-ad31-2d2a76ad6f94 could not be found. Jul 16 21:16:23.546039 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:16:23.546039 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:16:23.546039 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:16:23.552199 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:16:23.552199 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:16:23.552199 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:16:23.552199 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:16:23.552199 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:16:23.552199 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Jul 16 21:16:23.552199 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Jul 16 21:16:23.552199 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:16:23.552199 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:16:23.552199 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/images.py", line 107, in delete Jul 16 21:16:23.552199 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=explanation) Jul 16 21:16:23.552199 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Image not found. Jul 16 21:16:23.552199 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:16:23.552199 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:16:23.552199 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:16:23.552199 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:16:23.552199 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:16:23.552199 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:16:23.552199 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:16:23.552199 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:16:23.552199 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Image not found. Jul 16 21:16:23.552199 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:16:23.552199 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-19be8796-3ffd-48fe-8b87-c9de51084cd4 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Returning 404 to user: Image not found. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:16:23.552199 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-19be8796-3ffd-48fe-8b87-c9de51084cd4 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] 10.222.0.16 "DELETE /compute/v2.1/images/23300f39-e85b-4c2f-ad31-2d2a76ad6f94" status: 404 len: 62 microversion: 2.1 time: 0.058711 Jul 16 21:16:23.557820 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 992/1902] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:16:23 2022] DELETE /compute/v2.1/images/23300f39-e85b-4c2f-ad31-2d2a76ad6f94 => generated 62 bytes in 66 msecs (HTTP/1.1 404) 9 headers in 378 bytes (1 switches on core 0) Jul 16 21:16:23.567133 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-8468ad6a-9ecb-46c9-96ec-301778c78c97 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:23.567761 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-8468ad6a-9ecb-46c9-96ec-301778c78c97 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] [instance: 39e51d01-21e0-4dd5-bfd4-16b0f8b0f827] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:23.593981 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_api:SELECT=92,nova_api:UPDATE=2,nova_api:INSERT=6 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:16:23.597343 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8468ad6a-9ecb-46c9-96ec-301778c78c97 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:23.598162 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8468ad6a-9ecb-46c9-96ec-301778c78c97 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:23.599533 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8468ad6a-9ecb-46c9-96ec-301778c78c97 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:23.600914 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8468ad6a-9ecb-46c9-96ec-301778c78c97 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:23.623435 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-1df90ee3-57e5-4406-9124-1d9cfc97c87c tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] [instance: 11194822-5d42-45f4-99d0-88c70c808ddd] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:23.638666 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-8468ad6a-9ecb-46c9-96ec-301778c78c97 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] HTTP exception thrown: Instance 39e51d01-21e0-4dd5-bfd4-16b0f8b0f827 could not be found. Jul 16 21:16:23.639806 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1df90ee3-57e5-4406-9124-1d9cfc97c87c tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:23.642422 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1df90ee3-57e5-4406-9124-1d9cfc97c87c tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:23.643513 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1df90ee3-57e5-4406-9124-1d9cfc97c87c tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:23.644151 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-8468ad6a-9ecb-46c9-96ec-301778c78c97 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Request method failure captured: Jul 16 21:16:23.644151 ncdv-master-1 devstack@n-api.service[138680]: request: DELETE /compute/v2.1/servers/39e51d01-21e0-4dd5-bfd4-16b0f8b0f827 HTTP/1.1 Jul 16 21:16:23.644151 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:16:23.644151 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:16:23.644151 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:16:23.644151 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 0 Jul 16 21:16:23.644151 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:16:23.644151 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:16:23.644151 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:16:23.644151 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:16:23.644151 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0yoJFUAf_8Wtq8fI-wZdJwoKyg8rUEOC_eBMeqcRE2hk6JM7_ssDplugQwmk2S9xH7Oys7Ho2tw0kc9hmlY-jX_rR-I-9WVwQYFbyogU7ki99siEGGR9aVA2GKsp5nNcVHXBtjM3bI_zfa21XSWedvaSIVwZV8kEGixKYsBf2GMYKVI Jul 16 21:16:23.644151 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:16:23.644151 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:16:23.644151 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:16:23.644151 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:16:23.644151 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:16:23.644151 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:16:23.644151 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: 4337575622d04a26942905c9dc044ba7 Jul 16 21:16:23.644151 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-ImagesOneServerNegativeTestJSON-526509748 Jul 16 21:16:23.644151 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:16:23.644151 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:16:23.644151 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/4337575622d04a26942905c9dc044ba7"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/4337575622d04a26942905c9dc044ba7"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_4337575622d04a26942905c9dc044ba7"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/4337575622d04a26942905c9dc044ba7"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:16:23.644151 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-ImagesOneServerNegativeTestJSON-526509748 Jul 16 21:16:23.644151 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: 4337575622d04a26942905c9dc044ba7 Jul 16 21:16:23.644151 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-ImagesOneServerNegativeTestJSON-526509748 Jul 16 21:16:23.644151 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-ImagesOneServerNegativeTestJSON-526509748-project Jul 16 21:16:23.644151 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:16:23.644151 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:16:23.644151 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: 4e9df82f95ac4779859972403869096a Jul 16 21:16:23.644151 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-ImagesOneServerNegativeTestJSON-526509748-project Jul 16 21:16:23.644151 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 21:16:23.644151 ncdv-master-1 devstack@n-api.service[138680]: exception: Instance 39e51d01-21e0-4dd5-bfd4-16b0f8b0f827 could not be found. Jul 16 21:16:23.644151 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': '39e51d01-21e0-4dd5-bfd4-16b0f8b0f827'} Jul 16 21:16:23.644151 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:16:23.644151 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:16:23.644151 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:16:23.644151 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:16:23.644151 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:16:23.644151 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:16:23.644151 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:16:23.644151 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:16:23.644151 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:16:23.644151 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:16:23.644151 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 39e51d01-21e0-4dd5-bfd4-16b0f8b0f827 could not be found. Jul 16 21:16:23.644151 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:16:23.644151 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:16:23.644151 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:16:23.644151 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:16:23.644151 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:16:23.644151 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:16:23.644151 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:16:23.644151 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:16:23.644151 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:16:23.644151 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:16:23.644151 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Jul 16 21:16:23.644151 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Jul 16 21:16:23.644151 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Jul 16 21:16:23.644151 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Jul 16 21:16:23.644151 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:16:23.644151 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:16:23.644151 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:16:23.644151 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:16:23.648990 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 39e51d01-21e0-4dd5-bfd4-16b0f8b0f827 could not be found. Jul 16 21:16:23.648990 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:16:23.648990 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:16:23.648990 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:16:23.648990 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:16:23.648990 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:16:23.648990 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:16:23.648990 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:16:23.648990 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:16:23.648990 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 39e51d01-21e0-4dd5-bfd4-16b0f8b0f827 could not be found. Jul 16 21:16:23.648990 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:16:23.649850 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1df90ee3-57e5-4406-9124-1d9cfc97c87c tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:23.652849 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-8468ad6a-9ecb-46c9-96ec-301778c78c97 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Returning 404 to user: Instance 39e51d01-21e0-4dd5-bfd4-16b0f8b0f827 could not be found. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:16:23.654276 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-8468ad6a-9ecb-46c9-96ec-301778c78c97 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] 10.222.0.16 "DELETE /compute/v2.1/servers/39e51d01-21e0-4dd5-bfd4-16b0f8b0f827" status: 404 len: 111 microversion: 2.1 time: 0.091168 Jul 16 21:16:23.664103 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 993/1903] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:16:23 2022] DELETE /compute/v2.1/servers/39e51d01-21e0-4dd5-bfd4-16b0f8b0f827 => generated 111 bytes in 103 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:16:23.665613 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-63d89066-238b-413d-a7eb-d531fcbd508d tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:23.666394 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-63d89066-238b-413d-a7eb-d531fcbd508d tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] [instance: b9f2c527-6346-4403-841e-b7bd876a5fb7] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:23.674415 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-63d89066-238b-413d-a7eb-d531fcbd508d tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:23.675375 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-63d89066-238b-413d-a7eb-d531fcbd508d tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:23.676327 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-63d89066-238b-413d-a7eb-d531fcbd508d tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:23.677203 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-63d89066-238b-413d-a7eb-d531fcbd508d tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:23.712241 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-1df90ee3-57e5-4406-9124-1d9cfc97c87c tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] 10.222.0.16 "POST /compute/v2.1/servers/11194822-5d42-45f4-99d0-88c70c808ddd/action" status: 202 len: 1108 microversion: 2.1 time: 0.553499 Jul 16 21:16:23.714199 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 911/1904] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 21:16:23 2022] POST /compute/v2.1/servers/11194822-5d42-45f4-99d0-88c70c808ddd/action => generated 1108 bytes in 557 msecs (HTTP/1.1 202) 10 headers in 452 bytes (1 switches on core 0) Jul 16 21:16:23.723173 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-4dbc39d3-cf2c-4c53-ae9a-093c82d3668d tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:23.723173 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-4dbc39d3-cf2c-4c53-ae9a-093c82d3668d tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] [instance: 39e51d01-21e0-4dd5-bfd4-16b0f8b0f827] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:23.746329 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4dbc39d3-cf2c-4c53-ae9a-093c82d3668d tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:23.746794 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4dbc39d3-cf2c-4c53-ae9a-093c82d3668d tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:23.747655 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4dbc39d3-cf2c-4c53-ae9a-093c82d3668d tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:23.748104 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4dbc39d3-cf2c-4c53-ae9a-093c82d3668d tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:23.784925 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-4dbc39d3-cf2c-4c53-ae9a-093c82d3668d tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] HTTP exception thrown: Instance 39e51d01-21e0-4dd5-bfd4-16b0f8b0f827 could not be found. Jul 16 21:16:23.786907 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-4dbc39d3-cf2c-4c53-ae9a-093c82d3668d tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Request method failure captured: Jul 16 21:16:23.786907 ncdv-master-1 devstack@n-api.service[138681]: request: GET /compute/v2.1/servers/39e51d01-21e0-4dd5-bfd4-16b0f8b0f827 HTTP/1.1 Jul 16 21:16:23.786907 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:16:23.786907 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:16:23.786907 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:16:23.786907 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 21:16:23.786907 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:16:23.786907 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:16:23.786907 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:16:23.786907 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:16:23.786907 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0yoJFUAf_8Wtq8fI-wZdJwoKyg8rUEOC_eBMeqcRE2hk6JM7_ssDplugQwmk2S9xH7Oys7Ho2tw0kc9hmlY-jX_rR-I-9WVwQYFbyogU7ki99siEGGR9aVA2GKsp5nNcVHXBtjM3bI_zfa21XSWedvaSIVwZV8kEGixKYsBf2GMYKVI Jul 16 21:16:23.786907 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:16:23.786907 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:16:23.786907 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:16:23.786907 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:16:23.786907 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:16:23.786907 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:16:23.786907 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: 4337575622d04a26942905c9dc044ba7 Jul 16 21:16:23.786907 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-ImagesOneServerNegativeTestJSON-526509748 Jul 16 21:16:23.786907 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:16:23.786907 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:16:23.786907 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/4337575622d04a26942905c9dc044ba7"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/4337575622d04a26942905c9dc044ba7"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_4337575622d04a26942905c9dc044ba7"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/4337575622d04a26942905c9dc044ba7"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:16:23.786907 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-ImagesOneServerNegativeTestJSON-526509748 Jul 16 21:16:23.786907 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: 4337575622d04a26942905c9dc044ba7 Jul 16 21:16:23.786907 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-ImagesOneServerNegativeTestJSON-526509748 Jul 16 21:16:23.786907 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-ImagesOneServerNegativeTestJSON-526509748-project Jul 16 21:16:23.786907 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:16:23.786907 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:16:23.786907 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: 4e9df82f95ac4779859972403869096a Jul 16 21:16:23.786907 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-ImagesOneServerNegativeTestJSON-526509748-project Jul 16 21:16:23.786907 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:16:23.786907 ncdv-master-1 devstack@n-api.service[138681]: exception: Instance 39e51d01-21e0-4dd5-bfd4-16b0f8b0f827 could not be found. Jul 16 21:16:23.786907 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': '39e51d01-21e0-4dd5-bfd4-16b0f8b0f827'} Jul 16 21:16:23.786907 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:16:23.786907 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:16:23.786907 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:16:23.786907 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:16:23.786907 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:16:23.786907 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:16:23.786907 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:16:23.786907 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:16:23.786907 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:16:23.786907 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:16:23.786907 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 39e51d01-21e0-4dd5-bfd4-16b0f8b0f827 could not be found. Jul 16 21:16:23.786907 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:16:23.786907 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:16:23.786907 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:16:23.786907 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:16:23.786907 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:16:23.786907 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:16:23.786907 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:16:23.786907 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:16:23.786907 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:16:23.786907 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:16:23.786907 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:16:23.786907 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:16:23.786907 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:16:23.786907 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:16:23.786907 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:16:23.786907 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:16:23.786907 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 39e51d01-21e0-4dd5-bfd4-16b0f8b0f827 could not be found. Jul 16 21:16:23.786907 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:16:23.786907 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:16:23.798432 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:16:23.798432 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:16:23.798432 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:16:23.798432 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:16:23.798432 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:16:23.798432 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:16:23.798432 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 39e51d01-21e0-4dd5-bfd4-16b0f8b0f827 could not be found. Jul 16 21:16:23.798432 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:16:23.798432 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-4dbc39d3-cf2c-4c53-ae9a-093c82d3668d tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Returning 404 to user: Instance 39e51d01-21e0-4dd5-bfd4-16b0f8b0f827 could not be found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:16:23.798432 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-4dbc39d3-cf2c-4c53-ae9a-093c82d3668d tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] 10.222.0.16 "GET /compute/v2.1/servers/39e51d01-21e0-4dd5-bfd4-16b0f8b0f827" status: 404 len: 111 microversion: 2.1 time: 0.081209 Jul 16 21:16:23.798432 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 912/1905] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:16:23 2022] GET /compute/v2.1/servers/39e51d01-21e0-4dd5-bfd4-16b0f8b0f827 => generated 111 bytes in 82 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:16:23.803328 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-f43c9ac1-9746-4cce-a57c-0a3566114ac5 req-b66d3240-368d-413d-bd88-b6f3b8f51926 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "b9f2c527-6346-4403-841e-b7bd876a5fb7", "tag": "edbdb017-2f7f-440d-9d46-aacbd858ccb6"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:16:23.812332 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-f43c9ac1-9746-4cce-a57c-0a3566114ac5 req-b66d3240-368d-413d-bd88-b6f3b8f51926 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:23.813789 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-f43c9ac1-9746-4cce-a57c-0a3566114ac5 req-b66d3240-368d-413d-bd88-b6f3b8f51926 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:23.839187 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-f43c9ac1-9746-4cce-a57c-0a3566114ac5 req-b66d3240-368d-413d-bd88-b6f3b8f51926 service nova] Creating event network-changed:edbdb017-2f7f-440d-9d46-aacbd858ccb6 for instance b9f2c527-6346-4403-841e-b7bd876a5fb7 on nch2-master-1 Jul 16 21:16:23.845158 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-f43c9ac1-9746-4cce-a57c-0a3566114ac5 req-b66d3240-368d-413d-bd88-b6f3b8f51926 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.047197 Jul 16 21:16:23.849898 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 913/1906] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:16:23 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:16:23.852797 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-cbd74573-2a04-4f4a-977f-f0e027a469a7 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ImagesTestJSON-server-368706092", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452", "networks": [{"uuid": "e0956c94-52de-403b-bf7c-aa26a7179ba6"}]}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:16:23.911366 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-63d89066-238b-413d-a7eb-d531fcbd508d tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aa99be675e946fd995d71465b4cbec6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd82c2d92c67f469e863eb70bb7f9af84', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:23.920375 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-63d89066-238b-413d-a7eb-d531fcbd508d tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] 10.222.0.16 "GET /compute/v2.1/servers/b9f2c527-6346-4403-841e-b7bd876a5fb7" status: 200 len: 1484 microversion: 2.1 time: 0.259979 Jul 16 21:16:23.932901 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 994/1907] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:16:23 2022] GET /compute/v2.1/servers/b9f2c527-6346-4403-841e-b7bd876a5fb7 => generated 1484 bytes in 264 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:16:23.936656 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-a8b6c54c-3d64-40ca-aa50-b50ce893b8e4 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:23.936656 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-a8b6c54c-3d64-40ca-aa50-b50ce893b8e4 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] [instance: 11194822-5d42-45f4-99d0-88c70c808ddd] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:23.938248 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.network.neutron [None req-cbd74573-2a04-4f4a-977f-f0e027a469a7 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] validate_networks() for [('e0956c94-52de-403b-bf7c-aa26a7179ba6', None, None, None, None, None)] {{(pid=138681) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 16 21:16:23.950525 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a8b6c54c-3d64-40ca-aa50-b50ce893b8e4 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:23.954618 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a8b6c54c-3d64-40ca-aa50-b50ce893b8e4 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:23.955524 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a8b6c54c-3d64-40ca-aa50-b50ce893b8e4 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:23.956058 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a8b6c54c-3d64-40ca-aa50-b50ce893b8e4 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:24.389489 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-a8b6c54c-3d64-40ca-aa50-b50ce893b8e4 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd4144b5388b46a989a41a2f556bb508', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a90d4bcd25ff4bf68bdc194180fbbe09', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:24.398109 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-a8b6c54c-3d64-40ca-aa50-b50ce893b8e4 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] 10.222.0.16 "GET /compute/v2.1/servers/11194822-5d42-45f4-99d0-88c70c808ddd" status: 200 len: 1513 microversion: 2.1 time: 0.471750 Jul 16 21:16:24.398970 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 995/1908] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:16:23 2022] GET /compute/v2.1/servers/11194822-5d42-45f4-99d0-88c70c808ddd => generated 1513 bytes in 474 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:16:24.404832 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-8b885d06-caac-4210-8f58-e3bd9db6c60c tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Calling method '.version_select at 0x7fdda2b7f040>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:24.512448 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-8b885d06-caac-4210-8f58-e3bd9db6c60c tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] 10.222.0.16 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-660085568" status: 202 len: 0 microversion: 2.1 time: 0.112016 Jul 16 21:16:24.517398 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 996/1909] 10.222.0.16 () {58 vars in 1217 bytes} [Sat Jul 16 21:16:24 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-660085568 => generated 0 bytes in 117 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 21:16:24.523462 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-2c4e42db-b239-46c9-9afb-4b14e353bae5 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:24.526466 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-2c4e42db-b239-46c9-9afb-4b14e353bae5 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] [instance: 9657dddc-2416-4c95-aab3-d6b6f9673369] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:24.542679 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2c4e42db-b239-46c9-9afb-4b14e353bae5 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:24.543273 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2c4e42db-b239-46c9-9afb-4b14e353bae5 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:24.544188 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2c4e42db-b239-46c9-9afb-4b14e353bae5 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:24.544731 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2c4e42db-b239-46c9-9afb-4b14e353bae5 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:24.851549 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-2c4e42db-b239-46c9-9afb-4b14e353bae5 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '015ac1bbb2df487e99e5bddc60ee6923', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9f5f03e09936489fa1505188cefe3445', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:24.881531 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-2c4e42db-b239-46c9-9afb-4b14e353bae5 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '015ac1bbb2df487e99e5bddc60ee6923', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9f5f03e09936489fa1505188cefe3445', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:24.883414 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-2c4e42db-b239-46c9-9afb-4b14e353bae5 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '015ac1bbb2df487e99e5bddc60ee6923', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9f5f03e09936489fa1505188cefe3445', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:24.884881 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-2c4e42db-b239-46c9-9afb-4b14e353bae5 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] 10.222.0.16 "GET /compute/v2.1/servers/9657dddc-2416-4c95-aab3-d6b6f9673369" status: 200 len: 1273 microversion: 2.47 time: 0.365279 Jul 16 21:16:24.896912 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 997/1910] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:16:24 2022] GET /compute/v2.1/servers/9657dddc-2416-4c95-aab3-d6b6f9673369 => generated 1273 bytes in 367 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:16:25.301937 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-cbd74573-2a04-4f4a-977f-f0e027a469a7 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Flavor limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 16 21:16:25.302550 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-cbd74573-2a04-4f4a-977f-f0e027a469a7 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Image limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 16 21:16:25.304499 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-cbd74573-2a04-4f4a-977f-f0e027a469a7 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Flavor pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 16 21:16:25.304499 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-cbd74573-2a04-4f4a-977f-f0e027a469a7 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Image pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 16 21:16:25.304499 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-cbd74573-2a04-4f4a-977f-f0e027a469a7 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 16 21:16:25.325048 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cbd74573-2a04-4f4a-977f-f0e027a469a7 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:25.327703 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cbd74573-2a04-4f4a-977f-f0e027a469a7 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:25.327703 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cbd74573-2a04-4f4a-977f-f0e027a469a7 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:25.327703 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cbd74573-2a04-4f4a-977f-f0e027a469a7 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:25.417356 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-acc98984-7c7e-4b15-b332-3f9f29b8f59c tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:25.418397 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-acc98984-7c7e-4b15-b332-3f9f29b8f59c tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] [instance: 11194822-5d42-45f4-99d0-88c70c808ddd] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:25.430357 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-acc98984-7c7e-4b15-b332-3f9f29b8f59c tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:25.435745 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-acc98984-7c7e-4b15-b332-3f9f29b8f59c tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:25.436794 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-acc98984-7c7e-4b15-b332-3f9f29b8f59c tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:25.437355 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-acc98984-7c7e-4b15-b332-3f9f29b8f59c tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:25.444911 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-cbd74573-2a04-4f4a-977f-f0e027a469a7 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Getting quotas for project 24c64db8082648c8837c16352252f96d. Resources: {'ram', 'instances', 'cores'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:16:25.470245 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-cbd74573-2a04-4f4a-977f-f0e027a469a7 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Getting quotas for user daf3c9e6634e4ae1bfb492eae1f44c93 and project 24c64db8082648c8837c16352252f96d. Resources: {'ram', 'instances', 'cores'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:16:25.523260 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-cbd74573-2a04-4f4a-977f-f0e027a469a7 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Going to run 1 instances... {{(pid=138681) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 16 21:16:25.528341 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-cbd74573-2a04-4f4a-977f-f0e027a469a7 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] [instance: f03a2a83-e007-444a-b46b-70eb46bb242b] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138681) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 16 21:16:25.785111 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-cbd74573-2a04-4f4a-977f-f0e027a469a7 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 1.937287 Jul 16 21:16:25.793694 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 914/1911] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 21:16:23 2022] POST /compute/v2.1/servers => generated 380 bytes in 1947 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 16 21:16:25.812495 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-1749072b-9b3b-4025-a95f-64fa351121b6 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:25.814838 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-1749072b-9b3b-4025-a95f-64fa351121b6 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] [instance: f03a2a83-e007-444a-b46b-70eb46bb242b] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:25.982096 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-1749072b-9b3b-4025-a95f-64fa351121b6 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'daf3c9e6634e4ae1bfb492eae1f44c93', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '24c64db8082648c8837c16352252f96d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:25.998564 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-1749072b-9b3b-4025-a95f-64fa351121b6 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] 10.222.0.16 "GET /compute/v2.1/servers/f03a2a83-e007-444a-b46b-70eb46bb242b" status: 200 len: 1207 microversion: 2.1 time: 0.203478 Jul 16 21:16:25.999809 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 915/1912] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:16:25 2022] GET /compute/v2.1/servers/f03a2a83-e007-444a-b46b-70eb46bb242b => generated 1207 bytes in 205 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:16:26.016677 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-903c6ffb-4980-4150-a335-583e29d49135 req-19dcf9f1-611e-4705-b476-ae2374b5d5c3 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "93e327c3-c63d-439a-9b25-1a805f785949", "name": "network-vif-unplugged", "status": "completed", "tag": "4a074ea9-9de4-4dc1-aba4-632bf2d90076"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:16:26.033420 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-903c6ffb-4980-4150-a335-583e29d49135 req-19dcf9f1-611e-4705-b476-ae2374b5d5c3 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:26.034154 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-903c6ffb-4980-4150-a335-583e29d49135 req-19dcf9f1-611e-4705-b476-ae2374b5d5c3 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:26.066659 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-903c6ffb-4980-4150-a335-583e29d49135 req-19dcf9f1-611e-4705-b476-ae2374b5d5c3 service nova] Creating event network-vif-unplugged:4a074ea9-9de4-4dc1-aba4-632bf2d90076 for instance 93e327c3-c63d-439a-9b25-1a805f785949 on nch1-master-1 Jul 16 21:16:26.085304 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-903c6ffb-4980-4150-a335-583e29d49135 req-19dcf9f1-611e-4705-b476-ae2374b5d5c3 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.072265 Jul 16 21:16:26.085924 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 916/1913] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:16:25 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:16:26.090666 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-acc98984-7c7e-4b15-b332-3f9f29b8f59c tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd4144b5388b46a989a41a2f556bb508', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a90d4bcd25ff4bf68bdc194180fbbe09', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:26.099204 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-83194451-618b-4b33-9f0b-6520d8826412 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:26.099204 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-83194451-618b-4b33-9f0b-6520d8826412 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] [instance: 9657dddc-2416-4c95-aab3-d6b6f9673369] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:26.105829 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-83194451-618b-4b33-9f0b-6520d8826412 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:26.106279 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-83194451-618b-4b33-9f0b-6520d8826412 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:26.107716 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-83194451-618b-4b33-9f0b-6520d8826412 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:26.108194 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-83194451-618b-4b33-9f0b-6520d8826412 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:26.134565 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-acc98984-7c7e-4b15-b332-3f9f29b8f59c tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] 10.222.0.16 "GET /compute/v2.1/servers/11194822-5d42-45f4-99d0-88c70c808ddd" status: 200 len: 1513 microversion: 2.1 time: 0.718698 Jul 16 21:16:26.137802 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 998/1914] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:16:25 2022] GET /compute/v2.1/servers/11194822-5d42-45f4-99d0-88c70c808ddd => generated 1513 bytes in 725 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:16:26.254292 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-83194451-618b-4b33-9f0b-6520d8826412 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '015ac1bbb2df487e99e5bddc60ee6923', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9f5f03e09936489fa1505188cefe3445', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:26.259140 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=96,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:16:26.268452 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-83194451-618b-4b33-9f0b-6520d8826412 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '015ac1bbb2df487e99e5bddc60ee6923', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9f5f03e09936489fa1505188cefe3445', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:26.270345 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-83194451-618b-4b33-9f0b-6520d8826412 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '015ac1bbb2df487e99e5bddc60ee6923', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9f5f03e09936489fa1505188cefe3445', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:26.274565 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-83194451-618b-4b33-9f0b-6520d8826412 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] 10.222.0.16 "GET /compute/v2.1/servers/9657dddc-2416-4c95-aab3-d6b6f9673369" status: 200 len: 1337 microversion: 2.47 time: 0.186918 Jul 16 21:16:26.278911 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 917/1915] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:16:26 2022] GET /compute/v2.1/servers/9657dddc-2416-4c95-aab3-d6b6f9673369 => generated 1337 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:16:26.346847 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-50b98e9c-94d7-4102-8192-b9a4c8ad22c8 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:26.347648 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-50b98e9c-94d7-4102-8192-b9a4c8ad22c8 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] [instance: 6a391156-ffce-49c0-a79d-47b4544264f5] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:26.355767 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-50b98e9c-94d7-4102-8192-b9a4c8ad22c8 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:26.357920 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-50b98e9c-94d7-4102-8192-b9a4c8ad22c8 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:26.359922 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-50b98e9c-94d7-4102-8192-b9a4c8ad22c8 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:26.361465 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-50b98e9c-94d7-4102-8192-b9a4c8ad22c8 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:26.430103 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-50b98e9c-94d7-4102-8192-b9a4c8ad22c8 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] [instance: 6a391156-ffce-49c0-a79d-47b4544264f5] Going to try to terminate instance {{(pid=138680) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 16 21:16:26.939273 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-50b98e9c-94d7-4102-8192-b9a4c8ad22c8 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] 10.222.0.16 "DELETE /compute/v2.1/servers/6a391156-ffce-49c0-a79d-47b4544264f5" status: 204 len: 0 microversion: 2.1 time: 0.596593 Jul 16 21:16:26.940494 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 999/1916] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:16:26 2022] DELETE /compute/v2.1/servers/6a391156-ffce-49c0-a79d-47b4544264f5 => generated 0 bytes in 599 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 16 21:16:26.948401 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-66724426-68af-45d8-a475-abbbff1ee2c4 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:26.954280 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-66724426-68af-45d8-a475-abbbff1ee2c4 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] [instance: 6a391156-ffce-49c0-a79d-47b4544264f5] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:26.958715 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-66724426-68af-45d8-a475-abbbff1ee2c4 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:26.962120 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-66724426-68af-45d8-a475-abbbff1ee2c4 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:26.963042 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-66724426-68af-45d8-a475-abbbff1ee2c4 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:26.963493 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-66724426-68af-45d8-a475-abbbff1ee2c4 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:26.972978 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-b245d7e5-b9f2-4a1d-aa79-011c7184904b req-8801b655-8eb3-44eb-976e-bc3e82f86378 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "93e327c3-c63d-439a-9b25-1a805f785949", "name": "network-vif-deleted", "tag": "4a074ea9-9de4-4dc1-aba4-632bf2d90076"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:16:26.990474 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-b245d7e5-b9f2-4a1d-aa79-011c7184904b req-8801b655-8eb3-44eb-976e-bc3e82f86378 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:26.995271 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-b245d7e5-b9f2-4a1d-aa79-011c7184904b req-8801b655-8eb3-44eb-976e-bc3e82f86378 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:27.016700 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-b245d7e5-b9f2-4a1d-aa79-011c7184904b req-8801b655-8eb3-44eb-976e-bc3e82f86378 service nova] Creating event network-vif-deleted:4a074ea9-9de4-4dc1-aba4-632bf2d90076 for instance 93e327c3-c63d-439a-9b25-1a805f785949 on nch1-master-1 Jul 16 21:16:27.026383 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-b245d7e5-b9f2-4a1d-aa79-011c7184904b req-8801b655-8eb3-44eb-976e-bc3e82f86378 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.058872 Jul 16 21:16:27.027954 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1000/1917] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:16:26 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 62 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 16 21:16:27.034896 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-6e272291-d6f0-4fe0-b63e-073665bfe7b4 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:27.035690 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-6e272291-d6f0-4fe0-b63e-073665bfe7b4 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] [instance: f03a2a83-e007-444a-b46b-70eb46bb242b] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:27.132295 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-6e272291-d6f0-4fe0-b63e-073665bfe7b4 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'daf3c9e6634e4ae1bfb492eae1f44c93', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '24c64db8082648c8837c16352252f96d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:27.143629 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-6e272291-d6f0-4fe0-b63e-073665bfe7b4 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] 10.222.0.16 "GET /compute/v2.1/servers/f03a2a83-e007-444a-b46b-70eb46bb242b" status: 200 len: 1207 microversion: 2.1 time: 0.113739 Jul 16 21:16:27.144815 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1001/1918] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:16:27 2022] GET /compute/v2.1/servers/f03a2a83-e007-444a-b46b-70eb46bb242b => generated 1207 bytes in 116 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:16:27.157749 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-14c1a6df-11c1-45c7-8257-c4792f0d7c46 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:27.159063 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-14c1a6df-11c1-45c7-8257-c4792f0d7c46 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] [instance: 11194822-5d42-45f4-99d0-88c70c808ddd] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:27.173922 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-14c1a6df-11c1-45c7-8257-c4792f0d7c46 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:27.173922 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-14c1a6df-11c1-45c7-8257-c4792f0d7c46 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:27.173922 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-14c1a6df-11c1-45c7-8257-c4792f0d7c46 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:27.173922 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-14c1a6df-11c1-45c7-8257-c4792f0d7c46 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:27.222427 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-66724426-68af-45d8-a475-abbbff1ee2c4 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a69f9dee03d43beaafa966216b9e289', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9b769bf8f3ea4e368bf68de1df6538f9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:27.232642 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-66724426-68af-45d8-a475-abbbff1ee2c4 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] 10.222.0.16 "GET /compute/v2.1/servers/6a391156-ffce-49c0-a79d-47b4544264f5" status: 200 len: 1667 microversion: 2.1 time: 0.288084 Jul 16 21:16:27.237763 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 918/1919] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:16:26 2022] GET /compute/v2.1/servers/6a391156-ffce-49c0-a79d-47b4544264f5 => generated 1667 bytes in 292 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:16:27.311813 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-e1d41681-5ea2-403c-9621-6b48821fc5b3 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:27.317462 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-e1d41681-5ea2-403c-9621-6b48821fc5b3 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] [instance: 9657dddc-2416-4c95-aab3-d6b6f9673369] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:27.342962 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e1d41681-5ea2-403c-9621-6b48821fc5b3 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:27.344060 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e1d41681-5ea2-403c-9621-6b48821fc5b3 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:27.345196 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e1d41681-5ea2-403c-9621-6b48821fc5b3 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:27.345813 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e1d41681-5ea2-403c-9621-6b48821fc5b3 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:27.347067 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-14c1a6df-11c1-45c7-8257-c4792f0d7c46 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd4144b5388b46a989a41a2f556bb508', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a90d4bcd25ff4bf68bdc194180fbbe09', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:27.357506 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-14c1a6df-11c1-45c7-8257-c4792f0d7c46 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] 10.222.0.16 "GET /compute/v2.1/servers/11194822-5d42-45f4-99d0-88c70c808ddd" status: 200 len: 1513 microversion: 2.1 time: 0.204257 Jul 16 21:16:27.368781 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1002/1920] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:16:27 2022] GET /compute/v2.1/servers/11194822-5d42-45f4-99d0-88c70c808ddd => generated 1513 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:16:27.376226 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-362bb948-b778-40b4-b5ae-965bb7880cda tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Calling method '.version_select at 0x7fdd9cc3f040>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:27.402582 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-362bb948-b778-40b4-b5ae-965bb7880cda tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] HTTP exception thrown: Image not found. Jul 16 21:16:27.405348 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-362bb948-b778-40b4-b5ae-965bb7880cda tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Request method failure captured: Jul 16 21:16:27.405348 ncdv-master-1 devstack@n-api.service[138680]: request: DELETE /compute/v2.1/images/55ce9fb1-ab88-4209-bcf9-89b9a3541595 HTTP/1.1 Jul 16 21:16:27.405348 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:16:27.405348 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:16:27.405348 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:16:27.405348 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 0 Jul 16 21:16:27.405348 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:16:27.405348 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:16:27.405348 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:16:27.405348 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:16:27.405348 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0yoJFUAf_8Wtq8fI-wZdJwoKyg8rUEOC_eBMeqcRE2hk6JM7_ssDplugQwmk2S9xH7Oys7Ho2tw0kc9hmlY-jX_rR-I-9WVwQYFbyogU7ki99siEGGR9aVA2GKsp5nNcVHXBtjM3bI_zfa21XSWedvaSIVwZV8kEGixKYsBf2GMYKVI Jul 16 21:16:27.405348 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:16:27.405348 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:16:27.405348 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:16:27.405348 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:16:27.405348 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:16:27.405348 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:16:27.405348 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: 4337575622d04a26942905c9dc044ba7 Jul 16 21:16:27.405348 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-ImagesOneServerNegativeTestJSON-526509748 Jul 16 21:16:27.405348 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:16:27.405348 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:16:27.405348 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/4337575622d04a26942905c9dc044ba7"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/4337575622d04a26942905c9dc044ba7"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_4337575622d04a26942905c9dc044ba7"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/4337575622d04a26942905c9dc044ba7"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:16:27.405348 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-ImagesOneServerNegativeTestJSON-526509748 Jul 16 21:16:27.405348 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: 4337575622d04a26942905c9dc044ba7 Jul 16 21:16:27.405348 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-ImagesOneServerNegativeTestJSON-526509748 Jul 16 21:16:27.405348 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-ImagesOneServerNegativeTestJSON-526509748-project Jul 16 21:16:27.405348 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:16:27.405348 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:16:27.405348 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: 4e9df82f95ac4779859972403869096a Jul 16 21:16:27.405348 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-ImagesOneServerNegativeTestJSON-526509748-project Jul 16 21:16:27.405348 ncdv-master-1 devstack@n-api.service[138680]: method: Jul 16 21:16:27.405348 ncdv-master-1 devstack@n-api.service[138680]: exception: Image not found. Jul 16 21:16:27.405348 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': '55ce9fb1-ab88-4209-bcf9-89b9a3541595'} Jul 16 21:16:27.405348 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:16:27.405348 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:16:27.405348 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 724, in delete Jul 16 21:16:27.405348 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi self._client.call(context, 2, 'delete', args=(image_id,)) Jul 16 21:16:27.405348 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 191, in call Jul 16 21:16:27.405348 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi result = getattr(controller, method)(*args, **kwargs) Jul 16 21:16:27.405348 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/utils.py", line 670, in inner Jul 16 21:16:27.405348 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return RequestIdProxy(wrapped(*args, **kwargs)) Jul 16 21:16:27.405348 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/v2/images.py", line 386, in delete Jul 16 21:16:27.405348 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi resp, body = self.http_client.delete(url) Jul 16 21:16:27.405348 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/adapter.py", line 410, in delete Jul 16 21:16:27.405348 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return self.request(url, 'DELETE', **kwargs) Jul 16 21:16:27.405348 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 380, in request Jul 16 21:16:27.405348 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return self._handle_response(resp) Jul 16 21:16:27.405348 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 120, in _handle_response Jul 16 21:16:27.405348 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.from_response(resp, resp.content) Jul 16 21:16:27.405348 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi glanceclient.exc.HTTPNotFound: HTTP 404 Not Found: Failed to find image 55ce9fb1-ab88-4209-bcf9-89b9a3541595 to delete Jul 16 21:16:27.405348 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:16:27.405348 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:16:27.405348 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:16:27.405348 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:16:27.405348 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/images.py", line 104, in delete Jul 16 21:16:27.405348 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi self._image_api.delete(context, id) Jul 16 21:16:27.405348 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 1255, in delete Jul 16 21:16:27.405348 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return session.delete(context, image_id) Jul 16 21:16:27.405348 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 726, in delete Jul 16 21:16:27.405348 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.ImageNotFound(image_id=image_id) Jul 16 21:16:27.405348 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.ImageNotFound: Image 55ce9fb1-ab88-4209-bcf9-89b9a3541595 could not be found. Jul 16 21:16:27.405348 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:16:27.405348 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:16:27.405348 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:16:27.410270 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:16:27.410270 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:16:27.410270 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:16:27.410270 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:16:27.410270 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:16:27.410270 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Jul 16 21:16:27.410270 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Jul 16 21:16:27.410270 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:16:27.410270 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:16:27.410270 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/images.py", line 107, in delete Jul 16 21:16:27.410270 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=explanation) Jul 16 21:16:27.410270 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Image not found. Jul 16 21:16:27.410270 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:16:27.410270 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:16:27.410270 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:16:27.410270 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:16:27.410270 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:16:27.410270 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:16:27.410270 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:16:27.410270 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:16:27.410270 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Image not found. Jul 16 21:16:27.410270 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:16:27.410270 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-362bb948-b778-40b4-b5ae-965bb7880cda tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Returning 404 to user: Image not found. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:16:27.410270 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-362bb948-b778-40b4-b5ae-965bb7880cda tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] 10.222.0.16 "DELETE /compute/v2.1/images/55ce9fb1-ab88-4209-bcf9-89b9a3541595" status: 404 len: 62 microversion: 2.1 time: 0.035909 Jul 16 21:16:27.410270 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1003/1921] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:16:27 2022] DELETE /compute/v2.1/images/55ce9fb1-ab88-4209-bcf9-89b9a3541595 => generated 62 bytes in 39 msecs (HTTP/1.1 404) 9 headers in 378 bytes (1 switches on core 0) Jul 16 21:16:27.416409 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-90ac0144-47ae-4a62-b2d7-e2f4b9adb742 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:27.417032 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-90ac0144-47ae-4a62-b2d7-e2f4b9adb742 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] [instance: e66baae9-d488-46a4-8347-623030fc8b69] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:27.425819 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-90ac0144-47ae-4a62-b2d7-e2f4b9adb742 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:27.426312 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-90ac0144-47ae-4a62-b2d7-e2f4b9adb742 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:27.427255 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-90ac0144-47ae-4a62-b2d7-e2f4b9adb742 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:27.427665 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-90ac0144-47ae-4a62-b2d7-e2f4b9adb742 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:27.449841 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-e1d41681-5ea2-403c-9621-6b48821fc5b3 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '015ac1bbb2df487e99e5bddc60ee6923', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9f5f03e09936489fa1505188cefe3445', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:27.461996 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-90ac0144-47ae-4a62-b2d7-e2f4b9adb742 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] HTTP exception thrown: Instance e66baae9-d488-46a4-8347-623030fc8b69 could not be found. Jul 16 21:16:27.463806 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-e1d41681-5ea2-403c-9621-6b48821fc5b3 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '015ac1bbb2df487e99e5bddc60ee6923', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9f5f03e09936489fa1505188cefe3445', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:27.465669 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-90ac0144-47ae-4a62-b2d7-e2f4b9adb742 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Request method failure captured: Jul 16 21:16:27.465669 ncdv-master-1 devstack@n-api.service[138680]: request: DELETE /compute/v2.1/servers/e66baae9-d488-46a4-8347-623030fc8b69 HTTP/1.1 Jul 16 21:16:27.465669 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:16:27.465669 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:16:27.465669 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:16:27.465669 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 0 Jul 16 21:16:27.465669 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:16:27.465669 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:16:27.465669 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:16:27.465669 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:16:27.465669 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0yoJFUAf_8Wtq8fI-wZdJwoKyg8rUEOC_eBMeqcRE2hk6JM7_ssDplugQwmk2S9xH7Oys7Ho2tw0kc9hmlY-jX_rR-I-9WVwQYFbyogU7ki99siEGGR9aVA2GKsp5nNcVHXBtjM3bI_zfa21XSWedvaSIVwZV8kEGixKYsBf2GMYKVI Jul 16 21:16:27.465669 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:16:27.465669 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:16:27.465669 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:16:27.465669 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:16:27.465669 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:16:27.465669 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:16:27.465669 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: 4337575622d04a26942905c9dc044ba7 Jul 16 21:16:27.465669 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-ImagesOneServerNegativeTestJSON-526509748 Jul 16 21:16:27.465669 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:16:27.465669 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:16:27.465669 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/4337575622d04a26942905c9dc044ba7"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/4337575622d04a26942905c9dc044ba7"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_4337575622d04a26942905c9dc044ba7"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/4337575622d04a26942905c9dc044ba7"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:16:27.465669 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-ImagesOneServerNegativeTestJSON-526509748 Jul 16 21:16:27.465669 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: 4337575622d04a26942905c9dc044ba7 Jul 16 21:16:27.465669 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-ImagesOneServerNegativeTestJSON-526509748 Jul 16 21:16:27.465669 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-ImagesOneServerNegativeTestJSON-526509748-project Jul 16 21:16:27.465669 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:16:27.465669 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:16:27.465669 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: 4e9df82f95ac4779859972403869096a Jul 16 21:16:27.465669 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-ImagesOneServerNegativeTestJSON-526509748-project Jul 16 21:16:27.465669 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 21:16:27.465669 ncdv-master-1 devstack@n-api.service[138680]: exception: Instance e66baae9-d488-46a4-8347-623030fc8b69 could not be found. Jul 16 21:16:27.465669 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': 'e66baae9-d488-46a4-8347-623030fc8b69'} Jul 16 21:16:27.465669 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:16:27.465669 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:16:27.465669 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:16:27.465669 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:16:27.465669 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:16:27.465669 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:16:27.465669 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:16:27.465669 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:16:27.465669 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:16:27.465669 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:16:27.465669 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance e66baae9-d488-46a4-8347-623030fc8b69 could not be found. Jul 16 21:16:27.465669 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:16:27.465669 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:16:27.465669 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:16:27.465669 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:16:27.465669 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:16:27.465669 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:16:27.465669 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:16:27.465669 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:16:27.465669 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:16:27.465669 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:16:27.465669 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Jul 16 21:16:27.465669 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Jul 16 21:16:27.465669 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Jul 16 21:16:27.465669 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Jul 16 21:16:27.465669 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:16:27.465669 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:16:27.465669 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:16:27.465669 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:16:27.471162 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance e66baae9-d488-46a4-8347-623030fc8b69 could not be found. Jul 16 21:16:27.471162 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:16:27.471162 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:16:27.471162 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:16:27.471162 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:16:27.471162 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:16:27.471162 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:16:27.471162 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:16:27.471162 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:16:27.471162 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance e66baae9-d488-46a4-8347-623030fc8b69 could not be found. Jul 16 21:16:27.471162 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:16:27.472418 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-e1d41681-5ea2-403c-9621-6b48821fc5b3 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '015ac1bbb2df487e99e5bddc60ee6923', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9f5f03e09936489fa1505188cefe3445', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:27.472599 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-90ac0144-47ae-4a62-b2d7-e2f4b9adb742 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Returning 404 to user: Instance e66baae9-d488-46a4-8347-623030fc8b69 could not be found. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:16:27.472599 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-90ac0144-47ae-4a62-b2d7-e2f4b9adb742 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] 10.222.0.16 "DELETE /compute/v2.1/servers/e66baae9-d488-46a4-8347-623030fc8b69" status: 404 len: 111 microversion: 2.1 time: 0.055710 Jul 16 21:16:27.472822 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-e1d41681-5ea2-403c-9621-6b48821fc5b3 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] 10.222.0.16 "GET /compute/v2.1/servers/9657dddc-2416-4c95-aab3-d6b6f9673369" status: 200 len: 1335 microversion: 2.47 time: 0.176363 Jul 16 21:16:27.472924 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1004/1922] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:16:27 2022] DELETE /compute/v2.1/servers/e66baae9-d488-46a4-8347-623030fc8b69 => generated 111 bytes in 59 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:16:27.473021 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 919/1923] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:16:27 2022] GET /compute/v2.1/servers/9657dddc-2416-4c95-aab3-d6b6f9673369 => generated 1335 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 16 21:16:27.488869 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-36144a4c-3d04-47b3-bb9a-8aac703ef180 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:27.489495 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-36144a4c-3d04-47b3-bb9a-8aac703ef180 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] [instance: e66baae9-d488-46a4-8347-623030fc8b69] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:27.497501 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-36144a4c-3d04-47b3-bb9a-8aac703ef180 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:27.497954 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-36144a4c-3d04-47b3-bb9a-8aac703ef180 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:27.498731 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-36144a4c-3d04-47b3-bb9a-8aac703ef180 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:27.499211 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-36144a4c-3d04-47b3-bb9a-8aac703ef180 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:27.525586 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-36144a4c-3d04-47b3-bb9a-8aac703ef180 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] HTTP exception thrown: Instance e66baae9-d488-46a4-8347-623030fc8b69 could not be found. Jul 16 21:16:27.527557 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-36144a4c-3d04-47b3-bb9a-8aac703ef180 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Request method failure captured: Jul 16 21:16:27.527557 ncdv-master-1 devstack@n-api.service[138680]: request: GET /compute/v2.1/servers/e66baae9-d488-46a4-8347-623030fc8b69 HTTP/1.1 Jul 16 21:16:27.527557 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:16:27.527557 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:16:27.527557 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:16:27.527557 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 0 Jul 16 21:16:27.527557 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:16:27.527557 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:16:27.527557 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:16:27.527557 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:16:27.527557 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0yoJFUAf_8Wtq8fI-wZdJwoKyg8rUEOC_eBMeqcRE2hk6JM7_ssDplugQwmk2S9xH7Oys7Ho2tw0kc9hmlY-jX_rR-I-9WVwQYFbyogU7ki99siEGGR9aVA2GKsp5nNcVHXBtjM3bI_zfa21XSWedvaSIVwZV8kEGixKYsBf2GMYKVI Jul 16 21:16:27.527557 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:16:27.527557 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:16:27.527557 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:16:27.527557 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:16:27.527557 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:16:27.527557 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:16:27.527557 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: 4337575622d04a26942905c9dc044ba7 Jul 16 21:16:27.527557 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-ImagesOneServerNegativeTestJSON-526509748 Jul 16 21:16:27.527557 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:16:27.527557 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:16:27.527557 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/4337575622d04a26942905c9dc044ba7"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/4337575622d04a26942905c9dc044ba7"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_4337575622d04a26942905c9dc044ba7"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/4337575622d04a26942905c9dc044ba7"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:16:27.527557 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-ImagesOneServerNegativeTestJSON-526509748 Jul 16 21:16:27.527557 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: 4337575622d04a26942905c9dc044ba7 Jul 16 21:16:27.527557 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-ImagesOneServerNegativeTestJSON-526509748 Jul 16 21:16:27.527557 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-ImagesOneServerNegativeTestJSON-526509748-project Jul 16 21:16:27.527557 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:16:27.527557 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:16:27.527557 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: 4e9df82f95ac4779859972403869096a Jul 16 21:16:27.527557 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-ImagesOneServerNegativeTestJSON-526509748-project Jul 16 21:16:27.527557 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 21:16:27.527557 ncdv-master-1 devstack@n-api.service[138680]: exception: Instance e66baae9-d488-46a4-8347-623030fc8b69 could not be found. Jul 16 21:16:27.527557 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': 'e66baae9-d488-46a4-8347-623030fc8b69'} Jul 16 21:16:27.527557 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:16:27.527557 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:16:27.527557 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:16:27.527557 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:16:27.527557 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:16:27.527557 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:16:27.527557 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:16:27.527557 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:16:27.527557 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:16:27.527557 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:16:27.527557 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance e66baae9-d488-46a4-8347-623030fc8b69 could not be found. Jul 16 21:16:27.527557 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:16:27.527557 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:16:27.527557 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:16:27.527557 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:16:27.527557 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:16:27.527557 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:16:27.527557 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:16:27.527557 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:16:27.527557 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:16:27.527557 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:16:27.527557 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:16:27.527557 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:16:27.527557 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:16:27.527557 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:16:27.527557 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:16:27.527557 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:16:27.527557 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance e66baae9-d488-46a4-8347-623030fc8b69 could not be found. Jul 16 21:16:27.527557 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:16:27.527557 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:16:27.534267 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:16:27.534267 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:16:27.534267 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:16:27.534267 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:16:27.534267 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:16:27.534267 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:16:27.534267 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance e66baae9-d488-46a4-8347-623030fc8b69 could not be found. Jul 16 21:16:27.534267 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:16:27.540098 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-36144a4c-3d04-47b3-bb9a-8aac703ef180 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] Returning 404 to user: Instance e66baae9-d488-46a4-8347-623030fc8b69 could not be found. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:16:27.540864 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-36144a4c-3d04-47b3-bb9a-8aac703ef180 tempest-ImagesOneServerNegativeTestJSON-526509748 tempest-ImagesOneServerNegativeTestJSON-526509748-project] 10.222.0.16 "GET /compute/v2.1/servers/e66baae9-d488-46a4-8347-623030fc8b69" status: 404 len: 111 microversion: 2.1 time: 0.057241 Jul 16 21:16:27.541635 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1005/1924] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:16:27 2022] GET /compute/v2.1/servers/e66baae9-d488-46a4-8347-623030fc8b69 => generated 111 bytes in 69 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:16:28.161594 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-ec16e519-0909-452a-991a-920f0277f529 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:28.162969 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-ec16e519-0909-452a-991a-920f0277f529 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] [instance: f03a2a83-e007-444a-b46b-70eb46bb242b] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:28.171384 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ec16e519-0909-452a-991a-920f0277f529 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:28.171844 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ec16e519-0909-452a-991a-920f0277f529 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:28.174003 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ec16e519-0909-452a-991a-920f0277f529 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:28.174653 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ec16e519-0909-452a-991a-920f0277f529 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:28.251461 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-5e88c7e3-e0c0-4ffd-bddd-decead586cb8 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:28.252289 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-5e88c7e3-e0c0-4ffd-bddd-decead586cb8 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] [instance: 6a391156-ffce-49c0-a79d-47b4544264f5] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:28.268325 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5e88c7e3-e0c0-4ffd-bddd-decead586cb8 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:28.270425 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5e88c7e3-e0c0-4ffd-bddd-decead586cb8 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:28.272272 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5e88c7e3-e0c0-4ffd-bddd-decead586cb8 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:28.272920 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5e88c7e3-e0c0-4ffd-bddd-decead586cb8 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:28.744096 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-ec16e519-0909-452a-991a-920f0277f529 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'daf3c9e6634e4ae1bfb492eae1f44c93', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '24c64db8082648c8837c16352252f96d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:28.756045 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-ec16e519-0909-452a-991a-920f0277f529 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] 10.222.0.16 "GET /compute/v2.1/servers/f03a2a83-e007-444a-b46b-70eb46bb242b" status: 200 len: 1203 microversion: 2.1 time: 0.599127 Jul 16 21:16:28.761904 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 920/1925] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:16:28 2022] GET /compute/v2.1/servers/f03a2a83-e007-444a-b46b-70eb46bb242b => generated 1203 bytes in 605 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:16:28.770293 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-614e9877-d4ab-4b79-9a76-30399595eec1 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:28.784068 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-614e9877-d4ab-4b79-9a76-30399595eec1 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] [instance: 11194822-5d42-45f4-99d0-88c70c808ddd] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:28.790552 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-614e9877-d4ab-4b79-9a76-30399595eec1 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:28.791059 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-614e9877-d4ab-4b79-9a76-30399595eec1 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:28.791907 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-614e9877-d4ab-4b79-9a76-30399595eec1 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:28.792813 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-614e9877-d4ab-4b79-9a76-30399595eec1 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:29.337873 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-5e88c7e3-e0c0-4ffd-bddd-decead586cb8 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a69f9dee03d43beaafa966216b9e289', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9b769bf8f3ea4e368bf68de1df6538f9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:29.343066 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-614e9877-d4ab-4b79-9a76-30399595eec1 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd4144b5388b46a989a41a2f556bb508', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a90d4bcd25ff4bf68bdc194180fbbe09', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:29.346891 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=96,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:16:29.349645 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-5e88c7e3-e0c0-4ffd-bddd-decead586cb8 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] 10.222.0.16 "GET /compute/v2.1/servers/6a391156-ffce-49c0-a79d-47b4544264f5" status: 200 len: 1667 microversion: 2.1 time: 1.102795 Jul 16 21:16:29.360664 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1006/1926] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:16:28 2022] GET /compute/v2.1/servers/6a391156-ffce-49c0-a79d-47b4544264f5 => generated 1667 bytes in 1114 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:16:29.361053 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-614e9877-d4ab-4b79-9a76-30399595eec1 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] 10.222.0.16 "GET /compute/v2.1/servers/11194822-5d42-45f4-99d0-88c70c808ddd" status: 200 len: 1513 microversion: 2.1 time: 0.596212 Jul 16 21:16:29.366188 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 921/1927] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:16:28 2022] GET /compute/v2.1/servers/11194822-5d42-45f4-99d0-88c70c808ddd => generated 1513 bytes in 603 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:16:29.368633 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-5b6131b0-a540-410a-8eb3-7f1cbd9e31aa tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:29.369679 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-5b6131b0-a540-410a-8eb3-7f1cbd9e31aa tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] [instance: 9657dddc-2416-4c95-aab3-d6b6f9673369] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:29.381509 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5b6131b0-a540-410a-8eb3-7f1cbd9e31aa tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:29.383259 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5b6131b0-a540-410a-8eb3-7f1cbd9e31aa tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:29.385472 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5b6131b0-a540-410a-8eb3-7f1cbd9e31aa tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:29.386944 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5b6131b0-a540-410a-8eb3-7f1cbd9e31aa tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:29.519209 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-5b6131b0-a540-410a-8eb3-7f1cbd9e31aa tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '015ac1bbb2df487e99e5bddc60ee6923', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9f5f03e09936489fa1505188cefe3445', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:29.529071 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-5b6131b0-a540-410a-8eb3-7f1cbd9e31aa tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '015ac1bbb2df487e99e5bddc60ee6923', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9f5f03e09936489fa1505188cefe3445', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:29.532229 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-5b6131b0-a540-410a-8eb3-7f1cbd9e31aa tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '015ac1bbb2df487e99e5bddc60ee6923', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9f5f03e09936489fa1505188cefe3445', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:29.533749 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-5b6131b0-a540-410a-8eb3-7f1cbd9e31aa tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] 10.222.0.16 "GET /compute/v2.1/servers/9657dddc-2416-4c95-aab3-d6b6f9673369" status: 200 len: 1335 microversion: 2.47 time: 0.171452 Jul 16 21:16:29.534542 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1007/1928] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:16:29 2022] GET /compute/v2.1/servers/9657dddc-2416-4c95-aab3-d6b6f9673369 => generated 1335 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:16:29.778945 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-cbd02bd4-16d2-40e1-be60-f0dc3401e058 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:29.779728 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-cbd02bd4-16d2-40e1-be60-f0dc3401e058 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] [instance: f03a2a83-e007-444a-b46b-70eb46bb242b] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:29.787435 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cbd02bd4-16d2-40e1-be60-f0dc3401e058 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:29.788160 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cbd02bd4-16d2-40e1-be60-f0dc3401e058 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:29.788949 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cbd02bd4-16d2-40e1-be60-f0dc3401e058 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:29.789569 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cbd02bd4-16d2-40e1-be60-f0dc3401e058 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:29.958489 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-cbd02bd4-16d2-40e1-be60-f0dc3401e058 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'daf3c9e6634e4ae1bfb492eae1f44c93', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '24c64db8082648c8837c16352252f96d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:29.972714 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-cbd02bd4-16d2-40e1-be60-f0dc3401e058 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] 10.222.0.16 "GET /compute/v2.1/servers/f03a2a83-e007-444a-b46b-70eb46bb242b" status: 200 len: 1265 microversion: 2.1 time: 0.198376 Jul 16 21:16:29.974528 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 922/1929] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:16:29 2022] GET /compute/v2.1/servers/f03a2a83-e007-444a-b46b-70eb46bb242b => generated 1265 bytes in 201 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:16:30.284944 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-3c1ec1eb-2ddb-4af4-9792-4c328fcc5c60 req-0075a14a-a548-4f80-9743-433025f2c1b0 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "6a391156-ffce-49c0-a79d-47b4544264f5", "name": "network-vif-deleted", "tag": "a8de1bba-f28a-49ad-92e5-8a10d67f1e75"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:16:30.311912 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-3c1ec1eb-2ddb-4af4-9792-4c328fcc5c60 req-0075a14a-a548-4f80-9743-433025f2c1b0 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:30.312373 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-3c1ec1eb-2ddb-4af4-9792-4c328fcc5c60 req-0075a14a-a548-4f80-9743-433025f2c1b0 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:30.347051 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-3c1ec1eb-2ddb-4af4-9792-4c328fcc5c60 req-0075a14a-a548-4f80-9743-433025f2c1b0 service nova] Creating event network-vif-deleted:a8de1bba-f28a-49ad-92e5-8a10d67f1e75 for instance 6a391156-ffce-49c0-a79d-47b4544264f5 on nch1-master-1 Jul 16 21:16:30.355546 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-3c1ec1eb-2ddb-4af4-9792-4c328fcc5c60 req-0075a14a-a548-4f80-9743-433025f2c1b0 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.079448 Jul 16 21:16:30.356165 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1008/1930] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:16:30 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 16 21:16:30.367143 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-807a63d5-d6ab-4836-bfac-e96e1ab8ed1a tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:30.367766 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-807a63d5-d6ab-4836-bfac-e96e1ab8ed1a tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] [instance: 6a391156-ffce-49c0-a79d-47b4544264f5] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:30.396338 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-807a63d5-d6ab-4836-bfac-e96e1ab8ed1a tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:30.399768 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-807a63d5-d6ab-4836-bfac-e96e1ab8ed1a tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:30.403368 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-807a63d5-d6ab-4836-bfac-e96e1ab8ed1a tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:30.408277 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-0adb06e2-285a-447f-a2ae-d4ddd945d795 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:30.410192 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-807a63d5-d6ab-4836-bfac-e96e1ab8ed1a tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:30.413687 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-0adb06e2-285a-447f-a2ae-d4ddd945d795 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] [instance: 11194822-5d42-45f4-99d0-88c70c808ddd] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:30.483778 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0adb06e2-285a-447f-a2ae-d4ddd945d795 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:30.485955 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0adb06e2-285a-447f-a2ae-d4ddd945d795 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:30.487599 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0adb06e2-285a-447f-a2ae-d4ddd945d795 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:30.492743 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0adb06e2-285a-447f-a2ae-d4ddd945d795 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:30.687458 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-807a63d5-d6ab-4836-bfac-e96e1ab8ed1a tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a69f9dee03d43beaafa966216b9e289', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9b769bf8f3ea4e368bf68de1df6538f9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:30.713888 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-807a63d5-d6ab-4836-bfac-e96e1ab8ed1a tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] 10.222.0.16 "GET /compute/v2.1/servers/6a391156-ffce-49c0-a79d-47b4544264f5" status: 200 len: 1599 microversion: 2.1 time: 0.340244 Jul 16 21:16:30.715829 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 923/1931] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:16:30 2022] GET /compute/v2.1/servers/6a391156-ffce-49c0-a79d-47b4544264f5 => generated 1599 bytes in 353 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:16:30.734742 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-5383a677-d78f-45d0-946f-45efc229cbca tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:30.737393 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-5383a677-d78f-45d0-946f-45efc229cbca tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] [instance: 9657dddc-2416-4c95-aab3-d6b6f9673369] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:30.769898 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5383a677-d78f-45d0-946f-45efc229cbca tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:30.780028 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5383a677-d78f-45d0-946f-45efc229cbca tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.010s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:30.786578 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5383a677-d78f-45d0-946f-45efc229cbca tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:30.787662 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5383a677-d78f-45d0-946f-45efc229cbca tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:31.089948 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-5383a677-d78f-45d0-946f-45efc229cbca tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '015ac1bbb2df487e99e5bddc60ee6923', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9f5f03e09936489fa1505188cefe3445', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:31.106167 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-5383a677-d78f-45d0-946f-45efc229cbca tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '015ac1bbb2df487e99e5bddc60ee6923', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9f5f03e09936489fa1505188cefe3445', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:31.110017 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-5383a677-d78f-45d0-946f-45efc229cbca tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '015ac1bbb2df487e99e5bddc60ee6923', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9f5f03e09936489fa1505188cefe3445', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:31.116637 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-5383a677-d78f-45d0-946f-45efc229cbca tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] 10.222.0.16 "GET /compute/v2.1/servers/9657dddc-2416-4c95-aab3-d6b6f9673369" status: 200 len: 1356 microversion: 2.47 time: 0.397095 Jul 16 21:16:31.123402 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 924/1932] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:16:30 2022] GET /compute/v2.1/servers/9657dddc-2416-4c95-aab3-d6b6f9673369 => generated 1356 bytes in 407 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:16:31.130232 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-2c1c7173-ed93-4e6a-905d-a33dfcab6205 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:31.132741 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-2c1c7173-ed93-4e6a-905d-a33dfcab6205 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] [instance: f03a2a83-e007-444a-b46b-70eb46bb242b] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:31.157276 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2c1c7173-ed93-4e6a-905d-a33dfcab6205 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:31.159796 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2c1c7173-ed93-4e6a-905d-a33dfcab6205 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:31.160851 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2c1c7173-ed93-4e6a-905d-a33dfcab6205 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:31.163179 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2c1c7173-ed93-4e6a-905d-a33dfcab6205 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:31.409151 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-2c1c7173-ed93-4e6a-905d-a33dfcab6205 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'daf3c9e6634e4ae1bfb492eae1f44c93', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '24c64db8082648c8837c16352252f96d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:31.422869 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-2c1c7173-ed93-4e6a-905d-a33dfcab6205 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] 10.222.0.16 "GET /compute/v2.1/servers/f03a2a83-e007-444a-b46b-70eb46bb242b" status: 200 len: 1265 microversion: 2.1 time: 0.294181 Jul 16 21:16:31.427166 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 925/1933] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:16:31 2022] GET /compute/v2.1/servers/f03a2a83-e007-444a-b46b-70eb46bb242b => generated 1265 bytes in 303 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:16:31.437037 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-c697b4fe-732d-4ddb-a036-80d49dbd6b49 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Action: 'action', calling method: >, body: {"rebuild": {"imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9"}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:16:31.439903 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-c697b4fe-732d-4ddb-a036-80d49dbd6b49 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] [instance: 9657dddc-2416-4c95-aab3-d6b6f9673369] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:31.465836 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-0adb06e2-285a-447f-a2ae-d4ddd945d795 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd4144b5388b46a989a41a2f556bb508', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a90d4bcd25ff4bf68bdc194180fbbe09', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:31.479884 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-0adb06e2-285a-447f-a2ae-d4ddd945d795 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] 10.222.0.16 "GET /compute/v2.1/servers/11194822-5d42-45f4-99d0-88c70c808ddd" status: 200 len: 1513 microversion: 2.1 time: 1.096032 Jul 16 21:16:31.480506 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1009/1934] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:16:30 2022] GET /compute/v2.1/servers/11194822-5d42-45f4-99d0-88c70c808ddd => generated 1513 bytes in 1100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:16:31.481104 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c697b4fe-732d-4ddb-a036-80d49dbd6b49 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:31.481586 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c697b4fe-732d-4ddb-a036-80d49dbd6b49 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:31.482521 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c697b4fe-732d-4ddb-a036-80d49dbd6b49 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:31.482966 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c697b4fe-732d-4ddb-a036-80d49dbd6b49 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:31.688929 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-c697b4fe-732d-4ddb-a036-80d49dbd6b49 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Flavor limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 16 21:16:31.691447 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-c697b4fe-732d-4ddb-a036-80d49dbd6b49 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Image limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 16 21:16:31.699424 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-c697b4fe-732d-4ddb-a036-80d49dbd6b49 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Flavor pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 16 21:16:31.700419 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-c697b4fe-732d-4ddb-a036-80d49dbd6b49 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Image pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 16 21:16:31.703747 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-c697b4fe-732d-4ddb-a036-80d49dbd6b49 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 16 21:16:31.732729 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-39664fca-1401-40b4-9571-552fb5e6a381 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:31.733674 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-39664fca-1401-40b4-9571-552fb5e6a381 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] [instance: 6a391156-ffce-49c0-a79d-47b4544264f5] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:31.748948 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-39664fca-1401-40b4-9571-552fb5e6a381 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:31.756314 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-39664fca-1401-40b4-9571-552fb5e6a381 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:31.757814 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-39664fca-1401-40b4-9571-552fb5e6a381 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:31.758272 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-39664fca-1401-40b4-9571-552fb5e6a381 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:31.816138 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-39664fca-1401-40b4-9571-552fb5e6a381 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] HTTP exception thrown: Instance 6a391156-ffce-49c0-a79d-47b4544264f5 could not be found. Jul 16 21:16:31.837210 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-39664fca-1401-40b4-9571-552fb5e6a381 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Request method failure captured: Jul 16 21:16:31.837210 ncdv-master-1 devstack@n-api.service[138680]: request: GET /compute/v2.1/servers/6a391156-ffce-49c0-a79d-47b4544264f5 HTTP/1.1 Jul 16 21:16:31.837210 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:16:31.837210 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:16:31.837210 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:16:31.837210 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 0 Jul 16 21:16:31.837210 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:16:31.837210 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:16:31.837210 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:16:31.837210 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:16:31.837210 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0yjys5urr_ei5Seo_QKDPCqmR0EFBibpmCG26yjPCPljKUNzADf-WX077qzJM5iZPEhFd-9J9wlncEYo9S_mA0fOh-Gvf9LxFXEvyCYlVLvbQiN1HZ19aSAOzuz2ad1WkBxSsEap9oMo0DE5_h7t-ONjtOk159u5RHwmU9Cr_ibnvE8 Jul 16 21:16:31.837210 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:16:31.837210 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:16:31.837210 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:16:31.837210 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:16:31.837210 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:16:31.837210 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:16:31.837210 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: 9b769bf8f3ea4e368bf68de1df6538f9 Jul 16 21:16:31.837210 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-LiveMigrationTest-1001758322 Jul 16 21:16:31.837210 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:16:31.837210 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:16:31.837210 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/9b769bf8f3ea4e368bf68de1df6538f9"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/9b769bf8f3ea4e368bf68de1df6538f9"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_9b769bf8f3ea4e368bf68de1df6538f9"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/9b769bf8f3ea4e368bf68de1df6538f9"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:16:31.837210 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-LiveMigrationTest-1001758322 Jul 16 21:16:31.837210 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: 9b769bf8f3ea4e368bf68de1df6538f9 Jul 16 21:16:31.837210 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-LiveMigrationTest-1001758322 Jul 16 21:16:31.837210 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-LiveMigrationTest-1001758322-project Jul 16 21:16:31.837210 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:16:31.837210 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:16:31.837210 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: 6a69f9dee03d43beaafa966216b9e289 Jul 16 21:16:31.837210 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-LiveMigrationTest-1001758322-project Jul 16 21:16:31.837210 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 21:16:31.837210 ncdv-master-1 devstack@n-api.service[138680]: exception: Instance 6a391156-ffce-49c0-a79d-47b4544264f5 could not be found. Jul 16 21:16:31.837210 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': '6a391156-ffce-49c0-a79d-47b4544264f5'} Jul 16 21:16:31.837210 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:16:31.837210 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:16:31.837210 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:16:31.837210 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:16:31.837210 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:16:31.837210 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:16:31.837210 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:16:31.837210 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:16:31.837210 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:16:31.837210 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:16:31.837210 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 6a391156-ffce-49c0-a79d-47b4544264f5 could not be found. Jul 16 21:16:31.837210 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:16:31.837210 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:16:31.837210 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:16:31.837210 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:16:31.837210 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:16:31.837210 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:16:31.837210 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:16:31.837210 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:16:31.837210 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:16:31.837210 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:16:31.837210 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:16:31.837210 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:16:31.837210 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:16:31.837210 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:16:31.837210 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:16:31.837210 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:16:31.837210 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 6a391156-ffce-49c0-a79d-47b4544264f5 could not be found. Jul 16 21:16:31.837210 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:16:31.837210 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:16:31.837210 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:16:31.844567 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:16:31.844567 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:16:31.844567 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:16:31.844567 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:16:31.844567 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:16:31.844567 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 6a391156-ffce-49c0-a79d-47b4544264f5 could not be found. Jul 16 21:16:31.844567 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:16:31.844567 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-39664fca-1401-40b4-9571-552fb5e6a381 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Returning 404 to user: Instance 6a391156-ffce-49c0-a79d-47b4544264f5 could not be found. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:16:31.844567 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-39664fca-1401-40b4-9571-552fb5e6a381 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] 10.222.0.16 "GET /compute/v2.1/servers/6a391156-ffce-49c0-a79d-47b4544264f5" status: 404 len: 111 microversion: 2.1 time: 0.110369 Jul 16 21:16:31.844567 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1010/1935] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:16:31 2022] GET /compute/v2.1/servers/6a391156-ffce-49c0-a79d-47b4544264f5 => generated 111 bytes in 112 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Jul 16 21:16:31.958244 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-b2299d10-82a0-4934-8b64-8912995d846f tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ImagesNegativeTestJSON-server-874228757", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452", "networks": [{"uuid": "bb478118-ddbd-4c38-9a4f-f3bf823533c9"}]}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:16:32.094303 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.network.neutron [None req-b2299d10-82a0-4934-8b64-8912995d846f tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] validate_networks() for [('bb478118-ddbd-4c38-9a4f-f3bf823533c9', None, None, None, None, None)] {{(pid=138680) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 16 21:16:32.119793 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-c697b4fe-732d-4ddb-a036-80d49dbd6b49 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] [instance: 9657dddc-2416-4c95-aab3-d6b6f9673369] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:32.136610 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c697b4fe-732d-4ddb-a036-80d49dbd6b49 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:32.137069 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c697b4fe-732d-4ddb-a036-80d49dbd6b49 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:32.137859 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c697b4fe-732d-4ddb-a036-80d49dbd6b49 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:32.138297 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c697b4fe-732d-4ddb-a036-80d49dbd6b49 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:32.185744 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-c697b4fe-732d-4ddb-a036-80d49dbd6b49 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] 10.222.0.16 "POST /compute/v2.1/servers/9657dddc-2416-4c95-aab3-d6b6f9673369/action" status: 202 len: 1078 microversion: 2.47 time: 0.756833 Jul 16 21:16:32.188663 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 926/1936] 10.222.0.16 () {62 vars in 1318 bytes} [Sat Jul 16 21:16:31 2022] POST /compute/v2.1/servers/9657dddc-2416-4c95-aab3-d6b6f9673369/action => generated 1078 bytes in 759 msecs (HTTP/1.1 202) 10 headers in 454 bytes (1 switches on core 0) Jul 16 21:16:32.193214 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-4fddd48f-b3d5-4079-883d-891caad36734 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Calling method '.version_select at 0x7fdda2a60b80>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:32.313900 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell0:SELECT=4 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:16:32.359657 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-4fddd48f-b3d5-4079-883d-891caad36734 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] 10.222.0.16 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1244080767" status: 202 len: 0 microversion: 2.1 time: 0.171727 Jul 16 21:16:32.363831 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 927/1937] 10.222.0.16 () {58 vars in 1220 bytes} [Sat Jul 16 21:16:32 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1244080767 => generated 0 bytes in 173 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 21:16:32.365860 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-07e9d8c0-0693-49b1-9424-067c635395a2 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:32.366489 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-07e9d8c0-0693-49b1-9424-067c635395a2 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] [instance: 9657dddc-2416-4c95-aab3-d6b6f9673369] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:32.382040 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-07e9d8c0-0693-49b1-9424-067c635395a2 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:32.382763 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-07e9d8c0-0693-49b1-9424-067c635395a2 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:32.384342 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-07e9d8c0-0693-49b1-9424-067c635395a2 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:32.385141 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-07e9d8c0-0693-49b1-9424-067c635395a2 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:32.586167 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-07e9d8c0-0693-49b1-9424-067c635395a2 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '015ac1bbb2df487e99e5bddc60ee6923', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9f5f03e09936489fa1505188cefe3445', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:32.608136 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-07e9d8c0-0693-49b1-9424-067c635395a2 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '015ac1bbb2df487e99e5bddc60ee6923', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9f5f03e09936489fa1505188cefe3445', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:32.621133 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-07e9d8c0-0693-49b1-9424-067c635395a2 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '015ac1bbb2df487e99e5bddc60ee6923', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9f5f03e09936489fa1505188cefe3445', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:32.623223 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-07e9d8c0-0693-49b1-9424-067c635395a2 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] 10.222.0.16 "GET /compute/v2.1/servers/9657dddc-2416-4c95-aab3-d6b6f9673369" status: 200 len: 1365 microversion: 2.47 time: 0.261559 Jul 16 21:16:32.626382 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 928/1938] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:16:32 2022] GET /compute/v2.1/servers/9657dddc-2416-4c95-aab3-d6b6f9673369 => generated 1365 bytes in 265 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:16:32.644913 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-ba363421-465b-4c8a-840b-eccd8bd1925c tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:32.646024 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-ba363421-465b-4c8a-840b-eccd8bd1925c tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] [instance: f03a2a83-e007-444a-b46b-70eb46bb242b] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:32.663432 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ba363421-465b-4c8a-840b-eccd8bd1925c tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:32.667367 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ba363421-465b-4c8a-840b-eccd8bd1925c tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:32.668552 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ba363421-465b-4c8a-840b-eccd8bd1925c tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:32.670377 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ba363421-465b-4c8a-840b-eccd8bd1925c tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:33.116623 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-b2299d10-82a0-4934-8b64-8912995d846f tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] Flavor limits 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 16 21:16:33.117128 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-b2299d10-82a0-4934-8b64-8912995d846f tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] Image limits 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 16 21:16:33.117623 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-b2299d10-82a0-4934-8b64-8912995d846f tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] Flavor pref 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 16 21:16:33.118117 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-b2299d10-82a0-4934-8b64-8912995d846f tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] Image pref 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 16 21:16:33.118542 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-b2299d10-82a0-4934-8b64-8912995d846f tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 16 21:16:33.141433 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-ba363421-465b-4c8a-840b-eccd8bd1925c tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'daf3c9e6634e4ae1bfb492eae1f44c93', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '24c64db8082648c8837c16352252f96d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:33.152396 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-ba363421-465b-4c8a-840b-eccd8bd1925c tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] 10.222.0.16 "GET /compute/v2.1/servers/f03a2a83-e007-444a-b46b-70eb46bb242b" status: 200 len: 1307 microversion: 2.1 time: 0.524479 Jul 16 21:16:33.153525 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 929/1939] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:16:32 2022] GET /compute/v2.1/servers/f03a2a83-e007-444a-b46b-70eb46bb242b => generated 1307 bytes in 526 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:16:33.159159 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-8ab05932-a5a2-43bf-950d-9da5ac6eaa0d tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:33.160083 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-8ab05932-a5a2-43bf-950d-9da5ac6eaa0d tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] [instance: 11194822-5d42-45f4-99d0-88c70c808ddd] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:33.160233 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b2299d10-82a0-4934-8b64-8912995d846f tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:33.160622 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b2299d10-82a0-4934-8b64-8912995d846f tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:33.161447 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b2299d10-82a0-4934-8b64-8912995d846f tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:33.163673 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b2299d10-82a0-4934-8b64-8912995d846f tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:33.169478 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8ab05932-a5a2-43bf-950d-9da5ac6eaa0d tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:33.174743 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8ab05932-a5a2-43bf-950d-9da5ac6eaa0d tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:33.174743 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8ab05932-a5a2-43bf-950d-9da5ac6eaa0d tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:33.174743 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8ab05932-a5a2-43bf-950d-9da5ac6eaa0d tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:33.214686 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-b2299d10-82a0-4934-8b64-8912995d846f tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] Getting quotas for project 57c1e99ad9884d79812e95fbc81512da. Resources: {'ram', 'instances', 'cores'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:16:33.235671 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-b2299d10-82a0-4934-8b64-8912995d846f tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] Getting quotas for user b7a14721381b4f2cb4def5be76b7b645 and project 57c1e99ad9884d79812e95fbc81512da. Resources: {'ram', 'instances', 'cores'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:16:33.254936 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-b2299d10-82a0-4934-8b64-8912995d846f tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] Going to run 1 instances... {{(pid=138680) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 16 21:16:33.258749 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-b2299d10-82a0-4934-8b64-8912995d846f tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] [instance: e6844cd4-a382-45cc-ab21-bd7f2ede46e2] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138680) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 16 21:16:33.363681 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-8ab05932-a5a2-43bf-950d-9da5ac6eaa0d tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd4144b5388b46a989a41a2f556bb508', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a90d4bcd25ff4bf68bdc194180fbbe09', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:33.372274 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-8ab05932-a5a2-43bf-950d-9da5ac6eaa0d tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] 10.222.0.16 "GET /compute/v2.1/servers/11194822-5d42-45f4-99d0-88c70c808ddd" status: 200 len: 1513 microversion: 2.1 time: 0.217402 Jul 16 21:16:33.373071 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 930/1940] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:16:33 2022] GET /compute/v2.1/servers/11194822-5d42-45f4-99d0-88c70c808ddd => generated 1513 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:16:33.471761 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-b2299d10-82a0-4934-8b64-8912995d846f tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 1.630496 Jul 16 21:16:33.472517 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1011/1941] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 21:16:31 2022] POST /compute/v2.1/servers => generated 380 bytes in 1632 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 16 21:16:33.485086 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-d7af4eb5-f95e-4891-a72b-e784db66ea5e tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:33.486434 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-d7af4eb5-f95e-4891-a72b-e784db66ea5e tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] [instance: e6844cd4-a382-45cc-ab21-bd7f2ede46e2] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:33.565200 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-d7af4eb5-f95e-4891-a72b-e784db66ea5e tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b7a14721381b4f2cb4def5be76b7b645', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '57c1e99ad9884d79812e95fbc81512da', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:33.580879 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-d7af4eb5-f95e-4891-a72b-e784db66ea5e tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] 10.222.0.16 "GET /compute/v2.1/servers/e6844cd4-a382-45cc-ab21-bd7f2ede46e2" status: 200 len: 1215 microversion: 2.1 time: 0.095895 Jul 16 21:16:33.588981 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 931/1942] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:16:33 2022] GET /compute/v2.1/servers/e6844cd4-a382-45cc-ab21-bd7f2ede46e2 => generated 1215 bytes in 109 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:16:33.651578 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-42e623fa-4b37-4756-afb4-7e3e294c5549 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:33.652208 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-42e623fa-4b37-4756-afb4-7e3e294c5549 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] [instance: 9657dddc-2416-4c95-aab3-d6b6f9673369] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:33.667910 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-42e623fa-4b37-4756-afb4-7e3e294c5549 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:33.667910 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-42e623fa-4b37-4756-afb4-7e3e294c5549 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:33.667910 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-42e623fa-4b37-4756-afb4-7e3e294c5549 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:33.667910 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-42e623fa-4b37-4756-afb4-7e3e294c5549 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:33.745335 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-42e623fa-4b37-4756-afb4-7e3e294c5549 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '015ac1bbb2df487e99e5bddc60ee6923', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9f5f03e09936489fa1505188cefe3445', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:33.757946 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-42e623fa-4b37-4756-afb4-7e3e294c5549 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '015ac1bbb2df487e99e5bddc60ee6923', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9f5f03e09936489fa1505188cefe3445', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:33.759626 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-42e623fa-4b37-4756-afb4-7e3e294c5549 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '015ac1bbb2df487e99e5bddc60ee6923', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9f5f03e09936489fa1505188cefe3445', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:33.761164 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-42e623fa-4b37-4756-afb4-7e3e294c5549 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] 10.222.0.16 "GET /compute/v2.1/servers/9657dddc-2416-4c95-aab3-d6b6f9673369" status: 200 len: 1365 microversion: 2.47 time: 0.113461 Jul 16 21:16:33.761962 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1012/1943] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:16:33 2022] GET /compute/v2.1/servers/9657dddc-2416-4c95-aab3-d6b6f9673369 => generated 1365 bytes in 115 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:16:34.173007 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-941f7ebe-760e-4350-8971-b7c75d8fe079 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:34.174152 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-941f7ebe-760e-4350-8971-b7c75d8fe079 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] [instance: f03a2a83-e007-444a-b46b-70eb46bb242b] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:34.186502 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-941f7ebe-760e-4350-8971-b7c75d8fe079 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:34.190079 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-941f7ebe-760e-4350-8971-b7c75d8fe079 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:34.190079 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-941f7ebe-760e-4350-8971-b7c75d8fe079 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:34.190079 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-941f7ebe-760e-4350-8971-b7c75d8fe079 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:34.390145 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-c87497f7-db4a-450e-b49e-216b3ce93d05 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:34.393464 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-c87497f7-db4a-450e-b49e-216b3ce93d05 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] [instance: 11194822-5d42-45f4-99d0-88c70c808ddd] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:34.410148 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c87497f7-db4a-450e-b49e-216b3ce93d05 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:34.411084 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c87497f7-db4a-450e-b49e-216b3ce93d05 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:34.412286 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c87497f7-db4a-450e-b49e-216b3ce93d05 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:34.412993 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c87497f7-db4a-450e-b49e-216b3ce93d05 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:34.443286 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-941f7ebe-760e-4350-8971-b7c75d8fe079 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'daf3c9e6634e4ae1bfb492eae1f44c93', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '24c64db8082648c8837c16352252f96d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:34.452299 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-941f7ebe-760e-4350-8971-b7c75d8fe079 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] 10.222.0.16 "GET /compute/v2.1/servers/f03a2a83-e007-444a-b46b-70eb46bb242b" status: 200 len: 1307 microversion: 2.1 time: 0.284672 Jul 16 21:16:34.452730 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 932/1944] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:16:34 2022] GET /compute/v2.1/servers/f03a2a83-e007-444a-b46b-70eb46bb242b => generated 1307 bytes in 286 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:16:34.535887 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-2c9c3532-35f8-42e4-ba32-c14bbac379e1 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:34.536767 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-2c9c3532-35f8-42e4-ba32-c14bbac379e1 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] [instance: 4ee3ffbe-c16a-4847-a343-469c645e4b25] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:34.546580 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2c9c3532-35f8-42e4-ba32-c14bbac379e1 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:34.548553 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2c9c3532-35f8-42e4-ba32-c14bbac379e1 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:34.549602 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2c9c3532-35f8-42e4-ba32-c14bbac379e1 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:34.550277 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2c9c3532-35f8-42e4-ba32-c14bbac379e1 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:34.575908 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-c87497f7-db4a-450e-b49e-216b3ce93d05 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd4144b5388b46a989a41a2f556bb508', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a90d4bcd25ff4bf68bdc194180fbbe09', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:34.585938 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-c87497f7-db4a-450e-b49e-216b3ce93d05 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] 10.222.0.16 "GET /compute/v2.1/servers/11194822-5d42-45f4-99d0-88c70c808ddd" status: 200 len: 1513 microversion: 2.1 time: 0.200114 Jul 16 21:16:34.586528 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1013/1945] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:16:34 2022] GET /compute/v2.1/servers/11194822-5d42-45f4-99d0-88c70c808ddd => generated 1513 bytes in 201 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:16:34.589465 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-2c9c3532-35f8-42e4-ba32-c14bbac379e1 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] [instance: 4ee3ffbe-c16a-4847-a343-469c645e4b25] Going to try to terminate instance {{(pid=138681) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 16 21:16:34.593128 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-e277125c-dd23-475c-818e-8ad8c1ddc9a7 tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:34.594354 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-e277125c-dd23-475c-818e-8ad8c1ddc9a7 tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] [instance: e6844cd4-a382-45cc-ab21-bd7f2ede46e2] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:34.638605 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:16:34.662305 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-e277125c-dd23-475c-818e-8ad8c1ddc9a7 tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b7a14721381b4f2cb4def5be76b7b645', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '57c1e99ad9884d79812e95fbc81512da', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:34.684132 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-e277125c-dd23-475c-818e-8ad8c1ddc9a7 tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] 10.222.0.16 "GET /compute/v2.1/servers/e6844cd4-a382-45cc-ab21-bd7f2ede46e2" status: 200 len: 1215 microversion: 2.1 time: 0.094769 Jul 16 21:16:34.688438 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1014/1946] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:16:34 2022] GET /compute/v2.1/servers/e6844cd4-a382-45cc-ab21-bd7f2ede46e2 => generated 1215 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:16:34.780459 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-784db52c-c475-4cb8-bd66-1cb5da984377 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:34.790731 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-784db52c-c475-4cb8-bd66-1cb5da984377 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] [instance: 9657dddc-2416-4c95-aab3-d6b6f9673369] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:34.816779 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-784db52c-c475-4cb8-bd66-1cb5da984377 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:34.820543 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-784db52c-c475-4cb8-bd66-1cb5da984377 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:34.821697 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-784db52c-c475-4cb8-bd66-1cb5da984377 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:34.822571 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-784db52c-c475-4cb8-bd66-1cb5da984377 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:34.931781 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-784db52c-c475-4cb8-bd66-1cb5da984377 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '015ac1bbb2df487e99e5bddc60ee6923', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9f5f03e09936489fa1505188cefe3445', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:34.945468 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-784db52c-c475-4cb8-bd66-1cb5da984377 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '015ac1bbb2df487e99e5bddc60ee6923', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9f5f03e09936489fa1505188cefe3445', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:34.950418 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-784db52c-c475-4cb8-bd66-1cb5da984377 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '015ac1bbb2df487e99e5bddc60ee6923', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9f5f03e09936489fa1505188cefe3445', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:34.952242 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-784db52c-c475-4cb8-bd66-1cb5da984377 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] 10.222.0.16 "GET /compute/v2.1/servers/9657dddc-2416-4c95-aab3-d6b6f9673369" status: 200 len: 1365 microversion: 2.47 time: 0.175795 Jul 16 21:16:34.955433 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1015/1947] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:16:34 2022] GET /compute/v2.1/servers/9657dddc-2416-4c95-aab3-d6b6f9673369 => generated 1365 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:16:35.069529 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-2c9c3532-35f8-42e4-ba32-c14bbac379e1 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] 10.222.0.16 "DELETE /compute/v2.1/servers/4ee3ffbe-c16a-4847-a343-469c645e4b25" status: 204 len: 0 microversion: 2.1 time: 0.539237 Jul 16 21:16:35.070397 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 933/1948] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:16:34 2022] DELETE /compute/v2.1/servers/4ee3ffbe-c16a-4847-a343-469c645e4b25 => generated 0 bytes in 542 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Jul 16 21:16:35.079894 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-1df01905-fe4d-4fc7-a958-08371ca017e5 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:35.081033 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-1df01905-fe4d-4fc7-a958-08371ca017e5 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] [instance: 4ee3ffbe-c16a-4847-a343-469c645e4b25] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:35.094354 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-8174cd05-9b79-49da-89b9-f6c6bde0b3be req-d898c832-12a0-416a-b3c2-3a211cadb1bd service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "f03a2a83-e007-444a-b46b-70eb46bb242b", "tag": "1df74ef4-7ed5-4498-8310-01458a7e541a"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:16:35.099984 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1df01905-fe4d-4fc7-a958-08371ca017e5 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:35.101345 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1df01905-fe4d-4fc7-a958-08371ca017e5 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:35.102388 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1df01905-fe4d-4fc7-a958-08371ca017e5 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:35.103482 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1df01905-fe4d-4fc7-a958-08371ca017e5 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:35.116823 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-8174cd05-9b79-49da-89b9-f6c6bde0b3be req-d898c832-12a0-416a-b3c2-3a211cadb1bd service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:35.116823 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-8174cd05-9b79-49da-89b9-f6c6bde0b3be req-d898c832-12a0-416a-b3c2-3a211cadb1bd service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:35.138500 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-8174cd05-9b79-49da-89b9-f6c6bde0b3be req-d898c832-12a0-416a-b3c2-3a211cadb1bd service nova] Creating event network-changed:1df74ef4-7ed5-4498-8310-01458a7e541a for instance f03a2a83-e007-444a-b46b-70eb46bb242b on nch1-master-1 Jul 16 21:16:35.151399 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-8174cd05-9b79-49da-89b9-f6c6bde0b3be req-d898c832-12a0-416a-b3c2-3a211cadb1bd service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.063182 Jul 16 21:16:35.157743 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 934/1949] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:16:35 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 71 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:16:35.412251 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-1df01905-fe4d-4fc7-a958-08371ca017e5 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a69f9dee03d43beaafa966216b9e289', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9b769bf8f3ea4e368bf68de1df6538f9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:35.428715 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-1df01905-fe4d-4fc7-a958-08371ca017e5 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] 10.222.0.16 "GET /compute/v2.1/servers/4ee3ffbe-c16a-4847-a343-469c645e4b25" status: 200 len: 1485 microversion: 2.1 time: 0.353877 Jul 16 21:16:35.432088 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1016/1950] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:16:35 2022] GET /compute/v2.1/servers/4ee3ffbe-c16a-4847-a343-469c645e4b25 => generated 1485 bytes in 356 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:16:35.477102 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-3898f889-4f5f-4b57-acb2-11416338d1cc tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:35.482783 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-3898f889-4f5f-4b57-acb2-11416338d1cc tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] [instance: f03a2a83-e007-444a-b46b-70eb46bb242b] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:35.498429 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell0:SELECT=2 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:16:35.505681 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3898f889-4f5f-4b57-acb2-11416338d1cc tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:35.506302 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3898f889-4f5f-4b57-acb2-11416338d1cc tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:35.507784 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3898f889-4f5f-4b57-acb2-11416338d1cc tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:35.512747 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3898f889-4f5f-4b57-acb2-11416338d1cc tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:35.614056 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-bf6fdbf3-a4b9-4e0f-afb2-b5862e0124f7 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:35.614725 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-bf6fdbf3-a4b9-4e0f-afb2-b5862e0124f7 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] [instance: 11194822-5d42-45f4-99d0-88c70c808ddd] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:35.648328 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-bf6fdbf3-a4b9-4e0f-afb2-b5862e0124f7 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:35.650662 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-bf6fdbf3-a4b9-4e0f-afb2-b5862e0124f7 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:35.654336 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-bf6fdbf3-a4b9-4e0f-afb2-b5862e0124f7 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:35.664635 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-bf6fdbf3-a4b9-4e0f-afb2-b5862e0124f7 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.010s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:35.989093 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-3898f889-4f5f-4b57-acb2-11416338d1cc tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'daf3c9e6634e4ae1bfb492eae1f44c93', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '24c64db8082648c8837c16352252f96d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:36.006108 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-3898f889-4f5f-4b57-acb2-11416338d1cc tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] 10.222.0.16 "GET /compute/v2.1/servers/f03a2a83-e007-444a-b46b-70eb46bb242b" status: 200 len: 1307 microversion: 2.1 time: 0.539842 Jul 16 21:16:36.012193 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 935/1951] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:16:35 2022] GET /compute/v2.1/servers/f03a2a83-e007-444a-b46b-70eb46bb242b => generated 1307 bytes in 544 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:16:36.021736 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-cd2ccbb2-8fb2-485d-818e-15f278266879 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:36.022818 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-cd2ccbb2-8fb2-485d-818e-15f278266879 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] [instance: 3c5ab09e-0a97-4cfc-b249-a11ab221b5d2] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:36.026077 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-bf6fdbf3-a4b9-4e0f-afb2-b5862e0124f7 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd4144b5388b46a989a41a2f556bb508', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a90d4bcd25ff4bf68bdc194180fbbe09', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:36.034423 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cd2ccbb2-8fb2-485d-818e-15f278266879 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:36.035424 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cd2ccbb2-8fb2-485d-818e-15f278266879 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:36.038947 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-bf6fdbf3-a4b9-4e0f-afb2-b5862e0124f7 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] 10.222.0.16 "GET /compute/v2.1/servers/11194822-5d42-45f4-99d0-88c70c808ddd" status: 200 len: 1513 microversion: 2.1 time: 0.431308 Jul 16 21:16:36.040163 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1017/1952] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:16:35 2022] GET /compute/v2.1/servers/11194822-5d42-45f4-99d0-88c70c808ddd => generated 1513 bytes in 433 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:16:36.044957 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-166ab7c0-4861-4a56-99a3-ea1abd30fbd5 tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:36.048927 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-166ab7c0-4861-4a56-99a3-ea1abd30fbd5 tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] [instance: e6844cd4-a382-45cc-ab21-bd7f2ede46e2] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:36.057576 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cd2ccbb2-8fb2-485d-818e-15f278266879 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:36.058386 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cd2ccbb2-8fb2-485d-818e-15f278266879 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:36.061593 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-166ab7c0-4861-4a56-99a3-ea1abd30fbd5 tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:36.066402 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-166ab7c0-4861-4a56-99a3-ea1abd30fbd5 tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:36.068766 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-166ab7c0-4861-4a56-99a3-ea1abd30fbd5 tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:36.073010 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-166ab7c0-4861-4a56-99a3-ea1abd30fbd5 tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:36.107729 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-cd2ccbb2-8fb2-485d-818e-15f278266879 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] [instance: 3c5ab09e-0a97-4cfc-b249-a11ab221b5d2] Going to try to terminate instance {{(pid=138681) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 16 21:16:36.528517 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_api:SELECT=95,nova_api:INSERT=3,nova_api:DELETE=1,nova_api:UPDATE=1 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:16:36.628198 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-cd2ccbb2-8fb2-485d-818e-15f278266879 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] 10.222.0.16 "DELETE /compute/v2.1/servers/3c5ab09e-0a97-4cfc-b249-a11ab221b5d2" status: 204 len: 0 microversion: 2.25 time: 0.614441 Jul 16 21:16:36.635604 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 936/1953] 10.222.0.16 () {60 vars in 1279 bytes} [Sat Jul 16 21:16:36 2022] DELETE /compute/v2.1/servers/3c5ab09e-0a97-4cfc-b249-a11ab221b5d2 => generated 0 bytes in 616 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Jul 16 21:16:36.647410 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-f5e8b858-d3ac-47d1-a341-67b1a0ccdc6b tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:36.648146 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-f5e8b858-d3ac-47d1-a341-67b1a0ccdc6b tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] [instance: 9657dddc-2416-4c95-aab3-d6b6f9673369] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:36.654696 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f5e8b858-d3ac-47d1-a341-67b1a0ccdc6b tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:36.655190 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f5e8b858-d3ac-47d1-a341-67b1a0ccdc6b tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:36.656443 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f5e8b858-d3ac-47d1-a341-67b1a0ccdc6b tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:36.656631 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f5e8b858-d3ac-47d1-a341-67b1a0ccdc6b tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:36.978512 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-166ab7c0-4861-4a56-99a3-ea1abd30fbd5 tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b7a14721381b4f2cb4def5be76b7b645', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '57c1e99ad9884d79812e95fbc81512da', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:36.989162 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-f5e8b858-d3ac-47d1-a341-67b1a0ccdc6b tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '015ac1bbb2df487e99e5bddc60ee6923', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9f5f03e09936489fa1505188cefe3445', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:36.997581 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-f5e8b858-d3ac-47d1-a341-67b1a0ccdc6b tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '015ac1bbb2df487e99e5bddc60ee6923', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9f5f03e09936489fa1505188cefe3445', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:37.004278 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-f5e8b858-d3ac-47d1-a341-67b1a0ccdc6b tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '015ac1bbb2df487e99e5bddc60ee6923', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9f5f03e09936489fa1505188cefe3445', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:37.005303 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-f5e8b858-d3ac-47d1-a341-67b1a0ccdc6b tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] 10.222.0.16 "GET /compute/v2.1/servers/9657dddc-2416-4c95-aab3-d6b6f9673369" status: 200 len: 1365 microversion: 2.47 time: 0.371467 Jul 16 21:16:37.005760 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 937/1954] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:16:36 2022] GET /compute/v2.1/servers/9657dddc-2416-4c95-aab3-d6b6f9673369 => generated 1365 bytes in 373 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:16:37.013177 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-1bc2830f-9600-4a95-8492-8f410ef84e3b tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:37.013804 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-1bc2830f-9600-4a95-8492-8f410ef84e3b tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] [instance: 4ee3ffbe-c16a-4847-a343-469c645e4b25] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:37.021687 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1bc2830f-9600-4a95-8492-8f410ef84e3b tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:37.022418 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-166ab7c0-4861-4a56-99a3-ea1abd30fbd5 tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] 10.222.0.16 "GET /compute/v2.1/servers/e6844cd4-a382-45cc-ab21-bd7f2ede46e2" status: 200 len: 1211 microversion: 2.1 time: 0.980912 Jul 16 21:16:37.022634 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1bc2830f-9600-4a95-8492-8f410ef84e3b tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:37.023608 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1bc2830f-9600-4a95-8492-8f410ef84e3b tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:37.024336 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1bc2830f-9600-4a95-8492-8f410ef84e3b tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:37.036636 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1018/1955] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:16:36 2022] GET /compute/v2.1/servers/e6844cd4-a382-45cc-ab21-bd7f2ede46e2 => generated 1211 bytes in 986 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:16:37.041748 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-332f0ad6-06e6-425e-bcc1-d6083f88c679 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:37.041748 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-332f0ad6-06e6-425e-bcc1-d6083f88c679 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] [instance: 3c5ab09e-0a97-4cfc-b249-a11ab221b5d2] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:37.051248 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-332f0ad6-06e6-425e-bcc1-d6083f88c679 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:37.052178 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-332f0ad6-06e6-425e-bcc1-d6083f88c679 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:37.053320 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-332f0ad6-06e6-425e-bcc1-d6083f88c679 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:37.053779 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-332f0ad6-06e6-425e-bcc1-d6083f88c679 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:37.413550 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-1bc2830f-9600-4a95-8492-8f410ef84e3b tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a69f9dee03d43beaafa966216b9e289', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9b769bf8f3ea4e368bf68de1df6538f9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:37.423360 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-1bc2830f-9600-4a95-8492-8f410ef84e3b tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] 10.222.0.16 "GET /compute/v2.1/servers/4ee3ffbe-c16a-4847-a343-469c645e4b25" status: 200 len: 1485 microversion: 2.1 time: 0.416014 Jul 16 21:16:37.424495 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 938/1956] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:16:37 2022] GET /compute/v2.1/servers/4ee3ffbe-c16a-4847-a343-469c645e4b25 => generated 1485 bytes in 418 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:16:37.429672 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-ca89acf0-42e8-4106-a14a-96a8b97d2618 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:37.430997 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-ca89acf0-42e8-4106-a14a-96a8b97d2618 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] [instance: f03a2a83-e007-444a-b46b-70eb46bb242b] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:37.439052 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ca89acf0-42e8-4106-a14a-96a8b97d2618 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:37.439737 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ca89acf0-42e8-4106-a14a-96a8b97d2618 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:37.440744 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ca89acf0-42e8-4106-a14a-96a8b97d2618 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:37.441609 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ca89acf0-42e8-4106-a14a-96a8b97d2618 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:37.503966 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-332f0ad6-06e6-425e-bcc1-d6083f88c679 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:37.513713 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-332f0ad6-06e6-425e-bcc1-d6083f88c679 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:37.521201 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-332f0ad6-06e6-425e-bcc1-d6083f88c679 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:37.532243 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-332f0ad6-06e6-425e-bcc1-d6083f88c679 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] 10.222.0.16 "GET /compute/v2.1/servers/3c5ab09e-0a97-4cfc-b249-a11ab221b5d2" status: 200 len: 1727 microversion: 2.25 time: 0.494800 Jul 16 21:16:37.532948 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1019/1957] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:16:37 2022] GET /compute/v2.1/servers/3c5ab09e-0a97-4cfc-b249-a11ab221b5d2 => generated 1727 bytes in 505 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:16:37.549043 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-ab7ba4d4-29dc-4ed6-9ecf-c962e1182016 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:37.549965 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-ab7ba4d4-29dc-4ed6-9ecf-c962e1182016 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] [instance: 11194822-5d42-45f4-99d0-88c70c808ddd] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:37.557318 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ab7ba4d4-29dc-4ed6-9ecf-c962e1182016 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:37.558560 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ab7ba4d4-29dc-4ed6-9ecf-c962e1182016 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:37.566742 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ab7ba4d4-29dc-4ed6-9ecf-c962e1182016 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:37.571515 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ab7ba4d4-29dc-4ed6-9ecf-c962e1182016 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.011s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:37.904990 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-ca89acf0-42e8-4106-a14a-96a8b97d2618 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'daf3c9e6634e4ae1bfb492eae1f44c93', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '24c64db8082648c8837c16352252f96d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:37.917526 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-ca89acf0-42e8-4106-a14a-96a8b97d2618 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] 10.222.0.16 "GET /compute/v2.1/servers/f03a2a83-e007-444a-b46b-70eb46bb242b" status: 200 len: 1307 microversion: 2.1 time: 0.491673 Jul 16 21:16:37.929186 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 939/1958] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:16:37 2022] GET /compute/v2.1/servers/f03a2a83-e007-444a-b46b-70eb46bb242b => generated 1307 bytes in 504 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:16:37.934852 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-55107ff0-27bd-4705-92da-442f198ed486 req-9bb2b051-2c28-400b-bfbf-0cc690103bfa service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "4ee3ffbe-c16a-4847-a343-469c645e4b25", "name": "network-vif-deleted", "tag": "f54ad45e-ef5d-4b3e-ae39-d46f5d43bc88"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:16:37.945276 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-55107ff0-27bd-4705-92da-442f198ed486 req-9bb2b051-2c28-400b-bfbf-0cc690103bfa service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:37.946284 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-55107ff0-27bd-4705-92da-442f198ed486 req-9bb2b051-2c28-400b-bfbf-0cc690103bfa service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:37.962692 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-ab7ba4d4-29dc-4ed6-9ecf-c962e1182016 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd4144b5388b46a989a41a2f556bb508', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a90d4bcd25ff4bf68bdc194180fbbe09', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:37.965233 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-55107ff0-27bd-4705-92da-442f198ed486 req-9bb2b051-2c28-400b-bfbf-0cc690103bfa service nova] Creating event network-vif-deleted:f54ad45e-ef5d-4b3e-ae39-d46f5d43bc88 for instance 4ee3ffbe-c16a-4847-a343-469c645e4b25 on nch2-master-1 Jul 16 21:16:37.971475 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-55107ff0-27bd-4705-92da-442f198ed486 req-9bb2b051-2c28-400b-bfbf-0cc690103bfa service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.040404 Jul 16 21:16:37.972380 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 940/1959] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:16:37 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:16:37.982483 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-ab7ba4d4-29dc-4ed6-9ecf-c962e1182016 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] 10.222.0.16 "GET /compute/v2.1/servers/11194822-5d42-45f4-99d0-88c70c808ddd" status: 200 len: 1513 microversion: 2.1 time: 0.436401 Jul 16 21:16:37.984680 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1020/1960] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:16:37 2022] GET /compute/v2.1/servers/11194822-5d42-45f4-99d0-88c70c808ddd => generated 1513 bytes in 441 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:16:38.027300 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-1e2146a0-fe26-4af9-8d25-7a9b0e9275fa tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:38.028016 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-1e2146a0-fe26-4af9-8d25-7a9b0e9275fa tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] [instance: 9657dddc-2416-4c95-aab3-d6b6f9673369] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:38.036806 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1e2146a0-fe26-4af9-8d25-7a9b0e9275fa tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:38.037364 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1e2146a0-fe26-4af9-8d25-7a9b0e9275fa tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:38.038356 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1e2146a0-fe26-4af9-8d25-7a9b0e9275fa tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:38.038803 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1e2146a0-fe26-4af9-8d25-7a9b0e9275fa tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:38.049606 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-02fd483b-ef66-49c7-9601-b607d6b8d890 tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:38.050236 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-02fd483b-ef66-49c7-9601-b607d6b8d890 tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] [instance: e6844cd4-a382-45cc-ab21-bd7f2ede46e2] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:38.056696 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-02fd483b-ef66-49c7-9601-b607d6b8d890 tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:38.057159 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-02fd483b-ef66-49c7-9601-b607d6b8d890 tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:38.057939 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-02fd483b-ef66-49c7-9601-b607d6b8d890 tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:38.058444 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-02fd483b-ef66-49c7-9601-b607d6b8d890 tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:38.164801 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-1e2146a0-fe26-4af9-8d25-7a9b0e9275fa tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '015ac1bbb2df487e99e5bddc60ee6923', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9f5f03e09936489fa1505188cefe3445', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:38.175387 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-1e2146a0-fe26-4af9-8d25-7a9b0e9275fa tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '015ac1bbb2df487e99e5bddc60ee6923', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9f5f03e09936489fa1505188cefe3445', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:38.178202 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-1e2146a0-fe26-4af9-8d25-7a9b0e9275fa tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '015ac1bbb2df487e99e5bddc60ee6923', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9f5f03e09936489fa1505188cefe3445', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:38.180547 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-1e2146a0-fe26-4af9-8d25-7a9b0e9275fa tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] 10.222.0.16 "GET /compute/v2.1/servers/9657dddc-2416-4c95-aab3-d6b6f9673369" status: 200 len: 1365 microversion: 2.47 time: 0.157077 Jul 16 21:16:38.181607 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 941/1961] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:16:38 2022] GET /compute/v2.1/servers/9657dddc-2416-4c95-aab3-d6b6f9673369 => generated 1365 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:16:38.184384 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-02fd483b-ef66-49c7-9601-b607d6b8d890 tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b7a14721381b4f2cb4def5be76b7b645', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '57c1e99ad9884d79812e95fbc81512da', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:38.205918 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-02fd483b-ef66-49c7-9601-b607d6b8d890 tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] 10.222.0.16 "GET /compute/v2.1/servers/e6844cd4-a382-45cc-ab21-bd7f2ede46e2" status: 200 len: 1275 microversion: 2.1 time: 0.160125 Jul 16 21:16:38.206878 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1021/1962] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:16:38 2022] GET /compute/v2.1/servers/e6844cd4-a382-45cc-ab21-bd7f2ede46e2 => generated 1275 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:16:38.446909 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-42693aa4-14b3-42fe-ad8d-e4434a1cd836 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:38.447568 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-42693aa4-14b3-42fe-ad8d-e4434a1cd836 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] [instance: 4ee3ffbe-c16a-4847-a343-469c645e4b25] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:38.454604 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-42693aa4-14b3-42fe-ad8d-e4434a1cd836 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:38.455557 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-42693aa4-14b3-42fe-ad8d-e4434a1cd836 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:38.456334 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-42693aa4-14b3-42fe-ad8d-e4434a1cd836 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:38.456782 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-42693aa4-14b3-42fe-ad8d-e4434a1cd836 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:38.502929 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-42693aa4-14b3-42fe-ad8d-e4434a1cd836 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:38.503413 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-42693aa4-14b3-42fe-ad8d-e4434a1cd836 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:38.503881 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.objects.instance [None req-42693aa4-14b3-42fe-ad8d-e4434a1cd836 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lazy-loading 'fault' on Instance uuid 4ee3ffbe-c16a-4847-a343-469c645e4b25 {{(pid=138681) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:16:38.552196 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-3b766e65-0229-45a0-9af2-11a1d7a18aad tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:38.553309 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-3b766e65-0229-45a0-9af2-11a1d7a18aad tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] [instance: 3c5ab09e-0a97-4cfc-b249-a11ab221b5d2] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:38.564311 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3b766e65-0229-45a0-9af2-11a1d7a18aad tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:38.568844 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3b766e65-0229-45a0-9af2-11a1d7a18aad tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:38.574341 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3b766e65-0229-45a0-9af2-11a1d7a18aad tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:38.575366 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3b766e65-0229-45a0-9af2-11a1d7a18aad tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:38.614846 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-42693aa4-14b3-42fe-ad8d-e4434a1cd836 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a69f9dee03d43beaafa966216b9e289', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9b769bf8f3ea4e368bf68de1df6538f9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:38.627470 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-42693aa4-14b3-42fe-ad8d-e4434a1cd836 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] 10.222.0.16 "GET /compute/v2.1/servers/4ee3ffbe-c16a-4847-a343-469c645e4b25" status: 200 len: 1301 microversion: 2.1 time: 0.185421 Jul 16 21:16:38.632124 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 942/1963] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:16:38 2022] GET /compute/v2.1/servers/4ee3ffbe-c16a-4847-a343-469c645e4b25 => generated 1301 bytes in 191 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:16:38.690884 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-3b766e65-0229-45a0-9af2-11a1d7a18aad tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:38.700769 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-3b766e65-0229-45a0-9af2-11a1d7a18aad tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:38.702706 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-3b766e65-0229-45a0-9af2-11a1d7a18aad tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:38.706760 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-3b766e65-0229-45a0-9af2-11a1d7a18aad tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] 10.222.0.16 "GET /compute/v2.1/servers/3c5ab09e-0a97-4cfc-b249-a11ab221b5d2" status: 200 len: 1661 microversion: 2.25 time: 0.160837 Jul 16 21:16:38.707906 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1022/1964] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:16:38 2022] GET /compute/v2.1/servers/3c5ab09e-0a97-4cfc-b249-a11ab221b5d2 => generated 1661 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 16 21:16:39.165798 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-4ada152b-16a8-48da-99da-38a031b75e32 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:39.165798 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-4ada152b-16a8-48da-99da-38a031b75e32 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] [instance: f03a2a83-e007-444a-b46b-70eb46bb242b] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:39.165798 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4ada152b-16a8-48da-99da-38a031b75e32 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:39.165798 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4ada152b-16a8-48da-99da-38a031b75e32 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:39.165798 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4ada152b-16a8-48da-99da-38a031b75e32 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:39.165798 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4ada152b-16a8-48da-99da-38a031b75e32 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:39.169141 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-640619c1-bfa6-4f45-a6e1-eb9d6b70b6e1 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:39.169141 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-640619c1-bfa6-4f45-a6e1-eb9d6b70b6e1 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] [instance: 11194822-5d42-45f4-99d0-88c70c808ddd] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:39.169141 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-640619c1-bfa6-4f45-a6e1-eb9d6b70b6e1 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:39.169141 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-640619c1-bfa6-4f45-a6e1-eb9d6b70b6e1 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.009s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:39.169141 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-640619c1-bfa6-4f45-a6e1-eb9d6b70b6e1 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:39.169141 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-640619c1-bfa6-4f45-a6e1-eb9d6b70b6e1 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:39.246506 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-4ada152b-16a8-48da-99da-38a031b75e32 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'daf3c9e6634e4ae1bfb492eae1f44c93', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '24c64db8082648c8837c16352252f96d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:39.265231 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-4ada152b-16a8-48da-99da-38a031b75e32 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] 10.222.0.16 "GET /compute/v2.1/servers/f03a2a83-e007-444a-b46b-70eb46bb242b" status: 200 len: 1307 microversion: 2.1 time: 0.333637 Jul 16 21:16:39.270525 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 943/1965] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:16:38 2022] GET /compute/v2.1/servers/f03a2a83-e007-444a-b46b-70eb46bb242b => generated 1307 bytes in 340 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:16:39.285180 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-aeabed1a-ac39-46c5-a65b-e9c38dfec196 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:39.285853 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-aeabed1a-ac39-46c5-a65b-e9c38dfec196 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] [instance: 9657dddc-2416-4c95-aab3-d6b6f9673369] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:39.295541 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-aeabed1a-ac39-46c5-a65b-e9c38dfec196 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:39.296183 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-aeabed1a-ac39-46c5-a65b-e9c38dfec196 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:39.297905 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-aeabed1a-ac39-46c5-a65b-e9c38dfec196 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:39.299057 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-aeabed1a-ac39-46c5-a65b-e9c38dfec196 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:39.339482 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-640619c1-bfa6-4f45-a6e1-eb9d6b70b6e1 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd4144b5388b46a989a41a2f556bb508', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a90d4bcd25ff4bf68bdc194180fbbe09', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:39.349287 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-640619c1-bfa6-4f45-a6e1-eb9d6b70b6e1 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] 10.222.0.16 "GET /compute/v2.1/servers/11194822-5d42-45f4-99d0-88c70c808ddd" status: 200 len: 1513 microversion: 2.1 time: 0.351163 Jul 16 21:16:39.360176 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1023/1966] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:16:38 2022] GET /compute/v2.1/servers/11194822-5d42-45f4-99d0-88c70c808ddd => generated 1513 bytes in 363 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:16:39.368407 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-78f951db-d4c9-4294-8a11-0436f76538a5 tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:39.369369 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-78f951db-d4c9-4294-8a11-0436f76538a5 tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] [instance: e6844cd4-a382-45cc-ab21-bd7f2ede46e2] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:39.410374 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-78f951db-d4c9-4294-8a11-0436f76538a5 tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:39.412826 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-78f951db-d4c9-4294-8a11-0436f76538a5 tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:39.413477 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-aeabed1a-ac39-46c5-a65b-e9c38dfec196 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '015ac1bbb2df487e99e5bddc60ee6923', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9f5f03e09936489fa1505188cefe3445', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:39.414211 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-78f951db-d4c9-4294-8a11-0436f76538a5 tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:39.424740 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-78f951db-d4c9-4294-8a11-0436f76538a5 tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:39.430979 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-aeabed1a-ac39-46c5-a65b-e9c38dfec196 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '015ac1bbb2df487e99e5bddc60ee6923', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9f5f03e09936489fa1505188cefe3445', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:39.439636 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-aeabed1a-ac39-46c5-a65b-e9c38dfec196 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '015ac1bbb2df487e99e5bddc60ee6923', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9f5f03e09936489fa1505188cefe3445', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:39.450703 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-aeabed1a-ac39-46c5-a65b-e9c38dfec196 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] 10.222.0.16 "GET /compute/v2.1/servers/9657dddc-2416-4c95-aab3-d6b6f9673369" status: 200 len: 1365 microversion: 2.47 time: 0.177314 Jul 16 21:16:39.451863 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 944/1967] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:16:39 2022] GET /compute/v2.1/servers/9657dddc-2416-4c95-aab3-d6b6f9673369 => generated 1365 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:16:39.534405 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-78f951db-d4c9-4294-8a11-0436f76538a5 tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b7a14721381b4f2cb4def5be76b7b645', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '57c1e99ad9884d79812e95fbc81512da', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:39.544509 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-78f951db-d4c9-4294-8a11-0436f76538a5 tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] 10.222.0.16 "GET /compute/v2.1/servers/e6844cd4-a382-45cc-ab21-bd7f2ede46e2" status: 200 len: 1273 microversion: 2.1 time: 0.180140 Jul 16 21:16:39.545524 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1024/1968] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:16:39 2022] GET /compute/v2.1/servers/e6844cd4-a382-45cc-ab21-bd7f2ede46e2 => generated 1273 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:16:39.650689 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-d987af84-ac58-49d6-a5e6-45832c5c8111 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:39.651831 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-d987af84-ac58-49d6-a5e6-45832c5c8111 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] [instance: 4ee3ffbe-c16a-4847-a343-469c645e4b25] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:39.671888 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d987af84-ac58-49d6-a5e6-45832c5c8111 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:39.671888 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d987af84-ac58-49d6-a5e6-45832c5c8111 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:39.671888 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d987af84-ac58-49d6-a5e6-45832c5c8111 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:39.671888 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d987af84-ac58-49d6-a5e6-45832c5c8111 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:39.726486 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-d987af84-ac58-49d6-a5e6-45832c5c8111 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] HTTP exception thrown: Instance 4ee3ffbe-c16a-4847-a343-469c645e4b25 could not be found. Jul 16 21:16:39.726486 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-d987af84-ac58-49d6-a5e6-45832c5c8111 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Request method failure captured: Jul 16 21:16:39.726486 ncdv-master-1 devstack@n-api.service[138681]: request: GET /compute/v2.1/servers/4ee3ffbe-c16a-4847-a343-469c645e4b25 HTTP/1.1 Jul 16 21:16:39.726486 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:16:39.726486 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:16:39.726486 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:16:39.726486 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 21:16:39.726486 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:16:39.726486 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:16:39.726486 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:16:39.726486 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:16:39.726486 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0yjys5urr_ei5Seo_QKDPCqmR0EFBibpmCG26yjPCPljKUNzADf-WX077qzJM5iZPEhFd-9J9wlncEYo9S_mA0fOh-Gvf9LxFXEvyCYlVLvbQiN1HZ19aSAOzuz2ad1WkBxSsEap9oMo0DE5_h7t-ONjtOk159u5RHwmU9Cr_ibnvE8 Jul 16 21:16:39.726486 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:16:39.726486 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:16:39.726486 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:16:39.726486 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:16:39.726486 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:16:39.726486 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:16:39.726486 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: 9b769bf8f3ea4e368bf68de1df6538f9 Jul 16 21:16:39.726486 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-LiveMigrationTest-1001758322 Jul 16 21:16:39.726486 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:16:39.726486 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:16:39.726486 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/9b769bf8f3ea4e368bf68de1df6538f9"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/9b769bf8f3ea4e368bf68de1df6538f9"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_9b769bf8f3ea4e368bf68de1df6538f9"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/9b769bf8f3ea4e368bf68de1df6538f9"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:16:39.726486 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-LiveMigrationTest-1001758322 Jul 16 21:16:39.726486 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: 9b769bf8f3ea4e368bf68de1df6538f9 Jul 16 21:16:39.726486 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-LiveMigrationTest-1001758322 Jul 16 21:16:39.726486 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-LiveMigrationTest-1001758322-project Jul 16 21:16:39.726486 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:16:39.726486 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:16:39.726486 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: 6a69f9dee03d43beaafa966216b9e289 Jul 16 21:16:39.726486 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-LiveMigrationTest-1001758322-project Jul 16 21:16:39.726486 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:16:39.726486 ncdv-master-1 devstack@n-api.service[138681]: exception: Instance 4ee3ffbe-c16a-4847-a343-469c645e4b25 could not be found. Jul 16 21:16:39.726486 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': '4ee3ffbe-c16a-4847-a343-469c645e4b25'} Jul 16 21:16:39.726486 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:16:39.726486 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:16:39.726486 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:16:39.726486 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:16:39.726486 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:16:39.726486 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:16:39.726486 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:16:39.726486 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:16:39.726486 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:16:39.726486 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:16:39.726486 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 4ee3ffbe-c16a-4847-a343-469c645e4b25 could not be found. Jul 16 21:16:39.726486 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:16:39.726486 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:16:39.726486 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:16:39.726486 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:16:39.726486 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:16:39.726486 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:16:39.726486 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:16:39.726486 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:16:39.726486 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:16:39.726486 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:16:39.726486 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:16:39.726486 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:16:39.726486 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:16:39.726486 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:16:39.726486 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:16:39.726486 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:16:39.732369 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 4ee3ffbe-c16a-4847-a343-469c645e4b25 could not be found. Jul 16 21:16:39.732369 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:16:39.732369 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:16:39.732369 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:16:39.732369 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:16:39.732369 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:16:39.732369 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:16:39.732369 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:16:39.732369 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:16:39.732369 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 4ee3ffbe-c16a-4847-a343-469c645e4b25 could not be found. Jul 16 21:16:39.732369 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:16:39.732369 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-d987af84-ac58-49d6-a5e6-45832c5c8111 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Returning 404 to user: Instance 4ee3ffbe-c16a-4847-a343-469c645e4b25 could not be found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:16:39.732369 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-d987af84-ac58-49d6-a5e6-45832c5c8111 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] 10.222.0.16 "GET /compute/v2.1/servers/4ee3ffbe-c16a-4847-a343-469c645e4b25" status: 404 len: 111 microversion: 2.1 time: 0.070175 Jul 16 21:16:39.732369 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 945/1969] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:16:39 2022] GET /compute/v2.1/servers/4ee3ffbe-c16a-4847-a343-469c645e4b25 => generated 111 bytes in 73 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:16:39.733550 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-50e4ffa8-fedc-4db5-bbb9-d38b50980971 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:39.733550 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-50e4ffa8-fedc-4db5-bbb9-d38b50980971 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] [instance: 3c5ab09e-0a97-4cfc-b249-a11ab221b5d2] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:39.745128 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-a3672f00-b912-4847-bc3a-f205551fac39 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:39.746647 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-50e4ffa8-fedc-4db5-bbb9-d38b50980971 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:39.747451 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-a3672f00-b912-4847-bc3a-f205551fac39 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] [instance: c40ba20d-eb88-4ee8-be8f-ecbbd3315c48] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:39.750241 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-50e4ffa8-fedc-4db5-bbb9-d38b50980971 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:39.759416 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-50e4ffa8-fedc-4db5-bbb9-d38b50980971 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:39.759416 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-50e4ffa8-fedc-4db5-bbb9-d38b50980971 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:39.775423 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a3672f00-b912-4847-bc3a-f205551fac39 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:39.775795 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a3672f00-b912-4847-bc3a-f205551fac39 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:39.788178 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a3672f00-b912-4847-bc3a-f205551fac39 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:39.788684 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a3672f00-b912-4847-bc3a-f205551fac39 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.012s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:39.852901 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-a3672f00-b912-4847-bc3a-f205551fac39 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] [instance: c40ba20d-eb88-4ee8-be8f-ecbbd3315c48] Going to try to terminate instance {{(pid=138681) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 16 21:16:39.882498 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-50e4ffa8-fedc-4db5-bbb9-d38b50980971 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:39.884277 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-50e4ffa8-fedc-4db5-bbb9-d38b50980971 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:39.887651 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.objects.instance [None req-50e4ffa8-fedc-4db5-bbb9-d38b50980971 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lazy-loading 'fault' on Instance uuid 3c5ab09e-0a97-4cfc-b249-a11ab221b5d2 {{(pid=138680) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:16:39.976390 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-50e4ffa8-fedc-4db5-bbb9-d38b50980971 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:39.992157 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-50e4ffa8-fedc-4db5-bbb9-d38b50980971 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:39.997767 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-50e4ffa8-fedc-4db5-bbb9-d38b50980971 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:40.003047 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-50e4ffa8-fedc-4db5-bbb9-d38b50980971 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] 10.222.0.16 "GET /compute/v2.1/servers/3c5ab09e-0a97-4cfc-b249-a11ab221b5d2" status: 200 len: 1374 microversion: 2.25 time: 0.281069 Jul 16 21:16:40.003762 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1025/1970] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:16:39 2022] GET /compute/v2.1/servers/3c5ab09e-0a97-4cfc-b249-a11ab221b5d2 => generated 1374 bytes in 282 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:16:40.018628 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-c07209d0-680a-4d63-af2a-3bb5005fad85 req-788884aa-e4ed-4cae-809a-d6b723f0fcea service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "3c5ab09e-0a97-4cfc-b249-a11ab221b5d2", "name": "network-vif-deleted", "tag": "4f9dfe87-5bed-49a6-8b89-464c4930e746"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:16:40.034689 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-c07209d0-680a-4d63-af2a-3bb5005fad85 req-788884aa-e4ed-4cae-809a-d6b723f0fcea service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:40.035210 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-c07209d0-680a-4d63-af2a-3bb5005fad85 req-788884aa-e4ed-4cae-809a-d6b723f0fcea service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:40.039532 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=100 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:16:40.047700 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-c07209d0-680a-4d63-af2a-3bb5005fad85 req-788884aa-e4ed-4cae-809a-d6b723f0fcea service nova] Creating event network-vif-deleted:4f9dfe87-5bed-49a6-8b89-464c4930e746 for instance 3c5ab09e-0a97-4cfc-b249-a11ab221b5d2 on nch1-master-1 Jul 16 21:16:40.063704 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-c07209d0-680a-4d63-af2a-3bb5005fad85 req-788884aa-e4ed-4cae-809a-d6b723f0fcea service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.054100 Jul 16 21:16:40.071846 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1026/1971] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:16:40 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 65 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:16:40.289042 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-6ad6eaa2-dc72-4a25-b062-0b84c6dd0276 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:40.291316 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-6ad6eaa2-dc72-4a25-b062-0b84c6dd0276 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] [instance: f03a2a83-e007-444a-b46b-70eb46bb242b] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:40.304550 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6ad6eaa2-dc72-4a25-b062-0b84c6dd0276 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:40.306044 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6ad6eaa2-dc72-4a25-b062-0b84c6dd0276 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:40.307409 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6ad6eaa2-dc72-4a25-b062-0b84c6dd0276 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:40.308611 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6ad6eaa2-dc72-4a25-b062-0b84c6dd0276 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:40.365043 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-a3672f00-b912-4847-bc3a-f205551fac39 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] 10.222.0.16 "DELETE /compute/v2.1/servers/c40ba20d-eb88-4ee8-be8f-ecbbd3315c48" status: 204 len: 0 microversion: 2.1 time: 0.633512 Jul 16 21:16:40.366601 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 946/1972] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:16:39 2022] DELETE /compute/v2.1/servers/c40ba20d-eb88-4ee8-be8f-ecbbd3315c48 => generated 0 bytes in 637 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 16 21:16:40.374250 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-463c5f3c-fc17-4c9e-9fd5-f8e67d0806f2 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:40.377339 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-463c5f3c-fc17-4c9e-9fd5-f8e67d0806f2 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] [instance: 11194822-5d42-45f4-99d0-88c70c808ddd] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:40.385560 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-463c5f3c-fc17-4c9e-9fd5-f8e67d0806f2 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:40.386359 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-463c5f3c-fc17-4c9e-9fd5-f8e67d0806f2 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:40.389203 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-463c5f3c-fc17-4c9e-9fd5-f8e67d0806f2 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:40.391806 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-463c5f3c-fc17-4c9e-9fd5-f8e67d0806f2 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:40.542380 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-6ad6eaa2-dc72-4a25-b062-0b84c6dd0276 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'daf3c9e6634e4ae1bfb492eae1f44c93', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '24c64db8082648c8837c16352252f96d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:40.569797 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-6ad6eaa2-dc72-4a25-b062-0b84c6dd0276 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] 10.222.0.16 "GET /compute/v2.1/servers/f03a2a83-e007-444a-b46b-70eb46bb242b" status: 200 len: 1307 microversion: 2.1 time: 0.285872 Jul 16 21:16:40.578154 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1027/1973] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:16:40 2022] GET /compute/v2.1/servers/f03a2a83-e007-444a-b46b-70eb46bb242b => generated 1307 bytes in 295 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:16:40.586808 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-5abf5918-2f10-4302-b7e8-29e7cc7786d9 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:40.587613 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-5abf5918-2f10-4302-b7e8-29e7cc7786d9 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] [instance: c40ba20d-eb88-4ee8-be8f-ecbbd3315c48] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:40.595464 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5abf5918-2f10-4302-b7e8-29e7cc7786d9 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:40.595923 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5abf5918-2f10-4302-b7e8-29e7cc7786d9 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:40.596719 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5abf5918-2f10-4302-b7e8-29e7cc7786d9 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:40.597160 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5abf5918-2f10-4302-b7e8-29e7cc7786d9 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:40.603789 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-463c5f3c-fc17-4c9e-9fd5-f8e67d0806f2 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd4144b5388b46a989a41a2f556bb508', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a90d4bcd25ff4bf68bdc194180fbbe09', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:40.608589 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:UPDATE=3,nova_cell1:SELECT=94,nova_cell1:INSERT=3 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:16:40.617163 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-463c5f3c-fc17-4c9e-9fd5-f8e67d0806f2 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] 10.222.0.16 "GET /compute/v2.1/servers/11194822-5d42-45f4-99d0-88c70c808ddd" status: 200 len: 1513 microversion: 2.1 time: 0.248852 Jul 16 21:16:40.617757 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 947/1974] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:16:40 2022] GET /compute/v2.1/servers/11194822-5d42-45f4-99d0-88c70c808ddd => generated 1513 bytes in 250 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:16:40.643240 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-d9b1a3d8-1120-4641-83bc-d7d330a4b718 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:40.644157 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-d9b1a3d8-1120-4641-83bc-d7d330a4b718 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] [instance: 9657dddc-2416-4c95-aab3-d6b6f9673369] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:40.651503 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d9b1a3d8-1120-4641-83bc-d7d330a4b718 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:40.653316 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d9b1a3d8-1120-4641-83bc-d7d330a4b718 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:40.654167 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d9b1a3d8-1120-4641-83bc-d7d330a4b718 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:40.663791 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d9b1a3d8-1120-4641-83bc-d7d330a4b718 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:40.849640 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-d9b1a3d8-1120-4641-83bc-d7d330a4b718 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '015ac1bbb2df487e99e5bddc60ee6923', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9f5f03e09936489fa1505188cefe3445', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:40.865102 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-d9b1a3d8-1120-4641-83bc-d7d330a4b718 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '015ac1bbb2df487e99e5bddc60ee6923', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9f5f03e09936489fa1505188cefe3445', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:40.867540 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-d9b1a3d8-1120-4641-83bc-d7d330a4b718 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '015ac1bbb2df487e99e5bddc60ee6923', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9f5f03e09936489fa1505188cefe3445', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:40.888249 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-d9b1a3d8-1120-4641-83bc-d7d330a4b718 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] 10.222.0.16 "GET /compute/v2.1/servers/9657dddc-2416-4c95-aab3-d6b6f9673369" status: 200 len: 1365 microversion: 2.47 time: 0.258590 Jul 16 21:16:40.890014 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 948/1975] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:16:40 2022] GET /compute/v2.1/servers/9657dddc-2416-4c95-aab3-d6b6f9673369 => generated 1365 bytes in 262 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:16:40.918507 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-f2e373ac-f095-42b3-850b-50e6cc3eb407 tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:40.920183 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-f2e373ac-f095-42b3-850b-50e6cc3eb407 tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] [instance: e6844cd4-a382-45cc-ab21-bd7f2ede46e2] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:40.940129 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f2e373ac-f095-42b3-850b-50e6cc3eb407 tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:40.940500 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f2e373ac-f095-42b3-850b-50e6cc3eb407 tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:40.941527 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f2e373ac-f095-42b3-850b-50e6cc3eb407 tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:40.941984 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f2e373ac-f095-42b3-850b-50e6cc3eb407 tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:40.994119 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-5abf5918-2f10-4302-b7e8-29e7cc7786d9 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a69f9dee03d43beaafa966216b9e289', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9b769bf8f3ea4e368bf68de1df6538f9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:41.011188 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-5abf5918-2f10-4302-b7e8-29e7cc7786d9 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] 10.222.0.16 "GET /compute/v2.1/servers/c40ba20d-eb88-4ee8-be8f-ecbbd3315c48" status: 200 len: 1499 microversion: 2.1 time: 0.423739 Jul 16 21:16:41.011878 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1028/1976] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:16:40 2022] GET /compute/v2.1/servers/c40ba20d-eb88-4ee8-be8f-ecbbd3315c48 => generated 1499 bytes in 433 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:16:41.025305 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-b2f3d33b-90b6-49a3-8128-0521b9754692 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:41.026245 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-b2f3d33b-90b6-49a3-8128-0521b9754692 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] [instance: 3c5ab09e-0a97-4cfc-b249-a11ab221b5d2] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:41.033927 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b2f3d33b-90b6-49a3-8128-0521b9754692 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:41.034755 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b2f3d33b-90b6-49a3-8128-0521b9754692 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:41.035791 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b2f3d33b-90b6-49a3-8128-0521b9754692 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:41.036419 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b2f3d33b-90b6-49a3-8128-0521b9754692 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:41.055327 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-b2f3d33b-90b6-49a3-8128-0521b9754692 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] HTTP exception thrown: Instance 3c5ab09e-0a97-4cfc-b249-a11ab221b5d2 could not be found. Jul 16 21:16:41.057326 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-b2f3d33b-90b6-49a3-8128-0521b9754692 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Request method failure captured: Jul 16 21:16:41.057326 ncdv-master-1 devstack@n-api.service[138680]: request: GET /compute/v2.1/servers/3c5ab09e-0a97-4cfc-b249-a11ab221b5d2 HTTP/1.1 Jul 16 21:16:41.057326 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:16:41.057326 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:16:41.057326 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:16:41.057326 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 0 Jul 16 21:16:41.057326 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:16:41.057326 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:16:41.057326 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:16:41.057326 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:16:41.057326 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0ylBOHnbwqerAHhqONpFSBMhwk4nWfHPAHc-OTgMvb17y3jQh2-2K7JnAKTp7DvWJ_uO-bYr8xJfvvOUSkF97J9lR0DMJJ8o4htp9CLMoM61BjIIA0xJtMBeTA8oUgOHWMLAymuEaOlZbY4tC3l04tVeZqVEgsWsWeWmyuuhOz7GDcE Jul 16 21:16:41.057326 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:16:41.057326 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:16:41.057326 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:16:41.057326 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:16:41.057326 ncdv-master-1 devstack@n-api.service[138680]: X-Openstack-Nova-Api-Version: 2.25 Jul 16 21:16:41.057326 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:16:41.057326 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:16:41.057326 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: d9b92fc544c544f686ac2ab7ff5123b6 Jul 16 21:16:41.057326 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-LiveAutoBlockMigrationV225Test-1461646193 Jul 16 21:16:41.057326 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:16:41.057326 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:16:41.057326 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/d9b92fc544c544f686ac2ab7ff5123b6"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/d9b92fc544c544f686ac2ab7ff5123b6"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_d9b92fc544c544f686ac2ab7ff5123b6"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/d9b92fc544c544f686ac2ab7ff5123b6"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:16:41.057326 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-LiveAutoBlockMigrationV225Test-1461646193 Jul 16 21:16:41.057326 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: d9b92fc544c544f686ac2ab7ff5123b6 Jul 16 21:16:41.057326 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-LiveAutoBlockMigrationV225Test-1461646193 Jul 16 21:16:41.057326 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-LiveAutoBlockMigrationV225Test-1461646193-project Jul 16 21:16:41.057326 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:16:41.057326 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:16:41.057326 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: 578b87f863b34f57a571f7e56d44f118 Jul 16 21:16:41.057326 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-LiveAutoBlockMigrationV225Test-1461646193-project Jul 16 21:16:41.057326 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 21:16:41.057326 ncdv-master-1 devstack@n-api.service[138680]: exception: Instance 3c5ab09e-0a97-4cfc-b249-a11ab221b5d2 could not be found. Jul 16 21:16:41.057326 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': '3c5ab09e-0a97-4cfc-b249-a11ab221b5d2'} Jul 16 21:16:41.057326 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:16:41.057326 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:16:41.057326 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:16:41.057326 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:16:41.057326 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:16:41.057326 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:16:41.057326 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:16:41.057326 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:16:41.057326 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:16:41.057326 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:16:41.057326 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 3c5ab09e-0a97-4cfc-b249-a11ab221b5d2 could not be found. Jul 16 21:16:41.057326 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:16:41.057326 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:16:41.057326 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:16:41.057326 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:16:41.057326 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:16:41.057326 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:16:41.057326 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:16:41.057326 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:16:41.057326 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:16:41.057326 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:16:41.057326 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:16:41.057326 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:16:41.057326 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:16:41.057326 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:16:41.057326 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:16:41.057326 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:16:41.057326 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 3c5ab09e-0a97-4cfc-b249-a11ab221b5d2 could not be found. Jul 16 21:16:41.057326 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:16:41.061892 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:16:41.061892 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:16:41.061892 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:16:41.061892 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:16:41.061892 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:16:41.061892 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:16:41.061892 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:16:41.061892 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 3c5ab09e-0a97-4cfc-b249-a11ab221b5d2 could not be found. Jul 16 21:16:41.061892 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:16:41.065265 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-b2f3d33b-90b6-49a3-8128-0521b9754692 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Returning 404 to user: Instance 3c5ab09e-0a97-4cfc-b249-a11ab221b5d2 could not be found. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:16:41.066453 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-b2f3d33b-90b6-49a3-8128-0521b9754692 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] 10.222.0.16 "GET /compute/v2.1/servers/3c5ab09e-0a97-4cfc-b249-a11ab221b5d2" status: 404 len: 111 microversion: 2.25 time: 0.048129 Jul 16 21:16:41.067286 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1029/1977] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:16:41 2022] GET /compute/v2.1/servers/3c5ab09e-0a97-4cfc-b249-a11ab221b5d2 => generated 111 bytes in 49 msecs (HTTP/1.1 404) 9 headers in 381 bytes (2 switches on core 0) Jul 16 21:16:41.076698 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-bfac34e3-fa69-4a22-b6f8-d83135399c49 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Calling method '.version_select at 0x7fdda2b1a820>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:41.087165 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_api:SELECT=95,nova_api:DELETE=1,nova_api:UPDATE=1,nova_api:INSERT=3 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:16:41.185763 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-bfac34e3-fa69-4a22-b6f8-d83135399c49 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] 10.222.0.16 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-108238508" status: 204 len: 0 microversion: 2.25 time: 0.113766 Jul 16 21:16:41.187188 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1030/1978] 10.222.0.16 () {60 vars in 1258 bytes} [Sat Jul 16 21:16:41 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-108238508 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Jul 16 21:16:41.201046 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-87f9a3b2-6eb0-4744-bbbc-d6117456f397 req-31010b64-0bc2-4256-b151-fa1fa0c69de1 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "e6844cd4-a382-45cc-ab21-bd7f2ede46e2", "tag": "b38de153-224c-42ed-a714-30fda6897dfb"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:16:41.217731 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-87f9a3b2-6eb0-4744-bbbc-d6117456f397 req-31010b64-0bc2-4256-b151-fa1fa0c69de1 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:41.218198 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-87f9a3b2-6eb0-4744-bbbc-d6117456f397 req-31010b64-0bc2-4256-b151-fa1fa0c69de1 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:41.230855 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-87f9a3b2-6eb0-4744-bbbc-d6117456f397 req-31010b64-0bc2-4256-b151-fa1fa0c69de1 service nova] Creating event network-changed:b38de153-224c-42ed-a714-30fda6897dfb for instance e6844cd4-a382-45cc-ab21-bd7f2ede46e2 on nch1-master-1 Jul 16 21:16:41.242514 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-87f9a3b2-6eb0-4744-bbbc-d6117456f397 req-31010b64-0bc2-4256-b151-fa1fa0c69de1 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.045345 Jul 16 21:16:41.243663 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1031/1979] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:16:41 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:16:41.316842 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-f2e373ac-f095-42b3-850b-50e6cc3eb407 tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b7a14721381b4f2cb4def5be76b7b645', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '57c1e99ad9884d79812e95fbc81512da', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:41.325412 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-f2e373ac-f095-42b3-850b-50e6cc3eb407 tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] 10.222.0.16 "GET /compute/v2.1/servers/e6844cd4-a382-45cc-ab21-bd7f2ede46e2" status: 200 len: 1315 microversion: 2.1 time: 0.433878 Jul 16 21:16:41.326007 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 949/1980] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:16:40 2022] GET /compute/v2.1/servers/e6844cd4-a382-45cc-ab21-bd7f2ede46e2 => generated 1315 bytes in 435 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:16:41.590438 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-e7e5735c-00c2-45cd-98df-362dfd0c1fa3 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:41.591299 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-e7e5735c-00c2-45cd-98df-362dfd0c1fa3 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] [instance: f03a2a83-e007-444a-b46b-70eb46bb242b] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:41.602087 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e7e5735c-00c2-45cd-98df-362dfd0c1fa3 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:41.607135 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e7e5735c-00c2-45cd-98df-362dfd0c1fa3 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:41.608047 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e7e5735c-00c2-45cd-98df-362dfd0c1fa3 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:41.608513 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e7e5735c-00c2-45cd-98df-362dfd0c1fa3 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:41.634906 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-629c312f-7cae-4a0b-a77e-ea65f614cc56 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:41.636737 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-629c312f-7cae-4a0b-a77e-ea65f614cc56 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] [instance: 11194822-5d42-45f4-99d0-88c70c808ddd] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:41.649398 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-629c312f-7cae-4a0b-a77e-ea65f614cc56 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:41.649857 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-629c312f-7cae-4a0b-a77e-ea65f614cc56 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:41.650647 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-629c312f-7cae-4a0b-a77e-ea65f614cc56 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:41.651119 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-629c312f-7cae-4a0b-a77e-ea65f614cc56 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:41.972842 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-629c312f-7cae-4a0b-a77e-ea65f614cc56 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd4144b5388b46a989a41a2f556bb508', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a90d4bcd25ff4bf68bdc194180fbbe09', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:41.986358 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-629c312f-7cae-4a0b-a77e-ea65f614cc56 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] 10.222.0.16 "GET /compute/v2.1/servers/11194822-5d42-45f4-99d0-88c70c808ddd" status: 200 len: 1513 microversion: 2.1 time: 0.355713 Jul 16 21:16:41.987464 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 950/1981] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:16:41 2022] GET /compute/v2.1/servers/11194822-5d42-45f4-99d0-88c70c808ddd => generated 1513 bytes in 358 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:16:42.004179 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-d3eb1f02-5a07-4f28-8c85-8d5003474fa8 req-012df50d-27dd-4419-a217-ce16a8d54899 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "f03a2a83-e007-444a-b46b-70eb46bb242b", "name": "network-vif-plugged", "status": "completed", "tag": "1df74ef4-7ed5-4498-8310-01458a7e541a"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:16:42.023931 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-d3eb1f02-5a07-4f28-8c85-8d5003474fa8 req-012df50d-27dd-4419-a217-ce16a8d54899 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:42.024664 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-d3eb1f02-5a07-4f28-8c85-8d5003474fa8 req-012df50d-27dd-4419-a217-ce16a8d54899 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:42.038058 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-e7e5735c-00c2-45cd-98df-362dfd0c1fa3 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'daf3c9e6634e4ae1bfb492eae1f44c93', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '24c64db8082648c8837c16352252f96d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:42.038608 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-d3eb1f02-5a07-4f28-8c85-8d5003474fa8 req-012df50d-27dd-4419-a217-ce16a8d54899 service nova] Creating event network-vif-plugged:1df74ef4-7ed5-4498-8310-01458a7e541a for instance f03a2a83-e007-444a-b46b-70eb46bb242b on nch1-master-1 Jul 16 21:16:42.046195 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-d3eb1f02-5a07-4f28-8c85-8d5003474fa8 req-012df50d-27dd-4419-a217-ce16a8d54899 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.056578 Jul 16 21:16:42.046821 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 951/1982] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:16:41 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:16:42.047483 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-e7e5735c-00c2-45cd-98df-362dfd0c1fa3 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] 10.222.0.16 "GET /compute/v2.1/servers/f03a2a83-e007-444a-b46b-70eb46bb242b" status: 200 len: 1307 microversion: 2.1 time: 0.461678 Jul 16 21:16:42.052060 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-234dc6e0-1f00-4f51-a1ae-6929e110f1c6 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:42.052768 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-234dc6e0-1f00-4f51-a1ae-6929e110f1c6 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] [instance: 9657dddc-2416-4c95-aab3-d6b6f9673369] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:42.057065 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1032/1983] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:16:41 2022] GET /compute/v2.1/servers/f03a2a83-e007-444a-b46b-70eb46bb242b => generated 1307 bytes in 473 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:16:42.062724 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-b20c7fbb-86d7-4e55-af43-064e6141d118 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:42.063911 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-b20c7fbb-86d7-4e55-af43-064e6141d118 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] [instance: c40ba20d-eb88-4ee8-be8f-ecbbd3315c48] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:42.070717 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b20c7fbb-86d7-4e55-af43-064e6141d118 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:42.071139 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-234dc6e0-1f00-4f51-a1ae-6929e110f1c6 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:42.071271 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b20c7fbb-86d7-4e55-af43-064e6141d118 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:42.072026 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b20c7fbb-86d7-4e55-af43-064e6141d118 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:42.072353 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-234dc6e0-1f00-4f51-a1ae-6929e110f1c6 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.011s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:42.072486 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b20c7fbb-86d7-4e55-af43-064e6141d118 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:42.073244 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-234dc6e0-1f00-4f51-a1ae-6929e110f1c6 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:42.073694 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-234dc6e0-1f00-4f51-a1ae-6929e110f1c6 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:42.226114 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-b20c7fbb-86d7-4e55-af43-064e6141d118 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6a69f9dee03d43beaafa966216b9e289', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9b769bf8f3ea4e368bf68de1df6538f9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:42.231800 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-234dc6e0-1f00-4f51-a1ae-6929e110f1c6 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '015ac1bbb2df487e99e5bddc60ee6923', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9f5f03e09936489fa1505188cefe3445', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:42.243669 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-b20c7fbb-86d7-4e55-af43-064e6141d118 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] 10.222.0.16 "GET /compute/v2.1/servers/c40ba20d-eb88-4ee8-be8f-ecbbd3315c48" status: 200 len: 1457 microversion: 2.1 time: 0.185283 Jul 16 21:16:42.244510 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1033/1984] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:16:42 2022] GET /compute/v2.1/servers/c40ba20d-eb88-4ee8-be8f-ecbbd3315c48 => generated 1457 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:16:42.254462 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-1e382b9f-d146-483a-8e71-1ed0e1c6eddc req-acc738be-d2b2-43c1-820a-6eea6094aa21 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "c40ba20d-eb88-4ee8-be8f-ecbbd3315c48", "name": "network-vif-deleted", "tag": "a2fac50b-598b-43e0-8e39-546a5c1b4490"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:16:42.260888 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-234dc6e0-1f00-4f51-a1ae-6929e110f1c6 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '015ac1bbb2df487e99e5bddc60ee6923', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9f5f03e09936489fa1505188cefe3445', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:42.275571 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-234dc6e0-1f00-4f51-a1ae-6929e110f1c6 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '015ac1bbb2df487e99e5bddc60ee6923', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9f5f03e09936489fa1505188cefe3445', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:42.275571 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-234dc6e0-1f00-4f51-a1ae-6929e110f1c6 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] 10.222.0.16 "GET /compute/v2.1/servers/9657dddc-2416-4c95-aab3-d6b6f9673369" status: 200 len: 1365 microversion: 2.47 time: 0.216287 Jul 16 21:16:42.275849 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-1e382b9f-d146-483a-8e71-1ed0e1c6eddc req-acc738be-d2b2-43c1-820a-6eea6094aa21 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:42.275849 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-1e382b9f-d146-483a-8e71-1ed0e1c6eddc req-acc738be-d2b2-43c1-820a-6eea6094aa21 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:42.276071 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 952/1985] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:16:42 2022] GET /compute/v2.1/servers/9657dddc-2416-4c95-aab3-d6b6f9673369 => generated 1365 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:16:42.289734 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-1e382b9f-d146-483a-8e71-1ed0e1c6eddc req-acc738be-d2b2-43c1-820a-6eea6094aa21 service nova] Creating event network-vif-deleted:a2fac50b-598b-43e0-8e39-546a5c1b4490 for instance c40ba20d-eb88-4ee8-be8f-ecbbd3315c48 on nch2-master-1 Jul 16 21:16:42.305111 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-1e382b9f-d146-483a-8e71-1ed0e1c6eddc req-acc738be-d2b2-43c1-820a-6eea6094aa21 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.058971 Jul 16 21:16:42.305734 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1034/1986] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:16:42 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 60 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:16:42.344585 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-9bcc6dff-54f4-4e01-98a1-54c29456fec1 tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:42.345449 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-9bcc6dff-54f4-4e01-98a1-54c29456fec1 tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] [instance: e6844cd4-a382-45cc-ab21-bd7f2ede46e2] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:42.352420 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9bcc6dff-54f4-4e01-98a1-54c29456fec1 tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:42.353135 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9bcc6dff-54f4-4e01-98a1-54c29456fec1 tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:42.353919 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9bcc6dff-54f4-4e01-98a1-54c29456fec1 tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:42.354352 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9bcc6dff-54f4-4e01-98a1-54c29456fec1 tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:42.506535 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-9bcc6dff-54f4-4e01-98a1-54c29456fec1 tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b7a14721381b4f2cb4def5be76b7b645', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '57c1e99ad9884d79812e95fbc81512da', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:42.513297 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-9bcc6dff-54f4-4e01-98a1-54c29456fec1 tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] 10.222.0.16 "GET /compute/v2.1/servers/e6844cd4-a382-45cc-ab21-bd7f2ede46e2" status: 200 len: 1315 microversion: 2.1 time: 0.173538 Jul 16 21:16:42.515077 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 953/1987] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:16:42 2022] GET /compute/v2.1/servers/e6844cd4-a382-45cc-ab21-bd7f2ede46e2 => generated 1315 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:16:42.692392 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-8c42374c-5f8f-49f9-b723-e5713a7914d9 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:42.693517 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-8c42374c-5f8f-49f9-b723-e5713a7914d9 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] [instance: 0fb5b5b6-1796-4406-b276-326e1d7dd451] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:42.705409 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8c42374c-5f8f-49f9-b723-e5713a7914d9 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:42.706142 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8c42374c-5f8f-49f9-b723-e5713a7914d9 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:42.707407 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8c42374c-5f8f-49f9-b723-e5713a7914d9 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:42.713608 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8c42374c-5f8f-49f9-b723-e5713a7914d9 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:42.737913 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-8c42374c-5f8f-49f9-b723-e5713a7914d9 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] [instance: 0fb5b5b6-1796-4406-b276-326e1d7dd451] Going to try to terminate instance {{(pid=138680) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 16 21:16:43.006038 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-852715fb-c30f-407d-b1d6-fbd0206919b0 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:43.006737 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-852715fb-c30f-407d-b1d6-fbd0206919b0 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] [instance: 11194822-5d42-45f4-99d0-88c70c808ddd] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:43.015835 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-852715fb-c30f-407d-b1d6-fbd0206919b0 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:43.016291 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-852715fb-c30f-407d-b1d6-fbd0206919b0 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:43.017217 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-852715fb-c30f-407d-b1d6-fbd0206919b0 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:43.017819 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-852715fb-c30f-407d-b1d6-fbd0206919b0 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:43.168001 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-852715fb-c30f-407d-b1d6-fbd0206919b0 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd4144b5388b46a989a41a2f556bb508', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a90d4bcd25ff4bf68bdc194180fbbe09', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:43.175778 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-852715fb-c30f-407d-b1d6-fbd0206919b0 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] 10.222.0.16 "GET /compute/v2.1/servers/11194822-5d42-45f4-99d0-88c70c808ddd" status: 200 len: 1513 microversion: 2.1 time: 0.175158 Jul 16 21:16:43.176354 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 954/1988] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:16:42 2022] GET /compute/v2.1/servers/11194822-5d42-45f4-99d0-88c70c808ddd => generated 1513 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:16:43.181020 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-ad0c3a01-cb83-40b2-aad8-ed749adf45cc tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:43.181628 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-ad0c3a01-cb83-40b2-aad8-ed749adf45cc tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] [instance: f03a2a83-e007-444a-b46b-70eb46bb242b] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:43.187611 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ad0c3a01-cb83-40b2-aad8-ed749adf45cc tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:43.188186 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ad0c3a01-cb83-40b2-aad8-ed749adf45cc tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:43.188986 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ad0c3a01-cb83-40b2-aad8-ed749adf45cc tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:43.189421 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ad0c3a01-cb83-40b2-aad8-ed749adf45cc tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:43.193098 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell0:SELECT=2 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:16:43.334529 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-ad0c3a01-cb83-40b2-aad8-ed749adf45cc tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'daf3c9e6634e4ae1bfb492eae1f44c93', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '24c64db8082648c8837c16352252f96d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:43.341920 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-ad0c3a01-cb83-40b2-aad8-ed749adf45cc tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] 10.222.0.16 "GET /compute/v2.1/servers/f03a2a83-e007-444a-b46b-70eb46bb242b" status: 200 len: 1307 microversion: 2.1 time: 0.164492 Jul 16 21:16:43.342547 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 955/1989] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:16:43 2022] GET /compute/v2.1/servers/f03a2a83-e007-444a-b46b-70eb46bb242b => generated 1307 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:16:43.347826 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-189e1e52-7651-4c3f-ac2a-039d09623647 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:43.348444 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-189e1e52-7651-4c3f-ac2a-039d09623647 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] [instance: c40ba20d-eb88-4ee8-be8f-ecbbd3315c48] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:43.354721 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-189e1e52-7651-4c3f-ac2a-039d09623647 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:43.355219 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-189e1e52-7651-4c3f-ac2a-039d09623647 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:43.356036 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-189e1e52-7651-4c3f-ac2a-039d09623647 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:43.356475 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-189e1e52-7651-4c3f-ac2a-039d09623647 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:43.370975 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-189e1e52-7651-4c3f-ac2a-039d09623647 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] HTTP exception thrown: Instance c40ba20d-eb88-4ee8-be8f-ecbbd3315c48 could not be found. Jul 16 21:16:43.376333 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-189e1e52-7651-4c3f-ac2a-039d09623647 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Request method failure captured: Jul 16 21:16:43.376333 ncdv-master-1 devstack@n-api.service[138681]: request: GET /compute/v2.1/servers/c40ba20d-eb88-4ee8-be8f-ecbbd3315c48 HTTP/1.1 Jul 16 21:16:43.376333 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:16:43.376333 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:16:43.376333 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:16:43.376333 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 21:16:43.376333 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:16:43.376333 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:16:43.376333 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:16:43.376333 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:16:43.376333 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0yjys5urr_ei5Seo_QKDPCqmR0EFBibpmCG26yjPCPljKUNzADf-WX077qzJM5iZPEhFd-9J9wlncEYo9S_mA0fOh-Gvf9LxFXEvyCYlVLvbQiN1HZ19aSAOzuz2ad1WkBxSsEap9oMo0DE5_h7t-ONjtOk159u5RHwmU9Cr_ibnvE8 Jul 16 21:16:43.376333 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:16:43.376333 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:16:43.376333 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:16:43.376333 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:16:43.376333 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:16:43.376333 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:16:43.376333 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: 9b769bf8f3ea4e368bf68de1df6538f9 Jul 16 21:16:43.376333 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-LiveMigrationTest-1001758322 Jul 16 21:16:43.376333 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:16:43.376333 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:16:43.376333 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/9b769bf8f3ea4e368bf68de1df6538f9"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/9b769bf8f3ea4e368bf68de1df6538f9"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_9b769bf8f3ea4e368bf68de1df6538f9"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/9b769bf8f3ea4e368bf68de1df6538f9"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:16:43.376333 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-LiveMigrationTest-1001758322 Jul 16 21:16:43.376333 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: 9b769bf8f3ea4e368bf68de1df6538f9 Jul 16 21:16:43.376333 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-LiveMigrationTest-1001758322 Jul 16 21:16:43.376333 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-LiveMigrationTest-1001758322-project Jul 16 21:16:43.376333 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:16:43.376333 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:16:43.376333 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: 6a69f9dee03d43beaafa966216b9e289 Jul 16 21:16:43.376333 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-LiveMigrationTest-1001758322-project Jul 16 21:16:43.376333 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:16:43.376333 ncdv-master-1 devstack@n-api.service[138681]: exception: Instance c40ba20d-eb88-4ee8-be8f-ecbbd3315c48 could not be found. Jul 16 21:16:43.376333 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': 'c40ba20d-eb88-4ee8-be8f-ecbbd3315c48'} Jul 16 21:16:43.376333 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:16:43.376333 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:16:43.376333 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:16:43.376333 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:16:43.376333 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:16:43.376333 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:16:43.376333 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:16:43.376333 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:16:43.376333 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:16:43.376333 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:16:43.376333 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance c40ba20d-eb88-4ee8-be8f-ecbbd3315c48 could not be found. Jul 16 21:16:43.376333 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:16:43.376333 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:16:43.376333 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:16:43.376333 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:16:43.376333 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:16:43.376333 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:16:43.376333 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:16:43.376333 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:16:43.376333 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:16:43.376333 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:16:43.376333 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:16:43.376333 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:16:43.376333 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:16:43.376333 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:16:43.376333 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:16:43.376333 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:16:43.376333 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance c40ba20d-eb88-4ee8-be8f-ecbbd3315c48 could not be found. Jul 16 21:16:43.376333 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:16:43.376333 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:16:43.376333 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:16:43.381930 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:16:43.381930 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:16:43.381930 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:16:43.381930 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:16:43.381930 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:16:43.381930 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance c40ba20d-eb88-4ee8-be8f-ecbbd3315c48 could not be found. Jul 16 21:16:43.381930 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:16:43.381930 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-189e1e52-7651-4c3f-ac2a-039d09623647 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] Returning 404 to user: Instance c40ba20d-eb88-4ee8-be8f-ecbbd3315c48 could not be found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:16:43.381930 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-189e1e52-7651-4c3f-ac2a-039d09623647 tempest-LiveMigrationTest-1001758322 tempest-LiveMigrationTest-1001758322-project] 10.222.0.16 "GET /compute/v2.1/servers/c40ba20d-eb88-4ee8-be8f-ecbbd3315c48" status: 404 len: 111 microversion: 2.1 time: 0.030525 Jul 16 21:16:43.381930 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 956/1990] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:16:43 2022] GET /compute/v2.1/servers/c40ba20d-eb88-4ee8-be8f-ecbbd3315c48 => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:16:43.384496 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-9c8341d8-56e1-4a10-a2e6-0c7f79fd4601 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:43.385202 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-9c8341d8-56e1-4a10-a2e6-0c7f79fd4601 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] [instance: 9657dddc-2416-4c95-aab3-d6b6f9673369] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:43.393150 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9c8341d8-56e1-4a10-a2e6-0c7f79fd4601 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:43.394093 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9c8341d8-56e1-4a10-a2e6-0c7f79fd4601 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:43.394613 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9c8341d8-56e1-4a10-a2e6-0c7f79fd4601 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:43.394833 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9c8341d8-56e1-4a10-a2e6-0c7f79fd4601 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:43.425279 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-8c42374c-5f8f-49f9-b723-e5713a7914d9 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] 10.222.0.16 "DELETE /compute/v2.1/servers/0fb5b5b6-1796-4406-b276-326e1d7dd451" status: 204 len: 0 microversion: 2.25 time: 0.737361 Jul 16 21:16:43.426625 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1035/1991] 10.222.0.16 () {60 vars in 1279 bytes} [Sat Jul 16 21:16:42 2022] DELETE /compute/v2.1/servers/0fb5b5b6-1796-4406-b276-326e1d7dd451 => generated 0 bytes in 740 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Jul 16 21:16:43.441949 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-a09766c0-0785-4ab7-a22c-30616ce0399b tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:43.443230 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-a09766c0-0785-4ab7-a22c-30616ce0399b tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] [instance: 0fb5b5b6-1796-4406-b276-326e1d7dd451] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:43.457548 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a09766c0-0785-4ab7-a22c-30616ce0399b tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:43.460106 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a09766c0-0785-4ab7-a22c-30616ce0399b tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:43.466152 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a09766c0-0785-4ab7-a22c-30616ce0399b tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:43.466915 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a09766c0-0785-4ab7-a22c-30616ce0399b tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:43.513453 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-9c8341d8-56e1-4a10-a2e6-0c7f79fd4601 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '015ac1bbb2df487e99e5bddc60ee6923', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9f5f03e09936489fa1505188cefe3445', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:43.531814 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-9c8341d8-56e1-4a10-a2e6-0c7f79fd4601 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '015ac1bbb2df487e99e5bddc60ee6923', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9f5f03e09936489fa1505188cefe3445', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:43.531814 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-9c8341d8-56e1-4a10-a2e6-0c7f79fd4601 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '015ac1bbb2df487e99e5bddc60ee6923', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9f5f03e09936489fa1505188cefe3445', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:43.531814 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-9c8341d8-56e1-4a10-a2e6-0c7f79fd4601 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] 10.222.0.16 "GET /compute/v2.1/servers/9657dddc-2416-4c95-aab3-d6b6f9673369" status: 200 len: 1365 microversion: 2.47 time: 0.152917 Jul 16 21:16:43.531814 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 957/1992] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:16:43 2022] GET /compute/v2.1/servers/9657dddc-2416-4c95-aab3-d6b6f9673369 => generated 1365 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:16:43.539905 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-8331916a-62eb-4a3a-aeaf-ba9af8736944 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Calling method '.version_select at 0x7fdda2ac09d0>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:43.572046 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell0:SELECT=12 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:16:43.613412 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-8331916a-62eb-4a3a-aeaf-ba9af8736944 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] 10.222.0.16 "GET /compute/v2.1/images/a7ab265a-4515-4850-a8d5-c86e67137b7e" status: 200 len: 662 microversion: 2.1 time: 0.082630 Jul 16 21:16:43.614021 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 958/1993] 10.222.0.16 () {58 vars in 1232 bytes} [Sat Jul 16 21:16:43 2022] GET /compute/v2.1/images/a7ab265a-4515-4850-a8d5-c86e67137b7e => generated 662 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:16:43.619253 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-f1c79377-a23a-472f-b9b5-b2e099039cdc tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:43.619879 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-f1c79377-a23a-472f-b9b5-b2e099039cdc tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] [instance: e6844cd4-a382-45cc-ab21-bd7f2ede46e2] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:43.627302 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f1c79377-a23a-472f-b9b5-b2e099039cdc tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:43.627748 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f1c79377-a23a-472f-b9b5-b2e099039cdc tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:43.628971 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f1c79377-a23a-472f-b9b5-b2e099039cdc tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:43.630277 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f1c79377-a23a-472f-b9b5-b2e099039cdc tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:43.658567 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-a09766c0-0785-4ab7-a22c-30616ce0399b tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:43.667747 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-a09766c0-0785-4ab7-a22c-30616ce0399b tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:43.669582 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-a09766c0-0785-4ab7-a22c-30616ce0399b tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:43.671307 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-a09766c0-0785-4ab7-a22c-30616ce0399b tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] 10.222.0.16 "GET /compute/v2.1/servers/0fb5b5b6-1796-4406-b276-326e1d7dd451" status: 200 len: 1548 microversion: 2.25 time: 0.238106 Jul 16 21:16:43.672416 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1036/1994] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:16:43 2022] GET /compute/v2.1/servers/0fb5b5b6-1796-4406-b276-326e1d7dd451 => generated 1548 bytes in 241 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 16 21:16:43.686828 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-b6f6b6f3-151f-4607-bb57-ae0ea93aa69d tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Calling method '.version_select at 0x7fdda2b1aaf0>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:43.737985 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-b6f6b6f3-151f-4607-bb57-ae0ea93aa69d tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] 10.222.0.16 "GET /compute/v2.1/images/a7ab265a-4515-4850-a8d5-c86e67137b7e" status: 200 len: 662 microversion: 2.1 time: 0.064310 Jul 16 21:16:43.739350 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1037/1995] 10.222.0.16 () {58 vars in 1232 bytes} [Sat Jul 16 21:16:43 2022] GET /compute/v2.1/images/a7ab265a-4515-4850-a8d5-c86e67137b7e => generated 662 bytes in 66 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:16:43.800191 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-f1c79377-a23a-472f-b9b5-b2e099039cdc tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b7a14721381b4f2cb4def5be76b7b645', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '57c1e99ad9884d79812e95fbc81512da', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:43.809185 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-f1c79377-a23a-472f-b9b5-b2e099039cdc tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] 10.222.0.16 "GET /compute/v2.1/servers/e6844cd4-a382-45cc-ab21-bd7f2ede46e2" status: 200 len: 1315 microversion: 2.1 time: 0.194033 Jul 16 21:16:43.809939 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 959/1996] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:16:43 2022] GET /compute/v2.1/servers/e6844cd4-a382-45cc-ab21-bd7f2ede46e2 => generated 1315 bytes in 195 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:16:44.195205 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-99aae83b-cc43-447e-ab5d-6bfce30f9dd7 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:44.195990 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-99aae83b-cc43-447e-ab5d-6bfce30f9dd7 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] [instance: 11194822-5d42-45f4-99d0-88c70c808ddd] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:44.204654 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-99aae83b-cc43-447e-ab5d-6bfce30f9dd7 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:44.205109 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-99aae83b-cc43-447e-ab5d-6bfce30f9dd7 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:44.205937 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-99aae83b-cc43-447e-ab5d-6bfce30f9dd7 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:44.206383 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-99aae83b-cc43-447e-ab5d-6bfce30f9dd7 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:44.348048 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-99aae83b-cc43-447e-ab5d-6bfce30f9dd7 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd4144b5388b46a989a41a2f556bb508', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a90d4bcd25ff4bf68bdc194180fbbe09', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:44.357812 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-99aae83b-cc43-447e-ab5d-6bfce30f9dd7 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] 10.222.0.16 "GET /compute/v2.1/servers/11194822-5d42-45f4-99d0-88c70c808ddd" status: 200 len: 1513 microversion: 2.1 time: 0.168370 Jul 16 21:16:44.358435 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1038/1997] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:16:44 2022] GET /compute/v2.1/servers/11194822-5d42-45f4-99d0-88c70c808ddd => generated 1513 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:16:44.361338 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-9ef8a4dd-4c62-47cd-a749-e1b472ae008a tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:44.361962 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-9ef8a4dd-4c62-47cd-a749-e1b472ae008a tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] [instance: f03a2a83-e007-444a-b46b-70eb46bb242b] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:44.373974 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9ef8a4dd-4c62-47cd-a749-e1b472ae008a tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:44.377593 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9ef8a4dd-4c62-47cd-a749-e1b472ae008a tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:44.377593 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9ef8a4dd-4c62-47cd-a749-e1b472ae008a tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:44.377593 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9ef8a4dd-4c62-47cd-a749-e1b472ae008a tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:44.523439 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-9ef8a4dd-4c62-47cd-a749-e1b472ae008a tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'daf3c9e6634e4ae1bfb492eae1f44c93', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '24c64db8082648c8837c16352252f96d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:44.531775 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-9ef8a4dd-4c62-47cd-a749-e1b472ae008a tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] 10.222.0.16 "GET /compute/v2.1/servers/f03a2a83-e007-444a-b46b-70eb46bb242b" status: 200 len: 1486 microversion: 2.1 time: 0.174983 Jul 16 21:16:44.532800 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 960/1998] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:16:44 2022] GET /compute/v2.1/servers/f03a2a83-e007-444a-b46b-70eb46bb242b => generated 1486 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:16:44.549935 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-8c1c95b7-db1f-4483-b8de-3f388536dd78 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Action: 'action', calling method: >, body: {"os-stop": {}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:16:44.554837 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-d4da6114-a717-4a72-8e6b-ed7ee3f404eb tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:44.555378 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-d4da6114-a717-4a72-8e6b-ed7ee3f404eb tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] [instance: 9657dddc-2416-4c95-aab3-d6b6f9673369] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:44.556231 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8c1c95b7-db1f-4483-b8de-3f388536dd78 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:44.556685 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8c1c95b7-db1f-4483-b8de-3f388536dd78 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:44.562835 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d4da6114-a717-4a72-8e6b-ed7ee3f404eb tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:44.563362 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d4da6114-a717-4a72-8e6b-ed7ee3f404eb tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:44.564167 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d4da6114-a717-4a72-8e6b-ed7ee3f404eb tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:44.564610 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d4da6114-a717-4a72-8e6b-ed7ee3f404eb tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:44.620023 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-8c1c95b7-db1f-4483-b8de-3f388536dd78 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] [instance: f03a2a83-e007-444a-b46b-70eb46bb242b] Going to try to stop instance {{(pid=138680) force_stop /opt/stack/nova/nova/compute/api.py:2707}} Jul 16 21:16:44.662792 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-d4da6114-a717-4a72-8e6b-ed7ee3f404eb tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '015ac1bbb2df487e99e5bddc60ee6923', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9f5f03e09936489fa1505188cefe3445', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:44.672163 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-d4da6114-a717-4a72-8e6b-ed7ee3f404eb tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '015ac1bbb2df487e99e5bddc60ee6923', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9f5f03e09936489fa1505188cefe3445', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:44.674052 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-d4da6114-a717-4a72-8e6b-ed7ee3f404eb tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '015ac1bbb2df487e99e5bddc60ee6923', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9f5f03e09936489fa1505188cefe3445', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:44.675443 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-d4da6114-a717-4a72-8e6b-ed7ee3f404eb tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] 10.222.0.16 "GET /compute/v2.1/servers/9657dddc-2416-4c95-aab3-d6b6f9673369" status: 200 len: 1365 microversion: 2.47 time: 0.127779 Jul 16 21:16:44.676009 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 961/1999] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:16:44 2022] GET /compute/v2.1/servers/9657dddc-2416-4c95-aab3-d6b6f9673369 => generated 1365 bytes in 130 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 16 21:16:44.686137 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell0:SELECT=24 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:16:44.691139 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-7b14f1e6-e348-45ec-a74c-504815c4af27 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:44.691882 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-7b14f1e6-e348-45ec-a74c-504815c4af27 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] [instance: 0fb5b5b6-1796-4406-b276-326e1d7dd451] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:44.698483 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7b14f1e6-e348-45ec-a74c-504815c4af27 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:44.699717 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7b14f1e6-e348-45ec-a74c-504815c4af27 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:44.700596 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7b14f1e6-e348-45ec-a74c-504815c4af27 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:44.701112 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7b14f1e6-e348-45ec-a74c-504815c4af27 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:44.804457 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-8c1c95b7-db1f-4483-b8de-3f388536dd78 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] 10.222.0.16 "POST /compute/v2.1/servers/f03a2a83-e007-444a-b46b-70eb46bb242b/action" status: 202 len: 0 microversion: 2.1 time: 0.258743 Jul 16 21:16:44.805138 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1039/2000] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 21:16:44 2022] POST /compute/v2.1/servers/f03a2a83-e007-444a-b46b-70eb46bb242b/action => generated 0 bytes in 260 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Jul 16 21:16:44.822157 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-df756f81-4a04-4fe8-a6e7-d61e8a91dc70 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:44.824389 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-df756f81-4a04-4fe8-a6e7-d61e8a91dc70 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] [instance: f03a2a83-e007-444a-b46b-70eb46bb242b] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:44.836056 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-df756f81-4a04-4fe8-a6e7-d61e8a91dc70 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:44.838089 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-df756f81-4a04-4fe8-a6e7-d61e8a91dc70 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:44.839119 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-df756f81-4a04-4fe8-a6e7-d61e8a91dc70 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:44.839732 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-df756f81-4a04-4fe8-a6e7-d61e8a91dc70 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:44.941161 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-7b14f1e6-e348-45ec-a74c-504815c4af27 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:44.952265 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-7b14f1e6-e348-45ec-a74c-504815c4af27 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:44.954612 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-7b14f1e6-e348-45ec-a74c-504815c4af27 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:44.955887 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-7b14f1e6-e348-45ec-a74c-504815c4af27 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] 10.222.0.16 "GET /compute/v2.1/servers/0fb5b5b6-1796-4406-b276-326e1d7dd451" status: 200 len: 1548 microversion: 2.25 time: 0.269933 Jul 16 21:16:44.956496 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 962/2001] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:16:44 2022] GET /compute/v2.1/servers/0fb5b5b6-1796-4406-b276-326e1d7dd451 => generated 1548 bytes in 271 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 16 21:16:44.961261 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-a58eaab4-8366-4168-900b-14374792eae7 tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:44.962064 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-a58eaab4-8366-4168-900b-14374792eae7 tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] [instance: e6844cd4-a382-45cc-ab21-bd7f2ede46e2] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:44.969045 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a58eaab4-8366-4168-900b-14374792eae7 tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:44.974466 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a58eaab4-8366-4168-900b-14374792eae7 tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:44.974466 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a58eaab4-8366-4168-900b-14374792eae7 tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:44.974466 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a58eaab4-8366-4168-900b-14374792eae7 tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:45.114189 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-df756f81-4a04-4fe8-a6e7-d61e8a91dc70 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'daf3c9e6634e4ae1bfb492eae1f44c93', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '24c64db8082648c8837c16352252f96d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:45.129430 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-df756f81-4a04-4fe8-a6e7-d61e8a91dc70 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] 10.222.0.16 "GET /compute/v2.1/servers/f03a2a83-e007-444a-b46b-70eb46bb242b" status: 200 len: 1496 microversion: 2.1 time: 0.319659 Jul 16 21:16:45.130737 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1040/2002] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:16:44 2022] GET /compute/v2.1/servers/f03a2a83-e007-444a-b46b-70eb46bb242b => generated 1496 bytes in 322 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:16:45.175297 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-a58eaab4-8366-4168-900b-14374792eae7 tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b7a14721381b4f2cb4def5be76b7b645', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '57c1e99ad9884d79812e95fbc81512da', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:45.183478 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-a58eaab4-8366-4168-900b-14374792eae7 tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] 10.222.0.16 "GET /compute/v2.1/servers/e6844cd4-a382-45cc-ab21-bd7f2ede46e2" status: 200 len: 1315 microversion: 2.1 time: 0.225691 Jul 16 21:16:45.184318 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 963/2003] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:16:44 2022] GET /compute/v2.1/servers/e6844cd4-a382-45cc-ab21-bd7f2ede46e2 => generated 1315 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:16:45.376736 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-8794bd3d-2f36-4971-86cf-3f057f3e7227 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:45.377406 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-8794bd3d-2f36-4971-86cf-3f057f3e7227 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] [instance: 11194822-5d42-45f4-99d0-88c70c808ddd] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:45.385850 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8794bd3d-2f36-4971-86cf-3f057f3e7227 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:45.386310 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8794bd3d-2f36-4971-86cf-3f057f3e7227 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:45.387124 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8794bd3d-2f36-4971-86cf-3f057f3e7227 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:45.387563 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8794bd3d-2f36-4971-86cf-3f057f3e7227 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:45.565835 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-8794bd3d-2f36-4971-86cf-3f057f3e7227 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd4144b5388b46a989a41a2f556bb508', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a90d4bcd25ff4bf68bdc194180fbbe09', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:45.574213 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-8794bd3d-2f36-4971-86cf-3f057f3e7227 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] 10.222.0.16 "GET /compute/v2.1/servers/11194822-5d42-45f4-99d0-88c70c808ddd" status: 200 len: 1513 microversion: 2.1 time: 0.201877 Jul 16 21:16:45.586133 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1041/2004] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:16:45 2022] GET /compute/v2.1/servers/11194822-5d42-45f4-99d0-88c70c808ddd => generated 1513 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:16:45.608156 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-c7bfb655-bef5-44b5-9622-de29de47e08d tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Calling method '.version_select at 0x7fdda2ae6a60>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:45.662100 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-c7bfb655-bef5-44b5-9622-de29de47e08d tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] 10.222.0.16 "GET /compute/v2.1/images/c201a74f-a305-4be4-9ee5-cd6c3f65f771" status: 200 len: 662 microversion: 2.1 time: 0.058489 Jul 16 21:16:45.662931 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 964/2005] 10.222.0.16 () {58 vars in 1232 bytes} [Sat Jul 16 21:16:45 2022] GET /compute/v2.1/images/c201a74f-a305-4be4-9ee5-cd6c3f65f771 => generated 662 bytes in 60 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:16:45.676246 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-14fcc1c4-c0e3-4ed8-929c-518d36800ece tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Calling method '.version_select at 0x7fdda2b39820>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:45.697651 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-d9e1b9dd-ddd2-4a69-9e5d-f4374e4d114b tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:45.704991 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-d9e1b9dd-ddd2-4a69-9e5d-f4374e4d114b tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] [instance: 9657dddc-2416-4c95-aab3-d6b6f9673369] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:45.706324 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d9e1b9dd-ddd2-4a69-9e5d-f4374e4d114b tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:45.707264 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d9e1b9dd-ddd2-4a69-9e5d-f4374e4d114b tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:45.708159 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d9e1b9dd-ddd2-4a69-9e5d-f4374e4d114b tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:45.708843 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d9e1b9dd-ddd2-4a69-9e5d-f4374e4d114b tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:45.725736 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-14fcc1c4-c0e3-4ed8-929c-518d36800ece tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] 10.222.0.16 "GET /compute/v2.1/images/c201a74f-a305-4be4-9ee5-cd6c3f65f771" status: 200 len: 662 microversion: 2.1 time: 0.054744 Jul 16 21:16:45.726320 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1042/2006] 10.222.0.16 () {58 vars in 1232 bytes} [Sat Jul 16 21:16:45 2022] GET /compute/v2.1/images/c201a74f-a305-4be4-9ee5-cd6c3f65f771 => generated 662 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:16:45.809201 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-d9e1b9dd-ddd2-4a69-9e5d-f4374e4d114b tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '015ac1bbb2df487e99e5bddc60ee6923', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9f5f03e09936489fa1505188cefe3445', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:45.818325 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-d9e1b9dd-ddd2-4a69-9e5d-f4374e4d114b tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '015ac1bbb2df487e99e5bddc60ee6923', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9f5f03e09936489fa1505188cefe3445', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:45.820383 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-d9e1b9dd-ddd2-4a69-9e5d-f4374e4d114b tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '015ac1bbb2df487e99e5bddc60ee6923', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9f5f03e09936489fa1505188cefe3445', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:45.821563 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-d9e1b9dd-ddd2-4a69-9e5d-f4374e4d114b tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] 10.222.0.16 "GET /compute/v2.1/servers/9657dddc-2416-4c95-aab3-d6b6f9673369" status: 200 len: 1365 microversion: 2.47 time: 0.128750 Jul 16 21:16:45.822125 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 965/2007] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:16:45 2022] GET /compute/v2.1/servers/9657dddc-2416-4c95-aab3-d6b6f9673369 => generated 1365 bytes in 130 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 16 21:16:45.974590 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-16650929-1cfc-4f33-83d3-8bab24612573 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:45.976031 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-16650929-1cfc-4f33-83d3-8bab24612573 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] [instance: 0fb5b5b6-1796-4406-b276-326e1d7dd451] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:45.986236 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-16650929-1cfc-4f33-83d3-8bab24612573 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:45.986981 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-16650929-1cfc-4f33-83d3-8bab24612573 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:45.988039 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-16650929-1cfc-4f33-83d3-8bab24612573 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:45.988657 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-16650929-1cfc-4f33-83d3-8bab24612573 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:46.080844 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-16650929-1cfc-4f33-83d3-8bab24612573 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:46.093997 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-16650929-1cfc-4f33-83d3-8bab24612573 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:46.096279 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-16650929-1cfc-4f33-83d3-8bab24612573 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:46.097788 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-16650929-1cfc-4f33-83d3-8bab24612573 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] 10.222.0.16 "GET /compute/v2.1/servers/0fb5b5b6-1796-4406-b276-326e1d7dd451" status: 200 len: 1506 microversion: 2.25 time: 0.127183 Jul 16 21:16:46.099182 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1043/2008] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:16:45 2022] GET /compute/v2.1/servers/0fb5b5b6-1796-4406-b276-326e1d7dd451 => generated 1506 bytes in 129 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:16:46.150675 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-37ba5188-8fdd-4432-8309-e8ceed694d02 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:46.151770 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-37ba5188-8fdd-4432-8309-e8ceed694d02 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] [instance: f03a2a83-e007-444a-b46b-70eb46bb242b] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:46.195207 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-37ba5188-8fdd-4432-8309-e8ceed694d02 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:46.199480 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-37ba5188-8fdd-4432-8309-e8ceed694d02 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:46.203982 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-0cb79023-a031-4bb1-ba6f-2b7ee9cfd94b tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:46.205904 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-0cb79023-a031-4bb1-ba6f-2b7ee9cfd94b tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] [instance: e6844cd4-a382-45cc-ab21-bd7f2ede46e2] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:46.212684 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-37ba5188-8fdd-4432-8309-e8ceed694d02 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:46.213535 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-37ba5188-8fdd-4432-8309-e8ceed694d02 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:46.231997 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0cb79023-a031-4bb1-ba6f-2b7ee9cfd94b tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:46.231997 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0cb79023-a031-4bb1-ba6f-2b7ee9cfd94b tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:46.233156 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0cb79023-a031-4bb1-ba6f-2b7ee9cfd94b tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:46.233761 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0cb79023-a031-4bb1-ba6f-2b7ee9cfd94b tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:46.576545 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-37ba5188-8fdd-4432-8309-e8ceed694d02 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'daf3c9e6634e4ae1bfb492eae1f44c93', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '24c64db8082648c8837c16352252f96d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:46.584241 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-37ba5188-8fdd-4432-8309-e8ceed694d02 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] 10.222.0.16 "GET /compute/v2.1/servers/f03a2a83-e007-444a-b46b-70eb46bb242b" status: 200 len: 1496 microversion: 2.1 time: 0.438268 Jul 16 21:16:46.585152 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 966/2009] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:16:46 2022] GET /compute/v2.1/servers/f03a2a83-e007-444a-b46b-70eb46bb242b => generated 1496 bytes in 442 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:16:46.595770 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-bfa8b0d7-10cd-4c4c-9305-2bd1687f3a6a req-b4e6b1a6-b393-485f-a0ad-856024f8b220 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "0fb5b5b6-1796-4406-b276-326e1d7dd451", "name": "network-vif-deleted", "tag": "19f08e09-ff8e-43eb-b5f2-421d0eadc41c"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:16:46.608268 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-0cb79023-a031-4bb1-ba6f-2b7ee9cfd94b tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b7a14721381b4f2cb4def5be76b7b645', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '57c1e99ad9884d79812e95fbc81512da', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:46.618406 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-0cb79023-a031-4bb1-ba6f-2b7ee9cfd94b tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] 10.222.0.16 "GET /compute/v2.1/servers/e6844cd4-a382-45cc-ab21-bd7f2ede46e2" status: 200 len: 1315 microversion: 2.1 time: 0.421169 Jul 16 21:16:46.629365 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1044/2010] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:16:46 2022] GET /compute/v2.1/servers/e6844cd4-a382-45cc-ab21-bd7f2ede46e2 => generated 1315 bytes in 432 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:16:46.642198 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-bfa8b0d7-10cd-4c4c-9305-2bd1687f3a6a req-b4e6b1a6-b393-485f-a0ad-856024f8b220 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:46.643343 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-bfa8b0d7-10cd-4c4c-9305-2bd1687f3a6a req-b4e6b1a6-b393-485f-a0ad-856024f8b220 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:46.644974 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-b3823052-e2bc-4558-9573-c496cf3dfaa2 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:46.645699 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-b3823052-e2bc-4558-9573-c496cf3dfaa2 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] [instance: 11194822-5d42-45f4-99d0-88c70c808ddd] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:46.659775 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b3823052-e2bc-4558-9573-c496cf3dfaa2 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:46.661628 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-bfa8b0d7-10cd-4c4c-9305-2bd1687f3a6a req-b4e6b1a6-b393-485f-a0ad-856024f8b220 service nova] Creating event network-vif-deleted:19f08e09-ff8e-43eb-b5f2-421d0eadc41c for instance 0fb5b5b6-1796-4406-b276-326e1d7dd451 on nch1-master-1 Jul 16 21:16:46.665603 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b3823052-e2bc-4558-9573-c496cf3dfaa2 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:46.666272 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-bfa8b0d7-10cd-4c4c-9305-2bd1687f3a6a req-b4e6b1a6-b393-485f-a0ad-856024f8b220 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.078269 Jul 16 21:16:46.667361 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 967/2011] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:16:46 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:16:46.681019 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b3823052-e2bc-4558-9573-c496cf3dfaa2 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:46.681019 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b3823052-e2bc-4558-9573-c496cf3dfaa2 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:46.841356 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-3f53d606-9be4-4bf7-89e4-2541bd425d37 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:46.842074 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-3f53d606-9be4-4bf7-89e4-2541bd425d37 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] [instance: 9657dddc-2416-4c95-aab3-d6b6f9673369] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:46.849234 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3f53d606-9be4-4bf7-89e4-2541bd425d37 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:46.849396 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-b3823052-e2bc-4558-9573-c496cf3dfaa2 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd4144b5388b46a989a41a2f556bb508', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a90d4bcd25ff4bf68bdc194180fbbe09', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:46.849693 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3f53d606-9be4-4bf7-89e4-2541bd425d37 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:46.850482 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3f53d606-9be4-4bf7-89e4-2541bd425d37 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:46.850928 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3f53d606-9be4-4bf7-89e4-2541bd425d37 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:46.853492 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=96,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:16:46.858373 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-b3823052-e2bc-4558-9573-c496cf3dfaa2 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] 10.222.0.16 "GET /compute/v2.1/servers/11194822-5d42-45f4-99d0-88c70c808ddd" status: 200 len: 1513 microversion: 2.1 time: 0.225969 Jul 16 21:16:46.859992 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1045/2012] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:16:46 2022] GET /compute/v2.1/servers/11194822-5d42-45f4-99d0-88c70c808ddd => generated 1513 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:16:46.873860 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=100 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:16:46.931820 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-3f53d606-9be4-4bf7-89e4-2541bd425d37 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '015ac1bbb2df487e99e5bddc60ee6923', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9f5f03e09936489fa1505188cefe3445', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:46.940392 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-3f53d606-9be4-4bf7-89e4-2541bd425d37 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '015ac1bbb2df487e99e5bddc60ee6923', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9f5f03e09936489fa1505188cefe3445', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:46.941913 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-3f53d606-9be4-4bf7-89e4-2541bd425d37 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '015ac1bbb2df487e99e5bddc60ee6923', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9f5f03e09936489fa1505188cefe3445', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:46.943141 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-3f53d606-9be4-4bf7-89e4-2541bd425d37 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] 10.222.0.16 "GET /compute/v2.1/servers/9657dddc-2416-4c95-aab3-d6b6f9673369" status: 200 len: 1365 microversion: 2.47 time: 0.106099 Jul 16 21:16:46.943712 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 968/2013] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:16:46 2022] GET /compute/v2.1/servers/9657dddc-2416-4c95-aab3-d6b6f9673369 => generated 1365 bytes in 108 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 16 21:16:47.117215 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-6f62ce2c-c9f0-4f6a-ad55-dbe80b008caf tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:47.118364 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-6f62ce2c-c9f0-4f6a-ad55-dbe80b008caf tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] [instance: 0fb5b5b6-1796-4406-b276-326e1d7dd451] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:47.128114 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6f62ce2c-c9f0-4f6a-ad55-dbe80b008caf tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:47.128666 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6f62ce2c-c9f0-4f6a-ad55-dbe80b008caf tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:47.129437 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6f62ce2c-c9f0-4f6a-ad55-dbe80b008caf tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:47.130065 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6f62ce2c-c9f0-4f6a-ad55-dbe80b008caf tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:47.170697 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6f62ce2c-c9f0-4f6a-ad55-dbe80b008caf tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:47.171294 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6f62ce2c-c9f0-4f6a-ad55-dbe80b008caf tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:47.172285 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.objects.instance [None req-6f62ce2c-c9f0-4f6a-ad55-dbe80b008caf tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lazy-loading 'fault' on Instance uuid 0fb5b5b6-1796-4406-b276-326e1d7dd451 {{(pid=138680) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:16:47.217634 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-6f62ce2c-c9f0-4f6a-ad55-dbe80b008caf tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:47.226752 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-6f62ce2c-c9f0-4f6a-ad55-dbe80b008caf tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:47.228319 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-6f62ce2c-c9f0-4f6a-ad55-dbe80b008caf tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:47.232021 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-6f62ce2c-c9f0-4f6a-ad55-dbe80b008caf tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] 10.222.0.16 "GET /compute/v2.1/servers/0fb5b5b6-1796-4406-b276-326e1d7dd451" status: 200 len: 1352 microversion: 2.25 time: 0.118784 Jul 16 21:16:47.239389 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1046/2014] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:16:47 2022] GET /compute/v2.1/servers/0fb5b5b6-1796-4406-b276-326e1d7dd451 => generated 1352 bytes in 127 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:16:47.602092 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-74f95d4d-1f06-47ab-b9fc-b389ea6bbc65 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:47.602957 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-74f95d4d-1f06-47ab-b9fc-b389ea6bbc65 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] [instance: f03a2a83-e007-444a-b46b-70eb46bb242b] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:47.610280 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-74f95d4d-1f06-47ab-b9fc-b389ea6bbc65 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:47.610845 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-74f95d4d-1f06-47ab-b9fc-b389ea6bbc65 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:47.611654 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-74f95d4d-1f06-47ab-b9fc-b389ea6bbc65 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:47.612086 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-74f95d4d-1f06-47ab-b9fc-b389ea6bbc65 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:47.636972 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-8f536dd8-6aff-49e2-9327-870f74d46d11 tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:47.637715 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-8f536dd8-6aff-49e2-9327-870f74d46d11 tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] [instance: e6844cd4-a382-45cc-ab21-bd7f2ede46e2] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:47.644985 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8f536dd8-6aff-49e2-9327-870f74d46d11 tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:47.647426 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8f536dd8-6aff-49e2-9327-870f74d46d11 tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:47.648296 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8f536dd8-6aff-49e2-9327-870f74d46d11 tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:47.648763 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8f536dd8-6aff-49e2-9327-870f74d46d11 tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:47.902244 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-74f95d4d-1f06-47ab-b9fc-b389ea6bbc65 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'daf3c9e6634e4ae1bfb492eae1f44c93', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '24c64db8082648c8837c16352252f96d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:48.184149 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-74f95d4d-1f06-47ab-b9fc-b389ea6bbc65 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] 10.222.0.16 "GET /compute/v2.1/servers/f03a2a83-e007-444a-b46b-70eb46bb242b" status: 200 len: 1496 microversion: 2.1 time: 0.315683 Jul 16 21:16:48.184149 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 969/2015] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:16:47 2022] GET /compute/v2.1/servers/f03a2a83-e007-444a-b46b-70eb46bb242b => generated 1496 bytes in 318 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:16:48.184149 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-78559d1d-856d-40d7-8e7a-b5aa234da380 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Calling method '.version_select at 0x7fdda2ac0940>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:48.189733 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-8f536dd8-6aff-49e2-9327-870f74d46d11 tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b7a14721381b4f2cb4def5be76b7b645', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '57c1e99ad9884d79812e95fbc81512da', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:48.189733 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-8f536dd8-6aff-49e2-9327-870f74d46d11 tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] 10.222.0.16 "GET /compute/v2.1/servers/e6844cd4-a382-45cc-ab21-bd7f2ede46e2" status: 200 len: 1315 microversion: 2.1 time: 0.356159 Jul 16 21:16:48.189733 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1047/2016] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:16:47 2022] GET /compute/v2.1/servers/e6844cd4-a382-45cc-ab21-bd7f2ede46e2 => generated 1315 bytes in 358 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:16:48.189733 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-7c73e728-f006-4fb7-852c-13c95b9c21a4 req-68c45af1-8690-4355-85dc-f6dcd3228b14 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "e6844cd4-a382-45cc-ab21-bd7f2ede46e2", "name": "network-vif-plugged", "status": "completed", "tag": "b38de153-224c-42ed-a714-30fda6897dfb"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:16:48.189733 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-7c73e728-f006-4fb7-852c-13c95b9c21a4 req-68c45af1-8690-4355-85dc-f6dcd3228b14 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:48.189733 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-7c73e728-f006-4fb7-852c-13c95b9c21a4 req-68c45af1-8690-4355-85dc-f6dcd3228b14 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:48.189733 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-7c73e728-f006-4fb7-852c-13c95b9c21a4 req-68c45af1-8690-4355-85dc-f6dcd3228b14 service nova] Creating event network-vif-plugged:b38de153-224c-42ed-a714-30fda6897dfb for instance e6844cd4-a382-45cc-ab21-bd7f2ede46e2 on nch1-master-1 Jul 16 21:16:48.192222 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-78559d1d-856d-40d7-8e7a-b5aa234da380 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] 10.222.0.16 "GET /compute/v2.1/images/90124cc7-665a-40b9-adc1-59b89a3d6840" status: 200 len: 662 microversion: 2.1 time: 0.120867 Jul 16 21:16:48.192222 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 970/2017] 10.222.0.16 () {58 vars in 1232 bytes} [Sat Jul 16 21:16:47 2022] GET /compute/v2.1/images/90124cc7-665a-40b9-adc1-59b89a3d6840 => generated 662 bytes in 129 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:16:48.192222 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-1ae37edd-1dd0-411a-9585-e4c24a6ed028 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:48.192222 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-1ae37edd-1dd0-411a-9585-e4c24a6ed028 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] [instance: 11194822-5d42-45f4-99d0-88c70c808ddd] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:48.192623 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-7c73e728-f006-4fb7-852c-13c95b9c21a4 req-68c45af1-8690-4355-85dc-f6dcd3228b14 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.058270 Jul 16 21:16:48.192623 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1048/2018] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:16:47 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 67 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:16:48.192623 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-da5ca457-1150-4f54-b0af-2441edbfde7c tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:48.192623 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-da5ca457-1150-4f54-b0af-2441edbfde7c tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] [instance: 9657dddc-2416-4c95-aab3-d6b6f9673369] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:48.198553 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1ae37edd-1dd0-411a-9585-e4c24a6ed028 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:48.198553 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1ae37edd-1dd0-411a-9585-e4c24a6ed028 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:48.198553 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1ae37edd-1dd0-411a-9585-e4c24a6ed028 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:48.198553 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1ae37edd-1dd0-411a-9585-e4c24a6ed028 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:48.199391 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-da5ca457-1150-4f54-b0af-2441edbfde7c tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:48.199391 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-da5ca457-1150-4f54-b0af-2441edbfde7c tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:48.199391 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-da5ca457-1150-4f54-b0af-2441edbfde7c tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:48.199391 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-da5ca457-1150-4f54-b0af-2441edbfde7c tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:48.224265 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-da5ca457-1150-4f54-b0af-2441edbfde7c tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '015ac1bbb2df487e99e5bddc60ee6923', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9f5f03e09936489fa1505188cefe3445', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:48.236903 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-da5ca457-1150-4f54-b0af-2441edbfde7c tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '015ac1bbb2df487e99e5bddc60ee6923', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9f5f03e09936489fa1505188cefe3445', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:48.238718 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-da5ca457-1150-4f54-b0af-2441edbfde7c tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '015ac1bbb2df487e99e5bddc60ee6923', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9f5f03e09936489fa1505188cefe3445', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:48.244174 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-da5ca457-1150-4f54-b0af-2441edbfde7c tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] 10.222.0.16 "GET /compute/v2.1/servers/9657dddc-2416-4c95-aab3-d6b6f9673369" status: 200 len: 1365 microversion: 2.47 time: 0.184798 Jul 16 21:16:48.245299 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1049/2019] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:16:48 2022] GET /compute/v2.1/servers/9657dddc-2416-4c95-aab3-d6b6f9673369 => generated 1365 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:16:48.258160 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-f3019a53-bad9-4a5f-82aa-929bdb779f9e tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Calling method '.version_select at 0x7fdda2a62d30>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:48.309226 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-1ae37edd-1dd0-411a-9585-e4c24a6ed028 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd4144b5388b46a989a41a2f556bb508', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a90d4bcd25ff4bf68bdc194180fbbe09', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:48.314256 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-f3019a53-bad9-4a5f-82aa-929bdb779f9e tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] 10.222.0.16 "GET /compute/v2.1/images/90124cc7-665a-40b9-adc1-59b89a3d6840" status: 200 len: 662 microversion: 2.1 time: 0.066113 Jul 16 21:16:48.314943 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1050/2020] 10.222.0.16 () {58 vars in 1232 bytes} [Sat Jul 16 21:16:48 2022] GET /compute/v2.1/images/90124cc7-665a-40b9-adc1-59b89a3d6840 => generated 662 bytes in 68 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:16:48.317526 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-1ae37edd-1dd0-411a-9585-e4c24a6ed028 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] 10.222.0.16 "GET /compute/v2.1/servers/11194822-5d42-45f4-99d0-88c70c808ddd" status: 200 len: 1513 microversion: 2.1 time: 0.271003 Jul 16 21:16:48.318215 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 971/2021] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:16:48 2022] GET /compute/v2.1/servers/11194822-5d42-45f4-99d0-88c70c808ddd => generated 1513 bytes in 272 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:16:48.322269 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-a29fba5e-1b3f-4f60-9acb-664629e60e23 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:48.323132 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-a29fba5e-1b3f-4f60-9acb-664629e60e23 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] [instance: 0fb5b5b6-1796-4406-b276-326e1d7dd451] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:48.331579 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a29fba5e-1b3f-4f60-9acb-664629e60e23 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:48.332118 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a29fba5e-1b3f-4f60-9acb-664629e60e23 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:48.332843 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a29fba5e-1b3f-4f60-9acb-664629e60e23 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:48.333434 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a29fba5e-1b3f-4f60-9acb-664629e60e23 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:48.350171 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-a29fba5e-1b3f-4f60-9acb-664629e60e23 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] HTTP exception thrown: Instance 0fb5b5b6-1796-4406-b276-326e1d7dd451 could not be found. Jul 16 21:16:48.351899 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-a29fba5e-1b3f-4f60-9acb-664629e60e23 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Request method failure captured: Jul 16 21:16:48.351899 ncdv-master-1 devstack@n-api.service[138680]: request: GET /compute/v2.1/servers/0fb5b5b6-1796-4406-b276-326e1d7dd451 HTTP/1.1 Jul 16 21:16:48.351899 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:16:48.351899 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:16:48.351899 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:16:48.351899 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 0 Jul 16 21:16:48.351899 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:16:48.351899 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:16:48.351899 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:16:48.351899 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:16:48.351899 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0ylBOHnbwqerAHhqONpFSBMhwk4nWfHPAHc-OTgMvb17y3jQh2-2K7JnAKTp7DvWJ_uO-bYr8xJfvvOUSkF97J9lR0DMJJ8o4htp9CLMoM61BjIIA0xJtMBeTA8oUgOHWMLAymuEaOlZbY4tC3l04tVeZqVEgsWsWeWmyuuhOz7GDcE Jul 16 21:16:48.351899 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:16:48.351899 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:16:48.351899 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:16:48.351899 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:16:48.351899 ncdv-master-1 devstack@n-api.service[138680]: X-Openstack-Nova-Api-Version: 2.25 Jul 16 21:16:48.351899 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:16:48.351899 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:16:48.351899 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: d9b92fc544c544f686ac2ab7ff5123b6 Jul 16 21:16:48.351899 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-LiveAutoBlockMigrationV225Test-1461646193 Jul 16 21:16:48.351899 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:16:48.351899 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:16:48.351899 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/d9b92fc544c544f686ac2ab7ff5123b6"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/d9b92fc544c544f686ac2ab7ff5123b6"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_d9b92fc544c544f686ac2ab7ff5123b6"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/d9b92fc544c544f686ac2ab7ff5123b6"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:16:48.351899 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-LiveAutoBlockMigrationV225Test-1461646193 Jul 16 21:16:48.351899 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: d9b92fc544c544f686ac2ab7ff5123b6 Jul 16 21:16:48.351899 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-LiveAutoBlockMigrationV225Test-1461646193 Jul 16 21:16:48.351899 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-LiveAutoBlockMigrationV225Test-1461646193-project Jul 16 21:16:48.351899 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:16:48.351899 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:16:48.351899 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: 578b87f863b34f57a571f7e56d44f118 Jul 16 21:16:48.351899 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-LiveAutoBlockMigrationV225Test-1461646193-project Jul 16 21:16:48.351899 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 21:16:48.351899 ncdv-master-1 devstack@n-api.service[138680]: exception: Instance 0fb5b5b6-1796-4406-b276-326e1d7dd451 could not be found. Jul 16 21:16:48.351899 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': '0fb5b5b6-1796-4406-b276-326e1d7dd451'} Jul 16 21:16:48.351899 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:16:48.351899 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:16:48.351899 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:16:48.351899 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:16:48.351899 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:16:48.351899 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:16:48.351899 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:16:48.351899 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:16:48.351899 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:16:48.351899 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:16:48.351899 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 0fb5b5b6-1796-4406-b276-326e1d7dd451 could not be found. Jul 16 21:16:48.351899 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:16:48.351899 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:16:48.351899 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:16:48.351899 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:16:48.351899 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:16:48.351899 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:16:48.351899 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:16:48.351899 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:16:48.351899 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:16:48.351899 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:16:48.351899 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:16:48.351899 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:16:48.351899 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:16:48.351899 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:16:48.351899 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:16:48.351899 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:16:48.351899 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 0fb5b5b6-1796-4406-b276-326e1d7dd451 could not be found. Jul 16 21:16:48.351899 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:16:48.355238 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:16:48.355238 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:16:48.355238 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:16:48.355238 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:16:48.355238 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:16:48.355238 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:16:48.355238 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:16:48.355238 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 0fb5b5b6-1796-4406-b276-326e1d7dd451 could not be found. Jul 16 21:16:48.355238 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:16:48.356540 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-a29fba5e-1b3f-4f60-9acb-664629e60e23 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Returning 404 to user: Instance 0fb5b5b6-1796-4406-b276-326e1d7dd451 could not be found. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:16:48.357534 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-a29fba5e-1b3f-4f60-9acb-664629e60e23 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] 10.222.0.16 "GET /compute/v2.1/servers/0fb5b5b6-1796-4406-b276-326e1d7dd451" status: 404 len: 111 microversion: 2.25 time: 0.041242 Jul 16 21:16:48.359403 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1051/2022] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:16:48 2022] GET /compute/v2.1/servers/0fb5b5b6-1796-4406-b276-326e1d7dd451 => generated 111 bytes in 43 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Jul 16 21:16:48.368637 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-2606d357-6998-4875-9fbc-8f98b5e52090 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:48.369399 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-2606d357-6998-4875-9fbc-8f98b5e52090 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] [instance: de210dbd-1398-413f-9a0f-63cae300bd03] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:48.373690 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-70c01863-c230-43ac-8143-54382f197de3 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ListImageFiltersTestJSON-server-535219192", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452"}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:16:48.378330 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2606d357-6998-4875-9fbc-8f98b5e52090 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:48.379036 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2606d357-6998-4875-9fbc-8f98b5e52090 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:48.379891 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2606d357-6998-4875-9fbc-8f98b5e52090 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:48.380324 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2606d357-6998-4875-9fbc-8f98b5e52090 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:48.420745 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-2606d357-6998-4875-9fbc-8f98b5e52090 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] [instance: de210dbd-1398-413f-9a0f-63cae300bd03] Going to try to terminate instance {{(pid=138681) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 16 21:16:48.456783 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.network.neutron [None req-70c01863-c230-43ac-8143-54382f197de3 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] validate_networks() for None {{(pid=138680) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 16 21:16:48.714537 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-70c01863-c230-43ac-8143-54382f197de3 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Flavor limits 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 16 21:16:48.714963 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-70c01863-c230-43ac-8143-54382f197de3 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Image limits 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 16 21:16:48.715480 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-70c01863-c230-43ac-8143-54382f197de3 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Flavor pref 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 16 21:16:48.715892 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-70c01863-c230-43ac-8143-54382f197de3 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Image pref 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 16 21:16:48.716306 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-70c01863-c230-43ac-8143-54382f197de3 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 16 21:16:48.726771 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-70c01863-c230-43ac-8143-54382f197de3 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:48.727255 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-70c01863-c230-43ac-8143-54382f197de3 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:48.728067 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-70c01863-c230-43ac-8143-54382f197de3 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:48.728510 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-70c01863-c230-43ac-8143-54382f197de3 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:48.746442 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-70c01863-c230-43ac-8143-54382f197de3 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Getting quotas for project 7e84496d78d4481e9d7648128868bca2. Resources: {'ram', 'instances', 'cores'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:16:48.751783 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-70c01863-c230-43ac-8143-54382f197de3 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Getting quotas for user 71e9304fc50447d6aac99703b5b6e4e1 and project 7e84496d78d4481e9d7648128868bca2. Resources: {'ram', 'instances', 'cores'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:16:48.764133 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-70c01863-c230-43ac-8143-54382f197de3 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Going to run 1 instances... {{(pid=138680) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 16 21:16:48.766903 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-70c01863-c230-43ac-8143-54382f197de3 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] [instance: b9a4910a-f117-4834-b699-498c6431b9bb] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138680) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 16 21:16:48.825280 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-2606d357-6998-4875-9fbc-8f98b5e52090 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] 10.222.0.16 "DELETE /compute/v2.1/servers/de210dbd-1398-413f-9a0f-63cae300bd03" status: 204 len: 0 microversion: 2.25 time: 0.461172 Jul 16 21:16:48.828942 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 972/2023] 10.222.0.16 () {60 vars in 1279 bytes} [Sat Jul 16 21:16:48 2022] DELETE /compute/v2.1/servers/de210dbd-1398-413f-9a0f-63cae300bd03 => generated 0 bytes in 466 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Jul 16 21:16:48.838359 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-48101a11-91bf-4e0b-b030-d4a904b47b2c tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:48.839103 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-48101a11-91bf-4e0b-b030-d4a904b47b2c tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] [instance: de210dbd-1398-413f-9a0f-63cae300bd03] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:48.846605 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-48101a11-91bf-4e0b-b030-d4a904b47b2c tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:48.847086 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-48101a11-91bf-4e0b-b030-d4a904b47b2c tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:48.847870 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-48101a11-91bf-4e0b-b030-d4a904b47b2c tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:48.848422 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-48101a11-91bf-4e0b-b030-d4a904b47b2c tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:48.900868 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-70c01863-c230-43ac-8143-54382f197de3 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.531308 Jul 16 21:16:48.901735 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1052/2024] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 21:16:48 2022] POST /compute/v2.1/servers => generated 380 bytes in 533 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Jul 16 21:16:48.930718 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-1d752e26-316b-4c95-bbc2-2e48cda35303 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:48.931375 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-1d752e26-316b-4c95-bbc2-2e48cda35303 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] [instance: f03a2a83-e007-444a-b46b-70eb46bb242b] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:48.944961 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1d752e26-316b-4c95-bbc2-2e48cda35303 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:48.944961 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1d752e26-316b-4c95-bbc2-2e48cda35303 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:48.944961 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1d752e26-316b-4c95-bbc2-2e48cda35303 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:48.944961 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1d752e26-316b-4c95-bbc2-2e48cda35303 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:49.164666 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-48101a11-91bf-4e0b-b030-d4a904b47b2c tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:49.174462 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-48101a11-91bf-4e0b-b030-d4a904b47b2c tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:49.176207 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-48101a11-91bf-4e0b-b030-d4a904b47b2c tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:49.177401 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-48101a11-91bf-4e0b-b030-d4a904b47b2c tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] 10.222.0.16 "GET /compute/v2.1/servers/de210dbd-1398-413f-9a0f-63cae300bd03" status: 200 len: 1563 microversion: 2.25 time: 0.343420 Jul 16 21:16:49.188786 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 973/2025] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:16:48 2022] GET /compute/v2.1/servers/de210dbd-1398-413f-9a0f-63cae300bd03 => generated 1563 bytes in 345 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:16:49.188786 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-c1e7e7ef-55e0-4e7a-a7c5-5e6ddf0b163a tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ListImageFiltersTestJSON-server-177207096", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452"}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:16:49.234873 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-1d752e26-316b-4c95-bbc2-2e48cda35303 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'daf3c9e6634e4ae1bfb492eae1f44c93', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '24c64db8082648c8837c16352252f96d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:49.241969 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.network.neutron [None req-c1e7e7ef-55e0-4e7a-a7c5-5e6ddf0b163a tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] validate_networks() for None {{(pid=138681) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 16 21:16:49.243952 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-1d752e26-316b-4c95-bbc2-2e48cda35303 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] 10.222.0.16 "GET /compute/v2.1/servers/f03a2a83-e007-444a-b46b-70eb46bb242b" status: 200 len: 1496 microversion: 2.1 time: 0.317003 Jul 16 21:16:49.244548 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1053/2026] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:16:48 2022] GET /compute/v2.1/servers/f03a2a83-e007-444a-b46b-70eb46bb242b => generated 1496 bytes in 319 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:16:49.259830 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-33ab49bf-a364-423e-aa2e-2164553a0e04 tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:49.260459 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-33ab49bf-a364-423e-aa2e-2164553a0e04 tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] [instance: e6844cd4-a382-45cc-ab21-bd7f2ede46e2] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:49.268131 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-33ab49bf-a364-423e-aa2e-2164553a0e04 tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:49.269399 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-33ab49bf-a364-423e-aa2e-2164553a0e04 tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:49.269399 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-33ab49bf-a364-423e-aa2e-2164553a0e04 tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:49.269796 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-33ab49bf-a364-423e-aa2e-2164553a0e04 tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:49.922486 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-33ab49bf-a364-423e-aa2e-2164553a0e04 tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b7a14721381b4f2cb4def5be76b7b645', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '57c1e99ad9884d79812e95fbc81512da', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:49.931461 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-33ab49bf-a364-423e-aa2e-2164553a0e04 tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] 10.222.0.16 "GET /compute/v2.1/servers/e6844cd4-a382-45cc-ab21-bd7f2ede46e2" status: 200 len: 1315 microversion: 2.1 time: 0.675673 Jul 16 21:16:49.943037 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1054/2027] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:16:49 2022] GET /compute/v2.1/servers/e6844cd4-a382-45cc-ab21-bd7f2ede46e2 => generated 1315 bytes in 688 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:16:49.952102 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-5a221b8c-9971-4215-b6af-c2f38ee3469b tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:49.955194 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-5a221b8c-9971-4215-b6af-c2f38ee3469b tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] [instance: 9657dddc-2416-4c95-aab3-d6b6f9673369] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:49.961851 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5a221b8c-9971-4215-b6af-c2f38ee3469b tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:49.962346 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5a221b8c-9971-4215-b6af-c2f38ee3469b tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:49.963156 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5a221b8c-9971-4215-b6af-c2f38ee3469b tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:49.963664 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5a221b8c-9971-4215-b6af-c2f38ee3469b tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:50.050548 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-c1e7e7ef-55e0-4e7a-a7c5-5e6ddf0b163a tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Flavor limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 16 21:16:50.051107 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-c1e7e7ef-55e0-4e7a-a7c5-5e6ddf0b163a tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Image limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 16 21:16:50.051592 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-c1e7e7ef-55e0-4e7a-a7c5-5e6ddf0b163a tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Flavor pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 16 21:16:50.052034 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-c1e7e7ef-55e0-4e7a-a7c5-5e6ddf0b163a tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Image pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 16 21:16:50.052451 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-c1e7e7ef-55e0-4e7a-a7c5-5e6ddf0b163a tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 16 21:16:50.064641 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c1e7e7ef-55e0-4e7a-a7c5-5e6ddf0b163a tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:50.065154 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c1e7e7ef-55e0-4e7a-a7c5-5e6ddf0b163a tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:50.065915 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c1e7e7ef-55e0-4e7a-a7c5-5e6ddf0b163a tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:50.066521 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c1e7e7ef-55e0-4e7a-a7c5-5e6ddf0b163a tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:50.086637 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-c1e7e7ef-55e0-4e7a-a7c5-5e6ddf0b163a tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Getting quotas for project 7e84496d78d4481e9d7648128868bca2. Resources: {'ram', 'instances', 'cores'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:16:50.091740 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-c1e7e7ef-55e0-4e7a-a7c5-5e6ddf0b163a tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Getting quotas for user 71e9304fc50447d6aac99703b5b6e4e1 and project 7e84496d78d4481e9d7648128868bca2. Resources: {'ram', 'instances', 'cores'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:16:50.096716 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-5a221b8c-9971-4215-b6af-c2f38ee3469b tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '015ac1bbb2df487e99e5bddc60ee6923', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9f5f03e09936489fa1505188cefe3445', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:50.113768 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-5a221b8c-9971-4215-b6af-c2f38ee3469b tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '015ac1bbb2df487e99e5bddc60ee6923', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9f5f03e09936489fa1505188cefe3445', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:50.116015 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-5a221b8c-9971-4215-b6af-c2f38ee3469b tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '015ac1bbb2df487e99e5bddc60ee6923', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9f5f03e09936489fa1505188cefe3445', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:50.119447 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-5a221b8c-9971-4215-b6af-c2f38ee3469b tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] 10.222.0.16 "GET /compute/v2.1/servers/9657dddc-2416-4c95-aab3-d6b6f9673369" status: 200 len: 1365 microversion: 2.47 time: 0.174476 Jul 16 21:16:50.120564 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1055/2028] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:16:49 2022] GET /compute/v2.1/servers/9657dddc-2416-4c95-aab3-d6b6f9673369 => generated 1365 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:16:50.129204 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-b7255957-9346-4cab-b65e-69bee7ea16b7 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:50.130218 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-b7255957-9346-4cab-b65e-69bee7ea16b7 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] [instance: 11194822-5d42-45f4-99d0-88c70c808ddd] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:50.136621 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-c1e7e7ef-55e0-4e7a-a7c5-5e6ddf0b163a tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Going to run 1 instances... {{(pid=138681) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 16 21:16:50.138973 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-c1e7e7ef-55e0-4e7a-a7c5-5e6ddf0b163a tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] [instance: 6cb4e66a-eba2-42b7-91e8-09ee7b2a5a4f] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138681) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 16 21:16:50.139426 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b7255957-9346-4cab-b65e-69bee7ea16b7 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:50.142315 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b7255957-9346-4cab-b65e-69bee7ea16b7 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:50.143439 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b7255957-9346-4cab-b65e-69bee7ea16b7 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:50.145084 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b7255957-9346-4cab-b65e-69bee7ea16b7 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:50.195407 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_api:SELECT=95,nova_api:UPDATE=3,nova_api:INSERT=2 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:16:50.400674 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-b7255957-9346-4cab-b65e-69bee7ea16b7 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd4144b5388b46a989a41a2f556bb508', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a90d4bcd25ff4bf68bdc194180fbbe09', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:50.409725 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-c1e7e7ef-55e0-4e7a-a7c5-5e6ddf0b163a tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 1.230470 Jul 16 21:16:50.415097 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 974/2029] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 21:16:49 2022] POST /compute/v2.1/servers => generated 380 bytes in 1233 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 16 21:16:50.417557 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-b7255957-9346-4cab-b65e-69bee7ea16b7 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] 10.222.0.16 "GET /compute/v2.1/servers/11194822-5d42-45f4-99d0-88c70c808ddd" status: 200 len: 1513 microversion: 2.1 time: 0.294907 Jul 16 21:16:50.418235 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-b36d8243-5871-4034-9d95-3896798f2ba9 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:50.418834 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1056/2030] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:16:50 2022] GET /compute/v2.1/servers/11194822-5d42-45f4-99d0-88c70c808ddd => generated 1513 bytes in 297 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:16:50.418960 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-b36d8243-5871-4034-9d95-3896798f2ba9 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] [instance: de210dbd-1398-413f-9a0f-63cae300bd03] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:50.425058 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-977a7e87-b707-4711-bdc6-743111eeebd3 req-195f18c2-914d-4866-831f-c5c0b4a57202 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "de210dbd-1398-413f-9a0f-63cae300bd03", "name": "network-vif-deleted", "tag": "176a187b-2abc-455b-bbcd-a736dec0c33e"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:16:50.434398 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-977a7e87-b707-4711-bdc6-743111eeebd3 req-195f18c2-914d-4866-831f-c5c0b4a57202 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:50.434915 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-977a7e87-b707-4711-bdc6-743111eeebd3 req-195f18c2-914d-4866-831f-c5c0b4a57202 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:50.435816 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b36d8243-5871-4034-9d95-3896798f2ba9 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:50.436281 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b36d8243-5871-4034-9d95-3896798f2ba9 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:50.439581 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b36d8243-5871-4034-9d95-3896798f2ba9 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:50.441219 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b36d8243-5871-4034-9d95-3896798f2ba9 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:50.451149 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-977a7e87-b707-4711-bdc6-743111eeebd3 req-195f18c2-914d-4866-831f-c5c0b4a57202 service nova] Creating event network-vif-deleted:176a187b-2abc-455b-bbcd-a736dec0c33e for instance de210dbd-1398-413f-9a0f-63cae300bd03 on nch1-master-1 Jul 16 21:16:50.457596 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-977a7e87-b707-4711-bdc6-743111eeebd3 req-195f18c2-914d-4866-831f-c5c0b4a57202 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.037064 Jul 16 21:16:50.458764 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1057/2031] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:16:50 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:16:50.465145 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-d138d262-7999-4bbc-b0fb-c9001dd01dda tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:50.465991 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-d138d262-7999-4bbc-b0fb-c9001dd01dda tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] [instance: f03a2a83-e007-444a-b46b-70eb46bb242b] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:50.473713 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d138d262-7999-4bbc-b0fb-c9001dd01dda tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:50.474248 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d138d262-7999-4bbc-b0fb-c9001dd01dda tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:50.475077 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d138d262-7999-4bbc-b0fb-c9001dd01dda tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:50.475890 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d138d262-7999-4bbc-b0fb-c9001dd01dda tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:50.659880 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-b36d8243-5871-4034-9d95-3896798f2ba9 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:50.684059 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-b36d8243-5871-4034-9d95-3896798f2ba9 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:50.689220 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-b36d8243-5871-4034-9d95-3896798f2ba9 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:50.690728 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-b36d8243-5871-4034-9d95-3896798f2ba9 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] 10.222.0.16 "GET /compute/v2.1/servers/de210dbd-1398-413f-9a0f-63cae300bd03" status: 200 len: 1521 microversion: 2.25 time: 0.278309 Jul 16 21:16:50.703528 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 975/2032] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:16:50 2022] GET /compute/v2.1/servers/de210dbd-1398-413f-9a0f-63cae300bd03 => generated 1521 bytes in 291 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:16:50.708905 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-f40e8cb0-2f73-470e-9838-c2960877197d tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:50.709811 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-f40e8cb0-2f73-470e-9838-c2960877197d tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] [instance: 6cb4e66a-eba2-42b7-91e8-09ee7b2a5a4f] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:50.771295 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-d138d262-7999-4bbc-b0fb-c9001dd01dda tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'daf3c9e6634e4ae1bfb492eae1f44c93', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '24c64db8082648c8837c16352252f96d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:50.782692 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-d138d262-7999-4bbc-b0fb-c9001dd01dda tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] 10.222.0.16 "GET /compute/v2.1/servers/f03a2a83-e007-444a-b46b-70eb46bb242b" status: 200 len: 1496 microversion: 2.1 time: 0.320046 Jul 16 21:16:50.782692 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1058/2033] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:16:50 2022] GET /compute/v2.1/servers/f03a2a83-e007-444a-b46b-70eb46bb242b => generated 1496 bytes in 323 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:16:50.790976 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-f40e8cb0-2f73-470e-9838-c2960877197d tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '71e9304fc50447d6aac99703b5b6e4e1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e84496d78d4481e9d7648128868bca2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:50.798095 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-f40e8cb0-2f73-470e-9838-c2960877197d tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] 10.222.0.16 "GET /compute/v2.1/servers/6cb4e66a-eba2-42b7-91e8-09ee7b2a5a4f" status: 200 len: 1217 microversion: 2.1 time: 0.092996 Jul 16 21:16:50.798680 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 976/2034] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:16:50 2022] GET /compute/v2.1/servers/6cb4e66a-eba2-42b7-91e8-09ee7b2a5a4f => generated 1217 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:16:50.964067 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-3ca57c76-2304-42c9-90f2-07ebb1cebd92 tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:50.964723 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-3ca57c76-2304-42c9-90f2-07ebb1cebd92 tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] [instance: e6844cd4-a382-45cc-ab21-bd7f2ede46e2] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:50.972004 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3ca57c76-2304-42c9-90f2-07ebb1cebd92 tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:50.972541 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3ca57c76-2304-42c9-90f2-07ebb1cebd92 tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:50.973417 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3ca57c76-2304-42c9-90f2-07ebb1cebd92 tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:50.973923 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3ca57c76-2304-42c9-90f2-07ebb1cebd92 tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:51.139286 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-b77acb93-41a0-49b9-9429-3a60dba264a3 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:51.141336 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-b77acb93-41a0-49b9-9429-3a60dba264a3 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] [instance: 9657dddc-2416-4c95-aab3-d6b6f9673369] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:51.149026 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b77acb93-41a0-49b9-9429-3a60dba264a3 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:51.149714 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b77acb93-41a0-49b9-9429-3a60dba264a3 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:51.150683 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b77acb93-41a0-49b9-9429-3a60dba264a3 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:51.151835 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b77acb93-41a0-49b9-9429-3a60dba264a3 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:51.244241 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-3ca57c76-2304-42c9-90f2-07ebb1cebd92 tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b7a14721381b4f2cb4def5be76b7b645', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '57c1e99ad9884d79812e95fbc81512da', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:51.253474 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-b77acb93-41a0-49b9-9429-3a60dba264a3 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '015ac1bbb2df487e99e5bddc60ee6923', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9f5f03e09936489fa1505188cefe3445', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:51.267881 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-3ca57c76-2304-42c9-90f2-07ebb1cebd92 tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] 10.222.0.16 "GET /compute/v2.1/servers/e6844cd4-a382-45cc-ab21-bd7f2ede46e2" status: 200 len: 1503 microversion: 2.1 time: 0.297035 Jul 16 21:16:51.267881 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1059/2035] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:16:50 2022] GET /compute/v2.1/servers/e6844cd4-a382-45cc-ab21-bd7f2ede46e2 => generated 1503 bytes in 299 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:16:51.268090 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-b77acb93-41a0-49b9-9429-3a60dba264a3 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '015ac1bbb2df487e99e5bddc60ee6923', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9f5f03e09936489fa1505188cefe3445', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:51.268090 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-b77acb93-41a0-49b9-9429-3a60dba264a3 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '015ac1bbb2df487e99e5bddc60ee6923', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9f5f03e09936489fa1505188cefe3445', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:51.268090 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-b77acb93-41a0-49b9-9429-3a60dba264a3 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] 10.222.0.16 "GET /compute/v2.1/servers/9657dddc-2416-4c95-aab3-d6b6f9673369" status: 200 len: 1365 microversion: 2.47 time: 0.130602 Jul 16 21:16:51.268090 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 977/2036] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:16:51 2022] GET /compute/v2.1/servers/9657dddc-2416-4c95-aab3-d6b6f9673369 => generated 1365 bytes in 132 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:16:51.273759 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-2e11faab-08b5-43a9-8ca5-ac71b05010ec tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:51.274269 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-2e11faab-08b5-43a9-8ca5-ac71b05010ec tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] [instance: e6844cd4-a382-45cc-ab21-bd7f2ede46e2] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:51.282129 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2e11faab-08b5-43a9-8ca5-ac71b05010ec tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:51.282689 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2e11faab-08b5-43a9-8ca5-ac71b05010ec tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:51.283431 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2e11faab-08b5-43a9-8ca5-ac71b05010ec tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:51.283874 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2e11faab-08b5-43a9-8ca5-ac71b05010ec tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:51.313520 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-2e11faab-08b5-43a9-8ca5-ac71b05010ec tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] [instance: e6844cd4-a382-45cc-ab21-bd7f2ede46e2] Going to try to terminate instance {{(pid=138680) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 16 21:16:51.436336 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-93aa2057-08e3-4670-bf06-eeaa2cd43465 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:51.437071 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-93aa2057-08e3-4670-bf06-eeaa2cd43465 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] [instance: 11194822-5d42-45f4-99d0-88c70c808ddd] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:51.443726 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-93aa2057-08e3-4670-bf06-eeaa2cd43465 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:51.448598 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-93aa2057-08e3-4670-bf06-eeaa2cd43465 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:51.448598 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-93aa2057-08e3-4670-bf06-eeaa2cd43465 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:51.448598 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-93aa2057-08e3-4670-bf06-eeaa2cd43465 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:51.666355 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-93aa2057-08e3-4670-bf06-eeaa2cd43465 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd4144b5388b46a989a41a2f556bb508', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a90d4bcd25ff4bf68bdc194180fbbe09', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:51.675025 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-93aa2057-08e3-4670-bf06-eeaa2cd43465 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] 10.222.0.16 "GET /compute/v2.1/servers/11194822-5d42-45f4-99d0-88c70c808ddd" status: 200 len: 1513 microversion: 2.1 time: 0.242983 Jul 16 21:16:51.675859 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 978/2037] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:16:51 2022] GET /compute/v2.1/servers/11194822-5d42-45f4-99d0-88c70c808ddd => generated 1513 bytes in 245 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:16:51.709945 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-306daa58-23b5-4d41-87f0-e2c1c5a282d7 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:51.710806 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-306daa58-23b5-4d41-87f0-e2c1c5a282d7 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] [instance: de210dbd-1398-413f-9a0f-63cae300bd03] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:51.718609 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-306daa58-23b5-4d41-87f0-e2c1c5a282d7 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:51.719915 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-306daa58-23b5-4d41-87f0-e2c1c5a282d7 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:51.721133 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-306daa58-23b5-4d41-87f0-e2c1c5a282d7 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:51.721774 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-306daa58-23b5-4d41-87f0-e2c1c5a282d7 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:51.722937 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-2e11faab-08b5-43a9-8ca5-ac71b05010ec tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] 10.222.0.16 "DELETE /compute/v2.1/servers/e6844cd4-a382-45cc-ab21-bd7f2ede46e2" status: 204 len: 0 microversion: 2.1 time: 0.452701 Jul 16 21:16:51.728623 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1060/2038] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:16:51 2022] DELETE /compute/v2.1/servers/e6844cd4-a382-45cc-ab21-bd7f2ede46e2 => generated 0 bytes in 454 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 16 21:16:51.739257 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-33d75880-c620-4b37-a307-59f8a53ecaa9 tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:51.740640 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-33d75880-c620-4b37-a307-59f8a53ecaa9 tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] [instance: e6844cd4-a382-45cc-ab21-bd7f2ede46e2] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:51.749922 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-33d75880-c620-4b37-a307-59f8a53ecaa9 tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:51.750500 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-33d75880-c620-4b37-a307-59f8a53ecaa9 tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:51.751978 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-33d75880-c620-4b37-a307-59f8a53ecaa9 tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:51.752441 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-33d75880-c620-4b37-a307-59f8a53ecaa9 tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:51.760661 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-306daa58-23b5-4d41-87f0-e2c1c5a282d7 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:51.761356 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-306daa58-23b5-4d41-87f0-e2c1c5a282d7 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:51.762036 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.objects.instance [None req-306daa58-23b5-4d41-87f0-e2c1c5a282d7 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lazy-loading 'fault' on Instance uuid de210dbd-1398-413f-9a0f-63cae300bd03 {{(pid=138681) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:16:51.826443 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-306daa58-23b5-4d41-87f0-e2c1c5a282d7 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:51.839867 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-306daa58-23b5-4d41-87f0-e2c1c5a282d7 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:51.841789 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-306daa58-23b5-4d41-87f0-e2c1c5a282d7 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:51.846162 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-306daa58-23b5-4d41-87f0-e2c1c5a282d7 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] 10.222.0.16 "GET /compute/v2.1/servers/de210dbd-1398-413f-9a0f-63cae300bd03" status: 200 len: 1352 microversion: 2.25 time: 0.140208 Jul 16 21:16:51.846818 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 979/2039] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:16:51 2022] GET /compute/v2.1/servers/de210dbd-1398-413f-9a0f-63cae300bd03 => generated 1352 bytes in 142 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 16 21:16:51.851923 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-2579d4f8-16e8-4cce-ab4f-74dcf8de7793 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:51.852886 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-2579d4f8-16e8-4cce-ab4f-74dcf8de7793 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] [instance: f03a2a83-e007-444a-b46b-70eb46bb242b] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:51.860680 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2579d4f8-16e8-4cce-ab4f-74dcf8de7793 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:51.861581 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2579d4f8-16e8-4cce-ab4f-74dcf8de7793 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:51.862789 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2579d4f8-16e8-4cce-ab4f-74dcf8de7793 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:51.863403 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2579d4f8-16e8-4cce-ab4f-74dcf8de7793 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:52.074691 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-33d75880-c620-4b37-a307-59f8a53ecaa9 tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b7a14721381b4f2cb4def5be76b7b645', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '57c1e99ad9884d79812e95fbc81512da', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:52.089547 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-33d75880-c620-4b37-a307-59f8a53ecaa9 tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] 10.222.0.16 "GET /compute/v2.1/servers/e6844cd4-a382-45cc-ab21-bd7f2ede46e2" status: 200 len: 1509 microversion: 2.1 time: 0.356373 Jul 16 21:16:52.090627 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1061/2040] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:16:51 2022] GET /compute/v2.1/servers/e6844cd4-a382-45cc-ab21-bd7f2ede46e2 => generated 1509 bytes in 361 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:16:52.096404 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-7f76f61d-c5f1-4627-b668-82673ba8d293 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:52.097034 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-7f76f61d-c5f1-4627-b668-82673ba8d293 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] [instance: 6cb4e66a-eba2-42b7-91e8-09ee7b2a5a4f] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:52.121695 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7f76f61d-c5f1-4627-b668-82673ba8d293 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:52.122254 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7f76f61d-c5f1-4627-b668-82673ba8d293 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:52.128925 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7f76f61d-c5f1-4627-b668-82673ba8d293 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:52.129397 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7f76f61d-c5f1-4627-b668-82673ba8d293 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:52.152600 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-2579d4f8-16e8-4cce-ab4f-74dcf8de7793 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'daf3c9e6634e4ae1bfb492eae1f44c93', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '24c64db8082648c8837c16352252f96d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:52.168709 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-2579d4f8-16e8-4cce-ab4f-74dcf8de7793 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] 10.222.0.16 "GET /compute/v2.1/servers/f03a2a83-e007-444a-b46b-70eb46bb242b" status: 200 len: 1496 microversion: 2.1 time: 0.318466 Jul 16 21:16:52.178195 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 980/2041] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:16:51 2022] GET /compute/v2.1/servers/f03a2a83-e007-444a-b46b-70eb46bb242b => generated 1496 bytes in 322 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:16:52.223828 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-7f76f61d-c5f1-4627-b668-82673ba8d293 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '71e9304fc50447d6aac99703b5b6e4e1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e84496d78d4481e9d7648128868bca2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:52.231707 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-7f76f61d-c5f1-4627-b668-82673ba8d293 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] 10.222.0.16 "GET /compute/v2.1/servers/6cb4e66a-eba2-42b7-91e8-09ee7b2a5a4f" status: 200 len: 1221 microversion: 2.1 time: 0.139582 Jul 16 21:16:52.232286 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1062/2042] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:16:52 2022] GET /compute/v2.1/servers/6cb4e66a-eba2-42b7-91e8-09ee7b2a5a4f => generated 1221 bytes in 141 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:16:52.284788 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-0701dd7a-9434-4bf9-826b-b80652ecae52 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:52.285390 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-0701dd7a-9434-4bf9-826b-b80652ecae52 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] [instance: 9657dddc-2416-4c95-aab3-d6b6f9673369] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:52.291543 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0701dd7a-9434-4bf9-826b-b80652ecae52 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:52.291995 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0701dd7a-9434-4bf9-826b-b80652ecae52 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:52.292762 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0701dd7a-9434-4bf9-826b-b80652ecae52 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:52.293226 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0701dd7a-9434-4bf9-826b-b80652ecae52 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:52.380433 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-0701dd7a-9434-4bf9-826b-b80652ecae52 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '015ac1bbb2df487e99e5bddc60ee6923', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9f5f03e09936489fa1505188cefe3445', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:52.391454 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-0701dd7a-9434-4bf9-826b-b80652ecae52 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '015ac1bbb2df487e99e5bddc60ee6923', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9f5f03e09936489fa1505188cefe3445', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:52.392967 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-0701dd7a-9434-4bf9-826b-b80652ecae52 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '015ac1bbb2df487e99e5bddc60ee6923', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9f5f03e09936489fa1505188cefe3445', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:52.394400 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-0701dd7a-9434-4bf9-826b-b80652ecae52 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] 10.222.0.16 "GET /compute/v2.1/servers/9657dddc-2416-4c95-aab3-d6b6f9673369" status: 200 len: 1365 microversion: 2.47 time: 0.113566 Jul 16 21:16:52.406283 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 981/2043] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:16:52 2022] GET /compute/v2.1/servers/9657dddc-2416-4c95-aab3-d6b6f9673369 => generated 1365 bytes in 126 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:16:52.692414 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-361ef9d3-2cee-4b03-a164-2ea3839bb5b2 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:52.693032 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-361ef9d3-2cee-4b03-a164-2ea3839bb5b2 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] [instance: 11194822-5d42-45f4-99d0-88c70c808ddd] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:52.699711 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-361ef9d3-2cee-4b03-a164-2ea3839bb5b2 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:52.700104 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-361ef9d3-2cee-4b03-a164-2ea3839bb5b2 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:52.700884 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-361ef9d3-2cee-4b03-a164-2ea3839bb5b2 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:52.701431 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-361ef9d3-2cee-4b03-a164-2ea3839bb5b2 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:52.871128 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-f2d982ff-510e-47d8-997b-ec2ef0658e23 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:52.871128 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-f2d982ff-510e-47d8-997b-ec2ef0658e23 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] [instance: de210dbd-1398-413f-9a0f-63cae300bd03] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:52.874752 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f2d982ff-510e-47d8-997b-ec2ef0658e23 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:52.875507 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f2d982ff-510e-47d8-997b-ec2ef0658e23 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:52.876310 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f2d982ff-510e-47d8-997b-ec2ef0658e23 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:52.876747 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f2d982ff-510e-47d8-997b-ec2ef0658e23 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:52.915525 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f2d982ff-510e-47d8-997b-ec2ef0658e23 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:52.915715 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f2d982ff-510e-47d8-997b-ec2ef0658e23 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:52.916056 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.objects.instance [None req-f2d982ff-510e-47d8-997b-ec2ef0658e23 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lazy-loading 'fault' on Instance uuid de210dbd-1398-413f-9a0f-63cae300bd03 {{(pid=138681) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:16:52.980531 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-361ef9d3-2cee-4b03-a164-2ea3839bb5b2 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd4144b5388b46a989a41a2f556bb508', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a90d4bcd25ff4bf68bdc194180fbbe09', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:52.997074 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-361ef9d3-2cee-4b03-a164-2ea3839bb5b2 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] 10.222.0.16 "GET /compute/v2.1/servers/11194822-5d42-45f4-99d0-88c70c808ddd" status: 200 len: 1531 microversion: 2.1 time: 0.308285 Jul 16 21:16:53.007107 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-f2d982ff-510e-47d8-997b-ec2ef0658e23 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:53.011574 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1063/2044] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:16:52 2022] GET /compute/v2.1/servers/11194822-5d42-45f4-99d0-88c70c808ddd => generated 1531 bytes in 324 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:16:53.020346 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-f2d982ff-510e-47d8-997b-ec2ef0658e23 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:53.021876 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-f2d982ff-510e-47d8-997b-ec2ef0658e23 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '578b87f863b34f57a571f7e56d44f118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd9b92fc544c544f686ac2ab7ff5123b6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:53.023087 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-f2d982ff-510e-47d8-997b-ec2ef0658e23 tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] 10.222.0.16 "GET /compute/v2.1/servers/de210dbd-1398-413f-9a0f-63cae300bd03" status: 200 len: 1352 microversion: 2.25 time: 0.162232 Jul 16 21:16:53.023658 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 982/2045] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:16:52 2022] GET /compute/v2.1/servers/de210dbd-1398-413f-9a0f-63cae300bd03 => generated 1352 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 16 21:16:53.112548 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-50c64aa5-293f-4b0e-a663-8e19537b30e0 tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:53.112548 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-50c64aa5-293f-4b0e-a663-8e19537b30e0 tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] [instance: e6844cd4-a382-45cc-ab21-bd7f2ede46e2] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:53.116197 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-50c64aa5-293f-4b0e-a663-8e19537b30e0 tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:53.116654 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-50c64aa5-293f-4b0e-a663-8e19537b30e0 tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:53.117471 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-50c64aa5-293f-4b0e-a663-8e19537b30e0 tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:53.118070 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-50c64aa5-293f-4b0e-a663-8e19537b30e0 tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:53.122759 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:16:53.187825 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-d22cf0d0-7a38-4baa-b682-bc9cc8e10cee tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:53.188562 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-d22cf0d0-7a38-4baa-b682-bc9cc8e10cee tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] [instance: f03a2a83-e007-444a-b46b-70eb46bb242b] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:53.195129 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d22cf0d0-7a38-4baa-b682-bc9cc8e10cee tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:53.195609 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d22cf0d0-7a38-4baa-b682-bc9cc8e10cee tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:53.196392 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d22cf0d0-7a38-4baa-b682-bc9cc8e10cee tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:53.196849 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d22cf0d0-7a38-4baa-b682-bc9cc8e10cee tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:53.484254 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-50c64aa5-293f-4b0e-a663-8e19537b30e0 tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b7a14721381b4f2cb4def5be76b7b645', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '57c1e99ad9884d79812e95fbc81512da', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:53.528199 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-50c64aa5-293f-4b0e-a663-8e19537b30e0 tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] 10.222.0.16 "GET /compute/v2.1/servers/e6844cd4-a382-45cc-ab21-bd7f2ede46e2" status: 200 len: 1509 microversion: 2.1 time: 0.424218 Jul 16 21:16:53.531083 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1064/2046] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:16:53 2022] GET /compute/v2.1/servers/e6844cd4-a382-45cc-ab21-bd7f2ede46e2 => generated 1509 bytes in 428 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:16:53.543590 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-ddf750c8-8190-49fb-b404-29d1e60f1e7b tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:53.555980 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-ddf750c8-8190-49fb-b404-29d1e60f1e7b tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] [instance: 6cb4e66a-eba2-42b7-91e8-09ee7b2a5a4f] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:53.566622 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ddf750c8-8190-49fb-b404-29d1e60f1e7b tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:53.567940 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ddf750c8-8190-49fb-b404-29d1e60f1e7b tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:53.568093 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ddf750c8-8190-49fb-b404-29d1e60f1e7b tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:53.568588 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ddf750c8-8190-49fb-b404-29d1e60f1e7b tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:54.051441 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-d22cf0d0-7a38-4baa-b682-bc9cc8e10cee tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'daf3c9e6634e4ae1bfb492eae1f44c93', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '24c64db8082648c8837c16352252f96d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:54.059337 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-d22cf0d0-7a38-4baa-b682-bc9cc8e10cee tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] 10.222.0.16 "GET /compute/v2.1/servers/f03a2a83-e007-444a-b46b-70eb46bb242b" status: 200 len: 1496 microversion: 2.1 time: 0.877289 Jul 16 21:16:54.059928 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 983/2047] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:16:53 2022] GET /compute/v2.1/servers/f03a2a83-e007-444a-b46b-70eb46bb242b => generated 1496 bytes in 879 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:16:54.065169 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-d0e6542b-9cd8-4b03-80cb-ecd017465f2f tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:54.065935 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-d0e6542b-9cd8-4b03-80cb-ecd017465f2f tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] [instance: 9657dddc-2416-4c95-aab3-d6b6f9673369] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:54.072333 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d0e6542b-9cd8-4b03-80cb-ecd017465f2f tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:54.072796 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d0e6542b-9cd8-4b03-80cb-ecd017465f2f tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:54.073576 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d0e6542b-9cd8-4b03-80cb-ecd017465f2f tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:54.076639 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d0e6542b-9cd8-4b03-80cb-ecd017465f2f tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:54.117854 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-ddf750c8-8190-49fb-b404-29d1e60f1e7b tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '71e9304fc50447d6aac99703b5b6e4e1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e84496d78d4481e9d7648128868bca2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:54.126707 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-ddf750c8-8190-49fb-b404-29d1e60f1e7b tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] 10.222.0.16 "GET /compute/v2.1/servers/6cb4e66a-eba2-42b7-91e8-09ee7b2a5a4f" status: 200 len: 1277 microversion: 2.1 time: 0.592891 Jul 16 21:16:54.127525 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1065/2048] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:16:53 2022] GET /compute/v2.1/servers/6cb4e66a-eba2-42b7-91e8-09ee7b2a5a4f => generated 1277 bytes in 594 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:16:54.132081 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-4ccb9e14-ac87-4874-a326-f58966f7fb64 req-58377a13-cfba-4945-bb74-f1adf5770090 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "11194822-5d42-45f4-99d0-88c70c808ddd", "name": "network-vif-unplugged", "status": "completed", "tag": "1e6a5dec-046c-4c31-bb3f-1b8d00690085"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:16:54.140884 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-4ccb9e14-ac87-4874-a326-f58966f7fb64 req-58377a13-cfba-4945-bb74-f1adf5770090 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:54.141338 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-4ccb9e14-ac87-4874-a326-f58966f7fb64 req-58377a13-cfba-4945-bb74-f1adf5770090 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:54.155432 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-4ccb9e14-ac87-4874-a326-f58966f7fb64 req-58377a13-cfba-4945-bb74-f1adf5770090 service nova] Creating event network-vif-unplugged:1e6a5dec-046c-4c31-bb3f-1b8d00690085 for instance 11194822-5d42-45f4-99d0-88c70c808ddd on nch1-master-1 Jul 16 21:16:54.160949 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-4ccb9e14-ac87-4874-a326-f58966f7fb64 req-58377a13-cfba-4945-bb74-f1adf5770090 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.032468 Jul 16 21:16:54.161564 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1066/2049] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:16:54 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:16:54.168935 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-a53c29bc-6ef7-448a-b709-373156522043 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:54.173728 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-a53c29bc-6ef7-448a-b709-373156522043 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] [instance: 11194822-5d42-45f4-99d0-88c70c808ddd] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:54.176970 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a53c29bc-6ef7-448a-b709-373156522043 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:54.177430 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a53c29bc-6ef7-448a-b709-373156522043 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:54.178255 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a53c29bc-6ef7-448a-b709-373156522043 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:54.178707 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a53c29bc-6ef7-448a-b709-373156522043 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:54.213661 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-d0e6542b-9cd8-4b03-80cb-ecd017465f2f tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '015ac1bbb2df487e99e5bddc60ee6923', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9f5f03e09936489fa1505188cefe3445', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:54.224265 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-d0e6542b-9cd8-4b03-80cb-ecd017465f2f tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '015ac1bbb2df487e99e5bddc60ee6923', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9f5f03e09936489fa1505188cefe3445', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:54.225842 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-d0e6542b-9cd8-4b03-80cb-ecd017465f2f tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '015ac1bbb2df487e99e5bddc60ee6923', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9f5f03e09936489fa1505188cefe3445', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:54.227225 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-d0e6542b-9cd8-4b03-80cb-ecd017465f2f tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] 10.222.0.16 "GET /compute/v2.1/servers/9657dddc-2416-4c95-aab3-d6b6f9673369" status: 200 len: 1365 microversion: 2.47 time: 0.166330 Jul 16 21:16:54.227786 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 984/2050] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:16:54 2022] GET /compute/v2.1/servers/9657dddc-2416-4c95-aab3-d6b6f9673369 => generated 1365 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:16:54.232373 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-fdd5bec7-0194-46c7-ad87-ae79126494ab tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:54.232967 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-fdd5bec7-0194-46c7-ad87-ae79126494ab tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] [instance: de210dbd-1398-413f-9a0f-63cae300bd03] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:54.240061 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fdd5bec7-0194-46c7-ad87-ae79126494ab tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:54.240510 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fdd5bec7-0194-46c7-ad87-ae79126494ab tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:54.241284 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fdd5bec7-0194-46c7-ad87-ae79126494ab tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:54.241718 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fdd5bec7-0194-46c7-ad87-ae79126494ab tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:54.257263 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-fdd5bec7-0194-46c7-ad87-ae79126494ab tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] HTTP exception thrown: Instance de210dbd-1398-413f-9a0f-63cae300bd03 could not be found. Jul 16 21:16:54.258924 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-fdd5bec7-0194-46c7-ad87-ae79126494ab tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Request method failure captured: Jul 16 21:16:54.258924 ncdv-master-1 devstack@n-api.service[138681]: request: GET /compute/v2.1/servers/de210dbd-1398-413f-9a0f-63cae300bd03 HTTP/1.1 Jul 16 21:16:54.258924 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:16:54.258924 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:16:54.258924 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:16:54.258924 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 21:16:54.258924 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:16:54.258924 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:16:54.258924 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:16:54.258924 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:16:54.258924 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0ylBOHnbwqerAHhqONpFSBMhwk4nWfHPAHc-OTgMvb17y3jQh2-2K7JnAKTp7DvWJ_uO-bYr8xJfvvOUSkF97J9lR0DMJJ8o4htp9CLMoM61BjIIA0xJtMBeTA8oUgOHWMLAymuEaOlZbY4tC3l04tVeZqVEgsWsWeWmyuuhOz7GDcE Jul 16 21:16:54.258924 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:16:54.258924 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:16:54.258924 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:16:54.258924 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:16:54.258924 ncdv-master-1 devstack@n-api.service[138681]: X-Openstack-Nova-Api-Version: 2.25 Jul 16 21:16:54.258924 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:16:54.258924 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:16:54.258924 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: d9b92fc544c544f686ac2ab7ff5123b6 Jul 16 21:16:54.258924 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-LiveAutoBlockMigrationV225Test-1461646193 Jul 16 21:16:54.258924 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:16:54.258924 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:16:54.258924 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/d9b92fc544c544f686ac2ab7ff5123b6"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/d9b92fc544c544f686ac2ab7ff5123b6"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_d9b92fc544c544f686ac2ab7ff5123b6"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/d9b92fc544c544f686ac2ab7ff5123b6"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:16:54.258924 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-LiveAutoBlockMigrationV225Test-1461646193 Jul 16 21:16:54.258924 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: d9b92fc544c544f686ac2ab7ff5123b6 Jul 16 21:16:54.258924 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-LiveAutoBlockMigrationV225Test-1461646193 Jul 16 21:16:54.258924 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-LiveAutoBlockMigrationV225Test-1461646193-project Jul 16 21:16:54.258924 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:16:54.258924 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:16:54.258924 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: 578b87f863b34f57a571f7e56d44f118 Jul 16 21:16:54.258924 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-LiveAutoBlockMigrationV225Test-1461646193-project Jul 16 21:16:54.258924 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:16:54.258924 ncdv-master-1 devstack@n-api.service[138681]: exception: Instance de210dbd-1398-413f-9a0f-63cae300bd03 could not be found. Jul 16 21:16:54.258924 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': 'de210dbd-1398-413f-9a0f-63cae300bd03'} Jul 16 21:16:54.258924 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:16:54.258924 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:16:54.258924 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:16:54.258924 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:16:54.258924 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:16:54.258924 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:16:54.258924 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:16:54.258924 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:16:54.258924 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:16:54.258924 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:16:54.258924 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance de210dbd-1398-413f-9a0f-63cae300bd03 could not be found. Jul 16 21:16:54.258924 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:16:54.258924 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:16:54.258924 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:16:54.258924 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:16:54.258924 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:16:54.258924 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:16:54.258924 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:16:54.258924 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:16:54.258924 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:16:54.258924 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:16:54.258924 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:16:54.258924 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:16:54.258924 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:16:54.258924 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:16:54.258924 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:16:54.258924 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:16:54.258924 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance de210dbd-1398-413f-9a0f-63cae300bd03 could not be found. Jul 16 21:16:54.258924 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:16:54.262466 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:16:54.262466 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:16:54.262466 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:16:54.262466 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:16:54.262466 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:16:54.262466 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:16:54.262466 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:16:54.262466 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance de210dbd-1398-413f-9a0f-63cae300bd03 could not be found. Jul 16 21:16:54.262466 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:16:54.262466 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-fdd5bec7-0194-46c7-ad87-ae79126494ab tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] Returning 404 to user: Instance de210dbd-1398-413f-9a0f-63cae300bd03 could not be found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:16:54.262466 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-fdd5bec7-0194-46c7-ad87-ae79126494ab tempest-LiveAutoBlockMigrationV225Test-1461646193 tempest-LiveAutoBlockMigrationV225Test-1461646193-project] 10.222.0.16 "GET /compute/v2.1/servers/de210dbd-1398-413f-9a0f-63cae300bd03" status: 404 len: 111 microversion: 2.25 time: 0.031568 Jul 16 21:16:54.262466 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 985/2051] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:16:54 2022] GET /compute/v2.1/servers/de210dbd-1398-413f-9a0f-63cae300bd03 => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Jul 16 21:16:54.512632 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-a53c29bc-6ef7-448a-b709-373156522043 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd4144b5388b46a989a41a2f556bb508', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a90d4bcd25ff4bf68bdc194180fbbe09', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:54.521888 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-a53c29bc-6ef7-448a-b709-373156522043 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] 10.222.0.16 "GET /compute/v2.1/servers/11194822-5d42-45f4-99d0-88c70c808ddd" status: 200 len: 1519 microversion: 2.1 time: 0.357189 Jul 16 21:16:54.538232 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1067/2052] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:16:54 2022] GET /compute/v2.1/servers/11194822-5d42-45f4-99d0-88c70c808ddd => generated 1519 bytes in 365 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:16:54.549411 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-b35af1ff-fd19-4e68-9285-07ac833599c6 tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:54.550198 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-b35af1ff-fd19-4e68-9285-07ac833599c6 tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] [instance: e6844cd4-a382-45cc-ab21-bd7f2ede46e2] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:54.558502 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b35af1ff-fd19-4e68-9285-07ac833599c6 tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:54.558965 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b35af1ff-fd19-4e68-9285-07ac833599c6 tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:54.560843 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b35af1ff-fd19-4e68-9285-07ac833599c6 tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:54.560843 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b35af1ff-fd19-4e68-9285-07ac833599c6 tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:54.593131 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:16:54.809252 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-b35af1ff-fd19-4e68-9285-07ac833599c6 tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b7a14721381b4f2cb4def5be76b7b645', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '57c1e99ad9884d79812e95fbc81512da', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:54.818373 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-b35af1ff-fd19-4e68-9285-07ac833599c6 tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] 10.222.0.16 "GET /compute/v2.1/servers/e6844cd4-a382-45cc-ab21-bd7f2ede46e2" status: 200 len: 1509 microversion: 2.1 time: 0.274043 Jul 16 21:16:54.828150 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 986/2053] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:16:54 2022] GET /compute/v2.1/servers/e6844cd4-a382-45cc-ab21-bd7f2ede46e2 => generated 1509 bytes in 284 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:16:55.077346 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-aa585ce0-2389-487c-a942-1f0130cbdf64 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:55.078170 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-aa585ce0-2389-487c-a942-1f0130cbdf64 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] [instance: f03a2a83-e007-444a-b46b-70eb46bb242b] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:55.081451 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_api:SELECT=94,nova_api:DELETE=1,nova_api:UPDATE=2,nova_api:INSERT=3 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:16:55.087365 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-aa585ce0-2389-487c-a942-1f0130cbdf64 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:55.088735 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-aa585ce0-2389-487c-a942-1f0130cbdf64 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:55.089723 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-aa585ce0-2389-487c-a942-1f0130cbdf64 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:55.090423 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-aa585ce0-2389-487c-a942-1f0130cbdf64 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:55.144736 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-c32856e6-6220-493a-bbcf-6d15ffa4fa67 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:55.145536 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-c32856e6-6220-493a-bbcf-6d15ffa4fa67 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] [instance: 6cb4e66a-eba2-42b7-91e8-09ee7b2a5a4f] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:55.158468 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c32856e6-6220-493a-bbcf-6d15ffa4fa67 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:55.159108 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c32856e6-6220-493a-bbcf-6d15ffa4fa67 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:55.160143 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c32856e6-6220-493a-bbcf-6d15ffa4fa67 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:55.162871 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c32856e6-6220-493a-bbcf-6d15ffa4fa67 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:55.322625 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-c32856e6-6220-493a-bbcf-6d15ffa4fa67 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '71e9304fc50447d6aac99703b5b6e4e1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e84496d78d4481e9d7648128868bca2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:55.347179 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-c32856e6-6220-493a-bbcf-6d15ffa4fa67 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] 10.222.0.16 "GET /compute/v2.1/servers/6cb4e66a-eba2-42b7-91e8-09ee7b2a5a4f" status: 200 len: 1275 microversion: 2.1 time: 0.205724 Jul 16 21:16:55.348063 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 987/2054] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:16:55 2022] GET /compute/v2.1/servers/6cb4e66a-eba2-42b7-91e8-09ee7b2a5a4f => generated 1275 bytes in 208 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:16:55.364997 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-6f45610f-f0d1-408b-aaa2-3f0dfacff8e7 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:55.366796 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-6f45610f-f0d1-408b-aaa2-3f0dfacff8e7 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] [instance: 9657dddc-2416-4c95-aab3-d6b6f9673369] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:55.376280 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6f45610f-f0d1-408b-aaa2-3f0dfacff8e7 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:55.377078 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6f45610f-f0d1-408b-aaa2-3f0dfacff8e7 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:55.381244 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6f45610f-f0d1-408b-aaa2-3f0dfacff8e7 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:55.382421 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6f45610f-f0d1-408b-aaa2-3f0dfacff8e7 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:55.458005 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-aa585ce0-2389-487c-a942-1f0130cbdf64 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'daf3c9e6634e4ae1bfb492eae1f44c93', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '24c64db8082648c8837c16352252f96d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:55.466918 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-aa585ce0-2389-487c-a942-1f0130cbdf64 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] 10.222.0.16 "GET /compute/v2.1/servers/f03a2a83-e007-444a-b46b-70eb46bb242b" status: 200 len: 1496 microversion: 2.1 time: 0.393742 Jul 16 21:16:55.468259 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1068/2055] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:16:55 2022] GET /compute/v2.1/servers/f03a2a83-e007-444a-b46b-70eb46bb242b => generated 1496 bytes in 396 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:16:55.546414 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-bb0297f6-0084-47a2-970f-645a76ff7762 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:55.547712 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-bb0297f6-0084-47a2-970f-645a76ff7762 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] [instance: 11194822-5d42-45f4-99d0-88c70c808ddd] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:55.554433 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-bb0297f6-0084-47a2-970f-645a76ff7762 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:55.555280 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-bb0297f6-0084-47a2-970f-645a76ff7762 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:55.556291 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-bb0297f6-0084-47a2-970f-645a76ff7762 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:55.556942 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-bb0297f6-0084-47a2-970f-645a76ff7762 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:55.625376 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-6f45610f-f0d1-408b-aaa2-3f0dfacff8e7 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '015ac1bbb2df487e99e5bddc60ee6923', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9f5f03e09936489fa1505188cefe3445', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:55.639163 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-6f45610f-f0d1-408b-aaa2-3f0dfacff8e7 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '015ac1bbb2df487e99e5bddc60ee6923', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9f5f03e09936489fa1505188cefe3445', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:55.642197 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-6f45610f-f0d1-408b-aaa2-3f0dfacff8e7 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '015ac1bbb2df487e99e5bddc60ee6923', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9f5f03e09936489fa1505188cefe3445', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:55.644467 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-6f45610f-f0d1-408b-aaa2-3f0dfacff8e7 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] 10.222.0.16 "GET /compute/v2.1/servers/9657dddc-2416-4c95-aab3-d6b6f9673369" status: 200 len: 1365 microversion: 2.47 time: 0.283930 Jul 16 21:16:55.645298 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 988/2056] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:16:55 2022] GET /compute/v2.1/servers/9657dddc-2416-4c95-aab3-d6b6f9673369 => generated 1365 bytes in 286 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:16:55.835956 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-bb0297f6-0084-47a2-970f-645a76ff7762 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd4144b5388b46a989a41a2f556bb508', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a90d4bcd25ff4bf68bdc194180fbbe09', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:55.845764 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-963524ec-0172-4a88-9829-c999e2de1403 tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:55.846394 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-963524ec-0172-4a88-9829-c999e2de1403 tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] [instance: e6844cd4-a382-45cc-ab21-bd7f2ede46e2] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:55.859613 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-bb0297f6-0084-47a2-970f-645a76ff7762 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] 10.222.0.16 "GET /compute/v2.1/servers/11194822-5d42-45f4-99d0-88c70c808ddd" status: 200 len: 1519 microversion: 2.1 time: 0.316614 Jul 16 21:16:55.860452 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-963524ec-0172-4a88-9829-c999e2de1403 tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:55.861221 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-963524ec-0172-4a88-9829-c999e2de1403 tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:55.862236 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-963524ec-0172-4a88-9829-c999e2de1403 tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:55.863661 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-963524ec-0172-4a88-9829-c999e2de1403 tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:55.864834 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1069/2057] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:16:55 2022] GET /compute/v2.1/servers/11194822-5d42-45f4-99d0-88c70c808ddd => generated 1519 bytes in 323 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:16:56.273443 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-963524ec-0172-4a88-9829-c999e2de1403 tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b7a14721381b4f2cb4def5be76b7b645', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '57c1e99ad9884d79812e95fbc81512da', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:56.297275 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-963524ec-0172-4a88-9829-c999e2de1403 tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] 10.222.0.16 "GET /compute/v2.1/servers/e6844cd4-a382-45cc-ab21-bd7f2ede46e2" status: 200 len: 1509 microversion: 2.1 time: 0.455953 Jul 16 21:16:56.298082 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 989/2058] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:16:55 2022] GET /compute/v2.1/servers/e6844cd4-a382-45cc-ab21-bd7f2ede46e2 => generated 1509 bytes in 457 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:16:56.347805 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-2d7ae4a0-4589-4166-8d8b-3f375068416d req-572e4ca9-f527-441d-b29c-db487e5837a1 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "e6844cd4-a382-45cc-ab21-bd7f2ede46e2", "name": "network-vif-deleted", "tag": "b38de153-224c-42ed-a714-30fda6897dfb"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:16:56.360486 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-2d7ae4a0-4589-4166-8d8b-3f375068416d req-572e4ca9-f527-441d-b29c-db487e5837a1 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:56.368235 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-529eef21-77d6-48e6-b8c1-04d4f2d0bdb3 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:56.369283 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-529eef21-77d6-48e6-b8c1-04d4f2d0bdb3 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] [instance: 6cb4e66a-eba2-42b7-91e8-09ee7b2a5a4f] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:56.375240 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-2d7ae4a0-4589-4166-8d8b-3f375068416d req-572e4ca9-f527-441d-b29c-db487e5837a1 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.013s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:56.383897 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-529eef21-77d6-48e6-b8c1-04d4f2d0bdb3 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:56.387644 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-529eef21-77d6-48e6-b8c1-04d4f2d0bdb3 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:56.388779 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-529eef21-77d6-48e6-b8c1-04d4f2d0bdb3 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:56.390978 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-529eef21-77d6-48e6-b8c1-04d4f2d0bdb3 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:56.423665 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-2d7ae4a0-4589-4166-8d8b-3f375068416d req-572e4ca9-f527-441d-b29c-db487e5837a1 service nova] Creating event network-vif-deleted:b38de153-224c-42ed-a714-30fda6897dfb for instance e6844cd4-a382-45cc-ab21-bd7f2ede46e2 on nch1-master-1 Jul 16 21:16:56.430666 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-2d7ae4a0-4589-4166-8d8b-3f375068416d req-572e4ca9-f527-441d-b29c-db487e5837a1 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.091377 Jul 16 21:16:56.432599 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1070/2059] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:16:56 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:16:56.506273 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-8cca9af3-1a67-4bf1-a551-691abfceda50 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:56.507992 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-8cca9af3-1a67-4bf1-a551-691abfceda50 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] [instance: f03a2a83-e007-444a-b46b-70eb46bb242b] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:56.534735 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8cca9af3-1a67-4bf1-a551-691abfceda50 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.018s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:56.535490 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8cca9af3-1a67-4bf1-a551-691abfceda50 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:56.536654 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8cca9af3-1a67-4bf1-a551-691abfceda50 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:56.543377 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8cca9af3-1a67-4bf1-a551-691abfceda50 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:56.911043 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-529eef21-77d6-48e6-b8c1-04d4f2d0bdb3 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '71e9304fc50447d6aac99703b5b6e4e1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e84496d78d4481e9d7648128868bca2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:56.920847 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-529eef21-77d6-48e6-b8c1-04d4f2d0bdb3 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] 10.222.0.16 "GET /compute/v2.1/servers/6cb4e66a-eba2-42b7-91e8-09ee7b2a5a4f" status: 200 len: 1317 microversion: 2.1 time: 0.558369 Jul 16 21:16:56.921542 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 990/2060] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:16:56 2022] GET /compute/v2.1/servers/6cb4e66a-eba2-42b7-91e8-09ee7b2a5a4f => generated 1317 bytes in 560 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:16:56.927712 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-6091f647-053e-418e-8420-35ceeb25d9ae tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:56.928440 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-6091f647-053e-418e-8420-35ceeb25d9ae tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] [instance: 9657dddc-2416-4c95-aab3-d6b6f9673369] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:56.934870 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6091f647-053e-418e-8420-35ceeb25d9ae tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:56.935418 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6091f647-053e-418e-8420-35ceeb25d9ae tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:56.935623 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-8cca9af3-1a67-4bf1-a551-691abfceda50 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'daf3c9e6634e4ae1bfb492eae1f44c93', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '24c64db8082648c8837c16352252f96d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:56.936206 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6091f647-053e-418e-8420-35ceeb25d9ae tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:56.936775 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6091f647-053e-418e-8420-35ceeb25d9ae tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:56.945607 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-8cca9af3-1a67-4bf1-a551-691abfceda50 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] 10.222.0.16 "GET /compute/v2.1/servers/f03a2a83-e007-444a-b46b-70eb46bb242b" status: 200 len: 1496 microversion: 2.1 time: 0.454291 Jul 16 21:16:56.946235 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1071/2061] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:16:56 2022] GET /compute/v2.1/servers/f03a2a83-e007-444a-b46b-70eb46bb242b => generated 1496 bytes in 456 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:16:56.951872 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-78a026c9-5a09-4e56-a397-772abcbe1977 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:56.953157 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-78a026c9-5a09-4e56-a397-772abcbe1977 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] [instance: 11194822-5d42-45f4-99d0-88c70c808ddd] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:56.959747 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-78a026c9-5a09-4e56-a397-772abcbe1977 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:56.960393 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-78a026c9-5a09-4e56-a397-772abcbe1977 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:56.962542 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-78a026c9-5a09-4e56-a397-772abcbe1977 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:56.962542 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-78a026c9-5a09-4e56-a397-772abcbe1977 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:57.087698 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-6091f647-053e-418e-8420-35ceeb25d9ae tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '015ac1bbb2df487e99e5bddc60ee6923', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9f5f03e09936489fa1505188cefe3445', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:57.097419 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-6091f647-053e-418e-8420-35ceeb25d9ae tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '015ac1bbb2df487e99e5bddc60ee6923', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9f5f03e09936489fa1505188cefe3445', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:57.099195 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-6091f647-053e-418e-8420-35ceeb25d9ae tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '015ac1bbb2df487e99e5bddc60ee6923', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9f5f03e09936489fa1505188cefe3445', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:57.100628 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-6091f647-053e-418e-8420-35ceeb25d9ae tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] 10.222.0.16 "GET /compute/v2.1/servers/9657dddc-2416-4c95-aab3-d6b6f9673369" status: 200 len: 1365 microversion: 2.47 time: 0.177941 Jul 16 21:16:57.101730 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 991/2062] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:16:56 2022] GET /compute/v2.1/servers/9657dddc-2416-4c95-aab3-d6b6f9673369 => generated 1365 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:16:57.314722 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-6a7bad4c-ef2c-49f1-a0f6-e70737b92379 tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:57.316248 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-6a7bad4c-ef2c-49f1-a0f6-e70737b92379 tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] [instance: e6844cd4-a382-45cc-ab21-bd7f2ede46e2] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:57.330856 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6a7bad4c-ef2c-49f1-a0f6-e70737b92379 tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:57.331357 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6a7bad4c-ef2c-49f1-a0f6-e70737b92379 tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:57.332154 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6a7bad4c-ef2c-49f1-a0f6-e70737b92379 tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:57.332595 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6a7bad4c-ef2c-49f1-a0f6-e70737b92379 tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:57.352530 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-78a026c9-5a09-4e56-a397-772abcbe1977 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd4144b5388b46a989a41a2f556bb508', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a90d4bcd25ff4bf68bdc194180fbbe09', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:57.373009 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-78a026c9-5a09-4e56-a397-772abcbe1977 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] 10.222.0.16 "GET /compute/v2.1/servers/11194822-5d42-45f4-99d0-88c70c808ddd" status: 200 len: 1519 microversion: 2.1 time: 0.423394 Jul 16 21:16:57.373703 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1072/2063] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:16:56 2022] GET /compute/v2.1/servers/11194822-5d42-45f4-99d0-88c70c808ddd => generated 1519 bytes in 427 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:16:57.393070 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6a7bad4c-ef2c-49f1-a0f6-e70737b92379 tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:57.393746 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6a7bad4c-ef2c-49f1-a0f6-e70737b92379 tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:57.394395 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.objects.instance [None req-6a7bad4c-ef2c-49f1-a0f6-e70737b92379 tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] Lazy-loading 'fault' on Instance uuid e6844cd4-a382-45cc-ab21-bd7f2ede46e2 {{(pid=138681) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:16:57.487484 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-6a7bad4c-ef2c-49f1-a0f6-e70737b92379 tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b7a14721381b4f2cb4def5be76b7b645', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '57c1e99ad9884d79812e95fbc81512da', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:57.499899 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-6a7bad4c-ef2c-49f1-a0f6-e70737b92379 tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] 10.222.0.16 "GET /compute/v2.1/servers/e6844cd4-a382-45cc-ab21-bd7f2ede46e2" status: 200 len: 1305 microversion: 2.1 time: 0.189026 Jul 16 21:16:57.500680 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 992/2064] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:16:57 2022] GET /compute/v2.1/servers/e6844cd4-a382-45cc-ab21-bd7f2ede46e2 => generated 1305 bytes in 191 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:16:57.747385 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-1460e6ab-4915-4c6d-8f1c-88401ee2abf7 req-4fa1e4fb-ab2a-4e9c-bd0b-2ea49fd9be11 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "11194822-5d42-45f4-99d0-88c70c808ddd", "name": "network-vif-plugged", "status": "completed", "tag": "1e6a5dec-046c-4c31-bb3f-1b8d00690085"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:16:57.760252 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-1460e6ab-4915-4c6d-8f1c-88401ee2abf7 req-4fa1e4fb-ab2a-4e9c-bd0b-2ea49fd9be11 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:57.760911 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-1460e6ab-4915-4c6d-8f1c-88401ee2abf7 req-4fa1e4fb-ab2a-4e9c-bd0b-2ea49fd9be11 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:57.784303 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-1460e6ab-4915-4c6d-8f1c-88401ee2abf7 req-4fa1e4fb-ab2a-4e9c-bd0b-2ea49fd9be11 service nova] Creating event network-vif-plugged:1e6a5dec-046c-4c31-bb3f-1b8d00690085 for instance 11194822-5d42-45f4-99d0-88c70c808ddd on nch1-master-1 Jul 16 21:16:57.792811 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-1460e6ab-4915-4c6d-8f1c-88401ee2abf7 req-4fa1e4fb-ab2a-4e9c-bd0b-2ea49fd9be11 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.052694 Jul 16 21:16:57.803904 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1073/2065] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:16:57 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 65 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:16:57.938046 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-d78410a7-28d3-4cdd-818f-58541baad66f tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:57.938723 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-d78410a7-28d3-4cdd-818f-58541baad66f tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] [instance: 6cb4e66a-eba2-42b7-91e8-09ee7b2a5a4f] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:57.948883 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d78410a7-28d3-4cdd-818f-58541baad66f tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:57.949518 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d78410a7-28d3-4cdd-818f-58541baad66f tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:57.952958 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d78410a7-28d3-4cdd-818f-58541baad66f tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:57.954289 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d78410a7-28d3-4cdd-818f-58541baad66f tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:57.973698 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-ba2875a7-8e54-44bc-a2ce-c21f316e3cf3 req-74166f33-4b43-4d60-b645-2dfaae70d385 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "b9a4910a-f117-4834-b699-498c6431b9bb", "tag": "9ab69acd-f0a3-4355-b203-2c48e39cc3a1"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:16:58.000845 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-ba2875a7-8e54-44bc-a2ce-c21f316e3cf3 req-74166f33-4b43-4d60-b645-2dfaae70d385 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:58.003989 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-ba2875a7-8e54-44bc-a2ce-c21f316e3cf3 req-74166f33-4b43-4d60-b645-2dfaae70d385 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.010s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:58.032610 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-ba2875a7-8e54-44bc-a2ce-c21f316e3cf3 req-74166f33-4b43-4d60-b645-2dfaae70d385 service nova] Creating event network-changed:9ab69acd-f0a3-4355-b203-2c48e39cc3a1 for instance b9a4910a-f117-4834-b699-498c6431b9bb on nch2-master-1 Jul 16 21:16:58.039467 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-ba2875a7-8e54-44bc-a2ce-c21f316e3cf3 req-74166f33-4b43-4d60-b645-2dfaae70d385 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.080087 Jul 16 21:16:58.041030 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1074/2066] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:16:57 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:16:58.048220 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-864957bc-0a43-47f2-883b-98cdd3cd60fd tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:58.049280 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-864957bc-0a43-47f2-883b-98cdd3cd60fd tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] [instance: f03a2a83-e007-444a-b46b-70eb46bb242b] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:58.057273 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-864957bc-0a43-47f2-883b-98cdd3cd60fd tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:58.060933 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-864957bc-0a43-47f2-883b-98cdd3cd60fd tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:58.060933 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-864957bc-0a43-47f2-883b-98cdd3cd60fd tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:58.060933 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-864957bc-0a43-47f2-883b-98cdd3cd60fd tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:58.466486 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-864957bc-0a43-47f2-883b-98cdd3cd60fd tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'daf3c9e6634e4ae1bfb492eae1f44c93', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '24c64db8082648c8837c16352252f96d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:58.482565 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-864957bc-0a43-47f2-883b-98cdd3cd60fd tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] 10.222.0.16 "GET /compute/v2.1/servers/f03a2a83-e007-444a-b46b-70eb46bb242b" status: 200 len: 1496 microversion: 2.1 time: 0.436670 Jul 16 21:16:58.491596 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1075/2067] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:16:58 2022] GET /compute/v2.1/servers/f03a2a83-e007-444a-b46b-70eb46bb242b => generated 1496 bytes in 450 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:16:58.496675 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-03a65e12-bbc6-4551-bf55-916acfc24cac tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:58.497512 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-03a65e12-bbc6-4551-bf55-916acfc24cac tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] [instance: 9657dddc-2416-4c95-aab3-d6b6f9673369] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:58.506153 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-03a65e12-bbc6-4551-bf55-916acfc24cac tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:58.507122 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-03a65e12-bbc6-4551-bf55-916acfc24cac tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:58.508114 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-03a65e12-bbc6-4551-bf55-916acfc24cac tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:58.508791 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-03a65e12-bbc6-4551-bf55-916acfc24cac tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:58.556879 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-d78410a7-28d3-4cdd-818f-58541baad66f tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '71e9304fc50447d6aac99703b5b6e4e1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e84496d78d4481e9d7648128868bca2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:58.568850 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-d78410a7-28d3-4cdd-818f-58541baad66f tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] 10.222.0.16 "GET /compute/v2.1/servers/6cb4e66a-eba2-42b7-91e8-09ee7b2a5a4f" status: 200 len: 1317 microversion: 2.1 time: 0.634854 Jul 16 21:16:58.579595 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 993/2068] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:16:57 2022] GET /compute/v2.1/servers/6cb4e66a-eba2-42b7-91e8-09ee7b2a5a4f => generated 1317 bytes in 646 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:16:58.586972 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-ef812f75-97d5-47f2-85aa-cd06329b5ec1 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:58.587817 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-ef812f75-97d5-47f2-85aa-cd06329b5ec1 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] [instance: 11194822-5d42-45f4-99d0-88c70c808ddd] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:58.599723 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ef812f75-97d5-47f2-85aa-cd06329b5ec1 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:58.600384 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ef812f75-97d5-47f2-85aa-cd06329b5ec1 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:58.601314 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ef812f75-97d5-47f2-85aa-cd06329b5ec1 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:58.602665 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ef812f75-97d5-47f2-85aa-cd06329b5ec1 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:58.646537 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-03a65e12-bbc6-4551-bf55-916acfc24cac tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '015ac1bbb2df487e99e5bddc60ee6923', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9f5f03e09936489fa1505188cefe3445', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:58.658149 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-03a65e12-bbc6-4551-bf55-916acfc24cac tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '015ac1bbb2df487e99e5bddc60ee6923', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9f5f03e09936489fa1505188cefe3445', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:58.659723 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-03a65e12-bbc6-4551-bf55-916acfc24cac tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '015ac1bbb2df487e99e5bddc60ee6923', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9f5f03e09936489fa1505188cefe3445', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:58.661320 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-03a65e12-bbc6-4551-bf55-916acfc24cac tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] 10.222.0.16 "GET /compute/v2.1/servers/9657dddc-2416-4c95-aab3-d6b6f9673369" status: 200 len: 1365 microversion: 2.47 time: 0.168444 Jul 16 21:16:58.675263 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1076/2069] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:16:58 2022] GET /compute/v2.1/servers/9657dddc-2416-4c95-aab3-d6b6f9673369 => generated 1365 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:16:58.682343 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-c1704617-cb39-4b14-b75d-82e9b283a52b tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:58.683203 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-c1704617-cb39-4b14-b75d-82e9b283a52b tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] [instance: e6844cd4-a382-45cc-ab21-bd7f2ede46e2] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:58.695274 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c1704617-cb39-4b14-b75d-82e9b283a52b tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:58.696045 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c1704617-cb39-4b14-b75d-82e9b283a52b tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:58.697078 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c1704617-cb39-4b14-b75d-82e9b283a52b tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:58.697722 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c1704617-cb39-4b14-b75d-82e9b283a52b tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:58.714085 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-c1704617-cb39-4b14-b75d-82e9b283a52b tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] HTTP exception thrown: Instance e6844cd4-a382-45cc-ab21-bd7f2ede46e2 could not be found. Jul 16 21:16:58.717101 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-c1704617-cb39-4b14-b75d-82e9b283a52b tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] Request method failure captured: Jul 16 21:16:58.717101 ncdv-master-1 devstack@n-api.service[138680]: request: GET /compute/v2.1/servers/e6844cd4-a382-45cc-ab21-bd7f2ede46e2 HTTP/1.1 Jul 16 21:16:58.717101 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:16:58.717101 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:16:58.717101 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:16:58.717101 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 0 Jul 16 21:16:58.717101 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:16:58.717101 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:16:58.717101 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:16:58.717101 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:16:58.717101 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0yqvSs5UgxQxKA82EE830ekeZzD64HCYmKhqW4ubz2pm3RVOwENZ0Zu-pJqLcb3II1j4okuuS6d1EzSSqhj-LCfInXXWIIBtGJ3C4NuvjFkCNne7mTnaEoGL1Eq40mf85R2R38XmjzvDGni-yhyQ0K6VsJdJb-uuNQJDCPLFt47iex4 Jul 16 21:16:58.717101 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:16:58.717101 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:16:58.717101 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:16:58.717101 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:16:58.717101 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:16:58.717101 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:16:58.717101 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: 57c1e99ad9884d79812e95fbc81512da Jul 16 21:16:58.717101 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-ImagesNegativeTestJSON-1746139659 Jul 16 21:16:58.717101 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:16:58.717101 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:16:58.717101 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/57c1e99ad9884d79812e95fbc81512da"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/57c1e99ad9884d79812e95fbc81512da"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_57c1e99ad9884d79812e95fbc81512da"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/57c1e99ad9884d79812e95fbc81512da"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:16:58.717101 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-ImagesNegativeTestJSON-1746139659 Jul 16 21:16:58.717101 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: 57c1e99ad9884d79812e95fbc81512da Jul 16 21:16:58.717101 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-ImagesNegativeTestJSON-1746139659 Jul 16 21:16:58.717101 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-ImagesNegativeTestJSON-1746139659-project Jul 16 21:16:58.717101 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:16:58.717101 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:16:58.717101 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: b7a14721381b4f2cb4def5be76b7b645 Jul 16 21:16:58.717101 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-ImagesNegativeTestJSON-1746139659-project Jul 16 21:16:58.717101 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 21:16:58.717101 ncdv-master-1 devstack@n-api.service[138680]: exception: Instance e6844cd4-a382-45cc-ab21-bd7f2ede46e2 could not be found. Jul 16 21:16:58.717101 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': 'e6844cd4-a382-45cc-ab21-bd7f2ede46e2'} Jul 16 21:16:58.717101 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:16:58.717101 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:16:58.717101 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:16:58.717101 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:16:58.717101 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:16:58.717101 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:16:58.717101 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:16:58.717101 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:16:58.717101 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:16:58.717101 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:16:58.717101 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance e6844cd4-a382-45cc-ab21-bd7f2ede46e2 could not be found. Jul 16 21:16:58.717101 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:16:58.717101 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:16:58.717101 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:16:58.717101 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:16:58.717101 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:16:58.717101 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:16:58.717101 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:16:58.717101 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:16:58.717101 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:16:58.717101 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:16:58.717101 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:16:58.717101 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:16:58.717101 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:16:58.717101 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:16:58.717101 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:16:58.717101 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:16:58.717101 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance e6844cd4-a382-45cc-ab21-bd7f2ede46e2 could not be found. Jul 16 21:16:58.717101 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:16:58.717101 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:16:58.717101 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:16:58.720277 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:16:58.720277 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:16:58.720277 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:16:58.720277 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:16:58.720277 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:16:58.720277 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance e6844cd4-a382-45cc-ab21-bd7f2ede46e2 could not be found. Jul 16 21:16:58.720277 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:16:58.720277 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-c1704617-cb39-4b14-b75d-82e9b283a52b tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] Returning 404 to user: Instance e6844cd4-a382-45cc-ab21-bd7f2ede46e2 could not be found. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:16:58.720277 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-c1704617-cb39-4b14-b75d-82e9b283a52b tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] 10.222.0.16 "GET /compute/v2.1/servers/e6844cd4-a382-45cc-ab21-bd7f2ede46e2" status: 404 len: 111 microversion: 2.1 time: 0.042116 Jul 16 21:16:58.721551 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1077/2070] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:16:58 2022] GET /compute/v2.1/servers/e6844cd4-a382-45cc-ab21-bd7f2ede46e2 => generated 111 bytes in 45 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:16:58.739001 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-6985de0a-9025-48a7-9fe2-803f277076e0 tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] Action: 'action', calling method: >, body: {"createImage": {"name": "tempest-ImagesNegativeTestJSON-image-67541837", "metadata": {"image_type": "test"}}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:16:58.740204 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-6985de0a-9025-48a7-9fe2-803f277076e0 tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] [instance: e6844cd4-a382-45cc-ab21-bd7f2ede46e2] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:58.755181 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell0:SELECT=2 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:16:58.760038 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6985de0a-9025-48a7-9fe2-803f277076e0 tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:58.762617 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6985de0a-9025-48a7-9fe2-803f277076e0 tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:58.763766 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6985de0a-9025-48a7-9fe2-803f277076e0 tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:58.766563 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6985de0a-9025-48a7-9fe2-803f277076e0 tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:58.769543 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell0:SELECT=6 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:16:58.810817 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-6985de0a-9025-48a7-9fe2-803f277076e0 tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] HTTP exception thrown: Instance e6844cd4-a382-45cc-ab21-bd7f2ede46e2 could not be found. Jul 16 21:16:58.815721 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-6985de0a-9025-48a7-9fe2-803f277076e0 tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] Request method failure captured: Jul 16 21:16:58.815721 ncdv-master-1 devstack@n-api.service[138680]: request: POST /compute/v2.1/servers/e6844cd4-a382-45cc-ab21-bd7f2ede46e2/action HTTP/1.1 Jul 16 21:16:58.815721 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:16:58.815721 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:16:58.815721 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:16:58.815721 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 110 Jul 16 21:16:58.815721 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:16:58.815721 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:16:58.815721 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:16:58.815721 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:16:58.815721 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0yqvSs5UgxQxKA82EE830ekeZzD64HCYmKhqW4ubz2pm3RVOwENZ0Zu-pJqLcb3II1j4okuuS6d1EzSSqhj-LCfInXXWIIBtGJ3C4NuvjFkCNne7mTnaEoGL1Eq40mf85R2R38XmjzvDGni-yhyQ0K6VsJdJb-uuNQJDCPLFt47iex4 Jul 16 21:16:58.815721 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:16:58.815721 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:16:58.815721 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:16:58.815721 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:16:58.815721 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:16:58.815721 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:16:58.815721 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: 57c1e99ad9884d79812e95fbc81512da Jul 16 21:16:58.815721 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-ImagesNegativeTestJSON-1746139659 Jul 16 21:16:58.815721 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:16:58.815721 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:16:58.815721 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/57c1e99ad9884d79812e95fbc81512da"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/57c1e99ad9884d79812e95fbc81512da"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_57c1e99ad9884d79812e95fbc81512da"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/57c1e99ad9884d79812e95fbc81512da"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:16:58.815721 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-ImagesNegativeTestJSON-1746139659 Jul 16 21:16:58.815721 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: 57c1e99ad9884d79812e95fbc81512da Jul 16 21:16:58.815721 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-ImagesNegativeTestJSON-1746139659 Jul 16 21:16:58.815721 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-ImagesNegativeTestJSON-1746139659-project Jul 16 21:16:58.815721 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:16:58.815721 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:16:58.815721 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: b7a14721381b4f2cb4def5be76b7b645 Jul 16 21:16:58.815721 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-ImagesNegativeTestJSON-1746139659-project Jul 16 21:16:58.815721 ncdv-master-1 devstack@n-api.service[138680]: Jul 16 21:16:58.815721 ncdv-master-1 devstack@n-api.service[138680]: {"createImage": {"name": "tempest-ImagesNegativeTestJSON-image-67541837", "metadata": {"image_type": "test"}}} Jul 16 21:16:58.815721 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 21:16:58.815721 ncdv-master-1 devstack@n-api.service[138680]: exception: Instance e6844cd4-a382-45cc-ab21-bd7f2ede46e2 could not be found. Jul 16 21:16:58.815721 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': 'e6844cd4-a382-45cc-ab21-bd7f2ede46e2', 'body': {'createImage': {'name': 'tempest-ImagesNegativeTestJSON-image-67541837', 'metadata': {'image_type': 'test'}}}} Jul 16 21:16:58.815721 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:16:58.815721 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:16:58.815721 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:16:58.815721 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:16:58.815721 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:16:58.815721 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:16:58.815721 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:16:58.815721 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:16:58.815721 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:16:58.815721 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:16:58.815721 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance e6844cd4-a382-45cc-ab21-bd7f2ede46e2 could not be found. Jul 16 21:16:58.815721 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:16:58.815721 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:16:58.815721 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:16:58.815721 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:16:58.815721 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:16:58.815721 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:16:58.815721 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:16:58.815721 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:16:58.815721 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:16:58.815721 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:16:58.815721 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 16 21:16:58.815721 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 16 21:16:58.815721 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 16 21:16:58.815721 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 16 21:16:58.815721 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1316, in _action_create_image Jul 16 21:16:58.815721 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, id) Jul 16 21:16:58.815721 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:16:58.819204 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:16:58.819204 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:16:58.819204 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:16:58.819204 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance e6844cd4-a382-45cc-ab21-bd7f2ede46e2 could not be found. Jul 16 21:16:58.819204 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:16:58.819204 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:16:58.819204 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:16:58.819204 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:16:58.819204 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:16:58.819204 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:16:58.819204 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:16:58.819204 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:16:58.819204 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance e6844cd4-a382-45cc-ab21-bd7f2ede46e2 could not be found. Jul 16 21:16:58.819204 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:16:58.819204 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-6985de0a-9025-48a7-9fe2-803f277076e0 tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] Returning 404 to user: Instance e6844cd4-a382-45cc-ab21-bd7f2ede46e2 could not be found. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:16:58.819204 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-6985de0a-9025-48a7-9fe2-803f277076e0 tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] 10.222.0.16 "POST /compute/v2.1/servers/e6844cd4-a382-45cc-ab21-bd7f2ede46e2/action" status: 404 len: 111 microversion: 2.1 time: 0.087426 Jul 16 21:16:58.831971 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1078/2071] 10.222.0.16 () {60 vars in 1278 bytes} [Sat Jul 16 21:16:58 2022] POST /compute/v2.1/servers/e6844cd4-a382-45cc-ab21-bd7f2ede46e2/action => generated 111 bytes in 102 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:16:58.838647 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-66e41067-b405-46b5-8a74-0156466dab35 tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] Action: 'action', calling method: >, body: {"createImage": {"name": "tempest-ImagesNegativeTestJSON-image-466991438", "metadata": {"image_type": "test"}}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:16:58.847444 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-66e41067-b405-46b5-8a74-0156466dab35 tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] Failed to fetch instance by id tempest-invalid-1539026547 {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2926}} Jul 16 21:16:58.848367 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-66e41067-b405-46b5-8a74-0156466dab35 tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] HTTP exception thrown: Instance tempest-invalid-1539026547 could not be found. Jul 16 21:16:58.852995 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-66e41067-b405-46b5-8a74-0156466dab35 tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] Request method failure captured: Jul 16 21:16:58.852995 ncdv-master-1 devstack@n-api.service[138680]: request: POST /compute/v2.1/servers/tempest-invalid-1539026547/action HTTP/1.1 Jul 16 21:16:58.852995 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:16:58.852995 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:16:58.852995 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:16:58.852995 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 111 Jul 16 21:16:58.852995 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:16:58.852995 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:16:58.852995 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:16:58.852995 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:16:58.852995 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0yqvSs5UgxQxKA82EE830ekeZzD64HCYmKhqW4ubz2pm3RVOwENZ0Zu-pJqLcb3II1j4okuuS6d1EzSSqhj-LCfInXXWIIBtGJ3C4NuvjFkCNne7mTnaEoGL1Eq40mf85R2R38XmjzvDGni-yhyQ0K6VsJdJb-uuNQJDCPLFt47iex4 Jul 16 21:16:58.852995 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:16:58.852995 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:16:58.852995 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:16:58.852995 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:16:58.852995 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:16:58.852995 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:16:58.852995 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: 57c1e99ad9884d79812e95fbc81512da Jul 16 21:16:58.852995 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-ImagesNegativeTestJSON-1746139659 Jul 16 21:16:58.852995 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:16:58.852995 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:16:58.852995 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/57c1e99ad9884d79812e95fbc81512da"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/57c1e99ad9884d79812e95fbc81512da"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_57c1e99ad9884d79812e95fbc81512da"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/57c1e99ad9884d79812e95fbc81512da"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:16:58.852995 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-ImagesNegativeTestJSON-1746139659 Jul 16 21:16:58.852995 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: 57c1e99ad9884d79812e95fbc81512da Jul 16 21:16:58.852995 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-ImagesNegativeTestJSON-1746139659 Jul 16 21:16:58.852995 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-ImagesNegativeTestJSON-1746139659-project Jul 16 21:16:58.852995 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:16:58.852995 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:16:58.852995 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: b7a14721381b4f2cb4def5be76b7b645 Jul 16 21:16:58.852995 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-ImagesNegativeTestJSON-1746139659-project Jul 16 21:16:58.852995 ncdv-master-1 devstack@n-api.service[138680]: Jul 16 21:16:58.852995 ncdv-master-1 devstack@n-api.service[138680]: {"createImage": {"name": "tempest-ImagesNegativeTestJSON-image-466991438", "metadata": {"image_type": "test"}}} Jul 16 21:16:58.852995 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 21:16:58.852995 ncdv-master-1 devstack@n-api.service[138680]: exception: Instance tempest-invalid-1539026547 could not be found. Jul 16 21:16:58.852995 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': 'tempest-invalid-1539026547', 'body': {'createImage': {'name': 'tempest-ImagesNegativeTestJSON-image-466991438', 'metadata': {'image_type': 'test'}}}} Jul 16 21:16:58.852995 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:16:58.852995 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:16:58.852995 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:16:58.852995 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:16:58.852995 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2927, in get Jul 16 21:16:58.852995 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=instance_id) Jul 16 21:16:58.852995 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance tempest-invalid-1539026547 could not be found. Jul 16 21:16:58.852995 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:16:58.852995 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:16:58.852995 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:16:58.852995 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:16:58.852995 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:16:58.852995 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:16:58.852995 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:16:58.852995 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:16:58.852995 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:16:58.852995 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:16:58.852995 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 16 21:16:58.852995 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 16 21:16:58.852995 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 16 21:16:58.852995 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 16 21:16:58.852995 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1316, in _action_create_image Jul 16 21:16:58.852995 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, id) Jul 16 21:16:58.852995 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:16:58.852995 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:16:58.852995 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:16:58.852995 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:16:58.852995 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance tempest-invalid-1539026547 could not be found. Jul 16 21:16:58.856993 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:16:58.856993 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:16:58.856993 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:16:58.856993 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:16:58.856993 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:16:58.856993 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:16:58.856993 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:16:58.856993 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:16:58.856993 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance tempest-invalid-1539026547 could not be found. Jul 16 21:16:58.856993 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:16:58.869795 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-66e41067-b405-46b5-8a74-0156466dab35 tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] Returning 404 to user: Instance tempest-invalid-1539026547 could not be found. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:16:58.870824 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-66e41067-b405-46b5-8a74-0156466dab35 tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] 10.222.0.16 "POST /compute/v2.1/servers/tempest-invalid-1539026547/action" status: 404 len: 101 microversion: 2.1 time: 0.036101 Jul 16 21:16:58.880684 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1079/2072] 10.222.0.16 () {60 vars in 1248 bytes} [Sat Jul 16 21:16:58 2022] POST /compute/v2.1/servers/tempest-invalid-1539026547/action => generated 101 bytes in 48 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:16:58.927423 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-68d18190-56a3-4998-8ed7-9b81d526c35c tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] Action: 'action', calling method: >, body: {"createImage": {"name": "tempest-test-snap-1806144571"}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:16:58.933767 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-68d18190-56a3-4998-8ed7-9b81d526c35c tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] Failed to fetch instance by id aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2926}} Jul 16 21:16:58.938700 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-68d18190-56a3-4998-8ed7-9b81d526c35c tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] HTTP exception thrown: Instance aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa could not be found. Jul 16 21:16:58.941512 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-68d18190-56a3-4998-8ed7-9b81d526c35c tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] Request method failure captured: Jul 16 21:16:58.941512 ncdv-master-1 devstack@n-api.service[138680]: request: POST /compute/v2.1/servers/aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa/action HTTP/1.1 Jul 16 21:16:58.941512 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:16:58.941512 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:16:58.941512 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:16:58.941512 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 57 Jul 16 21:16:58.941512 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:16:58.941512 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:16:58.941512 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:16:58.941512 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:16:58.941512 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0yqvSs5UgxQxKA82EE830ekeZzD64HCYmKhqW4ubz2pm3RVOwENZ0Zu-pJqLcb3II1j4okuuS6d1EzSSqhj-LCfInXXWIIBtGJ3C4NuvjFkCNne7mTnaEoGL1Eq40mf85R2R38XmjzvDGni-yhyQ0K6VsJdJb-uuNQJDCPLFt47iex4 Jul 16 21:16:58.941512 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:16:58.941512 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:16:58.941512 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:16:58.941512 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:16:58.941512 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:16:58.941512 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:16:58.941512 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: 57c1e99ad9884d79812e95fbc81512da Jul 16 21:16:58.941512 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-ImagesNegativeTestJSON-1746139659 Jul 16 21:16:58.941512 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:16:58.941512 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:16:58.941512 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/57c1e99ad9884d79812e95fbc81512da"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/57c1e99ad9884d79812e95fbc81512da"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_57c1e99ad9884d79812e95fbc81512da"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/57c1e99ad9884d79812e95fbc81512da"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:16:58.941512 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-ImagesNegativeTestJSON-1746139659 Jul 16 21:16:58.941512 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: 57c1e99ad9884d79812e95fbc81512da Jul 16 21:16:58.941512 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-ImagesNegativeTestJSON-1746139659 Jul 16 21:16:58.941512 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-ImagesNegativeTestJSON-1746139659-project Jul 16 21:16:58.941512 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:16:58.941512 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:16:58.941512 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: b7a14721381b4f2cb4def5be76b7b645 Jul 16 21:16:58.941512 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-ImagesNegativeTestJSON-1746139659-project Jul 16 21:16:58.941512 ncdv-master-1 devstack@n-api.service[138680]: Jul 16 21:16:58.941512 ncdv-master-1 devstack@n-api.service[138680]: {"createImage": {"name": "tempest-test-snap-1806144571"}} Jul 16 21:16:58.941512 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 21:16:58.941512 ncdv-master-1 devstack@n-api.service[138680]: exception: Instance aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa could not be found. Jul 16 21:16:58.941512 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa', 'body': {'createImage': {'name': 'tempest-test-snap-1806144571'}}} Jul 16 21:16:58.941512 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:16:58.941512 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:16:58.941512 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:16:58.941512 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:16:58.941512 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2927, in get Jul 16 21:16:58.941512 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=instance_id) Jul 16 21:16:58.941512 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa could not be found. Jul 16 21:16:58.941512 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:16:58.941512 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:16:58.941512 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:16:58.941512 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:16:58.941512 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:16:58.941512 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:16:58.941512 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:16:58.941512 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:16:58.941512 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:16:58.941512 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:16:58.941512 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 16 21:16:58.941512 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 16 21:16:58.941512 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 16 21:16:58.941512 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 16 21:16:58.941512 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1316, in _action_create_image Jul 16 21:16:58.941512 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, id) Jul 16 21:16:58.941512 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:16:58.941512 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:16:58.941512 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:16:58.941512 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:16:58.941512 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa could not be found. Jul 16 21:16:58.941512 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:16:58.945014 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:16:58.945014 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:16:58.945014 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:16:58.945014 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:16:58.945014 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:16:58.945014 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:16:58.945014 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:16:58.945014 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa could not be found. Jul 16 21:16:58.945014 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:16:58.946998 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-68d18190-56a3-4998-8ed7-9b81d526c35c tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] Returning 404 to user: Instance aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa could not be found. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:16:58.948024 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-68d18190-56a3-4998-8ed7-9b81d526c35c tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] 10.222.0.16 "POST /compute/v2.1/servers/aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa/action" status: 404 len: 110 microversion: 2.1 time: 0.054037 Jul 16 21:16:58.955818 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1080/2073] 10.222.0.16 () {60 vars in 1274 bytes} [Sat Jul 16 21:16:58 2022] POST /compute/v2.1/servers/aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa/action => generated 110 bytes in 62 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Jul 16 21:16:58.980226 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-f7a251f2-3dc8-4baf-8fce-3cad459b92ce tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] Action: 'action', calling method: >, body: {"createImage": {"name": "tempest-test-snap-380558449"}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:16:58.981889 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-f7a251f2-3dc8-4baf-8fce-3cad459b92ce tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] Failed to fetch instance by id aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2926}} Jul 16 21:16:58.982564 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-f7a251f2-3dc8-4baf-8fce-3cad459b92ce tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] HTTP exception thrown: Instance aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa could not be found. Jul 16 21:16:58.987796 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-f7a251f2-3dc8-4baf-8fce-3cad459b92ce tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] Request method failure captured: Jul 16 21:16:58.987796 ncdv-master-1 devstack@n-api.service[138680]: request: POST /compute/v2.1/servers/aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa/action HTTP/1.1 Jul 16 21:16:58.987796 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:16:58.987796 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:16:58.987796 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:16:58.987796 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 56 Jul 16 21:16:58.987796 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:16:58.987796 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:16:58.987796 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:16:58.987796 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:16:58.987796 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0yqvSs5UgxQxKA82EE830ekeZzD64HCYmKhqW4ubz2pm3RVOwENZ0Zu-pJqLcb3II1j4okuuS6d1EzSSqhj-LCfInXXWIIBtGJ3C4NuvjFkCNne7mTnaEoGL1Eq40mf85R2R38XmjzvDGni-yhyQ0K6VsJdJb-uuNQJDCPLFt47iex4 Jul 16 21:16:58.987796 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:16:58.987796 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:16:58.987796 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:16:58.987796 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:16:58.987796 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:16:58.987796 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:16:58.987796 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: 57c1e99ad9884d79812e95fbc81512da Jul 16 21:16:58.987796 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-ImagesNegativeTestJSON-1746139659 Jul 16 21:16:58.987796 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:16:58.987796 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:16:58.987796 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/57c1e99ad9884d79812e95fbc81512da"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/57c1e99ad9884d79812e95fbc81512da"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_57c1e99ad9884d79812e95fbc81512da"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/57c1e99ad9884d79812e95fbc81512da"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:16:58.987796 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-ImagesNegativeTestJSON-1746139659 Jul 16 21:16:58.987796 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: 57c1e99ad9884d79812e95fbc81512da Jul 16 21:16:58.987796 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-ImagesNegativeTestJSON-1746139659 Jul 16 21:16:58.987796 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-ImagesNegativeTestJSON-1746139659-project Jul 16 21:16:58.987796 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:16:58.987796 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:16:58.987796 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: b7a14721381b4f2cb4def5be76b7b645 Jul 16 21:16:58.987796 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-ImagesNegativeTestJSON-1746139659-project Jul 16 21:16:58.987796 ncdv-master-1 devstack@n-api.service[138680]: Jul 16 21:16:58.987796 ncdv-master-1 devstack@n-api.service[138680]: {"createImage": {"name": "tempest-test-snap-380558449"}} Jul 16 21:16:58.987796 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 21:16:58.987796 ncdv-master-1 devstack@n-api.service[138680]: exception: Instance aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa could not be found. Jul 16 21:16:58.987796 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa', 'body': {'createImage': {'name': 'tempest-test-snap-380558449'}}} Jul 16 21:16:58.987796 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:16:58.987796 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:16:58.987796 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:16:58.987796 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:16:58.987796 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2927, in get Jul 16 21:16:58.987796 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=instance_id) Jul 16 21:16:58.987796 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa could not be found. Jul 16 21:16:58.987796 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:16:58.987796 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:16:58.987796 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:16:58.987796 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:16:58.987796 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:16:58.987796 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:16:58.987796 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:16:58.987796 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:16:58.987796 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:16:58.987796 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:16:58.987796 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 16 21:16:58.987796 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 16 21:16:58.987796 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 16 21:16:58.987796 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 16 21:16:58.987796 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1316, in _action_create_image Jul 16 21:16:58.987796 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, id) Jul 16 21:16:58.987796 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:16:58.987796 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:16:58.987796 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:16:58.987796 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:16:58.987796 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa could not be found. Jul 16 21:16:58.987796 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:16:58.990967 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:16:58.990967 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:16:58.990967 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:16:58.990967 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:16:58.990967 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:16:58.990967 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:16:58.990967 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:16:58.990967 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa could not be found. Jul 16 21:16:58.990967 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:16:58.991752 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-f7a251f2-3dc8-4baf-8fce-3cad459b92ce tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] Returning 404 to user: Instance aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa could not be found. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:16:58.998469 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-f7a251f2-3dc8-4baf-8fce-3cad459b92ce tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] 10.222.0.16 "POST /compute/v2.1/servers/aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa/action" status: 404 len: 112 microversion: 2.1 time: 0.039517 Jul 16 21:16:58.999830 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1081/2074] 10.222.0.16 () {60 vars in 1280 bytes} [Sat Jul 16 21:16:58 2022] POST /compute/v2.1/servers/aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa/action => generated 112 bytes in 43 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:16:59.010084 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-49f7f2e1-d0a2-41fe-9e9a-3d77b94f0572 tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:59.015710 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-49f7f2e1-d0a2-41fe-9e9a-3d77b94f0572 tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] [instance: e6844cd4-a382-45cc-ab21-bd7f2ede46e2] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:59.023179 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-49f7f2e1-d0a2-41fe-9e9a-3d77b94f0572 tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:59.023646 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-49f7f2e1-d0a2-41fe-9e9a-3d77b94f0572 tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:59.024432 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-49f7f2e1-d0a2-41fe-9e9a-3d77b94f0572 tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:59.024874 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-49f7f2e1-d0a2-41fe-9e9a-3d77b94f0572 tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:59.045829 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-49f7f2e1-d0a2-41fe-9e9a-3d77b94f0572 tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] HTTP exception thrown: Instance e6844cd4-a382-45cc-ab21-bd7f2ede46e2 could not be found. Jul 16 21:16:59.047557 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-49f7f2e1-d0a2-41fe-9e9a-3d77b94f0572 tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] Request method failure captured: Jul 16 21:16:59.047557 ncdv-master-1 devstack@n-api.service[138680]: request: DELETE /compute/v2.1/servers/e6844cd4-a382-45cc-ab21-bd7f2ede46e2 HTTP/1.1 Jul 16 21:16:59.047557 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:16:59.047557 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:16:59.047557 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:16:59.047557 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 0 Jul 16 21:16:59.047557 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:16:59.047557 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:16:59.047557 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:16:59.047557 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:16:59.047557 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0yqvSs5UgxQxKA82EE830ekeZzD64HCYmKhqW4ubz2pm3RVOwENZ0Zu-pJqLcb3II1j4okuuS6d1EzSSqhj-LCfInXXWIIBtGJ3C4NuvjFkCNne7mTnaEoGL1Eq40mf85R2R38XmjzvDGni-yhyQ0K6VsJdJb-uuNQJDCPLFt47iex4 Jul 16 21:16:59.047557 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:16:59.047557 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:16:59.047557 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:16:59.047557 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:16:59.047557 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:16:59.047557 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:16:59.047557 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: 57c1e99ad9884d79812e95fbc81512da Jul 16 21:16:59.047557 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-ImagesNegativeTestJSON-1746139659 Jul 16 21:16:59.047557 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:16:59.047557 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:16:59.047557 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/57c1e99ad9884d79812e95fbc81512da"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/57c1e99ad9884d79812e95fbc81512da"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_57c1e99ad9884d79812e95fbc81512da"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/57c1e99ad9884d79812e95fbc81512da"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:16:59.047557 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-ImagesNegativeTestJSON-1746139659 Jul 16 21:16:59.047557 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: 57c1e99ad9884d79812e95fbc81512da Jul 16 21:16:59.047557 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-ImagesNegativeTestJSON-1746139659 Jul 16 21:16:59.047557 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-ImagesNegativeTestJSON-1746139659-project Jul 16 21:16:59.047557 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:16:59.047557 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:16:59.047557 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: b7a14721381b4f2cb4def5be76b7b645 Jul 16 21:16:59.047557 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-ImagesNegativeTestJSON-1746139659-project Jul 16 21:16:59.047557 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 21:16:59.047557 ncdv-master-1 devstack@n-api.service[138680]: exception: Instance e6844cd4-a382-45cc-ab21-bd7f2ede46e2 could not be found. Jul 16 21:16:59.047557 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': 'e6844cd4-a382-45cc-ab21-bd7f2ede46e2'} Jul 16 21:16:59.047557 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:16:59.047557 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:16:59.047557 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:16:59.047557 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:16:59.047557 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:16:59.047557 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:16:59.047557 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:16:59.047557 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:16:59.047557 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:16:59.047557 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:16:59.047557 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance e6844cd4-a382-45cc-ab21-bd7f2ede46e2 could not be found. Jul 16 21:16:59.047557 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:16:59.047557 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:16:59.047557 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:16:59.047557 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:16:59.047557 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:16:59.047557 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:16:59.047557 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:16:59.047557 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:16:59.047557 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:16:59.047557 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:16:59.047557 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Jul 16 21:16:59.047557 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Jul 16 21:16:59.047557 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Jul 16 21:16:59.047557 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Jul 16 21:16:59.047557 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:16:59.047557 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:16:59.047557 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:16:59.047557 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:16:59.053640 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance e6844cd4-a382-45cc-ab21-bd7f2ede46e2 could not be found. Jul 16 21:16:59.053640 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:16:59.053640 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:16:59.053640 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:16:59.053640 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:16:59.053640 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:16:59.053640 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:16:59.053640 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:16:59.053640 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:16:59.053640 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance e6844cd4-a382-45cc-ab21-bd7f2ede46e2 could not be found. Jul 16 21:16:59.053640 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:16:59.053640 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-49f7f2e1-d0a2-41fe-9e9a-3d77b94f0572 tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] Returning 404 to user: Instance e6844cd4-a382-45cc-ab21-bd7f2ede46e2 could not be found. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:16:59.053640 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-49f7f2e1-d0a2-41fe-9e9a-3d77b94f0572 tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] 10.222.0.16 "DELETE /compute/v2.1/servers/e6844cd4-a382-45cc-ab21-bd7f2ede46e2" status: 404 len: 111 microversion: 2.1 time: 0.043131 Jul 16 21:16:59.053640 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1082/2075] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:16:59 2022] DELETE /compute/v2.1/servers/e6844cd4-a382-45cc-ab21-bd7f2ede46e2 => generated 111 bytes in 48 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:16:59.059676 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-363ec187-84c8-4357-ad42-5559a53e9b71 tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:59.060283 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-363ec187-84c8-4357-ad42-5559a53e9b71 tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] [instance: e6844cd4-a382-45cc-ab21-bd7f2ede46e2] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:59.068161 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-363ec187-84c8-4357-ad42-5559a53e9b71 tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:59.068624 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-363ec187-84c8-4357-ad42-5559a53e9b71 tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:59.069411 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-363ec187-84c8-4357-ad42-5559a53e9b71 tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:59.069858 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-363ec187-84c8-4357-ad42-5559a53e9b71 tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:59.097248 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-363ec187-84c8-4357-ad42-5559a53e9b71 tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] HTTP exception thrown: Instance e6844cd4-a382-45cc-ab21-bd7f2ede46e2 could not be found. Jul 16 21:16:59.101667 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-363ec187-84c8-4357-ad42-5559a53e9b71 tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] Request method failure captured: Jul 16 21:16:59.101667 ncdv-master-1 devstack@n-api.service[138680]: request: GET /compute/v2.1/servers/e6844cd4-a382-45cc-ab21-bd7f2ede46e2 HTTP/1.1 Jul 16 21:16:59.101667 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:16:59.101667 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:16:59.101667 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:16:59.101667 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 0 Jul 16 21:16:59.101667 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:16:59.101667 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:16:59.101667 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:16:59.101667 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:16:59.101667 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0yqvSs5UgxQxKA82EE830ekeZzD64HCYmKhqW4ubz2pm3RVOwENZ0Zu-pJqLcb3II1j4okuuS6d1EzSSqhj-LCfInXXWIIBtGJ3C4NuvjFkCNne7mTnaEoGL1Eq40mf85R2R38XmjzvDGni-yhyQ0K6VsJdJb-uuNQJDCPLFt47iex4 Jul 16 21:16:59.101667 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:16:59.101667 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:16:59.101667 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:16:59.101667 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:16:59.101667 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:16:59.101667 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:16:59.101667 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: 57c1e99ad9884d79812e95fbc81512da Jul 16 21:16:59.101667 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-ImagesNegativeTestJSON-1746139659 Jul 16 21:16:59.101667 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:16:59.101667 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:16:59.101667 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/57c1e99ad9884d79812e95fbc81512da"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/57c1e99ad9884d79812e95fbc81512da"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_57c1e99ad9884d79812e95fbc81512da"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/57c1e99ad9884d79812e95fbc81512da"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:16:59.101667 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-ImagesNegativeTestJSON-1746139659 Jul 16 21:16:59.101667 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: 57c1e99ad9884d79812e95fbc81512da Jul 16 21:16:59.101667 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-ImagesNegativeTestJSON-1746139659 Jul 16 21:16:59.101667 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-ImagesNegativeTestJSON-1746139659-project Jul 16 21:16:59.101667 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:16:59.101667 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:16:59.101667 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: b7a14721381b4f2cb4def5be76b7b645 Jul 16 21:16:59.101667 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-ImagesNegativeTestJSON-1746139659-project Jul 16 21:16:59.101667 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 21:16:59.101667 ncdv-master-1 devstack@n-api.service[138680]: exception: Instance e6844cd4-a382-45cc-ab21-bd7f2ede46e2 could not be found. Jul 16 21:16:59.101667 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': 'e6844cd4-a382-45cc-ab21-bd7f2ede46e2'} Jul 16 21:16:59.101667 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:16:59.101667 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:16:59.101667 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:16:59.101667 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:16:59.101667 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:16:59.101667 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:16:59.101667 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:16:59.101667 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:16:59.101667 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:16:59.101667 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:16:59.101667 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance e6844cd4-a382-45cc-ab21-bd7f2ede46e2 could not be found. Jul 16 21:16:59.101667 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:16:59.101667 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:16:59.101667 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:16:59.101667 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:16:59.101667 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:16:59.101667 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:16:59.101667 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:16:59.101667 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:16:59.101667 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:16:59.101667 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:16:59.101667 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:16:59.101667 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:16:59.101667 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:16:59.101667 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:16:59.101667 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:16:59.101667 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:16:59.101667 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance e6844cd4-a382-45cc-ab21-bd7f2ede46e2 could not be found. Jul 16 21:16:59.101667 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:16:59.101667 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:16:59.101667 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:16:59.105892 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:16:59.105892 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:16:59.105892 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:16:59.105892 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:16:59.105892 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:16:59.105892 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance e6844cd4-a382-45cc-ab21-bd7f2ede46e2 could not be found. Jul 16 21:16:59.105892 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:16:59.105892 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-363ec187-84c8-4357-ad42-5559a53e9b71 tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] Returning 404 to user: Instance e6844cd4-a382-45cc-ab21-bd7f2ede46e2 could not be found. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:16:59.105892 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-363ec187-84c8-4357-ad42-5559a53e9b71 tempest-ImagesNegativeTestJSON-1746139659 tempest-ImagesNegativeTestJSON-1746139659-project] 10.222.0.16 "GET /compute/v2.1/servers/e6844cd4-a382-45cc-ab21-bd7f2ede46e2" status: 404 len: 111 microversion: 2.1 time: 0.048724 Jul 16 21:16:59.109235 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1083/2076] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:16:59 2022] GET /compute/v2.1/servers/e6844cd4-a382-45cc-ab21-bd7f2ede46e2 => generated 111 bytes in 55 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:16:59.153146 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-ef812f75-97d5-47f2-85aa-cd06329b5ec1 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd4144b5388b46a989a41a2f556bb508', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a90d4bcd25ff4bf68bdc194180fbbe09', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:16:59.166068 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-ef812f75-97d5-47f2-85aa-cd06329b5ec1 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] 10.222.0.16 "GET /compute/v2.1/servers/11194822-5d42-45f4-99d0-88c70c808ddd" status: 200 len: 1519 microversion: 2.1 time: 0.584893 Jul 16 21:16:59.166830 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 994/2077] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:16:58 2022] GET /compute/v2.1/servers/11194822-5d42-45f4-99d0-88c70c808ddd => generated 1519 bytes in 586 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:16:59.497865 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-2e2716d1-1c6c-41a4-91fe-c967548e1067 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:59.498567 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-2e2716d1-1c6c-41a4-91fe-c967548e1067 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] [instance: f03a2a83-e007-444a-b46b-70eb46bb242b] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:59.507617 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2e2716d1-1c6c-41a4-91fe-c967548e1067 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:59.513832 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2e2716d1-1c6c-41a4-91fe-c967548e1067 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:59.518403 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2e2716d1-1c6c-41a4-91fe-c967548e1067 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:59.520631 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2e2716d1-1c6c-41a4-91fe-c967548e1067 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:59.590009 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-fabfcf3f-0385-411f-9fd4-df6c729818ae tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:16:59.590786 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-fabfcf3f-0385-411f-9fd4-df6c729818ae tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] [instance: 6cb4e66a-eba2-42b7-91e8-09ee7b2a5a4f] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:16:59.598574 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fabfcf3f-0385-411f-9fd4-df6c729818ae tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:59.599333 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fabfcf3f-0385-411f-9fd4-df6c729818ae tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:16:59.600216 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fabfcf3f-0385-411f-9fd4-df6c729818ae tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:16:59.600875 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fabfcf3f-0385-411f-9fd4-df6c729818ae tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:00.077470 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell0:SELECT=2 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:17:00.149796 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-2e2716d1-1c6c-41a4-91fe-c967548e1067 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'daf3c9e6634e4ae1bfb492eae1f44c93', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '24c64db8082648c8837c16352252f96d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:17:00.163293 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-2e2716d1-1c6c-41a4-91fe-c967548e1067 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] 10.222.0.16 "GET /compute/v2.1/servers/f03a2a83-e007-444a-b46b-70eb46bb242b" status: 200 len: 1496 microversion: 2.1 time: 0.669323 Jul 16 21:17:00.175093 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-fabfcf3f-0385-411f-9fd4-df6c729818ae tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '71e9304fc50447d6aac99703b5b6e4e1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e84496d78d4481e9d7648128868bca2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:17:00.175320 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1084/2078] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:16:59 2022] GET /compute/v2.1/servers/f03a2a83-e007-444a-b46b-70eb46bb242b => generated 1496 bytes in 682 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:17:00.182742 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-6a641ee8-a471-47f7-aec9-f46f0c08fdbc tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:17:00.184045 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-6a641ee8-a471-47f7-aec9-f46f0c08fdbc tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] [instance: 9657dddc-2416-4c95-aab3-d6b6f9673369] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:17:00.185076 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-fabfcf3f-0385-411f-9fd4-df6c729818ae tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] 10.222.0.16 "GET /compute/v2.1/servers/6cb4e66a-eba2-42b7-91e8-09ee7b2a5a4f" status: 200 len: 1317 microversion: 2.1 time: 0.598872 Jul 16 21:17:00.185976 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 995/2079] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:16:59 2022] GET /compute/v2.1/servers/6cb4e66a-eba2-42b7-91e8-09ee7b2a5a4f => generated 1317 bytes in 600 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:17:00.191927 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-9a1c87c7-1565-4b41-bdb7-65f4e599a29c req-f9cb0fc5-5991-4b28-a6ae-1ee0a3fc9cc5 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "6cb4e66a-eba2-42b7-91e8-09ee7b2a5a4f", "tag": "381fe698-72ef-4f6f-8b88-5df5cb8d8d96"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:17:00.202384 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-9a1c87c7-1565-4b41-bdb7-65f4e599a29c req-f9cb0fc5-5991-4b28-a6ae-1ee0a3fc9cc5 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:00.203252 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-9a1c87c7-1565-4b41-bdb7-65f4e599a29c req-f9cb0fc5-5991-4b28-a6ae-1ee0a3fc9cc5 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:00.206841 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6a641ee8-a471-47f7-aec9-f46f0c08fdbc tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:00.207427 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6a641ee8-a471-47f7-aec9-f46f0c08fdbc tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:00.209258 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6a641ee8-a471-47f7-aec9-f46f0c08fdbc tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:00.209746 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6a641ee8-a471-47f7-aec9-f46f0c08fdbc tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:00.230044 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-9a1c87c7-1565-4b41-bdb7-65f4e599a29c req-f9cb0fc5-5991-4b28-a6ae-1ee0a3fc9cc5 service nova] Creating event network-changed:381fe698-72ef-4f6f-8b88-5df5cb8d8d96 for instance 6cb4e66a-eba2-42b7-91e8-09ee7b2a5a4f on nch2-master-1 Jul 16 21:17:00.248033 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-9a1c87c7-1565-4b41-bdb7-65f4e599a29c req-f9cb0fc5-5991-4b28-a6ae-1ee0a3fc9cc5 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.059754 Jul 16 21:17:00.249833 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 996/2080] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:17:00 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 63 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:17:00.262276 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-8f99f2df-d286-4a65-bfc0-1054c829dc11 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:17:00.263241 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-8f99f2df-d286-4a65-bfc0-1054c829dc11 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] [instance: 11194822-5d42-45f4-99d0-88c70c808ddd] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:17:00.275120 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8f99f2df-d286-4a65-bfc0-1054c829dc11 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:00.275504 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8f99f2df-d286-4a65-bfc0-1054c829dc11 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:00.276290 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8f99f2df-d286-4a65-bfc0-1054c829dc11 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:00.276742 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8f99f2df-d286-4a65-bfc0-1054c829dc11 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:00.422350 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-6a641ee8-a471-47f7-aec9-f46f0c08fdbc tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '015ac1bbb2df487e99e5bddc60ee6923', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9f5f03e09936489fa1505188cefe3445', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:17:00.434897 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-6a641ee8-a471-47f7-aec9-f46f0c08fdbc tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '015ac1bbb2df487e99e5bddc60ee6923', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9f5f03e09936489fa1505188cefe3445', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:17:00.438466 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-6a641ee8-a471-47f7-aec9-f46f0c08fdbc tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '015ac1bbb2df487e99e5bddc60ee6923', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9f5f03e09936489fa1505188cefe3445', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:17:00.439936 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-6a641ee8-a471-47f7-aec9-f46f0c08fdbc tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] 10.222.0.16 "GET /compute/v2.1/servers/9657dddc-2416-4c95-aab3-d6b6f9673369" status: 200 len: 1365 microversion: 2.47 time: 0.263467 Jul 16 21:17:00.440648 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1085/2081] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:17:00 2022] GET /compute/v2.1/servers/9657dddc-2416-4c95-aab3-d6b6f9673369 => generated 1365 bytes in 265 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:17:00.681086 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-8f99f2df-d286-4a65-bfc0-1054c829dc11 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd4144b5388b46a989a41a2f556bb508', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a90d4bcd25ff4bf68bdc194180fbbe09', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:17:00.693461 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-8f99f2df-d286-4a65-bfc0-1054c829dc11 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] 10.222.0.16 "GET /compute/v2.1/servers/11194822-5d42-45f4-99d0-88c70c808ddd" status: 200 len: 1504 microversion: 2.1 time: 0.442299 Jul 16 21:17:00.694283 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 997/2082] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:17:00 2022] GET /compute/v2.1/servers/11194822-5d42-45f4-99d0-88c70c808ddd => generated 1504 bytes in 444 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:17:00.712004 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-2b61c467-7733-4bd4-81ad-e87730470a08 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:17:00.712785 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-2b61c467-7733-4bd4-81ad-e87730470a08 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] [instance: 11194822-5d42-45f4-99d0-88c70c808ddd] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:17:00.724903 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2b61c467-7733-4bd4-81ad-e87730470a08 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:00.726679 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2b61c467-7733-4bd4-81ad-e87730470a08 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:00.728237 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2b61c467-7733-4bd4-81ad-e87730470a08 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:00.729514 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2b61c467-7733-4bd4-81ad-e87730470a08 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:01.047282 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-2b61c467-7733-4bd4-81ad-e87730470a08 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd4144b5388b46a989a41a2f556bb508', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a90d4bcd25ff4bf68bdc194180fbbe09', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:17:01.056678 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-2b61c467-7733-4bd4-81ad-e87730470a08 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] 10.222.0.16 "GET /compute/v2.1/servers/11194822-5d42-45f4-99d0-88c70c808ddd" status: 200 len: 1504 microversion: 2.1 time: 0.349046 Jul 16 21:17:01.067988 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1086/2083] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:17:00 2022] GET /compute/v2.1/servers/11194822-5d42-45f4-99d0-88c70c808ddd => generated 1504 bytes in 361 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:17:01.074892 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-85749805-bc26-42c9-bef7-c0ea4cc94c57 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:17:01.075432 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-85749805-bc26-42c9-bef7-c0ea4cc94c57 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] [instance: 11194822-5d42-45f4-99d0-88c70c808ddd] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:17:01.078950 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell0:SELECT=8 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:17:01.085985 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-85749805-bc26-42c9-bef7-c0ea4cc94c57 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:01.086451 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-85749805-bc26-42c9-bef7-c0ea4cc94c57 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:01.087290 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-85749805-bc26-42c9-bef7-c0ea4cc94c57 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:01.090020 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-85749805-bc26-42c9-bef7-c0ea4cc94c57 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:01.182826 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-88f54700-4545-4caa-bff2-ba4d1a5b7702 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:17:01.184054 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-88f54700-4545-4caa-bff2-ba4d1a5b7702 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] [instance: f03a2a83-e007-444a-b46b-70eb46bb242b] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:17:01.190185 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-85749805-bc26-42c9-bef7-c0ea4cc94c57 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] [instance: 11194822-5d42-45f4-99d0-88c70c808ddd] Going to try to terminate instance {{(pid=138681) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 16 21:17:01.195588 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-88f54700-4545-4caa-bff2-ba4d1a5b7702 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:01.199935 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-88f54700-4545-4caa-bff2-ba4d1a5b7702 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:01.206841 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-88f54700-4545-4caa-bff2-ba4d1a5b7702 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:01.221095 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-88f54700-4545-4caa-bff2-ba4d1a5b7702 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:01.445546 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-88f54700-4545-4caa-bff2-ba4d1a5b7702 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'daf3c9e6634e4ae1bfb492eae1f44c93', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '24c64db8082648c8837c16352252f96d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:17:01.457297 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-88f54700-4545-4caa-bff2-ba4d1a5b7702 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] 10.222.0.16 "GET /compute/v2.1/servers/f03a2a83-e007-444a-b46b-70eb46bb242b" status: 200 len: 1496 microversion: 2.1 time: 0.279081 Jul 16 21:17:01.458257 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1087/2084] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:17:01 2022] GET /compute/v2.1/servers/f03a2a83-e007-444a-b46b-70eb46bb242b => generated 1496 bytes in 281 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:17:01.463545 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-d8ab14cd-3771-4f62-8f6a-542600cffb57 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:17:01.464213 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-d8ab14cd-3771-4f62-8f6a-542600cffb57 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] [instance: 6cb4e66a-eba2-42b7-91e8-09ee7b2a5a4f] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:17:01.471277 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d8ab14cd-3771-4f62-8f6a-542600cffb57 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:01.471748 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d8ab14cd-3771-4f62-8f6a-542600cffb57 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:01.472593 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d8ab14cd-3771-4f62-8f6a-542600cffb57 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:01.473043 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d8ab14cd-3771-4f62-8f6a-542600cffb57 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:01.661079 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-d8ab14cd-3771-4f62-8f6a-542600cffb57 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '71e9304fc50447d6aac99703b5b6e4e1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e84496d78d4481e9d7648128868bca2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:17:01.668953 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-d8ab14cd-3771-4f62-8f6a-542600cffb57 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] 10.222.0.16 "GET /compute/v2.1/servers/6cb4e66a-eba2-42b7-91e8-09ee7b2a5a4f" status: 200 len: 1317 microversion: 2.1 time: 0.209541 Jul 16 21:17:01.669542 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1088/2085] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:17:01 2022] GET /compute/v2.1/servers/6cb4e66a-eba2-42b7-91e8-09ee7b2a5a4f => generated 1317 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:17:01.680895 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-32ef0c9c-65e3-4953-ac4d-acb345c307bb tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:17:01.681993 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-32ef0c9c-65e3-4953-ac4d-acb345c307bb tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] [instance: 9657dddc-2416-4c95-aab3-d6b6f9673369] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:17:01.690321 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-85749805-bc26-42c9-bef7-c0ea4cc94c57 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] 10.222.0.16 "DELETE /compute/v2.1/servers/11194822-5d42-45f4-99d0-88c70c808ddd" status: 204 len: 0 microversion: 2.1 time: 0.619457 Jul 16 21:17:01.692521 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-32ef0c9c-65e3-4953-ac4d-acb345c307bb tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:01.692521 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-32ef0c9c-65e3-4953-ac4d-acb345c307bb tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:01.692721 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-32ef0c9c-65e3-4953-ac4d-acb345c307bb tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:01.693210 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-32ef0c9c-65e3-4953-ac4d-acb345c307bb tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:01.694922 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 998/2086] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:17:01 2022] DELETE /compute/v2.1/servers/11194822-5d42-45f4-99d0-88c70c808ddd => generated 0 bytes in 625 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 16 21:17:01.712338 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=100 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:17:01.797097 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-32ef0c9c-65e3-4953-ac4d-acb345c307bb tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '015ac1bbb2df487e99e5bddc60ee6923', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9f5f03e09936489fa1505188cefe3445', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:17:01.815727 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-32ef0c9c-65e3-4953-ac4d-acb345c307bb tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '015ac1bbb2df487e99e5bddc60ee6923', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9f5f03e09936489fa1505188cefe3445', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:17:01.818009 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-32ef0c9c-65e3-4953-ac4d-acb345c307bb tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '015ac1bbb2df487e99e5bddc60ee6923', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9f5f03e09936489fa1505188cefe3445', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:17:01.823399 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-32ef0c9c-65e3-4953-ac4d-acb345c307bb tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] 10.222.0.16 "GET /compute/v2.1/servers/9657dddc-2416-4c95-aab3-d6b6f9673369" status: 200 len: 1365 microversion: 2.47 time: 0.151888 Jul 16 21:17:01.824240 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1089/2087] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:17:01 2022] GET /compute/v2.1/servers/9657dddc-2416-4c95-aab3-d6b6f9673369 => generated 1365 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:17:01.883889 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-828ec50a-aa31-4cea-b022-6cea3a99a9bd tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerDiskConfigTestJSON-server-1317876656", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452", "networks": [{"uuid": "b137fafe-f0b3-4d44-9dfc-add7ee4d76f5"}]}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:17:01.996502 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.network.neutron [None req-828ec50a-aa31-4cea-b022-6cea3a99a9bd tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] validate_networks() for [('b137fafe-f0b3-4d44-9dfc-add7ee4d76f5', None, None, None, None, None)] {{(pid=138681) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 16 21:17:02.475172 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-268e7c40-f79b-4bc3-8ead-512a8f521792 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:17:02.475800 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-268e7c40-f79b-4bc3-8ead-512a8f521792 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] [instance: f03a2a83-e007-444a-b46b-70eb46bb242b] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:17:02.486747 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-268e7c40-f79b-4bc3-8ead-512a8f521792 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:02.487217 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-268e7c40-f79b-4bc3-8ead-512a8f521792 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:02.488041 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-268e7c40-f79b-4bc3-8ead-512a8f521792 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:02.488447 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-268e7c40-f79b-4bc3-8ead-512a8f521792 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:03.099598 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-268e7c40-f79b-4bc3-8ead-512a8f521792 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'daf3c9e6634e4ae1bfb492eae1f44c93', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '24c64db8082648c8837c16352252f96d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:17:03.115635 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-268e7c40-f79b-4bc3-8ead-512a8f521792 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] 10.222.0.16 "GET /compute/v2.1/servers/f03a2a83-e007-444a-b46b-70eb46bb242b" status: 200 len: 1496 microversion: 2.1 time: 0.644770 Jul 16 21:17:03.116541 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1090/2088] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:17:02 2022] GET /compute/v2.1/servers/f03a2a83-e007-444a-b46b-70eb46bb242b => generated 1496 bytes in 646 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:17:03.121693 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-de01cdd6-ff80-44e5-8620-1a2451a3d0b2 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:17:03.122525 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-de01cdd6-ff80-44e5-8620-1a2451a3d0b2 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] [instance: 6cb4e66a-eba2-42b7-91e8-09ee7b2a5a4f] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:17:03.135517 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-de01cdd6-ff80-44e5-8620-1a2451a3d0b2 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:03.137633 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-de01cdd6-ff80-44e5-8620-1a2451a3d0b2 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:03.147522 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-de01cdd6-ff80-44e5-8620-1a2451a3d0b2 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:03.148467 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-de01cdd6-ff80-44e5-8620-1a2451a3d0b2 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.011s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:03.305774 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-828ec50a-aa31-4cea-b022-6cea3a99a9bd tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Flavor limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 16 21:17:03.306341 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-828ec50a-aa31-4cea-b022-6cea3a99a9bd tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Image limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 16 21:17:03.307983 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-828ec50a-aa31-4cea-b022-6cea3a99a9bd tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Flavor pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 16 21:17:03.308410 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-828ec50a-aa31-4cea-b022-6cea3a99a9bd tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Image pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 16 21:17:03.309925 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-828ec50a-aa31-4cea-b022-6cea3a99a9bd tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 16 21:17:03.346246 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-828ec50a-aa31-4cea-b022-6cea3a99a9bd tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:03.346863 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-828ec50a-aa31-4cea-b022-6cea3a99a9bd tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:03.347901 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-828ec50a-aa31-4cea-b022-6cea3a99a9bd tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:03.348469 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-828ec50a-aa31-4cea-b022-6cea3a99a9bd tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:03.370206 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-828ec50a-aa31-4cea-b022-6cea3a99a9bd tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Getting quotas for project a90d4bcd25ff4bf68bdc194180fbbe09. Resources: {'ram', 'instances', 'cores'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:17:03.376423 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-828ec50a-aa31-4cea-b022-6cea3a99a9bd tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Getting quotas for user dd4144b5388b46a989a41a2f556bb508 and project a90d4bcd25ff4bf68bdc194180fbbe09. Resources: {'ram', 'instances', 'cores'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:17:03.391332 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-828ec50a-aa31-4cea-b022-6cea3a99a9bd tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Going to run 1 instances... {{(pid=138681) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 16 21:17:03.397240 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-828ec50a-aa31-4cea-b022-6cea3a99a9bd tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] [instance: 7e4b142d-f489-4220-adc3-7b348dc14ce9] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138681) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 16 21:17:03.545721 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-de01cdd6-ff80-44e5-8620-1a2451a3d0b2 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '71e9304fc50447d6aac99703b5b6e4e1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e84496d78d4481e9d7648128868bca2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:17:03.555125 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-de01cdd6-ff80-44e5-8620-1a2451a3d0b2 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] 10.222.0.16 "GET /compute/v2.1/servers/6cb4e66a-eba2-42b7-91e8-09ee7b2a5a4f" status: 200 len: 1317 microversion: 2.1 time: 0.437283 Jul 16 21:17:03.555653 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1091/2089] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:17:03 2022] GET /compute/v2.1/servers/6cb4e66a-eba2-42b7-91e8-09ee7b2a5a4f => generated 1317 bytes in 438 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:17:03.560544 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-efff3ec5-a478-43a8-836e-543169fe3809 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:17:03.562367 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-efff3ec5-a478-43a8-836e-543169fe3809 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] [instance: 9657dddc-2416-4c95-aab3-d6b6f9673369] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:17:03.569814 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-efff3ec5-a478-43a8-836e-543169fe3809 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:03.570655 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-efff3ec5-a478-43a8-836e-543169fe3809 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:03.571638 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-efff3ec5-a478-43a8-836e-543169fe3809 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:03.572277 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-efff3ec5-a478-43a8-836e-543169fe3809 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:03.604425 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-828ec50a-aa31-4cea-b022-6cea3a99a9bd tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 1.726591 Jul 16 21:17:03.607267 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 999/2090] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 21:17:01 2022] POST /compute/v2.1/servers => generated 380 bytes in 1730 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 16 21:17:03.702045 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-efff3ec5-a478-43a8-836e-543169fe3809 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '015ac1bbb2df487e99e5bddc60ee6923', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9f5f03e09936489fa1505188cefe3445', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:17:03.715387 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-efff3ec5-a478-43a8-836e-543169fe3809 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '015ac1bbb2df487e99e5bddc60ee6923', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9f5f03e09936489fa1505188cefe3445', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:17:03.716979 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-efff3ec5-a478-43a8-836e-543169fe3809 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '015ac1bbb2df487e99e5bddc60ee6923', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9f5f03e09936489fa1505188cefe3445', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:17:03.718728 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-efff3ec5-a478-43a8-836e-543169fe3809 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] 10.222.0.16 "GET /compute/v2.1/servers/9657dddc-2416-4c95-aab3-d6b6f9673369" status: 200 len: 1365 microversion: 2.47 time: 0.161712 Jul 16 21:17:03.719583 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1092/2091] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:17:03 2022] GET /compute/v2.1/servers/9657dddc-2416-4c95-aab3-d6b6f9673369 => generated 1365 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:17:03.736162 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-d52c0989-71c9-4dd6-8ac8-8bab8d8d2e84 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:17:03.736793 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-d52c0989-71c9-4dd6-8ac8-8bab8d8d2e84 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] [instance: 7e4b142d-f489-4220-adc3-7b348dc14ce9] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:17:03.830623 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-42f32a2c-d8b4-4509-b1e6-61773e30958a tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServersAdminTestJSON-server-270461627", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452"}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:17:03.882436 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-d52c0989-71c9-4dd6-8ac8-8bab8d8d2e84 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd4144b5388b46a989a41a2f556bb508', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a90d4bcd25ff4bf68bdc194180fbbe09', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:17:03.891694 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-d52c0989-71c9-4dd6-8ac8-8bab8d8d2e84 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] 10.222.0.16 "GET /compute/v2.1/servers/7e4b142d-f489-4220-adc3-7b348dc14ce9" status: 200 len: 1218 microversion: 2.1 time: 0.170818 Jul 16 21:17:03.901152 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1093/2092] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:17:03 2022] GET /compute/v2.1/servers/7e4b142d-f489-4220-adc3-7b348dc14ce9 => generated 1218 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:17:03.916002 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.network.neutron [None req-42f32a2c-d8b4-4509-b1e6-61773e30958a tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] validate_networks() for None {{(pid=138681) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 16 21:17:04.054214 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-975c7a4c-2ad2-4813-95a9-6e484ba8e834 req-70ee973f-b9a8-42a7-974b-26b73281448f service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "11194822-5d42-45f4-99d0-88c70c808ddd", "name": "network-vif-deleted", "tag": "1e6a5dec-046c-4c31-bb3f-1b8d00690085"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:17:04.075484 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-975c7a4c-2ad2-4813-95a9-6e484ba8e834 req-70ee973f-b9a8-42a7-974b-26b73281448f service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:04.075963 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-975c7a4c-2ad2-4813-95a9-6e484ba8e834 req-70ee973f-b9a8-42a7-974b-26b73281448f service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:04.095779 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-975c7a4c-2ad2-4813-95a9-6e484ba8e834 req-70ee973f-b9a8-42a7-974b-26b73281448f service nova] Creating event network-vif-deleted:1e6a5dec-046c-4c31-bb3f-1b8d00690085 for instance 11194822-5d42-45f4-99d0-88c70c808ddd on nch1-master-1 Jul 16 21:17:04.114648 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-975c7a4c-2ad2-4813-95a9-6e484ba8e834 req-70ee973f-b9a8-42a7-974b-26b73281448f service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.064173 Jul 16 21:17:04.117316 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1094/2093] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:17:04 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 68 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 16 21:17:04.141363 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-3e9d02ec-8f68-47a4-9e5f-075ca3b9301c tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:17:04.142434 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-3e9d02ec-8f68-47a4-9e5f-075ca3b9301c tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] [instance: f03a2a83-e007-444a-b46b-70eb46bb242b] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:17:04.150728 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3e9d02ec-8f68-47a4-9e5f-075ca3b9301c tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:04.151442 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3e9d02ec-8f68-47a4-9e5f-075ca3b9301c tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:04.152408 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3e9d02ec-8f68-47a4-9e5f-075ca3b9301c tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:04.152999 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3e9d02ec-8f68-47a4-9e5f-075ca3b9301c tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:05.028619 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-3e9d02ec-8f68-47a4-9e5f-075ca3b9301c tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'daf3c9e6634e4ae1bfb492eae1f44c93', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '24c64db8082648c8837c16352252f96d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:17:05.064232 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-3e9d02ec-8f68-47a4-9e5f-075ca3b9301c tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] 10.222.0.16 "GET /compute/v2.1/servers/f03a2a83-e007-444a-b46b-70eb46bb242b" status: 200 len: 1496 microversion: 2.1 time: 0.929622 Jul 16 21:17:05.075266 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1095/2094] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:17:04 2022] GET /compute/v2.1/servers/f03a2a83-e007-444a-b46b-70eb46bb242b => generated 1496 bytes in 933 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:17:05.075266 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-e616f036-5f6f-4dcf-961d-90ba0964a5f9 req-638ad075-f253-410e-9f79-a703acd3d512 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "b9a4910a-f117-4834-b699-498c6431b9bb", "name": "network-vif-plugged", "status": "completed", "tag": "9ab69acd-f0a3-4355-b203-2c48e39cc3a1"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:17:05.082108 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-42f32a2c-d8b4-4509-b1e6-61773e30958a tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Flavor limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 16 21:17:05.083262 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-42f32a2c-d8b4-4509-b1e6-61773e30958a tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Image limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 16 21:17:05.083993 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-42f32a2c-d8b4-4509-b1e6-61773e30958a tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Flavor pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 16 21:17:05.084552 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-42f32a2c-d8b4-4509-b1e6-61773e30958a tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Image pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 16 21:17:05.085118 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-42f32a2c-d8b4-4509-b1e6-61773e30958a tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 16 21:17:05.100584 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-42f32a2c-d8b4-4509-b1e6-61773e30958a tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:05.105998 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-42f32a2c-d8b4-4509-b1e6-61773e30958a tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:05.105998 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-42f32a2c-d8b4-4509-b1e6-61773e30958a tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:05.105998 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-42f32a2c-d8b4-4509-b1e6-61773e30958a tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:05.106285 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-e616f036-5f6f-4dcf-961d-90ba0964a5f9 req-638ad075-f253-410e-9f79-a703acd3d512 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:05.106285 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-e616f036-5f6f-4dcf-961d-90ba0964a5f9 req-638ad075-f253-410e-9f79-a703acd3d512 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:05.128156 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-e616f036-5f6f-4dcf-961d-90ba0964a5f9 req-638ad075-f253-410e-9f79-a703acd3d512 service nova] Creating event network-vif-plugged:9ab69acd-f0a3-4355-b203-2c48e39cc3a1 for instance b9a4910a-f117-4834-b699-498c6431b9bb on nch2-master-1 Jul 16 21:17:05.141363 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-42f32a2c-d8b4-4509-b1e6-61773e30958a tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Getting quotas for project 65d5bcfa02af4e8284ba1342197e2860. Resources: {'ram', 'instances', 'cores'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:17:05.147326 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-e616f036-5f6f-4dcf-961d-90ba0964a5f9 req-638ad075-f253-410e-9f79-a703acd3d512 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.075843 Jul 16 21:17:05.149345 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1096/2095] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:17:05 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:17:05.152358 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-42f32a2c-d8b4-4509-b1e6-61773e30958a tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Getting quotas for user 2f5bc1f2209f4eb186d80e385b67c774 and project 65d5bcfa02af4e8284ba1342197e2860. Resources: {'ram', 'instances', 'cores'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:17:05.166907 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-39d3934c-d135-465a-a435-ba026019fd23 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:17:05.167734 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-39d3934c-d135-465a-a435-ba026019fd23 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] [instance: 6cb4e66a-eba2-42b7-91e8-09ee7b2a5a4f] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:17:05.174804 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-42f32a2c-d8b4-4509-b1e6-61773e30958a tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Going to run 1 instances... {{(pid=138681) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 16 21:17:05.174804 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-42f32a2c-d8b4-4509-b1e6-61773e30958a tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] [instance: e7598b4a-f70c-4186-88d1-6fd0057f8656] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138681) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 16 21:17:05.190072 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-39d3934c-d135-465a-a435-ba026019fd23 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:05.190569 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-39d3934c-d135-465a-a435-ba026019fd23 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:05.193066 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-39d3934c-d135-465a-a435-ba026019fd23 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:05.194663 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-39d3934c-d135-465a-a435-ba026019fd23 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:05.423668 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-42f32a2c-d8b4-4509-b1e6-61773e30958a tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 1.815242 Jul 16 21:17:05.425419 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1000/2096] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 21:17:03 2022] POST /compute/v2.1/servers => generated 380 bytes in 1818 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 16 21:17:05.430640 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-5a6b093e-2d0c-4f7d-81c5-8ac5e7202c98 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:17:05.431689 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-5a6b093e-2d0c-4f7d-81c5-8ac5e7202c98 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] [instance: 9657dddc-2416-4c95-aab3-d6b6f9673369] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:17:05.438567 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-39d3934c-d135-465a-a435-ba026019fd23 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '71e9304fc50447d6aac99703b5b6e4e1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e84496d78d4481e9d7648128868bca2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:17:05.440119 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5a6b093e-2d0c-4f7d-81c5-8ac5e7202c98 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:05.440766 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5a6b093e-2d0c-4f7d-81c5-8ac5e7202c98 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:05.441983 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5a6b093e-2d0c-4f7d-81c5-8ac5e7202c98 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:05.442601 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5a6b093e-2d0c-4f7d-81c5-8ac5e7202c98 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:05.450802 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-39d3934c-d135-465a-a435-ba026019fd23 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] 10.222.0.16 "GET /compute/v2.1/servers/6cb4e66a-eba2-42b7-91e8-09ee7b2a5a4f" status: 200 len: 1317 microversion: 2.1 time: 0.298770 Jul 16 21:17:05.460908 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1097/2097] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:17:05 2022] GET /compute/v2.1/servers/6cb4e66a-eba2-42b7-91e8-09ee7b2a5a4f => generated 1317 bytes in 302 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:17:05.483708 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-f9954d76-4f19-4607-91d1-d6d91dcba9da tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:17:05.484364 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-f9954d76-4f19-4607-91d1-d6d91dcba9da tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] [instance: 7e4b142d-f489-4220-adc3-7b348dc14ce9] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:17:05.505774 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f9954d76-4f19-4607-91d1-d6d91dcba9da tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:05.506244 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f9954d76-4f19-4607-91d1-d6d91dcba9da tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:05.507325 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f9954d76-4f19-4607-91d1-d6d91dcba9da tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:05.507877 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f9954d76-4f19-4607-91d1-d6d91dcba9da tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:05.635604 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-5a6b093e-2d0c-4f7d-81c5-8ac5e7202c98 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '015ac1bbb2df487e99e5bddc60ee6923', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9f5f03e09936489fa1505188cefe3445', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:17:05.682218 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-5a6b093e-2d0c-4f7d-81c5-8ac5e7202c98 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '015ac1bbb2df487e99e5bddc60ee6923', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9f5f03e09936489fa1505188cefe3445', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:17:05.688395 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-5a6b093e-2d0c-4f7d-81c5-8ac5e7202c98 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '015ac1bbb2df487e99e5bddc60ee6923', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9f5f03e09936489fa1505188cefe3445', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:17:05.694224 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-5a6b093e-2d0c-4f7d-81c5-8ac5e7202c98 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] 10.222.0.16 "GET /compute/v2.1/servers/9657dddc-2416-4c95-aab3-d6b6f9673369" status: 200 len: 1371 microversion: 2.47 time: 0.264620 Jul 16 21:17:05.694224 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1001/2098] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:17:05 2022] GET /compute/v2.1/servers/9657dddc-2416-4c95-aab3-d6b6f9673369 => generated 1371 bytes in 268 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:17:05.701224 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-efdbd07c-2d69-4e16-a290-48dd5281dc1f tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServersAdminTestJSON-server-1188243810", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452"}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:17:05.716377 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-f9954d76-4f19-4607-91d1-d6d91dcba9da tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd4144b5388b46a989a41a2f556bb508', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a90d4bcd25ff4bf68bdc194180fbbe09', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:17:05.727342 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-f9954d76-4f19-4607-91d1-d6d91dcba9da tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] 10.222.0.16 "GET /compute/v2.1/servers/7e4b142d-f489-4220-adc3-7b348dc14ce9" status: 200 len: 1222 microversion: 2.1 time: 0.273496 Jul 16 21:17:05.737462 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1098/2099] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:17:05 2022] GET /compute/v2.1/servers/7e4b142d-f489-4220-adc3-7b348dc14ce9 => generated 1222 bytes in 284 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:17:05.779394 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.network.neutron [None req-efdbd07c-2d69-4e16-a290-48dd5281dc1f tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] validate_networks() for None {{(pid=138681) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 16 21:17:06.087297 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-68d9dd55-feb5-40ab-930c-d7f559031de4 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:17:06.088766 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-68d9dd55-feb5-40ab-930c-d7f559031de4 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] [instance: f03a2a83-e007-444a-b46b-70eb46bb242b] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:17:06.098479 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-68d9dd55-feb5-40ab-930c-d7f559031de4 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:06.099036 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-68d9dd55-feb5-40ab-930c-d7f559031de4 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:06.099902 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-68d9dd55-feb5-40ab-930c-d7f559031de4 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:06.100517 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-68d9dd55-feb5-40ab-930c-d7f559031de4 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:06.273498 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-efdbd07c-2d69-4e16-a290-48dd5281dc1f tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Flavor limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 16 21:17:06.274041 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-efdbd07c-2d69-4e16-a290-48dd5281dc1f tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Image limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 16 21:17:06.274841 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-efdbd07c-2d69-4e16-a290-48dd5281dc1f tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Flavor pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 16 21:17:06.275168 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-efdbd07c-2d69-4e16-a290-48dd5281dc1f tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Image pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 16 21:17:06.275610 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-efdbd07c-2d69-4e16-a290-48dd5281dc1f tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 16 21:17:06.277788 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_api:INSERT=7,nova_api:SELECT=92,nova_api:UPDATE=1 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:17:06.287162 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-efdbd07c-2d69-4e16-a290-48dd5281dc1f tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:06.287899 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-efdbd07c-2d69-4e16-a290-48dd5281dc1f tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:06.288888 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-efdbd07c-2d69-4e16-a290-48dd5281dc1f tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:06.289491 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-efdbd07c-2d69-4e16-a290-48dd5281dc1f tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:06.310144 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-efdbd07c-2d69-4e16-a290-48dd5281dc1f tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Getting quotas for project 65d5bcfa02af4e8284ba1342197e2860. Resources: {'ram', 'instances', 'cores'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:17:06.319172 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-efdbd07c-2d69-4e16-a290-48dd5281dc1f tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Getting quotas for user 2f5bc1f2209f4eb186d80e385b67c774 and project 65d5bcfa02af4e8284ba1342197e2860. Resources: {'ram', 'instances', 'cores'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:17:06.335506 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-efdbd07c-2d69-4e16-a290-48dd5281dc1f tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Going to run 1 instances... {{(pid=138681) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 16 21:17:06.337908 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-efdbd07c-2d69-4e16-a290-48dd5281dc1f tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] [instance: aa688a7e-0b28-4dfd-a159-8e19fd62ee02] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138681) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 16 21:17:06.486650 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-68d9dd55-feb5-40ab-930c-d7f559031de4 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'daf3c9e6634e4ae1bfb492eae1f44c93', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '24c64db8082648c8837c16352252f96d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:17:06.498782 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-68d9dd55-feb5-40ab-930c-d7f559031de4 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] 10.222.0.16 "GET /compute/v2.1/servers/f03a2a83-e007-444a-b46b-70eb46bb242b" status: 200 len: 1496 microversion: 2.1 time: 0.416039 Jul 16 21:17:06.500435 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1099/2100] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:17:06 2022] GET /compute/v2.1/servers/f03a2a83-e007-444a-b46b-70eb46bb242b => generated 1496 bytes in 419 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:17:06.516717 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-79e30428-8eb3-4590-b925-84962d75d1e0 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:17:06.517623 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-79e30428-8eb3-4590-b925-84962d75d1e0 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] [instance: 6cb4e66a-eba2-42b7-91e8-09ee7b2a5a4f] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:17:06.524242 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-79e30428-8eb3-4590-b925-84962d75d1e0 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:06.524883 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-79e30428-8eb3-4590-b925-84962d75d1e0 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:06.525818 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-79e30428-8eb3-4590-b925-84962d75d1e0 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:06.527464 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-79e30428-8eb3-4590-b925-84962d75d1e0 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:06.534893 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-efdbd07c-2d69-4e16-a290-48dd5281dc1f tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.839928 Jul 16 21:17:06.539496 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1002/2101] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 21:17:05 2022] POST /compute/v2.1/servers => generated 380 bytes in 845 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 16 21:17:06.552529 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-27019251-eca7-4bac-a6d9-6ec58976db1f tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:17:06.553282 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-27019251-eca7-4bac-a6d9-6ec58976db1f tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] [instance: aa688a7e-0b28-4dfd-a159-8e19fd62ee02] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:17:06.707305 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-27019251-eca7-4bac-a6d9-6ec58976db1f tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f5bc1f2209f4eb186d80e385b67c774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '65d5bcfa02af4e8284ba1342197e2860', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:17:06.719793 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-27019251-eca7-4bac-a6d9-6ec58976db1f tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] 10.222.0.16 "GET /compute/v2.1/servers/aa688a7e-0b28-4dfd-a159-8e19fd62ee02" status: 200 len: 1214 microversion: 2.1 time: 0.173102 Jul 16 21:17:06.731313 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1003/2102] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:17:06 2022] GET /compute/v2.1/servers/aa688a7e-0b28-4dfd-a159-8e19fd62ee02 => generated 1214 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:17:06.731313 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-b52e775a-0a48-4b12-9beb-3eb39becffda tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:17:06.731313 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-b52e775a-0a48-4b12-9beb-3eb39becffda tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] [instance: 9657dddc-2416-4c95-aab3-d6b6f9673369] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:17:06.751799 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b52e775a-0a48-4b12-9beb-3eb39becffda tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:06.751799 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b52e775a-0a48-4b12-9beb-3eb39becffda tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:06.761922 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b52e775a-0a48-4b12-9beb-3eb39becffda tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:06.761922 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b52e775a-0a48-4b12-9beb-3eb39becffda tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:06.789121 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:17:06.882477 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-79e30428-8eb3-4590-b925-84962d75d1e0 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '71e9304fc50447d6aac99703b5b6e4e1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e84496d78d4481e9d7648128868bca2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:17:06.884439 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-b52e775a-0a48-4b12-9beb-3eb39becffda tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '015ac1bbb2df487e99e5bddc60ee6923', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9f5f03e09936489fa1505188cefe3445', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:17:06.895184 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-79e30428-8eb3-4590-b925-84962d75d1e0 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] 10.222.0.16 "GET /compute/v2.1/servers/6cb4e66a-eba2-42b7-91e8-09ee7b2a5a4f" status: 200 len: 1317 microversion: 2.1 time: 0.383439 Jul 16 21:17:06.895714 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1100/2103] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:17:06 2022] GET /compute/v2.1/servers/6cb4e66a-eba2-42b7-91e8-09ee7b2a5a4f => generated 1317 bytes in 385 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:17:06.900027 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-b52e775a-0a48-4b12-9beb-3eb39becffda tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '015ac1bbb2df487e99e5bddc60ee6923', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9f5f03e09936489fa1505188cefe3445', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:17:06.902712 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-b52e775a-0a48-4b12-9beb-3eb39becffda tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '015ac1bbb2df487e99e5bddc60ee6923', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9f5f03e09936489fa1505188cefe3445', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:17:06.904480 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-b52e775a-0a48-4b12-9beb-3eb39becffda tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] 10.222.0.16 "GET /compute/v2.1/servers/9657dddc-2416-4c95-aab3-d6b6f9673369" status: 200 len: 1371 microversion: 2.47 time: 0.182743 Jul 16 21:17:06.906145 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-4e649f5b-cf45-4f08-b3da-e6f5e67f35ac tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:17:06.906684 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-4e649f5b-cf45-4f08-b3da-e6f5e67f35ac tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] [instance: 7e4b142d-f489-4220-adc3-7b348dc14ce9] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:17:06.907773 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1004/2104] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:17:06 2022] GET /compute/v2.1/servers/9657dddc-2416-4c95-aab3-d6b6f9673369 => generated 1371 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:17:06.925027 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4e649f5b-cf45-4f08-b3da-e6f5e67f35ac tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:06.927726 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4e649f5b-cf45-4f08-b3da-e6f5e67f35ac tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:06.927726 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4e649f5b-cf45-4f08-b3da-e6f5e67f35ac tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:06.927726 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4e649f5b-cf45-4f08-b3da-e6f5e67f35ac tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:07.049269 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-4e649f5b-cf45-4f08-b3da-e6f5e67f35ac tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd4144b5388b46a989a41a2f556bb508', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a90d4bcd25ff4bf68bdc194180fbbe09', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:17:07.057229 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-4e649f5b-cf45-4f08-b3da-e6f5e67f35ac tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] 10.222.0.16 "GET /compute/v2.1/servers/7e4b142d-f489-4220-adc3-7b348dc14ce9" status: 200 len: 1278 microversion: 2.1 time: 0.160465 Jul 16 21:17:07.058058 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1101/2105] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:17:06 2022] GET /compute/v2.1/servers/7e4b142d-f489-4220-adc3-7b348dc14ce9 => generated 1278 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:17:07.174867 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-03efb172-59da-4632-b1fa-f01e995347cd req-1f221c31-190c-43b9-ba83-5eec8c635f10 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "6cb4e66a-eba2-42b7-91e8-09ee7b2a5a4f", "name": "network-vif-plugged", "status": "completed", "tag": "381fe698-72ef-4f6f-8b88-5df5cb8d8d96"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:17:07.204881 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-03efb172-59da-4632-b1fa-f01e995347cd req-1f221c31-190c-43b9-ba83-5eec8c635f10 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:07.205418 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-03efb172-59da-4632-b1fa-f01e995347cd req-1f221c31-190c-43b9-ba83-5eec8c635f10 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:07.233867 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-03efb172-59da-4632-b1fa-f01e995347cd req-1f221c31-190c-43b9-ba83-5eec8c635f10 service nova] Creating event network-vif-plugged:381fe698-72ef-4f6f-8b88-5df5cb8d8d96 for instance 6cb4e66a-eba2-42b7-91e8-09ee7b2a5a4f on nch2-master-1 Jul 16 21:17:07.239370 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-03efb172-59da-4632-b1fa-f01e995347cd req-1f221c31-190c-43b9-ba83-5eec8c635f10 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.069885 Jul 16 21:17:07.243649 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1005/2106] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:17:07 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 75 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 16 21:17:07.520287 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-1e1831c4-6298-4025-ac17-095288592215 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:17:07.520940 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-1e1831c4-6298-4025-ac17-095288592215 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] [instance: f03a2a83-e007-444a-b46b-70eb46bb242b] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:17:07.535945 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1e1831c4-6298-4025-ac17-095288592215 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:07.538852 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1e1831c4-6298-4025-ac17-095288592215 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:07.541700 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1e1831c4-6298-4025-ac17-095288592215 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:07.542654 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1e1831c4-6298-4025-ac17-095288592215 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:07.750972 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-0a30d46a-1a6c-4d34-af55-86cf02a2f00e tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:17:07.752943 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-0a30d46a-1a6c-4d34-af55-86cf02a2f00e tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] [instance: aa688a7e-0b28-4dfd-a159-8e19fd62ee02] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:17:07.788812 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-1e1831c4-6298-4025-ac17-095288592215 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'daf3c9e6634e4ae1bfb492eae1f44c93', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '24c64db8082648c8837c16352252f96d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:17:07.799078 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-1e1831c4-6298-4025-ac17-095288592215 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] 10.222.0.16 "GET /compute/v2.1/servers/f03a2a83-e007-444a-b46b-70eb46bb242b" status: 200 len: 1496 microversion: 2.1 time: 0.285246 Jul 16 21:17:07.800402 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1102/2107] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:17:07 2022] GET /compute/v2.1/servers/f03a2a83-e007-444a-b46b-70eb46bb242b => generated 1496 bytes in 288 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:17:07.928819 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-974276bb-74e4-4820-8fef-eb0d0ddb1334 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:17:07.929501 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-974276bb-74e4-4820-8fef-eb0d0ddb1334 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] [instance: 6cb4e66a-eba2-42b7-91e8-09ee7b2a5a4f] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:17:07.948058 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-974276bb-74e4-4820-8fef-eb0d0ddb1334 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:07.950687 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-0a30d46a-1a6c-4d34-af55-86cf02a2f00e tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f5bc1f2209f4eb186d80e385b67c774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '65d5bcfa02af4e8284ba1342197e2860', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:17:07.951458 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-974276bb-74e4-4820-8fef-eb0d0ddb1334 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:07.952455 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-974276bb-74e4-4820-8fef-eb0d0ddb1334 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:07.953049 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-974276bb-74e4-4820-8fef-eb0d0ddb1334 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:07.963220 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-0a30d46a-1a6c-4d34-af55-86cf02a2f00e tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] 10.222.0.16 "GET /compute/v2.1/servers/aa688a7e-0b28-4dfd-a159-8e19fd62ee02" status: 200 len: 1214 microversion: 2.1 time: 0.228296 Jul 16 21:17:07.966399 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1006/2108] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:17:07 2022] GET /compute/v2.1/servers/aa688a7e-0b28-4dfd-a159-8e19fd62ee02 => generated 1214 bytes in 233 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:17:07.982124 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-a6a35031-3d58-42a4-b920-7701e5e90163 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:17:07.987094 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-a6a35031-3d58-42a4-b920-7701e5e90163 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] [instance: 9657dddc-2416-4c95-aab3-d6b6f9673369] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:17:07.997466 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a6a35031-3d58-42a4-b920-7701e5e90163 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:07.998136 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a6a35031-3d58-42a4-b920-7701e5e90163 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:07.999873 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a6a35031-3d58-42a4-b920-7701e5e90163 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:08.001052 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a6a35031-3d58-42a4-b920-7701e5e90163 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:08.179961 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-a6a35031-3d58-42a4-b920-7701e5e90163 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '015ac1bbb2df487e99e5bddc60ee6923', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9f5f03e09936489fa1505188cefe3445', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:17:08.196883 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-a6a35031-3d58-42a4-b920-7701e5e90163 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '015ac1bbb2df487e99e5bddc60ee6923', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9f5f03e09936489fa1505188cefe3445', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:17:08.199086 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-a6a35031-3d58-42a4-b920-7701e5e90163 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '015ac1bbb2df487e99e5bddc60ee6923', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9f5f03e09936489fa1505188cefe3445', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:17:08.202710 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-a6a35031-3d58-42a4-b920-7701e5e90163 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] 10.222.0.16 "GET /compute/v2.1/servers/9657dddc-2416-4c95-aab3-d6b6f9673369" status: 200 len: 1371 microversion: 2.47 time: 0.232850 Jul 16 21:17:08.204692 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1007/2109] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:17:07 2022] GET /compute/v2.1/servers/9657dddc-2416-4c95-aab3-d6b6f9673369 => generated 1371 bytes in 237 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:17:08.213240 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-c0969b16-93f4-460a-89fd-54bff27e434c tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:17:08.213922 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-c0969b16-93f4-460a-89fd-54bff27e434c tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] [instance: 7e4b142d-f489-4220-adc3-7b348dc14ce9] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:17:08.241736 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c0969b16-93f4-460a-89fd-54bff27e434c tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:08.242865 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c0969b16-93f4-460a-89fd-54bff27e434c tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:08.244286 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c0969b16-93f4-460a-89fd-54bff27e434c tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:08.244752 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c0969b16-93f4-460a-89fd-54bff27e434c tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:08.398224 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-974276bb-74e4-4820-8fef-eb0d0ddb1334 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '71e9304fc50447d6aac99703b5b6e4e1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e84496d78d4481e9d7648128868bca2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:17:08.411083 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-974276bb-74e4-4820-8fef-eb0d0ddb1334 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] 10.222.0.16 "GET /compute/v2.1/servers/6cb4e66a-eba2-42b7-91e8-09ee7b2a5a4f" status: 200 len: 1317 microversion: 2.1 time: 0.492688 Jul 16 21:17:08.411856 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1103/2110] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:17:07 2022] GET /compute/v2.1/servers/6cb4e66a-eba2-42b7-91e8-09ee7b2a5a4f => generated 1317 bytes in 494 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:17:08.487421 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-c0969b16-93f4-460a-89fd-54bff27e434c tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd4144b5388b46a989a41a2f556bb508', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a90d4bcd25ff4bf68bdc194180fbbe09', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:17:08.499765 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-c0969b16-93f4-460a-89fd-54bff27e434c tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] 10.222.0.16 "GET /compute/v2.1/servers/7e4b142d-f489-4220-adc3-7b348dc14ce9" status: 200 len: 1318 microversion: 2.1 time: 0.291693 Jul 16 21:17:08.500381 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1008/2111] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:17:08 2022] GET /compute/v2.1/servers/7e4b142d-f489-4220-adc3-7b348dc14ce9 => generated 1318 bytes in 295 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:17:08.817587 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-64604c7d-b4c7-4b38-bb48-52da1f45333a tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:17:08.818288 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-64604c7d-b4c7-4b38-bb48-52da1f45333a tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] [instance: f03a2a83-e007-444a-b46b-70eb46bb242b] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:17:08.829969 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-64604c7d-b4c7-4b38-bb48-52da1f45333a tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:08.836034 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-64604c7d-b4c7-4b38-bb48-52da1f45333a tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:08.845825 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-64604c7d-b4c7-4b38-bb48-52da1f45333a tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:08.846263 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-64604c7d-b4c7-4b38-bb48-52da1f45333a tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:08.986144 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-7d06bc48-7229-4d8a-848d-aa1b19d921e7 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:17:08.990232 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-7d06bc48-7229-4d8a-848d-aa1b19d921e7 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] [instance: aa688a7e-0b28-4dfd-a159-8e19fd62ee02] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:17:09.000546 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7d06bc48-7229-4d8a-848d-aa1b19d921e7 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:09.001669 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7d06bc48-7229-4d8a-848d-aa1b19d921e7 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:09.002834 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7d06bc48-7229-4d8a-848d-aa1b19d921e7 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:09.003392 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7d06bc48-7229-4d8a-848d-aa1b19d921e7 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:09.134863 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-7d06bc48-7229-4d8a-848d-aa1b19d921e7 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f5bc1f2209f4eb186d80e385b67c774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '65d5bcfa02af4e8284ba1342197e2860', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:17:09.148459 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-7d06bc48-7229-4d8a-848d-aa1b19d921e7 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] 10.222.0.16 "GET /compute/v2.1/servers/aa688a7e-0b28-4dfd-a159-8e19fd62ee02" status: 200 len: 1266 microversion: 2.1 time: 0.170660 Jul 16 21:17:09.158287 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1009/2112] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:17:08 2022] GET /compute/v2.1/servers/aa688a7e-0b28-4dfd-a159-8e19fd62ee02 => generated 1266 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:17:09.225548 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-626203fd-9309-41e1-be53-76406ca98aac tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:17:09.226848 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-626203fd-9309-41e1-be53-76406ca98aac tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] [instance: 9657dddc-2416-4c95-aab3-d6b6f9673369] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:17:09.237819 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-626203fd-9309-41e1-be53-76406ca98aac tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:09.239080 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-626203fd-9309-41e1-be53-76406ca98aac tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:09.240861 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-626203fd-9309-41e1-be53-76406ca98aac tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:09.241508 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-626203fd-9309-41e1-be53-76406ca98aac tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:09.290689 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-64604c7d-b4c7-4b38-bb48-52da1f45333a tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'daf3c9e6634e4ae1bfb492eae1f44c93', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '24c64db8082648c8837c16352252f96d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:17:09.302820 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-64604c7d-b4c7-4b38-bb48-52da1f45333a tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] 10.222.0.16 "GET /compute/v2.1/servers/f03a2a83-e007-444a-b46b-70eb46bb242b" status: 200 len: 1496 microversion: 2.1 time: 0.489449 Jul 16 21:17:09.304065 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1104/2113] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:17:08 2022] GET /compute/v2.1/servers/f03a2a83-e007-444a-b46b-70eb46bb242b => generated 1496 bytes in 492 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:17:09.379155 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-626203fd-9309-41e1-be53-76406ca98aac tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '015ac1bbb2df487e99e5bddc60ee6923', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9f5f03e09936489fa1505188cefe3445', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:17:09.395187 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-626203fd-9309-41e1-be53-76406ca98aac tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '015ac1bbb2df487e99e5bddc60ee6923', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9f5f03e09936489fa1505188cefe3445', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:17:09.402284 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-626203fd-9309-41e1-be53-76406ca98aac tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '015ac1bbb2df487e99e5bddc60ee6923', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9f5f03e09936489fa1505188cefe3445', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:17:09.403713 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-626203fd-9309-41e1-be53-76406ca98aac tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] 10.222.0.16 "GET /compute/v2.1/servers/9657dddc-2416-4c95-aab3-d6b6f9673369" status: 200 len: 1356 microversion: 2.47 time: 0.183018 Jul 16 21:17:09.404910 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1010/2114] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:17:09 2022] GET /compute/v2.1/servers/9657dddc-2416-4c95-aab3-d6b6f9673369 => generated 1356 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 16 21:17:09.431893 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-878b04b9-2485-4526-862a-8f323fafd670 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:17:09.435214 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-878b04b9-2485-4526-862a-8f323fafd670 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Policy check for os_compute_api:servers:allow_all_filters failed with credentials {'is_admin': False, 'user_id': '015ac1bbb2df487e99e5bddc60ee6923', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9f5f03e09936489fa1505188cefe3445', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:17:09.436442 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-878b04b9-2485-4526-862a-8f323fafd670 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Searching by: {'deleted': False, 'project_id': '9f5f03e09936489fa1505188cefe3445'} {{(pid=138681) get_all /opt/stack/nova/nova/compute/api.py:2962}} Jul 16 21:17:09.443891 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-878b04b9-2485-4526-862a-8f323fafd670 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:09.444502 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-878b04b9-2485-4526-862a-8f323fafd670 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:09.445630 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-878b04b9-2485-4526-862a-8f323fafd670 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:09.446669 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-878b04b9-2485-4526-862a-8f323fafd670 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:09.551029 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-fb1a5386-6c37-4b1f-b06a-54dd28409838 tempest-ListImageFiltersNegativeTestJSON-981970341 tempest-ListImageFiltersNegativeTestJSON-981970341-project] Calling method '.version_select at 0x7fdda2b3d160>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:17:09.588791 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-fb1a5386-6c37-4b1f-b06a-54dd28409838 tempest-ListImageFiltersNegativeTestJSON-981970341 tempest-ListImageFiltersNegativeTestJSON-981970341-project] HTTP exception thrown: Image not found. Jul 16 21:17:09.594069 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-fb1a5386-6c37-4b1f-b06a-54dd28409838 tempest-ListImageFiltersNegativeTestJSON-981970341 tempest-ListImageFiltersNegativeTestJSON-981970341-project] Request method failure captured: Jul 16 21:17:09.594069 ncdv-master-1 devstack@n-api.service[138680]: request: GET /compute/v2.1/images/20e50ad0-a01a-4f65-b915-7ae8ba2079fd HTTP/1.1 Jul 16 21:17:09.594069 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:17:09.594069 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:17:09.594069 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:17:09.594069 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 0 Jul 16 21:17:09.594069 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:17:09.594069 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:17:09.594069 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:17:09.594069 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:17:09.594069 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0yrVXajuhVd3uXFiOboTM57blhuUAW0wy2_pFHZsRd-UkIWz_eMOiZg6dzKkwaScvIedxDfk70mjSx2u9b3fkVB524DHQ8Uu3NxHJtB4W6slqRdPL4c6svr9FoluWs4-qt2n7S8176B_Bygw33r5On3MY-E3In05umlJae6_Vl6Tmtc Jul 16 21:17:09.594069 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:17:09.594069 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:17:09.594069 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:17:09.594069 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:17:09.594069 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:17:09.594069 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:17:09.594069 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: 2e8eb118531a44d3872ca521c699dee5 Jul 16 21:17:09.594069 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-ListImageFiltersNegativeTestJSON-981970341 Jul 16 21:17:09.594069 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:17:09.594069 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:17:09.594069 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/2e8eb118531a44d3872ca521c699dee5"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/2e8eb118531a44d3872ca521c699dee5"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_2e8eb118531a44d3872ca521c699dee5"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/2e8eb118531a44d3872ca521c699dee5"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:17:09.594069 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-ListImageFiltersNegativeTestJSON-981970341 Jul 16 21:17:09.594069 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: 2e8eb118531a44d3872ca521c699dee5 Jul 16 21:17:09.594069 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-ListImageFiltersNegativeTestJSON-981970341 Jul 16 21:17:09.594069 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-ListImageFiltersNegativeTestJSON-981970341-project Jul 16 21:17:09.594069 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:17:09.594069 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:17:09.594069 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: 1fd5581f5cf04fd798b9c0ac68a93f77 Jul 16 21:17:09.594069 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-ListImageFiltersNegativeTestJSON-981970341-project Jul 16 21:17:09.594069 ncdv-master-1 devstack@n-api.service[138680]: method: Jul 16 21:17:09.594069 ncdv-master-1 devstack@n-api.service[138680]: exception: Image not found. Jul 16 21:17:09.594069 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': '20e50ad0-a01a-4f65-b915-7ae8ba2079fd'} Jul 16 21:17:09.594069 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:17:09.594069 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:17:09.594069 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 285, in show Jul 16 21:17:09.594069 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi image = self._client.call(context, 2, 'get', args=(image_id,)) Jul 16 21:17:09.594069 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 191, in call Jul 16 21:17:09.594069 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi result = getattr(controller, method)(*args, **kwargs) Jul 16 21:17:09.594069 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/v2/images.py", line 197, in get Jul 16 21:17:09.594069 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return self._get(image_id) Jul 16 21:17:09.594069 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/utils.py", line 670, in inner Jul 16 21:17:09.594069 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return RequestIdProxy(wrapped(*args, **kwargs)) Jul 16 21:17:09.594069 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/v2/images.py", line 190, in _get Jul 16 21:17:09.594069 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi resp, body = self.http_client.get(url, headers=header) Jul 16 21:17:09.594069 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/adapter.py", line 395, in get Jul 16 21:17:09.594069 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return self.request(url, 'GET', **kwargs) Jul 16 21:17:09.594069 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 380, in request Jul 16 21:17:09.594069 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return self._handle_response(resp) Jul 16 21:17:09.594069 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 120, in _handle_response Jul 16 21:17:09.594069 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.from_response(resp, resp.content) Jul 16 21:17:09.594069 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi glanceclient.exc.HTTPNotFound: HTTP 404 Not Found: No image found with ID 20e50ad0-a01a-4f65-b915-7ae8ba2079fd Jul 16 21:17:09.594069 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:17:09.594069 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:17:09.594069 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:17:09.594069 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:17:09.594069 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/images.py", line 86, in show Jul 16 21:17:09.594069 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi image = self._image_api.get(context, id) Jul 16 21:17:09.594069 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 1205, in get Jul 16 21:17:09.594069 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return session.show(context, image_id, Jul 16 21:17:09.594069 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 287, in show Jul 16 21:17:09.594069 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi _reraise_translated_image_exception(image_id) Jul 16 21:17:09.594069 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 1031, in _reraise_translated_image_exception Jul 16 21:17:09.597285 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise new_exc.with_traceback(exc_trace) Jul 16 21:17:09.597285 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 285, in show Jul 16 21:17:09.597285 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi image = self._client.call(context, 2, 'get', args=(image_id,)) Jul 16 21:17:09.597285 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 191, in call Jul 16 21:17:09.597285 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi result = getattr(controller, method)(*args, **kwargs) Jul 16 21:17:09.597285 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/v2/images.py", line 197, in get Jul 16 21:17:09.597285 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return self._get(image_id) Jul 16 21:17:09.597285 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/utils.py", line 670, in inner Jul 16 21:17:09.597285 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return RequestIdProxy(wrapped(*args, **kwargs)) Jul 16 21:17:09.597285 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/v2/images.py", line 190, in _get Jul 16 21:17:09.597285 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi resp, body = self.http_client.get(url, headers=header) Jul 16 21:17:09.597285 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/adapter.py", line 395, in get Jul 16 21:17:09.597285 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return self.request(url, 'GET', **kwargs) Jul 16 21:17:09.597285 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 380, in request Jul 16 21:17:09.597285 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return self._handle_response(resp) Jul 16 21:17:09.597285 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 120, in _handle_response Jul 16 21:17:09.597285 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.from_response(resp, resp.content) Jul 16 21:17:09.597285 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.ImageNotFound: Image 20e50ad0-a01a-4f65-b915-7ae8ba2079fd could not be found. Jul 16 21:17:09.597285 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:17:09.597285 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:17:09.597285 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:17:09.597285 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:17:09.597285 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:17:09.597285 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:17:09.597285 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:17:09.597285 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:17:09.597285 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Jul 16 21:17:09.597285 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Jul 16 21:17:09.597285 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:17:09.597285 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:17:09.597285 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/images.py", line 89, in show Jul 16 21:17:09.597285 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=explanation) Jul 16 21:17:09.597285 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Image not found. Jul 16 21:17:09.597285 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:17:09.597285 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:17:09.597285 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:17:09.597285 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:17:09.597285 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:17:09.597285 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:17:09.597285 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:17:09.597285 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:17:09.597285 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Image not found. Jul 16 21:17:09.597285 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:17:09.601566 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-fb1a5386-6c37-4b1f-b06a-54dd28409838 tempest-ListImageFiltersNegativeTestJSON-981970341 tempest-ListImageFiltersNegativeTestJSON-981970341-project] Returning 404 to user: Image not found. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:17:09.607874 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-fb1a5386-6c37-4b1f-b06a-54dd28409838 tempest-ListImageFiltersNegativeTestJSON-981970341 tempest-ListImageFiltersNegativeTestJSON-981970341-project] 10.222.0.16 "GET /compute/v2.1/images/20e50ad0-a01a-4f65-b915-7ae8ba2079fd" status: 404 len: 62 microversion: 2.1 time: 0.298197 Jul 16 21:17:09.607874 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1105/2115] 10.222.0.16 () {58 vars in 1232 bytes} [Sat Jul 16 21:17:09 2022] GET /compute/v2.1/images/20e50ad0-a01a-4f65-b915-7ae8ba2079fd => generated 62 bytes in 300 msecs (HTTP/1.1 404) 9 headers in 378 bytes (1 switches on core 0) Jul 16 21:17:09.637569 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-4d745016-928e-43d8-bdba-a74b58bcd3d5 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:17:09.638260 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-4d745016-928e-43d8-bdba-a74b58bcd3d5 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] [instance: 6cb4e66a-eba2-42b7-91e8-09ee7b2a5a4f] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:17:09.684281 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4d745016-928e-43d8-bdba-a74b58bcd3d5 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:09.684750 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4d745016-928e-43d8-bdba-a74b58bcd3d5 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:09.689620 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4d745016-928e-43d8-bdba-a74b58bcd3d5 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:09.690144 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4d745016-928e-43d8-bdba-a74b58bcd3d5 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:09.695712 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.multi_cell_list [None req-878b04b9-2485-4526-862a-8f323fafd670 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Listed batch of 2 results from cell out of 1000 limit. Returned 2 total so far. {{(pid=138681) do_query /opt/stack/nova/nova/compute/multi_cell_list.py:373}} Jul 16 21:17:09.824115 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-878b04b9-2485-4526-862a-8f323fafd670 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '015ac1bbb2df487e99e5bddc60ee6923', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9f5f03e09936489fa1505188cefe3445', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:17:09.832787 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-878b04b9-2485-4526-862a-8f323fafd670 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:09.833363 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-878b04b9-2485-4526-862a-8f323fafd670 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:09.853962 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-878b04b9-2485-4526-862a-8f323fafd670 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '015ac1bbb2df487e99e5bddc60ee6923', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9f5f03e09936489fa1505188cefe3445', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:17:09.856013 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-878b04b9-2485-4526-862a-8f323fafd670 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '015ac1bbb2df487e99e5bddc60ee6923', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9f5f03e09936489fa1505188cefe3445', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:17:10.070532 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-878b04b9-2485-4526-862a-8f323fafd670 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] 10.222.0.16 "GET /compute/v2.1/servers/detail" status: 200 len: 2705 microversion: 2.47 time: 0.648374 Jul 16 21:17:10.071193 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1011/2116] 10.222.0.16 () {60 vars in 1186 bytes} [Sat Jul 16 21:17:09 2022] GET /compute/v2.1/servers/detail => generated 2705 bytes in 650 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 16 21:17:10.076001 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-968cc474-75eb-44ed-a37a-ba71c22d8f1f tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:17:10.077305 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-968cc474-75eb-44ed-a37a-ba71c22d8f1f tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] [instance: 7e4b142d-f489-4220-adc3-7b348dc14ce9] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:17:10.092560 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-968cc474-75eb-44ed-a37a-ba71c22d8f1f tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:10.093112 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-968cc474-75eb-44ed-a37a-ba71c22d8f1f tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:10.093982 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-968cc474-75eb-44ed-a37a-ba71c22d8f1f tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:10.097638 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-968cc474-75eb-44ed-a37a-ba71c22d8f1f tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:10.872250 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-4d745016-928e-43d8-bdba-a74b58bcd3d5 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '71e9304fc50447d6aac99703b5b6e4e1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e84496d78d4481e9d7648128868bca2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:17:10.893583 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-4d745016-928e-43d8-bdba-a74b58bcd3d5 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] 10.222.0.16 "GET /compute/v2.1/servers/6cb4e66a-eba2-42b7-91e8-09ee7b2a5a4f" status: 200 len: 1317 microversion: 2.1 time: 1.287679 Jul 16 21:17:10.894690 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1106/2117] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:17:09 2022] GET /compute/v2.1/servers/6cb4e66a-eba2-42b7-91e8-09ee7b2a5a4f => generated 1317 bytes in 1289 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:17:10.910221 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-c9a1ab32-1428-476d-acf1-0266c21cf359 req-c7308127-8515-479e-830b-bd1674c4ad89 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "7e4b142d-f489-4220-adc3-7b348dc14ce9", "tag": "d10b6c2f-f386-410e-a6a2-fbd06a8fceaf"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:17:10.920759 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-c9a1ab32-1428-476d-acf1-0266c21cf359 req-c7308127-8515-479e-830b-bd1674c4ad89 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:10.921208 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-c9a1ab32-1428-476d-acf1-0266c21cf359 req-c7308127-8515-479e-830b-bd1674c4ad89 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:10.934140 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-c9a1ab32-1428-476d-acf1-0266c21cf359 req-c7308127-8515-479e-830b-bd1674c4ad89 service nova] Creating event network-changed:d10b6c2f-f386-410e-a6a2-fbd06a8fceaf for instance 7e4b142d-f489-4220-adc3-7b348dc14ce9 on nch1-master-1 Jul 16 21:17:10.935370 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-968cc474-75eb-44ed-a37a-ba71c22d8f1f tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd4144b5388b46a989a41a2f556bb508', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a90d4bcd25ff4bf68bdc194180fbbe09', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:17:10.940352 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-c9a1ab32-1428-476d-acf1-0266c21cf359 req-c7308127-8515-479e-830b-bd1674c4ad89 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.035565 Jul 16 21:17:10.941168 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1107/2118] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:17:10 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:17:10.945493 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-968cc474-75eb-44ed-a37a-ba71c22d8f1f tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] 10.222.0.16 "GET /compute/v2.1/servers/7e4b142d-f489-4220-adc3-7b348dc14ce9" status: 200 len: 1318 microversion: 2.1 time: 0.873295 Jul 16 21:17:10.946077 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1012/2119] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:17:10 2022] GET /compute/v2.1/servers/7e4b142d-f489-4220-adc3-7b348dc14ce9 => generated 1318 bytes in 874 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:17:10.956572 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-1fe54f31-e6c8-4783-9113-6cb428548c96 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:17:10.957780 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-1fe54f31-e6c8-4783-9113-6cb428548c96 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] [instance: 9657dddc-2416-4c95-aab3-d6b6f9673369] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:17:10.966183 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1fe54f31-e6c8-4783-9113-6cb428548c96 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:10.966328 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-52c2a691-cd78-41ab-bc3a-19c0523a2d99 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:17:10.966995 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1fe54f31-e6c8-4783-9113-6cb428548c96 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:10.973288 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-52c2a691-cd78-41ab-bc3a-19c0523a2d99 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] [instance: aa688a7e-0b28-4dfd-a159-8e19fd62ee02] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:17:10.973423 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1fe54f31-e6c8-4783-9113-6cb428548c96 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:10.973423 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1fe54f31-e6c8-4783-9113-6cb428548c96 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:10.976763 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-52c2a691-cd78-41ab-bc3a-19c0523a2d99 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:10.979697 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-52c2a691-cd78-41ab-bc3a-19c0523a2d99 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:10.980530 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-52c2a691-cd78-41ab-bc3a-19c0523a2d99 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:10.980982 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-52c2a691-cd78-41ab-bc3a-19c0523a2d99 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:11.005142 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-1fe54f31-e6c8-4783-9113-6cb428548c96 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] [instance: 9657dddc-2416-4c95-aab3-d6b6f9673369] Going to try to terminate instance {{(pid=138680) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 16 21:17:11.059937 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=99,nova_cell1:UPDATE=1 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:17:11.106246 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-52c2a691-cd78-41ab-bc3a-19c0523a2d99 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f5bc1f2209f4eb186d80e385b67c774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '65d5bcfa02af4e8284ba1342197e2860', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:17:11.116385 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-52c2a691-cd78-41ab-bc3a-19c0523a2d99 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] 10.222.0.16 "GET /compute/v2.1/servers/aa688a7e-0b28-4dfd-a159-8e19fd62ee02" status: 200 len: 1272 microversion: 2.1 time: 0.168110 Jul 16 21:17:11.116385 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1013/2120] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:17:10 2022] GET /compute/v2.1/servers/aa688a7e-0b28-4dfd-a159-8e19fd62ee02 => generated 1272 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:17:11.129754 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-28ac2300-c75d-4e33-aac8-352fdc40f3a8 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:17:11.130372 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-28ac2300-c75d-4e33-aac8-352fdc40f3a8 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] [instance: f03a2a83-e007-444a-b46b-70eb46bb242b] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:17:11.136949 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-28ac2300-c75d-4e33-aac8-352fdc40f3a8 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:11.137396 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-28ac2300-c75d-4e33-aac8-352fdc40f3a8 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:11.138165 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-28ac2300-c75d-4e33-aac8-352fdc40f3a8 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:11.138599 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-28ac2300-c75d-4e33-aac8-352fdc40f3a8 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:11.426456 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-1fe54f31-e6c8-4783-9113-6cb428548c96 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] 10.222.0.16 "DELETE /compute/v2.1/servers/9657dddc-2416-4c95-aab3-d6b6f9673369" status: 204 len: 0 microversion: 2.47 time: 0.483740 Jul 16 21:17:11.434711 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1108/2121] 10.222.0.16 () {60 vars in 1279 bytes} [Sat Jul 16 21:17:10 2022] DELETE /compute/v2.1/servers/9657dddc-2416-4c95-aab3-d6b6f9673369 => generated 0 bytes in 485 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Jul 16 21:17:11.443155 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-47341c9e-50fb-4b02-9d7c-6f5ca7ce4bf5 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:17:11.443898 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-47341c9e-50fb-4b02-9d7c-6f5ca7ce4bf5 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] [instance: 9657dddc-2416-4c95-aab3-d6b6f9673369] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:17:11.450543 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-47341c9e-50fb-4b02-9d7c-6f5ca7ce4bf5 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:11.451847 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-47341c9e-50fb-4b02-9d7c-6f5ca7ce4bf5 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:11.453698 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-47341c9e-50fb-4b02-9d7c-6f5ca7ce4bf5 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:11.454213 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-47341c9e-50fb-4b02-9d7c-6f5ca7ce4bf5 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:11.644015 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-47341c9e-50fb-4b02-9d7c-6f5ca7ce4bf5 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '015ac1bbb2df487e99e5bddc60ee6923', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9f5f03e09936489fa1505188cefe3445', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:17:11.652350 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-47341c9e-50fb-4b02-9d7c-6f5ca7ce4bf5 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '015ac1bbb2df487e99e5bddc60ee6923', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9f5f03e09936489fa1505188cefe3445', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:17:11.654090 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-47341c9e-50fb-4b02-9d7c-6f5ca7ce4bf5 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '015ac1bbb2df487e99e5bddc60ee6923', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9f5f03e09936489fa1505188cefe3445', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:17:11.656071 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-47341c9e-50fb-4b02-9d7c-6f5ca7ce4bf5 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] 10.222.0.16 "GET /compute/v2.1/servers/9657dddc-2416-4c95-aab3-d6b6f9673369" status: 200 len: 1362 microversion: 2.47 time: 0.217229 Jul 16 21:17:11.667948 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1109/2122] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:17:11 2022] GET /compute/v2.1/servers/9657dddc-2416-4c95-aab3-d6b6f9673369 => generated 1362 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:17:11.881859 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-28ac2300-c75d-4e33-aac8-352fdc40f3a8 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'daf3c9e6634e4ae1bfb492eae1f44c93', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '24c64db8082648c8837c16352252f96d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:17:11.915327 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-5bc8f8cf-b2cc-4d53-bb14-0adc0f1601c2 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:17:11.918090 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-28ac2300-c75d-4e33-aac8-352fdc40f3a8 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] 10.222.0.16 "GET /compute/v2.1/servers/f03a2a83-e007-444a-b46b-70eb46bb242b" status: 200 len: 1496 microversion: 2.1 time: 0.800155 Jul 16 21:17:11.918932 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1014/2123] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:17:11 2022] GET /compute/v2.1/servers/f03a2a83-e007-444a-b46b-70eb46bb242b => generated 1496 bytes in 802 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:17:11.925294 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-5bc8f8cf-b2cc-4d53-bb14-0adc0f1601c2 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] [instance: 6cb4e66a-eba2-42b7-91e8-09ee7b2a5a4f] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:17:11.946259 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5bc8f8cf-b2cc-4d53-bb14-0adc0f1601c2 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:11.946850 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5bc8f8cf-b2cc-4d53-bb14-0adc0f1601c2 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:11.949340 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5bc8f8cf-b2cc-4d53-bb14-0adc0f1601c2 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:11.950232 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5bc8f8cf-b2cc-4d53-bb14-0adc0f1601c2 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:11.965483 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-fd029947-da1a-4917-8336-8059c14d08c4 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:17:11.966117 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-fd029947-da1a-4917-8336-8059c14d08c4 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] [instance: 7e4b142d-f489-4220-adc3-7b348dc14ce9] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:17:11.975037 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fd029947-da1a-4917-8336-8059c14d08c4 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:11.975520 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fd029947-da1a-4917-8336-8059c14d08c4 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:11.976311 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fd029947-da1a-4917-8336-8059c14d08c4 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:11.976756 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fd029947-da1a-4917-8336-8059c14d08c4 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:12.247821 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-fd029947-da1a-4917-8336-8059c14d08c4 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd4144b5388b46a989a41a2f556bb508', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a90d4bcd25ff4bf68bdc194180fbbe09', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:17:12.254415 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-fd029947-da1a-4917-8336-8059c14d08c4 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] 10.222.0.16 "GET /compute/v2.1/servers/7e4b142d-f489-4220-adc3-7b348dc14ce9" status: 200 len: 1318 microversion: 2.1 time: 0.294064 Jul 16 21:17:12.255706 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1015/2124] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:17:11 2022] GET /compute/v2.1/servers/7e4b142d-f489-4220-adc3-7b348dc14ce9 => generated 1318 bytes in 297 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:17:12.263054 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-d412aff7-422b-451c-abac-23a8645d4a49 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:17:12.266858 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-d412aff7-422b-451c-abac-23a8645d4a49 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] [instance: aa688a7e-0b28-4dfd-a159-8e19fd62ee02] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:17:12.276909 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d412aff7-422b-451c-abac-23a8645d4a49 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:12.277617 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d412aff7-422b-451c-abac-23a8645d4a49 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:12.278951 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d412aff7-422b-451c-abac-23a8645d4a49 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:12.280205 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d412aff7-422b-451c-abac-23a8645d4a49 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:12.282677 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-5bc8f8cf-b2cc-4d53-bb14-0adc0f1601c2 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '71e9304fc50447d6aac99703b5b6e4e1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e84496d78d4481e9d7648128868bca2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:17:12.291413 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-5bc8f8cf-b2cc-4d53-bb14-0adc0f1601c2 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] 10.222.0.16 "GET /compute/v2.1/servers/6cb4e66a-eba2-42b7-91e8-09ee7b2a5a4f" status: 200 len: 1466 microversion: 2.1 time: 0.383457 Jul 16 21:17:12.295124 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1110/2125] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:17:11 2022] GET /compute/v2.1/servers/6cb4e66a-eba2-42b7-91e8-09ee7b2a5a4f => generated 1466 bytes in 388 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:17:12.314004 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-a57e5f86-81fd-47a8-84ba-92bf3c01fa8d tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:17:12.314660 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-a57e5f86-81fd-47a8-84ba-92bf3c01fa8d tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] [instance: b9a4910a-f117-4834-b699-498c6431b9bb] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:17:12.321441 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a57e5f86-81fd-47a8-84ba-92bf3c01fa8d tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:12.321902 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a57e5f86-81fd-47a8-84ba-92bf3c01fa8d tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:12.322888 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a57e5f86-81fd-47a8-84ba-92bf3c01fa8d tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:12.323392 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a57e5f86-81fd-47a8-84ba-92bf3c01fa8d tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:12.376792 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-d412aff7-422b-451c-abac-23a8645d4a49 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f5bc1f2209f4eb186d80e385b67c774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '65d5bcfa02af4e8284ba1342197e2860', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:17:12.385731 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-d412aff7-422b-451c-abac-23a8645d4a49 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] 10.222.0.16 "GET /compute/v2.1/servers/aa688a7e-0b28-4dfd-a159-8e19fd62ee02" status: 200 len: 1272 microversion: 2.1 time: 0.128350 Jul 16 21:17:12.386311 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1016/2126] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:17:12 2022] GET /compute/v2.1/servers/aa688a7e-0b28-4dfd-a159-8e19fd62ee02 => generated 1272 bytes in 130 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:17:12.669837 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-a57e5f86-81fd-47a8-84ba-92bf3c01fa8d tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '71e9304fc50447d6aac99703b5b6e4e1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e84496d78d4481e9d7648128868bca2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:17:12.678574 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-a3228d1a-e4b5-45ec-a472-cb649dc9e396 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:17:12.679256 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-a3228d1a-e4b5-45ec-a472-cb649dc9e396 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] [instance: 9657dddc-2416-4c95-aab3-d6b6f9673369] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:17:12.682864 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-a57e5f86-81fd-47a8-84ba-92bf3c01fa8d tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] 10.222.0.16 "GET /compute/v2.1/servers/b9a4910a-f117-4834-b699-498c6431b9bb" status: 200 len: 1466 microversion: 2.1 time: 0.374743 Jul 16 21:17:12.687258 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1111/2127] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:17:12 2022] GET /compute/v2.1/servers/b9a4910a-f117-4834-b699-498c6431b9bb => generated 1466 bytes in 380 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:17:12.688321 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a3228d1a-e4b5-45ec-a472-cb649dc9e396 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:12.688778 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a3228d1a-e4b5-45ec-a472-cb649dc9e396 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:12.689589 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a3228d1a-e4b5-45ec-a472-cb649dc9e396 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:12.690102 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a3228d1a-e4b5-45ec-a472-cb649dc9e396 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:12.705065 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-410f0762-222e-4ed6-81dc-a668889bb25b tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Action: 'action', calling method: >, body: {"createImage": {"name": "tempest-ListImageFiltersTestJSON-image-1665324876"}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:17:12.706549 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-410f0762-222e-4ed6-81dc-a668889bb25b tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] [instance: b9a4910a-f117-4834-b699-498c6431b9bb] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:17:12.720972 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-410f0762-222e-4ed6-81dc-a668889bb25b tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:12.721426 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-410f0762-222e-4ed6-81dc-a668889bb25b tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:12.729983 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-410f0762-222e-4ed6-81dc-a668889bb25b tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:12.730934 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-410f0762-222e-4ed6-81dc-a668889bb25b tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:12.827290 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-a3228d1a-e4b5-45ec-a472-cb649dc9e396 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '015ac1bbb2df487e99e5bddc60ee6923', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9f5f03e09936489fa1505188cefe3445', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:17:12.838530 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-a3228d1a-e4b5-45ec-a472-cb649dc9e396 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '015ac1bbb2df487e99e5bddc60ee6923', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9f5f03e09936489fa1505188cefe3445', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:17:12.840377 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-a3228d1a-e4b5-45ec-a472-cb649dc9e396 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '015ac1bbb2df487e99e5bddc60ee6923', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9f5f03e09936489fa1505188cefe3445', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:17:12.841596 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-a3228d1a-e4b5-45ec-a472-cb649dc9e396 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] 10.222.0.16 "GET /compute/v2.1/servers/9657dddc-2416-4c95-aab3-d6b6f9673369" status: 200 len: 1362 microversion: 2.47 time: 0.170032 Jul 16 21:17:12.845141 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1017/2128] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:17:12 2022] GET /compute/v2.1/servers/9657dddc-2416-4c95-aab3-d6b6f9673369 => generated 1362 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 16 21:17:12.936313 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-7753c1ca-056d-4898-a40c-ee649c1ca5fc tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:17:12.937025 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-7753c1ca-056d-4898-a40c-ee649c1ca5fc tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] [instance: f03a2a83-e007-444a-b46b-70eb46bb242b] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:17:12.949533 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7753c1ca-056d-4898-a40c-ee649c1ca5fc tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:12.950311 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7753c1ca-056d-4898-a40c-ee649c1ca5fc tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:12.954702 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7753c1ca-056d-4898-a40c-ee649c1ca5fc tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:12.956777 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7753c1ca-056d-4898-a40c-ee649c1ca5fc tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:13.361489 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-7753c1ca-056d-4898-a40c-ee649c1ca5fc tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'daf3c9e6634e4ae1bfb492eae1f44c93', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '24c64db8082648c8837c16352252f96d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:17:13.370327 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-7753c1ca-056d-4898-a40c-ee649c1ca5fc tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] 10.222.0.16 "GET /compute/v2.1/servers/f03a2a83-e007-444a-b46b-70eb46bb242b" status: 200 len: 1496 microversion: 2.1 time: 0.438927 Jul 16 21:17:13.381378 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1018/2129] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:17:12 2022] GET /compute/v2.1/servers/f03a2a83-e007-444a-b46b-70eb46bb242b => generated 1496 bytes in 450 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:17:13.386714 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-f9e7cfad-7722-4694-82b0-af10a31dfaaa tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:17:13.387619 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-f9e7cfad-7722-4694-82b0-af10a31dfaaa tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] [instance: 7e4b142d-f489-4220-adc3-7b348dc14ce9] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:17:13.400237 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f9e7cfad-7722-4694-82b0-af10a31dfaaa tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:13.400889 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f9e7cfad-7722-4694-82b0-af10a31dfaaa tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:13.401902 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f9e7cfad-7722-4694-82b0-af10a31dfaaa tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:13.402856 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f9e7cfad-7722-4694-82b0-af10a31dfaaa tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:13.544481 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-410f0762-222e-4ed6-81dc-a668889bb25b tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] 10.222.0.16 "POST /compute/v2.1/servers/b9a4910a-f117-4834-b699-498c6431b9bb/action" status: 202 len: 0 microversion: 2.1 time: 0.841926 Jul 16 21:17:13.548307 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1112/2130] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 21:17:12 2022] POST /compute/v2.1/servers/b9a4910a-f117-4834-b699-498c6431b9bb/action => generated 0 bytes in 849 msecs (HTTP/1.1 202) 10 headers in 450 bytes (1 switches on core 0) Jul 16 21:17:13.565048 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-54084021-14a0-49bd-9ccd-43cc4969bb59 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:17:13.565966 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-54084021-14a0-49bd-9ccd-43cc4969bb59 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] [instance: aa688a7e-0b28-4dfd-a159-8e19fd62ee02] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:17:13.575401 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-54084021-14a0-49bd-9ccd-43cc4969bb59 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:13.575869 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-54084021-14a0-49bd-9ccd-43cc4969bb59 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:13.577378 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-54084021-14a0-49bd-9ccd-43cc4969bb59 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:13.577833 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-54084021-14a0-49bd-9ccd-43cc4969bb59 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:13.753225 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-f9e7cfad-7722-4694-82b0-af10a31dfaaa tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd4144b5388b46a989a41a2f556bb508', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a90d4bcd25ff4bf68bdc194180fbbe09', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:17:13.759315 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-f9e7cfad-7722-4694-82b0-af10a31dfaaa tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] 10.222.0.16 "GET /compute/v2.1/servers/7e4b142d-f489-4220-adc3-7b348dc14ce9" status: 200 len: 1318 microversion: 2.1 time: 0.376376 Jul 16 21:17:13.769149 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1019/2131] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:17:13 2022] GET /compute/v2.1/servers/7e4b142d-f489-4220-adc3-7b348dc14ce9 => generated 1318 bytes in 387 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:17:13.774367 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-f1b6a820-ba62-4600-9843-d284eab8f9dc tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Calling method '.version_select at 0x7fdda2ab3160>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:17:13.836359 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-f1b6a820-ba62-4600-9843-d284eab8f9dc tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] 10.222.0.16 "GET /compute/v2.1/images/245dff72-cf22-471e-9dc0-0e7271c7ed43" status: 200 len: 1455 microversion: 2.1 time: 0.065943 Jul 16 21:17:13.836949 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1020/2132] 10.222.0.16 () {58 vars in 1232 bytes} [Sat Jul 16 21:17:13 2022] GET /compute/v2.1/images/245dff72-cf22-471e-9dc0-0e7271c7ed43 => generated 1455 bytes in 67 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:17:13.864411 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-8cd500f7-3cf1-4ef0-b16d-105b5918958d tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:17:13.865104 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-8cd500f7-3cf1-4ef0-b16d-105b5918958d tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] [instance: 9657dddc-2416-4c95-aab3-d6b6f9673369] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:17:13.872318 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8cd500f7-3cf1-4ef0-b16d-105b5918958d tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:13.872777 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8cd500f7-3cf1-4ef0-b16d-105b5918958d tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:13.874405 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8cd500f7-3cf1-4ef0-b16d-105b5918958d tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:13.874545 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8cd500f7-3cf1-4ef0-b16d-105b5918958d tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:13.887533 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell0:SELECT=2 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:17:13.959866 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8cd500f7-3cf1-4ef0-b16d-105b5918958d tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:13.959866 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8cd500f7-3cf1-4ef0-b16d-105b5918958d tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:13.959866 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.objects.instance [None req-8cd500f7-3cf1-4ef0-b16d-105b5918958d tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lazy-loading 'fault' on Instance uuid 9657dddc-2416-4c95-aab3-d6b6f9673369 {{(pid=138681) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:17:14.137761 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-54084021-14a0-49bd-9ccd-43cc4969bb59 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f5bc1f2209f4eb186d80e385b67c774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '65d5bcfa02af4e8284ba1342197e2860', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:17:14.147909 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-54084021-14a0-49bd-9ccd-43cc4969bb59 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] 10.222.0.16 "GET /compute/v2.1/servers/aa688a7e-0b28-4dfd-a159-8e19fd62ee02" status: 200 len: 1314 microversion: 2.1 time: 0.596941 Jul 16 21:17:14.148969 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1113/2133] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:17:13 2022] GET /compute/v2.1/servers/aa688a7e-0b28-4dfd-a159-8e19fd62ee02 => generated 1314 bytes in 600 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:17:14.161648 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-8cd500f7-3cf1-4ef0-b16d-105b5918958d tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '015ac1bbb2df487e99e5bddc60ee6923', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9f5f03e09936489fa1505188cefe3445', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:17:14.170266 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-8cd500f7-3cf1-4ef0-b16d-105b5918958d tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '015ac1bbb2df487e99e5bddc60ee6923', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9f5f03e09936489fa1505188cefe3445', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:17:14.171855 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-8cd500f7-3cf1-4ef0-b16d-105b5918958d tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '015ac1bbb2df487e99e5bddc60ee6923', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9f5f03e09936489fa1505188cefe3445', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:17:14.173050 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-8cd500f7-3cf1-4ef0-b16d-105b5918958d tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] 10.222.0.16 "GET /compute/v2.1/servers/9657dddc-2416-4c95-aab3-d6b6f9673369" status: 200 len: 1367 microversion: 2.47 time: 0.313159 Jul 16 21:17:14.173851 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1021/2134] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:17:13 2022] GET /compute/v2.1/servers/9657dddc-2416-4c95-aab3-d6b6f9673369 => generated 1367 bytes in 315 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 16 21:17:14.394437 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-68a615eb-c1bb-4484-b348-bb259722d52a tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:17:14.395609 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-68a615eb-c1bb-4484-b348-bb259722d52a tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] [instance: f03a2a83-e007-444a-b46b-70eb46bb242b] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:17:14.405583 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-68a615eb-c1bb-4484-b348-bb259722d52a tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:14.406468 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-68a615eb-c1bb-4484-b348-bb259722d52a tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:14.407400 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-68a615eb-c1bb-4484-b348-bb259722d52a tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:14.408122 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-68a615eb-c1bb-4484-b348-bb259722d52a tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:14.649348 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-7182ccca-316a-4874-9dfa-3347f1360a97 req-27fd0c15-97b9-4265-9c10-4a171c6fc4cf service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "e7598b4a-f70c-4186-88d1-6fd0057f8656", "tag": "5e72e822-7cdd-445c-8d25-dc81877dc718"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:17:14.679767 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-7182ccca-316a-4874-9dfa-3347f1360a97 req-27fd0c15-97b9-4265-9c10-4a171c6fc4cf service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:14.680543 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-7182ccca-316a-4874-9dfa-3347f1360a97 req-27fd0c15-97b9-4265-9c10-4a171c6fc4cf service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:14.727067 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-68a615eb-c1bb-4484-b348-bb259722d52a tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'daf3c9e6634e4ae1bfb492eae1f44c93', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '24c64db8082648c8837c16352252f96d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:17:14.727963 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-7182ccca-316a-4874-9dfa-3347f1360a97 req-27fd0c15-97b9-4265-9c10-4a171c6fc4cf service nova] Creating event network-changed:5e72e822-7cdd-445c-8d25-dc81877dc718 for instance e7598b4a-f70c-4186-88d1-6fd0057f8656 on nch1-master-1 Jul 16 21:17:14.734268 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-7182ccca-316a-4874-9dfa-3347f1360a97 req-27fd0c15-97b9-4265-9c10-4a171c6fc4cf service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.095522 Jul 16 21:17:14.735106 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1022/2135] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:17:14 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:17:14.756870 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-68a615eb-c1bb-4484-b348-bb259722d52a tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] 10.222.0.16 "GET /compute/v2.1/servers/f03a2a83-e007-444a-b46b-70eb46bb242b" status: 200 len: 1496 microversion: 2.1 time: 0.370087 Jul 16 21:17:14.763381 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1114/2136] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:17:14 2022] GET /compute/v2.1/servers/f03a2a83-e007-444a-b46b-70eb46bb242b => generated 1496 bytes in 379 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:17:14.778368 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-9a667a57-aed7-4503-9065-4dbf6dccd426 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:17:14.779814 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-9a667a57-aed7-4503-9065-4dbf6dccd426 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] [instance: 7e4b142d-f489-4220-adc3-7b348dc14ce9] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:17:14.788779 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9a667a57-aed7-4503-9065-4dbf6dccd426 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:14.789453 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9a667a57-aed7-4503-9065-4dbf6dccd426 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:14.790574 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9a667a57-aed7-4503-9065-4dbf6dccd426 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:14.791195 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9a667a57-aed7-4503-9065-4dbf6dccd426 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:14.842771 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=100 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:17:14.850905 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-412854f3-97ed-48f9-a00b-5288ac7e7dc8 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Calling method '.version_select at 0x7fdda2a150d0>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:17:14.958930 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-412854f3-97ed-48f9-a00b-5288ac7e7dc8 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] 10.222.0.16 "GET /compute/v2.1/images/245dff72-cf22-471e-9dc0-0e7271c7ed43" status: 200 len: 1455 microversion: 2.1 time: 0.112360 Jul 16 21:17:14.960990 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1115/2137] 10.222.0.16 () {58 vars in 1232 bytes} [Sat Jul 16 21:17:14 2022] GET /compute/v2.1/images/245dff72-cf22-471e-9dc0-0e7271c7ed43 => generated 1455 bytes in 115 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:17:15.169051 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-92a50ab1-d526-4a37-bcd9-c94dc64fc4fd tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:17:15.171003 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-92a50ab1-d526-4a37-bcd9-c94dc64fc4fd tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] [instance: aa688a7e-0b28-4dfd-a159-8e19fd62ee02] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:17:15.191589 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-92a50ab1-d526-4a37-bcd9-c94dc64fc4fd tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.009s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:15.192385 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-92a50ab1-d526-4a37-bcd9-c94dc64fc4fd tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:15.193362 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-92a50ab1-d526-4a37-bcd9-c94dc64fc4fd tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:15.194347 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-92a50ab1-d526-4a37-bcd9-c94dc64fc4fd tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:15.346285 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-9a667a57-aed7-4503-9065-4dbf6dccd426 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd4144b5388b46a989a41a2f556bb508', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a90d4bcd25ff4bf68bdc194180fbbe09', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:17:15.354859 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-9a667a57-aed7-4503-9065-4dbf6dccd426 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] 10.222.0.16 "GET /compute/v2.1/servers/7e4b142d-f489-4220-adc3-7b348dc14ce9" status: 200 len: 1318 microversion: 2.1 time: 0.580449 Jul 16 21:17:15.364531 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1023/2138] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:17:14 2022] GET /compute/v2.1/servers/7e4b142d-f489-4220-adc3-7b348dc14ce9 => generated 1318 bytes in 592 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:17:15.384565 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-ca06f5e3-1753-4ec4-bbef-ab60a5014fd9 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:17:15.385194 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-ca06f5e3-1753-4ec4-bbef-ab60a5014fd9 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] [instance: 9657dddc-2416-4c95-aab3-d6b6f9673369] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:17:15.402754 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ca06f5e3-1753-4ec4-bbef-ab60a5014fd9 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:15.406121 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ca06f5e3-1753-4ec4-bbef-ab60a5014fd9 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:15.408245 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ca06f5e3-1753-4ec4-bbef-ab60a5014fd9 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:15.413679 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ca06f5e3-1753-4ec4-bbef-ab60a5014fd9 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:15.440123 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-ca06f5e3-1753-4ec4-bbef-ab60a5014fd9 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] HTTP exception thrown: Instance 9657dddc-2416-4c95-aab3-d6b6f9673369 could not be found. Jul 16 21:17:15.441990 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-ca06f5e3-1753-4ec4-bbef-ab60a5014fd9 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Request method failure captured: Jul 16 21:17:15.441990 ncdv-master-1 devstack@n-api.service[138681]: request: GET /compute/v2.1/servers/9657dddc-2416-4c95-aab3-d6b6f9673369 HTTP/1.1 Jul 16 21:17:15.441990 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:17:15.441990 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:17:15.441990 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:17:15.441990 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 21:17:15.441990 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:17:15.441990 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:17:15.441990 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:17:15.441990 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:17:15.441990 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0yqk_pXk-wKum3b8cCuT2pEaU9dBudX_cAtBJ9pskOXJYY-ofAku47SZoD93qXsTHLBEazC1J-w1Qcw50R8VujDLzTf9h8xBm8O8X1husFf7xOBV98tr4pV6N0QotOfB-rAq_gHVRYXqCGhTgE068IUohjigJzKTCWdzNDvOifq8IvA Jul 16 21:17:15.441990 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:17:15.441990 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:17:15.441990 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:17:15.441990 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:17:15.441990 ncdv-master-1 devstack@n-api.service[138681]: X-Openstack-Nova-Api-Version: 2.47 Jul 16 21:17:15.441990 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:17:15.441990 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:17:15.441990 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: 9f5f03e09936489fa1505188cefe3445 Jul 16 21:17:15.441990 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-ServerShowV247Test-1709646099 Jul 16 21:17:15.441990 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:17:15.441990 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:17:15.441990 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/9f5f03e09936489fa1505188cefe3445"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/9f5f03e09936489fa1505188cefe3445"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_9f5f03e09936489fa1505188cefe3445"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/9f5f03e09936489fa1505188cefe3445"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:17:15.441990 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-ServerShowV247Test-1709646099 Jul 16 21:17:15.441990 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: 9f5f03e09936489fa1505188cefe3445 Jul 16 21:17:15.441990 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-ServerShowV247Test-1709646099 Jul 16 21:17:15.441990 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-ServerShowV247Test-1709646099-project Jul 16 21:17:15.441990 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:17:15.441990 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:17:15.441990 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: 015ac1bbb2df487e99e5bddc60ee6923 Jul 16 21:17:15.441990 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-ServerShowV247Test-1709646099-project Jul 16 21:17:15.441990 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:17:15.441990 ncdv-master-1 devstack@n-api.service[138681]: exception: Instance 9657dddc-2416-4c95-aab3-d6b6f9673369 could not be found. Jul 16 21:17:15.441990 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': '9657dddc-2416-4c95-aab3-d6b6f9673369'} Jul 16 21:17:15.441990 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:17:15.441990 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:17:15.441990 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:17:15.441990 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:17:15.441990 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:17:15.441990 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:17:15.441990 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:17:15.441990 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:17:15.441990 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:17:15.441990 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:17:15.441990 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 9657dddc-2416-4c95-aab3-d6b6f9673369 could not be found. Jul 16 21:17:15.441990 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:17:15.441990 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:17:15.441990 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:17:15.441990 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:17:15.441990 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:17:15.441990 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:17:15.441990 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:17:15.441990 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:17:15.441990 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:17:15.441990 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:17:15.441990 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:17:15.441990 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:17:15.441990 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:17:15.441990 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:17:15.441990 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:17:15.441990 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:17:15.441990 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 9657dddc-2416-4c95-aab3-d6b6f9673369 could not be found. Jul 16 21:17:15.441990 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:17:15.441990 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:17:15.441990 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:17:15.451368 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:17:15.451368 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:17:15.451368 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:17:15.451368 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:17:15.451368 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:17:15.451368 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 9657dddc-2416-4c95-aab3-d6b6f9673369 could not be found. Jul 16 21:17:15.451368 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:17:15.451368 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-ca06f5e3-1753-4ec4-bbef-ab60a5014fd9 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Returning 404 to user: Instance 9657dddc-2416-4c95-aab3-d6b6f9673369 could not be found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:17:15.451368 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-ca06f5e3-1753-4ec4-bbef-ab60a5014fd9 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] 10.222.0.16 "GET /compute/v2.1/servers/9657dddc-2416-4c95-aab3-d6b6f9673369" status: 404 len: 111 microversion: 2.47 time: 0.066497 Jul 16 21:17:15.451368 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1024/2139] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:17:15 2022] GET /compute/v2.1/servers/9657dddc-2416-4c95-aab3-d6b6f9673369 => generated 111 bytes in 80 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Jul 16 21:17:15.457948 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-e58d6b5b-9b7c-4df3-b8d7-a014f980522b tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:17:15.458699 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-e58d6b5b-9b7c-4df3-b8d7-a014f980522b tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] [instance: 0bcfc499-2e0f-48b6-994a-e0cb74ac7b4f] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:17:15.467893 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e58d6b5b-9b7c-4df3-b8d7-a014f980522b tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:15.468520 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e58d6b5b-9b7c-4df3-b8d7-a014f980522b tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:15.469459 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e58d6b5b-9b7c-4df3-b8d7-a014f980522b tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:15.470048 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e58d6b5b-9b7c-4df3-b8d7-a014f980522b tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:15.497863 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-e58d6b5b-9b7c-4df3-b8d7-a014f980522b tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] [instance: 0bcfc499-2e0f-48b6-994a-e0cb74ac7b4f] Going to try to terminate instance {{(pid=138681) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 16 21:17:15.605528 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-92a50ab1-d526-4a37-bcd9-c94dc64fc4fd tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f5bc1f2209f4eb186d80e385b67c774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '65d5bcfa02af4e8284ba1342197e2860', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:17:15.615820 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-92a50ab1-d526-4a37-bcd9-c94dc64fc4fd tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] 10.222.0.16 "GET /compute/v2.1/servers/aa688a7e-0b28-4dfd-a159-8e19fd62ee02" status: 200 len: 1314 microversion: 2.1 time: 0.453749 Jul 16 21:17:15.616435 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1116/2140] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:17:15 2022] GET /compute/v2.1/servers/aa688a7e-0b28-4dfd-a159-8e19fd62ee02 => generated 1314 bytes in 455 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:17:15.774940 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-b31a04c2-3b18-45c7-a56b-be8645a413f6 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:17:15.776440 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-b31a04c2-3b18-45c7-a56b-be8645a413f6 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] [instance: f03a2a83-e007-444a-b46b-70eb46bb242b] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:17:15.788249 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b31a04c2-3b18-45c7-a56b-be8645a413f6 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:15.790393 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b31a04c2-3b18-45c7-a56b-be8645a413f6 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:15.791429 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b31a04c2-3b18-45c7-a56b-be8645a413f6 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:15.792209 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b31a04c2-3b18-45c7-a56b-be8645a413f6 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:16.030944 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-e58d6b5b-9b7c-4df3-b8d7-a014f980522b tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] 10.222.0.16 "DELETE /compute/v2.1/servers/0bcfc499-2e0f-48b6-994a-e0cb74ac7b4f" status: 204 len: 0 microversion: 2.47 time: 0.575787 Jul 16 21:17:16.033117 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1025/2141] 10.222.0.16 () {60 vars in 1279 bytes} [Sat Jul 16 21:17:15 2022] DELETE /compute/v2.1/servers/0bcfc499-2e0f-48b6-994a-e0cb74ac7b4f => generated 0 bytes in 580 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Jul 16 21:17:16.042241 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-62d3c0f8-d75d-4036-a13c-b7c0e9dafe5d tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Calling method '.version_select at 0x7fdda2a09820>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:17:16.113310 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-62d3c0f8-d75d-4036-a13c-b7c0e9dafe5d tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] 10.222.0.16 "GET /compute/v2.1/images/245dff72-cf22-471e-9dc0-0e7271c7ed43" status: 200 len: 1455 microversion: 2.1 time: 0.078624 Jul 16 21:17:16.114579 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1026/2142] 10.222.0.16 () {58 vars in 1232 bytes} [Sat Jul 16 21:17:16 2022] GET /compute/v2.1/images/245dff72-cf22-471e-9dc0-0e7271c7ed43 => generated 1455 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:17:16.124391 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-4f48eb33-6407-4d42-a54c-6a23f9782ef4 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:17:16.125907 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-4f48eb33-6407-4d42-a54c-6a23f9782ef4 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] [instance: 0bcfc499-2e0f-48b6-994a-e0cb74ac7b4f] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:17:16.134627 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4f48eb33-6407-4d42-a54c-6a23f9782ef4 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:16.135427 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4f48eb33-6407-4d42-a54c-6a23f9782ef4 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:16.137415 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4f48eb33-6407-4d42-a54c-6a23f9782ef4 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:16.137415 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4f48eb33-6407-4d42-a54c-6a23f9782ef4 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:16.536392 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-4f48eb33-6407-4d42-a54c-6a23f9782ef4 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '015ac1bbb2df487e99e5bddc60ee6923', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9f5f03e09936489fa1505188cefe3445', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:17:16.546721 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-4f48eb33-6407-4d42-a54c-6a23f9782ef4 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '015ac1bbb2df487e99e5bddc60ee6923', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9f5f03e09936489fa1505188cefe3445', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:17:16.548732 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-4f48eb33-6407-4d42-a54c-6a23f9782ef4 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '015ac1bbb2df487e99e5bddc60ee6923', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9f5f03e09936489fa1505188cefe3445', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:17:16.549957 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-4f48eb33-6407-4d42-a54c-6a23f9782ef4 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] 10.222.0.16 "GET /compute/v2.1/servers/0bcfc499-2e0f-48b6-994a-e0cb74ac7b4f" status: 200 len: 1362 microversion: 2.47 time: 0.434110 Jul 16 21:17:16.550684 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1027/2143] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:17:16 2022] GET /compute/v2.1/servers/0bcfc499-2e0f-48b6-994a-e0cb74ac7b4f => generated 1362 bytes in 435 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:17:16.555243 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-a53d0be2-3788-4a7c-b24e-c86abb132892 req-bced639a-e1b5-485e-becd-419047f64073 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "7e4b142d-f489-4220-adc3-7b348dc14ce9", "name": "network-vif-plugged", "status": "completed", "tag": "d10b6c2f-f386-410e-a6a2-fbd06a8fceaf"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:17:16.564555 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-a53d0be2-3788-4a7c-b24e-c86abb132892 req-bced639a-e1b5-485e-becd-419047f64073 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:16.565006 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-a53d0be2-3788-4a7c-b24e-c86abb132892 req-bced639a-e1b5-485e-becd-419047f64073 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:16.577177 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-a53d0be2-3788-4a7c-b24e-c86abb132892 req-bced639a-e1b5-485e-becd-419047f64073 service nova] Creating event network-vif-plugged:d10b6c2f-f386-410e-a6a2-fbd06a8fceaf for instance 7e4b142d-f489-4220-adc3-7b348dc14ce9 on nch1-master-1 Jul 16 21:17:16.582792 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-a53d0be2-3788-4a7c-b24e-c86abb132892 req-bced639a-e1b5-485e-becd-419047f64073 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.030911 Jul 16 21:17:16.583434 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1028/2144] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:17:16 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:17:16.588654 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-c0a1a459-b1be-4b2e-a1f0-4c9d962d257a tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:17:16.589362 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-c0a1a459-b1be-4b2e-a1f0-4c9d962d257a tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] [instance: 7e4b142d-f489-4220-adc3-7b348dc14ce9] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:17:16.598305 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c0a1a459-b1be-4b2e-a1f0-4c9d962d257a tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:16.598767 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c0a1a459-b1be-4b2e-a1f0-4c9d962d257a tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:16.599747 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c0a1a459-b1be-4b2e-a1f0-4c9d962d257a tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:16.600421 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c0a1a459-b1be-4b2e-a1f0-4c9d962d257a tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:16.987643 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-b31a04c2-3b18-45c7-a56b-be8645a413f6 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'daf3c9e6634e4ae1bfb492eae1f44c93', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '24c64db8082648c8837c16352252f96d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:17:17.030285 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-b31a04c2-3b18-45c7-a56b-be8645a413f6 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] 10.222.0.16 "GET /compute/v2.1/servers/f03a2a83-e007-444a-b46b-70eb46bb242b" status: 200 len: 1473 microversion: 2.1 time: 1.260233 Jul 16 21:17:17.034887 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1117/2145] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:17:15 2022] GET /compute/v2.1/servers/f03a2a83-e007-444a-b46b-70eb46bb242b => generated 1473 bytes in 1265 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:17:17.058316 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-e11346bc-a55a-48ec-9bfb-72eba2f6695e tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:17:17.060134 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-e11346bc-a55a-48ec-9bfb-72eba2f6695e tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] [instance: aa688a7e-0b28-4dfd-a159-8e19fd62ee02] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:17:17.096858 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e11346bc-a55a-48ec-9bfb-72eba2f6695e tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:17.097329 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e11346bc-a55a-48ec-9bfb-72eba2f6695e tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:17.098154 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e11346bc-a55a-48ec-9bfb-72eba2f6695e tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:17.098603 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e11346bc-a55a-48ec-9bfb-72eba2f6695e tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:17.123826 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-c0a1a459-b1be-4b2e-a1f0-4c9d962d257a tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd4144b5388b46a989a41a2f556bb508', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a90d4bcd25ff4bf68bdc194180fbbe09', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:17:17.142045 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-c0a1a459-b1be-4b2e-a1f0-4c9d962d257a tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] 10.222.0.16 "GET /compute/v2.1/servers/7e4b142d-f489-4220-adc3-7b348dc14ce9" status: 200 len: 1318 microversion: 2.1 time: 0.557297 Jul 16 21:17:17.151784 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1029/2146] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:17:16 2022] GET /compute/v2.1/servers/7e4b142d-f489-4220-adc3-7b348dc14ce9 => generated 1318 bytes in 559 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:17:17.151784 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-180fd5d8-8362-4eb0-b9a2-7218e95253e0 req-02d4c70a-37b1-4505-9af5-1957e3f21a4c service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "aa688a7e-0b28-4dfd-a159-8e19fd62ee02", "tag": "0eaf070b-3b01-4123-bf36-5947ef05d97a"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:17:17.164354 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-180fd5d8-8362-4eb0-b9a2-7218e95253e0 req-02d4c70a-37b1-4505-9af5-1957e3f21a4c service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:17.164802 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-180fd5d8-8362-4eb0-b9a2-7218e95253e0 req-02d4c70a-37b1-4505-9af5-1957e3f21a4c service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:17.181579 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-180fd5d8-8362-4eb0-b9a2-7218e95253e0 req-02d4c70a-37b1-4505-9af5-1957e3f21a4c service nova] Creating event network-changed:0eaf070b-3b01-4123-bf36-5947ef05d97a for instance aa688a7e-0b28-4dfd-a159-8e19fd62ee02 on nch1-master-1 Jul 16 21:17:17.186765 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-180fd5d8-8362-4eb0-b9a2-7218e95253e0 req-02d4c70a-37b1-4505-9af5-1957e3f21a4c service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.042068 Jul 16 21:17:17.188236 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1030/2147] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:17:17 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:17:17.195372 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-2275ec2e-8fd7-4f5c-9161-f0cd49bfe86e tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Action: 'action', calling method: >, body: {"createImage": {"name": "tempest-test-snap-1090445214"}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:17:17.196828 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-2275ec2e-8fd7-4f5c-9161-f0cd49bfe86e tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] [instance: f03a2a83-e007-444a-b46b-70eb46bb242b] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:17:17.207093 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2275ec2e-8fd7-4f5c-9161-f0cd49bfe86e tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:17.207559 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2275ec2e-8fd7-4f5c-9161-f0cd49bfe86e tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:17.208350 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2275ec2e-8fd7-4f5c-9161-f0cd49bfe86e tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:17.208790 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2275ec2e-8fd7-4f5c-9161-f0cd49bfe86e tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:17.292265 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-e11346bc-a55a-48ec-9bfb-72eba2f6695e tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f5bc1f2209f4eb186d80e385b67c774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '65d5bcfa02af4e8284ba1342197e2860', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:17:17.302555 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-e11346bc-a55a-48ec-9bfb-72eba2f6695e tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] 10.222.0.16 "GET /compute/v2.1/servers/aa688a7e-0b28-4dfd-a159-8e19fd62ee02" status: 200 len: 1314 microversion: 2.1 time: 0.264201 Jul 16 21:17:17.303835 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1118/2148] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:17:17 2022] GET /compute/v2.1/servers/aa688a7e-0b28-4dfd-a159-8e19fd62ee02 => generated 1314 bytes in 266 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:17:17.311604 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-43ba6ee9-d482-4e1f-bea7-7c62c1dec126 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Calling method '.version_select at 0x7fdda29715e0>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:17:17.368123 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-43ba6ee9-d482-4e1f-bea7-7c62c1dec126 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] 10.222.0.16 "GET /compute/v2.1/images/245dff72-cf22-471e-9dc0-0e7271c7ed43" status: 200 len: 1455 microversion: 2.1 time: 0.062930 Jul 16 21:17:17.368855 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1119/2149] 10.222.0.16 () {58 vars in 1232 bytes} [Sat Jul 16 21:17:17 2022] GET /compute/v2.1/images/245dff72-cf22-471e-9dc0-0e7271c7ed43 => generated 1455 bytes in 64 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:17:17.569456 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-5a13b47b-e2aa-400b-bdd0-8ed8f6c6d22d tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:17:17.572795 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-5a13b47b-e2aa-400b-bdd0-8ed8f6c6d22d tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] [instance: 0bcfc499-2e0f-48b6-994a-e0cb74ac7b4f] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:17:17.587099 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_api:SELECT=99,nova_api:UPDATE=1 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:17:17.595040 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5a13b47b-e2aa-400b-bdd0-8ed8f6c6d22d tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:17.595762 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5a13b47b-e2aa-400b-bdd0-8ed8f6c6d22d tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:17.597473 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5a13b47b-e2aa-400b-bdd0-8ed8f6c6d22d tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:17.598776 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5a13b47b-e2aa-400b-bdd0-8ed8f6c6d22d tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:17.678035 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-5a13b47b-e2aa-400b-bdd0-8ed8f6c6d22d tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '015ac1bbb2df487e99e5bddc60ee6923', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9f5f03e09936489fa1505188cefe3445', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:17:17.686888 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-5a13b47b-e2aa-400b-bdd0-8ed8f6c6d22d tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '015ac1bbb2df487e99e5bddc60ee6923', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9f5f03e09936489fa1505188cefe3445', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:17:17.688741 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-5a13b47b-e2aa-400b-bdd0-8ed8f6c6d22d tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '015ac1bbb2df487e99e5bddc60ee6923', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9f5f03e09936489fa1505188cefe3445', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:17:17.690149 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-5a13b47b-e2aa-400b-bdd0-8ed8f6c6d22d tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] 10.222.0.16 "GET /compute/v2.1/servers/0bcfc499-2e0f-48b6-994a-e0cb74ac7b4f" status: 200 len: 1362 microversion: 2.47 time: 0.125690 Jul 16 21:17:17.691073 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1120/2150] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:17:17 2022] GET /compute/v2.1/servers/0bcfc499-2e0f-48b6-994a-e0cb74ac7b4f => generated 1362 bytes in 127 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:17:17.838737 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-2275ec2e-8fd7-4f5c-9161-f0cd49bfe86e tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] 10.222.0.16 "POST /compute/v2.1/servers/f03a2a83-e007-444a-b46b-70eb46bb242b/action" status: 202 len: 0 microversion: 2.1 time: 0.648861 Jul 16 21:17:17.839304 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-399e2bf9-62c2-4e1e-8359-c91da4c7da8f tempest-QuotaClassesAdminTestJSON-1596052195 tempest-QuotaClassesAdminTestJSON-1596052195-project-admin] Calling method '.version_select at 0x7fdd9c6b6700>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:17:17.839985 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1031/2151] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 21:17:17 2022] POST /compute/v2.1/servers/f03a2a83-e007-444a-b46b-70eb46bb242b/action => generated 0 bytes in 651 msecs (HTTP/1.1 202) 10 headers in 450 bytes (1 switches on core 0) Jul 16 21:17:17.842120 ncdv-master-1 devstack@n-api.service[138680]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-quota-class-sets:show": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:17:17.842120 ncdv-master-1 devstack@n-api.service[138680]: warnings.warn(msg) Jul 16 21:17:17.860898 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-9d0940a7-3e87-40ae-8ee5-8d3922f32770 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Calling method '.version_select at 0x7fdda2af1280>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:17:17.874555 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-399e2bf9-62c2-4e1e-8359-c91da4c7da8f tempest-QuotaClassesAdminTestJSON-1596052195 tempest-QuotaClassesAdminTestJSON-1596052195-project-admin] 10.222.0.16 "GET /compute/v2.1/os-quota-class-sets/default" status: 200 len: 303 microversion: 2.1 time: 0.182132 Jul 16 21:17:17.875824 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1121/2152] 10.222.0.16 () {58 vars in 1184 bytes} [Sat Jul 16 21:17:17 2022] GET /compute/v2.1/os-quota-class-sets/default => generated 303 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:17:17.901008 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-2aa344f9-d364-4814-b1ee-e6cc95340521 tempest-QuotaClassesAdminTestJSON-1596052195 tempest-QuotaClassesAdminTestJSON-1596052195-project-admin] Action: 'update', calling method: .version_select at 0x7fdda2971790>, body: {"quota_class_set": {"cores": 120, "fixed_ips": 99, "floating_ips": 99, "injected_file_content_bytes": 10340, "injected_file_path_bytes": 355, "injected_files": 105, "instances": 110, "key_pairs": 200, "metadata_items": 228, "ram": 51300, "security_group_rules": 99, "security_groups": 99}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:17:17.903090 ncdv-master-1 devstack@n-api.service[138680]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-quota-class-sets:update": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:17:17.903090 ncdv-master-1 devstack@n-api.service[138680]: warnings.warn(msg) Jul 16 21:17:17.932546 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-9d0940a7-3e87-40ae-8ee5-8d3922f32770 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] 10.222.0.16 "GET /compute/v2.1/images/211719d4-d58e-474c-ade9-8ffb307248f3" status: 200 len: 1414 microversion: 2.1 time: 0.085104 Jul 16 21:17:17.933509 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1032/2153] 10.222.0.16 () {58 vars in 1232 bytes} [Sat Jul 16 21:17:17 2022] GET /compute/v2.1/images/211719d4-d58e-474c-ade9-8ffb307248f3 => generated 1414 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:17:18.161976 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-ce472032-2c87-4dd0-9908-acdc5fd96dd7 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:17:18.162830 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-ce472032-2c87-4dd0-9908-acdc5fd96dd7 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] [instance: 7e4b142d-f489-4220-adc3-7b348dc14ce9] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:17:18.188274 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ce472032-2c87-4dd0-9908-acdc5fd96dd7 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:18.188732 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ce472032-2c87-4dd0-9908-acdc5fd96dd7 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:18.189529 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ce472032-2c87-4dd0-9908-acdc5fd96dd7 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:18.189994 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ce472032-2c87-4dd0-9908-acdc5fd96dd7 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:18.668528 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-ce472032-2c87-4dd0-9908-acdc5fd96dd7 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd4144b5388b46a989a41a2f556bb508', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a90d4bcd25ff4bf68bdc194180fbbe09', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:17:18.691349 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-ce472032-2c87-4dd0-9908-acdc5fd96dd7 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] 10.222.0.16 "GET /compute/v2.1/servers/7e4b142d-f489-4220-adc3-7b348dc14ce9" status: 200 len: 1318 microversion: 2.1 time: 0.526686 Jul 16 21:17:18.691349 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1033/2154] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:17:18 2022] GET /compute/v2.1/servers/7e4b142d-f489-4220-adc3-7b348dc14ce9 => generated 1318 bytes in 528 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:17:18.695425 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-1fceb3d0-cc08-4e25-8723-462529153fa5 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:17:18.696190 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-1fceb3d0-cc08-4e25-8723-462529153fa5 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] [instance: aa688a7e-0b28-4dfd-a159-8e19fd62ee02] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:17:18.705607 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1fceb3d0-cc08-4e25-8723-462529153fa5 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:18.706413 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1fceb3d0-cc08-4e25-8723-462529153fa5 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:18.707313 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1fceb3d0-cc08-4e25-8723-462529153fa5 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:18.707987 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1fceb3d0-cc08-4e25-8723-462529153fa5 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:19.145922 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-1fceb3d0-cc08-4e25-8723-462529153fa5 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f5bc1f2209f4eb186d80e385b67c774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '65d5bcfa02af4e8284ba1342197e2860', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:17:19.145922 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-1fceb3d0-cc08-4e25-8723-462529153fa5 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] 10.222.0.16 "GET /compute/v2.1/servers/aa688a7e-0b28-4dfd-a159-8e19fd62ee02" status: 200 len: 1314 microversion: 2.1 time: 0.437873 Jul 16 21:17:19.145922 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1034/2155] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:17:18 2022] GET /compute/v2.1/servers/aa688a7e-0b28-4dfd-a159-8e19fd62ee02 => generated 1314 bytes in 439 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:17:19.145922 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-937afba1-4c1e-4d98-90b1-7f76da443a4b tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Calling method '.version_select at 0x7fdda2ab3c10>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:17:19.197533 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-937afba1-4c1e-4d98-90b1-7f76da443a4b tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] 10.222.0.16 "GET /compute/v2.1/images/245dff72-cf22-471e-9dc0-0e7271c7ed43" status: 200 len: 1455 microversion: 2.1 time: 0.066817 Jul 16 21:17:19.197533 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1035/2156] 10.222.0.16 () {58 vars in 1232 bytes} [Sat Jul 16 21:17:19 2022] GET /compute/v2.1/images/245dff72-cf22-471e-9dc0-0e7271c7ed43 => generated 1455 bytes in 68 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:17:19.198256 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-6cf78e9f-c579-4ccd-b543-14c6fa360354 req-7752bc86-0fe5-4e56-96ff-295c4e3110c6 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "b9a4910a-f117-4834-b699-498c6431b9bb", "name": "network-vif-plugged", "status": "completed", "tag": "9ab69acd-f0a3-4355-b203-2c48e39cc3a1"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:17:19.209685 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-6cf78e9f-c579-4ccd-b543-14c6fa360354 req-7752bc86-0fe5-4e56-96ff-295c4e3110c6 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:19.210289 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-6cf78e9f-c579-4ccd-b543-14c6fa360354 req-7752bc86-0fe5-4e56-96ff-295c4e3110c6 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:19.222869 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-6cf78e9f-c579-4ccd-b543-14c6fa360354 req-7752bc86-0fe5-4e56-96ff-295c4e3110c6 service nova] Creating event network-vif-plugged:9ab69acd-f0a3-4355-b203-2c48e39cc3a1 for instance b9a4910a-f117-4834-b699-498c6431b9bb on nch2-master-1 Jul 16 21:17:19.229223 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-6cf78e9f-c579-4ccd-b543-14c6fa360354 req-7752bc86-0fe5-4e56-96ff-295c4e3110c6 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.035460 Jul 16 21:17:19.230043 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1036/2157] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:17:19 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:17:19.235409 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-db632ee2-5acf-4d47-bcd7-836a7175d573 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:17:19.236401 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-db632ee2-5acf-4d47-bcd7-836a7175d573 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] [instance: 0bcfc499-2e0f-48b6-994a-e0cb74ac7b4f] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:17:19.243845 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-db632ee2-5acf-4d47-bcd7-836a7175d573 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:19.244394 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-db632ee2-5acf-4d47-bcd7-836a7175d573 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:19.245245 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-db632ee2-5acf-4d47-bcd7-836a7175d573 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:19.245848 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-db632ee2-5acf-4d47-bcd7-836a7175d573 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:19.292104 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-db632ee2-5acf-4d47-bcd7-836a7175d573 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:19.292800 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-db632ee2-5acf-4d47-bcd7-836a7175d573 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:19.293412 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.objects.instance [None req-db632ee2-5acf-4d47-bcd7-836a7175d573 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lazy-loading 'fault' on Instance uuid 0bcfc499-2e0f-48b6-994a-e0cb74ac7b4f {{(pid=138681) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:17:19.402672 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-db632ee2-5acf-4d47-bcd7-836a7175d573 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '015ac1bbb2df487e99e5bddc60ee6923', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9f5f03e09936489fa1505188cefe3445', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:17:19.419036 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-db632ee2-5acf-4d47-bcd7-836a7175d573 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '015ac1bbb2df487e99e5bddc60ee6923', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9f5f03e09936489fa1505188cefe3445', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:17:19.421139 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-db632ee2-5acf-4d47-bcd7-836a7175d573 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '015ac1bbb2df487e99e5bddc60ee6923', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9f5f03e09936489fa1505188cefe3445', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:17:19.422446 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-db632ee2-5acf-4d47-bcd7-836a7175d573 tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] 10.222.0.16 "GET /compute/v2.1/servers/0bcfc499-2e0f-48b6-994a-e0cb74ac7b4f" status: 200 len: 1367 microversion: 2.47 time: 0.191060 Jul 16 21:17:19.423046 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1037/2158] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:17:19 2022] GET /compute/v2.1/servers/0bcfc499-2e0f-48b6-994a-e0cb74ac7b4f => generated 1367 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:17:19.436858 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-ebcc20fa-4bf2-4b3d-9ba0-648ab1f478d8 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Calling method '.version_select at 0x7fdda2ac00d0>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:17:19.528230 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-ebcc20fa-4bf2-4b3d-9ba0-648ab1f478d8 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] 10.222.0.16 "GET /compute/v2.1/images/211719d4-d58e-474c-ade9-8ffb307248f3" status: 200 len: 1414 microversion: 2.1 time: 0.096033 Jul 16 21:17:19.529746 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1038/2159] 10.222.0.16 () {58 vars in 1232 bytes} [Sat Jul 16 21:17:19 2022] GET /compute/v2.1/images/211719d4-d58e-474c-ade9-8ffb307248f3 => generated 1414 bytes in 106 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:17:19.703341 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell0:SELECT=8 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:17:19.708148 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-62b6b452-98f6-4ad8-add3-d7c94b59f2bd tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:17:19.708990 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-62b6b452-98f6-4ad8-add3-d7c94b59f2bd tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] [instance: 7e4b142d-f489-4220-adc3-7b348dc14ce9] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:17:19.718725 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-62b6b452-98f6-4ad8-add3-d7c94b59f2bd tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:19.723374 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-62b6b452-98f6-4ad8-add3-d7c94b59f2bd tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:19.726247 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-62b6b452-98f6-4ad8-add3-d7c94b59f2bd tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:19.737774 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-62b6b452-98f6-4ad8-add3-d7c94b59f2bd tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.010s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:19.822478 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell0:SELECT=24 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:17:19.900606 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-2aa344f9-d364-4814-b1ee-e6cc95340521 tempest-QuotaClassesAdminTestJSON-1596052195 tempest-QuotaClassesAdminTestJSON-1596052195-project-admin] 10.222.0.16 "PUT /compute/v2.1/os-quota-class-sets/default" status: 200 len: 290 microversion: 2.1 time: 2.013021 Jul 16 21:17:19.905513 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1122/2160] 10.222.0.16 () {60 vars in 1205 bytes} [Sat Jul 16 21:17:17 2022] PUT /compute/v2.1/os-quota-class-sets/default => generated 290 bytes in 2018 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:17:19.915458 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-fd89ce48-0757-4ea6-93d8-857354083a09 tempest-QuotaClassesAdminTestJSON-1596052195 tempest-QuotaClassesAdminTestJSON-1596052195-project-admin] Calling method '.version_select at 0x7fdda29605e0>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:17:19.941155 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-fd89ce48-0757-4ea6-93d8-857354083a09 tempest-QuotaClassesAdminTestJSON-1596052195 tempest-QuotaClassesAdminTestJSON-1596052195-project-admin] 10.222.0.16 "GET /compute/v2.1/os-quota-class-sets/default" status: 200 len: 307 microversion: 2.1 time: 0.029780 Jul 16 21:17:19.942413 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1123/2161] 10.222.0.16 () {58 vars in 1184 bytes} [Sat Jul 16 21:17:19 2022] GET /compute/v2.1/os-quota-class-sets/default => generated 307 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 16 21:17:19.954541 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-94f54d87-b6fc-4694-96f6-49aadd5ed5e9 tempest-QuotaClassesAdminTestJSON-1596052195 tempest-QuotaClassesAdminTestJSON-1596052195-project-admin] Action: 'update', calling method: .version_select at 0x7fdda297a820>, body: {"quota_class_set": {"cores": 20, "fixed_ips": -1, "floating_ips": -1, "injected_file_content_bytes": 10240, "injected_file_path_bytes": 255, "injected_files": 5, "instances": 10, "key_pairs": 100, "metadata_items": 128, "ram": 51200, "security_group_rules": -1, "security_groups": -1}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:17:20.005268 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-62b6b452-98f6-4ad8-add3-d7c94b59f2bd tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd4144b5388b46a989a41a2f556bb508', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a90d4bcd25ff4bf68bdc194180fbbe09', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:17:20.019208 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-62b6b452-98f6-4ad8-add3-d7c94b59f2bd tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] 10.222.0.16 "GET /compute/v2.1/servers/7e4b142d-f489-4220-adc3-7b348dc14ce9" status: 200 len: 1506 microversion: 2.1 time: 0.316884 Jul 16 21:17:20.032140 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1039/2162] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:17:19 2022] GET /compute/v2.1/servers/7e4b142d-f489-4220-adc3-7b348dc14ce9 => generated 1506 bytes in 331 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:17:20.038572 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-4be65b53-d245-4a0c-9a6c-94fccbec240a tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:17:20.039419 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-4be65b53-d245-4a0c-9a6c-94fccbec240a tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] [instance: 7e4b142d-f489-4220-adc3-7b348dc14ce9] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:17:20.048250 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4be65b53-d245-4a0c-9a6c-94fccbec240a tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:20.048878 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4be65b53-d245-4a0c-9a6c-94fccbec240a tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:20.050029 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4be65b53-d245-4a0c-9a6c-94fccbec240a tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:20.050825 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4be65b53-d245-4a0c-9a6c-94fccbec240a tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:20.438199 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-4be65b53-d245-4a0c-9a6c-94fccbec240a tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd4144b5388b46a989a41a2f556bb508', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a90d4bcd25ff4bf68bdc194180fbbe09', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:17:20.452970 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-4be65b53-d245-4a0c-9a6c-94fccbec240a tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] 10.222.0.16 "GET /compute/v2.1/servers/7e4b142d-f489-4220-adc3-7b348dc14ce9" status: 200 len: 1506 microversion: 2.1 time: 0.419321 Jul 16 21:17:20.459210 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1040/2163] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:17:20 2022] GET /compute/v2.1/servers/7e4b142d-f489-4220-adc3-7b348dc14ce9 => generated 1506 bytes in 424 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:17:20.466318 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-8acdd87c-f3c6-46b5-922b-38b7fa56e97e tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:17:20.467611 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-8acdd87c-f3c6-46b5-922b-38b7fa56e97e tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] [instance: aa688a7e-0b28-4dfd-a159-8e19fd62ee02] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:17:20.496016 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8acdd87c-f3c6-46b5-922b-38b7fa56e97e tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:20.516372 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8acdd87c-f3c6-46b5-922b-38b7fa56e97e tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.017s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:20.516372 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8acdd87c-f3c6-46b5-922b-38b7fa56e97e tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:20.516372 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8acdd87c-f3c6-46b5-922b-38b7fa56e97e tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:21.018109 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-8acdd87c-f3c6-46b5-922b-38b7fa56e97e tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f5bc1f2209f4eb186d80e385b67c774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '65d5bcfa02af4e8284ba1342197e2860', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:17:21.034196 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-8acdd87c-f3c6-46b5-922b-38b7fa56e97e tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] 10.222.0.16 "GET /compute/v2.1/servers/aa688a7e-0b28-4dfd-a159-8e19fd62ee02" status: 200 len: 1314 microversion: 2.1 time: 0.576180 Jul 16 21:17:21.036945 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1041/2164] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:17:20 2022] GET /compute/v2.1/servers/aa688a7e-0b28-4dfd-a159-8e19fd62ee02 => generated 1314 bytes in 579 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:17:21.044140 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-215ca331-89d4-47c9-8147-141cb6c8446b tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Calling method '.version_select at 0x7fdda2b380d0>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:17:21.116353 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-215ca331-89d4-47c9-8147-141cb6c8446b tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] 10.222.0.16 "GET /compute/v2.1/images/245dff72-cf22-471e-9dc0-0e7271c7ed43" status: 200 len: 1455 microversion: 2.1 time: 0.078001 Jul 16 21:17:21.117203 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1042/2165] 10.222.0.16 () {58 vars in 1232 bytes} [Sat Jul 16 21:17:21 2022] GET /compute/v2.1/images/245dff72-cf22-471e-9dc0-0e7271c7ed43 => generated 1455 bytes in 80 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:17:21.131653 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-3893038a-3161-4a3b-9519-5ac8fc33b48f tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:17:21.132294 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-3893038a-3161-4a3b-9519-5ac8fc33b48f tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] [instance: 0bcfc499-2e0f-48b6-994a-e0cb74ac7b4f] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:17:21.140421 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3893038a-3161-4a3b-9519-5ac8fc33b48f tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:21.141138 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3893038a-3161-4a3b-9519-5ac8fc33b48f tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:21.142280 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3893038a-3161-4a3b-9519-5ac8fc33b48f tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:21.142860 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3893038a-3161-4a3b-9519-5ac8fc33b48f tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:21.164722 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-3893038a-3161-4a3b-9519-5ac8fc33b48f tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] HTTP exception thrown: Instance 0bcfc499-2e0f-48b6-994a-e0cb74ac7b4f could not be found. Jul 16 21:17:21.166945 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-3893038a-3161-4a3b-9519-5ac8fc33b48f tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Request method failure captured: Jul 16 21:17:21.166945 ncdv-master-1 devstack@n-api.service[138681]: request: GET /compute/v2.1/servers/0bcfc499-2e0f-48b6-994a-e0cb74ac7b4f HTTP/1.1 Jul 16 21:17:21.166945 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:17:21.166945 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:17:21.166945 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:17:21.166945 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 21:17:21.166945 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:17:21.166945 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:17:21.166945 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:17:21.166945 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:17:21.166945 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0yqk_pXk-wKum3b8cCuT2pEaU9dBudX_cAtBJ9pskOXJYY-ofAku47SZoD93qXsTHLBEazC1J-w1Qcw50R8VujDLzTf9h8xBm8O8X1husFf7xOBV98tr4pV6N0QotOfB-rAq_gHVRYXqCGhTgE068IUohjigJzKTCWdzNDvOifq8IvA Jul 16 21:17:21.166945 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:17:21.166945 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:17:21.166945 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:17:21.166945 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:17:21.166945 ncdv-master-1 devstack@n-api.service[138681]: X-Openstack-Nova-Api-Version: 2.47 Jul 16 21:17:21.166945 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:17:21.166945 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:17:21.166945 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: 9f5f03e09936489fa1505188cefe3445 Jul 16 21:17:21.166945 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-ServerShowV247Test-1709646099 Jul 16 21:17:21.166945 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:17:21.166945 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:17:21.166945 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/9f5f03e09936489fa1505188cefe3445"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/9f5f03e09936489fa1505188cefe3445"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_9f5f03e09936489fa1505188cefe3445"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/9f5f03e09936489fa1505188cefe3445"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:17:21.166945 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-ServerShowV247Test-1709646099 Jul 16 21:17:21.166945 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: 9f5f03e09936489fa1505188cefe3445 Jul 16 21:17:21.166945 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-ServerShowV247Test-1709646099 Jul 16 21:17:21.166945 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-ServerShowV247Test-1709646099-project Jul 16 21:17:21.166945 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:17:21.166945 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:17:21.166945 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: 015ac1bbb2df487e99e5bddc60ee6923 Jul 16 21:17:21.166945 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-ServerShowV247Test-1709646099-project Jul 16 21:17:21.166945 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:17:21.166945 ncdv-master-1 devstack@n-api.service[138681]: exception: Instance 0bcfc499-2e0f-48b6-994a-e0cb74ac7b4f could not be found. Jul 16 21:17:21.166945 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': '0bcfc499-2e0f-48b6-994a-e0cb74ac7b4f'} Jul 16 21:17:21.166945 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:17:21.166945 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:17:21.166945 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:17:21.166945 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:17:21.166945 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:17:21.166945 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:17:21.166945 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:17:21.166945 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:17:21.166945 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:17:21.166945 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:17:21.166945 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 0bcfc499-2e0f-48b6-994a-e0cb74ac7b4f could not be found. Jul 16 21:17:21.166945 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:17:21.166945 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:17:21.166945 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:17:21.166945 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:17:21.166945 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:17:21.166945 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:17:21.166945 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:17:21.166945 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:17:21.166945 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:17:21.166945 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:17:21.166945 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:17:21.166945 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:17:21.166945 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:17:21.166945 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:17:21.166945 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:17:21.166945 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:17:21.166945 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 0bcfc499-2e0f-48b6-994a-e0cb74ac7b4f could not be found. Jul 16 21:17:21.166945 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:17:21.166945 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:17:21.166945 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:17:21.171508 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:17:21.171508 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:17:21.171508 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:17:21.171508 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:17:21.171508 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:17:21.171508 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 0bcfc499-2e0f-48b6-994a-e0cb74ac7b4f could not be found. Jul 16 21:17:21.171508 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:17:21.174592 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-3893038a-3161-4a3b-9519-5ac8fc33b48f tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] Returning 404 to user: Instance 0bcfc499-2e0f-48b6-994a-e0cb74ac7b4f could not be found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:17:21.176419 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-3893038a-3161-4a3b-9519-5ac8fc33b48f tempest-ServerShowV247Test-1709646099 tempest-ServerShowV247Test-1709646099-project] 10.222.0.16 "GET /compute/v2.1/servers/0bcfc499-2e0f-48b6-994a-e0cb74ac7b4f" status: 404 len: 111 microversion: 2.47 time: 0.054471 Jul 16 21:17:21.178366 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1043/2166] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:17:21 2022] GET /compute/v2.1/servers/0bcfc499-2e0f-48b6-994a-e0cb74ac7b4f => generated 111 bytes in 60 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Jul 16 21:17:21.185997 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-849a4f5a-c745-4f79-aa01-bfd5165e15e3 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Action: 'update', calling method: >, body: {"server": {"OS-DCF:diskConfig": "AUTO"}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:17:21.187741 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-849a4f5a-c745-4f79-aa01-bfd5165e15e3 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] [instance: 7e4b142d-f489-4220-adc3-7b348dc14ce9] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:17:21.194321 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_api:SELECT=96,nova_api:INSERT=3,nova_api:UPDATE=1 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:17:21.197221 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-849a4f5a-c745-4f79-aa01-bfd5165e15e3 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:21.197921 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-849a4f5a-c745-4f79-aa01-bfd5165e15e3 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:21.198959 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-849a4f5a-c745-4f79-aa01-bfd5165e15e3 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:21.199629 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-849a4f5a-c745-4f79-aa01-bfd5165e15e3 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:21.278725 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=96,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:17:21.401583 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-849a4f5a-c745-4f79-aa01-bfd5165e15e3 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] 10.222.0.16 "PUT /compute/v2.1/servers/7e4b142d-f489-4220-adc3-7b348dc14ce9" status: 200 len: 1078 microversion: 2.1 time: 0.221630 Jul 16 21:17:21.413530 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1044/2167] 10.222.0.16 () {60 vars in 1255 bytes} [Sat Jul 16 21:17:21 2022] PUT /compute/v2.1/servers/7e4b142d-f489-4220-adc3-7b348dc14ce9 => generated 1078 bytes in 232 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:17:21.434376 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-76f2a6b5-2a2c-42f3-a79e-cdbcce24a4e9 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Calling method '.version_select at 0x7fdda2ae6d30>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:17:21.512687 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-76f2a6b5-2a2c-42f3-a79e-cdbcce24a4e9 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] 10.222.0.16 "GET /compute/v2.1/images/211719d4-d58e-474c-ade9-8ffb307248f3" status: 200 len: 1414 microversion: 2.1 time: 0.097029 Jul 16 21:17:21.514693 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1045/2168] 10.222.0.16 () {58 vars in 1232 bytes} [Sat Jul 16 21:17:21 2022] GET /compute/v2.1/images/211719d4-d58e-474c-ade9-8ffb307248f3 => generated 1414 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:17:21.525219 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-31696972-d281-40bc-b574-e196951fff08 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:17:21.525219 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-31696972-d281-40bc-b574-e196951fff08 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] [instance: 7e4b142d-f489-4220-adc3-7b348dc14ce9] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:17:21.529600 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-31696972-d281-40bc-b574-e196951fff08 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:21.530298 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-31696972-d281-40bc-b574-e196951fff08 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:21.531284 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-31696972-d281-40bc-b574-e196951fff08 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:21.531874 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-31696972-d281-40bc-b574-e196951fff08 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:21.632561 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-94f54d87-b6fc-4694-96f6-49aadd5ed5e9 tempest-QuotaClassesAdminTestJSON-1596052195 tempest-QuotaClassesAdminTestJSON-1596052195-project-admin] 10.222.0.16 "PUT /compute/v2.1/os-quota-class-sets/default" status: 200 len: 286 microversion: 2.1 time: 1.683401 Jul 16 21:17:21.636673 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1124/2169] 10.222.0.16 () {60 vars in 1205 bytes} [Sat Jul 16 21:17:19 2022] PUT /compute/v2.1/os-quota-class-sets/default => generated 286 bytes in 1685 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:17:21.770875 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-31696972-d281-40bc-b574-e196951fff08 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd4144b5388b46a989a41a2f556bb508', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a90d4bcd25ff4bf68bdc194180fbbe09', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:17:21.780208 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-31696972-d281-40bc-b574-e196951fff08 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] 10.222.0.16 "GET /compute/v2.1/servers/7e4b142d-f489-4220-adc3-7b348dc14ce9" status: 200 len: 1504 microversion: 2.1 time: 0.264239 Jul 16 21:17:21.780796 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1046/2170] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:17:21 2022] GET /compute/v2.1/servers/7e4b142d-f489-4220-adc3-7b348dc14ce9 => generated 1504 bytes in 265 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:17:21.805998 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-f4784248-ceda-4757-b272-18d24eb8ad26 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:17:21.806877 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-f4784248-ceda-4757-b272-18d24eb8ad26 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] [instance: 7e4b142d-f489-4220-adc3-7b348dc14ce9] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:17:21.829297 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f4784248-ceda-4757-b272-18d24eb8ad26 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:21.829297 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f4784248-ceda-4757-b272-18d24eb8ad26 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:21.829297 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f4784248-ceda-4757-b272-18d24eb8ad26 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:21.829297 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f4784248-ceda-4757-b272-18d24eb8ad26 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:22.054802 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-79c686e0-d095-495e-b5ba-f333c90a5519 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:17:22.055711 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-79c686e0-d095-495e-b5ba-f333c90a5519 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] [instance: aa688a7e-0b28-4dfd-a159-8e19fd62ee02] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:17:22.067427 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-79c686e0-d095-495e-b5ba-f333c90a5519 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:22.069181 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-79c686e0-d095-495e-b5ba-f333c90a5519 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:22.070017 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-79c686e0-d095-495e-b5ba-f333c90a5519 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:22.073514 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-79c686e0-d095-495e-b5ba-f333c90a5519 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:23.727922 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-f4784248-ceda-4757-b272-18d24eb8ad26 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd4144b5388b46a989a41a2f556bb508', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a90d4bcd25ff4bf68bdc194180fbbe09', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:17:23.758072 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-f4784248-ceda-4757-b272-18d24eb8ad26 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] 10.222.0.16 "GET /compute/v2.1/servers/7e4b142d-f489-4220-adc3-7b348dc14ce9" status: 200 len: 1504 microversion: 2.1 time: 1.963347 Jul 16 21:17:23.759129 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1125/2171] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:17:21 2022] GET /compute/v2.1/servers/7e4b142d-f489-4220-adc3-7b348dc14ce9 => generated 1504 bytes in 1967 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:17:23.788271 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-b5249db2-48b8-4672-9d7d-67bc8c36e7e9 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Calling method '.version_select at 0x7fdda2b3d160>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:17:23.805590 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-79c686e0-d095-495e-b5ba-f333c90a5519 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f5bc1f2209f4eb186d80e385b67c774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '65d5bcfa02af4e8284ba1342197e2860', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:17:23.817411 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-79c686e0-d095-495e-b5ba-f333c90a5519 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] 10.222.0.16 "GET /compute/v2.1/servers/aa688a7e-0b28-4dfd-a159-8e19fd62ee02" status: 200 len: 1314 microversion: 2.1 time: 1.767683 Jul 16 21:17:23.824145 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1047/2172] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:17:22 2022] GET /compute/v2.1/servers/aa688a7e-0b28-4dfd-a159-8e19fd62ee02 => generated 1314 bytes in 1775 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:17:23.839720 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-c8e0047b-53d3-41b5-b314-65d83a6df59d tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Calling method '.version_select at 0x7fdda2b8fe50>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:17:31.631531 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell0:SELECT=29,nova_cell0:UPDATE=7 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:17:31.811985 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_api:SELECT=49,nova_api:UPDATE=24,nova_api:INSERT=7 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:17:33.735355 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=75,nova_cell1:INSERT=2,nova_cell1:UPDATE=1 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:17:34.496914 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_api:SELECT=5 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:17:34.773194 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:UPDATE=1,nova_cell1:SELECT=13 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:17:37.843065 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-b5249db2-48b8-4672-9d7d-67bc8c36e7e9 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] 10.222.0.16 "GET /compute/v2.1/images/245dff72-cf22-471e-9dc0-0e7271c7ed43" status: 200 len: 1464 microversion: 2.1 time: 14.080094 Jul 16 21:17:37.850085 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1126/2173] 10.222.0.16 () {58 vars in 1232 bytes} [Sat Jul 16 21:17:23 2022] GET /compute/v2.1/images/245dff72-cf22-471e-9dc0-0e7271c7ed43 => generated 1464 bytes in 14082 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:17:37.852571 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-4d76ba2c-d147-4244-9eec-1726f0cb7856 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Action: 'action', calling method: >, body: {"rebuild": {"imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "OS-DCF:diskConfig": "MANUAL"}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:17:37.866683 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-c8e0047b-53d3-41b5-b314-65d83a6df59d tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] 10.222.0.16 "GET /compute/v2.1/images/211719d4-d58e-474c-ade9-8ffb307248f3" status: 200 len: 1414 microversion: 2.1 time: 14.040978 Jul 16 21:17:37.867729 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1048/2174] 10.222.0.16 () {58 vars in 1232 bytes} [Sat Jul 16 21:17:23 2022] GET /compute/v2.1/images/211719d4-d58e-474c-ade9-8ffb307248f3 => generated 1414 bytes in 14043 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:17:37.875675 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-4d76ba2c-d147-4244-9eec-1726f0cb7856 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] [instance: 7e4b142d-f489-4220-adc3-7b348dc14ce9] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:17:37.888655 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4d76ba2c-d147-4244-9eec-1726f0cb7856 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:37.889176 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4d76ba2c-d147-4244-9eec-1726f0cb7856 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:37.889975 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4d76ba2c-d147-4244-9eec-1726f0cb7856 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:37.890416 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4d76ba2c-d147-4244-9eec-1726f0cb7856 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:37.891364 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-506864e6-afd8-4786-a71b-eac567db66fe req-603bcc71-703f-4f8d-8452-49717f50690c service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "e7598b4a-f70c-4186-88d1-6fd0057f8656", "name": "network-vif-plugged", "status": "completed", "tag": "5e72e822-7cdd-445c-8d25-dc81877dc718"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:17:37.933386 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-506864e6-afd8-4786-a71b-eac567db66fe req-603bcc71-703f-4f8d-8452-49717f50690c service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:37.934072 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-506864e6-afd8-4786-a71b-eac567db66fe req-603bcc71-703f-4f8d-8452-49717f50690c service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:37.946399 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-506864e6-afd8-4786-a71b-eac567db66fe req-603bcc71-703f-4f8d-8452-49717f50690c service nova] Creating event network-vif-plugged:5e72e822-7cdd-445c-8d25-dc81877dc718 for instance e7598b4a-f70c-4186-88d1-6fd0057f8656 on nch1-master-1 Jul 16 21:17:37.952622 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-506864e6-afd8-4786-a71b-eac567db66fe req-603bcc71-703f-4f8d-8452-49717f50690c service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.083091 Jul 16 21:17:37.953833 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1049/2175] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:17:37 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:17:38.036984 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-4d76ba2c-d147-4244-9eec-1726f0cb7856 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Flavor limits 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 16 21:17:38.037692 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-4d76ba2c-d147-4244-9eec-1726f0cb7856 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Image limits 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 16 21:17:38.038346 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-4d76ba2c-d147-4244-9eec-1726f0cb7856 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Flavor pref 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 16 21:17:38.038913 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-4d76ba2c-d147-4244-9eec-1726f0cb7856 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Image pref 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 16 21:17:38.041060 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-4d76ba2c-d147-4244-9eec-1726f0cb7856 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 16 21:17:38.219839 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-4bd4a07e-9567-407c-b184-e0da47807bc7 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Action: 'create', calling method: .version_select at 0x7fdda2ab3c10>, body: {"security_group": {"name": "tempest-SecurityGroupsTestJSON-securitygroup-1978190092", "description": "tempest-description-1302632424"}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:17:38.337804 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-4d76ba2c-d147-4244-9eec-1726f0cb7856 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] [instance: 7e4b142d-f489-4220-adc3-7b348dc14ce9] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:17:38.345944 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4d76ba2c-d147-4244-9eec-1726f0cb7856 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:38.346695 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4d76ba2c-d147-4244-9eec-1726f0cb7856 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:38.347768 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4d76ba2c-d147-4244-9eec-1726f0cb7856 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:38.348451 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4d76ba2c-d147-4244-9eec-1726f0cb7856 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:38.399327 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-4d76ba2c-d147-4244-9eec-1726f0cb7856 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] 10.222.0.16 "POST /compute/v2.1/servers/7e4b142d-f489-4220-adc3-7b348dc14ce9/action" status: 202 len: 1110 microversion: 2.1 time: 0.546501 Jul 16 21:17:38.400404 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1127/2176] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 21:17:37 2022] POST /compute/v2.1/servers/7e4b142d-f489-4220-adc3-7b348dc14ce9/action => generated 1110 bytes in 555 msecs (HTTP/1.1 202) 10 headers in 452 bytes (1 switches on core 0) Jul 16 21:17:38.409858 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-c79e6884-932a-4f38-8797-09273a7cc9ab tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:17:38.414426 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-c79e6884-932a-4f38-8797-09273a7cc9ab tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] [instance: aa688a7e-0b28-4dfd-a159-8e19fd62ee02] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:17:38.427392 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c79e6884-932a-4f38-8797-09273a7cc9ab tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:38.428059 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c79e6884-932a-4f38-8797-09273a7cc9ab tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:38.435764 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c79e6884-932a-4f38-8797-09273a7cc9ab tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:38.437386 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c79e6884-932a-4f38-8797-09273a7cc9ab tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:38.821515 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-c79e6884-932a-4f38-8797-09273a7cc9ab tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f5bc1f2209f4eb186d80e385b67c774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '65d5bcfa02af4e8284ba1342197e2860', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:17:38.842631 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-c79e6884-932a-4f38-8797-09273a7cc9ab tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] 10.222.0.16 "GET /compute/v2.1/servers/aa688a7e-0b28-4dfd-a159-8e19fd62ee02" status: 200 len: 1314 microversion: 2.1 time: 0.440301 Jul 16 21:17:38.843891 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1128/2177] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:17:38 2022] GET /compute/v2.1/servers/aa688a7e-0b28-4dfd-a159-8e19fd62ee02 => generated 1314 bytes in 443 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:17:39.086862 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-3c754cf0-0eea-4db0-81dc-ee852f49153f tempest-QuotasAdminTestV236-2041262956 tempest-QuotasAdminTestV236-2041262956-project-admin] Action: 'update', calling method: .version_select at 0x7fdd9c6b11f0>, body: {"quota_set": {"ram": "5120"}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:17:39.161676 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3c754cf0-0eea-4db0-81dc-ee852f49153f tempest-QuotasAdminTestV236-2041262956 tempest-QuotasAdminTestV236-2041262956-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:39.162131 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3c754cf0-0eea-4db0-81dc-ee852f49153f tempest-QuotasAdminTestV236-2041262956 tempest-QuotasAdminTestV236-2041262956-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:39.162934 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3c754cf0-0eea-4db0-81dc-ee852f49153f tempest-QuotasAdminTestV236-2041262956 tempest-QuotasAdminTestV236-2041262956-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:39.163405 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3c754cf0-0eea-4db0-81dc-ee852f49153f tempest-QuotasAdminTestV236-2041262956 tempest-QuotasAdminTestV236-2041262956-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:39.453454 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-4bd4a07e-9567-407c-b184-e0da47807bc7 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] 10.222.0.16 "POST /compute/v2.1/os-security-groups" status: 200 len: 244 microversion: 2.1 time: 1.498220 Jul 16 21:17:39.454150 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1050/2178] 10.222.0.16 () {60 vars in 1179 bytes} [Sat Jul 16 21:17:37 2022] POST /compute/v2.1/os-security-groups => generated 244 bytes in 1500 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:17:39.459768 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-37057a2c-8cce-48e0-9c7b-455264ac2bb0 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Calling method '.version_select at 0x7fdda2b8f670>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:17:39.499509 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-3c754cf0-0eea-4db0-81dc-ee852f49153f tempest-QuotasAdminTestV236-2041262956 tempest-QuotasAdminTestV236-2041262956-project-admin] 10.222.0.16 "PUT /compute/v2.1/os-quota-sets/13e0f52a884440db827dba9fe97e8fef" status: 200 len: 240 microversion: 2.36 time: 0.653583 Jul 16 21:17:39.506756 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1129/2179] 10.222.0.16 () {62 vars in 1302 bytes} [Sat Jul 16 21:17:38 2022] PUT /compute/v2.1/os-quota-sets/13e0f52a884440db827dba9fe97e8fef => generated 240 bytes in 661 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 16 21:17:39.514280 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-d64f438e-b8ec-4122-b8a7-4f3afb61ebdd tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:17:39.515187 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-d64f438e-b8ec-4122-b8a7-4f3afb61ebdd tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] [instance: 7e4b142d-f489-4220-adc3-7b348dc14ce9] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:17:39.556319 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d64f438e-b8ec-4122-b8a7-4f3afb61ebdd tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:39.556779 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d64f438e-b8ec-4122-b8a7-4f3afb61ebdd tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:39.557567 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d64f438e-b8ec-4122-b8a7-4f3afb61ebdd tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:39.558008 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d64f438e-b8ec-4122-b8a7-4f3afb61ebdd tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:39.598096 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-37057a2c-8cce-48e0-9c7b-455264ac2bb0 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] 10.222.0.16 "GET /compute/v2.1/images/245dff72-cf22-471e-9dc0-0e7271c7ed43" status: 200 len: 1464 microversion: 2.1 time: 0.142470 Jul 16 21:17:39.599358 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1051/2180] 10.222.0.16 () {58 vars in 1232 bytes} [Sat Jul 16 21:17:39 2022] GET /compute/v2.1/images/245dff72-cf22-471e-9dc0-0e7271c7ed43 => generated 1464 bytes in 144 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:17:39.611264 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-82936b76-9e2b-411e-9c4a-d00119c4c92f tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Calling method '.version_select at 0x7fdda2ac0af0>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:17:39.674590 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-82936b76-9e2b-411e-9c4a-d00119c4c92f tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] 10.222.0.16 "GET /compute/v2.1/images/211719d4-d58e-474c-ade9-8ffb307248f3" status: 200 len: 1414 microversion: 2.1 time: 0.072539 Jul 16 21:17:39.675532 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1052/2181] 10.222.0.16 () {58 vars in 1232 bytes} [Sat Jul 16 21:17:39 2022] GET /compute/v2.1/images/211719d4-d58e-474c-ade9-8ffb307248f3 => generated 1414 bytes in 75 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:17:39.681077 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-be2321c6-8b5a-4135-93fb-d20771d823a4 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Action: 'create', calling method: .version_select at 0x7fdd9cdca0d0>, body: {"security_group": {"name": "tempest-SecurityGroupsTestJSON-securitygroup-2071679724", "description": "tempest-description-1957715051"}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:17:39.907318 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-d64f438e-b8ec-4122-b8a7-4f3afb61ebdd tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd4144b5388b46a989a41a2f556bb508', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a90d4bcd25ff4bf68bdc194180fbbe09', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:17:39.919402 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-d64f438e-b8ec-4122-b8a7-4f3afb61ebdd tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] 10.222.0.16 "GET /compute/v2.1/servers/7e4b142d-f489-4220-adc3-7b348dc14ce9" status: 200 len: 1515 microversion: 2.1 time: 0.407790 Jul 16 21:17:39.923099 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1130/2182] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:17:39 2022] GET /compute/v2.1/servers/7e4b142d-f489-4220-adc3-7b348dc14ce9 => generated 1515 bytes in 415 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:17:39.929862 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-4fb0d67a-9335-4676-bc33-27239d277c9b tempest-QuotasAdminTestV236-2041262956 tempest-QuotasAdminTestV236-2041262956-project-admin] Calling method '.version_select at 0x7fdd9c6b60d0>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:17:40.228207 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4fb0d67a-9335-4676-bc33-27239d277c9b tempest-QuotasAdminTestV236-2041262956 tempest-QuotasAdminTestV236-2041262956-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:40.228669 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4fb0d67a-9335-4676-bc33-27239d277c9b tempest-QuotasAdminTestV236-2041262956 tempest-QuotasAdminTestV236-2041262956-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:40.229474 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4fb0d67a-9335-4676-bc33-27239d277c9b tempest-QuotasAdminTestV236-2041262956 tempest-QuotasAdminTestV236-2041262956-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:40.230294 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4fb0d67a-9335-4676-bc33-27239d277c9b tempest-QuotasAdminTestV236-2041262956 tempest-QuotasAdminTestV236-2041262956-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:40.241937 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-be2321c6-8b5a-4135-93fb-d20771d823a4 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] 10.222.0.16 "POST /compute/v2.1/os-security-groups" status: 200 len: 244 microversion: 2.1 time: 0.560718 Jul 16 21:17:40.241937 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1053/2183] 10.222.0.16 () {60 vars in 1179 bytes} [Sat Jul 16 21:17:39 2022] POST /compute/v2.1/os-security-groups => generated 244 bytes in 562 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:17:40.243239 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-a4f91b3f-7afd-4752-9d86-94ab5f98c0f5 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:17:40.243997 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-a4f91b3f-7afd-4752-9d86-94ab5f98c0f5 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] [instance: b9a4910a-f117-4834-b699-498c6431b9bb] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:17:40.256454 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a4f91b3f-7afd-4752-9d86-94ab5f98c0f5 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:40.257097 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a4f91b3f-7afd-4752-9d86-94ab5f98c0f5 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:40.258068 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a4f91b3f-7afd-4752-9d86-94ab5f98c0f5 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:40.260006 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a4f91b3f-7afd-4752-9d86-94ab5f98c0f5 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:40.289896 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-4fb0d67a-9335-4676-bc33-27239d277c9b tempest-QuotasAdminTestV236-2041262956 tempest-QuotasAdminTestV236-2041262956-project-admin] 10.222.0.16 "GET /compute/v2.1/os-quota-sets/13e0f52a884440db827dba9fe97e8fef/detail" status: 200 len: 672 microversion: 2.36 time: 0.365130 Jul 16 21:17:40.291072 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1131/2184] 10.222.0.16 () {60 vars in 1303 bytes} [Sat Jul 16 21:17:39 2022] GET /compute/v2.1/os-quota-sets/13e0f52a884440db827dba9fe97e8fef/detail => generated 672 bytes in 368 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 16 21:17:40.319824 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-db550ebd-ebd0-4f68-9847-12d9beb1d1b5 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:17:40.321622 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-db550ebd-ebd0-4f68-9847-12d9beb1d1b5 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] [instance: aa688a7e-0b28-4dfd-a159-8e19fd62ee02] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:17:40.343975 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-db550ebd-ebd0-4f68-9847-12d9beb1d1b5 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:40.345650 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-db550ebd-ebd0-4f68-9847-12d9beb1d1b5 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:40.352255 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-db550ebd-ebd0-4f68-9847-12d9beb1d1b5 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:40.357231 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-db550ebd-ebd0-4f68-9847-12d9beb1d1b5 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:40.595852 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-a4f91b3f-7afd-4752-9d86-94ab5f98c0f5 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '71e9304fc50447d6aac99703b5b6e4e1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e84496d78d4481e9d7648128868bca2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:17:40.607882 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-a4f91b3f-7afd-4752-9d86-94ab5f98c0f5 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] 10.222.0.16 "GET /compute/v2.1/servers/b9a4910a-f117-4834-b699-498c6431b9bb" status: 200 len: 1466 microversion: 2.1 time: 0.368340 Jul 16 21:17:40.611367 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1054/2185] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:17:40 2022] GET /compute/v2.1/servers/b9a4910a-f117-4834-b699-498c6431b9bb => generated 1466 bytes in 372 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:17:40.626195 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-e4fd47cb-26e0-45f7-a2e0-2bb0defe9407 req-f4f87897-00a1-417a-b0a7-583d1b421e8f service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "aa688a7e-0b28-4dfd-a159-8e19fd62ee02", "name": "network-vif-plugged", "status": "completed", "tag": "0eaf070b-3b01-4123-bf36-5947ef05d97a"}, {"server_uuid": "f03a2a83-e007-444a-b46b-70eb46bb242b", "name": "network-vif-plugged", "status": "completed", "tag": "1df74ef4-7ed5-4498-8310-01458a7e541a"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:17:40.647382 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-e4fd47cb-26e0-45f7-a2e0-2bb0defe9407 req-f4f87897-00a1-417a-b0a7-583d1b421e8f service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:40.648055 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-e4fd47cb-26e0-45f7-a2e0-2bb0defe9407 req-f4f87897-00a1-417a-b0a7-583d1b421e8f service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:40.669510 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-e4fd47cb-26e0-45f7-a2e0-2bb0defe9407 req-f4f87897-00a1-417a-b0a7-583d1b421e8f service nova] Creating event network-vif-plugged:0eaf070b-3b01-4123-bf36-5947ef05d97a for instance aa688a7e-0b28-4dfd-a159-8e19fd62ee02 on nch1-master-1 Jul 16 21:17:40.670436 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-e4fd47cb-26e0-45f7-a2e0-2bb0defe9407 req-f4f87897-00a1-417a-b0a7-583d1b421e8f service nova] Creating event network-vif-plugged:1df74ef4-7ed5-4498-8310-01458a7e541a for instance f03a2a83-e007-444a-b46b-70eb46bb242b on nch1-master-1 Jul 16 21:17:40.678070 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-e4fd47cb-26e0-45f7-a2e0-2bb0defe9407 req-f4f87897-00a1-417a-b0a7-583d1b421e8f service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 354 microversion: 2.1 time: 0.065028 Jul 16 21:17:40.681838 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1055/2186] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:17:40 2022] POST /compute/v2.1/os-server-external-events => generated 354 bytes in 67 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:17:40.687136 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-c45032d7-2ed8-4381-937f-2a6c9af7bff0 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-SecurityGroupsTestJSON-server-1675315678", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452", "networks": [{"uuid": "8cd71feb-4bea-45d0-9b6a-2960b1a03de1"}]}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:17:40.707290 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-db550ebd-ebd0-4f68-9847-12d9beb1d1b5 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f5bc1f2209f4eb186d80e385b67c774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '65d5bcfa02af4e8284ba1342197e2860', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:17:40.728280 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-db550ebd-ebd0-4f68-9847-12d9beb1d1b5 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] 10.222.0.16 "GET /compute/v2.1/servers/aa688a7e-0b28-4dfd-a159-8e19fd62ee02" status: 200 len: 1314 microversion: 2.1 time: 0.427211 Jul 16 21:17:40.728946 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1132/2187] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:17:40 2022] GET /compute/v2.1/servers/aa688a7e-0b28-4dfd-a159-8e19fd62ee02 => generated 1314 bytes in 437 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:17:40.735652 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-712c3c00-3739-4acc-b122-97f59b849d43 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Action: 'action', calling method: >, body: {"createImage": {"name": "tempest-ListImageFiltersTestJSON-image-145359419"}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:17:40.737021 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-712c3c00-3739-4acc-b122-97f59b849d43 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] [instance: 6cb4e66a-eba2-42b7-91e8-09ee7b2a5a4f] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:17:40.744344 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-712c3c00-3739-4acc-b122-97f59b849d43 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:40.750626 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-712c3c00-3739-4acc-b122-97f59b849d43 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:40.750626 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-712c3c00-3739-4acc-b122-97f59b849d43 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:40.750626 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-712c3c00-3739-4acc-b122-97f59b849d43 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:40.782473 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.network.neutron [None req-c45032d7-2ed8-4381-937f-2a6c9af7bff0 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] validate_networks() for [('8cd71feb-4bea-45d0-9b6a-2960b1a03de1', None, None, None, None, None)] {{(pid=138681) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 16 21:17:41.543622 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-712c3c00-3739-4acc-b122-97f59b849d43 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] 10.222.0.16 "POST /compute/v2.1/servers/6cb4e66a-eba2-42b7-91e8-09ee7b2a5a4f/action" status: 202 len: 0 microversion: 2.1 time: 0.813044 Jul 16 21:17:41.549893 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1133/2188] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 21:17:40 2022] POST /compute/v2.1/servers/6cb4e66a-eba2-42b7-91e8-09ee7b2a5a4f/action => generated 0 bytes in 820 msecs (HTTP/1.1 202) 10 headers in 450 bytes (1 switches on core 0) Jul 16 21:17:41.558117 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-c5f72b36-5b45-4ae4-a400-3ff8c3d0a97b tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Calling method '.version_select at 0x7fdd9c6a5e50>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:17:41.638109 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-c45032d7-2ed8-4381-937f-2a6c9af7bff0 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Flavor limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 16 21:17:41.638109 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-c45032d7-2ed8-4381-937f-2a6c9af7bff0 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Image limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 16 21:17:41.638109 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-c45032d7-2ed8-4381-937f-2a6c9af7bff0 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Flavor pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 16 21:17:41.638109 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-c45032d7-2ed8-4381-937f-2a6c9af7bff0 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Image pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 16 21:17:41.639290 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-c45032d7-2ed8-4381-937f-2a6c9af7bff0 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 16 21:17:41.644447 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-c5f72b36-5b45-4ae4-a400-3ff8c3d0a97b tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] 10.222.0.16 "GET /compute/v2.1/images/211719d4-d58e-474c-ade9-8ffb307248f3" status: 200 len: 1423 microversion: 2.1 time: 0.092866 Jul 16 21:17:41.645046 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1134/2189] 10.222.0.16 () {58 vars in 1232 bytes} [Sat Jul 16 21:17:41 2022] GET /compute/v2.1/images/211719d4-d58e-474c-ade9-8ffb307248f3 => generated 1423 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:17:41.647902 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c45032d7-2ed8-4381-937f-2a6c9af7bff0 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:41.648358 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c45032d7-2ed8-4381-937f-2a6c9af7bff0 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:41.649163 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c45032d7-2ed8-4381-937f-2a6c9af7bff0 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:41.649578 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-1c5eab1f-2848-45b9-a872-8fd01ae84c96 tempest-QuotasAdminTestV236-2041262956 tempest-QuotasAdminTestV236-2041262956-project-admin] Action: 'update', calling method: .version_select at 0x7fdda2a15ca0>, body: {"quota_set": {"ram": "2048"}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:17:41.649698 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c45032d7-2ed8-4381-937f-2a6c9af7bff0 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:41.675859 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-c45032d7-2ed8-4381-937f-2a6c9af7bff0 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Getting quotas for project e353258f9c3048ef84683294e2b8579d. Resources: {'ram', 'instances', 'cores'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:17:41.685910 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-c45032d7-2ed8-4381-937f-2a6c9af7bff0 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Getting quotas for user d0008fffad8e4708b2c01e03da92c47d and project e353258f9c3048ef84683294e2b8579d. Resources: {'ram', 'instances', 'cores'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:17:41.702832 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-c45032d7-2ed8-4381-937f-2a6c9af7bff0 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Going to run 1 instances... {{(pid=138681) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 16 21:17:41.705481 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-c45032d7-2ed8-4381-937f-2a6c9af7bff0 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] [instance: b8565a82-250b-4f64-9d90-98b368e4bd57] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138681) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 16 21:17:41.748961 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1c5eab1f-2848-45b9-a872-8fd01ae84c96 tempest-QuotasAdminTestV236-2041262956 tempest-QuotasAdminTestV236-2041262956-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:41.749849 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1c5eab1f-2848-45b9-a872-8fd01ae84c96 tempest-QuotasAdminTestV236-2041262956 tempest-QuotasAdminTestV236-2041262956-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:41.760235 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1c5eab1f-2848-45b9-a872-8fd01ae84c96 tempest-QuotasAdminTestV236-2041262956 tempest-QuotasAdminTestV236-2041262956-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:41.760235 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1c5eab1f-2848-45b9-a872-8fd01ae84c96 tempest-QuotasAdminTestV236-2041262956 tempest-QuotasAdminTestV236-2041262956-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:41.793109 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1c5eab1f-2848-45b9-a872-8fd01ae84c96 tempest-QuotasAdminTestV236-2041262956 tempest-QuotasAdminTestV236-2041262956-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:41.793674 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1c5eab1f-2848-45b9-a872-8fd01ae84c96 tempest-QuotasAdminTestV236-2041262956 tempest-QuotasAdminTestV236-2041262956-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:41.795790 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1c5eab1f-2848-45b9-a872-8fd01ae84c96 tempest-QuotasAdminTestV236-2041262956 tempest-QuotasAdminTestV236-2041262956-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:41.796247 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1c5eab1f-2848-45b9-a872-8fd01ae84c96 tempest-QuotasAdminTestV236-2041262956 tempest-QuotasAdminTestV236-2041262956-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:41.944933 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-c45032d7-2ed8-4381-937f-2a6c9af7bff0 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 1.264384 Jul 16 21:17:41.945770 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1056/2190] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 21:17:40 2022] POST /compute/v2.1/servers => generated 380 bytes in 1266 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 16 21:17:41.950650 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-bd0e51fc-21c2-4fb8-9efc-3f2a7b27d5f5 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:17:41.951323 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-bd0e51fc-21c2-4fb8-9efc-3f2a7b27d5f5 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] [instance: 7e4b142d-f489-4220-adc3-7b348dc14ce9] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:17:41.958471 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-bd0e51fc-21c2-4fb8-9efc-3f2a7b27d5f5 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:41.958936 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-bd0e51fc-21c2-4fb8-9efc-3f2a7b27d5f5 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:41.959758 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-bd0e51fc-21c2-4fb8-9efc-3f2a7b27d5f5 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:41.960377 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-bd0e51fc-21c2-4fb8-9efc-3f2a7b27d5f5 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:41.990191 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-1c5eab1f-2848-45b9-a872-8fd01ae84c96 tempest-QuotasAdminTestV236-2041262956 tempest-QuotasAdminTestV236-2041262956-project-admin] 10.222.0.16 "PUT /compute/v2.1/os-quota-sets/13e0f52a884440db827dba9fe97e8fef?user_id=435411be15364e74a547015105a29c53" status: 200 len: 240 microversion: 2.36 time: 0.344043 Jul 16 21:17:42.000124 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1135/2191] 10.222.0.16 () {62 vars in 1383 bytes} [Sat Jul 16 21:17:41 2022] PUT /compute/v2.1/os-quota-sets/13e0f52a884440db827dba9fe97e8fef?user_id=435411be15364e74a547015105a29c53 => generated 240 bytes in 354 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 16 21:17:42.006916 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-c87b4765-54ad-4f3a-bdd5-9f831b1c79d9 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Calling method '.version_select at 0x7fdda2983700>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:17:42.077960 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-c87b4765-54ad-4f3a-bdd5-9f831b1c79d9 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] 10.222.0.16 "GET /compute/v2.1/images/1f9a9864-79f1-4d21-84c6-8d017682af53" status: 200 len: 1454 microversion: 2.1 time: 0.076466 Jul 16 21:17:42.084347 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1136/2192] 10.222.0.16 () {58 vars in 1232 bytes} [Sat Jul 16 21:17:42 2022] GET /compute/v2.1/images/1f9a9864-79f1-4d21-84c6-8d017682af53 => generated 1454 bytes in 78 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:17:42.092432 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-51b0f9ec-aeec-4dcc-9a40-941276ec7297 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Calling method '.version_select at 0x7fdd9c6608b0>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:17:42.181092 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-bd0e51fc-21c2-4fb8-9efc-3f2a7b27d5f5 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd4144b5388b46a989a41a2f556bb508', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a90d4bcd25ff4bf68bdc194180fbbe09', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:17:42.196291 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-bd0e51fc-21c2-4fb8-9efc-3f2a7b27d5f5 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] 10.222.0.16 "GET /compute/v2.1/servers/7e4b142d-f489-4220-adc3-7b348dc14ce9" status: 200 len: 1515 microversion: 2.1 time: 0.245769 Jul 16 21:17:42.196291 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1057/2193] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:17:41 2022] GET /compute/v2.1/servers/7e4b142d-f489-4220-adc3-7b348dc14ce9 => generated 1515 bytes in 247 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:17:42.198582 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-111e8a93-6b67-4f2c-be1c-b6c22a7c4b0f tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:17:42.200331 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-111e8a93-6b67-4f2c-be1c-b6c22a7c4b0f tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] [instance: aa688a7e-0b28-4dfd-a159-8e19fd62ee02] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:17:42.200466 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-51b0f9ec-aeec-4dcc-9a40-941276ec7297 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] 10.222.0.16 "GET /compute/v2.1/images/211719d4-d58e-474c-ade9-8ffb307248f3" status: 200 len: 1423 microversion: 2.1 time: 0.116409 Jul 16 21:17:42.204566 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1137/2194] 10.222.0.16 () {58 vars in 1232 bytes} [Sat Jul 16 21:17:42 2022] GET /compute/v2.1/images/211719d4-d58e-474c-ade9-8ffb307248f3 => generated 1423 bytes in 125 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:17:42.213406 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-09853b15-922c-423b-b799-7b8aecd63f0f tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:17:42.214175 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-09853b15-922c-423b-b799-7b8aecd63f0f tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] [instance: b8565a82-250b-4f64-9d90-98b368e4bd57] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:17:42.218786 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-111e8a93-6b67-4f2c-be1c-b6c22a7c4b0f tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:42.225263 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-111e8a93-6b67-4f2c-be1c-b6c22a7c4b0f tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:42.225984 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-111e8a93-6b67-4f2c-be1c-b6c22a7c4b0f tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:42.235817 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-111e8a93-6b67-4f2c-be1c-b6c22a7c4b0f tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:42.301484 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-09853b15-922c-423b-b799-7b8aecd63f0f tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd0008fffad8e4708b2c01e03da92c47d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e353258f9c3048ef84683294e2b8579d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:17:42.313461 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-09853b15-922c-423b-b799-7b8aecd63f0f tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] 10.222.0.16 "GET /compute/v2.1/servers/b8565a82-250b-4f64-9d90-98b368e4bd57" status: 200 len: 1216 microversion: 2.1 time: 0.107598 Jul 16 21:17:42.314060 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1138/2195] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:17:42 2022] GET /compute/v2.1/servers/b8565a82-250b-4f64-9d90-98b368e4bd57 => generated 1216 bytes in 109 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:17:42.318574 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-17015c0b-1892-4daa-bcb5-75c343720e5d tempest-QuotasAdminTestV236-2041262956 tempest-QuotasAdminTestV236-2041262956-project-admin] Calling method '.version_select at 0x7fdd9c6b9dc0>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:17:42.473193 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-17015c0b-1892-4daa-bcb5-75c343720e5d tempest-QuotasAdminTestV236-2041262956 tempest-QuotasAdminTestV236-2041262956-project-admin] 10.222.0.16 "GET /compute/v2.1/os-quota-sets/13e0f52a884440db827dba9fe97e8fef?user_id=435411be15364e74a547015105a29c53" status: 200 len: 282 microversion: 2.36 time: 0.156943 Jul 16 21:17:42.476789 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1139/2196] 10.222.0.16 () {60 vars in 1363 bytes} [Sat Jul 16 21:17:42 2022] GET /compute/v2.1/os-quota-sets/13e0f52a884440db827dba9fe97e8fef?user_id=435411be15364e74a547015105a29c53 => generated 282 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 16 21:17:42.498195 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-76361d7a-3a03-4eec-9391-99a06ec6dfa9 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Calling method '.version_select at 0x7fdd9c6b1c10>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:17:42.507385 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-111e8a93-6b67-4f2c-be1c-b6c22a7c4b0f tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f5bc1f2209f4eb186d80e385b67c774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '65d5bcfa02af4e8284ba1342197e2860', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:17:42.517196 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-111e8a93-6b67-4f2c-be1c-b6c22a7c4b0f tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] 10.222.0.16 "GET /compute/v2.1/servers/aa688a7e-0b28-4dfd-a159-8e19fd62ee02" status: 200 len: 1314 microversion: 2.1 time: 0.322441 Jul 16 21:17:42.518149 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1058/2197] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:17:42 2022] GET /compute/v2.1/servers/aa688a7e-0b28-4dfd-a159-8e19fd62ee02 => generated 1314 bytes in 324 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:17:42.729066 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-61393ecc-8738-4d19-a75e-d204469cf380 req-f06f1c45-a2df-43b0-a976-9b54c7bca64a service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "b9a4910a-f117-4834-b699-498c6431b9bb", "name": "network-vif-unplugged", "status": "completed", "tag": "9ab69acd-f0a3-4355-b203-2c48e39cc3a1"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:17:42.748930 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-61393ecc-8738-4d19-a75e-d204469cf380 req-f06f1c45-a2df-43b0-a976-9b54c7bca64a service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:42.749276 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-61393ecc-8738-4d19-a75e-d204469cf380 req-f06f1c45-a2df-43b0-a976-9b54c7bca64a service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:42.779128 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-61393ecc-8738-4d19-a75e-d204469cf380 req-f06f1c45-a2df-43b0-a976-9b54c7bca64a service nova] Creating event network-vif-unplugged:9ab69acd-f0a3-4355-b203-2c48e39cc3a1 for instance b9a4910a-f117-4834-b699-498c6431b9bb on nch2-master-1 Jul 16 21:17:42.787269 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-61393ecc-8738-4d19-a75e-d204469cf380 req-f06f1c45-a2df-43b0-a976-9b54c7bca64a service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.061118 Jul 16 21:17:42.787899 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1059/2198] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:17:42 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 64 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:17:43.093377 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-8e2ae5ea-8dd8-40a6-9393-08aaff041f82 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Calling method '.version_select at 0x7fdda2a58f70>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:17:43.172309 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-8e2ae5ea-8dd8-40a6-9393-08aaff041f82 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] 10.222.0.16 "GET /compute/v2.1/images/1f9a9864-79f1-4d21-84c6-8d017682af53" status: 200 len: 1454 microversion: 2.1 time: 0.084002 Jul 16 21:17:43.175083 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1060/2199] 10.222.0.16 () {58 vars in 1232 bytes} [Sat Jul 16 21:17:43 2022] GET /compute/v2.1/images/1f9a9864-79f1-4d21-84c6-8d017682af53 => generated 1454 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:17:43.211125 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-901523e7-a65d-4473-94b3-8bdef27a2c1d tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:17:43.212083 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-901523e7-a65d-4473-94b3-8bdef27a2c1d tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] [instance: 7e4b142d-f489-4220-adc3-7b348dc14ce9] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:17:43.224697 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-901523e7-a65d-4473-94b3-8bdef27a2c1d tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:43.225552 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-901523e7-a65d-4473-94b3-8bdef27a2c1d tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:43.229951 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-901523e7-a65d-4473-94b3-8bdef27a2c1d tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:43.230565 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-901523e7-a65d-4473-94b3-8bdef27a2c1d tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:43.506738 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-901523e7-a65d-4473-94b3-8bdef27a2c1d tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd4144b5388b46a989a41a2f556bb508', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a90d4bcd25ff4bf68bdc194180fbbe09', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:17:43.516288 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-901523e7-a65d-4473-94b3-8bdef27a2c1d tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] 10.222.0.16 "GET /compute/v2.1/servers/7e4b142d-f489-4220-adc3-7b348dc14ce9" status: 200 len: 1515 microversion: 2.1 time: 0.309684 Jul 16 21:17:43.517118 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1061/2200] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:17:43 2022] GET /compute/v2.1/servers/7e4b142d-f489-4220-adc3-7b348dc14ce9 => generated 1515 bytes in 311 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:17:43.535034 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-4793368f-c5d1-4729-886b-d79b3fe18947 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:17:43.536942 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-4793368f-c5d1-4729-886b-d79b3fe18947 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] [instance: b8565a82-250b-4f64-9d90-98b368e4bd57] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:17:43.681343 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-4793368f-c5d1-4729-886b-d79b3fe18947 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd0008fffad8e4708b2c01e03da92c47d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e353258f9c3048ef84683294e2b8579d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:17:43.697411 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-4793368f-c5d1-4729-886b-d79b3fe18947 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] 10.222.0.16 "GET /compute/v2.1/servers/b8565a82-250b-4f64-9d90-98b368e4bd57" status: 200 len: 1216 microversion: 2.1 time: 0.178323 Jul 16 21:17:43.710093 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1062/2201] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:17:43 2022] GET /compute/v2.1/servers/b8565a82-250b-4f64-9d90-98b368e4bd57 => generated 1216 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:17:43.723196 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-bf51e9ea-1b9f-4b7b-a87e-e640491d72fa tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:17:43.726023 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-bf51e9ea-1b9f-4b7b-a87e-e640491d72fa tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] [instance: aa688a7e-0b28-4dfd-a159-8e19fd62ee02] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:17:43.738244 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-bf51e9ea-1b9f-4b7b-a87e-e640491d72fa tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:43.739081 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-bf51e9ea-1b9f-4b7b-a87e-e640491d72fa tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:43.739893 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-bf51e9ea-1b9f-4b7b-a87e-e640491d72fa tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:43.740507 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-bf51e9ea-1b9f-4b7b-a87e-e640491d72fa tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:43.958410 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-bf51e9ea-1b9f-4b7b-a87e-e640491d72fa tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f5bc1f2209f4eb186d80e385b67c774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '65d5bcfa02af4e8284ba1342197e2860', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:17:43.966839 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-bf51e9ea-1b9f-4b7b-a87e-e640491d72fa tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] 10.222.0.16 "GET /compute/v2.1/servers/aa688a7e-0b28-4dfd-a159-8e19fd62ee02" status: 200 len: 1464 microversion: 2.1 time: 0.255368 Jul 16 21:17:43.972530 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1063/2202] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:17:43 2022] GET /compute/v2.1/servers/aa688a7e-0b28-4dfd-a159-8e19fd62ee02 => generated 1464 bytes in 257 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:17:43.986697 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-67fb0996-3d1c-4c93-9083-95f70d515cb1 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:17:43.987352 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-67fb0996-3d1c-4c93-9083-95f70d515cb1 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] [instance: e7598b4a-f70c-4186-88d1-6fd0057f8656] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:17:43.994182 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-67fb0996-3d1c-4c93-9083-95f70d515cb1 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:43.994684 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-67fb0996-3d1c-4c93-9083-95f70d515cb1 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:44.002518 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-67fb0996-3d1c-4c93-9083-95f70d515cb1 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:44.002518 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-67fb0996-3d1c-4c93-9083-95f70d515cb1 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:44.258879 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-76361d7a-3a03-4eec-9391-99a06ec6dfa9 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] 10.222.0.16 "DELETE /compute/v2.1/images/211719d4-d58e-474c-ade9-8ffb307248f3" status: 204 len: 0 microversion: 2.1 time: 1.780825 Jul 16 21:17:44.260975 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1140/2203] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:17:42 2022] DELETE /compute/v2.1/images/211719d4-d58e-474c-ade9-8ffb307248f3 => generated 0 bytes in 1783 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 16 21:17:44.267155 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-eae34d73-1bd1-4412-86e4-586f69243796 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Calling method '.version_select at 0x7fdda2960040>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:17:44.299560 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-67fb0996-3d1c-4c93-9083-95f70d515cb1 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f5bc1f2209f4eb186d80e385b67c774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '65d5bcfa02af4e8284ba1342197e2860', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:17:44.317939 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-67fb0996-3d1c-4c93-9083-95f70d515cb1 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] 10.222.0.16 "GET /compute/v2.1/servers/e7598b4a-f70c-4186-88d1-6fd0057f8656" status: 200 len: 1463 microversion: 2.1 time: 0.334946 Jul 16 21:17:44.318853 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1064/2204] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:17:43 2022] GET /compute/v2.1/servers/e7598b4a-f70c-4186-88d1-6fd0057f8656 => generated 1463 bytes in 337 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:17:44.328182 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-eae34d73-1bd1-4412-86e4-586f69243796 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] 10.222.0.16 "GET /compute/v2.1/images/1f9a9864-79f1-4d21-84c6-8d017682af53" status: 200 len: 1454 microversion: 2.1 time: 0.065971 Jul 16 21:17:44.329275 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1141/2205] 10.222.0.16 () {58 vars in 1232 bytes} [Sat Jul 16 21:17:44 2022] GET /compute/v2.1/images/1f9a9864-79f1-4d21-84c6-8d017682af53 => generated 1454 bytes in 67 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:17:44.331961 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-349c48b1-ac41-4f9f-ae45-ecde47445370 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:17:44.332471 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-349c48b1-ac41-4f9f-ae45-ecde47445370 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] [instance: f03a2a83-e007-444a-b46b-70eb46bb242b] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:17:44.338612 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-349c48b1-ac41-4f9f-ae45-ecde47445370 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:44.339088 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-349c48b1-ac41-4f9f-ae45-ecde47445370 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:44.339874 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-349c48b1-ac41-4f9f-ae45-ecde47445370 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:44.340341 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-349c48b1-ac41-4f9f-ae45-ecde47445370 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:44.373763 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-349c48b1-ac41-4f9f-ae45-ecde47445370 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] [instance: f03a2a83-e007-444a-b46b-70eb46bb242b] Going to try to terminate instance {{(pid=138681) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 16 21:17:44.462268 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-5aaea6f6-21be-4787-9378-ee7a46ab5ebe tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServersAdminTestJSON-server-160984979", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452"}, "os:scheduler_hints": {"same_host": "e7598b4a-f70c-4186-88d1-6fd0057f8656"}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:17:44.524168 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.network.neutron [None req-5aaea6f6-21be-4787-9378-ee7a46ab5ebe tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] validate_networks() for None {{(pid=138680) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 16 21:17:44.833988 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-349c48b1-ac41-4f9f-ae45-ecde47445370 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] 10.222.0.16 "DELETE /compute/v2.1/servers/f03a2a83-e007-444a-b46b-70eb46bb242b" status: 204 len: 0 microversion: 2.1 time: 0.513814 Jul 16 21:17:44.838480 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1065/2206] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:17:44 2022] DELETE /compute/v2.1/servers/f03a2a83-e007-444a-b46b-70eb46bb242b => generated 0 bytes in 519 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 16 21:17:44.853590 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-5a597ad5-c576-4fa3-8dbd-19aa5958a85c tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:17:44.855576 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-5a597ad5-c576-4fa3-8dbd-19aa5958a85c tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] [instance: 7e4b142d-f489-4220-adc3-7b348dc14ce9] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:17:44.875824 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5a597ad5-c576-4fa3-8dbd-19aa5958a85c tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:44.877375 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5a597ad5-c576-4fa3-8dbd-19aa5958a85c tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:44.878438 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5a597ad5-c576-4fa3-8dbd-19aa5958a85c tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:44.879245 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5a597ad5-c576-4fa3-8dbd-19aa5958a85c tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:45.226090 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-5aaea6f6-21be-4787-9378-ee7a46ab5ebe tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Flavor limits 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 16 21:17:45.226525 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-5aaea6f6-21be-4787-9378-ee7a46ab5ebe tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Image limits 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 16 21:17:45.227031 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-5aaea6f6-21be-4787-9378-ee7a46ab5ebe tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Flavor pref 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 16 21:17:45.227442 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-5aaea6f6-21be-4787-9378-ee7a46ab5ebe tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Image pref 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 16 21:17:45.227895 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-5aaea6f6-21be-4787-9378-ee7a46ab5ebe tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 16 21:17:45.239056 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5aaea6f6-21be-4787-9378-ee7a46ab5ebe tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:45.239940 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5aaea6f6-21be-4787-9378-ee7a46ab5ebe tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:45.240766 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5aaea6f6-21be-4787-9378-ee7a46ab5ebe tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:45.241213 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5aaea6f6-21be-4787-9378-ee7a46ab5ebe tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:45.265811 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-5aaea6f6-21be-4787-9378-ee7a46ab5ebe tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Getting quotas for project 65d5bcfa02af4e8284ba1342197e2860. Resources: {'ram', 'instances', 'cores'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:17:45.272600 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-5aaea6f6-21be-4787-9378-ee7a46ab5ebe tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Getting quotas for user 2f5bc1f2209f4eb186d80e385b67c774 and project 65d5bcfa02af4e8284ba1342197e2860. Resources: {'ram', 'instances', 'cores'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:17:45.287298 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-5a597ad5-c576-4fa3-8dbd-19aa5958a85c tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd4144b5388b46a989a41a2f556bb508', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a90d4bcd25ff4bf68bdc194180fbbe09', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:17:45.299436 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-5a597ad5-c576-4fa3-8dbd-19aa5958a85c tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] 10.222.0.16 "GET /compute/v2.1/servers/7e4b142d-f489-4220-adc3-7b348dc14ce9" status: 200 len: 1515 microversion: 2.1 time: 0.459121 Jul 16 21:17:45.300245 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1066/2207] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:17:44 2022] GET /compute/v2.1/servers/7e4b142d-f489-4220-adc3-7b348dc14ce9 => generated 1515 bytes in 461 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:17:45.304864 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-81f8900a-77fc-490d-94e3-798238a1c3cc tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:17:45.305296 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-5aaea6f6-21be-4787-9378-ee7a46ab5ebe tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Going to run 1 instances... {{(pid=138680) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 16 21:17:45.308013 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-5aaea6f6-21be-4787-9378-ee7a46ab5ebe tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] [instance: 720ef2b0-e7b3-4b36-b613-1b73d2bae2e9] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138680) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 16 21:17:45.313969 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-81f8900a-77fc-490d-94e3-798238a1c3cc tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] [instance: b8565a82-250b-4f64-9d90-98b368e4bd57] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:17:45.326537 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-81f8900a-77fc-490d-94e3-798238a1c3cc tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:45.326977 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-81f8900a-77fc-490d-94e3-798238a1c3cc tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:45.327851 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-81f8900a-77fc-490d-94e3-798238a1c3cc tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:45.328222 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-81f8900a-77fc-490d-94e3-798238a1c3cc tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:45.426353 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-81f8900a-77fc-490d-94e3-798238a1c3cc tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd0008fffad8e4708b2c01e03da92c47d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e353258f9c3048ef84683294e2b8579d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:17:45.437464 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-81f8900a-77fc-490d-94e3-798238a1c3cc tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] 10.222.0.16 "GET /compute/v2.1/servers/b8565a82-250b-4f64-9d90-98b368e4bd57" status: 200 len: 1268 microversion: 2.1 time: 0.136003 Jul 16 21:17:45.447178 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1067/2208] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:17:45 2022] GET /compute/v2.1/servers/b8565a82-250b-4f64-9d90-98b368e4bd57 => generated 1268 bytes in 137 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:17:45.453756 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-a7942f6d-68af-4fe7-ab47-d5e81e048546 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ImagesTestJSON-server-1525766916", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452", "networks": [{"uuid": "e0956c94-52de-403b-bf7c-aa26a7179ba6"}]}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:17:45.510422 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-5aaea6f6-21be-4787-9378-ee7a46ab5ebe tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 1.051992 Jul 16 21:17:45.512546 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1142/2209] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 21:17:44 2022] POST /compute/v2.1/servers => generated 380 bytes in 1055 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Jul 16 21:17:45.519698 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-8062d0f9-11b2-4484-9c67-feba225f3f10 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Calling method '.version_select at 0x7fdd9c6b2a60>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:17:45.578364 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.network.neutron [None req-a7942f6d-68af-4fe7-ab47-d5e81e048546 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] validate_networks() for [('e0956c94-52de-403b-bf7c-aa26a7179ba6', None, None, None, None, None)] {{(pid=138681) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 16 21:17:45.606869 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-8062d0f9-11b2-4484-9c67-feba225f3f10 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] 10.222.0.16 "GET /compute/v2.1/images/1f9a9864-79f1-4d21-84c6-8d017682af53" status: 200 len: 1454 microversion: 2.1 time: 0.092943 Jul 16 21:17:45.608415 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1143/2210] 10.222.0.16 () {58 vars in 1232 bytes} [Sat Jul 16 21:17:45 2022] GET /compute/v2.1/images/1f9a9864-79f1-4d21-84c6-8d017682af53 => generated 1454 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:17:45.614448 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-295e895e-8d01-4fb9-a0a1-670b139dc0f2 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:17:45.615311 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-295e895e-8d01-4fb9-a0a1-670b139dc0f2 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] [instance: 720ef2b0-e7b3-4b36-b613-1b73d2bae2e9] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:17:45.805966 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-295e895e-8d01-4fb9-a0a1-670b139dc0f2 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f5bc1f2209f4eb186d80e385b67c774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '65d5bcfa02af4e8284ba1342197e2860', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:17:45.814030 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-295e895e-8d01-4fb9-a0a1-670b139dc0f2 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] 10.222.0.16 "GET /compute/v2.1/servers/720ef2b0-e7b3-4b36-b613-1b73d2bae2e9" status: 200 len: 1213 microversion: 2.1 time: 0.204309 Jul 16 21:17:45.814621 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1144/2211] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:17:45 2022] GET /compute/v2.1/servers/720ef2b0-e7b3-4b36-b613-1b73d2bae2e9 => generated 1213 bytes in 205 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:17:46.317337 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-51fae2c4-f742-4bbe-acee-998e7e071c7d tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:17:46.317914 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-51fae2c4-f742-4bbe-acee-998e7e071c7d tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] [instance: 7e4b142d-f489-4220-adc3-7b348dc14ce9] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:17:46.326267 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-51fae2c4-f742-4bbe-acee-998e7e071c7d tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:46.337881 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-51fae2c4-f742-4bbe-acee-998e7e071c7d tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.012s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:46.340353 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-51fae2c4-f742-4bbe-acee-998e7e071c7d tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:46.340684 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-51fae2c4-f742-4bbe-acee-998e7e071c7d tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:47.003421 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-51fae2c4-f742-4bbe-acee-998e7e071c7d tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd4144b5388b46a989a41a2f556bb508', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a90d4bcd25ff4bf68bdc194180fbbe09', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:17:47.020831 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-51fae2c4-f742-4bbe-acee-998e7e071c7d tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] 10.222.0.16 "GET /compute/v2.1/servers/7e4b142d-f489-4220-adc3-7b348dc14ce9" status: 200 len: 1515 microversion: 2.1 time: 0.707077 Jul 16 21:17:47.021695 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1145/2212] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:17:46 2022] GET /compute/v2.1/servers/7e4b142d-f489-4220-adc3-7b348dc14ce9 => generated 1515 bytes in 709 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:17:47.026460 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-e5cbd2ae-fb50-4ecf-ad7d-ce55453c0745 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:17:47.037835 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-e5cbd2ae-fb50-4ecf-ad7d-ce55453c0745 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] [instance: b8565a82-250b-4f64-9d90-98b368e4bd57] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:17:47.045853 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e5cbd2ae-fb50-4ecf-ad7d-ce55453c0745 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:47.052066 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e5cbd2ae-fb50-4ecf-ad7d-ce55453c0745 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:47.053136 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e5cbd2ae-fb50-4ecf-ad7d-ce55453c0745 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:47.053731 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e5cbd2ae-fb50-4ecf-ad7d-ce55453c0745 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:47.101670 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-a7942f6d-68af-4fe7-ab47-d5e81e048546 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Flavor limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 16 21:17:47.102210 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-a7942f6d-68af-4fe7-ab47-d5e81e048546 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Image limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 16 21:17:47.102557 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-a7942f6d-68af-4fe7-ab47-d5e81e048546 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Flavor pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 16 21:17:47.102986 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-a7942f6d-68af-4fe7-ab47-d5e81e048546 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Image pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 16 21:17:47.103422 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-a7942f6d-68af-4fe7-ab47-d5e81e048546 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 16 21:17:47.142486 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a7942f6d-68af-4fe7-ab47-d5e81e048546 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:47.142944 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a7942f6d-68af-4fe7-ab47-d5e81e048546 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:47.143787 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a7942f6d-68af-4fe7-ab47-d5e81e048546 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:47.144246 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a7942f6d-68af-4fe7-ab47-d5e81e048546 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:47.191224 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-a7942f6d-68af-4fe7-ab47-d5e81e048546 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Getting quotas for project 24c64db8082648c8837c16352252f96d. Resources: {'ram', 'instances', 'cores'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:17:47.207739 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-a7942f6d-68af-4fe7-ab47-d5e81e048546 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Getting quotas for user daf3c9e6634e4ae1bfb492eae1f44c93 and project 24c64db8082648c8837c16352252f96d. Resources: {'ram', 'instances', 'cores'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:17:47.233708 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-e5cbd2ae-fb50-4ecf-ad7d-ce55453c0745 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd0008fffad8e4708b2c01e03da92c47d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e353258f9c3048ef84683294e2b8579d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:17:47.246425 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-a7942f6d-68af-4fe7-ab47-d5e81e048546 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Going to run 1 instances... {{(pid=138681) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 16 21:17:47.248761 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-a7942f6d-68af-4fe7-ab47-d5e81e048546 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] [instance: ed8dcb97-7546-45ef-9e9d-a9bb03a1b419] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138681) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 16 21:17:47.252999 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-e5cbd2ae-fb50-4ecf-ad7d-ce55453c0745 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] 10.222.0.16 "GET /compute/v2.1/servers/b8565a82-250b-4f64-9d90-98b368e4bd57" status: 200 len: 1274 microversion: 2.1 time: 0.220362 Jul 16 21:17:47.253974 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1146/2213] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:17:47 2022] GET /compute/v2.1/servers/b8565a82-250b-4f64-9d90-98b368e4bd57 => generated 1274 bytes in 231 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:17:47.280078 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-8b6a4185-a983-4ce9-9b5d-8fdd5751ea03 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Calling method '.version_select at 0x7fdd9c6bf670>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:17:47.388496 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-8b6a4185-a983-4ce9-9b5d-8fdd5751ea03 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] 10.222.0.16 "GET /compute/v2.1/images/1f9a9864-79f1-4d21-84c6-8d017682af53" status: 200 len: 1454 microversion: 2.1 time: 0.133329 Jul 16 21:17:47.389135 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1147/2214] 10.222.0.16 () {58 vars in 1232 bytes} [Sat Jul 16 21:17:47 2022] GET /compute/v2.1/images/1f9a9864-79f1-4d21-84c6-8d017682af53 => generated 1454 bytes in 135 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:17:47.418926 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-e08cd5fc-36ca-4e2c-a538-4be04bddb5f3 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:17:47.422902 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-e08cd5fc-36ca-4e2c-a538-4be04bddb5f3 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] [instance: 720ef2b0-e7b3-4b36-b613-1b73d2bae2e9] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:17:47.423704 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-a7942f6d-68af-4fe7-ab47-d5e81e048546 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 1.982757 Jul 16 21:17:47.429740 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1068/2215] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 21:17:45 2022] POST /compute/v2.1/servers => generated 380 bytes in 1990 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 16 21:17:47.450018 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-37b5164e-f9b3-4f17-9213-a0aee869ce0d req-2daa17a4-ce6f-41d0-9f7b-301e50668282 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "6cb4e66a-eba2-42b7-91e8-09ee7b2a5a4f", "name": "network-vif-plugged", "status": "completed", "tag": "381fe698-72ef-4f6f-8b88-5df5cb8d8d96"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:17:47.450692 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e08cd5fc-36ca-4e2c-a538-4be04bddb5f3 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:47.451485 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e08cd5fc-36ca-4e2c-a538-4be04bddb5f3 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:47.459536 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e08cd5fc-36ca-4e2c-a538-4be04bddb5f3 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:47.460413 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e08cd5fc-36ca-4e2c-a538-4be04bddb5f3 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:47.501377 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-37b5164e-f9b3-4f17-9213-a0aee869ce0d req-2daa17a4-ce6f-41d0-9f7b-301e50668282 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:47.501835 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-37b5164e-f9b3-4f17-9213-a0aee869ce0d req-2daa17a4-ce6f-41d0-9f7b-301e50668282 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:47.583655 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-37b5164e-f9b3-4f17-9213-a0aee869ce0d req-2daa17a4-ce6f-41d0-9f7b-301e50668282 service nova] Creating event network-vif-plugged:381fe698-72ef-4f6f-8b88-5df5cb8d8d96 for instance 6cb4e66a-eba2-42b7-91e8-09ee7b2a5a4f on nch2-master-1 Jul 16 21:17:47.604119 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-37b5164e-f9b3-4f17-9213-a0aee869ce0d req-2daa17a4-ce6f-41d0-9f7b-301e50668282 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.172226 Jul 16 21:17:47.607975 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1069/2216] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:17:47 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:17:47.616933 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-7f688086-5942-4719-91a3-85515e265e26 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:17:47.618820 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-7f688086-5942-4719-91a3-85515e265e26 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] [instance: ed8dcb97-7546-45ef-9e9d-a9bb03a1b419] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:17:48.080625 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-e08cd5fc-36ca-4e2c-a538-4be04bddb5f3 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f5bc1f2209f4eb186d80e385b67c774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '65d5bcfa02af4e8284ba1342197e2860', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:17:48.114644 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-e08cd5fc-36ca-4e2c-a538-4be04bddb5f3 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] 10.222.0.16 "GET /compute/v2.1/servers/720ef2b0-e7b3-4b36-b613-1b73d2bae2e9" status: 200 len: 1217 microversion: 2.1 time: 0.710417 Jul 16 21:17:48.124016 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1148/2217] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:17:47 2022] GET /compute/v2.1/servers/720ef2b0-e7b3-4b36-b613-1b73d2bae2e9 => generated 1217 bytes in 734 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:17:48.149390 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-fcfcb712-ae26-4ff4-81c8-b318d04c19db tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:17:48.150810 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-fcfcb712-ae26-4ff4-81c8-b318d04c19db tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] [instance: 7e4b142d-f489-4220-adc3-7b348dc14ce9] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:17:48.196554 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-fcfcb712-ae26-4ff4-81c8-b318d04c19db tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:48.197216 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-fcfcb712-ae26-4ff4-81c8-b318d04c19db tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:48.200341 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-fcfcb712-ae26-4ff4-81c8-b318d04c19db tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:48.200811 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-fcfcb712-ae26-4ff4-81c8-b318d04c19db tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:48.455198 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-7f688086-5942-4719-91a3-85515e265e26 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'daf3c9e6634e4ae1bfb492eae1f44c93', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '24c64db8082648c8837c16352252f96d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:17:48.466727 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-7f688086-5942-4719-91a3-85515e265e26 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] 10.222.0.16 "GET /compute/v2.1/servers/ed8dcb97-7546-45ef-9e9d-a9bb03a1b419" status: 200 len: 1208 microversion: 2.1 time: 0.855747 Jul 16 21:17:48.468046 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1070/2218] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:17:47 2022] GET /compute/v2.1/servers/ed8dcb97-7546-45ef-9e9d-a9bb03a1b419 => generated 1208 bytes in 859 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:17:48.473626 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-2455dbb7-be43-4172-a7fe-06b305bbd75b tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:17:48.474569 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-2455dbb7-be43-4172-a7fe-06b305bbd75b tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] [instance: b8565a82-250b-4f64-9d90-98b368e4bd57] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:17:48.490106 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2455dbb7-be43-4172-a7fe-06b305bbd75b tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:48.490963 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2455dbb7-be43-4172-a7fe-06b305bbd75b tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:48.492153 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2455dbb7-be43-4172-a7fe-06b305bbd75b tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:48.493278 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2455dbb7-be43-4172-a7fe-06b305bbd75b tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:48.660036 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-2455dbb7-be43-4172-a7fe-06b305bbd75b tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd0008fffad8e4708b2c01e03da92c47d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e353258f9c3048ef84683294e2b8579d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:17:48.667740 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-2455dbb7-be43-4172-a7fe-06b305bbd75b tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] 10.222.0.16 "GET /compute/v2.1/servers/b8565a82-250b-4f64-9d90-98b368e4bd57" status: 200 len: 1274 microversion: 2.1 time: 0.198245 Jul 16 21:17:48.685099 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1071/2219] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:17:48 2022] GET /compute/v2.1/servers/b8565a82-250b-4f64-9d90-98b368e4bd57 => generated 1274 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:17:48.690240 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-271a4247-d602-4def-ae45-2cefb16b0182 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Calling method '.version_select at 0x7fdda2a683a0>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:17:48.752700 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-fcfcb712-ae26-4ff4-81c8-b318d04c19db tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd4144b5388b46a989a41a2f556bb508', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a90d4bcd25ff4bf68bdc194180fbbe09', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:17:48.765821 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-fcfcb712-ae26-4ff4-81c8-b318d04c19db tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] 10.222.0.16 "GET /compute/v2.1/servers/7e4b142d-f489-4220-adc3-7b348dc14ce9" status: 200 len: 1521 microversion: 2.1 time: 0.640017 Jul 16 21:17:48.766419 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1149/2220] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:17:48 2022] GET /compute/v2.1/servers/7e4b142d-f489-4220-adc3-7b348dc14ce9 => generated 1521 bytes in 641 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:17:48.792093 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-271a4247-d602-4def-ae45-2cefb16b0182 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] 10.222.0.16 "GET /compute/v2.1/images/1f9a9864-79f1-4d21-84c6-8d017682af53" status: 200 len: 1454 microversion: 2.1 time: 0.105641 Jul 16 21:17:48.793890 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1072/2221] 10.222.0.16 () {58 vars in 1232 bytes} [Sat Jul 16 21:17:48 2022] GET /compute/v2.1/images/1f9a9864-79f1-4d21-84c6-8d017682af53 => generated 1454 bytes in 108 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:17:49.138155 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-54df50a3-21e9-4ae7-923b-e8536163cd65 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:17:49.138798 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-54df50a3-21e9-4ae7-923b-e8536163cd65 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] [instance: 720ef2b0-e7b3-4b36-b613-1b73d2bae2e9] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:17:49.149396 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-54df50a3-21e9-4ae7-923b-e8536163cd65 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:49.149907 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-54df50a3-21e9-4ae7-923b-e8536163cd65 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:49.150701 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-54df50a3-21e9-4ae7-923b-e8536163cd65 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:49.151180 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-54df50a3-21e9-4ae7-923b-e8536163cd65 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:49.488793 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-3bfbfc30-55ad-4313-9fa6-9ffcbe59feaf tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:17:49.490398 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-3bfbfc30-55ad-4313-9fa6-9ffcbe59feaf tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] [instance: ed8dcb97-7546-45ef-9e9d-a9bb03a1b419] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:17:49.522344 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3bfbfc30-55ad-4313-9fa6-9ffcbe59feaf tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:49.522817 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3bfbfc30-55ad-4313-9fa6-9ffcbe59feaf tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:49.528726 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3bfbfc30-55ad-4313-9fa6-9ffcbe59feaf tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:49.529183 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3bfbfc30-55ad-4313-9fa6-9ffcbe59feaf tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:49.564632 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-54df50a3-21e9-4ae7-923b-e8536163cd65 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f5bc1f2209f4eb186d80e385b67c774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '65d5bcfa02af4e8284ba1342197e2860', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:17:49.619386 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-54df50a3-21e9-4ae7-923b-e8536163cd65 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] 10.222.0.16 "GET /compute/v2.1/servers/720ef2b0-e7b3-4b36-b613-1b73d2bae2e9" status: 200 len: 1265 microversion: 2.1 time: 0.484647 Jul 16 21:17:49.624237 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1150/2222] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:17:49 2022] GET /compute/v2.1/servers/720ef2b0-e7b3-4b36-b613-1b73d2bae2e9 => generated 1265 bytes in 491 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:17:49.691230 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-a3cfb17e-1e72-431e-bfb5-542ff86d1b97 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:17:49.692883 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-a3cfb17e-1e72-431e-bfb5-542ff86d1b97 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] [instance: b8565a82-250b-4f64-9d90-98b368e4bd57] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:17:49.711773 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a3cfb17e-1e72-431e-bfb5-542ff86d1b97 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:49.712614 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a3cfb17e-1e72-431e-bfb5-542ff86d1b97 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:49.713562 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a3cfb17e-1e72-431e-bfb5-542ff86d1b97 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:49.715437 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a3cfb17e-1e72-431e-bfb5-542ff86d1b97 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:50.262567 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-3bfbfc30-55ad-4313-9fa6-9ffcbe59feaf tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'daf3c9e6634e4ae1bfb492eae1f44c93', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '24c64db8082648c8837c16352252f96d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:17:50.279432 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-3bfbfc30-55ad-4313-9fa6-9ffcbe59feaf tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] 10.222.0.16 "GET /compute/v2.1/servers/ed8dcb97-7546-45ef-9e9d-a9bb03a1b419" status: 200 len: 1212 microversion: 2.1 time: 0.792386 Jul 16 21:17:50.280587 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1073/2223] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:17:49 2022] GET /compute/v2.1/servers/ed8dcb97-7546-45ef-9e9d-a9bb03a1b419 => generated 1212 bytes in 801 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:17:50.297668 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-ba2e5e16-d0d9-469a-a21e-118d3892cf9c req-d5585511-0f57-48fd-abc4-1b4584b54183 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "f03a2a83-e007-444a-b46b-70eb46bb242b", "name": "network-vif-unplugged", "status": "completed", "tag": "1df74ef4-7ed5-4498-8310-01458a7e541a"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:17:50.326787 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-ba2e5e16-d0d9-469a-a21e-118d3892cf9c req-d5585511-0f57-48fd-abc4-1b4584b54183 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:50.327269 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-ba2e5e16-d0d9-469a-a21e-118d3892cf9c req-d5585511-0f57-48fd-abc4-1b4584b54183 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:50.377056 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-ba2e5e16-d0d9-469a-a21e-118d3892cf9c req-d5585511-0f57-48fd-abc4-1b4584b54183 service nova] Creating event network-vif-unplugged:1df74ef4-7ed5-4498-8310-01458a7e541a for instance f03a2a83-e007-444a-b46b-70eb46bb242b on nch1-master-1 Jul 16 21:17:50.416874 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-ba2e5e16-d0d9-469a-a21e-118d3892cf9c req-d5585511-0f57-48fd-abc4-1b4584b54183 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.116773 Jul 16 21:17:50.416874 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1074/2224] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:17:50 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 118 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:17:50.426365 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-2e866a69-8688-4f39-a086-631184e420a2 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:17:50.427314 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-2e866a69-8688-4f39-a086-631184e420a2 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] [instance: 7e4b142d-f489-4220-adc3-7b348dc14ce9] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:17:50.439426 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2e866a69-8688-4f39-a086-631184e420a2 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:50.442026 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2e866a69-8688-4f39-a086-631184e420a2 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:50.446754 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2e866a69-8688-4f39-a086-631184e420a2 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:50.449730 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2e866a69-8688-4f39-a086-631184e420a2 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:51.634526 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-a3cfb17e-1e72-431e-bfb5-542ff86d1b97 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd0008fffad8e4708b2c01e03da92c47d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e353258f9c3048ef84683294e2b8579d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:17:51.673015 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-2e866a69-8688-4f39-a086-631184e420a2 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd4144b5388b46a989a41a2f556bb508', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a90d4bcd25ff4bf68bdc194180fbbe09', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:17:51.685768 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-a3cfb17e-1e72-431e-bfb5-542ff86d1b97 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] 10.222.0.16 "GET /compute/v2.1/servers/b8565a82-250b-4f64-9d90-98b368e4bd57" status: 200 len: 1316 microversion: 2.1 time: 1.998200 Jul 16 21:17:51.692204 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1151/2225] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:17:49 2022] GET /compute/v2.1/servers/b8565a82-250b-4f64-9d90-98b368e4bd57 => generated 1316 bytes in 2002 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:17:51.695800 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-2e866a69-8688-4f39-a086-631184e420a2 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] 10.222.0.16 "GET /compute/v2.1/servers/7e4b142d-f489-4220-adc3-7b348dc14ce9" status: 200 len: 1521 microversion: 2.1 time: 1.279648 Jul 16 21:17:51.697965 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1075/2226] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:17:50 2022] GET /compute/v2.1/servers/7e4b142d-f489-4220-adc3-7b348dc14ce9 => generated 1521 bytes in 1289 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:17:51.721919 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-570a5b12-8f57-4b55-85b7-70f8bb305eb0 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Calling method '.version_select at 0x7fdd9c6b68b0>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:17:51.733989 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-cee4723c-c35a-41c1-96bf-5b1087e9f121 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:17:51.734914 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-cee4723c-c35a-41c1-96bf-5b1087e9f121 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] [instance: 720ef2b0-e7b3-4b36-b613-1b73d2bae2e9] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:17:51.823634 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cee4723c-c35a-41c1-96bf-5b1087e9f121 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:51.826994 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cee4723c-c35a-41c1-96bf-5b1087e9f121 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:51.832893 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cee4723c-c35a-41c1-96bf-5b1087e9f121 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:51.835155 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cee4723c-c35a-41c1-96bf-5b1087e9f121 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:51.874076 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:17:51.920350 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-570a5b12-8f57-4b55-85b7-70f8bb305eb0 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] 10.222.0.16 "GET /compute/v2.1/images/1f9a9864-79f1-4d21-84c6-8d017682af53" status: 200 len: 1454 microversion: 2.1 time: 0.226220 Jul 16 21:17:51.920974 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1152/2227] 10.222.0.16 () {58 vars in 1232 bytes} [Sat Jul 16 21:17:51 2022] GET /compute/v2.1/images/1f9a9864-79f1-4d21-84c6-8d017682af53 => generated 1454 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:17:51.929679 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-a1f3af52-62b5-4ba7-a972-ca3f2dc11d98 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:17:51.930525 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-a1f3af52-62b5-4ba7-a972-ca3f2dc11d98 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] [instance: ed8dcb97-7546-45ef-9e9d-a9bb03a1b419] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:17:51.939240 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_api:SELECT=95,nova_api:INSERT=5 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:17:51.941235 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a1f3af52-62b5-4ba7-a972-ca3f2dc11d98 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:51.941877 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a1f3af52-62b5-4ba7-a972-ca3f2dc11d98 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:51.942819 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a1f3af52-62b5-4ba7-a972-ca3f2dc11d98 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:51.943442 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a1f3af52-62b5-4ba7-a972-ca3f2dc11d98 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:51.963435 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=96,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:17:52.113608 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-cee4723c-c35a-41c1-96bf-5b1087e9f121 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f5bc1f2209f4eb186d80e385b67c774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '65d5bcfa02af4e8284ba1342197e2860', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:17:52.128374 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-cee4723c-c35a-41c1-96bf-5b1087e9f121 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] 10.222.0.16 "GET /compute/v2.1/servers/720ef2b0-e7b3-4b36-b613-1b73d2bae2e9" status: 200 len: 1271 microversion: 2.1 time: 0.428105 Jul 16 21:17:52.129340 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1076/2228] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:17:51 2022] GET /compute/v2.1/servers/720ef2b0-e7b3-4b36-b613-1b73d2bae2e9 => generated 1271 bytes in 430 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:17:52.139669 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-a1f3af52-62b5-4ba7-a972-ca3f2dc11d98 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'daf3c9e6634e4ae1bfb492eae1f44c93', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '24c64db8082648c8837c16352252f96d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:17:52.149171 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-a1f3af52-62b5-4ba7-a972-ca3f2dc11d98 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] 10.222.0.16 "GET /compute/v2.1/servers/ed8dcb97-7546-45ef-9e9d-a9bb03a1b419" status: 200 len: 1266 microversion: 2.1 time: 0.225329 Jul 16 21:17:52.149972 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1153/2229] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:17:51 2022] GET /compute/v2.1/servers/ed8dcb97-7546-45ef-9e9d-a9bb03a1b419 => generated 1266 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:17:52.389553 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-e16f6b89-89f8-4231-88d0-4d59a6bac681 tempest-QuotasAdminTestV257-1554560177 tempest-QuotasAdminTestV257-1554560177-project-admin] Action: 'update', calling method: .version_select at 0x7fdda2a58a60>, body: {"quota_set": {"ram": "5120"}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:17:52.570101 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-d72286dd-43f9-40ab-8591-7ff70024fb94 tempest-ServersNegativeTestMultiTenantJSON-1135003248 tempest-ServersNegativeTestMultiTenantJSON-1135003248-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServersNegativeTestMultiTenantJSON-server-1751248964", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452", "networks": [{"uuid": "fa401cc4-ef35-4721-97b9-708af9ea3be1"}]}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:17:52.600198 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e16f6b89-89f8-4231-88d0-4d59a6bac681 tempest-QuotasAdminTestV257-1554560177 tempest-QuotasAdminTestV257-1554560177-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:52.601880 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e16f6b89-89f8-4231-88d0-4d59a6bac681 tempest-QuotasAdminTestV257-1554560177 tempest-QuotasAdminTestV257-1554560177-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:52.602958 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e16f6b89-89f8-4231-88d0-4d59a6bac681 tempest-QuotasAdminTestV257-1554560177 tempest-QuotasAdminTestV257-1554560177-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:52.607743 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e16f6b89-89f8-4231-88d0-4d59a6bac681 tempest-QuotasAdminTestV257-1554560177 tempest-QuotasAdminTestV257-1554560177-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:52.881563 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.network.neutron [None req-d72286dd-43f9-40ab-8591-7ff70024fb94 tempest-ServersNegativeTestMultiTenantJSON-1135003248 tempest-ServersNegativeTestMultiTenantJSON-1135003248-project] validate_networks() for [('fa401cc4-ef35-4721-97b9-708af9ea3be1', None, None, None, None, None)] {{(pid=138680) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 16 21:17:53.282469 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-e16f6b89-89f8-4231-88d0-4d59a6bac681 tempest-QuotasAdminTestV257-1554560177 tempest-QuotasAdminTestV257-1554560177-project-admin] 10.222.0.16 "PUT /compute/v2.1/os-quota-sets/243dfbefda284034b9973693c393cf42" status: 200 len: 148 microversion: 2.57 time: 1.139379 Jul 16 21:17:53.282469 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1077/2230] 10.222.0.16 () {62 vars in 1302 bytes} [Sat Jul 16 21:17:52 2022] PUT /compute/v2.1/os-quota-sets/243dfbefda284034b9973693c393cf42 => generated 148 bytes in 1141 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 16 21:17:53.318170 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-e9fff94a-9cce-484f-8b75-d9c24770b7a7 req-edd52e8a-d453-4e2a-a0e2-4a8b8d0a26bd service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "7e4b142d-f489-4220-adc3-7b348dc14ce9", "name": "network-vif-unplugged", "status": "completed", "tag": "d10b6c2f-f386-410e-a6a2-fbd06a8fceaf"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:17:53.348013 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-e9fff94a-9cce-484f-8b75-d9c24770b7a7 req-edd52e8a-d453-4e2a-a0e2-4a8b8d0a26bd service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:53.352992 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-e9fff94a-9cce-484f-8b75-d9c24770b7a7 req-edd52e8a-d453-4e2a-a0e2-4a8b8d0a26bd service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:53.403270 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-e9fff94a-9cce-484f-8b75-d9c24770b7a7 req-edd52e8a-d453-4e2a-a0e2-4a8b8d0a26bd service nova] Creating event network-vif-unplugged:d10b6c2f-f386-410e-a6a2-fbd06a8fceaf for instance 7e4b142d-f489-4220-adc3-7b348dc14ce9 on nch1-master-1 Jul 16 21:17:53.407580 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-e9fff94a-9cce-484f-8b75-d9c24770b7a7 req-edd52e8a-d453-4e2a-a0e2-4a8b8d0a26bd service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.114875 Jul 16 21:17:53.412856 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1078/2231] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:17:53 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 129 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:17:53.422080 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-b4112720-d8f1-4eea-bdff-5d7469ed77ba tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:17:53.432034 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-b4112720-d8f1-4eea-bdff-5d7469ed77ba tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] [instance: b8565a82-250b-4f64-9d90-98b368e4bd57] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:17:53.440038 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_api:SELECT=92,nova_api:INSERT=7,nova_api:UPDATE=1 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:17:53.464390 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b4112720-d8f1-4eea-bdff-5d7469ed77ba tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:53.465693 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b4112720-d8f1-4eea-bdff-5d7469ed77ba tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.012s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:53.467511 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b4112720-d8f1-4eea-bdff-5d7469ed77ba tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:53.467973 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b4112720-d8f1-4eea-bdff-5d7469ed77ba tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:54.525237 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-b4112720-d8f1-4eea-bdff-5d7469ed77ba tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd0008fffad8e4708b2c01e03da92c47d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e353258f9c3048ef84683294e2b8579d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:17:54.546764 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-b4112720-d8f1-4eea-bdff-5d7469ed77ba tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] 10.222.0.16 "GET /compute/v2.1/servers/b8565a82-250b-4f64-9d90-98b368e4bd57" status: 200 len: 1316 microversion: 2.1 time: 1.129540 Jul 16 21:17:54.547393 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1079/2232] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:17:53 2022] GET /compute/v2.1/servers/b8565a82-250b-4f64-9d90-98b368e4bd57 => generated 1316 bytes in 1135 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:17:54.552530 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-d39d006b-a473-49ad-bdc4-0fc90dd2bba4 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:17:54.553153 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-d39d006b-a473-49ad-bdc4-0fc90dd2bba4 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] [instance: 7e4b142d-f489-4220-adc3-7b348dc14ce9] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:17:54.571512 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d39d006b-a473-49ad-bdc4-0fc90dd2bba4 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:54.571858 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d39d006b-a473-49ad-bdc4-0fc90dd2bba4 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:54.572658 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d39d006b-a473-49ad-bdc4-0fc90dd2bba4 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:54.573085 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d39d006b-a473-49ad-bdc4-0fc90dd2bba4 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:55.196736 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-d39d006b-a473-49ad-bdc4-0fc90dd2bba4 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd4144b5388b46a989a41a2f556bb508', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a90d4bcd25ff4bf68bdc194180fbbe09', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:17:55.213221 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-d39d006b-a473-49ad-bdc4-0fc90dd2bba4 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] 10.222.0.16 "GET /compute/v2.1/servers/7e4b142d-f489-4220-adc3-7b348dc14ce9" status: 200 len: 1521 microversion: 2.1 time: 0.664230 Jul 16 21:17:55.222949 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1080/2233] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:17:54 2022] GET /compute/v2.1/servers/7e4b142d-f489-4220-adc3-7b348dc14ce9 => generated 1521 bytes in 666 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:17:55.233760 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-1178e242-47c9-4b2a-b36a-6da1ea031371 req-64dd0337-4670-4294-8f29-060edc637a51 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "b8565a82-250b-4f64-9d90-98b368e4bd57", "tag": "54d1cb01-bab2-432d-8666-66691fd8c6de"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:17:55.244475 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-1178e242-47c9-4b2a-b36a-6da1ea031371 req-64dd0337-4670-4294-8f29-060edc637a51 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:55.245454 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-1178e242-47c9-4b2a-b36a-6da1ea031371 req-64dd0337-4670-4294-8f29-060edc637a51 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:55.251421 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell0:SELECT=11 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:17:55.274492 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-1178e242-47c9-4b2a-b36a-6da1ea031371 req-64dd0337-4670-4294-8f29-060edc637a51 service nova] Creating event network-changed:54d1cb01-bab2-432d-8666-66691fd8c6de for instance b8565a82-250b-4f64-9d90-98b368e4bd57 on nch2-master-1 Jul 16 21:17:55.283952 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-1178e242-47c9-4b2a-b36a-6da1ea031371 req-64dd0337-4670-4294-8f29-060edc637a51 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.068566 Jul 16 21:17:55.287567 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1081/2234] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:17:55 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 72 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:17:55.304053 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-15754146-c787-4066-b373-cd4047f3e055 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Calling method '.version_select at 0x7fdda2b7e5e0>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:17:55.328980 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-d72286dd-43f9-40ab-8591-7ff70024fb94 tempest-ServersNegativeTestMultiTenantJSON-1135003248 tempest-ServersNegativeTestMultiTenantJSON-1135003248-project] Flavor limits 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 16 21:17:55.337741 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-d72286dd-43f9-40ab-8591-7ff70024fb94 tempest-ServersNegativeTestMultiTenantJSON-1135003248 tempest-ServersNegativeTestMultiTenantJSON-1135003248-project] Image limits 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 16 21:17:55.339143 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-d72286dd-43f9-40ab-8591-7ff70024fb94 tempest-ServersNegativeTestMultiTenantJSON-1135003248 tempest-ServersNegativeTestMultiTenantJSON-1135003248-project] Flavor pref 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 16 21:17:55.340264 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-d72286dd-43f9-40ab-8591-7ff70024fb94 tempest-ServersNegativeTestMultiTenantJSON-1135003248 tempest-ServersNegativeTestMultiTenantJSON-1135003248-project] Image pref 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 16 21:17:55.341571 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-d72286dd-43f9-40ab-8591-7ff70024fb94 tempest-ServersNegativeTestMultiTenantJSON-1135003248 tempest-ServersNegativeTestMultiTenantJSON-1135003248-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 16 21:17:55.381270 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d72286dd-43f9-40ab-8591-7ff70024fb94 tempest-ServersNegativeTestMultiTenantJSON-1135003248 tempest-ServersNegativeTestMultiTenantJSON-1135003248-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:55.387992 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d72286dd-43f9-40ab-8591-7ff70024fb94 tempest-ServersNegativeTestMultiTenantJSON-1135003248 tempest-ServersNegativeTestMultiTenantJSON-1135003248-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.022s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:55.388963 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d72286dd-43f9-40ab-8591-7ff70024fb94 tempest-ServersNegativeTestMultiTenantJSON-1135003248 tempest-ServersNegativeTestMultiTenantJSON-1135003248-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:55.389386 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d72286dd-43f9-40ab-8591-7ff70024fb94 tempest-ServersNegativeTestMultiTenantJSON-1135003248 tempest-ServersNegativeTestMultiTenantJSON-1135003248-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:55.405899 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-15754146-c787-4066-b373-cd4047f3e055 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] 10.222.0.16 "GET /compute/v2.1/images/1f9a9864-79f1-4d21-84c6-8d017682af53" status: 200 len: 1454 microversion: 2.1 time: 0.114125 Jul 16 21:17:55.412035 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1082/2235] 10.222.0.16 () {58 vars in 1232 bytes} [Sat Jul 16 21:17:55 2022] GET /compute/v2.1/images/1f9a9864-79f1-4d21-84c6-8d017682af53 => generated 1454 bytes in 121 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:17:55.417054 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-86278374-bb4a-436e-b05b-6084f1f9b234 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:17:55.434220 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-86278374-bb4a-436e-b05b-6084f1f9b234 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] [instance: 720ef2b0-e7b3-4b36-b613-1b73d2bae2e9] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:17:55.448017 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-86278374-bb4a-436e-b05b-6084f1f9b234 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:55.448795 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-86278374-bb4a-436e-b05b-6084f1f9b234 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:55.449426 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-86278374-bb4a-436e-b05b-6084f1f9b234 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:55.449877 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-86278374-bb4a-436e-b05b-6084f1f9b234 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:55.452868 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-d72286dd-43f9-40ab-8591-7ff70024fb94 tempest-ServersNegativeTestMultiTenantJSON-1135003248 tempest-ServersNegativeTestMultiTenantJSON-1135003248-project] Getting quotas for project ba229aaef0ec4ddcadd1d940376c31ac. Resources: {'ram', 'instances', 'cores'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:17:55.473507 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-d72286dd-43f9-40ab-8591-7ff70024fb94 tempest-ServersNegativeTestMultiTenantJSON-1135003248 tempest-ServersNegativeTestMultiTenantJSON-1135003248-project] Getting quotas for user 50cdc134e712487196f6623c5cf14592 and project ba229aaef0ec4ddcadd1d940376c31ac. Resources: {'ram', 'instances', 'cores'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:17:55.507777 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-d72286dd-43f9-40ab-8591-7ff70024fb94 tempest-ServersNegativeTestMultiTenantJSON-1135003248 tempest-ServersNegativeTestMultiTenantJSON-1135003248-project] Going to run 1 instances... {{(pid=138680) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 16 21:17:55.510355 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-d72286dd-43f9-40ab-8591-7ff70024fb94 tempest-ServersNegativeTestMultiTenantJSON-1135003248 tempest-ServersNegativeTestMultiTenantJSON-1135003248-project] [instance: 43169d4f-1214-419a-aa1e-def65182ed5a] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138680) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 16 21:17:55.715738 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-d72286dd-43f9-40ab-8591-7ff70024fb94 tempest-ServersNegativeTestMultiTenantJSON-1135003248 tempest-ServersNegativeTestMultiTenantJSON-1135003248-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 3.473505 Jul 16 21:17:55.726562 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1154/2236] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 21:17:52 2022] POST /compute/v2.1/servers => generated 380 bytes in 3487 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 16 21:17:55.740469 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-c2ad0c1d-0e26-48af-b30f-cd4519db14c2 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:17:55.746624 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-c2ad0c1d-0e26-48af-b30f-cd4519db14c2 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] [instance: ed8dcb97-7546-45ef-9e9d-a9bb03a1b419] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:17:55.772743 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c2ad0c1d-0e26-48af-b30f-cd4519db14c2 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:55.776740 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c2ad0c1d-0e26-48af-b30f-cd4519db14c2 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:55.778811 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c2ad0c1d-0e26-48af-b30f-cd4519db14c2 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:55.781372 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c2ad0c1d-0e26-48af-b30f-cd4519db14c2 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:56.327761 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-86278374-bb4a-436e-b05b-6084f1f9b234 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f5bc1f2209f4eb186d80e385b67c774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '65d5bcfa02af4e8284ba1342197e2860', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:17:56.344872 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-86278374-bb4a-436e-b05b-6084f1f9b234 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] 10.222.0.16 "GET /compute/v2.1/servers/720ef2b0-e7b3-4b36-b613-1b73d2bae2e9" status: 200 len: 1313 microversion: 2.1 time: 0.930384 Jul 16 21:17:56.348125 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1083/2237] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:17:55 2022] GET /compute/v2.1/servers/720ef2b0-e7b3-4b36-b613-1b73d2bae2e9 => generated 1313 bytes in 936 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:17:56.373915 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-d49dc8b2-2f3a-4fd7-960b-b96ccc3aab3e tempest-QuotasAdminTestV257-1554560177 tempest-QuotasAdminTestV257-1554560177-project-admin] Calling method '.version_select at 0x7fdd9cdcaaf0>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:17:56.416854 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-c2ad0c1d-0e26-48af-b30f-cd4519db14c2 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'daf3c9e6634e4ae1bfb492eae1f44c93', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '24c64db8082648c8837c16352252f96d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:17:56.428756 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-c2ad0c1d-0e26-48af-b30f-cd4519db14c2 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] 10.222.0.16 "GET /compute/v2.1/servers/ed8dcb97-7546-45ef-9e9d-a9bb03a1b419" status: 200 len: 1308 microversion: 2.1 time: 0.700525 Jul 16 21:17:56.430018 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1155/2238] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:17:55 2022] GET /compute/v2.1/servers/ed8dcb97-7546-45ef-9e9d-a9bb03a1b419 => generated 1308 bytes in 703 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:17:56.438907 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-d02f4265-8084-4ac4-b91e-a6445ecb4655 req-815ac843-c23e-4a76-9bde-6fddd2e4324a service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "7e4b142d-f489-4220-adc3-7b348dc14ce9", "name": "network-vif-plugged", "status": "completed", "tag": "d10b6c2f-f386-410e-a6a2-fbd06a8fceaf"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:17:56.448827 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-d02f4265-8084-4ac4-b91e-a6445ecb4655 req-815ac843-c23e-4a76-9bde-6fddd2e4324a service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:56.450137 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-d02f4265-8084-4ac4-b91e-a6445ecb4655 req-815ac843-c23e-4a76-9bde-6fddd2e4324a service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:56.468879 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-d02f4265-8084-4ac4-b91e-a6445ecb4655 req-815ac843-c23e-4a76-9bde-6fddd2e4324a service nova] Creating event network-vif-plugged:d10b6c2f-f386-410e-a6a2-fbd06a8fceaf for instance 7e4b142d-f489-4220-adc3-7b348dc14ce9 on nch1-master-1 Jul 16 21:17:56.488153 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-d02f4265-8084-4ac4-b91e-a6445ecb4655 req-815ac843-c23e-4a76-9bde-6fddd2e4324a service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.055555 Jul 16 21:17:56.488572 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1156/2239] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:17:56 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:17:56.507891 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-af12b102-b87a-4c79-a18d-ed327bda9937 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:17:56.507891 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-af12b102-b87a-4c79-a18d-ed327bda9937 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] [instance: b8565a82-250b-4f64-9d90-98b368e4bd57] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:17:56.513353 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-af12b102-b87a-4c79-a18d-ed327bda9937 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:56.513819 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-af12b102-b87a-4c79-a18d-ed327bda9937 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:56.514629 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-af12b102-b87a-4c79-a18d-ed327bda9937 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:56.515271 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-af12b102-b87a-4c79-a18d-ed327bda9937 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:56.517527 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d49dc8b2-2f3a-4fd7-960b-b96ccc3aab3e tempest-QuotasAdminTestV257-1554560177 tempest-QuotasAdminTestV257-1554560177-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:56.518489 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d49dc8b2-2f3a-4fd7-960b-b96ccc3aab3e tempest-QuotasAdminTestV257-1554560177 tempest-QuotasAdminTestV257-1554560177-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:56.518730 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d49dc8b2-2f3a-4fd7-960b-b96ccc3aab3e tempest-QuotasAdminTestV257-1554560177 tempest-QuotasAdminTestV257-1554560177-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:56.519224 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d49dc8b2-2f3a-4fd7-960b-b96ccc3aab3e tempest-QuotasAdminTestV257-1554560177 tempest-QuotasAdminTestV257-1554560177-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:56.601191 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-d49dc8b2-2f3a-4fd7-960b-b96ccc3aab3e tempest-QuotasAdminTestV257-1554560177 tempest-QuotasAdminTestV257-1554560177-project-admin] 10.222.0.16 "GET /compute/v2.1/os-quota-sets/243dfbefda284034b9973693c393cf42/detail" status: 200 len: 463 microversion: 2.57 time: 0.244695 Jul 16 21:17:56.601821 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1084/2240] 10.222.0.16 () {60 vars in 1303 bytes} [Sat Jul 16 21:17:56 2022] GET /compute/v2.1/os-quota-sets/243dfbefda284034b9973693c393cf42/detail => generated 463 bytes in 253 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 16 21:17:56.607323 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-086c2342-ce7c-43bb-ae87-ae2f1fec1a9c tempest-ServersNegativeTestMultiTenantJSON-1135003248 tempest-ServersNegativeTestMultiTenantJSON-1135003248-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:17:56.617280 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-086c2342-ce7c-43bb-ae87-ae2f1fec1a9c tempest-ServersNegativeTestMultiTenantJSON-1135003248 tempest-ServersNegativeTestMultiTenantJSON-1135003248-project] [instance: 43169d4f-1214-419a-aa1e-def65182ed5a] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:17:56.943471 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-086c2342-ce7c-43bb-ae87-ae2f1fec1a9c tempest-ServersNegativeTestMultiTenantJSON-1135003248 tempest-ServersNegativeTestMultiTenantJSON-1135003248-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '50cdc134e712487196f6623c5cf14592', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ba229aaef0ec4ddcadd1d940376c31ac', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:17:56.958485 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-086c2342-ce7c-43bb-ae87-ae2f1fec1a9c tempest-ServersNegativeTestMultiTenantJSON-1135003248 tempest-ServersNegativeTestMultiTenantJSON-1135003248-project] 10.222.0.16 "GET /compute/v2.1/servers/43169d4f-1214-419a-aa1e-def65182ed5a" status: 200 len: 1228 microversion: 2.1 time: 0.352747 Jul 16 21:17:56.959226 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1085/2241] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:17:56 2022] GET /compute/v2.1/servers/43169d4f-1214-419a-aa1e-def65182ed5a => generated 1228 bytes in 357 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:17:56.975771 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-23970acc-120b-4da5-ac48-07f76f92e180 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:17:56.977213 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-23970acc-120b-4da5-ac48-07f76f92e180 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] [instance: 7e4b142d-f489-4220-adc3-7b348dc14ce9] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:17:56.989096 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-23970acc-120b-4da5-ac48-07f76f92e180 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:56.993469 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-23970acc-120b-4da5-ac48-07f76f92e180 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:56.993469 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-23970acc-120b-4da5-ac48-07f76f92e180 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:56.993469 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-23970acc-120b-4da5-ac48-07f76f92e180 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:57.796224 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-af12b102-b87a-4c79-a18d-ed327bda9937 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd0008fffad8e4708b2c01e03da92c47d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e353258f9c3048ef84683294e2b8579d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:17:57.810714 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-af12b102-b87a-4c79-a18d-ed327bda9937 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] 10.222.0.16 "GET /compute/v2.1/servers/b8565a82-250b-4f64-9d90-98b368e4bd57" status: 200 len: 1316 microversion: 2.1 time: 1.314678 Jul 16 21:17:57.810714 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1157/2242] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:17:56 2022] GET /compute/v2.1/servers/b8565a82-250b-4f64-9d90-98b368e4bd57 => generated 1316 bytes in 1322 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:17:57.829038 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-af37074b-943e-441f-acbd-e308f97ba429 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Calling method '.version_select at 0x7fdda29d1dc0>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:17:57.903886 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-af37074b-943e-441f-acbd-e308f97ba429 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] 10.222.0.16 "GET /compute/v2.1/images/1f9a9864-79f1-4d21-84c6-8d017682af53" status: 200 len: 1454 microversion: 2.1 time: 0.082365 Jul 16 21:17:57.904485 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1158/2243] 10.222.0.16 () {58 vars in 1232 bytes} [Sat Jul 16 21:17:57 2022] GET /compute/v2.1/images/1f9a9864-79f1-4d21-84c6-8d017682af53 => generated 1454 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:17:57.909589 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-9d4375de-078a-4c31-a928-b164ee9e9ab3 tempest-QuotasAdminTestV257-1554560177 tempest-QuotasAdminTestV257-1554560177-project-admin] Action: 'update', calling method: .version_select at 0x7fdda29b2c10>, body: {"quota_set": {"ram": "2048"}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:17:57.992379 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-23970acc-120b-4da5-ac48-07f76f92e180 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd4144b5388b46a989a41a2f556bb508', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a90d4bcd25ff4bf68bdc194180fbbe09', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:17:58.007666 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-23970acc-120b-4da5-ac48-07f76f92e180 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] 10.222.0.16 "GET /compute/v2.1/servers/7e4b142d-f489-4220-adc3-7b348dc14ce9" status: 200 len: 1521 microversion: 2.1 time: 1.036659 Jul 16 21:17:58.023492 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1086/2244] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:17:56 2022] GET /compute/v2.1/servers/7e4b142d-f489-4220-adc3-7b348dc14ce9 => generated 1521 bytes in 1045 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:17:58.038281 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-4722809c-2737-4a04-905a-5ec5c3c4033f tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:17:58.044479 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-4722809c-2737-4a04-905a-5ec5c3c4033f tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] [instance: 720ef2b0-e7b3-4b36-b613-1b73d2bae2e9] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:17:58.048930 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9d4375de-078a-4c31-a928-b164ee9e9ab3 tempest-QuotasAdminTestV257-1554560177 tempest-QuotasAdminTestV257-1554560177-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:58.049402 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9d4375de-078a-4c31-a928-b164ee9e9ab3 tempest-QuotasAdminTestV257-1554560177 tempest-QuotasAdminTestV257-1554560177-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:58.050225 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9d4375de-078a-4c31-a928-b164ee9e9ab3 tempest-QuotasAdminTestV257-1554560177 tempest-QuotasAdminTestV257-1554560177-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:58.050667 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9d4375de-078a-4c31-a928-b164ee9e9ab3 tempest-QuotasAdminTestV257-1554560177 tempest-QuotasAdminTestV257-1554560177-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:58.056628 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4722809c-2737-4a04-905a-5ec5c3c4033f tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:58.057223 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4722809c-2737-4a04-905a-5ec5c3c4033f tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:58.060169 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4722809c-2737-4a04-905a-5ec5c3c4033f tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:58.062593 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4722809c-2737-4a04-905a-5ec5c3c4033f tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:58.182788 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9d4375de-078a-4c31-a928-b164ee9e9ab3 tempest-QuotasAdminTestV257-1554560177 tempest-QuotasAdminTestV257-1554560177-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:58.183275 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9d4375de-078a-4c31-a928-b164ee9e9ab3 tempest-QuotasAdminTestV257-1554560177 tempest-QuotasAdminTestV257-1554560177-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:58.186079 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9d4375de-078a-4c31-a928-b164ee9e9ab3 tempest-QuotasAdminTestV257-1554560177 tempest-QuotasAdminTestV257-1554560177-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:58.187047 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9d4375de-078a-4c31-a928-b164ee9e9ab3 tempest-QuotasAdminTestV257-1554560177 tempest-QuotasAdminTestV257-1554560177-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:58.372737 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-9d4375de-078a-4c31-a928-b164ee9e9ab3 tempest-QuotasAdminTestV257-1554560177 tempest-QuotasAdminTestV257-1554560177-project-admin] 10.222.0.16 "PUT /compute/v2.1/os-quota-sets/243dfbefda284034b9973693c393cf42?user_id=26aed09f0744445aa4d9ae07f3c07dbe" status: 200 len: 148 microversion: 2.57 time: 0.466888 Jul 16 21:17:58.380326 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1159/2245] 10.222.0.16 () {62 vars in 1383 bytes} [Sat Jul 16 21:17:57 2022] PUT /compute/v2.1/os-quota-sets/243dfbefda284034b9973693c393cf42?user_id=26aed09f0744445aa4d9ae07f3c07dbe => generated 148 bytes in 475 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 16 21:17:58.392097 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-3c4b7a56-3290-433b-ad0c-95dafc689865 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:17:58.392730 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-3c4b7a56-3290-433b-ad0c-95dafc689865 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] [instance: ed8dcb97-7546-45ef-9e9d-a9bb03a1b419] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:17:58.409842 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3c4b7a56-3290-433b-ad0c-95dafc689865 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:58.410112 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3c4b7a56-3290-433b-ad0c-95dafc689865 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:58.413855 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3c4b7a56-3290-433b-ad0c-95dafc689865 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:58.414324 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3c4b7a56-3290-433b-ad0c-95dafc689865 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:58.800125 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-3c4b7a56-3290-433b-ad0c-95dafc689865 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'daf3c9e6634e4ae1bfb492eae1f44c93', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '24c64db8082648c8837c16352252f96d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:17:58.810389 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-3c4b7a56-3290-433b-ad0c-95dafc689865 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] 10.222.0.16 "GET /compute/v2.1/servers/ed8dcb97-7546-45ef-9e9d-a9bb03a1b419" status: 200 len: 1308 microversion: 2.1 time: 0.428872 Jul 16 21:17:58.814278 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1160/2246] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:17:58 2022] GET /compute/v2.1/servers/ed8dcb97-7546-45ef-9e9d-a9bb03a1b419 => generated 1308 bytes in 433 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:17:58.829370 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-bc6c679a-9c0d-48bb-a4d2-e7af830efc88 req-82656b5b-675d-4442-a8f5-2eea933da1c9 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "f03a2a83-e007-444a-b46b-70eb46bb242b", "name": "network-vif-deleted", "tag": "1df74ef4-7ed5-4498-8310-01458a7e541a"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:17:58.856609 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-bc6c679a-9c0d-48bb-a4d2-e7af830efc88 req-82656b5b-675d-4442-a8f5-2eea933da1c9 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:58.857123 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-bc6c679a-9c0d-48bb-a4d2-e7af830efc88 req-82656b5b-675d-4442-a8f5-2eea933da1c9 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.009s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:58.893764 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-bc6c679a-9c0d-48bb-a4d2-e7af830efc88 req-82656b5b-675d-4442-a8f5-2eea933da1c9 service nova] Creating event network-vif-deleted:1df74ef4-7ed5-4498-8310-01458a7e541a for instance f03a2a83-e007-444a-b46b-70eb46bb242b on nch1-master-1 Jul 16 21:17:58.902957 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-bc6c679a-9c0d-48bb-a4d2-e7af830efc88 req-82656b5b-675d-4442-a8f5-2eea933da1c9 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.085630 Jul 16 21:17:58.904261 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1161/2247] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:17:58 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:17:58.910049 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-78481ec0-8fd8-408b-8fc8-84bbecd184d3 tempest-ServersNegativeTestMultiTenantJSON-1135003248 tempest-ServersNegativeTestMultiTenantJSON-1135003248-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:17:58.911284 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-78481ec0-8fd8-408b-8fc8-84bbecd184d3 tempest-ServersNegativeTestMultiTenantJSON-1135003248 tempest-ServersNegativeTestMultiTenantJSON-1135003248-project] [instance: 43169d4f-1214-419a-aa1e-def65182ed5a] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:17:58.923649 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-78481ec0-8fd8-408b-8fc8-84bbecd184d3 tempest-ServersNegativeTestMultiTenantJSON-1135003248 tempest-ServersNegativeTestMultiTenantJSON-1135003248-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:58.924526 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-78481ec0-8fd8-408b-8fc8-84bbecd184d3 tempest-ServersNegativeTestMultiTenantJSON-1135003248 tempest-ServersNegativeTestMultiTenantJSON-1135003248-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:58.926429 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-78481ec0-8fd8-408b-8fc8-84bbecd184d3 tempest-ServersNegativeTestMultiTenantJSON-1135003248 tempest-ServersNegativeTestMultiTenantJSON-1135003248-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:58.927195 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-78481ec0-8fd8-408b-8fc8-84bbecd184d3 tempest-ServersNegativeTestMultiTenantJSON-1135003248 tempest-ServersNegativeTestMultiTenantJSON-1135003248-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:59.113123 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-4722809c-2737-4a04-905a-5ec5c3c4033f tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f5bc1f2209f4eb186d80e385b67c774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '65d5bcfa02af4e8284ba1342197e2860', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:17:59.124206 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-4722809c-2737-4a04-905a-5ec5c3c4033f tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] 10.222.0.16 "GET /compute/v2.1/servers/720ef2b0-e7b3-4b36-b613-1b73d2bae2e9" status: 200 len: 1313 microversion: 2.1 time: 1.108024 Jul 16 21:17:59.147941 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1087/2248] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:17:58 2022] GET /compute/v2.1/servers/720ef2b0-e7b3-4b36-b613-1b73d2bae2e9 => generated 1313 bytes in 1132 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:17:59.154270 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-1698df43-f8ee-41e1-b792-0d4d9724aea3 tempest-QuotasAdminTestV257-1554560177 tempest-QuotasAdminTestV257-1554560177-project-admin] Calling method '.version_select at 0x7fdda2b7e0d0>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:17:59.161045 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-78481ec0-8fd8-408b-8fc8-84bbecd184d3 tempest-ServersNegativeTestMultiTenantJSON-1135003248 tempest-ServersNegativeTestMultiTenantJSON-1135003248-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '50cdc134e712487196f6623c5cf14592', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ba229aaef0ec4ddcadd1d940376c31ac', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:17:59.176767 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-78481ec0-8fd8-408b-8fc8-84bbecd184d3 tempest-ServersNegativeTestMultiTenantJSON-1135003248 tempest-ServersNegativeTestMultiTenantJSON-1135003248-project] 10.222.0.16 "GET /compute/v2.1/servers/43169d4f-1214-419a-aa1e-def65182ed5a" status: 200 len: 1280 microversion: 2.1 time: 0.270808 Jul 16 21:17:59.181036 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1162/2249] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:17:58 2022] GET /compute/v2.1/servers/43169d4f-1214-419a-aa1e-def65182ed5a => generated 1280 bytes in 276 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:17:59.191553 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-b7c21b1e-c9d6-46e0-adca-42b8cb44f8c0 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:17:59.192425 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-b7c21b1e-c9d6-46e0-adca-42b8cb44f8c0 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] [instance: b8565a82-250b-4f64-9d90-98b368e4bd57] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:17:59.206945 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b7c21b1e-c9d6-46e0-adca-42b8cb44f8c0 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:59.213314 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b7c21b1e-c9d6-46e0-adca-42b8cb44f8c0 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:59.214572 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b7c21b1e-c9d6-46e0-adca-42b8cb44f8c0 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:59.216475 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b7c21b1e-c9d6-46e0-adca-42b8cb44f8c0 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:59.307326 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-1698df43-f8ee-41e1-b792-0d4d9724aea3 tempest-QuotasAdminTestV257-1554560177 tempest-QuotasAdminTestV257-1554560177-project-admin] 10.222.0.16 "GET /compute/v2.1/os-quota-sets/243dfbefda284034b9973693c393cf42?user_id=26aed09f0744445aa4d9ae07f3c07dbe" status: 200 len: 190 microversion: 2.57 time: 0.157965 Jul 16 21:17:59.308926 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1088/2250] 10.222.0.16 () {60 vars in 1363 bytes} [Sat Jul 16 21:17:59 2022] GET /compute/v2.1/os-quota-sets/243dfbefda284034b9973693c393cf42?user_id=26aed09f0744445aa4d9ae07f3c07dbe => generated 190 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 16 21:17:59.327732 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-28374ea3-98f8-4ce3-9b06-2fa023f3b66e tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Calling method '.version_select at 0x7fdd9cdf9670>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:17:59.406686 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-28374ea3-98f8-4ce3-9b06-2fa023f3b66e tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] 10.222.0.16 "GET /compute/v2.1/images/1f9a9864-79f1-4d21-84c6-8d017682af53" status: 200 len: 1454 microversion: 2.1 time: 0.095720 Jul 16 21:17:59.408702 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1089/2251] 10.222.0.16 () {58 vars in 1232 bytes} [Sat Jul 16 21:17:59 2022] GET /compute/v2.1/images/1f9a9864-79f1-4d21-84c6-8d017682af53 => generated 1454 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:17:59.424685 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-722834fa-fc6a-45e6-a70f-ec48bea2e75b tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:17:59.427946 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-722834fa-fc6a-45e6-a70f-ec48bea2e75b tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] [instance: 7e4b142d-f489-4220-adc3-7b348dc14ce9] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:17:59.444289 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-722834fa-fc6a-45e6-a70f-ec48bea2e75b tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:59.447278 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-722834fa-fc6a-45e6-a70f-ec48bea2e75b tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:17:59.448616 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-722834fa-fc6a-45e6-a70f-ec48bea2e75b tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:17:59.449405 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-722834fa-fc6a-45e6-a70f-ec48bea2e75b tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:00.286356 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-b7c21b1e-c9d6-46e0-adca-42b8cb44f8c0 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd0008fffad8e4708b2c01e03da92c47d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e353258f9c3048ef84683294e2b8579d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:18:00.295509 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-b7c21b1e-c9d6-46e0-adca-42b8cb44f8c0 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] 10.222.0.16 "GET /compute/v2.1/servers/b8565a82-250b-4f64-9d90-98b368e4bd57" status: 200 len: 1316 microversion: 2.1 time: 1.113149 Jul 16 21:18:00.298278 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1163/2252] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:17:59 2022] GET /compute/v2.1/servers/b8565a82-250b-4f64-9d90-98b368e4bd57 => generated 1316 bytes in 1116 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:18:00.312107 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-3b5f1904-e88f-4a92-bbc1-0324a292ca14 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:00.313041 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-3b5f1904-e88f-4a92-bbc1-0324a292ca14 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] [instance: ed8dcb97-7546-45ef-9e9d-a9bb03a1b419] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:18:00.329696 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3b5f1904-e88f-4a92-bbc1-0324a292ca14 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:00.330265 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3b5f1904-e88f-4a92-bbc1-0324a292ca14 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:00.333693 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3b5f1904-e88f-4a92-bbc1-0324a292ca14 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:00.335721 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3b5f1904-e88f-4a92-bbc1-0324a292ca14 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:00.633291 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-722834fa-fc6a-45e6-a70f-ec48bea2e75b tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd4144b5388b46a989a41a2f556bb508', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a90d4bcd25ff4bf68bdc194180fbbe09', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:18:00.648678 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-722834fa-fc6a-45e6-a70f-ec48bea2e75b tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] 10.222.0.16 "GET /compute/v2.1/servers/7e4b142d-f489-4220-adc3-7b348dc14ce9" status: 200 len: 1506 microversion: 2.1 time: 1.238660 Jul 16 21:18:00.659116 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1090/2253] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:17:59 2022] GET /compute/v2.1/servers/7e4b142d-f489-4220-adc3-7b348dc14ce9 => generated 1506 bytes in 1240 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:18:00.671333 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-f3b625e5-339b-49c8-8f30-00071d66f78a tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:00.671333 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-f3b625e5-339b-49c8-8f30-00071d66f78a tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] [instance: 720ef2b0-e7b3-4b36-b613-1b73d2bae2e9] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:18:00.679888 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f3b625e5-339b-49c8-8f30-00071d66f78a tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:00.686497 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f3b625e5-339b-49c8-8f30-00071d66f78a tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:00.686497 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f3b625e5-339b-49c8-8f30-00071d66f78a tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:00.686497 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f3b625e5-339b-49c8-8f30-00071d66f78a tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:01.187432 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-3b5f1904-e88f-4a92-bbc1-0324a292ca14 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'daf3c9e6634e4ae1bfb492eae1f44c93', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '24c64db8082648c8837c16352252f96d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:18:01.195823 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-3b5f1904-e88f-4a92-bbc1-0324a292ca14 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] 10.222.0.16 "GET /compute/v2.1/servers/ed8dcb97-7546-45ef-9e9d-a9bb03a1b419" status: 200 len: 1308 microversion: 2.1 time: 0.895726 Jul 16 21:18:01.196553 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1164/2254] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:18:00 2022] GET /compute/v2.1/servers/ed8dcb97-7546-45ef-9e9d-a9bb03a1b419 => generated 1308 bytes in 898 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:18:01.202025 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-448b168b-1d17-4b03-8253-4b1a102b27a2 tempest-ServersNegativeTestMultiTenantJSON-1135003248 tempest-ServersNegativeTestMultiTenantJSON-1135003248-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:01.202953 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-448b168b-1d17-4b03-8253-4b1a102b27a2 tempest-ServersNegativeTestMultiTenantJSON-1135003248 tempest-ServersNegativeTestMultiTenantJSON-1135003248-project] [instance: 43169d4f-1214-419a-aa1e-def65182ed5a] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:18:01.209986 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-448b168b-1d17-4b03-8253-4b1a102b27a2 tempest-ServersNegativeTestMultiTenantJSON-1135003248 tempest-ServersNegativeTestMultiTenantJSON-1135003248-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:01.210831 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-448b168b-1d17-4b03-8253-4b1a102b27a2 tempest-ServersNegativeTestMultiTenantJSON-1135003248 tempest-ServersNegativeTestMultiTenantJSON-1135003248-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:01.211822 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-448b168b-1d17-4b03-8253-4b1a102b27a2 tempest-ServersNegativeTestMultiTenantJSON-1135003248 tempest-ServersNegativeTestMultiTenantJSON-1135003248-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:01.212262 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-448b168b-1d17-4b03-8253-4b1a102b27a2 tempest-ServersNegativeTestMultiTenantJSON-1135003248 tempest-ServersNegativeTestMultiTenantJSON-1135003248-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:01.312203 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-f3b625e5-339b-49c8-8f30-00071d66f78a tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f5bc1f2209f4eb186d80e385b67c774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '65d5bcfa02af4e8284ba1342197e2860', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:18:01.319706 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-f3b625e5-339b-49c8-8f30-00071d66f78a tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] 10.222.0.16 "GET /compute/v2.1/servers/720ef2b0-e7b3-4b36-b613-1b73d2bae2e9" status: 200 len: 1313 microversion: 2.1 time: 0.668947 Jul 16 21:18:01.320282 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1091/2255] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:18:00 2022] GET /compute/v2.1/servers/720ef2b0-e7b3-4b36-b613-1b73d2bae2e9 => generated 1313 bytes in 670 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:18:01.330021 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-c52128d8-d60c-4ed2-b980-7cb09be7f28c tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Calling method '.version_select at 0x7fdd9cdf9310>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:01.351529 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-448b168b-1d17-4b03-8253-4b1a102b27a2 tempest-ServersNegativeTestMultiTenantJSON-1135003248 tempest-ServersNegativeTestMultiTenantJSON-1135003248-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '50cdc134e712487196f6623c5cf14592', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ba229aaef0ec4ddcadd1d940376c31ac', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:18:01.371946 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-448b168b-1d17-4b03-8253-4b1a102b27a2 tempest-ServersNegativeTestMultiTenantJSON-1135003248 tempest-ServersNegativeTestMultiTenantJSON-1135003248-project] 10.222.0.16 "GET /compute/v2.1/servers/43169d4f-1214-419a-aa1e-def65182ed5a" status: 200 len: 1286 microversion: 2.1 time: 0.173378 Jul 16 21:18:01.373641 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1165/2256] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:18:01 2022] GET /compute/v2.1/servers/43169d4f-1214-419a-aa1e-def65182ed5a => generated 1286 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:18:01.383733 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-c52128d8-d60c-4ed2-b980-7cb09be7f28c tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] 10.222.0.16 "GET /compute/v2.1/images/1f9a9864-79f1-4d21-84c6-8d017682af53" status: 200 len: 1454 microversion: 2.1 time: 0.062386 Jul 16 21:18:01.390066 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-5c8b7b32-c65a-486a-8826-7a8103864439 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:01.390551 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1092/2257] 10.222.0.16 () {58 vars in 1232 bytes} [Sat Jul 16 21:18:01 2022] GET /compute/v2.1/images/1f9a9864-79f1-4d21-84c6-8d017682af53 => generated 1454 bytes in 70 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:18:01.390780 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-5c8b7b32-c65a-486a-8826-7a8103864439 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] [instance: 7e4b142d-f489-4220-adc3-7b348dc14ce9] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:18:01.396416 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-ada500c3-25a6-477b-9d9c-2953f5a6c48b req-2da64720-6782-475a-aca4-2addd62c716b service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "b8565a82-250b-4f64-9d90-98b368e4bd57", "name": "network-vif-plugged", "status": "completed", "tag": "54d1cb01-bab2-432d-8666-66691fd8c6de"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:18:01.398665 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5c8b7b32-c65a-486a-8826-7a8103864439 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:01.399194 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5c8b7b32-c65a-486a-8826-7a8103864439 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:01.399970 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5c8b7b32-c65a-486a-8826-7a8103864439 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:01.400414 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5c8b7b32-c65a-486a-8826-7a8103864439 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:01.407271 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-ada500c3-25a6-477b-9d9c-2953f5a6c48b req-2da64720-6782-475a-aca4-2addd62c716b service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:01.407966 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-ada500c3-25a6-477b-9d9c-2953f5a6c48b req-2da64720-6782-475a-aca4-2addd62c716b service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:01.422552 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-ada500c3-25a6-477b-9d9c-2953f5a6c48b req-2da64720-6782-475a-aca4-2addd62c716b service nova] Creating event network-vif-plugged:54d1cb01-bab2-432d-8666-66691fd8c6de for instance b8565a82-250b-4f64-9d90-98b368e4bd57 on nch2-master-1 Jul 16 21:18:01.430917 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-ada500c3-25a6-477b-9d9c-2953f5a6c48b req-2da64720-6782-475a-aca4-2addd62c716b service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.036449 Jul 16 21:18:01.447153 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1093/2258] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:18:01 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:18:01.466184 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-3f89f72d-514b-4acf-809b-6710b361280e req-7d5d3f32-22ea-4e04-867d-5f174a5159d3 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "ed8dcb97-7546-45ef-9e9d-a9bb03a1b419", "tag": "033ca6ad-5781-4719-b3b5-986dcfd7de3d"}, {"name": "network-changed", "server_uuid": "720ef2b0-e7b3-4b36-b613-1b73d2bae2e9", "tag": "b4dd35f4-14d3-4d08-b800-60be4a0cba5b"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:18:01.496531 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-3f89f72d-514b-4acf-809b-6710b361280e req-7d5d3f32-22ea-4e04-867d-5f174a5159d3 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:01.497732 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-3f89f72d-514b-4acf-809b-6710b361280e req-7d5d3f32-22ea-4e04-867d-5f174a5159d3 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:01.514179 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-3f89f72d-514b-4acf-809b-6710b361280e req-7d5d3f32-22ea-4e04-867d-5f174a5159d3 service nova] Creating event network-changed:033ca6ad-5781-4719-b3b5-986dcfd7de3d for instance ed8dcb97-7546-45ef-9e9d-a9bb03a1b419 on nch2-master-1 Jul 16 21:18:01.517909 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-3f89f72d-514b-4acf-809b-6710b361280e req-7d5d3f32-22ea-4e04-867d-5f174a5159d3 service nova] Creating event network-changed:b4dd35f4-14d3-4d08-b800-60be4a0cba5b for instance 720ef2b0-e7b3-4b36-b613-1b73d2bae2e9 on nch1-master-1 Jul 16 21:18:01.528404 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-3f89f72d-514b-4acf-809b-6710b361280e req-7d5d3f32-22ea-4e04-867d-5f174a5159d3 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 346 microversion: 2.1 time: 0.078167 Jul 16 21:18:01.530601 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1094/2259] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:18:01 2022] POST /compute/v2.1/os-server-external-events => generated 346 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:18:01.536823 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-446f4d76-31ad-4096-b32e-51fbc6a88af1 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:01.537925 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-446f4d76-31ad-4096-b32e-51fbc6a88af1 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] [instance: b8565a82-250b-4f64-9d90-98b368e4bd57] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:18:01.545665 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-446f4d76-31ad-4096-b32e-51fbc6a88af1 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:01.546430 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-446f4d76-31ad-4096-b32e-51fbc6a88af1 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:01.551027 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-446f4d76-31ad-4096-b32e-51fbc6a88af1 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:01.551742 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-446f4d76-31ad-4096-b32e-51fbc6a88af1 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:02.111767 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-5c8b7b32-c65a-486a-8826-7a8103864439 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd4144b5388b46a989a41a2f556bb508', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a90d4bcd25ff4bf68bdc194180fbbe09', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:18:02.124084 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-5c8b7b32-c65a-486a-8826-7a8103864439 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] 10.222.0.16 "GET /compute/v2.1/servers/7e4b142d-f489-4220-adc3-7b348dc14ce9" status: 200 len: 1506 microversion: 2.1 time: 0.744396 Jul 16 21:18:02.124084 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1166/2260] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:18:01 2022] GET /compute/v2.1/servers/7e4b142d-f489-4220-adc3-7b348dc14ce9 => generated 1506 bytes in 749 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:18:02.142178 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-d1f47100-9739-4b60-ad5e-9788293e379e tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:02.147120 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-d1f47100-9739-4b60-ad5e-9788293e379e tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] [instance: 7e4b142d-f489-4220-adc3-7b348dc14ce9] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:18:02.156525 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d1f47100-9739-4b60-ad5e-9788293e379e tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:02.157599 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d1f47100-9739-4b60-ad5e-9788293e379e tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:02.159260 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d1f47100-9739-4b60-ad5e-9788293e379e tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:02.159979 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d1f47100-9739-4b60-ad5e-9788293e379e tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:02.194696 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-d1f47100-9739-4b60-ad5e-9788293e379e tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] [instance: 7e4b142d-f489-4220-adc3-7b348dc14ce9] Going to try to terminate instance {{(pid=138680) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 16 21:18:02.227529 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-446f4d76-31ad-4096-b32e-51fbc6a88af1 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd0008fffad8e4708b2c01e03da92c47d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e353258f9c3048ef84683294e2b8579d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:18:02.243463 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-446f4d76-31ad-4096-b32e-51fbc6a88af1 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] 10.222.0.16 "GET /compute/v2.1/servers/b8565a82-250b-4f64-9d90-98b368e4bd57" status: 200 len: 1316 microversion: 2.1 time: 0.711341 Jul 16 21:18:02.244583 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1095/2261] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:18:01 2022] GET /compute/v2.1/servers/b8565a82-250b-4f64-9d90-98b368e4bd57 => generated 1316 bytes in 713 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:18:02.250324 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-b96ca3ad-4f22-4e4f-9857-f194e7ba4be1 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:02.250931 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-b96ca3ad-4f22-4e4f-9857-f194e7ba4be1 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] [instance: ed8dcb97-7546-45ef-9e9d-a9bb03a1b419] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:18:02.257968 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b96ca3ad-4f22-4e4f-9857-f194e7ba4be1 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:02.258698 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b96ca3ad-4f22-4e4f-9857-f194e7ba4be1 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:02.259708 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b96ca3ad-4f22-4e4f-9857-f194e7ba4be1 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:02.260409 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b96ca3ad-4f22-4e4f-9857-f194e7ba4be1 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:02.593824 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-b96ca3ad-4f22-4e4f-9857-f194e7ba4be1 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'daf3c9e6634e4ae1bfb492eae1f44c93', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '24c64db8082648c8837c16352252f96d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:18:02.605056 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-b96ca3ad-4f22-4e4f-9857-f194e7ba4be1 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] 10.222.0.16 "GET /compute/v2.1/servers/ed8dcb97-7546-45ef-9e9d-a9bb03a1b419" status: 200 len: 1308 microversion: 2.1 time: 0.359159 Jul 16 21:18:02.619980 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1096/2262] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:18:02 2022] GET /compute/v2.1/servers/ed8dcb97-7546-45ef-9e9d-a9bb03a1b419 => generated 1308 bytes in 375 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:18:02.642420 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-85263085-0634-4822-bdf1-b7052aa3195b tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:02.643359 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-85263085-0634-4822-bdf1-b7052aa3195b tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] [instance: 720ef2b0-e7b3-4b36-b613-1b73d2bae2e9] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:18:02.653829 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-85263085-0634-4822-bdf1-b7052aa3195b tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:02.654290 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-85263085-0634-4822-bdf1-b7052aa3195b tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:02.655131 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-85263085-0634-4822-bdf1-b7052aa3195b tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:02.664230 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-d1f47100-9739-4b60-ad5e-9788293e379e tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] 10.222.0.16 "DELETE /compute/v2.1/servers/7e4b142d-f489-4220-adc3-7b348dc14ce9" status: 204 len: 0 microversion: 2.1 time: 0.525543 Jul 16 21:18:02.666788 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1167/2263] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:18:02 2022] DELETE /compute/v2.1/servers/7e4b142d-f489-4220-adc3-7b348dc14ce9 => generated 0 bytes in 529 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 16 21:18:02.669034 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-85263085-0634-4822-bdf1-b7052aa3195b tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.013s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:02.674020 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-18b4c1c8-0fdc-40cf-a36e-93c3e465ec1d tempest-ServersNegativeTestMultiTenantJSON-1135003248 tempest-ServersNegativeTestMultiTenantJSON-1135003248-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:02.674988 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-18b4c1c8-0fdc-40cf-a36e-93c3e465ec1d tempest-ServersNegativeTestMultiTenantJSON-1135003248 tempest-ServersNegativeTestMultiTenantJSON-1135003248-project] [instance: 43169d4f-1214-419a-aa1e-def65182ed5a] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:18:02.682216 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-18b4c1c8-0fdc-40cf-a36e-93c3e465ec1d tempest-ServersNegativeTestMultiTenantJSON-1135003248 tempest-ServersNegativeTestMultiTenantJSON-1135003248-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:02.682878 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-18b4c1c8-0fdc-40cf-a36e-93c3e465ec1d tempest-ServersNegativeTestMultiTenantJSON-1135003248 tempest-ServersNegativeTestMultiTenantJSON-1135003248-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:02.685240 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-18b4c1c8-0fdc-40cf-a36e-93c3e465ec1d tempest-ServersNegativeTestMultiTenantJSON-1135003248 tempest-ServersNegativeTestMultiTenantJSON-1135003248-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:02.686875 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-18b4c1c8-0fdc-40cf-a36e-93c3e465ec1d tempest-ServersNegativeTestMultiTenantJSON-1135003248 tempest-ServersNegativeTestMultiTenantJSON-1135003248-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:02.885496 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-18b4c1c8-0fdc-40cf-a36e-93c3e465ec1d tempest-ServersNegativeTestMultiTenantJSON-1135003248 tempest-ServersNegativeTestMultiTenantJSON-1135003248-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '50cdc134e712487196f6623c5cf14592', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ba229aaef0ec4ddcadd1d940376c31ac', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:18:02.900172 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-18b4c1c8-0fdc-40cf-a36e-93c3e465ec1d tempest-ServersNegativeTestMultiTenantJSON-1135003248 tempest-ServersNegativeTestMultiTenantJSON-1135003248-project] 10.222.0.16 "GET /compute/v2.1/servers/43169d4f-1214-419a-aa1e-def65182ed5a" status: 200 len: 1286 microversion: 2.1 time: 0.231217 Jul 16 21:18:02.903490 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1168/2264] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:18:02 2022] GET /compute/v2.1/servers/43169d4f-1214-419a-aa1e-def65182ed5a => generated 1286 bytes in 236 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:18:02.913933 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-ca163d89-74ae-4bfc-af61-c94e1a243e1a tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Calling method '.version_select at 0x7fdd9c6b9700>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:02.984873 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-ca163d89-74ae-4bfc-af61-c94e1a243e1a tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] 10.222.0.16 "GET /compute/v2.1/images/1f9a9864-79f1-4d21-84c6-8d017682af53" status: 200 len: 1454 microversion: 2.1 time: 0.073659 Jul 16 21:18:02.985187 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1169/2265] 10.222.0.16 () {58 vars in 1232 bytes} [Sat Jul 16 21:18:02 2022] GET /compute/v2.1/images/1f9a9864-79f1-4d21-84c6-8d017682af53 => generated 1454 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:18:02.994148 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-34fd4ca8-bbad-482a-ae27-e2014ab6da22 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerDiskConfigTestJSON-server-1068151968", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452", "networks": [{"uuid": "b137fafe-f0b3-4d44-9dfc-add7ee4d76f5"}]}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:18:03.081870 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.network.neutron [None req-34fd4ca8-bbad-482a-ae27-e2014ab6da22 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] validate_networks() for [('b137fafe-f0b3-4d44-9dfc-add7ee4d76f5', None, None, None, None, None)] {{(pid=138680) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 16 21:18:03.152960 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-85263085-0634-4822-bdf1-b7052aa3195b tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f5bc1f2209f4eb186d80e385b67c774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '65d5bcfa02af4e8284ba1342197e2860', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:18:03.162983 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-85263085-0634-4822-bdf1-b7052aa3195b tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] 10.222.0.16 "GET /compute/v2.1/servers/720ef2b0-e7b3-4b36-b613-1b73d2bae2e9" status: 200 len: 1313 microversion: 2.1 time: 0.540523 Jul 16 21:18:03.163938 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1097/2266] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:18:02 2022] GET /compute/v2.1/servers/720ef2b0-e7b3-4b36-b613-1b73d2bae2e9 => generated 1313 bytes in 543 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:18:03.265250 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-6e1d99ee-b696-4a78-bdea-96f04aba4171 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:03.266589 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-6e1d99ee-b696-4a78-bdea-96f04aba4171 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] [instance: b8565a82-250b-4f64-9d90-98b368e4bd57] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:18:03.280783 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6e1d99ee-b696-4a78-bdea-96f04aba4171 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:03.281253 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6e1d99ee-b696-4a78-bdea-96f04aba4171 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:03.282270 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6e1d99ee-b696-4a78-bdea-96f04aba4171 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:03.282719 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6e1d99ee-b696-4a78-bdea-96f04aba4171 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:03.583381 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-6e1d99ee-b696-4a78-bdea-96f04aba4171 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd0008fffad8e4708b2c01e03da92c47d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e353258f9c3048ef84683294e2b8579d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:18:03.593573 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-6e1d99ee-b696-4a78-bdea-96f04aba4171 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] 10.222.0.16 "GET /compute/v2.1/servers/b8565a82-250b-4f64-9d90-98b368e4bd57" status: 200 len: 1316 microversion: 2.1 time: 0.335624 Jul 16 21:18:03.604751 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1098/2267] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:18:03 2022] GET /compute/v2.1/servers/b8565a82-250b-4f64-9d90-98b368e4bd57 => generated 1316 bytes in 347 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:18:03.628294 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-62d12259-bdf1-4997-a49a-40ba1bbad559 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:03.629431 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-62d12259-bdf1-4997-a49a-40ba1bbad559 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] [instance: ed8dcb97-7546-45ef-9e9d-a9bb03a1b419] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:18:03.637644 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-62d12259-bdf1-4997-a49a-40ba1bbad559 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:03.638364 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-62d12259-bdf1-4997-a49a-40ba1bbad559 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:03.639378 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-62d12259-bdf1-4997-a49a-40ba1bbad559 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:03.640001 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-62d12259-bdf1-4997-a49a-40ba1bbad559 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:03.829788 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-34fd4ca8-bbad-482a-ae27-e2014ab6da22 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Flavor limits 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 16 21:18:03.830698 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-34fd4ca8-bbad-482a-ae27-e2014ab6da22 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Image limits 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 16 21:18:03.831301 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-34fd4ca8-bbad-482a-ae27-e2014ab6da22 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Flavor pref 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 16 21:18:03.831789 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-34fd4ca8-bbad-482a-ae27-e2014ab6da22 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Image pref 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 16 21:18:03.832290 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-34fd4ca8-bbad-482a-ae27-e2014ab6da22 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 16 21:18:03.842527 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell0:SELECT=55 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:18:03.845696 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-34fd4ca8-bbad-482a-ae27-e2014ab6da22 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:03.852458 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-34fd4ca8-bbad-482a-ae27-e2014ab6da22 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:03.854348 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-34fd4ca8-bbad-482a-ae27-e2014ab6da22 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:03.855113 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-34fd4ca8-bbad-482a-ae27-e2014ab6da22 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:03.879960 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-62d12259-bdf1-4997-a49a-40ba1bbad559 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'daf3c9e6634e4ae1bfb492eae1f44c93', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '24c64db8082648c8837c16352252f96d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:18:03.885481 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-34fd4ca8-bbad-482a-ae27-e2014ab6da22 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Getting quotas for project a90d4bcd25ff4bf68bdc194180fbbe09. Resources: {'ram', 'instances', 'cores'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:18:03.890852 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-62d12259-bdf1-4997-a49a-40ba1bbad559 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] 10.222.0.16 "GET /compute/v2.1/servers/ed8dcb97-7546-45ef-9e9d-a9bb03a1b419" status: 200 len: 1308 microversion: 2.1 time: 0.268602 Jul 16 21:18:03.891885 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1099/2268] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:18:03 2022] GET /compute/v2.1/servers/ed8dcb97-7546-45ef-9e9d-a9bb03a1b419 => generated 1308 bytes in 270 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:18:03.900063 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-34fd4ca8-bbad-482a-ae27-e2014ab6da22 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Getting quotas for user dd4144b5388b46a989a41a2f556bb508 and project a90d4bcd25ff4bf68bdc194180fbbe09. Resources: {'ram', 'instances', 'cores'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:18:03.916980 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-34fd4ca8-bbad-482a-ae27-e2014ab6da22 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Going to run 1 instances... {{(pid=138680) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 16 21:18:03.919427 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-34fd4ca8-bbad-482a-ae27-e2014ab6da22 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] [instance: 6baac85b-f81c-4e3a-9f96-633cffccfd74] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138680) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 16 21:18:03.922146 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-5a69d7bd-221a-402d-8e9d-d36adf552ede tempest-ServersNegativeTestMultiTenantJSON-1135003248 tempest-ServersNegativeTestMultiTenantJSON-1135003248-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:03.926254 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-5a69d7bd-221a-402d-8e9d-d36adf552ede tempest-ServersNegativeTestMultiTenantJSON-1135003248 tempest-ServersNegativeTestMultiTenantJSON-1135003248-project] [instance: 43169d4f-1214-419a-aa1e-def65182ed5a] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:18:03.939201 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5a69d7bd-221a-402d-8e9d-d36adf552ede tempest-ServersNegativeTestMultiTenantJSON-1135003248 tempest-ServersNegativeTestMultiTenantJSON-1135003248-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:03.939671 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5a69d7bd-221a-402d-8e9d-d36adf552ede tempest-ServersNegativeTestMultiTenantJSON-1135003248 tempest-ServersNegativeTestMultiTenantJSON-1135003248-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:03.941504 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5a69d7bd-221a-402d-8e9d-d36adf552ede tempest-ServersNegativeTestMultiTenantJSON-1135003248 tempest-ServersNegativeTestMultiTenantJSON-1135003248-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:03.942152 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5a69d7bd-221a-402d-8e9d-d36adf552ede tempest-ServersNegativeTestMultiTenantJSON-1135003248 tempest-ServersNegativeTestMultiTenantJSON-1135003248-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:04.101245 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-34fd4ca8-bbad-482a-ae27-e2014ab6da22 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 1.114735 Jul 16 21:18:04.102309 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1170/2269] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 21:18:02 2022] POST /compute/v2.1/servers => generated 380 bytes in 1117 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 16 21:18:04.112039 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-cb2455e5-d2fd-499d-824a-c8b79bc8208f tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Calling method '.version_select at 0x7fdd9c6b6f70>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:04.176032 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-cb2455e5-d2fd-499d-824a-c8b79bc8208f tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] 10.222.0.16 "GET /compute/v2.1/images/1f9a9864-79f1-4d21-84c6-8d017682af53" status: 200 len: 1454 microversion: 2.1 time: 0.071816 Jul 16 21:18:04.176929 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1171/2270] 10.222.0.16 () {58 vars in 1232 bytes} [Sat Jul 16 21:18:04 2022] GET /compute/v2.1/images/1f9a9864-79f1-4d21-84c6-8d017682af53 => generated 1454 bytes in 74 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:18:04.189850 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-70067ee3-c760-4ebd-90bf-85e0e623d1b5 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:04.190489 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-70067ee3-c760-4ebd-90bf-85e0e623d1b5 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] [instance: 6baac85b-f81c-4e3a-9f96-633cffccfd74] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:18:04.427168 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-70067ee3-c760-4ebd-90bf-85e0e623d1b5 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd4144b5388b46a989a41a2f556bb508', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a90d4bcd25ff4bf68bdc194180fbbe09', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:18:04.439053 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-70067ee3-c760-4ebd-90bf-85e0e623d1b5 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] 10.222.0.16 "GET /compute/v2.1/servers/6baac85b-f81c-4e3a-9f96-633cffccfd74" status: 200 len: 1218 microversion: 2.1 time: 0.260748 Jul 16 21:18:04.449573 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1172/2271] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:18:04 2022] GET /compute/v2.1/servers/6baac85b-f81c-4e3a-9f96-633cffccfd74 => generated 1218 bytes in 272 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:18:04.456298 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-af853136-aaf0-4d41-823c-34e711cf5ca6 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:04.456298 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-af853136-aaf0-4d41-823c-34e711cf5ca6 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] [instance: 720ef2b0-e7b3-4b36-b613-1b73d2bae2e9] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:18:04.462995 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-af853136-aaf0-4d41-823c-34e711cf5ca6 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:04.463492 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-af853136-aaf0-4d41-823c-34e711cf5ca6 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:04.464315 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-af853136-aaf0-4d41-823c-34e711cf5ca6 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:04.465378 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-af853136-aaf0-4d41-823c-34e711cf5ca6 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:04.588848 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-5a69d7bd-221a-402d-8e9d-d36adf552ede tempest-ServersNegativeTestMultiTenantJSON-1135003248 tempest-ServersNegativeTestMultiTenantJSON-1135003248-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '50cdc134e712487196f6623c5cf14592', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ba229aaef0ec4ddcadd1d940376c31ac', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:18:04.601231 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-5a69d7bd-221a-402d-8e9d-d36adf552ede tempest-ServersNegativeTestMultiTenantJSON-1135003248 tempest-ServersNegativeTestMultiTenantJSON-1135003248-project] 10.222.0.16 "GET /compute/v2.1/servers/43169d4f-1214-419a-aa1e-def65182ed5a" status: 200 len: 1328 microversion: 2.1 time: 0.683785 Jul 16 21:18:04.605981 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1100/2272] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:18:03 2022] GET /compute/v2.1/servers/43169d4f-1214-419a-aa1e-def65182ed5a => generated 1328 bytes in 689 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:18:04.613995 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-fdd09ff7-ed5c-4b59-90fd-1f354945c449 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:04.616366 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-fdd09ff7-ed5c-4b59-90fd-1f354945c449 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] [instance: b8565a82-250b-4f64-9d90-98b368e4bd57] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:18:04.626701 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fdd09ff7-ed5c-4b59-90fd-1f354945c449 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:04.627451 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fdd09ff7-ed5c-4b59-90fd-1f354945c449 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:04.628417 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fdd09ff7-ed5c-4b59-90fd-1f354945c449 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:04.629051 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fdd09ff7-ed5c-4b59-90fd-1f354945c449 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:05.502879 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-af853136-aaf0-4d41-823c-34e711cf5ca6 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f5bc1f2209f4eb186d80e385b67c774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '65d5bcfa02af4e8284ba1342197e2860', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:18:05.522829 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-af853136-aaf0-4d41-823c-34e711cf5ca6 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] 10.222.0.16 "GET /compute/v2.1/servers/720ef2b0-e7b3-4b36-b613-1b73d2bae2e9" status: 200 len: 1313 microversion: 2.1 time: 1.072008 Jul 16 21:18:05.535189 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1173/2273] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:18:04 2022] GET /compute/v2.1/servers/720ef2b0-e7b3-4b36-b613-1b73d2bae2e9 => generated 1313 bytes in 1085 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:18:05.540615 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-592c787f-ec58-486e-98c8-939edb43d9e5 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:05.541701 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-592c787f-ec58-486e-98c8-939edb43d9e5 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] [instance: ed8dcb97-7546-45ef-9e9d-a9bb03a1b419] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:18:05.549358 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-592c787f-ec58-486e-98c8-939edb43d9e5 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:05.550194 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-592c787f-ec58-486e-98c8-939edb43d9e5 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:05.551252 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-592c787f-ec58-486e-98c8-939edb43d9e5 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:05.552023 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-592c787f-ec58-486e-98c8-939edb43d9e5 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:05.654736 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-fdd09ff7-ed5c-4b59-90fd-1f354945c449 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd0008fffad8e4708b2c01e03da92c47d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e353258f9c3048ef84683294e2b8579d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:18:05.709417 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-fdd09ff7-ed5c-4b59-90fd-1f354945c449 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] 10.222.0.16 "GET /compute/v2.1/servers/b8565a82-250b-4f64-9d90-98b368e4bd57" status: 200 len: 1505 microversion: 2.1 time: 1.101466 Jul 16 21:18:05.712455 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1101/2274] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:18:04 2022] GET /compute/v2.1/servers/b8565a82-250b-4f64-9d90-98b368e4bd57 => generated 1505 bytes in 1105 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:18:05.733504 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-5d7332ea-bfe1-486d-8254-3f81aed72ee2 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Calling method '.version_select at 0x7fdd9cda2550>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:05.829875 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-5d7332ea-bfe1-486d-8254-3f81aed72ee2 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] 10.222.0.16 "GET /compute/v2.1/images/1f9a9864-79f1-4d21-84c6-8d017682af53" status: 200 len: 1454 microversion: 2.1 time: 0.115945 Jul 16 21:18:05.836709 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1102/2275] 10.222.0.16 () {58 vars in 1232 bytes} [Sat Jul 16 21:18:05 2022] GET /compute/v2.1/images/1f9a9864-79f1-4d21-84c6-8d017682af53 => generated 1454 bytes in 123 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:18:05.843591 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-2e70be4c-6015-4f47-b23d-92659ad62d82 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:05.844427 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-2e70be4c-6015-4f47-b23d-92659ad62d82 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] [instance: 6baac85b-f81c-4e3a-9f96-633cffccfd74] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:18:05.851877 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2e70be4c-6015-4f47-b23d-92659ad62d82 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:05.852694 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2e70be4c-6015-4f47-b23d-92659ad62d82 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:05.853782 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2e70be4c-6015-4f47-b23d-92659ad62d82 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:05.854454 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2e70be4c-6015-4f47-b23d-92659ad62d82 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:05.862599 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=100 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:18:06.075540 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-2e70be4c-6015-4f47-b23d-92659ad62d82 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd4144b5388b46a989a41a2f556bb508', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a90d4bcd25ff4bf68bdc194180fbbe09', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:18:06.083581 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-2e70be4c-6015-4f47-b23d-92659ad62d82 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] 10.222.0.16 "GET /compute/v2.1/servers/6baac85b-f81c-4e3a-9f96-633cffccfd74" status: 200 len: 1222 microversion: 2.1 time: 0.245279 Jul 16 21:18:06.084156 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1103/2276] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:18:05 2022] GET /compute/v2.1/servers/6baac85b-f81c-4e3a-9f96-633cffccfd74 => generated 1222 bytes in 247 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:18:06.088661 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-32cb2836-1924-4293-aa2f-18960bf1b0ba tempest-ServersNegativeTestMultiTenantJSON-1135003248 tempest-ServersNegativeTestMultiTenantJSON-1135003248-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:06.089312 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-32cb2836-1924-4293-aa2f-18960bf1b0ba tempest-ServersNegativeTestMultiTenantJSON-1135003248 tempest-ServersNegativeTestMultiTenantJSON-1135003248-project] [instance: 43169d4f-1214-419a-aa1e-def65182ed5a] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:18:06.095858 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-32cb2836-1924-4293-aa2f-18960bf1b0ba tempest-ServersNegativeTestMultiTenantJSON-1135003248 tempest-ServersNegativeTestMultiTenantJSON-1135003248-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:06.097283 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-32cb2836-1924-4293-aa2f-18960bf1b0ba tempest-ServersNegativeTestMultiTenantJSON-1135003248 tempest-ServersNegativeTestMultiTenantJSON-1135003248-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:06.098259 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-32cb2836-1924-4293-aa2f-18960bf1b0ba tempest-ServersNegativeTestMultiTenantJSON-1135003248 tempest-ServersNegativeTestMultiTenantJSON-1135003248-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:06.098868 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-32cb2836-1924-4293-aa2f-18960bf1b0ba tempest-ServersNegativeTestMultiTenantJSON-1135003248 tempest-ServersNegativeTestMultiTenantJSON-1135003248-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:06.119576 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-592c787f-ec58-486e-98c8-939edb43d9e5 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'daf3c9e6634e4ae1bfb492eae1f44c93', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '24c64db8082648c8837c16352252f96d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:18:06.130317 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-592c787f-ec58-486e-98c8-939edb43d9e5 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] 10.222.0.16 "GET /compute/v2.1/servers/ed8dcb97-7546-45ef-9e9d-a9bb03a1b419" status: 200 len: 1308 microversion: 2.1 time: 0.593252 Jul 16 21:18:06.131555 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1174/2277] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:18:05 2022] GET /compute/v2.1/servers/ed8dcb97-7546-45ef-9e9d-a9bb03a1b419 => generated 1308 bytes in 595 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:18:06.141545 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-a9b3029e-87e1-4975-9b12-dfe9cd123a6e tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Action: 'action', calling method: >, body: {"addSecurityGroup": {"name": "tempest-SecurityGroupsTestJSON-securitygroup-1978190092"}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:18:06.141545 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-a9b3029e-87e1-4975-9b12-dfe9cd123a6e tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] [instance: b8565a82-250b-4f64-9d90-98b368e4bd57] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:18:06.146631 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a9b3029e-87e1-4975-9b12-dfe9cd123a6e tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:06.147205 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a9b3029e-87e1-4975-9b12-dfe9cd123a6e tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:06.147987 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a9b3029e-87e1-4975-9b12-dfe9cd123a6e tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:06.148429 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a9b3029e-87e1-4975-9b12-dfe9cd123a6e tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:06.152900 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:18:06.544375 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.network.security_group_api [None req-a9b3029e-87e1-4975-9b12-dfe9cd123a6e tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Adding security group 83887366-b549-457d-8fc6-28eee153b69e to port 54d1cb01-bab2-432d-8666-66691fd8c6de Jul 16 21:18:06.567203 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell0:SELECT=8 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:18:06.580739 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-32cb2836-1924-4293-aa2f-18960bf1b0ba tempest-ServersNegativeTestMultiTenantJSON-1135003248 tempest-ServersNegativeTestMultiTenantJSON-1135003248-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '50cdc134e712487196f6623c5cf14592', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ba229aaef0ec4ddcadd1d940376c31ac', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:18:06.592260 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-32cb2836-1924-4293-aa2f-18960bf1b0ba tempest-ServersNegativeTestMultiTenantJSON-1135003248 tempest-ServersNegativeTestMultiTenantJSON-1135003248-project] 10.222.0.16 "GET /compute/v2.1/servers/43169d4f-1214-419a-aa1e-def65182ed5a" status: 200 len: 1328 microversion: 2.1 time: 0.507028 Jul 16 21:18:06.593196 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1104/2278] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:18:06 2022] GET /compute/v2.1/servers/43169d4f-1214-419a-aa1e-def65182ed5a => generated 1328 bytes in 508 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:18:06.616855 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-06b69548-82b2-4540-9b55-e90a29e378c4 req-04a03c09-b89d-4da3-ad13-50b660b1ab61 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "7e4b142d-f489-4220-adc3-7b348dc14ce9", "name": "network-vif-deleted", "tag": "d10b6c2f-f386-410e-a6a2-fbd06a8fceaf"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:18:06.634375 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-06b69548-82b2-4540-9b55-e90a29e378c4 req-04a03c09-b89d-4da3-ad13-50b660b1ab61 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:06.636516 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-06b69548-82b2-4540-9b55-e90a29e378c4 req-04a03c09-b89d-4da3-ad13-50b660b1ab61 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:06.668655 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-06b69548-82b2-4540-9b55-e90a29e378c4 req-04a03c09-b89d-4da3-ad13-50b660b1ab61 service nova] Creating event network-vif-deleted:d10b6c2f-f386-410e-a6a2-fbd06a8fceaf for instance 7e4b142d-f489-4220-adc3-7b348dc14ce9 on nch1-master-1 Jul 16 21:18:06.691647 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-06b69548-82b2-4540-9b55-e90a29e378c4 req-04a03c09-b89d-4da3-ad13-50b660b1ab61 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.090661 Jul 16 21:18:06.693038 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1105/2279] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:18:06 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:18:06.715200 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-f08334a0-e13d-4f82-98c9-7c792c7bbb30 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:06.715200 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-f08334a0-e13d-4f82-98c9-7c792c7bbb30 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] [instance: 720ef2b0-e7b3-4b36-b613-1b73d2bae2e9] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:18:06.722641 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f08334a0-e13d-4f82-98c9-7c792c7bbb30 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:06.723493 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f08334a0-e13d-4f82-98c9-7c792c7bbb30 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:06.724501 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f08334a0-e13d-4f82-98c9-7c792c7bbb30 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:06.725177 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f08334a0-e13d-4f82-98c9-7c792c7bbb30 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:07.248832 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-f08334a0-e13d-4f82-98c9-7c792c7bbb30 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f5bc1f2209f4eb186d80e385b67c774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '65d5bcfa02af4e8284ba1342197e2860', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:18:07.263466 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-f08334a0-e13d-4f82-98c9-7c792c7bbb30 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] 10.222.0.16 "GET /compute/v2.1/servers/720ef2b0-e7b3-4b36-b613-1b73d2bae2e9" status: 200 len: 1313 microversion: 2.1 time: 0.566193 Jul 16 21:18:07.283770 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1106/2280] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:18:06 2022] GET /compute/v2.1/servers/720ef2b0-e7b3-4b36-b613-1b73d2bae2e9 => generated 1313 bytes in 589 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:18:07.291262 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-e618c663-aea5-4643-8871-3ec426b7894c req-e2cfef24-1be3-4f94-968d-c5da14c63149 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "ed8dcb97-7546-45ef-9e9d-a9bb03a1b419", "name": "network-vif-plugged", "status": "completed", "tag": "033ca6ad-5781-4719-b3b5-986dcfd7de3d"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:18:07.319239 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-e618c663-aea5-4643-8871-3ec426b7894c req-e2cfef24-1be3-4f94-968d-c5da14c63149 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:07.319239 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-e618c663-aea5-4643-8871-3ec426b7894c req-e2cfef24-1be3-4f94-968d-c5da14c63149 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:07.333671 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-e618c663-aea5-4643-8871-3ec426b7894c req-e2cfef24-1be3-4f94-968d-c5da14c63149 service nova] Creating event network-vif-plugged:033ca6ad-5781-4719-b3b5-986dcfd7de3d for instance ed8dcb97-7546-45ef-9e9d-a9bb03a1b419 on nch2-master-1 Jul 16 21:18:07.340096 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-e618c663-aea5-4643-8871-3ec426b7894c req-e2cfef24-1be3-4f94-968d-c5da14c63149 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.053281 Jul 16 21:18:07.354393 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1107/2281] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:18:07 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 69 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:18:07.363900 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-ea257937-869f-479f-9bb2-2edc669a5f30 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Calling method '.version_select at 0x7fdd9cf90550>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:07.485857 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-ea257937-869f-479f-9bb2-2edc669a5f30 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] 10.222.0.16 "GET /compute/v2.1/images/1f9a9864-79f1-4d21-84c6-8d017682af53" status: 200 len: 1454 microversion: 2.1 time: 0.129703 Jul 16 21:18:07.486462 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1108/2282] 10.222.0.16 () {58 vars in 1232 bytes} [Sat Jul 16 21:18:07 2022] GET /compute/v2.1/images/1f9a9864-79f1-4d21-84c6-8d017682af53 => generated 1454 bytes in 131 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:18:07.494690 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-f8c038b2-fcd2-424c-b7f9-5ba21203b1bf tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:07.496037 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-f8c038b2-fcd2-424c-b7f9-5ba21203b1bf tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] [instance: 6baac85b-f81c-4e3a-9f96-633cffccfd74] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:18:07.507724 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f8c038b2-fcd2-424c-b7f9-5ba21203b1bf tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:07.508409 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f8c038b2-fcd2-424c-b7f9-5ba21203b1bf tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:07.510386 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f8c038b2-fcd2-424c-b7f9-5ba21203b1bf tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:07.511039 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f8c038b2-fcd2-424c-b7f9-5ba21203b1bf tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:07.732905 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-f8c038b2-fcd2-424c-b7f9-5ba21203b1bf tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd4144b5388b46a989a41a2f556bb508', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a90d4bcd25ff4bf68bdc194180fbbe09', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:18:07.745546 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-f8c038b2-fcd2-424c-b7f9-5ba21203b1bf tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] 10.222.0.16 "GET /compute/v2.1/servers/6baac85b-f81c-4e3a-9f96-633cffccfd74" status: 200 len: 1270 microversion: 2.1 time: 0.257727 Jul 16 21:18:07.758244 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1109/2283] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:18:07 2022] GET /compute/v2.1/servers/6baac85b-f81c-4e3a-9f96-633cffccfd74 => generated 1270 bytes in 271 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:18:07.765092 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-58dfcb8b-8adf-4e2f-8a4f-b4b9796c3c96 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:07.765907 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-58dfcb8b-8adf-4e2f-8a4f-b4b9796c3c96 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] [instance: ed8dcb97-7546-45ef-9e9d-a9bb03a1b419] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:18:07.784766 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-58dfcb8b-8adf-4e2f-8a4f-b4b9796c3c96 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:07.785978 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-58dfcb8b-8adf-4e2f-8a4f-b4b9796c3c96 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.011s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:07.787326 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-58dfcb8b-8adf-4e2f-8a4f-b4b9796c3c96 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:07.787976 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-58dfcb8b-8adf-4e2f-8a4f-b4b9796c3c96 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:08.187685 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-58dfcb8b-8adf-4e2f-8a4f-b4b9796c3c96 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'daf3c9e6634e4ae1bfb492eae1f44c93', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '24c64db8082648c8837c16352252f96d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:18:08.195865 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-58dfcb8b-8adf-4e2f-8a4f-b4b9796c3c96 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] 10.222.0.16 "GET /compute/v2.1/servers/ed8dcb97-7546-45ef-9e9d-a9bb03a1b419" status: 200 len: 1308 microversion: 2.1 time: 0.436043 Jul 16 21:18:08.196654 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1110/2284] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:18:07 2022] GET /compute/v2.1/servers/ed8dcb97-7546-45ef-9e9d-a9bb03a1b419 => generated 1308 bytes in 438 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:18:08.204072 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-7729c04b-791b-4774-a48d-c69728c41dcf tempest-ServersNegativeTestMultiTenantJSON-1135003248 tempest-ServersNegativeTestMultiTenantJSON-1135003248-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:08.204692 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-7729c04b-791b-4774-a48d-c69728c41dcf tempest-ServersNegativeTestMultiTenantJSON-1135003248 tempest-ServersNegativeTestMultiTenantJSON-1135003248-project] [instance: 43169d4f-1214-419a-aa1e-def65182ed5a] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:18:08.211423 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7729c04b-791b-4774-a48d-c69728c41dcf tempest-ServersNegativeTestMultiTenantJSON-1135003248 tempest-ServersNegativeTestMultiTenantJSON-1135003248-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:08.211873 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7729c04b-791b-4774-a48d-c69728c41dcf tempest-ServersNegativeTestMultiTenantJSON-1135003248 tempest-ServersNegativeTestMultiTenantJSON-1135003248-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:08.212693 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7729c04b-791b-4774-a48d-c69728c41dcf tempest-ServersNegativeTestMultiTenantJSON-1135003248 tempest-ServersNegativeTestMultiTenantJSON-1135003248-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:08.213126 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7729c04b-791b-4774-a48d-c69728c41dcf tempest-ServersNegativeTestMultiTenantJSON-1135003248 tempest-ServersNegativeTestMultiTenantJSON-1135003248-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:08.353073 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-a9b3029e-87e1-4975-9b12-dfe9cd123a6e tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] 10.222.0.16 "POST /compute/v2.1/servers/b8565a82-250b-4f64-9d90-98b368e4bd57/action" status: 202 len: 0 microversion: 2.1 time: 2.218208 Jul 16 21:18:08.353073 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1175/2285] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 21:18:06 2022] POST /compute/v2.1/servers/b8565a82-250b-4f64-9d90-98b368e4bd57/action => generated 0 bytes in 2220 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 21:18:08.590370 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-7729c04b-791b-4774-a48d-c69728c41dcf tempest-ServersNegativeTestMultiTenantJSON-1135003248 tempest-ServersNegativeTestMultiTenantJSON-1135003248-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '50cdc134e712487196f6623c5cf14592', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ba229aaef0ec4ddcadd1d940376c31ac', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:18:08.613208 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-7729c04b-791b-4774-a48d-c69728c41dcf tempest-ServersNegativeTestMultiTenantJSON-1135003248 tempest-ServersNegativeTestMultiTenantJSON-1135003248-project] 10.222.0.16 "GET /compute/v2.1/servers/43169d4f-1214-419a-aa1e-def65182ed5a" status: 200 len: 1328 microversion: 2.1 time: 0.415061 Jul 16 21:18:08.624735 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1111/2286] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:18:08 2022] GET /compute/v2.1/servers/43169d4f-1214-419a-aa1e-def65182ed5a => generated 1328 bytes in 427 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:18:08.631712 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-662f9ee4-553c-42a4-8a69-80acadc409ad tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:08.632424 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-662f9ee4-553c-42a4-8a69-80acadc409ad tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] [instance: 720ef2b0-e7b3-4b36-b613-1b73d2bae2e9] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:18:08.651624 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-662f9ee4-553c-42a4-8a69-80acadc409ad tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:08.652178 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-662f9ee4-553c-42a4-8a69-80acadc409ad tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:08.653025 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-662f9ee4-553c-42a4-8a69-80acadc409ad tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:08.653480 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-662f9ee4-553c-42a4-8a69-80acadc409ad tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:08.667999 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-4d5085e1-2724-4524-9b68-54baabd1d3bb tempest-SecurityGroupsTestAdminJSON-1430108405 tempest-SecurityGroupsTestAdminJSON-1430108405-project] Action: 'create', calling method: .version_select at 0x7fdd9c6b9310>, body: {"security_group": {"name": "tempest-securitygroup-1993657642", "description": "tempest-description-1110875975"}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:18:09.142690 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-662f9ee4-553c-42a4-8a69-80acadc409ad tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f5bc1f2209f4eb186d80e385b67c774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '65d5bcfa02af4e8284ba1342197e2860', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:18:09.150506 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-662f9ee4-553c-42a4-8a69-80acadc409ad tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] 10.222.0.16 "GET /compute/v2.1/servers/720ef2b0-e7b3-4b36-b613-1b73d2bae2e9" status: 200 len: 1313 microversion: 2.1 time: 0.524797 Jul 16 21:18:09.151240 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1112/2287] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:18:08 2022] GET /compute/v2.1/servers/720ef2b0-e7b3-4b36-b613-1b73d2bae2e9 => generated 1313 bytes in 526 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:18:09.157032 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-766d4155-630d-4629-b860-f7f096100df9 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Action: 'action', calling method: >, body: {"addSecurityGroup": {"name": "tempest-SecurityGroupsTestJSON-securitygroup-2071679724"}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:18:09.157587 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-766d4155-630d-4629-b860-f7f096100df9 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] [instance: b8565a82-250b-4f64-9d90-98b368e4bd57] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:18:09.168084 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-766d4155-630d-4629-b860-f7f096100df9 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:09.168084 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-766d4155-630d-4629-b860-f7f096100df9 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:09.168084 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-766d4155-630d-4629-b860-f7f096100df9 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:09.168084 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-766d4155-630d-4629-b860-f7f096100df9 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:09.299527 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell0:SELECT=32 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:18:09.509134 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.network.security_group_api [None req-766d4155-630d-4629-b860-f7f096100df9 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Adding security group ca1da2f5-509d-4012-b094-b91b1b3d0f60 to port 54d1cb01-bab2-432d-8666-66691fd8c6de Jul 16 21:18:10.444144 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-4d5085e1-2724-4524-9b68-54baabd1d3bb tempest-SecurityGroupsTestAdminJSON-1430108405 tempest-SecurityGroupsTestAdminJSON-1430108405-project] 10.222.0.16 "POST /compute/v2.1/os-security-groups" status: 200 len: 221 microversion: 2.1 time: 2.090968 Jul 16 21:18:10.444750 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1176/2288] 10.222.0.16 () {60 vars in 1179 bytes} [Sat Jul 16 21:18:08 2022] POST /compute/v2.1/os-security-groups => generated 221 bytes in 2092 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:18:10.451481 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-fe1baf1c-faa1-4647-9a70-5c4e3877e0e9 req-99c38ee9-9693-40a9-9927-c15445060639 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "b8565a82-250b-4f64-9d90-98b368e4bd57", "tag": "54d1cb01-bab2-432d-8666-66691fd8c6de"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:18:10.461340 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-fe1baf1c-faa1-4647-9a70-5c4e3877e0e9 req-99c38ee9-9693-40a9-9927-c15445060639 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:10.461788 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-fe1baf1c-faa1-4647-9a70-5c4e3877e0e9 req-99c38ee9-9693-40a9-9927-c15445060639 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:10.473731 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-fe1baf1c-faa1-4647-9a70-5c4e3877e0e9 req-99c38ee9-9693-40a9-9927-c15445060639 service nova] Creating event network-changed:54d1cb01-bab2-432d-8666-66691fd8c6de for instance b8565a82-250b-4f64-9d90-98b368e4bd57 on nch2-master-1 Jul 16 21:18:10.478988 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-fe1baf1c-faa1-4647-9a70-5c4e3877e0e9 req-99c38ee9-9693-40a9-9927-c15445060639 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.033133 Jul 16 21:18:10.480391 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1177/2289] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:18:10 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:18:10.485635 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-679d88f3-0a1f-4b8d-bd87-1d6146d15cfb tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Calling method '.version_select at 0x7fdda297a700>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:10.551479 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-679d88f3-0a1f-4b8d-bd87-1d6146d15cfb tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] 10.222.0.16 "GET /compute/v2.1/images/1f9a9864-79f1-4d21-84c6-8d017682af53" status: 200 len: 1463 microversion: 2.1 time: 0.062794 Jul 16 21:18:10.551479 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1178/2290] 10.222.0.16 () {58 vars in 1232 bytes} [Sat Jul 16 21:18:10 2022] GET /compute/v2.1/images/1f9a9864-79f1-4d21-84c6-8d017682af53 => generated 1463 bytes in 64 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:18:10.551479 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-33d7261f-9925-4a0f-bcc4-bac5cd692fcf tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:10.551976 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-33d7261f-9925-4a0f-bcc4-bac5cd692fcf tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] [instance: 6baac85b-f81c-4e3a-9f96-633cffccfd74] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:18:10.555437 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_api:SELECT=92,nova_api:INSERT=7,nova_api:UPDATE=1 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:18:10.560543 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-33d7261f-9925-4a0f-bcc4-bac5cd692fcf tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:10.561147 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-33d7261f-9925-4a0f-bcc4-bac5cd692fcf tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:10.562169 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-33d7261f-9925-4a0f-bcc4-bac5cd692fcf tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:10.562953 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-33d7261f-9925-4a0f-bcc4-bac5cd692fcf tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:10.595851 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-766d4155-630d-4629-b860-f7f096100df9 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] 10.222.0.16 "POST /compute/v2.1/servers/b8565a82-250b-4f64-9d90-98b368e4bd57/action" status: 202 len: 0 microversion: 2.1 time: 1.442843 Jul 16 21:18:10.597045 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1113/2291] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 21:18:09 2022] POST /compute/v2.1/servers/b8565a82-250b-4f64-9d90-98b368e4bd57/action => generated 0 bytes in 1445 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 21:18:10.606185 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-bc11575e-20da-4416-8ad8-8f9040088bfb tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:10.611063 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-bc11575e-20da-4416-8ad8-8f9040088bfb tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] [instance: ed8dcb97-7546-45ef-9e9d-a9bb03a1b419] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:18:10.614531 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-bc11575e-20da-4416-8ad8-8f9040088bfb tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:10.615245 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-bc11575e-20da-4416-8ad8-8f9040088bfb tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:10.616355 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-bc11575e-20da-4416-8ad8-8f9040088bfb tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:10.616977 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-bc11575e-20da-4416-8ad8-8f9040088bfb tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:10.929198 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-33d7261f-9925-4a0f-bcc4-bac5cd692fcf tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd4144b5388b46a989a41a2f556bb508', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a90d4bcd25ff4bf68bdc194180fbbe09', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:18:10.937494 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-33d7261f-9925-4a0f-bcc4-bac5cd692fcf tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] 10.222.0.16 "GET /compute/v2.1/servers/6baac85b-f81c-4e3a-9f96-633cffccfd74" status: 200 len: 1318 microversion: 2.1 time: 0.391054 Jul 16 21:18:10.938309 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1179/2292] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:18:10 2022] GET /compute/v2.1/servers/6baac85b-f81c-4e3a-9f96-633cffccfd74 => generated 1318 bytes in 393 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:18:10.950710 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-3a5823a1-d8fb-41e7-9474-7c1acedb41c5 req-df06a91b-189d-479d-b472-6816e176909b service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "720ef2b0-e7b3-4b36-b613-1b73d2bae2e9", "name": "network-vif-plugged", "status": "completed", "tag": "b4dd35f4-14d3-4d08-b800-60be4a0cba5b"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:18:10.959337 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-3a5823a1-d8fb-41e7-9474-7c1acedb41c5 req-df06a91b-189d-479d-b472-6816e176909b service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:10.959789 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-3a5823a1-d8fb-41e7-9474-7c1acedb41c5 req-df06a91b-189d-479d-b472-6816e176909b service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:10.972704 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-3a5823a1-d8fb-41e7-9474-7c1acedb41c5 req-df06a91b-189d-479d-b472-6816e176909b service nova] Creating event network-vif-plugged:b4dd35f4-14d3-4d08-b800-60be4a0cba5b for instance 720ef2b0-e7b3-4b36-b613-1b73d2bae2e9 on nch1-master-1 Jul 16 21:18:10.978363 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-3a5823a1-d8fb-41e7-9474-7c1acedb41c5 req-df06a91b-189d-479d-b472-6816e176909b service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.031328 Jul 16 21:18:10.978998 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1180/2293] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:18:10 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:18:10.985104 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-bd8bb34d-2dd9-4c1c-988d-3d278e7cd2dd tempest-ServersNegativeTestMultiTenantJSON-1135003248 tempest-ServersNegativeTestMultiTenantJSON-1135003248-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:10.985825 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-bd8bb34d-2dd9-4c1c-988d-3d278e7cd2dd tempest-ServersNegativeTestMultiTenantJSON-1135003248 tempest-ServersNegativeTestMultiTenantJSON-1135003248-project] [instance: 43169d4f-1214-419a-aa1e-def65182ed5a] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:18:10.992919 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-bd8bb34d-2dd9-4c1c-988d-3d278e7cd2dd tempest-ServersNegativeTestMultiTenantJSON-1135003248 tempest-ServersNegativeTestMultiTenantJSON-1135003248-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:10.993381 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-bd8bb34d-2dd9-4c1c-988d-3d278e7cd2dd tempest-ServersNegativeTestMultiTenantJSON-1135003248 tempest-ServersNegativeTestMultiTenantJSON-1135003248-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:10.994174 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-bd8bb34d-2dd9-4c1c-988d-3d278e7cd2dd tempest-ServersNegativeTestMultiTenantJSON-1135003248 tempest-ServersNegativeTestMultiTenantJSON-1135003248-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:10.994626 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-bd8bb34d-2dd9-4c1c-988d-3d278e7cd2dd tempest-ServersNegativeTestMultiTenantJSON-1135003248 tempest-ServersNegativeTestMultiTenantJSON-1135003248-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:11.016863 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-bc11575e-20da-4416-8ad8-8f9040088bfb tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'daf3c9e6634e4ae1bfb492eae1f44c93', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '24c64db8082648c8837c16352252f96d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:18:11.029264 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-bc11575e-20da-4416-8ad8-8f9040088bfb tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] 10.222.0.16 "GET /compute/v2.1/servers/ed8dcb97-7546-45ef-9e9d-a9bb03a1b419" status: 200 len: 1487 microversion: 2.1 time: 0.427560 Jul 16 21:18:11.043308 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1114/2294] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:18:10 2022] GET /compute/v2.1/servers/ed8dcb97-7546-45ef-9e9d-a9bb03a1b419 => generated 1487 bytes in 438 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:18:11.048316 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-19967ed4-22ea-45fc-925f-3edada831ee1 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:11.049020 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-19967ed4-22ea-45fc-925f-3edada831ee1 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] [instance: 720ef2b0-e7b3-4b36-b613-1b73d2bae2e9] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:18:11.061922 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-19967ed4-22ea-45fc-925f-3edada831ee1 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:11.062301 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-19967ed4-22ea-45fc-925f-3edada831ee1 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:11.063130 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-19967ed4-22ea-45fc-925f-3edada831ee1 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:11.064158 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-19967ed4-22ea-45fc-925f-3edada831ee1 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:11.528847 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-bd8bb34d-2dd9-4c1c-988d-3d278e7cd2dd tempest-ServersNegativeTestMultiTenantJSON-1135003248 tempest-ServersNegativeTestMultiTenantJSON-1135003248-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '50cdc134e712487196f6623c5cf14592', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ba229aaef0ec4ddcadd1d940376c31ac', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:18:11.536629 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-bd8bb34d-2dd9-4c1c-988d-3d278e7cd2dd tempest-ServersNegativeTestMultiTenantJSON-1135003248 tempest-ServersNegativeTestMultiTenantJSON-1135003248-project] 10.222.0.16 "GET /compute/v2.1/servers/43169d4f-1214-419a-aa1e-def65182ed5a" status: 200 len: 1328 microversion: 2.1 time: 0.555506 Jul 16 21:18:11.537214 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1181/2295] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:18:10 2022] GET /compute/v2.1/servers/43169d4f-1214-419a-aa1e-def65182ed5a => generated 1328 bytes in 557 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:18:11.541791 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-98d0e281-21ad-497f-9cf8-43474d13e1d4 tempest-SecurityGroupsTestAdminJSON-1430108405 tempest-SecurityGroupsTestAdminJSON-1430108405-project] Action: 'create', calling method: .version_select at 0x7fdd9c6b9b80>, body: {"security_group": {"name": "tempest-securitygroup-1638585513", "description": "tempest-description-2094423604"}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:18:11.569298 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-19967ed4-22ea-45fc-925f-3edada831ee1 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f5bc1f2209f4eb186d80e385b67c774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '65d5bcfa02af4e8284ba1342197e2860', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:18:11.578152 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-19967ed4-22ea-45fc-925f-3edada831ee1 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] 10.222.0.16 "GET /compute/v2.1/servers/720ef2b0-e7b3-4b36-b613-1b73d2bae2e9" status: 200 len: 1313 microversion: 2.1 time: 0.533642 Jul 16 21:18:11.588154 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1115/2296] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:18:11 2022] GET /compute/v2.1/servers/720ef2b0-e7b3-4b36-b613-1b73d2bae2e9 => generated 1313 bytes in 535 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:18:11.588154 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-c3db6382-e93a-49ee-bcdb-debe63fc3783 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Calling method '.version_select at 0x7fdda2b8f940>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:11.631737 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-c3db6382-e93a-49ee-bcdb-debe63fc3783 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] 10.222.0.16 "GET /compute/v2.1/images/1f9a9864-79f1-4d21-84c6-8d017682af53" status: 200 len: 1463 microversion: 2.1 time: 0.051880 Jul 16 21:18:11.633042 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1116/2297] 10.222.0.16 () {58 vars in 1232 bytes} [Sat Jul 16 21:18:11 2022] GET /compute/v2.1/images/1f9a9864-79f1-4d21-84c6-8d017682af53 => generated 1463 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:18:11.638705 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-2c85e892-ab4f-453b-aa95-a27202334c72 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:11.639399 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-2c85e892-ab4f-453b-aa95-a27202334c72 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] [instance: b8565a82-250b-4f64-9d90-98b368e4bd57] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:18:11.645684 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2c85e892-ab4f-453b-aa95-a27202334c72 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:11.646144 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2c85e892-ab4f-453b-aa95-a27202334c72 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:11.646926 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2c85e892-ab4f-453b-aa95-a27202334c72 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:11.647403 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2c85e892-ab4f-453b-aa95-a27202334c72 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:11.946268 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-2c85e892-ab4f-453b-aa95-a27202334c72 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] 10.222.0.16 "GET /compute/v2.1/servers/b8565a82-250b-4f64-9d90-98b368e4bd57/os-security-groups" status: 200 len: 1159 microversion: 2.1 time: 0.311998 Jul 16 21:18:11.947200 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1117/2298] 10.222.0.16 () {58 vars in 1292 bytes} [Sat Jul 16 21:18:11 2022] GET /compute/v2.1/servers/b8565a82-250b-4f64-9d90-98b368e4bd57/os-security-groups => generated 1159 bytes in 313 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:18:11.958240 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-4a0df430-b06d-488a-8503-3dd671d75c7f tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Action: 'action', calling method: >, body: {"suspend": {}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:18:11.958525 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-4a0df430-b06d-488a-8503-3dd671d75c7f tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] [instance: ed8dcb97-7546-45ef-9e9d-a9bb03a1b419] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:18:11.965524 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4a0df430-b06d-488a-8503-3dd671d75c7f tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:11.965985 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4a0df430-b06d-488a-8503-3dd671d75c7f tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:11.966884 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4a0df430-b06d-488a-8503-3dd671d75c7f tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:11.967364 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4a0df430-b06d-488a-8503-3dd671d75c7f tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:11.991464 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.objects.instance [None req-4a0df430-b06d-488a-8503-3dd671d75c7f tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lazy-loading 'flavor' on Instance uuid ed8dcb97-7546-45ef-9e9d-a9bb03a1b419 {{(pid=138681) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:18:12.157840 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-98d0e281-21ad-497f-9cf8-43474d13e1d4 tempest-SecurityGroupsTestAdminJSON-1430108405 tempest-SecurityGroupsTestAdminJSON-1430108405-project] 10.222.0.16 "POST /compute/v2.1/os-security-groups" status: 200 len: 221 microversion: 2.1 time: 0.619543 Jul 16 21:18:12.165396 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1182/2299] 10.222.0.16 () {60 vars in 1179 bytes} [Sat Jul 16 21:18:11 2022] POST /compute/v2.1/os-security-groups => generated 221 bytes in 627 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:18:12.172109 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-da2d9ea4-4f22-40a5-966f-098f9ace9048 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:12.172745 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-da2d9ea4-4f22-40a5-966f-098f9ace9048 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] [instance: 6cb4e66a-eba2-42b7-91e8-09ee7b2a5a4f] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:18:12.181591 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-da2d9ea4-4f22-40a5-966f-098f9ace9048 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:12.182440 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-da2d9ea4-4f22-40a5-966f-098f9ace9048 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:12.184717 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-da2d9ea4-4f22-40a5-966f-098f9ace9048 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:12.185499 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-da2d9ea4-4f22-40a5-966f-098f9ace9048 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:12.358764 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-da2d9ea4-4f22-40a5-966f-098f9ace9048 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '71e9304fc50447d6aac99703b5b6e4e1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e84496d78d4481e9d7648128868bca2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:18:12.367484 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-da2d9ea4-4f22-40a5-966f-098f9ace9048 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] 10.222.0.16 "GET /compute/v2.1/servers/6cb4e66a-eba2-42b7-91e8-09ee7b2a5a4f" status: 200 len: 1466 microversion: 2.1 time: 0.200656 Jul 16 21:18:12.367843 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-4a0df430-b06d-488a-8503-3dd671d75c7f tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] 10.222.0.16 "POST /compute/v2.1/servers/ed8dcb97-7546-45ef-9e9d-a9bb03a1b419/action" status: 202 len: 0 microversion: 2.1 time: 0.416952 Jul 16 21:18:12.368478 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1183/2300] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:18:12 2022] GET /compute/v2.1/servers/6cb4e66a-eba2-42b7-91e8-09ee7b2a5a4f => generated 1466 bytes in 202 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:18:12.371603 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1118/2301] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 21:18:11 2022] POST /compute/v2.1/servers/ed8dcb97-7546-45ef-9e9d-a9bb03a1b419/action => generated 0 bytes in 421 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 21:18:12.373980 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-3b6b5549-5def-42ce-b21d-ad19af84a011 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:12.374452 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-3bccb2d9-9893-4c4a-9aaa-03c6c590746d tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Action: 'create', calling method: .version_select at 0x7fdda2ac05e0>, body: {"security_group": {"name": " tempest-securitygroup -682819366 ", "description": "tempest-description-1530172964"}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:18:12.374944 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-3b6b5549-5def-42ce-b21d-ad19af84a011 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] [instance: 6baac85b-f81c-4e3a-9f96-633cffccfd74] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:18:12.381649 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3b6b5549-5def-42ce-b21d-ad19af84a011 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:12.382036 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3b6b5549-5def-42ce-b21d-ad19af84a011 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:12.382963 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3b6b5549-5def-42ce-b21d-ad19af84a011 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:12.383464 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3b6b5549-5def-42ce-b21d-ad19af84a011 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:12.623705 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-3b6b5549-5def-42ce-b21d-ad19af84a011 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd4144b5388b46a989a41a2f556bb508', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a90d4bcd25ff4bf68bdc194180fbbe09', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:18:12.632193 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-3b6b5549-5def-42ce-b21d-ad19af84a011 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] 10.222.0.16 "GET /compute/v2.1/servers/6baac85b-f81c-4e3a-9f96-633cffccfd74" status: 200 len: 1318 microversion: 2.1 time: 0.262315 Jul 16 21:18:12.633762 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1184/2302] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:18:12 2022] GET /compute/v2.1/servers/6baac85b-f81c-4e3a-9f96-633cffccfd74 => generated 1318 bytes in 265 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:18:12.704471 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-9905a2a7-e5d0-422b-8390-2b03574eef61 tempest-SecurityGroupsTestAdminJSON-1999303763 tempest-SecurityGroupsTestAdminJSON-1999303763-project-admin] Action: 'create', calling method: .version_select at 0x7fdda29b2430>, body: {"security_group": {"name": "tempest-securitygroup-1938867538", "description": "tempest-description-890316916"}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:18:12.928354 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-3bccb2d9-9893-4c4a-9aaa-03c6c590746d tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] 10.222.0.16 "POST /compute/v2.1/os-security-groups" status: 200 len: 223 microversion: 2.1 time: 0.558122 Jul 16 21:18:12.929790 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1119/2303] 10.222.0.16 () {60 vars in 1179 bytes} [Sat Jul 16 21:18:12 2022] POST /compute/v2.1/os-security-groups => generated 223 bytes in 560 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:18:12.935603 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-510c111e-8410-433e-bda0-a41c1ce53834 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:12.936786 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-510c111e-8410-433e-bda0-a41c1ce53834 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] [instance: ed8dcb97-7546-45ef-9e9d-a9bb03a1b419] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:18:12.943683 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-510c111e-8410-433e-bda0-a41c1ce53834 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:12.944143 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-510c111e-8410-433e-bda0-a41c1ce53834 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:12.945000 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-510c111e-8410-433e-bda0-a41c1ce53834 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:12.945444 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-510c111e-8410-433e-bda0-a41c1ce53834 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:13.214861 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-510c111e-8410-433e-bda0-a41c1ce53834 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'daf3c9e6634e4ae1bfb492eae1f44c93', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '24c64db8082648c8837c16352252f96d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:18:13.223224 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-510c111e-8410-433e-bda0-a41c1ce53834 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] 10.222.0.16 "GET /compute/v2.1/servers/ed8dcb97-7546-45ef-9e9d-a9bb03a1b419" status: 200 len: 1495 microversion: 2.1 time: 0.292365 Jul 16 21:18:13.223803 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1120/2304] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:18:12 2022] GET /compute/v2.1/servers/ed8dcb97-7546-45ef-9e9d-a9bb03a1b419 => generated 1495 bytes in 293 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:18:13.233333 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-e86588d8-4c3b-4228-acfd-1575c103981e tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Action: 'action', calling method: >, body: {"createImage": {"name": "tempest-ListImageFiltersTestJSON-image-962829589"}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:18:13.233333 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-e86588d8-4c3b-4228-acfd-1575c103981e tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] [instance: b9a4910a-f117-4834-b699-498c6431b9bb] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:18:13.236137 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e86588d8-4c3b-4228-acfd-1575c103981e tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:13.236592 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e86588d8-4c3b-4228-acfd-1575c103981e tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:13.237371 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e86588d8-4c3b-4228-acfd-1575c103981e tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:13.237810 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e86588d8-4c3b-4228-acfd-1575c103981e tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:13.564336 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-9905a2a7-e5d0-422b-8390-2b03574eef61 tempest-SecurityGroupsTestAdminJSON-1999303763 tempest-SecurityGroupsTestAdminJSON-1999303763-project-admin] 10.222.0.16 "POST /compute/v2.1/os-security-groups" status: 200 len: 220 microversion: 2.1 time: 0.929245 Jul 16 21:18:13.565082 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1185/2305] 10.222.0.16 () {60 vars in 1179 bytes} [Sat Jul 16 21:18:12 2022] POST /compute/v2.1/os-security-groups => generated 220 bytes in 931 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:18:13.569756 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-3a820401-5099-4f0d-b246-667ea743e866 req-d59e25f8-4988-4868-9c3d-71fe5ba7a5b7 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "43169d4f-1214-419a-aa1e-def65182ed5a", "tag": "17db3ebe-c69b-439f-9a4d-5f322d30e1ad"}, {"name": "network-changed", "server_uuid": "b8565a82-250b-4f64-9d90-98b368e4bd57", "tag": "54d1cb01-bab2-432d-8666-66691fd8c6de"}, {"name": "network-changed", "server_uuid": "6baac85b-f81c-4e3a-9f96-633cffccfd74", "tag": "92263c71-5a11-478f-8dc4-be33c5f55e4f"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:18:13.581806 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-3a820401-5099-4f0d-b246-667ea743e866 req-d59e25f8-4988-4868-9c3d-71fe5ba7a5b7 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:13.582267 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-3a820401-5099-4f0d-b246-667ea743e866 req-d59e25f8-4988-4868-9c3d-71fe5ba7a5b7 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:13.600673 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-3a820401-5099-4f0d-b246-667ea743e866 req-d59e25f8-4988-4868-9c3d-71fe5ba7a5b7 service nova] Creating event network-changed:17db3ebe-c69b-439f-9a4d-5f322d30e1ad for instance 43169d4f-1214-419a-aa1e-def65182ed5a on nch1-master-1 Jul 16 21:18:13.601194 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-3a820401-5099-4f0d-b246-667ea743e866 req-d59e25f8-4988-4868-9c3d-71fe5ba7a5b7 service nova] Creating event network-changed:54d1cb01-bab2-432d-8666-66691fd8c6de for instance b8565a82-250b-4f64-9d90-98b368e4bd57 on nch2-master-1 Jul 16 21:18:13.601710 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-3a820401-5099-4f0d-b246-667ea743e866 req-d59e25f8-4988-4868-9c3d-71fe5ba7a5b7 service nova] Creating event network-changed:92263c71-5a11-478f-8dc4-be33c5f55e4f for instance 6baac85b-f81c-4e3a-9f96-633cffccfd74 on nch1-master-1 Jul 16 21:18:13.612317 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-3a820401-5099-4f0d-b246-667ea743e866 req-d59e25f8-4988-4868-9c3d-71fe5ba7a5b7 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 513 microversion: 2.1 time: 0.046060 Jul 16 21:18:13.612932 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1186/2306] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:18:13 2022] POST /compute/v2.1/os-server-external-events => generated 513 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:18:13.618327 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-2b118d8f-b878-4b06-90b3-6110c46b530c tempest-ServersNegativeTestMultiTenantJSON-1135003248 tempest-ServersNegativeTestMultiTenantJSON-1135003248-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:13.619059 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-2b118d8f-b878-4b06-90b3-6110c46b530c tempest-ServersNegativeTestMultiTenantJSON-1135003248 tempest-ServersNegativeTestMultiTenantJSON-1135003248-project] [instance: 43169d4f-1214-419a-aa1e-def65182ed5a] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:18:13.625642 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2b118d8f-b878-4b06-90b3-6110c46b530c tempest-ServersNegativeTestMultiTenantJSON-1135003248 tempest-ServersNegativeTestMultiTenantJSON-1135003248-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:13.626260 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2b118d8f-b878-4b06-90b3-6110c46b530c tempest-ServersNegativeTestMultiTenantJSON-1135003248 tempest-ServersNegativeTestMultiTenantJSON-1135003248-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:13.626999 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2b118d8f-b878-4b06-90b3-6110c46b530c tempest-ServersNegativeTestMultiTenantJSON-1135003248 tempest-ServersNegativeTestMultiTenantJSON-1135003248-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:13.627602 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2b118d8f-b878-4b06-90b3-6110c46b530c tempest-ServersNegativeTestMultiTenantJSON-1135003248 tempest-ServersNegativeTestMultiTenantJSON-1135003248-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:13.724502 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=95,nova_cell1:UPDATE=3,nova_cell1:INSERT=2 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:18:13.811765 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-e86588d8-4c3b-4228-acfd-1575c103981e tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] 10.222.0.16 "POST /compute/v2.1/servers/b9a4910a-f117-4834-b699-498c6431b9bb/action" status: 202 len: 0 microversion: 2.1 time: 0.586617 Jul 16 21:18:13.817559 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1121/2307] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 21:18:13 2022] POST /compute/v2.1/servers/b9a4910a-f117-4834-b699-498c6431b9bb/action => generated 0 bytes in 589 msecs (HTTP/1.1 202) 10 headers in 450 bytes (1 switches on core 0) Jul 16 21:18:13.819666 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-b07c9a09-67ce-4d4e-b194-0eef44f7121d tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:13.820923 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-b07c9a09-67ce-4d4e-b194-0eef44f7121d tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] [instance: 720ef2b0-e7b3-4b36-b613-1b73d2bae2e9] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:18:13.829779 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b07c9a09-67ce-4d4e-b194-0eef44f7121d tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:13.830248 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b07c9a09-67ce-4d4e-b194-0eef44f7121d tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:13.831066 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b07c9a09-67ce-4d4e-b194-0eef44f7121d tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:13.831530 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b07c9a09-67ce-4d4e-b194-0eef44f7121d tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:13.871260 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell0:SELECT=9 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:18:14.021284 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-2b118d8f-b878-4b06-90b3-6110c46b530c tempest-ServersNegativeTestMultiTenantJSON-1135003248 tempest-ServersNegativeTestMultiTenantJSON-1135003248-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '50cdc134e712487196f6623c5cf14592', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ba229aaef0ec4ddcadd1d940376c31ac', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:18:14.028118 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-2b118d8f-b878-4b06-90b3-6110c46b530c tempest-ServersNegativeTestMultiTenantJSON-1135003248 tempest-ServersNegativeTestMultiTenantJSON-1135003248-project] 10.222.0.16 "GET /compute/v2.1/servers/43169d4f-1214-419a-aa1e-def65182ed5a" status: 200 len: 1328 microversion: 2.1 time: 0.413717 Jul 16 21:18:14.028699 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1187/2308] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:18:13 2022] GET /compute/v2.1/servers/43169d4f-1214-419a-aa1e-def65182ed5a => generated 1328 bytes in 415 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:18:14.033205 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-b8929556-b024-45ad-b95a-1c4e772fd4d9 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Calling method '.version_select at 0x7fdda2a15430>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:14.098485 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-b07c9a09-67ce-4d4e-b194-0eef44f7121d tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f5bc1f2209f4eb186d80e385b67c774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '65d5bcfa02af4e8284ba1342197e2860', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:18:14.106985 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-b8929556-b024-45ad-b95a-1c4e772fd4d9 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] 10.222.0.16 "GET /compute/v2.1/os-security-groups/b9e2f469-feba-4fb9-9432-0249b1ace909" status: 200 len: 223 microversion: 2.1 time: 0.077208 Jul 16 21:18:14.107400 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-b07c9a09-67ce-4d4e-b194-0eef44f7121d tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] 10.222.0.16 "GET /compute/v2.1/servers/720ef2b0-e7b3-4b36-b613-1b73d2bae2e9" status: 200 len: 1462 microversion: 2.1 time: 0.292278 Jul 16 21:18:14.111082 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1188/2309] 10.222.0.16 () {58 vars in 1268 bytes} [Sat Jul 16 21:18:14 2022] GET /compute/v2.1/os-security-groups/b9e2f469-feba-4fb9-9432-0249b1ace909 => generated 223 bytes in 79 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:18:14.111242 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1122/2310] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:18:13 2022] GET /compute/v2.1/servers/720ef2b0-e7b3-4b36-b613-1b73d2bae2e9 => generated 1462 bytes in 294 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:18:14.112661 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-9d38ad07-095b-4547-a391-c319f15578fa tempest-SecurityGroupsTestAdminJSON-1999303763 tempest-SecurityGroupsTestAdminJSON-1999303763-project-admin] Action: 'create', calling method: .version_select at 0x7fdd9cfa65e0>, body: {"security_group": {"name": "tempest-securitygroup-1125630027", "description": "tempest-description-1202117918"}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:18:14.123238 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-e9c25180-a309-4898-83cb-0cef652031ee req-20479999-bca8-4e5f-aa3d-e2ced580796f service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "6cb4e66a-eba2-42b7-91e8-09ee7b2a5a4f", "name": "network-vif-unplugged", "status": "completed", "tag": "381fe698-72ef-4f6f-8b88-5df5cb8d8d96"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:18:14.133003 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-e9c25180-a309-4898-83cb-0cef652031ee req-20479999-bca8-4e5f-aa3d-e2ced580796f service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:14.133464 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-e9c25180-a309-4898-83cb-0cef652031ee req-20479999-bca8-4e5f-aa3d-e2ced580796f service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:14.146318 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-e9c25180-a309-4898-83cb-0cef652031ee req-20479999-bca8-4e5f-aa3d-e2ced580796f service nova] Creating event network-vif-unplugged:381fe698-72ef-4f6f-8b88-5df5cb8d8d96 for instance 6cb4e66a-eba2-42b7-91e8-09ee7b2a5a4f on nch2-master-1 Jul 16 21:18:14.152073 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-e9c25180-a309-4898-83cb-0cef652031ee req-20479999-bca8-4e5f-aa3d-e2ced580796f service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.042147 Jul 16 21:18:14.154596 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1189/2311] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:18:14 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:18:14.160192 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-261d0fe7-d9cc-4a42-bd9f-0b22ef7df739 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:14.160904 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-261d0fe7-d9cc-4a42-bd9f-0b22ef7df739 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] [instance: 6baac85b-f81c-4e3a-9f96-633cffccfd74] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:18:14.167619 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-261d0fe7-d9cc-4a42-bd9f-0b22ef7df739 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:14.168354 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-261d0fe7-d9cc-4a42-bd9f-0b22ef7df739 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:14.169140 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-261d0fe7-d9cc-4a42-bd9f-0b22ef7df739 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:14.169729 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-261d0fe7-d9cc-4a42-bd9f-0b22ef7df739 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:14.377639 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-261d0fe7-d9cc-4a42-bd9f-0b22ef7df739 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd4144b5388b46a989a41a2f556bb508', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a90d4bcd25ff4bf68bdc194180fbbe09', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:18:14.386169 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-261d0fe7-d9cc-4a42-bd9f-0b22ef7df739 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] 10.222.0.16 "GET /compute/v2.1/servers/6baac85b-f81c-4e3a-9f96-633cffccfd74" status: 200 len: 1318 microversion: 2.1 time: 0.229651 Jul 16 21:18:14.387286 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1190/2312] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:18:14 2022] GET /compute/v2.1/servers/6baac85b-f81c-4e3a-9f96-633cffccfd74 => generated 1318 bytes in 232 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:18:14.392057 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-b7ce8e71-d20e-48b5-b592-7be4b3ce5cd0 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Calling method '.version_select at 0x7fdda2a98dc0>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:14.440459 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-b7ce8e71-d20e-48b5-b592-7be4b3ce5cd0 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] 10.222.0.16 "GET /compute/v2.1/images/04088d67-f393-4fbb-8c56-7c2bc534a560" status: 200 len: 1454 microversion: 2.1 time: 0.051914 Jul 16 21:18:14.441525 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1191/2313] 10.222.0.16 () {58 vars in 1232 bytes} [Sat Jul 16 21:18:14 2022] GET /compute/v2.1/images/04088d67-f393-4fbb-8c56-7c2bc534a560 => generated 1454 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:18:14.442695 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell0:SELECT=7 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:18:14.446766 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-c445aeb0-3717-4d6a-ae5e-81f1c6f93f95 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Calling method '.version_select at 0x7fdd9c6a5310>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:14.693203 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-9d38ad07-095b-4547-a391-c319f15578fa tempest-SecurityGroupsTestAdminJSON-1999303763 tempest-SecurityGroupsTestAdminJSON-1999303763-project-admin] 10.222.0.16 "POST /compute/v2.1/os-security-groups" status: 200 len: 221 microversion: 2.1 time: 0.584094 Jul 16 21:18:14.694005 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1123/2314] 10.222.0.16 () {60 vars in 1179 bytes} [Sat Jul 16 21:18:14 2022] POST /compute/v2.1/os-security-groups => generated 221 bytes in 585 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:18:14.821979 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-ea105a62-a4df-4b77-a39c-ec2cfb7ff6d4 tempest-ServersAdminTestJSON-1511367024 tempest-ServersAdminTestJSON-1511367024-project-admin] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:14.827779 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-ea105a62-a4df-4b77-a39c-ec2cfb7ff6d4 tempest-ServersAdminTestJSON-1511367024 tempest-ServersAdminTestJSON-1511367024-project-admin] Searching by: {'deleted': False, 'project_id': 'd9472fdbbcfb4b3cb81b052cc93d714b'} {{(pid=138681) get_all /opt/stack/nova/nova/compute/api.py:2962}} Jul 16 21:18:14.834321 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ea105a62-a4df-4b77-a39c-ec2cfb7ff6d4 tempest-ServersAdminTestJSON-1511367024 tempest-ServersAdminTestJSON-1511367024-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:14.834782 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ea105a62-a4df-4b77-a39c-ec2cfb7ff6d4 tempest-ServersAdminTestJSON-1511367024 tempest-ServersAdminTestJSON-1511367024-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:14.835662 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ea105a62-a4df-4b77-a39c-ec2cfb7ff6d4 tempest-ServersAdminTestJSON-1511367024 tempest-ServersAdminTestJSON-1511367024-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:14.836104 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ea105a62-a4df-4b77-a39c-ec2cfb7ff6d4 tempest-ServersAdminTestJSON-1511367024 tempest-ServersAdminTestJSON-1511367024-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:14.866921 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-c445aeb0-3717-4d6a-ae5e-81f1c6f93f95 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] 10.222.0.16 "DELETE /compute/v2.1/os-security-groups/b9e2f469-feba-4fb9-9432-0249b1ace909" status: 202 len: 0 microversion: 2.1 time: 0.424580 Jul 16 21:18:14.871648 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1192/2315] 10.222.0.16 () {58 vars in 1271 bytes} [Sat Jul 16 21:18:14 2022] DELETE /compute/v2.1/os-security-groups/b9e2f469-feba-4fb9-9432-0249b1ace909 => generated 0 bytes in 430 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 21:18:14.877049 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-d5297f06-3391-438f-a581-4d4d672afabe tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:14.881554 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-d5297f06-3391-438f-a581-4d4d672afabe tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] [instance: ed8dcb97-7546-45ef-9e9d-a9bb03a1b419] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:18:14.896701 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d5297f06-3391-438f-a581-4d4d672afabe tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:14.902659 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d5297f06-3391-438f-a581-4d4d672afabe tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:14.903821 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d5297f06-3391-438f-a581-4d4d672afabe tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:14.904276 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d5297f06-3391-438f-a581-4d4d672afabe tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:14.942190 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-ea105a62-a4df-4b77-a39c-ec2cfb7ff6d4 tempest-ServersAdminTestJSON-1511367024 tempest-ServersAdminTestJSON-1511367024-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/detail" status: 200 len: 15 microversion: 2.1 time: 0.246949 Jul 16 21:18:14.943103 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1124/2316] 10.222.0.16 () {58 vars in 1145 bytes} [Sat Jul 16 21:18:14 2022] GET /compute/v2.1/servers/detail => generated 15 bytes in 248 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Jul 16 21:18:14.947507 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-0517dc7e-4706-44c0-9327-78c00f0fcd5a tempest-SecurityGroupsTestAdminJSON-1999303763 tempest-SecurityGroupsTestAdminJSON-1999303763-project-admin] Calling method '.version_select at 0x7fdd9cfa69d0>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:15.560493 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-d5297f06-3391-438f-a581-4d4d672afabe tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'daf3c9e6634e4ae1bfb492eae1f44c93', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '24c64db8082648c8837c16352252f96d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:18:15.568098 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-d5297f06-3391-438f-a581-4d4d672afabe tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] 10.222.0.16 "GET /compute/v2.1/servers/ed8dcb97-7546-45ef-9e9d-a9bb03a1b419" status: 200 len: 1478 microversion: 2.1 time: 0.694956 Jul 16 21:18:15.568679 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1193/2317] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:18:14 2022] GET /compute/v2.1/servers/ed8dcb97-7546-45ef-9e9d-a9bb03a1b419 => generated 1478 bytes in 696 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:18:15.573308 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-8227b16e-388d-40d4-b223-e45cc1d6c41f tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Calling method '.version_select at 0x7fdd9c6b6160>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:15.625580 ncdv-master-1 devstack@n-api.service[138680]: DEBUG neutronclient.v2_0.client [None req-8227b16e-388d-40d4-b223-e45cc1d6c41f tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Error message: {"NeutronError": {"type": "SecurityGroupNotFound", "message": "Security group b9e2f469-feba-4fb9-9432-0249b1ace909 does not exist", "detail": ""}} {{(pid=138680) _handle_fault_response /usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py:262}} Jul 16 21:18:15.628246 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.network.security_group_api [None req-8227b16e-388d-40d4-b223-e45cc1d6c41f tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Neutron security group b9e2f469-feba-4fb9-9432-0249b1ace909 not found {{(pid=138680) get /opt/stack/nova/nova/network/security_group_api.py:313}} Jul 16 21:18:15.628246 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-8227b16e-388d-40d4-b223-e45cc1d6c41f tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] HTTP exception thrown: Security group b9e2f469-feba-4fb9-9432-0249b1ace909 does not exist Jul 16 21:18:15.628246 ncdv-master-1 devstack@n-api.service[138680]: Neutron server returns request_ids: ['req-d8dc9de6-80ea-41b8-9a76-b1ebcab8a0b2'] Jul 16 21:18:15.629825 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-8227b16e-388d-40d4-b223-e45cc1d6c41f tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Request method failure captured: Jul 16 21:18:15.629825 ncdv-master-1 devstack@n-api.service[138680]: request: GET /compute/v2.1/os-security-groups/b9e2f469-feba-4fb9-9432-0249b1ace909 HTTP/1.1 Jul 16 21:18:15.629825 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:18:15.629825 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:18:15.629825 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:18:15.629825 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 0 Jul 16 21:18:15.629825 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:18:15.629825 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:18:15.629825 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:18:15.629825 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:18:15.629825 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0yrk-HsfYKsyovLSaEtNxaoXy7a3rLGNlsRPiPP76ZvvrKkKyGd-uekosXQxpFgkRnItBdviN4xr49Rerg5AyvLSzsyRY-Um0r3BRAvyZrzmCDYUpWbXk1uyEIb3Af4U3Y3nXIJgJAuDVx9z2NGSvEGCsGz12WlyyGCNfVSADFcYJP8 Jul 16 21:18:15.629825 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:18:15.629825 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:18:15.629825 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:18:15.629825 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:18:15.629825 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:18:15.629825 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:18:15.629825 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: e353258f9c3048ef84683294e2b8579d Jul 16 21:18:15.629825 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-SecurityGroupsTestJSON-438340169 Jul 16 21:18:15.629825 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:18:15.629825 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:18:15.629825 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/e353258f9c3048ef84683294e2b8579d"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/e353258f9c3048ef84683294e2b8579d"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_e353258f9c3048ef84683294e2b8579d"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/e353258f9c3048ef84683294e2b8579d"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:18:15.629825 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-SecurityGroupsTestJSON-438340169 Jul 16 21:18:15.629825 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: e353258f9c3048ef84683294e2b8579d Jul 16 21:18:15.629825 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-SecurityGroupsTestJSON-438340169 Jul 16 21:18:15.629825 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-SecurityGroupsTestJSON-438340169-project Jul 16 21:18:15.629825 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:18:15.629825 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:18:15.629825 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: d0008fffad8e4708b2c01e03da92c47d Jul 16 21:18:15.629825 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-SecurityGroupsTestJSON-438340169-project Jul 16 21:18:15.629825 ncdv-master-1 devstack@n-api.service[138680]: method: Jul 16 21:18:15.629825 ncdv-master-1 devstack@n-api.service[138680]: exception: Security group b9e2f469-feba-4fb9-9432-0249b1ace909 does not exist Jul 16 21:18:15.629825 ncdv-master-1 devstack@n-api.service[138680]: Neutron server returns request_ids: ['req-d8dc9de6-80ea-41b8-9a76-b1ebcab8a0b2'] Jul 16 21:18:15.629825 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': 'b9e2f469-feba-4fb9-9432-0249b1ace909'} Jul 16 21:18:15.629825 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:18:15.629825 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:18:15.629825 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/security_group_api.py", line 309, in get Jul 16 21:18:15.629825 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi group = neutron.show_security_group(id).get('security_group') Jul 16 21:18:15.629825 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 197, in wrapper Jul 16 21:18:15.629825 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Jul 16 21:18:15.629825 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 1040, in show_security_group Jul 16 21:18:15.629825 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return self.get(self.security_group_path % (security_group), Jul 16 21:18:15.629825 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 197, in wrapper Jul 16 21:18:15.629825 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Jul 16 21:18:15.629825 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 356, in get Jul 16 21:18:15.629825 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return self.retry_request("GET", action, body=body, Jul 16 21:18:15.629825 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 197, in wrapper Jul 16 21:18:15.629825 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Jul 16 21:18:15.629825 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 333, in retry_request Jul 16 21:18:15.629825 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return self.do_request(method, action, body=body, Jul 16 21:18:15.629825 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 197, in wrapper Jul 16 21:18:15.629825 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Jul 16 21:18:15.629825 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 297, in do_request Jul 16 21:18:15.629825 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi self._handle_fault_response(status_code, replybody, resp) Jul 16 21:18:15.629825 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 197, in wrapper Jul 16 21:18:15.629825 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Jul 16 21:18:15.629825 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 272, in _handle_fault_response Jul 16 21:18:15.629825 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi exception_handler_v20(status_code, error_body) Jul 16 21:18:15.629825 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 90, in exception_handler_v20 Jul 16 21:18:15.629825 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise client_exc(message=error_message, Jul 16 21:18:15.629825 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi neutronclient.common.exceptions.NotFound: Security group b9e2f469-feba-4fb9-9432-0249b1ace909 does not exist Jul 16 21:18:15.634047 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-d8dc9de6-80ea-41b8-9a76-b1ebcab8a0b2'] Jul 16 21:18:15.634047 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:18:15.634047 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:18:15.634047 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:18:15.634047 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:18:15.634047 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/security_groups.py", line 158, in show Jul 16 21:18:15.634047 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi security_group = security_group_api.get(context, id) Jul 16 21:18:15.634047 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/security_group_api.py", line 314, in get Jul 16 21:18:15.634047 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.SecurityGroupNotFound(str(e)) Jul 16 21:18:15.634047 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.SecurityGroupNotFound: Security group b9e2f469-feba-4fb9-9432-0249b1ace909 does not exist Jul 16 21:18:15.634047 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-d8dc9de6-80ea-41b8-9a76-b1ebcab8a0b2'] Jul 16 21:18:15.634047 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:18:15.634047 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:18:15.634047 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:18:15.634047 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:18:15.634047 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:18:15.634047 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:18:15.634047 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:18:15.634047 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:18:15.634047 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Jul 16 21:18:15.634047 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Jul 16 21:18:15.634047 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:18:15.634047 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:18:15.634047 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/security_groups.py", line 160, in show Jul 16 21:18:15.634047 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=exp.format_message()) Jul 16 21:18:15.634047 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Security group b9e2f469-feba-4fb9-9432-0249b1ace909 does not exist Jul 16 21:18:15.634047 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-d8dc9de6-80ea-41b8-9a76-b1ebcab8a0b2'] Jul 16 21:18:15.634047 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:18:15.634047 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:18:15.634047 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:18:15.634047 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:18:15.634047 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:18:15.634047 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:18:15.634047 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:18:15.634047 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:18:15.634047 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Security group b9e2f469-feba-4fb9-9432-0249b1ace909 does not exist Jul 16 21:18:15.634047 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-d8dc9de6-80ea-41b8-9a76-b1ebcab8a0b2'] Jul 16 21:18:15.634047 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:18:15.634047 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-8227b16e-388d-40d4-b223-e45cc1d6c41f tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Returning 404 to user: Security group b9e2f469-feba-4fb9-9432-0249b1ace909 does not exist Jul 16 21:18:15.634047 ncdv-master-1 devstack@n-api.service[138680]: Neutron server returns request_ids: ['req-d8dc9de6-80ea-41b8-9a76-b1ebcab8a0b2'] {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:18:15.634047 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-8227b16e-388d-40d4-b223-e45cc1d6c41f tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] 10.222.0.16 "GET /compute/v2.1/os-security-groups/b9e2f469-feba-4fb9-9432-0249b1ace909" status: 404 len: 194 microversion: 2.1 time: 0.061640 Jul 16 21:18:15.634047 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1194/2318] 10.222.0.16 () {58 vars in 1268 bytes} [Sat Jul 16 21:18:15 2022] GET /compute/v2.1/os-security-groups/b9e2f469-feba-4fb9-9432-0249b1ace909 => generated 194 bytes in 63 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:18:15.638602 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-51b4be98-1b7a-42a6-96d2-eb826bc9789e tempest-ServersAdminTestJSON-1511367024 tempest-ServersAdminTestJSON-1511367024-project-admin] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:15.642521 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-51b4be98-1b7a-42a6-96d2-eb826bc9789e tempest-ServersAdminTestJSON-1511367024 tempest-ServersAdminTestJSON-1511367024-project-admin] Searching by: {'deleted': False} {{(pid=138680) get_all /opt/stack/nova/nova/compute/api.py:2962}} Jul 16 21:18:15.649748 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-51b4be98-1b7a-42a6-96d2-eb826bc9789e tempest-ServersAdminTestJSON-1511367024 tempest-ServersAdminTestJSON-1511367024-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:15.653028 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-51b4be98-1b7a-42a6-96d2-eb826bc9789e tempest-ServersAdminTestJSON-1511367024 tempest-ServersAdminTestJSON-1511367024-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:15.653028 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-51b4be98-1b7a-42a6-96d2-eb826bc9789e tempest-ServersAdminTestJSON-1511367024 tempest-ServersAdminTestJSON-1511367024-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:15.653028 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-51b4be98-1b7a-42a6-96d2-eb826bc9789e tempest-ServersAdminTestJSON-1511367024 tempest-ServersAdminTestJSON-1511367024-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:15.763279 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.multi_cell_list [None req-51b4be98-1b7a-42a6-96d2-eb826bc9789e tempest-ServersAdminTestJSON-1511367024 tempest-ServersAdminTestJSON-1511367024-project-admin] Listed batch of 10 results from cell out of 1000 limit. Returned 10 total so far. {{(pid=138680) do_query /opt/stack/nova/nova/compute/multi_cell_list.py:373}} Jul 16 21:18:15.844439 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-51b4be98-1b7a-42a6-96d2-eb826bc9789e tempest-ServersAdminTestJSON-1511367024 tempest-ServersAdminTestJSON-1511367024-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:15.844898 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-51b4be98-1b7a-42a6-96d2-eb826bc9789e tempest-ServersAdminTestJSON-1511367024 tempest-ServersAdminTestJSON-1511367024-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:16.729228 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-51b4be98-1b7a-42a6-96d2-eb826bc9789e tempest-ServersAdminTestJSON-1511367024 tempest-ServersAdminTestJSON-1511367024-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/detail?all_tenants=" status: 200 len: 16132 microversion: 2.1 time: 1.096181 Jul 16 21:18:16.730489 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1195/2319] 10.222.0.16 () {58 vars in 1170 bytes} [Sat Jul 16 21:18:15 2022] GET /compute/v2.1/servers/detail?all_tenants= => generated 16132 bytes in 1098 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 16 21:18:16.759880 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-1d062a94-7b9d-403d-ad08-c0c3b13cf40e tempest-ServersNegativeTestMultiTenantJSON-1135003248 tempest-ServersNegativeTestMultiTenantJSON-1135003248-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:16.762224 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-1d062a94-7b9d-403d-ad08-c0c3b13cf40e tempest-ServersNegativeTestMultiTenantJSON-1135003248 tempest-ServersNegativeTestMultiTenantJSON-1135003248-project] [instance: 43169d4f-1214-419a-aa1e-def65182ed5a] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:18:16.769118 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1d062a94-7b9d-403d-ad08-c0c3b13cf40e tempest-ServersNegativeTestMultiTenantJSON-1135003248 tempest-ServersNegativeTestMultiTenantJSON-1135003248-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:16.770151 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1d062a94-7b9d-403d-ad08-c0c3b13cf40e tempest-ServersNegativeTestMultiTenantJSON-1135003248 tempest-ServersNegativeTestMultiTenantJSON-1135003248-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:16.770937 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1d062a94-7b9d-403d-ad08-c0c3b13cf40e tempest-ServersNegativeTestMultiTenantJSON-1135003248 tempest-ServersNegativeTestMultiTenantJSON-1135003248-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:16.771704 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1d062a94-7b9d-403d-ad08-c0c3b13cf40e tempest-ServersNegativeTestMultiTenantJSON-1135003248 tempest-ServersNegativeTestMultiTenantJSON-1135003248-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:16.982524 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-1d062a94-7b9d-403d-ad08-c0c3b13cf40e tempest-ServersNegativeTestMultiTenantJSON-1135003248 tempest-ServersNegativeTestMultiTenantJSON-1135003248-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '50cdc134e712487196f6623c5cf14592', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ba229aaef0ec4ddcadd1d940376c31ac', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:18:16.991366 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-1d062a94-7b9d-403d-ad08-c0c3b13cf40e tempest-ServersNegativeTestMultiTenantJSON-1135003248 tempest-ServersNegativeTestMultiTenantJSON-1135003248-project] 10.222.0.16 "GET /compute/v2.1/servers/43169d4f-1214-419a-aa1e-def65182ed5a" status: 200 len: 1328 microversion: 2.1 time: 0.248486 Jul 16 21:18:16.991953 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1196/2320] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:18:16 2022] GET /compute/v2.1/servers/43169d4f-1214-419a-aa1e-def65182ed5a => generated 1328 bytes in 250 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:18:16.997337 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-3c08f7d0-a3d2-4691-94a2-b7be0bc821f3 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:16.998412 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-3c08f7d0-a3d2-4691-94a2-b7be0bc821f3 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] [instance: 6baac85b-f81c-4e3a-9f96-633cffccfd74] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:18:17.007536 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3c08f7d0-a3d2-4691-94a2-b7be0bc821f3 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:17.008018 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3c08f7d0-a3d2-4691-94a2-b7be0bc821f3 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:17.009406 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3c08f7d0-a3d2-4691-94a2-b7be0bc821f3 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:17.009866 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3c08f7d0-a3d2-4691-94a2-b7be0bc821f3 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:17.208157 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-3c08f7d0-a3d2-4691-94a2-b7be0bc821f3 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd4144b5388b46a989a41a2f556bb508', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a90d4bcd25ff4bf68bdc194180fbbe09', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:18:17.216986 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-3c08f7d0-a3d2-4691-94a2-b7be0bc821f3 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] 10.222.0.16 "GET /compute/v2.1/servers/6baac85b-f81c-4e3a-9f96-633cffccfd74" status: 200 len: 1318 microversion: 2.1 time: 0.223654 Jul 16 21:18:17.220387 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1197/2321] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:18:16 2022] GET /compute/v2.1/servers/6baac85b-f81c-4e3a-9f96-633cffccfd74 => generated 1318 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:18:17.222341 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-a93eadc7-986e-4952-87f4-46fee04da59f tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Calling method '.version_select at 0x7fdda29d1b80>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:17.267879 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-a93eadc7-986e-4952-87f4-46fee04da59f tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] 10.222.0.16 "GET /compute/v2.1/images/04088d67-f393-4fbb-8c56-7c2bc534a560" status: 200 len: 1454 microversion: 2.1 time: 0.049151 Jul 16 21:18:17.268633 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1198/2322] 10.222.0.16 () {58 vars in 1232 bytes} [Sat Jul 16 21:18:17 2022] GET /compute/v2.1/images/04088d67-f393-4fbb-8c56-7c2bc534a560 => generated 1454 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:18:17.273160 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-71ae61cf-5a66-43cd-97fe-97648e02b15a tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Action: 'action', calling method: >, body: {"createImage": {"name": "tempest-test-snap-851485691"}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:18:17.274233 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-71ae61cf-5a66-43cd-97fe-97648e02b15a tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] [instance: ed8dcb97-7546-45ef-9e9d-a9bb03a1b419] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:18:17.281270 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-71ae61cf-5a66-43cd-97fe-97648e02b15a tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:17.281731 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-71ae61cf-5a66-43cd-97fe-97648e02b15a tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:17.283967 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-71ae61cf-5a66-43cd-97fe-97648e02b15a tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:17.284433 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-71ae61cf-5a66-43cd-97fe-97648e02b15a tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:17.842627 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-71ae61cf-5a66-43cd-97fe-97648e02b15a tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] 10.222.0.16 "POST /compute/v2.1/servers/ed8dcb97-7546-45ef-9e9d-a9bb03a1b419/action" status: 202 len: 0 microversion: 2.1 time: 0.572896 Jul 16 21:18:17.848118 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1199/2323] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 21:18:17 2022] POST /compute/v2.1/servers/ed8dcb97-7546-45ef-9e9d-a9bb03a1b419/action => generated 0 bytes in 575 msecs (HTTP/1.1 202) 10 headers in 450 bytes (1 switches on core 0) Jul 16 21:18:17.849762 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-0517dc7e-4706-44c0-9327-78c00f0fcd5a tempest-SecurityGroupsTestAdminJSON-1999303763 tempest-SecurityGroupsTestAdminJSON-1999303763-project-admin] 10.222.0.16 "GET /compute/v2.1/os-security-groups?all_tenants=true" status: 200 len: 10824 microversion: 2.1 time: 2.905511 Jul 16 21:18:17.850209 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-130b0496-fa5d-431c-89f5-eb8567724e9b tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Action: 'create', calling method: .version_select at 0x7fdda2b02ee0>, body: {"security_group": {"name": "tempest-SecurityGroupsTestJSON-securitygroup-1562718073", "description": "tempest-description-1979157908"}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:18:17.863320 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1125/2324] 10.222.0.16 () {58 vars in 1190 bytes} [Sat Jul 16 21:18:14 2022] GET /compute/v2.1/os-security-groups?all_tenants=true => generated 10824 bytes in 2919 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 16 21:18:17.868625 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-9070e0fd-d7aa-448a-9be4-7bcbfb7099ec req-a6817ae4-db39-4d6a-9e45-8f553ec497f8 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "43169d4f-1214-419a-aa1e-def65182ed5a", "name": "network-vif-plugged", "status": "completed", "tag": "17db3ebe-c69b-439f-9a4d-5f322d30e1ad"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:18:17.880682 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-9070e0fd-d7aa-448a-9be4-7bcbfb7099ec req-a6817ae4-db39-4d6a-9e45-8f553ec497f8 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:17.881210 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-9070e0fd-d7aa-448a-9be4-7bcbfb7099ec req-a6817ae4-db39-4d6a-9e45-8f553ec497f8 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:17.893890 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-9070e0fd-d7aa-448a-9be4-7bcbfb7099ec req-a6817ae4-db39-4d6a-9e45-8f553ec497f8 service nova] Creating event network-vif-plugged:17db3ebe-c69b-439f-9a4d-5f322d30e1ad for instance 43169d4f-1214-419a-aa1e-def65182ed5a on nch1-master-1 Jul 16 21:18:17.898891 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-9070e0fd-d7aa-448a-9be4-7bcbfb7099ec req-a6817ae4-db39-4d6a-9e45-8f553ec497f8 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.034315 Jul 16 21:18:17.899722 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1126/2325] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:18:17 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:18:17.907467 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-82e3f142-655b-4666-ba8e-c78e889d9e95 tempest-ServersAdminTestJSON-1511367024 tempest-ServersAdminTestJSON-1511367024-project-admin] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:17.911498 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-82e3f142-655b-4666-ba8e-c78e889d9e95 tempest-ServersAdminTestJSON-1511367024 tempest-ServersAdminTestJSON-1511367024-project-admin] Searching by: {'deleted': False, 'project_id': 'd9472fdbbcfb4b3cb81b052cc93d714b'} {{(pid=138681) get_all /opt/stack/nova/nova/compute/api.py:2962}} Jul 16 21:18:17.916655 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-82e3f142-655b-4666-ba8e-c78e889d9e95 tempest-ServersAdminTestJSON-1511367024 tempest-ServersAdminTestJSON-1511367024-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:17.917735 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-82e3f142-655b-4666-ba8e-c78e889d9e95 tempest-ServersAdminTestJSON-1511367024 tempest-ServersAdminTestJSON-1511367024-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:17.918605 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-82e3f142-655b-4666-ba8e-c78e889d9e95 tempest-ServersAdminTestJSON-1511367024 tempest-ServersAdminTestJSON-1511367024-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:17.919074 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-82e3f142-655b-4666-ba8e-c78e889d9e95 tempest-ServersAdminTestJSON-1511367024 tempest-ServersAdminTestJSON-1511367024-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:17.948124 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-82e3f142-655b-4666-ba8e-c78e889d9e95 tempest-ServersAdminTestJSON-1511367024 tempest-ServersAdminTestJSON-1511367024-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/detail?tenant_id=65d5bcfa02af4e8284ba1342197e2860" status: 200 len: 15 microversion: 2.1 time: 0.044266 Jul 16 21:18:17.948696 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1127/2326] 10.222.0.16 () {58 vars in 1230 bytes} [Sat Jul 16 21:18:17 2022] GET /compute/v2.1/servers/detail?tenant_id=65d5bcfa02af4e8284ba1342197e2860 => generated 15 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Jul 16 21:18:17.953365 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-3e169d94-741d-4589-bc66-2eb978992769 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Calling method '.version_select at 0x7fdda2b38c10>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:18.000030 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-3e169d94-741d-4589-bc66-2eb978992769 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] 10.222.0.16 "GET /compute/v2.1/images/eee6448e-8c8a-42ef-b4df-d19c5d9ae531" status: 200 len: 1439 microversion: 2.1 time: 0.049709 Jul 16 21:18:18.001314 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1128/2327] 10.222.0.16 () {58 vars in 1232 bytes} [Sat Jul 16 21:18:17 2022] GET /compute/v2.1/images/eee6448e-8c8a-42ef-b4df-d19c5d9ae531 => generated 1439 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:18:18.006985 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-06c7db29-ebce-41c4-96db-26494be12675 tempest-SecurityGroupsTestAdminJSON-1430108405 tempest-SecurityGroupsTestAdminJSON-1430108405-project] Calling method '.version_select at 0x7fdd9cf90430>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:18.290538 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-06c7db29-ebce-41c4-96db-26494be12675 tempest-SecurityGroupsTestAdminJSON-1430108405 tempest-SecurityGroupsTestAdminJSON-1430108405-project] 10.222.0.16 "GET /compute/v2.1/os-security-groups?all_tenants=true" status: 200 len: 1113 microversion: 2.1 time: 0.287927 Jul 16 21:18:18.291153 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1129/2328] 10.222.0.16 () {58 vars in 1190 bytes} [Sat Jul 16 21:18:18 2022] GET /compute/v2.1/os-security-groups?all_tenants=true => generated 1113 bytes in 289 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:18:18.296799 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-e1ab342b-3e32-4229-84a7-bf5756d948e6 tempest-ServersAdminTestJSON-1511367024 tempest-ServersAdminTestJSON-1511367024-project-admin] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:18.302237 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-e1ab342b-3e32-4229-84a7-bf5756d948e6 tempest-ServersAdminTestJSON-1511367024 tempest-ServersAdminTestJSON-1511367024-project-admin] Searching by: {'tenant_id': '65d5bcfa02af4e8284ba1342197e2860', 'deleted': False} {{(pid=138681) get_all /opt/stack/nova/nova/compute/api.py:2962}} Jul 16 21:18:18.308679 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e1ab342b-3e32-4229-84a7-bf5756d948e6 tempest-ServersAdminTestJSON-1511367024 tempest-ServersAdminTestJSON-1511367024-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:18.308679 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e1ab342b-3e32-4229-84a7-bf5756d948e6 tempest-ServersAdminTestJSON-1511367024 tempest-ServersAdminTestJSON-1511367024-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:18.309470 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e1ab342b-3e32-4229-84a7-bf5756d948e6 tempest-ServersAdminTestJSON-1511367024 tempest-ServersAdminTestJSON-1511367024-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:18.309918 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e1ab342b-3e32-4229-84a7-bf5756d948e6 tempest-ServersAdminTestJSON-1511367024 tempest-ServersAdminTestJSON-1511367024-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:18.351275 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.multi_cell_list [None req-e1ab342b-3e32-4229-84a7-bf5756d948e6 tempest-ServersAdminTestJSON-1511367024 tempest-ServersAdminTestJSON-1511367024-project-admin] Listed batch of 3 results from cell out of 1000 limit. Returned 3 total so far. {{(pid=138681) do_query /opt/stack/nova/nova/compute/multi_cell_list.py:373}} Jul 16 21:18:18.445847 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e1ab342b-3e32-4229-84a7-bf5756d948e6 tempest-ServersAdminTestJSON-1511367024 tempest-ServersAdminTestJSON-1511367024-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:18.446311 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e1ab342b-3e32-4229-84a7-bf5756d948e6 tempest-ServersAdminTestJSON-1511367024 tempest-ServersAdminTestJSON-1511367024-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:18.465777 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-130b0496-fa5d-431c-89f5-eb8567724e9b tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] 10.222.0.16 "POST /compute/v2.1/os-security-groups" status: 200 len: 244 microversion: 2.1 time: 0.620416 Jul 16 21:18:18.466651 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1200/2329] 10.222.0.16 () {60 vars in 1179 bytes} [Sat Jul 16 21:18:17 2022] POST /compute/v2.1/os-security-groups => generated 244 bytes in 622 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:18:18.472902 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-dba21c9b-4c05-4403-b097-54c60f3aa1e0 tempest-ServersNegativeTestMultiTenantJSON-1135003248 tempest-ServersNegativeTestMultiTenantJSON-1135003248-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:18.474511 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-dba21c9b-4c05-4403-b097-54c60f3aa1e0 tempest-ServersNegativeTestMultiTenantJSON-1135003248 tempest-ServersNegativeTestMultiTenantJSON-1135003248-project] [instance: 43169d4f-1214-419a-aa1e-def65182ed5a] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:18:18.484051 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-dba21c9b-4c05-4403-b097-54c60f3aa1e0 tempest-ServersNegativeTestMultiTenantJSON-1135003248 tempest-ServersNegativeTestMultiTenantJSON-1135003248-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:18.484766 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-dba21c9b-4c05-4403-b097-54c60f3aa1e0 tempest-ServersNegativeTestMultiTenantJSON-1135003248 tempest-ServersNegativeTestMultiTenantJSON-1135003248-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:18.485707 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-dba21c9b-4c05-4403-b097-54c60f3aa1e0 tempest-ServersNegativeTestMultiTenantJSON-1135003248 tempest-ServersNegativeTestMultiTenantJSON-1135003248-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:18.490682 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-dba21c9b-4c05-4403-b097-54c60f3aa1e0 tempest-ServersNegativeTestMultiTenantJSON-1135003248 tempest-ServersNegativeTestMultiTenantJSON-1135003248-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:18.621621 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-e1ab342b-3e32-4229-84a7-bf5756d948e6 tempest-ServersAdminTestJSON-1511367024 tempest-ServersAdminTestJSON-1511367024-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/detail?all_tenants=&tenant_id=65d5bcfa02af4e8284ba1342197e2860" status: 200 len: 4825 microversion: 2.1 time: 0.329285 Jul 16 21:18:18.622239 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1130/2330] 10.222.0.16 () {58 vars in 1256 bytes} [Sat Jul 16 21:18:18 2022] GET /compute/v2.1/servers/detail?all_tenants=&tenant_id=65d5bcfa02af4e8284ba1342197e2860 => generated 4825 bytes in 331 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:18:18.627339 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-e7189688-e14a-4a11-9d80-4ee921ea6024 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:18.628088 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-e7189688-e14a-4a11-9d80-4ee921ea6024 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] [instance: 6baac85b-f81c-4e3a-9f96-633cffccfd74] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:18:18.643839 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e7189688-e14a-4a11-9d80-4ee921ea6024 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:18.645178 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e7189688-e14a-4a11-9d80-4ee921ea6024 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:18.646191 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e7189688-e14a-4a11-9d80-4ee921ea6024 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:18.647418 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e7189688-e14a-4a11-9d80-4ee921ea6024 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:18.694024 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-dba21c9b-4c05-4403-b097-54c60f3aa1e0 tempest-ServersNegativeTestMultiTenantJSON-1135003248 tempest-ServersNegativeTestMultiTenantJSON-1135003248-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '50cdc134e712487196f6623c5cf14592', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ba229aaef0ec4ddcadd1d940376c31ac', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:18:18.703732 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-dba21c9b-4c05-4403-b097-54c60f3aa1e0 tempest-ServersNegativeTestMultiTenantJSON-1135003248 tempest-ServersNegativeTestMultiTenantJSON-1135003248-project] 10.222.0.16 "GET /compute/v2.1/servers/43169d4f-1214-419a-aa1e-def65182ed5a" status: 200 len: 1328 microversion: 2.1 time: 0.235818 Jul 16 21:18:18.704392 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1201/2331] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:18:18 2022] GET /compute/v2.1/servers/43169d4f-1214-419a-aa1e-def65182ed5a => generated 1328 bytes in 237 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:18:18.709416 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-828579b6-d37b-49a6-b798-4c16a53be656 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Calling method '.version_select at 0x7fdd9c6b6b80>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:18.766124 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-828579b6-d37b-49a6-b798-4c16a53be656 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] 10.222.0.16 "GET /compute/v2.1/images/04088d67-f393-4fbb-8c56-7c2bc534a560" status: 200 len: 1454 microversion: 2.1 time: 0.059990 Jul 16 21:18:18.769576 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1202/2332] 10.222.0.16 () {58 vars in 1232 bytes} [Sat Jul 16 21:18:18 2022] GET /compute/v2.1/images/04088d67-f393-4fbb-8c56-7c2bc534a560 => generated 1454 bytes in 63 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:18:18.776533 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-b413e571-624e-4ffd-b136-83a8b32a6497 tempest-SecurityGroupsTestAdminJSON-1999303763 tempest-SecurityGroupsTestAdminJSON-1999303763-project-admin] Calling method '.version_select at 0x7fdd9c6b9280>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:18.880274 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-e7189688-e14a-4a11-9d80-4ee921ea6024 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd4144b5388b46a989a41a2f556bb508', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a90d4bcd25ff4bf68bdc194180fbbe09', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:18:18.889142 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-e7189688-e14a-4a11-9d80-4ee921ea6024 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] 10.222.0.16 "GET /compute/v2.1/servers/6baac85b-f81c-4e3a-9f96-633cffccfd74" status: 200 len: 1318 microversion: 2.1 time: 0.265562 Jul 16 21:18:18.889502 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1131/2333] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:18:18 2022] GET /compute/v2.1/servers/6baac85b-f81c-4e3a-9f96-633cffccfd74 => generated 1318 bytes in 267 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:18:18.894372 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-84b79729-00a5-4763-8579-d330d74a0b69 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Action: 'create', calling method: .version_select at 0x7fdd9cfa6f70>, body: {"security_group": {"name": "tempest-SecurityGroupsTestJSON-securitygroup-1483257755", "description": "tempest-description-417839566"}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:18:19.117853 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-b413e571-624e-4ffd-b136-83a8b32a6497 tempest-SecurityGroupsTestAdminJSON-1999303763 tempest-SecurityGroupsTestAdminJSON-1999303763-project-admin] 10.222.0.16 "DELETE /compute/v2.1/os-security-groups/bf900702-bad4-476d-a2c3-e97773244175" status: 202 len: 0 microversion: 2.1 time: 0.347607 Jul 16 21:18:19.118545 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1203/2334] 10.222.0.16 () {58 vars in 1271 bytes} [Sat Jul 16 21:18:18 2022] DELETE /compute/v2.1/os-security-groups/bf900702-bad4-476d-a2c3-e97773244175 => generated 0 bytes in 350 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 21:18:19.129732 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-6151b18c-2b1a-418f-a9a9-a622ea0f1949 tempest-ServersAdminTestJSON-1511367024 tempest-ServersAdminTestJSON-1511367024-project-admin] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:19.150449 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-6151b18c-2b1a-418f-a9a9-a622ea0f1949 tempest-ServersAdminTestJSON-1511367024 tempest-ServersAdminTestJSON-1511367024-project-admin] Searching by: {'tenant_id': 'd9472fdbbcfb4b3cb81b052cc93d714b', 'deleted': False} {{(pid=138680) get_all /opt/stack/nova/nova/compute/api.py:2962}} Jul 16 21:18:19.168561 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6151b18c-2b1a-418f-a9a9-a622ea0f1949 tempest-ServersAdminTestJSON-1511367024 tempest-ServersAdminTestJSON-1511367024-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:19.169351 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6151b18c-2b1a-418f-a9a9-a622ea0f1949 tempest-ServersAdminTestJSON-1511367024 tempest-ServersAdminTestJSON-1511367024-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:19.174792 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6151b18c-2b1a-418f-a9a9-a622ea0f1949 tempest-ServersAdminTestJSON-1511367024 tempest-ServersAdminTestJSON-1511367024-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:19.177938 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6151b18c-2b1a-418f-a9a9-a622ea0f1949 tempest-ServersAdminTestJSON-1511367024 tempest-ServersAdminTestJSON-1511367024-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:19.223692 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-6151b18c-2b1a-418f-a9a9-a622ea0f1949 tempest-ServersAdminTestJSON-1511367024 tempest-ServersAdminTestJSON-1511367024-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/detail?all_tenants=&tenant_id=d9472fdbbcfb4b3cb81b052cc93d714b" status: 200 len: 15 microversion: 2.1 time: 0.099929 Jul 16 21:18:19.228850 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1204/2335] 10.222.0.16 () {58 vars in 1256 bytes} [Sat Jul 16 21:18:19 2022] GET /compute/v2.1/servers/detail?all_tenants=&tenant_id=d9472fdbbcfb4b3cb81b052cc93d714b => generated 15 bytes in 106 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Jul 16 21:18:19.230195 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-e0ab9116-a031-4ca8-b59e-bb86a0ae2c97 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Calling method '.version_select at 0x7fdd9c65e670>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:19.289182 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-e0ab9116-a031-4ca8-b59e-bb86a0ae2c97 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] 10.222.0.16 "GET /compute/v2.1/images/eee6448e-8c8a-42ef-b4df-d19c5d9ae531" status: 200 len: 1439 microversion: 2.1 time: 0.063291 Jul 16 21:18:19.297587 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1205/2336] 10.222.0.16 () {58 vars in 1232 bytes} [Sat Jul 16 21:18:19 2022] GET /compute/v2.1/images/eee6448e-8c8a-42ef-b4df-d19c5d9ae531 => generated 1439 bytes in 72 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:18:19.303694 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-7ef0ba93-4d70-41ad-a7c8-62f228458589 tempest-SecurityGroupsTestAdminJSON-1999303763 tempest-SecurityGroupsTestAdminJSON-1999303763-project-admin] Calling method '.version_select at 0x7fdd9c6b91f0>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:19.393281 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-84b79729-00a5-4763-8579-d330d74a0b69 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] 10.222.0.16 "POST /compute/v2.1/os-security-groups" status: 200 len: 243 microversion: 2.1 time: 0.502242 Jul 16 21:18:19.399741 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1132/2337] 10.222.0.16 () {60 vars in 1179 bytes} [Sat Jul 16 21:18:18 2022] POST /compute/v2.1/os-security-groups => generated 243 bytes in 504 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:18:19.404582 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-96832fef-761b-4810-b678-d95e0f943f8d tempest-ServersAdminTestJSON-1511367024 tempest-ServersAdminTestJSON-1511367024-project-admin] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:19.409069 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-96832fef-761b-4810-b678-d95e0f943f8d tempest-ServersAdminTestJSON-1511367024 tempest-ServersAdminTestJSON-1511367024-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/detail?status=invalid_status" status: 200 len: 15 microversion: 2.1 time: 0.013405 Jul 16 21:18:19.410778 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1133/2338] 10.222.0.16 () {58 vars in 1188 bytes} [Sat Jul 16 21:18:19 2022] GET /compute/v2.1/servers/detail?status=invalid_status => generated 15 bytes in 16 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Jul 16 21:18:19.417954 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-346773aa-856a-46e4-b4c2-c547d59263f2 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Action: 'create', calling method: .version_select at 0x7fdd9cdf9280>, body: {"security_group": {"name": "tempest-SecurityGroupsTestJSON-securitygroup-1676107235", "description": "tempest-description-2060172822"}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:18:19.672499 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-7ef0ba93-4d70-41ad-a7c8-62f228458589 tempest-SecurityGroupsTestAdminJSON-1999303763 tempest-SecurityGroupsTestAdminJSON-1999303763-project-admin] 10.222.0.16 "DELETE /compute/v2.1/os-security-groups/184c0386-7280-4b9e-bee4-666b628b3008" status: 202 len: 0 microversion: 2.1 time: 0.373215 Jul 16 21:18:19.673170 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1206/2339] 10.222.0.16 () {58 vars in 1271 bytes} [Sat Jul 16 21:18:19 2022] DELETE /compute/v2.1/os-security-groups/184c0386-7280-4b9e-bee4-666b628b3008 => generated 0 bytes in 375 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 21:18:19.680734 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-3f838cc1-5eaa-4a5a-a7f6-fb9658e7b939 tempest-ServersAdminTestJSON-1511367024 tempest-ServersAdminTestJSON-1511367024-project-admin] Action: 'action', calling method: >, body: {"os-resetState": {"state": "error"}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:18:19.682941 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-3f838cc1-5eaa-4a5a-a7f6-fb9658e7b939 tempest-ServersAdminTestJSON-1511367024 tempest-ServersAdminTestJSON-1511367024-project-admin] [instance: e7598b4a-f70c-4186-88d1-6fd0057f8656] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:18:19.696395 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3f838cc1-5eaa-4a5a-a7f6-fb9658e7b939 tempest-ServersAdminTestJSON-1511367024 tempest-ServersAdminTestJSON-1511367024-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:19.698880 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3f838cc1-5eaa-4a5a-a7f6-fb9658e7b939 tempest-ServersAdminTestJSON-1511367024 tempest-ServersAdminTestJSON-1511367024-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:19.703464 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3f838cc1-5eaa-4a5a-a7f6-fb9658e7b939 tempest-ServersAdminTestJSON-1511367024 tempest-ServersAdminTestJSON-1511367024-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:19.704180 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3f838cc1-5eaa-4a5a-a7f6-fb9658e7b939 tempest-ServersAdminTestJSON-1511367024 tempest-ServersAdminTestJSON-1511367024-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:19.742928 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:18:19.920240 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-346773aa-856a-46e4-b4c2-c547d59263f2 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] 10.222.0.16 "POST /compute/v2.1/os-security-groups" status: 200 len: 244 microversion: 2.1 time: 0.508146 Jul 16 21:18:19.921499 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1134/2340] 10.222.0.16 () {60 vars in 1179 bytes} [Sat Jul 16 21:18:19 2022] POST /compute/v2.1/os-security-groups => generated 244 bytes in 510 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:18:19.926751 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-19fc93f6-6931-462d-86db-e82b26fd71e1 tempest-SecurityGroupsTestAdminJSON-1430108405 tempest-SecurityGroupsTestAdminJSON-1430108405-project] Calling method '.version_select at 0x7fdd9d4cc9d0>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:20.065626 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-3f838cc1-5eaa-4a5a-a7f6-fb9658e7b939 tempest-ServersAdminTestJSON-1511367024 tempest-ServersAdminTestJSON-1511367024-project-admin] 10.222.0.16 "POST /compute/v2.1/servers/e7598b4a-f70c-4186-88d1-6fd0057f8656/action" status: 202 len: 0 microversion: 2.1 time: 0.388318 Jul 16 21:18:20.066268 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1207/2341] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 21:18:19 2022] POST /compute/v2.1/servers/e7598b4a-f70c-4186-88d1-6fd0057f8656/action => generated 0 bytes in 393 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 21:18:20.076298 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-cd41a943-5663-4a0c-bf66-2d856d61ea90 tempest-ServersNegativeTestMultiTenantJSON-1135003248 tempest-ServersNegativeTestMultiTenantJSON-1135003248-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:20.077979 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-cd41a943-5663-4a0c-bf66-2d856d61ea90 tempest-ServersNegativeTestMultiTenantJSON-1135003248 tempest-ServersNegativeTestMultiTenantJSON-1135003248-project] [instance: 43169d4f-1214-419a-aa1e-def65182ed5a] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:18:20.095023 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cd41a943-5663-4a0c-bf66-2d856d61ea90 tempest-ServersNegativeTestMultiTenantJSON-1135003248 tempest-ServersNegativeTestMultiTenantJSON-1135003248-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:20.095745 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cd41a943-5663-4a0c-bf66-2d856d61ea90 tempest-ServersNegativeTestMultiTenantJSON-1135003248 tempest-ServersNegativeTestMultiTenantJSON-1135003248-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:20.099645 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cd41a943-5663-4a0c-bf66-2d856d61ea90 tempest-ServersNegativeTestMultiTenantJSON-1135003248 tempest-ServersNegativeTestMultiTenantJSON-1135003248-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:20.100341 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cd41a943-5663-4a0c-bf66-2d856d61ea90 tempest-ServersNegativeTestMultiTenantJSON-1135003248 tempest-ServersNegativeTestMultiTenantJSON-1135003248-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:20.270853 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-cd41a943-5663-4a0c-bf66-2d856d61ea90 tempest-ServersNegativeTestMultiTenantJSON-1135003248 tempest-ServersNegativeTestMultiTenantJSON-1135003248-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '50cdc134e712487196f6623c5cf14592', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ba229aaef0ec4ddcadd1d940376c31ac', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:18:20.282006 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-cd41a943-5663-4a0c-bf66-2d856d61ea90 tempest-ServersNegativeTestMultiTenantJSON-1135003248 tempest-ServersNegativeTestMultiTenantJSON-1135003248-project] 10.222.0.16 "GET /compute/v2.1/servers/43169d4f-1214-419a-aa1e-def65182ed5a" status: 200 len: 1527 microversion: 2.1 time: 0.214061 Jul 16 21:18:20.292223 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1208/2342] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:18:20 2022] GET /compute/v2.1/servers/43169d4f-1214-419a-aa1e-def65182ed5a => generated 1527 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:18:20.299556 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-19fc93f6-6931-462d-86db-e82b26fd71e1 tempest-SecurityGroupsTestAdminJSON-1430108405 tempest-SecurityGroupsTestAdminJSON-1430108405-project] 10.222.0.16 "DELETE /compute/v2.1/os-security-groups/0b1cd62e-74cf-42b0-a4bb-cdcb287c0fad" status: 202 len: 0 microversion: 2.1 time: 0.376539 Jul 16 21:18:20.300151 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1135/2343] 10.222.0.16 () {58 vars in 1271 bytes} [Sat Jul 16 21:18:19 2022] DELETE /compute/v2.1/os-security-groups/0b1cd62e-74cf-42b0-a4bb-cdcb287c0fad => generated 0 bytes in 378 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 21:18:20.302215 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-3affece5-a48b-4643-8f55-9c379bb25822 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Calling method '.version_select at 0x7fdd9ddb4b80>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:20.309022 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-45b87de2-e9b4-4b2a-a4b8-7ae4a8adbbf0 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:20.310325 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-45b87de2-e9b4-4b2a-a4b8-7ae4a8adbbf0 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] [instance: 6baac85b-f81c-4e3a-9f96-633cffccfd74] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:18:20.331352 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_api:SELECT=100 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:18:20.333151 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-45b87de2-e9b4-4b2a-a4b8-7ae4a8adbbf0 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:20.333610 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-45b87de2-e9b4-4b2a-a4b8-7ae4a8adbbf0 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:20.334966 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-45b87de2-e9b4-4b2a-a4b8-7ae4a8adbbf0 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:20.335583 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-45b87de2-e9b4-4b2a-a4b8-7ae4a8adbbf0 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:20.384281 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-3affece5-a48b-4643-8f55-9c379bb25822 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] 10.222.0.16 "GET /compute/v2.1/images/04088d67-f393-4fbb-8c56-7c2bc534a560" status: 200 len: 1454 microversion: 2.1 time: 0.089552 Jul 16 21:18:20.385975 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1209/2344] 10.222.0.16 () {58 vars in 1232 bytes} [Sat Jul 16 21:18:20 2022] GET /compute/v2.1/images/04088d67-f393-4fbb-8c56-7c2bc534a560 => generated 1454 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:18:20.404746 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-206ea0f7-3fdb-462a-8571-932ce6f61e01 req-6c999408-e65d-4c6f-90b1-4eed0af90f1a service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "b9a4910a-f117-4834-b699-498c6431b9bb", "name": "network-vif-plugged", "status": "completed", "tag": "9ab69acd-f0a3-4355-b203-2c48e39cc3a1"}, {"server_uuid": "6baac85b-f81c-4e3a-9f96-633cffccfd74", "name": "network-vif-plugged", "status": "completed", "tag": "92263c71-5a11-478f-8dc4-be33c5f55e4f"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:18:20.419399 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-206ea0f7-3fdb-462a-8571-932ce6f61e01 req-6c999408-e65d-4c6f-90b1-4eed0af90f1a service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:20.420246 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-206ea0f7-3fdb-462a-8571-932ce6f61e01 req-6c999408-e65d-4c6f-90b1-4eed0af90f1a service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:20.449364 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-206ea0f7-3fdb-462a-8571-932ce6f61e01 req-6c999408-e65d-4c6f-90b1-4eed0af90f1a service nova] Creating event network-vif-plugged:9ab69acd-f0a3-4355-b203-2c48e39cc3a1 for instance b9a4910a-f117-4834-b699-498c6431b9bb on nch2-master-1 Jul 16 21:18:20.450070 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-206ea0f7-3fdb-462a-8571-932ce6f61e01 req-6c999408-e65d-4c6f-90b1-4eed0af90f1a service nova] Creating event network-vif-plugged:92263c71-5a11-478f-8dc4-be33c5f55e4f for instance 6baac85b-f81c-4e3a-9f96-633cffccfd74 on nch1-master-1 Jul 16 21:18:20.470801 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-206ea0f7-3fdb-462a-8571-932ce6f61e01 req-6c999408-e65d-4c6f-90b1-4eed0af90f1a service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 354 microversion: 2.1 time: 0.078210 Jul 16 21:18:20.472126 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1210/2345] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:18:20 2022] POST /compute/v2.1/os-server-external-events => generated 354 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:18:20.481521 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-f1f2122b-03eb-42cb-8828-ff3da2c48c22 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Calling method '.version_select at 0x7fdda2ae0940>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:20.617446 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-45b87de2-e9b4-4b2a-a4b8-7ae4a8adbbf0 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd4144b5388b46a989a41a2f556bb508', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a90d4bcd25ff4bf68bdc194180fbbe09', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:18:20.626651 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-45b87de2-e9b4-4b2a-a4b8-7ae4a8adbbf0 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] 10.222.0.16 "GET /compute/v2.1/servers/6baac85b-f81c-4e3a-9f96-633cffccfd74" status: 200 len: 1318 microversion: 2.1 time: 0.325380 Jul 16 21:18:20.631795 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1136/2346] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:18:20 2022] GET /compute/v2.1/servers/6baac85b-f81c-4e3a-9f96-633cffccfd74 => generated 1318 bytes in 331 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:18:20.647180 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-c9e7c19d-797e-4d9e-bb9c-635fb32cf031 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:20.652179 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-c9e7c19d-797e-4d9e-bb9c-635fb32cf031 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Policy check for os_compute_api:servers:allow_all_filters failed with credentials {'is_admin': False, 'user_id': '2f5bc1f2209f4eb186d80e385b67c774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '65d5bcfa02af4e8284ba1342197e2860', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:18:20.654903 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-c9e7c19d-797e-4d9e-bb9c-635fb32cf031 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Searching by: {'vm_state': ['error'], 'deleted': False, 'project_id': '65d5bcfa02af4e8284ba1342197e2860'} {{(pid=138681) get_all /opt/stack/nova/nova/compute/api.py:2962}} Jul 16 21:18:20.659719 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c9e7c19d-797e-4d9e-bb9c-635fb32cf031 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:20.660059 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c9e7c19d-797e-4d9e-bb9c-635fb32cf031 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:20.660932 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c9e7c19d-797e-4d9e-bb9c-635fb32cf031 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:20.661362 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c9e7c19d-797e-4d9e-bb9c-635fb32cf031 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:20.808712 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.multi_cell_list [None req-c9e7c19d-797e-4d9e-bb9c-635fb32cf031 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Listed batch of 1 results from cell out of 1000 limit. Returned 1 total so far. {{(pid=138681) do_query /opt/stack/nova/nova/compute/multi_cell_list.py:373}} Jul 16 21:18:20.918254 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-c9e7c19d-797e-4d9e-bb9c-635fb32cf031 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] 10.222.0.16 "GET /compute/v2.1/servers?status=error" status: 200 len: 338 microversion: 2.1 time: 0.274398 Jul 16 21:18:20.919386 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1137/2347] 10.222.0.16 () {58 vars in 1149 bytes} [Sat Jul 16 21:18:20 2022] GET /compute/v2.1/servers?status=error => generated 338 bytes in 276 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:18:20.924571 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-863a60a1-2957-4656-b767-747532bb7a87 tempest-ServersNegativeTestMultiTenantJSON-1135003248 tempest-ServersNegativeTestMultiTenantJSON-1135003248-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:20.925335 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-863a60a1-2957-4656-b767-747532bb7a87 tempest-ServersNegativeTestMultiTenantJSON-1135003248 tempest-ServersNegativeTestMultiTenantJSON-1135003248-project] [instance: 43169d4f-1214-419a-aa1e-def65182ed5a] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:18:20.932444 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-f1f2122b-03eb-42cb-8828-ff3da2c48c22 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] 10.222.0.16 "GET /compute/v2.1/os-security-groups" status: 200 len: 1836 microversion: 2.1 time: 0.458782 Jul 16 21:18:20.932635 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-863a60a1-2957-4656-b767-747532bb7a87 tempest-ServersNegativeTestMultiTenantJSON-1135003248 tempest-ServersNegativeTestMultiTenantJSON-1135003248-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:20.935621 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1211/2348] 10.222.0.16 () {58 vars in 1157 bytes} [Sat Jul 16 21:18:20 2022] GET /compute/v2.1/os-security-groups => generated 1836 bytes in 460 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:18:20.935787 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-863a60a1-2957-4656-b767-747532bb7a87 tempest-ServersNegativeTestMultiTenantJSON-1135003248 tempest-ServersNegativeTestMultiTenantJSON-1135003248-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:20.938586 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-863a60a1-2957-4656-b767-747532bb7a87 tempest-ServersNegativeTestMultiTenantJSON-1135003248 tempest-ServersNegativeTestMultiTenantJSON-1135003248-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:20.939223 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-863a60a1-2957-4656-b767-747532bb7a87 tempest-ServersNegativeTestMultiTenantJSON-1135003248 tempest-ServersNegativeTestMultiTenantJSON-1135003248-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:20.951959 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-0418b8ef-2668-4c43-ab5e-a2f4ff78ccb9 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Calling method '.version_select at 0x7fdda2b2b670>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:21.087133 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-0418b8ef-2668-4c43-ab5e-a2f4ff78ccb9 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] 10.222.0.16 "GET /compute/v2.1/images/eee6448e-8c8a-42ef-b4df-d19c5d9ae531" status: 200 len: 1439 microversion: 2.1 time: 0.140428 Jul 16 21:18:21.090249 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1212/2349] 10.222.0.16 () {58 vars in 1232 bytes} [Sat Jul 16 21:18:20 2022] GET /compute/v2.1/images/eee6448e-8c8a-42ef-b4df-d19c5d9ae531 => generated 1439 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:18:21.099253 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-52a485ec-9ae6-4878-8725-30dd1cc2e905 tempest-SecurityGroupsTestAdminJSON-1430108405 tempest-SecurityGroupsTestAdminJSON-1430108405-project] Calling method '.version_select at 0x7fdda2b39700>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:21.215629 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-863a60a1-2957-4656-b767-747532bb7a87 tempest-ServersNegativeTestMultiTenantJSON-1135003248 tempest-ServersNegativeTestMultiTenantJSON-1135003248-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '50cdc134e712487196f6623c5cf14592', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ba229aaef0ec4ddcadd1d940376c31ac', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:18:21.224371 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-863a60a1-2957-4656-b767-747532bb7a87 tempest-ServersNegativeTestMultiTenantJSON-1135003248 tempest-ServersNegativeTestMultiTenantJSON-1135003248-project] 10.222.0.16 "GET /compute/v2.1/servers/43169d4f-1214-419a-aa1e-def65182ed5a" status: 200 len: 1527 microversion: 2.1 time: 0.303706 Jul 16 21:18:21.238107 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1138/2350] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:18:20 2022] GET /compute/v2.1/servers/43169d4f-1214-419a-aa1e-def65182ed5a => generated 1527 bytes in 318 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:18:21.244549 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-c57b9311-211e-4ea0-adc9-d899b499d4d7 tempest-ServersAdminTestJSON-1511367024 tempest-ServersAdminTestJSON-1511367024-project-admin] Action: 'action', calling method: >, body: {"os-resetState": {"state": "active"}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:18:21.246431 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-c57b9311-211e-4ea0-adc9-d899b499d4d7 tempest-ServersAdminTestJSON-1511367024 tempest-ServersAdminTestJSON-1511367024-project-admin] [instance: e7598b4a-f70c-4186-88d1-6fd0057f8656] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:18:21.253815 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c57b9311-211e-4ea0-adc9-d899b499d4d7 tempest-ServersAdminTestJSON-1511367024 tempest-ServersAdminTestJSON-1511367024-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:21.254558 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c57b9311-211e-4ea0-adc9-d899b499d4d7 tempest-ServersAdminTestJSON-1511367024 tempest-ServersAdminTestJSON-1511367024-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:21.255477 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c57b9311-211e-4ea0-adc9-d899b499d4d7 tempest-ServersAdminTestJSON-1511367024 tempest-ServersAdminTestJSON-1511367024-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:21.255930 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c57b9311-211e-4ea0-adc9-d899b499d4d7 tempest-ServersAdminTestJSON-1511367024 tempest-ServersAdminTestJSON-1511367024-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:21.472962 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-52a485ec-9ae6-4878-8725-30dd1cc2e905 tempest-SecurityGroupsTestAdminJSON-1430108405 tempest-SecurityGroupsTestAdminJSON-1430108405-project] 10.222.0.16 "DELETE /compute/v2.1/os-security-groups/818390ac-6b27-4fed-b58d-016df9dc7a5d" status: 202 len: 0 microversion: 2.1 time: 0.381467 Jul 16 21:18:21.474076 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1213/2351] 10.222.0.16 () {58 vars in 1271 bytes} [Sat Jul 16 21:18:21 2022] DELETE /compute/v2.1/os-security-groups/818390ac-6b27-4fed-b58d-016df9dc7a5d => generated 0 bytes in 383 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 21:18:21.487744 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-65c34757-1e55-4ead-91e6-a50c4c90a429 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Calling method '.version_select at 0x7fdda2aef670>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:21.669320 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-c57b9311-211e-4ea0-adc9-d899b499d4d7 tempest-ServersAdminTestJSON-1511367024 tempest-ServersAdminTestJSON-1511367024-project-admin] 10.222.0.16 "POST /compute/v2.1/servers/e7598b4a-f70c-4186-88d1-6fd0057f8656/action" status: 202 len: 0 microversion: 2.1 time: 0.429080 Jul 16 21:18:21.686446 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1139/2352] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 21:18:21 2022] POST /compute/v2.1/servers/e7598b4a-f70c-4186-88d1-6fd0057f8656/action => generated 0 bytes in 448 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 21:18:21.876777 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-2583ddcb-9293-4675-9225-063fb03c05f5 tempest-ServersNegativeTestMultiTenantJSON-131244321 tempest-ServersNegativeTestMultiTenantJSON-131244321-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:21.876777 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-2583ddcb-9293-4675-9225-063fb03c05f5 tempest-ServersNegativeTestMultiTenantJSON-131244321 tempest-ServersNegativeTestMultiTenantJSON-131244321-project] [instance: 43169d4f-1214-419a-aa1e-def65182ed5a] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:18:21.889406 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2583ddcb-9293-4675-9225-063fb03c05f5 tempest-ServersNegativeTestMultiTenantJSON-131244321 tempest-ServersNegativeTestMultiTenantJSON-131244321-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:21.889871 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2583ddcb-9293-4675-9225-063fb03c05f5 tempest-ServersNegativeTestMultiTenantJSON-131244321 tempest-ServersNegativeTestMultiTenantJSON-131244321-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:21.891256 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2583ddcb-9293-4675-9225-063fb03c05f5 tempest-ServersNegativeTestMultiTenantJSON-131244321 tempest-ServersNegativeTestMultiTenantJSON-131244321-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:21.891716 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2583ddcb-9293-4675-9225-063fb03c05f5 tempest-ServersNegativeTestMultiTenantJSON-131244321 tempest-ServersNegativeTestMultiTenantJSON-131244321-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:21.932865 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-2583ddcb-9293-4675-9225-063fb03c05f5 tempest-ServersNegativeTestMultiTenantJSON-131244321 tempest-ServersNegativeTestMultiTenantJSON-131244321-project] HTTP exception thrown: Instance 43169d4f-1214-419a-aa1e-def65182ed5a could not be found. Jul 16 21:18:21.936099 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-2583ddcb-9293-4675-9225-063fb03c05f5 tempest-ServersNegativeTestMultiTenantJSON-131244321 tempest-ServersNegativeTestMultiTenantJSON-131244321-project] Request method failure captured: Jul 16 21:18:21.936099 ncdv-master-1 devstack@n-api.service[138681]: request: DELETE /compute/v2.1/servers/43169d4f-1214-419a-aa1e-def65182ed5a HTTP/1.1 Jul 16 21:18:21.936099 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:18:21.936099 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:18:21.936099 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:18:21.936099 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 21:18:21.936099 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:18:21.936099 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:18:21.936099 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:18:21.936099 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:18:21.936099 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0yr__FFOdjSTNIPApk01zZYGqBdJK1Z_YC9fSAMUW8MU6g5Wdk_2k40S7plcRJRNo4ZZhECatT_sd7p1zjNuhZ4MdDr2H_HLJrR7xcoh_YjKkZrF7lfJPN2a2HyNjysf5urdaxnDJTtIgaPJaGPVGf1AJxC2SA8hXvi3lOUalG5jxEQ Jul 16 21:18:21.936099 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:18:21.936099 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:18:21.936099 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:18:21.936099 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:18:21.936099 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:18:21.936099 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:18:21.936099 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: d498eaa349e84dcab5c975eeb69d1d6a Jul 16 21:18:21.936099 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-ServersNegativeTestMultiTenantJSON-131244321 Jul 16 21:18:21.936099 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:18:21.936099 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:18:21.936099 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/d498eaa349e84dcab5c975eeb69d1d6a"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/d498eaa349e84dcab5c975eeb69d1d6a"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_d498eaa349e84dcab5c975eeb69d1d6a"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/d498eaa349e84dcab5c975eeb69d1d6a"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:18:21.936099 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-ServersNegativeTestMultiTenantJSON-131244321 Jul 16 21:18:21.936099 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: d498eaa349e84dcab5c975eeb69d1d6a Jul 16 21:18:21.936099 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-ServersNegativeTestMultiTenantJSON-131244321 Jul 16 21:18:21.936099 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-ServersNegativeTestMultiTenantJSON-131244321-project Jul 16 21:18:21.936099 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:18:21.936099 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:18:21.936099 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: d35b09fb00874c05ac96b0a20e61ba10 Jul 16 21:18:21.936099 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-ServersNegativeTestMultiTenantJSON-131244321-project Jul 16 21:18:21.936099 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:18:21.936099 ncdv-master-1 devstack@n-api.service[138681]: exception: Instance 43169d4f-1214-419a-aa1e-def65182ed5a could not be found. Jul 16 21:18:21.936099 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': '43169d4f-1214-419a-aa1e-def65182ed5a'} Jul 16 21:18:21.936099 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:18:21.936099 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:18:21.936099 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:18:21.936099 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:18:21.936099 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:18:21.936099 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:18:21.936099 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:18:21.936099 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:18:21.936099 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:18:21.936099 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:18:21.936099 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 43169d4f-1214-419a-aa1e-def65182ed5a could not be found. Jul 16 21:18:21.936099 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:18:21.936099 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:18:21.936099 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:18:21.936099 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:18:21.936099 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:18:21.936099 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:18:21.936099 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:18:21.936099 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:18:21.936099 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:18:21.936099 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:18:21.936099 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Jul 16 21:18:21.936099 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Jul 16 21:18:21.936099 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Jul 16 21:18:21.936099 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Jul 16 21:18:21.936099 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:18:21.936099 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:18:21.936099 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:18:21.936099 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:18:21.939749 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 43169d4f-1214-419a-aa1e-def65182ed5a could not be found. Jul 16 21:18:21.939749 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:18:21.939749 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:18:21.939749 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:18:21.939749 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:18:21.939749 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:18:21.939749 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:18:21.939749 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:18:21.939749 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:18:21.939749 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 43169d4f-1214-419a-aa1e-def65182ed5a could not be found. Jul 16 21:18:21.939749 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:18:21.939749 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-2583ddcb-9293-4675-9225-063fb03c05f5 tempest-ServersNegativeTestMultiTenantJSON-131244321 tempest-ServersNegativeTestMultiTenantJSON-131244321-project] Returning 404 to user: Instance 43169d4f-1214-419a-aa1e-def65182ed5a could not be found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:18:21.945465 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-2583ddcb-9293-4675-9225-063fb03c05f5 tempest-ServersNegativeTestMultiTenantJSON-131244321 tempest-ServersNegativeTestMultiTenantJSON-131244321-project] 10.222.0.16 "DELETE /compute/v2.1/servers/43169d4f-1214-419a-aa1e-def65182ed5a" status: 404 len: 111 microversion: 2.1 time: 0.257804 Jul 16 21:18:21.956203 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1140/2353] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:18:21 2022] DELETE /compute/v2.1/servers/43169d4f-1214-419a-aa1e-def65182ed5a => generated 111 bytes in 267 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:18:21.964417 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-4b627823-da20-4505-8729-e6c48e0852bf tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Calling method '.version_select at 0x7fdd9cfa6dc0>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:22.038080 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-65c34757-1e55-4ead-91e6-a50c4c90a429 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] 10.222.0.16 "DELETE /compute/v2.1/os-security-groups/5d9ea170-9999-4816-bc4f-070762e7181b" status: 202 len: 0 microversion: 2.1 time: 0.562093 Jul 16 21:18:22.038699 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1214/2354] 10.222.0.16 () {58 vars in 1271 bytes} [Sat Jul 16 21:18:21 2022] DELETE /compute/v2.1/os-security-groups/5d9ea170-9999-4816-bc4f-070762e7181b => generated 0 bytes in 564 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 21:18:22.048253 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-2d06041e-94da-4526-99fc-5258acbeff3a tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:22.049106 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-2d06041e-94da-4526-99fc-5258acbeff3a tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] [instance: 6baac85b-f81c-4e3a-9f96-633cffccfd74] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:18:22.061489 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2d06041e-94da-4526-99fc-5258acbeff3a tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:22.061663 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2d06041e-94da-4526-99fc-5258acbeff3a tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:22.067586 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2d06041e-94da-4526-99fc-5258acbeff3a tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:22.068901 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2d06041e-94da-4526-99fc-5258acbeff3a tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:22.316730 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-2d06041e-94da-4526-99fc-5258acbeff3a tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd4144b5388b46a989a41a2f556bb508', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a90d4bcd25ff4bf68bdc194180fbbe09', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:18:22.326125 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-2d06041e-94da-4526-99fc-5258acbeff3a tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] 10.222.0.16 "GET /compute/v2.1/servers/6baac85b-f81c-4e3a-9f96-633cffccfd74" status: 200 len: 1318 microversion: 2.1 time: 0.286134 Jul 16 21:18:22.326706 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1215/2355] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:18:22 2022] GET /compute/v2.1/servers/6baac85b-f81c-4e3a-9f96-633cffccfd74 => generated 1318 bytes in 287 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:18:22.348183 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-676132e1-c481-453b-8b15-cf9e9377f0f6 tempest-ServersAdminTestJSON-1511367024 tempest-ServersAdminTestJSON-1511367024-project-admin] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:22.349081 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-676132e1-c481-453b-8b15-cf9e9377f0f6 tempest-ServersAdminTestJSON-1511367024 tempest-ServersAdminTestJSON-1511367024-project-admin] [instance: e7598b4a-f70c-4186-88d1-6fd0057f8656] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:18:22.357254 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-676132e1-c481-453b-8b15-cf9e9377f0f6 tempest-ServersAdminTestJSON-1511367024 tempest-ServersAdminTestJSON-1511367024-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:22.357944 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-676132e1-c481-453b-8b15-cf9e9377f0f6 tempest-ServersAdminTestJSON-1511367024 tempest-ServersAdminTestJSON-1511367024-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:22.359459 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-676132e1-c481-453b-8b15-cf9e9377f0f6 tempest-ServersAdminTestJSON-1511367024 tempest-ServersAdminTestJSON-1511367024-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:22.360096 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-676132e1-c481-453b-8b15-cf9e9377f0f6 tempest-ServersAdminTestJSON-1511367024 tempest-ServersAdminTestJSON-1511367024-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:22.540040 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-676132e1-c481-453b-8b15-cf9e9377f0f6 tempest-ServersAdminTestJSON-1511367024 tempest-ServersAdminTestJSON-1511367024-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/e7598b4a-f70c-4186-88d1-6fd0057f8656" status: 200 len: 1614 microversion: 2.1 time: 0.211266 Jul 16 21:18:22.540629 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1216/2356] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:18:22 2022] GET /compute/v2.1/servers/e7598b4a-f70c-4186-88d1-6fd0057f8656 => generated 1614 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:18:22.546803 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-cdc80c21-b42a-416b-ab36-2f7cf2cc7092 tempest-ServersNegativeTestMultiTenantJSON-1135003248 tempest-ServersNegativeTestMultiTenantJSON-1135003248-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:22.547617 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-cdc80c21-b42a-416b-ab36-2f7cf2cc7092 tempest-ServersNegativeTestMultiTenantJSON-1135003248 tempest-ServersNegativeTestMultiTenantJSON-1135003248-project] [instance: 43169d4f-1214-419a-aa1e-def65182ed5a] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:18:22.554707 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cdc80c21-b42a-416b-ab36-2f7cf2cc7092 tempest-ServersNegativeTestMultiTenantJSON-1135003248 tempest-ServersNegativeTestMultiTenantJSON-1135003248-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:22.555207 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cdc80c21-b42a-416b-ab36-2f7cf2cc7092 tempest-ServersNegativeTestMultiTenantJSON-1135003248 tempest-ServersNegativeTestMultiTenantJSON-1135003248-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:22.556129 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cdc80c21-b42a-416b-ab36-2f7cf2cc7092 tempest-ServersNegativeTestMultiTenantJSON-1135003248 tempest-ServersNegativeTestMultiTenantJSON-1135003248-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:22.556765 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cdc80c21-b42a-416b-ab36-2f7cf2cc7092 tempest-ServersNegativeTestMultiTenantJSON-1135003248 tempest-ServersNegativeTestMultiTenantJSON-1135003248-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:22.740396 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-cdc80c21-b42a-416b-ab36-2f7cf2cc7092 tempest-ServersNegativeTestMultiTenantJSON-1135003248 tempest-ServersNegativeTestMultiTenantJSON-1135003248-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '50cdc134e712487196f6623c5cf14592', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ba229aaef0ec4ddcadd1d940376c31ac', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:18:22.749989 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-cdc80c21-b42a-416b-ab36-2f7cf2cc7092 tempest-ServersNegativeTestMultiTenantJSON-1135003248 tempest-ServersNegativeTestMultiTenantJSON-1135003248-project] 10.222.0.16 "GET /compute/v2.1/servers/43169d4f-1214-419a-aa1e-def65182ed5a" status: 200 len: 1527 microversion: 2.1 time: 0.207930 Jul 16 21:18:22.762275 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1217/2357] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:18:22 2022] GET /compute/v2.1/servers/43169d4f-1214-419a-aa1e-def65182ed5a => generated 1527 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:18:22.766829 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-cafb5fe5-aca3-4af9-b060-328bd6d75146 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Calling method '.version_select at 0x7fdda2b2b160>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:22.792521 ncdv-master-1 devstack@n-api.service[138680]: DEBUG neutronclient.v2_0.client [None req-cafb5fe5-aca3-4af9-b060-328bd6d75146 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Error message: {"NeutronError": {"type": "SecurityGroupNotFound", "message": "Security group 5d9ea170-9999-4816-bc4f-070762e7181b does not exist", "detail": ""}} {{(pid=138680) _handle_fault_response /usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py:262}} Jul 16 21:18:22.793487 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.network.security_group_api [None req-cafb5fe5-aca3-4af9-b060-328bd6d75146 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Neutron security group 5d9ea170-9999-4816-bc4f-070762e7181b not found {{(pid=138680) get /opt/stack/nova/nova/network/security_group_api.py:313}} Jul 16 21:18:22.794094 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-cafb5fe5-aca3-4af9-b060-328bd6d75146 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] HTTP exception thrown: Security group 5d9ea170-9999-4816-bc4f-070762e7181b does not exist Jul 16 21:18:22.794094 ncdv-master-1 devstack@n-api.service[138680]: Neutron server returns request_ids: ['req-bceaa606-3283-440a-b0ef-fbdda3a50397'] Jul 16 21:18:22.796251 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-cafb5fe5-aca3-4af9-b060-328bd6d75146 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Request method failure captured: Jul 16 21:18:22.796251 ncdv-master-1 devstack@n-api.service[138680]: request: GET /compute/v2.1/os-security-groups/5d9ea170-9999-4816-bc4f-070762e7181b HTTP/1.1 Jul 16 21:18:22.796251 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:18:22.796251 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:18:22.796251 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:18:22.796251 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 0 Jul 16 21:18:22.796251 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:18:22.796251 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:18:22.796251 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:18:22.796251 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:18:22.796251 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0yrk-HsfYKsyovLSaEtNxaoXy7a3rLGNlsRPiPP76ZvvrKkKyGd-uekosXQxpFgkRnItBdviN4xr49Rerg5AyvLSzsyRY-Um0r3BRAvyZrzmCDYUpWbXk1uyEIb3Af4U3Y3nXIJgJAuDVx9z2NGSvEGCsGz12WlyyGCNfVSADFcYJP8 Jul 16 21:18:22.796251 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:18:22.796251 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:18:22.796251 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:18:22.796251 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:18:22.796251 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:18:22.796251 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:18:22.796251 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: e353258f9c3048ef84683294e2b8579d Jul 16 21:18:22.796251 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-SecurityGroupsTestJSON-438340169 Jul 16 21:18:22.796251 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:18:22.796251 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:18:22.796251 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/e353258f9c3048ef84683294e2b8579d"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/e353258f9c3048ef84683294e2b8579d"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_e353258f9c3048ef84683294e2b8579d"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/e353258f9c3048ef84683294e2b8579d"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:18:22.796251 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-SecurityGroupsTestJSON-438340169 Jul 16 21:18:22.796251 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: e353258f9c3048ef84683294e2b8579d Jul 16 21:18:22.796251 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-SecurityGroupsTestJSON-438340169 Jul 16 21:18:22.796251 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-SecurityGroupsTestJSON-438340169-project Jul 16 21:18:22.796251 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:18:22.796251 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:18:22.796251 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: d0008fffad8e4708b2c01e03da92c47d Jul 16 21:18:22.796251 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-SecurityGroupsTestJSON-438340169-project Jul 16 21:18:22.796251 ncdv-master-1 devstack@n-api.service[138680]: method: Jul 16 21:18:22.796251 ncdv-master-1 devstack@n-api.service[138680]: exception: Security group 5d9ea170-9999-4816-bc4f-070762e7181b does not exist Jul 16 21:18:22.796251 ncdv-master-1 devstack@n-api.service[138680]: Neutron server returns request_ids: ['req-bceaa606-3283-440a-b0ef-fbdda3a50397'] Jul 16 21:18:22.796251 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': '5d9ea170-9999-4816-bc4f-070762e7181b'} Jul 16 21:18:22.796251 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:18:22.796251 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:18:22.796251 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/security_group_api.py", line 309, in get Jul 16 21:18:22.796251 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi group = neutron.show_security_group(id).get('security_group') Jul 16 21:18:22.796251 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 197, in wrapper Jul 16 21:18:22.796251 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Jul 16 21:18:22.796251 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 1040, in show_security_group Jul 16 21:18:22.796251 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return self.get(self.security_group_path % (security_group), Jul 16 21:18:22.796251 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 197, in wrapper Jul 16 21:18:22.796251 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Jul 16 21:18:22.796251 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 356, in get Jul 16 21:18:22.796251 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return self.retry_request("GET", action, body=body, Jul 16 21:18:22.796251 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 197, in wrapper Jul 16 21:18:22.796251 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Jul 16 21:18:22.796251 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 333, in retry_request Jul 16 21:18:22.796251 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return self.do_request(method, action, body=body, Jul 16 21:18:22.796251 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 197, in wrapper Jul 16 21:18:22.796251 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Jul 16 21:18:22.796251 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 297, in do_request Jul 16 21:18:22.796251 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi self._handle_fault_response(status_code, replybody, resp) Jul 16 21:18:22.796251 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 197, in wrapper Jul 16 21:18:22.796251 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Jul 16 21:18:22.796251 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 272, in _handle_fault_response Jul 16 21:18:22.796251 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi exception_handler_v20(status_code, error_body) Jul 16 21:18:22.796251 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 90, in exception_handler_v20 Jul 16 21:18:22.796251 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise client_exc(message=error_message, Jul 16 21:18:22.796251 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi neutronclient.common.exceptions.NotFound: Security group 5d9ea170-9999-4816-bc4f-070762e7181b does not exist Jul 16 21:18:22.800394 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-bceaa606-3283-440a-b0ef-fbdda3a50397'] Jul 16 21:18:22.800394 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:18:22.800394 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:18:22.800394 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:18:22.800394 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:18:22.800394 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/security_groups.py", line 158, in show Jul 16 21:18:22.800394 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi security_group = security_group_api.get(context, id) Jul 16 21:18:22.800394 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/security_group_api.py", line 314, in get Jul 16 21:18:22.800394 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.SecurityGroupNotFound(str(e)) Jul 16 21:18:22.800394 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.SecurityGroupNotFound: Security group 5d9ea170-9999-4816-bc4f-070762e7181b does not exist Jul 16 21:18:22.800394 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-bceaa606-3283-440a-b0ef-fbdda3a50397'] Jul 16 21:18:22.800394 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:18:22.800394 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:18:22.800394 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:18:22.800394 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:18:22.800394 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:18:22.800394 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:18:22.800394 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:18:22.800394 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:18:22.800394 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Jul 16 21:18:22.800394 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Jul 16 21:18:22.800394 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:18:22.800394 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:18:22.800394 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/security_groups.py", line 160, in show Jul 16 21:18:22.800394 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=exp.format_message()) Jul 16 21:18:22.800394 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Security group 5d9ea170-9999-4816-bc4f-070762e7181b does not exist Jul 16 21:18:22.800394 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-bceaa606-3283-440a-b0ef-fbdda3a50397'] Jul 16 21:18:22.800394 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:18:22.800394 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:18:22.800394 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:18:22.800394 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:18:22.800394 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:18:22.800394 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:18:22.800394 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:18:22.800394 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:18:22.800394 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Security group 5d9ea170-9999-4816-bc4f-070762e7181b does not exist Jul 16 21:18:22.800394 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-bceaa606-3283-440a-b0ef-fbdda3a50397'] Jul 16 21:18:22.800394 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:18:22.800394 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-cafb5fe5-aca3-4af9-b060-328bd6d75146 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Returning 404 to user: Security group 5d9ea170-9999-4816-bc4f-070762e7181b does not exist Jul 16 21:18:22.800394 ncdv-master-1 devstack@n-api.service[138680]: Neutron server returns request_ids: ['req-bceaa606-3283-440a-b0ef-fbdda3a50397'] {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:18:22.800394 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-cafb5fe5-aca3-4af9-b060-328bd6d75146 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] 10.222.0.16 "GET /compute/v2.1/os-security-groups/5d9ea170-9999-4816-bc4f-070762e7181b" status: 404 len: 194 microversion: 2.1 time: 0.037175 Jul 16 21:18:22.804233 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1218/2358] 10.222.0.16 () {58 vars in 1268 bytes} [Sat Jul 16 21:18:22 2022] GET /compute/v2.1/os-security-groups/5d9ea170-9999-4816-bc4f-070762e7181b => generated 194 bytes in 42 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:18:22.809821 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-9d25f666-d721-41fe-84b1-df254aff71ec tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Calling method '.version_select at 0x7fdda29b21f0>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:25.830686 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell0:SELECT=2 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:18:28.436773 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell0:SELECT=2 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:18:29.195527 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell0:SELECT=4 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:18:30.691808 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell0:SELECT=8 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:18:31.686352 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-4b627823-da20-4505-8729-e6c48e0852bf tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] 10.222.0.16 "GET /compute/v2.1/images/04088d67-f393-4fbb-8c56-7c2bc534a560" status: 200 len: 1463 microversion: 2.1 time: 9.728952 Jul 16 21:18:31.686928 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1141/2359] 10.222.0.16 () {58 vars in 1232 bytes} [Sat Jul 16 21:18:21 2022] GET /compute/v2.1/images/04088d67-f393-4fbb-8c56-7c2bc534a560 => generated 1463 bytes in 9730 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:18:31.691653 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-9d25f666-d721-41fe-84b1-df254aff71ec tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] 10.222.0.16 "GET /compute/v2.1/images/eee6448e-8c8a-42ef-b4df-d19c5d9ae531" status: 200 len: 1439 microversion: 2.1 time: 8.886159 Jul 16 21:18:31.698037 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1219/2360] 10.222.0.16 () {58 vars in 1232 bytes} [Sat Jul 16 21:18:22 2022] GET /compute/v2.1/images/eee6448e-8c8a-42ef-b4df-d19c5d9ae531 => generated 1439 bytes in 8892 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:18:31.699837 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-9417becb-9661-411b-9ea0-fdf0177cf3c7 tempest-ServersAdminTestJSON-1511367024 tempest-ServersAdminTestJSON-1511367024-project-admin] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:31.701981 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-9417becb-9661-411b-9ea0-fdf0177cf3c7 tempest-ServersAdminTestJSON-1511367024 tempest-ServersAdminTestJSON-1511367024-project-admin] [instance: e7598b4a-f70c-4186-88d1-6fd0057f8656] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:18:31.705973 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-525eab4b-08a7-4f1d-94bb-bf384a1df5f6 req-a0cc58c0-3c38-4815-8e43-64d62a31c7c0 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "ed8dcb97-7546-45ef-9e9d-a9bb03a1b419", "name": "network-vif-plugged", "status": "completed", "tag": "033ca6ad-5781-4719-b3b5-986dcfd7de3d"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:18:31.716608 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9417becb-9661-411b-9ea0-fdf0177cf3c7 tempest-ServersAdminTestJSON-1511367024 tempest-ServersAdminTestJSON-1511367024-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:31.717108 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9417becb-9661-411b-9ea0-fdf0177cf3c7 tempest-ServersAdminTestJSON-1511367024 tempest-ServersAdminTestJSON-1511367024-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:31.717394 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-525eab4b-08a7-4f1d-94bb-bf384a1df5f6 req-a0cc58c0-3c38-4815-8e43-64d62a31c7c0 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:31.717923 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9417becb-9661-411b-9ea0-fdf0177cf3c7 tempest-ServersAdminTestJSON-1511367024 tempest-ServersAdminTestJSON-1511367024-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:31.718330 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-525eab4b-08a7-4f1d-94bb-bf384a1df5f6 req-a0cc58c0-3c38-4815-8e43-64d62a31c7c0 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:31.718456 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9417becb-9661-411b-9ea0-fdf0177cf3c7 tempest-ServersAdminTestJSON-1511367024 tempest-ServersAdminTestJSON-1511367024-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:31.729956 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-525eab4b-08a7-4f1d-94bb-bf384a1df5f6 req-a0cc58c0-3c38-4815-8e43-64d62a31c7c0 service nova] Creating event network-vif-plugged:033ca6ad-5781-4719-b3b5-986dcfd7de3d for instance ed8dcb97-7546-45ef-9e9d-a9bb03a1b419 on nch2-master-1 Jul 16 21:18:31.735555 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-525eab4b-08a7-4f1d-94bb-bf384a1df5f6 req-a0cc58c0-3c38-4815-8e43-64d62a31c7c0 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.035893 Jul 16 21:18:31.736722 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1220/2361] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:18:31 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:18:31.742366 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-dcfc81b0-6e6f-4d20-a74e-20226714db0a tempest-ServersNegativeTestMultiTenantJSON-131244321 tempest-ServersNegativeTestMultiTenantJSON-131244321-project] Action: 'update', calling method: >, body: {"server": {"name": "43169d4f-1214-419a-aa1e-def65182ed5a_new"}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:18:31.743961 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-dcfc81b0-6e6f-4d20-a74e-20226714db0a tempest-ServersNegativeTestMultiTenantJSON-131244321 tempest-ServersNegativeTestMultiTenantJSON-131244321-project] [instance: 43169d4f-1214-419a-aa1e-def65182ed5a] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:18:31.751449 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-dcfc81b0-6e6f-4d20-a74e-20226714db0a tempest-ServersNegativeTestMultiTenantJSON-131244321 tempest-ServersNegativeTestMultiTenantJSON-131244321-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:31.752176 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-dcfc81b0-6e6f-4d20-a74e-20226714db0a tempest-ServersNegativeTestMultiTenantJSON-131244321 tempest-ServersNegativeTestMultiTenantJSON-131244321-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:31.753120 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-dcfc81b0-6e6f-4d20-a74e-20226714db0a tempest-ServersNegativeTestMultiTenantJSON-131244321 tempest-ServersNegativeTestMultiTenantJSON-131244321-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:31.754015 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-dcfc81b0-6e6f-4d20-a74e-20226714db0a tempest-ServersNegativeTestMultiTenantJSON-131244321 tempest-ServersNegativeTestMultiTenantJSON-131244321-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:31.770141 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-dcfc81b0-6e6f-4d20-a74e-20226714db0a tempest-ServersNegativeTestMultiTenantJSON-131244321 tempest-ServersNegativeTestMultiTenantJSON-131244321-project] HTTP exception thrown: Instance 43169d4f-1214-419a-aa1e-def65182ed5a could not be found. Jul 16 21:18:31.771976 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-dcfc81b0-6e6f-4d20-a74e-20226714db0a tempest-ServersNegativeTestMultiTenantJSON-131244321 tempest-ServersNegativeTestMultiTenantJSON-131244321-project] Request method failure captured: Jul 16 21:18:31.771976 ncdv-master-1 devstack@n-api.service[138680]: request: PUT /compute/v2.1/servers/43169d4f-1214-419a-aa1e-def65182ed5a HTTP/1.1 Jul 16 21:18:31.771976 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:18:31.771976 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:18:31.771976 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:18:31.771976 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 64 Jul 16 21:18:31.771976 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:18:31.771976 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:18:31.771976 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:18:31.771976 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:18:31.771976 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0yr__FFOdjSTNIPApk01zZYGqBdJK1Z_YC9fSAMUW8MU6g5Wdk_2k40S7plcRJRNo4ZZhECatT_sd7p1zjNuhZ4MdDr2H_HLJrR7xcoh_YjKkZrF7lfJPN2a2HyNjysf5urdaxnDJTtIgaPJaGPVGf1AJxC2SA8hXvi3lOUalG5jxEQ Jul 16 21:18:31.771976 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:18:31.771976 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:18:31.771976 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:18:31.771976 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:18:31.771976 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:18:31.771976 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:18:31.771976 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: d498eaa349e84dcab5c975eeb69d1d6a Jul 16 21:18:31.771976 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-ServersNegativeTestMultiTenantJSON-131244321 Jul 16 21:18:31.771976 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:18:31.771976 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:18:31.771976 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/d498eaa349e84dcab5c975eeb69d1d6a"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/d498eaa349e84dcab5c975eeb69d1d6a"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_d498eaa349e84dcab5c975eeb69d1d6a"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/d498eaa349e84dcab5c975eeb69d1d6a"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:18:31.771976 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-ServersNegativeTestMultiTenantJSON-131244321 Jul 16 21:18:31.771976 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: d498eaa349e84dcab5c975eeb69d1d6a Jul 16 21:18:31.771976 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-ServersNegativeTestMultiTenantJSON-131244321 Jul 16 21:18:31.771976 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-ServersNegativeTestMultiTenantJSON-131244321-project Jul 16 21:18:31.771976 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:18:31.771976 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:18:31.771976 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: d35b09fb00874c05ac96b0a20e61ba10 Jul 16 21:18:31.771976 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-ServersNegativeTestMultiTenantJSON-131244321-project Jul 16 21:18:31.771976 ncdv-master-1 devstack@n-api.service[138680]: Jul 16 21:18:31.771976 ncdv-master-1 devstack@n-api.service[138680]: {"server": {"name": "43169d4f-1214-419a-aa1e-def65182ed5a_new"}} Jul 16 21:18:31.771976 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 21:18:31.771976 ncdv-master-1 devstack@n-api.service[138680]: exception: Instance 43169d4f-1214-419a-aa1e-def65182ed5a could not be found. Jul 16 21:18:31.771976 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': '43169d4f-1214-419a-aa1e-def65182ed5a', 'body': {'server': {'name': '43169d4f-1214-419a-aa1e-def65182ed5a_new'}}} Jul 16 21:18:31.771976 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:18:31.771976 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:18:31.771976 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:18:31.771976 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:18:31.771976 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:18:31.771976 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:18:31.771976 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:18:31.771976 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:18:31.771976 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:18:31.771976 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:18:31.771976 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 43169d4f-1214-419a-aa1e-def65182ed5a could not be found. Jul 16 21:18:31.771976 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:18:31.771976 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:18:31.771976 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:18:31.771976 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:18:31.771976 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:18:31.771976 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:18:31.771976 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:18:31.771976 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:18:31.771976 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:18:31.771976 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:18:31.771976 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 16 21:18:31.771976 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 16 21:18:31.771976 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 16 21:18:31.771976 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 16 21:18:31.771976 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 16 21:18:31.771976 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 16 21:18:31.771976 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi [Previous line repeated 1 more time] Jul 16 21:18:31.771976 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 915, in update Jul 16 21:18:31.776043 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_server(ctxt, req, id, is_detail=True) Jul 16 21:18:31.776043 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:18:31.776043 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:18:31.776043 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:18:31.776043 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:18:31.776043 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 43169d4f-1214-419a-aa1e-def65182ed5a could not be found. Jul 16 21:18:31.776043 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:18:31.776043 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:18:31.776043 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:18:31.776043 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:18:31.776043 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:18:31.776043 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:18:31.776043 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:18:31.776043 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:18:31.776043 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 43169d4f-1214-419a-aa1e-def65182ed5a could not be found. Jul 16 21:18:31.776043 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:18:31.776043 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-dcfc81b0-6e6f-4d20-a74e-20226714db0a tempest-ServersNegativeTestMultiTenantJSON-131244321 tempest-ServersNegativeTestMultiTenantJSON-131244321-project] Returning 404 to user: Instance 43169d4f-1214-419a-aa1e-def65182ed5a could not be found. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:18:31.776043 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-dcfc81b0-6e6f-4d20-a74e-20226714db0a tempest-ServersNegativeTestMultiTenantJSON-131244321 tempest-ServersNegativeTestMultiTenantJSON-131244321-project] 10.222.0.16 "PUT /compute/v2.1/servers/43169d4f-1214-419a-aa1e-def65182ed5a" status: 404 len: 111 microversion: 2.1 time: 0.035499 Jul 16 21:18:31.776043 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1221/2362] 10.222.0.16 () {60 vars in 1255 bytes} [Sat Jul 16 21:18:31 2022] PUT /compute/v2.1/servers/43169d4f-1214-419a-aa1e-def65182ed5a => generated 111 bytes in 37 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:18:31.779698 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-ec15b1c8-6169-4e50-8db3-ef77165f0bf0 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Calling method '.version_select at 0x7fdda2b90e50>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:31.909320 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-9417becb-9661-411b-9ea0-fdf0177cf3c7 tempest-ServersAdminTestJSON-1511367024 tempest-ServersAdminTestJSON-1511367024-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/e7598b4a-f70c-4186-88d1-6fd0057f8656" status: 200 len: 1614 microversion: 2.1 time: 0.220974 Jul 16 21:18:31.910635 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1142/2363] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:18:31 2022] GET /compute/v2.1/servers/e7598b4a-f70c-4186-88d1-6fd0057f8656 => generated 1614 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:18:31.915941 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-063ec06d-411e-4375-b230-286e1009d44a tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:31.917795 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-063ec06d-411e-4375-b230-286e1009d44a tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] [instance: 6baac85b-f81c-4e3a-9f96-633cffccfd74] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:18:31.925365 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-063ec06d-411e-4375-b230-286e1009d44a tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:31.926633 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-063ec06d-411e-4375-b230-286e1009d44a tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:31.927236 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-063ec06d-411e-4375-b230-286e1009d44a tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:31.927834 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-063ec06d-411e-4375-b230-286e1009d44a tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:32.134119 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-063ec06d-411e-4375-b230-286e1009d44a tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd4144b5388b46a989a41a2f556bb508', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a90d4bcd25ff4bf68bdc194180fbbe09', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:18:32.153556 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-063ec06d-411e-4375-b230-286e1009d44a tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] 10.222.0.16 "GET /compute/v2.1/servers/6baac85b-f81c-4e3a-9f96-633cffccfd74" status: 200 len: 1507 microversion: 2.1 time: 0.241517 Jul 16 21:18:32.165198 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1143/2364] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:18:31 2022] GET /compute/v2.1/servers/6baac85b-f81c-4e3a-9f96-633cffccfd74 => generated 1507 bytes in 254 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:18:32.203209 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-ec15b1c8-6169-4e50-8db3-ef77165f0bf0 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] 10.222.0.16 "DELETE /compute/v2.1/os-security-groups/171f6d9b-ebb3-4079-a770-ac1c15c2977a" status: 202 len: 0 microversion: 2.1 time: 0.427677 Jul 16 21:18:32.208111 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1222/2365] 10.222.0.16 () {58 vars in 1271 bytes} [Sat Jul 16 21:18:31 2022] DELETE /compute/v2.1/os-security-groups/171f6d9b-ebb3-4079-a770-ac1c15c2977a => generated 0 bytes in 433 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 21:18:32.223647 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-1ea16466-d0c8-415c-92ba-3963302fd60e tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Calling method '.version_select at 0x7fdd9c65eee0>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:32.290470 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-1ea16466-d0c8-415c-92ba-3963302fd60e tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] 10.222.0.16 "GET /compute/v2.1/images/04088d67-f393-4fbb-8c56-7c2bc534a560" status: 200 len: 1463 microversion: 2.1 time: 0.080714 Jul 16 21:18:32.291869 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1223/2366] 10.222.0.16 () {58 vars in 1232 bytes} [Sat Jul 16 21:18:32 2022] GET /compute/v2.1/images/04088d67-f393-4fbb-8c56-7c2bc534a560 => generated 1463 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:18:32.297598 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-6fb85f9a-7acc-4506-ab5b-c4eae5a0a085 tempest-ServersNegativeTestMultiTenantJSON-1135003248 tempest-ServersNegativeTestMultiTenantJSON-1135003248-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:32.298366 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-6fb85f9a-7acc-4506-ab5b-c4eae5a0a085 tempest-ServersNegativeTestMultiTenantJSON-1135003248 tempest-ServersNegativeTestMultiTenantJSON-1135003248-project] [instance: 43169d4f-1214-419a-aa1e-def65182ed5a] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:18:32.307746 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6fb85f9a-7acc-4506-ab5b-c4eae5a0a085 tempest-ServersNegativeTestMultiTenantJSON-1135003248 tempest-ServersNegativeTestMultiTenantJSON-1135003248-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:32.308903 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6fb85f9a-7acc-4506-ab5b-c4eae5a0a085 tempest-ServersNegativeTestMultiTenantJSON-1135003248 tempest-ServersNegativeTestMultiTenantJSON-1135003248-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:32.309383 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6fb85f9a-7acc-4506-ab5b-c4eae5a0a085 tempest-ServersNegativeTestMultiTenantJSON-1135003248 tempest-ServersNegativeTestMultiTenantJSON-1135003248-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:32.310097 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6fb85f9a-7acc-4506-ab5b-c4eae5a0a085 tempest-ServersNegativeTestMultiTenantJSON-1135003248 tempest-ServersNegativeTestMultiTenantJSON-1135003248-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:32.331838 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-d0df8547-8d42-458c-8a6f-510956ec0edf tempest-ListImagesTestJSON-945500242 tempest-ListImagesTestJSON-945500242-project] Calling method '.version_select at 0x7fdd9d087ee0>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:32.340685 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-6fb85f9a-7acc-4506-ab5b-c4eae5a0a085 tempest-ServersNegativeTestMultiTenantJSON-1135003248 tempest-ServersNegativeTestMultiTenantJSON-1135003248-project] [instance: 43169d4f-1214-419a-aa1e-def65182ed5a] Going to try to terminate instance {{(pid=138680) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 16 21:18:32.387325 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-d0df8547-8d42-458c-8a6f-510956ec0edf tempest-ListImagesTestJSON-945500242 tempest-ListImagesTestJSON-945500242-project] 10.222.0.16 "GET /compute/v2.1/images/1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9" status: 200 len: 784 microversion: 2.1 time: 0.216870 Jul 16 21:18:32.388543 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1144/2367] 10.222.0.16 () {58 vars in 1232 bytes} [Sat Jul 16 21:18:32 2022] GET /compute/v2.1/images/1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9 => generated 784 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:18:32.393286 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-a924c260-fc01-47c3-b4aa-5f83d2c2aef2 tempest-ServersAdminTestJSON-1511367024 tempest-ServersAdminTestJSON-1511367024-project-admin] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:32.398203 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-a924c260-fc01-47c3-b4aa-5f83d2c2aef2 tempest-ServersAdminTestJSON-1511367024 tempest-ServersAdminTestJSON-1511367024-project-admin] Searching by: {'host': 'nch1-master-1', 'deleted': False} {{(pid=138681) get_all /opt/stack/nova/nova/compute/api.py:2962}} Jul 16 21:18:32.404344 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a924c260-fc01-47c3-b4aa-5f83d2c2aef2 tempest-ServersAdminTestJSON-1511367024 tempest-ServersAdminTestJSON-1511367024-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:32.404799 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a924c260-fc01-47c3-b4aa-5f83d2c2aef2 tempest-ServersAdminTestJSON-1511367024 tempest-ServersAdminTestJSON-1511367024-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:32.405608 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a924c260-fc01-47c3-b4aa-5f83d2c2aef2 tempest-ServersAdminTestJSON-1511367024 tempest-ServersAdminTestJSON-1511367024-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:32.406141 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a924c260-fc01-47c3-b4aa-5f83d2c2aef2 tempest-ServersAdminTestJSON-1511367024 tempest-ServersAdminTestJSON-1511367024-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:32.522366 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.multi_cell_list [None req-a924c260-fc01-47c3-b4aa-5f83d2c2aef2 tempest-ServersAdminTestJSON-1511367024 tempest-ServersAdminTestJSON-1511367024-project-admin] Listed batch of 5 results from cell out of 1000 limit. Returned 5 total so far. {{(pid=138681) do_query /opt/stack/nova/nova/compute/multi_cell_list.py:373}} Jul 16 21:18:32.594564 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-a924c260-fc01-47c3-b4aa-5f83d2c2aef2 tempest-ServersAdminTestJSON-1511367024 tempest-ServersAdminTestJSON-1511367024-project-admin] 10.222.0.16 "GET /compute/v2.1/servers?host=nch1-master-1&all_tenants=1" status: 200 len: 1659 microversion: 2.1 time: 0.204814 Jul 16 21:18:32.595600 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1145/2368] 10.222.0.16 () {58 vars in 1189 bytes} [Sat Jul 16 21:18:32 2022] GET /compute/v2.1/servers?host=nch1-master-1&all_tenants=1 => generated 1659 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:18:32.600348 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-727170f3-049f-48c5-997c-c25fe14afed0 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:32.601331 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-727170f3-049f-48c5-997c-c25fe14afed0 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] [instance: 6baac85b-f81c-4e3a-9f96-633cffccfd74] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:18:32.609620 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-727170f3-049f-48c5-997c-c25fe14afed0 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:32.609620 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-727170f3-049f-48c5-997c-c25fe14afed0 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:32.618395 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-727170f3-049f-48c5-997c-c25fe14afed0 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:32.618395 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-727170f3-049f-48c5-997c-c25fe14afed0 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:32.702557 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-6fb85f9a-7acc-4506-ab5b-c4eae5a0a085 tempest-ServersNegativeTestMultiTenantJSON-1135003248 tempest-ServersNegativeTestMultiTenantJSON-1135003248-project] 10.222.0.16 "DELETE /compute/v2.1/servers/43169d4f-1214-419a-aa1e-def65182ed5a" status: 204 len: 0 microversion: 2.1 time: 0.408870 Jul 16 21:18:32.703434 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1224/2369] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:18:32 2022] DELETE /compute/v2.1/servers/43169d4f-1214-419a-aa1e-def65182ed5a => generated 0 bytes in 411 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 16 21:18:32.709930 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-8758c677-f7ac-4ba3-9c48-5db5d7414ee6 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Calling method '.version_select at 0x7fdda2b39310>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:32.756656 ncdv-master-1 devstack@n-api.service[138680]: DEBUG neutronclient.v2_0.client [None req-8758c677-f7ac-4ba3-9c48-5db5d7414ee6 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Error message: {"NeutronError": {"type": "SecurityGroupNotFound", "message": "Security group 171f6d9b-ebb3-4079-a770-ac1c15c2977a does not exist", "detail": ""}} {{(pid=138680) _handle_fault_response /usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py:262}} Jul 16 21:18:32.758141 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.network.security_group_api [None req-8758c677-f7ac-4ba3-9c48-5db5d7414ee6 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Neutron security group 171f6d9b-ebb3-4079-a770-ac1c15c2977a not found {{(pid=138680) get /opt/stack/nova/nova/network/security_group_api.py:313}} Jul 16 21:18:32.763678 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-8758c677-f7ac-4ba3-9c48-5db5d7414ee6 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] HTTP exception thrown: Security group 171f6d9b-ebb3-4079-a770-ac1c15c2977a does not exist Jul 16 21:18:32.763678 ncdv-master-1 devstack@n-api.service[138680]: Neutron server returns request_ids: ['req-ebf8f7a6-cc0e-45ee-bbde-48861021b998'] Jul 16 21:18:32.765803 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-8758c677-f7ac-4ba3-9c48-5db5d7414ee6 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Request method failure captured: Jul 16 21:18:32.765803 ncdv-master-1 devstack@n-api.service[138680]: request: GET /compute/v2.1/os-security-groups/171f6d9b-ebb3-4079-a770-ac1c15c2977a HTTP/1.1 Jul 16 21:18:32.765803 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:18:32.765803 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:18:32.765803 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:18:32.765803 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 0 Jul 16 21:18:32.765803 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:18:32.765803 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:18:32.765803 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:18:32.765803 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:18:32.765803 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0yrk-HsfYKsyovLSaEtNxaoXy7a3rLGNlsRPiPP76ZvvrKkKyGd-uekosXQxpFgkRnItBdviN4xr49Rerg5AyvLSzsyRY-Um0r3BRAvyZrzmCDYUpWbXk1uyEIb3Af4U3Y3nXIJgJAuDVx9z2NGSvEGCsGz12WlyyGCNfVSADFcYJP8 Jul 16 21:18:32.765803 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:18:32.765803 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:18:32.765803 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:18:32.765803 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:18:32.765803 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:18:32.765803 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:18:32.765803 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: e353258f9c3048ef84683294e2b8579d Jul 16 21:18:32.765803 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-SecurityGroupsTestJSON-438340169 Jul 16 21:18:32.765803 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:18:32.765803 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:18:32.765803 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/e353258f9c3048ef84683294e2b8579d"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/e353258f9c3048ef84683294e2b8579d"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_e353258f9c3048ef84683294e2b8579d"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/e353258f9c3048ef84683294e2b8579d"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:18:32.765803 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-SecurityGroupsTestJSON-438340169 Jul 16 21:18:32.765803 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: e353258f9c3048ef84683294e2b8579d Jul 16 21:18:32.765803 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-SecurityGroupsTestJSON-438340169 Jul 16 21:18:32.765803 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-SecurityGroupsTestJSON-438340169-project Jul 16 21:18:32.765803 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:18:32.765803 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:18:32.765803 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: d0008fffad8e4708b2c01e03da92c47d Jul 16 21:18:32.765803 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-SecurityGroupsTestJSON-438340169-project Jul 16 21:18:32.765803 ncdv-master-1 devstack@n-api.service[138680]: method: Jul 16 21:18:32.765803 ncdv-master-1 devstack@n-api.service[138680]: exception: Security group 171f6d9b-ebb3-4079-a770-ac1c15c2977a does not exist Jul 16 21:18:32.765803 ncdv-master-1 devstack@n-api.service[138680]: Neutron server returns request_ids: ['req-ebf8f7a6-cc0e-45ee-bbde-48861021b998'] Jul 16 21:18:32.765803 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': '171f6d9b-ebb3-4079-a770-ac1c15c2977a'} Jul 16 21:18:32.765803 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:18:32.765803 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:18:32.765803 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/security_group_api.py", line 309, in get Jul 16 21:18:32.765803 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi group = neutron.show_security_group(id).get('security_group') Jul 16 21:18:32.765803 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 197, in wrapper Jul 16 21:18:32.765803 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Jul 16 21:18:32.765803 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 1040, in show_security_group Jul 16 21:18:32.765803 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return self.get(self.security_group_path % (security_group), Jul 16 21:18:32.765803 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 197, in wrapper Jul 16 21:18:32.765803 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Jul 16 21:18:32.765803 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 356, in get Jul 16 21:18:32.765803 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return self.retry_request("GET", action, body=body, Jul 16 21:18:32.765803 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 197, in wrapper Jul 16 21:18:32.765803 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Jul 16 21:18:32.765803 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 333, in retry_request Jul 16 21:18:32.765803 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return self.do_request(method, action, body=body, Jul 16 21:18:32.765803 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 197, in wrapper Jul 16 21:18:32.765803 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Jul 16 21:18:32.765803 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 297, in do_request Jul 16 21:18:32.765803 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi self._handle_fault_response(status_code, replybody, resp) Jul 16 21:18:32.765803 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 197, in wrapper Jul 16 21:18:32.765803 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Jul 16 21:18:32.765803 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 272, in _handle_fault_response Jul 16 21:18:32.765803 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi exception_handler_v20(status_code, error_body) Jul 16 21:18:32.765803 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 90, in exception_handler_v20 Jul 16 21:18:32.765803 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise client_exc(message=error_message, Jul 16 21:18:32.765803 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi neutronclient.common.exceptions.NotFound: Security group 171f6d9b-ebb3-4079-a770-ac1c15c2977a does not exist Jul 16 21:18:32.776558 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-ebf8f7a6-cc0e-45ee-bbde-48861021b998'] Jul 16 21:18:32.776558 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:18:32.776558 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:18:32.776558 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:18:32.776558 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:18:32.776558 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/security_groups.py", line 158, in show Jul 16 21:18:32.776558 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi security_group = security_group_api.get(context, id) Jul 16 21:18:32.776558 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/security_group_api.py", line 314, in get Jul 16 21:18:32.776558 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.SecurityGroupNotFound(str(e)) Jul 16 21:18:32.776558 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.SecurityGroupNotFound: Security group 171f6d9b-ebb3-4079-a770-ac1c15c2977a does not exist Jul 16 21:18:32.776558 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-ebf8f7a6-cc0e-45ee-bbde-48861021b998'] Jul 16 21:18:32.776558 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:18:32.776558 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:18:32.776558 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:18:32.776558 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:18:32.776558 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:18:32.776558 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:18:32.776558 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:18:32.776558 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:18:32.776558 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Jul 16 21:18:32.776558 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Jul 16 21:18:32.776558 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:18:32.776558 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:18:32.776558 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/security_groups.py", line 160, in show Jul 16 21:18:32.776558 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=exp.format_message()) Jul 16 21:18:32.776558 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Security group 171f6d9b-ebb3-4079-a770-ac1c15c2977a does not exist Jul 16 21:18:32.776558 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-ebf8f7a6-cc0e-45ee-bbde-48861021b998'] Jul 16 21:18:32.776558 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:18:32.776558 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:18:32.776558 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:18:32.776558 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:18:32.776558 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:18:32.776558 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:18:32.776558 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:18:32.776558 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:18:32.776558 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Security group 171f6d9b-ebb3-4079-a770-ac1c15c2977a does not exist Jul 16 21:18:32.776558 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-ebf8f7a6-cc0e-45ee-bbde-48861021b998'] Jul 16 21:18:32.776558 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:18:32.776558 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-8758c677-f7ac-4ba3-9c48-5db5d7414ee6 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Returning 404 to user: Security group 171f6d9b-ebb3-4079-a770-ac1c15c2977a does not exist Jul 16 21:18:32.776558 ncdv-master-1 devstack@n-api.service[138680]: Neutron server returns request_ids: ['req-ebf8f7a6-cc0e-45ee-bbde-48861021b998'] {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:18:32.776558 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-8758c677-f7ac-4ba3-9c48-5db5d7414ee6 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] 10.222.0.16 "GET /compute/v2.1/os-security-groups/171f6d9b-ebb3-4079-a770-ac1c15c2977a" status: 404 len: 194 microversion: 2.1 time: 0.062606 Jul 16 21:18:32.776558 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1225/2370] 10.222.0.16 () {58 vars in 1268 bytes} [Sat Jul 16 21:18:32 2022] GET /compute/v2.1/os-security-groups/171f6d9b-ebb3-4079-a770-ac1c15c2977a => generated 194 bytes in 64 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:18:32.782955 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-cc079506-eb03-4db9-97ba-96f278901375 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:32.787381 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-cc079506-eb03-4db9-97ba-96f278901375 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] [instance: b9a4910a-f117-4834-b699-498c6431b9bb] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:18:32.798894 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cc079506-eb03-4db9-97ba-96f278901375 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:32.799387 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cc079506-eb03-4db9-97ba-96f278901375 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:32.800690 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cc079506-eb03-4db9-97ba-96f278901375 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:32.800690 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cc079506-eb03-4db9-97ba-96f278901375 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:32.817946 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-727170f3-049f-48c5-997c-c25fe14afed0 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd4144b5388b46a989a41a2f556bb508', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a90d4bcd25ff4bf68bdc194180fbbe09', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:18:32.834252 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-727170f3-049f-48c5-997c-c25fe14afed0 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] 10.222.0.16 "GET /compute/v2.1/servers/6baac85b-f81c-4e3a-9f96-633cffccfd74" status: 200 len: 1507 microversion: 2.1 time: 0.237588 Jul 16 21:18:32.835321 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1146/2371] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:18:32 2022] GET /compute/v2.1/servers/6baac85b-f81c-4e3a-9f96-633cffccfd74 => generated 1507 bytes in 239 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:18:32.848037 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-ef5fac33-fb66-412f-833e-92a57f3371be tempest-ListImagesTestJSON-945500242 tempest-ListImagesTestJSON-945500242-project] Calling method '.version_select at 0x7fdd9d2678b0>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:32.950872 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-ef5fac33-fb66-412f-833e-92a57f3371be tempest-ListImagesTestJSON-945500242 tempest-ListImagesTestJSON-945500242-project] 10.222.0.16 "GET /compute/v2.1/images" status: 200 len: 445 microversion: 2.1 time: 0.114016 Jul 16 21:18:32.951609 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1147/2372] 10.222.0.16 () {58 vars in 1121 bytes} [Sat Jul 16 21:18:32 2022] GET /compute/v2.1/images => generated 445 bytes in 116 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:18:32.957474 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-3d83f479-4646-4277-af5e-822b5863a113 tempest-ServersAdminTestJSON-1511367024 tempest-ServersAdminTestJSON-1511367024-project-admin] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:32.962149 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-3d83f479-4646-4277-af5e-822b5863a113 tempest-ServersAdminTestJSON-1511367024 tempest-ServersAdminTestJSON-1511367024-project-admin] Searching by: {'host': 'nonexistent_host', 'deleted': False} {{(pid=138681) get_all /opt/stack/nova/nova/compute/api.py:2962}} Jul 16 21:18:32.972204 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3d83f479-4646-4277-af5e-822b5863a113 tempest-ServersAdminTestJSON-1511367024 tempest-ServersAdminTestJSON-1511367024-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:32.972940 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3d83f479-4646-4277-af5e-822b5863a113 tempest-ServersAdminTestJSON-1511367024 tempest-ServersAdminTestJSON-1511367024-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:32.973857 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3d83f479-4646-4277-af5e-822b5863a113 tempest-ServersAdminTestJSON-1511367024 tempest-ServersAdminTestJSON-1511367024-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:32.974357 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3d83f479-4646-4277-af5e-822b5863a113 tempest-ServersAdminTestJSON-1511367024 tempest-ServersAdminTestJSON-1511367024-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:32.982351 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-cc079506-eb03-4db9-97ba-96f278901375 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '71e9304fc50447d6aac99703b5b6e4e1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e84496d78d4481e9d7648128868bca2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:18:32.990845 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-cc079506-eb03-4db9-97ba-96f278901375 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] 10.222.0.16 "GET /compute/v2.1/servers/b9a4910a-f117-4834-b699-498c6431b9bb" status: 200 len: 1466 microversion: 2.1 time: 0.221923 Jul 16 21:18:32.998903 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-3d83f479-4646-4277-af5e-822b5863a113 tempest-ServersAdminTestJSON-1511367024 tempest-ServersAdminTestJSON-1511367024-project-admin] 10.222.0.16 "GET /compute/v2.1/servers?host=nonexistent_host&all_tenants=1" status: 200 len: 15 microversion: 2.1 time: 0.045916 Jul 16 21:18:32.999547 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1148/2373] 10.222.0.16 () {58 vars in 1195 bytes} [Sat Jul 16 21:18:32 2022] GET /compute/v2.1/servers?host=nonexistent_host&all_tenants=1 => generated 15 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Jul 16 21:18:33.003244 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1226/2374] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:18:32 2022] GET /compute/v2.1/servers/b9a4910a-f117-4834-b699-498c6431b9bb => generated 1466 bytes in 235 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:18:33.011241 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-3aadb3bb-bbcc-40ba-afd4-0ee2adb8647c tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Calling method '.version_select at 0x7fdd9dbfc940>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:33.011839 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-2fd5e11b-c5eb-4ca9-8afb-4006a513af43 tempest-ServersNegativeTestMultiTenantJSON-1135003248 tempest-ServersNegativeTestMultiTenantJSON-1135003248-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:33.012487 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-2fd5e11b-c5eb-4ca9-8afb-4006a513af43 tempest-ServersNegativeTestMultiTenantJSON-1135003248 tempest-ServersNegativeTestMultiTenantJSON-1135003248-project] [instance: 43169d4f-1214-419a-aa1e-def65182ed5a] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:18:33.020425 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2fd5e11b-c5eb-4ca9-8afb-4006a513af43 tempest-ServersNegativeTestMultiTenantJSON-1135003248 tempest-ServersNegativeTestMultiTenantJSON-1135003248-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:33.026613 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2fd5e11b-c5eb-4ca9-8afb-4006a513af43 tempest-ServersNegativeTestMultiTenantJSON-1135003248 tempest-ServersNegativeTestMultiTenantJSON-1135003248-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:33.026613 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2fd5e11b-c5eb-4ca9-8afb-4006a513af43 tempest-ServersNegativeTestMultiTenantJSON-1135003248 tempest-ServersNegativeTestMultiTenantJSON-1135003248-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:33.026613 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2fd5e11b-c5eb-4ca9-8afb-4006a513af43 tempest-ServersNegativeTestMultiTenantJSON-1135003248 tempest-ServersNegativeTestMultiTenantJSON-1135003248-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:33.071087 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-3aadb3bb-bbcc-40ba-afd4-0ee2adb8647c tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] 10.222.0.16 "GET /compute/v2.1/images/eee6448e-8c8a-42ef-b4df-d19c5d9ae531" status: 200 len: 1439 microversion: 2.1 time: 0.062305 Jul 16 21:18:33.071087 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1149/2375] 10.222.0.16 () {58 vars in 1232 bytes} [Sat Jul 16 21:18:32 2022] GET /compute/v2.1/images/eee6448e-8c8a-42ef-b4df-d19c5d9ae531 => generated 1439 bytes in 64 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:18:33.071087 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-0e9566c7-c5a3-4fcf-be15-f36072b9ce99 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Calling method '.version_select at 0x7fdd9d087dc0>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:33.217364 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-2fd5e11b-c5eb-4ca9-8afb-4006a513af43 tempest-ServersNegativeTestMultiTenantJSON-1135003248 tempest-ServersNegativeTestMultiTenantJSON-1135003248-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '50cdc134e712487196f6623c5cf14592', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ba229aaef0ec4ddcadd1d940376c31ac', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:18:33.225892 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-2fd5e11b-c5eb-4ca9-8afb-4006a513af43 tempest-ServersNegativeTestMultiTenantJSON-1135003248 tempest-ServersNegativeTestMultiTenantJSON-1135003248-project] 10.222.0.16 "GET /compute/v2.1/servers/43169d4f-1214-419a-aa1e-def65182ed5a" status: 200 len: 1533 microversion: 2.1 time: 0.221119 Jul 16 21:18:33.226496 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1227/2376] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:18:33 2022] GET /compute/v2.1/servers/43169d4f-1214-419a-aa1e-def65182ed5a => generated 1533 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:18:33.231819 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-7c7fcd52-7ebf-4a34-8fce-3678705094b0 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Action: 'update', calling method: >, body: {"server": {"OS-DCF:diskConfig": "AUTO"}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:18:33.233446 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-7c7fcd52-7ebf-4a34-8fce-3678705094b0 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] [instance: 6baac85b-f81c-4e3a-9f96-633cffccfd74] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:18:33.241323 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7c7fcd52-7ebf-4a34-8fce-3678705094b0 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:33.241785 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7c7fcd52-7ebf-4a34-8fce-3678705094b0 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:33.242785 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7c7fcd52-7ebf-4a34-8fce-3678705094b0 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:33.243498 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7c7fcd52-7ebf-4a34-8fce-3678705094b0 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:33.799089 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-7c7fcd52-7ebf-4a34-8fce-3678705094b0 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] 10.222.0.16 "PUT /compute/v2.1/servers/6baac85b-f81c-4e3a-9f96-633cffccfd74" status: 200 len: 1079 microversion: 2.1 time: 0.571238 Jul 16 21:18:33.799749 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1228/2377] 10.222.0.16 () {60 vars in 1255 bytes} [Sat Jul 16 21:18:33 2022] PUT /compute/v2.1/servers/6baac85b-f81c-4e3a-9f96-633cffccfd74 => generated 1079 bytes in 573 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:18:33.804371 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-87d66ccf-cf39-4db7-9033-5a65fd3bc343 tempest-ListImagesTestJSON-945500242 tempest-ListImagesTestJSON-945500242-project] Calling method '.version_select at 0x7fdda29fbe50>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:33.828836 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-0e9566c7-c5a3-4fcf-be15-f36072b9ce99 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] 10.222.0.16 "DELETE /compute/v2.1/os-security-groups/0dec7f11-e8ff-4171-a3ac-2aef7047a4b6" status: 202 len: 0 microversion: 2.1 time: 0.756971 Jul 16 21:18:33.828836 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1150/2378] 10.222.0.16 () {58 vars in 1271 bytes} [Sat Jul 16 21:18:33 2022] DELETE /compute/v2.1/os-security-groups/0dec7f11-e8ff-4171-a3ac-2aef7047a4b6 => generated 0 bytes in 761 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 21:18:33.830458 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-c9905300-a81d-4737-b9f4-631b15ab4a64 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Calling method '.version_select at 0x7fdda2a68e50>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:33.924607 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-87d66ccf-cf39-4db7-9033-5a65fd3bc343 tempest-ListImagesTestJSON-945500242 tempest-ListImagesTestJSON-945500242-project] 10.222.0.16 "GET /compute/v2.1/images/detail" status: 200 len: 787 microversion: 2.1 time: 0.123044 Jul 16 21:18:33.925839 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1229/2379] 10.222.0.16 () {58 vars in 1142 bytes} [Sat Jul 16 21:18:33 2022] GET /compute/v2.1/images/detail => generated 787 bytes in 125 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:18:33.933789 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-c1af4b96-2318-4a8d-8108-76c7c3ab71b3 tempest-ServersAdminTestJSON-1511367024 tempest-ServersAdminTestJSON-1511367024-project-admin] Action: 'action', calling method: >, body: {"os-resetState": {"state": "error"}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:18:33.935963 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-c1af4b96-2318-4a8d-8108-76c7c3ab71b3 tempest-ServersAdminTestJSON-1511367024 tempest-ServersAdminTestJSON-1511367024-project-admin] [instance: e7598b4a-f70c-4186-88d1-6fd0057f8656] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:18:33.945372 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c1af4b96-2318-4a8d-8108-76c7c3ab71b3 tempest-ServersAdminTestJSON-1511367024 tempest-ServersAdminTestJSON-1511367024-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:33.946146 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c1af4b96-2318-4a8d-8108-76c7c3ab71b3 tempest-ServersAdminTestJSON-1511367024 tempest-ServersAdminTestJSON-1511367024-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:33.947541 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c1af4b96-2318-4a8d-8108-76c7c3ab71b3 tempest-ServersAdminTestJSON-1511367024 tempest-ServersAdminTestJSON-1511367024-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:33.948835 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c1af4b96-2318-4a8d-8108-76c7c3ab71b3 tempest-ServersAdminTestJSON-1511367024 tempest-ServersAdminTestJSON-1511367024-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:33.958680 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-c9905300-a81d-4737-b9f4-631b15ab4a64 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] 10.222.0.16 "GET /compute/v2.1/images?changes-since=2022-07-16T21%3A16%3A46Z" status: 200 len: 1902 microversion: 2.1 time: 0.132114 Jul 16 21:18:33.967393 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1151/2380] 10.222.0.16 () {58 vars in 1198 bytes} [Sat Jul 16 21:18:33 2022] GET /compute/v2.1/images?changes-since=2022-07-16T21%3A16%3A46Z => generated 1902 bytes in 141 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:18:33.972593 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-2c6368dd-013b-48a4-8579-52f4812eb9f6 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:33.973579 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-2c6368dd-013b-48a4-8579-52f4812eb9f6 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] [instance: 6baac85b-f81c-4e3a-9f96-633cffccfd74] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:18:33.984925 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2c6368dd-013b-48a4-8579-52f4812eb9f6 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:33.985652 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2c6368dd-013b-48a4-8579-52f4812eb9f6 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:33.986601 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2c6368dd-013b-48a4-8579-52f4812eb9f6 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:33.987233 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2c6368dd-013b-48a4-8579-52f4812eb9f6 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:34.162249 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-2c6368dd-013b-48a4-8579-52f4812eb9f6 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd4144b5388b46a989a41a2f556bb508', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a90d4bcd25ff4bf68bdc194180fbbe09', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:18:34.171564 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-2c6368dd-013b-48a4-8579-52f4812eb9f6 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] 10.222.0.16 "GET /compute/v2.1/servers/6baac85b-f81c-4e3a-9f96-633cffccfd74" status: 200 len: 1505 microversion: 2.1 time: 0.202716 Jul 16 21:18:34.175264 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1152/2381] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:18:33 2022] GET /compute/v2.1/servers/6baac85b-f81c-4e3a-9f96-633cffccfd74 => generated 1505 bytes in 204 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:18:34.180934 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-0dad8d28-6a2e-41ff-9a05-6eb6ef500080 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Calling method '.version_select at 0x7fdd9d7a3940>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:34.256984 ncdv-master-1 devstack@n-api.service[138681]: DEBUG neutronclient.v2_0.client [None req-0dad8d28-6a2e-41ff-9a05-6eb6ef500080 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Error message: {"NeutronError": {"type": "SecurityGroupNotFound", "message": "Security group 0dec7f11-e8ff-4171-a3ac-2aef7047a4b6 does not exist", "detail": ""}} {{(pid=138681) _handle_fault_response /usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py:262}} Jul 16 21:18:34.257706 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.network.security_group_api [None req-0dad8d28-6a2e-41ff-9a05-6eb6ef500080 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Neutron security group 0dec7f11-e8ff-4171-a3ac-2aef7047a4b6 not found {{(pid=138681) get /opt/stack/nova/nova/network/security_group_api.py:313}} Jul 16 21:18:34.258416 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-0dad8d28-6a2e-41ff-9a05-6eb6ef500080 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] HTTP exception thrown: Security group 0dec7f11-e8ff-4171-a3ac-2aef7047a4b6 does not exist Jul 16 21:18:34.258416 ncdv-master-1 devstack@n-api.service[138681]: Neutron server returns request_ids: ['req-e5803829-d1b9-4d77-b4da-b33761e0e4b2'] Jul 16 21:18:34.261339 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-0dad8d28-6a2e-41ff-9a05-6eb6ef500080 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Request method failure captured: Jul 16 21:18:34.261339 ncdv-master-1 devstack@n-api.service[138681]: request: GET /compute/v2.1/os-security-groups/0dec7f11-e8ff-4171-a3ac-2aef7047a4b6 HTTP/1.1 Jul 16 21:18:34.261339 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:18:34.261339 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:18:34.261339 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:18:34.261339 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 21:18:34.261339 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:18:34.261339 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:18:34.261339 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:18:34.261339 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:18:34.261339 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0yrk-HsfYKsyovLSaEtNxaoXy7a3rLGNlsRPiPP76ZvvrKkKyGd-uekosXQxpFgkRnItBdviN4xr49Rerg5AyvLSzsyRY-Um0r3BRAvyZrzmCDYUpWbXk1uyEIb3Af4U3Y3nXIJgJAuDVx9z2NGSvEGCsGz12WlyyGCNfVSADFcYJP8 Jul 16 21:18:34.261339 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:18:34.261339 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:18:34.261339 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:18:34.261339 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:18:34.261339 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:18:34.261339 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:18:34.261339 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: e353258f9c3048ef84683294e2b8579d Jul 16 21:18:34.261339 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-SecurityGroupsTestJSON-438340169 Jul 16 21:18:34.261339 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:18:34.261339 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:18:34.261339 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/e353258f9c3048ef84683294e2b8579d"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/e353258f9c3048ef84683294e2b8579d"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_e353258f9c3048ef84683294e2b8579d"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/e353258f9c3048ef84683294e2b8579d"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:18:34.261339 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-SecurityGroupsTestJSON-438340169 Jul 16 21:18:34.261339 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: e353258f9c3048ef84683294e2b8579d Jul 16 21:18:34.261339 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-SecurityGroupsTestJSON-438340169 Jul 16 21:18:34.261339 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-SecurityGroupsTestJSON-438340169-project Jul 16 21:18:34.261339 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:18:34.261339 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:18:34.261339 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: d0008fffad8e4708b2c01e03da92c47d Jul 16 21:18:34.261339 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-SecurityGroupsTestJSON-438340169-project Jul 16 21:18:34.261339 ncdv-master-1 devstack@n-api.service[138681]: method: Jul 16 21:18:34.261339 ncdv-master-1 devstack@n-api.service[138681]: exception: Security group 0dec7f11-e8ff-4171-a3ac-2aef7047a4b6 does not exist Jul 16 21:18:34.261339 ncdv-master-1 devstack@n-api.service[138681]: Neutron server returns request_ids: ['req-e5803829-d1b9-4d77-b4da-b33761e0e4b2'] Jul 16 21:18:34.261339 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': '0dec7f11-e8ff-4171-a3ac-2aef7047a4b6'} Jul 16 21:18:34.261339 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:18:34.261339 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:18:34.261339 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/security_group_api.py", line 309, in get Jul 16 21:18:34.261339 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi group = neutron.show_security_group(id).get('security_group') Jul 16 21:18:34.261339 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 197, in wrapper Jul 16 21:18:34.261339 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Jul 16 21:18:34.261339 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 1040, in show_security_group Jul 16 21:18:34.261339 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return self.get(self.security_group_path % (security_group), Jul 16 21:18:34.261339 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 197, in wrapper Jul 16 21:18:34.261339 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Jul 16 21:18:34.261339 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 356, in get Jul 16 21:18:34.261339 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return self.retry_request("GET", action, body=body, Jul 16 21:18:34.261339 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 197, in wrapper Jul 16 21:18:34.261339 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Jul 16 21:18:34.261339 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 333, in retry_request Jul 16 21:18:34.261339 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return self.do_request(method, action, body=body, Jul 16 21:18:34.261339 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 197, in wrapper Jul 16 21:18:34.261339 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Jul 16 21:18:34.261339 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 297, in do_request Jul 16 21:18:34.261339 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi self._handle_fault_response(status_code, replybody, resp) Jul 16 21:18:34.261339 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 197, in wrapper Jul 16 21:18:34.261339 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Jul 16 21:18:34.261339 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 272, in _handle_fault_response Jul 16 21:18:34.261339 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi exception_handler_v20(status_code, error_body) Jul 16 21:18:34.261339 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 90, in exception_handler_v20 Jul 16 21:18:34.261339 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise client_exc(message=error_message, Jul 16 21:18:34.261339 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi neutronclient.common.exceptions.NotFound: Security group 0dec7f11-e8ff-4171-a3ac-2aef7047a4b6 does not exist Jul 16 21:18:34.269238 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-e5803829-d1b9-4d77-b4da-b33761e0e4b2'] Jul 16 21:18:34.269238 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:18:34.269238 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:18:34.269238 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:18:34.269238 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:18:34.269238 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/security_groups.py", line 158, in show Jul 16 21:18:34.269238 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi security_group = security_group_api.get(context, id) Jul 16 21:18:34.269238 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/security_group_api.py", line 314, in get Jul 16 21:18:34.269238 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.SecurityGroupNotFound(str(e)) Jul 16 21:18:34.269238 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.SecurityGroupNotFound: Security group 0dec7f11-e8ff-4171-a3ac-2aef7047a4b6 does not exist Jul 16 21:18:34.269238 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-e5803829-d1b9-4d77-b4da-b33761e0e4b2'] Jul 16 21:18:34.269238 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:18:34.269238 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:18:34.269238 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:18:34.269238 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:18:34.269238 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:18:34.269238 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:18:34.269238 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:18:34.269238 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:18:34.269238 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Jul 16 21:18:34.269238 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Jul 16 21:18:34.269238 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:18:34.269238 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:18:34.269238 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/security_groups.py", line 160, in show Jul 16 21:18:34.269238 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=exp.format_message()) Jul 16 21:18:34.269238 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Security group 0dec7f11-e8ff-4171-a3ac-2aef7047a4b6 does not exist Jul 16 21:18:34.269238 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-e5803829-d1b9-4d77-b4da-b33761e0e4b2'] Jul 16 21:18:34.269238 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:18:34.269238 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:18:34.269238 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:18:34.269238 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:18:34.269238 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:18:34.269238 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:18:34.269238 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:18:34.269238 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:18:34.269238 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Security group 0dec7f11-e8ff-4171-a3ac-2aef7047a4b6 does not exist Jul 16 21:18:34.269238 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-e5803829-d1b9-4d77-b4da-b33761e0e4b2'] Jul 16 21:18:34.269238 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:18:34.269238 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-0dad8d28-6a2e-41ff-9a05-6eb6ef500080 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Returning 404 to user: Security group 0dec7f11-e8ff-4171-a3ac-2aef7047a4b6 does not exist Jul 16 21:18:34.269238 ncdv-master-1 devstack@n-api.service[138681]: Neutron server returns request_ids: ['req-e5803829-d1b9-4d77-b4da-b33761e0e4b2'] {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:18:34.269238 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-0dad8d28-6a2e-41ff-9a05-6eb6ef500080 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] 10.222.0.16 "GET /compute/v2.1/os-security-groups/0dec7f11-e8ff-4171-a3ac-2aef7047a4b6" status: 404 len: 194 microversion: 2.1 time: 0.091234 Jul 16 21:18:34.269238 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1153/2382] 10.222.0.16 () {58 vars in 1268 bytes} [Sat Jul 16 21:18:34 2022] GET /compute/v2.1/os-security-groups/0dec7f11-e8ff-4171-a3ac-2aef7047a4b6 => generated 194 bytes in 96 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:18:34.273579 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-bc1a586f-7b33-4749-8fb5-d121da422a74 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Calling method '.version_select at 0x7fdd9d0870d0>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:34.349538 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-c1af4b96-2318-4a8d-8108-76c7c3ab71b3 tempest-ServersAdminTestJSON-1511367024 tempest-ServersAdminTestJSON-1511367024-project-admin] 10.222.0.16 "POST /compute/v2.1/servers/e7598b4a-f70c-4186-88d1-6fd0057f8656/action" status: 202 len: 0 microversion: 2.1 time: 0.420716 Jul 16 21:18:34.349538 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1230/2383] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 21:18:33 2022] POST /compute/v2.1/servers/e7598b4a-f70c-4186-88d1-6fd0057f8656/action => generated 0 bytes in 422 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 21:18:34.353215 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-65f39818-cc8d-465f-9638-e7d15e21e711 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Calling method '.version_select at 0x7fdda2a5d3a0>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:34.368486 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-bc1a586f-7b33-4749-8fb5-d121da422a74 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] 10.222.0.16 "GET /compute/v2.1/images?name=tempest-ListImageFiltersTestJSON-image-1673862976" status: 200 len: 485 microversion: 2.1 time: 0.098508 Jul 16 21:18:34.369094 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1154/2384] 10.222.0.16 () {58 vars in 1230 bytes} [Sat Jul 16 21:18:34 2022] GET /compute/v2.1/images?name=tempest-ListImageFiltersTestJSON-image-1673862976 => generated 485 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:18:34.373690 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-142d24e4-51be-484e-92db-9c8c1430337b tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:34.379381 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-142d24e4-51be-484e-92db-9c8c1430337b tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] [instance: 6baac85b-f81c-4e3a-9f96-633cffccfd74] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:18:34.389055 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-142d24e4-51be-484e-92db-9c8c1430337b tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:34.392580 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-142d24e4-51be-484e-92db-9c8c1430337b tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:34.394020 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-142d24e4-51be-484e-92db-9c8c1430337b tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:34.394824 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-142d24e4-51be-484e-92db-9c8c1430337b tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:34.419305 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-65f39818-cc8d-465f-9638-e7d15e21e711 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] 10.222.0.16 "GET /compute/v2.1/images/eee6448e-8c8a-42ef-b4df-d19c5d9ae531" status: 200 len: 1439 microversion: 2.1 time: 0.069516 Jul 16 21:18:34.424851 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1231/2385] 10.222.0.16 () {58 vars in 1232 bytes} [Sat Jul 16 21:18:34 2022] GET /compute/v2.1/images/eee6448e-8c8a-42ef-b4df-d19c5d9ae531 => generated 1439 bytes in 71 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:18:34.425139 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-e603ce62-8168-485b-aa5f-1b76e792bb95 tempest-ServersNegativeTestMultiTenantJSON-1135003248 tempest-ServersNegativeTestMultiTenantJSON-1135003248-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:34.425663 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-e603ce62-8168-485b-aa5f-1b76e792bb95 tempest-ServersNegativeTestMultiTenantJSON-1135003248 tempest-ServersNegativeTestMultiTenantJSON-1135003248-project] [instance: 43169d4f-1214-419a-aa1e-def65182ed5a] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:18:34.433715 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e603ce62-8168-485b-aa5f-1b76e792bb95 tempest-ServersNegativeTestMultiTenantJSON-1135003248 tempest-ServersNegativeTestMultiTenantJSON-1135003248-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:34.435823 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e603ce62-8168-485b-aa5f-1b76e792bb95 tempest-ServersNegativeTestMultiTenantJSON-1135003248 tempest-ServersNegativeTestMultiTenantJSON-1135003248-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:34.435823 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e603ce62-8168-485b-aa5f-1b76e792bb95 tempest-ServersNegativeTestMultiTenantJSON-1135003248 tempest-ServersNegativeTestMultiTenantJSON-1135003248-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:34.435823 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e603ce62-8168-485b-aa5f-1b76e792bb95 tempest-ServersNegativeTestMultiTenantJSON-1135003248 tempest-ServersNegativeTestMultiTenantJSON-1135003248-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:34.463341 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=97,nova_cell1:INSERT=1,nova_cell1:UPDATE=2 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:18:34.778418 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-142d24e4-51be-484e-92db-9c8c1430337b tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd4144b5388b46a989a41a2f556bb508', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a90d4bcd25ff4bf68bdc194180fbbe09', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:18:34.779408 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-e603ce62-8168-485b-aa5f-1b76e792bb95 tempest-ServersNegativeTestMultiTenantJSON-1135003248 tempest-ServersNegativeTestMultiTenantJSON-1135003248-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '50cdc134e712487196f6623c5cf14592', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ba229aaef0ec4ddcadd1d940376c31ac', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:18:34.795255 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=91,nova_cell1:INSERT=3,nova_cell1:UPDATE=6 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:18:34.803449 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-142d24e4-51be-484e-92db-9c8c1430337b tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] 10.222.0.16 "GET /compute/v2.1/servers/6baac85b-f81c-4e3a-9f96-633cffccfd74" status: 200 len: 1505 microversion: 2.1 time: 0.431960 Jul 16 21:18:34.805517 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-e603ce62-8168-485b-aa5f-1b76e792bb95 tempest-ServersNegativeTestMultiTenantJSON-1135003248 tempest-ServersNegativeTestMultiTenantJSON-1135003248-project] 10.222.0.16 "GET /compute/v2.1/servers/43169d4f-1214-419a-aa1e-def65182ed5a" status: 200 len: 1533 microversion: 2.1 time: 0.383833 Jul 16 21:18:34.817074 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1155/2386] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:18:34 2022] GET /compute/v2.1/servers/6baac85b-f81c-4e3a-9f96-633cffccfd74 => generated 1505 bytes in 438 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:18:34.819630 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1232/2387] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:18:34 2022] GET /compute/v2.1/servers/43169d4f-1214-419a-aa1e-def65182ed5a => generated 1533 bytes in 399 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:18:34.825178 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-bf0a0658-28ca-48db-a48d-542cb643723d tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Action: 'action', calling method: >, body: {"rebuild": {"imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9"}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:18:34.826684 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-bf0a0658-28ca-48db-a48d-542cb643723d tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] [instance: e7598b4a-f70c-4186-88d1-6fd0057f8656] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:18:34.830760 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-898df0c1-f08e-4694-a5bc-2b10c8da7682 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Calling method '.version_select at 0x7fdd9d087700>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:34.837284 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-bf0a0658-28ca-48db-a48d-542cb643723d tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:34.838718 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-bf0a0658-28ca-48db-a48d-542cb643723d tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:34.839686 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-bf0a0658-28ca-48db-a48d-542cb643723d tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:34.840218 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-bf0a0658-28ca-48db-a48d-542cb643723d tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:34.949749 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-bf0a0658-28ca-48db-a48d-542cb643723d tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Flavor limits 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 16 21:18:34.950179 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-bf0a0658-28ca-48db-a48d-542cb643723d tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Image limits 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 16 21:18:34.950664 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-bf0a0658-28ca-48db-a48d-542cb643723d tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Flavor pref 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 16 21:18:34.951112 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-bf0a0658-28ca-48db-a48d-542cb643723d tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Image pref 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 16 21:18:34.951535 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-bf0a0658-28ca-48db-a48d-542cb643723d tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 16 21:18:35.176787 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-898df0c1-f08e-4694-a5bc-2b10c8da7682 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] 10.222.0.16 "GET /compute/v2.1/os-security-groups" status: 200 len: 1159 microversion: 2.1 time: 0.368110 Jul 16 21:18:35.177546 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1156/2388] 10.222.0.16 () {58 vars in 1157 bytes} [Sat Jul 16 21:18:34 2022] GET /compute/v2.1/os-security-groups => generated 1159 bytes in 369 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:18:35.182887 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-bafdce7f-b796-4366-94f3-e65d8455fa7b tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Calling method '.version_select at 0x7fdd9cf903a0>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:35.286446 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-bf0a0658-28ca-48db-a48d-542cb643723d tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] [instance: e7598b4a-f70c-4186-88d1-6fd0057f8656] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:18:35.291072 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-bafdce7f-b796-4366-94f3-e65d8455fa7b tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] 10.222.0.16 "GET /compute/v2.1/images?server=b9a4910a-f117-4834-b699-498c6431b9bb" status: 200 len: 957 microversion: 2.1 time: 0.111902 Jul 16 21:18:35.292270 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1157/2389] 10.222.0.16 () {58 vars in 1208 bytes} [Sat Jul 16 21:18:35 2022] GET /compute/v2.1/images?server=b9a4910a-f117-4834-b699-498c6431b9bb => generated 957 bytes in 114 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:18:35.304146 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-bf0a0658-28ca-48db-a48d-542cb643723d tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:35.304788 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-bf0a0658-28ca-48db-a48d-542cb643723d tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:35.305596 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-bf0a0658-28ca-48db-a48d-542cb643723d tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:35.305921 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-56b5e6ac-7d6e-4cf8-88b4-3de6fc570703 req-ea266a28-cbae-41a4-8984-d8b5fcdb5f80 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "b9a4910a-f117-4834-b699-498c6431b9bb", "name": "network-vif-unplugged", "status": "completed", "tag": "9ab69acd-f0a3-4355-b203-2c48e39cc3a1"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:18:35.306433 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-bf0a0658-28ca-48db-a48d-542cb643723d tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:35.319642 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-56b5e6ac-7d6e-4cf8-88b4-3de6fc570703 req-ea266a28-cbae-41a4-8984-d8b5fcdb5f80 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:35.319642 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-56b5e6ac-7d6e-4cf8-88b4-3de6fc570703 req-ea266a28-cbae-41a4-8984-d8b5fcdb5f80 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:35.328358 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-56b5e6ac-7d6e-4cf8-88b4-3de6fc570703 req-ea266a28-cbae-41a4-8984-d8b5fcdb5f80 service nova] Creating event network-vif-unplugged:9ab69acd-f0a3-4355-b203-2c48e39cc3a1 for instance b9a4910a-f117-4834-b699-498c6431b9bb on nch2-master-1 Jul 16 21:18:35.333714 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-56b5e6ac-7d6e-4cf8-88b4-3de6fc570703 req-ea266a28-cbae-41a4-8984-d8b5fcdb5f80 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.032819 Jul 16 21:18:35.334595 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1158/2390] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:18:35 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:18:35.345859 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-d1d935f7-04ad-4aa1-9617-c9d069f5a304 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Action: 'update', calling method: >, body: {"server": {"OS-DCF:diskConfig": "MANUAL"}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:18:35.347206 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-d1d935f7-04ad-4aa1-9617-c9d069f5a304 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] [instance: 6baac85b-f81c-4e3a-9f96-633cffccfd74] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:18:35.349015 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-bf0a0658-28ca-48db-a48d-542cb643723d tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] 10.222.0.16 "POST /compute/v2.1/servers/e7598b4a-f70c-4186-88d1-6fd0057f8656/action" status: 202 len: 1067 microversion: 2.1 time: 0.527870 Jul 16 21:18:35.359278 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1233/2391] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 21:18:34 2022] POST /compute/v2.1/servers/e7598b4a-f70c-4186-88d1-6fd0057f8656/action => generated 1067 bytes in 530 msecs (HTTP/1.1 202) 10 headers in 452 bytes (1 switches on core 0) Jul 16 21:18:35.359476 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d1d935f7-04ad-4aa1-9617-c9d069f5a304 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:35.359476 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d1d935f7-04ad-4aa1-9617-c9d069f5a304 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:35.359476 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d1d935f7-04ad-4aa1-9617-c9d069f5a304 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:35.359476 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d1d935f7-04ad-4aa1-9617-c9d069f5a304 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:35.487790 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-d1d935f7-04ad-4aa1-9617-c9d069f5a304 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] 10.222.0.16 "PUT /compute/v2.1/servers/6baac85b-f81c-4e3a-9f96-633cffccfd74" status: 200 len: 1081 microversion: 2.1 time: 0.147750 Jul 16 21:18:35.495451 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1159/2392] 10.222.0.16 () {60 vars in 1255 bytes} [Sat Jul 16 21:18:35 2022] PUT /compute/v2.1/servers/6baac85b-f81c-4e3a-9f96-633cffccfd74 => generated 1081 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:18:35.498886 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-c760e718-f68e-4cf5-bfd4-3c32c59cc258 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Action: 'create', calling method: .version_select at 0x7fdd9d221dc0>, body: {"security_group": {"name": "tempest-SecurityGroupsTestJSON-securitygroup-833364650", "description": "tempest-description-258878237"}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:18:35.596770 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-09a39e33-76ea-434f-b578-50e5b43eaacb req-fc6d5351-d010-4227-801c-3b8e32c05cba service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "43169d4f-1214-419a-aa1e-def65182ed5a", "name": "network-vif-deleted", "tag": "17db3ebe-c69b-439f-9a4d-5f322d30e1ad"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:18:35.605135 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-09a39e33-76ea-434f-b578-50e5b43eaacb req-fc6d5351-d010-4227-801c-3b8e32c05cba service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:35.605582 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-09a39e33-76ea-434f-b578-50e5b43eaacb req-fc6d5351-d010-4227-801c-3b8e32c05cba service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:35.617090 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-09a39e33-76ea-434f-b578-50e5b43eaacb req-fc6d5351-d010-4227-801c-3b8e32c05cba service nova] Creating event network-vif-deleted:17db3ebe-c69b-439f-9a4d-5f322d30e1ad for instance 43169d4f-1214-419a-aa1e-def65182ed5a on nch1-master-1 Jul 16 21:18:35.622725 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-09a39e33-76ea-434f-b578-50e5b43eaacb req-fc6d5351-d010-4227-801c-3b8e32c05cba service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.270882 Jul 16 21:18:35.623635 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1234/2393] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:18:35 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 273 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:18:35.628967 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-3f921b43-c8f6-4890-9d56-fe568c83885e tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Calling method '.version_select at 0x7fdd9c6b6160>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:35.680228 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-3f921b43-c8f6-4890-9d56-fe568c83885e tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] 10.222.0.16 "GET /compute/v2.1/images?server=http%3A%2F%2F10.222.0.16%2Fcompute%2Fv2.1%2Fservers%2F6cb4e66a-eba2-42b7-91e8-09ee7b2a5a4f" status: 200 len: 484 microversion: 2.1 time: 0.055365 Jul 16 21:18:35.680955 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1235/2394] 10.222.0.16 () {58 vars in 1316 bytes} [Sat Jul 16 21:18:35 2022] GET /compute/v2.1/images?server=http%3A%2F%2F10.222.0.16%2Fcompute%2Fv2.1%2Fservers%2F6cb4e66a-eba2-42b7-91e8-09ee7b2a5a4f => generated 484 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:18:35.686250 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-b9898bc6-ecfc-4284-8276-ec7f298d821d tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:35.687247 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-b9898bc6-ecfc-4284-8276-ec7f298d821d tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] [instance: e7598b4a-f70c-4186-88d1-6fd0057f8656] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:18:35.694239 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b9898bc6-ecfc-4284-8276-ec7f298d821d tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:35.694702 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b9898bc6-ecfc-4284-8276-ec7f298d821d tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:35.695504 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b9898bc6-ecfc-4284-8276-ec7f298d821d tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:35.695951 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b9898bc6-ecfc-4284-8276-ec7f298d821d tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:35.876129 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-b9898bc6-ecfc-4284-8276-ec7f298d821d tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f5bc1f2209f4eb186d80e385b67c774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '65d5bcfa02af4e8284ba1342197e2860', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:18:35.886581 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-b9898bc6-ecfc-4284-8276-ec7f298d821d tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] 10.222.0.16 "GET /compute/v2.1/servers/e7598b4a-f70c-4186-88d1-6fd0057f8656" status: 200 len: 1471 microversion: 2.1 time: 0.203474 Jul 16 21:18:35.894009 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1236/2395] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:18:35 2022] GET /compute/v2.1/servers/e7598b4a-f70c-4186-88d1-6fd0057f8656 => generated 1471 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:18:35.894009 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-2b46ac0a-dc51-4b18-bc2d-74cce6b6916a tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Calling method '.version_select at 0x7fdda29fbf70>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:35.939546 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-2b46ac0a-dc51-4b18-bc2d-74cce6b6916a tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] 10.222.0.16 "GET /compute/v2.1/images/eee6448e-8c8a-42ef-b4df-d19c5d9ae531" status: 200 len: 1448 microversion: 2.1 time: 0.050402 Jul 16 21:18:35.943900 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1237/2396] 10.222.0.16 () {58 vars in 1232 bytes} [Sat Jul 16 21:18:35 2022] GET /compute/v2.1/images/eee6448e-8c8a-42ef-b4df-d19c5d9ae531 => generated 1448 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:18:35.950092 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-77add7cf-ee02-4e9d-97b9-f3dfc69321a7 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:35.950879 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-77add7cf-ee02-4e9d-97b9-f3dfc69321a7 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] [instance: 6baac85b-f81c-4e3a-9f96-633cffccfd74] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:18:35.961280 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-77add7cf-ee02-4e9d-97b9-f3dfc69321a7 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:35.961853 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-77add7cf-ee02-4e9d-97b9-f3dfc69321a7 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:35.962525 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-77add7cf-ee02-4e9d-97b9-f3dfc69321a7 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:35.963745 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-77add7cf-ee02-4e9d-97b9-f3dfc69321a7 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:36.048018 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-c760e718-f68e-4cf5-bfd4-3c32c59cc258 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] 10.222.0.16 "POST /compute/v2.1/os-security-groups" status: 200 len: 242 microversion: 2.1 time: 0.557363 Jul 16 21:18:36.053757 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1160/2397] 10.222.0.16 () {60 vars in 1179 bytes} [Sat Jul 16 21:18:35 2022] POST /compute/v2.1/os-security-groups => generated 242 bytes in 565 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:18:36.059941 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-57ac353f-54cc-4a0c-90b8-699bb6ef9e88 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Calling method '.version_select at 0x7fdd9d52eca0>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:36.114312 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-57ac353f-54cc-4a0c-90b8-699bb6ef9e88 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] 10.222.0.16 "GET /compute/v2.1/images?server=http%3A%2F%2F10.222.0.16%2Fcompute%2Fservers%2F6cb4e66a-eba2-42b7-91e8-09ee7b2a5a4f" status: 200 len: 484 microversion: 2.1 time: 0.059075 Jul 16 21:18:36.115231 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1161/2398] 10.222.0.16 () {58 vars in 1302 bytes} [Sat Jul 16 21:18:36 2022] GET /compute/v2.1/images?server=http%3A%2F%2F10.222.0.16%2Fcompute%2Fservers%2F6cb4e66a-eba2-42b7-91e8-09ee7b2a5a4f => generated 484 bytes in 61 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:18:36.121299 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-5c75ad7e-21ac-469e-ab57-84745b44eb6b tempest-ServersNegativeTestMultiTenantJSON-1135003248 tempest-ServersNegativeTestMultiTenantJSON-1135003248-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:36.121299 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-5c75ad7e-21ac-469e-ab57-84745b44eb6b tempest-ServersNegativeTestMultiTenantJSON-1135003248 tempest-ServersNegativeTestMultiTenantJSON-1135003248-project] [instance: 43169d4f-1214-419a-aa1e-def65182ed5a] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:18:36.128109 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5c75ad7e-21ac-469e-ab57-84745b44eb6b tempest-ServersNegativeTestMultiTenantJSON-1135003248 tempest-ServersNegativeTestMultiTenantJSON-1135003248-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:36.128970 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5c75ad7e-21ac-469e-ab57-84745b44eb6b tempest-ServersNegativeTestMultiTenantJSON-1135003248 tempest-ServersNegativeTestMultiTenantJSON-1135003248-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:36.129894 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5c75ad7e-21ac-469e-ab57-84745b44eb6b tempest-ServersNegativeTestMultiTenantJSON-1135003248 tempest-ServersNegativeTestMultiTenantJSON-1135003248-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:36.130483 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5c75ad7e-21ac-469e-ab57-84745b44eb6b tempest-ServersNegativeTestMultiTenantJSON-1135003248 tempest-ServersNegativeTestMultiTenantJSON-1135003248-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:36.131747 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-77add7cf-ee02-4e9d-97b9-f3dfc69321a7 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd4144b5388b46a989a41a2f556bb508', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a90d4bcd25ff4bf68bdc194180fbbe09', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:18:36.140580 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-77add7cf-ee02-4e9d-97b9-f3dfc69321a7 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] 10.222.0.16 "GET /compute/v2.1/servers/6baac85b-f81c-4e3a-9f96-633cffccfd74" status: 200 len: 1507 microversion: 2.1 time: 0.199363 Jul 16 21:18:36.141299 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1238/2399] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:18:35 2022] GET /compute/v2.1/servers/6baac85b-f81c-4e3a-9f96-633cffccfd74 => generated 1507 bytes in 201 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:18:36.145890 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-8485bc9e-5e49-45e4-acfb-c9fc551d2ecf tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Calling method '.version_select at 0x7fdda2b1ad30>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:36.169862 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5c75ad7e-21ac-469e-ab57-84745b44eb6b tempest-ServersNegativeTestMultiTenantJSON-1135003248 tempest-ServersNegativeTestMultiTenantJSON-1135003248-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:36.170370 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5c75ad7e-21ac-469e-ab57-84745b44eb6b tempest-ServersNegativeTestMultiTenantJSON-1135003248 tempest-ServersNegativeTestMultiTenantJSON-1135003248-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:36.170953 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.objects.instance [None req-5c75ad7e-21ac-469e-ab57-84745b44eb6b tempest-ServersNegativeTestMultiTenantJSON-1135003248 tempest-ServersNegativeTestMultiTenantJSON-1135003248-project] Lazy-loading 'fault' on Instance uuid 43169d4f-1214-419a-aa1e-def65182ed5a {{(pid=138681) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:18:36.200029 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-8485bc9e-5e49-45e4-acfb-c9fc551d2ecf tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] 10.222.0.16 "GET /compute/v2.1/images/eee6448e-8c8a-42ef-b4df-d19c5d9ae531" status: 200 len: 1448 microversion: 2.1 time: 0.057457 Jul 16 21:18:36.200605 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1239/2400] 10.222.0.16 () {58 vars in 1232 bytes} [Sat Jul 16 21:18:36 2022] GET /compute/v2.1/images/eee6448e-8c8a-42ef-b4df-d19c5d9ae531 => generated 1448 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:18:36.205557 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-12482b92-73a5-47ae-8678-8a8842f37636 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Action: 'create', calling method: .version_select at 0x7fdda2b7f0d0>, body: {"security_group": {"name": "tempest-SecurityGroupsTestJSON-securitygroup-25448692", "description": "tempest-description-298946738"}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:18:36.229957 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-5c75ad7e-21ac-469e-ab57-84745b44eb6b tempest-ServersNegativeTestMultiTenantJSON-1135003248 tempest-ServersNegativeTestMultiTenantJSON-1135003248-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '50cdc134e712487196f6623c5cf14592', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ba229aaef0ec4ddcadd1d940376c31ac', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:18:36.237269 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-5c75ad7e-21ac-469e-ab57-84745b44eb6b tempest-ServersNegativeTestMultiTenantJSON-1135003248 tempest-ServersNegativeTestMultiTenantJSON-1135003248-project] 10.222.0.16 "GET /compute/v2.1/servers/43169d4f-1214-419a-aa1e-def65182ed5a" status: 200 len: 1318 microversion: 2.1 time: 0.121070 Jul 16 21:18:36.238201 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1162/2401] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:18:36 2022] GET /compute/v2.1/servers/43169d4f-1214-419a-aa1e-def65182ed5a => generated 1318 bytes in 123 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:18:36.242852 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-256e50b4-fd65-42c3-8490-0aa8864d555d tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Calling method '.version_select at 0x7fdd9d519160>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:36.357357 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-256e50b4-fd65-42c3-8490-0aa8864d555d tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] 10.222.0.16 "GET /compute/v2.1/images?status=ACTIVE" status: 200 len: 3281 microversion: 2.1 time: 0.117928 Jul 16 21:18:36.357953 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1163/2402] 10.222.0.16 () {58 vars in 1148 bytes} [Sat Jul 16 21:18:36 2022] GET /compute/v2.1/images?status=ACTIVE => generated 3281 bytes in 119 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:18:36.362468 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-7d31aef7-037a-4459-acc0-f76992975720 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:36.363073 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-7d31aef7-037a-4459-acc0-f76992975720 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] [instance: 6baac85b-f81c-4e3a-9f96-633cffccfd74] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:18:36.370080 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7d31aef7-037a-4459-acc0-f76992975720 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:36.373415 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7d31aef7-037a-4459-acc0-f76992975720 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:36.373415 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7d31aef7-037a-4459-acc0-f76992975720 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:36.373415 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7d31aef7-037a-4459-acc0-f76992975720 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:36.558189 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-7d31aef7-037a-4459-acc0-f76992975720 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd4144b5388b46a989a41a2f556bb508', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a90d4bcd25ff4bf68bdc194180fbbe09', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:18:36.567202 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-7d31aef7-037a-4459-acc0-f76992975720 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] 10.222.0.16 "GET /compute/v2.1/servers/6baac85b-f81c-4e3a-9f96-633cffccfd74" status: 200 len: 1507 microversion: 2.1 time: 0.207677 Jul 16 21:18:36.568499 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1164/2403] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:18:36 2022] GET /compute/v2.1/servers/6baac85b-f81c-4e3a-9f96-633cffccfd74 => generated 1507 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:18:36.574163 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-a180a03c-dd0b-4bbb-a395-aa817cf65757 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Calling method '.version_select at 0x7fdd9d519790>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:36.815384 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-12482b92-73a5-47ae-8678-8a8842f37636 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] 10.222.0.16 "POST /compute/v2.1/os-security-groups" status: 200 len: 241 microversion: 2.1 time: 0.613281 Jul 16 21:18:36.820419 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1240/2404] 10.222.0.16 () {60 vars in 1179 bytes} [Sat Jul 16 21:18:36 2022] POST /compute/v2.1/os-security-groups => generated 241 bytes in 615 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:18:36.820712 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-9aa61c2e-7dbb-4e7c-881b-f358c751ea7d tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Calling method '.version_select at 0x7fdda2b90f70>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:36.924290 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-9aa61c2e-7dbb-4e7c-881b-f358c751ea7d tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] 10.222.0.16 "GET /compute/v2.1/images?type=snapshot" status: 200 len: 1429 microversion: 2.1 time: 0.107244 Jul 16 21:18:36.925168 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1241/2405] 10.222.0.16 () {58 vars in 1148 bytes} [Sat Jul 16 21:18:36 2022] GET /compute/v2.1/images?type=snapshot => generated 1429 bytes in 108 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:18:36.929890 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-c97bd43c-8bce-47e4-ab57-269c619d3635 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:36.930397 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-c97bd43c-8bce-47e4-ab57-269c619d3635 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] [instance: 6baac85b-f81c-4e3a-9f96-633cffccfd74] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:18:36.937264 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c97bd43c-8bce-47e4-ab57-269c619d3635 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:36.937720 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c97bd43c-8bce-47e4-ab57-269c619d3635 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:36.938488 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c97bd43c-8bce-47e4-ab57-269c619d3635 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:36.938927 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c97bd43c-8bce-47e4-ab57-269c619d3635 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:36.965750 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-c97bd43c-8bce-47e4-ab57-269c619d3635 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] [instance: 6baac85b-f81c-4e3a-9f96-633cffccfd74] Going to try to terminate instance {{(pid=138680) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 16 21:18:37.333259 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-c97bd43c-8bce-47e4-ab57-269c619d3635 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] 10.222.0.16 "DELETE /compute/v2.1/servers/6baac85b-f81c-4e3a-9f96-633cffccfd74" status: 204 len: 0 microversion: 2.1 time: 0.406466 Jul 16 21:18:37.334452 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1242/2406] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:18:36 2022] DELETE /compute/v2.1/servers/6baac85b-f81c-4e3a-9f96-633cffccfd74 => generated 0 bytes in 409 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 16 21:18:37.459589 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-efb4a587-a7f6-4e2e-b529-2c1ff2f8b645 tempest-KeyPairsV2TestJSON-30838616 tempest-KeyPairsV2TestJSON-30838616-project] Action: 'create', calling method: .version_select at 0x7fdda2aef430>, body: {"keypair": {"name": "tempest-keypair-983547489", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBBv0rhMaVWZKqAaX5qSk44jLi6XJsyWjh72R6gnHxHjtY/NRgvZHsCldRYTTI/Yw8vguopJ82WyRf0vqGhKR8ZUitpqWmRgucLcaBHELeYQIz2N+DbgOIWYhOu1YpfgoBQ=="}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:18:37.474233 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-efb4a587-a7f6-4e2e-b529-2c1ff2f8b645 tempest-KeyPairsV2TestJSON-30838616 tempest-KeyPairsV2TestJSON-30838616-project] Getting quotas for project 205121b2249944eb9b2e1401cf693663. Resources: {'key_pairs'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:18:37.483002 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-efb4a587-a7f6-4e2e-b529-2c1ff2f8b645 tempest-KeyPairsV2TestJSON-30838616 tempest-KeyPairsV2TestJSON-30838616-project] Getting quotas for user e848e24fe4344314a82f574a1bf1283f and project 205121b2249944eb9b2e1401cf693663. Resources: {'key_pairs'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:18:37.501730 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_api:SELECT=97,nova_api:UPDATE=2,nova_api:INSERT=1 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:18:37.594950 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-efb4a587-a7f6-4e2e-b529-2c1ff2f8b645 tempest-KeyPairsV2TestJSON-30838616 tempest-KeyPairsV2TestJSON-30838616-project] 10.222.0.16 "POST /compute/v2.1/os-keypairs" status: 200 len: 385 microversion: 2.1 time: 0.258544 Jul 16 21:18:37.595524 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1243/2407] 10.222.0.16 () {60 vars in 1158 bytes} [Sat Jul 16 21:18:37 2022] POST /compute/v2.1/os-keypairs => generated 385 bytes in 261 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:18:37.600204 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-fd389b82-708d-4154-9f7d-c7e63c82caa8 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:37.600944 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-fd389b82-708d-4154-9f7d-c7e63c82caa8 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] [instance: e7598b4a-f70c-4186-88d1-6fd0057f8656] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:18:37.608237 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-fd389b82-708d-4154-9f7d-c7e63c82caa8 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:37.609032 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-fd389b82-708d-4154-9f7d-c7e63c82caa8 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:37.609975 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-fd389b82-708d-4154-9f7d-c7e63c82caa8 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:37.611478 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-fd389b82-708d-4154-9f7d-c7e63c82caa8 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:37.757405 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-fd389b82-708d-4154-9f7d-c7e63c82caa8 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f5bc1f2209f4eb186d80e385b67c774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '65d5bcfa02af4e8284ba1342197e2860', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:18:37.765307 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-fd389b82-708d-4154-9f7d-c7e63c82caa8 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] 10.222.0.16 "GET /compute/v2.1/servers/e7598b4a-f70c-4186-88d1-6fd0057f8656" status: 200 len: 1471 microversion: 2.1 time: 0.168693 Jul 16 21:18:37.765875 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1244/2408] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:18:37 2022] GET /compute/v2.1/servers/e7598b4a-f70c-4186-88d1-6fd0057f8656 => generated 1471 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:18:37.770375 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-9ee970e0-4444-45ec-ab25-ef813e894b13 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-SecurityGroupsTestJSON-server-1940801925", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452", "networks": [{"uuid": "8cd71feb-4bea-45d0-9b6a-2960b1a03de1"}]}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:18:37.822875 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.network.neutron [None req-9ee970e0-4444-45ec-ab25-ef813e894b13 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] validate_networks() for [('8cd71feb-4bea-45d0-9b6a-2960b1a03de1', None, None, None, None, None)] {{(pid=138680) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 16 21:18:37.975466 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-a180a03c-dd0b-4bbb-a395-aa817cf65757 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] 10.222.0.16 "DELETE /compute/v2.1/images/eee6448e-8c8a-42ef-b4df-d19c5d9ae531" status: 204 len: 0 microversion: 2.1 time: 1.404895 Jul 16 21:18:37.976048 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1165/2409] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:18:36 2022] DELETE /compute/v2.1/images/eee6448e-8c8a-42ef-b4df-d19c5d9ae531 => generated 0 bytes in 1407 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 16 21:18:37.982375 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-2e6e11bf-647c-4e66-9751-6d89c241ed4e tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Calling method '.version_select at 0x7fdd9d087b80>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:38.064299 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-2e6e11bf-647c-4e66-9751-6d89c241ed4e tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] 10.222.0.16 "GET /compute/v2.1/images?limit=1" status: 200 len: 621 microversion: 2.1 time: 0.087211 Jul 16 21:18:38.072111 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1166/2410] 10.222.0.16 () {58 vars in 1136 bytes} [Sat Jul 16 21:18:37 2022] GET /compute/v2.1/images?limit=1 => generated 621 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:18:38.072111 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-e8d8f49b-4280-4cb5-ad8e-e7de908eb8e5 tempest-ServersNegativeTestMultiTenantJSON-1135003248 tempest-ServersNegativeTestMultiTenantJSON-1135003248-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:38.072111 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-e8d8f49b-4280-4cb5-ad8e-e7de908eb8e5 tempest-ServersNegativeTestMultiTenantJSON-1135003248 tempest-ServersNegativeTestMultiTenantJSON-1135003248-project] [instance: 43169d4f-1214-419a-aa1e-def65182ed5a] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:18:38.078891 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e8d8f49b-4280-4cb5-ad8e-e7de908eb8e5 tempest-ServersNegativeTestMultiTenantJSON-1135003248 tempest-ServersNegativeTestMultiTenantJSON-1135003248-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:38.079436 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e8d8f49b-4280-4cb5-ad8e-e7de908eb8e5 tempest-ServersNegativeTestMultiTenantJSON-1135003248 tempest-ServersNegativeTestMultiTenantJSON-1135003248-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:38.080264 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e8d8f49b-4280-4cb5-ad8e-e7de908eb8e5 tempest-ServersNegativeTestMultiTenantJSON-1135003248 tempest-ServersNegativeTestMultiTenantJSON-1135003248-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:38.080775 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e8d8f49b-4280-4cb5-ad8e-e7de908eb8e5 tempest-ServersNegativeTestMultiTenantJSON-1135003248 tempest-ServersNegativeTestMultiTenantJSON-1135003248-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:38.097984 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-e8d8f49b-4280-4cb5-ad8e-e7de908eb8e5 tempest-ServersNegativeTestMultiTenantJSON-1135003248 tempest-ServersNegativeTestMultiTenantJSON-1135003248-project] HTTP exception thrown: Instance 43169d4f-1214-419a-aa1e-def65182ed5a could not be found. Jul 16 21:18:38.099946 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-e8d8f49b-4280-4cb5-ad8e-e7de908eb8e5 tempest-ServersNegativeTestMultiTenantJSON-1135003248 tempest-ServersNegativeTestMultiTenantJSON-1135003248-project] Request method failure captured: Jul 16 21:18:38.099946 ncdv-master-1 devstack@n-api.service[138681]: request: GET /compute/v2.1/servers/43169d4f-1214-419a-aa1e-def65182ed5a HTTP/1.1 Jul 16 21:18:38.099946 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:18:38.099946 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:18:38.099946 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:18:38.099946 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 21:18:38.099946 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:18:38.099946 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:18:38.099946 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:18:38.099946 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:18:38.099946 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0yryifRXARMsLC12lbEY2MUHYyD_6KxWP3wy6bVLRK2bKAJT3OGjueaRD7XM5oJJv7PDs-y_pJtoOp9AuxA2v53jq8Ii-HEqQOKE8rA0y8Ucw8-wrxmYtIl8NCNLf0XFf10iaKOQnvxmhvG1NJqYcIxI0dHU3Vch8Ce3BkTy8dR-ORw Jul 16 21:18:38.099946 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:18:38.099946 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:18:38.099946 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:18:38.099946 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:18:38.099946 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:18:38.099946 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:18:38.099946 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: ba229aaef0ec4ddcadd1d940376c31ac Jul 16 21:18:38.099946 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-ServersNegativeTestMultiTenantJSON-1135003248 Jul 16 21:18:38.099946 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:18:38.099946 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:18:38.099946 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/ba229aaef0ec4ddcadd1d940376c31ac"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/ba229aaef0ec4ddcadd1d940376c31ac"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_ba229aaef0ec4ddcadd1d940376c31ac"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/ba229aaef0ec4ddcadd1d940376c31ac"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:18:38.099946 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-ServersNegativeTestMultiTenantJSON-1135003248 Jul 16 21:18:38.099946 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: ba229aaef0ec4ddcadd1d940376c31ac Jul 16 21:18:38.099946 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-ServersNegativeTestMultiTenantJSON-1135003248 Jul 16 21:18:38.099946 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-ServersNegativeTestMultiTenantJSON-1135003248-project Jul 16 21:18:38.099946 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:18:38.099946 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:18:38.099946 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: 50cdc134e712487196f6623c5cf14592 Jul 16 21:18:38.099946 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-ServersNegativeTestMultiTenantJSON-1135003248-project Jul 16 21:18:38.099946 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:18:38.099946 ncdv-master-1 devstack@n-api.service[138681]: exception: Instance 43169d4f-1214-419a-aa1e-def65182ed5a could not be found. Jul 16 21:18:38.099946 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': '43169d4f-1214-419a-aa1e-def65182ed5a'} Jul 16 21:18:38.099946 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:18:38.099946 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:18:38.099946 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:18:38.099946 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:18:38.099946 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:18:38.099946 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:18:38.099946 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:18:38.099946 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:18:38.099946 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:18:38.099946 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:18:38.099946 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 43169d4f-1214-419a-aa1e-def65182ed5a could not be found. Jul 16 21:18:38.099946 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:18:38.099946 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:18:38.099946 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:18:38.099946 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:18:38.099946 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:18:38.099946 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:18:38.099946 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:18:38.099946 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:18:38.099946 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:18:38.099946 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:18:38.099946 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:18:38.099946 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:18:38.099946 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:18:38.099946 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:18:38.099946 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:18:38.099946 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:18:38.099946 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 43169d4f-1214-419a-aa1e-def65182ed5a could not be found. Jul 16 21:18:38.099946 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:18:38.109238 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:18:38.109238 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:18:38.109238 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:18:38.109238 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:18:38.109238 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:18:38.109238 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:18:38.109238 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:18:38.109238 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 43169d4f-1214-419a-aa1e-def65182ed5a could not be found. Jul 16 21:18:38.109238 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:18:38.109238 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-e8d8f49b-4280-4cb5-ad8e-e7de908eb8e5 tempest-ServersNegativeTestMultiTenantJSON-1135003248 tempest-ServersNegativeTestMultiTenantJSON-1135003248-project] Returning 404 to user: Instance 43169d4f-1214-419a-aa1e-def65182ed5a could not be found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:18:38.109238 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-e8d8f49b-4280-4cb5-ad8e-e7de908eb8e5 tempest-ServersNegativeTestMultiTenantJSON-1135003248 tempest-ServersNegativeTestMultiTenantJSON-1135003248-project] 10.222.0.16 "GET /compute/v2.1/servers/43169d4f-1214-419a-aa1e-def65182ed5a" status: 404 len: 111 microversion: 2.1 time: 0.035581 Jul 16 21:18:38.109238 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1167/2411] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:18:38 2022] GET /compute/v2.1/servers/43169d4f-1214-419a-aa1e-def65182ed5a => generated 111 bytes in 37 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:18:38.109238 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-07d3592a-8ec6-4246-902e-862f1ec5d2ce tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:38.109238 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-07d3592a-8ec6-4246-902e-862f1ec5d2ce tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] [instance: 6baac85b-f81c-4e3a-9f96-633cffccfd74] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:18:38.114932 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-07d3592a-8ec6-4246-902e-862f1ec5d2ce tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:38.115370 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-07d3592a-8ec6-4246-902e-862f1ec5d2ce tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:38.116116 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-07d3592a-8ec6-4246-902e-862f1ec5d2ce tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:38.116552 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-07d3592a-8ec6-4246-902e-862f1ec5d2ce tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:38.147239 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-07d3592a-8ec6-4246-902e-862f1ec5d2ce tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] [instance: 6baac85b-f81c-4e3a-9f96-633cffccfd74] Going to try to terminate instance {{(pid=138681) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 16 21:18:38.223238 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-9ee970e0-4444-45ec-ab25-ef813e894b13 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Flavor limits 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 16 21:18:38.223753 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-9ee970e0-4444-45ec-ab25-ef813e894b13 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Image limits 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 16 21:18:38.224166 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-9ee970e0-4444-45ec-ab25-ef813e894b13 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Flavor pref 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 16 21:18:38.224622 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-9ee970e0-4444-45ec-ab25-ef813e894b13 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Image pref 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 16 21:18:38.225217 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-9ee970e0-4444-45ec-ab25-ef813e894b13 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 16 21:18:38.235555 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9ee970e0-4444-45ec-ab25-ef813e894b13 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:38.236011 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9ee970e0-4444-45ec-ab25-ef813e894b13 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:38.236818 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9ee970e0-4444-45ec-ab25-ef813e894b13 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:38.237498 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9ee970e0-4444-45ec-ab25-ef813e894b13 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:38.256048 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-9ee970e0-4444-45ec-ab25-ef813e894b13 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Getting quotas for project e353258f9c3048ef84683294e2b8579d. Resources: {'ram', 'instances', 'cores'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:18:38.262281 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-9ee970e0-4444-45ec-ab25-ef813e894b13 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Getting quotas for user d0008fffad8e4708b2c01e03da92c47d and project e353258f9c3048ef84683294e2b8579d. Resources: {'ram', 'instances', 'cores'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:18:38.274915 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-9ee970e0-4444-45ec-ab25-ef813e894b13 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Going to run 1 instances... {{(pid=138680) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 16 21:18:38.277955 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-9ee970e0-4444-45ec-ab25-ef813e894b13 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] [instance: c88b191a-1ae5-4952-8afa-14aee5567a7f] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138680) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 16 21:18:38.317302 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.compute.api [None req-07d3592a-8ec6-4246-902e-862f1ec5d2ce tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] [instance: 6baac85b-f81c-4e3a-9f96-633cffccfd74] Instance is already in deleting state, ignoring this request Jul 16 21:18:38.318127 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-07d3592a-8ec6-4246-902e-862f1ec5d2ce tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] 10.222.0.16 "DELETE /compute/v2.1/servers/6baac85b-f81c-4e3a-9f96-633cffccfd74" status: 204 len: 0 microversion: 2.1 time: 0.215290 Jul 16 21:18:38.318689 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1168/2412] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:18:38 2022] DELETE /compute/v2.1/servers/6baac85b-f81c-4e3a-9f96-633cffccfd74 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 16 21:18:38.324336 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-5e06ff48-a591-463d-aa51-12356818da93 tempest-KeyPairsV2TestJSON-30838616 tempest-KeyPairsV2TestJSON-30838616-project] Calling method '.version_select at 0x7fdd9cfa6040>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:38.332125 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-5e06ff48-a591-463d-aa51-12356818da93 tempest-KeyPairsV2TestJSON-30838616 tempest-KeyPairsV2TestJSON-30838616-project] 10.222.0.16 "GET /compute/v2.1/os-keypairs/tempest-keypair-983547489" status: 200 len: 497 microversion: 2.1 time: 0.012410 Jul 16 21:18:38.332674 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1169/2413] 10.222.0.16 () {58 vars in 1214 bytes} [Sat Jul 16 21:18:38 2022] GET /compute/v2.1/os-keypairs/tempest-keypair-983547489 => generated 497 bytes in 13 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:18:38.342242 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-a9073c07-6ccd-4a01-9864-5995fec95c62 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:38.342848 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-a9073c07-6ccd-4a01-9864-5995fec95c62 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] [instance: ed8dcb97-7546-45ef-9e9d-a9bb03a1b419] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:18:38.349082 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a9073c07-6ccd-4a01-9864-5995fec95c62 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:38.349533 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a9073c07-6ccd-4a01-9864-5995fec95c62 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:38.350303 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a9073c07-6ccd-4a01-9864-5995fec95c62 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:38.350739 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a9073c07-6ccd-4a01-9864-5995fec95c62 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:38.376892 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-a9073c07-6ccd-4a01-9864-5995fec95c62 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] [instance: ed8dcb97-7546-45ef-9e9d-a9bb03a1b419] Going to try to terminate instance {{(pid=138681) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 16 21:18:38.417395 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-9ee970e0-4444-45ec-ab25-ef813e894b13 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.650234 Jul 16 21:18:38.418325 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1245/2414] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 21:18:37 2022] POST /compute/v2.1/servers => generated 380 bytes in 652 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 16 21:18:38.423370 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-e4388363-b292-425e-acbe-9c7284e33124 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Calling method '.version_select at 0x7fdda2bc5dc0>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:38.580507 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-e4388363-b292-425e-acbe-9c7284e33124 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] 10.222.0.16 "GET /compute/v2.1/images/detail?changes-since=2022-07-16T21%3A16%3A41Z" status: 200 len: 6334 microversion: 2.1 time: 0.160940 Jul 16 21:18:38.581703 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1246/2415] 10.222.0.16 () {58 vars in 1219 bytes} [Sat Jul 16 21:18:38 2022] GET /compute/v2.1/images/detail?changes-since=2022-07-16T21%3A16%3A41Z => generated 6334 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:18:38.586748 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-beb662b4-44d1-4113-b66f-4797b384e102 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:38.587472 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-beb662b4-44d1-4113-b66f-4797b384e102 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] [instance: 6baac85b-f81c-4e3a-9f96-633cffccfd74] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:18:38.597438 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-beb662b4-44d1-4113-b66f-4797b384e102 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:38.597978 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-beb662b4-44d1-4113-b66f-4797b384e102 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:38.598775 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-beb662b4-44d1-4113-b66f-4797b384e102 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:38.599292 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-beb662b4-44d1-4113-b66f-4797b384e102 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:38.771910 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-a9073c07-6ccd-4a01-9864-5995fec95c62 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] 10.222.0.16 "DELETE /compute/v2.1/servers/ed8dcb97-7546-45ef-9e9d-a9bb03a1b419" status: 204 len: 0 microversion: 2.1 time: 0.437146 Jul 16 21:18:38.771910 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1170/2416] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:18:38 2022] DELETE /compute/v2.1/servers/ed8dcb97-7546-45ef-9e9d-a9bb03a1b419 => generated 0 bytes in 438 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 16 21:18:38.778740 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-4e348610-b31c-4272-adbe-4d57fd841cd4 tempest-KeyPairsV2TestJSON-30838616 tempest-KeyPairsV2TestJSON-30838616-project] Calling method '.version_select at 0x7fdd9cda21f0>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:38.823143 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-beb662b4-44d1-4113-b66f-4797b384e102 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd4144b5388b46a989a41a2f556bb508', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a90d4bcd25ff4bf68bdc194180fbbe09', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:18:38.833294 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-beb662b4-44d1-4113-b66f-4797b384e102 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] 10.222.0.16 "GET /compute/v2.1/servers/6baac85b-f81c-4e3a-9f96-633cffccfd74" status: 200 len: 1513 microversion: 2.1 time: 0.248277 Jul 16 21:18:38.833952 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1247/2417] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:18:38 2022] GET /compute/v2.1/servers/6baac85b-f81c-4e3a-9f96-633cffccfd74 => generated 1513 bytes in 252 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:18:38.848483 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-46aa7fcf-efef-4e0c-ada8-7aa86b9546b1 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:38.849133 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-46aa7fcf-efef-4e0c-ada8-7aa86b9546b1 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] [instance: c88b191a-1ae5-4952-8afa-14aee5567a7f] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:18:38.924283 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-4e348610-b31c-4272-adbe-4d57fd841cd4 tempest-KeyPairsV2TestJSON-30838616 tempest-KeyPairsV2TestJSON-30838616-project] 10.222.0.16 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-983547489" status: 202 len: 0 microversion: 2.1 time: 0.151148 Jul 16 21:18:38.925234 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1171/2418] 10.222.0.16 () {58 vars in 1217 bytes} [Sat Jul 16 21:18:38 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-983547489 => generated 0 bytes in 153 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 21:18:38.932081 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-f438816a-a0e9-4968-af31-1b670c0b7efa tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Calling method '.version_select at 0x7fdd9d2670d0>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:39.049192 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-f438816a-a0e9-4968-af31-1b670c0b7efa tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] 10.222.0.16 "GET /compute/v2.1/images/detail?name=tempest-ListImageFiltersTestJSON-image-1673862976" status: 200 len: 665 microversion: 2.1 time: 0.122841 Jul 16 21:18:39.059158 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1172/2419] 10.222.0.16 () {58 vars in 1251 bytes} [Sat Jul 16 21:18:38 2022] GET /compute/v2.1/images/detail?name=tempest-ListImageFiltersTestJSON-image-1673862976 => generated 665 bytes in 133 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:18:39.064544 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-cc39578e-2641-4f69-9d0b-6dca14b3213f req-1e29a445-2ea9-413d-ae6c-ee11f934cdb2 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "ed8dcb97-7546-45ef-9e9d-a9bb03a1b419", "name": "network-vif-unplugged", "status": "completed", "tag": "033ca6ad-5781-4719-b3b5-986dcfd7de3d"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:18:39.075283 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-cc39578e-2641-4f69-9d0b-6dca14b3213f req-1e29a445-2ea9-413d-ae6c-ee11f934cdb2 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:39.075981 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-cc39578e-2641-4f69-9d0b-6dca14b3213f req-1e29a445-2ea9-413d-ae6c-ee11f934cdb2 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:39.089286 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-cc39578e-2641-4f69-9d0b-6dca14b3213f req-1e29a445-2ea9-413d-ae6c-ee11f934cdb2 service nova] Creating event network-vif-unplugged:033ca6ad-5781-4719-b3b5-986dcfd7de3d for instance ed8dcb97-7546-45ef-9e9d-a9bb03a1b419 on nch2-master-1 Jul 16 21:18:39.095185 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-cc39578e-2641-4f69-9d0b-6dca14b3213f req-1e29a445-2ea9-413d-ae6c-ee11f934cdb2 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.034300 Jul 16 21:18:39.095865 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1173/2420] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:18:39 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:18:39.101872 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-f31f7fec-977a-4a16-9319-88ada2fbf226 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:39.102754 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-f31f7fec-977a-4a16-9319-88ada2fbf226 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] [instance: e7598b4a-f70c-4186-88d1-6fd0057f8656] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:18:39.110841 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f31f7fec-977a-4a16-9319-88ada2fbf226 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:39.111327 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f31f7fec-977a-4a16-9319-88ada2fbf226 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:39.112308 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f31f7fec-977a-4a16-9319-88ada2fbf226 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:39.112752 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f31f7fec-977a-4a16-9319-88ada2fbf226 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:39.309606 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-46aa7fcf-efef-4e0c-ada8-7aa86b9546b1 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd0008fffad8e4708b2c01e03da92c47d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e353258f9c3048ef84683294e2b8579d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:18:39.320059 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-46aa7fcf-efef-4e0c-ada8-7aa86b9546b1 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] 10.222.0.16 "GET /compute/v2.1/servers/c88b191a-1ae5-4952-8afa-14aee5567a7f" status: 200 len: 1216 microversion: 2.1 time: 0.484970 Jul 16 21:18:39.321348 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1248/2421] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:18:38 2022] GET /compute/v2.1/servers/c88b191a-1ae5-4952-8afa-14aee5567a7f => generated 1216 bytes in 487 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:18:39.327493 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-39da215e-fb8d-403e-9037-ca5f21d61168 tempest-KeyPairsV2TestJSON-30838616 tempest-KeyPairsV2TestJSON-30838616-project] Action: 'create', calling method: .version_select at 0x7fdda2bc5670>, body: {"keypair": {"name": "tempest-keypair-1137674212", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBBl7WNZrx7Mw53/BA/NXBQ32G1P53CWK5KI8ODj1My/HCA1VAYFSnonZxh8nkGoveZKg+mhdS+jssiRc+mZaQB8NTJvfwunbRD4ZaB/CDaEwDiX2xVUHp7nMGkORC26y+Q=="}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:18:39.349525 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-39da215e-fb8d-403e-9037-ca5f21d61168 tempest-KeyPairsV2TestJSON-30838616 tempest-KeyPairsV2TestJSON-30838616-project] Getting quotas for project 205121b2249944eb9b2e1401cf693663. Resources: {'key_pairs'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:18:39.357194 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-39da215e-fb8d-403e-9037-ca5f21d61168 tempest-KeyPairsV2TestJSON-30838616 tempest-KeyPairsV2TestJSON-30838616-project] Getting quotas for user e848e24fe4344314a82f574a1bf1283f and project 205121b2249944eb9b2e1401cf693663. Resources: {'key_pairs'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:18:39.499591 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-39da215e-fb8d-403e-9037-ca5f21d61168 tempest-KeyPairsV2TestJSON-30838616 tempest-KeyPairsV2TestJSON-30838616-project] 10.222.0.16 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.176378 Jul 16 21:18:39.500936 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1249/2422] 10.222.0.16 () {60 vars in 1158 bytes} [Sat Jul 16 21:18:39 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:18:39.509522 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-18b649bc-024a-419c-bf63-1b94c9f94867 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ImagesTestJSON-server-590205140", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452", "networks": [{"uuid": "e0956c94-52de-403b-bf7c-aa26a7179ba6"}]}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:18:39.510748 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-f31f7fec-977a-4a16-9319-88ada2fbf226 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f5bc1f2209f4eb186d80e385b67c774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '65d5bcfa02af4e8284ba1342197e2860', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:18:39.518850 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-f31f7fec-977a-4a16-9319-88ada2fbf226 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] 10.222.0.16 "GET /compute/v2.1/servers/e7598b4a-f70c-4186-88d1-6fd0057f8656" status: 200 len: 1471 microversion: 2.1 time: 0.421811 Jul 16 21:18:39.529177 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1174/2423] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:18:39 2022] GET /compute/v2.1/servers/e7598b4a-f70c-4186-88d1-6fd0057f8656 => generated 1471 bytes in 433 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:18:39.539410 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-66098b95-4602-4d8f-85c9-1c4f24cd3e58 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Calling method '.version_select at 0x7fdd9cdc4940>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:39.604424 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.network.neutron [None req-18b649bc-024a-419c-bf63-1b94c9f94867 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] validate_networks() for [('e0956c94-52de-403b-bf7c-aa26a7179ba6', None, None, None, None, None)] {{(pid=138680) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 16 21:18:39.623530 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-66098b95-4602-4d8f-85c9-1c4f24cd3e58 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] 10.222.0.16 "GET /compute/v2.1/images/detail?server=http%3A%2F%2F10.222.0.16%2Fcompute%2Fv2.1%2Fservers%2F6cb4e66a-eba2-42b7-91e8-09ee7b2a5a4f" status: 200 len: 1466 microversion: 2.1 time: 0.093081 Jul 16 21:18:39.624119 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1175/2424] 10.222.0.16 () {58 vars in 1337 bytes} [Sat Jul 16 21:18:39 2022] GET /compute/v2.1/images/detail?server=http%3A%2F%2F10.222.0.16%2Fcompute%2Fv2.1%2Fservers%2F6cb4e66a-eba2-42b7-91e8-09ee7b2a5a4f => generated 1466 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:18:39.632481 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-ceb21b28-d1e2-4fae-99d3-ceef66f5352f tempest-KeyPairsV2TestJSON-30838616 tempest-KeyPairsV2TestJSON-30838616-project] Calling method '.version_select at 0x7fdd9d2575e0>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:39.739617 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-ceb21b28-d1e2-4fae-99d3-ceef66f5352f tempest-KeyPairsV2TestJSON-30838616 tempest-KeyPairsV2TestJSON-30838616-project] 10.222.0.16 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1137674212" status: 202 len: 0 microversion: 2.1 time: 0.113954 Jul 16 21:18:39.744596 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1176/2425] 10.222.0.16 () {58 vars in 1220 bytes} [Sat Jul 16 21:18:39 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1137674212 => generated 0 bytes in 116 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 21:18:39.755312 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-b59b6b50-f747-4491-a150-98c69a6c2fd5 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Calling method '.version_select at 0x7fdd9d087b80>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:39.820754 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-b59b6b50-f747-4491-a150-98c69a6c2fd5 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] 10.222.0.16 "GET /compute/v2.1/images/detail?server=http%3A%2F%2F10.222.0.16%2Fcompute%2Fservers%2F6cb4e66a-eba2-42b7-91e8-09ee7b2a5a4f" status: 200 len: 1466 microversion: 2.1 time: 0.079337 Jul 16 21:18:39.821390 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1177/2426] 10.222.0.16 () {58 vars in 1323 bytes} [Sat Jul 16 21:18:39 2022] GET /compute/v2.1/images/detail?server=http%3A%2F%2F10.222.0.16%2Fcompute%2Fservers%2F6cb4e66a-eba2-42b7-91e8-09ee7b2a5a4f => generated 1466 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:18:39.827240 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-4ebfbbca-ca8d-4539-ba81-204c9f7dfc53 tempest-KeyPairsV2TestJSON-30838616 tempest-KeyPairsV2TestJSON-30838616-project] Action: 'create', calling method: .version_select at 0x7fdd9d221160>, body: {"keypair": {"name": "tempest-keypair-2108714585", "public_key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCsNe3/1ILNCqFyfYWDeTKLD6jEXC2OQHLmietMWW+/vdaZq7KZEwO0jhglaFjU1mpqq4Gz5RX156sCTNM9vRbwKAxfsdF9laBYVsex3m3Wmui3uYrKyumsoJn2g9GNnG1PI1mrVjZ61i0GY3khna+wzlTpCCmy5HNlrmbj3XLqBUpipTOXmsnr4sChzC53KCd8LXuwc1i/CZPvF+3XipvAgFSE53pCtLOeB1kYMOBaiUPLQTWXR3JpckqFIQwhIH0zoHlJvZE8hh90XcPojYN56tI0OlrGqojbediJYD0rUsJu4weZpbn8vilb3JuDY+jwssnSA8wzBx3A/8y9Pp1B nova@ubuntu"}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:18:39.856395 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-4ebfbbca-ca8d-4539-ba81-204c9f7dfc53 tempest-KeyPairsV2TestJSON-30838616 tempest-KeyPairsV2TestJSON-30838616-project] Getting quotas for project 205121b2249944eb9b2e1401cf693663. Resources: {'key_pairs'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:18:39.862652 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-4ebfbbca-ca8d-4539-ba81-204c9f7dfc53 tempest-KeyPairsV2TestJSON-30838616 tempest-KeyPairsV2TestJSON-30838616-project] Getting quotas for user e848e24fe4344314a82f574a1bf1283f and project 205121b2249944eb9b2e1401cf693663. Resources: {'key_pairs'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:18:39.958559 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-4ebfbbca-ca8d-4539-ba81-204c9f7dfc53 tempest-KeyPairsV2TestJSON-30838616 tempest-KeyPairsV2TestJSON-30838616-project] 10.222.0.16 "POST /compute/v2.1/os-keypairs" status: 200 len: 574 microversion: 2.1 time: 0.136023 Jul 16 21:18:39.963896 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1178/2427] 10.222.0.16 () {60 vars in 1158 bytes} [Sat Jul 16 21:18:39 2022] POST /compute/v2.1/os-keypairs => generated 574 bytes in 137 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:18:39.965781 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-d96f98c5-5d7a-4dec-b487-44d66ff1e233 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Calling method '.version_select at 0x7fdd9d6dcca0>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:40.068562 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-d96f98c5-5d7a-4dec-b487-44d66ff1e233 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] 10.222.0.16 "GET /compute/v2.1/images/detail?status=ACTIVE" status: 200 len: 7109 microversion: 2.1 time: 0.106946 Jul 16 21:18:40.069301 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1179/2428] 10.222.0.16 () {58 vars in 1169 bytes} [Sat Jul 16 21:18:39 2022] GET /compute/v2.1/images/detail?status=ACTIVE => generated 7109 bytes in 109 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:18:40.087693 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-209af9dd-048d-4d98-8814-494b15aed741 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:40.100875 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-209af9dd-048d-4d98-8814-494b15aed741 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] [instance: 6baac85b-f81c-4e3a-9f96-633cffccfd74] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:18:40.117079 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-209af9dd-048d-4d98-8814-494b15aed741 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:40.117781 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-209af9dd-048d-4d98-8814-494b15aed741 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:40.119876 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-209af9dd-048d-4d98-8814-494b15aed741 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:40.120829 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-209af9dd-048d-4d98-8814-494b15aed741 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:40.231446 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-209af9dd-048d-4d98-8814-494b15aed741 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd4144b5388b46a989a41a2f556bb508', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a90d4bcd25ff4bf68bdc194180fbbe09', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:18:40.241510 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-209af9dd-048d-4d98-8814-494b15aed741 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] 10.222.0.16 "GET /compute/v2.1/servers/6baac85b-f81c-4e3a-9f96-633cffccfd74" status: 200 len: 1471 microversion: 2.1 time: 0.171102 Jul 16 21:18:40.242590 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1180/2429] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:18:40 2022] GET /compute/v2.1/servers/6baac85b-f81c-4e3a-9f96-633cffccfd74 => generated 1471 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:18:40.248888 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-ac077a64-00b6-46ff-be8e-5e80e45c1634 tempest-KeyPairsV2TestJSON-30838616 tempest-KeyPairsV2TestJSON-30838616-project] Calling method '.version_select at 0x7fdd9dbfc940>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:40.292779 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-18b649bc-024a-419c-bf63-1b94c9f94867 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Flavor limits 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 16 21:18:40.293217 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-18b649bc-024a-419c-bf63-1b94c9f94867 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Image limits 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 16 21:18:40.293704 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-18b649bc-024a-419c-bf63-1b94c9f94867 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Flavor pref 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 16 21:18:40.294843 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-18b649bc-024a-419c-bf63-1b94c9f94867 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Image pref 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 16 21:18:40.295297 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-18b649bc-024a-419c-bf63-1b94c9f94867 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 16 21:18:40.312629 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-18b649bc-024a-419c-bf63-1b94c9f94867 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:40.313284 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-18b649bc-024a-419c-bf63-1b94c9f94867 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:40.313891 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-18b649bc-024a-419c-bf63-1b94c9f94867 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:40.314342 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-18b649bc-024a-419c-bf63-1b94c9f94867 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:40.363257 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-18b649bc-024a-419c-bf63-1b94c9f94867 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Getting quotas for project 24c64db8082648c8837c16352252f96d. Resources: {'ram', 'instances', 'cores'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:18:40.370974 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-18b649bc-024a-419c-bf63-1b94c9f94867 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Getting quotas for user daf3c9e6634e4ae1bfb492eae1f44c93 and project 24c64db8082648c8837c16352252f96d. Resources: {'ram', 'instances', 'cores'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:18:40.387502 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-18b649bc-024a-419c-bf63-1b94c9f94867 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Going to run 1 instances... {{(pid=138680) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 16 21:18:40.389875 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-18b649bc-024a-419c-bf63-1b94c9f94867 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] [instance: d265192b-536b-4884-9fe8-74cf20fc7ef9] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138680) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 16 21:18:40.428500 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-ac077a64-00b6-46ff-be8e-5e80e45c1634 tempest-KeyPairsV2TestJSON-30838616 tempest-KeyPairsV2TestJSON-30838616-project] 10.222.0.16 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-2108714585" status: 202 len: 0 microversion: 2.1 time: 0.184592 Jul 16 21:18:40.429188 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1181/2430] 10.222.0.16 () {58 vars in 1220 bytes} [Sat Jul 16 21:18:40 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-2108714585 => generated 0 bytes in 186 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 21:18:40.445055 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-4bbeafc8-8fe0-4a24-ae36-6e43095faa64 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Calling method '.version_select at 0x7fdd9cda24c0>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:40.533313 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-4bbeafc8-8fe0-4a24-ae36-6e43095faa64 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] 10.222.0.16 "GET /compute/v2.1/images/detail?type=snapshot" status: 200 len: 4375 microversion: 2.1 time: 0.092080 Jul 16 21:18:40.537353 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1182/2431] 10.222.0.16 () {58 vars in 1169 bytes} [Sat Jul 16 21:18:40 2022] GET /compute/v2.1/images/detail?type=snapshot => generated 4375 bytes in 108 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:18:40.564521 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-18b649bc-024a-419c-bf63-1b94c9f94867 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 1.062191 Jul 16 21:18:40.565754 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1250/2432] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 21:18:39 2022] POST /compute/v2.1/servers => generated 380 bytes in 1064 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 16 21:18:40.576006 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-bed6fd12-7bc5-403a-b52f-eaedaaee6c53 req-bf2e796a-5d77-4f9f-84d3-d8a4852a0e99 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "6baac85b-f81c-4e3a-9f96-633cffccfd74", "name": "network-vif-deleted", "tag": "92263c71-5a11-478f-8dc4-be33c5f55e4f"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:18:40.576159 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-709bb5b3-9e1e-4bb0-9719-8896c68e025a tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:40.576159 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-709bb5b3-9e1e-4bb0-9719-8896c68e025a tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] [instance: c88b191a-1ae5-4952-8afa-14aee5567a7f] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:18:40.584930 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-709bb5b3-9e1e-4bb0-9719-8896c68e025a tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:40.588355 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-709bb5b3-9e1e-4bb0-9719-8896c68e025a tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:40.589533 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-709bb5b3-9e1e-4bb0-9719-8896c68e025a tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:40.590449 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-709bb5b3-9e1e-4bb0-9719-8896c68e025a tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:40.595154 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-bed6fd12-7bc5-403a-b52f-eaedaaee6c53 req-bf2e796a-5d77-4f9f-84d3-d8a4852a0e99 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:40.595652 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-bed6fd12-7bc5-403a-b52f-eaedaaee6c53 req-bf2e796a-5d77-4f9f-84d3-d8a4852a0e99 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:40.609249 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-bed6fd12-7bc5-403a-b52f-eaedaaee6c53 req-bf2e796a-5d77-4f9f-84d3-d8a4852a0e99 service nova] Creating event network-vif-deleted:92263c71-5a11-478f-8dc4-be33c5f55e4f for instance 6baac85b-f81c-4e3a-9f96-633cffccfd74 on nch1-master-1 Jul 16 21:18:40.615358 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-bed6fd12-7bc5-403a-b52f-eaedaaee6c53 req-bf2e796a-5d77-4f9f-84d3-d8a4852a0e99 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.067667 Jul 16 21:18:40.616663 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1183/2433] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:18:40 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 70 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:18:40.624608 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-edac38e6-3cbe-4eb6-9a39-b72e18c2a497 tempest-KeyPairsV2TestJSON-30838616 tempest-KeyPairsV2TestJSON-30838616-project] Action: 'create', calling method: .version_select at 0x7fdd9d087160>, body: {"keypair": {"name": "tempest-KeyPairsV2TestJSON-keypair-114969803", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBDqFh5FmM1HMu0eXrfGDcih/D93wV9IrKNRjE0lKfGQxcZOIinW+zxtUd/dXxmUrqlQUUWggqsTK74YNOT/zKy1DHRfkCiXIhApBh39FbJJagSlcFppOCUNqe1x8SUo8Bw=="}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:18:40.641001 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-edac38e6-3cbe-4eb6-9a39-b72e18c2a497 tempest-KeyPairsV2TestJSON-30838616 tempest-KeyPairsV2TestJSON-30838616-project] Getting quotas for project 205121b2249944eb9b2e1401cf693663. Resources: {'key_pairs'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:18:40.646885 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-edac38e6-3cbe-4eb6-9a39-b72e18c2a497 tempest-KeyPairsV2TestJSON-30838616 tempest-KeyPairsV2TestJSON-30838616-project] Getting quotas for user e848e24fe4344314a82f574a1bf1283f and project 205121b2249944eb9b2e1401cf693663. Resources: {'key_pairs'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:18:40.739877 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-709bb5b3-9e1e-4bb0-9719-8896c68e025a tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd0008fffad8e4708b2c01e03da92c47d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e353258f9c3048ef84683294e2b8579d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:18:40.752730 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-709bb5b3-9e1e-4bb0-9719-8896c68e025a tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] 10.222.0.16 "GET /compute/v2.1/servers/c88b191a-1ae5-4952-8afa-14aee5567a7f" status: 200 len: 1268 microversion: 2.1 time: 0.185678 Jul 16 21:18:40.759427 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1251/2434] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:18:40 2022] GET /compute/v2.1/servers/c88b191a-1ae5-4952-8afa-14aee5567a7f => generated 1268 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:18:40.766511 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-edac38e6-3cbe-4eb6-9a39-b72e18c2a497 tempest-KeyPairsV2TestJSON-30838616 tempest-KeyPairsV2TestJSON-30838616-project] 10.222.0.16 "POST /compute/v2.1/os-keypairs" status: 200 len: 404 microversion: 2.1 time: 0.147259 Jul 16 21:18:40.767042 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-3d820ca3-0e66-4815-9eeb-1bd0c66ffdbc tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:40.782230 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1184/2435] 10.222.0.16 () {60 vars in 1158 bytes} [Sat Jul 16 21:18:40 2022] POST /compute/v2.1/os-keypairs => generated 404 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:18:40.782384 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-3d820ca3-0e66-4815-9eeb-1bd0c66ffdbc tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] [instance: e7598b4a-f70c-4186-88d1-6fd0057f8656] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:18:40.782495 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-75ca82bd-50ae-4a5d-b750-9d03fb1b69d7 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Calling method '.version_select at 0x7fdd9d8d3280>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:40.803187 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3d820ca3-0e66-4815-9eeb-1bd0c66ffdbc tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:40.805944 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3d820ca3-0e66-4815-9eeb-1bd0c66ffdbc tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:40.810037 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3d820ca3-0e66-4815-9eeb-1bd0c66ffdbc tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:40.810512 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3d820ca3-0e66-4815-9eeb-1bd0c66ffdbc tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:40.848172 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-75ca82bd-50ae-4a5d-b750-9d03fb1b69d7 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] 10.222.0.16 "GET /compute/v2.1/images/1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9" status: 200 len: 784 microversion: 2.1 time: 0.079364 Jul 16 21:18:40.849792 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1185/2436] 10.222.0.16 () {58 vars in 1232 bytes} [Sat Jul 16 21:18:40 2022] GET /compute/v2.1/images/1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9 => generated 784 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:18:40.872778 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-3248b77a-58ee-4d5e-83e0-4b1590f14f08 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:40.875439 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-3248b77a-58ee-4d5e-83e0-4b1590f14f08 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] [instance: d265192b-536b-4884-9fe8-74cf20fc7ef9] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:18:40.993360 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-3248b77a-58ee-4d5e-83e0-4b1590f14f08 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'daf3c9e6634e4ae1bfb492eae1f44c93', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '24c64db8082648c8837c16352252f96d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:18:41.002275 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-3248b77a-58ee-4d5e-83e0-4b1590f14f08 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] 10.222.0.16 "GET /compute/v2.1/servers/d265192b-536b-4884-9fe8-74cf20fc7ef9" status: 200 len: 1207 microversion: 2.1 time: 0.144339 Jul 16 21:18:41.003099 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1186/2437] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:18:40 2022] GET /compute/v2.1/servers/d265192b-536b-4884-9fe8-74cf20fc7ef9 => generated 1207 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:18:41.017176 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-09a5eff4-e1fb-4443-85df-d7d97356fb9c tempest-KeyPairsV2TestJSON-30838616 tempest-KeyPairsV2TestJSON-30838616-project] Action: 'create', calling method: .version_select at 0x7fdd9d8d3700>, body: {"keypair": {"name": "tempest-KeyPairsV2TestJSON-keypair-15560015", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBI/NVZvyQkPP6TbTdOeKkFEu4I9XQXAfvxvI8avNgS0m6pNZbGZeBGdtwyoRoCbcnK0UsUcfZ+8hHbiU5qTpNcsSe/XBukYtj7q+JMmBJU8ptb1dgdCrYfQMc7Aimw6YQA=="}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:18:41.040650 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-09a5eff4-e1fb-4443-85df-d7d97356fb9c tempest-KeyPairsV2TestJSON-30838616 tempest-KeyPairsV2TestJSON-30838616-project] Getting quotas for project 205121b2249944eb9b2e1401cf693663. Resources: {'key_pairs'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:18:41.043936 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_api:SELECT=94,nova_api:UPDATE=1,nova_api:DELETE=3,nova_api:INSERT=2 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:18:41.048259 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-09a5eff4-e1fb-4443-85df-d7d97356fb9c tempest-KeyPairsV2TestJSON-30838616 tempest-KeyPairsV2TestJSON-30838616-project] Getting quotas for user e848e24fe4344314a82f574a1bf1283f and project 205121b2249944eb9b2e1401cf693663. Resources: {'key_pairs'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:18:41.170339 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-09a5eff4-e1fb-4443-85df-d7d97356fb9c tempest-KeyPairsV2TestJSON-30838616 tempest-KeyPairsV2TestJSON-30838616-project] 10.222.0.16 "POST /compute/v2.1/os-keypairs" status: 200 len: 403 microversion: 2.1 time: 0.165907 Jul 16 21:18:41.180960 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1187/2438] 10.222.0.16 () {60 vars in 1158 bytes} [Sat Jul 16 21:18:41 2022] POST /compute/v2.1/os-keypairs => generated 403 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:18:41.189185 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-0ec55dbb-6aaa-45db-a422-510026f3d1dc tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Calling method '.version_select at 0x7fdd9d267550>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:41.222358 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-3d820ca3-0e66-4815-9eeb-1bd0c66ffdbc tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f5bc1f2209f4eb186d80e385b67c774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '65d5bcfa02af4e8284ba1342197e2860', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:18:41.230925 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-3d820ca3-0e66-4815-9eeb-1bd0c66ffdbc tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] 10.222.0.16 "GET /compute/v2.1/servers/e7598b4a-f70c-4186-88d1-6fd0057f8656" status: 200 len: 1471 microversion: 2.1 time: 0.469453 Jul 16 21:18:41.231608 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1252/2439] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:18:40 2022] GET /compute/v2.1/servers/e7598b4a-f70c-4186-88d1-6fd0057f8656 => generated 1471 bytes in 471 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:18:41.244603 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-2e8eab00-a2a7-4def-b8cc-f606456caf64 tempest-KeyPairsV2TestJSON-30838616 tempest-KeyPairsV2TestJSON-30838616-project] Action: 'create', calling method: .version_select at 0x7fdda29fb670>, body: {"keypair": {"name": "tempest-KeyPairsV2TestJSON-keypair-2110016697", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBNOjeHUWC3f5F4r03W9NTnEFSi2147uCp5mwK/xjC7bHOg5M9GU/IPeP6VXJqJKOrbaiNs2IK+tFXG7xPejtQOVb7+2TxeAovuftkaIkVaG6l92KlvFOP4DQOgOFsd9uJw=="}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:18:41.259409 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-2e8eab00-a2a7-4def-b8cc-f606456caf64 tempest-KeyPairsV2TestJSON-30838616 tempest-KeyPairsV2TestJSON-30838616-project] Getting quotas for project 205121b2249944eb9b2e1401cf693663. Resources: {'key_pairs'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:18:41.261439 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-0ec55dbb-6aaa-45db-a422-510026f3d1dc tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] 10.222.0.16 "GET /compute/v2.1/images/detail?limit=1" status: 200 len: 1610 microversion: 2.1 time: 0.078369 Jul 16 21:18:41.262401 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1188/2440] 10.222.0.16 () {58 vars in 1157 bytes} [Sat Jul 16 21:18:41 2022] GET /compute/v2.1/images/detail?limit=1 => generated 1610 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:18:41.267496 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-4147b8b4-4579-494a-af6d-aa868e9c7589 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:41.268118 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-4147b8b4-4579-494a-af6d-aa868e9c7589 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] [instance: 6baac85b-f81c-4e3a-9f96-633cffccfd74] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:18:41.273218 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-2e8eab00-a2a7-4def-b8cc-f606456caf64 tempest-KeyPairsV2TestJSON-30838616 tempest-KeyPairsV2TestJSON-30838616-project] Getting quotas for user e848e24fe4344314a82f574a1bf1283f and project 205121b2249944eb9b2e1401cf693663. Resources: {'key_pairs'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:18:41.274707 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4147b8b4-4579-494a-af6d-aa868e9c7589 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:41.275684 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4147b8b4-4579-494a-af6d-aa868e9c7589 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:41.277086 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4147b8b4-4579-494a-af6d-aa868e9c7589 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:41.277744 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4147b8b4-4579-494a-af6d-aa868e9c7589 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:41.317894 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4147b8b4-4579-494a-af6d-aa868e9c7589 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:41.318358 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4147b8b4-4579-494a-af6d-aa868e9c7589 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:41.318824 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.objects.instance [None req-4147b8b4-4579-494a-af6d-aa868e9c7589 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lazy-loading 'fault' on Instance uuid 6baac85b-f81c-4e3a-9f96-633cffccfd74 {{(pid=138681) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:18:41.381851 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-2e8eab00-a2a7-4def-b8cc-f606456caf64 tempest-KeyPairsV2TestJSON-30838616 tempest-KeyPairsV2TestJSON-30838616-project] 10.222.0.16 "POST /compute/v2.1/os-keypairs" status: 200 len: 405 microversion: 2.1 time: 0.148410 Jul 16 21:18:41.382931 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1253/2441] 10.222.0.16 () {60 vars in 1158 bytes} [Sat Jul 16 21:18:41 2022] POST /compute/v2.1/os-keypairs => generated 405 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:18:41.388799 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-34026ff8-705a-4233-a9bf-d262376775ab tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Calling method '.version_select at 0x7fdd9cc3f940>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:41.449645 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-4147b8b4-4579-494a-af6d-aa868e9c7589 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dd4144b5388b46a989a41a2f556bb508', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a90d4bcd25ff4bf68bdc194180fbbe09', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:18:41.468705 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-4147b8b4-4579-494a-af6d-aa868e9c7589 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] 10.222.0.16 "GET /compute/v2.1/servers/6baac85b-f81c-4e3a-9f96-633cffccfd74" status: 200 len: 1308 microversion: 2.1 time: 0.197791 Jul 16 21:18:41.469559 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1189/2442] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:18:41 2022] GET /compute/v2.1/servers/6baac85b-f81c-4e3a-9f96-633cffccfd74 => generated 1308 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:18:41.481268 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-c0abeee2-1378-45ab-ac4f-f95e53b65749 tempest-KeyPairsV2TestJSON-30838616 tempest-KeyPairsV2TestJSON-30838616-project] Calling method '.version_select at 0x7fdd9d519310>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:41.489853 ncdv-master-1 devstack@n-api.service[138681]: WARNING oslo_db.sqlalchemy.utils [None req-c0abeee2-1378-45ab-ac4f-f95e53b65749 tempest-KeyPairsV2TestJSON-30838616 tempest-KeyPairsV2TestJSON-30838616-project] Unique keys not in sort_keys. The sorting order may be unstable. Jul 16 21:18:41.504751 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-c0abeee2-1378-45ab-ac4f-f95e53b65749 tempest-KeyPairsV2TestJSON-30838616 tempest-KeyPairsV2TestJSON-30838616-project] 10.222.0.16 "GET /compute/v2.1/os-keypairs" status: 200 len: 1091 microversion: 2.1 time: 0.034268 Jul 16 21:18:41.511360 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1190/2443] 10.222.0.16 () {58 vars in 1136 bytes} [Sat Jul 16 21:18:41 2022] GET /compute/v2.1/os-keypairs => generated 1091 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:18:41.519911 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-a375358b-5981-403a-ad43-0031172f6a17 tempest-KeyPairsV2TestJSON-30838616 tempest-KeyPairsV2TestJSON-30838616-project] Calling method '.version_select at 0x7fdd9d1861f0>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:41.663804 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-a375358b-5981-403a-ad43-0031172f6a17 tempest-KeyPairsV2TestJSON-30838616 tempest-KeyPairsV2TestJSON-30838616-project] 10.222.0.16 "DELETE /compute/v2.1/os-keypairs/tempest-KeyPairsV2TestJSON-keypair-2110016697" status: 202 len: 0 microversion: 2.1 time: 0.151137 Jul 16 21:18:41.664922 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1191/2444] 10.222.0.16 () {58 vars in 1277 bytes} [Sat Jul 16 21:18:41 2022] DELETE /compute/v2.1/os-keypairs/tempest-KeyPairsV2TestJSON-keypair-2110016697 => generated 0 bytes in 153 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 21:18:41.681089 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-246dd1b0-3958-436e-be49-9e9b3765ef85 tempest-KeyPairsV2TestJSON-30838616 tempest-KeyPairsV2TestJSON-30838616-project] Calling method '.version_select at 0x7fdd9d267e50>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:41.818100 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-246dd1b0-3958-436e-be49-9e9b3765ef85 tempest-KeyPairsV2TestJSON-30838616 tempest-KeyPairsV2TestJSON-30838616-project] 10.222.0.16 "DELETE /compute/v2.1/os-keypairs/tempest-KeyPairsV2TestJSON-keypair-15560015" status: 202 len: 0 microversion: 2.1 time: 0.141256 Jul 16 21:18:41.821668 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1192/2445] 10.222.0.16 () {58 vars in 1271 bytes} [Sat Jul 16 21:18:41 2022] DELETE /compute/v2.1/os-keypairs/tempest-KeyPairsV2TestJSON-keypair-15560015 => generated 0 bytes in 146 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 21:18:41.830419 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-bbdfe8be-4799-46d0-a728-68d366b95cd4 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:41.831076 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-bbdfe8be-4799-46d0-a728-68d366b95cd4 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] [instance: c88b191a-1ae5-4952-8afa-14aee5567a7f] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:18:41.837706 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-bbdfe8be-4799-46d0-a728-68d366b95cd4 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:41.838267 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-bbdfe8be-4799-46d0-a728-68d366b95cd4 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:41.839326 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-bbdfe8be-4799-46d0-a728-68d366b95cd4 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:41.839856 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-bbdfe8be-4799-46d0-a728-68d366b95cd4 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:41.995545 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-bbdfe8be-4799-46d0-a728-68d366b95cd4 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd0008fffad8e4708b2c01e03da92c47d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e353258f9c3048ef84683294e2b8579d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:18:42.006140 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-bbdfe8be-4799-46d0-a728-68d366b95cd4 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] 10.222.0.16 "GET /compute/v2.1/servers/c88b191a-1ae5-4952-8afa-14aee5567a7f" status: 200 len: 1274 microversion: 2.1 time: 0.182444 Jul 16 21:18:42.006960 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1193/2446] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:18:41 2022] GET /compute/v2.1/servers/c88b191a-1ae5-4952-8afa-14aee5567a7f => generated 1274 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:18:42.014244 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-528303b7-ba95-462d-8dc7-125865db3446 tempest-KeyPairsV2TestJSON-30838616 tempest-KeyPairsV2TestJSON-30838616-project] Calling method '.version_select at 0x7fdd9d519550>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:42.176478 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-528303b7-ba95-462d-8dc7-125865db3446 tempest-KeyPairsV2TestJSON-30838616 tempest-KeyPairsV2TestJSON-30838616-project] 10.222.0.16 "DELETE /compute/v2.1/os-keypairs/tempest-KeyPairsV2TestJSON-keypair-114969803" status: 202 len: 0 microversion: 2.1 time: 0.168203 Jul 16 21:18:42.177097 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1194/2447] 10.222.0.16 () {58 vars in 1274 bytes} [Sat Jul 16 21:18:42 2022] DELETE /compute/v2.1/os-keypairs/tempest-KeyPairsV2TestJSON-keypair-114969803 => generated 0 bytes in 169 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 21:18:42.182525 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-660def73-0a24-4a4a-aa4e-e86cef7517dd tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:42.183214 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-660def73-0a24-4a4a-aa4e-e86cef7517dd tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] [instance: d265192b-536b-4884-9fe8-74cf20fc7ef9] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:18:42.193109 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-660def73-0a24-4a4a-aa4e-e86cef7517dd tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:42.193629 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-660def73-0a24-4a4a-aa4e-e86cef7517dd tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:42.194958 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-660def73-0a24-4a4a-aa4e-e86cef7517dd tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:42.195579 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-660def73-0a24-4a4a-aa4e-e86cef7517dd tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:42.305118 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-660def73-0a24-4a4a-aa4e-e86cef7517dd tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'daf3c9e6634e4ae1bfb492eae1f44c93', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '24c64db8082648c8837c16352252f96d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:18:42.350003 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-660def73-0a24-4a4a-aa4e-e86cef7517dd tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] 10.222.0.16 "GET /compute/v2.1/servers/d265192b-536b-4884-9fe8-74cf20fc7ef9" status: 200 len: 1211 microversion: 2.1 time: 0.171846 Jul 16 21:18:42.350605 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1195/2448] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:18:42 2022] GET /compute/v2.1/servers/d265192b-536b-4884-9fe8-74cf20fc7ef9 => generated 1211 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:18:42.363857 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-807d7e52-4e3b-410e-8c83-208144a70793 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:42.366774 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-807d7e52-4e3b-410e-8c83-208144a70793 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] [instance: e7598b4a-f70c-4186-88d1-6fd0057f8656] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:18:42.380725 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-807d7e52-4e3b-410e-8c83-208144a70793 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:42.381196 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-807d7e52-4e3b-410e-8c83-208144a70793 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:42.383235 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-807d7e52-4e3b-410e-8c83-208144a70793 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:42.385940 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-807d7e52-4e3b-410e-8c83-208144a70793 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:42.417546 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=96,nova_cell1:UPDATE=3,nova_cell1:INSERT=1 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:18:42.672234 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-807d7e52-4e3b-410e-8c83-208144a70793 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f5bc1f2209f4eb186d80e385b67c774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '65d5bcfa02af4e8284ba1342197e2860', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:18:42.685506 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-807d7e52-4e3b-410e-8c83-208144a70793 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] 10.222.0.16 "GET /compute/v2.1/servers/e7598b4a-f70c-4186-88d1-6fd0057f8656" status: 200 len: 1471 microversion: 2.1 time: 0.330415 Jul 16 21:18:42.690835 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1196/2449] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:18:42 2022] GET /compute/v2.1/servers/e7598b4a-f70c-4186-88d1-6fd0057f8656 => generated 1471 bytes in 335 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:18:42.696343 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-3b24b072-daa8-4779-aba1-f68b224ae67a tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:42.697651 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-3b24b072-daa8-4779-aba1-f68b224ae67a tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] [instance: 6baac85b-f81c-4e3a-9f96-633cffccfd74] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:18:42.705659 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3b24b072-daa8-4779-aba1-f68b224ae67a tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:42.706151 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3b24b072-daa8-4779-aba1-f68b224ae67a tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:42.707178 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3b24b072-daa8-4779-aba1-f68b224ae67a tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:42.707731 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3b24b072-daa8-4779-aba1-f68b224ae67a tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:42.746768 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-3b24b072-daa8-4779-aba1-f68b224ae67a tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] HTTP exception thrown: Instance 6baac85b-f81c-4e3a-9f96-633cffccfd74 could not be found. Jul 16 21:18:42.748557 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-3b24b072-daa8-4779-aba1-f68b224ae67a tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Request method failure captured: Jul 16 21:18:42.748557 ncdv-master-1 devstack@n-api.service[138681]: request: GET /compute/v2.1/servers/6baac85b-f81c-4e3a-9f96-633cffccfd74 HTTP/1.1 Jul 16 21:18:42.748557 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:18:42.748557 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:18:42.748557 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:18:42.748557 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 21:18:42.748557 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:18:42.748557 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:18:42.748557 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:18:42.748557 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:18:42.748557 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0yqDuVl2xIz2wViVyfJejvvtUZm-mKINsHNSlOQlGCd_acdEn65QKbASWdLu3HI9dn0H56_KoirEcMTwKxd5XhFG89zgeglS6pwD7zqjxXMLIej2J5FN7Ma0YQylulb9Sa7eygzRGv-UtWjaxTP8BM9NnmzYl3XkDKlDS31Gk3wf-wE Jul 16 21:18:42.748557 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:18:42.748557 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:18:42.748557 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:18:42.748557 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:18:42.748557 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:18:42.748557 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:18:42.748557 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: a90d4bcd25ff4bf68bdc194180fbbe09 Jul 16 21:18:42.748557 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-ServerDiskConfigTestJSON-760881222 Jul 16 21:18:42.748557 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:18:42.748557 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:18:42.748557 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/a90d4bcd25ff4bf68bdc194180fbbe09"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/a90d4bcd25ff4bf68bdc194180fbbe09"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_a90d4bcd25ff4bf68bdc194180fbbe09"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/a90d4bcd25ff4bf68bdc194180fbbe09"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:18:42.748557 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-ServerDiskConfigTestJSON-760881222 Jul 16 21:18:42.748557 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: a90d4bcd25ff4bf68bdc194180fbbe09 Jul 16 21:18:42.748557 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-ServerDiskConfigTestJSON-760881222 Jul 16 21:18:42.748557 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-ServerDiskConfigTestJSON-760881222-project Jul 16 21:18:42.748557 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:18:42.748557 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:18:42.748557 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: dd4144b5388b46a989a41a2f556bb508 Jul 16 21:18:42.748557 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-ServerDiskConfigTestJSON-760881222-project Jul 16 21:18:42.748557 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:18:42.748557 ncdv-master-1 devstack@n-api.service[138681]: exception: Instance 6baac85b-f81c-4e3a-9f96-633cffccfd74 could not be found. Jul 16 21:18:42.748557 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': '6baac85b-f81c-4e3a-9f96-633cffccfd74'} Jul 16 21:18:42.748557 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:18:42.748557 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:18:42.748557 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:18:42.748557 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:18:42.748557 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:18:42.748557 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:18:42.748557 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:18:42.748557 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:18:42.748557 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:18:42.748557 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:18:42.748557 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 6baac85b-f81c-4e3a-9f96-633cffccfd74 could not be found. Jul 16 21:18:42.748557 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:18:42.748557 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:18:42.748557 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:18:42.748557 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:18:42.748557 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:18:42.748557 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:18:42.748557 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:18:42.748557 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:18:42.748557 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:18:42.748557 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:18:42.748557 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:18:42.748557 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:18:42.748557 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:18:42.748557 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:18:42.748557 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:18:42.748557 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:18:42.748557 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 6baac85b-f81c-4e3a-9f96-633cffccfd74 could not be found. Jul 16 21:18:42.748557 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:18:42.748557 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:18:42.748557 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:18:42.752148 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:18:42.752148 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:18:42.752148 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:18:42.752148 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:18:42.752148 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:18:42.752148 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 6baac85b-f81c-4e3a-9f96-633cffccfd74 could not be found. Jul 16 21:18:42.752148 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:18:42.752148 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-3b24b072-daa8-4779-aba1-f68b224ae67a tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Returning 404 to user: Instance 6baac85b-f81c-4e3a-9f96-633cffccfd74 could not be found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:18:42.752148 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-3b24b072-daa8-4779-aba1-f68b224ae67a tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] 10.222.0.16 "GET /compute/v2.1/servers/6baac85b-f81c-4e3a-9f96-633cffccfd74" status: 404 len: 111 microversion: 2.1 time: 0.057637 Jul 16 21:18:42.752148 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1197/2450] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:18:42 2022] GET /compute/v2.1/servers/6baac85b-f81c-4e3a-9f96-633cffccfd74 => generated 111 bytes in 63 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:18:42.760462 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-5410c4a7-7459-475e-a9fc-9debdad04c68 req-c6eb36f3-0c8f-4c36-a45d-e82d4e7aa23a service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "ed8dcb97-7546-45ef-9e9d-a9bb03a1b419", "name": "network-vif-deleted", "tag": "033ca6ad-5781-4719-b3b5-986dcfd7de3d"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:18:42.770181 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-5410c4a7-7459-475e-a9fc-9debdad04c68 req-c6eb36f3-0c8f-4c36-a45d-e82d4e7aa23a service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:42.770635 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-5410c4a7-7459-475e-a9fc-9debdad04c68 req-c6eb36f3-0c8f-4c36-a45d-e82d4e7aa23a service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:42.784543 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-5410c4a7-7459-475e-a9fc-9debdad04c68 req-c6eb36f3-0c8f-4c36-a45d-e82d4e7aa23a service nova] Creating event network-vif-deleted:033ca6ad-5781-4719-b3b5-986dcfd7de3d for instance ed8dcb97-7546-45ef-9e9d-a9bb03a1b419 on nch2-master-1 Jul 16 21:18:42.789922 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-5410c4a7-7459-475e-a9fc-9debdad04c68 req-c6eb36f3-0c8f-4c36-a45d-e82d4e7aa23a service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.037881 Jul 16 21:18:42.791069 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1198/2451] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:18:42 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:18:42.796594 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-7be92a1d-1afc-4ba8-ab1a-33c707156855 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:42.797379 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-7be92a1d-1afc-4ba8-ab1a-33c707156855 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] [instance: 7e4b142d-f489-4220-adc3-7b348dc14ce9] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:18:42.804673 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7be92a1d-1afc-4ba8-ab1a-33c707156855 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:42.805144 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7be92a1d-1afc-4ba8-ab1a-33c707156855 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:42.806661 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7be92a1d-1afc-4ba8-ab1a-33c707156855 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:42.807497 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7be92a1d-1afc-4ba8-ab1a-33c707156855 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:42.827965 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-7be92a1d-1afc-4ba8-ab1a-33c707156855 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] HTTP exception thrown: Instance 7e4b142d-f489-4220-adc3-7b348dc14ce9 could not be found. Jul 16 21:18:42.829693 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-7be92a1d-1afc-4ba8-ab1a-33c707156855 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Request method failure captured: Jul 16 21:18:42.829693 ncdv-master-1 devstack@n-api.service[138681]: request: DELETE /compute/v2.1/servers/7e4b142d-f489-4220-adc3-7b348dc14ce9 HTTP/1.1 Jul 16 21:18:42.829693 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:18:42.829693 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:18:42.829693 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:18:42.829693 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 21:18:42.829693 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:18:42.829693 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:18:42.829693 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:18:42.829693 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:18:42.829693 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0yqDuVl2xIz2wViVyfJejvvtUZm-mKINsHNSlOQlGCd_acdEn65QKbASWdLu3HI9dn0H56_KoirEcMTwKxd5XhFG89zgeglS6pwD7zqjxXMLIej2J5FN7Ma0YQylulb9Sa7eygzRGv-UtWjaxTP8BM9NnmzYl3XkDKlDS31Gk3wf-wE Jul 16 21:18:42.829693 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:18:42.829693 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:18:42.829693 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:18:42.829693 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:18:42.829693 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:18:42.829693 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:18:42.829693 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: a90d4bcd25ff4bf68bdc194180fbbe09 Jul 16 21:18:42.829693 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-ServerDiskConfigTestJSON-760881222 Jul 16 21:18:42.829693 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:18:42.829693 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:18:42.829693 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/a90d4bcd25ff4bf68bdc194180fbbe09"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/a90d4bcd25ff4bf68bdc194180fbbe09"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_a90d4bcd25ff4bf68bdc194180fbbe09"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/a90d4bcd25ff4bf68bdc194180fbbe09"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:18:42.829693 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-ServerDiskConfigTestJSON-760881222 Jul 16 21:18:42.829693 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: a90d4bcd25ff4bf68bdc194180fbbe09 Jul 16 21:18:42.829693 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-ServerDiskConfigTestJSON-760881222 Jul 16 21:18:42.829693 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-ServerDiskConfigTestJSON-760881222-project Jul 16 21:18:42.829693 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:18:42.829693 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:18:42.829693 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: dd4144b5388b46a989a41a2f556bb508 Jul 16 21:18:42.829693 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-ServerDiskConfigTestJSON-760881222-project Jul 16 21:18:42.829693 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:18:42.829693 ncdv-master-1 devstack@n-api.service[138681]: exception: Instance 7e4b142d-f489-4220-adc3-7b348dc14ce9 could not be found. Jul 16 21:18:42.829693 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': '7e4b142d-f489-4220-adc3-7b348dc14ce9'} Jul 16 21:18:42.829693 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:18:42.829693 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:18:42.829693 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:18:42.829693 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:18:42.829693 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:18:42.829693 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:18:42.829693 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:18:42.829693 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:18:42.829693 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:18:42.829693 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:18:42.829693 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 7e4b142d-f489-4220-adc3-7b348dc14ce9 could not be found. Jul 16 21:18:42.829693 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:18:42.829693 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:18:42.829693 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:18:42.829693 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:18:42.829693 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:18:42.829693 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:18:42.829693 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:18:42.829693 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:18:42.829693 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:18:42.829693 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:18:42.829693 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Jul 16 21:18:42.829693 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Jul 16 21:18:42.829693 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Jul 16 21:18:42.829693 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Jul 16 21:18:42.829693 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:18:42.829693 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:18:42.829693 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:18:42.829693 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:18:42.833050 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 7e4b142d-f489-4220-adc3-7b348dc14ce9 could not be found. Jul 16 21:18:42.833050 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:18:42.833050 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:18:42.833050 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:18:42.833050 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:18:42.833050 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:18:42.833050 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:18:42.833050 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:18:42.833050 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:18:42.833050 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 7e4b142d-f489-4220-adc3-7b348dc14ce9 could not be found. Jul 16 21:18:42.833050 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:18:42.833050 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-7be92a1d-1afc-4ba8-ab1a-33c707156855 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Returning 404 to user: Instance 7e4b142d-f489-4220-adc3-7b348dc14ce9 could not be found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:18:42.839379 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-7be92a1d-1afc-4ba8-ab1a-33c707156855 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] 10.222.0.16 "DELETE /compute/v2.1/servers/7e4b142d-f489-4220-adc3-7b348dc14ce9" status: 404 len: 111 microversion: 2.1 time: 0.046716 Jul 16 21:18:42.840266 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1199/2452] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:18:42 2022] DELETE /compute/v2.1/servers/7e4b142d-f489-4220-adc3-7b348dc14ce9 => generated 111 bytes in 48 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:18:42.849031 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-df50abca-549d-4164-952f-081be6a2d395 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:42.849979 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-df50abca-549d-4164-952f-081be6a2d395 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] [instance: 7e4b142d-f489-4220-adc3-7b348dc14ce9] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:18:42.859255 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-df50abca-549d-4164-952f-081be6a2d395 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:42.859255 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-df50abca-549d-4164-952f-081be6a2d395 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:42.859255 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-df50abca-549d-4164-952f-081be6a2d395 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:42.859255 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-df50abca-549d-4164-952f-081be6a2d395 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:42.883653 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-df50abca-549d-4164-952f-081be6a2d395 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] HTTP exception thrown: Instance 7e4b142d-f489-4220-adc3-7b348dc14ce9 could not be found. Jul 16 21:18:42.886271 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-df50abca-549d-4164-952f-081be6a2d395 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Request method failure captured: Jul 16 21:18:42.886271 ncdv-master-1 devstack@n-api.service[138681]: request: GET /compute/v2.1/servers/7e4b142d-f489-4220-adc3-7b348dc14ce9 HTTP/1.1 Jul 16 21:18:42.886271 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:18:42.886271 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:18:42.886271 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:18:42.886271 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 21:18:42.886271 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:18:42.886271 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:18:42.886271 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:18:42.886271 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:18:42.886271 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0yqDuVl2xIz2wViVyfJejvvtUZm-mKINsHNSlOQlGCd_acdEn65QKbASWdLu3HI9dn0H56_KoirEcMTwKxd5XhFG89zgeglS6pwD7zqjxXMLIej2J5FN7Ma0YQylulb9Sa7eygzRGv-UtWjaxTP8BM9NnmzYl3XkDKlDS31Gk3wf-wE Jul 16 21:18:42.886271 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:18:42.886271 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:18:42.886271 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:18:42.886271 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:18:42.886271 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:18:42.886271 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:18:42.886271 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: a90d4bcd25ff4bf68bdc194180fbbe09 Jul 16 21:18:42.886271 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-ServerDiskConfigTestJSON-760881222 Jul 16 21:18:42.886271 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:18:42.886271 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:18:42.886271 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/a90d4bcd25ff4bf68bdc194180fbbe09"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/a90d4bcd25ff4bf68bdc194180fbbe09"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_a90d4bcd25ff4bf68bdc194180fbbe09"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/a90d4bcd25ff4bf68bdc194180fbbe09"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:18:42.886271 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-ServerDiskConfigTestJSON-760881222 Jul 16 21:18:42.886271 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: a90d4bcd25ff4bf68bdc194180fbbe09 Jul 16 21:18:42.886271 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-ServerDiskConfigTestJSON-760881222 Jul 16 21:18:42.886271 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-ServerDiskConfigTestJSON-760881222-project Jul 16 21:18:42.886271 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:18:42.886271 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:18:42.886271 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: dd4144b5388b46a989a41a2f556bb508 Jul 16 21:18:42.886271 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-ServerDiskConfigTestJSON-760881222-project Jul 16 21:18:42.886271 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:18:42.886271 ncdv-master-1 devstack@n-api.service[138681]: exception: Instance 7e4b142d-f489-4220-adc3-7b348dc14ce9 could not be found. Jul 16 21:18:42.886271 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': '7e4b142d-f489-4220-adc3-7b348dc14ce9'} Jul 16 21:18:42.886271 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:18:42.886271 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:18:42.886271 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:18:42.886271 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:18:42.886271 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:18:42.886271 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:18:42.886271 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:18:42.886271 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:18:42.886271 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:18:42.886271 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:18:42.886271 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 7e4b142d-f489-4220-adc3-7b348dc14ce9 could not be found. Jul 16 21:18:42.886271 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:18:42.886271 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:18:42.886271 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:18:42.886271 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:18:42.886271 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:18:42.886271 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:18:42.886271 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:18:42.886271 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:18:42.886271 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:18:42.886271 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:18:42.886271 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:18:42.886271 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:18:42.886271 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:18:42.886271 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:18:42.886271 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:18:42.886271 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:18:42.886271 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 7e4b142d-f489-4220-adc3-7b348dc14ce9 could not be found. Jul 16 21:18:42.886271 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:18:42.886271 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:18:42.886271 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:18:42.889833 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:18:42.889833 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:18:42.889833 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:18:42.889833 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:18:42.889833 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:18:42.889833 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 7e4b142d-f489-4220-adc3-7b348dc14ce9 could not be found. Jul 16 21:18:42.889833 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:18:42.889833 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-df50abca-549d-4164-952f-081be6a2d395 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Returning 404 to user: Instance 7e4b142d-f489-4220-adc3-7b348dc14ce9 could not be found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:18:42.889833 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-df50abca-549d-4164-952f-081be6a2d395 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] 10.222.0.16 "GET /compute/v2.1/servers/7e4b142d-f489-4220-adc3-7b348dc14ce9" status: 404 len: 111 microversion: 2.1 time: 0.042659 Jul 16 21:18:42.889833 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1200/2453] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:18:42 2022] GET /compute/v2.1/servers/7e4b142d-f489-4220-adc3-7b348dc14ce9 => generated 111 bytes in 44 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Jul 16 21:18:42.896550 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-909a5afb-0a53-4193-9928-6b01f3600497 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:42.897290 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-909a5afb-0a53-4193-9928-6b01f3600497 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] [instance: 11194822-5d42-45f4-99d0-88c70c808ddd] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:18:42.906513 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-909a5afb-0a53-4193-9928-6b01f3600497 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:42.917118 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-909a5afb-0a53-4193-9928-6b01f3600497 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:42.918412 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-909a5afb-0a53-4193-9928-6b01f3600497 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:42.919526 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-909a5afb-0a53-4193-9928-6b01f3600497 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:42.936189 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-34026ff8-705a-4233-a9bf-d262376775ab tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] 10.222.0.16 "DELETE /compute/v2.1/images/04088d67-f393-4fbb-8c56-7c2bc534a560" status: 204 len: 0 microversion: 2.1 time: 1.551987 Jul 16 21:18:42.936776 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1254/2454] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:18:41 2022] DELETE /compute/v2.1/images/04088d67-f393-4fbb-8c56-7c2bc534a560 => generated 0 bytes in 1553 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 16 21:18:42.944613 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-278fc5be-a16c-4876-92b7-a5a6ff118daa tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Calling method '.version_select at 0x7fdd9d321d30>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:42.979749 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-909a5afb-0a53-4193-9928-6b01f3600497 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] HTTP exception thrown: Instance 11194822-5d42-45f4-99d0-88c70c808ddd could not be found. Jul 16 21:18:42.982127 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-909a5afb-0a53-4193-9928-6b01f3600497 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Request method failure captured: Jul 16 21:18:42.982127 ncdv-master-1 devstack@n-api.service[138681]: request: DELETE /compute/v2.1/servers/11194822-5d42-45f4-99d0-88c70c808ddd HTTP/1.1 Jul 16 21:18:42.982127 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:18:42.982127 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:18:42.982127 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:18:42.982127 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 21:18:42.982127 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:18:42.982127 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:18:42.982127 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:18:42.982127 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:18:42.982127 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0yqDuVl2xIz2wViVyfJejvvtUZm-mKINsHNSlOQlGCd_acdEn65QKbASWdLu3HI9dn0H56_KoirEcMTwKxd5XhFG89zgeglS6pwD7zqjxXMLIej2J5FN7Ma0YQylulb9Sa7eygzRGv-UtWjaxTP8BM9NnmzYl3XkDKlDS31Gk3wf-wE Jul 16 21:18:42.982127 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:18:42.982127 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:18:42.982127 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:18:42.982127 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:18:42.982127 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:18:42.982127 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:18:42.982127 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: a90d4bcd25ff4bf68bdc194180fbbe09 Jul 16 21:18:42.982127 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-ServerDiskConfigTestJSON-760881222 Jul 16 21:18:42.982127 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:18:42.982127 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:18:42.982127 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/a90d4bcd25ff4bf68bdc194180fbbe09"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/a90d4bcd25ff4bf68bdc194180fbbe09"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_a90d4bcd25ff4bf68bdc194180fbbe09"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/a90d4bcd25ff4bf68bdc194180fbbe09"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:18:42.982127 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-ServerDiskConfigTestJSON-760881222 Jul 16 21:18:42.982127 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: a90d4bcd25ff4bf68bdc194180fbbe09 Jul 16 21:18:42.982127 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-ServerDiskConfigTestJSON-760881222 Jul 16 21:18:42.982127 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-ServerDiskConfigTestJSON-760881222-project Jul 16 21:18:42.982127 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:18:42.982127 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:18:42.982127 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: dd4144b5388b46a989a41a2f556bb508 Jul 16 21:18:42.982127 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-ServerDiskConfigTestJSON-760881222-project Jul 16 21:18:42.982127 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:18:42.982127 ncdv-master-1 devstack@n-api.service[138681]: exception: Instance 11194822-5d42-45f4-99d0-88c70c808ddd could not be found. Jul 16 21:18:42.982127 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': '11194822-5d42-45f4-99d0-88c70c808ddd'} Jul 16 21:18:42.982127 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:18:42.982127 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:18:42.982127 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:18:42.982127 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:18:42.982127 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:18:42.982127 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:18:42.982127 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:18:42.982127 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:18:42.982127 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:18:42.982127 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:18:42.982127 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 11194822-5d42-45f4-99d0-88c70c808ddd could not be found. Jul 16 21:18:42.982127 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:18:42.982127 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:18:42.982127 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:18:42.982127 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:18:42.982127 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:18:42.982127 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:18:42.982127 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:18:42.982127 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:18:42.982127 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:18:42.982127 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:18:42.982127 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Jul 16 21:18:42.982127 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Jul 16 21:18:42.982127 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Jul 16 21:18:42.982127 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Jul 16 21:18:42.982127 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:18:42.982127 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:18:42.982127 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:18:42.982127 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:18:42.992275 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 11194822-5d42-45f4-99d0-88c70c808ddd could not be found. Jul 16 21:18:42.992275 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:18:42.992275 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:18:42.992275 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:18:42.992275 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:18:42.992275 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:18:42.992275 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:18:42.992275 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:18:42.992275 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:18:42.992275 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 11194822-5d42-45f4-99d0-88c70c808ddd could not be found. Jul 16 21:18:42.992275 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:18:42.992275 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-909a5afb-0a53-4193-9928-6b01f3600497 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Returning 404 to user: Instance 11194822-5d42-45f4-99d0-88c70c808ddd could not be found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:18:42.992275 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-909a5afb-0a53-4193-9928-6b01f3600497 tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] 10.222.0.16 "DELETE /compute/v2.1/servers/11194822-5d42-45f4-99d0-88c70c808ddd" status: 404 len: 111 microversion: 2.1 time: 0.092859 Jul 16 21:18:42.994508 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1201/2455] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:18:42 2022] DELETE /compute/v2.1/servers/11194822-5d42-45f4-99d0-88c70c808ddd => generated 111 bytes in 102 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Jul 16 21:18:43.000128 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-d8103044-041c-4cae-a99c-64db8d4d892a tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:43.000758 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-d8103044-041c-4cae-a99c-64db8d4d892a tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] [instance: 11194822-5d42-45f4-99d0-88c70c808ddd] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:18:43.003044 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell0:SELECT=4 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:18:43.008544 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d8103044-041c-4cae-a99c-64db8d4d892a tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:43.009005 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d8103044-041c-4cae-a99c-64db8d4d892a tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:43.010047 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d8103044-041c-4cae-a99c-64db8d4d892a tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:43.010717 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d8103044-041c-4cae-a99c-64db8d4d892a tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:43.031666 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-d8103044-041c-4cae-a99c-64db8d4d892a tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] HTTP exception thrown: Instance 11194822-5d42-45f4-99d0-88c70c808ddd could not be found. Jul 16 21:18:43.033674 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-d8103044-041c-4cae-a99c-64db8d4d892a tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Request method failure captured: Jul 16 21:18:43.033674 ncdv-master-1 devstack@n-api.service[138681]: request: GET /compute/v2.1/servers/11194822-5d42-45f4-99d0-88c70c808ddd HTTP/1.1 Jul 16 21:18:43.033674 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:18:43.033674 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:18:43.033674 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:18:43.033674 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 21:18:43.033674 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:18:43.033674 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:18:43.033674 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:18:43.033674 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:18:43.033674 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0yqDuVl2xIz2wViVyfJejvvtUZm-mKINsHNSlOQlGCd_acdEn65QKbASWdLu3HI9dn0H56_KoirEcMTwKxd5XhFG89zgeglS6pwD7zqjxXMLIej2J5FN7Ma0YQylulb9Sa7eygzRGv-UtWjaxTP8BM9NnmzYl3XkDKlDS31Gk3wf-wE Jul 16 21:18:43.033674 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:18:43.033674 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:18:43.033674 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:18:43.033674 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:18:43.033674 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:18:43.033674 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:18:43.033674 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: a90d4bcd25ff4bf68bdc194180fbbe09 Jul 16 21:18:43.033674 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-ServerDiskConfigTestJSON-760881222 Jul 16 21:18:43.033674 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:18:43.033674 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:18:43.033674 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/a90d4bcd25ff4bf68bdc194180fbbe09"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/a90d4bcd25ff4bf68bdc194180fbbe09"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_a90d4bcd25ff4bf68bdc194180fbbe09"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/a90d4bcd25ff4bf68bdc194180fbbe09"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:18:43.033674 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-ServerDiskConfigTestJSON-760881222 Jul 16 21:18:43.033674 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: a90d4bcd25ff4bf68bdc194180fbbe09 Jul 16 21:18:43.033674 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-ServerDiskConfigTestJSON-760881222 Jul 16 21:18:43.033674 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-ServerDiskConfigTestJSON-760881222-project Jul 16 21:18:43.033674 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:18:43.033674 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:18:43.033674 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: dd4144b5388b46a989a41a2f556bb508 Jul 16 21:18:43.033674 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-ServerDiskConfigTestJSON-760881222-project Jul 16 21:18:43.033674 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:18:43.033674 ncdv-master-1 devstack@n-api.service[138681]: exception: Instance 11194822-5d42-45f4-99d0-88c70c808ddd could not be found. Jul 16 21:18:43.033674 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': '11194822-5d42-45f4-99d0-88c70c808ddd'} Jul 16 21:18:43.033674 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:18:43.033674 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:18:43.033674 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:18:43.033674 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:18:43.033674 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:18:43.033674 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:18:43.033674 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:18:43.033674 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:18:43.033674 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:18:43.033674 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:18:43.033674 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 11194822-5d42-45f4-99d0-88c70c808ddd could not be found. Jul 16 21:18:43.033674 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:18:43.033674 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:18:43.033674 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:18:43.033674 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:18:43.033674 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:18:43.033674 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:18:43.033674 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:18:43.033674 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:18:43.033674 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:18:43.033674 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:18:43.033674 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:18:43.033674 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:18:43.033674 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:18:43.033674 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:18:43.033674 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:18:43.033674 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:18:43.033674 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 11194822-5d42-45f4-99d0-88c70c808ddd could not be found. Jul 16 21:18:43.033674 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:18:43.033674 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:18:43.033674 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:18:43.037074 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:18:43.037074 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:18:43.037074 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:18:43.037074 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:18:43.037074 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:18:43.037074 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 11194822-5d42-45f4-99d0-88c70c808ddd could not be found. Jul 16 21:18:43.037074 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:18:43.039645 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-d8103044-041c-4cae-a99c-64db8d4d892a tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] Returning 404 to user: Instance 11194822-5d42-45f4-99d0-88c70c808ddd could not be found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:18:43.041435 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-d8103044-041c-4cae-a99c-64db8d4d892a tempest-ServerDiskConfigTestJSON-760881222 tempest-ServerDiskConfigTestJSON-760881222-project] 10.222.0.16 "GET /compute/v2.1/servers/11194822-5d42-45f4-99d0-88c70c808ddd" status: 404 len: 111 microversion: 2.1 time: 0.044947 Jul 16 21:18:43.042207 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1202/2456] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:18:42 2022] GET /compute/v2.1/servers/11194822-5d42-45f4-99d0-88c70c808ddd => generated 111 bytes in 47 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:18:43.047723 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-5f95dfd8-cb24-48b7-89ba-00540b7ff460 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:43.048598 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-5f95dfd8-cb24-48b7-89ba-00540b7ff460 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] [instance: c88b191a-1ae5-4952-8afa-14aee5567a7f] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:18:43.058683 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5f95dfd8-cb24-48b7-89ba-00540b7ff460 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:43.059516 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5f95dfd8-cb24-48b7-89ba-00540b7ff460 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:43.060525 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5f95dfd8-cb24-48b7-89ba-00540b7ff460 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:43.061184 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5f95dfd8-cb24-48b7-89ba-00540b7ff460 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:43.368568 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-5f95dfd8-cb24-48b7-89ba-00540b7ff460 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd0008fffad8e4708b2c01e03da92c47d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e353258f9c3048ef84683294e2b8579d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:18:43.377334 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-5f95dfd8-cb24-48b7-89ba-00540b7ff460 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] 10.222.0.16 "GET /compute/v2.1/servers/c88b191a-1ae5-4952-8afa-14aee5567a7f" status: 200 len: 1316 microversion: 2.1 time: 0.333918 Jul 16 21:18:43.391747 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1203/2457] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:18:43 2022] GET /compute/v2.1/servers/c88b191a-1ae5-4952-8afa-14aee5567a7f => generated 1316 bytes in 349 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:18:43.398509 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-944ecb03-05f8-4aa5-a31b-412c63604e5e tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:43.401103 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-944ecb03-05f8-4aa5-a31b-412c63604e5e tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] [instance: d265192b-536b-4884-9fe8-74cf20fc7ef9] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:18:43.419030 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-944ecb03-05f8-4aa5-a31b-412c63604e5e tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:43.424286 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-944ecb03-05f8-4aa5-a31b-412c63604e5e tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:43.424286 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-944ecb03-05f8-4aa5-a31b-412c63604e5e tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:43.424286 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-944ecb03-05f8-4aa5-a31b-412c63604e5e tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:43.640548 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-944ecb03-05f8-4aa5-a31b-412c63604e5e tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'daf3c9e6634e4ae1bfb492eae1f44c93', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '24c64db8082648c8837c16352252f96d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:18:43.655322 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-944ecb03-05f8-4aa5-a31b-412c63604e5e tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] 10.222.0.16 "GET /compute/v2.1/servers/d265192b-536b-4884-9fe8-74cf20fc7ef9" status: 200 len: 1259 microversion: 2.1 time: 0.260654 Jul 16 21:18:43.658288 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1204/2458] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:18:43 2022] GET /compute/v2.1/servers/d265192b-536b-4884-9fe8-74cf20fc7ef9 => generated 1259 bytes in 266 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:18:43.704534 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-bbb72dc4-483a-42ee-8174-42fc31195e1a tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:43.705403 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-bbb72dc4-483a-42ee-8174-42fc31195e1a tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] [instance: e7598b4a-f70c-4186-88d1-6fd0057f8656] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:18:43.716900 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-bbb72dc4-483a-42ee-8174-42fc31195e1a tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:43.717613 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-bbb72dc4-483a-42ee-8174-42fc31195e1a tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:43.718637 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-bbb72dc4-483a-42ee-8174-42fc31195e1a tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:43.719264 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-bbb72dc4-483a-42ee-8174-42fc31195e1a tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:44.411839 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-bbb72dc4-483a-42ee-8174-42fc31195e1a tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f5bc1f2209f4eb186d80e385b67c774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '65d5bcfa02af4e8284ba1342197e2860', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:18:44.422045 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-bbb72dc4-483a-42ee-8174-42fc31195e1a tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] 10.222.0.16 "GET /compute/v2.1/servers/e7598b4a-f70c-4186-88d1-6fd0057f8656" status: 200 len: 1471 microversion: 2.1 time: 0.721976 Jul 16 21:18:44.423393 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1205/2459] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:18:43 2022] GET /compute/v2.1/servers/e7598b4a-f70c-4186-88d1-6fd0057f8656 => generated 1471 bytes in 724 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:18:44.430093 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-06c1fc49-3455-47a1-8413-09594e856520 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:44.431106 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-06c1fc49-3455-47a1-8413-09594e856520 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] [instance: c88b191a-1ae5-4952-8afa-14aee5567a7f] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:18:44.440858 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-06c1fc49-3455-47a1-8413-09594e856520 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:44.441576 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-06c1fc49-3455-47a1-8413-09594e856520 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:44.442834 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-06c1fc49-3455-47a1-8413-09594e856520 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:44.444662 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-06c1fc49-3455-47a1-8413-09594e856520 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:44.910987 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-06c1fc49-3455-47a1-8413-09594e856520 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd0008fffad8e4708b2c01e03da92c47d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e353258f9c3048ef84683294e2b8579d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:18:44.925831 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-06c1fc49-3455-47a1-8413-09594e856520 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] 10.222.0.16 "GET /compute/v2.1/servers/c88b191a-1ae5-4952-8afa-14aee5567a7f" status: 200 len: 1316 microversion: 2.1 time: 0.501069 Jul 16 21:18:44.936392 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1206/2460] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:18:44 2022] GET /compute/v2.1/servers/c88b191a-1ae5-4952-8afa-14aee5567a7f => generated 1316 bytes in 512 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:18:44.941695 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-0ebcc78d-386f-45cc-84f1-9aeced46da7f tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:44.948622 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-0ebcc78d-386f-45cc-84f1-9aeced46da7f tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] [instance: d265192b-536b-4884-9fe8-74cf20fc7ef9] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:18:44.960225 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0ebcc78d-386f-45cc-84f1-9aeced46da7f tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:44.964528 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0ebcc78d-386f-45cc-84f1-9aeced46da7f tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:44.966255 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0ebcc78d-386f-45cc-84f1-9aeced46da7f tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:44.966782 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0ebcc78d-386f-45cc-84f1-9aeced46da7f tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:45.052885 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-0ebcc78d-386f-45cc-84f1-9aeced46da7f tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'daf3c9e6634e4ae1bfb492eae1f44c93', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '24c64db8082648c8837c16352252f96d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:18:45.063215 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-0ebcc78d-386f-45cc-84f1-9aeced46da7f tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] 10.222.0.16 "GET /compute/v2.1/servers/d265192b-536b-4884-9fe8-74cf20fc7ef9" status: 200 len: 1265 microversion: 2.1 time: 0.125362 Jul 16 21:18:45.063965 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1207/2461] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:18:44 2022] GET /compute/v2.1/servers/d265192b-536b-4884-9fe8-74cf20fc7ef9 => generated 1265 bytes in 127 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:18:45.187278 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-278fc5be-a16c-4876-92b7-a5a6ff118daa tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] 10.222.0.16 "DELETE /compute/v2.1/images/1f9a9864-79f1-4d21-84c6-8d017682af53" status: 204 len: 0 microversion: 2.1 time: 2.246383 Jul 16 21:18:45.187869 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1255/2462] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:18:42 2022] DELETE /compute/v2.1/images/1f9a9864-79f1-4d21-84c6-8d017682af53 => generated 0 bytes in 2248 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Jul 16 21:18:45.196948 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-f712ae01-e054-476c-be9a-0603a940e2b9 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Calling method '.version_select at 0x7fdd9cda2940>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:45.440097 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-2ca06e04-1576-4ada-a0b3-aa36f13498f0 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:45.441062 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-2ca06e04-1576-4ada-a0b3-aa36f13498f0 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] [instance: e7598b4a-f70c-4186-88d1-6fd0057f8656] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:18:45.449409 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2ca06e04-1576-4ada-a0b3-aa36f13498f0 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:45.450397 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2ca06e04-1576-4ada-a0b3-aa36f13498f0 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:45.451630 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2ca06e04-1576-4ada-a0b3-aa36f13498f0 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:45.452319 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2ca06e04-1576-4ada-a0b3-aa36f13498f0 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:45.847107 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-2ca06e04-1576-4ada-a0b3-aa36f13498f0 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f5bc1f2209f4eb186d80e385b67c774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '65d5bcfa02af4e8284ba1342197e2860', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:18:45.857345 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-2ca06e04-1576-4ada-a0b3-aa36f13498f0 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] 10.222.0.16 "GET /compute/v2.1/servers/e7598b4a-f70c-4186-88d1-6fd0057f8656" status: 200 len: 1471 microversion: 2.1 time: 0.421612 Jul 16 21:18:45.858342 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1256/2463] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:18:45 2022] GET /compute/v2.1/servers/e7598b4a-f70c-4186-88d1-6fd0057f8656 => generated 1471 bytes in 423 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:18:45.864845 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-ebed7206-4bff-44fa-9541-58c4e3cc7203 req-5a9c9d98-410d-4262-a773-1b79d66ebfb0 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "c88b191a-1ae5-4952-8afa-14aee5567a7f", "tag": "9b680cff-b0fe-4eb6-91dd-54ab2451c6f2"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:18:45.887287 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-ebed7206-4bff-44fa-9541-58c4e3cc7203 req-5a9c9d98-410d-4262-a773-1b79d66ebfb0 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:45.887998 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-ebed7206-4bff-44fa-9541-58c4e3cc7203 req-5a9c9d98-410d-4262-a773-1b79d66ebfb0 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:45.950764 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-ebed7206-4bff-44fa-9541-58c4e3cc7203 req-5a9c9d98-410d-4262-a773-1b79d66ebfb0 service nova] Creating event network-changed:9b680cff-b0fe-4eb6-91dd-54ab2451c6f2 for instance c88b191a-1ae5-4952-8afa-14aee5567a7f on nch1-master-1 Jul 16 21:18:45.960051 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-ebed7206-4bff-44fa-9541-58c4e3cc7203 req-5a9c9d98-410d-4262-a773-1b79d66ebfb0 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.100150 Jul 16 21:18:45.964319 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1257/2464] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:18:45 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:18:45.980610 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-9bb01dde-84a8-401b-b123-1e6eb57a8476 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:45.981342 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-9bb01dde-84a8-401b-b123-1e6eb57a8476 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] [instance: c88b191a-1ae5-4952-8afa-14aee5567a7f] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:18:45.988006 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9bb01dde-84a8-401b-b123-1e6eb57a8476 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:45.997094 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9bb01dde-84a8-401b-b123-1e6eb57a8476 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:45.998099 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9bb01dde-84a8-401b-b123-1e6eb57a8476 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:45.998562 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9bb01dde-84a8-401b-b123-1e6eb57a8476 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:46.840029 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-9bb01dde-84a8-401b-b123-1e6eb57a8476 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd0008fffad8e4708b2c01e03da92c47d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e353258f9c3048ef84683294e2b8579d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:18:46.849439 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-9bb01dde-84a8-401b-b123-1e6eb57a8476 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] 10.222.0.16 "GET /compute/v2.1/servers/c88b191a-1ae5-4952-8afa-14aee5567a7f" status: 200 len: 1316 microversion: 2.1 time: 0.883846 Jul 16 21:18:46.851452 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1258/2465] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:18:45 2022] GET /compute/v2.1/servers/c88b191a-1ae5-4952-8afa-14aee5567a7f => generated 1316 bytes in 887 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:18:46.877332 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-43b27d7e-3ff9-4811-8c37-e6382b385ddc tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:46.883954 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-43b27d7e-3ff9-4811-8c37-e6382b385ddc tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] [instance: d265192b-536b-4884-9fe8-74cf20fc7ef9] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:18:46.895672 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-43b27d7e-3ff9-4811-8c37-e6382b385ddc tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:46.896502 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-43b27d7e-3ff9-4811-8c37-e6382b385ddc tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:46.897503 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-43b27d7e-3ff9-4811-8c37-e6382b385ddc tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:46.900991 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-43b27d7e-3ff9-4811-8c37-e6382b385ddc tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:47.143999 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-f712ae01-e054-476c-be9a-0603a940e2b9 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] 10.222.0.16 "DELETE /compute/v2.1/images/245dff72-cf22-471e-9dc0-0e7271c7ed43" status: 204 len: 0 microversion: 2.1 time: 1.950680 Jul 16 21:18:47.148852 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1208/2466] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:18:45 2022] DELETE /compute/v2.1/images/245dff72-cf22-471e-9dc0-0e7271c7ed43 => generated 0 bytes in 1953 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 16 21:18:47.306914 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-28d76587-17e6-4603-9295-eecc991ebd5b tempest-KeyPairsV22TestJSON-865907648 tempest-KeyPairsV22TestJSON-865907648-project] Action: 'create', calling method: .version_select at 0x7fdd9d0e24c0>, body: {"keypair": {"name": "tempest-keypair-295988021", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBAVGn+BSYiWwS3CmzTyyyvfSWKYxcfvti4cPiLYi1VbIWB4ba6RT+rf6TfKhQZG+JZ3T4/81YdApxcJaCiEdc4THRXp2Zv69U7xPOax55nl6kyE9YnJeEWwpOgAePaCclQ=="}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:18:47.331809 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-28d76587-17e6-4603-9295-eecc991ebd5b tempest-KeyPairsV22TestJSON-865907648 tempest-KeyPairsV22TestJSON-865907648-project] Getting quotas for project 50e54e839e35463898c9c47af127a018. Resources: {'key_pairs'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:18:47.337774 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-28d76587-17e6-4603-9295-eecc991ebd5b tempest-KeyPairsV22TestJSON-865907648 tempest-KeyPairsV22TestJSON-865907648-project] Getting quotas for user ee0b62d71e9b4610b77eedd8f9d3d0a0 and project 50e54e839e35463898c9c47af127a018. Resources: {'key_pairs'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:18:47.437845 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-28d76587-17e6-4603-9295-eecc991ebd5b tempest-KeyPairsV22TestJSON-865907648 tempest-KeyPairsV22TestJSON-865907648-project] 10.222.0.16 "POST /compute/v2.1/os-keypairs" status: 201 len: 400 microversion: 2.2 time: 0.286447 Jul 16 21:18:47.445679 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1209/2467] 10.222.0.16 () {62 vars in 1198 bytes} [Sat Jul 16 21:18:47 2022] POST /compute/v2.1/os-keypairs => generated 400 bytes in 294 msecs (HTTP/1.1 201) 9 headers in 362 bytes (1 switches on core 0) Jul 16 21:18:47.453152 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-3a3c361f-913f-4d5d-9c22-33f5af1caaf2 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:47.454811 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-3a3c361f-913f-4d5d-9c22-33f5af1caaf2 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] [instance: e7598b4a-f70c-4186-88d1-6fd0057f8656] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:18:47.466784 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3a3c361f-913f-4d5d-9c22-33f5af1caaf2 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:47.468537 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3a3c361f-913f-4d5d-9c22-33f5af1caaf2 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:47.469421 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3a3c361f-913f-4d5d-9c22-33f5af1caaf2 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:47.470181 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3a3c361f-913f-4d5d-9c22-33f5af1caaf2 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:48.018822 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-43b27d7e-3ff9-4811-8c37-e6382b385ddc tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'daf3c9e6634e4ae1bfb492eae1f44c93', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '24c64db8082648c8837c16352252f96d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:18:48.027852 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-43b27d7e-3ff9-4811-8c37-e6382b385ddc tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] 10.222.0.16 "GET /compute/v2.1/servers/d265192b-536b-4884-9fe8-74cf20fc7ef9" status: 200 len: 1307 microversion: 2.1 time: 1.162670 Jul 16 21:18:48.028437 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1259/2468] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:18:46 2022] GET /compute/v2.1/servers/d265192b-536b-4884-9fe8-74cf20fc7ef9 => generated 1307 bytes in 1176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:18:48.043420 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-2437c72d-e562-44f4-81e2-bc09ee61ba5d tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:48.052048 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-2437c72d-e562-44f4-81e2-bc09ee61ba5d tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] [instance: 6cb4e66a-eba2-42b7-91e8-09ee7b2a5a4f] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:18:48.052048 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2437c72d-e562-44f4-81e2-bc09ee61ba5d tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:48.052048 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2437c72d-e562-44f4-81e2-bc09ee61ba5d tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:48.052583 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2437c72d-e562-44f4-81e2-bc09ee61ba5d tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:48.053205 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2437c72d-e562-44f4-81e2-bc09ee61ba5d tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:48.076289 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=96,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:18:48.086891 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-2437c72d-e562-44f4-81e2-bc09ee61ba5d tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] [instance: 6cb4e66a-eba2-42b7-91e8-09ee7b2a5a4f] Going to try to terminate instance {{(pid=138680) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 16 21:18:48.095308 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-3a3c361f-913f-4d5d-9c22-33f5af1caaf2 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f5bc1f2209f4eb186d80e385b67c774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '65d5bcfa02af4e8284ba1342197e2860', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:18:48.106518 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-3a3c361f-913f-4d5d-9c22-33f5af1caaf2 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] 10.222.0.16 "GET /compute/v2.1/servers/e7598b4a-f70c-4186-88d1-6fd0057f8656" status: 200 len: 1471 microversion: 2.1 time: 0.661197 Jul 16 21:18:48.107454 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1210/2469] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:18:47 2022] GET /compute/v2.1/servers/e7598b4a-f70c-4186-88d1-6fd0057f8656 => generated 1471 bytes in 663 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:18:48.113107 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-ad3e14d7-db82-42a4-9b24-2c5acbfc41b9 tempest-KeyPairsV22TestJSON-865907648 tempest-KeyPairsV22TestJSON-865907648-project] Calling method '.version_select at 0x7fdd9d233790>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:48.122032 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-ad3e14d7-db82-42a4-9b24-2c5acbfc41b9 tempest-KeyPairsV22TestJSON-865907648 tempest-KeyPairsV22TestJSON-865907648-project] 10.222.0.16 "GET /compute/v2.1/os-keypairs/tempest-keypair-295988021" status: 200 len: 512 microversion: 2.2 time: 0.013342 Jul 16 21:18:48.123216 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1211/2470] 10.222.0.16 () {60 vars in 1254 bytes} [Sat Jul 16 21:18:48 2022] GET /compute/v2.1/os-keypairs/tempest-keypair-295988021 => generated 512 bytes in 15 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:18:48.129441 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-99b0abd3-afd2-4111-a335-26a67c87f57e tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:48.130424 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-99b0abd3-afd2-4111-a335-26a67c87f57e tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] [instance: c88b191a-1ae5-4952-8afa-14aee5567a7f] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:18:48.138465 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-99b0abd3-afd2-4111-a335-26a67c87f57e tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:48.139988 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-99b0abd3-afd2-4111-a335-26a67c87f57e tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:48.140956 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-99b0abd3-afd2-4111-a335-26a67c87f57e tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:48.141563 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-99b0abd3-afd2-4111-a335-26a67c87f57e tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:48.453172 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-2437c72d-e562-44f4-81e2-bc09ee61ba5d tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] 10.222.0.16 "DELETE /compute/v2.1/servers/6cb4e66a-eba2-42b7-91e8-09ee7b2a5a4f" status: 204 len: 0 microversion: 2.1 time: 0.413882 Jul 16 21:18:48.458303 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1260/2471] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:18:48 2022] DELETE /compute/v2.1/servers/6cb4e66a-eba2-42b7-91e8-09ee7b2a5a4f => generated 0 bytes in 426 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 16 21:18:48.470198 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-9acecdfa-9dac-4fce-aa04-4e51a16691b6 tempest-KeyPairsV22TestJSON-865907648 tempest-KeyPairsV22TestJSON-865907648-project] Calling method '.version_select at 0x7fdd9cc47310>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:48.473456 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-99b0abd3-afd2-4111-a335-26a67c87f57e tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd0008fffad8e4708b2c01e03da92c47d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e353258f9c3048ef84683294e2b8579d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:18:48.496595 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-99b0abd3-afd2-4111-a335-26a67c87f57e tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] 10.222.0.16 "GET /compute/v2.1/servers/c88b191a-1ae5-4952-8afa-14aee5567a7f" status: 200 len: 1316 microversion: 2.1 time: 0.371853 Jul 16 21:18:48.507882 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1212/2472] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:18:48 2022] GET /compute/v2.1/servers/c88b191a-1ae5-4952-8afa-14aee5567a7f => generated 1316 bytes in 384 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:18:48.515544 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-b6c6a9b4-0db6-40e1-9c41-847c95734c2c tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:48.516463 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-b6c6a9b4-0db6-40e1-9c41-847c95734c2c tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] [instance: 6cb4e66a-eba2-42b7-91e8-09ee7b2a5a4f] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:18:48.523192 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b6c6a9b4-0db6-40e1-9c41-847c95734c2c tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:48.523892 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b6c6a9b4-0db6-40e1-9c41-847c95734c2c tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:48.524863 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b6c6a9b4-0db6-40e1-9c41-847c95734c2c tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:48.525705 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b6c6a9b4-0db6-40e1-9c41-847c95734c2c tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:48.584572 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-9acecdfa-9dac-4fce-aa04-4e51a16691b6 tempest-KeyPairsV22TestJSON-865907648 tempest-KeyPairsV22TestJSON-865907648-project] 10.222.0.16 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-295988021" status: 204 len: 0 microversion: 2.2 time: 0.124043 Jul 16 21:18:48.585508 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1261/2473] 10.222.0.16 () {60 vars in 1257 bytes} [Sat Jul 16 21:18:48 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-295988021 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 16 21:18:48.598573 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-6ec5fbf9-3bdb-4f0a-92c9-e1feef4baef8 tempest-KeyPairsV22TestJSON-865907648 tempest-KeyPairsV22TestJSON-865907648-project] Action: 'create', calling method: .version_select at 0x7fdd9cc3f040>, body: {"keypair": {"name": "tempest-keypair-1433087934", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBFmkri8ATNRt5+0Vb8auQgsvZaP+AeJUSZ3axC/dkuDO6nHH6cs41VwIdbu2CCMxQu3lgGMtnYggb+lXUf+Z2sMk4SefaWy8fz5LxIVm4AT9aO7xTkASCBVOQqbZg7D3UQ=="}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:18:48.622626 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-6ec5fbf9-3bdb-4f0a-92c9-e1feef4baef8 tempest-KeyPairsV22TestJSON-865907648 tempest-KeyPairsV22TestJSON-865907648-project] Getting quotas for project 50e54e839e35463898c9c47af127a018. Resources: {'key_pairs'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:18:48.654671 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-6ec5fbf9-3bdb-4f0a-92c9-e1feef4baef8 tempest-KeyPairsV22TestJSON-865907648 tempest-KeyPairsV22TestJSON-865907648-project] Getting quotas for user ee0b62d71e9b4610b77eedd8f9d3d0a0 and project 50e54e839e35463898c9c47af127a018. Resources: {'key_pairs'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:18:48.701237 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_api:SELECT=89,nova_api:INSERT=9,nova_api:UPDATE=1,nova_api:DELETE=1 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:18:48.818802 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-6ec5fbf9-3bdb-4f0a-92c9-e1feef4baef8 tempest-KeyPairsV22TestJSON-865907648 tempest-KeyPairsV22TestJSON-865907648-project] 10.222.0.16 "POST /compute/v2.1/os-keypairs" status: 201 len: 401 microversion: 2.2 time: 0.224830 Jul 16 21:18:48.820961 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1262/2474] 10.222.0.16 () {62 vars in 1198 bytes} [Sat Jul 16 21:18:48 2022] POST /compute/v2.1/os-keypairs => generated 401 bytes in 228 msecs (HTTP/1.1 201) 9 headers in 362 bytes (2 switches on core 0) Jul 16 21:18:48.835093 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-d9c0792f-bb7f-4d93-9f78-4fdcb80a83ae tempest-KeyPairsV22TestJSON-865907648 tempest-KeyPairsV22TestJSON-865907648-project] Calling method '.version_select at 0x7fdda2bc5820>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:48.870862 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-b6c6a9b4-0db6-40e1-9c41-847c95734c2c tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '71e9304fc50447d6aac99703b5b6e4e1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e84496d78d4481e9d7648128868bca2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:18:48.880896 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-b6c6a9b4-0db6-40e1-9c41-847c95734c2c tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] 10.222.0.16 "GET /compute/v2.1/servers/6cb4e66a-eba2-42b7-91e8-09ee7b2a5a4f" status: 200 len: 1472 microversion: 2.1 time: 0.371597 Jul 16 21:18:48.882884 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1213/2475] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:18:48 2022] GET /compute/v2.1/servers/6cb4e66a-eba2-42b7-91e8-09ee7b2a5a4f => generated 1472 bytes in 373 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:18:48.934395 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-d9c0792f-bb7f-4d93-9f78-4fdcb80a83ae tempest-KeyPairsV22TestJSON-865907648 tempest-KeyPairsV22TestJSON-865907648-project] 10.222.0.16 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1433087934" status: 204 len: 0 microversion: 2.2 time: 0.106751 Jul 16 21:18:48.935002 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1263/2476] 10.222.0.16 () {60 vars in 1260 bytes} [Sat Jul 16 21:18:48 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1433087934 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 16 21:18:48.954773 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-2d4cf314-4c1f-4b95-b42e-5a4423c8dac8 tempest-KeyPairsV22TestJSON-865907648 tempest-KeyPairsV22TestJSON-865907648-project] Action: 'create', calling method: .version_select at 0x7fdd9dd3bca0>, body: {"keypair": {"name": "tempest-keypair-15312032", "public_key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCsNe3/1ILNCqFyfYWDeTKLD6jEXC2OQHLmietMWW+/vdaZq7KZEwO0jhglaFjU1mpqq4Gz5RX156sCTNM9vRbwKAxfsdF9laBYVsex3m3Wmui3uYrKyumsoJn2g9GNnG1PI1mrVjZ61i0GY3khna+wzlTpCCmy5HNlrmbj3XLqBUpipTOXmsnr4sChzC53KCd8LXuwc1i/CZPvF+3XipvAgFSE53pCtLOeB1kYMOBaiUPLQTWXR3JpckqFIQwhIH0zoHlJvZE8hh90XcPojYN56tI0OlrGqojbediJYD0rUsJu4weZpbn8vilb3JuDY+jwssnSA8wzBx3A/8y9Pp1B nova@ubuntu"}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:18:48.989292 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-2d4cf314-4c1f-4b95-b42e-5a4423c8dac8 tempest-KeyPairsV22TestJSON-865907648 tempest-KeyPairsV22TestJSON-865907648-project] Getting quotas for project 50e54e839e35463898c9c47af127a018. Resources: {'key_pairs'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:18:48.998912 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-2d4cf314-4c1f-4b95-b42e-5a4423c8dac8 tempest-KeyPairsV22TestJSON-865907648 tempest-KeyPairsV22TestJSON-865907648-project] Getting quotas for user ee0b62d71e9b4610b77eedd8f9d3d0a0 and project 50e54e839e35463898c9c47af127a018. Resources: {'key_pairs'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:18:49.054051 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-70302c42-d573-4266-9d14-ba17bde29a9e tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:49.056576 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-70302c42-d573-4266-9d14-ba17bde29a9e tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] [instance: d265192b-536b-4884-9fe8-74cf20fc7ef9] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:18:49.074811 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-70302c42-d573-4266-9d14-ba17bde29a9e tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:49.076096 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-70302c42-d573-4266-9d14-ba17bde29a9e tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:49.077216 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-70302c42-d573-4266-9d14-ba17bde29a9e tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:49.078950 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-70302c42-d573-4266-9d14-ba17bde29a9e tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:49.132923 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-2d4cf314-4c1f-4b95-b42e-5a4423c8dac8 tempest-KeyPairsV22TestJSON-865907648 tempest-KeyPairsV22TestJSON-865907648-project] 10.222.0.16 "POST /compute/v2.1/os-keypairs" status: 201 len: 587 microversion: 2.2 time: 0.192284 Jul 16 21:18:49.134183 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1214/2477] 10.222.0.16 () {62 vars in 1198 bytes} [Sat Jul 16 21:18:48 2022] POST /compute/v2.1/os-keypairs => generated 587 bytes in 194 msecs (HTTP/1.1 201) 9 headers in 362 bytes (2 switches on core 0) Jul 16 21:18:49.144028 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-67d0b673-8c56-45ca-871b-cab5af836647 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:49.144920 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-67d0b673-8c56-45ca-871b-cab5af836647 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] [instance: e7598b4a-f70c-4186-88d1-6fd0057f8656] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:18:49.163505 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-67d0b673-8c56-45ca-871b-cab5af836647 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:49.163505 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-67d0b673-8c56-45ca-871b-cab5af836647 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:49.163505 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-67d0b673-8c56-45ca-871b-cab5af836647 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:49.163505 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-67d0b673-8c56-45ca-871b-cab5af836647 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:50.032648 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-70302c42-d573-4266-9d14-ba17bde29a9e tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'daf3c9e6634e4ae1bfb492eae1f44c93', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '24c64db8082648c8837c16352252f96d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:18:50.041571 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-70302c42-d573-4266-9d14-ba17bde29a9e tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] 10.222.0.16 "GET /compute/v2.1/servers/d265192b-536b-4884-9fe8-74cf20fc7ef9" status: 200 len: 1307 microversion: 2.1 time: 0.998915 Jul 16 21:18:50.042162 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1264/2478] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:18:49 2022] GET /compute/v2.1/servers/d265192b-536b-4884-9fe8-74cf20fc7ef9 => generated 1307 bytes in 1000 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:18:50.054892 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-dbf111c1-6e95-42ef-90e5-0aca41b20d8b tempest-KeyPairsV22TestJSON-865907648 tempest-KeyPairsV22TestJSON-865907648-project] Calling method '.version_select at 0x7fdda2b909d0>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:50.108396 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-67d0b673-8c56-45ca-871b-cab5af836647 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f5bc1f2209f4eb186d80e385b67c774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '65d5bcfa02af4e8284ba1342197e2860', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:18:50.116413 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-67d0b673-8c56-45ca-871b-cab5af836647 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] 10.222.0.16 "GET /compute/v2.1/servers/e7598b4a-f70c-4186-88d1-6fd0057f8656" status: 200 len: 1471 microversion: 2.1 time: 0.979962 Jul 16 21:18:50.116994 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1215/2479] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:18:49 2022] GET /compute/v2.1/servers/e7598b4a-f70c-4186-88d1-6fd0057f8656 => generated 1471 bytes in 982 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:18:50.132968 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-dbf111c1-6e95-42ef-90e5-0aca41b20d8b tempest-KeyPairsV22TestJSON-865907648 tempest-KeyPairsV22TestJSON-865907648-project] 10.222.0.16 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-15312032" status: 204 len: 0 microversion: 2.2 time: 0.087313 Jul 16 21:18:50.133077 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-616ee9c8-377a-4d3a-876f-d7da89acb058 req-6b6e93c8-099c-49bb-a184-c47be2fab4a5 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "d265192b-536b-4884-9fe8-74cf20fc7ef9", "tag": "8a340e39-7064-4a42-b19e-e37841899e42"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:18:50.133198 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1265/2480] 10.222.0.16 () {60 vars in 1254 bytes} [Sat Jul 16 21:18:50 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-15312032 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 16 21:18:50.140580 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-b9de9d76-6107-44eb-af12-45b57ef93801 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:50.141553 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-b9de9d76-6107-44eb-af12-45b57ef93801 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] [instance: c88b191a-1ae5-4952-8afa-14aee5567a7f] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:18:50.142135 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-616ee9c8-377a-4d3a-876f-d7da89acb058 req-6b6e93c8-099c-49bb-a184-c47be2fab4a5 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:50.142637 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-616ee9c8-377a-4d3a-876f-d7da89acb058 req-6b6e93c8-099c-49bb-a184-c47be2fab4a5 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:50.148356 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b9de9d76-6107-44eb-af12-45b57ef93801 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:50.148925 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b9de9d76-6107-44eb-af12-45b57ef93801 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:50.150455 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b9de9d76-6107-44eb-af12-45b57ef93801 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:50.151431 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b9de9d76-6107-44eb-af12-45b57ef93801 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:50.155141 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-616ee9c8-377a-4d3a-876f-d7da89acb058 req-6b6e93c8-099c-49bb-a184-c47be2fab4a5 service nova] Creating event network-changed:8a340e39-7064-4a42-b19e-e37841899e42 for instance d265192b-536b-4884-9fe8-74cf20fc7ef9 on nch2-master-1 Jul 16 21:18:50.161187 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-616ee9c8-377a-4d3a-876f-d7da89acb058 req-6b6e93c8-099c-49bb-a184-c47be2fab4a5 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.033558 Jul 16 21:18:50.162053 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1216/2481] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:18:50 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:18:50.168201 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-04057480-9319-4122-8fd3-9ac7a5dcfd0c tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:50.169887 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-04057480-9319-4122-8fd3-9ac7a5dcfd0c tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] [instance: 6cb4e66a-eba2-42b7-91e8-09ee7b2a5a4f] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:18:50.177976 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-04057480-9319-4122-8fd3-9ac7a5dcfd0c tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:50.178614 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-04057480-9319-4122-8fd3-9ac7a5dcfd0c tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:50.179664 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-04057480-9319-4122-8fd3-9ac7a5dcfd0c tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:50.180252 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-04057480-9319-4122-8fd3-9ac7a5dcfd0c tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:50.342385 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell0:SELECT=4 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:18:50.733790 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-b9de9d76-6107-44eb-af12-45b57ef93801 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd0008fffad8e4708b2c01e03da92c47d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e353258f9c3048ef84683294e2b8579d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:18:50.752809 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-b9de9d76-6107-44eb-af12-45b57ef93801 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] 10.222.0.16 "GET /compute/v2.1/servers/c88b191a-1ae5-4952-8afa-14aee5567a7f" status: 200 len: 1316 microversion: 2.1 time: 0.613397 Jul 16 21:18:50.752809 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1266/2482] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:18:50 2022] GET /compute/v2.1/servers/c88b191a-1ae5-4952-8afa-14aee5567a7f => generated 1316 bytes in 615 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:18:50.758491 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-650f9a63-08e9-4643-b643-160907692444 tempest-KeyPairsV22TestJSON-865907648 tempest-KeyPairsV22TestJSON-865907648-project] Action: 'create', calling method: .version_select at 0x7fdda2a15310>, body: {"keypair": {"name": "tempest-KeyPairsV22TestJSON-keypair-1203152553", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBOKu/tDAPfdNu9J7FbDIqi4YVwVt/Gw5uc1qr3hqUbHUuFkVdqtsZG68KDQAKwOwZzsnggvxDEBMf2szMVN0pBqLm9bydRUv61iRCG/6L7fR1W1WshmjJ/067xh/MsR+Nw=="}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:18:50.800701 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-650f9a63-08e9-4643-b643-160907692444 tempest-KeyPairsV22TestJSON-865907648 tempest-KeyPairsV22TestJSON-865907648-project] Getting quotas for project 50e54e839e35463898c9c47af127a018. Resources: {'key_pairs'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:18:50.815925 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-650f9a63-08e9-4643-b643-160907692444 tempest-KeyPairsV22TestJSON-865907648 tempest-KeyPairsV22TestJSON-865907648-project] Getting quotas for user ee0b62d71e9b4610b77eedd8f9d3d0a0 and project 50e54e839e35463898c9c47af127a018. Resources: {'key_pairs'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:18:50.877373 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-04057480-9319-4122-8fd3-9ac7a5dcfd0c tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '71e9304fc50447d6aac99703b5b6e4e1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e84496d78d4481e9d7648128868bca2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:18:50.898214 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-04057480-9319-4122-8fd3-9ac7a5dcfd0c tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] 10.222.0.16 "GET /compute/v2.1/servers/6cb4e66a-eba2-42b7-91e8-09ee7b2a5a4f" status: 200 len: 1472 microversion: 2.1 time: 0.734397 Jul 16 21:18:50.899388 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1217/2483] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:18:50 2022] GET /compute/v2.1/servers/6cb4e66a-eba2-42b7-91e8-09ee7b2a5a4f => generated 1472 bytes in 737 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:18:50.933033 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-650f9a63-08e9-4643-b643-160907692444 tempest-KeyPairsV22TestJSON-865907648 tempest-KeyPairsV22TestJSON-865907648-project] 10.222.0.16 "POST /compute/v2.1/os-keypairs" status: 201 len: 421 microversion: 2.2 time: 0.178476 Jul 16 21:18:50.933820 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1267/2484] 10.222.0.16 () {62 vars in 1198 bytes} [Sat Jul 16 21:18:50 2022] POST /compute/v2.1/os-keypairs => generated 421 bytes in 186 msecs (HTTP/1.1 201) 9 headers in 362 bytes (1 switches on core 0) Jul 16 21:18:50.947800 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-ec558a9f-2c81-46a4-93ed-125d0bb043b9 tempest-KeyPairsV22TestJSON-865907648 tempest-KeyPairsV22TestJSON-865907648-project] Action: 'create', calling method: .version_select at 0x7fdd9d221790>, body: {"keypair": {"name": "tempest-KeyPairsV22TestJSON-keypair-1684602812", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBDarfL0FufrpgqgZZ3HqAWG5SLbOHIZ/jZ8AUuvIvJCYHY3PUsCGP1rxJ1ro7745IcJ5sn0VEdyom1kykF5exjl0k6pk4G8Wl70OpQ7OXnAs/4e7js8E2nkLgOLYeZXo1g=="}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:18:50.982291 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-ec558a9f-2c81-46a4-93ed-125d0bb043b9 tempest-KeyPairsV22TestJSON-865907648 tempest-KeyPairsV22TestJSON-865907648-project] Getting quotas for project 50e54e839e35463898c9c47af127a018. Resources: {'key_pairs'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:18:50.992250 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-ec558a9f-2c81-46a4-93ed-125d0bb043b9 tempest-KeyPairsV22TestJSON-865907648 tempest-KeyPairsV22TestJSON-865907648-project] Getting quotas for user ee0b62d71e9b4610b77eedd8f9d3d0a0 and project 50e54e839e35463898c9c47af127a018. Resources: {'key_pairs'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:18:51.006537 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_api:SELECT=94,nova_api:INSERT=3,nova_api:DELETE=3 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:18:51.059591 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-c4db95e5-f081-4641-a409-adaeb6f93924 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:51.060898 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-c4db95e5-f081-4641-a409-adaeb6f93924 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] [instance: d265192b-536b-4884-9fe8-74cf20fc7ef9] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:18:51.068093 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c4db95e5-f081-4641-a409-adaeb6f93924 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:51.068834 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c4db95e5-f081-4641-a409-adaeb6f93924 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:51.069829 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c4db95e5-f081-4641-a409-adaeb6f93924 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:51.070479 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c4db95e5-f081-4641-a409-adaeb6f93924 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:51.102301 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-ec558a9f-2c81-46a4-93ed-125d0bb043b9 tempest-KeyPairsV22TestJSON-865907648 tempest-KeyPairsV22TestJSON-865907648-project] 10.222.0.16 "POST /compute/v2.1/os-keypairs" status: 201 len: 421 microversion: 2.2 time: 0.159750 Jul 16 21:18:51.104047 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1218/2485] 10.222.0.16 () {62 vars in 1198 bytes} [Sat Jul 16 21:18:50 2022] POST /compute/v2.1/os-keypairs => generated 421 bytes in 162 msecs (HTTP/1.1 201) 9 headers in 362 bytes (1 switches on core 0) Jul 16 21:18:51.120237 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-91f139f2-262d-4562-8720-a5755a743f9f tempest-KeyPairsV22TestJSON-865907648 tempest-KeyPairsV22TestJSON-865907648-project] Action: 'create', calling method: .version_select at 0x7fdd9cdca310>, body: {"keypair": {"name": "tempest-KeyPairsV22TestJSON-keypair-1557221917", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBPuuhnXwqPiQYSzlGTewmdDNX/aleTblkPxck5ib6FVvnmGXag+GjL6n6P/tvW8baf9/Y7f1y9KoYfwVxLdAezIcEzsh9WVJBsZd4uGUog/+Objnt6eLxM9/7XOLlKU66g=="}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:18:51.143370 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-91f139f2-262d-4562-8720-a5755a743f9f tempest-KeyPairsV22TestJSON-865907648 tempest-KeyPairsV22TestJSON-865907648-project] Getting quotas for project 50e54e839e35463898c9c47af127a018. Resources: {'key_pairs'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:18:51.151590 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-91f139f2-262d-4562-8720-a5755a743f9f tempest-KeyPairsV22TestJSON-865907648 tempest-KeyPairsV22TestJSON-865907648-project] Getting quotas for user ee0b62d71e9b4610b77eedd8f9d3d0a0 and project 50e54e839e35463898c9c47af127a018. Resources: {'key_pairs'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:18:51.277830 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-91f139f2-262d-4562-8720-a5755a743f9f tempest-KeyPairsV22TestJSON-865907648 tempest-KeyPairsV22TestJSON-865907648-project] 10.222.0.16 "POST /compute/v2.1/os-keypairs" status: 201 len: 421 microversion: 2.2 time: 0.164859 Jul 16 21:18:51.279454 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1219/2486] 10.222.0.16 () {62 vars in 1198 bytes} [Sat Jul 16 21:18:51 2022] POST /compute/v2.1/os-keypairs => generated 421 bytes in 168 msecs (HTTP/1.1 201) 9 headers in 362 bytes (1 switches on core 0) Jul 16 21:18:51.297496 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-7705b5a5-3bbf-4188-9ca3-30e2f03580cb tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:51.298754 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-7705b5a5-3bbf-4188-9ca3-30e2f03580cb tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] [instance: e7598b4a-f70c-4186-88d1-6fd0057f8656] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:18:51.317056 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7705b5a5-3bbf-4188-9ca3-30e2f03580cb tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:51.317799 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7705b5a5-3bbf-4188-9ca3-30e2f03580cb tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:51.319242 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7705b5a5-3bbf-4188-9ca3-30e2f03580cb tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:51.321711 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7705b5a5-3bbf-4188-9ca3-30e2f03580cb tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:51.848899 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-c4db95e5-f081-4641-a409-adaeb6f93924 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'daf3c9e6634e4ae1bfb492eae1f44c93', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '24c64db8082648c8837c16352252f96d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:18:51.857160 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-c4db95e5-f081-4641-a409-adaeb6f93924 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] 10.222.0.16 "GET /compute/v2.1/servers/d265192b-536b-4884-9fe8-74cf20fc7ef9" status: 200 len: 1307 microversion: 2.1 time: 0.802513 Jul 16 21:18:51.867074 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1268/2487] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:18:51 2022] GET /compute/v2.1/servers/d265192b-536b-4884-9fe8-74cf20fc7ef9 => generated 1307 bytes in 804 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:18:51.877385 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-719d9b5e-517a-4d6e-8f71-1a62ff707712 tempest-KeyPairsV22TestJSON-865907648 tempest-KeyPairsV22TestJSON-865907648-project] Calling method '.version_select at 0x7fdda29d1550>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:51.880942 ncdv-master-1 devstack@n-api.service[138680]: WARNING oslo_db.sqlalchemy.utils [None req-719d9b5e-517a-4d6e-8f71-1a62ff707712 tempest-KeyPairsV22TestJSON-865907648 tempest-KeyPairsV22TestJSON-865907648-project] Unique keys not in sort_keys. The sorting order may be unstable. Jul 16 21:18:51.888391 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-719d9b5e-517a-4d6e-8f71-1a62ff707712 tempest-KeyPairsV22TestJSON-865907648 tempest-KeyPairsV22TestJSON-865907648-project] 10.222.0.16 "GET /compute/v2.1/os-keypairs" status: 200 len: 1142 microversion: 2.2 time: 0.029339 Jul 16 21:18:51.889250 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1269/2488] 10.222.0.16 () {60 vars in 1176 bytes} [Sat Jul 16 21:18:51 2022] GET /compute/v2.1/os-keypairs => generated 1142 bytes in 31 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:18:51.893933 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-6207a225-5220-4b63-b75a-a038e1de3c59 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:51.894546 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-6207a225-5220-4b63-b75a-a038e1de3c59 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] [instance: c88b191a-1ae5-4952-8afa-14aee5567a7f] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:18:51.901692 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6207a225-5220-4b63-b75a-a038e1de3c59 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:51.902157 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6207a225-5220-4b63-b75a-a038e1de3c59 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:51.902950 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6207a225-5220-4b63-b75a-a038e1de3c59 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:51.903427 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6207a225-5220-4b63-b75a-a038e1de3c59 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:51.918064 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-7705b5a5-3bbf-4188-9ca3-30e2f03580cb tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f5bc1f2209f4eb186d80e385b67c774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '65d5bcfa02af4e8284ba1342197e2860', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:18:51.927482 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-7705b5a5-3bbf-4188-9ca3-30e2f03580cb tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] 10.222.0.16 "GET /compute/v2.1/servers/e7598b4a-f70c-4186-88d1-6fd0057f8656" status: 200 len: 1471 microversion: 2.1 time: 0.645810 Jul 16 21:18:51.928646 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1220/2489] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:18:51 2022] GET /compute/v2.1/servers/e7598b4a-f70c-4186-88d1-6fd0057f8656 => generated 1471 bytes in 648 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:18:51.934921 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-f4e828ab-c955-4eeb-845f-f3281db83d8c tempest-KeyPairsV22TestJSON-865907648 tempest-KeyPairsV22TestJSON-865907648-project] Calling method '.version_select at 0x7fdd9cffc5e0>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:52.028630 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-f4e828ab-c955-4eeb-845f-f3281db83d8c tempest-KeyPairsV22TestJSON-865907648 tempest-KeyPairsV22TestJSON-865907648-project] 10.222.0.16 "DELETE /compute/v2.1/os-keypairs/tempest-KeyPairsV22TestJSON-keypair-1557221917" status: 204 len: 0 microversion: 2.2 time: 0.098624 Jul 16 21:18:52.030013 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1221/2490] 10.222.0.16 () {60 vars in 1320 bytes} [Sat Jul 16 21:18:51 2022] DELETE /compute/v2.1/os-keypairs/tempest-KeyPairsV22TestJSON-keypair-1557221917 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 16 21:18:52.037008 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-7047fcf4-5578-4b28-afdf-158e07b5884b tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:52.037639 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-7047fcf4-5578-4b28-afdf-158e07b5884b tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] [instance: 6cb4e66a-eba2-42b7-91e8-09ee7b2a5a4f] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:18:52.045328 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7047fcf4-5578-4b28-afdf-158e07b5884b tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:52.045916 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7047fcf4-5578-4b28-afdf-158e07b5884b tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:52.046746 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7047fcf4-5578-4b28-afdf-158e07b5884b tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:52.047233 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7047fcf4-5578-4b28-afdf-158e07b5884b tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:52.218030 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-6207a225-5220-4b63-b75a-a038e1de3c59 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd0008fffad8e4708b2c01e03da92c47d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e353258f9c3048ef84683294e2b8579d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:18:52.226194 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-6207a225-5220-4b63-b75a-a038e1de3c59 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] 10.222.0.16 "GET /compute/v2.1/servers/c88b191a-1ae5-4952-8afa-14aee5567a7f" status: 200 len: 1316 microversion: 2.1 time: 0.335892 Jul 16 21:18:52.228247 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1270/2491] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:18:51 2022] GET /compute/v2.1/servers/c88b191a-1ae5-4952-8afa-14aee5567a7f => generated 1316 bytes in 338 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:18:52.233381 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-0b47fab9-3b3c-474e-acb4-0cbf5fd62dcf tempest-KeyPairsV22TestJSON-865907648 tempest-KeyPairsV22TestJSON-865907648-project] Calling method '.version_select at 0x7fdda2aefa60>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:52.330767 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-0b47fab9-3b3c-474e-acb4-0cbf5fd62dcf tempest-KeyPairsV22TestJSON-865907648 tempest-KeyPairsV22TestJSON-865907648-project] 10.222.0.16 "DELETE /compute/v2.1/os-keypairs/tempest-KeyPairsV22TestJSON-keypair-1684602812" status: 204 len: 0 microversion: 2.2 time: 0.100196 Jul 16 21:18:52.332042 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1271/2492] 10.222.0.16 () {60 vars in 1320 bytes} [Sat Jul 16 21:18:52 2022] DELETE /compute/v2.1/os-keypairs/tempest-KeyPairsV22TestJSON-keypair-1684602812 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 16 21:18:52.344180 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-87851802-d618-40d9-9d1f-dab5af647f02 tempest-KeyPairsV22TestJSON-865907648 tempest-KeyPairsV22TestJSON-865907648-project] Calling method '.version_select at 0x7fdd9cc3fdc0>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:52.427307 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-7047fcf4-5578-4b28-afdf-158e07b5884b tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '71e9304fc50447d6aac99703b5b6e4e1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e84496d78d4481e9d7648128868bca2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:18:52.445680 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-87851802-d618-40d9-9d1f-dab5af647f02 tempest-KeyPairsV22TestJSON-865907648 tempest-KeyPairsV22TestJSON-865907648-project] 10.222.0.16 "DELETE /compute/v2.1/os-keypairs/tempest-KeyPairsV22TestJSON-keypair-1203152553" status: 204 len: 0 microversion: 2.2 time: 0.109826 Jul 16 21:18:52.446401 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1272/2493] 10.222.0.16 () {60 vars in 1320 bytes} [Sat Jul 16 21:18:52 2022] DELETE /compute/v2.1/os-keypairs/tempest-KeyPairsV22TestJSON-keypair-1203152553 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 16 21:18:52.449483 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-7047fcf4-5578-4b28-afdf-158e07b5884b tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] 10.222.0.16 "GET /compute/v2.1/servers/6cb4e66a-eba2-42b7-91e8-09ee7b2a5a4f" status: 200 len: 1472 microversion: 2.1 time: 0.416832 Jul 16 21:18:52.452327 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-6643c6dd-9e98-468c-a534-e614a0d57aeb req-2d1890ac-e7ca-41f2-8375-6befe8b19e84 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "6cb4e66a-eba2-42b7-91e8-09ee7b2a5a4f", "name": "network-vif-deleted", "tag": "381fe698-72ef-4f6f-8b88-5df5cb8d8d96"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:18:52.459456 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1222/2494] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:18:52 2022] GET /compute/v2.1/servers/6cb4e66a-eba2-42b7-91e8-09ee7b2a5a4f => generated 1472 bytes in 429 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:18:52.463947 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-6643c6dd-9e98-468c-a534-e614a0d57aeb req-2d1890ac-e7ca-41f2-8375-6befe8b19e84 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:52.464400 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-6643c6dd-9e98-468c-a534-e614a0d57aeb req-2d1890ac-e7ca-41f2-8375-6befe8b19e84 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:52.465211 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-fef6e855-7201-41a8-9241-5b7995ba115e tempest-KeyPairsV22TestJSON-865907648 tempest-KeyPairsV22TestJSON-865907648-project] Action: 'create', calling method: .version_select at 0x7fdd9d96a940>, body: {"keypair": {"name": "tempest-keypair-570822837", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBBVlvJ7xf+VgirxAbeRRbbsAMuLCJbwWEx59cBDYAbhzRP2592PMgeQvFlOuozJDvPwZaUuYAJBchxccAJ6My7/VbaQxWrgL/vAG5I2e5WXUAWrIMEf9AZDJifIjxMvcWw=="}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:18:52.477243 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-6643c6dd-9e98-468c-a534-e614a0d57aeb req-2d1890ac-e7ca-41f2-8375-6befe8b19e84 service nova] Creating event network-vif-deleted:381fe698-72ef-4f6f-8b88-5df5cb8d8d96 for instance 6cb4e66a-eba2-42b7-91e8-09ee7b2a5a4f on nch2-master-1 Jul 16 21:18:52.480214 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-fef6e855-7201-41a8-9241-5b7995ba115e tempest-KeyPairsV22TestJSON-865907648 tempest-KeyPairsV22TestJSON-865907648-project] Getting quotas for project 50e54e839e35463898c9c47af127a018. Resources: {'key_pairs'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:18:52.486598 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-6643c6dd-9e98-468c-a534-e614a0d57aeb req-2d1890ac-e7ca-41f2-8375-6befe8b19e84 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.036344 Jul 16 21:18:52.487076 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-fef6e855-7201-41a8-9241-5b7995ba115e tempest-KeyPairsV22TestJSON-865907648 tempest-KeyPairsV22TestJSON-865907648-project] Getting quotas for user ee0b62d71e9b4610b77eedd8f9d3d0a0 and project 50e54e839e35463898c9c47af127a018. Resources: {'key_pairs'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:18:52.487399 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1273/2495] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:18:52 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:18:52.610337 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-fef6e855-7201-41a8-9241-5b7995ba115e tempest-KeyPairsV22TestJSON-865907648 tempest-KeyPairsV22TestJSON-865907648-project] 10.222.0.16 "POST /compute/v2.1/os-keypairs" status: 201 len: 400 microversion: 2.2 time: 0.149472 Jul 16 21:18:52.615374 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1223/2496] 10.222.0.16 () {62 vars in 1198 bytes} [Sat Jul 16 21:18:52 2022] POST /compute/v2.1/os-keypairs => generated 400 bytes in 151 msecs (HTTP/1.1 201) 9 headers in 362 bytes (1 switches on core 0) Jul 16 21:18:52.625492 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-bbaccf66-aea1-483d-801e-eac5ebe232de tempest-KeyPairsV22TestJSON-865907648 tempest-KeyPairsV22TestJSON-865907648-project] Calling method '.version_select at 0x7fdd9c6b61f0>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:52.636068 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-bbaccf66-aea1-483d-801e-eac5ebe232de tempest-KeyPairsV22TestJSON-865907648 tempest-KeyPairsV22TestJSON-865907648-project] 10.222.0.16 "GET /compute/v2.1/os-keypairs/tempest-keypair-570822837" status: 200 len: 512 microversion: 2.2 time: 0.018688 Jul 16 21:18:52.643341 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1274/2497] 10.222.0.16 () {60 vars in 1254 bytes} [Sat Jul 16 21:18:52 2022] GET /compute/v2.1/os-keypairs/tempest-keypair-570822837 => generated 512 bytes in 26 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:18:52.648032 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-ee03a954-ab37-4f1a-acea-16b5460518b6 tempest-KeyPairsV22TestJSON-865907648 tempest-KeyPairsV22TestJSON-865907648-project] Calling method '.version_select at 0x7fdd9d087ee0>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:52.650962 ncdv-master-1 devstack@n-api.service[138681]: WARNING oslo_db.sqlalchemy.utils [None req-ee03a954-ab37-4f1a-acea-16b5460518b6 tempest-KeyPairsV22TestJSON-865907648 tempest-KeyPairsV22TestJSON-865907648-project] Unique keys not in sort_keys. The sorting order may be unstable. Jul 16 21:18:52.667161 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-ee03a954-ab37-4f1a-acea-16b5460518b6 tempest-KeyPairsV22TestJSON-865907648 tempest-KeyPairsV22TestJSON-865907648-project] 10.222.0.16 "GET /compute/v2.1/os-keypairs" status: 200 len: 369 microversion: 2.2 time: 0.022974 Jul 16 21:18:52.667748 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1224/2498] 10.222.0.16 () {60 vars in 1176 bytes} [Sat Jul 16 21:18:52 2022] GET /compute/v2.1/os-keypairs => generated 369 bytes in 24 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 16 21:18:52.702089 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-ab6aed3e-44c8-4352-ad68-fb69db5b3a24 tempest-KeyPairsV22TestJSON-865907648 tempest-KeyPairsV22TestJSON-865907648-project] Calling method '.version_select at 0x7fdda2b39ee0>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:52.864272 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-ab6aed3e-44c8-4352-ad68-fb69db5b3a24 tempest-KeyPairsV22TestJSON-865907648 tempest-KeyPairsV22TestJSON-865907648-project] 10.222.0.16 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-570822837" status: 204 len: 0 microversion: 2.2 time: 0.187839 Jul 16 21:18:52.865387 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1275/2499] 10.222.0.16 () {60 vars in 1257 bytes} [Sat Jul 16 21:18:52 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-570822837 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 16 21:18:52.879198 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-0780f3b3-28de-4bbc-a161-10c43b0cfa28 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:52.879974 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-0780f3b3-28de-4bbc-a161-10c43b0cfa28 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] [instance: d265192b-536b-4884-9fe8-74cf20fc7ef9] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:18:52.888738 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-49a43d49-0b96-4833-925c-b1e8eb98dbb7 tempest-KeyPairsV22TestJSON-865907648 tempest-KeyPairsV22TestJSON-865907648-project] Action: 'create', calling method: .version_select at 0x7fdda2bc5160>, body: {"keypair": {"name": "tempest-keypair-1442909616", "type": "x509"}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:18:52.897431 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0780f3b3-28de-4bbc-a161-10c43b0cfa28 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:52.898366 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0780f3b3-28de-4bbc-a161-10c43b0cfa28 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:52.900432 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0780f3b3-28de-4bbc-a161-10c43b0cfa28 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:52.900883 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0780f3b3-28de-4bbc-a161-10c43b0cfa28 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:52.919877 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-49a43d49-0b96-4833-925c-b1e8eb98dbb7 tempest-KeyPairsV22TestJSON-865907648 tempest-KeyPairsV22TestJSON-865907648-project] Getting quotas for project 50e54e839e35463898c9c47af127a018. Resources: {'key_pairs'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:18:52.927953 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-49a43d49-0b96-4833-925c-b1e8eb98dbb7 tempest-KeyPairsV22TestJSON-865907648 tempest-KeyPairsV22TestJSON-865907648-project] Getting quotas for user ee0b62d71e9b4610b77eedd8f9d3d0a0 and project 50e54e839e35463898c9c47af127a018. Resources: {'key_pairs'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:18:52.948050 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.processutils [None req-49a43d49-0b96-4833-925c-b1e8eb98dbb7 tempest-KeyPairsV22TestJSON-865907648 tempest-KeyPairsV22TestJSON-865907648-project] Running cmd (subprocess): openssl req -x509 -nodes -days 3650 -config /tmp/tmp9l2psv_0/temp.conf -newkey rsa:2048 -outform PEM -keyout /tmp/tmp9l2psv_0/temp.key -subj /CN=ee0b62d71e9b4610b77eedd8f9d3d0a0 -extensions v3_req_client {{(pid=138680) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Jul 16 21:18:53.063974 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-0780f3b3-28de-4bbc-a161-10c43b0cfa28 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'daf3c9e6634e4ae1bfb492eae1f44c93', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '24c64db8082648c8837c16352252f96d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:18:53.074189 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-0780f3b3-28de-4bbc-a161-10c43b0cfa28 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] 10.222.0.16 "GET /compute/v2.1/servers/d265192b-536b-4884-9fe8-74cf20fc7ef9" status: 200 len: 1307 microversion: 2.1 time: 0.200978 Jul 16 21:18:53.074984 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1225/2500] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:18:52 2022] GET /compute/v2.1/servers/d265192b-536b-4884-9fe8-74cf20fc7ef9 => generated 1307 bytes in 203 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:18:53.079895 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-6942cc63-68a0-45a5-9989-ab70e1ad817a tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:53.080699 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-6942cc63-68a0-45a5-9989-ab70e1ad817a tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] [instance: e7598b4a-f70c-4186-88d1-6fd0057f8656] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:18:53.088247 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6942cc63-68a0-45a5-9989-ab70e1ad817a tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:53.089117 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6942cc63-68a0-45a5-9989-ab70e1ad817a tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:53.090105 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6942cc63-68a0-45a5-9989-ab70e1ad817a tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:53.090546 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6942cc63-68a0-45a5-9989-ab70e1ad817a tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:53.120766 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=100 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:18:53.269164 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-6942cc63-68a0-45a5-9989-ab70e1ad817a tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f5bc1f2209f4eb186d80e385b67c774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '65d5bcfa02af4e8284ba1342197e2860', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:18:53.277931 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-6942cc63-68a0-45a5-9989-ab70e1ad817a tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] 10.222.0.16 "GET /compute/v2.1/servers/e7598b4a-f70c-4186-88d1-6fd0057f8656" status: 200 len: 1471 microversion: 2.1 time: 0.201685 Jul 16 21:18:53.278608 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1226/2501] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:18:53 2022] GET /compute/v2.1/servers/e7598b4a-f70c-4186-88d1-6fd0057f8656 => generated 1471 bytes in 203 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:18:53.289614 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.processutils [None req-49a43d49-0b96-4833-925c-b1e8eb98dbb7 tempest-KeyPairsV22TestJSON-865907648 tempest-KeyPairsV22TestJSON-865907648-project] CMD "openssl req -x509 -nodes -days 3650 -config /tmp/tmp9l2psv_0/temp.conf -newkey rsa:2048 -outform PEM -keyout /tmp/tmp9l2psv_0/temp.key -subj /CN=ee0b62d71e9b4610b77eedd8f9d3d0a0 -extensions v3_req_client" returned: 0 in 0.341s {{(pid=138680) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Jul 16 21:18:53.292479 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.processutils [None req-49a43d49-0b96-4833-925c-b1e8eb98dbb7 tempest-KeyPairsV22TestJSON-865907648 tempest-KeyPairsV22TestJSON-865907648-project] Running cmd (subprocess): openssl pkcs12 -export -inkey /tmp/tmp9l2psv_0/temp.key -password pass: {{(pid=138680) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Jul 16 21:18:53.375682 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.processutils [None req-49a43d49-0b96-4833-925c-b1e8eb98dbb7 tempest-KeyPairsV22TestJSON-865907648 tempest-KeyPairsV22TestJSON-865907648-project] CMD "openssl pkcs12 -export -inkey /tmp/tmp9l2psv_0/temp.key -password pass:" returned: 0 in 0.082s {{(pid=138680) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Jul 16 21:18:53.506311 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-e667df2f-f761-4773-a481-bbf439688735 req-74a368a7-a1bd-471b-9401-7b5ce799316d service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "c88b191a-1ae5-4952-8afa-14aee5567a7f", "name": "network-vif-plugged", "status": "completed", "tag": "9b680cff-b0fe-4eb6-91dd-54ab2451c6f2"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:18:53.514743 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-e667df2f-f761-4773-a481-bbf439688735 req-74a368a7-a1bd-471b-9401-7b5ce799316d service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:53.515238 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-e667df2f-f761-4773-a481-bbf439688735 req-74a368a7-a1bd-471b-9401-7b5ce799316d service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:53.527524 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-e667df2f-f761-4773-a481-bbf439688735 req-74a368a7-a1bd-471b-9401-7b5ce799316d service nova] Creating event network-vif-plugged:9b680cff-b0fe-4eb6-91dd-54ab2451c6f2 for instance c88b191a-1ae5-4952-8afa-14aee5567a7f on nch1-master-1 Jul 16 21:18:53.532725 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-e667df2f-f761-4773-a481-bbf439688735 req-74a368a7-a1bd-471b-9401-7b5ce799316d service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.252324 Jul 16 21:18:53.533307 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1227/2502] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:18:53 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 254 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:18:53.538805 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-34314483-6e39-4db2-ba44-5e6644371a3f tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:53.539522 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-34314483-6e39-4db2-ba44-5e6644371a3f tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] [instance: c88b191a-1ae5-4952-8afa-14aee5567a7f] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:18:53.547618 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-34314483-6e39-4db2-ba44-5e6644371a3f tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:53.548074 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-34314483-6e39-4db2-ba44-5e6644371a3f tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:53.548838 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-34314483-6e39-4db2-ba44-5e6644371a3f tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:53.549272 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-34314483-6e39-4db2-ba44-5e6644371a3f tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:53.697320 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-49a43d49-0b96-4833-925c-b1e8eb98dbb7 tempest-KeyPairsV22TestJSON-865907648 tempest-KeyPairsV22TestJSON-865907648-project] Getting quotas for project 50e54e839e35463898c9c47af127a018. Resources: {'key_pairs'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:18:53.708788 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-49a43d49-0b96-4833-925c-b1e8eb98dbb7 tempest-KeyPairsV22TestJSON-865907648 tempest-KeyPairsV22TestJSON-865907648-project] Getting quotas for user ee0b62d71e9b4610b77eedd8f9d3d0a0 and project 50e54e839e35463898c9c47af127a018. Resources: {'key_pairs'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:18:53.738313 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-49a43d49-0b96-4833-925c-b1e8eb98dbb7 tempest-KeyPairsV22TestJSON-865907648 tempest-KeyPairsV22TestJSON-865907648-project] 10.222.0.16 "POST /compute/v2.1/os-keypairs" status: 201 len: 4693 microversion: 2.2 time: 0.860101 Jul 16 21:18:53.744827 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1276/2503] 10.222.0.16 () {62 vars in 1197 bytes} [Sat Jul 16 21:18:52 2022] POST /compute/v2.1/os-keypairs => generated 4693 bytes in 867 msecs (HTTP/1.1 201) 9 headers in 363 bytes (1 switches on core 0) Jul 16 21:18:53.752384 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-cb9eb145-157d-4d01-b733-b77e1599feca tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:53.753527 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-cb9eb145-157d-4d01-b733-b77e1599feca tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] [instance: 6cb4e66a-eba2-42b7-91e8-09ee7b2a5a4f] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:18:53.757288 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-34314483-6e39-4db2-ba44-5e6644371a3f tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd0008fffad8e4708b2c01e03da92c47d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e353258f9c3048ef84683294e2b8579d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:18:53.761532 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cb9eb145-157d-4d01-b733-b77e1599feca tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:53.762222 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cb9eb145-157d-4d01-b733-b77e1599feca tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:53.763573 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cb9eb145-157d-4d01-b733-b77e1599feca tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:53.764225 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cb9eb145-157d-4d01-b733-b77e1599feca tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:53.765695 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-34314483-6e39-4db2-ba44-5e6644371a3f tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] 10.222.0.16 "GET /compute/v2.1/servers/c88b191a-1ae5-4952-8afa-14aee5567a7f" status: 200 len: 1316 microversion: 2.1 time: 0.231218 Jul 16 21:18:53.766587 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1228/2504] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:18:53 2022] GET /compute/v2.1/servers/c88b191a-1ae5-4952-8afa-14aee5567a7f => generated 1316 bytes in 233 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:18:53.771142 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-27811ad6-cd6f-4d62-926d-b63fd15d2c3a tempest-KeyPairsV22TestJSON-865907648 tempest-KeyPairsV22TestJSON-865907648-project] Calling method '.version_select at 0x7fdd9dbbd8b0>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:53.780141 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-27811ad6-cd6f-4d62-926d-b63fd15d2c3a tempest-KeyPairsV22TestJSON-865907648 tempest-KeyPairsV22TestJSON-865907648-project] 10.222.0.16 "GET /compute/v2.1/os-keypairs/tempest-keypair-1442909616" status: 200 len: 1534 microversion: 2.2 time: 0.012497 Jul 16 21:18:53.780887 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1229/2505] 10.222.0.16 () {60 vars in 1257 bytes} [Sat Jul 16 21:18:53 2022] GET /compute/v2.1/os-keypairs/tempest-keypair-1442909616 => generated 1534 bytes in 14 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:18:53.792747 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-8f4ebe11-0edf-407e-8718-edfe7f0e99c0 tempest-KeyPairsV22TestJSON-865907648 tempest-KeyPairsV22TestJSON-865907648-project] Calling method '.version_select at 0x7fdd9d8d3160>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:53.795445 ncdv-master-1 devstack@n-api.service[138681]: WARNING oslo_db.sqlalchemy.utils [None req-8f4ebe11-0edf-407e-8718-edfe7f0e99c0 tempest-KeyPairsV22TestJSON-865907648 tempest-KeyPairsV22TestJSON-865907648-project] Unique keys not in sort_keys. The sorting order may be unstable. Jul 16 21:18:53.800544 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-8f4ebe11-0edf-407e-8718-edfe7f0e99c0 tempest-KeyPairsV22TestJSON-865907648 tempest-KeyPairsV22TestJSON-865907648-project] 10.222.0.16 "GET /compute/v2.1/os-keypairs" status: 200 len: 1391 microversion: 2.2 time: 0.013661 Jul 16 21:18:53.802571 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1230/2506] 10.222.0.16 () {60 vars in 1176 bytes} [Sat Jul 16 21:18:53 2022] GET /compute/v2.1/os-keypairs => generated 1391 bytes in 16 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:18:53.804642 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cb9eb145-157d-4d01-b733-b77e1599feca tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:53.805102 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cb9eb145-157d-4d01-b733-b77e1599feca tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:53.805575 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.objects.instance [None req-cb9eb145-157d-4d01-b733-b77e1599feca tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Lazy-loading 'fault' on Instance uuid 6cb4e66a-eba2-42b7-91e8-09ee7b2a5a4f {{(pid=138680) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:18:53.815237 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-70ef14d6-0149-45c1-869f-e40bba25f74e tempest-KeyPairsV22TestJSON-865907648 tempest-KeyPairsV22TestJSON-865907648-project] Calling method '.version_select at 0x7fdd9d186040>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:53.850614 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-cb9eb145-157d-4d01-b733-b77e1599feca tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '71e9304fc50447d6aac99703b5b6e4e1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e84496d78d4481e9d7648128868bca2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:18:53.859959 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-cb9eb145-157d-4d01-b733-b77e1599feca tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] 10.222.0.16 "GET /compute/v2.1/servers/6cb4e66a-eba2-42b7-91e8-09ee7b2a5a4f" status: 200 len: 1307 microversion: 2.1 time: 0.113343 Jul 16 21:18:53.865889 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1277/2507] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:18:53 2022] GET /compute/v2.1/servers/6cb4e66a-eba2-42b7-91e8-09ee7b2a5a4f => generated 1307 bytes in 115 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:18:53.936015 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-70ef14d6-0149-45c1-869f-e40bba25f74e tempest-KeyPairsV22TestJSON-865907648 tempest-KeyPairsV22TestJSON-865907648-project] 10.222.0.16 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1442909616" status: 204 len: 0 microversion: 2.2 time: 0.124363 Jul 16 21:18:53.936732 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1231/2508] 10.222.0.16 () {60 vars in 1260 bytes} [Sat Jul 16 21:18:53 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1442909616 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Jul 16 21:18:54.092031 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-9baa8807-961b-42f4-a0b3-bea27a3b63cb tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:54.092739 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-9baa8807-961b-42f4-a0b3-bea27a3b63cb tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] [instance: d265192b-536b-4884-9fe8-74cf20fc7ef9] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:18:54.099168 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9baa8807-961b-42f4-a0b3-bea27a3b63cb tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:54.099692 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9baa8807-961b-42f4-a0b3-bea27a3b63cb tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:54.100479 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9baa8807-961b-42f4-a0b3-bea27a3b63cb tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:54.100919 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9baa8807-961b-42f4-a0b3-bea27a3b63cb tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:54.238868 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-9baa8807-961b-42f4-a0b3-bea27a3b63cb tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'daf3c9e6634e4ae1bfb492eae1f44c93', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '24c64db8082648c8837c16352252f96d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:18:54.248412 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-9baa8807-961b-42f4-a0b3-bea27a3b63cb tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] 10.222.0.16 "GET /compute/v2.1/servers/d265192b-536b-4884-9fe8-74cf20fc7ef9" status: 200 len: 1307 microversion: 2.1 time: 0.160612 Jul 16 21:18:54.248939 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1278/2509] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:18:54 2022] GET /compute/v2.1/servers/d265192b-536b-4884-9fe8-74cf20fc7ef9 => generated 1307 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:18:54.297605 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-1c198471-b5d2-436f-850f-5a946e543dd8 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:54.298279 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-1c198471-b5d2-436f-850f-5a946e543dd8 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] [instance: e7598b4a-f70c-4186-88d1-6fd0057f8656] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:18:54.305182 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1c198471-b5d2-436f-850f-5a946e543dd8 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:54.305657 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1c198471-b5d2-436f-850f-5a946e543dd8 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:54.306906 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1c198471-b5d2-436f-850f-5a946e543dd8 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:54.307483 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1c198471-b5d2-436f-850f-5a946e543dd8 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:54.453464 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-1c198471-b5d2-436f-850f-5a946e543dd8 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f5bc1f2209f4eb186d80e385b67c774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '65d5bcfa02af4e8284ba1342197e2860', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:18:54.462729 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-1c198471-b5d2-436f-850f-5a946e543dd8 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] 10.222.0.16 "GET /compute/v2.1/servers/e7598b4a-f70c-4186-88d1-6fd0057f8656" status: 200 len: 1477 microversion: 2.1 time: 0.169382 Jul 16 21:18:54.463076 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1232/2510] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:18:54 2022] GET /compute/v2.1/servers/e7598b4a-f70c-4186-88d1-6fd0057f8656 => generated 1477 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:18:54.784319 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-d036d9aa-11ee-426d-8d4c-73f199b49fc8 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:54.785199 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-d036d9aa-11ee-426d-8d4c-73f199b49fc8 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] [instance: c88b191a-1ae5-4952-8afa-14aee5567a7f] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:18:54.793213 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d036d9aa-11ee-426d-8d4c-73f199b49fc8 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:54.793674 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d036d9aa-11ee-426d-8d4c-73f199b49fc8 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:54.794842 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d036d9aa-11ee-426d-8d4c-73f199b49fc8 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:54.795325 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d036d9aa-11ee-426d-8d4c-73f199b49fc8 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:54.877428 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-61606b57-adbe-4fa5-86e4-f166bdb86509 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:54.878271 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-61606b57-adbe-4fa5-86e4-f166bdb86509 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] [instance: 6cb4e66a-eba2-42b7-91e8-09ee7b2a5a4f] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:18:54.889626 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-61606b57-adbe-4fa5-86e4-f166bdb86509 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:54.895098 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-61606b57-adbe-4fa5-86e4-f166bdb86509 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:54.895098 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-61606b57-adbe-4fa5-86e4-f166bdb86509 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:54.895098 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-61606b57-adbe-4fa5-86e4-f166bdb86509 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:54.911901 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-61606b57-adbe-4fa5-86e4-f166bdb86509 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] HTTP exception thrown: Instance 6cb4e66a-eba2-42b7-91e8-09ee7b2a5a4f could not be found. Jul 16 21:18:54.913840 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-61606b57-adbe-4fa5-86e4-f166bdb86509 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Request method failure captured: Jul 16 21:18:54.913840 ncdv-master-1 devstack@n-api.service[138681]: request: GET /compute/v2.1/servers/6cb4e66a-eba2-42b7-91e8-09ee7b2a5a4f HTTP/1.1 Jul 16 21:18:54.913840 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:18:54.913840 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:18:54.913840 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:18:54.913840 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 21:18:54.913840 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:18:54.913840 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:18:54.913840 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:18:54.913840 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:18:54.913840 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0yq45M5tdqjee-T32BMtcEMJOIgSLcQky5K2YTxccde0BMgVBvCXGDb0O3En8ZLM5b4_J77ztcSDPXbQynL5j5ji8vdJgPr_1r_BGJmnN0395RLQHDO2cSXFUsZiVB7_gRTfKV68KCmigPiVLqn2Ocm8smB3OqtAekwKb3BDVve30Pc Jul 16 21:18:54.913840 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:18:54.913840 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:18:54.913840 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:18:54.913840 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:18:54.913840 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:18:54.913840 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:18:54.913840 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: 7e84496d78d4481e9d7648128868bca2 Jul 16 21:18:54.913840 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-ListImageFiltersTestJSON-1083290149 Jul 16 21:18:54.913840 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:18:54.913840 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:18:54.913840 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/7e84496d78d4481e9d7648128868bca2"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/7e84496d78d4481e9d7648128868bca2"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_7e84496d78d4481e9d7648128868bca2"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/7e84496d78d4481e9d7648128868bca2"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:18:54.913840 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-ListImageFiltersTestJSON-1083290149 Jul 16 21:18:54.913840 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: 7e84496d78d4481e9d7648128868bca2 Jul 16 21:18:54.913840 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-ListImageFiltersTestJSON-1083290149 Jul 16 21:18:54.913840 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-ListImageFiltersTestJSON-1083290149-project Jul 16 21:18:54.913840 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:18:54.913840 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:18:54.913840 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: 71e9304fc50447d6aac99703b5b6e4e1 Jul 16 21:18:54.913840 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-ListImageFiltersTestJSON-1083290149-project Jul 16 21:18:54.913840 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:18:54.913840 ncdv-master-1 devstack@n-api.service[138681]: exception: Instance 6cb4e66a-eba2-42b7-91e8-09ee7b2a5a4f could not be found. Jul 16 21:18:54.913840 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': '6cb4e66a-eba2-42b7-91e8-09ee7b2a5a4f'} Jul 16 21:18:54.913840 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:18:54.913840 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:18:54.913840 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:18:54.913840 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:18:54.913840 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:18:54.913840 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:18:54.913840 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:18:54.913840 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:18:54.913840 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:18:54.913840 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:18:54.913840 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 6cb4e66a-eba2-42b7-91e8-09ee7b2a5a4f could not be found. Jul 16 21:18:54.913840 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:18:54.913840 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:18:54.913840 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:18:54.913840 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:18:54.913840 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:18:54.913840 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:18:54.913840 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:18:54.913840 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:18:54.913840 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:18:54.913840 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:18:54.913840 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:18:54.913840 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:18:54.913840 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:18:54.913840 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:18:54.913840 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:18:54.913840 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:18:54.913840 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 6cb4e66a-eba2-42b7-91e8-09ee7b2a5a4f could not be found. Jul 16 21:18:54.913840 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:18:54.913840 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:18:54.913840 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:18:54.918286 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:18:54.918286 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:18:54.918286 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:18:54.918286 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:18:54.918286 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:18:54.918286 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 6cb4e66a-eba2-42b7-91e8-09ee7b2a5a4f could not be found. Jul 16 21:18:54.918286 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:18:54.920364 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-61606b57-adbe-4fa5-86e4-f166bdb86509 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Returning 404 to user: Instance 6cb4e66a-eba2-42b7-91e8-09ee7b2a5a4f could not be found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:18:54.921481 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-61606b57-adbe-4fa5-86e4-f166bdb86509 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] 10.222.0.16 "GET /compute/v2.1/servers/6cb4e66a-eba2-42b7-91e8-09ee7b2a5a4f" status: 404 len: 111 microversion: 2.1 time: 0.048280 Jul 16 21:18:54.922576 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1233/2511] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:18:54 2022] GET /compute/v2.1/servers/6cb4e66a-eba2-42b7-91e8-09ee7b2a5a4f => generated 111 bytes in 50 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:18:54.937121 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-e293b503-a1a9-4895-a080-34ed6c0224b2 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:54.937431 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-e293b503-a1a9-4895-a080-34ed6c0224b2 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] [instance: b9a4910a-f117-4834-b699-498c6431b9bb] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:18:54.949713 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e293b503-a1a9-4895-a080-34ed6c0224b2 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:54.950470 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e293b503-a1a9-4895-a080-34ed6c0224b2 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:54.951615 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e293b503-a1a9-4895-a080-34ed6c0224b2 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:54.952244 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e293b503-a1a9-4895-a080-34ed6c0224b2 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:54.984237 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-e293b503-a1a9-4895-a080-34ed6c0224b2 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] [instance: b9a4910a-f117-4834-b699-498c6431b9bb] Going to try to terminate instance {{(pid=138681) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 16 21:18:55.151892 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-d036d9aa-11ee-426d-8d4c-73f199b49fc8 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd0008fffad8e4708b2c01e03da92c47d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e353258f9c3048ef84683294e2b8579d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:18:55.159994 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-d036d9aa-11ee-426d-8d4c-73f199b49fc8 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] 10.222.0.16 "GET /compute/v2.1/servers/c88b191a-1ae5-4952-8afa-14aee5567a7f" status: 200 len: 1316 microversion: 2.1 time: 0.379572 Jul 16 21:18:55.170923 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1279/2512] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:18:54 2022] GET /compute/v2.1/servers/c88b191a-1ae5-4952-8afa-14aee5567a7f => generated 1316 bytes in 391 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:18:55.361992 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-e293b503-a1a9-4895-a080-34ed6c0224b2 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] 10.222.0.16 "DELETE /compute/v2.1/servers/b9a4910a-f117-4834-b699-498c6431b9bb" status: 204 len: 0 microversion: 2.1 time: 0.428927 Jul 16 21:18:55.363264 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1234/2513] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:18:54 2022] DELETE /compute/v2.1/servers/b9a4910a-f117-4834-b699-498c6431b9bb => generated 0 bytes in 431 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 16 21:18:55.375139 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-692bf27b-4481-4523-abb6-9d990016bc72 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:55.376109 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-692bf27b-4481-4523-abb6-9d990016bc72 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] [instance: d265192b-536b-4884-9fe8-74cf20fc7ef9] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:18:55.384494 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-0b7ca605-0c7a-4f9c-a419-f42aab606c2e tempest-VolumesSnapshotsTestJSON-851680556 tempest-VolumesSnapshotsTestJSON-851680556-project] Action: 'create', calling method: .version_select at 0x7fdd9cc473a0>, body: {"volume": {"size": 1, "display_name": "tempest-VolumesSnapshotsTestJSON-volume-1584104127"}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:18:55.387402 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-692bf27b-4481-4523-abb6-9d990016bc72 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:55.387946 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-692bf27b-4481-4523-abb6-9d990016bc72 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:55.388812 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-692bf27b-4481-4523-abb6-9d990016bc72 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:55.389330 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-692bf27b-4481-4523-abb6-9d990016bc72 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:55.634533 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-692bf27b-4481-4523-abb6-9d990016bc72 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'daf3c9e6634e4ae1bfb492eae1f44c93', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '24c64db8082648c8837c16352252f96d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:18:55.641864 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-692bf27b-4481-4523-abb6-9d990016bc72 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] 10.222.0.16 "GET /compute/v2.1/servers/d265192b-536b-4884-9fe8-74cf20fc7ef9" status: 200 len: 1307 microversion: 2.1 time: 0.274010 Jul 16 21:18:55.642455 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1235/2514] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:18:55 2022] GET /compute/v2.1/servers/d265192b-536b-4884-9fe8-74cf20fc7ef9 => generated 1307 bytes in 275 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:18:55.656824 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-b01c9ab9-24dc-4624-a029-7cc11cbd6f9d tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:55.657393 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-b01c9ab9-24dc-4624-a029-7cc11cbd6f9d tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] [instance: b9a4910a-f117-4834-b699-498c6431b9bb] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:18:55.664113 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b01c9ab9-24dc-4624-a029-7cc11cbd6f9d tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:55.664567 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b01c9ab9-24dc-4624-a029-7cc11cbd6f9d tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:55.665421 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b01c9ab9-24dc-4624-a029-7cc11cbd6f9d tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:55.665866 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b01c9ab9-24dc-4624-a029-7cc11cbd6f9d tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:55.850088 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-b01c9ab9-24dc-4624-a029-7cc11cbd6f9d tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '71e9304fc50447d6aac99703b5b6e4e1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e84496d78d4481e9d7648128868bca2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:18:55.866794 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-b01c9ab9-24dc-4624-a029-7cc11cbd6f9d tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] 10.222.0.16 "GET /compute/v2.1/servers/b9a4910a-f117-4834-b699-498c6431b9bb" status: 200 len: 1472 microversion: 2.1 time: 0.213701 Jul 16 21:18:55.867596 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1236/2515] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:18:55 2022] GET /compute/v2.1/servers/b9a4910a-f117-4834-b699-498c6431b9bb => generated 1472 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:18:55.876338 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-e94c2652-20d2-43de-904a-269a6c54900f tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:55.877244 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-e94c2652-20d2-43de-904a-269a6c54900f tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] [instance: e7598b4a-f70c-4186-88d1-6fd0057f8656] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:18:55.885402 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e94c2652-20d2-43de-904a-269a6c54900f tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:55.887427 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e94c2652-20d2-43de-904a-269a6c54900f tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:55.887427 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e94c2652-20d2-43de-904a-269a6c54900f tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:55.888658 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e94c2652-20d2-43de-904a-269a6c54900f tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:56.049347 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-e94c2652-20d2-43de-904a-269a6c54900f tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f5bc1f2209f4eb186d80e385b67c774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '65d5bcfa02af4e8284ba1342197e2860', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:18:56.057971 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-e94c2652-20d2-43de-904a-269a6c54900f tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] 10.222.0.16 "GET /compute/v2.1/servers/e7598b4a-f70c-4186-88d1-6fd0057f8656" status: 200 len: 1477 microversion: 2.1 time: 0.188603 Jul 16 21:18:56.058556 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1237/2516] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:18:55 2022] GET /compute/v2.1/servers/e7598b4a-f70c-4186-88d1-6fd0057f8656 => generated 1477 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:18:56.177091 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-f64b84c9-8303-4820-b765-f73f5f348fad tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:56.177857 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-f64b84c9-8303-4820-b765-f73f5f348fad tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] [instance: c88b191a-1ae5-4952-8afa-14aee5567a7f] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:18:56.185205 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f64b84c9-8303-4820-b765-f73f5f348fad tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:56.189172 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f64b84c9-8303-4820-b765-f73f5f348fad tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:56.189172 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f64b84c9-8303-4820-b765-f73f5f348fad tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:56.189172 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f64b84c9-8303-4820-b765-f73f5f348fad tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:56.244901 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-0b7ca605-0c7a-4f9c-a419-f42aab606c2e tempest-VolumesSnapshotsTestJSON-851680556 tempest-VolumesSnapshotsTestJSON-851680556-project] 10.222.0.16 "POST /compute/v2.1/os-volumes" status: 200 len: 338 microversion: 2.1 time: 0.999822 Jul 16 21:18:56.246329 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1280/2517] 10.222.0.16 () {60 vars in 1154 bytes} [Sat Jul 16 21:18:55 2022] POST /compute/v2.1/os-volumes => generated 338 bytes in 1002 msecs (HTTP/1.1 200) 10 headers in 448 bytes (1 switches on core 0) Jul 16 21:18:56.260873 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-58b59b1d-a9c6-4b1d-8dce-c9e8e69628dd tempest-VolumesSnapshotsTestJSON-851680556 tempest-VolumesSnapshotsTestJSON-851680556-project] Calling method '.version_select at 0x7fdda2b2bf70>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:56.360336 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-f64b84c9-8303-4820-b765-f73f5f348fad tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd0008fffad8e4708b2c01e03da92c47d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e353258f9c3048ef84683294e2b8579d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:18:56.361676 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-58b59b1d-a9c6-4b1d-8dce-c9e8e69628dd tempest-VolumesSnapshotsTestJSON-851680556 tempest-VolumesSnapshotsTestJSON-851680556-project] 10.222.0.16 "GET /compute/v2.1/os-volumes/4450505c-3ac5-42c2-b94c-b8f945007d50" status: 200 len: 338 microversion: 2.1 time: 0.107736 Jul 16 21:18:56.362442 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1281/2518] 10.222.0.16 () {58 vars in 1244 bytes} [Sat Jul 16 21:18:56 2022] GET /compute/v2.1/os-volumes/4450505c-3ac5-42c2-b94c-b8f945007d50 => generated 338 bytes in 109 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:18:56.374603 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-f64b84c9-8303-4820-b765-f73f5f348fad tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] 10.222.0.16 "GET /compute/v2.1/servers/c88b191a-1ae5-4952-8afa-14aee5567a7f" status: 200 len: 1505 microversion: 2.1 time: 0.201429 Jul 16 21:18:56.385749 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1238/2519] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:18:56 2022] GET /compute/v2.1/servers/c88b191a-1ae5-4952-8afa-14aee5567a7f => generated 1505 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:18:56.398859 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-49d51aec-4fec-47aa-ad8c-aead0e2369be tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Action: 'action', calling method: >, body: {"addSecurityGroup": {"name": "tempest-SecurityGroupsTestJSON-securitygroup-833364650"}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:18:56.400412 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-49d51aec-4fec-47aa-ad8c-aead0e2369be tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] [instance: c88b191a-1ae5-4952-8afa-14aee5567a7f] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:18:56.411950 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-49d51aec-4fec-47aa-ad8c-aead0e2369be tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:56.411950 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-49d51aec-4fec-47aa-ad8c-aead0e2369be tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:56.411950 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-49d51aec-4fec-47aa-ad8c-aead0e2369be tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:56.411950 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-49d51aec-4fec-47aa-ad8c-aead0e2369be tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:56.557405 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.network.security_group_api [None req-49d51aec-4fec-47aa-ad8c-aead0e2369be tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Adding security group 89aaff03-f3d3-48f7-965b-66c2a37029e0 to port 9b680cff-b0fe-4eb6-91dd-54ab2451c6f2 Jul 16 21:18:56.663553 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-d9414de3-19b4-41ef-bfba-9766967090c1 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:56.666851 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-d9414de3-19b4-41ef-bfba-9766967090c1 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] [instance: d265192b-536b-4884-9fe8-74cf20fc7ef9] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:18:56.688425 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d9414de3-19b4-41ef-bfba-9766967090c1 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:56.695967 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d9414de3-19b4-41ef-bfba-9766967090c1 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:56.697169 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d9414de3-19b4-41ef-bfba-9766967090c1 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:56.698597 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d9414de3-19b4-41ef-bfba-9766967090c1 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:56.954926 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-d9414de3-19b4-41ef-bfba-9766967090c1 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'daf3c9e6634e4ae1bfb492eae1f44c93', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '24c64db8082648c8837c16352252f96d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:18:56.967331 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-d9414de3-19b4-41ef-bfba-9766967090c1 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] 10.222.0.16 "GET /compute/v2.1/servers/d265192b-536b-4884-9fe8-74cf20fc7ef9" status: 200 len: 1307 microversion: 2.1 time: 0.311707 Jul 16 21:18:56.977178 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1239/2520] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:18:56 2022] GET /compute/v2.1/servers/d265192b-536b-4884-9fe8-74cf20fc7ef9 => generated 1307 bytes in 322 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:18:56.982662 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-f152b7ca-5b7a-4517-aee7-2894709241c6 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:56.983307 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-f152b7ca-5b7a-4517-aee7-2894709241c6 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] [instance: b9a4910a-f117-4834-b699-498c6431b9bb] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:18:56.990073 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f152b7ca-5b7a-4517-aee7-2894709241c6 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:56.994806 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f152b7ca-5b7a-4517-aee7-2894709241c6 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:56.994806 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f152b7ca-5b7a-4517-aee7-2894709241c6 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:56.994806 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f152b7ca-5b7a-4517-aee7-2894709241c6 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:57.191855 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-f152b7ca-5b7a-4517-aee7-2894709241c6 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '71e9304fc50447d6aac99703b5b6e4e1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e84496d78d4481e9d7648128868bca2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:18:57.203548 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-f152b7ca-5b7a-4517-aee7-2894709241c6 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] 10.222.0.16 "GET /compute/v2.1/servers/b9a4910a-f117-4834-b699-498c6431b9bb" status: 200 len: 1472 microversion: 2.1 time: 0.224583 Jul 16 21:18:57.204307 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1240/2521] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:18:56 2022] GET /compute/v2.1/servers/b9a4910a-f117-4834-b699-498c6431b9bb => generated 1472 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:18:57.211777 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-eae1cd8c-8f2e-4ad4-806a-28aed0bb5d64 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:57.215867 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-eae1cd8c-8f2e-4ad4-806a-28aed0bb5d64 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] [instance: e7598b4a-f70c-4186-88d1-6fd0057f8656] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:18:57.224337 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-eae1cd8c-8f2e-4ad4-806a-28aed0bb5d64 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:57.225003 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-eae1cd8c-8f2e-4ad4-806a-28aed0bb5d64 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:57.226100 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-eae1cd8c-8f2e-4ad4-806a-28aed0bb5d64 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:57.226855 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-eae1cd8c-8f2e-4ad4-806a-28aed0bb5d64 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:57.434684 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-49d51aec-4fec-47aa-ad8c-aead0e2369be tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] 10.222.0.16 "POST /compute/v2.1/servers/c88b191a-1ae5-4952-8afa-14aee5567a7f/action" status: 202 len: 0 microversion: 2.1 time: 1.034731 Jul 16 21:18:57.434684 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1282/2522] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 21:18:56 2022] POST /compute/v2.1/servers/c88b191a-1ae5-4952-8afa-14aee5567a7f/action => generated 0 bytes in 1036 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 21:18:57.441501 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-782c6570-ea3d-4724-8a2a-86ffb2d4ac4a tempest-VolumesSnapshotsTestJSON-851680556 tempest-VolumesSnapshotsTestJSON-851680556-project] Calling method '.version_select at 0x7fdda2bc83a0>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:57.539885 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-782c6570-ea3d-4724-8a2a-86ffb2d4ac4a tempest-VolumesSnapshotsTestJSON-851680556 tempest-VolumesSnapshotsTestJSON-851680556-project] 10.222.0.16 "GET /compute/v2.1/os-volumes/4450505c-3ac5-42c2-b94c-b8f945007d50" status: 200 len: 339 microversion: 2.1 time: 0.105694 Jul 16 21:18:57.547303 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1283/2523] 10.222.0.16 () {58 vars in 1244 bytes} [Sat Jul 16 21:18:57 2022] GET /compute/v2.1/os-volumes/4450505c-3ac5-42c2-b94c-b8f945007d50 => generated 339 bytes in 112 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:18:57.551174 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-8bb1254d-ff73-4783-bc2b-3ce0a14ec04f tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Calling method '.version_select at 0x7fdd9cc47700>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:57.776940 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-eae1cd8c-8f2e-4ad4-806a-28aed0bb5d64 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f5bc1f2209f4eb186d80e385b67c774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '65d5bcfa02af4e8284ba1342197e2860', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:18:57.785626 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-eae1cd8c-8f2e-4ad4-806a-28aed0bb5d64 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] 10.222.0.16 "GET /compute/v2.1/servers/e7598b4a-f70c-4186-88d1-6fd0057f8656" status: 200 len: 1477 microversion: 2.1 time: 0.579298 Jul 16 21:18:57.786745 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1241/2524] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:18:57 2022] GET /compute/v2.1/servers/e7598b4a-f70c-4186-88d1-6fd0057f8656 => generated 1477 bytes in 582 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:18:57.793877 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-214b9674-0d26-4fa9-9d90-ff757045d1a3 req-af39708c-ef40-44b8-b0cc-b357aac211c2 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "c88b191a-1ae5-4952-8afa-14aee5567a7f", "tag": "9b680cff-b0fe-4eb6-91dd-54ab2451c6f2"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:18:57.808991 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-214b9674-0d26-4fa9-9d90-ff757045d1a3 req-af39708c-ef40-44b8-b0cc-b357aac211c2 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:57.809708 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-214b9674-0d26-4fa9-9d90-ff757045d1a3 req-af39708c-ef40-44b8-b0cc-b357aac211c2 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:57.833956 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-214b9674-0d26-4fa9-9d90-ff757045d1a3 req-af39708c-ef40-44b8-b0cc-b357aac211c2 service nova] Creating event network-changed:9b680cff-b0fe-4eb6-91dd-54ab2451c6f2 for instance c88b191a-1ae5-4952-8afa-14aee5567a7f on nch1-master-1 Jul 16 21:18:57.840963 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-214b9674-0d26-4fa9-9d90-ff757045d1a3 req-af39708c-ef40-44b8-b0cc-b357aac211c2 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.052173 Jul 16 21:18:57.845589 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1242/2525] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:18:57 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:18:57.849534 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-d8a3f53e-5e7f-42d0-9673-e33d309e143e tempest-VolumesSnapshotsTestJSON-851680556 tempest-VolumesSnapshotsTestJSON-851680556-project] Action: 'create', calling method: .version_select at 0x7fdda03d6ee0>, body: {"snapshot": {"volume_id": "4450505c-3ac5-42c2-b94c-b8f945007d50", "display_name": "tempest-VolumesSnapshotsTestJSON-Snapshot-835757387"}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:18:58.167416 ncdv-master-1 devstack@n-api.service[138680]: DEBUG neutronclient.v2_0.client [None req-8bb1254d-ff73-4783-bc2b-3ce0a14ec04f tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Error message: {"NeutronError": {"type": "SecurityGroupInUse", "message": "Security Group 89aaff03-f3d3-48f7-965b-66c2a37029e0 in use.", "detail": ""}} {{(pid=138680) _handle_fault_response /usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py:262}} Jul 16 21:18:58.170070 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-8bb1254d-ff73-4783-bc2b-3ce0a14ec04f tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] HTTP exception thrown: Security Group 89aaff03-f3d3-48f7-965b-66c2a37029e0 in use. Jul 16 21:18:58.170070 ncdv-master-1 devstack@n-api.service[138680]: Neutron server returns request_ids: ['req-12984a38-ef14-46ad-9c45-00346ce062ee'] Jul 16 21:18:58.180116 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-8bb1254d-ff73-4783-bc2b-3ce0a14ec04f tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Request method failure captured: Jul 16 21:18:58.180116 ncdv-master-1 devstack@n-api.service[138680]: request: DELETE /compute/v2.1/os-security-groups/89aaff03-f3d3-48f7-965b-66c2a37029e0 HTTP/1.1 Jul 16 21:18:58.180116 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:18:58.180116 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:18:58.180116 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:18:58.180116 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 0 Jul 16 21:18:58.180116 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:18:58.180116 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:18:58.180116 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:18:58.180116 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:18:58.180116 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0yrk-HsfYKsyovLSaEtNxaoXy7a3rLGNlsRPiPP76ZvvrKkKyGd-uekosXQxpFgkRnItBdviN4xr49Rerg5AyvLSzsyRY-Um0r3BRAvyZrzmCDYUpWbXk1uyEIb3Af4U3Y3nXIJgJAuDVx9z2NGSvEGCsGz12WlyyGCNfVSADFcYJP8 Jul 16 21:18:58.180116 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:18:58.180116 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:18:58.180116 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:18:58.180116 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:18:58.180116 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:18:58.180116 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:18:58.180116 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: e353258f9c3048ef84683294e2b8579d Jul 16 21:18:58.180116 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-SecurityGroupsTestJSON-438340169 Jul 16 21:18:58.180116 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:18:58.180116 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:18:58.180116 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/e353258f9c3048ef84683294e2b8579d"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/e353258f9c3048ef84683294e2b8579d"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_e353258f9c3048ef84683294e2b8579d"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/e353258f9c3048ef84683294e2b8579d"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:18:58.180116 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-SecurityGroupsTestJSON-438340169 Jul 16 21:18:58.180116 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: e353258f9c3048ef84683294e2b8579d Jul 16 21:18:58.180116 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-SecurityGroupsTestJSON-438340169 Jul 16 21:18:58.180116 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-SecurityGroupsTestJSON-438340169-project Jul 16 21:18:58.180116 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:18:58.180116 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:18:58.180116 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: d0008fffad8e4708b2c01e03da92c47d Jul 16 21:18:58.180116 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-SecurityGroupsTestJSON-438340169-project Jul 16 21:18:58.180116 ncdv-master-1 devstack@n-api.service[138680]: method: Jul 16 21:18:58.180116 ncdv-master-1 devstack@n-api.service[138680]: exception: Security Group 89aaff03-f3d3-48f7-965b-66c2a37029e0 in use. Jul 16 21:18:58.180116 ncdv-master-1 devstack@n-api.service[138680]: Neutron server returns request_ids: ['req-12984a38-ef14-46ad-9c45-00346ce062ee'] Jul 16 21:18:58.180116 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': '89aaff03-f3d3-48f7-965b-66c2a37029e0'} Jul 16 21:18:58.180116 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:18:58.180116 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:18:58.180116 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/security_group_api.py", line 359, in destroy Jul 16 21:18:58.180116 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi neutron.delete_security_group(security_group['id']) Jul 16 21:18:58.180116 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 197, in wrapper Jul 16 21:18:58.180116 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Jul 16 21:18:58.180116 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 1045, in delete_security_group Jul 16 21:18:58.180116 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return self.delete(self.security_group_path % (security_group)) Jul 16 21:18:58.180116 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 197, in wrapper Jul 16 21:18:58.180116 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Jul 16 21:18:58.180116 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 352, in delete Jul 16 21:18:58.180116 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return self.retry_request("DELETE", action, body=body, Jul 16 21:18:58.180116 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 197, in wrapper Jul 16 21:18:58.180116 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Jul 16 21:18:58.180116 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 333, in retry_request Jul 16 21:18:58.180116 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return self.do_request(method, action, body=body, Jul 16 21:18:58.180116 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 197, in wrapper Jul 16 21:18:58.180116 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Jul 16 21:18:58.180116 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 297, in do_request Jul 16 21:18:58.180116 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi self._handle_fault_response(status_code, replybody, resp) Jul 16 21:18:58.180116 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 197, in wrapper Jul 16 21:18:58.180116 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Jul 16 21:18:58.180116 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 272, in _handle_fault_response Jul 16 21:18:58.180116 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi exception_handler_v20(status_code, error_body) Jul 16 21:18:58.180116 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 90, in exception_handler_v20 Jul 16 21:18:58.180116 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise client_exc(message=error_message, Jul 16 21:18:58.180116 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi neutronclient.common.exceptions.Conflict: Security Group 89aaff03-f3d3-48f7-965b-66c2a37029e0 in use. Jul 16 21:18:58.184884 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-12984a38-ef14-46ad-9c45-00346ce062ee'] Jul 16 21:18:58.184884 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:18:58.184884 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:18:58.184884 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:18:58.184884 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:18:58.184884 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/security_groups.py", line 179, in delete Jul 16 21:18:58.184884 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi security_group_api.destroy(context, security_group) Jul 16 21:18:58.184884 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/security_group_api.py", line 364, in destroy Jul 16 21:18:58.184884 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.Invalid(str(e)) Jul 16 21:18:58.184884 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.Invalid: Security Group 89aaff03-f3d3-48f7-965b-66c2a37029e0 in use. Jul 16 21:18:58.184884 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-12984a38-ef14-46ad-9c45-00346ce062ee'] Jul 16 21:18:58.184884 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:18:58.184884 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:18:58.184884 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:18:58.184884 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:18:58.184884 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:18:58.184884 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:18:58.184884 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:18:58.184884 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:18:58.184884 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Jul 16 21:18:58.184884 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Jul 16 21:18:58.184884 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:18:58.184884 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:18:58.184884 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/security_groups.py", line 183, in delete Jul 16 21:18:58.184884 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=exp.format_message()) Jul 16 21:18:58.184884 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Security Group 89aaff03-f3d3-48f7-965b-66c2a37029e0 in use. Jul 16 21:18:58.184884 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-12984a38-ef14-46ad-9c45-00346ce062ee'] Jul 16 21:18:58.184884 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:18:58.184884 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:18:58.184884 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:18:58.184884 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:18:58.184884 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:18:58.184884 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:18:58.184884 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:18:58.184884 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:18:58.184884 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Security Group 89aaff03-f3d3-48f7-965b-66c2a37029e0 in use. Jul 16 21:18:58.184884 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-12984a38-ef14-46ad-9c45-00346ce062ee'] Jul 16 21:18:58.184884 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:18:58.191154 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-8bb1254d-ff73-4783-bc2b-3ce0a14ec04f tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Returning 400 to user: Security Group 89aaff03-f3d3-48f7-965b-66c2a37029e0 in use. Jul 16 21:18:58.191154 ncdv-master-1 devstack@n-api.service[138680]: Neutron server returns request_ids: ['req-12984a38-ef14-46ad-9c45-00346ce062ee'] {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:18:58.198205 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-8bb1254d-ff73-4783-bc2b-3ce0a14ec04f tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] 10.222.0.16 "DELETE /compute/v2.1/os-security-groups/89aaff03-f3d3-48f7-965b-66c2a37029e0" status: 400 len: 185 microversion: 2.1 time: 0.648747 Jul 16 21:18:58.198205 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1284/2526] 10.222.0.16 () {58 vars in 1271 bytes} [Sat Jul 16 21:18:57 2022] DELETE /compute/v2.1/os-security-groups/89aaff03-f3d3-48f7-965b-66c2a37029e0 => generated 185 bytes in 654 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Jul 16 21:18:58.204057 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-77b0c0e3-0ab5-490e-8406-060219a7b6ed req-1775f4be-f735-460a-9f5f-24ca58339660 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "e7598b4a-f70c-4186-88d1-6fd0057f8656", "name": "network-vif-unplugged", "status": "completed", "tag": "5e72e822-7cdd-445c-8d25-dc81877dc718"}, {"server_uuid": "d265192b-536b-4884-9fe8-74cf20fc7ef9", "name": "network-vif-plugged", "status": "completed", "tag": "8a340e39-7064-4a42-b19e-e37841899e42"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:18:58.215654 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-77b0c0e3-0ab5-490e-8406-060219a7b6ed req-1775f4be-f735-460a-9f5f-24ca58339660 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:58.216279 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-77b0c0e3-0ab5-490e-8406-060219a7b6ed req-1775f4be-f735-460a-9f5f-24ca58339660 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:58.239314 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-77b0c0e3-0ab5-490e-8406-060219a7b6ed req-1775f4be-f735-460a-9f5f-24ca58339660 service nova] Creating event network-vif-unplugged:5e72e822-7cdd-445c-8d25-dc81877dc718 for instance e7598b4a-f70c-4186-88d1-6fd0057f8656 on nch1-master-1 Jul 16 21:18:58.240208 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-77b0c0e3-0ab5-490e-8406-060219a7b6ed req-1775f4be-f735-460a-9f5f-24ca58339660 service nova] Creating event network-vif-plugged:8a340e39-7064-4a42-b19e-e37841899e42 for instance d265192b-536b-4884-9fe8-74cf20fc7ef9 on nch2-master-1 Jul 16 21:18:58.256657 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-77b0c0e3-0ab5-490e-8406-060219a7b6ed req-1775f4be-f735-460a-9f5f-24ca58339660 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 356 microversion: 2.1 time: 0.057345 Jul 16 21:18:58.257668 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1285/2527] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:18:58 2022] POST /compute/v2.1/os-server-external-events => generated 356 bytes in 60 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:18:58.267400 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-a2ffdee8-f9e0-487b-8a1f-9a735c431536 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:58.269031 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-a2ffdee8-f9e0-487b-8a1f-9a735c431536 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] [instance: d265192b-536b-4884-9fe8-74cf20fc7ef9] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:18:58.297129 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a2ffdee8-f9e0-487b-8a1f-9a735c431536 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:58.299881 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a2ffdee8-f9e0-487b-8a1f-9a735c431536 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:58.301509 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a2ffdee8-f9e0-487b-8a1f-9a735c431536 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:58.302181 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a2ffdee8-f9e0-487b-8a1f-9a735c431536 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:58.621745 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-d8a3f53e-5e7f-42d0-9673-e33d309e143e tempest-VolumesSnapshotsTestJSON-851680556 tempest-VolumesSnapshotsTestJSON-851680556-project] 10.222.0.16 "POST /compute/v2.1/os-snapshots" status: 200 len: 286 microversion: 2.1 time: 0.778075 Jul 16 21:18:58.627610 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1243/2528] 10.222.0.16 () {60 vars in 1161 bytes} [Sat Jul 16 21:18:57 2022] POST /compute/v2.1/os-snapshots => generated 286 bytes in 785 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:18:58.637576 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-2463e5db-245c-4da5-8b25-62e9d36f1e55 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Action: 'action', calling method: >, body: {"reboot": {"type": "HARD"}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:18:58.644677 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-2463e5db-245c-4da5-8b25-62e9d36f1e55 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] [instance: c88b191a-1ae5-4952-8afa-14aee5567a7f] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:18:58.656104 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2463e5db-245c-4da5-8b25-62e9d36f1e55 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:58.657538 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2463e5db-245c-4da5-8b25-62e9d36f1e55 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:58.658613 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2463e5db-245c-4da5-8b25-62e9d36f1e55 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:58.660344 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2463e5db-245c-4da5-8b25-62e9d36f1e55 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:58.990457 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-2463e5db-245c-4da5-8b25-62e9d36f1e55 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] 10.222.0.16 "POST /compute/v2.1/servers/c88b191a-1ae5-4952-8afa-14aee5567a7f/action" status: 202 len: 0 microversion: 2.1 time: 0.355493 Jul 16 21:18:58.992300 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1244/2529] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 21:18:58 2022] POST /compute/v2.1/servers/c88b191a-1ae5-4952-8afa-14aee5567a7f/action => generated 0 bytes in 361 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 21:18:59.007321 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-99f7df6a-ffd0-4848-8b5b-7434e7ae94f0 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:59.008190 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-99f7df6a-ffd0-4848-8b5b-7434e7ae94f0 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] [instance: b9a4910a-f117-4834-b699-498c6431b9bb] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:18:59.014932 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-99f7df6a-ffd0-4848-8b5b-7434e7ae94f0 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:59.016791 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-99f7df6a-ffd0-4848-8b5b-7434e7ae94f0 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:59.017667 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-99f7df6a-ffd0-4848-8b5b-7434e7ae94f0 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:59.018187 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-99f7df6a-ffd0-4848-8b5b-7434e7ae94f0 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:59.170758 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-a2ffdee8-f9e0-487b-8a1f-9a735c431536 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'daf3c9e6634e4ae1bfb492eae1f44c93', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '24c64db8082648c8837c16352252f96d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:18:59.184628 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-a2ffdee8-f9e0-487b-8a1f-9a735c431536 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] 10.222.0.16 "GET /compute/v2.1/servers/d265192b-536b-4884-9fe8-74cf20fc7ef9" status: 200 len: 1307 microversion: 2.1 time: 0.925128 Jul 16 21:18:59.185231 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1286/2530] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:18:58 2022] GET /compute/v2.1/servers/d265192b-536b-4884-9fe8-74cf20fc7ef9 => generated 1307 bytes in 927 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:18:59.199444 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-d2ede908-d0f3-4a78-8df8-f50fda3aa548 tempest-VolumesSnapshotsTestJSON-851680556 tempest-VolumesSnapshotsTestJSON-851680556-project] Calling method '.version_select at 0x7fdd9d1c1a60>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:59.325250 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-d2ede908-d0f3-4a78-8df8-f50fda3aa548 tempest-VolumesSnapshotsTestJSON-851680556 tempest-VolumesSnapshotsTestJSON-851680556-project] 10.222.0.16 "GET /compute/v2.1/os-snapshots/b6ec78c5-d133-485b-8009-7628a73d4739" status: 200 len: 286 microversion: 2.1 time: 0.133531 Jul 16 21:18:59.325621 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1287/2531] 10.222.0.16 () {58 vars in 1250 bytes} [Sat Jul 16 21:18:59 2022] GET /compute/v2.1/os-snapshots/b6ec78c5-d133-485b-8009-7628a73d4739 => generated 286 bytes in 140 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:18:59.330901 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-16b974d5-33f6-4281-af60-bc498ffeb59b tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:59.331522 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-16b974d5-33f6-4281-af60-bc498ffeb59b tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] [instance: e7598b4a-f70c-4186-88d1-6fd0057f8656] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:18:59.347394 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-16b974d5-33f6-4281-af60-bc498ffeb59b tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:59.350578 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-16b974d5-33f6-4281-af60-bc498ffeb59b tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:59.357373 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-16b974d5-33f6-4281-af60-bc498ffeb59b tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:59.359078 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-16b974d5-33f6-4281-af60-bc498ffeb59b tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:59.518678 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-99f7df6a-ffd0-4848-8b5b-7434e7ae94f0 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '71e9304fc50447d6aac99703b5b6e4e1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e84496d78d4481e9d7648128868bca2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:18:59.533026 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-99f7df6a-ffd0-4848-8b5b-7434e7ae94f0 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] 10.222.0.16 "GET /compute/v2.1/servers/b9a4910a-f117-4834-b699-498c6431b9bb" status: 200 len: 1472 microversion: 2.1 time: 0.537291 Jul 16 21:18:59.535793 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1245/2532] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:18:58 2022] GET /compute/v2.1/servers/b9a4910a-f117-4834-b699-498c6431b9bb => generated 1472 bytes in 543 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:18:59.552067 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-855b9726-ece5-4e4b-bf12-f60f032ef1b5 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:18:59.552986 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-855b9726-ece5-4e4b-bf12-f60f032ef1b5 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] [instance: c88b191a-1ae5-4952-8afa-14aee5567a7f] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:18:59.567658 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-855b9726-ece5-4e4b-bf12-f60f032ef1b5 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:59.572158 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-855b9726-ece5-4e4b-bf12-f60f032ef1b5 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:59.573259 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-855b9726-ece5-4e4b-bf12-f60f032ef1b5 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:18:59.574000 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-855b9726-ece5-4e4b-bf12-f60f032ef1b5 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:18:59.610718 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=96,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:19:00.084749 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-855b9726-ece5-4e4b-bf12-f60f032ef1b5 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd0008fffad8e4708b2c01e03da92c47d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e353258f9c3048ef84683294e2b8579d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:19:00.098999 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-855b9726-ece5-4e4b-bf12-f60f032ef1b5 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] 10.222.0.16 "GET /compute/v2.1/servers/c88b191a-1ae5-4952-8afa-14aee5567a7f" status: 200 len: 1575 microversion: 2.1 time: 0.559229 Jul 16 21:19:00.143398 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1246/2533] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:18:59 2022] GET /compute/v2.1/servers/c88b191a-1ae5-4952-8afa-14aee5567a7f => generated 1575 bytes in 604 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:19:00.158121 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-c1e34e9a-87f6-4ee3-b1bb-c82596f494b4 tempest-VolumesSnapshotsTestJSON-851680556 tempest-VolumesSnapshotsTestJSON-851680556-project] Calling method '.version_select at 0x7fdd9d0a05e0>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:19:00.167176 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-16b974d5-33f6-4281-af60-bc498ffeb59b tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f5bc1f2209f4eb186d80e385b67c774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '65d5bcfa02af4e8284ba1342197e2860', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:19:00.189690 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-16b974d5-33f6-4281-af60-bc498ffeb59b tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] 10.222.0.16 "GET /compute/v2.1/servers/e7598b4a-f70c-4186-88d1-6fd0057f8656" status: 200 len: 1477 microversion: 2.1 time: 0.862658 Jul 16 21:19:00.191399 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1288/2534] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:18:59 2022] GET /compute/v2.1/servers/e7598b4a-f70c-4186-88d1-6fd0057f8656 => generated 1477 bytes in 865 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:19:00.197424 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-36a9d2bf-df52-4929-88a3-579456c9484e req-9a156992-4f88-4181-8427-8c8b4de41cd3 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "b9a4910a-f117-4834-b699-498c6431b9bb", "name": "network-vif-deleted", "tag": "9ab69acd-f0a3-4355-b203-2c48e39cc3a1"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:19:00.207641 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-36a9d2bf-df52-4929-88a3-579456c9484e req-9a156992-4f88-4181-8427-8c8b4de41cd3 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:00.208094 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-36a9d2bf-df52-4929-88a3-579456c9484e req-9a156992-4f88-4181-8427-8c8b4de41cd3 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:00.221839 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-36a9d2bf-df52-4929-88a3-579456c9484e req-9a156992-4f88-4181-8427-8c8b4de41cd3 service nova] Creating event network-vif-deleted:9ab69acd-f0a3-4355-b203-2c48e39cc3a1 for instance b9a4910a-f117-4834-b699-498c6431b9bb on nch2-master-1 Jul 16 21:19:00.228216 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-36a9d2bf-df52-4929-88a3-579456c9484e req-9a156992-4f88-4181-8427-8c8b4de41cd3 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.035210 Jul 16 21:19:00.229402 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1289/2535] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:19:00 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:19:00.236109 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-f2b54195-ba51-434b-98b8-8fcedf0b21d9 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:19:00.237364 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-f2b54195-ba51-434b-98b8-8fcedf0b21d9 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] [instance: d265192b-536b-4884-9fe8-74cf20fc7ef9] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:19:00.241480 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-c1e34e9a-87f6-4ee3-b1bb-c82596f494b4 tempest-VolumesSnapshotsTestJSON-851680556 tempest-VolumesSnapshotsTestJSON-851680556-project] 10.222.0.16 "GET /compute/v2.1/os-snapshots" status: 200 len: 289 microversion: 2.1 time: 0.093792 Jul 16 21:19:00.242234 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1247/2536] 10.222.0.16 () {58 vars in 1139 bytes} [Sat Jul 16 21:19:00 2022] GET /compute/v2.1/os-snapshots => generated 289 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:19:00.253823 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f2b54195-ba51-434b-98b8-8fcedf0b21d9 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:00.254555 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f2b54195-ba51-434b-98b8-8fcedf0b21d9 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:00.255164 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-0ec0357b-b314-4a4e-8138-ed787264bd63 tempest-VolumesSnapshotsTestJSON-851680556 tempest-VolumesSnapshotsTestJSON-851680556-project] Calling method '.version_select at 0x7fdd9d221f70>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:19:00.255800 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f2b54195-ba51-434b-98b8-8fcedf0b21d9 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:00.256433 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f2b54195-ba51-434b-98b8-8fcedf0b21d9 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:00.308928 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-0ec0357b-b314-4a4e-8138-ed787264bd63 tempest-VolumesSnapshotsTestJSON-851680556 tempest-VolumesSnapshotsTestJSON-851680556-project] 10.222.0.16 "GET /compute/v2.1/os-snapshots/b6ec78c5-d133-485b-8009-7628a73d4739" status: 200 len: 286 microversion: 2.1 time: 0.059580 Jul 16 21:19:00.310064 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1248/2537] 10.222.0.16 () {58 vars in 1250 bytes} [Sat Jul 16 21:19:00 2022] GET /compute/v2.1/os-snapshots/b6ec78c5-d133-485b-8009-7628a73d4739 => generated 286 bytes in 61 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 16 21:19:00.523522 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-f2b54195-ba51-434b-98b8-8fcedf0b21d9 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'daf3c9e6634e4ae1bfb492eae1f44c93', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '24c64db8082648c8837c16352252f96d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:19:00.539643 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-f2b54195-ba51-434b-98b8-8fcedf0b21d9 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] 10.222.0.16 "GET /compute/v2.1/servers/d265192b-536b-4884-9fe8-74cf20fc7ef9" status: 200 len: 1307 microversion: 2.1 time: 0.308216 Jul 16 21:19:00.556745 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1290/2538] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:19:00 2022] GET /compute/v2.1/servers/d265192b-536b-4884-9fe8-74cf20fc7ef9 => generated 1307 bytes in 327 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:19:00.557926 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-fbf82509-eca0-4427-b97f-a4acf1342368 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:19:00.558703 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-fbf82509-eca0-4427-b97f-a4acf1342368 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] [instance: b9a4910a-f117-4834-b699-498c6431b9bb] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:19:00.571893 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fbf82509-eca0-4427-b97f-a4acf1342368 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:00.573737 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fbf82509-eca0-4427-b97f-a4acf1342368 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:00.574702 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fbf82509-eca0-4427-b97f-a4acf1342368 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:00.575340 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fbf82509-eca0-4427-b97f-a4acf1342368 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:00.641392 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fbf82509-eca0-4427-b97f-a4acf1342368 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:00.642429 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fbf82509-eca0-4427-b97f-a4acf1342368 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:00.643146 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.objects.instance [None req-fbf82509-eca0-4427-b97f-a4acf1342368 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Lazy-loading 'fault' on Instance uuid b9a4910a-f117-4834-b699-498c6431b9bb {{(pid=138681) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:19:00.756956 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-fbf82509-eca0-4427-b97f-a4acf1342368 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '71e9304fc50447d6aac99703b5b6e4e1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e84496d78d4481e9d7648128868bca2', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:19:00.767457 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-fbf82509-eca0-4427-b97f-a4acf1342368 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] 10.222.0.16 "GET /compute/v2.1/servers/b9a4910a-f117-4834-b699-498c6431b9bb" status: 200 len: 1307 microversion: 2.1 time: 0.217741 Jul 16 21:19:00.768098 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1249/2539] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:19:00 2022] GET /compute/v2.1/servers/b9a4910a-f117-4834-b699-498c6431b9bb => generated 1307 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:19:01.126173 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-83936b53-17d8-4311-9f5a-b68a7cd78af2 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:19:01.127306 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-83936b53-17d8-4311-9f5a-b68a7cd78af2 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] [instance: c88b191a-1ae5-4952-8afa-14aee5567a7f] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:19:01.145753 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-83936b53-17d8-4311-9f5a-b68a7cd78af2 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:01.146381 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-83936b53-17d8-4311-9f5a-b68a7cd78af2 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:01.158522 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-83936b53-17d8-4311-9f5a-b68a7cd78af2 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:01.159746 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-83936b53-17d8-4311-9f5a-b68a7cd78af2 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:01.210797 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-bd150486-2007-4e9b-bffb-5e31eaeeb048 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:19:01.211472 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-bd150486-2007-4e9b-bffb-5e31eaeeb048 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] [instance: e7598b4a-f70c-4186-88d1-6fd0057f8656] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:19:01.225786 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-bd150486-2007-4e9b-bffb-5e31eaeeb048 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:01.229510 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-bd150486-2007-4e9b-bffb-5e31eaeeb048 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:01.233105 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-bd150486-2007-4e9b-bffb-5e31eaeeb048 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:01.233729 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-bd150486-2007-4e9b-bffb-5e31eaeeb048 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:01.506670 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-bd150486-2007-4e9b-bffb-5e31eaeeb048 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f5bc1f2209f4eb186d80e385b67c774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '65d5bcfa02af4e8284ba1342197e2860', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:19:01.555715 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-bd150486-2007-4e9b-bffb-5e31eaeeb048 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] 10.222.0.16 "GET /compute/v2.1/servers/e7598b4a-f70c-4186-88d1-6fd0057f8656" status: 200 len: 1477 microversion: 2.1 time: 0.348090 Jul 16 21:19:01.556431 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1250/2540] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:19:01 2022] GET /compute/v2.1/servers/e7598b4a-f70c-4186-88d1-6fd0057f8656 => generated 1477 bytes in 353 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:19:01.569372 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-bfbacb64-0daf-4c6b-a5ef-df67ae8e3e1f tempest-VolumesSnapshotsTestJSON-851680556 tempest-VolumesSnapshotsTestJSON-851680556-project] Calling method '.version_select at 0x7fdd9d879550>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:19:01.651299 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-bfbacb64-0daf-4c6b-a5ef-df67ae8e3e1f tempest-VolumesSnapshotsTestJSON-851680556 tempest-VolumesSnapshotsTestJSON-851680556-project] 10.222.0.16 "GET /compute/v2.1/os-snapshots/b6ec78c5-d133-485b-8009-7628a73d4739" status: 200 len: 286 microversion: 2.1 time: 0.092926 Jul 16 21:19:01.656402 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1251/2541] 10.222.0.16 () {58 vars in 1250 bytes} [Sat Jul 16 21:19:01 2022] GET /compute/v2.1/os-snapshots/b6ec78c5-d133-485b-8009-7628a73d4739 => generated 286 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:19:01.663525 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-9e30ec81-acfa-4f9b-b2b0-75ba47a3016a tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:19:01.664243 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-9e30ec81-acfa-4f9b-b2b0-75ba47a3016a tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] [instance: d265192b-536b-4884-9fe8-74cf20fc7ef9] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:19:01.672595 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9e30ec81-acfa-4f9b-b2b0-75ba47a3016a tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:01.673116 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9e30ec81-acfa-4f9b-b2b0-75ba47a3016a tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:01.674078 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9e30ec81-acfa-4f9b-b2b0-75ba47a3016a tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:01.676341 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9e30ec81-acfa-4f9b-b2b0-75ba47a3016a tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:01.687330 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-83936b53-17d8-4311-9f5a-b68a7cd78af2 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd0008fffad8e4708b2c01e03da92c47d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e353258f9c3048ef84683294e2b8579d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:19:01.701478 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-83936b53-17d8-4311-9f5a-b68a7cd78af2 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] 10.222.0.16 "GET /compute/v2.1/servers/c88b191a-1ae5-4952-8afa-14aee5567a7f" status: 200 len: 1575 microversion: 2.1 time: 0.580167 Jul 16 21:19:01.706054 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1291/2542] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:19:01 2022] GET /compute/v2.1/servers/c88b191a-1ae5-4952-8afa-14aee5567a7f => generated 1575 bytes in 585 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:19:01.788806 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-9e65c322-7a3e-4c99-9fd4-cc9e9dc80034 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:19:01.789427 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-9e65c322-7a3e-4c99-9fd4-cc9e9dc80034 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] [instance: b9a4910a-f117-4834-b699-498c6431b9bb] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:19:01.805027 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9e65c322-7a3e-4c99-9fd4-cc9e9dc80034 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:01.805537 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9e65c322-7a3e-4c99-9fd4-cc9e9dc80034 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:01.806494 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9e65c322-7a3e-4c99-9fd4-cc9e9dc80034 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:01.806945 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9e65c322-7a3e-4c99-9fd4-cc9e9dc80034 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:01.827609 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-9e65c322-7a3e-4c99-9fd4-cc9e9dc80034 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] HTTP exception thrown: Instance b9a4910a-f117-4834-b699-498c6431b9bb could not be found. Jul 16 21:19:01.829617 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-9e65c322-7a3e-4c99-9fd4-cc9e9dc80034 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Request method failure captured: Jul 16 21:19:01.829617 ncdv-master-1 devstack@n-api.service[138680]: request: GET /compute/v2.1/servers/b9a4910a-f117-4834-b699-498c6431b9bb HTTP/1.1 Jul 16 21:19:01.829617 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:19:01.829617 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:19:01.829617 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:19:01.829617 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 0 Jul 16 21:19:01.829617 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:19:01.829617 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:19:01.829617 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:19:01.829617 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:19:01.829617 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0yq45M5tdqjee-T32BMtcEMJOIgSLcQky5K2YTxccde0BMgVBvCXGDb0O3En8ZLM5b4_J77ztcSDPXbQynL5j5ji8vdJgPr_1r_BGJmnN0395RLQHDO2cSXFUsZiVB7_gRTfKV68KCmigPiVLqn2Ocm8smB3OqtAekwKb3BDVve30Pc Jul 16 21:19:01.829617 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:19:01.829617 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:19:01.829617 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:19:01.829617 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:19:01.829617 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:19:01.829617 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:19:01.829617 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: 7e84496d78d4481e9d7648128868bca2 Jul 16 21:19:01.829617 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-ListImageFiltersTestJSON-1083290149 Jul 16 21:19:01.829617 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:19:01.829617 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:19:01.829617 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/7e84496d78d4481e9d7648128868bca2"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/7e84496d78d4481e9d7648128868bca2"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_7e84496d78d4481e9d7648128868bca2"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/7e84496d78d4481e9d7648128868bca2"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:19:01.829617 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-ListImageFiltersTestJSON-1083290149 Jul 16 21:19:01.829617 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: 7e84496d78d4481e9d7648128868bca2 Jul 16 21:19:01.829617 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-ListImageFiltersTestJSON-1083290149 Jul 16 21:19:01.829617 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-ListImageFiltersTestJSON-1083290149-project Jul 16 21:19:01.829617 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:19:01.829617 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:19:01.829617 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: 71e9304fc50447d6aac99703b5b6e4e1 Jul 16 21:19:01.829617 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-ListImageFiltersTestJSON-1083290149-project Jul 16 21:19:01.829617 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 21:19:01.829617 ncdv-master-1 devstack@n-api.service[138680]: exception: Instance b9a4910a-f117-4834-b699-498c6431b9bb could not be found. Jul 16 21:19:01.829617 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': 'b9a4910a-f117-4834-b699-498c6431b9bb'} Jul 16 21:19:01.829617 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:19:01.829617 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:19:01.829617 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:19:01.829617 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:19:01.829617 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:19:01.829617 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:19:01.829617 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:19:01.829617 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:19:01.829617 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:19:01.829617 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:19:01.829617 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance b9a4910a-f117-4834-b699-498c6431b9bb could not be found. Jul 16 21:19:01.829617 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:19:01.829617 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:19:01.829617 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:19:01.829617 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:19:01.829617 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:19:01.829617 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:19:01.829617 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:19:01.829617 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:19:01.829617 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:19:01.829617 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:19:01.829617 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:19:01.829617 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:19:01.829617 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:19:01.829617 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:19:01.829617 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:19:01.829617 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:19:01.829617 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance b9a4910a-f117-4834-b699-498c6431b9bb could not be found. Jul 16 21:19:01.829617 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:19:01.829617 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:19:01.829617 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:19:01.833846 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:19:01.833846 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:19:01.833846 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:19:01.833846 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:19:01.833846 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:19:01.833846 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance b9a4910a-f117-4834-b699-498c6431b9bb could not be found. Jul 16 21:19:01.833846 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:19:01.835529 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-9e65c322-7a3e-4c99-9fd4-cc9e9dc80034 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Returning 404 to user: Instance b9a4910a-f117-4834-b699-498c6431b9bb could not be found. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:19:01.837780 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-9e65c322-7a3e-4c99-9fd4-cc9e9dc80034 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] 10.222.0.16 "GET /compute/v2.1/servers/b9a4910a-f117-4834-b699-498c6431b9bb" status: 404 len: 111 microversion: 2.1 time: 0.051897 Jul 16 21:19:01.841991 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1292/2543] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:19:01 2022] GET /compute/v2.1/servers/b9a4910a-f117-4834-b699-498c6431b9bb => generated 111 bytes in 58 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:19:01.847455 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-7b0948f7-9049-4ec1-a6c4-c665992b41cb tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Calling method '.version_select at 0x7fdd9d321670>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:19:01.920595 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-9e30ec81-acfa-4f9b-b2b0-75ba47a3016a tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'daf3c9e6634e4ae1bfb492eae1f44c93', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '24c64db8082648c8837c16352252f96d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:19:01.929926 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-9e30ec81-acfa-4f9b-b2b0-75ba47a3016a tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] 10.222.0.16 "GET /compute/v2.1/servers/d265192b-536b-4884-9fe8-74cf20fc7ef9" status: 200 len: 1487 microversion: 2.1 time: 0.271992 Jul 16 21:19:01.931312 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1252/2544] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:19:01 2022] GET /compute/v2.1/servers/d265192b-536b-4884-9fe8-74cf20fc7ef9 => generated 1487 bytes in 274 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:19:01.956488 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-2683995e-8af0-48d1-8cb1-fc2961130f67 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Action: 'action', calling method: >, body: {"createImage": {"name": "tempest-test-snap-706056877"}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:19:01.958314 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-2683995e-8af0-48d1-8cb1-fc2961130f67 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] [instance: d265192b-536b-4884-9fe8-74cf20fc7ef9] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:19:01.982600 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2683995e-8af0-48d1-8cb1-fc2961130f67 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:01.984153 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2683995e-8af0-48d1-8cb1-fc2961130f67 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:01.987038 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2683995e-8af0-48d1-8cb1-fc2961130f67 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:01.988610 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2683995e-8af0-48d1-8cb1-fc2961130f67 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:02.500713 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell0:SELECT=34 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:19:02.852995 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-2683995e-8af0-48d1-8cb1-fc2961130f67 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] 10.222.0.16 "POST /compute/v2.1/servers/d265192b-536b-4884-9fe8-74cf20fc7ef9/action" status: 202 len: 0 microversion: 2.1 time: 0.904860 Jul 16 21:19:02.867376 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1253/2545] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 21:19:01 2022] POST /compute/v2.1/servers/d265192b-536b-4884-9fe8-74cf20fc7ef9/action => generated 0 bytes in 921 msecs (HTTP/1.1 202) 10 headers in 450 bytes (1 switches on core 0) Jul 16 21:19:02.888940 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-98dd3747-e23f-48af-89b4-fc15690803aa req-c5c332a6-78ce-4ac6-b43a-35949cc8e4ff service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "e7598b4a-f70c-4186-88d1-6fd0057f8656", "name": "network-vif-plugged", "status": "completed", "tag": "5e72e822-7cdd-445c-8d25-dc81877dc718"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:19:02.916841 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-98dd3747-e23f-48af-89b4-fc15690803aa req-c5c332a6-78ce-4ac6-b43a-35949cc8e4ff service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:02.917365 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-98dd3747-e23f-48af-89b4-fc15690803aa req-c5c332a6-78ce-4ac6-b43a-35949cc8e4ff service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:02.945137 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-98dd3747-e23f-48af-89b4-fc15690803aa req-c5c332a6-78ce-4ac6-b43a-35949cc8e4ff service nova] Creating event network-vif-plugged:5e72e822-7cdd-445c-8d25-dc81877dc718 for instance e7598b4a-f70c-4186-88d1-6fd0057f8656 on nch1-master-1 Jul 16 21:19:02.956455 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-98dd3747-e23f-48af-89b4-fc15690803aa req-c5c332a6-78ce-4ac6-b43a-35949cc8e4ff service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.082470 Jul 16 21:19:02.957733 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1254/2546] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:19:02 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:19:02.974033 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-f38d088b-9fc0-4afb-9ec1-70e0c5698eb8 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:19:02.975445 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-f38d088b-9fc0-4afb-9ec1-70e0c5698eb8 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] [instance: e7598b4a-f70c-4186-88d1-6fd0057f8656] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:19:02.982786 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_api:SELECT=94,nova_api:INSERT=3,nova_api:DELETE=2,nova_api:UPDATE=1 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:19:02.985413 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f38d088b-9fc0-4afb-9ec1-70e0c5698eb8 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:02.986366 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f38d088b-9fc0-4afb-9ec1-70e0c5698eb8 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:02.988151 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f38d088b-9fc0-4afb-9ec1-70e0c5698eb8 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:02.989661 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f38d088b-9fc0-4afb-9ec1-70e0c5698eb8 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:03.062510 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-7b0948f7-9049-4ec1-a6c4-c665992b41cb tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] 10.222.0.16 "DELETE /compute/v2.1/images/90124cc7-665a-40b9-adc1-59b89a3d6840" status: 204 len: 0 microversion: 2.1 time: 1.219287 Jul 16 21:19:03.063450 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1293/2547] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:19:01 2022] DELETE /compute/v2.1/images/90124cc7-665a-40b9-adc1-59b89a3d6840 => generated 0 bytes in 1221 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 16 21:19:03.073613 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-689c9833-5a62-4de3-80ee-75a56197e0c6 tempest-VolumesSnapshotsTestJSON-851680556 tempest-VolumesSnapshotsTestJSON-851680556-project] Calling method '.version_select at 0x7fdda2bc8b80>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:19:03.138719 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-689c9833-5a62-4de3-80ee-75a56197e0c6 tempest-VolumesSnapshotsTestJSON-851680556 tempest-VolumesSnapshotsTestJSON-851680556-project] 10.222.0.16 "GET /compute/v2.1/os-snapshots/b6ec78c5-d133-485b-8009-7628a73d4739" status: 200 len: 287 microversion: 2.1 time: 0.073743 Jul 16 21:19:03.144706 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1294/2548] 10.222.0.16 () {58 vars in 1250 bytes} [Sat Jul 16 21:19:03 2022] GET /compute/v2.1/os-snapshots/b6ec78c5-d133-485b-8009-7628a73d4739 => generated 287 bytes in 80 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:19:03.150916 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-59fabee2-2204-4f0c-8711-2bde553120b1 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:19:03.154507 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-59fabee2-2204-4f0c-8711-2bde553120b1 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] [instance: c88b191a-1ae5-4952-8afa-14aee5567a7f] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:19:03.172327 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-59fabee2-2204-4f0c-8711-2bde553120b1 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:03.173323 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-59fabee2-2204-4f0c-8711-2bde553120b1 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:03.174378 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-59fabee2-2204-4f0c-8711-2bde553120b1 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:03.175335 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-59fabee2-2204-4f0c-8711-2bde553120b1 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:03.468831 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-f38d088b-9fc0-4afb-9ec1-70e0c5698eb8 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f5bc1f2209f4eb186d80e385b67c774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '65d5bcfa02af4e8284ba1342197e2860', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:19:03.481115 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-f38d088b-9fc0-4afb-9ec1-70e0c5698eb8 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] 10.222.0.16 "GET /compute/v2.1/servers/e7598b4a-f70c-4186-88d1-6fd0057f8656" status: 200 len: 1477 microversion: 2.1 time: 0.519562 Jul 16 21:19:03.484489 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1255/2549] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:19:02 2022] GET /compute/v2.1/servers/e7598b4a-f70c-4186-88d1-6fd0057f8656 => generated 1477 bytes in 526 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:19:03.502512 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-98af916c-9944-434f-a894-3bf1db9006a9 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Calling method '.version_select at 0x7fdd9d0e2790>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:19:03.568625 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-98af916c-9944-434f-a894-3bf1db9006a9 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] 10.222.0.16 "GET /compute/v2.1/images/3cd0a9f0-af2b-4760-a6d2-51f198a2555a" status: 200 len: 1413 microversion: 2.1 time: 0.082821 Jul 16 21:19:03.575060 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1256/2550] 10.222.0.16 () {58 vars in 1232 bytes} [Sat Jul 16 21:19:03 2022] GET /compute/v2.1/images/3cd0a9f0-af2b-4760-a6d2-51f198a2555a => generated 1413 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:19:03.575060 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-01904d5a-bafa-4c9b-90d6-5ea0f5653725 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Calling method '.version_select at 0x7fdd9d519f70>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:19:03.657523 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-59fabee2-2204-4f0c-8711-2bde553120b1 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd0008fffad8e4708b2c01e03da92c47d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e353258f9c3048ef84683294e2b8579d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:19:03.677203 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-59fabee2-2204-4f0c-8711-2bde553120b1 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] 10.222.0.16 "GET /compute/v2.1/servers/c88b191a-1ae5-4952-8afa-14aee5567a7f" status: 200 len: 1573 microversion: 2.1 time: 0.530421 Jul 16 21:19:03.689880 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1295/2551] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:19:03 2022] GET /compute/v2.1/servers/c88b191a-1ae5-4952-8afa-14aee5567a7f => generated 1573 bytes in 543 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:19:03.718067 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-2b346903-4553-4f28-9a57-1ffcc26ebce4 tempest-VolumesSnapshotsTestJSON-851680556 tempest-VolumesSnapshotsTestJSON-851680556-project] Calling method '.version_select at 0x7fdda2ae0940>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:19:03.738170 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell0:SELECT=42 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:19:03.897289 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-2b346903-4553-4f28-9a57-1ffcc26ebce4 tempest-VolumesSnapshotsTestJSON-851680556 tempest-VolumesSnapshotsTestJSON-851680556-project] 10.222.0.16 "DELETE /compute/v2.1/os-snapshots/b6ec78c5-d133-485b-8009-7628a73d4739" status: 202 len: 0 microversion: 2.1 time: 0.202882 Jul 16 21:19:03.897480 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1296/2552] 10.222.0.16 () {58 vars in 1253 bytes} [Sat Jul 16 21:19:03 2022] DELETE /compute/v2.1/os-snapshots/b6ec78c5-d133-485b-8009-7628a73d4739 => generated 0 bytes in 207 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 21:19:03.903661 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-6135c751-0fbb-4795-b649-9e5da28a616f tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Calling method '.version_select at 0x7fdda2bc5e50>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:19:03.957786 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-6135c751-0fbb-4795-b649-9e5da28a616f tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] 10.222.0.16 "GET /compute/v2.1/images/3cd0a9f0-af2b-4760-a6d2-51f198a2555a" status: 200 len: 1413 microversion: 2.1 time: 0.058481 Jul 16 21:19:03.958435 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1297/2553] 10.222.0.16 () {58 vars in 1232 bytes} [Sat Jul 16 21:19:03 2022] GET /compute/v2.1/images/3cd0a9f0-af2b-4760-a6d2-51f198a2555a => generated 1413 bytes in 60 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:19:03.965579 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-8e8f0db0-2193-41ba-8cd4-bc0801772765 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Action: 'action', calling method: >, body: {"addSecurityGroup": {"name": "tempest-SecurityGroupsTestJSON-securitygroup-25448692"}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:19:03.966138 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-8e8f0db0-2193-41ba-8cd4-bc0801772765 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] [instance: c88b191a-1ae5-4952-8afa-14aee5567a7f] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:19:03.972780 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8e8f0db0-2193-41ba-8cd4-bc0801772765 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:03.973391 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8e8f0db0-2193-41ba-8cd4-bc0801772765 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:03.974236 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8e8f0db0-2193-41ba-8cd4-bc0801772765 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:03.974668 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8e8f0db0-2193-41ba-8cd4-bc0801772765 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:03.979769 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:19:04.126828 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.network.security_group_api [None req-8e8f0db0-2193-41ba-8cd4-bc0801772765 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Adding security group c176d065-4e8f-45fb-99d9-e12db1c30218 to port 9b680cff-b0fe-4eb6-91dd-54ab2451c6f2 Jul 16 21:19:04.515584 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-01904d5a-bafa-4c9b-90d6-5ea0f5653725 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] 10.222.0.16 "DELETE /compute/v2.1/images/c201a74f-a305-4be4-9ee5-cd6c3f65f771" status: 204 len: 0 microversion: 2.1 time: 0.944909 Jul 16 21:19:04.516451 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1257/2554] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:19:03 2022] DELETE /compute/v2.1/images/c201a74f-a305-4be4-9ee5-cd6c3f65f771 => generated 0 bytes in 946 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 16 21:19:04.787373 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-8e8f0db0-2193-41ba-8cd4-bc0801772765 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] 10.222.0.16 "POST /compute/v2.1/servers/c88b191a-1ae5-4952-8afa-14aee5567a7f/action" status: 202 len: 0 microversion: 2.1 time: 0.827742 Jul 16 21:19:04.788443 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1298/2555] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 21:19:03 2022] POST /compute/v2.1/servers/c88b191a-1ae5-4952-8afa-14aee5567a7f/action => generated 0 bytes in 829 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 21:19:04.793617 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-09042272-bb0b-408c-9805-caaa1a100ae5 tempest-VolumesSnapshotsTestJSON-851680556 tempest-VolumesSnapshotsTestJSON-851680556-project] Calling method '.version_select at 0x7fdd9cc353a0>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:19:04.806701 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-e981bf20-695e-4bd9-9581-0e6c9cc362e6 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-tempest.common.compute-instance-1558455757", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452", "min_count": 2, "networks": [{"uuid": "f5bb1e92-80f6-44be-a5b9-c44a678f4b55"}]}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:19:04.878985 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-09042272-bb0b-408c-9805-caaa1a100ae5 tempest-VolumesSnapshotsTestJSON-851680556 tempest-VolumesSnapshotsTestJSON-851680556-project] 10.222.0.16 "GET /compute/v2.1/os-snapshots/b6ec78c5-d133-485b-8009-7628a73d4739" status: 200 len: 286 microversion: 2.1 time: 0.089434 Jul 16 21:19:04.880508 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1299/2556] 10.222.0.16 () {58 vars in 1250 bytes} [Sat Jul 16 21:19:04 2022] GET /compute/v2.1/os-snapshots/b6ec78c5-d133-485b-8009-7628a73d4739 => generated 286 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:19:04.886036 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-e73361c8-6b0e-46dd-bb7a-5cdeb8b85bb4 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Calling method '.version_select at 0x7fdd9c6b99d0>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:19:04.940016 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.network.neutron [None req-e981bf20-695e-4bd9-9581-0e6c9cc362e6 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] validate_networks() for [('f5bb1e92-80f6-44be-a5b9-c44a678f4b55', None, None, None, None, None)] {{(pid=138681) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 16 21:19:05.306677 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-e73361c8-6b0e-46dd-bb7a-5cdeb8b85bb4 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] 10.222.0.16 "DELETE /compute/v2.1/images/3cd0a9f0-af2b-4760-a6d2-51f198a2555a" status: 204 len: 0 microversion: 2.1 time: 0.422766 Jul 16 21:19:05.308309 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1300/2557] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:19:04 2022] DELETE /compute/v2.1/images/3cd0a9f0-af2b-4760-a6d2-51f198a2555a => generated 0 bytes in 427 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 16 21:19:05.318785 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-4bfef110-3088-4d34-a3e3-8d4634403034 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:19:05.333903 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-4bfef110-3088-4d34-a3e3-8d4634403034 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] [instance: e7598b4a-f70c-4186-88d1-6fd0057f8656] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:19:05.347642 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4bfef110-3088-4d34-a3e3-8d4634403034 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:05.355387 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4bfef110-3088-4d34-a3e3-8d4634403034 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:05.356824 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4bfef110-3088-4d34-a3e3-8d4634403034 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:05.360780 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4bfef110-3088-4d34-a3e3-8d4634403034 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:05.637448 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-4bfef110-3088-4d34-a3e3-8d4634403034 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f5bc1f2209f4eb186d80e385b67c774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '65d5bcfa02af4e8284ba1342197e2860', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:19:05.638607 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-e981bf20-695e-4bd9-9581-0e6c9cc362e6 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Flavor limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 16 21:19:05.639125 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-e981bf20-695e-4bd9-9581-0e6c9cc362e6 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Image limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 16 21:19:05.640257 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-e981bf20-695e-4bd9-9581-0e6c9cc362e6 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Flavor pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 16 21:19:05.640681 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-e981bf20-695e-4bd9-9581-0e6c9cc362e6 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Image pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 16 21:19:05.642100 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-e981bf20-695e-4bd9-9581-0e6c9cc362e6 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 16 21:19:05.652385 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-4bfef110-3088-4d34-a3e3-8d4634403034 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] 10.222.0.16 "GET /compute/v2.1/servers/e7598b4a-f70c-4186-88d1-6fd0057f8656" status: 200 len: 1463 microversion: 2.1 time: 0.339030 Jul 16 21:19:05.653005 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1301/2558] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:19:05 2022] GET /compute/v2.1/servers/e7598b4a-f70c-4186-88d1-6fd0057f8656 => generated 1463 bytes in 344 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:19:05.661737 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-e82e9787-d37d-42c0-be3b-6f73f3742fe6 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] Calling method '.version_select at 0x7fdda29d1550>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:19:05.666795 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e981bf20-695e-4bd9-9581-0e6c9cc362e6 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:05.667526 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e981bf20-695e-4bd9-9581-0e6c9cc362e6 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:05.668984 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e981bf20-695e-4bd9-9581-0e6c9cc362e6 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:05.669581 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e981bf20-695e-4bd9-9581-0e6c9cc362e6 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:05.694095 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-e981bf20-695e-4bd9-9581-0e6c9cc362e6 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Getting quotas for project a092ce9ef40943f6b6ace6cd28481b71. Resources: {'ram', 'instances', 'cores'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:19:05.700614 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-e981bf20-695e-4bd9-9581-0e6c9cc362e6 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Getting quotas for user eebb224a56814f92b12ebebebc6bc8a3 and project a092ce9ef40943f6b6ace6cd28481b71. Resources: {'ram', 'instances', 'cores'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:19:05.718840 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-e981bf20-695e-4bd9-9581-0e6c9cc362e6 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Going to run 2 instances... {{(pid=138681) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 16 21:19:05.721393 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-e981bf20-695e-4bd9-9581-0e6c9cc362e6 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] [instance: 21522211-8382-4958-ad2a-cff26f4f2305] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138681) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 16 21:19:05.838947 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-e981bf20-695e-4bd9-9581-0e6c9cc362e6 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] [instance: c16e5f09-f813-414e-b233-c08348078624] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9',instance=,instance_uuid=21522211-8382-4958-ad2a-cff26f4f2305,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138681) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 16 21:19:06.030286 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-e981bf20-695e-4bd9-9581-0e6c9cc362e6 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 1.512503 Jul 16 21:19:06.034684 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1258/2559] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 21:19:04 2022] POST /compute/v2.1/servers => generated 380 bytes in 1518 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 16 21:19:06.043698 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-a867c7fe-5631-4334-8269-0a186a756cea tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Calling method '.version_select at 0x7fdd9cfa6280>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:19:06.172842 ncdv-master-1 devstack@n-api.service[138681]: DEBUG neutronclient.v2_0.client [None req-a867c7fe-5631-4334-8269-0a186a756cea tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Error message: {"NeutronError": {"type": "SecurityGroupInUse", "message": "Security Group c176d065-4e8f-45fb-99d9-e12db1c30218 in use.", "detail": ""}} {{(pid=138681) _handle_fault_response /usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py:262}} Jul 16 21:19:06.174174 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-a867c7fe-5631-4334-8269-0a186a756cea tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] HTTP exception thrown: Security Group c176d065-4e8f-45fb-99d9-e12db1c30218 in use. Jul 16 21:19:06.174174 ncdv-master-1 devstack@n-api.service[138681]: Neutron server returns request_ids: ['req-b705a791-4770-40b8-bf78-12100b120539'] Jul 16 21:19:06.178143 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-a867c7fe-5631-4334-8269-0a186a756cea tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Request method failure captured: Jul 16 21:19:06.178143 ncdv-master-1 devstack@n-api.service[138681]: request: DELETE /compute/v2.1/os-security-groups/c176d065-4e8f-45fb-99d9-e12db1c30218 HTTP/1.1 Jul 16 21:19:06.178143 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:19:06.178143 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:19:06.178143 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:19:06.178143 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 21:19:06.178143 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:19:06.178143 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:19:06.178143 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:19:06.178143 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:19:06.178143 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0yrk-HsfYKsyovLSaEtNxaoXy7a3rLGNlsRPiPP76ZvvrKkKyGd-uekosXQxpFgkRnItBdviN4xr49Rerg5AyvLSzsyRY-Um0r3BRAvyZrzmCDYUpWbXk1uyEIb3Af4U3Y3nXIJgJAuDVx9z2NGSvEGCsGz12WlyyGCNfVSADFcYJP8 Jul 16 21:19:06.178143 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:19:06.178143 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:19:06.178143 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:19:06.178143 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:19:06.178143 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:19:06.178143 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:19:06.178143 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: e353258f9c3048ef84683294e2b8579d Jul 16 21:19:06.178143 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-SecurityGroupsTestJSON-438340169 Jul 16 21:19:06.178143 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:19:06.178143 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:19:06.178143 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/e353258f9c3048ef84683294e2b8579d"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/e353258f9c3048ef84683294e2b8579d"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_e353258f9c3048ef84683294e2b8579d"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/e353258f9c3048ef84683294e2b8579d"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:19:06.178143 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-SecurityGroupsTestJSON-438340169 Jul 16 21:19:06.178143 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: e353258f9c3048ef84683294e2b8579d Jul 16 21:19:06.178143 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-SecurityGroupsTestJSON-438340169 Jul 16 21:19:06.178143 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-SecurityGroupsTestJSON-438340169-project Jul 16 21:19:06.178143 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:19:06.178143 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:19:06.178143 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: d0008fffad8e4708b2c01e03da92c47d Jul 16 21:19:06.178143 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-SecurityGroupsTestJSON-438340169-project Jul 16 21:19:06.178143 ncdv-master-1 devstack@n-api.service[138681]: method: Jul 16 21:19:06.178143 ncdv-master-1 devstack@n-api.service[138681]: exception: Security Group c176d065-4e8f-45fb-99d9-e12db1c30218 in use. Jul 16 21:19:06.178143 ncdv-master-1 devstack@n-api.service[138681]: Neutron server returns request_ids: ['req-b705a791-4770-40b8-bf78-12100b120539'] Jul 16 21:19:06.178143 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': 'c176d065-4e8f-45fb-99d9-e12db1c30218'} Jul 16 21:19:06.178143 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:19:06.178143 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:19:06.178143 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/security_group_api.py", line 359, in destroy Jul 16 21:19:06.178143 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi neutron.delete_security_group(security_group['id']) Jul 16 21:19:06.178143 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 197, in wrapper Jul 16 21:19:06.178143 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Jul 16 21:19:06.178143 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 1045, in delete_security_group Jul 16 21:19:06.178143 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return self.delete(self.security_group_path % (security_group)) Jul 16 21:19:06.178143 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 197, in wrapper Jul 16 21:19:06.178143 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Jul 16 21:19:06.178143 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 352, in delete Jul 16 21:19:06.178143 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return self.retry_request("DELETE", action, body=body, Jul 16 21:19:06.178143 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 197, in wrapper Jul 16 21:19:06.178143 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Jul 16 21:19:06.178143 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 333, in retry_request Jul 16 21:19:06.178143 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return self.do_request(method, action, body=body, Jul 16 21:19:06.178143 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 197, in wrapper Jul 16 21:19:06.178143 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Jul 16 21:19:06.178143 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 297, in do_request Jul 16 21:19:06.178143 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi self._handle_fault_response(status_code, replybody, resp) Jul 16 21:19:06.178143 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 197, in wrapper Jul 16 21:19:06.178143 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Jul 16 21:19:06.178143 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 272, in _handle_fault_response Jul 16 21:19:06.178143 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi exception_handler_v20(status_code, error_body) Jul 16 21:19:06.178143 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 90, in exception_handler_v20 Jul 16 21:19:06.178143 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise client_exc(message=error_message, Jul 16 21:19:06.178143 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi neutronclient.common.exceptions.Conflict: Security Group c176d065-4e8f-45fb-99d9-e12db1c30218 in use. Jul 16 21:19:06.188628 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-b705a791-4770-40b8-bf78-12100b120539'] Jul 16 21:19:06.188628 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:19:06.188628 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:19:06.188628 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:19:06.188628 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:19:06.188628 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/security_groups.py", line 179, in delete Jul 16 21:19:06.188628 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi security_group_api.destroy(context, security_group) Jul 16 21:19:06.188628 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/security_group_api.py", line 364, in destroy Jul 16 21:19:06.188628 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.Invalid(str(e)) Jul 16 21:19:06.188628 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.Invalid: Security Group c176d065-4e8f-45fb-99d9-e12db1c30218 in use. Jul 16 21:19:06.188628 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-b705a791-4770-40b8-bf78-12100b120539'] Jul 16 21:19:06.188628 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:19:06.188628 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:19:06.188628 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:19:06.188628 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:19:06.188628 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:19:06.188628 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:19:06.188628 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:19:06.188628 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:19:06.188628 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Jul 16 21:19:06.188628 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Jul 16 21:19:06.188628 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:19:06.188628 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:19:06.188628 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/security_groups.py", line 183, in delete Jul 16 21:19:06.188628 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=exp.format_message()) Jul 16 21:19:06.188628 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Security Group c176d065-4e8f-45fb-99d9-e12db1c30218 in use. Jul 16 21:19:06.188628 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-b705a791-4770-40b8-bf78-12100b120539'] Jul 16 21:19:06.188628 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:19:06.188628 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:19:06.188628 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:19:06.188628 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:19:06.188628 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:19:06.188628 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:19:06.188628 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:19:06.188628 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:19:06.188628 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Security Group c176d065-4e8f-45fb-99d9-e12db1c30218 in use. Jul 16 21:19:06.188628 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-b705a791-4770-40b8-bf78-12100b120539'] Jul 16 21:19:06.188628 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:19:06.202386 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-a867c7fe-5631-4334-8269-0a186a756cea tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Returning 400 to user: Security Group c176d065-4e8f-45fb-99d9-e12db1c30218 in use. Jul 16 21:19:06.202386 ncdv-master-1 devstack@n-api.service[138681]: Neutron server returns request_ids: ['req-b705a791-4770-40b8-bf78-12100b120539'] {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:19:06.203753 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-a867c7fe-5631-4334-8269-0a186a756cea tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] 10.222.0.16 "DELETE /compute/v2.1/os-security-groups/c176d065-4e8f-45fb-99d9-e12db1c30218" status: 400 len: 185 microversion: 2.1 time: 0.165644 Jul 16 21:19:06.213236 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1259/2560] 10.222.0.16 () {58 vars in 1271 bytes} [Sat Jul 16 21:19:06 2022] DELETE /compute/v2.1/os-security-groups/c176d065-4e8f-45fb-99d9-e12db1c30218 => generated 185 bytes in 178 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Jul 16 21:19:06.225439 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-9409f00f-7627-4c96-81ac-efb24411d7f7 req-81bbfb7c-cdcf-48e0-9923-8f1856b198f6 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "c88b191a-1ae5-4952-8afa-14aee5567a7f", "tag": "9b680cff-b0fe-4eb6-91dd-54ab2451c6f2"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:19:06.255065 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-9409f00f-7627-4c96-81ac-efb24411d7f7 req-81bbfb7c-cdcf-48e0-9923-8f1856b198f6 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:06.255632 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-9409f00f-7627-4c96-81ac-efb24411d7f7 req-81bbfb7c-cdcf-48e0-9923-8f1856b198f6 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:06.276622 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-9409f00f-7627-4c96-81ac-efb24411d7f7 req-81bbfb7c-cdcf-48e0-9923-8f1856b198f6 service nova] Creating event network-changed:9b680cff-b0fe-4eb6-91dd-54ab2451c6f2 for instance c88b191a-1ae5-4952-8afa-14aee5567a7f on nch1-master-1 Jul 16 21:19:06.286131 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-9409f00f-7627-4c96-81ac-efb24411d7f7 req-81bbfb7c-cdcf-48e0-9923-8f1856b198f6 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.070823 Jul 16 21:19:06.288169 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1260/2561] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:19:06 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 74 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:19:06.311266 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-82d4d511-4b35-400d-b2e8-88987983fadc tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Calling method '.version_select at 0x7fdd9cf979d0>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:19:06.336591 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-e82e9787-d37d-42c0-be3b-6f73f3742fe6 tempest-ListImageFiltersTestJSON-1083290149 tempest-ListImageFiltersTestJSON-1083290149-project] 10.222.0.16 "DELETE /compute/v2.1/images/a7ab265a-4515-4850-a8d5-c86e67137b7e" status: 204 len: 0 microversion: 2.1 time: 0.676750 Jul 16 21:19:06.336591 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1302/2562] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:19:05 2022] DELETE /compute/v2.1/images/a7ab265a-4515-4850-a8d5-c86e67137b7e => generated 0 bytes in 680 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 16 21:19:06.349270 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-2ed78329-08fd-4900-b0f4-29584190a859 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:19:06.351637 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-2ed78329-08fd-4900-b0f4-29584190a859 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] [instance: e7598b4a-f70c-4186-88d1-6fd0057f8656] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:19:06.358821 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_api:SELECT=93,nova_api:DELETE=5,nova_api:INSERT=2 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:19:06.365825 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2ed78329-08fd-4900-b0f4-29584190a859 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:06.368055 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2ed78329-08fd-4900-b0f4-29584190a859 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:06.368403 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-82d4d511-4b35-400d-b2e8-88987983fadc tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] HTTP exception thrown: Image not found. Jul 16 21:19:06.368789 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2ed78329-08fd-4900-b0f4-29584190a859 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:06.369446 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2ed78329-08fd-4900-b0f4-29584190a859 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:06.370348 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-82d4d511-4b35-400d-b2e8-88987983fadc tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Request method failure captured: Jul 16 21:19:06.370348 ncdv-master-1 devstack@n-api.service[138681]: request: GET /compute/v2.1/images/3cd0a9f0-af2b-4760-a6d2-51f198a2555a HTTP/1.1 Jul 16 21:19:06.370348 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:19:06.370348 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:19:06.370348 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:19:06.370348 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 21:19:06.370348 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:19:06.370348 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:19:06.370348 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:19:06.370348 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:19:06.370348 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0yptIZULOjVMHbouVcndclXAnMKC6qm1VtMGoKOnZzI-fUPx3L5lhuz06xvHQzr3ooL8P53cH83yaSSJWcV0fJCtKGdbdY71S9WCM2F11ZTapGri39-eD6M8L5ZmeIRv6VTRaJtzqhadxTTGkGwVf5FzILXoamLnE3Jdb08lelPg3B4 Jul 16 21:19:06.370348 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:19:06.370348 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:19:06.370348 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:19:06.370348 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:19:06.370348 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:19:06.370348 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:19:06.370348 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: 24c64db8082648c8837c16352252f96d Jul 16 21:19:06.370348 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-ImagesTestJSON-1532827163 Jul 16 21:19:06.370348 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:19:06.370348 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:19:06.370348 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/24c64db8082648c8837c16352252f96d"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/24c64db8082648c8837c16352252f96d"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_24c64db8082648c8837c16352252f96d"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/24c64db8082648c8837c16352252f96d"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:19:06.370348 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-ImagesTestJSON-1532827163 Jul 16 21:19:06.370348 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: 24c64db8082648c8837c16352252f96d Jul 16 21:19:06.370348 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-ImagesTestJSON-1532827163 Jul 16 21:19:06.370348 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-ImagesTestJSON-1532827163-project Jul 16 21:19:06.370348 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:19:06.370348 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:19:06.370348 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: daf3c9e6634e4ae1bfb492eae1f44c93 Jul 16 21:19:06.370348 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-ImagesTestJSON-1532827163-project Jul 16 21:19:06.370348 ncdv-master-1 devstack@n-api.service[138681]: method: Jul 16 21:19:06.370348 ncdv-master-1 devstack@n-api.service[138681]: exception: Image not found. Jul 16 21:19:06.370348 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': '3cd0a9f0-af2b-4760-a6d2-51f198a2555a'} Jul 16 21:19:06.370348 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:19:06.370348 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:19:06.370348 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 285, in show Jul 16 21:19:06.370348 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi image = self._client.call(context, 2, 'get', args=(image_id,)) Jul 16 21:19:06.370348 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 191, in call Jul 16 21:19:06.370348 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi result = getattr(controller, method)(*args, **kwargs) Jul 16 21:19:06.370348 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/v2/images.py", line 197, in get Jul 16 21:19:06.370348 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return self._get(image_id) Jul 16 21:19:06.370348 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/utils.py", line 670, in inner Jul 16 21:19:06.370348 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return RequestIdProxy(wrapped(*args, **kwargs)) Jul 16 21:19:06.370348 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/v2/images.py", line 190, in _get Jul 16 21:19:06.370348 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi resp, body = self.http_client.get(url, headers=header) Jul 16 21:19:06.370348 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/adapter.py", line 395, in get Jul 16 21:19:06.370348 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return self.request(url, 'GET', **kwargs) Jul 16 21:19:06.370348 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 380, in request Jul 16 21:19:06.370348 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return self._handle_response(resp) Jul 16 21:19:06.370348 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 120, in _handle_response Jul 16 21:19:06.370348 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.from_response(resp, resp.content) Jul 16 21:19:06.370348 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi glanceclient.exc.HTTPNotFound: HTTP 404 Not Found: No image found with ID 3cd0a9f0-af2b-4760-a6d2-51f198a2555a Jul 16 21:19:06.370348 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:19:06.370348 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:19:06.370348 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:19:06.370348 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:19:06.370348 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/images.py", line 86, in show Jul 16 21:19:06.370348 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi image = self._image_api.get(context, id) Jul 16 21:19:06.370348 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 1205, in get Jul 16 21:19:06.370348 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return session.show(context, image_id, Jul 16 21:19:06.370348 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 287, in show Jul 16 21:19:06.370348 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi _reraise_translated_image_exception(image_id) Jul 16 21:19:06.370348 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 1031, in _reraise_translated_image_exception Jul 16 21:19:06.370348 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise new_exc.with_traceback(exc_trace) Jul 16 21:19:06.374944 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 285, in show Jul 16 21:19:06.374944 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi image = self._client.call(context, 2, 'get', args=(image_id,)) Jul 16 21:19:06.374944 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 191, in call Jul 16 21:19:06.374944 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi result = getattr(controller, method)(*args, **kwargs) Jul 16 21:19:06.374944 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/v2/images.py", line 197, in get Jul 16 21:19:06.374944 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return self._get(image_id) Jul 16 21:19:06.374944 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/utils.py", line 670, in inner Jul 16 21:19:06.374944 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return RequestIdProxy(wrapped(*args, **kwargs)) Jul 16 21:19:06.374944 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/v2/images.py", line 190, in _get Jul 16 21:19:06.374944 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi resp, body = self.http_client.get(url, headers=header) Jul 16 21:19:06.374944 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/adapter.py", line 395, in get Jul 16 21:19:06.374944 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return self.request(url, 'GET', **kwargs) Jul 16 21:19:06.374944 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 380, in request Jul 16 21:19:06.374944 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return self._handle_response(resp) Jul 16 21:19:06.374944 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 120, in _handle_response Jul 16 21:19:06.374944 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.from_response(resp, resp.content) Jul 16 21:19:06.374944 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.ImageNotFound: Image 3cd0a9f0-af2b-4760-a6d2-51f198a2555a could not be found. Jul 16 21:19:06.374944 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:19:06.374944 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:19:06.374944 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:19:06.374944 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:19:06.374944 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:19:06.374944 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:19:06.374944 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:19:06.374944 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:19:06.374944 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Jul 16 21:19:06.374944 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Jul 16 21:19:06.374944 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:19:06.374944 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:19:06.374944 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/images.py", line 89, in show Jul 16 21:19:06.374944 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=explanation) Jul 16 21:19:06.374944 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Image not found. Jul 16 21:19:06.374944 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:19:06.374944 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:19:06.374944 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:19:06.374944 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:19:06.374944 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:19:06.374944 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:19:06.374944 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:19:06.374944 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:19:06.374944 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Image not found. Jul 16 21:19:06.374944 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:19:06.374944 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-82d4d511-4b35-400d-b2e8-88987983fadc tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Returning 404 to user: Image not found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:19:06.374944 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-82d4d511-4b35-400d-b2e8-88987983fadc tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] 10.222.0.16 "GET /compute/v2.1/images/3cd0a9f0-af2b-4760-a6d2-51f198a2555a" status: 404 len: 62 microversion: 2.1 time: 0.083820 Jul 16 21:19:06.384026 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1261/2563] 10.222.0.16 () {58 vars in 1232 bytes} [Sat Jul 16 21:19:06 2022] GET /compute/v2.1/images/3cd0a9f0-af2b-4760-a6d2-51f198a2555a => generated 62 bytes in 86 msecs (HTTP/1.1 404) 9 headers in 378 bytes (1 switches on core 0) Jul 16 21:19:06.396872 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-81ac7787-7858-44a2-8f66-2e9b3d872a92 tempest-VolumesSnapshotsTestJSON-851680556 tempest-VolumesSnapshotsTestJSON-851680556-project] Calling method '.version_select at 0x7fdd9dd3bca0>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:19:06.465982 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-81ac7787-7858-44a2-8f66-2e9b3d872a92 tempest-VolumesSnapshotsTestJSON-851680556 tempest-VolumesSnapshotsTestJSON-851680556-project] 10.222.0.16 "GET /compute/v2.1/os-snapshots/b6ec78c5-d133-485b-8009-7628a73d4739" status: 200 len: 286 microversion: 2.1 time: 0.088044 Jul 16 21:19:06.470890 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1262/2564] 10.222.0.16 () {58 vars in 1250 bytes} [Sat Jul 16 21:19:06 2022] GET /compute/v2.1/os-snapshots/b6ec78c5-d133-485b-8009-7628a73d4739 => generated 286 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:19:06.476515 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-17048370-58dd-4224-ae25-3ba3c5bc4355 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:19:06.480988 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-17048370-58dd-4224-ae25-3ba3c5bc4355 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Policy check for os_compute_api:servers:allow_all_filters failed with credentials {'is_admin': False, 'user_id': 'eebb224a56814f92b12ebebebc6bc8a3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a092ce9ef40943f6b6ace6cd28481b71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:19:06.482655 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-17048370-58dd-4224-ae25-3ba3c5bc4355 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Searching by: {'deleted': False, 'project_id': 'a092ce9ef40943f6b6ace6cd28481b71'} {{(pid=138681) get_all /opt/stack/nova/nova/compute/api.py:2962}} Jul 16 21:19:06.521481 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-17048370-58dd-4224-ae25-3ba3c5bc4355 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:06.522120 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-17048370-58dd-4224-ae25-3ba3c5bc4355 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:06.524963 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-17048370-58dd-4224-ae25-3ba3c5bc4355 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:06.525424 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-17048370-58dd-4224-ae25-3ba3c5bc4355 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:06.790132 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-17048370-58dd-4224-ae25-3ba3c5bc4355 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] 10.222.0.16 "GET /compute/v2.1/servers" status: 200 len: 677 microversion: 2.1 time: 0.316367 Jul 16 21:19:06.796504 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1263/2565] 10.222.0.16 () {58 vars in 1124 bytes} [Sat Jul 16 21:19:06 2022] GET /compute/v2.1/servers => generated 677 bytes in 319 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:19:06.796655 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-ed8bc202-a760-49eb-a8b7-43c3f72482dd tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:19:06.797230 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-ed8bc202-a760-49eb-a8b7-43c3f72482dd tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] [instance: c88b191a-1ae5-4952-8afa-14aee5567a7f] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:19:06.807601 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ed8bc202-a760-49eb-a8b7-43c3f72482dd tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:06.808347 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ed8bc202-a760-49eb-a8b7-43c3f72482dd tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:06.810179 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ed8bc202-a760-49eb-a8b7-43c3f72482dd tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:06.810179 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ed8bc202-a760-49eb-a8b7-43c3f72482dd tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:06.880985 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-2ed78329-08fd-4900-b0f4-29584190a859 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f5bc1f2209f4eb186d80e385b67c774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '65d5bcfa02af4e8284ba1342197e2860', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:19:06.898592 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-2ed78329-08fd-4900-b0f4-29584190a859 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] 10.222.0.16 "GET /compute/v2.1/servers/e7598b4a-f70c-4186-88d1-6fd0057f8656" status: 200 len: 1463 microversion: 2.1 time: 0.558556 Jul 16 21:19:06.899265 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1303/2566] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:19:06 2022] GET /compute/v2.1/servers/e7598b4a-f70c-4186-88d1-6fd0057f8656 => generated 1463 bytes in 565 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:19:06.911601 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-ed8bc202-a760-49eb-a8b7-43c3f72482dd tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] [instance: c88b191a-1ae5-4952-8afa-14aee5567a7f] Going to try to terminate instance {{(pid=138681) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 16 21:19:06.921958 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-b7ef984e-337e-4cdc-b625-00bd7e6daffc tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:19:06.922605 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-b7ef984e-337e-4cdc-b625-00bd7e6daffc tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] [instance: d265192b-536b-4884-9fe8-74cf20fc7ef9] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:19:06.942864 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b7ef984e-337e-4cdc-b625-00bd7e6daffc tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:06.942864 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b7ef984e-337e-4cdc-b625-00bd7e6daffc tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:06.942864 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b7ef984e-337e-4cdc-b625-00bd7e6daffc tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:06.945515 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b7ef984e-337e-4cdc-b625-00bd7e6daffc tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:07.236282 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-b7ef984e-337e-4cdc-b625-00bd7e6daffc tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'daf3c9e6634e4ae1bfb492eae1f44c93', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '24c64db8082648c8837c16352252f96d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:19:07.248513 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-b7ef984e-337e-4cdc-b625-00bd7e6daffc tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] 10.222.0.16 "GET /compute/v2.1/servers/d265192b-536b-4884-9fe8-74cf20fc7ef9" status: 200 len: 1487 microversion: 2.1 time: 0.338765 Jul 16 21:19:07.253270 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1304/2567] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:19:06 2022] GET /compute/v2.1/servers/d265192b-536b-4884-9fe8-74cf20fc7ef9 => generated 1487 bytes in 353 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:19:07.266778 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-c8f4a793-67a6-44c3-80ba-325418a638ba tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:19:07.278216 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-c8f4a793-67a6-44c3-80ba-325418a638ba tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] [instance: c16e5f09-f813-414e-b233-c08348078624] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:19:07.418869 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-c8f4a793-67a6-44c3-80ba-325418a638ba tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eebb224a56814f92b12ebebebc6bc8a3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a092ce9ef40943f6b6ace6cd28481b71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:19:07.426747 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-c8f4a793-67a6-44c3-80ba-325418a638ba tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] 10.222.0.16 "GET /compute/v2.1/servers/c16e5f09-f813-414e-b233-c08348078624" status: 200 len: 1220 microversion: 2.1 time: 0.171882 Jul 16 21:19:07.441361 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1305/2568] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:19:07 2022] GET /compute/v2.1/servers/c16e5f09-f813-414e-b233-c08348078624 => generated 1220 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:19:07.446518 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-87f8c399-cae5-4010-81b0-989c116a778d tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Action: 'action', calling method: >, body: {"rebuild": {"imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9"}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:19:07.448526 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-87f8c399-cae5-4010-81b0-989c116a778d tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] [instance: e7598b4a-f70c-4186-88d1-6fd0057f8656] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:19:07.459391 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-87f8c399-cae5-4010-81b0-989c116a778d tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:07.462214 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-87f8c399-cae5-4010-81b0-989c116a778d tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:07.463957 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-87f8c399-cae5-4010-81b0-989c116a778d tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:07.464433 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-87f8c399-cae5-4010-81b0-989c116a778d tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:07.491326 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-ed8bc202-a760-49eb-a8b7-43c3f72482dd tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] 10.222.0.16 "DELETE /compute/v2.1/servers/c88b191a-1ae5-4952-8afa-14aee5567a7f" status: 204 len: 0 microversion: 2.1 time: 0.698316 Jul 16 21:19:07.491947 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1264/2569] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:19:06 2022] DELETE /compute/v2.1/servers/c88b191a-1ae5-4952-8afa-14aee5567a7f => generated 0 bytes in 700 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 16 21:19:07.501091 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-622d90f2-91fc-4945-bc2d-bedcf83e835b tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:19:07.507117 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-622d90f2-91fc-4945-bc2d-bedcf83e835b tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] [instance: d265192b-536b-4884-9fe8-74cf20fc7ef9] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:19:07.521331 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-622d90f2-91fc-4945-bc2d-bedcf83e835b tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:07.521807 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-622d90f2-91fc-4945-bc2d-bedcf83e835b tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:07.522884 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-622d90f2-91fc-4945-bc2d-bedcf83e835b tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:07.523394 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-622d90f2-91fc-4945-bc2d-bedcf83e835b tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:07.617157 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-622d90f2-91fc-4945-bc2d-bedcf83e835b tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] [instance: d265192b-536b-4884-9fe8-74cf20fc7ef9] Going to try to terminate instance {{(pid=138681) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 16 21:19:07.661577 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-87f8c399-cae5-4010-81b0-989c116a778d tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Flavor limits 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 16 21:19:07.662012 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-87f8c399-cae5-4010-81b0-989c116a778d tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Image limits 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 16 21:19:07.665900 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-87f8c399-cae5-4010-81b0-989c116a778d tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Flavor pref 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 16 21:19:07.666857 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-87f8c399-cae5-4010-81b0-989c116a778d tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Image pref 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 16 21:19:07.667706 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-87f8c399-cae5-4010-81b0-989c116a778d tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 16 21:19:08.040497 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-87f8c399-cae5-4010-81b0-989c116a778d tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] [instance: e7598b4a-f70c-4186-88d1-6fd0057f8656] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:19:08.047956 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-87f8c399-cae5-4010-81b0-989c116a778d tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:08.048871 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-87f8c399-cae5-4010-81b0-989c116a778d tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:08.050615 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-87f8c399-cae5-4010-81b0-989c116a778d tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:08.051119 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-87f8c399-cae5-4010-81b0-989c116a778d tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:08.068041 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-622d90f2-91fc-4945-bc2d-bedcf83e835b tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] 10.222.0.16 "DELETE /compute/v2.1/servers/d265192b-536b-4884-9fe8-74cf20fc7ef9" status: 204 len: 0 microversion: 2.1 time: 0.570935 Jul 16 21:19:08.068636 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1265/2570] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:19:07 2022] DELETE /compute/v2.1/servers/d265192b-536b-4884-9fe8-74cf20fc7ef9 => generated 0 bytes in 576 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 16 21:19:08.091519 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-8d728b87-6e4f-4dd6-b6d5-b3e0eccfd8fa tempest-VolumesSnapshotsTestJSON-851680556 tempest-VolumesSnapshotsTestJSON-851680556-project] Calling method '.version_select at 0x7fdd9d137790>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:19:08.094560 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-87f8c399-cae5-4010-81b0-989c116a778d tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] 10.222.0.16 "POST /compute/v2.1/servers/e7598b4a-f70c-4186-88d1-6fd0057f8656/action" status: 202 len: 1067 microversion: 2.1 time: 0.657262 Jul 16 21:19:08.108230 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1306/2571] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 21:19:07 2022] POST /compute/v2.1/servers/e7598b4a-f70c-4186-88d1-6fd0057f8656/action => generated 1067 bytes in 674 msecs (HTTP/1.1 202) 10 headers in 452 bytes (1 switches on core 0) Jul 16 21:19:08.129444 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-55efa871-85a2-482a-9749-e5db7309cb61 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:19:08.130667 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-55efa871-85a2-482a-9749-e5db7309cb61 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] [instance: c88b191a-1ae5-4952-8afa-14aee5567a7f] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:19:08.136823 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-8d728b87-6e4f-4dd6-b6d5-b3e0eccfd8fa tempest-VolumesSnapshotsTestJSON-851680556 tempest-VolumesSnapshotsTestJSON-851680556-project] HTTP exception thrown: Snapshot b6ec78c5-d133-485b-8009-7628a73d4739 could not be found. Jul 16 21:19:08.140887 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-55efa871-85a2-482a-9749-e5db7309cb61 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:08.141455 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-55efa871-85a2-482a-9749-e5db7309cb61 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:08.142461 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-55efa871-85a2-482a-9749-e5db7309cb61 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:08.143029 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-55efa871-85a2-482a-9749-e5db7309cb61 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:08.168620 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-8d728b87-6e4f-4dd6-b6d5-b3e0eccfd8fa tempest-VolumesSnapshotsTestJSON-851680556 tempest-VolumesSnapshotsTestJSON-851680556-project] Request method failure captured: Jul 16 21:19:08.168620 ncdv-master-1 devstack@n-api.service[138681]: request: GET /compute/v2.1/os-snapshots/b6ec78c5-d133-485b-8009-7628a73d4739 HTTP/1.1 Jul 16 21:19:08.168620 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:19:08.168620 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:19:08.168620 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:19:08.168620 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 21:19:08.168620 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:19:08.168620 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:19:08.168620 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:19:08.168620 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:19:08.168620 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0ys_bQ2YkbojsHK1HUJf-q5eS4WAKF6XcJ7jYKIj3a8lR37PmQ0zZ5DyXkNSC8x8kLgg3bQW6slrse7LirtZA91amTfVZ6Kr3gaUumm-yoTjKpsyiJvqY1GYtuGlfRXmsIYTP5FABjs_4cPzEKKvoKI8h2KKC02FgQGNg_a7XC56cnE Jul 16 21:19:08.168620 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:19:08.168620 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:19:08.168620 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:19:08.168620 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:19:08.168620 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:19:08.168620 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:19:08.168620 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: 942725b966874321b6ccd6f628787d06 Jul 16 21:19:08.168620 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-VolumesSnapshotsTestJSON-851680556 Jul 16 21:19:08.168620 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:19:08.168620 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:19:08.168620 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/942725b966874321b6ccd6f628787d06"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/942725b966874321b6ccd6f628787d06"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_942725b966874321b6ccd6f628787d06"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/942725b966874321b6ccd6f628787d06"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:19:08.168620 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-VolumesSnapshotsTestJSON-851680556 Jul 16 21:19:08.168620 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: 942725b966874321b6ccd6f628787d06 Jul 16 21:19:08.168620 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-VolumesSnapshotsTestJSON-851680556 Jul 16 21:19:08.168620 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-VolumesSnapshotsTestJSON-851680556-project Jul 16 21:19:08.168620 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:19:08.168620 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:19:08.168620 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: 6a3dbbb2b39447d884c56dfd0db7e42e Jul 16 21:19:08.168620 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-VolumesSnapshotsTestJSON-851680556-project Jul 16 21:19:08.168620 ncdv-master-1 devstack@n-api.service[138681]: method: Jul 16 21:19:08.168620 ncdv-master-1 devstack@n-api.service[138681]: exception: Snapshot b6ec78c5-d133-485b-8009-7628a73d4739 could not be found. Jul 16 21:19:08.168620 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': 'b6ec78c5-d133-485b-8009-7628a73d4739'} Jul 16 21:19:08.168620 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:19:08.168620 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:19:08.168620 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 464, in wrapper Jul 16 21:19:08.168620 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi res = method(self, ctx, snapshot_id, *args, **kwargs) Jul 16 21:19:08.168620 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 701, in get_snapshot Jul 16 21:19:08.168620 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi item = cinderclient(context).volume_snapshots.get(snapshot_id) Jul 16 21:19:08.168620 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volume_snapshots.py", line 160, in get Jul 16 21:19:08.168620 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return self._get("/snapshots/%s" % snapshot_id, "snapshot") Jul 16 21:19:08.168620 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Jul 16 21:19:08.168620 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Jul 16 21:19:08.168620 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Jul 16 21:19:08.168620 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Jul 16 21:19:08.168620 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jul 16 21:19:08.168620 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jul 16 21:19:08.168620 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jul 16 21:19:08.168620 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jul 16 21:19:08.168620 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Snapshot b6ec78c5-d133-485b-8009-7628a73d4739 could not be found. (HTTP 404) (Request-ID: req-30f77308-c81a-4948-8941-da6594488c86) Jul 16 21:19:08.168620 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:19:08.168620 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:19:08.168620 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:19:08.168620 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:19:08.168620 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 617, in show Jul 16 21:19:08.168620 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get_snapshot(context, id) Jul 16 21:19:08.168620 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Jul 16 21:19:08.168620 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Jul 16 21:19:08.168620 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 466, in wrapper Jul 16 21:19:08.168620 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi _reraise(exception.SnapshotNotFound(snapshot_id=snapshot_id)) Jul 16 21:19:08.168620 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Jul 16 21:19:08.168620 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Jul 16 21:19:08.176969 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 464, in wrapper Jul 16 21:19:08.176969 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi res = method(self, ctx, snapshot_id, *args, **kwargs) Jul 16 21:19:08.176969 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 701, in get_snapshot Jul 16 21:19:08.176969 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi item = cinderclient(context).volume_snapshots.get(snapshot_id) Jul 16 21:19:08.176969 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volume_snapshots.py", line 160, in get Jul 16 21:19:08.176969 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return self._get("/snapshots/%s" % snapshot_id, "snapshot") Jul 16 21:19:08.176969 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Jul 16 21:19:08.176969 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Jul 16 21:19:08.176969 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Jul 16 21:19:08.176969 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Jul 16 21:19:08.176969 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jul 16 21:19:08.176969 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jul 16 21:19:08.176969 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jul 16 21:19:08.176969 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jul 16 21:19:08.176969 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.SnapshotNotFound: Snapshot b6ec78c5-d133-485b-8009-7628a73d4739 could not be found. Jul 16 21:19:08.176969 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:19:08.176969 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:19:08.176969 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:19:08.176969 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:19:08.176969 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:19:08.176969 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:19:08.176969 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:19:08.176969 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:19:08.176969 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Jul 16 21:19:08.176969 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Jul 16 21:19:08.176969 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:19:08.176969 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:19:08.176969 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 619, in show Jul 16 21:19:08.176969 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:19:08.176969 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Snapshot b6ec78c5-d133-485b-8009-7628a73d4739 could not be found. Jul 16 21:19:08.176969 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:19:08.176969 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:19:08.176969 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:19:08.176969 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:19:08.176969 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:19:08.176969 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:19:08.176969 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:19:08.176969 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:19:08.176969 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Snapshot b6ec78c5-d133-485b-8009-7628a73d4739 could not be found. Jul 16 21:19:08.176969 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:19:08.176969 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-8d728b87-6e4f-4dd6-b6d5-b3e0eccfd8fa tempest-VolumesSnapshotsTestJSON-851680556 tempest-VolumesSnapshotsTestJSON-851680556-project] Returning 404 to user: Snapshot b6ec78c5-d133-485b-8009-7628a73d4739 could not be found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:19:08.176969 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-8d728b87-6e4f-4dd6-b6d5-b3e0eccfd8fa tempest-VolumesSnapshotsTestJSON-851680556 tempest-VolumesSnapshotsTestJSON-851680556-project] 10.222.0.16 "GET /compute/v2.1/os-snapshots/b6ec78c5-d133-485b-8009-7628a73d4739" status: 404 len: 111 microversion: 2.1 time: 0.085847 Jul 16 21:19:08.176969 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1266/2572] 10.222.0.16 () {58 vars in 1250 bytes} [Sat Jul 16 21:19:08 2022] GET /compute/v2.1/os-snapshots/b6ec78c5-d133-485b-8009-7628a73d4739 => generated 111 bytes in 87 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:19:08.201032 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-b9c9f98d-1112-4f64-90b9-efdb5269b9cd tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Calling method '.version_select at 0x7fdd9d8b14c0>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:19:08.246805 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-b9c9f98d-1112-4f64-90b9-efdb5269b9cd tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] HTTP exception thrown: Image not found. Jul 16 21:19:08.251112 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-b9c9f98d-1112-4f64-90b9-efdb5269b9cd tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Request method failure captured: Jul 16 21:19:08.251112 ncdv-master-1 devstack@n-api.service[138681]: request: DELETE /compute/v2.1/images/3cd0a9f0-af2b-4760-a6d2-51f198a2555a HTTP/1.1 Jul 16 21:19:08.251112 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:19:08.251112 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:19:08.251112 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:19:08.251112 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 21:19:08.251112 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:19:08.251112 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:19:08.251112 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:19:08.251112 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:19:08.251112 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0yptIZULOjVMHbouVcndclXAnMKC6qm1VtMGoKOnZzI-fUPx3L5lhuz06xvHQzr3ooL8P53cH83yaSSJWcV0fJCtKGdbdY71S9WCM2F11ZTapGri39-eD6M8L5ZmeIRv6VTRaJtzqhadxTTGkGwVf5FzILXoamLnE3Jdb08lelPg3B4 Jul 16 21:19:08.251112 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:19:08.251112 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:19:08.251112 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:19:08.251112 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:19:08.251112 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:19:08.251112 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:19:08.251112 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: 24c64db8082648c8837c16352252f96d Jul 16 21:19:08.251112 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-ImagesTestJSON-1532827163 Jul 16 21:19:08.251112 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:19:08.251112 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:19:08.251112 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/24c64db8082648c8837c16352252f96d"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/24c64db8082648c8837c16352252f96d"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_24c64db8082648c8837c16352252f96d"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/24c64db8082648c8837c16352252f96d"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:19:08.251112 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-ImagesTestJSON-1532827163 Jul 16 21:19:08.251112 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: 24c64db8082648c8837c16352252f96d Jul 16 21:19:08.251112 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-ImagesTestJSON-1532827163 Jul 16 21:19:08.251112 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-ImagesTestJSON-1532827163-project Jul 16 21:19:08.251112 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:19:08.251112 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:19:08.251112 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: daf3c9e6634e4ae1bfb492eae1f44c93 Jul 16 21:19:08.251112 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-ImagesTestJSON-1532827163-project Jul 16 21:19:08.251112 ncdv-master-1 devstack@n-api.service[138681]: method: Jul 16 21:19:08.251112 ncdv-master-1 devstack@n-api.service[138681]: exception: Image not found. Jul 16 21:19:08.251112 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': '3cd0a9f0-af2b-4760-a6d2-51f198a2555a'} Jul 16 21:19:08.251112 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:19:08.251112 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:19:08.251112 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 724, in delete Jul 16 21:19:08.251112 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi self._client.call(context, 2, 'delete', args=(image_id,)) Jul 16 21:19:08.251112 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 191, in call Jul 16 21:19:08.251112 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi result = getattr(controller, method)(*args, **kwargs) Jul 16 21:19:08.251112 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/utils.py", line 670, in inner Jul 16 21:19:08.251112 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return RequestIdProxy(wrapped(*args, **kwargs)) Jul 16 21:19:08.251112 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/v2/images.py", line 386, in delete Jul 16 21:19:08.251112 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi resp, body = self.http_client.delete(url) Jul 16 21:19:08.251112 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/adapter.py", line 410, in delete Jul 16 21:19:08.251112 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return self.request(url, 'DELETE', **kwargs) Jul 16 21:19:08.251112 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 380, in request Jul 16 21:19:08.251112 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return self._handle_response(resp) Jul 16 21:19:08.251112 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 120, in _handle_response Jul 16 21:19:08.251112 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.from_response(resp, resp.content) Jul 16 21:19:08.251112 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi glanceclient.exc.HTTPNotFound: HTTP 404 Not Found: Failed to find image 3cd0a9f0-af2b-4760-a6d2-51f198a2555a to delete Jul 16 21:19:08.251112 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:19:08.251112 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:19:08.251112 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:19:08.251112 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:19:08.251112 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/images.py", line 104, in delete Jul 16 21:19:08.251112 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi self._image_api.delete(context, id) Jul 16 21:19:08.251112 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 1255, in delete Jul 16 21:19:08.251112 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return session.delete(context, image_id) Jul 16 21:19:08.251112 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 726, in delete Jul 16 21:19:08.251112 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.ImageNotFound(image_id=image_id) Jul 16 21:19:08.251112 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.ImageNotFound: Image 3cd0a9f0-af2b-4760-a6d2-51f198a2555a could not be found. Jul 16 21:19:08.251112 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:19:08.251112 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:19:08.251112 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:19:08.251112 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:19:08.256459 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:19:08.256459 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:19:08.256459 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:19:08.256459 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:19:08.256459 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Jul 16 21:19:08.256459 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Jul 16 21:19:08.256459 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:19:08.256459 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:19:08.256459 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/images.py", line 107, in delete Jul 16 21:19:08.256459 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=explanation) Jul 16 21:19:08.256459 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Image not found. Jul 16 21:19:08.256459 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:19:08.256459 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:19:08.256459 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:19:08.256459 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:19:08.256459 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:19:08.256459 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:19:08.256459 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:19:08.256459 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:19:08.256459 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Image not found. Jul 16 21:19:08.256459 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:19:08.265258 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-b9c9f98d-1112-4f64-90b9-efdb5269b9cd tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Returning 404 to user: Image not found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:19:08.265258 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-b9c9f98d-1112-4f64-90b9-efdb5269b9cd tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] 10.222.0.16 "DELETE /compute/v2.1/images/3cd0a9f0-af2b-4760-a6d2-51f198a2555a" status: 404 len: 62 microversion: 2.1 time: 0.085493 Jul 16 21:19:08.265258 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1267/2573] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:19:08 2022] DELETE /compute/v2.1/images/3cd0a9f0-af2b-4760-a6d2-51f198a2555a => generated 62 bytes in 91 msecs (HTTP/1.1 404) 9 headers in 378 bytes (1 switches on core 0) Jul 16 21:19:08.277641 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-6d834405-f4b8-48c0-9c33-922bc5534e5a tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:19:08.278783 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-6d834405-f4b8-48c0-9c33-922bc5534e5a tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] [instance: e7598b4a-f70c-4186-88d1-6fd0057f8656] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:19:08.287588 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6d834405-f4b8-48c0-9c33-922bc5534e5a tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:08.288555 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6d834405-f4b8-48c0-9c33-922bc5534e5a tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:08.290274 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6d834405-f4b8-48c0-9c33-922bc5534e5a tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:08.290959 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6d834405-f4b8-48c0-9c33-922bc5534e5a tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:09.041256 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-6d834405-f4b8-48c0-9c33-922bc5534e5a tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f5bc1f2209f4eb186d80e385b67c774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '65d5bcfa02af4e8284ba1342197e2860', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:19:09.062604 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-6d834405-f4b8-48c0-9c33-922bc5534e5a tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] 10.222.0.16 "GET /compute/v2.1/servers/e7598b4a-f70c-4186-88d1-6fd0057f8656" status: 200 len: 1472 microversion: 2.1 time: 0.798829 Jul 16 21:19:09.063981 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1268/2574] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:19:08 2022] GET /compute/v2.1/servers/e7598b4a-f70c-4186-88d1-6fd0057f8656 => generated 1472 bytes in 800 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:19:09.065110 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-55efa871-85a2-482a-9749-e5db7309cb61 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd0008fffad8e4708b2c01e03da92c47d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e353258f9c3048ef84683294e2b8579d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:19:09.081567 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-55efa871-85a2-482a-9749-e5db7309cb61 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] 10.222.0.16 "GET /compute/v2.1/servers/c88b191a-1ae5-4952-8afa-14aee5567a7f" status: 200 len: 1646 microversion: 2.1 time: 0.969024 Jul 16 21:19:09.082281 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1307/2575] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:19:08 2022] GET /compute/v2.1/servers/c88b191a-1ae5-4952-8afa-14aee5567a7f => generated 1646 bytes in 974 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:19:09.085802 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-991f94ab-c90f-41be-844e-aa2f9468bcdb tempest-VolumesSnapshotsTestJSON-851680556 tempest-VolumesSnapshotsTestJSON-851680556-project] Calling method '.version_select at 0x7fdd9d519e50>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:19:09.100302 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-56000fd6-c4e0-4af3-b7a9-d33d35c51ba0 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:19:09.100953 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-56000fd6-c4e0-4af3-b7a9-d33d35c51ba0 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] [instance: d265192b-536b-4884-9fe8-74cf20fc7ef9] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:19:09.128990 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-56000fd6-c4e0-4af3-b7a9-d33d35c51ba0 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:09.129480 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-56000fd6-c4e0-4af3-b7a9-d33d35c51ba0 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:09.130278 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-56000fd6-c4e0-4af3-b7a9-d33d35c51ba0 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:09.130719 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-56000fd6-c4e0-4af3-b7a9-d33d35c51ba0 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:09.200257 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-56000fd6-c4e0-4af3-b7a9-d33d35c51ba0 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] [instance: d265192b-536b-4884-9fe8-74cf20fc7ef9] Going to try to terminate instance {{(pid=138680) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 16 21:19:09.366556 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.compute.api [None req-56000fd6-c4e0-4af3-b7a9-d33d35c51ba0 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] [instance: d265192b-536b-4884-9fe8-74cf20fc7ef9] Instance is already in deleting state, ignoring this request Jul 16 21:19:09.370294 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-56000fd6-c4e0-4af3-b7a9-d33d35c51ba0 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] 10.222.0.16 "DELETE /compute/v2.1/servers/d265192b-536b-4884-9fe8-74cf20fc7ef9" status: 204 len: 0 microversion: 2.1 time: 0.280972 Jul 16 21:19:09.371429 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1308/2576] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:19:09 2022] DELETE /compute/v2.1/servers/d265192b-536b-4884-9fe8-74cf20fc7ef9 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 16 21:19:09.378622 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-988e39e2-d397-4076-a812-b4fa47da56cc tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:19:09.379308 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-988e39e2-d397-4076-a812-b4fa47da56cc tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] [instance: c16e5f09-f813-414e-b233-c08348078624] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:19:09.420803 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-988e39e2-d397-4076-a812-b4fa47da56cc tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:09.422174 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-988e39e2-d397-4076-a812-b4fa47da56cc tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:09.423055 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-988e39e2-d397-4076-a812-b4fa47da56cc tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:09.423509 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-988e39e2-d397-4076-a812-b4fa47da56cc tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:09.452144 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-991f94ab-c90f-41be-844e-aa2f9468bcdb tempest-VolumesSnapshotsTestJSON-851680556 tempest-VolumesSnapshotsTestJSON-851680556-project] 10.222.0.16 "DELETE /compute/v2.1/os-volumes/4450505c-3ac5-42c2-b94c-b8f945007d50" status: 202 len: 0 microversion: 2.1 time: 0.385208 Jul 16 21:19:09.460246 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1269/2577] 10.222.0.16 () {58 vars in 1247 bytes} [Sat Jul 16 21:19:09 2022] DELETE /compute/v2.1/os-volumes/4450505c-3ac5-42c2-b94c-b8f945007d50 => generated 0 bytes in 395 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 21:19:09.485689 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-d2e8b09d-9dcc-468a-92f4-94d5984004e8 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:19:09.485689 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-d2e8b09d-9dcc-468a-92f4-94d5984004e8 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] [instance: d265192b-536b-4884-9fe8-74cf20fc7ef9] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:19:09.495907 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d2e8b09d-9dcc-468a-92f4-94d5984004e8 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:09.499561 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d2e8b09d-9dcc-468a-92f4-94d5984004e8 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:09.509103 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d2e8b09d-9dcc-468a-92f4-94d5984004e8 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:09.509103 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d2e8b09d-9dcc-468a-92f4-94d5984004e8 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:09.962872 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-988e39e2-d397-4076-a812-b4fa47da56cc tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eebb224a56814f92b12ebebebc6bc8a3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a092ce9ef40943f6b6ace6cd28481b71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:19:09.980302 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-988e39e2-d397-4076-a812-b4fa47da56cc tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] 10.222.0.16 "GET /compute/v2.1/servers/c16e5f09-f813-414e-b233-c08348078624" status: 200 len: 1224 microversion: 2.1 time: 0.607272 Jul 16 21:19:09.986118 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1309/2578] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:19:09 2022] GET /compute/v2.1/servers/c16e5f09-f813-414e-b233-c08348078624 => generated 1224 bytes in 614 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:19:09.998484 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-a9bdd4ce-4054-4675-a563-e59917e4df2e tempest-VolumesSnapshotsTestJSON-851680556 tempest-VolumesSnapshotsTestJSON-851680556-project] Calling method '.version_select at 0x7fdd9c6b9430>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:19:10.158347 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-a9bdd4ce-4054-4675-a563-e59917e4df2e tempest-VolumesSnapshotsTestJSON-851680556 tempest-VolumesSnapshotsTestJSON-851680556-project] 10.222.0.16 "GET /compute/v2.1/os-volumes/4450505c-3ac5-42c2-b94c-b8f945007d50" status: 200 len: 338 microversion: 2.1 time: 0.170457 Jul 16 21:19:10.159320 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1310/2579] 10.222.0.16 () {58 vars in 1244 bytes} [Sat Jul 16 21:19:09 2022] GET /compute/v2.1/os-volumes/4450505c-3ac5-42c2-b94c-b8f945007d50 => generated 338 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:19:10.343935 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-d2e8b09d-9dcc-468a-92f4-94d5984004e8 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'daf3c9e6634e4ae1bfb492eae1f44c93', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '24c64db8082648c8837c16352252f96d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:19:10.369142 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-d2e8b09d-9dcc-468a-92f4-94d5984004e8 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] 10.222.0.16 "GET /compute/v2.1/servers/d265192b-536b-4884-9fe8-74cf20fc7ef9" status: 200 len: 1493 microversion: 2.1 time: 0.905990 Jul 16 21:19:10.383685 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1270/2580] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:19:09 2022] GET /compute/v2.1/servers/d265192b-536b-4884-9fe8-74cf20fc7ef9 => generated 1493 bytes in 922 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:19:10.401537 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-10c73974-7f72-4f89-9011-6f3cb213260a req-574ddbe8-9e15-4e72-9020-cfc109803a36 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "d265192b-536b-4884-9fe8-74cf20fc7ef9", "name": "network-vif-plugged", "status": "completed", "tag": "8a340e39-7064-4a42-b19e-e37841899e42"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:19:10.436219 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-10c73974-7f72-4f89-9011-6f3cb213260a req-574ddbe8-9e15-4e72-9020-cfc109803a36 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:10.436219 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-10c73974-7f72-4f89-9011-6f3cb213260a req-574ddbe8-9e15-4e72-9020-cfc109803a36 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:10.450227 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-10c73974-7f72-4f89-9011-6f3cb213260a req-574ddbe8-9e15-4e72-9020-cfc109803a36 service nova] Creating event network-vif-plugged:8a340e39-7064-4a42-b19e-e37841899e42 for instance d265192b-536b-4884-9fe8-74cf20fc7ef9 on nch2-master-1 Jul 16 21:19:10.456105 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-10c73974-7f72-4f89-9011-6f3cb213260a req-574ddbe8-9e15-4e72-9020-cfc109803a36 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.070794 Jul 16 21:19:10.457360 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1271/2581] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:19:10 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 73 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:19:10.463080 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-183d419b-d111-47ee-b2e4-3a09d7af0a2b tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:19:10.463986 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-183d419b-d111-47ee-b2e4-3a09d7af0a2b tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] [instance: e7598b4a-f70c-4186-88d1-6fd0057f8656] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:19:10.472019 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-183d419b-d111-47ee-b2e4-3a09d7af0a2b tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:10.472719 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-183d419b-d111-47ee-b2e4-3a09d7af0a2b tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:10.473723 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-183d419b-d111-47ee-b2e4-3a09d7af0a2b tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:10.474315 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-183d419b-d111-47ee-b2e4-3a09d7af0a2b tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:10.507366 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=94,nova_cell1:UPDATE=3,nova_cell1:INSERT=3 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:19:10.585770 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-1158e605-66f2-4d1b-a214-25af9dd27472 tempest-SecurityGroupRulesNegativeTestJSON-1399499068 tempest-SecurityGroupRulesNegativeTestJSON-1399499068-project] Action: 'create', calling method: .version_select at 0x7fdda2bc89d0>, body: {"security_group": {"name": "tempest-SecurityGroupRulesNegativeTestJSON-securitygroup-1469368896", "description": "tempest-description-874434349"}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:19:10.953139 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-183d419b-d111-47ee-b2e4-3a09d7af0a2b tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f5bc1f2209f4eb186d80e385b67c774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '65d5bcfa02af4e8284ba1342197e2860', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:19:10.963352 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-183d419b-d111-47ee-b2e4-3a09d7af0a2b tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] 10.222.0.16 "GET /compute/v2.1/servers/e7598b4a-f70c-4186-88d1-6fd0057f8656" status: 200 len: 1472 microversion: 2.1 time: 0.504568 Jul 16 21:19:10.964053 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1272/2582] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:19:10 2022] GET /compute/v2.1/servers/e7598b4a-f70c-4186-88d1-6fd0057f8656 => generated 1472 bytes in 506 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:19:10.978840 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-3603fb3b-09ef-4ce7-9822-57940d1843bc tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:19:10.979713 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-3603fb3b-09ef-4ce7-9822-57940d1843bc tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] [instance: c88b191a-1ae5-4952-8afa-14aee5567a7f] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:19:10.986370 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3603fb3b-09ef-4ce7-9822-57940d1843bc tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:10.986918 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3603fb3b-09ef-4ce7-9822-57940d1843bc tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:10.987992 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3603fb3b-09ef-4ce7-9822-57940d1843bc tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:10.988753 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3603fb3b-09ef-4ce7-9822-57940d1843bc tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:11.139433 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-3603fb3b-09ef-4ce7-9822-57940d1843bc tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd0008fffad8e4708b2c01e03da92c47d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e353258f9c3048ef84683294e2b8579d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:19:11.148230 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-3603fb3b-09ef-4ce7-9822-57940d1843bc tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] 10.222.0.16 "GET /compute/v2.1/servers/c88b191a-1ae5-4952-8afa-14aee5567a7f" status: 200 len: 1469 microversion: 2.1 time: 0.182796 Jul 16 21:19:11.149057 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1273/2583] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:19:10 2022] GET /compute/v2.1/servers/c88b191a-1ae5-4952-8afa-14aee5567a7f => generated 1469 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:19:11.257330 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-660732bb-4cd9-4d16-a6c4-879cd9a21f7a tempest-ServersAaction247Test-725202529 tempest-ServersAaction247Test-725202529-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServersAaction247Test-server-1870879937", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452", "networks": "none"}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:19:11.372813 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-660732bb-4cd9-4d16-a6c4-879cd9a21f7a tempest-ServersAaction247Test-725202529 tempest-ServersAaction247Test-725202529-project] Flavor limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 16 21:19:11.375269 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-660732bb-4cd9-4d16-a6c4-879cd9a21f7a tempest-ServersAaction247Test-725202529 tempest-ServersAaction247Test-725202529-project] Image limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 16 21:19:11.376544 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-660732bb-4cd9-4d16-a6c4-879cd9a21f7a tempest-ServersAaction247Test-725202529 tempest-ServersAaction247Test-725202529-project] Flavor pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 16 21:19:11.378521 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-660732bb-4cd9-4d16-a6c4-879cd9a21f7a tempest-ServersAaction247Test-725202529 tempest-ServersAaction247Test-725202529-project] Image pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 16 21:19:11.378972 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-660732bb-4cd9-4d16-a6c4-879cd9a21f7a tempest-ServersAaction247Test-725202529 tempest-ServersAaction247Test-725202529-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 16 21:19:11.427185 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-660732bb-4cd9-4d16-a6c4-879cd9a21f7a tempest-ServersAaction247Test-725202529 tempest-ServersAaction247Test-725202529-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.020s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:11.428097 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-660732bb-4cd9-4d16-a6c4-879cd9a21f7a tempest-ServersAaction247Test-725202529 tempest-ServersAaction247Test-725202529-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:11.431756 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-660732bb-4cd9-4d16-a6c4-879cd9a21f7a tempest-ServersAaction247Test-725202529 tempest-ServersAaction247Test-725202529-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:11.432486 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-660732bb-4cd9-4d16-a6c4-879cd9a21f7a tempest-ServersAaction247Test-725202529 tempest-ServersAaction247Test-725202529-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:11.505893 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-660732bb-4cd9-4d16-a6c4-879cd9a21f7a tempest-ServersAaction247Test-725202529 tempest-ServersAaction247Test-725202529-project] Getting quotas for project f2984288b4cd4610b2f32e895f3e5d72. Resources: {'ram', 'instances', 'cores'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:19:11.515076 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-660732bb-4cd9-4d16-a6c4-879cd9a21f7a tempest-ServersAaction247Test-725202529 tempest-ServersAaction247Test-725202529-project] Getting quotas for user a8d48b52db1545a4a51027a5c2c7fcce and project f2984288b4cd4610b2f32e895f3e5d72. Resources: {'ram', 'instances', 'cores'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:19:11.557905 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-660732bb-4cd9-4d16-a6c4-879cd9a21f7a tempest-ServersAaction247Test-725202529 tempest-ServersAaction247Test-725202529-project] Going to run 1 instances... {{(pid=138681) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 16 21:19:11.560572 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-660732bb-4cd9-4d16-a6c4-879cd9a21f7a tempest-ServersAaction247Test-725202529 tempest-ServersAaction247Test-725202529-project] [instance: a51adaf1-9c26-47d0-92c2-b6bb065a5ee8] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138681) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 16 21:19:11.702020 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-1158e605-66f2-4d1b-a214-25af9dd27472 tempest-SecurityGroupRulesNegativeTestJSON-1399499068 tempest-SecurityGroupRulesNegativeTestJSON-1399499068-project] 10.222.0.16 "POST /compute/v2.1/os-security-groups" status: 200 len: 255 microversion: 2.1 time: 1.541418 Jul 16 21:19:11.703372 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1311/2584] 10.222.0.16 () {60 vars in 1179 bytes} [Sat Jul 16 21:19:10 2022] POST /compute/v2.1/os-security-groups => generated 255 bytes in 1543 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:19:11.710102 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-19f6049a-216f-4816-8e7e-19eae78f5531 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:19:11.711908 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-19f6049a-216f-4816-8e7e-19eae78f5531 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] [instance: c16e5f09-f813-414e-b233-c08348078624] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:19:11.734023 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-19f6049a-216f-4816-8e7e-19eae78f5531 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:11.736982 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-19f6049a-216f-4816-8e7e-19eae78f5531 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:11.744581 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-19f6049a-216f-4816-8e7e-19eae78f5531 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:11.745279 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-19f6049a-216f-4816-8e7e-19eae78f5531 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:11.778468 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-660732bb-4cd9-4d16-a6c4-879cd9a21f7a tempest-ServersAaction247Test-725202529 tempest-ServersAaction247Test-725202529-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.47 time: 0.622307 Jul 16 21:19:11.779828 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1274/2585] 10.222.0.16 () {62 vars in 1187 bytes} [Sat Jul 16 21:19:11 2022] POST /compute/v2.1/servers => generated 380 bytes in 630 msecs (HTTP/1.1 202) 10 headers in 453 bytes (1 switches on core 0) Jul 16 21:19:11.823643 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-9cd6b29b-0e9a-4011-bea5-182b717f9611 tempest-VolumesSnapshotsTestJSON-851680556 tempest-VolumesSnapshotsTestJSON-851680556-project] Calling method '.version_select at 0x7fdd9d0e23a0>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:19:11.942284 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-19f6049a-216f-4816-8e7e-19eae78f5531 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eebb224a56814f92b12ebebebc6bc8a3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a092ce9ef40943f6b6ace6cd28481b71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:19:11.966047 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-9cd6b29b-0e9a-4011-bea5-182b717f9611 tempest-VolumesSnapshotsTestJSON-851680556 tempest-VolumesSnapshotsTestJSON-851680556-project] 10.222.0.16 "GET /compute/v2.1/os-volumes/4450505c-3ac5-42c2-b94c-b8f945007d50" status: 200 len: 338 microversion: 2.1 time: 0.183582 Jul 16 21:19:11.967175 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1275/2586] 10.222.0.16 () {58 vars in 1244 bytes} [Sat Jul 16 21:19:11 2022] GET /compute/v2.1/os-volumes/4450505c-3ac5-42c2-b94c-b8f945007d50 => generated 338 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:19:11.976212 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-badbd3ac-a4fb-4d63-ac86-b8e16c09e4d1 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:19:11.978526 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-badbd3ac-a4fb-4d63-ac86-b8e16c09e4d1 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] [instance: d265192b-536b-4884-9fe8-74cf20fc7ef9] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:19:11.980159 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-19f6049a-216f-4816-8e7e-19eae78f5531 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] 10.222.0.16 "GET /compute/v2.1/servers/c16e5f09-f813-414e-b233-c08348078624" status: 200 len: 1278 microversion: 2.1 time: 0.272599 Jul 16 21:19:11.999071 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1312/2587] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:19:11 2022] GET /compute/v2.1/servers/c16e5f09-f813-414e-b233-c08348078624 => generated 1278 bytes in 286 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:19:12.006249 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-badbd3ac-a4fb-4d63-ac86-b8e16c09e4d1 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:12.007001 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-badbd3ac-a4fb-4d63-ac86-b8e16c09e4d1 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:12.008699 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-badbd3ac-a4fb-4d63-ac86-b8e16c09e4d1 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:12.009377 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-badbd3ac-a4fb-4d63-ac86-b8e16c09e4d1 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:12.012879 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-db75d4b8-49dd-48a0-bf99-a0eabf1ebc9f req-ef0d02a6-310d-459c-aeeb-a55fb2c6d953 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "c88b191a-1ae5-4952-8afa-14aee5567a7f", "name": "network-vif-deleted", "tag": "9b680cff-b0fe-4eb6-91dd-54ab2451c6f2"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:19:12.042633 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-db75d4b8-49dd-48a0-bf99-a0eabf1ebc9f req-ef0d02a6-310d-459c-aeeb-a55fb2c6d953 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:12.047702 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-db75d4b8-49dd-48a0-bf99-a0eabf1ebc9f req-ef0d02a6-310d-459c-aeeb-a55fb2c6d953 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:12.088924 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-db75d4b8-49dd-48a0-bf99-a0eabf1ebc9f req-ef0d02a6-310d-459c-aeeb-a55fb2c6d953 service nova] Creating event network-vif-deleted:9b680cff-b0fe-4eb6-91dd-54ab2451c6f2 for instance c88b191a-1ae5-4952-8afa-14aee5567a7f on nch1-master-1 Jul 16 21:19:12.109665 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-db75d4b8-49dd-48a0-bf99-a0eabf1ebc9f req-ef0d02a6-310d-459c-aeeb-a55fb2c6d953 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.113670 Jul 16 21:19:12.111854 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1313/2588] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:19:11 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 121 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:19:12.119566 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-0e0a59c8-627b-49cc-9b8c-4403ed741c49 tempest-SecurityGroupRulesNegativeTestJSON-1399499068 tempest-SecurityGroupRulesNegativeTestJSON-1399499068-project] Action: 'create', calling method: .version_select at 0x7fdd9cc3f550>, body: {"security_group_rule": {"parent_group_id": "cbd89553-301f-4fde-8e7e-ca4e1602fcb9", "ip_protocol": "tcp", "from_port": 22, "to_port": 22}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:19:12.188314 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-badbd3ac-a4fb-4d63-ac86-b8e16c09e4d1 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'daf3c9e6634e4ae1bfb492eae1f44c93', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '24c64db8082648c8837c16352252f96d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:19:12.198635 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-badbd3ac-a4fb-4d63-ac86-b8e16c09e4d1 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] 10.222.0.16 "GET /compute/v2.1/servers/d265192b-536b-4884-9fe8-74cf20fc7ef9" status: 200 len: 1292 microversion: 2.1 time: 0.230339 Jul 16 21:19:12.200088 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1276/2589] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:19:11 2022] GET /compute/v2.1/servers/d265192b-536b-4884-9fe8-74cf20fc7ef9 => generated 1292 bytes in 232 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:19:12.221923 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-7e42e0b7-4668-4023-959a-bfe34d6dc47a tempest-ServersAaction247Test-725202529 tempest-ServersAaction247Test-725202529-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:19:12.222855 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-7e42e0b7-4668-4023-959a-bfe34d6dc47a tempest-ServersAaction247Test-725202529 tempest-ServersAaction247Test-725202529-project] [instance: a51adaf1-9c26-47d0-92c2-b6bb065a5ee8] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:19:12.517886 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-7e42e0b7-4668-4023-959a-bfe34d6dc47a tempest-ServersAaction247Test-725202529 tempest-ServersAaction247Test-725202529-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a8d48b52db1545a4a51027a5c2c7fcce', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f2984288b4cd4610b2f32e895f3e5d72', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:19:12.530585 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-7e42e0b7-4668-4023-959a-bfe34d6dc47a tempest-ServersAaction247Test-725202529 tempest-ServersAaction247Test-725202529-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'a8d48b52db1545a4a51027a5c2c7fcce', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f2984288b4cd4610b2f32e895f3e5d72', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:19:12.532967 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-7e42e0b7-4668-4023-959a-bfe34d6dc47a tempest-ServersAaction247Test-725202529 tempest-ServersAaction247Test-725202529-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'a8d48b52db1545a4a51027a5c2c7fcce', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f2984288b4cd4610b2f32e895f3e5d72', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:19:12.534537 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-7e42e0b7-4668-4023-959a-bfe34d6dc47a tempest-ServersAaction247Test-725202529 tempest-ServersAaction247Test-725202529-project] 10.222.0.16 "GET /compute/v2.1/servers/a51adaf1-9c26-47d0-92c2-b6bb065a5ee8" status: 200 len: 1281 microversion: 2.47 time: 0.333146 Jul 16 21:19:12.535906 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1277/2590] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:19:12 2022] GET /compute/v2.1/servers/a51adaf1-9c26-47d0-92c2-b6bb065a5ee8 => generated 1281 bytes in 335 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:19:12.543320 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-2fe3cecc-c642-4732-a2f5-8de3f1d204bf tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:19:12.544263 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-2fe3cecc-c642-4732-a2f5-8de3f1d204bf tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] [instance: e7598b4a-f70c-4186-88d1-6fd0057f8656] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:19:12.556488 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2fe3cecc-c642-4732-a2f5-8de3f1d204bf tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:12.557073 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2fe3cecc-c642-4732-a2f5-8de3f1d204bf tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:12.557972 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2fe3cecc-c642-4732-a2f5-8de3f1d204bf tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:12.558805 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2fe3cecc-c642-4732-a2f5-8de3f1d204bf tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:12.797568 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-2fe3cecc-c642-4732-a2f5-8de3f1d204bf tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f5bc1f2209f4eb186d80e385b67c774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '65d5bcfa02af4e8284ba1342197e2860', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:19:12.808058 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-2fe3cecc-c642-4732-a2f5-8de3f1d204bf tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] 10.222.0.16 "GET /compute/v2.1/servers/e7598b4a-f70c-4186-88d1-6fd0057f8656" status: 200 len: 1472 microversion: 2.1 time: 0.270918 Jul 16 21:19:12.819248 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1278/2591] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:19:12 2022] GET /compute/v2.1/servers/e7598b4a-f70c-4186-88d1-6fd0057f8656 => generated 1472 bytes in 283 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:19:12.825225 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-1c5e1e3b-5a67-498d-9491-e5a7681a4175 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:19:12.826146 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-1c5e1e3b-5a67-498d-9491-e5a7681a4175 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] [instance: c88b191a-1ae5-4952-8afa-14aee5567a7f] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:19:12.837548 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1c5e1e3b-5a67-498d-9491-e5a7681a4175 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:12.838365 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1c5e1e3b-5a67-498d-9491-e5a7681a4175 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:12.839554 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1c5e1e3b-5a67-498d-9491-e5a7681a4175 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:12.840243 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1c5e1e3b-5a67-498d-9491-e5a7681a4175 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:12.874595 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-1c5e1e3b-5a67-498d-9491-e5a7681a4175 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] HTTP exception thrown: Instance c88b191a-1ae5-4952-8afa-14aee5567a7f could not be found. Jul 16 21:19:12.876703 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-1c5e1e3b-5a67-498d-9491-e5a7681a4175 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Request method failure captured: Jul 16 21:19:12.876703 ncdv-master-1 devstack@n-api.service[138681]: request: GET /compute/v2.1/servers/c88b191a-1ae5-4952-8afa-14aee5567a7f HTTP/1.1 Jul 16 21:19:12.876703 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:19:12.876703 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:19:12.876703 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:19:12.876703 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 21:19:12.876703 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:19:12.876703 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:19:12.876703 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:19:12.876703 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:19:12.876703 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0yrk-HsfYKsyovLSaEtNxaoXy7a3rLGNlsRPiPP76ZvvrKkKyGd-uekosXQxpFgkRnItBdviN4xr49Rerg5AyvLSzsyRY-Um0r3BRAvyZrzmCDYUpWbXk1uyEIb3Af4U3Y3nXIJgJAuDVx9z2NGSvEGCsGz12WlyyGCNfVSADFcYJP8 Jul 16 21:19:12.876703 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:19:12.876703 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:19:12.876703 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:19:12.876703 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:19:12.876703 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:19:12.876703 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:19:12.876703 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: e353258f9c3048ef84683294e2b8579d Jul 16 21:19:12.876703 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-SecurityGroupsTestJSON-438340169 Jul 16 21:19:12.876703 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:19:12.876703 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:19:12.876703 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/e353258f9c3048ef84683294e2b8579d"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/e353258f9c3048ef84683294e2b8579d"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_e353258f9c3048ef84683294e2b8579d"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/e353258f9c3048ef84683294e2b8579d"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:19:12.876703 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-SecurityGroupsTestJSON-438340169 Jul 16 21:19:12.876703 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: e353258f9c3048ef84683294e2b8579d Jul 16 21:19:12.876703 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-SecurityGroupsTestJSON-438340169 Jul 16 21:19:12.876703 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-SecurityGroupsTestJSON-438340169-project Jul 16 21:19:12.876703 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:19:12.876703 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:19:12.876703 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: d0008fffad8e4708b2c01e03da92c47d Jul 16 21:19:12.876703 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-SecurityGroupsTestJSON-438340169-project Jul 16 21:19:12.876703 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:19:12.876703 ncdv-master-1 devstack@n-api.service[138681]: exception: Instance c88b191a-1ae5-4952-8afa-14aee5567a7f could not be found. Jul 16 21:19:12.876703 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': 'c88b191a-1ae5-4952-8afa-14aee5567a7f'} Jul 16 21:19:12.876703 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:19:12.876703 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:19:12.876703 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:19:12.876703 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:19:12.876703 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:19:12.876703 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:19:12.876703 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:19:12.876703 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:19:12.876703 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:19:12.876703 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:19:12.876703 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance c88b191a-1ae5-4952-8afa-14aee5567a7f could not be found. Jul 16 21:19:12.876703 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:19:12.876703 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:19:12.876703 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:19:12.876703 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:19:12.876703 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:19:12.876703 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:19:12.876703 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:19:12.876703 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:19:12.876703 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:19:12.876703 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:19:12.876703 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:19:12.876703 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:19:12.876703 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:19:12.876703 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:19:12.876703 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:19:12.876703 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:19:12.876703 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance c88b191a-1ae5-4952-8afa-14aee5567a7f could not be found. Jul 16 21:19:12.876703 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:19:12.876703 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:19:12.876703 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:19:12.881371 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:19:12.881371 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:19:12.881371 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:19:12.881371 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:19:12.881371 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:19:12.881371 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance c88b191a-1ae5-4952-8afa-14aee5567a7f could not be found. Jul 16 21:19:12.881371 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:19:12.881856 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-1c5e1e3b-5a67-498d-9491-e5a7681a4175 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Returning 404 to user: Instance c88b191a-1ae5-4952-8afa-14aee5567a7f could not be found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:19:12.882846 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-1c5e1e3b-5a67-498d-9491-e5a7681a4175 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] 10.222.0.16 "GET /compute/v2.1/servers/c88b191a-1ae5-4952-8afa-14aee5567a7f" status: 404 len: 111 microversion: 2.1 time: 0.062268 Jul 16 21:19:12.883736 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1279/2592] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:19:12 2022] GET /compute/v2.1/servers/c88b191a-1ae5-4952-8afa-14aee5567a7f => generated 111 bytes in 64 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:19:12.892273 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-1d857054-4951-4574-81df-904d56b17d7e tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Calling method '.version_select at 0x7fdd9da679d0>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:19:12.970098 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-0e0a59c8-627b-49cc-9b8c-4403ed741c49 tempest-SecurityGroupRulesNegativeTestJSON-1399499068 tempest-SecurityGroupRulesNegativeTestJSON-1399499068-project] 10.222.0.16 "POST /compute/v2.1/os-security-group-rules" status: 200 len: 232 microversion: 2.1 time: 0.855846 Jul 16 21:19:12.977588 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1314/2593] 10.222.0.16 () {60 vars in 1194 bytes} [Sat Jul 16 21:19:12 2022] POST /compute/v2.1/os-security-group-rules => generated 232 bytes in 864 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:19:12.983148 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-7b264ea0-e1bc-4c39-bfbe-5e884762ef02 tempest-VolumesSnapshotsTestJSON-851680556 tempest-VolumesSnapshotsTestJSON-851680556-project] Calling method '.version_select at 0x7fdd9c6b9430>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:19:13.134911 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-7b264ea0-e1bc-4c39-bfbe-5e884762ef02 tempest-VolumesSnapshotsTestJSON-851680556 tempest-VolumesSnapshotsTestJSON-851680556-project] 10.222.0.16 "GET /compute/v2.1/os-volumes/4450505c-3ac5-42c2-b94c-b8f945007d50" status: 200 len: 338 microversion: 2.1 time: 0.155823 Jul 16 21:19:13.145153 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1315/2594] 10.222.0.16 () {58 vars in 1244 bytes} [Sat Jul 16 21:19:12 2022] GET /compute/v2.1/os-volumes/4450505c-3ac5-42c2-b94c-b8f945007d50 => generated 338 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:19:13.150920 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-9bba8590-ffc8-411c-984a-d6a7d019dd18 tempest-SecurityGroupRulesNegativeTestJSON-1399499068 tempest-SecurityGroupRulesNegativeTestJSON-1399499068-project] Action: 'create', calling method: .version_select at 0x7fdda2b7f0d0>, body: {"security_group_rule": {"parent_group_id": "cbd89553-301f-4fde-8e7e-ca4e1602fcb9", "ip_protocol": "tcp", "from_port": 22, "to_port": 22}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:19:13.303046 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-9bba8590-ffc8-411c-984a-d6a7d019dd18 tempest-SecurityGroupRulesNegativeTestJSON-1399499068 tempest-SecurityGroupRulesNegativeTestJSON-1399499068-project] HTTP exception thrown: This rule already exists in group cbd89553-301f-4fde-8e7e-ca4e1602fcb9 Jul 16 21:19:13.305875 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-9bba8590-ffc8-411c-984a-d6a7d019dd18 tempest-SecurityGroupRulesNegativeTestJSON-1399499068 tempest-SecurityGroupRulesNegativeTestJSON-1399499068-project] Request method failure captured: Jul 16 21:19:13.305875 ncdv-master-1 devstack@n-api.service[138680]: request: POST /compute/v2.1/os-security-group-rules HTTP/1.1 Jul 16 21:19:13.305875 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:19:13.305875 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:19:13.305875 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:19:13.305875 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 138 Jul 16 21:19:13.305875 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:19:13.305875 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:19:13.305875 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:19:13.305875 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:19:13.305875 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0ytNkmtXKapUZpFKPGUA8hf3ATbduJ0XtzorjW7amdLo0WS0Qbkqbi3B3wBROLLwxfXJZxCXXmOW4by85qgjbrIW-gGZ2c_YrmiVSX-WID5VqAWhi2CmsmYLeeu_6bw9QJDULh6nweMasTNNM3cnBfI9QUQLgKaJIeA4QYZkK1W0reQ Jul 16 21:19:13.305875 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:19:13.305875 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:19:13.305875 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:19:13.305875 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:19:13.305875 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:19:13.305875 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:19:13.305875 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: 89b4d12252b94d54b50cc15238af6c2f Jul 16 21:19:13.305875 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-SecurityGroupRulesNegativeTestJSON-1399499068 Jul 16 21:19:13.305875 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:19:13.305875 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:19:13.305875 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/89b4d12252b94d54b50cc15238af6c2f"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/89b4d12252b94d54b50cc15238af6c2f"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_89b4d12252b94d54b50cc15238af6c2f"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/89b4d12252b94d54b50cc15238af6c2f"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:19:13.305875 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-SecurityGroupRulesNegativeTestJSON-1399499068 Jul 16 21:19:13.305875 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: 89b4d12252b94d54b50cc15238af6c2f Jul 16 21:19:13.305875 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-SecurityGroupRulesNegativeTestJSON-1399499068 Jul 16 21:19:13.305875 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-SecurityGroupRulesNegativeTestJSON-1399499068-project Jul 16 21:19:13.305875 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:19:13.305875 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:19:13.305875 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: 7ea30eb37fbf469194845261a19e8d4f Jul 16 21:19:13.305875 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-SecurityGroupRulesNegativeTestJSON-1399499068-project Jul 16 21:19:13.305875 ncdv-master-1 devstack@n-api.service[138680]: Jul 16 21:19:13.305875 ncdv-master-1 devstack@n-api.service[138680]: {"security_group_rule": {"parent_group_id": "cbd89553-301f-4fde-8e7e-ca4e1602fcb9", "ip_protocol": "tcp", "from_port": 22, "to_port": 22}} Jul 16 21:19:13.305875 ncdv-master-1 devstack@n-api.service[138680]: method: Jul 16 21:19:13.305875 ncdv-master-1 devstack@n-api.service[138680]: exception: This rule already exists in group cbd89553-301f-4fde-8e7e-ca4e1602fcb9 Jul 16 21:19:13.305875 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'body': {'security_group_rule': {'parent_group_id': 'cbd89553-301f-4fde-8e7e-ca4e1602fcb9', 'ip_protocol': 'tcp', 'from_port': 22, 'to_port': 22}}} Jul 16 21:19:13.305875 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:19:13.305875 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:19:13.305875 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/security_groups.py", line 324, in create Jul 16 21:19:13.305875 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi security_group_api.create_security_group_rule( Jul 16 21:19:13.305875 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/security_group_api.py", line 186, in create_security_group_rule Jul 16 21:19:13.305875 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.Invalid(msg) Jul 16 21:19:13.305875 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.Invalid: This rule already exists in group cbd89553-301f-4fde-8e7e-ca4e1602fcb9 Jul 16 21:19:13.305875 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:19:13.305875 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:19:13.305875 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:19:13.305875 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:19:13.305875 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:19:13.305875 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:19:13.305875 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:19:13.305875 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:19:13.305875 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Jul 16 21:19:13.305875 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Jul 16 21:19:13.305875 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:19:13.305875 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:19:13.305875 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/security_groups.py", line 327, in create Jul 16 21:19:13.305875 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=exp.format_message()) Jul 16 21:19:13.305875 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: This rule already exists in group cbd89553-301f-4fde-8e7e-ca4e1602fcb9 Jul 16 21:19:13.305875 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:19:13.305875 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:19:13.305875 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:19:13.305875 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:19:13.305875 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:19:13.305875 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:19:13.309788 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:19:13.309788 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:19:13.309788 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: This rule already exists in group cbd89553-301f-4fde-8e7e-ca4e1602fcb9 Jul 16 21:19:13.309788 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:19:13.312609 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-9bba8590-ffc8-411c-984a-d6a7d019dd18 tempest-SecurityGroupRulesNegativeTestJSON-1399499068 tempest-SecurityGroupRulesNegativeTestJSON-1399499068-project] Returning 400 to user: This rule already exists in group cbd89553-301f-4fde-8e7e-ca4e1602fcb9 {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:19:13.313664 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-9bba8590-ffc8-411c-984a-d6a7d019dd18 tempest-SecurityGroupRulesNegativeTestJSON-1399499068 tempest-SecurityGroupRulesNegativeTestJSON-1399499068-project] 10.222.0.16 "POST /compute/v2.1/os-security-group-rules" status: 400 len: 114 microversion: 2.1 time: 0.169131 Jul 16 21:19:13.318483 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1316/2595] 10.222.0.16 () {60 vars in 1194 bytes} [Sat Jul 16 21:19:13 2022] POST /compute/v2.1/os-security-group-rules => generated 114 bytes in 174 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Jul 16 21:19:13.327841 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-8faa5ef7-b8dd-4151-b262-745e8884c5f3 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:19:13.329769 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-8faa5ef7-b8dd-4151-b262-745e8884c5f3 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] [instance: c16e5f09-f813-414e-b233-c08348078624] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:19:13.349527 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8faa5ef7-b8dd-4151-b262-745e8884c5f3 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:13.350213 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8faa5ef7-b8dd-4151-b262-745e8884c5f3 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:13.351218 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8faa5ef7-b8dd-4151-b262-745e8884c5f3 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:13.352037 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8faa5ef7-b8dd-4151-b262-745e8884c5f3 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:13.572724 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-1d857054-4951-4574-81df-904d56b17d7e tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] 10.222.0.16 "DELETE /compute/v2.1/os-security-groups/89aaff03-f3d3-48f7-965b-66c2a37029e0" status: 202 len: 0 microversion: 2.1 time: 0.684209 Jul 16 21:19:13.576870 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1280/2596] 10.222.0.16 () {58 vars in 1271 bytes} [Sat Jul 16 21:19:12 2022] DELETE /compute/v2.1/os-security-groups/89aaff03-f3d3-48f7-965b-66c2a37029e0 => generated 0 bytes in 689 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 21:19:13.581905 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-36b5fd42-2e31-4072-af3e-ef80f4369ee7 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:19:13.582529 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-36b5fd42-2e31-4072-af3e-ef80f4369ee7 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] [instance: d265192b-536b-4884-9fe8-74cf20fc7ef9] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:19:13.608842 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-36b5fd42-2e31-4072-af3e-ef80f4369ee7 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:13.612586 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-36b5fd42-2e31-4072-af3e-ef80f4369ee7 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:13.613730 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-36b5fd42-2e31-4072-af3e-ef80f4369ee7 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:13.615479 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-36b5fd42-2e31-4072-af3e-ef80f4369ee7 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:13.648359 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-36b5fd42-2e31-4072-af3e-ef80f4369ee7 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] HTTP exception thrown: Instance d265192b-536b-4884-9fe8-74cf20fc7ef9 could not be found. Jul 16 21:19:13.654934 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-36b5fd42-2e31-4072-af3e-ef80f4369ee7 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Request method failure captured: Jul 16 21:19:13.654934 ncdv-master-1 devstack@n-api.service[138681]: request: GET /compute/v2.1/servers/d265192b-536b-4884-9fe8-74cf20fc7ef9 HTTP/1.1 Jul 16 21:19:13.654934 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:19:13.654934 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:19:13.654934 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:19:13.654934 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 21:19:13.654934 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:19:13.654934 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:19:13.654934 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:19:13.654934 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:19:13.654934 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0yptIZULOjVMHbouVcndclXAnMKC6qm1VtMGoKOnZzI-fUPx3L5lhuz06xvHQzr3ooL8P53cH83yaSSJWcV0fJCtKGdbdY71S9WCM2F11ZTapGri39-eD6M8L5ZmeIRv6VTRaJtzqhadxTTGkGwVf5FzILXoamLnE3Jdb08lelPg3B4 Jul 16 21:19:13.654934 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:19:13.654934 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:19:13.654934 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:19:13.654934 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:19:13.654934 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:19:13.654934 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:19:13.654934 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: 24c64db8082648c8837c16352252f96d Jul 16 21:19:13.654934 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-ImagesTestJSON-1532827163 Jul 16 21:19:13.654934 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:19:13.654934 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:19:13.654934 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/24c64db8082648c8837c16352252f96d"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/24c64db8082648c8837c16352252f96d"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_24c64db8082648c8837c16352252f96d"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/24c64db8082648c8837c16352252f96d"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:19:13.654934 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-ImagesTestJSON-1532827163 Jul 16 21:19:13.654934 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: 24c64db8082648c8837c16352252f96d Jul 16 21:19:13.654934 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-ImagesTestJSON-1532827163 Jul 16 21:19:13.654934 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-ImagesTestJSON-1532827163-project Jul 16 21:19:13.654934 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:19:13.654934 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:19:13.654934 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: daf3c9e6634e4ae1bfb492eae1f44c93 Jul 16 21:19:13.654934 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-ImagesTestJSON-1532827163-project Jul 16 21:19:13.654934 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:19:13.654934 ncdv-master-1 devstack@n-api.service[138681]: exception: Instance d265192b-536b-4884-9fe8-74cf20fc7ef9 could not be found. Jul 16 21:19:13.654934 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': 'd265192b-536b-4884-9fe8-74cf20fc7ef9'} Jul 16 21:19:13.654934 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:19:13.654934 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:19:13.654934 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:19:13.654934 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:19:13.654934 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:19:13.654934 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:19:13.654934 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:19:13.654934 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:19:13.654934 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:19:13.654934 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:19:13.654934 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance d265192b-536b-4884-9fe8-74cf20fc7ef9 could not be found. Jul 16 21:19:13.654934 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:19:13.654934 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:19:13.654934 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:19:13.654934 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:19:13.654934 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:19:13.654934 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:19:13.654934 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:19:13.654934 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:19:13.654934 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:19:13.654934 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:19:13.654934 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:19:13.654934 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:19:13.654934 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:19:13.654934 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:19:13.654934 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:19:13.654934 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:19:13.654934 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance d265192b-536b-4884-9fe8-74cf20fc7ef9 could not be found. Jul 16 21:19:13.654934 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:19:13.654934 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:19:13.654934 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:19:13.654934 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:19:13.659285 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:19:13.659285 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:19:13.659285 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:19:13.659285 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:19:13.659285 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance d265192b-536b-4884-9fe8-74cf20fc7ef9 could not be found. Jul 16 21:19:13.659285 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:19:13.659285 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-36b5fd42-2e31-4072-af3e-ef80f4369ee7 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Returning 404 to user: Instance d265192b-536b-4884-9fe8-74cf20fc7ef9 could not be found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:19:13.664146 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-36b5fd42-2e31-4072-af3e-ef80f4369ee7 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] 10.222.0.16 "GET /compute/v2.1/servers/d265192b-536b-4884-9fe8-74cf20fc7ef9" status: 404 len: 111 microversion: 2.1 time: 0.086048 Jul 16 21:19:13.668355 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1281/2597] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:19:13 2022] GET /compute/v2.1/servers/d265192b-536b-4884-9fe8-74cf20fc7ef9 => generated 111 bytes in 91 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:19:13.674632 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-9efbf446-1759-4d73-91d2-7e1a688778f0 tempest-SecurityGroupRulesNegativeTestJSON-1399499068 tempest-SecurityGroupRulesNegativeTestJSON-1399499068-project] Calling method '.version_select at 0x7fdd9cfacf70>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:19:13.798436 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-8faa5ef7-b8dd-4151-b262-745e8884c5f3 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eebb224a56814f92b12ebebebc6bc8a3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a092ce9ef40943f6b6ace6cd28481b71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:19:13.830215 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-8faa5ef7-b8dd-4151-b262-745e8884c5f3 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] 10.222.0.16 "GET /compute/v2.1/servers/c16e5f09-f813-414e-b233-c08348078624" status: 200 len: 1320 microversion: 2.1 time: 0.509909 Jul 16 21:19:13.849859 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1317/2598] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:19:13 2022] GET /compute/v2.1/servers/c16e5f09-f813-414e-b233-c08348078624 => generated 1320 bytes in 530 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:19:13.860050 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-a128d28c-a71c-4c2e-b7da-2bb94c2af145 tempest-ServersAaction247Test-725202529 tempest-ServersAaction247Test-725202529-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:19:13.860953 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-a128d28c-a71c-4c2e-b7da-2bb94c2af145 tempest-ServersAaction247Test-725202529 tempest-ServersAaction247Test-725202529-project] [instance: a51adaf1-9c26-47d0-92c2-b6bb065a5ee8] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:19:13.873769 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a128d28c-a71c-4c2e-b7da-2bb94c2af145 tempest-ServersAaction247Test-725202529 tempest-ServersAaction247Test-725202529-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:13.874213 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a128d28c-a71c-4c2e-b7da-2bb94c2af145 tempest-ServersAaction247Test-725202529 tempest-ServersAaction247Test-725202529-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:13.874998 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a128d28c-a71c-4c2e-b7da-2bb94c2af145 tempest-ServersAaction247Test-725202529 tempest-ServersAaction247Test-725202529-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:13.881161 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a128d28c-a71c-4c2e-b7da-2bb94c2af145 tempest-ServersAaction247Test-725202529 tempest-ServersAaction247Test-725202529-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:14.001231 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-a128d28c-a71c-4c2e-b7da-2bb94c2af145 tempest-ServersAaction247Test-725202529 tempest-ServersAaction247Test-725202529-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a8d48b52db1545a4a51027a5c2c7fcce', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f2984288b4cd4610b2f32e895f3e5d72', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:19:14.018103 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-a128d28c-a71c-4c2e-b7da-2bb94c2af145 tempest-ServersAaction247Test-725202529 tempest-ServersAaction247Test-725202529-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'a8d48b52db1545a4a51027a5c2c7fcce', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f2984288b4cd4610b2f32e895f3e5d72', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:19:14.021243 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-a128d28c-a71c-4c2e-b7da-2bb94c2af145 tempest-ServersAaction247Test-725202529 tempest-ServersAaction247Test-725202529-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'a8d48b52db1545a4a51027a5c2c7fcce', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f2984288b4cd4610b2f32e895f3e5d72', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:19:14.026846 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-a128d28c-a71c-4c2e-b7da-2bb94c2af145 tempest-ServersAaction247Test-725202529 tempest-ServersAaction247Test-725202529-project] 10.222.0.16 "GET /compute/v2.1/servers/a51adaf1-9c26-47d0-92c2-b6bb065a5ee8" status: 200 len: 1285 microversion: 2.47 time: 0.172163 Jul 16 21:19:14.026846 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1318/2599] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:19:13 2022] GET /compute/v2.1/servers/a51adaf1-9c26-47d0-92c2-b6bb065a5ee8 => generated 1285 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:19:14.031770 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-2a71cfd8-b6e2-4867-8468-6cd7163d2b02 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Calling method '.version_select at 0x7fdd9d321700>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:19:14.428452 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-9efbf446-1759-4d73-91d2-7e1a688778f0 tempest-SecurityGroupRulesNegativeTestJSON-1399499068 tempest-SecurityGroupRulesNegativeTestJSON-1399499068-project] 10.222.0.16 "DELETE /compute/v2.1/os-security-group-rules/3dbef6cb-b5b0-4f3a-9c6a-2b09a747ce43" status: 202 len: 0 microversion: 2.1 time: 0.758635 Jul 16 21:19:14.429878 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1282/2600] 10.222.0.16 () {58 vars in 1286 bytes} [Sat Jul 16 21:19:13 2022] DELETE /compute/v2.1/os-security-group-rules/3dbef6cb-b5b0-4f3a-9c6a-2b09a747ce43 => generated 0 bytes in 760 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 21:19:14.437698 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-72034361-9023-4103-9c2a-26dab190f6bc tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Calling method '.version_select at 0x7fdd9d069790>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:19:14.466899 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-72034361-9023-4103-9c2a-26dab190f6bc tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] HTTP exception thrown: Image not found. Jul 16 21:19:14.469559 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-72034361-9023-4103-9c2a-26dab190f6bc tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Request method failure captured: Jul 16 21:19:14.469559 ncdv-master-1 devstack@n-api.service[138681]: request: DELETE /compute/v2.1/images/eee6448e-8c8a-42ef-b4df-d19c5d9ae531 HTTP/1.1 Jul 16 21:19:14.469559 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:19:14.469559 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:19:14.469559 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:19:14.469559 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 21:19:14.469559 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:19:14.469559 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:19:14.469559 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:19:14.469559 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:19:14.469559 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0yptIZULOjVMHbouVcndclXAnMKC6qm1VtMGoKOnZzI-fUPx3L5lhuz06xvHQzr3ooL8P53cH83yaSSJWcV0fJCtKGdbdY71S9WCM2F11ZTapGri39-eD6M8L5ZmeIRv6VTRaJtzqhadxTTGkGwVf5FzILXoamLnE3Jdb08lelPg3B4 Jul 16 21:19:14.469559 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:19:14.469559 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:19:14.469559 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:19:14.469559 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:19:14.469559 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:19:14.469559 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:19:14.469559 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: 24c64db8082648c8837c16352252f96d Jul 16 21:19:14.469559 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-ImagesTestJSON-1532827163 Jul 16 21:19:14.469559 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:19:14.469559 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:19:14.469559 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/24c64db8082648c8837c16352252f96d"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/24c64db8082648c8837c16352252f96d"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_24c64db8082648c8837c16352252f96d"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/24c64db8082648c8837c16352252f96d"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:19:14.469559 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-ImagesTestJSON-1532827163 Jul 16 21:19:14.469559 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: 24c64db8082648c8837c16352252f96d Jul 16 21:19:14.469559 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-ImagesTestJSON-1532827163 Jul 16 21:19:14.469559 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-ImagesTestJSON-1532827163-project Jul 16 21:19:14.469559 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:19:14.469559 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:19:14.469559 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: daf3c9e6634e4ae1bfb492eae1f44c93 Jul 16 21:19:14.469559 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-ImagesTestJSON-1532827163-project Jul 16 21:19:14.469559 ncdv-master-1 devstack@n-api.service[138681]: method: Jul 16 21:19:14.469559 ncdv-master-1 devstack@n-api.service[138681]: exception: Image not found. Jul 16 21:19:14.469559 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': 'eee6448e-8c8a-42ef-b4df-d19c5d9ae531'} Jul 16 21:19:14.469559 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:19:14.469559 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:19:14.469559 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 724, in delete Jul 16 21:19:14.469559 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi self._client.call(context, 2, 'delete', args=(image_id,)) Jul 16 21:19:14.469559 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 191, in call Jul 16 21:19:14.469559 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi result = getattr(controller, method)(*args, **kwargs) Jul 16 21:19:14.469559 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/utils.py", line 670, in inner Jul 16 21:19:14.469559 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return RequestIdProxy(wrapped(*args, **kwargs)) Jul 16 21:19:14.469559 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/v2/images.py", line 386, in delete Jul 16 21:19:14.469559 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi resp, body = self.http_client.delete(url) Jul 16 21:19:14.469559 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/adapter.py", line 410, in delete Jul 16 21:19:14.469559 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return self.request(url, 'DELETE', **kwargs) Jul 16 21:19:14.469559 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 380, in request Jul 16 21:19:14.469559 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return self._handle_response(resp) Jul 16 21:19:14.469559 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 120, in _handle_response Jul 16 21:19:14.469559 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.from_response(resp, resp.content) Jul 16 21:19:14.469559 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi glanceclient.exc.HTTPNotFound: HTTP 404 Not Found: Failed to find image eee6448e-8c8a-42ef-b4df-d19c5d9ae531 to delete Jul 16 21:19:14.469559 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:19:14.469559 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:19:14.469559 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:19:14.469559 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:19:14.469559 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/images.py", line 104, in delete Jul 16 21:19:14.469559 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi self._image_api.delete(context, id) Jul 16 21:19:14.469559 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 1255, in delete Jul 16 21:19:14.469559 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return session.delete(context, image_id) Jul 16 21:19:14.469559 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 726, in delete Jul 16 21:19:14.469559 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.ImageNotFound(image_id=image_id) Jul 16 21:19:14.469559 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.ImageNotFound: Image eee6448e-8c8a-42ef-b4df-d19c5d9ae531 could not be found. Jul 16 21:19:14.469559 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:19:14.469559 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:19:14.469559 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:19:14.469559 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:19:14.473434 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:19:14.473434 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:19:14.473434 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:19:14.473434 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:19:14.473434 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Jul 16 21:19:14.473434 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Jul 16 21:19:14.473434 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:19:14.473434 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:19:14.473434 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/images.py", line 107, in delete Jul 16 21:19:14.473434 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=explanation) Jul 16 21:19:14.473434 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Image not found. Jul 16 21:19:14.473434 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:19:14.473434 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:19:14.473434 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:19:14.473434 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:19:14.473434 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:19:14.473434 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:19:14.473434 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:19:14.473434 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:19:14.473434 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Image not found. Jul 16 21:19:14.473434 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:19:14.473434 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-72034361-9023-4103-9c2a-26dab190f6bc tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Returning 404 to user: Image not found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:19:14.473434 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-72034361-9023-4103-9c2a-26dab190f6bc tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] 10.222.0.16 "DELETE /compute/v2.1/images/eee6448e-8c8a-42ef-b4df-d19c5d9ae531" status: 404 len: 62 microversion: 2.1 time: 0.039042 Jul 16 21:19:14.473434 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1283/2601] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:19:14 2022] DELETE /compute/v2.1/images/eee6448e-8c8a-42ef-b4df-d19c5d9ae531 => generated 62 bytes in 42 msecs (HTTP/1.1 404) 9 headers in 378 bytes (1 switches on core 0) Jul 16 21:19:14.479055 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-602b2322-b59a-48f7-9c62-0c1b51bdd3fd tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:19:14.479869 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-602b2322-b59a-48f7-9c62-0c1b51bdd3fd tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] [instance: e7598b4a-f70c-4186-88d1-6fd0057f8656] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:19:14.486374 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-602b2322-b59a-48f7-9c62-0c1b51bdd3fd tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:14.486830 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-602b2322-b59a-48f7-9c62-0c1b51bdd3fd tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:14.487619 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-602b2322-b59a-48f7-9c62-0c1b51bdd3fd tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:14.488124 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-602b2322-b59a-48f7-9c62-0c1b51bdd3fd tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:14.579957 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-2a71cfd8-b6e2-4867-8468-6cd7163d2b02 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] 10.222.0.16 "DELETE /compute/v2.1/os-security-groups/c176d065-4e8f-45fb-99d9-e12db1c30218" status: 202 len: 0 microversion: 2.1 time: 0.554035 Jul 16 21:19:14.580629 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1319/2602] 10.222.0.16 () {58 vars in 1271 bytes} [Sat Jul 16 21:19:14 2022] DELETE /compute/v2.1/os-security-groups/c176d065-4e8f-45fb-99d9-e12db1c30218 => generated 0 bytes in 555 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 21:19:14.596159 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-8576dd1f-5c46-4adf-a9fd-47c9c5aa8200 tempest-VolumesSnapshotsTestJSON-851680556 tempest-VolumesSnapshotsTestJSON-851680556-project] Calling method '.version_select at 0x7fdd9cc47b80>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:19:14.659459 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-602b2322-b59a-48f7-9c62-0c1b51bdd3fd tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f5bc1f2209f4eb186d80e385b67c774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '65d5bcfa02af4e8284ba1342197e2860', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:19:14.662052 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-8576dd1f-5c46-4adf-a9fd-47c9c5aa8200 tempest-VolumesSnapshotsTestJSON-851680556 tempest-VolumesSnapshotsTestJSON-851680556-project] 10.222.0.16 "GET /compute/v2.1/os-volumes/4450505c-3ac5-42c2-b94c-b8f945007d50" status: 200 len: 338 microversion: 2.1 time: 0.078702 Jul 16 21:19:14.662655 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1320/2603] 10.222.0.16 () {58 vars in 1244 bytes} [Sat Jul 16 21:19:14 2022] GET /compute/v2.1/os-volumes/4450505c-3ac5-42c2-b94c-b8f945007d50 => generated 338 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:19:14.673547 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-602b2322-b59a-48f7-9c62-0c1b51bdd3fd tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] 10.222.0.16 "GET /compute/v2.1/servers/e7598b4a-f70c-4186-88d1-6fd0057f8656" status: 200 len: 1472 microversion: 2.1 time: 0.199783 Jul 16 21:19:14.675362 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1284/2604] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:19:14 2022] GET /compute/v2.1/servers/e7598b4a-f70c-4186-88d1-6fd0057f8656 => generated 1472 bytes in 202 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:19:14.677057 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-c4414e23-ed42-4103-822b-17f4c6e555b5 req-d3bd166d-038e-4359-afba-0822c07b68e5 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "d265192b-536b-4884-9fe8-74cf20fc7ef9", "name": "network-vif-deleted", "tag": "8a340e39-7064-4a42-b19e-e37841899e42"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:19:14.691263 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-bdbc0aad-8070-4bd4-894c-8d8a3e3f1ab3 tempest-SecurityGroupRulesNegativeTestJSON-1399499068 tempest-SecurityGroupRulesNegativeTestJSON-1399499068-project] Action: 'create', calling method: .version_select at 0x7fdd9d221550>, body: {"security_group": {"name": "tempest-SecurityGroupRulesNegativeTestJSON-securitygroup-1861497632", "description": "tempest-description-789665872"}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:19:14.700972 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-c4414e23-ed42-4103-822b-17f4c6e555b5 req-d3bd166d-038e-4359-afba-0822c07b68e5 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:14.701431 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-c4414e23-ed42-4103-822b-17f4c6e555b5 req-d3bd166d-038e-4359-afba-0822c07b68e5 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:14.716121 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-c4414e23-ed42-4103-822b-17f4c6e555b5 req-d3bd166d-038e-4359-afba-0822c07b68e5 service nova] Creating event network-vif-deleted:8a340e39-7064-4a42-b19e-e37841899e42 for instance d265192b-536b-4884-9fe8-74cf20fc7ef9 on nch2-master-1 Jul 16 21:19:14.724948 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-c4414e23-ed42-4103-822b-17f4c6e555b5 req-d3bd166d-038e-4359-afba-0822c07b68e5 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.060393 Jul 16 21:19:14.726514 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1321/2605] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:19:14 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 63 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:19:14.737513 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-32223064-8fc5-46ee-b150-7fcceb936122 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:19:14.739172 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-32223064-8fc5-46ee-b150-7fcceb936122 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] [instance: ed8dcb97-7546-45ef-9e9d-a9bb03a1b419] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:19:14.749282 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-32223064-8fc5-46ee-b150-7fcceb936122 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:14.750199 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-32223064-8fc5-46ee-b150-7fcceb936122 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:14.751256 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-32223064-8fc5-46ee-b150-7fcceb936122 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:14.751909 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-32223064-8fc5-46ee-b150-7fcceb936122 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:14.773323 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-32223064-8fc5-46ee-b150-7fcceb936122 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] HTTP exception thrown: Instance ed8dcb97-7546-45ef-9e9d-a9bb03a1b419 could not be found. Jul 16 21:19:14.776281 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-32223064-8fc5-46ee-b150-7fcceb936122 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Request method failure captured: Jul 16 21:19:14.776281 ncdv-master-1 devstack@n-api.service[138680]: request: DELETE /compute/v2.1/servers/ed8dcb97-7546-45ef-9e9d-a9bb03a1b419 HTTP/1.1 Jul 16 21:19:14.776281 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:19:14.776281 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:19:14.776281 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:19:14.776281 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 0 Jul 16 21:19:14.776281 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:19:14.776281 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:19:14.776281 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:19:14.776281 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:19:14.776281 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0yptIZULOjVMHbouVcndclXAnMKC6qm1VtMGoKOnZzI-fUPx3L5lhuz06xvHQzr3ooL8P53cH83yaSSJWcV0fJCtKGdbdY71S9WCM2F11ZTapGri39-eD6M8L5ZmeIRv6VTRaJtzqhadxTTGkGwVf5FzILXoamLnE3Jdb08lelPg3B4 Jul 16 21:19:14.776281 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:19:14.776281 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:19:14.776281 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:19:14.776281 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:19:14.776281 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:19:14.776281 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:19:14.776281 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: 24c64db8082648c8837c16352252f96d Jul 16 21:19:14.776281 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-ImagesTestJSON-1532827163 Jul 16 21:19:14.776281 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:19:14.776281 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:19:14.776281 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/24c64db8082648c8837c16352252f96d"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/24c64db8082648c8837c16352252f96d"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_24c64db8082648c8837c16352252f96d"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/24c64db8082648c8837c16352252f96d"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:19:14.776281 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-ImagesTestJSON-1532827163 Jul 16 21:19:14.776281 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: 24c64db8082648c8837c16352252f96d Jul 16 21:19:14.776281 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-ImagesTestJSON-1532827163 Jul 16 21:19:14.776281 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-ImagesTestJSON-1532827163-project Jul 16 21:19:14.776281 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:19:14.776281 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:19:14.776281 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: daf3c9e6634e4ae1bfb492eae1f44c93 Jul 16 21:19:14.776281 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-ImagesTestJSON-1532827163-project Jul 16 21:19:14.776281 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 21:19:14.776281 ncdv-master-1 devstack@n-api.service[138680]: exception: Instance ed8dcb97-7546-45ef-9e9d-a9bb03a1b419 could not be found. Jul 16 21:19:14.776281 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': 'ed8dcb97-7546-45ef-9e9d-a9bb03a1b419'} Jul 16 21:19:14.776281 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:19:14.776281 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:19:14.776281 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:19:14.776281 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:19:14.776281 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:19:14.776281 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:19:14.776281 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:19:14.776281 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:19:14.776281 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:19:14.776281 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:19:14.776281 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance ed8dcb97-7546-45ef-9e9d-a9bb03a1b419 could not be found. Jul 16 21:19:14.776281 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:19:14.776281 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:19:14.776281 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:19:14.776281 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:19:14.776281 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:19:14.776281 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:19:14.776281 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:19:14.776281 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:19:14.776281 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:19:14.776281 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:19:14.776281 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Jul 16 21:19:14.776281 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Jul 16 21:19:14.776281 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Jul 16 21:19:14.776281 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Jul 16 21:19:14.776281 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:19:14.776281 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:19:14.776281 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:19:14.776281 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:19:14.776281 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance ed8dcb97-7546-45ef-9e9d-a9bb03a1b419 could not be found. Jul 16 21:19:14.780796 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:19:14.780796 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:19:14.780796 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:19:14.780796 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:19:14.780796 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:19:14.780796 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:19:14.780796 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:19:14.780796 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:19:14.780796 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance ed8dcb97-7546-45ef-9e9d-a9bb03a1b419 could not be found. Jul 16 21:19:14.780796 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:19:14.780796 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-32223064-8fc5-46ee-b150-7fcceb936122 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Returning 404 to user: Instance ed8dcb97-7546-45ef-9e9d-a9bb03a1b419 could not be found. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:19:14.783681 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-32223064-8fc5-46ee-b150-7fcceb936122 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] 10.222.0.16 "DELETE /compute/v2.1/servers/ed8dcb97-7546-45ef-9e9d-a9bb03a1b419" status: 404 len: 111 microversion: 2.1 time: 0.052994 Jul 16 21:19:14.785644 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1322/2606] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:19:14 2022] DELETE /compute/v2.1/servers/ed8dcb97-7546-45ef-9e9d-a9bb03a1b419 => generated 111 bytes in 58 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:19:14.792883 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-5d56ede5-73a5-4714-9283-70212776e21f tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Action: 'create', calling method: .version_select at 0x7fdd9cc3fdc0>, body: {"security_group": {"name": "tempest-SecurityGroupsTestJSON-securitygroup-1793322749", "description": "tempest-description-1429199255"}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:19:15.679506 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-bdbc0aad-8070-4bd4-894c-8d8a3e3f1ab3 tempest-SecurityGroupRulesNegativeTestJSON-1399499068 tempest-SecurityGroupRulesNegativeTestJSON-1399499068-project] 10.222.0.16 "POST /compute/v2.1/os-security-groups" status: 200 len: 255 microversion: 2.1 time: 1.002134 Jul 16 21:19:15.682070 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1285/2607] 10.222.0.16 () {60 vars in 1179 bytes} [Sat Jul 16 21:19:14 2022] POST /compute/v2.1/os-security-groups => generated 255 bytes in 1006 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:19:15.693618 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-0e22c2d3-87a0-431f-a469-82a51fe02cca tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:19:15.694332 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-0e22c2d3-87a0-431f-a469-82a51fe02cca tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] [instance: ed8dcb97-7546-45ef-9e9d-a9bb03a1b419] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:19:15.704219 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0e22c2d3-87a0-431f-a469-82a51fe02cca tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:15.704946 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0e22c2d3-87a0-431f-a469-82a51fe02cca tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:15.705883 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0e22c2d3-87a0-431f-a469-82a51fe02cca tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:15.706474 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0e22c2d3-87a0-431f-a469-82a51fe02cca tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:15.735676 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-0e22c2d3-87a0-431f-a469-82a51fe02cca tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] HTTP exception thrown: Instance ed8dcb97-7546-45ef-9e9d-a9bb03a1b419 could not be found. Jul 16 21:19:15.737512 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-0e22c2d3-87a0-431f-a469-82a51fe02cca tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Request method failure captured: Jul 16 21:19:15.737512 ncdv-master-1 devstack@n-api.service[138681]: request: GET /compute/v2.1/servers/ed8dcb97-7546-45ef-9e9d-a9bb03a1b419 HTTP/1.1 Jul 16 21:19:15.737512 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:19:15.737512 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:19:15.737512 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:19:15.737512 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 21:19:15.737512 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:19:15.737512 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:19:15.737512 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:19:15.737512 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:19:15.737512 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0yptIZULOjVMHbouVcndclXAnMKC6qm1VtMGoKOnZzI-fUPx3L5lhuz06xvHQzr3ooL8P53cH83yaSSJWcV0fJCtKGdbdY71S9WCM2F11ZTapGri39-eD6M8L5ZmeIRv6VTRaJtzqhadxTTGkGwVf5FzILXoamLnE3Jdb08lelPg3B4 Jul 16 21:19:15.737512 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:19:15.737512 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:19:15.737512 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:19:15.737512 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:19:15.737512 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:19:15.737512 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:19:15.737512 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: 24c64db8082648c8837c16352252f96d Jul 16 21:19:15.737512 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-ImagesTestJSON-1532827163 Jul 16 21:19:15.737512 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:19:15.737512 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:19:15.737512 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/24c64db8082648c8837c16352252f96d"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/24c64db8082648c8837c16352252f96d"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_24c64db8082648c8837c16352252f96d"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/24c64db8082648c8837c16352252f96d"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:19:15.737512 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-ImagesTestJSON-1532827163 Jul 16 21:19:15.737512 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: 24c64db8082648c8837c16352252f96d Jul 16 21:19:15.737512 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-ImagesTestJSON-1532827163 Jul 16 21:19:15.737512 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-ImagesTestJSON-1532827163-project Jul 16 21:19:15.737512 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:19:15.737512 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:19:15.737512 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: daf3c9e6634e4ae1bfb492eae1f44c93 Jul 16 21:19:15.737512 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-ImagesTestJSON-1532827163-project Jul 16 21:19:15.737512 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:19:15.737512 ncdv-master-1 devstack@n-api.service[138681]: exception: Instance ed8dcb97-7546-45ef-9e9d-a9bb03a1b419 could not be found. Jul 16 21:19:15.737512 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': 'ed8dcb97-7546-45ef-9e9d-a9bb03a1b419'} Jul 16 21:19:15.737512 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:19:15.737512 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:19:15.737512 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:19:15.737512 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:19:15.737512 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:19:15.737512 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:19:15.737512 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:19:15.737512 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:19:15.737512 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:19:15.737512 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:19:15.737512 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance ed8dcb97-7546-45ef-9e9d-a9bb03a1b419 could not be found. Jul 16 21:19:15.737512 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:19:15.737512 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:19:15.737512 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:19:15.737512 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:19:15.737512 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:19:15.737512 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:19:15.737512 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:19:15.737512 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:19:15.737512 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:19:15.737512 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:19:15.737512 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:19:15.737512 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:19:15.737512 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:19:15.737512 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:19:15.737512 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:19:15.737512 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:19:15.737512 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance ed8dcb97-7546-45ef-9e9d-a9bb03a1b419 could not be found. Jul 16 21:19:15.737512 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:19:15.737512 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:19:15.737512 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:19:15.737512 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:19:15.741014 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:19:15.741014 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:19:15.741014 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:19:15.741014 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:19:15.741014 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance ed8dcb97-7546-45ef-9e9d-a9bb03a1b419 could not be found. Jul 16 21:19:15.741014 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:19:15.747207 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-0e22c2d3-87a0-431f-a469-82a51fe02cca tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Returning 404 to user: Instance ed8dcb97-7546-45ef-9e9d-a9bb03a1b419 could not be found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:19:15.748452 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-0e22c2d3-87a0-431f-a469-82a51fe02cca tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] 10.222.0.16 "GET /compute/v2.1/servers/ed8dcb97-7546-45ef-9e9d-a9bb03a1b419" status: 404 len: 111 microversion: 2.1 time: 0.064983 Jul 16 21:19:15.751209 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1286/2608] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:19:15 2022] GET /compute/v2.1/servers/ed8dcb97-7546-45ef-9e9d-a9bb03a1b419 => generated 111 bytes in 68 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:19:15.758303 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-15c1459c-f486-426e-8685-1df57d94f304 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:19:15.759192 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-15c1459c-f486-426e-8685-1df57d94f304 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] [instance: c16e5f09-f813-414e-b233-c08348078624] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:19:15.770289 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-5d56ede5-73a5-4714-9283-70212776e21f tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] 10.222.0.16 "POST /compute/v2.1/os-security-groups" status: 200 len: 244 microversion: 2.1 time: 0.982882 Jul 16 21:19:15.774035 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-15c1459c-f486-426e-8685-1df57d94f304 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:15.775163 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-15c1459c-f486-426e-8685-1df57d94f304 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:15.777746 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1323/2609] 10.222.0.16 () {60 vars in 1179 bytes} [Sat Jul 16 21:19:14 2022] POST /compute/v2.1/os-security-groups => generated 244 bytes in 991 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:19:15.778144 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-15c1459c-f486-426e-8685-1df57d94f304 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:15.778753 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-15c1459c-f486-426e-8685-1df57d94f304 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:15.788231 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-e0d7d3aa-d8de-4d54-a49d-b2baffd8cc97 tempest-ServersAaction247Test-725202529 tempest-ServersAaction247Test-725202529-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:19:15.789979 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-e0d7d3aa-d8de-4d54-a49d-b2baffd8cc97 tempest-ServersAaction247Test-725202529 tempest-ServersAaction247Test-725202529-project] [instance: a51adaf1-9c26-47d0-92c2-b6bb065a5ee8] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:19:15.800624 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e0d7d3aa-d8de-4d54-a49d-b2baffd8cc97 tempest-ServersAaction247Test-725202529 tempest-ServersAaction247Test-725202529-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:15.801653 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e0d7d3aa-d8de-4d54-a49d-b2baffd8cc97 tempest-ServersAaction247Test-725202529 tempest-ServersAaction247Test-725202529-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:15.805561 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e0d7d3aa-d8de-4d54-a49d-b2baffd8cc97 tempest-ServersAaction247Test-725202529 tempest-ServersAaction247Test-725202529-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:15.805561 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e0d7d3aa-d8de-4d54-a49d-b2baffd8cc97 tempest-ServersAaction247Test-725202529 tempest-ServersAaction247Test-725202529-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:15.932998 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-e0d7d3aa-d8de-4d54-a49d-b2baffd8cc97 tempest-ServersAaction247Test-725202529 tempest-ServersAaction247Test-725202529-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a8d48b52db1545a4a51027a5c2c7fcce', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f2984288b4cd4610b2f32e895f3e5d72', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:19:15.951162 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-e0d7d3aa-d8de-4d54-a49d-b2baffd8cc97 tempest-ServersAaction247Test-725202529 tempest-ServersAaction247Test-725202529-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'a8d48b52db1545a4a51027a5c2c7fcce', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f2984288b4cd4610b2f32e895f3e5d72', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:19:15.951162 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-e0d7d3aa-d8de-4d54-a49d-b2baffd8cc97 tempest-ServersAaction247Test-725202529 tempest-ServersAaction247Test-725202529-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'a8d48b52db1545a4a51027a5c2c7fcce', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f2984288b4cd4610b2f32e895f3e5d72', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:19:15.951162 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-e0d7d3aa-d8de-4d54-a49d-b2baffd8cc97 tempest-ServersAaction247Test-725202529 tempest-ServersAaction247Test-725202529-project] 10.222.0.16 "GET /compute/v2.1/servers/a51adaf1-9c26-47d0-92c2-b6bb065a5ee8" status: 200 len: 1339 microversion: 2.47 time: 0.163667 Jul 16 21:19:15.951162 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1324/2610] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:19:15 2022] GET /compute/v2.1/servers/a51adaf1-9c26-47d0-92c2-b6bb065a5ee8 => generated 1339 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:19:15.962316 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-609c6ccd-8553-41a4-9f0f-a6178d084459 req-8bae6395-29fe-4842-b618-2ecda0e502f3 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "21522211-8382-4958-ad2a-cff26f4f2305", "tag": "9805a0f5-a6ed-4cb8-ac51-b1d03a7a9ec6"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:19:15.971588 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-609c6ccd-8553-41a4-9f0f-a6178d084459 req-8bae6395-29fe-4842-b618-2ecda0e502f3 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:15.972047 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-609c6ccd-8553-41a4-9f0f-a6178d084459 req-8bae6395-29fe-4842-b618-2ecda0e502f3 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:15.986270 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-609c6ccd-8553-41a4-9f0f-a6178d084459 req-8bae6395-29fe-4842-b618-2ecda0e502f3 service nova] Creating event network-changed:9805a0f5-a6ed-4cb8-ac51-b1d03a7a9ec6 for instance 21522211-8382-4958-ad2a-cff26f4f2305 on nch2-master-1 Jul 16 21:19:15.993364 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-609c6ccd-8553-41a4-9f0f-a6178d084459 req-8bae6395-29fe-4842-b618-2ecda0e502f3 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.041202 Jul 16 21:19:15.994521 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1325/2611] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:19:15 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:19:16.001787 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-debab421-b822-4937-8c4d-075faf66199c tempest-VolumesSnapshotsTestJSON-851680556 tempest-VolumesSnapshotsTestJSON-851680556-project] Calling method '.version_select at 0x7fdda2b1ae50>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:19:16.087619 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-debab421-b822-4937-8c4d-075faf66199c tempest-VolumesSnapshotsTestJSON-851680556 tempest-VolumesSnapshotsTestJSON-851680556-project] 10.222.0.16 "GET /compute/v2.1/os-volumes/4450505c-3ac5-42c2-b94c-b8f945007d50" status: 200 len: 338 microversion: 2.1 time: 0.091262 Jul 16 21:19:16.088219 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1326/2612] 10.222.0.16 () {58 vars in 1244 bytes} [Sat Jul 16 21:19:15 2022] GET /compute/v2.1/os-volumes/4450505c-3ac5-42c2-b94c-b8f945007d50 => generated 338 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:19:16.095304 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-d8542782-52ab-423a-afdc-32dd6086b062 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:19:16.096153 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-d8542782-52ab-423a-afdc-32dd6086b062 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] [instance: e7598b4a-f70c-4186-88d1-6fd0057f8656] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:19:16.105092 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d8542782-52ab-423a-afdc-32dd6086b062 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:16.106311 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d8542782-52ab-423a-afdc-32dd6086b062 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:16.108598 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d8542782-52ab-423a-afdc-32dd6086b062 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:16.109272 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d8542782-52ab-423a-afdc-32dd6086b062 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:16.122659 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-15c1459c-f486-426e-8685-1df57d94f304 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eebb224a56814f92b12ebebebc6bc8a3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a092ce9ef40943f6b6ace6cd28481b71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:19:16.131314 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-15c1459c-f486-426e-8685-1df57d94f304 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] 10.222.0.16 "GET /compute/v2.1/servers/c16e5f09-f813-414e-b233-c08348078624" status: 200 len: 1320 microversion: 2.1 time: 0.378727 Jul 16 21:19:16.132362 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1287/2613] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:19:15 2022] GET /compute/v2.1/servers/c16e5f09-f813-414e-b233-c08348078624 => generated 1320 bytes in 380 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:19:16.147184 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:19:16.155983 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-090f179f-2229-42c9-81ce-b83221531789 tempest-SecurityGroupRulesNegativeTestJSON-1399499068 tempest-SecurityGroupRulesNegativeTestJSON-1399499068-project] Action: 'create', calling method: .version_select at 0x7fdd9d090430>, body: {"security_group_rule": {"parent_group_id": "2f2e2e5c-6442-41c9-9ea0-a8fc5950b918", "ip_protocol": "tcp", "from_port": 1252175047, "to_port": 22}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:19:16.354068 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-090f179f-2229-42c9-81ce-b83221531789 tempest-SecurityGroupRulesNegativeTestJSON-1399499068 tempest-SecurityGroupRulesNegativeTestJSON-1399499068-project] HTTP exception thrown: Invalid port range 1252175047:22. Former value cannot be greater than the later Jul 16 21:19:16.355813 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-090f179f-2229-42c9-81ce-b83221531789 tempest-SecurityGroupRulesNegativeTestJSON-1399499068 tempest-SecurityGroupRulesNegativeTestJSON-1399499068-project] Request method failure captured: Jul 16 21:19:16.355813 ncdv-master-1 devstack@n-api.service[138681]: request: POST /compute/v2.1/os-security-group-rules HTTP/1.1 Jul 16 21:19:16.355813 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:19:16.355813 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:19:16.355813 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:19:16.355813 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 146 Jul 16 21:19:16.355813 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:19:16.355813 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:19:16.355813 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:19:16.355813 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:19:16.355813 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0ytNkmtXKapUZpFKPGUA8hf3ATbduJ0XtzorjW7amdLo0WS0Qbkqbi3B3wBROLLwxfXJZxCXXmOW4by85qgjbrIW-gGZ2c_YrmiVSX-WID5VqAWhi2CmsmYLeeu_6bw9QJDULh6nweMasTNNM3cnBfI9QUQLgKaJIeA4QYZkK1W0reQ Jul 16 21:19:16.355813 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:19:16.355813 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:19:16.355813 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:19:16.355813 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:19:16.355813 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:19:16.355813 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:19:16.355813 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: 89b4d12252b94d54b50cc15238af6c2f Jul 16 21:19:16.355813 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-SecurityGroupRulesNegativeTestJSON-1399499068 Jul 16 21:19:16.355813 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:19:16.355813 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:19:16.355813 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/89b4d12252b94d54b50cc15238af6c2f"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/89b4d12252b94d54b50cc15238af6c2f"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_89b4d12252b94d54b50cc15238af6c2f"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/89b4d12252b94d54b50cc15238af6c2f"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:19:16.355813 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-SecurityGroupRulesNegativeTestJSON-1399499068 Jul 16 21:19:16.355813 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: 89b4d12252b94d54b50cc15238af6c2f Jul 16 21:19:16.355813 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-SecurityGroupRulesNegativeTestJSON-1399499068 Jul 16 21:19:16.355813 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-SecurityGroupRulesNegativeTestJSON-1399499068-project Jul 16 21:19:16.355813 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:19:16.355813 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:19:16.355813 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: 7ea30eb37fbf469194845261a19e8d4f Jul 16 21:19:16.355813 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-SecurityGroupRulesNegativeTestJSON-1399499068-project Jul 16 21:19:16.355813 ncdv-master-1 devstack@n-api.service[138681]: Jul 16 21:19:16.355813 ncdv-master-1 devstack@n-api.service[138681]: {"security_group_rule": {"parent_group_id": "2f2e2e5c-6442-41c9-9ea0-a8fc5950b918", "ip_protocol": "tcp", "from_port": 1252175047, "to_port": 22}} Jul 16 21:19:16.355813 ncdv-master-1 devstack@n-api.service[138681]: method: Jul 16 21:19:16.355813 ncdv-master-1 devstack@n-api.service[138681]: exception: Invalid port range 1252175047:22. Former value cannot be greater than the later Jul 16 21:19:16.355813 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'body': {'security_group_rule': {'parent_group_id': '2f2e2e5c-6442-41c9-9ea0-a8fc5950b918', 'ip_protocol': 'tcp', 'from_port': 1252175047, 'to_port': 22}}} Jul 16 21:19:16.355813 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:19:16.355813 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:19:16.355813 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/security_groups.py", line 294, in create Jul 16 21:19:16.355813 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi new_rule = self._rule_args_to_dict(context, Jul 16 21:19:16.355813 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/security_groups.py", line 346, in _rule_args_to_dict Jul 16 21:19:16.355813 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return security_group_api.new_cidr_ingress_rule( Jul 16 21:19:16.355813 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/security_group_api.py", line 103, in new_cidr_ingress_rule Jul 16 21:19:16.355813 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return _new_ingress_rule( Jul 16 21:19:16.355813 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/security_group_api.py", line 150, in _new_ingress_rule Jul 16 21:19:16.355813 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.InvalidPortRange(from_port=from_port, Jul 16 21:19:16.355813 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.InvalidPortRange: Invalid port range 1252175047:22. Former value cannot be greater than the later Jul 16 21:19:16.355813 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:19:16.355813 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:19:16.355813 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:19:16.355813 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:19:16.355813 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:19:16.355813 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:19:16.355813 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:19:16.355813 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:19:16.355813 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Jul 16 21:19:16.355813 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Jul 16 21:19:16.355813 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:19:16.355813 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:19:16.355813 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/security_groups.py", line 301, in create Jul 16 21:19:16.355813 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=exp.format_message()) Jul 16 21:19:16.355813 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid port range 1252175047:22. Former value cannot be greater than the later Jul 16 21:19:16.355813 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:19:16.359301 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:19:16.359301 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:19:16.359301 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:19:16.359301 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:19:16.359301 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:19:16.359301 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:19:16.359301 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:19:16.359301 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid port range 1252175047:22. Former value cannot be greater than the later Jul 16 21:19:16.359301 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:19:16.359301 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-090f179f-2229-42c9-81ce-b83221531789 tempest-SecurityGroupRulesNegativeTestJSON-1399499068 tempest-SecurityGroupRulesNegativeTestJSON-1399499068-project] Returning 400 to user: Invalid port range 1252175047:22. Former value cannot be greater than the later {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:19:16.359301 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-090f179f-2229-42c9-81ce-b83221531789 tempest-SecurityGroupRulesNegativeTestJSON-1399499068 tempest-SecurityGroupRulesNegativeTestJSON-1399499068-project] 10.222.0.16 "POST /compute/v2.1/os-security-group-rules" status: 400 len: 123 microversion: 2.1 time: 0.223654 Jul 16 21:19:16.364646 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1288/2614] 10.222.0.16 () {60 vars in 1194 bytes} [Sat Jul 16 21:19:16 2022] POST /compute/v2.1/os-security-group-rules => generated 123 bytes in 232 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Jul 16 21:19:16.372020 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-55de1ee2-6972-4833-87cc-2701a0b4c086 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Calling method '.version_select at 0x7fdd9d0cc0d0>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:19:16.421606 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-55de1ee2-6972-4833-87cc-2701a0b4c086 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] HTTP exception thrown: Image not found. Jul 16 21:19:16.422878 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-55de1ee2-6972-4833-87cc-2701a0b4c086 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Request method failure captured: Jul 16 21:19:16.422878 ncdv-master-1 devstack@n-api.service[138681]: request: DELETE /compute/v2.1/images/211719d4-d58e-474c-ade9-8ffb307248f3 HTTP/1.1 Jul 16 21:19:16.422878 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:19:16.422878 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:19:16.422878 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:19:16.422878 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 21:19:16.422878 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:19:16.422878 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:19:16.422878 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:19:16.422878 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:19:16.422878 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0yptIZULOjVMHbouVcndclXAnMKC6qm1VtMGoKOnZzI-fUPx3L5lhuz06xvHQzr3ooL8P53cH83yaSSJWcV0fJCtKGdbdY71S9WCM2F11ZTapGri39-eD6M8L5ZmeIRv6VTRaJtzqhadxTTGkGwVf5FzILXoamLnE3Jdb08lelPg3B4 Jul 16 21:19:16.422878 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:19:16.422878 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:19:16.422878 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:19:16.422878 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:19:16.422878 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:19:16.422878 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:19:16.422878 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: 24c64db8082648c8837c16352252f96d Jul 16 21:19:16.422878 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-ImagesTestJSON-1532827163 Jul 16 21:19:16.422878 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:19:16.422878 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:19:16.422878 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/24c64db8082648c8837c16352252f96d"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/24c64db8082648c8837c16352252f96d"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_24c64db8082648c8837c16352252f96d"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/24c64db8082648c8837c16352252f96d"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:19:16.422878 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-ImagesTestJSON-1532827163 Jul 16 21:19:16.422878 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: 24c64db8082648c8837c16352252f96d Jul 16 21:19:16.422878 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-ImagesTestJSON-1532827163 Jul 16 21:19:16.422878 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-ImagesTestJSON-1532827163-project Jul 16 21:19:16.422878 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:19:16.422878 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:19:16.422878 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: daf3c9e6634e4ae1bfb492eae1f44c93 Jul 16 21:19:16.422878 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-ImagesTestJSON-1532827163-project Jul 16 21:19:16.422878 ncdv-master-1 devstack@n-api.service[138681]: method: Jul 16 21:19:16.422878 ncdv-master-1 devstack@n-api.service[138681]: exception: Image not found. Jul 16 21:19:16.422878 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': '211719d4-d58e-474c-ade9-8ffb307248f3'} Jul 16 21:19:16.422878 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:19:16.422878 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:19:16.422878 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 724, in delete Jul 16 21:19:16.422878 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi self._client.call(context, 2, 'delete', args=(image_id,)) Jul 16 21:19:16.422878 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 191, in call Jul 16 21:19:16.422878 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi result = getattr(controller, method)(*args, **kwargs) Jul 16 21:19:16.422878 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/utils.py", line 670, in inner Jul 16 21:19:16.422878 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return RequestIdProxy(wrapped(*args, **kwargs)) Jul 16 21:19:16.422878 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/v2/images.py", line 386, in delete Jul 16 21:19:16.422878 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi resp, body = self.http_client.delete(url) Jul 16 21:19:16.422878 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/adapter.py", line 410, in delete Jul 16 21:19:16.422878 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return self.request(url, 'DELETE', **kwargs) Jul 16 21:19:16.422878 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 380, in request Jul 16 21:19:16.422878 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return self._handle_response(resp) Jul 16 21:19:16.422878 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 120, in _handle_response Jul 16 21:19:16.422878 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.from_response(resp, resp.content) Jul 16 21:19:16.422878 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi glanceclient.exc.HTTPNotFound: HTTP 404 Not Found: Failed to find image 211719d4-d58e-474c-ade9-8ffb307248f3 to delete Jul 16 21:19:16.422878 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:19:16.422878 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:19:16.422878 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:19:16.422878 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:19:16.422878 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/images.py", line 104, in delete Jul 16 21:19:16.422878 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi self._image_api.delete(context, id) Jul 16 21:19:16.422878 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 1255, in delete Jul 16 21:19:16.422878 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return session.delete(context, image_id) Jul 16 21:19:16.422878 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 726, in delete Jul 16 21:19:16.422878 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.ImageNotFound(image_id=image_id) Jul 16 21:19:16.422878 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.ImageNotFound: Image 211719d4-d58e-474c-ade9-8ffb307248f3 could not be found. Jul 16 21:19:16.422878 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:19:16.422878 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:19:16.422878 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:19:16.422878 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:19:16.426961 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:19:16.426961 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:19:16.426961 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:19:16.426961 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:19:16.426961 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Jul 16 21:19:16.426961 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Jul 16 21:19:16.426961 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:19:16.426961 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:19:16.426961 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/images.py", line 107, in delete Jul 16 21:19:16.426961 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=explanation) Jul 16 21:19:16.426961 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Image not found. Jul 16 21:19:16.426961 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:19:16.426961 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:19:16.426961 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:19:16.426961 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:19:16.426961 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:19:16.426961 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:19:16.426961 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:19:16.426961 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:19:16.426961 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Image not found. Jul 16 21:19:16.426961 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:19:16.426961 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-55de1ee2-6972-4833-87cc-2701a0b4c086 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Returning 404 to user: Image not found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:19:16.426961 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-55de1ee2-6972-4833-87cc-2701a0b4c086 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] 10.222.0.16 "DELETE /compute/v2.1/images/211719d4-d58e-474c-ade9-8ffb307248f3" status: 404 len: 62 microversion: 2.1 time: 0.057449 Jul 16 21:19:16.426961 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1289/2615] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:19:16 2022] DELETE /compute/v2.1/images/211719d4-d58e-474c-ade9-8ffb307248f3 => generated 62 bytes in 60 msecs (HTTP/1.1 404) 9 headers in 378 bytes (1 switches on core 0) Jul 16 21:19:16.430483 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-c0001e74-154e-4bed-87fb-adaa51d8204b tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Action: 'update', calling method: .version_select at 0x7fdd9cfac0d0>, body: {"security_group": {"name": "tempest-sg-hth-1947925254", "description": "tempest-description-hth-570985559"}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:19:16.516638 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-d8542782-52ab-423a-afdc-32dd6086b062 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f5bc1f2209f4eb186d80e385b67c774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '65d5bcfa02af4e8284ba1342197e2860', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:19:16.526918 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-d8542782-52ab-423a-afdc-32dd6086b062 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] 10.222.0.16 "GET /compute/v2.1/servers/e7598b4a-f70c-4186-88d1-6fd0057f8656" status: 200 len: 1472 microversion: 2.1 time: 0.437244 Jul 16 21:19:16.527642 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1327/2616] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:19:16 2022] GET /compute/v2.1/servers/e7598b4a-f70c-4186-88d1-6fd0057f8656 => generated 1472 bytes in 439 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:19:16.534108 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-2fbce82b-9722-4b6d-880d-f8d9979a62d0 tempest-SecurityGroupRulesNegativeTestJSON-1399499068 tempest-SecurityGroupRulesNegativeTestJSON-1399499068-project] Action: 'create', calling method: .version_select at 0x7fdd9d4e00d0>, body: {"security_group_rule": {"parent_group_id": "tempest-non_int_id-1250132104", "ip_protocol": "tcp", "from_port": 22, "to_port": 22}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:19:16.536968 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-2fbce82b-9722-4b6d-880d-f8d9979a62d0 tempest-SecurityGroupRulesNegativeTestJSON-1399499068 tempest-SecurityGroupRulesNegativeTestJSON-1399499068-project] HTTP exception thrown: Security group id should be uuid Jul 16 21:19:16.542716 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-2fbce82b-9722-4b6d-880d-f8d9979a62d0 tempest-SecurityGroupRulesNegativeTestJSON-1399499068 tempest-SecurityGroupRulesNegativeTestJSON-1399499068-project] Request method failure captured: Jul 16 21:19:16.542716 ncdv-master-1 devstack@n-api.service[138680]: request: POST /compute/v2.1/os-security-group-rules HTTP/1.1 Jul 16 21:19:16.542716 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:19:16.542716 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:19:16.542716 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:19:16.542716 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 131 Jul 16 21:19:16.542716 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:19:16.542716 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:19:16.542716 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:19:16.542716 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:19:16.542716 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0ytNkmtXKapUZpFKPGUA8hf3ATbduJ0XtzorjW7amdLo0WS0Qbkqbi3B3wBROLLwxfXJZxCXXmOW4by85qgjbrIW-gGZ2c_YrmiVSX-WID5VqAWhi2CmsmYLeeu_6bw9QJDULh6nweMasTNNM3cnBfI9QUQLgKaJIeA4QYZkK1W0reQ Jul 16 21:19:16.542716 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:19:16.542716 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:19:16.542716 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:19:16.542716 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:19:16.542716 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:19:16.542716 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:19:16.542716 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: 89b4d12252b94d54b50cc15238af6c2f Jul 16 21:19:16.542716 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-SecurityGroupRulesNegativeTestJSON-1399499068 Jul 16 21:19:16.542716 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:19:16.542716 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:19:16.542716 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/89b4d12252b94d54b50cc15238af6c2f"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/89b4d12252b94d54b50cc15238af6c2f"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_89b4d12252b94d54b50cc15238af6c2f"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/89b4d12252b94d54b50cc15238af6c2f"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:19:16.542716 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-SecurityGroupRulesNegativeTestJSON-1399499068 Jul 16 21:19:16.542716 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: 89b4d12252b94d54b50cc15238af6c2f Jul 16 21:19:16.542716 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-SecurityGroupRulesNegativeTestJSON-1399499068 Jul 16 21:19:16.542716 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-SecurityGroupRulesNegativeTestJSON-1399499068-project Jul 16 21:19:16.542716 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:19:16.542716 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:19:16.542716 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: 7ea30eb37fbf469194845261a19e8d4f Jul 16 21:19:16.542716 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-SecurityGroupRulesNegativeTestJSON-1399499068-project Jul 16 21:19:16.542716 ncdv-master-1 devstack@n-api.service[138680]: Jul 16 21:19:16.542716 ncdv-master-1 devstack@n-api.service[138680]: {"security_group_rule": {"parent_group_id": "tempest-non_int_id-1250132104", "ip_protocol": "tcp", "from_port": 22, "to_port": 22}} Jul 16 21:19:16.542716 ncdv-master-1 devstack@n-api.service[138680]: method: Jul 16 21:19:16.542716 ncdv-master-1 devstack@n-api.service[138680]: exception: Security group id should be uuid Jul 16 21:19:16.542716 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'body': {'security_group_rule': {'parent_group_id': 'tempest-non_int_id-1250132104', 'ip_protocol': 'tcp', 'from_port': 22, 'to_port': 22}}} Jul 16 21:19:16.542716 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:19:16.542716 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:19:16.542716 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/security_groups.py", line 285, in create Jul 16 21:19:16.542716 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi parent_group_id = security_group_api.validate_id( Jul 16 21:19:16.542716 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/security_group_api.py", line 51, in validate_id Jul 16 21:19:16.542716 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.Invalid(msg) Jul 16 21:19:16.542716 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.Invalid: Security group id should be uuid Jul 16 21:19:16.542716 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:19:16.542716 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:19:16.542716 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:19:16.542716 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:19:16.542716 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:19:16.542716 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:19:16.542716 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:19:16.542716 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:19:16.542716 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Jul 16 21:19:16.542716 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Jul 16 21:19:16.542716 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:19:16.542716 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:19:16.542716 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/security_groups.py", line 301, in create Jul 16 21:19:16.542716 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=exp.format_message()) Jul 16 21:19:16.542716 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Security group id should be uuid Jul 16 21:19:16.542716 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:19:16.542716 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:19:16.542716 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:19:16.542716 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:19:16.542716 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:19:16.542716 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:19:16.542716 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:19:16.542716 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:19:16.546488 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Security group id should be uuid Jul 16 21:19:16.546488 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:19:16.546488 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-2fbce82b-9722-4b6d-880d-f8d9979a62d0 tempest-SecurityGroupRulesNegativeTestJSON-1399499068 tempest-SecurityGroupRulesNegativeTestJSON-1399499068-project] Returning 400 to user: Security group id should be uuid {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:19:16.546488 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-2fbce82b-9722-4b6d-880d-f8d9979a62d0 tempest-SecurityGroupRulesNegativeTestJSON-1399499068 tempest-SecurityGroupRulesNegativeTestJSON-1399499068-project] 10.222.0.16 "POST /compute/v2.1/os-security-group-rules" status: 400 len: 76 microversion: 2.1 time: 0.015474 Jul 16 21:19:16.546488 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1328/2617] 10.222.0.16 () {60 vars in 1194 bytes} [Sat Jul 16 21:19:16 2022] POST /compute/v2.1/os-security-group-rules => generated 76 bytes in 16 msecs (HTTP/1.1 400) 9 headers in 380 bytes (1 switches on core 0) Jul 16 21:19:16.552120 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-ab9ea3cd-c2f7-4f29-9dc0-3e69e9a92959 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:19:16.553103 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-ab9ea3cd-c2f7-4f29-9dc0-3e69e9a92959 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] [instance: f03a2a83-e007-444a-b46b-70eb46bb242b] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:19:16.564154 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ab9ea3cd-c2f7-4f29-9dc0-3e69e9a92959 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:16.565328 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ab9ea3cd-c2f7-4f29-9dc0-3e69e9a92959 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:16.566427 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ab9ea3cd-c2f7-4f29-9dc0-3e69e9a92959 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:16.566942 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ab9ea3cd-c2f7-4f29-9dc0-3e69e9a92959 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:16.596198 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-ab9ea3cd-c2f7-4f29-9dc0-3e69e9a92959 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] HTTP exception thrown: Instance f03a2a83-e007-444a-b46b-70eb46bb242b could not be found. Jul 16 21:19:16.598144 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-ab9ea3cd-c2f7-4f29-9dc0-3e69e9a92959 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Request method failure captured: Jul 16 21:19:16.598144 ncdv-master-1 devstack@n-api.service[138680]: request: DELETE /compute/v2.1/servers/f03a2a83-e007-444a-b46b-70eb46bb242b HTTP/1.1 Jul 16 21:19:16.598144 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:19:16.598144 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:19:16.598144 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:19:16.598144 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 0 Jul 16 21:19:16.598144 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:19:16.598144 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:19:16.598144 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:19:16.598144 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:19:16.598144 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0yptIZULOjVMHbouVcndclXAnMKC6qm1VtMGoKOnZzI-fUPx3L5lhuz06xvHQzr3ooL8P53cH83yaSSJWcV0fJCtKGdbdY71S9WCM2F11ZTapGri39-eD6M8L5ZmeIRv6VTRaJtzqhadxTTGkGwVf5FzILXoamLnE3Jdb08lelPg3B4 Jul 16 21:19:16.598144 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:19:16.598144 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:19:16.598144 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:19:16.598144 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:19:16.598144 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:19:16.598144 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:19:16.598144 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: 24c64db8082648c8837c16352252f96d Jul 16 21:19:16.598144 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-ImagesTestJSON-1532827163 Jul 16 21:19:16.598144 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:19:16.598144 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:19:16.598144 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/24c64db8082648c8837c16352252f96d"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/24c64db8082648c8837c16352252f96d"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_24c64db8082648c8837c16352252f96d"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/24c64db8082648c8837c16352252f96d"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:19:16.598144 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-ImagesTestJSON-1532827163 Jul 16 21:19:16.598144 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: 24c64db8082648c8837c16352252f96d Jul 16 21:19:16.598144 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-ImagesTestJSON-1532827163 Jul 16 21:19:16.598144 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-ImagesTestJSON-1532827163-project Jul 16 21:19:16.598144 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:19:16.598144 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:19:16.598144 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: daf3c9e6634e4ae1bfb492eae1f44c93 Jul 16 21:19:16.598144 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-ImagesTestJSON-1532827163-project Jul 16 21:19:16.598144 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 21:19:16.598144 ncdv-master-1 devstack@n-api.service[138680]: exception: Instance f03a2a83-e007-444a-b46b-70eb46bb242b could not be found. Jul 16 21:19:16.598144 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': 'f03a2a83-e007-444a-b46b-70eb46bb242b'} Jul 16 21:19:16.598144 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:19:16.598144 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:19:16.598144 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:19:16.598144 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:19:16.598144 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:19:16.598144 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:19:16.598144 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:19:16.598144 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:19:16.598144 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:19:16.598144 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:19:16.598144 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance f03a2a83-e007-444a-b46b-70eb46bb242b could not be found. Jul 16 21:19:16.598144 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:19:16.598144 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:19:16.598144 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:19:16.598144 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:19:16.598144 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:19:16.598144 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:19:16.598144 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:19:16.598144 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:19:16.598144 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:19:16.598144 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:19:16.598144 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Jul 16 21:19:16.598144 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Jul 16 21:19:16.598144 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Jul 16 21:19:16.598144 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Jul 16 21:19:16.598144 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:19:16.598144 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:19:16.598144 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:19:16.598144 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:19:16.598144 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance f03a2a83-e007-444a-b46b-70eb46bb242b could not be found. Jul 16 21:19:16.604471 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:19:16.604471 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:19:16.604471 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:19:16.604471 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:19:16.604471 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:19:16.604471 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:19:16.604471 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:19:16.604471 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:19:16.604471 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance f03a2a83-e007-444a-b46b-70eb46bb242b could not be found. Jul 16 21:19:16.604471 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:19:16.604471 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-ab9ea3cd-c2f7-4f29-9dc0-3e69e9a92959 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Returning 404 to user: Instance f03a2a83-e007-444a-b46b-70eb46bb242b could not be found. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:19:16.604471 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-ab9ea3cd-c2f7-4f29-9dc0-3e69e9a92959 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] 10.222.0.16 "DELETE /compute/v2.1/servers/f03a2a83-e007-444a-b46b-70eb46bb242b" status: 404 len: 111 microversion: 2.1 time: 0.054114 Jul 16 21:19:16.604471 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1329/2618] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:19:16 2022] DELETE /compute/v2.1/servers/f03a2a83-e007-444a-b46b-70eb46bb242b => generated 111 bytes in 58 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:19:16.611049 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-3e364811-6abc-45fe-b242-574a7c300cd2 tempest-SecurityGroupRulesNegativeTestJSON-1399499068 tempest-SecurityGroupRulesNegativeTestJSON-1399499068-project] Action: 'create', calling method: .version_select at 0x7fdd9d3ea3a0>, body: {"security_group": {"name": "tempest-SecurityGroupRulesNegativeTestJSON-securitygroup-1847845763", "description": "tempest-description-200925555"}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:19:16.835520 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-c0001e74-154e-4bed-87fb-adaa51d8204b tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] 10.222.0.16 "PUT /compute/v2.1/os-security-groups/26173f30-c76f-4699-87d6-a73ad84078ba" status: 200 len: 217 microversion: 2.1 time: 0.409508 Jul 16 21:19:16.836605 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1290/2619] 10.222.0.16 () {60 vars in 1289 bytes} [Sat Jul 16 21:19:16 2022] PUT /compute/v2.1/os-security-groups/26173f30-c76f-4699-87d6-a73ad84078ba => generated 217 bytes in 411 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:19:16.846428 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-5469d951-c03f-4626-99e6-776e97f3e328 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:19:16.847389 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-5469d951-c03f-4626-99e6-776e97f3e328 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] [instance: f03a2a83-e007-444a-b46b-70eb46bb242b] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:19:16.855751 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5469d951-c03f-4626-99e6-776e97f3e328 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:16.855751 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5469d951-c03f-4626-99e6-776e97f3e328 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:16.856208 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5469d951-c03f-4626-99e6-776e97f3e328 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:16.856655 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5469d951-c03f-4626-99e6-776e97f3e328 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:16.871622 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-5469d951-c03f-4626-99e6-776e97f3e328 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] HTTP exception thrown: Instance f03a2a83-e007-444a-b46b-70eb46bb242b could not be found. Jul 16 21:19:16.873309 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-5469d951-c03f-4626-99e6-776e97f3e328 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Request method failure captured: Jul 16 21:19:16.873309 ncdv-master-1 devstack@n-api.service[138681]: request: GET /compute/v2.1/servers/f03a2a83-e007-444a-b46b-70eb46bb242b HTTP/1.1 Jul 16 21:19:16.873309 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:19:16.873309 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:19:16.873309 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:19:16.873309 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 21:19:16.873309 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:19:16.873309 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:19:16.873309 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:19:16.873309 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:19:16.873309 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0yptIZULOjVMHbouVcndclXAnMKC6qm1VtMGoKOnZzI-fUPx3L5lhuz06xvHQzr3ooL8P53cH83yaSSJWcV0fJCtKGdbdY71S9WCM2F11ZTapGri39-eD6M8L5ZmeIRv6VTRaJtzqhadxTTGkGwVf5FzILXoamLnE3Jdb08lelPg3B4 Jul 16 21:19:16.873309 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:19:16.873309 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:19:16.873309 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:19:16.873309 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:19:16.873309 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:19:16.873309 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:19:16.873309 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: 24c64db8082648c8837c16352252f96d Jul 16 21:19:16.873309 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-ImagesTestJSON-1532827163 Jul 16 21:19:16.873309 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:19:16.873309 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:19:16.873309 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/24c64db8082648c8837c16352252f96d"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/24c64db8082648c8837c16352252f96d"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_24c64db8082648c8837c16352252f96d"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/24c64db8082648c8837c16352252f96d"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:19:16.873309 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-ImagesTestJSON-1532827163 Jul 16 21:19:16.873309 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: 24c64db8082648c8837c16352252f96d Jul 16 21:19:16.873309 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-ImagesTestJSON-1532827163 Jul 16 21:19:16.873309 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-ImagesTestJSON-1532827163-project Jul 16 21:19:16.873309 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:19:16.873309 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:19:16.873309 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: daf3c9e6634e4ae1bfb492eae1f44c93 Jul 16 21:19:16.873309 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-ImagesTestJSON-1532827163-project Jul 16 21:19:16.873309 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:19:16.873309 ncdv-master-1 devstack@n-api.service[138681]: exception: Instance f03a2a83-e007-444a-b46b-70eb46bb242b could not be found. Jul 16 21:19:16.873309 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': 'f03a2a83-e007-444a-b46b-70eb46bb242b'} Jul 16 21:19:16.873309 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:19:16.873309 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:19:16.873309 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:19:16.873309 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:19:16.873309 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:19:16.873309 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:19:16.873309 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:19:16.873309 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:19:16.873309 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:19:16.873309 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:19:16.873309 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance f03a2a83-e007-444a-b46b-70eb46bb242b could not be found. Jul 16 21:19:16.873309 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:19:16.873309 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:19:16.873309 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:19:16.873309 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:19:16.873309 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:19:16.873309 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:19:16.873309 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:19:16.873309 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:19:16.873309 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:19:16.873309 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:19:16.873309 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:19:16.873309 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:19:16.873309 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:19:16.873309 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:19:16.873309 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:19:16.873309 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:19:16.873309 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance f03a2a83-e007-444a-b46b-70eb46bb242b could not be found. Jul 16 21:19:16.873309 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:19:16.873309 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:19:16.873309 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:19:16.873309 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:19:16.878082 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:19:16.878082 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:19:16.878082 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:19:16.878082 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:19:16.878082 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance f03a2a83-e007-444a-b46b-70eb46bb242b could not be found. Jul 16 21:19:16.878082 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:19:16.878082 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-5469d951-c03f-4626-99e6-776e97f3e328 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Returning 404 to user: Instance f03a2a83-e007-444a-b46b-70eb46bb242b could not be found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:19:16.878082 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-5469d951-c03f-4626-99e6-776e97f3e328 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] 10.222.0.16 "GET /compute/v2.1/servers/f03a2a83-e007-444a-b46b-70eb46bb242b" status: 404 len: 111 microversion: 2.1 time: 0.036927 Jul 16 21:19:16.878082 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1291/2620] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:19:16 2022] GET /compute/v2.1/servers/f03a2a83-e007-444a-b46b-70eb46bb242b => generated 111 bytes in 38 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:19:16.888204 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-d35a380e-a05c-44c4-9dd5-d1bf98ee1f3e req-651a3b6c-cf6c-4606-a6c9-3a2066baf28e service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "c16e5f09-f813-414e-b233-c08348078624", "tag": "882bd182-7d81-4db5-8bdc-c8b2c9c996a4"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:19:16.889006 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-d35a380e-a05c-44c4-9dd5-d1bf98ee1f3e req-651a3b6c-cf6c-4606-a6c9-3a2066baf28e service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:16.889470 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-d35a380e-a05c-44c4-9dd5-d1bf98ee1f3e req-651a3b6c-cf6c-4606-a6c9-3a2066baf28e service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:16.903528 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-d35a380e-a05c-44c4-9dd5-d1bf98ee1f3e req-651a3b6c-cf6c-4606-a6c9-3a2066baf28e service nova] Creating event network-changed:882bd182-7d81-4db5-8bdc-c8b2c9c996a4 for instance c16e5f09-f813-414e-b233-c08348078624 on nch1-master-1 Jul 16 21:19:16.908535 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-d35a380e-a05c-44c4-9dd5-d1bf98ee1f3e req-651a3b6c-cf6c-4606-a6c9-3a2066baf28e service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.032184 Jul 16 21:19:16.909414 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1292/2621] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:19:16 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:19:16.914478 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-cf10f703-b038-4793-94fb-127a11b3b82f tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Calling method '.version_select at 0x7fdd9d137d30>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:19:16.962748 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-cf10f703-b038-4793-94fb-127a11b3b82f tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] 10.222.0.16 "GET /compute/v2.1/os-security-groups/26173f30-c76f-4699-87d6-a73ad84078ba" status: 200 len: 217 microversion: 2.1 time: 0.052285 Jul 16 21:19:16.963390 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1293/2622] 10.222.0.16 () {58 vars in 1268 bytes} [Sat Jul 16 21:19:16 2022] GET /compute/v2.1/os-security-groups/26173f30-c76f-4699-87d6-a73ad84078ba => generated 217 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:19:16.967880 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-399f5fbe-4157-41e6-8468-c8330339619d tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Calling method '.version_select at 0x7fdd9cf97310>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:19:16.989201 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-399f5fbe-4157-41e6-8468-c8330339619d tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] HTTP exception thrown: Image not found. Jul 16 21:19:16.991141 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-399f5fbe-4157-41e6-8468-c8330339619d tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Request method failure captured: Jul 16 21:19:16.991141 ncdv-master-1 devstack@n-api.service[138681]: request: DELETE /compute/v2.1/images/5af5e2eb-8145-46b4-abad-4d2a1e3630fc HTTP/1.1 Jul 16 21:19:16.991141 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:19:16.991141 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:19:16.991141 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:19:16.991141 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 21:19:16.991141 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:19:16.991141 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:19:16.991141 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:19:16.991141 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:19:16.991141 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0yptIZULOjVMHbouVcndclXAnMKC6qm1VtMGoKOnZzI-fUPx3L5lhuz06xvHQzr3ooL8P53cH83yaSSJWcV0fJCtKGdbdY71S9WCM2F11ZTapGri39-eD6M8L5ZmeIRv6VTRaJtzqhadxTTGkGwVf5FzILXoamLnE3Jdb08lelPg3B4 Jul 16 21:19:16.991141 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:19:16.991141 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:19:16.991141 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:19:16.991141 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:19:16.991141 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:19:16.991141 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:19:16.991141 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: 24c64db8082648c8837c16352252f96d Jul 16 21:19:16.991141 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-ImagesTestJSON-1532827163 Jul 16 21:19:16.991141 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:19:16.991141 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:19:16.991141 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/24c64db8082648c8837c16352252f96d"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/24c64db8082648c8837c16352252f96d"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_24c64db8082648c8837c16352252f96d"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/24c64db8082648c8837c16352252f96d"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:19:16.991141 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-ImagesTestJSON-1532827163 Jul 16 21:19:16.991141 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: 24c64db8082648c8837c16352252f96d Jul 16 21:19:16.991141 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-ImagesTestJSON-1532827163 Jul 16 21:19:16.991141 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-ImagesTestJSON-1532827163-project Jul 16 21:19:16.991141 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:19:16.991141 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:19:16.991141 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: daf3c9e6634e4ae1bfb492eae1f44c93 Jul 16 21:19:16.991141 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-ImagesTestJSON-1532827163-project Jul 16 21:19:16.991141 ncdv-master-1 devstack@n-api.service[138681]: method: Jul 16 21:19:16.991141 ncdv-master-1 devstack@n-api.service[138681]: exception: Image not found. Jul 16 21:19:16.991141 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': '5af5e2eb-8145-46b4-abad-4d2a1e3630fc'} Jul 16 21:19:16.991141 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:19:16.991141 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:19:16.991141 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 724, in delete Jul 16 21:19:16.991141 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi self._client.call(context, 2, 'delete', args=(image_id,)) Jul 16 21:19:16.991141 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 191, in call Jul 16 21:19:16.991141 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi result = getattr(controller, method)(*args, **kwargs) Jul 16 21:19:16.991141 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/utils.py", line 670, in inner Jul 16 21:19:16.991141 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return RequestIdProxy(wrapped(*args, **kwargs)) Jul 16 21:19:16.991141 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/v2/images.py", line 386, in delete Jul 16 21:19:16.991141 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi resp, body = self.http_client.delete(url) Jul 16 21:19:16.991141 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/adapter.py", line 410, in delete Jul 16 21:19:16.991141 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return self.request(url, 'DELETE', **kwargs) Jul 16 21:19:16.991141 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 380, in request Jul 16 21:19:16.991141 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return self._handle_response(resp) Jul 16 21:19:16.991141 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 120, in _handle_response Jul 16 21:19:16.991141 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.from_response(resp, resp.content) Jul 16 21:19:16.991141 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi glanceclient.exc.HTTPNotFound: HTTP 404 Not Found: Failed to find image 5af5e2eb-8145-46b4-abad-4d2a1e3630fc to delete Jul 16 21:19:16.991141 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:19:16.991141 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:19:16.991141 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:19:16.991141 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:19:16.991141 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/images.py", line 104, in delete Jul 16 21:19:16.991141 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi self._image_api.delete(context, id) Jul 16 21:19:16.991141 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 1255, in delete Jul 16 21:19:16.991141 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return session.delete(context, image_id) Jul 16 21:19:16.991141 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 726, in delete Jul 16 21:19:16.991141 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.ImageNotFound(image_id=image_id) Jul 16 21:19:16.991141 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.ImageNotFound: Image 5af5e2eb-8145-46b4-abad-4d2a1e3630fc could not be found. Jul 16 21:19:16.991141 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:19:16.991141 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:19:16.991141 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:19:16.991141 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:19:16.996680 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:19:16.996680 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:19:16.996680 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:19:16.996680 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:19:16.996680 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Jul 16 21:19:16.996680 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Jul 16 21:19:16.996680 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:19:16.996680 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:19:16.996680 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/images.py", line 107, in delete Jul 16 21:19:16.996680 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=explanation) Jul 16 21:19:16.996680 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Image not found. Jul 16 21:19:16.996680 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:19:16.996680 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:19:16.996680 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:19:16.996680 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:19:16.996680 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:19:16.996680 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:19:16.996680 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:19:16.996680 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:19:16.996680 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Image not found. Jul 16 21:19:16.996680 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:19:16.996680 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-399f5fbe-4157-41e6-8468-c8330339619d tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Returning 404 to user: Image not found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:19:16.996680 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-399f5fbe-4157-41e6-8468-c8330339619d tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] 10.222.0.16 "DELETE /compute/v2.1/images/5af5e2eb-8145-46b4-abad-4d2a1e3630fc" status: 404 len: 62 microversion: 2.1 time: 0.028858 Jul 16 21:19:16.996680 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1294/2623] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:19:16 2022] DELETE /compute/v2.1/images/5af5e2eb-8145-46b4-abad-4d2a1e3630fc => generated 62 bytes in 30 msecs (HTTP/1.1 404) 9 headers in 378 bytes (1 switches on core 0) Jul 16 21:19:16.998597 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-0406104f-e94f-4623-a6d7-cf0595ac798c tempest-ServersAaction247Test-725202529 tempest-ServersAaction247Test-725202529-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:19:16.999192 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-0406104f-e94f-4623-a6d7-cf0595ac798c tempest-ServersAaction247Test-725202529 tempest-ServersAaction247Test-725202529-project] [instance: a51adaf1-9c26-47d0-92c2-b6bb065a5ee8] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:19:17.008785 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0406104f-e94f-4623-a6d7-cf0595ac798c tempest-ServersAaction247Test-725202529 tempest-ServersAaction247Test-725202529-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:17.008785 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0406104f-e94f-4623-a6d7-cf0595ac798c tempest-ServersAaction247Test-725202529 tempest-ServersAaction247Test-725202529-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:17.008785 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0406104f-e94f-4623-a6d7-cf0595ac798c tempest-ServersAaction247Test-725202529 tempest-ServersAaction247Test-725202529-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:17.009302 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0406104f-e94f-4623-a6d7-cf0595ac798c tempest-ServersAaction247Test-725202529 tempest-ServersAaction247Test-725202529-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:17.075836 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-0406104f-e94f-4623-a6d7-cf0595ac798c tempest-ServersAaction247Test-725202529 tempest-ServersAaction247Test-725202529-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a8d48b52db1545a4a51027a5c2c7fcce', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f2984288b4cd4610b2f32e895f3e5d72', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:19:17.087283 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-0406104f-e94f-4623-a6d7-cf0595ac798c tempest-ServersAaction247Test-725202529 tempest-ServersAaction247Test-725202529-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'a8d48b52db1545a4a51027a5c2c7fcce', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f2984288b4cd4610b2f32e895f3e5d72', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:19:17.087283 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-0406104f-e94f-4623-a6d7-cf0595ac798c tempest-ServersAaction247Test-725202529 tempest-ServersAaction247Test-725202529-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'a8d48b52db1545a4a51027a5c2c7fcce', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f2984288b4cd4610b2f32e895f3e5d72', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:19:17.088160 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-0406104f-e94f-4623-a6d7-cf0595ac798c tempest-ServersAaction247Test-725202529 tempest-ServersAaction247Test-725202529-project] 10.222.0.16 "GET /compute/v2.1/servers/a51adaf1-9c26-47d0-92c2-b6bb065a5ee8" status: 200 len: 1339 microversion: 2.47 time: 0.093352 Jul 16 21:19:17.089052 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1295/2624] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:19:16 2022] GET /compute/v2.1/servers/a51adaf1-9c26-47d0-92c2-b6bb065a5ee8 => generated 1339 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:19:17.094139 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-485b593b-69fe-4873-9358-ce75a5ab3cd6 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Calling method '.version_select at 0x7fdd9d090430>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:19:17.143845 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-3e364811-6abc-45fe-b242-574a7c300cd2 tempest-SecurityGroupRulesNegativeTestJSON-1399499068 tempest-SecurityGroupRulesNegativeTestJSON-1399499068-project] 10.222.0.16 "POST /compute/v2.1/os-security-groups" status: 200 len: 255 microversion: 2.1 time: 0.538946 Jul 16 21:19:17.144485 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1330/2625] 10.222.0.16 () {60 vars in 1179 bytes} [Sat Jul 16 21:19:16 2022] POST /compute/v2.1/os-security-groups => generated 255 bytes in 540 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:19:17.150270 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-16407cf2-0f84-4b49-a6ee-f41bf9ddb163 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:19:17.151174 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-16407cf2-0f84-4b49-a6ee-f41bf9ddb163 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] [instance: 93e327c3-c63d-439a-9b25-1a805f785949] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:19:17.158306 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-16407cf2-0f84-4b49-a6ee-f41bf9ddb163 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:17.158771 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-16407cf2-0f84-4b49-a6ee-f41bf9ddb163 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:17.159629 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-16407cf2-0f84-4b49-a6ee-f41bf9ddb163 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:17.160111 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-16407cf2-0f84-4b49-a6ee-f41bf9ddb163 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:17.176452 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-16407cf2-0f84-4b49-a6ee-f41bf9ddb163 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] HTTP exception thrown: Instance 93e327c3-c63d-439a-9b25-1a805f785949 could not be found. Jul 16 21:19:17.178710 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-16407cf2-0f84-4b49-a6ee-f41bf9ddb163 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Request method failure captured: Jul 16 21:19:17.178710 ncdv-master-1 devstack@n-api.service[138680]: request: DELETE /compute/v2.1/servers/93e327c3-c63d-439a-9b25-1a805f785949 HTTP/1.1 Jul 16 21:19:17.178710 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:19:17.178710 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:19:17.178710 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:19:17.178710 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 0 Jul 16 21:19:17.178710 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:19:17.178710 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:19:17.178710 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:19:17.178710 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:19:17.178710 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0yptIZULOjVMHbouVcndclXAnMKC6qm1VtMGoKOnZzI-fUPx3L5lhuz06xvHQzr3ooL8P53cH83yaSSJWcV0fJCtKGdbdY71S9WCM2F11ZTapGri39-eD6M8L5ZmeIRv6VTRaJtzqhadxTTGkGwVf5FzILXoamLnE3Jdb08lelPg3B4 Jul 16 21:19:17.178710 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:19:17.178710 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:19:17.178710 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:19:17.178710 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:19:17.178710 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:19:17.178710 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:19:17.178710 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: 24c64db8082648c8837c16352252f96d Jul 16 21:19:17.178710 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-ImagesTestJSON-1532827163 Jul 16 21:19:17.178710 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:19:17.178710 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:19:17.178710 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/24c64db8082648c8837c16352252f96d"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/24c64db8082648c8837c16352252f96d"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_24c64db8082648c8837c16352252f96d"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/24c64db8082648c8837c16352252f96d"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:19:17.178710 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-ImagesTestJSON-1532827163 Jul 16 21:19:17.178710 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: 24c64db8082648c8837c16352252f96d Jul 16 21:19:17.178710 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-ImagesTestJSON-1532827163 Jul 16 21:19:17.178710 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-ImagesTestJSON-1532827163-project Jul 16 21:19:17.178710 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:19:17.178710 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:19:17.178710 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: daf3c9e6634e4ae1bfb492eae1f44c93 Jul 16 21:19:17.178710 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-ImagesTestJSON-1532827163-project Jul 16 21:19:17.178710 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 21:19:17.178710 ncdv-master-1 devstack@n-api.service[138680]: exception: Instance 93e327c3-c63d-439a-9b25-1a805f785949 could not be found. Jul 16 21:19:17.178710 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': '93e327c3-c63d-439a-9b25-1a805f785949'} Jul 16 21:19:17.178710 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:19:17.178710 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:19:17.178710 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:19:17.178710 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:19:17.178710 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:19:17.178710 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:19:17.178710 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:19:17.178710 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:19:17.178710 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:19:17.178710 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:19:17.178710 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 93e327c3-c63d-439a-9b25-1a805f785949 could not be found. Jul 16 21:19:17.178710 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:19:17.178710 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:19:17.178710 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:19:17.178710 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:19:17.178710 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:19:17.178710 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:19:17.178710 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:19:17.178710 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:19:17.178710 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:19:17.178710 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:19:17.178710 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Jul 16 21:19:17.178710 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Jul 16 21:19:17.178710 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Jul 16 21:19:17.178710 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Jul 16 21:19:17.178710 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:19:17.178710 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:19:17.178710 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:19:17.178710 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:19:17.178710 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 93e327c3-c63d-439a-9b25-1a805f785949 could not be found. Jul 16 21:19:17.183411 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:19:17.183411 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:19:17.183411 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:19:17.183411 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:19:17.183411 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:19:17.183411 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:19:17.183411 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:19:17.183411 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:19:17.183411 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 93e327c3-c63d-439a-9b25-1a805f785949 could not be found. Jul 16 21:19:17.183411 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:19:17.185429 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-16407cf2-0f84-4b49-a6ee-f41bf9ddb163 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Returning 404 to user: Instance 93e327c3-c63d-439a-9b25-1a805f785949 could not be found. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:19:17.186482 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-16407cf2-0f84-4b49-a6ee-f41bf9ddb163 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] 10.222.0.16 "DELETE /compute/v2.1/servers/93e327c3-c63d-439a-9b25-1a805f785949" status: 404 len: 111 microversion: 2.1 time: 0.040783 Jul 16 21:19:17.187464 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1331/2626] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:19:17 2022] DELETE /compute/v2.1/servers/93e327c3-c63d-439a-9b25-1a805f785949 => generated 111 bytes in 42 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:19:17.192345 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-c34ab2e3-b99b-4712-956e-ebb9fe733dac tempest-VolumesSnapshotsTestJSON-851680556 tempest-VolumesSnapshotsTestJSON-851680556-project] Calling method '.version_select at 0x7fdda2b7f940>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:19:17.221520 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-c34ab2e3-b99b-4712-956e-ebb9fe733dac tempest-VolumesSnapshotsTestJSON-851680556 tempest-VolumesSnapshotsTestJSON-851680556-project] HTTP exception thrown: Volume 4450505c-3ac5-42c2-b94c-b8f945007d50 could not be found. Jul 16 21:19:17.223617 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-c34ab2e3-b99b-4712-956e-ebb9fe733dac tempest-VolumesSnapshotsTestJSON-851680556 tempest-VolumesSnapshotsTestJSON-851680556-project] Request method failure captured: Jul 16 21:19:17.223617 ncdv-master-1 devstack@n-api.service[138680]: request: GET /compute/v2.1/os-volumes/4450505c-3ac5-42c2-b94c-b8f945007d50 HTTP/1.1 Jul 16 21:19:17.223617 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:19:17.223617 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:19:17.223617 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:19:17.223617 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 0 Jul 16 21:19:17.223617 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:19:17.223617 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:19:17.223617 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:19:17.223617 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:19:17.223617 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0ys_bQ2YkbojsHK1HUJf-q5eS4WAKF6XcJ7jYKIj3a8lR37PmQ0zZ5DyXkNSC8x8kLgg3bQW6slrse7LirtZA91amTfVZ6Kr3gaUumm-yoTjKpsyiJvqY1GYtuGlfRXmsIYTP5FABjs_4cPzEKKvoKI8h2KKC02FgQGNg_a7XC56cnE Jul 16 21:19:17.223617 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:19:17.223617 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:19:17.223617 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:19:17.223617 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:19:17.223617 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:19:17.223617 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:19:17.223617 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: 942725b966874321b6ccd6f628787d06 Jul 16 21:19:17.223617 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-VolumesSnapshotsTestJSON-851680556 Jul 16 21:19:17.223617 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:19:17.223617 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:19:17.223617 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/942725b966874321b6ccd6f628787d06"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/942725b966874321b6ccd6f628787d06"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_942725b966874321b6ccd6f628787d06"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/942725b966874321b6ccd6f628787d06"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:19:17.223617 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-VolumesSnapshotsTestJSON-851680556 Jul 16 21:19:17.223617 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: 942725b966874321b6ccd6f628787d06 Jul 16 21:19:17.223617 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-VolumesSnapshotsTestJSON-851680556 Jul 16 21:19:17.223617 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-VolumesSnapshotsTestJSON-851680556-project Jul 16 21:19:17.223617 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:19:17.223617 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:19:17.223617 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: 6a3dbbb2b39447d884c56dfd0db7e42e Jul 16 21:19:17.223617 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-VolumesSnapshotsTestJSON-851680556-project Jul 16 21:19:17.223617 ncdv-master-1 devstack@n-api.service[138680]: method: Jul 16 21:19:17.223617 ncdv-master-1 devstack@n-api.service[138680]: exception: Volume 4450505c-3ac5-42c2-b94c-b8f945007d50 could not be found. Jul 16 21:19:17.223617 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': '4450505c-3ac5-42c2-b94c-b8f945007d50'} Jul 16 21:19:17.223617 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:19:17.223617 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:19:17.223617 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jul 16 21:19:17.223617 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jul 16 21:19:17.223617 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Jul 16 21:19:17.223617 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi item = cinderclient( Jul 16 21:19:17.223617 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Jul 16 21:19:17.223617 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Jul 16 21:19:17.223617 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Jul 16 21:19:17.223617 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Jul 16 21:19:17.223617 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Jul 16 21:19:17.223617 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Jul 16 21:19:17.223617 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jul 16 21:19:17.223617 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jul 16 21:19:17.223617 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jul 16 21:19:17.223617 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jul 16 21:19:17.223617 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume 4450505c-3ac5-42c2-b94c-b8f945007d50 could not be found. (HTTP 404) (Request-ID: req-3552ac8f-8838-4a98-800a-929ba29333a8) Jul 16 21:19:17.223617 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:19:17.223617 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:19:17.223617 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:19:17.223617 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:19:17.223617 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 120, in show Jul 16 21:19:17.223617 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get(context, id) Jul 16 21:19:17.223617 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Jul 16 21:19:17.223617 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Jul 16 21:19:17.223617 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Jul 16 21:19:17.223617 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Jul 16 21:19:17.223617 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Jul 16 21:19:17.223617 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Jul 16 21:19:17.223617 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jul 16 21:19:17.228838 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jul 16 21:19:17.228838 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Jul 16 21:19:17.228838 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi item = cinderclient( Jul 16 21:19:17.228838 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Jul 16 21:19:17.228838 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Jul 16 21:19:17.228838 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Jul 16 21:19:17.228838 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Jul 16 21:19:17.228838 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Jul 16 21:19:17.228838 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Jul 16 21:19:17.228838 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jul 16 21:19:17.228838 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jul 16 21:19:17.228838 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jul 16 21:19:17.228838 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jul 16 21:19:17.228838 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume 4450505c-3ac5-42c2-b94c-b8f945007d50 could not be found. Jul 16 21:19:17.228838 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:19:17.228838 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:19:17.228838 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:19:17.228838 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:19:17.228838 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:19:17.228838 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:19:17.228838 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:19:17.228838 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:19:17.228838 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Jul 16 21:19:17.228838 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Jul 16 21:19:17.228838 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:19:17.228838 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:19:17.228838 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 122, in show Jul 16 21:19:17.228838 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:19:17.228838 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume 4450505c-3ac5-42c2-b94c-b8f945007d50 could not be found. Jul 16 21:19:17.228838 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:19:17.228838 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:19:17.228838 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:19:17.228838 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:19:17.228838 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:19:17.228838 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:19:17.228838 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:19:17.228838 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:19:17.228838 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume 4450505c-3ac5-42c2-b94c-b8f945007d50 could not be found. Jul 16 21:19:17.228838 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:19:17.228838 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-c34ab2e3-b99b-4712-956e-ebb9fe733dac tempest-VolumesSnapshotsTestJSON-851680556 tempest-VolumesSnapshotsTestJSON-851680556-project] Returning 404 to user: Volume 4450505c-3ac5-42c2-b94c-b8f945007d50 could not be found. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:19:17.228838 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-c34ab2e3-b99b-4712-956e-ebb9fe733dac tempest-VolumesSnapshotsTestJSON-851680556 tempest-VolumesSnapshotsTestJSON-851680556-project] 10.222.0.16 "GET /compute/v2.1/os-volumes/4450505c-3ac5-42c2-b94c-b8f945007d50" status: 404 len: 109 microversion: 2.1 time: 0.036469 Jul 16 21:19:17.228838 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1332/2627] 10.222.0.16 () {58 vars in 1244 bytes} [Sat Jul 16 21:19:17 2022] GET /compute/v2.1/os-volumes/4450505c-3ac5-42c2-b94c-b8f945007d50 => generated 109 bytes in 37 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:19:17.236427 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-fcaf094f-6072-4f77-819e-cddb26551422 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:19:17.237106 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-fcaf094f-6072-4f77-819e-cddb26551422 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] [instance: c16e5f09-f813-414e-b233-c08348078624] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:19:17.244461 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-fcaf094f-6072-4f77-819e-cddb26551422 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:17.244942 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-fcaf094f-6072-4f77-819e-cddb26551422 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:17.245697 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-fcaf094f-6072-4f77-819e-cddb26551422 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:17.246137 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-fcaf094f-6072-4f77-819e-cddb26551422 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:17.388748 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-fcaf094f-6072-4f77-819e-cddb26551422 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eebb224a56814f92b12ebebebc6bc8a3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a092ce9ef40943f6b6ace6cd28481b71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:19:17.396541 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-fcaf094f-6072-4f77-819e-cddb26551422 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] 10.222.0.16 "GET /compute/v2.1/servers/c16e5f09-f813-414e-b233-c08348078624" status: 200 len: 1320 microversion: 2.1 time: 0.170035 Jul 16 21:19:17.401924 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1333/2628] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:19:17 2022] GET /compute/v2.1/servers/c16e5f09-f813-414e-b233-c08348078624 => generated 1320 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:19:17.401924 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-0737fe38-f7a4-4c95-8979-f91af6b803ea tempest-SecurityGroupRulesNegativeTestJSON-1399499068 tempest-SecurityGroupRulesNegativeTestJSON-1399499068-project] Action: 'create', calling method: .version_select at 0x7fdd9cc35040>, body: {"security_group_rule": {"parent_group_id": "65d24cc4-8c2d-4fb8-bf7b-a64bb6225fe1", "ip_protocol": "tempest-999-960908298", "from_port": 22, "to_port": 22}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:19:17.424148 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell0:SELECT=2 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:19:17.428405 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-485b593b-69fe-4873-9358-ce75a5ab3cd6 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] 10.222.0.16 "DELETE /compute/v2.1/os-security-groups/26173f30-c76f-4699-87d6-a73ad84078ba" status: 202 len: 0 microversion: 2.1 time: 0.338135 Jul 16 21:19:17.432555 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1296/2629] 10.222.0.16 () {58 vars in 1271 bytes} [Sat Jul 16 21:19:17 2022] DELETE /compute/v2.1/os-security-groups/26173f30-c76f-4699-87d6-a73ad84078ba => generated 0 bytes in 343 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 21:19:17.437387 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-7ca6a2e3-993f-4ca5-a4ab-5dc2e5d72dc8 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:19:17.437997 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-7ca6a2e3-993f-4ca5-a4ab-5dc2e5d72dc8 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] [instance: 93e327c3-c63d-439a-9b25-1a805f785949] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:19:17.444103 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7ca6a2e3-993f-4ca5-a4ab-5dc2e5d72dc8 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:17.445558 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7ca6a2e3-993f-4ca5-a4ab-5dc2e5d72dc8 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:17.445558 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7ca6a2e3-993f-4ca5-a4ab-5dc2e5d72dc8 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:17.445854 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7ca6a2e3-993f-4ca5-a4ab-5dc2e5d72dc8 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:17.461501 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-7ca6a2e3-993f-4ca5-a4ab-5dc2e5d72dc8 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] HTTP exception thrown: Instance 93e327c3-c63d-439a-9b25-1a805f785949 could not be found. Jul 16 21:19:17.463289 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-7ca6a2e3-993f-4ca5-a4ab-5dc2e5d72dc8 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Request method failure captured: Jul 16 21:19:17.463289 ncdv-master-1 devstack@n-api.service[138681]: request: GET /compute/v2.1/servers/93e327c3-c63d-439a-9b25-1a805f785949 HTTP/1.1 Jul 16 21:19:17.463289 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:19:17.463289 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:19:17.463289 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:19:17.463289 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 21:19:17.463289 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:19:17.463289 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:19:17.463289 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:19:17.463289 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:19:17.463289 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0yptIZULOjVMHbouVcndclXAnMKC6qm1VtMGoKOnZzI-fUPx3L5lhuz06xvHQzr3ooL8P53cH83yaSSJWcV0fJCtKGdbdY71S9WCM2F11ZTapGri39-eD6M8L5ZmeIRv6VTRaJtzqhadxTTGkGwVf5FzILXoamLnE3Jdb08lelPg3B4 Jul 16 21:19:17.463289 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:19:17.463289 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:19:17.463289 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:19:17.463289 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:19:17.463289 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:19:17.463289 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:19:17.463289 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: 24c64db8082648c8837c16352252f96d Jul 16 21:19:17.463289 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-ImagesTestJSON-1532827163 Jul 16 21:19:17.463289 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:19:17.463289 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:19:17.463289 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/24c64db8082648c8837c16352252f96d"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/24c64db8082648c8837c16352252f96d"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_24c64db8082648c8837c16352252f96d"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/24c64db8082648c8837c16352252f96d"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:19:17.463289 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-ImagesTestJSON-1532827163 Jul 16 21:19:17.463289 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: 24c64db8082648c8837c16352252f96d Jul 16 21:19:17.463289 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-ImagesTestJSON-1532827163 Jul 16 21:19:17.463289 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-ImagesTestJSON-1532827163-project Jul 16 21:19:17.463289 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:19:17.463289 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:19:17.463289 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: daf3c9e6634e4ae1bfb492eae1f44c93 Jul 16 21:19:17.463289 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-ImagesTestJSON-1532827163-project Jul 16 21:19:17.463289 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:19:17.463289 ncdv-master-1 devstack@n-api.service[138681]: exception: Instance 93e327c3-c63d-439a-9b25-1a805f785949 could not be found. Jul 16 21:19:17.463289 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': '93e327c3-c63d-439a-9b25-1a805f785949'} Jul 16 21:19:17.463289 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:19:17.463289 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:19:17.463289 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:19:17.463289 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:19:17.463289 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:19:17.463289 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:19:17.463289 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:19:17.463289 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:19:17.463289 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:19:17.463289 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:19:17.463289 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 93e327c3-c63d-439a-9b25-1a805f785949 could not be found. Jul 16 21:19:17.463289 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:19:17.463289 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:19:17.463289 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:19:17.463289 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:19:17.463289 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:19:17.463289 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:19:17.463289 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:19:17.463289 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:19:17.463289 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:19:17.463289 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:19:17.463289 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:19:17.463289 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:19:17.463289 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:19:17.463289 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:19:17.463289 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:19:17.463289 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:19:17.463289 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 93e327c3-c63d-439a-9b25-1a805f785949 could not be found. Jul 16 21:19:17.463289 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:19:17.463289 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:19:17.463289 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:19:17.463289 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:19:17.467030 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:19:17.467030 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:19:17.467030 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:19:17.467030 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:19:17.467030 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 93e327c3-c63d-439a-9b25-1a805f785949 could not be found. Jul 16 21:19:17.467030 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:19:17.467030 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-7ca6a2e3-993f-4ca5-a4ab-5dc2e5d72dc8 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] Returning 404 to user: Instance 93e327c3-c63d-439a-9b25-1a805f785949 could not be found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:19:17.467030 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-7ca6a2e3-993f-4ca5-a4ab-5dc2e5d72dc8 tempest-ImagesTestJSON-1532827163 tempest-ImagesTestJSON-1532827163-project] 10.222.0.16 "GET /compute/v2.1/servers/93e327c3-c63d-439a-9b25-1a805f785949" status: 404 len: 111 microversion: 2.1 time: 0.031045 Jul 16 21:19:17.467030 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1297/2630] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:19:17 2022] GET /compute/v2.1/servers/93e327c3-c63d-439a-9b25-1a805f785949 => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:19:17.467548 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-0737fe38-f7a4-4c95-8979-f91af6b803ea tempest-SecurityGroupRulesNegativeTestJSON-1399499068 tempest-SecurityGroupRulesNegativeTestJSON-1399499068-project] HTTP exception thrown: Invalid IP protocol tempest-999-960908298. Jul 16 21:19:17.468910 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-0737fe38-f7a4-4c95-8979-f91af6b803ea tempest-SecurityGroupRulesNegativeTestJSON-1399499068 tempest-SecurityGroupRulesNegativeTestJSON-1399499068-project] Request method failure captured: Jul 16 21:19:17.468910 ncdv-master-1 devstack@n-api.service[138680]: request: POST /compute/v2.1/os-security-group-rules HTTP/1.1 Jul 16 21:19:17.468910 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:19:17.468910 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:19:17.468910 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:19:17.468910 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 156 Jul 16 21:19:17.468910 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:19:17.468910 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:19:17.468910 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:19:17.468910 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:19:17.468910 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0ytNkmtXKapUZpFKPGUA8hf3ATbduJ0XtzorjW7amdLo0WS0Qbkqbi3B3wBROLLwxfXJZxCXXmOW4by85qgjbrIW-gGZ2c_YrmiVSX-WID5VqAWhi2CmsmYLeeu_6bw9QJDULh6nweMasTNNM3cnBfI9QUQLgKaJIeA4QYZkK1W0reQ Jul 16 21:19:17.468910 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:19:17.468910 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:19:17.468910 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:19:17.468910 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:19:17.468910 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:19:17.468910 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:19:17.468910 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: 89b4d12252b94d54b50cc15238af6c2f Jul 16 21:19:17.468910 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-SecurityGroupRulesNegativeTestJSON-1399499068 Jul 16 21:19:17.468910 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:19:17.468910 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:19:17.468910 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/89b4d12252b94d54b50cc15238af6c2f"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/89b4d12252b94d54b50cc15238af6c2f"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_89b4d12252b94d54b50cc15238af6c2f"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/89b4d12252b94d54b50cc15238af6c2f"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:19:17.468910 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-SecurityGroupRulesNegativeTestJSON-1399499068 Jul 16 21:19:17.468910 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: 89b4d12252b94d54b50cc15238af6c2f Jul 16 21:19:17.468910 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-SecurityGroupRulesNegativeTestJSON-1399499068 Jul 16 21:19:17.468910 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-SecurityGroupRulesNegativeTestJSON-1399499068-project Jul 16 21:19:17.468910 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:19:17.468910 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:19:17.468910 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: 7ea30eb37fbf469194845261a19e8d4f Jul 16 21:19:17.468910 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-SecurityGroupRulesNegativeTestJSON-1399499068-project Jul 16 21:19:17.468910 ncdv-master-1 devstack@n-api.service[138680]: Jul 16 21:19:17.468910 ncdv-master-1 devstack@n-api.service[138680]: {"security_group_rule": {"parent_group_id": "65d24cc4-8c2d-4fb8-bf7b-a64bb6225fe1", "ip_protocol": "tempest-999-960908298", "from_port": 22, "to_port": 22}} Jul 16 21:19:17.468910 ncdv-master-1 devstack@n-api.service[138680]: method: Jul 16 21:19:17.468910 ncdv-master-1 devstack@n-api.service[138680]: exception: Invalid IP protocol tempest-999-960908298. Jul 16 21:19:17.468910 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'body': {'security_group_rule': {'parent_group_id': '65d24cc4-8c2d-4fb8-bf7b-a64bb6225fe1', 'ip_protocol': 'tempest-999-960908298', 'from_port': 22, 'to_port': 22}}} Jul 16 21:19:17.468910 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:19:17.468910 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:19:17.468910 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/security_groups.py", line 294, in create Jul 16 21:19:17.468910 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi new_rule = self._rule_args_to_dict(context, Jul 16 21:19:17.468910 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/security_groups.py", line 346, in _rule_args_to_dict Jul 16 21:19:17.468910 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return security_group_api.new_cidr_ingress_rule( Jul 16 21:19:17.468910 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/security_group_api.py", line 103, in new_cidr_ingress_rule Jul 16 21:19:17.468910 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return _new_ingress_rule( Jul 16 21:19:17.468910 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/security_group_api.py", line 144, in _new_ingress_rule Jul 16 21:19:17.468910 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.InvalidIpProtocol(protocol=ip_protocol) Jul 16 21:19:17.468910 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.InvalidIpProtocol: Invalid IP protocol tempest-999-960908298. Jul 16 21:19:17.468910 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:19:17.468910 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:19:17.468910 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:19:17.468910 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:19:17.468910 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:19:17.468910 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:19:17.468910 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:19:17.468910 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:19:17.468910 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Jul 16 21:19:17.468910 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Jul 16 21:19:17.468910 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:19:17.468910 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:19:17.468910 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/security_groups.py", line 301, in create Jul 16 21:19:17.468910 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=exp.format_message()) Jul 16 21:19:17.468910 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid IP protocol tempest-999-960908298. Jul 16 21:19:17.468910 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:19:17.468910 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:19:17.479027 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:19:17.479027 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:19:17.479027 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:19:17.479027 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:19:17.479027 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:19:17.479027 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:19:17.479027 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid IP protocol tempest-999-960908298. Jul 16 21:19:17.479027 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:19:17.479027 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-0737fe38-f7a4-4c95-8979-f91af6b803ea tempest-SecurityGroupRulesNegativeTestJSON-1399499068 tempest-SecurityGroupRulesNegativeTestJSON-1399499068-project] Returning 400 to user: Invalid IP protocol tempest-999-960908298. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:19:17.479027 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-0737fe38-f7a4-4c95-8979-f91af6b803ea tempest-SecurityGroupRulesNegativeTestJSON-1399499068 tempest-SecurityGroupRulesNegativeTestJSON-1399499068-project] 10.222.0.16 "POST /compute/v2.1/os-security-group-rules" status: 400 len: 86 microversion: 2.1 time: 0.072319 Jul 16 21:19:17.479027 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1334/2631] 10.222.0.16 () {60 vars in 1194 bytes} [Sat Jul 16 21:19:17 2022] POST /compute/v2.1/os-security-group-rules => generated 86 bytes in 73 msecs (HTTP/1.1 400) 9 headers in 380 bytes (1 switches on core 0) Jul 16 21:19:17.479027 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-61946084-58f9-437a-8a2b-cdbeef488616 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:19:17.479027 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-61946084-58f9-437a-8a2b-cdbeef488616 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] [instance: c88b191a-1ae5-4952-8afa-14aee5567a7f] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:19:17.484696 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-61946084-58f9-437a-8a2b-cdbeef488616 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:17.485157 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-61946084-58f9-437a-8a2b-cdbeef488616 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:17.485934 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-61946084-58f9-437a-8a2b-cdbeef488616 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:17.486500 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-61946084-58f9-437a-8a2b-cdbeef488616 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:17.487240 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-40039fbe-0426-4d33-83f3-3de524f1eb66 tempest-VolumesSnapshotsTestJSON-851680556 tempest-VolumesSnapshotsTestJSON-851680556-project] Calling method '.version_select at 0x7fdd9d090790>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:19:17.509337 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-61946084-58f9-437a-8a2b-cdbeef488616 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] HTTP exception thrown: Instance c88b191a-1ae5-4952-8afa-14aee5567a7f could not be found. Jul 16 21:19:17.511220 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-61946084-58f9-437a-8a2b-cdbeef488616 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Request method failure captured: Jul 16 21:19:17.511220 ncdv-master-1 devstack@n-api.service[138680]: request: DELETE /compute/v2.1/servers/c88b191a-1ae5-4952-8afa-14aee5567a7f HTTP/1.1 Jul 16 21:19:17.511220 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:19:17.511220 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:19:17.511220 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:19:17.511220 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 0 Jul 16 21:19:17.511220 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:19:17.511220 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:19:17.511220 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:19:17.511220 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:19:17.511220 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0yrk-HsfYKsyovLSaEtNxaoXy7a3rLGNlsRPiPP76ZvvrKkKyGd-uekosXQxpFgkRnItBdviN4xr49Rerg5AyvLSzsyRY-Um0r3BRAvyZrzmCDYUpWbXk1uyEIb3Af4U3Y3nXIJgJAuDVx9z2NGSvEGCsGz12WlyyGCNfVSADFcYJP8 Jul 16 21:19:17.511220 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:19:17.511220 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:19:17.511220 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:19:17.511220 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:19:17.511220 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:19:17.511220 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:19:17.511220 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: e353258f9c3048ef84683294e2b8579d Jul 16 21:19:17.511220 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-SecurityGroupsTestJSON-438340169 Jul 16 21:19:17.511220 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:19:17.511220 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:19:17.511220 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/e353258f9c3048ef84683294e2b8579d"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/e353258f9c3048ef84683294e2b8579d"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_e353258f9c3048ef84683294e2b8579d"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/e353258f9c3048ef84683294e2b8579d"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:19:17.511220 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-SecurityGroupsTestJSON-438340169 Jul 16 21:19:17.511220 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: e353258f9c3048ef84683294e2b8579d Jul 16 21:19:17.511220 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-SecurityGroupsTestJSON-438340169 Jul 16 21:19:17.511220 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-SecurityGroupsTestJSON-438340169-project Jul 16 21:19:17.511220 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:19:17.511220 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:19:17.511220 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: d0008fffad8e4708b2c01e03da92c47d Jul 16 21:19:17.511220 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-SecurityGroupsTestJSON-438340169-project Jul 16 21:19:17.511220 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 21:19:17.511220 ncdv-master-1 devstack@n-api.service[138680]: exception: Instance c88b191a-1ae5-4952-8afa-14aee5567a7f could not be found. Jul 16 21:19:17.511220 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': 'c88b191a-1ae5-4952-8afa-14aee5567a7f'} Jul 16 21:19:17.511220 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:19:17.511220 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:19:17.511220 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:19:17.511220 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:19:17.511220 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:19:17.511220 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:19:17.511220 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:19:17.511220 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:19:17.511220 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:19:17.511220 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:19:17.511220 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance c88b191a-1ae5-4952-8afa-14aee5567a7f could not be found. Jul 16 21:19:17.511220 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:19:17.511220 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:19:17.511220 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:19:17.511220 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:19:17.511220 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:19:17.511220 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:19:17.511220 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:19:17.511220 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:19:17.511220 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:19:17.511220 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:19:17.511220 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Jul 16 21:19:17.511220 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Jul 16 21:19:17.511220 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Jul 16 21:19:17.511220 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Jul 16 21:19:17.511220 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:19:17.511220 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:19:17.511220 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:19:17.511220 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:19:17.515524 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance c88b191a-1ae5-4952-8afa-14aee5567a7f could not be found. Jul 16 21:19:17.515524 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:19:17.515524 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:19:17.515524 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:19:17.515524 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:19:17.515524 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:19:17.515524 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:19:17.515524 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:19:17.515524 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:19:17.515524 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance c88b191a-1ae5-4952-8afa-14aee5567a7f could not be found. Jul 16 21:19:17.515524 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:19:17.515524 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-61946084-58f9-437a-8a2b-cdbeef488616 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Returning 404 to user: Instance c88b191a-1ae5-4952-8afa-14aee5567a7f could not be found. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:19:17.515524 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-61946084-58f9-437a-8a2b-cdbeef488616 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] 10.222.0.16 "DELETE /compute/v2.1/servers/c88b191a-1ae5-4952-8afa-14aee5567a7f" status: 404 len: 111 microversion: 2.1 time: 0.040694 Jul 16 21:19:17.515524 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1335/2632] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:19:17 2022] DELETE /compute/v2.1/servers/c88b191a-1ae5-4952-8afa-14aee5567a7f => generated 111 bytes in 42 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:19:17.520287 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-9d69ec4f-3fa0-4f5f-97f5-7f3b741186e3 tempest-SecurityGroupRulesNegativeTestJSON-1399499068 tempest-SecurityGroupRulesNegativeTestJSON-1399499068-project] Action: 'create', calling method: .version_select at 0x7fdd9cc35550>, body: {"security_group": {"name": "tempest-SecurityGroupRulesNegativeTestJSON-securitygroup-831813123", "description": "tempest-description-711060677"}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:19:17.525700 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-40039fbe-0426-4d33-83f3-3de524f1eb66 tempest-VolumesSnapshotsTestJSON-851680556 tempest-VolumesSnapshotsTestJSON-851680556-project] HTTP exception thrown: Volume 4450505c-3ac5-42c2-b94c-b8f945007d50 could not be found. Jul 16 21:19:17.531513 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-40039fbe-0426-4d33-83f3-3de524f1eb66 tempest-VolumesSnapshotsTestJSON-851680556 tempest-VolumesSnapshotsTestJSON-851680556-project] Request method failure captured: Jul 16 21:19:17.531513 ncdv-master-1 devstack@n-api.service[138681]: request: DELETE /compute/v2.1/os-volumes/4450505c-3ac5-42c2-b94c-b8f945007d50 HTTP/1.1 Jul 16 21:19:17.531513 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:19:17.531513 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:19:17.531513 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:19:17.531513 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 21:19:17.531513 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:19:17.531513 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:19:17.531513 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:19:17.531513 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:19:17.531513 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0ys_bQ2YkbojsHK1HUJf-q5eS4WAKF6XcJ7jYKIj3a8lR37PmQ0zZ5DyXkNSC8x8kLgg3bQW6slrse7LirtZA91amTfVZ6Kr3gaUumm-yoTjKpsyiJvqY1GYtuGlfRXmsIYTP5FABjs_4cPzEKKvoKI8h2KKC02FgQGNg_a7XC56cnE Jul 16 21:19:17.531513 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:19:17.531513 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:19:17.531513 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:19:17.531513 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:19:17.531513 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:19:17.531513 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:19:17.531513 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: 942725b966874321b6ccd6f628787d06 Jul 16 21:19:17.531513 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-VolumesSnapshotsTestJSON-851680556 Jul 16 21:19:17.531513 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:19:17.531513 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:19:17.531513 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/942725b966874321b6ccd6f628787d06"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/942725b966874321b6ccd6f628787d06"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_942725b966874321b6ccd6f628787d06"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/942725b966874321b6ccd6f628787d06"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:19:17.531513 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-VolumesSnapshotsTestJSON-851680556 Jul 16 21:19:17.531513 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: 942725b966874321b6ccd6f628787d06 Jul 16 21:19:17.531513 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-VolumesSnapshotsTestJSON-851680556 Jul 16 21:19:17.531513 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-VolumesSnapshotsTestJSON-851680556-project Jul 16 21:19:17.531513 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:19:17.531513 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:19:17.531513 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: 6a3dbbb2b39447d884c56dfd0db7e42e Jul 16 21:19:17.531513 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-VolumesSnapshotsTestJSON-851680556-project Jul 16 21:19:17.531513 ncdv-master-1 devstack@n-api.service[138681]: method: Jul 16 21:19:17.531513 ncdv-master-1 devstack@n-api.service[138681]: exception: Volume 4450505c-3ac5-42c2-b94c-b8f945007d50 could not be found. Jul 16 21:19:17.531513 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': '4450505c-3ac5-42c2-b94c-b8f945007d50'} Jul 16 21:19:17.531513 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:19:17.531513 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:19:17.531513 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jul 16 21:19:17.531513 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jul 16 21:19:17.531513 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Jul 16 21:19:17.531513 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Jul 16 21:19:17.531513 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Jul 16 21:19:17.531513 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return self._delete(loc) Jul 16 21:19:17.531513 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Jul 16 21:19:17.531513 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Jul 16 21:19:17.531513 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Jul 16 21:19:17.531513 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Jul 16 21:19:17.531513 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jul 16 21:19:17.531513 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jul 16 21:19:17.531513 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jul 16 21:19:17.531513 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jul 16 21:19:17.531513 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume 4450505c-3ac5-42c2-b94c-b8f945007d50 could not be found. (HTTP 404) (Request-ID: req-f4c37b84-e343-474a-a117-b95161c2598c) Jul 16 21:19:17.531513 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:19:17.531513 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:19:17.531513 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:19:17.531513 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:19:17.531513 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 136, in delete Jul 16 21:19:17.531513 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi self.volume_api.delete(context, id) Jul 16 21:19:17.531513 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Jul 16 21:19:17.531513 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Jul 16 21:19:17.531513 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Jul 16 21:19:17.531513 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Jul 16 21:19:17.531513 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Jul 16 21:19:17.531513 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Jul 16 21:19:17.531513 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jul 16 21:19:17.544218 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jul 16 21:19:17.544218 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Jul 16 21:19:17.544218 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Jul 16 21:19:17.544218 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Jul 16 21:19:17.544218 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return self._delete(loc) Jul 16 21:19:17.544218 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Jul 16 21:19:17.544218 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Jul 16 21:19:17.544218 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Jul 16 21:19:17.544218 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Jul 16 21:19:17.544218 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jul 16 21:19:17.544218 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jul 16 21:19:17.544218 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jul 16 21:19:17.544218 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jul 16 21:19:17.544218 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume 4450505c-3ac5-42c2-b94c-b8f945007d50 could not be found. Jul 16 21:19:17.544218 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:19:17.544218 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:19:17.544218 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:19:17.544218 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:19:17.544218 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:19:17.544218 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:19:17.544218 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:19:17.544218 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:19:17.544218 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Jul 16 21:19:17.544218 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Jul 16 21:19:17.544218 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:19:17.544218 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:19:17.544218 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 140, in delete Jul 16 21:19:17.544218 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:19:17.544218 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume 4450505c-3ac5-42c2-b94c-b8f945007d50 could not be found. Jul 16 21:19:17.544218 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:19:17.544218 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:19:17.544218 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:19:17.544218 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:19:17.544218 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:19:17.544218 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:19:17.544218 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:19:17.544218 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:19:17.544218 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume 4450505c-3ac5-42c2-b94c-b8f945007d50 could not be found. Jul 16 21:19:17.544218 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:19:17.544218 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-40039fbe-0426-4d33-83f3-3de524f1eb66 tempest-VolumesSnapshotsTestJSON-851680556 tempest-VolumesSnapshotsTestJSON-851680556-project] Returning 404 to user: Volume 4450505c-3ac5-42c2-b94c-b8f945007d50 could not be found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:19:17.544218 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-40039fbe-0426-4d33-83f3-3de524f1eb66 tempest-VolumesSnapshotsTestJSON-851680556 tempest-VolumesSnapshotsTestJSON-851680556-project] 10.222.0.16 "DELETE /compute/v2.1/os-volumes/4450505c-3ac5-42c2-b94c-b8f945007d50" status: 404 len: 109 microversion: 2.1 time: 0.066850 Jul 16 21:19:17.544218 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1298/2633] 10.222.0.16 () {58 vars in 1247 bytes} [Sat Jul 16 21:19:17 2022] DELETE /compute/v2.1/os-volumes/4450505c-3ac5-42c2-b94c-b8f945007d50 => generated 109 bytes in 68 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:19:17.544218 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-3a95dd20-5de4-4dbc-b8d9-70dbfcd776c1 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:19:17.544218 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-3a95dd20-5de4-4dbc-b8d9-70dbfcd776c1 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] [instance: c88b191a-1ae5-4952-8afa-14aee5567a7f] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:19:17.547874 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3a95dd20-5de4-4dbc-b8d9-70dbfcd776c1 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:17.548329 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3a95dd20-5de4-4dbc-b8d9-70dbfcd776c1 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:17.549115 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3a95dd20-5de4-4dbc-b8d9-70dbfcd776c1 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:17.550002 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3a95dd20-5de4-4dbc-b8d9-70dbfcd776c1 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:17.576070 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-3a95dd20-5de4-4dbc-b8d9-70dbfcd776c1 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] HTTP exception thrown: Instance c88b191a-1ae5-4952-8afa-14aee5567a7f could not be found. Jul 16 21:19:17.577746 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-3a95dd20-5de4-4dbc-b8d9-70dbfcd776c1 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Request method failure captured: Jul 16 21:19:17.577746 ncdv-master-1 devstack@n-api.service[138681]: request: GET /compute/v2.1/servers/c88b191a-1ae5-4952-8afa-14aee5567a7f HTTP/1.1 Jul 16 21:19:17.577746 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:19:17.577746 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:19:17.577746 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:19:17.577746 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 21:19:17.577746 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:19:17.577746 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:19:17.577746 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:19:17.577746 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:19:17.577746 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0yrk-HsfYKsyovLSaEtNxaoXy7a3rLGNlsRPiPP76ZvvrKkKyGd-uekosXQxpFgkRnItBdviN4xr49Rerg5AyvLSzsyRY-Um0r3BRAvyZrzmCDYUpWbXk1uyEIb3Af4U3Y3nXIJgJAuDVx9z2NGSvEGCsGz12WlyyGCNfVSADFcYJP8 Jul 16 21:19:17.577746 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:19:17.577746 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:19:17.577746 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:19:17.577746 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:19:17.577746 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:19:17.577746 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:19:17.577746 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: e353258f9c3048ef84683294e2b8579d Jul 16 21:19:17.577746 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-SecurityGroupsTestJSON-438340169 Jul 16 21:19:17.577746 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:19:17.577746 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:19:17.577746 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/e353258f9c3048ef84683294e2b8579d"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/e353258f9c3048ef84683294e2b8579d"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_e353258f9c3048ef84683294e2b8579d"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/e353258f9c3048ef84683294e2b8579d"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:19:17.577746 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-SecurityGroupsTestJSON-438340169 Jul 16 21:19:17.577746 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: e353258f9c3048ef84683294e2b8579d Jul 16 21:19:17.577746 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-SecurityGroupsTestJSON-438340169 Jul 16 21:19:17.577746 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-SecurityGroupsTestJSON-438340169-project Jul 16 21:19:17.577746 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:19:17.577746 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:19:17.577746 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: d0008fffad8e4708b2c01e03da92c47d Jul 16 21:19:17.577746 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-SecurityGroupsTestJSON-438340169-project Jul 16 21:19:17.577746 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:19:17.577746 ncdv-master-1 devstack@n-api.service[138681]: exception: Instance c88b191a-1ae5-4952-8afa-14aee5567a7f could not be found. Jul 16 21:19:17.577746 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': 'c88b191a-1ae5-4952-8afa-14aee5567a7f'} Jul 16 21:19:17.577746 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:19:17.577746 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:19:17.577746 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:19:17.577746 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:19:17.577746 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:19:17.577746 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:19:17.577746 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:19:17.577746 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:19:17.577746 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:19:17.577746 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:19:17.577746 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance c88b191a-1ae5-4952-8afa-14aee5567a7f could not be found. Jul 16 21:19:17.577746 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:19:17.577746 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:19:17.577746 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:19:17.577746 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:19:17.577746 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:19:17.577746 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:19:17.577746 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:19:17.577746 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:19:17.577746 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:19:17.577746 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:19:17.577746 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:19:17.577746 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:19:17.577746 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:19:17.577746 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:19:17.577746 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:19:17.577746 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:19:17.577746 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance c88b191a-1ae5-4952-8afa-14aee5567a7f could not be found. Jul 16 21:19:17.577746 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:19:17.577746 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:19:17.577746 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:19:17.582121 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:19:17.582121 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:19:17.582121 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:19:17.582121 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:19:17.582121 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:19:17.582121 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance c88b191a-1ae5-4952-8afa-14aee5567a7f could not be found. Jul 16 21:19:17.582121 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:19:17.582121 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-3a95dd20-5de4-4dbc-b8d9-70dbfcd776c1 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Returning 404 to user: Instance c88b191a-1ae5-4952-8afa-14aee5567a7f could not be found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:19:17.582121 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-3a95dd20-5de4-4dbc-b8d9-70dbfcd776c1 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] 10.222.0.16 "GET /compute/v2.1/servers/c88b191a-1ae5-4952-8afa-14aee5567a7f" status: 404 len: 111 microversion: 2.1 time: 0.044641 Jul 16 21:19:17.582121 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1299/2634] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:19:17 2022] GET /compute/v2.1/servers/c88b191a-1ae5-4952-8afa-14aee5567a7f => generated 111 bytes in 46 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:19:17.584446 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-2c1e6f23-626c-4744-b0a4-62cc96da557a tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:19:17.585061 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-2c1e6f23-626c-4744-b0a4-62cc96da557a tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] [instance: e7598b4a-f70c-4186-88d1-6fd0057f8656] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:19:17.592074 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2c1e6f23-626c-4744-b0a4-62cc96da557a tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:17.592537 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2c1e6f23-626c-4744-b0a4-62cc96da557a tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:17.593351 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2c1e6f23-626c-4744-b0a4-62cc96da557a tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:17.593803 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2c1e6f23-626c-4744-b0a4-62cc96da557a tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:17.892362 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-2c1e6f23-626c-4744-b0a4-62cc96da557a tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f5bc1f2209f4eb186d80e385b67c774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '65d5bcfa02af4e8284ba1342197e2860', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:19:17.923458 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-2c1e6f23-626c-4744-b0a4-62cc96da557a tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] 10.222.0.16 "GET /compute/v2.1/servers/e7598b4a-f70c-4186-88d1-6fd0057f8656" status: 200 len: 1472 microversion: 2.1 time: 0.342432 Jul 16 21:19:17.924365 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1300/2635] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:19:17 2022] GET /compute/v2.1/servers/e7598b4a-f70c-4186-88d1-6fd0057f8656 => generated 1472 bytes in 344 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:19:17.930594 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-e56fb94b-d688-4b64-9ed4-a2c426a54850 tempest-VolumesSnapshotsTestJSON-851680556 tempest-VolumesSnapshotsTestJSON-851680556-project] Calling method '.version_select at 0x7fdd9d086af0>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:19:17.965434 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-e56fb94b-d688-4b64-9ed4-a2c426a54850 tempest-VolumesSnapshotsTestJSON-851680556 tempest-VolumesSnapshotsTestJSON-851680556-project] HTTP exception thrown: Volume 4450505c-3ac5-42c2-b94c-b8f945007d50 could not be found. Jul 16 21:19:17.967697 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-e56fb94b-d688-4b64-9ed4-a2c426a54850 tempest-VolumesSnapshotsTestJSON-851680556 tempest-VolumesSnapshotsTestJSON-851680556-project] Request method failure captured: Jul 16 21:19:17.967697 ncdv-master-1 devstack@n-api.service[138681]: request: GET /compute/v2.1/os-volumes/4450505c-3ac5-42c2-b94c-b8f945007d50 HTTP/1.1 Jul 16 21:19:17.967697 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:19:17.967697 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:19:17.967697 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:19:17.967697 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 21:19:17.967697 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:19:17.967697 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:19:17.967697 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:19:17.967697 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:19:17.967697 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0ys_bQ2YkbojsHK1HUJf-q5eS4WAKF6XcJ7jYKIj3a8lR37PmQ0zZ5DyXkNSC8x8kLgg3bQW6slrse7LirtZA91amTfVZ6Kr3gaUumm-yoTjKpsyiJvqY1GYtuGlfRXmsIYTP5FABjs_4cPzEKKvoKI8h2KKC02FgQGNg_a7XC56cnE Jul 16 21:19:17.967697 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:19:17.967697 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:19:17.967697 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:19:17.967697 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:19:17.967697 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:19:17.967697 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:19:17.967697 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: 942725b966874321b6ccd6f628787d06 Jul 16 21:19:17.967697 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-VolumesSnapshotsTestJSON-851680556 Jul 16 21:19:17.967697 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:19:17.967697 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:19:17.967697 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/942725b966874321b6ccd6f628787d06"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/942725b966874321b6ccd6f628787d06"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_942725b966874321b6ccd6f628787d06"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/942725b966874321b6ccd6f628787d06"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:19:17.967697 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-VolumesSnapshotsTestJSON-851680556 Jul 16 21:19:17.967697 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: 942725b966874321b6ccd6f628787d06 Jul 16 21:19:17.967697 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-VolumesSnapshotsTestJSON-851680556 Jul 16 21:19:17.967697 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-VolumesSnapshotsTestJSON-851680556-project Jul 16 21:19:17.967697 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:19:17.967697 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:19:17.967697 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: 6a3dbbb2b39447d884c56dfd0db7e42e Jul 16 21:19:17.967697 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-VolumesSnapshotsTestJSON-851680556-project Jul 16 21:19:17.967697 ncdv-master-1 devstack@n-api.service[138681]: method: Jul 16 21:19:17.967697 ncdv-master-1 devstack@n-api.service[138681]: exception: Volume 4450505c-3ac5-42c2-b94c-b8f945007d50 could not be found. Jul 16 21:19:17.967697 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': '4450505c-3ac5-42c2-b94c-b8f945007d50'} Jul 16 21:19:17.967697 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:19:17.967697 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:19:17.967697 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jul 16 21:19:17.967697 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jul 16 21:19:17.967697 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Jul 16 21:19:17.967697 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi item = cinderclient( Jul 16 21:19:17.967697 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Jul 16 21:19:17.967697 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Jul 16 21:19:17.967697 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Jul 16 21:19:17.967697 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Jul 16 21:19:17.967697 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Jul 16 21:19:17.967697 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Jul 16 21:19:17.967697 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jul 16 21:19:17.967697 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jul 16 21:19:17.967697 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jul 16 21:19:17.967697 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jul 16 21:19:17.967697 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume 4450505c-3ac5-42c2-b94c-b8f945007d50 could not be found. (HTTP 404) (Request-ID: req-1da485b9-d952-498d-b6c1-10c1967bfa64) Jul 16 21:19:17.967697 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:19:17.967697 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:19:17.967697 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:19:17.967697 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:19:17.967697 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 120, in show Jul 16 21:19:17.967697 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get(context, id) Jul 16 21:19:17.967697 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Jul 16 21:19:17.967697 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Jul 16 21:19:17.967697 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Jul 16 21:19:17.967697 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Jul 16 21:19:17.967697 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Jul 16 21:19:17.967697 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Jul 16 21:19:17.967697 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jul 16 21:19:17.972202 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jul 16 21:19:17.972202 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Jul 16 21:19:17.972202 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi item = cinderclient( Jul 16 21:19:17.972202 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Jul 16 21:19:17.972202 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Jul 16 21:19:17.972202 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Jul 16 21:19:17.972202 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Jul 16 21:19:17.972202 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Jul 16 21:19:17.972202 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Jul 16 21:19:17.972202 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jul 16 21:19:17.972202 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jul 16 21:19:17.972202 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jul 16 21:19:17.972202 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jul 16 21:19:17.972202 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume 4450505c-3ac5-42c2-b94c-b8f945007d50 could not be found. Jul 16 21:19:17.972202 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:19:17.972202 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:19:17.972202 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:19:17.972202 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:19:17.972202 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:19:17.972202 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:19:17.972202 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:19:17.972202 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:19:17.972202 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Jul 16 21:19:17.972202 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Jul 16 21:19:17.972202 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:19:17.972202 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:19:17.972202 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 122, in show Jul 16 21:19:17.972202 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:19:17.972202 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume 4450505c-3ac5-42c2-b94c-b8f945007d50 could not be found. Jul 16 21:19:17.972202 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:19:17.972202 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:19:17.972202 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:19:17.972202 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:19:17.972202 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:19:17.972202 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:19:17.972202 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:19:17.972202 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:19:17.972202 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume 4450505c-3ac5-42c2-b94c-b8f945007d50 could not be found. Jul 16 21:19:17.972202 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:19:17.972202 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-e56fb94b-d688-4b64-9ed4-a2c426a54850 tempest-VolumesSnapshotsTestJSON-851680556 tempest-VolumesSnapshotsTestJSON-851680556-project] Returning 404 to user: Volume 4450505c-3ac5-42c2-b94c-b8f945007d50 could not be found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:19:17.975528 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-e56fb94b-d688-4b64-9ed4-a2c426a54850 tempest-VolumesSnapshotsTestJSON-851680556 tempest-VolumesSnapshotsTestJSON-851680556-project] 10.222.0.16 "GET /compute/v2.1/os-volumes/4450505c-3ac5-42c2-b94c-b8f945007d50" status: 404 len: 109 microversion: 2.1 time: 0.047442 Jul 16 21:19:17.975528 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1301/2636] 10.222.0.16 () {58 vars in 1244 bytes} [Sat Jul 16 21:19:17 2022] GET /compute/v2.1/os-volumes/4450505c-3ac5-42c2-b94c-b8f945007d50 => generated 109 bytes in 49 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:19:17.985142 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-8eb0ec31-63c4-4168-96ac-0adc3b6a90ed tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Calling method '.version_select at 0x7fdd9cfac5e0>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:19:18.051772 ncdv-master-1 devstack@n-api.service[138681]: DEBUG neutronclient.v2_0.client [None req-8eb0ec31-63c4-4168-96ac-0adc3b6a90ed tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Error message: {"NeutronError": {"type": "SecurityGroupNotFound", "message": "Security group c176d065-4e8f-45fb-99d9-e12db1c30218 does not exist", "detail": ""}} {{(pid=138681) _handle_fault_response /usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py:262}} Jul 16 21:19:18.052393 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.network.security_group_api [None req-8eb0ec31-63c4-4168-96ac-0adc3b6a90ed tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Neutron security group c176d065-4e8f-45fb-99d9-e12db1c30218 not found {{(pid=138681) get /opt/stack/nova/nova/network/security_group_api.py:313}} Jul 16 21:19:18.052956 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-8eb0ec31-63c4-4168-96ac-0adc3b6a90ed tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] HTTP exception thrown: Security group c176d065-4e8f-45fb-99d9-e12db1c30218 does not exist Jul 16 21:19:18.052956 ncdv-master-1 devstack@n-api.service[138681]: Neutron server returns request_ids: ['req-d1e8fd91-149c-4b3b-8561-4932deee668d'] Jul 16 21:19:18.056346 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-8eb0ec31-63c4-4168-96ac-0adc3b6a90ed tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Request method failure captured: Jul 16 21:19:18.056346 ncdv-master-1 devstack@n-api.service[138681]: request: DELETE /compute/v2.1/os-security-groups/c176d065-4e8f-45fb-99d9-e12db1c30218 HTTP/1.1 Jul 16 21:19:18.056346 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:19:18.056346 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:19:18.056346 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:19:18.056346 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 21:19:18.056346 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:19:18.056346 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:19:18.056346 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:19:18.056346 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:19:18.056346 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0yrk-HsfYKsyovLSaEtNxaoXy7a3rLGNlsRPiPP76ZvvrKkKyGd-uekosXQxpFgkRnItBdviN4xr49Rerg5AyvLSzsyRY-Um0r3BRAvyZrzmCDYUpWbXk1uyEIb3Af4U3Y3nXIJgJAuDVx9z2NGSvEGCsGz12WlyyGCNfVSADFcYJP8 Jul 16 21:19:18.056346 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:19:18.056346 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:19:18.056346 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:19:18.056346 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:19:18.056346 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:19:18.056346 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:19:18.056346 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: e353258f9c3048ef84683294e2b8579d Jul 16 21:19:18.056346 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-SecurityGroupsTestJSON-438340169 Jul 16 21:19:18.056346 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:19:18.056346 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:19:18.056346 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/e353258f9c3048ef84683294e2b8579d"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/e353258f9c3048ef84683294e2b8579d"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_e353258f9c3048ef84683294e2b8579d"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/e353258f9c3048ef84683294e2b8579d"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:19:18.056346 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-SecurityGroupsTestJSON-438340169 Jul 16 21:19:18.056346 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: e353258f9c3048ef84683294e2b8579d Jul 16 21:19:18.056346 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-SecurityGroupsTestJSON-438340169 Jul 16 21:19:18.056346 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-SecurityGroupsTestJSON-438340169-project Jul 16 21:19:18.056346 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:19:18.056346 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:19:18.056346 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: d0008fffad8e4708b2c01e03da92c47d Jul 16 21:19:18.056346 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-SecurityGroupsTestJSON-438340169-project Jul 16 21:19:18.056346 ncdv-master-1 devstack@n-api.service[138681]: method: Jul 16 21:19:18.056346 ncdv-master-1 devstack@n-api.service[138681]: exception: Security group c176d065-4e8f-45fb-99d9-e12db1c30218 does not exist Jul 16 21:19:18.056346 ncdv-master-1 devstack@n-api.service[138681]: Neutron server returns request_ids: ['req-d1e8fd91-149c-4b3b-8561-4932deee668d'] Jul 16 21:19:18.056346 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': 'c176d065-4e8f-45fb-99d9-e12db1c30218'} Jul 16 21:19:18.056346 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:19:18.056346 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:19:18.056346 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/security_group_api.py", line 309, in get Jul 16 21:19:18.056346 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi group = neutron.show_security_group(id).get('security_group') Jul 16 21:19:18.056346 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 197, in wrapper Jul 16 21:19:18.056346 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Jul 16 21:19:18.056346 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 1040, in show_security_group Jul 16 21:19:18.056346 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return self.get(self.security_group_path % (security_group), Jul 16 21:19:18.056346 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 197, in wrapper Jul 16 21:19:18.056346 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Jul 16 21:19:18.056346 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 356, in get Jul 16 21:19:18.056346 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return self.retry_request("GET", action, body=body, Jul 16 21:19:18.056346 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 197, in wrapper Jul 16 21:19:18.056346 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Jul 16 21:19:18.056346 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 333, in retry_request Jul 16 21:19:18.056346 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return self.do_request(method, action, body=body, Jul 16 21:19:18.056346 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 197, in wrapper Jul 16 21:19:18.056346 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Jul 16 21:19:18.056346 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 297, in do_request Jul 16 21:19:18.056346 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi self._handle_fault_response(status_code, replybody, resp) Jul 16 21:19:18.056346 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 197, in wrapper Jul 16 21:19:18.056346 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Jul 16 21:19:18.056346 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 272, in _handle_fault_response Jul 16 21:19:18.056346 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi exception_handler_v20(status_code, error_body) Jul 16 21:19:18.056346 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 90, in exception_handler_v20 Jul 16 21:19:18.056346 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise client_exc(message=error_message, Jul 16 21:19:18.056346 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi neutronclient.common.exceptions.NotFound: Security group c176d065-4e8f-45fb-99d9-e12db1c30218 does not exist Jul 16 21:19:18.062727 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-d1e8fd91-149c-4b3b-8561-4932deee668d'] Jul 16 21:19:18.062727 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:19:18.062727 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:19:18.062727 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:19:18.062727 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:19:18.062727 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/security_groups.py", line 178, in delete Jul 16 21:19:18.062727 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi security_group = security_group_api.get(context, id) Jul 16 21:19:18.062727 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/security_group_api.py", line 314, in get Jul 16 21:19:18.062727 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.SecurityGroupNotFound(str(e)) Jul 16 21:19:18.062727 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.SecurityGroupNotFound: Security group c176d065-4e8f-45fb-99d9-e12db1c30218 does not exist Jul 16 21:19:18.062727 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-d1e8fd91-149c-4b3b-8561-4932deee668d'] Jul 16 21:19:18.062727 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:19:18.062727 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:19:18.062727 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:19:18.062727 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:19:18.062727 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:19:18.062727 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:19:18.062727 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:19:18.062727 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:19:18.062727 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Jul 16 21:19:18.062727 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Jul 16 21:19:18.062727 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:19:18.062727 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:19:18.062727 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/security_groups.py", line 181, in delete Jul 16 21:19:18.062727 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=exp.format_message()) Jul 16 21:19:18.062727 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Security group c176d065-4e8f-45fb-99d9-e12db1c30218 does not exist Jul 16 21:19:18.062727 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-d1e8fd91-149c-4b3b-8561-4932deee668d'] Jul 16 21:19:18.062727 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:19:18.062727 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:19:18.062727 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:19:18.062727 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:19:18.062727 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:19:18.062727 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:19:18.062727 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:19:18.062727 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:19:18.062727 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Security group c176d065-4e8f-45fb-99d9-e12db1c30218 does not exist Jul 16 21:19:18.062727 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-d1e8fd91-149c-4b3b-8561-4932deee668d'] Jul 16 21:19:18.062727 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:19:18.062727 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-8eb0ec31-63c4-4168-96ac-0adc3b6a90ed tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Returning 404 to user: Security group c176d065-4e8f-45fb-99d9-e12db1c30218 does not exist Jul 16 21:19:18.062727 ncdv-master-1 devstack@n-api.service[138681]: Neutron server returns request_ids: ['req-d1e8fd91-149c-4b3b-8561-4932deee668d'] {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:19:18.062727 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-8eb0ec31-63c4-4168-96ac-0adc3b6a90ed tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] 10.222.0.16 "DELETE /compute/v2.1/os-security-groups/c176d065-4e8f-45fb-99d9-e12db1c30218" status: 404 len: 194 microversion: 2.1 time: 0.082788 Jul 16 21:19:18.062727 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1302/2637] 10.222.0.16 () {58 vars in 1271 bytes} [Sat Jul 16 21:19:17 2022] DELETE /compute/v2.1/os-security-groups/c176d065-4e8f-45fb-99d9-e12db1c30218 => generated 194 bytes in 84 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:19:18.072024 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-b5357b06-d89c-4a8e-ac02-d08a9fd24211 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Calling method '.version_select at 0x7fdda2a68430>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:19:18.105360 ncdv-master-1 devstack@n-api.service[138681]: DEBUG neutronclient.v2_0.client [None req-b5357b06-d89c-4a8e-ac02-d08a9fd24211 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Error message: {"NeutronError": {"type": "SecurityGroupNotFound", "message": "Security group 89aaff03-f3d3-48f7-965b-66c2a37029e0 does not exist", "detail": ""}} {{(pid=138681) _handle_fault_response /usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py:262}} Jul 16 21:19:18.105360 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.network.security_group_api [None req-b5357b06-d89c-4a8e-ac02-d08a9fd24211 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Neutron security group 89aaff03-f3d3-48f7-965b-66c2a37029e0 not found {{(pid=138681) get /opt/stack/nova/nova/network/security_group_api.py:313}} Jul 16 21:19:18.105842 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-b5357b06-d89c-4a8e-ac02-d08a9fd24211 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] HTTP exception thrown: Security group 89aaff03-f3d3-48f7-965b-66c2a37029e0 does not exist Jul 16 21:19:18.105842 ncdv-master-1 devstack@n-api.service[138681]: Neutron server returns request_ids: ['req-91c71876-afeb-4537-9cb9-74dde240eee9'] Jul 16 21:19:18.107682 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-b5357b06-d89c-4a8e-ac02-d08a9fd24211 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Request method failure captured: Jul 16 21:19:18.107682 ncdv-master-1 devstack@n-api.service[138681]: request: DELETE /compute/v2.1/os-security-groups/89aaff03-f3d3-48f7-965b-66c2a37029e0 HTTP/1.1 Jul 16 21:19:18.107682 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:19:18.107682 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:19:18.107682 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:19:18.107682 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 21:19:18.107682 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:19:18.107682 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:19:18.107682 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:19:18.107682 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:19:18.107682 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0yrk-HsfYKsyovLSaEtNxaoXy7a3rLGNlsRPiPP76ZvvrKkKyGd-uekosXQxpFgkRnItBdviN4xr49Rerg5AyvLSzsyRY-Um0r3BRAvyZrzmCDYUpWbXk1uyEIb3Af4U3Y3nXIJgJAuDVx9z2NGSvEGCsGz12WlyyGCNfVSADFcYJP8 Jul 16 21:19:18.107682 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:19:18.107682 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:19:18.107682 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:19:18.107682 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:19:18.107682 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:19:18.107682 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:19:18.107682 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: e353258f9c3048ef84683294e2b8579d Jul 16 21:19:18.107682 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-SecurityGroupsTestJSON-438340169 Jul 16 21:19:18.107682 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:19:18.107682 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:19:18.107682 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/e353258f9c3048ef84683294e2b8579d"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/e353258f9c3048ef84683294e2b8579d"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_e353258f9c3048ef84683294e2b8579d"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/e353258f9c3048ef84683294e2b8579d"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:19:18.107682 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-SecurityGroupsTestJSON-438340169 Jul 16 21:19:18.107682 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: e353258f9c3048ef84683294e2b8579d Jul 16 21:19:18.107682 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-SecurityGroupsTestJSON-438340169 Jul 16 21:19:18.107682 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-SecurityGroupsTestJSON-438340169-project Jul 16 21:19:18.107682 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:19:18.107682 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:19:18.107682 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: d0008fffad8e4708b2c01e03da92c47d Jul 16 21:19:18.107682 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-SecurityGroupsTestJSON-438340169-project Jul 16 21:19:18.107682 ncdv-master-1 devstack@n-api.service[138681]: method: Jul 16 21:19:18.107682 ncdv-master-1 devstack@n-api.service[138681]: exception: Security group 89aaff03-f3d3-48f7-965b-66c2a37029e0 does not exist Jul 16 21:19:18.107682 ncdv-master-1 devstack@n-api.service[138681]: Neutron server returns request_ids: ['req-91c71876-afeb-4537-9cb9-74dde240eee9'] Jul 16 21:19:18.107682 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': '89aaff03-f3d3-48f7-965b-66c2a37029e0'} Jul 16 21:19:18.107682 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:19:18.107682 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:19:18.107682 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/security_group_api.py", line 309, in get Jul 16 21:19:18.107682 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi group = neutron.show_security_group(id).get('security_group') Jul 16 21:19:18.107682 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 197, in wrapper Jul 16 21:19:18.107682 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Jul 16 21:19:18.107682 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 1040, in show_security_group Jul 16 21:19:18.107682 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return self.get(self.security_group_path % (security_group), Jul 16 21:19:18.107682 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 197, in wrapper Jul 16 21:19:18.107682 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Jul 16 21:19:18.107682 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 356, in get Jul 16 21:19:18.107682 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return self.retry_request("GET", action, body=body, Jul 16 21:19:18.107682 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 197, in wrapper Jul 16 21:19:18.107682 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Jul 16 21:19:18.107682 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 333, in retry_request Jul 16 21:19:18.107682 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return self.do_request(method, action, body=body, Jul 16 21:19:18.107682 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 197, in wrapper Jul 16 21:19:18.107682 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Jul 16 21:19:18.107682 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 297, in do_request Jul 16 21:19:18.107682 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi self._handle_fault_response(status_code, replybody, resp) Jul 16 21:19:18.107682 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 197, in wrapper Jul 16 21:19:18.107682 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Jul 16 21:19:18.107682 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 272, in _handle_fault_response Jul 16 21:19:18.107682 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi exception_handler_v20(status_code, error_body) Jul 16 21:19:18.107682 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 90, in exception_handler_v20 Jul 16 21:19:18.107682 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise client_exc(message=error_message, Jul 16 21:19:18.107682 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi neutronclient.common.exceptions.NotFound: Security group 89aaff03-f3d3-48f7-965b-66c2a37029e0 does not exist Jul 16 21:19:18.111976 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-91c71876-afeb-4537-9cb9-74dde240eee9'] Jul 16 21:19:18.111976 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:19:18.111976 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:19:18.111976 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:19:18.111976 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:19:18.111976 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/security_groups.py", line 178, in delete Jul 16 21:19:18.111976 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi security_group = security_group_api.get(context, id) Jul 16 21:19:18.111976 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/security_group_api.py", line 314, in get Jul 16 21:19:18.111976 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.SecurityGroupNotFound(str(e)) Jul 16 21:19:18.111976 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.SecurityGroupNotFound: Security group 89aaff03-f3d3-48f7-965b-66c2a37029e0 does not exist Jul 16 21:19:18.111976 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-91c71876-afeb-4537-9cb9-74dde240eee9'] Jul 16 21:19:18.111976 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:19:18.111976 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:19:18.111976 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:19:18.111976 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:19:18.111976 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:19:18.111976 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:19:18.111976 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:19:18.111976 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:19:18.111976 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Jul 16 21:19:18.111976 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Jul 16 21:19:18.111976 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:19:18.111976 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:19:18.111976 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/security_groups.py", line 181, in delete Jul 16 21:19:18.111976 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=exp.format_message()) Jul 16 21:19:18.111976 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Security group 89aaff03-f3d3-48f7-965b-66c2a37029e0 does not exist Jul 16 21:19:18.111976 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-91c71876-afeb-4537-9cb9-74dde240eee9'] Jul 16 21:19:18.111976 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:19:18.111976 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:19:18.111976 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:19:18.111976 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:19:18.111976 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:19:18.111976 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:19:18.111976 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:19:18.111976 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:19:18.111976 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Security group 89aaff03-f3d3-48f7-965b-66c2a37029e0 does not exist Jul 16 21:19:18.111976 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-91c71876-afeb-4537-9cb9-74dde240eee9'] Jul 16 21:19:18.111976 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:19:18.114610 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-9d69ec4f-3fa0-4f5f-97f5-7f3b741186e3 tempest-SecurityGroupRulesNegativeTestJSON-1399499068 tempest-SecurityGroupRulesNegativeTestJSON-1399499068-project] 10.222.0.16 "POST /compute/v2.1/os-security-groups" status: 200 len: 254 microversion: 2.1 time: 0.594641 Jul 16 21:19:18.115916 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-b5357b06-d89c-4a8e-ac02-d08a9fd24211 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Returning 404 to user: Security group 89aaff03-f3d3-48f7-965b-66c2a37029e0 does not exist Jul 16 21:19:18.115916 ncdv-master-1 devstack@n-api.service[138681]: Neutron server returns request_ids: ['req-91c71876-afeb-4537-9cb9-74dde240eee9'] {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:19:18.116219 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1336/2638] 10.222.0.16 () {60 vars in 1179 bytes} [Sat Jul 16 21:19:17 2022] POST /compute/v2.1/os-security-groups => generated 254 bytes in 602 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:19:18.117760 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-b5357b06-d89c-4a8e-ac02-d08a9fd24211 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] 10.222.0.16 "DELETE /compute/v2.1/os-security-groups/89aaff03-f3d3-48f7-965b-66c2a37029e0" status: 404 len: 194 microversion: 2.1 time: 0.053531 Jul 16 21:19:18.118986 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1303/2639] 10.222.0.16 () {58 vars in 1271 bytes} [Sat Jul 16 21:19:18 2022] DELETE /compute/v2.1/os-security-groups/89aaff03-f3d3-48f7-965b-66c2a37029e0 => generated 194 bytes in 56 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:19:18.121270 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-4f76f9e9-6b05-4870-be7d-f2071464f279 tempest-ServersAaction247Test-725202529 tempest-ServersAaction247Test-725202529-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:19:18.121898 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-4f76f9e9-6b05-4870-be7d-f2071464f279 tempest-ServersAaction247Test-725202529 tempest-ServersAaction247Test-725202529-project] [instance: a51adaf1-9c26-47d0-92c2-b6bb065a5ee8] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:19:18.125755 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-29720421-c7cb-4851-9533-0315b9f181f5 tempest-SecurityGroupRulesNegativeTestJSON-1399499068 tempest-SecurityGroupRulesNegativeTestJSON-1399499068-project] Action: 'create', calling method: .version_select at 0x7fdd9d090b80>, body: {"security_group_rule": {"parent_group_id": "171e329c-c428-4558-abb4-cf001d25e916", "ip_protocol": "tcp", "from_port": 22, "to_port": 21}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:19:18.136910 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4f76f9e9-6b05-4870-be7d-f2071464f279 tempest-ServersAaction247Test-725202529 tempest-ServersAaction247Test-725202529-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:18.137373 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4f76f9e9-6b05-4870-be7d-f2071464f279 tempest-ServersAaction247Test-725202529 tempest-ServersAaction247Test-725202529-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:18.138160 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4f76f9e9-6b05-4870-be7d-f2071464f279 tempest-ServersAaction247Test-725202529 tempest-ServersAaction247Test-725202529-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:18.138896 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4f76f9e9-6b05-4870-be7d-f2071464f279 tempest-ServersAaction247Test-725202529 tempest-ServersAaction247Test-725202529-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:18.232998 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-4f76f9e9-6b05-4870-be7d-f2071464f279 tempest-ServersAaction247Test-725202529 tempest-ServersAaction247Test-725202529-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a8d48b52db1545a4a51027a5c2c7fcce', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f2984288b4cd4610b2f32e895f3e5d72', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:19:18.242882 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-4f76f9e9-6b05-4870-be7d-f2071464f279 tempest-ServersAaction247Test-725202529 tempest-ServersAaction247Test-725202529-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'a8d48b52db1545a4a51027a5c2c7fcce', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f2984288b4cd4610b2f32e895f3e5d72', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:19:18.244852 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-4f76f9e9-6b05-4870-be7d-f2071464f279 tempest-ServersAaction247Test-725202529 tempest-ServersAaction247Test-725202529-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'a8d48b52db1545a4a51027a5c2c7fcce', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f2984288b4cd4610b2f32e895f3e5d72', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:19:18.246069 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-4f76f9e9-6b05-4870-be7d-f2071464f279 tempest-ServersAaction247Test-725202529 tempest-ServersAaction247Test-725202529-project] 10.222.0.16 "GET /compute/v2.1/servers/a51adaf1-9c26-47d0-92c2-b6bb065a5ee8" status: 200 len: 1360 microversion: 2.47 time: 0.129053 Jul 16 21:19:18.246753 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1337/2640] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:19:18 2022] GET /compute/v2.1/servers/a51adaf1-9c26-47d0-92c2-b6bb065a5ee8 => generated 1360 bytes in 130 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:19:18.255895 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-2586b367-5891-4657-a30f-3d4798f5d160 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Calling method '.version_select at 0x7fdda2b02700>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:19:18.276908 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-29720421-c7cb-4851-9533-0315b9f181f5 tempest-SecurityGroupRulesNegativeTestJSON-1399499068 tempest-SecurityGroupRulesNegativeTestJSON-1399499068-project] HTTP exception thrown: Invalid port range 22:21. Former value cannot be greater than the later Jul 16 21:19:18.278165 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-29720421-c7cb-4851-9533-0315b9f181f5 tempest-SecurityGroupRulesNegativeTestJSON-1399499068 tempest-SecurityGroupRulesNegativeTestJSON-1399499068-project] Request method failure captured: Jul 16 21:19:18.278165 ncdv-master-1 devstack@n-api.service[138681]: request: POST /compute/v2.1/os-security-group-rules HTTP/1.1 Jul 16 21:19:18.278165 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:19:18.278165 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:19:18.278165 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:19:18.278165 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 138 Jul 16 21:19:18.278165 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:19:18.278165 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:19:18.278165 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:19:18.278165 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:19:18.278165 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0ytNkmtXKapUZpFKPGUA8hf3ATbduJ0XtzorjW7amdLo0WS0Qbkqbi3B3wBROLLwxfXJZxCXXmOW4by85qgjbrIW-gGZ2c_YrmiVSX-WID5VqAWhi2CmsmYLeeu_6bw9QJDULh6nweMasTNNM3cnBfI9QUQLgKaJIeA4QYZkK1W0reQ Jul 16 21:19:18.278165 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:19:18.278165 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:19:18.278165 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:19:18.278165 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:19:18.278165 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:19:18.278165 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:19:18.278165 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: 89b4d12252b94d54b50cc15238af6c2f Jul 16 21:19:18.278165 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-SecurityGroupRulesNegativeTestJSON-1399499068 Jul 16 21:19:18.278165 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:19:18.278165 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:19:18.278165 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/89b4d12252b94d54b50cc15238af6c2f"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/89b4d12252b94d54b50cc15238af6c2f"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_89b4d12252b94d54b50cc15238af6c2f"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/89b4d12252b94d54b50cc15238af6c2f"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:19:18.278165 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-SecurityGroupRulesNegativeTestJSON-1399499068 Jul 16 21:19:18.278165 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: 89b4d12252b94d54b50cc15238af6c2f Jul 16 21:19:18.278165 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-SecurityGroupRulesNegativeTestJSON-1399499068 Jul 16 21:19:18.278165 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-SecurityGroupRulesNegativeTestJSON-1399499068-project Jul 16 21:19:18.278165 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:19:18.278165 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:19:18.278165 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: 7ea30eb37fbf469194845261a19e8d4f Jul 16 21:19:18.278165 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-SecurityGroupRulesNegativeTestJSON-1399499068-project Jul 16 21:19:18.278165 ncdv-master-1 devstack@n-api.service[138681]: Jul 16 21:19:18.278165 ncdv-master-1 devstack@n-api.service[138681]: {"security_group_rule": {"parent_group_id": "171e329c-c428-4558-abb4-cf001d25e916", "ip_protocol": "tcp", "from_port": 22, "to_port": 21}} Jul 16 21:19:18.278165 ncdv-master-1 devstack@n-api.service[138681]: method: Jul 16 21:19:18.278165 ncdv-master-1 devstack@n-api.service[138681]: exception: Invalid port range 22:21. Former value cannot be greater than the later Jul 16 21:19:18.278165 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'body': {'security_group_rule': {'parent_group_id': '171e329c-c428-4558-abb4-cf001d25e916', 'ip_protocol': 'tcp', 'from_port': 22, 'to_port': 21}}} Jul 16 21:19:18.278165 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:19:18.278165 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:19:18.278165 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/security_groups.py", line 294, in create Jul 16 21:19:18.278165 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi new_rule = self._rule_args_to_dict(context, Jul 16 21:19:18.278165 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/security_groups.py", line 346, in _rule_args_to_dict Jul 16 21:19:18.278165 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return security_group_api.new_cidr_ingress_rule( Jul 16 21:19:18.278165 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/security_group_api.py", line 103, in new_cidr_ingress_rule Jul 16 21:19:18.278165 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return _new_ingress_rule( Jul 16 21:19:18.278165 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/security_group_api.py", line 150, in _new_ingress_rule Jul 16 21:19:18.278165 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.InvalidPortRange(from_port=from_port, Jul 16 21:19:18.278165 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.InvalidPortRange: Invalid port range 22:21. Former value cannot be greater than the later Jul 16 21:19:18.278165 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:19:18.278165 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:19:18.278165 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:19:18.278165 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:19:18.278165 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:19:18.278165 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:19:18.278165 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:19:18.278165 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:19:18.278165 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Jul 16 21:19:18.278165 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Jul 16 21:19:18.278165 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:19:18.278165 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:19:18.278165 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/security_groups.py", line 301, in create Jul 16 21:19:18.278165 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=exp.format_message()) Jul 16 21:19:18.278165 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid port range 22:21. Former value cannot be greater than the later Jul 16 21:19:18.278165 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:19:18.288306 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:19:18.288306 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:19:18.288306 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:19:18.288306 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:19:18.288306 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:19:18.288306 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:19:18.288306 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:19:18.288306 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid port range 22:21. Former value cannot be greater than the later Jul 16 21:19:18.288306 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:19:18.288306 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-29720421-c7cb-4851-9533-0315b9f181f5 tempest-SecurityGroupRulesNegativeTestJSON-1399499068 tempest-SecurityGroupRulesNegativeTestJSON-1399499068-project] Returning 400 to user: Invalid port range 22:21. Former value cannot be greater than the later {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:19:18.288306 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-29720421-c7cb-4851-9533-0315b9f181f5 tempest-SecurityGroupRulesNegativeTestJSON-1399499068 tempest-SecurityGroupRulesNegativeTestJSON-1399499068-project] 10.222.0.16 "POST /compute/v2.1/os-security-group-rules" status: 400 len: 115 microversion: 2.1 time: 0.159279 Jul 16 21:19:18.288306 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1304/2641] 10.222.0.16 () {60 vars in 1194 bytes} [Sat Jul 16 21:19:18 2022] POST /compute/v2.1/os-security-group-rules => generated 115 bytes in 160 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Jul 16 21:19:18.290706 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-0a024f10-3f7f-427c-8a97-22d65979504c tempest-ServersAaction247Test-725202529 tempest-ServersAaction247Test-725202529-project] Action: 'action', calling method: >, body: {"createBackup": {"backup_type": "daily", "rotation": 2, "name": "tempest-backup-1-126316777"}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:19:18.292101 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-0a024f10-3f7f-427c-8a97-22d65979504c tempest-ServersAaction247Test-725202529 tempest-ServersAaction247Test-725202529-project] [instance: a51adaf1-9c26-47d0-92c2-b6bb065a5ee8] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:19:18.300866 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0a024f10-3f7f-427c-8a97-22d65979504c tempest-ServersAaction247Test-725202529 tempest-ServersAaction247Test-725202529-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:18.301430 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0a024f10-3f7f-427c-8a97-22d65979504c tempest-ServersAaction247Test-725202529 tempest-ServersAaction247Test-725202529-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:18.303157 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0a024f10-3f7f-427c-8a97-22d65979504c tempest-ServersAaction247Test-725202529 tempest-ServersAaction247Test-725202529-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:18.303874 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0a024f10-3f7f-427c-8a97-22d65979504c tempest-ServersAaction247Test-725202529 tempest-ServersAaction247Test-725202529-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:18.323760 ncdv-master-1 devstack@n-api.service[138680]: DEBUG neutronclient.v2_0.client [None req-2586b367-5891-4657-a30f-3d4798f5d160 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Error message: {"NeutronError": {"type": "SecurityGroupNotFound", "message": "Security group 0dec7f11-e8ff-4171-a3ac-2aef7047a4b6 does not exist", "detail": ""}} {{(pid=138680) _handle_fault_response /usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py:262}} Jul 16 21:19:18.325408 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.network.security_group_api [None req-2586b367-5891-4657-a30f-3d4798f5d160 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Neutron security group 0dec7f11-e8ff-4171-a3ac-2aef7047a4b6 not found {{(pid=138680) get /opt/stack/nova/nova/network/security_group_api.py:313}} Jul 16 21:19:18.326826 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-2586b367-5891-4657-a30f-3d4798f5d160 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] HTTP exception thrown: Security group 0dec7f11-e8ff-4171-a3ac-2aef7047a4b6 does not exist Jul 16 21:19:18.326826 ncdv-master-1 devstack@n-api.service[138680]: Neutron server returns request_ids: ['req-87fbcc09-6599-46b5-bc1e-ddf7156caedf'] Jul 16 21:19:18.330361 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-2586b367-5891-4657-a30f-3d4798f5d160 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Request method failure captured: Jul 16 21:19:18.330361 ncdv-master-1 devstack@n-api.service[138680]: request: DELETE /compute/v2.1/os-security-groups/0dec7f11-e8ff-4171-a3ac-2aef7047a4b6 HTTP/1.1 Jul 16 21:19:18.330361 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:19:18.330361 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:19:18.330361 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:19:18.330361 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 0 Jul 16 21:19:18.330361 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:19:18.330361 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:19:18.330361 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:19:18.330361 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:19:18.330361 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0yrk-HsfYKsyovLSaEtNxaoXy7a3rLGNlsRPiPP76ZvvrKkKyGd-uekosXQxpFgkRnItBdviN4xr49Rerg5AyvLSzsyRY-Um0r3BRAvyZrzmCDYUpWbXk1uyEIb3Af4U3Y3nXIJgJAuDVx9z2NGSvEGCsGz12WlyyGCNfVSADFcYJP8 Jul 16 21:19:18.330361 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:19:18.330361 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:19:18.330361 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:19:18.330361 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:19:18.330361 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:19:18.330361 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:19:18.330361 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: e353258f9c3048ef84683294e2b8579d Jul 16 21:19:18.330361 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-SecurityGroupsTestJSON-438340169 Jul 16 21:19:18.330361 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:19:18.330361 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:19:18.330361 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/e353258f9c3048ef84683294e2b8579d"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/e353258f9c3048ef84683294e2b8579d"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_e353258f9c3048ef84683294e2b8579d"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/e353258f9c3048ef84683294e2b8579d"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:19:18.330361 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-SecurityGroupsTestJSON-438340169 Jul 16 21:19:18.330361 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: e353258f9c3048ef84683294e2b8579d Jul 16 21:19:18.330361 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-SecurityGroupsTestJSON-438340169 Jul 16 21:19:18.330361 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-SecurityGroupsTestJSON-438340169-project Jul 16 21:19:18.330361 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:19:18.330361 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:19:18.330361 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: d0008fffad8e4708b2c01e03da92c47d Jul 16 21:19:18.330361 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-SecurityGroupsTestJSON-438340169-project Jul 16 21:19:18.330361 ncdv-master-1 devstack@n-api.service[138680]: method: Jul 16 21:19:18.330361 ncdv-master-1 devstack@n-api.service[138680]: exception: Security group 0dec7f11-e8ff-4171-a3ac-2aef7047a4b6 does not exist Jul 16 21:19:18.330361 ncdv-master-1 devstack@n-api.service[138680]: Neutron server returns request_ids: ['req-87fbcc09-6599-46b5-bc1e-ddf7156caedf'] Jul 16 21:19:18.330361 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': '0dec7f11-e8ff-4171-a3ac-2aef7047a4b6'} Jul 16 21:19:18.330361 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:19:18.330361 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:19:18.330361 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/security_group_api.py", line 309, in get Jul 16 21:19:18.330361 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi group = neutron.show_security_group(id).get('security_group') Jul 16 21:19:18.330361 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 197, in wrapper Jul 16 21:19:18.330361 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Jul 16 21:19:18.330361 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 1040, in show_security_group Jul 16 21:19:18.330361 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return self.get(self.security_group_path % (security_group), Jul 16 21:19:18.330361 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 197, in wrapper Jul 16 21:19:18.330361 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Jul 16 21:19:18.330361 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 356, in get Jul 16 21:19:18.330361 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return self.retry_request("GET", action, body=body, Jul 16 21:19:18.330361 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 197, in wrapper Jul 16 21:19:18.330361 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Jul 16 21:19:18.330361 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 333, in retry_request Jul 16 21:19:18.330361 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return self.do_request(method, action, body=body, Jul 16 21:19:18.330361 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 197, in wrapper Jul 16 21:19:18.330361 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Jul 16 21:19:18.330361 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 297, in do_request Jul 16 21:19:18.330361 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi self._handle_fault_response(status_code, replybody, resp) Jul 16 21:19:18.330361 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 197, in wrapper Jul 16 21:19:18.330361 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Jul 16 21:19:18.330361 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 272, in _handle_fault_response Jul 16 21:19:18.330361 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi exception_handler_v20(status_code, error_body) Jul 16 21:19:18.330361 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 90, in exception_handler_v20 Jul 16 21:19:18.330361 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise client_exc(message=error_message, Jul 16 21:19:18.330361 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi neutronclient.common.exceptions.NotFound: Security group 0dec7f11-e8ff-4171-a3ac-2aef7047a4b6 does not exist Jul 16 21:19:18.335113 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-87fbcc09-6599-46b5-bc1e-ddf7156caedf'] Jul 16 21:19:18.335113 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:19:18.335113 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:19:18.335113 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:19:18.335113 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:19:18.335113 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/security_groups.py", line 178, in delete Jul 16 21:19:18.335113 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi security_group = security_group_api.get(context, id) Jul 16 21:19:18.335113 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/security_group_api.py", line 314, in get Jul 16 21:19:18.335113 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.SecurityGroupNotFound(str(e)) Jul 16 21:19:18.335113 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.SecurityGroupNotFound: Security group 0dec7f11-e8ff-4171-a3ac-2aef7047a4b6 does not exist Jul 16 21:19:18.335113 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-87fbcc09-6599-46b5-bc1e-ddf7156caedf'] Jul 16 21:19:18.335113 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:19:18.335113 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:19:18.335113 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:19:18.335113 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:19:18.335113 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:19:18.335113 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:19:18.335113 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:19:18.335113 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:19:18.335113 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Jul 16 21:19:18.335113 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Jul 16 21:19:18.335113 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:19:18.335113 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:19:18.335113 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/security_groups.py", line 181, in delete Jul 16 21:19:18.335113 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=exp.format_message()) Jul 16 21:19:18.335113 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Security group 0dec7f11-e8ff-4171-a3ac-2aef7047a4b6 does not exist Jul 16 21:19:18.335113 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-87fbcc09-6599-46b5-bc1e-ddf7156caedf'] Jul 16 21:19:18.335113 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:19:18.335113 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:19:18.335113 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:19:18.335113 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:19:18.335113 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:19:18.335113 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:19:18.335113 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:19:18.335113 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:19:18.335113 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Security group 0dec7f11-e8ff-4171-a3ac-2aef7047a4b6 does not exist Jul 16 21:19:18.335113 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-87fbcc09-6599-46b5-bc1e-ddf7156caedf'] Jul 16 21:19:18.335113 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:19:18.340735 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-2586b367-5891-4657-a30f-3d4798f5d160 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Returning 404 to user: Security group 0dec7f11-e8ff-4171-a3ac-2aef7047a4b6 does not exist Jul 16 21:19:18.340735 ncdv-master-1 devstack@n-api.service[138680]: Neutron server returns request_ids: ['req-87fbcc09-6599-46b5-bc1e-ddf7156caedf'] {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:19:18.341902 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-2586b367-5891-4657-a30f-3d4798f5d160 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] 10.222.0.16 "DELETE /compute/v2.1/os-security-groups/0dec7f11-e8ff-4171-a3ac-2aef7047a4b6" status: 404 len: 194 microversion: 2.1 time: 0.094021 Jul 16 21:19:18.343497 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1338/2642] 10.222.0.16 () {58 vars in 1271 bytes} [Sat Jul 16 21:19:18 2022] DELETE /compute/v2.1/os-security-groups/0dec7f11-e8ff-4171-a3ac-2aef7047a4b6 => generated 194 bytes in 96 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:19:18.352256 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-37bc57a9-ebf6-4f66-9d60-f13d857e9019 tempest-SecurityGroupRulesNegativeTestJSON-1399499068 tempest-SecurityGroupRulesNegativeTestJSON-1399499068-project] Action: 'create', calling method: .version_select at 0x7fdd9cc35940>, body: {"security_group": {"name": "tempest-SecurityGroupRulesNegativeTestJSON-securitygroup-1255471776", "description": "tempest-description-812304793"}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:19:18.861602 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-37bc57a9-ebf6-4f66-9d60-f13d857e9019 tempest-SecurityGroupRulesNegativeTestJSON-1399499068 tempest-SecurityGroupRulesNegativeTestJSON-1399499068-project] 10.222.0.16 "POST /compute/v2.1/os-security-groups" status: 200 len: 255 microversion: 2.1 time: 0.515573 Jul 16 21:19:18.869939 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1339/2643] 10.222.0.16 () {60 vars in 1179 bytes} [Sat Jul 16 21:19:18 2022] POST /compute/v2.1/os-security-groups => generated 255 bytes in 526 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:19:18.881374 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-738b1680-0ac0-46ef-bff0-62cc6c73049e tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Calling method '.version_select at 0x7fdd9cc35550>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:19:18.922337 ncdv-master-1 devstack@n-api.service[138680]: DEBUG neutronclient.v2_0.client [None req-738b1680-0ac0-46ef-bff0-62cc6c73049e tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Error message: {"NeutronError": {"type": "SecurityGroupNotFound", "message": "Security group 171f6d9b-ebb3-4079-a770-ac1c15c2977a does not exist", "detail": ""}} {{(pid=138680) _handle_fault_response /usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py:262}} Jul 16 21:19:18.924502 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.network.security_group_api [None req-738b1680-0ac0-46ef-bff0-62cc6c73049e tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Neutron security group 171f6d9b-ebb3-4079-a770-ac1c15c2977a not found {{(pid=138680) get /opt/stack/nova/nova/network/security_group_api.py:313}} Jul 16 21:19:18.925046 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-738b1680-0ac0-46ef-bff0-62cc6c73049e tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] HTTP exception thrown: Security group 171f6d9b-ebb3-4079-a770-ac1c15c2977a does not exist Jul 16 21:19:18.925046 ncdv-master-1 devstack@n-api.service[138680]: Neutron server returns request_ids: ['req-3d09404e-bae7-4537-8d52-4ce340595df7'] Jul 16 21:19:18.928130 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-738b1680-0ac0-46ef-bff0-62cc6c73049e tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Request method failure captured: Jul 16 21:19:18.928130 ncdv-master-1 devstack@n-api.service[138680]: request: DELETE /compute/v2.1/os-security-groups/171f6d9b-ebb3-4079-a770-ac1c15c2977a HTTP/1.1 Jul 16 21:19:18.928130 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:19:18.928130 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:19:18.928130 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:19:18.928130 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 0 Jul 16 21:19:18.928130 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:19:18.928130 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:19:18.928130 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:19:18.928130 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:19:18.928130 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0yrk-HsfYKsyovLSaEtNxaoXy7a3rLGNlsRPiPP76ZvvrKkKyGd-uekosXQxpFgkRnItBdviN4xr49Rerg5AyvLSzsyRY-Um0r3BRAvyZrzmCDYUpWbXk1uyEIb3Af4U3Y3nXIJgJAuDVx9z2NGSvEGCsGz12WlyyGCNfVSADFcYJP8 Jul 16 21:19:18.928130 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:19:18.928130 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:19:18.928130 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:19:18.928130 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:19:18.928130 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:19:18.928130 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:19:18.928130 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: e353258f9c3048ef84683294e2b8579d Jul 16 21:19:18.928130 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-SecurityGroupsTestJSON-438340169 Jul 16 21:19:18.928130 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:19:18.928130 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:19:18.928130 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/e353258f9c3048ef84683294e2b8579d"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/e353258f9c3048ef84683294e2b8579d"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_e353258f9c3048ef84683294e2b8579d"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/e353258f9c3048ef84683294e2b8579d"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:19:18.928130 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-SecurityGroupsTestJSON-438340169 Jul 16 21:19:18.928130 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: e353258f9c3048ef84683294e2b8579d Jul 16 21:19:18.928130 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-SecurityGroupsTestJSON-438340169 Jul 16 21:19:18.928130 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-SecurityGroupsTestJSON-438340169-project Jul 16 21:19:18.928130 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:19:18.928130 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:19:18.928130 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: d0008fffad8e4708b2c01e03da92c47d Jul 16 21:19:18.928130 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-SecurityGroupsTestJSON-438340169-project Jul 16 21:19:18.928130 ncdv-master-1 devstack@n-api.service[138680]: method: Jul 16 21:19:18.928130 ncdv-master-1 devstack@n-api.service[138680]: exception: Security group 171f6d9b-ebb3-4079-a770-ac1c15c2977a does not exist Jul 16 21:19:18.928130 ncdv-master-1 devstack@n-api.service[138680]: Neutron server returns request_ids: ['req-3d09404e-bae7-4537-8d52-4ce340595df7'] Jul 16 21:19:18.928130 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': '171f6d9b-ebb3-4079-a770-ac1c15c2977a'} Jul 16 21:19:18.928130 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:19:18.928130 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:19:18.928130 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/security_group_api.py", line 309, in get Jul 16 21:19:18.928130 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi group = neutron.show_security_group(id).get('security_group') Jul 16 21:19:18.928130 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 197, in wrapper Jul 16 21:19:18.928130 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Jul 16 21:19:18.928130 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 1040, in show_security_group Jul 16 21:19:18.928130 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return self.get(self.security_group_path % (security_group), Jul 16 21:19:18.928130 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 197, in wrapper Jul 16 21:19:18.928130 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Jul 16 21:19:18.928130 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 356, in get Jul 16 21:19:18.928130 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return self.retry_request("GET", action, body=body, Jul 16 21:19:18.928130 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 197, in wrapper Jul 16 21:19:18.928130 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Jul 16 21:19:18.928130 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 333, in retry_request Jul 16 21:19:18.928130 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return self.do_request(method, action, body=body, Jul 16 21:19:18.928130 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 197, in wrapper Jul 16 21:19:18.928130 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Jul 16 21:19:18.928130 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 297, in do_request Jul 16 21:19:18.928130 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi self._handle_fault_response(status_code, replybody, resp) Jul 16 21:19:18.928130 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 197, in wrapper Jul 16 21:19:18.928130 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Jul 16 21:19:18.928130 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 272, in _handle_fault_response Jul 16 21:19:18.928130 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi exception_handler_v20(status_code, error_body) Jul 16 21:19:18.928130 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 90, in exception_handler_v20 Jul 16 21:19:18.928130 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise client_exc(message=error_message, Jul 16 21:19:18.928130 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi neutronclient.common.exceptions.NotFound: Security group 171f6d9b-ebb3-4079-a770-ac1c15c2977a does not exist Jul 16 21:19:18.933263 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-3d09404e-bae7-4537-8d52-4ce340595df7'] Jul 16 21:19:18.933263 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:19:18.933263 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:19:18.933263 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:19:18.933263 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:19:18.933263 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/security_groups.py", line 178, in delete Jul 16 21:19:18.933263 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi security_group = security_group_api.get(context, id) Jul 16 21:19:18.933263 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/security_group_api.py", line 314, in get Jul 16 21:19:18.933263 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.SecurityGroupNotFound(str(e)) Jul 16 21:19:18.933263 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.SecurityGroupNotFound: Security group 171f6d9b-ebb3-4079-a770-ac1c15c2977a does not exist Jul 16 21:19:18.933263 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-3d09404e-bae7-4537-8d52-4ce340595df7'] Jul 16 21:19:18.933263 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:19:18.933263 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:19:18.933263 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:19:18.933263 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:19:18.933263 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:19:18.933263 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:19:18.933263 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:19:18.933263 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:19:18.933263 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Jul 16 21:19:18.933263 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Jul 16 21:19:18.933263 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:19:18.933263 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:19:18.933263 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/security_groups.py", line 181, in delete Jul 16 21:19:18.933263 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=exp.format_message()) Jul 16 21:19:18.933263 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Security group 171f6d9b-ebb3-4079-a770-ac1c15c2977a does not exist Jul 16 21:19:18.933263 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-3d09404e-bae7-4537-8d52-4ce340595df7'] Jul 16 21:19:18.933263 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:19:18.933263 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:19:18.933263 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:19:18.933263 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:19:18.933263 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:19:18.933263 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:19:18.933263 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:19:18.933263 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:19:18.933263 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Security group 171f6d9b-ebb3-4079-a770-ac1c15c2977a does not exist Jul 16 21:19:18.933263 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-3d09404e-bae7-4537-8d52-4ce340595df7'] Jul 16 21:19:18.933263 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:19:18.933263 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-738b1680-0ac0-46ef-bff0-62cc6c73049e tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Returning 404 to user: Security group 171f6d9b-ebb3-4079-a770-ac1c15c2977a does not exist Jul 16 21:19:18.933263 ncdv-master-1 devstack@n-api.service[138680]: Neutron server returns request_ids: ['req-3d09404e-bae7-4537-8d52-4ce340595df7'] {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:19:18.933263 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-738b1680-0ac0-46ef-bff0-62cc6c73049e tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] 10.222.0.16 "DELETE /compute/v2.1/os-security-groups/171f6d9b-ebb3-4079-a770-ac1c15c2977a" status: 404 len: 194 microversion: 2.1 time: 0.060549 Jul 16 21:19:18.933263 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1340/2644] 10.222.0.16 () {58 vars in 1271 bytes} [Sat Jul 16 21:19:18 2022] DELETE /compute/v2.1/os-security-groups/171f6d9b-ebb3-4079-a770-ac1c15c2977a => generated 194 bytes in 62 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:19:18.940306 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-8532e46b-0229-4170-8287-6b8bb80d3b2f tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:19:18.940306 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-8532e46b-0229-4170-8287-6b8bb80d3b2f tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] [instance: c16e5f09-f813-414e-b233-c08348078624] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:19:18.948414 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-0a024f10-3f7f-427c-8a97-22d65979504c tempest-ServersAaction247Test-725202529 tempest-ServersAaction247Test-725202529-project] 10.222.0.16 "POST /compute/v2.1/servers/a51adaf1-9c26-47d0-92c2-b6bb065a5ee8/action" status: 202 len: 52 microversion: 2.47 time: 0.666066 Jul 16 21:19:18.951445 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8532e46b-0229-4170-8287-6b8bb80d3b2f tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:18.951923 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8532e46b-0229-4170-8287-6b8bb80d3b2f tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:18.953088 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8532e46b-0229-4170-8287-6b8bb80d3b2f tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:18.953540 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8532e46b-0229-4170-8287-6b8bb80d3b2f tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:18.955559 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1305/2645] 10.222.0.16 () {62 vars in 1318 bytes} [Sat Jul 16 21:19:18 2022] POST /compute/v2.1/servers/a51adaf1-9c26-47d0-92c2-b6bb065a5ee8/action => generated 52 bytes in 674 msecs (HTTP/1.1 202) 9 headers in 364 bytes (1 switches on core 0) Jul 16 21:19:18.961709 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-cd518d52-a243-4d1a-ba6e-d6539d5c2b63 tempest-SecurityGroupRulesNegativeTestJSON-1399499068 tempest-SecurityGroupRulesNegativeTestJSON-1399499068-project] Action: 'create', calling method: .version_select at 0x7fdd9cf0f700>, body: {"security_group_rule": {"parent_group_id": "386428d1-8e69-4752-85e2-9f71d5be2bb7", "ip_protocol": "tcp", "from_port": 22, "to_port": 984987791}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:19:19.107908 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-cd518d52-a243-4d1a-ba6e-d6539d5c2b63 tempest-SecurityGroupRulesNegativeTestJSON-1399499068 tempest-SecurityGroupRulesNegativeTestJSON-1399499068-project] HTTP exception thrown: Invalid port range 22:984987791. Valid TCP ports should be between 1-65535 Jul 16 21:19:19.115690 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-cd518d52-a243-4d1a-ba6e-d6539d5c2b63 tempest-SecurityGroupRulesNegativeTestJSON-1399499068 tempest-SecurityGroupRulesNegativeTestJSON-1399499068-project] Request method failure captured: Jul 16 21:19:19.115690 ncdv-master-1 devstack@n-api.service[138681]: request: POST /compute/v2.1/os-security-group-rules HTTP/1.1 Jul 16 21:19:19.115690 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:19:19.115690 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:19:19.115690 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:19:19.115690 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 145 Jul 16 21:19:19.115690 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:19:19.115690 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:19:19.115690 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:19:19.115690 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:19:19.115690 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0ytNkmtXKapUZpFKPGUA8hf3ATbduJ0XtzorjW7amdLo0WS0Qbkqbi3B3wBROLLwxfXJZxCXXmOW4by85qgjbrIW-gGZ2c_YrmiVSX-WID5VqAWhi2CmsmYLeeu_6bw9QJDULh6nweMasTNNM3cnBfI9QUQLgKaJIeA4QYZkK1W0reQ Jul 16 21:19:19.115690 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:19:19.115690 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:19:19.115690 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:19:19.115690 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:19:19.115690 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:19:19.115690 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:19:19.115690 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: 89b4d12252b94d54b50cc15238af6c2f Jul 16 21:19:19.115690 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-SecurityGroupRulesNegativeTestJSON-1399499068 Jul 16 21:19:19.115690 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:19:19.115690 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:19:19.115690 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/89b4d12252b94d54b50cc15238af6c2f"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/89b4d12252b94d54b50cc15238af6c2f"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_89b4d12252b94d54b50cc15238af6c2f"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/89b4d12252b94d54b50cc15238af6c2f"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:19:19.115690 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-SecurityGroupRulesNegativeTestJSON-1399499068 Jul 16 21:19:19.115690 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: 89b4d12252b94d54b50cc15238af6c2f Jul 16 21:19:19.115690 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-SecurityGroupRulesNegativeTestJSON-1399499068 Jul 16 21:19:19.115690 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-SecurityGroupRulesNegativeTestJSON-1399499068-project Jul 16 21:19:19.115690 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:19:19.115690 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:19:19.115690 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: 7ea30eb37fbf469194845261a19e8d4f Jul 16 21:19:19.115690 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-SecurityGroupRulesNegativeTestJSON-1399499068-project Jul 16 21:19:19.115690 ncdv-master-1 devstack@n-api.service[138681]: Jul 16 21:19:19.115690 ncdv-master-1 devstack@n-api.service[138681]: {"security_group_rule": {"parent_group_id": "386428d1-8e69-4752-85e2-9f71d5be2bb7", "ip_protocol": "tcp", "from_port": 22, "to_port": 984987791}} Jul 16 21:19:19.115690 ncdv-master-1 devstack@n-api.service[138681]: method: Jul 16 21:19:19.115690 ncdv-master-1 devstack@n-api.service[138681]: exception: Invalid port range 22:984987791. Valid TCP ports should be between 1-65535 Jul 16 21:19:19.115690 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'body': {'security_group_rule': {'parent_group_id': '386428d1-8e69-4752-85e2-9f71d5be2bb7', 'ip_protocol': 'tcp', 'from_port': 22, 'to_port': 984987791}}} Jul 16 21:19:19.115690 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:19:19.115690 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:19:19.115690 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/security_groups.py", line 294, in create Jul 16 21:19:19.115690 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi new_rule = self._rule_args_to_dict(context, Jul 16 21:19:19.115690 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/security_groups.py", line 346, in _rule_args_to_dict Jul 16 21:19:19.115690 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return security_group_api.new_cidr_ingress_rule( Jul 16 21:19:19.115690 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/security_group_api.py", line 103, in new_cidr_ingress_rule Jul 16 21:19:19.115690 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return _new_ingress_rule( Jul 16 21:19:19.115690 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/security_group_api.py", line 157, in _new_ingress_rule Jul 16 21:19:19.115690 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.InvalidPortRange(from_port=from_port, Jul 16 21:19:19.115690 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.InvalidPortRange: Invalid port range 22:984987791. Valid TCP ports should be between 1-65535 Jul 16 21:19:19.115690 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:19:19.115690 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:19:19.115690 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:19:19.115690 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:19:19.115690 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:19:19.115690 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:19:19.115690 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:19:19.115690 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:19:19.115690 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Jul 16 21:19:19.115690 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Jul 16 21:19:19.115690 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:19:19.115690 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:19:19.115690 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/security_groups.py", line 301, in create Jul 16 21:19:19.115690 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=exp.format_message()) Jul 16 21:19:19.115690 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid port range 22:984987791. Valid TCP ports should be between 1-65535 Jul 16 21:19:19.115690 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:19:19.120624 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:19:19.120624 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:19:19.120624 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:19:19.120624 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:19:19.120624 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:19:19.120624 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:19:19.120624 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:19:19.120624 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid port range 22:984987791. Valid TCP ports should be between 1-65535 Jul 16 21:19:19.120624 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:19:19.136532 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-cd518d52-a243-4d1a-ba6e-d6539d5c2b63 tempest-SecurityGroupRulesNegativeTestJSON-1399499068 tempest-SecurityGroupRulesNegativeTestJSON-1399499068-project] Returning 400 to user: Invalid port range 22:984987791. Valid TCP ports should be between 1-65535 {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:19:19.137965 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-cd518d52-a243-4d1a-ba6e-d6539d5c2b63 tempest-SecurityGroupRulesNegativeTestJSON-1399499068 tempest-SecurityGroupRulesNegativeTestJSON-1399499068-project] 10.222.0.16 "POST /compute/v2.1/os-security-group-rules" status: 400 len: 118 microversion: 2.1 time: 0.180935 Jul 16 21:19:19.142806 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1306/2646] 10.222.0.16 () {60 vars in 1194 bytes} [Sat Jul 16 21:19:18 2022] POST /compute/v2.1/os-security-group-rules => generated 118 bytes in 187 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Jul 16 21:19:19.153795 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-d253d33a-dc01-4f9f-81a0-7614351aa8f4 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:19:19.154734 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-d253d33a-dc01-4f9f-81a0-7614351aa8f4 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] [instance: e7598b4a-f70c-4186-88d1-6fd0057f8656] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:19:19.161786 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_api:SELECT=89,nova_api:INSERT=9,nova_api:UPDATE=2 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:19:19.170092 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d253d33a-dc01-4f9f-81a0-7614351aa8f4 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:19.170801 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d253d33a-dc01-4f9f-81a0-7614351aa8f4 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:19.172192 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d253d33a-dc01-4f9f-81a0-7614351aa8f4 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:19.172955 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d253d33a-dc01-4f9f-81a0-7614351aa8f4 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:19.222007 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-8532e46b-0229-4170-8287-6b8bb80d3b2f tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eebb224a56814f92b12ebebebc6bc8a3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a092ce9ef40943f6b6ace6cd28481b71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:19:19.237588 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-8532e46b-0229-4170-8287-6b8bb80d3b2f tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] 10.222.0.16 "GET /compute/v2.1/servers/c16e5f09-f813-414e-b233-c08348078624" status: 200 len: 1320 microversion: 2.1 time: 0.304000 Jul 16 21:19:19.239570 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1341/2647] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:19:18 2022] GET /compute/v2.1/servers/c16e5f09-f813-414e-b233-c08348078624 => generated 1320 bytes in 306 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:19:19.245944 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-d37aa4b6-280b-4e0d-a4b4-2f796b9c88e7 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Calling method '.version_select at 0x7fdd9d3ea280>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:19:19.302324 ncdv-master-1 devstack@n-api.service[138680]: DEBUG neutronclient.v2_0.client [None req-d37aa4b6-280b-4e0d-a4b4-2f796b9c88e7 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Error message: {"NeutronError": {"type": "SecurityGroupNotFound", "message": "Security group 5d9ea170-9999-4816-bc4f-070762e7181b does not exist", "detail": ""}} {{(pid=138680) _handle_fault_response /usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py:262}} Jul 16 21:19:19.302969 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.network.security_group_api [None req-d37aa4b6-280b-4e0d-a4b4-2f796b9c88e7 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Neutron security group 5d9ea170-9999-4816-bc4f-070762e7181b not found {{(pid=138680) get /opt/stack/nova/nova/network/security_group_api.py:313}} Jul 16 21:19:19.303703 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-d37aa4b6-280b-4e0d-a4b4-2f796b9c88e7 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] HTTP exception thrown: Security group 5d9ea170-9999-4816-bc4f-070762e7181b does not exist Jul 16 21:19:19.303703 ncdv-master-1 devstack@n-api.service[138680]: Neutron server returns request_ids: ['req-3479e75c-91e6-49bc-966e-ebe78c312428'] Jul 16 21:19:19.306611 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-d37aa4b6-280b-4e0d-a4b4-2f796b9c88e7 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Request method failure captured: Jul 16 21:19:19.306611 ncdv-master-1 devstack@n-api.service[138680]: request: DELETE /compute/v2.1/os-security-groups/5d9ea170-9999-4816-bc4f-070762e7181b HTTP/1.1 Jul 16 21:19:19.306611 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:19:19.306611 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:19:19.306611 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:19:19.306611 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 0 Jul 16 21:19:19.306611 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:19:19.306611 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:19:19.306611 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:19:19.306611 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:19:19.306611 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0yrk-HsfYKsyovLSaEtNxaoXy7a3rLGNlsRPiPP76ZvvrKkKyGd-uekosXQxpFgkRnItBdviN4xr49Rerg5AyvLSzsyRY-Um0r3BRAvyZrzmCDYUpWbXk1uyEIb3Af4U3Y3nXIJgJAuDVx9z2NGSvEGCsGz12WlyyGCNfVSADFcYJP8 Jul 16 21:19:19.306611 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:19:19.306611 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:19:19.306611 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:19:19.306611 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:19:19.306611 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:19:19.306611 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:19:19.306611 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: e353258f9c3048ef84683294e2b8579d Jul 16 21:19:19.306611 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-SecurityGroupsTestJSON-438340169 Jul 16 21:19:19.306611 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:19:19.306611 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:19:19.306611 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/e353258f9c3048ef84683294e2b8579d"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/e353258f9c3048ef84683294e2b8579d"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_e353258f9c3048ef84683294e2b8579d"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/e353258f9c3048ef84683294e2b8579d"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:19:19.306611 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-SecurityGroupsTestJSON-438340169 Jul 16 21:19:19.306611 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: e353258f9c3048ef84683294e2b8579d Jul 16 21:19:19.306611 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-SecurityGroupsTestJSON-438340169 Jul 16 21:19:19.306611 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-SecurityGroupsTestJSON-438340169-project Jul 16 21:19:19.306611 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:19:19.306611 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:19:19.306611 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: d0008fffad8e4708b2c01e03da92c47d Jul 16 21:19:19.306611 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-SecurityGroupsTestJSON-438340169-project Jul 16 21:19:19.306611 ncdv-master-1 devstack@n-api.service[138680]: method: Jul 16 21:19:19.306611 ncdv-master-1 devstack@n-api.service[138680]: exception: Security group 5d9ea170-9999-4816-bc4f-070762e7181b does not exist Jul 16 21:19:19.306611 ncdv-master-1 devstack@n-api.service[138680]: Neutron server returns request_ids: ['req-3479e75c-91e6-49bc-966e-ebe78c312428'] Jul 16 21:19:19.306611 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': '5d9ea170-9999-4816-bc4f-070762e7181b'} Jul 16 21:19:19.306611 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:19:19.306611 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:19:19.306611 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/security_group_api.py", line 309, in get Jul 16 21:19:19.306611 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi group = neutron.show_security_group(id).get('security_group') Jul 16 21:19:19.306611 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 197, in wrapper Jul 16 21:19:19.306611 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Jul 16 21:19:19.306611 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 1040, in show_security_group Jul 16 21:19:19.306611 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return self.get(self.security_group_path % (security_group), Jul 16 21:19:19.306611 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 197, in wrapper Jul 16 21:19:19.306611 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Jul 16 21:19:19.306611 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 356, in get Jul 16 21:19:19.306611 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return self.retry_request("GET", action, body=body, Jul 16 21:19:19.306611 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 197, in wrapper Jul 16 21:19:19.306611 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Jul 16 21:19:19.306611 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 333, in retry_request Jul 16 21:19:19.306611 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return self.do_request(method, action, body=body, Jul 16 21:19:19.306611 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 197, in wrapper Jul 16 21:19:19.306611 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Jul 16 21:19:19.306611 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 297, in do_request Jul 16 21:19:19.306611 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi self._handle_fault_response(status_code, replybody, resp) Jul 16 21:19:19.306611 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 197, in wrapper Jul 16 21:19:19.306611 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Jul 16 21:19:19.306611 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 272, in _handle_fault_response Jul 16 21:19:19.306611 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi exception_handler_v20(status_code, error_body) Jul 16 21:19:19.306611 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 90, in exception_handler_v20 Jul 16 21:19:19.306611 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise client_exc(message=error_message, Jul 16 21:19:19.306611 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi neutronclient.common.exceptions.NotFound: Security group 5d9ea170-9999-4816-bc4f-070762e7181b does not exist Jul 16 21:19:19.310062 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-3479e75c-91e6-49bc-966e-ebe78c312428'] Jul 16 21:19:19.310062 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:19:19.310062 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:19:19.310062 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:19:19.310062 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:19:19.310062 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/security_groups.py", line 178, in delete Jul 16 21:19:19.310062 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi security_group = security_group_api.get(context, id) Jul 16 21:19:19.310062 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/security_group_api.py", line 314, in get Jul 16 21:19:19.310062 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.SecurityGroupNotFound(str(e)) Jul 16 21:19:19.310062 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.SecurityGroupNotFound: Security group 5d9ea170-9999-4816-bc4f-070762e7181b does not exist Jul 16 21:19:19.310062 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-3479e75c-91e6-49bc-966e-ebe78c312428'] Jul 16 21:19:19.310062 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:19:19.310062 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:19:19.310062 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:19:19.310062 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:19:19.310062 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:19:19.310062 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:19:19.310062 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:19:19.310062 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:19:19.310062 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Jul 16 21:19:19.310062 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Jul 16 21:19:19.310062 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:19:19.310062 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:19:19.310062 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/security_groups.py", line 181, in delete Jul 16 21:19:19.310062 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=exp.format_message()) Jul 16 21:19:19.310062 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Security group 5d9ea170-9999-4816-bc4f-070762e7181b does not exist Jul 16 21:19:19.310062 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-3479e75c-91e6-49bc-966e-ebe78c312428'] Jul 16 21:19:19.310062 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:19:19.310062 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:19:19.310062 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:19:19.310062 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:19:19.310062 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:19:19.310062 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:19:19.310062 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:19:19.310062 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:19:19.310062 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Security group 5d9ea170-9999-4816-bc4f-070762e7181b does not exist Jul 16 21:19:19.310062 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-3479e75c-91e6-49bc-966e-ebe78c312428'] Jul 16 21:19:19.310062 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:19:19.310062 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-d37aa4b6-280b-4e0d-a4b4-2f796b9c88e7 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Returning 404 to user: Security group 5d9ea170-9999-4816-bc4f-070762e7181b does not exist Jul 16 21:19:19.310062 ncdv-master-1 devstack@n-api.service[138680]: Neutron server returns request_ids: ['req-3479e75c-91e6-49bc-966e-ebe78c312428'] {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:19:19.310062 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-d37aa4b6-280b-4e0d-a4b4-2f796b9c88e7 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] 10.222.0.16 "DELETE /compute/v2.1/os-security-groups/5d9ea170-9999-4816-bc4f-070762e7181b" status: 404 len: 194 microversion: 2.1 time: 0.067261 Jul 16 21:19:19.310062 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1342/2648] 10.222.0.16 () {58 vars in 1271 bytes} [Sat Jul 16 21:19:19 2022] DELETE /compute/v2.1/os-security-groups/5d9ea170-9999-4816-bc4f-070762e7181b => generated 194 bytes in 68 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:19:19.318215 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-00f4ca1f-2e64-4c69-85f4-5e18cba8390b tempest-ServersAaction247Test-725202529 tempest-ServersAaction247Test-725202529-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:19:19.318215 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-00f4ca1f-2e64-4c69-85f4-5e18cba8390b tempest-ServersAaction247Test-725202529 tempest-ServersAaction247Test-725202529-project] [instance: a51adaf1-9c26-47d0-92c2-b6bb065a5ee8] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:19:19.324096 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-00f4ca1f-2e64-4c69-85f4-5e18cba8390b tempest-ServersAaction247Test-725202529 tempest-ServersAaction247Test-725202529-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:19.324554 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-00f4ca1f-2e64-4c69-85f4-5e18cba8390b tempest-ServersAaction247Test-725202529 tempest-ServersAaction247Test-725202529-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:19.325338 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-00f4ca1f-2e64-4c69-85f4-5e18cba8390b tempest-ServersAaction247Test-725202529 tempest-ServersAaction247Test-725202529-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:19.325984 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-00f4ca1f-2e64-4c69-85f4-5e18cba8390b tempest-ServersAaction247Test-725202529 tempest-ServersAaction247Test-725202529-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:19.358342 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-00f4ca1f-2e64-4c69-85f4-5e18cba8390b tempest-ServersAaction247Test-725202529 tempest-ServersAaction247Test-725202529-project] [instance: a51adaf1-9c26-47d0-92c2-b6bb065a5ee8] Going to try to terminate instance {{(pid=138680) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 16 21:19:19.403826 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-d253d33a-dc01-4f9f-81a0-7614351aa8f4 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f5bc1f2209f4eb186d80e385b67c774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '65d5bcfa02af4e8284ba1342197e2860', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:19:19.412672 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-d253d33a-dc01-4f9f-81a0-7614351aa8f4 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] 10.222.0.16 "GET /compute/v2.1/servers/e7598b4a-f70c-4186-88d1-6fd0057f8656" status: 200 len: 1472 microversion: 2.1 time: 0.268329 Jul 16 21:19:19.413462 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1307/2649] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:19:19 2022] GET /compute/v2.1/servers/e7598b4a-f70c-4186-88d1-6fd0057f8656 => generated 1472 bytes in 270 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:19:19.419450 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-b228e305-bbbc-476e-b63f-51b11a4a4650 tempest-SecurityGroupRulesNegativeTestJSON-1399499068 tempest-SecurityGroupRulesNegativeTestJSON-1399499068-project] Action: 'create', calling method: .version_select at 0x7fdd9cf0ff70>, body: {"security_group_rule": {"parent_group_id": "0f4ae75e-40bf-4aa1-81b2-6540e1aec4e0", "ip_protocol": "tcp", "from_port": 22, "to_port": 22}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:19:19.442534 ncdv-master-1 devstack@n-api.service[138681]: DEBUG neutronclient.v2_0.client [None req-b228e305-bbbc-476e-b63f-51b11a4a4650 tempest-SecurityGroupRulesNegativeTestJSON-1399499068 tempest-SecurityGroupRulesNegativeTestJSON-1399499068-project] Error message: {"NeutronError": {"type": "SecurityGroupNotFound", "message": "Security group 0f4ae75e-40bf-4aa1-81b2-6540e1aec4e0 does not exist", "detail": ""}} {{(pid=138681) _handle_fault_response /usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py:262}} Jul 16 21:19:19.443368 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.network.security_group_api [None req-b228e305-bbbc-476e-b63f-51b11a4a4650 tempest-SecurityGroupRulesNegativeTestJSON-1399499068 tempest-SecurityGroupRulesNegativeTestJSON-1399499068-project] Neutron security group 0f4ae75e-40bf-4aa1-81b2-6540e1aec4e0 not found {{(pid=138681) get /opt/stack/nova/nova/network/security_group_api.py:313}} Jul 16 21:19:19.443883 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-b228e305-bbbc-476e-b63f-51b11a4a4650 tempest-SecurityGroupRulesNegativeTestJSON-1399499068 tempest-SecurityGroupRulesNegativeTestJSON-1399499068-project] HTTP exception thrown: Security group 0f4ae75e-40bf-4aa1-81b2-6540e1aec4e0 does not exist Jul 16 21:19:19.443883 ncdv-master-1 devstack@n-api.service[138681]: Neutron server returns request_ids: ['req-cae3e2a5-6a3e-4561-bff1-5081a9214f18'] Jul 16 21:19:19.445906 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-b228e305-bbbc-476e-b63f-51b11a4a4650 tempest-SecurityGroupRulesNegativeTestJSON-1399499068 tempest-SecurityGroupRulesNegativeTestJSON-1399499068-project] Request method failure captured: Jul 16 21:19:19.445906 ncdv-master-1 devstack@n-api.service[138681]: request: POST /compute/v2.1/os-security-group-rules HTTP/1.1 Jul 16 21:19:19.445906 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:19:19.445906 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:19:19.445906 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:19:19.445906 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 138 Jul 16 21:19:19.445906 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:19:19.445906 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:19:19.445906 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:19:19.445906 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:19:19.445906 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0ytNkmtXKapUZpFKPGUA8hf3ATbduJ0XtzorjW7amdLo0WS0Qbkqbi3B3wBROLLwxfXJZxCXXmOW4by85qgjbrIW-gGZ2c_YrmiVSX-WID5VqAWhi2CmsmYLeeu_6bw9QJDULh6nweMasTNNM3cnBfI9QUQLgKaJIeA4QYZkK1W0reQ Jul 16 21:19:19.445906 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:19:19.445906 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:19:19.445906 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:19:19.445906 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:19:19.445906 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:19:19.445906 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:19:19.445906 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: 89b4d12252b94d54b50cc15238af6c2f Jul 16 21:19:19.445906 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-SecurityGroupRulesNegativeTestJSON-1399499068 Jul 16 21:19:19.445906 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:19:19.445906 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:19:19.445906 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/89b4d12252b94d54b50cc15238af6c2f"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/89b4d12252b94d54b50cc15238af6c2f"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_89b4d12252b94d54b50cc15238af6c2f"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/89b4d12252b94d54b50cc15238af6c2f"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:19:19.445906 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-SecurityGroupRulesNegativeTestJSON-1399499068 Jul 16 21:19:19.445906 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: 89b4d12252b94d54b50cc15238af6c2f Jul 16 21:19:19.445906 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-SecurityGroupRulesNegativeTestJSON-1399499068 Jul 16 21:19:19.445906 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-SecurityGroupRulesNegativeTestJSON-1399499068-project Jul 16 21:19:19.445906 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:19:19.445906 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:19:19.445906 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: 7ea30eb37fbf469194845261a19e8d4f Jul 16 21:19:19.445906 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-SecurityGroupRulesNegativeTestJSON-1399499068-project Jul 16 21:19:19.445906 ncdv-master-1 devstack@n-api.service[138681]: Jul 16 21:19:19.445906 ncdv-master-1 devstack@n-api.service[138681]: {"security_group_rule": {"parent_group_id": "0f4ae75e-40bf-4aa1-81b2-6540e1aec4e0", "ip_protocol": "tcp", "from_port": 22, "to_port": 22}} Jul 16 21:19:19.445906 ncdv-master-1 devstack@n-api.service[138681]: method: Jul 16 21:19:19.445906 ncdv-master-1 devstack@n-api.service[138681]: exception: Security group 0f4ae75e-40bf-4aa1-81b2-6540e1aec4e0 does not exist Jul 16 21:19:19.445906 ncdv-master-1 devstack@n-api.service[138681]: Neutron server returns request_ids: ['req-cae3e2a5-6a3e-4561-bff1-5081a9214f18'] Jul 16 21:19:19.445906 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'body': {'security_group_rule': {'parent_group_id': '0f4ae75e-40bf-4aa1-81b2-6540e1aec4e0', 'ip_protocol': 'tcp', 'from_port': 22, 'to_port': 22}}} Jul 16 21:19:19.445906 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:19:19.445906 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:19:19.445906 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/security_group_api.py", line 309, in get Jul 16 21:19:19.445906 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi group = neutron.show_security_group(id).get('security_group') Jul 16 21:19:19.445906 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 197, in wrapper Jul 16 21:19:19.445906 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Jul 16 21:19:19.445906 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 1040, in show_security_group Jul 16 21:19:19.445906 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return self.get(self.security_group_path % (security_group), Jul 16 21:19:19.445906 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 197, in wrapper Jul 16 21:19:19.445906 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Jul 16 21:19:19.445906 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 356, in get Jul 16 21:19:19.445906 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return self.retry_request("GET", action, body=body, Jul 16 21:19:19.445906 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 197, in wrapper Jul 16 21:19:19.445906 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Jul 16 21:19:19.445906 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 333, in retry_request Jul 16 21:19:19.445906 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return self.do_request(method, action, body=body, Jul 16 21:19:19.445906 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 197, in wrapper Jul 16 21:19:19.445906 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Jul 16 21:19:19.445906 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 297, in do_request Jul 16 21:19:19.445906 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi self._handle_fault_response(status_code, replybody, resp) Jul 16 21:19:19.445906 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 197, in wrapper Jul 16 21:19:19.445906 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Jul 16 21:19:19.445906 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 272, in _handle_fault_response Jul 16 21:19:19.445906 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi exception_handler_v20(status_code, error_body) Jul 16 21:19:19.445906 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 90, in exception_handler_v20 Jul 16 21:19:19.450352 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise client_exc(message=error_message, Jul 16 21:19:19.450352 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi neutronclient.common.exceptions.NotFound: Security group 0f4ae75e-40bf-4aa1-81b2-6540e1aec4e0 does not exist Jul 16 21:19:19.450352 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-cae3e2a5-6a3e-4561-bff1-5081a9214f18'] Jul 16 21:19:19.450352 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:19:19.450352 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:19:19.450352 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:19:19.450352 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:19:19.450352 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/security_groups.py", line 287, in create Jul 16 21:19:19.450352 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi security_group = security_group_api.get( Jul 16 21:19:19.450352 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/security_group_api.py", line 314, in get Jul 16 21:19:19.450352 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.SecurityGroupNotFound(str(e)) Jul 16 21:19:19.450352 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.SecurityGroupNotFound: Security group 0f4ae75e-40bf-4aa1-81b2-6540e1aec4e0 does not exist Jul 16 21:19:19.450352 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-cae3e2a5-6a3e-4561-bff1-5081a9214f18'] Jul 16 21:19:19.450352 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:19:19.450352 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:19:19.450352 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:19:19.450352 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:19:19.450352 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:19:19.450352 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:19:19.450352 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:19:19.450352 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:19:19.450352 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Jul 16 21:19:19.450352 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Jul 16 21:19:19.450352 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:19:19.450352 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:19:19.450352 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/security_groups.py", line 303, in create Jul 16 21:19:19.450352 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=exp.format_message()) Jul 16 21:19:19.450352 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Security group 0f4ae75e-40bf-4aa1-81b2-6540e1aec4e0 does not exist Jul 16 21:19:19.450352 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-cae3e2a5-6a3e-4561-bff1-5081a9214f18'] Jul 16 21:19:19.450352 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:19:19.450352 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:19:19.450352 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:19:19.450352 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:19:19.450352 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:19:19.450352 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:19:19.450352 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:19:19.450352 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:19:19.450352 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Security group 0f4ae75e-40bf-4aa1-81b2-6540e1aec4e0 does not exist Jul 16 21:19:19.450352 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-cae3e2a5-6a3e-4561-bff1-5081a9214f18'] Jul 16 21:19:19.450352 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:19:19.450352 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-b228e305-bbbc-476e-b63f-51b11a4a4650 tempest-SecurityGroupRulesNegativeTestJSON-1399499068 tempest-SecurityGroupRulesNegativeTestJSON-1399499068-project] Returning 404 to user: Security group 0f4ae75e-40bf-4aa1-81b2-6540e1aec4e0 does not exist Jul 16 21:19:19.450352 ncdv-master-1 devstack@n-api.service[138681]: Neutron server returns request_ids: ['req-cae3e2a5-6a3e-4561-bff1-5081a9214f18'] {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:19:19.450352 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-b228e305-bbbc-476e-b63f-51b11a4a4650 tempest-SecurityGroupRulesNegativeTestJSON-1399499068 tempest-SecurityGroupRulesNegativeTestJSON-1399499068-project] 10.222.0.16 "POST /compute/v2.1/os-security-group-rules" status: 404 len: 194 microversion: 2.1 time: 0.031897 Jul 16 21:19:19.457094 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1308/2650] 10.222.0.16 () {60 vars in 1194 bytes} [Sat Jul 16 21:19:19 2022] POST /compute/v2.1/os-security-group-rules => generated 194 bytes in 39 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:19:19.457662 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-6f88635c-246c-4a51-9ca5-509a628f3dea tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Calling method '.version_select at 0x7fdd9d519160>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:19:19.480518 ncdv-master-1 devstack@n-api.service[138681]: DEBUG neutronclient.v2_0.client [None req-6f88635c-246c-4a51-9ca5-509a628f3dea tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Error message: {"NeutronError": {"type": "SecurityGroupNotFound", "message": "Security group b9e2f469-feba-4fb9-9432-0249b1ace909 does not exist", "detail": ""}} {{(pid=138681) _handle_fault_response /usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py:262}} Jul 16 21:19:19.481119 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.network.security_group_api [None req-6f88635c-246c-4a51-9ca5-509a628f3dea tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Neutron security group b9e2f469-feba-4fb9-9432-0249b1ace909 not found {{(pid=138681) get /opt/stack/nova/nova/network/security_group_api.py:313}} Jul 16 21:19:19.481680 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-6f88635c-246c-4a51-9ca5-509a628f3dea tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] HTTP exception thrown: Security group b9e2f469-feba-4fb9-9432-0249b1ace909 does not exist Jul 16 21:19:19.481680 ncdv-master-1 devstack@n-api.service[138681]: Neutron server returns request_ids: ['req-17853438-7644-4029-a29f-f3637037c08f'] Jul 16 21:19:19.484658 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-6f88635c-246c-4a51-9ca5-509a628f3dea tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Request method failure captured: Jul 16 21:19:19.484658 ncdv-master-1 devstack@n-api.service[138681]: request: DELETE /compute/v2.1/os-security-groups/b9e2f469-feba-4fb9-9432-0249b1ace909 HTTP/1.1 Jul 16 21:19:19.484658 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:19:19.484658 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:19:19.484658 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:19:19.484658 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 21:19:19.484658 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:19:19.484658 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:19:19.484658 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:19:19.484658 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:19:19.484658 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0yrk-HsfYKsyovLSaEtNxaoXy7a3rLGNlsRPiPP76ZvvrKkKyGd-uekosXQxpFgkRnItBdviN4xr49Rerg5AyvLSzsyRY-Um0r3BRAvyZrzmCDYUpWbXk1uyEIb3Af4U3Y3nXIJgJAuDVx9z2NGSvEGCsGz12WlyyGCNfVSADFcYJP8 Jul 16 21:19:19.484658 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:19:19.484658 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:19:19.484658 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:19:19.484658 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:19:19.484658 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:19:19.484658 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:19:19.484658 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: e353258f9c3048ef84683294e2b8579d Jul 16 21:19:19.484658 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-SecurityGroupsTestJSON-438340169 Jul 16 21:19:19.484658 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:19:19.484658 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:19:19.484658 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/e353258f9c3048ef84683294e2b8579d"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/e353258f9c3048ef84683294e2b8579d"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_e353258f9c3048ef84683294e2b8579d"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/e353258f9c3048ef84683294e2b8579d"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:19:19.484658 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-SecurityGroupsTestJSON-438340169 Jul 16 21:19:19.484658 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: e353258f9c3048ef84683294e2b8579d Jul 16 21:19:19.484658 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-SecurityGroupsTestJSON-438340169 Jul 16 21:19:19.484658 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-SecurityGroupsTestJSON-438340169-project Jul 16 21:19:19.484658 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:19:19.484658 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:19:19.484658 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: d0008fffad8e4708b2c01e03da92c47d Jul 16 21:19:19.484658 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-SecurityGroupsTestJSON-438340169-project Jul 16 21:19:19.484658 ncdv-master-1 devstack@n-api.service[138681]: method: Jul 16 21:19:19.484658 ncdv-master-1 devstack@n-api.service[138681]: exception: Security group b9e2f469-feba-4fb9-9432-0249b1ace909 does not exist Jul 16 21:19:19.484658 ncdv-master-1 devstack@n-api.service[138681]: Neutron server returns request_ids: ['req-17853438-7644-4029-a29f-f3637037c08f'] Jul 16 21:19:19.484658 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': 'b9e2f469-feba-4fb9-9432-0249b1ace909'} Jul 16 21:19:19.484658 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:19:19.484658 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:19:19.484658 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/security_group_api.py", line 309, in get Jul 16 21:19:19.484658 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi group = neutron.show_security_group(id).get('security_group') Jul 16 21:19:19.484658 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 197, in wrapper Jul 16 21:19:19.484658 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Jul 16 21:19:19.484658 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 1040, in show_security_group Jul 16 21:19:19.484658 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return self.get(self.security_group_path % (security_group), Jul 16 21:19:19.484658 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 197, in wrapper Jul 16 21:19:19.484658 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Jul 16 21:19:19.484658 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 356, in get Jul 16 21:19:19.484658 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return self.retry_request("GET", action, body=body, Jul 16 21:19:19.484658 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 197, in wrapper Jul 16 21:19:19.484658 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Jul 16 21:19:19.484658 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 333, in retry_request Jul 16 21:19:19.484658 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return self.do_request(method, action, body=body, Jul 16 21:19:19.484658 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 197, in wrapper Jul 16 21:19:19.484658 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Jul 16 21:19:19.484658 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 297, in do_request Jul 16 21:19:19.484658 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi self._handle_fault_response(status_code, replybody, resp) Jul 16 21:19:19.484658 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 197, in wrapper Jul 16 21:19:19.484658 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Jul 16 21:19:19.484658 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 272, in _handle_fault_response Jul 16 21:19:19.484658 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi exception_handler_v20(status_code, error_body) Jul 16 21:19:19.484658 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 90, in exception_handler_v20 Jul 16 21:19:19.484658 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise client_exc(message=error_message, Jul 16 21:19:19.484658 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi neutronclient.common.exceptions.NotFound: Security group b9e2f469-feba-4fb9-9432-0249b1ace909 does not exist Jul 16 21:19:19.488153 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-17853438-7644-4029-a29f-f3637037c08f'] Jul 16 21:19:19.488153 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:19:19.488153 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:19:19.488153 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:19:19.488153 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:19:19.488153 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/security_groups.py", line 178, in delete Jul 16 21:19:19.488153 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi security_group = security_group_api.get(context, id) Jul 16 21:19:19.488153 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/security_group_api.py", line 314, in get Jul 16 21:19:19.488153 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.SecurityGroupNotFound(str(e)) Jul 16 21:19:19.488153 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.SecurityGroupNotFound: Security group b9e2f469-feba-4fb9-9432-0249b1ace909 does not exist Jul 16 21:19:19.488153 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-17853438-7644-4029-a29f-f3637037c08f'] Jul 16 21:19:19.488153 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:19:19.488153 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:19:19.488153 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:19:19.488153 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:19:19.488153 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:19:19.488153 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:19:19.488153 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:19:19.488153 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:19:19.488153 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Jul 16 21:19:19.488153 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Jul 16 21:19:19.488153 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:19:19.488153 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:19:19.488153 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/security_groups.py", line 181, in delete Jul 16 21:19:19.488153 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=exp.format_message()) Jul 16 21:19:19.488153 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Security group b9e2f469-feba-4fb9-9432-0249b1ace909 does not exist Jul 16 21:19:19.488153 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-17853438-7644-4029-a29f-f3637037c08f'] Jul 16 21:19:19.488153 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:19:19.488153 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:19:19.488153 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:19:19.488153 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:19:19.488153 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:19:19.488153 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:19:19.488153 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:19:19.488153 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:19:19.488153 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Security group b9e2f469-feba-4fb9-9432-0249b1ace909 does not exist Jul 16 21:19:19.488153 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-17853438-7644-4029-a29f-f3637037c08f'] Jul 16 21:19:19.488153 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:19:19.488153 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-6f88635c-246c-4a51-9ca5-509a628f3dea tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Returning 404 to user: Security group b9e2f469-feba-4fb9-9432-0249b1ace909 does not exist Jul 16 21:19:19.488153 ncdv-master-1 devstack@n-api.service[138681]: Neutron server returns request_ids: ['req-17853438-7644-4029-a29f-f3637037c08f'] {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:19:19.488153 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-6f88635c-246c-4a51-9ca5-509a628f3dea tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] 10.222.0.16 "DELETE /compute/v2.1/os-security-groups/b9e2f469-feba-4fb9-9432-0249b1ace909" status: 404 len: 194 microversion: 2.1 time: 0.032392 Jul 16 21:19:19.488153 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1309/2651] 10.222.0.16 () {58 vars in 1271 bytes} [Sat Jul 16 21:19:19 2022] DELETE /compute/v2.1/os-security-groups/b9e2f469-feba-4fb9-9432-0249b1ace909 => generated 194 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:19:19.493978 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-086d6ccd-d585-4d39-9935-3d8c09a65992 tempest-SecurityGroupRulesNegativeTestJSON-1399499068 tempest-SecurityGroupRulesNegativeTestJSON-1399499068-project] Calling method '.version_select at 0x7fdd9cf0f550>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:19:19.515034 ncdv-master-1 devstack@n-api.service[138681]: DEBUG neutronclient.v2_0.client [None req-086d6ccd-d585-4d39-9935-3d8c09a65992 tempest-SecurityGroupRulesNegativeTestJSON-1399499068 tempest-SecurityGroupRulesNegativeTestJSON-1399499068-project] Error message: {"NeutronError": {"type": "SecurityGroupRuleNotFound", "message": "Security group rule 5a186cfa-3c84-4244-a8d8-c20525594fd4 does not exist", "detail": ""}} {{(pid=138681) _handle_fault_response /usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py:262}} Jul 16 21:19:19.515646 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.network.security_group_api [None req-086d6ccd-d585-4d39-9935-3d8c09a65992 tempest-SecurityGroupRulesNegativeTestJSON-1399499068 tempest-SecurityGroupRulesNegativeTestJSON-1399499068-project] Neutron security group rule 5a186cfa-3c84-4244-a8d8-c20525594fd4 not found {{(pid=138681) get_rule /opt/stack/nova/nova/network/security_group_api.py:463}} Jul 16 21:19:19.516170 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-086d6ccd-d585-4d39-9935-3d8c09a65992 tempest-SecurityGroupRulesNegativeTestJSON-1399499068 tempest-SecurityGroupRulesNegativeTestJSON-1399499068-project] HTTP exception thrown: Security group rule 5a186cfa-3c84-4244-a8d8-c20525594fd4 does not exist Jul 16 21:19:19.516170 ncdv-master-1 devstack@n-api.service[138681]: Neutron server returns request_ids: ['req-f1c1152a-e41d-4e12-bd4a-727390b5b9ce'] Jul 16 21:19:19.517991 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-086d6ccd-d585-4d39-9935-3d8c09a65992 tempest-SecurityGroupRulesNegativeTestJSON-1399499068 tempest-SecurityGroupRulesNegativeTestJSON-1399499068-project] Request method failure captured: Jul 16 21:19:19.517991 ncdv-master-1 devstack@n-api.service[138681]: request: DELETE /compute/v2.1/os-security-group-rules/5a186cfa-3c84-4244-a8d8-c20525594fd4 HTTP/1.1 Jul 16 21:19:19.517991 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:19:19.517991 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:19:19.517991 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:19:19.517991 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 21:19:19.517991 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:19:19.517991 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:19:19.517991 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:19:19.517991 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:19:19.517991 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0ytNkmtXKapUZpFKPGUA8hf3ATbduJ0XtzorjW7amdLo0WS0Qbkqbi3B3wBROLLwxfXJZxCXXmOW4by85qgjbrIW-gGZ2c_YrmiVSX-WID5VqAWhi2CmsmYLeeu_6bw9QJDULh6nweMasTNNM3cnBfI9QUQLgKaJIeA4QYZkK1W0reQ Jul 16 21:19:19.517991 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:19:19.517991 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:19:19.517991 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:19:19.517991 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:19:19.517991 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:19:19.517991 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:19:19.517991 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: 89b4d12252b94d54b50cc15238af6c2f Jul 16 21:19:19.517991 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-SecurityGroupRulesNegativeTestJSON-1399499068 Jul 16 21:19:19.517991 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:19:19.517991 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:19:19.517991 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/89b4d12252b94d54b50cc15238af6c2f"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/89b4d12252b94d54b50cc15238af6c2f"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_89b4d12252b94d54b50cc15238af6c2f"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/89b4d12252b94d54b50cc15238af6c2f"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:19:19.517991 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-SecurityGroupRulesNegativeTestJSON-1399499068 Jul 16 21:19:19.517991 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: 89b4d12252b94d54b50cc15238af6c2f Jul 16 21:19:19.517991 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-SecurityGroupRulesNegativeTestJSON-1399499068 Jul 16 21:19:19.517991 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-SecurityGroupRulesNegativeTestJSON-1399499068-project Jul 16 21:19:19.517991 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:19:19.517991 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:19:19.517991 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: 7ea30eb37fbf469194845261a19e8d4f Jul 16 21:19:19.517991 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-SecurityGroupRulesNegativeTestJSON-1399499068-project Jul 16 21:19:19.517991 ncdv-master-1 devstack@n-api.service[138681]: method: Jul 16 21:19:19.517991 ncdv-master-1 devstack@n-api.service[138681]: exception: Security group rule 5a186cfa-3c84-4244-a8d8-c20525594fd4 does not exist Jul 16 21:19:19.517991 ncdv-master-1 devstack@n-api.service[138681]: Neutron server returns request_ids: ['req-f1c1152a-e41d-4e12-bd4a-727390b5b9ce'] Jul 16 21:19:19.517991 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': '5a186cfa-3c84-4244-a8d8-c20525594fd4'} Jul 16 21:19:19.517991 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:19:19.517991 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:19:19.517991 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/security_group_api.py", line 459, in get_rule Jul 16 21:19:19.517991 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi rule = neutron.show_security_group_rule( Jul 16 21:19:19.517991 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 197, in wrapper Jul 16 21:19:19.517991 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Jul 16 21:19:19.517991 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 1064, in show_security_group_rule Jul 16 21:19:19.517991 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return self.get(self.security_group_rule_path % (security_group_rule), Jul 16 21:19:19.517991 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 197, in wrapper Jul 16 21:19:19.517991 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Jul 16 21:19:19.517991 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 356, in get Jul 16 21:19:19.517991 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return self.retry_request("GET", action, body=body, Jul 16 21:19:19.517991 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 197, in wrapper Jul 16 21:19:19.517991 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Jul 16 21:19:19.517991 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 333, in retry_request Jul 16 21:19:19.517991 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return self.do_request(method, action, body=body, Jul 16 21:19:19.517991 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 197, in wrapper Jul 16 21:19:19.517991 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Jul 16 21:19:19.517991 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 297, in do_request Jul 16 21:19:19.517991 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi self._handle_fault_response(status_code, replybody, resp) Jul 16 21:19:19.517991 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 197, in wrapper Jul 16 21:19:19.517991 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Jul 16 21:19:19.517991 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 272, in _handle_fault_response Jul 16 21:19:19.517991 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi exception_handler_v20(status_code, error_body) Jul 16 21:19:19.517991 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 90, in exception_handler_v20 Jul 16 21:19:19.517991 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise client_exc(message=error_message, Jul 16 21:19:19.517991 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi neutronclient.common.exceptions.NotFound: Security group rule 5a186cfa-3c84-4244-a8d8-c20525594fd4 does not exist Jul 16 21:19:19.527189 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-f1c1152a-e41d-4e12-bd4a-727390b5b9ce'] Jul 16 21:19:19.527189 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:19:19.527189 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:19:19.527189 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:19:19.527189 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:19:19.527189 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/security_groups.py", line 359, in delete Jul 16 21:19:19.527189 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi rule = security_group_api.get_rule(context, id) Jul 16 21:19:19.527189 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/security_group_api.py", line 464, in get_rule Jul 16 21:19:19.527189 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.SecurityGroupNotFound(str(e)) Jul 16 21:19:19.527189 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.SecurityGroupNotFound: Security group rule 5a186cfa-3c84-4244-a8d8-c20525594fd4 does not exist Jul 16 21:19:19.527189 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-f1c1152a-e41d-4e12-bd4a-727390b5b9ce'] Jul 16 21:19:19.527189 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:19:19.527189 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:19:19.527189 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:19:19.527189 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:19:19.527189 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:19:19.527189 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:19:19.527189 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:19:19.527189 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:19:19.527189 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Jul 16 21:19:19.527189 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Jul 16 21:19:19.527189 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:19:19.527189 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:19:19.527189 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/security_groups.py", line 365, in delete Jul 16 21:19:19.527189 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=exp.format_message()) Jul 16 21:19:19.527189 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Security group rule 5a186cfa-3c84-4244-a8d8-c20525594fd4 does not exist Jul 16 21:19:19.527189 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-f1c1152a-e41d-4e12-bd4a-727390b5b9ce'] Jul 16 21:19:19.527189 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:19:19.527189 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:19:19.527189 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:19:19.527189 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:19:19.527189 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:19:19.527189 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:19:19.527189 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:19:19.527189 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:19:19.527189 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Security group rule 5a186cfa-3c84-4244-a8d8-c20525594fd4 does not exist Jul 16 21:19:19.527189 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-f1c1152a-e41d-4e12-bd4a-727390b5b9ce'] Jul 16 21:19:19.527189 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:19:19.527189 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-086d6ccd-d585-4d39-9935-3d8c09a65992 tempest-SecurityGroupRulesNegativeTestJSON-1399499068 tempest-SecurityGroupRulesNegativeTestJSON-1399499068-project] Returning 404 to user: Security group rule 5a186cfa-3c84-4244-a8d8-c20525594fd4 does not exist Jul 16 21:19:19.527189 ncdv-master-1 devstack@n-api.service[138681]: Neutron server returns request_ids: ['req-f1c1152a-e41d-4e12-bd4a-727390b5b9ce'] {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:19:19.527189 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-086d6ccd-d585-4d39-9935-3d8c09a65992 tempest-SecurityGroupRulesNegativeTestJSON-1399499068 tempest-SecurityGroupRulesNegativeTestJSON-1399499068-project] 10.222.0.16 "DELETE /compute/v2.1/os-security-group-rules/5a186cfa-3c84-4244-a8d8-c20525594fd4" status: 404 len: 199 microversion: 2.1 time: 0.032114 Jul 16 21:19:19.527189 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1310/2652] 10.222.0.16 () {58 vars in 1286 bytes} [Sat Jul 16 21:19:19 2022] DELETE /compute/v2.1/os-security-group-rules/5a186cfa-3c84-4244-a8d8-c20525594fd4 => generated 199 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:19:19.527189 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-9d2ec647-2900-483b-90ec-2386c1aa736f tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:19:19.527189 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-9d2ec647-2900-483b-90ec-2386c1aa736f tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] [instance: b8565a82-250b-4f64-9d90-98b368e4bd57] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:19:19.533646 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9d2ec647-2900-483b-90ec-2386c1aa736f tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:19.534108 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9d2ec647-2900-483b-90ec-2386c1aa736f tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:19.534893 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9d2ec647-2900-483b-90ec-2386c1aa736f tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:19.535439 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9d2ec647-2900-483b-90ec-2386c1aa736f tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:19.565096 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-9d2ec647-2900-483b-90ec-2386c1aa736f tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] [instance: b8565a82-250b-4f64-9d90-98b368e4bd57] Going to try to terminate instance {{(pid=138681) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 16 21:19:19.776961 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-00f4ca1f-2e64-4c69-85f4-5e18cba8390b tempest-ServersAaction247Test-725202529 tempest-ServersAaction247Test-725202529-project] 10.222.0.16 "DELETE /compute/v2.1/servers/a51adaf1-9c26-47d0-92c2-b6bb065a5ee8" status: 204 len: 0 microversion: 2.47 time: 0.466515 Jul 16 21:19:19.779627 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1343/2653] 10.222.0.16 () {60 vars in 1279 bytes} [Sat Jul 16 21:19:19 2022] DELETE /compute/v2.1/servers/a51adaf1-9c26-47d0-92c2-b6bb065a5ee8 => generated 0 bytes in 470 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Jul 16 21:19:19.787452 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-7254fae2-0c71-4065-a3ed-b6516bd5d9eb tempest-SecurityGroupRulesNegativeTestJSON-1399499068 tempest-SecurityGroupRulesNegativeTestJSON-1399499068-project] Calling method '.version_select at 0x7fdd9cc35550>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:19:19.907027 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-9d2ec647-2900-483b-90ec-2386c1aa736f tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] 10.222.0.16 "DELETE /compute/v2.1/servers/b8565a82-250b-4f64-9d90-98b368e4bd57" status: 204 len: 0 microversion: 2.1 time: 0.385389 Jul 16 21:19:19.908354 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1311/2654] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:19:19 2022] DELETE /compute/v2.1/servers/b8565a82-250b-4f64-9d90-98b368e4bd57 => generated 0 bytes in 387 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 16 21:19:19.920124 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-7237485d-78d7-4e80-ba5f-bfeebac6cb9c tempest-ServersAaction247Test-725202529 tempest-ServersAaction247Test-725202529-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:19:19.922189 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-7237485d-78d7-4e80-ba5f-bfeebac6cb9c tempest-ServersAaction247Test-725202529 tempest-ServersAaction247Test-725202529-project] [instance: a51adaf1-9c26-47d0-92c2-b6bb065a5ee8] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:19:19.942706 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7237485d-78d7-4e80-ba5f-bfeebac6cb9c tempest-ServersAaction247Test-725202529 tempest-ServersAaction247Test-725202529-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:19.942706 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7237485d-78d7-4e80-ba5f-bfeebac6cb9c tempest-ServersAaction247Test-725202529 tempest-ServersAaction247Test-725202529-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:19.943969 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7237485d-78d7-4e80-ba5f-bfeebac6cb9c tempest-ServersAaction247Test-725202529 tempest-ServersAaction247Test-725202529-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:19.944426 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7237485d-78d7-4e80-ba5f-bfeebac6cb9c tempest-ServersAaction247Test-725202529 tempest-ServersAaction247Test-725202529-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:19.972945 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=96,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:19:20.083162 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-7237485d-78d7-4e80-ba5f-bfeebac6cb9c tempest-ServersAaction247Test-725202529 tempest-ServersAaction247Test-725202529-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a8d48b52db1545a4a51027a5c2c7fcce', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f2984288b4cd4610b2f32e895f3e5d72', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:19:20.097033 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-7237485d-78d7-4e80-ba5f-bfeebac6cb9c tempest-ServersAaction247Test-725202529 tempest-ServersAaction247Test-725202529-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'a8d48b52db1545a4a51027a5c2c7fcce', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f2984288b4cd4610b2f32e895f3e5d72', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:19:20.100076 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-7237485d-78d7-4e80-ba5f-bfeebac6cb9c tempest-ServersAaction247Test-725202529 tempest-ServersAaction247Test-725202529-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'a8d48b52db1545a4a51027a5c2c7fcce', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f2984288b4cd4610b2f32e895f3e5d72', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:19:20.102904 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-7237485d-78d7-4e80-ba5f-bfeebac6cb9c tempest-ServersAaction247Test-725202529 tempest-ServersAaction247Test-725202529-project] 10.222.0.16 "GET /compute/v2.1/servers/a51adaf1-9c26-47d0-92c2-b6bb065a5ee8" status: 200 len: 1366 microversion: 2.47 time: 0.191865 Jul 16 21:19:20.105904 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1312/2655] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:19:19 2022] GET /compute/v2.1/servers/a51adaf1-9c26-47d0-92c2-b6bb065a5ee8 => generated 1366 bytes in 197 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:19:20.119161 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-08889e56-9b79-4331-a058-c17bd91550e8 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:19:20.120196 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-08889e56-9b79-4331-a058-c17bd91550e8 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] [instance: b8565a82-250b-4f64-9d90-98b368e4bd57] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:19:20.144425 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-08889e56-9b79-4331-a058-c17bd91550e8 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:20.147244 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-08889e56-9b79-4331-a058-c17bd91550e8 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:20.151629 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-08889e56-9b79-4331-a058-c17bd91550e8 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:20.152316 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-08889e56-9b79-4331-a058-c17bd91550e8 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:20.332391 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-7254fae2-0c71-4065-a3ed-b6516bd5d9eb tempest-SecurityGroupRulesNegativeTestJSON-1399499068 tempest-SecurityGroupRulesNegativeTestJSON-1399499068-project] 10.222.0.16 "DELETE /compute/v2.1/os-security-groups/386428d1-8e69-4752-85e2-9f71d5be2bb7" status: 202 len: 0 microversion: 2.1 time: 0.550733 Jul 16 21:19:20.333420 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1344/2656] 10.222.0.16 () {58 vars in 1271 bytes} [Sat Jul 16 21:19:19 2022] DELETE /compute/v2.1/os-security-groups/386428d1-8e69-4752-85e2-9f71d5be2bb7 => generated 0 bytes in 553 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 21:19:20.340660 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-22ac9e4f-c984-4b42-8c34-b1413223246e tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:19:20.342306 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-22ac9e4f-c984-4b42-8c34-b1413223246e tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] [instance: c16e5f09-f813-414e-b233-c08348078624] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:19:20.353102 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-22ac9e4f-c984-4b42-8c34-b1413223246e tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:20.353684 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-22ac9e4f-c984-4b42-8c34-b1413223246e tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:20.355858 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-22ac9e4f-c984-4b42-8c34-b1413223246e tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:20.357245 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-22ac9e4f-c984-4b42-8c34-b1413223246e tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:20.438077 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-08889e56-9b79-4331-a058-c17bd91550e8 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd0008fffad8e4708b2c01e03da92c47d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e353258f9c3048ef84683294e2b8579d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:19:20.448995 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-08889e56-9b79-4331-a058-c17bd91550e8 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] 10.222.0.16 "GET /compute/v2.1/servers/b8565a82-250b-4f64-9d90-98b368e4bd57" status: 200 len: 1649 microversion: 2.1 time: 0.341343 Jul 16 21:19:20.454504 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1313/2657] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:19:20 2022] GET /compute/v2.1/servers/b8565a82-250b-4f64-9d90-98b368e4bd57 => generated 1649 bytes in 343 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:19:20.463635 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-086823a5-4498-4a50-b68c-336cbc379ec8 tempest-SecurityGroupRulesNegativeTestJSON-1399499068 tempest-SecurityGroupRulesNegativeTestJSON-1399499068-project] Calling method '.version_select at 0x7fdd9d0cce50>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:19:20.611663 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-22ac9e4f-c984-4b42-8c34-b1413223246e tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eebb224a56814f92b12ebebebc6bc8a3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a092ce9ef40943f6b6ace6cd28481b71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:19:20.624172 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-22ac9e4f-c984-4b42-8c34-b1413223246e tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] 10.222.0.16 "GET /compute/v2.1/servers/c16e5f09-f813-414e-b233-c08348078624" status: 200 len: 1320 microversion: 2.1 time: 0.289071 Jul 16 21:19:20.625091 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1345/2658] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:19:20 2022] GET /compute/v2.1/servers/c16e5f09-f813-414e-b233-c08348078624 => generated 1320 bytes in 291 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:19:20.640470 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-4060da01-fc6a-4ab2-b2a4-066ba6381420 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:19:20.647086 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-4060da01-fc6a-4ab2-b2a4-066ba6381420 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] [instance: e7598b4a-f70c-4186-88d1-6fd0057f8656] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:19:20.652856 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4060da01-fc6a-4ab2-b2a4-066ba6381420 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:20.653743 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4060da01-fc6a-4ab2-b2a4-066ba6381420 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:20.655246 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4060da01-fc6a-4ab2-b2a4-066ba6381420 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:20.656007 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4060da01-fc6a-4ab2-b2a4-066ba6381420 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:20.877129 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-086823a5-4498-4a50-b68c-336cbc379ec8 tempest-SecurityGroupRulesNegativeTestJSON-1399499068 tempest-SecurityGroupRulesNegativeTestJSON-1399499068-project] 10.222.0.16 "DELETE /compute/v2.1/os-security-groups/171e329c-c428-4558-abb4-cf001d25e916" status: 202 len: 0 microversion: 2.1 time: 0.422964 Jul 16 21:19:20.878538 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1314/2659] 10.222.0.16 () {58 vars in 1271 bytes} [Sat Jul 16 21:19:20 2022] DELETE /compute/v2.1/os-security-groups/171e329c-c428-4558-abb4-cf001d25e916 => generated 0 bytes in 427 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 21:19:20.890053 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-9f56af07-d685-4ef0-b90f-ce6b8f7ba412 tempest-SecurityGroupRulesNegativeTestJSON-1399499068 tempest-SecurityGroupRulesNegativeTestJSON-1399499068-project] Calling method '.version_select at 0x7fdd9cf525e0>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:19:20.935050 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-4060da01-fc6a-4ab2-b2a4-066ba6381420 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f5bc1f2209f4eb186d80e385b67c774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '65d5bcfa02af4e8284ba1342197e2860', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:19:20.944094 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-4060da01-fc6a-4ab2-b2a4-066ba6381420 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] 10.222.0.16 "GET /compute/v2.1/servers/e7598b4a-f70c-4186-88d1-6fd0057f8656" status: 200 len: 1472 microversion: 2.1 time: 0.307747 Jul 16 21:19:20.947816 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1346/2660] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:19:20 2022] GET /compute/v2.1/servers/e7598b4a-f70c-4186-88d1-6fd0057f8656 => generated 1472 bytes in 312 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:19:21.125612 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-3dbf1926-46db-42b2-935f-6222378a7321 tempest-ServersAaction247Test-725202529 tempest-ServersAaction247Test-725202529-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:19:21.126539 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-3dbf1926-46db-42b2-935f-6222378a7321 tempest-ServersAaction247Test-725202529 tempest-ServersAaction247Test-725202529-project] [instance: a51adaf1-9c26-47d0-92c2-b6bb065a5ee8] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:19:21.140288 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3dbf1926-46db-42b2-935f-6222378a7321 tempest-ServersAaction247Test-725202529 tempest-ServersAaction247Test-725202529-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:21.141051 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3dbf1926-46db-42b2-935f-6222378a7321 tempest-ServersAaction247Test-725202529 tempest-ServersAaction247Test-725202529-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:21.142191 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3dbf1926-46db-42b2-935f-6222378a7321 tempest-ServersAaction247Test-725202529 tempest-ServersAaction247Test-725202529-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:21.142653 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3dbf1926-46db-42b2-935f-6222378a7321 tempest-ServersAaction247Test-725202529 tempest-ServersAaction247Test-725202529-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:21.242717 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-3dbf1926-46db-42b2-935f-6222378a7321 tempest-ServersAaction247Test-725202529 tempest-ServersAaction247Test-725202529-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a8d48b52db1545a4a51027a5c2c7fcce', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f2984288b4cd4610b2f32e895f3e5d72', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:19:21.260518 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-3dbf1926-46db-42b2-935f-6222378a7321 tempest-ServersAaction247Test-725202529 tempest-ServersAaction247Test-725202529-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'a8d48b52db1545a4a51027a5c2c7fcce', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f2984288b4cd4610b2f32e895f3e5d72', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:19:21.264894 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-9f56af07-d685-4ef0-b90f-ce6b8f7ba412 tempest-SecurityGroupRulesNegativeTestJSON-1399499068 tempest-SecurityGroupRulesNegativeTestJSON-1399499068-project] 10.222.0.16 "DELETE /compute/v2.1/os-security-groups/65d24cc4-8c2d-4fb8-bf7b-a64bb6225fe1" status: 202 len: 0 microversion: 2.1 time: 0.380140 Jul 16 21:19:21.266191 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1315/2661] 10.222.0.16 () {58 vars in 1271 bytes} [Sat Jul 16 21:19:20 2022] DELETE /compute/v2.1/os-security-groups/65d24cc4-8c2d-4fb8-bf7b-a64bb6225fe1 => generated 0 bytes in 384 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 21:19:21.271588 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-3dbf1926-46db-42b2-935f-6222378a7321 tempest-ServersAaction247Test-725202529 tempest-ServersAaction247Test-725202529-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'a8d48b52db1545a4a51027a5c2c7fcce', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f2984288b4cd4610b2f32e895f3e5d72', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:19:21.274967 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-3dbf1926-46db-42b2-935f-6222378a7321 tempest-ServersAaction247Test-725202529 tempest-ServersAaction247Test-725202529-project] 10.222.0.16 "GET /compute/v2.1/servers/a51adaf1-9c26-47d0-92c2-b6bb065a5ee8" status: 200 len: 1366 microversion: 2.47 time: 0.155417 Jul 16 21:19:21.276717 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-93ff2063-998b-4f67-88c5-034423e6cc0b tempest-SecurityGroupRulesNegativeTestJSON-1399499068 tempest-SecurityGroupRulesNegativeTestJSON-1399499068-project] Calling method '.version_select at 0x7fdd9d038b80>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:19:21.288245 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1347/2662] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:19:21 2022] GET /compute/v2.1/servers/a51adaf1-9c26-47d0-92c2-b6bb065a5ee8 => generated 1366 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:19:21.453902 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell0:SELECT=6 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:19:21.469531 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-0fa38065-7439-4797-abe6-e75a1921db55 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:19:21.470792 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-0fa38065-7439-4797-abe6-e75a1921db55 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] [instance: b8565a82-250b-4f64-9d90-98b368e4bd57] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:19:21.478453 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0fa38065-7439-4797-abe6-e75a1921db55 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:21.478999 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0fa38065-7439-4797-abe6-e75a1921db55 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:21.479851 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0fa38065-7439-4797-abe6-e75a1921db55 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:21.480482 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0fa38065-7439-4797-abe6-e75a1921db55 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:21.696823 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-0fa38065-7439-4797-abe6-e75a1921db55 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd0008fffad8e4708b2c01e03da92c47d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e353258f9c3048ef84683294e2b8579d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:19:21.704548 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-0fa38065-7439-4797-abe6-e75a1921db55 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] 10.222.0.16 "GET /compute/v2.1/servers/b8565a82-250b-4f64-9d90-98b368e4bd57" status: 200 len: 1649 microversion: 2.1 time: 0.241423 Jul 16 21:19:21.705122 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1348/2663] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:19:21 2022] GET /compute/v2.1/servers/b8565a82-250b-4f64-9d90-98b368e4bd57 => generated 1649 bytes in 243 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:19:21.710120 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-23717f0b-694c-45ea-a747-483362047092 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:19:21.711870 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-23717f0b-694c-45ea-a747-483362047092 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] [instance: c16e5f09-f813-414e-b233-c08348078624] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:19:21.718217 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-23717f0b-694c-45ea-a747-483362047092 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:21.718798 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-23717f0b-694c-45ea-a747-483362047092 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:21.719675 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-23717f0b-694c-45ea-a747-483362047092 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:21.720186 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-23717f0b-694c-45ea-a747-483362047092 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:21.740450 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-93ff2063-998b-4f67-88c5-034423e6cc0b tempest-SecurityGroupRulesNegativeTestJSON-1399499068 tempest-SecurityGroupRulesNegativeTestJSON-1399499068-project] 10.222.0.16 "DELETE /compute/v2.1/os-security-groups/2f2e2e5c-6442-41c9-9ea0-a8fc5950b918" status: 202 len: 0 microversion: 2.1 time: 0.470073 Jul 16 21:19:21.741333 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1316/2664] 10.222.0.16 () {58 vars in 1271 bytes} [Sat Jul 16 21:19:21 2022] DELETE /compute/v2.1/os-security-groups/2f2e2e5c-6442-41c9-9ea0-a8fc5950b918 => generated 0 bytes in 471 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Jul 16 21:19:21.750928 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-b1650c2b-0ba0-4b6a-bf94-42a42fc067cd tempest-SecurityGroupRulesNegativeTestJSON-1399499068 tempest-SecurityGroupRulesNegativeTestJSON-1399499068-project] Calling method '.version_select at 0x7fdd9d0aa4c0>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:19:22.007845 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-23717f0b-694c-45ea-a747-483362047092 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eebb224a56814f92b12ebebebc6bc8a3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a092ce9ef40943f6b6ace6cd28481b71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:19:22.020569 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-23717f0b-694c-45ea-a747-483362047092 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] 10.222.0.16 "GET /compute/v2.1/servers/c16e5f09-f813-414e-b233-c08348078624" status: 200 len: 1320 microversion: 2.1 time: 0.314217 Jul 16 21:19:22.025607 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1349/2665] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:19:21 2022] GET /compute/v2.1/servers/c16e5f09-f813-414e-b233-c08348078624 => generated 1320 bytes in 320 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:19:22.032788 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-43fffd61-6e0d-45cc-82a5-24f536dcbff3 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:19:22.033941 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-43fffd61-6e0d-45cc-82a5-24f536dcbff3 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] [instance: e7598b4a-f70c-4186-88d1-6fd0057f8656] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:19:22.042740 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-43fffd61-6e0d-45cc-82a5-24f536dcbff3 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:22.043993 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-43fffd61-6e0d-45cc-82a5-24f536dcbff3 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:22.045024 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-43fffd61-6e0d-45cc-82a5-24f536dcbff3 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:22.045891 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-43fffd61-6e0d-45cc-82a5-24f536dcbff3 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:22.297408 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-43fffd61-6e0d-45cc-82a5-24f536dcbff3 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f5bc1f2209f4eb186d80e385b67c774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '65d5bcfa02af4e8284ba1342197e2860', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:19:22.308434 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-43fffd61-6e0d-45cc-82a5-24f536dcbff3 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] 10.222.0.16 "GET /compute/v2.1/servers/e7598b4a-f70c-4186-88d1-6fd0057f8656" status: 200 len: 1472 microversion: 2.1 time: 0.281502 Jul 16 21:19:22.312407 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1350/2666] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:19:22 2022] GET /compute/v2.1/servers/e7598b4a-f70c-4186-88d1-6fd0057f8656 => generated 1472 bytes in 285 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:19:22.324181 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-8b013abe-44f7-43ae-99e1-e48b21e459c0 tempest-ServersAaction247Test-725202529 tempest-ServersAaction247Test-725202529-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:19:22.325128 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-8b013abe-44f7-43ae-99e1-e48b21e459c0 tempest-ServersAaction247Test-725202529 tempest-ServersAaction247Test-725202529-project] [instance: a51adaf1-9c26-47d0-92c2-b6bb065a5ee8] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:19:22.341709 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8b013abe-44f7-43ae-99e1-e48b21e459c0 tempest-ServersAaction247Test-725202529 tempest-ServersAaction247Test-725202529-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:22.342454 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8b013abe-44f7-43ae-99e1-e48b21e459c0 tempest-ServersAaction247Test-725202529 tempest-ServersAaction247Test-725202529-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:22.362174 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-b1650c2b-0ba0-4b6a-bf94-42a42fc067cd tempest-SecurityGroupRulesNegativeTestJSON-1399499068 tempest-SecurityGroupRulesNegativeTestJSON-1399499068-project] 10.222.0.16 "DELETE /compute/v2.1/os-security-groups/cbd89553-301f-4fde-8e7e-ca4e1602fcb9" status: 202 len: 0 microversion: 2.1 time: 0.616640 Jul 16 21:19:22.363597 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1317/2667] 10.222.0.16 () {58 vars in 1271 bytes} [Sat Jul 16 21:19:21 2022] DELETE /compute/v2.1/os-security-groups/cbd89553-301f-4fde-8e7e-ca4e1602fcb9 => generated 0 bytes in 619 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Jul 16 21:19:22.373979 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8b013abe-44f7-43ae-99e1-e48b21e459c0 tempest-ServersAaction247Test-725202529 tempest-ServersAaction247Test-725202529-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:22.373979 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8b013abe-44f7-43ae-99e1-e48b21e459c0 tempest-ServersAaction247Test-725202529 tempest-ServersAaction247Test-725202529-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.029s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:22.719983 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell0:SELECT=14 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:19:22.727447 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-6c2d5f2d-de7d-4b30-9299-3ab118fb0036 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:19:22.728431 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-6c2d5f2d-de7d-4b30-9299-3ab118fb0036 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] [instance: b8565a82-250b-4f64-9d90-98b368e4bd57] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:19:22.740102 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6c2d5f2d-de7d-4b30-9299-3ab118fb0036 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:22.741470 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6c2d5f2d-de7d-4b30-9299-3ab118fb0036 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:22.744327 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6c2d5f2d-de7d-4b30-9299-3ab118fb0036 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:22.745023 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6c2d5f2d-de7d-4b30-9299-3ab118fb0036 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:22.859896 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-8b013abe-44f7-43ae-99e1-e48b21e459c0 tempest-ServersAaction247Test-725202529 tempest-ServersAaction247Test-725202529-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a8d48b52db1545a4a51027a5c2c7fcce', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f2984288b4cd4610b2f32e895f3e5d72', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:19:22.868602 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-8b013abe-44f7-43ae-99e1-e48b21e459c0 tempest-ServersAaction247Test-725202529 tempest-ServersAaction247Test-725202529-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'a8d48b52db1545a4a51027a5c2c7fcce', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f2984288b4cd4610b2f32e895f3e5d72', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:19:22.881228 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-8b013abe-44f7-43ae-99e1-e48b21e459c0 tempest-ServersAaction247Test-725202529 tempest-ServersAaction247Test-725202529-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'a8d48b52db1545a4a51027a5c2c7fcce', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f2984288b4cd4610b2f32e895f3e5d72', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:19:22.882663 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-8b013abe-44f7-43ae-99e1-e48b21e459c0 tempest-ServersAaction247Test-725202529 tempest-ServersAaction247Test-725202529-project] 10.222.0.16 "GET /compute/v2.1/servers/a51adaf1-9c26-47d0-92c2-b6bb065a5ee8" status: 200 len: 1366 microversion: 2.47 time: 0.569192 Jul 16 21:19:22.883635 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1351/2668] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:19:22 2022] GET /compute/v2.1/servers/a51adaf1-9c26-47d0-92c2-b6bb065a5ee8 => generated 1366 bytes in 571 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:19:23.045992 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-58205044-268a-4a9a-9332-a252ebd665de tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:19:23.046628 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-58205044-268a-4a9a-9332-a252ebd665de tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] [instance: c16e5f09-f813-414e-b233-c08348078624] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:19:23.062120 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-58205044-268a-4a9a-9332-a252ebd665de tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:23.062769 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-58205044-268a-4a9a-9332-a252ebd665de tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:23.063973 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-58205044-268a-4a9a-9332-a252ebd665de tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:23.064569 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-58205044-268a-4a9a-9332-a252ebd665de tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:23.197060 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-6c2d5f2d-de7d-4b30-9299-3ab118fb0036 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd0008fffad8e4708b2c01e03da92c47d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e353258f9c3048ef84683294e2b8579d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:19:23.207986 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-6c2d5f2d-de7d-4b30-9299-3ab118fb0036 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] 10.222.0.16 "GET /compute/v2.1/servers/b8565a82-250b-4f64-9d90-98b368e4bd57" status: 200 len: 1649 microversion: 2.1 time: 0.488855 Jul 16 21:19:23.209050 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1318/2669] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:19:22 2022] GET /compute/v2.1/servers/b8565a82-250b-4f64-9d90-98b368e4bd57 => generated 1649 bytes in 491 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:19:23.226992 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-423a82b4-0b29-45cc-ac1f-3db93f62bdb2 req-fff51590-96a6-48bd-8697-dd3c274f87b6 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "21522211-8382-4958-ad2a-cff26f4f2305", "name": "network-vif-plugged", "status": "completed", "tag": "9805a0f5-a6ed-4cb8-ac51-b1d03a7a9ec6"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:19:23.240775 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-423a82b4-0b29-45cc-ac1f-3db93f62bdb2 req-fff51590-96a6-48bd-8697-dd3c274f87b6 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:23.246933 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-423a82b4-0b29-45cc-ac1f-3db93f62bdb2 req-fff51590-96a6-48bd-8697-dd3c274f87b6 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:23.264855 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-423a82b4-0b29-45cc-ac1f-3db93f62bdb2 req-fff51590-96a6-48bd-8697-dd3c274f87b6 service nova] Creating event network-vif-plugged:9805a0f5-a6ed-4cb8-ac51-b1d03a7a9ec6 for instance 21522211-8382-4958-ad2a-cff26f4f2305 on nch2-master-1 Jul 16 21:19:23.270393 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-423a82b4-0b29-45cc-ac1f-3db93f62bdb2 req-fff51590-96a6-48bd-8697-dd3c274f87b6 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.059992 Jul 16 21:19:23.273902 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1319/2670] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:19:23 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 62 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:19:23.328157 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-6393d17a-2620-4cb3-bd59-cbf28077785e tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:19:23.329261 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-6393d17a-2620-4cb3-bd59-cbf28077785e tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] [instance: e7598b4a-f70c-4186-88d1-6fd0057f8656] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:19:23.356090 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6393d17a-2620-4cb3-bd59-cbf28077785e tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.014s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:23.356904 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6393d17a-2620-4cb3-bd59-cbf28077785e tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:23.359813 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6393d17a-2620-4cb3-bd59-cbf28077785e tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:23.360492 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6393d17a-2620-4cb3-bd59-cbf28077785e tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:23.390046 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-58205044-268a-4a9a-9332-a252ebd665de tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eebb224a56814f92b12ebebebc6bc8a3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a092ce9ef40943f6b6ace6cd28481b71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:19:23.398389 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-58205044-268a-4a9a-9332-a252ebd665de tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] 10.222.0.16 "GET /compute/v2.1/servers/c16e5f09-f813-414e-b233-c08348078624" status: 200 len: 1320 microversion: 2.1 time: 0.361245 Jul 16 21:19:23.413534 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1352/2671] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:19:23 2022] GET /compute/v2.1/servers/c16e5f09-f813-414e-b233-c08348078624 => generated 1320 bytes in 377 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:19:23.893518 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-6393d17a-2620-4cb3-bd59-cbf28077785e tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f5bc1f2209f4eb186d80e385b67c774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '65d5bcfa02af4e8284ba1342197e2860', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:19:23.905468 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-6393d17a-2620-4cb3-bd59-cbf28077785e tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] 10.222.0.16 "GET /compute/v2.1/servers/e7598b4a-f70c-4186-88d1-6fd0057f8656" status: 200 len: 1472 microversion: 2.1 time: 0.581759 Jul 16 21:19:23.910692 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1320/2672] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:19:23 2022] GET /compute/v2.1/servers/e7598b4a-f70c-4186-88d1-6fd0057f8656 => generated 1472 bytes in 588 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:19:23.920382 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-842c6172-59a3-43bc-b2ea-e61293de3edb tempest-ServersAaction247Test-725202529 tempest-ServersAaction247Test-725202529-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:19:23.922373 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-842c6172-59a3-43bc-b2ea-e61293de3edb tempest-ServersAaction247Test-725202529 tempest-ServersAaction247Test-725202529-project] [instance: a51adaf1-9c26-47d0-92c2-b6bb065a5ee8] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:19:23.930746 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-842c6172-59a3-43bc-b2ea-e61293de3edb tempest-ServersAaction247Test-725202529 tempest-ServersAaction247Test-725202529-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:23.931565 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-842c6172-59a3-43bc-b2ea-e61293de3edb tempest-ServersAaction247Test-725202529 tempest-ServersAaction247Test-725202529-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:23.934094 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-842c6172-59a3-43bc-b2ea-e61293de3edb tempest-ServersAaction247Test-725202529 tempest-ServersAaction247Test-725202529-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:23.934535 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-842c6172-59a3-43bc-b2ea-e61293de3edb tempest-ServersAaction247Test-725202529 tempest-ServersAaction247Test-725202529-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:23.974213 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-842c6172-59a3-43bc-b2ea-e61293de3edb tempest-ServersAaction247Test-725202529 tempest-ServersAaction247Test-725202529-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:23.974592 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-842c6172-59a3-43bc-b2ea-e61293de3edb tempest-ServersAaction247Test-725202529 tempest-ServersAaction247Test-725202529-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:23.975304 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.objects.instance [None req-842c6172-59a3-43bc-b2ea-e61293de3edb tempest-ServersAaction247Test-725202529 tempest-ServersAaction247Test-725202529-project] Lazy-loading 'fault' on Instance uuid a51adaf1-9c26-47d0-92c2-b6bb065a5ee8 {{(pid=138680) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:19:24.041956 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-842c6172-59a3-43bc-b2ea-e61293de3edb tempest-ServersAaction247Test-725202529 tempest-ServersAaction247Test-725202529-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a8d48b52db1545a4a51027a5c2c7fcce', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f2984288b4cd4610b2f32e895f3e5d72', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:19:24.063575 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-842c6172-59a3-43bc-b2ea-e61293de3edb tempest-ServersAaction247Test-725202529 tempest-ServersAaction247Test-725202529-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'a8d48b52db1545a4a51027a5c2c7fcce', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f2984288b4cd4610b2f32e895f3e5d72', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:19:24.068995 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-842c6172-59a3-43bc-b2ea-e61293de3edb tempest-ServersAaction247Test-725202529 tempest-ServersAaction247Test-725202529-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'a8d48b52db1545a4a51027a5c2c7fcce', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f2984288b4cd4610b2f32e895f3e5d72', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:19:24.072060 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-842c6172-59a3-43bc-b2ea-e61293de3edb tempest-ServersAaction247Test-725202529 tempest-ServersAaction247Test-725202529-project] 10.222.0.16 "GET /compute/v2.1/servers/a51adaf1-9c26-47d0-92c2-b6bb065a5ee8" status: 200 len: 1371 microversion: 2.47 time: 0.174214 Jul 16 21:19:24.074770 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1353/2673] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:19:23 2022] GET /compute/v2.1/servers/a51adaf1-9c26-47d0-92c2-b6bb065a5ee8 => generated 1371 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:19:24.228567 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-a9889c6c-337b-400c-84b0-e158133d37da tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:19:24.229032 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-a9889c6c-337b-400c-84b0-e158133d37da tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] [instance: b8565a82-250b-4f64-9d90-98b368e4bd57] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:19:24.237168 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a9889c6c-337b-400c-84b0-e158133d37da tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:24.237601 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a9889c6c-337b-400c-84b0-e158133d37da tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:24.238963 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a9889c6c-337b-400c-84b0-e158133d37da tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:24.239584 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a9889c6c-337b-400c-84b0-e158133d37da tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:24.246335 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-1a9110d5-21ff-4302-a824-26f1c63b8e45 req-2dbfb119-c73b-489f-80d3-e1fffe99c6b9 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "b8565a82-250b-4f64-9d90-98b368e4bd57", "name": "network-vif-deleted", "tag": "54d1cb01-bab2-432d-8666-66691fd8c6de"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:19:24.256461 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-1a9110d5-21ff-4302-a824-26f1c63b8e45 req-2dbfb119-c73b-489f-80d3-e1fffe99c6b9 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:24.256955 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-1a9110d5-21ff-4302-a824-26f1c63b8e45 req-2dbfb119-c73b-489f-80d3-e1fffe99c6b9 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:24.272127 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:19:24.282258 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-1a9110d5-21ff-4302-a824-26f1c63b8e45 req-2dbfb119-c73b-489f-80d3-e1fffe99c6b9 service nova] Creating event network-vif-deleted:54d1cb01-bab2-432d-8666-66691fd8c6de for instance b8565a82-250b-4f64-9d90-98b368e4bd57 on nch2-master-1 Jul 16 21:19:24.291878 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-1a9110d5-21ff-4302-a824-26f1c63b8e45 req-2dbfb119-c73b-489f-80d3-e1fffe99c6b9 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.050221 Jul 16 21:19:24.348859 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1354/2674] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:19:24 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 16 21:19:24.360082 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-a9889c6c-337b-400c-84b0-e158133d37da tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd0008fffad8e4708b2c01e03da92c47d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e353258f9c3048ef84683294e2b8579d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:19:24.370971 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-a9889c6c-337b-400c-84b0-e158133d37da tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] 10.222.0.16 "GET /compute/v2.1/servers/b8565a82-250b-4f64-9d90-98b368e4bd57" status: 200 len: 1469 microversion: 2.1 time: 0.149100 Jul 16 21:19:24.374128 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1321/2675] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:19:24 2022] GET /compute/v2.1/servers/b8565a82-250b-4f64-9d90-98b368e4bd57 => generated 1469 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:19:24.421355 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-d1ccb7fb-544c-4838-b16c-8cc7d1d6a687 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:19:24.422076 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-d1ccb7fb-544c-4838-b16c-8cc7d1d6a687 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] [instance: c16e5f09-f813-414e-b233-c08348078624] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:19:24.428554 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d1ccb7fb-544c-4838-b16c-8cc7d1d6a687 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:24.429017 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d1ccb7fb-544c-4838-b16c-8cc7d1d6a687 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:24.430166 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d1ccb7fb-544c-4838-b16c-8cc7d1d6a687 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:24.430623 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d1ccb7fb-544c-4838-b16c-8cc7d1d6a687 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:24.571513 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-d1ccb7fb-544c-4838-b16c-8cc7d1d6a687 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eebb224a56814f92b12ebebebc6bc8a3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a092ce9ef40943f6b6ace6cd28481b71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:19:24.579590 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-d1ccb7fb-544c-4838-b16c-8cc7d1d6a687 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] 10.222.0.16 "GET /compute/v2.1/servers/c16e5f09-f813-414e-b233-c08348078624" status: 200 len: 1320 microversion: 2.1 time: 0.162032 Jul 16 21:19:24.579876 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1355/2676] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:19:24 2022] GET /compute/v2.1/servers/c16e5f09-f813-414e-b233-c08348078624 => generated 1320 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:19:24.925540 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-7616d68d-214b-482a-96a6-4f7057797276 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:19:24.926861 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-7616d68d-214b-482a-96a6-4f7057797276 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] [instance: e7598b4a-f70c-4186-88d1-6fd0057f8656] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:19:24.935140 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7616d68d-214b-482a-96a6-4f7057797276 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:24.935784 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7616d68d-214b-482a-96a6-4f7057797276 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:24.939494 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7616d68d-214b-482a-96a6-4f7057797276 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:24.940295 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7616d68d-214b-482a-96a6-4f7057797276 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:25.094917 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-f795f7d6-b71c-41e0-87e0-659dbf7bbb5c tempest-ServersAaction247Test-725202529 tempest-ServersAaction247Test-725202529-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:19:25.096639 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-f795f7d6-b71c-41e0-87e0-659dbf7bbb5c tempest-ServersAaction247Test-725202529 tempest-ServersAaction247Test-725202529-project] [instance: a51adaf1-9c26-47d0-92c2-b6bb065a5ee8] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:19:25.118713 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f795f7d6-b71c-41e0-87e0-659dbf7bbb5c tempest-ServersAaction247Test-725202529 tempest-ServersAaction247Test-725202529-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:25.118713 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f795f7d6-b71c-41e0-87e0-659dbf7bbb5c tempest-ServersAaction247Test-725202529 tempest-ServersAaction247Test-725202529-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:25.118713 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f795f7d6-b71c-41e0-87e0-659dbf7bbb5c tempest-ServersAaction247Test-725202529 tempest-ServersAaction247Test-725202529-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:25.118713 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f795f7d6-b71c-41e0-87e0-659dbf7bbb5c tempest-ServersAaction247Test-725202529 tempest-ServersAaction247Test-725202529-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:25.128777 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-f795f7d6-b71c-41e0-87e0-659dbf7bbb5c tempest-ServersAaction247Test-725202529 tempest-ServersAaction247Test-725202529-project] HTTP exception thrown: Instance a51adaf1-9c26-47d0-92c2-b6bb065a5ee8 could not be found. Jul 16 21:19:25.130900 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-f795f7d6-b71c-41e0-87e0-659dbf7bbb5c tempest-ServersAaction247Test-725202529 tempest-ServersAaction247Test-725202529-project] Request method failure captured: Jul 16 21:19:25.130900 ncdv-master-1 devstack@n-api.service[138680]: request: GET /compute/v2.1/servers/a51adaf1-9c26-47d0-92c2-b6bb065a5ee8 HTTP/1.1 Jul 16 21:19:25.130900 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:19:25.130900 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:19:25.130900 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:19:25.130900 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 0 Jul 16 21:19:25.130900 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:19:25.130900 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:19:25.130900 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:19:25.130900 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:19:25.130900 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0ytOkmdhjfWKFztnZ3_JOFd0i1auBXItG-Ef9kCjRY0Kc_Tp1dDIdbHSri1fyxJewFxyCms9Iuhwau3W3-1Jd_p2So5UPopUgwcnHVE6jy6wwXi3ROlViH97uhXnnueMKQxsoAWzZFxCCqKxqvzSv_4A7Lf3BYA3Q71aAergCKv8Wfw Jul 16 21:19:25.130900 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:19:25.130900 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:19:25.130900 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:19:25.130900 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:19:25.130900 ncdv-master-1 devstack@n-api.service[138680]: X-Openstack-Nova-Api-Version: 2.47 Jul 16 21:19:25.130900 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:19:25.130900 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:19:25.130900 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: f2984288b4cd4610b2f32e895f3e5d72 Jul 16 21:19:25.130900 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-ServersAaction247Test-725202529 Jul 16 21:19:25.130900 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:19:25.130900 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:19:25.130900 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/f2984288b4cd4610b2f32e895f3e5d72"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/f2984288b4cd4610b2f32e895f3e5d72"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_f2984288b4cd4610b2f32e895f3e5d72"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/f2984288b4cd4610b2f32e895f3e5d72"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:19:25.130900 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-ServersAaction247Test-725202529 Jul 16 21:19:25.130900 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: f2984288b4cd4610b2f32e895f3e5d72 Jul 16 21:19:25.130900 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-ServersAaction247Test-725202529 Jul 16 21:19:25.130900 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-ServersAaction247Test-725202529-project Jul 16 21:19:25.130900 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:19:25.130900 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:19:25.130900 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: a8d48b52db1545a4a51027a5c2c7fcce Jul 16 21:19:25.130900 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-ServersAaction247Test-725202529-project Jul 16 21:19:25.130900 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 21:19:25.130900 ncdv-master-1 devstack@n-api.service[138680]: exception: Instance a51adaf1-9c26-47d0-92c2-b6bb065a5ee8 could not be found. Jul 16 21:19:25.130900 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': 'a51adaf1-9c26-47d0-92c2-b6bb065a5ee8'} Jul 16 21:19:25.130900 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:19:25.130900 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:19:25.130900 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:19:25.130900 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:19:25.130900 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:19:25.130900 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:19:25.130900 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:19:25.130900 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:19:25.130900 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:19:25.130900 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:19:25.130900 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance a51adaf1-9c26-47d0-92c2-b6bb065a5ee8 could not be found. Jul 16 21:19:25.130900 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:19:25.130900 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:19:25.130900 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:19:25.130900 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:19:25.130900 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:19:25.130900 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:19:25.130900 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:19:25.130900 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:19:25.130900 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:19:25.130900 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:19:25.130900 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:19:25.130900 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:19:25.130900 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:19:25.130900 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:19:25.130900 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:19:25.130900 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:19:25.130900 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance a51adaf1-9c26-47d0-92c2-b6bb065a5ee8 could not be found. Jul 16 21:19:25.130900 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:19:25.130900 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:19:25.130900 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:19:25.136794 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:19:25.136794 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:19:25.136794 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:19:25.136794 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:19:25.136794 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:19:25.136794 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance a51adaf1-9c26-47d0-92c2-b6bb065a5ee8 could not be found. Jul 16 21:19:25.136794 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:19:25.139972 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-f795f7d6-b71c-41e0-87e0-659dbf7bbb5c tempest-ServersAaction247Test-725202529 tempest-ServersAaction247Test-725202529-project] Returning 404 to user: Instance a51adaf1-9c26-47d0-92c2-b6bb065a5ee8 could not be found. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:19:25.141057 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-f795f7d6-b71c-41e0-87e0-659dbf7bbb5c tempest-ServersAaction247Test-725202529 tempest-ServersAaction247Test-725202529-project] 10.222.0.16 "GET /compute/v2.1/servers/a51adaf1-9c26-47d0-92c2-b6bb065a5ee8" status: 404 len: 111 microversion: 2.47 time: 0.052608 Jul 16 21:19:25.142219 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1356/2677] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:19:25 2022] GET /compute/v2.1/servers/a51adaf1-9c26-47d0-92c2-b6bb065a5ee8 => generated 111 bytes in 55 msecs (HTTP/1.1 404) 9 headers in 381 bytes (2 switches on core 0) Jul 16 21:19:25.261663 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-7616d68d-214b-482a-96a6-4f7057797276 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f5bc1f2209f4eb186d80e385b67c774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '65d5bcfa02af4e8284ba1342197e2860', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:19:25.272223 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-7616d68d-214b-482a-96a6-4f7057797276 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] 10.222.0.16 "GET /compute/v2.1/servers/e7598b4a-f70c-4186-88d1-6fd0057f8656" status: 200 len: 1472 microversion: 2.1 time: 0.350503 Jul 16 21:19:25.275437 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1322/2678] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:19:24 2022] GET /compute/v2.1/servers/e7598b4a-f70c-4186-88d1-6fd0057f8656 => generated 1472 bytes in 354 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:19:25.295602 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-03a3e42d-0016-4caa-9935-b98bf9df6911 req-6ab6b89e-d7bf-4180-9616-778eee9668cc service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "c16e5f09-f813-414e-b233-c08348078624", "name": "network-vif-plugged", "status": "completed", "tag": "882bd182-7d81-4db5-8bdc-c8b2c9c996a4"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:19:25.307309 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-03a3e42d-0016-4caa-9935-b98bf9df6911 req-6ab6b89e-d7bf-4180-9616-778eee9668cc service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:25.308073 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-03a3e42d-0016-4caa-9935-b98bf9df6911 req-6ab6b89e-d7bf-4180-9616-778eee9668cc service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:25.322107 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-03a3e42d-0016-4caa-9935-b98bf9df6911 req-6ab6b89e-d7bf-4180-9616-778eee9668cc service nova] Creating event network-vif-plugged:882bd182-7d81-4db5-8bdc-c8b2c9c996a4 for instance c16e5f09-f813-414e-b233-c08348078624 on nch1-master-1 Jul 16 21:19:25.329226 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-03a3e42d-0016-4caa-9935-b98bf9df6911 req-6ab6b89e-d7bf-4180-9616-778eee9668cc service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.037055 Jul 16 21:19:25.331622 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1357/2679] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:19:25 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:19:25.391412 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-2b0a2003-5420-4ece-b85a-b23f5c832e80 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:19:25.393861 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-2b0a2003-5420-4ece-b85a-b23f5c832e80 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] [instance: b8565a82-250b-4f64-9d90-98b368e4bd57] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:19:25.407786 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2b0a2003-5420-4ece-b85a-b23f5c832e80 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:25.408332 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2b0a2003-5420-4ece-b85a-b23f5c832e80 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:25.409462 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2b0a2003-5420-4ece-b85a-b23f5c832e80 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:25.411172 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2b0a2003-5420-4ece-b85a-b23f5c832e80 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:25.464364 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2b0a2003-5420-4ece-b85a-b23f5c832e80 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:25.464826 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2b0a2003-5420-4ece-b85a-b23f5c832e80 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:25.465449 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.objects.instance [None req-2b0a2003-5420-4ece-b85a-b23f5c832e80 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Lazy-loading 'fault' on Instance uuid b8565a82-250b-4f64-9d90-98b368e4bd57 {{(pid=138681) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:19:25.522274 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-2b0a2003-5420-4ece-b85a-b23f5c832e80 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd0008fffad8e4708b2c01e03da92c47d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e353258f9c3048ef84683294e2b8579d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:19:25.538389 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-2b0a2003-5420-4ece-b85a-b23f5c832e80 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] 10.222.0.16 "GET /compute/v2.1/servers/b8565a82-250b-4f64-9d90-98b368e4bd57" status: 200 len: 1306 microversion: 2.1 time: 0.153097 Jul 16 21:19:25.539621 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1323/2680] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:19:25 2022] GET /compute/v2.1/servers/b8565a82-250b-4f64-9d90-98b368e4bd57 => generated 1306 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:19:25.597529 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-67e3d699-7bdf-4acc-9fa3-ad275be9fe78 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:19:25.598660 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-67e3d699-7bdf-4acc-9fa3-ad275be9fe78 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] [instance: c16e5f09-f813-414e-b233-c08348078624] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:19:25.606790 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-67e3d699-7bdf-4acc-9fa3-ad275be9fe78 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:25.607588 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-67e3d699-7bdf-4acc-9fa3-ad275be9fe78 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:25.608843 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-67e3d699-7bdf-4acc-9fa3-ad275be9fe78 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:25.609440 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-67e3d699-7bdf-4acc-9fa3-ad275be9fe78 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:25.761361 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-67e3d699-7bdf-4acc-9fa3-ad275be9fe78 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eebb224a56814f92b12ebebebc6bc8a3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a092ce9ef40943f6b6ace6cd28481b71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:19:25.769573 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-67e3d699-7bdf-4acc-9fa3-ad275be9fe78 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] 10.222.0.16 "GET /compute/v2.1/servers/c16e5f09-f813-414e-b233-c08348078624" status: 200 len: 1320 microversion: 2.1 time: 0.176378 Jul 16 21:19:25.770680 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1358/2681] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:19:25 2022] GET /compute/v2.1/servers/c16e5f09-f813-414e-b233-c08348078624 => generated 1320 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:19:26.292471 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-41f9f705-4292-44fd-867f-4cc91ea20400 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:19:26.293076 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-41f9f705-4292-44fd-867f-4cc91ea20400 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] [instance: e7598b4a-f70c-4186-88d1-6fd0057f8656] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:19:26.300365 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-41f9f705-4292-44fd-867f-4cc91ea20400 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:26.300823 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-41f9f705-4292-44fd-867f-4cc91ea20400 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:26.301600 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-41f9f705-4292-44fd-867f-4cc91ea20400 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:26.302767 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-41f9f705-4292-44fd-867f-4cc91ea20400 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:26.489383 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-41f9f705-4292-44fd-867f-4cc91ea20400 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f5bc1f2209f4eb186d80e385b67c774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '65d5bcfa02af4e8284ba1342197e2860', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:19:26.497118 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-41f9f705-4292-44fd-867f-4cc91ea20400 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] 10.222.0.16 "GET /compute/v2.1/servers/e7598b4a-f70c-4186-88d1-6fd0057f8656" status: 200 len: 1472 microversion: 2.1 time: 0.208684 Jul 16 21:19:26.498020 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1324/2682] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:19:26 2022] GET /compute/v2.1/servers/e7598b4a-f70c-4186-88d1-6fd0057f8656 => generated 1472 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:19:26.557813 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-c7087a36-43db-4a1f-8bab-ecd60cd3bdf5 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:19:26.559066 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-c7087a36-43db-4a1f-8bab-ecd60cd3bdf5 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] [instance: b8565a82-250b-4f64-9d90-98b368e4bd57] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:19:26.567418 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c7087a36-43db-4a1f-8bab-ecd60cd3bdf5 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:26.567863 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c7087a36-43db-4a1f-8bab-ecd60cd3bdf5 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:26.568681 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c7087a36-43db-4a1f-8bab-ecd60cd3bdf5 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:26.569121 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c7087a36-43db-4a1f-8bab-ecd60cd3bdf5 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:26.596975 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-c7087a36-43db-4a1f-8bab-ecd60cd3bdf5 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] HTTP exception thrown: Instance b8565a82-250b-4f64-9d90-98b368e4bd57 could not be found. Jul 16 21:19:26.598897 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-c7087a36-43db-4a1f-8bab-ecd60cd3bdf5 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Request method failure captured: Jul 16 21:19:26.598897 ncdv-master-1 devstack@n-api.service[138680]: request: GET /compute/v2.1/servers/b8565a82-250b-4f64-9d90-98b368e4bd57 HTTP/1.1 Jul 16 21:19:26.598897 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:19:26.598897 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:19:26.598897 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:19:26.598897 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 0 Jul 16 21:19:26.598897 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:19:26.598897 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:19:26.598897 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:19:26.598897 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:19:26.598897 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0yrk-HsfYKsyovLSaEtNxaoXy7a3rLGNlsRPiPP76ZvvrKkKyGd-uekosXQxpFgkRnItBdviN4xr49Rerg5AyvLSzsyRY-Um0r3BRAvyZrzmCDYUpWbXk1uyEIb3Af4U3Y3nXIJgJAuDVx9z2NGSvEGCsGz12WlyyGCNfVSADFcYJP8 Jul 16 21:19:26.598897 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:19:26.598897 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:19:26.598897 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:19:26.598897 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:19:26.598897 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:19:26.598897 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:19:26.598897 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: e353258f9c3048ef84683294e2b8579d Jul 16 21:19:26.598897 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-SecurityGroupsTestJSON-438340169 Jul 16 21:19:26.598897 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:19:26.598897 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:19:26.598897 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/e353258f9c3048ef84683294e2b8579d"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/e353258f9c3048ef84683294e2b8579d"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_e353258f9c3048ef84683294e2b8579d"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/e353258f9c3048ef84683294e2b8579d"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:19:26.598897 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-SecurityGroupsTestJSON-438340169 Jul 16 21:19:26.598897 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: e353258f9c3048ef84683294e2b8579d Jul 16 21:19:26.598897 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-SecurityGroupsTestJSON-438340169 Jul 16 21:19:26.598897 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-SecurityGroupsTestJSON-438340169-project Jul 16 21:19:26.598897 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:19:26.598897 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:19:26.598897 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: d0008fffad8e4708b2c01e03da92c47d Jul 16 21:19:26.598897 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-SecurityGroupsTestJSON-438340169-project Jul 16 21:19:26.598897 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 21:19:26.598897 ncdv-master-1 devstack@n-api.service[138680]: exception: Instance b8565a82-250b-4f64-9d90-98b368e4bd57 could not be found. Jul 16 21:19:26.598897 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': 'b8565a82-250b-4f64-9d90-98b368e4bd57'} Jul 16 21:19:26.598897 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:19:26.598897 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:19:26.598897 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:19:26.598897 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:19:26.598897 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:19:26.598897 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:19:26.598897 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:19:26.598897 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:19:26.598897 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:19:26.598897 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:19:26.598897 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance b8565a82-250b-4f64-9d90-98b368e4bd57 could not be found. Jul 16 21:19:26.598897 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:19:26.598897 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:19:26.598897 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:19:26.598897 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:19:26.598897 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:19:26.598897 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:19:26.598897 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:19:26.598897 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:19:26.598897 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:19:26.598897 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:19:26.598897 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:19:26.598897 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:19:26.598897 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:19:26.598897 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:19:26.598897 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:19:26.598897 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:19:26.598897 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance b8565a82-250b-4f64-9d90-98b368e4bd57 could not be found. Jul 16 21:19:26.598897 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:19:26.598897 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:19:26.598897 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:19:26.604338 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:19:26.604338 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:19:26.604338 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:19:26.604338 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:19:26.604338 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:19:26.604338 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance b8565a82-250b-4f64-9d90-98b368e4bd57 could not be found. Jul 16 21:19:26.604338 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:19:26.605187 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-c7087a36-43db-4a1f-8bab-ecd60cd3bdf5 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Returning 404 to user: Instance b8565a82-250b-4f64-9d90-98b368e4bd57 could not be found. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:19:26.607219 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-c7087a36-43db-4a1f-8bab-ecd60cd3bdf5 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] 10.222.0.16 "GET /compute/v2.1/servers/b8565a82-250b-4f64-9d90-98b368e4bd57" status: 404 len: 111 microversion: 2.1 time: 0.054360 Jul 16 21:19:26.608406 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1359/2683] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:19:26 2022] GET /compute/v2.1/servers/b8565a82-250b-4f64-9d90-98b368e4bd57 => generated 111 bytes in 56 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:19:26.617242 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-73347e7d-17f2-4d70-b593-b02afa88d3df tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Calling method '.version_select at 0x7fdd9d8c5e50>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:19:26.788812 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-db59097a-7d33-40b4-8329-8723714f36fc tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:19:26.789448 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-db59097a-7d33-40b4-8329-8723714f36fc tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] [instance: c16e5f09-f813-414e-b233-c08348078624] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:19:26.800321 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-db59097a-7d33-40b4-8329-8723714f36fc tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:26.800783 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-db59097a-7d33-40b4-8329-8723714f36fc tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:26.806322 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-db59097a-7d33-40b4-8329-8723714f36fc tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:26.806322 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-db59097a-7d33-40b4-8329-8723714f36fc tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:27.095901 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-db59097a-7d33-40b4-8329-8723714f36fc tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eebb224a56814f92b12ebebebc6bc8a3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a092ce9ef40943f6b6ace6cd28481b71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:19:27.104650 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-db59097a-7d33-40b4-8329-8723714f36fc tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] 10.222.0.16 "GET /compute/v2.1/servers/c16e5f09-f813-414e-b233-c08348078624" status: 200 len: 1320 microversion: 2.1 time: 0.321285 Jul 16 21:19:27.117796 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1360/2684] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:19:26 2022] GET /compute/v2.1/servers/c16e5f09-f813-414e-b233-c08348078624 => generated 1320 bytes in 335 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:19:27.321873 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-73347e7d-17f2-4d70-b593-b02afa88d3df tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] 10.222.0.16 "DELETE /compute/v2.1/os-security-groups/ca1da2f5-509d-4012-b094-b91b1b3d0f60" status: 202 len: 0 microversion: 2.1 time: 0.708838 Jul 16 21:19:27.322482 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1325/2685] 10.222.0.16 () {58 vars in 1271 bytes} [Sat Jul 16 21:19:26 2022] DELETE /compute/v2.1/os-security-groups/ca1da2f5-509d-4012-b094-b91b1b3d0f60 => generated 0 bytes in 710 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 21:19:27.330902 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-6d15d3a7-c954-4f76-b311-707271e9f418 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] Calling method '.version_select at 0x7fdda2b1ae50>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:19:27.517548 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-b1039f8c-f304-458e-89e6-374e5e093d0c tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:19:27.518632 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-b1039f8c-f304-458e-89e6-374e5e093d0c tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] [instance: e7598b4a-f70c-4186-88d1-6fd0057f8656] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:19:27.527801 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b1039f8c-f304-458e-89e6-374e5e093d0c tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:27.529071 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b1039f8c-f304-458e-89e6-374e5e093d0c tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:27.530057 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b1039f8c-f304-458e-89e6-374e5e093d0c tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:27.530679 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b1039f8c-f304-458e-89e6-374e5e093d0c tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:27.825263 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-b1039f8c-f304-458e-89e6-374e5e093d0c tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f5bc1f2209f4eb186d80e385b67c774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '65d5bcfa02af4e8284ba1342197e2860', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:19:27.834040 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-b1039f8c-f304-458e-89e6-374e5e093d0c tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] 10.222.0.16 "GET /compute/v2.1/servers/e7598b4a-f70c-4186-88d1-6fd0057f8656" status: 200 len: 1472 microversion: 2.1 time: 0.321437 Jul 16 21:19:27.845012 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1326/2686] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:19:27 2022] GET /compute/v2.1/servers/e7598b4a-f70c-4186-88d1-6fd0057f8656 => generated 1472 bytes in 334 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:19:27.880287 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-6d15d3a7-c954-4f76-b311-707271e9f418 tempest-SecurityGroupsTestJSON-438340169 tempest-SecurityGroupsTestJSON-438340169-project] 10.222.0.16 "DELETE /compute/v2.1/os-security-groups/83887366-b549-457d-8fc6-28eee153b69e" status: 202 len: 0 microversion: 2.1 time: 0.553454 Jul 16 21:19:27.881040 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1361/2687] 10.222.0.16 () {58 vars in 1271 bytes} [Sat Jul 16 21:19:27 2022] DELETE /compute/v2.1/os-security-groups/83887366-b549-457d-8fc6-28eee153b69e => generated 0 bytes in 555 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 21:19:28.144848 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-30d12958-8ce4-4617-b281-bd91f81a0f02 tempest-AbsoluteLimitsTestJSON-942098090 tempest-AbsoluteLimitsTestJSON-942098090-project] Calling method '.version_select at 0x7fdd9d0b5dc0>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:19:28.149229 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-18bf9e5b-ce7e-437a-a2ad-da1d0752c30d tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:19:28.150382 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-18bf9e5b-ce7e-437a-a2ad-da1d0752c30d tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] [instance: c16e5f09-f813-414e-b233-c08348078624] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:19:28.162707 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-18bf9e5b-ce7e-437a-a2ad-da1d0752c30d tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:28.163212 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-18bf9e5b-ce7e-437a-a2ad-da1d0752c30d tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:28.173754 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-30d12958-8ce4-4617-b281-bd91f81a0f02 tempest-AbsoluteLimitsTestJSON-942098090 tempest-AbsoluteLimitsTestJSON-942098090-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:28.174787 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-18bf9e5b-ce7e-437a-a2ad-da1d0752c30d tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:28.175291 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-18bf9e5b-ce7e-437a-a2ad-da1d0752c30d tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:28.179347 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-30d12958-8ce4-4617-b281-bd91f81a0f02 tempest-AbsoluteLimitsTestJSON-942098090 tempest-AbsoluteLimitsTestJSON-942098090-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:28.192366 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-30d12958-8ce4-4617-b281-bd91f81a0f02 tempest-AbsoluteLimitsTestJSON-942098090 tempest-AbsoluteLimitsTestJSON-942098090-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:28.194410 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-30d12958-8ce4-4617-b281-bd91f81a0f02 tempest-AbsoluteLimitsTestJSON-942098090 tempest-AbsoluteLimitsTestJSON-942098090-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.009s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:28.286599 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-30d12958-8ce4-4617-b281-bd91f81a0f02 tempest-AbsoluteLimitsTestJSON-942098090 tempest-AbsoluteLimitsTestJSON-942098090-project] 10.222.0.16 "GET /compute/v2.1/limits" status: 200 len: 511 microversion: 2.1 time: 0.417399 Jul 16 21:19:28.286599 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1327/2688] 10.222.0.16 () {58 vars in 1121 bytes} [Sat Jul 16 21:19:27 2022] GET /compute/v2.1/limits => generated 511 bytes in 419 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:19:28.529992 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-18bf9e5b-ce7e-437a-a2ad-da1d0752c30d tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eebb224a56814f92b12ebebebc6bc8a3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a092ce9ef40943f6b6ace6cd28481b71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:19:28.540329 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-18bf9e5b-ce7e-437a-a2ad-da1d0752c30d tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] 10.222.0.16 "GET /compute/v2.1/servers/c16e5f09-f813-414e-b233-c08348078624" status: 200 len: 1508 microversion: 2.1 time: 0.405363 Jul 16 21:19:28.544788 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1362/2689] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:19:28 2022] GET /compute/v2.1/servers/c16e5f09-f813-414e-b233-c08348078624 => generated 1508 bytes in 410 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:19:28.561098 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-d97c5706-5acc-4547-8070-190feefdea53 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:19:28.563215 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-d97c5706-5acc-4547-8070-190feefdea53 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] [instance: 21522211-8382-4958-ad2a-cff26f4f2305] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:19:28.573700 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d97c5706-5acc-4547-8070-190feefdea53 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:28.574743 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d97c5706-5acc-4547-8070-190feefdea53 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:28.575846 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d97c5706-5acc-4547-8070-190feefdea53 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:28.576609 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d97c5706-5acc-4547-8070-190feefdea53 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:28.853548 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-93ff3cc4-2613-45ee-8899-95bbdce1b2f6 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:19:28.857166 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-93ff3cc4-2613-45ee-8899-95bbdce1b2f6 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] [instance: e7598b4a-f70c-4186-88d1-6fd0057f8656] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:19:28.861542 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-d97c5706-5acc-4547-8070-190feefdea53 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eebb224a56814f92b12ebebebc6bc8a3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a092ce9ef40943f6b6ace6cd28481b71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:19:28.873080 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-93ff3cc4-2613-45ee-8899-95bbdce1b2f6 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:28.873816 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-93ff3cc4-2613-45ee-8899-95bbdce1b2f6 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:28.875173 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-93ff3cc4-2613-45ee-8899-95bbdce1b2f6 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:28.875988 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-93ff3cc4-2613-45ee-8899-95bbdce1b2f6 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:28.877465 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-d97c5706-5acc-4547-8070-190feefdea53 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] 10.222.0.16 "GET /compute/v2.1/servers/21522211-8382-4958-ad2a-cff26f4f2305" status: 200 len: 1509 microversion: 2.1 time: 0.320752 Jul 16 21:19:28.881719 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1328/2690] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:19:28 2022] GET /compute/v2.1/servers/21522211-8382-4958-ad2a-cff26f4f2305 => generated 1509 bytes in 326 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:19:28.899677 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-5dad3722-1868-4a15-a834-090205c38961 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:19:28.901058 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-5dad3722-1868-4a15-a834-090205c38961 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] [instance: 21522211-8382-4958-ad2a-cff26f4f2305] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:19:28.911307 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5dad3722-1868-4a15-a834-090205c38961 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:28.914457 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5dad3722-1868-4a15-a834-090205c38961 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:28.915580 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5dad3722-1868-4a15-a834-090205c38961 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:28.916254 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5dad3722-1868-4a15-a834-090205c38961 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:28.977586 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-5dad3722-1868-4a15-a834-090205c38961 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] [instance: 21522211-8382-4958-ad2a-cff26f4f2305] Going to try to terminate instance {{(pid=138681) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 16 21:19:29.175506 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-93ff3cc4-2613-45ee-8899-95bbdce1b2f6 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f5bc1f2209f4eb186d80e385b67c774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '65d5bcfa02af4e8284ba1342197e2860', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:19:29.186397 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-93ff3cc4-2613-45ee-8899-95bbdce1b2f6 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] 10.222.0.16 "GET /compute/v2.1/servers/e7598b4a-f70c-4186-88d1-6fd0057f8656" status: 200 len: 1472 microversion: 2.1 time: 0.337859 Jul 16 21:19:29.196972 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1363/2691] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:19:28 2022] GET /compute/v2.1/servers/e7598b4a-f70c-4186-88d1-6fd0057f8656 => generated 1472 bytes in 349 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:19:29.455349 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-5dad3722-1868-4a15-a834-090205c38961 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] 10.222.0.16 "DELETE /compute/v2.1/servers/21522211-8382-4958-ad2a-cff26f4f2305" status: 204 len: 0 microversion: 2.1 time: 0.560074 Jul 16 21:19:29.455349 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1329/2692] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:19:28 2022] DELETE /compute/v2.1/servers/21522211-8382-4958-ad2a-cff26f4f2305 => generated 0 bytes in 562 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 16 21:19:29.464406 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-2cd021c1-1901-4fec-bd2f-33b370658862 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:19:29.465821 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-2cd021c1-1901-4fec-bd2f-33b370658862 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] [instance: c16e5f09-f813-414e-b233-c08348078624] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:19:29.473505 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2cd021c1-1901-4fec-bd2f-33b370658862 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:29.474145 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2cd021c1-1901-4fec-bd2f-33b370658862 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:29.475178 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2cd021c1-1901-4fec-bd2f-33b370658862 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:29.476359 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2cd021c1-1901-4fec-bd2f-33b370658862 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:29.515330 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-2cd021c1-1901-4fec-bd2f-33b370658862 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] [instance: c16e5f09-f813-414e-b233-c08348078624] Going to try to terminate instance {{(pid=138680) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 16 21:19:29.875241 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_api:SELECT=99,nova_api:UPDATE=1 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:19:29.968080 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-2cd021c1-1901-4fec-bd2f-33b370658862 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] 10.222.0.16 "DELETE /compute/v2.1/servers/c16e5f09-f813-414e-b233-c08348078624" status: 204 len: 0 microversion: 2.1 time: 0.509315 Jul 16 21:19:29.968681 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1364/2693] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:19:29 2022] DELETE /compute/v2.1/servers/c16e5f09-f813-414e-b233-c08348078624 => generated 0 bytes in 511 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 16 21:19:30.169884 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-ac0d1ed2-52b4-4968-842b-2cd93aa37537 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-MultipleCreateTestJSON-server-2114837005", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452", "min_count": 1, "max_count": 2, "return_reservation_id": true, "networks": [{"uuid": "f5bb1e92-80f6-44be-a5b9-c44a678f4b55"}]}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:19:30.210537 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-9a505cd1-8ec4-4cdf-be5c-244801c82c73 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:19:30.214613 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-9a505cd1-8ec4-4cdf-be5c-244801c82c73 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] [instance: e7598b4a-f70c-4186-88d1-6fd0057f8656] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:19:30.228359 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9a505cd1-8ec4-4cdf-be5c-244801c82c73 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:30.245253 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9a505cd1-8ec4-4cdf-be5c-244801c82c73 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:30.250544 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9a505cd1-8ec4-4cdf-be5c-244801c82c73 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:30.250544 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9a505cd1-8ec4-4cdf-be5c-244801c82c73 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:30.311702 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.network.neutron [None req-ac0d1ed2-52b4-4968-842b-2cd93aa37537 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] validate_networks() for [('f5bb1e92-80f6-44be-a5b9-c44a678f4b55', None, None, None, None, None)] {{(pid=138681) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 16 21:19:30.682406 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-9a505cd1-8ec4-4cdf-be5c-244801c82c73 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f5bc1f2209f4eb186d80e385b67c774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '65d5bcfa02af4e8284ba1342197e2860', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:19:30.690399 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-9a505cd1-8ec4-4cdf-be5c-244801c82c73 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] 10.222.0.16 "GET /compute/v2.1/servers/e7598b4a-f70c-4186-88d1-6fd0057f8656" status: 200 len: 1472 microversion: 2.1 time: 0.490105 Jul 16 21:19:30.701767 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1365/2694] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:19:30 2022] GET /compute/v2.1/servers/e7598b4a-f70c-4186-88d1-6fd0057f8656 => generated 1472 bytes in 502 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:19:31.389564 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-ac0d1ed2-52b4-4968-842b-2cd93aa37537 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Flavor limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 16 21:19:31.390016 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-ac0d1ed2-52b4-4968-842b-2cd93aa37537 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Image limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 16 21:19:31.390454 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-ac0d1ed2-52b4-4968-842b-2cd93aa37537 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Flavor pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 16 21:19:31.391073 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-ac0d1ed2-52b4-4968-842b-2cd93aa37537 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Image pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 16 21:19:31.391748 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-ac0d1ed2-52b4-4968-842b-2cd93aa37537 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 16 21:19:31.406629 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ac0d1ed2-52b4-4968-842b-2cd93aa37537 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:31.406971 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ac0d1ed2-52b4-4968-842b-2cd93aa37537 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:31.416761 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ac0d1ed2-52b4-4968-842b-2cd93aa37537 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:31.417222 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ac0d1ed2-52b4-4968-842b-2cd93aa37537 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:31.456486 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-ac0d1ed2-52b4-4968-842b-2cd93aa37537 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Getting quotas for project a092ce9ef40943f6b6ace6cd28481b71. Resources: {'ram', 'instances', 'cores'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:19:31.470810 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-ac0d1ed2-52b4-4968-842b-2cd93aa37537 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Getting quotas for user eebb224a56814f92b12ebebebc6bc8a3 and project a092ce9ef40943f6b6ace6cd28481b71. Resources: {'ram', 'instances', 'cores'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:19:31.511855 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-ac0d1ed2-52b4-4968-842b-2cd93aa37537 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Going to run 2 instances... {{(pid=138681) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 16 21:19:31.516194 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-ac0d1ed2-52b4-4968-842b-2cd93aa37537 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] [instance: 9d2cd0fc-1101-4e21-8179-c1dd4605594f] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138681) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 16 21:19:31.683357 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-ac0d1ed2-52b4-4968-842b-2cd93aa37537 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] [instance: a56f8463-6006-4805-9021-1babc2d98f0f] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9',instance=,instance_uuid=9d2cd0fc-1101-4e21-8179-c1dd4605594f,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138681) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 16 21:19:31.711192 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-dcaca190-d147-496a-aa53-1ea306c09efc tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:19:31.714026 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-dcaca190-d147-496a-aa53-1ea306c09efc tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] [instance: e7598b4a-f70c-4186-88d1-6fd0057f8656] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:19:31.726283 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-dcaca190-d147-496a-aa53-1ea306c09efc tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:31.727062 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-dcaca190-d147-496a-aa53-1ea306c09efc tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:31.728197 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-dcaca190-d147-496a-aa53-1ea306c09efc tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:31.729029 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-dcaca190-d147-496a-aa53-1ea306c09efc tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:31.960857 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-ac0d1ed2-52b4-4968-842b-2cd93aa37537 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 32 microversion: 2.1 time: 1.787451 Jul 16 21:19:31.961569 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1330/2695] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 21:19:30 2022] POST /compute/v2.1/servers => generated 32 bytes in 1800 msecs (HTTP/1.1 202) 9 headers in 362 bytes (1 switches on core 0) Jul 16 21:19:31.974821 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-39359617-be23-40c0-9383-693fdcb960e6 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:19:32.009596 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-39359617-be23-40c0-9383-693fdcb960e6 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Policy check for os_compute_api:servers:allow_all_filters failed with credentials {'is_admin': False, 'user_id': 'eebb224a56814f92b12ebebebc6bc8a3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a092ce9ef40943f6b6ace6cd28481b71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:19:32.012261 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-39359617-be23-40c0-9383-693fdcb960e6 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Searching by: {'deleted': False, 'project_id': 'a092ce9ef40943f6b6ace6cd28481b71'} {{(pid=138681) get_all /opt/stack/nova/nova/compute/api.py:2962}} Jul 16 21:19:32.034217 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-39359617-be23-40c0-9383-693fdcb960e6 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:32.035076 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-39359617-be23-40c0-9383-693fdcb960e6 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:32.037452 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-39359617-be23-40c0-9383-693fdcb960e6 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:32.038092 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-39359617-be23-40c0-9383-693fdcb960e6 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:32.090805 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.multi_cell_list [None req-39359617-be23-40c0-9383-693fdcb960e6 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Listed batch of 2 results from cell out of 998 limit. Returned 2 total so far. {{(pid=138681) do_query /opt/stack/nova/nova/compute/multi_cell_list.py:373}} Jul 16 21:19:32.196425 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-39359617-be23-40c0-9383-693fdcb960e6 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] 10.222.0.16 "GET /compute/v2.1/servers" status: 200 len: 1337 microversion: 2.1 time: 0.225397 Jul 16 21:19:32.203286 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1331/2696] 10.222.0.16 () {58 vars in 1124 bytes} [Sat Jul 16 21:19:31 2022] GET /compute/v2.1/servers => generated 1337 bytes in 237 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:19:32.212704 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-e2c9d033-8175-478e-86f7-396277073a87 req-bb1c181c-204e-45c8-b880-bac5eb616538 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "21522211-8382-4958-ad2a-cff26f4f2305", "name": "network-vif-deleted", "tag": "9805a0f5-a6ed-4cb8-ac51-b1d03a7a9ec6"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:19:32.239328 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-e2c9d033-8175-478e-86f7-396277073a87 req-bb1c181c-204e-45c8-b880-bac5eb616538 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:32.239328 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-e2c9d033-8175-478e-86f7-396277073a87 req-bb1c181c-204e-45c8-b880-bac5eb616538 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:32.269593 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-e2c9d033-8175-478e-86f7-396277073a87 req-bb1c181c-204e-45c8-b880-bac5eb616538 service nova] Creating event network-vif-deleted:9805a0f5-a6ed-4cb8-ac51-b1d03a7a9ec6 for instance 21522211-8382-4958-ad2a-cff26f4f2305 on nch2-master-1 Jul 16 21:19:32.274175 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-e2c9d033-8175-478e-86f7-396277073a87 req-bb1c181c-204e-45c8-b880-bac5eb616538 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.069513 Jul 16 21:19:32.275613 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1332/2697] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:19:32 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 72 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:19:32.298729 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-8fa32593-17b6-4427-89d9-7a4d0dfd89e0 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:19:32.298729 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-8fa32593-17b6-4427-89d9-7a4d0dfd89e0 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] [instance: a56f8463-6006-4805-9021-1babc2d98f0f] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:19:32.545532 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-8fa32593-17b6-4427-89d9-7a4d0dfd89e0 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eebb224a56814f92b12ebebebc6bc8a3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a092ce9ef40943f6b6ace6cd28481b71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:19:32.555105 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-8fa32593-17b6-4427-89d9-7a4d0dfd89e0 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] 10.222.0.16 "GET /compute/v2.1/servers/a56f8463-6006-4805-9021-1babc2d98f0f" status: 200 len: 1218 microversion: 2.1 time: 0.277243 Jul 16 21:19:32.555966 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1333/2698] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:19:32 2022] GET /compute/v2.1/servers/a56f8463-6006-4805-9021-1babc2d98f0f => generated 1218 bytes in 280 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:19:32.692156 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-a7288c2f-e193-4351-bf1f-19799b338bb4 req-f7202eda-b3cd-4e69-9257-d7e5c55bca54 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "c16e5f09-f813-414e-b233-c08348078624", "name": "network-vif-deleted", "tag": "882bd182-7d81-4db5-8bdc-c8b2c9c996a4"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:19:32.710915 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-a7288c2f-e193-4351-bf1f-19799b338bb4 req-f7202eda-b3cd-4e69-9257-d7e5c55bca54 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:32.711597 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-a7288c2f-e193-4351-bf1f-19799b338bb4 req-f7202eda-b3cd-4e69-9257-d7e5c55bca54 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:32.720957 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-dcaca190-d147-496a-aa53-1ea306c09efc tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f5bc1f2209f4eb186d80e385b67c774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '65d5bcfa02af4e8284ba1342197e2860', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:19:32.731535 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-dcaca190-d147-496a-aa53-1ea306c09efc tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] 10.222.0.16 "GET /compute/v2.1/servers/e7598b4a-f70c-4186-88d1-6fd0057f8656" status: 200 len: 1472 microversion: 2.1 time: 1.026419 Jul 16 21:19:32.732592 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1366/2699] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:19:31 2022] GET /compute/v2.1/servers/e7598b4a-f70c-4186-88d1-6fd0057f8656 => generated 1472 bytes in 1028 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:19:32.750219 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-a7288c2f-e193-4351-bf1f-19799b338bb4 req-f7202eda-b3cd-4e69-9257-d7e5c55bca54 service nova] Creating event network-vif-deleted:882bd182-7d81-4db5-8bdc-c8b2c9c996a4 for instance c16e5f09-f813-414e-b233-c08348078624 on nch1-master-1 Jul 16 21:19:32.759549 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-a7288c2f-e193-4351-bf1f-19799b338bb4 req-f7202eda-b3cd-4e69-9257-d7e5c55bca54 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.072067 Jul 16 21:19:32.760180 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1334/2700] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:19:32 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 74 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:19:33.577513 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-695aa728-b882-4c45-81c1-49bfba191789 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:19:33.579987 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-695aa728-b882-4c45-81c1-49bfba191789 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] [instance: a56f8463-6006-4805-9021-1babc2d98f0f] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:19:33.707568 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-695aa728-b882-4c45-81c1-49bfba191789 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eebb224a56814f92b12ebebebc6bc8a3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a092ce9ef40943f6b6ace6cd28481b71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:19:33.720889 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-695aa728-b882-4c45-81c1-49bfba191789 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] 10.222.0.16 "GET /compute/v2.1/servers/a56f8463-6006-4805-9021-1babc2d98f0f" status: 200 len: 1218 microversion: 2.1 time: 0.148622 Jul 16 21:19:33.721778 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1367/2701] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:19:33 2022] GET /compute/v2.1/servers/a56f8463-6006-4805-9021-1babc2d98f0f => generated 1218 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:19:33.750332 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-17000a39-b37c-4e3e-b08e-b81a7e242242 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:19:33.752673 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-17000a39-b37c-4e3e-b08e-b81a7e242242 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] [instance: e7598b4a-f70c-4186-88d1-6fd0057f8656] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:19:33.765511 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-17000a39-b37c-4e3e-b08e-b81a7e242242 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:33.766018 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-17000a39-b37c-4e3e-b08e-b81a7e242242 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:33.766836 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-17000a39-b37c-4e3e-b08e-b81a7e242242 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:33.767301 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-17000a39-b37c-4e3e-b08e-b81a7e242242 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:34.143102 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-17000a39-b37c-4e3e-b08e-b81a7e242242 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f5bc1f2209f4eb186d80e385b67c774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '65d5bcfa02af4e8284ba1342197e2860', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:19:34.155298 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-17000a39-b37c-4e3e-b08e-b81a7e242242 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] 10.222.0.16 "GET /compute/v2.1/servers/e7598b4a-f70c-4186-88d1-6fd0057f8656" status: 200 len: 1472 microversion: 2.1 time: 0.409612 Jul 16 21:19:34.156570 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1335/2702] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:19:33 2022] GET /compute/v2.1/servers/e7598b4a-f70c-4186-88d1-6fd0057f8656 => generated 1472 bytes in 412 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:19:34.737717 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-99e98718-9803-4d41-b23a-b2fc189ffa9e tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:19:34.739117 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-99e98718-9803-4d41-b23a-b2fc189ffa9e tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] [instance: a56f8463-6006-4805-9021-1babc2d98f0f] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:19:34.842413 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-99e98718-9803-4d41-b23a-b2fc189ffa9e tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eebb224a56814f92b12ebebebc6bc8a3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a092ce9ef40943f6b6ace6cd28481b71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:19:34.851863 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-99e98718-9803-4d41-b23a-b2fc189ffa9e tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] 10.222.0.16 "GET /compute/v2.1/servers/a56f8463-6006-4805-9021-1babc2d98f0f" status: 200 len: 1218 microversion: 2.1 time: 0.117417 Jul 16 21:19:34.860499 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1368/2703] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:19:34 2022] GET /compute/v2.1/servers/a56f8463-6006-4805-9021-1babc2d98f0f => generated 1218 bytes in 127 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:19:35.180485 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-d3f753c9-019b-4b98-b46a-6ec081c22293 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:19:35.180485 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-d3f753c9-019b-4b98-b46a-6ec081c22293 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] [instance: e7598b4a-f70c-4186-88d1-6fd0057f8656] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:19:35.190060 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d3f753c9-019b-4b98-b46a-6ec081c22293 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:35.196150 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d3f753c9-019b-4b98-b46a-6ec081c22293 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:35.197132 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d3f753c9-019b-4b98-b46a-6ec081c22293 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:35.197734 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d3f753c9-019b-4b98-b46a-6ec081c22293 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:35.202586 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:19:35.557195 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-d3f753c9-019b-4b98-b46a-6ec081c22293 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f5bc1f2209f4eb186d80e385b67c774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '65d5bcfa02af4e8284ba1342197e2860', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:19:35.564702 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-d3f753c9-019b-4b98-b46a-6ec081c22293 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] 10.222.0.16 "GET /compute/v2.1/servers/e7598b4a-f70c-4186-88d1-6fd0057f8656" status: 200 len: 1472 microversion: 2.1 time: 0.394827 Jul 16 21:19:35.565727 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1336/2704] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:19:35 2022] GET /compute/v2.1/servers/e7598b4a-f70c-4186-88d1-6fd0057f8656 => generated 1472 bytes in 396 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:19:35.888837 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-22688a48-0409-4ad7-a8fc-e187f3a7fd85 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:19:35.890916 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-22688a48-0409-4ad7-a8fc-e187f3a7fd85 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] [instance: a56f8463-6006-4805-9021-1babc2d98f0f] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:19:35.909575 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-22688a48-0409-4ad7-a8fc-e187f3a7fd85 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:35.913205 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-22688a48-0409-4ad7-a8fc-e187f3a7fd85 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:35.914674 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-22688a48-0409-4ad7-a8fc-e187f3a7fd85 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:35.915154 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-22688a48-0409-4ad7-a8fc-e187f3a7fd85 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:36.153157 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-22688a48-0409-4ad7-a8fc-e187f3a7fd85 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eebb224a56814f92b12ebebebc6bc8a3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a092ce9ef40943f6b6ace6cd28481b71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:19:36.176180 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-22688a48-0409-4ad7-a8fc-e187f3a7fd85 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] 10.222.0.16 "GET /compute/v2.1/servers/a56f8463-6006-4805-9021-1babc2d98f0f" status: 200 len: 1222 microversion: 2.1 time: 0.299792 Jul 16 21:19:36.177266 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1369/2705] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:19:35 2022] GET /compute/v2.1/servers/a56f8463-6006-4805-9021-1babc2d98f0f => generated 1222 bytes in 302 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:19:36.585640 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-512fd37c-9dbc-46aa-b9a4-3805cc104553 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:19:36.587210 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-512fd37c-9dbc-46aa-b9a4-3805cc104553 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] [instance: e7598b4a-f70c-4186-88d1-6fd0057f8656] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:19:36.601205 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-512fd37c-9dbc-46aa-b9a4-3805cc104553 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:36.601888 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-512fd37c-9dbc-46aa-b9a4-3805cc104553 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:36.602937 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-512fd37c-9dbc-46aa-b9a4-3805cc104553 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:36.603780 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-512fd37c-9dbc-46aa-b9a4-3805cc104553 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:37.196814 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-032fc829-a44b-4d7e-b880-1f8aaeb2dc08 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:19:37.199243 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-032fc829-a44b-4d7e-b880-1f8aaeb2dc08 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] [instance: a56f8463-6006-4805-9021-1babc2d98f0f] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:19:37.217323 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-032fc829-a44b-4d7e-b880-1f8aaeb2dc08 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:37.217820 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-032fc829-a44b-4d7e-b880-1f8aaeb2dc08 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:37.237073 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-032fc829-a44b-4d7e-b880-1f8aaeb2dc08 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:37.240188 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-032fc829-a44b-4d7e-b880-1f8aaeb2dc08 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.016s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:37.446623 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-032fc829-a44b-4d7e-b880-1f8aaeb2dc08 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eebb224a56814f92b12ebebebc6bc8a3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a092ce9ef40943f6b6ace6cd28481b71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:19:37.462113 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-032fc829-a44b-4d7e-b880-1f8aaeb2dc08 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] 10.222.0.16 "GET /compute/v2.1/servers/a56f8463-6006-4805-9021-1babc2d98f0f" status: 200 len: 1270 microversion: 2.1 time: 0.271063 Jul 16 21:19:37.463070 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1370/2706] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:19:37 2022] GET /compute/v2.1/servers/a56f8463-6006-4805-9021-1babc2d98f0f => generated 1270 bytes in 274 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:19:37.895755 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-512fd37c-9dbc-46aa-b9a4-3805cc104553 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f5bc1f2209f4eb186d80e385b67c774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '65d5bcfa02af4e8284ba1342197e2860', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:19:37.912460 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-512fd37c-9dbc-46aa-b9a4-3805cc104553 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] 10.222.0.16 "GET /compute/v2.1/servers/e7598b4a-f70c-4186-88d1-6fd0057f8656" status: 200 len: 1478 microversion: 2.1 time: 1.332528 Jul 16 21:19:37.914010 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1337/2707] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:19:36 2022] GET /compute/v2.1/servers/e7598b4a-f70c-4186-88d1-6fd0057f8656 => generated 1478 bytes in 1336 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:19:38.481255 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-bcb879b0-26f8-4bdc-8c6a-ac2330472307 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:19:38.482101 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-bcb879b0-26f8-4bdc-8c6a-ac2330472307 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] [instance: a56f8463-6006-4805-9021-1babc2d98f0f] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:19:38.520745 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-bcb879b0-26f8-4bdc-8c6a-ac2330472307 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:38.521455 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-bcb879b0-26f8-4bdc-8c6a-ac2330472307 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:38.522957 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-bcb879b0-26f8-4bdc-8c6a-ac2330472307 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:38.523696 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-bcb879b0-26f8-4bdc-8c6a-ac2330472307 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:38.859124 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-bcb879b0-26f8-4bdc-8c6a-ac2330472307 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eebb224a56814f92b12ebebebc6bc8a3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a092ce9ef40943f6b6ace6cd28481b71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:19:38.875151 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-bcb879b0-26f8-4bdc-8c6a-ac2330472307 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] 10.222.0.16 "GET /compute/v2.1/servers/a56f8463-6006-4805-9021-1babc2d98f0f" status: 200 len: 1288 microversion: 2.1 time: 0.394260 Jul 16 21:19:38.890639 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1371/2708] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:19:38 2022] GET /compute/v2.1/servers/a56f8463-6006-4805-9021-1babc2d98f0f => generated 1288 bytes in 415 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:19:38.933700 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-14ed1ef9-caac-4e3d-9e3c-694e0ac3bab5 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:19:38.934320 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-14ed1ef9-caac-4e3d-9e3c-694e0ac3bab5 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] [instance: e7598b4a-f70c-4186-88d1-6fd0057f8656] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:19:38.955412 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-14ed1ef9-caac-4e3d-9e3c-694e0ac3bab5 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:38.967178 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-14ed1ef9-caac-4e3d-9e3c-694e0ac3bab5 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.011s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:38.971276 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-14ed1ef9-caac-4e3d-9e3c-694e0ac3bab5 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:38.971276 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-14ed1ef9-caac-4e3d-9e3c-694e0ac3bab5 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:39.607287 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-14ed1ef9-caac-4e3d-9e3c-694e0ac3bab5 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f5bc1f2209f4eb186d80e385b67c774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '65d5bcfa02af4e8284ba1342197e2860', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:19:39.631616 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-14ed1ef9-caac-4e3d-9e3c-694e0ac3bab5 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] 10.222.0.16 "GET /compute/v2.1/servers/e7598b4a-f70c-4186-88d1-6fd0057f8656" status: 200 len: 1478 microversion: 2.1 time: 0.692412 Jul 16 21:19:39.638239 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1338/2709] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:19:38 2022] GET /compute/v2.1/servers/e7598b4a-f70c-4186-88d1-6fd0057f8656 => generated 1478 bytes in 712 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:19:39.895357 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-f13f66db-d60a-471d-8631-c6fa47c39484 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:19:39.896178 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-f13f66db-d60a-471d-8631-c6fa47c39484 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] [instance: a56f8463-6006-4805-9021-1babc2d98f0f] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:19:39.907451 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f13f66db-d60a-471d-8631-c6fa47c39484 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:39.912414 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f13f66db-d60a-471d-8631-c6fa47c39484 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:39.915039 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f13f66db-d60a-471d-8631-c6fa47c39484 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:39.922497 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f13f66db-d60a-471d-8631-c6fa47c39484 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:40.529856 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-f13f66db-d60a-471d-8631-c6fa47c39484 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eebb224a56814f92b12ebebebc6bc8a3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a092ce9ef40943f6b6ace6cd28481b71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:19:40.555176 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-f13f66db-d60a-471d-8631-c6fa47c39484 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] 10.222.0.16 "GET /compute/v2.1/servers/a56f8463-6006-4805-9021-1babc2d98f0f" status: 200 len: 1276 microversion: 2.1 time: 0.664145 Jul 16 21:19:40.561248 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1372/2710] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:19:39 2022] GET /compute/v2.1/servers/a56f8463-6006-4805-9021-1babc2d98f0f => generated 1276 bytes in 671 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:19:40.640007 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-535764b2-a949-45ae-a921-a4a2ba9e1f60 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:19:40.641240 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-535764b2-a949-45ae-a921-a4a2ba9e1f60 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] [instance: e7598b4a-f70c-4186-88d1-6fd0057f8656] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:19:40.649303 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-535764b2-a949-45ae-a921-a4a2ba9e1f60 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:40.649781 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-535764b2-a949-45ae-a921-a4a2ba9e1f60 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:40.650558 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-535764b2-a949-45ae-a921-a4a2ba9e1f60 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:40.651025 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-535764b2-a949-45ae-a921-a4a2ba9e1f60 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:41.089512 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-535764b2-a949-45ae-a921-a4a2ba9e1f60 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f5bc1f2209f4eb186d80e385b67c774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '65d5bcfa02af4e8284ba1342197e2860', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:19:41.115187 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-535764b2-a949-45ae-a921-a4a2ba9e1f60 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] 10.222.0.16 "GET /compute/v2.1/servers/e7598b4a-f70c-4186-88d1-6fd0057f8656" status: 200 len: 1463 microversion: 2.1 time: 0.477929 Jul 16 21:19:41.127080 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1339/2711] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:19:40 2022] GET /compute/v2.1/servers/e7598b4a-f70c-4186-88d1-6fd0057f8656 => generated 1463 bytes in 482 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:19:41.148170 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-0cc22659-890f-495f-8749-c96851bba419 tempest-ServersAdminTestJSON-1511367024 tempest-ServersAdminTestJSON-1511367024-project-admin] Action: 'action', calling method: >, body: {"os-resetState": {"state": "error"}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:19:41.150442 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-0cc22659-890f-495f-8749-c96851bba419 tempest-ServersAdminTestJSON-1511367024 tempest-ServersAdminTestJSON-1511367024-project-admin] [instance: e7598b4a-f70c-4186-88d1-6fd0057f8656] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:19:41.170170 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0cc22659-890f-495f-8749-c96851bba419 tempest-ServersAdminTestJSON-1511367024 tempest-ServersAdminTestJSON-1511367024-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:41.174592 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0cc22659-890f-495f-8749-c96851bba419 tempest-ServersAdminTestJSON-1511367024 tempest-ServersAdminTestJSON-1511367024-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:41.178228 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0cc22659-890f-495f-8749-c96851bba419 tempest-ServersAdminTestJSON-1511367024 tempest-ServersAdminTestJSON-1511367024-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:41.179604 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0cc22659-890f-495f-8749-c96851bba419 tempest-ServersAdminTestJSON-1511367024 tempest-ServersAdminTestJSON-1511367024-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:41.399497 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=96,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:19:41.577879 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-9a6176a8-3139-41cc-be72-15bea8d0460b tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:19:41.578775 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-9a6176a8-3139-41cc-be72-15bea8d0460b tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] [instance: a56f8463-6006-4805-9021-1babc2d98f0f] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:19:41.591527 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9a6176a8-3139-41cc-be72-15bea8d0460b tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:41.595391 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9a6176a8-3139-41cc-be72-15bea8d0460b tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:41.595391 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9a6176a8-3139-41cc-be72-15bea8d0460b tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:41.595391 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9a6176a8-3139-41cc-be72-15bea8d0460b tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:41.637799 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-0cc22659-890f-495f-8749-c96851bba419 tempest-ServersAdminTestJSON-1511367024 tempest-ServersAdminTestJSON-1511367024-project-admin] 10.222.0.16 "POST /compute/v2.1/servers/e7598b4a-f70c-4186-88d1-6fd0057f8656/action" status: 202 len: 0 microversion: 2.1 time: 0.495338 Jul 16 21:19:41.642342 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1373/2712] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 21:19:41 2022] POST /compute/v2.1/servers/e7598b4a-f70c-4186-88d1-6fd0057f8656/action => generated 0 bytes in 501 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 21:19:41.649711 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-8eff224c-cd55-49d4-a83f-c73fcbf6ebd8 tempest-ServersAdminTestJSON-1511367024 tempest-ServersAdminTestJSON-1511367024-project-admin] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:19:41.650626 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-8eff224c-cd55-49d4-a83f-c73fcbf6ebd8 tempest-ServersAdminTestJSON-1511367024 tempest-ServersAdminTestJSON-1511367024-project-admin] [instance: e7598b4a-f70c-4186-88d1-6fd0057f8656] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:19:41.679867 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8eff224c-cd55-49d4-a83f-c73fcbf6ebd8 tempest-ServersAdminTestJSON-1511367024 tempest-ServersAdminTestJSON-1511367024-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:41.683357 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8eff224c-cd55-49d4-a83f-c73fcbf6ebd8 tempest-ServersAdminTestJSON-1511367024 tempest-ServersAdminTestJSON-1511367024-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:41.687789 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8eff224c-cd55-49d4-a83f-c73fcbf6ebd8 tempest-ServersAdminTestJSON-1511367024 tempest-ServersAdminTestJSON-1511367024-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:41.692956 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8eff224c-cd55-49d4-a83f-c73fcbf6ebd8 tempest-ServersAdminTestJSON-1511367024 tempest-ServersAdminTestJSON-1511367024-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:41.833866 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8eff224c-cd55-49d4-a83f-c73fcbf6ebd8 tempest-ServersAdminTestJSON-1511367024 tempest-ServersAdminTestJSON-1511367024-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:41.835431 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8eff224c-cd55-49d4-a83f-c73fcbf6ebd8 tempest-ServersAdminTestJSON-1511367024 tempest-ServersAdminTestJSON-1511367024-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:41.836516 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.objects.instance [None req-8eff224c-cd55-49d4-a83f-c73fcbf6ebd8 tempest-ServersAdminTestJSON-1511367024 tempest-ServersAdminTestJSON-1511367024-project-admin] Lazy-loading 'fault' on Instance uuid e7598b4a-f70c-4186-88d1-6fd0057f8656 {{(pid=138680) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:19:42.056498 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell0:SELECT=6 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:19:42.369106 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-9a6176a8-3139-41cc-be72-15bea8d0460b tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eebb224a56814f92b12ebebebc6bc8a3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a092ce9ef40943f6b6ace6cd28481b71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:19:42.380500 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-9a6176a8-3139-41cc-be72-15bea8d0460b tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] 10.222.0.16 "GET /compute/v2.1/servers/a56f8463-6006-4805-9021-1babc2d98f0f" status: 200 len: 1318 microversion: 2.1 time: 0.806892 Jul 16 21:19:42.395282 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1340/2713] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:19:41 2022] GET /compute/v2.1/servers/a56f8463-6006-4805-9021-1babc2d98f0f => generated 1318 bytes in 822 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:19:42.646875 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-8eff224c-cd55-49d4-a83f-c73fcbf6ebd8 tempest-ServersAdminTestJSON-1511367024 tempest-ServersAdminTestJSON-1511367024-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/e7598b4a-f70c-4186-88d1-6fd0057f8656" status: 200 len: 1597 microversion: 2.1 time: 1.003092 Jul 16 21:19:42.647597 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1374/2714] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:19:41 2022] GET /compute/v2.1/servers/e7598b4a-f70c-4186-88d1-6fd0057f8656 => generated 1597 bytes in 1005 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:19:42.661010 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell0:SELECT=10 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:19:42.669694 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-0f2882f3-53dc-483c-914e-225a549417e4 tempest-ServersAdminTestJSON-1511367024 tempest-ServersAdminTestJSON-1511367024-project-admin] Action: 'action', calling method: >, body: {"os-resetState": {"state": "active"}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:19:42.670975 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-0f2882f3-53dc-483c-914e-225a549417e4 tempest-ServersAdminTestJSON-1511367024 tempest-ServersAdminTestJSON-1511367024-project-admin] [instance: e7598b4a-f70c-4186-88d1-6fd0057f8656] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:19:42.700209 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0f2882f3-53dc-483c-914e-225a549417e4 tempest-ServersAdminTestJSON-1511367024 tempest-ServersAdminTestJSON-1511367024-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:42.700742 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0f2882f3-53dc-483c-914e-225a549417e4 tempest-ServersAdminTestJSON-1511367024 tempest-ServersAdminTestJSON-1511367024-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:42.701549 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0f2882f3-53dc-483c-914e-225a549417e4 tempest-ServersAdminTestJSON-1511367024 tempest-ServersAdminTestJSON-1511367024-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:42.705550 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0f2882f3-53dc-483c-914e-225a549417e4 tempest-ServersAdminTestJSON-1511367024 tempest-ServersAdminTestJSON-1511367024-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:43.169842 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-0f2882f3-53dc-483c-914e-225a549417e4 tempest-ServersAdminTestJSON-1511367024 tempest-ServersAdminTestJSON-1511367024-project-admin] 10.222.0.16 "POST /compute/v2.1/servers/e7598b4a-f70c-4186-88d1-6fd0057f8656/action" status: 202 len: 0 microversion: 2.1 time: 0.509378 Jul 16 21:19:43.174226 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1341/2715] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 21:19:42 2022] POST /compute/v2.1/servers/e7598b4a-f70c-4186-88d1-6fd0057f8656/action => generated 0 bytes in 514 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 21:19:43.180059 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-31eca5f6-e951-4c3e-b1db-4a440ab3b255 tempest-ServersAdminTestJSON-1511367024 tempest-ServersAdminTestJSON-1511367024-project-admin] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:19:43.180829 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-31eca5f6-e951-4c3e-b1db-4a440ab3b255 tempest-ServersAdminTestJSON-1511367024 tempest-ServersAdminTestJSON-1511367024-project-admin] [instance: e7598b4a-f70c-4186-88d1-6fd0057f8656] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:19:43.199674 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-31eca5f6-e951-4c3e-b1db-4a440ab3b255 tempest-ServersAdminTestJSON-1511367024 tempest-ServersAdminTestJSON-1511367024-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:43.200596 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-31eca5f6-e951-4c3e-b1db-4a440ab3b255 tempest-ServersAdminTestJSON-1511367024 tempest-ServersAdminTestJSON-1511367024-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:43.203836 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-31eca5f6-e951-4c3e-b1db-4a440ab3b255 tempest-ServersAdminTestJSON-1511367024 tempest-ServersAdminTestJSON-1511367024-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:43.208891 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-31eca5f6-e951-4c3e-b1db-4a440ab3b255 tempest-ServersAdminTestJSON-1511367024 tempest-ServersAdminTestJSON-1511367024-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:43.404377 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-7eb635f5-f4d0-402e-b15b-85453b14ad27 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:19:43.405095 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-7eb635f5-f4d0-402e-b15b-85453b14ad27 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] [instance: a56f8463-6006-4805-9021-1babc2d98f0f] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:19:43.417261 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7eb635f5-f4d0-402e-b15b-85453b14ad27 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:43.418247 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7eb635f5-f4d0-402e-b15b-85453b14ad27 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:43.420430 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7eb635f5-f4d0-402e-b15b-85453b14ad27 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:43.424458 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7eb635f5-f4d0-402e-b15b-85453b14ad27 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:44.536011 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-31eca5f6-e951-4c3e-b1db-4a440ab3b255 tempest-ServersAdminTestJSON-1511367024 tempest-ServersAdminTestJSON-1511367024-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/e7598b4a-f70c-4186-88d1-6fd0057f8656" status: 200 len: 1614 microversion: 2.1 time: 1.361216 Jul 16 21:19:44.544803 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1375/2716] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:19:43 2022] GET /compute/v2.1/servers/e7598b4a-f70c-4186-88d1-6fd0057f8656 => generated 1614 bytes in 1370 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:19:44.549287 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-972821f2-7243-4fa2-9385-f8554b744bb2 req-c5331b2b-3c3a-4bcb-9a7f-05bf38b02729 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "9d2cd0fc-1101-4e21-8179-c1dd4605594f", "tag": "4589c1e7-5239-4b55-9840-6b8a096c6324"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:19:44.576490 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-972821f2-7243-4fa2-9385-f8554b744bb2 req-c5331b2b-3c3a-4bcb-9a7f-05bf38b02729 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:44.577151 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-972821f2-7243-4fa2-9385-f8554b744bb2 req-c5331b2b-3c3a-4bcb-9a7f-05bf38b02729 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:44.604227 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-972821f2-7243-4fa2-9385-f8554b744bb2 req-c5331b2b-3c3a-4bcb-9a7f-05bf38b02729 service nova] Creating event network-changed:4589c1e7-5239-4b55-9840-6b8a096c6324 for instance 9d2cd0fc-1101-4e21-8179-c1dd4605594f on nch2-master-1 Jul 16 21:19:44.616288 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-972821f2-7243-4fa2-9385-f8554b744bb2 req-c5331b2b-3c3a-4bcb-9a7f-05bf38b02729 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.071208 Jul 16 21:19:44.617070 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1376/2717] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:19:44 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 73 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:19:44.630393 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-9dc751d0-f7cd-484c-852d-f98f1fd0be1c req-d0d9d5db-7ae3-4e22-9718-fe36e1fabf79 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "e7598b4a-f70c-4186-88d1-6fd0057f8656", "name": "network-vif-plugged", "status": "completed", "tag": "5e72e822-7cdd-445c-8d25-dc81877dc718"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:19:44.654101 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-9dc751d0-f7cd-484c-852d-f98f1fd0be1c req-d0d9d5db-7ae3-4e22-9718-fe36e1fabf79 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:44.654713 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-9dc751d0-f7cd-484c-852d-f98f1fd0be1c req-d0d9d5db-7ae3-4e22-9718-fe36e1fabf79 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:44.687661 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-9dc751d0-f7cd-484c-852d-f98f1fd0be1c req-d0d9d5db-7ae3-4e22-9718-fe36e1fabf79 service nova] Creating event network-vif-plugged:5e72e822-7cdd-445c-8d25-dc81877dc718 for instance e7598b4a-f70c-4186-88d1-6fd0057f8656 on nch1-master-1 Jul 16 21:19:44.695165 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-9dc751d0-f7cd-484c-852d-f98f1fd0be1c req-d0d9d5db-7ae3-4e22-9718-fe36e1fabf79 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.075162 Jul 16 21:19:44.702941 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1377/2718] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:19:44 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:19:44.713089 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-cad98a73-6943-448c-a165-e624482daf8e tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:19:44.719828 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-cad98a73-6943-448c-a165-e624482daf8e tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] [instance: 720ef2b0-e7b3-4b36-b613-1b73d2bae2e9] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:19:44.733394 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cad98a73-6943-448c-a165-e624482daf8e tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:44.734570 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cad98a73-6943-448c-a165-e624482daf8e tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:44.737854 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cad98a73-6943-448c-a165-e624482daf8e tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:44.738577 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cad98a73-6943-448c-a165-e624482daf8e tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:44.789993 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-cad98a73-6943-448c-a165-e624482daf8e tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] [instance: 720ef2b0-e7b3-4b36-b613-1b73d2bae2e9] Going to try to terminate instance {{(pid=138680) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 16 21:19:44.860608 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell0:SELECT=4 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:19:44.868817 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-7eb635f5-f4d0-402e-b15b-85453b14ad27 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eebb224a56814f92b12ebebebc6bc8a3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a092ce9ef40943f6b6ace6cd28481b71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:19:44.882076 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-7eb635f5-f4d0-402e-b15b-85453b14ad27 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] 10.222.0.16 "GET /compute/v2.1/servers/a56f8463-6006-4805-9021-1babc2d98f0f" status: 200 len: 1318 microversion: 2.1 time: 1.483160 Jul 16 21:19:44.887172 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1342/2719] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:19:43 2022] GET /compute/v2.1/servers/a56f8463-6006-4805-9021-1babc2d98f0f => generated 1318 bytes in 1486 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:19:45.340524 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-cad98a73-6943-448c-a165-e624482daf8e tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] 10.222.0.16 "DELETE /compute/v2.1/servers/720ef2b0-e7b3-4b36-b613-1b73d2bae2e9" status: 204 len: 0 microversion: 2.1 time: 0.635354 Jul 16 21:19:45.341153 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1378/2720] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:19:44 2022] DELETE /compute/v2.1/servers/720ef2b0-e7b3-4b36-b613-1b73d2bae2e9 => generated 0 bytes in 637 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 16 21:19:45.350313 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-910a814a-b3e5-4bb9-9c2a-8e158b6d3f8b tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:19:45.351239 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-910a814a-b3e5-4bb9-9c2a-8e158b6d3f8b tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] [instance: 720ef2b0-e7b3-4b36-b613-1b73d2bae2e9] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:19:45.370696 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-910a814a-b3e5-4bb9-9c2a-8e158b6d3f8b tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:45.371276 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-910a814a-b3e5-4bb9-9c2a-8e158b6d3f8b tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:45.374654 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-910a814a-b3e5-4bb9-9c2a-8e158b6d3f8b tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:45.375502 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-910a814a-b3e5-4bb9-9c2a-8e158b6d3f8b tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:45.672894 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-4f7ec88a-e405-4c1d-a1e3-2da0a7ec74d4 tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] Action: 'action', calling method: >, body: {"os-getConsoleOutput": {}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:19:45.673900 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-4f7ec88a-e405-4c1d-a1e3-2da0a7ec74d4 tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] [instance: b9f2c527-6346-4403-841e-b7bd876a5fb7] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:19:45.691269 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4f7ec88a-e405-4c1d-a1e3-2da0a7ec74d4 tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:45.698979 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4f7ec88a-e405-4c1d-a1e3-2da0a7ec74d4 tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:45.698979 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4f7ec88a-e405-4c1d-a1e3-2da0a7ec74d4 tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:45.704211 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4f7ec88a-e405-4c1d-a1e3-2da0a7ec74d4 tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:46.111147 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-910a814a-b3e5-4bb9-9c2a-8e158b6d3f8b tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f5bc1f2209f4eb186d80e385b67c774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '65d5bcfa02af4e8284ba1342197e2860', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:19:46.119775 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-910a814a-b3e5-4bb9-9c2a-8e158b6d3f8b tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] 10.222.0.16 "GET /compute/v2.1/servers/720ef2b0-e7b3-4b36-b613-1b73d2bae2e9" status: 200 len: 1468 microversion: 2.1 time: 0.774193 Jul 16 21:19:46.121477 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1343/2721] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:19:45 2022] GET /compute/v2.1/servers/720ef2b0-e7b3-4b36-b613-1b73d2bae2e9 => generated 1468 bytes in 776 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:19:46.135563 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-dbfe38f8-1b74-43c1-be92-f19f6d0fd040 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:19:46.136451 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-dbfe38f8-1b74-43c1-be92-f19f6d0fd040 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] [instance: a56f8463-6006-4805-9021-1babc2d98f0f] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:19:46.153445 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-dbfe38f8-1b74-43c1-be92-f19f6d0fd040 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:46.154116 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-dbfe38f8-1b74-43c1-be92-f19f6d0fd040 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:46.155315 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-dbfe38f8-1b74-43c1-be92-f19f6d0fd040 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:46.156010 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-dbfe38f8-1b74-43c1-be92-f19f6d0fd040 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:46.168387 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-4f7ec88a-e405-4c1d-a1e3-2da0a7ec74d4 tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] 10.222.0.16 "POST /compute/v2.1/servers/b9f2c527-6346-4403-841e-b7bd876a5fb7/action" status: 200 len: 24569 microversion: 2.1 time: 0.501926 Jul 16 21:19:46.174687 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1379/2722] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 21:19:45 2022] POST /compute/v2.1/servers/b9f2c527-6346-4403-841e-b7bd876a5fb7/action => generated 24569 bytes in 509 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 16 21:19:46.193387 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-090a7ce7-3dcd-4a76-b561-fb39dd74a743 tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:19:46.196428 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-090a7ce7-3dcd-4a76-b561-fb39dd74a743 tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] Policy check for os_compute_api:servers:allow_all_filters failed with credentials {'is_admin': False, 'user_id': '1aa99be675e946fd995d71465b4cbec6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd82c2d92c67f469e863eb70bb7f9af84', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:19:46.197225 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-090a7ce7-3dcd-4a76-b561-fb39dd74a743 tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] Searching by: {'deleted': False, 'project_id': 'd82c2d92c67f469e863eb70bb7f9af84'} {{(pid=138680) get_all /opt/stack/nova/nova/compute/api.py:2962}} Jul 16 21:19:46.243087 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-090a7ce7-3dcd-4a76-b561-fb39dd74a743 tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:46.246112 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-090a7ce7-3dcd-4a76-b561-fb39dd74a743 tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:46.247525 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-090a7ce7-3dcd-4a76-b561-fb39dd74a743 tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:46.249894 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-090a7ce7-3dcd-4a76-b561-fb39dd74a743 tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:46.308386 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.multi_cell_list [None req-090a7ce7-3dcd-4a76-b561-fb39dd74a743 tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] Listed batch of 1 results from cell out of 1000 limit. Returned 1 total so far. {{(pid=138680) do_query /opt/stack/nova/nova/compute/multi_cell_list.py:373}} Jul 16 21:19:46.349222 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-090a7ce7-3dcd-4a76-b561-fb39dd74a743 tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] 10.222.0.16 "GET /compute/v2.1/servers" status: 200 len: 343 microversion: 2.1 time: 0.161273 Jul 16 21:19:46.350885 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1380/2723] 10.222.0.16 () {58 vars in 1124 bytes} [Sat Jul 16 21:19:46 2022] GET /compute/v2.1/servers => generated 343 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 16 21:19:46.700902 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-742bf6e0-4402-492a-8770-b40e6fabd294 tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] Action: 'create', calling method: .version_select at 0x7fdda2bc8700>, body: {"keypair": {"name": "tempest-keypair-1642910678", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBFkiYpn/1A1Ih//bsOUOrmmsDlbkzVRG4q0rPXWVyAoyngn/tJ7pFYXu5Yvs+zAKJEJDyl6Wrw/QPj36QpoXchlBKHIOkMbCHU4sclF5uV2mSXTR1s2+0iyNYCqEtqLZnA=="}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:19:46.743540 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-742bf6e0-4402-492a-8770-b40e6fabd294 tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] Getting quotas for project 0be222897b3a4864a39e9fc7d481340a. Resources: {'key_pairs'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:19:46.753250 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-742bf6e0-4402-492a-8770-b40e6fabd294 tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] Getting quotas for user 1f72b1e0b4124480bfd53b5c22a343e0 and project 0be222897b3a4864a39e9fc7d481340a. Resources: {'key_pairs'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:19:46.867307 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-742bf6e0-4402-492a-8770-b40e6fabd294 tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] 10.222.0.16 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.514021 Jul 16 21:19:46.868537 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1381/2724] 10.222.0.16 () {60 vars in 1158 bytes} [Sat Jul 16 21:19:46 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 517 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:19:46.877052 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-9ac7b1d7-d136-4781-b570-96ae3a0ee09e tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:19:46.890345 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-9ac7b1d7-d136-4781-b570-96ae3a0ee09e tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] Policy check for os_compute_api:servers:allow_all_filters failed with credentials {'is_admin': False, 'user_id': '1aa99be675e946fd995d71465b4cbec6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd82c2d92c67f469e863eb70bb7f9af84', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:19:46.891211 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-9ac7b1d7-d136-4781-b570-96ae3a0ee09e tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] Searching by: {'deleted': False, 'project_id': 'd82c2d92c67f469e863eb70bb7f9af84'} {{(pid=138680) get_all /opt/stack/nova/nova/compute/api.py:2962}} Jul 16 21:19:46.904565 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9ac7b1d7-d136-4781-b570-96ae3a0ee09e tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:46.905655 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9ac7b1d7-d136-4781-b570-96ae3a0ee09e tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:46.908247 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9ac7b1d7-d136-4781-b570-96ae3a0ee09e tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:46.908912 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9ac7b1d7-d136-4781-b570-96ae3a0ee09e tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:46.948239 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.multi_cell_list [None req-9ac7b1d7-d136-4781-b570-96ae3a0ee09e tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] Listed batch of 1 results from cell out of 1000 limit. Returned 1 total so far. {{(pid=138680) do_query /opt/stack/nova/nova/compute/multi_cell_list.py:373}} Jul 16 21:19:47.028679 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-9ac7b1d7-d136-4781-b570-96ae3a0ee09e tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aa99be675e946fd995d71465b4cbec6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd82c2d92c67f469e863eb70bb7f9af84', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:19:47.058511 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9ac7b1d7-d136-4781-b570-96ae3a0ee09e tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:47.060957 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9ac7b1d7-d136-4781-b570-96ae3a0ee09e tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:47.839859 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-dbfe38f8-1b74-43c1-be92-f19f6d0fd040 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eebb224a56814f92b12ebebebc6bc8a3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a092ce9ef40943f6b6ace6cd28481b71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:19:47.858533 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-dbfe38f8-1b74-43c1-be92-f19f6d0fd040 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] 10.222.0.16 "GET /compute/v2.1/servers/a56f8463-6006-4805-9021-1babc2d98f0f" status: 200 len: 1318 microversion: 2.1 time: 1.734323 Jul 16 21:19:47.859770 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1344/2725] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:19:46 2022] GET /compute/v2.1/servers/a56f8463-6006-4805-9021-1babc2d98f0f => generated 1318 bytes in 1737 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:19:47.874488 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-9a3b2bf2-90a8-4d15-bea5-46cce735c9f8 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:19:47.875389 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-9a3b2bf2-90a8-4d15-bea5-46cce735c9f8 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] [instance: 720ef2b0-e7b3-4b36-b613-1b73d2bae2e9] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:19:47.884635 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9a3b2bf2-90a8-4d15-bea5-46cce735c9f8 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:47.897488 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9a3b2bf2-90a8-4d15-bea5-46cce735c9f8 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:47.901188 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9a3b2bf2-90a8-4d15-bea5-46cce735c9f8 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:47.901740 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9a3b2bf2-90a8-4d15-bea5-46cce735c9f8 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:48.380151 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-9ac7b1d7-d136-4781-b570-96ae3a0ee09e tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] 10.222.0.16 "GET /compute/v2.1/servers/detail" status: 200 len: 1606 microversion: 2.1 time: 1.496616 Jul 16 21:19:48.380151 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1382/2726] 10.222.0.16 () {58 vars in 1145 bytes} [Sat Jul 16 21:19:46 2022] GET /compute/v2.1/servers/detail => generated 1606 bytes in 1499 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:19:48.390598 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-bb7522e9-b25d-4ac8-8851-6194f27a8d4c tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:19:48.407746 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-bb7522e9-b25d-4ac8-8851-6194f27a8d4c tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] 10.222.0.16 "GET /compute/v2.1/flavors/452" status: 200 len: 370 microversion: 2.1 time: 0.021475 Jul 16 21:19:48.411071 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1383/2727] 10.222.0.16 () {58 vars in 1136 bytes} [Sat Jul 16 21:19:48 2022] GET /compute/v2.1/flavors/452 => generated 370 bytes in 24 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 16 21:19:48.885388 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-b4d9a2b1-dbcb-4578-8a6a-1a33a163436e tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:19:48.886151 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-b4d9a2b1-dbcb-4578-8a6a-1a33a163436e tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] [instance: a56f8463-6006-4805-9021-1babc2d98f0f] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:19:48.907250 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-9a3b2bf2-90a8-4d15-bea5-46cce735c9f8 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f5bc1f2209f4eb186d80e385b67c774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '65d5bcfa02af4e8284ba1342197e2860', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:19:48.914382 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b4d9a2b1-dbcb-4578-8a6a-1a33a163436e tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:48.914844 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b4d9a2b1-dbcb-4578-8a6a-1a33a163436e tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:48.920428 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b4d9a2b1-dbcb-4578-8a6a-1a33a163436e tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:48.921088 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b4d9a2b1-dbcb-4578-8a6a-1a33a163436e tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:48.923158 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-9a3b2bf2-90a8-4d15-bea5-46cce735c9f8 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] 10.222.0.16 "GET /compute/v2.1/servers/720ef2b0-e7b3-4b36-b613-1b73d2bae2e9" status: 200 len: 1468 microversion: 2.1 time: 1.047056 Jul 16 21:19:48.924143 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1345/2728] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:19:47 2022] GET /compute/v2.1/servers/720ef2b0-e7b3-4b36-b613-1b73d2bae2e9 => generated 1468 bytes in 1055 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:19:49.013324 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-e1ba5628-f2b3-46b3-b433-67b4dec1f4db req-601b5651-efef-449d-816b-de38a1d20956 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "a56f8463-6006-4805-9021-1babc2d98f0f", "tag": "d407f4a6-f978-413f-821d-6a31fd8785c8"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:19:49.026254 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_api:SELECT=92,nova_api:UPDATE=2,nova_api:INSERT=6 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:19:49.038902 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-e1ba5628-f2b3-46b3-b433-67b4dec1f4db req-601b5651-efef-449d-816b-de38a1d20956 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:49.041423 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-e1ba5628-f2b3-46b3-b433-67b4dec1f4db req-601b5651-efef-449d-816b-de38a1d20956 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:49.081231 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-e1ba5628-f2b3-46b3-b433-67b4dec1f4db req-601b5651-efef-449d-816b-de38a1d20956 service nova] Creating event network-changed:d407f4a6-f978-413f-821d-6a31fd8785c8 for instance a56f8463-6006-4805-9021-1babc2d98f0f on nch1-master-1 Jul 16 21:19:49.094390 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-e1ba5628-f2b3-46b3-b433-67b4dec1f4db req-601b5651-efef-449d-816b-de38a1d20956 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.089317 Jul 16 21:19:49.096613 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1346/2729] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:19:49 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 16 21:19:49.943593 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-014e8c2f-e3ff-4568-aaf1-fbb68a75e61f tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:19:49.944721 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-014e8c2f-e3ff-4568-aaf1-fbb68a75e61f tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] [instance: 720ef2b0-e7b3-4b36-b613-1b73d2bae2e9] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:19:49.965561 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-014e8c2f-e3ff-4568-aaf1-fbb68a75e61f tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:49.966820 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-014e8c2f-e3ff-4568-aaf1-fbb68a75e61f tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:49.967890 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-014e8c2f-e3ff-4568-aaf1-fbb68a75e61f tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:49.968960 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-014e8c2f-e3ff-4568-aaf1-fbb68a75e61f tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:50.811237 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-b4d9a2b1-dbcb-4578-8a6a-1a33a163436e tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eebb224a56814f92b12ebebebc6bc8a3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a092ce9ef40943f6b6ace6cd28481b71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:19:50.822810 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-b4d9a2b1-dbcb-4578-8a6a-1a33a163436e tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] 10.222.0.16 "GET /compute/v2.1/servers/a56f8463-6006-4805-9021-1babc2d98f0f" status: 200 len: 1318 microversion: 2.1 time: 1.941498 Jul 16 21:19:50.837391 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1384/2730] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:19:48 2022] GET /compute/v2.1/servers/a56f8463-6006-4805-9021-1babc2d98f0f => generated 1318 bytes in 1956 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:19:50.989150 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-014e8c2f-e3ff-4568-aaf1-fbb68a75e61f tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f5bc1f2209f4eb186d80e385b67c774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '65d5bcfa02af4e8284ba1342197e2860', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:19:51.029807 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-014e8c2f-e3ff-4568-aaf1-fbb68a75e61f tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] 10.222.0.16 "GET /compute/v2.1/servers/720ef2b0-e7b3-4b36-b613-1b73d2bae2e9" status: 200 len: 1468 microversion: 2.1 time: 1.091573 Jul 16 21:19:51.031188 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1347/2731] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:19:49 2022] GET /compute/v2.1/servers/720ef2b0-e7b3-4b36-b613-1b73d2bae2e9 => generated 1468 bytes in 1094 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:19:51.844421 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-355daa6b-f83f-4be2-9831-c0595f4b55f0 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:19:51.845038 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-355daa6b-f83f-4be2-9831-c0595f4b55f0 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] [instance: a56f8463-6006-4805-9021-1babc2d98f0f] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:19:51.862191 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-355daa6b-f83f-4be2-9831-c0595f4b55f0 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:51.862943 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-355daa6b-f83f-4be2-9831-c0595f4b55f0 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:51.863772 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-355daa6b-f83f-4be2-9831-c0595f4b55f0 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:51.864222 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-355daa6b-f83f-4be2-9831-c0595f4b55f0 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:51.967881 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-9613ddd7-2810-4cca-a9eb-7a391ca1ec47 req-cb9d6089-1f94-4a36-8579-38204658a0be service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "720ef2b0-e7b3-4b36-b613-1b73d2bae2e9", "name": "network-vif-deleted", "tag": "b4dd35f4-14d3-4d08-b800-60be4a0cba5b"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:19:52.001632 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-9613ddd7-2810-4cca-a9eb-7a391ca1ec47 req-cb9d6089-1f94-4a36-8579-38204658a0be service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:52.002309 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-9613ddd7-2810-4cca-a9eb-7a391ca1ec47 req-cb9d6089-1f94-4a36-8579-38204658a0be service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:52.030542 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-9613ddd7-2810-4cca-a9eb-7a391ca1ec47 req-cb9d6089-1f94-4a36-8579-38204658a0be service nova] Creating event network-vif-deleted:b4dd35f4-14d3-4d08-b800-60be4a0cba5b for instance 720ef2b0-e7b3-4b36-b613-1b73d2bae2e9 on nch1-master-1 Jul 16 21:19:52.048138 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-9613ddd7-2810-4cca-a9eb-7a391ca1ec47 req-cb9d6089-1f94-4a36-8579-38204658a0be service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.084711 Jul 16 21:19:52.056005 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1348/2732] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:19:51 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 16 21:19:52.067128 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-80935af6-974e-4b43-9ea8-e984c3e7c865 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:19:52.069752 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-80935af6-974e-4b43-9ea8-e984c3e7c865 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] [instance: 720ef2b0-e7b3-4b36-b613-1b73d2bae2e9] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:19:52.088621 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-80935af6-974e-4b43-9ea8-e984c3e7c865 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:52.089388 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-80935af6-974e-4b43-9ea8-e984c3e7c865 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:52.090399 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-80935af6-974e-4b43-9ea8-e984c3e7c865 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:52.091113 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-80935af6-974e-4b43-9ea8-e984c3e7c865 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:52.588170 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-80935af6-974e-4b43-9ea8-e984c3e7c865 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f5bc1f2209f4eb186d80e385b67c774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '65d5bcfa02af4e8284ba1342197e2860', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:19:52.606729 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-80935af6-974e-4b43-9ea8-e984c3e7c865 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] 10.222.0.16 "GET /compute/v2.1/servers/720ef2b0-e7b3-4b36-b613-1b73d2bae2e9" status: 200 len: 1426 microversion: 2.1 time: 0.549137 Jul 16 21:19:52.609657 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1349/2733] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:19:52 2022] GET /compute/v2.1/servers/720ef2b0-e7b3-4b36-b613-1b73d2bae2e9 => generated 1426 bytes in 553 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:19:53.109821 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-355daa6b-f83f-4be2-9831-c0595f4b55f0 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eebb224a56814f92b12ebebebc6bc8a3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a092ce9ef40943f6b6ace6cd28481b71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:19:53.125570 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-355daa6b-f83f-4be2-9831-c0595f4b55f0 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] 10.222.0.16 "GET /compute/v2.1/servers/a56f8463-6006-4805-9021-1babc2d98f0f" status: 200 len: 1318 microversion: 2.1 time: 1.284999 Jul 16 21:19:53.127101 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1385/2734] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:19:51 2022] GET /compute/v2.1/servers/a56f8463-6006-4805-9021-1babc2d98f0f => generated 1318 bytes in 1287 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:19:53.247206 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-3bafb7cd-2bf2-475a-b7f7-816a07c0457c tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServersTestManualDisk-server-665854763", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452", "metadata": {"hello": "world"}, "accessIPv4": "1.1.1.1", "accessIPv6": "0000:0000:0000:0000:0000:babe:220.12.22.2", "adminPass": "***", "networks": [{"uuid": "eb4b973e-9e58-4ee0-be45-4d674dadbe28"}], "security_groups": [{"name": "tempest-securitygroup--135924805"}], "key_name": "tempest-keypair-1642910678", "user_data": "IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIHVidW50dSB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH51YnVudHUvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=", "OS-DCF:diskConfig": "}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:19:53.438202 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-69a10e6c-6337-4fa7-9da5-5d43a513fd9a req-53012757-b10e-40cf-8a54-199b9f8d7ec4 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "9d2cd0fc-1101-4e21-8179-c1dd4605594f", "name": "network-vif-plugged", "status": "completed", "tag": "4589c1e7-5239-4b55-9840-6b8a096c6324"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:19:53.465194 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-69a10e6c-6337-4fa7-9da5-5d43a513fd9a req-53012757-b10e-40cf-8a54-199b9f8d7ec4 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:53.466000 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-69a10e6c-6337-4fa7-9da5-5d43a513fd9a req-53012757-b10e-40cf-8a54-199b9f8d7ec4 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:53.484717 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-69a10e6c-6337-4fa7-9da5-5d43a513fd9a req-53012757-b10e-40cf-8a54-199b9f8d7ec4 service nova] Creating event network-vif-plugged:4589c1e7-5239-4b55-9840-6b8a096c6324 for instance 9d2cd0fc-1101-4e21-8179-c1dd4605594f on nch2-master-1 Jul 16 21:19:53.495269 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-69a10e6c-6337-4fa7-9da5-5d43a513fd9a req-53012757-b10e-40cf-8a54-199b9f8d7ec4 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.062092 Jul 16 21:19:53.498700 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1386/2735] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:19:53 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 68 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:19:53.629791 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-27a4b735-af51-49a6-a3bf-7d70cf114ccf tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:19:53.631164 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-27a4b735-af51-49a6-a3bf-7d70cf114ccf tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] [instance: 720ef2b0-e7b3-4b36-b613-1b73d2bae2e9] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:19:53.656109 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-27a4b735-af51-49a6-a3bf-7d70cf114ccf tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:53.656788 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-27a4b735-af51-49a6-a3bf-7d70cf114ccf tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:53.658010 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-27a4b735-af51-49a6-a3bf-7d70cf114ccf tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:53.658616 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-27a4b735-af51-49a6-a3bf-7d70cf114ccf tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:53.784984 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-27a4b735-af51-49a6-a3bf-7d70cf114ccf tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:53.786713 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-27a4b735-af51-49a6-a3bf-7d70cf114ccf tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:53.788593 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.objects.instance [None req-27a4b735-af51-49a6-a3bf-7d70cf114ccf tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lazy-loading 'fault' on Instance uuid 720ef2b0-e7b3-4b36-b613-1b73d2bae2e9 {{(pid=138680) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:19:53.992816 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-27a4b735-af51-49a6-a3bf-7d70cf114ccf tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f5bc1f2209f4eb186d80e385b67c774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '65d5bcfa02af4e8284ba1342197e2860', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:19:54.000990 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.network.neutron [None req-3bafb7cd-2bf2-475a-b7f7-816a07c0457c tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] validate_networks() for [('eb4b973e-9e58-4ee0-be45-4d674dadbe28', None, None, None, None, None)] {{(pid=138681) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 16 21:19:54.009099 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-27a4b735-af51-49a6-a3bf-7d70cf114ccf tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] 10.222.0.16 "GET /compute/v2.1/servers/720ef2b0-e7b3-4b36-b613-1b73d2bae2e9" status: 200 len: 1303 microversion: 2.1 time: 0.387002 Jul 16 21:19:54.010263 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1387/2736] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:19:53 2022] GET /compute/v2.1/servers/720ef2b0-e7b3-4b36-b613-1b73d2bae2e9 => generated 1303 bytes in 389 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:19:54.144847 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-81a26259-3130-4d13-8bec-ecfed8b899d7 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:19:54.145564 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-81a26259-3130-4d13-8bec-ecfed8b899d7 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] [instance: a56f8463-6006-4805-9021-1babc2d98f0f] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:19:54.153898 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-81a26259-3130-4d13-8bec-ecfed8b899d7 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:54.154600 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-81a26259-3130-4d13-8bec-ecfed8b899d7 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:54.155420 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-81a26259-3130-4d13-8bec-ecfed8b899d7 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:54.155876 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-81a26259-3130-4d13-8bec-ecfed8b899d7 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:54.688350 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-81a26259-3130-4d13-8bec-ecfed8b899d7 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eebb224a56814f92b12ebebebc6bc8a3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a092ce9ef40943f6b6ace6cd28481b71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:19:54.721698 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-81a26259-3130-4d13-8bec-ecfed8b899d7 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] 10.222.0.16 "GET /compute/v2.1/servers/a56f8463-6006-4805-9021-1babc2d98f0f" status: 200 len: 1318 microversion: 2.1 time: 0.579121 Jul 16 21:19:54.727398 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1388/2737] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:19:54 2022] GET /compute/v2.1/servers/a56f8463-6006-4805-9021-1babc2d98f0f => generated 1318 bytes in 584 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:19:55.590232 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-d6c7ccdf-dcdd-469d-8a30-d72924ab4297 tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Action: 'create', calling method: .version_select at 0x7fdd9d242040>, body: {"server_group": {"name": "tempest-ServerGroupTestJSON-Server-Group-1240723198", "policies": ["affinity"]}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:19:55.645622 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-d6c7ccdf-dcdd-469d-8a30-d72924ab4297 tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Getting quotas for project 772bd01188344e288824730238fee49b. Resources: {'server_groups'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:19:55.666419 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-d6c7ccdf-dcdd-469d-8a30-d72924ab4297 tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Getting quotas for user fe680ccdc60e410b812794de8aa73e0f and project 772bd01188344e288824730238fee49b. Resources: {'server_groups'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:19:55.892505 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-d6c7ccdf-dcdd-469d-8a30-d72924ab4297 tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Getting quotas for project 772bd01188344e288824730238fee49b. Resources: {'server_groups'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:19:55.907628 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-d6c7ccdf-dcdd-469d-8a30-d72924ab4297 tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Getting quotas for user fe680ccdc60e410b812794de8aa73e0f and project 772bd01188344e288824730238fee49b. Resources: {'server_groups'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:19:55.911977 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_api:UPDATE=2,nova_api:SELECT=95,nova_api:INSERT=3 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:19:55.923963 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-d6c7ccdf-dcdd-469d-8a30-d72924ab4297 tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] 10.222.0.16 "POST /compute/v2.1/os-server-groups" status: 200 len: 184 microversion: 2.1 time: 1.061086 Jul 16 21:19:55.925174 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1389/2738] 10.222.0.16 () {60 vars in 1173 bytes} [Sat Jul 16 21:19:54 2022] POST /compute/v2.1/os-server-groups => generated 184 bytes in 1063 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 16 21:19:55.933997 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-8076702b-2599-4fff-8bcd-44c5852cacbd tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:19:55.942237 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-8076702b-2599-4fff-8bcd-44c5852cacbd tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] [instance: 720ef2b0-e7b3-4b36-b613-1b73d2bae2e9] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:19:55.953378 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8076702b-2599-4fff-8bcd-44c5852cacbd tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:55.955120 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8076702b-2599-4fff-8bcd-44c5852cacbd tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:55.965874 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8076702b-2599-4fff-8bcd-44c5852cacbd tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:55.965874 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8076702b-2599-4fff-8bcd-44c5852cacbd tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:55.985558 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-8076702b-2599-4fff-8bcd-44c5852cacbd tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] HTTP exception thrown: Instance 720ef2b0-e7b3-4b36-b613-1b73d2bae2e9 could not be found. Jul 16 21:19:55.987652 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-8076702b-2599-4fff-8bcd-44c5852cacbd tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Request method failure captured: Jul 16 21:19:55.987652 ncdv-master-1 devstack@n-api.service[138680]: request: GET /compute/v2.1/servers/720ef2b0-e7b3-4b36-b613-1b73d2bae2e9 HTTP/1.1 Jul 16 21:19:55.987652 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:19:55.987652 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:19:55.987652 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:19:55.987652 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 0 Jul 16 21:19:55.987652 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:19:55.987652 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:19:55.987652 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:19:55.987652 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:19:55.987652 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0yrNpNpQByhQ1578GareI_bLbzFzwt1X7DjqAJogmQKBk7pzKioqjlhICyrqQjyNEkmOeTkxKX8xROhDwwvvteGCjPW-Knwb8qWdwKE0LlYrz9U5NzOQ8ukFz_cfNcdWpVOqqWw9XjDKSvggxCIEhYLmedQYWwv9L2-iYQUT-tM1e3g Jul 16 21:19:55.987652 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:19:55.987652 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:19:55.987652 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:19:55.987652 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:19:55.987652 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:19:55.987652 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:19:55.987652 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: 65d5bcfa02af4e8284ba1342197e2860 Jul 16 21:19:55.987652 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-ServersAdminTestJSON-1438684720 Jul 16 21:19:55.987652 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:19:55.987652 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:19:55.987652 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/65d5bcfa02af4e8284ba1342197e2860"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/65d5bcfa02af4e8284ba1342197e2860"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_65d5bcfa02af4e8284ba1342197e2860"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/65d5bcfa02af4e8284ba1342197e2860"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:19:55.987652 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-ServersAdminTestJSON-1438684720 Jul 16 21:19:55.987652 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: 65d5bcfa02af4e8284ba1342197e2860 Jul 16 21:19:55.987652 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-ServersAdminTestJSON-1438684720 Jul 16 21:19:55.987652 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-ServersAdminTestJSON-1438684720-project Jul 16 21:19:55.987652 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:19:55.987652 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:19:55.987652 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: 2f5bc1f2209f4eb186d80e385b67c774 Jul 16 21:19:55.987652 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-ServersAdminTestJSON-1438684720-project Jul 16 21:19:55.987652 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 21:19:55.987652 ncdv-master-1 devstack@n-api.service[138680]: exception: Instance 720ef2b0-e7b3-4b36-b613-1b73d2bae2e9 could not be found. Jul 16 21:19:55.987652 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': '720ef2b0-e7b3-4b36-b613-1b73d2bae2e9'} Jul 16 21:19:55.987652 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:19:55.987652 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:19:55.987652 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:19:55.987652 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:19:55.987652 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:19:55.987652 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:19:55.987652 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:19:55.987652 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:19:55.987652 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:19:55.987652 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:19:55.987652 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 720ef2b0-e7b3-4b36-b613-1b73d2bae2e9 could not be found. Jul 16 21:19:55.987652 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:19:55.987652 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:19:55.987652 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:19:55.987652 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:19:55.987652 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:19:55.987652 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:19:55.987652 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:19:55.987652 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:19:55.987652 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:19:55.987652 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:19:55.987652 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:19:55.987652 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:19:55.987652 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:19:55.987652 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:19:55.987652 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:19:55.987652 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:19:55.987652 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 720ef2b0-e7b3-4b36-b613-1b73d2bae2e9 could not be found. Jul 16 21:19:55.987652 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:19:55.987652 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:19:55.987652 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:19:55.991909 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:19:55.991909 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:19:55.991909 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:19:55.991909 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:19:55.991909 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:19:55.991909 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 720ef2b0-e7b3-4b36-b613-1b73d2bae2e9 could not be found. Jul 16 21:19:55.991909 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:19:55.993953 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-8076702b-2599-4fff-8bcd-44c5852cacbd tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Returning 404 to user: Instance 720ef2b0-e7b3-4b36-b613-1b73d2bae2e9 could not be found. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:19:55.995186 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-8076702b-2599-4fff-8bcd-44c5852cacbd tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] 10.222.0.16 "GET /compute/v2.1/servers/720ef2b0-e7b3-4b36-b613-1b73d2bae2e9" status: 404 len: 111 microversion: 2.1 time: 0.068791 Jul 16 21:19:55.995970 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1390/2739] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:19:55 2022] GET /compute/v2.1/servers/720ef2b0-e7b3-4b36-b613-1b73d2bae2e9 => generated 111 bytes in 70 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:19:56.001447 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-2ac0fc94-8885-48ec-8c66-f39b7d8a69c9 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:19:56.002334 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-2ac0fc94-8885-48ec-8c66-f39b7d8a69c9 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] [instance: a56f8463-6006-4805-9021-1babc2d98f0f] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:19:56.012033 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2ac0fc94-8885-48ec-8c66-f39b7d8a69c9 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:56.012831 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2ac0fc94-8885-48ec-8c66-f39b7d8a69c9 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:56.013927 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2ac0fc94-8885-48ec-8c66-f39b7d8a69c9 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:56.014729 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2ac0fc94-8885-48ec-8c66-f39b7d8a69c9 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:56.040231 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-3bafb7cd-2bf2-475a-b7f7-816a07c0457c tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] Getting quotas for project 0be222897b3a4864a39e9fc7d481340a. Resources: dict_keys(['metadata_items']) {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:19:56.045845 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-3bafb7cd-2bf2-475a-b7f7-816a07c0457c tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] Getting quotas for user 1f72b1e0b4124480bfd53b5c22a343e0 and project 0be222897b3a4864a39e9fc7d481340a. Resources: dict_keys(['metadata_items']) {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:19:56.075829 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-3bafb7cd-2bf2-475a-b7f7-816a07c0457c tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] Flavor limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 16 21:19:56.080614 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-3bafb7cd-2bf2-475a-b7f7-816a07c0457c tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] Image limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 16 21:19:56.082126 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-3bafb7cd-2bf2-475a-b7f7-816a07c0457c tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] Flavor pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 16 21:19:56.082832 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-3bafb7cd-2bf2-475a-b7f7-816a07c0457c tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] Image pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 16 21:19:56.083351 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-3bafb7cd-2bf2-475a-b7f7-816a07c0457c tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 16 21:19:56.097391 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3bafb7cd-2bf2-475a-b7f7-816a07c0457c tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:56.097391 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3bafb7cd-2bf2-475a-b7f7-816a07c0457c tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:56.097391 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3bafb7cd-2bf2-475a-b7f7-816a07c0457c tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:56.097391 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3bafb7cd-2bf2-475a-b7f7-816a07c0457c tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:56.118957 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-3bafb7cd-2bf2-475a-b7f7-816a07c0457c tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] Getting quotas for project 0be222897b3a4864a39e9fc7d481340a. Resources: {'ram', 'instances', 'cores'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:19:56.125312 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-3bafb7cd-2bf2-475a-b7f7-816a07c0457c tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] Getting quotas for user 1f72b1e0b4124480bfd53b5c22a343e0 and project 0be222897b3a4864a39e9fc7d481340a. Resources: {'ram', 'instances', 'cores'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:19:56.151351 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-3bafb7cd-2bf2-475a-b7f7-816a07c0457c tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] Going to run 1 instances... {{(pid=138681) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 16 21:19:56.154582 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-3bafb7cd-2bf2-475a-b7f7-816a07c0457c tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] [instance: 0a9564cb-c375-47e9-bfe6-e68d4700442b] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138681) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 16 21:19:56.306339 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-2ac0fc94-8885-48ec-8c66-f39b7d8a69c9 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eebb224a56814f92b12ebebebc6bc8a3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a092ce9ef40943f6b6ace6cd28481b71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:19:56.320026 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-2ac0fc94-8885-48ec-8c66-f39b7d8a69c9 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] 10.222.0.16 "GET /compute/v2.1/servers/a56f8463-6006-4805-9021-1babc2d98f0f" status: 200 len: 1318 microversion: 2.1 time: 0.318900 Jul 16 21:19:56.320026 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1391/2740] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:19:55 2022] GET /compute/v2.1/servers/a56f8463-6006-4805-9021-1babc2d98f0f => generated 1318 bytes in 320 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:19:56.333828 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-1d8c4b5f-1fee-45be-8deb-cc682e4c272c tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Action: 'create', calling method: .version_select at 0x7fdd9cc3fee0>, body: {"server_group": {"name": "tempest-server-group-1689613382", "policies": ["affinity"]}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:19:56.359633 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-1d8c4b5f-1fee-45be-8deb-cc682e4c272c tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Getting quotas for project 772bd01188344e288824730238fee49b. Resources: {'server_groups'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:19:56.367426 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-1d8c4b5f-1fee-45be-8deb-cc682e4c272c tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Getting quotas for user fe680ccdc60e410b812794de8aa73e0f and project 772bd01188344e288824730238fee49b. Resources: {'server_groups'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:19:56.372800 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-3bafb7cd-2bf2-475a-b7f7-816a07c0457c tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 408 microversion: 2.1 time: 3.132095 Jul 16 21:19:56.373846 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1350/2741] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 21:19:53 2022] POST /compute/v2.1/servers => generated 408 bytes in 3134 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 16 21:19:56.386878 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-645d454c-5c0e-429e-9884-a91d40110079 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:19:56.388462 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-645d454c-5c0e-429e-9884-a91d40110079 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] [instance: aa688a7e-0b28-4dfd-a159-8e19fd62ee02] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:19:56.421415 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-645d454c-5c0e-429e-9884-a91d40110079 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:56.423373 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-645d454c-5c0e-429e-9884-a91d40110079 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:56.425504 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-645d454c-5c0e-429e-9884-a91d40110079 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:56.428384 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-645d454c-5c0e-429e-9884-a91d40110079 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:56.485484 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-645d454c-5c0e-429e-9884-a91d40110079 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] [instance: aa688a7e-0b28-4dfd-a159-8e19fd62ee02] Going to try to terminate instance {{(pid=138681) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 16 21:19:56.566191 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-1d8c4b5f-1fee-45be-8deb-cc682e4c272c tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Getting quotas for project 772bd01188344e288824730238fee49b. Resources: {'server_groups'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:19:56.588063 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-1d8c4b5f-1fee-45be-8deb-cc682e4c272c tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Getting quotas for user fe680ccdc60e410b812794de8aa73e0f and project 772bd01188344e288824730238fee49b. Resources: {'server_groups'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:19:56.605313 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-1d8c4b5f-1fee-45be-8deb-cc682e4c272c tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] 10.222.0.16 "POST /compute/v2.1/os-server-groups" status: 200 len: 164 microversion: 2.1 time: 0.287321 Jul 16 21:19:56.610678 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1392/2742] 10.222.0.16 () {60 vars in 1172 bytes} [Sat Jul 16 21:19:56 2022] POST /compute/v2.1/os-server-groups => generated 164 bytes in 289 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:19:56.654076 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=96,nova_cell1:INSERT=1,nova_cell1:UPDATE=3 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:19:56.873181 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-9ebbc5ec-eeeb-474f-8d7d-a20162aa320f tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] Action: 'create', calling method: .version_select at 0x7fdd9d3adaf0>, body: {"keypair": {"name": "tempest-keypair-1918580201", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBMkDlXRpHVfj5r5sn4DeSgkiGGRpwmt8jZSywdXwE0I5xbsoFNHCx9xTA7OXmmajdrVpSIKchRFlkM0qyCQRZ+jwPkUoXUuNBfwZawbKAOzsVwH44pFCzMwaHdinM/BoKg=="}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:19:56.892339 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-9ebbc5ec-eeeb-474f-8d7d-a20162aa320f tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] Getting quotas for project 837a8f9e06e141dcbd5f7bd895016d49. Resources: {'key_pairs'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:19:56.900542 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-9ebbc5ec-eeeb-474f-8d7d-a20162aa320f tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] Getting quotas for user c6ba562873fa4f4fbd0b635f1c9dcdf2 and project 837a8f9e06e141dcbd5f7bd895016d49. Resources: {'key_pairs'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:19:56.911967 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-645d454c-5c0e-429e-9884-a91d40110079 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] 10.222.0.16 "DELETE /compute/v2.1/servers/aa688a7e-0b28-4dfd-a159-8e19fd62ee02" status: 204 len: 0 microversion: 2.1 time: 0.527307 Jul 16 21:19:56.912828 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1351/2743] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:19:56 2022] DELETE /compute/v2.1/servers/aa688a7e-0b28-4dfd-a159-8e19fd62ee02 => generated 0 bytes in 532 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 16 21:19:56.918816 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-7e4237b5-0629-443f-926c-d1fad618be4b tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:19:56.921214 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-7e4237b5-0629-443f-926c-d1fad618be4b tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] [instance: 0a9564cb-c375-47e9-bfe6-e68d4700442b] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:19:56.935612 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell0:SELECT=4 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:19:57.020679 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-7e4237b5-0629-443f-926c-d1fad618be4b tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1f72b1e0b4124480bfd53b5c22a343e0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0be222897b3a4864a39e9fc7d481340a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:19:57.057400 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-7e4237b5-0629-443f-926c-d1fad618be4b tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] 10.222.0.16 "GET /compute/v2.1/servers/0a9564cb-c375-47e9-bfe6-e68d4700442b" status: 200 len: 1277 microversion: 2.1 time: 0.135485 Jul 16 21:19:57.057400 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1352/2744] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:19:56 2022] GET /compute/v2.1/servers/0a9564cb-c375-47e9-bfe6-e68d4700442b => generated 1277 bytes in 143 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:19:57.070823 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-8d2edfe5-8790-4146-9ca6-bc99e57fe5ba tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Action: 'create', calling method: .version_select at 0x7fdd9d0aadc0>, body: {"server_group": {"name": "tempest-server-group-1689613382", "policies": ["affinity"]}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:19:57.128745 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-8d2edfe5-8790-4146-9ca6-bc99e57fe5ba tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Getting quotas for project 772bd01188344e288824730238fee49b. Resources: {'server_groups'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:19:57.137740 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-8d2edfe5-8790-4146-9ca6-bc99e57fe5ba tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Getting quotas for user fe680ccdc60e410b812794de8aa73e0f and project 772bd01188344e288824730238fee49b. Resources: {'server_groups'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:19:57.146210 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-9ebbc5ec-eeeb-474f-8d7d-a20162aa320f tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] 10.222.0.16 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.537906 Jul 16 21:19:57.147056 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1393/2745] 10.222.0.16 () {60 vars in 1158 bytes} [Sat Jul 16 21:19:56 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 540 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:19:57.166236 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-4f44d957-3d59-40b1-b998-0dc879d7e82a tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:19:57.166837 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-4f44d957-3d59-40b1-b998-0dc879d7e82a tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] [instance: aa688a7e-0b28-4dfd-a159-8e19fd62ee02] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:19:57.203494 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4f44d957-3d59-40b1-b998-0dc879d7e82a tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:57.204030 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4f44d957-3d59-40b1-b998-0dc879d7e82a tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:57.205029 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4f44d957-3d59-40b1-b998-0dc879d7e82a tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:57.207653 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4f44d957-3d59-40b1-b998-0dc879d7e82a tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:57.340513 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-8d2edfe5-8790-4146-9ca6-bc99e57fe5ba tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Getting quotas for project 772bd01188344e288824730238fee49b. Resources: {'server_groups'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:19:57.366451 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-8d2edfe5-8790-4146-9ca6-bc99e57fe5ba tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Getting quotas for user fe680ccdc60e410b812794de8aa73e0f and project 772bd01188344e288824730238fee49b. Resources: {'server_groups'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:19:57.388699 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-8d2edfe5-8790-4146-9ca6-bc99e57fe5ba tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] 10.222.0.16 "POST /compute/v2.1/os-server-groups" status: 200 len: 164 microversion: 2.1 time: 0.329171 Jul 16 21:19:57.389717 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1353/2746] 10.222.0.16 () {60 vars in 1172 bytes} [Sat Jul 16 21:19:57 2022] POST /compute/v2.1/os-server-groups => generated 164 bytes in 333 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:19:57.404107 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-094e7fce-dee5-4132-9878-ac312d3040a8 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:19:57.404839 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-094e7fce-dee5-4132-9878-ac312d3040a8 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] [instance: a56f8463-6006-4805-9021-1babc2d98f0f] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:19:57.413244 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-094e7fce-dee5-4132-9878-ac312d3040a8 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:57.413903 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-094e7fce-dee5-4132-9878-ac312d3040a8 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:57.414762 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-094e7fce-dee5-4132-9878-ac312d3040a8 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:57.415343 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-094e7fce-dee5-4132-9878-ac312d3040a8 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:57.703488 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-4f44d957-3d59-40b1-b998-0dc879d7e82a tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f5bc1f2209f4eb186d80e385b67c774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '65d5bcfa02af4e8284ba1342197e2860', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:19:57.706809 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:19:57.715559 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-4f44d957-3d59-40b1-b998-0dc879d7e82a tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] 10.222.0.16 "GET /compute/v2.1/servers/aa688a7e-0b28-4dfd-a159-8e19fd62ee02" status: 200 len: 1470 microversion: 2.1 time: 0.566839 Jul 16 21:19:57.716897 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1394/2747] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:19:57 2022] GET /compute/v2.1/servers/aa688a7e-0b28-4dfd-a159-8e19fd62ee02 => generated 1470 bytes in 569 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:19:57.726181 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-3377dbea-1cbe-4823-84ee-e44d0165138c tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:19:57.855735 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-3377dbea-1cbe-4823-84ee-e44d0165138c tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] 10.222.0.16 "DELETE /compute/v2.1/os-server-groups/6f360547-03a6-4526-9bde-3144198d30f4" status: 204 len: 0 microversion: 2.1 time: 0.137535 Jul 16 21:19:57.858247 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1395/2748] 10.222.0.16 () {58 vars in 1265 bytes} [Sat Jul 16 21:19:57 2022] DELETE /compute/v2.1/os-server-groups/6f360547-03a6-4526-9bde-3144198d30f4 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 16 21:19:57.872433 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-8a869fae-cfb5-4f9f-b84a-737ad79e3cc7 tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:19:57.900605 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-8a869fae-cfb5-4f9f-b84a-737ad79e3cc7 tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] 10.222.0.16 "GET /compute/v2.1/os-server-groups" status: 200 len: 335 microversion: 2.1 time: 0.036114 Jul 16 21:19:57.903675 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1396/2749] 10.222.0.16 () {58 vars in 1151 bytes} [Sat Jul 16 21:19:57 2022] GET /compute/v2.1/os-server-groups => generated 335 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:19:57.914431 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-96b01eba-4d76-4c0a-a0ac-4737b89b4454 tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:19:57.917365 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-094e7fce-dee5-4132-9878-ac312d3040a8 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eebb224a56814f92b12ebebebc6bc8a3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a092ce9ef40943f6b6ace6cd28481b71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:19:57.931260 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-094e7fce-dee5-4132-9878-ac312d3040a8 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] 10.222.0.16 "GET /compute/v2.1/servers/a56f8463-6006-4805-9021-1babc2d98f0f" status: 200 len: 1318 microversion: 2.1 time: 0.532373 Jul 16 21:19:57.932374 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1354/2750] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:19:57 2022] GET /compute/v2.1/servers/a56f8463-6006-4805-9021-1babc2d98f0f => generated 1318 bytes in 536 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:19:58.025116 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-96b01eba-4d76-4c0a-a0ac-4737b89b4454 tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] 10.222.0.16 "DELETE /compute/v2.1/os-server-groups/970ebe29-6d90-49c2-8d63-f506221aecc5" status: 204 len: 0 microversion: 2.1 time: 0.115478 Jul 16 21:19:58.026737 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1397/2751] 10.222.0.16 () {58 vars in 1265 bytes} [Sat Jul 16 21:19:57 2022] DELETE /compute/v2.1/os-server-groups/970ebe29-6d90-49c2-8d63-f506221aecc5 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 16 21:19:58.034295 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-fb4ed88e-64cd-4ca5-99a0-683fc7bf1dc6 tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:19:58.071861 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-fb4ed88e-64cd-4ca5-99a0-683fc7bf1dc6 tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] 10.222.0.16 "GET /compute/v2.1/os-server-groups" status: 200 len: 187 microversion: 2.1 time: 0.041464 Jul 16 21:19:58.076487 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-eedfe774-018f-4cdf-a8c5-311a7d8a9ff9 tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:19:58.076487 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-eedfe774-018f-4cdf-a8c5-311a7d8a9ff9 tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] [instance: 0a9564cb-c375-47e9-bfe6-e68d4700442b] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:19:58.078764 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1355/2752] 10.222.0.16 () {58 vars in 1151 bytes} [Sat Jul 16 21:19:58 2022] GET /compute/v2.1/os-server-groups => generated 187 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:19:58.085694 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-b80351bd-b0fc-4c72-989d-484f4555a0ca tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Action: 'create', calling method: .version_select at 0x7fdd9cdc4160>, body: {"server_group": {"name": "tempest-server-group-960171593", "policies": ["affinity"]}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:19:58.106074 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-b80351bd-b0fc-4c72-989d-484f4555a0ca tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Getting quotas for project 772bd01188344e288824730238fee49b. Resources: {'server_groups'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:19:58.112582 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-b80351bd-b0fc-4c72-989d-484f4555a0ca tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Getting quotas for user fe680ccdc60e410b812794de8aa73e0f and project 772bd01188344e288824730238fee49b. Resources: {'server_groups'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:19:58.211257 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-eedfe774-018f-4cdf-a8c5-311a7d8a9ff9 tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1f72b1e0b4124480bfd53b5c22a343e0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0be222897b3a4864a39e9fc7d481340a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:19:58.255605 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-eedfe774-018f-4cdf-a8c5-311a7d8a9ff9 tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] 10.222.0.16 "GET /compute/v2.1/servers/0a9564cb-c375-47e9-bfe6-e68d4700442b" status: 200 len: 1277 microversion: 2.1 time: 0.185223 Jul 16 21:19:58.257141 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1398/2753] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:19:58 2022] GET /compute/v2.1/servers/0a9564cb-c375-47e9-bfe6-e68d4700442b => generated 1277 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:19:58.480612 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-b80351bd-b0fc-4c72-989d-484f4555a0ca tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Getting quotas for project 772bd01188344e288824730238fee49b. Resources: {'server_groups'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:19:58.504797 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-b80351bd-b0fc-4c72-989d-484f4555a0ca tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Getting quotas for user fe680ccdc60e410b812794de8aa73e0f and project 772bd01188344e288824730238fee49b. Resources: {'server_groups'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:19:58.508606 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_api:SELECT=92,nova_api:INSERT=7,nova_api:UPDATE=1 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:19:58.546978 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-b80351bd-b0fc-4c72-989d-484f4555a0ca tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] 10.222.0.16 "POST /compute/v2.1/os-server-groups" status: 200 len: 163 microversion: 2.1 time: 0.467005 Jul 16 21:19:58.547617 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1356/2754] 10.222.0.16 () {60 vars in 1172 bytes} [Sat Jul 16 21:19:58 2022] POST /compute/v2.1/os-server-groups => generated 163 bytes in 468 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:19:58.558776 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-dd13ed44-9207-47f4-b15d-6cc27b89cb1f tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:19:58.734266 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-900ce8b6-e0d5-4f02-9854-20aa4d01c6dd tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:19:58.735204 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-900ce8b6-e0d5-4f02-9854-20aa4d01c6dd tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] [instance: aa688a7e-0b28-4dfd-a159-8e19fd62ee02] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:19:58.743165 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-900ce8b6-e0d5-4f02-9854-20aa4d01c6dd tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:58.744019 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-900ce8b6-e0d5-4f02-9854-20aa4d01c6dd tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:58.745027 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-900ce8b6-e0d5-4f02-9854-20aa4d01c6dd tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:58.745963 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-900ce8b6-e0d5-4f02-9854-20aa4d01c6dd tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:58.757886 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-dd13ed44-9207-47f4-b15d-6cc27b89cb1f tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] 10.222.0.16 "DELETE /compute/v2.1/os-server-groups/dd6aa1fc-935c-443a-81fb-19099bb00610" status: 204 len: 0 microversion: 2.1 time: 0.203821 Jul 16 21:19:58.759453 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1399/2755] 10.222.0.16 () {58 vars in 1265 bytes} [Sat Jul 16 21:19:58 2022] DELETE /compute/v2.1/os-server-groups/dd6aa1fc-935c-443a-81fb-19099bb00610 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 16 21:19:58.767141 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-74c7c4cd-983b-48bb-99bf-301f13b58284 tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:19:58.799116 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-74c7c4cd-983b-48bb-99bf-301f13b58284 tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] 10.222.0.16 "GET /compute/v2.1/os-server-groups" status: 200 len: 187 microversion: 2.1 time: 0.035435 Jul 16 21:19:58.815359 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1400/2756] 10.222.0.16 () {58 vars in 1151 bytes} [Sat Jul 16 21:19:58 2022] GET /compute/v2.1/os-server-groups => generated 187 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:19:58.821380 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-b9f969b4-55b9-40bd-9cb7-14b73e260ff5 tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Action: 'create', calling method: .version_select at 0x7fdd9c6b9e50>, body: {"server_group": {"name": "tempest-server-group-304506694", "policies": ["anti-affinity"]}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:19:58.862555 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-b9f969b4-55b9-40bd-9cb7-14b73e260ff5 tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Getting quotas for project 772bd01188344e288824730238fee49b. Resources: {'server_groups'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:19:58.875394 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-b9f969b4-55b9-40bd-9cb7-14b73e260ff5 tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Getting quotas for user fe680ccdc60e410b812794de8aa73e0f and project 772bd01188344e288824730238fee49b. Resources: {'server_groups'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:19:59.057236 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-b9f969b4-55b9-40bd-9cb7-14b73e260ff5 tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Getting quotas for project 772bd01188344e288824730238fee49b. Resources: {'server_groups'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:19:59.066051 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-b9f969b4-55b9-40bd-9cb7-14b73e260ff5 tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Getting quotas for user fe680ccdc60e410b812794de8aa73e0f and project 772bd01188344e288824730238fee49b. Resources: {'server_groups'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:19:59.085639 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-b9f969b4-55b9-40bd-9cb7-14b73e260ff5 tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] 10.222.0.16 "POST /compute/v2.1/os-server-groups" status: 200 len: 168 microversion: 2.1 time: 0.268161 Jul 16 21:19:59.086534 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1401/2757] 10.222.0.16 () {60 vars in 1172 bytes} [Sat Jul 16 21:19:58 2022] POST /compute/v2.1/os-server-groups => generated 168 bytes in 270 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:19:59.097826 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-d23e8702-f109-4c61-a829-16e979e1b49c tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:19:59.115548 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-d23e8702-f109-4c61-a829-16e979e1b49c tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] [instance: a56f8463-6006-4805-9021-1babc2d98f0f] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:19:59.123139 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d23e8702-f109-4c61-a829-16e979e1b49c tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:59.124123 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d23e8702-f109-4c61-a829-16e979e1b49c tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:59.125249 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d23e8702-f109-4c61-a829-16e979e1b49c tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:59.125842 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d23e8702-f109-4c61-a829-16e979e1b49c tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:59.384206 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-900ce8b6-e0d5-4f02-9854-20aa4d01c6dd tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f5bc1f2209f4eb186d80e385b67c774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '65d5bcfa02af4e8284ba1342197e2860', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:19:59.395152 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-900ce8b6-e0d5-4f02-9854-20aa4d01c6dd tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] 10.222.0.16 "GET /compute/v2.1/servers/aa688a7e-0b28-4dfd-a159-8e19fd62ee02" status: 200 len: 1470 microversion: 2.1 time: 0.665008 Jul 16 21:19:59.397348 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1357/2758] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:19:58 2022] GET /compute/v2.1/servers/aa688a7e-0b28-4dfd-a159-8e19fd62ee02 => generated 1470 bytes in 668 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:19:59.411780 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-7d00f91e-d3e4-4f60-a2c0-b7780985cd8a tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:19:59.427110 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-d23e8702-f109-4c61-a829-16e979e1b49c tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eebb224a56814f92b12ebebebc6bc8a3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a092ce9ef40943f6b6ace6cd28481b71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:19:59.438578 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-d23e8702-f109-4c61-a829-16e979e1b49c tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] 10.222.0.16 "GET /compute/v2.1/servers/a56f8463-6006-4805-9021-1babc2d98f0f" status: 200 len: 1318 microversion: 2.1 time: 0.344326 Jul 16 21:19:59.448840 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1402/2759] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:19:59 2022] GET /compute/v2.1/servers/a56f8463-6006-4805-9021-1babc2d98f0f => generated 1318 bytes in 355 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:19:59.554293 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-7d00f91e-d3e4-4f60-a2c0-b7780985cd8a tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] 10.222.0.16 "DELETE /compute/v2.1/os-server-groups/4a10cd0b-f064-4621-961d-0ee3925657ef" status: 204 len: 0 microversion: 2.1 time: 0.151975 Jul 16 21:19:59.555571 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1358/2760] 10.222.0.16 () {58 vars in 1265 bytes} [Sat Jul 16 21:19:59 2022] DELETE /compute/v2.1/os-server-groups/4a10cd0b-f064-4621-961d-0ee3925657ef => generated 0 bytes in 157 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 16 21:19:59.567207 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-9075fbe0-3f34-40d6-8c5e-d13471e3d14c tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:19:59.569989 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-9075fbe0-3f34-40d6-8c5e-d13471e3d14c tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] [instance: 0a9564cb-c375-47e9-bfe6-e68d4700442b] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:19:59.582174 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9075fbe0-3f34-40d6-8c5e-d13471e3d14c tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:59.582730 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9075fbe0-3f34-40d6-8c5e-d13471e3d14c tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:59.584465 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9075fbe0-3f34-40d6-8c5e-d13471e3d14c tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:19:59.585406 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9075fbe0-3f34-40d6-8c5e-d13471e3d14c tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:19:59.602174 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-b3373dbd-5e42-4995-a327-d9a73d4c13eb tempest-SecurityGroupsNegativeTestJSON-448634586 tempest-SecurityGroupsNegativeTestJSON-448634586-project] Calling method '.version_select at 0x7fdd9d7c33a0>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:19:59.637287 ncdv-master-1 devstack@n-api.service[138680]: DEBUG neutronclient.v2_0.client [None req-b3373dbd-5e42-4995-a327-d9a73d4c13eb tempest-SecurityGroupsNegativeTestJSON-448634586 tempest-SecurityGroupsNegativeTestJSON-448634586-project] Error message: {"NeutronError": {"type": "SecurityGroupNotFound", "message": "Security group f074fce6-2c5a-424e-9d1d-18482f760a58 does not exist", "detail": ""}} {{(pid=138680) _handle_fault_response /usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py:262}} Jul 16 21:19:59.638811 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.network.security_group_api [None req-b3373dbd-5e42-4995-a327-d9a73d4c13eb tempest-SecurityGroupsNegativeTestJSON-448634586 tempest-SecurityGroupsNegativeTestJSON-448634586-project] Neutron security group f074fce6-2c5a-424e-9d1d-18482f760a58 not found {{(pid=138680) get /opt/stack/nova/nova/network/security_group_api.py:313}} Jul 16 21:19:59.638811 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-b3373dbd-5e42-4995-a327-d9a73d4c13eb tempest-SecurityGroupsNegativeTestJSON-448634586 tempest-SecurityGroupsNegativeTestJSON-448634586-project] HTTP exception thrown: Security group f074fce6-2c5a-424e-9d1d-18482f760a58 does not exist Jul 16 21:19:59.638811 ncdv-master-1 devstack@n-api.service[138680]: Neutron server returns request_ids: ['req-5fce9de2-56f4-4674-8078-1c077e83d418'] Jul 16 21:19:59.640697 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-b3373dbd-5e42-4995-a327-d9a73d4c13eb tempest-SecurityGroupsNegativeTestJSON-448634586 tempest-SecurityGroupsNegativeTestJSON-448634586-project] Request method failure captured: Jul 16 21:19:59.640697 ncdv-master-1 devstack@n-api.service[138680]: request: DELETE /compute/v2.1/os-security-groups/f074fce6-2c5a-424e-9d1d-18482f760a58 HTTP/1.1 Jul 16 21:19:59.640697 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:19:59.640697 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:19:59.640697 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:19:59.640697 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 0 Jul 16 21:19:59.640697 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:19:59.640697 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:19:59.640697 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:19:59.640697 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:19:59.640697 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0yt_ymXnmnlBefd4L74fkZK-MBunE4X9xmzQTfcTj2uXA0DwtttILe9S_ZZU4TyN65WBepNA2fANs8uBj7BhBHLuerjdECnqpUBAhcOwa__91XCZTXcvMcKNelBeydlTkeHX_Ec6v9VCcW32w5T8ou4e1K5WWpMzMDvJjkmnfvwuuJo Jul 16 21:19:59.640697 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:19:59.640697 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:19:59.640697 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:19:59.640697 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:19:59.640697 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:19:59.640697 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:19:59.640697 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: 6f40e610a1bf49a2a6913c4c6c6735b6 Jul 16 21:19:59.640697 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-SecurityGroupsNegativeTestJSON-448634586 Jul 16 21:19:59.640697 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:19:59.640697 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:19:59.640697 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/6f40e610a1bf49a2a6913c4c6c6735b6"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/6f40e610a1bf49a2a6913c4c6c6735b6"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_6f40e610a1bf49a2a6913c4c6c6735b6"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/6f40e610a1bf49a2a6913c4c6c6735b6"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:19:59.640697 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-SecurityGroupsNegativeTestJSON-448634586 Jul 16 21:19:59.640697 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: 6f40e610a1bf49a2a6913c4c6c6735b6 Jul 16 21:19:59.640697 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-SecurityGroupsNegativeTestJSON-448634586 Jul 16 21:19:59.640697 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-SecurityGroupsNegativeTestJSON-448634586-project Jul 16 21:19:59.640697 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:19:59.640697 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:19:59.640697 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: 27a08402c73946178157b98e3e4f1e82 Jul 16 21:19:59.640697 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-SecurityGroupsNegativeTestJSON-448634586-project Jul 16 21:19:59.640697 ncdv-master-1 devstack@n-api.service[138680]: method: Jul 16 21:19:59.640697 ncdv-master-1 devstack@n-api.service[138680]: exception: Security group f074fce6-2c5a-424e-9d1d-18482f760a58 does not exist Jul 16 21:19:59.640697 ncdv-master-1 devstack@n-api.service[138680]: Neutron server returns request_ids: ['req-5fce9de2-56f4-4674-8078-1c077e83d418'] Jul 16 21:19:59.640697 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': 'f074fce6-2c5a-424e-9d1d-18482f760a58'} Jul 16 21:19:59.640697 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:19:59.640697 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:19:59.640697 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/security_group_api.py", line 309, in get Jul 16 21:19:59.640697 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi group = neutron.show_security_group(id).get('security_group') Jul 16 21:19:59.640697 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 197, in wrapper Jul 16 21:19:59.640697 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Jul 16 21:19:59.640697 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 1040, in show_security_group Jul 16 21:19:59.640697 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return self.get(self.security_group_path % (security_group), Jul 16 21:19:59.640697 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 197, in wrapper Jul 16 21:19:59.640697 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Jul 16 21:19:59.640697 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 356, in get Jul 16 21:19:59.640697 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return self.retry_request("GET", action, body=body, Jul 16 21:19:59.640697 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 197, in wrapper Jul 16 21:19:59.640697 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Jul 16 21:19:59.640697 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 333, in retry_request Jul 16 21:19:59.640697 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return self.do_request(method, action, body=body, Jul 16 21:19:59.640697 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 197, in wrapper Jul 16 21:19:59.640697 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Jul 16 21:19:59.640697 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 297, in do_request Jul 16 21:19:59.640697 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi self._handle_fault_response(status_code, replybody, resp) Jul 16 21:19:59.640697 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 197, in wrapper Jul 16 21:19:59.640697 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Jul 16 21:19:59.640697 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 272, in _handle_fault_response Jul 16 21:19:59.640697 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi exception_handler_v20(status_code, error_body) Jul 16 21:19:59.640697 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 90, in exception_handler_v20 Jul 16 21:19:59.640697 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise client_exc(message=error_message, Jul 16 21:19:59.640697 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi neutronclient.common.exceptions.NotFound: Security group f074fce6-2c5a-424e-9d1d-18482f760a58 does not exist Jul 16 21:19:59.647273 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-5fce9de2-56f4-4674-8078-1c077e83d418'] Jul 16 21:19:59.647273 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:19:59.647273 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:19:59.647273 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:19:59.647273 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:19:59.647273 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/security_groups.py", line 178, in delete Jul 16 21:19:59.647273 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi security_group = security_group_api.get(context, id) Jul 16 21:19:59.647273 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/security_group_api.py", line 314, in get Jul 16 21:19:59.647273 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.SecurityGroupNotFound(str(e)) Jul 16 21:19:59.647273 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.SecurityGroupNotFound: Security group f074fce6-2c5a-424e-9d1d-18482f760a58 does not exist Jul 16 21:19:59.647273 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-5fce9de2-56f4-4674-8078-1c077e83d418'] Jul 16 21:19:59.647273 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:19:59.647273 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:19:59.647273 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:19:59.647273 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:19:59.647273 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:19:59.647273 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:19:59.647273 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:19:59.647273 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:19:59.647273 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Jul 16 21:19:59.647273 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Jul 16 21:19:59.647273 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:19:59.647273 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:19:59.647273 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/security_groups.py", line 181, in delete Jul 16 21:19:59.647273 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=exp.format_message()) Jul 16 21:19:59.647273 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Security group f074fce6-2c5a-424e-9d1d-18482f760a58 does not exist Jul 16 21:19:59.647273 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-5fce9de2-56f4-4674-8078-1c077e83d418'] Jul 16 21:19:59.647273 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:19:59.647273 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:19:59.647273 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:19:59.647273 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:19:59.647273 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:19:59.647273 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:19:59.647273 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:19:59.647273 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:19:59.647273 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Security group f074fce6-2c5a-424e-9d1d-18482f760a58 does not exist Jul 16 21:19:59.647273 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-5fce9de2-56f4-4674-8078-1c077e83d418'] Jul 16 21:19:59.647273 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:19:59.647273 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-b3373dbd-5e42-4995-a327-d9a73d4c13eb tempest-SecurityGroupsNegativeTestJSON-448634586 tempest-SecurityGroupsNegativeTestJSON-448634586-project] Returning 404 to user: Security group f074fce6-2c5a-424e-9d1d-18482f760a58 does not exist Jul 16 21:19:59.647273 ncdv-master-1 devstack@n-api.service[138680]: Neutron server returns request_ids: ['req-5fce9de2-56f4-4674-8078-1c077e83d418'] {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:19:59.647273 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-b3373dbd-5e42-4995-a327-d9a73d4c13eb tempest-SecurityGroupsNegativeTestJSON-448634586 tempest-SecurityGroupsNegativeTestJSON-448634586-project] 10.222.0.16 "DELETE /compute/v2.1/os-security-groups/f074fce6-2c5a-424e-9d1d-18482f760a58" status: 404 len: 194 microversion: 2.1 time: 0.191606 Jul 16 21:19:59.647273 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1403/2761] 10.222.0.16 () {58 vars in 1271 bytes} [Sat Jul 16 21:19:59 2022] DELETE /compute/v2.1/os-security-groups/f074fce6-2c5a-424e-9d1d-18482f760a58 => generated 194 bytes in 193 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:19:59.652513 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-47e96db6-4ff3-484f-a1c7-16c7fbb222bc tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:19:59.674027 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-47e96db6-4ff3-484f-a1c7-16c7fbb222bc tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] 10.222.0.16 "GET /compute/v2.1/os-server-groups" status: 200 len: 187 microversion: 2.1 time: 0.030625 Jul 16 21:19:59.675226 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1404/2762] 10.222.0.16 () {58 vars in 1151 bytes} [Sat Jul 16 21:19:59 2022] GET /compute/v2.1/os-server-groups => generated 187 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:19:59.682248 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-f5a290df-e157-44d9-baac-e9ed19e07923 tempest-SecurityGroupsNegativeTestJSON-448634586 tempest-SecurityGroupsNegativeTestJSON-448634586-project] 10.222.0.16 "DELETE /compute/v2.1/os-security-groups/" status: 404 len: 112 microversion: - time: 0.005823 Jul 16 21:19:59.683276 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1405/2763] 10.222.0.16 () {58 vars in 1163 bytes} [Sat Jul 16 21:19:59 2022] DELETE /compute/v2.1/os-security-groups/ => generated 112 bytes in 7 msecs (HTTP/1.1 404) 5 headers in 228 bytes (1 switches on core 0) Jul 16 21:19:59.693162 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-be42a069-c3ac-4d77-b203-f08029d33cba tempest-SecurityGroupsNegativeTestJSON-448634586 tempest-SecurityGroupsNegativeTestJSON-448634586-project] Calling method '.version_select at 0x7fdd9cc35940>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:19:59.701986 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_api:SELECT=86,nova_api:INSERT=5,nova_api:DELETE=9 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:19:59.714854 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-9075fbe0-3f34-40d6-8c5e-d13471e3d14c tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1f72b1e0b4124480bfd53b5c22a343e0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0be222897b3a4864a39e9fc7d481340a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:19:59.722196 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-9075fbe0-3f34-40d6-8c5e-d13471e3d14c tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] 10.222.0.16 "GET /compute/v2.1/servers/0a9564cb-c375-47e9-bfe6-e68d4700442b" status: 200 len: 1329 microversion: 2.1 time: 0.164181 Jul 16 21:19:59.722784 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1359/2764] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:19:59 2022] GET /compute/v2.1/servers/0a9564cb-c375-47e9-bfe6-e68d4700442b => generated 1329 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:19:59.832589 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-d7c7baa2-d4e8-4416-b1e1-015d02e7f09d tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerGroupTestJSON-server-1348019279", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452", "networks": [{"uuid": "8d5e0ca5-2915-4edb-b98f-20f4995bc25f"}]}, "os:scheduler_hints": {"group": "129ae0b9-7a56-4a16-8a71-8c4b96c9564d"}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:19:59.928356 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.network.neutron [None req-d7c7baa2-d4e8-4416-b1e1-015d02e7f09d tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] validate_networks() for [('8d5e0ca5-2915-4edb-b98f-20f4995bc25f', None, None, None, None, None)] {{(pid=138681) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 16 21:20:00.174544 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-be42a069-c3ac-4d77-b203-f08029d33cba tempest-SecurityGroupsNegativeTestJSON-448634586 tempest-SecurityGroupsNegativeTestJSON-448634586-project] 10.222.0.16 "GET /compute/v2.1/os-security-groups" status: 200 len: 707 microversion: 2.1 time: 0.486402 Jul 16 21:20:00.176170 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1406/2765] 10.222.0.16 () {58 vars in 1157 bytes} [Sat Jul 16 21:19:59 2022] GET /compute/v2.1/os-security-groups => generated 707 bytes in 489 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:20:00.182289 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-b807cc43-5d1f-4efa-9a41-56d73d48a37d req-b5f0085c-7fef-4e27-aab0-bb0bc32689ac service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "aa688a7e-0b28-4dfd-a159-8e19fd62ee02", "name": "network-vif-deleted", "tag": "0eaf070b-3b01-4123-bf36-5947ef05d97a"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:20:00.206837 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-b807cc43-5d1f-4efa-9a41-56d73d48a37d req-b5f0085c-7fef-4e27-aab0-bb0bc32689ac service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:00.211830 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-b807cc43-5d1f-4efa-9a41-56d73d48a37d req-b5f0085c-7fef-4e27-aab0-bb0bc32689ac service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:00.265321 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-b807cc43-5d1f-4efa-9a41-56d73d48a37d req-b5f0085c-7fef-4e27-aab0-bb0bc32689ac service nova] Creating event network-vif-deleted:0eaf070b-3b01-4123-bf36-5947ef05d97a for instance aa688a7e-0b28-4dfd-a159-8e19fd62ee02 on nch1-master-1 Jul 16 21:20:00.284920 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-b807cc43-5d1f-4efa-9a41-56d73d48a37d req-b5f0085c-7fef-4e27-aab0-bb0bc32689ac service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.107223 Jul 16 21:20:00.286406 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1407/2766] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:20:00 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 110 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:20:00.332927 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-5de297b1-ed98-44aa-8f98-58edd6751e86 tempest-SecurityGroupsNegativeTestJSON-448634586 tempest-SecurityGroupsNegativeTestJSON-448634586-project] Calling method '.version_select at 0x7fdd9d0ad0d0>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:20:01.114188 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-d7c7baa2-d4e8-4416-b1e1-015d02e7f09d tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Flavor limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 16 21:20:01.119559 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-d7c7baa2-d4e8-4416-b1e1-015d02e7f09d tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Image limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 16 21:20:01.120252 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-d7c7baa2-d4e8-4416-b1e1-015d02e7f09d tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Flavor pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 16 21:20:01.120684 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-d7c7baa2-d4e8-4416-b1e1-015d02e7f09d tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Image pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 16 21:20:01.121114 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-d7c7baa2-d4e8-4416-b1e1-015d02e7f09d tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 16 21:20:01.151577 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d7c7baa2-d4e8-4416-b1e1-015d02e7f09d tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:01.152431 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d7c7baa2-d4e8-4416-b1e1-015d02e7f09d tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:01.153638 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d7c7baa2-d4e8-4416-b1e1-015d02e7f09d tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:01.154245 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d7c7baa2-d4e8-4416-b1e1-015d02e7f09d tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:01.157105 ncdv-master-1 devstack@n-api.service[138680]: DEBUG neutronclient.v2_0.client [None req-5de297b1-ed98-44aa-8f98-58edd6751e86 tempest-SecurityGroupsNegativeTestJSON-448634586 tempest-SecurityGroupsNegativeTestJSON-448634586-project] Error message: {"NeutronError": {"type": "SecurityGroupCannotRemoveDefault", "message": "Insufficient rights for removing default security group.", "detail": ""}} {{(pid=138680) _handle_fault_response /usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py:262}} Jul 16 21:20:01.157835 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-5de297b1-ed98-44aa-8f98-58edd6751e86 tempest-SecurityGroupsNegativeTestJSON-448634586 tempest-SecurityGroupsNegativeTestJSON-448634586-project] HTTP exception thrown: Insufficient rights for removing default security group. Jul 16 21:20:01.157835 ncdv-master-1 devstack@n-api.service[138680]: Neutron server returns request_ids: ['req-8baf2363-b21a-4190-8403-49f8f36b22b1'] Jul 16 21:20:01.163663 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-5de297b1-ed98-44aa-8f98-58edd6751e86 tempest-SecurityGroupsNegativeTestJSON-448634586 tempest-SecurityGroupsNegativeTestJSON-448634586-project] Request method failure captured: Jul 16 21:20:01.163663 ncdv-master-1 devstack@n-api.service[138680]: request: DELETE /compute/v2.1/os-security-groups/2dcbeaf3-697f-4a6c-8da1-fdd7b5f15c61 HTTP/1.1 Jul 16 21:20:01.163663 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:20:01.163663 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:20:01.163663 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:20:01.163663 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 0 Jul 16 21:20:01.163663 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:20:01.163663 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:20:01.163663 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:20:01.163663 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:20:01.163663 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0yt_ymXnmnlBefd4L74fkZK-MBunE4X9xmzQTfcTj2uXA0DwtttILe9S_ZZU4TyN65WBepNA2fANs8uBj7BhBHLuerjdECnqpUBAhcOwa__91XCZTXcvMcKNelBeydlTkeHX_Ec6v9VCcW32w5T8ou4e1K5WWpMzMDvJjkmnfvwuuJo Jul 16 21:20:01.163663 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:20:01.163663 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:20:01.163663 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:20:01.163663 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:20:01.163663 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:20:01.163663 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:20:01.163663 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: 6f40e610a1bf49a2a6913c4c6c6735b6 Jul 16 21:20:01.163663 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-SecurityGroupsNegativeTestJSON-448634586 Jul 16 21:20:01.163663 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:20:01.163663 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:20:01.163663 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/6f40e610a1bf49a2a6913c4c6c6735b6"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/6f40e610a1bf49a2a6913c4c6c6735b6"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_6f40e610a1bf49a2a6913c4c6c6735b6"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/6f40e610a1bf49a2a6913c4c6c6735b6"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:20:01.163663 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-SecurityGroupsNegativeTestJSON-448634586 Jul 16 21:20:01.163663 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: 6f40e610a1bf49a2a6913c4c6c6735b6 Jul 16 21:20:01.163663 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-SecurityGroupsNegativeTestJSON-448634586 Jul 16 21:20:01.163663 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-SecurityGroupsNegativeTestJSON-448634586-project Jul 16 21:20:01.163663 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:20:01.163663 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:20:01.163663 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: 27a08402c73946178157b98e3e4f1e82 Jul 16 21:20:01.163663 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-SecurityGroupsNegativeTestJSON-448634586-project Jul 16 21:20:01.163663 ncdv-master-1 devstack@n-api.service[138680]: method: Jul 16 21:20:01.163663 ncdv-master-1 devstack@n-api.service[138680]: exception: Insufficient rights for removing default security group. Jul 16 21:20:01.163663 ncdv-master-1 devstack@n-api.service[138680]: Neutron server returns request_ids: ['req-8baf2363-b21a-4190-8403-49f8f36b22b1'] Jul 16 21:20:01.163663 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': '2dcbeaf3-697f-4a6c-8da1-fdd7b5f15c61'} Jul 16 21:20:01.163663 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:20:01.163663 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:20:01.163663 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/security_group_api.py", line 359, in destroy Jul 16 21:20:01.163663 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi neutron.delete_security_group(security_group['id']) Jul 16 21:20:01.163663 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 197, in wrapper Jul 16 21:20:01.163663 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Jul 16 21:20:01.163663 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 1045, in delete_security_group Jul 16 21:20:01.163663 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return self.delete(self.security_group_path % (security_group)) Jul 16 21:20:01.163663 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 197, in wrapper Jul 16 21:20:01.163663 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Jul 16 21:20:01.163663 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 352, in delete Jul 16 21:20:01.163663 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return self.retry_request("DELETE", action, body=body, Jul 16 21:20:01.163663 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 197, in wrapper Jul 16 21:20:01.163663 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Jul 16 21:20:01.163663 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 333, in retry_request Jul 16 21:20:01.163663 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return self.do_request(method, action, body=body, Jul 16 21:20:01.163663 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 197, in wrapper Jul 16 21:20:01.163663 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Jul 16 21:20:01.163663 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 297, in do_request Jul 16 21:20:01.163663 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi self._handle_fault_response(status_code, replybody, resp) Jul 16 21:20:01.163663 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 197, in wrapper Jul 16 21:20:01.163663 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Jul 16 21:20:01.163663 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 272, in _handle_fault_response Jul 16 21:20:01.163663 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi exception_handler_v20(status_code, error_body) Jul 16 21:20:01.163663 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 90, in exception_handler_v20 Jul 16 21:20:01.163663 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise client_exc(message=error_message, Jul 16 21:20:01.163663 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi neutronclient.common.exceptions.Conflict: Insufficient rights for removing default security group. Jul 16 21:20:01.176625 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-8baf2363-b21a-4190-8403-49f8f36b22b1'] Jul 16 21:20:01.176625 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:20:01.176625 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:20:01.176625 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:20:01.176625 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:20:01.176625 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/security_groups.py", line 179, in delete Jul 16 21:20:01.176625 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi security_group_api.destroy(context, security_group) Jul 16 21:20:01.176625 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/security_group_api.py", line 364, in destroy Jul 16 21:20:01.176625 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.Invalid(str(e)) Jul 16 21:20:01.176625 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.Invalid: Insufficient rights for removing default security group. Jul 16 21:20:01.176625 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-8baf2363-b21a-4190-8403-49f8f36b22b1'] Jul 16 21:20:01.176625 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:20:01.176625 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:20:01.176625 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:20:01.176625 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:20:01.176625 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:20:01.176625 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:20:01.176625 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:20:01.176625 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:20:01.176625 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Jul 16 21:20:01.176625 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Jul 16 21:20:01.176625 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:20:01.176625 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:20:01.176625 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/security_groups.py", line 183, in delete Jul 16 21:20:01.176625 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=exp.format_message()) Jul 16 21:20:01.176625 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Insufficient rights for removing default security group. Jul 16 21:20:01.176625 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-8baf2363-b21a-4190-8403-49f8f36b22b1'] Jul 16 21:20:01.176625 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:20:01.176625 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:20:01.176625 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:20:01.176625 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:20:01.176625 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:20:01.176625 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:20:01.176625 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:20:01.176625 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:20:01.176625 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Insufficient rights for removing default security group. Jul 16 21:20:01.176625 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-8baf2363-b21a-4190-8403-49f8f36b22b1'] Jul 16 21:20:01.176625 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:20:01.176625 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-5de297b1-ed98-44aa-8f98-58edd6751e86 tempest-SecurityGroupsNegativeTestJSON-448634586 tempest-SecurityGroupsNegativeTestJSON-448634586-project] Returning 400 to user: Insufficient rights for removing default security group. Jul 16 21:20:01.176625 ncdv-master-1 devstack@n-api.service[138680]: Neutron server returns request_ids: ['req-8baf2363-b21a-4190-8403-49f8f36b22b1'] {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:20:01.176625 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-5de297b1-ed98-44aa-8f98-58edd6751e86 tempest-SecurityGroupsNegativeTestJSON-448634586 tempest-SecurityGroupsNegativeTestJSON-448634586-project] 10.222.0.16 "DELETE /compute/v2.1/os-security-groups/2dcbeaf3-697f-4a6c-8da1-fdd7b5f15c61" status: 400 len: 182 microversion: 2.1 time: 0.880136 Jul 16 21:20:01.176625 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1408/2767] 10.222.0.16 () {58 vars in 1271 bytes} [Sat Jul 16 21:20:00 2022] DELETE /compute/v2.1/os-security-groups/2dcbeaf3-697f-4a6c-8da1-fdd7b5f15c61 => generated 182 bytes in 881 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Jul 16 21:20:01.180646 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-431ca5db-290a-4764-b558-aaab82255e95 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:20:01.181421 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-431ca5db-290a-4764-b558-aaab82255e95 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] [instance: aa688a7e-0b28-4dfd-a159-8e19fd62ee02] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:20:01.191393 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-431ca5db-290a-4764-b558-aaab82255e95 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:01.192134 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-431ca5db-290a-4764-b558-aaab82255e95 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:01.194322 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-431ca5db-290a-4764-b558-aaab82255e95 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:01.194964 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-431ca5db-290a-4764-b558-aaab82255e95 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:01.203392 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-d7c7baa2-d4e8-4416-b1e1-015d02e7f09d tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Getting quotas for project 772bd01188344e288824730238fee49b. Resources: {'ram', 'instances', 'cores'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:20:01.222847 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-d7c7baa2-d4e8-4416-b1e1-015d02e7f09d tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Getting quotas for user fe680ccdc60e410b812794de8aa73e0f and project 772bd01188344e288824730238fee49b. Resources: {'ram', 'instances', 'cores'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:20:01.252634 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-d7c7baa2-d4e8-4416-b1e1-015d02e7f09d tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Going to run 1 instances... {{(pid=138681) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 16 21:20:01.257410 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-431ca5db-290a-4764-b558-aaab82255e95 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:01.259000 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-431ca5db-290a-4764-b558-aaab82255e95 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:01.264974 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.objects.instance [None req-431ca5db-290a-4764-b558-aaab82255e95 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lazy-loading 'fault' on Instance uuid aa688a7e-0b28-4dfd-a159-8e19fd62ee02 {{(pid=138680) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:20:01.265130 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-d7c7baa2-d4e8-4416-b1e1-015d02e7f09d tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] [instance: 84aa4a51-e376-4829-9a38-2ea447ebb413] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138681) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 16 21:20:01.334035 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-431ca5db-290a-4764-b558-aaab82255e95 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f5bc1f2209f4eb186d80e385b67c774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '65d5bcfa02af4e8284ba1342197e2860', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:20:01.347145 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-431ca5db-290a-4764-b558-aaab82255e95 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] 10.222.0.16 "GET /compute/v2.1/servers/aa688a7e-0b28-4dfd-a159-8e19fd62ee02" status: 200 len: 1304 microversion: 2.1 time: 0.177017 Jul 16 21:20:01.348341 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1409/2768] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:20:01 2022] GET /compute/v2.1/servers/aa688a7e-0b28-4dfd-a159-8e19fd62ee02 => generated 1304 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:20:01.353928 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-f35fc21d-6580-4100-8ec6-f5a132e7755a tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:20:01.354783 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-f35fc21d-6580-4100-8ec6-f5a132e7755a tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] [instance: a56f8463-6006-4805-9021-1babc2d98f0f] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:20:01.364425 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f35fc21d-6580-4100-8ec6-f5a132e7755a tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:01.368048 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f35fc21d-6580-4100-8ec6-f5a132e7755a tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:01.368048 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f35fc21d-6580-4100-8ec6-f5a132e7755a tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:01.368048 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f35fc21d-6580-4100-8ec6-f5a132e7755a tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:01.507445 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-d7c7baa2-d4e8-4416-b1e1-015d02e7f09d tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Getting quotas for project 772bd01188344e288824730238fee49b. Resources: {'server_group_members'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:20:01.523853 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-d7c7baa2-d4e8-4416-b1e1-015d02e7f09d tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Getting quotas for user fe680ccdc60e410b812794de8aa73e0f and project 772bd01188344e288824730238fee49b. Resources: {'server_group_members'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:20:01.619247 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-f35fc21d-6580-4100-8ec6-f5a132e7755a tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eebb224a56814f92b12ebebebc6bc8a3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a092ce9ef40943f6b6ace6cd28481b71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:20:01.634174 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-f35fc21d-6580-4100-8ec6-f5a132e7755a tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] 10.222.0.16 "GET /compute/v2.1/servers/a56f8463-6006-4805-9021-1babc2d98f0f" status: 200 len: 1318 microversion: 2.1 time: 0.284484 Jul 16 21:20:01.635088 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1410/2769] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:20:01 2022] GET /compute/v2.1/servers/a56f8463-6006-4805-9021-1babc2d98f0f => generated 1318 bytes in 286 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:20:01.642357 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-5ff5b933-816c-4495-9ece-12a507fd16ab tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:20:01.643395 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-5ff5b933-816c-4495-9ece-12a507fd16ab tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] [instance: 0a9564cb-c375-47e9-bfe6-e68d4700442b] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:20:01.651317 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5ff5b933-816c-4495-9ece-12a507fd16ab tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:01.654366 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5ff5b933-816c-4495-9ece-12a507fd16ab tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:01.654366 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5ff5b933-816c-4495-9ece-12a507fd16ab tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:01.654366 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5ff5b933-816c-4495-9ece-12a507fd16ab tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:01.696958 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-d7c7baa2-d4e8-4416-b1e1-015d02e7f09d tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Getting quotas for project 772bd01188344e288824730238fee49b. Resources: {'server_group_members'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:20:01.702979 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-d7c7baa2-d4e8-4416-b1e1-015d02e7f09d tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Getting quotas for user fe680ccdc60e410b812794de8aa73e0f and project 772bd01188344e288824730238fee49b. Resources: {'server_group_members'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:20:01.781729 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-5ff5b933-816c-4495-9ece-12a507fd16ab tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1f72b1e0b4124480bfd53b5c22a343e0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0be222897b3a4864a39e9fc7d481340a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:20:01.795070 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-5ff5b933-816c-4495-9ece-12a507fd16ab tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] 10.222.0.16 "GET /compute/v2.1/servers/0a9564cb-c375-47e9-bfe6-e68d4700442b" status: 200 len: 1335 microversion: 2.1 time: 0.157768 Jul 16 21:20:01.807240 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1411/2770] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:20:01 2022] GET /compute/v2.1/servers/0a9564cb-c375-47e9-bfe6-e68d4700442b => generated 1335 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:20:01.809065 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-d7c7baa2-d4e8-4416-b1e1-015d02e7f09d tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 1.980880 Jul 16 21:20:01.809963 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1360/2771] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 21:19:59 2022] POST /compute/v2.1/servers => generated 380 bytes in 1985 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Jul 16 21:20:01.813074 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-14c82a61-770c-456c-a576-b5567634b2c2 req-fdd95cca-c994-4439-b545-9810ff80052b service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "a56f8463-6006-4805-9021-1babc2d98f0f", "name": "network-vif-plugged", "status": "completed", "tag": "d407f4a6-f978-413f-821d-6a31fd8785c8"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:20:01.824887 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-215b3912-4edf-4b9e-9519-584ee433da96 tempest-SecurityGroupsNegativeTestJSON-448634586 tempest-SecurityGroupsNegativeTestJSON-448634586-project] Calling method '.version_select at 0x7fdda29ab3a0>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:20:01.833351 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-14c82a61-770c-456c-a576-b5567634b2c2 req-fdd95cca-c994-4439-b545-9810ff80052b service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:01.834571 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-14c82a61-770c-456c-a576-b5567634b2c2 req-fdd95cca-c994-4439-b545-9810ff80052b service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:01.889769 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-14c82a61-770c-456c-a576-b5567634b2c2 req-fdd95cca-c994-4439-b545-9810ff80052b service nova] Creating event network-vif-plugged:d407f4a6-f978-413f-821d-6a31fd8785c8 for instance a56f8463-6006-4805-9021-1babc2d98f0f on nch1-master-1 Jul 16 21:20:01.899647 ncdv-master-1 devstack@n-api.service[138681]: DEBUG neutronclient.v2_0.client [None req-215b3912-4edf-4b9e-9519-584ee433da96 tempest-SecurityGroupsNegativeTestJSON-448634586 tempest-SecurityGroupsNegativeTestJSON-448634586-project] Error message: {"NeutronError": {"type": "SecurityGroupNotFound", "message": "Security group 6f4159fc-9681-47e7-b0b7-a535874f71d0 does not exist", "detail": ""}} {{(pid=138681) _handle_fault_response /usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py:262}} Jul 16 21:20:01.900541 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.network.security_group_api [None req-215b3912-4edf-4b9e-9519-584ee433da96 tempest-SecurityGroupsNegativeTestJSON-448634586 tempest-SecurityGroupsNegativeTestJSON-448634586-project] Neutron security group 6f4159fc-9681-47e7-b0b7-a535874f71d0 not found {{(pid=138681) get /opt/stack/nova/nova/network/security_group_api.py:313}} Jul 16 21:20:01.901262 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-215b3912-4edf-4b9e-9519-584ee433da96 tempest-SecurityGroupsNegativeTestJSON-448634586 tempest-SecurityGroupsNegativeTestJSON-448634586-project] HTTP exception thrown: Security group 6f4159fc-9681-47e7-b0b7-a535874f71d0 does not exist Jul 16 21:20:01.901262 ncdv-master-1 devstack@n-api.service[138681]: Neutron server returns request_ids: ['req-32417f29-df59-43e6-b794-7717a8f63ee9'] Jul 16 21:20:01.904601 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-215b3912-4edf-4b9e-9519-584ee433da96 tempest-SecurityGroupsNegativeTestJSON-448634586 tempest-SecurityGroupsNegativeTestJSON-448634586-project] Request method failure captured: Jul 16 21:20:01.904601 ncdv-master-1 devstack@n-api.service[138681]: request: GET /compute/v2.1/os-security-groups/6f4159fc-9681-47e7-b0b7-a535874f71d0 HTTP/1.1 Jul 16 21:20:01.904601 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:20:01.904601 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:20:01.904601 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:20:01.904601 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 21:20:01.904601 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:20:01.904601 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:20:01.904601 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:20:01.904601 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:20:01.904601 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0yt_ymXnmnlBefd4L74fkZK-MBunE4X9xmzQTfcTj2uXA0DwtttILe9S_ZZU4TyN65WBepNA2fANs8uBj7BhBHLuerjdECnqpUBAhcOwa__91XCZTXcvMcKNelBeydlTkeHX_Ec6v9VCcW32w5T8ou4e1K5WWpMzMDvJjkmnfvwuuJo Jul 16 21:20:01.904601 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:20:01.904601 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:20:01.904601 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:20:01.904601 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:20:01.904601 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:20:01.904601 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:20:01.904601 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: 6f40e610a1bf49a2a6913c4c6c6735b6 Jul 16 21:20:01.904601 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-SecurityGroupsNegativeTestJSON-448634586 Jul 16 21:20:01.904601 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:20:01.904601 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:20:01.904601 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/6f40e610a1bf49a2a6913c4c6c6735b6"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/6f40e610a1bf49a2a6913c4c6c6735b6"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_6f40e610a1bf49a2a6913c4c6c6735b6"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/6f40e610a1bf49a2a6913c4c6c6735b6"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:20:01.904601 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-SecurityGroupsNegativeTestJSON-448634586 Jul 16 21:20:01.904601 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: 6f40e610a1bf49a2a6913c4c6c6735b6 Jul 16 21:20:01.904601 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-SecurityGroupsNegativeTestJSON-448634586 Jul 16 21:20:01.904601 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-SecurityGroupsNegativeTestJSON-448634586-project Jul 16 21:20:01.904601 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:20:01.904601 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:20:01.904601 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: 27a08402c73946178157b98e3e4f1e82 Jul 16 21:20:01.904601 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-SecurityGroupsNegativeTestJSON-448634586-project Jul 16 21:20:01.904601 ncdv-master-1 devstack@n-api.service[138681]: method: Jul 16 21:20:01.904601 ncdv-master-1 devstack@n-api.service[138681]: exception: Security group 6f4159fc-9681-47e7-b0b7-a535874f71d0 does not exist Jul 16 21:20:01.904601 ncdv-master-1 devstack@n-api.service[138681]: Neutron server returns request_ids: ['req-32417f29-df59-43e6-b794-7717a8f63ee9'] Jul 16 21:20:01.904601 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': '6f4159fc-9681-47e7-b0b7-a535874f71d0'} Jul 16 21:20:01.904601 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:20:01.904601 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:20:01.904601 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/security_group_api.py", line 309, in get Jul 16 21:20:01.904601 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi group = neutron.show_security_group(id).get('security_group') Jul 16 21:20:01.904601 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 197, in wrapper Jul 16 21:20:01.904601 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Jul 16 21:20:01.904601 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 1040, in show_security_group Jul 16 21:20:01.904601 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return self.get(self.security_group_path % (security_group), Jul 16 21:20:01.904601 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 197, in wrapper Jul 16 21:20:01.904601 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Jul 16 21:20:01.904601 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 356, in get Jul 16 21:20:01.904601 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return self.retry_request("GET", action, body=body, Jul 16 21:20:01.904601 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 197, in wrapper Jul 16 21:20:01.904601 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Jul 16 21:20:01.904601 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 333, in retry_request Jul 16 21:20:01.904601 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return self.do_request(method, action, body=body, Jul 16 21:20:01.904601 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 197, in wrapper Jul 16 21:20:01.904601 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Jul 16 21:20:01.904601 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 297, in do_request Jul 16 21:20:01.904601 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi self._handle_fault_response(status_code, replybody, resp) Jul 16 21:20:01.904601 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 197, in wrapper Jul 16 21:20:01.904601 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Jul 16 21:20:01.904601 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 272, in _handle_fault_response Jul 16 21:20:01.904601 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi exception_handler_v20(status_code, error_body) Jul 16 21:20:01.904601 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 90, in exception_handler_v20 Jul 16 21:20:01.904601 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise client_exc(message=error_message, Jul 16 21:20:01.904601 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi neutronclient.common.exceptions.NotFound: Security group 6f4159fc-9681-47e7-b0b7-a535874f71d0 does not exist Jul 16 21:20:01.911048 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-32417f29-df59-43e6-b794-7717a8f63ee9'] Jul 16 21:20:01.911048 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:20:01.911048 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:20:01.911048 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:20:01.911048 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:20:01.911048 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/security_groups.py", line 158, in show Jul 16 21:20:01.911048 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi security_group = security_group_api.get(context, id) Jul 16 21:20:01.911048 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/security_group_api.py", line 314, in get Jul 16 21:20:01.911048 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.SecurityGroupNotFound(str(e)) Jul 16 21:20:01.911048 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.SecurityGroupNotFound: Security group 6f4159fc-9681-47e7-b0b7-a535874f71d0 does not exist Jul 16 21:20:01.911048 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-32417f29-df59-43e6-b794-7717a8f63ee9'] Jul 16 21:20:01.911048 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:20:01.911048 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:20:01.911048 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:20:01.911048 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:20:01.911048 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:20:01.911048 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:20:01.911048 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:20:01.911048 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:20:01.911048 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Jul 16 21:20:01.911048 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Jul 16 21:20:01.911048 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:20:01.911048 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:20:01.911048 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/security_groups.py", line 160, in show Jul 16 21:20:01.911048 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=exp.format_message()) Jul 16 21:20:01.911048 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Security group 6f4159fc-9681-47e7-b0b7-a535874f71d0 does not exist Jul 16 21:20:01.911048 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-32417f29-df59-43e6-b794-7717a8f63ee9'] Jul 16 21:20:01.911048 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:20:01.911048 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:20:01.911048 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:20:01.911048 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:20:01.911048 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:20:01.911048 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:20:01.911048 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:20:01.911048 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:20:01.911048 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Security group 6f4159fc-9681-47e7-b0b7-a535874f71d0 does not exist Jul 16 21:20:01.911048 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-32417f29-df59-43e6-b794-7717a8f63ee9'] Jul 16 21:20:01.911048 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:20:01.911048 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-215b3912-4edf-4b9e-9519-584ee433da96 tempest-SecurityGroupsNegativeTestJSON-448634586 tempest-SecurityGroupsNegativeTestJSON-448634586-project] Returning 404 to user: Security group 6f4159fc-9681-47e7-b0b7-a535874f71d0 does not exist Jul 16 21:20:01.911048 ncdv-master-1 devstack@n-api.service[138681]: Neutron server returns request_ids: ['req-32417f29-df59-43e6-b794-7717a8f63ee9'] {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:20:01.911048 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-215b3912-4edf-4b9e-9519-584ee433da96 tempest-SecurityGroupsNegativeTestJSON-448634586 tempest-SecurityGroupsNegativeTestJSON-448634586-project] 10.222.0.16 "GET /compute/v2.1/os-security-groups/6f4159fc-9681-47e7-b0b7-a535874f71d0" status: 404 len: 194 microversion: 2.1 time: 0.095711 Jul 16 21:20:01.911048 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1361/2772] 10.222.0.16 () {58 vars in 1268 bytes} [Sat Jul 16 21:20:01 2022] GET /compute/v2.1/os-security-groups/6f4159fc-9681-47e7-b0b7-a535874f71d0 => generated 194 bytes in 97 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:20:01.920790 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-d529d0f8-ef9f-4fa4-bbbe-e9dff03c202e tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:20:01.921881 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-d529d0f8-ef9f-4fa4-bbbe-e9dff03c202e tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] [instance: 84aa4a51-e376-4829-9a38-2ea447ebb413] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:20:01.925461 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-14c82a61-770c-456c-a576-b5567634b2c2 req-fdd95cca-c994-4439-b545-9810ff80052b service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.112649 Jul 16 21:20:01.929967 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1412/2773] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:20:01 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 122 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:20:01.946043 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-63ec2c01-3099-47c0-a6c3-b08586f18725 tempest-SecurityGroupsNegativeTestJSON-448634586 tempest-SecurityGroupsNegativeTestJSON-448634586-project] Action: 'update', calling method: .version_select at 0x7fdd9d3ea310>, body: {"security_group": {"name": "tempest-sg-134363798", "description": "tempest-description-1972526295"}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:20:01.999699 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-d529d0f8-ef9f-4fa4-bbbe-e9dff03c202e tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fe680ccdc60e410b812794de8aa73e0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '772bd01188344e288824730238fee49b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:20:02.008933 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-d529d0f8-ef9f-4fa4-bbbe-e9dff03c202e tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] 10.222.0.16 "GET /compute/v2.1/servers/84aa4a51-e376-4829-9a38-2ea447ebb413" status: 200 len: 1213 microversion: 2.1 time: 0.093080 Jul 16 21:20:02.015811 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1362/2774] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:20:01 2022] GET /compute/v2.1/servers/84aa4a51-e376-4829-9a38-2ea447ebb413 => generated 1213 bytes in 108 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:20:02.069376 ncdv-master-1 devstack@n-api.service[138680]: DEBUG neutronclient.v2_0.client [None req-63ec2c01-3099-47c0-a6c3-b08586f18725 tempest-SecurityGroupsNegativeTestJSON-448634586 tempest-SecurityGroupsNegativeTestJSON-448634586-project] Error message: {"NeutronError": {"type": "SecurityGroupNotFound", "message": "Security group 107eede4-1ac5-4c55-b36a-a794adb0b14f does not exist", "detail": ""}} {{(pid=138680) _handle_fault_response /usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py:262}} Jul 16 21:20:02.070210 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.network.security_group_api [None req-63ec2c01-3099-47c0-a6c3-b08586f18725 tempest-SecurityGroupsNegativeTestJSON-448634586 tempest-SecurityGroupsNegativeTestJSON-448634586-project] Neutron security group 107eede4-1ac5-4c55-b36a-a794adb0b14f not found {{(pid=138680) get /opt/stack/nova/nova/network/security_group_api.py:313}} Jul 16 21:20:02.070897 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-63ec2c01-3099-47c0-a6c3-b08586f18725 tempest-SecurityGroupsNegativeTestJSON-448634586 tempest-SecurityGroupsNegativeTestJSON-448634586-project] HTTP exception thrown: Security group 107eede4-1ac5-4c55-b36a-a794adb0b14f does not exist Jul 16 21:20:02.070897 ncdv-master-1 devstack@n-api.service[138680]: Neutron server returns request_ids: ['req-8468b322-e8bc-411e-957b-590a612c5750'] Jul 16 21:20:02.073275 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-63ec2c01-3099-47c0-a6c3-b08586f18725 tempest-SecurityGroupsNegativeTestJSON-448634586 tempest-SecurityGroupsNegativeTestJSON-448634586-project] Request method failure captured: Jul 16 21:20:02.073275 ncdv-master-1 devstack@n-api.service[138680]: request: PUT /compute/v2.1/os-security-groups/107eede4-1ac5-4c55-b36a-a794adb0b14f HTTP/1.1 Jul 16 21:20:02.073275 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:20:02.073275 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:20:02.073275 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:20:02.073275 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 101 Jul 16 21:20:02.073275 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:20:02.073275 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:20:02.073275 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:20:02.073275 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:20:02.073275 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0yt_ymXnmnlBefd4L74fkZK-MBunE4X9xmzQTfcTj2uXA0DwtttILe9S_ZZU4TyN65WBepNA2fANs8uBj7BhBHLuerjdECnqpUBAhcOwa__91XCZTXcvMcKNelBeydlTkeHX_Ec6v9VCcW32w5T8ou4e1K5WWpMzMDvJjkmnfvwuuJo Jul 16 21:20:02.073275 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:20:02.073275 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:20:02.073275 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:20:02.073275 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:20:02.073275 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:20:02.073275 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:20:02.073275 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: 6f40e610a1bf49a2a6913c4c6c6735b6 Jul 16 21:20:02.073275 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-SecurityGroupsNegativeTestJSON-448634586 Jul 16 21:20:02.073275 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:20:02.073275 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:20:02.073275 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/6f40e610a1bf49a2a6913c4c6c6735b6"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/6f40e610a1bf49a2a6913c4c6c6735b6"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_6f40e610a1bf49a2a6913c4c6c6735b6"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/6f40e610a1bf49a2a6913c4c6c6735b6"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:20:02.073275 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-SecurityGroupsNegativeTestJSON-448634586 Jul 16 21:20:02.073275 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: 6f40e610a1bf49a2a6913c4c6c6735b6 Jul 16 21:20:02.073275 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-SecurityGroupsNegativeTestJSON-448634586 Jul 16 21:20:02.073275 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-SecurityGroupsNegativeTestJSON-448634586-project Jul 16 21:20:02.073275 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:20:02.073275 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:20:02.073275 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: 27a08402c73946178157b98e3e4f1e82 Jul 16 21:20:02.073275 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-SecurityGroupsNegativeTestJSON-448634586-project Jul 16 21:20:02.073275 ncdv-master-1 devstack@n-api.service[138680]: Jul 16 21:20:02.073275 ncdv-master-1 devstack@n-api.service[138680]: {"security_group": {"name": "tempest-sg-134363798", "description": "tempest-description-1972526295"}} Jul 16 21:20:02.073275 ncdv-master-1 devstack@n-api.service[138680]: method: Jul 16 21:20:02.073275 ncdv-master-1 devstack@n-api.service[138680]: exception: Security group 107eede4-1ac5-4c55-b36a-a794adb0b14f does not exist Jul 16 21:20:02.073275 ncdv-master-1 devstack@n-api.service[138680]: Neutron server returns request_ids: ['req-8468b322-e8bc-411e-957b-590a612c5750'] Jul 16 21:20:02.073275 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': '107eede4-1ac5-4c55-b36a-a794adb0b14f', 'body': {'security_group': {'name': 'tempest-sg-134363798', 'description': 'tempest-description-1972526295'}}} Jul 16 21:20:02.073275 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:20:02.073275 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:20:02.073275 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/security_group_api.py", line 309, in get Jul 16 21:20:02.073275 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi group = neutron.show_security_group(id).get('security_group') Jul 16 21:20:02.073275 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 197, in wrapper Jul 16 21:20:02.073275 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Jul 16 21:20:02.073275 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 1040, in show_security_group Jul 16 21:20:02.073275 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return self.get(self.security_group_path % (security_group), Jul 16 21:20:02.073275 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 197, in wrapper Jul 16 21:20:02.073275 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Jul 16 21:20:02.073275 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 356, in get Jul 16 21:20:02.073275 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return self.retry_request("GET", action, body=body, Jul 16 21:20:02.073275 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 197, in wrapper Jul 16 21:20:02.073275 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Jul 16 21:20:02.073275 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 333, in retry_request Jul 16 21:20:02.073275 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return self.do_request(method, action, body=body, Jul 16 21:20:02.073275 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 197, in wrapper Jul 16 21:20:02.073275 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Jul 16 21:20:02.073275 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 297, in do_request Jul 16 21:20:02.073275 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi self._handle_fault_response(status_code, replybody, resp) Jul 16 21:20:02.073275 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 197, in wrapper Jul 16 21:20:02.073275 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Jul 16 21:20:02.073275 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 272, in _handle_fault_response Jul 16 21:20:02.073275 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi exception_handler_v20(status_code, error_body) Jul 16 21:20:02.073275 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 90, in exception_handler_v20 Jul 16 21:20:02.077486 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise client_exc(message=error_message, Jul 16 21:20:02.077486 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi neutronclient.common.exceptions.NotFound: Security group 107eede4-1ac5-4c55-b36a-a794adb0b14f does not exist Jul 16 21:20:02.077486 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-8468b322-e8bc-411e-957b-590a612c5750'] Jul 16 21:20:02.077486 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:20:02.077486 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:20:02.077486 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:20:02.077486 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:20:02.077486 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/security_groups.py", line 246, in update Jul 16 21:20:02.077486 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi security_group = security_group_api.get(context, id) Jul 16 21:20:02.077486 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/security_group_api.py", line 314, in get Jul 16 21:20:02.077486 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.SecurityGroupNotFound(str(e)) Jul 16 21:20:02.077486 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.SecurityGroupNotFound: Security group 107eede4-1ac5-4c55-b36a-a794adb0b14f does not exist Jul 16 21:20:02.077486 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-8468b322-e8bc-411e-957b-590a612c5750'] Jul 16 21:20:02.077486 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:20:02.077486 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:20:02.077486 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:20:02.077486 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:20:02.077486 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:20:02.077486 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:20:02.077486 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:20:02.077486 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:20:02.077486 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Jul 16 21:20:02.077486 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Jul 16 21:20:02.077486 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:20:02.077486 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:20:02.077486 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/security_groups.py", line 248, in update Jul 16 21:20:02.077486 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=exp.format_message()) Jul 16 21:20:02.077486 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Security group 107eede4-1ac5-4c55-b36a-a794adb0b14f does not exist Jul 16 21:20:02.077486 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-8468b322-e8bc-411e-957b-590a612c5750'] Jul 16 21:20:02.077486 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:20:02.077486 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:20:02.077486 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:20:02.077486 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:20:02.077486 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:20:02.077486 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:20:02.077486 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:20:02.077486 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:20:02.077486 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Security group 107eede4-1ac5-4c55-b36a-a794adb0b14f does not exist Jul 16 21:20:02.077486 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-8468b322-e8bc-411e-957b-590a612c5750'] Jul 16 21:20:02.077486 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:20:02.077486 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-63ec2c01-3099-47c0-a6c3-b08586f18725 tempest-SecurityGroupsNegativeTestJSON-448634586 tempest-SecurityGroupsNegativeTestJSON-448634586-project] Returning 404 to user: Security group 107eede4-1ac5-4c55-b36a-a794adb0b14f does not exist Jul 16 21:20:02.077486 ncdv-master-1 devstack@n-api.service[138680]: Neutron server returns request_ids: ['req-8468b322-e8bc-411e-957b-590a612c5750'] {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:20:02.081191 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-63ec2c01-3099-47c0-a6c3-b08586f18725 tempest-SecurityGroupsNegativeTestJSON-448634586 tempest-SecurityGroupsNegativeTestJSON-448634586-project] 10.222.0.16 "PUT /compute/v2.1/os-security-groups/107eede4-1ac5-4c55-b36a-a794adb0b14f" status: 404 len: 194 microversion: 2.1 time: 0.145149 Jul 16 21:20:02.081191 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1413/2775] 10.222.0.16 () {60 vars in 1289 bytes} [Sat Jul 16 21:20:01 2022] PUT /compute/v2.1/os-security-groups/107eede4-1ac5-4c55-b36a-a794adb0b14f => generated 194 bytes in 148 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:20:02.370131 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-5bef7fbf-b008-4a2a-91b4-bfe8c038c46c tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:20:02.372004 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-5bef7fbf-b008-4a2a-91b4-bfe8c038c46c tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] [instance: aa688a7e-0b28-4dfd-a159-8e19fd62ee02] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:20:02.381397 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5bef7fbf-b008-4a2a-91b4-bfe8c038c46c tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:02.382550 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5bef7fbf-b008-4a2a-91b4-bfe8c038c46c tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:02.384930 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5bef7fbf-b008-4a2a-91b4-bfe8c038c46c tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:02.400608 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5bef7fbf-b008-4a2a-91b4-bfe8c038c46c tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:02.429279 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-5bef7fbf-b008-4a2a-91b4-bfe8c038c46c tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] HTTP exception thrown: Instance aa688a7e-0b28-4dfd-a159-8e19fd62ee02 could not be found. Jul 16 21:20:02.431781 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-5bef7fbf-b008-4a2a-91b4-bfe8c038c46c tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Request method failure captured: Jul 16 21:20:02.431781 ncdv-master-1 devstack@n-api.service[138681]: request: GET /compute/v2.1/servers/aa688a7e-0b28-4dfd-a159-8e19fd62ee02 HTTP/1.1 Jul 16 21:20:02.431781 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:20:02.431781 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:20:02.431781 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:20:02.431781 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 21:20:02.431781 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:20:02.431781 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:20:02.431781 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:20:02.431781 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:20:02.431781 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0yrNpNpQByhQ1578GareI_bLbzFzwt1X7DjqAJogmQKBk7pzKioqjlhICyrqQjyNEkmOeTkxKX8xROhDwwvvteGCjPW-Knwb8qWdwKE0LlYrz9U5NzOQ8ukFz_cfNcdWpVOqqWw9XjDKSvggxCIEhYLmedQYWwv9L2-iYQUT-tM1e3g Jul 16 21:20:02.431781 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:20:02.431781 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:20:02.431781 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:20:02.431781 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:20:02.431781 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:20:02.431781 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:20:02.431781 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: 65d5bcfa02af4e8284ba1342197e2860 Jul 16 21:20:02.431781 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-ServersAdminTestJSON-1438684720 Jul 16 21:20:02.431781 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:20:02.431781 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:20:02.431781 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/65d5bcfa02af4e8284ba1342197e2860"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/65d5bcfa02af4e8284ba1342197e2860"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_65d5bcfa02af4e8284ba1342197e2860"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/65d5bcfa02af4e8284ba1342197e2860"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:20:02.431781 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-ServersAdminTestJSON-1438684720 Jul 16 21:20:02.431781 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: 65d5bcfa02af4e8284ba1342197e2860 Jul 16 21:20:02.431781 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-ServersAdminTestJSON-1438684720 Jul 16 21:20:02.431781 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-ServersAdminTestJSON-1438684720-project Jul 16 21:20:02.431781 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:20:02.431781 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:20:02.431781 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: 2f5bc1f2209f4eb186d80e385b67c774 Jul 16 21:20:02.431781 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-ServersAdminTestJSON-1438684720-project Jul 16 21:20:02.431781 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:20:02.431781 ncdv-master-1 devstack@n-api.service[138681]: exception: Instance aa688a7e-0b28-4dfd-a159-8e19fd62ee02 could not be found. Jul 16 21:20:02.431781 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': 'aa688a7e-0b28-4dfd-a159-8e19fd62ee02'} Jul 16 21:20:02.431781 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:20:02.431781 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:20:02.431781 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:20:02.431781 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:20:02.431781 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:20:02.431781 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:20:02.431781 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:20:02.431781 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:20:02.431781 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:20:02.431781 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:20:02.431781 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance aa688a7e-0b28-4dfd-a159-8e19fd62ee02 could not be found. Jul 16 21:20:02.431781 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:20:02.431781 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:20:02.431781 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:20:02.431781 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:20:02.431781 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:20:02.431781 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:20:02.431781 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:20:02.431781 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:20:02.431781 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:20:02.431781 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:20:02.431781 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:20:02.431781 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:20:02.431781 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:20:02.431781 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:20:02.431781 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:20:02.431781 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:20:02.431781 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance aa688a7e-0b28-4dfd-a159-8e19fd62ee02 could not be found. Jul 16 21:20:02.431781 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:20:02.431781 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:20:02.431781 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:20:02.438053 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:20:02.438053 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:20:02.438053 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:20:02.438053 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:20:02.438053 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:20:02.438053 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance aa688a7e-0b28-4dfd-a159-8e19fd62ee02 could not be found. Jul 16 21:20:02.438053 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:20:02.440557 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-5bef7fbf-b008-4a2a-91b4-bfe8c038c46c tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Returning 404 to user: Instance aa688a7e-0b28-4dfd-a159-8e19fd62ee02 could not be found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:20:02.441681 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-5bef7fbf-b008-4a2a-91b4-bfe8c038c46c tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] 10.222.0.16 "GET /compute/v2.1/servers/aa688a7e-0b28-4dfd-a159-8e19fd62ee02" status: 404 len: 111 microversion: 2.1 time: 0.080383 Jul 16 21:20:02.442513 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1363/2776] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:20:02 2022] GET /compute/v2.1/servers/aa688a7e-0b28-4dfd-a159-8e19fd62ee02 => generated 111 bytes in 82 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:20:02.454680 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-37a27cd1-7489-4ad0-b755-243baa77c50b tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:20:02.456083 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-37a27cd1-7489-4ad0-b755-243baa77c50b tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] [instance: e7598b4a-f70c-4186-88d1-6fd0057f8656] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:20:02.470726 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-37a27cd1-7489-4ad0-b755-243baa77c50b tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:02.475603 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-37a27cd1-7489-4ad0-b755-243baa77c50b tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:02.477596 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-37a27cd1-7489-4ad0-b755-243baa77c50b tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:02.477596 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-37a27cd1-7489-4ad0-b755-243baa77c50b tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:02.523929 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-37a27cd1-7489-4ad0-b755-243baa77c50b tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] [instance: e7598b4a-f70c-4186-88d1-6fd0057f8656] Going to try to terminate instance {{(pid=138680) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 16 21:20:02.654769 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-2617c899-4073-465c-a9e1-4704a63ec44f tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:20:02.655683 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-2617c899-4073-465c-a9e1-4704a63ec44f tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] [instance: a56f8463-6006-4805-9021-1babc2d98f0f] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:20:02.672807 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2617c899-4073-465c-a9e1-4704a63ec44f tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:02.674680 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2617c899-4073-465c-a9e1-4704a63ec44f tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:02.677898 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2617c899-4073-465c-a9e1-4704a63ec44f tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:02.684005 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2617c899-4073-465c-a9e1-4704a63ec44f tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:03.012035 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-37a27cd1-7489-4ad0-b755-243baa77c50b tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] 10.222.0.16 "DELETE /compute/v2.1/servers/e7598b4a-f70c-4186-88d1-6fd0057f8656" status: 204 len: 0 microversion: 2.1 time: 0.563863 Jul 16 21:20:03.013778 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1414/2777] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:20:02 2022] DELETE /compute/v2.1/servers/e7598b4a-f70c-4186-88d1-6fd0057f8656 => generated 0 bytes in 567 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 16 21:20:03.021843 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-3802edeb-f2f9-479a-a2dc-477d6f6f6c02 tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:20:03.024063 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-3802edeb-f2f9-479a-a2dc-477d6f6f6c02 tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] [instance: 0a9564cb-c375-47e9-bfe6-e68d4700442b] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:20:03.046569 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3802edeb-f2f9-479a-a2dc-477d6f6f6c02 tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:03.048973 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3802edeb-f2f9-479a-a2dc-477d6f6f6c02 tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:03.049921 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3802edeb-f2f9-479a-a2dc-477d6f6f6c02 tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:03.051129 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3802edeb-f2f9-479a-a2dc-477d6f6f6c02 tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:03.236735 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-3802edeb-f2f9-479a-a2dc-477d6f6f6c02 tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1f72b1e0b4124480bfd53b5c22a343e0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0be222897b3a4864a39e9fc7d481340a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:20:03.249980 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-3802edeb-f2f9-479a-a2dc-477d6f6f6c02 tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] 10.222.0.16 "GET /compute/v2.1/servers/0a9564cb-c375-47e9-bfe6-e68d4700442b" status: 200 len: 1335 microversion: 2.1 time: 0.234249 Jul 16 21:20:03.260317 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1415/2778] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:20:03 2022] GET /compute/v2.1/servers/0a9564cb-c375-47e9-bfe6-e68d4700442b => generated 1335 bytes in 245 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:20:03.265374 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-22136fd0-a73f-4a4e-ab80-4053dcbd0632 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:20:03.274782 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-22136fd0-a73f-4a4e-ab80-4053dcbd0632 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] [instance: e7598b4a-f70c-4186-88d1-6fd0057f8656] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:20:03.298642 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-22136fd0-a73f-4a4e-ab80-4053dcbd0632 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:03.299316 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-22136fd0-a73f-4a4e-ab80-4053dcbd0632 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:03.301866 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-22136fd0-a73f-4a4e-ab80-4053dcbd0632 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:03.305338 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-22136fd0-a73f-4a4e-ab80-4053dcbd0632 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:03.341090 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-2617c899-4073-465c-a9e1-4704a63ec44f tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eebb224a56814f92b12ebebebc6bc8a3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a092ce9ef40943f6b6ace6cd28481b71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:20:03.353856 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-2617c899-4073-465c-a9e1-4704a63ec44f tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] 10.222.0.16 "GET /compute/v2.1/servers/a56f8463-6006-4805-9021-1babc2d98f0f" status: 200 len: 1318 microversion: 2.1 time: 0.705894 Jul 16 21:20:03.361303 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1364/2779] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:20:02 2022] GET /compute/v2.1/servers/a56f8463-6006-4805-9021-1babc2d98f0f => generated 1318 bytes in 714 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:20:03.376984 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-67477672-2f07-4785-b9d6-97e44e187192 tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:20:03.378293 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-67477672-2f07-4785-b9d6-97e44e187192 tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] [instance: 84aa4a51-e376-4829-9a38-2ea447ebb413] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:20:03.781628 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-67477672-2f07-4785-b9d6-97e44e187192 tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fe680ccdc60e410b812794de8aa73e0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '772bd01188344e288824730238fee49b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:20:03.807655 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-67477672-2f07-4785-b9d6-97e44e187192 tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] 10.222.0.16 "GET /compute/v2.1/servers/84aa4a51-e376-4829-9a38-2ea447ebb413" status: 200 len: 1213 microversion: 2.1 time: 0.433463 Jul 16 21:20:03.808912 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1365/2780] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:20:03 2022] GET /compute/v2.1/servers/84aa4a51-e376-4829-9a38-2ea447ebb413 => generated 1213 bytes in 447 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:20:03.814253 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-f8249777-fdbf-44ec-9af0-92211e08c6f8 tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] Action: 'create', calling method: >, body: {"server": {"name": "guest-instance-1.domain.com", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452", "adminPass": "***", "accessIPv4": "2.2.2.2", "networks": [{"uuid": "ded9b24a-d333-4b92-976d-dfec1646dea5"}], "security_groups": [{"name": "tempest-securitygroup--1047759228"}], "key_name": "tempest-keypair-1918580201", "user_data": "}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:20:03.898383 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-22136fd0-a73f-4a4e-ab80-4053dcbd0632 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f5bc1f2209f4eb186d80e385b67c774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '65d5bcfa02af4e8284ba1342197e2860', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:20:03.922255 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-22136fd0-a73f-4a4e-ab80-4053dcbd0632 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] 10.222.0.16 "GET /compute/v2.1/servers/e7598b4a-f70c-4186-88d1-6fd0057f8656" status: 200 len: 1469 microversion: 2.1 time: 0.660874 Jul 16 21:20:03.923331 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1416/2781] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:20:03 2022] GET /compute/v2.1/servers/e7598b4a-f70c-4186-88d1-6fd0057f8656 => generated 1469 bytes in 662 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:20:04.080074 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.network.neutron [None req-f8249777-fdbf-44ec-9af0-92211e08c6f8 tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] validate_networks() for [('ded9b24a-d333-4b92-976d-dfec1646dea5', None, None, None, None, None)] {{(pid=138681) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 16 21:20:04.267511 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-0a41037e-7b26-4f41-b354-d08315b0c7c3 tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:20:04.268144 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-0a41037e-7b26-4f41-b354-d08315b0c7c3 tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] [instance: 0a9564cb-c375-47e9-bfe6-e68d4700442b] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:20:04.287352 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0a41037e-7b26-4f41-b354-d08315b0c7c3 tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:04.288296 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0a41037e-7b26-4f41-b354-d08315b0c7c3 tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:04.291233 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0a41037e-7b26-4f41-b354-d08315b0c7c3 tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:04.293908 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0a41037e-7b26-4f41-b354-d08315b0c7c3 tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:05.518509 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-0a41037e-7b26-4f41-b354-d08315b0c7c3 tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1f72b1e0b4124480bfd53b5c22a343e0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0be222897b3a4864a39e9fc7d481340a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:20:05.528311 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-0a41037e-7b26-4f41-b354-d08315b0c7c3 tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] 10.222.0.16 "GET /compute/v2.1/servers/0a9564cb-c375-47e9-bfe6-e68d4700442b" status: 200 len: 1402 microversion: 2.1 time: 1.264794 Jul 16 21:20:05.529508 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1417/2782] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:20:04 2022] GET /compute/v2.1/servers/0a9564cb-c375-47e9-bfe6-e68d4700442b => generated 1402 bytes in 1267 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:20:05.535650 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-b73a18e3-f3d8-4a20-ad8d-7a8bc5d2db81 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:20:05.536267 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-b73a18e3-f3d8-4a20-ad8d-7a8bc5d2db81 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] [instance: a56f8463-6006-4805-9021-1babc2d98f0f] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:20:05.543675 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b73a18e3-f3d8-4a20-ad8d-7a8bc5d2db81 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:05.544622 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b73a18e3-f3d8-4a20-ad8d-7a8bc5d2db81 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:05.545577 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b73a18e3-f3d8-4a20-ad8d-7a8bc5d2db81 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:05.546210 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b73a18e3-f3d8-4a20-ad8d-7a8bc5d2db81 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:05.898046 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-f8249777-fdbf-44ec-9af0-92211e08c6f8 tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] Flavor limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 16 21:20:05.900324 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-f8249777-fdbf-44ec-9af0-92211e08c6f8 tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] Image limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 16 21:20:05.900982 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-f8249777-fdbf-44ec-9af0-92211e08c6f8 tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] Flavor pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 16 21:20:05.901622 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-f8249777-fdbf-44ec-9af0-92211e08c6f8 tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] Image pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 16 21:20:05.902242 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-f8249777-fdbf-44ec-9af0-92211e08c6f8 tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 16 21:20:05.917105 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f8249777-fdbf-44ec-9af0-92211e08c6f8 tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:05.922296 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f8249777-fdbf-44ec-9af0-92211e08c6f8 tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:05.923946 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f8249777-fdbf-44ec-9af0-92211e08c6f8 tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:05.924567 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f8249777-fdbf-44ec-9af0-92211e08c6f8 tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:05.997441 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-f8249777-fdbf-44ec-9af0-92211e08c6f8 tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] Getting quotas for project 837a8f9e06e141dcbd5f7bd895016d49. Resources: {'ram', 'instances', 'cores'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:20:06.009460 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-f8249777-fdbf-44ec-9af0-92211e08c6f8 tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] Getting quotas for user c6ba562873fa4f4fbd0b635f1c9dcdf2 and project 837a8f9e06e141dcbd5f7bd895016d49. Resources: {'ram', 'instances', 'cores'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:20:06.023433 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-b73a18e3-f3d8-4a20-ad8d-7a8bc5d2db81 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eebb224a56814f92b12ebebebc6bc8a3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a092ce9ef40943f6b6ace6cd28481b71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:20:06.033147 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-f8249777-fdbf-44ec-9af0-92211e08c6f8 tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] Going to run 1 instances... {{(pid=138681) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 16 21:20:06.036215 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-f8249777-fdbf-44ec-9af0-92211e08c6f8 tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] [instance: 6cba399c-58c3-4850-8065-0fbc3301fb2c] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138681) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 16 21:20:06.039338 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-b73a18e3-f3d8-4a20-ad8d-7a8bc5d2db81 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] 10.222.0.16 "GET /compute/v2.1/servers/a56f8463-6006-4805-9021-1babc2d98f0f" status: 200 len: 1506 microversion: 2.1 time: 0.508403 Jul 16 21:20:06.040419 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1418/2783] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:20:05 2022] GET /compute/v2.1/servers/a56f8463-6006-4805-9021-1babc2d98f0f => generated 1506 bytes in 510 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:20:06.048840 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-4f4e486b-386b-4bd5-8980-cc0e0755c247 tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:20:06.052274 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-4f4e486b-386b-4bd5-8980-cc0e0755c247 tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] [instance: 84aa4a51-e376-4829-9a38-2ea447ebb413] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:20:06.065237 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4f4e486b-386b-4bd5-8980-cc0e0755c247 tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:06.065833 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4f4e486b-386b-4bd5-8980-cc0e0755c247 tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:06.066726 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4f4e486b-386b-4bd5-8980-cc0e0755c247 tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:06.067591 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4f4e486b-386b-4bd5-8980-cc0e0755c247 tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:06.192190 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-4f4e486b-386b-4bd5-8980-cc0e0755c247 tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fe680ccdc60e410b812794de8aa73e0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '772bd01188344e288824730238fee49b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:20:06.217620 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-4f4e486b-386b-4bd5-8980-cc0e0755c247 tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] 10.222.0.16 "GET /compute/v2.1/servers/84aa4a51-e376-4829-9a38-2ea447ebb413" status: 200 len: 1265 microversion: 2.1 time: 0.175091 Jul 16 21:20:06.219733 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1419/2784] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:20:06 2022] GET /compute/v2.1/servers/84aa4a51-e376-4829-9a38-2ea447ebb413 => generated 1265 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:20:06.244374 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-f8249777-fdbf-44ec-9af0-92211e08c6f8 tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 409 microversion: 2.1 time: 2.434333 Jul 16 21:20:06.245046 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1366/2785] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 21:20:03 2022] POST /compute/v2.1/servers => generated 409 bytes in 2436 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 16 21:20:06.249413 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-6a809026-6846-4497-a1b2-fef52f81200a tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:20:06.256958 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-6a809026-6846-4497-a1b2-fef52f81200a tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] [instance: e7598b4a-f70c-4186-88d1-6fd0057f8656] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:20:06.266022 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-da52472a-f841-472f-9b3e-df1c007ca6d1 req-469fcfd7-5833-4f35-ae93-122a2354a29c service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "0a9564cb-c375-47e9-bfe6-e68d4700442b", "tag": "22eac33c-df13-4324-a3f6-14093cd60ead"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:20:06.287892 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-da52472a-f841-472f-9b3e-df1c007ca6d1 req-469fcfd7-5833-4f35-ae93-122a2354a29c service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:06.288557 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-da52472a-f841-472f-9b3e-df1c007ca6d1 req-469fcfd7-5833-4f35-ae93-122a2354a29c service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:06.293520 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6a809026-6846-4497-a1b2-fef52f81200a tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:06.293520 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6a809026-6846-4497-a1b2-fef52f81200a tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:06.293520 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6a809026-6846-4497-a1b2-fef52f81200a tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:06.293520 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6a809026-6846-4497-a1b2-fef52f81200a tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:06.304645 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-da52472a-f841-472f-9b3e-df1c007ca6d1 req-469fcfd7-5833-4f35-ae93-122a2354a29c service nova] Creating event network-changed:22eac33c-df13-4324-a3f6-14093cd60ead for instance 0a9564cb-c375-47e9-bfe6-e68d4700442b on nch2-master-1 Jul 16 21:20:06.311116 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-da52472a-f841-472f-9b3e-df1c007ca6d1 req-469fcfd7-5833-4f35-ae93-122a2354a29c service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.057052 Jul 16 21:20:06.313469 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1367/2786] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:20:06 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 60 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:20:06.322850 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-2e10b9de-c1c5-43fd-aa5e-e5e356358fbf tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:20:06.323855 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-2e10b9de-c1c5-43fd-aa5e-e5e356358fbf tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] [instance: 9d2cd0fc-1101-4e21-8179-c1dd4605594f] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:20:06.332316 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2e10b9de-c1c5-43fd-aa5e-e5e356358fbf tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:06.333832 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2e10b9de-c1c5-43fd-aa5e-e5e356358fbf tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:06.335305 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2e10b9de-c1c5-43fd-aa5e-e5e356358fbf tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:06.336525 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2e10b9de-c1c5-43fd-aa5e-e5e356358fbf tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:06.956363 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-2e10b9de-c1c5-43fd-aa5e-e5e356358fbf tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eebb224a56814f92b12ebebebc6bc8a3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a092ce9ef40943f6b6ace6cd28481b71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:20:06.967991 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-2e10b9de-c1c5-43fd-aa5e-e5e356358fbf tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] 10.222.0.16 "GET /compute/v2.1/servers/9d2cd0fc-1101-4e21-8179-c1dd4605594f" status: 200 len: 1507 microversion: 2.1 time: 0.652697 Jul 16 21:20:06.968637 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1368/2787] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:20:06 2022] GET /compute/v2.1/servers/9d2cd0fc-1101-4e21-8179-c1dd4605594f => generated 1507 bytes in 654 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:20:06.975758 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-b586bf2e-65ba-4110-97a2-84fa6d71429c tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:20:06.976382 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-b586bf2e-65ba-4110-97a2-84fa6d71429c tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] [instance: 6cba399c-58c3-4850-8065-0fbc3301fb2c] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:20:07.158079 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-b586bf2e-65ba-4110-97a2-84fa6d71429c tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c6ba562873fa4f4fbd0b635f1c9dcdf2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '837a8f9e06e141dcbd5f7bd895016d49', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:20:07.167262 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-b586bf2e-65ba-4110-97a2-84fa6d71429c tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] 10.222.0.16 "GET /compute/v2.1/servers/6cba399c-58c3-4850-8065-0fbc3301fb2c" status: 200 len: 1226 microversion: 2.1 time: 0.197224 Jul 16 21:20:07.178475 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1369/2788] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:20:06 2022] GET /compute/v2.1/servers/6cba399c-58c3-4850-8065-0fbc3301fb2c => generated 1226 bytes in 199 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:20:07.178475 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-34e57ee6-003a-400b-b58b-4d1b68877a8a tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:20:07.178475 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-34e57ee6-003a-400b-b58b-4d1b68877a8a tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] [instance: 0a9564cb-c375-47e9-bfe6-e68d4700442b] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:20:07.192268 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-6a809026-6846-4497-a1b2-fef52f81200a tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f5bc1f2209f4eb186d80e385b67c774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '65d5bcfa02af4e8284ba1342197e2860', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:20:07.198299 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-34e57ee6-003a-400b-b58b-4d1b68877a8a tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:07.198994 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-34e57ee6-003a-400b-b58b-4d1b68877a8a tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:07.199840 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-34e57ee6-003a-400b-b58b-4d1b68877a8a tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:07.200279 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-34e57ee6-003a-400b-b58b-4d1b68877a8a tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:07.200650 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-6a809026-6846-4497-a1b2-fef52f81200a tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] 10.222.0.16 "GET /compute/v2.1/servers/e7598b4a-f70c-4186-88d1-6fd0057f8656" status: 200 len: 1469 microversion: 2.1 time: 0.979669 Jul 16 21:20:07.201252 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1420/2789] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:20:06 2022] GET /compute/v2.1/servers/e7598b4a-f70c-4186-88d1-6fd0057f8656 => generated 1469 bytes in 981 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:20:07.216527 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-d94f1198-c575-429b-9395-ce5f6e9f938d tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:20:07.217294 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-d94f1198-c575-429b-9395-ce5f6e9f938d tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] [instance: 9d2cd0fc-1101-4e21-8179-c1dd4605594f] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:20:07.237784 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d94f1198-c575-429b-9395-ce5f6e9f938d tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:07.240347 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d94f1198-c575-429b-9395-ce5f6e9f938d tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:07.240347 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d94f1198-c575-429b-9395-ce5f6e9f938d tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:07.240823 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d94f1198-c575-429b-9395-ce5f6e9f938d tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:07.311542 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-d94f1198-c575-429b-9395-ce5f6e9f938d tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] [instance: 9d2cd0fc-1101-4e21-8179-c1dd4605594f] Going to try to terminate instance {{(pid=138680) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 16 21:20:07.694658 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-d94f1198-c575-429b-9395-ce5f6e9f938d tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] 10.222.0.16 "DELETE /compute/v2.1/servers/9d2cd0fc-1101-4e21-8179-c1dd4605594f" status: 204 len: 0 microversion: 2.1 time: 0.486854 Jul 16 21:20:07.698044 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1421/2790] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:20:07 2022] DELETE /compute/v2.1/servers/9d2cd0fc-1101-4e21-8179-c1dd4605594f => generated 0 bytes in 489 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 16 21:20:07.704879 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-9c17df3d-494e-4852-99f4-97471c42f842 tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:20:07.705954 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-9c17df3d-494e-4852-99f4-97471c42f842 tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] [instance: 84aa4a51-e376-4829-9a38-2ea447ebb413] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:20:07.713294 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9c17df3d-494e-4852-99f4-97471c42f842 tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:07.714294 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9c17df3d-494e-4852-99f4-97471c42f842 tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:07.715670 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9c17df3d-494e-4852-99f4-97471c42f842 tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:07.716547 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9c17df3d-494e-4852-99f4-97471c42f842 tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:07.723476 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=96,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:20:07.727412 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-34e57ee6-003a-400b-b58b-4d1b68877a8a tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1f72b1e0b4124480bfd53b5c22a343e0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0be222897b3a4864a39e9fc7d481340a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:20:07.736304 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-34e57ee6-003a-400b-b58b-4d1b68877a8a tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] 10.222.0.16 "GET /compute/v2.1/servers/0a9564cb-c375-47e9-bfe6-e68d4700442b" status: 200 len: 1402 microversion: 2.1 time: 0.567041 Jul 16 21:20:07.736904 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1370/2791] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:20:07 2022] GET /compute/v2.1/servers/0a9564cb-c375-47e9-bfe6-e68d4700442b => generated 1402 bytes in 568 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:20:07.751600 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-7d0b0f42-cdec-42a8-ab05-b3a8804b4b29 req-c60f73c4-561b-4fe1-821b-c6a6786a6a10 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "e7598b4a-f70c-4186-88d1-6fd0057f8656", "name": "network-vif-deleted", "tag": "5e72e822-7cdd-445c-8d25-dc81877dc718"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:20:07.760935 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-7d0b0f42-cdec-42a8-ab05-b3a8804b4b29 req-c60f73c4-561b-4fe1-821b-c6a6786a6a10 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:07.761640 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-7d0b0f42-cdec-42a8-ab05-b3a8804b4b29 req-c60f73c4-561b-4fe1-821b-c6a6786a6a10 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:07.775913 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-7d0b0f42-cdec-42a8-ab05-b3a8804b4b29 req-c60f73c4-561b-4fe1-821b-c6a6786a6a10 service nova] Creating event network-vif-deleted:5e72e822-7cdd-445c-8d25-dc81877dc718 for instance e7598b4a-f70c-4186-88d1-6fd0057f8656 on nch1-master-1 Jul 16 21:20:07.780970 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-7d0b0f42-cdec-42a8-ab05-b3a8804b4b29 req-c60f73c4-561b-4fe1-821b-c6a6786a6a10 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.039229 Jul 16 21:20:07.781651 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1371/2792] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:20:07 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:20:07.787599 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-b8a2fd62-2893-446f-8037-189eb75297cb tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:20:07.788520 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-b8a2fd62-2893-446f-8037-189eb75297cb tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] [instance: a56f8463-6006-4805-9021-1babc2d98f0f] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:20:07.792852 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_api:SELECT=90,nova_api:DELETE=3,nova_api:INSERT=7 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:20:07.797007 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b8a2fd62-2893-446f-8037-189eb75297cb tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:07.797615 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b8a2fd62-2893-446f-8037-189eb75297cb tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:07.801521 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b8a2fd62-2893-446f-8037-189eb75297cb tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:07.801521 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b8a2fd62-2893-446f-8037-189eb75297cb tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:07.834167 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-b8a2fd62-2893-446f-8037-189eb75297cb tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] [instance: a56f8463-6006-4805-9021-1babc2d98f0f] Going to try to terminate instance {{(pid=138681) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 16 21:20:07.840622 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-9c17df3d-494e-4852-99f4-97471c42f842 tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fe680ccdc60e410b812794de8aa73e0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '772bd01188344e288824730238fee49b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:20:07.849858 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-9c17df3d-494e-4852-99f4-97471c42f842 tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] 10.222.0.16 "GET /compute/v2.1/servers/84aa4a51-e376-4829-9a38-2ea447ebb413" status: 200 len: 1271 microversion: 2.1 time: 0.148995 Jul 16 21:20:07.851035 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1422/2793] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:20:07 2022] GET /compute/v2.1/servers/84aa4a51-e376-4829-9a38-2ea447ebb413 => generated 1271 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:20:08.172013 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-b8a2fd62-2893-446f-8037-189eb75297cb tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] 10.222.0.16 "DELETE /compute/v2.1/servers/a56f8463-6006-4805-9021-1babc2d98f0f" status: 204 len: 0 microversion: 2.1 time: 0.388741 Jul 16 21:20:08.174048 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1372/2794] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:20:07 2022] DELETE /compute/v2.1/servers/a56f8463-6006-4805-9021-1babc2d98f0f => generated 0 bytes in 390 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 16 21:20:08.184423 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-ec46a05a-4db7-4063-86fc-90cf527f2648 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:20:08.185268 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-ec46a05a-4db7-4063-86fc-90cf527f2648 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] [instance: 9d2cd0fc-1101-4e21-8179-c1dd4605594f] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:20:08.186569 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-66fa4f43-ac09-421a-8014-a2844bb900fa tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:20:08.187654 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-66fa4f43-ac09-421a-8014-a2844bb900fa tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] [instance: 6cba399c-58c3-4850-8065-0fbc3301fb2c] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:20:08.192272 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ec46a05a-4db7-4063-86fc-90cf527f2648 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:08.193200 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ec46a05a-4db7-4063-86fc-90cf527f2648 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:08.194743 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ec46a05a-4db7-4063-86fc-90cf527f2648 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:08.195381 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ec46a05a-4db7-4063-86fc-90cf527f2648 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:08.196675 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-66fa4f43-ac09-421a-8014-a2844bb900fa tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:08.197963 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-66fa4f43-ac09-421a-8014-a2844bb900fa tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:08.205204 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-66fa4f43-ac09-421a-8014-a2844bb900fa tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:08.205204 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-66fa4f43-ac09-421a-8014-a2844bb900fa tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:08.568236 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-66fa4f43-ac09-421a-8014-a2844bb900fa tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c6ba562873fa4f4fbd0b635f1c9dcdf2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '837a8f9e06e141dcbd5f7bd895016d49', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:20:08.580389 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-66fa4f43-ac09-421a-8014-a2844bb900fa tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] 10.222.0.16 "GET /compute/v2.1/servers/6cba399c-58c3-4850-8065-0fbc3301fb2c" status: 200 len: 1230 microversion: 2.1 time: 0.398490 Jul 16 21:20:08.581699 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1373/2795] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:20:08 2022] GET /compute/v2.1/servers/6cba399c-58c3-4850-8065-0fbc3301fb2c => generated 1230 bytes in 400 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:20:08.589185 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-2360ff97-fdc9-4b0e-9828-327dbbf53c31 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:20:08.589185 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-2360ff97-fdc9-4b0e-9828-327dbbf53c31 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] [instance: e7598b4a-f70c-4186-88d1-6fd0057f8656] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:20:08.602458 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2360ff97-fdc9-4b0e-9828-327dbbf53c31 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:08.603298 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2360ff97-fdc9-4b0e-9828-327dbbf53c31 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:08.604244 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2360ff97-fdc9-4b0e-9828-327dbbf53c31 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:08.604865 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2360ff97-fdc9-4b0e-9828-327dbbf53c31 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:08.658629 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2360ff97-fdc9-4b0e-9828-327dbbf53c31 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:08.659484 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2360ff97-fdc9-4b0e-9828-327dbbf53c31 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:08.660209 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.objects.instance [None req-2360ff97-fdc9-4b0e-9828-327dbbf53c31 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lazy-loading 'fault' on Instance uuid e7598b4a-f70c-4186-88d1-6fd0057f8656 {{(pid=138681) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:20:08.669530 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-ec46a05a-4db7-4063-86fc-90cf527f2648 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eebb224a56814f92b12ebebebc6bc8a3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a092ce9ef40943f6b6ace6cd28481b71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:20:08.683173 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-ec46a05a-4db7-4063-86fc-90cf527f2648 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] 10.222.0.16 "GET /compute/v2.1/servers/9d2cd0fc-1101-4e21-8179-c1dd4605594f" status: 200 len: 1513 microversion: 2.1 time: 0.498942 Jul 16 21:20:08.683173 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1423/2796] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:20:08 2022] GET /compute/v2.1/servers/9d2cd0fc-1101-4e21-8179-c1dd4605594f => generated 1513 bytes in 501 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:20:08.717007 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-2360ff97-fdc9-4b0e-9828-327dbbf53c31 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2f5bc1f2209f4eb186d80e385b67c774', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '65d5bcfa02af4e8284ba1342197e2860', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:20:08.733854 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-2360ff97-fdc9-4b0e-9828-327dbbf53c31 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] 10.222.0.16 "GET /compute/v2.1/servers/e7598b4a-f70c-4186-88d1-6fd0057f8656" status: 200 len: 1303 microversion: 2.1 time: 0.150244 Jul 16 21:20:08.740259 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1374/2797] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:20:08 2022] GET /compute/v2.1/servers/e7598b4a-f70c-4186-88d1-6fd0057f8656 => generated 1303 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:20:08.756535 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-9c0da373-1ddb-4f25-a349-06d2b5dfaa71 tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:20:08.757501 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-9c0da373-1ddb-4f25-a349-06d2b5dfaa71 tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] [instance: 0a9564cb-c375-47e9-bfe6-e68d4700442b] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:20:08.764368 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9c0da373-1ddb-4f25-a349-06d2b5dfaa71 tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:08.765024 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9c0da373-1ddb-4f25-a349-06d2b5dfaa71 tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:08.765986 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9c0da373-1ddb-4f25-a349-06d2b5dfaa71 tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:08.766587 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9c0da373-1ddb-4f25-a349-06d2b5dfaa71 tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:08.870418 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-6bdf72bf-20bf-467b-96f7-6dba07ac7cba tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:20:08.871079 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-6bdf72bf-20bf-467b-96f7-6dba07ac7cba tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] [instance: 84aa4a51-e376-4829-9a38-2ea447ebb413] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:20:08.880241 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6bdf72bf-20bf-467b-96f7-6dba07ac7cba tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:08.881642 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6bdf72bf-20bf-467b-96f7-6dba07ac7cba tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:08.882480 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6bdf72bf-20bf-467b-96f7-6dba07ac7cba tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:08.882925 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6bdf72bf-20bf-467b-96f7-6dba07ac7cba tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:09.076460 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell0:SELECT=36 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:20:09.233246 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-9c0da373-1ddb-4f25-a349-06d2b5dfaa71 tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1f72b1e0b4124480bfd53b5c22a343e0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0be222897b3a4864a39e9fc7d481340a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:20:09.240448 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-6bdf72bf-20bf-467b-96f7-6dba07ac7cba tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fe680ccdc60e410b812794de8aa73e0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '772bd01188344e288824730238fee49b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:20:09.247261 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-9c0da373-1ddb-4f25-a349-06d2b5dfaa71 tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] 10.222.0.16 "GET /compute/v2.1/servers/0a9564cb-c375-47e9-bfe6-e68d4700442b" status: 200 len: 1402 microversion: 2.1 time: 0.495535 Jul 16 21:20:09.248685 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1424/2798] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:20:08 2022] GET /compute/v2.1/servers/0a9564cb-c375-47e9-bfe6-e68d4700442b => generated 1402 bytes in 498 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:20:09.268173 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-6bdf72bf-20bf-467b-96f7-6dba07ac7cba tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] 10.222.0.16 "GET /compute/v2.1/servers/84aa4a51-e376-4829-9a38-2ea447ebb413" status: 200 len: 1313 microversion: 2.1 time: 0.404790 Jul 16 21:20:09.272811 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1375/2799] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:20:08 2022] GET /compute/v2.1/servers/84aa4a51-e376-4829-9a38-2ea447ebb413 => generated 1313 bytes in 410 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:20:09.603991 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-67bbae1e-5e56-405d-82bb-b1b7399afb9d tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:20:09.608225 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-67bbae1e-5e56-405d-82bb-b1b7399afb9d tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] [instance: 6cba399c-58c3-4850-8065-0fbc3301fb2c] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:20:09.636367 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-67bbae1e-5e56-405d-82bb-b1b7399afb9d tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:09.644396 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-67bbae1e-5e56-405d-82bb-b1b7399afb9d tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:09.647302 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-67bbae1e-5e56-405d-82bb-b1b7399afb9d tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:09.648428 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-67bbae1e-5e56-405d-82bb-b1b7399afb9d tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:09.697263 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-91e3acdd-e2da-49c8-87cb-b29df6716efd tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:20:09.697899 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-91e3acdd-e2da-49c8-87cb-b29df6716efd tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] [instance: 9d2cd0fc-1101-4e21-8179-c1dd4605594f] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:20:09.711948 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-91e3acdd-e2da-49c8-87cb-b29df6716efd tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:09.712324 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-91e3acdd-e2da-49c8-87cb-b29df6716efd tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:09.713117 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-91e3acdd-e2da-49c8-87cb-b29df6716efd tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:09.713562 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-91e3acdd-e2da-49c8-87cb-b29df6716efd tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:09.809074 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-67bbae1e-5e56-405d-82bb-b1b7399afb9d tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c6ba562873fa4f4fbd0b635f1c9dcdf2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '837a8f9e06e141dcbd5f7bd895016d49', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:20:09.818085 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-67bbae1e-5e56-405d-82bb-b1b7399afb9d tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] 10.222.0.16 "GET /compute/v2.1/servers/6cba399c-58c3-4850-8065-0fbc3301fb2c" status: 200 len: 1222 microversion: 2.1 time: 0.218223 Jul 16 21:20:09.837719 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1425/2800] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:20:09 2022] GET /compute/v2.1/servers/6cba399c-58c3-4850-8065-0fbc3301fb2c => generated 1222 bytes in 236 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:20:09.851683 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-5ee9d43b-24bf-43bc-b997-b42324ae76e9 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:20:09.855348 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-5ee9d43b-24bf-43bc-b997-b42324ae76e9 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] [instance: e7598b4a-f70c-4186-88d1-6fd0057f8656] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:20:09.876604 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5ee9d43b-24bf-43bc-b997-b42324ae76e9 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:09.876604 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5ee9d43b-24bf-43bc-b997-b42324ae76e9 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:09.880759 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5ee9d43b-24bf-43bc-b997-b42324ae76e9 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:09.881466 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5ee9d43b-24bf-43bc-b997-b42324ae76e9 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:09.903985 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-5ee9d43b-24bf-43bc-b997-b42324ae76e9 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] HTTP exception thrown: Instance e7598b4a-f70c-4186-88d1-6fd0057f8656 could not be found. Jul 16 21:20:09.905806 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-5ee9d43b-24bf-43bc-b997-b42324ae76e9 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Request method failure captured: Jul 16 21:20:09.905806 ncdv-master-1 devstack@n-api.service[138680]: request: GET /compute/v2.1/servers/e7598b4a-f70c-4186-88d1-6fd0057f8656 HTTP/1.1 Jul 16 21:20:09.905806 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:20:09.905806 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:20:09.905806 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:20:09.905806 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 0 Jul 16 21:20:09.905806 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:20:09.905806 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:20:09.905806 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:20:09.905806 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:20:09.905806 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0yrNpNpQByhQ1578GareI_bLbzFzwt1X7DjqAJogmQKBk7pzKioqjlhICyrqQjyNEkmOeTkxKX8xROhDwwvvteGCjPW-Knwb8qWdwKE0LlYrz9U5NzOQ8ukFz_cfNcdWpVOqqWw9XjDKSvggxCIEhYLmedQYWwv9L2-iYQUT-tM1e3g Jul 16 21:20:09.905806 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:20:09.905806 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:20:09.905806 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:20:09.905806 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:20:09.905806 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:20:09.905806 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:20:09.905806 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: 65d5bcfa02af4e8284ba1342197e2860 Jul 16 21:20:09.905806 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-ServersAdminTestJSON-1438684720 Jul 16 21:20:09.905806 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:20:09.905806 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:20:09.905806 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/65d5bcfa02af4e8284ba1342197e2860"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/65d5bcfa02af4e8284ba1342197e2860"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_65d5bcfa02af4e8284ba1342197e2860"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/65d5bcfa02af4e8284ba1342197e2860"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:20:09.905806 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-ServersAdminTestJSON-1438684720 Jul 16 21:20:09.905806 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: 65d5bcfa02af4e8284ba1342197e2860 Jul 16 21:20:09.905806 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-ServersAdminTestJSON-1438684720 Jul 16 21:20:09.905806 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-ServersAdminTestJSON-1438684720-project Jul 16 21:20:09.905806 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:20:09.905806 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:20:09.905806 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: 2f5bc1f2209f4eb186d80e385b67c774 Jul 16 21:20:09.905806 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-ServersAdminTestJSON-1438684720-project Jul 16 21:20:09.905806 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 21:20:09.905806 ncdv-master-1 devstack@n-api.service[138680]: exception: Instance e7598b4a-f70c-4186-88d1-6fd0057f8656 could not be found. Jul 16 21:20:09.905806 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': 'e7598b4a-f70c-4186-88d1-6fd0057f8656'} Jul 16 21:20:09.905806 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:20:09.905806 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:20:09.905806 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:20:09.905806 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:20:09.905806 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:20:09.905806 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:20:09.905806 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:20:09.905806 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:20:09.905806 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:20:09.905806 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:20:09.905806 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance e7598b4a-f70c-4186-88d1-6fd0057f8656 could not be found. Jul 16 21:20:09.905806 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:20:09.905806 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:20:09.905806 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:20:09.905806 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:20:09.905806 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:20:09.905806 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:20:09.905806 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:20:09.905806 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:20:09.905806 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:20:09.905806 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:20:09.905806 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:20:09.905806 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:20:09.905806 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:20:09.905806 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:20:09.905806 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:20:09.905806 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:20:09.905806 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance e7598b4a-f70c-4186-88d1-6fd0057f8656 could not be found. Jul 16 21:20:09.905806 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:20:09.905806 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:20:09.905806 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:20:09.910744 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:20:09.910744 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:20:09.910744 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:20:09.910744 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:20:09.910744 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:20:09.910744 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance e7598b4a-f70c-4186-88d1-6fd0057f8656 could not be found. Jul 16 21:20:09.910744 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:20:09.916540 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-5ee9d43b-24bf-43bc-b997-b42324ae76e9 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] Returning 404 to user: Instance e7598b4a-f70c-4186-88d1-6fd0057f8656 could not be found. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:20:09.916540 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-5ee9d43b-24bf-43bc-b997-b42324ae76e9 tempest-ServersAdminTestJSON-1438684720 tempest-ServersAdminTestJSON-1438684720-project] 10.222.0.16 "GET /compute/v2.1/servers/e7598b4a-f70c-4186-88d1-6fd0057f8656" status: 404 len: 111 microversion: 2.1 time: 0.075971 Jul 16 21:20:09.920181 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1426/2801] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:20:09 2022] GET /compute/v2.1/servers/e7598b4a-f70c-4186-88d1-6fd0057f8656 => generated 111 bytes in 82 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:20:10.268064 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-b553343d-5fc8-40f6-8b12-c4b52d54a339 tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:20:10.270174 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-b553343d-5fc8-40f6-8b12-c4b52d54a339 tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] [instance: 0a9564cb-c375-47e9-bfe6-e68d4700442b] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:20:10.293523 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b553343d-5fc8-40f6-8b12-c4b52d54a339 tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:10.296330 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b553343d-5fc8-40f6-8b12-c4b52d54a339 tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:10.298568 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-91e3acdd-e2da-49c8-87cb-b29df6716efd tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eebb224a56814f92b12ebebebc6bc8a3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a092ce9ef40943f6b6ace6cd28481b71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:20:10.299363 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b553343d-5fc8-40f6-8b12-c4b52d54a339 tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:10.300952 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b553343d-5fc8-40f6-8b12-c4b52d54a339 tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:10.314617 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-91e3acdd-e2da-49c8-87cb-b29df6716efd tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] 10.222.0.16 "GET /compute/v2.1/servers/9d2cd0fc-1101-4e21-8179-c1dd4605594f" status: 200 len: 1513 microversion: 2.1 time: 0.623413 Jul 16 21:20:10.315795 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1376/2802] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:20:09 2022] GET /compute/v2.1/servers/9d2cd0fc-1101-4e21-8179-c1dd4605594f => generated 1513 bytes in 625 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:20:10.320794 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-9c836a6e-3604-4f0c-814f-024620057f4d tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:20:10.329723 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-9c836a6e-3604-4f0c-814f-024620057f4d tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] [instance: 84aa4a51-e376-4829-9a38-2ea447ebb413] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:20:10.345188 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9c836a6e-3604-4f0c-814f-024620057f4d tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:10.345645 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9c836a6e-3604-4f0c-814f-024620057f4d tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:10.346423 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9c836a6e-3604-4f0c-814f-024620057f4d tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:10.346942 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9c836a6e-3604-4f0c-814f-024620057f4d tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:10.932195 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-b553343d-5fc8-40f6-8b12-c4b52d54a339 tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1f72b1e0b4124480bfd53b5c22a343e0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0be222897b3a4864a39e9fc7d481340a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:20:10.946930 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-b553343d-5fc8-40f6-8b12-c4b52d54a339 tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] 10.222.0.16 "GET /compute/v2.1/servers/0a9564cb-c375-47e9-bfe6-e68d4700442b" status: 200 len: 1402 microversion: 2.1 time: 0.684338 Jul 16 21:20:10.958764 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1427/2803] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:20:10 2022] GET /compute/v2.1/servers/0a9564cb-c375-47e9-bfe6-e68d4700442b => generated 1402 bytes in 689 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:20:10.963711 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-be2189b3-46e3-41ba-a3bc-76f669678c6d tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:20:10.965067 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-be2189b3-46e3-41ba-a3bc-76f669678c6d tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] [instance: 6cba399c-58c3-4850-8065-0fbc3301fb2c] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:20:10.979672 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-be2189b3-46e3-41ba-a3bc-76f669678c6d tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:10.980194 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-be2189b3-46e3-41ba-a3bc-76f669678c6d tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:10.980922 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-be2189b3-46e3-41ba-a3bc-76f669678c6d tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:10.981439 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-be2189b3-46e3-41ba-a3bc-76f669678c6d tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:11.038904 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-9c836a6e-3604-4f0c-814f-024620057f4d tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fe680ccdc60e410b812794de8aa73e0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '772bd01188344e288824730238fee49b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:20:11.052830 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-9c836a6e-3604-4f0c-814f-024620057f4d tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] 10.222.0.16 "GET /compute/v2.1/servers/84aa4a51-e376-4829-9a38-2ea447ebb413" status: 200 len: 1313 microversion: 2.1 time: 0.730501 Jul 16 21:20:11.056770 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1377/2804] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:20:10 2022] GET /compute/v2.1/servers/84aa4a51-e376-4829-9a38-2ea447ebb413 => generated 1313 bytes in 740 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:20:11.265441 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-be2189b3-46e3-41ba-a3bc-76f669678c6d tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c6ba562873fa4f4fbd0b635f1c9dcdf2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '837a8f9e06e141dcbd5f7bd895016d49', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:20:11.277957 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-be2189b3-46e3-41ba-a3bc-76f669678c6d tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] 10.222.0.16 "GET /compute/v2.1/servers/6cba399c-58c3-4850-8065-0fbc3301fb2c" status: 200 len: 1296 microversion: 2.1 time: 0.324876 Jul 16 21:20:11.283399 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1428/2805] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:20:10 2022] GET /compute/v2.1/servers/6cba399c-58c3-4850-8065-0fbc3301fb2c => generated 1296 bytes in 332 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:20:11.332052 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-af0944d8-982e-4608-9905-95f4489a212a tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:20:11.332689 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-af0944d8-982e-4608-9905-95f4489a212a tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] [instance: 9d2cd0fc-1101-4e21-8179-c1dd4605594f] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:20:11.347308 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-af0944d8-982e-4608-9905-95f4489a212a tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:11.352488 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-af0944d8-982e-4608-9905-95f4489a212a tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:11.353589 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-af0944d8-982e-4608-9905-95f4489a212a tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:11.359754 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-af0944d8-982e-4608-9905-95f4489a212a tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:11.416959 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=97,nova_cell1:INSERT=2,nova_cell1:UPDATE=1 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:20:11.596079 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-2f1d4144-5d94-4654-b3de-6fdfab557cb5 req-a7b2e36d-6252-4499-a14a-5cb86e79a42c service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "0a9564cb-c375-47e9-bfe6-e68d4700442b", "name": "network-vif-plugged", "status": "completed", "tag": "22eac33c-df13-4324-a3f6-14093cd60ead"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:20:11.618720 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-2f1d4144-5d94-4654-b3de-6fdfab557cb5 req-a7b2e36d-6252-4499-a14a-5cb86e79a42c service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:11.623391 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-2f1d4144-5d94-4654-b3de-6fdfab557cb5 req-a7b2e36d-6252-4499-a14a-5cb86e79a42c service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:11.660436 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-2f1d4144-5d94-4654-b3de-6fdfab557cb5 req-a7b2e36d-6252-4499-a14a-5cb86e79a42c service nova] Creating event network-vif-plugged:22eac33c-df13-4324-a3f6-14093cd60ead for instance 0a9564cb-c375-47e9-bfe6-e68d4700442b on nch2-master-1 Jul 16 21:20:11.666603 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-af0944d8-982e-4608-9905-95f4489a212a tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eebb224a56814f92b12ebebebc6bc8a3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a092ce9ef40943f6b6ace6cd28481b71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:20:11.671457 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-2f1d4144-5d94-4654-b3de-6fdfab557cb5 req-a7b2e36d-6252-4499-a14a-5cb86e79a42c service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.078120 Jul 16 21:20:11.675365 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1429/2806] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:20:11 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:20:11.684841 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-af0944d8-982e-4608-9905-95f4489a212a tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] 10.222.0.16 "GET /compute/v2.1/servers/9d2cd0fc-1101-4e21-8179-c1dd4605594f" status: 200 len: 1471 microversion: 2.1 time: 0.353234 Jul 16 21:20:11.689576 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1378/2807] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:20:11 2022] GET /compute/v2.1/servers/9d2cd0fc-1101-4e21-8179-c1dd4605594f => generated 1471 bytes in 363 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:20:11.948543 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-0ee70909-077a-4984-8c5e-5b227fb76d80 req-258fdfaf-03d6-46fe-945b-bf193d2a92f4 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "9d2cd0fc-1101-4e21-8179-c1dd4605594f", "name": "network-vif-deleted", "tag": "4589c1e7-5239-4b55-9840-6b8a096c6324"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:20:11.974600 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-71c904b2-f888-4b77-9d4a-b1cb15f6970b tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:20:11.975378 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-71c904b2-f888-4b77-9d4a-b1cb15f6970b tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] [instance: 0a9564cb-c375-47e9-bfe6-e68d4700442b] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:20:11.975522 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-0ee70909-077a-4984-8c5e-5b227fb76d80 req-258fdfaf-03d6-46fe-945b-bf193d2a92f4 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:11.976092 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-0ee70909-077a-4984-8c5e-5b227fb76d80 req-258fdfaf-03d6-46fe-945b-bf193d2a92f4 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:11.995589 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-71c904b2-f888-4b77-9d4a-b1cb15f6970b tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:11.995962 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-71c904b2-f888-4b77-9d4a-b1cb15f6970b tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:11.999197 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-71c904b2-f888-4b77-9d4a-b1cb15f6970b tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:11.999781 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-71c904b2-f888-4b77-9d4a-b1cb15f6970b tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:12.013210 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-0ee70909-077a-4984-8c5e-5b227fb76d80 req-258fdfaf-03d6-46fe-945b-bf193d2a92f4 service nova] Creating event network-vif-deleted:4589c1e7-5239-4b55-9840-6b8a096c6324 for instance 9d2cd0fc-1101-4e21-8179-c1dd4605594f on nch2-master-1 Jul 16 21:20:12.028164 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-0ee70909-077a-4984-8c5e-5b227fb76d80 req-258fdfaf-03d6-46fe-945b-bf193d2a92f4 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.086500 Jul 16 21:20:12.029352 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1430/2808] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:20:11 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 16 21:20:12.073355 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-52f861e7-ee8d-4a7c-b565-ac1f52167fb4 tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:20:12.074510 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-52f861e7-ee8d-4a7c-b565-ac1f52167fb4 tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] [instance: 84aa4a51-e376-4829-9a38-2ea447ebb413] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:20:12.086867 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-52f861e7-ee8d-4a7c-b565-ac1f52167fb4 tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:12.093668 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-52f861e7-ee8d-4a7c-b565-ac1f52167fb4 tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:12.094658 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-52f861e7-ee8d-4a7c-b565-ac1f52167fb4 tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:12.095356 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-52f861e7-ee8d-4a7c-b565-ac1f52167fb4 tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:12.757014 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-52f861e7-ee8d-4a7c-b565-ac1f52167fb4 tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fe680ccdc60e410b812794de8aa73e0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '772bd01188344e288824730238fee49b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:20:12.766946 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-52f861e7-ee8d-4a7c-b565-ac1f52167fb4 tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] 10.222.0.16 "GET /compute/v2.1/servers/84aa4a51-e376-4829-9a38-2ea447ebb413" status: 200 len: 1313 microversion: 2.1 time: 0.697887 Jul 16 21:20:12.774048 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1431/2809] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:20:12 2022] GET /compute/v2.1/servers/84aa4a51-e376-4829-9a38-2ea447ebb413 => generated 1313 bytes in 706 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:20:12.791183 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-f85fac51-a64e-4094-8624-761d1b272168 tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:20:12.791183 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-f85fac51-a64e-4094-8624-761d1b272168 tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] [instance: 6cba399c-58c3-4850-8065-0fbc3301fb2c] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:20:12.793941 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f85fac51-a64e-4094-8624-761d1b272168 tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:12.794844 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f85fac51-a64e-4094-8624-761d1b272168 tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:12.796063 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f85fac51-a64e-4094-8624-761d1b272168 tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:12.796755 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f85fac51-a64e-4094-8624-761d1b272168 tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:12.861770 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-71c904b2-f888-4b77-9d4a-b1cb15f6970b tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1f72b1e0b4124480bfd53b5c22a343e0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0be222897b3a4864a39e9fc7d481340a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:20:12.871589 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-71c904b2-f888-4b77-9d4a-b1cb15f6970b tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] 10.222.0.16 "GET /compute/v2.1/servers/0a9564cb-c375-47e9-bfe6-e68d4700442b" status: 200 len: 1402 microversion: 2.1 time: 0.909118 Jul 16 21:20:12.874969 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1379/2810] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:20:11 2022] GET /compute/v2.1/servers/0a9564cb-c375-47e9-bfe6-e68d4700442b => generated 1402 bytes in 913 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:20:12.882988 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-e51dc08a-ae3f-4f1f-b3c2-cbe807e577dd tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:20:12.885547 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-e51dc08a-ae3f-4f1f-b3c2-cbe807e577dd tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] [instance: 9d2cd0fc-1101-4e21-8179-c1dd4605594f] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:20:12.893045 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e51dc08a-ae3f-4f1f-b3c2-cbe807e577dd tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:12.893744 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e51dc08a-ae3f-4f1f-b3c2-cbe807e577dd tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:12.894741 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e51dc08a-ae3f-4f1f-b3c2-cbe807e577dd tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:12.904221 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e51dc08a-ae3f-4f1f-b3c2-cbe807e577dd tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:12.947656 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e51dc08a-ae3f-4f1f-b3c2-cbe807e577dd tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:12.948122 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e51dc08a-ae3f-4f1f-b3c2-cbe807e577dd tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:12.948717 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.objects.instance [None req-e51dc08a-ae3f-4f1f-b3c2-cbe807e577dd tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Lazy-loading 'fault' on Instance uuid 9d2cd0fc-1101-4e21-8179-c1dd4605594f {{(pid=138681) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:20:13.296001 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-e51dc08a-ae3f-4f1f-b3c2-cbe807e577dd tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eebb224a56814f92b12ebebebc6bc8a3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a092ce9ef40943f6b6ace6cd28481b71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:20:13.311204 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-e51dc08a-ae3f-4f1f-b3c2-cbe807e577dd tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] 10.222.0.16 "GET /compute/v2.1/servers/9d2cd0fc-1101-4e21-8179-c1dd4605594f" status: 200 len: 1308 microversion: 2.1 time: 0.432740 Jul 16 21:20:13.312938 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1380/2811] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:20:12 2022] GET /compute/v2.1/servers/9d2cd0fc-1101-4e21-8179-c1dd4605594f => generated 1308 bytes in 435 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:20:13.506436 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-f85fac51-a64e-4094-8624-761d1b272168 tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c6ba562873fa4f4fbd0b635f1c9dcdf2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '837a8f9e06e141dcbd5f7bd895016d49', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:20:13.515785 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-f85fac51-a64e-4094-8624-761d1b272168 tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] 10.222.0.16 "GET /compute/v2.1/servers/6cba399c-58c3-4850-8065-0fbc3301fb2c" status: 200 len: 1352 microversion: 2.1 time: 0.739729 Jul 16 21:20:13.516381 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1432/2812] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:20:12 2022] GET /compute/v2.1/servers/6cba399c-58c3-4850-8065-0fbc3301fb2c => generated 1352 bytes in 741 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:20:13.793233 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-068d79f2-d87b-4668-8d13-bd2f6f6f4e0e tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:20:13.797705 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-068d79f2-d87b-4668-8d13-bd2f6f6f4e0e tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] [instance: 84aa4a51-e376-4829-9a38-2ea447ebb413] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:20:13.801919 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-068d79f2-d87b-4668-8d13-bd2f6f6f4e0e tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:13.802672 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-068d79f2-d87b-4668-8d13-bd2f6f6f4e0e tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:13.803966 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-068d79f2-d87b-4668-8d13-bd2f6f6f4e0e tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:13.805470 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-068d79f2-d87b-4668-8d13-bd2f6f6f4e0e tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:13.891516 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-9df59305-6208-4a31-b86d-203d053343bb tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:20:13.893236 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-9df59305-6208-4a31-b86d-203d053343bb tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] [instance: 0a9564cb-c375-47e9-bfe6-e68d4700442b] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:20:13.904165 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9df59305-6208-4a31-b86d-203d053343bb tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:13.904682 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9df59305-6208-4a31-b86d-203d053343bb tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:13.905492 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9df59305-6208-4a31-b86d-203d053343bb tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:13.906032 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9df59305-6208-4a31-b86d-203d053343bb tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:14.222338 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-068d79f2-d87b-4668-8d13-bd2f6f6f4e0e tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fe680ccdc60e410b812794de8aa73e0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '772bd01188344e288824730238fee49b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:20:14.236366 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-068d79f2-d87b-4668-8d13-bd2f6f6f4e0e tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] 10.222.0.16 "GET /compute/v2.1/servers/84aa4a51-e376-4829-9a38-2ea447ebb413" status: 200 len: 1313 microversion: 2.1 time: 0.443852 Jul 16 21:20:14.237585 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1381/2813] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:20:13 2022] GET /compute/v2.1/servers/84aa4a51-e376-4829-9a38-2ea447ebb413 => generated 1313 bytes in 452 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:20:14.248178 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-181f4247-e341-4826-8439-819ecd990d3f req-973674f1-3c14-4f7f-aeaf-9c02a00c84ac service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "a56f8463-6006-4805-9021-1babc2d98f0f", "name": "network-vif-deleted", "tag": "d407f4a6-f978-413f-821d-6a31fd8785c8"}, {"name": "network-changed", "server_uuid": "84aa4a51-e376-4829-9a38-2ea447ebb413", "tag": "b361b1b8-4e34-435d-b0ca-1bb7c92fb2fd"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:20:14.276967 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-181f4247-e341-4826-8439-819ecd990d3f req-973674f1-3c14-4f7f-aeaf-9c02a00c84ac service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:14.277850 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-181f4247-e341-4826-8439-819ecd990d3f req-973674f1-3c14-4f7f-aeaf-9c02a00c84ac service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:14.297630 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-181f4247-e341-4826-8439-819ecd990d3f req-973674f1-3c14-4f7f-aeaf-9c02a00c84ac service nova] Creating event network-vif-deleted:d407f4a6-f978-413f-821d-6a31fd8785c8 for instance a56f8463-6006-4805-9021-1babc2d98f0f on nch1-master-1 Jul 16 21:20:14.297630 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-181f4247-e341-4826-8439-819ecd990d3f req-973674f1-3c14-4f7f-aeaf-9c02a00c84ac service nova] Creating event network-changed:b361b1b8-4e34-435d-b0ca-1bb7c92fb2fd for instance 84aa4a51-e376-4829-9a38-2ea447ebb413 on nch1-master-1 Jul 16 21:20:14.305002 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-181f4247-e341-4826-8439-819ecd990d3f req-973674f1-3c14-4f7f-aeaf-9c02a00c84ac service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 350 microversion: 2.1 time: 0.065948 Jul 16 21:20:14.305628 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1382/2814] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:20:14 2022] POST /compute/v2.1/os-server-external-events => generated 350 bytes in 67 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:20:14.311521 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-9df59305-6208-4a31-b86d-203d053343bb tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1f72b1e0b4124480bfd53b5c22a343e0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0be222897b3a4864a39e9fc7d481340a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:20:14.321516 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-9df59305-6208-4a31-b86d-203d053343bb tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] 10.222.0.16 "GET /compute/v2.1/servers/0a9564cb-c375-47e9-bfe6-e68d4700442b" status: 200 len: 1402 microversion: 2.1 time: 0.434188 Jul 16 21:20:14.322113 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1433/2815] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:20:13 2022] GET /compute/v2.1/servers/0a9564cb-c375-47e9-bfe6-e68d4700442b => generated 1402 bytes in 436 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:20:14.332809 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-32ae8618-5d52-4024-a54a-f650ecf5ba4a tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:20:14.334812 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-32ae8618-5d52-4024-a54a-f650ecf5ba4a tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] [instance: 9d2cd0fc-1101-4e21-8179-c1dd4605594f] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:20:14.341463 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-32ae8618-5d52-4024-a54a-f650ecf5ba4a tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:14.341909 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-32ae8618-5d52-4024-a54a-f650ecf5ba4a tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:14.352002 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-32ae8618-5d52-4024-a54a-f650ecf5ba4a tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.009s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:14.356740 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-32ae8618-5d52-4024-a54a-f650ecf5ba4a tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:14.370100 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-32ae8618-5d52-4024-a54a-f650ecf5ba4a tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] HTTP exception thrown: Instance 9d2cd0fc-1101-4e21-8179-c1dd4605594f could not be found. Jul 16 21:20:14.372079 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-32ae8618-5d52-4024-a54a-f650ecf5ba4a tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Request method failure captured: Jul 16 21:20:14.372079 ncdv-master-1 devstack@n-api.service[138681]: request: GET /compute/v2.1/servers/9d2cd0fc-1101-4e21-8179-c1dd4605594f HTTP/1.1 Jul 16 21:20:14.372079 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:20:14.372079 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:20:14.372079 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:20:14.372079 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 21:20:14.372079 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:20:14.372079 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:20:14.372079 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:20:14.372079 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:20:14.372079 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0ytHvsWW2V-xDxUDHKm8iBJmB6hisOmzLa9bX37LW-R5voU4GNo1F50zz6JJlqhQPjExG3AVUtKv0jYdNgl7lAdeGREJlzb_inip0wO1M2WFJvdMsWGtEHz_jZAMR7rp2tEwqFOU1iIldo4UykC31P1gYCG1gGcn9q4NmcREEdnxrOY Jul 16 21:20:14.372079 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:20:14.372079 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:20:14.372079 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:20:14.372079 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:20:14.372079 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:20:14.372079 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:20:14.372079 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: a092ce9ef40943f6b6ace6cd28481b71 Jul 16 21:20:14.372079 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-MultipleCreateTestJSON-874278387 Jul 16 21:20:14.372079 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:20:14.372079 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:20:14.372079 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/a092ce9ef40943f6b6ace6cd28481b71"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/a092ce9ef40943f6b6ace6cd28481b71"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_a092ce9ef40943f6b6ace6cd28481b71"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/a092ce9ef40943f6b6ace6cd28481b71"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:20:14.372079 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-MultipleCreateTestJSON-874278387 Jul 16 21:20:14.372079 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: a092ce9ef40943f6b6ace6cd28481b71 Jul 16 21:20:14.372079 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-MultipleCreateTestJSON-874278387 Jul 16 21:20:14.372079 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-MultipleCreateTestJSON-874278387-project Jul 16 21:20:14.372079 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:20:14.372079 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:20:14.372079 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: eebb224a56814f92b12ebebebc6bc8a3 Jul 16 21:20:14.372079 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-MultipleCreateTestJSON-874278387-project Jul 16 21:20:14.372079 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:20:14.372079 ncdv-master-1 devstack@n-api.service[138681]: exception: Instance 9d2cd0fc-1101-4e21-8179-c1dd4605594f could not be found. Jul 16 21:20:14.372079 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': '9d2cd0fc-1101-4e21-8179-c1dd4605594f'} Jul 16 21:20:14.372079 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:20:14.372079 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:20:14.372079 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:20:14.372079 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:20:14.372079 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:20:14.372079 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:20:14.372079 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:20:14.372079 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:20:14.372079 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:20:14.372079 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:20:14.372079 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 9d2cd0fc-1101-4e21-8179-c1dd4605594f could not be found. Jul 16 21:20:14.372079 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:20:14.372079 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:20:14.372079 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:20:14.372079 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:20:14.372079 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:20:14.372079 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:20:14.372079 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:20:14.372079 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:20:14.372079 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:20:14.372079 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:20:14.372079 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:20:14.372079 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:20:14.372079 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:20:14.372079 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:20:14.372079 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:20:14.372079 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:20:14.372079 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 9d2cd0fc-1101-4e21-8179-c1dd4605594f could not be found. Jul 16 21:20:14.372079 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:20:14.372079 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:20:14.372079 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:20:14.376210 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:20:14.376210 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:20:14.376210 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:20:14.376210 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:20:14.376210 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:20:14.376210 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 9d2cd0fc-1101-4e21-8179-c1dd4605594f could not be found. Jul 16 21:20:14.376210 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:20:14.388758 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-32ae8618-5d52-4024-a54a-f650ecf5ba4a tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Returning 404 to user: Instance 9d2cd0fc-1101-4e21-8179-c1dd4605594f could not be found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:20:14.390036 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-32ae8618-5d52-4024-a54a-f650ecf5ba4a tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] 10.222.0.16 "GET /compute/v2.1/servers/9d2cd0fc-1101-4e21-8179-c1dd4605594f" status: 404 len: 111 microversion: 2.1 time: 0.061603 Jul 16 21:20:14.390630 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1383/2816] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:20:14 2022] GET /compute/v2.1/servers/9d2cd0fc-1101-4e21-8179-c1dd4605594f => generated 111 bytes in 63 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:20:14.400438 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-933f5503-524d-46bd-91c9-597ffcdc9364 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:20:14.400629 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-933f5503-524d-46bd-91c9-597ffcdc9364 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] [instance: a56f8463-6006-4805-9021-1babc2d98f0f] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:20:14.407955 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-933f5503-524d-46bd-91c9-597ffcdc9364 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:14.408732 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-933f5503-524d-46bd-91c9-597ffcdc9364 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:14.409768 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-933f5503-524d-46bd-91c9-597ffcdc9364 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:14.415359 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-933f5503-524d-46bd-91c9-597ffcdc9364 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:14.426533 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-933f5503-524d-46bd-91c9-597ffcdc9364 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] HTTP exception thrown: Instance a56f8463-6006-4805-9021-1babc2d98f0f could not be found. Jul 16 21:20:14.429221 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-933f5503-524d-46bd-91c9-597ffcdc9364 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Request method failure captured: Jul 16 21:20:14.429221 ncdv-master-1 devstack@n-api.service[138680]: request: GET /compute/v2.1/servers/a56f8463-6006-4805-9021-1babc2d98f0f HTTP/1.1 Jul 16 21:20:14.429221 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:20:14.429221 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:20:14.429221 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:20:14.429221 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 0 Jul 16 21:20:14.429221 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:20:14.429221 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:20:14.429221 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:20:14.429221 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:20:14.429221 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0ytHvsWW2V-xDxUDHKm8iBJmB6hisOmzLa9bX37LW-R5voU4GNo1F50zz6JJlqhQPjExG3AVUtKv0jYdNgl7lAdeGREJlzb_inip0wO1M2WFJvdMsWGtEHz_jZAMR7rp2tEwqFOU1iIldo4UykC31P1gYCG1gGcn9q4NmcREEdnxrOY Jul 16 21:20:14.429221 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:20:14.429221 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:20:14.429221 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:20:14.429221 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:20:14.429221 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:20:14.429221 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:20:14.429221 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: a092ce9ef40943f6b6ace6cd28481b71 Jul 16 21:20:14.429221 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-MultipleCreateTestJSON-874278387 Jul 16 21:20:14.429221 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:20:14.429221 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:20:14.429221 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/a092ce9ef40943f6b6ace6cd28481b71"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/a092ce9ef40943f6b6ace6cd28481b71"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_a092ce9ef40943f6b6ace6cd28481b71"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/a092ce9ef40943f6b6ace6cd28481b71"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:20:14.429221 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-MultipleCreateTestJSON-874278387 Jul 16 21:20:14.429221 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: a092ce9ef40943f6b6ace6cd28481b71 Jul 16 21:20:14.429221 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-MultipleCreateTestJSON-874278387 Jul 16 21:20:14.429221 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-MultipleCreateTestJSON-874278387-project Jul 16 21:20:14.429221 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:20:14.429221 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:20:14.429221 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: eebb224a56814f92b12ebebebc6bc8a3 Jul 16 21:20:14.429221 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-MultipleCreateTestJSON-874278387-project Jul 16 21:20:14.429221 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 21:20:14.429221 ncdv-master-1 devstack@n-api.service[138680]: exception: Instance a56f8463-6006-4805-9021-1babc2d98f0f could not be found. Jul 16 21:20:14.429221 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': 'a56f8463-6006-4805-9021-1babc2d98f0f'} Jul 16 21:20:14.429221 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:20:14.429221 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:20:14.429221 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:20:14.429221 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:20:14.429221 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:20:14.429221 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:20:14.429221 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:20:14.429221 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:20:14.429221 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:20:14.429221 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:20:14.429221 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance a56f8463-6006-4805-9021-1babc2d98f0f could not be found. Jul 16 21:20:14.429221 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:20:14.429221 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:20:14.429221 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:20:14.429221 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:20:14.429221 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:20:14.429221 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:20:14.429221 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:20:14.429221 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:20:14.429221 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:20:14.429221 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:20:14.429221 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:20:14.429221 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:20:14.429221 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:20:14.429221 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:20:14.429221 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:20:14.429221 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:20:14.429221 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance a56f8463-6006-4805-9021-1babc2d98f0f could not be found. Jul 16 21:20:14.429221 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:20:14.429221 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:20:14.429221 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:20:14.436668 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:20:14.436668 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:20:14.436668 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:20:14.436668 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:20:14.436668 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:20:14.436668 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance a56f8463-6006-4805-9021-1babc2d98f0f could not be found. Jul 16 21:20:14.436668 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:20:14.436668 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-933f5503-524d-46bd-91c9-597ffcdc9364 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] Returning 404 to user: Instance a56f8463-6006-4805-9021-1babc2d98f0f could not be found. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:20:14.436668 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-933f5503-524d-46bd-91c9-597ffcdc9364 tempest-MultipleCreateTestJSON-874278387 tempest-MultipleCreateTestJSON-874278387-project] 10.222.0.16 "GET /compute/v2.1/servers/a56f8463-6006-4805-9021-1babc2d98f0f" status: 404 len: 111 microversion: 2.1 time: 0.035689 Jul 16 21:20:14.440524 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1434/2817] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:20:14 2022] GET /compute/v2.1/servers/a56f8463-6006-4805-9021-1babc2d98f0f => generated 111 bytes in 46 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Jul 16 21:20:14.536785 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-eed67a6a-db99-43d0-a62c-7b2c7fc54ccc tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:20:14.537504 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-eed67a6a-db99-43d0-a62c-7b2c7fc54ccc tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] [instance: 6cba399c-58c3-4850-8065-0fbc3301fb2c] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:20:14.544296 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-eed67a6a-db99-43d0-a62c-7b2c7fc54ccc tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:14.544850 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-eed67a6a-db99-43d0-a62c-7b2c7fc54ccc tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:14.545887 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-eed67a6a-db99-43d0-a62c-7b2c7fc54ccc tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:14.546991 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-eed67a6a-db99-43d0-a62c-7b2c7fc54ccc tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:14.731546 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-eed67a6a-db99-43d0-a62c-7b2c7fc54ccc tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c6ba562873fa4f4fbd0b635f1c9dcdf2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '837a8f9e06e141dcbd5f7bd895016d49', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:20:14.741156 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-eed67a6a-db99-43d0-a62c-7b2c7fc54ccc tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] 10.222.0.16 "GET /compute/v2.1/servers/6cba399c-58c3-4850-8065-0fbc3301fb2c" status: 200 len: 1352 microversion: 2.1 time: 0.211132 Jul 16 21:20:14.741986 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1435/2818] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:20:14 2022] GET /compute/v2.1/servers/6cba399c-58c3-4850-8065-0fbc3301fb2c => generated 1352 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:20:14.906811 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-536b07dd-ab31-44e1-9cd7-18e2fdda7a2c req-4ea2e34c-d3ea-4afd-b40f-c1f932684e85 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "6cba399c-58c3-4850-8065-0fbc3301fb2c", "tag": "607e9a0d-96c5-4d29-a42f-425502256e42"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:20:14.922762 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-536b07dd-ab31-44e1-9cd7-18e2fdda7a2c req-4ea2e34c-d3ea-4afd-b40f-c1f932684e85 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:14.923249 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-536b07dd-ab31-44e1-9cd7-18e2fdda7a2c req-4ea2e34c-d3ea-4afd-b40f-c1f932684e85 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:14.946942 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-536b07dd-ab31-44e1-9cd7-18e2fdda7a2c req-4ea2e34c-d3ea-4afd-b40f-c1f932684e85 service nova] Creating event network-changed:607e9a0d-96c5-4d29-a42f-425502256e42 for instance 6cba399c-58c3-4850-8065-0fbc3301fb2c on nch2-master-1 Jul 16 21:20:14.952370 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-536b07dd-ab31-44e1-9cd7-18e2fdda7a2c req-4ea2e34c-d3ea-4afd-b40f-c1f932684e85 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.517340 Jul 16 21:20:14.953324 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1384/2819] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:20:14 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 519 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 16 21:20:15.255801 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-bff82228-b204-4d07-9426-861de521a45d tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:20:15.257286 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-bff82228-b204-4d07-9426-861de521a45d tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] [instance: 84aa4a51-e376-4829-9a38-2ea447ebb413] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:20:15.266611 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-bff82228-b204-4d07-9426-861de521a45d tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:15.268132 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-bff82228-b204-4d07-9426-861de521a45d tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:15.269253 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-bff82228-b204-4d07-9426-861de521a45d tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:15.269773 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-bff82228-b204-4d07-9426-861de521a45d tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:15.345394 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-05ef9d8a-f589-449d-b404-a1f7aaa5b82f tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:20:15.345773 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-05ef9d8a-f589-449d-b404-a1f7aaa5b82f tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] [instance: 0a9564cb-c375-47e9-bfe6-e68d4700442b] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:20:15.357725 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-05ef9d8a-f589-449d-b404-a1f7aaa5b82f tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:15.359704 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-05ef9d8a-f589-449d-b404-a1f7aaa5b82f tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:15.362175 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-05ef9d8a-f589-449d-b404-a1f7aaa5b82f tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:15.363250 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-05ef9d8a-f589-449d-b404-a1f7aaa5b82f tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:15.625374 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-bff82228-b204-4d07-9426-861de521a45d tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fe680ccdc60e410b812794de8aa73e0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '772bd01188344e288824730238fee49b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:20:15.639302 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-bff82228-b204-4d07-9426-861de521a45d tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] 10.222.0.16 "GET /compute/v2.1/servers/84aa4a51-e376-4829-9a38-2ea447ebb413" status: 200 len: 1313 microversion: 2.1 time: 0.389137 Jul 16 21:20:15.640273 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1436/2820] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:20:15 2022] GET /compute/v2.1/servers/84aa4a51-e376-4829-9a38-2ea447ebb413 => generated 1313 bytes in 391 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:20:15.738904 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-05ef9d8a-f589-449d-b404-a1f7aaa5b82f tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1f72b1e0b4124480bfd53b5c22a343e0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0be222897b3a4864a39e9fc7d481340a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:20:15.749068 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-05ef9d8a-f589-449d-b404-a1f7aaa5b82f tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] 10.222.0.16 "GET /compute/v2.1/servers/0a9564cb-c375-47e9-bfe6-e68d4700442b" status: 200 len: 1589 microversion: 2.1 time: 0.412706 Jul 16 21:20:15.749981 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1385/2821] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:20:15 2022] GET /compute/v2.1/servers/0a9564cb-c375-47e9-bfe6-e68d4700442b => generated 1589 bytes in 415 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:20:15.759626 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-6a98dc74-4dbf-4694-8ce6-ab973d333994 tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:20:15.763393 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-6a98dc74-4dbf-4694-8ce6-ab973d333994 tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] [instance: 6cba399c-58c3-4850-8065-0fbc3301fb2c] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:20:15.769600 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-09568209-1f7b-4a89-94e2-8b1f94b9fdfe tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:20:15.769600 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-09568209-1f7b-4a89-94e2-8b1f94b9fdfe tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] [instance: 0a9564cb-c375-47e9-bfe6-e68d4700442b] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:20:15.788804 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-09568209-1f7b-4a89-94e2-8b1f94b9fdfe tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:15.794144 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-09568209-1f7b-4a89-94e2-8b1f94b9fdfe tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:15.794306 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6a98dc74-4dbf-4694-8ce6-ab973d333994 tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.014s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:15.794397 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-09568209-1f7b-4a89-94e2-8b1f94b9fdfe tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:15.794397 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-09568209-1f7b-4a89-94e2-8b1f94b9fdfe tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:15.794563 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6a98dc74-4dbf-4694-8ce6-ab973d333994 tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:15.796871 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6a98dc74-4dbf-4694-8ce6-ab973d333994 tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:15.797560 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6a98dc74-4dbf-4694-8ce6-ab973d333994 tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:15.932623 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-09568209-1f7b-4a89-94e2-8b1f94b9fdfe tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] 10.222.0.16 "GET /compute/v2.1/servers/0a9564cb-c375-47e9-bfe6-e68d4700442b/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.168039 Jul 16 21:20:15.933750 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1386/2822] 10.222.0.16 () {58 vars in 1274 bytes} [Sat Jul 16 21:20:15 2022] GET /compute/v2.1/servers/0a9564cb-c375-47e9-bfe6-e68d4700442b/os-interface => generated 284 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 16 21:20:16.040925 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-6a98dc74-4dbf-4694-8ce6-ab973d333994 tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c6ba562873fa4f4fbd0b635f1c9dcdf2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '837a8f9e06e141dcbd5f7bd895016d49', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:20:16.055570 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-6a98dc74-4dbf-4694-8ce6-ab973d333994 tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] 10.222.0.16 "GET /compute/v2.1/servers/6cba399c-58c3-4850-8065-0fbc3301fb2c" status: 200 len: 1352 microversion: 2.1 time: 0.300001 Jul 16 21:20:16.057201 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1437/2823] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:20:15 2022] GET /compute/v2.1/servers/6cba399c-58c3-4850-8065-0fbc3301fb2c => generated 1352 bytes in 303 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:20:16.358059 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell0:SELECT=6 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:20:16.657356 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-01dd8fce-81ea-4aa6-a8a1-1abe806bdd6b tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:20:16.658661 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-01dd8fce-81ea-4aa6-a8a1-1abe806bdd6b tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] [instance: 84aa4a51-e376-4829-9a38-2ea447ebb413] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:20:16.667173 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-01dd8fce-81ea-4aa6-a8a1-1abe806bdd6b tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:16.667935 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-01dd8fce-81ea-4aa6-a8a1-1abe806bdd6b tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:16.668584 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-01dd8fce-81ea-4aa6-a8a1-1abe806bdd6b tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:16.669102 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-01dd8fce-81ea-4aa6-a8a1-1abe806bdd6b tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:16.787301 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-3ebd5ac9-2dd4-40c7-864b-5ee8433ad4ff tempest-AZV2TestJSON-659418679 tempest-AZV2TestJSON-659418679-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:20:16.790630 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3ebd5ac9-2dd4-40c7-864b-5ee8433ad4ff tempest-AZV2TestJSON-659418679 tempest-AZV2TestJSON-659418679-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:16.791802 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3ebd5ac9-2dd4-40c7-864b-5ee8433ad4ff tempest-AZV2TestJSON-659418679 tempest-AZV2TestJSON-659418679-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:16.794454 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3ebd5ac9-2dd4-40c7-864b-5ee8433ad4ff tempest-AZV2TestJSON-659418679 tempest-AZV2TestJSON-659418679-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:16.803584 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3ebd5ac9-2dd4-40c7-864b-5ee8433ad4ff tempest-AZV2TestJSON-659418679 tempest-AZV2TestJSON-659418679-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:16.882027 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-3ebd5ac9-2dd4-40c7-864b-5ee8433ad4ff tempest-AZV2TestJSON-659418679 tempest-AZV2TestJSON-659418679-project] 10.222.0.16 "GET /compute/v2.1/os-availability-zone" status: 200 len: 97 microversion: 2.1 time: 0.524668 Jul 16 21:20:16.889217 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-01dd8fce-81ea-4aa6-a8a1-1abe806bdd6b tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fe680ccdc60e410b812794de8aa73e0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '772bd01188344e288824730238fee49b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:20:16.889374 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1387/2824] 10.222.0.16 () {58 vars in 1163 bytes} [Sat Jul 16 21:20:16 2022] GET /compute/v2.1/os-availability-zone => generated 97 bytes in 530 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Jul 16 21:20:16.898611 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-01dd8fce-81ea-4aa6-a8a1-1abe806bdd6b tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] 10.222.0.16 "GET /compute/v2.1/servers/84aa4a51-e376-4829-9a38-2ea447ebb413" status: 200 len: 1313 microversion: 2.1 time: 0.245618 Jul 16 21:20:16.900256 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1438/2825] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:20:16 2022] GET /compute/v2.1/servers/84aa4a51-e376-4829-9a38-2ea447ebb413 => generated 1313 bytes in 248 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:20:17.073410 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-b7c29627-8e12-4ac8-a999-6bd897f2b227 tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:20:17.074153 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-b7c29627-8e12-4ac8-a999-6bd897f2b227 tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] [instance: 6cba399c-58c3-4850-8065-0fbc3301fb2c] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:20:17.086446 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b7c29627-8e12-4ac8-a999-6bd897f2b227 tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:17.087164 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b7c29627-8e12-4ac8-a999-6bd897f2b227 tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:17.088591 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b7c29627-8e12-4ac8-a999-6bd897f2b227 tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:17.089767 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b7c29627-8e12-4ac8-a999-6bd897f2b227 tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:17.163427 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell0:SELECT=54 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:20:17.249321 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-b7c29627-8e12-4ac8-a999-6bd897f2b227 tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c6ba562873fa4f4fbd0b635f1c9dcdf2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '837a8f9e06e141dcbd5f7bd895016d49', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:20:17.258365 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-b7c29627-8e12-4ac8-a999-6bd897f2b227 tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] 10.222.0.16 "GET /compute/v2.1/servers/6cba399c-58c3-4850-8065-0fbc3301fb2c" status: 200 len: 1352 microversion: 2.1 time: 0.189226 Jul 16 21:20:17.259035 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1388/2826] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:20:17 2022] GET /compute/v2.1/servers/6cba399c-58c3-4850-8065-0fbc3301fb2c => generated 1352 bytes in 191 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:20:17.919346 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-d026d81c-924e-43d3-a7c8-c80e29a0f522 tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:20:17.927878 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-d026d81c-924e-43d3-a7c8-c80e29a0f522 tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] [instance: 84aa4a51-e376-4829-9a38-2ea447ebb413] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:20:17.931313 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d026d81c-924e-43d3-a7c8-c80e29a0f522 tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:17.938525 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d026d81c-924e-43d3-a7c8-c80e29a0f522 tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:17.940279 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d026d81c-924e-43d3-a7c8-c80e29a0f522 tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:17.942257 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d026d81c-924e-43d3-a7c8-c80e29a0f522 tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:18.183616 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-5e3856c2-6158-4ea5-a868-8531f27f7b14 tempest-ServicesAdminNegativeTestJSON-1545737068 tempest-ServicesAdminNegativeTestJSON-1545737068-project-admin] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:20:18.184963 ncdv-master-1 devstack@n-api.service[138681]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-services:list": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:20:18.184963 ncdv-master-1 devstack@n-api.service[138681]: warnings.warn(msg) Jul 16 21:20:18.188384 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5e3856c2-6158-4ea5-a868-8531f27f7b14 tempest-ServicesAdminNegativeTestJSON-1545737068 tempest-ServicesAdminNegativeTestJSON-1545737068-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:18.191481 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5e3856c2-6158-4ea5-a868-8531f27f7b14 tempest-ServicesAdminNegativeTestJSON-1545737068 tempest-ServicesAdminNegativeTestJSON-1545737068-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:18.191481 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5e3856c2-6158-4ea5-a868-8531f27f7b14 tempest-ServicesAdminNegativeTestJSON-1545737068 tempest-ServicesAdminNegativeTestJSON-1545737068-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:18.191481 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5e3856c2-6158-4ea5-a868-8531f27f7b14 tempest-ServicesAdminNegativeTestJSON-1545737068 tempest-ServicesAdminNegativeTestJSON-1545737068-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:18.226092 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-5e3856c2-6158-4ea5-a868-8531f27f7b14 tempest-ServicesAdminNegativeTestJSON-1545737068 tempest-ServicesAdminNegativeTestJSON-1545737068-project-admin] 10.222.0.16 "GET /compute/v2.1/os-services" status: 200 len: 947 microversion: 2.1 time: 0.262209 Jul 16 21:20:18.226981 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1389/2827] 10.222.0.16 () {58 vars in 1136 bytes} [Sat Jul 16 21:20:17 2022] GET /compute/v2.1/os-services => generated 947 bytes in 264 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:20:18.229603 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-d026d81c-924e-43d3-a7c8-c80e29a0f522 tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fe680ccdc60e410b812794de8aa73e0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '772bd01188344e288824730238fee49b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:20:18.238185 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-d70f70c3-0be9-4201-9d38-f87d0864ae13 tempest-ServicesAdminNegativeTestJSON-1545737068 tempest-ServicesAdminNegativeTestJSON-1545737068-project-admin] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:20:18.240829 ncdv-master-1 devstack@n-api.service[138681]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-services:list": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:20:18.240829 ncdv-master-1 devstack@n-api.service[138681]: warnings.warn(msg) Jul 16 21:20:18.242988 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d70f70c3-0be9-4201-9d38-f87d0864ae13 tempest-ServicesAdminNegativeTestJSON-1545737068 tempest-ServicesAdminNegativeTestJSON-1545737068-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:18.243637 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d70f70c3-0be9-4201-9d38-f87d0864ae13 tempest-ServicesAdminNegativeTestJSON-1545737068 tempest-ServicesAdminNegativeTestJSON-1545737068-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:18.244607 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d70f70c3-0be9-4201-9d38-f87d0864ae13 tempest-ServicesAdminNegativeTestJSON-1545737068 tempest-ServicesAdminNegativeTestJSON-1545737068-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:18.245201 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d70f70c3-0be9-4201-9d38-f87d0864ae13 tempest-ServicesAdminNegativeTestJSON-1545737068 tempest-ServicesAdminNegativeTestJSON-1545737068-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:18.246635 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-d026d81c-924e-43d3-a7c8-c80e29a0f522 tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] 10.222.0.16 "GET /compute/v2.1/servers/84aa4a51-e376-4829-9a38-2ea447ebb413" status: 200 len: 1313 microversion: 2.1 time: 0.333064 Jul 16 21:20:18.247653 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1439/2828] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:20:17 2022] GET /compute/v2.1/servers/84aa4a51-e376-4829-9a38-2ea447ebb413 => generated 1313 bytes in 335 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:20:18.276107 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-35f95b11-0c7c-42a1-b378-3f018d32659b tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:20:18.276647 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-d70f70c3-0be9-4201-9d38-f87d0864ae13 tempest-ServicesAdminNegativeTestJSON-1545737068 tempest-ServicesAdminNegativeTestJSON-1545737068-project-admin] 10.222.0.16 "GET /compute/v2.1/os-services" status: 200 len: 947 microversion: 2.1 time: 0.042443 Jul 16 21:20:18.276795 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-35f95b11-0c7c-42a1-b378-3f018d32659b tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] [instance: 6cba399c-58c3-4850-8065-0fbc3301fb2c] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:20:18.278961 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1390/2829] 10.222.0.16 () {58 vars in 1169 bytes} [Sat Jul 16 21:20:18 2022] GET /compute/v2.1/os-services?xxx=nova-compute => generated 947 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 16 21:20:18.283052 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-35f95b11-0c7c-42a1-b378-3f018d32659b tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:18.283671 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-35f95b11-0c7c-42a1-b378-3f018d32659b tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:18.284946 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-35f95b11-0c7c-42a1-b378-3f018d32659b tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:18.285544 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-35f95b11-0c7c-42a1-b378-3f018d32659b tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:18.294486 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-5e266d93-5221-4e50-aacb-2ccdfa68f20e tempest-ServicesAdminNegativeTestJSON-1545737068 tempest-ServicesAdminNegativeTestJSON-1545737068-project-admin] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:20:18.295991 ncdv-master-1 devstack@n-api.service[138681]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-services:list": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:20:18.295991 ncdv-master-1 devstack@n-api.service[138681]: warnings.warn(msg) Jul 16 21:20:18.297400 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5e266d93-5221-4e50-aacb-2ccdfa68f20e tempest-ServicesAdminNegativeTestJSON-1545737068 tempest-ServicesAdminNegativeTestJSON-1545737068-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:18.298098 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5e266d93-5221-4e50-aacb-2ccdfa68f20e tempest-ServicesAdminNegativeTestJSON-1545737068 tempest-ServicesAdminNegativeTestJSON-1545737068-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:18.299076 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5e266d93-5221-4e50-aacb-2ccdfa68f20e tempest-ServicesAdminNegativeTestJSON-1545737068 tempest-ServicesAdminNegativeTestJSON-1545737068-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:18.300281 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5e266d93-5221-4e50-aacb-2ccdfa68f20e tempest-ServicesAdminNegativeTestJSON-1545737068 tempest-ServicesAdminNegativeTestJSON-1545737068-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:18.329822 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-5e266d93-5221-4e50-aacb-2ccdfa68f20e tempest-ServicesAdminNegativeTestJSON-1545737068 tempest-ServicesAdminNegativeTestJSON-1545737068-project-admin] 10.222.0.16 "GET /compute/v2.1/os-services" status: 200 len: 947 microversion: 2.1 time: 0.039028 Jul 16 21:20:18.337233 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1391/2830] 10.222.0.16 () {58 vars in 1136 bytes} [Sat Jul 16 21:20:18 2022] GET /compute/v2.1/os-services => generated 947 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:20:18.344813 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-f6a1cce8-00e6-412c-92eb-23f26f8b68be tempest-ServicesAdminNegativeTestJSON-1545737068 tempest-ServicesAdminNegativeTestJSON-1545737068-project-admin] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:20:18.347752 ncdv-master-1 devstack@n-api.service[138681]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-services:list": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:20:18.347752 ncdv-master-1 devstack@n-api.service[138681]: warnings.warn(msg) Jul 16 21:20:18.349951 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f6a1cce8-00e6-412c-92eb-23f26f8b68be tempest-ServicesAdminNegativeTestJSON-1545737068 tempest-ServicesAdminNegativeTestJSON-1545737068-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:18.350604 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f6a1cce8-00e6-412c-92eb-23f26f8b68be tempest-ServicesAdminNegativeTestJSON-1545737068 tempest-ServicesAdminNegativeTestJSON-1545737068-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:18.351622 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f6a1cce8-00e6-412c-92eb-23f26f8b68be tempest-ServicesAdminNegativeTestJSON-1545737068 tempest-ServicesAdminNegativeTestJSON-1545737068-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:18.352236 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f6a1cce8-00e6-412c-92eb-23f26f8b68be tempest-ServicesAdminNegativeTestJSON-1545737068 tempest-ServicesAdminNegativeTestJSON-1545737068-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:18.382882 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-f6a1cce8-00e6-412c-92eb-23f26f8b68be tempest-ServicesAdminNegativeTestJSON-1545737068 tempest-ServicesAdminNegativeTestJSON-1545737068-project-admin] 10.222.0.16 "GET /compute/v2.1/os-services?host=ncdv-master-1&binary=xxx" status: 200 len: 16 microversion: 2.1 time: 0.043060 Jul 16 21:20:18.390258 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1392/2831] 10.222.0.16 () {58 vars in 1195 bytes} [Sat Jul 16 21:20:18 2022] GET /compute/v2.1/os-services?host=ncdv-master-1&binary=xxx => generated 16 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Jul 16 21:20:18.395831 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-0558e083-74db-4bb5-8582-afe099b2ac30 tempest-ServicesAdminNegativeTestJSON-1545737068 tempest-ServicesAdminNegativeTestJSON-1545737068-project-admin] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:20:18.397576 ncdv-master-1 devstack@n-api.service[138681]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-services:list": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:20:18.397576 ncdv-master-1 devstack@n-api.service[138681]: warnings.warn(msg) Jul 16 21:20:18.399262 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0558e083-74db-4bb5-8582-afe099b2ac30 tempest-ServicesAdminNegativeTestJSON-1545737068 tempest-ServicesAdminNegativeTestJSON-1545737068-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:18.399872 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0558e083-74db-4bb5-8582-afe099b2ac30 tempest-ServicesAdminNegativeTestJSON-1545737068 tempest-ServicesAdminNegativeTestJSON-1545737068-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:18.400942 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0558e083-74db-4bb5-8582-afe099b2ac30 tempest-ServicesAdminNegativeTestJSON-1545737068 tempest-ServicesAdminNegativeTestJSON-1545737068-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:18.401531 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0558e083-74db-4bb5-8582-afe099b2ac30 tempest-ServicesAdminNegativeTestJSON-1545737068 tempest-ServicesAdminNegativeTestJSON-1545737068-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:18.437400 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-0558e083-74db-4bb5-8582-afe099b2ac30 tempest-ServicesAdminNegativeTestJSON-1545737068 tempest-ServicesAdminNegativeTestJSON-1545737068-project-admin] 10.222.0.16 "GET /compute/v2.1/os-services" status: 200 len: 947 microversion: 2.1 time: 0.044317 Jul 16 21:20:18.437400 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1393/2832] 10.222.0.16 () {58 vars in 1136 bytes} [Sat Jul 16 21:20:18 2022] GET /compute/v2.1/os-services => generated 947 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:20:18.449693 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-07576fc1-b05f-4b4c-86d7-6acce2b6d8f5 tempest-ServicesAdminNegativeTestJSON-1545737068 tempest-ServicesAdminNegativeTestJSON-1545737068-project-admin] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:20:18.451766 ncdv-master-1 devstack@n-api.service[138681]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-services:list": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:20:18.451766 ncdv-master-1 devstack@n-api.service[138681]: warnings.warn(msg) Jul 16 21:20:18.453230 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-07576fc1-b05f-4b4c-86d7-6acce2b6d8f5 tempest-ServicesAdminNegativeTestJSON-1545737068 tempest-ServicesAdminNegativeTestJSON-1545737068-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:18.453870 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-07576fc1-b05f-4b4c-86d7-6acce2b6d8f5 tempest-ServicesAdminNegativeTestJSON-1545737068 tempest-ServicesAdminNegativeTestJSON-1545737068-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:18.455054 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-07576fc1-b05f-4b4c-86d7-6acce2b6d8f5 tempest-ServicesAdminNegativeTestJSON-1545737068 tempest-ServicesAdminNegativeTestJSON-1545737068-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:18.455784 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-07576fc1-b05f-4b4c-86d7-6acce2b6d8f5 tempest-ServicesAdminNegativeTestJSON-1545737068 tempest-ServicesAdminNegativeTestJSON-1545737068-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:18.469651 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-35f95b11-0c7c-42a1-b378-3f018d32659b tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c6ba562873fa4f4fbd0b635f1c9dcdf2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '837a8f9e06e141dcbd5f7bd895016d49', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:20:18.478811 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-35f95b11-0c7c-42a1-b378-3f018d32659b tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] 10.222.0.16 "GET /compute/v2.1/servers/6cba399c-58c3-4850-8065-0fbc3301fb2c" status: 200 len: 1352 microversion: 2.1 time: 0.207273 Jul 16 21:20:18.480213 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1440/2833] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:20:18 2022] GET /compute/v2.1/servers/6cba399c-58c3-4850-8065-0fbc3301fb2c => generated 1352 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:20:18.492536 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-07576fc1-b05f-4b4c-86d7-6acce2b6d8f5 tempest-ServicesAdminNegativeTestJSON-1545737068 tempest-ServicesAdminNegativeTestJSON-1545737068-project-admin] 10.222.0.16 "GET /compute/v2.1/os-services?host=xxx&binary=nova-scheduler" status: 200 len: 16 microversion: 2.1 time: 0.047400 Jul 16 21:20:18.493196 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1394/2834] 10.222.0.16 () {58 vars in 1197 bytes} [Sat Jul 16 21:20:18 2022] GET /compute/v2.1/os-services?host=xxx&binary=nova-scheduler => generated 16 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Jul 16 21:20:18.605650 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-0e3a10da-a0bc-49fc-bba2-bac0d824e4ca tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:20:18.606280 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-0e3a10da-a0bc-49fc-bba2-bac0d824e4ca tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] [instance: 0a9564cb-c375-47e9-bfe6-e68d4700442b] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:20:18.637903 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0e3a10da-a0bc-49fc-bba2-bac0d824e4ca tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:18.640508 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0e3a10da-a0bc-49fc-bba2-bac0d824e4ca tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.011s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:18.645112 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0e3a10da-a0bc-49fc-bba2-bac0d824e4ca tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:18.646829 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0e3a10da-a0bc-49fc-bba2-bac0d824e4ca tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:18.951858 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-a3160e98-3881-40d2-88b9-adf6d05915e9 tempest-ServicesAdminNegativeTestJSON-355610019 tempest-ServicesAdminNegativeTestJSON-355610019-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:20:18.953403 ncdv-master-1 devstack@n-api.service[138680]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-services:list": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:20:18.953403 ncdv-master-1 devstack@n-api.service[138680]: warnings.warn(msg) Jul 16 21:20:18.954751 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-a3160e98-3881-40d2-88b9-adf6d05915e9 tempest-ServicesAdminNegativeTestJSON-355610019 tempest-ServicesAdminNegativeTestJSON-355610019-project] Policy check for os_compute_api:os-services:list failed with credentials {'is_admin': False, 'user_id': '9df596bcfbfc48ff8fa292551b14b9fa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '21b58c588e03487795f46b458b5a8094', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:20:18.958673 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-a3160e98-3881-40d2-88b9-adf6d05915e9 tempest-ServicesAdminNegativeTestJSON-355610019 tempest-ServicesAdminNegativeTestJSON-355610019-project] Request method failure captured: Jul 16 21:20:18.958673 ncdv-master-1 devstack@n-api.service[138680]: request: GET /compute/v2.1/os-services HTTP/1.1 Jul 16 21:20:18.958673 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:20:18.958673 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:20:18.958673 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:20:18.958673 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 0 Jul 16 21:20:18.958673 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:20:18.958673 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:20:18.958673 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:20:18.958673 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:20:18.958673 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0yuQRIcecREH6OHRwkJxkahlH99CQ7Vq_tikI5neEM2X5sm253ch0t90TEqPPVRB2IY6kCAx-HVc-lm08uPCwU1jQIiv0uRT4Jl1PHmRrNzeLmJqgEc6VJwUd0jduuQ4qBvx5f-Njno7cPwztMd_QrzDG0IU5Owm8dAcjCzhydLXZfU Jul 16 21:20:18.958673 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:20:18.958673 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:20:18.958673 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:20:18.958673 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:20:18.958673 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:20:18.958673 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:20:18.958673 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: 21b58c588e03487795f46b458b5a8094 Jul 16 21:20:18.958673 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-ServicesAdminNegativeTestJSON-355610019 Jul 16 21:20:18.958673 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:20:18.958673 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:20:18.958673 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/21b58c588e03487795f46b458b5a8094"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/21b58c588e03487795f46b458b5a8094"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_21b58c588e03487795f46b458b5a8094"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/21b58c588e03487795f46b458b5a8094"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:20:18.958673 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-ServicesAdminNegativeTestJSON-355610019 Jul 16 21:20:18.958673 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: 21b58c588e03487795f46b458b5a8094 Jul 16 21:20:18.958673 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-ServicesAdminNegativeTestJSON-355610019 Jul 16 21:20:18.958673 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-ServicesAdminNegativeTestJSON-355610019-project Jul 16 21:20:18.958673 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:20:18.958673 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:20:18.958673 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: 9df596bcfbfc48ff8fa292551b14b9fa Jul 16 21:20:18.958673 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-ServicesAdminNegativeTestJSON-355610019-project Jul 16 21:20:18.958673 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 21:20:18.958673 ncdv-master-1 devstack@n-api.service[138680]: exception: Policy doesn't allow os_compute_api:os-services:list to be performed. Jul 16 21:20:18.958673 ncdv-master-1 devstack@n-api.service[138680]: action_args: {} Jul 16 21:20:18.958673 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:20:18.958673 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:20:18.958673 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:20:18.958673 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:20:18.958673 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:20:18.958673 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:20:18.958673 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper Jul 16 21:20:18.958673 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 16 21:20:18.958673 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper Jul 16 21:20:18.958673 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 16 21:20:18.958673 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:20:18.958673 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:20:18.958673 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/services.py", line 387, in index Jul 16 21:20:18.958673 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi context.can(services_policies.BASE_POLICY_NAME % 'list', target={}) Jul 16 21:20:18.958673 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/context.py", line 240, in can Jul 16 21:20:18.958673 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return policy.authorize(self, action, target) Jul 16 21:20:18.958673 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 199, in authorize Jul 16 21:20:18.958673 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi LOG.debug('Policy check for %(action)s failed with credentials ' Jul 16 21:20:18.958673 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 16 21:20:18.958673 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi self.force_reraise() Jul 16 21:20:18.958673 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 16 21:20:18.958673 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise self.value Jul 16 21:20:18.958673 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 186, in authorize Jul 16 21:20:18.958673 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi result = _ENFORCER.authorize(action, target, context, Jul 16 21:20:18.958673 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1184, in authorize Jul 16 21:20:18.958673 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return self.enforce( Jul 16 21:20:18.958673 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1088, in enforce Jul 16 21:20:18.958673 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc(*args, **kwargs) Jul 16 21:20:18.958673 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.PolicyNotAuthorized: Policy doesn't allow os_compute_api:os-services:list to be performed. Jul 16 21:20:18.958673 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:20:18.962368 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:20:18.962368 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:20:18.962368 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:20:18.962368 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:20:18.962368 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:20:18.962368 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 354, in __exit__ Jul 16 21:20:18.962368 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(webob.exc.HTTPForbidden( Jul 16 21:20:18.962368 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Policy doesn't allow os_compute_api:os-services:list to be performed. Jul 16 21:20:18.962368 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:20:18.962368 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-a3160e98-3881-40d2-88b9-adf6d05915e9 tempest-ServicesAdminNegativeTestJSON-355610019 tempest-ServicesAdminNegativeTestJSON-355610019-project] Returning 403 to user: Policy doesn't allow os_compute_api:os-services:list to be performed. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:20:18.962368 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-a3160e98-3881-40d2-88b9-adf6d05915e9 tempest-ServicesAdminNegativeTestJSON-355610019 tempest-ServicesAdminNegativeTestJSON-355610019-project] 10.222.0.16 "GET /compute/v2.1/os-services" status: 403 len: 112 microversion: 2.1 time: 0.459715 Jul 16 21:20:18.970526 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1441/2835] 10.222.0.16 () {58 vars in 1136 bytes} [Sat Jul 16 21:20:18 2022] GET /compute/v2.1/os-services => generated 112 bytes in 468 msecs (HTTP/1.1 403) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:20:18.981340 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-0a100cab-1f86-4806-a3e0-5d0f0e4efe64 req-0d4d9a43-84ae-42f0-9050-2c9a78d7b072 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "0a9564cb-c375-47e9-bfe6-e68d4700442b", "tag": "22eac33c-df13-4324-a3f6-14093cd60ead"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:20:19.005669 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-0a100cab-1f86-4806-a3e0-5d0f0e4efe64 req-0d4d9a43-84ae-42f0-9050-2c9a78d7b072 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:19.008529 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-0a100cab-1f86-4806-a3e0-5d0f0e4efe64 req-0d4d9a43-84ae-42f0-9050-2c9a78d7b072 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:19.055593 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-0a100cab-1f86-4806-a3e0-5d0f0e4efe64 req-0d4d9a43-84ae-42f0-9050-2c9a78d7b072 service nova] Creating event network-changed:22eac33c-df13-4324-a3f6-14093cd60ead for instance 0a9564cb-c375-47e9-bfe6-e68d4700442b on nch2-master-1 Jul 16 21:20:19.062779 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-0e3a10da-a0bc-49fc-bba2-bac0d824e4ca tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1f72b1e0b4124480bfd53b5c22a343e0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0be222897b3a4864a39e9fc7d481340a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:20:19.071215 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-0a100cab-1f86-4806-a3e0-5d0f0e4efe64 req-0d4d9a43-84ae-42f0-9050-2c9a78d7b072 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.101139 Jul 16 21:20:19.075577 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-0e3a10da-a0bc-49fc-bba2-bac0d824e4ca tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] 10.222.0.16 "GET /compute/v2.1/servers/0a9564cb-c375-47e9-bfe6-e68d4700442b" status: 200 len: 1589 microversion: 2.1 time: 0.476854 Jul 16 21:20:19.076482 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1395/2836] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:20:18 2022] GET /compute/v2.1/servers/0a9564cb-c375-47e9-bfe6-e68d4700442b => generated 1589 bytes in 478 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:20:19.089970 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1442/2837] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:20:18 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 121 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:20:19.272851 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-7988120f-0c8b-44ef-978a-eb19d64be22e tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:20:19.274683 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-7988120f-0c8b-44ef-978a-eb19d64be22e tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] [instance: 84aa4a51-e376-4829-9a38-2ea447ebb413] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:20:19.293399 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7988120f-0c8b-44ef-978a-eb19d64be22e tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:19.293859 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7988120f-0c8b-44ef-978a-eb19d64be22e tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:19.294662 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7988120f-0c8b-44ef-978a-eb19d64be22e tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:19.295282 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7988120f-0c8b-44ef-978a-eb19d64be22e tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:19.498943 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-12f43894-89bd-4c88-9569-fb519e9a875e tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:20:19.499882 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-12f43894-89bd-4c88-9569-fb519e9a875e tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] [instance: 6cba399c-58c3-4850-8065-0fbc3301fb2c] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:20:19.512094 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-12f43894-89bd-4c88-9569-fb519e9a875e tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:19.513599 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-12f43894-89bd-4c88-9569-fb519e9a875e tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:19.515784 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-12f43894-89bd-4c88-9569-fb519e9a875e tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:19.516422 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-12f43894-89bd-4c88-9569-fb519e9a875e tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:19.523881 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=100 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:20:19.640712 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-7988120f-0c8b-44ef-978a-eb19d64be22e tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fe680ccdc60e410b812794de8aa73e0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '772bd01188344e288824730238fee49b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:20:19.652151 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-7988120f-0c8b-44ef-978a-eb19d64be22e tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] 10.222.0.16 "GET /compute/v2.1/servers/84aa4a51-e376-4829-9a38-2ea447ebb413" status: 200 len: 1313 microversion: 2.1 time: 0.385854 Jul 16 21:20:19.652765 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1396/2838] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:20:19 2022] GET /compute/v2.1/servers/84aa4a51-e376-4829-9a38-2ea447ebb413 => generated 1313 bytes in 388 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:20:19.997790 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-12f43894-89bd-4c88-9569-fb519e9a875e tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c6ba562873fa4f4fbd0b635f1c9dcdf2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '837a8f9e06e141dcbd5f7bd895016d49', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:20:20.017098 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-12f43894-89bd-4c88-9569-fb519e9a875e tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] 10.222.0.16 "GET /compute/v2.1/servers/6cba399c-58c3-4850-8065-0fbc3301fb2c" status: 200 len: 1352 microversion: 2.1 time: 0.521294 Jul 16 21:20:20.037196 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1443/2839] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:20:19 2022] GET /compute/v2.1/servers/6cba399c-58c3-4850-8065-0fbc3301fb2c => generated 1352 bytes in 538 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:20:20.110449 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-c45c7e56-a3c2-4e42-8e8a-b15b0d424333 req-df78b3e6-47c3-43ed-8ba3-96ef6d198542 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "84aa4a51-e376-4829-9a38-2ea447ebb413", "name": "network-vif-plugged", "status": "completed", "tag": "b361b1b8-4e34-435d-b0ca-1bb7c92fb2fd"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:20:20.130900 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-c45c7e56-a3c2-4e42-8e8a-b15b0d424333 req-df78b3e6-47c3-43ed-8ba3-96ef6d198542 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:20.131444 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-c45c7e56-a3c2-4e42-8e8a-b15b0d424333 req-df78b3e6-47c3-43ed-8ba3-96ef6d198542 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:20.164266 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-c45c7e56-a3c2-4e42-8e8a-b15b0d424333 req-df78b3e6-47c3-43ed-8ba3-96ef6d198542 service nova] Creating event network-vif-plugged:b361b1b8-4e34-435d-b0ca-1bb7c92fb2fd for instance 84aa4a51-e376-4829-9a38-2ea447ebb413 on nch1-master-1 Jul 16 21:20:20.183927 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-c45c7e56-a3c2-4e42-8e8a-b15b0d424333 req-df78b3e6-47c3-43ed-8ba3-96ef6d198542 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.077778 Jul 16 21:20:20.187833 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1397/2840] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:20:20 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:20:20.669509 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-78a59975-44cf-461b-9416-cd2089498013 tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:20:20.670402 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-78a59975-44cf-461b-9416-cd2089498013 tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] [instance: 84aa4a51-e376-4829-9a38-2ea447ebb413] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:20:20.678520 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-78a59975-44cf-461b-9416-cd2089498013 tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:20.679240 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-78a59975-44cf-461b-9416-cd2089498013 tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:20.680209 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-78a59975-44cf-461b-9416-cd2089498013 tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:20.680850 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-78a59975-44cf-461b-9416-cd2089498013 tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:20.928975 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-78a59975-44cf-461b-9416-cd2089498013 tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fe680ccdc60e410b812794de8aa73e0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '772bd01188344e288824730238fee49b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:20:20.939748 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-78a59975-44cf-461b-9416-cd2089498013 tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] 10.222.0.16 "GET /compute/v2.1/servers/84aa4a51-e376-4829-9a38-2ea447ebb413" status: 200 len: 1313 microversion: 2.1 time: 0.274412 Jul 16 21:20:20.941929 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1444/2841] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:20:20 2022] GET /compute/v2.1/servers/84aa4a51-e376-4829-9a38-2ea447ebb413 => generated 1313 bytes in 277 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:20:21.035104 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-69cca4bd-1d11-42a7-a575-547e6b8bf404 tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:20:21.036292 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-69cca4bd-1d11-42a7-a575-547e6b8bf404 tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] [instance: 6cba399c-58c3-4850-8065-0fbc3301fb2c] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:20:21.043520 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-69cca4bd-1d11-42a7-a575-547e6b8bf404 tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:21.043973 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-69cca4bd-1d11-42a7-a575-547e6b8bf404 tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:21.044750 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-69cca4bd-1d11-42a7-a575-547e6b8bf404 tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:21.045188 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-69cca4bd-1d11-42a7-a575-547e6b8bf404 tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:21.260378 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-69cca4bd-1d11-42a7-a575-547e6b8bf404 tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c6ba562873fa4f4fbd0b635f1c9dcdf2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '837a8f9e06e141dcbd5f7bd895016d49', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:20:21.272281 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-69cca4bd-1d11-42a7-a575-547e6b8bf404 tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] 10.222.0.16 "GET /compute/v2.1/servers/6cba399c-58c3-4850-8065-0fbc3301fb2c" status: 200 len: 1352 microversion: 2.1 time: 0.241112 Jul 16 21:20:21.283445 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1398/2842] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:20:21 2022] GET /compute/v2.1/servers/6cba399c-58c3-4850-8065-0fbc3301fb2c => generated 1352 bytes in 253 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:20:21.636776 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-115f8073-eaeb-411b-ac80-a599a415c617 tempest-VolumesGetTestJSON-1435260878 tempest-VolumesGetTestJSON-1435260878-project] Action: 'create', calling method: .version_select at 0x7fdd9d08e280>, body: {"volume": {"size": 1, "display_name": "tempest-VolumesGetTestJSON-Volume-989658209", "metadata": {"Type": "work"}}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:20:21.966896 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-c5200784-8670-4dc8-8d79-66026f594e26 tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:20:21.967884 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-c5200784-8670-4dc8-8d79-66026f594e26 tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] [instance: 84aa4a51-e376-4829-9a38-2ea447ebb413] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:20:21.998209 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c5200784-8670-4dc8-8d79-66026f594e26 tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:21.998677 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c5200784-8670-4dc8-8d79-66026f594e26 tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:22.002529 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c5200784-8670-4dc8-8d79-66026f594e26 tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:22.002983 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c5200784-8670-4dc8-8d79-66026f594e26 tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:22.210624 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-c5200784-8670-4dc8-8d79-66026f594e26 tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fe680ccdc60e410b812794de8aa73e0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '772bd01188344e288824730238fee49b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:20:22.215932 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-c5200784-8670-4dc8-8d79-66026f594e26 tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] 10.222.0.16 "GET /compute/v2.1/servers/84aa4a51-e376-4829-9a38-2ea447ebb413" status: 200 len: 1498 microversion: 2.1 time: 0.258816 Jul 16 21:20:22.216550 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1399/2843] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:20:21 2022] GET /compute/v2.1/servers/84aa4a51-e376-4829-9a38-2ea447ebb413 => generated 1498 bytes in 260 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:20:22.234005 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-4676c101-c4fe-455a-bf6b-b41844805914 tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:20:22.250105 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.compute.server_groups [None req-4676c101-c4fe-455a-bf6b-b41844805914 tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Querying cell 327e8df1-39cc-4b36-9ea6-6a9b88ed54fc(cell1) for 1 instances {{(pid=138681) _get_not_deleted /opt/stack/nova/nova/api/openstack/compute/server_groups.py:69}} Jul 16 21:20:22.251148 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4676c101-c4fe-455a-bf6b-b41844805914 tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:22.251594 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4676c101-c4fe-455a-bf6b-b41844805914 tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:22.277394 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-4676c101-c4fe-455a-bf6b-b41844805914 tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] 10.222.0.16 "GET /compute/v2.1/os-server-groups/129ae0b9-7a56-4a16-8a71-8c4b96c9564d" status: 200 len: 222 microversion: 2.1 time: 0.047339 Jul 16 21:20:22.282672 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1400/2844] 10.222.0.16 () {58 vars in 1262 bytes} [Sat Jul 16 21:20:22 2022] GET /compute/v2.1/os-server-groups/129ae0b9-7a56-4a16-8a71-8c4b96c9564d => generated 222 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 16 21:20:22.285163 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-044169fb-c9c4-46d5-9dca-ceaeeb3f6235 req-39ce97dc-7499-4a88-a2f6-121b22ebaa4d service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "6cba399c-58c3-4850-8065-0fbc3301fb2c", "name": "network-vif-plugged", "status": "completed", "tag": "607e9a0d-96c5-4d29-a42f-425502256e42"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:20:22.294546 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-044169fb-c9c4-46d5-9dca-ceaeeb3f6235 req-39ce97dc-7499-4a88-a2f6-121b22ebaa4d service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:22.295150 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-044169fb-c9c4-46d5-9dca-ceaeeb3f6235 req-39ce97dc-7499-4a88-a2f6-121b22ebaa4d service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:22.307539 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-044169fb-c9c4-46d5-9dca-ceaeeb3f6235 req-39ce97dc-7499-4a88-a2f6-121b22ebaa4d service nova] Creating event network-vif-plugged:607e9a0d-96c5-4d29-a42f-425502256e42 for instance 6cba399c-58c3-4850-8065-0fbc3301fb2c on nch2-master-1 Jul 16 21:20:22.312857 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-044169fb-c9c4-46d5-9dca-ceaeeb3f6235 req-39ce97dc-7499-4a88-a2f6-121b22ebaa4d service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.031461 Jul 16 21:20:22.313336 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1401/2845] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:20:22 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:20:22.319544 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-cc51df23-ec6c-45ca-bf66-6d20ffca307d tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:20:22.320165 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-cc51df23-ec6c-45ca-bf66-6d20ffca307d tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] [instance: 84aa4a51-e376-4829-9a38-2ea447ebb413] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:20:22.342908 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cc51df23-ec6c-45ca-bf66-6d20ffca307d tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:22.343395 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cc51df23-ec6c-45ca-bf66-6d20ffca307d tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:22.344168 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cc51df23-ec6c-45ca-bf66-6d20ffca307d tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:22.344608 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cc51df23-ec6c-45ca-bf66-6d20ffca307d tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:22.377955 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-cc51df23-ec6c-45ca-bf66-6d20ffca307d tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] [instance: 84aa4a51-e376-4829-9a38-2ea447ebb413] Going to try to terminate instance {{(pid=138681) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 16 21:20:22.410299 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=100 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:20:22.563974 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-115f8073-eaeb-411b-ac80-a599a415c617 tempest-VolumesGetTestJSON-1435260878 tempest-VolumesGetTestJSON-1435260878-project] 10.222.0.16 "POST /compute/v2.1/os-volumes" status: 200 len: 345 microversion: 2.1 time: 1.140141 Jul 16 21:20:22.563974 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1445/2846] 10.222.0.16 () {60 vars in 1155 bytes} [Sat Jul 16 21:20:21 2022] POST /compute/v2.1/os-volumes => generated 345 bytes in 1141 msecs (HTTP/1.1 200) 10 headers in 448 bytes (1 switches on core 0) Jul 16 21:20:22.563974 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-cd43d9b9-283a-4b3d-b965-8159440fe124 tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:20:22.563974 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-cd43d9b9-283a-4b3d-b965-8159440fe124 tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] [instance: 6cba399c-58c3-4850-8065-0fbc3301fb2c] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:20:22.571192 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cd43d9b9-283a-4b3d-b965-8159440fe124 tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:22.571657 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cd43d9b9-283a-4b3d-b965-8159440fe124 tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:22.572460 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cd43d9b9-283a-4b3d-b965-8159440fe124 tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:22.572906 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cd43d9b9-283a-4b3d-b965-8159440fe124 tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:22.763662 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-cc51df23-ec6c-45ca-bf66-6d20ffca307d tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] 10.222.0.16 "DELETE /compute/v2.1/servers/84aa4a51-e376-4829-9a38-2ea447ebb413" status: 204 len: 0 microversion: 2.1 time: 0.448564 Jul 16 21:20:22.764062 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-cd43d9b9-283a-4b3d-b965-8159440fe124 tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c6ba562873fa4f4fbd0b635f1c9dcdf2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '837a8f9e06e141dcbd5f7bd895016d49', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:20:22.767994 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1402/2847] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:20:22 2022] DELETE /compute/v2.1/servers/84aa4a51-e376-4829-9a38-2ea447ebb413 => generated 0 bytes in 454 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 16 21:20:22.774898 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-2de32d58-5e54-4ba7-9c28-2e9398c244c6 tempest-VolumesGetTestJSON-1435260878 tempest-VolumesGetTestJSON-1435260878-project] Calling method '.version_select at 0x7fdd9c57d550>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:20:22.782522 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-cd43d9b9-283a-4b3d-b965-8159440fe124 tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] 10.222.0.16 "GET /compute/v2.1/servers/6cba399c-58c3-4850-8065-0fbc3301fb2c" status: 200 len: 1352 microversion: 2.1 time: 0.223037 Jul 16 21:20:22.783160 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1446/2848] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:20:22 2022] GET /compute/v2.1/servers/6cba399c-58c3-4850-8065-0fbc3301fb2c => generated 1352 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:20:22.789887 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-3b0591cc-4e53-4640-bc0b-c30131f5238d tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:20:22.790816 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-3b0591cc-4e53-4640-bc0b-c30131f5238d tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] [instance: 84aa4a51-e376-4829-9a38-2ea447ebb413] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:20:22.798390 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3b0591cc-4e53-4640-bc0b-c30131f5238d tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:22.799184 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3b0591cc-4e53-4640-bc0b-c30131f5238d tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:22.800188 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3b0591cc-4e53-4640-bc0b-c30131f5238d tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:22.801148 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3b0591cc-4e53-4640-bc0b-c30131f5238d tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:22.861717 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-2de32d58-5e54-4ba7-9c28-2e9398c244c6 tempest-VolumesGetTestJSON-1435260878 tempest-VolumesGetTestJSON-1435260878-project] 10.222.0.16 "GET /compute/v2.1/os-volumes/d1414df7-4173-492a-ae5f-849dda58e82c" status: 200 len: 345 microversion: 2.1 time: 0.092240 Jul 16 21:20:22.862874 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1403/2849] 10.222.0.16 () {58 vars in 1244 bytes} [Sat Jul 16 21:20:22 2022] GET /compute/v2.1/os-volumes/d1414df7-4173-492a-ae5f-849dda58e82c => generated 345 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:20:23.000743 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-3b0591cc-4e53-4640-bc0b-c30131f5238d tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fe680ccdc60e410b812794de8aa73e0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '772bd01188344e288824730238fee49b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:20:23.009062 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-3b0591cc-4e53-4640-bc0b-c30131f5238d tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] 10.222.0.16 "GET /compute/v2.1/servers/84aa4a51-e376-4829-9a38-2ea447ebb413" status: 200 len: 1504 microversion: 2.1 time: 0.224679 Jul 16 21:20:23.009651 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1447/2850] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:20:22 2022] GET /compute/v2.1/servers/84aa4a51-e376-4829-9a38-2ea447ebb413 => generated 1504 bytes in 226 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:20:23.802428 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-5584f79a-aebf-44fd-8d40-f58b2d7d30d0 tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:20:23.803292 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-5584f79a-aebf-44fd-8d40-f58b2d7d30d0 tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] [instance: 6cba399c-58c3-4850-8065-0fbc3301fb2c] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:20:23.810361 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5584f79a-aebf-44fd-8d40-f58b2d7d30d0 tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:23.810886 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5584f79a-aebf-44fd-8d40-f58b2d7d30d0 tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:23.811695 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5584f79a-aebf-44fd-8d40-f58b2d7d30d0 tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:23.812142 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5584f79a-aebf-44fd-8d40-f58b2d7d30d0 tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:23.875901 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-1f719628-cbf9-4c90-b4d9-b510424d085c tempest-VolumesGetTestJSON-1435260878 tempest-VolumesGetTestJSON-1435260878-project] Calling method '.version_select at 0x7fdda2aef670>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:20:23.943871 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-1f719628-cbf9-4c90-b4d9-b510424d085c tempest-VolumesGetTestJSON-1435260878 tempest-VolumesGetTestJSON-1435260878-project] 10.222.0.16 "GET /compute/v2.1/os-volumes/d1414df7-4173-492a-ae5f-849dda58e82c" status: 200 len: 345 microversion: 2.1 time: 0.072356 Jul 16 21:20:23.944685 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1448/2851] 10.222.0.16 () {58 vars in 1244 bytes} [Sat Jul 16 21:20:23 2022] GET /compute/v2.1/os-volumes/d1414df7-4173-492a-ae5f-849dda58e82c => generated 345 bytes in 74 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 16 21:20:24.032456 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-628ba558-1fd6-4352-a36e-1425ef14d73b tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:20:24.034002 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-628ba558-1fd6-4352-a36e-1425ef14d73b tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] [instance: 84aa4a51-e376-4829-9a38-2ea447ebb413] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:20:24.036263 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_api:SELECT=98,nova_api:UPDATE=2 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:20:24.070379 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-628ba558-1fd6-4352-a36e-1425ef14d73b tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:24.071229 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-628ba558-1fd6-4352-a36e-1425ef14d73b tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:24.072445 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-628ba558-1fd6-4352-a36e-1425ef14d73b tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:24.073098 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-628ba558-1fd6-4352-a36e-1425ef14d73b tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:24.405972 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-628ba558-1fd6-4352-a36e-1425ef14d73b tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fe680ccdc60e410b812794de8aa73e0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '772bd01188344e288824730238fee49b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:20:24.414056 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-5584f79a-aebf-44fd-8d40-f58b2d7d30d0 tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c6ba562873fa4f4fbd0b635f1c9dcdf2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '837a8f9e06e141dcbd5f7bd895016d49', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:20:24.419216 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-628ba558-1fd6-4352-a36e-1425ef14d73b tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] 10.222.0.16 "GET /compute/v2.1/servers/84aa4a51-e376-4829-9a38-2ea447ebb413" status: 200 len: 1504 microversion: 2.1 time: 0.395334 Jul 16 21:20:24.420625 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1449/2852] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:20:24 2022] GET /compute/v2.1/servers/84aa4a51-e376-4829-9a38-2ea447ebb413 => generated 1504 bytes in 398 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:20:24.427441 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-5584f79a-aebf-44fd-8d40-f58b2d7d30d0 tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] 10.222.0.16 "GET /compute/v2.1/servers/6cba399c-58c3-4850-8065-0fbc3301fb2c" status: 200 len: 1352 microversion: 2.1 time: 0.630128 Jul 16 21:20:24.431799 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1404/2853] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:20:23 2022] GET /compute/v2.1/servers/6cba399c-58c3-4850-8065-0fbc3301fb2c => generated 1352 bytes in 635 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:20:24.910525 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-167107e4-0ac0-48ea-9975-6ef458ac1293 req-e82783a4-468f-4abf-8825-c8209d89e556 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "84aa4a51-e376-4829-9a38-2ea447ebb413", "name": "network-vif-deleted", "tag": "b361b1b8-4e34-435d-b0ca-1bb7c92fb2fd"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:20:24.922546 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-167107e4-0ac0-48ea-9975-6ef458ac1293 req-e82783a4-468f-4abf-8825-c8209d89e556 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:24.923632 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-167107e4-0ac0-48ea-9975-6ef458ac1293 req-e82783a4-468f-4abf-8825-c8209d89e556 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:24.943213 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-167107e4-0ac0-48ea-9975-6ef458ac1293 req-e82783a4-468f-4abf-8825-c8209d89e556 service nova] Creating event network-vif-deleted:b361b1b8-4e34-435d-b0ca-1bb7c92fb2fd for instance 84aa4a51-e376-4829-9a38-2ea447ebb413 on nch1-master-1 Jul 16 21:20:24.952329 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-167107e4-0ac0-48ea-9975-6ef458ac1293 req-e82783a4-468f-4abf-8825-c8209d89e556 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.047931 Jul 16 21:20:24.953044 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1450/2854] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:20:24 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:20:24.963251 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-a6df27c8-1b82-4e26-9a24-d05d99c005c1 tempest-VolumesGetTestJSON-1435260878 tempest-VolumesGetTestJSON-1435260878-project] Calling method '.version_select at 0x7fdd9d519f70>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:20:25.033246 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-a6df27c8-1b82-4e26-9a24-d05d99c005c1 tempest-VolumesGetTestJSON-1435260878 tempest-VolumesGetTestJSON-1435260878-project] 10.222.0.16 "GET /compute/v2.1/os-volumes/d1414df7-4173-492a-ae5f-849dda58e82c" status: 200 len: 346 microversion: 2.1 time: 0.080199 Jul 16 21:20:25.034313 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1405/2855] 10.222.0.16 () {58 vars in 1244 bytes} [Sat Jul 16 21:20:24 2022] GET /compute/v2.1/os-volumes/d1414df7-4173-492a-ae5f-849dda58e82c => generated 346 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:20:25.047601 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-69e89d10-7098-4e63-a550-f8c0c4938aa9 tempest-VolumesGetTestJSON-1435260878 tempest-VolumesGetTestJSON-1435260878-project] Calling method '.version_select at 0x7fdd9cfeea60>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:20:25.149053 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-69e89d10-7098-4e63-a550-f8c0c4938aa9 tempest-VolumesGetTestJSON-1435260878 tempest-VolumesGetTestJSON-1435260878-project] 10.222.0.16 "GET /compute/v2.1/os-volumes/d1414df7-4173-492a-ae5f-849dda58e82c" status: 200 len: 346 microversion: 2.1 time: 0.105819 Jul 16 21:20:25.149894 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1451/2856] 10.222.0.16 () {58 vars in 1244 bytes} [Sat Jul 16 21:20:25 2022] GET /compute/v2.1/os-volumes/d1414df7-4173-492a-ae5f-849dda58e82c => generated 346 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:20:25.162472 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-79f1a657-1f4d-451a-921a-9e90abaf6815 tempest-VolumesGetTestJSON-1435260878 tempest-VolumesGetTestJSON-1435260878-project] Calling method '.version_select at 0x7fdd9c579ca0>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:20:25.439093 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-c0d27b28-0f19-4412-9854-1f40985f5558 tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:20:25.440171 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-c0d27b28-0f19-4412-9854-1f40985f5558 tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] [instance: 84aa4a51-e376-4829-9a38-2ea447ebb413] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:20:25.447819 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c0d27b28-0f19-4412-9854-1f40985f5558 tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:25.450547 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c0d27b28-0f19-4412-9854-1f40985f5558 tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:25.451703 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c0d27b28-0f19-4412-9854-1f40985f5558 tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:25.452430 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c0d27b28-0f19-4412-9854-1f40985f5558 tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:25.483117 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-79f1a657-1f4d-451a-921a-9e90abaf6815 tempest-VolumesGetTestJSON-1435260878 tempest-VolumesGetTestJSON-1435260878-project] 10.222.0.16 "DELETE /compute/v2.1/os-volumes/d1414df7-4173-492a-ae5f-849dda58e82c" status: 202 len: 0 microversion: 2.1 time: 0.325161 Jul 16 21:20:25.488587 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1406/2857] 10.222.0.16 () {58 vars in 1247 bytes} [Sat Jul 16 21:20:25 2022] DELETE /compute/v2.1/os-volumes/d1414df7-4173-492a-ae5f-849dda58e82c => generated 0 bytes in 327 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 21:20:25.499079 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-794453ee-47ef-492e-a872-279df1a21ac0 tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:20:25.499947 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-794453ee-47ef-492e-a872-279df1a21ac0 tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] [instance: 6cba399c-58c3-4850-8065-0fbc3301fb2c] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:20:25.516971 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c0d27b28-0f19-4412-9854-1f40985f5558 tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:25.517588 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c0d27b28-0f19-4412-9854-1f40985f5558 tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:25.518410 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.objects.instance [None req-c0d27b28-0f19-4412-9854-1f40985f5558 tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Lazy-loading 'fault' on Instance uuid 84aa4a51-e376-4829-9a38-2ea447ebb413 {{(pid=138680) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:20:25.519035 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-794453ee-47ef-492e-a872-279df1a21ac0 tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:25.519666 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-794453ee-47ef-492e-a872-279df1a21ac0 tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:25.520696 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-794453ee-47ef-492e-a872-279df1a21ac0 tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:25.521235 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-794453ee-47ef-492e-a872-279df1a21ac0 tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:25.580462 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-c0d27b28-0f19-4412-9854-1f40985f5558 tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fe680ccdc60e410b812794de8aa73e0f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '772bd01188344e288824730238fee49b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:20:25.589048 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-c0d27b28-0f19-4412-9854-1f40985f5558 tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] 10.222.0.16 "GET /compute/v2.1/servers/84aa4a51-e376-4829-9a38-2ea447ebb413" status: 200 len: 1303 microversion: 2.1 time: 0.154567 Jul 16 21:20:25.589876 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1452/2858] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:20:25 2022] GET /compute/v2.1/servers/84aa4a51-e376-4829-9a38-2ea447ebb413 => generated 1303 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:20:25.594862 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-837d1c35-218a-4db1-9215-a753e84d182e tempest-VolumesGetTestJSON-1435260878 tempest-VolumesGetTestJSON-1435260878-project] Calling method '.version_select at 0x7fdd9cfeeca0>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:20:25.646799 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-837d1c35-218a-4db1-9215-a753e84d182e tempest-VolumesGetTestJSON-1435260878 tempest-VolumesGetTestJSON-1435260878-project] 10.222.0.16 "GET /compute/v2.1/os-volumes/d1414df7-4173-492a-ae5f-849dda58e82c" status: 200 len: 345 microversion: 2.1 time: 0.055634 Jul 16 21:20:25.647423 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1453/2859] 10.222.0.16 () {58 vars in 1244 bytes} [Sat Jul 16 21:20:25 2022] GET /compute/v2.1/os-volumes/d1414df7-4173-492a-ae5f-849dda58e82c => generated 345 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:20:25.722371 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-794453ee-47ef-492e-a872-279df1a21ac0 tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c6ba562873fa4f4fbd0b635f1c9dcdf2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '837a8f9e06e141dcbd5f7bd895016d49', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:20:25.731760 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-794453ee-47ef-492e-a872-279df1a21ac0 tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] 10.222.0.16 "GET /compute/v2.1/servers/6cba399c-58c3-4850-8065-0fbc3301fb2c" status: 200 len: 1543 microversion: 2.1 time: 0.245528 Jul 16 21:20:25.732429 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1407/2860] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:20:25 2022] GET /compute/v2.1/servers/6cba399c-58c3-4850-8065-0fbc3301fb2c => generated 1543 bytes in 247 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:20:25.755228 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-1a540fad-30e3-4f45-99e5-b80765b3b182 tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:20:25.755228 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-1a540fad-30e3-4f45-99e5-b80765b3b182 tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] [instance: 6cba399c-58c3-4850-8065-0fbc3301fb2c] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:20:25.761853 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1a540fad-30e3-4f45-99e5-b80765b3b182 tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:25.762391 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1a540fad-30e3-4f45-99e5-b80765b3b182 tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:25.763275 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1a540fad-30e3-4f45-99e5-b80765b3b182 tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:25.763789 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1a540fad-30e3-4f45-99e5-b80765b3b182 tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:25.927679 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-1a540fad-30e3-4f45-99e5-b80765b3b182 tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] 10.222.0.16 "GET /compute/v2.1/servers/6cba399c-58c3-4850-8065-0fbc3301fb2c/os-interface" status: 200 len: 285 microversion: 2.1 time: 0.177413 Jul 16 21:20:25.930029 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1454/2861] 10.222.0.16 () {58 vars in 1274 bytes} [Sat Jul 16 21:20:25 2022] GET /compute/v2.1/servers/6cba399c-58c3-4850-8065-0fbc3301fb2c/os-interface => generated 285 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:20:26.606897 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-39236364-501f-4817-ac37-4037cfa410b0 tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:20:26.607601 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-39236364-501f-4817-ac37-4037cfa410b0 tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] [instance: 84aa4a51-e376-4829-9a38-2ea447ebb413] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:20:26.614625 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-39236364-501f-4817-ac37-4037cfa410b0 tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:26.615106 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-39236364-501f-4817-ac37-4037cfa410b0 tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:26.615863 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-39236364-501f-4817-ac37-4037cfa410b0 tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:26.616302 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-39236364-501f-4817-ac37-4037cfa410b0 tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:26.629901 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-39236364-501f-4817-ac37-4037cfa410b0 tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] HTTP exception thrown: Instance 84aa4a51-e376-4829-9a38-2ea447ebb413 could not be found. Jul 16 21:20:26.631542 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-39236364-501f-4817-ac37-4037cfa410b0 tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Request method failure captured: Jul 16 21:20:26.631542 ncdv-master-1 devstack@n-api.service[138681]: request: GET /compute/v2.1/servers/84aa4a51-e376-4829-9a38-2ea447ebb413 HTTP/1.1 Jul 16 21:20:26.631542 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:20:26.631542 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:20:26.631542 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:20:26.631542 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 21:20:26.631542 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:20:26.631542 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:20:26.631542 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:20:26.631542 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:20:26.631542 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0yt6As_n2V35_SjRmQhSF7a6QitkyeIu0F-jOVqebNp0HM84P8d1z7a2uuT_Zwuj8DRDlVYNphI_CfLGnrMeXPwEWkp3zvFd6xj0Kc3oSnr8T3EZu6xDDmEBMiXub1HkMFYIj11TnKacr30S5O8ds98eL7YFJyHNi9EFgWKOb6dRtEg Jul 16 21:20:26.631542 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:20:26.631542 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:20:26.631542 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:20:26.631542 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:20:26.631542 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:20:26.631542 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:20:26.631542 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: 772bd01188344e288824730238fee49b Jul 16 21:20:26.631542 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-ServerGroupTestJSON-2037884880 Jul 16 21:20:26.631542 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:20:26.631542 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:20:26.631542 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/772bd01188344e288824730238fee49b"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/772bd01188344e288824730238fee49b"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_772bd01188344e288824730238fee49b"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/772bd01188344e288824730238fee49b"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:20:26.631542 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-ServerGroupTestJSON-2037884880 Jul 16 21:20:26.631542 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: 772bd01188344e288824730238fee49b Jul 16 21:20:26.631542 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-ServerGroupTestJSON-2037884880 Jul 16 21:20:26.631542 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-ServerGroupTestJSON-2037884880-project Jul 16 21:20:26.631542 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:20:26.631542 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:20:26.631542 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: fe680ccdc60e410b812794de8aa73e0f Jul 16 21:20:26.631542 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-ServerGroupTestJSON-2037884880-project Jul 16 21:20:26.631542 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:20:26.631542 ncdv-master-1 devstack@n-api.service[138681]: exception: Instance 84aa4a51-e376-4829-9a38-2ea447ebb413 could not be found. Jul 16 21:20:26.631542 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': '84aa4a51-e376-4829-9a38-2ea447ebb413'} Jul 16 21:20:26.631542 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:20:26.631542 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:20:26.631542 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:20:26.631542 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:20:26.631542 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:20:26.631542 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:20:26.631542 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:20:26.631542 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:20:26.631542 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:20:26.631542 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:20:26.631542 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 84aa4a51-e376-4829-9a38-2ea447ebb413 could not be found. Jul 16 21:20:26.631542 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:20:26.631542 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:20:26.631542 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:20:26.631542 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:20:26.631542 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:20:26.631542 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:20:26.631542 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:20:26.631542 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:20:26.631542 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:20:26.631542 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:20:26.631542 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:20:26.631542 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:20:26.631542 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:20:26.631542 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:20:26.631542 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:20:26.631542 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:20:26.631542 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 84aa4a51-e376-4829-9a38-2ea447ebb413 could not be found. Jul 16 21:20:26.631542 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:20:26.631542 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:20:26.631542 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:20:26.636403 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:20:26.636403 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:20:26.636403 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:20:26.636403 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:20:26.636403 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:20:26.636403 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 84aa4a51-e376-4829-9a38-2ea447ebb413 could not be found. Jul 16 21:20:26.636403 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:20:26.636403 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-39236364-501f-4817-ac37-4037cfa410b0 tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Returning 404 to user: Instance 84aa4a51-e376-4829-9a38-2ea447ebb413 could not be found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:20:26.636403 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-39236364-501f-4817-ac37-4037cfa410b0 tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] 10.222.0.16 "GET /compute/v2.1/servers/84aa4a51-e376-4829-9a38-2ea447ebb413" status: 404 len: 111 microversion: 2.1 time: 0.029900 Jul 16 21:20:26.636403 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1408/2862] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:20:26 2022] GET /compute/v2.1/servers/84aa4a51-e376-4829-9a38-2ea447ebb413 => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:20:26.645498 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-b1720ed1-903d-4f38-a1d4-a63bf7ca5b68 tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:20:26.658778 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-923edd81-843e-40c3-9649-f45b0ef1cb7f tempest-VolumesGetTestJSON-1435260878 tempest-VolumesGetTestJSON-1435260878-project] Calling method '.version_select at 0x7fdd9c5798b0>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:20:26.659356 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.compute.server_groups [None req-b1720ed1-903d-4f38-a1d4-a63bf7ca5b68 tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Querying cell 327e8df1-39cc-4b36-9ea6-6a9b88ed54fc(cell1) for 1 instances {{(pid=138680) _get_not_deleted /opt/stack/nova/nova/api/openstack/compute/server_groups.py:69}} Jul 16 21:20:26.660097 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b1720ed1-903d-4f38-a1d4-a63bf7ca5b68 tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:26.660542 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b1720ed1-903d-4f38-a1d4-a63bf7ca5b68 tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:26.686095 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-b1720ed1-903d-4f38-a1d4-a63bf7ca5b68 tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] 10.222.0.16 "GET /compute/v2.1/os-server-groups" status: 200 len: 187 microversion: 2.1 time: 0.045628 Jul 16 21:20:26.686682 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1455/2863] 10.222.0.16 () {58 vars in 1151 bytes} [Sat Jul 16 21:20:26 2022] GET /compute/v2.1/os-server-groups => generated 187 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:20:26.700239 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-3401925c-504b-4fb2-b851-e43f8a43ffd4 tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:20:26.715196 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.compute.server_groups [None req-3401925c-504b-4fb2-b851-e43f8a43ffd4 tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Querying cell 327e8df1-39cc-4b36-9ea6-6a9b88ed54fc(cell1) for 1 instances {{(pid=138680) _get_not_deleted /opt/stack/nova/nova/api/openstack/compute/server_groups.py:69}} Jul 16 21:20:26.716194 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3401925c-504b-4fb2-b851-e43f8a43ffd4 tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:26.716622 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-923edd81-843e-40c3-9649-f45b0ef1cb7f tempest-VolumesGetTestJSON-1435260878 tempest-VolumesGetTestJSON-1435260878-project] 10.222.0.16 "GET /compute/v2.1/os-volumes/d1414df7-4173-492a-ae5f-849dda58e82c" status: 200 len: 345 microversion: 2.1 time: 0.061560 Jul 16 21:20:26.717001 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3401925c-504b-4fb2-b851-e43f8a43ffd4 tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:26.717198 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1409/2864] 10.222.0.16 () {58 vars in 1244 bytes} [Sat Jul 16 21:20:26 2022] GET /compute/v2.1/os-volumes/d1414df7-4173-492a-ae5f-849dda58e82c => generated 345 bytes in 63 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 16 21:20:26.731993 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-3401925c-504b-4fb2-b851-e43f8a43ffd4 tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] 10.222.0.16 "GET /compute/v2.1/os-server-groups/129ae0b9-7a56-4a16-8a71-8c4b96c9564d" status: 200 len: 184 microversion: 2.1 time: 0.038279 Jul 16 21:20:26.732817 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1456/2865] 10.222.0.16 () {58 vars in 1262 bytes} [Sat Jul 16 21:20:26 2022] GET /compute/v2.1/os-server-groups/129ae0b9-7a56-4a16-8a71-8c4b96c9564d => generated 184 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:20:26.746280 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-40a45a46-da62-4a5a-baf8-cdf37c6b9f61 tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:20:26.747297 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-40a45a46-da62-4a5a-baf8-cdf37c6b9f61 tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] [instance: 84aa4a51-e376-4829-9a38-2ea447ebb413] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:20:26.754370 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_api:SELECT=98,nova_api:UPDATE=2 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:20:26.757074 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-40a45a46-da62-4a5a-baf8-cdf37c6b9f61 tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:26.757531 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-40a45a46-da62-4a5a-baf8-cdf37c6b9f61 tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:26.758311 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-40a45a46-da62-4a5a-baf8-cdf37c6b9f61 tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:26.758749 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-40a45a46-da62-4a5a-baf8-cdf37c6b9f61 tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:26.779779 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-40a45a46-da62-4a5a-baf8-cdf37c6b9f61 tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] HTTP exception thrown: Instance 84aa4a51-e376-4829-9a38-2ea447ebb413 could not be found. Jul 16 21:20:26.781953 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-40a45a46-da62-4a5a-baf8-cdf37c6b9f61 tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Request method failure captured: Jul 16 21:20:26.781953 ncdv-master-1 devstack@n-api.service[138681]: request: DELETE /compute/v2.1/servers/84aa4a51-e376-4829-9a38-2ea447ebb413 HTTP/1.1 Jul 16 21:20:26.781953 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:20:26.781953 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:20:26.781953 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:20:26.781953 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 21:20:26.781953 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:20:26.781953 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:20:26.781953 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:20:26.781953 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:20:26.781953 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0yt6As_n2V35_SjRmQhSF7a6QitkyeIu0F-jOVqebNp0HM84P8d1z7a2uuT_Zwuj8DRDlVYNphI_CfLGnrMeXPwEWkp3zvFd6xj0Kc3oSnr8T3EZu6xDDmEBMiXub1HkMFYIj11TnKacr30S5O8ds98eL7YFJyHNi9EFgWKOb6dRtEg Jul 16 21:20:26.781953 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:20:26.781953 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:20:26.781953 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:20:26.781953 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:20:26.781953 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:20:26.781953 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:20:26.781953 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: 772bd01188344e288824730238fee49b Jul 16 21:20:26.781953 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-ServerGroupTestJSON-2037884880 Jul 16 21:20:26.781953 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:20:26.781953 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:20:26.781953 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/772bd01188344e288824730238fee49b"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/772bd01188344e288824730238fee49b"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_772bd01188344e288824730238fee49b"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/772bd01188344e288824730238fee49b"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:20:26.781953 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-ServerGroupTestJSON-2037884880 Jul 16 21:20:26.781953 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: 772bd01188344e288824730238fee49b Jul 16 21:20:26.781953 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-ServerGroupTestJSON-2037884880 Jul 16 21:20:26.781953 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-ServerGroupTestJSON-2037884880-project Jul 16 21:20:26.781953 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:20:26.781953 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:20:26.781953 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: fe680ccdc60e410b812794de8aa73e0f Jul 16 21:20:26.781953 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-ServerGroupTestJSON-2037884880-project Jul 16 21:20:26.781953 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:20:26.781953 ncdv-master-1 devstack@n-api.service[138681]: exception: Instance 84aa4a51-e376-4829-9a38-2ea447ebb413 could not be found. Jul 16 21:20:26.781953 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': '84aa4a51-e376-4829-9a38-2ea447ebb413'} Jul 16 21:20:26.781953 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:20:26.781953 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:20:26.781953 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:20:26.781953 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:20:26.781953 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:20:26.781953 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:20:26.781953 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:20:26.781953 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:20:26.781953 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:20:26.781953 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:20:26.781953 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 84aa4a51-e376-4829-9a38-2ea447ebb413 could not be found. Jul 16 21:20:26.781953 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:20:26.781953 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:20:26.781953 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:20:26.781953 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:20:26.781953 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:20:26.781953 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:20:26.781953 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:20:26.781953 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:20:26.781953 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:20:26.781953 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:20:26.781953 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Jul 16 21:20:26.781953 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Jul 16 21:20:26.781953 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Jul 16 21:20:26.781953 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Jul 16 21:20:26.781953 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:20:26.781953 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:20:26.781953 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:20:26.781953 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:20:26.791685 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 84aa4a51-e376-4829-9a38-2ea447ebb413 could not be found. Jul 16 21:20:26.791685 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:20:26.791685 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:20:26.791685 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:20:26.791685 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:20:26.791685 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:20:26.791685 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:20:26.791685 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:20:26.791685 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:20:26.791685 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 84aa4a51-e376-4829-9a38-2ea447ebb413 could not be found. Jul 16 21:20:26.791685 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:20:26.796653 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-40a45a46-da62-4a5a-baf8-cdf37c6b9f61 tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Returning 404 to user: Instance 84aa4a51-e376-4829-9a38-2ea447ebb413 could not be found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:20:26.798752 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-40a45a46-da62-4a5a-baf8-cdf37c6b9f61 tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] 10.222.0.16 "DELETE /compute/v2.1/servers/84aa4a51-e376-4829-9a38-2ea447ebb413" status: 404 len: 111 microversion: 2.1 time: 0.057343 Jul 16 21:20:26.804400 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1410/2866] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:20:26 2022] DELETE /compute/v2.1/servers/84aa4a51-e376-4829-9a38-2ea447ebb413 => generated 111 bytes in 64 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:20:26.811605 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-57fe4072-3292-45a3-bd4c-e6353ddff446 tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:20:26.813068 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-57fe4072-3292-45a3-bd4c-e6353ddff446 tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] [instance: 84aa4a51-e376-4829-9a38-2ea447ebb413] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:20:26.826800 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-57fe4072-3292-45a3-bd4c-e6353ddff446 tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:26.827603 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-57fe4072-3292-45a3-bd4c-e6353ddff446 tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:26.828203 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-57fe4072-3292-45a3-bd4c-e6353ddff446 tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:26.829439 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-57fe4072-3292-45a3-bd4c-e6353ddff446 tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:26.844753 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-57fe4072-3292-45a3-bd4c-e6353ddff446 tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] HTTP exception thrown: Instance 84aa4a51-e376-4829-9a38-2ea447ebb413 could not be found. Jul 16 21:20:26.847157 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-57fe4072-3292-45a3-bd4c-e6353ddff446 tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Request method failure captured: Jul 16 21:20:26.847157 ncdv-master-1 devstack@n-api.service[138680]: request: GET /compute/v2.1/servers/84aa4a51-e376-4829-9a38-2ea447ebb413 HTTP/1.1 Jul 16 21:20:26.847157 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:20:26.847157 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:20:26.847157 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:20:26.847157 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 0 Jul 16 21:20:26.847157 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:20:26.847157 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:20:26.847157 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:20:26.847157 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:20:26.847157 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0yt6As_n2V35_SjRmQhSF7a6QitkyeIu0F-jOVqebNp0HM84P8d1z7a2uuT_Zwuj8DRDlVYNphI_CfLGnrMeXPwEWkp3zvFd6xj0Kc3oSnr8T3EZu6xDDmEBMiXub1HkMFYIj11TnKacr30S5O8ds98eL7YFJyHNi9EFgWKOb6dRtEg Jul 16 21:20:26.847157 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:20:26.847157 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:20:26.847157 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:20:26.847157 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:20:26.847157 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:20:26.847157 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:20:26.847157 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: 772bd01188344e288824730238fee49b Jul 16 21:20:26.847157 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-ServerGroupTestJSON-2037884880 Jul 16 21:20:26.847157 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:20:26.847157 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:20:26.847157 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/772bd01188344e288824730238fee49b"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/772bd01188344e288824730238fee49b"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_772bd01188344e288824730238fee49b"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/772bd01188344e288824730238fee49b"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:20:26.847157 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-ServerGroupTestJSON-2037884880 Jul 16 21:20:26.847157 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: 772bd01188344e288824730238fee49b Jul 16 21:20:26.847157 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-ServerGroupTestJSON-2037884880 Jul 16 21:20:26.847157 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-ServerGroupTestJSON-2037884880-project Jul 16 21:20:26.847157 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:20:26.847157 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:20:26.847157 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: fe680ccdc60e410b812794de8aa73e0f Jul 16 21:20:26.847157 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-ServerGroupTestJSON-2037884880-project Jul 16 21:20:26.847157 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 21:20:26.847157 ncdv-master-1 devstack@n-api.service[138680]: exception: Instance 84aa4a51-e376-4829-9a38-2ea447ebb413 could not be found. Jul 16 21:20:26.847157 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': '84aa4a51-e376-4829-9a38-2ea447ebb413'} Jul 16 21:20:26.847157 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:20:26.847157 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:20:26.847157 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:20:26.847157 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:20:26.847157 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:20:26.847157 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:20:26.847157 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:20:26.847157 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:20:26.847157 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:20:26.847157 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:20:26.847157 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 84aa4a51-e376-4829-9a38-2ea447ebb413 could not be found. Jul 16 21:20:26.847157 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:20:26.847157 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:20:26.847157 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:20:26.847157 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:20:26.847157 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:20:26.847157 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:20:26.847157 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:20:26.847157 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:20:26.847157 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:20:26.847157 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:20:26.847157 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:20:26.847157 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:20:26.847157 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:20:26.847157 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:20:26.847157 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:20:26.847157 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:20:26.847157 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 84aa4a51-e376-4829-9a38-2ea447ebb413 could not be found. Jul 16 21:20:26.847157 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:20:26.847157 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:20:26.847157 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:20:26.856113 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:20:26.856113 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:20:26.856113 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:20:26.856113 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:20:26.856113 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:20:26.856113 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 84aa4a51-e376-4829-9a38-2ea447ebb413 could not be found. Jul 16 21:20:26.856113 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:20:26.856113 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-57fe4072-3292-45a3-bd4c-e6353ddff446 tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Returning 404 to user: Instance 84aa4a51-e376-4829-9a38-2ea447ebb413 could not be found. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:20:26.856113 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-57fe4072-3292-45a3-bd4c-e6353ddff446 tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] 10.222.0.16 "GET /compute/v2.1/servers/84aa4a51-e376-4829-9a38-2ea447ebb413" status: 404 len: 111 microversion: 2.1 time: 0.049572 Jul 16 21:20:26.860918 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1457/2867] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:20:26 2022] GET /compute/v2.1/servers/84aa4a51-e376-4829-9a38-2ea447ebb413 => generated 111 bytes in 56 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:20:26.865015 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-e76f9549-caeb-4b33-9f81-7f7664fbed73 tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:20:26.873589 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-e76f9549-caeb-4b33-9f81-7f7664fbed73 tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] HTTP exception thrown: Instance group 4a10cd0b-f064-4621-961d-0ee3925657ef could not be found. Jul 16 21:20:26.882677 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-e76f9549-caeb-4b33-9f81-7f7664fbed73 tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Request method failure captured: Jul 16 21:20:26.882677 ncdv-master-1 devstack@n-api.service[138681]: request: DELETE /compute/v2.1/os-server-groups/4a10cd0b-f064-4621-961d-0ee3925657ef HTTP/1.1 Jul 16 21:20:26.882677 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:20:26.882677 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:20:26.882677 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:20:26.882677 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 21:20:26.882677 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:20:26.882677 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:20:26.882677 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:20:26.882677 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:20:26.882677 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0yt6As_n2V35_SjRmQhSF7a6QitkyeIu0F-jOVqebNp0HM84P8d1z7a2uuT_Zwuj8DRDlVYNphI_CfLGnrMeXPwEWkp3zvFd6xj0Kc3oSnr8T3EZu6xDDmEBMiXub1HkMFYIj11TnKacr30S5O8ds98eL7YFJyHNi9EFgWKOb6dRtEg Jul 16 21:20:26.882677 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:20:26.882677 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:20:26.882677 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:20:26.882677 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:20:26.882677 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:20:26.882677 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:20:26.882677 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: 772bd01188344e288824730238fee49b Jul 16 21:20:26.882677 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-ServerGroupTestJSON-2037884880 Jul 16 21:20:26.882677 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:20:26.882677 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:20:26.882677 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/772bd01188344e288824730238fee49b"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/772bd01188344e288824730238fee49b"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_772bd01188344e288824730238fee49b"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/772bd01188344e288824730238fee49b"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:20:26.882677 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-ServerGroupTestJSON-2037884880 Jul 16 21:20:26.882677 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: 772bd01188344e288824730238fee49b Jul 16 21:20:26.882677 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-ServerGroupTestJSON-2037884880 Jul 16 21:20:26.882677 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-ServerGroupTestJSON-2037884880-project Jul 16 21:20:26.882677 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:20:26.882677 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:20:26.882677 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: fe680ccdc60e410b812794de8aa73e0f Jul 16 21:20:26.882677 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-ServerGroupTestJSON-2037884880-project Jul 16 21:20:26.882677 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:20:26.882677 ncdv-master-1 devstack@n-api.service[138681]: exception: Instance group 4a10cd0b-f064-4621-961d-0ee3925657ef could not be found. Jul 16 21:20:26.882677 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': '4a10cd0b-f064-4621-961d-0ee3925657ef'} Jul 16 21:20:26.882677 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:20:26.882677 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:20:26.882677 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/server_groups.py", line 139, in delete Jul 16 21:20:26.882677 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi sg = objects.InstanceGroup.get_by_uuid(context, id) Jul 16 21:20:26.882677 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Jul 16 21:20:26.882677 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Jul 16 21:20:26.882677 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/instance_group.py", line 373, in get_by_uuid Jul 16 21:20:26.882677 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi db_group = cls._get_from_db_by_uuid(context, uuid) Jul 16 21:20:26.882677 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1010, in wrapper Jul 16 21:20:26.882677 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return fn(*args, **kwargs) Jul 16 21:20:26.882677 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/instance_group.py", line 228, in _get_from_db_by_uuid Jul 16 21:20:26.882677 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.InstanceGroupNotFound(group_uuid=uuid) Jul 16 21:20:26.882677 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.InstanceGroupNotFound: Instance group 4a10cd0b-f064-4621-961d-0ee3925657ef could not be found. Jul 16 21:20:26.882677 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:20:26.882677 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:20:26.882677 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:20:26.882677 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:20:26.882677 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:20:26.882677 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:20:26.882677 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:20:26.882677 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:20:26.882677 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:20:26.882677 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:20:26.882677 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/server_groups.py", line 141, in delete Jul 16 21:20:26.882677 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:20:26.882677 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance group 4a10cd0b-f064-4621-961d-0ee3925657ef could not be found. Jul 16 21:20:26.882677 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:20:26.882677 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:20:26.882677 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:20:26.882677 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:20:26.889606 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:20:26.889606 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:20:26.889606 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:20:26.889606 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:20:26.889606 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance group 4a10cd0b-f064-4621-961d-0ee3925657ef could not be found. Jul 16 21:20:26.889606 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:20:26.889606 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-e76f9549-caeb-4b33-9f81-7f7664fbed73 tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Returning 404 to user: Instance group 4a10cd0b-f064-4621-961d-0ee3925657ef could not be found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:20:26.889606 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-e76f9549-caeb-4b33-9f81-7f7664fbed73 tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] 10.222.0.16 "DELETE /compute/v2.1/os-server-groups/4a10cd0b-f064-4621-961d-0ee3925657ef" status: 404 len: 117 microversion: 2.1 time: 0.024648 Jul 16 21:20:26.889606 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1411/2868] 10.222.0.16 () {58 vars in 1265 bytes} [Sat Jul 16 21:20:26 2022] DELETE /compute/v2.1/os-server-groups/4a10cd0b-f064-4621-961d-0ee3925657ef => generated 117 bytes in 26 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:20:26.894097 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-a05a4a0d-aa68-4a00-8c70-c3b6739afae9 tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:20:26.905168 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-a05a4a0d-aa68-4a00-8c70-c3b6739afae9 tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] HTTP exception thrown: Instance group dd6aa1fc-935c-443a-81fb-19099bb00610 could not be found. Jul 16 21:20:26.908551 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-a05a4a0d-aa68-4a00-8c70-c3b6739afae9 tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Request method failure captured: Jul 16 21:20:26.908551 ncdv-master-1 devstack@n-api.service[138680]: request: DELETE /compute/v2.1/os-server-groups/dd6aa1fc-935c-443a-81fb-19099bb00610 HTTP/1.1 Jul 16 21:20:26.908551 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:20:26.908551 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:20:26.908551 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:20:26.908551 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 0 Jul 16 21:20:26.908551 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:20:26.908551 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:20:26.908551 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:20:26.908551 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:20:26.908551 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0yt6As_n2V35_SjRmQhSF7a6QitkyeIu0F-jOVqebNp0HM84P8d1z7a2uuT_Zwuj8DRDlVYNphI_CfLGnrMeXPwEWkp3zvFd6xj0Kc3oSnr8T3EZu6xDDmEBMiXub1HkMFYIj11TnKacr30S5O8ds98eL7YFJyHNi9EFgWKOb6dRtEg Jul 16 21:20:26.908551 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:20:26.908551 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:20:26.908551 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:20:26.908551 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:20:26.908551 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:20:26.908551 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:20:26.908551 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: 772bd01188344e288824730238fee49b Jul 16 21:20:26.908551 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-ServerGroupTestJSON-2037884880 Jul 16 21:20:26.908551 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:20:26.908551 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:20:26.908551 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/772bd01188344e288824730238fee49b"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/772bd01188344e288824730238fee49b"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_772bd01188344e288824730238fee49b"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/772bd01188344e288824730238fee49b"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:20:26.908551 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-ServerGroupTestJSON-2037884880 Jul 16 21:20:26.908551 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: 772bd01188344e288824730238fee49b Jul 16 21:20:26.908551 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-ServerGroupTestJSON-2037884880 Jul 16 21:20:26.908551 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-ServerGroupTestJSON-2037884880-project Jul 16 21:20:26.908551 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:20:26.908551 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:20:26.908551 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: fe680ccdc60e410b812794de8aa73e0f Jul 16 21:20:26.908551 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-ServerGroupTestJSON-2037884880-project Jul 16 21:20:26.908551 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 21:20:26.908551 ncdv-master-1 devstack@n-api.service[138680]: exception: Instance group dd6aa1fc-935c-443a-81fb-19099bb00610 could not be found. Jul 16 21:20:26.908551 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': 'dd6aa1fc-935c-443a-81fb-19099bb00610'} Jul 16 21:20:26.908551 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:20:26.908551 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:20:26.908551 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/server_groups.py", line 139, in delete Jul 16 21:20:26.908551 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi sg = objects.InstanceGroup.get_by_uuid(context, id) Jul 16 21:20:26.908551 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Jul 16 21:20:26.908551 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Jul 16 21:20:26.908551 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/instance_group.py", line 373, in get_by_uuid Jul 16 21:20:26.908551 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi db_group = cls._get_from_db_by_uuid(context, uuid) Jul 16 21:20:26.908551 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1010, in wrapper Jul 16 21:20:26.908551 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return fn(*args, **kwargs) Jul 16 21:20:26.908551 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/instance_group.py", line 228, in _get_from_db_by_uuid Jul 16 21:20:26.908551 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.InstanceGroupNotFound(group_uuid=uuid) Jul 16 21:20:26.908551 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.InstanceGroupNotFound: Instance group dd6aa1fc-935c-443a-81fb-19099bb00610 could not be found. Jul 16 21:20:26.908551 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:20:26.908551 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:20:26.908551 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:20:26.908551 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:20:26.908551 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:20:26.908551 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:20:26.908551 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:20:26.908551 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:20:26.908551 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:20:26.908551 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:20:26.908551 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/server_groups.py", line 141, in delete Jul 16 21:20:26.908551 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:20:26.908551 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance group dd6aa1fc-935c-443a-81fb-19099bb00610 could not be found. Jul 16 21:20:26.908551 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:20:26.908551 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:20:26.908551 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:20:26.908551 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:20:26.913643 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:20:26.913643 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:20:26.913643 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:20:26.913643 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:20:26.913643 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance group dd6aa1fc-935c-443a-81fb-19099bb00610 could not be found. Jul 16 21:20:26.913643 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:20:26.913643 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-a05a4a0d-aa68-4a00-8c70-c3b6739afae9 tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Returning 404 to user: Instance group dd6aa1fc-935c-443a-81fb-19099bb00610 could not be found. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:20:26.913643 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-a05a4a0d-aa68-4a00-8c70-c3b6739afae9 tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] 10.222.0.16 "DELETE /compute/v2.1/os-server-groups/dd6aa1fc-935c-443a-81fb-19099bb00610" status: 404 len: 117 microversion: 2.1 time: 0.018743 Jul 16 21:20:26.913643 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1458/2869] 10.222.0.16 () {58 vars in 1265 bytes} [Sat Jul 16 21:20:26 2022] DELETE /compute/v2.1/os-server-groups/dd6aa1fc-935c-443a-81fb-19099bb00610 => generated 117 bytes in 20 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:20:26.918537 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-14db1726-c494-4829-a83b-2cc8ffef7fc5 tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:20:26.931118 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-14db1726-c494-4829-a83b-2cc8ffef7fc5 tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] HTTP exception thrown: Instance group 970ebe29-6d90-49c2-8d63-f506221aecc5 could not be found. Jul 16 21:20:26.933215 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-14db1726-c494-4829-a83b-2cc8ffef7fc5 tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Request method failure captured: Jul 16 21:20:26.933215 ncdv-master-1 devstack@n-api.service[138681]: request: DELETE /compute/v2.1/os-server-groups/970ebe29-6d90-49c2-8d63-f506221aecc5 HTTP/1.1 Jul 16 21:20:26.933215 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:20:26.933215 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:20:26.933215 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:20:26.933215 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 21:20:26.933215 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:20:26.933215 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:20:26.933215 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:20:26.933215 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:20:26.933215 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0yt6As_n2V35_SjRmQhSF7a6QitkyeIu0F-jOVqebNp0HM84P8d1z7a2uuT_Zwuj8DRDlVYNphI_CfLGnrMeXPwEWkp3zvFd6xj0Kc3oSnr8T3EZu6xDDmEBMiXub1HkMFYIj11TnKacr30S5O8ds98eL7YFJyHNi9EFgWKOb6dRtEg Jul 16 21:20:26.933215 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:20:26.933215 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:20:26.933215 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:20:26.933215 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:20:26.933215 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:20:26.933215 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:20:26.933215 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: 772bd01188344e288824730238fee49b Jul 16 21:20:26.933215 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-ServerGroupTestJSON-2037884880 Jul 16 21:20:26.933215 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:20:26.933215 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:20:26.933215 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/772bd01188344e288824730238fee49b"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/772bd01188344e288824730238fee49b"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_772bd01188344e288824730238fee49b"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/772bd01188344e288824730238fee49b"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:20:26.933215 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-ServerGroupTestJSON-2037884880 Jul 16 21:20:26.933215 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: 772bd01188344e288824730238fee49b Jul 16 21:20:26.933215 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-ServerGroupTestJSON-2037884880 Jul 16 21:20:26.933215 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-ServerGroupTestJSON-2037884880-project Jul 16 21:20:26.933215 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:20:26.933215 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:20:26.933215 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: fe680ccdc60e410b812794de8aa73e0f Jul 16 21:20:26.933215 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-ServerGroupTestJSON-2037884880-project Jul 16 21:20:26.933215 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:20:26.933215 ncdv-master-1 devstack@n-api.service[138681]: exception: Instance group 970ebe29-6d90-49c2-8d63-f506221aecc5 could not be found. Jul 16 21:20:26.933215 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': '970ebe29-6d90-49c2-8d63-f506221aecc5'} Jul 16 21:20:26.933215 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:20:26.933215 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:20:26.933215 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/server_groups.py", line 139, in delete Jul 16 21:20:26.933215 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi sg = objects.InstanceGroup.get_by_uuid(context, id) Jul 16 21:20:26.933215 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Jul 16 21:20:26.933215 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Jul 16 21:20:26.933215 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/instance_group.py", line 373, in get_by_uuid Jul 16 21:20:26.933215 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi db_group = cls._get_from_db_by_uuid(context, uuid) Jul 16 21:20:26.933215 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1010, in wrapper Jul 16 21:20:26.933215 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return fn(*args, **kwargs) Jul 16 21:20:26.933215 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/instance_group.py", line 228, in _get_from_db_by_uuid Jul 16 21:20:26.933215 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.InstanceGroupNotFound(group_uuid=uuid) Jul 16 21:20:26.933215 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.InstanceGroupNotFound: Instance group 970ebe29-6d90-49c2-8d63-f506221aecc5 could not be found. Jul 16 21:20:26.933215 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:20:26.933215 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:20:26.933215 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:20:26.933215 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:20:26.933215 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:20:26.933215 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:20:26.933215 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:20:26.933215 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:20:26.933215 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:20:26.933215 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:20:26.933215 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/server_groups.py", line 141, in delete Jul 16 21:20:26.933215 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:20:26.933215 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance group 970ebe29-6d90-49c2-8d63-f506221aecc5 could not be found. Jul 16 21:20:26.933215 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:20:26.933215 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:20:26.933215 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:20:26.933215 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:20:26.937763 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:20:26.937763 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:20:26.937763 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:20:26.937763 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:20:26.937763 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance group 970ebe29-6d90-49c2-8d63-f506221aecc5 could not be found. Jul 16 21:20:26.937763 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:20:26.937763 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-14db1726-c494-4829-a83b-2cc8ffef7fc5 tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Returning 404 to user: Instance group 970ebe29-6d90-49c2-8d63-f506221aecc5 could not be found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:20:26.937763 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-14db1726-c494-4829-a83b-2cc8ffef7fc5 tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] 10.222.0.16 "DELETE /compute/v2.1/os-server-groups/970ebe29-6d90-49c2-8d63-f506221aecc5" status: 404 len: 117 microversion: 2.1 time: 0.019712 Jul 16 21:20:26.938310 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1412/2870] 10.222.0.16 () {58 vars in 1265 bytes} [Sat Jul 16 21:20:26 2022] DELETE /compute/v2.1/os-server-groups/970ebe29-6d90-49c2-8d63-f506221aecc5 => generated 117 bytes in 24 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:20:26.943254 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-0e5a35c9-f043-4bc6-9a72-ae3c53a06d4b tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:20:26.951214 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-0e5a35c9-f043-4bc6-9a72-ae3c53a06d4b tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] HTTP exception thrown: Instance group 6f360547-03a6-4526-9bde-3144198d30f4 could not be found. Jul 16 21:20:26.952754 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-0e5a35c9-f043-4bc6-9a72-ae3c53a06d4b tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Request method failure captured: Jul 16 21:20:26.952754 ncdv-master-1 devstack@n-api.service[138680]: request: DELETE /compute/v2.1/os-server-groups/6f360547-03a6-4526-9bde-3144198d30f4 HTTP/1.1 Jul 16 21:20:26.952754 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:20:26.952754 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:20:26.952754 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:20:26.952754 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 0 Jul 16 21:20:26.952754 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:20:26.952754 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:20:26.952754 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:20:26.952754 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:20:26.952754 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0yt6As_n2V35_SjRmQhSF7a6QitkyeIu0F-jOVqebNp0HM84P8d1z7a2uuT_Zwuj8DRDlVYNphI_CfLGnrMeXPwEWkp3zvFd6xj0Kc3oSnr8T3EZu6xDDmEBMiXub1HkMFYIj11TnKacr30S5O8ds98eL7YFJyHNi9EFgWKOb6dRtEg Jul 16 21:20:26.952754 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:20:26.952754 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:20:26.952754 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:20:26.952754 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:20:26.952754 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:20:26.952754 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:20:26.952754 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: 772bd01188344e288824730238fee49b Jul 16 21:20:26.952754 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-ServerGroupTestJSON-2037884880 Jul 16 21:20:26.952754 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:20:26.952754 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:20:26.952754 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/772bd01188344e288824730238fee49b"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/772bd01188344e288824730238fee49b"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_772bd01188344e288824730238fee49b"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/772bd01188344e288824730238fee49b"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:20:26.952754 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-ServerGroupTestJSON-2037884880 Jul 16 21:20:26.952754 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: 772bd01188344e288824730238fee49b Jul 16 21:20:26.952754 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-ServerGroupTestJSON-2037884880 Jul 16 21:20:26.952754 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-ServerGroupTestJSON-2037884880-project Jul 16 21:20:26.952754 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:20:26.952754 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:20:26.952754 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: fe680ccdc60e410b812794de8aa73e0f Jul 16 21:20:26.952754 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-ServerGroupTestJSON-2037884880-project Jul 16 21:20:26.952754 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 21:20:26.952754 ncdv-master-1 devstack@n-api.service[138680]: exception: Instance group 6f360547-03a6-4526-9bde-3144198d30f4 could not be found. Jul 16 21:20:26.952754 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': '6f360547-03a6-4526-9bde-3144198d30f4'} Jul 16 21:20:26.952754 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:20:26.952754 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:20:26.952754 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/server_groups.py", line 139, in delete Jul 16 21:20:26.952754 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi sg = objects.InstanceGroup.get_by_uuid(context, id) Jul 16 21:20:26.952754 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Jul 16 21:20:26.952754 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Jul 16 21:20:26.952754 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/instance_group.py", line 373, in get_by_uuid Jul 16 21:20:26.952754 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi db_group = cls._get_from_db_by_uuid(context, uuid) Jul 16 21:20:26.952754 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1010, in wrapper Jul 16 21:20:26.952754 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return fn(*args, **kwargs) Jul 16 21:20:26.952754 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/instance_group.py", line 228, in _get_from_db_by_uuid Jul 16 21:20:26.952754 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.InstanceGroupNotFound(group_uuid=uuid) Jul 16 21:20:26.952754 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.InstanceGroupNotFound: Instance group 6f360547-03a6-4526-9bde-3144198d30f4 could not be found. Jul 16 21:20:26.952754 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:20:26.952754 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:20:26.952754 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:20:26.952754 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:20:26.952754 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:20:26.952754 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:20:26.952754 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:20:26.952754 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:20:26.952754 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:20:26.952754 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:20:26.952754 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/server_groups.py", line 141, in delete Jul 16 21:20:26.952754 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:20:26.952754 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance group 6f360547-03a6-4526-9bde-3144198d30f4 could not be found. Jul 16 21:20:26.952754 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:20:26.952754 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:20:26.952754 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:20:26.952754 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:20:26.961434 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:20:26.961434 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:20:26.961434 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:20:26.961434 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:20:26.961434 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance group 6f360547-03a6-4526-9bde-3144198d30f4 could not be found. Jul 16 21:20:26.961434 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:20:26.961434 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-0e5a35c9-f043-4bc6-9a72-ae3c53a06d4b tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Returning 404 to user: Instance group 6f360547-03a6-4526-9bde-3144198d30f4 could not be found. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:20:26.961434 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-0e5a35c9-f043-4bc6-9a72-ae3c53a06d4b tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] 10.222.0.16 "DELETE /compute/v2.1/os-server-groups/6f360547-03a6-4526-9bde-3144198d30f4" status: 404 len: 117 microversion: 2.1 time: 0.015351 Jul 16 21:20:26.961434 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1459/2871] 10.222.0.16 () {58 vars in 1265 bytes} [Sat Jul 16 21:20:26 2022] DELETE /compute/v2.1/os-server-groups/6f360547-03a6-4526-9bde-3144198d30f4 => generated 117 bytes in 16 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:20:26.968357 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-a63f2206-6c8e-4682-9c1b-7696476d7945 tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:20:27.087902 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-a63f2206-6c8e-4682-9c1b-7696476d7945 tempest-ServerGroupTestJSON-2037884880 tempest-ServerGroupTestJSON-2037884880-project] 10.222.0.16 "DELETE /compute/v2.1/os-server-groups/129ae0b9-7a56-4a16-8a71-8c4b96c9564d" status: 204 len: 0 microversion: 2.1 time: 0.123675 Jul 16 21:20:27.089151 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1413/2872] 10.222.0.16 () {58 vars in 1265 bytes} [Sat Jul 16 21:20:26 2022] DELETE /compute/v2.1/os-server-groups/129ae0b9-7a56-4a16-8a71-8c4b96c9564d => generated 0 bytes in 126 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 16 21:20:27.729870 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-a78a233c-e719-4bc7-9ee6-6f8c74e5efc6 tempest-VolumesGetTestJSON-1435260878 tempest-VolumesGetTestJSON-1435260878-project] Calling method '.version_select at 0x7fdd9d03eb80>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:20:27.785117 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-a78a233c-e719-4bc7-9ee6-6f8c74e5efc6 tempest-VolumesGetTestJSON-1435260878 tempest-VolumesGetTestJSON-1435260878-project] 10.222.0.16 "GET /compute/v2.1/os-volumes/d1414df7-4173-492a-ae5f-849dda58e82c" status: 200 len: 345 microversion: 2.1 time: 0.059264 Jul 16 21:20:27.791343 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1460/2873] 10.222.0.16 () {58 vars in 1244 bytes} [Sat Jul 16 21:20:27 2022] GET /compute/v2.1/os-volumes/d1414df7-4173-492a-ae5f-849dda58e82c => generated 345 bytes in 66 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:20:27.908023 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-c913d60e-b1ef-49a1-8736-56fc3bd4d960 req-de78184e-f357-42df-ab74-fa8a5760ea85 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "6cba399c-58c3-4850-8065-0fbc3301fb2c", "tag": "607e9a0d-96c5-4d29-a42f-425502256e42"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:20:27.920765 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-c913d60e-b1ef-49a1-8736-56fc3bd4d960 req-de78184e-f357-42df-ab74-fa8a5760ea85 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:27.921528 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-c913d60e-b1ef-49a1-8736-56fc3bd4d960 req-de78184e-f357-42df-ab74-fa8a5760ea85 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:27.937730 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-c913d60e-b1ef-49a1-8736-56fc3bd4d960 req-de78184e-f357-42df-ab74-fa8a5760ea85 service nova] Creating event network-changed:607e9a0d-96c5-4d29-a42f-425502256e42 for instance 6cba399c-58c3-4850-8065-0fbc3301fb2c on nch2-master-1 Jul 16 21:20:27.950902 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-c913d60e-b1ef-49a1-8736-56fc3bd4d960 req-de78184e-f357-42df-ab74-fa8a5760ea85 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.047642 Jul 16 21:20:27.951653 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1414/2874] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:20:27 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:20:28.800405 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-dd67f9ac-0801-412a-9ff9-1879bfd28f89 tempest-VolumesGetTestJSON-1435260878 tempest-VolumesGetTestJSON-1435260878-project] Calling method '.version_select at 0x7fdd9cea6550>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:20:28.900369 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-dd67f9ac-0801-412a-9ff9-1879bfd28f89 tempest-VolumesGetTestJSON-1435260878 tempest-VolumesGetTestJSON-1435260878-project] 10.222.0.16 "GET /compute/v2.1/os-volumes/d1414df7-4173-492a-ae5f-849dda58e82c" status: 200 len: 345 microversion: 2.1 time: 0.106483 Jul 16 21:20:28.905354 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1461/2875] 10.222.0.16 () {58 vars in 1244 bytes} [Sat Jul 16 21:20:28 2022] GET /compute/v2.1/os-volumes/d1414df7-4173-492a-ae5f-849dda58e82c => generated 345 bytes in 112 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:20:29.910128 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell0:SELECT=14 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:20:29.917050 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-414a76a5-6a1f-4fcf-9b08-d8dd97d2e330 tempest-VolumesGetTestJSON-1435260878 tempest-VolumesGetTestJSON-1435260878-project] Calling method '.version_select at 0x7fdd9c51b790>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:20:30.010003 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-414a76a5-6a1f-4fcf-9b08-d8dd97d2e330 tempest-VolumesGetTestJSON-1435260878 tempest-VolumesGetTestJSON-1435260878-project] 10.222.0.16 "GET /compute/v2.1/os-volumes/d1414df7-4173-492a-ae5f-849dda58e82c" status: 200 len: 345 microversion: 2.1 time: 0.100397 Jul 16 21:20:30.015320 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1415/2876] 10.222.0.16 () {58 vars in 1244 bytes} [Sat Jul 16 21:20:29 2022] GET /compute/v2.1/os-volumes/d1414df7-4173-492a-ae5f-849dda58e82c => generated 345 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:20:31.032717 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-da0d7e65-33e5-41de-8773-ef52ba587752 tempest-VolumesGetTestJSON-1435260878 tempest-VolumesGetTestJSON-1435260878-project] Calling method '.version_select at 0x7fdd9d1a9820>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:20:31.218263 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-da0d7e65-33e5-41de-8773-ef52ba587752 tempest-VolumesGetTestJSON-1435260878 tempest-VolumesGetTestJSON-1435260878-project] 10.222.0.16 "GET /compute/v2.1/os-volumes/d1414df7-4173-492a-ae5f-849dda58e82c" status: 200 len: 345 microversion: 2.1 time: 0.200230 Jul 16 21:20:31.219912 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1462/2877] 10.222.0.16 () {58 vars in 1244 bytes} [Sat Jul 16 21:20:31 2022] GET /compute/v2.1/os-volumes/d1414df7-4173-492a-ae5f-849dda58e82c => generated 345 bytes in 202 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:20:32.233559 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-c3fe05d2-700b-43d1-a30b-6c0fa4843970 tempest-VolumesGetTestJSON-1435260878 tempest-VolumesGetTestJSON-1435260878-project] Calling method '.version_select at 0x7fdda2a16670>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:20:32.325824 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-c3fe05d2-700b-43d1-a30b-6c0fa4843970 tempest-VolumesGetTestJSON-1435260878 tempest-VolumesGetTestJSON-1435260878-project] 10.222.0.16 "GET /compute/v2.1/os-volumes/d1414df7-4173-492a-ae5f-849dda58e82c" status: 200 len: 345 microversion: 2.1 time: 0.098526 Jul 16 21:20:32.326085 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1416/2878] 10.222.0.16 () {58 vars in 1244 bytes} [Sat Jul 16 21:20:32 2022] GET /compute/v2.1/os-volumes/d1414df7-4173-492a-ae5f-849dda58e82c => generated 345 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:20:33.342131 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-8bfa3959-d1bd-4ee5-9983-d88cc50c76fa tempest-VolumesGetTestJSON-1435260878 tempest-VolumesGetTestJSON-1435260878-project] Calling method '.version_select at 0x7fdd9d03e550>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:20:33.436813 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-8bfa3959-d1bd-4ee5-9983-d88cc50c76fa tempest-VolumesGetTestJSON-1435260878 tempest-VolumesGetTestJSON-1435260878-project] 10.222.0.16 "GET /compute/v2.1/os-volumes/d1414df7-4173-492a-ae5f-849dda58e82c" status: 200 len: 345 microversion: 2.1 time: 0.100718 Jul 16 21:20:33.437589 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1463/2879] 10.222.0.16 () {58 vars in 1244 bytes} [Sat Jul 16 21:20:33 2022] GET /compute/v2.1/os-volumes/d1414df7-4173-492a-ae5f-849dda58e82c => generated 345 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:20:34.450311 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-e8a62891-4587-4bec-8cfd-135d99a47ceb tempest-VolumesGetTestJSON-1435260878 tempest-VolumesGetTestJSON-1435260878-project] Calling method '.version_select at 0x7fdd9c581040>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:20:34.549023 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-e8a62891-4587-4bec-8cfd-135d99a47ceb tempest-VolumesGetTestJSON-1435260878 tempest-VolumesGetTestJSON-1435260878-project] 10.222.0.16 "GET /compute/v2.1/os-volumes/d1414df7-4173-492a-ae5f-849dda58e82c" status: 200 len: 345 microversion: 2.1 time: 0.103452 Jul 16 21:20:34.555770 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1417/2880] 10.222.0.16 () {58 vars in 1244 bytes} [Sat Jul 16 21:20:34 2022] GET /compute/v2.1/os-volumes/d1414df7-4173-492a-ae5f-849dda58e82c => generated 345 bytes in 111 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:20:35.570500 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-055577f9-5635-4358-b8df-c9316d79e808 tempest-VolumesGetTestJSON-1435260878 tempest-VolumesGetTestJSON-1435260878-project] Calling method '.version_select at 0x7fdda2aef670>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:20:35.635130 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-055577f9-5635-4358-b8df-c9316d79e808 tempest-VolumesGetTestJSON-1435260878 tempest-VolumesGetTestJSON-1435260878-project] HTTP exception thrown: Volume d1414df7-4173-492a-ae5f-849dda58e82c could not be found. Jul 16 21:20:35.638354 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-055577f9-5635-4358-b8df-c9316d79e808 tempest-VolumesGetTestJSON-1435260878 tempest-VolumesGetTestJSON-1435260878-project] Request method failure captured: Jul 16 21:20:35.638354 ncdv-master-1 devstack@n-api.service[138680]: request: GET /compute/v2.1/os-volumes/d1414df7-4173-492a-ae5f-849dda58e82c HTTP/1.1 Jul 16 21:20:35.638354 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:20:35.638354 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:20:35.638354 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:20:35.638354 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 0 Jul 16 21:20:35.638354 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:20:35.638354 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:20:35.638354 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:20:35.638354 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:20:35.638354 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0yuVJn2T4L_gy9AeF9T5b8OUHaZU75d1PHSVeEGG_7PlVjkdgbtzAfNdxZhW_wjXUHUH00H33tnvRF67XsIbs2w5syrxgFx2tD-K0bf3qPa05N9urCamocFExYbr09YAK823X5SReAPZbdVrDCIljFSn8jWPYqsgx_eQS586ez8gqNQ Jul 16 21:20:35.638354 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:20:35.638354 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:20:35.638354 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:20:35.638354 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:20:35.638354 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:20:35.638354 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:20:35.638354 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: 28489f72346f422f9d44afcf5393acc4 Jul 16 21:20:35.638354 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-VolumesGetTestJSON-1435260878 Jul 16 21:20:35.638354 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:20:35.638354 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:20:35.638354 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "tempest-type-466225789", "name": "tempest-service-1065416560", "endpoints": []}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/28489f72346f422f9d44afcf5393acc4"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/28489f72346f422f9d44afcf5393acc4"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_28489f72346f422f9d44afcf5393acc4"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/28489f72346f422f9d44afcf5393acc4"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:20:35.638354 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-VolumesGetTestJSON-1435260878 Jul 16 21:20:35.638354 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: 28489f72346f422f9d44afcf5393acc4 Jul 16 21:20:35.638354 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-VolumesGetTestJSON-1435260878 Jul 16 21:20:35.638354 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-VolumesGetTestJSON-1435260878-project Jul 16 21:20:35.638354 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:20:35.638354 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:20:35.638354 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: 9fb89c3c454243dfbb48e3a1971c9491 Jul 16 21:20:35.638354 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-VolumesGetTestJSON-1435260878-project Jul 16 21:20:35.638354 ncdv-master-1 devstack@n-api.service[138680]: method: Jul 16 21:20:35.638354 ncdv-master-1 devstack@n-api.service[138680]: exception: Volume d1414df7-4173-492a-ae5f-849dda58e82c could not be found. Jul 16 21:20:35.638354 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': 'd1414df7-4173-492a-ae5f-849dda58e82c'} Jul 16 21:20:35.638354 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:20:35.638354 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:20:35.638354 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jul 16 21:20:35.638354 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jul 16 21:20:35.638354 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Jul 16 21:20:35.638354 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi item = cinderclient( Jul 16 21:20:35.638354 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Jul 16 21:20:35.638354 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Jul 16 21:20:35.638354 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Jul 16 21:20:35.638354 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Jul 16 21:20:35.638354 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Jul 16 21:20:35.638354 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Jul 16 21:20:35.638354 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jul 16 21:20:35.638354 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jul 16 21:20:35.638354 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jul 16 21:20:35.638354 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jul 16 21:20:35.638354 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume d1414df7-4173-492a-ae5f-849dda58e82c could not be found. (HTTP 404) (Request-ID: req-3eb256f7-90d3-4768-9133-69581f800fcf) Jul 16 21:20:35.638354 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:20:35.638354 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:20:35.638354 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:20:35.638354 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:20:35.638354 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 120, in show Jul 16 21:20:35.638354 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get(context, id) Jul 16 21:20:35.638354 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Jul 16 21:20:35.638354 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Jul 16 21:20:35.638354 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Jul 16 21:20:35.638354 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Jul 16 21:20:35.638354 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Jul 16 21:20:35.638354 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Jul 16 21:20:35.643746 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jul 16 21:20:35.643746 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jul 16 21:20:35.643746 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Jul 16 21:20:35.643746 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi item = cinderclient( Jul 16 21:20:35.643746 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Jul 16 21:20:35.643746 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Jul 16 21:20:35.643746 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Jul 16 21:20:35.643746 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Jul 16 21:20:35.643746 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Jul 16 21:20:35.643746 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Jul 16 21:20:35.643746 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jul 16 21:20:35.643746 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jul 16 21:20:35.643746 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jul 16 21:20:35.643746 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jul 16 21:20:35.643746 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume d1414df7-4173-492a-ae5f-849dda58e82c could not be found. Jul 16 21:20:35.643746 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:20:35.643746 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:20:35.643746 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:20:35.643746 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:20:35.643746 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:20:35.643746 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:20:35.643746 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:20:35.643746 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:20:35.643746 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Jul 16 21:20:35.643746 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Jul 16 21:20:35.643746 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:20:35.643746 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:20:35.643746 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 122, in show Jul 16 21:20:35.643746 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:20:35.643746 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume d1414df7-4173-492a-ae5f-849dda58e82c could not be found. Jul 16 21:20:35.643746 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:20:35.643746 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:20:35.643746 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:20:35.643746 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:20:35.643746 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:20:35.643746 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:20:35.643746 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:20:35.643746 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:20:35.643746 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume d1414df7-4173-492a-ae5f-849dda58e82c could not be found. Jul 16 21:20:35.643746 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:20:35.652139 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-055577f9-5635-4358-b8df-c9316d79e808 tempest-VolumesGetTestJSON-1435260878 tempest-VolumesGetTestJSON-1435260878-project] Returning 404 to user: Volume d1414df7-4173-492a-ae5f-849dda58e82c could not be found. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:20:35.653642 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-055577f9-5635-4358-b8df-c9316d79e808 tempest-VolumesGetTestJSON-1435260878 tempest-VolumesGetTestJSON-1435260878-project] 10.222.0.16 "GET /compute/v2.1/os-volumes/d1414df7-4173-492a-ae5f-849dda58e82c" status: 404 len: 109 microversion: 2.1 time: 0.087948 Jul 16 21:20:35.662802 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1464/2881] 10.222.0.16 () {58 vars in 1244 bytes} [Sat Jul 16 21:20:35 2022] GET /compute/v2.1/os-volumes/d1414df7-4173-492a-ae5f-849dda58e82c => generated 109 bytes in 98 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:20:37.698409 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=49 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:20:37.699306 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_api:SELECT=23 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:20:37.831230 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-800b61b1-1829-4849-b627-39ef7e597d0b tempest-QuotasTestJSON-1382213646 tempest-QuotasTestJSON-1382213646-project] Calling method '.version_select at 0x7fdd9c52fdc0>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:20:37.932677 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:UPDATE=1,nova_cell1:SELECT=22,nova_cell1:INSERT=1 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:20:38.058322 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-68bca602-ca63-48b1-b45c-e5a1134ee1d1 tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerPasswordTestJSON-server-1893190673", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452", "networks": [{"uuid": "4289ce04-b635-4798-b0d2-743f2a93f7c8"}]}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:20:38.117014 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.network.neutron [None req-68bca602-ca63-48b1-b45c-e5a1134ee1d1 tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] validate_networks() for [('4289ce04-b635-4798-b0d2-743f2a93f7c8', None, None, None, None, None)] {{(pid=138680) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 16 21:20:38.632214 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-800b61b1-1829-4849-b627-39ef7e597d0b tempest-QuotasTestJSON-1382213646 tempest-QuotasTestJSON-1382213646-project] 10.222.0.16 "GET /compute/v2.1/os-quota-sets/c7b8d5d113e44e8a8e8e024c8bde9c1e/defaults" status: 200 len: 371 microversion: 2.1 time: 1.117453 Jul 16 21:20:38.633666 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1418/2882] 10.222.0.16 () {58 vars in 1268 bytes} [Sat Jul 16 21:20:37 2022] GET /compute/v2.1/os-quota-sets/c7b8d5d113e44e8a8e8e024c8bde9c1e/defaults => generated 371 bytes in 1120 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:20:38.645166 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-8f022607-b550-42d1-bff3-7ccdd3ad6373 tempest-QuotasTestJSON-1382213646 tempest-QuotasTestJSON-1382213646-project] Calling method '.version_select at 0x7fdd9d0b5940>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:20:39.265349 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-68bca602-ca63-48b1-b45c-e5a1134ee1d1 tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] Flavor limits 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 16 21:20:39.266931 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-68bca602-ca63-48b1-b45c-e5a1134ee1d1 tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] Image limits 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 16 21:20:39.267754 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-68bca602-ca63-48b1-b45c-e5a1134ee1d1 tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] Flavor pref 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 16 21:20:39.272165 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-68bca602-ca63-48b1-b45c-e5a1134ee1d1 tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] Image pref 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 16 21:20:39.272165 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-68bca602-ca63-48b1-b45c-e5a1134ee1d1 tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 16 21:20:39.288346 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-68bca602-ca63-48b1-b45c-e5a1134ee1d1 tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:39.289311 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-68bca602-ca63-48b1-b45c-e5a1134ee1d1 tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:39.290883 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-68bca602-ca63-48b1-b45c-e5a1134ee1d1 tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:39.295041 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-68bca602-ca63-48b1-b45c-e5a1134ee1d1 tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:39.324697 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-68bca602-ca63-48b1-b45c-e5a1134ee1d1 tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] Getting quotas for project f163566907554620bc94de3b960a4f13. Resources: {'ram', 'instances', 'cores'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:20:39.337602 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-68bca602-ca63-48b1-b45c-e5a1134ee1d1 tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] Getting quotas for user d3bb6f6d546c4bd090e72c149afe5660 and project f163566907554620bc94de3b960a4f13. Resources: {'ram', 'instances', 'cores'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:20:39.359463 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-68bca602-ca63-48b1-b45c-e5a1134ee1d1 tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] Going to run 1 instances... {{(pid=138680) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 16 21:20:39.368229 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-68bca602-ca63-48b1-b45c-e5a1134ee1d1 tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] [instance: fa4d69b3-8beb-4dfa-b655-c5c26dcac34f] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138680) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 16 21:20:39.368592 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-8f022607-b550-42d1-bff3-7ccdd3ad6373 tempest-QuotasTestJSON-1382213646 tempest-QuotasTestJSON-1382213646-project] 10.222.0.16 "GET /compute/v2.1/os-quota-sets/c7b8d5d113e44e8a8e8e024c8bde9c1e" status: 200 len: 371 microversion: 2.1 time: 0.728814 Jul 16 21:20:39.369200 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1419/2883] 10.222.0.16 () {58 vars in 1241 bytes} [Sat Jul 16 21:20:38 2022] GET /compute/v2.1/os-quota-sets/c7b8d5d113e44e8a8e8e024c8bde9c1e => generated 371 bytes in 730 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 16 21:20:39.390041 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-afd518bb-b2cb-4a77-bd30-c42e05c29e87 tempest-QuotasTestJSON-1382213646 tempest-QuotasTestJSON-1382213646-project] Calling method '.version_select at 0x7fdda0401c10>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:20:39.545308 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-68bca602-ca63-48b1-b45c-e5a1134ee1d1 tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 1.857023 Jul 16 21:20:39.553795 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1465/2884] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 21:20:37 2022] POST /compute/v2.1/servers => generated 380 bytes in 1866 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 16 21:20:39.564191 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-ac22eb02-2cba-4286-8fcf-039b3ebbff3c tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:20:39.564987 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-ac22eb02-2cba-4286-8fcf-039b3ebbff3c tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] [instance: fa4d69b3-8beb-4dfa-b655-c5c26dcac34f] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:20:39.775486 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-ac22eb02-2cba-4286-8fcf-039b3ebbff3c tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd3bb6f6d546c4bd090e72c149afe5660', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f163566907554620bc94de3b960a4f13', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:20:39.784383 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-ac22eb02-2cba-4286-8fcf-039b3ebbff3c tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] 10.222.0.16 "GET /compute/v2.1/servers/fa4d69b3-8beb-4dfa-b655-c5c26dcac34f" status: 200 len: 1216 microversion: 2.1 time: 0.224548 Jul 16 21:20:39.797121 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1466/2885] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:20:39 2022] GET /compute/v2.1/servers/fa4d69b3-8beb-4dfa-b655-c5c26dcac34f => generated 1216 bytes in 238 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:20:39.876224 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-afd518bb-b2cb-4a77-bd30-c42e05c29e87 tempest-QuotasTestJSON-1382213646 tempest-QuotasTestJSON-1382213646-project] 10.222.0.16 "GET /compute/v2.1/os-quota-sets/c7b8d5d113e44e8a8e8e024c8bde9c1e/defaults" status: 200 len: 371 microversion: 2.1 time: 0.496064 Jul 16 21:20:39.877394 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1420/2886] 10.222.0.16 () {58 vars in 1268 bytes} [Sat Jul 16 21:20:39 2022] GET /compute/v2.1/os-quota-sets/c7b8d5d113e44e8a8e8e024c8bde9c1e/defaults => generated 371 bytes in 498 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:20:39.893202 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-d867df3d-8fd2-4da5-a152-5a78311ae4bd tempest-QuotasTestJSON-1382213646 tempest-QuotasTestJSON-1382213646-project] Calling method '.version_select at 0x7fdd9d4d9e50>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:20:40.211410 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-d867df3d-8fd2-4da5-a152-5a78311ae4bd tempest-QuotasTestJSON-1382213646 tempest-QuotasTestJSON-1382213646-project] 10.222.0.16 "GET /compute/v2.1/os-quota-sets/c7b8d5d113e44e8a8e8e024c8bde9c1e" status: 200 len: 371 microversion: 2.1 time: 0.324271 Jul 16 21:20:40.212833 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1467/2887] 10.222.0.16 () {58 vars in 1241 bytes} [Sat Jul 16 21:20:39 2022] GET /compute/v2.1/os-quota-sets/c7b8d5d113e44e8a8e8e024c8bde9c1e => generated 371 bytes in 326 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:20:40.223531 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-a989f0bf-ed8d-4eea-87ac-7282369bdcd0 tempest-QuotasTestJSON-1382213646 tempest-QuotasTestJSON-1382213646-project] Calling method '.version_select at 0x7fdd9c5813a0>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:20:40.407235 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-a989f0bf-ed8d-4eea-87ac-7282369bdcd0 tempest-QuotasTestJSON-1382213646 tempest-QuotasTestJSON-1382213646-project] 10.222.0.16 "GET /compute/v2.1/os-quota-sets/c7b8d5d113e44e8a8e8e024c8bde9c1e?user_id=b99d79db9d374453803e7715cc6a8599" status: 200 len: 371 microversion: 2.1 time: 0.187991 Jul 16 21:20:40.416085 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1421/2888] 10.222.0.16 () {58 vars in 1322 bytes} [Sat Jul 16 21:20:40 2022] GET /compute/v2.1/os-quota-sets/c7b8d5d113e44e8a8e8e024c8bde9c1e?user_id=b99d79db9d374453803e7715cc6a8599 => generated 371 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:20:40.803653 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-9197e625-a9d8-4ef2-8e91-28838ee0df81 tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:20:40.805447 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-9197e625-a9d8-4ef2-8e91-28838ee0df81 tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] [instance: fa4d69b3-8beb-4dfa-b655-c5c26dcac34f] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:20:40.953481 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-9197e625-a9d8-4ef2-8e91-28838ee0df81 tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd3bb6f6d546c4bd090e72c149afe5660', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f163566907554620bc94de3b960a4f13', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:20:40.962194 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-9197e625-a9d8-4ef2-8e91-28838ee0df81 tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] 10.222.0.16 "GET /compute/v2.1/servers/fa4d69b3-8beb-4dfa-b655-c5c26dcac34f" status: 200 len: 1216 microversion: 2.1 time: 0.162570 Jul 16 21:20:40.963275 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1468/2889] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:20:40 2022] GET /compute/v2.1/servers/fa4d69b3-8beb-4dfa-b655-c5c26dcac34f => generated 1216 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:20:41.980654 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-bc3d1dd3-803d-4049-8681-76aeafbb51ef tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:20:41.982085 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-bc3d1dd3-803d-4049-8681-76aeafbb51ef tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] [instance: fa4d69b3-8beb-4dfa-b655-c5c26dcac34f] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:20:42.011974 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-bc3d1dd3-803d-4049-8681-76aeafbb51ef tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:42.012626 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-bc3d1dd3-803d-4049-8681-76aeafbb51ef tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:42.016006 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-bc3d1dd3-803d-4049-8681-76aeafbb51ef tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:42.017175 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-bc3d1dd3-803d-4049-8681-76aeafbb51ef tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:42.267595 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-bc3d1dd3-803d-4049-8681-76aeafbb51ef tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd3bb6f6d546c4bd090e72c149afe5660', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f163566907554620bc94de3b960a4f13', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:20:42.278818 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-bc3d1dd3-803d-4049-8681-76aeafbb51ef tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] 10.222.0.16 "GET /compute/v2.1/servers/fa4d69b3-8beb-4dfa-b655-c5c26dcac34f" status: 200 len: 1212 microversion: 2.1 time: 0.302291 Jul 16 21:20:42.293593 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1422/2890] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:20:41 2022] GET /compute/v2.1/servers/fa4d69b3-8beb-4dfa-b655-c5c26dcac34f => generated 1212 bytes in 318 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:20:43.297413 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-93d62446-adf8-4cef-93af-aa1b4a177cb7 tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:20:43.297966 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-93d62446-adf8-4cef-93af-aa1b4a177cb7 tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] [instance: fa4d69b3-8beb-4dfa-b655-c5c26dcac34f] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:20:43.306746 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-93d62446-adf8-4cef-93af-aa1b4a177cb7 tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:43.307229 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-93d62446-adf8-4cef-93af-aa1b4a177cb7 tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:43.308172 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-93d62446-adf8-4cef-93af-aa1b4a177cb7 tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:43.308618 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-93d62446-adf8-4cef-93af-aa1b4a177cb7 tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:43.405467 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-93d62446-adf8-4cef-93af-aa1b4a177cb7 tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd3bb6f6d546c4bd090e72c149afe5660', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f163566907554620bc94de3b960a4f13', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:20:43.418756 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-93d62446-adf8-4cef-93af-aa1b4a177cb7 tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] 10.222.0.16 "GET /compute/v2.1/servers/fa4d69b3-8beb-4dfa-b655-c5c26dcac34f" status: 200 len: 1286 microversion: 2.1 time: 0.124973 Jul 16 21:20:43.425350 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1469/2891] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:20:43 2022] GET /compute/v2.1/servers/fa4d69b3-8beb-4dfa-b655-c5c26dcac34f => generated 1286 bytes in 133 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:20:44.438317 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-c616f716-579f-4a2e-ab93-481858c46b05 tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:20:44.439663 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-c616f716-579f-4a2e-ab93-481858c46b05 tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] [instance: fa4d69b3-8beb-4dfa-b655-c5c26dcac34f] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:20:44.448521 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c616f716-579f-4a2e-ab93-481858c46b05 tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:44.449354 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c616f716-579f-4a2e-ab93-481858c46b05 tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:44.450405 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c616f716-579f-4a2e-ab93-481858c46b05 tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:44.450999 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c616f716-579f-4a2e-ab93-481858c46b05 tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:44.913360 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-c616f716-579f-4a2e-ab93-481858c46b05 tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd3bb6f6d546c4bd090e72c149afe5660', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f163566907554620bc94de3b960a4f13', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:20:44.921700 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-c616f716-579f-4a2e-ab93-481858c46b05 tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] 10.222.0.16 "GET /compute/v2.1/servers/fa4d69b3-8beb-4dfa-b655-c5c26dcac34f" status: 200 len: 1316 microversion: 2.1 time: 0.487715 Jul 16 21:20:44.933126 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1423/2892] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:20:44 2022] GET /compute/v2.1/servers/fa4d69b3-8beb-4dfa-b655-c5c26dcac34f => generated 1316 bytes in 497 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:20:45.950593 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-98072d18-2a52-46f6-8224-ff0e21032642 tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:20:45.951878 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-98072d18-2a52-46f6-8224-ff0e21032642 tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] [instance: fa4d69b3-8beb-4dfa-b655-c5c26dcac34f] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:20:45.963559 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-98072d18-2a52-46f6-8224-ff0e21032642 tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:45.964278 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-98072d18-2a52-46f6-8224-ff0e21032642 tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:45.965477 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-98072d18-2a52-46f6-8224-ff0e21032642 tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:45.966398 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-98072d18-2a52-46f6-8224-ff0e21032642 tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:46.254958 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-98072d18-2a52-46f6-8224-ff0e21032642 tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd3bb6f6d546c4bd090e72c149afe5660', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f163566907554620bc94de3b960a4f13', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:20:46.275797 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-98072d18-2a52-46f6-8224-ff0e21032642 tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] 10.222.0.16 "GET /compute/v2.1/servers/fa4d69b3-8beb-4dfa-b655-c5c26dcac34f" status: 200 len: 1316 microversion: 2.1 time: 0.331425 Jul 16 21:20:46.276400 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1470/2893] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:20:45 2022] GET /compute/v2.1/servers/fa4d69b3-8beb-4dfa-b655-c5c26dcac34f => generated 1316 bytes in 333 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:20:46.794460 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-24b5d06d-e91f-4d0e-849c-1023aa688388 req-3beea054-8a4e-4001-be2b-b6ed39ffca2d service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "fa4d69b3-8beb-4dfa-b655-c5c26dcac34f", "tag": "dc02eaab-b399-4f24-92e0-d5cefb6ea105"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:20:46.824336 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-24b5d06d-e91f-4d0e-849c-1023aa688388 req-3beea054-8a4e-4001-be2b-b6ed39ffca2d service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:46.825059 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-24b5d06d-e91f-4d0e-849c-1023aa688388 req-3beea054-8a4e-4001-be2b-b6ed39ffca2d service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:46.843247 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-24b5d06d-e91f-4d0e-849c-1023aa688388 req-3beea054-8a4e-4001-be2b-b6ed39ffca2d service nova] Creating event network-changed:dc02eaab-b399-4f24-92e0-d5cefb6ea105 for instance fa4d69b3-8beb-4dfa-b655-c5c26dcac34f on nch1-master-1 Jul 16 21:20:46.860434 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-24b5d06d-e91f-4d0e-849c-1023aa688388 req-3beea054-8a4e-4001-be2b-b6ed39ffca2d service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.069774 Jul 16 21:20:46.861552 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1424/2894] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:20:46 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 71 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:20:47.293216 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-3eaa939f-9bed-41b9-b416-c955949e12b8 tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:20:47.294434 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-3eaa939f-9bed-41b9-b416-c955949e12b8 tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] [instance: fa4d69b3-8beb-4dfa-b655-c5c26dcac34f] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:20:47.302330 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3eaa939f-9bed-41b9-b416-c955949e12b8 tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:47.302793 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3eaa939f-9bed-41b9-b416-c955949e12b8 tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:47.303612 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3eaa939f-9bed-41b9-b416-c955949e12b8 tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:47.304052 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3eaa939f-9bed-41b9-b416-c955949e12b8 tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:47.546688 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-3eaa939f-9bed-41b9-b416-c955949e12b8 tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd3bb6f6d546c4bd090e72c149afe5660', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f163566907554620bc94de3b960a4f13', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:20:47.557876 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-3eaa939f-9bed-41b9-b416-c955949e12b8 tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] 10.222.0.16 "GET /compute/v2.1/servers/fa4d69b3-8beb-4dfa-b655-c5c26dcac34f" status: 200 len: 1316 microversion: 2.1 time: 0.268452 Jul 16 21:20:47.572871 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1471/2895] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:20:47 2022] GET /compute/v2.1/servers/fa4d69b3-8beb-4dfa-b655-c5c26dcac34f => generated 1316 bytes in 284 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:20:48.581394 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-16e9eed5-c81d-421a-8585-0ec26dcdeafa tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:20:48.582555 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-16e9eed5-c81d-421a-8585-0ec26dcdeafa tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] [instance: fa4d69b3-8beb-4dfa-b655-c5c26dcac34f] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:20:48.595053 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-16e9eed5-c81d-421a-8585-0ec26dcdeafa tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:48.598683 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-16e9eed5-c81d-421a-8585-0ec26dcdeafa tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:48.600648 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-16e9eed5-c81d-421a-8585-0ec26dcdeafa tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:48.601365 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-16e9eed5-c81d-421a-8585-0ec26dcdeafa tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:48.813704 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-16e9eed5-c81d-421a-8585-0ec26dcdeafa tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd3bb6f6d546c4bd090e72c149afe5660', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f163566907554620bc94de3b960a4f13', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:20:48.830295 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-16e9eed5-c81d-421a-8585-0ec26dcdeafa tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] 10.222.0.16 "GET /compute/v2.1/servers/fa4d69b3-8beb-4dfa-b655-c5c26dcac34f" status: 200 len: 1316 microversion: 2.1 time: 0.253477 Jul 16 21:20:48.831217 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1425/2896] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:20:48 2022] GET /compute/v2.1/servers/fa4d69b3-8beb-4dfa-b655-c5c26dcac34f => generated 1316 bytes in 255 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:20:49.850410 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-d6caa21f-0ae6-4ea1-8a44-28c150af7814 tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:20:49.851064 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-d6caa21f-0ae6-4ea1-8a44-28c150af7814 tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] [instance: fa4d69b3-8beb-4dfa-b655-c5c26dcac34f] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:20:49.855317 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell0:SELECT=2 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:20:49.861175 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d6caa21f-0ae6-4ea1-8a44-28c150af7814 tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:49.862651 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d6caa21f-0ae6-4ea1-8a44-28c150af7814 tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:49.863699 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d6caa21f-0ae6-4ea1-8a44-28c150af7814 tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:49.864319 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d6caa21f-0ae6-4ea1-8a44-28c150af7814 tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:50.051893 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-d6caa21f-0ae6-4ea1-8a44-28c150af7814 tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd3bb6f6d546c4bd090e72c149afe5660', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f163566907554620bc94de3b960a4f13', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:20:50.059508 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-d6caa21f-0ae6-4ea1-8a44-28c150af7814 tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] 10.222.0.16 "GET /compute/v2.1/servers/fa4d69b3-8beb-4dfa-b655-c5c26dcac34f" status: 200 len: 1316 microversion: 2.1 time: 0.214856 Jul 16 21:20:50.060104 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1472/2897] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:20:49 2022] GET /compute/v2.1/servers/fa4d69b3-8beb-4dfa-b655-c5c26dcac34f => generated 1316 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:20:51.080320 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-1bbfde66-0fa1-492b-ac81-9a0805e66e19 tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:20:51.083644 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-1bbfde66-0fa1-492b-ac81-9a0805e66e19 tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] [instance: fa4d69b3-8beb-4dfa-b655-c5c26dcac34f] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:20:51.088160 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell0:SELECT=6 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:20:51.121939 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1bbfde66-0fa1-492b-ac81-9a0805e66e19 tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:51.124022 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1bbfde66-0fa1-492b-ac81-9a0805e66e19 tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:51.126370 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1bbfde66-0fa1-492b-ac81-9a0805e66e19 tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:51.128026 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1bbfde66-0fa1-492b-ac81-9a0805e66e19 tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:51.453933 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-1bbfde66-0fa1-492b-ac81-9a0805e66e19 tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd3bb6f6d546c4bd090e72c149afe5660', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f163566907554620bc94de3b960a4f13', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:20:51.461723 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-1bbfde66-0fa1-492b-ac81-9a0805e66e19 tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] 10.222.0.16 "GET /compute/v2.1/servers/fa4d69b3-8beb-4dfa-b655-c5c26dcac34f" status: 200 len: 1316 microversion: 2.1 time: 0.387863 Jul 16 21:20:51.471182 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1426/2898] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:20:51 2022] GET /compute/v2.1/servers/fa4d69b3-8beb-4dfa-b655-c5c26dcac34f => generated 1316 bytes in 398 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:20:52.481706 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-d12eb8fc-2870-43a2-bb06-7ddea563357a tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:20:52.482276 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-d12eb8fc-2870-43a2-bb06-7ddea563357a tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] [instance: fa4d69b3-8beb-4dfa-b655-c5c26dcac34f] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:20:52.486827 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell0:SELECT=12 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:20:52.493452 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d12eb8fc-2870-43a2-bb06-7ddea563357a tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:52.494183 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d12eb8fc-2870-43a2-bb06-7ddea563357a tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:52.495034 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d12eb8fc-2870-43a2-bb06-7ddea563357a tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:52.495559 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d12eb8fc-2870-43a2-bb06-7ddea563357a tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:53.024298 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-d12eb8fc-2870-43a2-bb06-7ddea563357a tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd3bb6f6d546c4bd090e72c149afe5660', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f163566907554620bc94de3b960a4f13', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:20:53.045224 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-d12eb8fc-2870-43a2-bb06-7ddea563357a tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] 10.222.0.16 "GET /compute/v2.1/servers/fa4d69b3-8beb-4dfa-b655-c5c26dcac34f" status: 200 len: 1316 microversion: 2.1 time: 0.569842 Jul 16 21:20:53.047568 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1473/2899] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:20:52 2022] GET /compute/v2.1/servers/fa4d69b3-8beb-4dfa-b655-c5c26dcac34f => generated 1316 bytes in 573 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:20:54.068812 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-65c4ae72-a7ae-41be-9a87-51b8670e7b5f tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:20:54.070312 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-65c4ae72-a7ae-41be-9a87-51b8670e7b5f tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] [instance: fa4d69b3-8beb-4dfa-b655-c5c26dcac34f] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:20:54.092563 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-65c4ae72-a7ae-41be-9a87-51b8670e7b5f tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:54.092563 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-65c4ae72-a7ae-41be-9a87-51b8670e7b5f tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:54.092563 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-65c4ae72-a7ae-41be-9a87-51b8670e7b5f tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:54.092563 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-65c4ae72-a7ae-41be-9a87-51b8670e7b5f tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:54.325856 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-5274d7ef-7832-4773-8367-9224659a358e tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:20:54.329567 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-5274d7ef-7832-4773-8367-9224659a358e tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] Policy check for os_compute_api:servers:allow_all_filters failed with credentials {'is_admin': False, 'user_id': '1f72b1e0b4124480bfd53b5c22a343e0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0be222897b3a4864a39e9fc7d481340a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:20:54.331457 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-5274d7ef-7832-4773-8367-9224659a358e tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] Searching by: {'deleted': False, 'project_id': '0be222897b3a4864a39e9fc7d481340a'} {{(pid=138680) get_all /opt/stack/nova/nova/compute/api.py:2962}} Jul 16 21:20:54.356669 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5274d7ef-7832-4773-8367-9224659a358e tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:54.357126 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5274d7ef-7832-4773-8367-9224659a358e tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:54.375347 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5274d7ef-7832-4773-8367-9224659a358e tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:54.375945 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5274d7ef-7832-4773-8367-9224659a358e tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.018s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:54.444060 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.multi_cell_list [None req-5274d7ef-7832-4773-8367-9224659a358e tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] Listed batch of 1 results from cell out of 1000 limit. Returned 1 total so far. {{(pid=138680) do_query /opt/stack/nova/nova/compute/multi_cell_list.py:373}} Jul 16 21:20:54.524874 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-65c4ae72-a7ae-41be-9a87-51b8670e7b5f tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd3bb6f6d546c4bd090e72c149afe5660', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f163566907554620bc94de3b960a4f13', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:20:54.537400 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-65c4ae72-a7ae-41be-9a87-51b8670e7b5f tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] 10.222.0.16 "GET /compute/v2.1/servers/fa4d69b3-8beb-4dfa-b655-c5c26dcac34f" status: 200 len: 1316 microversion: 2.1 time: 0.473107 Jul 16 21:20:54.540403 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1427/2900] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:20:54 2022] GET /compute/v2.1/servers/fa4d69b3-8beb-4dfa-b655-c5c26dcac34f => generated 1316 bytes in 477 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:20:54.557873 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-5274d7ef-7832-4773-8367-9224659a358e tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] 10.222.0.16 "GET /compute/v2.1/servers" status: 200 len: 339 microversion: 2.1 time: 0.207782 Jul 16 21:20:54.567086 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1474/2901] 10.222.0.16 () {58 vars in 1124 bytes} [Sat Jul 16 21:20:54 2022] GET /compute/v2.1/servers => generated 339 bytes in 246 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:20:54.579331 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-2d79e142-b889-4cd3-a522-98741a25f9f7 tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:20:54.583756 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-2d79e142-b889-4cd3-a522-98741a25f9f7 tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] Policy check for os_compute_api:servers:allow_all_filters failed with credentials {'is_admin': False, 'user_id': '1f72b1e0b4124480bfd53b5c22a343e0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0be222897b3a4864a39e9fc7d481340a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:20:54.587251 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-2d79e142-b889-4cd3-a522-98741a25f9f7 tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] Searching by: {'deleted': False, 'project_id': '0be222897b3a4864a39e9fc7d481340a'} {{(pid=138681) get_all /opt/stack/nova/nova/compute/api.py:2962}} Jul 16 21:20:54.599519 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2d79e142-b889-4cd3-a522-98741a25f9f7 tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:54.602139 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2d79e142-b889-4cd3-a522-98741a25f9f7 tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:54.603258 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2d79e142-b889-4cd3-a522-98741a25f9f7 tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:54.604496 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2d79e142-b889-4cd3-a522-98741a25f9f7 tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:54.673161 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.multi_cell_list [None req-2d79e142-b889-4cd3-a522-98741a25f9f7 tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] Listed batch of 1 results from cell out of 1000 limit. Returned 1 total so far. {{(pid=138681) do_query /opt/stack/nova/nova/compute/multi_cell_list.py:373}} Jul 16 21:20:54.770114 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-2d79e142-b889-4cd3-a522-98741a25f9f7 tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1f72b1e0b4124480bfd53b5c22a343e0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0be222897b3a4864a39e9fc7d481340a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:20:54.790151 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2d79e142-b889-4cd3-a522-98741a25f9f7 tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:54.793259 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2d79e142-b889-4cd3-a522-98741a25f9f7 tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:55.205488 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-2d79e142-b889-4cd3-a522-98741a25f9f7 tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] 10.222.0.16 "GET /compute/v2.1/servers/detail" status: 200 len: 1710 microversion: 2.1 time: 0.629993 Jul 16 21:20:55.215411 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1428/2902] 10.222.0.16 () {58 vars in 1145 bytes} [Sat Jul 16 21:20:54 2022] GET /compute/v2.1/servers/detail => generated 1710 bytes in 640 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:20:55.228303 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-ccb5ca6e-06e1-410c-bfda-fb0a087fc96b tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:20:55.245475 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-ccb5ca6e-06e1-410c-bfda-fb0a087fc96b tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] 10.222.0.16 "GET /compute/v2.1/flavors/452" status: 200 len: 370 microversion: 2.1 time: 0.020927 Jul 16 21:20:55.246468 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1475/2903] 10.222.0.16 () {58 vars in 1136 bytes} [Sat Jul 16 21:20:55 2022] GET /compute/v2.1/flavors/452 => generated 370 bytes in 24 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:20:55.556319 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-eaf30211-b192-48cf-91d5-34c43daec047 tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:20:55.557049 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-eaf30211-b192-48cf-91d5-34c43daec047 tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] [instance: fa4d69b3-8beb-4dfa-b655-c5c26dcac34f] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:20:55.566115 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-eaf30211-b192-48cf-91d5-34c43daec047 tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:55.566708 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-eaf30211-b192-48cf-91d5-34c43daec047 tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:55.575059 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-eaf30211-b192-48cf-91d5-34c43daec047 tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:55.575196 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-eaf30211-b192-48cf-91d5-34c43daec047 tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:55.639846 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-9b5d3110-4b6c-4dd6-9e87-466cc073290e tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:20:55.640437 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-9b5d3110-4b6c-4dd6-9e87-466cc073290e tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] [instance: 0a9564cb-c375-47e9-bfe6-e68d4700442b] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:20:55.702381 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9b5d3110-4b6c-4dd6-9e87-466cc073290e tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:55.703096 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9b5d3110-4b6c-4dd6-9e87-466cc073290e tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.022s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:55.703966 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9b5d3110-4b6c-4dd6-9e87-466cc073290e tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:55.704416 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9b5d3110-4b6c-4dd6-9e87-466cc073290e tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:55.918879 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-9b5d3110-4b6c-4dd6-9e87-466cc073290e tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] [instance: 0a9564cb-c375-47e9-bfe6-e68d4700442b] Going to try to terminate instance {{(pid=138680) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 16 21:20:56.010340 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-eaf30211-b192-48cf-91d5-34c43daec047 tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd3bb6f6d546c4bd090e72c149afe5660', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f163566907554620bc94de3b960a4f13', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:20:56.022425 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-eaf30211-b192-48cf-91d5-34c43daec047 tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] 10.222.0.16 "GET /compute/v2.1/servers/fa4d69b3-8beb-4dfa-b655-c5c26dcac34f" status: 200 len: 1316 microversion: 2.1 time: 0.467322 Jul 16 21:20:56.027222 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1429/2904] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:20:55 2022] GET /compute/v2.1/servers/fa4d69b3-8beb-4dfa-b655-c5c26dcac34f => generated 1316 bytes in 476 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:20:56.526776 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-9b5d3110-4b6c-4dd6-9e87-466cc073290e tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] 10.222.0.16 "DELETE /compute/v2.1/servers/0a9564cb-c375-47e9-bfe6-e68d4700442b" status: 204 len: 0 microversion: 2.1 time: 0.890512 Jul 16 21:20:56.531344 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1476/2905] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:20:55 2022] DELETE /compute/v2.1/servers/0a9564cb-c375-47e9-bfe6-e68d4700442b => generated 0 bytes in 893 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 16 21:20:56.539540 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-0cdeb21e-678e-4c8b-ac1c-8c8e20ae12ac tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:20:56.540406 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-0cdeb21e-678e-4c8b-ac1c-8c8e20ae12ac tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] [instance: 0a9564cb-c375-47e9-bfe6-e68d4700442b] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:20:56.567777 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0cdeb21e-678e-4c8b-ac1c-8c8e20ae12ac tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:56.567777 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0cdeb21e-678e-4c8b-ac1c-8c8e20ae12ac tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:56.568541 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0cdeb21e-678e-4c8b-ac1c-8c8e20ae12ac tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:56.570800 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0cdeb21e-678e-4c8b-ac1c-8c8e20ae12ac tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:56.965962 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-e4d9ee06-9a57-44ba-95dc-12c558d0d526 req-8745a3bf-ecf1-45b2-ac48-ff28f263ad75 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "fa4d69b3-8beb-4dfa-b655-c5c26dcac34f", "name": "network-vif-plugged", "status": "completed", "tag": "dc02eaab-b399-4f24-92e0-d5cefb6ea105"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:20:56.986659 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-e4d9ee06-9a57-44ba-95dc-12c558d0d526 req-8745a3bf-ecf1-45b2-ac48-ff28f263ad75 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:56.988388 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-e4d9ee06-9a57-44ba-95dc-12c558d0d526 req-8745a3bf-ecf1-45b2-ac48-ff28f263ad75 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:57.011420 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-e4d9ee06-9a57-44ba-95dc-12c558d0d526 req-8745a3bf-ecf1-45b2-ac48-ff28f263ad75 service nova] Creating event network-vif-plugged:dc02eaab-b399-4f24-92e0-d5cefb6ea105 for instance fa4d69b3-8beb-4dfa-b655-c5c26dcac34f on nch1-master-1 Jul 16 21:20:57.020646 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-0cdeb21e-678e-4c8b-ac1c-8c8e20ae12ac tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1f72b1e0b4124480bfd53b5c22a343e0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0be222897b3a4864a39e9fc7d481340a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:20:57.031252 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-e4d9ee06-9a57-44ba-95dc-12c558d0d526 req-8745a3bf-ecf1-45b2-ac48-ff28f263ad75 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.069466 Jul 16 21:20:57.035186 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1477/2906] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:20:56 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 75 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 16 21:20:57.037136 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-0cdeb21e-678e-4c8b-ac1c-8c8e20ae12ac tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] 10.222.0.16 "GET /compute/v2.1/servers/0a9564cb-c375-47e9-bfe6-e68d4700442b" status: 200 len: 1713 microversion: 2.1 time: 0.503560 Jul 16 21:20:57.038239 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1430/2907] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:20:56 2022] GET /compute/v2.1/servers/0a9564cb-c375-47e9-bfe6-e68d4700442b => generated 1713 bytes in 506 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:20:57.059651 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-a0d4882d-8b2d-4d45-9851-5c9de7e7358d tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:20:57.060761 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-a0d4882d-8b2d-4d45-9851-5c9de7e7358d tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] [instance: fa4d69b3-8beb-4dfa-b655-c5c26dcac34f] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:20:57.071352 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a0d4882d-8b2d-4d45-9851-5c9de7e7358d tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:57.072735 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a0d4882d-8b2d-4d45-9851-5c9de7e7358d tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:57.073773 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a0d4882d-8b2d-4d45-9851-5c9de7e7358d tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:57.074396 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a0d4882d-8b2d-4d45-9851-5c9de7e7358d tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:57.638467 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-a0d4882d-8b2d-4d45-9851-5c9de7e7358d tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd3bb6f6d546c4bd090e72c149afe5660', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f163566907554620bc94de3b960a4f13', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:20:57.649993 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-a0d4882d-8b2d-4d45-9851-5c9de7e7358d tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] 10.222.0.16 "GET /compute/v2.1/servers/fa4d69b3-8beb-4dfa-b655-c5c26dcac34f" status: 200 len: 1316 microversion: 2.1 time: 0.594850 Jul 16 21:20:57.652271 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1478/2908] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:20:57 2022] GET /compute/v2.1/servers/fa4d69b3-8beb-4dfa-b655-c5c26dcac34f => generated 1316 bytes in 598 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:20:58.064906 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-d22ec7fd-f940-40a1-a740-9e69c54ef0a6 tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:20:58.064906 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-d22ec7fd-f940-40a1-a740-9e69c54ef0a6 tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] [instance: 0a9564cb-c375-47e9-bfe6-e68d4700442b] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:20:58.087876 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d22ec7fd-f940-40a1-a740-9e69c54ef0a6 tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:58.087876 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d22ec7fd-f940-40a1-a740-9e69c54ef0a6 tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:58.088381 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d22ec7fd-f940-40a1-a740-9e69c54ef0a6 tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:58.093225 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d22ec7fd-f940-40a1-a740-9e69c54ef0a6 tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:58.629350 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-d22ec7fd-f940-40a1-a740-9e69c54ef0a6 tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1f72b1e0b4124480bfd53b5c22a343e0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0be222897b3a4864a39e9fc7d481340a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:20:58.643049 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-d22ec7fd-f940-40a1-a740-9e69c54ef0a6 tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] 10.222.0.16 "GET /compute/v2.1/servers/0a9564cb-c375-47e9-bfe6-e68d4700442b" status: 200 len: 1713 microversion: 2.1 time: 0.581516 Jul 16 21:20:58.647466 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1431/2909] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:20:58 2022] GET /compute/v2.1/servers/0a9564cb-c375-47e9-bfe6-e68d4700442b => generated 1713 bytes in 590 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:20:58.675431 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-687d736a-c372-43d7-9195-fd6be2a364af tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:20:58.677041 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-687d736a-c372-43d7-9195-fd6be2a364af tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] [instance: fa4d69b3-8beb-4dfa-b655-c5c26dcac34f] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:20:58.719706 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-687d736a-c372-43d7-9195-fd6be2a364af tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:58.726519 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-687d736a-c372-43d7-9195-fd6be2a364af tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:58.733479 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-687d736a-c372-43d7-9195-fd6be2a364af tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:58.734285 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-687d736a-c372-43d7-9195-fd6be2a364af tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:59.134846 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-687d736a-c372-43d7-9195-fd6be2a364af tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd3bb6f6d546c4bd090e72c149afe5660', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f163566907554620bc94de3b960a4f13', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:20:59.171174 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-687d736a-c372-43d7-9195-fd6be2a364af tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] 10.222.0.16 "GET /compute/v2.1/servers/fa4d69b3-8beb-4dfa-b655-c5c26dcac34f" status: 200 len: 1316 microversion: 2.1 time: 0.497561 Jul 16 21:20:59.191047 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1479/2910] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:20:58 2022] GET /compute/v2.1/servers/fa4d69b3-8beb-4dfa-b655-c5c26dcac34f => generated 1316 bytes in 504 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:20:59.671661 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-e6221212-09ac-41a1-82e2-7539ebb34fbf tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:20:59.673410 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-e6221212-09ac-41a1-82e2-7539ebb34fbf tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] [instance: 0a9564cb-c375-47e9-bfe6-e68d4700442b] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:20:59.710527 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e6221212-09ac-41a1-82e2-7539ebb34fbf tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:59.716201 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e6221212-09ac-41a1-82e2-7539ebb34fbf tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:20:59.719357 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e6221212-09ac-41a1-82e2-7539ebb34fbf tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:20:59.719971 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e6221212-09ac-41a1-82e2-7539ebb34fbf tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:21:00.064178 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-894d07ac-ff81-42df-8168-8581bad69f09 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Action: 'create', calling method: .version_select at 0x7fdd9c57d940>, body: {"keypair": {"name": "tempest-keypair-1504167600", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBHES7+NsbHSJa9BYfVPzawU+6J81KWt3BqZNwaiUTqjIBhjDj7FiaSLBfTrpro3Gc2MvggE16EHRAy9JT/FWNaWzCivgbiC8HMnlquxIssigSaJFXaNSViizCegVqwVKlQ=="}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:21:00.071684 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_api:SELECT=48,nova_api:DELETE=3 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:21:00.108738 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-894d07ac-ff81-42df-8168-8581bad69f09 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Getting quotas for project 22ca92a86e6c40d59071e58ce4085568. Resources: {'key_pairs'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:21:00.142843 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-894d07ac-ff81-42df-8168-8581bad69f09 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Getting quotas for user bba77f6ef42547f1aa8480e27868bd4a and project 22ca92a86e6c40d59071e58ce4085568. Resources: {'key_pairs'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:21:00.320824 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-894d07ac-ff81-42df-8168-8581bad69f09 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] 10.222.0.16 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 1.363558 Jul 16 21:21:00.327840 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1432/2911] 10.222.0.16 () {60 vars in 1158 bytes} [Sat Jul 16 21:20:58 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 1372 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:21:00.339680 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-79324195-d2d5-42d0-a4f8-90cdf9e35ee2 tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:21:00.340600 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-79324195-d2d5-42d0-a4f8-90cdf9e35ee2 tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] [instance: fa4d69b3-8beb-4dfa-b655-c5c26dcac34f] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:21:00.376007 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-79324195-d2d5-42d0-a4f8-90cdf9e35ee2 tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:21:00.379165 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-79324195-d2d5-42d0-a4f8-90cdf9e35ee2 tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:21:00.384545 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-79324195-d2d5-42d0-a4f8-90cdf9e35ee2 tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:21:00.384545 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-79324195-d2d5-42d0-a4f8-90cdf9e35ee2 tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:21:00.524903 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-e6221212-09ac-41a1-82e2-7539ebb34fbf tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1f72b1e0b4124480bfd53b5c22a343e0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0be222897b3a4864a39e9fc7d481340a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:21:00.536652 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-e6221212-09ac-41a1-82e2-7539ebb34fbf tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] 10.222.0.16 "GET /compute/v2.1/servers/0a9564cb-c375-47e9-bfe6-e68d4700442b" status: 200 len: 1713 microversion: 2.1 time: 0.871600 Jul 16 21:21:00.537273 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1480/2912] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:20:59 2022] GET /compute/v2.1/servers/0a9564cb-c375-47e9-bfe6-e68d4700442b => generated 1713 bytes in 873 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:21:01.234201 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-79324195-d2d5-42d0-a4f8-90cdf9e35ee2 tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd3bb6f6d546c4bd090e72c149afe5660', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f163566907554620bc94de3b960a4f13', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:21:01.276584 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-79324195-d2d5-42d0-a4f8-90cdf9e35ee2 tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] 10.222.0.16 "GET /compute/v2.1/servers/fa4d69b3-8beb-4dfa-b655-c5c26dcac34f" status: 200 len: 1505 microversion: 2.1 time: 0.943998 Jul 16 21:21:01.292896 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1433/2913] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:21:00 2022] GET /compute/v2.1/servers/fa4d69b3-8beb-4dfa-b655-c5c26dcac34f => generated 1505 bytes in 965 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:21:01.321724 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-94abd82c-f478-4280-ab8d-b3a6b845395c tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:21:01.322300 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-94abd82c-f478-4280-ab8d-b3a6b845395c tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] [instance: fa4d69b3-8beb-4dfa-b655-c5c26dcac34f] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:21:01.354712 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-94abd82c-f478-4280-ab8d-b3a6b845395c tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:21:01.355221 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-94abd82c-f478-4280-ab8d-b3a6b845395c tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:21:01.356147 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-94abd82c-f478-4280-ab8d-b3a6b845395c tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:21:01.356618 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-94abd82c-f478-4280-ab8d-b3a6b845395c tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:21:01.573599 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-511406b7-d3ad-4656-841a-332cd2ddac34 tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:21:01.574482 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-511406b7-d3ad-4656-841a-332cd2ddac34 tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] [instance: 0a9564cb-c375-47e9-bfe6-e68d4700442b] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:21:01.598575 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-511406b7-d3ad-4656-841a-332cd2ddac34 tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:21:01.603701 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-511406b7-d3ad-4656-841a-332cd2ddac34 tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:21:01.606197 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-511406b7-d3ad-4656-841a-332cd2ddac34 tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:21:01.621105 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-511406b7-d3ad-4656-841a-332cd2ddac34 tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.015s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:21:01.690786 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-94abd82c-f478-4280-ab8d-b3a6b845395c tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] 10.222.0.16 "DELETE /compute/v2.1/servers/fa4d69b3-8beb-4dfa-b655-c5c26dcac34f/os-server-password" status: 204 len: 0 microversion: 2.1 time: 0.372497 Jul 16 21:21:01.692168 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1481/2914] 10.222.0.16 () {58 vars in 1295 bytes} [Sat Jul 16 21:21:01 2022] DELETE /compute/v2.1/servers/fa4d69b3-8beb-4dfa-b655-c5c26dcac34f/os-server-password => generated 0 bytes in 375 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 16 21:21:01.698278 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-0b5fa0bc-9f9a-4fad-acb4-bf8bd16484c0 req-d9cd0f0b-64cb-4744-861b-73cfe0d743ac service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "0a9564cb-c375-47e9-bfe6-e68d4700442b", "name": "network-vif-deleted", "tag": "22eac33c-df13-4324-a3f6-14093cd60ead"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:21:01.769611 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-0b5fa0bc-9f9a-4fad-acb4-bf8bd16484c0 req-d9cd0f0b-64cb-4744-861b-73cfe0d743ac service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:21:01.770658 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-0b5fa0bc-9f9a-4fad-acb4-bf8bd16484c0 req-d9cd0f0b-64cb-4744-861b-73cfe0d743ac service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:21:01.810380 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-0b5fa0bc-9f9a-4fad-acb4-bf8bd16484c0 req-d9cd0f0b-64cb-4744-861b-73cfe0d743ac service nova] Creating event network-vif-deleted:22eac33c-df13-4324-a3f6-14093cd60ead for instance 0a9564cb-c375-47e9-bfe6-e68d4700442b on nch2-master-1 Jul 16 21:21:01.844645 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-0b5fa0bc-9f9a-4fad-acb4-bf8bd16484c0 req-d9cd0f0b-64cb-4744-861b-73cfe0d743ac service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.147300 Jul 16 21:21:01.851085 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1482/2915] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:21:01 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:21:01.880040 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-671f7a11-8a5b-4de4-9e41-cc867790c7a8 tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:21:01.881866 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-671f7a11-8a5b-4de4-9e41-cc867790c7a8 tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] [instance: fa4d69b3-8beb-4dfa-b655-c5c26dcac34f] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:21:01.911260 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-671f7a11-8a5b-4de4-9e41-cc867790c7a8 tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:21:01.918969 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-671f7a11-8a5b-4de4-9e41-cc867790c7a8 tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:21:01.920086 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-671f7a11-8a5b-4de4-9e41-cc867790c7a8 tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:21:01.921410 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-671f7a11-8a5b-4de4-9e41-cc867790c7a8 tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:21:02.024221 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-671f7a11-8a5b-4de4-9e41-cc867790c7a8 tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] 10.222.0.16 "GET /compute/v2.1/servers/fa4d69b3-8beb-4dfa-b655-c5c26dcac34f/os-server-password" status: 200 len: 16 microversion: 2.1 time: 0.164446 Jul 16 21:21:02.025692 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1483/2916] 10.222.0.16 () {58 vars in 1292 bytes} [Sat Jul 16 21:21:01 2022] GET /compute/v2.1/servers/fa4d69b3-8beb-4dfa-b655-c5c26dcac34f/os-server-password => generated 16 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Jul 16 21:21:02.039909 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-85c09f1b-1c3e-431b-bc77-1ab665be7fa9 tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:21:02.043575 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-85c09f1b-1c3e-431b-bc77-1ab665be7fa9 tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] [instance: fa4d69b3-8beb-4dfa-b655-c5c26dcac34f] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:21:02.066484 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-85c09f1b-1c3e-431b-bc77-1ab665be7fa9 tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:21:02.066996 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-85c09f1b-1c3e-431b-bc77-1ab665be7fa9 tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:21:02.067873 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-85c09f1b-1c3e-431b-bc77-1ab665be7fa9 tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:21:02.068318 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-85c09f1b-1c3e-431b-bc77-1ab665be7fa9 tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:21:02.135532 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-511406b7-d3ad-4656-841a-332cd2ddac34 tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1f72b1e0b4124480bfd53b5c22a343e0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0be222897b3a4864a39e9fc7d481340a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:21:02.141210 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=93,nova_cell1:UPDATE=2,nova_cell1:INSERT=5 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:21:02.157743 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-511406b7-d3ad-4656-841a-332cd2ddac34 tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] 10.222.0.16 "GET /compute/v2.1/servers/0a9564cb-c375-47e9-bfe6-e68d4700442b" status: 200 len: 1646 microversion: 2.1 time: 0.589098 Jul 16 21:21:02.158656 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1434/2917] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:21:01 2022] GET /compute/v2.1/servers/0a9564cb-c375-47e9-bfe6-e68d4700442b => generated 1646 bytes in 591 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:21:02.161675 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-85c09f1b-1c3e-431b-bc77-1ab665be7fa9 tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] [instance: fa4d69b3-8beb-4dfa-b655-c5c26dcac34f] Going to try to terminate instance {{(pid=138680) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 16 21:21:02.726096 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-85c09f1b-1c3e-431b-bc77-1ab665be7fa9 tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] 10.222.0.16 "DELETE /compute/v2.1/servers/fa4d69b3-8beb-4dfa-b655-c5c26dcac34f" status: 204 len: 0 microversion: 2.1 time: 0.690670 Jul 16 21:21:02.726096 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1484/2918] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:21:02 2022] DELETE /compute/v2.1/servers/fa4d69b3-8beb-4dfa-b655-c5c26dcac34f => generated 0 bytes in 694 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 16 21:21:02.734916 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-a7b99e5a-faee-41d2-8482-4a48ea1ae2a7 tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:21:02.736239 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-a7b99e5a-faee-41d2-8482-4a48ea1ae2a7 tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] [instance: fa4d69b3-8beb-4dfa-b655-c5c26dcac34f] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:21:02.770596 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a7b99e5a-faee-41d2-8482-4a48ea1ae2a7 tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:21:02.776357 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a7b99e5a-faee-41d2-8482-4a48ea1ae2a7 tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.017s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:21:02.777297 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a7b99e5a-faee-41d2-8482-4a48ea1ae2a7 tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:21:02.794075 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a7b99e5a-faee-41d2-8482-4a48ea1ae2a7 tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:21:03.130867 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-a7b99e5a-faee-41d2-8482-4a48ea1ae2a7 tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd3bb6f6d546c4bd090e72c149afe5660', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f163566907554620bc94de3b960a4f13', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:21:03.161522 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-a7b99e5a-faee-41d2-8482-4a48ea1ae2a7 tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] 10.222.0.16 "GET /compute/v2.1/servers/fa4d69b3-8beb-4dfa-b655-c5c26dcac34f" status: 200 len: 1511 microversion: 2.1 time: 0.431417 Jul 16 21:21:03.162871 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1435/2919] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:21:02 2022] GET /compute/v2.1/servers/fa4d69b3-8beb-4dfa-b655-c5c26dcac34f => generated 1511 bytes in 433 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:21:03.188756 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-6e66ee9b-b56f-4832-a07a-ccd75a9c4c21 tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:21:03.190603 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-6e66ee9b-b56f-4832-a07a-ccd75a9c4c21 tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] [instance: 0a9564cb-c375-47e9-bfe6-e68d4700442b] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:21:03.201555 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6e66ee9b-b56f-4832-a07a-ccd75a9c4c21 tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:21:03.204657 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6e66ee9b-b56f-4832-a07a-ccd75a9c4c21 tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:21:03.204657 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6e66ee9b-b56f-4832-a07a-ccd75a9c4c21 tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:21:03.205259 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6e66ee9b-b56f-4832-a07a-ccd75a9c4c21 tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:21:03.260484 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6e66ee9b-b56f-4832-a07a-ccd75a9c4c21 tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:21:03.260484 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6e66ee9b-b56f-4832-a07a-ccd75a9c4c21 tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:21:03.260484 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.objects.instance [None req-6e66ee9b-b56f-4832-a07a-ccd75a9c4c21 tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] Lazy-loading 'fault' on Instance uuid 0a9564cb-c375-47e9-bfe6-e68d4700442b {{(pid=138680) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:21:03.396260 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-6e66ee9b-b56f-4832-a07a-ccd75a9c4c21 tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1f72b1e0b4124480bfd53b5c22a343e0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0be222897b3a4864a39e9fc7d481340a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:21:03.413476 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-6e66ee9b-b56f-4832-a07a-ccd75a9c4c21 tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] 10.222.0.16 "GET /compute/v2.1/servers/0a9564cb-c375-47e9-bfe6-e68d4700442b" status: 200 len: 1367 microversion: 2.1 time: 0.231020 Jul 16 21:21:03.425739 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1485/2920] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:21:03 2022] GET /compute/v2.1/servers/0a9564cb-c375-47e9-bfe6-e68d4700442b => generated 1367 bytes in 246 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:21:04.183897 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-ea77f975-4d01-4966-946f-41447cbf6848 tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:21:04.191321 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-ea77f975-4d01-4966-946f-41447cbf6848 tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] [instance: fa4d69b3-8beb-4dfa-b655-c5c26dcac34f] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:21:04.203387 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ea77f975-4d01-4966-946f-41447cbf6848 tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:21:04.211044 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ea77f975-4d01-4966-946f-41447cbf6848 tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:21:04.212071 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ea77f975-4d01-4966-946f-41447cbf6848 tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:21:04.216978 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ea77f975-4d01-4966-946f-41447cbf6848 tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:21:04.440530 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-a7663c98-966e-401b-b805-cab07ea4dbe7 tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:21:04.442358 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-a7663c98-966e-401b-b805-cab07ea4dbe7 tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] [instance: 0a9564cb-c375-47e9-bfe6-e68d4700442b] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:21:04.465728 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell0:SELECT=2 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:21:04.470563 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a7663c98-966e-401b-b805-cab07ea4dbe7 tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:21:04.472345 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a7663c98-966e-401b-b805-cab07ea4dbe7 tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.011s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:21:04.477628 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a7663c98-966e-401b-b805-cab07ea4dbe7 tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:21:04.479331 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a7663c98-966e-401b-b805-cab07ea4dbe7 tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:21:04.520342 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-a7663c98-966e-401b-b805-cab07ea4dbe7 tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] HTTP exception thrown: Instance 0a9564cb-c375-47e9-bfe6-e68d4700442b could not be found. Jul 16 21:21:04.531315 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-a7663c98-966e-401b-b805-cab07ea4dbe7 tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] Request method failure captured: Jul 16 21:21:04.531315 ncdv-master-1 devstack@n-api.service[138680]: request: GET /compute/v2.1/servers/0a9564cb-c375-47e9-bfe6-e68d4700442b HTTP/1.1 Jul 16 21:21:04.531315 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:21:04.531315 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:21:04.531315 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:21:04.531315 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 0 Jul 16 21:21:04.531315 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:21:04.531315 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:21:04.531315 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:21:04.531315 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:21:04.531315 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0ytyrlolLl8kKrtcVppE89HeuWQhXN0wSXm13yl6eeIpFIceuQepMR6-wKvJi5-MY2txOYpbHEdzJtWjZrQEdGfM4kSgUbesFtdh33NDA2VAVWK7WNHGEvT6pjcZY2kLERJ4kTY4KdgsLf8uR9A5Ydy_pDq1YOwiAcQt8vEnkheOQOg Jul 16 21:21:04.531315 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:21:04.531315 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:21:04.531315 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:21:04.531315 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:21:04.531315 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:21:04.531315 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:21:04.531315 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: 0be222897b3a4864a39e9fc7d481340a Jul 16 21:21:04.531315 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-ServersTestManualDisk-1287576747 Jul 16 21:21:04.531315 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:21:04.531315 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:21:04.531315 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "tempest-type-314677192", "name": "tempest-service-239606739", "endpoints": [{"region": "tempest-region-1341024243", "internalURL": "https://url-1881773296.com"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/0be222897b3a4864a39e9fc7d481340a"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/0be222897b3a4864a39e9fc7d481340a"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_0be222897b3a4864a39e9fc7d481340a"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/0be222897b3a4864a39e9fc7d481340a"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "tempest-type-134202145", "name": "tempest-service-1489181580", "endpoints": [{"region": "tempest-region-292581853", "publicURL": "https://url-1812817479.com"}, {"region": "tempest-region-369744987", "publicURL": "https://url-1949724457.com"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:21:04.531315 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-ServersTestManualDisk-1287576747 Jul 16 21:21:04.531315 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: 0be222897b3a4864a39e9fc7d481340a Jul 16 21:21:04.531315 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-ServersTestManualDisk-1287576747 Jul 16 21:21:04.531315 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-ServersTestManualDisk-1287576747-project Jul 16 21:21:04.531315 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:21:04.531315 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:21:04.531315 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: 1f72b1e0b4124480bfd53b5c22a343e0 Jul 16 21:21:04.531315 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-ServersTestManualDisk-1287576747-project Jul 16 21:21:04.531315 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 21:21:04.531315 ncdv-master-1 devstack@n-api.service[138680]: exception: Instance 0a9564cb-c375-47e9-bfe6-e68d4700442b could not be found. Jul 16 21:21:04.531315 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': '0a9564cb-c375-47e9-bfe6-e68d4700442b'} Jul 16 21:21:04.531315 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:21:04.531315 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:21:04.531315 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:21:04.531315 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:21:04.531315 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:21:04.531315 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:21:04.531315 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:21:04.531315 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:21:04.531315 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:21:04.531315 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:21:04.531315 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 0a9564cb-c375-47e9-bfe6-e68d4700442b could not be found. Jul 16 21:21:04.531315 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:21:04.531315 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:21:04.531315 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:21:04.531315 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:21:04.531315 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:21:04.531315 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:21:04.531315 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:21:04.531315 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:21:04.531315 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:21:04.531315 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:21:04.531315 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:21:04.531315 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:21:04.531315 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:21:04.531315 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:21:04.531315 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:21:04.569199 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:21:04.569199 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 0a9564cb-c375-47e9-bfe6-e68d4700442b could not be found. Jul 16 21:21:04.569199 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:21:04.569199 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:21:04.569199 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:21:04.569199 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:21:04.569199 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:21:04.569199 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:21:04.569199 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:21:04.569199 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:21:04.569199 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 0a9564cb-c375-47e9-bfe6-e68d4700442b could not be found. Jul 16 21:21:04.569199 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:21:04.569199 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-a7663c98-966e-401b-b805-cab07ea4dbe7 tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] Returning 404 to user: Instance 0a9564cb-c375-47e9-bfe6-e68d4700442b could not be found. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:21:04.569199 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-a7663c98-966e-401b-b805-cab07ea4dbe7 tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] 10.222.0.16 "GET /compute/v2.1/servers/0a9564cb-c375-47e9-bfe6-e68d4700442b" status: 404 len: 111 microversion: 2.1 time: 0.097288 Jul 16 21:21:04.569199 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1486/2921] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:21:04 2022] GET /compute/v2.1/servers/0a9564cb-c375-47e9-bfe6-e68d4700442b => generated 111 bytes in 99 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:21:04.569199 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-786a62a1-ab90-4855-b85b-62be1037665b tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] Calling method '.version_select at 0x7fdda2b029d0>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:21:04.634034 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell0:SELECT=2 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:21:04.676958 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-ea77f975-4d01-4966-946f-41447cbf6848 tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd3bb6f6d546c4bd090e72c149afe5660', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f163566907554620bc94de3b960a4f13', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:21:04.687911 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-ea77f975-4d01-4966-946f-41447cbf6848 tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] 10.222.0.16 "GET /compute/v2.1/servers/fa4d69b3-8beb-4dfa-b655-c5c26dcac34f" status: 200 len: 1511 microversion: 2.1 time: 0.508720 Jul 16 21:21:04.689075 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1436/2922] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:21:04 2022] GET /compute/v2.1/servers/fa4d69b3-8beb-4dfa-b655-c5c26dcac34f => generated 1511 bytes in 510 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:21:04.774642 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-786a62a1-ab90-4855-b85b-62be1037665b tempest-ServersTestManualDisk-1287576747 tempest-ServersTestManualDisk-1287576747-project] 10.222.0.16 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1642910678" status: 202 len: 0 microversion: 2.1 time: 0.235811 Jul 16 21:21:04.779205 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1487/2923] 10.222.0.16 () {58 vars in 1220 bytes} [Sat Jul 16 21:21:04 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1642910678 => generated 0 bytes in 241 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 21:21:05.429524 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-d34f1c03-e01f-4728-92df-e0d292bcf7db req-aa01e343-813e-41ae-b075-f7f228dfd6cf service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "fa4d69b3-8beb-4dfa-b655-c5c26dcac34f", "name": "network-vif-deleted", "tag": "dc02eaab-b399-4f24-92e0-d5cefb6ea105"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:21:05.478910 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-d34f1c03-e01f-4728-92df-e0d292bcf7db req-aa01e343-813e-41ae-b075-f7f228dfd6cf service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:21:05.479817 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-d34f1c03-e01f-4728-92df-e0d292bcf7db req-aa01e343-813e-41ae-b075-f7f228dfd6cf service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.024s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:21:05.532639 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-d34f1c03-e01f-4728-92df-e0d292bcf7db req-aa01e343-813e-41ae-b075-f7f228dfd6cf service nova] Creating event network-vif-deleted:dc02eaab-b399-4f24-92e0-d5cefb6ea105 for instance fa4d69b3-8beb-4dfa-b655-c5c26dcac34f on nch1-master-1 Jul 16 21:21:05.565574 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-d34f1c03-e01f-4728-92df-e0d292bcf7db req-aa01e343-813e-41ae-b075-f7f228dfd6cf service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.149505 Jul 16 21:21:05.569706 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1437/2924] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:21:05 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:21:05.715679 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-3254e509-20be-4cdf-998f-920dbfc76c66 tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:21:05.716937 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-3254e509-20be-4cdf-998f-920dbfc76c66 tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] [instance: fa4d69b3-8beb-4dfa-b655-c5c26dcac34f] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:21:05.732767 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3254e509-20be-4cdf-998f-920dbfc76c66 tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:21:05.736071 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3254e509-20be-4cdf-998f-920dbfc76c66 tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:21:05.738552 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3254e509-20be-4cdf-998f-920dbfc76c66 tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:21:05.739601 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3254e509-20be-4cdf-998f-920dbfc76c66 tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:21:06.465163 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-3254e509-20be-4cdf-998f-920dbfc76c66 tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd3bb6f6d546c4bd090e72c149afe5660', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f163566907554620bc94de3b960a4f13', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:21:06.478670 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-3254e509-20be-4cdf-998f-920dbfc76c66 tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] 10.222.0.16 "GET /compute/v2.1/servers/fa4d69b3-8beb-4dfa-b655-c5c26dcac34f" status: 200 len: 1301 microversion: 2.1 time: 0.774065 Jul 16 21:21:06.480435 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1488/2925] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:21:05 2022] GET /compute/v2.1/servers/fa4d69b3-8beb-4dfa-b655-c5c26dcac34f => generated 1301 bytes in 779 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:21:06.730720 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-d9b73105-244f-4dea-b093-bd08a2ac2ffa tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeTestJSON-server-652391072", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "57ded3f4-cdf1-4dc2-a2d8-ef8ca61bdef4"}], "security_groups": [{"name": "tempest-securitygroup--1529969768"}], "key_name": "tempest-keypair-1504167600", "user_data": "}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:21:06.985369 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.network.neutron [None req-d9b73105-244f-4dea-b093-bd08a2ac2ffa tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] validate_networks() for [('57ded3f4-cdf1-4dc2-a2d8-ef8ca61bdef4', None, None, None, None, None)] {{(pid=138681) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 16 21:21:07.497940 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-b17eb4a5-5fb8-4ddf-b777-ba9f1cea0fa4 tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:21:07.498563 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-b17eb4a5-5fb8-4ddf-b777-ba9f1cea0fa4 tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] [instance: fa4d69b3-8beb-4dfa-b655-c5c26dcac34f] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:21:07.522196 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b17eb4a5-5fb8-4ddf-b777-ba9f1cea0fa4 tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:21:07.524221 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b17eb4a5-5fb8-4ddf-b777-ba9f1cea0fa4 tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:21:07.525928 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b17eb4a5-5fb8-4ddf-b777-ba9f1cea0fa4 tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:21:07.533697 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b17eb4a5-5fb8-4ddf-b777-ba9f1cea0fa4 tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:21:07.578562 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-b17eb4a5-5fb8-4ddf-b777-ba9f1cea0fa4 tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] HTTP exception thrown: Instance fa4d69b3-8beb-4dfa-b655-c5c26dcac34f could not be found. Jul 16 21:21:07.582577 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-b17eb4a5-5fb8-4ddf-b777-ba9f1cea0fa4 tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] Request method failure captured: Jul 16 21:21:07.582577 ncdv-master-1 devstack@n-api.service[138680]: request: GET /compute/v2.1/servers/fa4d69b3-8beb-4dfa-b655-c5c26dcac34f HTTP/1.1 Jul 16 21:21:07.582577 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:21:07.582577 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:21:07.582577 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:21:07.582577 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 0 Jul 16 21:21:07.582577 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:21:07.582577 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:21:07.582577 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:21:07.582577 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:21:07.582577 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0yuldoV-ne91-iZkvnX4PYhWdF1b3KC2htAH7buCkiKpktR_2b448qQ6UFWtcm5h_HIDHiN7X6QJxb7B7A02ob2Zu7ZO6Juve2Wey7IN6Pk8jOYDftlIr5D7M6a0IquEdNBayu5WOQkb8atF_aBh3C5n454kfKTAJUQ0UJGzP02US_4 Jul 16 21:21:07.582577 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:21:07.582577 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:21:07.582577 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:21:07.582577 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:21:07.582577 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:21:07.582577 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:21:07.582577 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: f163566907554620bc94de3b960a4f13 Jul 16 21:21:07.582577 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-ServerPasswordTestJSON-2022348653 Jul 16 21:21:07.582577 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:21:07.582577 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:21:07.582577 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/f163566907554620bc94de3b960a4f13"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/f163566907554620bc94de3b960a4f13"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_f163566907554620bc94de3b960a4f13"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/f163566907554620bc94de3b960a4f13"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:21:07.582577 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-ServerPasswordTestJSON-2022348653 Jul 16 21:21:07.582577 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: f163566907554620bc94de3b960a4f13 Jul 16 21:21:07.582577 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-ServerPasswordTestJSON-2022348653 Jul 16 21:21:07.582577 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-ServerPasswordTestJSON-2022348653-project Jul 16 21:21:07.582577 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:21:07.582577 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:21:07.582577 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: d3bb6f6d546c4bd090e72c149afe5660 Jul 16 21:21:07.582577 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-ServerPasswordTestJSON-2022348653-project Jul 16 21:21:07.582577 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 21:21:07.582577 ncdv-master-1 devstack@n-api.service[138680]: exception: Instance fa4d69b3-8beb-4dfa-b655-c5c26dcac34f could not be found. Jul 16 21:21:07.582577 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': 'fa4d69b3-8beb-4dfa-b655-c5c26dcac34f'} Jul 16 21:21:07.582577 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:21:07.582577 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:21:07.582577 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:21:07.582577 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:21:07.582577 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:21:07.582577 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:21:07.582577 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:21:07.582577 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:21:07.582577 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:21:07.582577 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:21:07.582577 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance fa4d69b3-8beb-4dfa-b655-c5c26dcac34f could not be found. Jul 16 21:21:07.582577 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:21:07.582577 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:21:07.582577 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:21:07.582577 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:21:07.582577 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:21:07.582577 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:21:07.582577 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:21:07.582577 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:21:07.582577 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:21:07.582577 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:21:07.582577 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:21:07.582577 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:21:07.582577 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:21:07.582577 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:21:07.582577 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:21:07.582577 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:21:07.582577 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance fa4d69b3-8beb-4dfa-b655-c5c26dcac34f could not be found. Jul 16 21:21:07.582577 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:21:07.582577 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:21:07.582577 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:21:07.588065 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:21:07.588065 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:21:07.588065 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:21:07.588065 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:21:07.588065 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:21:07.588065 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance fa4d69b3-8beb-4dfa-b655-c5c26dcac34f could not be found. Jul 16 21:21:07.588065 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:21:07.590258 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-b17eb4a5-5fb8-4ddf-b777-ba9f1cea0fa4 tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] Returning 404 to user: Instance fa4d69b3-8beb-4dfa-b655-c5c26dcac34f could not be found. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:21:07.591531 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-b17eb4a5-5fb8-4ddf-b777-ba9f1cea0fa4 tempest-ServerPasswordTestJSON-2022348653 tempest-ServerPasswordTestJSON-2022348653-project] 10.222.0.16 "GET /compute/v2.1/servers/fa4d69b3-8beb-4dfa-b655-c5c26dcac34f" status: 404 len: 111 microversion: 2.1 time: 0.097405 Jul 16 21:21:07.593252 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1489/2926] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:21:07 2022] GET /compute/v2.1/servers/fa4d69b3-8beb-4dfa-b655-c5c26dcac34f => generated 111 bytes in 100 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Jul 16 21:21:07.922649 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-d9b73105-244f-4dea-b093-bd08a2ac2ffa tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Flavor limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 16 21:21:07.923739 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-d9b73105-244f-4dea-b093-bd08a2ac2ffa tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Image limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 16 21:21:07.924699 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-d9b73105-244f-4dea-b093-bd08a2ac2ffa tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Flavor pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 16 21:21:07.925549 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-d9b73105-244f-4dea-b093-bd08a2ac2ffa tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Image pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 16 21:21:07.926185 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-d9b73105-244f-4dea-b093-bd08a2ac2ffa tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 16 21:21:07.941587 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d9b73105-244f-4dea-b093-bd08a2ac2ffa tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:21:07.946823 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d9b73105-244f-4dea-b093-bd08a2ac2ffa tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:21:07.948049 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d9b73105-244f-4dea-b093-bd08a2ac2ffa tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:21:07.948714 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d9b73105-244f-4dea-b093-bd08a2ac2ffa tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:21:08.024163 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-d9b73105-244f-4dea-b093-bd08a2ac2ffa tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Getting quotas for project 22ca92a86e6c40d59071e58ce4085568. Resources: {'ram', 'instances', 'cores'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:21:08.032319 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-d9b73105-244f-4dea-b093-bd08a2ac2ffa tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Getting quotas for user bba77f6ef42547f1aa8480e27868bd4a and project 22ca92a86e6c40d59071e58ce4085568. Resources: {'ram', 'instances', 'cores'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:21:08.052731 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-d9b73105-244f-4dea-b093-bd08a2ac2ffa tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Going to run 1 instances... {{(pid=138681) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 16 21:21:08.055631 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-d9b73105-244f-4dea-b093-bd08a2ac2ffa tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] [instance: f114e124-2903-4f30-b055-9f493e3e9e6c] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138681) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 16 21:21:08.269215 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-d9b73105-244f-4dea-b093-bd08a2ac2ffa tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 402 microversion: 2.1 time: 1.545324 Jul 16 21:21:08.273988 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1438/2927] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 21:21:06 2022] POST /compute/v2.1/servers => generated 402 bytes in 1551 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 16 21:21:08.286129 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-5b770c1b-eb0f-411a-9af1-3daa5cd888b2 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:21:08.286802 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-5b770c1b-eb0f-411a-9af1-3daa5cd888b2 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] [instance: f114e124-2903-4f30-b055-9f493e3e9e6c] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:21:08.443129 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-5b770c1b-eb0f-411a-9af1-3daa5cd888b2 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bba77f6ef42547f1aa8480e27868bd4a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '22ca92a86e6c40d59071e58ce4085568', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:21:08.451464 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-5b770c1b-eb0f-411a-9af1-3daa5cd888b2 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] 10.222.0.16 "GET /compute/v2.1/servers/f114e124-2903-4f30-b055-9f493e3e9e6c" status: 200 len: 1237 microversion: 2.1 time: 0.170595 Jul 16 21:21:08.461483 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1490/2928] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:21:08 2022] GET /compute/v2.1/servers/f114e124-2903-4f30-b055-9f493e3e9e6c => generated 1237 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:21:08.710556 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-86dffc34-18e0-47f9-84ad-ff3b623ed4b3 tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:21:08.712989 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-86dffc34-18e0-47f9-84ad-ff3b623ed4b3 tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] [instance: 6cba399c-58c3-4850-8065-0fbc3301fb2c] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:21:08.735357 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-86dffc34-18e0-47f9-84ad-ff3b623ed4b3 tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:21:08.735357 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-86dffc34-18e0-47f9-84ad-ff3b623ed4b3 tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:21:08.735357 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-86dffc34-18e0-47f9-84ad-ff3b623ed4b3 tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:21:08.738061 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-86dffc34-18e0-47f9-84ad-ff3b623ed4b3 tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:21:08.842792 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-86dffc34-18e0-47f9-84ad-ff3b623ed4b3 tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] [instance: 6cba399c-58c3-4850-8065-0fbc3301fb2c] Going to try to terminate instance {{(pid=138681) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 16 21:21:08.941341 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=99,nova_cell1:UPDATE=1 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:21:09.353395 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-86dffc34-18e0-47f9-84ad-ff3b623ed4b3 tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] 10.222.0.16 "DELETE /compute/v2.1/servers/6cba399c-58c3-4850-8065-0fbc3301fb2c" status: 204 len: 0 microversion: 2.1 time: 0.644428 Jul 16 21:21:09.353395 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1439/2929] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:21:08 2022] DELETE /compute/v2.1/servers/6cba399c-58c3-4850-8065-0fbc3301fb2c => generated 0 bytes in 646 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Jul 16 21:21:09.359432 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-344b0880-96c1-4012-a303-a0af54af50c1 tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:21:09.360640 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-344b0880-96c1-4012-a303-a0af54af50c1 tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] [instance: 6cba399c-58c3-4850-8065-0fbc3301fb2c] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:21:09.376129 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_api:SELECT=85,nova_api:INSERT=3,nova_api:UPDATE=2,nova_api:DELETE=1 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:21:09.379342 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-344b0880-96c1-4012-a303-a0af54af50c1 tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:21:09.381553 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-344b0880-96c1-4012-a303-a0af54af50c1 tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:21:09.388920 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-344b0880-96c1-4012-a303-a0af54af50c1 tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:21:09.390780 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-344b0880-96c1-4012-a303-a0af54af50c1 tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:21:09.481246 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-dd8e9834-3b3e-4f1e-bd01-503b9452effc tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:21:09.482019 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-dd8e9834-3b3e-4f1e-bd01-503b9452effc tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] [instance: f114e124-2903-4f30-b055-9f493e3e9e6c] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:21:09.684933 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-dd8e9834-3b3e-4f1e-bd01-503b9452effc tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bba77f6ef42547f1aa8480e27868bd4a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '22ca92a86e6c40d59071e58ce4085568', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:21:09.701129 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-dd8e9834-3b3e-4f1e-bd01-503b9452effc tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] 10.222.0.16 "GET /compute/v2.1/servers/f114e124-2903-4f30-b055-9f493e3e9e6c" status: 200 len: 1237 microversion: 2.1 time: 0.222160 Jul 16 21:21:09.716262 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1440/2930] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:21:09 2022] GET /compute/v2.1/servers/f114e124-2903-4f30-b055-9f493e3e9e6c => generated 1237 bytes in 243 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:21:09.742258 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-344b0880-96c1-4012-a303-a0af54af50c1 tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c6ba562873fa4f4fbd0b635f1c9dcdf2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '837a8f9e06e141dcbd5f7bd895016d49', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:21:09.758407 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-344b0880-96c1-4012-a303-a0af54af50c1 tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] 10.222.0.16 "GET /compute/v2.1/servers/6cba399c-58c3-4850-8065-0fbc3301fb2c" status: 200 len: 1668 microversion: 2.1 time: 0.403419 Jul 16 21:21:09.763959 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1491/2931] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:21:09 2022] GET /compute/v2.1/servers/6cba399c-58c3-4850-8065-0fbc3301fb2c => generated 1668 bytes in 409 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:21:10.719213 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-d6964600-2024-4871-8fcc-ac1809d67e2a tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:21:10.719931 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-d6964600-2024-4871-8fcc-ac1809d67e2a tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] [instance: f114e124-2903-4f30-b055-9f493e3e9e6c] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:21:10.734389 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d6964600-2024-4871-8fcc-ac1809d67e2a tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:21:10.735041 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d6964600-2024-4871-8fcc-ac1809d67e2a tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:21:10.735981 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d6964600-2024-4871-8fcc-ac1809d67e2a tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:21:10.737733 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d6964600-2024-4871-8fcc-ac1809d67e2a tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:21:10.782117 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-7379c00a-a817-40a6-b2cc-3d4a7db26e39 tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:21:10.783387 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-7379c00a-a817-40a6-b2cc-3d4a7db26e39 tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] [instance: 6cba399c-58c3-4850-8065-0fbc3301fb2c] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:21:10.822254 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7379c00a-a817-40a6-b2cc-3d4a7db26e39 tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:21:10.825420 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7379c00a-a817-40a6-b2cc-3d4a7db26e39 tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:21:10.826610 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7379c00a-a817-40a6-b2cc-3d4a7db26e39 tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:21:10.833625 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7379c00a-a817-40a6-b2cc-3d4a7db26e39 tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:21:10.871991 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-d6964600-2024-4871-8fcc-ac1809d67e2a tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bba77f6ef42547f1aa8480e27868bd4a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '22ca92a86e6c40d59071e58ce4085568', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:21:10.888140 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-d6964600-2024-4871-8fcc-ac1809d67e2a tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] 10.222.0.16 "GET /compute/v2.1/servers/f114e124-2903-4f30-b055-9f493e3e9e6c" status: 200 len: 1289 microversion: 2.1 time: 0.172632 Jul 16 21:21:10.903035 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1441/2932] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:21:10 2022] GET /compute/v2.1/servers/f114e124-2903-4f30-b055-9f493e3e9e6c => generated 1289 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:21:11.415412 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-7379c00a-a817-40a6-b2cc-3d4a7db26e39 tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c6ba562873fa4f4fbd0b635f1c9dcdf2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '837a8f9e06e141dcbd5f7bd895016d49', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:21:11.427428 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-7379c00a-a817-40a6-b2cc-3d4a7db26e39 tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] 10.222.0.16 "GET /compute/v2.1/servers/6cba399c-58c3-4850-8065-0fbc3301fb2c" status: 200 len: 1668 microversion: 2.1 time: 0.648576 Jul 16 21:21:11.428255 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1492/2933] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:21:10 2022] GET /compute/v2.1/servers/6cba399c-58c3-4850-8065-0fbc3301fb2c => generated 1668 bytes in 652 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:21:11.938410 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-cc1c26a0-31e8-4598-b793-dd5729debcb7 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:21:11.940637 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-cc1c26a0-31e8-4598-b793-dd5729debcb7 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] [instance: f114e124-2903-4f30-b055-9f493e3e9e6c] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:21:11.988627 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cc1c26a0-31e8-4598-b793-dd5729debcb7 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:21:11.988627 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cc1c26a0-31e8-4598-b793-dd5729debcb7 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:21:11.988627 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cc1c26a0-31e8-4598-b793-dd5729debcb7 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:21:11.988627 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cc1c26a0-31e8-4598-b793-dd5729debcb7 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:21:12.144352 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-cc1c26a0-31e8-4598-b793-dd5729debcb7 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bba77f6ef42547f1aa8480e27868bd4a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '22ca92a86e6c40d59071e58ce4085568', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:21:12.169841 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-cc1c26a0-31e8-4598-b793-dd5729debcb7 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] 10.222.0.16 "GET /compute/v2.1/servers/f114e124-2903-4f30-b055-9f493e3e9e6c" status: 200 len: 1307 microversion: 2.1 time: 0.247618 Jul 16 21:21:12.175824 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1442/2934] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:21:11 2022] GET /compute/v2.1/servers/f114e124-2903-4f30-b055-9f493e3e9e6c => generated 1307 bytes in 254 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:21:12.450443 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-69d5b824-9a70-4b82-9550-e1a8f3bb8033 tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:21:12.451166 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-69d5b824-9a70-4b82-9550-e1a8f3bb8033 tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] [instance: 6cba399c-58c3-4850-8065-0fbc3301fb2c] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:21:12.469564 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-69d5b824-9a70-4b82-9550-e1a8f3bb8033 tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:21:12.473290 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-69d5b824-9a70-4b82-9550-e1a8f3bb8033 tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:21:12.477935 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-69d5b824-9a70-4b82-9550-e1a8f3bb8033 tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:21:12.477935 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-69d5b824-9a70-4b82-9550-e1a8f3bb8033 tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:21:13.206693 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-9ff262fb-21e3-46e8-8afa-412bf0bb36fb tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:21:13.206693 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-9ff262fb-21e3-46e8-8afa-412bf0bb36fb tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] [instance: f114e124-2903-4f30-b055-9f493e3e9e6c] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:21:13.214180 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9ff262fb-21e3-46e8-8afa-412bf0bb36fb tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:21:13.214902 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9ff262fb-21e3-46e8-8afa-412bf0bb36fb tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:21:13.215898 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9ff262fb-21e3-46e8-8afa-412bf0bb36fb tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:21:13.216498 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9ff262fb-21e3-46e8-8afa-412bf0bb36fb tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:21:13.255565 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-69d5b824-9a70-4b82-9550-e1a8f3bb8033 tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c6ba562873fa4f4fbd0b635f1c9dcdf2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '837a8f9e06e141dcbd5f7bd895016d49', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:21:13.269822 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-69d5b824-9a70-4b82-9550-e1a8f3bb8033 tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] 10.222.0.16 "GET /compute/v2.1/servers/6cba399c-58c3-4850-8065-0fbc3301fb2c" status: 200 len: 1668 microversion: 2.1 time: 0.824919 Jul 16 21:21:13.278824 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1493/2935] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:21:12 2022] GET /compute/v2.1/servers/6cba399c-58c3-4850-8065-0fbc3301fb2c => generated 1668 bytes in 835 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:21:13.429361 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-9ff262fb-21e3-46e8-8afa-412bf0bb36fb tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bba77f6ef42547f1aa8480e27868bd4a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '22ca92a86e6c40d59071e58ce4085568', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:21:13.446378 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-9ff262fb-21e3-46e8-8afa-412bf0bb36fb tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] 10.222.0.16 "GET /compute/v2.1/servers/f114e124-2903-4f30-b055-9f493e3e9e6c" status: 200 len: 1295 microversion: 2.1 time: 0.258006 Jul 16 21:21:13.447201 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1443/2936] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:21:13 2022] GET /compute/v2.1/servers/f114e124-2903-4f30-b055-9f493e3e9e6c => generated 1295 bytes in 260 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:21:14.295402 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-f032e66a-3bab-4ac7-9503-759a581dac04 tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:21:14.295960 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-f032e66a-3bab-4ac7-9503-759a581dac04 tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] [instance: 6cba399c-58c3-4850-8065-0fbc3301fb2c] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:21:14.311904 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f032e66a-3bab-4ac7-9503-759a581dac04 tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:21:14.315121 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f032e66a-3bab-4ac7-9503-759a581dac04 tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:21:14.315121 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f032e66a-3bab-4ac7-9503-759a581dac04 tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:21:14.315121 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f032e66a-3bab-4ac7-9503-759a581dac04 tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:21:14.468803 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-7c2dfbc8-8080-4db9-9669-4499cd29f1ad tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:21:14.469423 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-7c2dfbc8-8080-4db9-9669-4499cd29f1ad tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] [instance: f114e124-2903-4f30-b055-9f493e3e9e6c] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:21:14.491639 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7c2dfbc8-8080-4db9-9669-4499cd29f1ad tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:21:14.492361 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7c2dfbc8-8080-4db9-9669-4499cd29f1ad tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:21:14.496320 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7c2dfbc8-8080-4db9-9669-4499cd29f1ad tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:21:14.500287 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7c2dfbc8-8080-4db9-9669-4499cd29f1ad tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:21:14.560664 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-f032e66a-3bab-4ac7-9503-759a581dac04 tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c6ba562873fa4f4fbd0b635f1c9dcdf2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '837a8f9e06e141dcbd5f7bd895016d49', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:21:14.571396 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-f032e66a-3bab-4ac7-9503-759a581dac04 tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] 10.222.0.16 "GET /compute/v2.1/servers/6cba399c-58c3-4850-8065-0fbc3301fb2c" status: 200 len: 1600 microversion: 2.1 time: 0.280159 Jul 16 21:21:14.585619 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1494/2937] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:21:14 2022] GET /compute/v2.1/servers/6cba399c-58c3-4850-8065-0fbc3301fb2c => generated 1600 bytes in 294 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:21:15.246637 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-7c2dfbc8-8080-4db9-9669-4499cd29f1ad tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bba77f6ef42547f1aa8480e27868bd4a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '22ca92a86e6c40d59071e58ce4085568', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:21:15.260130 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-7c2dfbc8-8080-4db9-9669-4499cd29f1ad tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] 10.222.0.16 "GET /compute/v2.1/servers/f114e124-2903-4f30-b055-9f493e3e9e6c" status: 200 len: 1363 microversion: 2.1 time: 0.794731 Jul 16 21:21:15.262153 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1444/2938] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:21:14 2022] GET /compute/v2.1/servers/f114e124-2903-4f30-b055-9f493e3e9e6c => generated 1363 bytes in 798 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:21:15.582719 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-42cefc29-de31-4476-a7b9-c94619f4c0d3 req-59ab0488-bf98-44c7-9843-e7d7c8ff48bc service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "6cba399c-58c3-4850-8065-0fbc3301fb2c", "name": "network-vif-deleted", "tag": "607e9a0d-96c5-4d29-a42f-425502256e42"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:21:15.592700 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-cf4b8c43-8929-4251-b21a-e12fb18d4d07 tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:21:15.593597 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-cf4b8c43-8929-4251-b21a-e12fb18d4d07 tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] [instance: 6cba399c-58c3-4850-8065-0fbc3301fb2c] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:21:15.608770 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-42cefc29-de31-4476-a7b9-c94619f4c0d3 req-59ab0488-bf98-44c7-9843-e7d7c8ff48bc service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:21:15.612900 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-42cefc29-de31-4476-a7b9-c94619f4c0d3 req-59ab0488-bf98-44c7-9843-e7d7c8ff48bc service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:21:15.617111 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cf4b8c43-8929-4251-b21a-e12fb18d4d07 tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:21:15.617831 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cf4b8c43-8929-4251-b21a-e12fb18d4d07 tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:21:15.624317 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cf4b8c43-8929-4251-b21a-e12fb18d4d07 tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:21:15.625153 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cf4b8c43-8929-4251-b21a-e12fb18d4d07 tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:21:15.656443 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-42cefc29-de31-4476-a7b9-c94619f4c0d3 req-59ab0488-bf98-44c7-9843-e7d7c8ff48bc service nova] Creating event network-vif-deleted:607e9a0d-96c5-4d29-a42f-425502256e42 for instance 6cba399c-58c3-4850-8065-0fbc3301fb2c on nch2-master-1 Jul 16 21:21:15.679875 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-42cefc29-de31-4476-a7b9-c94619f4c0d3 req-59ab0488-bf98-44c7-9843-e7d7c8ff48bc service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.109562 Jul 16 21:21:15.685124 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1495/2939] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:21:15 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 116 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:21:15.776377 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-cf4b8c43-8929-4251-b21a-e12fb18d4d07 tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c6ba562873fa4f4fbd0b635f1c9dcdf2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '837a8f9e06e141dcbd5f7bd895016d49', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:21:15.790547 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-cf4b8c43-8929-4251-b21a-e12fb18d4d07 tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] 10.222.0.16 "GET /compute/v2.1/servers/6cba399c-58c3-4850-8065-0fbc3301fb2c" status: 200 len: 1600 microversion: 2.1 time: 0.202090 Jul 16 21:21:15.800836 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1445/2940] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:21:15 2022] GET /compute/v2.1/servers/6cba399c-58c3-4850-8065-0fbc3301fb2c => generated 1600 bytes in 203 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:21:16.281070 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-360dcdee-ea61-4aea-b5d6-cde34c8476f0 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:21:16.282134 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-360dcdee-ea61-4aea-b5d6-cde34c8476f0 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] [instance: f114e124-2903-4f30-b055-9f493e3e9e6c] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:21:16.294880 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-360dcdee-ea61-4aea-b5d6-cde34c8476f0 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:21:16.295861 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-360dcdee-ea61-4aea-b5d6-cde34c8476f0 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:21:16.299361 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-360dcdee-ea61-4aea-b5d6-cde34c8476f0 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:21:16.301861 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-360dcdee-ea61-4aea-b5d6-cde34c8476f0 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:21:16.600086 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-360dcdee-ea61-4aea-b5d6-cde34c8476f0 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bba77f6ef42547f1aa8480e27868bd4a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '22ca92a86e6c40d59071e58ce4085568', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:21:16.622281 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-360dcdee-ea61-4aea-b5d6-cde34c8476f0 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] 10.222.0.16 "GET /compute/v2.1/servers/f114e124-2903-4f30-b055-9f493e3e9e6c" status: 200 len: 1363 microversion: 2.1 time: 0.345958 Jul 16 21:21:16.639979 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1496/2941] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:21:16 2022] GET /compute/v2.1/servers/f114e124-2903-4f30-b055-9f493e3e9e6c => generated 1363 bytes in 364 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:21:16.811481 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-18b0b19e-59de-494b-9e71-024c512ca599 tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:21:16.812644 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-18b0b19e-59de-494b-9e71-024c512ca599 tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] [instance: 6cba399c-58c3-4850-8065-0fbc3301fb2c] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:21:16.839304 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-18b0b19e-59de-494b-9e71-024c512ca599 tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:21:16.840295 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-18b0b19e-59de-494b-9e71-024c512ca599 tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:21:16.843533 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-18b0b19e-59de-494b-9e71-024c512ca599 tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:21:16.848990 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-18b0b19e-59de-494b-9e71-024c512ca599 tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:21:16.959842 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-18b0b19e-59de-494b-9e71-024c512ca599 tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:21:16.962370 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-18b0b19e-59de-494b-9e71-024c512ca599 tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:21:16.963062 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.objects.instance [None req-18b0b19e-59de-494b-9e71-024c512ca599 tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] Lazy-loading 'fault' on Instance uuid 6cba399c-58c3-4850-8065-0fbc3301fb2c {{(pid=138681) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:21:17.220011 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-18b0b19e-59de-494b-9e71-024c512ca599 tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c6ba562873fa4f4fbd0b635f1c9dcdf2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '837a8f9e06e141dcbd5f7bd895016d49', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:21:17.238207 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-18b0b19e-59de-494b-9e71-024c512ca599 tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] 10.222.0.16 "GET /compute/v2.1/servers/6cba399c-58c3-4850-8065-0fbc3301fb2c" status: 200 len: 1316 microversion: 2.1 time: 0.433347 Jul 16 21:21:17.238813 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1446/2942] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:21:16 2022] GET /compute/v2.1/servers/6cba399c-58c3-4850-8065-0fbc3301fb2c => generated 1316 bytes in 435 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:21:17.665646 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-052c4812-1e6f-4e35-b59e-da8321555d9d tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:21:17.667339 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-052c4812-1e6f-4e35-b59e-da8321555d9d tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] [instance: f114e124-2903-4f30-b055-9f493e3e9e6c] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:21:17.678084 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-052c4812-1e6f-4e35-b59e-da8321555d9d tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:21:17.682510 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-052c4812-1e6f-4e35-b59e-da8321555d9d tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:21:17.682510 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-052c4812-1e6f-4e35-b59e-da8321555d9d tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:21:17.682510 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-052c4812-1e6f-4e35-b59e-da8321555d9d tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:21:18.063364 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-d7dbf646-5d14-4ce6-aa94-a440243d236d req-a034194d-6234-40ca-b0bd-e584d6e87b50 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "f114e124-2903-4f30-b055-9f493e3e9e6c", "tag": "f802c615-4c34-4b94-9946-382deb12b3b2"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:21:18.073952 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell0:SELECT=2 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:21:18.087472 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-d7dbf646-5d14-4ce6-aa94-a440243d236d req-a034194d-6234-40ca-b0bd-e584d6e87b50 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:21:18.088964 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-d7dbf646-5d14-4ce6-aa94-a440243d236d req-a034194d-6234-40ca-b0bd-e584d6e87b50 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:21:18.122784 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-d7dbf646-5d14-4ce6-aa94-a440243d236d req-a034194d-6234-40ca-b0bd-e584d6e87b50 service nova] Creating event network-changed:f802c615-4c34-4b94-9946-382deb12b3b2 for instance f114e124-2903-4f30-b055-9f493e3e9e6c on nch1-master-1 Jul 16 21:21:18.137309 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-d7dbf646-5d14-4ce6-aa94-a440243d236d req-a034194d-6234-40ca-b0bd-e584d6e87b50 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.080177 Jul 16 21:21:18.138418 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1447/2943] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:21:18 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:21:18.147173 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-052c4812-1e6f-4e35-b59e-da8321555d9d tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bba77f6ef42547f1aa8480e27868bd4a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '22ca92a86e6c40d59071e58ce4085568', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:21:18.169964 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-052c4812-1e6f-4e35-b59e-da8321555d9d tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] 10.222.0.16 "GET /compute/v2.1/servers/f114e124-2903-4f30-b055-9f493e3e9e6c" status: 200 len: 1363 microversion: 2.1 time: 0.517686 Jul 16 21:21:18.171179 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1497/2944] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:21:17 2022] GET /compute/v2.1/servers/f114e124-2903-4f30-b055-9f493e3e9e6c => generated 1363 bytes in 519 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:21:18.257218 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-65f6143c-5745-4719-adf5-531a6d857a0a tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:21:18.257999 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-65f6143c-5745-4719-adf5-531a6d857a0a tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] [instance: 6cba399c-58c3-4850-8065-0fbc3301fb2c] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:21:18.275167 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-65f6143c-5745-4719-adf5-531a6d857a0a tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:21:18.275642 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-65f6143c-5745-4719-adf5-531a6d857a0a tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:21:18.276449 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-65f6143c-5745-4719-adf5-531a6d857a0a tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:21:18.279551 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-65f6143c-5745-4719-adf5-531a6d857a0a tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:21:18.312070 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-65f6143c-5745-4719-adf5-531a6d857a0a tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] HTTP exception thrown: Instance 6cba399c-58c3-4850-8065-0fbc3301fb2c could not be found. Jul 16 21:21:18.316178 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-65f6143c-5745-4719-adf5-531a6d857a0a tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] Request method failure captured: Jul 16 21:21:18.316178 ncdv-master-1 devstack@n-api.service[138681]: request: GET /compute/v2.1/servers/6cba399c-58c3-4850-8065-0fbc3301fb2c HTTP/1.1 Jul 16 21:21:18.316178 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:21:18.316178 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:21:18.316178 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:21:18.316178 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 21:21:18.316178 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:21:18.316178 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:21:18.316178 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:21:18.316178 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:21:18.316178 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0yt889rKfSQMojVuncvtjrG2ctJl6ui98IVnPZl1CCCIF_sYvklgTfE_LOA-ICn9g3wci6JzWNZKHxR1BLqFs-TvbXFOKmRCl5d7lhP_MtfaF4t5SpRkkXMLFYdrvb-E1I59Fd8oYnVt5EXZAX89Cs2QOVutY2sfWkTg1bTwnXFZY4I Jul 16 21:21:18.316178 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:21:18.316178 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:21:18.316178 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:21:18.316178 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:21:18.316178 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:21:18.316178 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:21:18.316178 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: 837a8f9e06e141dcbd5f7bd895016d49 Jul 16 21:21:18.316178 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-ServersTestFqdnHostnames-1193871565 Jul 16 21:21:18.316178 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:21:18.316178 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:21:18.316178 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/837a8f9e06e141dcbd5f7bd895016d49"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/837a8f9e06e141dcbd5f7bd895016d49"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_837a8f9e06e141dcbd5f7bd895016d49"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/837a8f9e06e141dcbd5f7bd895016d49"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:21:18.316178 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-ServersTestFqdnHostnames-1193871565 Jul 16 21:21:18.316178 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: 837a8f9e06e141dcbd5f7bd895016d49 Jul 16 21:21:18.316178 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-ServersTestFqdnHostnames-1193871565 Jul 16 21:21:18.316178 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-ServersTestFqdnHostnames-1193871565-project Jul 16 21:21:18.316178 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:21:18.316178 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:21:18.316178 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: c6ba562873fa4f4fbd0b635f1c9dcdf2 Jul 16 21:21:18.316178 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-ServersTestFqdnHostnames-1193871565-project Jul 16 21:21:18.316178 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:21:18.316178 ncdv-master-1 devstack@n-api.service[138681]: exception: Instance 6cba399c-58c3-4850-8065-0fbc3301fb2c could not be found. Jul 16 21:21:18.316178 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': '6cba399c-58c3-4850-8065-0fbc3301fb2c'} Jul 16 21:21:18.316178 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:21:18.316178 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:21:18.316178 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:21:18.316178 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:21:18.316178 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:21:18.316178 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:21:18.316178 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:21:18.316178 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:21:18.316178 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:21:18.316178 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:21:18.316178 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 6cba399c-58c3-4850-8065-0fbc3301fb2c could not be found. Jul 16 21:21:18.316178 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:21:18.316178 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:21:18.316178 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:21:18.316178 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:21:18.316178 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:21:18.316178 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:21:18.316178 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:21:18.316178 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:21:18.316178 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:21:18.316178 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:21:18.316178 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:21:18.316178 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:21:18.316178 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:21:18.316178 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:21:18.316178 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:21:18.316178 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:21:18.316178 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 6cba399c-58c3-4850-8065-0fbc3301fb2c could not be found. Jul 16 21:21:18.316178 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:21:18.316178 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:21:18.316178 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:21:18.320136 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:21:18.320136 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:21:18.320136 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:21:18.320136 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:21:18.320136 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:21:18.320136 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 6cba399c-58c3-4850-8065-0fbc3301fb2c could not be found. Jul 16 21:21:18.320136 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:21:18.320136 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-65f6143c-5745-4719-adf5-531a6d857a0a tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] Returning 404 to user: Instance 6cba399c-58c3-4850-8065-0fbc3301fb2c could not be found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:21:18.320644 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-65f6143c-5745-4719-adf5-531a6d857a0a tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] 10.222.0.16 "GET /compute/v2.1/servers/6cba399c-58c3-4850-8065-0fbc3301fb2c" status: 404 len: 111 microversion: 2.1 time: 0.065858 Jul 16 21:21:18.325703 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1448/2945] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:21:18 2022] GET /compute/v2.1/servers/6cba399c-58c3-4850-8065-0fbc3301fb2c => generated 111 bytes in 75 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:21:18.334401 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-1a2515f1-18c8-408c-906a-aa3d3fa389c0 tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] Calling method '.version_select at 0x7fdda29c8280>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:21:18.451797 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell0:SELECT=2 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:21:18.495206 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-1a2515f1-18c8-408c-906a-aa3d3fa389c0 tempest-ServersTestFqdnHostnames-1193871565 tempest-ServersTestFqdnHostnames-1193871565-project] 10.222.0.16 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1918580201" status: 202 len: 0 microversion: 2.1 time: 0.167704 Jul 16 21:21:18.499038 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1498/2946] 10.222.0.16 () {58 vars in 1220 bytes} [Sat Jul 16 21:21:18 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1918580201 => generated 0 bytes in 171 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 21:21:19.187484 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-9bfe2af3-1708-4c49-968d-806e05e49f88 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:21:19.188110 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-9bfe2af3-1708-4c49-968d-806e05e49f88 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] [instance: f114e124-2903-4f30-b055-9f493e3e9e6c] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:21:19.211172 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9bfe2af3-1708-4c49-968d-806e05e49f88 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:21:19.211649 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9bfe2af3-1708-4c49-968d-806e05e49f88 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:21:19.212640 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9bfe2af3-1708-4c49-968d-806e05e49f88 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:21:19.215783 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9bfe2af3-1708-4c49-968d-806e05e49f88 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:21:19.524950 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-9bfe2af3-1708-4c49-968d-806e05e49f88 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bba77f6ef42547f1aa8480e27868bd4a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '22ca92a86e6c40d59071e58ce4085568', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:21:19.542045 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-9bfe2af3-1708-4c49-968d-806e05e49f88 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] 10.222.0.16 "GET /compute/v2.1/servers/f114e124-2903-4f30-b055-9f493e3e9e6c" status: 200 len: 1363 microversion: 2.1 time: 0.358682 Jul 16 21:21:19.543511 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1449/2947] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:21:19 2022] GET /compute/v2.1/servers/f114e124-2903-4f30-b055-9f493e3e9e6c => generated 1363 bytes in 361 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:21:20.566775 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-bbda81d5-640d-4cae-82b9-592080c4fec2 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:21:20.568488 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-bbda81d5-640d-4cae-82b9-592080c4fec2 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] [instance: f114e124-2903-4f30-b055-9f493e3e9e6c] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:21:20.577801 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-bbda81d5-640d-4cae-82b9-592080c4fec2 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:21:20.579133 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-bbda81d5-640d-4cae-82b9-592080c4fec2 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:21:20.580793 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-bbda81d5-640d-4cae-82b9-592080c4fec2 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:21:20.581420 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-bbda81d5-640d-4cae-82b9-592080c4fec2 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:21:20.864334 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-bbda81d5-640d-4cae-82b9-592080c4fec2 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bba77f6ef42547f1aa8480e27868bd4a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '22ca92a86e6c40d59071e58ce4085568', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:21:20.875477 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-bbda81d5-640d-4cae-82b9-592080c4fec2 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] 10.222.0.16 "GET /compute/v2.1/servers/f114e124-2903-4f30-b055-9f493e3e9e6c" status: 200 len: 1363 microversion: 2.1 time: 0.316339 Jul 16 21:21:20.878189 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1499/2948] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:21:20 2022] GET /compute/v2.1/servers/f114e124-2903-4f30-b055-9f493e3e9e6c => generated 1363 bytes in 320 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:21:21.900088 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-50b09ca9-ec41-424c-8101-9424a0dd8f29 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:21:21.901844 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-50b09ca9-ec41-424c-8101-9424a0dd8f29 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] [instance: f114e124-2903-4f30-b055-9f493e3e9e6c] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:21:21.923405 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell0:SELECT=14 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:21:21.939655 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-50b09ca9-ec41-424c-8101-9424a0dd8f29 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:21:21.941600 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-50b09ca9-ec41-424c-8101-9424a0dd8f29 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:21:21.942720 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-50b09ca9-ec41-424c-8101-9424a0dd8f29 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:21:21.943733 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-50b09ca9-ec41-424c-8101-9424a0dd8f29 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:21:22.207194 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-50b09ca9-ec41-424c-8101-9424a0dd8f29 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bba77f6ef42547f1aa8480e27868bd4a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '22ca92a86e6c40d59071e58ce4085568', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:21:22.228547 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-50b09ca9-ec41-424c-8101-9424a0dd8f29 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] 10.222.0.16 "GET /compute/v2.1/servers/f114e124-2903-4f30-b055-9f493e3e9e6c" status: 200 len: 1363 microversion: 2.1 time: 0.333431 Jul 16 21:21:22.240614 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1450/2949] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:21:21 2022] GET /compute/v2.1/servers/f114e124-2903-4f30-b055-9f493e3e9e6c => generated 1363 bytes in 347 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:21:23.247758 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-f55c9abc-6e71-416d-adc8-190ba4b45054 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:21:23.248772 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-f55c9abc-6e71-416d-adc8-190ba4b45054 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] [instance: f114e124-2903-4f30-b055-9f493e3e9e6c] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:21:23.266243 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f55c9abc-6e71-416d-adc8-190ba4b45054 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:21:23.269917 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f55c9abc-6e71-416d-adc8-190ba4b45054 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:21:23.272077 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f55c9abc-6e71-416d-adc8-190ba4b45054 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:21:23.272783 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f55c9abc-6e71-416d-adc8-190ba4b45054 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:21:23.625910 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-f55c9abc-6e71-416d-adc8-190ba4b45054 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bba77f6ef42547f1aa8480e27868bd4a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '22ca92a86e6c40d59071e58ce4085568', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:21:23.637396 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-f55c9abc-6e71-416d-adc8-190ba4b45054 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] 10.222.0.16 "GET /compute/v2.1/servers/f114e124-2903-4f30-b055-9f493e3e9e6c" status: 200 len: 1363 microversion: 2.1 time: 0.393146 Jul 16 21:21:23.644958 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1500/2950] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:21:23 2022] GET /compute/v2.1/servers/f114e124-2903-4f30-b055-9f493e3e9e6c => generated 1363 bytes in 402 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:21:24.661772 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-c89e54f2-0232-4de8-9be8-7fb258d3b478 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:21:24.662395 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-c89e54f2-0232-4de8-9be8-7fb258d3b478 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] [instance: f114e124-2903-4f30-b055-9f493e3e9e6c] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:21:24.684476 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c89e54f2-0232-4de8-9be8-7fb258d3b478 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:21:24.690540 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c89e54f2-0232-4de8-9be8-7fb258d3b478 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:21:24.691655 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c89e54f2-0232-4de8-9be8-7fb258d3b478 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:21:24.692257 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c89e54f2-0232-4de8-9be8-7fb258d3b478 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:21:25.354792 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-c89e54f2-0232-4de8-9be8-7fb258d3b478 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bba77f6ef42547f1aa8480e27868bd4a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '22ca92a86e6c40d59071e58ce4085568', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:21:25.368052 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-c89e54f2-0232-4de8-9be8-7fb258d3b478 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] 10.222.0.16 "GET /compute/v2.1/servers/f114e124-2903-4f30-b055-9f493e3e9e6c" status: 200 len: 1363 microversion: 2.1 time: 0.711453 Jul 16 21:21:25.369140 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1451/2951] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:21:24 2022] GET /compute/v2.1/servers/f114e124-2903-4f30-b055-9f493e3e9e6c => generated 1363 bytes in 713 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:21:26.355722 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-d379642e-684c-4cb2-9cbd-112e914d90a1 req-5bb1df74-e665-4349-990d-3f3c249e2580 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "f114e124-2903-4f30-b055-9f493e3e9e6c", "name": "network-vif-plugged", "status": "completed", "tag": "f802c615-4c34-4b94-9946-382deb12b3b2"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:21:26.369044 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-d379642e-684c-4cb2-9cbd-112e914d90a1 req-5bb1df74-e665-4349-990d-3f3c249e2580 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:21:26.379201 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-d379642e-684c-4cb2-9cbd-112e914d90a1 req-5bb1df74-e665-4349-990d-3f3c249e2580 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:21:26.391041 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-6b7759ec-5194-4926-af45-7c9ab6f085bd tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:21:26.391758 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-6b7759ec-5194-4926-af45-7c9ab6f085bd tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] [instance: f114e124-2903-4f30-b055-9f493e3e9e6c] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:21:26.404410 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-d379642e-684c-4cb2-9cbd-112e914d90a1 req-5bb1df74-e665-4349-990d-3f3c249e2580 service nova] Creating event network-vif-plugged:f802c615-4c34-4b94-9946-382deb12b3b2 for instance f114e124-2903-4f30-b055-9f493e3e9e6c on nch1-master-1 Jul 16 21:21:26.406567 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6b7759ec-5194-4926-af45-7c9ab6f085bd tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:21:26.407216 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6b7759ec-5194-4926-af45-7c9ab6f085bd tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:21:26.409032 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6b7759ec-5194-4926-af45-7c9ab6f085bd tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:21:26.410801 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6b7759ec-5194-4926-af45-7c9ab6f085bd tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:21:26.421326 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-d379642e-684c-4cb2-9cbd-112e914d90a1 req-5bb1df74-e665-4349-990d-3f3c249e2580 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.069373 Jul 16 21:21:26.425523 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1501/2952] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:21:26 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 75 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:21:26.935232 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-6b7759ec-5194-4926-af45-7c9ab6f085bd tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bba77f6ef42547f1aa8480e27868bd4a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '22ca92a86e6c40d59071e58ce4085568', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:21:26.947157 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-6b7759ec-5194-4926-af45-7c9ab6f085bd tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] 10.222.0.16 "GET /compute/v2.1/servers/f114e124-2903-4f30-b055-9f493e3e9e6c" status: 200 len: 1363 microversion: 2.1 time: 0.565193 Jul 16 21:21:26.953680 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1452/2953] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:21:26 2022] GET /compute/v2.1/servers/f114e124-2903-4f30-b055-9f493e3e9e6c => generated 1363 bytes in 569 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:21:27.264066 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-c958bc40-960c-40bb-bcac-a2c608c9977e tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Action: 'create', calling method: .version_select at 0x7fdd9d1bb040>, body: {"keypair": {"name": "tempest-keypair-492746964", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBE6EOQdYdn5grAPLGftEQvx6ROYjloVvCs82PBE/QCGB3iT6UpwPAT6YQYMN8qEsZ3q2GuwTAYQ0eG3pKbEgnhB+np4+KDuj4GdIfrIpGlUTZoVTCkmJX3MyIl0KnOQqWw=="}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:21:27.279409 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-c958bc40-960c-40bb-bcac-a2c608c9977e tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Getting quotas for project 3e1f6ffc3fb64dba8c4474641e796fb8. Resources: {'key_pairs'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:21:27.284764 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-c958bc40-960c-40bb-bcac-a2c608c9977e tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Getting quotas for user 029f1334779e4bcdb3f4f2aed564a075 and project 3e1f6ffc3fb64dba8c4474641e796fb8. Resources: {'key_pairs'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:21:27.422256 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-c958bc40-960c-40bb-bcac-a2c608c9977e tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] 10.222.0.16 "POST /compute/v2.1/os-keypairs" status: 200 len: 385 microversion: 2.1 time: 0.353509 Jul 16 21:21:27.422906 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1502/2954] 10.222.0.16 () {60 vars in 1158 bytes} [Sat Jul 16 21:21:27 2022] POST /compute/v2.1/os-keypairs => generated 385 bytes in 355 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:21:27.969488 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-5b4b696c-b943-44eb-b53a-68e4fe129050 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:21:27.971302 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-5b4b696c-b943-44eb-b53a-68e4fe129050 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] [instance: f114e124-2903-4f30-b055-9f493e3e9e6c] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:21:27.981060 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5b4b696c-b943-44eb-b53a-68e4fe129050 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:21:27.986603 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5b4b696c-b943-44eb-b53a-68e4fe129050 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:21:27.992192 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5b4b696c-b943-44eb-b53a-68e4fe129050 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:21:27.993035 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5b4b696c-b943-44eb-b53a-68e4fe129050 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:21:28.309942 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-5b4b696c-b943-44eb-b53a-68e4fe129050 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bba77f6ef42547f1aa8480e27868bd4a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '22ca92a86e6c40d59071e58ce4085568', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:21:28.318963 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-5b4b696c-b943-44eb-b53a-68e4fe129050 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] 10.222.0.16 "GET /compute/v2.1/servers/f114e124-2903-4f30-b055-9f493e3e9e6c" status: 200 len: 1363 microversion: 2.1 time: 0.353465 Jul 16 21:21:28.319575 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1453/2955] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:21:27 2022] GET /compute/v2.1/servers/f114e124-2903-4f30-b055-9f493e3e9e6c => generated 1363 bytes in 355 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:21:29.336599 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-89637195-79f3-4573-b637-858c9880907e tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:21:29.337396 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-89637195-79f3-4573-b637-858c9880907e tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] [instance: f114e124-2903-4f30-b055-9f493e3e9e6c] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:21:29.353343 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-89637195-79f3-4573-b637-858c9880907e tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:21:29.355923 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-89637195-79f3-4573-b637-858c9880907e tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:21:29.356837 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-89637195-79f3-4573-b637-858c9880907e tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:21:29.358571 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-89637195-79f3-4573-b637-858c9880907e tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:21:29.741885 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-89637195-79f3-4573-b637-858c9880907e tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bba77f6ef42547f1aa8480e27868bd4a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '22ca92a86e6c40d59071e58ce4085568', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:21:29.756491 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-89637195-79f3-4573-b637-858c9880907e tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] 10.222.0.16 "GET /compute/v2.1/servers/f114e124-2903-4f30-b055-9f493e3e9e6c" status: 200 len: 1550 microversion: 2.1 time: 0.423092 Jul 16 21:21:29.760561 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1503/2956] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:21:29 2022] GET /compute/v2.1/servers/f114e124-2903-4f30-b055-9f493e3e9e6c => generated 1550 bytes in 429 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:21:29.784905 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-f95a540e-18e7-483c-a27c-84e0080e2c4a tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:21:29.786998 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-f95a540e-18e7-483c-a27c-84e0080e2c4a tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] [instance: f114e124-2903-4f30-b055-9f493e3e9e6c] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:21:29.806911 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f95a540e-18e7-483c-a27c-84e0080e2c4a tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:21:29.807757 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f95a540e-18e7-483c-a27c-84e0080e2c4a tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:21:29.808789 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f95a540e-18e7-483c-a27c-84e0080e2c4a tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:21:29.809260 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f95a540e-18e7-483c-a27c-84e0080e2c4a tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:21:30.021187 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-f95a540e-18e7-483c-a27c-84e0080e2c4a tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] 10.222.0.16 "GET /compute/v2.1/servers/f114e124-2903-4f30-b055-9f493e3e9e6c/os-interface" status: 200 len: 285 microversion: 2.1 time: 0.245747 Jul 16 21:21:30.021805 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1454/2957] 10.222.0.16 () {58 vars in 1274 bytes} [Sat Jul 16 21:21:29 2022] GET /compute/v2.1/servers/f114e124-2903-4f30-b055-9f493e3e9e6c/os-interface => generated 285 bytes in 248 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:21:33.879658 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-92aafc2c-d281-4943-978e-86e1c00db892 req-99215274-9e72-428f-9c35-67b36d3bfee8 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "f114e124-2903-4f30-b055-9f493e3e9e6c", "tag": "f802c615-4c34-4b94-9946-382deb12b3b2"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:21:33.896893 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-92aafc2c-d281-4943-978e-86e1c00db892 req-99215274-9e72-428f-9c35-67b36d3bfee8 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:21:33.901161 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-92aafc2c-d281-4943-978e-86e1c00db892 req-99215274-9e72-428f-9c35-67b36d3bfee8 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:21:33.926204 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-92aafc2c-d281-4943-978e-86e1c00db892 req-99215274-9e72-428f-9c35-67b36d3bfee8 service nova] Creating event network-changed:f802c615-4c34-4b94-9946-382deb12b3b2 for instance f114e124-2903-4f30-b055-9f493e3e9e6c on nch1-master-1 Jul 16 21:21:33.941967 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-92aafc2c-d281-4943-978e-86e1c00db892 req-99215274-9e72-428f-9c35-67b36d3bfee8 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.066904 Jul 16 21:21:33.944367 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1504/2958] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:21:33 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 70 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:21:34.101516 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-13c85d9b-4e58-4e31-b7de-15e2c4d21f0e tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachSCSIVolumeTestJSON-server-1763639039", "imageRef": "cffbb16d-5cb8-459a-b417-cf170a24beb9", "flavorRef": "452", "config_drive": true, "networks": [{"uuid": "3f66cb92-004b-485d-9cfd-bbc620e07fbd"}], "security_groups": [{"name": "tempest-securitygroup--1502749186"}], "key_name": "tempest-keypair-492746964", "user_data": "IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIHVidW50dSB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH51YnVudHUvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo="}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:21:34.110207 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_api:SELECT=77,nova_api:INSERT=4,nova_api:UPDATE=1 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:21:34.366705 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.network.neutron [None req-13c85d9b-4e58-4e31-b7de-15e2c4d21f0e tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] validate_networks() for [('3f66cb92-004b-485d-9cfd-bbc620e07fbd', None, None, None, None, None)] {{(pid=138681) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 16 21:21:36.011224 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-13c85d9b-4e58-4e31-b7de-15e2c4d21f0e tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Flavor limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 16 21:21:36.014247 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-13c85d9b-4e58-4e31-b7de-15e2c4d21f0e tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Image limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 16 21:21:36.014247 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-13c85d9b-4e58-4e31-b7de-15e2c4d21f0e tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Flavor pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 16 21:21:36.014717 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-13c85d9b-4e58-4e31-b7de-15e2c4d21f0e tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Image pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 16 21:21:36.015191 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-13c85d9b-4e58-4e31-b7de-15e2c4d21f0e tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 16 21:21:36.048800 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-13c85d9b-4e58-4e31-b7de-15e2c4d21f0e tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:21:36.050609 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-13c85d9b-4e58-4e31-b7de-15e2c4d21f0e tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:21:36.056501 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-13c85d9b-4e58-4e31-b7de-15e2c4d21f0e tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:21:36.058580 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-13c85d9b-4e58-4e31-b7de-15e2c4d21f0e tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:21:36.124053 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-13c85d9b-4e58-4e31-b7de-15e2c4d21f0e tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Getting quotas for project 3e1f6ffc3fb64dba8c4474641e796fb8. Resources: {'ram', 'instances', 'cores'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:21:36.138310 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-13c85d9b-4e58-4e31-b7de-15e2c4d21f0e tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Getting quotas for user 029f1334779e4bcdb3f4f2aed564a075 and project 3e1f6ffc3fb64dba8c4474641e796fb8. Resources: {'ram', 'instances', 'cores'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:21:36.177887 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-13c85d9b-4e58-4e31-b7de-15e2c4d21f0e tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Going to run 1 instances... {{(pid=138681) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 16 21:21:36.189110 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-13c85d9b-4e58-4e31-b7de-15e2c4d21f0e tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] [instance: fa075dcc-e175-449a-9914-d1ee19131f33] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='cffbb16d-5cb8-459a-b417-cf170a24beb9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138681) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 16 21:21:36.390467 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-4bc5c068-3e85-42ec-b4cd-b341fa0ccb15 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ListServersNegativeTestJSON-server-1902559838", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452", "min_count": 3, "networks": [{"uuid": "5f04e26a-7e76-4c3d-a154-6695c537474d"}]}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:21:36.557264 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.network.neutron [None req-4bc5c068-3e85-42ec-b4cd-b341fa0ccb15 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] validate_networks() for [('5f04e26a-7e76-4c3d-a154-6695c537474d', None, None, None, None, None)] {{(pid=138680) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 16 21:21:36.597040 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-13c85d9b-4e58-4e31-b7de-15e2c4d21f0e tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 406 microversion: 2.1 time: 2.503010 Jul 16 21:21:36.598153 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1455/2959] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 21:21:34 2022] POST /compute/v2.1/servers => generated 406 bytes in 2508 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 16 21:21:36.613382 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-f453e364-0009-4b19-96cc-f6ddf662eed7 tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:21:36.617180 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-f453e364-0009-4b19-96cc-f6ddf662eed7 tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] [instance: fa075dcc-e175-449a-9914-d1ee19131f33] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:21:36.850734 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-f453e364-0009-4b19-96cc-f6ddf662eed7 tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '029f1334779e4bcdb3f4f2aed564a075', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e1f6ffc3fb64dba8c4474641e796fb8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:21:36.858629 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-f453e364-0009-4b19-96cc-f6ddf662eed7 tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] 10.222.0.16 "GET /compute/v2.1/servers/fa075dcc-e175-449a-9914-d1ee19131f33" status: 200 len: 1245 microversion: 2.1 time: 0.254027 Jul 16 21:21:36.863383 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1456/2960] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:21:36 2022] GET /compute/v2.1/servers/fa075dcc-e175-449a-9914-d1ee19131f33 => generated 1245 bytes in 260 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:21:37.293005 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell0:SELECT=4 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:21:37.807330 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-4bc5c068-3e85-42ec-b4cd-b341fa0ccb15 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Flavor limits 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 16 21:21:37.808463 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-4bc5c068-3e85-42ec-b4cd-b341fa0ccb15 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Image limits 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 16 21:21:37.809206 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-4bc5c068-3e85-42ec-b4cd-b341fa0ccb15 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Flavor pref 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 16 21:21:37.809937 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-4bc5c068-3e85-42ec-b4cd-b341fa0ccb15 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Image pref 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 16 21:21:37.810651 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-4bc5c068-3e85-42ec-b4cd-b341fa0ccb15 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 16 21:21:37.828780 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4bc5c068-3e85-42ec-b4cd-b341fa0ccb15 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:21:37.830208 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4bc5c068-3e85-42ec-b4cd-b341fa0ccb15 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:21:37.831451 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4bc5c068-3e85-42ec-b4cd-b341fa0ccb15 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:21:37.832223 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4bc5c068-3e85-42ec-b4cd-b341fa0ccb15 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:21:37.858787 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=89,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:21:37.876899 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-4bc5c068-3e85-42ec-b4cd-b341fa0ccb15 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Getting quotas for project c95a2d1e0a034c7ca4a2fcb9fd060d53. Resources: {'ram', 'instances', 'cores'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:21:37.879857 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-b57b83f5-e364-4112-936a-813e08b67ed1 tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:21:37.880316 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-4bc5c068-3e85-42ec-b4cd-b341fa0ccb15 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Getting quotas for user fa4c2791e14d4f88883a0af422e4ff0c and project c95a2d1e0a034c7ca4a2fcb9fd060d53. Resources: {'ram', 'instances', 'cores'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:21:37.880521 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-b57b83f5-e364-4112-936a-813e08b67ed1 tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] [instance: fa075dcc-e175-449a-9914-d1ee19131f33] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:21:37.895851 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-4bc5c068-3e85-42ec-b4cd-b341fa0ccb15 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Going to run 3 instances... {{(pid=138680) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 16 21:21:37.898978 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-4bc5c068-3e85-42ec-b4cd-b341fa0ccb15 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] [instance: 8acee151-2b52-4ca3-b2e1-18a19834b0e2] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138680) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 16 21:21:38.018345 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-b57b83f5-e364-4112-936a-813e08b67ed1 tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '029f1334779e4bcdb3f4f2aed564a075', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e1f6ffc3fb64dba8c4474641e796fb8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:21:38.033262 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-b57b83f5-e364-4112-936a-813e08b67ed1 tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] 10.222.0.16 "GET /compute/v2.1/servers/fa075dcc-e175-449a-9914-d1ee19131f33" status: 200 len: 1245 microversion: 2.1 time: 0.156780 Jul 16 21:21:38.046783 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1457/2961] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:21:37 2022] GET /compute/v2.1/servers/fa075dcc-e175-449a-9914-d1ee19131f33 => generated 1245 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:21:38.082941 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-4bc5c068-3e85-42ec-b4cd-b341fa0ccb15 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] [instance: 9e7787d1-264a-4707-8604-3e8b70133376] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9',instance=,instance_uuid=8acee151-2b52-4ca3-b2e1-18a19834b0e2,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138680) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 16 21:21:38.316704 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-4bc5c068-3e85-42ec-b4cd-b341fa0ccb15 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] [instance: 94d9ca2e-1c4d-49ce-918e-02479909b75b] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9',instance=,instance_uuid=9e7787d1-264a-4707-8604-3e8b70133376,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138680) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 16 21:21:38.745725 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-4bc5c068-3e85-42ec-b4cd-b341fa0ccb15 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 2.630362 Jul 16 21:21:38.757462 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1505/2962] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 21:21:36 2022] POST /compute/v2.1/servers => generated 380 bytes in 2643 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 16 21:21:38.757645 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-d75aba42-a48d-4f8e-b837-cf60b9d1c9aa tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerRescueTestJSONUnderV235-server-1509881048", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "}]}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:21:38.762582 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-9ed33476-aa3a-4e08-b6e2-6b6dfff96c81 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:21:38.786647 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-9ed33476-aa3a-4e08-b6e2-6b6dfff96c81 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Policy check for os_compute_api:servers:allow_all_filters failed with credentials {'is_admin': False, 'user_id': 'fa4c2791e14d4f88883a0af422e4ff0c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c95a2d1e0a034c7ca4a2fcb9fd060d53', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:21:38.789989 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-9ed33476-aa3a-4e08-b6e2-6b6dfff96c81 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Searching by: {'deleted': False, 'project_id': 'c95a2d1e0a034c7ca4a2fcb9fd060d53'} {{(pid=138680) get_all /opt/stack/nova/nova/compute/api.py:2962}} Jul 16 21:21:38.829904 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9ed33476-aa3a-4e08-b6e2-6b6dfff96c81 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:21:38.831450 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9ed33476-aa3a-4e08-b6e2-6b6dfff96c81 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:21:38.833557 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9ed33476-aa3a-4e08-b6e2-6b6dfff96c81 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:21:38.835207 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9ed33476-aa3a-4e08-b6e2-6b6dfff96c81 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:21:39.032851 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.network.neutron [None req-d75aba42-a48d-4f8e-b837-cf60b9d1c9aa tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] validate_networks() for [('728dbccf-f950-46d2-9c0b-1f3c0f2fd7ae', None, None, None, None, None)] {{(pid=138681) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 16 21:21:39.034566 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-9ed33476-aa3a-4e08-b6e2-6b6dfff96c81 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] 10.222.0.16 "GET /compute/v2.1/servers" status: 200 len: 1018 microversion: 2.1 time: 0.142058 Jul 16 21:21:39.034566 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1506/2963] 10.222.0.16 () {58 vars in 1124 bytes} [Sat Jul 16 21:21:38 2022] GET /compute/v2.1/servers => generated 1018 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:21:39.034566 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-2be1d167-cbc7-4ce7-be48-18213ea28a42 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:21:39.034566 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-2be1d167-cbc7-4ce7-be48-18213ea28a42 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] [instance: 94d9ca2e-1c4d-49ce-918e-02479909b75b] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:21:39.577840 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-2be1d167-cbc7-4ce7-be48-18213ea28a42 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa4c2791e14d4f88883a0af422e4ff0c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c95a2d1e0a034c7ca4a2fcb9fd060d53', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:21:39.597076 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-2be1d167-cbc7-4ce7-be48-18213ea28a42 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] 10.222.0.16 "GET /compute/v2.1/servers/94d9ca2e-1c4d-49ce-918e-02479909b75b" status: 200 len: 1223 microversion: 2.1 time: 0.682587 Jul 16 21:21:39.605920 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1507/2964] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:21:38 2022] GET /compute/v2.1/servers/94d9ca2e-1c4d-49ce-918e-02479909b75b => generated 1223 bytes in 692 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:21:39.629233 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-cb1348a1-28b9-4dc8-b612-726abc8f1786 tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:21:39.630272 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-cb1348a1-28b9-4dc8-b612-726abc8f1786 tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] [instance: fa075dcc-e175-449a-9914-d1ee19131f33] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:21:39.674327 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cb1348a1-28b9-4dc8-b612-726abc8f1786 tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:21:39.675994 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cb1348a1-28b9-4dc8-b612-726abc8f1786 tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:21:39.678311 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cb1348a1-28b9-4dc8-b612-726abc8f1786 tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:21:39.679504 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cb1348a1-28b9-4dc8-b612-726abc8f1786 tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:21:39.895259 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-cb1348a1-28b9-4dc8-b612-726abc8f1786 tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '029f1334779e4bcdb3f4f2aed564a075', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e1f6ffc3fb64dba8c4474641e796fb8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:21:39.914020 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-cb1348a1-28b9-4dc8-b612-726abc8f1786 tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] 10.222.0.16 "GET /compute/v2.1/servers/fa075dcc-e175-449a-9914-d1ee19131f33" status: 200 len: 1241 microversion: 2.1 time: 0.295038 Jul 16 21:21:39.914020 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1508/2965] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:21:39 2022] GET /compute/v2.1/servers/fa075dcc-e175-449a-9914-d1ee19131f33 => generated 1241 bytes in 303 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:21:40.624160 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-3fb0f8f5-5a0c-4fe5-9482-154206600438 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:21:40.624780 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-3fb0f8f5-5a0c-4fe5-9482-154206600438 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] [instance: 94d9ca2e-1c4d-49ce-918e-02479909b75b] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:21:40.736008 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-d75aba42-a48d-4f8e-b837-cf60b9d1c9aa tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Flavor limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 16 21:21:40.736571 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-d75aba42-a48d-4f8e-b837-cf60b9d1c9aa tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Image limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 16 21:21:40.737246 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-d75aba42-a48d-4f8e-b837-cf60b9d1c9aa tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Flavor pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 16 21:21:40.737821 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-d75aba42-a48d-4f8e-b837-cf60b9d1c9aa tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Image pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 16 21:21:40.738398 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-d75aba42-a48d-4f8e-b837-cf60b9d1c9aa tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 16 21:21:40.792119 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d75aba42-a48d-4f8e-b837-cf60b9d1c9aa tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:21:40.792828 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d75aba42-a48d-4f8e-b837-cf60b9d1c9aa tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:21:40.793354 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d75aba42-a48d-4f8e-b837-cf60b9d1c9aa tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:21:40.795464 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d75aba42-a48d-4f8e-b837-cf60b9d1c9aa tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:21:40.836808 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=82,nova_cell1:INSERT=1 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:21:40.878728 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-d75aba42-a48d-4f8e-b837-cf60b9d1c9aa tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Getting quotas for project 65644a8df5d84df3b35c894560e0a5c4. Resources: {'ram', 'instances', 'cores'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:21:40.892899 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-d75aba42-a48d-4f8e-b837-cf60b9d1c9aa tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Getting quotas for user aaf2292818b749f0ae43c66bca929be9 and project 65644a8df5d84df3b35c894560e0a5c4. Resources: {'ram', 'instances', 'cores'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:21:40.921676 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-d75aba42-a48d-4f8e-b837-cf60b9d1c9aa tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Going to run 1 instances... {{(pid=138681) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 16 21:21:40.925442 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-d75aba42-a48d-4f8e-b837-cf60b9d1c9aa tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] [instance: 583e0c15-3053-429d-81bf-95a4d1f99cbf] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138681) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 16 21:21:41.017155 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-3fb0f8f5-5a0c-4fe5-9482-154206600438 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa4c2791e14d4f88883a0af422e4ff0c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c95a2d1e0a034c7ca4a2fcb9fd060d53', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:21:41.034955 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-3fb0f8f5-5a0c-4fe5-9482-154206600438 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] 10.222.0.16 "GET /compute/v2.1/servers/94d9ca2e-1c4d-49ce-918e-02479909b75b" status: 200 len: 1223 microversion: 2.1 time: 0.415684 Jul 16 21:21:41.035603 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1509/2966] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:21:40 2022] GET /compute/v2.1/servers/94d9ca2e-1c4d-49ce-918e-02479909b75b => generated 1223 bytes in 417 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:21:41.085613 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-ee63637b-96aa-465d-88b1-ffe8f7b155fc tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:21:41.086656 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-ee63637b-96aa-465d-88b1-ffe8f7b155fc tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] [instance: fa075dcc-e175-449a-9914-d1ee19131f33] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:21:41.118889 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ee63637b-96aa-465d-88b1-ffe8f7b155fc tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:21:41.124298 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ee63637b-96aa-465d-88b1-ffe8f7b155fc tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:21:41.129052 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ee63637b-96aa-465d-88b1-ffe8f7b155fc tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:21:41.129707 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ee63637b-96aa-465d-88b1-ffe8f7b155fc tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:21:41.225871 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-d75aba42-a48d-4f8e-b837-cf60b9d1c9aa tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 383 microversion: 2.1 time: 3.141655 Jul 16 21:21:41.231632 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1458/2967] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 21:21:38 2022] POST /compute/v2.1/servers => generated 383 bytes in 3146 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 16 21:21:41.253572 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-f5647235-7bb1-484e-84c4-c84bf9ad743c tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:21:41.254203 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-f5647235-7bb1-484e-84c4-c84bf9ad743c tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] [instance: 583e0c15-3053-429d-81bf-95a4d1f99cbf] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:21:41.567217 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-ee63637b-96aa-465d-88b1-ffe8f7b155fc tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '029f1334779e4bcdb3f4f2aed564a075', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e1f6ffc3fb64dba8c4474641e796fb8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:21:41.578274 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-ee63637b-96aa-465d-88b1-ffe8f7b155fc tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] 10.222.0.16 "GET /compute/v2.1/servers/fa075dcc-e175-449a-9914-d1ee19131f33" status: 200 len: 1315 microversion: 2.1 time: 0.525935 Jul 16 21:21:41.582507 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1510/2968] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:21:41 2022] GET /compute/v2.1/servers/fa075dcc-e175-449a-9914-d1ee19131f33 => generated 1315 bytes in 532 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:21:41.645388 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-f5647235-7bb1-484e-84c4-c84bf9ad743c tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'aaf2292818b749f0ae43c66bca929be9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '65644a8df5d84df3b35c894560e0a5c4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:21:41.660531 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-f5647235-7bb1-484e-84c4-c84bf9ad743c tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] 10.222.0.16 "GET /compute/v2.1/servers/583e0c15-3053-429d-81bf-95a4d1f99cbf" status: 200 len: 1223 microversion: 2.1 time: 0.415542 Jul 16 21:21:41.663269 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1459/2969] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:21:41 2022] GET /compute/v2.1/servers/583e0c15-3053-429d-81bf-95a4d1f99cbf => generated 1223 bytes in 424 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:21:42.054487 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-9738ff5f-f293-442e-9657-6c3b590e37e6 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:21:42.054942 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-9738ff5f-f293-442e-9657-6c3b590e37e6 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] [instance: 94d9ca2e-1c4d-49ce-918e-02479909b75b] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:21:42.237497 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-9738ff5f-f293-442e-9657-6c3b590e37e6 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa4c2791e14d4f88883a0af422e4ff0c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c95a2d1e0a034c7ca4a2fcb9fd060d53', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:21:42.251706 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-9738ff5f-f293-442e-9657-6c3b590e37e6 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] 10.222.0.16 "GET /compute/v2.1/servers/94d9ca2e-1c4d-49ce-918e-02479909b75b" status: 200 len: 1223 microversion: 2.1 time: 0.201690 Jul 16 21:21:42.253376 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1511/2970] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:21:42 2022] GET /compute/v2.1/servers/94d9ca2e-1c4d-49ce-918e-02479909b75b => generated 1223 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:21:42.611788 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-cce32517-4cb4-4613-b6c9-24410dc5bb54 tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:21:42.612314 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-cce32517-4cb4-4613-b6c9-24410dc5bb54 tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] [instance: fa075dcc-e175-449a-9914-d1ee19131f33] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:21:42.633317 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cce32517-4cb4-4613-b6c9-24410dc5bb54 tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:21:42.637446 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cce32517-4cb4-4613-b6c9-24410dc5bb54 tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:21:42.638365 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cce32517-4cb4-4613-b6c9-24410dc5bb54 tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:21:42.638831 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cce32517-4cb4-4613-b6c9-24410dc5bb54 tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:21:42.681357 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-75220856-1638-418a-9ca5-50a663e8cf5c tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:21:42.682639 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-75220856-1638-418a-9ca5-50a663e8cf5c tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] [instance: 583e0c15-3053-429d-81bf-95a4d1f99cbf] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:21:43.226739 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-75220856-1638-418a-9ca5-50a663e8cf5c tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'aaf2292818b749f0ae43c66bca929be9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '65644a8df5d84df3b35c894560e0a5c4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:21:43.239304 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-75220856-1638-418a-9ca5-50a663e8cf5c tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] 10.222.0.16 "GET /compute/v2.1/servers/583e0c15-3053-429d-81bf-95a4d1f99cbf" status: 200 len: 1223 microversion: 2.1 time: 0.561831 Jul 16 21:21:43.242414 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1512/2971] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:21:42 2022] GET /compute/v2.1/servers/583e0c15-3053-429d-81bf-95a4d1f99cbf => generated 1223 bytes in 564 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:21:43.274907 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-8fdc6a54-7e14-46f1-87c1-fab6b2910a54 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:21:43.276729 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-8fdc6a54-7e14-46f1-87c1-fab6b2910a54 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] [instance: 94d9ca2e-1c4d-49ce-918e-02479909b75b] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:21:43.545948 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-8fdc6a54-7e14-46f1-87c1-fab6b2910a54 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa4c2791e14d4f88883a0af422e4ff0c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c95a2d1e0a034c7ca4a2fcb9fd060d53', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:21:43.562096 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-8fdc6a54-7e14-46f1-87c1-fab6b2910a54 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] 10.222.0.16 "GET /compute/v2.1/servers/94d9ca2e-1c4d-49ce-918e-02479909b75b" status: 200 len: 1223 microversion: 2.1 time: 0.294738 Jul 16 21:21:43.567491 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1513/2972] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:21:43 2022] GET /compute/v2.1/servers/94d9ca2e-1c4d-49ce-918e-02479909b75b => generated 1223 bytes in 304 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:21:43.896278 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-cce32517-4cb4-4613-b6c9-24410dc5bb54 tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '029f1334779e4bcdb3f4f2aed564a075', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e1f6ffc3fb64dba8c4474641e796fb8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:21:43.911729 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-cce32517-4cb4-4613-b6c9-24410dc5bb54 tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] 10.222.0.16 "GET /compute/v2.1/servers/fa075dcc-e175-449a-9914-d1ee19131f33" status: 200 len: 1371 microversion: 2.1 time: 1.312443 Jul 16 21:21:43.926706 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1460/2973] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:21:42 2022] GET /compute/v2.1/servers/fa075dcc-e175-449a-9914-d1ee19131f33 => generated 1371 bytes in 1328 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:21:44.257579 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-98d10157-52fe-4139-a1a9-a773a461058c tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:21:44.258446 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-98d10157-52fe-4139-a1a9-a773a461058c tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] [instance: 583e0c15-3053-429d-81bf-95a4d1f99cbf] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:21:44.276690 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-98d10157-52fe-4139-a1a9-a773a461058c tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:21:44.288300 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-98d10157-52fe-4139-a1a9-a773a461058c tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:21:44.292513 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-98d10157-52fe-4139-a1a9-a773a461058c tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:21:44.293601 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-98d10157-52fe-4139-a1a9-a773a461058c tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:21:44.473568 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-98d10157-52fe-4139-a1a9-a773a461058c tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'aaf2292818b749f0ae43c66bca929be9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '65644a8df5d84df3b35c894560e0a5c4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:21:44.499567 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-98d10157-52fe-4139-a1a9-a773a461058c tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] 10.222.0.16 "GET /compute/v2.1/servers/583e0c15-3053-429d-81bf-95a4d1f99cbf" status: 200 len: 1219 microversion: 2.1 time: 0.238940 Jul 16 21:21:44.502771 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1514/2974] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:21:44 2022] GET /compute/v2.1/servers/583e0c15-3053-429d-81bf-95a4d1f99cbf => generated 1219 bytes in 251 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:21:44.584751 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-8d0cdf33-dc0c-4c15-891b-f6c3e2ca78de tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:21:44.586064 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-8d0cdf33-dc0c-4c15-891b-f6c3e2ca78de tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] [instance: 94d9ca2e-1c4d-49ce-918e-02479909b75b] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:21:44.616388 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8d0cdf33-dc0c-4c15-891b-f6c3e2ca78de tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:21:44.618944 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8d0cdf33-dc0c-4c15-891b-f6c3e2ca78de tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:21:44.622555 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8d0cdf33-dc0c-4c15-891b-f6c3e2ca78de tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:21:44.626232 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8d0cdf33-dc0c-4c15-891b-f6c3e2ca78de tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:21:44.942180 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-95b58559-1d5e-42e9-99f3-0345c0f03778 tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:21:44.943071 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-95b58559-1d5e-42e9-99f3-0345c0f03778 tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] [instance: fa075dcc-e175-449a-9914-d1ee19131f33] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:21:44.951788 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_api:SELECT=81,nova_api:DELETE=1,nova_api:INSERT=10 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:21:44.956859 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-95b58559-1d5e-42e9-99f3-0345c0f03778 tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:21:44.960836 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-95b58559-1d5e-42e9-99f3-0345c0f03778 tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:21:44.962116 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-95b58559-1d5e-42e9-99f3-0345c0f03778 tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:21:44.963410 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-95b58559-1d5e-42e9-99f3-0345c0f03778 tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:21:45.011067 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-8d0cdf33-dc0c-4c15-891b-f6c3e2ca78de tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa4c2791e14d4f88883a0af422e4ff0c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c95a2d1e0a034c7ca4a2fcb9fd060d53', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:21:45.035615 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-8d0cdf33-dc0c-4c15-891b-f6c3e2ca78de tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] 10.222.0.16 "GET /compute/v2.1/servers/94d9ca2e-1c4d-49ce-918e-02479909b75b" status: 200 len: 1227 microversion: 2.1 time: 0.453744 Jul 16 21:21:45.036767 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1461/2975] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:21:44 2022] GET /compute/v2.1/servers/94d9ca2e-1c4d-49ce-918e-02479909b75b => generated 1227 bytes in 460 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:21:45.521386 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-d6be0f37-5365-4ad5-8c2c-241aabbbe5d3 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:21:45.521610 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-d6be0f37-5365-4ad5-8c2c-241aabbbe5d3 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] [instance: 583e0c15-3053-429d-81bf-95a4d1f99cbf] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:21:45.538549 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d6be0f37-5365-4ad5-8c2c-241aabbbe5d3 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:21:45.538549 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d6be0f37-5365-4ad5-8c2c-241aabbbe5d3 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:21:45.539205 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d6be0f37-5365-4ad5-8c2c-241aabbbe5d3 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:21:45.539205 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d6be0f37-5365-4ad5-8c2c-241aabbbe5d3 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:21:45.697011 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-d6be0f37-5365-4ad5-8c2c-241aabbbe5d3 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'aaf2292818b749f0ae43c66bca929be9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '65644a8df5d84df3b35c894560e0a5c4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:21:45.711656 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-d6be0f37-5365-4ad5-8c2c-241aabbbe5d3 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] 10.222.0.16 "GET /compute/v2.1/servers/583e0c15-3053-429d-81bf-95a4d1f99cbf" status: 200 len: 1275 microversion: 2.1 time: 0.196410 Jul 16 21:21:45.712273 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1462/2976] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:21:45 2022] GET /compute/v2.1/servers/583e0c15-3053-429d-81bf-95a4d1f99cbf => generated 1275 bytes in 198 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:21:45.768490 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-95b58559-1d5e-42e9-99f3-0345c0f03778 tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '029f1334779e4bcdb3f4f2aed564a075', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e1f6ffc3fb64dba8c4474641e796fb8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:21:45.777822 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-95b58559-1d5e-42e9-99f3-0345c0f03778 tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] 10.222.0.16 "GET /compute/v2.1/servers/fa075dcc-e175-449a-9914-d1ee19131f33" status: 200 len: 1371 microversion: 2.1 time: 0.846471 Jul 16 21:21:45.788954 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1515/2977] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:21:44 2022] GET /compute/v2.1/servers/fa075dcc-e175-449a-9914-d1ee19131f33 => generated 1371 bytes in 857 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:21:46.053748 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-7f69ca37-b90b-4903-bfd1-93b12aed2c68 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:21:46.054413 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-7f69ca37-b90b-4903-bfd1-93b12aed2c68 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] [instance: 94d9ca2e-1c4d-49ce-918e-02479909b75b] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:21:46.072424 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7f69ca37-b90b-4903-bfd1-93b12aed2c68 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:21:46.073469 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7f69ca37-b90b-4903-bfd1-93b12aed2c68 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:21:46.074455 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7f69ca37-b90b-4903-bfd1-93b12aed2c68 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:21:46.075093 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7f69ca37-b90b-4903-bfd1-93b12aed2c68 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:21:46.555518 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-7f69ca37-b90b-4903-bfd1-93b12aed2c68 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa4c2791e14d4f88883a0af422e4ff0c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c95a2d1e0a034c7ca4a2fcb9fd060d53', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:21:46.584789 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-7f69ca37-b90b-4903-bfd1-93b12aed2c68 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] 10.222.0.16 "GET /compute/v2.1/servers/94d9ca2e-1c4d-49ce-918e-02479909b75b" status: 200 len: 1275 microversion: 2.1 time: 0.535265 Jul 16 21:21:46.623687 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1463/2978] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:21:46 2022] GET /compute/v2.1/servers/94d9ca2e-1c4d-49ce-918e-02479909b75b => generated 1275 bytes in 575 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:21:46.728878 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-db675168-7b28-439d-bee8-684c7f6efaec req-16efc996-6717-4255-a442-21335f09e2fe service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "fa075dcc-e175-449a-9914-d1ee19131f33", "tag": "49869004-34cd-4d2b-8bf2-b7cbd8448d51"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:21:46.730129 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-860bcb9c-e4d0-40dc-91c4-a05cd6cc776f tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:21:46.730763 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-860bcb9c-e4d0-40dc-91c4-a05cd6cc776f tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] [instance: 583e0c15-3053-429d-81bf-95a4d1f99cbf] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:21:46.745730 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-860bcb9c-e4d0-40dc-91c4-a05cd6cc776f tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:21:46.751489 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-860bcb9c-e4d0-40dc-91c4-a05cd6cc776f tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:21:46.754474 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-860bcb9c-e4d0-40dc-91c4-a05cd6cc776f tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:21:46.759264 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-860bcb9c-e4d0-40dc-91c4-a05cd6cc776f tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:21:46.768244 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-db675168-7b28-439d-bee8-684c7f6efaec req-16efc996-6717-4255-a442-21335f09e2fe service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:21:46.769811 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-db675168-7b28-439d-bee8-684c7f6efaec req-16efc996-6717-4255-a442-21335f09e2fe service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:21:46.814090 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-db675168-7b28-439d-bee8-684c7f6efaec req-16efc996-6717-4255-a442-21335f09e2fe service nova] Creating event network-changed:49869004-34cd-4d2b-8bf2-b7cbd8448d51 for instance fa075dcc-e175-449a-9914-d1ee19131f33 on nch2-master-1 Jul 16 21:21:46.826427 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-db675168-7b28-439d-bee8-684c7f6efaec req-16efc996-6717-4255-a442-21335f09e2fe service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.112729 Jul 16 21:21:46.827591 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1516/2979] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:21:46 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 115 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 16 21:21:46.840761 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-8fdd288e-4a17-4091-bdf9-ac0861437d31 tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:21:46.841506 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-8fdd288e-4a17-4091-bdf9-ac0861437d31 tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] [instance: fa075dcc-e175-449a-9914-d1ee19131f33] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:21:46.868151 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8fdd288e-4a17-4091-bdf9-ac0861437d31 tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:21:46.868629 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8fdd288e-4a17-4091-bdf9-ac0861437d31 tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:21:46.869924 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8fdd288e-4a17-4091-bdf9-ac0861437d31 tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:21:46.869924 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8fdd288e-4a17-4091-bdf9-ac0861437d31 tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:21:47.404114 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-860bcb9c-e4d0-40dc-91c4-a05cd6cc776f tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'aaf2292818b749f0ae43c66bca929be9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '65644a8df5d84df3b35c894560e0a5c4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:21:47.431715 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-860bcb9c-e4d0-40dc-91c4-a05cd6cc776f tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] 10.222.0.16 "GET /compute/v2.1/servers/583e0c15-3053-429d-81bf-95a4d1f99cbf" status: 200 len: 1281 microversion: 2.1 time: 0.706733 Jul 16 21:21:47.447355 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1464/2980] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:21:46 2022] GET /compute/v2.1/servers/583e0c15-3053-429d-81bf-95a4d1f99cbf => generated 1281 bytes in 713 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:21:47.632626 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-37e4676c-7063-4ad0-996d-b075de2ae51c tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:21:47.633213 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-37e4676c-7063-4ad0-996d-b075de2ae51c tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] [instance: 94d9ca2e-1c4d-49ce-918e-02479909b75b] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:21:47.653433 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-37e4676c-7063-4ad0-996d-b075de2ae51c tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:21:47.655396 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-37e4676c-7063-4ad0-996d-b075de2ae51c tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:21:47.660632 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-37e4676c-7063-4ad0-996d-b075de2ae51c tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:21:47.661305 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-37e4676c-7063-4ad0-996d-b075de2ae51c tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:21:48.404212 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-37e4676c-7063-4ad0-996d-b075de2ae51c tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa4c2791e14d4f88883a0af422e4ff0c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c95a2d1e0a034c7ca4a2fcb9fd060d53', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:21:48.431767 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-37e4676c-7063-4ad0-996d-b075de2ae51c tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] 10.222.0.16 "GET /compute/v2.1/servers/94d9ca2e-1c4d-49ce-918e-02479909b75b" status: 200 len: 1281 microversion: 2.1 time: 0.792903 Jul 16 21:21:48.431767 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1465/2981] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:21:47 2022] GET /compute/v2.1/servers/94d9ca2e-1c4d-49ce-918e-02479909b75b => generated 1281 bytes in 794 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:21:48.458228 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-0d341a27-99e5-4397-9bcf-84fee80361cd tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:21:48.458878 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-0d341a27-99e5-4397-9bcf-84fee80361cd tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] [instance: 583e0c15-3053-429d-81bf-95a4d1f99cbf] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:21:48.480849 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0d341a27-99e5-4397-9bcf-84fee80361cd tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:21:48.481488 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0d341a27-99e5-4397-9bcf-84fee80361cd tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:21:48.483669 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0d341a27-99e5-4397-9bcf-84fee80361cd tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:21:48.484263 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0d341a27-99e5-4397-9bcf-84fee80361cd tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:21:48.870442 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell0:SELECT=4 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:21:49.328037 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-8fdd288e-4a17-4091-bdf9-ac0861437d31 tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '029f1334779e4bcdb3f4f2aed564a075', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e1f6ffc3fb64dba8c4474641e796fb8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:21:49.337164 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-8fdd288e-4a17-4091-bdf9-ac0861437d31 tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] 10.222.0.16 "GET /compute/v2.1/servers/fa075dcc-e175-449a-9914-d1ee19131f33" status: 200 len: 1371 microversion: 2.1 time: 2.508315 Jul 16 21:21:49.342543 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1517/2982] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:21:46 2022] GET /compute/v2.1/servers/fa075dcc-e175-449a-9914-d1ee19131f33 => generated 1371 bytes in 2514 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:21:49.373482 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-0d341a27-99e5-4397-9bcf-84fee80361cd tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'aaf2292818b749f0ae43c66bca929be9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '65644a8df5d84df3b35c894560e0a5c4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:21:49.386100 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-0d341a27-99e5-4397-9bcf-84fee80361cd tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] 10.222.0.16 "GET /compute/v2.1/servers/583e0c15-3053-429d-81bf-95a4d1f99cbf" status: 200 len: 1281 microversion: 2.1 time: 0.932261 Jul 16 21:21:49.398197 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1466/2983] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:21:48 2022] GET /compute/v2.1/servers/583e0c15-3053-429d-81bf-95a4d1f99cbf => generated 1281 bytes in 945 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:21:49.453071 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-56d874a2-25fd-4d9a-8aeb-ebde3c14175d tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:21:49.460000 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-56d874a2-25fd-4d9a-8aeb-ebde3c14175d tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] [instance: 94d9ca2e-1c4d-49ce-918e-02479909b75b] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:21:49.486866 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-56d874a2-25fd-4d9a-8aeb-ebde3c14175d tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:21:49.487377 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-56d874a2-25fd-4d9a-8aeb-ebde3c14175d tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:21:49.488199 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-56d874a2-25fd-4d9a-8aeb-ebde3c14175d tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:21:49.494726 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-56d874a2-25fd-4d9a-8aeb-ebde3c14175d tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:21:49.774883 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-56d874a2-25fd-4d9a-8aeb-ebde3c14175d tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa4c2791e14d4f88883a0af422e4ff0c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c95a2d1e0a034c7ca4a2fcb9fd060d53', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:21:49.789422 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-56d874a2-25fd-4d9a-8aeb-ebde3c14175d tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] 10.222.0.16 "GET /compute/v2.1/servers/94d9ca2e-1c4d-49ce-918e-02479909b75b" status: 200 len: 1281 microversion: 2.1 time: 0.348886 Jul 16 21:21:49.790661 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1467/2984] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:21:49 2022] GET /compute/v2.1/servers/94d9ca2e-1c4d-49ce-918e-02479909b75b => generated 1281 bytes in 354 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:21:49.911058 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-428a3691-1870-46d1-99b6-89054f8bd6f7 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-DeleteServersTestJSON-server-622952095", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452", "networks": [{"uuid": "fc3b835d-0a7f-4948-9fcb-1edfd6b9a8df"}]}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:21:50.359101 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-1e84b076-9a2d-4c09-b47e-67f5293e2e16 tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:21:50.368867 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-1e84b076-9a2d-4c09-b47e-67f5293e2e16 tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] [instance: fa075dcc-e175-449a-9914-d1ee19131f33] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:21:50.391591 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1e84b076-9a2d-4c09-b47e-67f5293e2e16 tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:21:50.392321 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1e84b076-9a2d-4c09-b47e-67f5293e2e16 tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:21:50.402843 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1e84b076-9a2d-4c09-b47e-67f5293e2e16 tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:21:50.404801 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1e84b076-9a2d-4c09-b47e-67f5293e2e16 tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:21:50.660283 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.network.neutron [None req-428a3691-1870-46d1-99b6-89054f8bd6f7 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] validate_networks() for [('fc3b835d-0a7f-4948-9fcb-1edfd6b9a8df', None, None, None, None, None)] {{(pid=138680) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 16 21:21:51.136701 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell0:SELECT=4 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:21:51.652236 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell0:SELECT=18 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:21:52.345150 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-1e84b076-9a2d-4c09-b47e-67f5293e2e16 tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '029f1334779e4bcdb3f4f2aed564a075', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e1f6ffc3fb64dba8c4474641e796fb8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:21:52.361434 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-1e84b076-9a2d-4c09-b47e-67f5293e2e16 tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] 10.222.0.16 "GET /compute/v2.1/servers/fa075dcc-e175-449a-9914-d1ee19131f33" status: 200 len: 1371 microversion: 2.1 time: 2.006163 Jul 16 21:21:52.374820 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1468/2985] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:21:50 2022] GET /compute/v2.1/servers/fa075dcc-e175-449a-9914-d1ee19131f33 => generated 1371 bytes in 2020 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:21:52.387467 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-74aad575-1fef-47e9-a157-f4ec656e1b43 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:21:52.389049 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-74aad575-1fef-47e9-a157-f4ec656e1b43 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] [instance: 583e0c15-3053-429d-81bf-95a4d1f99cbf] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:21:52.398204 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-74aad575-1fef-47e9-a157-f4ec656e1b43 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:21:52.398642 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-74aad575-1fef-47e9-a157-f4ec656e1b43 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:21:52.399657 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-74aad575-1fef-47e9-a157-f4ec656e1b43 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:21:52.400257 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-74aad575-1fef-47e9-a157-f4ec656e1b43 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:21:52.608042 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-74aad575-1fef-47e9-a157-f4ec656e1b43 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'aaf2292818b749f0ae43c66bca929be9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '65644a8df5d84df3b35c894560e0a5c4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:21:52.621004 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-74aad575-1fef-47e9-a157-f4ec656e1b43 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] 10.222.0.16 "GET /compute/v2.1/servers/583e0c15-3053-429d-81bf-95a4d1f99cbf" status: 200 len: 1281 microversion: 2.1 time: 0.241328 Jul 16 21:21:52.623688 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1469/2986] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:21:52 2022] GET /compute/v2.1/servers/583e0c15-3053-429d-81bf-95a4d1f99cbf => generated 1281 bytes in 248 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:21:52.634381 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-315f1581-0237-49a1-aec9-0400510accb1 req-ac7f36b0-5af4-4cf4-833d-cd5fd481042b service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "8acee151-2b52-4ca3-b2e1-18a19834b0e2", "tag": "b550a22e-98c4-4464-b5db-88766e75d282"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:21:52.647385 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-315f1581-0237-49a1-aec9-0400510accb1 req-ac7f36b0-5af4-4cf4-833d-cd5fd481042b service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:21:52.648179 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-315f1581-0237-49a1-aec9-0400510accb1 req-ac7f36b0-5af4-4cf4-833d-cd5fd481042b service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:21:52.677385 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-315f1581-0237-49a1-aec9-0400510accb1 req-ac7f36b0-5af4-4cf4-833d-cd5fd481042b service nova] Creating event network-changed:b550a22e-98c4-4464-b5db-88766e75d282 for instance 8acee151-2b52-4ca3-b2e1-18a19834b0e2 on nch2-master-1 Jul 16 21:21:52.683853 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-315f1581-0237-49a1-aec9-0400510accb1 req-ac7f36b0-5af4-4cf4-833d-cd5fd481042b service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.058133 Jul 16 21:21:52.685481 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1470/2987] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:21:52 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 61 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:21:52.711625 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-45b1cf12-ab24-459a-9b7d-f30ac199f36a tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:21:52.713040 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-45b1cf12-ab24-459a-9b7d-f30ac199f36a tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] [instance: 94d9ca2e-1c4d-49ce-918e-02479909b75b] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:21:52.725845 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-45b1cf12-ab24-459a-9b7d-f30ac199f36a tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:21:52.726566 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-45b1cf12-ab24-459a-9b7d-f30ac199f36a tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:21:52.727681 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-45b1cf12-ab24-459a-9b7d-f30ac199f36a tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:21:52.729372 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-45b1cf12-ab24-459a-9b7d-f30ac199f36a tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:21:52.891431 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-45b1cf12-ab24-459a-9b7d-f30ac199f36a tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa4c2791e14d4f88883a0af422e4ff0c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c95a2d1e0a034c7ca4a2fcb9fd060d53', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:21:52.904537 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-45b1cf12-ab24-459a-9b7d-f30ac199f36a tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] 10.222.0.16 "GET /compute/v2.1/servers/94d9ca2e-1c4d-49ce-918e-02479909b75b" status: 200 len: 1281 microversion: 2.1 time: 0.206161 Jul 16 21:21:52.907108 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1471/2988] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:21:52 2022] GET /compute/v2.1/servers/94d9ca2e-1c4d-49ce-918e-02479909b75b => generated 1281 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:21:53.284870 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-428a3691-1870-46d1-99b6-89054f8bd6f7 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Flavor limits 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 16 21:21:53.285783 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-428a3691-1870-46d1-99b6-89054f8bd6f7 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Image limits 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 16 21:21:53.286863 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-428a3691-1870-46d1-99b6-89054f8bd6f7 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Flavor pref 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 16 21:21:53.287783 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-428a3691-1870-46d1-99b6-89054f8bd6f7 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Image pref 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 16 21:21:53.291581 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-428a3691-1870-46d1-99b6-89054f8bd6f7 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 16 21:21:53.338858 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-428a3691-1870-46d1-99b6-89054f8bd6f7 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:21:53.340382 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-428a3691-1870-46d1-99b6-89054f8bd6f7 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:21:53.343211 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-428a3691-1870-46d1-99b6-89054f8bd6f7 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:21:53.344957 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-428a3691-1870-46d1-99b6-89054f8bd6f7 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:21:53.392620 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-720f49f8-4e50-43a5-a444-db65cf73bc9d tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:21:53.393917 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-720f49f8-4e50-43a5-a444-db65cf73bc9d tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] [instance: fa075dcc-e175-449a-9914-d1ee19131f33] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:21:53.405127 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-720f49f8-4e50-43a5-a444-db65cf73bc9d tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:21:53.408856 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-720f49f8-4e50-43a5-a444-db65cf73bc9d tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:21:53.411690 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-720f49f8-4e50-43a5-a444-db65cf73bc9d tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:21:53.416080 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-720f49f8-4e50-43a5-a444-db65cf73bc9d tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:21:53.432624 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-428a3691-1870-46d1-99b6-89054f8bd6f7 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Getting quotas for project d31be8475a6a49d6bfe8132dd050c6c0. Resources: {'ram', 'instances', 'cores'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:21:53.469660 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-428a3691-1870-46d1-99b6-89054f8bd6f7 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Getting quotas for user 6f2e974ab5aa48dfb4b00f810dcedb63 and project d31be8475a6a49d6bfe8132dd050c6c0. Resources: {'ram', 'instances', 'cores'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:21:53.524299 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-428a3691-1870-46d1-99b6-89054f8bd6f7 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Going to run 1 instances... {{(pid=138680) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 16 21:21:53.542806 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-428a3691-1870-46d1-99b6-89054f8bd6f7 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: a4f887a4-84e8-4b14-beca-e6ad747ba616] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138680) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 16 21:21:53.849492 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-428a3691-1870-46d1-99b6-89054f8bd6f7 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 4.505350 Jul 16 21:21:53.864178 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1518/2989] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 21:21:49 2022] POST /compute/v2.1/servers => generated 380 bytes in 4519 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 16 21:21:53.874001 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-a80e4302-26fe-4708-ab4b-f89e7d0aeb8d tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:21:53.889823 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-a80e4302-26fe-4708-ab4b-f89e7d0aeb8d tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] [instance: 583e0c15-3053-429d-81bf-95a4d1f99cbf] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:21:53.905906 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a80e4302-26fe-4708-ab4b-f89e7d0aeb8d tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:21:53.908694 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a80e4302-26fe-4708-ab4b-f89e7d0aeb8d tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:21:53.909917 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a80e4302-26fe-4708-ab4b-f89e7d0aeb8d tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:21:53.913525 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a80e4302-26fe-4708-ab4b-f89e7d0aeb8d tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:21:54.399224 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-720f49f8-4e50-43a5-a444-db65cf73bc9d tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '029f1334779e4bcdb3f4f2aed564a075', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e1f6ffc3fb64dba8c4474641e796fb8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:21:54.420409 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-720f49f8-4e50-43a5-a444-db65cf73bc9d tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] 10.222.0.16 "GET /compute/v2.1/servers/fa075dcc-e175-449a-9914-d1ee19131f33" status: 200 len: 1371 microversion: 2.1 time: 1.034070 Jul 16 21:21:54.430813 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1472/2990] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:21:53 2022] GET /compute/v2.1/servers/fa075dcc-e175-449a-9914-d1ee19131f33 => generated 1371 bytes in 1045 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:21:54.455699 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-63a8a87a-d303-47e7-80ed-96d2b4c7a8cb tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:21:54.456337 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-63a8a87a-d303-47e7-80ed-96d2b4c7a8cb tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: a4f887a4-84e8-4b14-beca-e6ad747ba616] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:21:54.661204 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-63a8a87a-d303-47e7-80ed-96d2b4c7a8cb tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f2e974ab5aa48dfb4b00f810dcedb63', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd31be8475a6a49d6bfe8132dd050c6c0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:21:54.685150 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-63a8a87a-d303-47e7-80ed-96d2b4c7a8cb tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "GET /compute/v2.1/servers/a4f887a4-84e8-4b14-beca-e6ad747ba616" status: 200 len: 1214 microversion: 2.1 time: 0.240291 Jul 16 21:21:54.691434 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1473/2991] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:21:54 2022] GET /compute/v2.1/servers/a4f887a4-84e8-4b14-beca-e6ad747ba616 => generated 1214 bytes in 260 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:21:54.700253 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-e1e5919d-c24d-46f1-aaa8-849523a84587 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:21:54.701239 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-e1e5919d-c24d-46f1-aaa8-849523a84587 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] [instance: 94d9ca2e-1c4d-49ce-918e-02479909b75b] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:21:54.740776 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e1e5919d-c24d-46f1-aaa8-849523a84587 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:21:54.748215 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e1e5919d-c24d-46f1-aaa8-849523a84587 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.029s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:21:54.749454 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e1e5919d-c24d-46f1-aaa8-849523a84587 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:21:54.752777 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e1e5919d-c24d-46f1-aaa8-849523a84587 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:21:55.401450 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-a80e4302-26fe-4708-ab4b-f89e7d0aeb8d tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'aaf2292818b749f0ae43c66bca929be9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '65644a8df5d84df3b35c894560e0a5c4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:21:55.422537 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-a80e4302-26fe-4708-ab4b-f89e7d0aeb8d tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] 10.222.0.16 "GET /compute/v2.1/servers/583e0c15-3053-429d-81bf-95a4d1f99cbf" status: 200 len: 1323 microversion: 2.1 time: 1.554509 Jul 16 21:21:55.435496 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1519/2992] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:21:53 2022] GET /compute/v2.1/servers/583e0c15-3053-429d-81bf-95a4d1f99cbf => generated 1323 bytes in 1572 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:21:55.448451 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-f28119db-bc8c-4045-bd41-573dd990712b tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:21:55.451365 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-f28119db-bc8c-4045-bd41-573dd990712b tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] [instance: fa075dcc-e175-449a-9914-d1ee19131f33] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:21:55.483791 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f28119db-bc8c-4045-bd41-573dd990712b tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:21:55.483791 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f28119db-bc8c-4045-bd41-573dd990712b tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:21:55.483791 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f28119db-bc8c-4045-bd41-573dd990712b tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:21:55.483791 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f28119db-bc8c-4045-bd41-573dd990712b tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:21:55.565554 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-e1e5919d-c24d-46f1-aaa8-849523a84587 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa4c2791e14d4f88883a0af422e4ff0c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c95a2d1e0a034c7ca4a2fcb9fd060d53', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:21:55.573941 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-e1e5919d-c24d-46f1-aaa8-849523a84587 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] 10.222.0.16 "GET /compute/v2.1/servers/94d9ca2e-1c4d-49ce-918e-02479909b75b" status: 200 len: 1323 microversion: 2.1 time: 0.880363 Jul 16 21:21:55.576365 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1474/2993] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:21:54 2022] GET /compute/v2.1/servers/94d9ca2e-1c4d-49ce-918e-02479909b75b => generated 1323 bytes in 883 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:21:55.713701 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-c2c42c61-c966-4112-ad5f-281054f5d04c tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:21:55.714498 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-c2c42c61-c966-4112-ad5f-281054f5d04c tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: a4f887a4-84e8-4b14-beca-e6ad747ba616] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:21:55.822589 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-c2c42c61-c966-4112-ad5f-281054f5d04c tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f2e974ab5aa48dfb4b00f810dcedb63', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd31be8475a6a49d6bfe8132dd050c6c0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:21:55.836568 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-c2c42c61-c966-4112-ad5f-281054f5d04c tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "GET /compute/v2.1/servers/a4f887a4-84e8-4b14-beca-e6ad747ba616" status: 200 len: 1214 microversion: 2.1 time: 0.130288 Jul 16 21:21:55.838142 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1475/2994] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:21:55 2022] GET /compute/v2.1/servers/a4f887a4-84e8-4b14-beca-e6ad747ba616 => generated 1214 bytes in 133 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:21:55.850924 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-f28119db-bc8c-4045-bd41-573dd990712b tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '029f1334779e4bcdb3f4f2aed564a075', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e1f6ffc3fb64dba8c4474641e796fb8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:21:55.860717 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-f28119db-bc8c-4045-bd41-573dd990712b tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] 10.222.0.16 "GET /compute/v2.1/servers/fa075dcc-e175-449a-9914-d1ee19131f33" status: 200 len: 1371 microversion: 2.1 time: 0.416057 Jul 16 21:21:55.872168 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1520/2995] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:21:55 2022] GET /compute/v2.1/servers/fa075dcc-e175-449a-9914-d1ee19131f33 => generated 1371 bytes in 428 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:21:56.472043 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-0ebab76c-7272-4036-b7b7-e4f1885af3b0 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:21:56.473051 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-0ebab76c-7272-4036-b7b7-e4f1885af3b0 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] [instance: 583e0c15-3053-429d-81bf-95a4d1f99cbf] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:21:56.502540 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0ebab76c-7272-4036-b7b7-e4f1885af3b0 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:21:56.506696 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0ebab76c-7272-4036-b7b7-e4f1885af3b0 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:21:56.521201 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0ebab76c-7272-4036-b7b7-e4f1885af3b0 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:21:56.523896 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0ebab76c-7272-4036-b7b7-e4f1885af3b0 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:21:56.593606 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-6f0897fd-3fcd-4194-b0c7-528fbd670025 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:21:56.595227 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-6f0897fd-3fcd-4194-b0c7-528fbd670025 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] [instance: 94d9ca2e-1c4d-49ce-918e-02479909b75b] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:21:56.638426 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6f0897fd-3fcd-4194-b0c7-528fbd670025 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:21:56.642167 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6f0897fd-3fcd-4194-b0c7-528fbd670025 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:21:56.646085 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6f0897fd-3fcd-4194-b0c7-528fbd670025 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:21:56.646085 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6f0897fd-3fcd-4194-b0c7-528fbd670025 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:21:57.886873 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-0ebab76c-7272-4036-b7b7-e4f1885af3b0 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'aaf2292818b749f0ae43c66bca929be9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '65644a8df5d84df3b35c894560e0a5c4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:21:57.920126 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-0ebab76c-7272-4036-b7b7-e4f1885af3b0 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] 10.222.0.16 "GET /compute/v2.1/servers/583e0c15-3053-429d-81bf-95a4d1f99cbf" status: 200 len: 1323 microversion: 2.1 time: 1.469014 Jul 16 21:21:57.923807 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1476/2996] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:21:56 2022] GET /compute/v2.1/servers/583e0c15-3053-429d-81bf-95a4d1f99cbf => generated 1323 bytes in 1476 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:21:57.931332 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-c6373e44-00a1-4c4d-9b33-5849a9911800 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:21:57.941071 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-c6373e44-00a1-4c4d-9b33-5849a9911800 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: a4f887a4-84e8-4b14-beca-e6ad747ba616] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:21:57.972595 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c6373e44-00a1-4c4d-9b33-5849a9911800 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:21:57.976730 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c6373e44-00a1-4c4d-9b33-5849a9911800 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:21:57.977944 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c6373e44-00a1-4c4d-9b33-5849a9911800 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:21:57.979152 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c6373e44-00a1-4c4d-9b33-5849a9911800 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:21:58.212309 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-c6373e44-00a1-4c4d-9b33-5849a9911800 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f2e974ab5aa48dfb4b00f810dcedb63', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd31be8475a6a49d6bfe8132dd050c6c0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:21:58.231914 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-6f0897fd-3fcd-4194-b0c7-528fbd670025 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa4c2791e14d4f88883a0af422e4ff0c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c95a2d1e0a034c7ca4a2fcb9fd060d53', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:21:58.268313 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-6f0897fd-3fcd-4194-b0c7-528fbd670025 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] 10.222.0.16 "GET /compute/v2.1/servers/94d9ca2e-1c4d-49ce-918e-02479909b75b" status: 200 len: 1323 microversion: 2.1 time: 1.673678 Jul 16 21:21:58.269342 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-c6373e44-00a1-4c4d-9b33-5849a9911800 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "GET /compute/v2.1/servers/a4f887a4-84e8-4b14-beca-e6ad747ba616" status: 200 len: 1266 microversion: 2.1 time: 0.338664 Jul 16 21:21:58.270299 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1521/2997] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:21:56 2022] GET /compute/v2.1/servers/94d9ca2e-1c4d-49ce-918e-02479909b75b => generated 1323 bytes in 1684 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:21:58.273219 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1477/2998] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:21:57 2022] GET /compute/v2.1/servers/a4f887a4-84e8-4b14-beca-e6ad747ba616 => generated 1266 bytes in 349 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:21:58.296385 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-177cb498-5234-4036-ab7c-5a5ba269594c tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:21:58.296385 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-177cb498-5234-4036-ab7c-5a5ba269594c tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] [instance: fa075dcc-e175-449a-9914-d1ee19131f33] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:21:58.327854 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-177cb498-5234-4036-ab7c-5a5ba269594c tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:21:58.333879 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-177cb498-5234-4036-ab7c-5a5ba269594c tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:21:58.334750 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-177cb498-5234-4036-ab7c-5a5ba269594c tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:21:58.339885 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-177cb498-5234-4036-ab7c-5a5ba269594c tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:21:58.739420 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-32a02650-96fd-4c66-ad1b-54c43116b60c req-d86d4f1b-ab5c-48cb-ac87-a9935fda9b8c service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "583e0c15-3053-429d-81bf-95a4d1f99cbf", "tag": "40900dd3-d3df-4e26-80dd-0776ab4b1d20"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:21:58.771382 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-32a02650-96fd-4c66-ad1b-54c43116b60c req-d86d4f1b-ab5c-48cb-ac87-a9935fda9b8c service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:21:58.772806 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-32a02650-96fd-4c66-ad1b-54c43116b60c req-d86d4f1b-ab5c-48cb-ac87-a9935fda9b8c service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:21:58.815694 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-32a02650-96fd-4c66-ad1b-54c43116b60c req-d86d4f1b-ab5c-48cb-ac87-a9935fda9b8c service nova] Creating event network-changed:40900dd3-d3df-4e26-80dd-0776ab4b1d20 for instance 583e0c15-3053-429d-81bf-95a4d1f99cbf on nch1-master-1 Jul 16 21:21:58.857736 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-32a02650-96fd-4c66-ad1b-54c43116b60c req-d86d4f1b-ab5c-48cb-ac87-a9935fda9b8c service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.124293 Jul 16 21:21:58.870723 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1478/2999] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:21:58 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 138 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:21:58.944094 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-96f997fd-4fd5-4ba1-9aa2-885d5849b521 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:21:58.946202 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-96f997fd-4fd5-4ba1-9aa2-885d5849b521 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] [instance: 583e0c15-3053-429d-81bf-95a4d1f99cbf] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:21:58.972996 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-96f997fd-4fd5-4ba1-9aa2-885d5849b521 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:21:58.975781 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-96f997fd-4fd5-4ba1-9aa2-885d5849b521 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:21:58.977844 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-96f997fd-4fd5-4ba1-9aa2-885d5849b521 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:21:58.980044 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-96f997fd-4fd5-4ba1-9aa2-885d5849b521 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:00.236627 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-177cb498-5234-4036-ab7c-5a5ba269594c tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '029f1334779e4bcdb3f4f2aed564a075', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e1f6ffc3fb64dba8c4474641e796fb8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:22:00.249415 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-177cb498-5234-4036-ab7c-5a5ba269594c tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] 10.222.0.16 "GET /compute/v2.1/servers/fa075dcc-e175-449a-9914-d1ee19131f33" status: 200 len: 1371 microversion: 2.1 time: 1.974785 Jul 16 21:22:00.254004 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1522/3000] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:21:58 2022] GET /compute/v2.1/servers/fa075dcc-e175-449a-9914-d1ee19131f33 => generated 1371 bytes in 1981 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:22:00.268190 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-121a6a3f-2338-4c68-94d5-da6b87f09e2e tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:22:00.278469 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-121a6a3f-2338-4c68-94d5-da6b87f09e2e tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] [instance: 94d9ca2e-1c4d-49ce-918e-02479909b75b] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:22:00.287194 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-121a6a3f-2338-4c68-94d5-da6b87f09e2e tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:00.288307 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-121a6a3f-2338-4c68-94d5-da6b87f09e2e tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:00.291044 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-121a6a3f-2338-4c68-94d5-da6b87f09e2e tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:00.291044 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-121a6a3f-2338-4c68-94d5-da6b87f09e2e tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:00.419326 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-96f997fd-4fd5-4ba1-9aa2-885d5849b521 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'aaf2292818b749f0ae43c66bca929be9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '65644a8df5d84df3b35c894560e0a5c4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:22:00.435183 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=100 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:22:00.438227 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-96f997fd-4fd5-4ba1-9aa2-885d5849b521 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] 10.222.0.16 "GET /compute/v2.1/servers/583e0c15-3053-429d-81bf-95a4d1f99cbf" status: 200 len: 1323 microversion: 2.1 time: 1.498919 Jul 16 21:22:00.447232 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1479/3001] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:21:58 2022] GET /compute/v2.1/servers/583e0c15-3053-429d-81bf-95a4d1f99cbf => generated 1323 bytes in 1509 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:22:00.456434 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-9697a870-9e2d-42e4-96d9-604bb8878fbe tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:22:00.458181 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-9697a870-9e2d-42e4-96d9-604bb8878fbe tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: a4f887a4-84e8-4b14-beca-e6ad747ba616] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:22:00.477476 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9697a870-9e2d-42e4-96d9-604bb8878fbe tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:00.478257 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9697a870-9e2d-42e4-96d9-604bb8878fbe tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:00.480373 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9697a870-9e2d-42e4-96d9-604bb8878fbe tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:00.481284 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9697a870-9e2d-42e4-96d9-604bb8878fbe tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:01.162238 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-121a6a3f-2338-4c68-94d5-da6b87f09e2e tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa4c2791e14d4f88883a0af422e4ff0c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c95a2d1e0a034c7ca4a2fcb9fd060d53', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:22:01.174617 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-121a6a3f-2338-4c68-94d5-da6b87f09e2e tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] 10.222.0.16 "GET /compute/v2.1/servers/94d9ca2e-1c4d-49ce-918e-02479909b75b" status: 200 len: 1323 microversion: 2.1 time: 0.918677 Jul 16 21:22:01.181821 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1523/3002] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:22:00 2022] GET /compute/v2.1/servers/94d9ca2e-1c4d-49ce-918e-02479909b75b => generated 1323 bytes in 926 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:22:01.242153 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-9697a870-9e2d-42e4-96d9-604bb8878fbe tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f2e974ab5aa48dfb4b00f810dcedb63', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd31be8475a6a49d6bfe8132dd050c6c0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:22:01.253781 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-9697a870-9e2d-42e4-96d9-604bb8878fbe tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "GET /compute/v2.1/servers/a4f887a4-84e8-4b14-beca-e6ad747ba616" status: 200 len: 1314 microversion: 2.1 time: 0.805010 Jul 16 21:22:01.254374 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1480/3003] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:22:00 2022] GET /compute/v2.1/servers/a4f887a4-84e8-4b14-beca-e6ad747ba616 => generated 1314 bytes in 806 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:22:01.269984 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-9b58d341-4b58-4bde-8823-a0c7bd39e96c tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:22:01.271259 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-9b58d341-4b58-4bde-8823-a0c7bd39e96c tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] [instance: fa075dcc-e175-449a-9914-d1ee19131f33] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:22:01.278301 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9b58d341-4b58-4bde-8823-a0c7bd39e96c tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:01.278971 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9b58d341-4b58-4bde-8823-a0c7bd39e96c tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:01.279774 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9b58d341-4b58-4bde-8823-a0c7bd39e96c tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:01.280412 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9b58d341-4b58-4bde-8823-a0c7bd39e96c tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:01.463886 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-a8e4e0a0-56b5-4077-8089-6ef7ba14e12f tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:22:01.467197 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-a8e4e0a0-56b5-4077-8089-6ef7ba14e12f tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] [instance: 583e0c15-3053-429d-81bf-95a4d1f99cbf] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:22:01.500466 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a8e4e0a0-56b5-4077-8089-6ef7ba14e12f tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:01.500466 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a8e4e0a0-56b5-4077-8089-6ef7ba14e12f tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:01.500466 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a8e4e0a0-56b5-4077-8089-6ef7ba14e12f tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:01.500466 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a8e4e0a0-56b5-4077-8089-6ef7ba14e12f tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:01.783099 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-9b58d341-4b58-4bde-8823-a0c7bd39e96c tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '029f1334779e4bcdb3f4f2aed564a075', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e1f6ffc3fb64dba8c4474641e796fb8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:22:01.805798 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-9b58d341-4b58-4bde-8823-a0c7bd39e96c tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] 10.222.0.16 "GET /compute/v2.1/servers/fa075dcc-e175-449a-9914-d1ee19131f33" status: 200 len: 1371 microversion: 2.1 time: 0.538451 Jul 16 21:22:01.806460 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1524/3004] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:22:01 2022] GET /compute/v2.1/servers/fa075dcc-e175-449a-9914-d1ee19131f33 => generated 1371 bytes in 542 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:22:01.826776 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-ed6d6a1f-ec79-432c-9211-22ade4d855c6 req-2beaefee-65fb-4007-b5e7-749d6565650d service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "94d9ca2e-1c4d-49ce-918e-02479909b75b", "tag": "782384db-d832-4551-9c6b-3eb3e7a7352b"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:22:01.837258 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-ed6d6a1f-ec79-432c-9211-22ade4d855c6 req-2beaefee-65fb-4007-b5e7-749d6565650d service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:01.837855 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-ed6d6a1f-ec79-432c-9211-22ade4d855c6 req-2beaefee-65fb-4007-b5e7-749d6565650d service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:01.862944 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-ed6d6a1f-ec79-432c-9211-22ade4d855c6 req-2beaefee-65fb-4007-b5e7-749d6565650d service nova] Creating event network-changed:782384db-d832-4551-9c6b-3eb3e7a7352b for instance 94d9ca2e-1c4d-49ce-918e-02479909b75b on nch2-master-1 Jul 16 21:22:01.880124 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-ed6d6a1f-ec79-432c-9211-22ade4d855c6 req-2beaefee-65fb-4007-b5e7-749d6565650d service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.072324 Jul 16 21:22:01.881520 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1525/3005] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:22:01 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 74 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:22:01.888462 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-c6074fba-a664-4bc2-be4e-33ff2414017a req-672013b3-6078-48a6-ab77-06f2f49ae6d1 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "8acee151-2b52-4ca3-b2e1-18a19834b0e2", "name": "network-vif-plugged", "status": "completed", "tag": "b550a22e-98c4-4464-b5db-88766e75d282"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:22:01.916471 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-c6074fba-a664-4bc2-be4e-33ff2414017a req-672013b3-6078-48a6-ab77-06f2f49ae6d1 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:01.917242 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-c6074fba-a664-4bc2-be4e-33ff2414017a req-672013b3-6078-48a6-ab77-06f2f49ae6d1 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:01.944183 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-c6074fba-a664-4bc2-be4e-33ff2414017a req-672013b3-6078-48a6-ab77-06f2f49ae6d1 service nova] Creating event network-vif-plugged:b550a22e-98c4-4464-b5db-88766e75d282 for instance 8acee151-2b52-4ca3-b2e1-18a19834b0e2 on nch2-master-1 Jul 16 21:22:01.957232 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-c6074fba-a664-4bc2-be4e-33ff2414017a req-672013b3-6078-48a6-ab77-06f2f49ae6d1 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.073883 Jul 16 21:22:01.958328 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1526/3006] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:22:01 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 76 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:22:01.958899 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-a8e4e0a0-56b5-4077-8089-6ef7ba14e12f tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'aaf2292818b749f0ae43c66bca929be9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '65644a8df5d84df3b35c894560e0a5c4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:22:01.967347 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-7576be3c-1c29-47cc-97bb-e1e3738a6b87 req-c7b57caf-a542-43fc-b13e-943d3f01d4ef service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "a4f887a4-84e8-4b14-beca-e6ad747ba616", "tag": "ced0cb52-5511-4c5a-9ebd-aab291a021f0"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:22:01.967997 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-a8e4e0a0-56b5-4077-8089-6ef7ba14e12f tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] 10.222.0.16 "GET /compute/v2.1/servers/583e0c15-3053-429d-81bf-95a4d1f99cbf" status: 200 len: 1323 microversion: 2.1 time: 0.509042 Jul 16 21:22:01.969346 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1481/3007] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:22:01 2022] GET /compute/v2.1/servers/583e0c15-3053-429d-81bf-95a4d1f99cbf => generated 1323 bytes in 512 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:22:01.994094 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-7576be3c-1c29-47cc-97bb-e1e3738a6b87 req-c7b57caf-a542-43fc-b13e-943d3f01d4ef service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:01.995594 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-7576be3c-1c29-47cc-97bb-e1e3738a6b87 req-c7b57caf-a542-43fc-b13e-943d3f01d4ef service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:02.015219 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-7576be3c-1c29-47cc-97bb-e1e3738a6b87 req-c7b57caf-a542-43fc-b13e-943d3f01d4ef service nova] Creating event network-changed:ced0cb52-5511-4c5a-9ebd-aab291a021f0 for instance a4f887a4-84e8-4b14-beca-e6ad747ba616 on nch1-master-1 Jul 16 21:22:02.022430 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-7576be3c-1c29-47cc-97bb-e1e3738a6b87 req-c7b57caf-a542-43fc-b13e-943d3f01d4ef service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.062366 Jul 16 21:22:02.027844 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1527/3008] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:22:01 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 69 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:22:02.194198 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-0a138ad6-2fac-41ed-8650-6c38fe7d9a72 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:22:02.195056 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-0a138ad6-2fac-41ed-8650-6c38fe7d9a72 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] [instance: 94d9ca2e-1c4d-49ce-918e-02479909b75b] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:22:02.204777 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0a138ad6-2fac-41ed-8650-6c38fe7d9a72 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:02.210840 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0a138ad6-2fac-41ed-8650-6c38fe7d9a72 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:02.211679 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0a138ad6-2fac-41ed-8650-6c38fe7d9a72 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:02.212416 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0a138ad6-2fac-41ed-8650-6c38fe7d9a72 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:02.283654 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-d8f5acac-c143-4f3f-bc34-5cd567bf3224 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:22:02.286330 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-d8f5acac-c143-4f3f-bc34-5cd567bf3224 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: a4f887a4-84e8-4b14-beca-e6ad747ba616] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:22:02.302570 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d8f5acac-c143-4f3f-bc34-5cd567bf3224 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:02.307434 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d8f5acac-c143-4f3f-bc34-5cd567bf3224 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:02.311928 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d8f5acac-c143-4f3f-bc34-5cd567bf3224 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:02.313672 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d8f5acac-c143-4f3f-bc34-5cd567bf3224 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:02.702328 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-0a138ad6-2fac-41ed-8650-6c38fe7d9a72 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa4c2791e14d4f88883a0af422e4ff0c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c95a2d1e0a034c7ca4a2fcb9fd060d53', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:22:02.711212 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-d8f5acac-c143-4f3f-bc34-5cd567bf3224 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f2e974ab5aa48dfb4b00f810dcedb63', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd31be8475a6a49d6bfe8132dd050c6c0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:22:02.721163 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-0a138ad6-2fac-41ed-8650-6c38fe7d9a72 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] 10.222.0.16 "GET /compute/v2.1/servers/94d9ca2e-1c4d-49ce-918e-02479909b75b" status: 200 len: 1323 microversion: 2.1 time: 0.532406 Jul 16 21:22:02.723217 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-d8f5acac-c143-4f3f-bc34-5cd567bf3224 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "GET /compute/v2.1/servers/a4f887a4-84e8-4b14-beca-e6ad747ba616" status: 200 len: 1314 microversion: 2.1 time: 0.454638 Jul 16 21:22:02.729628 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1482/3009] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:22:02 2022] GET /compute/v2.1/servers/94d9ca2e-1c4d-49ce-918e-02479909b75b => generated 1323 bytes in 542 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:22:02.731416 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1528/3010] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:22:02 2022] GET /compute/v2.1/servers/a4f887a4-84e8-4b14-beca-e6ad747ba616 => generated 1314 bytes in 464 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:22:02.824963 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-a4ec454a-94fd-46fb-b0e3-c43efccd5220 tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:22:02.826439 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-a4ec454a-94fd-46fb-b0e3-c43efccd5220 tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] [instance: fa075dcc-e175-449a-9914-d1ee19131f33] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:22:02.833980 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a4ec454a-94fd-46fb-b0e3-c43efccd5220 tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:02.834443 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a4ec454a-94fd-46fb-b0e3-c43efccd5220 tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:02.835253 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a4ec454a-94fd-46fb-b0e3-c43efccd5220 tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:02.835871 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a4ec454a-94fd-46fb-b0e3-c43efccd5220 tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:02.989619 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-9248521e-96c6-442b-b82a-d5cf54fde45f tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:22:02.990638 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-9248521e-96c6-442b-b82a-d5cf54fde45f tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] [instance: 583e0c15-3053-429d-81bf-95a4d1f99cbf] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:22:02.998444 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9248521e-96c6-442b-b82a-d5cf54fde45f tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:03.000184 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9248521e-96c6-442b-b82a-d5cf54fde45f tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:03.001211 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9248521e-96c6-442b-b82a-d5cf54fde45f tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:03.002262 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9248521e-96c6-442b-b82a-d5cf54fde45f tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:03.386808 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell0:SELECT=2 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:22:03.493434 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell0:SELECT=22 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:22:03.649188 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-a4ec454a-94fd-46fb-b0e3-c43efccd5220 tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '029f1334779e4bcdb3f4f2aed564a075', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e1f6ffc3fb64dba8c4474641e796fb8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:22:03.660171 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-a4ec454a-94fd-46fb-b0e3-c43efccd5220 tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] 10.222.0.16 "GET /compute/v2.1/servers/fa075dcc-e175-449a-9914-d1ee19131f33" status: 200 len: 1371 microversion: 2.1 time: 0.840630 Jul 16 21:22:03.661137 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1483/3011] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:22:02 2022] GET /compute/v2.1/servers/fa075dcc-e175-449a-9914-d1ee19131f33 => generated 1371 bytes in 842 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:22:03.709021 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-9248521e-96c6-442b-b82a-d5cf54fde45f tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'aaf2292818b749f0ae43c66bca929be9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '65644a8df5d84df3b35c894560e0a5c4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:22:03.719851 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-9248521e-96c6-442b-b82a-d5cf54fde45f tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] 10.222.0.16 "GET /compute/v2.1/servers/583e0c15-3053-429d-81bf-95a4d1f99cbf" status: 200 len: 1323 microversion: 2.1 time: 0.737655 Jul 16 21:22:03.721339 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1529/3012] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:22:02 2022] GET /compute/v2.1/servers/583e0c15-3053-429d-81bf-95a4d1f99cbf => generated 1323 bytes in 740 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:22:03.756722 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-261fae72-a628-4a7f-9c4e-df575783d242 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:22:03.758526 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-261fae72-a628-4a7f-9c4e-df575783d242 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] [instance: 94d9ca2e-1c4d-49ce-918e-02479909b75b] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:22:03.770306 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-261fae72-a628-4a7f-9c4e-df575783d242 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:03.775686 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-2951a854-1275-4ab5-848d-2ac73d7fb852 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:22:03.776254 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-261fae72-a628-4a7f-9c4e-df575783d242 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:03.776690 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-2951a854-1275-4ab5-848d-2ac73d7fb852 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: a4f887a4-84e8-4b14-beca-e6ad747ba616] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:22:03.777630 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-261fae72-a628-4a7f-9c4e-df575783d242 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:03.778092 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-261fae72-a628-4a7f-9c4e-df575783d242 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:03.785381 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2951a854-1275-4ab5-848d-2ac73d7fb852 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:03.787305 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2951a854-1275-4ab5-848d-2ac73d7fb852 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:03.788389 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2951a854-1275-4ab5-848d-2ac73d7fb852 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:03.789086 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2951a854-1275-4ab5-848d-2ac73d7fb852 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:04.289349 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-2951a854-1275-4ab5-848d-2ac73d7fb852 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f2e974ab5aa48dfb4b00f810dcedb63', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd31be8475a6a49d6bfe8132dd050c6c0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:22:04.306415 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-2951a854-1275-4ab5-848d-2ac73d7fb852 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "GET /compute/v2.1/servers/a4f887a4-84e8-4b14-beca-e6ad747ba616" status: 200 len: 1314 microversion: 2.1 time: 0.542182 Jul 16 21:22:04.309332 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1530/3013] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:22:03 2022] GET /compute/v2.1/servers/a4f887a4-84e8-4b14-beca-e6ad747ba616 => generated 1314 bytes in 546 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:22:04.315100 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-74875cc7-5978-443a-baf1-ad6f04bb8d40 req-031a6729-df9a-4c7d-b1fe-e34ded973820 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "9e7787d1-264a-4707-8604-3e8b70133376", "tag": "b5213034-c4ce-4421-b1cd-864c1901ff82"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:22:04.318032 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-261fae72-a628-4a7f-9c4e-df575783d242 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa4c2791e14d4f88883a0af422e4ff0c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c95a2d1e0a034c7ca4a2fcb9fd060d53', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:22:04.325801 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-74875cc7-5978-443a-baf1-ad6f04bb8d40 req-031a6729-df9a-4c7d-b1fe-e34ded973820 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:04.326253 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-74875cc7-5978-443a-baf1-ad6f04bb8d40 req-031a6729-df9a-4c7d-b1fe-e34ded973820 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:04.328261 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-261fae72-a628-4a7f-9c4e-df575783d242 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] 10.222.0.16 "GET /compute/v2.1/servers/94d9ca2e-1c4d-49ce-918e-02479909b75b" status: 200 len: 1323 microversion: 2.1 time: 0.576423 Jul 16 21:22:04.339143 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1484/3014] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:22:03 2022] GET /compute/v2.1/servers/94d9ca2e-1c4d-49ce-918e-02479909b75b => generated 1323 bytes in 589 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:22:04.340057 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-74875cc7-5978-443a-baf1-ad6f04bb8d40 req-031a6729-df9a-4c7d-b1fe-e34ded973820 service nova] Creating event network-changed:b5213034-c4ce-4421-b1cd-864c1901ff82 for instance 9e7787d1-264a-4707-8604-3e8b70133376 on nch1-master-1 Jul 16 21:22:04.346424 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-74875cc7-5978-443a-baf1-ad6f04bb8d40 req-031a6729-df9a-4c7d-b1fe-e34ded973820 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.035817 Jul 16 21:22:04.347924 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1531/3015] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:22:04 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:22:04.678232 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-c1c298b0-b187-46cd-b676-b960c2e5e6f6 tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:22:04.683110 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-c1c298b0-b187-46cd-b676-b960c2e5e6f6 tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] [instance: fa075dcc-e175-449a-9914-d1ee19131f33] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:22:04.692379 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_api:SELECT=94,nova_api:INSERT=6 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:22:04.696997 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c1c298b0-b187-46cd-b676-b960c2e5e6f6 tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:04.697640 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c1c298b0-b187-46cd-b676-b960c2e5e6f6 tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:04.698476 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c1c298b0-b187-46cd-b676-b960c2e5e6f6 tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:04.699105 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c1c298b0-b187-46cd-b676-b960c2e5e6f6 tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:04.716208 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-2ab6544b-ca26-48d8-8024-63d92084e7b1 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:22:04.720329 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-2ab6544b-ca26-48d8-8024-63d92084e7b1 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] [instance: f114e124-2903-4f30-b055-9f493e3e9e6c] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:22:04.725944 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2ab6544b-ca26-48d8-8024-63d92084e7b1 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:04.726697 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2ab6544b-ca26-48d8-8024-63d92084e7b1 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:04.727750 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2ab6544b-ca26-48d8-8024-63d92084e7b1 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:04.728452 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2ab6544b-ca26-48d8-8024-63d92084e7b1 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:04.753186 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=100 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:22:04.757752 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-2ab6544b-ca26-48d8-8024-63d92084e7b1 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] 10.222.0.16 "GET /compute/v2.1/servers/f114e124-2903-4f30-b055-9f493e3e9e6c/ips" status: 200 len: 147 microversion: 2.1 time: 0.046867 Jul 16 21:22:04.758760 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1532/3016] 10.222.0.16 () {58 vars in 1247 bytes} [Sat Jul 16 21:22:04 2022] GET /compute/v2.1/servers/f114e124-2903-4f30-b055-9f493e3e9e6c/ips => generated 147 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:22:04.763738 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-62b9ac92-fa46-47d8-b6b9-446eaa34a1aa tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:22:04.766495 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-62b9ac92-fa46-47d8-b6b9-446eaa34a1aa tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] [instance: 583e0c15-3053-429d-81bf-95a4d1f99cbf] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:22:04.771443 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-62b9ac92-fa46-47d8-b6b9-446eaa34a1aa tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:04.772189 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-62b9ac92-fa46-47d8-b6b9-446eaa34a1aa tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:04.773383 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-62b9ac92-fa46-47d8-b6b9-446eaa34a1aa tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:04.774087 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-62b9ac92-fa46-47d8-b6b9-446eaa34a1aa tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:04.906531 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-c1c298b0-b187-46cd-b676-b960c2e5e6f6 tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '029f1334779e4bcdb3f4f2aed564a075', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e1f6ffc3fb64dba8c4474641e796fb8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:22:04.918558 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-c1c298b0-b187-46cd-b676-b960c2e5e6f6 tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] 10.222.0.16 "GET /compute/v2.1/servers/fa075dcc-e175-449a-9914-d1ee19131f33" status: 200 len: 1371 microversion: 2.1 time: 0.243975 Jul 16 21:22:04.919191 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1485/3017] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:22:04 2022] GET /compute/v2.1/servers/fa075dcc-e175-449a-9914-d1ee19131f33 => generated 1371 bytes in 246 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:22:04.946373 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-62b9ac92-fa46-47d8-b6b9-446eaa34a1aa tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'aaf2292818b749f0ae43c66bca929be9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '65644a8df5d84df3b35c894560e0a5c4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:22:04.955878 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-62b9ac92-fa46-47d8-b6b9-446eaa34a1aa tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] 10.222.0.16 "GET /compute/v2.1/servers/583e0c15-3053-429d-81bf-95a4d1f99cbf" status: 200 len: 1323 microversion: 2.1 time: 0.195832 Jul 16 21:22:04.957637 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1533/3018] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:22:04 2022] GET /compute/v2.1/servers/583e0c15-3053-429d-81bf-95a4d1f99cbf => generated 1323 bytes in 198 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:22:05.324578 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-1bd113a4-d805-4b1e-ac59-cd2d5a63b077 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:22:05.325210 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-1bd113a4-d805-4b1e-ac59-cd2d5a63b077 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: a4f887a4-84e8-4b14-beca-e6ad747ba616] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:22:05.336015 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1bd113a4-d805-4b1e-ac59-cd2d5a63b077 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:05.336945 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1bd113a4-d805-4b1e-ac59-cd2d5a63b077 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:05.346971 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-87f7819c-200b-4119-ad1e-e44f13c80ab6 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:22:05.347928 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-87f7819c-200b-4119-ad1e-e44f13c80ab6 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] [instance: 94d9ca2e-1c4d-49ce-918e-02479909b75b] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:22:05.353574 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1bd113a4-d805-4b1e-ac59-cd2d5a63b077 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:05.354578 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1bd113a4-d805-4b1e-ac59-cd2d5a63b077 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.016s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:05.359952 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-87f7819c-200b-4119-ad1e-e44f13c80ab6 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:05.360807 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-87f7819c-200b-4119-ad1e-e44f13c80ab6 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:05.364601 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-87f7819c-200b-4119-ad1e-e44f13c80ab6 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:05.372622 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-87f7819c-200b-4119-ad1e-e44f13c80ab6 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:05.693166 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-87f7819c-200b-4119-ad1e-e44f13c80ab6 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa4c2791e14d4f88883a0af422e4ff0c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c95a2d1e0a034c7ca4a2fcb9fd060d53', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:22:05.709046 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-87f7819c-200b-4119-ad1e-e44f13c80ab6 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] 10.222.0.16 "GET /compute/v2.1/servers/94d9ca2e-1c4d-49ce-918e-02479909b75b" status: 200 len: 1323 microversion: 2.1 time: 0.367957 Jul 16 21:22:05.719628 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1534/3019] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:22:05 2022] GET /compute/v2.1/servers/94d9ca2e-1c4d-49ce-918e-02479909b75b => generated 1323 bytes in 379 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:22:05.725471 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-1bd113a4-d805-4b1e-ac59-cd2d5a63b077 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f2e974ab5aa48dfb4b00f810dcedb63', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd31be8475a6a49d6bfe8132dd050c6c0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:22:05.737683 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-1bd113a4-d805-4b1e-ac59-cd2d5a63b077 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "GET /compute/v2.1/servers/a4f887a4-84e8-4b14-beca-e6ad747ba616" status: 200 len: 1314 microversion: 2.1 time: 0.417285 Jul 16 21:22:05.738799 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1486/3020] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:22:05 2022] GET /compute/v2.1/servers/a4f887a4-84e8-4b14-beca-e6ad747ba616 => generated 1314 bytes in 419 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:22:05.938744 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-8bc04b70-3f84-4b6f-9ea7-f7fba67b675c tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:22:05.939684 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-8bc04b70-3f84-4b6f-9ea7-f7fba67b675c tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] [instance: fa075dcc-e175-449a-9914-d1ee19131f33] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:22:05.947844 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8bc04b70-3f84-4b6f-9ea7-f7fba67b675c tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:05.948312 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8bc04b70-3f84-4b6f-9ea7-f7fba67b675c tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:05.949132 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8bc04b70-3f84-4b6f-9ea7-f7fba67b675c tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:05.949580 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8bc04b70-3f84-4b6f-9ea7-f7fba67b675c tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:05.972687 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-1a3e09b7-6034-4c7b-b785-d498734a72a7 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:22:05.978120 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-1a3e09b7-6034-4c7b-b785-d498734a72a7 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] [instance: 583e0c15-3053-429d-81bf-95a4d1f99cbf] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:22:05.978817 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell0:SELECT=4 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:22:05.980957 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1a3e09b7-6034-4c7b-b785-d498734a72a7 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:05.981593 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1a3e09b7-6034-4c7b-b785-d498734a72a7 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:05.982603 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1a3e09b7-6034-4c7b-b785-d498734a72a7 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:05.983221 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1a3e09b7-6034-4c7b-b785-d498734a72a7 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:06.221265 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-1a3e09b7-6034-4c7b-b785-d498734a72a7 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'aaf2292818b749f0ae43c66bca929be9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '65644a8df5d84df3b35c894560e0a5c4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:22:06.230459 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-8bc04b70-3f84-4b6f-9ea7-f7fba67b675c tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '029f1334779e4bcdb3f4f2aed564a075', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e1f6ffc3fb64dba8c4474641e796fb8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:22:06.236594 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-1a3e09b7-6034-4c7b-b785-d498734a72a7 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] 10.222.0.16 "GET /compute/v2.1/servers/583e0c15-3053-429d-81bf-95a4d1f99cbf" status: 200 len: 1323 microversion: 2.1 time: 0.263299 Jul 16 21:22:06.236594 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1487/3021] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:22:05 2022] GET /compute/v2.1/servers/583e0c15-3053-429d-81bf-95a4d1f99cbf => generated 1323 bytes in 268 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:22:06.245431 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-8bc04b70-3f84-4b6f-9ea7-f7fba67b675c tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] 10.222.0.16 "GET /compute/v2.1/servers/fa075dcc-e175-449a-9914-d1ee19131f33" status: 200 len: 1371 microversion: 2.1 time: 0.312511 Jul 16 21:22:06.259667 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-f453d861-09ff-4131-b1dd-8c0d2be11cad req-76b875ee-fb0f-4069-b354-0a207dc01ed3 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "583e0c15-3053-429d-81bf-95a4d1f99cbf", "name": "network-vif-plugged", "status": "completed", "tag": "40900dd3-d3df-4e26-80dd-0776ab4b1d20"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:22:06.273544 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1535/3022] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:22:05 2022] GET /compute/v2.1/servers/fa075dcc-e175-449a-9914-d1ee19131f33 => generated 1371 bytes in 341 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:22:06.289784 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-f453d861-09ff-4131-b1dd-8c0d2be11cad req-76b875ee-fb0f-4069-b354-0a207dc01ed3 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:06.290701 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-f453d861-09ff-4131-b1dd-8c0d2be11cad req-76b875ee-fb0f-4069-b354-0a207dc01ed3 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:06.312940 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-f453d861-09ff-4131-b1dd-8c0d2be11cad req-76b875ee-fb0f-4069-b354-0a207dc01ed3 service nova] Creating event network-vif-plugged:40900dd3-d3df-4e26-80dd-0776ab4b1d20 for instance 583e0c15-3053-429d-81bf-95a4d1f99cbf on nch1-master-1 Jul 16 21:22:06.318974 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-f453d861-09ff-4131-b1dd-8c0d2be11cad req-76b875ee-fb0f-4069-b354-0a207dc01ed3 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.079551 Jul 16 21:22:06.327240 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1488/3023] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:22:06 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:22:06.726289 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-21f6aec6-2097-44a6-bf0c-9758e1a7d60c tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:22:06.726913 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-21f6aec6-2097-44a6-bf0c-9758e1a7d60c tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] [instance: 94d9ca2e-1c4d-49ce-918e-02479909b75b] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:22:06.734098 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-21f6aec6-2097-44a6-bf0c-9758e1a7d60c tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:06.734561 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-21f6aec6-2097-44a6-bf0c-9758e1a7d60c tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:06.735375 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-21f6aec6-2097-44a6-bf0c-9758e1a7d60c tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:06.735824 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-21f6aec6-2097-44a6-bf0c-9758e1a7d60c tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:06.763800 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-4baf6948-6c01-48f2-882f-a03de87daa0c tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:22:06.764807 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-4baf6948-6c01-48f2-882f-a03de87daa0c tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: a4f887a4-84e8-4b14-beca-e6ad747ba616] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:22:06.778595 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4baf6948-6c01-48f2-882f-a03de87daa0c tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:06.779179 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4baf6948-6c01-48f2-882f-a03de87daa0c tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:06.779980 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4baf6948-6c01-48f2-882f-a03de87daa0c tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:06.785869 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4baf6948-6c01-48f2-882f-a03de87daa0c tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:07.115344 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-21f6aec6-2097-44a6-bf0c-9758e1a7d60c tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa4c2791e14d4f88883a0af422e4ff0c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c95a2d1e0a034c7ca4a2fcb9fd060d53', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:22:07.129292 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-4baf6948-6c01-48f2-882f-a03de87daa0c tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f2e974ab5aa48dfb4b00f810dcedb63', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd31be8475a6a49d6bfe8132dd050c6c0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:22:07.144362 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-4baf6948-6c01-48f2-882f-a03de87daa0c tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "GET /compute/v2.1/servers/a4f887a4-84e8-4b14-beca-e6ad747ba616" status: 200 len: 1314 microversion: 2.1 time: 0.391953 Jul 16 21:22:07.145890 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1489/3024] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:22:06 2022] GET /compute/v2.1/servers/a4f887a4-84e8-4b14-beca-e6ad747ba616 => generated 1314 bytes in 394 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:22:07.149448 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-21f6aec6-2097-44a6-bf0c-9758e1a7d60c tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] 10.222.0.16 "GET /compute/v2.1/servers/94d9ca2e-1c4d-49ce-918e-02479909b75b" status: 200 len: 1323 microversion: 2.1 time: 0.427525 Jul 16 21:22:07.155164 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1536/3025] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:22:06 2022] GET /compute/v2.1/servers/94d9ca2e-1c4d-49ce-918e-02479909b75b => generated 1323 bytes in 434 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:22:07.264942 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-d4e06b1f-774a-4135-87a5-21ace18d69da tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:22:07.265710 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-d4e06b1f-774a-4135-87a5-21ace18d69da tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] [instance: fa075dcc-e175-449a-9914-d1ee19131f33] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:22:07.276578 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d4e06b1f-774a-4135-87a5-21ace18d69da tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:07.277069 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d4e06b1f-774a-4135-87a5-21ace18d69da tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:07.278438 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d4e06b1f-774a-4135-87a5-21ace18d69da tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:07.278889 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d4e06b1f-774a-4135-87a5-21ace18d69da tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:07.329620 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-cf87ac22-c550-41ce-ae4b-8c2106eb5752 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:22:07.330915 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-cf87ac22-c550-41ce-ae4b-8c2106eb5752 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] [instance: 583e0c15-3053-429d-81bf-95a4d1f99cbf] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:22:07.339388 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cf87ac22-c550-41ce-ae4b-8c2106eb5752 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:07.340587 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cf87ac22-c550-41ce-ae4b-8c2106eb5752 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:07.341603 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cf87ac22-c550-41ce-ae4b-8c2106eb5752 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:07.342325 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cf87ac22-c550-41ce-ae4b-8c2106eb5752 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:07.797043 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-d4e06b1f-774a-4135-87a5-21ace18d69da tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '029f1334779e4bcdb3f4f2aed564a075', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e1f6ffc3fb64dba8c4474641e796fb8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:22:07.805830 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-d4e06b1f-774a-4135-87a5-21ace18d69da tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] 10.222.0.16 "GET /compute/v2.1/servers/fa075dcc-e175-449a-9914-d1ee19131f33" status: 200 len: 1371 microversion: 2.1 time: 0.545063 Jul 16 21:22:07.807594 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1490/3026] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:22:07 2022] GET /compute/v2.1/servers/fa075dcc-e175-449a-9914-d1ee19131f33 => generated 1371 bytes in 547 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:22:07.856199 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-cf87ac22-c550-41ce-ae4b-8c2106eb5752 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'aaf2292818b749f0ae43c66bca929be9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '65644a8df5d84df3b35c894560e0a5c4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:22:07.871107 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-cf87ac22-c550-41ce-ae4b-8c2106eb5752 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] 10.222.0.16 "GET /compute/v2.1/servers/583e0c15-3053-429d-81bf-95a4d1f99cbf" status: 200 len: 1323 microversion: 2.1 time: 0.546762 Jul 16 21:22:07.871709 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1537/3027] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:22:07 2022] GET /compute/v2.1/servers/583e0c15-3053-429d-81bf-95a4d1f99cbf => generated 1323 bytes in 550 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:22:08.164469 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-01a0195c-0d3b-4418-ade8-de07315bd8a8 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:22:08.165296 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-01a0195c-0d3b-4418-ade8-de07315bd8a8 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: a4f887a4-84e8-4b14-beca-e6ad747ba616] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:22:08.172218 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-01a0195c-0d3b-4418-ade8-de07315bd8a8 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:08.172780 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-01a0195c-0d3b-4418-ade8-de07315bd8a8 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:08.172916 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-1d19e990-786f-42ca-925e-46d03f93fc80 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:22:08.173633 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-1d19e990-786f-42ca-925e-46d03f93fc80 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] [instance: 94d9ca2e-1c4d-49ce-918e-02479909b75b] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:22:08.174526 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-01a0195c-0d3b-4418-ade8-de07315bd8a8 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:08.175056 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-01a0195c-0d3b-4418-ade8-de07315bd8a8 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:08.185808 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1d19e990-786f-42ca-925e-46d03f93fc80 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:08.187515 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1d19e990-786f-42ca-925e-46d03f93fc80 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:08.189550 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1d19e990-786f-42ca-925e-46d03f93fc80 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:08.190565 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1d19e990-786f-42ca-925e-46d03f93fc80 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:08.406173 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-01a0195c-0d3b-4418-ade8-de07315bd8a8 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f2e974ab5aa48dfb4b00f810dcedb63', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd31be8475a6a49d6bfe8132dd050c6c0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:22:08.416063 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-01a0195c-0d3b-4418-ade8-de07315bd8a8 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "GET /compute/v2.1/servers/a4f887a4-84e8-4b14-beca-e6ad747ba616" status: 200 len: 1314 microversion: 2.1 time: 0.256554 Jul 16 21:22:08.417812 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1491/3028] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:22:08 2022] GET /compute/v2.1/servers/a4f887a4-84e8-4b14-beca-e6ad747ba616 => generated 1314 bytes in 259 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:22:08.424141 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-07e8dfdf-6a09-4943-b12e-910bdc526e95 req-ae0728b3-8292-4273-84e4-43a1bb7ebc23 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "fa075dcc-e175-449a-9914-d1ee19131f33", "name": "network-vif-plugged", "status": "completed", "tag": "49869004-34cd-4d2b-8bf2-b7cbd8448d51"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:22:08.437233 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-1d19e990-786f-42ca-925e-46d03f93fc80 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa4c2791e14d4f88883a0af422e4ff0c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c95a2d1e0a034c7ca4a2fcb9fd060d53', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:22:08.441703 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-07e8dfdf-6a09-4943-b12e-910bdc526e95 req-ae0728b3-8292-4273-84e4-43a1bb7ebc23 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:08.446358 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-07e8dfdf-6a09-4943-b12e-910bdc526e95 req-ae0728b3-8292-4273-84e4-43a1bb7ebc23 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:08.446669 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-1d19e990-786f-42ca-925e-46d03f93fc80 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] 10.222.0.16 "GET /compute/v2.1/servers/94d9ca2e-1c4d-49ce-918e-02479909b75b" status: 200 len: 1323 microversion: 2.1 time: 0.282104 Jul 16 21:22:08.447218 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1538/3029] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:22:08 2022] GET /compute/v2.1/servers/94d9ca2e-1c4d-49ce-918e-02479909b75b => generated 1323 bytes in 284 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:22:08.466674 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-07e8dfdf-6a09-4943-b12e-910bdc526e95 req-ae0728b3-8292-4273-84e4-43a1bb7ebc23 service nova] Creating event network-vif-plugged:49869004-34cd-4d2b-8bf2-b7cbd8448d51 for instance fa075dcc-e175-449a-9914-d1ee19131f33 on nch2-master-1 Jul 16 21:22:08.478515 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-07e8dfdf-6a09-4943-b12e-910bdc526e95 req-ae0728b3-8292-4273-84e4-43a1bb7ebc23 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.059175 Jul 16 21:22:08.484441 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1492/3030] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:22:08 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 65 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:22:08.831212 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-664de63a-d72b-46b3-9507-1d76c5aa1908 tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:22:08.831840 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-664de63a-d72b-46b3-9507-1d76c5aa1908 tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] [instance: fa075dcc-e175-449a-9914-d1ee19131f33] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:22:08.837998 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-664de63a-d72b-46b3-9507-1d76c5aa1908 tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:08.838463 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-664de63a-d72b-46b3-9507-1d76c5aa1908 tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:08.839263 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-664de63a-d72b-46b3-9507-1d76c5aa1908 tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:08.839709 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-664de63a-d72b-46b3-9507-1d76c5aa1908 tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:08.889306 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-5ac221db-0a79-4409-b223-621849f1ec99 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:22:08.890663 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-5ac221db-0a79-4409-b223-621849f1ec99 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] [instance: 583e0c15-3053-429d-81bf-95a4d1f99cbf] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:22:08.897172 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5ac221db-0a79-4409-b223-621849f1ec99 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:08.902466 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5ac221db-0a79-4409-b223-621849f1ec99 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:08.902466 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5ac221db-0a79-4409-b223-621849f1ec99 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:08.902466 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5ac221db-0a79-4409-b223-621849f1ec99 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:09.033575 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-664de63a-d72b-46b3-9507-1d76c5aa1908 tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '029f1334779e4bcdb3f4f2aed564a075', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e1f6ffc3fb64dba8c4474641e796fb8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:22:09.044000 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-664de63a-d72b-46b3-9507-1d76c5aa1908 tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] 10.222.0.16 "GET /compute/v2.1/servers/fa075dcc-e175-449a-9914-d1ee19131f33" status: 200 len: 1371 microversion: 2.1 time: 0.217254 Jul 16 21:22:09.055624 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1539/3031] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:22:08 2022] GET /compute/v2.1/servers/fa075dcc-e175-449a-9914-d1ee19131f33 => generated 1371 bytes in 237 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:22:09.082179 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-5ac221db-0a79-4409-b223-621849f1ec99 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'aaf2292818b749f0ae43c66bca929be9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '65644a8df5d84df3b35c894560e0a5c4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:22:09.090916 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-5ac221db-0a79-4409-b223-621849f1ec99 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] 10.222.0.16 "GET /compute/v2.1/servers/583e0c15-3053-429d-81bf-95a4d1f99cbf" status: 200 len: 1518 microversion: 2.1 time: 0.206305 Jul 16 21:22:09.091632 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1493/3032] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:22:08 2022] GET /compute/v2.1/servers/583e0c15-3053-429d-81bf-95a4d1f99cbf => generated 1518 bytes in 208 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:22:09.092809 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-f0042231-4d26-484c-a73a-4ffb4bd992cc tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "7734c56d-3537-4b9a-b18e-78dda7314333"}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:22:09.094334 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-f0042231-4d26-484c-a73a-4ffb4bd992cc tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] [instance: f114e124-2903-4f30-b055-9f493e3e9e6c] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:22:09.101136 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f0042231-4d26-484c-a73a-4ffb4bd992cc tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:09.101606 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f0042231-4d26-484c-a73a-4ffb4bd992cc tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:09.102386 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f0042231-4d26-484c-a73a-4ffb4bd992cc tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:09.102838 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f0042231-4d26-484c-a73a-4ffb4bd992cc tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:09.111689 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-59f0393f-08eb-4d74-a584-288a7e33ca51 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Action: 'action', calling method: >, body: {"rescue": {"adminPass": "***"}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:22:09.119973 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-59f0393f-08eb-4d74-a584-288a7e33ca51 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] [instance: 583e0c15-3053-429d-81bf-95a4d1f99cbf] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:22:09.121304 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-59f0393f-08eb-4d74-a584-288a7e33ca51 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:09.122216 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-59f0393f-08eb-4d74-a584-288a7e33ca51 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:09.123035 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-59f0393f-08eb-4d74-a584-288a7e33ca51 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:09.123745 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-59f0393f-08eb-4d74-a584-288a7e33ca51 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:09.152715 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.objects.instance [None req-59f0393f-08eb-4d74-a584-288a7e33ca51 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lazy-loading 'flavor' on Instance uuid 583e0c15-3053-429d-81bf-95a4d1f99cbf {{(pid=138681) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:22:09.628338 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-59f0393f-08eb-4d74-a584-288a7e33ca51 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] 10.222.0.16 "POST /compute/v2.1/servers/583e0c15-3053-429d-81bf-95a4d1f99cbf/action" status: 200 len: 32 microversion: 2.1 time: 0.522351 Jul 16 21:22:09.629420 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1494/3033] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 21:22:09 2022] POST /compute/v2.1/servers/583e0c15-3053-429d-81bf-95a4d1f99cbf/action => generated 32 bytes in 525 msecs (HTTP/1.1 200) 9 headers in 356 bytes (2 switches on core 0) Jul 16 21:22:09.635617 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-0c056b27-ab11-4d26-b495-0f6da0ca8fac tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:22:09.641592 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-0c056b27-ab11-4d26-b495-0f6da0ca8fac tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: a4f887a4-84e8-4b14-beca-e6ad747ba616] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:22:09.644100 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0c056b27-ab11-4d26-b495-0f6da0ca8fac tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:09.644747 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0c056b27-ab11-4d26-b495-0f6da0ca8fac tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:09.645685 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0c056b27-ab11-4d26-b495-0f6da0ca8fac tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:09.646288 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0c056b27-ab11-4d26-b495-0f6da0ca8fac tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:09.823854 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-0c056b27-ab11-4d26-b495-0f6da0ca8fac tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f2e974ab5aa48dfb4b00f810dcedb63', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd31be8475a6a49d6bfe8132dd050c6c0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:22:09.823854 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:22:09.832299 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-0c056b27-ab11-4d26-b495-0f6da0ca8fac tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "GET /compute/v2.1/servers/a4f887a4-84e8-4b14-beca-e6ad747ba616" status: 200 len: 1314 microversion: 2.1 time: 0.201076 Jul 16 21:22:09.834971 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1495/3034] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:22:09 2022] GET /compute/v2.1/servers/a4f887a4-84e8-4b14-beca-e6ad747ba616 => generated 1314 bytes in 204 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:22:09.839936 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-bfe51d92-8d08-4ce3-b49a-b034b406b000 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:22:09.840734 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-bfe51d92-8d08-4ce3-b49a-b034b406b000 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] [instance: 94d9ca2e-1c4d-49ce-918e-02479909b75b] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:22:09.848988 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-bfe51d92-8d08-4ce3-b49a-b034b406b000 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:09.849214 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-bfe51d92-8d08-4ce3-b49a-b034b406b000 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:09.850139 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-bfe51d92-8d08-4ce3-b49a-b034b406b000 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:09.850582 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-bfe51d92-8d08-4ce3-b49a-b034b406b000 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:10.088948 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-f0042231-4d26-484c-a73a-4ffb4bd992cc tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] 10.222.0.16 "POST /compute/v2.1/servers/f114e124-2903-4f30-b055-9f493e3e9e6c/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 1.000624 Jul 16 21:22:10.089551 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1540/3035] 10.222.0.16 () {60 vars in 1322 bytes} [Sat Jul 16 21:22:09 2022] POST /compute/v2.1/servers/f114e124-2903-4f30-b055-9f493e3e9e6c/os-volume_attachments => generated 194 bytes in 1002 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 16 21:22:10.094323 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-bfe51d92-8d08-4ce3-b49a-b034b406b000 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa4c2791e14d4f88883a0af422e4ff0c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c95a2d1e0a034c7ca4a2fcb9fd060d53', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:22:10.103789 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-bfe51d92-8d08-4ce3-b49a-b034b406b000 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] 10.222.0.16 "GET /compute/v2.1/servers/94d9ca2e-1c4d-49ce-918e-02479909b75b" status: 200 len: 1323 microversion: 2.1 time: 0.267160 Jul 16 21:22:10.105151 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1496/3036] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:22:09 2022] GET /compute/v2.1/servers/94d9ca2e-1c4d-49ce-918e-02479909b75b => generated 1323 bytes in 269 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:22:10.106923 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-c4d72780-0002-4303-aee1-3c724b737fa2 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:22:10.107699 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-c4d72780-0002-4303-aee1-3c724b737fa2 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] [instance: 583e0c15-3053-429d-81bf-95a4d1f99cbf] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:22:10.111537 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-ea932101-3713-4e14-9a81-2c9fde6b4b44 tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:22:10.112134 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-ea932101-3713-4e14-9a81-2c9fde6b4b44 tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] [instance: fa075dcc-e175-449a-9914-d1ee19131f33] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:22:10.118486 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ea932101-3713-4e14-9a81-2c9fde6b4b44 tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:10.119185 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ea932101-3713-4e14-9a81-2c9fde6b4b44 tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:10.120311 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ea932101-3713-4e14-9a81-2c9fde6b4b44 tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:10.121377 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ea932101-3713-4e14-9a81-2c9fde6b4b44 tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:10.129906 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c4d72780-0002-4303-aee1-3c724b737fa2 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:10.130383 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c4d72780-0002-4303-aee1-3c724b737fa2 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:10.131268 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c4d72780-0002-4303-aee1-3c724b737fa2 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:10.132522 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c4d72780-0002-4303-aee1-3c724b737fa2 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:10.501057 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-c4d72780-0002-4303-aee1-3c724b737fa2 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'aaf2292818b749f0ae43c66bca929be9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '65644a8df5d84df3b35c894560e0a5c4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:22:10.509904 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-c4d72780-0002-4303-aee1-3c724b737fa2 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] 10.222.0.16 "GET /compute/v2.1/servers/583e0c15-3053-429d-81bf-95a4d1f99cbf" status: 200 len: 1524 microversion: 2.1 time: 0.419159 Jul 16 21:22:10.526680 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1541/3037] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:22:10 2022] GET /compute/v2.1/servers/583e0c15-3053-429d-81bf-95a4d1f99cbf => generated 1524 bytes in 436 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:22:10.531136 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-ea932101-3713-4e14-9a81-2c9fde6b4b44 tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '029f1334779e4bcdb3f4f2aed564a075', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e1f6ffc3fb64dba8c4474641e796fb8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:22:10.537457 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-d427b3ce-e213-43e3-b3f4-dadb2c215e95 req-ae15b61c-d71d-484e-820c-ca33e5674f54 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "94d9ca2e-1c4d-49ce-918e-02479909b75b", "name": "network-vif-plugged", "status": "completed", "tag": "782384db-d832-4551-9c6b-3eb3e7a7352b"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:22:10.540334 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-ea932101-3713-4e14-9a81-2c9fde6b4b44 tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] 10.222.0.16 "GET /compute/v2.1/servers/fa075dcc-e175-449a-9914-d1ee19131f33" status: 200 len: 1371 microversion: 2.1 time: 0.433733 Jul 16 21:22:10.540980 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1497/3038] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:22:10 2022] GET /compute/v2.1/servers/fa075dcc-e175-449a-9914-d1ee19131f33 => generated 1371 bytes in 435 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:22:10.559053 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-d427b3ce-e213-43e3-b3f4-dadb2c215e95 req-ae15b61c-d71d-484e-820c-ca33e5674f54 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:10.559492 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-d427b3ce-e213-43e3-b3f4-dadb2c215e95 req-ae15b61c-d71d-484e-820c-ca33e5674f54 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:10.576656 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-d427b3ce-e213-43e3-b3f4-dadb2c215e95 req-ae15b61c-d71d-484e-820c-ca33e5674f54 service nova] Creating event network-vif-plugged:782384db-d832-4551-9c6b-3eb3e7a7352b for instance 94d9ca2e-1c4d-49ce-918e-02479909b75b on nch2-master-1 Jul 16 21:22:10.584049 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-d427b3ce-e213-43e3-b3f4-dadb2c215e95 req-ae15b61c-d71d-484e-820c-ca33e5674f54 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.055526 Jul 16 21:22:10.585284 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1542/3039] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:22:10 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:22:10.850529 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-a248ecf7-aa01-44da-bb95-6a5feedc5b63 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:22:10.851408 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-a248ecf7-aa01-44da-bb95-6a5feedc5b63 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: a4f887a4-84e8-4b14-beca-e6ad747ba616] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:22:10.858002 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a248ecf7-aa01-44da-bb95-6a5feedc5b63 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:10.858464 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a248ecf7-aa01-44da-bb95-6a5feedc5b63 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:10.859305 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a248ecf7-aa01-44da-bb95-6a5feedc5b63 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:10.859865 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a248ecf7-aa01-44da-bb95-6a5feedc5b63 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:11.025133 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-a248ecf7-aa01-44da-bb95-6a5feedc5b63 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f2e974ab5aa48dfb4b00f810dcedb63', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd31be8475a6a49d6bfe8132dd050c6c0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:22:11.034439 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-a248ecf7-aa01-44da-bb95-6a5feedc5b63 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "GET /compute/v2.1/servers/a4f887a4-84e8-4b14-beca-e6ad747ba616" status: 200 len: 1314 microversion: 2.1 time: 0.187913 Jul 16 21:22:11.039060 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1498/3040] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:22:10 2022] GET /compute/v2.1/servers/a4f887a4-84e8-4b14-beca-e6ad747ba616 => generated 1314 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:22:11.127945 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-83a324ee-2092-44fa-b470-b27eb32fef74 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:22:11.128543 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-83a324ee-2092-44fa-b470-b27eb32fef74 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] [instance: 94d9ca2e-1c4d-49ce-918e-02479909b75b] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:22:11.136767 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-83a324ee-2092-44fa-b470-b27eb32fef74 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:11.137232 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-83a324ee-2092-44fa-b470-b27eb32fef74 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:11.138054 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-83a324ee-2092-44fa-b470-b27eb32fef74 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:11.140981 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-83a324ee-2092-44fa-b470-b27eb32fef74 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:11.309006 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-83a324ee-2092-44fa-b470-b27eb32fef74 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa4c2791e14d4f88883a0af422e4ff0c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c95a2d1e0a034c7ca4a2fcb9fd060d53', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:22:11.319050 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-83a324ee-2092-44fa-b470-b27eb32fef74 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] 10.222.0.16 "GET /compute/v2.1/servers/94d9ca2e-1c4d-49ce-918e-02479909b75b" status: 200 len: 1323 microversion: 2.1 time: 0.195960 Jul 16 21:22:11.322359 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1543/3041] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:22:11 2022] GET /compute/v2.1/servers/94d9ca2e-1c4d-49ce-918e-02479909b75b => generated 1323 bytes in 198 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:22:11.540493 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-49b14de2-18d4-41e3-88b4-c34593871778 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:22:11.541133 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-49b14de2-18d4-41e3-88b4-c34593871778 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] [instance: 583e0c15-3053-429d-81bf-95a4d1f99cbf] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:22:11.550379 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-49b14de2-18d4-41e3-88b4-c34593871778 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:11.551263 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-49b14de2-18d4-41e3-88b4-c34593871778 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:11.552427 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-49b14de2-18d4-41e3-88b4-c34593871778 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:11.553040 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-49b14de2-18d4-41e3-88b4-c34593871778 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:11.556919 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-fffb433d-b67a-4ec1-a43a-1f65e382d724 tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:22:11.557549 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-fffb433d-b67a-4ec1-a43a-1f65e382d724 tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] [instance: fa075dcc-e175-449a-9914-d1ee19131f33] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:22:11.569803 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-fffb433d-b67a-4ec1-a43a-1f65e382d724 tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:11.569803 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-fffb433d-b67a-4ec1-a43a-1f65e382d724 tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:11.569803 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-fffb433d-b67a-4ec1-a43a-1f65e382d724 tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:11.569803 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-fffb433d-b67a-4ec1-a43a-1f65e382d724 tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:11.770804 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-fffb433d-b67a-4ec1-a43a-1f65e382d724 tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '029f1334779e4bcdb3f4f2aed564a075', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e1f6ffc3fb64dba8c4474641e796fb8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:22:11.786334 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-49b14de2-18d4-41e3-88b4-c34593871778 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'aaf2292818b749f0ae43c66bca929be9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '65644a8df5d84df3b35c894560e0a5c4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:22:11.795637 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-49b14de2-18d4-41e3-88b4-c34593871778 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] 10.222.0.16 "GET /compute/v2.1/servers/583e0c15-3053-429d-81bf-95a4d1f99cbf" status: 200 len: 1524 microversion: 2.1 time: 0.259530 Jul 16 21:22:11.797300 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1499/3042] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:22:11 2022] GET /compute/v2.1/servers/583e0c15-3053-429d-81bf-95a4d1f99cbf => generated 1524 bytes in 263 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:22:11.806713 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-fffb433d-b67a-4ec1-a43a-1f65e382d724 tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] 10.222.0.16 "GET /compute/v2.1/servers/fa075dcc-e175-449a-9914-d1ee19131f33" status: 200 len: 1556 microversion: 2.1 time: 0.251744 Jul 16 21:22:11.818784 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1544/3043] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:22:11 2022] GET /compute/v2.1/servers/fa075dcc-e175-449a-9914-d1ee19131f33 => generated 1556 bytes in 257 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:22:11.832649 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-cb04c95c-83e5-4f32-9147-9a63427ecb29 tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:22:11.833196 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-cb04c95c-83e5-4f32-9147-9a63427ecb29 tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] [instance: fa075dcc-e175-449a-9914-d1ee19131f33] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:22:11.841578 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cb04c95c-83e5-4f32-9147-9a63427ecb29 tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:11.846453 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cb04c95c-83e5-4f32-9147-9a63427ecb29 tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:11.846453 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cb04c95c-83e5-4f32-9147-9a63427ecb29 tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:11.846453 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cb04c95c-83e5-4f32-9147-9a63427ecb29 tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:11.950714 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-cb04c95c-83e5-4f32-9147-9a63427ecb29 tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] 10.222.0.16 "GET /compute/v2.1/servers/fa075dcc-e175-449a-9914-d1ee19131f33/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.126403 Jul 16 21:22:11.956304 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1500/3044] 10.222.0.16 () {58 vars in 1274 bytes} [Sat Jul 16 21:22:11 2022] GET /compute/v2.1/servers/fa075dcc-e175-449a-9914-d1ee19131f33/os-interface => generated 284 bytes in 133 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:22:12.052517 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-32a63669-15fc-4ef1-a8cc-4397bdc38c46 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:22:12.053465 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-32a63669-15fc-4ef1-a8cc-4397bdc38c46 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: a4f887a4-84e8-4b14-beca-e6ad747ba616] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:22:12.060188 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-32a63669-15fc-4ef1-a8cc-4397bdc38c46 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:12.060856 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-32a63669-15fc-4ef1-a8cc-4397bdc38c46 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:12.062380 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-32a63669-15fc-4ef1-a8cc-4397bdc38c46 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:12.062847 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-32a63669-15fc-4ef1-a8cc-4397bdc38c46 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:12.336372 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-c88c06d1-521b-4c03-8e9c-6dda0ed770cd tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:22:12.339090 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-c88c06d1-521b-4c03-8e9c-6dda0ed770cd tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] [instance: 94d9ca2e-1c4d-49ce-918e-02479909b75b] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:22:12.343659 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c88c06d1-521b-4c03-8e9c-6dda0ed770cd tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:12.344167 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c88c06d1-521b-4c03-8e9c-6dda0ed770cd tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:12.345114 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c88c06d1-521b-4c03-8e9c-6dda0ed770cd tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:12.345887 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c88c06d1-521b-4c03-8e9c-6dda0ed770cd tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:12.357784 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-32a63669-15fc-4ef1-a8cc-4397bdc38c46 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f2e974ab5aa48dfb4b00f810dcedb63', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd31be8475a6a49d6bfe8132dd050c6c0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:22:12.371415 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-32a63669-15fc-4ef1-a8cc-4397bdc38c46 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "GET /compute/v2.1/servers/a4f887a4-84e8-4b14-beca-e6ad747ba616" status: 200 len: 1314 microversion: 2.1 time: 0.322599 Jul 16 21:22:12.372546 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1545/3045] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:22:12 2022] GET /compute/v2.1/servers/a4f887a4-84e8-4b14-beca-e6ad747ba616 => generated 1314 bytes in 324 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:22:12.600465 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-f53f2379-8cdf-467c-8cab-b6409943802c req-c32b7e9a-b1db-48c9-bc33-79cd3a099075 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "9e7787d1-264a-4707-8604-3e8b70133376", "name": "network-vif-plugged", "status": "completed", "tag": "b5213034-c4ce-4421-b1cd-864c1901ff82"}, {"server_uuid": "a4f887a4-84e8-4b14-beca-e6ad747ba616", "name": "network-vif-plugged", "status": "completed", "tag": "ced0cb52-5511-4c5a-9ebd-aab291a021f0"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:22:12.612756 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-f53f2379-8cdf-467c-8cab-b6409943802c req-c32b7e9a-b1db-48c9-bc33-79cd3a099075 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:12.613160 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-f53f2379-8cdf-467c-8cab-b6409943802c req-c32b7e9a-b1db-48c9-bc33-79cd3a099075 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:12.628042 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-f53f2379-8cdf-467c-8cab-b6409943802c req-c32b7e9a-b1db-48c9-bc33-79cd3a099075 service nova] Creating event network-vif-plugged:b5213034-c4ce-4421-b1cd-864c1901ff82 for instance 9e7787d1-264a-4707-8604-3e8b70133376 on nch1-master-1 Jul 16 21:22:12.629277 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-f53f2379-8cdf-467c-8cab-b6409943802c req-c32b7e9a-b1db-48c9-bc33-79cd3a099075 service nova] Creating event network-vif-plugged:ced0cb52-5511-4c5a-9ebd-aab291a021f0 for instance a4f887a4-84e8-4b14-beca-e6ad747ba616 on nch1-master-1 Jul 16 21:22:12.640103 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-f53f2379-8cdf-467c-8cab-b6409943802c req-c32b7e9a-b1db-48c9-bc33-79cd3a099075 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 354 microversion: 2.1 time: 0.039105 Jul 16 21:22:12.640103 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1546/3046] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:22:12 2022] POST /compute/v2.1/os-server-external-events => generated 354 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 16 21:22:12.689088 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-c88c06d1-521b-4c03-8e9c-6dda0ed770cd tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa4c2791e14d4f88883a0af422e4ff0c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c95a2d1e0a034c7ca4a2fcb9fd060d53', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:22:12.698062 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-c88c06d1-521b-4c03-8e9c-6dda0ed770cd tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] 10.222.0.16 "GET /compute/v2.1/servers/94d9ca2e-1c4d-49ce-918e-02479909b75b" status: 200 len: 1515 microversion: 2.1 time: 0.365080 Jul 16 21:22:12.698998 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1501/3047] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:22:12 2022] GET /compute/v2.1/servers/94d9ca2e-1c4d-49ce-918e-02479909b75b => generated 1515 bytes in 367 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:22:12.716704 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-70811e55-b109-4d90-b2ac-665bd4f4907a tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:22:12.717458 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-70811e55-b109-4d90-b2ac-665bd4f4907a tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] [instance: 9e7787d1-264a-4707-8604-3e8b70133376] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:22:12.724897 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-70811e55-b109-4d90-b2ac-665bd4f4907a tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:12.725367 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-70811e55-b109-4d90-b2ac-665bd4f4907a tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:12.726152 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-70811e55-b109-4d90-b2ac-665bd4f4907a tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:12.726602 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-70811e55-b109-4d90-b2ac-665bd4f4907a tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:12.813943 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-9cd4ab0b-870f-40d4-b7c4-1f7bad3d0019 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:22:12.814507 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-9cd4ab0b-870f-40d4-b7c4-1f7bad3d0019 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] [instance: 583e0c15-3053-429d-81bf-95a4d1f99cbf] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:22:12.824824 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9cd4ab0b-870f-40d4-b7c4-1f7bad3d0019 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:12.824824 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9cd4ab0b-870f-40d4-b7c4-1f7bad3d0019 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:12.824824 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9cd4ab0b-870f-40d4-b7c4-1f7bad3d0019 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:12.824824 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9cd4ab0b-870f-40d4-b7c4-1f7bad3d0019 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:13.263442 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-70811e55-b109-4d90-b2ac-665bd4f4907a tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa4c2791e14d4f88883a0af422e4ff0c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c95a2d1e0a034c7ca4a2fcb9fd060d53', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:22:13.271957 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-70811e55-b109-4d90-b2ac-665bd4f4907a tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] 10.222.0.16 "GET /compute/v2.1/servers/9e7787d1-264a-4707-8604-3e8b70133376" status: 200 len: 1323 microversion: 2.1 time: 0.559501 Jul 16 21:22:13.272764 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1547/3048] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:22:12 2022] GET /compute/v2.1/servers/9e7787d1-264a-4707-8604-3e8b70133376 => generated 1323 bytes in 561 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:22:13.300062 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-9cd4ab0b-870f-40d4-b7c4-1f7bad3d0019 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'aaf2292818b749f0ae43c66bca929be9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '65644a8df5d84df3b35c894560e0a5c4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:22:13.309031 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-9cd4ab0b-870f-40d4-b7c4-1f7bad3d0019 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] 10.222.0.16 "GET /compute/v2.1/servers/583e0c15-3053-429d-81bf-95a4d1f99cbf" status: 200 len: 1524 microversion: 2.1 time: 0.499304 Jul 16 21:22:13.309616 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1502/3049] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:22:12 2022] GET /compute/v2.1/servers/583e0c15-3053-429d-81bf-95a4d1f99cbf => generated 1524 bytes in 501 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:22:13.389425 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-261b1e13-3887-44b3-b593-b7e0110391ea tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:22:13.390270 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-261b1e13-3887-44b3-b593-b7e0110391ea tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: a4f887a4-84e8-4b14-beca-e6ad747ba616] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:22:13.405034 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-261b1e13-3887-44b3-b593-b7e0110391ea tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:13.406029 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-261b1e13-3887-44b3-b593-b7e0110391ea tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:13.406813 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-261b1e13-3887-44b3-b593-b7e0110391ea tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:13.407274 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-261b1e13-3887-44b3-b593-b7e0110391ea tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:13.954503 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-261b1e13-3887-44b3-b593-b7e0110391ea tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f2e974ab5aa48dfb4b00f810dcedb63', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd31be8475a6a49d6bfe8132dd050c6c0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:22:13.965979 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-261b1e13-3887-44b3-b593-b7e0110391ea tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "GET /compute/v2.1/servers/a4f887a4-84e8-4b14-beca-e6ad747ba616" status: 200 len: 1314 microversion: 2.1 time: 0.579707 Jul 16 21:22:13.966945 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1548/3050] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:22:13 2022] GET /compute/v2.1/servers/a4f887a4-84e8-4b14-beca-e6ad747ba616 => generated 1314 bytes in 582 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:22:14.108002 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-8fbf4a9e-30a0-40ba-9014-1465e422fa3c req-e201b8da-bf23-4482-b503-4b966f721bce service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "fa075dcc-e175-449a-9914-d1ee19131f33", "tag": "49869004-34cd-4d2b-8bf2-b7cbd8448d51"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:22:14.121400 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-8fbf4a9e-30a0-40ba-9014-1465e422fa3c req-e201b8da-bf23-4482-b503-4b966f721bce service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:14.121400 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-8fbf4a9e-30a0-40ba-9014-1465e422fa3c req-e201b8da-bf23-4482-b503-4b966f721bce service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:14.132593 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-8fbf4a9e-30a0-40ba-9014-1465e422fa3c req-e201b8da-bf23-4482-b503-4b966f721bce service nova] Creating event network-changed:49869004-34cd-4d2b-8bf2-b7cbd8448d51 for instance fa075dcc-e175-449a-9914-d1ee19131f33 on nch2-master-1 Jul 16 21:22:14.137648 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-8fbf4a9e-30a0-40ba-9014-1465e422fa3c req-e201b8da-bf23-4482-b503-4b966f721bce service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.033610 Jul 16 21:22:14.138258 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1503/3051] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:22:14 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:22:14.290894 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-7f66e37e-3031-4477-be2e-eba5c39b5e0a tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:22:14.292134 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-7f66e37e-3031-4477-be2e-eba5c39b5e0a tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] [instance: 9e7787d1-264a-4707-8604-3e8b70133376] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:22:14.301063 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7f66e37e-3031-4477-be2e-eba5c39b5e0a tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:14.301815 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7f66e37e-3031-4477-be2e-eba5c39b5e0a tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:14.302892 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7f66e37e-3031-4477-be2e-eba5c39b5e0a tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:14.303813 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7f66e37e-3031-4477-be2e-eba5c39b5e0a tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:14.308809 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:22:14.330313 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-a6a2b6c2-7d7c-48f0-8e25-b36b8500fa03 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:22:14.331113 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-a6a2b6c2-7d7c-48f0-8e25-b36b8500fa03 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] [instance: 583e0c15-3053-429d-81bf-95a4d1f99cbf] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:22:14.340361 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a6a2b6c2-7d7c-48f0-8e25-b36b8500fa03 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:14.341021 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a6a2b6c2-7d7c-48f0-8e25-b36b8500fa03 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:14.341984 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a6a2b6c2-7d7c-48f0-8e25-b36b8500fa03 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:14.343174 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a6a2b6c2-7d7c-48f0-8e25-b36b8500fa03 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:14.726078 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-7f66e37e-3031-4477-be2e-eba5c39b5e0a tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa4c2791e14d4f88883a0af422e4ff0c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c95a2d1e0a034c7ca4a2fcb9fd060d53', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:22:14.735800 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-7f66e37e-3031-4477-be2e-eba5c39b5e0a tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] 10.222.0.16 "GET /compute/v2.1/servers/9e7787d1-264a-4707-8604-3e8b70133376" status: 200 len: 1323 microversion: 2.1 time: 0.449978 Jul 16 21:22:14.736423 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1549/3052] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:22:14 2022] GET /compute/v2.1/servers/9e7787d1-264a-4707-8604-3e8b70133376 => generated 1323 bytes in 452 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:22:14.748145 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-a6a2b6c2-7d7c-48f0-8e25-b36b8500fa03 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'aaf2292818b749f0ae43c66bca929be9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '65644a8df5d84df3b35c894560e0a5c4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:22:14.757579 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-a6a2b6c2-7d7c-48f0-8e25-b36b8500fa03 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] 10.222.0.16 "GET /compute/v2.1/servers/583e0c15-3053-429d-81bf-95a4d1f99cbf" status: 200 len: 1524 microversion: 2.1 time: 0.433337 Jul 16 21:22:14.758163 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1504/3053] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:22:14 2022] GET /compute/v2.1/servers/583e0c15-3053-429d-81bf-95a4d1f99cbf => generated 1524 bytes in 435 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:22:14.995864 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-f538b656-ec66-416d-8e36-5822a8babd73 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:22:14.998345 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-f538b656-ec66-416d-8e36-5822a8babd73 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: a4f887a4-84e8-4b14-beca-e6ad747ba616] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:22:15.008524 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f538b656-ec66-416d-8e36-5822a8babd73 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:15.013419 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f538b656-ec66-416d-8e36-5822a8babd73 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:15.013419 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f538b656-ec66-416d-8e36-5822a8babd73 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:15.013419 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f538b656-ec66-416d-8e36-5822a8babd73 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:15.322125 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-f538b656-ec66-416d-8e36-5822a8babd73 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f2e974ab5aa48dfb4b00f810dcedb63', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd31be8475a6a49d6bfe8132dd050c6c0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:22:15.331783 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-f538b656-ec66-416d-8e36-5822a8babd73 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "GET /compute/v2.1/servers/a4f887a4-84e8-4b14-beca-e6ad747ba616" status: 200 len: 1502 microversion: 2.1 time: 0.351620 Jul 16 21:22:15.333511 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1550/3054] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:22:14 2022] GET /compute/v2.1/servers/a4f887a4-84e8-4b14-beca-e6ad747ba616 => generated 1502 bytes in 355 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:22:15.354297 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-befa13ef-b51c-4c30-960f-aacd52ff2ce8 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:22:15.354874 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-befa13ef-b51c-4c30-960f-aacd52ff2ce8 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: a4f887a4-84e8-4b14-beca-e6ad747ba616] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:22:15.363120 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-befa13ef-b51c-4c30-960f-aacd52ff2ce8 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:15.363595 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-befa13ef-b51c-4c30-960f-aacd52ff2ce8 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:15.365573 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-befa13ef-b51c-4c30-960f-aacd52ff2ce8 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:15.367671 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-befa13ef-b51c-4c30-960f-aacd52ff2ce8 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:15.397328 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-befa13ef-b51c-4c30-960f-aacd52ff2ce8 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: a4f887a4-84e8-4b14-beca-e6ad747ba616] Going to try to terminate instance {{(pid=138681) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 16 21:22:15.752951 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-11580271-1504-413b-9b8e-9120027809e4 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:22:15.753667 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-11580271-1504-413b-9b8e-9120027809e4 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] [instance: 9e7787d1-264a-4707-8604-3e8b70133376] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:22:15.764388 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-11580271-1504-413b-9b8e-9120027809e4 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:15.764845 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-11580271-1504-413b-9b8e-9120027809e4 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:15.765633 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-11580271-1504-413b-9b8e-9120027809e4 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:15.766079 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-11580271-1504-413b-9b8e-9120027809e4 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:15.803495 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-befa13ef-b51c-4c30-960f-aacd52ff2ce8 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "DELETE /compute/v2.1/servers/a4f887a4-84e8-4b14-beca-e6ad747ba616" status: 204 len: 0 microversion: 2.1 time: 0.455694 Jul 16 21:22:15.805545 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1505/3055] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:22:15 2022] DELETE /compute/v2.1/servers/a4f887a4-84e8-4b14-beca-e6ad747ba616 => generated 0 bytes in 459 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 16 21:22:15.812548 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-16bd0cb2-a225-4a46-98b1-826b850aff08 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:22:15.813372 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-16bd0cb2-a225-4a46-98b1-826b850aff08 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] [instance: 583e0c15-3053-429d-81bf-95a4d1f99cbf] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:22:15.826010 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-16bd0cb2-a225-4a46-98b1-826b850aff08 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:15.826629 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-16bd0cb2-a225-4a46-98b1-826b850aff08 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:15.827730 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-16bd0cb2-a225-4a46-98b1-826b850aff08 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:15.828417 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-16bd0cb2-a225-4a46-98b1-826b850aff08 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:16.052691 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-11580271-1504-413b-9b8e-9120027809e4 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa4c2791e14d4f88883a0af422e4ff0c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c95a2d1e0a034c7ca4a2fcb9fd060d53', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:22:16.060466 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-11580271-1504-413b-9b8e-9120027809e4 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] 10.222.0.16 "GET /compute/v2.1/servers/9e7787d1-264a-4707-8604-3e8b70133376" status: 200 len: 1516 microversion: 2.1 time: 0.311803 Jul 16 21:22:16.061345 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1551/3056] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:22:15 2022] GET /compute/v2.1/servers/9e7787d1-264a-4707-8604-3e8b70133376 => generated 1516 bytes in 313 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:22:16.078382 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-6a1f45e4-5929-4453-b887-a183640e6e7c tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:22:16.079269 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-6a1f45e4-5929-4453-b887-a183640e6e7c tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: a4f887a4-84e8-4b14-beca-e6ad747ba616] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:22:16.085794 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6a1f45e4-5929-4453-b887-a183640e6e7c tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:16.086493 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6a1f45e4-5929-4453-b887-a183640e6e7c tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:16.087464 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6a1f45e4-5929-4453-b887-a183640e6e7c tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:16.088066 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6a1f45e4-5929-4453-b887-a183640e6e7c tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:16.106624 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-16bd0cb2-a225-4a46-98b1-826b850aff08 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'aaf2292818b749f0ae43c66bca929be9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '65644a8df5d84df3b35c894560e0a5c4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:22:16.115532 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-16bd0cb2-a225-4a46-98b1-826b850aff08 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] 10.222.0.16 "GET /compute/v2.1/servers/583e0c15-3053-429d-81bf-95a4d1f99cbf" status: 200 len: 1524 microversion: 2.1 time: 0.308401 Jul 16 21:22:16.116644 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1506/3057] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:22:15 2022] GET /compute/v2.1/servers/583e0c15-3053-429d-81bf-95a4d1f99cbf => generated 1524 bytes in 310 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:22:16.125638 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-ad593fd1-196e-4817-8499-b50ecd4f903e tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:22:16.126485 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-ad593fd1-196e-4817-8499-b50ecd4f903e tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] [instance: 8acee151-2b52-4ca3-b2e1-18a19834b0e2] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:22:16.139510 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ad593fd1-196e-4817-8499-b50ecd4f903e tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:16.139510 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ad593fd1-196e-4817-8499-b50ecd4f903e tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:16.139510 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ad593fd1-196e-4817-8499-b50ecd4f903e tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:16.139510 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ad593fd1-196e-4817-8499-b50ecd4f903e tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:16.295586 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-6a1f45e4-5929-4453-b887-a183640e6e7c tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f2e974ab5aa48dfb4b00f810dcedb63', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd31be8475a6a49d6bfe8132dd050c6c0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:22:16.304951 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-6a1f45e4-5929-4453-b887-a183640e6e7c tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "GET /compute/v2.1/servers/a4f887a4-84e8-4b14-beca-e6ad747ba616" status: 200 len: 1508 microversion: 2.1 time: 0.232499 Jul 16 21:22:16.313460 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1552/3058] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:22:16 2022] GET /compute/v2.1/servers/a4f887a4-84e8-4b14-beca-e6ad747ba616 => generated 1508 bytes in 242 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:22:16.333845 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-ad593fd1-196e-4817-8499-b50ecd4f903e tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa4c2791e14d4f88883a0af422e4ff0c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c95a2d1e0a034c7ca4a2fcb9fd060d53', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:22:16.342115 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-ad593fd1-196e-4817-8499-b50ecd4f903e tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] 10.222.0.16 "GET /compute/v2.1/servers/8acee151-2b52-4ca3-b2e1-18a19834b0e2" status: 200 len: 1515 microversion: 2.1 time: 0.224183 Jul 16 21:22:16.342784 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1507/3059] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:22:16 2022] GET /compute/v2.1/servers/8acee151-2b52-4ca3-b2e1-18a19834b0e2 => generated 1515 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:22:16.359273 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-13d91883-3009-40fc-badc-1b629c3c8396 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:22:16.360152 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-13d91883-3009-40fc-badc-1b629c3c8396 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] [instance: 8acee151-2b52-4ca3-b2e1-18a19834b0e2] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:22:16.366550 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-13d91883-3009-40fc-badc-1b629c3c8396 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:16.367235 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-13d91883-3009-40fc-badc-1b629c3c8396 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:16.368225 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-13d91883-3009-40fc-badc-1b629c3c8396 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:16.368674 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-13d91883-3009-40fc-badc-1b629c3c8396 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:16.397392 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-13d91883-3009-40fc-badc-1b629c3c8396 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] [instance: 8acee151-2b52-4ca3-b2e1-18a19834b0e2] Going to try to terminate instance {{(pid=138680) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 16 21:22:16.747156 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-6ba04082-2010-47eb-8317-d3e5055effea tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Action: 'action', calling method: >, body: {"os-stop": {}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:22:16.754858 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6ba04082-2010-47eb-8317-d3e5055effea tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:16.758017 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6ba04082-2010-47eb-8317-d3e5055effea tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:16.817541 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-6ba04082-2010-47eb-8317-d3e5055effea tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] [instance: f114e124-2903-4f30-b055-9f493e3e9e6c] Going to try to stop instance {{(pid=138681) force_stop /opt/stack/nova/nova/compute/api.py:2707}} Jul 16 21:22:16.884234 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-13d91883-3009-40fc-badc-1b629c3c8396 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] 10.222.0.16 "DELETE /compute/v2.1/servers/8acee151-2b52-4ca3-b2e1-18a19834b0e2" status: 204 len: 0 microversion: 2.1 time: 0.529268 Jul 16 21:22:16.884856 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1553/3060] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:22:16 2022] DELETE /compute/v2.1/servers/8acee151-2b52-4ca3-b2e1-18a19834b0e2 => generated 0 bytes in 531 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 16 21:22:16.893326 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-65e5ef56-32ca-4c17-b8b1-774e758756e4 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:22:16.894057 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-65e5ef56-32ca-4c17-b8b1-774e758756e4 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] [instance: 8acee151-2b52-4ca3-b2e1-18a19834b0e2] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:22:16.900050 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-65e5ef56-32ca-4c17-b8b1-774e758756e4 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:16.900497 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-65e5ef56-32ca-4c17-b8b1-774e758756e4 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:16.901277 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-65e5ef56-32ca-4c17-b8b1-774e758756e4 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:16.901715 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-65e5ef56-32ca-4c17-b8b1-774e758756e4 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:17.049659 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-6ba04082-2010-47eb-8317-d3e5055effea tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] 10.222.0.16 "POST /compute/v2.1/servers/f114e124-2903-4f30-b055-9f493e3e9e6c/action" status: 202 len: 0 microversion: 2.1 time: 0.306181 Jul 16 21:22:17.053173 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1508/3061] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 21:22:16 2022] POST /compute/v2.1/servers/f114e124-2903-4f30-b055-9f493e3e9e6c/action => generated 0 bytes in 308 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 21:22:17.054780 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-65e5ef56-32ca-4c17-b8b1-774e758756e4 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa4c2791e14d4f88883a0af422e4ff0c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c95a2d1e0a034c7ca4a2fcb9fd060d53', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:22:17.065577 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-65e5ef56-32ca-4c17-b8b1-774e758756e4 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] 10.222.0.16 "GET /compute/v2.1/servers/8acee151-2b52-4ca3-b2e1-18a19834b0e2" status: 200 len: 1521 microversion: 2.1 time: 0.176403 Jul 16 21:22:17.065774 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-fe1c1836-ae83-4dd7-bac5-f25fadeed6e7 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:22:17.066165 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1554/3062] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:22:16 2022] GET /compute/v2.1/servers/8acee151-2b52-4ca3-b2e1-18a19834b0e2 => generated 1521 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:22:17.066381 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-fe1c1836-ae83-4dd7-bac5-f25fadeed6e7 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] [instance: f114e124-2903-4f30-b055-9f493e3e9e6c] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:22:17.081239 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fe1c1836-ae83-4dd7-bac5-f25fadeed6e7 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:17.082015 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fe1c1836-ae83-4dd7-bac5-f25fadeed6e7 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:17.083245 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fe1c1836-ae83-4dd7-bac5-f25fadeed6e7 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:17.083966 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fe1c1836-ae83-4dd7-bac5-f25fadeed6e7 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:17.134886 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-47305734-3a8a-450d-baf6-d399606e8ad8 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:22:17.136001 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-47305734-3a8a-450d-baf6-d399606e8ad8 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] [instance: 583e0c15-3053-429d-81bf-95a4d1f99cbf] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:22:17.138353 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_api:SELECT=96,nova_api:INSERT=3,nova_api:UPDATE=1 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:22:17.147548 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-47305734-3a8a-450d-baf6-d399606e8ad8 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:17.148301 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-47305734-3a8a-450d-baf6-d399606e8ad8 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:17.149359 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-47305734-3a8a-450d-baf6-d399606e8ad8 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:17.150096 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-47305734-3a8a-450d-baf6-d399606e8ad8 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:17.389544 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-fe1c1836-ae83-4dd7-bac5-f25fadeed6e7 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bba77f6ef42547f1aa8480e27868bd4a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '22ca92a86e6c40d59071e58ce4085568', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:22:17.399403 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-fe1c1836-ae83-4dd7-bac5-f25fadeed6e7 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] 10.222.0.16 "GET /compute/v2.1/servers/f114e124-2903-4f30-b055-9f493e3e9e6c" status: 200 len: 1725 microversion: 2.1 time: 0.343240 Jul 16 21:22:17.406280 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1509/3063] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:22:17 2022] GET /compute/v2.1/servers/f114e124-2903-4f30-b055-9f493e3e9e6c => generated 1725 bytes in 346 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:22:17.408349 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-a18b6acd-c8a6-4dc9-83a3-edcb4c62fbf2 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:22:17.409319 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-a18b6acd-c8a6-4dc9-83a3-edcb4c62fbf2 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: a4f887a4-84e8-4b14-beca-e6ad747ba616] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:22:17.417344 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a18b6acd-c8a6-4dc9-83a3-edcb4c62fbf2 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:17.418278 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a18b6acd-c8a6-4dc9-83a3-edcb4c62fbf2 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:17.419548 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a18b6acd-c8a6-4dc9-83a3-edcb4c62fbf2 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:17.420207 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a18b6acd-c8a6-4dc9-83a3-edcb4c62fbf2 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:17.426825 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-47305734-3a8a-450d-baf6-d399606e8ad8 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'aaf2292818b749f0ae43c66bca929be9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '65644a8df5d84df3b35c894560e0a5c4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:22:17.438337 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-47305734-3a8a-450d-baf6-d399606e8ad8 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] 10.222.0.16 "GET /compute/v2.1/servers/583e0c15-3053-429d-81bf-95a4d1f99cbf" status: 200 len: 1524 microversion: 2.1 time: 0.308098 Jul 16 21:22:17.439355 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1555/3064] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:22:17 2022] GET /compute/v2.1/servers/583e0c15-3053-429d-81bf-95a4d1f99cbf => generated 1524 bytes in 310 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:22:17.674740 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-a18b6acd-c8a6-4dc9-83a3-edcb4c62fbf2 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f2e974ab5aa48dfb4b00f810dcedb63', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd31be8475a6a49d6bfe8132dd050c6c0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:22:17.677338 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=96,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:22:17.684563 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-a18b6acd-c8a6-4dc9-83a3-edcb4c62fbf2 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "GET /compute/v2.1/servers/a4f887a4-84e8-4b14-beca-e6ad747ba616" status: 200 len: 1508 microversion: 2.1 time: 0.282288 Jul 16 21:22:17.685152 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1510/3065] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:22:17 2022] GET /compute/v2.1/servers/a4f887a4-84e8-4b14-beca-e6ad747ba616 => generated 1508 bytes in 283 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:22:18.050953 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-b2f8216f-119b-4595-a0bd-271148872533 req-cf874ce1-a240-4520-9007-67cfda831e36 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "a4f887a4-84e8-4b14-beca-e6ad747ba616", "name": "network-vif-deleted", "tag": "ced0cb52-5511-4c5a-9ebd-aab291a021f0"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:22:18.061999 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-b2f8216f-119b-4595-a0bd-271148872533 req-cf874ce1-a240-4520-9007-67cfda831e36 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:18.062698 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-b2f8216f-119b-4595-a0bd-271148872533 req-cf874ce1-a240-4520-9007-67cfda831e36 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:18.076423 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-b2f8216f-119b-4595-a0bd-271148872533 req-cf874ce1-a240-4520-9007-67cfda831e36 service nova] Creating event network-vif-deleted:ced0cb52-5511-4c5a-9ebd-aab291a021f0 for instance a4f887a4-84e8-4b14-beca-e6ad747ba616 on nch1-master-1 Jul 16 21:22:18.083549 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-b2f8216f-119b-4595-a0bd-271148872533 req-cf874ce1-a240-4520-9007-67cfda831e36 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.037302 Jul 16 21:22:18.083766 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-3e5cd104-4068-4960-afa5-0769a6b08494 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:22:18.084310 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-3e5cd104-4068-4960-afa5-0769a6b08494 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] [instance: 8acee151-2b52-4ca3-b2e1-18a19834b0e2] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:22:18.085508 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1556/3066] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:22:18 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:22:18.104317 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3e5cd104-4068-4960-afa5-0769a6b08494 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:18.104896 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3e5cd104-4068-4960-afa5-0769a6b08494 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:18.105694 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3e5cd104-4068-4960-afa5-0769a6b08494 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:18.106125 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3e5cd104-4068-4960-afa5-0769a6b08494 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:18.276136 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-3e5cd104-4068-4960-afa5-0769a6b08494 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa4c2791e14d4f88883a0af422e4ff0c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c95a2d1e0a034c7ca4a2fcb9fd060d53', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:22:18.283717 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-3e5cd104-4068-4960-afa5-0769a6b08494 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] 10.222.0.16 "GET /compute/v2.1/servers/8acee151-2b52-4ca3-b2e1-18a19834b0e2" status: 200 len: 1521 microversion: 2.1 time: 0.204246 Jul 16 21:22:18.284301 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1511/3067] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:22:18 2022] GET /compute/v2.1/servers/8acee151-2b52-4ca3-b2e1-18a19834b0e2 => generated 1521 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:22:18.418411 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-6a89d29a-3faa-46c6-a8f3-952764b994aa tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:22:18.419155 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-6a89d29a-3faa-46c6-a8f3-952764b994aa tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] [instance: f114e124-2903-4f30-b055-9f493e3e9e6c] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:22:18.425965 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6a89d29a-3faa-46c6-a8f3-952764b994aa tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:18.426422 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6a89d29a-3faa-46c6-a8f3-952764b994aa tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:18.427301 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6a89d29a-3faa-46c6-a8f3-952764b994aa tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:18.428368 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6a89d29a-3faa-46c6-a8f3-952764b994aa tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:18.456885 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-411ff72d-3df7-4fd0-925d-86034ff60473 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:22:18.457794 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-411ff72d-3df7-4fd0-925d-86034ff60473 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] [instance: 583e0c15-3053-429d-81bf-95a4d1f99cbf] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:22:18.466269 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-411ff72d-3df7-4fd0-925d-86034ff60473 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:18.466873 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-411ff72d-3df7-4fd0-925d-86034ff60473 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:18.467803 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-411ff72d-3df7-4fd0-925d-86034ff60473 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:18.468429 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-411ff72d-3df7-4fd0-925d-86034ff60473 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:18.756813 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-6a89d29a-3faa-46c6-a8f3-952764b994aa tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bba77f6ef42547f1aa8480e27868bd4a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '22ca92a86e6c40d59071e58ce4085568', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:22:18.766883 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-6a89d29a-3faa-46c6-a8f3-952764b994aa tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] 10.222.0.16 "GET /compute/v2.1/servers/f114e124-2903-4f30-b055-9f493e3e9e6c" status: 200 len: 1725 microversion: 2.1 time: 0.352803 Jul 16 21:22:18.767911 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1557/3068] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:22:18 2022] GET /compute/v2.1/servers/f114e124-2903-4f30-b055-9f493e3e9e6c => generated 1725 bytes in 355 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:22:18.784481 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-786c0e5b-4a35-47ba-99da-9b47e37d75f3 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:22:18.786273 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-786c0e5b-4a35-47ba-99da-9b47e37d75f3 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: a4f887a4-84e8-4b14-beca-e6ad747ba616] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:22:18.789063 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-411ff72d-3df7-4fd0-925d-86034ff60473 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'aaf2292818b749f0ae43c66bca929be9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '65644a8df5d84df3b35c894560e0a5c4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:22:18.793190 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-786c0e5b-4a35-47ba-99da-9b47e37d75f3 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:18.793670 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-786c0e5b-4a35-47ba-99da-9b47e37d75f3 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:18.794657 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-786c0e5b-4a35-47ba-99da-9b47e37d75f3 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:18.795151 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-786c0e5b-4a35-47ba-99da-9b47e37d75f3 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:18.801348 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-411ff72d-3df7-4fd0-925d-86034ff60473 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] 10.222.0.16 "GET /compute/v2.1/servers/583e0c15-3053-429d-81bf-95a4d1f99cbf" status: 200 len: 1524 microversion: 2.1 time: 0.348752 Jul 16 21:22:18.802260 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1512/3069] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:22:18 2022] GET /compute/v2.1/servers/583e0c15-3053-429d-81bf-95a4d1f99cbf => generated 1524 bytes in 350 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:22:18.841841 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-786c0e5b-4a35-47ba-99da-9b47e37d75f3 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:18.842392 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-786c0e5b-4a35-47ba-99da-9b47e37d75f3 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:18.842951 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.objects.instance [None req-786c0e5b-4a35-47ba-99da-9b47e37d75f3 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lazy-loading 'fault' on Instance uuid a4f887a4-84e8-4b14-beca-e6ad747ba616 {{(pid=138680) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:22:18.906651 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-786c0e5b-4a35-47ba-99da-9b47e37d75f3 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f2e974ab5aa48dfb4b00f810dcedb63', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd31be8475a6a49d6bfe8132dd050c6c0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:22:18.917866 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-786c0e5b-4a35-47ba-99da-9b47e37d75f3 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "GET /compute/v2.1/servers/a4f887a4-84e8-4b14-beca-e6ad747ba616" status: 200 len: 1304 microversion: 2.1 time: 0.137556 Jul 16 21:22:18.918455 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1558/3070] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:22:18 2022] GET /compute/v2.1/servers/a4f887a4-84e8-4b14-beca-e6ad747ba616 => generated 1304 bytes in 139 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:22:19.307666 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-674c6bb2-a320-4131-b63c-0395c6b959ef tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:22:19.309748 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-674c6bb2-a320-4131-b63c-0395c6b959ef tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] [instance: 8acee151-2b52-4ca3-b2e1-18a19834b0e2] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:22:19.327160 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-674c6bb2-a320-4131-b63c-0395c6b959ef tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:19.328087 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-674c6bb2-a320-4131-b63c-0395c6b959ef tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:19.330487 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-674c6bb2-a320-4131-b63c-0395c6b959ef tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:19.331179 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-674c6bb2-a320-4131-b63c-0395c6b959ef tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:19.404549 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-674c6bb2-a320-4131-b63c-0395c6b959ef tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa4c2791e14d4f88883a0af422e4ff0c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c95a2d1e0a034c7ca4a2fcb9fd060d53', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:22:19.413262 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-674c6bb2-a320-4131-b63c-0395c6b959ef tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] 10.222.0.16 "GET /compute/v2.1/servers/8acee151-2b52-4ca3-b2e1-18a19834b0e2" status: 200 len: 1479 microversion: 2.1 time: 0.114609 Jul 16 21:22:19.423348 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1513/3071] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:22:19 2022] GET /compute/v2.1/servers/8acee151-2b52-4ca3-b2e1-18a19834b0e2 => generated 1479 bytes in 126 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:22:19.785339 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-50263d37-bd59-4047-8523-86d807328ade tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:22:19.785963 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-50263d37-bd59-4047-8523-86d807328ade tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] [instance: f114e124-2903-4f30-b055-9f493e3e9e6c] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:22:19.793171 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-50263d37-bd59-4047-8523-86d807328ade tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:19.793753 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-50263d37-bd59-4047-8523-86d807328ade tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:19.794536 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-50263d37-bd59-4047-8523-86d807328ade tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:19.794981 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-50263d37-bd59-4047-8523-86d807328ade tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:19.833279 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-76756bfe-f02d-4969-a842-fbbe37388746 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:22:19.834571 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-76756bfe-f02d-4969-a842-fbbe37388746 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] [instance: 583e0c15-3053-429d-81bf-95a4d1f99cbf] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:22:19.842462 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-76756bfe-f02d-4969-a842-fbbe37388746 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:19.843739 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-76756bfe-f02d-4969-a842-fbbe37388746 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:19.844521 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-76756bfe-f02d-4969-a842-fbbe37388746 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:19.844964 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-76756bfe-f02d-4969-a842-fbbe37388746 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:20.003458 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-50263d37-bd59-4047-8523-86d807328ade tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bba77f6ef42547f1aa8480e27868bd4a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '22ca92a86e6c40d59071e58ce4085568', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:22:20.009417 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-50263d37-bd59-4047-8523-86d807328ade tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] 10.222.0.16 "GET /compute/v2.1/servers/f114e124-2903-4f30-b055-9f493e3e9e6c" status: 200 len: 1725 microversion: 2.1 time: 0.228623 Jul 16 21:22:20.029867 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-76756bfe-f02d-4969-a842-fbbe37388746 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'aaf2292818b749f0ae43c66bca929be9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '65644a8df5d84df3b35c894560e0a5c4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:22:20.038879 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-76756bfe-f02d-4969-a842-fbbe37388746 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] 10.222.0.16 "GET /compute/v2.1/servers/583e0c15-3053-429d-81bf-95a4d1f99cbf" status: 200 len: 1524 microversion: 2.1 time: 0.224438 Jul 16 21:22:20.042527 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1559/3072] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:22:19 2022] GET /compute/v2.1/servers/f114e124-2903-4f30-b055-9f493e3e9e6c => generated 1725 bytes in 262 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:22:20.049824 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1514/3073] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:22:19 2022] GET /compute/v2.1/servers/583e0c15-3053-429d-81bf-95a4d1f99cbf => generated 1524 bytes in 236 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:22:20.054491 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-a210ec09-9581-4324-b1c1-bf797775e8d3 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:22:20.054491 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-a210ec09-9581-4324-b1c1-bf797775e8d3 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: a4f887a4-84e8-4b14-beca-e6ad747ba616] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:22:20.059328 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a210ec09-9581-4324-b1c1-bf797775e8d3 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:20.059782 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a210ec09-9581-4324-b1c1-bf797775e8d3 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:20.060574 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a210ec09-9581-4324-b1c1-bf797775e8d3 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:20.060996 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a210ec09-9581-4324-b1c1-bf797775e8d3 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:20.078510 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-a210ec09-9581-4324-b1c1-bf797775e8d3 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] HTTP exception thrown: Instance a4f887a4-84e8-4b14-beca-e6ad747ba616 could not be found. Jul 16 21:22:20.081290 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-a210ec09-9581-4324-b1c1-bf797775e8d3 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Request method failure captured: Jul 16 21:22:20.081290 ncdv-master-1 devstack@n-api.service[138680]: request: GET /compute/v2.1/servers/a4f887a4-84e8-4b14-beca-e6ad747ba616 HTTP/1.1 Jul 16 21:22:20.081290 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:22:20.081290 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:22:20.081290 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:22:20.081290 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 0 Jul 16 21:22:20.081290 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:22:20.081290 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:22:20.081290 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:22:20.081290 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:22:20.081290 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0yvs4CI48W3ZCHoQ8huRZwHvqHxAdwrwrvk1v_ETBPx6FU4DdiPUjkjq4hN_4g_fHLAB_XZ9-AAN8Pyvy0KON2X25WlmyilO65MDHbIW9Xy9v9USVPAu_Ce-ze5X9UIHyned0AUneE4UPGiLLIkoXeoS5ekiq1pi209pud2lyUAuHgw Jul 16 21:22:20.081290 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:22:20.081290 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:22:20.081290 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:22:20.081290 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:22:20.081290 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:22:20.081290 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:22:20.081290 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: d31be8475a6a49d6bfe8132dd050c6c0 Jul 16 21:22:20.081290 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-DeleteServersTestJSON-1541892181 Jul 16 21:22:20.081290 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:22:20.081290 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:22:20.081290 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/d31be8475a6a49d6bfe8132dd050c6c0"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/d31be8475a6a49d6bfe8132dd050c6c0"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_d31be8475a6a49d6bfe8132dd050c6c0"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/d31be8475a6a49d6bfe8132dd050c6c0"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:22:20.081290 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-DeleteServersTestJSON-1541892181 Jul 16 21:22:20.081290 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: d31be8475a6a49d6bfe8132dd050c6c0 Jul 16 21:22:20.081290 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-DeleteServersTestJSON-1541892181 Jul 16 21:22:20.081290 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-DeleteServersTestJSON-1541892181-project Jul 16 21:22:20.081290 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:22:20.081290 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:22:20.081290 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: 6f2e974ab5aa48dfb4b00f810dcedb63 Jul 16 21:22:20.081290 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-DeleteServersTestJSON-1541892181-project Jul 16 21:22:20.081290 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 21:22:20.081290 ncdv-master-1 devstack@n-api.service[138680]: exception: Instance a4f887a4-84e8-4b14-beca-e6ad747ba616 could not be found. Jul 16 21:22:20.081290 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': 'a4f887a4-84e8-4b14-beca-e6ad747ba616'} Jul 16 21:22:20.081290 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:22:20.081290 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:22:20.081290 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:22:20.081290 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:22:20.081290 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:22:20.081290 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:22:20.081290 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:22:20.081290 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:22:20.081290 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:22:20.081290 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:22:20.081290 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance a4f887a4-84e8-4b14-beca-e6ad747ba616 could not be found. Jul 16 21:22:20.081290 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:22:20.081290 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:22:20.081290 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:22:20.081290 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:22:20.081290 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:22:20.081290 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:22:20.081290 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:22:20.081290 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:22:20.081290 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:22:20.081290 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:22:20.081290 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:22:20.081290 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:22:20.081290 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:22:20.081290 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:22:20.081290 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:22:20.081290 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:22:20.081290 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance a4f887a4-84e8-4b14-beca-e6ad747ba616 could not be found. Jul 16 21:22:20.081290 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:22:20.081290 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:22:20.081290 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:22:20.086926 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:22:20.086926 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:22:20.086926 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:22:20.086926 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:22:20.086926 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:22:20.086926 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance a4f887a4-84e8-4b14-beca-e6ad747ba616 could not be found. Jul 16 21:22:20.086926 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:22:20.086926 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-a210ec09-9581-4324-b1c1-bf797775e8d3 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Returning 404 to user: Instance a4f887a4-84e8-4b14-beca-e6ad747ba616 could not be found. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:22:20.086926 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-a210ec09-9581-4324-b1c1-bf797775e8d3 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "GET /compute/v2.1/servers/a4f887a4-84e8-4b14-beca-e6ad747ba616" status: 404 len: 111 microversion: 2.1 time: 0.037947 Jul 16 21:22:20.086926 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1560/3074] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:22:20 2022] GET /compute/v2.1/servers/a4f887a4-84e8-4b14-beca-e6ad747ba616 => generated 111 bytes in 39 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:22:20.099869 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-f8b6c01b-59d7-4297-8e61-11246ef72b86 req-58bce18d-35c6-412b-80d5-9170e2c7ee9b service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "8acee151-2b52-4ca3-b2e1-18a19834b0e2", "name": "network-vif-deleted", "tag": "b550a22e-98c4-4464-b5db-88766e75d282"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:22:20.108710 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-f8b6c01b-59d7-4297-8e61-11246ef72b86 req-58bce18d-35c6-412b-80d5-9170e2c7ee9b service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:20.109335 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-f8b6c01b-59d7-4297-8e61-11246ef72b86 req-58bce18d-35c6-412b-80d5-9170e2c7ee9b service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:20.121259 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-f8b6c01b-59d7-4297-8e61-11246ef72b86 req-58bce18d-35c6-412b-80d5-9170e2c7ee9b service nova] Creating event network-vif-deleted:b550a22e-98c4-4464-b5db-88766e75d282 for instance 8acee151-2b52-4ca3-b2e1-18a19834b0e2 on nch2-master-1 Jul 16 21:22:20.127860 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-f8b6c01b-59d7-4297-8e61-11246ef72b86 req-58bce18d-35c6-412b-80d5-9170e2c7ee9b service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.031482 Jul 16 21:22:20.128669 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1515/3075] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:22:20 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:22:20.231260 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-23fb1693-9df4-419c-ba70-07f17e6e9413 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-DeleteServersTestJSON-server-240007313", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452", "networks": [{"uuid": "fc3b835d-0a7f-4948-9fcb-1edfd6b9a8df"}]}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:22:20.294297 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.network.neutron [None req-23fb1693-9df4-419c-ba70-07f17e6e9413 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] validate_networks() for [('fc3b835d-0a7f-4948-9fcb-1edfd6b9a8df', None, None, None, None, None)] {{(pid=138680) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 16 21:22:20.432712 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-385ec18f-e397-4088-8e3e-95a19362dad4 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:22:20.433449 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-385ec18f-e397-4088-8e3e-95a19362dad4 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] [instance: 8acee151-2b52-4ca3-b2e1-18a19834b0e2] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:22:20.440566 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-385ec18f-e397-4088-8e3e-95a19362dad4 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:20.441018 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-385ec18f-e397-4088-8e3e-95a19362dad4 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:20.441797 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-385ec18f-e397-4088-8e3e-95a19362dad4 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:20.442235 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-385ec18f-e397-4088-8e3e-95a19362dad4 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:20.502487 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-385ec18f-e397-4088-8e3e-95a19362dad4 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:20.503075 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-385ec18f-e397-4088-8e3e-95a19362dad4 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:20.503551 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.objects.instance [None req-385ec18f-e397-4088-8e3e-95a19362dad4 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Lazy-loading 'fault' on Instance uuid 8acee151-2b52-4ca3-b2e1-18a19834b0e2 {{(pid=138681) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:22:20.889422 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-385ec18f-e397-4088-8e3e-95a19362dad4 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa4c2791e14d4f88883a0af422e4ff0c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c95a2d1e0a034c7ca4a2fcb9fd060d53', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:22:20.898191 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-385ec18f-e397-4088-8e3e-95a19362dad4 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] 10.222.0.16 "GET /compute/v2.1/servers/8acee151-2b52-4ca3-b2e1-18a19834b0e2" status: 200 len: 1313 microversion: 2.1 time: 0.470941 Jul 16 21:22:20.899348 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1516/3076] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:22:20 2022] GET /compute/v2.1/servers/8acee151-2b52-4ca3-b2e1-18a19834b0e2 => generated 1313 bytes in 473 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:22:21.011210 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-23fb1693-9df4-419c-ba70-07f17e6e9413 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Flavor limits 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 16 21:22:21.011726 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-23fb1693-9df4-419c-ba70-07f17e6e9413 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Image limits 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 16 21:22:21.012174 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-23fb1693-9df4-419c-ba70-07f17e6e9413 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Flavor pref 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 16 21:22:21.012873 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-23fb1693-9df4-419c-ba70-07f17e6e9413 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Image pref 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 16 21:22:21.013291 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-23fb1693-9df4-419c-ba70-07f17e6e9413 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 16 21:22:21.023290 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-23fb1693-9df4-419c-ba70-07f17e6e9413 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:21.023745 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-23fb1693-9df4-419c-ba70-07f17e6e9413 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:21.024549 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-23fb1693-9df4-419c-ba70-07f17e6e9413 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:21.024989 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-23fb1693-9df4-419c-ba70-07f17e6e9413 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:21.027875 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-473c513a-9682-49ce-8ee2-c17db5c00616 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:22:21.028510 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-473c513a-9682-49ce-8ee2-c17db5c00616 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] [instance: f114e124-2903-4f30-b055-9f493e3e9e6c] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:22:21.035818 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-473c513a-9682-49ce-8ee2-c17db5c00616 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:21.036445 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-473c513a-9682-49ce-8ee2-c17db5c00616 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:21.037242 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-473c513a-9682-49ce-8ee2-c17db5c00616 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:21.037692 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-473c513a-9682-49ce-8ee2-c17db5c00616 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:21.046703 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-23fb1693-9df4-419c-ba70-07f17e6e9413 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Getting quotas for project d31be8475a6a49d6bfe8132dd050c6c0. Resources: {'ram', 'instances', 'cores'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:22:21.052863 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-23fb1693-9df4-419c-ba70-07f17e6e9413 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Getting quotas for user 6f2e974ab5aa48dfb4b00f810dcedb63 and project d31be8475a6a49d6bfe8132dd050c6c0. Resources: {'ram', 'instances', 'cores'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:22:21.080761 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-23fb1693-9df4-419c-ba70-07f17e6e9413 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Going to run 1 instances... {{(pid=138680) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 16 21:22:21.083101 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-23fb1693-9df4-419c-ba70-07f17e6e9413 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 32df0911-cfa6-4410-840a-4098a9fef0b4] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138680) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 16 21:22:21.187563 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-473c513a-9682-49ce-8ee2-c17db5c00616 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bba77f6ef42547f1aa8480e27868bd4a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '22ca92a86e6c40d59071e58ce4085568', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:22:21.196271 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-473c513a-9682-49ce-8ee2-c17db5c00616 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] 10.222.0.16 "GET /compute/v2.1/servers/f114e124-2903-4f30-b055-9f493e3e9e6c" status: 200 len: 1725 microversion: 2.1 time: 0.172346 Jul 16 21:22:21.196869 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1517/3077] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:22:21 2022] GET /compute/v2.1/servers/f114e124-2903-4f30-b055-9f493e3e9e6c => generated 1725 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:22:21.206042 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-091ea3c1-7b8f-4464-9aaa-265c661aa238 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:22:21.206042 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-091ea3c1-7b8f-4464-9aaa-265c661aa238 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] [instance: 583e0c15-3053-429d-81bf-95a4d1f99cbf] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:22:21.208452 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-091ea3c1-7b8f-4464-9aaa-265c661aa238 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:21.208918 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-091ea3c1-7b8f-4464-9aaa-265c661aa238 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:21.209695 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-091ea3c1-7b8f-4464-9aaa-265c661aa238 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:21.210165 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-091ea3c1-7b8f-4464-9aaa-265c661aa238 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:21.247819 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-23fb1693-9df4-419c-ba70-07f17e6e9413 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 1.011569 Jul 16 21:22:21.247819 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1561/3078] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 21:22:20 2022] POST /compute/v2.1/servers => generated 380 bytes in 1013 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 16 21:22:21.260471 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-ebafd0f5-970a-498b-847d-058e2eb2a31c tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:22:21.261677 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-ebafd0f5-970a-498b-847d-058e2eb2a31c tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 32df0911-cfa6-4410-840a-4098a9fef0b4] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:22:21.350922 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-ebafd0f5-970a-498b-847d-058e2eb2a31c tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f2e974ab5aa48dfb4b00f810dcedb63', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd31be8475a6a49d6bfe8132dd050c6c0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:22:21.359902 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-ebafd0f5-970a-498b-847d-058e2eb2a31c tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "GET /compute/v2.1/servers/32df0911-cfa6-4410-840a-4098a9fef0b4" status: 200 len: 1214 microversion: 2.1 time: 0.103630 Jul 16 21:22:21.361199 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1562/3079] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:22:21 2022] GET /compute/v2.1/servers/32df0911-cfa6-4410-840a-4098a9fef0b4 => generated 1214 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:22:21.410932 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-091ea3c1-7b8f-4464-9aaa-265c661aa238 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'aaf2292818b749f0ae43c66bca929be9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '65644a8df5d84df3b35c894560e0a5c4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:22:21.423685 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-091ea3c1-7b8f-4464-9aaa-265c661aa238 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] 10.222.0.16 "GET /compute/v2.1/servers/583e0c15-3053-429d-81bf-95a4d1f99cbf" status: 200 len: 1524 microversion: 2.1 time: 0.221944 Jul 16 21:22:21.423685 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1518/3080] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:22:21 2022] GET /compute/v2.1/servers/583e0c15-3053-429d-81bf-95a4d1f99cbf => generated 1524 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:22:21.915632 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-3a118ab9-568b-48a3-9b83-0c31a7801bb9 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:22:21.916251 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-3a118ab9-568b-48a3-9b83-0c31a7801bb9 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] [instance: 8acee151-2b52-4ca3-b2e1-18a19834b0e2] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:22:21.923660 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3a118ab9-568b-48a3-9b83-0c31a7801bb9 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:21.924226 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3a118ab9-568b-48a3-9b83-0c31a7801bb9 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:21.925009 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3a118ab9-568b-48a3-9b83-0c31a7801bb9 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:21.925480 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3a118ab9-568b-48a3-9b83-0c31a7801bb9 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:21.940673 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-3a118ab9-568b-48a3-9b83-0c31a7801bb9 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] HTTP exception thrown: Instance 8acee151-2b52-4ca3-b2e1-18a19834b0e2 could not be found. Jul 16 21:22:21.942230 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-3a118ab9-568b-48a3-9b83-0c31a7801bb9 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Request method failure captured: Jul 16 21:22:21.942230 ncdv-master-1 devstack@n-api.service[138680]: request: GET /compute/v2.1/servers/8acee151-2b52-4ca3-b2e1-18a19834b0e2 HTTP/1.1 Jul 16 21:22:21.942230 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:22:21.942230 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:22:21.942230 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:22:21.942230 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 0 Jul 16 21:22:21.942230 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:22:21.942230 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:22:21.942230 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:22:21.942230 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:22:21.942230 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0yvfxxMh1kQhaEsgLF2SOq9-6tkycdPHJwpB0voh-EodC0AvZcoQjlWZQijLUgRQKz2_W2NQj1Lqw2r8EDFvWo3AgwmaOhVrEZAQN9n0U0N5inF9PKjVbZSkNFL0ZbZQ8MJbijHlSUH_9tZn9_wPW1QgQaVuiTwbt8cjpwg_quZbwtU Jul 16 21:22:21.942230 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:22:21.942230 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:22:21.942230 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:22:21.942230 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:22:21.942230 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:22:21.942230 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:22:21.942230 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: c95a2d1e0a034c7ca4a2fcb9fd060d53 Jul 16 21:22:21.942230 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-ListServersNegativeTestJSON-982002908 Jul 16 21:22:21.942230 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:22:21.942230 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:22:21.942230 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/c95a2d1e0a034c7ca4a2fcb9fd060d53"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/c95a2d1e0a034c7ca4a2fcb9fd060d53"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_c95a2d1e0a034c7ca4a2fcb9fd060d53"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/c95a2d1e0a034c7ca4a2fcb9fd060d53"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:22:21.942230 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-ListServersNegativeTestJSON-982002908 Jul 16 21:22:21.942230 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: c95a2d1e0a034c7ca4a2fcb9fd060d53 Jul 16 21:22:21.942230 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-ListServersNegativeTestJSON-982002908 Jul 16 21:22:21.942230 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-ListServersNegativeTestJSON-982002908-project Jul 16 21:22:21.942230 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:22:21.942230 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:22:21.942230 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: fa4c2791e14d4f88883a0af422e4ff0c Jul 16 21:22:21.942230 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-ListServersNegativeTestJSON-982002908-project Jul 16 21:22:21.942230 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 21:22:21.942230 ncdv-master-1 devstack@n-api.service[138680]: exception: Instance 8acee151-2b52-4ca3-b2e1-18a19834b0e2 could not be found. Jul 16 21:22:21.942230 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': '8acee151-2b52-4ca3-b2e1-18a19834b0e2'} Jul 16 21:22:21.942230 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:22:21.942230 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:22:21.942230 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:22:21.942230 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:22:21.942230 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:22:21.942230 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:22:21.942230 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:22:21.942230 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:22:21.942230 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:22:21.942230 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:22:21.942230 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 8acee151-2b52-4ca3-b2e1-18a19834b0e2 could not be found. Jul 16 21:22:21.942230 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:22:21.942230 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:22:21.942230 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:22:21.942230 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:22:21.942230 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:22:21.942230 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:22:21.942230 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:22:21.942230 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:22:21.942230 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:22:21.942230 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:22:21.942230 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:22:21.942230 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:22:21.942230 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:22:21.942230 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:22:21.942230 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:22:21.942230 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:22:21.942230 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 8acee151-2b52-4ca3-b2e1-18a19834b0e2 could not be found. Jul 16 21:22:21.942230 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:22:21.942230 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:22:21.942230 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:22:21.946311 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:22:21.946311 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:22:21.946311 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:22:21.946311 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:22:21.946311 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:22:21.946311 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 8acee151-2b52-4ca3-b2e1-18a19834b0e2 could not be found. Jul 16 21:22:21.946311 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:22:21.946311 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-3a118ab9-568b-48a3-9b83-0c31a7801bb9 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Returning 404 to user: Instance 8acee151-2b52-4ca3-b2e1-18a19834b0e2 could not be found. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:22:21.946311 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-3a118ab9-568b-48a3-9b83-0c31a7801bb9 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] 10.222.0.16 "GET /compute/v2.1/servers/8acee151-2b52-4ca3-b2e1-18a19834b0e2" status: 404 len: 111 microversion: 2.1 time: 0.031995 Jul 16 21:22:21.946311 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1563/3081] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:22:21 2022] GET /compute/v2.1/servers/8acee151-2b52-4ca3-b2e1-18a19834b0e2 => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:22:21.957955 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-0be1ae34-74b9-44f4-971a-16113452f82d tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:22:21.961818 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-0be1ae34-74b9-44f4-971a-16113452f82d tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Policy check for os_compute_api:servers:allow_all_filters failed with credentials {'is_admin': False, 'user_id': 'fa4c2791e14d4f88883a0af422e4ff0c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c95a2d1e0a034c7ca4a2fcb9fd060d53', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:22:21.962733 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-0be1ae34-74b9-44f4-971a-16113452f82d tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Searching by: {'changes-since': datetime.datetime(2051, 1, 1, 12, 34, tzinfo=datetime.timezone.utc), 'vm_state': ['active'], 'project_id': 'c95a2d1e0a034c7ca4a2fcb9fd060d53'} {{(pid=138681) get_all /opt/stack/nova/nova/compute/api.py:2962}} Jul 16 21:22:21.973987 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0be1ae34-74b9-44f4-971a-16113452f82d tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:21.974444 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0be1ae34-74b9-44f4-971a-16113452f82d tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:21.975262 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0be1ae34-74b9-44f4-971a-16113452f82d tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:21.975700 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0be1ae34-74b9-44f4-971a-16113452f82d tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:22.063897 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-0be1ae34-74b9-44f4-971a-16113452f82d tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] 10.222.0.16 "GET /compute/v2.1/servers?changes-since=2051-01-01T12%3A34%3A00Z&status=ACTIVE" status: 200 len: 15 microversion: 2.1 time: 0.114252 Jul 16 21:22:22.064488 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1519/3082] 10.222.0.16 () {58 vars in 1229 bytes} [Sat Jul 16 21:22:21 2022] GET /compute/v2.1/servers?changes-since=2051-01-01T12%3A34%3A00Z&status=ACTIVE => generated 15 bytes in 116 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Jul 16 21:22:22.076316 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-544f4128-447c-4b0e-ab97-26a8c13685c1 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:22:22.079579 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-544f4128-447c-4b0e-ab97-26a8c13685c1 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Request method failure captured: Jul 16 21:22:22.079579 ncdv-master-1 devstack@n-api.service[138680]: request: GET /compute/v2.1/servers?changes-since=2011%2F01%2F01 HTTP/1.1 Jul 16 21:22:22.079579 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:22:22.079579 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:22:22.079579 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:22:22.079579 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 0 Jul 16 21:22:22.079579 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:22:22.079579 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:22:22.079579 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:22:22.079579 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:22:22.079579 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0yvfxxMh1kQhaEsgLF2SOq9-6tkycdPHJwpB0voh-EodC0AvZcoQjlWZQijLUgRQKz2_W2NQj1Lqw2r8EDFvWo3AgwmaOhVrEZAQN9n0U0N5inF9PKjVbZSkNFL0ZbZQ8MJbijHlSUH_9tZn9_wPW1QgQaVuiTwbt8cjpwg_quZbwtU Jul 16 21:22:22.079579 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:22:22.079579 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:22:22.079579 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:22:22.079579 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:22:22.079579 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:22:22.079579 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:22:22.079579 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: c95a2d1e0a034c7ca4a2fcb9fd060d53 Jul 16 21:22:22.079579 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-ListServersNegativeTestJSON-982002908 Jul 16 21:22:22.079579 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:22:22.079579 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:22:22.079579 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/c95a2d1e0a034c7ca4a2fcb9fd060d53"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/c95a2d1e0a034c7ca4a2fcb9fd060d53"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_c95a2d1e0a034c7ca4a2fcb9fd060d53"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/c95a2d1e0a034c7ca4a2fcb9fd060d53"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:22:22.079579 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-ListServersNegativeTestJSON-982002908 Jul 16 21:22:22.079579 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: c95a2d1e0a034c7ca4a2fcb9fd060d53 Jul 16 21:22:22.079579 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-ListServersNegativeTestJSON-982002908 Jul 16 21:22:22.079579 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-ListServersNegativeTestJSON-982002908-project Jul 16 21:22:22.079579 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:22:22.079579 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:22:22.079579 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: fa4c2791e14d4f88883a0af422e4ff0c Jul 16 21:22:22.079579 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-ListServersNegativeTestJSON-982002908-project Jul 16 21:22:22.079579 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 21:22:22.079579 ncdv-master-1 devstack@n-api.service[138680]: exception: Invalid input for query parameters changes-since. Value: 2011/01/01. '2011/01/01' is not a 'date-time' Jul 16 21:22:22.079579 ncdv-master-1 devstack@n-api.service[138680]: action_args: {} Jul 16 21:22:22.079579 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:22:22.079579 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:22:22.079579 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 279, in validate Jul 16 21:22:22.079579 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Jul 16 21:22:22.079579 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/lib/python3/dist-packages/jsonschema/validators.py", line 353, in validate Jul 16 21:22:22.079579 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise error Jul 16 21:22:22.079579 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: '2011/01/01' is not a 'date-time' Jul 16 21:22:22.079579 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:22:22.079579 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Failed validating 'format' in schema['properties']['changes-since']['items']: Jul 16 21:22:22.079579 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi {'format': 'date-time', 'type': 'string'} Jul 16 21:22:22.079579 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:22:22.079579 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi On instance[0]: Jul 16 21:22:22.079579 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi '2011/01/01' Jul 16 21:22:22.079579 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:22:22.079579 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:22:22.079579 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:22:22.079579 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:22:22.079579 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:22:22.079579 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:22:22.079579 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:22:22.079579 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:22:22.079579 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:22:22.079579 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:22:22.079579 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper Jul 16 21:22:22.079579 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 16 21:22:22.079579 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper Jul 16 21:22:22.079579 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 16 21:22:22.079579 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper Jul 16 21:22:22.079579 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 16 21:22:22.079579 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi [Previous line repeated 1 more time] Jul 16 21:22:22.079579 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 181, in wrapper Jul 16 21:22:22.079579 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi if _schema_validation_helper(query_params_schema, Jul 16 21:22:22.079579 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Jul 16 21:22:22.086628 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Jul 16 21:22:22.086628 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 308, in validate Jul 16 21:22:22.086628 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Jul 16 21:22:22.086628 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for query parameters changes-since. Value: 2011/01/01. '2011/01/01' is not a 'date-time' Jul 16 21:22:22.086628 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:22:22.086628 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:22:22.086628 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:22:22.086628 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:22:22.086628 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:22:22.086628 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:22:22.086628 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Jul 16 21:22:22.086628 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Jul 16 21:22:22.086628 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for query parameters changes-since. Value: 2011/01/01. '2011/01/01' is not a 'date-time' Jul 16 21:22:22.086628 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:22:22.086628 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-544f4128-447c-4b0e-ab97-26a8c13685c1 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Returning 400 to user: Invalid input for query parameters changes-since. Value: 2011/01/01. '2011/01/01' is not a 'date-time' {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:22:22.086628 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-544f4128-447c-4b0e-ab97-26a8c13685c1 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] 10.222.0.16 "GET /compute/v2.1/servers?changes-since=2011%2F01%2F01" status: 400 len: 146 microversion: 2.1 time: 0.008653 Jul 16 21:22:22.086628 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1564/3083] 10.222.0.16 () {58 vars in 1181 bytes} [Sat Jul 16 21:22:22 2022] GET /compute/v2.1/servers?changes-since=2011%2F01%2F01 => generated 146 bytes in 10 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Jul 16 21:22:22.091197 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-41d8fd1f-b47c-48d9-963a-fbe6973e337a tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:22:22.095149 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-41d8fd1f-b47c-48d9-963a-fbe6973e337a tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Policy check for os_compute_api:servers:allow_all_filters failed with credentials {'is_admin': False, 'user_id': 'fa4c2791e14d4f88883a0af422e4ff0c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c95a2d1e0a034c7ca4a2fcb9fd060d53', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:22:22.095869 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-41d8fd1f-b47c-48d9-963a-fbe6973e337a tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Searching by: {'deleted': False, 'project_id': 'c95a2d1e0a034c7ca4a2fcb9fd060d53'} {{(pid=138681) get_all /opt/stack/nova/nova/compute/api.py:2962}} Jul 16 21:22:22.101828 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-41d8fd1f-b47c-48d9-963a-fbe6973e337a tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:22.102300 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-41d8fd1f-b47c-48d9-963a-fbe6973e337a tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:22.103093 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-41d8fd1f-b47c-48d9-963a-fbe6973e337a tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:22.103530 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-41d8fd1f-b47c-48d9-963a-fbe6973e337a tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:22.135302 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.multi_cell_list [None req-41d8fd1f-b47c-48d9-963a-fbe6973e337a tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Listed batch of 2 results from cell out of 1000 limit. Returned 2 total so far. {{(pid=138681) do_query /opt/stack/nova/nova/compute/multi_cell_list.py:373}} Jul 16 21:22:22.168986 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-41d8fd1f-b47c-48d9-963a-fbe6973e337a tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] 10.222.0.16 "GET /compute/v2.1/servers" status: 200 len: 683 microversion: 2.1 time: 0.082309 Jul 16 21:22:22.169897 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1520/3084] 10.222.0.16 () {58 vars in 1124 bytes} [Sat Jul 16 21:22:22 2022] GET /compute/v2.1/servers => generated 683 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:22:22.180838 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-aa536883-81a2-44eb-a958-de4b79a9b375 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:22:22.184644 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-aa536883-81a2-44eb-a958-de4b79a9b375 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Policy check for os_compute_api:servers:allow_all_filters failed with credentials {'is_admin': False, 'user_id': 'fa4c2791e14d4f88883a0af422e4ff0c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c95a2d1e0a034c7ca4a2fcb9fd060d53', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:22:22.185115 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.compute.servers [None req-aa536883-81a2-44eb-a958-de4b79a9b375 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Removing options 'limit' from query {{(pid=138680) remove_invalid_options /opt/stack/nova/nova/api/openstack/compute/servers.py:1487}} Jul 16 21:22:22.190144 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-aa536883-81a2-44eb-a958-de4b79a9b375 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Searching by: {'deleted': False, 'project_id': 'c95a2d1e0a034c7ca4a2fcb9fd060d53'} {{(pid=138680) get_all /opt/stack/nova/nova/compute/api.py:2962}} Jul 16 21:22:22.193272 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-aa536883-81a2-44eb-a958-de4b79a9b375 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:22.193731 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-aa536883-81a2-44eb-a958-de4b79a9b375 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:22.194602 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-aa536883-81a2-44eb-a958-de4b79a9b375 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:22.195102 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-aa536883-81a2-44eb-a958-de4b79a9b375 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:22.224605 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-4cb3f3a0-ee93-470f-a57d-4cfe31c5265d tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:22:22.224605 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-4cb3f3a0-ee93-470f-a57d-4cfe31c5265d tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] [instance: f114e124-2903-4f30-b055-9f493e3e9e6c] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:22:22.224605 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4cb3f3a0-ee93-470f-a57d-4cfe31c5265d tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:22.224605 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4cb3f3a0-ee93-470f-a57d-4cfe31c5265d tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:22.224605 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4cb3f3a0-ee93-470f-a57d-4cfe31c5265d tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:22.224605 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4cb3f3a0-ee93-470f-a57d-4cfe31c5265d tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:22.226094 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.multi_cell_list [None req-aa536883-81a2-44eb-a958-de4b79a9b375 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Listed batch of 2 results from cell out of 102 limit. Returned 2 total so far. {{(pid=138680) do_query /opt/stack/nova/nova/compute/multi_cell_list.py:373}} Jul 16 21:22:22.259083 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-aa536883-81a2-44eb-a958-de4b79a9b375 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] 10.222.0.16 "GET /compute/v2.1/servers?limit=102" status: 200 len: 683 microversion: 2.1 time: 0.081082 Jul 16 21:22:22.259083 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1565/3085] 10.222.0.16 () {58 vars in 1143 bytes} [Sat Jul 16 21:22:22 2022] GET /compute/v2.1/servers?limit=102 => generated 683 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:22:22.271750 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-c1d8c13d-3d6f-4824-a6b3-2b4205a710a6 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:22:22.274590 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-c1d8c13d-3d6f-4824-a6b3-2b4205a710a6 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Request method failure captured: Jul 16 21:22:22.274590 ncdv-master-1 devstack@n-api.service[138680]: request: GET /compute/v2.1/servers?limit=-1 HTTP/1.1 Jul 16 21:22:22.274590 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:22:22.274590 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:22:22.274590 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:22:22.274590 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 0 Jul 16 21:22:22.274590 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:22:22.274590 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:22:22.274590 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:22:22.274590 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:22:22.274590 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0yvfxxMh1kQhaEsgLF2SOq9-6tkycdPHJwpB0voh-EodC0AvZcoQjlWZQijLUgRQKz2_W2NQj1Lqw2r8EDFvWo3AgwmaOhVrEZAQN9n0U0N5inF9PKjVbZSkNFL0ZbZQ8MJbijHlSUH_9tZn9_wPW1QgQaVuiTwbt8cjpwg_quZbwtU Jul 16 21:22:22.274590 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:22:22.274590 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:22:22.274590 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:22:22.274590 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:22:22.274590 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:22:22.274590 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:22:22.274590 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: c95a2d1e0a034c7ca4a2fcb9fd060d53 Jul 16 21:22:22.274590 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-ListServersNegativeTestJSON-982002908 Jul 16 21:22:22.274590 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:22:22.274590 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:22:22.274590 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/c95a2d1e0a034c7ca4a2fcb9fd060d53"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/c95a2d1e0a034c7ca4a2fcb9fd060d53"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_c95a2d1e0a034c7ca4a2fcb9fd060d53"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/c95a2d1e0a034c7ca4a2fcb9fd060d53"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:22:22.274590 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-ListServersNegativeTestJSON-982002908 Jul 16 21:22:22.274590 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: c95a2d1e0a034c7ca4a2fcb9fd060d53 Jul 16 21:22:22.274590 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-ListServersNegativeTestJSON-982002908 Jul 16 21:22:22.274590 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-ListServersNegativeTestJSON-982002908-project Jul 16 21:22:22.274590 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:22:22.274590 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:22:22.274590 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: fa4c2791e14d4f88883a0af422e4ff0c Jul 16 21:22:22.274590 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-ListServersNegativeTestJSON-982002908-project Jul 16 21:22:22.274590 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 21:22:22.274590 ncdv-master-1 devstack@n-api.service[138680]: exception: Invalid input for query parameters limit. Value: -1. '-1' does not match '^[0-9]*$' Jul 16 21:22:22.274590 ncdv-master-1 devstack@n-api.service[138680]: action_args: {} Jul 16 21:22:22.274590 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:22:22.274590 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:22:22.274590 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 279, in validate Jul 16 21:22:22.274590 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Jul 16 21:22:22.274590 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/lib/python3/dist-packages/jsonschema/validators.py", line 353, in validate Jul 16 21:22:22.274590 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise error Jul 16 21:22:22.274590 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: '-1' does not match '^[0-9]*$' Jul 16 21:22:22.274590 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:22:22.274590 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Failed validating 'pattern' in schema['properties']['limit']['items']: Jul 16 21:22:22.274590 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi {'minLength': 1, Jul 16 21:22:22.274590 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi 'minimum': 0, Jul 16 21:22:22.274590 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi 'pattern': '^[0-9]*$', Jul 16 21:22:22.274590 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi 'type': ['integer', 'string']} Jul 16 21:22:22.274590 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:22:22.274590 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi On instance[0]: Jul 16 21:22:22.274590 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi '-1' Jul 16 21:22:22.274590 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:22:22.274590 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:22:22.274590 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:22:22.274590 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:22:22.274590 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:22:22.274590 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:22:22.274590 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:22:22.274590 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:22:22.274590 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:22:22.274590 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:22:22.274590 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper Jul 16 21:22:22.274590 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 16 21:22:22.274590 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper Jul 16 21:22:22.274590 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 16 21:22:22.274590 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper Jul 16 21:22:22.274590 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 16 21:22:22.274590 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi [Previous line repeated 1 more time] Jul 16 21:22:22.274590 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 181, in wrapper Jul 16 21:22:22.274590 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi if _schema_validation_helper(query_params_schema, Jul 16 21:22:22.289817 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Jul 16 21:22:22.289817 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Jul 16 21:22:22.289817 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 308, in validate Jul 16 21:22:22.289817 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Jul 16 21:22:22.289817 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for query parameters limit. Value: -1. '-1' does not match '^[0-9]*$' Jul 16 21:22:22.289817 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:22:22.289817 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:22:22.289817 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:22:22.289817 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:22:22.289817 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:22:22.289817 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:22:22.289817 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Jul 16 21:22:22.289817 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Jul 16 21:22:22.289817 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for query parameters limit. Value: -1. '-1' does not match '^[0-9]*$' Jul 16 21:22:22.289817 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:22:22.289817 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-c1d8c13d-3d6f-4824-a6b3-2b4205a710a6 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Returning 400 to user: Invalid input for query parameters limit. Value: -1. '-1' does not match '^[0-9]*$' {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:22:22.289817 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-c1d8c13d-3d6f-4824-a6b3-2b4205a710a6 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] 10.222.0.16 "GET /compute/v2.1/servers?limit=-1" status: 400 len: 127 microversion: 2.1 time: 0.009428 Jul 16 21:22:22.289817 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1566/3086] 10.222.0.16 () {58 vars in 1141 bytes} [Sat Jul 16 21:22:22 2022] GET /compute/v2.1/servers?limit=-1 => generated 127 bytes in 11 msecs (HTTP/1.1 400) 9 headers in 381 bytes (2 switches on core 0) Jul 16 21:22:22.289817 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-92131292-78f8-40d6-affe-696397ba7bbb tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:22:22.291382 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-92131292-78f8-40d6-affe-696397ba7bbb tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Request method failure captured: Jul 16 21:22:22.291382 ncdv-master-1 devstack@n-api.service[138680]: request: GET /compute/v2.1/servers?limit=testing HTTP/1.1 Jul 16 21:22:22.291382 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:22:22.291382 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:22:22.291382 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:22:22.291382 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 0 Jul 16 21:22:22.291382 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:22:22.291382 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:22:22.291382 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:22:22.291382 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:22:22.291382 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0yvfxxMh1kQhaEsgLF2SOq9-6tkycdPHJwpB0voh-EodC0AvZcoQjlWZQijLUgRQKz2_W2NQj1Lqw2r8EDFvWo3AgwmaOhVrEZAQN9n0U0N5inF9PKjVbZSkNFL0ZbZQ8MJbijHlSUH_9tZn9_wPW1QgQaVuiTwbt8cjpwg_quZbwtU Jul 16 21:22:22.291382 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:22:22.291382 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:22:22.291382 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:22:22.291382 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:22:22.291382 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:22:22.291382 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:22:22.291382 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: c95a2d1e0a034c7ca4a2fcb9fd060d53 Jul 16 21:22:22.291382 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-ListServersNegativeTestJSON-982002908 Jul 16 21:22:22.291382 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:22:22.291382 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:22:22.291382 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/c95a2d1e0a034c7ca4a2fcb9fd060d53"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/c95a2d1e0a034c7ca4a2fcb9fd060d53"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_c95a2d1e0a034c7ca4a2fcb9fd060d53"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/c95a2d1e0a034c7ca4a2fcb9fd060d53"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:22:22.291382 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-ListServersNegativeTestJSON-982002908 Jul 16 21:22:22.291382 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: c95a2d1e0a034c7ca4a2fcb9fd060d53 Jul 16 21:22:22.291382 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-ListServersNegativeTestJSON-982002908 Jul 16 21:22:22.291382 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-ListServersNegativeTestJSON-982002908-project Jul 16 21:22:22.291382 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:22:22.291382 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:22:22.291382 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: fa4c2791e14d4f88883a0af422e4ff0c Jul 16 21:22:22.291382 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-ListServersNegativeTestJSON-982002908-project Jul 16 21:22:22.291382 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 21:22:22.291382 ncdv-master-1 devstack@n-api.service[138680]: exception: Invalid input for query parameters limit. Value: testing. 'testing' does not match '^[0-9]*$' Jul 16 21:22:22.291382 ncdv-master-1 devstack@n-api.service[138680]: action_args: {} Jul 16 21:22:22.291382 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:22:22.291382 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:22:22.291382 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 279, in validate Jul 16 21:22:22.291382 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Jul 16 21:22:22.291382 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/lib/python3/dist-packages/jsonschema/validators.py", line 353, in validate Jul 16 21:22:22.291382 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise error Jul 16 21:22:22.291382 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: 'testing' does not match '^[0-9]*$' Jul 16 21:22:22.291382 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:22:22.291382 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Failed validating 'pattern' in schema['properties']['limit']['items']: Jul 16 21:22:22.291382 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi {'minLength': 1, Jul 16 21:22:22.291382 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi 'minimum': 0, Jul 16 21:22:22.291382 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi 'pattern': '^[0-9]*$', Jul 16 21:22:22.291382 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi 'type': ['integer', 'string']} Jul 16 21:22:22.291382 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:22:22.291382 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi On instance[0]: Jul 16 21:22:22.291382 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi 'testing' Jul 16 21:22:22.291382 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:22:22.291382 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:22:22.291382 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:22:22.291382 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:22:22.291382 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:22:22.291382 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:22:22.291382 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:22:22.291382 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:22:22.291382 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:22:22.291382 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:22:22.291382 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper Jul 16 21:22:22.291382 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 16 21:22:22.291382 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper Jul 16 21:22:22.291382 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 16 21:22:22.291382 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper Jul 16 21:22:22.291382 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 16 21:22:22.291382 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi [Previous line repeated 1 more time] Jul 16 21:22:22.291382 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 181, in wrapper Jul 16 21:22:22.291382 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi if _schema_validation_helper(query_params_schema, Jul 16 21:22:22.300717 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Jul 16 21:22:22.300717 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Jul 16 21:22:22.300717 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 308, in validate Jul 16 21:22:22.300717 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Jul 16 21:22:22.300717 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for query parameters limit. Value: testing. 'testing' does not match '^[0-9]*$' Jul 16 21:22:22.300717 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:22:22.300717 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:22:22.300717 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:22:22.300717 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:22:22.300717 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:22:22.300717 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:22:22.300717 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Jul 16 21:22:22.300717 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Jul 16 21:22:22.300717 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for query parameters limit. Value: testing. 'testing' does not match '^[0-9]*$' Jul 16 21:22:22.300717 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:22:22.300717 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-92131292-78f8-40d6-affe-696397ba7bbb tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Returning 400 to user: Invalid input for query parameters limit. Value: testing. 'testing' does not match '^[0-9]*$' {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:22:22.300717 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-92131292-78f8-40d6-affe-696397ba7bbb tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] 10.222.0.16 "GET /compute/v2.1/servers?limit=testing" status: 400 len: 137 microversion: 2.1 time: 0.008847 Jul 16 21:22:22.300717 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1567/3087] 10.222.0.16 () {58 vars in 1151 bytes} [Sat Jul 16 21:22:22 2022] GET /compute/v2.1/servers?limit=testing => generated 137 bytes in 10 msecs (HTTP/1.1 400) 9 headers in 381 bytes (2 switches on core 0) Jul 16 21:22:22.304758 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-1e562dc5-420f-4256-8a68-ff2328b3697a tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:22:22.308725 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-1e562dc5-420f-4256-8a68-ff2328b3697a tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Policy check for os_compute_api:servers:allow_all_filters failed with credentials {'is_admin': False, 'user_id': 'fa4c2791e14d4f88883a0af422e4ff0c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c95a2d1e0a034c7ca4a2fcb9fd060d53', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:22:22.309398 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-1e562dc5-420f-4256-8a68-ff2328b3697a tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Searching by: {'flavor': 'non_existing_flavor', 'deleted': False, 'project_id': 'c95a2d1e0a034c7ca4a2fcb9fd060d53'} {{(pid=138680) get_all /opt/stack/nova/nova/compute/api.py:2962}} Jul 16 21:22:22.317896 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.compute.servers [None req-1e562dc5-420f-4256-8a68-ff2328b3697a tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Flavor 'non_existing_flavor' could not be found {{(pid=138680) _get_servers /opt/stack/nova/nova/api/openstack/compute/servers.py:336}} Jul 16 21:22:22.318988 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-1e562dc5-420f-4256-8a68-ff2328b3697a tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] 10.222.0.16 "GET /compute/v2.1/servers?flavor=non_existing_flavor" status: 200 len: 15 microversion: 2.1 time: 0.020443 Jul 16 21:22:22.319652 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1568/3088] 10.222.0.16 () {58 vars in 1177 bytes} [Sat Jul 16 21:22:22 2022] GET /compute/v2.1/servers?flavor=non_existing_flavor => generated 15 bytes in 22 msecs (HTTP/1.1 200) 9 headers in 356 bytes (2 switches on core 0) Jul 16 21:22:22.331387 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-46b882e2-329b-4b3a-b2e4-87b652e658db tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:22:22.334721 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-46b882e2-329b-4b3a-b2e4-87b652e658db tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Policy check for os_compute_api:servers:allow_all_filters failed with credentials {'is_admin': False, 'user_id': 'fa4c2791e14d4f88883a0af422e4ff0c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c95a2d1e0a034c7ca4a2fcb9fd060d53', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:22:22.335425 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-46b882e2-329b-4b3a-b2e4-87b652e658db tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Searching by: {'image': 'non_existing_image', 'deleted': False, 'project_id': 'c95a2d1e0a034c7ca4a2fcb9fd060d53'} {{(pid=138680) get_all /opt/stack/nova/nova/compute/api.py:2962}} Jul 16 21:22:22.341019 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-46b882e2-329b-4b3a-b2e4-87b652e658db tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:22.341471 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-46b882e2-329b-4b3a-b2e4-87b652e658db tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:22.342256 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-46b882e2-329b-4b3a-b2e4-87b652e658db tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:22.342787 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-46b882e2-329b-4b3a-b2e4-87b652e658db tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:22.393717 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-4cb3f3a0-ee93-470f-a57d-4cfe31c5265d tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bba77f6ef42547f1aa8480e27868bd4a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '22ca92a86e6c40d59071e58ce4085568', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:22:22.403523 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-4cb3f3a0-ee93-470f-a57d-4cfe31c5265d tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] 10.222.0.16 "GET /compute/v2.1/servers/f114e124-2903-4f30-b055-9f493e3e9e6c" status: 200 len: 1725 microversion: 2.1 time: 0.193352 Jul 16 21:22:22.406201 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1521/3089] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:22:22 2022] GET /compute/v2.1/servers/f114e124-2903-4f30-b055-9f493e3e9e6c => generated 1725 bytes in 195 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:22:22.419556 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-d2c85f77-5e28-4992-bedf-1f5679919572 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:22:22.420166 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-d2c85f77-5e28-4992-bedf-1f5679919572 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 32df0911-cfa6-4410-840a-4098a9fef0b4] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:22:22.428040 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d2c85f77-5e28-4992-bedf-1f5679919572 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:22.428499 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d2c85f77-5e28-4992-bedf-1f5679919572 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:22.429278 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d2c85f77-5e28-4992-bedf-1f5679919572 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:22.429714 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d2c85f77-5e28-4992-bedf-1f5679919572 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:22.433320 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-46b882e2-329b-4b3a-b2e4-87b652e658db tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] 10.222.0.16 "GET /compute/v2.1/servers?image=non_existing_image" status: 200 len: 15 microversion: 2.1 time: 0.106246 Jul 16 21:22:22.434564 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1569/3090] 10.222.0.16 () {58 vars in 1173 bytes} [Sat Jul 16 21:22:22 2022] GET /compute/v2.1/servers?image=non_existing_image => generated 15 bytes in 108 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Jul 16 21:22:22.440381 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-317989a6-612b-47c3-988d-b1f355bb3ea2 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:22:22.441153 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-317989a6-612b-47c3-988d-b1f355bb3ea2 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] [instance: 583e0c15-3053-429d-81bf-95a4d1f99cbf] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:22:22.448178 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-317989a6-612b-47c3-988d-b1f355bb3ea2 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:22.451174 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-317989a6-612b-47c3-988d-b1f355bb3ea2 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:22.451174 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-317989a6-612b-47c3-988d-b1f355bb3ea2 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:22.451174 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-317989a6-612b-47c3-988d-b1f355bb3ea2 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:22.509287 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-d2c85f77-5e28-4992-bedf-1f5679919572 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f2e974ab5aa48dfb4b00f810dcedb63', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd31be8475a6a49d6bfe8132dd050c6c0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:22:22.517668 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-d2c85f77-5e28-4992-bedf-1f5679919572 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "GET /compute/v2.1/servers/32df0911-cfa6-4410-840a-4098a9fef0b4" status: 200 len: 1218 microversion: 2.1 time: 0.110738 Jul 16 21:22:22.518428 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1522/3091] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:22:22 2022] GET /compute/v2.1/servers/32df0911-cfa6-4410-840a-4098a9fef0b4 => generated 1218 bytes in 114 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:22:22.523352 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-75b3806f-fc4c-4702-99d8-bf0e8e104d0f tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:22:22.527213 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-75b3806f-fc4c-4702-99d8-bf0e8e104d0f tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Policy check for os_compute_api:servers:allow_all_filters failed with credentials {'is_admin': False, 'user_id': 'fa4c2791e14d4f88883a0af422e4ff0c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c95a2d1e0a034c7ca4a2fcb9fd060d53', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:22:22.528072 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-75b3806f-fc4c-4702-99d8-bf0e8e104d0f tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Searching by: {'name': 'non_existing_server_name', 'deleted': False, 'project_id': 'c95a2d1e0a034c7ca4a2fcb9fd060d53'} {{(pid=138681) get_all /opt/stack/nova/nova/compute/api.py:2962}} Jul 16 21:22:22.540068 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-75b3806f-fc4c-4702-99d8-bf0e8e104d0f tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:22.540525 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-75b3806f-fc4c-4702-99d8-bf0e8e104d0f tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:22.542991 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-75b3806f-fc4c-4702-99d8-bf0e8e104d0f tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:22.542991 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-75b3806f-fc4c-4702-99d8-bf0e8e104d0f tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:22.632579 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-317989a6-612b-47c3-988d-b1f355bb3ea2 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'aaf2292818b749f0ae43c66bca929be9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '65644a8df5d84df3b35c894560e0a5c4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:22:22.640667 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-317989a6-612b-47c3-988d-b1f355bb3ea2 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] 10.222.0.16 "GET /compute/v2.1/servers/583e0c15-3053-429d-81bf-95a4d1f99cbf" status: 200 len: 1524 microversion: 2.1 time: 0.204845 Jul 16 21:22:22.641254 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1570/3092] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:22:22 2022] GET /compute/v2.1/servers/583e0c15-3053-429d-81bf-95a4d1f99cbf => generated 1524 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:22:22.650763 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-75b3806f-fc4c-4702-99d8-bf0e8e104d0f tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] 10.222.0.16 "GET /compute/v2.1/servers?name=non_existing_server_name" status: 200 len: 15 microversion: 2.1 time: 0.130889 Jul 16 21:22:22.651388 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1523/3093] 10.222.0.16 () {58 vars in 1183 bytes} [Sat Jul 16 21:22:22 2022] GET /compute/v2.1/servers?name=non_existing_server_name => generated 15 bytes in 132 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Jul 16 21:22:22.662825 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-a8bf1683-1843-43c4-a3bf-bdec2bff3da6 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:22:22.666180 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-a8bf1683-1843-43c4-a3bf-bdec2bff3da6 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Policy check for os_compute_api:servers:allow_all_filters failed with credentials {'is_admin': False, 'user_id': 'fa4c2791e14d4f88883a0af422e4ff0c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c95a2d1e0a034c7ca4a2fcb9fd060d53', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:22:22.666938 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-a8bf1683-1843-43c4-a3bf-bdec2bff3da6 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Searching by: {'deleted': False, 'project_id': 'c95a2d1e0a034c7ca4a2fcb9fd060d53'} {{(pid=138681) get_all /opt/stack/nova/nova/compute/api.py:2962}} Jul 16 21:22:22.673229 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a8bf1683-1843-43c4-a3bf-bdec2bff3da6 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:22.673939 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a8bf1683-1843-43c4-a3bf-bdec2bff3da6 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:22.674899 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a8bf1683-1843-43c4-a3bf-bdec2bff3da6 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:22.675422 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a8bf1683-1843-43c4-a3bf-bdec2bff3da6 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:22.705166 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.multi_cell_list [None req-a8bf1683-1843-43c4-a3bf-bdec2bff3da6 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Listed batch of 2 results from cell out of 1000 limit. Returned 2 total so far. {{(pid=138681) do_query /opt/stack/nova/nova/compute/multi_cell_list.py:373}} Jul 16 21:22:22.748514 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-a8bf1683-1843-43c4-a3bf-bdec2bff3da6 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa4c2791e14d4f88883a0af422e4ff0c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c95a2d1e0a034c7ca4a2fcb9fd060d53', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:22:22.756103 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a8bf1683-1843-43c4-a3bf-bdec2bff3da6 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:22.756566 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a8bf1683-1843-43c4-a3bf-bdec2bff3da6 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:22.865093 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-d56e7337-6cc4-421e-a0d6-42bb5876995c tempest-PortsAdminExtendedAttrsTestJSON-1373442467 tempest-PortsAdminExtendedAttrsTestJSON-1373442467-project-admin] Calling method '.version_select at 0x7fdd9c564ee0>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:22:22.866412 ncdv-master-1 devstack@n-api.service[138680]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-hypervisors:list": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:22:22.866412 ncdv-master-1 devstack@n-api.service[138680]: warnings.warn(msg) Jul 16 21:22:22.868489 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d56e7337-6cc4-421e-a0d6-42bb5876995c tempest-PortsAdminExtendedAttrsTestJSON-1373442467 tempest-PortsAdminExtendedAttrsTestJSON-1373442467-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:22.870049 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d56e7337-6cc4-421e-a0d6-42bb5876995c tempest-PortsAdminExtendedAttrsTestJSON-1373442467 tempest-PortsAdminExtendedAttrsTestJSON-1373442467-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:22.897894 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d56e7337-6cc4-421e-a0d6-42bb5876995c tempest-PortsAdminExtendedAttrsTestJSON-1373442467 tempest-PortsAdminExtendedAttrsTestJSON-1373442467-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:22.898756 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d56e7337-6cc4-421e-a0d6-42bb5876995c tempest-PortsAdminExtendedAttrsTestJSON-1373442467 tempest-PortsAdminExtendedAttrsTestJSON-1373442467-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:22.912167 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d56e7337-6cc4-421e-a0d6-42bb5876995c tempest-PortsAdminExtendedAttrsTestJSON-1373442467 tempest-PortsAdminExtendedAttrsTestJSON-1373442467-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:22.912643 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d56e7337-6cc4-421e-a0d6-42bb5876995c tempest-PortsAdminExtendedAttrsTestJSON-1373442467 tempest-PortsAdminExtendedAttrsTestJSON-1373442467-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:22.913184 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-a8bf1683-1843-43c4-a3bf-bdec2bff3da6 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] 10.222.0.16 "GET /compute/v2.1/servers/detail" status: 200 len: 3024 microversion: 2.1 time: 0.253923 Jul 16 21:22:22.914059 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1524/3094] 10.222.0.16 () {58 vars in 1145 bytes} [Sat Jul 16 21:22:22 2022] GET /compute/v2.1/servers/detail => generated 3024 bytes in 255 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:22:22.914848 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:22:22.921073 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-d56e7337-6cc4-421e-a0d6-42bb5876995c tempest-PortsAdminExtendedAttrsTestJSON-1373442467 tempest-PortsAdminExtendedAttrsTestJSON-1373442467-project-admin] 10.222.0.16 "GET /compute/v2.1/os-hypervisors" status: 200 len: 191 microversion: 2.1 time: 0.278641 Jul 16 21:22:22.923210 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1571/3095] 10.222.0.16 () {58 vars in 1145 bytes} [Sat Jul 16 21:22:22 2022] GET /compute/v2.1/os-hypervisors => generated 191 bytes in 280 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:22:22.932387 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-a8861da3-5b76-417c-a55e-bd5d0de8643e tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:22:22.936388 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-a8861da3-5b76-417c-a55e-bd5d0de8643e tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Policy check for os_compute_api:servers:allow_all_filters failed with credentials {'is_admin': False, 'user_id': 'fa4c2791e14d4f88883a0af422e4ff0c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c95a2d1e0a034c7ca4a2fcb9fd060d53', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:22:22.937422 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-a8861da3-5b76-417c-a55e-bd5d0de8643e tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] 10.222.0.16 "GET /compute/v2.1/servers?status=non_existing_status" status: 200 len: 15 microversion: 2.1 time: 0.008854 Jul 16 21:22:22.938190 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1525/3096] 10.222.0.16 () {58 vars in 1177 bytes} [Sat Jul 16 21:22:22 2022] GET /compute/v2.1/servers?status=non_existing_status => generated 15 bytes in 10 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Jul 16 21:22:22.949017 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-dd1bc42f-b1ec-44f1-9946-db6c8036afb7 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:22:22.953196 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-dd1bc42f-b1ec-44f1-9946-db6c8036afb7 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Policy check for os_compute_api:servers:allow_all_filters failed with credentials {'is_admin': False, 'user_id': 'fa4c2791e14d4f88883a0af422e4ff0c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c95a2d1e0a034c7ca4a2fcb9fd060d53', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:22:22.953932 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-dd1bc42f-b1ec-44f1-9946-db6c8036afb7 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Searching by: {'deleted': False, 'project_id': 'c95a2d1e0a034c7ca4a2fcb9fd060d53'} {{(pid=138680) get_all /opt/stack/nova/nova/compute/api.py:2962}} Jul 16 21:22:22.959298 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-dd1bc42f-b1ec-44f1-9946-db6c8036afb7 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:22.959746 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-dd1bc42f-b1ec-44f1-9946-db6c8036afb7 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:22.960540 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-dd1bc42f-b1ec-44f1-9946-db6c8036afb7 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:22.960976 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-dd1bc42f-b1ec-44f1-9946-db6c8036afb7 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:22.986688 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.multi_cell_list [None req-dd1bc42f-b1ec-44f1-9946-db6c8036afb7 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Listed batch of 2 results from cell out of 1000 limit. Returned 2 total so far. {{(pid=138680) do_query /opt/stack/nova/nova/compute/multi_cell_list.py:373}} Jul 16 21:22:23.014210 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-dd1bc42f-b1ec-44f1-9946-db6c8036afb7 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] 10.222.0.16 "GET /compute/v2.1/servers" status: 200 len: 683 microversion: 2.1 time: 0.069069 Jul 16 21:22:23.015560 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1572/3097] 10.222.0.16 () {58 vars in 1124 bytes} [Sat Jul 16 21:22:22 2022] GET /compute/v2.1/servers => generated 683 bytes in 71 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:22:23.027698 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-cb179f9d-1438-486c-be97-22337412e1a1 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:22:23.028239 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-cb179f9d-1438-486c-be97-22337412e1a1 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] [instance: 8acee151-2b52-4ca3-b2e1-18a19834b0e2] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:22:23.037353 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cb179f9d-1438-486c-be97-22337412e1a1 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:23.038280 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cb179f9d-1438-486c-be97-22337412e1a1 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:23.039258 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cb179f9d-1438-486c-be97-22337412e1a1 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:23.039886 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cb179f9d-1438-486c-be97-22337412e1a1 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:23.055753 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-cb179f9d-1438-486c-be97-22337412e1a1 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] HTTP exception thrown: Instance 8acee151-2b52-4ca3-b2e1-18a19834b0e2 could not be found. Jul 16 21:22:23.057651 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-cb179f9d-1438-486c-be97-22337412e1a1 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Request method failure captured: Jul 16 21:22:23.057651 ncdv-master-1 devstack@n-api.service[138681]: request: DELETE /compute/v2.1/servers/8acee151-2b52-4ca3-b2e1-18a19834b0e2 HTTP/1.1 Jul 16 21:22:23.057651 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:22:23.057651 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:22:23.057651 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:22:23.057651 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 21:22:23.057651 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:22:23.057651 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:22:23.057651 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:22:23.057651 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:22:23.057651 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0yvfxxMh1kQhaEsgLF2SOq9-6tkycdPHJwpB0voh-EodC0AvZcoQjlWZQijLUgRQKz2_W2NQj1Lqw2r8EDFvWo3AgwmaOhVrEZAQN9n0U0N5inF9PKjVbZSkNFL0ZbZQ8MJbijHlSUH_9tZn9_wPW1QgQaVuiTwbt8cjpwg_quZbwtU Jul 16 21:22:23.057651 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:22:23.057651 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:22:23.057651 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:22:23.057651 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:22:23.057651 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:22:23.057651 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:22:23.057651 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: c95a2d1e0a034c7ca4a2fcb9fd060d53 Jul 16 21:22:23.057651 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-ListServersNegativeTestJSON-982002908 Jul 16 21:22:23.057651 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:22:23.057651 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:22:23.057651 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/c95a2d1e0a034c7ca4a2fcb9fd060d53"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/c95a2d1e0a034c7ca4a2fcb9fd060d53"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_c95a2d1e0a034c7ca4a2fcb9fd060d53"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/c95a2d1e0a034c7ca4a2fcb9fd060d53"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:22:23.057651 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-ListServersNegativeTestJSON-982002908 Jul 16 21:22:23.057651 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: c95a2d1e0a034c7ca4a2fcb9fd060d53 Jul 16 21:22:23.057651 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-ListServersNegativeTestJSON-982002908 Jul 16 21:22:23.057651 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-ListServersNegativeTestJSON-982002908-project Jul 16 21:22:23.057651 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:22:23.057651 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:22:23.057651 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: fa4c2791e14d4f88883a0af422e4ff0c Jul 16 21:22:23.057651 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-ListServersNegativeTestJSON-982002908-project Jul 16 21:22:23.057651 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:22:23.057651 ncdv-master-1 devstack@n-api.service[138681]: exception: Instance 8acee151-2b52-4ca3-b2e1-18a19834b0e2 could not be found. Jul 16 21:22:23.057651 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': '8acee151-2b52-4ca3-b2e1-18a19834b0e2'} Jul 16 21:22:23.057651 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:22:23.057651 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:22:23.057651 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:22:23.057651 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:22:23.057651 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:22:23.057651 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:22:23.057651 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:22:23.057651 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:22:23.057651 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:22:23.057651 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:22:23.057651 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 8acee151-2b52-4ca3-b2e1-18a19834b0e2 could not be found. Jul 16 21:22:23.057651 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:22:23.057651 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:22:23.057651 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:22:23.057651 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:22:23.057651 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:22:23.057651 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:22:23.057651 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:22:23.057651 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:22:23.057651 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:22:23.057651 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:22:23.057651 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Jul 16 21:22:23.057651 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Jul 16 21:22:23.057651 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Jul 16 21:22:23.057651 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Jul 16 21:22:23.057651 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:22:23.057651 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:22:23.057651 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:22:23.057651 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:22:23.062104 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 8acee151-2b52-4ca3-b2e1-18a19834b0e2 could not be found. Jul 16 21:22:23.062104 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:22:23.062104 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:22:23.062104 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:22:23.062104 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:22:23.062104 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:22:23.062104 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:22:23.062104 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:22:23.062104 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:22:23.062104 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 8acee151-2b52-4ca3-b2e1-18a19834b0e2 could not be found. Jul 16 21:22:23.062104 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:22:23.063867 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-cb179f9d-1438-486c-be97-22337412e1a1 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Returning 404 to user: Instance 8acee151-2b52-4ca3-b2e1-18a19834b0e2 could not be found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:22:23.064921 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-cb179f9d-1438-486c-be97-22337412e1a1 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] 10.222.0.16 "DELETE /compute/v2.1/servers/8acee151-2b52-4ca3-b2e1-18a19834b0e2" status: 404 len: 111 microversion: 2.1 time: 0.042169 Jul 16 21:22:23.065679 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1526/3098] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:22:23 2022] DELETE /compute/v2.1/servers/8acee151-2b52-4ca3-b2e1-18a19834b0e2 => generated 111 bytes in 43 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:22:23.075452 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-955a198c-911c-451e-8bbf-513465acb6e1 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:22:23.076499 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-955a198c-911c-451e-8bbf-513465acb6e1 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] [instance: 9e7787d1-264a-4707-8604-3e8b70133376] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:22:23.087559 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-955a198c-911c-451e-8bbf-513465acb6e1 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:23.089126 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-955a198c-911c-451e-8bbf-513465acb6e1 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:23.089921 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-955a198c-911c-451e-8bbf-513465acb6e1 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:23.090372 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-955a198c-911c-451e-8bbf-513465acb6e1 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:23.118242 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-955a198c-911c-451e-8bbf-513465acb6e1 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] [instance: 9e7787d1-264a-4707-8604-3e8b70133376] Going to try to terminate instance {{(pid=138680) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 16 21:22:23.422649 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-bfa69a60-6450-408f-94f8-3d6106a02ea1 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:22:23.424253 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-bfa69a60-6450-408f-94f8-3d6106a02ea1 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] [instance: f114e124-2903-4f30-b055-9f493e3e9e6c] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:22:23.431649 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-bfa69a60-6450-408f-94f8-3d6106a02ea1 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:23.432550 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-bfa69a60-6450-408f-94f8-3d6106a02ea1 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:23.433831 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-bfa69a60-6450-408f-94f8-3d6106a02ea1 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:23.434532 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-bfa69a60-6450-408f-94f8-3d6106a02ea1 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:23.525248 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-955a198c-911c-451e-8bbf-513465acb6e1 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] 10.222.0.16 "DELETE /compute/v2.1/servers/9e7787d1-264a-4707-8604-3e8b70133376" status: 204 len: 0 microversion: 2.1 time: 0.454795 Jul 16 21:22:23.528139 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1573/3099] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:22:23 2022] DELETE /compute/v2.1/servers/9e7787d1-264a-4707-8604-3e8b70133376 => generated 0 bytes in 459 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Jul 16 21:22:23.538751 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-81a1a2d3-c387-420f-8c8e-9b7adbf91163 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:22:23.539799 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-81a1a2d3-c387-420f-8c8e-9b7adbf91163 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 32df0911-cfa6-4410-840a-4098a9fef0b4] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:22:23.559758 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-81a1a2d3-c387-420f-8c8e-9b7adbf91163 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:23.562586 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-81a1a2d3-c387-420f-8c8e-9b7adbf91163 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:23.565916 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-81a1a2d3-c387-420f-8c8e-9b7adbf91163 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:23.566651 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-81a1a2d3-c387-420f-8c8e-9b7adbf91163 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:23.616797 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-bfa69a60-6450-408f-94f8-3d6106a02ea1 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bba77f6ef42547f1aa8480e27868bd4a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '22ca92a86e6c40d59071e58ce4085568', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:22:23.627920 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-bfa69a60-6450-408f-94f8-3d6106a02ea1 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] 10.222.0.16 "GET /compute/v2.1/servers/f114e124-2903-4f30-b055-9f493e3e9e6c" status: 200 len: 1725 microversion: 2.1 time: 0.210624 Jul 16 21:22:23.629395 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1527/3100] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:22:23 2022] GET /compute/v2.1/servers/f114e124-2903-4f30-b055-9f493e3e9e6c => generated 1725 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:22:23.634901 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-f89cb0df-fe48-4335-9145-41cd21d5f39e tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:22:23.635892 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-f89cb0df-fe48-4335-9145-41cd21d5f39e tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] [instance: 94d9ca2e-1c4d-49ce-918e-02479909b75b] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:22:23.644333 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f89cb0df-fe48-4335-9145-41cd21d5f39e tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:23.645002 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f89cb0df-fe48-4335-9145-41cd21d5f39e tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:23.645919 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f89cb0df-fe48-4335-9145-41cd21d5f39e tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:23.646466 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f89cb0df-fe48-4335-9145-41cd21d5f39e tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:23.658479 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-81a1a2d3-c387-420f-8c8e-9b7adbf91163 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f2e974ab5aa48dfb4b00f810dcedb63', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd31be8475a6a49d6bfe8132dd050c6c0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:22:23.666145 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-81a1a2d3-c387-420f-8c8e-9b7adbf91163 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "GET /compute/v2.1/servers/32df0911-cfa6-4410-840a-4098a9fef0b4" status: 200 len: 1274 microversion: 2.1 time: 0.132977 Jul 16 21:22:23.666729 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1574/3101] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:22:23 2022] GET /compute/v2.1/servers/32df0911-cfa6-4410-840a-4098a9fef0b4 => generated 1274 bytes in 134 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:22:23.671963 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-37c0cd54-4cd8-47b8-87a0-37025ffef20d tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:22:23.672690 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-37c0cd54-4cd8-47b8-87a0-37025ffef20d tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] [instance: 583e0c15-3053-429d-81bf-95a4d1f99cbf] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:22:23.687622 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-f89cb0df-fe48-4335-9145-41cd21d5f39e tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] [instance: 94d9ca2e-1c4d-49ce-918e-02479909b75b] Going to try to terminate instance {{(pid=138681) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 16 21:22:23.692600 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-37c0cd54-4cd8-47b8-87a0-37025ffef20d tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:23.692600 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-37c0cd54-4cd8-47b8-87a0-37025ffef20d tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:23.692600 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-37c0cd54-4cd8-47b8-87a0-37025ffef20d tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:23.692600 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-37c0cd54-4cd8-47b8-87a0-37025ffef20d tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:23.731258 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=99,nova_cell1:UPDATE=1 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:22:23.843105 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-37c0cd54-4cd8-47b8-87a0-37025ffef20d tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'aaf2292818b749f0ae43c66bca929be9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '65644a8df5d84df3b35c894560e0a5c4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:22:23.850871 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-37c0cd54-4cd8-47b8-87a0-37025ffef20d tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] 10.222.0.16 "GET /compute/v2.1/servers/583e0c15-3053-429d-81bf-95a4d1f99cbf" status: 200 len: 1524 microversion: 2.1 time: 0.182952 Jul 16 21:22:23.851507 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1575/3102] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:22:23 2022] GET /compute/v2.1/servers/583e0c15-3053-429d-81bf-95a4d1f99cbf => generated 1524 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:22:23.943306 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_api:SELECT=98,nova_api:UPDATE=2 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:22:24.043773 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-f89cb0df-fe48-4335-9145-41cd21d5f39e tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] 10.222.0.16 "DELETE /compute/v2.1/servers/94d9ca2e-1c4d-49ce-918e-02479909b75b" status: 204 len: 0 microversion: 2.1 time: 0.412636 Jul 16 21:22:24.048604 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1528/3103] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:22:23 2022] DELETE /compute/v2.1/servers/94d9ca2e-1c4d-49ce-918e-02479909b75b => generated 0 bytes in 414 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 16 21:22:24.053783 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-276362be-3be7-4037-97bc-2f344e8ce9ce tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:22:24.054334 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-276362be-3be7-4037-97bc-2f344e8ce9ce tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] [instance: 8acee151-2b52-4ca3-b2e1-18a19834b0e2] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:22:24.063117 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-276362be-3be7-4037-97bc-2f344e8ce9ce tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:24.063552 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-276362be-3be7-4037-97bc-2f344e8ce9ce tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:24.064327 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-276362be-3be7-4037-97bc-2f344e8ce9ce tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:24.064766 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-276362be-3be7-4037-97bc-2f344e8ce9ce tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:24.078885 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-276362be-3be7-4037-97bc-2f344e8ce9ce tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] HTTP exception thrown: Instance 8acee151-2b52-4ca3-b2e1-18a19834b0e2 could not be found. Jul 16 21:22:24.080569 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-276362be-3be7-4037-97bc-2f344e8ce9ce tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Request method failure captured: Jul 16 21:22:24.080569 ncdv-master-1 devstack@n-api.service[138680]: request: GET /compute/v2.1/servers/8acee151-2b52-4ca3-b2e1-18a19834b0e2 HTTP/1.1 Jul 16 21:22:24.080569 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:22:24.080569 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:22:24.080569 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:22:24.080569 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 0 Jul 16 21:22:24.080569 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:22:24.080569 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:22:24.080569 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:22:24.080569 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:22:24.080569 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0yvfxxMh1kQhaEsgLF2SOq9-6tkycdPHJwpB0voh-EodC0AvZcoQjlWZQijLUgRQKz2_W2NQj1Lqw2r8EDFvWo3AgwmaOhVrEZAQN9n0U0N5inF9PKjVbZSkNFL0ZbZQ8MJbijHlSUH_9tZn9_wPW1QgQaVuiTwbt8cjpwg_quZbwtU Jul 16 21:22:24.080569 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:22:24.080569 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:22:24.080569 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:22:24.080569 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:22:24.080569 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:22:24.080569 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:22:24.080569 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: c95a2d1e0a034c7ca4a2fcb9fd060d53 Jul 16 21:22:24.080569 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-ListServersNegativeTestJSON-982002908 Jul 16 21:22:24.080569 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:22:24.080569 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:22:24.080569 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/c95a2d1e0a034c7ca4a2fcb9fd060d53"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/c95a2d1e0a034c7ca4a2fcb9fd060d53"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_c95a2d1e0a034c7ca4a2fcb9fd060d53"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/c95a2d1e0a034c7ca4a2fcb9fd060d53"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:22:24.080569 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-ListServersNegativeTestJSON-982002908 Jul 16 21:22:24.080569 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: c95a2d1e0a034c7ca4a2fcb9fd060d53 Jul 16 21:22:24.080569 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-ListServersNegativeTestJSON-982002908 Jul 16 21:22:24.080569 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-ListServersNegativeTestJSON-982002908-project Jul 16 21:22:24.080569 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:22:24.080569 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:22:24.080569 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: fa4c2791e14d4f88883a0af422e4ff0c Jul 16 21:22:24.080569 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-ListServersNegativeTestJSON-982002908-project Jul 16 21:22:24.080569 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 21:22:24.080569 ncdv-master-1 devstack@n-api.service[138680]: exception: Instance 8acee151-2b52-4ca3-b2e1-18a19834b0e2 could not be found. Jul 16 21:22:24.080569 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': '8acee151-2b52-4ca3-b2e1-18a19834b0e2'} Jul 16 21:22:24.080569 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:22:24.080569 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:22:24.080569 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:22:24.080569 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:22:24.080569 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:22:24.080569 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:22:24.080569 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:22:24.080569 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:22:24.080569 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:22:24.080569 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:22:24.080569 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 8acee151-2b52-4ca3-b2e1-18a19834b0e2 could not be found. Jul 16 21:22:24.080569 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:22:24.080569 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:22:24.080569 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:22:24.080569 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:22:24.080569 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:22:24.080569 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:22:24.080569 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:22:24.080569 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:22:24.080569 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:22:24.080569 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:22:24.080569 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:22:24.080569 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:22:24.080569 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:22:24.080569 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:22:24.080569 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:22:24.080569 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:22:24.080569 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 8acee151-2b52-4ca3-b2e1-18a19834b0e2 could not be found. Jul 16 21:22:24.080569 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:22:24.080569 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:22:24.080569 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:22:24.085698 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:22:24.085698 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:22:24.085698 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:22:24.085698 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:22:24.085698 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:22:24.085698 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 8acee151-2b52-4ca3-b2e1-18a19834b0e2 could not be found. Jul 16 21:22:24.085698 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:22:24.085698 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-276362be-3be7-4037-97bc-2f344e8ce9ce tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Returning 404 to user: Instance 8acee151-2b52-4ca3-b2e1-18a19834b0e2 could not be found. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:22:24.085698 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-276362be-3be7-4037-97bc-2f344e8ce9ce tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] 10.222.0.16 "GET /compute/v2.1/servers/8acee151-2b52-4ca3-b2e1-18a19834b0e2" status: 404 len: 111 microversion: 2.1 time: 0.032161 Jul 16 21:22:24.085698 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1576/3104] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:22:24 2022] GET /compute/v2.1/servers/8acee151-2b52-4ca3-b2e1-18a19834b0e2 => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:22:24.091460 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-b30cbdce-8230-40a4-a995-a2b367fbecba tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:22:24.092116 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-b30cbdce-8230-40a4-a995-a2b367fbecba tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] [instance: 9e7787d1-264a-4707-8604-3e8b70133376] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:22:24.099570 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b30cbdce-8230-40a4-a995-a2b367fbecba tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:24.100018 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b30cbdce-8230-40a4-a995-a2b367fbecba tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:24.100781 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b30cbdce-8230-40a4-a995-a2b367fbecba tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:24.101213 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b30cbdce-8230-40a4-a995-a2b367fbecba tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:24.282806 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-b30cbdce-8230-40a4-a995-a2b367fbecba tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa4c2791e14d4f88883a0af422e4ff0c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c95a2d1e0a034c7ca4a2fcb9fd060d53', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:22:24.293990 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-b30cbdce-8230-40a4-a995-a2b367fbecba tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] 10.222.0.16 "GET /compute/v2.1/servers/9e7787d1-264a-4707-8604-3e8b70133376" status: 200 len: 1522 microversion: 2.1 time: 0.206702 Jul 16 21:22:24.295295 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1529/3105] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:22:24 2022] GET /compute/v2.1/servers/9e7787d1-264a-4707-8604-3e8b70133376 => generated 1522 bytes in 209 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:22:24.647278 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-30e3fec2-1d3c-41e3-b448-f3bed0c5c8f3 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:22:24.647890 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-30e3fec2-1d3c-41e3-b448-f3bed0c5c8f3 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] [instance: f114e124-2903-4f30-b055-9f493e3e9e6c] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:22:24.654386 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-30e3fec2-1d3c-41e3-b448-f3bed0c5c8f3 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:24.654911 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-30e3fec2-1d3c-41e3-b448-f3bed0c5c8f3 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:24.655710 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-30e3fec2-1d3c-41e3-b448-f3bed0c5c8f3 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:24.656149 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-30e3fec2-1d3c-41e3-b448-f3bed0c5c8f3 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:24.685632 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-ffc8ebd5-65f5-41ad-8f07-9df1e4b1d3ef tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:22:24.687614 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-ffc8ebd5-65f5-41ad-8f07-9df1e4b1d3ef tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 32df0911-cfa6-4410-840a-4098a9fef0b4] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:22:24.694321 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ffc8ebd5-65f5-41ad-8f07-9df1e4b1d3ef tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:24.701708 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ffc8ebd5-65f5-41ad-8f07-9df1e4b1d3ef tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:24.701708 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ffc8ebd5-65f5-41ad-8f07-9df1e4b1d3ef tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:24.701708 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ffc8ebd5-65f5-41ad-8f07-9df1e4b1d3ef tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:24.801753 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-ffc8ebd5-65f5-41ad-8f07-9df1e4b1d3ef tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f2e974ab5aa48dfb4b00f810dcedb63', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd31be8475a6a49d6bfe8132dd050c6c0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:22:24.816061 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-ffc8ebd5-65f5-41ad-8f07-9df1e4b1d3ef tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "GET /compute/v2.1/servers/32df0911-cfa6-4410-840a-4098a9fef0b4" status: 200 len: 1274 microversion: 2.1 time: 0.134780 Jul 16 21:22:24.816671 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1530/3106] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:22:24 2022] GET /compute/v2.1/servers/32df0911-cfa6-4410-840a-4098a9fef0b4 => generated 1274 bytes in 136 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:22:24.876279 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-adea48b1-59e4-4ddd-b274-306a4513ed90 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:22:24.876872 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-adea48b1-59e4-4ddd-b274-306a4513ed90 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] [instance: 583e0c15-3053-429d-81bf-95a4d1f99cbf] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:22:24.883583 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-adea48b1-59e4-4ddd-b274-306a4513ed90 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:24.884089 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-adea48b1-59e4-4ddd-b274-306a4513ed90 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:24.885440 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-adea48b1-59e4-4ddd-b274-306a4513ed90 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:24.885896 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-adea48b1-59e4-4ddd-b274-306a4513ed90 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:25.002679 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-30e3fec2-1d3c-41e3-b448-f3bed0c5c8f3 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bba77f6ef42547f1aa8480e27868bd4a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '22ca92a86e6c40d59071e58ce4085568', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:22:25.013928 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-30e3fec2-1d3c-41e3-b448-f3bed0c5c8f3 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] 10.222.0.16 "GET /compute/v2.1/servers/f114e124-2903-4f30-b055-9f493e3e9e6c" status: 200 len: 1725 microversion: 2.1 time: 0.371487 Jul 16 21:22:25.029350 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1577/3107] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:22:24 2022] GET /compute/v2.1/servers/f114e124-2903-4f30-b055-9f493e3e9e6c => generated 1725 bytes in 388 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:22:25.208623 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-adea48b1-59e4-4ddd-b274-306a4513ed90 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'aaf2292818b749f0ae43c66bca929be9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '65644a8df5d84df3b35c894560e0a5c4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:22:25.225836 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-adea48b1-59e4-4ddd-b274-306a4513ed90 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] 10.222.0.16 "GET /compute/v2.1/servers/583e0c15-3053-429d-81bf-95a4d1f99cbf" status: 200 len: 1524 microversion: 2.1 time: 0.354619 Jul 16 21:22:25.231380 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1531/3108] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:22:24 2022] GET /compute/v2.1/servers/583e0c15-3053-429d-81bf-95a4d1f99cbf => generated 1524 bytes in 361 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:22:25.326502 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-c60907d5-bd56-4cb7-8151-d20099e445ee tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:22:25.331504 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-c60907d5-bd56-4cb7-8151-d20099e445ee tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] [instance: 9e7787d1-264a-4707-8604-3e8b70133376] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:22:25.342102 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c60907d5-bd56-4cb7-8151-d20099e445ee tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:25.342933 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c60907d5-bd56-4cb7-8151-d20099e445ee tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:25.344075 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c60907d5-bd56-4cb7-8151-d20099e445ee tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:25.344547 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c60907d5-bd56-4cb7-8151-d20099e445ee tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:25.721631 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-c60907d5-bd56-4cb7-8151-d20099e445ee tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa4c2791e14d4f88883a0af422e4ff0c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c95a2d1e0a034c7ca4a2fcb9fd060d53', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:22:25.734278 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-c60907d5-bd56-4cb7-8151-d20099e445ee tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] 10.222.0.16 "GET /compute/v2.1/servers/9e7787d1-264a-4707-8604-3e8b70133376" status: 200 len: 1522 microversion: 2.1 time: 0.422716 Jul 16 21:22:25.738679 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1578/3109] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:22:25 2022] GET /compute/v2.1/servers/9e7787d1-264a-4707-8604-3e8b70133376 => generated 1522 bytes in 426 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:22:25.835441 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-21e3867c-822a-4822-8034-5b9dcfa1d71d tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:22:25.836267 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-21e3867c-822a-4822-8034-5b9dcfa1d71d tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 32df0911-cfa6-4410-840a-4098a9fef0b4] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:22:25.845978 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-21e3867c-822a-4822-8034-5b9dcfa1d71d tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:25.846593 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-21e3867c-822a-4822-8034-5b9dcfa1d71d tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:25.847630 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-21e3867c-822a-4822-8034-5b9dcfa1d71d tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:25.851467 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-21e3867c-822a-4822-8034-5b9dcfa1d71d tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:26.022847 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-21e3867c-822a-4822-8034-5b9dcfa1d71d tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f2e974ab5aa48dfb4b00f810dcedb63', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd31be8475a6a49d6bfe8132dd050c6c0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:22:26.032983 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-21e3867c-822a-4822-8034-5b9dcfa1d71d tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "GET /compute/v2.1/servers/32df0911-cfa6-4410-840a-4098a9fef0b4" status: 200 len: 1272 microversion: 2.1 time: 0.201935 Jul 16 21:22:26.033799 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1532/3110] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:22:25 2022] GET /compute/v2.1/servers/32df0911-cfa6-4410-840a-4098a9fef0b4 => generated 1272 bytes in 204 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:22:26.037139 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-24271543-f76f-4c37-898b-6b8ae6a90436 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:22:26.041416 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-24271543-f76f-4c37-898b-6b8ae6a90436 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] [instance: f114e124-2903-4f30-b055-9f493e3e9e6c] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:22:26.047248 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-24271543-f76f-4c37-898b-6b8ae6a90436 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:26.047917 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-24271543-f76f-4c37-898b-6b8ae6a90436 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:26.053803 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-24271543-f76f-4c37-898b-6b8ae6a90436 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:26.054273 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-24271543-f76f-4c37-898b-6b8ae6a90436 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:26.244660 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-7fece662-a016-4ec4-84af-266fed307f90 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:22:26.245385 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-7fece662-a016-4ec4-84af-266fed307f90 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] [instance: 583e0c15-3053-429d-81bf-95a4d1f99cbf] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:22:26.259497 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7fece662-a016-4ec4-84af-266fed307f90 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:26.260555 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7fece662-a016-4ec4-84af-266fed307f90 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:26.261539 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7fece662-a016-4ec4-84af-266fed307f90 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:26.262330 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7fece662-a016-4ec4-84af-266fed307f90 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:26.571397 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-24271543-f76f-4c37-898b-6b8ae6a90436 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bba77f6ef42547f1aa8480e27868bd4a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '22ca92a86e6c40d59071e58ce4085568', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:22:26.581997 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-24271543-f76f-4c37-898b-6b8ae6a90436 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] 10.222.0.16 "GET /compute/v2.1/servers/f114e124-2903-4f30-b055-9f493e3e9e6c" status: 200 len: 1725 microversion: 2.1 time: 0.548790 Jul 16 21:22:26.583237 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1579/3111] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:22:26 2022] GET /compute/v2.1/servers/f114e124-2903-4f30-b055-9f493e3e9e6c => generated 1725 bytes in 550 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:22:26.588104 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-60492eaf-fc60-4bfb-a1cc-b60d6449c062 req-a87bf059-3851-4217-b83e-2b97979b79d8 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "9e7787d1-264a-4707-8604-3e8b70133376", "name": "network-vif-deleted", "tag": "b5213034-c4ce-4421-b1cd-864c1901ff82"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:22:26.596588 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-60492eaf-fc60-4bfb-a1cc-b60d6449c062 req-a87bf059-3851-4217-b83e-2b97979b79d8 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:26.597042 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-60492eaf-fc60-4bfb-a1cc-b60d6449c062 req-a87bf059-3851-4217-b83e-2b97979b79d8 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:26.609651 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-60492eaf-fc60-4bfb-a1cc-b60d6449c062 req-a87bf059-3851-4217-b83e-2b97979b79d8 service nova] Creating event network-vif-deleted:b5213034-c4ce-4421-b1cd-864c1901ff82 for instance 9e7787d1-264a-4707-8604-3e8b70133376 on nch1-master-1 Jul 16 21:22:26.614899 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-60492eaf-fc60-4bfb-a1cc-b60d6449c062 req-a87bf059-3851-4217-b83e-2b97979b79d8 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.030277 Jul 16 21:22:26.615547 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1580/3112] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:22:26 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:22:26.692436 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-7fece662-a016-4ec4-84af-266fed307f90 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'aaf2292818b749f0ae43c66bca929be9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '65644a8df5d84df3b35c894560e0a5c4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:22:26.702852 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-7fece662-a016-4ec4-84af-266fed307f90 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] 10.222.0.16 "GET /compute/v2.1/servers/583e0c15-3053-429d-81bf-95a4d1f99cbf" status: 200 len: 1524 microversion: 2.1 time: 0.462814 Jul 16 21:22:26.703930 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1533/3113] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:22:26 2022] GET /compute/v2.1/servers/583e0c15-3053-429d-81bf-95a4d1f99cbf => generated 1524 bytes in 465 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:22:26.758310 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-f84e3c5a-3b8f-4d76-a55d-d864aafffd8f tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:22:26.759396 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-f84e3c5a-3b8f-4d76-a55d-d864aafffd8f tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] [instance: 9e7787d1-264a-4707-8604-3e8b70133376] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:22:26.767495 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f84e3c5a-3b8f-4d76-a55d-d864aafffd8f tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:26.769335 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f84e3c5a-3b8f-4d76-a55d-d864aafffd8f tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:26.769335 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f84e3c5a-3b8f-4d76-a55d-d864aafffd8f tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:26.769335 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f84e3c5a-3b8f-4d76-a55d-d864aafffd8f tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:26.855334 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-f84e3c5a-3b8f-4d76-a55d-d864aafffd8f tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa4c2791e14d4f88883a0af422e4ff0c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c95a2d1e0a034c7ca4a2fcb9fd060d53', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:22:26.874790 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-f84e3c5a-3b8f-4d76-a55d-d864aafffd8f tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] 10.222.0.16 "GET /compute/v2.1/servers/9e7787d1-264a-4707-8604-3e8b70133376" status: 200 len: 1308 microversion: 2.1 time: 0.123616 Jul 16 21:22:26.880258 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1581/3114] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:22:26 2022] GET /compute/v2.1/servers/9e7787d1-264a-4707-8604-3e8b70133376 => generated 1308 bytes in 130 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:22:27.057339 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-b9018559-9d90-4076-a911-f489b055e798 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:22:27.058401 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-b9018559-9d90-4076-a911-f489b055e798 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 32df0911-cfa6-4410-840a-4098a9fef0b4] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:22:27.066566 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b9018559-9d90-4076-a911-f489b055e798 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:27.067764 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b9018559-9d90-4076-a911-f489b055e798 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:27.069133 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b9018559-9d90-4076-a911-f489b055e798 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:27.069968 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b9018559-9d90-4076-a911-f489b055e798 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:27.377748 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-b9018559-9d90-4076-a911-f489b055e798 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f2e974ab5aa48dfb4b00f810dcedb63', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd31be8475a6a49d6bfe8132dd050c6c0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:22:27.387313 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-b9018559-9d90-4076-a911-f489b055e798 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "GET /compute/v2.1/servers/32df0911-cfa6-4410-840a-4098a9fef0b4" status: 200 len: 1314 microversion: 2.1 time: 0.334352 Jul 16 21:22:27.397023 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1534/3115] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:22:27 2022] GET /compute/v2.1/servers/32df0911-cfa6-4410-840a-4098a9fef0b4 => generated 1314 bytes in 336 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:22:27.599761 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-261c12ad-a0b6-462f-8c54-3a03cea6f88f tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:22:27.600388 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-261c12ad-a0b6-462f-8c54-3a03cea6f88f tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] [instance: f114e124-2903-4f30-b055-9f493e3e9e6c] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:22:27.607458 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-261c12ad-a0b6-462f-8c54-3a03cea6f88f tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:27.608058 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-261c12ad-a0b6-462f-8c54-3a03cea6f88f tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:27.609008 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-261c12ad-a0b6-462f-8c54-3a03cea6f88f tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:27.609459 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-261c12ad-a0b6-462f-8c54-3a03cea6f88f tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:27.721646 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-38dd0cd9-eb91-4409-b272-87107ca246c7 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:22:27.723374 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-38dd0cd9-eb91-4409-b272-87107ca246c7 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] [instance: 583e0c15-3053-429d-81bf-95a4d1f99cbf] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:22:27.729969 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-38dd0cd9-eb91-4409-b272-87107ca246c7 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:27.730426 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-38dd0cd9-eb91-4409-b272-87107ca246c7 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:27.731302 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-38dd0cd9-eb91-4409-b272-87107ca246c7 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:27.731827 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-38dd0cd9-eb91-4409-b272-87107ca246c7 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:28.312030 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-261c12ad-a0b6-462f-8c54-3a03cea6f88f tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bba77f6ef42547f1aa8480e27868bd4a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '22ca92a86e6c40d59071e58ce4085568', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:22:28.320236 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-261c12ad-a0b6-462f-8c54-3a03cea6f88f tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] 10.222.0.16 "GET /compute/v2.1/servers/f114e124-2903-4f30-b055-9f493e3e9e6c" status: 200 len: 1725 microversion: 2.1 time: 0.724427 Jul 16 21:22:28.320804 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1582/3116] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:22:27 2022] GET /compute/v2.1/servers/f114e124-2903-4f30-b055-9f493e3e9e6c => generated 1725 bytes in 726 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:22:28.326232 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-0f492773-7f98-48d0-89b9-9267aa6d8c35 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:22:28.326880 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-0f492773-7f98-48d0-89b9-9267aa6d8c35 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] [instance: 9e7787d1-264a-4707-8604-3e8b70133376] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:22:28.333388 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0f492773-7f98-48d0-89b9-9267aa6d8c35 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:28.333846 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0f492773-7f98-48d0-89b9-9267aa6d8c35 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:28.334615 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0f492773-7f98-48d0-89b9-9267aa6d8c35 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:28.335074 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0f492773-7f98-48d0-89b9-9267aa6d8c35 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:28.351117 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-0f492773-7f98-48d0-89b9-9267aa6d8c35 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] HTTP exception thrown: Instance 9e7787d1-264a-4707-8604-3e8b70133376 could not be found. Jul 16 21:22:28.352754 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-0f492773-7f98-48d0-89b9-9267aa6d8c35 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Request method failure captured: Jul 16 21:22:28.352754 ncdv-master-1 devstack@n-api.service[138680]: request: GET /compute/v2.1/servers/9e7787d1-264a-4707-8604-3e8b70133376 HTTP/1.1 Jul 16 21:22:28.352754 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:22:28.352754 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:22:28.352754 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:22:28.352754 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 0 Jul 16 21:22:28.352754 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:22:28.352754 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:22:28.352754 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:22:28.352754 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:22:28.352754 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0yvfxxMh1kQhaEsgLF2SOq9-6tkycdPHJwpB0voh-EodC0AvZcoQjlWZQijLUgRQKz2_W2NQj1Lqw2r8EDFvWo3AgwmaOhVrEZAQN9n0U0N5inF9PKjVbZSkNFL0ZbZQ8MJbijHlSUH_9tZn9_wPW1QgQaVuiTwbt8cjpwg_quZbwtU Jul 16 21:22:28.352754 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:22:28.352754 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:22:28.352754 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:22:28.352754 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:22:28.352754 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:22:28.352754 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:22:28.352754 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: c95a2d1e0a034c7ca4a2fcb9fd060d53 Jul 16 21:22:28.352754 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-ListServersNegativeTestJSON-982002908 Jul 16 21:22:28.352754 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:22:28.352754 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:22:28.352754 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/c95a2d1e0a034c7ca4a2fcb9fd060d53"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/c95a2d1e0a034c7ca4a2fcb9fd060d53"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_c95a2d1e0a034c7ca4a2fcb9fd060d53"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/c95a2d1e0a034c7ca4a2fcb9fd060d53"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:22:28.352754 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-ListServersNegativeTestJSON-982002908 Jul 16 21:22:28.352754 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: c95a2d1e0a034c7ca4a2fcb9fd060d53 Jul 16 21:22:28.352754 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-ListServersNegativeTestJSON-982002908 Jul 16 21:22:28.352754 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-ListServersNegativeTestJSON-982002908-project Jul 16 21:22:28.352754 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:22:28.352754 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:22:28.352754 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: fa4c2791e14d4f88883a0af422e4ff0c Jul 16 21:22:28.352754 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-ListServersNegativeTestJSON-982002908-project Jul 16 21:22:28.352754 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 21:22:28.352754 ncdv-master-1 devstack@n-api.service[138680]: exception: Instance 9e7787d1-264a-4707-8604-3e8b70133376 could not be found. Jul 16 21:22:28.352754 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': '9e7787d1-264a-4707-8604-3e8b70133376'} Jul 16 21:22:28.352754 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:22:28.352754 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:22:28.352754 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:22:28.352754 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:22:28.352754 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:22:28.352754 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:22:28.352754 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:22:28.352754 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:22:28.352754 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:22:28.352754 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:22:28.352754 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 9e7787d1-264a-4707-8604-3e8b70133376 could not be found. Jul 16 21:22:28.352754 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:22:28.352754 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:22:28.352754 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:22:28.352754 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:22:28.352754 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:22:28.352754 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:22:28.352754 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:22:28.352754 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:22:28.352754 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:22:28.352754 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:22:28.352754 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:22:28.352754 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:22:28.352754 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:22:28.352754 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:22:28.352754 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:22:28.352754 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:22:28.352754 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 9e7787d1-264a-4707-8604-3e8b70133376 could not be found. Jul 16 21:22:28.352754 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:22:28.352754 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:22:28.352754 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:22:28.358777 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:22:28.358777 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:22:28.358777 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:22:28.358777 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:22:28.358777 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:22:28.358777 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 9e7787d1-264a-4707-8604-3e8b70133376 could not be found. Jul 16 21:22:28.358777 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:22:28.358777 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-0f492773-7f98-48d0-89b9-9267aa6d8c35 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Returning 404 to user: Instance 9e7787d1-264a-4707-8604-3e8b70133376 could not be found. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:22:28.358777 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-0f492773-7f98-48d0-89b9-9267aa6d8c35 tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] 10.222.0.16 "GET /compute/v2.1/servers/9e7787d1-264a-4707-8604-3e8b70133376" status: 404 len: 111 microversion: 2.1 time: 0.031459 Jul 16 21:22:28.358777 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1583/3117] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:22:28 2022] GET /compute/v2.1/servers/9e7787d1-264a-4707-8604-3e8b70133376 => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:22:28.363760 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-2c782ee5-4594-48cd-a485-d8976e77fd2e tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:22:28.364369 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-2c782ee5-4594-48cd-a485-d8976e77fd2e tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] [instance: 94d9ca2e-1c4d-49ce-918e-02479909b75b] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:22:28.378408 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2c782ee5-4594-48cd-a485-d8976e77fd2e tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:28.378854 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2c782ee5-4594-48cd-a485-d8976e77fd2e tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:28.379689 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2c782ee5-4594-48cd-a485-d8976e77fd2e tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:28.380232 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2c782ee5-4594-48cd-a485-d8976e77fd2e tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:28.384916 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-38dd0cd9-eb91-4409-b272-87107ca246c7 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'aaf2292818b749f0ae43c66bca929be9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '65644a8df5d84df3b35c894560e0a5c4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:22:28.398107 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-38dd0cd9-eb91-4409-b272-87107ca246c7 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] 10.222.0.16 "GET /compute/v2.1/servers/583e0c15-3053-429d-81bf-95a4d1f99cbf" status: 200 len: 1524 microversion: 2.1 time: 0.681202 Jul 16 21:22:28.398324 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-2c782ee5-4594-48cd-a485-d8976e77fd2e tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] HTTP exception thrown: Instance 94d9ca2e-1c4d-49ce-918e-02479909b75b could not be found. Jul 16 21:22:28.399947 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-2c782ee5-4594-48cd-a485-d8976e77fd2e tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Request method failure captured: Jul 16 21:22:28.399947 ncdv-master-1 devstack@n-api.service[138680]: request: GET /compute/v2.1/servers/94d9ca2e-1c4d-49ce-918e-02479909b75b HTTP/1.1 Jul 16 21:22:28.399947 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:22:28.399947 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:22:28.399947 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:22:28.399947 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 0 Jul 16 21:22:28.399947 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:22:28.399947 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:22:28.399947 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:22:28.399947 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:22:28.399947 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0yvfxxMh1kQhaEsgLF2SOq9-6tkycdPHJwpB0voh-EodC0AvZcoQjlWZQijLUgRQKz2_W2NQj1Lqw2r8EDFvWo3AgwmaOhVrEZAQN9n0U0N5inF9PKjVbZSkNFL0ZbZQ8MJbijHlSUH_9tZn9_wPW1QgQaVuiTwbt8cjpwg_quZbwtU Jul 16 21:22:28.399947 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:22:28.399947 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:22:28.399947 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:22:28.399947 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:22:28.399947 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:22:28.399947 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:22:28.399947 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: c95a2d1e0a034c7ca4a2fcb9fd060d53 Jul 16 21:22:28.399947 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-ListServersNegativeTestJSON-982002908 Jul 16 21:22:28.399947 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:22:28.399947 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:22:28.399947 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/c95a2d1e0a034c7ca4a2fcb9fd060d53"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/c95a2d1e0a034c7ca4a2fcb9fd060d53"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_c95a2d1e0a034c7ca4a2fcb9fd060d53"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/c95a2d1e0a034c7ca4a2fcb9fd060d53"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:22:28.399947 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-ListServersNegativeTestJSON-982002908 Jul 16 21:22:28.399947 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: c95a2d1e0a034c7ca4a2fcb9fd060d53 Jul 16 21:22:28.399947 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-ListServersNegativeTestJSON-982002908 Jul 16 21:22:28.399947 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-ListServersNegativeTestJSON-982002908-project Jul 16 21:22:28.399947 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:22:28.399947 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:22:28.399947 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: fa4c2791e14d4f88883a0af422e4ff0c Jul 16 21:22:28.399947 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-ListServersNegativeTestJSON-982002908-project Jul 16 21:22:28.399947 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 21:22:28.399947 ncdv-master-1 devstack@n-api.service[138680]: exception: Instance 94d9ca2e-1c4d-49ce-918e-02479909b75b could not be found. Jul 16 21:22:28.399947 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': '94d9ca2e-1c4d-49ce-918e-02479909b75b'} Jul 16 21:22:28.399947 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:22:28.399947 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:22:28.399947 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:22:28.399947 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:22:28.399947 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:22:28.399947 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:22:28.399947 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:22:28.399947 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:22:28.399947 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:22:28.399947 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:22:28.399947 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 94d9ca2e-1c4d-49ce-918e-02479909b75b could not be found. Jul 16 21:22:28.399947 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:22:28.399947 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:22:28.399947 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:22:28.399947 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:22:28.399947 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:22:28.399947 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:22:28.399947 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:22:28.399947 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:22:28.399947 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:22:28.399947 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:22:28.399947 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:22:28.399947 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:22:28.399947 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:22:28.399947 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:22:28.399947 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:22:28.399947 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:22:28.399947 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 94d9ca2e-1c4d-49ce-918e-02479909b75b could not be found. Jul 16 21:22:28.399947 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:22:28.399947 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:22:28.399947 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:22:28.405275 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:22:28.405275 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:22:28.405275 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:22:28.405275 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:22:28.405275 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:22:28.405275 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 94d9ca2e-1c4d-49ce-918e-02479909b75b could not be found. Jul 16 21:22:28.405275 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:22:28.405275 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-2c782ee5-4594-48cd-a485-d8976e77fd2e tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] Returning 404 to user: Instance 94d9ca2e-1c4d-49ce-918e-02479909b75b could not be found. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:22:28.405275 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-2c782ee5-4594-48cd-a485-d8976e77fd2e tempest-ListServersNegativeTestJSON-982002908 tempest-ListServersNegativeTestJSON-982002908-project] 10.222.0.16 "GET /compute/v2.1/servers/94d9ca2e-1c4d-49ce-918e-02479909b75b" status: 404 len: 111 microversion: 2.1 time: 0.042798 Jul 16 21:22:28.407356 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1584/3118] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:22:28 2022] GET /compute/v2.1/servers/94d9ca2e-1c4d-49ce-918e-02479909b75b => generated 111 bytes in 48 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Jul 16 21:22:28.410610 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1535/3119] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:22:27 2022] GET /compute/v2.1/servers/583e0c15-3053-429d-81bf-95a4d1f99cbf => generated 1524 bytes in 695 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:22:28.412085 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-5ffdc6f0-2f65-4ec4-9ae0-115884bcd684 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:22:28.412719 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-5ffdc6f0-2f65-4ec4-9ae0-115884bcd684 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 32df0911-cfa6-4410-840a-4098a9fef0b4] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:22:28.418852 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5ffdc6f0-2f65-4ec4-9ae0-115884bcd684 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:28.419350 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5ffdc6f0-2f65-4ec4-9ae0-115884bcd684 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:28.420139 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5ffdc6f0-2f65-4ec4-9ae0-115884bcd684 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:28.421170 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5ffdc6f0-2f65-4ec4-9ae0-115884bcd684 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:28.676109 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-b2738af9-eada-4164-a735-43232395b19c req-e782b6b3-79eb-498e-88a1-3385de8c5663 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "94d9ca2e-1c4d-49ce-918e-02479909b75b", "name": "network-vif-deleted", "tag": "782384db-d832-4551-9c6b-3eb3e7a7352b"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:22:28.685077 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-b2738af9-eada-4164-a735-43232395b19c req-e782b6b3-79eb-498e-88a1-3385de8c5663 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:28.685588 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-b2738af9-eada-4164-a735-43232395b19c req-e782b6b3-79eb-498e-88a1-3385de8c5663 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:28.699414 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-b2738af9-eada-4164-a735-43232395b19c req-e782b6b3-79eb-498e-88a1-3385de8c5663 service nova] Creating event network-vif-deleted:782384db-d832-4551-9c6b-3eb3e7a7352b for instance 94d9ca2e-1c4d-49ce-918e-02479909b75b on nch2-master-1 Jul 16 21:22:28.705030 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-b2738af9-eada-4164-a735-43232395b19c req-e782b6b3-79eb-498e-88a1-3385de8c5663 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.032639 Jul 16 21:22:28.705865 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1536/3120] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:22:28 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:22:28.787147 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-5ffdc6f0-2f65-4ec4-9ae0-115884bcd684 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f2e974ab5aa48dfb4b00f810dcedb63', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd31be8475a6a49d6bfe8132dd050c6c0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:22:28.794865 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-5ffdc6f0-2f65-4ec4-9ae0-115884bcd684 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "GET /compute/v2.1/servers/32df0911-cfa6-4410-840a-4098a9fef0b4" status: 200 len: 1314 microversion: 2.1 time: 0.386425 Jul 16 21:22:28.795987 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1585/3121] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:22:28 2022] GET /compute/v2.1/servers/32df0911-cfa6-4410-840a-4098a9fef0b4 => generated 1314 bytes in 388 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:22:29.340122 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-7eec71c3-9b8b-4d1e-811c-f19f158c0944 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:22:29.340753 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-7eec71c3-9b8b-4d1e-811c-f19f158c0944 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] [instance: f114e124-2903-4f30-b055-9f493e3e9e6c] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:22:29.365454 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7eec71c3-9b8b-4d1e-811c-f19f158c0944 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:29.365454 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7eec71c3-9b8b-4d1e-811c-f19f158c0944 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:29.365454 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7eec71c3-9b8b-4d1e-811c-f19f158c0944 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:29.365454 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7eec71c3-9b8b-4d1e-811c-f19f158c0944 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:29.417376 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-ef02c1ba-96d2-4f98-b488-57c01c3ae918 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:22:29.418291 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-ef02c1ba-96d2-4f98-b488-57c01c3ae918 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] [instance: 583e0c15-3053-429d-81bf-95a4d1f99cbf] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:22:29.433766 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ef02c1ba-96d2-4f98-b488-57c01c3ae918 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:29.434223 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ef02c1ba-96d2-4f98-b488-57c01c3ae918 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:29.437418 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ef02c1ba-96d2-4f98-b488-57c01c3ae918 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:29.438041 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ef02c1ba-96d2-4f98-b488-57c01c3ae918 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:29.803630 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-7eec71c3-9b8b-4d1e-811c-f19f158c0944 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bba77f6ef42547f1aa8480e27868bd4a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '22ca92a86e6c40d59071e58ce4085568', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:22:29.822080 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-7eec71c3-9b8b-4d1e-811c-f19f158c0944 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] 10.222.0.16 "GET /compute/v2.1/servers/f114e124-2903-4f30-b055-9f493e3e9e6c" status: 200 len: 1725 microversion: 2.1 time: 0.487376 Jul 16 21:22:29.825492 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1537/3122] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:22:29 2022] GET /compute/v2.1/servers/f114e124-2903-4f30-b055-9f493e3e9e6c => generated 1725 bytes in 492 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:22:29.835832 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-d9af6015-b723-4e67-b5fa-c24a21b5b0b1 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:22:29.835832 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-d9af6015-b723-4e67-b5fa-c24a21b5b0b1 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 32df0911-cfa6-4410-840a-4098a9fef0b4] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:22:29.848614 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d9af6015-b723-4e67-b5fa-c24a21b5b0b1 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:29.850525 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d9af6015-b723-4e67-b5fa-c24a21b5b0b1 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:29.852323 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d9af6015-b723-4e67-b5fa-c24a21b5b0b1 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:29.853179 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d9af6015-b723-4e67-b5fa-c24a21b5b0b1 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:29.867766 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-ef02c1ba-96d2-4f98-b488-57c01c3ae918 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'aaf2292818b749f0ae43c66bca929be9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '65644a8df5d84df3b35c894560e0a5c4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:22:29.879821 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-ef02c1ba-96d2-4f98-b488-57c01c3ae918 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] 10.222.0.16 "GET /compute/v2.1/servers/583e0c15-3053-429d-81bf-95a4d1f99cbf" status: 200 len: 1524 microversion: 2.1 time: 0.466918 Jul 16 21:22:29.890751 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1586/3123] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:22:29 2022] GET /compute/v2.1/servers/583e0c15-3053-429d-81bf-95a4d1f99cbf => generated 1524 bytes in 472 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:22:30.075442 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-d9af6015-b723-4e67-b5fa-c24a21b5b0b1 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f2e974ab5aa48dfb4b00f810dcedb63', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd31be8475a6a49d6bfe8132dd050c6c0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:22:30.084470 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-d9af6015-b723-4e67-b5fa-c24a21b5b0b1 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "GET /compute/v2.1/servers/32df0911-cfa6-4410-840a-4098a9fef0b4" status: 200 len: 1314 microversion: 2.1 time: 0.255602 Jul 16 21:22:30.086712 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1538/3124] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:22:29 2022] GET /compute/v2.1/servers/32df0911-cfa6-4410-840a-4098a9fef0b4 => generated 1314 bytes in 258 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:22:30.731269 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-a3dffe81-4e0c-47ac-8ce4-20d5cafa3cc0 req-6492058c-fdd0-46b5-a620-a8420486f3a0 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "32df0911-cfa6-4410-840a-4098a9fef0b4", "tag": "37e2d383-525b-40f5-ae01-66bf83a6301d"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:22:30.744620 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-a3dffe81-4e0c-47ac-8ce4-20d5cafa3cc0 req-6492058c-fdd0-46b5-a620-a8420486f3a0 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:30.745147 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-a3dffe81-4e0c-47ac-8ce4-20d5cafa3cc0 req-6492058c-fdd0-46b5-a620-a8420486f3a0 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:30.781637 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-a3dffe81-4e0c-47ac-8ce4-20d5cafa3cc0 req-6492058c-fdd0-46b5-a620-a8420486f3a0 service nova] Creating event network-changed:37e2d383-525b-40f5-ae01-66bf83a6301d for instance 32df0911-cfa6-4410-840a-4098a9fef0b4 on nch1-master-1 Jul 16 21:22:30.788508 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-a3dffe81-4e0c-47ac-8ce4-20d5cafa3cc0 req-6492058c-fdd0-46b5-a620-a8420486f3a0 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.061182 Jul 16 21:22:30.789406 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1587/3125] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:22:30 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 64 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:22:30.851948 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-d36790ec-12a8-4505-be5f-454a04b25a39 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:22:30.852562 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-d36790ec-12a8-4505-be5f-454a04b25a39 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] [instance: f114e124-2903-4f30-b055-9f493e3e9e6c] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:22:30.860576 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d36790ec-12a8-4505-be5f-454a04b25a39 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:30.861354 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d36790ec-12a8-4505-be5f-454a04b25a39 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:30.862299 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d36790ec-12a8-4505-be5f-454a04b25a39 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:30.862901 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d36790ec-12a8-4505-be5f-454a04b25a39 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:30.899721 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-2f5b3d85-b8f6-4832-89a1-8d9b0ad544fb tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:22:30.900639 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-2f5b3d85-b8f6-4832-89a1-8d9b0ad544fb tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] [instance: 583e0c15-3053-429d-81bf-95a4d1f99cbf] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:22:30.907962 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2f5b3d85-b8f6-4832-89a1-8d9b0ad544fb tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:30.908694 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2f5b3d85-b8f6-4832-89a1-8d9b0ad544fb tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:30.909819 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2f5b3d85-b8f6-4832-89a1-8d9b0ad544fb tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:30.913173 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2f5b3d85-b8f6-4832-89a1-8d9b0ad544fb tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:31.337937 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-d36790ec-12a8-4505-be5f-454a04b25a39 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bba77f6ef42547f1aa8480e27868bd4a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '22ca92a86e6c40d59071e58ce4085568', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:22:31.349734 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-d36790ec-12a8-4505-be5f-454a04b25a39 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] 10.222.0.16 "GET /compute/v2.1/servers/f114e124-2903-4f30-b055-9f493e3e9e6c" status: 200 len: 1725 microversion: 2.1 time: 0.501572 Jul 16 21:22:31.351031 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1539/3126] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:22:30 2022] GET /compute/v2.1/servers/f114e124-2903-4f30-b055-9f493e3e9e6c => generated 1725 bytes in 503 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:22:31.356656 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell0:SELECT=8 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:22:31.365146 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-b6e624b6-ac03-4800-b585-5c555c08075a tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:22:31.366156 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-b6e624b6-ac03-4800-b585-5c555c08075a tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 32df0911-cfa6-4410-840a-4098a9fef0b4] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:22:31.379570 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b6e624b6-ac03-4800-b585-5c555c08075a tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:31.380345 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b6e624b6-ac03-4800-b585-5c555c08075a tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:31.381438 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b6e624b6-ac03-4800-b585-5c555c08075a tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:31.382165 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b6e624b6-ac03-4800-b585-5c555c08075a tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:31.385976 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-2f5b3d85-b8f6-4832-89a1-8d9b0ad544fb tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'aaf2292818b749f0ae43c66bca929be9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '65644a8df5d84df3b35c894560e0a5c4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:22:31.408244 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-2f5b3d85-b8f6-4832-89a1-8d9b0ad544fb tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] 10.222.0.16 "GET /compute/v2.1/servers/583e0c15-3053-429d-81bf-95a4d1f99cbf" status: 200 len: 1524 microversion: 2.1 time: 0.511610 Jul 16 21:22:31.408973 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1588/3127] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:22:30 2022] GET /compute/v2.1/servers/583e0c15-3053-429d-81bf-95a4d1f99cbf => generated 1524 bytes in 516 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:22:31.578155 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-b6e624b6-ac03-4800-b585-5c555c08075a tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f2e974ab5aa48dfb4b00f810dcedb63', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd31be8475a6a49d6bfe8132dd050c6c0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:22:31.586957 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-b6e624b6-ac03-4800-b585-5c555c08075a tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "GET /compute/v2.1/servers/32df0911-cfa6-4410-840a-4098a9fef0b4" status: 200 len: 1314 microversion: 2.1 time: 0.234617 Jul 16 21:22:31.587584 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1540/3128] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:22:31 2022] GET /compute/v2.1/servers/32df0911-cfa6-4410-840a-4098a9fef0b4 => generated 1314 bytes in 236 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:22:32.368843 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-fde6b8c5-50b9-4ec3-885b-fd131fa3f023 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:22:32.369799 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-fde6b8c5-50b9-4ec3-885b-fd131fa3f023 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] [instance: f114e124-2903-4f30-b055-9f493e3e9e6c] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:22:32.377810 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-fde6b8c5-50b9-4ec3-885b-fd131fa3f023 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:32.378425 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-fde6b8c5-50b9-4ec3-885b-fd131fa3f023 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:32.379491 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-fde6b8c5-50b9-4ec3-885b-fd131fa3f023 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:32.380110 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-fde6b8c5-50b9-4ec3-885b-fd131fa3f023 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:32.426286 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-5074472e-9900-466a-bbe0-b756a622d0e5 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:22:32.426909 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-5074472e-9900-466a-bbe0-b756a622d0e5 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] [instance: 583e0c15-3053-429d-81bf-95a4d1f99cbf] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:22:32.437346 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5074472e-9900-466a-bbe0-b756a622d0e5 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:32.437818 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5074472e-9900-466a-bbe0-b756a622d0e5 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:32.438645 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5074472e-9900-466a-bbe0-b756a622d0e5 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:32.439124 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5074472e-9900-466a-bbe0-b756a622d0e5 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:32.620408 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-fde6b8c5-50b9-4ec3-885b-fd131fa3f023 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bba77f6ef42547f1aa8480e27868bd4a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '22ca92a86e6c40d59071e58ce4085568', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:22:32.623991 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:22:32.631989 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-fde6b8c5-50b9-4ec3-885b-fd131fa3f023 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] 10.222.0.16 "GET /compute/v2.1/servers/f114e124-2903-4f30-b055-9f493e3e9e6c" status: 200 len: 1725 microversion: 2.1 time: 0.267922 Jul 16 21:22:32.633484 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1589/3129] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:22:32 2022] GET /compute/v2.1/servers/f114e124-2903-4f30-b055-9f493e3e9e6c => generated 1725 bytes in 269 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:22:32.638368 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-c9af352f-74bb-47a6-9eda-3adbdd142d1d tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:22:32.639505 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-c9af352f-74bb-47a6-9eda-3adbdd142d1d tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 32df0911-cfa6-4410-840a-4098a9fef0b4] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:22:32.647155 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c9af352f-74bb-47a6-9eda-3adbdd142d1d tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:32.647808 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c9af352f-74bb-47a6-9eda-3adbdd142d1d tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:32.648802 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c9af352f-74bb-47a6-9eda-3adbdd142d1d tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:32.649411 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c9af352f-74bb-47a6-9eda-3adbdd142d1d tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:32.659288 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-5074472e-9900-466a-bbe0-b756a622d0e5 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'aaf2292818b749f0ae43c66bca929be9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '65644a8df5d84df3b35c894560e0a5c4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:22:32.677542 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-5074472e-9900-466a-bbe0-b756a622d0e5 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] 10.222.0.16 "GET /compute/v2.1/servers/583e0c15-3053-429d-81bf-95a4d1f99cbf" status: 200 len: 1524 microversion: 2.1 time: 0.255632 Jul 16 21:22:32.679851 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1541/3130] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:22:32 2022] GET /compute/v2.1/servers/583e0c15-3053-429d-81bf-95a4d1f99cbf => generated 1524 bytes in 259 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:22:32.895610 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-c9af352f-74bb-47a6-9eda-3adbdd142d1d tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f2e974ab5aa48dfb4b00f810dcedb63', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd31be8475a6a49d6bfe8132dd050c6c0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:22:32.915475 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-c9af352f-74bb-47a6-9eda-3adbdd142d1d tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "GET /compute/v2.1/servers/32df0911-cfa6-4410-840a-4098a9fef0b4" status: 200 len: 1314 microversion: 2.1 time: 0.279758 Jul 16 21:22:32.915810 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1590/3131] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:22:32 2022] GET /compute/v2.1/servers/32df0911-cfa6-4410-840a-4098a9fef0b4 => generated 1314 bytes in 282 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:22:33.650173 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-9b3bf0e0-adf4-4815-ab6c-e99edf71e709 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:22:33.650792 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-9b3bf0e0-adf4-4815-ab6c-e99edf71e709 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] [instance: f114e124-2903-4f30-b055-9f493e3e9e6c] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:22:33.652858 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell0:SELECT=8 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:22:33.653942 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell0:SELECT=2 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:22:33.661434 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9b3bf0e0-adf4-4815-ab6c-e99edf71e709 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:33.661434 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9b3bf0e0-adf4-4815-ab6c-e99edf71e709 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:33.662422 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9b3bf0e0-adf4-4815-ab6c-e99edf71e709 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:33.662422 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9b3bf0e0-adf4-4815-ab6c-e99edf71e709 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:33.696996 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-e9657420-f34f-4b2c-b95e-c5eb80f7e0a6 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:22:33.697935 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-e9657420-f34f-4b2c-b95e-c5eb80f7e0a6 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] [instance: 583e0c15-3053-429d-81bf-95a4d1f99cbf] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:22:33.700601 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell0:SELECT=8 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:22:33.707408 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e9657420-f34f-4b2c-b95e-c5eb80f7e0a6 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:33.707923 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e9657420-f34f-4b2c-b95e-c5eb80f7e0a6 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:33.708850 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e9657420-f34f-4b2c-b95e-c5eb80f7e0a6 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:33.709660 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e9657420-f34f-4b2c-b95e-c5eb80f7e0a6 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:33.869510 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-9b3bf0e0-adf4-4815-ab6c-e99edf71e709 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bba77f6ef42547f1aa8480e27868bd4a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '22ca92a86e6c40d59071e58ce4085568', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:22:33.885687 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-9b3bf0e0-adf4-4815-ab6c-e99edf71e709 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] 10.222.0.16 "GET /compute/v2.1/servers/f114e124-2903-4f30-b055-9f493e3e9e6c" status: 200 len: 1702 microversion: 2.1 time: 0.239798 Jul 16 21:22:33.889817 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1542/3132] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:22:33 2022] GET /compute/v2.1/servers/f114e124-2903-4f30-b055-9f493e3e9e6c => generated 1702 bytes in 245 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:22:33.906571 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-ccdb36b4-1a4f-4a18-a436-e05ddc8eade8 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Action: 'action', calling method: >, body: {"os-start": {}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:22:33.913845 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ccdb36b4-1a4f-4a18-a436-e05ddc8eade8 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:33.914439 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ccdb36b4-1a4f-4a18-a436-e05ddc8eade8 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:33.930908 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-e9657420-f34f-4b2c-b95e-c5eb80f7e0a6 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'aaf2292818b749f0ae43c66bca929be9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '65644a8df5d84df3b35c894560e0a5c4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:22:33.940581 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-e9657420-f34f-4b2c-b95e-c5eb80f7e0a6 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] 10.222.0.16 "GET /compute/v2.1/servers/583e0c15-3053-429d-81bf-95a4d1f99cbf" status: 200 len: 1524 microversion: 2.1 time: 0.247543 Jul 16 21:22:33.941525 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1591/3133] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:22:33 2022] GET /compute/v2.1/servers/583e0c15-3053-429d-81bf-95a4d1f99cbf => generated 1524 bytes in 249 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:22:33.941780 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-ccdb36b4-1a4f-4a18-a436-e05ddc8eade8 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] [instance: f114e124-2903-4f30-b055-9f493e3e9e6c] Going to try to start instance {{(pid=138681) start /opt/stack/nova/nova/compute/api.py:2730}} Jul 16 21:22:33.953110 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-3a52413d-84ec-45b5-8f54-dae18318263a tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:22:33.954175 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-3a52413d-84ec-45b5-8f54-dae18318263a tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 32df0911-cfa6-4410-840a-4098a9fef0b4] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:22:33.962677 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3a52413d-84ec-45b5-8f54-dae18318263a tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:33.962999 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3a52413d-84ec-45b5-8f54-dae18318263a tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:33.963979 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3a52413d-84ec-45b5-8f54-dae18318263a tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:33.964497 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3a52413d-84ec-45b5-8f54-dae18318263a tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:34.141441 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-3a52413d-84ec-45b5-8f54-dae18318263a tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f2e974ab5aa48dfb4b00f810dcedb63', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd31be8475a6a49d6bfe8132dd050c6c0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:22:34.148380 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-ccdb36b4-1a4f-4a18-a436-e05ddc8eade8 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] 10.222.0.16 "POST /compute/v2.1/servers/f114e124-2903-4f30-b055-9f493e3e9e6c/action" status: 202 len: 0 microversion: 2.1 time: 0.245541 Jul 16 21:22:34.150362 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1543/3134] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 21:22:33 2022] POST /compute/v2.1/servers/f114e124-2903-4f30-b055-9f493e3e9e6c/action => generated 0 bytes in 248 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Jul 16 21:22:34.158323 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-3c1c87a6-7361-4031-b74a-b631b17e2e5c tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:22:34.159079 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-3c1c87a6-7361-4031-b74a-b631b17e2e5c tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] [instance: f114e124-2903-4f30-b055-9f493e3e9e6c] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:22:34.171166 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3c1c87a6-7361-4031-b74a-b631b17e2e5c tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:34.176346 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3c1c87a6-7361-4031-b74a-b631b17e2e5c tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:34.177187 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3c1c87a6-7361-4031-b74a-b631b17e2e5c tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:34.177634 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3c1c87a6-7361-4031-b74a-b631b17e2e5c tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:34.189241 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-3a52413d-84ec-45b5-8f54-dae18318263a tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "GET /compute/v2.1/servers/32df0911-cfa6-4410-840a-4098a9fef0b4" status: 200 len: 1314 microversion: 2.1 time: 0.239970 Jul 16 21:22:34.191173 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1592/3135] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:22:33 2022] GET /compute/v2.1/servers/32df0911-cfa6-4410-840a-4098a9fef0b4 => generated 1314 bytes in 246 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:22:34.199443 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=97,nova_cell1:INSERT=2,nova_cell1:UPDATE=1 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:22:34.408013 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-3c1c87a6-7361-4031-b74a-b631b17e2e5c tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bba77f6ef42547f1aa8480e27868bd4a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '22ca92a86e6c40d59071e58ce4085568', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:22:34.418391 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-3c1c87a6-7361-4031-b74a-b631b17e2e5c tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] 10.222.0.16 "GET /compute/v2.1/servers/f114e124-2903-4f30-b055-9f493e3e9e6c" status: 200 len: 1711 microversion: 2.1 time: 0.264440 Jul 16 21:22:34.419238 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1544/3136] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:22:34 2022] GET /compute/v2.1/servers/f114e124-2903-4f30-b055-9f493e3e9e6c => generated 1711 bytes in 266 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:22:34.960804 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-0e3ef6f0-b92a-4f8b-ab3e-99a7af244cee tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:22:34.962336 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-0e3ef6f0-b92a-4f8b-ab3e-99a7af244cee tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] [instance: 583e0c15-3053-429d-81bf-95a4d1f99cbf] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:22:34.972425 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0e3ef6f0-b92a-4f8b-ab3e-99a7af244cee tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:34.972970 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0e3ef6f0-b92a-4f8b-ab3e-99a7af244cee tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:34.973749 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0e3ef6f0-b92a-4f8b-ab3e-99a7af244cee tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:34.974185 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0e3ef6f0-b92a-4f8b-ab3e-99a7af244cee tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:35.120605 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-0e3ef6f0-b92a-4f8b-ab3e-99a7af244cee tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'aaf2292818b749f0ae43c66bca929be9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '65644a8df5d84df3b35c894560e0a5c4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:22:35.132513 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-0e3ef6f0-b92a-4f8b-ab3e-99a7af244cee tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] 10.222.0.16 "GET /compute/v2.1/servers/583e0c15-3053-429d-81bf-95a4d1f99cbf" status: 200 len: 1524 microversion: 2.1 time: 0.178321 Jul 16 21:22:35.133871 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1593/3137] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:22:34 2022] GET /compute/v2.1/servers/583e0c15-3053-429d-81bf-95a4d1f99cbf => generated 1524 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:22:35.207843 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-63f7b09e-d7d5-4845-b9c8-0ac237ce4c94 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:22:35.208642 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-63f7b09e-d7d5-4845-b9c8-0ac237ce4c94 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 32df0911-cfa6-4410-840a-4098a9fef0b4] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:22:35.217301 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-63f7b09e-d7d5-4845-b9c8-0ac237ce4c94 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:35.218211 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-63f7b09e-d7d5-4845-b9c8-0ac237ce4c94 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:35.222556 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-63f7b09e-d7d5-4845-b9c8-0ac237ce4c94 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:35.222556 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-63f7b09e-d7d5-4845-b9c8-0ac237ce4c94 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:35.371311 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-63f7b09e-d7d5-4845-b9c8-0ac237ce4c94 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f2e974ab5aa48dfb4b00f810dcedb63', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd31be8475a6a49d6bfe8132dd050c6c0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:22:35.379238 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-63f7b09e-d7d5-4845-b9c8-0ac237ce4c94 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "GET /compute/v2.1/servers/32df0911-cfa6-4410-840a-4098a9fef0b4" status: 200 len: 1314 microversion: 2.1 time: 0.175983 Jul 16 21:22:35.379820 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1545/3138] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:22:35 2022] GET /compute/v2.1/servers/32df0911-cfa6-4410-840a-4098a9fef0b4 => generated 1314 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:22:35.437139 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-c2e472b1-ec3c-4ab6-ace0-d603240167af tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:22:35.437789 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-c2e472b1-ec3c-4ab6-ace0-d603240167af tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] [instance: f114e124-2903-4f30-b055-9f493e3e9e6c] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:22:35.444061 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c2e472b1-ec3c-4ab6-ace0-d603240167af tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:35.444579 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c2e472b1-ec3c-4ab6-ace0-d603240167af tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:35.445350 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c2e472b1-ec3c-4ab6-ace0-d603240167af tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:35.445814 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c2e472b1-ec3c-4ab6-ace0-d603240167af tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:35.605631 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-c2e472b1-ec3c-4ab6-ace0-d603240167af tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bba77f6ef42547f1aa8480e27868bd4a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '22ca92a86e6c40d59071e58ce4085568', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:22:35.614974 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-c2e472b1-ec3c-4ab6-ace0-d603240167af tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] 10.222.0.16 "GET /compute/v2.1/servers/f114e124-2903-4f30-b055-9f493e3e9e6c" status: 200 len: 1711 microversion: 2.1 time: 0.182061 Jul 16 21:22:35.625910 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1594/3139] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:22:35 2022] GET /compute/v2.1/servers/f114e124-2903-4f30-b055-9f493e3e9e6c => generated 1711 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:22:36.149946 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-3af1f171-b295-4582-a4d9-4ae54671856e tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:22:36.150565 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-3af1f171-b295-4582-a4d9-4ae54671856e tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] [instance: 583e0c15-3053-429d-81bf-95a4d1f99cbf] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:22:36.157117 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3af1f171-b295-4582-a4d9-4ae54671856e tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:36.157633 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3af1f171-b295-4582-a4d9-4ae54671856e tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:36.158346 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3af1f171-b295-4582-a4d9-4ae54671856e tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:36.158789 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3af1f171-b295-4582-a4d9-4ae54671856e tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:36.318818 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-3af1f171-b295-4582-a4d9-4ae54671856e tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'aaf2292818b749f0ae43c66bca929be9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '65644a8df5d84df3b35c894560e0a5c4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:22:36.326425 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-3af1f171-b295-4582-a4d9-4ae54671856e tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] 10.222.0.16 "GET /compute/v2.1/servers/583e0c15-3053-429d-81bf-95a4d1f99cbf" status: 200 len: 1524 microversion: 2.1 time: 0.180699 Jul 16 21:22:36.327346 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1546/3140] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:22:36 2022] GET /compute/v2.1/servers/583e0c15-3053-429d-81bf-95a4d1f99cbf => generated 1524 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:22:36.361807 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-2382c154-d99f-41c1-987b-78f01011c742 req-a4443e0c-bc88-4847-92db-347d59b94321 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "32df0911-cfa6-4410-840a-4098a9fef0b4", "name": "network-vif-plugged", "status": "completed", "tag": "37e2d383-525b-40f5-ae01-66bf83a6301d"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:22:36.367605 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_api:SELECT=96,nova_api:INSERT=3,nova_api:UPDATE=1 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:22:36.372679 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-2382c154-d99f-41c1-987b-78f01011c742 req-a4443e0c-bc88-4847-92db-347d59b94321 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:36.373166 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-2382c154-d99f-41c1-987b-78f01011c742 req-a4443e0c-bc88-4847-92db-347d59b94321 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:36.385248 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-2382c154-d99f-41c1-987b-78f01011c742 req-a4443e0c-bc88-4847-92db-347d59b94321 service nova] Creating event network-vif-plugged:37e2d383-525b-40f5-ae01-66bf83a6301d for instance 32df0911-cfa6-4410-840a-4098a9fef0b4 on nch1-master-1 Jul 16 21:22:36.390400 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-2382c154-d99f-41c1-987b-78f01011c742 req-a4443e0c-bc88-4847-92db-347d59b94321 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.033254 Jul 16 21:22:36.391522 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1595/3141] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:22:36 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:22:36.396790 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-0fb52df0-b036-43f5-ad24-1a282e2c8214 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:22:36.397340 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-0fb52df0-b036-43f5-ad24-1a282e2c8214 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 32df0911-cfa6-4410-840a-4098a9fef0b4] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:22:36.403904 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0fb52df0-b036-43f5-ad24-1a282e2c8214 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:36.404553 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0fb52df0-b036-43f5-ad24-1a282e2c8214 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:36.405731 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0fb52df0-b036-43f5-ad24-1a282e2c8214 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:36.405861 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0fb52df0-b036-43f5-ad24-1a282e2c8214 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:36.633712 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-8787e935-2309-4ea5-a5c2-a22172f5f3da tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:22:36.634670 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-8787e935-2309-4ea5-a5c2-a22172f5f3da tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] [instance: f114e124-2903-4f30-b055-9f493e3e9e6c] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:22:36.641798 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8787e935-2309-4ea5-a5c2-a22172f5f3da tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:36.642642 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8787e935-2309-4ea5-a5c2-a22172f5f3da tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:36.643773 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8787e935-2309-4ea5-a5c2-a22172f5f3da tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:36.644406 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8787e935-2309-4ea5-a5c2-a22172f5f3da tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:36.668168 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-0fb52df0-b036-43f5-ad24-1a282e2c8214 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f2e974ab5aa48dfb4b00f810dcedb63', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd31be8475a6a49d6bfe8132dd050c6c0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:22:36.679643 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-0fb52df0-b036-43f5-ad24-1a282e2c8214 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "GET /compute/v2.1/servers/32df0911-cfa6-4410-840a-4098a9fef0b4" status: 200 len: 1314 microversion: 2.1 time: 0.286705 Jul 16 21:22:36.680910 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1547/3142] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:22:36 2022] GET /compute/v2.1/servers/32df0911-cfa6-4410-840a-4098a9fef0b4 => generated 1314 bytes in 288 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:22:36.832347 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-8787e935-2309-4ea5-a5c2-a22172f5f3da tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bba77f6ef42547f1aa8480e27868bd4a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '22ca92a86e6c40d59071e58ce4085568', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:22:36.840688 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-8787e935-2309-4ea5-a5c2-a22172f5f3da tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] 10.222.0.16 "GET /compute/v2.1/servers/f114e124-2903-4f30-b055-9f493e3e9e6c" status: 200 len: 1715 microversion: 2.1 time: 0.211717 Jul 16 21:22:36.841349 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1596/3143] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:22:36 2022] GET /compute/v2.1/servers/f114e124-2903-4f30-b055-9f493e3e9e6c => generated 1715 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:22:37.344200 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-fb4407d6-db00-4fc7-8eb2-e504b3894953 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:22:37.344837 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-fb4407d6-db00-4fc7-8eb2-e504b3894953 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] [instance: 583e0c15-3053-429d-81bf-95a4d1f99cbf] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:22:37.355358 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fb4407d6-db00-4fc7-8eb2-e504b3894953 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:37.355823 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fb4407d6-db00-4fc7-8eb2-e504b3894953 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:37.356604 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fb4407d6-db00-4fc7-8eb2-e504b3894953 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:37.357050 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fb4407d6-db00-4fc7-8eb2-e504b3894953 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:37.609943 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-fb4407d6-db00-4fc7-8eb2-e504b3894953 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'aaf2292818b749f0ae43c66bca929be9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '65644a8df5d84df3b35c894560e0a5c4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:22:37.619283 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-fb4407d6-db00-4fc7-8eb2-e504b3894953 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] 10.222.0.16 "GET /compute/v2.1/servers/583e0c15-3053-429d-81bf-95a4d1f99cbf" status: 200 len: 1524 microversion: 2.1 time: 0.279416 Jul 16 21:22:37.619877 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1548/3144] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:22:37 2022] GET /compute/v2.1/servers/583e0c15-3053-429d-81bf-95a4d1f99cbf => generated 1524 bytes in 281 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:22:37.699696 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-10e0a458-48e0-46f6-bb6a-6b20febe117f tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:22:37.700325 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-10e0a458-48e0-46f6-bb6a-6b20febe117f tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 32df0911-cfa6-4410-840a-4098a9fef0b4] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:22:37.708361 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-10e0a458-48e0-46f6-bb6a-6b20febe117f tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:37.708885 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-10e0a458-48e0-46f6-bb6a-6b20febe117f tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:37.709656 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-10e0a458-48e0-46f6-bb6a-6b20febe117f tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:37.710158 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-10e0a458-48e0-46f6-bb6a-6b20febe117f tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:37.859617 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-10e0a458-48e0-46f6-bb6a-6b20febe117f tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f2e974ab5aa48dfb4b00f810dcedb63', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd31be8475a6a49d6bfe8132dd050c6c0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:22:37.868088 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-10e0a458-48e0-46f6-bb6a-6b20febe117f tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "GET /compute/v2.1/servers/32df0911-cfa6-4410-840a-4098a9fef0b4" status: 200 len: 1314 microversion: 2.1 time: 0.173650 Jul 16 21:22:37.871853 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1597/3145] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:22:37 2022] GET /compute/v2.1/servers/32df0911-cfa6-4410-840a-4098a9fef0b4 => generated 1314 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:22:38.637164 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-e62dedd5-dc56-4ad0-bdc5-fabc847f2c69 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:22:38.637791 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-e62dedd5-dc56-4ad0-bdc5-fabc847f2c69 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] [instance: 583e0c15-3053-429d-81bf-95a4d1f99cbf] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:22:38.645724 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e62dedd5-dc56-4ad0-bdc5-fabc847f2c69 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:38.646392 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e62dedd5-dc56-4ad0-bdc5-fabc847f2c69 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:38.647193 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e62dedd5-dc56-4ad0-bdc5-fabc847f2c69 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:38.647823 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e62dedd5-dc56-4ad0-bdc5-fabc847f2c69 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:38.906937 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-d34034d0-a878-4afc-97ff-f7f5f65fe2ba tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:22:38.907691 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-d34034d0-a878-4afc-97ff-f7f5f65fe2ba tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 32df0911-cfa6-4410-840a-4098a9fef0b4] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:22:38.924725 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d34034d0-a878-4afc-97ff-f7f5f65fe2ba tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:38.926819 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d34034d0-a878-4afc-97ff-f7f5f65fe2ba tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:38.928870 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d34034d0-a878-4afc-97ff-f7f5f65fe2ba tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:38.929351 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d34034d0-a878-4afc-97ff-f7f5f65fe2ba tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:39.162363 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-e62dedd5-dc56-4ad0-bdc5-fabc847f2c69 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'aaf2292818b749f0ae43c66bca929be9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '65644a8df5d84df3b35c894560e0a5c4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:22:39.173228 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-e62dedd5-dc56-4ad0-bdc5-fabc847f2c69 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] 10.222.0.16 "GET /compute/v2.1/servers/583e0c15-3053-429d-81bf-95a4d1f99cbf" status: 200 len: 1524 microversion: 2.1 time: 0.539912 Jul 16 21:22:39.175333 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1549/3146] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:22:38 2022] GET /compute/v2.1/servers/583e0c15-3053-429d-81bf-95a4d1f99cbf => generated 1524 bytes in 542 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:22:39.291079 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-d34034d0-a878-4afc-97ff-f7f5f65fe2ba tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f2e974ab5aa48dfb4b00f810dcedb63', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd31be8475a6a49d6bfe8132dd050c6c0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:22:39.303082 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-d34034d0-a878-4afc-97ff-f7f5f65fe2ba tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "GET /compute/v2.1/servers/32df0911-cfa6-4410-840a-4098a9fef0b4" status: 200 len: 1502 microversion: 2.1 time: 0.415092 Jul 16 21:22:39.303082 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1598/3147] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:22:38 2022] GET /compute/v2.1/servers/32df0911-cfa6-4410-840a-4098a9fef0b4 => generated 1502 bytes in 416 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:22:40.192323 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-2ec8cbef-bfcd-4937-b773-55f18e6589a7 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:22:40.192927 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-2ec8cbef-bfcd-4937-b773-55f18e6589a7 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] [instance: 583e0c15-3053-429d-81bf-95a4d1f99cbf] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:22:40.200298 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2ec8cbef-bfcd-4937-b773-55f18e6589a7 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:40.200984 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2ec8cbef-bfcd-4937-b773-55f18e6589a7 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:40.202171 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2ec8cbef-bfcd-4937-b773-55f18e6589a7 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:40.204001 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2ec8cbef-bfcd-4937-b773-55f18e6589a7 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:40.606786 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-2ec8cbef-bfcd-4937-b773-55f18e6589a7 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'aaf2292818b749f0ae43c66bca929be9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '65644a8df5d84df3b35c894560e0a5c4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:22:40.615627 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-2ec8cbef-bfcd-4937-b773-55f18e6589a7 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] 10.222.0.16 "GET /compute/v2.1/servers/583e0c15-3053-429d-81bf-95a4d1f99cbf" status: 200 len: 1504 microversion: 2.1 time: 0.429170 Jul 16 21:22:40.628793 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1550/3148] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:22:40 2022] GET /compute/v2.1/servers/583e0c15-3053-429d-81bf-95a4d1f99cbf => generated 1504 bytes in 443 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:22:40.635238 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-131ae4de-3484-4275-9a09-0332839b287c tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Action: 'create', calling method: .version_select at 0x7fdd9c4595e0>, body: {"security_group": {"name": "tempest-ServerRescueTestJSONUnderV235-securitygroup-205358439", "description": "tempest-description-741934429"}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:22:41.184691 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-131ae4de-3484-4275-9a09-0332839b287c tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] 10.222.0.16 "POST /compute/v2.1/os-security-groups" status: 200 len: 249 microversion: 2.1 time: 0.554496 Jul 16 21:22:41.189665 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1599/3149] 10.222.0.16 () {60 vars in 1179 bytes} [Sat Jul 16 21:22:40 2022] POST /compute/v2.1/os-security-groups => generated 249 bytes in 556 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 16 21:22:41.197423 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-7777f307-8f1b-4099-abcb-caed718dc8d3 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Action: 'action', calling method: >, body: {"addSecurityGroup": {"name": "tempest-ServerRescueTestJSONUnderV235-securitygroup-205358439"}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:22:41.197991 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-7777f307-8f1b-4099-abcb-caed718dc8d3 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] [instance: 583e0c15-3053-429d-81bf-95a4d1f99cbf] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:22:41.207255 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7777f307-8f1b-4099-abcb-caed718dc8d3 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:41.207741 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7777f307-8f1b-4099-abcb-caed718dc8d3 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:41.208939 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7777f307-8f1b-4099-abcb-caed718dc8d3 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:41.209394 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7777f307-8f1b-4099-abcb-caed718dc8d3 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:41.491527 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.network.security_group_api [None req-7777f307-8f1b-4099-abcb-caed718dc8d3 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Adding security group 0e1e4131-16d9-409c-927c-9257d8d5843b to port 40900dd3-d3df-4e26-80dd-0776ab4b1d20 Jul 16 21:22:42.269759 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-5f21b11d-0e59-4209-be52-c5280c5d8111 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "71861a5b-1ca1-4935-9349-fbb30a4fc8bc"}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:22:42.271541 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-5f21b11d-0e59-4209-be52-c5280c5d8111 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 32df0911-cfa6-4410-840a-4098a9fef0b4] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:22:42.280512 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5f21b11d-0e59-4209-be52-c5280c5d8111 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:42.281735 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5f21b11d-0e59-4209-be52-c5280c5d8111 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:42.283067 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5f21b11d-0e59-4209-be52-c5280c5d8111 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:42.283679 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5f21b11d-0e59-4209-be52-c5280c5d8111 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:42.530328 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-7777f307-8f1b-4099-abcb-caed718dc8d3 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] 10.222.0.16 "POST /compute/v2.1/servers/583e0c15-3053-429d-81bf-95a4d1f99cbf/action" status: 202 len: 0 microversion: 2.1 time: 1.337408 Jul 16 21:22:42.531190 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1551/3150] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 21:22:41 2022] POST /compute/v2.1/servers/583e0c15-3053-429d-81bf-95a4d1f99cbf/action => generated 0 bytes in 1339 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 21:22:42.544056 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-13188ae0-0b84-4b06-99e4-e5c4ad27b3a2 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Action: 'action', calling method: >, body: {"removeSecurityGroup": {"name": "tempest-ServerRescueTestJSONUnderV235-securitygroup-205358439"}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:22:42.549924 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-13188ae0-0b84-4b06-99e4-e5c4ad27b3a2 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] [instance: 583e0c15-3053-429d-81bf-95a4d1f99cbf] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:22:42.556748 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-13188ae0-0b84-4b06-99e4-e5c4ad27b3a2 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:42.557508 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-13188ae0-0b84-4b06-99e4-e5c4ad27b3a2 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:42.558526 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-13188ae0-0b84-4b06-99e4-e5c4ad27b3a2 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:42.559189 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-13188ae0-0b84-4b06-99e4-e5c4ad27b3a2 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:42.818211 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.network.security_group_api [None req-13188ae0-0b84-4b06-99e4-e5c4ad27b3a2 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Removing security group 0e1e4131-16d9-409c-927c-9257d8d5843b from port 40900dd3-d3df-4e26-80dd-0776ab4b1d20 Jul 16 21:22:43.390459 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-5f21b11d-0e59-4209-be52-c5280c5d8111 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "POST /compute/v2.1/servers/32df0911-cfa6-4410-840a-4098a9fef0b4/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 1.126777 Jul 16 21:22:43.393197 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1600/3151] 10.222.0.16 () {60 vars in 1322 bytes} [Sat Jul 16 21:22:42 2022] POST /compute/v2.1/servers/32df0911-cfa6-4410-840a-4098a9fef0b4/os-volume_attachments => generated 194 bytes in 1130 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:22:43.425839 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-1d368140-34b2-4287-ab77-488a0e61c7e5 req-9bd36a27-a51b-4def-a045-37f0e67aa43d service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "583e0c15-3053-429d-81bf-95a4d1f99cbf", "tag": "40900dd3-d3df-4e26-80dd-0776ab4b1d20"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:22:43.445278 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-1d368140-34b2-4287-ab77-488a0e61c7e5 req-9bd36a27-a51b-4def-a045-37f0e67aa43d service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:43.445737 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-1d368140-34b2-4287-ab77-488a0e61c7e5 req-9bd36a27-a51b-4def-a045-37f0e67aa43d service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:43.467898 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-1d368140-34b2-4287-ab77-488a0e61c7e5 req-9bd36a27-a51b-4def-a045-37f0e67aa43d service nova] Creating event network-changed:40900dd3-d3df-4e26-80dd-0776ab4b1d20 for instance 583e0c15-3053-429d-81bf-95a4d1f99cbf on nch1-master-1 Jul 16 21:22:43.473835 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-1d368140-34b2-4287-ab77-488a0e61c7e5 req-9bd36a27-a51b-4def-a045-37f0e67aa43d service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.077099 Jul 16 21:22:43.483067 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1601/3152] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:22:43 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:22:43.812404 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-13188ae0-0b84-4b06-99e4-e5c4ad27b3a2 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] 10.222.0.16 "POST /compute/v2.1/servers/583e0c15-3053-429d-81bf-95a4d1f99cbf/action" status: 202 len: 0 microversion: 2.1 time: 1.275769 Jul 16 21:22:43.821424 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1552/3153] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 21:22:42 2022] POST /compute/v2.1/servers/583e0c15-3053-429d-81bf-95a4d1f99cbf/action => generated 0 bytes in 1286 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 21:22:43.827344 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-121b4257-b914-4c2e-bcc5-e046ef31e6c5 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Action: 'create', calling method: .version_select at 0x7fdd9c512940>, body: {"pool": null} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:22:44.365042 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-b856901a-1771-4443-be12-123a510773d9 tempest-ServerAddressesNegativeTestJSON-1415290150 tempest-ServerAddressesNegativeTestJSON-1415290150-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerAddressesNegativeTestJSON-server-1768914926", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452", "networks": [{"uuid": "b452fc21-3bff-42cf-b524-7595174c6676"}]}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:22:44.427152 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.network.neutron [None req-b856901a-1771-4443-be12-123a510773d9 tempest-ServerAddressesNegativeTestJSON-1415290150 tempest-ServerAddressesNegativeTestJSON-1415290150-project] validate_networks() for [('b452fc21-3bff-42cf-b524-7595174c6676', None, None, None, None, None)] {{(pid=138681) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 16 21:22:45.874431 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-b856901a-1771-4443-be12-123a510773d9 tempest-ServerAddressesNegativeTestJSON-1415290150 tempest-ServerAddressesNegativeTestJSON-1415290150-project] Flavor limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 16 21:22:45.875771 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-b856901a-1771-4443-be12-123a510773d9 tempest-ServerAddressesNegativeTestJSON-1415290150 tempest-ServerAddressesNegativeTestJSON-1415290150-project] Image limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 16 21:22:45.876474 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-b856901a-1771-4443-be12-123a510773d9 tempest-ServerAddressesNegativeTestJSON-1415290150 tempest-ServerAddressesNegativeTestJSON-1415290150-project] Flavor pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 16 21:22:45.877031 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-b856901a-1771-4443-be12-123a510773d9 tempest-ServerAddressesNegativeTestJSON-1415290150 tempest-ServerAddressesNegativeTestJSON-1415290150-project] Image pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 16 21:22:45.877540 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-b856901a-1771-4443-be12-123a510773d9 tempest-ServerAddressesNegativeTestJSON-1415290150 tempest-ServerAddressesNegativeTestJSON-1415290150-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 16 21:22:45.900755 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b856901a-1771-4443-be12-123a510773d9 tempest-ServerAddressesNegativeTestJSON-1415290150 tempest-ServerAddressesNegativeTestJSON-1415290150-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:45.904869 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b856901a-1771-4443-be12-123a510773d9 tempest-ServerAddressesNegativeTestJSON-1415290150 tempest-ServerAddressesNegativeTestJSON-1415290150-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:45.906862 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b856901a-1771-4443-be12-123a510773d9 tempest-ServerAddressesNegativeTestJSON-1415290150 tempest-ServerAddressesNegativeTestJSON-1415290150-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:45.909692 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b856901a-1771-4443-be12-123a510773d9 tempest-ServerAddressesNegativeTestJSON-1415290150 tempest-ServerAddressesNegativeTestJSON-1415290150-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:45.943426 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-b856901a-1771-4443-be12-123a510773d9 tempest-ServerAddressesNegativeTestJSON-1415290150 tempest-ServerAddressesNegativeTestJSON-1415290150-project] Getting quotas for project 7c0ebaf2e3714e42b3bda726e2133522. Resources: {'ram', 'instances', 'cores'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:22:45.952257 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-b856901a-1771-4443-be12-123a510773d9 tempest-ServerAddressesNegativeTestJSON-1415290150 tempest-ServerAddressesNegativeTestJSON-1415290150-project] Getting quotas for user 3ab7326ef48042fe961a7eed5c46db24 and project 7c0ebaf2e3714e42b3bda726e2133522. Resources: {'ram', 'instances', 'cores'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:22:45.973228 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-b856901a-1771-4443-be12-123a510773d9 tempest-ServerAddressesNegativeTestJSON-1415290150 tempest-ServerAddressesNegativeTestJSON-1415290150-project] Going to run 1 instances... {{(pid=138681) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 16 21:22:45.976347 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-b856901a-1771-4443-be12-123a510773d9 tempest-ServerAddressesNegativeTestJSON-1415290150 tempest-ServerAddressesNegativeTestJSON-1415290150-project] [instance: 43a4dc6f-5d4d-45d7-bc69-0c754c2ec27a] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138681) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 16 21:22:46.184248 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-b856901a-1771-4443-be12-123a510773d9 tempest-ServerAddressesNegativeTestJSON-1415290150 tempest-ServerAddressesNegativeTestJSON-1415290150-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 1.994407 Jul 16 21:22:46.190253 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1553/3154] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 21:22:44 2022] POST /compute/v2.1/servers => generated 380 bytes in 2003 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Jul 16 21:22:46.196013 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-f216bc91-c134-4e9b-a1af-6c1a24db9135 req-4a4d836d-9723-45ef-b6e3-e5787787d196 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "583e0c15-3053-429d-81bf-95a4d1f99cbf", "tag": "40900dd3-d3df-4e26-80dd-0776ab4b1d20"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:22:46.210971 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-f216bc91-c134-4e9b-a1af-6c1a24db9135 req-4a4d836d-9723-45ef-b6e3-e5787787d196 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:46.210971 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-f216bc91-c134-4e9b-a1af-6c1a24db9135 req-4a4d836d-9723-45ef-b6e3-e5787787d196 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:46.231395 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-f216bc91-c134-4e9b-a1af-6c1a24db9135 req-4a4d836d-9723-45ef-b6e3-e5787787d196 service nova] Creating event network-changed:40900dd3-d3df-4e26-80dd-0776ab4b1d20 for instance 583e0c15-3053-429d-81bf-95a4d1f99cbf on nch1-master-1 Jul 16 21:22:46.240168 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-f216bc91-c134-4e9b-a1af-6c1a24db9135 req-4a4d836d-9723-45ef-b6e3-e5787787d196 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.048263 Jul 16 21:22:46.241438 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1554/3155] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:22:46 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:22:46.246884 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-d837f38d-f790-4b5c-9963-f86753f3e01d tempest-ServerAddressesNegativeTestJSON-1415290150 tempest-ServerAddressesNegativeTestJSON-1415290150-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:22:46.247639 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-d837f38d-f790-4b5c-9963-f86753f3e01d tempest-ServerAddressesNegativeTestJSON-1415290150 tempest-ServerAddressesNegativeTestJSON-1415290150-project] [instance: 43a4dc6f-5d4d-45d7-bc69-0c754c2ec27a] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:22:46.433998 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-d837f38d-f790-4b5c-9963-f86753f3e01d tempest-ServerAddressesNegativeTestJSON-1415290150 tempest-ServerAddressesNegativeTestJSON-1415290150-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ab7326ef48042fe961a7eed5c46db24', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c0ebaf2e3714e42b3bda726e2133522', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:22:46.444384 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-d837f38d-f790-4b5c-9963-f86753f3e01d tempest-ServerAddressesNegativeTestJSON-1415290150 tempest-ServerAddressesNegativeTestJSON-1415290150-project] 10.222.0.16 "GET /compute/v2.1/servers/43a4dc6f-5d4d-45d7-bc69-0c754c2ec27a" status: 200 len: 1225 microversion: 2.1 time: 0.201116 Jul 16 21:22:46.446738 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1555/3156] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:22:46 2022] GET /compute/v2.1/servers/43a4dc6f-5d4d-45d7-bc69-0c754c2ec27a => generated 1225 bytes in 205 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:22:47.464540 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-3d6105c3-cd88-4bea-9704-4cb2b64773ce tempest-ServerAddressesNegativeTestJSON-1415290150 tempest-ServerAddressesNegativeTestJSON-1415290150-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:22:47.468863 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-3d6105c3-cd88-4bea-9704-4cb2b64773ce tempest-ServerAddressesNegativeTestJSON-1415290150 tempest-ServerAddressesNegativeTestJSON-1415290150-project] [instance: 43a4dc6f-5d4d-45d7-bc69-0c754c2ec27a] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:22:47.553676 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-3d6105c3-cd88-4bea-9704-4cb2b64773ce tempest-ServerAddressesNegativeTestJSON-1415290150 tempest-ServerAddressesNegativeTestJSON-1415290150-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ab7326ef48042fe961a7eed5c46db24', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c0ebaf2e3714e42b3bda726e2133522', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:22:47.566166 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-3d6105c3-cd88-4bea-9704-4cb2b64773ce tempest-ServerAddressesNegativeTestJSON-1415290150 tempest-ServerAddressesNegativeTestJSON-1415290150-project] 10.222.0.16 "GET /compute/v2.1/servers/43a4dc6f-5d4d-45d7-bc69-0c754c2ec27a" status: 200 len: 1225 microversion: 2.1 time: 0.106554 Jul 16 21:22:47.568308 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1556/3157] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:22:47 2022] GET /compute/v2.1/servers/43a4dc6f-5d4d-45d7-bc69-0c754c2ec27a => generated 1225 bytes in 111 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:22:47.870398 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-85997114-1240-4d1e-93f4-454bd7980f44 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:22:47.871378 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-85997114-1240-4d1e-93f4-454bd7980f44 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 32df0911-cfa6-4410-840a-4098a9fef0b4] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:22:47.885923 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-85997114-1240-4d1e-93f4-454bd7980f44 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:47.886607 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-85997114-1240-4d1e-93f4-454bd7980f44 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:47.891853 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-85997114-1240-4d1e-93f4-454bd7980f44 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:47.893277 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-85997114-1240-4d1e-93f4-454bd7980f44 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:47.950483 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-85997114-1240-4d1e-93f4-454bd7980f44 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 32df0911-cfa6-4410-840a-4098a9fef0b4] Going to try to terminate instance {{(pid=138681) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 16 21:22:48.024508 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-121b4257-b914-4c2e-bcc5-e046ef31e6c5 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] 10.222.0.16 "POST /compute/v2.1/os-floating-ips" status: 200 len: 142 microversion: 2.1 time: 4.201287 Jul 16 21:22:48.031040 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1602/3158] 10.222.0.16 () {60 vars in 1169 bytes} [Sat Jul 16 21:22:43 2022] POST /compute/v2.1/os-floating-ips => generated 142 bytes in 4208 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:22:48.044002 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-724f599a-4736-4fc0-ac65-97a9b7c149b2 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Action: 'action', calling method: , body: {"addFloatingIp": {"address": "172.35.1.247"}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:22:48.049666 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-724f599a-4736-4fc0-ac65-97a9b7c149b2 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] [instance: 583e0c15-3053-429d-81bf-95a4d1f99cbf] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:22:48.063406 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-724f599a-4736-4fc0-ac65-97a9b7c149b2 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:48.064010 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-724f599a-4736-4fc0-ac65-97a9b7c149b2 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:48.064824 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-724f599a-4736-4fc0-ac65-97a9b7c149b2 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:48.067116 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-724f599a-4736-4fc0-ac65-97a9b7c149b2 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:48.136257 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-724f599a-4736-4fc0-ac65-97a9b7c149b2 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Acquired lock "refresh_cache-583e0c15-3053-429d-81bf-95a4d1f99cbf" {{(pid=138680) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:294}} Jul 16 21:22:48.533911 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-85997114-1240-4d1e-93f4-454bd7980f44 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "DELETE /compute/v2.1/servers/32df0911-cfa6-4410-840a-4098a9fef0b4" status: 204 len: 0 microversion: 2.1 time: 0.667401 Jul 16 21:22:48.535161 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1557/3159] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:22:47 2022] DELETE /compute/v2.1/servers/32df0911-cfa6-4410-840a-4098a9fef0b4 => generated 0 bytes in 670 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 16 21:22:48.543931 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-8d22da14-f239-4273-93ee-4467582878b2 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:22:48.545111 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-8d22da14-f239-4273-93ee-4467582878b2 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 32df0911-cfa6-4410-840a-4098a9fef0b4] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:22:48.552698 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8d22da14-f239-4273-93ee-4467582878b2 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:48.553900 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8d22da14-f239-4273-93ee-4467582878b2 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:48.555455 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8d22da14-f239-4273-93ee-4467582878b2 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:48.556686 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8d22da14-f239-4273-93ee-4467582878b2 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:48.904425 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-8d22da14-f239-4273-93ee-4467582878b2 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f2e974ab5aa48dfb4b00f810dcedb63', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd31be8475a6a49d6bfe8132dd050c6c0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:22:48.920823 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-8d22da14-f239-4273-93ee-4467582878b2 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "GET /compute/v2.1/servers/32df0911-cfa6-4410-840a-4098a9fef0b4" status: 200 len: 1554 microversion: 2.1 time: 0.381124 Jul 16 21:22:48.922638 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1558/3160] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:22:48 2022] GET /compute/v2.1/servers/32df0911-cfa6-4410-840a-4098a9fef0b4 => generated 1554 bytes in 384 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:22:48.937341 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-be723b5b-a92d-4c8a-9db6-84e077e2a73b tempest-ServerAddressesNegativeTestJSON-1415290150 tempest-ServerAddressesNegativeTestJSON-1415290150-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:22:48.938258 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-be723b5b-a92d-4c8a-9db6-84e077e2a73b tempest-ServerAddressesNegativeTestJSON-1415290150 tempest-ServerAddressesNegativeTestJSON-1415290150-project] [instance: 43a4dc6f-5d4d-45d7-bc69-0c754c2ec27a] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:22:48.949824 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-be723b5b-a92d-4c8a-9db6-84e077e2a73b tempest-ServerAddressesNegativeTestJSON-1415290150 tempest-ServerAddressesNegativeTestJSON-1415290150-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:48.950198 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-be723b5b-a92d-4c8a-9db6-84e077e2a73b tempest-ServerAddressesNegativeTestJSON-1415290150 tempest-ServerAddressesNegativeTestJSON-1415290150-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:48.950989 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-be723b5b-a92d-4c8a-9db6-84e077e2a73b tempest-ServerAddressesNegativeTestJSON-1415290150 tempest-ServerAddressesNegativeTestJSON-1415290150-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:48.951606 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-be723b5b-a92d-4c8a-9db6-84e077e2a73b tempest-ServerAddressesNegativeTestJSON-1415290150 tempest-ServerAddressesNegativeTestJSON-1415290150-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:49.059495 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-be723b5b-a92d-4c8a-9db6-84e077e2a73b tempest-ServerAddressesNegativeTestJSON-1415290150 tempest-ServerAddressesNegativeTestJSON-1415290150-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ab7326ef48042fe961a7eed5c46db24', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c0ebaf2e3714e42b3bda726e2133522', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:22:49.070735 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-be723b5b-a92d-4c8a-9db6-84e077e2a73b tempest-ServerAddressesNegativeTestJSON-1415290150 tempest-ServerAddressesNegativeTestJSON-1415290150-project] 10.222.0.16 "GET /compute/v2.1/servers/43a4dc6f-5d4d-45d7-bc69-0c754c2ec27a" status: 200 len: 1277 microversion: 2.1 time: 0.146718 Jul 16 21:22:49.071816 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1559/3161] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:22:48 2022] GET /compute/v2.1/servers/43a4dc6f-5d4d-45d7-bc69-0c754c2ec27a => generated 1277 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:22:49.939335 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-1c250656-6279-44a6-8ae0-0ddd9ee1a5c7 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:22:49.940051 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-1c250656-6279-44a6-8ae0-0ddd9ee1a5c7 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 32df0911-cfa6-4410-840a-4098a9fef0b4] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:22:49.946994 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1c250656-6279-44a6-8ae0-0ddd9ee1a5c7 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:49.947503 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1c250656-6279-44a6-8ae0-0ddd9ee1a5c7 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:49.948489 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1c250656-6279-44a6-8ae0-0ddd9ee1a5c7 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:49.948935 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1c250656-6279-44a6-8ae0-0ddd9ee1a5c7 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:50.330193 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-1c250656-6279-44a6-8ae0-0ddd9ee1a5c7 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f2e974ab5aa48dfb4b00f810dcedb63', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd31be8475a6a49d6bfe8132dd050c6c0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:22:50.344928 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-1c250656-6279-44a6-8ae0-0ddd9ee1a5c7 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "GET /compute/v2.1/servers/32df0911-cfa6-4410-840a-4098a9fef0b4" status: 200 len: 1554 microversion: 2.1 time: 0.408781 Jul 16 21:22:50.363733 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1560/3162] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:22:49 2022] GET /compute/v2.1/servers/32df0911-cfa6-4410-840a-4098a9fef0b4 => generated 1554 bytes in 429 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:22:50.369330 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.network.neutron [None req-724f599a-4736-4fc0-ac65-97a9b7c149b2 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] [instance: 583e0c15-3053-429d-81bf-95a4d1f99cbf] Building network info cache for instance {{(pid=138680) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1990}} Jul 16 21:22:50.372606 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-50fb607c-0ad8-442c-9a44-628d13b315fa tempest-ServerAddressesNegativeTestJSON-1415290150 tempest-ServerAddressesNegativeTestJSON-1415290150-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:22:50.373455 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-50fb607c-0ad8-442c-9a44-628d13b315fa tempest-ServerAddressesNegativeTestJSON-1415290150 tempest-ServerAddressesNegativeTestJSON-1415290150-project] [instance: 43a4dc6f-5d4d-45d7-bc69-0c754c2ec27a] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:22:50.380934 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-50fb607c-0ad8-442c-9a44-628d13b315fa tempest-ServerAddressesNegativeTestJSON-1415290150 tempest-ServerAddressesNegativeTestJSON-1415290150-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:50.381393 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-50fb607c-0ad8-442c-9a44-628d13b315fa tempest-ServerAddressesNegativeTestJSON-1415290150 tempest-ServerAddressesNegativeTestJSON-1415290150-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:50.382174 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-50fb607c-0ad8-442c-9a44-628d13b315fa tempest-ServerAddressesNegativeTestJSON-1415290150 tempest-ServerAddressesNegativeTestJSON-1415290150-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:50.382698 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-50fb607c-0ad8-442c-9a44-628d13b315fa tempest-ServerAddressesNegativeTestJSON-1415290150 tempest-ServerAddressesNegativeTestJSON-1415290150-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:50.507805 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-50fb607c-0ad8-442c-9a44-628d13b315fa tempest-ServerAddressesNegativeTestJSON-1415290150 tempest-ServerAddressesNegativeTestJSON-1415290150-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ab7326ef48042fe961a7eed5c46db24', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c0ebaf2e3714e42b3bda726e2133522', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:22:50.519750 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-50fb607c-0ad8-442c-9a44-628d13b315fa tempest-ServerAddressesNegativeTestJSON-1415290150 tempest-ServerAddressesNegativeTestJSON-1415290150-project] 10.222.0.16 "GET /compute/v2.1/servers/43a4dc6f-5d4d-45d7-bc69-0c754c2ec27a" status: 200 len: 1283 microversion: 2.1 time: 0.154779 Jul 16 21:22:50.520626 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1561/3163] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:22:50 2022] GET /compute/v2.1/servers/43a4dc6f-5d4d-45d7-bc69-0c754c2ec27a => generated 1283 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:22:50.538789 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-088ee560-edea-45d2-b596-63ee078e9000 req-6e7f1f5d-3aee-45ee-8ed3-c967e28433e5 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "583e0c15-3053-429d-81bf-95a4d1f99cbf", "tag": "40900dd3-d3df-4e26-80dd-0776ab4b1d20"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:22:50.563588 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-088ee560-edea-45d2-b596-63ee078e9000 req-6e7f1f5d-3aee-45ee-8ed3-c967e28433e5 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:50.564043 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-088ee560-edea-45d2-b596-63ee078e9000 req-6e7f1f5d-3aee-45ee-8ed3-c967e28433e5 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:50.603721 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-088ee560-edea-45d2-b596-63ee078e9000 req-6e7f1f5d-3aee-45ee-8ed3-c967e28433e5 service nova] Creating event network-changed:40900dd3-d3df-4e26-80dd-0776ab4b1d20 for instance 583e0c15-3053-429d-81bf-95a4d1f99cbf on nch1-master-1 Jul 16 21:22:50.610579 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-088ee560-edea-45d2-b596-63ee078e9000 req-6e7f1f5d-3aee-45ee-8ed3-c967e28433e5 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.087970 Jul 16 21:22:50.612422 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1562/3164] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:22:50 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:22:51.370415 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-9342ea32-f33a-4dce-b614-8099d2ee7f98 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:22:51.371475 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-9342ea32-f33a-4dce-b614-8099d2ee7f98 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 32df0911-cfa6-4410-840a-4098a9fef0b4] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:22:51.384136 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9342ea32-f33a-4dce-b614-8099d2ee7f98 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:51.384835 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9342ea32-f33a-4dce-b614-8099d2ee7f98 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:51.385884 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9342ea32-f33a-4dce-b614-8099d2ee7f98 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:51.386904 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9342ea32-f33a-4dce-b614-8099d2ee7f98 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:51.984938 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-9342ea32-f33a-4dce-b614-8099d2ee7f98 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f2e974ab5aa48dfb4b00f810dcedb63', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd31be8475a6a49d6bfe8132dd050c6c0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:22:52.002423 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-9342ea32-f33a-4dce-b614-8099d2ee7f98 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "GET /compute/v2.1/servers/32df0911-cfa6-4410-840a-4098a9fef0b4" status: 200 len: 1554 microversion: 2.1 time: 0.638391 Jul 16 21:22:52.003156 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1563/3165] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:22:51 2022] GET /compute/v2.1/servers/32df0911-cfa6-4410-840a-4098a9fef0b4 => generated 1554 bytes in 640 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:22:52.008379 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-3afb105a-b715-4fa0-84cf-0ec22b545321 tempest-ServerAddressesNegativeTestJSON-1415290150 tempest-ServerAddressesNegativeTestJSON-1415290150-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:22:52.009174 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-3afb105a-b715-4fa0-84cf-0ec22b545321 tempest-ServerAddressesNegativeTestJSON-1415290150 tempest-ServerAddressesNegativeTestJSON-1415290150-project] [instance: 43a4dc6f-5d4d-45d7-bc69-0c754c2ec27a] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:22:52.016461 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3afb105a-b715-4fa0-84cf-0ec22b545321 tempest-ServerAddressesNegativeTestJSON-1415290150 tempest-ServerAddressesNegativeTestJSON-1415290150-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:52.016912 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3afb105a-b715-4fa0-84cf-0ec22b545321 tempest-ServerAddressesNegativeTestJSON-1415290150 tempest-ServerAddressesNegativeTestJSON-1415290150-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:52.017883 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3afb105a-b715-4fa0-84cf-0ec22b545321 tempest-ServerAddressesNegativeTestJSON-1415290150 tempest-ServerAddressesNegativeTestJSON-1415290150-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:52.018321 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3afb105a-b715-4fa0-84cf-0ec22b545321 tempest-ServerAddressesNegativeTestJSON-1415290150 tempest-ServerAddressesNegativeTestJSON-1415290150-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:52.022976 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:22:52.137889 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-3afb105a-b715-4fa0-84cf-0ec22b545321 tempest-ServerAddressesNegativeTestJSON-1415290150 tempest-ServerAddressesNegativeTestJSON-1415290150-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ab7326ef48042fe961a7eed5c46db24', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c0ebaf2e3714e42b3bda726e2133522', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:22:52.147794 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-3afb105a-b715-4fa0-84cf-0ec22b545321 tempest-ServerAddressesNegativeTestJSON-1415290150 tempest-ServerAddressesNegativeTestJSON-1415290150-project] 10.222.0.16 "GET /compute/v2.1/servers/43a4dc6f-5d4d-45d7-bc69-0c754c2ec27a" status: 200 len: 1283 microversion: 2.1 time: 0.142686 Jul 16 21:22:52.158181 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1564/3166] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:22:52 2022] GET /compute/v2.1/servers/43a4dc6f-5d4d-45d7-bc69-0c754c2ec27a => generated 1283 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:22:52.362785 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-214c551d-1d2b-4a9d-a648-096564251b2f tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "f4c1e544-bdaf-4a33-bf91-08a306e68029"}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:22:52.365630 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-214c551d-1d2b-4a9d-a648-096564251b2f tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] [instance: fa075dcc-e175-449a-9914-d1ee19131f33] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:22:52.379272 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-214c551d-1d2b-4a9d-a648-096564251b2f tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:52.380653 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-214c551d-1d2b-4a9d-a648-096564251b2f tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:52.382476 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-214c551d-1d2b-4a9d-a648-096564251b2f tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:52.383774 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-214c551d-1d2b-4a9d-a648-096564251b2f tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:52.812486 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.network.neutron [None req-724f599a-4736-4fc0-ac65-97a9b7c149b2 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] [instance: 583e0c15-3053-429d-81bf-95a4d1f99cbf] Updating instance_info_cache with network_info: [{"id": "40900dd3-d3df-4e26-80dd-0776ab4b1d20", "address": "fa:16:3e:79:c2:70", "network": {"id": "728dbccf-f950-46d2-9c0b-1f3c0f2fd7ae", "bridge": null, "label": "tempest-ServerRescueTestJSONUnderV235-935378440-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.35.1.247", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "65644a8df5d84df3b35c894560e0a5c4", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false, "bound_drivers": {"0": "hyperv"}}, "devname": "tap40900dd3-d3", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=138680) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:117}} Jul 16 21:22:52.873201 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-724f599a-4736-4fc0-ac65-97a9b7c149b2 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Releasing lock "refresh_cache-583e0c15-3053-429d-81bf-95a4d1f99cbf" {{(pid=138680) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:312}} Jul 16 21:22:52.878153 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-724f599a-4736-4fc0-ac65-97a9b7c149b2 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] 10.222.0.16 "POST /compute/v2.1/servers/583e0c15-3053-429d-81bf-95a4d1f99cbf/action" status: 202 len: 0 microversion: 2.1 time: 4.840215 Jul 16 21:22:52.884836 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1603/3167] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 21:22:48 2022] POST /compute/v2.1/servers/583e0c15-3053-429d-81bf-95a4d1f99cbf/action => generated 0 bytes in 4851 msecs (HTTP/1.1 202) 9 headers in 369 bytes (1 switches on core 0) Jul 16 21:22:52.891500 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-f08709a8-6869-4c9e-9c58-f7e27b6ee0be req-018a572c-2748-4705-94bf-e7b010991101 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "32df0911-cfa6-4410-840a-4098a9fef0b4", "name": "network-vif-deleted", "tag": "37e2d383-525b-40f5-ae01-66bf83a6301d"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:22:52.906014 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-f08709a8-6869-4c9e-9c58-f7e27b6ee0be req-018a572c-2748-4705-94bf-e7b010991101 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:52.906014 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-f08709a8-6869-4c9e-9c58-f7e27b6ee0be req-018a572c-2748-4705-94bf-e7b010991101 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:52.917820 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-f08709a8-6869-4c9e-9c58-f7e27b6ee0be req-018a572c-2748-4705-94bf-e7b010991101 service nova] Creating event network-vif-deleted:37e2d383-525b-40f5-ae01-66bf83a6301d for instance 32df0911-cfa6-4410-840a-4098a9fef0b4 on nch1-master-1 Jul 16 21:22:52.925266 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-f08709a8-6869-4c9e-9c58-f7e27b6ee0be req-018a572c-2748-4705-94bf-e7b010991101 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.038457 Jul 16 21:22:52.931751 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1604/3168] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:22:52 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:22:52.940525 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-197ffcb1-86ff-439e-bc03-66b9f6fd6c65 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Action: 'action', calling method: , body: {"removeFloatingIp": {"address": "172.35.1.247"}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:22:53.170990 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-197ffcb1-86ff-439e-bc03-66b9f6fd6c65 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] [instance: 583e0c15-3053-429d-81bf-95a4d1f99cbf] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:22:53.178396 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-197ffcb1-86ff-439e-bc03-66b9f6fd6c65 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:53.179217 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-197ffcb1-86ff-439e-bc03-66b9f6fd6c65 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:53.180361 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-197ffcb1-86ff-439e-bc03-66b9f6fd6c65 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:53.181006 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-197ffcb1-86ff-439e-bc03-66b9f6fd6c65 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:53.470472 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-214c551d-1d2b-4a9d-a648-096564251b2f tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] 10.222.0.16 "POST /compute/v2.1/servers/fa075dcc-e175-449a-9914-d1ee19131f33/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 1.122728 Jul 16 21:22:53.477217 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1565/3169] 10.222.0.16 () {60 vars in 1322 bytes} [Sat Jul 16 21:22:52 2022] POST /compute/v2.1/servers/fa075dcc-e175-449a-9914-d1ee19131f33/os-volume_attachments => generated 194 bytes in 1130 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:22:53.485455 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-ceb2d4e3-97c6-4230-9568-0f7a7caf482b tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:22:53.486549 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-ceb2d4e3-97c6-4230-9568-0f7a7caf482b tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 32df0911-cfa6-4410-840a-4098a9fef0b4] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:22:53.493191 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_api:SELECT=96,nova_api:INSERT=3,nova_api:UPDATE=1 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:22:53.496040 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ceb2d4e3-97c6-4230-9568-0f7a7caf482b tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:53.496377 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ceb2d4e3-97c6-4230-9568-0f7a7caf482b tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:53.499785 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ceb2d4e3-97c6-4230-9568-0f7a7caf482b tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:53.500572 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ceb2d4e3-97c6-4230-9568-0f7a7caf482b tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:53.536754 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ceb2d4e3-97c6-4230-9568-0f7a7caf482b tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:53.538037 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ceb2d4e3-97c6-4230-9568-0f7a7caf482b tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:53.538737 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.objects.instance [None req-ceb2d4e3-97c6-4230-9568-0f7a7caf482b tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lazy-loading 'fault' on Instance uuid 32df0911-cfa6-4410-840a-4098a9fef0b4 {{(pid=138681) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:22:53.641428 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-197ffcb1-86ff-439e-bc03-66b9f6fd6c65 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Acquired lock "refresh_cache-583e0c15-3053-429d-81bf-95a4d1f99cbf" {{(pid=138680) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:294}} Jul 16 21:22:53.678248 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-ceb2d4e3-97c6-4230-9568-0f7a7caf482b tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f2e974ab5aa48dfb4b00f810dcedb63', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd31be8475a6a49d6bfe8132dd050c6c0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:22:53.690283 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-ceb2d4e3-97c6-4230-9568-0f7a7caf482b tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "GET /compute/v2.1/servers/32df0911-cfa6-4410-840a-4098a9fef0b4" status: 200 len: 1350 microversion: 2.1 time: 0.211257 Jul 16 21:22:53.690911 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1566/3170] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:22:53 2022] GET /compute/v2.1/servers/32df0911-cfa6-4410-840a-4098a9fef0b4 => generated 1350 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:22:53.705846 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-e9e3368f-2115-47f4-8fa6-1297aeb43493 tempest-ServerAddressesNegativeTestJSON-1415290150 tempest-ServerAddressesNegativeTestJSON-1415290150-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:22:53.706693 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-e9e3368f-2115-47f4-8fa6-1297aeb43493 tempest-ServerAddressesNegativeTestJSON-1415290150 tempest-ServerAddressesNegativeTestJSON-1415290150-project] [instance: 43a4dc6f-5d4d-45d7-bc69-0c754c2ec27a] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:22:53.713759 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e9e3368f-2115-47f4-8fa6-1297aeb43493 tempest-ServerAddressesNegativeTestJSON-1415290150 tempest-ServerAddressesNegativeTestJSON-1415290150-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:53.714783 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e9e3368f-2115-47f4-8fa6-1297aeb43493 tempest-ServerAddressesNegativeTestJSON-1415290150 tempest-ServerAddressesNegativeTestJSON-1415290150-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:53.715922 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e9e3368f-2115-47f4-8fa6-1297aeb43493 tempest-ServerAddressesNegativeTestJSON-1415290150 tempest-ServerAddressesNegativeTestJSON-1415290150-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:53.716536 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e9e3368f-2115-47f4-8fa6-1297aeb43493 tempest-ServerAddressesNegativeTestJSON-1415290150 tempest-ServerAddressesNegativeTestJSON-1415290150-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:54.190295 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-e9e3368f-2115-47f4-8fa6-1297aeb43493 tempest-ServerAddressesNegativeTestJSON-1415290150 tempest-ServerAddressesNegativeTestJSON-1415290150-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ab7326ef48042fe961a7eed5c46db24', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c0ebaf2e3714e42b3bda726e2133522', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:22:54.207453 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-e9e3368f-2115-47f4-8fa6-1297aeb43493 tempest-ServerAddressesNegativeTestJSON-1415290150 tempest-ServerAddressesNegativeTestJSON-1415290150-project] 10.222.0.16 "GET /compute/v2.1/servers/43a4dc6f-5d4d-45d7-bc69-0c754c2ec27a" status: 200 len: 1325 microversion: 2.1 time: 0.505690 Jul 16 21:22:54.208496 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1567/3171] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:22:53 2022] GET /compute/v2.1/servers/43a4dc6f-5d4d-45d7-bc69-0c754c2ec27a => generated 1325 bytes in 517 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:22:54.709827 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-12b00e65-30a2-4771-9d40-3139063e61ef tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:22:54.711457 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-12b00e65-30a2-4771-9d40-3139063e61ef tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 32df0911-cfa6-4410-840a-4098a9fef0b4] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:22:54.719574 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-12b00e65-30a2-4771-9d40-3139063e61ef tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:54.720412 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-12b00e65-30a2-4771-9d40-3139063e61ef tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:54.721626 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-12b00e65-30a2-4771-9d40-3139063e61ef tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:54.722295 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-12b00e65-30a2-4771-9d40-3139063e61ef tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:54.741594 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-12b00e65-30a2-4771-9d40-3139063e61ef tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] HTTP exception thrown: Instance 32df0911-cfa6-4410-840a-4098a9fef0b4 could not be found. Jul 16 21:22:54.743451 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-12b00e65-30a2-4771-9d40-3139063e61ef tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Request method failure captured: Jul 16 21:22:54.743451 ncdv-master-1 devstack@n-api.service[138681]: request: GET /compute/v2.1/servers/32df0911-cfa6-4410-840a-4098a9fef0b4 HTTP/1.1 Jul 16 21:22:54.743451 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:22:54.743451 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:22:54.743451 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:22:54.743451 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 21:22:54.743451 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:22:54.743451 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:22:54.743451 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:22:54.743451 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:22:54.743451 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0yvs4CI48W3ZCHoQ8huRZwHvqHxAdwrwrvk1v_ETBPx6FU4DdiPUjkjq4hN_4g_fHLAB_XZ9-AAN8Pyvy0KON2X25WlmyilO65MDHbIW9Xy9v9USVPAu_Ce-ze5X9UIHyned0AUneE4UPGiLLIkoXeoS5ekiq1pi209pud2lyUAuHgw Jul 16 21:22:54.743451 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:22:54.743451 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:22:54.743451 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:22:54.743451 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:22:54.743451 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:22:54.743451 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:22:54.743451 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: d31be8475a6a49d6bfe8132dd050c6c0 Jul 16 21:22:54.743451 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-DeleteServersTestJSON-1541892181 Jul 16 21:22:54.743451 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:22:54.743451 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:22:54.743451 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/d31be8475a6a49d6bfe8132dd050c6c0"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/d31be8475a6a49d6bfe8132dd050c6c0"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_d31be8475a6a49d6bfe8132dd050c6c0"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/d31be8475a6a49d6bfe8132dd050c6c0"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:22:54.743451 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-DeleteServersTestJSON-1541892181 Jul 16 21:22:54.743451 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: d31be8475a6a49d6bfe8132dd050c6c0 Jul 16 21:22:54.743451 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-DeleteServersTestJSON-1541892181 Jul 16 21:22:54.743451 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-DeleteServersTestJSON-1541892181-project Jul 16 21:22:54.743451 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:22:54.743451 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:22:54.743451 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: 6f2e974ab5aa48dfb4b00f810dcedb63 Jul 16 21:22:54.743451 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-DeleteServersTestJSON-1541892181-project Jul 16 21:22:54.743451 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:22:54.743451 ncdv-master-1 devstack@n-api.service[138681]: exception: Instance 32df0911-cfa6-4410-840a-4098a9fef0b4 could not be found. Jul 16 21:22:54.743451 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': '32df0911-cfa6-4410-840a-4098a9fef0b4'} Jul 16 21:22:54.743451 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:22:54.743451 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:22:54.743451 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:22:54.743451 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:22:54.743451 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:22:54.743451 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:22:54.743451 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:22:54.743451 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:22:54.743451 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:22:54.743451 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:22:54.743451 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 32df0911-cfa6-4410-840a-4098a9fef0b4 could not be found. Jul 16 21:22:54.743451 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:22:54.743451 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:22:54.743451 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:22:54.743451 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:22:54.743451 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:22:54.743451 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:22:54.743451 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:22:54.743451 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:22:54.743451 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:22:54.743451 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:22:54.743451 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:22:54.743451 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:22:54.743451 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:22:54.743451 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:22:54.743451 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:22:54.743451 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:22:54.743451 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 32df0911-cfa6-4410-840a-4098a9fef0b4 could not be found. Jul 16 21:22:54.743451 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:22:54.743451 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:22:54.743451 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:22:54.749966 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:22:54.749966 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:22:54.749966 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:22:54.749966 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:22:54.749966 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:22:54.749966 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 32df0911-cfa6-4410-840a-4098a9fef0b4 could not be found. Jul 16 21:22:54.749966 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:22:54.750937 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-12b00e65-30a2-4771-9d40-3139063e61ef tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Returning 404 to user: Instance 32df0911-cfa6-4410-840a-4098a9fef0b4 could not be found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:22:54.755276 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-12b00e65-30a2-4771-9d40-3139063e61ef tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "GET /compute/v2.1/servers/32df0911-cfa6-4410-840a-4098a9fef0b4" status: 404 len: 111 microversion: 2.1 time: 0.047639 Jul 16 21:22:54.755276 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1568/3172] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:22:54 2022] GET /compute/v2.1/servers/32df0911-cfa6-4410-840a-4098a9fef0b4 => generated 111 bytes in 49 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:22:54.883414 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-86a738a2-a34b-4f87-85e6-be22957ad4d6 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:22:54.885949 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-86a738a2-a34b-4f87-85e6-be22957ad4d6 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 32df0911-cfa6-4410-840a-4098a9fef0b4] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:22:54.897026 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-86a738a2-a34b-4f87-85e6-be22957ad4d6 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:54.898362 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-86a738a2-a34b-4f87-85e6-be22957ad4d6 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:54.899382 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-86a738a2-a34b-4f87-85e6-be22957ad4d6 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:54.901162 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-86a738a2-a34b-4f87-85e6-be22957ad4d6 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:54.916700 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-86a738a2-a34b-4f87-85e6-be22957ad4d6 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] HTTP exception thrown: Instance 32df0911-cfa6-4410-840a-4098a9fef0b4 could not be found. Jul 16 21:22:54.918591 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-86a738a2-a34b-4f87-85e6-be22957ad4d6 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Request method failure captured: Jul 16 21:22:54.918591 ncdv-master-1 devstack@n-api.service[138681]: request: GET /compute/v2.1/servers/32df0911-cfa6-4410-840a-4098a9fef0b4/os-volume_attachments HTTP/1.1 Jul 16 21:22:54.918591 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:22:54.918591 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:22:54.918591 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:22:54.918591 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 21:22:54.918591 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:22:54.918591 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:22:54.918591 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:22:54.918591 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:22:54.918591 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0yvs4CI48W3ZCHoQ8huRZwHvqHxAdwrwrvk1v_ETBPx6FU4DdiPUjkjq4hN_4g_fHLAB_XZ9-AAN8Pyvy0KON2X25WlmyilO65MDHbIW9Xy9v9USVPAu_Ce-ze5X9UIHyned0AUneE4UPGiLLIkoXeoS5ekiq1pi209pud2lyUAuHgw Jul 16 21:22:54.918591 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:22:54.918591 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:22:54.918591 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:22:54.918591 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:22:54.918591 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:22:54.918591 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:22:54.918591 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: d31be8475a6a49d6bfe8132dd050c6c0 Jul 16 21:22:54.918591 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-DeleteServersTestJSON-1541892181 Jul 16 21:22:54.918591 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:22:54.918591 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:22:54.918591 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/d31be8475a6a49d6bfe8132dd050c6c0"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/d31be8475a6a49d6bfe8132dd050c6c0"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_d31be8475a6a49d6bfe8132dd050c6c0"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/d31be8475a6a49d6bfe8132dd050c6c0"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:22:54.918591 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-DeleteServersTestJSON-1541892181 Jul 16 21:22:54.918591 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: d31be8475a6a49d6bfe8132dd050c6c0 Jul 16 21:22:54.918591 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-DeleteServersTestJSON-1541892181 Jul 16 21:22:54.918591 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-DeleteServersTestJSON-1541892181-project Jul 16 21:22:54.918591 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:22:54.918591 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:22:54.918591 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: 6f2e974ab5aa48dfb4b00f810dcedb63 Jul 16 21:22:54.918591 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-DeleteServersTestJSON-1541892181-project Jul 16 21:22:54.918591 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:22:54.918591 ncdv-master-1 devstack@n-api.service[138681]: exception: Instance 32df0911-cfa6-4410-840a-4098a9fef0b4 could not be found. Jul 16 21:22:54.918591 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'server_id': '32df0911-cfa6-4410-840a-4098a9fef0b4'} Jul 16 21:22:54.918591 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:22:54.918591 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:22:54.918591 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:22:54.918591 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:22:54.918591 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:22:54.918591 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:22:54.918591 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:22:54.918591 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:22:54.918591 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:22:54.918591 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:22:54.918591 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 32df0911-cfa6-4410-840a-4098a9fef0b4 could not be found. Jul 16 21:22:54.918591 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:22:54.918591 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:22:54.918591 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:22:54.918591 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:22:54.918591 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:22:54.918591 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:22:54.918591 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:22:54.918591 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:22:54.918591 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:22:54.918591 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:22:54.918591 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper Jul 16 21:22:54.918591 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 16 21:22:54.918591 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper Jul 16 21:22:54.918591 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 16 21:22:54.918591 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 303, in index Jul 16 21:22:54.918591 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, server_id) Jul 16 21:22:54.918591 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:22:54.918591 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:22:54.922624 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 32df0911-cfa6-4410-840a-4098a9fef0b4 could not be found. Jul 16 21:22:54.922624 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:22:54.922624 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:22:54.922624 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:22:54.922624 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:22:54.922624 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:22:54.922624 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:22:54.922624 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:22:54.922624 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:22:54.922624 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 32df0911-cfa6-4410-840a-4098a9fef0b4 could not be found. Jul 16 21:22:54.922624 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:22:54.929335 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-86a738a2-a34b-4f87-85e6-be22957ad4d6 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Returning 404 to user: Instance 32df0911-cfa6-4410-840a-4098a9fef0b4 could not be found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:22:54.929335 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-86a738a2-a34b-4f87-85e6-be22957ad4d6 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "GET /compute/v2.1/servers/32df0911-cfa6-4410-840a-4098a9fef0b4/os-volume_attachments" status: 404 len: 111 microversion: 2.1 time: 0.050432 Jul 16 21:22:54.931072 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1569/3173] 10.222.0.16 () {58 vars in 1301 bytes} [Sat Jul 16 21:22:54 2022] GET /compute/v2.1/servers/32df0911-cfa6-4410-840a-4098a9fef0b4/os-volume_attachments => generated 111 bytes in 57 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Jul 16 21:22:55.105484 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-e0f11e0a-30c6-49b8-8621-d708dcf8a915 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-DeleteServersTestJSON-server-1990054598", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452", "networks": [{"uuid": "fc3b835d-0a7f-4948-9fcb-1edfd6b9a8df"}]}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:22:55.190627 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.network.neutron [None req-e0f11e0a-30c6-49b8-8621-d708dcf8a915 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] validate_networks() for [('fc3b835d-0a7f-4948-9fcb-1edfd6b9a8df', None, None, None, None, None)] {{(pid=138681) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 16 21:22:55.929424 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell0:SELECT=2 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:22:56.022371 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.network.neutron [None req-197ffcb1-86ff-439e-bc03-66b9f6fd6c65 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] [instance: 583e0c15-3053-429d-81bf-95a4d1f99cbf] Building network info cache for instance {{(pid=138680) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1990}} Jul 16 21:22:56.728350 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-e0f11e0a-30c6-49b8-8621-d708dcf8a915 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Flavor limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 16 21:22:56.728783 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-e0f11e0a-30c6-49b8-8621-d708dcf8a915 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Image limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 16 21:22:56.729264 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-e0f11e0a-30c6-49b8-8621-d708dcf8a915 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Flavor pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 16 21:22:56.729683 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-e0f11e0a-30c6-49b8-8621-d708dcf8a915 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Image pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 16 21:22:56.730292 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-e0f11e0a-30c6-49b8-8621-d708dcf8a915 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 16 21:22:56.747634 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e0f11e0a-30c6-49b8-8621-d708dcf8a915 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:56.747634 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e0f11e0a-30c6-49b8-8621-d708dcf8a915 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:56.747634 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e0f11e0a-30c6-49b8-8621-d708dcf8a915 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:56.747634 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e0f11e0a-30c6-49b8-8621-d708dcf8a915 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:56.773099 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-e0f11e0a-30c6-49b8-8621-d708dcf8a915 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Getting quotas for project d31be8475a6a49d6bfe8132dd050c6c0. Resources: {'ram', 'instances', 'cores'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:22:56.786081 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-e0f11e0a-30c6-49b8-8621-d708dcf8a915 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Getting quotas for user 6f2e974ab5aa48dfb4b00f810dcedb63 and project d31be8475a6a49d6bfe8132dd050c6c0. Resources: {'ram', 'instances', 'cores'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:22:56.832587 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-e0f11e0a-30c6-49b8-8621-d708dcf8a915 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Going to run 1 instances... {{(pid=138681) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 16 21:22:56.836414 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-e0f11e0a-30c6-49b8-8621-d708dcf8a915 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: e5d606fa-6108-494c-846f-d40e3273cc64] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138681) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 16 21:22:57.098299 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-e0f11e0a-30c6-49b8-8621-d708dcf8a915 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 1.997750 Jul 16 21:22:57.104254 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1570/3174] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 21:22:55 2022] POST /compute/v2.1/servers => generated 380 bytes in 2004 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Jul 16 21:22:57.110582 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-b4eb7e02-a7b0-45c6-8c84-461e8fa18232 tempest-ServerAddressesNegativeTestJSON-1415290150 tempest-ServerAddressesNegativeTestJSON-1415290150-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:22:57.111476 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-b4eb7e02-a7b0-45c6-8c84-461e8fa18232 tempest-ServerAddressesNegativeTestJSON-1415290150 tempest-ServerAddressesNegativeTestJSON-1415290150-project] [instance: 43a4dc6f-5d4d-45d7-bc69-0c754c2ec27a] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:22:57.132987 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b4eb7e02-a7b0-45c6-8c84-461e8fa18232 tempest-ServerAddressesNegativeTestJSON-1415290150 tempest-ServerAddressesNegativeTestJSON-1415290150-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:57.133897 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b4eb7e02-a7b0-45c6-8c84-461e8fa18232 tempest-ServerAddressesNegativeTestJSON-1415290150 tempest-ServerAddressesNegativeTestJSON-1415290150-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:57.134285 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b4eb7e02-a7b0-45c6-8c84-461e8fa18232 tempest-ServerAddressesNegativeTestJSON-1415290150 tempest-ServerAddressesNegativeTestJSON-1415290150-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:57.135356 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b4eb7e02-a7b0-45c6-8c84-461e8fa18232 tempest-ServerAddressesNegativeTestJSON-1415290150 tempest-ServerAddressesNegativeTestJSON-1415290150-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:57.547618 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-b4eb7e02-a7b0-45c6-8c84-461e8fa18232 tempest-ServerAddressesNegativeTestJSON-1415290150 tempest-ServerAddressesNegativeTestJSON-1415290150-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ab7326ef48042fe961a7eed5c46db24', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c0ebaf2e3714e42b3bda726e2133522', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:22:57.549095 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.network.neutron [None req-197ffcb1-86ff-439e-bc03-66b9f6fd6c65 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] [instance: 583e0c15-3053-429d-81bf-95a4d1f99cbf] Updating instance_info_cache with network_info: [{"id": "40900dd3-d3df-4e26-80dd-0776ab4b1d20", "address": "fa:16:3e:79:c2:70", "network": {"id": "728dbccf-f950-46d2-9c0b-1f3c0f2fd7ae", "bridge": null, "label": "tempest-ServerRescueTestJSONUnderV235-935378440-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "65644a8df5d84df3b35c894560e0a5c4", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false, "bound_drivers": {"0": "hyperv"}}, "devname": "tap40900dd3-d3", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=138680) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:117}} Jul 16 21:22:57.560561 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-b4eb7e02-a7b0-45c6-8c84-461e8fa18232 tempest-ServerAddressesNegativeTestJSON-1415290150 tempest-ServerAddressesNegativeTestJSON-1415290150-project] 10.222.0.16 "GET /compute/v2.1/servers/43a4dc6f-5d4d-45d7-bc69-0c754c2ec27a" status: 200 len: 1325 microversion: 2.1 time: 0.452358 Jul 16 21:22:57.560561 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1571/3175] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:22:57 2022] GET /compute/v2.1/servers/43a4dc6f-5d4d-45d7-bc69-0c754c2ec27a => generated 1325 bytes in 454 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:22:57.564788 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-4e2e79ec-912a-406c-aedb-c35b4b2aa2c9 req-a64306a9-bcf3-463a-b9f8-866cb4bef9fd service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "43a4dc6f-5d4d-45d7-bc69-0c754c2ec27a", "tag": "4737b0a0-7745-4eb0-a84a-a81682e7532e"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:22:57.579443 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-4e2e79ec-912a-406c-aedb-c35b4b2aa2c9 req-a64306a9-bcf3-463a-b9f8-866cb4bef9fd service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:57.579900 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-4e2e79ec-912a-406c-aedb-c35b4b2aa2c9 req-a64306a9-bcf3-463a-b9f8-866cb4bef9fd service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:57.599831 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-4e2e79ec-912a-406c-aedb-c35b4b2aa2c9 req-a64306a9-bcf3-463a-b9f8-866cb4bef9fd service nova] Creating event network-changed:4737b0a0-7745-4eb0-a84a-a81682e7532e for instance 43a4dc6f-5d4d-45d7-bc69-0c754c2ec27a on nch2-master-1 Jul 16 21:22:57.605478 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-4e2e79ec-912a-406c-aedb-c35b4b2aa2c9 req-a64306a9-bcf3-463a-b9f8-866cb4bef9fd service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.044916 Jul 16 21:22:57.606735 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1572/3176] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:22:57 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:22:57.615706 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-22f9e031-a200-47b5-a8e6-f50c58b66f6f tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:22:57.616447 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-22f9e031-a200-47b5-a8e6-f50c58b66f6f tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: e5d606fa-6108-494c-846f-d40e3273cc64] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:22:57.636718 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-197ffcb1-86ff-439e-bc03-66b9f6fd6c65 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Releasing lock "refresh_cache-583e0c15-3053-429d-81bf-95a4d1f99cbf" {{(pid=138680) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:312}} Jul 16 21:22:57.637601 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-197ffcb1-86ff-439e-bc03-66b9f6fd6c65 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] 10.222.0.16 "POST /compute/v2.1/servers/583e0c15-3053-429d-81bf-95a4d1f99cbf/action" status: 202 len: 0 microversion: 2.1 time: 4.704398 Jul 16 21:22:57.638222 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1605/3177] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 21:22:52 2022] POST /compute/v2.1/servers/583e0c15-3053-429d-81bf-95a4d1f99cbf/action => generated 0 bytes in 4706 msecs (HTTP/1.1 202) 9 headers in 369 bytes (1 switches on core 0) Jul 16 21:22:57.650395 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-988dc8f9-5107-411a-828f-7269e196c09b tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Calling method '.version_select at 0x7fdd9c4f35e0>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:22:57.766246 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-22f9e031-a200-47b5-a8e6-f50c58b66f6f tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f2e974ab5aa48dfb4b00f810dcedb63', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd31be8475a6a49d6bfe8132dd050c6c0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:22:57.773378 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-22f9e031-a200-47b5-a8e6-f50c58b66f6f tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "GET /compute/v2.1/servers/e5d606fa-6108-494c-846f-d40e3273cc64" status: 200 len: 1215 microversion: 2.1 time: 0.165263 Jul 16 21:22:57.774253 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1573/3178] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:22:57 2022] GET /compute/v2.1/servers/e5d606fa-6108-494c-846f-d40e3273cc64 => generated 1215 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:22:57.793091 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-b3a6f3a4-66d0-4d6c-bbb1-ee5efef33326 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:22:57.793676 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-b3a6f3a4-66d0-4d6c-bbb1-ee5efef33326 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: e5d606fa-6108-494c-846f-d40e3273cc64] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:22:57.812826 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-b3a6f3a4-66d0-4d6c-bbb1-ee5efef33326 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: e5d606fa-6108-494c-846f-d40e3273cc64] Going to try to terminate instance {{(pid=138681) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 16 21:22:58.474282 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.scheduler.client.report [None req-b3a6f3a4-66d0-4d6c-bbb1-ee5efef33326 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Deleted allocations for instance e5d606fa-6108-494c-846f-d40e3273cc64 Jul 16 21:22:58.562475 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-b3a6f3a4-66d0-4d6c-bbb1-ee5efef33326 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "DELETE /compute/v2.1/servers/e5d606fa-6108-494c-846f-d40e3273cc64" status: 204 len: 0 microversion: 2.1 time: 0.774146 Jul 16 21:22:58.563432 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1574/3179] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:22:57 2022] DELETE /compute/v2.1/servers/e5d606fa-6108-494c-846f-d40e3273cc64 => generated 0 bytes in 777 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Jul 16 21:22:58.573970 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-caba2fb6-49fe-4739-bb9e-0ecbc2e52883 tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:22:58.575094 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-caba2fb6-49fe-4739-bb9e-0ecbc2e52883 tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] [instance: fa075dcc-e175-449a-9914-d1ee19131f33] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:22:58.582673 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-caba2fb6-49fe-4739-bb9e-0ecbc2e52883 tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:58.583510 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-caba2fb6-49fe-4739-bb9e-0ecbc2e52883 tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:58.584474 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-caba2fb6-49fe-4739-bb9e-0ecbc2e52883 tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:58.585085 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-caba2fb6-49fe-4739-bb9e-0ecbc2e52883 tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:58.621756 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-caba2fb6-49fe-4739-bb9e-0ecbc2e52883 tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] 10.222.0.16 "GET /compute/v2.1/servers/fa075dcc-e175-449a-9914-d1ee19131f33/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.054476 Jul 16 21:22:58.622672 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1575/3180] 10.222.0.16 () {58 vars in 1301 bytes} [Sat Jul 16 21:22:58 2022] GET /compute/v2.1/servers/fa075dcc-e175-449a-9914-d1ee19131f33/os-volume_attachments => generated 197 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:22:58.628888 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-02d479d8-d71a-44b9-8bf7-f204621cab3e tempest-ServerAddressesNegativeTestJSON-1415290150 tempest-ServerAddressesNegativeTestJSON-1415290150-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:22:58.629774 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-02d479d8-d71a-44b9-8bf7-f204621cab3e tempest-ServerAddressesNegativeTestJSON-1415290150 tempest-ServerAddressesNegativeTestJSON-1415290150-project] [instance: 43a4dc6f-5d4d-45d7-bc69-0c754c2ec27a] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:22:58.636778 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-02d479d8-d71a-44b9-8bf7-f204621cab3e tempest-ServerAddressesNegativeTestJSON-1415290150 tempest-ServerAddressesNegativeTestJSON-1415290150-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:58.637233 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-02d479d8-d71a-44b9-8bf7-f204621cab3e tempest-ServerAddressesNegativeTestJSON-1415290150 tempest-ServerAddressesNegativeTestJSON-1415290150-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:58.638011 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-02d479d8-d71a-44b9-8bf7-f204621cab3e tempest-ServerAddressesNegativeTestJSON-1415290150 tempest-ServerAddressesNegativeTestJSON-1415290150-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:58.638451 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-02d479d8-d71a-44b9-8bf7-f204621cab3e tempest-ServerAddressesNegativeTestJSON-1415290150 tempest-ServerAddressesNegativeTestJSON-1415290150-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:58.929412 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-02d479d8-d71a-44b9-8bf7-f204621cab3e tempest-ServerAddressesNegativeTestJSON-1415290150 tempest-ServerAddressesNegativeTestJSON-1415290150-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ab7326ef48042fe961a7eed5c46db24', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c0ebaf2e3714e42b3bda726e2133522', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:22:58.949375 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-02d479d8-d71a-44b9-8bf7-f204621cab3e tempest-ServerAddressesNegativeTestJSON-1415290150 tempest-ServerAddressesNegativeTestJSON-1415290150-project] 10.222.0.16 "GET /compute/v2.1/servers/43a4dc6f-5d4d-45d7-bc69-0c754c2ec27a" status: 200 len: 1325 microversion: 2.1 time: 0.325364 Jul 16 21:22:58.949970 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1576/3181] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:22:58 2022] GET /compute/v2.1/servers/43a4dc6f-5d4d-45d7-bc69-0c754c2ec27a => generated 1325 bytes in 327 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:22:58.957424 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-1e2d20d2-42c0-4cf8-ad7d-4da95bdb1212 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:22:58.958034 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-1e2d20d2-42c0-4cf8-ad7d-4da95bdb1212 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: e5d606fa-6108-494c-846f-d40e3273cc64] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:22:58.971219 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1e2d20d2-42c0-4cf8-ad7d-4da95bdb1212 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:58.971219 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1e2d20d2-42c0-4cf8-ad7d-4da95bdb1212 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:58.971219 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1e2d20d2-42c0-4cf8-ad7d-4da95bdb1212 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:58.971219 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1e2d20d2-42c0-4cf8-ad7d-4da95bdb1212 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:58.985716 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-1e2d20d2-42c0-4cf8-ad7d-4da95bdb1212 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] HTTP exception thrown: Instance e5d606fa-6108-494c-846f-d40e3273cc64 could not be found. Jul 16 21:22:58.987736 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-1e2d20d2-42c0-4cf8-ad7d-4da95bdb1212 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Request method failure captured: Jul 16 21:22:58.987736 ncdv-master-1 devstack@n-api.service[138681]: request: GET /compute/v2.1/servers/e5d606fa-6108-494c-846f-d40e3273cc64 HTTP/1.1 Jul 16 21:22:58.987736 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:22:58.987736 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:22:58.987736 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:22:58.987736 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 21:22:58.987736 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:22:58.987736 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:22:58.987736 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:22:58.987736 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:22:58.987736 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0yvs4CI48W3ZCHoQ8huRZwHvqHxAdwrwrvk1v_ETBPx6FU4DdiPUjkjq4hN_4g_fHLAB_XZ9-AAN8Pyvy0KON2X25WlmyilO65MDHbIW9Xy9v9USVPAu_Ce-ze5X9UIHyned0AUneE4UPGiLLIkoXeoS5ekiq1pi209pud2lyUAuHgw Jul 16 21:22:58.987736 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:22:58.987736 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:22:58.987736 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:22:58.987736 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:22:58.987736 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:22:58.987736 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:22:58.987736 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: d31be8475a6a49d6bfe8132dd050c6c0 Jul 16 21:22:58.987736 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-DeleteServersTestJSON-1541892181 Jul 16 21:22:58.987736 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:22:58.987736 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:22:58.987736 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/d31be8475a6a49d6bfe8132dd050c6c0"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/d31be8475a6a49d6bfe8132dd050c6c0"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_d31be8475a6a49d6bfe8132dd050c6c0"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/d31be8475a6a49d6bfe8132dd050c6c0"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:22:58.987736 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-DeleteServersTestJSON-1541892181 Jul 16 21:22:58.987736 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: d31be8475a6a49d6bfe8132dd050c6c0 Jul 16 21:22:58.987736 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-DeleteServersTestJSON-1541892181 Jul 16 21:22:58.987736 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-DeleteServersTestJSON-1541892181-project Jul 16 21:22:58.987736 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:22:58.987736 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:22:58.987736 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: 6f2e974ab5aa48dfb4b00f810dcedb63 Jul 16 21:22:58.987736 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-DeleteServersTestJSON-1541892181-project Jul 16 21:22:58.987736 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:22:58.987736 ncdv-master-1 devstack@n-api.service[138681]: exception: Instance e5d606fa-6108-494c-846f-d40e3273cc64 could not be found. Jul 16 21:22:58.987736 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': 'e5d606fa-6108-494c-846f-d40e3273cc64'} Jul 16 21:22:58.987736 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:22:58.987736 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:22:58.987736 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:22:58.987736 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:22:58.987736 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:22:58.987736 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:22:58.987736 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:22:58.987736 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:22:58.987736 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:22:58.987736 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:22:58.987736 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance e5d606fa-6108-494c-846f-d40e3273cc64 could not be found. Jul 16 21:22:58.987736 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:22:58.987736 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:22:58.987736 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:22:58.987736 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:22:58.987736 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:22:58.987736 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:22:58.987736 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:22:58.987736 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:22:58.987736 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:22:58.987736 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:22:58.987736 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:22:58.987736 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:22:58.987736 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:22:58.987736 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:22:58.987736 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:22:58.987736 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:22:58.987736 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance e5d606fa-6108-494c-846f-d40e3273cc64 could not be found. Jul 16 21:22:58.987736 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:22:58.987736 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:22:58.987736 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:22:58.992369 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:22:58.992369 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:22:58.992369 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:22:58.992369 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:22:58.992369 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:22:58.992369 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance e5d606fa-6108-494c-846f-d40e3273cc64 could not be found. Jul 16 21:22:58.992369 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:22:58.992369 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-1e2d20d2-42c0-4cf8-ad7d-4da95bdb1212 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Returning 404 to user: Instance e5d606fa-6108-494c-846f-d40e3273cc64 could not be found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:22:58.992369 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-1e2d20d2-42c0-4cf8-ad7d-4da95bdb1212 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "GET /compute/v2.1/servers/e5d606fa-6108-494c-846f-d40e3273cc64" status: 404 len: 111 microversion: 2.1 time: 0.038240 Jul 16 21:22:59.003293 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1577/3182] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:22:58 2022] GET /compute/v2.1/servers/e5d606fa-6108-494c-846f-d40e3273cc64 => generated 111 bytes in 53 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:22:59.008572 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-d29dc8ae-21c5-45e8-9d16-045ca100e701 tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:22:59.009483 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-d29dc8ae-21c5-45e8-9d16-045ca100e701 tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] [instance: fa075dcc-e175-449a-9914-d1ee19131f33] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:22:59.018222 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d29dc8ae-21c5-45e8-9d16-045ca100e701 tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:59.019211 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d29dc8ae-21c5-45e8-9d16-045ca100e701 tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:59.020037 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d29dc8ae-21c5-45e8-9d16-045ca100e701 tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:59.020665 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d29dc8ae-21c5-45e8-9d16-045ca100e701 tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:59.116421 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.objects.instance [None req-d29dc8ae-21c5-45e8-9d16-045ca100e701 tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Lazy-loading 'services' on Instance uuid fa075dcc-e175-449a-9914-d1ee19131f33 {{(pid=138681) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:22:59.331563 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-d29dc8ae-21c5-45e8-9d16-045ca100e701 tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] 10.222.0.16 "DELETE /compute/v2.1/servers/fa075dcc-e175-449a-9914-d1ee19131f33/os-volume_attachments/f4c1e544-bdaf-4a33-bf91-08a306e68029" status: 202 len: 0 microversion: 2.1 time: 0.326294 Jul 16 21:22:59.332654 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1578/3183] 10.222.0.16 () {58 vars in 1415 bytes} [Sat Jul 16 21:22:59 2022] DELETE /compute/v2.1/servers/fa075dcc-e175-449a-9914-d1ee19131f33/os-volume_attachments/f4c1e544-bdaf-4a33-bf91-08a306e68029 => generated 0 bytes in 329 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 21:22:59.355312 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-6e7dec2a-1957-4a3b-b982-a36b7741945d tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-DeleteServersTestJSON-server-1723530471", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452", "networks": [{"uuid": "fc3b835d-0a7f-4948-9fcb-1edfd6b9a8df"}]}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:22:59.441341 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.network.neutron [None req-6e7dec2a-1957-4a3b-b982-a36b7741945d tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] validate_networks() for [('fc3b835d-0a7f-4948-9fcb-1edfd6b9a8df', None, None, None, None, None)] {{(pid=138681) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 16 21:22:59.878133 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-988dc8f9-5107-411a-828f-7269e196c09b tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] 10.222.0.16 "DELETE /compute/v2.1/os-floating-ips/879d11f2-3eaa-4379-a2e5-3fde4318394c" status: 202 len: 0 microversion: 2.1 time: 2.233896 Jul 16 21:22:59.879158 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1606/3184] 10.222.0.16 () {58 vars in 1262 bytes} [Sat Jul 16 21:22:57 2022] DELETE /compute/v2.1/os-floating-ips/879d11f2-3eaa-4379-a2e5-3fde4318394c => generated 0 bytes in 2235 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 21:22:59.885486 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-b38b798a-a75d-4228-ac86-155f0f156fed req-2e391ea7-9c55-4aa8-9eff-5ab7bfbb62f3 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "583e0c15-3053-429d-81bf-95a4d1f99cbf", "tag": "40900dd3-d3df-4e26-80dd-0776ab4b1d20"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:22:59.896672 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-b38b798a-a75d-4228-ac86-155f0f156fed req-2e391ea7-9c55-4aa8-9eff-5ab7bfbb62f3 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:22:59.897470 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-b38b798a-a75d-4228-ac86-155f0f156fed req-2e391ea7-9c55-4aa8-9eff-5ab7bfbb62f3 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:22:59.910157 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-b38b798a-a75d-4228-ac86-155f0f156fed req-2e391ea7-9c55-4aa8-9eff-5ab7bfbb62f3 service nova] Creating event network-changed:40900dd3-d3df-4e26-80dd-0776ab4b1d20 for instance 583e0c15-3053-429d-81bf-95a4d1f99cbf on nch1-master-1 Jul 16 21:22:59.915528 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-b38b798a-a75d-4228-ac86-155f0f156fed req-2e391ea7-9c55-4aa8-9eff-5ab7bfbb62f3 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.035034 Jul 16 21:22:59.916139 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1607/3185] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:22:59 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:22:59.921484 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-ed50b552-e5a3-4898-af23-ca95ad3dc423 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Calling method '.version_select at 0x7fdd9c470d30>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:23:00.987573 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-6e7dec2a-1957-4a3b-b982-a36b7741945d tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Flavor limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 16 21:23:00.988090 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-6e7dec2a-1957-4a3b-b982-a36b7741945d tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Image limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 16 21:23:00.988463 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-6e7dec2a-1957-4a3b-b982-a36b7741945d tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Flavor pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 16 21:23:00.988874 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-6e7dec2a-1957-4a3b-b982-a36b7741945d tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Image pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 16 21:23:00.989285 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-6e7dec2a-1957-4a3b-b982-a36b7741945d tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 16 21:23:00.999444 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6e7dec2a-1957-4a3b-b982-a36b7741945d tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:01.000091 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6e7dec2a-1957-4a3b-b982-a36b7741945d tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:01.000930 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6e7dec2a-1957-4a3b-b982-a36b7741945d tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:01.002560 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6e7dec2a-1957-4a3b-b982-a36b7741945d tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:01.022072 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-6e7dec2a-1957-4a3b-b982-a36b7741945d tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Getting quotas for project d31be8475a6a49d6bfe8132dd050c6c0. Resources: {'ram', 'instances', 'cores'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:23:01.027437 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-6e7dec2a-1957-4a3b-b982-a36b7741945d tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Getting quotas for user 6f2e974ab5aa48dfb4b00f810dcedb63 and project d31be8475a6a49d6bfe8132dd050c6c0. Resources: {'ram', 'instances', 'cores'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:23:01.041367 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-6e7dec2a-1957-4a3b-b982-a36b7741945d tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Going to run 1 instances... {{(pid=138681) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 16 21:23:01.043812 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-6e7dec2a-1957-4a3b-b982-a36b7741945d tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 1cb418f7-6aea-42e6-88a2-007564d925c4] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138681) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 16 21:23:01.211843 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-ed50b552-e5a3-4898-af23-ca95ad3dc423 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] 10.222.0.16 "DELETE /compute/v2.1/os-security-groups/0e1e4131-16d9-409c-927c-9257d8d5843b" status: 202 len: 0 microversion: 2.1 time: 1.290574 Jul 16 21:23:01.211843 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1608/3186] 10.222.0.16 () {58 vars in 1271 bytes} [Sat Jul 16 21:22:59 2022] DELETE /compute/v2.1/os-security-groups/0e1e4131-16d9-409c-927c-9257d8d5843b => generated 0 bytes in 1292 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 21:23:01.216352 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-3a11cb5e-4d6d-4a99-9eff-1fce66a0af36 tempest-ServerAddressesNegativeTestJSON-1415290150 tempest-ServerAddressesNegativeTestJSON-1415290150-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:23:01.219353 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-3a11cb5e-4d6d-4a99-9eff-1fce66a0af36 tempest-ServerAddressesNegativeTestJSON-1415290150 tempest-ServerAddressesNegativeTestJSON-1415290150-project] [instance: 43a4dc6f-5d4d-45d7-bc69-0c754c2ec27a] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:23:01.260802 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3a11cb5e-4d6d-4a99-9eff-1fce66a0af36 tempest-ServerAddressesNegativeTestJSON-1415290150 tempest-ServerAddressesNegativeTestJSON-1415290150-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:01.261854 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-6e7dec2a-1957-4a3b-b982-a36b7741945d tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 1.924991 Jul 16 21:23:01.262805 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1579/3187] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 21:22:59 2022] POST /compute/v2.1/servers => generated 380 bytes in 1930 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 16 21:23:01.270276 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3a11cb5e-4d6d-4a99-9eff-1fce66a0af36 tempest-ServerAddressesNegativeTestJSON-1415290150 tempest-ServerAddressesNegativeTestJSON-1415290150-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:01.272816 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3a11cb5e-4d6d-4a99-9eff-1fce66a0af36 tempest-ServerAddressesNegativeTestJSON-1415290150 tempest-ServerAddressesNegativeTestJSON-1415290150-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:01.273288 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3a11cb5e-4d6d-4a99-9eff-1fce66a0af36 tempest-ServerAddressesNegativeTestJSON-1415290150 tempest-ServerAddressesNegativeTestJSON-1415290150-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:01.281032 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-2d5a9e9e-5b75-49b8-8c49-baf2e001c211 tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:23:01.281032 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-2d5a9e9e-5b75-49b8-8c49-baf2e001c211 tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] [instance: fa075dcc-e175-449a-9914-d1ee19131f33] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:23:01.291518 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2d5a9e9e-5b75-49b8-8c49-baf2e001c211 tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:01.298236 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2d5a9e9e-5b75-49b8-8c49-baf2e001c211 tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:01.298236 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2d5a9e9e-5b75-49b8-8c49-baf2e001c211 tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:01.298236 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2d5a9e9e-5b75-49b8-8c49-baf2e001c211 tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:01.360794 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-2d5a9e9e-5b75-49b8-8c49-baf2e001c211 tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] 10.222.0.16 "GET /compute/v2.1/servers/fa075dcc-e175-449a-9914-d1ee19131f33/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.096223 Jul 16 21:23:01.362044 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1580/3188] 10.222.0.16 () {58 vars in 1301 bytes} [Sat Jul 16 21:23:01 2022] GET /compute/v2.1/servers/fa075dcc-e175-449a-9914-d1ee19131f33/os-volume_attachments => generated 25 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Jul 16 21:23:01.367562 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-20a4c1f6-8e2e-456d-b5e1-5ddc35c5d11d tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:23:01.372931 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-20a4c1f6-8e2e-456d-b5e1-5ddc35c5d11d tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] [instance: 583e0c15-3053-429d-81bf-95a4d1f99cbf] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:23:01.377788 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-20a4c1f6-8e2e-456d-b5e1-5ddc35c5d11d tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:01.378787 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-20a4c1f6-8e2e-456d-b5e1-5ddc35c5d11d tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:01.379828 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-20a4c1f6-8e2e-456d-b5e1-5ddc35c5d11d tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:01.380520 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-20a4c1f6-8e2e-456d-b5e1-5ddc35c5d11d tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:01.487287 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-20a4c1f6-8e2e-456d-b5e1-5ddc35c5d11d tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] [instance: 583e0c15-3053-429d-81bf-95a4d1f99cbf] Going to try to terminate instance {{(pid=138681) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 16 21:23:01.693823 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-3a11cb5e-4d6d-4a99-9eff-1fce66a0af36 tempest-ServerAddressesNegativeTestJSON-1415290150 tempest-ServerAddressesNegativeTestJSON-1415290150-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ab7326ef48042fe961a7eed5c46db24', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c0ebaf2e3714e42b3bda726e2133522', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:23:01.704257 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-3a11cb5e-4d6d-4a99-9eff-1fce66a0af36 tempest-ServerAddressesNegativeTestJSON-1415290150 tempest-ServerAddressesNegativeTestJSON-1415290150-project] 10.222.0.16 "GET /compute/v2.1/servers/43a4dc6f-5d4d-45d7-bc69-0c754c2ec27a" status: 200 len: 1325 microversion: 2.1 time: 0.493791 Jul 16 21:23:01.705271 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1609/3189] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:23:01 2022] GET /compute/v2.1/servers/43a4dc6f-5d4d-45d7-bc69-0c754c2ec27a => generated 1325 bytes in 496 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:23:01.724877 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-cd2ca28f-642d-4e77-ba10-89ad256c3891 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:23:01.725777 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-cd2ca28f-642d-4e77-ba10-89ad256c3891 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 1cb418f7-6aea-42e6-88a2-007564d925c4] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:23:01.806656 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-cd2ca28f-642d-4e77-ba10-89ad256c3891 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f2e974ab5aa48dfb4b00f810dcedb63', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd31be8475a6a49d6bfe8132dd050c6c0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:23:01.831734 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-cd2ca28f-642d-4e77-ba10-89ad256c3891 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "GET /compute/v2.1/servers/1cb418f7-6aea-42e6-88a2-007564d925c4" status: 200 len: 1215 microversion: 2.1 time: 0.124632 Jul 16 21:23:01.832723 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1610/3190] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:23:01 2022] GET /compute/v2.1/servers/1cb418f7-6aea-42e6-88a2-007564d925c4 => generated 1215 bytes in 127 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:23:01.841994 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-c0366db7-0fd6-4bd2-acc8-937036963fc3 tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:23:01.841994 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-c0366db7-0fd6-4bd2-acc8-937036963fc3 tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] [instance: fa075dcc-e175-449a-9914-d1ee19131f33] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:23:01.847525 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c0366db7-0fd6-4bd2-acc8-937036963fc3 tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:01.847931 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c0366db7-0fd6-4bd2-acc8-937036963fc3 tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:01.849029 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c0366db7-0fd6-4bd2-acc8-937036963fc3 tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:01.850020 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c0366db7-0fd6-4bd2-acc8-937036963fc3 tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:01.875200 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=91,nova_cell1:UPDATE=4,nova_cell1:INSERT=1,nova_cell1:SAVEPOINT=2,nova_cell1:RELEASE=2 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:23:01.894961 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-c0366db7-0fd6-4bd2-acc8-937036963fc3 tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] 10.222.0.16 "GET /compute/v2.1/servers/fa075dcc-e175-449a-9914-d1ee19131f33/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.060837 Jul 16 21:23:01.895735 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1611/3191] 10.222.0.16 () {58 vars in 1301 bytes} [Sat Jul 16 21:23:01 2022] GET /compute/v2.1/servers/fa075dcc-e175-449a-9914-d1ee19131f33/os-volume_attachments => generated 25 bytes in 62 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Jul 16 21:23:01.946013 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-20a4c1f6-8e2e-456d-b5e1-5ddc35c5d11d tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] 10.222.0.16 "DELETE /compute/v2.1/servers/583e0c15-3053-429d-81bf-95a4d1f99cbf" status: 204 len: 0 microversion: 2.1 time: 0.582402 Jul 16 21:23:01.947719 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1581/3192] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:23:01 2022] DELETE /compute/v2.1/servers/583e0c15-3053-429d-81bf-95a4d1f99cbf => generated 0 bytes in 585 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 16 21:23:01.957251 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-d36d6fb9-79c1-4055-baec-0c6ee47ca8b1 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:23:01.958022 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-d36d6fb9-79c1-4055-baec-0c6ee47ca8b1 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] [instance: 583e0c15-3053-429d-81bf-95a4d1f99cbf] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:23:01.965871 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d36d6fb9-79c1-4055-baec-0c6ee47ca8b1 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:01.966694 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d36d6fb9-79c1-4055-baec-0c6ee47ca8b1 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:01.968574 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d36d6fb9-79c1-4055-baec-0c6ee47ca8b1 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:01.969247 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d36d6fb9-79c1-4055-baec-0c6ee47ca8b1 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:02.000309 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-8663406e-ad5d-4f98-85ec-6b9e149ae082 tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:23:02.000309 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-8663406e-ad5d-4f98-85ec-6b9e149ae082 tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] [instance: fa075dcc-e175-449a-9914-d1ee19131f33] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:23:02.017077 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8663406e-ad5d-4f98-85ec-6b9e149ae082 tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:02.023293 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8663406e-ad5d-4f98-85ec-6b9e149ae082 tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:02.023293 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8663406e-ad5d-4f98-85ec-6b9e149ae082 tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:02.032352 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8663406e-ad5d-4f98-85ec-6b9e149ae082 tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:02.071983 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-8663406e-ad5d-4f98-85ec-6b9e149ae082 tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] [instance: fa075dcc-e175-449a-9914-d1ee19131f33] Going to try to terminate instance {{(pid=138681) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 16 21:23:02.079833 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=94,nova_cell1:UPDATE=3,nova_cell1:INSERT=3 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:23:02.305349 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-d36d6fb9-79c1-4055-baec-0c6ee47ca8b1 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'aaf2292818b749f0ae43c66bca929be9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '65644a8df5d84df3b35c894560e0a5c4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:23:02.317081 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-d36d6fb9-79c1-4055-baec-0c6ee47ca8b1 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] 10.222.0.16 "GET /compute/v2.1/servers/583e0c15-3053-429d-81bf-95a4d1f99cbf" status: 200 len: 1510 microversion: 2.1 time: 0.364856 Jul 16 21:23:02.319621 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1612/3193] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:23:01 2022] GET /compute/v2.1/servers/583e0c15-3053-429d-81bf-95a4d1f99cbf => generated 1510 bytes in 368 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:23:02.562696 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-8663406e-ad5d-4f98-85ec-6b9e149ae082 tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] 10.222.0.16 "DELETE /compute/v2.1/servers/fa075dcc-e175-449a-9914-d1ee19131f33" status: 204 len: 0 microversion: 2.1 time: 0.568202 Jul 16 21:23:02.563518 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1582/3194] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:23:01 2022] DELETE /compute/v2.1/servers/fa075dcc-e175-449a-9914-d1ee19131f33 => generated 0 bytes in 570 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 16 21:23:02.571651 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-576efc86-00f8-48f5-8f3d-7d410e70dfad tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:23:02.572276 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-576efc86-00f8-48f5-8f3d-7d410e70dfad tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] [instance: fa075dcc-e175-449a-9914-d1ee19131f33] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:23:02.578393 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-576efc86-00f8-48f5-8f3d-7d410e70dfad tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:02.578851 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-576efc86-00f8-48f5-8f3d-7d410e70dfad tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:02.579650 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-576efc86-00f8-48f5-8f3d-7d410e70dfad tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:02.580094 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-576efc86-00f8-48f5-8f3d-7d410e70dfad tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:02.725731 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-fc94a2d8-b045-4a12-b092-655626142b3a tempest-ServerAddressesNegativeTestJSON-1415290150 tempest-ServerAddressesNegativeTestJSON-1415290150-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:23:02.726912 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-fc94a2d8-b045-4a12-b092-655626142b3a tempest-ServerAddressesNegativeTestJSON-1415290150 tempest-ServerAddressesNegativeTestJSON-1415290150-project] [instance: 43a4dc6f-5d4d-45d7-bc69-0c754c2ec27a] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:23:02.738167 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fc94a2d8-b045-4a12-b092-655626142b3a tempest-ServerAddressesNegativeTestJSON-1415290150 tempest-ServerAddressesNegativeTestJSON-1415290150-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:02.739105 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fc94a2d8-b045-4a12-b092-655626142b3a tempest-ServerAddressesNegativeTestJSON-1415290150 tempest-ServerAddressesNegativeTestJSON-1415290150-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:02.740341 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fc94a2d8-b045-4a12-b092-655626142b3a tempest-ServerAddressesNegativeTestJSON-1415290150 tempest-ServerAddressesNegativeTestJSON-1415290150-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:02.741314 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fc94a2d8-b045-4a12-b092-655626142b3a tempest-ServerAddressesNegativeTestJSON-1415290150 tempest-ServerAddressesNegativeTestJSON-1415290150-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:02.794331 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-576efc86-00f8-48f5-8f3d-7d410e70dfad tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '029f1334779e4bcdb3f4f2aed564a075', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e1f6ffc3fb64dba8c4474641e796fb8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:23:02.809712 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-576efc86-00f8-48f5-8f3d-7d410e70dfad tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] 10.222.0.16 "GET /compute/v2.1/servers/fa075dcc-e175-449a-9914-d1ee19131f33" status: 200 len: 1680 microversion: 2.1 time: 0.241238 Jul 16 21:23:02.811836 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1613/3195] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:23:02 2022] GET /compute/v2.1/servers/fa075dcc-e175-449a-9914-d1ee19131f33 => generated 1680 bytes in 245 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:23:02.850395 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-0b684a91-c1da-4ea5-b34f-07372baf18b9 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:23:02.851664 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-0b684a91-c1da-4ea5-b34f-07372baf18b9 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 1cb418f7-6aea-42e6-88a2-007564d925c4] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:23:02.865306 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0b684a91-c1da-4ea5-b34f-07372baf18b9 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:02.869558 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0b684a91-c1da-4ea5-b34f-07372baf18b9 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:02.871824 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0b684a91-c1da-4ea5-b34f-07372baf18b9 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:02.872463 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0b684a91-c1da-4ea5-b34f-07372baf18b9 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:02.991259 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-0b684a91-c1da-4ea5-b34f-07372baf18b9 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f2e974ab5aa48dfb4b00f810dcedb63', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd31be8475a6a49d6bfe8132dd050c6c0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:23:03.003505 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-0b684a91-c1da-4ea5-b34f-07372baf18b9 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "GET /compute/v2.1/servers/1cb418f7-6aea-42e6-88a2-007564d925c4" status: 200 len: 1219 microversion: 2.1 time: 0.157701 Jul 16 21:23:03.005001 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1614/3196] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:23:02 2022] GET /compute/v2.1/servers/1cb418f7-6aea-42e6-88a2-007564d925c4 => generated 1219 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:23:03.023964 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-fc94a2d8-b045-4a12-b092-655626142b3a tempest-ServerAddressesNegativeTestJSON-1415290150 tempest-ServerAddressesNegativeTestJSON-1415290150-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ab7326ef48042fe961a7eed5c46db24', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c0ebaf2e3714e42b3bda726e2133522', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:23:03.032963 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-fc94a2d8-b045-4a12-b092-655626142b3a tempest-ServerAddressesNegativeTestJSON-1415290150 tempest-ServerAddressesNegativeTestJSON-1415290150-project] 10.222.0.16 "GET /compute/v2.1/servers/43a4dc6f-5d4d-45d7-bc69-0c754c2ec27a" status: 200 len: 1325 microversion: 2.1 time: 0.314129 Jul 16 21:23:03.043255 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1583/3197] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:23:02 2022] GET /compute/v2.1/servers/43a4dc6f-5d4d-45d7-bc69-0c754c2ec27a => generated 1325 bytes in 326 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:23:03.334452 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-870a0415-6e65-48e7-905e-4aaf0eda1705 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:23:03.335098 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-870a0415-6e65-48e7-905e-4aaf0eda1705 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] [instance: 583e0c15-3053-429d-81bf-95a4d1f99cbf] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:23:03.344308 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-870a0415-6e65-48e7-905e-4aaf0eda1705 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:03.344768 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-870a0415-6e65-48e7-905e-4aaf0eda1705 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:03.345757 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-870a0415-6e65-48e7-905e-4aaf0eda1705 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:03.346403 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-870a0415-6e65-48e7-905e-4aaf0eda1705 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:03.567369 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-870a0415-6e65-48e7-905e-4aaf0eda1705 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'aaf2292818b749f0ae43c66bca929be9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '65644a8df5d84df3b35c894560e0a5c4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:23:03.575759 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-870a0415-6e65-48e7-905e-4aaf0eda1705 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] 10.222.0.16 "GET /compute/v2.1/servers/583e0c15-3053-429d-81bf-95a4d1f99cbf" status: 200 len: 1510 microversion: 2.1 time: 0.245128 Jul 16 21:23:03.576532 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1615/3198] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:23:03 2022] GET /compute/v2.1/servers/583e0c15-3053-429d-81bf-95a4d1f99cbf => generated 1510 bytes in 247 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:23:03.830675 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-79fa88ca-b0d8-420a-966c-f7eea959717b tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:23:03.831433 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-79fa88ca-b0d8-420a-966c-f7eea959717b tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] [instance: fa075dcc-e175-449a-9914-d1ee19131f33] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:23:03.861083 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-79fa88ca-b0d8-420a-966c-f7eea959717b tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:03.861565 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-79fa88ca-b0d8-420a-966c-f7eea959717b tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:03.863552 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-79fa88ca-b0d8-420a-966c-f7eea959717b tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:03.868343 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-79fa88ca-b0d8-420a-966c-f7eea959717b tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:04.023195 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-b46534c6-6633-474e-9943-d0a7f21cfa46 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:23:04.024403 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-b46534c6-6633-474e-9943-d0a7f21cfa46 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 1cb418f7-6aea-42e6-88a2-007564d925c4] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:23:04.047292 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b46534c6-6633-474e-9943-d0a7f21cfa46 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:04.047776 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b46534c6-6633-474e-9943-d0a7f21cfa46 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:04.048600 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b46534c6-6633-474e-9943-d0a7f21cfa46 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:04.052861 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b46534c6-6633-474e-9943-d0a7f21cfa46 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:04.123186 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-79fa88ca-b0d8-420a-966c-f7eea959717b tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '029f1334779e4bcdb3f4f2aed564a075', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e1f6ffc3fb64dba8c4474641e796fb8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:23:04.134794 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-79fa88ca-b0d8-420a-966c-f7eea959717b tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] 10.222.0.16 "GET /compute/v2.1/servers/fa075dcc-e175-449a-9914-d1ee19131f33" status: 200 len: 1680 microversion: 2.1 time: 0.310956 Jul 16 21:23:04.137346 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1584/3199] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:23:03 2022] GET /compute/v2.1/servers/fa075dcc-e175-449a-9914-d1ee19131f33 => generated 1680 bytes in 315 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:23:04.152244 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-80c86974-60d5-49e9-b379-66abac97248e req-b7e2389d-554c-4602-818a-7983160c8beb service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "43a4dc6f-5d4d-45d7-bc69-0c754c2ec27a", "name": "network-vif-plugged", "status": "completed", "tag": "4737b0a0-7745-4eb0-a84a-a81682e7532e"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:23:04.193327 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-80c86974-60d5-49e9-b379-66abac97248e req-b7e2389d-554c-4602-818a-7983160c8beb service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:04.193826 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-80c86974-60d5-49e9-b379-66abac97248e req-b7e2389d-554c-4602-818a-7983160c8beb service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:04.199188 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-b46534c6-6633-474e-9943-d0a7f21cfa46 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f2e974ab5aa48dfb4b00f810dcedb63', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd31be8475a6a49d6bfe8132dd050c6c0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:23:04.211735 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-80c86974-60d5-49e9-b379-66abac97248e req-b7e2389d-554c-4602-818a-7983160c8beb service nova] Creating event network-vif-plugged:4737b0a0-7745-4eb0-a84a-a81682e7532e for instance 43a4dc6f-5d4d-45d7-bc69-0c754c2ec27a on nch2-master-1 Jul 16 21:23:04.221629 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-80c86974-60d5-49e9-b379-66abac97248e req-b7e2389d-554c-4602-818a-7983160c8beb service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.082735 Jul 16 21:23:04.222574 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1585/3200] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:23:04 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:23:04.225481 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-b46534c6-6633-474e-9943-d0a7f21cfa46 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "GET /compute/v2.1/servers/1cb418f7-6aea-42e6-88a2-007564d925c4" status: 200 len: 1219 microversion: 2.1 time: 0.208346 Jul 16 21:23:04.238853 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1616/3201] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:23:04 2022] GET /compute/v2.1/servers/1cb418f7-6aea-42e6-88a2-007564d925c4 => generated 1219 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:23:04.246049 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-b059bdb1-b17c-47b1-84d0-18bbabe85d88 tempest-ServerAddressesNegativeTestJSON-1415290150 tempest-ServerAddressesNegativeTestJSON-1415290150-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:23:04.247078 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-b059bdb1-b17c-47b1-84d0-18bbabe85d88 tempest-ServerAddressesNegativeTestJSON-1415290150 tempest-ServerAddressesNegativeTestJSON-1415290150-project] [instance: 43a4dc6f-5d4d-45d7-bc69-0c754c2ec27a] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:23:04.256927 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b059bdb1-b17c-47b1-84d0-18bbabe85d88 tempest-ServerAddressesNegativeTestJSON-1415290150 tempest-ServerAddressesNegativeTestJSON-1415290150-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:04.257386 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b059bdb1-b17c-47b1-84d0-18bbabe85d88 tempest-ServerAddressesNegativeTestJSON-1415290150 tempest-ServerAddressesNegativeTestJSON-1415290150-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:04.258182 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b059bdb1-b17c-47b1-84d0-18bbabe85d88 tempest-ServerAddressesNegativeTestJSON-1415290150 tempest-ServerAddressesNegativeTestJSON-1415290150-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:04.258609 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b059bdb1-b17c-47b1-84d0-18bbabe85d88 tempest-ServerAddressesNegativeTestJSON-1415290150 tempest-ServerAddressesNegativeTestJSON-1415290150-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:04.603549 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-5e6450fd-cb1a-4b0f-bd31-a5ff97171233 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:23:04.605075 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-5e6450fd-cb1a-4b0f-bd31-a5ff97171233 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] [instance: 583e0c15-3053-429d-81bf-95a4d1f99cbf] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:23:04.626867 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5e6450fd-cb1a-4b0f-bd31-a5ff97171233 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:04.628849 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5e6450fd-cb1a-4b0f-bd31-a5ff97171233 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:04.630126 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5e6450fd-cb1a-4b0f-bd31-a5ff97171233 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:04.632209 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5e6450fd-cb1a-4b0f-bd31-a5ff97171233 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:04.851203 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-b059bdb1-b17c-47b1-84d0-18bbabe85d88 tempest-ServerAddressesNegativeTestJSON-1415290150 tempest-ServerAddressesNegativeTestJSON-1415290150-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ab7326ef48042fe961a7eed5c46db24', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c0ebaf2e3714e42b3bda726e2133522', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:23:04.877344 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-b059bdb1-b17c-47b1-84d0-18bbabe85d88 tempest-ServerAddressesNegativeTestJSON-1415290150 tempest-ServerAddressesNegativeTestJSON-1415290150-project] 10.222.0.16 "GET /compute/v2.1/servers/43a4dc6f-5d4d-45d7-bc69-0c754c2ec27a" status: 200 len: 1325 microversion: 2.1 time: 0.653074 Jul 16 21:23:04.878491 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1586/3202] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:23:04 2022] GET /compute/v2.1/servers/43a4dc6f-5d4d-45d7-bc69-0c754c2ec27a => generated 1325 bytes in 655 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:23:05.155135 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-3091b344-9ec3-47c8-82d3-d91bc13b5789 tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:23:05.155772 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-3091b344-9ec3-47c8-82d3-d91bc13b5789 tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] [instance: fa075dcc-e175-449a-9914-d1ee19131f33] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:23:05.159598 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_api:SELECT=90,nova_api:INSERT=6,nova_api:DELETE=1,nova_api:UPDATE=3 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:23:05.165463 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3091b344-9ec3-47c8-82d3-d91bc13b5789 tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:05.170957 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3091b344-9ec3-47c8-82d3-d91bc13b5789 tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:05.170957 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3091b344-9ec3-47c8-82d3-d91bc13b5789 tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:05.170957 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3091b344-9ec3-47c8-82d3-d91bc13b5789 tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:05.171462 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-5e6450fd-cb1a-4b0f-bd31-a5ff97171233 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'aaf2292818b749f0ae43c66bca929be9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '65644a8df5d84df3b35c894560e0a5c4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:23:05.182105 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-5e6450fd-cb1a-4b0f-bd31-a5ff97171233 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] 10.222.0.16 "GET /compute/v2.1/servers/583e0c15-3053-429d-81bf-95a4d1f99cbf" status: 200 len: 1510 microversion: 2.1 time: 0.591372 Jul 16 21:23:05.187045 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1617/3203] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:23:04 2022] GET /compute/v2.1/servers/583e0c15-3053-429d-81bf-95a4d1f99cbf => generated 1510 bytes in 598 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:23:05.247221 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-f7b9b51a-ac1b-45c1-b475-2eb9e53fb1bc tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:23:05.247861 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-f7b9b51a-ac1b-45c1-b475-2eb9e53fb1bc tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 1cb418f7-6aea-42e6-88a2-007564d925c4] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:23:05.261334 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f7b9b51a-ac1b-45c1-b475-2eb9e53fb1bc tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:05.262563 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f7b9b51a-ac1b-45c1-b475-2eb9e53fb1bc tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:05.262683 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f7b9b51a-ac1b-45c1-b475-2eb9e53fb1bc tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:05.263221 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f7b9b51a-ac1b-45c1-b475-2eb9e53fb1bc tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:05.476217 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-f7b9b51a-ac1b-45c1-b475-2eb9e53fb1bc tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f2e974ab5aa48dfb4b00f810dcedb63', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd31be8475a6a49d6bfe8132dd050c6c0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:23:05.487851 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-f7b9b51a-ac1b-45c1-b475-2eb9e53fb1bc tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "GET /compute/v2.1/servers/1cb418f7-6aea-42e6-88a2-007564d925c4" status: 200 len: 1267 microversion: 2.1 time: 0.245370 Jul 16 21:23:05.489442 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1618/3204] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:23:05 2022] GET /compute/v2.1/servers/1cb418f7-6aea-42e6-88a2-007564d925c4 => generated 1267 bytes in 248 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:23:05.896772 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-9bd5799a-2e0b-4aa0-8081-782f3e8b33a5 tempest-ServerAddressesNegativeTestJSON-1415290150 tempest-ServerAddressesNegativeTestJSON-1415290150-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:23:05.898770 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-9bd5799a-2e0b-4aa0-8081-782f3e8b33a5 tempest-ServerAddressesNegativeTestJSON-1415290150 tempest-ServerAddressesNegativeTestJSON-1415290150-project] [instance: 43a4dc6f-5d4d-45d7-bc69-0c754c2ec27a] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:23:05.908786 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9bd5799a-2e0b-4aa0-8081-782f3e8b33a5 tempest-ServerAddressesNegativeTestJSON-1415290150 tempest-ServerAddressesNegativeTestJSON-1415290150-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:05.909562 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9bd5799a-2e0b-4aa0-8081-782f3e8b33a5 tempest-ServerAddressesNegativeTestJSON-1415290150 tempest-ServerAddressesNegativeTestJSON-1415290150-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:05.910223 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9bd5799a-2e0b-4aa0-8081-782f3e8b33a5 tempest-ServerAddressesNegativeTestJSON-1415290150 tempest-ServerAddressesNegativeTestJSON-1415290150-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:05.910652 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9bd5799a-2e0b-4aa0-8081-782f3e8b33a5 tempest-ServerAddressesNegativeTestJSON-1415290150 tempest-ServerAddressesNegativeTestJSON-1415290150-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:06.032083 ncdv-master-1 devstack@n-api.service[138681]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jul 16 21:23:06.032366 ncdv-master-1 devstack@n-api.service[138681]: Sat Jul 16 21:23:06 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/fa075dcc-e175-449a-9914-d1ee19131f33 (ip 10.222.0.16) !!! Jul 16 21:23:06.347660 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-3091b344-9ec3-47c8-82d3-d91bc13b5789 tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '029f1334779e4bcdb3f4f2aed564a075', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e1f6ffc3fb64dba8c4474641e796fb8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:23:06.364052 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-3091b344-9ec3-47c8-82d3-d91bc13b5789 tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] 10.222.0.16 "GET /compute/v2.1/servers/fa075dcc-e175-449a-9914-d1ee19131f33" status: 200 len: 1680 microversion: 2.1 time: 1.209384 Jul 16 21:23:06.364052 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1587/3205] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:23:05 2022] GET /compute/v2.1/servers/fa075dcc-e175-449a-9914-d1ee19131f33 => generated 1680 bytes in 1212 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:23:06.371314 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-1418204a-7241-4fbc-929e-2b559410156c tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:23:06.376446 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-1418204a-7241-4fbc-929e-2b559410156c tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] [instance: 583e0c15-3053-429d-81bf-95a4d1f99cbf] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:23:06.391596 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1418204a-7241-4fbc-929e-2b559410156c tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:06.392354 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1418204a-7241-4fbc-929e-2b559410156c tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:06.393463 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1418204a-7241-4fbc-929e-2b559410156c tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:06.394075 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1418204a-7241-4fbc-929e-2b559410156c tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:06.652399 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-9bd5799a-2e0b-4aa0-8081-782f3e8b33a5 tempest-ServerAddressesNegativeTestJSON-1415290150 tempest-ServerAddressesNegativeTestJSON-1415290150-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ab7326ef48042fe961a7eed5c46db24', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c0ebaf2e3714e42b3bda726e2133522', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:23:06.672720 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-9bd5799a-2e0b-4aa0-8081-782f3e8b33a5 tempest-ServerAddressesNegativeTestJSON-1415290150 tempest-ServerAddressesNegativeTestJSON-1415290150-project] 10.222.0.16 "GET /compute/v2.1/servers/43a4dc6f-5d4d-45d7-bc69-0c754c2ec27a" status: 200 len: 1325 microversion: 2.1 time: 0.780597 Jul 16 21:23:06.685101 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1619/3206] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:23:05 2022] GET /compute/v2.1/servers/43a4dc6f-5d4d-45d7-bc69-0c754c2ec27a => generated 1325 bytes in 793 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:23:06.716367 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-8adcb281-0059-4cf9-9901-188f8e197e87 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:23:06.717791 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-8adcb281-0059-4cf9-9901-188f8e197e87 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 1cb418f7-6aea-42e6-88a2-007564d925c4] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:23:06.729675 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8adcb281-0059-4cf9-9901-188f8e197e87 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:06.730517 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8adcb281-0059-4cf9-9901-188f8e197e87 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:06.731726 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8adcb281-0059-4cf9-9901-188f8e197e87 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:06.732384 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8adcb281-0059-4cf9-9901-188f8e197e87 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:06.862651 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-1418204a-7241-4fbc-929e-2b559410156c tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'aaf2292818b749f0ae43c66bca929be9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '65644a8df5d84df3b35c894560e0a5c4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:23:06.870656 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-8adcb281-0059-4cf9-9901-188f8e197e87 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f2e974ab5aa48dfb4b00f810dcedb63', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd31be8475a6a49d6bfe8132dd050c6c0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:23:06.882368 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-1418204a-7241-4fbc-929e-2b559410156c tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] 10.222.0.16 "GET /compute/v2.1/servers/583e0c15-3053-429d-81bf-95a4d1f99cbf" status: 200 len: 1510 microversion: 2.1 time: 0.518172 Jul 16 21:23:06.892684 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1588/3207] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:23:06 2022] GET /compute/v2.1/servers/583e0c15-3053-429d-81bf-95a4d1f99cbf => generated 1510 bytes in 520 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:23:06.896293 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-8adcb281-0059-4cf9-9901-188f8e197e87 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "GET /compute/v2.1/servers/1cb418f7-6aea-42e6-88a2-007564d925c4" status: 200 len: 1273 microversion: 2.1 time: 0.211222 Jul 16 21:23:06.898331 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1620/3208] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:23:06 2022] GET /compute/v2.1/servers/1cb418f7-6aea-42e6-88a2-007564d925c4 => generated 1273 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:23:06.900188 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-4998d296-089a-4e3c-a7da-cd45b10b3ad7 req-d651318a-a918-4542-962e-51d21f94635e service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "583e0c15-3053-429d-81bf-95a4d1f99cbf", "name": "network-vif-deleted", "tag": "40900dd3-d3df-4e26-80dd-0776ab4b1d20"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:23:06.911452 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-4998d296-089a-4e3c-a7da-cd45b10b3ad7 req-d651318a-a918-4542-962e-51d21f94635e service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:06.912183 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-4998d296-089a-4e3c-a7da-cd45b10b3ad7 req-d651318a-a918-4542-962e-51d21f94635e service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:06.930468 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-4998d296-089a-4e3c-a7da-cd45b10b3ad7 req-d651318a-a918-4542-962e-51d21f94635e service nova] Creating event network-vif-deleted:40900dd3-d3df-4e26-80dd-0776ab4b1d20 for instance 583e0c15-3053-429d-81bf-95a4d1f99cbf on nch1-master-1 Jul 16 21:23:06.937876 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-4998d296-089a-4e3c-a7da-cd45b10b3ad7 req-d651318a-a918-4542-962e-51d21f94635e service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.052809 Jul 16 21:23:06.938774 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1589/3209] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:23:06 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:23:07.380977 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-ee5114d0-a5c8-468d-939c-e1685550911b tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:23:07.381556 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-ee5114d0-a5c8-468d-939c-e1685550911b tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] [instance: fa075dcc-e175-449a-9914-d1ee19131f33] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:23:07.389550 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ee5114d0-a5c8-468d-939c-e1685550911b tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:07.390380 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ee5114d0-a5c8-468d-939c-e1685550911b tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:07.393145 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ee5114d0-a5c8-468d-939c-e1685550911b tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:07.393800 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ee5114d0-a5c8-468d-939c-e1685550911b tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:07.669788 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-ee5114d0-a5c8-468d-939c-e1685550911b tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '029f1334779e4bcdb3f4f2aed564a075', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e1f6ffc3fb64dba8c4474641e796fb8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:23:07.679048 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-ee5114d0-a5c8-468d-939c-e1685550911b tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] 10.222.0.16 "GET /compute/v2.1/servers/fa075dcc-e175-449a-9914-d1ee19131f33" status: 200 len: 1612 microversion: 2.1 time: 0.301554 Jul 16 21:23:07.690657 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1621/3210] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:23:07 2022] GET /compute/v2.1/servers/fa075dcc-e175-449a-9914-d1ee19131f33 => generated 1612 bytes in 314 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:23:07.711964 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-b3d41c4d-3f43-48d8-b5f6-d7332be2ed58 tempest-ServerAddressesNegativeTestJSON-1415290150 tempest-ServerAddressesNegativeTestJSON-1415290150-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:23:07.712846 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-b3d41c4d-3f43-48d8-b5f6-d7332be2ed58 tempest-ServerAddressesNegativeTestJSON-1415290150 tempest-ServerAddressesNegativeTestJSON-1415290150-project] [instance: 43a4dc6f-5d4d-45d7-bc69-0c754c2ec27a] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:23:07.723327 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b3d41c4d-3f43-48d8-b5f6-d7332be2ed58 tempest-ServerAddressesNegativeTestJSON-1415290150 tempest-ServerAddressesNegativeTestJSON-1415290150-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:07.724134 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b3d41c4d-3f43-48d8-b5f6-d7332be2ed58 tempest-ServerAddressesNegativeTestJSON-1415290150 tempest-ServerAddressesNegativeTestJSON-1415290150-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:07.725621 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b3d41c4d-3f43-48d8-b5f6-d7332be2ed58 tempest-ServerAddressesNegativeTestJSON-1415290150 tempest-ServerAddressesNegativeTestJSON-1415290150-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:07.726281 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b3d41c4d-3f43-48d8-b5f6-d7332be2ed58 tempest-ServerAddressesNegativeTestJSON-1415290150 tempest-ServerAddressesNegativeTestJSON-1415290150-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:07.901725 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-a9bd8c69-9f52-4efe-ae68-5e4fdcb4e2e1 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:23:07.903478 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-a9bd8c69-9f52-4efe-ae68-5e4fdcb4e2e1 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] [instance: 583e0c15-3053-429d-81bf-95a4d1f99cbf] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:23:07.914001 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a9bd8c69-9f52-4efe-ae68-5e4fdcb4e2e1 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:07.914001 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a9bd8c69-9f52-4efe-ae68-5e4fdcb4e2e1 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:07.916354 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a9bd8c69-9f52-4efe-ae68-5e4fdcb4e2e1 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:07.917406 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a9bd8c69-9f52-4efe-ae68-5e4fdcb4e2e1 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:08.055490 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a9bd8c69-9f52-4efe-ae68-5e4fdcb4e2e1 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:08.056559 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a9bd8c69-9f52-4efe-ae68-5e4fdcb4e2e1 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:08.058020 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.objects.instance [None req-a9bd8c69-9f52-4efe-ae68-5e4fdcb4e2e1 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lazy-loading 'fault' on Instance uuid 583e0c15-3053-429d-81bf-95a4d1f99cbf {{(pid=138680) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:23:08.174477 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-a9bd8c69-9f52-4efe-ae68-5e4fdcb4e2e1 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'aaf2292818b749f0ae43c66bca929be9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '65644a8df5d84df3b35c894560e0a5c4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:23:08.190201 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-a9bd8c69-9f52-4efe-ae68-5e4fdcb4e2e1 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] 10.222.0.16 "GET /compute/v2.1/servers/583e0c15-3053-429d-81bf-95a4d1f99cbf" status: 200 len: 1313 microversion: 2.1 time: 0.292208 Jul 16 21:23:08.204599 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1622/3211] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:23:07 2022] GET /compute/v2.1/servers/583e0c15-3053-429d-81bf-95a4d1f99cbf => generated 1313 bytes in 308 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:23:08.212339 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-4a6e6d29-6b10-43cd-8709-d226dcdb7107 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:23:08.213313 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-4a6e6d29-6b10-43cd-8709-d226dcdb7107 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 1cb418f7-6aea-42e6-88a2-007564d925c4] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:23:08.221685 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4a6e6d29-6b10-43cd-8709-d226dcdb7107 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:08.222512 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4a6e6d29-6b10-43cd-8709-d226dcdb7107 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:08.224581 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4a6e6d29-6b10-43cd-8709-d226dcdb7107 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:08.225333 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4a6e6d29-6b10-43cd-8709-d226dcdb7107 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:08.268620 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-b3d41c4d-3f43-48d8-b5f6-d7332be2ed58 tempest-ServerAddressesNegativeTestJSON-1415290150 tempest-ServerAddressesNegativeTestJSON-1415290150-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ab7326ef48042fe961a7eed5c46db24', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c0ebaf2e3714e42b3bda726e2133522', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:23:08.277319 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-b3d41c4d-3f43-48d8-b5f6-d7332be2ed58 tempest-ServerAddressesNegativeTestJSON-1415290150 tempest-ServerAddressesNegativeTestJSON-1415290150-project] 10.222.0.16 "GET /compute/v2.1/servers/43a4dc6f-5d4d-45d7-bc69-0c754c2ec27a" status: 200 len: 1523 microversion: 2.1 time: 0.576767 Jul 16 21:23:08.281402 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1590/3212] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:23:07 2022] GET /compute/v2.1/servers/43a4dc6f-5d4d-45d7-bc69-0c754c2ec27a => generated 1523 bytes in 582 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:23:08.309559 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-69b68e75-b77a-4ed5-9784-2f3ce657fa8f tempest-ServerAddressesNegativeTestJSON-1415290150 tempest-ServerAddressesNegativeTestJSON-1415290150-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:23:08.310541 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-69b68e75-b77a-4ed5-9784-2f3ce657fa8f tempest-ServerAddressesNegativeTestJSON-1415290150 tempest-ServerAddressesNegativeTestJSON-1415290150-project] [instance: 43a4dc6f-5d4d-45d7-bc69-0c754c2ec27a] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:23:08.324469 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-69b68e75-b77a-4ed5-9784-2f3ce657fa8f tempest-ServerAddressesNegativeTestJSON-1415290150 tempest-ServerAddressesNegativeTestJSON-1415290150-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:08.325209 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-69b68e75-b77a-4ed5-9784-2f3ce657fa8f tempest-ServerAddressesNegativeTestJSON-1415290150 tempest-ServerAddressesNegativeTestJSON-1415290150-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:08.326366 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-69b68e75-b77a-4ed5-9784-2f3ce657fa8f tempest-ServerAddressesNegativeTestJSON-1415290150 tempest-ServerAddressesNegativeTestJSON-1415290150-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:08.327041 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-69b68e75-b77a-4ed5-9784-2f3ce657fa8f tempest-ServerAddressesNegativeTestJSON-1415290150 tempest-ServerAddressesNegativeTestJSON-1415290150-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:08.365276 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-69b68e75-b77a-4ed5-9784-2f3ce657fa8f tempest-ServerAddressesNegativeTestJSON-1415290150 tempest-ServerAddressesNegativeTestJSON-1415290150-project] HTTP exception thrown: Instance is not a member of specified network Jul 16 21:23:08.378981 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-4a6e6d29-6b10-43cd-8709-d226dcdb7107 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f2e974ab5aa48dfb4b00f810dcedb63', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd31be8475a6a49d6bfe8132dd050c6c0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:23:08.381849 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-69b68e75-b77a-4ed5-9784-2f3ce657fa8f tempest-ServerAddressesNegativeTestJSON-1415290150 tempest-ServerAddressesNegativeTestJSON-1415290150-project] Request method failure captured: Jul 16 21:23:08.381849 ncdv-master-1 devstack@n-api.service[138681]: request: GET /compute/v2.1/servers/43a4dc6f-5d4d-45d7-bc69-0c754c2ec27a/ips/invalid HTTP/1.1 Jul 16 21:23:08.381849 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:23:08.381849 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:23:08.381849 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:23:08.381849 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 21:23:08.381849 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:23:08.381849 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:23:08.381849 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:23:08.381849 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:23:08.381849 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0ywkRCV6NNaXk9MtxSRzjRYA4uiu8nVVP2REzd6SnqbgshubqNj0v_3TbWIK63J-IyaMZKkOK2K7t6ZqzT7w_9EGm5KNKhsnau4FHgmclpIu3I5edQQKzyWMc86RwtEygiyD06flnxu7mFd7F5iCzMr2uIcs7XKCJR1_GHAWq2C8AUk Jul 16 21:23:08.381849 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:23:08.381849 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:23:08.381849 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:23:08.381849 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:23:08.381849 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:23:08.381849 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:23:08.381849 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: 7c0ebaf2e3714e42b3bda726e2133522 Jul 16 21:23:08.381849 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-ServerAddressesNegativeTestJSON-1415290150 Jul 16 21:23:08.381849 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:23:08.381849 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:23:08.381849 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/7c0ebaf2e3714e42b3bda726e2133522"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/7c0ebaf2e3714e42b3bda726e2133522"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_7c0ebaf2e3714e42b3bda726e2133522"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/7c0ebaf2e3714e42b3bda726e2133522"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:23:08.381849 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-ServerAddressesNegativeTestJSON-1415290150 Jul 16 21:23:08.381849 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: 7c0ebaf2e3714e42b3bda726e2133522 Jul 16 21:23:08.381849 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-ServerAddressesNegativeTestJSON-1415290150 Jul 16 21:23:08.381849 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-ServerAddressesNegativeTestJSON-1415290150-project Jul 16 21:23:08.381849 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:23:08.381849 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:23:08.381849 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: 3ab7326ef48042fe961a7eed5c46db24 Jul 16 21:23:08.381849 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-ServerAddressesNegativeTestJSON-1415290150-project Jul 16 21:23:08.381849 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:23:08.381849 ncdv-master-1 devstack@n-api.service[138681]: exception: Instance is not a member of specified network Jul 16 21:23:08.381849 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'server_id': '43a4dc6f-5d4d-45d7-bc69-0c754c2ec27a', 'id': 'invalid'} Jul 16 21:23:08.381849 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:23:08.381849 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:23:08.381849 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:23:08.381849 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:23:08.381849 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:23:08.381849 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:23:08.381849 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:23:08.381849 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:23:08.381849 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/ips.py", line 52, in show Jul 16 21:23:08.381849 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=msg) Jul 16 21:23:08.381849 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance is not a member of specified network Jul 16 21:23:08.381849 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:23:08.381849 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:23:08.381849 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:23:08.381849 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:23:08.381849 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:23:08.381849 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:23:08.381849 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:23:08.381849 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:23:08.381849 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance is not a member of specified network Jul 16 21:23:08.381849 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:23:08.385583 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-69b68e75-b77a-4ed5-9784-2f3ce657fa8f tempest-ServerAddressesNegativeTestJSON-1415290150 tempest-ServerAddressesNegativeTestJSON-1415290150-project] Returning 404 to user: Instance is not a member of specified network {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:23:08.385583 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-69b68e75-b77a-4ed5-9784-2f3ce657fa8f tempest-ServerAddressesNegativeTestJSON-1415290150 tempest-ServerAddressesNegativeTestJSON-1415290150-project] 10.222.0.16 "GET /compute/v2.1/servers/43a4dc6f-5d4d-45d7-bc69-0c754c2ec27a/ips/invalid" status: 404 len: 91 microversion: 2.1 time: 0.079116 Jul 16 21:23:08.385583 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1591/3213] 10.222.0.16 () {58 vars in 1271 bytes} [Sat Jul 16 21:23:08 2022] GET /compute/v2.1/servers/43a4dc6f-5d4d-45d7-bc69-0c754c2ec27a/ips/invalid => generated 91 bytes in 81 msecs (HTTP/1.1 404) 9 headers in 378 bytes (1 switches on core 0) Jul 16 21:23:08.394723 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-92c4ef85-08eb-4350-a87c-fa495e00251e tempest-ServerAddressesNegativeTestJSON-1415290150 tempest-ServerAddressesNegativeTestJSON-1415290150-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:23:08.395636 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-92c4ef85-08eb-4350-a87c-fa495e00251e tempest-ServerAddressesNegativeTestJSON-1415290150 tempest-ServerAddressesNegativeTestJSON-1415290150-project] Failed to fetch instance by id 999 {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2926}} Jul 16 21:23:08.396214 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-92c4ef85-08eb-4350-a87c-fa495e00251e tempest-ServerAddressesNegativeTestJSON-1415290150 tempest-ServerAddressesNegativeTestJSON-1415290150-project] HTTP exception thrown: Instance 999 could not be found. Jul 16 21:23:08.397651 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-92c4ef85-08eb-4350-a87c-fa495e00251e tempest-ServerAddressesNegativeTestJSON-1415290150 tempest-ServerAddressesNegativeTestJSON-1415290150-project] Request method failure captured: Jul 16 21:23:08.397651 ncdv-master-1 devstack@n-api.service[138681]: request: GET /compute/v2.1/servers/999/ips HTTP/1.1 Jul 16 21:23:08.397651 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:23:08.397651 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:23:08.397651 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:23:08.397651 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 21:23:08.397651 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:23:08.397651 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:23:08.397651 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:23:08.397651 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:23:08.397651 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0ywkRCV6NNaXk9MtxSRzjRYA4uiu8nVVP2REzd6SnqbgshubqNj0v_3TbWIK63J-IyaMZKkOK2K7t6ZqzT7w_9EGm5KNKhsnau4FHgmclpIu3I5edQQKzyWMc86RwtEygiyD06flnxu7mFd7F5iCzMr2uIcs7XKCJR1_GHAWq2C8AUk Jul 16 21:23:08.397651 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:23:08.397651 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:23:08.397651 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:23:08.397651 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:23:08.397651 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:23:08.397651 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:23:08.397651 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: 7c0ebaf2e3714e42b3bda726e2133522 Jul 16 21:23:08.397651 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-ServerAddressesNegativeTestJSON-1415290150 Jul 16 21:23:08.397651 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:23:08.397651 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:23:08.397651 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/7c0ebaf2e3714e42b3bda726e2133522"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/7c0ebaf2e3714e42b3bda726e2133522"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_7c0ebaf2e3714e42b3bda726e2133522"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/7c0ebaf2e3714e42b3bda726e2133522"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:23:08.397651 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-ServerAddressesNegativeTestJSON-1415290150 Jul 16 21:23:08.397651 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: 7c0ebaf2e3714e42b3bda726e2133522 Jul 16 21:23:08.397651 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-ServerAddressesNegativeTestJSON-1415290150 Jul 16 21:23:08.397651 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-ServerAddressesNegativeTestJSON-1415290150-project Jul 16 21:23:08.397651 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:23:08.397651 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:23:08.397651 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: 3ab7326ef48042fe961a7eed5c46db24 Jul 16 21:23:08.397651 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-ServerAddressesNegativeTestJSON-1415290150-project Jul 16 21:23:08.397651 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:23:08.397651 ncdv-master-1 devstack@n-api.service[138681]: exception: Instance 999 could not be found. Jul 16 21:23:08.397651 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'server_id': '999'} Jul 16 21:23:08.397651 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:23:08.397651 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:23:08.397651 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:23:08.397651 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:23:08.397651 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2927, in get Jul 16 21:23:08.397651 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=instance_id) Jul 16 21:23:08.397651 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 999 could not be found. Jul 16 21:23:08.397651 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:23:08.397651 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:23:08.397651 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:23:08.397651 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:23:08.397651 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:23:08.397651 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:23:08.397651 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:23:08.397651 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:23:08.397651 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:23:08.397651 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:23:08.397651 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/ips.py", line 37, in index Jul 16 21:23:08.397651 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self._compute_api, context, server_id) Jul 16 21:23:08.397651 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:23:08.397651 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:23:08.397651 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 999 could not be found. Jul 16 21:23:08.397651 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:23:08.397651 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:23:08.397651 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:23:08.397651 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:23:08.397651 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:23:08.397651 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:23:08.397651 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:23:08.397651 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:23:08.397651 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 999 could not be found. Jul 16 21:23:08.397651 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:23:08.402162 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-92c4ef85-08eb-4350-a87c-fa495e00251e tempest-ServerAddressesNegativeTestJSON-1415290150 tempest-ServerAddressesNegativeTestJSON-1415290150-project] Returning 404 to user: Instance 999 could not be found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:23:08.402162 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-92c4ef85-08eb-4350-a87c-fa495e00251e tempest-ServerAddressesNegativeTestJSON-1415290150 tempest-ServerAddressesNegativeTestJSON-1415290150-project] 10.222.0.16 "GET /compute/v2.1/servers/999/ips" status: 404 len: 78 microversion: 2.1 time: 0.009059 Jul 16 21:23:08.402162 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1592/3214] 10.222.0.16 () {58 vars in 1148 bytes} [Sat Jul 16 21:23:08 2022] GET /compute/v2.1/servers/999/ips => generated 78 bytes in 10 msecs (HTTP/1.1 404) 9 headers in 378 bytes (1 switches on core 0) Jul 16 21:23:08.404547 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-4a6e6d29-6b10-43cd-8709-d226dcdb7107 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "GET /compute/v2.1/servers/1cb418f7-6aea-42e6-88a2-007564d925c4" status: 200 len: 1273 microversion: 2.1 time: 0.187764 Jul 16 21:23:08.408272 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1623/3215] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:23:08 2022] GET /compute/v2.1/servers/1cb418f7-6aea-42e6-88a2-007564d925c4 => generated 1273 bytes in 203 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:23:08.418961 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-e4335bca-721a-4524-999e-bdd0de66d201 tempest-ServerAddressesNegativeTestJSON-1415290150 tempest-ServerAddressesNegativeTestJSON-1415290150-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:23:08.419496 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-e4335bca-721a-4524-999e-bdd0de66d201 tempest-ServerAddressesNegativeTestJSON-1415290150 tempest-ServerAddressesNegativeTestJSON-1415290150-project] [instance: 43a4dc6f-5d4d-45d7-bc69-0c754c2ec27a] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:23:08.442600 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e4335bca-721a-4524-999e-bdd0de66d201 tempest-ServerAddressesNegativeTestJSON-1415290150 tempest-ServerAddressesNegativeTestJSON-1415290150-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:08.443186 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e4335bca-721a-4524-999e-bdd0de66d201 tempest-ServerAddressesNegativeTestJSON-1415290150 tempest-ServerAddressesNegativeTestJSON-1415290150-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:08.444214 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e4335bca-721a-4524-999e-bdd0de66d201 tempest-ServerAddressesNegativeTestJSON-1415290150 tempest-ServerAddressesNegativeTestJSON-1415290150-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:08.444789 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e4335bca-721a-4524-999e-bdd0de66d201 tempest-ServerAddressesNegativeTestJSON-1415290150 tempest-ServerAddressesNegativeTestJSON-1415290150-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:08.472376 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-e4335bca-721a-4524-999e-bdd0de66d201 tempest-ServerAddressesNegativeTestJSON-1415290150 tempest-ServerAddressesNegativeTestJSON-1415290150-project] [instance: 43a4dc6f-5d4d-45d7-bc69-0c754c2ec27a] Going to try to terminate instance {{(pid=138681) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 16 21:23:08.696892 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-1e55f505-cbac-4a11-81df-12879238dd71 tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:23:08.697428 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-1e55f505-cbac-4a11-81df-12879238dd71 tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] [instance: fa075dcc-e175-449a-9914-d1ee19131f33] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:23:08.703902 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1e55f505-cbac-4a11-81df-12879238dd71 tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:08.704586 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1e55f505-cbac-4a11-81df-12879238dd71 tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:08.705527 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1e55f505-cbac-4a11-81df-12879238dd71 tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:08.706127 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1e55f505-cbac-4a11-81df-12879238dd71 tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:08.753626 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1e55f505-cbac-4a11-81df-12879238dd71 tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:08.754358 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1e55f505-cbac-4a11-81df-12879238dd71 tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:08.754996 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.objects.instance [None req-1e55f505-cbac-4a11-81df-12879238dd71 tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Lazy-loading 'fault' on Instance uuid fa075dcc-e175-449a-9914-d1ee19131f33 {{(pid=138680) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:23:08.775837 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-e4335bca-721a-4524-999e-bdd0de66d201 tempest-ServerAddressesNegativeTestJSON-1415290150 tempest-ServerAddressesNegativeTestJSON-1415290150-project] 10.222.0.16 "DELETE /compute/v2.1/servers/43a4dc6f-5d4d-45d7-bc69-0c754c2ec27a" status: 204 len: 0 microversion: 2.1 time: 0.366310 Jul 16 21:23:08.778496 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1593/3216] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:23:08 2022] DELETE /compute/v2.1/servers/43a4dc6f-5d4d-45d7-bc69-0c754c2ec27a => generated 0 bytes in 370 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 16 21:23:08.789853 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-a4e0948e-4917-4ecc-807e-2d35e26159d8 tempest-ServerAddressesNegativeTestJSON-1415290150 tempest-ServerAddressesNegativeTestJSON-1415290150-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:23:08.794147 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-a4e0948e-4917-4ecc-807e-2d35e26159d8 tempest-ServerAddressesNegativeTestJSON-1415290150 tempest-ServerAddressesNegativeTestJSON-1415290150-project] [instance: 43a4dc6f-5d4d-45d7-bc69-0c754c2ec27a] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:23:08.811253 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a4e0948e-4917-4ecc-807e-2d35e26159d8 tempest-ServerAddressesNegativeTestJSON-1415290150 tempest-ServerAddressesNegativeTestJSON-1415290150-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:08.811799 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a4e0948e-4917-4ecc-807e-2d35e26159d8 tempest-ServerAddressesNegativeTestJSON-1415290150 tempest-ServerAddressesNegativeTestJSON-1415290150-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:08.812659 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a4e0948e-4917-4ecc-807e-2d35e26159d8 tempest-ServerAddressesNegativeTestJSON-1415290150 tempest-ServerAddressesNegativeTestJSON-1415290150-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:08.813168 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a4e0948e-4917-4ecc-807e-2d35e26159d8 tempest-ServerAddressesNegativeTestJSON-1415290150 tempest-ServerAddressesNegativeTestJSON-1415290150-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:08.903619 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-1e55f505-cbac-4a11-81df-12879238dd71 tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '029f1334779e4bcdb3f4f2aed564a075', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e1f6ffc3fb64dba8c4474641e796fb8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:23:08.918646 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-1e55f505-cbac-4a11-81df-12879238dd71 tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] 10.222.0.16 "GET /compute/v2.1/servers/fa075dcc-e175-449a-9914-d1ee19131f33" status: 200 len: 1331 microversion: 2.1 time: 0.225589 Jul 16 21:23:08.931261 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1624/3217] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:23:08 2022] GET /compute/v2.1/servers/fa075dcc-e175-449a-9914-d1ee19131f33 => generated 1331 bytes in 239 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:23:08.977235 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-346cf172-a6b2-47ca-be60-ddc3ef6e1857 req-4f28a6e2-6e2f-4986-868a-98da3aefa647 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "fa075dcc-e175-449a-9914-d1ee19131f33", "name": "network-vif-deleted", "tag": "49869004-34cd-4d2b-8bf2-b7cbd8448d51"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:23:09.001980 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-346cf172-a6b2-47ca-be60-ddc3ef6e1857 req-4f28a6e2-6e2f-4986-868a-98da3aefa647 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:09.005054 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-346cf172-a6b2-47ca-be60-ddc3ef6e1857 req-4f28a6e2-6e2f-4986-868a-98da3aefa647 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:09.037809 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-346cf172-a6b2-47ca-be60-ddc3ef6e1857 req-4f28a6e2-6e2f-4986-868a-98da3aefa647 service nova] Creating event network-vif-deleted:49869004-34cd-4d2b-8bf2-b7cbd8448d51 for instance fa075dcc-e175-449a-9914-d1ee19131f33 on nch2-master-1 Jul 16 21:23:09.050703 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-346cf172-a6b2-47ca-be60-ddc3ef6e1857 req-4f28a6e2-6e2f-4986-868a-98da3aefa647 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.080284 Jul 16 21:23:09.051797 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1625/3218] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:23:08 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:23:09.090516 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-a877d8eb-dd69-4996-b614-6699a1744fb7 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:23:09.093805 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-a877d8eb-dd69-4996-b614-6699a1744fb7 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] [instance: f114e124-2903-4f30-b055-9f493e3e9e6c] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:23:09.101758 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a877d8eb-dd69-4996-b614-6699a1744fb7 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:09.102227 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a877d8eb-dd69-4996-b614-6699a1744fb7 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:09.103612 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a877d8eb-dd69-4996-b614-6699a1744fb7 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:09.104128 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a877d8eb-dd69-4996-b614-6699a1744fb7 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:09.208467 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-a4e0948e-4917-4ecc-807e-2d35e26159d8 tempest-ServerAddressesNegativeTestJSON-1415290150 tempest-ServerAddressesNegativeTestJSON-1415290150-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ab7326ef48042fe961a7eed5c46db24', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c0ebaf2e3714e42b3bda726e2133522', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:23:09.215271 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-a4e0948e-4917-4ecc-807e-2d35e26159d8 tempest-ServerAddressesNegativeTestJSON-1415290150 tempest-ServerAddressesNegativeTestJSON-1415290150-project] 10.222.0.16 "GET /compute/v2.1/servers/43a4dc6f-5d4d-45d7-bc69-0c754c2ec27a" status: 200 len: 1529 microversion: 2.1 time: 0.431364 Jul 16 21:23:09.218590 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1594/3219] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:23:08 2022] GET /compute/v2.1/servers/43a4dc6f-5d4d-45d7-bc69-0c754c2ec27a => generated 1529 bytes in 436 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:23:09.229063 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-4011d770-eb50-4912-ad45-a51cde82c858 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:23:09.239360 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.objects.instance [None req-a877d8eb-dd69-4996-b614-6699a1744fb7 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lazy-loading 'services' on Instance uuid f114e124-2903-4f30-b055-9f493e3e9e6c {{(pid=138680) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:23:09.239938 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-4011d770-eb50-4912-ad45-a51cde82c858 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] [instance: 583e0c15-3053-429d-81bf-95a4d1f99cbf] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:23:09.255044 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4011d770-eb50-4912-ad45-a51cde82c858 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:09.255763 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4011d770-eb50-4912-ad45-a51cde82c858 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:09.256776 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4011d770-eb50-4912-ad45-a51cde82c858 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:09.257279 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4011d770-eb50-4912-ad45-a51cde82c858 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:09.296298 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-4011d770-eb50-4912-ad45-a51cde82c858 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] HTTP exception thrown: Instance 583e0c15-3053-429d-81bf-95a4d1f99cbf could not be found. Jul 16 21:23:09.298686 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-4011d770-eb50-4912-ad45-a51cde82c858 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Request method failure captured: Jul 16 21:23:09.298686 ncdv-master-1 devstack@n-api.service[138681]: request: GET /compute/v2.1/servers/583e0c15-3053-429d-81bf-95a4d1f99cbf HTTP/1.1 Jul 16 21:23:09.298686 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:23:09.298686 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:23:09.298686 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:23:09.298686 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 21:23:09.298686 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:23:09.298686 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:23:09.298686 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:23:09.298686 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:23:09.298686 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0yvhcWChos9BPc5M9iEHGYvPQzsomAdcS5MsL8XhOqLua3SuKyjZpXrhd0rudyICHCp_Hm88OIddNnlewL24O3YECF7zSPVnlXrLvt4tV8I_QcE15a183juS-VFME42X6bCZkTP0P0AFfI36Ps4vQJL_9FT_dIrXsYJlcFjMb6_LWLs Jul 16 21:23:09.298686 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:23:09.298686 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:23:09.298686 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:23:09.298686 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:23:09.298686 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:23:09.298686 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:23:09.298686 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: 65644a8df5d84df3b35c894560e0a5c4 Jul 16 21:23:09.298686 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-ServerRescueTestJSONUnderV235-139726075 Jul 16 21:23:09.298686 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:23:09.298686 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:23:09.298686 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/65644a8df5d84df3b35c894560e0a5c4"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/65644a8df5d84df3b35c894560e0a5c4"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_65644a8df5d84df3b35c894560e0a5c4"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/65644a8df5d84df3b35c894560e0a5c4"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:23:09.298686 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-ServerRescueTestJSONUnderV235-139726075 Jul 16 21:23:09.298686 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: 65644a8df5d84df3b35c894560e0a5c4 Jul 16 21:23:09.298686 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-ServerRescueTestJSONUnderV235-139726075 Jul 16 21:23:09.298686 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-ServerRescueTestJSONUnderV235-139726075-project Jul 16 21:23:09.298686 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:23:09.298686 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:23:09.298686 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: aaf2292818b749f0ae43c66bca929be9 Jul 16 21:23:09.298686 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-ServerRescueTestJSONUnderV235-139726075-project Jul 16 21:23:09.298686 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:23:09.298686 ncdv-master-1 devstack@n-api.service[138681]: exception: Instance 583e0c15-3053-429d-81bf-95a4d1f99cbf could not be found. Jul 16 21:23:09.298686 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': '583e0c15-3053-429d-81bf-95a4d1f99cbf'} Jul 16 21:23:09.298686 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:23:09.298686 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:23:09.298686 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:23:09.298686 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:23:09.298686 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:23:09.298686 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:23:09.298686 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:23:09.298686 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:23:09.298686 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:23:09.298686 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:23:09.298686 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 583e0c15-3053-429d-81bf-95a4d1f99cbf could not be found. Jul 16 21:23:09.298686 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:23:09.298686 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:23:09.298686 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:23:09.298686 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:23:09.298686 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:23:09.298686 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:23:09.298686 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:23:09.298686 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:23:09.298686 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:23:09.298686 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:23:09.298686 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:23:09.298686 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:23:09.298686 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:23:09.298686 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:23:09.298686 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:23:09.298686 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:23:09.298686 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 583e0c15-3053-429d-81bf-95a4d1f99cbf could not be found. Jul 16 21:23:09.298686 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:23:09.298686 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:23:09.304561 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:23:09.304561 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:23:09.304561 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:23:09.304561 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:23:09.304561 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:23:09.304561 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:23:09.304561 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 583e0c15-3053-429d-81bf-95a4d1f99cbf could not be found. Jul 16 21:23:09.304561 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:23:09.306852 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-4011d770-eb50-4912-ad45-a51cde82c858 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] Returning 404 to user: Instance 583e0c15-3053-429d-81bf-95a4d1f99cbf could not be found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:23:09.307827 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-4011d770-eb50-4912-ad45-a51cde82c858 tempest-ServerRescueTestJSONUnderV235-139726075 tempest-ServerRescueTestJSONUnderV235-139726075-project] 10.222.0.16 "GET /compute/v2.1/servers/583e0c15-3053-429d-81bf-95a4d1f99cbf" status: 404 len: 111 microversion: 2.1 time: 0.087978 Jul 16 21:23:09.308590 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1595/3220] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:23:09 2022] GET /compute/v2.1/servers/583e0c15-3053-429d-81bf-95a4d1f99cbf => generated 111 bytes in 89 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:23:09.424972 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-453e5516-0e72-4945-b21f-230ac9c9226b tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:23:09.427240 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-453e5516-0e72-4945-b21f-230ac9c9226b tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 1cb418f7-6aea-42e6-88a2-007564d925c4] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:23:09.444936 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-453e5516-0e72-4945-b21f-230ac9c9226b tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:09.445767 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-453e5516-0e72-4945-b21f-230ac9c9226b tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:09.447542 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-453e5516-0e72-4945-b21f-230ac9c9226b tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:09.450165 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-453e5516-0e72-4945-b21f-230ac9c9226b tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:09.528455 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-a877d8eb-dd69-4996-b614-6699a1744fb7 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] 10.222.0.16 "DELETE /compute/v2.1/servers/f114e124-2903-4f30-b055-9f493e3e9e6c/os-volume_attachments/7734c56d-3537-4b9a-b18e-78dda7314333" status: 202 len: 0 microversion: 2.1 time: 0.446329 Jul 16 21:23:09.530426 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1626/3221] 10.222.0.16 () {58 vars in 1415 bytes} [Sat Jul 16 21:23:09 2022] DELETE /compute/v2.1/servers/f114e124-2903-4f30-b055-9f493e3e9e6c/os-volume_attachments/7734c56d-3537-4b9a-b18e-78dda7314333 => generated 0 bytes in 450 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Jul 16 21:23:09.941403 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-f77b309c-8d39-4c1f-b80e-82ff1868863b tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:23:09.943373 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-f77b309c-8d39-4c1f-b80e-82ff1868863b tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] [instance: fa075dcc-e175-449a-9914-d1ee19131f33] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:23:09.945737 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-453e5516-0e72-4945-b21f-230ac9c9226b tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f2e974ab5aa48dfb4b00f810dcedb63', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd31be8475a6a49d6bfe8132dd050c6c0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:23:09.953811 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f77b309c-8d39-4c1f-b80e-82ff1868863b tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:09.954999 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f77b309c-8d39-4c1f-b80e-82ff1868863b tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:09.956803 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f77b309c-8d39-4c1f-b80e-82ff1868863b tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:09.956929 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-453e5516-0e72-4945-b21f-230ac9c9226b tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "GET /compute/v2.1/servers/1cb418f7-6aea-42e6-88a2-007564d925c4" status: 200 len: 1315 microversion: 2.1 time: 0.538420 Jul 16 21:23:09.957673 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1596/3222] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:23:09 2022] GET /compute/v2.1/servers/1cb418f7-6aea-42e6-88a2-007564d925c4 => generated 1315 bytes in 540 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:23:09.958102 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f77b309c-8d39-4c1f-b80e-82ff1868863b tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:09.963158 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=99,nova_cell1:INSERT=1 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:23:10.002882 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f77b309c-8d39-4c1f-b80e-82ff1868863b tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:10.003507 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f77b309c-8d39-4c1f-b80e-82ff1868863b tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:10.004043 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.objects.instance [None req-f77b309c-8d39-4c1f-b80e-82ff1868863b tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Lazy-loading 'fault' on Instance uuid fa075dcc-e175-449a-9914-d1ee19131f33 {{(pid=138680) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:23:10.102089 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-f77b309c-8d39-4c1f-b80e-82ff1868863b tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '029f1334779e4bcdb3f4f2aed564a075', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e1f6ffc3fb64dba8c4474641e796fb8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:23:10.111867 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-f77b309c-8d39-4c1f-b80e-82ff1868863b tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] 10.222.0.16 "GET /compute/v2.1/servers/fa075dcc-e175-449a-9914-d1ee19131f33" status: 200 len: 1331 microversion: 2.1 time: 0.178181 Jul 16 21:23:10.112837 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1627/3223] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:23:09 2022] GET /compute/v2.1/servers/fa075dcc-e175-449a-9914-d1ee19131f33 => generated 1331 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:23:10.246278 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-2b9a71ef-06c3-418d-b3cf-7bda5ba6e5c5 tempest-ServerAddressesNegativeTestJSON-1415290150 tempest-ServerAddressesNegativeTestJSON-1415290150-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:23:10.246914 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-2b9a71ef-06c3-418d-b3cf-7bda5ba6e5c5 tempest-ServerAddressesNegativeTestJSON-1415290150 tempest-ServerAddressesNegativeTestJSON-1415290150-project] [instance: 43a4dc6f-5d4d-45d7-bc69-0c754c2ec27a] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:23:10.263560 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2b9a71ef-06c3-418d-b3cf-7bda5ba6e5c5 tempest-ServerAddressesNegativeTestJSON-1415290150 tempest-ServerAddressesNegativeTestJSON-1415290150-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:10.266596 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2b9a71ef-06c3-418d-b3cf-7bda5ba6e5c5 tempest-ServerAddressesNegativeTestJSON-1415290150 tempest-ServerAddressesNegativeTestJSON-1415290150-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:10.267797 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2b9a71ef-06c3-418d-b3cf-7bda5ba6e5c5 tempest-ServerAddressesNegativeTestJSON-1415290150 tempest-ServerAddressesNegativeTestJSON-1415290150-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:10.270063 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2b9a71ef-06c3-418d-b3cf-7bda5ba6e5c5 tempest-ServerAddressesNegativeTestJSON-1415290150 tempest-ServerAddressesNegativeTestJSON-1415290150-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:10.592591 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-2b9a71ef-06c3-418d-b3cf-7bda5ba6e5c5 tempest-ServerAddressesNegativeTestJSON-1415290150 tempest-ServerAddressesNegativeTestJSON-1415290150-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ab7326ef48042fe961a7eed5c46db24', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c0ebaf2e3714e42b3bda726e2133522', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:23:10.607308 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-2b9a71ef-06c3-418d-b3cf-7bda5ba6e5c5 tempest-ServerAddressesNegativeTestJSON-1415290150 tempest-ServerAddressesNegativeTestJSON-1415290150-project] 10.222.0.16 "GET /compute/v2.1/servers/43a4dc6f-5d4d-45d7-bc69-0c754c2ec27a" status: 200 len: 1529 microversion: 2.1 time: 0.375608 Jul 16 21:23:10.608422 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1597/3224] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:23:10 2022] GET /compute/v2.1/servers/43a4dc6f-5d4d-45d7-bc69-0c754c2ec27a => generated 1529 bytes in 378 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:23:10.711255 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-59839e5f-177a-4d42-970c-e4694623cceb tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Action: 'action', calling method: >, body: {"os-stop": {}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:23:10.720055 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_api:SELECT=66 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:23:10.727031 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-59839e5f-177a-4d42-970c-e4694623cceb tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:10.727893 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-59839e5f-177a-4d42-970c-e4694623cceb tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:10.763531 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-59839e5f-177a-4d42-970c-e4694623cceb tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] [instance: f114e124-2903-4f30-b055-9f493e3e9e6c] Going to try to stop instance {{(pid=138680) force_stop /opt/stack/nova/nova/compute/api.py:2707}} Jul 16 21:23:10.990773 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-de6cc837-75b0-459e-961a-2a625e297b37 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:23:10.992273 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-de6cc837-75b0-459e-961a-2a625e297b37 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 1cb418f7-6aea-42e6-88a2-007564d925c4] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:23:11.007041 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-59839e5f-177a-4d42-970c-e4694623cceb tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] 10.222.0.16 "POST /compute/v2.1/servers/f114e124-2903-4f30-b055-9f493e3e9e6c/action" status: 202 len: 0 microversion: 2.1 time: 0.295829 Jul 16 21:23:11.007041 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1628/3225] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 21:23:10 2022] POST /compute/v2.1/servers/f114e124-2903-4f30-b055-9f493e3e9e6c/action => generated 0 bytes in 298 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 21:23:11.020327 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-1f19627b-d611-4696-b118-b3e671cdfb05 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:23:11.021042 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-1f19627b-d611-4696-b118-b3e671cdfb05 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] [instance: f114e124-2903-4f30-b055-9f493e3e9e6c] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:23:11.024658 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell0:SELECT=4 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:23:11.030781 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-de6cc837-75b0-459e-961a-2a625e297b37 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:11.035425 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-de6cc837-75b0-459e-961a-2a625e297b37 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:11.036540 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-de6cc837-75b0-459e-961a-2a625e297b37 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:11.039196 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-de6cc837-75b0-459e-961a-2a625e297b37 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:11.040671 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1f19627b-d611-4696-b118-b3e671cdfb05 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:11.041403 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell0:SELECT=26 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:23:11.045030 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1f19627b-d611-4696-b118-b3e671cdfb05 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:11.052403 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1f19627b-d611-4696-b118-b3e671cdfb05 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:11.052873 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1f19627b-d611-4696-b118-b3e671cdfb05 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:11.499066 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-de6cc837-75b0-459e-961a-2a625e297b37 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f2e974ab5aa48dfb4b00f810dcedb63', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd31be8475a6a49d6bfe8132dd050c6c0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:23:11.513410 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-de6cc837-75b0-459e-961a-2a625e297b37 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "GET /compute/v2.1/servers/1cb418f7-6aea-42e6-88a2-007564d925c4" status: 200 len: 1315 microversion: 2.1 time: 0.541765 Jul 16 21:23:11.514624 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1598/3226] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:23:10 2022] GET /compute/v2.1/servers/1cb418f7-6aea-42e6-88a2-007564d925c4 => generated 1315 bytes in 544 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:23:11.520269 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-c6e6da23-4623-4ccd-83c6-98e2d3620b05 tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:23:11.521323 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-c6e6da23-4623-4ccd-83c6-98e2d3620b05 tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] [instance: fa075dcc-e175-449a-9914-d1ee19131f33] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:23:11.533557 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c6e6da23-4623-4ccd-83c6-98e2d3620b05 tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:11.533557 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c6e6da23-4623-4ccd-83c6-98e2d3620b05 tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:11.533557 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c6e6da23-4623-4ccd-83c6-98e2d3620b05 tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:11.533557 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c6e6da23-4623-4ccd-83c6-98e2d3620b05 tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:11.535722 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-1f19627b-d611-4696-b118-b3e671cdfb05 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bba77f6ef42547f1aa8480e27868bd4a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '22ca92a86e6c40d59071e58ce4085568', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:23:11.550006 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-c6e6da23-4623-4ccd-83c6-98e2d3620b05 tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] HTTP exception thrown: Instance fa075dcc-e175-449a-9914-d1ee19131f33 could not be found. Jul 16 21:23:11.550856 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-1f19627b-d611-4696-b118-b3e671cdfb05 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] 10.222.0.16 "GET /compute/v2.1/servers/f114e124-2903-4f30-b055-9f493e3e9e6c" status: 200 len: 1679 microversion: 2.1 time: 0.541786 Jul 16 21:23:11.551669 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1629/3227] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:23:11 2022] GET /compute/v2.1/servers/f114e124-2903-4f30-b055-9f493e3e9e6c => generated 1679 bytes in 543 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:23:11.552626 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-c6e6da23-4623-4ccd-83c6-98e2d3620b05 tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Request method failure captured: Jul 16 21:23:11.552626 ncdv-master-1 devstack@n-api.service[138681]: request: GET /compute/v2.1/servers/fa075dcc-e175-449a-9914-d1ee19131f33 HTTP/1.1 Jul 16 21:23:11.552626 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:23:11.552626 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:23:11.552626 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:23:11.552626 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 21:23:11.552626 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:23:11.552626 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:23:11.552626 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:23:11.552626 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:23:11.552626 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0yukIMJ48tRgCB3diR5B2yTdgfCCN7-OZTMmtI4TH-F_hINl86vSpH64fFhpc-NX4fmSyldmbblkIl17gqyuR9fatu2VYxfrnJDqp4uflU3P7tQcLJTnZudoBO8vc10rI0Ks_An4UJorcnH-Ege1HUXGbj4nJCfyRf8zOh_B8VIV5H0 Jul 16 21:23:11.552626 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:23:11.552626 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:23:11.552626 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:23:11.552626 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:23:11.552626 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:23:11.552626 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:23:11.552626 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: 3e1f6ffc3fb64dba8c4474641e796fb8 Jul 16 21:23:11.552626 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-AttachSCSIVolumeTestJSON-288658511 Jul 16 21:23:11.552626 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:23:11.552626 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:23:11.552626 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/3e1f6ffc3fb64dba8c4474641e796fb8"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/3e1f6ffc3fb64dba8c4474641e796fb8"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_3e1f6ffc3fb64dba8c4474641e796fb8"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/3e1f6ffc3fb64dba8c4474641e796fb8"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:23:11.552626 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-AttachSCSIVolumeTestJSON-288658511 Jul 16 21:23:11.552626 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: 3e1f6ffc3fb64dba8c4474641e796fb8 Jul 16 21:23:11.552626 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-AttachSCSIVolumeTestJSON-288658511 Jul 16 21:23:11.552626 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-AttachSCSIVolumeTestJSON-288658511-project Jul 16 21:23:11.552626 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:23:11.552626 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:23:11.552626 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: 029f1334779e4bcdb3f4f2aed564a075 Jul 16 21:23:11.552626 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-AttachSCSIVolumeTestJSON-288658511-project Jul 16 21:23:11.552626 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:23:11.552626 ncdv-master-1 devstack@n-api.service[138681]: exception: Instance fa075dcc-e175-449a-9914-d1ee19131f33 could not be found. Jul 16 21:23:11.552626 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': 'fa075dcc-e175-449a-9914-d1ee19131f33'} Jul 16 21:23:11.552626 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:23:11.552626 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:23:11.552626 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:23:11.552626 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:23:11.552626 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:23:11.552626 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:23:11.552626 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:23:11.552626 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:23:11.552626 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:23:11.552626 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:23:11.552626 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance fa075dcc-e175-449a-9914-d1ee19131f33 could not be found. Jul 16 21:23:11.552626 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:23:11.552626 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:23:11.552626 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:23:11.552626 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:23:11.552626 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:23:11.552626 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:23:11.552626 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:23:11.552626 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:23:11.552626 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:23:11.552626 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:23:11.552626 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:23:11.552626 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:23:11.552626 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:23:11.552626 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:23:11.552626 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:23:11.552626 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:23:11.552626 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance fa075dcc-e175-449a-9914-d1ee19131f33 could not be found. Jul 16 21:23:11.552626 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:23:11.552626 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:23:11.552626 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:23:11.556634 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:23:11.556634 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:23:11.556634 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:23:11.556634 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:23:11.556634 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:23:11.556634 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance fa075dcc-e175-449a-9914-d1ee19131f33 could not be found. Jul 16 21:23:11.556634 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:23:11.558685 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-c6e6da23-4623-4ccd-83c6-98e2d3620b05 tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Returning 404 to user: Instance fa075dcc-e175-449a-9914-d1ee19131f33 could not be found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:23:11.559848 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-c6e6da23-4623-4ccd-83c6-98e2d3620b05 tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] 10.222.0.16 "GET /compute/v2.1/servers/fa075dcc-e175-449a-9914-d1ee19131f33" status: 404 len: 111 microversion: 2.1 time: 0.043676 Jul 16 21:23:11.561129 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1599/3228] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:23:11 2022] GET /compute/v2.1/servers/fa075dcc-e175-449a-9914-d1ee19131f33 => generated 111 bytes in 45 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:23:11.571849 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-ffdfd1fe-51b2-4104-a4a9-01886639ccd7 tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Calling method '.version_select at 0x7fdd9c4f3280>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:23:11.633537 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-54da9038-6e8e-4c7f-a3fb-32fa59996cd7 tempest-ServerAddressesNegativeTestJSON-1415290150 tempest-ServerAddressesNegativeTestJSON-1415290150-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:23:11.634348 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-54da9038-6e8e-4c7f-a3fb-32fa59996cd7 tempest-ServerAddressesNegativeTestJSON-1415290150 tempest-ServerAddressesNegativeTestJSON-1415290150-project] [instance: 43a4dc6f-5d4d-45d7-bc69-0c754c2ec27a] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:23:11.641665 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-54da9038-6e8e-4c7f-a3fb-32fa59996cd7 tempest-ServerAddressesNegativeTestJSON-1415290150 tempest-ServerAddressesNegativeTestJSON-1415290150-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:11.642477 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-54da9038-6e8e-4c7f-a3fb-32fa59996cd7 tempest-ServerAddressesNegativeTestJSON-1415290150 tempest-ServerAddressesNegativeTestJSON-1415290150-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:11.643498 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-54da9038-6e8e-4c7f-a3fb-32fa59996cd7 tempest-ServerAddressesNegativeTestJSON-1415290150 tempest-ServerAddressesNegativeTestJSON-1415290150-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:11.644049 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-54da9038-6e8e-4c7f-a3fb-32fa59996cd7 tempest-ServerAddressesNegativeTestJSON-1415290150 tempest-ServerAddressesNegativeTestJSON-1415290150-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:11.648631 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=96,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:23:11.712553 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-ffdfd1fe-51b2-4104-a4a9-01886639ccd7 tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] 10.222.0.16 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-492746964" status: 202 len: 0 microversion: 2.1 time: 0.144530 Jul 16 21:23:11.713295 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1630/3229] 10.222.0.16 () {58 vars in 1217 bytes} [Sat Jul 16 21:23:11 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-492746964 => generated 0 bytes in 146 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 21:23:11.840070 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell0:SELECT=2 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:23:11.846378 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-b19e022e-48ce-4192-b721-11d45066f794 req-54a03db3-e283-4896-bbf0-eddc13c00a04 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "1cb418f7-6aea-42e6-88a2-007564d925c4", "tag": "ac40e60b-9f21-4db1-9b3a-182c4a3e4316"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:23:11.864934 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-b19e022e-48ce-4192-b721-11d45066f794 req-54a03db3-e283-4896-bbf0-eddc13c00a04 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:11.868025 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-b19e022e-48ce-4192-b721-11d45066f794 req-54a03db3-e283-4896-bbf0-eddc13c00a04 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:11.880264 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-b19e022e-48ce-4192-b721-11d45066f794 req-54a03db3-e283-4896-bbf0-eddc13c00a04 service nova] Creating event network-changed:ac40e60b-9f21-4db1-9b3a-182c4a3e4316 for instance 1cb418f7-6aea-42e6-88a2-007564d925c4 on nch1-master-1 Jul 16 21:23:11.886728 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-b19e022e-48ce-4192-b721-11d45066f794 req-54a03db3-e283-4896-bbf0-eddc13c00a04 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.049899 Jul 16 21:23:11.887385 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1631/3230] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:23:11 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:23:12.072958 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-54da9038-6e8e-4c7f-a3fb-32fa59996cd7 tempest-ServerAddressesNegativeTestJSON-1415290150 tempest-ServerAddressesNegativeTestJSON-1415290150-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ab7326ef48042fe961a7eed5c46db24', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c0ebaf2e3714e42b3bda726e2133522', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:23:12.084683 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-54da9038-6e8e-4c7f-a3fb-32fa59996cd7 tempest-ServerAddressesNegativeTestJSON-1415290150 tempest-ServerAddressesNegativeTestJSON-1415290150-project] 10.222.0.16 "GET /compute/v2.1/servers/43a4dc6f-5d4d-45d7-bc69-0c754c2ec27a" status: 200 len: 1529 microversion: 2.1 time: 0.455218 Jul 16 21:23:12.096335 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1600/3231] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:23:11 2022] GET /compute/v2.1/servers/43a4dc6f-5d4d-45d7-bc69-0c754c2ec27a => generated 1529 bytes in 468 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:23:12.535283 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-30746458-f770-47f5-901c-13c50b3782bf tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:23:12.537015 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-30746458-f770-47f5-901c-13c50b3782bf tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 1cb418f7-6aea-42e6-88a2-007564d925c4] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:23:12.543950 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-30746458-f770-47f5-901c-13c50b3782bf tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:12.544617 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-30746458-f770-47f5-901c-13c50b3782bf tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:12.545630 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-30746458-f770-47f5-901c-13c50b3782bf tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:12.546241 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-30746458-f770-47f5-901c-13c50b3782bf tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:12.569915 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-9000d35e-6ff7-4f4f-96a2-e2b6c6aa4983 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:23:12.570548 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-9000d35e-6ff7-4f4f-96a2-e2b6c6aa4983 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] [instance: f114e124-2903-4f30-b055-9f493e3e9e6c] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:23:12.579405 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9000d35e-6ff7-4f4f-96a2-e2b6c6aa4983 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:12.590204 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9000d35e-6ff7-4f4f-96a2-e2b6c6aa4983 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:12.591213 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9000d35e-6ff7-4f4f-96a2-e2b6c6aa4983 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:12.592452 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9000d35e-6ff7-4f4f-96a2-e2b6c6aa4983 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:13.330612 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-30746458-f770-47f5-901c-13c50b3782bf tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f2e974ab5aa48dfb4b00f810dcedb63', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd31be8475a6a49d6bfe8132dd050c6c0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:23:13.346463 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-30746458-f770-47f5-901c-13c50b3782bf tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "GET /compute/v2.1/servers/1cb418f7-6aea-42e6-88a2-007564d925c4" status: 200 len: 1315 microversion: 2.1 time: 0.815760 Jul 16 21:23:13.347720 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1632/3232] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:23:12 2022] GET /compute/v2.1/servers/1cb418f7-6aea-42e6-88a2-007564d925c4 => generated 1315 bytes in 820 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:23:13.357231 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-07887534-b150-43ad-a586-c1602ee54833 tempest-ServerAddressesNegativeTestJSON-1415290150 tempest-ServerAddressesNegativeTestJSON-1415290150-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:23:13.358827 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-07887534-b150-43ad-a586-c1602ee54833 tempest-ServerAddressesNegativeTestJSON-1415290150 tempest-ServerAddressesNegativeTestJSON-1415290150-project] [instance: 43a4dc6f-5d4d-45d7-bc69-0c754c2ec27a] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:23:13.365748 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-07887534-b150-43ad-a586-c1602ee54833 tempest-ServerAddressesNegativeTestJSON-1415290150 tempest-ServerAddressesNegativeTestJSON-1415290150-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:13.367470 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-07887534-b150-43ad-a586-c1602ee54833 tempest-ServerAddressesNegativeTestJSON-1415290150 tempest-ServerAddressesNegativeTestJSON-1415290150-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:13.367470 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-07887534-b150-43ad-a586-c1602ee54833 tempest-ServerAddressesNegativeTestJSON-1415290150 tempest-ServerAddressesNegativeTestJSON-1415290150-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:13.369141 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-07887534-b150-43ad-a586-c1602ee54833 tempest-ServerAddressesNegativeTestJSON-1415290150 tempest-ServerAddressesNegativeTestJSON-1415290150-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:13.450633 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-07887534-b150-43ad-a586-c1602ee54833 tempest-ServerAddressesNegativeTestJSON-1415290150 tempest-ServerAddressesNegativeTestJSON-1415290150-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:13.453134 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-07887534-b150-43ad-a586-c1602ee54833 tempest-ServerAddressesNegativeTestJSON-1415290150 tempest-ServerAddressesNegativeTestJSON-1415290150-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:13.455856 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.objects.instance [None req-07887534-b150-43ad-a586-c1602ee54833 tempest-ServerAddressesNegativeTestJSON-1415290150 tempest-ServerAddressesNegativeTestJSON-1415290150-project] Lazy-loading 'fault' on Instance uuid 43a4dc6f-5d4d-45d7-bc69-0c754c2ec27a {{(pid=138680) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:23:13.512626 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-9000d35e-6ff7-4f4f-96a2-e2b6c6aa4983 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bba77f6ef42547f1aa8480e27868bd4a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '22ca92a86e6c40d59071e58ce4085568', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:23:13.531728 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-9000d35e-6ff7-4f4f-96a2-e2b6c6aa4983 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] 10.222.0.16 "GET /compute/v2.1/servers/f114e124-2903-4f30-b055-9f493e3e9e6c" status: 200 len: 1679 microversion: 2.1 time: 0.962847 Jul 16 21:23:13.533376 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1601/3233] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:23:12 2022] GET /compute/v2.1/servers/f114e124-2903-4f30-b055-9f493e3e9e6c => generated 1679 bytes in 967 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:23:13.750273 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-07887534-b150-43ad-a586-c1602ee54833 tempest-ServerAddressesNegativeTestJSON-1415290150 tempest-ServerAddressesNegativeTestJSON-1415290150-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ab7326ef48042fe961a7eed5c46db24', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c0ebaf2e3714e42b3bda726e2133522', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:23:13.763303 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-07887534-b150-43ad-a586-c1602ee54833 tempest-ServerAddressesNegativeTestJSON-1415290150 tempest-ServerAddressesNegativeTestJSON-1415290150-project] 10.222.0.16 "GET /compute/v2.1/servers/43a4dc6f-5d4d-45d7-bc69-0c754c2ec27a" status: 200 len: 1315 microversion: 2.1 time: 0.413878 Jul 16 21:23:13.774635 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1633/3234] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:23:13 2022] GET /compute/v2.1/servers/43a4dc6f-5d4d-45d7-bc69-0c754c2ec27a => generated 1315 bytes in 426 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:23:13.921467 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-650d66b6-f948-4d33-9e3f-b670d5cebab3 req-2068ba6f-2e9c-44b4-b7ef-a29fa4d9bc89 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "43a4dc6f-5d4d-45d7-bc69-0c754c2ec27a", "name": "network-vif-deleted", "tag": "4737b0a0-7745-4eb0-a84a-a81682e7532e"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:23:13.934206 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-650d66b6-f948-4d33-9e3f-b670d5cebab3 req-2068ba6f-2e9c-44b4-b7ef-a29fa4d9bc89 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:13.934648 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-650d66b6-f948-4d33-9e3f-b670d5cebab3 req-2068ba6f-2e9c-44b4-b7ef-a29fa4d9bc89 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:13.956675 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-650d66b6-f948-4d33-9e3f-b670d5cebab3 req-2068ba6f-2e9c-44b4-b7ef-a29fa4d9bc89 service nova] Creating event network-vif-deleted:4737b0a0-7745-4eb0-a84a-a81682e7532e for instance 43a4dc6f-5d4d-45d7-bc69-0c754c2ec27a on nch2-master-1 Jul 16 21:23:13.965583 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-650d66b6-f948-4d33-9e3f-b670d5cebab3 req-2068ba6f-2e9c-44b4-b7ef-a29fa4d9bc89 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.048309 Jul 16 21:23:13.966250 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1602/3235] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:23:13 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:23:14.365837 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-cf6b0623-80a3-4110-bf46-ce2f4682f8b2 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:23:14.372269 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-cf6b0623-80a3-4110-bf46-ce2f4682f8b2 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 1cb418f7-6aea-42e6-88a2-007564d925c4] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:23:14.383188 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cf6b0623-80a3-4110-bf46-ce2f4682f8b2 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:14.383658 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cf6b0623-80a3-4110-bf46-ce2f4682f8b2 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:14.386447 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cf6b0623-80a3-4110-bf46-ce2f4682f8b2 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:14.390506 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cf6b0623-80a3-4110-bf46-ce2f4682f8b2 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:14.552177 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-24095c07-dffe-4443-a39f-fc11c03a6efc tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:23:14.554327 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-24095c07-dffe-4443-a39f-fc11c03a6efc tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] [instance: f114e124-2903-4f30-b055-9f493e3e9e6c] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:23:14.580940 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-24095c07-dffe-4443-a39f-fc11c03a6efc tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:14.581407 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-24095c07-dffe-4443-a39f-fc11c03a6efc tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:14.588110 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-24095c07-dffe-4443-a39f-fc11c03a6efc tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:14.588676 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-24095c07-dffe-4443-a39f-fc11c03a6efc tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:14.730126 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-cf6b0623-80a3-4110-bf46-ce2f4682f8b2 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f2e974ab5aa48dfb4b00f810dcedb63', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd31be8475a6a49d6bfe8132dd050c6c0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:23:14.746018 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-cf6b0623-80a3-4110-bf46-ce2f4682f8b2 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "GET /compute/v2.1/servers/1cb418f7-6aea-42e6-88a2-007564d925c4" status: 200 len: 1315 microversion: 2.1 time: 0.385214 Jul 16 21:23:14.760765 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1634/3236] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:23:14 2022] GET /compute/v2.1/servers/1cb418f7-6aea-42e6-88a2-007564d925c4 => generated 1315 bytes in 401 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:23:14.784330 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-d6d3b556-fccd-4862-ad1d-aa5287c47cf2 tempest-ServerAddressesNegativeTestJSON-1415290150 tempest-ServerAddressesNegativeTestJSON-1415290150-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:23:14.785263 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-d6d3b556-fccd-4862-ad1d-aa5287c47cf2 tempest-ServerAddressesNegativeTestJSON-1415290150 tempest-ServerAddressesNegativeTestJSON-1415290150-project] [instance: 43a4dc6f-5d4d-45d7-bc69-0c754c2ec27a] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:23:14.793444 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d6d3b556-fccd-4862-ad1d-aa5287c47cf2 tempest-ServerAddressesNegativeTestJSON-1415290150 tempest-ServerAddressesNegativeTestJSON-1415290150-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:14.793825 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d6d3b556-fccd-4862-ad1d-aa5287c47cf2 tempest-ServerAddressesNegativeTestJSON-1415290150 tempest-ServerAddressesNegativeTestJSON-1415290150-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:14.795972 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d6d3b556-fccd-4862-ad1d-aa5287c47cf2 tempest-ServerAddressesNegativeTestJSON-1415290150 tempest-ServerAddressesNegativeTestJSON-1415290150-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:14.796445 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d6d3b556-fccd-4862-ad1d-aa5287c47cf2 tempest-ServerAddressesNegativeTestJSON-1415290150 tempest-ServerAddressesNegativeTestJSON-1415290150-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:14.807886 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-24095c07-dffe-4443-a39f-fc11c03a6efc tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bba77f6ef42547f1aa8480e27868bd4a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '22ca92a86e6c40d59071e58ce4085568', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:23:14.816429 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-24095c07-dffe-4443-a39f-fc11c03a6efc tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] 10.222.0.16 "GET /compute/v2.1/servers/f114e124-2903-4f30-b055-9f493e3e9e6c" status: 200 len: 1679 microversion: 2.1 time: 0.269909 Jul 16 21:23:14.817537 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1603/3237] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:23:14 2022] GET /compute/v2.1/servers/f114e124-2903-4f30-b055-9f493e3e9e6c => generated 1679 bytes in 272 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:23:14.829568 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-d6d3b556-fccd-4862-ad1d-aa5287c47cf2 tempest-ServerAddressesNegativeTestJSON-1415290150 tempest-ServerAddressesNegativeTestJSON-1415290150-project] HTTP exception thrown: Instance 43a4dc6f-5d4d-45d7-bc69-0c754c2ec27a could not be found. Jul 16 21:23:14.831292 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-d6d3b556-fccd-4862-ad1d-aa5287c47cf2 tempest-ServerAddressesNegativeTestJSON-1415290150 tempest-ServerAddressesNegativeTestJSON-1415290150-project] Request method failure captured: Jul 16 21:23:14.831292 ncdv-master-1 devstack@n-api.service[138680]: request: GET /compute/v2.1/servers/43a4dc6f-5d4d-45d7-bc69-0c754c2ec27a HTTP/1.1 Jul 16 21:23:14.831292 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:23:14.831292 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:23:14.831292 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:23:14.831292 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 0 Jul 16 21:23:14.831292 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:23:14.831292 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:23:14.831292 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:23:14.831292 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:23:14.831292 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0ywkRCV6NNaXk9MtxSRzjRYA4uiu8nVVP2REzd6SnqbgshubqNj0v_3TbWIK63J-IyaMZKkOK2K7t6ZqzT7w_9EGm5KNKhsnau4FHgmclpIu3I5edQQKzyWMc86RwtEygiyD06flnxu7mFd7F5iCzMr2uIcs7XKCJR1_GHAWq2C8AUk Jul 16 21:23:14.831292 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:23:14.831292 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:23:14.831292 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:23:14.831292 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:23:14.831292 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:23:14.831292 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:23:14.831292 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: 7c0ebaf2e3714e42b3bda726e2133522 Jul 16 21:23:14.831292 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-ServerAddressesNegativeTestJSON-1415290150 Jul 16 21:23:14.831292 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:23:14.831292 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:23:14.831292 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/7c0ebaf2e3714e42b3bda726e2133522"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/7c0ebaf2e3714e42b3bda726e2133522"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_7c0ebaf2e3714e42b3bda726e2133522"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/7c0ebaf2e3714e42b3bda726e2133522"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:23:14.831292 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-ServerAddressesNegativeTestJSON-1415290150 Jul 16 21:23:14.831292 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: 7c0ebaf2e3714e42b3bda726e2133522 Jul 16 21:23:14.831292 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-ServerAddressesNegativeTestJSON-1415290150 Jul 16 21:23:14.831292 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-ServerAddressesNegativeTestJSON-1415290150-project Jul 16 21:23:14.831292 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:23:14.831292 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:23:14.831292 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: 3ab7326ef48042fe961a7eed5c46db24 Jul 16 21:23:14.831292 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-ServerAddressesNegativeTestJSON-1415290150-project Jul 16 21:23:14.831292 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 21:23:14.831292 ncdv-master-1 devstack@n-api.service[138680]: exception: Instance 43a4dc6f-5d4d-45d7-bc69-0c754c2ec27a could not be found. Jul 16 21:23:14.831292 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': '43a4dc6f-5d4d-45d7-bc69-0c754c2ec27a'} Jul 16 21:23:14.831292 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:23:14.831292 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:23:14.831292 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:23:14.831292 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:23:14.831292 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:23:14.831292 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:23:14.831292 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:23:14.831292 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:23:14.831292 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:23:14.831292 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:23:14.831292 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 43a4dc6f-5d4d-45d7-bc69-0c754c2ec27a could not be found. Jul 16 21:23:14.831292 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:23:14.831292 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:23:14.831292 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:23:14.831292 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:23:14.831292 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:23:14.831292 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:23:14.831292 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:23:14.831292 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:23:14.831292 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:23:14.831292 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:23:14.831292 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:23:14.831292 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:23:14.831292 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:23:14.831292 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:23:14.831292 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:23:14.831292 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:23:14.831292 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 43a4dc6f-5d4d-45d7-bc69-0c754c2ec27a could not be found. Jul 16 21:23:14.831292 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:23:14.831292 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:23:14.835849 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:23:14.835849 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:23:14.835849 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:23:14.835849 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:23:14.835849 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:23:14.835849 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:23:14.835849 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 43a4dc6f-5d4d-45d7-bc69-0c754c2ec27a could not be found. Jul 16 21:23:14.835849 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:23:14.835849 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-d6d3b556-fccd-4862-ad1d-aa5287c47cf2 tempest-ServerAddressesNegativeTestJSON-1415290150 tempest-ServerAddressesNegativeTestJSON-1415290150-project] Returning 404 to user: Instance 43a4dc6f-5d4d-45d7-bc69-0c754c2ec27a could not be found. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:23:14.835849 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-d6d3b556-fccd-4862-ad1d-aa5287c47cf2 tempest-ServerAddressesNegativeTestJSON-1415290150 tempest-ServerAddressesNegativeTestJSON-1415290150-project] 10.222.0.16 "GET /compute/v2.1/servers/43a4dc6f-5d4d-45d7-bc69-0c754c2ec27a" status: 404 len: 111 microversion: 2.1 time: 0.054039 Jul 16 21:23:14.835849 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1635/3238] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:23:14 2022] GET /compute/v2.1/servers/43a4dc6f-5d4d-45d7-bc69-0c754c2ec27a => generated 111 bytes in 57 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Jul 16 21:23:15.765255 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-6068ddfb-275d-40d7-bf2a-3f0259813e4d tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:23:15.766511 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-6068ddfb-275d-40d7-bf2a-3f0259813e4d tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 1cb418f7-6aea-42e6-88a2-007564d925c4] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:23:15.777493 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6068ddfb-275d-40d7-bf2a-3f0259813e4d tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:15.781879 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6068ddfb-275d-40d7-bf2a-3f0259813e4d tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:15.781879 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6068ddfb-275d-40d7-bf2a-3f0259813e4d tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:15.781879 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6068ddfb-275d-40d7-bf2a-3f0259813e4d tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:15.834184 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-5d52be32-4824-4245-9b62-071da5bb7e34 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:23:15.835603 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-5d52be32-4824-4245-9b62-071da5bb7e34 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] [instance: f114e124-2903-4f30-b055-9f493e3e9e6c] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:23:15.854055 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5d52be32-4824-4245-9b62-071da5bb7e34 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:15.854449 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5d52be32-4824-4245-9b62-071da5bb7e34 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:15.855567 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5d52be32-4824-4245-9b62-071da5bb7e34 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:15.856172 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5d52be32-4824-4245-9b62-071da5bb7e34 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:16.297970 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-5d52be32-4824-4245-9b62-071da5bb7e34 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bba77f6ef42547f1aa8480e27868bd4a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '22ca92a86e6c40d59071e58ce4085568', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:23:16.314314 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-5d52be32-4824-4245-9b62-071da5bb7e34 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] 10.222.0.16 "GET /compute/v2.1/servers/f114e124-2903-4f30-b055-9f493e3e9e6c" status: 200 len: 1679 microversion: 2.1 time: 0.482075 Jul 16 21:23:16.315078 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1636/3239] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:23:15 2022] GET /compute/v2.1/servers/f114e124-2903-4f30-b055-9f493e3e9e6c => generated 1679 bytes in 485 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:23:16.376975 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-6068ddfb-275d-40d7-bf2a-3f0259813e4d tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f2e974ab5aa48dfb4b00f810dcedb63', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd31be8475a6a49d6bfe8132dd050c6c0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:23:16.389887 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-6068ddfb-275d-40d7-bf2a-3f0259813e4d tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "GET /compute/v2.1/servers/1cb418f7-6aea-42e6-88a2-007564d925c4" status: 200 len: 1315 microversion: 2.1 time: 0.629282 Jul 16 21:23:16.392185 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1604/3240] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:23:15 2022] GET /compute/v2.1/servers/1cb418f7-6aea-42e6-88a2-007564d925c4 => generated 1315 bytes in 632 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:23:16.647435 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-ea772e38-84a6-47d2-bc52-894511c8c31e tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:23:16.647944 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-ea772e38-84a6-47d2-bc52-894511c8c31e tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] [instance: b9f2c527-6346-4403-841e-b7bd876a5fb7] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:23:16.657181 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ea772e38-84a6-47d2-bc52-894511c8c31e tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:16.660989 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ea772e38-84a6-47d2-bc52-894511c8c31e tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:16.663998 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ea772e38-84a6-47d2-bc52-894511c8c31e tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:16.664461 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ea772e38-84a6-47d2-bc52-894511c8c31e tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:16.728190 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-ea772e38-84a6-47d2-bc52-894511c8c31e tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] [instance: b9f2c527-6346-4403-841e-b7bd876a5fb7] Going to try to terminate instance {{(pid=138680) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 16 21:23:17.198601 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-ea772e38-84a6-47d2-bc52-894511c8c31e tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] 10.222.0.16 "DELETE /compute/v2.1/servers/b9f2c527-6346-4403-841e-b7bd876a5fb7" status: 204 len: 0 microversion: 2.1 time: 0.879367 Jul 16 21:23:17.201076 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1637/3241] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:23:16 2022] DELETE /compute/v2.1/servers/b9f2c527-6346-4403-841e-b7bd876a5fb7 => generated 0 bytes in 885 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 16 21:23:17.210082 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-02148b2e-5918-493b-8087-89df8a5f3d28 tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:23:17.211551 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-02148b2e-5918-493b-8087-89df8a5f3d28 tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] [instance: b9f2c527-6346-4403-841e-b7bd876a5fb7] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:23:17.220575 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-02148b2e-5918-493b-8087-89df8a5f3d28 tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:17.222750 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-02148b2e-5918-493b-8087-89df8a5f3d28 tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:17.222750 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-02148b2e-5918-493b-8087-89df8a5f3d28 tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:17.223270 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-02148b2e-5918-493b-8087-89df8a5f3d28 tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:17.333502 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-547c2036-273a-479b-b2b5-e5cc3d5738d7 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:23:17.334320 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-547c2036-273a-479b-b2b5-e5cc3d5738d7 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] [instance: f114e124-2903-4f30-b055-9f493e3e9e6c] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:23:17.340835 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-547c2036-273a-479b-b2b5-e5cc3d5738d7 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:17.341374 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-547c2036-273a-479b-b2b5-e5cc3d5738d7 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:17.343080 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-547c2036-273a-479b-b2b5-e5cc3d5738d7 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:17.343777 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-547c2036-273a-479b-b2b5-e5cc3d5738d7 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:17.808273 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-02148b2e-5918-493b-8087-89df8a5f3d28 tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aa99be675e946fd995d71465b4cbec6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd82c2d92c67f469e863eb70bb7f9af84', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:23:17.818156 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-02148b2e-5918-493b-8087-89df8a5f3d28 tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] 10.222.0.16 "GET /compute/v2.1/servers/b9f2c527-6346-4403-841e-b7bd876a5fb7" status: 200 len: 1609 microversion: 2.1 time: 0.612614 Jul 16 21:23:17.828825 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1605/3242] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:23:17 2022] GET /compute/v2.1/servers/b9f2c527-6346-4403-841e-b7bd876a5fb7 => generated 1609 bytes in 624 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:23:17.833755 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-abf652e5-80a9-43d4-942a-1b5d2cb80257 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:23:17.834376 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-abf652e5-80a9-43d4-942a-1b5d2cb80257 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 1cb418f7-6aea-42e6-88a2-007564d925c4] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:23:17.840817 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-abf652e5-80a9-43d4-942a-1b5d2cb80257 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:17.842085 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-abf652e5-80a9-43d4-942a-1b5d2cb80257 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:17.843188 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-abf652e5-80a9-43d4-942a-1b5d2cb80257 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:17.843998 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-abf652e5-80a9-43d4-942a-1b5d2cb80257 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:17.863353 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-547c2036-273a-479b-b2b5-e5cc3d5738d7 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bba77f6ef42547f1aa8480e27868bd4a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '22ca92a86e6c40d59071e58ce4085568', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:23:17.876551 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-547c2036-273a-479b-b2b5-e5cc3d5738d7 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] 10.222.0.16 "GET /compute/v2.1/servers/f114e124-2903-4f30-b055-9f493e3e9e6c" status: 200 len: 1679 microversion: 2.1 time: 0.548509 Jul 16 21:23:17.886168 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1638/3243] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:23:17 2022] GET /compute/v2.1/servers/f114e124-2903-4f30-b055-9f493e3e9e6c => generated 1679 bytes in 550 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:23:18.348430 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-abf652e5-80a9-43d4-942a-1b5d2cb80257 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f2e974ab5aa48dfb4b00f810dcedb63', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd31be8475a6a49d6bfe8132dd050c6c0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:23:18.359279 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-abf652e5-80a9-43d4-942a-1b5d2cb80257 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "GET /compute/v2.1/servers/1cb418f7-6aea-42e6-88a2-007564d925c4" status: 200 len: 1315 microversion: 2.1 time: 0.529182 Jul 16 21:23:18.360068 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1606/3244] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:23:17 2022] GET /compute/v2.1/servers/1cb418f7-6aea-42e6-88a2-007564d925c4 => generated 1315 bytes in 531 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:23:18.491809 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-54436c5e-bc35-4ee8-aee1-00b160119fd5 req-69dae6b5-affa-4429-ae8d-7fae54239c5c service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "1cb418f7-6aea-42e6-88a2-007564d925c4", "name": "network-vif-plugged", "status": "completed", "tag": "ac40e60b-9f21-4db1-9b3a-182c4a3e4316"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:23:18.502302 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-54436c5e-bc35-4ee8-aee1-00b160119fd5 req-69dae6b5-affa-4429-ae8d-7fae54239c5c service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:18.502900 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-54436c5e-bc35-4ee8-aee1-00b160119fd5 req-69dae6b5-affa-4429-ae8d-7fae54239c5c service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:18.516763 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-54436c5e-bc35-4ee8-aee1-00b160119fd5 req-69dae6b5-affa-4429-ae8d-7fae54239c5c service nova] Creating event network-vif-plugged:ac40e60b-9f21-4db1-9b3a-182c4a3e4316 for instance 1cb418f7-6aea-42e6-88a2-007564d925c4 on nch1-master-1 Jul 16 21:23:18.535873 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-54436c5e-bc35-4ee8-aee1-00b160119fd5 req-69dae6b5-affa-4429-ae8d-7fae54239c5c service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.035749 Jul 16 21:23:18.535873 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1639/3245] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:23:18 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:23:18.835934 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-7a2edfb2-13ce-4e30-bf58-1ef12b7d02e7 tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:23:18.836822 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-7a2edfb2-13ce-4e30-bf58-1ef12b7d02e7 tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] [instance: b9f2c527-6346-4403-841e-b7bd876a5fb7] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:23:18.845066 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7a2edfb2-13ce-4e30-bf58-1ef12b7d02e7 tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:18.845768 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7a2edfb2-13ce-4e30-bf58-1ef12b7d02e7 tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:18.846562 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7a2edfb2-13ce-4e30-bf58-1ef12b7d02e7 tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:18.847025 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7a2edfb2-13ce-4e30-bf58-1ef12b7d02e7 tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:18.895049 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-311df3c8-b9dc-4dbc-bbde-ac9bb7c1a6b3 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:23:18.895809 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-311df3c8-b9dc-4dbc-bbde-ac9bb7c1a6b3 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] [instance: f114e124-2903-4f30-b055-9f493e3e9e6c] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:23:18.903948 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-311df3c8-b9dc-4dbc-bbde-ac9bb7c1a6b3 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:18.904746 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-311df3c8-b9dc-4dbc-bbde-ac9bb7c1a6b3 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:18.905493 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-311df3c8-b9dc-4dbc-bbde-ac9bb7c1a6b3 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:18.906031 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-311df3c8-b9dc-4dbc-bbde-ac9bb7c1a6b3 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:19.313115 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-7a2edfb2-13ce-4e30-bf58-1ef12b7d02e7 tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aa99be675e946fd995d71465b4cbec6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd82c2d92c67f469e863eb70bb7f9af84', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:23:19.323951 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-7a2edfb2-13ce-4e30-bf58-1ef12b7d02e7 tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] 10.222.0.16 "GET /compute/v2.1/servers/b9f2c527-6346-4403-841e-b7bd876a5fb7" status: 200 len: 1609 microversion: 2.1 time: 0.491986 Jul 16 21:23:19.324565 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1607/3246] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:23:18 2022] GET /compute/v2.1/servers/b9f2c527-6346-4403-841e-b7bd876a5fb7 => generated 1609 bytes in 493 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:23:19.378426 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-33327c4d-a999-402c-a97a-116e5d569f85 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:23:19.379778 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-33327c4d-a999-402c-a97a-116e5d569f85 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 1cb418f7-6aea-42e6-88a2-007564d925c4] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:23:19.387422 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-33327c4d-a999-402c-a97a-116e5d569f85 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:19.388155 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-33327c4d-a999-402c-a97a-116e5d569f85 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:19.389708 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-33327c4d-a999-402c-a97a-116e5d569f85 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:19.390315 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-33327c4d-a999-402c-a97a-116e5d569f85 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:19.423506 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-311df3c8-b9dc-4dbc-bbde-ac9bb7c1a6b3 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bba77f6ef42547f1aa8480e27868bd4a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '22ca92a86e6c40d59071e58ce4085568', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:23:19.440265 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-311df3c8-b9dc-4dbc-bbde-ac9bb7c1a6b3 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] 10.222.0.16 "GET /compute/v2.1/servers/f114e124-2903-4f30-b055-9f493e3e9e6c" status: 200 len: 1679 microversion: 2.1 time: 0.549598 Jul 16 21:23:19.440868 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1640/3247] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:23:18 2022] GET /compute/v2.1/servers/f114e124-2903-4f30-b055-9f493e3e9e6c => generated 1679 bytes in 552 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:23:19.738978 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-33327c4d-a999-402c-a97a-116e5d569f85 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f2e974ab5aa48dfb4b00f810dcedb63', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd31be8475a6a49d6bfe8132dd050c6c0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:23:19.750671 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-33327c4d-a999-402c-a97a-116e5d569f85 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "GET /compute/v2.1/servers/1cb418f7-6aea-42e6-88a2-007564d925c4" status: 200 len: 1315 microversion: 2.1 time: 0.377615 Jul 16 21:23:19.760767 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1608/3248] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:23:19 2022] GET /compute/v2.1/servers/1cb418f7-6aea-42e6-88a2-007564d925c4 => generated 1315 bytes in 379 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:23:20.343328 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-bf533095-b94e-4b8f-8add-e7a52abeeb20 tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:23:20.343948 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-bf533095-b94e-4b8f-8add-e7a52abeeb20 tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] [instance: b9f2c527-6346-4403-841e-b7bd876a5fb7] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:23:20.351699 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-bf533095-b94e-4b8f-8add-e7a52abeeb20 tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:20.352418 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-bf533095-b94e-4b8f-8add-e7a52abeeb20 tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:20.353362 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-bf533095-b94e-4b8f-8add-e7a52abeeb20 tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:20.353961 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-bf533095-b94e-4b8f-8add-e7a52abeeb20 tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:20.463720 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-0ba943b6-43b1-469b-887f-426ee075b1a8 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:23:20.464689 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-0ba943b6-43b1-469b-887f-426ee075b1a8 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] [instance: f114e124-2903-4f30-b055-9f493e3e9e6c] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:23:20.481161 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0ba943b6-43b1-469b-887f-426ee075b1a8 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:20.481645 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0ba943b6-43b1-469b-887f-426ee075b1a8 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:20.486367 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0ba943b6-43b1-469b-887f-426ee075b1a8 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:20.486928 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0ba943b6-43b1-469b-887f-426ee075b1a8 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:20.593683 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-bf533095-b94e-4b8f-8add-e7a52abeeb20 tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aa99be675e946fd995d71465b4cbec6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd82c2d92c67f469e863eb70bb7f9af84', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:23:20.619371 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-bf533095-b94e-4b8f-8add-e7a52abeeb20 tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] 10.222.0.16 "GET /compute/v2.1/servers/b9f2c527-6346-4403-841e-b7bd876a5fb7" status: 200 len: 1541 microversion: 2.1 time: 0.266305 Jul 16 21:23:20.619371 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1641/3249] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:23:20 2022] GET /compute/v2.1/servers/b9f2c527-6346-4403-841e-b7bd876a5fb7 => generated 1541 bytes in 272 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:23:20.781279 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-ea8987ae-884c-4390-b473-e9b7c89d75ea tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:23:20.782273 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-ea8987ae-884c-4390-b473-e9b7c89d75ea tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 1cb418f7-6aea-42e6-88a2-007564d925c4] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:23:20.791551 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ea8987ae-884c-4390-b473-e9b7c89d75ea tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:20.792341 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ea8987ae-884c-4390-b473-e9b7c89d75ea tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:20.793277 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ea8987ae-884c-4390-b473-e9b7c89d75ea tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:20.793799 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ea8987ae-884c-4390-b473-e9b7c89d75ea tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:20.989798 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-0ba943b6-43b1-469b-887f-426ee075b1a8 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bba77f6ef42547f1aa8480e27868bd4a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '22ca92a86e6c40d59071e58ce4085568', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:23:20.999687 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-0ba943b6-43b1-469b-887f-426ee075b1a8 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] 10.222.0.16 "GET /compute/v2.1/servers/f114e124-2903-4f30-b055-9f493e3e9e6c" status: 200 len: 1679 microversion: 2.1 time: 0.540010 Jul 16 21:23:21.000919 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1609/3250] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:23:20 2022] GET /compute/v2.1/servers/f114e124-2903-4f30-b055-9f493e3e9e6c => generated 1679 bytes in 542 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:23:21.223468 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-89018ad1-4875-4465-87ed-1d197535c07b req-635e6bf9-17f3-4534-860f-5fef954b3b6b service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "b9f2c527-6346-4403-841e-b7bd876a5fb7", "name": "network-vif-deleted", "tag": "edbdb017-2f7f-440d-9d46-aacbd858ccb6"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:23:21.239023 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-ea8987ae-884c-4390-b473-e9b7c89d75ea tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f2e974ab5aa48dfb4b00f810dcedb63', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd31be8475a6a49d6bfe8132dd050c6c0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:23:21.255271 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-ea8987ae-884c-4390-b473-e9b7c89d75ea tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "GET /compute/v2.1/servers/1cb418f7-6aea-42e6-88a2-007564d925c4" status: 200 len: 1502 microversion: 2.1 time: 0.489601 Jul 16 21:23:21.260903 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1642/3251] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:23:20 2022] GET /compute/v2.1/servers/1cb418f7-6aea-42e6-88a2-007564d925c4 => generated 1502 bytes in 498 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:23:21.266862 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-89018ad1-4875-4465-87ed-1d197535c07b req-635e6bf9-17f3-4534-860f-5fef954b3b6b service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:21.267879 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-89018ad1-4875-4465-87ed-1d197535c07b req-635e6bf9-17f3-4534-860f-5fef954b3b6b service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:21.284011 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-e4af0735-b9cb-452e-aff9-fe785da73194 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Action: 'action', calling method: >, body: {"pause": {}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:23:21.298212 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-e4af0735-b9cb-452e-aff9-fe785da73194 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 1cb418f7-6aea-42e6-88a2-007564d925c4] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:23:21.298353 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-89018ad1-4875-4465-87ed-1d197535c07b req-635e6bf9-17f3-4534-860f-5fef954b3b6b service nova] Creating event network-vif-deleted:edbdb017-2f7f-440d-9d46-aacbd858ccb6 for instance b9f2c527-6346-4403-841e-b7bd876a5fb7 on nch2-master-1 Jul 16 21:23:21.298353 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-89018ad1-4875-4465-87ed-1d197535c07b req-635e6bf9-17f3-4534-860f-5fef954b3b6b service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.083780 Jul 16 21:23:21.298353 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1610/3252] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:23:21 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 16 21:23:21.303067 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e4af0735-b9cb-452e-aff9-fe785da73194 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:21.304037 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e4af0735-b9cb-452e-aff9-fe785da73194 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:21.307506 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e4af0735-b9cb-452e-aff9-fe785da73194 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:21.308611 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e4af0735-b9cb-452e-aff9-fe785da73194 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:21.348854 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=96,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:23:21.636189 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-cb4618fa-8711-4168-a1d0-f35c462b1233 tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:23:21.637095 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-cb4618fa-8711-4168-a1d0-f35c462b1233 tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] [instance: b9f2c527-6346-4403-841e-b7bd876a5fb7] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:23:21.654418 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-e4af0735-b9cb-452e-aff9-fe785da73194 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "POST /compute/v2.1/servers/1cb418f7-6aea-42e6-88a2-007564d925c4/action" status: 202 len: 0 microversion: 2.1 time: 0.379280 Jul 16 21:23:21.659404 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1643/3253] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 21:23:21 2022] POST /compute/v2.1/servers/1cb418f7-6aea-42e6-88a2-007564d925c4/action => generated 0 bytes in 386 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 21:23:21.690312 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-32137e79-aa84-4a6a-8790-1a3638db5aed tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:23:21.697125 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cb4618fa-8711-4168-a1d0-f35c462b1233 tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:21.697591 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cb4618fa-8711-4168-a1d0-f35c462b1233 tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:21.698414 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cb4618fa-8711-4168-a1d0-f35c462b1233 tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:21.698858 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cb4618fa-8711-4168-a1d0-f35c462b1233 tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:21.701997 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-32137e79-aa84-4a6a-8790-1a3638db5aed tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 1cb418f7-6aea-42e6-88a2-007564d925c4] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:23:21.721955 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-32137e79-aa84-4a6a-8790-1a3638db5aed tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:21.721955 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-32137e79-aa84-4a6a-8790-1a3638db5aed tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:21.721955 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-32137e79-aa84-4a6a-8790-1a3638db5aed tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:21.721955 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-32137e79-aa84-4a6a-8790-1a3638db5aed tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:21.988568 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-cb4618fa-8711-4168-a1d0-f35c462b1233 tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aa99be675e946fd995d71465b4cbec6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd82c2d92c67f469e863eb70bb7f9af84', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:23:21.997978 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-cb4618fa-8711-4168-a1d0-f35c462b1233 tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] 10.222.0.16 "GET /compute/v2.1/servers/b9f2c527-6346-4403-841e-b7bd876a5fb7" status: 200 len: 1252 microversion: 2.1 time: 0.372496 Jul 16 21:23:22.004839 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1611/3254] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:23:21 2022] GET /compute/v2.1/servers/b9f2c527-6346-4403-841e-b7bd876a5fb7 => generated 1252 bytes in 377 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:23:22.018522 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-5837525e-740d-43f4-b741-60e335a87a9b tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:23:22.019179 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-5837525e-740d-43f4-b741-60e335a87a9b tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] [instance: f114e124-2903-4f30-b055-9f493e3e9e6c] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:23:22.025531 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5837525e-740d-43f4-b741-60e335a87a9b tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:22.025990 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5837525e-740d-43f4-b741-60e335a87a9b tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:22.027656 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5837525e-740d-43f4-b741-60e335a87a9b tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:22.027656 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5837525e-740d-43f4-b741-60e335a87a9b tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:22.432236 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-32137e79-aa84-4a6a-8790-1a3638db5aed tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f2e974ab5aa48dfb4b00f810dcedb63', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd31be8475a6a49d6bfe8132dd050c6c0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:23:22.452231 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-32137e79-aa84-4a6a-8790-1a3638db5aed tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "GET /compute/v2.1/servers/1cb418f7-6aea-42e6-88a2-007564d925c4" status: 200 len: 1507 microversion: 2.1 time: 0.789894 Jul 16 21:23:22.453129 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1644/3255] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:23:21 2022] GET /compute/v2.1/servers/1cb418f7-6aea-42e6-88a2-007564d925c4 => generated 1507 bytes in 793 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:23:22.550337 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-5837525e-740d-43f4-b741-60e335a87a9b tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bba77f6ef42547f1aa8480e27868bd4a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '22ca92a86e6c40d59071e58ce4085568', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:23:22.558342 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-5837525e-740d-43f4-b741-60e335a87a9b tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] 10.222.0.16 "GET /compute/v2.1/servers/f114e124-2903-4f30-b055-9f493e3e9e6c" status: 200 len: 1656 microversion: 2.1 time: 0.544995 Jul 16 21:23:22.558948 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1612/3256] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:23:22 2022] GET /compute/v2.1/servers/f114e124-2903-4f30-b055-9f493e3e9e6c => generated 1656 bytes in 546 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:23:22.584887 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-3e007da1-d491-4c84-a4b8-9a0e4a449fa8 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Action: 'action', calling method: >, body: {"os-start": {}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:23:22.608162 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3e007da1-d491-4c84-a4b8-9a0e4a449fa8 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:22.610732 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3e007da1-d491-4c84-a4b8-9a0e4a449fa8 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:22.682632 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-3e007da1-d491-4c84-a4b8-9a0e4a449fa8 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] [instance: f114e124-2903-4f30-b055-9f493e3e9e6c] Going to try to start instance {{(pid=138680) start /opt/stack/nova/nova/compute/api.py:2730}} Jul 16 21:23:22.894201 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-3e007da1-d491-4c84-a4b8-9a0e4a449fa8 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] 10.222.0.16 "POST /compute/v2.1/servers/f114e124-2903-4f30-b055-9f493e3e9e6c/action" status: 202 len: 0 microversion: 2.1 time: 0.314738 Jul 16 21:23:22.895770 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1645/3257] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 21:23:22 2022] POST /compute/v2.1/servers/f114e124-2903-4f30-b055-9f493e3e9e6c/action => generated 0 bytes in 317 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 21:23:22.907060 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-71998202-4d16-468e-8616-061a4bd5b7bd tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:23:22.907742 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-71998202-4d16-468e-8616-061a4bd5b7bd tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] [instance: f114e124-2903-4f30-b055-9f493e3e9e6c] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:23:22.921756 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-71998202-4d16-468e-8616-061a4bd5b7bd tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:22.923348 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-71998202-4d16-468e-8616-061a4bd5b7bd tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:22.923348 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-71998202-4d16-468e-8616-061a4bd5b7bd tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:22.923527 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-71998202-4d16-468e-8616-061a4bd5b7bd tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:23.032903 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-2beaf60e-463a-4b0d-b0ae-a37f1dd250bd tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:23:23.035195 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-2beaf60e-463a-4b0d-b0ae-a37f1dd250bd tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] [instance: b9f2c527-6346-4403-841e-b7bd876a5fb7] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:23:23.050241 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2beaf60e-463a-4b0d-b0ae-a37f1dd250bd tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:23.051278 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2beaf60e-463a-4b0d-b0ae-a37f1dd250bd tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:23.052563 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2beaf60e-463a-4b0d-b0ae-a37f1dd250bd tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:23.053697 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2beaf60e-463a-4b0d-b0ae-a37f1dd250bd tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:23.129698 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2beaf60e-463a-4b0d-b0ae-a37f1dd250bd tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:23.130217 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2beaf60e-463a-4b0d-b0ae-a37f1dd250bd tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:23.130986 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.objects.instance [None req-2beaf60e-463a-4b0d-b0ae-a37f1dd250bd tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] Lazy-loading 'fault' on Instance uuid b9f2c527-6346-4403-841e-b7bd876a5fb7 {{(pid=138680) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:23:23.254128 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-71998202-4d16-468e-8616-061a4bd5b7bd tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bba77f6ef42547f1aa8480e27868bd4a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '22ca92a86e6c40d59071e58ce4085568', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:23:23.273939 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-71998202-4d16-468e-8616-061a4bd5b7bd tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] 10.222.0.16 "GET /compute/v2.1/servers/f114e124-2903-4f30-b055-9f493e3e9e6c" status: 200 len: 1665 microversion: 2.1 time: 0.372959 Jul 16 21:23:23.275180 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1613/3258] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:23:22 2022] GET /compute/v2.1/servers/f114e124-2903-4f30-b055-9f493e3e9e6c => generated 1665 bytes in 375 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:23:23.364215 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-2beaf60e-463a-4b0d-b0ae-a37f1dd250bd tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1aa99be675e946fd995d71465b4cbec6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd82c2d92c67f469e863eb70bb7f9af84', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:23:23.373426 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-2beaf60e-463a-4b0d-b0ae-a37f1dd250bd tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] 10.222.0.16 "GET /compute/v2.1/servers/b9f2c527-6346-4403-841e-b7bd876a5fb7" status: 200 len: 1211 microversion: 2.1 time: 0.359677 Jul 16 21:23:23.374684 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1646/3259] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:23:23 2022] GET /compute/v2.1/servers/b9f2c527-6346-4403-841e-b7bd876a5fb7 => generated 1211 bytes in 363 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:23:23.470646 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-7e19b728-3f07-4a64-8113-bb23464e27e0 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:23:23.471574 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-7e19b728-3f07-4a64-8113-bb23464e27e0 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 1cb418f7-6aea-42e6-88a2-007564d925c4] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:23:23.478039 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7e19b728-3f07-4a64-8113-bb23464e27e0 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:23.478766 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7e19b728-3f07-4a64-8113-bb23464e27e0 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:23.479744 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7e19b728-3f07-4a64-8113-bb23464e27e0 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:23.480528 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7e19b728-3f07-4a64-8113-bb23464e27e0 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:23.882374 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-7e19b728-3f07-4a64-8113-bb23464e27e0 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f2e974ab5aa48dfb4b00f810dcedb63', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd31be8475a6a49d6bfe8132dd050c6c0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:23:23.899327 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-7e19b728-3f07-4a64-8113-bb23464e27e0 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "GET /compute/v2.1/servers/1cb418f7-6aea-42e6-88a2-007564d925c4" status: 200 len: 1487 microversion: 2.1 time: 0.432200 Jul 16 21:23:23.916891 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1614/3260] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:23:23 2022] GET /compute/v2.1/servers/1cb418f7-6aea-42e6-88a2-007564d925c4 => generated 1487 bytes in 451 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:23:23.920628 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-dca83f4f-1d53-412d-9967-62d740cd7f9f tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:23:23.921143 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-dca83f4f-1d53-412d-9967-62d740cd7f9f tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 1cb418f7-6aea-42e6-88a2-007564d925c4] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:23:23.927758 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-dca83f4f-1d53-412d-9967-62d740cd7f9f tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:23.928488 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-dca83f4f-1d53-412d-9967-62d740cd7f9f tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:23.929982 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-dca83f4f-1d53-412d-9967-62d740cd7f9f tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:23.930614 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-dca83f4f-1d53-412d-9967-62d740cd7f9f tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:23.974346 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-dca83f4f-1d53-412d-9967-62d740cd7f9f tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 1cb418f7-6aea-42e6-88a2-007564d925c4] Going to try to terminate instance {{(pid=138680) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 16 21:23:24.300728 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-4470af9f-8aea-497c-ab85-75bfde82e147 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:23:24.301824 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-4470af9f-8aea-497c-ab85-75bfde82e147 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] [instance: f114e124-2903-4f30-b055-9f493e3e9e6c] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:23:24.319164 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4470af9f-8aea-497c-ab85-75bfde82e147 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:24.321788 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4470af9f-8aea-497c-ab85-75bfde82e147 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:24.325853 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4470af9f-8aea-497c-ab85-75bfde82e147 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:24.326614 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4470af9f-8aea-497c-ab85-75bfde82e147 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:24.413098 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-dca83f4f-1d53-412d-9967-62d740cd7f9f tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "DELETE /compute/v2.1/servers/1cb418f7-6aea-42e6-88a2-007564d925c4" status: 204 len: 0 microversion: 2.1 time: 0.496141 Jul 16 21:23:24.413849 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1647/3261] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:23:23 2022] DELETE /compute/v2.1/servers/1cb418f7-6aea-42e6-88a2-007564d925c4 => generated 0 bytes in 498 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Jul 16 21:23:24.420440 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-24519c68-2c90-455f-89ad-fd751d1fbefc tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:23:24.422612 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-24519c68-2c90-455f-89ad-fd751d1fbefc tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] [instance: b9f2c527-6346-4403-841e-b7bd876a5fb7] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:23:24.436088 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-24519c68-2c90-455f-89ad-fd751d1fbefc tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:24.440919 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-24519c68-2c90-455f-89ad-fd751d1fbefc tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:24.441995 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-24519c68-2c90-455f-89ad-fd751d1fbefc tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:24.445129 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-24519c68-2c90-455f-89ad-fd751d1fbefc tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:24.484131 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-24519c68-2c90-455f-89ad-fd751d1fbefc tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] HTTP exception thrown: Instance b9f2c527-6346-4403-841e-b7bd876a5fb7 could not be found. Jul 16 21:23:24.489815 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-24519c68-2c90-455f-89ad-fd751d1fbefc tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] Request method failure captured: Jul 16 21:23:24.489815 ncdv-master-1 devstack@n-api.service[138680]: request: GET /compute/v2.1/servers/b9f2c527-6346-4403-841e-b7bd876a5fb7 HTTP/1.1 Jul 16 21:23:24.489815 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:23:24.489815 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:23:24.489815 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:23:24.489815 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 0 Jul 16 21:23:24.489815 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:23:24.489815 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:23:24.489815 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:23:24.489815 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:23:24.489815 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0yp2cWKLxMuCpoLH_T6kdTTUYLwVc98ACHSJj5-oUdSIWnt1kX-B6eYNkENsXqiKDYaHZ1ePkgn3QDlaKRsSkQVt7mH_4PvGvFsMsLG3HXdqUl_6oz-5-dXylT_EdegwJUy3VLWc6zHMLwxBng0c7BuenSPlL_PulBG4XYiyEx30zzs Jul 16 21:23:24.489815 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:23:24.489815 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:23:24.489815 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:23:24.489815 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:23:24.489815 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:23:24.489815 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:23:24.489815 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: d82c2d92c67f469e863eb70bb7f9af84 Jul 16 21:23:24.489815 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-ServersTestBootFromVolume-1080567873 Jul 16 21:23:24.489815 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:23:24.489815 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:23:24.489815 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/d82c2d92c67f469e863eb70bb7f9af84"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/d82c2d92c67f469e863eb70bb7f9af84"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_d82c2d92c67f469e863eb70bb7f9af84"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/d82c2d92c67f469e863eb70bb7f9af84"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:23:24.489815 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-ServersTestBootFromVolume-1080567873 Jul 16 21:23:24.489815 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: d82c2d92c67f469e863eb70bb7f9af84 Jul 16 21:23:24.489815 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-ServersTestBootFromVolume-1080567873 Jul 16 21:23:24.489815 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-ServersTestBootFromVolume-1080567873-project Jul 16 21:23:24.489815 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:23:24.489815 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:23:24.489815 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: 1aa99be675e946fd995d71465b4cbec6 Jul 16 21:23:24.489815 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-ServersTestBootFromVolume-1080567873-project Jul 16 21:23:24.489815 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 21:23:24.489815 ncdv-master-1 devstack@n-api.service[138680]: exception: Instance b9f2c527-6346-4403-841e-b7bd876a5fb7 could not be found. Jul 16 21:23:24.489815 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': 'b9f2c527-6346-4403-841e-b7bd876a5fb7'} Jul 16 21:23:24.489815 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:23:24.489815 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:23:24.489815 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:23:24.489815 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:23:24.489815 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:23:24.489815 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:23:24.489815 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:23:24.489815 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:23:24.489815 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:23:24.489815 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:23:24.489815 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance b9f2c527-6346-4403-841e-b7bd876a5fb7 could not be found. Jul 16 21:23:24.489815 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:23:24.489815 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:23:24.489815 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:23:24.489815 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:23:24.489815 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:23:24.489815 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:23:24.489815 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:23:24.489815 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:23:24.489815 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:23:24.489815 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:23:24.489815 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:23:24.489815 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:23:24.489815 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:23:24.489815 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:23:24.489815 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:23:24.489815 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:23:24.489815 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance b9f2c527-6346-4403-841e-b7bd876a5fb7 could not be found. Jul 16 21:23:24.489815 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:23:24.489815 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:23:24.489815 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:23:24.494987 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:23:24.494987 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:23:24.494987 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:23:24.494987 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:23:24.494987 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:23:24.494987 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance b9f2c527-6346-4403-841e-b7bd876a5fb7 could not be found. Jul 16 21:23:24.494987 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:23:24.497240 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-24519c68-2c90-455f-89ad-fd751d1fbefc tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] Returning 404 to user: Instance b9f2c527-6346-4403-841e-b7bd876a5fb7 could not be found. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:23:24.499862 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-24519c68-2c90-455f-89ad-fd751d1fbefc tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] 10.222.0.16 "GET /compute/v2.1/servers/b9f2c527-6346-4403-841e-b7bd876a5fb7" status: 404 len: 111 microversion: 2.1 time: 0.083848 Jul 16 21:23:24.501085 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1648/3262] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:23:24 2022] GET /compute/v2.1/servers/b9f2c527-6346-4403-841e-b7bd876a5fb7 => generated 111 bytes in 86 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:23:24.508748 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-6cf062ae-43b0-4025-9f08-2e2a1d639f28 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:23:24.509754 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-6cf062ae-43b0-4025-9f08-2e2a1d639f28 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 1cb418f7-6aea-42e6-88a2-007564d925c4] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:23:24.519930 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6cf062ae-43b0-4025-9f08-2e2a1d639f28 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:24.521619 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6cf062ae-43b0-4025-9f08-2e2a1d639f28 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:24.522697 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6cf062ae-43b0-4025-9f08-2e2a1d639f28 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:24.523349 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6cf062ae-43b0-4025-9f08-2e2a1d639f28 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:24.889533 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-4470af9f-8aea-497c-ab85-75bfde82e147 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bba77f6ef42547f1aa8480e27868bd4a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '22ca92a86e6c40d59071e58ce4085568', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:23:24.910777 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-4470af9f-8aea-497c-ab85-75bfde82e147 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] 10.222.0.16 "GET /compute/v2.1/servers/f114e124-2903-4f30-b055-9f493e3e9e6c" status: 200 len: 1665 microversion: 2.1 time: 0.620389 Jul 16 21:23:24.912000 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-6cf062ae-43b0-4025-9f08-2e2a1d639f28 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f2e974ab5aa48dfb4b00f810dcedb63', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd31be8475a6a49d6bfe8132dd050c6c0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:23:24.918981 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1615/3263] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:23:24 2022] GET /compute/v2.1/servers/f114e124-2903-4f30-b055-9f493e3e9e6c => generated 1665 bytes in 632 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:23:24.929683 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-64b7d964-675c-4e47-8b7b-a6852dc1a335 tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] Calling method '.version_select at 0x7fdd9cfa6c10>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:23:24.929931 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-6cf062ae-43b0-4025-9f08-2e2a1d639f28 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "GET /compute/v2.1/servers/1cb418f7-6aea-42e6-88a2-007564d925c4" status: 200 len: 1493 microversion: 2.1 time: 0.427160 Jul 16 21:23:24.931443 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1649/3264] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:23:24 2022] GET /compute/v2.1/servers/1cb418f7-6aea-42e6-88a2-007564d925c4 => generated 1493 bytes in 430 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:23:25.072138 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-64b7d964-675c-4e47-8b7b-a6852dc1a335 tempest-ServersTestBootFromVolume-1080567873 tempest-ServersTestBootFromVolume-1080567873-project] 10.222.0.16 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1395237950" status: 202 len: 0 microversion: 2.1 time: 0.151316 Jul 16 21:23:25.072736 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1616/3265] 10.222.0.16 () {58 vars in 1220 bytes} [Sat Jul 16 21:23:24 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1395237950 => generated 0 bytes in 153 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 21:23:25.925247 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-dddf2ee2-b534-4dab-95d8-10c1ec303d32 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:23:25.925872 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-dddf2ee2-b534-4dab-95d8-10c1ec303d32 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] [instance: f114e124-2903-4f30-b055-9f493e3e9e6c] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:23:25.933116 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-dddf2ee2-b534-4dab-95d8-10c1ec303d32 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:25.934290 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-dddf2ee2-b534-4dab-95d8-10c1ec303d32 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:25.935540 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-dddf2ee2-b534-4dab-95d8-10c1ec303d32 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:25.936276 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-dddf2ee2-b534-4dab-95d8-10c1ec303d32 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:25.953896 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-7bc265dd-c6ff-4643-b964-50ff8e9f8a54 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:23:25.957126 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-7bc265dd-c6ff-4643-b964-50ff8e9f8a54 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 1cb418f7-6aea-42e6-88a2-007564d925c4] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:23:25.969421 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7bc265dd-c6ff-4643-b964-50ff8e9f8a54 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:25.970252 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7bc265dd-c6ff-4643-b964-50ff8e9f8a54 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:25.971651 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7bc265dd-c6ff-4643-b964-50ff8e9f8a54 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:25.972298 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7bc265dd-c6ff-4643-b964-50ff8e9f8a54 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:26.435567 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-7bc265dd-c6ff-4643-b964-50ff8e9f8a54 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f2e974ab5aa48dfb4b00f810dcedb63', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd31be8475a6a49d6bfe8132dd050c6c0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:23:26.446738 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-7bc265dd-c6ff-4643-b964-50ff8e9f8a54 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "GET /compute/v2.1/servers/1cb418f7-6aea-42e6-88a2-007564d925c4" status: 200 len: 1493 microversion: 2.1 time: 0.497641 Jul 16 21:23:26.447544 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1617/3266] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:23:25 2022] GET /compute/v2.1/servers/1cb418f7-6aea-42e6-88a2-007564d925c4 => generated 1493 bytes in 499 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:23:26.456259 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-8ee0e28e-0073-4083-9add-bd4768644fa0 tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:23:26.467306 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-8ee0e28e-0073-4083-9add-bd4768644fa0 tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] [instance: fa075dcc-e175-449a-9914-d1ee19131f33] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:23:26.482514 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8ee0e28e-0073-4083-9add-bd4768644fa0 tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:26.482514 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8ee0e28e-0073-4083-9add-bd4768644fa0 tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:26.482514 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8ee0e28e-0073-4083-9add-bd4768644fa0 tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:26.483380 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8ee0e28e-0073-4083-9add-bd4768644fa0 tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:26.499665 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-8ee0e28e-0073-4083-9add-bd4768644fa0 tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] HTTP exception thrown: Instance fa075dcc-e175-449a-9914-d1ee19131f33 could not be found. Jul 16 21:23:26.502790 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-8ee0e28e-0073-4083-9add-bd4768644fa0 tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Request method failure captured: Jul 16 21:23:26.502790 ncdv-master-1 devstack@n-api.service[138681]: request: DELETE /compute/v2.1/servers/fa075dcc-e175-449a-9914-d1ee19131f33 HTTP/1.1 Jul 16 21:23:26.502790 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:23:26.502790 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:23:26.502790 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:23:26.502790 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 21:23:26.502790 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:23:26.502790 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:23:26.502790 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:23:26.502790 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:23:26.502790 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0yukIMJ48tRgCB3diR5B2yTdgfCCN7-OZTMmtI4TH-F_hINl86vSpH64fFhpc-NX4fmSyldmbblkIl17gqyuR9fatu2VYxfrnJDqp4uflU3P7tQcLJTnZudoBO8vc10rI0Ks_An4UJorcnH-Ege1HUXGbj4nJCfyRf8zOh_B8VIV5H0 Jul 16 21:23:26.502790 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:23:26.502790 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:23:26.502790 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:23:26.502790 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:23:26.502790 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:23:26.502790 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:23:26.502790 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: 3e1f6ffc3fb64dba8c4474641e796fb8 Jul 16 21:23:26.502790 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-AttachSCSIVolumeTestJSON-288658511 Jul 16 21:23:26.502790 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:23:26.502790 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:23:26.502790 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/3e1f6ffc3fb64dba8c4474641e796fb8"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/3e1f6ffc3fb64dba8c4474641e796fb8"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_3e1f6ffc3fb64dba8c4474641e796fb8"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/3e1f6ffc3fb64dba8c4474641e796fb8"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:23:26.502790 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-AttachSCSIVolumeTestJSON-288658511 Jul 16 21:23:26.502790 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: 3e1f6ffc3fb64dba8c4474641e796fb8 Jul 16 21:23:26.502790 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-AttachSCSIVolumeTestJSON-288658511 Jul 16 21:23:26.502790 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-AttachSCSIVolumeTestJSON-288658511-project Jul 16 21:23:26.502790 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:23:26.502790 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:23:26.502790 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: 029f1334779e4bcdb3f4f2aed564a075 Jul 16 21:23:26.502790 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-AttachSCSIVolumeTestJSON-288658511-project Jul 16 21:23:26.502790 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:23:26.502790 ncdv-master-1 devstack@n-api.service[138681]: exception: Instance fa075dcc-e175-449a-9914-d1ee19131f33 could not be found. Jul 16 21:23:26.502790 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': 'fa075dcc-e175-449a-9914-d1ee19131f33'} Jul 16 21:23:26.502790 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:23:26.502790 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:23:26.502790 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:23:26.502790 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:23:26.502790 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:23:26.502790 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:23:26.502790 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:23:26.502790 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:23:26.502790 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:23:26.502790 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:23:26.502790 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance fa075dcc-e175-449a-9914-d1ee19131f33 could not be found. Jul 16 21:23:26.502790 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:23:26.502790 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:23:26.502790 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:23:26.502790 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:23:26.502790 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:23:26.502790 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:23:26.502790 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:23:26.502790 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:23:26.502790 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:23:26.502790 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:23:26.502790 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Jul 16 21:23:26.502790 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Jul 16 21:23:26.502790 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Jul 16 21:23:26.502790 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Jul 16 21:23:26.502790 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:23:26.502790 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:23:26.502790 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:23:26.502790 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:23:26.514834 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance fa075dcc-e175-449a-9914-d1ee19131f33 could not be found. Jul 16 21:23:26.514834 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:23:26.514834 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:23:26.514834 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:23:26.514834 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:23:26.514834 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:23:26.514834 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:23:26.514834 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:23:26.514834 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:23:26.514834 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance fa075dcc-e175-449a-9914-d1ee19131f33 could not be found. Jul 16 21:23:26.514834 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:23:26.514834 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-8ee0e28e-0073-4083-9add-bd4768644fa0 tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Returning 404 to user: Instance fa075dcc-e175-449a-9914-d1ee19131f33 could not be found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:23:26.514834 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-8ee0e28e-0073-4083-9add-bd4768644fa0 tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] 10.222.0.16 "DELETE /compute/v2.1/servers/fa075dcc-e175-449a-9914-d1ee19131f33" status: 404 len: 111 microversion: 2.1 time: 0.055895 Jul 16 21:23:26.514834 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1618/3267] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:23:26 2022] DELETE /compute/v2.1/servers/fa075dcc-e175-449a-9914-d1ee19131f33 => generated 111 bytes in 57 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:23:26.521759 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-b0c6a823-2ed2-4474-8dcb-2aaa9983209d tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:23:26.523617 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-b0c6a823-2ed2-4474-8dcb-2aaa9983209d tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] [instance: fa075dcc-e175-449a-9914-d1ee19131f33] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:23:26.533711 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b0c6a823-2ed2-4474-8dcb-2aaa9983209d tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:26.534476 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b0c6a823-2ed2-4474-8dcb-2aaa9983209d tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:26.535576 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b0c6a823-2ed2-4474-8dcb-2aaa9983209d tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:26.536992 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b0c6a823-2ed2-4474-8dcb-2aaa9983209d tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:26.555697 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-b0c6a823-2ed2-4474-8dcb-2aaa9983209d tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] HTTP exception thrown: Instance fa075dcc-e175-449a-9914-d1ee19131f33 could not be found. Jul 16 21:23:26.561761 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-dddf2ee2-b534-4dab-95d8-10c1ec303d32 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bba77f6ef42547f1aa8480e27868bd4a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '22ca92a86e6c40d59071e58ce4085568', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:23:26.561903 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-b0c6a823-2ed2-4474-8dcb-2aaa9983209d tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Request method failure captured: Jul 16 21:23:26.561903 ncdv-master-1 devstack@n-api.service[138681]: request: GET /compute/v2.1/servers/fa075dcc-e175-449a-9914-d1ee19131f33 HTTP/1.1 Jul 16 21:23:26.561903 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:23:26.561903 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:23:26.561903 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:23:26.561903 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 21:23:26.561903 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:23:26.561903 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:23:26.561903 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:23:26.561903 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:23:26.561903 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0yukIMJ48tRgCB3diR5B2yTdgfCCN7-OZTMmtI4TH-F_hINl86vSpH64fFhpc-NX4fmSyldmbblkIl17gqyuR9fatu2VYxfrnJDqp4uflU3P7tQcLJTnZudoBO8vc10rI0Ks_An4UJorcnH-Ege1HUXGbj4nJCfyRf8zOh_B8VIV5H0 Jul 16 21:23:26.561903 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:23:26.561903 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:23:26.561903 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:23:26.561903 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:23:26.561903 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:23:26.561903 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:23:26.561903 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: 3e1f6ffc3fb64dba8c4474641e796fb8 Jul 16 21:23:26.561903 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-AttachSCSIVolumeTestJSON-288658511 Jul 16 21:23:26.561903 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:23:26.561903 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:23:26.561903 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/3e1f6ffc3fb64dba8c4474641e796fb8"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/3e1f6ffc3fb64dba8c4474641e796fb8"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_3e1f6ffc3fb64dba8c4474641e796fb8"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/3e1f6ffc3fb64dba8c4474641e796fb8"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:23:26.561903 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-AttachSCSIVolumeTestJSON-288658511 Jul 16 21:23:26.561903 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: 3e1f6ffc3fb64dba8c4474641e796fb8 Jul 16 21:23:26.561903 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-AttachSCSIVolumeTestJSON-288658511 Jul 16 21:23:26.561903 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-AttachSCSIVolumeTestJSON-288658511-project Jul 16 21:23:26.561903 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:23:26.561903 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:23:26.561903 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: 029f1334779e4bcdb3f4f2aed564a075 Jul 16 21:23:26.561903 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-AttachSCSIVolumeTestJSON-288658511-project Jul 16 21:23:26.561903 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:23:26.561903 ncdv-master-1 devstack@n-api.service[138681]: exception: Instance fa075dcc-e175-449a-9914-d1ee19131f33 could not be found. Jul 16 21:23:26.561903 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': 'fa075dcc-e175-449a-9914-d1ee19131f33'} Jul 16 21:23:26.561903 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:23:26.561903 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:23:26.561903 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:23:26.561903 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:23:26.561903 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:23:26.561903 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:23:26.561903 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:23:26.561903 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:23:26.561903 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:23:26.561903 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:23:26.561903 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance fa075dcc-e175-449a-9914-d1ee19131f33 could not be found. Jul 16 21:23:26.561903 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:23:26.561903 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:23:26.561903 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:23:26.561903 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:23:26.561903 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:23:26.561903 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:23:26.561903 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:23:26.561903 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:23:26.561903 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:23:26.561903 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:23:26.561903 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:23:26.561903 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:23:26.561903 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:23:26.561903 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:23:26.561903 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:23:26.561903 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:23:26.561903 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance fa075dcc-e175-449a-9914-d1ee19131f33 could not be found. Jul 16 21:23:26.561903 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:23:26.561903 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:23:26.561903 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:23:26.565789 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:23:26.565789 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:23:26.565789 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:23:26.565789 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:23:26.565789 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:23:26.565789 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance fa075dcc-e175-449a-9914-d1ee19131f33 could not be found. Jul 16 21:23:26.565789 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:23:26.565789 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-b0c6a823-2ed2-4474-8dcb-2aaa9983209d tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] Returning 404 to user: Instance fa075dcc-e175-449a-9914-d1ee19131f33 could not be found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:23:26.565789 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-b0c6a823-2ed2-4474-8dcb-2aaa9983209d tempest-AttachSCSIVolumeTestJSON-288658511 tempest-AttachSCSIVolumeTestJSON-288658511-project] 10.222.0.16 "GET /compute/v2.1/servers/fa075dcc-e175-449a-9914-d1ee19131f33" status: 404 len: 111 microversion: 2.1 time: 0.049678 Jul 16 21:23:26.565789 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1619/3268] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:23:26 2022] GET /compute/v2.1/servers/fa075dcc-e175-449a-9914-d1ee19131f33 => generated 111 bytes in 51 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Jul 16 21:23:26.569277 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-dddf2ee2-b534-4dab-95d8-10c1ec303d32 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] 10.222.0.16 "GET /compute/v2.1/servers/f114e124-2903-4f30-b055-9f493e3e9e6c" status: 200 len: 1665 microversion: 2.1 time: 0.647886 Jul 16 21:23:26.571058 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1650/3269] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:23:25 2022] GET /compute/v2.1/servers/f114e124-2903-4f30-b055-9f493e3e9e6c => generated 1665 bytes in 649 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:23:27.465878 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-27c22113-85fc-45fd-839c-6a33a29e072c tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:23:27.469218 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-27c22113-85fc-45fd-839c-6a33a29e072c tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 1cb418f7-6aea-42e6-88a2-007564d925c4] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:23:27.479533 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-27c22113-85fc-45fd-839c-6a33a29e072c tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:27.479998 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-27c22113-85fc-45fd-839c-6a33a29e072c tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:27.480786 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-27c22113-85fc-45fd-839c-6a33a29e072c tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:27.481598 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-27c22113-85fc-45fd-839c-6a33a29e072c tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:27.499998 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=100 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:23:27.586278 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-c9907c69-7046-4ba7-8e4b-62ec96b538ee tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:23:27.586903 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-c9907c69-7046-4ba7-8e4b-62ec96b538ee tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] [instance: f114e124-2903-4f30-b055-9f493e3e9e6c] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:23:27.599828 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c9907c69-7046-4ba7-8e4b-62ec96b538ee tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:27.600680 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c9907c69-7046-4ba7-8e4b-62ec96b538ee tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:27.602970 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c9907c69-7046-4ba7-8e4b-62ec96b538ee tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:27.603765 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c9907c69-7046-4ba7-8e4b-62ec96b538ee tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:28.678380 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-c9907c69-7046-4ba7-8e4b-62ec96b538ee tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bba77f6ef42547f1aa8480e27868bd4a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '22ca92a86e6c40d59071e58ce4085568', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:23:28.694336 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-c9907c69-7046-4ba7-8e4b-62ec96b538ee tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] 10.222.0.16 "GET /compute/v2.1/servers/f114e124-2903-4f30-b055-9f493e3e9e6c" status: 200 len: 1669 microversion: 2.1 time: 1.110349 Jul 16 21:23:28.696474 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1651/3270] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:23:27 2022] GET /compute/v2.1/servers/f114e124-2903-4f30-b055-9f493e3e9e6c => generated 1669 bytes in 1115 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:23:28.744616 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-27c22113-85fc-45fd-839c-6a33a29e072c tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f2e974ab5aa48dfb4b00f810dcedb63', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd31be8475a6a49d6bfe8132dd050c6c0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:23:28.756238 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-27c22113-85fc-45fd-839c-6a33a29e072c tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "GET /compute/v2.1/servers/1cb418f7-6aea-42e6-88a2-007564d925c4" status: 200 len: 1493 microversion: 2.1 time: 1.293927 Jul 16 21:23:28.757618 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1620/3271] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:23:27 2022] GET /compute/v2.1/servers/1cb418f7-6aea-42e6-88a2-007564d925c4 => generated 1493 bytes in 1296 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:23:28.795621 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-7b4d157d-3f1b-4264-916b-1d694dd3430e req-d9551aed-f021-482c-8c78-36cb592e3eb1 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "1cb418f7-6aea-42e6-88a2-007564d925c4", "name": "network-vif-deleted", "tag": "ac40e60b-9f21-4db1-9b3a-182c4a3e4316"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:23:28.816009 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-7b4d157d-3f1b-4264-916b-1d694dd3430e req-d9551aed-f021-482c-8c78-36cb592e3eb1 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:28.817133 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-7b4d157d-3f1b-4264-916b-1d694dd3430e req-d9551aed-f021-482c-8c78-36cb592e3eb1 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:28.845309 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-7b4d157d-3f1b-4264-916b-1d694dd3430e req-d9551aed-f021-482c-8c78-36cb592e3eb1 service nova] Creating event network-vif-deleted:ac40e60b-9f21-4db1-9b3a-182c4a3e4316 for instance 1cb418f7-6aea-42e6-88a2-007564d925c4 on nch1-master-1 Jul 16 21:23:28.851387 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-7b4d157d-3f1b-4264-916b-1d694dd3430e req-d9551aed-f021-482c-8c78-36cb592e3eb1 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.059896 Jul 16 21:23:28.852512 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1652/3272] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:23:28 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 62 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:23:29.774481 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-2cf868fa-b1ec-43ec-9202-dbdb73315e56 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:23:29.775686 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-2cf868fa-b1ec-43ec-9202-dbdb73315e56 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 1cb418f7-6aea-42e6-88a2-007564d925c4] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:23:29.794786 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2cf868fa-b1ec-43ec-9202-dbdb73315e56 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:29.795287 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2cf868fa-b1ec-43ec-9202-dbdb73315e56 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:29.798549 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2cf868fa-b1ec-43ec-9202-dbdb73315e56 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:29.800934 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2cf868fa-b1ec-43ec-9202-dbdb73315e56 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:29.875728 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2cf868fa-b1ec-43ec-9202-dbdb73315e56 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:29.876394 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2cf868fa-b1ec-43ec-9202-dbdb73315e56 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:29.877316 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.objects.instance [None req-2cf868fa-b1ec-43ec-9202-dbdb73315e56 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lazy-loading 'fault' on Instance uuid 1cb418f7-6aea-42e6-88a2-007564d925c4 {{(pid=138681) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:23:29.971719 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-2cf868fa-b1ec-43ec-9202-dbdb73315e56 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f2e974ab5aa48dfb4b00f810dcedb63', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd31be8475a6a49d6bfe8132dd050c6c0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:23:29.986659 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-2cf868fa-b1ec-43ec-9202-dbdb73315e56 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "GET /compute/v2.1/servers/1cb418f7-6aea-42e6-88a2-007564d925c4" status: 200 len: 1305 microversion: 2.1 time: 0.216391 Jul 16 21:23:29.990541 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1621/3273] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:23:29 2022] GET /compute/v2.1/servers/1cb418f7-6aea-42e6-88a2-007564d925c4 => generated 1305 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:23:31.016963 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-5bdeef1a-e657-4934-8e08-be26266bec69 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:23:31.016963 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-5bdeef1a-e657-4934-8e08-be26266bec69 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 1cb418f7-6aea-42e6-88a2-007564d925c4] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:23:31.021650 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5bdeef1a-e657-4934-8e08-be26266bec69 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:31.022527 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5bdeef1a-e657-4934-8e08-be26266bec69 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:31.023576 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5bdeef1a-e657-4934-8e08-be26266bec69 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:31.024220 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5bdeef1a-e657-4934-8e08-be26266bec69 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:31.051711 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-5bdeef1a-e657-4934-8e08-be26266bec69 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] HTTP exception thrown: Instance 1cb418f7-6aea-42e6-88a2-007564d925c4 could not be found. Jul 16 21:23:31.053833 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-5bdeef1a-e657-4934-8e08-be26266bec69 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Request method failure captured: Jul 16 21:23:31.053833 ncdv-master-1 devstack@n-api.service[138680]: request: GET /compute/v2.1/servers/1cb418f7-6aea-42e6-88a2-007564d925c4 HTTP/1.1 Jul 16 21:23:31.053833 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:23:31.053833 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:23:31.053833 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:23:31.053833 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 0 Jul 16 21:23:31.053833 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:23:31.053833 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:23:31.053833 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:23:31.053833 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:23:31.053833 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0yvs4CI48W3ZCHoQ8huRZwHvqHxAdwrwrvk1v_ETBPx6FU4DdiPUjkjq4hN_4g_fHLAB_XZ9-AAN8Pyvy0KON2X25WlmyilO65MDHbIW9Xy9v9USVPAu_Ce-ze5X9UIHyned0AUneE4UPGiLLIkoXeoS5ekiq1pi209pud2lyUAuHgw Jul 16 21:23:31.053833 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:23:31.053833 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:23:31.053833 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:23:31.053833 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:23:31.053833 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:23:31.053833 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:23:31.053833 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: d31be8475a6a49d6bfe8132dd050c6c0 Jul 16 21:23:31.053833 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-DeleteServersTestJSON-1541892181 Jul 16 21:23:31.053833 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:23:31.053833 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:23:31.053833 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/d31be8475a6a49d6bfe8132dd050c6c0"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/d31be8475a6a49d6bfe8132dd050c6c0"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_d31be8475a6a49d6bfe8132dd050c6c0"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/d31be8475a6a49d6bfe8132dd050c6c0"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:23:31.053833 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-DeleteServersTestJSON-1541892181 Jul 16 21:23:31.053833 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: d31be8475a6a49d6bfe8132dd050c6c0 Jul 16 21:23:31.053833 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-DeleteServersTestJSON-1541892181 Jul 16 21:23:31.053833 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-DeleteServersTestJSON-1541892181-project Jul 16 21:23:31.053833 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:23:31.053833 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:23:31.053833 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: 6f2e974ab5aa48dfb4b00f810dcedb63 Jul 16 21:23:31.053833 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-DeleteServersTestJSON-1541892181-project Jul 16 21:23:31.053833 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 21:23:31.053833 ncdv-master-1 devstack@n-api.service[138680]: exception: Instance 1cb418f7-6aea-42e6-88a2-007564d925c4 could not be found. Jul 16 21:23:31.053833 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': '1cb418f7-6aea-42e6-88a2-007564d925c4'} Jul 16 21:23:31.053833 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:23:31.053833 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:23:31.053833 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:23:31.053833 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:23:31.053833 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:23:31.053833 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:23:31.053833 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:23:31.053833 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:23:31.053833 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:23:31.053833 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:23:31.053833 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 1cb418f7-6aea-42e6-88a2-007564d925c4 could not be found. Jul 16 21:23:31.053833 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:23:31.053833 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:23:31.053833 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:23:31.053833 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:23:31.053833 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:23:31.053833 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:23:31.053833 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:23:31.053833 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:23:31.053833 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:23:31.053833 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:23:31.053833 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:23:31.053833 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:23:31.053833 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:23:31.053833 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:23:31.053833 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:23:31.053833 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:23:31.053833 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 1cb418f7-6aea-42e6-88a2-007564d925c4 could not be found. Jul 16 21:23:31.053833 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:23:31.053833 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:23:31.053833 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:23:31.059360 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:23:31.059360 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:23:31.059360 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:23:31.059360 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:23:31.059360 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:23:31.059360 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 1cb418f7-6aea-42e6-88a2-007564d925c4 could not be found. Jul 16 21:23:31.059360 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:23:31.063176 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-5bdeef1a-e657-4934-8e08-be26266bec69 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Returning 404 to user: Instance 1cb418f7-6aea-42e6-88a2-007564d925c4 could not be found. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:23:31.072006 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-5bdeef1a-e657-4934-8e08-be26266bec69 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "GET /compute/v2.1/servers/1cb418f7-6aea-42e6-88a2-007564d925c4" status: 404 len: 111 microversion: 2.1 time: 0.058208 Jul 16 21:23:31.072006 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1653/3274] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:23:31 2022] GET /compute/v2.1/servers/1cb418f7-6aea-42e6-88a2-007564d925c4 => generated 111 bytes in 60 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:23:31.195582 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-7457c750-39dc-4ed7-aa17-c78ae14c00c8 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-DeleteServersTestJSON-server-520135129", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452", "networks": [{"uuid": "fc3b835d-0a7f-4948-9fcb-1edfd6b9a8df"}]}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:23:31.276774 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.network.neutron [None req-7457c750-39dc-4ed7-aa17-c78ae14c00c8 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] validate_networks() for [('fc3b835d-0a7f-4948-9fcb-1edfd6b9a8df', None, None, None, None, None)] {{(pid=138680) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 16 21:23:31.344628 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-75b0d559-6daf-4d3d-a7a6-7cdb1ba4f15f tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-\u20a1-485362613", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452", "networks": [{"uuid": "2202d4ad-313c-4607-b9fc-eb98bac9afd0"}]}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:23:31.403629 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.network.neutron [None req-75b0d559-6daf-4d3d-a7a6-7cdb1ba4f15f tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] validate_networks() for [('2202d4ad-313c-4607-b9fc-eb98bac9afd0', None, None, None, None, None)] {{(pid=138681) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 16 21:23:34.330606 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-7457c750-39dc-4ed7-aa17-c78ae14c00c8 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Flavor limits 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 16 21:23:34.331712 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-7457c750-39dc-4ed7-aa17-c78ae14c00c8 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Image limits 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 16 21:23:34.332721 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-7457c750-39dc-4ed7-aa17-c78ae14c00c8 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Flavor pref 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 16 21:23:34.333322 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-7457c750-39dc-4ed7-aa17-c78ae14c00c8 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Image pref 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 16 21:23:34.333745 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-7457c750-39dc-4ed7-aa17-c78ae14c00c8 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 16 21:23:34.344870 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7457c750-39dc-4ed7-aa17-c78ae14c00c8 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:34.345425 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7457c750-39dc-4ed7-aa17-c78ae14c00c8 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:34.346483 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7457c750-39dc-4ed7-aa17-c78ae14c00c8 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:34.347001 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7457c750-39dc-4ed7-aa17-c78ae14c00c8 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:34.371630 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-7457c750-39dc-4ed7-aa17-c78ae14c00c8 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Getting quotas for project d31be8475a6a49d6bfe8132dd050c6c0. Resources: {'ram', 'instances', 'cores'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:23:34.378353 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-7457c750-39dc-4ed7-aa17-c78ae14c00c8 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Getting quotas for user 6f2e974ab5aa48dfb4b00f810dcedb63 and project d31be8475a6a49d6bfe8132dd050c6c0. Resources: {'ram', 'instances', 'cores'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:23:34.392262 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-7457c750-39dc-4ed7-aa17-c78ae14c00c8 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Going to run 1 instances... {{(pid=138680) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 16 21:23:34.394566 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-7457c750-39dc-4ed7-aa17-c78ae14c00c8 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 8e6eab25-f25e-46a2-9c72-9424c872fb84] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138680) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 16 21:23:34.457449 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-75b0d559-6daf-4d3d-a7a6-7cdb1ba4f15f tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Flavor limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 16 21:23:34.457970 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-75b0d559-6daf-4d3d-a7a6-7cdb1ba4f15f tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Image limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 16 21:23:34.458331 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-75b0d559-6daf-4d3d-a7a6-7cdb1ba4f15f tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Flavor pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 16 21:23:34.458950 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-75b0d559-6daf-4d3d-a7a6-7cdb1ba4f15f tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Image pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 16 21:23:34.459398 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-75b0d559-6daf-4d3d-a7a6-7cdb1ba4f15f tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 16 21:23:34.469373 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-75b0d559-6daf-4d3d-a7a6-7cdb1ba4f15f tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:34.469939 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-75b0d559-6daf-4d3d-a7a6-7cdb1ba4f15f tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:34.470945 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-75b0d559-6daf-4d3d-a7a6-7cdb1ba4f15f tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:34.471464 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-75b0d559-6daf-4d3d-a7a6-7cdb1ba4f15f tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:34.498818 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-75b0d559-6daf-4d3d-a7a6-7cdb1ba4f15f tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Getting quotas for project 1dad823e240149edb30e592380249a13. Resources: {'ram', 'instances', 'cores'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:23:34.505467 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-75b0d559-6daf-4d3d-a7a6-7cdb1ba4f15f tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Getting quotas for user 027dcc48e4fd4c36a5004096ad5e43f8 and project 1dad823e240149edb30e592380249a13. Resources: {'ram', 'instances', 'cores'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:23:34.522528 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-75b0d559-6daf-4d3d-a7a6-7cdb1ba4f15f tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Going to run 1 instances... {{(pid=138681) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 16 21:23:34.525337 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-75b0d559-6daf-4d3d-a7a6-7cdb1ba4f15f tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: e6774019-149e-4804-b0ab-e3fcd137851e] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138681) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 16 21:23:34.565032 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-7457c750-39dc-4ed7-aa17-c78ae14c00c8 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 3.373195 Jul 16 21:23:34.571696 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1654/3275] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 21:23:31 2022] POST /compute/v2.1/servers => generated 380 bytes in 3380 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 16 21:23:34.577191 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-a00218d7-84a6-488d-a1c5-d8839b6d58e6 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:23:34.578420 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-a00218d7-84a6-488d-a1c5-d8839b6d58e6 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 8e6eab25-f25e-46a2-9c72-9424c872fb84] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:23:34.687544 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-75b0d559-6daf-4d3d-a7a6-7cdb1ba4f15f tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 3.528344 Jul 16 21:23:34.692080 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1622/3276] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 21:23:31 2022] POST /compute/v2.1/servers => generated 380 bytes in 3534 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 16 21:23:34.727876 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-60c8eb65-0b80-4fd6-be93-8daa6029fde0 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:23:34.730293 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-60c8eb65-0b80-4fd6-be93-8daa6029fde0 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: e6774019-149e-4804-b0ab-e3fcd137851e] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:23:35.127478 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-a00218d7-84a6-488d-a1c5-d8839b6d58e6 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f2e974ab5aa48dfb4b00f810dcedb63', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd31be8475a6a49d6bfe8132dd050c6c0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:23:35.140670 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-a00218d7-84a6-488d-a1c5-d8839b6d58e6 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "GET /compute/v2.1/servers/8e6eab25-f25e-46a2-9c72-9424c872fb84" status: 200 len: 1214 microversion: 2.1 time: 0.567103 Jul 16 21:23:35.141611 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1655/3277] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:23:34 2022] GET /compute/v2.1/servers/8e6eab25-f25e-46a2-9c72-9424c872fb84 => generated 1214 bytes in 569 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:23:35.204614 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-60c8eb65-0b80-4fd6-be93-8daa6029fde0 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '027dcc48e4fd4c36a5004096ad5e43f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dad823e240149edb30e592380249a13', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:23:35.211553 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-60c8eb65-0b80-4fd6-be93-8daa6029fde0 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "GET /compute/v2.1/servers/e6774019-149e-4804-b0ab-e3fcd137851e" status: 200 len: 1192 microversion: 2.1 time: 0.513110 Jul 16 21:23:35.212239 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1623/3278] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:23:34 2022] GET /compute/v2.1/servers/e6774019-149e-4804-b0ab-e3fcd137851e => generated 1192 bytes in 515 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:23:36.160972 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-6b247bf2-6370-40fb-8d1d-16504393b5ab tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:23:36.161610 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-6b247bf2-6370-40fb-8d1d-16504393b5ab tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 8e6eab25-f25e-46a2-9c72-9424c872fb84] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:23:36.189743 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6b247bf2-6370-40fb-8d1d-16504393b5ab tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:36.190409 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6b247bf2-6370-40fb-8d1d-16504393b5ab tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.012s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:36.191293 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6b247bf2-6370-40fb-8d1d-16504393b5ab tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:36.191771 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6b247bf2-6370-40fb-8d1d-16504393b5ab tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:36.228688 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-81376411-ae90-4d37-b1b2-33278d5ef50e tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:23:36.231692 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-81376411-ae90-4d37-b1b2-33278d5ef50e tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: e6774019-149e-4804-b0ab-e3fcd137851e] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:23:36.244504 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_api:SELECT=95,nova_api:UPDATE=1,nova_api:DELETE=1,nova_api:INSERT=3 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:23:36.432190 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-81376411-ae90-4d37-b1b2-33278d5ef50e tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '027dcc48e4fd4c36a5004096ad5e43f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dad823e240149edb30e592380249a13', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:23:36.437425 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-6b247bf2-6370-40fb-8d1d-16504393b5ab tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f2e974ab5aa48dfb4b00f810dcedb63', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd31be8475a6a49d6bfe8132dd050c6c0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:23:36.457556 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-81376411-ae90-4d37-b1b2-33278d5ef50e tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "GET /compute/v2.1/servers/e6774019-149e-4804-b0ab-e3fcd137851e" status: 200 len: 1192 microversion: 2.1 time: 0.232674 Jul 16 21:23:36.460277 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-6b247bf2-6370-40fb-8d1d-16504393b5ab tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "GET /compute/v2.1/servers/8e6eab25-f25e-46a2-9c72-9424c872fb84" status: 200 len: 1218 microversion: 2.1 time: 0.305645 Jul 16 21:23:36.464528 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1624/3279] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:23:36 2022] GET /compute/v2.1/servers/e6774019-149e-4804-b0ab-e3fcd137851e => generated 1192 bytes in 240 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:23:36.474353 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1656/3280] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:23:36 2022] GET /compute/v2.1/servers/8e6eab25-f25e-46a2-9c72-9424c872fb84 => generated 1218 bytes in 314 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:23:37.486105 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-aa2a7557-9ea3-44e8-a966-ead5c7c3bf9f tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:23:37.486884 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-aa2a7557-9ea3-44e8-a966-ead5c7c3bf9f tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: e6774019-149e-4804-b0ab-e3fcd137851e] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:23:37.490436 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-0385dafc-bda0-4654-9a75-cd9a90564b18 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:23:37.491695 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-0385dafc-bda0-4654-9a75-cd9a90564b18 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 8e6eab25-f25e-46a2-9c72-9424c872fb84] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:23:37.505777 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0385dafc-bda0-4654-9a75-cd9a90564b18 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:37.506709 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0385dafc-bda0-4654-9a75-cd9a90564b18 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:37.507952 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0385dafc-bda0-4654-9a75-cd9a90564b18 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:37.508952 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0385dafc-bda0-4654-9a75-cd9a90564b18 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:37.519390 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-aa2a7557-9ea3-44e8-a966-ead5c7c3bf9f tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:37.520141 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-aa2a7557-9ea3-44e8-a966-ead5c7c3bf9f tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:37.521150 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-aa2a7557-9ea3-44e8-a966-ead5c7c3bf9f tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:37.522161 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-aa2a7557-9ea3-44e8-a966-ead5c7c3bf9f tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:37.718067 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-aa2a7557-9ea3-44e8-a966-ead5c7c3bf9f tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '027dcc48e4fd4c36a5004096ad5e43f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dad823e240149edb30e592380249a13', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:23:37.726756 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-0385dafc-bda0-4654-9a75-cd9a90564b18 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f2e974ab5aa48dfb4b00f810dcedb63', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd31be8475a6a49d6bfe8132dd050c6c0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:23:37.753056 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-0385dafc-bda0-4654-9a75-cd9a90564b18 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "GET /compute/v2.1/servers/8e6eab25-f25e-46a2-9c72-9424c872fb84" status: 200 len: 1266 microversion: 2.1 time: 0.268817 Jul 16 21:23:37.754316 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1657/3281] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:23:37 2022] GET /compute/v2.1/servers/8e6eab25-f25e-46a2-9c72-9424c872fb84 => generated 1266 bytes in 271 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:23:37.754872 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-aa2a7557-9ea3-44e8-a966-ead5c7c3bf9f tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "GET /compute/v2.1/servers/e6774019-149e-4804-b0ab-e3fcd137851e" status: 200 len: 1244 microversion: 2.1 time: 0.275932 Jul 16 21:23:37.769327 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1625/3282] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:23:37 2022] GET /compute/v2.1/servers/e6774019-149e-4804-b0ab-e3fcd137851e => generated 1244 bytes in 292 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:23:38.775560 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-87eabec7-e0f9-4205-bab9-743146640ef3 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:23:38.776344 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-87eabec7-e0f9-4205-bab9-743146640ef3 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 8e6eab25-f25e-46a2-9c72-9424c872fb84] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:23:38.776848 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-78dbb2b6-3001-46ef-968c-b428f18ce02f tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:23:38.777829 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-78dbb2b6-3001-46ef-968c-b428f18ce02f tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: e6774019-149e-4804-b0ab-e3fcd137851e] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:23:38.788283 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-78dbb2b6-3001-46ef-968c-b428f18ce02f tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:38.788983 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-78dbb2b6-3001-46ef-968c-b428f18ce02f tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:38.790091 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-78dbb2b6-3001-46ef-968c-b428f18ce02f tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:38.790953 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-78dbb2b6-3001-46ef-968c-b428f18ce02f tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:38.797843 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-87eabec7-e0f9-4205-bab9-743146640ef3 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:38.804770 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-87eabec7-e0f9-4205-bab9-743146640ef3 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:38.806315 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-87eabec7-e0f9-4205-bab9-743146640ef3 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:38.807127 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-87eabec7-e0f9-4205-bab9-743146640ef3 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:38.946352 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-87eabec7-e0f9-4205-bab9-743146640ef3 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f2e974ab5aa48dfb4b00f810dcedb63', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd31be8475a6a49d6bfe8132dd050c6c0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:23:38.958751 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-87eabec7-e0f9-4205-bab9-743146640ef3 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "GET /compute/v2.1/servers/8e6eab25-f25e-46a2-9c72-9424c872fb84" status: 200 len: 1272 microversion: 2.1 time: 0.190561 Jul 16 21:23:38.960047 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1658/3283] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:23:38 2022] GET /compute/v2.1/servers/8e6eab25-f25e-46a2-9c72-9424c872fb84 => generated 1272 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:23:39.079879 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-78dbb2b6-3001-46ef-968c-b428f18ce02f tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '027dcc48e4fd4c36a5004096ad5e43f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dad823e240149edb30e592380249a13', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:23:39.092638 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-78dbb2b6-3001-46ef-968c-b428f18ce02f tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "GET /compute/v2.1/servers/e6774019-149e-4804-b0ab-e3fcd137851e" status: 200 len: 1250 microversion: 2.1 time: 0.319340 Jul 16 21:23:39.102451 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1626/3284] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:23:38 2022] GET /compute/v2.1/servers/e6774019-149e-4804-b0ab-e3fcd137851e => generated 1250 bytes in 331 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:23:39.978221 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-c403cccd-2425-49a7-b121-db325d5641dd tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:23:39.981788 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-c403cccd-2425-49a7-b121-db325d5641dd tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 8e6eab25-f25e-46a2-9c72-9424c872fb84] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:23:39.995818 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c403cccd-2425-49a7-b121-db325d5641dd tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:39.997193 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c403cccd-2425-49a7-b121-db325d5641dd tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:40.001307 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c403cccd-2425-49a7-b121-db325d5641dd tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:40.003428 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c403cccd-2425-49a7-b121-db325d5641dd tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:40.110513 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-fd578767-2cbe-4c06-a16e-a3883fae0ae0 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:23:40.111338 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-fd578767-2cbe-4c06-a16e-a3883fae0ae0 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: e6774019-149e-4804-b0ab-e3fcd137851e] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:23:40.138335 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fd578767-2cbe-4c06-a16e-a3883fae0ae0 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:40.140328 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fd578767-2cbe-4c06-a16e-a3883fae0ae0 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:40.142868 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fd578767-2cbe-4c06-a16e-a3883fae0ae0 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:40.143692 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fd578767-2cbe-4c06-a16e-a3883fae0ae0 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:40.267086 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-c403cccd-2425-49a7-b121-db325d5641dd tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f2e974ab5aa48dfb4b00f810dcedb63', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd31be8475a6a49d6bfe8132dd050c6c0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:23:40.275293 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=94,nova_cell1:UPDATE=3,nova_cell1:INSERT=3 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:23:40.278313 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-c403cccd-2425-49a7-b121-db325d5641dd tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "GET /compute/v2.1/servers/8e6eab25-f25e-46a2-9c72-9424c872fb84" status: 200 len: 1272 microversion: 2.1 time: 0.304743 Jul 16 21:23:40.279450 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1659/3285] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:23:39 2022] GET /compute/v2.1/servers/8e6eab25-f25e-46a2-9c72-9424c872fb84 => generated 1272 bytes in 307 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:23:40.354238 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-fd578767-2cbe-4c06-a16e-a3883fae0ae0 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '027dcc48e4fd4c36a5004096ad5e43f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dad823e240149edb30e592380249a13', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:23:40.374002 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-fd578767-2cbe-4c06-a16e-a3883fae0ae0 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "GET /compute/v2.1/servers/e6774019-149e-4804-b0ab-e3fcd137851e" status: 200 len: 1250 microversion: 2.1 time: 0.267916 Jul 16 21:23:40.384986 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1627/3286] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:23:40 2022] GET /compute/v2.1/servers/e6774019-149e-4804-b0ab-e3fcd137851e => generated 1250 bytes in 279 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:23:41.296846 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-46cb3bae-e824-47fa-9fd1-5bb64a2c1433 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:23:41.297469 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-46cb3bae-e824-47fa-9fd1-5bb64a2c1433 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 8e6eab25-f25e-46a2-9c72-9424c872fb84] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:23:41.306775 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-46cb3bae-e824-47fa-9fd1-5bb64a2c1433 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:41.319233 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-46cb3bae-e824-47fa-9fd1-5bb64a2c1433 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:41.320227 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-46cb3bae-e824-47fa-9fd1-5bb64a2c1433 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:41.320671 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-46cb3bae-e824-47fa-9fd1-5bb64a2c1433 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:41.405573 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-fdcfb8bd-9df6-4d37-8661-fbedae782b11 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:23:41.406193 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-fdcfb8bd-9df6-4d37-8661-fbedae782b11 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: e6774019-149e-4804-b0ab-e3fcd137851e] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:23:41.426422 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fdcfb8bd-9df6-4d37-8661-fbedae782b11 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:41.427139 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fdcfb8bd-9df6-4d37-8661-fbedae782b11 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:41.428215 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fdcfb8bd-9df6-4d37-8661-fbedae782b11 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:41.428843 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fdcfb8bd-9df6-4d37-8661-fbedae782b11 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:41.565356 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-fdcfb8bd-9df6-4d37-8661-fbedae782b11 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '027dcc48e4fd4c36a5004096ad5e43f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dad823e240149edb30e592380249a13', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:23:41.580140 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-fdcfb8bd-9df6-4d37-8661-fbedae782b11 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "GET /compute/v2.1/servers/e6774019-149e-4804-b0ab-e3fcd137851e" status: 200 len: 1250 microversion: 2.1 time: 0.192488 Jul 16 21:23:41.582380 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1628/3287] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:23:41 2022] GET /compute/v2.1/servers/e6774019-149e-4804-b0ab-e3fcd137851e => generated 1250 bytes in 195 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:23:41.921881 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-46cb3bae-e824-47fa-9fd1-5bb64a2c1433 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f2e974ab5aa48dfb4b00f810dcedb63', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd31be8475a6a49d6bfe8132dd050c6c0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:23:41.941152 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-46cb3bae-e824-47fa-9fd1-5bb64a2c1433 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "GET /compute/v2.1/servers/8e6eab25-f25e-46a2-9c72-9424c872fb84" status: 200 len: 1314 microversion: 2.1 time: 0.639256 Jul 16 21:23:41.951254 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1660/3288] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:23:41 2022] GET /compute/v2.1/servers/8e6eab25-f25e-46a2-9c72-9424c872fb84 => generated 1314 bytes in 651 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:23:42.611260 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-0666b27d-7f6c-4582-bbf4-347185cb1b1c tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:23:42.611915 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-0666b27d-7f6c-4582-bbf4-347185cb1b1c tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: e6774019-149e-4804-b0ab-e3fcd137851e] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:23:42.625114 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0666b27d-7f6c-4582-bbf4-347185cb1b1c tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:42.625574 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0666b27d-7f6c-4582-bbf4-347185cb1b1c tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:42.626672 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0666b27d-7f6c-4582-bbf4-347185cb1b1c tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:42.627152 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0666b27d-7f6c-4582-bbf4-347185cb1b1c tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:42.959250 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-f6ad714d-0796-46cd-b1ca-e2d9b08c4f58 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:23:42.959884 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-f6ad714d-0796-46cd-b1ca-e2d9b08c4f58 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 8e6eab25-f25e-46a2-9c72-9424c872fb84] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:23:42.968641 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_api:SELECT=94,nova_api:DELETE=1,nova_api:UPDATE=2,nova_api:INSERT=3 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:23:42.979497 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f6ad714d-0796-46cd-b1ca-e2d9b08c4f58 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:42.979967 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f6ad714d-0796-46cd-b1ca-e2d9b08c4f58 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:42.980791 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f6ad714d-0796-46cd-b1ca-e2d9b08c4f58 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:42.981569 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f6ad714d-0796-46cd-b1ca-e2d9b08c4f58 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:43.731702 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-0666b27d-7f6c-4582-bbf4-347185cb1b1c tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '027dcc48e4fd4c36a5004096ad5e43f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dad823e240149edb30e592380249a13', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:23:43.740511 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-0666b27d-7f6c-4582-bbf4-347185cb1b1c tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "GET /compute/v2.1/servers/e6774019-149e-4804-b0ab-e3fcd137851e" status: 200 len: 1292 microversion: 2.1 time: 1.145355 Jul 16 21:23:43.741251 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1629/3289] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:23:42 2022] GET /compute/v2.1/servers/e6774019-149e-4804-b0ab-e3fcd137851e => generated 1292 bytes in 1148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:23:43.872014 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-f6ad714d-0796-46cd-b1ca-e2d9b08c4f58 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f2e974ab5aa48dfb4b00f810dcedb63', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd31be8475a6a49d6bfe8132dd050c6c0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:23:43.880546 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-f6ad714d-0796-46cd-b1ca-e2d9b08c4f58 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "GET /compute/v2.1/servers/8e6eab25-f25e-46a2-9c72-9424c872fb84" status: 200 len: 1314 microversion: 2.1 time: 0.925273 Jul 16 21:23:43.881247 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1661/3290] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:23:42 2022] GET /compute/v2.1/servers/8e6eab25-f25e-46a2-9c72-9424c872fb84 => generated 1314 bytes in 927 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:23:44.761207 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-f1751da1-2b90-4632-9351-7a0f7eb7e997 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:23:44.764543 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-f1751da1-2b90-4632-9351-7a0f7eb7e997 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: e6774019-149e-4804-b0ab-e3fcd137851e] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:23:44.771624 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell0:SELECT=2 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:23:44.774896 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f1751da1-2b90-4632-9351-7a0f7eb7e997 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:44.775862 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f1751da1-2b90-4632-9351-7a0f7eb7e997 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:44.777488 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f1751da1-2b90-4632-9351-7a0f7eb7e997 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:44.782261 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f1751da1-2b90-4632-9351-7a0f7eb7e997 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:44.897116 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-642bb0c4-9584-4fb9-a50b-fdb93fc80a64 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:23:44.897738 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-642bb0c4-9584-4fb9-a50b-fdb93fc80a64 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 8e6eab25-f25e-46a2-9c72-9424c872fb84] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:23:44.902081 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell0:SELECT=2 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:23:44.913684 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-642bb0c4-9584-4fb9-a50b-fdb93fc80a64 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:44.915475 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-642bb0c4-9584-4fb9-a50b-fdb93fc80a64 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:44.916466 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-642bb0c4-9584-4fb9-a50b-fdb93fc80a64 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:44.917078 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-642bb0c4-9584-4fb9-a50b-fdb93fc80a64 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:45.137537 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell0:SELECT=8 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:23:45.782471 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-642bb0c4-9584-4fb9-a50b-fdb93fc80a64 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f2e974ab5aa48dfb4b00f810dcedb63', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd31be8475a6a49d6bfe8132dd050c6c0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:23:45.803836 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-642bb0c4-9584-4fb9-a50b-fdb93fc80a64 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "GET /compute/v2.1/servers/8e6eab25-f25e-46a2-9c72-9424c872fb84" status: 200 len: 1314 microversion: 2.1 time: 0.909323 Jul 16 21:23:45.816725 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1662/3291] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:23:44 2022] GET /compute/v2.1/servers/8e6eab25-f25e-46a2-9c72-9424c872fb84 => generated 1314 bytes in 924 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:23:45.877953 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-f1751da1-2b90-4632-9351-7a0f7eb7e997 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '027dcc48e4fd4c36a5004096ad5e43f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dad823e240149edb30e592380249a13', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:23:45.882523 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-f1751da1-2b90-4632-9351-7a0f7eb7e997 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "GET /compute/v2.1/servers/e6774019-149e-4804-b0ab-e3fcd137851e" status: 200 len: 1292 microversion: 2.1 time: 1.127364 Jul 16 21:23:45.887674 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1630/3292] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:23:44 2022] GET /compute/v2.1/servers/e6774019-149e-4804-b0ab-e3fcd137851e => generated 1292 bytes in 1132 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:23:46.645350 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-6991e2fb-5f0e-4a26-bd74-3ace01beb2f1 req-557c635e-64bd-494c-b2b8-ec623521f3c3 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "8e6eab25-f25e-46a2-9c72-9424c872fb84", "tag": "4a81f7ad-929b-4f37-b5ac-e484b09be512"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:23:46.662826 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-6991e2fb-5f0e-4a26-bd74-3ace01beb2f1 req-557c635e-64bd-494c-b2b8-ec623521f3c3 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:46.663546 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-6991e2fb-5f0e-4a26-bd74-3ace01beb2f1 req-557c635e-64bd-494c-b2b8-ec623521f3c3 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:46.679526 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-6991e2fb-5f0e-4a26-bd74-3ace01beb2f1 req-557c635e-64bd-494c-b2b8-ec623521f3c3 service nova] Creating event network-changed:4a81f7ad-929b-4f37-b5ac-e484b09be512 for instance 8e6eab25-f25e-46a2-9c72-9424c872fb84 on nch2-master-1 Jul 16 21:23:46.685405 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-6991e2fb-5f0e-4a26-bd74-3ace01beb2f1 req-557c635e-64bd-494c-b2b8-ec623521f3c3 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.272819 Jul 16 21:23:46.686115 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1663/3293] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:23:46 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 274 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 16 21:23:46.836456 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-40a4c369-ae3e-428b-bfff-28cb4c4a7621 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:23:46.837551 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-40a4c369-ae3e-428b-bfff-28cb4c4a7621 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 8e6eab25-f25e-46a2-9c72-9424c872fb84] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:23:46.844052 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell0:SELECT=10 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:23:46.851457 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-40a4c369-ae3e-428b-bfff-28cb4c4a7621 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:46.858644 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-40a4c369-ae3e-428b-bfff-28cb4c4a7621 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:46.858644 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-40a4c369-ae3e-428b-bfff-28cb4c4a7621 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:46.858644 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-40a4c369-ae3e-428b-bfff-28cb4c4a7621 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:46.906624 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-d981eb22-22c8-44a7-84b9-3db597a068bc tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:23:46.907680 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-d981eb22-22c8-44a7-84b9-3db597a068bc tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: e6774019-149e-4804-b0ab-e3fcd137851e] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:23:46.922412 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d981eb22-22c8-44a7-84b9-3db597a068bc tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:46.922412 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d981eb22-22c8-44a7-84b9-3db597a068bc tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:46.922412 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d981eb22-22c8-44a7-84b9-3db597a068bc tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:46.922412 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d981eb22-22c8-44a7-84b9-3db597a068bc tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:47.988453 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-40a4c369-ae3e-428b-bfff-28cb4c4a7621 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f2e974ab5aa48dfb4b00f810dcedb63', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd31be8475a6a49d6bfe8132dd050c6c0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:23:47.999548 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-40a4c369-ae3e-428b-bfff-28cb4c4a7621 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "GET /compute/v2.1/servers/8e6eab25-f25e-46a2-9c72-9424c872fb84" status: 200 len: 1314 microversion: 2.1 time: 1.170772 Jul 16 21:23:48.000443 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1631/3294] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:23:46 2022] GET /compute/v2.1/servers/8e6eab25-f25e-46a2-9c72-9424c872fb84 => generated 1314 bytes in 1183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:23:48.056401 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-d981eb22-22c8-44a7-84b9-3db597a068bc tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '027dcc48e4fd4c36a5004096ad5e43f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dad823e240149edb30e592380249a13', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:23:48.065133 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-d981eb22-22c8-44a7-84b9-3db597a068bc tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "GET /compute/v2.1/servers/e6774019-149e-4804-b0ab-e3fcd137851e" status: 200 len: 1292 microversion: 2.1 time: 1.163577 Jul 16 21:23:48.069157 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1664/3295] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:23:46 2022] GET /compute/v2.1/servers/e6774019-149e-4804-b0ab-e3fcd137851e => generated 1292 bytes in 1169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:23:48.718237 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-2f45b8a1-fedd-45b7-bba9-4ff0518f4971 req-6c501fc4-d376-441c-9522-ad6cb9e485ae service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "e6774019-149e-4804-b0ab-e3fcd137851e", "tag": "7893417f-7147-4ca4-88d9-d55421fdc8d5"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:23:48.744612 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-2f45b8a1-fedd-45b7-bba9-4ff0518f4971 req-6c501fc4-d376-441c-9522-ad6cb9e485ae service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:48.745295 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-2f45b8a1-fedd-45b7-bba9-4ff0518f4971 req-6c501fc4-d376-441c-9522-ad6cb9e485ae service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:48.758447 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-2f45b8a1-fedd-45b7-bba9-4ff0518f4971 req-6c501fc4-d376-441c-9522-ad6cb9e485ae service nova] Creating event network-changed:7893417f-7147-4ca4-88d9-d55421fdc8d5 for instance e6774019-149e-4804-b0ab-e3fcd137851e on nch2-master-1 Jul 16 21:23:48.767279 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-2f45b8a1-fedd-45b7-bba9-4ff0518f4971 req-6c501fc4-d376-441c-9522-ad6cb9e485ae service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.052634 Jul 16 21:23:48.768840 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1632/3296] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:23:48 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:23:49.019371 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-2962c556-432e-41e7-8642-6a4f2ae63429 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:23:49.020351 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-2962c556-432e-41e7-8642-6a4f2ae63429 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 8e6eab25-f25e-46a2-9c72-9424c872fb84] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:23:49.032887 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2962c556-432e-41e7-8642-6a4f2ae63429 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:49.038656 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2962c556-432e-41e7-8642-6a4f2ae63429 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:49.039947 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2962c556-432e-41e7-8642-6a4f2ae63429 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:49.040646 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2962c556-432e-41e7-8642-6a4f2ae63429 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:49.092352 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-55081ad1-1178-4393-87d6-ea0a7a18e0f3 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:23:49.093831 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-55081ad1-1178-4393-87d6-ea0a7a18e0f3 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: e6774019-149e-4804-b0ab-e3fcd137851e] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:23:49.116931 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-55081ad1-1178-4393-87d6-ea0a7a18e0f3 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:49.118458 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-55081ad1-1178-4393-87d6-ea0a7a18e0f3 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:49.121073 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-55081ad1-1178-4393-87d6-ea0a7a18e0f3 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:49.130117 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-55081ad1-1178-4393-87d6-ea0a7a18e0f3 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.009s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:50.092348 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-55081ad1-1178-4393-87d6-ea0a7a18e0f3 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '027dcc48e4fd4c36a5004096ad5e43f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dad823e240149edb30e592380249a13', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:23:50.092880 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-2962c556-432e-41e7-8642-6a4f2ae63429 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f2e974ab5aa48dfb4b00f810dcedb63', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd31be8475a6a49d6bfe8132dd050c6c0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:23:50.109087 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-2962c556-432e-41e7-8642-6a4f2ae63429 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "GET /compute/v2.1/servers/8e6eab25-f25e-46a2-9c72-9424c872fb84" status: 200 len: 1314 microversion: 2.1 time: 1.095680 Jul 16 21:23:50.122247 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-55081ad1-1178-4393-87d6-ea0a7a18e0f3 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "GET /compute/v2.1/servers/e6774019-149e-4804-b0ab-e3fcd137851e" status: 200 len: 1292 microversion: 2.1 time: 1.027304 Jul 16 21:23:50.122247 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1633/3297] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:23:49 2022] GET /compute/v2.1/servers/e6774019-149e-4804-b0ab-e3fcd137851e => generated 1292 bytes in 1030 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:23:50.122846 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1665/3298] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:23:49 2022] GET /compute/v2.1/servers/8e6eab25-f25e-46a2-9c72-9424c872fb84 => generated 1314 bytes in 1102 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:23:50.224063 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-60e39fb6-9bcb-4ecd-a847-27b8a1cbe2c9 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerRescueNegativeTestJSON-server-1730361556", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "}]}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:23:50.327370 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.network.neutron [None req-60e39fb6-9bcb-4ecd-a847-27b8a1cbe2c9 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] validate_networks() for [('61f22a51-464f-433b-a3f7-919d551a0f2d', None, None, None, None, None)] {{(pid=138681) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 16 21:23:51.131226 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-4c329e98-4d03-45e5-bff0-3b5dac489c91 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:23:51.132673 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-4c329e98-4d03-45e5-bff0-3b5dac489c91 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: e6774019-149e-4804-b0ab-e3fcd137851e] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:23:51.148578 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4c329e98-4d03-45e5-bff0-3b5dac489c91 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:51.153306 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4c329e98-4d03-45e5-bff0-3b5dac489c91 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:51.154226 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4c329e98-4d03-45e5-bff0-3b5dac489c91 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:51.154687 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4c329e98-4d03-45e5-bff0-3b5dac489c91 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:51.897906 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-4c329e98-4d03-45e5-bff0-3b5dac489c91 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '027dcc48e4fd4c36a5004096ad5e43f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dad823e240149edb30e592380249a13', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:23:51.910437 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-4c329e98-4d03-45e5-bff0-3b5dac489c91 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "GET /compute/v2.1/servers/e6774019-149e-4804-b0ab-e3fcd137851e" status: 200 len: 1292 microversion: 2.1 time: 0.784971 Jul 16 21:23:51.946305 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1666/3299] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:23:51 2022] GET /compute/v2.1/servers/e6774019-149e-4804-b0ab-e3fcd137851e => generated 1292 bytes in 814 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:23:51.962551 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-aecd24c5-0979-4148-b71b-5db8ff193abf tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:23:51.976249 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-aecd24c5-0979-4148-b71b-5db8ff193abf tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 8e6eab25-f25e-46a2-9c72-9424c872fb84] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:23:51.983619 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-aecd24c5-0979-4148-b71b-5db8ff193abf tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:51.984092 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-aecd24c5-0979-4148-b71b-5db8ff193abf tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:51.985103 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-aecd24c5-0979-4148-b71b-5db8ff193abf tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:51.985711 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-aecd24c5-0979-4148-b71b-5db8ff193abf tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:52.177458 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-60e39fb6-9bcb-4ecd-a847-27b8a1cbe2c9 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Flavor limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 16 21:23:52.177991 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-60e39fb6-9bcb-4ecd-a847-27b8a1cbe2c9 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Image limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 16 21:23:52.178504 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-60e39fb6-9bcb-4ecd-a847-27b8a1cbe2c9 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Flavor pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 16 21:23:52.178786 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-60e39fb6-9bcb-4ecd-a847-27b8a1cbe2c9 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Image pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 16 21:23:52.179252 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-60e39fb6-9bcb-4ecd-a847-27b8a1cbe2c9 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 16 21:23:52.191285 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-60e39fb6-9bcb-4ecd-a847-27b8a1cbe2c9 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:52.192033 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-60e39fb6-9bcb-4ecd-a847-27b8a1cbe2c9 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:52.193567 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-60e39fb6-9bcb-4ecd-a847-27b8a1cbe2c9 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:52.195476 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-60e39fb6-9bcb-4ecd-a847-27b8a1cbe2c9 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:52.254489 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-60e39fb6-9bcb-4ecd-a847-27b8a1cbe2c9 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Getting quotas for project da56fc78f06142aba80ca4c6e3d60475. Resources: {'ram', 'instances', 'cores'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:23:52.291527 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-60e39fb6-9bcb-4ecd-a847-27b8a1cbe2c9 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Getting quotas for user bcc9aa2a091c490599f9a9c41163e2eb and project da56fc78f06142aba80ca4c6e3d60475. Resources: {'ram', 'instances', 'cores'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:23:52.327613 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-60e39fb6-9bcb-4ecd-a847-27b8a1cbe2c9 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Going to run 1 instances... {{(pid=138681) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 16 21:23:52.329997 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-60e39fb6-9bcb-4ecd-a847-27b8a1cbe2c9 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] [instance: 92fbfe5e-07b4-44ec-9729-1a9230f4c02c] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138681) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 16 21:23:52.453615 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-60e39fb6-9bcb-4ecd-a847-27b8a1cbe2c9 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 383 microversion: 2.1 time: 2.327291 Jul 16 21:23:52.454779 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1634/3300] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 21:23:50 2022] POST /compute/v2.1/servers => generated 383 bytes in 2342 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 16 21:23:52.481803 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-e721cf76-6f6a-4b40-9a4f-114ef8ab1e64 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:23:52.482464 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-e721cf76-6f6a-4b40-9a4f-114ef8ab1e64 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] [instance: 92fbfe5e-07b4-44ec-9729-1a9230f4c02c] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:23:52.703395 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-e721cf76-6f6a-4b40-9a4f-114ef8ab1e64 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcc9aa2a091c490599f9a9c41163e2eb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da56fc78f06142aba80ca4c6e3d60475', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:23:52.708153 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-e721cf76-6f6a-4b40-9a4f-114ef8ab1e64 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] 10.222.0.16 "GET /compute/v2.1/servers/92fbfe5e-07b4-44ec-9729-1a9230f4c02c" status: 200 len: 1222 microversion: 2.1 time: 0.247514 Jul 16 21:23:52.715231 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1635/3301] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:23:52 2022] GET /compute/v2.1/servers/92fbfe5e-07b4-44ec-9729-1a9230f4c02c => generated 1222 bytes in 255 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:23:52.920734 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-1ee53fbd-4996-4432-8cd5-e49954f771c5 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerRescueNegativeTestJSON-server-1002204881", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "}]}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:23:53.031038 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.network.neutron [None req-1ee53fbd-4996-4432-8cd5-e49954f771c5 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] validate_networks() for [('61f22a51-464f-433b-a3f7-919d551a0f2d', None, None, None, None, None)] {{(pid=138681) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 16 21:23:53.584169 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-aecd24c5-0979-4148-b71b-5db8ff193abf tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f2e974ab5aa48dfb4b00f810dcedb63', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd31be8475a6a49d6bfe8132dd050c6c0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:23:53.607996 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-aecd24c5-0979-4148-b71b-5db8ff193abf tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "GET /compute/v2.1/servers/8e6eab25-f25e-46a2-9c72-9424c872fb84" status: 200 len: 1314 microversion: 2.1 time: 1.654044 Jul 16 21:23:53.609256 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1667/3302] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:23:51 2022] GET /compute/v2.1/servers/8e6eab25-f25e-46a2-9c72-9424c872fb84 => generated 1314 bytes in 1662 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:23:53.621578 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-ced790b7-20ef-447a-9992-ce0fb564cb57 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:23:53.622464 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-ced790b7-20ef-447a-9992-ce0fb564cb57 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: e6774019-149e-4804-b0ab-e3fcd137851e] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:23:53.641530 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ced790b7-20ef-447a-9992-ce0fb564cb57 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:53.642388 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ced790b7-20ef-447a-9992-ce0fb564cb57 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:53.643457 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ced790b7-20ef-447a-9992-ce0fb564cb57 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:53.644235 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ced790b7-20ef-447a-9992-ce0fb564cb57 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:54.582073 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-ced790b7-20ef-447a-9992-ce0fb564cb57 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '027dcc48e4fd4c36a5004096ad5e43f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dad823e240149edb30e592380249a13', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:23:54.598374 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-ced790b7-20ef-447a-9992-ce0fb564cb57 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "GET /compute/v2.1/servers/e6774019-149e-4804-b0ab-e3fcd137851e" status: 200 len: 1292 microversion: 2.1 time: 0.984184 Jul 16 21:23:54.598374 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1668/3303] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:23:53 2022] GET /compute/v2.1/servers/e6774019-149e-4804-b0ab-e3fcd137851e => generated 1292 bytes in 986 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:23:54.805125 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-bb698d6f-254b-40a2-8c83-c95ac0783582 req-eb80c2b6-95ec-449f-a283-642f578135fe service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "8e6eab25-f25e-46a2-9c72-9424c872fb84", "name": "network-vif-plugged", "status": "completed", "tag": "4a81f7ad-929b-4f37-b5ac-e484b09be512"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:23:54.818047 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-1ee53fbd-4996-4432-8cd5-e49954f771c5 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Flavor limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 16 21:23:54.818988 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-bb698d6f-254b-40a2-8c83-c95ac0783582 req-eb80c2b6-95ec-449f-a283-642f578135fe service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:54.819886 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-bb698d6f-254b-40a2-8c83-c95ac0783582 req-eb80c2b6-95ec-449f-a283-642f578135fe service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:54.820159 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-1ee53fbd-4996-4432-8cd5-e49954f771c5 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Image limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 16 21:23:54.820674 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-1ee53fbd-4996-4432-8cd5-e49954f771c5 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Flavor pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 16 21:23:54.821133 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-1ee53fbd-4996-4432-8cd5-e49954f771c5 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Image pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 16 21:23:54.821547 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-1ee53fbd-4996-4432-8cd5-e49954f771c5 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 16 21:23:54.837230 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1ee53fbd-4996-4432-8cd5-e49954f771c5 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:54.840421 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1ee53fbd-4996-4432-8cd5-e49954f771c5 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:54.841311 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1ee53fbd-4996-4432-8cd5-e49954f771c5 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:54.844580 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1ee53fbd-4996-4432-8cd5-e49954f771c5 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:54.846357 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-bb698d6f-254b-40a2-8c83-c95ac0783582 req-eb80c2b6-95ec-449f-a283-642f578135fe service nova] Creating event network-vif-plugged:4a81f7ad-929b-4f37-b5ac-e484b09be512 for instance 8e6eab25-f25e-46a2-9c72-9424c872fb84 on nch2-master-1 Jul 16 21:23:54.856866 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-bb698d6f-254b-40a2-8c83-c95ac0783582 req-eb80c2b6-95ec-449f-a283-642f578135fe service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.225794 Jul 16 21:23:54.856866 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1669/3304] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:23:54 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 255 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:23:54.866190 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-eda2171b-58c8-4a6f-bb23-7ab312472745 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:23:54.867917 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-eda2171b-58c8-4a6f-bb23-7ab312472745 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 8e6eab25-f25e-46a2-9c72-9424c872fb84] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:23:54.889419 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-1ee53fbd-4996-4432-8cd5-e49954f771c5 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Getting quotas for project da56fc78f06142aba80ca4c6e3d60475. Resources: {'ram', 'instances', 'cores'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:23:54.896541 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-eda2171b-58c8-4a6f-bb23-7ab312472745 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:54.897396 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-eda2171b-58c8-4a6f-bb23-7ab312472745 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:54.898430 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-eda2171b-58c8-4a6f-bb23-7ab312472745 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:54.899192 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-eda2171b-58c8-4a6f-bb23-7ab312472745 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:54.918955 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-1ee53fbd-4996-4432-8cd5-e49954f771c5 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Getting quotas for user bcc9aa2a091c490599f9a9c41163e2eb and project da56fc78f06142aba80ca4c6e3d60475. Resources: {'ram', 'instances', 'cores'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:23:54.953076 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-1ee53fbd-4996-4432-8cd5-e49954f771c5 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Going to run 1 instances... {{(pid=138681) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 16 21:23:54.955629 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-1ee53fbd-4996-4432-8cd5-e49954f771c5 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] [instance: 867b2ab0-9301-414b-b045-85057d642971] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138681) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 16 21:23:55.182231 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-1ee53fbd-4996-4432-8cd5-e49954f771c5 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 383 microversion: 2.1 time: 2.268623 Jul 16 21:23:55.183976 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1636/3305] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 21:23:52 2022] POST /compute/v2.1/servers => generated 383 bytes in 2273 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Jul 16 21:23:55.196550 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-5d3f7636-95ab-4efa-9416-179f833e4ebc tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:23:55.202046 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-5d3f7636-95ab-4efa-9416-179f833e4ebc tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] [instance: 867b2ab0-9301-414b-b045-85057d642971] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:23:55.408366 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-5d3f7636-95ab-4efa-9416-179f833e4ebc tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcc9aa2a091c490599f9a9c41163e2eb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da56fc78f06142aba80ca4c6e3d60475', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:23:55.414412 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-eda2171b-58c8-4a6f-bb23-7ab312472745 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f2e974ab5aa48dfb4b00f810dcedb63', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd31be8475a6a49d6bfe8132dd050c6c0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:23:55.418991 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-5d3f7636-95ab-4efa-9416-179f833e4ebc tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] 10.222.0.16 "GET /compute/v2.1/servers/867b2ab0-9301-414b-b045-85057d642971" status: 200 len: 1222 microversion: 2.1 time: 0.228982 Jul 16 21:23:55.420559 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1637/3306] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:23:55 2022] GET /compute/v2.1/servers/867b2ab0-9301-414b-b045-85057d642971 => generated 1222 bytes in 231 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:23:55.427481 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-eda2171b-58c8-4a6f-bb23-7ab312472745 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "GET /compute/v2.1/servers/8e6eab25-f25e-46a2-9c72-9424c872fb84" status: 200 len: 1314 microversion: 2.1 time: 0.571935 Jul 16 21:23:55.436326 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1670/3307] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:23:54 2022] GET /compute/v2.1/servers/8e6eab25-f25e-46a2-9c72-9424c872fb84 => generated 1314 bytes in 574 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:23:55.613482 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-dc008db0-93ae-4856-a860-3e56e3458ed2 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:23:55.617689 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-dc008db0-93ae-4856-a860-3e56e3458ed2 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: e6774019-149e-4804-b0ab-e3fcd137851e] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:23:55.620742 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-dc008db0-93ae-4856-a860-3e56e3458ed2 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:55.622027 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-dc008db0-93ae-4856-a860-3e56e3458ed2 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:55.623043 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-dc008db0-93ae-4856-a860-3e56e3458ed2 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:55.623687 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-dc008db0-93ae-4856-a860-3e56e3458ed2 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:55.931396 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-dc008db0-93ae-4856-a860-3e56e3458ed2 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '027dcc48e4fd4c36a5004096ad5e43f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dad823e240149edb30e592380249a13', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:23:55.943437 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-dc008db0-93ae-4856-a860-3e56e3458ed2 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "GET /compute/v2.1/servers/e6774019-149e-4804-b0ab-e3fcd137851e" status: 200 len: 1292 microversion: 2.1 time: 0.333294 Jul 16 21:23:55.944029 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1638/3308] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:23:55 2022] GET /compute/v2.1/servers/e6774019-149e-4804-b0ab-e3fcd137851e => generated 1292 bytes in 335 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:23:56.440149 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-3bbc5edb-a723-4fac-9e17-23e2d4e71224 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:23:56.441411 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-3bbc5edb-a723-4fac-9e17-23e2d4e71224 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] [instance: 867b2ab0-9301-414b-b045-85057d642971] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:23:56.450458 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-0458b9e2-9a0d-4681-b92a-ec1e0232ae10 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:23:56.451433 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-0458b9e2-9a0d-4681-b92a-ec1e0232ae10 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 8e6eab25-f25e-46a2-9c72-9424c872fb84] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:23:56.462996 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0458b9e2-9a0d-4681-b92a-ec1e0232ae10 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:56.465324 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0458b9e2-9a0d-4681-b92a-ec1e0232ae10 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:56.479260 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0458b9e2-9a0d-4681-b92a-ec1e0232ae10 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:56.481113 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0458b9e2-9a0d-4681-b92a-ec1e0232ae10 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.013s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:56.545036 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-3bbc5edb-a723-4fac-9e17-23e2d4e71224 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcc9aa2a091c490599f9a9c41163e2eb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da56fc78f06142aba80ca4c6e3d60475', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:23:56.566740 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-3bbc5edb-a723-4fac-9e17-23e2d4e71224 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] 10.222.0.16 "GET /compute/v2.1/servers/867b2ab0-9301-414b-b045-85057d642971" status: 200 len: 1222 microversion: 2.1 time: 0.131719 Jul 16 21:23:56.578250 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1671/3309] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:23:56 2022] GET /compute/v2.1/servers/867b2ab0-9301-414b-b045-85057d642971 => generated 1222 bytes in 146 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:23:56.905052 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-0458b9e2-9a0d-4681-b92a-ec1e0232ae10 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f2e974ab5aa48dfb4b00f810dcedb63', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd31be8475a6a49d6bfe8132dd050c6c0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:23:56.922126 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-0458b9e2-9a0d-4681-b92a-ec1e0232ae10 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "GET /compute/v2.1/servers/8e6eab25-f25e-46a2-9c72-9424c872fb84" status: 200 len: 1314 microversion: 2.1 time: 0.473045 Jul 16 21:23:56.922738 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1639/3310] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:23:56 2022] GET /compute/v2.1/servers/8e6eab25-f25e-46a2-9c72-9424c872fb84 => generated 1314 bytes in 478 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:23:56.952452 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-16a4eb65-b8dd-4ba7-b310-2b2ae1916101 req-196adbb8-07a8-4d1e-85a3-b81519e34518 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "e6774019-149e-4804-b0ab-e3fcd137851e", "name": "network-vif-plugged", "status": "completed", "tag": "7893417f-7147-4ca4-88d9-d55421fdc8d5"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:23:56.960626 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-767147df-d7d1-4127-95af-25a8beb83972 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:23:56.961948 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-767147df-d7d1-4127-95af-25a8beb83972 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: e6774019-149e-4804-b0ab-e3fcd137851e] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:23:56.983695 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-767147df-d7d1-4127-95af-25a8beb83972 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:56.984329 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-767147df-d7d1-4127-95af-25a8beb83972 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:56.985615 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-767147df-d7d1-4127-95af-25a8beb83972 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:56.987586 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-16a4eb65-b8dd-4ba7-b310-2b2ae1916101 req-196adbb8-07a8-4d1e-85a3-b81519e34518 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:56.988115 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-767147df-d7d1-4127-95af-25a8beb83972 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:56.990690 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-16a4eb65-b8dd-4ba7-b310-2b2ae1916101 req-196adbb8-07a8-4d1e-85a3-b81519e34518 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:57.019639 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-16a4eb65-b8dd-4ba7-b310-2b2ae1916101 req-196adbb8-07a8-4d1e-85a3-b81519e34518 service nova] Creating event network-vif-plugged:7893417f-7147-4ca4-88d9-d55421fdc8d5 for instance e6774019-149e-4804-b0ab-e3fcd137851e on nch2-master-1 Jul 16 21:23:57.038839 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-16a4eb65-b8dd-4ba7-b310-2b2ae1916101 req-196adbb8-07a8-4d1e-85a3-b81519e34518 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.091245 Jul 16 21:23:57.040796 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1672/3311] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:23:56 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:23:57.385573 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-767147df-d7d1-4127-95af-25a8beb83972 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '027dcc48e4fd4c36a5004096ad5e43f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dad823e240149edb30e592380249a13', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:23:57.397182 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-767147df-d7d1-4127-95af-25a8beb83972 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "GET /compute/v2.1/servers/e6774019-149e-4804-b0ab-e3fcd137851e" status: 200 len: 1292 microversion: 2.1 time: 0.441064 Jul 16 21:23:57.401745 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1640/3312] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:23:56 2022] GET /compute/v2.1/servers/e6774019-149e-4804-b0ab-e3fcd137851e => generated 1292 bytes in 446 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:23:57.588276 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-9ae9b5a4-2724-4db7-98f6-f4f6e67a84b9 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:23:57.590357 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-9ae9b5a4-2724-4db7-98f6-f4f6e67a84b9 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] [instance: 867b2ab0-9301-414b-b045-85057d642971] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:23:57.602243 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9ae9b5a4-2724-4db7-98f6-f4f6e67a84b9 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:57.602719 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9ae9b5a4-2724-4db7-98f6-f4f6e67a84b9 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:57.603818 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9ae9b5a4-2724-4db7-98f6-f4f6e67a84b9 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:57.606478 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9ae9b5a4-2724-4db7-98f6-f4f6e67a84b9 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:57.736317 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-9ae9b5a4-2724-4db7-98f6-f4f6e67a84b9 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcc9aa2a091c490599f9a9c41163e2eb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da56fc78f06142aba80ca4c6e3d60475', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:23:57.746802 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-9ae9b5a4-2724-4db7-98f6-f4f6e67a84b9 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] 10.222.0.16 "GET /compute/v2.1/servers/867b2ab0-9301-414b-b045-85057d642971" status: 200 len: 1274 microversion: 2.1 time: 0.163673 Jul 16 21:23:57.747449 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1673/3313] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:23:57 2022] GET /compute/v2.1/servers/867b2ab0-9301-414b-b045-85057d642971 => generated 1274 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:23:57.940907 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-13f756f1-a815-45e6-b59d-5090f62fe92b tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:23:57.941856 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-13f756f1-a815-45e6-b59d-5090f62fe92b tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 8e6eab25-f25e-46a2-9c72-9424c872fb84] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:23:57.950014 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-13f756f1-a815-45e6-b59d-5090f62fe92b tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:57.950902 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-13f756f1-a815-45e6-b59d-5090f62fe92b tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:57.952610 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-13f756f1-a815-45e6-b59d-5090f62fe92b tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:57.953394 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-13f756f1-a815-45e6-b59d-5090f62fe92b tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:58.002775 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=86 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:23:58.420495 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-a3c7513b-ba4f-4e93-8859-3e6bfb943838 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:23:58.424503 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-a3c7513b-ba4f-4e93-8859-3e6bfb943838 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: e6774019-149e-4804-b0ab-e3fcd137851e] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:23:58.432282 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a3c7513b-ba4f-4e93-8859-3e6bfb943838 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:58.454327 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a3c7513b-ba4f-4e93-8859-3e6bfb943838 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:58.461602 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a3c7513b-ba4f-4e93-8859-3e6bfb943838 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:58.462350 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a3c7513b-ba4f-4e93-8859-3e6bfb943838 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:58.763618 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-13f756f1-a815-45e6-b59d-5090f62fe92b tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f2e974ab5aa48dfb4b00f810dcedb63', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd31be8475a6a49d6bfe8132dd050c6c0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:23:58.795765 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-13f756f1-a815-45e6-b59d-5090f62fe92b tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "GET /compute/v2.1/servers/8e6eab25-f25e-46a2-9c72-9424c872fb84" status: 200 len: 1501 microversion: 2.1 time: 0.858601 Jul 16 21:23:58.799925 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1641/3314] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:23:57 2022] GET /compute/v2.1/servers/8e6eab25-f25e-46a2-9c72-9424c872fb84 => generated 1501 bytes in 865 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:23:58.829686 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-11a08f28-1fc3-4a74-9709-38feaa43b007 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:23:58.836532 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-11a08f28-1fc3-4a74-9709-38feaa43b007 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] [instance: 867b2ab0-9301-414b-b045-85057d642971] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:23:58.870902 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-11a08f28-1fc3-4a74-9709-38feaa43b007 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:58.878748 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-11a08f28-1fc3-4a74-9709-38feaa43b007 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:58.880627 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-11a08f28-1fc3-4a74-9709-38feaa43b007 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:58.882378 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-11a08f28-1fc3-4a74-9709-38feaa43b007 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:59.128309 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-11a08f28-1fc3-4a74-9709-38feaa43b007 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcc9aa2a091c490599f9a9c41163e2eb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da56fc78f06142aba80ca4c6e3d60475', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:23:59.141180 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-11a08f28-1fc3-4a74-9709-38feaa43b007 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] 10.222.0.16 "GET /compute/v2.1/servers/867b2ab0-9301-414b-b045-85057d642971" status: 200 len: 1280 microversion: 2.1 time: 0.339614 Jul 16 21:23:59.141795 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1642/3315] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:23:58 2022] GET /compute/v2.1/servers/867b2ab0-9301-414b-b045-85057d642971 => generated 1280 bytes in 341 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:23:59.158847 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-18895714-5486-400e-a503-1530c29f8136 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Action: 'action', calling method: >, body: {"shelve": {}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:23:59.159487 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-18895714-5486-400e-a503-1530c29f8136 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 8e6eab25-f25e-46a2-9c72-9424c872fb84] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:23:59.184865 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-18895714-5486-400e-a503-1530c29f8136 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:59.185324 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-18895714-5486-400e-a503-1530c29f8136 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:59.186110 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-18895714-5486-400e-a503-1530c29f8136 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:23:59.190328 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-18895714-5486-400e-a503-1530c29f8136 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:23:59.259262 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.objects.instance [None req-18895714-5486-400e-a503-1530c29f8136 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lazy-loading 'flavor' on Instance uuid 8e6eab25-f25e-46a2-9c72-9424c872fb84 {{(pid=138681) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:23:59.259832 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-a3c7513b-ba4f-4e93-8859-3e6bfb943838 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '027dcc48e4fd4c36a5004096ad5e43f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dad823e240149edb30e592380249a13', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:23:59.273947 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-a3c7513b-ba4f-4e93-8859-3e6bfb943838 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "GET /compute/v2.1/servers/e6774019-149e-4804-b0ab-e3fcd137851e" status: 200 len: 1292 microversion: 2.1 time: 0.859923 Jul 16 21:23:59.286819 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1674/3316] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:23:58 2022] GET /compute/v2.1/servers/e6774019-149e-4804-b0ab-e3fcd137851e => generated 1292 bytes in 873 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:23:59.980198 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-18895714-5486-400e-a503-1530c29f8136 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "POST /compute/v2.1/servers/8e6eab25-f25e-46a2-9c72-9424c872fb84/action" status: 202 len: 0 microversion: 2.1 time: 0.831414 Jul 16 21:23:59.981078 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1643/3317] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 21:23:59 2022] POST /compute/v2.1/servers/8e6eab25-f25e-46a2-9c72-9424c872fb84/action => generated 0 bytes in 839 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 21:23:59.990660 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-f8be306e-37a1-46b2-ae46-9835ce9d0803 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:23:59.991468 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-f8be306e-37a1-46b2-ae46-9835ce9d0803 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 8e6eab25-f25e-46a2-9c72-9424c872fb84] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:24:00.000954 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f8be306e-37a1-46b2-ae46-9835ce9d0803 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:00.001801 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f8be306e-37a1-46b2-ae46-9835ce9d0803 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:00.003693 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f8be306e-37a1-46b2-ae46-9835ce9d0803 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:00.004361 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f8be306e-37a1-46b2-ae46-9835ce9d0803 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:00.170047 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-333bb32c-4087-458f-9630-7d4001275472 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:24:00.171853 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-333bb32c-4087-458f-9630-7d4001275472 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] [instance: 867b2ab0-9301-414b-b045-85057d642971] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:24:00.190158 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-333bb32c-4087-458f-9630-7d4001275472 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:00.190956 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-333bb32c-4087-458f-9630-7d4001275472 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:00.192161 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-333bb32c-4087-458f-9630-7d4001275472 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:00.192479 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-333bb32c-4087-458f-9630-7d4001275472 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:00.981186 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-f8be306e-37a1-46b2-ae46-9835ce9d0803 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f2e974ab5aa48dfb4b00f810dcedb63', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd31be8475a6a49d6bfe8132dd050c6c0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:24:01.014067 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-333bb32c-4087-458f-9630-7d4001275472 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcc9aa2a091c490599f9a9c41163e2eb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da56fc78f06142aba80ca4c6e3d60475', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:24:01.017134 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-f8be306e-37a1-46b2-ae46-9835ce9d0803 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "GET /compute/v2.1/servers/8e6eab25-f25e-46a2-9c72-9424c872fb84" status: 200 len: 1507 microversion: 2.1 time: 1.027557 Jul 16 21:24:01.022961 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1675/3318] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:23:59 2022] GET /compute/v2.1/servers/8e6eab25-f25e-46a2-9c72-9424c872fb84 => generated 1507 bytes in 1035 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:24:01.025425 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-333bb32c-4087-458f-9630-7d4001275472 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] 10.222.0.16 "GET /compute/v2.1/servers/867b2ab0-9301-414b-b045-85057d642971" status: 200 len: 1322 microversion: 2.1 time: 0.870018 Jul 16 21:24:01.038420 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1644/3319] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:24:00 2022] GET /compute/v2.1/servers/867b2ab0-9301-414b-b045-85057d642971 => generated 1322 bytes in 885 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:24:01.043261 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-70cedc66-1ebb-4d73-af12-d504bfbda8bf tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:24:01.044146 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-70cedc66-1ebb-4d73-af12-d504bfbda8bf tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: e6774019-149e-4804-b0ab-e3fcd137851e] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:24:01.051482 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-144a1e47-f77d-4ccc-8492-43734bd09000 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:24:01.056116 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-144a1e47-f77d-4ccc-8492-43734bd09000 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] [instance: f114e124-2903-4f30-b055-9f493e3e9e6c] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:24:01.065328 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-70cedc66-1ebb-4d73-af12-d504bfbda8bf tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:01.068481 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-70cedc66-1ebb-4d73-af12-d504bfbda8bf tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:01.071578 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-70cedc66-1ebb-4d73-af12-d504bfbda8bf tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:01.073833 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-70cedc66-1ebb-4d73-af12-d504bfbda8bf tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:01.076450 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-144a1e47-f77d-4ccc-8492-43734bd09000 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:01.079279 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-144a1e47-f77d-4ccc-8492-43734bd09000 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:01.080109 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-144a1e47-f77d-4ccc-8492-43734bd09000 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:01.080571 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-144a1e47-f77d-4ccc-8492-43734bd09000 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:01.169469 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-144a1e47-f77d-4ccc-8492-43734bd09000 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] 10.222.0.16 "GET /compute/v2.1/servers/f114e124-2903-4f30-b055-9f493e3e9e6c/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.129804 Jul 16 21:24:01.175350 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1645/3320] 10.222.0.16 () {58 vars in 1301 bytes} [Sat Jul 16 21:24:01 2022] GET /compute/v2.1/servers/f114e124-2903-4f30-b055-9f493e3e9e6c/os-volume_attachments => generated 25 bytes in 136 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Jul 16 21:24:01.262470 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-85dc93ef-e08b-480b-a24e-48eba1150661 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:24:01.263262 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-85dc93ef-e08b-480b-a24e-48eba1150661 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] [instance: f114e124-2903-4f30-b055-9f493e3e9e6c] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:24:01.269706 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-85dc93ef-e08b-480b-a24e-48eba1150661 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:01.270091 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-85dc93ef-e08b-480b-a24e-48eba1150661 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:01.271121 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-85dc93ef-e08b-480b-a24e-48eba1150661 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:01.271759 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-85dc93ef-e08b-480b-a24e-48eba1150661 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:01.320315 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-85dc93ef-e08b-480b-a24e-48eba1150661 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] [instance: f114e124-2903-4f30-b055-9f493e3e9e6c] Going to try to terminate instance {{(pid=138681) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 16 21:24:01.592534 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-70cedc66-1ebb-4d73-af12-d504bfbda8bf tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '027dcc48e4fd4c36a5004096ad5e43f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dad823e240149edb30e592380249a13', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:24:01.601064 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-70cedc66-1ebb-4d73-af12-d504bfbda8bf tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "GET /compute/v2.1/servers/e6774019-149e-4804-b0ab-e3fcd137851e" status: 200 len: 1473 microversion: 2.1 time: 0.574191 Jul 16 21:24:01.601799 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1676/3321] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:24:01 2022] GET /compute/v2.1/servers/e6774019-149e-4804-b0ab-e3fcd137851e => generated 1473 bytes in 578 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:24:02.037903 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-9a9ff7fc-2911-4e29-991c-f6f9ca567f79 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:24:02.038666 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-9a9ff7fc-2911-4e29-991c-f6f9ca567f79 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 8e6eab25-f25e-46a2-9c72-9424c872fb84] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:24:02.049391 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9a9ff7fc-2911-4e29-991c-f6f9ca567f79 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:02.049391 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9a9ff7fc-2911-4e29-991c-f6f9ca567f79 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:02.049391 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9a9ff7fc-2911-4e29-991c-f6f9ca567f79 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:02.049391 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9a9ff7fc-2911-4e29-991c-f6f9ca567f79 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:02.223882 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-85dc93ef-e08b-480b-a24e-48eba1150661 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] 10.222.0.16 "DELETE /compute/v2.1/servers/f114e124-2903-4f30-b055-9f493e3e9e6c" status: 204 len: 0 microversion: 2.1 time: 0.970014 Jul 16 21:24:02.227348 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1646/3322] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:24:01 2022] DELETE /compute/v2.1/servers/f114e124-2903-4f30-b055-9f493e3e9e6c => generated 0 bytes in 972 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 16 21:24:02.234278 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-2fbce17a-7113-4aab-9a03-525077cfa34c tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:24:02.238849 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-2fbce17a-7113-4aab-9a03-525077cfa34c tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] [instance: 867b2ab0-9301-414b-b045-85057d642971] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:24:02.258463 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2fbce17a-7113-4aab-9a03-525077cfa34c tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:02.259177 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2fbce17a-7113-4aab-9a03-525077cfa34c tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:02.265125 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2fbce17a-7113-4aab-9a03-525077cfa34c tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:02.265878 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2fbce17a-7113-4aab-9a03-525077cfa34c tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:02.606703 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-9a9ff7fc-2911-4e29-991c-f6f9ca567f79 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f2e974ab5aa48dfb4b00f810dcedb63', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd31be8475a6a49d6bfe8132dd050c6c0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:24:02.613230 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-2fbce17a-7113-4aab-9a03-525077cfa34c tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcc9aa2a091c490599f9a9c41163e2eb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da56fc78f06142aba80ca4c6e3d60475', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:24:02.621544 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-9a9ff7fc-2911-4e29-991c-f6f9ca567f79 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "GET /compute/v2.1/servers/8e6eab25-f25e-46a2-9c72-9424c872fb84" status: 200 len: 1507 microversion: 2.1 time: 0.581306 Jul 16 21:24:02.623696 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1677/3323] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:24:02 2022] GET /compute/v2.1/servers/8e6eab25-f25e-46a2-9c72-9424c872fb84 => generated 1507 bytes in 591 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:24:02.626502 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-2fbce17a-7113-4aab-9a03-525077cfa34c tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] 10.222.0.16 "GET /compute/v2.1/servers/867b2ab0-9301-414b-b045-85057d642971" status: 200 len: 1322 microversion: 2.1 time: 0.400779 Jul 16 21:24:02.631229 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1647/3324] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:24:02 2022] GET /compute/v2.1/servers/867b2ab0-9301-414b-b045-85057d642971 => generated 1322 bytes in 406 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:24:02.654361 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-7dd11710-3a20-41dc-93ee-5aede3de9047 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:24:02.656926 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-20fdd962-500e-4b48-81d1-98f8cffa99b3 req-6de7da04-9398-4581-a393-f9ef8ff1dca2 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "92fbfe5e-07b4-44ec-9729-1a9230f4c02c", "tag": "93df1436-7df4-45cf-b96b-37dafbf7a590"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:24:02.657327 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-7dd11710-3a20-41dc-93ee-5aede3de9047 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] [instance: f114e124-2903-4f30-b055-9f493e3e9e6c] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:24:02.682563 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7dd11710-3a20-41dc-93ee-5aede3de9047 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:02.689498 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7dd11710-3a20-41dc-93ee-5aede3de9047 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:02.690496 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7dd11710-3a20-41dc-93ee-5aede3de9047 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:02.690956 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7dd11710-3a20-41dc-93ee-5aede3de9047 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:02.694332 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-20fdd962-500e-4b48-81d1-98f8cffa99b3 req-6de7da04-9398-4581-a393-f9ef8ff1dca2 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:02.697493 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-20fdd962-500e-4b48-81d1-98f8cffa99b3 req-6de7da04-9398-4581-a393-f9ef8ff1dca2 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:02.739162 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-20fdd962-500e-4b48-81d1-98f8cffa99b3 req-6de7da04-9398-4581-a393-f9ef8ff1dca2 service nova] Creating event network-changed:93df1436-7df4-45cf-b96b-37dafbf7a590 for instance 92fbfe5e-07b4-44ec-9729-1a9230f4c02c on nch1-master-1 Jul 16 21:24:02.760748 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-20fdd962-500e-4b48-81d1-98f8cffa99b3 req-6de7da04-9398-4581-a393-f9ef8ff1dca2 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.117996 Jul 16 21:24:02.764454 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1648/3325] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:24:02 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 131 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:24:02.783497 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-b6d24ee9-5e99-4d70-bc2d-3bb9453db4df tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServersTestJSON-server-1338113604", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "}]}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:24:02.905701 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.network.neutron [None req-b6d24ee9-5e99-4d70-bc2d-3bb9453db4df tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] validate_networks() for [('2202d4ad-313c-4607-b9fc-eb98bac9afd0', None, None, None, None, None)] {{(pid=138681) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 16 21:24:03.157133 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-7dd11710-3a20-41dc-93ee-5aede3de9047 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bba77f6ef42547f1aa8480e27868bd4a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '22ca92a86e6c40d59071e58ce4085568', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:24:03.170215 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-7dd11710-3a20-41dc-93ee-5aede3de9047 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] 10.222.0.16 "GET /compute/v2.1/servers/f114e124-2903-4f30-b055-9f493e3e9e6c" status: 200 len: 1675 microversion: 2.1 time: 0.540720 Jul 16 21:24:03.186930 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1678/3326] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:24:02 2022] GET /compute/v2.1/servers/f114e124-2903-4f30-b055-9f493e3e9e6c => generated 1675 bytes in 559 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:24:03.643535 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-4235f8a2-5985-48c3-8379-bb9915f0da00 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:24:03.655526 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-4235f8a2-5985-48c3-8379-bb9915f0da00 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 8e6eab25-f25e-46a2-9c72-9424c872fb84] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:24:03.671397 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4235f8a2-5985-48c3-8379-bb9915f0da00 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:03.674248 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4235f8a2-5985-48c3-8379-bb9915f0da00 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:03.675455 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4235f8a2-5985-48c3-8379-bb9915f0da00 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:03.683297 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4235f8a2-5985-48c3-8379-bb9915f0da00 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:03.722937 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=100 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:24:04.330021 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-b6d24ee9-5e99-4d70-bc2d-3bb9453db4df tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Flavor limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 16 21:24:04.331255 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-b6d24ee9-5e99-4d70-bc2d-3bb9453db4df tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Image limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 16 21:24:04.333584 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-b6d24ee9-5e99-4d70-bc2d-3bb9453db4df tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Flavor pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 16 21:24:04.334093 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-b6d24ee9-5e99-4d70-bc2d-3bb9453db4df tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Image pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 16 21:24:04.335028 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-b6d24ee9-5e99-4d70-bc2d-3bb9453db4df tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 16 21:24:04.353822 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b6d24ee9-5e99-4d70-bc2d-3bb9453db4df tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:04.359312 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b6d24ee9-5e99-4d70-bc2d-3bb9453db4df tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:04.359312 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b6d24ee9-5e99-4d70-bc2d-3bb9453db4df tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:04.359312 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b6d24ee9-5e99-4d70-bc2d-3bb9453db4df tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:04.391613 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-b6d24ee9-5e99-4d70-bc2d-3bb9453db4df tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Getting quotas for project 1dad823e240149edb30e592380249a13. Resources: {'ram', 'instances', 'cores'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:24:04.407538 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-b6d24ee9-5e99-4d70-bc2d-3bb9453db4df tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Getting quotas for user 027dcc48e4fd4c36a5004096ad5e43f8 and project 1dad823e240149edb30e592380249a13. Resources: {'ram', 'instances', 'cores'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:24:04.434639 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-b6d24ee9-5e99-4d70-bc2d-3bb9453db4df tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Going to run 1 instances... {{(pid=138681) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 16 21:24:04.437118 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-b6d24ee9-5e99-4d70-bc2d-3bb9453db4df tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: b442581d-3a9f-4a24-9fae-c2abf958b2bd] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138681) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 16 21:24:04.457340 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_api:SELECT=93,nova_api:INSERT=6,nova_api:UPDATE=1 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:24:04.585073 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-b6d24ee9-5e99-4d70-bc2d-3bb9453db4df tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 1.816694 Jul 16 21:24:04.585755 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1649/3327] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 21:24:02 2022] POST /compute/v2.1/servers => generated 380 bytes in 1821 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 16 21:24:04.589586 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-4235f8a2-5985-48c3-8379-bb9915f0da00 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f2e974ab5aa48dfb4b00f810dcedb63', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd31be8475a6a49d6bfe8132dd050c6c0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:24:04.608126 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-4235f8a2-5985-48c3-8379-bb9915f0da00 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "GET /compute/v2.1/servers/8e6eab25-f25e-46a2-9c72-9424c872fb84" status: 200 len: 1507 microversion: 2.1 time: 0.963904 Jul 16 21:24:04.609916 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-59a1c55e-6ac6-4283-bd46-651a0dcce3d7 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:24:04.611125 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1679/3328] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:24:03 2022] GET /compute/v2.1/servers/8e6eab25-f25e-46a2-9c72-9424c872fb84 => generated 1507 bytes in 975 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:24:04.620014 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-556e57a5-7788-46d2-9689-963426ae3102 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:24:04.621441 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-556e57a5-7788-46d2-9689-963426ae3102 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] [instance: f114e124-2903-4f30-b055-9f493e3e9e6c] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:24:04.624946 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-59a1c55e-6ac6-4283-bd46-651a0dcce3d7 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] [instance: 867b2ab0-9301-414b-b045-85057d642971] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:24:04.635817 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-556e57a5-7788-46d2-9689-963426ae3102 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:04.642028 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-556e57a5-7788-46d2-9689-963426ae3102 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:04.644403 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-556e57a5-7788-46d2-9689-963426ae3102 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:04.646307 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-556e57a5-7788-46d2-9689-963426ae3102 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:04.657925 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-59a1c55e-6ac6-4283-bd46-651a0dcce3d7 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:04.658443 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-59a1c55e-6ac6-4283-bd46-651a0dcce3d7 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:04.668723 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-59a1c55e-6ac6-4283-bd46-651a0dcce3d7 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:04.671223 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-59a1c55e-6ac6-4283-bd46-651a0dcce3d7 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:05.290559 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-556e57a5-7788-46d2-9689-963426ae3102 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bba77f6ef42547f1aa8480e27868bd4a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '22ca92a86e6c40d59071e58ce4085568', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:24:05.294979 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-59a1c55e-6ac6-4283-bd46-651a0dcce3d7 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcc9aa2a091c490599f9a9c41163e2eb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da56fc78f06142aba80ca4c6e3d60475', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:24:05.306693 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-556e57a5-7788-46d2-9689-963426ae3102 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] 10.222.0.16 "GET /compute/v2.1/servers/f114e124-2903-4f30-b055-9f493e3e9e6c" status: 200 len: 1675 microversion: 2.1 time: 0.694228 Jul 16 21:24:05.306866 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-59a1c55e-6ac6-4283-bd46-651a0dcce3d7 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] 10.222.0.16 "GET /compute/v2.1/servers/867b2ab0-9301-414b-b045-85057d642971" status: 200 len: 1322 microversion: 2.1 time: 0.705490 Jul 16 21:24:05.307515 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1650/3329] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:24:04 2022] GET /compute/v2.1/servers/867b2ab0-9301-414b-b045-85057d642971 => generated 1322 bytes in 721 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:24:05.318227 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1680/3330] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:24:04 2022] GET /compute/v2.1/servers/f114e124-2903-4f30-b055-9f493e3e9e6c => generated 1675 bytes in 706 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:24:05.333840 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-f5f9db9f-53b3-4fd1-9ffd-e7fa6da33e85 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:24:05.334382 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-f5f9db9f-53b3-4fd1-9ffd-e7fa6da33e85 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: b442581d-3a9f-4a24-9fae-c2abf958b2bd] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:24:05.385350 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-f5f9db9f-53b3-4fd1-9ffd-e7fa6da33e85 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: b442581d-3a9f-4a24-9fae-c2abf958b2bd] Going to try to terminate instance {{(pid=138680) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 16 21:24:05.815419 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-725ed13b-8f7d-49c0-a584-bb2ac9498206 tempest-FlavorsV2TestJSON-1450577714 tempest-FlavorsV2TestJSON-1450577714-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:24:05.831721 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-725ed13b-8f7d-49c0-a584-bb2ac9498206 tempest-FlavorsV2TestJSON-1450577714 tempest-FlavorsV2TestJSON-1450577714-project] 10.222.0.16 "GET /compute/v2.1/flavors/452" status: 200 len: 370 microversion: 2.1 time: 0.523067 Jul 16 21:24:05.833319 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1651/3331] 10.222.0.16 () {58 vars in 1136 bytes} [Sat Jul 16 21:24:05 2022] GET /compute/v2.1/flavors/452 => generated 370 bytes in 525 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:24:05.844502 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-2783560e-abb5-4088-9124-3b70ac6e9a60 req-3e7b9299-e05e-4359-a8f4-63bde0b8e56c service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "867b2ab0-9301-414b-b045-85057d642971", "tag": "f3410245-1acc-427c-a9fb-aae6491051d9"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:24:05.880401 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-2783560e-abb5-4088-9124-3b70ac6e9a60 req-3e7b9299-e05e-4359-a8f4-63bde0b8e56c service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:05.883723 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-2783560e-abb5-4088-9124-3b70ac6e9a60 req-3e7b9299-e05e-4359-a8f4-63bde0b8e56c service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:05.923192 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-2783560e-abb5-4088-9124-3b70ac6e9a60 req-3e7b9299-e05e-4359-a8f4-63bde0b8e56c service nova] Creating event network-changed:f3410245-1acc-427c-a9fb-aae6491051d9 for instance 867b2ab0-9301-414b-b045-85057d642971 on nch1-master-1 Jul 16 21:24:05.934623 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-2783560e-abb5-4088-9124-3b70ac6e9a60 req-3e7b9299-e05e-4359-a8f4-63bde0b8e56c service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.099723 Jul 16 21:24:05.935613 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1652/3332] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:24:05 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:24:05.946055 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-90a11d68-4e63-40b9-912e-1c3041bdcdac tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:24:05.947051 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-90a11d68-4e63-40b9-912e-1c3041bdcdac tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 8e6eab25-f25e-46a2-9c72-9424c872fb84] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:24:05.967179 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-90a11d68-4e63-40b9-912e-1c3041bdcdac tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:05.967640 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-90a11d68-4e63-40b9-912e-1c3041bdcdac tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:05.973238 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-90a11d68-4e63-40b9-912e-1c3041bdcdac tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:05.973238 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-90a11d68-4e63-40b9-912e-1c3041bdcdac tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:06.682389 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.scheduler.client.report [None req-f5f9db9f-53b3-4fd1-9ffd-e7fa6da33e85 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Deleted allocations for instance b442581d-3a9f-4a24-9fae-c2abf958b2bd Jul 16 21:24:06.763256 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-90a11d68-4e63-40b9-912e-1c3041bdcdac tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f2e974ab5aa48dfb4b00f810dcedb63', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd31be8475a6a49d6bfe8132dd050c6c0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:24:06.783306 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-90a11d68-4e63-40b9-912e-1c3041bdcdac tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "GET /compute/v2.1/servers/8e6eab25-f25e-46a2-9c72-9424c872fb84" status: 200 len: 1507 microversion: 2.1 time: 0.841760 Jul 16 21:24:06.784176 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1653/3333] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:24:05 2022] GET /compute/v2.1/servers/8e6eab25-f25e-46a2-9c72-9424c872fb84 => generated 1507 bytes in 848 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:24:06.797274 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-e14b22aa-38e6-458b-9e53-ecabbde0dedb tempest-FlavorsV2TestJSON-1450577714 tempest-FlavorsV2TestJSON-1450577714-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:24:06.817480 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-f5f9db9f-53b3-4fd1-9ffd-e7fa6da33e85 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "DELETE /compute/v2.1/servers/b442581d-3a9f-4a24-9fae-c2abf958b2bd" status: 204 len: 0 microversion: 2.1 time: 1.497966 Jul 16 21:24:06.820827 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-e14b22aa-38e6-458b-9e53-ecabbde0dedb tempest-FlavorsV2TestJSON-1450577714 tempest-FlavorsV2TestJSON-1450577714-project] 10.222.0.16 "GET /compute/v2.1/flavors" status: 200 len: 2594 microversion: 2.1 time: 0.035319 Jul 16 21:24:06.821852 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1681/3334] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:24:05 2022] DELETE /compute/v2.1/servers/b442581d-3a9f-4a24-9fae-c2abf958b2bd => generated 0 bytes in 1500 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 16 21:24:06.822609 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1654/3335] 10.222.0.16 () {58 vars in 1124 bytes} [Sat Jul 16 21:24:06 2022] GET /compute/v2.1/flavors => generated 2594 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:24:06.828320 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-fa64cbd0-84fa-43ca-af14-0b331520a820 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:24:06.831842 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-1a2be5dd-a1c9-4c10-a906-ddee9f61a2cd req-d54e2575-8147-484a-aad4-46cc837a8af1 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "f114e124-2903-4f30-b055-9f493e3e9e6c", "name": "network-vif-deleted", "tag": "f802c615-4c34-4b94-9946-382deb12b3b2"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:24:06.832830 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-fa64cbd0-84fa-43ca-af14-0b331520a820 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] [instance: f114e124-2903-4f30-b055-9f493e3e9e6c] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:24:06.843374 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fa64cbd0-84fa-43ca-af14-0b331520a820 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:06.844564 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fa64cbd0-84fa-43ca-af14-0b331520a820 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:06.845613 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fa64cbd0-84fa-43ca-af14-0b331520a820 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:06.846494 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fa64cbd0-84fa-43ca-af14-0b331520a820 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:06.852923 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-1a2be5dd-a1c9-4c10-a906-ddee9f61a2cd req-d54e2575-8147-484a-aad4-46cc837a8af1 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:06.854097 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-1a2be5dd-a1c9-4c10-a906-ddee9f61a2cd req-d54e2575-8147-484a-aad4-46cc837a8af1 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:06.894925 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-1a2be5dd-a1c9-4c10-a906-ddee9f61a2cd req-d54e2575-8147-484a-aad4-46cc837a8af1 service nova] Creating event network-vif-deleted:f802c615-4c34-4b94-9946-382deb12b3b2 for instance f114e124-2903-4f30-b055-9f493e3e9e6c on nch1-master-1 Jul 16 21:24:06.905401 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-1a2be5dd-a1c9-4c10-a906-ddee9f61a2cd req-d54e2575-8147-484a-aad4-46cc837a8af1 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.081901 Jul 16 21:24:06.912170 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1682/3336] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:24:06 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:24:06.921618 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-ac3d15db-dac9-4dfb-ad6d-b86f2f28e154 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:24:06.927668 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-ac3d15db-dac9-4dfb-ad6d-b86f2f28e154 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] [instance: 867b2ab0-9301-414b-b045-85057d642971] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:24:06.942215 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ac3d15db-dac9-4dfb-ad6d-b86f2f28e154 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:06.943241 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ac3d15db-dac9-4dfb-ad6d-b86f2f28e154 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:06.944807 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ac3d15db-dac9-4dfb-ad6d-b86f2f28e154 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:06.945459 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ac3d15db-dac9-4dfb-ad6d-b86f2f28e154 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:07.005184 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-fa64cbd0-84fa-43ca-af14-0b331520a820 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bba77f6ef42547f1aa8480e27868bd4a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '22ca92a86e6c40d59071e58ce4085568', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:24:07.019296 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-fa64cbd0-84fa-43ca-af14-0b331520a820 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] 10.222.0.16 "GET /compute/v2.1/servers/f114e124-2903-4f30-b055-9f493e3e9e6c" status: 200 len: 1322 microversion: 2.1 time: 0.193214 Jul 16 21:24:07.019296 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1655/3337] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:24:06 2022] GET /compute/v2.1/servers/f114e124-2903-4f30-b055-9f493e3e9e6c => generated 1322 bytes in 195 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:24:07.025245 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-c0b5255c-c8d0-4410-9fb3-257c2719f79c tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:24:07.026240 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-c0b5255c-c8d0-4410-9fb3-257c2719f79c tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: b442581d-3a9f-4a24-9fae-c2abf958b2bd] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:24:07.034143 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c0b5255c-c8d0-4410-9fb3-257c2719f79c tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:07.035055 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c0b5255c-c8d0-4410-9fb3-257c2719f79c tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:07.036075 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c0b5255c-c8d0-4410-9fb3-257c2719f79c tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:07.036731 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c0b5255c-c8d0-4410-9fb3-257c2719f79c tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:07.083038 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-c0b5255c-c8d0-4410-9fb3-257c2719f79c tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] HTTP exception thrown: Instance b442581d-3a9f-4a24-9fae-c2abf958b2bd could not be found. Jul 16 21:24:07.086529 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-c0b5255c-c8d0-4410-9fb3-257c2719f79c tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Request method failure captured: Jul 16 21:24:07.086529 ncdv-master-1 devstack@n-api.service[138681]: request: GET /compute/v2.1/servers/b442581d-3a9f-4a24-9fae-c2abf958b2bd HTTP/1.1 Jul 16 21:24:07.086529 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:24:07.086529 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:24:07.086529 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:24:07.086529 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 21:24:07.086529 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:24:07.086529 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:24:07.086529 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:24:07.086529 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:24:07.086529 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0yxS9NCWgI1-O55Z_AF16JzmczB_p-s4P30aJMz-9zu9VxcHuIV00tsfB9hOBtHTZ_sI8GPeOOvUrqYZLgOBzCbOMr4ZO506_XIRjMCtHQ0umzvcB9V9iFSNFoiIe_b2jKm-OyHj1IB8YVBCAcwDKANlsCw5UaAoIABK4dLLFjaVpEo Jul 16 21:24:07.086529 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:24:07.086529 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:24:07.086529 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:24:07.086529 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:24:07.086529 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:24:07.086529 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:24:07.086529 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: 1dad823e240149edb30e592380249a13 Jul 16 21:24:07.086529 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-ServersTestJSON-1529284206 Jul 16 21:24:07.086529 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:24:07.086529 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:24:07.086529 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/1dad823e240149edb30e592380249a13"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/1dad823e240149edb30e592380249a13"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_1dad823e240149edb30e592380249a13"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/1dad823e240149edb30e592380249a13"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:24:07.086529 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-ServersTestJSON-1529284206 Jul 16 21:24:07.086529 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: 1dad823e240149edb30e592380249a13 Jul 16 21:24:07.086529 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-ServersTestJSON-1529284206 Jul 16 21:24:07.086529 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-ServersTestJSON-1529284206-project Jul 16 21:24:07.086529 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:24:07.086529 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:24:07.086529 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: 027dcc48e4fd4c36a5004096ad5e43f8 Jul 16 21:24:07.086529 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-ServersTestJSON-1529284206-project Jul 16 21:24:07.086529 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:24:07.086529 ncdv-master-1 devstack@n-api.service[138681]: exception: Instance b442581d-3a9f-4a24-9fae-c2abf958b2bd could not be found. Jul 16 21:24:07.086529 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': 'b442581d-3a9f-4a24-9fae-c2abf958b2bd'} Jul 16 21:24:07.086529 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:24:07.086529 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:24:07.086529 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:24:07.086529 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:24:07.086529 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:24:07.086529 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:24:07.086529 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:24:07.086529 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:24:07.086529 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:24:07.086529 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:24:07.086529 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance b442581d-3a9f-4a24-9fae-c2abf958b2bd could not be found. Jul 16 21:24:07.086529 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:24:07.086529 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:24:07.086529 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:24:07.086529 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:24:07.086529 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:24:07.086529 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:24:07.086529 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:24:07.086529 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:24:07.086529 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:24:07.086529 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:24:07.086529 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:24:07.086529 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:24:07.086529 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:24:07.086529 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:24:07.086529 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:24:07.086529 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:24:07.086529 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance b442581d-3a9f-4a24-9fae-c2abf958b2bd could not be found. Jul 16 21:24:07.086529 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:24:07.086529 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:24:07.086529 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:24:07.086529 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:24:07.094886 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:24:07.094886 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:24:07.094886 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:24:07.094886 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:24:07.094886 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance b442581d-3a9f-4a24-9fae-c2abf958b2bd could not be found. Jul 16 21:24:07.094886 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:24:07.098464 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-c0b5255c-c8d0-4410-9fb3-257c2719f79c tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Returning 404 to user: Instance b442581d-3a9f-4a24-9fae-c2abf958b2bd could not be found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:24:07.099678 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-c0b5255c-c8d0-4410-9fb3-257c2719f79c tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "GET /compute/v2.1/servers/b442581d-3a9f-4a24-9fae-c2abf958b2bd" status: 404 len: 111 microversion: 2.1 time: 0.080228 Jul 16 21:24:07.101894 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1656/3338] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:24:07 2022] GET /compute/v2.1/servers/b442581d-3a9f-4a24-9fae-c2abf958b2bd => generated 111 bytes in 83 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:24:07.112424 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-ad0f6ea8-1e41-4d77-8cb4-6ee3f34fa6cb tempest-FlavorsV2TestJSON-1450577714 tempest-FlavorsV2TestJSON-1450577714-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:24:07.133453 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-ad0f6ea8-1e41-4d77-8cb4-6ee3f34fa6cb tempest-FlavorsV2TestJSON-1450577714 tempest-FlavorsV2TestJSON-1450577714-project] 10.222.0.16 "GET /compute/v2.1/flavors/452" status: 200 len: 370 microversion: 2.1 time: 0.028584 Jul 16 21:24:07.140545 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1657/3339] 10.222.0.16 () {58 vars in 1136 bytes} [Sat Jul 16 21:24:07 2022] GET /compute/v2.1/flavors/452 => generated 370 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:24:07.154504 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-b6df9b7f-4eca-4ced-ac89-62ec91d04262 tempest-FlavorsV2TestJSON-1450577714 tempest-FlavorsV2TestJSON-1450577714-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:24:07.174046 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-b6df9b7f-4eca-4ced-ac89-62ec91d04262 tempest-FlavorsV2TestJSON-1450577714 tempest-FlavorsV2TestJSON-1450577714-project] 10.222.0.16 "GET /compute/v2.1/flavors/452" status: 200 len: 370 microversion: 2.1 time: 0.024245 Jul 16 21:24:07.180340 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1658/3340] 10.222.0.16 () {58 vars in 1136 bytes} [Sat Jul 16 21:24:07 2022] GET /compute/v2.1/flavors/452 => generated 370 bytes in 31 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:24:07.185932 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-8b7a130a-ee52-41ae-9b23-b3a7fda3dcf5 tempest-FlavorsV2TestJSON-1450577714 tempest-FlavorsV2TestJSON-1450577714-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:24:07.217027 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-8b7a130a-ee52-41ae-9b23-b3a7fda3dcf5 tempest-FlavorsV2TestJSON-1450577714 tempest-FlavorsV2TestJSON-1450577714-project] 10.222.0.16 "GET /compute/v2.1/flavors/detail?minDisk=2" status: 200 len: 3209 microversion: 2.1 time: 0.034754 Jul 16 21:24:07.228950 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1659/3341] 10.222.0.16 () {58 vars in 1164 bytes} [Sat Jul 16 21:24:07 2022] GET /compute/v2.1/flavors/detail?minDisk=2 => generated 3209 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:24:07.235778 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-264cab9a-226e-4751-94ee-4c916af855cc tempest-FlavorsV2TestJSON-1450577714 tempest-FlavorsV2TestJSON-1450577714-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:24:07.241459 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-ac3d15db-dac9-4dfb-ad6d-b86f2f28e154 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcc9aa2a091c490599f9a9c41163e2eb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da56fc78f06142aba80ca4c6e3d60475', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:24:07.251107 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-ac3d15db-dac9-4dfb-ad6d-b86f2f28e154 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] 10.222.0.16 "GET /compute/v2.1/servers/867b2ab0-9301-414b-b045-85057d642971" status: 200 len: 1322 microversion: 2.1 time: 0.337709 Jul 16 21:24:07.267623 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1683/3342] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:24:06 2022] GET /compute/v2.1/servers/867b2ab0-9301-414b-b045-85057d642971 => generated 1322 bytes in 355 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:24:07.273788 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-29b9053a-31eb-46e6-b7df-1da0f46a701b tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServersTestJSON-server-729131452", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452", "accessIPv6": "2001:2001::3", "networks": [{"uuid": "2202d4ad-313c-4607-b9fc-eb98bac9afd0"}]}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:24:07.277115 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-264cab9a-226e-4751-94ee-4c916af855cc tempest-FlavorsV2TestJSON-1450577714 tempest-FlavorsV2TestJSON-1450577714-project] 10.222.0.16 "GET /compute/v2.1/flavors/452" status: 200 len: 370 microversion: 2.1 time: 0.045660 Jul 16 21:24:07.281443 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1660/3343] 10.222.0.16 () {58 vars in 1136 bytes} [Sat Jul 16 21:24:07 2022] GET /compute/v2.1/flavors/452 => generated 370 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:24:07.290461 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-abd15671-6b2f-4ef5-9c20-98e27f1f8c8c tempest-FlavorsV2TestJSON-1450577714 tempest-FlavorsV2TestJSON-1450577714-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:24:07.320987 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-abd15671-6b2f-4ef5-9c20-98e27f1f8c8c tempest-FlavorsV2TestJSON-1450577714 tempest-FlavorsV2TestJSON-1450577714-project] 10.222.0.16 "GET /compute/v2.1/flavors/detail?minRam=1025" status: 200 len: 2145 microversion: 2.1 time: 0.034309 Jul 16 21:24:07.331923 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1661/3344] 10.222.0.16 () {58 vars in 1168 bytes} [Sat Jul 16 21:24:07 2022] GET /compute/v2.1/flavors/detail?minRam=1025 => generated 2145 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:24:07.338896 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-edddbf8c-d469-40ad-9e6d-27f8dfd1f9f2 tempest-FlavorsV2TestJSON-1450577714 tempest-FlavorsV2TestJSON-1450577714-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:24:07.352379 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-edddbf8c-d469-40ad-9e6d-27f8dfd1f9f2 tempest-FlavorsV2TestJSON-1450577714 tempest-FlavorsV2TestJSON-1450577714-project] 10.222.0.16 "GET /compute/v2.1/flavors/detail?limit=1" status: 200 len: 476 microversion: 2.1 time: 0.018960 Jul 16 21:24:07.362506 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1662/3345] 10.222.0.16 () {58 vars in 1160 bytes} [Sat Jul 16 21:24:07 2022] GET /compute/v2.1/flavors/detail?limit=1 => generated 476 bytes in 27 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:24:07.374678 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.network.neutron [None req-29b9053a-31eb-46e6-b7df-1da0f46a701b tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] validate_networks() for [('2202d4ad-313c-4607-b9fc-eb98bac9afd0', None, None, None, None, None)] {{(pid=138680) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 16 21:24:07.378753 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-dac2dbcc-b9e3-47a6-8460-b3feb7579f91 tempest-FlavorsV2TestJSON-1450577714 tempest-FlavorsV2TestJSON-1450577714-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:24:07.392884 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-dac2dbcc-b9e3-47a6-8460-b3feb7579f91 tempest-FlavorsV2TestJSON-1450577714 tempest-FlavorsV2TestJSON-1450577714-project] 10.222.0.16 "GET /compute/v2.1/flavors/452" status: 200 len: 370 microversion: 2.1 time: 0.030188 Jul 16 21:24:07.406195 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1663/3346] 10.222.0.16 () {58 vars in 1136 bytes} [Sat Jul 16 21:24:07 2022] GET /compute/v2.1/flavors/452 => generated 370 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:24:07.413243 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-056d2ee2-8f53-4c0e-beb2-e892a798019b tempest-FlavorsV2TestJSON-1450577714 tempest-FlavorsV2TestJSON-1450577714-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:24:07.491711 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-056d2ee2-8f53-4c0e-beb2-e892a798019b tempest-FlavorsV2TestJSON-1450577714 tempest-FlavorsV2TestJSON-1450577714-project] 10.222.0.16 "GET /compute/v2.1/flavors/detail?marker=452" status: 200 len: 2500 microversion: 2.1 time: 0.083985 Jul 16 21:24:07.506708 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1664/3347] 10.222.0.16 () {58 vars in 1166 bytes} [Sat Jul 16 21:24:07 2022] GET /compute/v2.1/flavors/detail?marker=452 => generated 2500 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:24:07.515363 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-33132c2e-6736-41ad-aee5-e1a266bc2275 tempest-FlavorsV2TestJSON-1450577714 tempest-FlavorsV2TestJSON-1450577714-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:24:07.536786 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-33132c2e-6736-41ad-aee5-e1a266bc2275 tempest-FlavorsV2TestJSON-1450577714 tempest-FlavorsV2TestJSON-1450577714-project] 10.222.0.16 "GET /compute/v2.1/flavors/452" status: 200 len: 370 microversion: 2.1 time: 0.026780 Jul 16 21:24:07.548488 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1665/3348] 10.222.0.16 () {58 vars in 1136 bytes} [Sat Jul 16 21:24:07 2022] GET /compute/v2.1/flavors/452 => generated 370 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:24:07.554436 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-4880cd8a-5ece-43dc-b0be-915ac27adec1 tempest-FlavorsV2TestJSON-1450577714 tempest-FlavorsV2TestJSON-1450577714-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:24:07.595297 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-4880cd8a-5ece-43dc-b0be-915ac27adec1 tempest-FlavorsV2TestJSON-1450577714 tempest-FlavorsV2TestJSON-1450577714-project] 10.222.0.16 "GET /compute/v2.1/flavors?minDisk=2" status: 200 len: 1663 microversion: 2.1 time: 0.045171 Jul 16 21:24:07.603421 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1666/3349] 10.222.0.16 () {58 vars in 1143 bytes} [Sat Jul 16 21:24:07 2022] GET /compute/v2.1/flavors?minDisk=2 => generated 1663 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:24:07.610091 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-bd529bf0-5cb4-4287-b2a4-e91de5925d27 tempest-FlavorsV2TestJSON-1450577714 tempest-FlavorsV2TestJSON-1450577714-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:24:07.620905 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-bd529bf0-5cb4-4287-b2a4-e91de5925d27 tempest-FlavorsV2TestJSON-1450577714 tempest-FlavorsV2TestJSON-1450577714-project] 10.222.0.16 "GET /compute/v2.1/flavors/452" status: 200 len: 370 microversion: 2.1 time: 0.015386 Jul 16 21:24:07.627870 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1667/3350] 10.222.0.16 () {58 vars in 1136 bytes} [Sat Jul 16 21:24:07 2022] GET /compute/v2.1/flavors/452 => generated 370 bytes in 24 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:24:07.635569 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-0d4b1260-b438-4fd3-b01b-ad78b28feab3 tempest-FlavorsV2TestJSON-1450577714 tempest-FlavorsV2TestJSON-1450577714-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:24:07.654926 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-0d4b1260-b438-4fd3-b01b-ad78b28feab3 tempest-FlavorsV2TestJSON-1450577714 tempest-FlavorsV2TestJSON-1450577714-project] 10.222.0.16 "GET /compute/v2.1/flavors?minRam=1025" status: 200 len: 1111 microversion: 2.1 time: 0.024746 Jul 16 21:24:07.663316 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1668/3351] 10.222.0.16 () {58 vars in 1147 bytes} [Sat Jul 16 21:24:07 2022] GET /compute/v2.1/flavors?minRam=1025 => generated 1111 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:24:07.669622 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-f0d0f451-f320-4223-8fe0-c05bee4bc744 tempest-FlavorsV2TestJSON-1450577714 tempest-FlavorsV2TestJSON-1450577714-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:24:07.684154 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-f0d0f451-f320-4223-8fe0-c05bee4bc744 tempest-FlavorsV2TestJSON-1450577714 tempest-FlavorsV2TestJSON-1450577714-project] 10.222.0.16 "GET /compute/v2.1/flavors?limit=1" status: 200 len: 299 microversion: 2.1 time: 0.019358 Jul 16 21:24:07.685331 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1669/3352] 10.222.0.16 () {58 vars in 1139 bytes} [Sat Jul 16 21:24:07 2022] GET /compute/v2.1/flavors?limit=1 => generated 299 bytes in 21 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:24:07.701522 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-031754b1-d1d5-4a31-a00b-34d6cee4bce9 tempest-FlavorsV2TestJSON-1450577714 tempest-FlavorsV2TestJSON-1450577714-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:24:07.711372 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-031754b1-d1d5-4a31-a00b-34d6cee4bce9 tempest-FlavorsV2TestJSON-1450577714 tempest-FlavorsV2TestJSON-1450577714-project] 10.222.0.16 "GET /compute/v2.1/flavors/452" status: 200 len: 370 microversion: 2.1 time: 0.016872 Jul 16 21:24:07.719556 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1670/3353] 10.222.0.16 () {58 vars in 1136 bytes} [Sat Jul 16 21:24:07 2022] GET /compute/v2.1/flavors/452 => generated 370 bytes in 26 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 16 21:24:07.740181 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-f38e6a3c-defa-46e9-8435-9d960bbb76ef tempest-FlavorsV2TestJSON-1450577714 tempest-FlavorsV2TestJSON-1450577714-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:24:07.778778 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-f38e6a3c-defa-46e9-8435-9d960bbb76ef tempest-FlavorsV2TestJSON-1450577714 tempest-FlavorsV2TestJSON-1450577714-project] 10.222.0.16 "GET /compute/v2.1/flavors?marker=452" status: 200 len: 1300 microversion: 2.1 time: 0.048404 Jul 16 21:24:07.811210 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1671/3354] 10.222.0.16 () {58 vars in 1145 bytes} [Sat Jul 16 21:24:07 2022] GET /compute/v2.1/flavors?marker=452 => generated 1300 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:24:07.819303 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-ae977632-b7c3-4df7-aa50-173447a9bd50 tempest-FlavorsV2TestJSON-1450577714 tempest-FlavorsV2TestJSON-1450577714-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:24:07.917815 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-ae977632-b7c3-4df7-aa50-173447a9bd50 tempest-FlavorsV2TestJSON-1450577714 tempest-FlavorsV2TestJSON-1450577714-project] 10.222.0.16 "GET /compute/v2.1/flavors/detail" status: 200 len: 4991 microversion: 2.1 time: 0.100990 Jul 16 21:24:07.927079 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1672/3355] 10.222.0.16 () {58 vars in 1145 bytes} [Sat Jul 16 21:24:07 2022] GET /compute/v2.1/flavors/detail => generated 4991 bytes in 111 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:24:07.945162 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-70a15b18-0539-4ac2-b139-1b06b818c282 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:24:07.946013 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-70a15b18-0539-4ac2-b139-1b06b818c282 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 8e6eab25-f25e-46a2-9c72-9424c872fb84] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:24:07.996906 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-70a15b18-0539-4ac2-b139-1b06b818c282 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:07.997542 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-70a15b18-0539-4ac2-b139-1b06b818c282 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.009s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:08.003257 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-70a15b18-0539-4ac2-b139-1b06b818c282 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:08.005157 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-70a15b18-0539-4ac2-b139-1b06b818c282 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:08.414784 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-29b9053a-31eb-46e6-b7df-1da0f46a701b tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Flavor limits 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 16 21:24:08.415594 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-29b9053a-31eb-46e6-b7df-1da0f46a701b tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Image limits 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 16 21:24:08.416090 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-29b9053a-31eb-46e6-b7df-1da0f46a701b tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Flavor pref 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 16 21:24:08.416652 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-29b9053a-31eb-46e6-b7df-1da0f46a701b tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Image pref 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 16 21:24:08.418294 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-29b9053a-31eb-46e6-b7df-1da0f46a701b tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 16 21:24:08.437590 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-29b9053a-31eb-46e6-b7df-1da0f46a701b tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:08.468128 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-29b9053a-31eb-46e6-b7df-1da0f46a701b tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:08.468128 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-29b9053a-31eb-46e6-b7df-1da0f46a701b tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:08.468128 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-29b9053a-31eb-46e6-b7df-1da0f46a701b tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:08.472653 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-70a15b18-0539-4ac2-b139-1b06b818c282 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f2e974ab5aa48dfb4b00f810dcedb63', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd31be8475a6a49d6bfe8132dd050c6c0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:24:08.481662 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-70a15b18-0539-4ac2-b139-1b06b818c282 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "GET /compute/v2.1/servers/8e6eab25-f25e-46a2-9c72-9424c872fb84" status: 200 len: 1507 microversion: 2.1 time: 0.548992 Jul 16 21:24:08.483676 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-29b9053a-31eb-46e6-b7df-1da0f46a701b tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Getting quotas for project 1dad823e240149edb30e592380249a13. Resources: {'ram', 'instances', 'cores'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:24:08.487467 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1673/3356] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:24:07 2022] GET /compute/v2.1/servers/8e6eab25-f25e-46a2-9c72-9424c872fb84 => generated 1507 bytes in 563 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:24:08.504044 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-3e24d27e-4392-4315-bb98-0a66fec3424f tempest-FlavorsV2TestJSON-1450577714 tempest-FlavorsV2TestJSON-1450577714-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:24:08.511850 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-29b9053a-31eb-46e6-b7df-1da0f46a701b tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Getting quotas for user 027dcc48e4fd4c36a5004096ad5e43f8 and project 1dad823e240149edb30e592380249a13. Resources: {'ram', 'instances', 'cores'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:24:08.526291 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-3e24d27e-4392-4315-bb98-0a66fec3424f tempest-FlavorsV2TestJSON-1450577714 tempest-FlavorsV2TestJSON-1450577714-project] 10.222.0.16 "GET /compute/v2.1/flavors/452" status: 200 len: 370 microversion: 2.1 time: 0.036822 Jul 16 21:24:08.530088 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1674/3357] 10.222.0.16 () {58 vars in 1136 bytes} [Sat Jul 16 21:24:08 2022] GET /compute/v2.1/flavors/452 => generated 370 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:24:08.551347 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-29b9053a-31eb-46e6-b7df-1da0f46a701b tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Going to run 1 instances... {{(pid=138680) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 16 21:24:08.556198 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-29b9053a-31eb-46e6-b7df-1da0f46a701b tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: 083a6d48-6623-437b-8649-92a693c68ebf] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138680) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 16 21:24:08.572207 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-f021ef4d-67bd-4d58-95fa-41cbb6ac6249 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:24:08.574010 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-f021ef4d-67bd-4d58-95fa-41cbb6ac6249 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] [instance: f114e124-2903-4f30-b055-9f493e3e9e6c] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:24:08.593591 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f021ef4d-67bd-4d58-95fa-41cbb6ac6249 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:08.594430 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f021ef4d-67bd-4d58-95fa-41cbb6ac6249 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:08.595555 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f021ef4d-67bd-4d58-95fa-41cbb6ac6249 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:08.596181 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f021ef4d-67bd-4d58-95fa-41cbb6ac6249 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:08.664496 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-f021ef4d-67bd-4d58-95fa-41cbb6ac6249 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] HTTP exception thrown: Instance f114e124-2903-4f30-b055-9f493e3e9e6c could not be found. Jul 16 21:24:08.670621 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-f021ef4d-67bd-4d58-95fa-41cbb6ac6249 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Request method failure captured: Jul 16 21:24:08.670621 ncdv-master-1 devstack@n-api.service[138681]: request: GET /compute/v2.1/servers/f114e124-2903-4f30-b055-9f493e3e9e6c HTTP/1.1 Jul 16 21:24:08.670621 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:24:08.670621 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:24:08.670621 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:24:08.670621 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 21:24:08.670621 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:24:08.670621 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:24:08.670621 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:24:08.670621 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:24:08.670621 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0yu6GvklgrJz64l4jI_9XrerhdTTPfYTDqcboWClQTVUOsFR3X72A-yD-xnMKndBGdnSSARa9ZpL_5E_A7dDhsSNRAFKwYpwFmXgR867L_9gOcd2pdrdnzmP_tHWcFxJFbkFQsl_MPEJMXgACkcSj5Ps4CNmUG4pTC9v8iTd4ac5zTs Jul 16 21:24:08.670621 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:24:08.670621 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:24:08.670621 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:24:08.670621 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:24:08.670621 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:24:08.670621 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:24:08.670621 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: 22ca92a86e6c40d59071e58ce4085568 Jul 16 21:24:08.670621 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-AttachVolumeTestJSON-1168541016 Jul 16 21:24:08.670621 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:24:08.670621 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:24:08.670621 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/22ca92a86e6c40d59071e58ce4085568"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/22ca92a86e6c40d59071e58ce4085568"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_22ca92a86e6c40d59071e58ce4085568"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/22ca92a86e6c40d59071e58ce4085568"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:24:08.670621 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-AttachVolumeTestJSON-1168541016 Jul 16 21:24:08.670621 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: 22ca92a86e6c40d59071e58ce4085568 Jul 16 21:24:08.670621 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-AttachVolumeTestJSON-1168541016 Jul 16 21:24:08.670621 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-AttachVolumeTestJSON-1168541016-project Jul 16 21:24:08.670621 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:24:08.670621 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:24:08.670621 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: bba77f6ef42547f1aa8480e27868bd4a Jul 16 21:24:08.670621 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-AttachVolumeTestJSON-1168541016-project Jul 16 21:24:08.670621 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:24:08.670621 ncdv-master-1 devstack@n-api.service[138681]: exception: Instance f114e124-2903-4f30-b055-9f493e3e9e6c could not be found. Jul 16 21:24:08.670621 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': 'f114e124-2903-4f30-b055-9f493e3e9e6c'} Jul 16 21:24:08.670621 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:24:08.670621 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:24:08.670621 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:24:08.670621 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:24:08.670621 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:24:08.670621 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:24:08.670621 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:24:08.670621 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:24:08.670621 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:24:08.670621 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:24:08.670621 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance f114e124-2903-4f30-b055-9f493e3e9e6c could not be found. Jul 16 21:24:08.670621 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:24:08.670621 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:24:08.670621 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:24:08.670621 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:24:08.670621 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:24:08.670621 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:24:08.670621 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:24:08.670621 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:24:08.670621 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:24:08.670621 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:24:08.670621 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:24:08.670621 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:24:08.670621 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:24:08.670621 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:24:08.670621 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:24:08.670621 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:24:08.670621 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance f114e124-2903-4f30-b055-9f493e3e9e6c could not be found. Jul 16 21:24:08.670621 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:24:08.670621 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:24:08.670621 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:24:08.683080 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:24:08.683080 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:24:08.683080 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:24:08.683080 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:24:08.683080 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:24:08.683080 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance f114e124-2903-4f30-b055-9f493e3e9e6c could not be found. Jul 16 21:24:08.683080 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:24:08.683080 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-f021ef4d-67bd-4d58-95fa-41cbb6ac6249 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Returning 404 to user: Instance f114e124-2903-4f30-b055-9f493e3e9e6c could not be found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:24:08.691475 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-f021ef4d-67bd-4d58-95fa-41cbb6ac6249 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] 10.222.0.16 "GET /compute/v2.1/servers/f114e124-2903-4f30-b055-9f493e3e9e6c" status: 404 len: 111 microversion: 2.1 time: 0.125796 Jul 16 21:24:08.694637 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1675/3358] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:24:08 2022] GET /compute/v2.1/servers/f114e124-2903-4f30-b055-9f493e3e9e6c => generated 111 bytes in 164 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:24:08.724129 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-414c225f-89b4-4ac6-bd91-74c593f95cfe tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:24:08.724771 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-414c225f-89b4-4ac6-bd91-74c593f95cfe tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] [instance: 867b2ab0-9301-414b-b045-85057d642971] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:24:08.734190 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-29b9053a-31eb-46e6-b7df-1da0f46a701b tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 1.464375 Jul 16 21:24:08.739503 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1684/3359] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 21:24:07 2022] POST /compute/v2.1/servers => generated 380 bytes in 1468 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 16 21:24:08.757669 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-414c225f-89b4-4ac6-bd91-74c593f95cfe tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:08.758142 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-414c225f-89b4-4ac6-bd91-74c593f95cfe tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:08.758860 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-ccd51282-6311-43ba-93a5-a635aa53f57e tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Calling method '.version_select at 0x7fdd9c5740d0>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:24:08.761856 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-414c225f-89b4-4ac6-bd91-74c593f95cfe tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:08.762329 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-414c225f-89b4-4ac6-bd91-74c593f95cfe tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:08.816909 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=95,nova_cell1:UPDATE=3,nova_cell1:INSERT=2 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:24:08.912059 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-ccd51282-6311-43ba-93a5-a635aa53f57e tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] 10.222.0.16 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1504167600" status: 202 len: 0 microversion: 2.1 time: 0.174379 Jul 16 21:24:08.912654 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1685/3360] 10.222.0.16 () {58 vars in 1220 bytes} [Sat Jul 16 21:24:08 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1504167600 => generated 0 bytes in 176 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 21:24:08.924907 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-fefba20b-f176-41fe-89d1-9164fade7146 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:24:08.928879 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-fefba20b-f176-41fe-89d1-9164fade7146 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: 083a6d48-6623-437b-8649-92a693c68ebf] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:24:09.100997 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-fefba20b-f176-41fe-89d1-9164fade7146 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '027dcc48e4fd4c36a5004096ad5e43f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dad823e240149edb30e592380249a13', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:24:09.107817 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-fefba20b-f176-41fe-89d1-9164fade7146 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "GET /compute/v2.1/servers/083a6d48-6623-437b-8649-92a693c68ebf" status: 200 len: 1220 microversion: 2.1 time: 0.188086 Jul 16 21:24:09.111459 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1686/3361] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:24:08 2022] GET /compute/v2.1/servers/083a6d48-6623-437b-8649-92a693c68ebf => generated 1220 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:24:09.166133 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-414c225f-89b4-4ac6-bd91-74c593f95cfe tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcc9aa2a091c490599f9a9c41163e2eb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da56fc78f06142aba80ca4c6e3d60475', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:24:09.176773 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-414c225f-89b4-4ac6-bd91-74c593f95cfe tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] 10.222.0.16 "GET /compute/v2.1/servers/867b2ab0-9301-414b-b045-85057d642971" status: 200 len: 1322 microversion: 2.1 time: 0.479600 Jul 16 21:24:09.178368 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1676/3362] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:24:08 2022] GET /compute/v2.1/servers/867b2ab0-9301-414b-b045-85057d642971 => generated 1322 bytes in 483 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:24:09.504683 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-6190ab4b-36f3-41f3-bf42-6b40d8cd5411 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:24:09.505322 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-6190ab4b-36f3-41f3-bf42-6b40d8cd5411 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 8e6eab25-f25e-46a2-9c72-9424c872fb84] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:24:09.522256 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6190ab4b-36f3-41f3-bf42-6b40d8cd5411 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:09.522788 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6190ab4b-36f3-41f3-bf42-6b40d8cd5411 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:09.523665 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6190ab4b-36f3-41f3-bf42-6b40d8cd5411 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:09.526574 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6190ab4b-36f3-41f3-bf42-6b40d8cd5411 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:09.945400 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-6190ab4b-36f3-41f3-bf42-6b40d8cd5411 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f2e974ab5aa48dfb4b00f810dcedb63', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd31be8475a6a49d6bfe8132dd050c6c0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:24:09.959708 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-6190ab4b-36f3-41f3-bf42-6b40d8cd5411 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "GET /compute/v2.1/servers/8e6eab25-f25e-46a2-9c72-9424c872fb84" status: 200 len: 1507 microversion: 2.1 time: 0.456957 Jul 16 21:24:09.960328 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1687/3363] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:24:09 2022] GET /compute/v2.1/servers/8e6eab25-f25e-46a2-9c72-9424c872fb84 => generated 1507 bytes in 460 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:24:10.127627 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-9cb07ae1-a10e-4a2f-a48d-6f650b52bb18 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:24:10.128524 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-9cb07ae1-a10e-4a2f-a48d-6f650b52bb18 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: 083a6d48-6623-437b-8649-92a693c68ebf] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:24:10.145565 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9cb07ae1-a10e-4a2f-a48d-6f650b52bb18 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:10.146279 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9cb07ae1-a10e-4a2f-a48d-6f650b52bb18 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:10.147267 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9cb07ae1-a10e-4a2f-a48d-6f650b52bb18 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:10.147988 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9cb07ae1-a10e-4a2f-a48d-6f650b52bb18 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:10.197914 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-cab6c031-fbd3-4a2f-b3bc-06182dbcc88b tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:24:10.199371 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-cab6c031-fbd3-4a2f-b3bc-06182dbcc88b tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] [instance: 867b2ab0-9301-414b-b045-85057d642971] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:24:10.219508 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cab6c031-fbd3-4a2f-b3bc-06182dbcc88b tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:10.222275 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cab6c031-fbd3-4a2f-b3bc-06182dbcc88b tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:10.223480 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cab6c031-fbd3-4a2f-b3bc-06182dbcc88b tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:10.226689 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cab6c031-fbd3-4a2f-b3bc-06182dbcc88b tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:10.288852 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-9cb07ae1-a10e-4a2f-a48d-6f650b52bb18 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '027dcc48e4fd4c36a5004096ad5e43f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dad823e240149edb30e592380249a13', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:24:10.299662 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-9cb07ae1-a10e-4a2f-a48d-6f650b52bb18 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "GET /compute/v2.1/servers/083a6d48-6623-437b-8649-92a693c68ebf" status: 200 len: 1224 microversion: 2.1 time: 0.177615 Jul 16 21:24:10.300562 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1677/3364] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:24:10 2022] GET /compute/v2.1/servers/083a6d48-6623-437b-8649-92a693c68ebf => generated 1224 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:24:10.804647 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-cab6c031-fbd3-4a2f-b3bc-06182dbcc88b tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcc9aa2a091c490599f9a9c41163e2eb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da56fc78f06142aba80ca4c6e3d60475', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:24:10.824394 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-cab6c031-fbd3-4a2f-b3bc-06182dbcc88b tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] 10.222.0.16 "GET /compute/v2.1/servers/867b2ab0-9301-414b-b045-85057d642971" status: 200 len: 1322 microversion: 2.1 time: 0.623288 Jul 16 21:24:10.827469 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1688/3365] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:24:10 2022] GET /compute/v2.1/servers/867b2ab0-9301-414b-b045-85057d642971 => generated 1322 bytes in 638 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:24:10.983190 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-e164cc3a-a4e1-4cba-bddc-5e0e51d12c89 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:24:10.983855 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-e164cc3a-a4e1-4cba-bddc-5e0e51d12c89 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 8e6eab25-f25e-46a2-9c72-9424c872fb84] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:24:10.993499 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e164cc3a-a4e1-4cba-bddc-5e0e51d12c89 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:10.994290 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e164cc3a-a4e1-4cba-bddc-5e0e51d12c89 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:10.995973 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e164cc3a-a4e1-4cba-bddc-5e0e51d12c89 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:10.996813 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e164cc3a-a4e1-4cba-bddc-5e0e51d12c89 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:11.310090 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-e164cc3a-a4e1-4cba-bddc-5e0e51d12c89 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f2e974ab5aa48dfb4b00f810dcedb63', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd31be8475a6a49d6bfe8132dd050c6c0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:24:11.318071 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-54ad7f5d-5383-4302-8be2-e162876cd6f1 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:24:11.318702 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-54ad7f5d-5383-4302-8be2-e162876cd6f1 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: 083a6d48-6623-437b-8649-92a693c68ebf] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:24:11.325733 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-e164cc3a-a4e1-4cba-bddc-5e0e51d12c89 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "GET /compute/v2.1/servers/8e6eab25-f25e-46a2-9c72-9424c872fb84" status: 200 len: 1507 microversion: 2.1 time: 0.350399 Jul 16 21:24:11.337175 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-54ad7f5d-5383-4302-8be2-e162876cd6f1 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:11.337175 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-54ad7f5d-5383-4302-8be2-e162876cd6f1 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:11.337175 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-54ad7f5d-5383-4302-8be2-e162876cd6f1 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:11.337175 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-54ad7f5d-5383-4302-8be2-e162876cd6f1 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:11.339754 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1678/3366] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:24:10 2022] GET /compute/v2.1/servers/8e6eab25-f25e-46a2-9c72-9424c872fb84 => generated 1507 bytes in 366 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:24:11.375958 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-4a0f1d3b-9719-4f7f-a759-7fe534db346d tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Action: 'create', calling method: .version_select at 0x7fdd9cdca8b0>, body: {"keypair": {"name": "tempest-keypair-415855574", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBLlSVTvdUW1lxzlQSyuCZIckyuQ/cHXHBYCJClHSOqLviEDnhmeORdHG6CFX1AqIjWCfyz6mw+zx2ItYjOWyP9UKyMXpfwUXe3WLxMTucKIXL4rUw5xmt2D3JFwwl9tBqA=="}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:24:11.431165 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-4a0f1d3b-9719-4f7f-a759-7fe534db346d tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Getting quotas for project 22ca92a86e6c40d59071e58ce4085568. Resources: {'key_pairs'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:24:11.452898 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-4a0f1d3b-9719-4f7f-a759-7fe534db346d tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Getting quotas for user bba77f6ef42547f1aa8480e27868bd4a and project 22ca92a86e6c40d59071e58ce4085568. Resources: {'key_pairs'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:24:11.496490 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-54ad7f5d-5383-4302-8be2-e162876cd6f1 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '027dcc48e4fd4c36a5004096ad5e43f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dad823e240149edb30e592380249a13', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:24:11.507499 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-54ad7f5d-5383-4302-8be2-e162876cd6f1 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "GET /compute/v2.1/servers/083a6d48-6623-437b-8649-92a693c68ebf" status: 200 len: 1272 microversion: 2.1 time: 0.193393 Jul 16 21:24:11.513219 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1689/3367] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:24:11 2022] GET /compute/v2.1/servers/083a6d48-6623-437b-8649-92a693c68ebf => generated 1272 bytes in 195 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:24:11.652621 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-4a0f1d3b-9719-4f7f-a759-7fe534db346d tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] 10.222.0.16 "POST /compute/v2.1/os-keypairs" status: 200 len: 385 microversion: 2.1 time: 0.309112 Jul 16 21:24:11.653930 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1679/3368] 10.222.0.16 () {60 vars in 1158 bytes} [Sat Jul 16 21:24:11 2022] POST /compute/v2.1/os-keypairs => generated 385 bytes in 311 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:24:11.843770 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-911f7462-94d9-4d0d-b1da-9df06cc069ce tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:24:11.844406 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-911f7462-94d9-4d0d-b1da-9df06cc069ce tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] [instance: 867b2ab0-9301-414b-b045-85057d642971] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:24:11.860667 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-911f7462-94d9-4d0d-b1da-9df06cc069ce tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:11.861142 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-911f7462-94d9-4d0d-b1da-9df06cc069ce tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:11.861937 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-911f7462-94d9-4d0d-b1da-9df06cc069ce tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:11.862757 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-911f7462-94d9-4d0d-b1da-9df06cc069ce tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:12.293163 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-3641cda9-24ba-45d8-a5cd-849c5a2c5d04 req-1df55d13-465f-4c31-b473-3409e03814f1 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "867b2ab0-9301-414b-b045-85057d642971", "name": "network-vif-plugged", "status": "completed", "tag": "f3410245-1acc-427c-a9fb-aae6491051d9"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:24:12.315274 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-3641cda9-24ba-45d8-a5cd-849c5a2c5d04 req-1df55d13-465f-4c31-b473-3409e03814f1 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:12.319239 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-3641cda9-24ba-45d8-a5cd-849c5a2c5d04 req-1df55d13-465f-4c31-b473-3409e03814f1 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:12.328784 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-911f7462-94d9-4d0d-b1da-9df06cc069ce tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcc9aa2a091c490599f9a9c41163e2eb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da56fc78f06142aba80ca4c6e3d60475', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:24:12.339770 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-911f7462-94d9-4d0d-b1da-9df06cc069ce tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] 10.222.0.16 "GET /compute/v2.1/servers/867b2ab0-9301-414b-b045-85057d642971" status: 200 len: 1322 microversion: 2.1 time: 0.499112 Jul 16 21:24:12.343463 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-3641cda9-24ba-45d8-a5cd-849c5a2c5d04 req-1df55d13-465f-4c31-b473-3409e03814f1 service nova] Creating event network-vif-plugged:f3410245-1acc-427c-a9fb-aae6491051d9 for instance 867b2ab0-9301-414b-b045-85057d642971 on nch1-master-1 Jul 16 21:24:12.343606 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1690/3369] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:24:11 2022] GET /compute/v2.1/servers/867b2ab0-9301-414b-b045-85057d642971 => generated 1322 bytes in 504 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:24:12.348325 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-3d900023-a6b3-4179-9a47-c55653271186 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:24:12.351316 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-3641cda9-24ba-45d8-a5cd-849c5a2c5d04 req-1df55d13-465f-4c31-b473-3409e03814f1 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.060278 Jul 16 21:24:12.353213 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1680/3370] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:24:12 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 65 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 16 21:24:12.355334 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-3d900023-a6b3-4179-9a47-c55653271186 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 8e6eab25-f25e-46a2-9c72-9424c872fb84] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:24:12.367334 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3d900023-a6b3-4179-9a47-c55653271186 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:12.368929 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3d900023-a6b3-4179-9a47-c55653271186 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:12.373522 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3d900023-a6b3-4179-9a47-c55653271186 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:12.373829 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3d900023-a6b3-4179-9a47-c55653271186 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:12.534031 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-982ff6ff-2238-4bcb-aedb-582adf0d04c7 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:24:12.535037 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-982ff6ff-2238-4bcb-aedb-582adf0d04c7 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: 083a6d48-6623-437b-8649-92a693c68ebf] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:24:12.544018 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-982ff6ff-2238-4bcb-aedb-582adf0d04c7 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:12.548090 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-982ff6ff-2238-4bcb-aedb-582adf0d04c7 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:12.551627 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-982ff6ff-2238-4bcb-aedb-582adf0d04c7 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:12.551627 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-982ff6ff-2238-4bcb-aedb-582adf0d04c7 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:12.745758 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-982ff6ff-2238-4bcb-aedb-582adf0d04c7 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '027dcc48e4fd4c36a5004096ad5e43f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dad823e240149edb30e592380249a13', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:24:12.759543 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-982ff6ff-2238-4bcb-aedb-582adf0d04c7 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "GET /compute/v2.1/servers/083a6d48-6623-437b-8649-92a693c68ebf" status: 200 len: 1278 microversion: 2.1 time: 0.234423 Jul 16 21:24:12.763786 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1681/3371] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:24:12 2022] GET /compute/v2.1/servers/083a6d48-6623-437b-8649-92a693c68ebf => generated 1278 bytes in 236 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:24:12.832095 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-3d900023-a6b3-4179-9a47-c55653271186 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f2e974ab5aa48dfb4b00f810dcedb63', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd31be8475a6a49d6bfe8132dd050c6c0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:24:12.851293 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-3d900023-a6b3-4179-9a47-c55653271186 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "GET /compute/v2.1/servers/8e6eab25-f25e-46a2-9c72-9424c872fb84" status: 200 len: 1507 microversion: 2.1 time: 0.501244 Jul 16 21:24:12.851293 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1691/3372] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:24:12 2022] GET /compute/v2.1/servers/8e6eab25-f25e-46a2-9c72-9424c872fb84 => generated 1507 bytes in 505 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:24:13.363737 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-070680bb-c582-43e6-a6a8-e2bbd31dd053 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:24:13.365227 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-070680bb-c582-43e6-a6a8-e2bbd31dd053 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] [instance: 867b2ab0-9301-414b-b045-85057d642971] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:24:13.375046 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-070680bb-c582-43e6-a6a8-e2bbd31dd053 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:13.375509 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-070680bb-c582-43e6-a6a8-e2bbd31dd053 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:13.380787 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-070680bb-c582-43e6-a6a8-e2bbd31dd053 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:13.381641 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-070680bb-c582-43e6-a6a8-e2bbd31dd053 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:13.704470 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-070680bb-c582-43e6-a6a8-e2bbd31dd053 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcc9aa2a091c490599f9a9c41163e2eb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da56fc78f06142aba80ca4c6e3d60475', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:24:13.723470 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-070680bb-c582-43e6-a6a8-e2bbd31dd053 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] 10.222.0.16 "GET /compute/v2.1/servers/867b2ab0-9301-414b-b045-85057d642971" status: 200 len: 1322 microversion: 2.1 time: 0.366275 Jul 16 21:24:13.728040 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1682/3373] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:24:13 2022] GET /compute/v2.1/servers/867b2ab0-9301-414b-b045-85057d642971 => generated 1322 bytes in 375 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:24:13.781482 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-666836d4-36ae-4006-96e1-a3361328eee4 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:24:13.782200 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-666836d4-36ae-4006-96e1-a3361328eee4 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: 083a6d48-6623-437b-8649-92a693c68ebf] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:24:13.802056 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-666836d4-36ae-4006-96e1-a3361328eee4 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:13.803447 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-666836d4-36ae-4006-96e1-a3361328eee4 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:13.805871 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-666836d4-36ae-4006-96e1-a3361328eee4 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:13.806444 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-666836d4-36ae-4006-96e1-a3361328eee4 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:13.870960 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-ee56df6c-d134-4e96-b3da-3bb17fd7d7b3 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:24:13.871998 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-ee56df6c-d134-4e96-b3da-3bb17fd7d7b3 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 8e6eab25-f25e-46a2-9c72-9424c872fb84] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:24:13.888101 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ee56df6c-d134-4e96-b3da-3bb17fd7d7b3 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:13.889116 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ee56df6c-d134-4e96-b3da-3bb17fd7d7b3 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:13.890504 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ee56df6c-d134-4e96-b3da-3bb17fd7d7b3 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:13.891859 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ee56df6c-d134-4e96-b3da-3bb17fd7d7b3 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:14.374638 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell0:SELECT=6 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:24:14.970014 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-666836d4-36ae-4006-96e1-a3361328eee4 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '027dcc48e4fd4c36a5004096ad5e43f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dad823e240149edb30e592380249a13', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:24:14.981174 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-666836d4-36ae-4006-96e1-a3361328eee4 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "GET /compute/v2.1/servers/083a6d48-6623-437b-8649-92a693c68ebf" status: 200 len: 1320 microversion: 2.1 time: 1.205055 Jul 16 21:24:14.981823 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1692/3374] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:24:13 2022] GET /compute/v2.1/servers/083a6d48-6623-437b-8649-92a693c68ebf => generated 1320 bytes in 1209 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:24:15.005300 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-c9977313-5631-49f2-b2b1-3cf3f6dd3e8b tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:24:15.006531 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-c9977313-5631-49f2-b2b1-3cf3f6dd3e8b tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] [instance: 867b2ab0-9301-414b-b045-85057d642971] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:24:15.013114 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_api:SELECT=92,nova_api:DELETE=2,nova_api:UPDATE=1,nova_api:INSERT=3 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:24:15.028233 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c9977313-5631-49f2-b2b1-3cf3f6dd3e8b tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:15.028233 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c9977313-5631-49f2-b2b1-3cf3f6dd3e8b tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:15.028233 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c9977313-5631-49f2-b2b1-3cf3f6dd3e8b tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:15.028233 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c9977313-5631-49f2-b2b1-3cf3f6dd3e8b tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:15.097112 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-ee56df6c-d134-4e96-b3da-3bb17fd7d7b3 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f2e974ab5aa48dfb4b00f810dcedb63', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd31be8475a6a49d6bfe8132dd050c6c0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:24:15.114271 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-ee56df6c-d134-4e96-b3da-3bb17fd7d7b3 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "GET /compute/v2.1/servers/8e6eab25-f25e-46a2-9c72-9424c872fb84" status: 200 len: 1507 microversion: 2.1 time: 1.250586 Jul 16 21:24:15.115150 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1683/3375] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:24:13 2022] GET /compute/v2.1/servers/8e6eab25-f25e-46a2-9c72-9424c872fb84 => generated 1507 bytes in 1253 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:24:15.484758 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-c1d607ea-8a56-40e7-9d5f-95bcb9b18868 req-8775f12f-b5ee-4681-a219-addd711c38f1 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "92fbfe5e-07b4-44ec-9729-1a9230f4c02c", "name": "network-vif-plugged", "status": "completed", "tag": "93df1436-7df4-45cf-b96b-37dafbf7a590"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:24:15.511810 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-c1d607ea-8a56-40e7-9d5f-95bcb9b18868 req-8775f12f-b5ee-4681-a219-addd711c38f1 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:15.513003 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-c1d607ea-8a56-40e7-9d5f-95bcb9b18868 req-8775f12f-b5ee-4681-a219-addd711c38f1 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:15.544508 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-c1d607ea-8a56-40e7-9d5f-95bcb9b18868 req-8775f12f-b5ee-4681-a219-addd711c38f1 service nova] Creating event network-vif-plugged:93df1436-7df4-45cf-b96b-37dafbf7a590 for instance 92fbfe5e-07b4-44ec-9729-1a9230f4c02c on nch1-master-1 Jul 16 21:24:15.556035 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-c1d607ea-8a56-40e7-9d5f-95bcb9b18868 req-8775f12f-b5ee-4681-a219-addd711c38f1 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.077744 Jul 16 21:24:15.556731 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1684/3376] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:24:15 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 79 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 16 21:24:15.758154 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-c9977313-5631-49f2-b2b1-3cf3f6dd3e8b tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcc9aa2a091c490599f9a9c41163e2eb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da56fc78f06142aba80ca4c6e3d60475', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:24:15.769049 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-c9977313-5631-49f2-b2b1-3cf3f6dd3e8b tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] 10.222.0.16 "GET /compute/v2.1/servers/867b2ab0-9301-414b-b045-85057d642971" status: 200 len: 1516 microversion: 2.1 time: 0.784661 Jul 16 21:24:15.771278 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1693/3377] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:24:14 2022] GET /compute/v2.1/servers/867b2ab0-9301-414b-b045-85057d642971 => generated 1516 bytes in 788 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:24:15.789281 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-006961c1-f126-4063-9ff2-4aad338c6734 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Action: 'action', calling method: >, body: {"rescue": {"adminPass": "***"}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:24:15.790346 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-006961c1-f126-4063-9ff2-4aad338c6734 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] [instance: 867b2ab0-9301-414b-b045-85057d642971] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:24:15.809043 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-006961c1-f126-4063-9ff2-4aad338c6734 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:15.809636 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-006961c1-f126-4063-9ff2-4aad338c6734 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:15.810498 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-006961c1-f126-4063-9ff2-4aad338c6734 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:15.810947 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-006961c1-f126-4063-9ff2-4aad338c6734 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:15.899739 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.objects.instance [None req-006961c1-f126-4063-9ff2-4aad338c6734 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lazy-loading 'flavor' on Instance uuid 867b2ab0-9301-414b-b045-85057d642971 {{(pid=138681) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:24:15.999969 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-6645c99a-eccc-432b-96ea-fd1a0b4b3f38 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:24:16.001674 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-6645c99a-eccc-432b-96ea-fd1a0b4b3f38 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: 083a6d48-6623-437b-8649-92a693c68ebf] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:24:16.019345 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6645c99a-eccc-432b-96ea-fd1a0b4b3f38 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:16.024030 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6645c99a-eccc-432b-96ea-fd1a0b4b3f38 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:16.025144 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6645c99a-eccc-432b-96ea-fd1a0b4b3f38 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:16.025759 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6645c99a-eccc-432b-96ea-fd1a0b4b3f38 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:16.365427 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-006961c1-f126-4063-9ff2-4aad338c6734 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] 10.222.0.16 "POST /compute/v2.1/servers/867b2ab0-9301-414b-b045-85057d642971/action" status: 200 len: 32 microversion: 2.1 time: 0.580576 Jul 16 21:24:16.366313 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1685/3378] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 21:24:15 2022] POST /compute/v2.1/servers/867b2ab0-9301-414b-b045-85057d642971/action => generated 32 bytes in 583 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Jul 16 21:24:16.391504 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-3360ed14-9fe9-401f-a3d2-e4fb18c43f7f tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:24:16.392474 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-3360ed14-9fe9-401f-a3d2-e4fb18c43f7f tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 8e6eab25-f25e-46a2-9c72-9424c872fb84] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:24:16.407030 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3360ed14-9fe9-401f-a3d2-e4fb18c43f7f tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:16.407721 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3360ed14-9fe9-401f-a3d2-e4fb18c43f7f tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:16.410515 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3360ed14-9fe9-401f-a3d2-e4fb18c43f7f tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:16.410515 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3360ed14-9fe9-401f-a3d2-e4fb18c43f7f tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:16.532238 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-6645c99a-eccc-432b-96ea-fd1a0b4b3f38 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '027dcc48e4fd4c36a5004096ad5e43f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dad823e240149edb30e592380249a13', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:24:16.540636 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-6645c99a-eccc-432b-96ea-fd1a0b4b3f38 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "GET /compute/v2.1/servers/083a6d48-6623-437b-8649-92a693c68ebf" status: 200 len: 1320 microversion: 2.1 time: 0.546130 Jul 16 21:24:16.554447 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1694/3379] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:24:15 2022] GET /compute/v2.1/servers/083a6d48-6623-437b-8649-92a693c68ebf => generated 1320 bytes in 561 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:24:16.562545 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-2bc70072-452e-433f-9d09-402cb251bd99 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:24:16.563539 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-2bc70072-452e-433f-9d09-402cb251bd99 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] [instance: 867b2ab0-9301-414b-b045-85057d642971] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:24:16.570785 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2bc70072-452e-433f-9d09-402cb251bd99 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:16.571402 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2bc70072-452e-433f-9d09-402cb251bd99 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:16.574588 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2bc70072-452e-433f-9d09-402cb251bd99 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:16.575214 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2bc70072-452e-433f-9d09-402cb251bd99 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:17.130451 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-3360ed14-9fe9-401f-a3d2-e4fb18c43f7f tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f2e974ab5aa48dfb4b00f810dcedb63', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd31be8475a6a49d6bfe8132dd050c6c0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:24:17.148969 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-3360ed14-9fe9-401f-a3d2-e4fb18c43f7f tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "GET /compute/v2.1/servers/8e6eab25-f25e-46a2-9c72-9424c872fb84" status: 200 len: 1507 microversion: 2.1 time: 0.781197 Jul 16 21:24:17.151990 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1686/3380] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:24:16 2022] GET /compute/v2.1/servers/8e6eab25-f25e-46a2-9c72-9424c872fb84 => generated 1507 bytes in 785 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:24:17.443272 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-2bc70072-452e-433f-9d09-402cb251bd99 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcc9aa2a091c490599f9a9c41163e2eb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da56fc78f06142aba80ca4c6e3d60475', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:24:17.460823 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-2bc70072-452e-433f-9d09-402cb251bd99 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] 10.222.0.16 "GET /compute/v2.1/servers/867b2ab0-9301-414b-b045-85057d642971" status: 200 len: 1522 microversion: 2.1 time: 0.904403 Jul 16 21:24:17.482602 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1695/3381] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:24:16 2022] GET /compute/v2.1/servers/867b2ab0-9301-414b-b045-85057d642971 => generated 1522 bytes in 923 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:24:17.560260 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-1c5fbf21-bcc9-4fb6-861f-3ec13e9c35b9 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:24:17.560877 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-1c5fbf21-bcc9-4fb6-861f-3ec13e9c35b9 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: 083a6d48-6623-437b-8649-92a693c68ebf] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:24:17.581310 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1c5fbf21-bcc9-4fb6-861f-3ec13e9c35b9 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:17.588301 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1c5fbf21-bcc9-4fb6-861f-3ec13e9c35b9 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:17.589980 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1c5fbf21-bcc9-4fb6-861f-3ec13e9c35b9 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:17.590409 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1c5fbf21-bcc9-4fb6-861f-3ec13e9c35b9 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:18.084413 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-1c5fbf21-bcc9-4fb6-861f-3ec13e9c35b9 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '027dcc48e4fd4c36a5004096ad5e43f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dad823e240149edb30e592380249a13', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:24:18.093207 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-1c5fbf21-bcc9-4fb6-861f-3ec13e9c35b9 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "GET /compute/v2.1/servers/083a6d48-6623-437b-8649-92a693c68ebf" status: 200 len: 1320 microversion: 2.1 time: 0.537417 Jul 16 21:24:18.093797 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1687/3382] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:24:17 2022] GET /compute/v2.1/servers/083a6d48-6623-437b-8649-92a693c68ebf => generated 1320 bytes in 539 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:24:18.170334 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-bf01a0db-d8d1-4617-897e-d13ff5448470 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:24:18.170964 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-bf01a0db-d8d1-4617-897e-d13ff5448470 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 8e6eab25-f25e-46a2-9c72-9424c872fb84] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:24:18.178033 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-bf01a0db-d8d1-4617-897e-d13ff5448470 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:18.178493 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-bf01a0db-d8d1-4617-897e-d13ff5448470 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:18.188705 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-bf01a0db-d8d1-4617-897e-d13ff5448470 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:18.189833 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-bf01a0db-d8d1-4617-897e-d13ff5448470 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:18.464941 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell0:SELECT=2 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:24:18.491484 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-ef92804a-4d78-4781-840e-97e9ec7c1601 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:24:18.496257 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-ef92804a-4d78-4781-840e-97e9ec7c1601 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] [instance: 867b2ab0-9301-414b-b045-85057d642971] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:24:18.516639 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ef92804a-4d78-4781-840e-97e9ec7c1601 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:18.518036 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ef92804a-4d78-4781-840e-97e9ec7c1601 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:18.520120 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ef92804a-4d78-4781-840e-97e9ec7c1601 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:18.525014 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ef92804a-4d78-4781-840e-97e9ec7c1601 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:18.999103 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-bf01a0db-d8d1-4617-897e-d13ff5448470 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f2e974ab5aa48dfb4b00f810dcedb63', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd31be8475a6a49d6bfe8132dd050c6c0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:24:19.012940 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-bf01a0db-d8d1-4617-897e-d13ff5448470 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "GET /compute/v2.1/servers/8e6eab25-f25e-46a2-9c72-9424c872fb84" status: 200 len: 1507 microversion: 2.1 time: 0.848209 Jul 16 21:24:19.017144 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1696/3383] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:24:18 2022] GET /compute/v2.1/servers/8e6eab25-f25e-46a2-9c72-9424c872fb84 => generated 1507 bytes in 853 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:24:19.117198 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-94e3dff4-8817-4a6a-b9be-d0bfefae1f0e tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:24:19.121310 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-94e3dff4-8817-4a6a-b9be-d0bfefae1f0e tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: 083a6d48-6623-437b-8649-92a693c68ebf] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:24:19.135097 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell0:SELECT=12 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:24:19.143511 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-94e3dff4-8817-4a6a-b9be-d0bfefae1f0e tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:19.144168 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-94e3dff4-8817-4a6a-b9be-d0bfefae1f0e tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:19.145158 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-94e3dff4-8817-4a6a-b9be-d0bfefae1f0e tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:19.145781 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-94e3dff4-8817-4a6a-b9be-d0bfefae1f0e tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:20.287827 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-ef92804a-4d78-4781-840e-97e9ec7c1601 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcc9aa2a091c490599f9a9c41163e2eb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da56fc78f06142aba80ca4c6e3d60475', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:24:20.300576 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-ef92804a-4d78-4781-840e-97e9ec7c1601 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] 10.222.0.16 "GET /compute/v2.1/servers/867b2ab0-9301-414b-b045-85057d642971" status: 200 len: 1522 microversion: 2.1 time: 1.825603 Jul 16 21:24:20.302420 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1688/3384] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:24:18 2022] GET /compute/v2.1/servers/867b2ab0-9301-414b-b045-85057d642971 => generated 1522 bytes in 1827 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:24:20.310415 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-335616c1-9978-49c0-a410-df3ee23f51d8 req-b36a07b8-9657-42bd-b621-7d456916c6e4 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "083a6d48-6623-437b-8649-92a693c68ebf", "tag": "207483d6-106d-4a32-a4fd-eb046d0d1476"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:24:20.322466 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-335616c1-9978-49c0-a410-df3ee23f51d8 req-b36a07b8-9657-42bd-b621-7d456916c6e4 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:20.323594 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-335616c1-9978-49c0-a410-df3ee23f51d8 req-b36a07b8-9657-42bd-b621-7d456916c6e4 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:20.344138 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-335616c1-9978-49c0-a410-df3ee23f51d8 req-b36a07b8-9657-42bd-b621-7d456916c6e4 service nova] Creating event network-changed:207483d6-106d-4a32-a4fd-eb046d0d1476 for instance 083a6d48-6623-437b-8649-92a693c68ebf on nch2-master-1 Jul 16 21:24:20.362016 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-335616c1-9978-49c0-a410-df3ee23f51d8 req-b36a07b8-9657-42bd-b621-7d456916c6e4 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.052349 Jul 16 21:24:20.362016 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1689/3385] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:24:20 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:24:20.385905 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-10942275-d9aa-4e1a-84bf-307b08e6c9cc tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:24:20.386944 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-10942275-d9aa-4e1a-84bf-307b08e6c9cc tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 8e6eab25-f25e-46a2-9c72-9424c872fb84] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:24:20.401199 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-10942275-d9aa-4e1a-84bf-307b08e6c9cc tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:20.401777 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-10942275-d9aa-4e1a-84bf-307b08e6c9cc tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:20.403920 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-10942275-d9aa-4e1a-84bf-307b08e6c9cc tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:20.405068 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-10942275-d9aa-4e1a-84bf-307b08e6c9cc tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:20.632435 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-94e3dff4-8817-4a6a-b9be-d0bfefae1f0e tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '027dcc48e4fd4c36a5004096ad5e43f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dad823e240149edb30e592380249a13', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:24:20.662676 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-94e3dff4-8817-4a6a-b9be-d0bfefae1f0e tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "GET /compute/v2.1/servers/083a6d48-6623-437b-8649-92a693c68ebf" status: 200 len: 1320 microversion: 2.1 time: 1.556375 Jul 16 21:24:20.682200 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1697/3386] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:24:19 2022] GET /compute/v2.1/servers/083a6d48-6623-437b-8649-92a693c68ebf => generated 1320 bytes in 1573 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:24:21.098810 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-10942275-d9aa-4e1a-84bf-307b08e6c9cc tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f2e974ab5aa48dfb4b00f810dcedb63', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd31be8475a6a49d6bfe8132dd050c6c0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:24:21.132418 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-10942275-d9aa-4e1a-84bf-307b08e6c9cc tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "GET /compute/v2.1/servers/8e6eab25-f25e-46a2-9c72-9424c872fb84" status: 200 len: 1507 microversion: 2.1 time: 0.768400 Jul 16 21:24:21.133621 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1690/3387] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:24:20 2022] GET /compute/v2.1/servers/8e6eab25-f25e-46a2-9c72-9424c872fb84 => generated 1507 bytes in 772 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:24:21.321250 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-a5ea5d97-4340-426a-bbbf-f868dfdcb327 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:24:21.321893 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-a5ea5d97-4340-426a-bbbf-f868dfdcb327 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] [instance: 867b2ab0-9301-414b-b045-85057d642971] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:24:21.346748 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a5ea5d97-4340-426a-bbbf-f868dfdcb327 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:21.347289 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a5ea5d97-4340-426a-bbbf-f868dfdcb327 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:21.350110 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a5ea5d97-4340-426a-bbbf-f868dfdcb327 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:21.350988 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a5ea5d97-4340-426a-bbbf-f868dfdcb327 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:21.683409 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-39dd7e83-4239-40c0-9bd8-a5ff7270a3fe tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:24:21.684995 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-39dd7e83-4239-40c0-9bd8-a5ff7270a3fe tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: 083a6d48-6623-437b-8649-92a693c68ebf] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:24:21.703888 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell0:SELECT=26 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:24:21.707407 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-39dd7e83-4239-40c0-9bd8-a5ff7270a3fe tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:21.708177 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-39dd7e83-4239-40c0-9bd8-a5ff7270a3fe tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:21.709316 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-39dd7e83-4239-40c0-9bd8-a5ff7270a3fe tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:21.710210 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-39dd7e83-4239-40c0-9bd8-a5ff7270a3fe tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:22.143261 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-a5ea5d97-4340-426a-bbbf-f868dfdcb327 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcc9aa2a091c490599f9a9c41163e2eb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da56fc78f06142aba80ca4c6e3d60475', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:24:22.151097 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-a5ea5d97-4340-426a-bbbf-f868dfdcb327 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] 10.222.0.16 "GET /compute/v2.1/servers/867b2ab0-9301-414b-b045-85057d642971" status: 200 len: 1522 microversion: 2.1 time: 0.836537 Jul 16 21:24:22.152040 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1698/3388] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:24:21 2022] GET /compute/v2.1/servers/867b2ab0-9301-414b-b045-85057d642971 => generated 1522 bytes in 838 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:24:22.156890 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-1a131fa0-a3bb-43b7-acfe-806183cfd57b tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:24:22.157516 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-1a131fa0-a3bb-43b7-acfe-806183cfd57b tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 8e6eab25-f25e-46a2-9c72-9424c872fb84] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:24:22.163505 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1a131fa0-a3bb-43b7-acfe-806183cfd57b tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:22.163954 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1a131fa0-a3bb-43b7-acfe-806183cfd57b tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:22.164909 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1a131fa0-a3bb-43b7-acfe-806183cfd57b tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:22.165378 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1a131fa0-a3bb-43b7-acfe-806183cfd57b tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:22.190653 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-39dd7e83-4239-40c0-9bd8-a5ff7270a3fe tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '027dcc48e4fd4c36a5004096ad5e43f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dad823e240149edb30e592380249a13', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:24:22.204142 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-39dd7e83-4239-40c0-9bd8-a5ff7270a3fe tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "GET /compute/v2.1/servers/083a6d48-6623-437b-8649-92a693c68ebf" status: 200 len: 1320 microversion: 2.1 time: 0.526118 Jul 16 21:24:22.204726 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1691/3389] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:24:21 2022] GET /compute/v2.1/servers/083a6d48-6623-437b-8649-92a693c68ebf => generated 1320 bytes in 528 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:24:22.399405 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-1a131fa0-a3bb-43b7-acfe-806183cfd57b tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f2e974ab5aa48dfb4b00f810dcedb63', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd31be8475a6a49d6bfe8132dd050c6c0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:24:22.411557 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-3e8d97b4-8384-4aa9-8716-6a7152841b64 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeTestJSON-server-1363221205", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "57ded3f4-cdf1-4dc2-a2d8-ef8ca61bdef4"}], "security_groups": [{"name": "tempest-securitygroup--313422349"}], "key_name": "tempest-keypair-415855574", "user_data": "}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:24:22.419348 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_api:INSERT=4,nova_api:SELECT=96 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:24:22.432893 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-1a131fa0-a3bb-43b7-acfe-806183cfd57b tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "GET /compute/v2.1/servers/8e6eab25-f25e-46a2-9c72-9424c872fb84" status: 200 len: 1507 microversion: 2.1 time: 0.276783 Jul 16 21:24:22.446534 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1699/3390] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:24:22 2022] GET /compute/v2.1/servers/8e6eab25-f25e-46a2-9c72-9424c872fb84 => generated 1507 bytes in 294 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:24:22.636319 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.network.neutron [None req-3e8d97b4-8384-4aa9-8716-6a7152841b64 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] validate_networks() for [('57ded3f4-cdf1-4dc2-a2d8-ef8ca61bdef4', None, None, None, None, None)] {{(pid=138681) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 16 21:24:23.168896 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-ee0b8b94-6af0-4016-ba17-724c96d3d0e5 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:24:23.170187 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-ee0b8b94-6af0-4016-ba17-724c96d3d0e5 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] [instance: 867b2ab0-9301-414b-b045-85057d642971] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:24:23.176550 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ee0b8b94-6af0-4016-ba17-724c96d3d0e5 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:23.177259 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ee0b8b94-6af0-4016-ba17-724c96d3d0e5 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:23.178893 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ee0b8b94-6af0-4016-ba17-724c96d3d0e5 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:23.179372 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ee0b8b94-6af0-4016-ba17-724c96d3d0e5 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:23.197529 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=100 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:24:23.634175 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-ee0b8b94-6af0-4016-ba17-724c96d3d0e5 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcc9aa2a091c490599f9a9c41163e2eb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da56fc78f06142aba80ca4c6e3d60475', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:24:23.640322 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-3e8d97b4-8384-4aa9-8716-6a7152841b64 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Flavor limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 16 21:24:23.641433 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-3e8d97b4-8384-4aa9-8716-6a7152841b64 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Image limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 16 21:24:23.641924 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-3e8d97b4-8384-4aa9-8716-6a7152841b64 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Flavor pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 16 21:24:23.642344 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-3e8d97b4-8384-4aa9-8716-6a7152841b64 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Image pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 16 21:24:23.642764 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-3e8d97b4-8384-4aa9-8716-6a7152841b64 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 16 21:24:23.643263 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-ee0b8b94-6af0-4016-ba17-724c96d3d0e5 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] 10.222.0.16 "GET /compute/v2.1/servers/867b2ab0-9301-414b-b045-85057d642971" status: 200 len: 1522 microversion: 2.1 time: 0.478963 Jul 16 21:24:23.644350 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1700/3391] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:24:23 2022] GET /compute/v2.1/servers/867b2ab0-9301-414b-b045-85057d642971 => generated 1522 bytes in 481 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:24:23.649248 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-96266452-3d94-4273-9bd6-a6751a043ac0 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:24:23.650645 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-96266452-3d94-4273-9bd6-a6751a043ac0 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: 083a6d48-6623-437b-8649-92a693c68ebf] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:24:23.657642 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-96266452-3d94-4273-9bd6-a6751a043ac0 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:23.658172 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-96266452-3d94-4273-9bd6-a6751a043ac0 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:23.659161 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-96266452-3d94-4273-9bd6-a6751a043ac0 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:23.659813 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-96266452-3d94-4273-9bd6-a6751a043ac0 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:23.662031 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3e8d97b4-8384-4aa9-8716-6a7152841b64 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:23.662531 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3e8d97b4-8384-4aa9-8716-6a7152841b64 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:23.663357 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3e8d97b4-8384-4aa9-8716-6a7152841b64 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:23.663925 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3e8d97b4-8384-4aa9-8716-6a7152841b64 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:23.683499 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-3e8d97b4-8384-4aa9-8716-6a7152841b64 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Getting quotas for project 22ca92a86e6c40d59071e58ce4085568. Resources: {'ram', 'instances', 'cores'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:24:23.689589 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-3e8d97b4-8384-4aa9-8716-6a7152841b64 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Getting quotas for user bba77f6ef42547f1aa8480e27868bd4a and project 22ca92a86e6c40d59071e58ce4085568. Resources: {'ram', 'instances', 'cores'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:24:23.704732 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-3e8d97b4-8384-4aa9-8716-6a7152841b64 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Going to run 1 instances... {{(pid=138681) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 16 21:24:23.707301 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-3e8d97b4-8384-4aa9-8716-6a7152841b64 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] [instance: 2fad683a-f21f-4a9c-8323-e9380ef7f9ef] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138681) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 16 21:24:23.835790 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-3e8d97b4-8384-4aa9-8716-6a7152841b64 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 401 microversion: 2.1 time: 1.428892 Jul 16 21:24:23.844823 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1692/3392] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 21:24:22 2022] POST /compute/v2.1/servers => generated 401 bytes in 1439 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Jul 16 21:24:23.850910 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-e4b801eb-314b-427a-b49d-9f316c094f49 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:24:23.859494 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-e4b801eb-314b-427a-b49d-9f316c094f49 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 8e6eab25-f25e-46a2-9c72-9424c872fb84] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:24:23.865440 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e4b801eb-314b-427a-b49d-9f316c094f49 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:23.865887 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e4b801eb-314b-427a-b49d-9f316c094f49 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:23.873489 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e4b801eb-314b-427a-b49d-9f316c094f49 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:23.873952 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e4b801eb-314b-427a-b49d-9f316c094f49 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:24.416080 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-96266452-3d94-4273-9bd6-a6751a043ac0 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '027dcc48e4fd4c36a5004096ad5e43f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dad823e240149edb30e592380249a13', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:24:24.424592 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-e4b801eb-314b-427a-b49d-9f316c094f49 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f2e974ab5aa48dfb4b00f810dcedb63', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd31be8475a6a49d6bfe8132dd050c6c0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:24:24.425090 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-96266452-3d94-4273-9bd6-a6751a043ac0 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "GET /compute/v2.1/servers/083a6d48-6623-437b-8649-92a693c68ebf" status: 200 len: 1320 microversion: 2.1 time: 0.779334 Jul 16 21:24:24.426701 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1701/3393] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:24:23 2022] GET /compute/v2.1/servers/083a6d48-6623-437b-8649-92a693c68ebf => generated 1320 bytes in 781 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:24:24.439450 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-ea27a230-208b-4733-a037-a93a12dfb9eb tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:24:24.445414 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-ea27a230-208b-4733-a037-a93a12dfb9eb tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] [instance: 2fad683a-f21f-4a9c-8323-e9380ef7f9ef] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:24:24.452221 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-e4b801eb-314b-427a-b49d-9f316c094f49 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "GET /compute/v2.1/servers/8e6eab25-f25e-46a2-9c72-9424c872fb84" status: 200 len: 1507 microversion: 2.1 time: 0.605683 Jul 16 21:24:24.454723 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1693/3394] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:24:23 2022] GET /compute/v2.1/servers/8e6eab25-f25e-46a2-9c72-9424c872fb84 => generated 1507 bytes in 609 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:24:24.617450 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-ea27a230-208b-4733-a037-a93a12dfb9eb tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bba77f6ef42547f1aa8480e27868bd4a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '22ca92a86e6c40d59071e58ce4085568', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:24:24.633184 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-ea27a230-208b-4733-a037-a93a12dfb9eb tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] 10.222.0.16 "GET /compute/v2.1/servers/2fad683a-f21f-4a9c-8323-e9380ef7f9ef" status: 200 len: 1237 microversion: 2.1 time: 0.201387 Jul 16 21:24:24.633184 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1702/3395] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:24:24 2022] GET /compute/v2.1/servers/2fad683a-f21f-4a9c-8323-e9380ef7f9ef => generated 1237 bytes in 203 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:24:24.663662 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-0fb5c7f1-f9b5-4682-8e03-182a99d27fb5 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:24:24.663662 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-0fb5c7f1-f9b5-4682-8e03-182a99d27fb5 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] [instance: 867b2ab0-9301-414b-b045-85057d642971] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:24:24.686017 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0fb5c7f1-f9b5-4682-8e03-182a99d27fb5 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:24.686484 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0fb5c7f1-f9b5-4682-8e03-182a99d27fb5 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:24.688672 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0fb5c7f1-f9b5-4682-8e03-182a99d27fb5 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:24.689445 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0fb5c7f1-f9b5-4682-8e03-182a99d27fb5 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:24.945168 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-0fb5c7f1-f9b5-4682-8e03-182a99d27fb5 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcc9aa2a091c490599f9a9c41163e2eb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da56fc78f06142aba80ca4c6e3d60475', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:24:24.948378 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:24:24.956714 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-0fb5c7f1-f9b5-4682-8e03-182a99d27fb5 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] 10.222.0.16 "GET /compute/v2.1/servers/867b2ab0-9301-414b-b045-85057d642971" status: 200 len: 1522 microversion: 2.1 time: 0.299619 Jul 16 21:24:24.980006 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1694/3396] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:24:24 2022] GET /compute/v2.1/servers/867b2ab0-9301-414b-b045-85057d642971 => generated 1522 bytes in 324 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:24:25.442959 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-9d98fd91-1e4e-4102-a72a-e23eb7461983 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:24:25.444953 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-9d98fd91-1e4e-4102-a72a-e23eb7461983 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: 083a6d48-6623-437b-8649-92a693c68ebf] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:24:25.454244 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9d98fd91-1e4e-4102-a72a-e23eb7461983 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:25.455406 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9d98fd91-1e4e-4102-a72a-e23eb7461983 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:25.456495 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9d98fd91-1e4e-4102-a72a-e23eb7461983 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:25.457373 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9d98fd91-1e4e-4102-a72a-e23eb7461983 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:25.470437 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-12b65b1c-68f9-40b6-ba20-05eab198102d tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:24:25.471383 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-12b65b1c-68f9-40b6-ba20-05eab198102d tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 8e6eab25-f25e-46a2-9c72-9424c872fb84] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:24:25.493110 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-12b65b1c-68f9-40b6-ba20-05eab198102d tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:25.501538 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-12b65b1c-68f9-40b6-ba20-05eab198102d tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:25.504651 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-12b65b1c-68f9-40b6-ba20-05eab198102d tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:25.508035 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-12b65b1c-68f9-40b6-ba20-05eab198102d tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:25.873964 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-9d98fd91-1e4e-4102-a72a-e23eb7461983 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '027dcc48e4fd4c36a5004096ad5e43f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dad823e240149edb30e592380249a13', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:24:25.880025 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-12b65b1c-68f9-40b6-ba20-05eab198102d tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f2e974ab5aa48dfb4b00f810dcedb63', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd31be8475a6a49d6bfe8132dd050c6c0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:24:25.889495 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-12b65b1c-68f9-40b6-ba20-05eab198102d tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "GET /compute/v2.1/servers/8e6eab25-f25e-46a2-9c72-9424c872fb84" status: 200 len: 1507 microversion: 2.1 time: 0.423046 Jul 16 21:24:25.890129 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1695/3397] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:24:25 2022] GET /compute/v2.1/servers/8e6eab25-f25e-46a2-9c72-9424c872fb84 => generated 1507 bytes in 425 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:24:25.902793 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-9d98fd91-1e4e-4102-a72a-e23eb7461983 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "GET /compute/v2.1/servers/083a6d48-6623-437b-8649-92a693c68ebf" status: 200 len: 1320 microversion: 2.1 time: 0.459327 Jul 16 21:24:25.903497 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1703/3398] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:24:25 2022] GET /compute/v2.1/servers/083a6d48-6623-437b-8649-92a693c68ebf => generated 1320 bytes in 466 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:24:25.910568 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-d18a06f1-ca0b-42cf-8b08-eb9adb0f16ea tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:24:25.911285 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-d18a06f1-ca0b-42cf-8b08-eb9adb0f16ea tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] [instance: 2fad683a-f21f-4a9c-8323-e9380ef7f9ef] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:24:25.942307 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d18a06f1-ca0b-42cf-8b08-eb9adb0f16ea tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:25.942307 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d18a06f1-ca0b-42cf-8b08-eb9adb0f16ea tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:25.942307 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d18a06f1-ca0b-42cf-8b08-eb9adb0f16ea tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:25.942612 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d18a06f1-ca0b-42cf-8b08-eb9adb0f16ea tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:25.976190 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-5639f8ff-82ae-4153-8c57-b3fa0da708ca tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:24:25.976927 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-5639f8ff-82ae-4153-8c57-b3fa0da708ca tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] [instance: 867b2ab0-9301-414b-b045-85057d642971] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:24:25.996225 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5639f8ff-82ae-4153-8c57-b3fa0da708ca tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:25.997921 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5639f8ff-82ae-4153-8c57-b3fa0da708ca tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:25.999652 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5639f8ff-82ae-4153-8c57-b3fa0da708ca tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:26.000260 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5639f8ff-82ae-4153-8c57-b3fa0da708ca tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:26.093919 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-d18a06f1-ca0b-42cf-8b08-eb9adb0f16ea tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bba77f6ef42547f1aa8480e27868bd4a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '22ca92a86e6c40d59071e58ce4085568', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:24:26.118396 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-d18a06f1-ca0b-42cf-8b08-eb9adb0f16ea tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] 10.222.0.16 "GET /compute/v2.1/servers/2fad683a-f21f-4a9c-8323-e9380ef7f9ef" status: 200 len: 1233 microversion: 2.1 time: 0.225754 Jul 16 21:24:26.119036 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1696/3399] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:24:25 2022] GET /compute/v2.1/servers/2fad683a-f21f-4a9c-8323-e9380ef7f9ef => generated 1233 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:24:26.305799 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-5639f8ff-82ae-4153-8c57-b3fa0da708ca tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcc9aa2a091c490599f9a9c41163e2eb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da56fc78f06142aba80ca4c6e3d60475', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:24:26.339196 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-5639f8ff-82ae-4153-8c57-b3fa0da708ca tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] 10.222.0.16 "GET /compute/v2.1/servers/867b2ab0-9301-414b-b045-85057d642971" status: 200 len: 1522 microversion: 2.1 time: 0.353093 Jul 16 21:24:26.342981 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1704/3400] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:24:25 2022] GET /compute/v2.1/servers/867b2ab0-9301-414b-b045-85057d642971 => generated 1522 bytes in 372 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:24:26.529630 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-bd989a26-900a-45c4-a8ed-eac7b6e9acc1 req-825a519e-6d3f-4ab0-9a34-99a1109237a2 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "083a6d48-6623-437b-8649-92a693c68ebf", "name": "network-vif-plugged", "status": "completed", "tag": "207483d6-106d-4a32-a4fd-eb046d0d1476"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:24:26.541129 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-bd989a26-900a-45c4-a8ed-eac7b6e9acc1 req-825a519e-6d3f-4ab0-9a34-99a1109237a2 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:26.542199 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-bd989a26-900a-45c4-a8ed-eac7b6e9acc1 req-825a519e-6d3f-4ab0-9a34-99a1109237a2 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:26.554641 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-bd989a26-900a-45c4-a8ed-eac7b6e9acc1 req-825a519e-6d3f-4ab0-9a34-99a1109237a2 service nova] Creating event network-vif-plugged:207483d6-106d-4a32-a4fd-eb046d0d1476 for instance 083a6d48-6623-437b-8649-92a693c68ebf on nch2-master-1 Jul 16 21:24:26.560013 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-bd989a26-900a-45c4-a8ed-eac7b6e9acc1 req-825a519e-6d3f-4ab0-9a34-99a1109237a2 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.040001 Jul 16 21:24:26.561311 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1697/3401] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:24:26 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:24:26.907531 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-251ef6ac-114f-4246-87bf-0e4fb7c29717 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:24:26.908175 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-251ef6ac-114f-4246-87bf-0e4fb7c29717 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 8e6eab25-f25e-46a2-9c72-9424c872fb84] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:24:26.922635 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-ef56c9e4-b4e0-4754-8bec-e05919ea4232 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:24:26.923413 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-ef56c9e4-b4e0-4754-8bec-e05919ea4232 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: 083a6d48-6623-437b-8649-92a693c68ebf] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:24:26.935196 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-251ef6ac-114f-4246-87bf-0e4fb7c29717 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:26.935665 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-251ef6ac-114f-4246-87bf-0e4fb7c29717 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:26.936632 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-251ef6ac-114f-4246-87bf-0e4fb7c29717 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:26.937094 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-251ef6ac-114f-4246-87bf-0e4fb7c29717 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:26.943534 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ef56c9e4-b4e0-4754-8bec-e05919ea4232 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:26.944005 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ef56c9e4-b4e0-4754-8bec-e05919ea4232 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:26.956364 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ef56c9e4-b4e0-4754-8bec-e05919ea4232 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:26.957280 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ef56c9e4-b4e0-4754-8bec-e05919ea4232 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.012s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:27.283027 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-251ef6ac-114f-4246-87bf-0e4fb7c29717 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f2e974ab5aa48dfb4b00f810dcedb63', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd31be8475a6a49d6bfe8132dd050c6c0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:24:27.298815 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-251ef6ac-114f-4246-87bf-0e4fb7c29717 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "GET /compute/v2.1/servers/8e6eab25-f25e-46a2-9c72-9424c872fb84" status: 200 len: 1507 microversion: 2.1 time: 0.395594 Jul 16 21:24:27.301235 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1705/3402] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:24:26 2022] GET /compute/v2.1/servers/8e6eab25-f25e-46a2-9c72-9424c872fb84 => generated 1507 bytes in 399 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:24:27.311048 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-da602b50-d101-49a2-9582-5d882c906926 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:24:27.313806 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-da602b50-d101-49a2-9582-5d882c906926 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] [instance: 2fad683a-f21f-4a9c-8323-e9380ef7f9ef] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:24:27.327858 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-da602b50-d101-49a2-9582-5d882c906926 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:27.328361 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-da602b50-d101-49a2-9582-5d882c906926 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:27.331810 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-da602b50-d101-49a2-9582-5d882c906926 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:27.332120 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-da602b50-d101-49a2-9582-5d882c906926 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:27.361221 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-ef56c9e4-b4e0-4754-8bec-e05919ea4232 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '027dcc48e4fd4c36a5004096ad5e43f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dad823e240149edb30e592380249a13', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:24:27.372334 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-ef56c9e4-b4e0-4754-8bec-e05919ea4232 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "GET /compute/v2.1/servers/083a6d48-6623-437b-8649-92a693c68ebf" status: 200 len: 1320 microversion: 2.1 time: 0.455977 Jul 16 21:24:27.372928 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1698/3403] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:24:26 2022] GET /compute/v2.1/servers/083a6d48-6623-437b-8649-92a693c68ebf => generated 1320 bytes in 458 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:24:27.378434 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-7a6bcf18-a9dd-45a9-baf5-53f531fae49a tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:24:27.383132 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-7a6bcf18-a9dd-45a9-baf5-53f531fae49a tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] [instance: 867b2ab0-9301-414b-b045-85057d642971] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:24:27.391425 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7a6bcf18-a9dd-45a9-baf5-53f531fae49a tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:27.392071 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7a6bcf18-a9dd-45a9-baf5-53f531fae49a tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:27.393636 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7a6bcf18-a9dd-45a9-baf5-53f531fae49a tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:27.394379 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7a6bcf18-a9dd-45a9-baf5-53f531fae49a tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:27.532709 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-da602b50-d101-49a2-9582-5d882c906926 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bba77f6ef42547f1aa8480e27868bd4a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '22ca92a86e6c40d59071e58ce4085568', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:24:27.541619 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-da602b50-d101-49a2-9582-5d882c906926 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] 10.222.0.16 "GET /compute/v2.1/servers/2fad683a-f21f-4a9c-8323-e9380ef7f9ef" status: 200 len: 1307 microversion: 2.1 time: 0.236219 Jul 16 21:24:27.573249 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1706/3404] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:24:27 2022] GET /compute/v2.1/servers/2fad683a-f21f-4a9c-8323-e9380ef7f9ef => generated 1307 bytes in 265 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:24:27.704454 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-7a6bcf18-a9dd-45a9-baf5-53f531fae49a tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcc9aa2a091c490599f9a9c41163e2eb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da56fc78f06142aba80ca4c6e3d60475', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:24:27.715233 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-7a6bcf18-a9dd-45a9-baf5-53f531fae49a tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] 10.222.0.16 "GET /compute/v2.1/servers/867b2ab0-9301-414b-b045-85057d642971" status: 200 len: 1522 microversion: 2.1 time: 0.340131 Jul 16 21:24:27.717048 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1699/3405] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:24:27 2022] GET /compute/v2.1/servers/867b2ab0-9301-414b-b045-85057d642971 => generated 1522 bytes in 342 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:24:28.325705 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-81c9a3b0-6d28-4581-93cd-434684e1faee tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:24:28.326530 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-81c9a3b0-6d28-4581-93cd-434684e1faee tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 8e6eab25-f25e-46a2-9c72-9424c872fb84] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:24:28.341210 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-81c9a3b0-6d28-4581-93cd-434684e1faee tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:28.341724 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-81c9a3b0-6d28-4581-93cd-434684e1faee tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:28.343979 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-81c9a3b0-6d28-4581-93cd-434684e1faee tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:28.347118 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-81c9a3b0-6d28-4581-93cd-434684e1faee tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:28.390429 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-cff1cb4d-c0ac-4bb2-87b0-ee17c3f9c2b1 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:24:28.391332 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-cff1cb4d-c0ac-4bb2-87b0-ee17c3f9c2b1 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: 083a6d48-6623-437b-8649-92a693c68ebf] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:24:28.408669 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cff1cb4d-c0ac-4bb2-87b0-ee17c3f9c2b1 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:28.414918 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cff1cb4d-c0ac-4bb2-87b0-ee17c3f9c2b1 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:28.420506 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cff1cb4d-c0ac-4bb2-87b0-ee17c3f9c2b1 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:28.421092 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cff1cb4d-c0ac-4bb2-87b0-ee17c3f9c2b1 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:28.855088 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-81c9a3b0-6d28-4581-93cd-434684e1faee tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f2e974ab5aa48dfb4b00f810dcedb63', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd31be8475a6a49d6bfe8132dd050c6c0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:24:28.883045 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-81c9a3b0-6d28-4581-93cd-434684e1faee tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "GET /compute/v2.1/servers/8e6eab25-f25e-46a2-9c72-9424c872fb84" status: 200 len: 1507 microversion: 2.1 time: 0.565847 Jul 16 21:24:28.888571 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1707/3406] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:24:28 2022] GET /compute/v2.1/servers/8e6eab25-f25e-46a2-9c72-9424c872fb84 => generated 1507 bytes in 571 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:24:28.903226 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-95935091-13df-4501-a6aa-4df36885cc40 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:24:28.903920 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-95935091-13df-4501-a6aa-4df36885cc40 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] [instance: 2fad683a-f21f-4a9c-8323-e9380ef7f9ef] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:24:28.921243 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-95935091-13df-4501-a6aa-4df36885cc40 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:28.922511 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-95935091-13df-4501-a6aa-4df36885cc40 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:28.924830 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-95935091-13df-4501-a6aa-4df36885cc40 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:28.925291 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-95935091-13df-4501-a6aa-4df36885cc40 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:28.961153 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-cff1cb4d-c0ac-4bb2-87b0-ee17c3f9c2b1 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '027dcc48e4fd4c36a5004096ad5e43f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dad823e240149edb30e592380249a13', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:24:28.982866 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-cff1cb4d-c0ac-4bb2-87b0-ee17c3f9c2b1 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "GET /compute/v2.1/servers/083a6d48-6623-437b-8649-92a693c68ebf" status: 200 len: 1501 microversion: 2.1 time: 0.586333 Jul 16 21:24:28.982866 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1700/3407] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:24:28 2022] GET /compute/v2.1/servers/083a6d48-6623-437b-8649-92a693c68ebf => generated 1501 bytes in 588 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:24:29.026796 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-95a1f532-d799-453b-9268-c31cef3c94c5 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:24:29.026796 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-95a1f532-d799-453b-9268-c31cef3c94c5 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] [instance: 867b2ab0-9301-414b-b045-85057d642971] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:24:29.042369 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-95a1f532-d799-453b-9268-c31cef3c94c5 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:29.042843 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-95a1f532-d799-453b-9268-c31cef3c94c5 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:29.059422 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-95a1f532-d799-453b-9268-c31cef3c94c5 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:29.059890 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-95a1f532-d799-453b-9268-c31cef3c94c5 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:29.411416 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-95935091-13df-4501-a6aa-4df36885cc40 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bba77f6ef42547f1aa8480e27868bd4a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '22ca92a86e6c40d59071e58ce4085568', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:24:29.420808 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-95935091-13df-4501-a6aa-4df36885cc40 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] 10.222.0.16 "GET /compute/v2.1/servers/2fad683a-f21f-4a9c-8323-e9380ef7f9ef" status: 200 len: 1362 microversion: 2.1 time: 0.524996 Jul 16 21:24:29.421524 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1708/3408] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:24:28 2022] GET /compute/v2.1/servers/2fad683a-f21f-4a9c-8323-e9380ef7f9ef => generated 1362 bytes in 532 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:24:29.428247 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-074c2a3d-d945-469d-90bf-f47e21cd8bdb tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:24:29.429642 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-074c2a3d-d945-469d-90bf-f47e21cd8bdb tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: 083a6d48-6623-437b-8649-92a693c68ebf] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:24:29.438276 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-074c2a3d-d945-469d-90bf-f47e21cd8bdb tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:29.438874 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-074c2a3d-d945-469d-90bf-f47e21cd8bdb tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:29.439839 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-074c2a3d-d945-469d-90bf-f47e21cd8bdb tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:29.440766 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-074c2a3d-d945-469d-90bf-f47e21cd8bdb tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:29.444292 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-95a1f532-d799-453b-9268-c31cef3c94c5 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcc9aa2a091c490599f9a9c41163e2eb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da56fc78f06142aba80ca4c6e3d60475', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:24:29.455165 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-95a1f532-d799-453b-9268-c31cef3c94c5 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] 10.222.0.16 "GET /compute/v2.1/servers/867b2ab0-9301-414b-b045-85057d642971" status: 200 len: 1522 microversion: 2.1 time: 0.480838 Jul 16 21:24:29.456282 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1701/3409] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:24:28 2022] GET /compute/v2.1/servers/867b2ab0-9301-414b-b045-85057d642971 => generated 1522 bytes in 483 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:24:29.629181 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-f1931177-769a-4c92-8b71-2f2084a62359 tempest-FloatingIPsNegativeTestJSON-420908 tempest-FloatingIPsNegativeTestJSON-420908-project] Calling method '.version_select at 0x7fdd9dbfc940>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:24:29.737545 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-f1931177-769a-4c92-8b71-2f2084a62359 tempest-FloatingIPsNegativeTestJSON-420908 tempest-FloatingIPsNegativeTestJSON-420908-project] 10.222.0.16 "GET /compute/v2.1/os-floating-ips" status: 200 len: 20 microversion: 2.1 time: 0.279716 Jul 16 21:24:29.743464 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1702/3410] 10.222.0.16 () {58 vars in 1148 bytes} [Sat Jul 16 21:24:29 2022] GET /compute/v2.1/os-floating-ips => generated 20 bytes in 286 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Jul 16 21:24:29.753770 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-483824eb-bff7-4182-9f9c-c00c01f4769e tempest-FloatingIPsNegativeTestJSON-420908 tempest-FloatingIPsNegativeTestJSON-420908-project] Action: 'create', calling method: .version_select at 0x7fdd9d107dc0>, body: {"pool": "non_exist_pool"} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:24:29.833610 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-483824eb-bff7-4182-9f9c-c00c01f4769e tempest-FloatingIPsNegativeTestJSON-420908 tempest-FloatingIPsNegativeTestJSON-420908-project] HTTP exception thrown: Floating IP pool not found. Jul 16 21:24:29.840738 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-483824eb-bff7-4182-9f9c-c00c01f4769e tempest-FloatingIPsNegativeTestJSON-420908 tempest-FloatingIPsNegativeTestJSON-420908-project] Request method failure captured: Jul 16 21:24:29.840738 ncdv-master-1 devstack@n-api.service[138681]: request: POST /compute/v2.1/os-floating-ips HTTP/1.1 Jul 16 21:24:29.840738 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:24:29.840738 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:24:29.840738 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:24:29.840738 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 26 Jul 16 21:24:29.840738 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:24:29.840738 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:24:29.840738 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:24:29.840738 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:24:29.840738 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0yyMMQPxQBrmy2yBz1XhV6eSkgzpl5kXWcBdzqaM7DNP2YrzLiR2pW1N4LmgRnbi0UFwIHwNc2DAgF6L3Ajh_Xl_SexvWsx6kREPwhNZ0-CeYQVCxR4ubU8OoBJxX52N1bo7fz0bnqFB4-_1Fa7LJxzteZc-AFTqQaVrE56DvDWAqlo Jul 16 21:24:29.840738 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:24:29.840738 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:24:29.840738 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:24:29.840738 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:24:29.840738 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:24:29.840738 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:24:29.840738 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: b5b3367821b84df7bafc93dea7905e37 Jul 16 21:24:29.840738 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-FloatingIPsNegativeTestJSON-420908 Jul 16 21:24:29.840738 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:24:29.840738 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:24:29.840738 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/b5b3367821b84df7bafc93dea7905e37"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/b5b3367821b84df7bafc93dea7905e37"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_b5b3367821b84df7bafc93dea7905e37"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/b5b3367821b84df7bafc93dea7905e37"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:24:29.840738 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-FloatingIPsNegativeTestJSON-420908 Jul 16 21:24:29.840738 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: b5b3367821b84df7bafc93dea7905e37 Jul 16 21:24:29.840738 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-FloatingIPsNegativeTestJSON-420908 Jul 16 21:24:29.840738 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-FloatingIPsNegativeTestJSON-420908-project Jul 16 21:24:29.840738 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:24:29.840738 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:24:29.840738 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: ab1e26051847460f85bfedf8f9c0336b Jul 16 21:24:29.840738 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-FloatingIPsNegativeTestJSON-420908-project Jul 16 21:24:29.840738 ncdv-master-1 devstack@n-api.service[138681]: Jul 16 21:24:29.840738 ncdv-master-1 devstack@n-api.service[138681]: {"pool": "non_exist_pool"} Jul 16 21:24:29.840738 ncdv-master-1 devstack@n-api.service[138681]: method: Jul 16 21:24:29.840738 ncdv-master-1 devstack@n-api.service[138681]: exception: Floating IP pool not found. Jul 16 21:24:29.840738 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'body': {'pool': 'non_exist_pool'}} Jul 16 21:24:29.840738 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:24:29.840738 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:24:29.840738 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/floating_ips.py", line 127, in create Jul 16 21:24:29.840738 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi address = self.network_api.allocate_floating_ip(context, pool) Jul 16 21:24:29.840738 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 2975, in allocate_floating_ip Jul 16 21:24:29.840738 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi pool_id = self._get_floating_ip_pool_id_by_name_or_id(client, pool) Jul 16 21:24:29.840738 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 2965, in _get_floating_ip_pool_id_by_name_or_id Jul 16 21:24:29.840738 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.FloatingIpPoolNotFound() Jul 16 21:24:29.840738 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.FloatingIpPoolNotFound: Floating IP pool not found. Jul 16 21:24:29.840738 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:24:29.840738 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:24:29.840738 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:24:29.840738 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:24:29.840738 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:24:29.840738 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:24:29.840738 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:24:29.840738 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:24:29.840738 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Jul 16 21:24:29.840738 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Jul 16 21:24:29.840738 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:24:29.840738 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:24:29.840738 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/floating_ips.py", line 142, in create Jul 16 21:24:29.840738 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:24:29.840738 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Floating IP pool not found. Jul 16 21:24:29.840738 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:24:29.840738 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:24:29.840738 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:24:29.840738 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:24:29.840738 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:24:29.840738 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:24:29.840738 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:24:29.840738 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:24:29.845907 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Floating IP pool not found. Jul 16 21:24:29.845907 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:24:29.845907 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-483824eb-bff7-4182-9f9c-c00c01f4769e tempest-FloatingIPsNegativeTestJSON-420908 tempest-FloatingIPsNegativeTestJSON-420908-project] Returning 404 to user: Floating IP pool not found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:24:29.845907 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-483824eb-bff7-4182-9f9c-c00c01f4769e tempest-FloatingIPsNegativeTestJSON-420908 tempest-FloatingIPsNegativeTestJSON-420908-project] 10.222.0.16 "POST /compute/v2.1/os-floating-ips" status: 404 len: 73 microversion: 2.1 time: 0.095434 Jul 16 21:24:29.845907 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1703/3411] 10.222.0.16 () {60 vars in 1169 bytes} [Sat Jul 16 21:24:29 2022] POST /compute/v2.1/os-floating-ips => generated 73 bytes in 97 msecs (HTTP/1.1 404) 9 headers in 378 bytes (1 switches on core 0) Jul 16 21:24:29.847043 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-074c2a3d-d945-469d-90bf-f47e21cd8bdb tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '027dcc48e4fd4c36a5004096ad5e43f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dad823e240149edb30e592380249a13', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:24:29.858996 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-074c2a3d-d945-469d-90bf-f47e21cd8bdb tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "GET /compute/v2.1/servers/083a6d48-6623-437b-8649-92a693c68ebf" status: 200 len: 1501 microversion: 2.1 time: 0.435791 Jul 16 21:24:29.861385 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-d2e78d39-3cbc-4448-9881-520b1201f6a6 tempest-FloatingIPsNegativeTestJSON-420908 tempest-FloatingIPsNegativeTestJSON-420908-project] Calling method '.version_select at 0x7fdda2a58af0>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:24:29.864157 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1709/3412] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:24:29 2022] GET /compute/v2.1/servers/083a6d48-6623-437b-8649-92a693c68ebf => generated 1501 bytes in 442 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:24:29.883164 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-4f87b775-fa23-46b2-82bc-66c95940d806 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:24:29.885553 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-4f87b775-fa23-46b2-82bc-66c95940d806 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: 083a6d48-6623-437b-8649-92a693c68ebf] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:24:29.895635 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4f87b775-fa23-46b2-82bc-66c95940d806 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:29.896097 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4f87b775-fa23-46b2-82bc-66c95940d806 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:29.896889 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4f87b775-fa23-46b2-82bc-66c95940d806 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:29.897395 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4f87b775-fa23-46b2-82bc-66c95940d806 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:29.946324 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-4f87b775-fa23-46b2-82bc-66c95940d806 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: 083a6d48-6623-437b-8649-92a693c68ebf] Going to try to terminate instance {{(pid=138680) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 16 21:24:29.965299 ncdv-master-1 devstack@n-api.service[138681]: DEBUG neutronclient.v2_0.client [None req-d2e78d39-3cbc-4448-9881-520b1201f6a6 tempest-FloatingIPsNegativeTestJSON-420908 tempest-FloatingIPsNegativeTestJSON-420908-project] Error message: {"NeutronError": {"type": "FloatingIPNotFound", "message": "Floating IP 9aaeef6d-2e10-4b66-9f9c-c3fb753c9ff2 could not be found", "detail": ""}} {{(pid=138681) _handle_fault_response /usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py:262}} Jul 16 21:24:29.965299 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-d2e78d39-3cbc-4448-9881-520b1201f6a6 tempest-FloatingIPsNegativeTestJSON-420908 tempest-FloatingIPsNegativeTestJSON-420908-project] HTTP exception thrown: Floating IP not found for ID 9aaeef6d-2e10-4b66-9f9c-c3fb753c9ff2 Jul 16 21:24:29.973922 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-d2e78d39-3cbc-4448-9881-520b1201f6a6 tempest-FloatingIPsNegativeTestJSON-420908 tempest-FloatingIPsNegativeTestJSON-420908-project] Request method failure captured: Jul 16 21:24:29.973922 ncdv-master-1 devstack@n-api.service[138681]: request: DELETE /compute/v2.1/os-floating-ips/9aaeef6d-2e10-4b66-9f9c-c3fb753c9ff2 HTTP/1.1 Jul 16 21:24:29.973922 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:24:29.973922 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:24:29.973922 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:24:29.973922 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 21:24:29.973922 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:24:29.973922 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:24:29.973922 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:24:29.973922 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:24:29.973922 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0yyMMQPxQBrmy2yBz1XhV6eSkgzpl5kXWcBdzqaM7DNP2YrzLiR2pW1N4LmgRnbi0UFwIHwNc2DAgF6L3Ajh_Xl_SexvWsx6kREPwhNZ0-CeYQVCxR4ubU8OoBJxX52N1bo7fz0bnqFB4-_1Fa7LJxzteZc-AFTqQaVrE56DvDWAqlo Jul 16 21:24:29.973922 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:24:29.973922 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:24:29.973922 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:24:29.973922 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:24:29.973922 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:24:29.973922 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:24:29.973922 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: b5b3367821b84df7bafc93dea7905e37 Jul 16 21:24:29.973922 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-FloatingIPsNegativeTestJSON-420908 Jul 16 21:24:29.973922 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:24:29.973922 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:24:29.973922 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/b5b3367821b84df7bafc93dea7905e37"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/b5b3367821b84df7bafc93dea7905e37"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_b5b3367821b84df7bafc93dea7905e37"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/b5b3367821b84df7bafc93dea7905e37"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:24:29.973922 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-FloatingIPsNegativeTestJSON-420908 Jul 16 21:24:29.973922 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: b5b3367821b84df7bafc93dea7905e37 Jul 16 21:24:29.973922 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-FloatingIPsNegativeTestJSON-420908 Jul 16 21:24:29.973922 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-FloatingIPsNegativeTestJSON-420908-project Jul 16 21:24:29.973922 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:24:29.973922 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:24:29.973922 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: ab1e26051847460f85bfedf8f9c0336b Jul 16 21:24:29.973922 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-FloatingIPsNegativeTestJSON-420908-project Jul 16 21:24:29.973922 ncdv-master-1 devstack@n-api.service[138681]: method: Jul 16 21:24:29.973922 ncdv-master-1 devstack@n-api.service[138681]: exception: Floating IP not found for ID 9aaeef6d-2e10-4b66-9f9c-c3fb753c9ff2 Jul 16 21:24:29.973922 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': '9aaeef6d-2e10-4b66-9f9c-c3fb753c9ff2'} Jul 16 21:24:29.973922 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:24:29.973922 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:24:29.973922 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 2827, in get_floating_ip Jul 16 21:24:29.973922 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi fip = client.show_floatingip(id)['floatingip'] Jul 16 21:24:29.973922 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 197, in wrapper Jul 16 21:24:29.973922 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Jul 16 21:24:29.973922 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 1006, in show_floatingip Jul 16 21:24:29.973922 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return self.get(self.floatingip_path % (floatingip), params=_params) Jul 16 21:24:29.973922 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 197, in wrapper Jul 16 21:24:29.973922 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Jul 16 21:24:29.973922 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 356, in get Jul 16 21:24:29.973922 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return self.retry_request("GET", action, body=body, Jul 16 21:24:29.973922 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 197, in wrapper Jul 16 21:24:29.973922 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Jul 16 21:24:29.973922 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 333, in retry_request Jul 16 21:24:29.973922 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return self.do_request(method, action, body=body, Jul 16 21:24:29.973922 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 197, in wrapper Jul 16 21:24:29.973922 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Jul 16 21:24:29.973922 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 297, in do_request Jul 16 21:24:29.973922 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi self._handle_fault_response(status_code, replybody, resp) Jul 16 21:24:29.973922 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 197, in wrapper Jul 16 21:24:29.973922 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Jul 16 21:24:29.973922 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 272, in _handle_fault_response Jul 16 21:24:29.973922 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi exception_handler_v20(status_code, error_body) Jul 16 21:24:29.973922 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 90, in exception_handler_v20 Jul 16 21:24:29.973922 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise client_exc(message=error_message, Jul 16 21:24:29.973922 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi neutronclient.common.exceptions.NotFound: Floating IP 9aaeef6d-2e10-4b66-9f9c-c3fb753c9ff2 could not be found Jul 16 21:24:29.973922 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-8b5223bc-bcb8-41c4-91ed-31dd6a737733'] Jul 16 21:24:29.973922 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:24:29.979986 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:24:29.979986 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:24:29.979986 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:24:29.979986 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/floating_ips.py", line 158, in delete Jul 16 21:24:29.979986 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi floating_ip = self.network_api.get_floating_ip(context, id) Jul 16 21:24:29.979986 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 2830, in get_floating_ip Jul 16 21:24:29.979986 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.FloatingIpNotFound(id=id) Jul 16 21:24:29.979986 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.FloatingIpNotFound: Floating IP not found for ID 9aaeef6d-2e10-4b66-9f9c-c3fb753c9ff2. Jul 16 21:24:29.979986 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:24:29.979986 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:24:29.979986 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:24:29.979986 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:24:29.979986 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:24:29.979986 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:24:29.979986 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:24:29.979986 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:24:29.979986 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Jul 16 21:24:29.979986 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Jul 16 21:24:29.979986 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:24:29.979986 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:24:29.979986 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/floating_ips.py", line 161, in delete Jul 16 21:24:29.979986 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=msg) Jul 16 21:24:29.979986 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Floating IP not found for ID 9aaeef6d-2e10-4b66-9f9c-c3fb753c9ff2 Jul 16 21:24:29.979986 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:24:29.979986 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:24:29.979986 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:24:29.979986 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:24:29.979986 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:24:29.979986 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:24:29.979986 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:24:29.979986 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:24:29.979986 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Floating IP not found for ID 9aaeef6d-2e10-4b66-9f9c-c3fb753c9ff2 Jul 16 21:24:29.979986 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:24:29.989219 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-d2e78d39-3cbc-4448-9881-520b1201f6a6 tempest-FloatingIPsNegativeTestJSON-420908 tempest-FloatingIPsNegativeTestJSON-420908-project] Returning 404 to user: Floating IP not found for ID 9aaeef6d-2e10-4b66-9f9c-c3fb753c9ff2 {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:24:29.989219 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-d2e78d39-3cbc-4448-9881-520b1201f6a6 tempest-FloatingIPsNegativeTestJSON-420908 tempest-FloatingIPsNegativeTestJSON-420908-project] 10.222.0.16 "DELETE /compute/v2.1/os-floating-ips/9aaeef6d-2e10-4b66-9f9c-c3fb753c9ff2" status: 404 len: 111 microversion: 2.1 time: 0.134313 Jul 16 21:24:29.989219 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1704/3413] 10.222.0.16 () {58 vars in 1262 bytes} [Sat Jul 16 21:24:29 2022] DELETE /compute/v2.1/os-floating-ips/9aaeef6d-2e10-4b66-9f9c-c3fb753c9ff2 => generated 111 bytes in 135 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:24:29.996745 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-753ce5b2-7ec2-4423-b119-7bc03cc78a42 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:24:29.997787 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-753ce5b2-7ec2-4423-b119-7bc03cc78a42 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 8e6eab25-f25e-46a2-9c72-9424c872fb84] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:24:30.018557 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-753ce5b2-7ec2-4423-b119-7bc03cc78a42 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:30.018967 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-753ce5b2-7ec2-4423-b119-7bc03cc78a42 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:30.019785 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-753ce5b2-7ec2-4423-b119-7bc03cc78a42 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:30.020767 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-753ce5b2-7ec2-4423-b119-7bc03cc78a42 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:30.370730 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-4f87b775-fa23-46b2-82bc-66c95940d806 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "DELETE /compute/v2.1/servers/083a6d48-6623-437b-8649-92a693c68ebf" status: 204 len: 0 microversion: 2.1 time: 0.480607 Jul 16 21:24:30.380913 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1710/3414] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:24:29 2022] DELETE /compute/v2.1/servers/083a6d48-6623-437b-8649-92a693c68ebf => generated 0 bytes in 501 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 16 21:24:30.414218 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-e7ffb0dd-4204-4f9b-9e3e-c9d341fa78de req-bda0d361-516e-419b-96b0-8b5620458caa service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "2fad683a-f21f-4a9c-8323-e9380ef7f9ef", "tag": "aff74b78-59ed-4a31-9d5b-3c3e6aa5d30c"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:24:30.457790 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-e7ffb0dd-4204-4f9b-9e3e-c9d341fa78de req-bda0d361-516e-419b-96b0-8b5620458caa service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:30.459392 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-e7ffb0dd-4204-4f9b-9e3e-c9d341fa78de req-bda0d361-516e-419b-96b0-8b5620458caa service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:30.493376 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-e7ffb0dd-4204-4f9b-9e3e-c9d341fa78de req-bda0d361-516e-419b-96b0-8b5620458caa service nova] Creating event network-changed:aff74b78-59ed-4a31-9d5b-3c3e6aa5d30c for instance 2fad683a-f21f-4a9c-8323-e9380ef7f9ef on nch1-master-1 Jul 16 21:24:30.508354 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-e7ffb0dd-4204-4f9b-9e3e-c9d341fa78de req-bda0d361-516e-419b-96b0-8b5620458caa service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.125957 Jul 16 21:24:30.511409 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1711/3415] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:24:30 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 132 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:24:30.518073 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-33135f81-8cdc-4006-8b47-f0df2a23a87f tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:24:30.519057 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-33135f81-8cdc-4006-8b47-f0df2a23a87f tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: 083a6d48-6623-437b-8649-92a693c68ebf] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:24:30.527525 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-33135f81-8cdc-4006-8b47-f0df2a23a87f tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:30.527656 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-33135f81-8cdc-4006-8b47-f0df2a23a87f tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:30.528660 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-33135f81-8cdc-4006-8b47-f0df2a23a87f tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:30.529111 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-33135f81-8cdc-4006-8b47-f0df2a23a87f tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:30.773764 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-753ce5b2-7ec2-4423-b119-7bc03cc78a42 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f2e974ab5aa48dfb4b00f810dcedb63', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd31be8475a6a49d6bfe8132dd050c6c0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:24:30.787689 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-753ce5b2-7ec2-4423-b119-7bc03cc78a42 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "GET /compute/v2.1/servers/8e6eab25-f25e-46a2-9c72-9424c872fb84" status: 200 len: 1507 microversion: 2.1 time: 0.799856 Jul 16 21:24:30.798897 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1705/3416] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:24:29 2022] GET /compute/v2.1/servers/8e6eab25-f25e-46a2-9c72-9424c872fb84 => generated 1507 bytes in 804 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:24:30.799025 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-33135f81-8cdc-4006-8b47-f0df2a23a87f tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '027dcc48e4fd4c36a5004096ad5e43f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dad823e240149edb30e592380249a13', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:24:30.805408 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-1f6287d1-9ae8-4feb-bf2e-18500f03995b tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:24:30.806373 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-1f6287d1-9ae8-4feb-bf2e-18500f03995b tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] [instance: 2fad683a-f21f-4a9c-8323-e9380ef7f9ef] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:24:30.815891 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1f6287d1-9ae8-4feb-bf2e-18500f03995b tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:30.817008 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1f6287d1-9ae8-4feb-bf2e-18500f03995b tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:30.818891 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1f6287d1-9ae8-4feb-bf2e-18500f03995b tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:30.818891 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1f6287d1-9ae8-4feb-bf2e-18500f03995b tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:30.831688 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-33135f81-8cdc-4006-8b47-f0df2a23a87f tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "GET /compute/v2.1/servers/083a6d48-6623-437b-8649-92a693c68ebf" status: 200 len: 1507 microversion: 2.1 time: 0.312358 Jul 16 21:24:30.842904 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1712/3417] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:24:30 2022] GET /compute/v2.1/servers/083a6d48-6623-437b-8649-92a693c68ebf => generated 1507 bytes in 330 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:24:30.861020 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-ecf6249c-0778-4047-a5c2-d66a2bacc71d tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:24:30.862928 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-ecf6249c-0778-4047-a5c2-d66a2bacc71d tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] [instance: 867b2ab0-9301-414b-b045-85057d642971] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:24:30.899341 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ecf6249c-0778-4047-a5c2-d66a2bacc71d tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:30.899813 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ecf6249c-0778-4047-a5c2-d66a2bacc71d tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:30.905270 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ecf6249c-0778-4047-a5c2-d66a2bacc71d tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:30.907900 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ecf6249c-0778-4047-a5c2-d66a2bacc71d tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:31.931651 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-ecf6249c-0778-4047-a5c2-d66a2bacc71d tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcc9aa2a091c490599f9a9c41163e2eb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da56fc78f06142aba80ca4c6e3d60475', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:24:31.946159 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-ecf6249c-0778-4047-a5c2-d66a2bacc71d tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] 10.222.0.16 "GET /compute/v2.1/servers/867b2ab0-9301-414b-b045-85057d642971" status: 200 len: 1522 microversion: 2.1 time: 1.101031 Jul 16 21:24:31.947613 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1713/3418] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:24:30 2022] GET /compute/v2.1/servers/867b2ab0-9301-414b-b045-85057d642971 => generated 1522 bytes in 1104 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:24:31.966818 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-93a879d6-0fa9-46a1-8c68-3ebcb9171ab0 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:24:31.980509 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-93a879d6-0fa9-46a1-8c68-3ebcb9171ab0 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 8e6eab25-f25e-46a2-9c72-9424c872fb84] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:24:31.988500 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-93a879d6-0fa9-46a1-8c68-3ebcb9171ab0 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:31.988968 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-93a879d6-0fa9-46a1-8c68-3ebcb9171ab0 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:32.001129 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-93a879d6-0fa9-46a1-8c68-3ebcb9171ab0 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:32.001938 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-93a879d6-0fa9-46a1-8c68-3ebcb9171ab0 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:32.069442 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-1f6287d1-9ae8-4feb-bf2e-18500f03995b tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bba77f6ef42547f1aa8480e27868bd4a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '22ca92a86e6c40d59071e58ce4085568', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:24:32.082606 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-1f6287d1-9ae8-4feb-bf2e-18500f03995b tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] 10.222.0.16 "GET /compute/v2.1/servers/2fad683a-f21f-4a9c-8323-e9380ef7f9ef" status: 200 len: 1362 microversion: 2.1 time: 1.286214 Jul 16 21:24:32.086678 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1706/3419] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:24:30 2022] GET /compute/v2.1/servers/2fad683a-f21f-4a9c-8323-e9380ef7f9ef => generated 1362 bytes in 1294 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:24:32.097358 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-62fac77a-9173-402e-bd6a-2c87cf897af6 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:24:32.100086 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-62fac77a-9173-402e-bd6a-2c87cf897af6 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: 083a6d48-6623-437b-8649-92a693c68ebf] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:24:32.114231 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-62fac77a-9173-402e-bd6a-2c87cf897af6 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:32.115091 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-62fac77a-9173-402e-bd6a-2c87cf897af6 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:32.123070 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-62fac77a-9173-402e-bd6a-2c87cf897af6 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:32.124370 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-62fac77a-9173-402e-bd6a-2c87cf897af6 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:32.545360 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-93a879d6-0fa9-46a1-8c68-3ebcb9171ab0 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f2e974ab5aa48dfb4b00f810dcedb63', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd31be8475a6a49d6bfe8132dd050c6c0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:24:32.555109 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-93a879d6-0fa9-46a1-8c68-3ebcb9171ab0 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "GET /compute/v2.1/servers/8e6eab25-f25e-46a2-9c72-9424c872fb84" status: 200 len: 1528 microversion: 2.1 time: 0.595027 Jul 16 21:24:32.564000 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1714/3420] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:24:31 2022] GET /compute/v2.1/servers/8e6eab25-f25e-46a2-9c72-9424c872fb84 => generated 1528 bytes in 616 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:24:32.566249 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-62fac77a-9173-402e-bd6a-2c87cf897af6 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '027dcc48e4fd4c36a5004096ad5e43f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dad823e240149edb30e592380249a13', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:24:32.581681 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-62fac77a-9173-402e-bd6a-2c87cf897af6 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "GET /compute/v2.1/servers/083a6d48-6623-437b-8649-92a693c68ebf" status: 200 len: 1507 microversion: 2.1 time: 0.493434 Jul 16 21:24:32.582390 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1707/3421] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:24:32 2022] GET /compute/v2.1/servers/083a6d48-6623-437b-8649-92a693c68ebf => generated 1507 bytes in 495 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:24:32.965141 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-d1490f5f-a6c4-4aca-8790-a9c36ffd8e98 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:24:32.966380 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-d1490f5f-a6c4-4aca-8790-a9c36ffd8e98 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] [instance: 867b2ab0-9301-414b-b045-85057d642971] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:24:32.986469 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d1490f5f-a6c4-4aca-8790-a9c36ffd8e98 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:32.986469 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d1490f5f-a6c4-4aca-8790-a9c36ffd8e98 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:32.986469 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d1490f5f-a6c4-4aca-8790-a9c36ffd8e98 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:32.986469 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d1490f5f-a6c4-4aca-8790-a9c36ffd8e98 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:33.102434 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-280cffe9-8dbf-4e0e-bc2e-ff89b65feb7a tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:24:33.103189 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-280cffe9-8dbf-4e0e-bc2e-ff89b65feb7a tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] [instance: 2fad683a-f21f-4a9c-8323-e9380ef7f9ef] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:24:33.123355 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-280cffe9-8dbf-4e0e-bc2e-ff89b65feb7a tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:33.128738 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-280cffe9-8dbf-4e0e-bc2e-ff89b65feb7a tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:33.132063 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-280cffe9-8dbf-4e0e-bc2e-ff89b65feb7a tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:33.132063 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-280cffe9-8dbf-4e0e-bc2e-ff89b65feb7a tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:33.509664 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-d1490f5f-a6c4-4aca-8790-a9c36ffd8e98 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcc9aa2a091c490599f9a9c41163e2eb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da56fc78f06142aba80ca4c6e3d60475', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:24:33.517561 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-d1490f5f-a6c4-4aca-8790-a9c36ffd8e98 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] 10.222.0.16 "GET /compute/v2.1/servers/867b2ab0-9301-414b-b045-85057d642971" status: 200 len: 1522 microversion: 2.1 time: 0.556756 Jul 16 21:24:33.520817 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1715/3422] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:24:32 2022] GET /compute/v2.1/servers/867b2ab0-9301-414b-b045-85057d642971 => generated 1522 bytes in 561 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:24:33.534838 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-94ffb889-4a20-4524-a4f2-2da3d1255fe5 req-36fe530d-79be-41c7-818f-3ecb503737f4 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "083a6d48-6623-437b-8649-92a693c68ebf", "name": "network-vif-deleted", "tag": "207483d6-106d-4a32-a4fd-eb046d0d1476"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:24:33.574715 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-94ffb889-4a20-4524-a4f2-2da3d1255fe5 req-36fe530d-79be-41c7-818f-3ecb503737f4 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:33.581858 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-94ffb889-4a20-4524-a4f2-2da3d1255fe5 req-36fe530d-79be-41c7-818f-3ecb503737f4 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:33.621314 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-94ffb889-4a20-4524-a4f2-2da3d1255fe5 req-36fe530d-79be-41c7-818f-3ecb503737f4 service nova] Creating event network-vif-deleted:207483d6-106d-4a32-a4fd-eb046d0d1476 for instance 083a6d48-6623-437b-8649-92a693c68ebf on nch2-master-1 Jul 16 21:24:33.639295 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-280cffe9-8dbf-4e0e-bc2e-ff89b65feb7a tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bba77f6ef42547f1aa8480e27868bd4a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '22ca92a86e6c40d59071e58ce4085568', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:24:33.648859 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-94ffb889-4a20-4524-a4f2-2da3d1255fe5 req-36fe530d-79be-41c7-818f-3ecb503737f4 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.126482 Jul 16 21:24:33.649798 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1716/3423] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:24:33 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 128 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:24:33.658952 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-280cffe9-8dbf-4e0e-bc2e-ff89b65feb7a tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] 10.222.0.16 "GET /compute/v2.1/servers/2fad683a-f21f-4a9c-8323-e9380ef7f9ef" status: 200 len: 1362 microversion: 2.1 time: 0.560495 Jul 16 21:24:33.660331 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1708/3424] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:24:33 2022] GET /compute/v2.1/servers/2fad683a-f21f-4a9c-8323-e9380ef7f9ef => generated 1362 bytes in 562 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:24:33.665725 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-9ddd0cb8-a7cc-4135-b668-ad838986c796 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:24:33.666801 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-9ddd0cb8-a7cc-4135-b668-ad838986c796 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: 083a6d48-6623-437b-8649-92a693c68ebf] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:24:33.670580 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-8471f132-6963-4d8b-85fe-1bc252cad360 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:24:33.674145 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-8471f132-6963-4d8b-85fe-1bc252cad360 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 8e6eab25-f25e-46a2-9c72-9424c872fb84] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:24:33.675110 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell0:SELECT=2 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:24:33.683514 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9ddd0cb8-a7cc-4135-b668-ad838986c796 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:33.683514 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9ddd0cb8-a7cc-4135-b668-ad838986c796 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:33.683514 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9ddd0cb8-a7cc-4135-b668-ad838986c796 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:33.683514 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9ddd0cb8-a7cc-4135-b668-ad838986c796 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:33.698522 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8471f132-6963-4d8b-85fe-1bc252cad360 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:33.698522 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8471f132-6963-4d8b-85fe-1bc252cad360 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:33.699388 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8471f132-6963-4d8b-85fe-1bc252cad360 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:33.699881 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8471f132-6963-4d8b-85fe-1bc252cad360 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:33.725927 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell0:SELECT=6 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:24:33.956534 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-9ddd0cb8-a7cc-4135-b668-ad838986c796 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '027dcc48e4fd4c36a5004096ad5e43f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dad823e240149edb30e592380249a13', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:24:33.983995 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-9ddd0cb8-a7cc-4135-b668-ad838986c796 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "GET /compute/v2.1/servers/083a6d48-6623-437b-8649-92a693c68ebf" status: 200 len: 1305 microversion: 2.1 time: 0.315724 Jul 16 21:24:33.987948 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1709/3425] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:24:33 2022] GET /compute/v2.1/servers/083a6d48-6623-437b-8649-92a693c68ebf => generated 1305 bytes in 326 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:24:34.085090 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-8471f132-6963-4d8b-85fe-1bc252cad360 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f2e974ab5aa48dfb4b00f810dcedb63', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd31be8475a6a49d6bfe8132dd050c6c0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:24:34.090464 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:24:34.100352 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-8471f132-6963-4d8b-85fe-1bc252cad360 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "GET /compute/v2.1/servers/8e6eab25-f25e-46a2-9c72-9424c872fb84" status: 200 len: 1528 microversion: 2.1 time: 0.446787 Jul 16 21:24:34.111428 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1717/3426] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:24:33 2022] GET /compute/v2.1/servers/8e6eab25-f25e-46a2-9c72-9424c872fb84 => generated 1528 bytes in 451 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:24:34.536836 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-17237c10-e798-4e91-a1ee-282674294883 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:24:34.538088 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-17237c10-e798-4e91-a1ee-282674294883 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] [instance: 867b2ab0-9301-414b-b045-85057d642971] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:24:34.554710 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-17237c10-e798-4e91-a1ee-282674294883 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:34.555451 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-17237c10-e798-4e91-a1ee-282674294883 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:34.560405 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-17237c10-e798-4e91-a1ee-282674294883 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:34.561121 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-17237c10-e798-4e91-a1ee-282674294883 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:34.677784 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-62be2e88-7e0f-4f1c-a003-d50cf8abc5d5 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:24:34.678626 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-62be2e88-7e0f-4f1c-a003-d50cf8abc5d5 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] [instance: 2fad683a-f21f-4a9c-8323-e9380ef7f9ef] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:24:34.683112 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell0:SELECT=2 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:24:34.700183 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-62be2e88-7e0f-4f1c-a003-d50cf8abc5d5 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:34.702752 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-62be2e88-7e0f-4f1c-a003-d50cf8abc5d5 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:34.707382 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-62be2e88-7e0f-4f1c-a003-d50cf8abc5d5 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:34.708573 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-62be2e88-7e0f-4f1c-a003-d50cf8abc5d5 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:34.878139 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-17237c10-e798-4e91-a1ee-282674294883 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcc9aa2a091c490599f9a9c41163e2eb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da56fc78f06142aba80ca4c6e3d60475', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:24:34.887692 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-17237c10-e798-4e91-a1ee-282674294883 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] 10.222.0.16 "GET /compute/v2.1/servers/867b2ab0-9301-414b-b045-85057d642971" status: 200 len: 1522 microversion: 2.1 time: 0.355572 Jul 16 21:24:34.901614 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1710/3427] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:24:34 2022] GET /compute/v2.1/servers/867b2ab0-9301-414b-b045-85057d642971 => generated 1522 bytes in 370 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:24:34.990636 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-62be2e88-7e0f-4f1c-a003-d50cf8abc5d5 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bba77f6ef42547f1aa8480e27868bd4a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '22ca92a86e6c40d59071e58ce4085568', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:24:35.011446 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-62be2e88-7e0f-4f1c-a003-d50cf8abc5d5 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] 10.222.0.16 "GET /compute/v2.1/servers/2fad683a-f21f-4a9c-8323-e9380ef7f9ef" status: 200 len: 1362 microversion: 2.1 time: 0.327984 Jul 16 21:24:35.011446 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1718/3428] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:24:34 2022] GET /compute/v2.1/servers/2fad683a-f21f-4a9c-8323-e9380ef7f9ef => generated 1362 bytes in 329 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:24:35.026840 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-59d6cdf7-2bcc-4053-a67c-364a344091d0 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:24:35.027426 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-59d6cdf7-2bcc-4053-a67c-364a344091d0 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: 083a6d48-6623-437b-8649-92a693c68ebf] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:24:35.036313 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-59d6cdf7-2bcc-4053-a67c-364a344091d0 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:35.036839 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-59d6cdf7-2bcc-4053-a67c-364a344091d0 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:35.037959 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-59d6cdf7-2bcc-4053-a67c-364a344091d0 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:35.038576 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-59d6cdf7-2bcc-4053-a67c-364a344091d0 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:35.055602 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-59d6cdf7-2bcc-4053-a67c-364a344091d0 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] HTTP exception thrown: Instance 083a6d48-6623-437b-8649-92a693c68ebf could not be found. Jul 16 21:24:35.058381 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-59d6cdf7-2bcc-4053-a67c-364a344091d0 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Request method failure captured: Jul 16 21:24:35.058381 ncdv-master-1 devstack@n-api.service[138681]: request: GET /compute/v2.1/servers/083a6d48-6623-437b-8649-92a693c68ebf HTTP/1.1 Jul 16 21:24:35.058381 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:24:35.058381 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:24:35.058381 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:24:35.058381 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 21:24:35.058381 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:24:35.058381 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:24:35.058381 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:24:35.058381 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:24:35.058381 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0yxS9NCWgI1-O55Z_AF16JzmczB_p-s4P30aJMz-9zu9VxcHuIV00tsfB9hOBtHTZ_sI8GPeOOvUrqYZLgOBzCbOMr4ZO506_XIRjMCtHQ0umzvcB9V9iFSNFoiIe_b2jKm-OyHj1IB8YVBCAcwDKANlsCw5UaAoIABK4dLLFjaVpEo Jul 16 21:24:35.058381 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:24:35.058381 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:24:35.058381 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:24:35.058381 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:24:35.058381 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:24:35.058381 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:24:35.058381 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: 1dad823e240149edb30e592380249a13 Jul 16 21:24:35.058381 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-ServersTestJSON-1529284206 Jul 16 21:24:35.058381 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:24:35.058381 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:24:35.058381 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/1dad823e240149edb30e592380249a13"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/1dad823e240149edb30e592380249a13"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_1dad823e240149edb30e592380249a13"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/1dad823e240149edb30e592380249a13"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:24:35.058381 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-ServersTestJSON-1529284206 Jul 16 21:24:35.058381 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: 1dad823e240149edb30e592380249a13 Jul 16 21:24:35.058381 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-ServersTestJSON-1529284206 Jul 16 21:24:35.058381 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-ServersTestJSON-1529284206-project Jul 16 21:24:35.058381 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:24:35.058381 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:24:35.058381 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: 027dcc48e4fd4c36a5004096ad5e43f8 Jul 16 21:24:35.058381 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-ServersTestJSON-1529284206-project Jul 16 21:24:35.058381 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:24:35.058381 ncdv-master-1 devstack@n-api.service[138681]: exception: Instance 083a6d48-6623-437b-8649-92a693c68ebf could not be found. Jul 16 21:24:35.058381 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': '083a6d48-6623-437b-8649-92a693c68ebf'} Jul 16 21:24:35.058381 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:24:35.058381 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:24:35.058381 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:24:35.058381 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:24:35.058381 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:24:35.058381 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:24:35.058381 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:24:35.058381 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:24:35.058381 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:24:35.058381 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:24:35.058381 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 083a6d48-6623-437b-8649-92a693c68ebf could not be found. Jul 16 21:24:35.058381 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:24:35.058381 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:24:35.058381 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:24:35.058381 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:24:35.058381 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:24:35.058381 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:24:35.058381 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:24:35.058381 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:24:35.058381 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:24:35.058381 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:24:35.058381 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:24:35.058381 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:24:35.058381 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:24:35.058381 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:24:35.058381 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:24:35.058381 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:24:35.058381 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 083a6d48-6623-437b-8649-92a693c68ebf could not be found. Jul 16 21:24:35.058381 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:24:35.058381 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:24:35.058381 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:24:35.058381 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:24:35.065311 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:24:35.065311 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:24:35.065311 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:24:35.065311 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:24:35.065311 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 083a6d48-6623-437b-8649-92a693c68ebf could not be found. Jul 16 21:24:35.065311 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:24:35.066124 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-59d6cdf7-2bcc-4053-a67c-364a344091d0 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Returning 404 to user: Instance 083a6d48-6623-437b-8649-92a693c68ebf could not be found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:24:35.068009 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-59d6cdf7-2bcc-4053-a67c-364a344091d0 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "GET /compute/v2.1/servers/083a6d48-6623-437b-8649-92a693c68ebf" status: 404 len: 111 microversion: 2.1 time: 0.064996 Jul 16 21:24:35.069188 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1711/3429] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:24:34 2022] GET /compute/v2.1/servers/083a6d48-6623-437b-8649-92a693c68ebf => generated 111 bytes in 72 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Jul 16 21:24:35.083144 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-e9273fda-413d-4ea2-ad70-a0d2cdea7777 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Action: 'create', calling method: .version_select at 0x7fdd9c469820>, body: {"keypair": {"name": "tempest-key-1030576937", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBKjlnuxZlM+2DE8IDVrP/5yjmVPpxM43ENQkeSN34jXfmsB8WnvdkYi6SabCEE4aI1N90US6Et3BkhX+w3YZGXqqWUv/Q6YcG1FT+DxKAUlZPGNaVUpl34ptFefNo469Nw=="}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:24:35.098516 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-e9273fda-413d-4ea2-ad70-a0d2cdea7777 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Getting quotas for project 1dad823e240149edb30e592380249a13. Resources: {'key_pairs'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:24:35.104674 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-e9273fda-413d-4ea2-ad70-a0d2cdea7777 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Getting quotas for user 027dcc48e4fd4c36a5004096ad5e43f8 and project 1dad823e240149edb30e592380249a13. Resources: {'key_pairs'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:24:35.122523 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-0a7336a2-e1cb-4c67-b540-f34e15f59927 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:24:35.122523 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-0a7336a2-e1cb-4c67-b540-f34e15f59927 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 8e6eab25-f25e-46a2-9c72-9424c872fb84] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:24:35.127317 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0a7336a2-e1cb-4c67-b540-f34e15f59927 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:35.127776 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0a7336a2-e1cb-4c67-b540-f34e15f59927 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:35.128585 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0a7336a2-e1cb-4c67-b540-f34e15f59927 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:35.129029 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0a7336a2-e1cb-4c67-b540-f34e15f59927 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:35.252878 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-e9273fda-413d-4ea2-ad70-a0d2cdea7777 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "POST /compute/v2.1/os-keypairs" status: 200 len: 382 microversion: 2.1 time: 0.173694 Jul 16 21:24:35.259420 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1719/3430] 10.222.0.16 () {60 vars in 1158 bytes} [Sat Jul 16 21:24:35 2022] POST /compute/v2.1/os-keypairs => generated 382 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:24:35.264422 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-97482cc5-594c-4008-994e-5b430477be3d tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '.version_select at 0x7fdd9c6b1c10>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:24:35.267173 ncdv-master-1 devstack@n-api.service[138680]: WARNING oslo_db.sqlalchemy.utils [None req-97482cc5-594c-4008-994e-5b430477be3d tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Unique keys not in sort_keys. The sorting order may be unstable. Jul 16 21:24:35.283917 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-97482cc5-594c-4008-994e-5b430477be3d tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "GET /compute/v2.1/os-keypairs" status: 200 len: 351 microversion: 2.1 time: 0.019450 Jul 16 21:24:35.283917 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1720/3431] 10.222.0.16 () {58 vars in 1136 bytes} [Sat Jul 16 21:24:35 2022] GET /compute/v2.1/os-keypairs => generated 351 bytes in 22 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:24:35.451703 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-0a7336a2-e1cb-4c67-b540-f34e15f59927 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f2e974ab5aa48dfb4b00f810dcedb63', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd31be8475a6a49d6bfe8132dd050c6c0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:24:35.466772 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-0a7336a2-e1cb-4c67-b540-f34e15f59927 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "GET /compute/v2.1/servers/8e6eab25-f25e-46a2-9c72-9424c872fb84" status: 200 len: 1523 microversion: 2.1 time: 0.348572 Jul 16 21:24:35.467871 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1712/3432] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:24:35 2022] GET /compute/v2.1/servers/8e6eab25-f25e-46a2-9c72-9424c872fb84 => generated 1523 bytes in 354 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:24:35.484869 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-1f642a4b-a912-4b2c-839b-6e06dba4278e tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServersTestJSON-server-235518694", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452", "key_name": "tempest-key-1030576937", "networks": [{"uuid": "2202d4ad-313c-4607-b9fc-eb98bac9afd0"}]}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:24:35.595510 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.network.neutron [None req-1f642a4b-a912-4b2c-839b-6e06dba4278e tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] validate_networks() for [('2202d4ad-313c-4607-b9fc-eb98bac9afd0', None, None, None, None, None)] {{(pid=138680) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 16 21:24:35.909236 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-70b1d62f-0aab-4a3b-92ea-a3d71b0f48b6 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:24:35.910858 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-70b1d62f-0aab-4a3b-92ea-a3d71b0f48b6 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] [instance: 867b2ab0-9301-414b-b045-85057d642971] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:24:35.924734 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-70b1d62f-0aab-4a3b-92ea-a3d71b0f48b6 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:35.928330 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-70b1d62f-0aab-4a3b-92ea-a3d71b0f48b6 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:35.928330 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-70b1d62f-0aab-4a3b-92ea-a3d71b0f48b6 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:35.934389 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-70b1d62f-0aab-4a3b-92ea-a3d71b0f48b6 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:36.434024 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-70b1d62f-0aab-4a3b-92ea-a3d71b0f48b6 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcc9aa2a091c490599f9a9c41163e2eb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da56fc78f06142aba80ca4c6e3d60475', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:24:36.447691 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-70b1d62f-0aab-4a3b-92ea-a3d71b0f48b6 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] 10.222.0.16 "GET /compute/v2.1/servers/867b2ab0-9301-414b-b045-85057d642971" status: 200 len: 1522 microversion: 2.1 time: 0.541876 Jul 16 21:24:36.451316 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1713/3433] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:24:35 2022] GET /compute/v2.1/servers/867b2ab0-9301-414b-b045-85057d642971 => generated 1522 bytes in 548 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:24:36.462493 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-0c40f98c-a01d-49e3-9b7c-b3d1ad81f8ac tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:24:36.465190 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-0c40f98c-a01d-49e3-9b7c-b3d1ad81f8ac tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] [instance: 2fad683a-f21f-4a9c-8323-e9380ef7f9ef] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:24:36.480169 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0c40f98c-a01d-49e3-9b7c-b3d1ad81f8ac tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:36.483793 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0c40f98c-a01d-49e3-9b7c-b3d1ad81f8ac tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:36.484465 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0c40f98c-a01d-49e3-9b7c-b3d1ad81f8ac tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:36.485494 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0c40f98c-a01d-49e3-9b7c-b3d1ad81f8ac tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:36.767800 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-1f642a4b-a912-4b2c-839b-6e06dba4278e tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Flavor limits 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 16 21:24:36.768887 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-1f642a4b-a912-4b2c-839b-6e06dba4278e tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Image limits 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 16 21:24:36.769547 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-1f642a4b-a912-4b2c-839b-6e06dba4278e tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Flavor pref 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 16 21:24:36.770145 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-1f642a4b-a912-4b2c-839b-6e06dba4278e tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Image pref 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 16 21:24:36.770728 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-1f642a4b-a912-4b2c-839b-6e06dba4278e tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 16 21:24:36.792109 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1f642a4b-a912-4b2c-839b-6e06dba4278e tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:36.797060 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1f642a4b-a912-4b2c-839b-6e06dba4278e tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:36.797060 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1f642a4b-a912-4b2c-839b-6e06dba4278e tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:36.797060 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1f642a4b-a912-4b2c-839b-6e06dba4278e tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:36.841317 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-1f642a4b-a912-4b2c-839b-6e06dba4278e tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Getting quotas for project 1dad823e240149edb30e592380249a13. Resources: {'ram', 'instances', 'cores'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:24:36.854859 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-1f642a4b-a912-4b2c-839b-6e06dba4278e tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Getting quotas for user 027dcc48e4fd4c36a5004096ad5e43f8 and project 1dad823e240149edb30e592380249a13. Resources: {'ram', 'instances', 'cores'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:24:36.909154 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-1f642a4b-a912-4b2c-839b-6e06dba4278e tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Going to run 1 instances... {{(pid=138680) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 16 21:24:36.917950 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-1f642a4b-a912-4b2c-839b-6e06dba4278e tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: c9901d0a-8a17-4e6c-ba98-6bd587d410e0] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138680) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 16 21:24:36.932284 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-0c40f98c-a01d-49e3-9b7c-b3d1ad81f8ac tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bba77f6ef42547f1aa8480e27868bd4a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '22ca92a86e6c40d59071e58ce4085568', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:24:36.946142 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-0c40f98c-a01d-49e3-9b7c-b3d1ad81f8ac tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] 10.222.0.16 "GET /compute/v2.1/servers/2fad683a-f21f-4a9c-8323-e9380ef7f9ef" status: 200 len: 1362 microversion: 2.1 time: 0.490498 Jul 16 21:24:36.958568 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1714/3434] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:24:36 2022] GET /compute/v2.1/servers/2fad683a-f21f-4a9c-8323-e9380ef7f9ef => generated 1362 bytes in 506 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:24:36.970728 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-3e9ac7e9-8a27-4f4d-a009-b68e12dea6ac tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:24:36.972627 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-3e9ac7e9-8a27-4f4d-a009-b68e12dea6ac tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 8e6eab25-f25e-46a2-9c72-9424c872fb84] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:24:36.989923 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3e9ac7e9-8a27-4f4d-a009-b68e12dea6ac tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:36.990385 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3e9ac7e9-8a27-4f4d-a009-b68e12dea6ac tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:36.998163 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3e9ac7e9-8a27-4f4d-a009-b68e12dea6ac tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:36.998163 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3e9ac7e9-8a27-4f4d-a009-b68e12dea6ac tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:37.169459 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-1f642a4b-a912-4b2c-839b-6e06dba4278e tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 1.689277 Jul 16 21:24:37.174675 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1721/3435] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 21:24:35 2022] POST /compute/v2.1/servers => generated 380 bytes in 1696 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 16 21:24:37.185300 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-ce0c9a05-9b0c-431a-91cf-48648bf6b984 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:24:37.191777 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-ce0c9a05-9b0c-431a-91cf-48648bf6b984 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: c9901d0a-8a17-4e6c-ba98-6bd587d410e0] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:24:37.472072 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-ce0c9a05-9b0c-431a-91cf-48648bf6b984 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '027dcc48e4fd4c36a5004096ad5e43f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dad823e240149edb30e592380249a13', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:24:37.487200 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-ce0c9a05-9b0c-431a-91cf-48648bf6b984 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "GET /compute/v2.1/servers/c9901d0a-8a17-4e6c-ba98-6bd587d410e0" status: 200 len: 1228 microversion: 2.1 time: 0.303672 Jul 16 21:24:37.487980 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1722/3436] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:24:37 2022] GET /compute/v2.1/servers/c9901d0a-8a17-4e6c-ba98-6bd587d410e0 => generated 1228 bytes in 308 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:24:37.499131 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-2ae608e9-9267-43f9-8d21-10af94109970 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:24:37.499870 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-2ae608e9-9267-43f9-8d21-10af94109970 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] [instance: 867b2ab0-9301-414b-b045-85057d642971] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:24:37.519232 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2ae608e9-9267-43f9-8d21-10af94109970 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:37.523413 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2ae608e9-9267-43f9-8d21-10af94109970 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:37.523413 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2ae608e9-9267-43f9-8d21-10af94109970 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:37.523413 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2ae608e9-9267-43f9-8d21-10af94109970 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:37.794979 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-3e9ac7e9-8a27-4f4d-a009-b68e12dea6ac tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f2e974ab5aa48dfb4b00f810dcedb63', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd31be8475a6a49d6bfe8132dd050c6c0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:24:37.798333 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=100 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:24:37.839565 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-3e9ac7e9-8a27-4f4d-a009-b68e12dea6ac tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "GET /compute/v2.1/servers/8e6eab25-f25e-46a2-9c72-9424c872fb84" status: 200 len: 1523 microversion: 2.1 time: 0.873512 Jul 16 21:24:37.842145 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1715/3437] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:24:36 2022] GET /compute/v2.1/servers/8e6eab25-f25e-46a2-9c72-9424c872fb84 => generated 1523 bytes in 883 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:24:37.965714 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-e14b5ecc-c7c4-4bd6-bac1-43f2b7323d6b tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:24:37.966831 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-e14b5ecc-c7c4-4bd6-bac1-43f2b7323d6b tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] [instance: 2fad683a-f21f-4a9c-8323-e9380ef7f9ef] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:24:37.984488 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e14b5ecc-c7c4-4bd6-bac1-43f2b7323d6b tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:37.984951 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e14b5ecc-c7c4-4bd6-bac1-43f2b7323d6b tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:37.985780 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e14b5ecc-c7c4-4bd6-bac1-43f2b7323d6b tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:37.986413 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e14b5ecc-c7c4-4bd6-bac1-43f2b7323d6b tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:38.067433 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-2ae608e9-9267-43f9-8d21-10af94109970 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcc9aa2a091c490599f9a9c41163e2eb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da56fc78f06142aba80ca4c6e3d60475', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:24:38.076667 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-2ae608e9-9267-43f9-8d21-10af94109970 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] 10.222.0.16 "GET /compute/v2.1/servers/867b2ab0-9301-414b-b045-85057d642971" status: 200 len: 1522 microversion: 2.1 time: 0.587498 Jul 16 21:24:38.077256 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1723/3438] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:24:37 2022] GET /compute/v2.1/servers/867b2ab0-9301-414b-b045-85057d642971 => generated 1522 bytes in 589 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:24:38.223333 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-e14b5ecc-c7c4-4bd6-bac1-43f2b7323d6b tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bba77f6ef42547f1aa8480e27868bd4a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '22ca92a86e6c40d59071e58ce4085568', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:24:38.242495 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-e14b5ecc-c7c4-4bd6-bac1-43f2b7323d6b tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] 10.222.0.16 "GET /compute/v2.1/servers/2fad683a-f21f-4a9c-8323-e9380ef7f9ef" status: 200 len: 1362 microversion: 2.1 time: 0.281099 Jul 16 21:24:38.255693 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1716/3439] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:24:37 2022] GET /compute/v2.1/servers/2fad683a-f21f-4a9c-8323-e9380ef7f9ef => generated 1362 bytes in 295 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:24:38.261424 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-ec6bfc7c-9ea8-4de4-8212-b6093091bc1c req-53c9bd76-e9c5-47ef-b1f9-b71a077b86cf service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "8e6eab25-f25e-46a2-9c72-9424c872fb84", "name": "network-vif-plugged", "status": "completed", "tag": "4a81f7ad-929b-4f37-b5ac-e484b09be512"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:24:38.294753 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-ec6bfc7c-9ea8-4de4-8212-b6093091bc1c req-53c9bd76-e9c5-47ef-b1f9-b71a077b86cf service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:38.295556 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-ec6bfc7c-9ea8-4de4-8212-b6093091bc1c req-53c9bd76-e9c5-47ef-b1f9-b71a077b86cf service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:38.317265 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-ec6bfc7c-9ea8-4de4-8212-b6093091bc1c req-53c9bd76-e9c5-47ef-b1f9-b71a077b86cf service nova] Creating event network-vif-plugged:4a81f7ad-929b-4f37-b5ac-e484b09be512 for instance 8e6eab25-f25e-46a2-9c72-9424c872fb84 on nch2-master-1 Jul 16 21:24:38.328658 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-ec6bfc7c-9ea8-4de4-8212-b6093091bc1c req-53c9bd76-e9c5-47ef-b1f9-b71a077b86cf service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.083488 Jul 16 21:24:38.340143 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1724/3440] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:24:38 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 16 21:24:38.506323 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-55fda62a-cbd5-45df-bebe-d39355db4e2b tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:24:38.507305 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-55fda62a-cbd5-45df-bebe-d39355db4e2b tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: c9901d0a-8a17-4e6c-ba98-6bd587d410e0] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:24:38.620913 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-55fda62a-cbd5-45df-bebe-d39355db4e2b tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '027dcc48e4fd4c36a5004096ad5e43f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dad823e240149edb30e592380249a13', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:24:38.631860 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-55fda62a-cbd5-45df-bebe-d39355db4e2b tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "GET /compute/v2.1/servers/c9901d0a-8a17-4e6c-ba98-6bd587d410e0" status: 200 len: 1228 microversion: 2.1 time: 0.129069 Jul 16 21:24:38.633212 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1717/3441] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:24:38 2022] GET /compute/v2.1/servers/c9901d0a-8a17-4e6c-ba98-6bd587d410e0 => generated 1228 bytes in 132 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:24:38.870290 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-89795eff-4ec8-4c5a-9985-84221aa571e7 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:24:38.871139 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-89795eff-4ec8-4c5a-9985-84221aa571e7 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 8e6eab25-f25e-46a2-9c72-9424c872fb84] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:24:38.896006 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_api:SELECT=95,nova_api:UPDATE=1,nova_api:INSERT=4 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:24:38.898614 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-89795eff-4ec8-4c5a-9985-84221aa571e7 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:38.900704 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-89795eff-4ec8-4c5a-9985-84221aa571e7 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:38.901787 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-89795eff-4ec8-4c5a-9985-84221aa571e7 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:38.904107 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-89795eff-4ec8-4c5a-9985-84221aa571e7 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:39.100398 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-d046b628-56d6-4e4b-ba27-77116a543fee tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:24:39.102085 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-d046b628-56d6-4e4b-ba27-77116a543fee tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] [instance: 867b2ab0-9301-414b-b045-85057d642971] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:24:39.110038 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d046b628-56d6-4e4b-ba27-77116a543fee tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:39.112560 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d046b628-56d6-4e4b-ba27-77116a543fee tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:39.112560 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d046b628-56d6-4e4b-ba27-77116a543fee tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:39.112560 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d046b628-56d6-4e4b-ba27-77116a543fee tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:39.557444 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-89795eff-4ec8-4c5a-9985-84221aa571e7 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f2e974ab5aa48dfb4b00f810dcedb63', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd31be8475a6a49d6bfe8132dd050c6c0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:24:39.612086 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-89795eff-4ec8-4c5a-9985-84221aa571e7 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "GET /compute/v2.1/servers/8e6eab25-f25e-46a2-9c72-9424c872fb84" status: 200 len: 1523 microversion: 2.1 time: 0.742779 Jul 16 21:24:39.614978 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1725/3442] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:24:38 2022] GET /compute/v2.1/servers/8e6eab25-f25e-46a2-9c72-9424c872fb84 => generated 1523 bytes in 760 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:24:39.653469 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-451a3e49-9abf-420f-aba1-af64d2eb7d36 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:24:39.659278 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-451a3e49-9abf-420f-aba1-af64d2eb7d36 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] [instance: 2fad683a-f21f-4a9c-8323-e9380ef7f9ef] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:24:39.732971 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-451a3e49-9abf-420f-aba1-af64d2eb7d36 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:39.733453 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-451a3e49-9abf-420f-aba1-af64d2eb7d36 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:39.734277 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-451a3e49-9abf-420f-aba1-af64d2eb7d36 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:39.743866 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-d046b628-56d6-4e4b-ba27-77116a543fee tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcc9aa2a091c490599f9a9c41163e2eb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da56fc78f06142aba80ca4c6e3d60475', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:24:39.752432 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-451a3e49-9abf-420f-aba1-af64d2eb7d36 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.018s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:39.753421 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-d046b628-56d6-4e4b-ba27-77116a543fee tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] 10.222.0.16 "GET /compute/v2.1/servers/867b2ab0-9301-414b-b045-85057d642971" status: 200 len: 1522 microversion: 2.1 time: 0.659147 Jul 16 21:24:39.767382 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1718/3443] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:24:39 2022] GET /compute/v2.1/servers/867b2ab0-9301-414b-b045-85057d642971 => generated 1522 bytes in 674 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:24:39.772744 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-48d458fb-c950-4fb6-8d25-a4ab7a994dca tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:24:39.773629 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-48d458fb-c950-4fb6-8d25-a4ab7a994dca tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: c9901d0a-8a17-4e6c-ba98-6bd587d410e0] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:24:39.785043 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-48d458fb-c950-4fb6-8d25-a4ab7a994dca tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:39.785857 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-48d458fb-c950-4fb6-8d25-a4ab7a994dca tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:39.786845 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-48d458fb-c950-4fb6-8d25-a4ab7a994dca tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:39.790892 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-48d458fb-c950-4fb6-8d25-a4ab7a994dca tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:40.071416 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-48d458fb-c950-4fb6-8d25-a4ab7a994dca tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '027dcc48e4fd4c36a5004096ad5e43f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dad823e240149edb30e592380249a13', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:24:40.089869 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-48d458fb-c950-4fb6-8d25-a4ab7a994dca tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "GET /compute/v2.1/servers/c9901d0a-8a17-4e6c-ba98-6bd587d410e0" status: 200 len: 1224 microversion: 2.1 time: 0.320962 Jul 16 21:24:40.093368 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1719/3444] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:24:39 2022] GET /compute/v2.1/servers/c9901d0a-8a17-4e6c-ba98-6bd587d410e0 => generated 1224 bytes in 323 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:24:40.242735 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-451a3e49-9abf-420f-aba1-af64d2eb7d36 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bba77f6ef42547f1aa8480e27868bd4a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '22ca92a86e6c40d59071e58ce4085568', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:24:40.253335 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-451a3e49-9abf-420f-aba1-af64d2eb7d36 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] 10.222.0.16 "GET /compute/v2.1/servers/2fad683a-f21f-4a9c-8323-e9380ef7f9ef" status: 200 len: 1362 microversion: 2.1 time: 0.636044 Jul 16 21:24:40.258445 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1726/3445] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:24:39 2022] GET /compute/v2.1/servers/2fad683a-f21f-4a9c-8323-e9380ef7f9ef => generated 1362 bytes in 643 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:24:40.388864 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-8ef67a1c-a5ba-4f2c-94e0-8e9c0f512a78 req-327ac33a-2055-45dd-af0f-45ae080391c0 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "2fad683a-f21f-4a9c-8323-e9380ef7f9ef", "name": "network-vif-plugged", "status": "completed", "tag": "aff74b78-59ed-4a31-9d5b-3c3e6aa5d30c"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:24:40.400956 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-8ef67a1c-a5ba-4f2c-94e0-8e9c0f512a78 req-327ac33a-2055-45dd-af0f-45ae080391c0 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:40.401545 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-8ef67a1c-a5ba-4f2c-94e0-8e9c0f512a78 req-327ac33a-2055-45dd-af0f-45ae080391c0 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:40.417371 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-8ef67a1c-a5ba-4f2c-94e0-8e9c0f512a78 req-327ac33a-2055-45dd-af0f-45ae080391c0 service nova] Creating event network-vif-plugged:aff74b78-59ed-4a31-9d5b-3c3e6aa5d30c for instance 2fad683a-f21f-4a9c-8323-e9380ef7f9ef on nch1-master-1 Jul 16 21:24:40.423477 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-8ef67a1c-a5ba-4f2c-94e0-8e9c0f512a78 req-327ac33a-2055-45dd-af0f-45ae080391c0 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.039524 Jul 16 21:24:40.425226 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1720/3446] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:24:40 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:24:40.634482 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-efd56ffe-4552-4bde-b6a3-99342753fba5 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:24:40.636866 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-efd56ffe-4552-4bde-b6a3-99342753fba5 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 8e6eab25-f25e-46a2-9c72-9424c872fb84] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:24:40.654298 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-efd56ffe-4552-4bde-b6a3-99342753fba5 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:40.658156 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-efd56ffe-4552-4bde-b6a3-99342753fba5 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:40.659291 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-efd56ffe-4552-4bde-b6a3-99342753fba5 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:40.662034 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-efd56ffe-4552-4bde-b6a3-99342753fba5 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:40.776001 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-4b9ec32f-9c3d-41ac-9931-332c53ab0c0b tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:24:40.777703 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-4b9ec32f-9c3d-41ac-9931-332c53ab0c0b tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] [instance: 867b2ab0-9301-414b-b045-85057d642971] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:24:40.806444 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4b9ec32f-9c3d-41ac-9931-332c53ab0c0b tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:40.806905 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4b9ec32f-9c3d-41ac-9931-332c53ab0c0b tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:40.815248 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4b9ec32f-9c3d-41ac-9931-332c53ab0c0b tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:40.815784 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4b9ec32f-9c3d-41ac-9931-332c53ab0c0b tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:41.346405 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-efd56ffe-4552-4bde-b6a3-99342753fba5 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f2e974ab5aa48dfb4b00f810dcedb63', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd31be8475a6a49d6bfe8132dd050c6c0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:24:41.367859 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-efd56ffe-4552-4bde-b6a3-99342753fba5 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "GET /compute/v2.1/servers/8e6eab25-f25e-46a2-9c72-9424c872fb84" status: 200 len: 1523 microversion: 2.1 time: 0.738973 Jul 16 21:24:41.372670 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1727/3447] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:24:40 2022] GET /compute/v2.1/servers/8e6eab25-f25e-46a2-9c72-9424c872fb84 => generated 1523 bytes in 747 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:24:41.388996 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-7e99685c-c673-46f8-9358-936213575c66 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:24:41.391836 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-7e99685c-c673-46f8-9358-936213575c66 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: c9901d0a-8a17-4e6c-ba98-6bd587d410e0] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:24:41.404181 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7e99685c-c673-46f8-9358-936213575c66 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:41.406672 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7e99685c-c673-46f8-9358-936213575c66 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:41.411628 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7e99685c-c673-46f8-9358-936213575c66 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:41.412812 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7e99685c-c673-46f8-9358-936213575c66 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:41.507362 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-4b9ec32f-9c3d-41ac-9931-332c53ab0c0b tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcc9aa2a091c490599f9a9c41163e2eb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da56fc78f06142aba80ca4c6e3d60475', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:24:41.525499 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-4b9ec32f-9c3d-41ac-9931-332c53ab0c0b tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] 10.222.0.16 "GET /compute/v2.1/servers/867b2ab0-9301-414b-b045-85057d642971" status: 200 len: 1522 microversion: 2.1 time: 0.754037 Jul 16 21:24:41.529042 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1721/3448] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:24:40 2022] GET /compute/v2.1/servers/867b2ab0-9301-414b-b045-85057d642971 => generated 1522 bytes in 760 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:24:41.543700 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-d1af9564-525b-4e65-9df8-8fbc1c1370ef tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:24:41.544582 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-d1af9564-525b-4e65-9df8-8fbc1c1370ef tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] [instance: 2fad683a-f21f-4a9c-8323-e9380ef7f9ef] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:24:41.562874 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d1af9564-525b-4e65-9df8-8fbc1c1370ef tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:41.563678 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d1af9564-525b-4e65-9df8-8fbc1c1370ef tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:41.564803 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d1af9564-525b-4e65-9df8-8fbc1c1370ef tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:41.567248 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d1af9564-525b-4e65-9df8-8fbc1c1370ef tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:42.544660 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-7e99685c-c673-46f8-9358-936213575c66 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '027dcc48e4fd4c36a5004096ad5e43f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dad823e240149edb30e592380249a13', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:24:42.562710 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-7e99685c-c673-46f8-9358-936213575c66 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "GET /compute/v2.1/servers/c9901d0a-8a17-4e6c-ba98-6bd587d410e0" status: 200 len: 1286 microversion: 2.1 time: 1.182721 Jul 16 21:24:42.563507 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1728/3449] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:24:41 2022] GET /compute/v2.1/servers/c9901d0a-8a17-4e6c-ba98-6bd587d410e0 => generated 1286 bytes in 1187 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:24:42.578565 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-f7adf1d5-3c46-465e-b848-cc4ed496c3ed tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:24:42.583326 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-f7adf1d5-3c46-465e-b848-cc4ed496c3ed tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 8e6eab25-f25e-46a2-9c72-9424c872fb84] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:24:42.600730 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f7adf1d5-3c46-465e-b848-cc4ed496c3ed tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:42.609225 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f7adf1d5-3c46-465e-b848-cc4ed496c3ed tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:42.609225 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f7adf1d5-3c46-465e-b848-cc4ed496c3ed tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:42.609225 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f7adf1d5-3c46-465e-b848-cc4ed496c3ed tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:42.922227 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-d1af9564-525b-4e65-9df8-8fbc1c1370ef tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bba77f6ef42547f1aa8480e27868bd4a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '22ca92a86e6c40d59071e58ce4085568', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:24:42.940587 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-d1af9564-525b-4e65-9df8-8fbc1c1370ef tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] 10.222.0.16 "GET /compute/v2.1/servers/2fad683a-f21f-4a9c-8323-e9380ef7f9ef" status: 200 len: 1362 microversion: 2.1 time: 1.409882 Jul 16 21:24:42.941334 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1722/3450] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:24:41 2022] GET /compute/v2.1/servers/2fad683a-f21f-4a9c-8323-e9380ef7f9ef => generated 1362 bytes in 1412 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:24:42.976874 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-2bb4a41e-86c2-4043-99ab-2771bd95d156 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:24:42.977762 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-2bb4a41e-86c2-4043-99ab-2771bd95d156 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] [instance: 867b2ab0-9301-414b-b045-85057d642971] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:24:42.999881 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2bb4a41e-86c2-4043-99ab-2771bd95d156 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:43.000490 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2bb4a41e-86c2-4043-99ab-2771bd95d156 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:43.001404 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2bb4a41e-86c2-4043-99ab-2771bd95d156 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:43.001967 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2bb4a41e-86c2-4043-99ab-2771bd95d156 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:43.009201 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-f7adf1d5-3c46-465e-b848-cc4ed496c3ed tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f2e974ab5aa48dfb4b00f810dcedb63', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd31be8475a6a49d6bfe8132dd050c6c0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:24:43.026792 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-f7adf1d5-3c46-465e-b848-cc4ed496c3ed tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "GET /compute/v2.1/servers/8e6eab25-f25e-46a2-9c72-9424c872fb84" status: 200 len: 1523 microversion: 2.1 time: 0.460701 Jul 16 21:24:43.027901 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1729/3451] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:24:42 2022] GET /compute/v2.1/servers/8e6eab25-f25e-46a2-9c72-9424c872fb84 => generated 1523 bytes in 463 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:24:43.248731 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-2bb4a41e-86c2-4043-99ab-2771bd95d156 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcc9aa2a091c490599f9a9c41163e2eb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da56fc78f06142aba80ca4c6e3d60475', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:24:43.260990 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-2bb4a41e-86c2-4043-99ab-2771bd95d156 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] 10.222.0.16 "GET /compute/v2.1/servers/867b2ab0-9301-414b-b045-85057d642971" status: 200 len: 1522 microversion: 2.1 time: 0.303787 Jul 16 21:24:43.262158 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1723/3452] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:24:42 2022] GET /compute/v2.1/servers/867b2ab0-9301-414b-b045-85057d642971 => generated 1522 bytes in 307 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:24:43.580954 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-5bf30359-da1a-4eb2-bc2a-0874740e2c21 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:24:43.581640 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-5bf30359-da1a-4eb2-bc2a-0874740e2c21 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: c9901d0a-8a17-4e6c-ba98-6bd587d410e0] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:24:43.647701 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5bf30359-da1a-4eb2-bc2a-0874740e2c21 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:43.655466 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5bf30359-da1a-4eb2-bc2a-0874740e2c21 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:43.661208 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5bf30359-da1a-4eb2-bc2a-0874740e2c21 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:43.661943 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5bf30359-da1a-4eb2-bc2a-0874740e2c21 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:43.810845 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-5bf30359-da1a-4eb2-bc2a-0874740e2c21 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '027dcc48e4fd4c36a5004096ad5e43f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dad823e240149edb30e592380249a13', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:24:43.819692 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-5bf30359-da1a-4eb2-bc2a-0874740e2c21 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "GET /compute/v2.1/servers/c9901d0a-8a17-4e6c-ba98-6bd587d410e0" status: 200 len: 1286 microversion: 2.1 time: 0.243336 Jul 16 21:24:43.824861 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1730/3453] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:24:43 2022] GET /compute/v2.1/servers/c9901d0a-8a17-4e6c-ba98-6bd587d410e0 => generated 1286 bytes in 249 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:24:43.960621 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-e044142a-2efb-455b-a0a3-15fac7018aa0 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:24:43.961843 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-e044142a-2efb-455b-a0a3-15fac7018aa0 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] [instance: 2fad683a-f21f-4a9c-8323-e9380ef7f9ef] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:24:43.984627 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e044142a-2efb-455b-a0a3-15fac7018aa0 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:43.984627 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e044142a-2efb-455b-a0a3-15fac7018aa0 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:43.988325 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e044142a-2efb-455b-a0a3-15fac7018aa0 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:43.989029 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e044142a-2efb-455b-a0a3-15fac7018aa0 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:44.045345 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-ccb2979a-a262-401b-9602-1506617055fb tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:24:44.046258 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-ccb2979a-a262-401b-9602-1506617055fb tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 8e6eab25-f25e-46a2-9c72-9424c872fb84] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:24:44.065969 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ccb2979a-a262-401b-9602-1506617055fb tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:44.068152 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ccb2979a-a262-401b-9602-1506617055fb tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:44.072718 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ccb2979a-a262-401b-9602-1506617055fb tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:44.079080 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ccb2979a-a262-401b-9602-1506617055fb tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.009s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:44.972931 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-ccb2979a-a262-401b-9602-1506617055fb tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f2e974ab5aa48dfb4b00f810dcedb63', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd31be8475a6a49d6bfe8132dd050c6c0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:24:44.984270 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-ccb2979a-a262-401b-9602-1506617055fb tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "GET /compute/v2.1/servers/8e6eab25-f25e-46a2-9c72-9424c872fb84" status: 200 len: 1523 microversion: 2.1 time: 0.944452 Jul 16 21:24:44.992745 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1731/3454] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:24:44 2022] GET /compute/v2.1/servers/8e6eab25-f25e-46a2-9c72-9424c872fb84 => generated 1523 bytes in 953 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:24:45.002110 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-0ce8f542-1e61-47ff-b43d-54fe28c32fe0 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:24:45.008465 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-0ce8f542-1e61-47ff-b43d-54fe28c32fe0 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] [instance: 867b2ab0-9301-414b-b045-85057d642971] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:24:45.042705 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0ce8f542-1e61-47ff-b43d-54fe28c32fe0 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:45.049531 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0ce8f542-1e61-47ff-b43d-54fe28c32fe0 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:45.065999 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0ce8f542-1e61-47ff-b43d-54fe28c32fe0 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:45.065999 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0ce8f542-1e61-47ff-b43d-54fe28c32fe0 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:45.086855 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-e044142a-2efb-455b-a0a3-15fac7018aa0 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bba77f6ef42547f1aa8480e27868bd4a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '22ca92a86e6c40d59071e58ce4085568', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:24:45.101921 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-e044142a-2efb-455b-a0a3-15fac7018aa0 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] 10.222.0.16 "GET /compute/v2.1/servers/2fad683a-f21f-4a9c-8323-e9380ef7f9ef" status: 200 len: 1548 microversion: 2.1 time: 1.147682 Jul 16 21:24:45.106282 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1724/3455] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:24:43 2022] GET /compute/v2.1/servers/2fad683a-f21f-4a9c-8323-e9380ef7f9ef => generated 1548 bytes in 1153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:24:45.121078 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-300c2e5f-daa5-4fab-9a83-1576c52a9274 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:24:45.123881 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-300c2e5f-daa5-4fab-9a83-1576c52a9274 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: c9901d0a-8a17-4e6c-ba98-6bd587d410e0] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:24:45.139231 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-300c2e5f-daa5-4fab-9a83-1576c52a9274 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:45.140156 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-300c2e5f-daa5-4fab-9a83-1576c52a9274 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:45.144124 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-300c2e5f-daa5-4fab-9a83-1576c52a9274 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:45.145311 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-300c2e5f-daa5-4fab-9a83-1576c52a9274 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:45.926484 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-0ce8f542-1e61-47ff-b43d-54fe28c32fe0 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcc9aa2a091c490599f9a9c41163e2eb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da56fc78f06142aba80ca4c6e3d60475', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:24:45.936786 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-0ce8f542-1e61-47ff-b43d-54fe28c32fe0 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] 10.222.0.16 "GET /compute/v2.1/servers/867b2ab0-9301-414b-b045-85057d642971" status: 200 len: 1522 microversion: 2.1 time: 0.941060 Jul 16 21:24:45.941291 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1732/3456] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:24:44 2022] GET /compute/v2.1/servers/867b2ab0-9301-414b-b045-85057d642971 => generated 1522 bytes in 948 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:24:45.945716 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-300c2e5f-daa5-4fab-9a83-1576c52a9274 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '027dcc48e4fd4c36a5004096ad5e43f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dad823e240149edb30e592380249a13', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:24:45.952259 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-9ba8a5c6-312b-4ead-a161-b7df47801875 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:24:45.953270 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-9ba8a5c6-312b-4ead-a161-b7df47801875 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] [instance: 2fad683a-f21f-4a9c-8323-e9380ef7f9ef] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:24:45.969960 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9ba8a5c6-312b-4ead-a161-b7df47801875 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:45.971841 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9ba8a5c6-312b-4ead-a161-b7df47801875 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:45.975641 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-300c2e5f-daa5-4fab-9a83-1576c52a9274 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "GET /compute/v2.1/servers/c9901d0a-8a17-4e6c-ba98-6bd587d410e0" status: 200 len: 1328 microversion: 2.1 time: 0.866757 Jul 16 21:24:45.976397 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1725/3457] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:24:45 2022] GET /compute/v2.1/servers/c9901d0a-8a17-4e6c-ba98-6bd587d410e0 => generated 1328 bytes in 869 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:24:45.977080 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9ba8a5c6-312b-4ead-a161-b7df47801875 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:45.978609 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9ba8a5c6-312b-4ead-a161-b7df47801875 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:45.997668 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-a0524fc4-3f24-4723-bd70-27854afeff65 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Action: 'create', calling method: .version_select at 0x7fdda2a60ca0>, body: {"keypair": {"name": "tempest-keypair-1847534343", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBPP9Y514WF6DdwWjdBKYurt43I6lZSWet1PI8P8qXOCGcwQvzWyXvl12TuV0MKca72AHOnSGu90/R+Jji1f5E6W+vjIR+kIi2VO8WDoL+cm6iORU4Eosw3qX3bHGNr3n4A=="}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:24:46.048740 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-a0524fc4-3f24-4723-bd70-27854afeff65 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Getting quotas for project cfc2e42a0df346f4adef152ba2e3fa71. Resources: {'key_pairs'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:24:46.067523 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-a0524fc4-3f24-4723-bd70-27854afeff65 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Getting quotas for user a23fafbab18d41ee82b84bbdf94cf0fe and project cfc2e42a0df346f4adef152ba2e3fa71. Resources: {'key_pairs'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:24:46.167302 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-a0524fc4-3f24-4723-bd70-27854afeff65 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] 10.222.0.16 "POST /compute/v2.1/os-keypairs" status: 201 len: 401 microversion: 2.32 time: 0.185196 Jul 16 21:24:46.174994 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1726/3458] 10.222.0.16 () {62 vars in 1199 bytes} [Sat Jul 16 21:24:45 2022] POST /compute/v2.1/os-keypairs => generated 401 bytes in 193 msecs (HTTP/1.1 201) 9 headers in 364 bytes (1 switches on core 0) Jul 16 21:24:46.213244 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-6ead6355-eef9-40f3-8f33-8e8bcb78cea9 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:24:46.216318 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-6ead6355-eef9-40f3-8f33-8e8bcb78cea9 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 8e6eab25-f25e-46a2-9c72-9424c872fb84] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:24:46.238364 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6ead6355-eef9-40f3-8f33-8e8bcb78cea9 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:46.238941 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6ead6355-eef9-40f3-8f33-8e8bcb78cea9 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:46.240095 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6ead6355-eef9-40f3-8f33-8e8bcb78cea9 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:46.240827 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6ead6355-eef9-40f3-8f33-8e8bcb78cea9 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:46.243366 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-9ba8a5c6-312b-4ead-a161-b7df47801875 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] 10.222.0.16 "GET /compute/v2.1/servers/2fad683a-f21f-4a9c-8323-e9380ef7f9ef/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.298190 Jul 16 21:24:46.243999 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1733/3459] 10.222.0.16 () {58 vars in 1274 bytes} [Sat Jul 16 21:24:45 2022] GET /compute/v2.1/servers/2fad683a-f21f-4a9c-8323-e9380ef7f9ef/os-interface => generated 284 bytes in 301 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:24:46.732720 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-6ead6355-eef9-40f3-8f33-8e8bcb78cea9 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f2e974ab5aa48dfb4b00f810dcedb63', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd31be8475a6a49d6bfe8132dd050c6c0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:24:46.744639 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-6ead6355-eef9-40f3-8f33-8e8bcb78cea9 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "GET /compute/v2.1/servers/8e6eab25-f25e-46a2-9c72-9424c872fb84" status: 200 len: 1523 microversion: 2.1 time: 0.564360 Jul 16 21:24:46.745557 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1727/3460] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:24:46 2022] GET /compute/v2.1/servers/8e6eab25-f25e-46a2-9c72-9424c872fb84 => generated 1523 bytes in 568 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:24:46.955333 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell0:SELECT=2 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:24:46.971371 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-61a7fceb-b1ad-4fb7-95a7-e052784292b5 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:24:46.972018 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-61a7fceb-b1ad-4fb7-95a7-e052784292b5 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] [instance: 867b2ab0-9301-414b-b045-85057d642971] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:24:46.995641 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-afe8e1d4-d9ff-4baa-8921-3b14115a1ce9 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:24:46.996274 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-afe8e1d4-d9ff-4baa-8921-3b14115a1ce9 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: c9901d0a-8a17-4e6c-ba98-6bd587d410e0] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:24:47.006904 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-61a7fceb-b1ad-4fb7-95a7-e052784292b5 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:47.015644 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-afe8e1d4-d9ff-4baa-8921-3b14115a1ce9 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:47.016112 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-afe8e1d4-d9ff-4baa-8921-3b14115a1ce9 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:47.016914 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-afe8e1d4-d9ff-4baa-8921-3b14115a1ce9 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:47.017367 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-afe8e1d4-d9ff-4baa-8921-3b14115a1ce9 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:47.018878 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-61a7fceb-b1ad-4fb7-95a7-e052784292b5 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:47.026048 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-61a7fceb-b1ad-4fb7-95a7-e052784292b5 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:47.026628 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-61a7fceb-b1ad-4fb7-95a7-e052784292b5 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:47.476736 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell0:SELECT=12 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:24:47.680168 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-afe8e1d4-d9ff-4baa-8921-3b14115a1ce9 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '027dcc48e4fd4c36a5004096ad5e43f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dad823e240149edb30e592380249a13', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:24:47.691336 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-afe8e1d4-d9ff-4baa-8921-3b14115a1ce9 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "GET /compute/v2.1/servers/c9901d0a-8a17-4e6c-ba98-6bd587d410e0" status: 200 len: 1328 microversion: 2.1 time: 0.701382 Jul 16 21:24:47.694404 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-61a7fceb-b1ad-4fb7-95a7-e052784292b5 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcc9aa2a091c490599f9a9c41163e2eb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da56fc78f06142aba80ca4c6e3d60475', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:24:47.704354 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1728/3461] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:24:46 2022] GET /compute/v2.1/servers/c9901d0a-8a17-4e6c-ba98-6bd587d410e0 => generated 1328 bytes in 715 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:24:47.706053 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-61a7fceb-b1ad-4fb7-95a7-e052784292b5 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] 10.222.0.16 "GET /compute/v2.1/servers/867b2ab0-9301-414b-b045-85057d642971" status: 200 len: 1502 microversion: 2.1 time: 0.751694 Jul 16 21:24:47.720891 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1734/3462] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:24:46 2022] GET /compute/v2.1/servers/867b2ab0-9301-414b-b045-85057d642971 => generated 1502 bytes in 767 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:24:47.729282 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-13cba3a2-b927-4b20-afca-f4464d7fb918 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:24:47.730673 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-13cba3a2-b927-4b20-afca-f4464d7fb918 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] [instance: 92fbfe5e-07b4-44ec-9729-1a9230f4c02c] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:24:47.739355 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-13cba3a2-b927-4b20-afca-f4464d7fb918 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:47.740930 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-13cba3a2-b927-4b20-afca-f4464d7fb918 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:47.742390 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-13cba3a2-b927-4b20-afca-f4464d7fb918 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:47.743135 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-13cba3a2-b927-4b20-afca-f4464d7fb918 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:47.933010 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-00b65b55-b6c0-448f-bd13-cad487580a7d tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ImagesOneServerTestJSON-server-900223469", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452"}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:24:48.023258 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.network.neutron [None req-00b65b55-b6c0-448f-bd13-cad487580a7d tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] validate_networks() for None {{(pid=138681) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 16 21:24:48.195594 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-13cba3a2-b927-4b20-afca-f4464d7fb918 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcc9aa2a091c490599f9a9c41163e2eb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da56fc78f06142aba80ca4c6e3d60475', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:24:48.214845 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-13cba3a2-b927-4b20-afca-f4464d7fb918 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] 10.222.0.16 "GET /compute/v2.1/servers/92fbfe5e-07b4-44ec-9729-1a9230f4c02c" status: 200 len: 1517 microversion: 2.1 time: 0.491744 Jul 16 21:24:48.218086 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1735/3463] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:24:47 2022] GET /compute/v2.1/servers/92fbfe5e-07b4-44ec-9729-1a9230f4c02c => generated 1517 bytes in 496 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:24:48.225058 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-a97b00f3-87b9-4e56-b17e-0369c1b5f075 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:24:48.226203 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-a97b00f3-87b9-4e56-b17e-0369c1b5f075 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 8e6eab25-f25e-46a2-9c72-9424c872fb84] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:24:48.236631 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a97b00f3-87b9-4e56-b17e-0369c1b5f075 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:48.237211 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a97b00f3-87b9-4e56-b17e-0369c1b5f075 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:48.238056 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a97b00f3-87b9-4e56-b17e-0369c1b5f075 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:48.246088 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a97b00f3-87b9-4e56-b17e-0369c1b5f075 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:48.896707 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-00b65b55-b6c0-448f-bd13-cad487580a7d tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Flavor limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 16 21:24:48.900469 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-00b65b55-b6c0-448f-bd13-cad487580a7d tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Image limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 16 21:24:48.900469 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-00b65b55-b6c0-448f-bd13-cad487580a7d tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Flavor pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 16 21:24:48.900469 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-00b65b55-b6c0-448f-bd13-cad487580a7d tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Image pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 16 21:24:48.900469 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-00b65b55-b6c0-448f-bd13-cad487580a7d tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 16 21:24:48.923865 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-00b65b55-b6c0-448f-bd13-cad487580a7d tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:48.924331 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-00b65b55-b6c0-448f-bd13-cad487580a7d tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:48.925156 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-00b65b55-b6c0-448f-bd13-cad487580a7d tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:48.925602 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-00b65b55-b6c0-448f-bd13-cad487580a7d tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:48.954011 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_api:SELECT=96,nova_api:INSERT=4 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:24:48.969349 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-00b65b55-b6c0-448f-bd13-cad487580a7d tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Getting quotas for project 336b3ef8a8744bf3b86854b32000bda6. Resources: {'ram', 'instances', 'cores'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:24:48.978654 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-00b65b55-b6c0-448f-bd13-cad487580a7d tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Getting quotas for user 3b0b46c0c1f74adaaaf65d46db7b2618 and project 336b3ef8a8744bf3b86854b32000bda6. Resources: {'ram', 'instances', 'cores'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:24:49.008427 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-a97b00f3-87b9-4e56-b17e-0369c1b5f075 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f2e974ab5aa48dfb4b00f810dcedb63', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd31be8475a6a49d6bfe8132dd050c6c0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:24:49.026498 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-a97b00f3-87b9-4e56-b17e-0369c1b5f075 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "GET /compute/v2.1/servers/8e6eab25-f25e-46a2-9c72-9424c872fb84" status: 200 len: 1523 microversion: 2.1 time: 0.806645 Jul 16 21:24:49.028209 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1736/3464] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:24:48 2022] GET /compute/v2.1/servers/8e6eab25-f25e-46a2-9c72-9424c872fb84 => generated 1523 bytes in 809 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:24:49.031369 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-00b65b55-b6c0-448f-bd13-cad487580a7d tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Going to run 1 instances... {{(pid=138681) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 16 21:24:49.039789 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-00b65b55-b6c0-448f-bd13-cad487580a7d tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] [instance: f00bea08-4180-45d6-ba01-4b104f5615d6] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138681) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 16 21:24:49.052751 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-9625f736-7afb-44b7-8f72-61daf5b80035 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Action: 'action', calling method: >, body: {"rescue": {}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:24:49.054215 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-9625f736-7afb-44b7-8f72-61daf5b80035 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] [instance: dc30c0e0-fc83-47a9-aca2-97d8c95e9ecc] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:24:49.072608 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-9625f736-7afb-44b7-8f72-61daf5b80035 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] HTTP exception thrown: Instance dc30c0e0-fc83-47a9-aca2-97d8c95e9ecc could not be found. Jul 16 21:24:49.077833 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-9625f736-7afb-44b7-8f72-61daf5b80035 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Request method failure captured: Jul 16 21:24:49.077833 ncdv-master-1 devstack@n-api.service[138680]: request: POST /compute/v2.1/servers/dc30c0e0-fc83-47a9-aca2-97d8c95e9ecc/action HTTP/1.1 Jul 16 21:24:49.077833 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:24:49.077833 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:24:49.077833 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:24:49.077833 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 14 Jul 16 21:24:49.077833 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:24:49.077833 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:24:49.077833 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:24:49.077833 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:24:49.077833 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0yxlBvXdj6P284bimRWVNOZAMmAVjIOasTJ1njzyaS12Nm5AKzVbxoKHi72FFVSaWe28lVKwWFtzjqUTy3NQWDNW-obFHfOCoFCCR4GNQ-_ruXI6NnMsKND6oz7nIdWgqUgB4Gac48-MGLdaneUoqvnHJrpsJjhWmKgGoKbdR8M7y0I Jul 16 21:24:49.077833 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:24:49.077833 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:24:49.077833 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:24:49.077833 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:24:49.077833 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:24:49.077833 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:24:49.077833 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: da56fc78f06142aba80ca4c6e3d60475 Jul 16 21:24:49.077833 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-439221844 Jul 16 21:24:49.077833 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:24:49.077833 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:24:49.077833 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/da56fc78f06142aba80ca4c6e3d60475"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/da56fc78f06142aba80ca4c6e3d60475"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_da56fc78f06142aba80ca4c6e3d60475"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/da56fc78f06142aba80ca4c6e3d60475"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:24:49.077833 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-ServerRescueNegativeTestJSON-439221844 Jul 16 21:24:49.077833 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: da56fc78f06142aba80ca4c6e3d60475 Jul 16 21:24:49.077833 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-439221844 Jul 16 21:24:49.077833 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-ServerRescueNegativeTestJSON-439221844-project Jul 16 21:24:49.077833 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:24:49.077833 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:24:49.077833 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: bcc9aa2a091c490599f9a9c41163e2eb Jul 16 21:24:49.077833 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-ServerRescueNegativeTestJSON-439221844-project Jul 16 21:24:49.077833 ncdv-master-1 devstack@n-api.service[138680]: Jul 16 21:24:49.077833 ncdv-master-1 devstack@n-api.service[138680]: {"rescue": {}} Jul 16 21:24:49.077833 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 21:24:49.077833 ncdv-master-1 devstack@n-api.service[138680]: exception: Instance dc30c0e0-fc83-47a9-aca2-97d8c95e9ecc could not be found. Jul 16 21:24:49.077833 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': 'dc30c0e0-fc83-47a9-aca2-97d8c95e9ecc', 'body': {'rescue': {}}} Jul 16 21:24:49.077833 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:24:49.077833 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:24:49.077833 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:24:49.077833 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:24:49.077833 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:24:49.077833 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:24:49.077833 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2898, in _get_instance Jul 16 21:24:49.077833 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=instance_uuid) Jul 16 21:24:49.077833 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance dc30c0e0-fc83-47a9-aca2-97d8c95e9ecc could not be found. Jul 16 21:24:49.077833 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:24:49.077833 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:24:49.077833 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:24:49.077833 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:24:49.077833 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:24:49.077833 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:24:49.077833 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:24:49.077833 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:24:49.077833 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:24:49.077833 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:24:49.077833 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 16 21:24:49.077833 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 16 21:24:49.077833 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/rescue.py", line 53, in _rescue Jul 16 21:24:49.077833 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, id) Jul 16 21:24:49.077833 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:24:49.077833 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:24:49.077833 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance dc30c0e0-fc83-47a9-aca2-97d8c95e9ecc could not be found. Jul 16 21:24:49.077833 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:24:49.077833 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:24:49.077833 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:24:49.077833 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:24:49.083366 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:24:49.083366 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:24:49.083366 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:24:49.083366 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:24:49.083366 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance dc30c0e0-fc83-47a9-aca2-97d8c95e9ecc could not be found. Jul 16 21:24:49.083366 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:24:49.085174 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-9625f736-7afb-44b7-8f72-61daf5b80035 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Returning 404 to user: Instance dc30c0e0-fc83-47a9-aca2-97d8c95e9ecc could not be found. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:24:49.086296 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-9625f736-7afb-44b7-8f72-61daf5b80035 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] 10.222.0.16 "POST /compute/v2.1/servers/dc30c0e0-fc83-47a9-aca2-97d8c95e9ecc/action" status: 404 len: 111 microversion: 2.1 time: 0.052328 Jul 16 21:24:49.092804 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1737/3465] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 21:24:49 2022] POST /compute/v2.1/servers/dc30c0e0-fc83-47a9-aca2-97d8c95e9ecc/action => generated 111 bytes in 61 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:24:49.111862 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-769cf170-6c09-4d66-888a-ac2fc0972510 req-6db46b7b-45ac-4829-9231-18f47103e385 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "c9901d0a-8a17-4e6c-ba98-6bd587d410e0", "tag": "443db70b-054d-4606-9eee-d742416cd5a6"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:24:49.130808 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-769cf170-6c09-4d66-888a-ac2fc0972510 req-6db46b7b-45ac-4829-9231-18f47103e385 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:49.131308 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-769cf170-6c09-4d66-888a-ac2fc0972510 req-6db46b7b-45ac-4829-9231-18f47103e385 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:49.147361 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-769cf170-6c09-4d66-888a-ac2fc0972510 req-6db46b7b-45ac-4829-9231-18f47103e385 service nova] Creating event network-changed:443db70b-054d-4606-9eee-d742416cd5a6 for instance c9901d0a-8a17-4e6c-ba98-6bd587d410e0 on nch2-master-1 Jul 16 21:24:49.153099 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-769cf170-6c09-4d66-888a-ac2fc0972510 req-6db46b7b-45ac-4829-9231-18f47103e385 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.057262 Jul 16 21:24:49.154820 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1738/3466] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:24:49 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 61 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:24:49.164137 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-ee798404-0386-46bb-a0c6-d9d7198678f5 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:24:49.165144 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-ee798404-0386-46bb-a0c6-d9d7198678f5 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: c9901d0a-8a17-4e6c-ba98-6bd587d410e0] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:24:49.173093 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ee798404-0386-46bb-a0c6-d9d7198678f5 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:49.173891 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ee798404-0386-46bb-a0c6-d9d7198678f5 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:49.174877 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ee798404-0386-46bb-a0c6-d9d7198678f5 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:49.175568 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ee798404-0386-46bb-a0c6-d9d7198678f5 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:49.212900 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-00b65b55-b6c0-448f-bd13-cad487580a7d tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 1.505304 Jul 16 21:24:49.213566 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1729/3467] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 21:24:47 2022] POST /compute/v2.1/servers => generated 380 bytes in 1509 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 16 21:24:49.226565 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-92ac1163-096e-4777-a99e-b5ccaffdbd15 req-8c909b9c-2a75-421f-8e02-eab15162197b service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "2fad683a-f21f-4a9c-8323-e9380ef7f9ef", "tag": "aff74b78-59ed-4a31-9d5b-3c3e6aa5d30c"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:24:49.264756 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-92ac1163-096e-4777-a99e-b5ccaffdbd15 req-8c909b9c-2a75-421f-8e02-eab15162197b service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:49.264756 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-92ac1163-096e-4777-a99e-b5ccaffdbd15 req-8c909b9c-2a75-421f-8e02-eab15162197b service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:49.295307 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-92ac1163-096e-4777-a99e-b5ccaffdbd15 req-8c909b9c-2a75-421f-8e02-eab15162197b service nova] Creating event network-changed:aff74b78-59ed-4a31-9d5b-3c3e6aa5d30c for instance 2fad683a-f21f-4a9c-8323-e9380ef7f9ef on nch1-master-1 Jul 16 21:24:49.305673 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-92ac1163-096e-4777-a99e-b5ccaffdbd15 req-8c909b9c-2a75-421f-8e02-eab15162197b service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.088447 Jul 16 21:24:49.308994 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1730/3468] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:24:49 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:24:49.321202 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-58ef3ffc-136e-41a6-94d7-3c6187ce204e tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Action: 'action', calling method: >, body: {"pause": {}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:24:49.326004 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-58ef3ffc-136e-41a6-94d7-3c6187ce204e tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] [instance: 92fbfe5e-07b4-44ec-9729-1a9230f4c02c] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:24:49.336843 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-58ef3ffc-136e-41a6-94d7-3c6187ce204e tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:49.339939 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-58ef3ffc-136e-41a6-94d7-3c6187ce204e tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:49.341005 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-58ef3ffc-136e-41a6-94d7-3c6187ce204e tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:49.342859 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-58ef3ffc-136e-41a6-94d7-3c6187ce204e tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:49.659837 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-58ef3ffc-136e-41a6-94d7-3c6187ce204e tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] 10.222.0.16 "POST /compute/v2.1/servers/92fbfe5e-07b4-44ec-9729-1a9230f4c02c/action" status: 202 len: 0 microversion: 2.1 time: 0.349265 Jul 16 21:24:49.662557 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1731/3469] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 21:24:49 2022] POST /compute/v2.1/servers/92fbfe5e-07b4-44ec-9729-1a9230f4c02c/action => generated 0 bytes in 351 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 21:24:49.669862 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-8f3cf335-584d-4136-9b14-5e2eaa2dcee6 tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:24:49.673402 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-8f3cf335-584d-4136-9b14-5e2eaa2dcee6 tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] [instance: f00bea08-4180-45d6-ba01-4b104f5615d6] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:24:49.987493 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-8f3cf335-584d-4136-9b14-5e2eaa2dcee6 tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3b0b46c0c1f74adaaaf65d46db7b2618', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '336b3ef8a8744bf3b86854b32000bda6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:24:50.011748 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-8f3cf335-584d-4136-9b14-5e2eaa2dcee6 tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] 10.222.0.16 "GET /compute/v2.1/servers/f00bea08-4180-45d6-ba01-4b104f5615d6" status: 200 len: 1216 microversion: 2.1 time: 0.348177 Jul 16 21:24:50.014098 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1732/3470] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:24:49 2022] GET /compute/v2.1/servers/f00bea08-4180-45d6-ba01-4b104f5615d6 => generated 1216 bytes in 353 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:24:50.069005 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-e96c98ab-e13a-44cd-a433-dca97b0ef5c0 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:24:50.073532 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-e96c98ab-e13a-44cd-a433-dca97b0ef5c0 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] [instance: 92fbfe5e-07b4-44ec-9729-1a9230f4c02c] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:24:50.096823 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e96c98ab-e13a-44cd-a433-dca97b0ef5c0 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:50.105453 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e96c98ab-e13a-44cd-a433-dca97b0ef5c0 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:50.106532 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e96c98ab-e13a-44cd-a433-dca97b0ef5c0 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:50.107121 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e96c98ab-e13a-44cd-a433-dca97b0ef5c0 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:50.297342 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-ee798404-0386-46bb-a0c6-d9d7198678f5 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '027dcc48e4fd4c36a5004096ad5e43f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dad823e240149edb30e592380249a13', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:24:50.311100 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-ee798404-0386-46bb-a0c6-d9d7198678f5 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "GET /compute/v2.1/servers/c9901d0a-8a17-4e6c-ba98-6bd587d410e0" status: 200 len: 1328 microversion: 2.1 time: 1.154626 Jul 16 21:24:50.323204 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1739/3471] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:24:49 2022] GET /compute/v2.1/servers/c9901d0a-8a17-4e6c-ba98-6bd587d410e0 => generated 1328 bytes in 1167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:24:50.330429 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-69243fec-1dd5-4f36-8f5e-d94c3c45b319 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:24:50.331531 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-69243fec-1dd5-4f36-8f5e-d94c3c45b319 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 8e6eab25-f25e-46a2-9c72-9424c872fb84] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:24:50.354388 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-69243fec-1dd5-4f36-8f5e-d94c3c45b319 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:50.355635 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-69243fec-1dd5-4f36-8f5e-d94c3c45b319 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:50.357558 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-69243fec-1dd5-4f36-8f5e-d94c3c45b319 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:50.358112 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-69243fec-1dd5-4f36-8f5e-d94c3c45b319 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:50.567275 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-e96c98ab-e13a-44cd-a433-dca97b0ef5c0 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcc9aa2a091c490599f9a9c41163e2eb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da56fc78f06142aba80ca4c6e3d60475', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:24:50.579085 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-e96c98ab-e13a-44cd-a433-dca97b0ef5c0 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] 10.222.0.16 "GET /compute/v2.1/servers/92fbfe5e-07b4-44ec-9729-1a9230f4c02c" status: 200 len: 1522 microversion: 2.1 time: 0.528525 Jul 16 21:24:50.579695 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1733/3472] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:24:50 2022] GET /compute/v2.1/servers/92fbfe5e-07b4-44ec-9729-1a9230f4c02c => generated 1522 bytes in 531 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:24:50.720345 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-69243fec-1dd5-4f36-8f5e-d94c3c45b319 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f2e974ab5aa48dfb4b00f810dcedb63', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd31be8475a6a49d6bfe8132dd050c6c0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:24:50.731746 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=100 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:24:50.737593 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-69243fec-1dd5-4f36-8f5e-d94c3c45b319 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "GET /compute/v2.1/servers/8e6eab25-f25e-46a2-9c72-9424c872fb84" status: 200 len: 1523 microversion: 2.1 time: 0.412906 Jul 16 21:24:50.738178 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1740/3473] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:24:50 2022] GET /compute/v2.1/servers/8e6eab25-f25e-46a2-9c72-9424c872fb84 => generated 1523 bytes in 414 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:24:51.047339 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-ecf37aa3-d59b-4363-a776-9828fecb99a9 tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:24:51.048260 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-ecf37aa3-d59b-4363-a776-9828fecb99a9 tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] [instance: f00bea08-4180-45d6-ba01-4b104f5615d6] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:24:51.187418 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-ecf37aa3-d59b-4363-a776-9828fecb99a9 tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3b0b46c0c1f74adaaaf65d46db7b2618', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '336b3ef8a8744bf3b86854b32000bda6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:24:51.197337 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-ecf37aa3-d59b-4363-a776-9828fecb99a9 tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] 10.222.0.16 "GET /compute/v2.1/servers/f00bea08-4180-45d6-ba01-4b104f5615d6" status: 200 len: 1216 microversion: 2.1 time: 0.155552 Jul 16 21:24:51.201541 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1734/3474] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:24:51 2022] GET /compute/v2.1/servers/f00bea08-4180-45d6-ba01-4b104f5615d6 => generated 1216 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:24:51.329589 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-1ccab498-2e00-48cd-b28d-672e3933d9d9 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:24:51.330428 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-1ccab498-2e00-48cd-b28d-672e3933d9d9 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: c9901d0a-8a17-4e6c-ba98-6bd587d410e0] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:24:51.340630 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1ccab498-2e00-48cd-b28d-672e3933d9d9 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:51.342553 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1ccab498-2e00-48cd-b28d-672e3933d9d9 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:51.344414 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1ccab498-2e00-48cd-b28d-672e3933d9d9 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:51.346672 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1ccab498-2e00-48cd-b28d-672e3933d9d9 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:51.469274 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-c6df4a88-bc09-479f-b0cf-d5e47766b27f tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-device-tagging-server-402975455", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452", "config_drive": true, "networks": [{"uuid": "9e60117b-2f38-43c2-826d-b414daf6bb38"}, {"port": "02eaaf37-d146-4778-a757-4ba313f73f8b", "tag": "port-1"}, {"port": "178c5641-50ae-4f11-97c8-41f1bfa01abe", "tag": "port-2"}, {"uuid": "a5dde854-7dfc-4d3f-8ecf-37c8ff9f32ce", "tag": "net-1"}, {"uuid": "a5dde854-7dfc-4d3f-8ecf-37c8ff9f32ce"}, {"uuid": "37673a28-61b8-4603-9c1e-e82b92aba310", "fixed_ip": "10.2.2.100", "tag": "net-2-100"}, {"uuid": "37673a28-61b8-4603-9c1e-e82b92aba310", "fixed_ip": "10.2.2.200", "tag": "net-2-200"}], "block_device_mapping_v2": [{"uuid": "6c632192-292e-476a-92f7-a3a26a1725e6", "source_type": "volume", "destination_type": "volume", "boot_index": 0, "tag": "boot"}, {"uuid": "83999d90-80b2-4c0c-8aff-bec74809b4d9", "source_type": "volume", "destination_type": "volume", "boot_index": 1, "tag": "other"}, {"uuid": "f0883a5a-4e31-4e72-9db6-7d56ee50023e", "source_type": "volume", "destination_type": "volume", "boot_index": 2}], "security_groups": [{"name": "tempest-securitygroup--801515692"}], "key_name": "tempest-keypair-1847534343", "user_data": "IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIHVidW50dSB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH51YnVudHUvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo="}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:24:52.043026 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-1ccab498-2e00-48cd-b28d-672e3933d9d9 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '027dcc48e4fd4c36a5004096ad5e43f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dad823e240149edb30e592380249a13', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:24:52.053614 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-1ccab498-2e00-48cd-b28d-672e3933d9d9 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "GET /compute/v2.1/servers/c9901d0a-8a17-4e6c-ba98-6bd587d410e0" status: 200 len: 1328 microversion: 2.1 time: 0.728614 Jul 16 21:24:52.059821 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1741/3475] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:24:51 2022] GET /compute/v2.1/servers/c9901d0a-8a17-4e6c-ba98-6bd587d410e0 => generated 1328 bytes in 735 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:24:52.064913 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-f73e6fe4-af0f-4a5b-908c-07605b3214a3 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:24:52.068790 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.network.neutron [None req-c6df4a88-bc09-479f-b0cf-d5e47766b27f tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] validate_networks() for [('9e60117b-2f38-43c2-826d-b414daf6bb38', None, None, None, None, None), (None, None, '02eaaf37-d146-4778-a757-4ba313f73f8b', None, None, None), (None, None, '178c5641-50ae-4f11-97c8-41f1bfa01abe', None, None, None), ('a5dde854-7dfc-4d3f-8ecf-37c8ff9f32ce', None, None, None, None, None), ('a5dde854-7dfc-4d3f-8ecf-37c8ff9f32ce', None, None, None, None, None), ('37673a28-61b8-4603-9c1e-e82b92aba310', '10.2.2.100', None, None, None, None), ('37673a28-61b8-4603-9c1e-e82b92aba310', '10.2.2.200', None, None, None, None)] {{(pid=138681) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 16 21:24:52.081093 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-f73e6fe4-af0f-4a5b-908c-07605b3214a3 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] [instance: 92fbfe5e-07b4-44ec-9729-1a9230f4c02c] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:24:52.094429 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f73e6fe4-af0f-4a5b-908c-07605b3214a3 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:52.101070 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f73e6fe4-af0f-4a5b-908c-07605b3214a3 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:52.102039 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f73e6fe4-af0f-4a5b-908c-07605b3214a3 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:52.102764 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f73e6fe4-af0f-4a5b-908c-07605b3214a3 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:52.385434 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-f73e6fe4-af0f-4a5b-908c-07605b3214a3 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcc9aa2a091c490599f9a9c41163e2eb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da56fc78f06142aba80ca4c6e3d60475', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:24:52.394076 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-f73e6fe4-af0f-4a5b-908c-07605b3214a3 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] 10.222.0.16 "GET /compute/v2.1/servers/92fbfe5e-07b4-44ec-9729-1a9230f4c02c" status: 200 len: 1502 microversion: 2.1 time: 0.332878 Jul 16 21:24:52.416364 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1742/3476] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:24:52 2022] GET /compute/v2.1/servers/92fbfe5e-07b4-44ec-9729-1a9230f4c02c => generated 1502 bytes in 355 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:24:52.423452 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-ce265cf4-d541-44b7-910e-723a117d28ec tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:24:52.431470 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-ce265cf4-d541-44b7-910e-723a117d28ec tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 8e6eab25-f25e-46a2-9c72-9424c872fb84] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:24:52.446424 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ce265cf4-d541-44b7-910e-723a117d28ec tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:52.447937 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ce265cf4-d541-44b7-910e-723a117d28ec tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:52.449036 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ce265cf4-d541-44b7-910e-723a117d28ec tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:52.449665 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ce265cf4-d541-44b7-910e-723a117d28ec tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:52.804792 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-ce265cf4-d541-44b7-910e-723a117d28ec tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f2e974ab5aa48dfb4b00f810dcedb63', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd31be8475a6a49d6bfe8132dd050c6c0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:24:52.819655 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-ce265cf4-d541-44b7-910e-723a117d28ec tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "GET /compute/v2.1/servers/8e6eab25-f25e-46a2-9c72-9424c872fb84" status: 200 len: 1523 microversion: 2.1 time: 0.400730 Jul 16 21:24:52.821007 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1743/3477] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:24:52 2022] GET /compute/v2.1/servers/8e6eab25-f25e-46a2-9c72-9424c872fb84 => generated 1523 bytes in 402 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:24:52.833821 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-a55e28a3-7e74-488a-9f31-e9be0e40d58e tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:24:52.835779 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-a55e28a3-7e74-488a-9f31-e9be0e40d58e tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] [instance: f00bea08-4180-45d6-ba01-4b104f5615d6] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:24:52.848539 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a55e28a3-7e74-488a-9f31-e9be0e40d58e tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:52.849288 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a55e28a3-7e74-488a-9f31-e9be0e40d58e tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:52.850834 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a55e28a3-7e74-488a-9f31-e9be0e40d58e tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:52.851598 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a55e28a3-7e74-488a-9f31-e9be0e40d58e tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:53.075771 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-a55e28a3-7e74-488a-9f31-e9be0e40d58e tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3b0b46c0c1f74adaaaf65d46db7b2618', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '336b3ef8a8744bf3b86854b32000bda6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:24:53.089995 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-a55e28a3-7e74-488a-9f31-e9be0e40d58e tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] 10.222.0.16 "GET /compute/v2.1/servers/f00bea08-4180-45d6-ba01-4b104f5615d6" status: 200 len: 1212 microversion: 2.1 time: 0.263258 Jul 16 21:24:53.097982 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1744/3478] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:24:52 2022] GET /compute/v2.1/servers/f00bea08-4180-45d6-ba01-4b104f5615d6 => generated 1212 bytes in 277 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:24:53.120008 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-1d3ced54-42cb-488b-ab0a-2350625290e4 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Action: 'action', calling method: >, body: {"rescue": {}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:24:53.125540 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-1d3ced54-42cb-488b-ab0a-2350625290e4 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] [instance: 92fbfe5e-07b4-44ec-9729-1a9230f4c02c] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:24:53.140489 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1d3ced54-42cb-488b-ab0a-2350625290e4 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:53.142774 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1d3ced54-42cb-488b-ab0a-2350625290e4 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:53.146983 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1d3ced54-42cb-488b-ab0a-2350625290e4 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:53.148193 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1d3ced54-42cb-488b-ab0a-2350625290e4 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:53.230223 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.objects.instance [None req-1d3ced54-42cb-488b-ab0a-2350625290e4 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lazy-loading 'flavor' on Instance uuid 92fbfe5e-07b4-44ec-9729-1a9230f4c02c {{(pid=138680) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:24:53.340026 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-1d3ced54-42cb-488b-ab0a-2350625290e4 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] HTTP exception thrown: Cannot 'rescue' instance 92fbfe5e-07b4-44ec-9729-1a9230f4c02c while it is in vm_state paused Jul 16 21:24:53.343486 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-1d3ced54-42cb-488b-ab0a-2350625290e4 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Request method failure captured: Jul 16 21:24:53.343486 ncdv-master-1 devstack@n-api.service[138680]: request: POST /compute/v2.1/servers/92fbfe5e-07b4-44ec-9729-1a9230f4c02c/action HTTP/1.1 Jul 16 21:24:53.343486 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:24:53.343486 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:24:53.343486 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:24:53.343486 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 14 Jul 16 21:24:53.343486 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:24:53.343486 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:24:53.343486 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:24:53.343486 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:24:53.343486 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0yxlBvXdj6P284bimRWVNOZAMmAVjIOasTJ1njzyaS12Nm5AKzVbxoKHi72FFVSaWe28lVKwWFtzjqUTy3NQWDNW-obFHfOCoFCCR4GNQ-_ruXI6NnMsKND6oz7nIdWgqUgB4Gac48-MGLdaneUoqvnHJrpsJjhWmKgGoKbdR8M7y0I Jul 16 21:24:53.343486 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:24:53.343486 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:24:53.343486 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:24:53.343486 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:24:53.343486 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:24:53.343486 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:24:53.343486 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: da56fc78f06142aba80ca4c6e3d60475 Jul 16 21:24:53.343486 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-439221844 Jul 16 21:24:53.343486 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:24:53.343486 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:24:53.343486 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/da56fc78f06142aba80ca4c6e3d60475"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/da56fc78f06142aba80ca4c6e3d60475"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_da56fc78f06142aba80ca4c6e3d60475"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/da56fc78f06142aba80ca4c6e3d60475"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:24:53.343486 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-ServerRescueNegativeTestJSON-439221844 Jul 16 21:24:53.343486 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: da56fc78f06142aba80ca4c6e3d60475 Jul 16 21:24:53.343486 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-439221844 Jul 16 21:24:53.343486 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-ServerRescueNegativeTestJSON-439221844-project Jul 16 21:24:53.343486 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:24:53.343486 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:24:53.343486 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: bcc9aa2a091c490599f9a9c41163e2eb Jul 16 21:24:53.343486 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-ServerRescueNegativeTestJSON-439221844-project Jul 16 21:24:53.343486 ncdv-master-1 devstack@n-api.service[138680]: Jul 16 21:24:53.343486 ncdv-master-1 devstack@n-api.service[138680]: {"rescue": {}} Jul 16 21:24:53.343486 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 21:24:53.343486 ncdv-master-1 devstack@n-api.service[138680]: exception: Cannot 'rescue' instance 92fbfe5e-07b4-44ec-9729-1a9230f4c02c while it is in vm_state paused Jul 16 21:24:53.343486 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': '92fbfe5e-07b4-44ec-9729-1a9230f4c02c', 'body': {'rescue': {}}} Jul 16 21:24:53.343486 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:24:53.343486 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:24:53.343486 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/rescue.py", line 62, in _rescue Jul 16 21:24:53.343486 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi self.compute_api.rescue(context, instance, Jul 16 21:24:53.343486 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 275, in inner Jul 16 21:24:53.343486 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Jul 16 21:24:53.343486 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner Jul 16 21:24:53.343486 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Jul 16 21:24:53.343486 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 145, in inner Jul 16 21:24:53.343486 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.InstanceInvalidState( Jul 16 21:24:53.343486 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.InstanceInvalidState: Instance 92fbfe5e-07b4-44ec-9729-1a9230f4c02c in vm_state paused. Cannot rescue while the instance is in this state. Jul 16 21:24:53.343486 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:24:53.343486 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:24:53.343486 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:24:53.343486 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:24:53.343486 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:24:53.343486 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:24:53.343486 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:24:53.343486 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:24:53.343486 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:24:53.343486 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:24:53.343486 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 16 21:24:53.343486 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 16 21:24:53.343486 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/rescue.py", line 72, in _rescue Jul 16 21:24:53.343486 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi common.raise_http_conflict_for_instance_invalid_state( Jul 16 21:24:53.343486 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 347, in raise_http_conflict_for_instance_invalid_state Jul 16 21:24:53.343486 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPConflict(explanation=msg) Jul 16 21:24:53.343486 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPConflict: Cannot 'rescue' instance 92fbfe5e-07b4-44ec-9729-1a9230f4c02c while it is in vm_state paused Jul 16 21:24:53.343486 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:24:53.349256 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:24:53.349256 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:24:53.349256 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:24:53.349256 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:24:53.349256 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:24:53.349256 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:24:53.349256 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:24:53.349256 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Cannot 'rescue' instance 92fbfe5e-07b4-44ec-9729-1a9230f4c02c while it is in vm_state paused Jul 16 21:24:53.349256 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:24:53.351716 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-1d3ced54-42cb-488b-ab0a-2350625290e4 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Returning 409 to user: Cannot 'rescue' instance 92fbfe5e-07b4-44ec-9729-1a9230f4c02c while it is in vm_state paused {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:24:53.357261 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-1d3ced54-42cb-488b-ab0a-2350625290e4 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] 10.222.0.16 "POST /compute/v2.1/servers/92fbfe5e-07b4-44ec-9729-1a9230f4c02c/action" status: 409 len: 144 microversion: 2.1 time: 0.254946 Jul 16 21:24:53.357261 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1745/3479] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 21:24:53 2022] POST /compute/v2.1/servers/92fbfe5e-07b4-44ec-9729-1a9230f4c02c/action => generated 144 bytes in 256 msecs (HTTP/1.1 409) 9 headers in 378 bytes (1 switches on core 0) Jul 16 21:24:53.362105 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-bb5147ee-6a63-4b7c-bee3-21b5c8e5460f tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:24:53.363556 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-bb5147ee-6a63-4b7c-bee3-21b5c8e5460f tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: c9901d0a-8a17-4e6c-ba98-6bd587d410e0] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:24:53.382913 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-bb5147ee-6a63-4b7c-bee3-21b5c8e5460f tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:53.383763 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-bb5147ee-6a63-4b7c-bee3-21b5c8e5460f tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:53.384874 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-bb5147ee-6a63-4b7c-bee3-21b5c8e5460f tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:53.386315 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-bb5147ee-6a63-4b7c-bee3-21b5c8e5460f tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:53.893695 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-bb5147ee-6a63-4b7c-bee3-21b5c8e5460f tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '027dcc48e4fd4c36a5004096ad5e43f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dad823e240149edb30e592380249a13', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:24:53.911304 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-bb5147ee-6a63-4b7c-bee3-21b5c8e5460f tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "GET /compute/v2.1/servers/c9901d0a-8a17-4e6c-ba98-6bd587d410e0" status: 200 len: 1328 microversion: 2.1 time: 0.554326 Jul 16 21:24:53.919266 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1746/3480] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:24:53 2022] GET /compute/v2.1/servers/c9901d0a-8a17-4e6c-ba98-6bd587d410e0 => generated 1328 bytes in 563 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:24:53.931434 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-c3302f26-05c4-46ab-b7b2-8fec18e6ff0c tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Action: 'action', calling method: >, body: {"unpause": {}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:24:53.932257 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-c3302f26-05c4-46ab-b7b2-8fec18e6ff0c tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] [instance: 92fbfe5e-07b4-44ec-9729-1a9230f4c02c] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:24:53.947645 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c3302f26-05c4-46ab-b7b2-8fec18e6ff0c tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:53.948611 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c3302f26-05c4-46ab-b7b2-8fec18e6ff0c tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:53.950920 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c3302f26-05c4-46ab-b7b2-8fec18e6ff0c tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:53.951588 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c3302f26-05c4-46ab-b7b2-8fec18e6ff0c tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:54.250151 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-c3302f26-05c4-46ab-b7b2-8fec18e6ff0c tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] 10.222.0.16 "POST /compute/v2.1/servers/92fbfe5e-07b4-44ec-9729-1a9230f4c02c/action" status: 202 len: 0 microversion: 2.1 time: 0.329066 Jul 16 21:24:54.251516 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1747/3481] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 21:24:53 2022] POST /compute/v2.1/servers/92fbfe5e-07b4-44ec-9729-1a9230f4c02c/action => generated 0 bytes in 331 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 21:24:54.284466 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-b76f245b-1655-48b7-9fdf-d2feac2b523a tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:24:54.288382 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-b76f245b-1655-48b7-9fdf-d2feac2b523a tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 8e6eab25-f25e-46a2-9c72-9424c872fb84] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:24:54.309831 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b76f245b-1655-48b7-9fdf-d2feac2b523a tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:54.310657 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b76f245b-1655-48b7-9fdf-d2feac2b523a tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:54.313580 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b76f245b-1655-48b7-9fdf-d2feac2b523a tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:54.314216 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b76f245b-1655-48b7-9fdf-d2feac2b523a tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:54.689813 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-b76f245b-1655-48b7-9fdf-d2feac2b523a tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f2e974ab5aa48dfb4b00f810dcedb63', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd31be8475a6a49d6bfe8132dd050c6c0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:24:54.701790 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-b76f245b-1655-48b7-9fdf-d2feac2b523a tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "GET /compute/v2.1/servers/8e6eab25-f25e-46a2-9c72-9424c872fb84" status: 200 len: 1523 microversion: 2.1 time: 0.445881 Jul 16 21:24:54.702739 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1748/3482] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:24:54 2022] GET /compute/v2.1/servers/8e6eab25-f25e-46a2-9c72-9424c872fb84 => generated 1523 bytes in 450 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:24:54.715704 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-728a7628-8d8c-4a59-b573-772252546269 tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:24:54.720901 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-728a7628-8d8c-4a59-b573-772252546269 tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] [instance: f00bea08-4180-45d6-ba01-4b104f5615d6] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:24:54.730525 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-728a7628-8d8c-4a59-b573-772252546269 tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:54.731757 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-728a7628-8d8c-4a59-b573-772252546269 tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:54.735997 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-728a7628-8d8c-4a59-b573-772252546269 tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:54.736573 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-728a7628-8d8c-4a59-b573-772252546269 tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:54.878414 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-728a7628-8d8c-4a59-b573-772252546269 tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3b0b46c0c1f74adaaaf65d46db7b2618', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '336b3ef8a8744bf3b86854b32000bda6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:24:54.894615 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-728a7628-8d8c-4a59-b573-772252546269 tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] 10.222.0.16 "GET /compute/v2.1/servers/f00bea08-4180-45d6-ba01-4b104f5615d6" status: 200 len: 1274 microversion: 2.1 time: 0.187565 Jul 16 21:24:54.895254 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1749/3483] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:24:54 2022] GET /compute/v2.1/servers/f00bea08-4180-45d6-ba01-4b104f5615d6 => generated 1274 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:24:54.901994 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-cf992439-bbb4-43cf-9535-16bb25acab2a tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:24:54.902753 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-cf992439-bbb4-43cf-9535-16bb25acab2a tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] [instance: 92fbfe5e-07b4-44ec-9729-1a9230f4c02c] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:24:54.911455 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cf992439-bbb4-43cf-9535-16bb25acab2a tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:54.911975 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cf992439-bbb4-43cf-9535-16bb25acab2a tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:54.912776 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cf992439-bbb4-43cf-9535-16bb25acab2a tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:54.913260 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cf992439-bbb4-43cf-9535-16bb25acab2a tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:55.255268 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-cf992439-bbb4-43cf-9535-16bb25acab2a tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcc9aa2a091c490599f9a9c41163e2eb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da56fc78f06142aba80ca4c6e3d60475', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:24:55.270368 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-cf992439-bbb4-43cf-9535-16bb25acab2a tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] 10.222.0.16 "GET /compute/v2.1/servers/92fbfe5e-07b4-44ec-9729-1a9230f4c02c" status: 200 len: 1517 microversion: 2.1 time: 0.367267 Jul 16 21:24:55.271190 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1750/3484] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:24:54 2022] GET /compute/v2.1/servers/92fbfe5e-07b4-44ec-9729-1a9230f4c02c => generated 1517 bytes in 375 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:24:55.290525 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-f066e1b5-174d-42c4-b7d0-16e08ffa0c56 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:24:55.296901 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-f066e1b5-174d-42c4-b7d0-16e08ffa0c56 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: c9901d0a-8a17-4e6c-ba98-6bd587d410e0] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:24:55.304958 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f066e1b5-174d-42c4-b7d0-16e08ffa0c56 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:55.305449 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f066e1b5-174d-42c4-b7d0-16e08ffa0c56 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:55.306830 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f066e1b5-174d-42c4-b7d0-16e08ffa0c56 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:55.307793 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f066e1b5-174d-42c4-b7d0-16e08ffa0c56 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:55.851155 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-f066e1b5-174d-42c4-b7d0-16e08ffa0c56 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '027dcc48e4fd4c36a5004096ad5e43f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dad823e240149edb30e592380249a13', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:24:55.878936 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-f066e1b5-174d-42c4-b7d0-16e08ffa0c56 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "GET /compute/v2.1/servers/c9901d0a-8a17-4e6c-ba98-6bd587d410e0" status: 200 len: 1328 microversion: 2.1 time: 0.594178 Jul 16 21:24:55.891066 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1751/3485] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:24:55 2022] GET /compute/v2.1/servers/c9901d0a-8a17-4e6c-ba98-6bd587d410e0 => generated 1328 bytes in 608 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:24:55.891066 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-4a2f5aaa-b40c-40e5-973f-f6ccd8a60151 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:24:55.891832 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-4a2f5aaa-b40c-40e5-973f-f6ccd8a60151 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 8e6eab25-f25e-46a2-9c72-9424c872fb84] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:24:55.898933 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4a2f5aaa-b40c-40e5-973f-f6ccd8a60151 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:55.899513 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4a2f5aaa-b40c-40e5-973f-f6ccd8a60151 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:55.900498 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4a2f5aaa-b40c-40e5-973f-f6ccd8a60151 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:55.900950 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4a2f5aaa-b40c-40e5-973f-f6ccd8a60151 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:56.288581 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-4a2f5aaa-b40c-40e5-973f-f6ccd8a60151 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f2e974ab5aa48dfb4b00f810dcedb63', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd31be8475a6a49d6bfe8132dd050c6c0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:24:56.312101 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-4a2f5aaa-b40c-40e5-973f-f6ccd8a60151 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "GET /compute/v2.1/servers/8e6eab25-f25e-46a2-9c72-9424c872fb84" status: 200 len: 1523 microversion: 2.1 time: 0.428164 Jul 16 21:24:56.326093 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1752/3486] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:24:55 2022] GET /compute/v2.1/servers/8e6eab25-f25e-46a2-9c72-9424c872fb84 => generated 1523 bytes in 445 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:24:56.334722 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-a4830ef3-5f84-4249-b8ae-692071f3f3b4 tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:24:56.337382 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-a4830ef3-5f84-4249-b8ae-692071f3f3b4 tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] [instance: f00bea08-4180-45d6-ba01-4b104f5615d6] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:24:56.347393 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a4830ef3-5f84-4249-b8ae-692071f3f3b4 tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:56.348192 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a4830ef3-5f84-4249-b8ae-692071f3f3b4 tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:56.349878 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a4830ef3-5f84-4249-b8ae-692071f3f3b4 tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:56.350943 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a4830ef3-5f84-4249-b8ae-692071f3f3b4 tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:56.544844 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-a4830ef3-5f84-4249-b8ae-692071f3f3b4 tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3b0b46c0c1f74adaaaf65d46db7b2618', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '336b3ef8a8744bf3b86854b32000bda6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:24:56.556996 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-a4830ef3-5f84-4249-b8ae-692071f3f3b4 tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] 10.222.0.16 "GET /compute/v2.1/servers/f00bea08-4180-45d6-ba01-4b104f5615d6" status: 200 len: 1274 microversion: 2.1 time: 0.228519 Jul 16 21:24:56.562085 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1753/3487] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:24:56 2022] GET /compute/v2.1/servers/f00bea08-4180-45d6-ba01-4b104f5615d6 => generated 1274 bytes in 235 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:24:56.902792 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-40c054f4-3d51-404f-9714-650dcd59c4c1 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:24:56.903586 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-40c054f4-3d51-404f-9714-650dcd59c4c1 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: c9901d0a-8a17-4e6c-ba98-6bd587d410e0] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:24:56.919961 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-40c054f4-3d51-404f-9714-650dcd59c4c1 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:56.926879 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-40c054f4-3d51-404f-9714-650dcd59c4c1 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:56.931339 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-40c054f4-3d51-404f-9714-650dcd59c4c1 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:56.932837 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-40c054f4-3d51-404f-9714-650dcd59c4c1 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:57.480956 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-40c054f4-3d51-404f-9714-650dcd59c4c1 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '027dcc48e4fd4c36a5004096ad5e43f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dad823e240149edb30e592380249a13', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:24:57.497652 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-40c054f4-3d51-404f-9714-650dcd59c4c1 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "GET /compute/v2.1/servers/c9901d0a-8a17-4e6c-ba98-6bd587d410e0" status: 200 len: 1328 microversion: 2.1 time: 0.599773 Jul 16 21:24:57.520248 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1754/3488] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:24:56 2022] GET /compute/v2.1/servers/c9901d0a-8a17-4e6c-ba98-6bd587d410e0 => generated 1328 bytes in 623 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:24:57.538007 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-c65a6327-0c0d-45bf-8b6f-f8b41d9ab549 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:24:57.538959 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-c65a6327-0c0d-45bf-8b6f-f8b41d9ab549 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 8e6eab25-f25e-46a2-9c72-9424c872fb84] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:24:57.588601 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c65a6327-0c0d-45bf-8b6f-f8b41d9ab549 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:57.592962 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c65a6327-0c0d-45bf-8b6f-f8b41d9ab549 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:57.600033 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c65a6327-0c0d-45bf-8b6f-f8b41d9ab549 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:57.600555 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c65a6327-0c0d-45bf-8b6f-f8b41d9ab549 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:57.935792 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-c65a6327-0c0d-45bf-8b6f-f8b41d9ab549 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f2e974ab5aa48dfb4b00f810dcedb63', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd31be8475a6a49d6bfe8132dd050c6c0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:24:57.949736 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-c65a6327-0c0d-45bf-8b6f-f8b41d9ab549 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "GET /compute/v2.1/servers/8e6eab25-f25e-46a2-9c72-9424c872fb84" status: 200 len: 1523 microversion: 2.1 time: 0.423109 Jul 16 21:24:57.950873 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1755/3489] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:24:57 2022] GET /compute/v2.1/servers/8e6eab25-f25e-46a2-9c72-9424c872fb84 => generated 1523 bytes in 428 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:24:57.963219 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-89ba70d3-22fb-459f-b46a-d6f812877eb3 req-7e55cde2-816f-452e-829a-ec29ebd9b7e9 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "c9901d0a-8a17-4e6c-ba98-6bd587d410e0", "name": "network-vif-plugged", "status": "completed", "tag": "443db70b-054d-4606-9eee-d742416cd5a6"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:24:57.977802 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-89ba70d3-22fb-459f-b46a-d6f812877eb3 req-7e55cde2-816f-452e-829a-ec29ebd9b7e9 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:57.978260 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-89ba70d3-22fb-459f-b46a-d6f812877eb3 req-7e55cde2-816f-452e-829a-ec29ebd9b7e9 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:57.998107 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-89ba70d3-22fb-459f-b46a-d6f812877eb3 req-7e55cde2-816f-452e-829a-ec29ebd9b7e9 service nova] Creating event network-vif-plugged:443db70b-054d-4606-9eee-d742416cd5a6 for instance c9901d0a-8a17-4e6c-ba98-6bd587d410e0 on nch2-master-1 Jul 16 21:24:58.003905 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-89ba70d3-22fb-459f-b46a-d6f812877eb3 req-7e55cde2-816f-452e-829a-ec29ebd9b7e9 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.051640 Jul 16 21:24:58.005408 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1756/3490] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:24:57 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:24:58.012744 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-f1829e87-fa26-4c1b-9c34-a8c4c8bd0f65 tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:24:58.013869 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-f1829e87-fa26-4c1b-9c34-a8c4c8bd0f65 tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] [instance: f00bea08-4180-45d6-ba01-4b104f5615d6] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:24:58.034895 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f1829e87-fa26-4c1b-9c34-a8c4c8bd0f65 tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:58.035740 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f1829e87-fa26-4c1b-9c34-a8c4c8bd0f65 tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:58.036709 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f1829e87-fa26-4c1b-9c34-a8c4c8bd0f65 tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:58.038109 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f1829e87-fa26-4c1b-9c34-a8c4c8bd0f65 tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:58.044685 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:24:58.419818 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-c6df4a88-bc09-479f-b0cf-d5e47766b27f tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Flavor limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 16 21:24:58.420843 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-c6df4a88-bc09-479f-b0cf-d5e47766b27f tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Image limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 16 21:24:58.424269 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-c6df4a88-bc09-479f-b0cf-d5e47766b27f tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Flavor pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 16 21:24:58.425482 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-c6df4a88-bc09-479f-b0cf-d5e47766b27f tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Image pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 16 21:24:58.426906 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-c6df4a88-bc09-479f-b0cf-d5e47766b27f tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 16 21:24:58.442530 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c6df4a88-bc09-479f-b0cf-d5e47766b27f tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:58.443813 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c6df4a88-bc09-479f-b0cf-d5e47766b27f tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:58.445577 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c6df4a88-bc09-479f-b0cf-d5e47766b27f tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:58.447323 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c6df4a88-bc09-479f-b0cf-d5e47766b27f tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:58.450224 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-f1829e87-fa26-4c1b-9c34-a8c4c8bd0f65 tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3b0b46c0c1f74adaaaf65d46db7b2618', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '336b3ef8a8744bf3b86854b32000bda6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:24:58.474976 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-f1829e87-fa26-4c1b-9c34-a8c4c8bd0f65 tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] 10.222.0.16 "GET /compute/v2.1/servers/f00bea08-4180-45d6-ba01-4b104f5615d6" status: 200 len: 1316 microversion: 2.1 time: 0.468097 Jul 16 21:24:58.483836 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-c6df4a88-bc09-479f-b0cf-d5e47766b27f tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Getting quotas for project cfc2e42a0df346f4adef152ba2e3fa71. Resources: {'ram', 'instances', 'cores'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:24:58.485155 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1757/3491] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:24:58 2022] GET /compute/v2.1/servers/f00bea08-4180-45d6-ba01-4b104f5615d6 => generated 1316 bytes in 479 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:24:58.494351 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-c6df4a88-bc09-479f-b0cf-d5e47766b27f tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Getting quotas for user a23fafbab18d41ee82b84bbdf94cf0fe and project cfc2e42a0df346f4adef152ba2e3fa71. Resources: {'ram', 'instances', 'cores'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:24:58.515729 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-b0ce1d79-a096-461c-b5a3-d8aaad765573 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:24:58.516635 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-b0ce1d79-a096-461c-b5a3-d8aaad765573 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: c9901d0a-8a17-4e6c-ba98-6bd587d410e0] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:24:58.527604 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b0ce1d79-a096-461c-b5a3-d8aaad765573 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:58.528618 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b0ce1d79-a096-461c-b5a3-d8aaad765573 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:58.529591 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b0ce1d79-a096-461c-b5a3-d8aaad765573 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:58.530368 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b0ce1d79-a096-461c-b5a3-d8aaad765573 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:58.695420 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-c6df4a88-bc09-479f-b0cf-d5e47766b27f tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Going to run 1 instances... {{(pid=138681) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 16 21:24:58.698826 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-c6df4a88-bc09-479f-b0cf-d5e47766b27f tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] [instance: 4eb3d8d6-79ae-4109-a5a7-835c459d1719] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=False,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag='boot',updated_at=,uuid=,volume_id='6c632192-292e-476a-92f7-a3a26a1725e6',volume_size=None,volume_type=None), BlockDeviceMapping(attachment_id=,boot_index=1,connection_info=None,created_at=,delete_on_termination=False,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag='other',updated_at=,uuid=,volume_id='83999d90-80b2-4c0c-8aff-bec74809b4d9',volume_size=None,volume_type=None), BlockDeviceMapping(attachment_id=,boot_index=2,connection_info=None,created_at=,delete_on_termination=False,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag=None,updated_at=,uuid=,volume_id='f0883a5a-4e31-4e72-9db6-7d56ee50023e',volume_size=None,volume_type=None)] {{(pid=138681) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 16 21:24:59.156603 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-b0ce1d79-a096-461c-b5a3-d8aaad765573 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '027dcc48e4fd4c36a5004096ad5e43f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dad823e240149edb30e592380249a13', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:24:59.171948 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-b0ce1d79-a096-461c-b5a3-d8aaad765573 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "GET /compute/v2.1/servers/c9901d0a-8a17-4e6c-ba98-6bd587d410e0" status: 200 len: 1328 microversion: 2.1 time: 0.656544 Jul 16 21:24:59.177874 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1758/3492] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:24:58 2022] GET /compute/v2.1/servers/c9901d0a-8a17-4e6c-ba98-6bd587d410e0 => generated 1328 bytes in 667 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:24:59.190445 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-3454f6fb-6d80-4593-bd18-1ef444ada6c5 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Action: 'action', calling method: >, body: {"rescue": {"adminPass": "***"}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:24:59.191564 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-3454f6fb-6d80-4593-bd18-1ef444ada6c5 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] [instance: 92fbfe5e-07b4-44ec-9729-1a9230f4c02c] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:24:59.223506 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3454f6fb-6d80-4593-bd18-1ef444ada6c5 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:59.224709 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3454f6fb-6d80-4593-bd18-1ef444ada6c5 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:59.225801 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3454f6fb-6d80-4593-bd18-1ef444ada6c5 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:59.226548 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3454f6fb-6d80-4593-bd18-1ef444ada6c5 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:59.270883 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.objects.instance [None req-3454f6fb-6d80-4593-bd18-1ef444ada6c5 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lazy-loading 'flavor' on Instance uuid 92fbfe5e-07b4-44ec-9729-1a9230f4c02c {{(pid=138680) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:24:59.714668 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-3454f6fb-6d80-4593-bd18-1ef444ada6c5 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] 10.222.0.16 "POST /compute/v2.1/servers/92fbfe5e-07b4-44ec-9729-1a9230f4c02c/action" status: 200 len: 32 microversion: 2.1 time: 0.535124 Jul 16 21:24:59.722911 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1759/3493] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 21:24:59 2022] POST /compute/v2.1/servers/92fbfe5e-07b4-44ec-9729-1a9230f4c02c/action => generated 32 bytes in 544 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Jul 16 21:24:59.729376 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-baafb7c4-16f3-41b4-b238-1d8bf8c3213c tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:24:59.730110 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-baafb7c4-16f3-41b4-b238-1d8bf8c3213c tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 8e6eab25-f25e-46a2-9c72-9424c872fb84] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:24:59.742097 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-baafb7c4-16f3-41b4-b238-1d8bf8c3213c tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:59.742544 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-baafb7c4-16f3-41b4-b238-1d8bf8c3213c tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:24:59.744036 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-baafb7c4-16f3-41b4-b238-1d8bf8c3213c tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:24:59.744500 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-baafb7c4-16f3-41b4-b238-1d8bf8c3213c tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:00.091745 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-baafb7c4-16f3-41b4-b238-1d8bf8c3213c tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f2e974ab5aa48dfb4b00f810dcedb63', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd31be8475a6a49d6bfe8132dd050c6c0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:00.100464 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-baafb7c4-16f3-41b4-b238-1d8bf8c3213c tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "GET /compute/v2.1/servers/8e6eab25-f25e-46a2-9c72-9424c872fb84" status: 200 len: 1505 microversion: 2.1 time: 0.376278 Jul 16 21:25:00.111107 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1760/3494] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:24:59 2022] GET /compute/v2.1/servers/8e6eab25-f25e-46a2-9c72-9424c872fb84 => generated 1505 bytes in 388 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:25:00.123694 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-fe5fc6db-348d-4d44-a12f-3443b68b7fb0 tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:00.123694 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-fe5fc6db-348d-4d44-a12f-3443b68b7fb0 tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] [instance: f00bea08-4180-45d6-ba01-4b104f5615d6] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:25:00.126742 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-fe5fc6db-348d-4d44-a12f-3443b68b7fb0 tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:00.127513 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-fe5fc6db-348d-4d44-a12f-3443b68b7fb0 tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:00.128548 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-fe5fc6db-348d-4d44-a12f-3443b68b7fb0 tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:00.129223 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-fe5fc6db-348d-4d44-a12f-3443b68b7fb0 tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:00.333304 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-c6df4a88-bc09-479f-b0cf-d5e47766b27f tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 405 microversion: 2.32 time: 8.868155 Jul 16 21:25:00.335469 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1735/3495] 10.222.0.16 () {62 vars in 1188 bytes} [Sat Jul 16 21:24:51 2022] POST /compute/v2.1/servers => generated 405 bytes in 8871 msecs (HTTP/1.1 202) 10 headers in 453 bytes (1 switches on core 0) Jul 16 21:25:00.355832 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-3c7f2461-e431-42d6-8911-2cbc5d823ac8 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:00.358649 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-3c7f2461-e431-42d6-8911-2cbc5d823ac8 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] [instance: 92fbfe5e-07b4-44ec-9729-1a9230f4c02c] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:25:00.388288 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3c7f2461-e431-42d6-8911-2cbc5d823ac8 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:00.392433 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3c7f2461-e431-42d6-8911-2cbc5d823ac8 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:00.393599 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3c7f2461-e431-42d6-8911-2cbc5d823ac8 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:00.394610 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3c7f2461-e431-42d6-8911-2cbc5d823ac8 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:00.490715 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-fe5fc6db-348d-4d44-a12f-3443b68b7fb0 tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3b0b46c0c1f74adaaaf65d46db7b2618', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '336b3ef8a8744bf3b86854b32000bda6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:00.503993 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-fe5fc6db-348d-4d44-a12f-3443b68b7fb0 tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] 10.222.0.16 "GET /compute/v2.1/servers/f00bea08-4180-45d6-ba01-4b104f5615d6" status: 200 len: 1316 microversion: 2.1 time: 0.391012 Jul 16 21:25:00.520540 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1761/3496] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:25:00 2022] GET /compute/v2.1/servers/f00bea08-4180-45d6-ba01-4b104f5615d6 => generated 1316 bytes in 408 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:25:00.533884 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-def0bc10-51fa-4305-80e1-df62a4ee17af req-ca624faf-2ed9-4d67-a3b5-a48f619c6556 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "f00bea08-4180-45d6-ba01-4b104f5615d6", "tag": "541fa980-4b14-4bc2-af0f-d4bf299e66ca"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:25:00.553454 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-def0bc10-51fa-4305-80e1-df62a4ee17af req-ca624faf-2ed9-4d67-a3b5-a48f619c6556 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:00.560097 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-def0bc10-51fa-4305-80e1-df62a4ee17af req-ca624faf-2ed9-4d67-a3b5-a48f619c6556 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:00.591323 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-def0bc10-51fa-4305-80e1-df62a4ee17af req-ca624faf-2ed9-4d67-a3b5-a48f619c6556 service nova] Creating event network-changed:541fa980-4b14-4bc2-af0f-d4bf299e66ca for instance f00bea08-4180-45d6-ba01-4b104f5615d6 on nch2-master-1 Jul 16 21:25:00.611501 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-def0bc10-51fa-4305-80e1-df62a4ee17af req-ca624faf-2ed9-4d67-a3b5-a48f619c6556 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.088057 Jul 16 21:25:00.612686 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1762/3497] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:25:00 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:25:00.618429 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-d987eb9a-ec66-442f-9d0f-cdbacc371f53 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:00.626874 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-d987eb9a-ec66-442f-9d0f-cdbacc371f53 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: c9901d0a-8a17-4e6c-ba98-6bd587d410e0] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:25:00.639472 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d987eb9a-ec66-442f-9d0f-cdbacc371f53 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:00.640598 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d987eb9a-ec66-442f-9d0f-cdbacc371f53 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:00.641948 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d987eb9a-ec66-442f-9d0f-cdbacc371f53 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:00.642565 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d987eb9a-ec66-442f-9d0f-cdbacc371f53 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:00.900079 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-3c7f2461-e431-42d6-8911-2cbc5d823ac8 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcc9aa2a091c490599f9a9c41163e2eb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da56fc78f06142aba80ca4c6e3d60475', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:00.914256 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-3c7f2461-e431-42d6-8911-2cbc5d823ac8 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] 10.222.0.16 "GET /compute/v2.1/servers/92fbfe5e-07b4-44ec-9729-1a9230f4c02c" status: 200 len: 1523 microversion: 2.1 time: 0.575923 Jul 16 21:25:00.918662 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1736/3498] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:25:00 2022] GET /compute/v2.1/servers/92fbfe5e-07b4-44ec-9729-1a9230f4c02c => generated 1523 bytes in 582 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:25:00.934675 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-604c9f21-b2c3-4379-bb8a-6105ca72c782 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:00.939547 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-604c9f21-b2c3-4379-bb8a-6105ca72c782 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] [instance: 4eb3d8d6-79ae-4109-a5a7-835c459d1719] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:25:01.034198 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-d987eb9a-ec66-442f-9d0f-cdbacc371f53 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '027dcc48e4fd4c36a5004096ad5e43f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dad823e240149edb30e592380249a13', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:01.053365 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-d987eb9a-ec66-442f-9d0f-cdbacc371f53 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "GET /compute/v2.1/servers/c9901d0a-8a17-4e6c-ba98-6bd587d410e0" status: 200 len: 1508 microversion: 2.1 time: 0.439247 Jul 16 21:25:01.059102 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1763/3499] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:25:00 2022] GET /compute/v2.1/servers/c9901d0a-8a17-4e6c-ba98-6bd587d410e0 => generated 1508 bytes in 446 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:25:01.085551 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-6b8bfa91-2d23-4de3-b3cd-3f9153bf4f61 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:01.086799 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-6b8bfa91-2d23-4de3-b3cd-3f9153bf4f61 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: c9901d0a-8a17-4e6c-ba98-6bd587d410e0] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:25:01.102423 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6b8bfa91-2d23-4de3-b3cd-3f9153bf4f61 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:01.105430 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6b8bfa91-2d23-4de3-b3cd-3f9153bf4f61 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:01.108417 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6b8bfa91-2d23-4de3-b3cd-3f9153bf4f61 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:01.109372 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6b8bfa91-2d23-4de3-b3cd-3f9153bf4f61 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:01.149044 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-604c9f21-b2c3-4379-bb8a-6105ca72c782 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a23fafbab18d41ee82b84bbdf94cf0fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfc2e42a0df346f4adef152ba2e3fa71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:01.162238 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-604c9f21-b2c3-4379-bb8a-6105ca72c782 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'a23fafbab18d41ee82b84bbdf94cf0fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfc2e42a0df346f4adef152ba2e3fa71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:01.165131 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-604c9f21-b2c3-4379-bb8a-6105ca72c782 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'a23fafbab18d41ee82b84bbdf94cf0fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfc2e42a0df346f4adef152ba2e3fa71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:01.167132 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-604c9f21-b2c3-4379-bb8a-6105ca72c782 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] 10.222.0.16 "GET /compute/v2.1/servers/4eb3d8d6-79ae-4109-a5a7-835c459d1719" status: 200 len: 1285 microversion: 2.32 time: 0.246741 Jul 16 21:25:01.167965 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1737/3500] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:25:00 2022] GET /compute/v2.1/servers/4eb3d8d6-79ae-4109-a5a7-835c459d1719 => generated 1285 bytes in 248 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:25:01.173713 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-6e95b87c-ce72-466d-a81f-ebe48dc8ee34 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:01.175595 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-6e95b87c-ce72-466d-a81f-ebe48dc8ee34 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 8e6eab25-f25e-46a2-9c72-9424c872fb84] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:25:01.187524 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6e95b87c-ce72-466d-a81f-ebe48dc8ee34 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:01.188080 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6e95b87c-ce72-466d-a81f-ebe48dc8ee34 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:01.189361 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6e95b87c-ce72-466d-a81f-ebe48dc8ee34 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:01.190957 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6e95b87c-ce72-466d-a81f-ebe48dc8ee34 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:01.569046 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-6e95b87c-ce72-466d-a81f-ebe48dc8ee34 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f2e974ab5aa48dfb4b00f810dcedb63', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd31be8475a6a49d6bfe8132dd050c6c0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:01.594858 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-6e95b87c-ce72-466d-a81f-ebe48dc8ee34 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "GET /compute/v2.1/servers/8e6eab25-f25e-46a2-9c72-9424c872fb84" status: 200 len: 1505 microversion: 2.1 time: 0.424474 Jul 16 21:25:01.595563 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1738/3501] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:25:01 2022] GET /compute/v2.1/servers/8e6eab25-f25e-46a2-9c72-9424c872fb84 => generated 1505 bytes in 427 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:25:01.605192 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-29b722e5-13f4-4d4a-961b-52abcfa773af tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:01.610923 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-29b722e5-13f4-4d4a-961b-52abcfa773af tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] [instance: f00bea08-4180-45d6-ba01-4b104f5615d6] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:25:01.628745 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-29b722e5-13f4-4d4a-961b-52abcfa773af tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:01.629565 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-29b722e5-13f4-4d4a-961b-52abcfa773af tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:01.630527 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-29b722e5-13f4-4d4a-961b-52abcfa773af tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:01.631198 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-29b722e5-13f4-4d4a-961b-52abcfa773af tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:01.651614 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:25:01.656938 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-6b8bfa91-2d23-4de3-b3cd-3f9153bf4f61 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '027dcc48e4fd4c36a5004096ad5e43f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dad823e240149edb30e592380249a13', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:01.706787 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-6b8bfa91-2d23-4de3-b3cd-3f9153bf4f61 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "GET /compute/v2.1/servers/c9901d0a-8a17-4e6c-ba98-6bd587d410e0" status: 200 len: 1508 microversion: 2.1 time: 0.616731 Jul 16 21:25:01.715002 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1764/3502] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:25:01 2022] GET /compute/v2.1/servers/c9901d0a-8a17-4e6c-ba98-6bd587d410e0 => generated 1508 bytes in 642 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:25:01.735956 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-b2d47011-93c9-41ef-8892-e3098759017d tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:01.736733 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-b2d47011-93c9-41ef-8892-e3098759017d tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: c9901d0a-8a17-4e6c-ba98-6bd587d410e0] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:25:01.760431 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b2d47011-93c9-41ef-8892-e3098759017d tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:01.760910 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b2d47011-93c9-41ef-8892-e3098759017d tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:01.763085 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b2d47011-93c9-41ef-8892-e3098759017d tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:01.764250 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b2d47011-93c9-41ef-8892-e3098759017d tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:01.795729 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-b2d47011-93c9-41ef-8892-e3098759017d tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: c9901d0a-8a17-4e6c-ba98-6bd587d410e0] Going to try to terminate instance {{(pid=138680) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 16 21:25:01.933059 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-29b722e5-13f4-4d4a-961b-52abcfa773af tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3b0b46c0c1f74adaaaf65d46db7b2618', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '336b3ef8a8744bf3b86854b32000bda6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:01.942456 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-29b722e5-13f4-4d4a-961b-52abcfa773af tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] 10.222.0.16 "GET /compute/v2.1/servers/f00bea08-4180-45d6-ba01-4b104f5615d6" status: 200 len: 1316 microversion: 2.1 time: 0.344580 Jul 16 21:25:01.953675 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1739/3503] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:25:01 2022] GET /compute/v2.1/servers/f00bea08-4180-45d6-ba01-4b104f5615d6 => generated 1316 bytes in 356 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:25:01.962855 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-28d6d840-54cc-4a3b-a271-950c02c2e5bc tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:01.963697 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-28d6d840-54cc-4a3b-a271-950c02c2e5bc tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] [instance: 92fbfe5e-07b4-44ec-9729-1a9230f4c02c] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:25:01.974598 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-28d6d840-54cc-4a3b-a271-950c02c2e5bc tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:01.976391 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-28d6d840-54cc-4a3b-a271-950c02c2e5bc tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:01.978049 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-28d6d840-54cc-4a3b-a271-950c02c2e5bc tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:01.980152 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-28d6d840-54cc-4a3b-a271-950c02c2e5bc tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:02.143706 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-28d6d840-54cc-4a3b-a271-950c02c2e5bc tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcc9aa2a091c490599f9a9c41163e2eb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da56fc78f06142aba80ca4c6e3d60475', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:02.152558 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-28d6d840-54cc-4a3b-a271-950c02c2e5bc tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] 10.222.0.16 "GET /compute/v2.1/servers/92fbfe5e-07b4-44ec-9729-1a9230f4c02c" status: 200 len: 1523 microversion: 2.1 time: 0.197341 Jul 16 21:25:02.153444 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1740/3504] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:25:01 2022] GET /compute/v2.1/servers/92fbfe5e-07b4-44ec-9729-1a9230f4c02c => generated 1523 bytes in 199 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:25:02.165724 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-b2d47011-93c9-41ef-8892-e3098759017d tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "DELETE /compute/v2.1/servers/c9901d0a-8a17-4e6c-ba98-6bd587d410e0" status: 204 len: 0 microversion: 2.1 time: 0.435659 Jul 16 21:25:02.166364 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1765/3505] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:25:01 2022] DELETE /compute/v2.1/servers/c9901d0a-8a17-4e6c-ba98-6bd587d410e0 => generated 0 bytes in 438 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Jul 16 21:25:02.175534 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-ff0084ad-e3b0-4679-a1e4-0173c2292439 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:02.176152 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-ff0084ad-e3b0-4679-a1e4-0173c2292439 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: c9901d0a-8a17-4e6c-ba98-6bd587d410e0] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:25:02.184133 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ff0084ad-e3b0-4679-a1e4-0173c2292439 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:02.188532 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ff0084ad-e3b0-4679-a1e4-0173c2292439 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:02.188532 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ff0084ad-e3b0-4679-a1e4-0173c2292439 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:02.190652 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-8cbc74ff-a9e5-4c38-91da-35c2ebc30e46 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:02.190801 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ff0084ad-e3b0-4679-a1e4-0173c2292439 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:02.190923 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-8cbc74ff-a9e5-4c38-91da-35c2ebc30e46 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] [instance: 4eb3d8d6-79ae-4109-a5a7-835c459d1719] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:25:02.289154 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-8cbc74ff-a9e5-4c38-91da-35c2ebc30e46 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a23fafbab18d41ee82b84bbdf94cf0fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfc2e42a0df346f4adef152ba2e3fa71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:02.305974 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-8cbc74ff-a9e5-4c38-91da-35c2ebc30e46 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'a23fafbab18d41ee82b84bbdf94cf0fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfc2e42a0df346f4adef152ba2e3fa71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:02.309039 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-8cbc74ff-a9e5-4c38-91da-35c2ebc30e46 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'a23fafbab18d41ee82b84bbdf94cf0fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfc2e42a0df346f4adef152ba2e3fa71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:02.319988 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-8cbc74ff-a9e5-4c38-91da-35c2ebc30e46 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] 10.222.0.16 "GET /compute/v2.1/servers/4eb3d8d6-79ae-4109-a5a7-835c459d1719" status: 200 len: 1285 microversion: 2.32 time: 0.131693 Jul 16 21:25:02.323672 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1766/3506] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:25:02 2022] GET /compute/v2.1/servers/4eb3d8d6-79ae-4109-a5a7-835c459d1719 => generated 1285 bytes in 142 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:25:02.343657 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-b23c94e6-2112-474a-9e2f-b9b9e6f109aa req-ca4da695-93a4-468b-b87a-08a33165ddce service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "8e6eab25-f25e-46a2-9c72-9424c872fb84", "name": "network-vif-unplugged", "status": "completed", "tag": "4a81f7ad-929b-4f37-b5ac-e484b09be512"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:25:02.355444 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-b23c94e6-2112-474a-9e2f-b9b9e6f109aa req-ca4da695-93a4-468b-b87a-08a33165ddce service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:02.356355 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-b23c94e6-2112-474a-9e2f-b9b9e6f109aa req-ca4da695-93a4-468b-b87a-08a33165ddce service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:02.374742 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-b23c94e6-2112-474a-9e2f-b9b9e6f109aa req-ca4da695-93a4-468b-b87a-08a33165ddce service nova] Creating event network-vif-unplugged:4a81f7ad-929b-4f37-b5ac-e484b09be512 for instance 8e6eab25-f25e-46a2-9c72-9424c872fb84 on nch2-master-1 Jul 16 21:25:02.378199 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-ff0084ad-e3b0-4679-a1e4-0173c2292439 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '027dcc48e4fd4c36a5004096ad5e43f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dad823e240149edb30e592380249a13', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:02.381210 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-b23c94e6-2112-474a-9e2f-b9b9e6f109aa req-ca4da695-93a4-468b-b87a-08a33165ddce service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.044561 Jul 16 21:25:02.382097 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1767/3507] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:25:02 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:25:02.387272 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-ff0084ad-e3b0-4679-a1e4-0173c2292439 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "GET /compute/v2.1/servers/c9901d0a-8a17-4e6c-ba98-6bd587d410e0" status: 200 len: 1514 microversion: 2.1 time: 0.216505 Jul 16 21:25:02.387863 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1741/3508] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:25:02 2022] GET /compute/v2.1/servers/c9901d0a-8a17-4e6c-ba98-6bd587d410e0 => generated 1514 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:25:02.612652 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-1e19ecf9-5b77-4a48-824a-be26bc7754a3 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:02.613356 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-1e19ecf9-5b77-4a48-824a-be26bc7754a3 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 8e6eab25-f25e-46a2-9c72-9424c872fb84] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:25:02.626960 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1e19ecf9-5b77-4a48-824a-be26bc7754a3 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:02.631134 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1e19ecf9-5b77-4a48-824a-be26bc7754a3 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:02.631134 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1e19ecf9-5b77-4a48-824a-be26bc7754a3 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:02.631134 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1e19ecf9-5b77-4a48-824a-be26bc7754a3 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:02.783310 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-1e19ecf9-5b77-4a48-824a-be26bc7754a3 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f2e974ab5aa48dfb4b00f810dcedb63', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd31be8475a6a49d6bfe8132dd050c6c0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:02.800704 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-1e19ecf9-5b77-4a48-824a-be26bc7754a3 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "GET /compute/v2.1/servers/8e6eab25-f25e-46a2-9c72-9424c872fb84" status: 200 len: 1505 microversion: 2.1 time: 0.192261 Jul 16 21:25:02.801290 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1768/3509] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:25:02 2022] GET /compute/v2.1/servers/8e6eab25-f25e-46a2-9c72-9424c872fb84 => generated 1505 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:25:02.959365 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-4c8a06c0-4531-479a-8931-6042dcbc2c13 tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:02.960051 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-4c8a06c0-4531-479a-8931-6042dcbc2c13 tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] [instance: f00bea08-4180-45d6-ba01-4b104f5615d6] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:25:02.967478 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4c8a06c0-4531-479a-8931-6042dcbc2c13 tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:02.967932 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4c8a06c0-4531-479a-8931-6042dcbc2c13 tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:02.968709 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4c8a06c0-4531-479a-8931-6042dcbc2c13 tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:02.969149 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4c8a06c0-4531-479a-8931-6042dcbc2c13 tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:03.169531 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-23fa0ab0-85e4-454c-8c8d-41d65f2d134b tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:03.170155 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-23fa0ab0-85e4-454c-8c8d-41d65f2d134b tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] [instance: 92fbfe5e-07b4-44ec-9729-1a9230f4c02c] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:25:03.177561 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-23fa0ab0-85e4-454c-8c8d-41d65f2d134b tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:03.178338 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-23fa0ab0-85e4-454c-8c8d-41d65f2d134b tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:03.179302 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-23fa0ab0-85e4-454c-8c8d-41d65f2d134b tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:03.179862 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-23fa0ab0-85e4-454c-8c8d-41d65f2d134b tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:03.188312 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-4c8a06c0-4531-479a-8931-6042dcbc2c13 tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3b0b46c0c1f74adaaaf65d46db7b2618', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '336b3ef8a8744bf3b86854b32000bda6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:03.202546 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-4c8a06c0-4531-479a-8931-6042dcbc2c13 tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] 10.222.0.16 "GET /compute/v2.1/servers/f00bea08-4180-45d6-ba01-4b104f5615d6" status: 200 len: 1316 microversion: 2.1 time: 0.245442 Jul 16 21:25:03.207204 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1742/3510] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:25:02 2022] GET /compute/v2.1/servers/f00bea08-4180-45d6-ba01-4b104f5615d6 => generated 1316 bytes in 252 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:25:03.342859 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-da4c6185-e1fa-44e7-a5c3-3cdc1c9125d3 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:03.343635 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-da4c6185-e1fa-44e7-a5c3-3cdc1c9125d3 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] [instance: 4eb3d8d6-79ae-4109-a5a7-835c459d1719] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:25:03.349736 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-da4c6185-e1fa-44e7-a5c3-3cdc1c9125d3 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:03.350503 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-da4c6185-e1fa-44e7-a5c3-3cdc1c9125d3 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:03.351301 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-da4c6185-e1fa-44e7-a5c3-3cdc1c9125d3 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:03.351745 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-da4c6185-e1fa-44e7-a5c3-3cdc1c9125d3 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:03.377224 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-23fa0ab0-85e4-454c-8c8d-41d65f2d134b tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcc9aa2a091c490599f9a9c41163e2eb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da56fc78f06142aba80ca4c6e3d60475', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:03.387281 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-23fa0ab0-85e4-454c-8c8d-41d65f2d134b tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] 10.222.0.16 "GET /compute/v2.1/servers/92fbfe5e-07b4-44ec-9729-1a9230f4c02c" status: 200 len: 1523 microversion: 2.1 time: 0.221493 Jul 16 21:25:03.388115 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1769/3511] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:25:03 2022] GET /compute/v2.1/servers/92fbfe5e-07b4-44ec-9729-1a9230f4c02c => generated 1523 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:25:03.411040 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-9068a780-588f-4c9a-bd45-589cc2853450 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:03.411794 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-9068a780-588f-4c9a-bd45-589cc2853450 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: c9901d0a-8a17-4e6c-ba98-6bd587d410e0] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:25:03.418262 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9068a780-588f-4c9a-bd45-589cc2853450 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:03.418784 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9068a780-588f-4c9a-bd45-589cc2853450 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:03.422579 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-da4c6185-e1fa-44e7-a5c3-3cdc1c9125d3 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a23fafbab18d41ee82b84bbdf94cf0fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfc2e42a0df346f4adef152ba2e3fa71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:03.423184 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9068a780-588f-4c9a-bd45-589cc2853450 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:03.423713 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9068a780-588f-4c9a-bd45-589cc2853450 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:03.435814 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-da4c6185-e1fa-44e7-a5c3-3cdc1c9125d3 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'a23fafbab18d41ee82b84bbdf94cf0fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfc2e42a0df346f4adef152ba2e3fa71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:03.438123 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-da4c6185-e1fa-44e7-a5c3-3cdc1c9125d3 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'a23fafbab18d41ee82b84bbdf94cf0fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfc2e42a0df346f4adef152ba2e3fa71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:03.439611 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-da4c6185-e1fa-44e7-a5c3-3cdc1c9125d3 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] 10.222.0.16 "GET /compute/v2.1/servers/4eb3d8d6-79ae-4109-a5a7-835c459d1719" status: 200 len: 1527 microversion: 2.32 time: 0.101001 Jul 16 21:25:03.453318 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1743/3512] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:25:03 2022] GET /compute/v2.1/servers/4eb3d8d6-79ae-4109-a5a7-835c459d1719 => generated 1527 bytes in 114 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 16 21:25:03.600615 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-9068a780-588f-4c9a-bd45-589cc2853450 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '027dcc48e4fd4c36a5004096ad5e43f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dad823e240149edb30e592380249a13', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:03.610382 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-9068a780-588f-4c9a-bd45-589cc2853450 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "GET /compute/v2.1/servers/c9901d0a-8a17-4e6c-ba98-6bd587d410e0" status: 200 len: 1514 microversion: 2.1 time: 0.209725 Jul 16 21:25:03.611184 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1770/3513] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:25:03 2022] GET /compute/v2.1/servers/c9901d0a-8a17-4e6c-ba98-6bd587d410e0 => generated 1514 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:25:03.818879 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-1aa852c1-66d4-458c-80db-4614775d2ea6 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:03.819842 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-1aa852c1-66d4-458c-80db-4614775d2ea6 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 8e6eab25-f25e-46a2-9c72-9424c872fb84] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:25:03.827813 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1aa852c1-66d4-458c-80db-4614775d2ea6 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:03.828267 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1aa852c1-66d4-458c-80db-4614775d2ea6 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:03.829036 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1aa852c1-66d4-458c-80db-4614775d2ea6 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:03.829505 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1aa852c1-66d4-458c-80db-4614775d2ea6 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:03.979091 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-1aa852c1-66d4-458c-80db-4614775d2ea6 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f2e974ab5aa48dfb4b00f810dcedb63', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd31be8475a6a49d6bfe8132dd050c6c0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:03.989679 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-1aa852c1-66d4-458c-80db-4614775d2ea6 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "GET /compute/v2.1/servers/8e6eab25-f25e-46a2-9c72-9424c872fb84" status: 200 len: 1505 microversion: 2.1 time: 0.174316 Jul 16 21:25:03.990484 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1744/3514] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:25:03 2022] GET /compute/v2.1/servers/8e6eab25-f25e-46a2-9c72-9424c872fb84 => generated 1505 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:25:04.223594 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-e2d5f13a-01c9-4648-9503-110e9a8142e5 tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:04.224302 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-e2d5f13a-01c9-4648-9503-110e9a8142e5 tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] [instance: f00bea08-4180-45d6-ba01-4b104f5615d6] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:25:04.227376 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_api:SELECT=99,nova_api:UPDATE=1 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:25:04.237779 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e2d5f13a-01c9-4648-9503-110e9a8142e5 tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:04.238239 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e2d5f13a-01c9-4648-9503-110e9a8142e5 tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:04.239044 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e2d5f13a-01c9-4648-9503-110e9a8142e5 tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:04.243675 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e2d5f13a-01c9-4648-9503-110e9a8142e5 tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:04.405007 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-c5f071ed-8fab-4c67-b28e-95cdecef0091 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:04.405628 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-c5f071ed-8fab-4c67-b28e-95cdecef0091 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] [instance: 92fbfe5e-07b4-44ec-9729-1a9230f4c02c] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:25:04.415301 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-e2d5f13a-01c9-4648-9503-110e9a8142e5 tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3b0b46c0c1f74adaaaf65d46db7b2618', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '336b3ef8a8744bf3b86854b32000bda6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:04.415702 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c5f071ed-8fab-4c67-b28e-95cdecef0091 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:04.415702 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c5f071ed-8fab-4c67-b28e-95cdecef0091 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:04.415702 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c5f071ed-8fab-4c67-b28e-95cdecef0091 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:04.415702 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c5f071ed-8fab-4c67-b28e-95cdecef0091 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:04.419651 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-e2d5f13a-01c9-4648-9503-110e9a8142e5 tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] 10.222.0.16 "GET /compute/v2.1/servers/f00bea08-4180-45d6-ba01-4b104f5615d6" status: 200 len: 1316 microversion: 2.1 time: 0.201935 Jul 16 21:25:04.420278 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1771/3515] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:25:04 2022] GET /compute/v2.1/servers/f00bea08-4180-45d6-ba01-4b104f5615d6 => generated 1316 bytes in 203 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:25:04.459761 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-6d8984b0-e960-4dc1-878d-47f08eae6808 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:04.460388 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-6d8984b0-e960-4dc1-878d-47f08eae6808 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] [instance: 4eb3d8d6-79ae-4109-a5a7-835c459d1719] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:25:04.466617 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6d8984b0-e960-4dc1-878d-47f08eae6808 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:04.472024 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6d8984b0-e960-4dc1-878d-47f08eae6808 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:04.472024 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6d8984b0-e960-4dc1-878d-47f08eae6808 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:04.472024 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6d8984b0-e960-4dc1-878d-47f08eae6808 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:04.484720 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=95,nova_cell1:UPDATE=3,nova_cell1:INSERT=2 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:25:04.536839 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-6d8984b0-e960-4dc1-878d-47f08eae6808 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a23fafbab18d41ee82b84bbdf94cf0fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfc2e42a0df346f4adef152ba2e3fa71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:04.546804 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-6d8984b0-e960-4dc1-878d-47f08eae6808 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'a23fafbab18d41ee82b84bbdf94cf0fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfc2e42a0df346f4adef152ba2e3fa71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:04.548344 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-6d8984b0-e960-4dc1-878d-47f08eae6808 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'a23fafbab18d41ee82b84bbdf94cf0fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfc2e42a0df346f4adef152ba2e3fa71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:04.549725 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-6d8984b0-e960-4dc1-878d-47f08eae6808 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] 10.222.0.16 "GET /compute/v2.1/servers/4eb3d8d6-79ae-4109-a5a7-835c459d1719" status: 200 len: 1575 microversion: 2.32 time: 0.093957 Jul 16 21:25:04.550296 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1772/3516] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:25:04 2022] GET /compute/v2.1/servers/4eb3d8d6-79ae-4109-a5a7-835c459d1719 => generated 1575 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 16 21:25:04.582155 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-c5f071ed-8fab-4c67-b28e-95cdecef0091 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcc9aa2a091c490599f9a9c41163e2eb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da56fc78f06142aba80ca4c6e3d60475', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:04.592152 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-c5f071ed-8fab-4c67-b28e-95cdecef0091 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] 10.222.0.16 "GET /compute/v2.1/servers/92fbfe5e-07b4-44ec-9729-1a9230f4c02c" status: 200 len: 1523 microversion: 2.1 time: 0.191221 Jul 16 21:25:04.592761 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1745/3517] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:25:04 2022] GET /compute/v2.1/servers/92fbfe5e-07b4-44ec-9729-1a9230f4c02c => generated 1523 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:25:04.629344 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-47500372-9b40-44e2-8281-d9534cd6a2e2 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:04.631634 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-47500372-9b40-44e2-8281-d9534cd6a2e2 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: c9901d0a-8a17-4e6c-ba98-6bd587d410e0] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:25:04.638239 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-47500372-9b40-44e2-8281-d9534cd6a2e2 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:04.638946 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-47500372-9b40-44e2-8281-d9534cd6a2e2 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:04.640130 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-47500372-9b40-44e2-8281-d9534cd6a2e2 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:04.641047 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-47500372-9b40-44e2-8281-d9534cd6a2e2 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:04.798590 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-47500372-9b40-44e2-8281-d9534cd6a2e2 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '027dcc48e4fd4c36a5004096ad5e43f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dad823e240149edb30e592380249a13', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:04.807973 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-47500372-9b40-44e2-8281-d9534cd6a2e2 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "GET /compute/v2.1/servers/c9901d0a-8a17-4e6c-ba98-6bd587d410e0" status: 200 len: 1514 microversion: 2.1 time: 0.182754 Jul 16 21:25:04.808891 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1773/3518] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:25:04 2022] GET /compute/v2.1/servers/c9901d0a-8a17-4e6c-ba98-6bd587d410e0 => generated 1514 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:25:05.025057 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-0d2121de-6318-4715-ab22-016180ff90d5 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:05.026359 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-0d2121de-6318-4715-ab22-016180ff90d5 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 8e6eab25-f25e-46a2-9c72-9424c872fb84] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:25:05.033134 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0d2121de-6318-4715-ab22-016180ff90d5 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:05.033832 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0d2121de-6318-4715-ab22-016180ff90d5 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:05.034965 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0d2121de-6318-4715-ab22-016180ff90d5 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:05.035436 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0d2121de-6318-4715-ab22-016180ff90d5 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:05.187892 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-0d2121de-6318-4715-ab22-016180ff90d5 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f2e974ab5aa48dfb4b00f810dcedb63', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd31be8475a6a49d6bfe8132dd050c6c0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:05.196547 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-0d2121de-6318-4715-ab22-016180ff90d5 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "GET /compute/v2.1/servers/8e6eab25-f25e-46a2-9c72-9424c872fb84" status: 200 len: 1505 microversion: 2.1 time: 0.178487 Jul 16 21:25:05.197400 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1746/3519] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:25:05 2022] GET /compute/v2.1/servers/8e6eab25-f25e-46a2-9c72-9424c872fb84 => generated 1505 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:25:05.437643 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-8a04342e-0330-4271-862e-b4b969260ec9 tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:05.439241 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-8a04342e-0330-4271-862e-b4b969260ec9 tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] [instance: f00bea08-4180-45d6-ba01-4b104f5615d6] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:25:05.449647 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8a04342e-0330-4271-862e-b4b969260ec9 tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:05.450109 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8a04342e-0330-4271-862e-b4b969260ec9 tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:05.450910 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8a04342e-0330-4271-862e-b4b969260ec9 tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:05.452177 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8a04342e-0330-4271-862e-b4b969260ec9 tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:05.569473 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-b6821ff9-71b9-4312-9555-a1ef81ffa6e8 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:05.570661 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-b6821ff9-71b9-4312-9555-a1ef81ffa6e8 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] [instance: 4eb3d8d6-79ae-4109-a5a7-835c459d1719] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:25:05.585243 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b6821ff9-71b9-4312-9555-a1ef81ffa6e8 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:05.588058 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b6821ff9-71b9-4312-9555-a1ef81ffa6e8 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:05.592055 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b6821ff9-71b9-4312-9555-a1ef81ffa6e8 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:05.592055 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b6821ff9-71b9-4312-9555-a1ef81ffa6e8 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:05.721161 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-b6821ff9-71b9-4312-9555-a1ef81ffa6e8 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a23fafbab18d41ee82b84bbdf94cf0fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfc2e42a0df346f4adef152ba2e3fa71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:05.734281 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-b6821ff9-71b9-4312-9555-a1ef81ffa6e8 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'a23fafbab18d41ee82b84bbdf94cf0fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfc2e42a0df346f4adef152ba2e3fa71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:05.735823 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-b6821ff9-71b9-4312-9555-a1ef81ffa6e8 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'a23fafbab18d41ee82b84bbdf94cf0fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfc2e42a0df346f4adef152ba2e3fa71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:05.736981 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-b6821ff9-71b9-4312-9555-a1ef81ffa6e8 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] 10.222.0.16 "GET /compute/v2.1/servers/4eb3d8d6-79ae-4109-a5a7-835c459d1719" status: 200 len: 1593 microversion: 2.32 time: 0.171907 Jul 16 21:25:05.748087 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1747/3520] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:25:05 2022] GET /compute/v2.1/servers/4eb3d8d6-79ae-4109-a5a7-835c459d1719 => generated 1593 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 16 21:25:05.757136 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-f2a9a4d8-63d7-4277-b6e6-dc89d72cc1f2 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:05.767408 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-f2a9a4d8-63d7-4277-b6e6-dc89d72cc1f2 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] [instance: 92fbfe5e-07b4-44ec-9729-1a9230f4c02c] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:25:05.798763 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f2a9a4d8-63d7-4277-b6e6-dc89d72cc1f2 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:05.801860 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f2a9a4d8-63d7-4277-b6e6-dc89d72cc1f2 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:05.807907 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f2a9a4d8-63d7-4277-b6e6-dc89d72cc1f2 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:05.808553 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f2a9a4d8-63d7-4277-b6e6-dc89d72cc1f2 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:05.876744 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-8a04342e-0330-4271-862e-b4b969260ec9 tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3b0b46c0c1f74adaaaf65d46db7b2618', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '336b3ef8a8744bf3b86854b32000bda6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:05.886191 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-8a04342e-0330-4271-862e-b4b969260ec9 tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] 10.222.0.16 "GET /compute/v2.1/servers/f00bea08-4180-45d6-ba01-4b104f5615d6" status: 200 len: 1316 microversion: 2.1 time: 0.452708 Jul 16 21:25:05.890878 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1774/3521] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:25:05 2022] GET /compute/v2.1/servers/f00bea08-4180-45d6-ba01-4b104f5615d6 => generated 1316 bytes in 458 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:25:05.895866 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-50db8938-0e2a-44cc-8686-c3c8956ad7c7 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:05.903956 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-50db8938-0e2a-44cc-8686-c3c8956ad7c7 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: c9901d0a-8a17-4e6c-ba98-6bd587d410e0] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:25:05.915335 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-50db8938-0e2a-44cc-8686-c3c8956ad7c7 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:05.916201 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-50db8938-0e2a-44cc-8686-c3c8956ad7c7 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:05.921129 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-50db8938-0e2a-44cc-8686-c3c8956ad7c7 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:05.921785 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-50db8938-0e2a-44cc-8686-c3c8956ad7c7 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:06.268701 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-f2a9a4d8-63d7-4277-b6e6-dc89d72cc1f2 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcc9aa2a091c490599f9a9c41163e2eb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da56fc78f06142aba80ca4c6e3d60475', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:06.282495 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-f2a9a4d8-63d7-4277-b6e6-dc89d72cc1f2 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] 10.222.0.16 "GET /compute/v2.1/servers/92fbfe5e-07b4-44ec-9729-1a9230f4c02c" status: 200 len: 1523 microversion: 2.1 time: 0.530224 Jul 16 21:25:06.285750 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1748/3522] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:25:05 2022] GET /compute/v2.1/servers/92fbfe5e-07b4-44ec-9729-1a9230f4c02c => generated 1523 bytes in 534 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:25:06.301763 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-c55db029-4791-4316-8ce9-41083890133a tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:06.303131 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-c55db029-4791-4316-8ce9-41083890133a tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 8e6eab25-f25e-46a2-9c72-9424c872fb84] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:25:06.311442 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c55db029-4791-4316-8ce9-41083890133a tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:06.312671 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c55db029-4791-4316-8ce9-41083890133a tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:06.313869 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c55db029-4791-4316-8ce9-41083890133a tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:06.314658 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c55db029-4791-4316-8ce9-41083890133a tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:06.565052 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-50db8938-0e2a-44cc-8686-c3c8956ad7c7 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '027dcc48e4fd4c36a5004096ad5e43f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dad823e240149edb30e592380249a13', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:06.577817 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-50db8938-0e2a-44cc-8686-c3c8956ad7c7 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "GET /compute/v2.1/servers/c9901d0a-8a17-4e6c-ba98-6bd587d410e0" status: 200 len: 1514 microversion: 2.1 time: 0.685572 Jul 16 21:25:06.578434 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1775/3523] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:25:05 2022] GET /compute/v2.1/servers/c9901d0a-8a17-4e6c-ba98-6bd587d410e0 => generated 1514 bytes in 687 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:25:06.771286 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-230e2472-ad25-4a90-811b-4f7cda87909e tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:06.772368 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-230e2472-ad25-4a90-811b-4f7cda87909e tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] [instance: 4eb3d8d6-79ae-4109-a5a7-835c459d1719] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:25:06.781718 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-230e2472-ad25-4a90-811b-4f7cda87909e tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:06.782573 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-230e2472-ad25-4a90-811b-4f7cda87909e tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:06.783573 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-230e2472-ad25-4a90-811b-4f7cda87909e tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:06.784217 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-230e2472-ad25-4a90-811b-4f7cda87909e tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:07.189170 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-c55db029-4791-4316-8ce9-41083890133a tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f2e974ab5aa48dfb4b00f810dcedb63', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd31be8475a6a49d6bfe8132dd050c6c0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:07.201156 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-c55db029-4791-4316-8ce9-41083890133a tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "GET /compute/v2.1/servers/8e6eab25-f25e-46a2-9c72-9424c872fb84" status: 200 len: 1508 microversion: 2.1 time: 0.903587 Jul 16 21:25:07.202702 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1749/3524] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:25:06 2022] GET /compute/v2.1/servers/8e6eab25-f25e-46a2-9c72-9424c872fb84 => generated 1508 bytes in 916 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:25:07.207952 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-01d1860f-dab8-433e-8d4c-3d6f139bae50 req-d92dce79-18d2-484a-ba1d-40bab86bd425 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "f00bea08-4180-45d6-ba01-4b104f5615d6", "name": "network-vif-plugged", "status": "completed", "tag": "541fa980-4b14-4bc2-af0f-d4bf299e66ca"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:25:07.217086 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-01d1860f-dab8-433e-8d4c-3d6f139bae50 req-d92dce79-18d2-484a-ba1d-40bab86bd425 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:07.217975 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-01d1860f-dab8-433e-8d4c-3d6f139bae50 req-d92dce79-18d2-484a-ba1d-40bab86bd425 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:07.226454 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-230e2472-ad25-4a90-811b-4f7cda87909e tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a23fafbab18d41ee82b84bbdf94cf0fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfc2e42a0df346f4adef152ba2e3fa71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:07.235901 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-01d1860f-dab8-433e-8d4c-3d6f139bae50 req-d92dce79-18d2-484a-ba1d-40bab86bd425 service nova] Creating event network-vif-plugged:541fa980-4b14-4bc2-af0f-d4bf299e66ca for instance f00bea08-4180-45d6-ba01-4b104f5615d6 on nch2-master-1 Jul 16 21:25:07.244310 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-230e2472-ad25-4a90-811b-4f7cda87909e tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'a23fafbab18d41ee82b84bbdf94cf0fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfc2e42a0df346f4adef152ba2e3fa71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:07.245158 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-01d1860f-dab8-433e-8d4c-3d6f139bae50 req-d92dce79-18d2-484a-ba1d-40bab86bd425 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.039825 Jul 16 21:25:07.246467 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1750/3525] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:25:07 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:25:07.254114 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-230e2472-ad25-4a90-811b-4f7cda87909e tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'a23fafbab18d41ee82b84bbdf94cf0fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfc2e42a0df346f4adef152ba2e3fa71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:07.255383 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-230e2472-ad25-4a90-811b-4f7cda87909e tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] 10.222.0.16 "GET /compute/v2.1/servers/4eb3d8d6-79ae-4109-a5a7-835c459d1719" status: 200 len: 1593 microversion: 2.32 time: 0.488854 Jul 16 21:25:07.255953 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1776/3526] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:25:06 2022] GET /compute/v2.1/servers/4eb3d8d6-79ae-4109-a5a7-835c459d1719 => generated 1593 bytes in 490 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 16 21:25:07.262067 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-12f37198-2fb8-4f7e-ba19-b58e7da47e19 tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:07.263194 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-12f37198-2fb8-4f7e-ba19-b58e7da47e19 tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] [instance: f00bea08-4180-45d6-ba01-4b104f5615d6] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:25:07.268109 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-121ab998-3faf-49a2-83d9-baf0d17b8e0b tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:07.271354 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-121ab998-3faf-49a2-83d9-baf0d17b8e0b tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 8e6eab25-f25e-46a2-9c72-9424c872fb84] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:25:07.272479 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-12f37198-2fb8-4f7e-ba19-b58e7da47e19 tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:07.274254 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-12f37198-2fb8-4f7e-ba19-b58e7da47e19 tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:07.275956 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-12f37198-2fb8-4f7e-ba19-b58e7da47e19 tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:07.276653 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-12f37198-2fb8-4f7e-ba19-b58e7da47e19 tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:07.296530 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-121ab998-3faf-49a2-83d9-baf0d17b8e0b tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:07.299886 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-121ab998-3faf-49a2-83d9-baf0d17b8e0b tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:07.301063 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-121ab998-3faf-49a2-83d9-baf0d17b8e0b tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:07.302749 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-121ab998-3faf-49a2-83d9-baf0d17b8e0b tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:07.391048 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-121ab998-3faf-49a2-83d9-baf0d17b8e0b tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 8e6eab25-f25e-46a2-9c72-9424c872fb84] Going to try to terminate instance {{(pid=138680) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 16 21:25:07.431381 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.compute.api [None req-121ab998-3faf-49a2-83d9-baf0d17b8e0b tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 8e6eab25-f25e-46a2-9c72-9424c872fb84] Working on deleting snapshot 6d8860f3-3fb5-42a8-959d-10a22b0d96ce from shelved instance... Jul 16 21:25:07.730239 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-12f37198-2fb8-4f7e-ba19-b58e7da47e19 tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3b0b46c0c1f74adaaaf65d46db7b2618', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '336b3ef8a8744bf3b86854b32000bda6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:07.738121 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-12f37198-2fb8-4f7e-ba19-b58e7da47e19 tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] 10.222.0.16 "GET /compute/v2.1/servers/f00bea08-4180-45d6-ba01-4b104f5615d6" status: 200 len: 1316 microversion: 2.1 time: 0.487300 Jul 16 21:25:07.745832 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1751/3527] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:25:07 2022] GET /compute/v2.1/servers/f00bea08-4180-45d6-ba01-4b104f5615d6 => generated 1316 bytes in 499 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:25:07.750922 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-8b061b7e-819f-450e-8588-b378884fbb25 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:07.756313 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-8b061b7e-819f-450e-8588-b378884fbb25 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] [instance: 92fbfe5e-07b4-44ec-9729-1a9230f4c02c] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:25:07.771592 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8b061b7e-819f-450e-8588-b378884fbb25 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:07.774099 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8b061b7e-819f-450e-8588-b378884fbb25 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:07.775052 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8b061b7e-819f-450e-8588-b378884fbb25 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:07.775371 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8b061b7e-819f-450e-8588-b378884fbb25 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:08.063545 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-8b061b7e-819f-450e-8588-b378884fbb25 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcc9aa2a091c490599f9a9c41163e2eb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da56fc78f06142aba80ca4c6e3d60475', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:08.081723 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-8b061b7e-819f-450e-8588-b378884fbb25 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] 10.222.0.16 "GET /compute/v2.1/servers/92fbfe5e-07b4-44ec-9729-1a9230f4c02c" status: 200 len: 1523 microversion: 2.1 time: 0.333740 Jul 16 21:25:08.083067 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1752/3528] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:25:07 2022] GET /compute/v2.1/servers/92fbfe5e-07b4-44ec-9729-1a9230f4c02c => generated 1523 bytes in 336 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:25:08.089537 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-28458a9e-2742-4017-9eb3-bd39a9681e99 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:08.090359 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-28458a9e-2742-4017-9eb3-bd39a9681e99 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: c9901d0a-8a17-4e6c-ba98-6bd587d410e0] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:25:08.098655 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-28458a9e-2742-4017-9eb3-bd39a9681e99 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:08.099835 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-28458a9e-2742-4017-9eb3-bd39a9681e99 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:08.100945 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-28458a9e-2742-4017-9eb3-bd39a9681e99 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:08.101588 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-28458a9e-2742-4017-9eb3-bd39a9681e99 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:08.188114 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-28458a9e-2742-4017-9eb3-bd39a9681e99 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '027dcc48e4fd4c36a5004096ad5e43f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dad823e240149edb30e592380249a13', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:08.196330 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-28458a9e-2742-4017-9eb3-bd39a9681e99 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "GET /compute/v2.1/servers/c9901d0a-8a17-4e6c-ba98-6bd587d410e0" status: 200 len: 1472 microversion: 2.1 time: 0.111863 Jul 16 21:25:08.197537 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1753/3529] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:25:08 2022] GET /compute/v2.1/servers/c9901d0a-8a17-4e6c-ba98-6bd587d410e0 => generated 1472 bytes in 114 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:25:08.207130 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-fa85af3d-a810-4acd-a1f6-4df9ec95081c req-4d6fc56f-c360-4305-a49f-8b86eb6f4028 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "c9901d0a-8a17-4e6c-ba98-6bd587d410e0", "name": "network-vif-deleted", "tag": "443db70b-054d-4606-9eee-d742416cd5a6"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:25:08.216499 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-fa85af3d-a810-4acd-a1f6-4df9ec95081c req-4d6fc56f-c360-4305-a49f-8b86eb6f4028 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:08.217191 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-fa85af3d-a810-4acd-a1f6-4df9ec95081c req-4d6fc56f-c360-4305-a49f-8b86eb6f4028 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:08.265110 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-fa85af3d-a810-4acd-a1f6-4df9ec95081c req-4d6fc56f-c360-4305-a49f-8b86eb6f4028 service nova] Creating event network-vif-deleted:443db70b-054d-4606-9eee-d742416cd5a6 for instance c9901d0a-8a17-4e6c-ba98-6bd587d410e0 on nch2-master-1 Jul 16 21:25:08.277242 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-fa85af3d-a810-4acd-a1f6-4df9ec95081c req-4d6fc56f-c360-4305-a49f-8b86eb6f4028 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.076995 Jul 16 21:25:08.282606 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1754/3530] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:25:08 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 80 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:25:08.286380 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-e5149ae0-2763-4b7a-ad29-0a556272faaf tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:08.286538 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-e5149ae0-2763-4b7a-ad29-0a556272faaf tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] [instance: 4eb3d8d6-79ae-4109-a5a7-835c459d1719] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:25:08.294057 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e5149ae0-2763-4b7a-ad29-0a556272faaf tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:08.294562 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e5149ae0-2763-4b7a-ad29-0a556272faaf tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:08.295450 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e5149ae0-2763-4b7a-ad29-0a556272faaf tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:08.295900 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e5149ae0-2763-4b7a-ad29-0a556272faaf tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:08.467936 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell0:SELECT=4 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:25:08.519076 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-e5149ae0-2763-4b7a-ad29-0a556272faaf tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a23fafbab18d41ee82b84bbdf94cf0fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfc2e42a0df346f4adef152ba2e3fa71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:08.553653 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-e5149ae0-2763-4b7a-ad29-0a556272faaf tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'a23fafbab18d41ee82b84bbdf94cf0fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfc2e42a0df346f4adef152ba2e3fa71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:08.555829 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-e5149ae0-2763-4b7a-ad29-0a556272faaf tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'a23fafbab18d41ee82b84bbdf94cf0fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfc2e42a0df346f4adef152ba2e3fa71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:08.557915 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-e5149ae0-2763-4b7a-ad29-0a556272faaf tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] 10.222.0.16 "GET /compute/v2.1/servers/4eb3d8d6-79ae-4109-a5a7-835c459d1719" status: 200 len: 1660 microversion: 2.32 time: 0.277799 Jul 16 21:25:08.559055 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1755/3531] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:25:08 2022] GET /compute/v2.1/servers/4eb3d8d6-79ae-4109-a5a7-835c459d1719 => generated 1660 bytes in 281 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:25:08.762696 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-754b84b6-e152-4f6f-b47e-5b9e5df1f94b tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:08.763387 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-754b84b6-e152-4f6f-b47e-5b9e5df1f94b tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] [instance: f00bea08-4180-45d6-ba01-4b104f5615d6] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:25:08.781130 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-754b84b6-e152-4f6f-b47e-5b9e5df1f94b tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:08.781891 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-754b84b6-e152-4f6f-b47e-5b9e5df1f94b tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:08.787890 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-754b84b6-e152-4f6f-b47e-5b9e5df1f94b tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:08.788591 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-754b84b6-e152-4f6f-b47e-5b9e5df1f94b tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:09.056389 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-754b84b6-e152-4f6f-b47e-5b9e5df1f94b tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3b0b46c0c1f74adaaaf65d46db7b2618', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '336b3ef8a8744bf3b86854b32000bda6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:09.072479 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-754b84b6-e152-4f6f-b47e-5b9e5df1f94b tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] 10.222.0.16 "GET /compute/v2.1/servers/f00bea08-4180-45d6-ba01-4b104f5615d6" status: 200 len: 1316 microversion: 2.1 time: 0.314432 Jul 16 21:25:09.073717 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1756/3532] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:25:08 2022] GET /compute/v2.1/servers/f00bea08-4180-45d6-ba01-4b104f5615d6 => generated 1316 bytes in 316 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:25:09.099410 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-632ae5b9-b63c-4f2c-93f4-216dc6c2283b tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:09.100324 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-632ae5b9-b63c-4f2c-93f4-216dc6c2283b tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] [instance: 92fbfe5e-07b4-44ec-9729-1a9230f4c02c] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:25:09.112792 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-632ae5b9-b63c-4f2c-93f4-216dc6c2283b tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:09.113287 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-632ae5b9-b63c-4f2c-93f4-216dc6c2283b tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:09.114073 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-632ae5b9-b63c-4f2c-93f4-216dc6c2283b tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:09.114536 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-632ae5b9-b63c-4f2c-93f4-216dc6c2283b tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:09.119361 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=100 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:25:09.368962 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-121ab998-3faf-49a2-83d9-baf0d17b8e0b tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 8e6eab25-f25e-46a2-9c72-9424c872fb84] Doing local delete in cell 327e8df1-39cc-4b36-9ea6-6a9b88ed54fc(cell1) {{(pid=138680) _delete /opt/stack/nova/nova/compute/api.py:2489}} Jul 16 21:25:09.369926 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-121ab998-3faf-49a2-83d9-baf0d17b8e0b tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:09.370465 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-121ab998-3faf-49a2-83d9-baf0d17b8e0b tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:09.370982 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.compute.api [None req-121ab998-3faf-49a2-83d9-baf0d17b8e0b tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 8e6eab25-f25e-46a2-9c72-9424c872fb84] instance is in SHELVED_OFFLOADED state, cleanup the instance's info from database. Jul 16 21:25:09.375723 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.network.neutron [None req-121ab998-3faf-49a2-83d9-baf0d17b8e0b tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 8e6eab25-f25e-46a2-9c72-9424c872fb84] deallocate_for_instance() {{(pid=138680) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1783}} Jul 16 21:25:09.466678 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-632ae5b9-b63c-4f2c-93f4-216dc6c2283b tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcc9aa2a091c490599f9a9c41163e2eb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da56fc78f06142aba80ca4c6e3d60475', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:09.481530 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-632ae5b9-b63c-4f2c-93f4-216dc6c2283b tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] 10.222.0.16 "GET /compute/v2.1/servers/92fbfe5e-07b4-44ec-9729-1a9230f4c02c" status: 200 len: 1523 microversion: 2.1 time: 0.379744 Jul 16 21:25:09.482220 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1757/3533] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:25:09 2022] GET /compute/v2.1/servers/92fbfe5e-07b4-44ec-9729-1a9230f4c02c => generated 1523 bytes in 387 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:25:09.495486 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-eb684fc3-0613-43a2-9f59-da1452baf6b4 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:09.496349 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-eb684fc3-0613-43a2-9f59-da1452baf6b4 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: c9901d0a-8a17-4e6c-ba98-6bd587d410e0] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:25:09.508455 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-eb684fc3-0613-43a2-9f59-da1452baf6b4 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:09.510181 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-eb684fc3-0613-43a2-9f59-da1452baf6b4 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:09.512365 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-eb684fc3-0613-43a2-9f59-da1452baf6b4 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:09.513094 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-eb684fc3-0613-43a2-9f59-da1452baf6b4 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:09.558096 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-eb684fc3-0613-43a2-9f59-da1452baf6b4 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:09.558689 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-eb684fc3-0613-43a2-9f59-da1452baf6b4 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:09.559201 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.objects.instance [None req-eb684fc3-0613-43a2-9f59-da1452baf6b4 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lazy-loading 'fault' on Instance uuid c9901d0a-8a17-4e6c-ba98-6bd587d410e0 {{(pid=138681) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:25:09.639409 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-eb684fc3-0613-43a2-9f59-da1452baf6b4 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '027dcc48e4fd4c36a5004096ad5e43f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dad823e240149edb30e592380249a13', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:09.646514 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-eb684fc3-0613-43a2-9f59-da1452baf6b4 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "GET /compute/v2.1/servers/c9901d0a-8a17-4e6c-ba98-6bd587d410e0" status: 200 len: 1318 microversion: 2.1 time: 0.163241 Jul 16 21:25:09.647342 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1758/3534] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:25:09 2022] GET /compute/v2.1/servers/c9901d0a-8a17-4e6c-ba98-6bd587d410e0 => generated 1318 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:25:09.653595 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-8a5a8665-154b-4ce1-8beb-39d48cac0a0e tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:09.655193 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-8a5a8665-154b-4ce1-8beb-39d48cac0a0e tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] [instance: 4eb3d8d6-79ae-4109-a5a7-835c459d1719] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:25:09.665300 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8a5a8665-154b-4ce1-8beb-39d48cac0a0e tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:09.667973 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8a5a8665-154b-4ce1-8beb-39d48cac0a0e tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:09.667973 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8a5a8665-154b-4ce1-8beb-39d48cac0a0e tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:09.667973 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8a5a8665-154b-4ce1-8beb-39d48cac0a0e tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:09.946487 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-8a5a8665-154b-4ce1-8beb-39d48cac0a0e tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a23fafbab18d41ee82b84bbdf94cf0fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfc2e42a0df346f4adef152ba2e3fa71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:09.955666 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-8a5a8665-154b-4ce1-8beb-39d48cac0a0e tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'a23fafbab18d41ee82b84bbdf94cf0fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfc2e42a0df346f4adef152ba2e3fa71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:09.957096 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-8a5a8665-154b-4ce1-8beb-39d48cac0a0e tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'a23fafbab18d41ee82b84bbdf94cf0fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfc2e42a0df346f4adef152ba2e3fa71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:09.958240 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-8a5a8665-154b-4ce1-8beb-39d48cac0a0e tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] 10.222.0.16 "GET /compute/v2.1/servers/4eb3d8d6-79ae-4109-a5a7-835c459d1719" status: 200 len: 1706 microversion: 2.32 time: 0.309515 Jul 16 21:25:09.958807 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1759/3535] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:25:09 2022] GET /compute/v2.1/servers/4eb3d8d6-79ae-4109-a5a7-835c459d1719 => generated 1706 bytes in 311 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:25:10.093485 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-5b1dd721-81c8-4b18-a4a0-ed375215a434 tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:10.094191 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-5b1dd721-81c8-4b18-a4a0-ed375215a434 tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] [instance: f00bea08-4180-45d6-ba01-4b104f5615d6] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:25:10.101912 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5b1dd721-81c8-4b18-a4a0-ed375215a434 tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:10.102835 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5b1dd721-81c8-4b18-a4a0-ed375215a434 tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:10.103714 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5b1dd721-81c8-4b18-a4a0-ed375215a434 tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:10.104159 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5b1dd721-81c8-4b18-a4a0-ed375215a434 tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:10.313923 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-5b1dd721-81c8-4b18-a4a0-ed375215a434 tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3b0b46c0c1f74adaaaf65d46db7b2618', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '336b3ef8a8744bf3b86854b32000bda6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:10.326173 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-5b1dd721-81c8-4b18-a4a0-ed375215a434 tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] 10.222.0.16 "GET /compute/v2.1/servers/f00bea08-4180-45d6-ba01-4b104f5615d6" status: 200 len: 1466 microversion: 2.1 time: 0.237214 Jul 16 21:25:10.331127 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1760/3536] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:25:10 2022] GET /compute/v2.1/servers/f00bea08-4180-45d6-ba01-4b104f5615d6 => generated 1466 bytes in 240 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:25:10.345442 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-ba31e211-877d-496d-b569-533481bad78f tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Action: 'action', calling method: >, body: {"createImage": {"name": "tempest-image-602177477", "metadata": {"image_type": "test"}}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:25:10.346971 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-ba31e211-877d-496d-b569-533481bad78f tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] [instance: f00bea08-4180-45d6-ba01-4b104f5615d6] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:25:10.354291 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ba31e211-877d-496d-b569-533481bad78f tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:10.359297 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ba31e211-877d-496d-b569-533481bad78f tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:10.359297 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ba31e211-877d-496d-b569-533481bad78f tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:10.359297 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ba31e211-877d-496d-b569-533481bad78f tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:10.429374 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_api:SELECT=94,nova_api:INSERT=6 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:25:10.436368 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-ba31e211-877d-496d-b569-533481bad78f tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Getting quotas for project 336b3ef8a8744bf3b86854b32000bda6. Resources: dict_keys(['metadata_items']) {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:25:10.442269 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-ba31e211-877d-496d-b569-533481bad78f tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Getting quotas for user 3b0b46c0c1f74adaaaf65d46db7b2618 and project 336b3ef8a8744bf3b86854b32000bda6. Resources: dict_keys(['metadata_items']) {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:25:10.471184 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.network.neutron [None req-121ab998-3faf-49a2-83d9-baf0d17b8e0b tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 8e6eab25-f25e-46a2-9c72-9424c872fb84] Updating instance_info_cache with network_info: [] {{(pid=138680) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:117}} Jul 16 21:25:11.053364 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-121ab998-3faf-49a2-83d9-baf0d17b8e0b tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "DELETE /compute/v2.1/servers/8e6eab25-f25e-46a2-9c72-9424c872fb84" status: 204 len: 0 microversion: 2.1 time: 3.792708 Jul 16 21:25:11.054277 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1777/3537] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:25:07 2022] DELETE /compute/v2.1/servers/8e6eab25-f25e-46a2-9c72-9424c872fb84 => generated 0 bytes in 3794 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 16 21:25:11.058263 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-ba31e211-877d-496d-b569-533481bad78f tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] 10.222.0.16 "POST /compute/v2.1/servers/f00bea08-4180-45d6-ba01-4b104f5615d6/action" status: 202 len: 0 microversion: 2.1 time: 0.716605 Jul 16 21:25:11.058860 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1761/3538] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 21:25:10 2022] POST /compute/v2.1/servers/f00bea08-4180-45d6-ba01-4b104f5615d6/action => generated 0 bytes in 718 msecs (HTTP/1.1 202) 10 headers in 450 bytes (2 switches on core 0) Jul 16 21:25:11.065809 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-e9b49805-f5f5-4b23-b3e6-7a88ffb12262 req-07a6da0c-4dbe-49a3-ac25-cf17339b36f3 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "8e6eab25-f25e-46a2-9c72-9424c872fb84", "name": "network-vif-deleted", "tag": "4a81f7ad-929b-4f37-b5ac-e484b09be512"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:25:11.077470 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-e9b49805-f5f5-4b23-b3e6-7a88ffb12262 req-07a6da0c-4dbe-49a3-ac25-cf17339b36f3 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:11.079775 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-e9b49805-f5f5-4b23-b3e6-7a88ffb12262 req-07a6da0c-4dbe-49a3-ac25-cf17339b36f3 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:11.079928 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-59599a36-17c0-4eb2-9cfe-e14889153b6f tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:11.079928 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-59599a36-17c0-4eb2-9cfe-e14889153b6f tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] [instance: 92fbfe5e-07b4-44ec-9729-1a9230f4c02c] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:25:11.087095 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-59599a36-17c0-4eb2-9cfe-e14889153b6f tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:11.087559 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-59599a36-17c0-4eb2-9cfe-e14889153b6f tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:11.088339 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-59599a36-17c0-4eb2-9cfe-e14889153b6f tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:11.089003 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-59599a36-17c0-4eb2-9cfe-e14889153b6f tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:11.091962 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.compute.server_external_events [req-e9b49805-f5f5-4b23-b3e6-7a88ffb12262 req-07a6da0c-4dbe-49a3-ac25-cf17339b36f3 service nova] Unable to find a host for instance 8e6eab25-f25e-46a2-9c72-9424c872fb84. Dropping event network-vif-deleted {{(pid=138680) create /opt/stack/nova/nova/api/openstack/compute/server_external_events.py:143}} Jul 16 21:25:11.093837 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-e9b49805-f5f5-4b23-b3e6-7a88ffb12262 req-07a6da0c-4dbe-49a3-ac25-cf17339b36f3 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 207 len: 180 microversion: 2.1 time: 0.037444 Jul 16 21:25:11.094690 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1778/3539] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:25:11 2022] POST /compute/v2.1/os-server-external-events => generated 180 bytes in 40 msecs (HTTP/1.1 207) 9 headers in 367 bytes (1 switches on core 0) Jul 16 21:25:11.099850 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-c117ce8e-7f66-42df-ac3a-201a77cd3252 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:11.100534 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-c117ce8e-7f66-42df-ac3a-201a77cd3252 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: c9901d0a-8a17-4e6c-ba98-6bd587d410e0] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:25:11.109113 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c117ce8e-7f66-42df-ac3a-201a77cd3252 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:11.109960 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c117ce8e-7f66-42df-ac3a-201a77cd3252 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:11.111171 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c117ce8e-7f66-42df-ac3a-201a77cd3252 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:11.111887 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c117ce8e-7f66-42df-ac3a-201a77cd3252 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:11.136357 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-c117ce8e-7f66-42df-ac3a-201a77cd3252 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] HTTP exception thrown: Instance c9901d0a-8a17-4e6c-ba98-6bd587d410e0 could not be found. Jul 16 21:25:11.137316 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-c117ce8e-7f66-42df-ac3a-201a77cd3252 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Request method failure captured: Jul 16 21:25:11.137316 ncdv-master-1 devstack@n-api.service[138680]: request: GET /compute/v2.1/servers/c9901d0a-8a17-4e6c-ba98-6bd587d410e0 HTTP/1.1 Jul 16 21:25:11.137316 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:25:11.137316 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:25:11.137316 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:25:11.137316 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 0 Jul 16 21:25:11.137316 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:25:11.137316 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:25:11.137316 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:25:11.137316 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:25:11.137316 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0yxS9NCWgI1-O55Z_AF16JzmczB_p-s4P30aJMz-9zu9VxcHuIV00tsfB9hOBtHTZ_sI8GPeOOvUrqYZLgOBzCbOMr4ZO506_XIRjMCtHQ0umzvcB9V9iFSNFoiIe_b2jKm-OyHj1IB8YVBCAcwDKANlsCw5UaAoIABK4dLLFjaVpEo Jul 16 21:25:11.137316 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:25:11.137316 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:25:11.137316 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:25:11.137316 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:25:11.137316 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:25:11.137316 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:25:11.137316 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: 1dad823e240149edb30e592380249a13 Jul 16 21:25:11.137316 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-ServersTestJSON-1529284206 Jul 16 21:25:11.137316 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:25:11.137316 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:25:11.137316 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/1dad823e240149edb30e592380249a13"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/1dad823e240149edb30e592380249a13"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_1dad823e240149edb30e592380249a13"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/1dad823e240149edb30e592380249a13"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:25:11.137316 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-ServersTestJSON-1529284206 Jul 16 21:25:11.137316 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: 1dad823e240149edb30e592380249a13 Jul 16 21:25:11.137316 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-ServersTestJSON-1529284206 Jul 16 21:25:11.137316 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-ServersTestJSON-1529284206-project Jul 16 21:25:11.137316 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:25:11.137316 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:25:11.137316 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: 027dcc48e4fd4c36a5004096ad5e43f8 Jul 16 21:25:11.137316 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-ServersTestJSON-1529284206-project Jul 16 21:25:11.137316 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 21:25:11.137316 ncdv-master-1 devstack@n-api.service[138680]: exception: Instance c9901d0a-8a17-4e6c-ba98-6bd587d410e0 could not be found. Jul 16 21:25:11.137316 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': 'c9901d0a-8a17-4e6c-ba98-6bd587d410e0'} Jul 16 21:25:11.137316 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:25:11.137316 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:25:11.137316 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:25:11.137316 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:25:11.137316 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:25:11.137316 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:25:11.137316 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:25:11.137316 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:25:11.137316 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:25:11.137316 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:25:11.137316 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance c9901d0a-8a17-4e6c-ba98-6bd587d410e0 could not be found. Jul 16 21:25:11.137316 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:25:11.137316 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:25:11.137316 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:25:11.137316 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:25:11.137316 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:25:11.137316 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:25:11.137316 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:25:11.137316 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:25:11.137316 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:25:11.137316 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:25:11.137316 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:25:11.137316 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:25:11.137316 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:25:11.137316 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:25:11.137316 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:25:11.137316 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:25:11.137316 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance c9901d0a-8a17-4e6c-ba98-6bd587d410e0 could not be found. Jul 16 21:25:11.137316 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:25:11.137316 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:25:11.137316 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:25:11.137316 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:25:11.142765 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:25:11.142765 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:25:11.142765 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:25:11.142765 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:25:11.142765 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance c9901d0a-8a17-4e6c-ba98-6bd587d410e0 could not be found. Jul 16 21:25:11.142765 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:25:11.142765 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-c117ce8e-7f66-42df-ac3a-201a77cd3252 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Returning 404 to user: Instance c9901d0a-8a17-4e6c-ba98-6bd587d410e0 could not be found. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:25:11.142765 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-c117ce8e-7f66-42df-ac3a-201a77cd3252 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "GET /compute/v2.1/servers/c9901d0a-8a17-4e6c-ba98-6bd587d410e0" status: 404 len: 111 microversion: 2.1 time: 0.042339 Jul 16 21:25:11.142765 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1779/3540] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:25:11 2022] GET /compute/v2.1/servers/c9901d0a-8a17-4e6c-ba98-6bd587d410e0 => generated 111 bytes in 43 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:25:11.148215 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-5d8b5f7c-4423-4e9a-9a16-ffce62aa1734 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:11.148215 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-5d8b5f7c-4423-4e9a-9a16-ffce62aa1734 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] [instance: 4eb3d8d6-79ae-4109-a5a7-835c459d1719] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:25:11.156601 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell0:SELECT=24 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:25:11.160980 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5d8b5f7c-4423-4e9a-9a16-ffce62aa1734 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:11.161741 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5d8b5f7c-4423-4e9a-9a16-ffce62aa1734 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:11.164674 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5d8b5f7c-4423-4e9a-9a16-ffce62aa1734 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:11.164674 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5d8b5f7c-4423-4e9a-9a16-ffce62aa1734 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:11.410680 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-59599a36-17c0-4eb2-9cfe-e14889153b6f tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcc9aa2a091c490599f9a9c41163e2eb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da56fc78f06142aba80ca4c6e3d60475', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:11.424328 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-59599a36-17c0-4eb2-9cfe-e14889153b6f tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] 10.222.0.16 "GET /compute/v2.1/servers/92fbfe5e-07b4-44ec-9729-1a9230f4c02c" status: 200 len: 1523 microversion: 2.1 time: 0.349838 Jul 16 21:25:11.430817 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1762/3541] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:25:11 2022] GET /compute/v2.1/servers/92fbfe5e-07b4-44ec-9729-1a9230f4c02c => generated 1523 bytes in 371 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:25:11.439200 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-12125933-a857-45ea-ba98-5ad283871bd7 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:11.439832 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-12125933-a857-45ea-ba98-5ad283871bd7 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 8e6eab25-f25e-46a2-9c72-9424c872fb84] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:25:11.454307 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-5d8b5f7c-4423-4e9a-9a16-ffce62aa1734 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a23fafbab18d41ee82b84bbdf94cf0fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfc2e42a0df346f4adef152ba2e3fa71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:11.468319 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-12125933-a857-45ea-ba98-5ad283871bd7 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:11.468775 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-12125933-a857-45ea-ba98-5ad283871bd7 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:11.469739 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-12125933-a857-45ea-ba98-5ad283871bd7 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:11.470179 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-12125933-a857-45ea-ba98-5ad283871bd7 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:11.473526 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-5d8b5f7c-4423-4e9a-9a16-ffce62aa1734 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'a23fafbab18d41ee82b84bbdf94cf0fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfc2e42a0df346f4adef152ba2e3fa71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:11.475043 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-5d8b5f7c-4423-4e9a-9a16-ffce62aa1734 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'a23fafbab18d41ee82b84bbdf94cf0fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfc2e42a0df346f4adef152ba2e3fa71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:11.482378 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-5d8b5f7c-4423-4e9a-9a16-ffce62aa1734 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] 10.222.0.16 "GET /compute/v2.1/servers/4eb3d8d6-79ae-4109-a5a7-835c459d1719" status: 200 len: 1798 microversion: 2.32 time: 0.342468 Jul 16 21:25:11.495763 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1780/3542] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:25:11 2022] GET /compute/v2.1/servers/4eb3d8d6-79ae-4109-a5a7-835c459d1719 => generated 1798 bytes in 344 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:25:11.501236 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-1390a584-bbd3-471a-9b92-6ddcb3062f7a tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Calling method '.version_select at 0x7fdd9d06aca0>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:11.517624 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-12125933-a857-45ea-ba98-5ad283871bd7 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] HTTP exception thrown: Instance 8e6eab25-f25e-46a2-9c72-9424c872fb84 could not be found. Jul 16 21:25:11.519747 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-12125933-a857-45ea-ba98-5ad283871bd7 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Request method failure captured: Jul 16 21:25:11.519747 ncdv-master-1 devstack@n-api.service[138681]: request: GET /compute/v2.1/servers/8e6eab25-f25e-46a2-9c72-9424c872fb84 HTTP/1.1 Jul 16 21:25:11.519747 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:25:11.519747 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:25:11.519747 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:25:11.519747 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 21:25:11.519747 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:25:11.519747 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:25:11.519747 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:25:11.519747 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:25:11.519747 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0yvs4CI48W3ZCHoQ8huRZwHvqHxAdwrwrvk1v_ETBPx6FU4DdiPUjkjq4hN_4g_fHLAB_XZ9-AAN8Pyvy0KON2X25WlmyilO65MDHbIW9Xy9v9USVPAu_Ce-ze5X9UIHyned0AUneE4UPGiLLIkoXeoS5ekiq1pi209pud2lyUAuHgw Jul 16 21:25:11.519747 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:25:11.519747 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:25:11.519747 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:25:11.519747 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:25:11.519747 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:25:11.519747 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:25:11.519747 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: d31be8475a6a49d6bfe8132dd050c6c0 Jul 16 21:25:11.519747 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-DeleteServersTestJSON-1541892181 Jul 16 21:25:11.519747 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:25:11.519747 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:25:11.519747 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/d31be8475a6a49d6bfe8132dd050c6c0"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/d31be8475a6a49d6bfe8132dd050c6c0"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_d31be8475a6a49d6bfe8132dd050c6c0"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/d31be8475a6a49d6bfe8132dd050c6c0"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:25:11.519747 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-DeleteServersTestJSON-1541892181 Jul 16 21:25:11.519747 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: d31be8475a6a49d6bfe8132dd050c6c0 Jul 16 21:25:11.519747 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-DeleteServersTestJSON-1541892181 Jul 16 21:25:11.519747 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-DeleteServersTestJSON-1541892181-project Jul 16 21:25:11.519747 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:25:11.519747 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:25:11.519747 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: 6f2e974ab5aa48dfb4b00f810dcedb63 Jul 16 21:25:11.519747 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-DeleteServersTestJSON-1541892181-project Jul 16 21:25:11.519747 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:25:11.519747 ncdv-master-1 devstack@n-api.service[138681]: exception: Instance 8e6eab25-f25e-46a2-9c72-9424c872fb84 could not be found. Jul 16 21:25:11.519747 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': '8e6eab25-f25e-46a2-9c72-9424c872fb84'} Jul 16 21:25:11.519747 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:25:11.519747 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:25:11.519747 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:25:11.519747 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:25:11.519747 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:25:11.519747 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:25:11.519747 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:25:11.519747 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:25:11.519747 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:25:11.519747 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:25:11.519747 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 8e6eab25-f25e-46a2-9c72-9424c872fb84 could not be found. Jul 16 21:25:11.519747 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:25:11.519747 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:25:11.519747 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:25:11.519747 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:25:11.519747 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:25:11.519747 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:25:11.519747 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:25:11.519747 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:25:11.519747 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:25:11.519747 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:25:11.519747 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:25:11.519747 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:25:11.519747 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:25:11.519747 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:25:11.519747 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:25:11.519747 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:25:11.519747 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 8e6eab25-f25e-46a2-9c72-9424c872fb84 could not be found. Jul 16 21:25:11.519747 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:25:11.519747 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:25:11.519747 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:25:11.526860 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:25:11.526860 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:25:11.526860 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:25:11.526860 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:25:11.526860 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:25:11.526860 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 8e6eab25-f25e-46a2-9c72-9424c872fb84 could not be found. Jul 16 21:25:11.526860 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:25:11.526860 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-12125933-a857-45ea-ba98-5ad283871bd7 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Returning 404 to user: Instance 8e6eab25-f25e-46a2-9c72-9424c872fb84 could not be found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:25:11.526860 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-12125933-a857-45ea-ba98-5ad283871bd7 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "GET /compute/v2.1/servers/8e6eab25-f25e-46a2-9c72-9424c872fb84" status: 404 len: 111 microversion: 2.1 time: 0.089277 Jul 16 21:25:11.526860 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1763/3543] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:25:11 2022] GET /compute/v2.1/servers/8e6eab25-f25e-46a2-9c72-9424c872fb84 => generated 111 bytes in 90 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:25:11.535927 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-6430ec9d-3f11-48f6-ba4c-5503b055c510 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '.version_select at 0x7fdda2b89670>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:11.593874 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-1390a584-bbd3-471a-9b92-6ddcb3062f7a tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] 10.222.0.16 "GET /compute/v2.1/images/706863d6-a902-4b5c-abc5-1e3e8a4c0e7a" status: 200 len: 1421 microversion: 2.1 time: 0.105849 Jul 16 21:25:11.594551 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1781/3544] 10.222.0.16 () {58 vars in 1232 bytes} [Sat Jul 16 21:25:11 2022] GET /compute/v2.1/images/706863d6-a902-4b5c-abc5-1e3e8a4c0e7a => generated 1421 bytes in 111 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:25:11.667469 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-6d0c11e0-f010-46d0-9a6f-e6a0847e5fdb tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-DeleteServersTestJSON-server-344441769", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452", "networks": [{"uuid": "fc3b835d-0a7f-4948-9fcb-1edfd6b9a8df"}]}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:25:11.685102 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-6430ec9d-3f11-48f6-ba4c-5503b055c510 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "DELETE /compute/v2.1/os-keypairs/tempest-key-1030576937" status: 202 len: 0 microversion: 2.1 time: 0.160393 Jul 16 21:25:11.685783 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1764/3545] 10.222.0.16 () {58 vars in 1208 bytes} [Sat Jul 16 21:25:11 2022] DELETE /compute/v2.1/os-keypairs/tempest-key-1030576937 => generated 0 bytes in 163 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 21:25:11.725178 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.network.neutron [None req-6d0c11e0-f010-46d0-9a6f-e6a0847e5fdb tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] validate_networks() for [('fc3b835d-0a7f-4948-9fcb-1edfd6b9a8df', None, None, None, None, None)] {{(pid=138680) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 16 21:25:11.798966 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-4e053337-103e-4846-a083-11bcdd214d64 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServersTestJSON-server-1313984590", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452", "networks": [{"uuid": "2202d4ad-313c-4607-b9fc-eb98bac9afd0"}]}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:25:11.897065 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.network.neutron [None req-4e053337-103e-4846-a083-11bcdd214d64 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] validate_networks() for [('2202d4ad-313c-4607-b9fc-eb98bac9afd0', None, None, None, None, None)] {{(pid=138681) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 16 21:25:12.301428 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell0:SELECT=2 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:25:12.457060 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-6d0c11e0-f010-46d0-9a6f-e6a0847e5fdb tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Flavor limits 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 16 21:25:12.457503 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-6d0c11e0-f010-46d0-9a6f-e6a0847e5fdb tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Image limits 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 16 21:25:12.457994 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-6d0c11e0-f010-46d0-9a6f-e6a0847e5fdb tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Flavor pref 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 16 21:25:12.458411 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-6d0c11e0-f010-46d0-9a6f-e6a0847e5fdb tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Image pref 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 16 21:25:12.458835 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-6d0c11e0-f010-46d0-9a6f-e6a0847e5fdb tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 16 21:25:12.472250 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6d0c11e0-f010-46d0-9a6f-e6a0847e5fdb tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:12.472706 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6d0c11e0-f010-46d0-9a6f-e6a0847e5fdb tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:12.473763 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6d0c11e0-f010-46d0-9a6f-e6a0847e5fdb tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:12.475208 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6d0c11e0-f010-46d0-9a6f-e6a0847e5fdb tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:12.505273 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-6d0c11e0-f010-46d0-9a6f-e6a0847e5fdb tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Getting quotas for project d31be8475a6a49d6bfe8132dd050c6c0. Resources: {'ram', 'instances', 'cores'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:25:12.514987 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-6d0c11e0-f010-46d0-9a6f-e6a0847e5fdb tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Getting quotas for user 6f2e974ab5aa48dfb4b00f810dcedb63 and project d31be8475a6a49d6bfe8132dd050c6c0. Resources: {'ram', 'instances', 'cores'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:25:12.539833 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-6d0c11e0-f010-46d0-9a6f-e6a0847e5fdb tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Going to run 1 instances... {{(pid=138680) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 16 21:25:12.542562 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-6d0c11e0-f010-46d0-9a6f-e6a0847e5fdb tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 00c8a753-5bf0-448c-8107-3946aa9bf7ae] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138680) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 16 21:25:12.577296 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-4e053337-103e-4846-a083-11bcdd214d64 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Flavor limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 16 21:25:12.577792 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-4e053337-103e-4846-a083-11bcdd214d64 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Image limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 16 21:25:12.578198 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-4e053337-103e-4846-a083-11bcdd214d64 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Flavor pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 16 21:25:12.578711 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-4e053337-103e-4846-a083-11bcdd214d64 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Image pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 16 21:25:12.579158 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-4e053337-103e-4846-a083-11bcdd214d64 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 16 21:25:12.589848 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4e053337-103e-4846-a083-11bcdd214d64 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:12.590904 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4e053337-103e-4846-a083-11bcdd214d64 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:12.594426 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4e053337-103e-4846-a083-11bcdd214d64 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:12.594426 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4e053337-103e-4846-a083-11bcdd214d64 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:12.612007 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-4e053337-103e-4846-a083-11bcdd214d64 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Getting quotas for project 1dad823e240149edb30e592380249a13. Resources: {'ram', 'instances', 'cores'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:25:12.618079 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-4e053337-103e-4846-a083-11bcdd214d64 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Getting quotas for user 027dcc48e4fd4c36a5004096ad5e43f8 and project 1dad823e240149edb30e592380249a13. Resources: {'ram', 'instances', 'cores'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:25:12.632351 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-4e053337-103e-4846-a083-11bcdd214d64 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Going to run 1 instances... {{(pid=138681) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 16 21:25:12.635734 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-4e053337-103e-4846-a083-11bcdd214d64 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: 05358d0e-ed2f-443c-835c-f77cd929289e] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138681) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 16 21:25:12.758804 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-6d0c11e0-f010-46d0-9a6f-e6a0847e5fdb tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 1.095349 Jul 16 21:25:12.760139 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1782/3546] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 21:25:11 2022] POST /compute/v2.1/servers => generated 380 bytes in 1098 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Jul 16 21:25:12.764115 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-c32e7794-202a-4935-af0b-b32647e39d33 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:12.764722 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-c32e7794-202a-4935-af0b-b32647e39d33 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] [instance: 92fbfe5e-07b4-44ec-9729-1a9230f4c02c] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:25:12.772213 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c32e7794-202a-4935-af0b-b32647e39d33 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:12.774402 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c32e7794-202a-4935-af0b-b32647e39d33 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:12.775424 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c32e7794-202a-4935-af0b-b32647e39d33 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:12.775864 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c32e7794-202a-4935-af0b-b32647e39d33 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:12.796344 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-4e053337-103e-4846-a083-11bcdd214d64 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 1.001848 Jul 16 21:25:12.802447 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1765/3547] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 21:25:11 2022] POST /compute/v2.1/servers => generated 380 bytes in 1009 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Jul 16 21:25:12.808038 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-a78a3acd-cd2d-4669-8b70-06a49f7dc090 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:12.808857 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-a78a3acd-cd2d-4669-8b70-06a49f7dc090 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] [instance: 4eb3d8d6-79ae-4109-a5a7-835c459d1719] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:25:12.818275 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a78a3acd-cd2d-4669-8b70-06a49f7dc090 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:12.823209 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a78a3acd-cd2d-4669-8b70-06a49f7dc090 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:12.823209 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a78a3acd-cd2d-4669-8b70-06a49f7dc090 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:12.823209 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a78a3acd-cd2d-4669-8b70-06a49f7dc090 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:13.116333 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-c32e7794-202a-4935-af0b-b32647e39d33 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcc9aa2a091c490599f9a9c41163e2eb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da56fc78f06142aba80ca4c6e3d60475', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:13.120640 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-c32e7794-202a-4935-af0b-b32647e39d33 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] 10.222.0.16 "GET /compute/v2.1/servers/92fbfe5e-07b4-44ec-9729-1a9230f4c02c" status: 200 len: 1523 microversion: 2.1 time: 0.360034 Jul 16 21:25:13.130152 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1783/3548] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:25:12 2022] GET /compute/v2.1/servers/92fbfe5e-07b4-44ec-9729-1a9230f4c02c => generated 1523 bytes in 361 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:25:13.133786 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-de69a098-ad44-421b-a26e-a6e4f567c0b2 tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Calling method '.version_select at 0x7fdd9d0ad3a0>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:13.156961 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-a78a3acd-cd2d-4669-8b70-06a49f7dc090 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a23fafbab18d41ee82b84bbdf94cf0fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfc2e42a0df346f4adef152ba2e3fa71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:13.166884 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-a78a3acd-cd2d-4669-8b70-06a49f7dc090 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'a23fafbab18d41ee82b84bbdf94cf0fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfc2e42a0df346f4adef152ba2e3fa71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:13.168624 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-a78a3acd-cd2d-4669-8b70-06a49f7dc090 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'a23fafbab18d41ee82b84bbdf94cf0fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfc2e42a0df346f4adef152ba2e3fa71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:13.169925 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-a78a3acd-cd2d-4669-8b70-06a49f7dc090 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] 10.222.0.16 "GET /compute/v2.1/servers/4eb3d8d6-79ae-4109-a5a7-835c459d1719" status: 200 len: 1844 microversion: 2.32 time: 0.366158 Jul 16 21:25:13.170823 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1766/3549] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:25:12 2022] GET /compute/v2.1/servers/4eb3d8d6-79ae-4109-a5a7-835c459d1719 => generated 1844 bytes in 368 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:25:13.180114 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-28c21bac-f7af-4b91-96f6-773aa711fbc1 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:13.180114 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-28c21bac-f7af-4b91-96f6-773aa711fbc1 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 00c8a753-5bf0-448c-8107-3946aa9bf7ae] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:25:13.189447 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-de69a098-ad44-421b-a26e-a6e4f567c0b2 tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] 10.222.0.16 "GET /compute/v2.1/images/706863d6-a902-4b5c-abc5-1e3e8a4c0e7a" status: 200 len: 1421 microversion: 2.1 time: 0.067094 Jul 16 21:25:13.190074 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1784/3550] 10.222.0.16 () {58 vars in 1232 bytes} [Sat Jul 16 21:25:13 2022] GET /compute/v2.1/images/706863d6-a902-4b5c-abc5-1e3e8a4c0e7a => generated 1421 bytes in 68 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:25:13.196173 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-d5ef95cf-de25-4a7a-ac2f-3dbc85dd0193 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:13.196932 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-d5ef95cf-de25-4a7a-ac2f-3dbc85dd0193 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: 05358d0e-ed2f-443c-835c-f77cd929289e] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:25:13.237801 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-28c21bac-f7af-4b91-96f6-773aa711fbc1 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f2e974ab5aa48dfb4b00f810dcedb63', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd31be8475a6a49d6bfe8132dd050c6c0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:13.247665 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-28c21bac-f7af-4b91-96f6-773aa711fbc1 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "GET /compute/v2.1/servers/00c8a753-5bf0-448c-8107-3946aa9bf7ae" status: 200 len: 1214 microversion: 2.1 time: 0.075483 Jul 16 21:25:13.248258 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1767/3551] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:25:13 2022] GET /compute/v2.1/servers/00c8a753-5bf0-448c-8107-3946aa9bf7ae => generated 1214 bytes in 77 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:25:13.282297 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-d5ef95cf-de25-4a7a-ac2f-3dbc85dd0193 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '027dcc48e4fd4c36a5004096ad5e43f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dad823e240149edb30e592380249a13', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:13.290039 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-d5ef95cf-de25-4a7a-ac2f-3dbc85dd0193 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "GET /compute/v2.1/servers/05358d0e-ed2f-443c-835c-f77cd929289e" status: 200 len: 1209 microversion: 2.1 time: 0.098513 Jul 16 21:25:13.299078 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1785/3552] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:25:13 2022] GET /compute/v2.1/servers/05358d0e-ed2f-443c-835c-f77cd929289e => generated 1209 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:25:13.811534 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-94f36a55-7830-4780-9175-d2acc0082f8f req-bb47bc96-1361-47eb-a7fa-8a0164ab3682 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "4eb3d8d6-79ae-4109-a5a7-835c459d1719", "tag": "fb878298-9eb7-4cd5-96d9-305373704495"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:25:13.822539 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-94f36a55-7830-4780-9175-d2acc0082f8f req-bb47bc96-1361-47eb-a7fa-8a0164ab3682 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:13.826040 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-94f36a55-7830-4780-9175-d2acc0082f8f req-bb47bc96-1361-47eb-a7fa-8a0164ab3682 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:13.839700 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-94f36a55-7830-4780-9175-d2acc0082f8f req-bb47bc96-1361-47eb-a7fa-8a0164ab3682 service nova] Creating event network-changed:fb878298-9eb7-4cd5-96d9-305373704495 for instance 4eb3d8d6-79ae-4109-a5a7-835c459d1719 on nch1-master-1 Jul 16 21:25:13.857727 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-94f36a55-7830-4780-9175-d2acc0082f8f req-bb47bc96-1361-47eb-a7fa-8a0164ab3682 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.049987 Jul 16 21:25:13.868127 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1768/3553] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:25:13 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 61 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:25:14.138084 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-4438242d-ea41-483d-97a9-9a1ea0751dc1 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:14.139559 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-4438242d-ea41-483d-97a9-9a1ea0751dc1 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] [instance: 92fbfe5e-07b4-44ec-9729-1a9230f4c02c] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:25:14.159941 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4438242d-ea41-483d-97a9-9a1ea0751dc1 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:14.160630 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4438242d-ea41-483d-97a9-9a1ea0751dc1 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:14.163144 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4438242d-ea41-483d-97a9-9a1ea0751dc1 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:14.164640 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4438242d-ea41-483d-97a9-9a1ea0751dc1 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:14.190852 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-69b31753-fd1e-4ac8-b9f7-c74f2ff50600 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:14.192146 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-69b31753-fd1e-4ac8-b9f7-c74f2ff50600 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] [instance: 4eb3d8d6-79ae-4109-a5a7-835c459d1719] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:25:14.221978 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-69b31753-fd1e-4ac8-b9f7-c74f2ff50600 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:14.229310 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-69b31753-fd1e-4ac8-b9f7-c74f2ff50600 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:14.230834 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-69b31753-fd1e-4ac8-b9f7-c74f2ff50600 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:14.234577 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-69b31753-fd1e-4ac8-b9f7-c74f2ff50600 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:14.637123 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-4438242d-ea41-483d-97a9-9a1ea0751dc1 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcc9aa2a091c490599f9a9c41163e2eb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da56fc78f06142aba80ca4c6e3d60475', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:14.646381 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-4438242d-ea41-483d-97a9-9a1ea0751dc1 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] 10.222.0.16 "GET /compute/v2.1/servers/92fbfe5e-07b4-44ec-9729-1a9230f4c02c" status: 200 len: 1523 microversion: 2.1 time: 0.512510 Jul 16 21:25:14.661856 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1786/3554] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:25:14 2022] GET /compute/v2.1/servers/92fbfe5e-07b4-44ec-9729-1a9230f4c02c => generated 1523 bytes in 529 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:25:14.668297 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-c31b860a-41c3-4e11-a928-df30cd59431b tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Calling method '.version_select at 0x7fdd9d1590d0>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:14.731115 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-69b31753-fd1e-4ac8-b9f7-c74f2ff50600 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a23fafbab18d41ee82b84bbdf94cf0fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfc2e42a0df346f4adef152ba2e3fa71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:14.746221 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-69b31753-fd1e-4ac8-b9f7-c74f2ff50600 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'a23fafbab18d41ee82b84bbdf94cf0fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfc2e42a0df346f4adef152ba2e3fa71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:14.750797 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-69b31753-fd1e-4ac8-b9f7-c74f2ff50600 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'a23fafbab18d41ee82b84bbdf94cf0fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfc2e42a0df346f4adef152ba2e3fa71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:14.756556 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-69b31753-fd1e-4ac8-b9f7-c74f2ff50600 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] 10.222.0.16 "GET /compute/v2.1/servers/4eb3d8d6-79ae-4109-a5a7-835c459d1719" status: 200 len: 1832 microversion: 2.32 time: 0.569861 Jul 16 21:25:14.760549 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-c31b860a-41c3-4e11-a928-df30cd59431b tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] 10.222.0.16 "GET /compute/v2.1/images/706863d6-a902-4b5c-abc5-1e3e8a4c0e7a" status: 200 len: 1421 microversion: 2.1 time: 0.095601 Jul 16 21:25:14.760549 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1787/3555] 10.222.0.16 () {58 vars in 1232 bytes} [Sat Jul 16 21:25:14 2022] GET /compute/v2.1/images/706863d6-a902-4b5c-abc5-1e3e8a4c0e7a => generated 1421 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:25:14.760774 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1769/3556] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:25:14 2022] GET /compute/v2.1/servers/4eb3d8d6-79ae-4109-a5a7-835c459d1719 => generated 1832 bytes in 575 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 16 21:25:14.777310 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-57a8d994-4f6b-45da-9353-1054977606d0 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:14.778198 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-57a8d994-4f6b-45da-9353-1054977606d0 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 00c8a753-5bf0-448c-8107-3946aa9bf7ae] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:25:14.781633 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-cc240616-261b-4d6a-94e8-f404ba64da6e tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:14.785073 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-57a8d994-4f6b-45da-9353-1054977606d0 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:14.788905 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-57a8d994-4f6b-45da-9353-1054977606d0 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:14.788905 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-57a8d994-4f6b-45da-9353-1054977606d0 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:14.788905 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-57a8d994-4f6b-45da-9353-1054977606d0 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:14.789845 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-cc240616-261b-4d6a-94e8-f404ba64da6e tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: 05358d0e-ed2f-443c-835c-f77cd929289e] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:25:14.799382 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cc240616-261b-4d6a-94e8-f404ba64da6e tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:14.799986 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cc240616-261b-4d6a-94e8-f404ba64da6e tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:14.801215 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cc240616-261b-4d6a-94e8-f404ba64da6e tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:14.801536 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cc240616-261b-4d6a-94e8-f404ba64da6e tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:14.890103 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-cc240616-261b-4d6a-94e8-f404ba64da6e tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '027dcc48e4fd4c36a5004096ad5e43f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dad823e240149edb30e592380249a13', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:14.902251 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-cc240616-261b-4d6a-94e8-f404ba64da6e tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "GET /compute/v2.1/servers/05358d0e-ed2f-443c-835c-f77cd929289e" status: 200 len: 1205 microversion: 2.1 time: 0.141050 Jul 16 21:25:14.904334 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1770/3557] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:25:14 2022] GET /compute/v2.1/servers/05358d0e-ed2f-443c-835c-f77cd929289e => generated 1205 bytes in 143 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:25:14.936591 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-57a8d994-4f6b-45da-9353-1054977606d0 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f2e974ab5aa48dfb4b00f810dcedb63', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd31be8475a6a49d6bfe8132dd050c6c0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:14.940216 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=80,nova_cell1:UPDATE=16,nova_cell1:SAVEPOINT=1,nova_cell1:RELEASE=1,nova_cell1:DELETE=2 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:25:14.948576 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-57a8d994-4f6b-45da-9353-1054977606d0 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "GET /compute/v2.1/servers/00c8a753-5bf0-448c-8107-3946aa9bf7ae" status: 200 len: 1266 microversion: 2.1 time: 0.178291 Jul 16 21:25:14.949739 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1788/3558] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:25:14 2022] GET /compute/v2.1/servers/00c8a753-5bf0-448c-8107-3946aa9bf7ae => generated 1266 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:25:15.430133 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-c849a60f-3e53-4206-acda-59c0193858ca req-1c779bcc-e51a-4e9d-9427-13e2bb8674b5 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "4eb3d8d6-79ae-4109-a5a7-835c459d1719", "tag": "02eaaf37-d146-4778-a757-4ba313f73f8b"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:25:15.439499 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-c849a60f-3e53-4206-acda-59c0193858ca req-1c779bcc-e51a-4e9d-9427-13e2bb8674b5 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:15.440044 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-c849a60f-3e53-4206-acda-59c0193858ca req-1c779bcc-e51a-4e9d-9427-13e2bb8674b5 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:15.451603 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-c849a60f-3e53-4206-acda-59c0193858ca req-1c779bcc-e51a-4e9d-9427-13e2bb8674b5 service nova] Creating event network-changed:02eaaf37-d146-4778-a757-4ba313f73f8b for instance 4eb3d8d6-79ae-4109-a5a7-835c459d1719 on nch1-master-1 Jul 16 21:25:15.458876 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-c849a60f-3e53-4206-acda-59c0193858ca req-1c779bcc-e51a-4e9d-9427-13e2bb8674b5 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.032983 Jul 16 21:25:15.459597 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1771/3559] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:25:15 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:25:15.670625 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-1f4353f3-3145-4d27-9c22-becbe9ae1bb7 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:15.670625 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-1f4353f3-3145-4d27-9c22-becbe9ae1bb7 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] [instance: 92fbfe5e-07b4-44ec-9729-1a9230f4c02c] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:25:15.680483 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1f4353f3-3145-4d27-9c22-becbe9ae1bb7 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:15.682348 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1f4353f3-3145-4d27-9c22-becbe9ae1bb7 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:15.683388 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1f4353f3-3145-4d27-9c22-becbe9ae1bb7 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:15.684164 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1f4353f3-3145-4d27-9c22-becbe9ae1bb7 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:15.776473 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-9541f1c5-441a-4882-8e2e-c44f65efcfa2 tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Calling method '.version_select at 0x7fdd9c320280>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:15.880096 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-9541f1c5-441a-4882-8e2e-c44f65efcfa2 tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] 10.222.0.16 "GET /compute/v2.1/images/706863d6-a902-4b5c-abc5-1e3e8a4c0e7a" status: 200 len: 1421 microversion: 2.1 time: 0.107929 Jul 16 21:25:15.882115 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1772/3560] 10.222.0.16 () {58 vars in 1232 bytes} [Sat Jul 16 21:25:15 2022] GET /compute/v2.1/images/706863d6-a902-4b5c-abc5-1e3e8a4c0e7a => generated 1421 bytes in 110 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:25:15.897379 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-0adb6131-6f7b-4495-aaa3-8b83024b0e06 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:15.899886 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-0adb6131-6f7b-4495-aaa3-8b83024b0e06 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] [instance: 4eb3d8d6-79ae-4109-a5a7-835c459d1719] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:25:15.918163 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0adb6131-6f7b-4495-aaa3-8b83024b0e06 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:15.918913 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0adb6131-6f7b-4495-aaa3-8b83024b0e06 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:15.941801 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0adb6131-6f7b-4495-aaa3-8b83024b0e06 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:15.942760 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0adb6131-6f7b-4495-aaa3-8b83024b0e06 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:16.809945 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-1f4353f3-3145-4d27-9c22-becbe9ae1bb7 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcc9aa2a091c490599f9a9c41163e2eb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da56fc78f06142aba80ca4c6e3d60475', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:16.817685 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-1f4353f3-3145-4d27-9c22-becbe9ae1bb7 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] 10.222.0.16 "GET /compute/v2.1/servers/92fbfe5e-07b4-44ec-9729-1a9230f4c02c" status: 200 len: 1523 microversion: 2.1 time: 1.157102 Jul 16 21:25:16.828126 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1789/3561] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:25:15 2022] GET /compute/v2.1/servers/92fbfe5e-07b4-44ec-9729-1a9230f4c02c => generated 1523 bytes in 1167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:25:16.834041 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-da1e85bb-cd74-42c5-b661-58dc9cf9b74e tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:16.834612 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-da1e85bb-cd74-42c5-b661-58dc9cf9b74e tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: 05358d0e-ed2f-443c-835c-f77cd929289e] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:25:16.850815 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-da1e85bb-cd74-42c5-b661-58dc9cf9b74e tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:16.851401 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-da1e85bb-cd74-42c5-b661-58dc9cf9b74e tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:16.852488 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-da1e85bb-cd74-42c5-b661-58dc9cf9b74e tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:16.853090 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-da1e85bb-cd74-42c5-b661-58dc9cf9b74e tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:17.009399 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-da1e85bb-cd74-42c5-b661-58dc9cf9b74e tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '027dcc48e4fd4c36a5004096ad5e43f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dad823e240149edb30e592380249a13', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:17.029489 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-da1e85bb-cd74-42c5-b661-58dc9cf9b74e tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "GET /compute/v2.1/servers/05358d0e-ed2f-443c-835c-f77cd929289e" status: 200 len: 1267 microversion: 2.1 time: 0.200226 Jul 16 21:25:17.030701 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1790/3562] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:25:16 2022] GET /compute/v2.1/servers/05358d0e-ed2f-443c-835c-f77cd929289e => generated 1267 bytes in 203 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:25:17.035643 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-e997ed58-6c25-40d3-8e31-60625b816365 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:17.036480 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-e997ed58-6c25-40d3-8e31-60625b816365 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 00c8a753-5bf0-448c-8107-3946aa9bf7ae] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:25:17.043608 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e997ed58-6c25-40d3-8e31-60625b816365 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:17.044069 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e997ed58-6c25-40d3-8e31-60625b816365 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:17.045050 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e997ed58-6c25-40d3-8e31-60625b816365 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:17.045656 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e997ed58-6c25-40d3-8e31-60625b816365 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:17.056451 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-0adb6131-6f7b-4495-aaa3-8b83024b0e06 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a23fafbab18d41ee82b84bbdf94cf0fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfc2e42a0df346f4adef152ba2e3fa71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:17.068427 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-0adb6131-6f7b-4495-aaa3-8b83024b0e06 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'a23fafbab18d41ee82b84bbdf94cf0fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfc2e42a0df346f4adef152ba2e3fa71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:17.070131 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-0adb6131-6f7b-4495-aaa3-8b83024b0e06 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'a23fafbab18d41ee82b84bbdf94cf0fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfc2e42a0df346f4adef152ba2e3fa71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:17.071982 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-0adb6131-6f7b-4495-aaa3-8b83024b0e06 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] 10.222.0.16 "GET /compute/v2.1/servers/4eb3d8d6-79ae-4109-a5a7-835c459d1719" status: 200 len: 1853 microversion: 2.32 time: 1.188337 Jul 16 21:25:17.072563 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1773/3563] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:25:15 2022] GET /compute/v2.1/servers/4eb3d8d6-79ae-4109-a5a7-835c459d1719 => generated 1853 bytes in 1190 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:25:17.094434 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-3e30c29c-7d55-4d43-80b8-57671da1d0a3 req-7ad18d4a-38cf-4dbd-ba5e-501d99095b42 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "f00bea08-4180-45d6-ba01-4b104f5615d6", "name": "network-vif-plugged", "status": "completed", "tag": "541fa980-4b14-4bc2-af0f-d4bf299e66ca"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:25:17.104126 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-3e30c29c-7d55-4d43-80b8-57671da1d0a3 req-7ad18d4a-38cf-4dbd-ba5e-501d99095b42 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:17.105720 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-3e30c29c-7d55-4d43-80b8-57671da1d0a3 req-7ad18d4a-38cf-4dbd-ba5e-501d99095b42 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:17.119118 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-3e30c29c-7d55-4d43-80b8-57671da1d0a3 req-7ad18d4a-38cf-4dbd-ba5e-501d99095b42 service nova] Creating event network-vif-plugged:541fa980-4b14-4bc2-af0f-d4bf299e66ca for instance f00bea08-4180-45d6-ba01-4b104f5615d6 on nch2-master-1 Jul 16 21:25:17.125618 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-3e30c29c-7d55-4d43-80b8-57671da1d0a3 req-7ad18d4a-38cf-4dbd-ba5e-501d99095b42 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.040874 Jul 16 21:25:17.127885 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1774/3564] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:25:17 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:25:17.136892 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-17066808-074d-41ef-be97-72f64d6fd325 tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Calling method '.version_select at 0x7fdd9c3310d0>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:17.186231 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-e997ed58-6c25-40d3-8e31-60625b816365 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f2e974ab5aa48dfb4b00f810dcedb63', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd31be8475a6a49d6bfe8132dd050c6c0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:17.196179 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-17066808-074d-41ef-be97-72f64d6fd325 tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] 10.222.0.16 "GET /compute/v2.1/images/706863d6-a902-4b5c-abc5-1e3e8a4c0e7a" status: 200 len: 1421 microversion: 2.1 time: 0.065721 Jul 16 21:25:17.201646 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1775/3565] 10.222.0.16 () {58 vars in 1232 bytes} [Sat Jul 16 21:25:17 2022] GET /compute/v2.1/images/706863d6-a902-4b5c-abc5-1e3e8a4c0e7a => generated 1421 bytes in 68 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:25:17.235626 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-e997ed58-6c25-40d3-8e31-60625b816365 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "GET /compute/v2.1/servers/00c8a753-5bf0-448c-8107-3946aa9bf7ae" status: 200 len: 1272 microversion: 2.1 time: 0.203211 Jul 16 21:25:17.236776 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1791/3566] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:25:17 2022] GET /compute/v2.1/servers/00c8a753-5bf0-448c-8107-3946aa9bf7ae => generated 1272 bytes in 205 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:25:17.394747 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-3f92101b-6764-4ce3-8991-a2a84264d77f req-cd9781b1-f4ad-45ad-bfe5-443ed91825ed service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "4eb3d8d6-79ae-4109-a5a7-835c459d1719", "tag": "178c5641-50ae-4f11-97c8-41f1bfa01abe"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:25:17.415980 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-3f92101b-6764-4ce3-8991-a2a84264d77f req-cd9781b1-f4ad-45ad-bfe5-443ed91825ed service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:17.415980 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-3f92101b-6764-4ce3-8991-a2a84264d77f req-cd9781b1-f4ad-45ad-bfe5-443ed91825ed service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:17.427752 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-3f92101b-6764-4ce3-8991-a2a84264d77f req-cd9781b1-f4ad-45ad-bfe5-443ed91825ed service nova] Creating event network-changed:178c5641-50ae-4f11-97c8-41f1bfa01abe for instance 4eb3d8d6-79ae-4109-a5a7-835c459d1719 on nch1-master-1 Jul 16 21:25:17.434643 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-3f92101b-6764-4ce3-8991-a2a84264d77f req-cd9781b1-f4ad-45ad-bfe5-443ed91825ed service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.236103 Jul 16 21:25:17.444082 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1776/3567] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:25:17 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 242 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:25:17.836372 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-1fed1f77-fac3-4ca1-82f5-fa3b1f8828f2 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:17.838070 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-1fed1f77-fac3-4ca1-82f5-fa3b1f8828f2 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] [instance: 92fbfe5e-07b4-44ec-9729-1a9230f4c02c] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:25:17.862961 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1fed1f77-fac3-4ca1-82f5-fa3b1f8828f2 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:17.864880 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1fed1f77-fac3-4ca1-82f5-fa3b1f8828f2 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:17.866247 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1fed1f77-fac3-4ca1-82f5-fa3b1f8828f2 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:17.870642 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1fed1f77-fac3-4ca1-82f5-fa3b1f8828f2 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:18.062104 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-01775529-3794-4ec3-9ba6-26cd231aa365 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:18.064147 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-01775529-3794-4ec3-9ba6-26cd231aa365 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: 05358d0e-ed2f-443c-835c-f77cd929289e] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:25:18.078553 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-01775529-3794-4ec3-9ba6-26cd231aa365 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:18.079623 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-01775529-3794-4ec3-9ba6-26cd231aa365 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:18.080741 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-01775529-3794-4ec3-9ba6-26cd231aa365 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:18.081501 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-01775529-3794-4ec3-9ba6-26cd231aa365 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:18.228476 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-01775529-3794-4ec3-9ba6-26cd231aa365 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '027dcc48e4fd4c36a5004096ad5e43f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dad823e240149edb30e592380249a13', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:18.258263 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-01775529-3794-4ec3-9ba6-26cd231aa365 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "GET /compute/v2.1/servers/05358d0e-ed2f-443c-835c-f77cd929289e" status: 200 len: 1267 microversion: 2.1 time: 0.199185 Jul 16 21:25:18.270433 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1777/3568] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:25:18 2022] GET /compute/v2.1/servers/05358d0e-ed2f-443c-835c-f77cd929289e => generated 1267 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:25:18.286219 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-18fb764f-50ef-4f1e-9185-c3948c40867b tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:18.290306 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-18fb764f-50ef-4f1e-9185-c3948c40867b tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] [instance: 4eb3d8d6-79ae-4109-a5a7-835c459d1719] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:25:18.310091 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-18fb764f-50ef-4f1e-9185-c3948c40867b tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:18.310091 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-18fb764f-50ef-4f1e-9185-c3948c40867b tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:18.310091 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-18fb764f-50ef-4f1e-9185-c3948c40867b tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:18.310091 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-18fb764f-50ef-4f1e-9185-c3948c40867b tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:18.399398 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-1fed1f77-fac3-4ca1-82f5-fa3b1f8828f2 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcc9aa2a091c490599f9a9c41163e2eb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da56fc78f06142aba80ca4c6e3d60475', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:18.408208 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-1fed1f77-fac3-4ca1-82f5-fa3b1f8828f2 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] 10.222.0.16 "GET /compute/v2.1/servers/92fbfe5e-07b4-44ec-9729-1a9230f4c02c" status: 200 len: 1523 microversion: 2.1 time: 0.576305 Jul 16 21:25:18.409358 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1792/3569] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:25:17 2022] GET /compute/v2.1/servers/92fbfe5e-07b4-44ec-9729-1a9230f4c02c => generated 1523 bytes in 578 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:25:18.418500 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-8580572a-caf3-443c-884d-655285495662 tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Calling method '.version_select at 0x7fdd9d3ad550>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:18.472800 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-8580572a-caf3-443c-884d-655285495662 tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] 10.222.0.16 "GET /compute/v2.1/images/706863d6-a902-4b5c-abc5-1e3e8a4c0e7a" status: 200 len: 1421 microversion: 2.1 time: 0.062036 Jul 16 21:25:18.479499 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1793/3570] 10.222.0.16 () {58 vars in 1232 bytes} [Sat Jul 16 21:25:18 2022] GET /compute/v2.1/images/706863d6-a902-4b5c-abc5-1e3e8a4c0e7a => generated 1421 bytes in 69 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:25:18.485014 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-6d31821f-35d9-4638-b966-06d7f561e6da tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:18.485838 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-6d31821f-35d9-4638-b966-06d7f561e6da tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 00c8a753-5bf0-448c-8107-3946aa9bf7ae] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:25:18.493549 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6d31821f-35d9-4638-b966-06d7f561e6da tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:18.494300 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6d31821f-35d9-4638-b966-06d7f561e6da tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:18.495302 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6d31821f-35d9-4638-b966-06d7f561e6da tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:18.495923 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6d31821f-35d9-4638-b966-06d7f561e6da tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:18.747889 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-6d31821f-35d9-4638-b966-06d7f561e6da tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f2e974ab5aa48dfb4b00f810dcedb63', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd31be8475a6a49d6bfe8132dd050c6c0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:18.780674 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-6d31821f-35d9-4638-b966-06d7f561e6da tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "GET /compute/v2.1/servers/00c8a753-5bf0-448c-8107-3946aa9bf7ae" status: 200 len: 1272 microversion: 2.1 time: 0.298245 Jul 16 21:25:18.786469 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1794/3571] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:25:18 2022] GET /compute/v2.1/servers/00c8a753-5bf0-448c-8107-3946aa9bf7ae => generated 1272 bytes in 302 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:25:19.230086 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-18fb764f-50ef-4f1e-9185-c3948c40867b tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a23fafbab18d41ee82b84bbdf94cf0fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfc2e42a0df346f4adef152ba2e3fa71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:19.254638 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-18fb764f-50ef-4f1e-9185-c3948c40867b tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'a23fafbab18d41ee82b84bbdf94cf0fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfc2e42a0df346f4adef152ba2e3fa71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:19.258986 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-18fb764f-50ef-4f1e-9185-c3948c40867b tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'a23fafbab18d41ee82b84bbdf94cf0fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfc2e42a0df346f4adef152ba2e3fa71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:19.261879 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-18fb764f-50ef-4f1e-9185-c3948c40867b tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] 10.222.0.16 "GET /compute/v2.1/servers/4eb3d8d6-79ae-4109-a5a7-835c459d1719" status: 200 len: 1874 microversion: 2.32 time: 0.989452 Jul 16 21:25:19.263163 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1778/3572] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:25:18 2022] GET /compute/v2.1/servers/4eb3d8d6-79ae-4109-a5a7-835c459d1719 => generated 1874 bytes in 992 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:25:19.276310 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-73298f2c-2976-46a9-957f-d01f7574bd60 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:19.276943 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-73298f2c-2976-46a9-957f-d01f7574bd60 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: 05358d0e-ed2f-443c-835c-f77cd929289e] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:25:19.299324 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-73298f2c-2976-46a9-957f-d01f7574bd60 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:19.300576 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-73298f2c-2976-46a9-957f-d01f7574bd60 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:19.302679 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-73298f2c-2976-46a9-957f-d01f7574bd60 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:19.303423 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-73298f2c-2976-46a9-957f-d01f7574bd60 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:19.429946 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-191f7636-706f-4c1f-906a-fedf36c966ea tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:19.430410 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-191f7636-706f-4c1f-906a-fedf36c966ea tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] [instance: 92fbfe5e-07b4-44ec-9729-1a9230f4c02c] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:25:19.455820 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-191f7636-706f-4c1f-906a-fedf36c966ea tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:19.455820 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-191f7636-706f-4c1f-906a-fedf36c966ea tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:19.455820 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-191f7636-706f-4c1f-906a-fedf36c966ea tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:19.455820 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-191f7636-706f-4c1f-906a-fedf36c966ea tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:19.486519 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:25:19.965545 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-73298f2c-2976-46a9-957f-d01f7574bd60 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '027dcc48e4fd4c36a5004096ad5e43f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dad823e240149edb30e592380249a13', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:19.977191 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-73298f2c-2976-46a9-957f-d01f7574bd60 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "GET /compute/v2.1/servers/05358d0e-ed2f-443c-835c-f77cd929289e" status: 200 len: 1309 microversion: 2.1 time: 0.705487 Jul 16 21:25:19.977936 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1795/3573] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:25:19 2022] GET /compute/v2.1/servers/05358d0e-ed2f-443c-835c-f77cd929289e => generated 1309 bytes in 707 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:25:19.985567 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-b3d16ae1-510e-4c91-8d36-9bccfeae7740 tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Calling method '.version_select at 0x7fdd9cea6040>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:20.020251 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-191f7636-706f-4c1f-906a-fedf36c966ea tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcc9aa2a091c490599f9a9c41163e2eb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da56fc78f06142aba80ca4c6e3d60475', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:20.029635 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-191f7636-706f-4c1f-906a-fedf36c966ea tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] 10.222.0.16 "GET /compute/v2.1/servers/92fbfe5e-07b4-44ec-9729-1a9230f4c02c" status: 200 len: 1523 microversion: 2.1 time: 0.605406 Jul 16 21:25:20.031816 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1779/3574] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:25:19 2022] GET /compute/v2.1/servers/92fbfe5e-07b4-44ec-9729-1a9230f4c02c => generated 1523 bytes in 608 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:25:20.036791 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-b5dd38ff-6b71-4fba-8b2e-230865992ad5 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:20.138143 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-b5dd38ff-6b71-4fba-8b2e-230865992ad5 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 00c8a753-5bf0-448c-8107-3946aa9bf7ae] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:25:20.139117 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-b3d16ae1-510e-4c91-8d36-9bccfeae7740 tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] 10.222.0.16 "GET /compute/v2.1/images/706863d6-a902-4b5c-abc5-1e3e8a4c0e7a" status: 200 len: 1421 microversion: 2.1 time: 0.060960 Jul 16 21:25:20.139117 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1796/3575] 10.222.0.16 () {58 vars in 1232 bytes} [Sat Jul 16 21:25:19 2022] GET /compute/v2.1/images/706863d6-a902-4b5c-abc5-1e3e8a4c0e7a => generated 1421 bytes in 62 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:25:20.139309 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b5dd38ff-6b71-4fba-8b2e-230865992ad5 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:20.139309 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b5dd38ff-6b71-4fba-8b2e-230865992ad5 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:20.139309 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b5dd38ff-6b71-4fba-8b2e-230865992ad5 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:20.139309 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b5dd38ff-6b71-4fba-8b2e-230865992ad5 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:20.282851 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-18f4ed64-985c-403b-9864-7b83e1996b11 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:20.284387 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-18f4ed64-985c-403b-9864-7b83e1996b11 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] [instance: 4eb3d8d6-79ae-4109-a5a7-835c459d1719] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:25:20.297936 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-18f4ed64-985c-403b-9864-7b83e1996b11 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:20.298945 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-18f4ed64-985c-403b-9864-7b83e1996b11 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:20.299916 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-18f4ed64-985c-403b-9864-7b83e1996b11 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:20.300558 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-18f4ed64-985c-403b-9864-7b83e1996b11 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:20.418949 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-b5dd38ff-6b71-4fba-8b2e-230865992ad5 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f2e974ab5aa48dfb4b00f810dcedb63', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd31be8475a6a49d6bfe8132dd050c6c0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:20.429824 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-b5dd38ff-6b71-4fba-8b2e-230865992ad5 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "GET /compute/v2.1/servers/00c8a753-5bf0-448c-8107-3946aa9bf7ae" status: 200 len: 1314 microversion: 2.1 time: 0.395932 Jul 16 21:25:20.441764 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1780/3576] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:25:20 2022] GET /compute/v2.1/servers/00c8a753-5bf0-448c-8107-3946aa9bf7ae => generated 1314 bytes in 409 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:25:20.545813 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-cbd6ac6d-0fde-42c7-b86a-913b8693472f req-a687e02f-a2da-4068-aac1-063f12b97240 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "4eb3d8d6-79ae-4109-a5a7-835c459d1719", "tag": "f55f6499-e005-4119-ad39-6c8db2fb1c53"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:25:20.560758 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-cbd6ac6d-0fde-42c7-b86a-913b8693472f req-a687e02f-a2da-4068-aac1-063f12b97240 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:20.561359 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-cbd6ac6d-0fde-42c7-b86a-913b8693472f req-a687e02f-a2da-4068-aac1-063f12b97240 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:20.582538 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-cbd6ac6d-0fde-42c7-b86a-913b8693472f req-a687e02f-a2da-4068-aac1-063f12b97240 service nova] Creating event network-changed:f55f6499-e005-4119-ad39-6c8db2fb1c53 for instance 4eb3d8d6-79ae-4109-a5a7-835c459d1719 on nch1-master-1 Jul 16 21:25:20.589022 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-cbd6ac6d-0fde-42c7-b86a-913b8693472f req-a687e02f-a2da-4068-aac1-063f12b97240 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.046863 Jul 16 21:25:20.589840 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1781/3577] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:25:20 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:25:20.999492 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-a28e4f36-7e45-4ce4-b6cc-38e609e432c5 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:21.000355 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-a28e4f36-7e45-4ce4-b6cc-38e609e432c5 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: 05358d0e-ed2f-443c-835c-f77cd929289e] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:25:21.011146 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a28e4f36-7e45-4ce4-b6cc-38e609e432c5 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:21.011880 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a28e4f36-7e45-4ce4-b6cc-38e609e432c5 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:21.013413 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a28e4f36-7e45-4ce4-b6cc-38e609e432c5 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:21.014934 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a28e4f36-7e45-4ce4-b6cc-38e609e432c5 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:21.235594 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-18f4ed64-985c-403b-9864-7b83e1996b11 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a23fafbab18d41ee82b84bbdf94cf0fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfc2e42a0df346f4adef152ba2e3fa71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:21.253688 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-18f4ed64-985c-403b-9864-7b83e1996b11 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'a23fafbab18d41ee82b84bbdf94cf0fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfc2e42a0df346f4adef152ba2e3fa71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:21.258961 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-18f4ed64-985c-403b-9864-7b83e1996b11 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'a23fafbab18d41ee82b84bbdf94cf0fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfc2e42a0df346f4adef152ba2e3fa71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:21.262287 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-18f4ed64-985c-403b-9864-7b83e1996b11 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] 10.222.0.16 "GET /compute/v2.1/servers/4eb3d8d6-79ae-4109-a5a7-835c459d1719" status: 200 len: 1874 microversion: 2.32 time: 0.984646 Jul 16 21:25:21.273521 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1797/3578] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:25:20 2022] GET /compute/v2.1/servers/4eb3d8d6-79ae-4109-a5a7-835c459d1719 => generated 1874 bytes in 995 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 16 21:25:21.285864 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-0a9fa6e0-453b-4d72-a657-07f04b533f69 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:21.287267 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-0a9fa6e0-453b-4d72-a657-07f04b533f69 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] [instance: 92fbfe5e-07b4-44ec-9729-1a9230f4c02c] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:25:21.295380 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0a9fa6e0-453b-4d72-a657-07f04b533f69 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:21.298286 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0a9fa6e0-453b-4d72-a657-07f04b533f69 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:21.298286 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0a9fa6e0-453b-4d72-a657-07f04b533f69 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:21.298286 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0a9fa6e0-453b-4d72-a657-07f04b533f69 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:22.103145 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-a28e4f36-7e45-4ce4-b6cc-38e609e432c5 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '027dcc48e4fd4c36a5004096ad5e43f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dad823e240149edb30e592380249a13', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:22.112459 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-a28e4f36-7e45-4ce4-b6cc-38e609e432c5 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "GET /compute/v2.1/servers/05358d0e-ed2f-443c-835c-f77cd929289e" status: 200 len: 1309 microversion: 2.1 time: 1.120520 Jul 16 21:25:22.113385 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1782/3579] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:25:20 2022] GET /compute/v2.1/servers/05358d0e-ed2f-443c-835c-f77cd929289e => generated 1309 bytes in 1124 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:25:22.120188 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-42ff179a-a346-4701-bd00-2eacf0c2a75e tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Calling method '.version_select at 0x7fdd9cdcaaf0>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:22.177257 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-42ff179a-a346-4701-bd00-2eacf0c2a75e tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] 10.222.0.16 "GET /compute/v2.1/images/706863d6-a902-4b5c-abc5-1e3e8a4c0e7a" status: 200 len: 1421 microversion: 2.1 time: 0.062612 Jul 16 21:25:22.184028 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1783/3580] 10.222.0.16 () {58 vars in 1232 bytes} [Sat Jul 16 21:25:22 2022] GET /compute/v2.1/images/706863d6-a902-4b5c-abc5-1e3e8a4c0e7a => generated 1421 bytes in 64 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:25:22.186139 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-1a661089-e548-4454-87fc-f78d3977fddc tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:22.187555 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-1a661089-e548-4454-87fc-f78d3977fddc tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 00c8a753-5bf0-448c-8107-3946aa9bf7ae] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:25:22.198613 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1a661089-e548-4454-87fc-f78d3977fddc tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:22.198613 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1a661089-e548-4454-87fc-f78d3977fddc tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:22.199550 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1a661089-e548-4454-87fc-f78d3977fddc tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:22.200211 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1a661089-e548-4454-87fc-f78d3977fddc tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:22.275212 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-0a9fa6e0-453b-4d72-a657-07f04b533f69 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcc9aa2a091c490599f9a9c41163e2eb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da56fc78f06142aba80ca4c6e3d60475', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:22.295508 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-0a9fa6e0-453b-4d72-a657-07f04b533f69 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] 10.222.0.16 "GET /compute/v2.1/servers/92fbfe5e-07b4-44ec-9729-1a9230f4c02c" status: 200 len: 1503 microversion: 2.1 time: 1.020462 Jul 16 21:25:22.299774 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1798/3581] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:25:21 2022] GET /compute/v2.1/servers/92fbfe5e-07b4-44ec-9729-1a9230f4c02c => generated 1503 bytes in 1025 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:25:22.317251 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-2f245944-d9ef-40aa-9ce6-19640b4c46fa tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:22.322879 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-2f245944-d9ef-40aa-9ce6-19640b4c46fa tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] [instance: 4eb3d8d6-79ae-4109-a5a7-835c459d1719] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:25:22.334839 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2f245944-d9ef-40aa-9ce6-19640b4c46fa tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:22.335867 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2f245944-d9ef-40aa-9ce6-19640b4c46fa tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:22.336712 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2f245944-d9ef-40aa-9ce6-19640b4c46fa tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:22.337154 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2f245944-d9ef-40aa-9ce6-19640b4c46fa tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:22.486257 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell0:SELECT=2 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:25:22.602081 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell0:SELECT=2 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:25:22.830610 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-1a661089-e548-4454-87fc-f78d3977fddc tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f2e974ab5aa48dfb4b00f810dcedb63', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd31be8475a6a49d6bfe8132dd050c6c0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:22.845491 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-1a661089-e548-4454-87fc-f78d3977fddc tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "GET /compute/v2.1/servers/00c8a753-5bf0-448c-8107-3946aa9bf7ae" status: 200 len: 1314 microversion: 2.1 time: 0.665217 Jul 16 21:25:22.856992 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1784/3582] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:25:22 2022] GET /compute/v2.1/servers/00c8a753-5bf0-448c-8107-3946aa9bf7ae => generated 1314 bytes in 678 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:25:22.870903 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-2ed57072-97f3-43d0-b3dd-d09d6a336880 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "48e72420-c2e0-466c-8349-298ea1959ee1"}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:25:22.872420 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-2ed57072-97f3-43d0-b3dd-d09d6a336880 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] [instance: 92fbfe5e-07b4-44ec-9729-1a9230f4c02c] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:25:22.880399 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2ed57072-97f3-43d0-b3dd-d09d6a336880 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:22.881116 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2ed57072-97f3-43d0-b3dd-d09d6a336880 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:22.882140 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2ed57072-97f3-43d0-b3dd-d09d6a336880 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:22.882963 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2ed57072-97f3-43d0-b3dd-d09d6a336880 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:22.911046 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-2ed57072-97f3-43d0-b3dd-d09d6a336880 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] HTTP exception thrown: Cannot 'attach_volume' instance 92fbfe5e-07b4-44ec-9729-1a9230f4c02c while it is in vm_state rescued Jul 16 21:25:22.913147 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-2ed57072-97f3-43d0-b3dd-d09d6a336880 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Request method failure captured: Jul 16 21:25:22.913147 ncdv-master-1 devstack@n-api.service[138681]: request: POST /compute/v2.1/servers/92fbfe5e-07b4-44ec-9729-1a9230f4c02c/os-volume_attachments HTTP/1.1 Jul 16 21:25:22.913147 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:25:22.913147 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:25:22.913147 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:25:22.913147 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 74 Jul 16 21:25:22.913147 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:25:22.913147 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:25:22.913147 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:25:22.913147 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:25:22.913147 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0yxlBvXdj6P284bimRWVNOZAMmAVjIOasTJ1njzyaS12Nm5AKzVbxoKHi72FFVSaWe28lVKwWFtzjqUTy3NQWDNW-obFHfOCoFCCR4GNQ-_ruXI6NnMsKND6oz7nIdWgqUgB4Gac48-MGLdaneUoqvnHJrpsJjhWmKgGoKbdR8M7y0I Jul 16 21:25:22.913147 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:25:22.913147 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:25:22.913147 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:25:22.913147 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:25:22.913147 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:25:22.913147 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:25:22.913147 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: da56fc78f06142aba80ca4c6e3d60475 Jul 16 21:25:22.913147 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-439221844 Jul 16 21:25:22.913147 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:25:22.913147 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:25:22.913147 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/da56fc78f06142aba80ca4c6e3d60475"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/da56fc78f06142aba80ca4c6e3d60475"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_da56fc78f06142aba80ca4c6e3d60475"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/da56fc78f06142aba80ca4c6e3d60475"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:25:22.913147 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-ServerRescueNegativeTestJSON-439221844 Jul 16 21:25:22.913147 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: da56fc78f06142aba80ca4c6e3d60475 Jul 16 21:25:22.913147 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-439221844 Jul 16 21:25:22.913147 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-ServerRescueNegativeTestJSON-439221844-project Jul 16 21:25:22.913147 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:25:22.913147 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:25:22.913147 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: bcc9aa2a091c490599f9a9c41163e2eb Jul 16 21:25:22.913147 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-ServerRescueNegativeTestJSON-439221844-project Jul 16 21:25:22.913147 ncdv-master-1 devstack@n-api.service[138681]: Jul 16 21:25:22.913147 ncdv-master-1 devstack@n-api.service[138681]: {"volumeAttachment": {"volumeId": "48e72420-c2e0-466c-8349-298ea1959ee1"}} Jul 16 21:25:22.913147 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:25:22.913147 ncdv-master-1 devstack@n-api.service[138681]: exception: Cannot 'attach_volume' instance 92fbfe5e-07b4-44ec-9729-1a9230f4c02c while it is in vm_state rescued Jul 16 21:25:22.913147 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'server_id': '92fbfe5e-07b4-44ec-9729-1a9230f4c02c', 'body': {'volumeAttachment': {'volumeId': '48e72420-c2e0-466c-8349-298ea1959ee1'}}} Jul 16 21:25:22.913147 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:25:22.913147 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:25:22.913147 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 388, in create Jul 16 21:25:22.913147 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi device = self.compute_api.attach_volume( Jul 16 21:25:22.913147 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner Jul 16 21:25:22.913147 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Jul 16 21:25:22.913147 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 145, in inner Jul 16 21:25:22.913147 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.InstanceInvalidState( Jul 16 21:25:22.913147 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.InstanceInvalidState: Instance 92fbfe5e-07b4-44ec-9729-1a9230f4c02c in vm_state rescued. Cannot attach_volume while the instance is in this state. Jul 16 21:25:22.913147 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:25:22.913147 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:25:22.913147 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:25:22.913147 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:25:22.913147 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:25:22.913147 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:25:22.913147 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:25:22.913147 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:25:22.913147 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:25:22.913147 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:25:22.913147 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 16 21:25:22.913147 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 16 21:25:22.913147 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 16 21:25:22.913147 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 16 21:25:22.913147 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 16 21:25:22.913147 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 16 21:25:22.913147 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 398, in create Jul 16 21:25:22.913147 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi common.raise_http_conflict_for_instance_invalid_state(state_error, Jul 16 21:25:22.918464 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 347, in raise_http_conflict_for_instance_invalid_state Jul 16 21:25:22.918464 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPConflict(explanation=msg) Jul 16 21:25:22.918464 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPConflict: Cannot 'attach_volume' instance 92fbfe5e-07b4-44ec-9729-1a9230f4c02c while it is in vm_state rescued Jul 16 21:25:22.918464 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:25:22.918464 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:25:22.918464 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:25:22.918464 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:25:22.918464 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:25:22.918464 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:25:22.918464 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:25:22.918464 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:25:22.918464 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Cannot 'attach_volume' instance 92fbfe5e-07b4-44ec-9729-1a9230f4c02c while it is in vm_state rescued Jul 16 21:25:22.918464 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:25:22.925321 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-2ed57072-97f3-43d0-b3dd-d09d6a336880 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Returning 409 to user: Cannot 'attach_volume' instance 92fbfe5e-07b4-44ec-9729-1a9230f4c02c while it is in vm_state rescued {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:25:22.926646 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-2ed57072-97f3-43d0-b3dd-d09d6a336880 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] 10.222.0.16 "POST /compute/v2.1/servers/92fbfe5e-07b4-44ec-9729-1a9230f4c02c/os-volume_attachments" status: 409 len: 152 microversion: 2.1 time: 0.068124 Jul 16 21:25:22.931347 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1785/3583] 10.222.0.16 () {60 vars in 1322 bytes} [Sat Jul 16 21:25:22 2022] POST /compute/v2.1/servers/92fbfe5e-07b4-44ec-9729-1a9230f4c02c/os-volume_attachments => generated 152 bytes in 73 msecs (HTTP/1.1 409) 9 headers in 378 bytes (1 switches on core 0) Jul 16 21:25:22.937164 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-4f57de37-bc77-478e-a612-5603808db180 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Action: 'action', calling method: >, body: {"unrescue": {}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:25:22.938406 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-4f57de37-bc77-478e-a612-5603808db180 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] [instance: 92fbfe5e-07b4-44ec-9729-1a9230f4c02c] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:25:22.941371 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-2f245944-d9ef-40aa-9ce6-19640b4c46fa tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a23fafbab18d41ee82b84bbdf94cf0fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfc2e42a0df346f4adef152ba2e3fa71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:22.946315 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4f57de37-bc77-478e-a612-5603808db180 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:22.947213 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4f57de37-bc77-478e-a612-5603808db180 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:22.948231 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4f57de37-bc77-478e-a612-5603808db180 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:22.948902 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4f57de37-bc77-478e-a612-5603808db180 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:22.980585 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-2f245944-d9ef-40aa-9ce6-19640b4c46fa tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'a23fafbab18d41ee82b84bbdf94cf0fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfc2e42a0df346f4adef152ba2e3fa71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:22.982119 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-2f245944-d9ef-40aa-9ce6-19640b4c46fa tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'a23fafbab18d41ee82b84bbdf94cf0fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfc2e42a0df346f4adef152ba2e3fa71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:22.995475 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-2f245944-d9ef-40aa-9ce6-19640b4c46fa tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] 10.222.0.16 "GET /compute/v2.1/servers/4eb3d8d6-79ae-4109-a5a7-835c459d1719" status: 200 len: 1874 microversion: 2.32 time: 0.682405 Jul 16 21:25:23.002743 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1799/3584] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:25:22 2022] GET /compute/v2.1/servers/4eb3d8d6-79ae-4109-a5a7-835c459d1719 => generated 1874 bytes in 702 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:25:23.131326 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-29604488-b865-42fb-a7ed-37b352cddfac tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:23.132782 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-29604488-b865-42fb-a7ed-37b352cddfac tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: 05358d0e-ed2f-443c-835c-f77cd929289e] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:25:23.145414 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-29604488-b865-42fb-a7ed-37b352cddfac tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:23.152872 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-29604488-b865-42fb-a7ed-37b352cddfac tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:23.153640 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-29604488-b865-42fb-a7ed-37b352cddfac tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:23.156411 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-29604488-b865-42fb-a7ed-37b352cddfac tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:23.229467 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-4f57de37-bc77-478e-a612-5603808db180 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] 10.222.0.16 "POST /compute/v2.1/servers/92fbfe5e-07b4-44ec-9729-1a9230f4c02c/action" status: 202 len: 0 microversion: 2.1 time: 0.295934 Jul 16 21:25:23.231706 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1786/3585] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 21:25:22 2022] POST /compute/v2.1/servers/92fbfe5e-07b4-44ec-9729-1a9230f4c02c/action => generated 0 bytes in 299 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 21:25:23.256317 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-274ff930-fa70-4a96-afb0-5fbefc6e9680 tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Calling method '.version_select at 0x7fdd9cdca0d0>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:23.265884 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell0:SELECT=8 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:25:23.322312 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-274ff930-fa70-4a96-afb0-5fbefc6e9680 tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] 10.222.0.16 "GET /compute/v2.1/images/706863d6-a902-4b5c-abc5-1e3e8a4c0e7a" status: 200 len: 1421 microversion: 2.1 time: 0.076995 Jul 16 21:25:23.323756 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1787/3586] 10.222.0.16 () {58 vars in 1232 bytes} [Sat Jul 16 21:25:23 2022] GET /compute/v2.1/images/706863d6-a902-4b5c-abc5-1e3e8a4c0e7a => generated 1421 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:25:23.333018 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-7f9fa44c-750f-448e-a549-aaea05efe204 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:23.334108 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-7f9fa44c-750f-448e-a549-aaea05efe204 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] [instance: 92fbfe5e-07b4-44ec-9729-1a9230f4c02c] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:25:23.356537 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7f9fa44c-750f-448e-a549-aaea05efe204 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:23.357365 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7f9fa44c-750f-448e-a549-aaea05efe204 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:23.358427 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7f9fa44c-750f-448e-a549-aaea05efe204 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:23.359159 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7f9fa44c-750f-448e-a549-aaea05efe204 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:23.941328 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-29604488-b865-42fb-a7ed-37b352cddfac tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '027dcc48e4fd4c36a5004096ad5e43f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dad823e240149edb30e592380249a13', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:23.951984 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-29604488-b865-42fb-a7ed-37b352cddfac tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "GET /compute/v2.1/servers/05358d0e-ed2f-443c-835c-f77cd929289e" status: 200 len: 1309 microversion: 2.1 time: 0.825856 Jul 16 21:25:23.965932 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1800/3587] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:25:23 2022] GET /compute/v2.1/servers/05358d0e-ed2f-443c-835c-f77cd929289e => generated 1309 bytes in 838 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:25:23.970356 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-c698e694-3fc4-4a6c-b7ec-d87c7f7a8f4f tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:23.970603 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-c698e694-3fc4-4a6c-b7ec-d87c7f7a8f4f tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] [instance: 2fad683a-f21f-4a9c-8323-e9380ef7f9ef] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:25:23.983214 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c698e694-3fc4-4a6c-b7ec-d87c7f7a8f4f tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:23.984221 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c698e694-3fc4-4a6c-b7ec-d87c7f7a8f4f tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:23.985491 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c698e694-3fc4-4a6c-b7ec-d87c7f7a8f4f tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:23.986269 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c698e694-3fc4-4a6c-b7ec-d87c7f7a8f4f tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:24.003997 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-7f9fa44c-750f-448e-a549-aaea05efe204 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcc9aa2a091c490599f9a9c41163e2eb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da56fc78f06142aba80ca4c6e3d60475', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:24.016010 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-7f9fa44c-750f-448e-a549-aaea05efe204 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] 10.222.0.16 "GET /compute/v2.1/servers/92fbfe5e-07b4-44ec-9729-1a9230f4c02c" status: 200 len: 1511 microversion: 2.1 time: 0.690775 Jul 16 21:25:24.018184 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1788/3588] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:25:23 2022] GET /compute/v2.1/servers/92fbfe5e-07b4-44ec-9729-1a9230f4c02c => generated 1511 bytes in 692 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:25:24.024049 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-c698e694-3fc4-4a6c-b7ec-d87c7f7a8f4f tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] 10.222.0.16 "GET /compute/v2.1/servers/2fad683a-f21f-4a9c-8323-e9380ef7f9ef/ips" status: 200 len: 145 microversion: 2.1 time: 0.058514 Jul 16 21:25:24.025466 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1801/3589] 10.222.0.16 () {58 vars in 1247 bytes} [Sat Jul 16 21:25:23 2022] GET /compute/v2.1/servers/2fad683a-f21f-4a9c-8323-e9380ef7f9ef/ips => generated 145 bytes in 61 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:25:24.031557 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-9cf817f9-ec59-4cdb-ad81-91ba7cb928ae tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:24.032712 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-9cf817f9-ec59-4cdb-ad81-91ba7cb928ae tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 00c8a753-5bf0-448c-8107-3946aa9bf7ae] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:25:24.035400 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-13824018-6f08-4654-9f49-c25f122263e5 req-79a2aede-e965-423f-8010-3658a816f696 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "05358d0e-ed2f-443c-835c-f77cd929289e", "tag": "9675fc80-e092-4333-8050-341c379735a0"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:25:24.041169 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9cf817f9-ec59-4cdb-ad81-91ba7cb928ae tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:24.041648 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9cf817f9-ec59-4cdb-ad81-91ba7cb928ae tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:24.042598 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9cf817f9-ec59-4cdb-ad81-91ba7cb928ae tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:24.043145 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9cf817f9-ec59-4cdb-ad81-91ba7cb928ae tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:24.048682 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-13824018-6f08-4654-9f49-c25f122263e5 req-79a2aede-e965-423f-8010-3658a816f696 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:24.049362 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-13824018-6f08-4654-9f49-c25f122263e5 req-79a2aede-e965-423f-8010-3658a816f696 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:24.087181 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-13824018-6f08-4654-9f49-c25f122263e5 req-79a2aede-e965-423f-8010-3658a816f696 service nova] Creating event network-changed:9675fc80-e092-4333-8050-341c379735a0 for instance 05358d0e-ed2f-443c-835c-f77cd929289e on nch2-master-1 Jul 16 21:25:24.097961 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-13824018-6f08-4654-9f49-c25f122263e5 req-79a2aede-e965-423f-8010-3658a816f696 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.077780 Jul 16 21:25:24.099469 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1789/3590] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:25:24 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 80 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:25:24.109130 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-f08c85d9-4d94-4d76-bc75-7c4d6fe48c45 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:24.112030 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-f08c85d9-4d94-4d76-bc75-7c4d6fe48c45 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] [instance: 4eb3d8d6-79ae-4109-a5a7-835c459d1719] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:25:24.122153 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f08c85d9-4d94-4d76-bc75-7c4d6fe48c45 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:24.122824 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f08c85d9-4d94-4d76-bc75-7c4d6fe48c45 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:24.123836 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f08c85d9-4d94-4d76-bc75-7c4d6fe48c45 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:24.124406 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f08c85d9-4d94-4d76-bc75-7c4d6fe48c45 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:24.146603 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell0:SELECT=8 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:25:24.597379 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-9cf817f9-ec59-4cdb-ad81-91ba7cb928ae tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f2e974ab5aa48dfb4b00f810dcedb63', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd31be8475a6a49d6bfe8132dd050c6c0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:24.610081 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-9cf817f9-ec59-4cdb-ad81-91ba7cb928ae tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "GET /compute/v2.1/servers/00c8a753-5bf0-448c-8107-3946aa9bf7ae" status: 200 len: 1314 microversion: 2.1 time: 0.582061 Jul 16 21:25:24.616069 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1802/3591] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:25:24 2022] GET /compute/v2.1/servers/00c8a753-5bf0-448c-8107-3946aa9bf7ae => generated 1314 bytes in 590 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:25:24.625651 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-6112e60f-411a-4961-b29e-aadf258bdd96 tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Calling method '.version_select at 0x7fdd9d434940>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:24.730258 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-6112e60f-411a-4961-b29e-aadf258bdd96 tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] 10.222.0.16 "GET /compute/v2.1/images/706863d6-a902-4b5c-abc5-1e3e8a4c0e7a" status: 200 len: 1421 microversion: 2.1 time: 0.111898 Jul 16 21:25:24.730860 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1803/3592] 10.222.0.16 () {58 vars in 1232 bytes} [Sat Jul 16 21:25:24 2022] GET /compute/v2.1/images/706863d6-a902-4b5c-abc5-1e3e8a4c0e7a => generated 1421 bytes in 114 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:25:24.910762 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-f08c85d9-4d94-4d76-bc75-7c4d6fe48c45 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a23fafbab18d41ee82b84bbdf94cf0fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfc2e42a0df346f4adef152ba2e3fa71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:24.930105 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-f08c85d9-4d94-4d76-bc75-7c4d6fe48c45 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'a23fafbab18d41ee82b84bbdf94cf0fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfc2e42a0df346f4adef152ba2e3fa71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:24.932211 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-f08c85d9-4d94-4d76-bc75-7c4d6fe48c45 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'a23fafbab18d41ee82b84bbdf94cf0fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfc2e42a0df346f4adef152ba2e3fa71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:24.939109 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-f08c85d9-4d94-4d76-bc75-7c4d6fe48c45 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] 10.222.0.16 "GET /compute/v2.1/servers/4eb3d8d6-79ae-4109-a5a7-835c459d1719" status: 200 len: 1874 microversion: 2.32 time: 0.835100 Jul 16 21:25:24.940107 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1790/3593] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:25:24 2022] GET /compute/v2.1/servers/4eb3d8d6-79ae-4109-a5a7-835c459d1719 => generated 1874 bytes in 840 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:25:24.970078 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-ead0f708-9485-4dea-b441-c4d7522e38c9 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:24.971395 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-ead0f708-9485-4dea-b441-c4d7522e38c9 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: 05358d0e-ed2f-443c-835c-f77cd929289e] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:25:24.986374 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ead0f708-9485-4dea-b441-c4d7522e38c9 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:24.986949 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ead0f708-9485-4dea-b441-c4d7522e38c9 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:24.987898 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ead0f708-9485-4dea-b441-c4d7522e38c9 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:24.988575 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ead0f708-9485-4dea-b441-c4d7522e38c9 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:25.040124 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-148bef93-2a5b-4a67-8138-9d38996ac643 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:25.040604 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-148bef93-2a5b-4a67-8138-9d38996ac643 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] [instance: 92fbfe5e-07b4-44ec-9729-1a9230f4c02c] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:25:25.054537 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-148bef93-2a5b-4a67-8138-9d38996ac643 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:25.055217 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-148bef93-2a5b-4a67-8138-9d38996ac643 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:25.061145 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-148bef93-2a5b-4a67-8138-9d38996ac643 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:25.062886 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-148bef93-2a5b-4a67-8138-9d38996ac643 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:26.052560 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-ead0f708-9485-4dea-b441-c4d7522e38c9 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '027dcc48e4fd4c36a5004096ad5e43f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dad823e240149edb30e592380249a13', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:26.072804 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-ead0f708-9485-4dea-b441-c4d7522e38c9 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "GET /compute/v2.1/servers/05358d0e-ed2f-443c-835c-f77cd929289e" status: 200 len: 1309 microversion: 2.1 time: 1.106354 Jul 16 21:25:26.076799 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1804/3594] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:25:24 2022] GET /compute/v2.1/servers/05358d0e-ed2f-443c-835c-f77cd929289e => generated 1309 bytes in 1111 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:25:26.086980 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-f2cb2456-66ff-477f-9b34-e6b3fcfc720b tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:26.088868 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-f2cb2456-66ff-477f-9b34-e6b3fcfc720b tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 00c8a753-5bf0-448c-8107-3946aa9bf7ae] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:25:26.099504 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f2cb2456-66ff-477f-9b34-e6b3fcfc720b tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:26.100292 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f2cb2456-66ff-477f-9b34-e6b3fcfc720b tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:26.102313 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f2cb2456-66ff-477f-9b34-e6b3fcfc720b tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:26.102931 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f2cb2456-66ff-477f-9b34-e6b3fcfc720b tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:26.218874 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-148bef93-2a5b-4a67-8138-9d38996ac643 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcc9aa2a091c490599f9a9c41163e2eb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da56fc78f06142aba80ca4c6e3d60475', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:26.240168 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-148bef93-2a5b-4a67-8138-9d38996ac643 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] 10.222.0.16 "GET /compute/v2.1/servers/92fbfe5e-07b4-44ec-9729-1a9230f4c02c" status: 200 len: 1511 microversion: 2.1 time: 1.204863 Jul 16 21:25:26.241031 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1791/3595] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:25:25 2022] GET /compute/v2.1/servers/92fbfe5e-07b4-44ec-9729-1a9230f4c02c => generated 1511 bytes in 1208 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:25:26.255136 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-dce58895-dcc3-49ba-91b0-a97d3c4fc0a8 tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Calling method '.version_select at 0x7fdd9c331c10>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:26.328098 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-dce58895-dcc3-49ba-91b0-a97d3c4fc0a8 tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] 10.222.0.16 "GET /compute/v2.1/images/706863d6-a902-4b5c-abc5-1e3e8a4c0e7a" status: 200 len: 1421 microversion: 2.1 time: 0.077561 Jul 16 21:25:26.334389 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1792/3596] 10.222.0.16 () {58 vars in 1232 bytes} [Sat Jul 16 21:25:26 2022] GET /compute/v2.1/images/706863d6-a902-4b5c-abc5-1e3e8a4c0e7a => generated 1421 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:25:26.345809 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-2e2bc66c-cba1-4aaf-9ef6-7c67d63bcb38 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:26.346489 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-2e2bc66c-cba1-4aaf-9ef6-7c67d63bcb38 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] [instance: 4eb3d8d6-79ae-4109-a5a7-835c459d1719] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:25:26.355072 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2e2bc66c-cba1-4aaf-9ef6-7c67d63bcb38 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:26.355799 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2e2bc66c-cba1-4aaf-9ef6-7c67d63bcb38 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:26.356781 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2e2bc66c-cba1-4aaf-9ef6-7c67d63bcb38 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:26.357398 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2e2bc66c-cba1-4aaf-9ef6-7c67d63bcb38 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:26.923406 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-f2cb2456-66ff-477f-9b34-e6b3fcfc720b tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f2e974ab5aa48dfb4b00f810dcedb63', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd31be8475a6a49d6bfe8132dd050c6c0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:26.939106 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-f2cb2456-66ff-477f-9b34-e6b3fcfc720b tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "GET /compute/v2.1/servers/00c8a753-5bf0-448c-8107-3946aa9bf7ae" status: 200 len: 1314 microversion: 2.1 time: 0.860759 Jul 16 21:25:26.951505 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1805/3597] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:25:26 2022] GET /compute/v2.1/servers/00c8a753-5bf0-448c-8107-3946aa9bf7ae => generated 1314 bytes in 874 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:25:26.957105 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-b2669453-03df-49df-a1e2-4bd53a933a4f req-376a87f5-bca3-4767-b2c7-b835e77492b9 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "00c8a753-5bf0-448c-8107-3946aa9bf7ae", "tag": "bb33931c-0e67-4fab-ae33-70ed7930c4be"}, {"name": "network-changed", "server_uuid": "4eb3d8d6-79ae-4109-a5a7-835c459d1719", "tag": "3581de64-9f10-4ee1-9fbc-52628f1ddcfa"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:25:26.975770 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-b2669453-03df-49df-a1e2-4bd53a933a4f req-376a87f5-bca3-4767-b2c7-b835e77492b9 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:26.976967 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-b2669453-03df-49df-a1e2-4bd53a933a4f req-376a87f5-bca3-4767-b2c7-b835e77492b9 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:26.993890 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-b2669453-03df-49df-a1e2-4bd53a933a4f req-376a87f5-bca3-4767-b2c7-b835e77492b9 service nova] Creating event network-changed:bb33931c-0e67-4fab-ae33-70ed7930c4be for instance 00c8a753-5bf0-448c-8107-3946aa9bf7ae on nch2-master-1 Jul 16 21:25:26.994706 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-b2669453-03df-49df-a1e2-4bd53a933a4f req-376a87f5-bca3-4767-b2c7-b835e77492b9 service nova] Creating event network-changed:3581de64-9f10-4ee1-9fbc-52628f1ddcfa for instance 4eb3d8d6-79ae-4109-a5a7-835c459d1719 on nch1-master-1 Jul 16 21:25:27.003438 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-b2669453-03df-49df-a1e2-4bd53a933a4f req-376a87f5-bca3-4767-b2c7-b835e77492b9 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 346 microversion: 2.1 time: 0.050340 Jul 16 21:25:27.004678 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1806/3598] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:25:26 2022] POST /compute/v2.1/os-server-external-events => generated 346 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:25:27.050349 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-ca5390e6-7270-47e8-adb9-90b0199008ed tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "7b097a34-dea9-4828-bbfe-ec7905fbb026"}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:25:27.051838 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-ca5390e6-7270-47e8-adb9-90b0199008ed tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] [instance: 2fad683a-f21f-4a9c-8323-e9380ef7f9ef] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:25:27.060472 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ca5390e6-7270-47e8-adb9-90b0199008ed tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:27.061722 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ca5390e6-7270-47e8-adb9-90b0199008ed tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:27.062785 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ca5390e6-7270-47e8-adb9-90b0199008ed tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:27.064514 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ca5390e6-7270-47e8-adb9-90b0199008ed tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:27.130994 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-2e2bc66c-cba1-4aaf-9ef6-7c67d63bcb38 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a23fafbab18d41ee82b84bbdf94cf0fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfc2e42a0df346f4adef152ba2e3fa71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:27.145650 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-2e2bc66c-cba1-4aaf-9ef6-7c67d63bcb38 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'a23fafbab18d41ee82b84bbdf94cf0fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfc2e42a0df346f4adef152ba2e3fa71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:27.147362 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-2e2bc66c-cba1-4aaf-9ef6-7c67d63bcb38 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'a23fafbab18d41ee82b84bbdf94cf0fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfc2e42a0df346f4adef152ba2e3fa71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:27.148920 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-2e2bc66c-cba1-4aaf-9ef6-7c67d63bcb38 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] 10.222.0.16 "GET /compute/v2.1/servers/4eb3d8d6-79ae-4109-a5a7-835c459d1719" status: 200 len: 1874 microversion: 2.32 time: 0.813515 Jul 16 21:25:27.162913 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1793/3599] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:25:26 2022] GET /compute/v2.1/servers/4eb3d8d6-79ae-4109-a5a7-835c459d1719 => generated 1874 bytes in 828 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:25:27.170517 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-03b43226-9668-4e3f-985b-7aef79d9bfc6 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:27.171626 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-03b43226-9668-4e3f-985b-7aef79d9bfc6 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: 05358d0e-ed2f-443c-835c-f77cd929289e] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:25:27.180332 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-03b43226-9668-4e3f-985b-7aef79d9bfc6 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:27.180798 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-03b43226-9668-4e3f-985b-7aef79d9bfc6 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:27.183892 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-03b43226-9668-4e3f-985b-7aef79d9bfc6 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:27.183892 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-03b43226-9668-4e3f-985b-7aef79d9bfc6 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:27.724260 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-03b43226-9668-4e3f-985b-7aef79d9bfc6 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '027dcc48e4fd4c36a5004096ad5e43f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dad823e240149edb30e592380249a13', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:27.734388 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-03b43226-9668-4e3f-985b-7aef79d9bfc6 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "GET /compute/v2.1/servers/05358d0e-ed2f-443c-835c-f77cd929289e" status: 200 len: 1309 microversion: 2.1 time: 0.569427 Jul 16 21:25:27.735283 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1794/3600] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:25:27 2022] GET /compute/v2.1/servers/05358d0e-ed2f-443c-835c-f77cd929289e => generated 1309 bytes in 572 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:25:27.740057 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-060833f0-1098-4cf1-837f-e988bba64929 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:27.741197 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-060833f0-1098-4cf1-837f-e988bba64929 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] [instance: 92fbfe5e-07b4-44ec-9729-1a9230f4c02c] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:25:27.750010 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-060833f0-1098-4cf1-837f-e988bba64929 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:27.750559 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-060833f0-1098-4cf1-837f-e988bba64929 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:27.751762 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-060833f0-1098-4cf1-837f-e988bba64929 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:27.752990 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-060833f0-1098-4cf1-837f-e988bba64929 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:27.979854 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-060833f0-1098-4cf1-837f-e988bba64929 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcc9aa2a091c490599f9a9c41163e2eb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da56fc78f06142aba80ca4c6e3d60475', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:27.987670 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-060833f0-1098-4cf1-837f-e988bba64929 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] 10.222.0.16 "GET /compute/v2.1/servers/92fbfe5e-07b4-44ec-9729-1a9230f4c02c" status: 200 len: 1511 microversion: 2.1 time: 0.251107 Jul 16 21:25:27.988461 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1795/3601] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:25:27 2022] GET /compute/v2.1/servers/92fbfe5e-07b4-44ec-9729-1a9230f4c02c => generated 1511 bytes in 252 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:25:27.996049 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-83b49129-2eac-4f83-8847-f217ea838a2f tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Calling method '.version_select at 0x7fdda2a609d0>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:28.057891 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-83b49129-2eac-4f83-8847-f217ea838a2f tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] 10.222.0.16 "GET /compute/v2.1/images/706863d6-a902-4b5c-abc5-1e3e8a4c0e7a" status: 200 len: 1421 microversion: 2.1 time: 0.068059 Jul 16 21:25:28.060573 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1796/3602] 10.222.0.16 () {58 vars in 1232 bytes} [Sat Jul 16 21:25:27 2022] GET /compute/v2.1/images/706863d6-a902-4b5c-abc5-1e3e8a4c0e7a => generated 1421 bytes in 69 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:25:28.064214 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-aa74862b-b81e-4882-b965-53149bc92071 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:28.065017 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-aa74862b-b81e-4882-b965-53149bc92071 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 00c8a753-5bf0-448c-8107-3946aa9bf7ae] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:25:28.072352 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-aa74862b-b81e-4882-b965-53149bc92071 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:28.072803 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-aa74862b-b81e-4882-b965-53149bc92071 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:28.073573 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-aa74862b-b81e-4882-b965-53149bc92071 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:28.074006 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-aa74862b-b81e-4882-b965-53149bc92071 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:28.703637 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-aa74862b-b81e-4882-b965-53149bc92071 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f2e974ab5aa48dfb4b00f810dcedb63', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd31be8475a6a49d6bfe8132dd050c6c0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:28.703637 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-aa74862b-b81e-4882-b965-53149bc92071 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "GET /compute/v2.1/servers/00c8a753-5bf0-448c-8107-3946aa9bf7ae" status: 200 len: 1314 microversion: 2.1 time: 0.623054 Jul 16 21:25:28.703637 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1797/3603] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:25:28 2022] GET /compute/v2.1/servers/00c8a753-5bf0-448c-8107-3946aa9bf7ae => generated 1314 bytes in 624 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:25:28.703637 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-215a9158-a959-498f-973f-ee844d9f6ba4 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:28.703637 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-215a9158-a959-498f-973f-ee844d9f6ba4 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] [instance: 4eb3d8d6-79ae-4109-a5a7-835c459d1719] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:25:28.703637 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-215a9158-a959-498f-973f-ee844d9f6ba4 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:28.703637 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-215a9158-a959-498f-973f-ee844d9f6ba4 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:28.703637 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-215a9158-a959-498f-973f-ee844d9f6ba4 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:28.703637 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-215a9158-a959-498f-973f-ee844d9f6ba4 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:29.225384 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-ca5390e6-7270-47e8-adb9-90b0199008ed tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] 10.222.0.16 "POST /compute/v2.1/servers/2fad683a-f21f-4a9c-8323-e9380ef7f9ef/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 2.179106 Jul 16 21:25:29.229011 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1807/3604] 10.222.0.16 () {60 vars in 1322 bytes} [Sat Jul 16 21:25:27 2022] POST /compute/v2.1/servers/2fad683a-f21f-4a9c-8323-e9380ef7f9ef/os-volume_attachments => generated 194 bytes in 2183 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 16 21:25:29.252901 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-425c489c-ff6e-431e-bbda-7c9d92274a7b tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:29.256129 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-425c489c-ff6e-431e-bbda-7c9d92274a7b tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: 05358d0e-ed2f-443c-835c-f77cd929289e] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:25:29.280544 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-425c489c-ff6e-431e-bbda-7c9d92274a7b tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:29.281589 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-425c489c-ff6e-431e-bbda-7c9d92274a7b tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:29.282811 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-425c489c-ff6e-431e-bbda-7c9d92274a7b tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:29.285249 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-425c489c-ff6e-431e-bbda-7c9d92274a7b tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:29.319546 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:25:29.840267 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-215a9158-a959-498f-973f-ee844d9f6ba4 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a23fafbab18d41ee82b84bbdf94cf0fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfc2e42a0df346f4adef152ba2e3fa71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:29.878418 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-215a9158-a959-498f-973f-ee844d9f6ba4 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'a23fafbab18d41ee82b84bbdf94cf0fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfc2e42a0df346f4adef152ba2e3fa71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:29.881466 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-215a9158-a959-498f-973f-ee844d9f6ba4 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'a23fafbab18d41ee82b84bbdf94cf0fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfc2e42a0df346f4adef152ba2e3fa71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:29.885925 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-215a9158-a959-498f-973f-ee844d9f6ba4 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] 10.222.0.16 "GET /compute/v2.1/servers/4eb3d8d6-79ae-4109-a5a7-835c459d1719" status: 200 len: 1874 microversion: 2.32 time: 1.200138 Jul 16 21:25:29.887092 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1798/3605] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:25:28 2022] GET /compute/v2.1/servers/4eb3d8d6-79ae-4109-a5a7-835c459d1719 => generated 1874 bytes in 1203 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:25:29.893339 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-dcca4fb2-088d-400e-9e13-2b8837f60068 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:29.895727 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-dcca4fb2-088d-400e-9e13-2b8837f60068 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] [instance: 92fbfe5e-07b4-44ec-9729-1a9230f4c02c] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:25:29.905146 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-dcca4fb2-088d-400e-9e13-2b8837f60068 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:29.906658 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-dcca4fb2-088d-400e-9e13-2b8837f60068 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:29.908183 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-dcca4fb2-088d-400e-9e13-2b8837f60068 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:29.909260 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-dcca4fb2-088d-400e-9e13-2b8837f60068 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:30.109782 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-425c489c-ff6e-431e-bbda-7c9d92274a7b tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '027dcc48e4fd4c36a5004096ad5e43f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dad823e240149edb30e592380249a13', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:30.117624 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-425c489c-ff6e-431e-bbda-7c9d92274a7b tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "GET /compute/v2.1/servers/05358d0e-ed2f-443c-835c-f77cd929289e" status: 200 len: 1309 microversion: 2.1 time: 0.878588 Jul 16 21:25:30.128823 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1808/3606] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:25:29 2022] GET /compute/v2.1/servers/05358d0e-ed2f-443c-835c-f77cd929289e => generated 1309 bytes in 890 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:25:30.131418 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-cd294a43-fb15-472d-9661-b2c926aa07cb req-f37f1cbf-68ae-4146-9642-b766300815ec service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "4eb3d8d6-79ae-4109-a5a7-835c459d1719", "tag": "b73090b3-6ab1-4f2c-b09d-274c9f89f289"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:25:30.153610 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-cd294a43-fb15-472d-9661-b2c926aa07cb req-f37f1cbf-68ae-4146-9642-b766300815ec service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:30.153992 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-cd294a43-fb15-472d-9661-b2c926aa07cb req-f37f1cbf-68ae-4146-9642-b766300815ec service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:30.180613 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-cd294a43-fb15-472d-9661-b2c926aa07cb req-f37f1cbf-68ae-4146-9642-b766300815ec service nova] Creating event network-changed:b73090b3-6ab1-4f2c-b09d-274c9f89f289 for instance 4eb3d8d6-79ae-4109-a5a7-835c459d1719 on nch1-master-1 Jul 16 21:25:30.190406 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-cd294a43-fb15-472d-9661-b2c926aa07cb req-f37f1cbf-68ae-4146-9642-b766300815ec service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.070706 Jul 16 21:25:30.191655 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1809/3607] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:25:30 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 73 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:25:30.198371 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-39db75ee-1c02-430a-a7a7-f88aef1c8f5e tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Calling method '.version_select at 0x7fdd9cea84c0>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:30.267421 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-39db75ee-1c02-430a-a7a7-f88aef1c8f5e tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] 10.222.0.16 "GET /compute/v2.1/images/706863d6-a902-4b5c-abc5-1e3e8a4c0e7a" status: 200 len: 1430 microversion: 2.1 time: 0.074300 Jul 16 21:25:30.276898 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1810/3608] 10.222.0.16 () {58 vars in 1232 bytes} [Sat Jul 16 21:25:30 2022] GET /compute/v2.1/images/706863d6-a902-4b5c-abc5-1e3e8a4c0e7a => generated 1430 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:25:30.282295 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-7094e781-fe48-4fd5-8e34-69eb9260efbc tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:30.283277 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-7094e781-fe48-4fd5-8e34-69eb9260efbc tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 00c8a753-5bf0-448c-8107-3946aa9bf7ae] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:25:30.293219 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7094e781-fe48-4fd5-8e34-69eb9260efbc tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:30.293864 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7094e781-fe48-4fd5-8e34-69eb9260efbc tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:30.294803 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7094e781-fe48-4fd5-8e34-69eb9260efbc tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:30.295445 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7094e781-fe48-4fd5-8e34-69eb9260efbc tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:30.495912 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-dcca4fb2-088d-400e-9e13-2b8837f60068 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcc9aa2a091c490599f9a9c41163e2eb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da56fc78f06142aba80ca4c6e3d60475', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:30.504636 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-dcca4fb2-088d-400e-9e13-2b8837f60068 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] 10.222.0.16 "GET /compute/v2.1/servers/92fbfe5e-07b4-44ec-9729-1a9230f4c02c" status: 200 len: 1517 microversion: 2.1 time: 0.616243 Jul 16 21:25:30.508244 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1799/3609] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:25:29 2022] GET /compute/v2.1/servers/92fbfe5e-07b4-44ec-9729-1a9230f4c02c => generated 1517 bytes in 620 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:25:30.517729 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-9f728767-45d3-496e-a17e-838faf8dd84a tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Calling method '.version_select at 0x7fdd9c331160>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:30.604547 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-9f728767-45d3-496e-a17e-838faf8dd84a tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] 10.222.0.16 "GET /compute/v2.1/images/706863d6-a902-4b5c-abc5-1e3e8a4c0e7a" status: 200 len: 1430 microversion: 2.1 time: 0.083534 Jul 16 21:25:30.614614 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1800/3610] 10.222.0.16 () {58 vars in 1232 bytes} [Sat Jul 16 21:25:30 2022] GET /compute/v2.1/images/706863d6-a902-4b5c-abc5-1e3e8a4c0e7a => generated 1430 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:25:30.627310 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-82c649f3-0fb2-4252-8731-4122c8743716 tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:30.627929 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-82c649f3-0fb2-4252-8731-4122c8743716 tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] [instance: f00bea08-4180-45d6-ba01-4b104f5615d6] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:25:30.654444 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-82c649f3-0fb2-4252-8731-4122c8743716 tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:30.660442 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-82c649f3-0fb2-4252-8731-4122c8743716 tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:30.661290 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-82c649f3-0fb2-4252-8731-4122c8743716 tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:30.665361 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-82c649f3-0fb2-4252-8731-4122c8743716 tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:31.117960 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-7094e781-fe48-4fd5-8e34-69eb9260efbc tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f2e974ab5aa48dfb4b00f810dcedb63', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd31be8475a6a49d6bfe8132dd050c6c0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:31.131294 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-7094e781-fe48-4fd5-8e34-69eb9260efbc tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "GET /compute/v2.1/servers/00c8a753-5bf0-448c-8107-3946aa9bf7ae" status: 200 len: 1314 microversion: 2.1 time: 0.852771 Jul 16 21:25:31.133077 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1811/3611] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:25:30 2022] GET /compute/v2.1/servers/00c8a753-5bf0-448c-8107-3946aa9bf7ae => generated 1314 bytes in 855 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:25:31.155569 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-0c025c41-070c-4d7d-b004-3f2ea17ef061 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:31.156736 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-0c025c41-070c-4d7d-b004-3f2ea17ef061 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] [instance: 4eb3d8d6-79ae-4109-a5a7-835c459d1719] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:25:31.165124 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0c025c41-070c-4d7d-b004-3f2ea17ef061 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:31.165741 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0c025c41-070c-4d7d-b004-3f2ea17ef061 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:31.166790 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0c025c41-070c-4d7d-b004-3f2ea17ef061 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:31.167385 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0c025c41-070c-4d7d-b004-3f2ea17ef061 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:31.394066 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-82c649f3-0fb2-4252-8731-4122c8743716 tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3b0b46c0c1f74adaaaf65d46db7b2618', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '336b3ef8a8744bf3b86854b32000bda6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:31.401282 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:25:31.403248 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-82c649f3-0fb2-4252-8731-4122c8743716 tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] 10.222.0.16 "GET /compute/v2.1/servers/f00bea08-4180-45d6-ba01-4b104f5615d6" status: 200 len: 1479 microversion: 2.1 time: 0.787492 Jul 16 21:25:31.407348 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1801/3612] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:25:30 2022] GET /compute/v2.1/servers/f00bea08-4180-45d6-ba01-4b104f5615d6 => generated 1479 bytes in 792 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:25:31.430121 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-04981a4b-a20e-42d2-b94a-d1b66d3a3788 req-9cbfbaae-a115-4804-8abb-cb1ccf4b1262 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "05358d0e-ed2f-443c-835c-f77cd929289e", "name": "network-vif-plugged", "status": "completed", "tag": "9675fc80-e092-4333-8050-341c379735a0"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:25:31.447456 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-04981a4b-a20e-42d2-b94a-d1b66d3a3788 req-9cbfbaae-a115-4804-8abb-cb1ccf4b1262 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:31.452900 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-04981a4b-a20e-42d2-b94a-d1b66d3a3788 req-9cbfbaae-a115-4804-8abb-cb1ccf4b1262 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:31.463419 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-04981a4b-a20e-42d2-b94a-d1b66d3a3788 req-9cbfbaae-a115-4804-8abb-cb1ccf4b1262 service nova] Creating event network-vif-plugged:9675fc80-e092-4333-8050-341c379735a0 for instance 05358d0e-ed2f-443c-835c-f77cd929289e on nch2-master-1 Jul 16 21:25:31.468754 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-04981a4b-a20e-42d2-b94a-d1b66d3a3788 req-9cbfbaae-a115-4804-8abb-cb1ccf4b1262 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.059189 Jul 16 21:25:31.469822 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1802/3613] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:25:31 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 62 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:25:31.477586 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-87d9ff2c-2048-4d2f-a876-8cfbc8f8aea2 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:31.478439 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-87d9ff2c-2048-4d2f-a876-8cfbc8f8aea2 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: 05358d0e-ed2f-443c-835c-f77cd929289e] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:25:31.486365 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-87d9ff2c-2048-4d2f-a876-8cfbc8f8aea2 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:31.487141 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-87d9ff2c-2048-4d2f-a876-8cfbc8f8aea2 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:31.489041 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-87d9ff2c-2048-4d2f-a876-8cfbc8f8aea2 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:31.489944 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-87d9ff2c-2048-4d2f-a876-8cfbc8f8aea2 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:31.723104 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-0c025c41-070c-4d7d-b004-3f2ea17ef061 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a23fafbab18d41ee82b84bbdf94cf0fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfc2e42a0df346f4adef152ba2e3fa71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:31.738229 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-0c025c41-070c-4d7d-b004-3f2ea17ef061 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'a23fafbab18d41ee82b84bbdf94cf0fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfc2e42a0df346f4adef152ba2e3fa71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:31.739883 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-0c025c41-070c-4d7d-b004-3f2ea17ef061 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'a23fafbab18d41ee82b84bbdf94cf0fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfc2e42a0df346f4adef152ba2e3fa71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:31.743439 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-0c025c41-070c-4d7d-b004-3f2ea17ef061 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] 10.222.0.16 "GET /compute/v2.1/servers/4eb3d8d6-79ae-4109-a5a7-835c459d1719" status: 200 len: 1874 microversion: 2.32 time: 0.604886 Jul 16 21:25:31.744383 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1812/3614] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:25:31 2022] GET /compute/v2.1/servers/4eb3d8d6-79ae-4109-a5a7-835c459d1719 => generated 1874 bytes in 610 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:25:31.817750 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-87d9ff2c-2048-4d2f-a876-8cfbc8f8aea2 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '027dcc48e4fd4c36a5004096ad5e43f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dad823e240149edb30e592380249a13', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:31.833117 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-87d9ff2c-2048-4d2f-a876-8cfbc8f8aea2 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "GET /compute/v2.1/servers/05358d0e-ed2f-443c-835c-f77cd929289e" status: 200 len: 1309 microversion: 2.1 time: 0.361786 Jul 16 21:25:31.833702 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1803/3615] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:25:31 2022] GET /compute/v2.1/servers/05358d0e-ed2f-443c-835c-f77cd929289e => generated 1309 bytes in 363 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:25:32.152736 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-006c9fd3-6f4a-4f9b-acbf-08668b02c484 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:32.153288 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-006c9fd3-6f4a-4f9b-acbf-08668b02c484 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 00c8a753-5bf0-448c-8107-3946aa9bf7ae] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:25:32.167308 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-006c9fd3-6f4a-4f9b-acbf-08668b02c484 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:32.167677 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-006c9fd3-6f4a-4f9b-acbf-08668b02c484 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:32.168459 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-006c9fd3-6f4a-4f9b-acbf-08668b02c484 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:32.168905 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-006c9fd3-6f4a-4f9b-acbf-08668b02c484 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:32.233770 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-54e76346-3836-4828-8640-f725a9283bfe req-39f1796a-051f-44c2-b17d-1548944440cc service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "4eb3d8d6-79ae-4109-a5a7-835c459d1719", "tag": "0ae97a58-1fb7-46af-ad9b-e1ebb2af5923"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:25:32.244216 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-54e76346-3836-4828-8640-f725a9283bfe req-39f1796a-051f-44c2-b17d-1548944440cc service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:32.244668 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-54e76346-3836-4828-8640-f725a9283bfe req-39f1796a-051f-44c2-b17d-1548944440cc service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:32.258862 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-54e76346-3836-4828-8640-f725a9283bfe req-39f1796a-051f-44c2-b17d-1548944440cc service nova] Creating event network-changed:0ae97a58-1fb7-46af-ad9b-e1ebb2af5923 for instance 4eb3d8d6-79ae-4109-a5a7-835c459d1719 on nch1-master-1 Jul 16 21:25:32.265104 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-54e76346-3836-4828-8640-f725a9283bfe req-39f1796a-051f-44c2-b17d-1548944440cc service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.034904 Jul 16 21:25:32.268018 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1804/3616] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:25:32 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 16 21:25:32.382827 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-006c9fd3-6f4a-4f9b-acbf-08668b02c484 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f2e974ab5aa48dfb4b00f810dcedb63', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd31be8475a6a49d6bfe8132dd050c6c0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:32.390741 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-006c9fd3-6f4a-4f9b-acbf-08668b02c484 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "GET /compute/v2.1/servers/00c8a753-5bf0-448c-8107-3946aa9bf7ae" status: 200 len: 1314 microversion: 2.1 time: 0.244872 Jul 16 21:25:32.391370 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1813/3617] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:25:32 2022] GET /compute/v2.1/servers/00c8a753-5bf0-448c-8107-3946aa9bf7ae => generated 1314 bytes in 246 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:25:32.423893 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-9edd2245-51fb-4592-8824-6ff2f88806af tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:32.424547 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-9edd2245-51fb-4592-8824-6ff2f88806af tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] [instance: f00bea08-4180-45d6-ba01-4b104f5615d6] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:25:32.431364 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9edd2245-51fb-4592-8824-6ff2f88806af tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:32.431822 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9edd2245-51fb-4592-8824-6ff2f88806af tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:32.432593 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9edd2245-51fb-4592-8824-6ff2f88806af tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:32.433034 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9edd2245-51fb-4592-8824-6ff2f88806af tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:32.594943 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-9edd2245-51fb-4592-8824-6ff2f88806af tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3b0b46c0c1f74adaaaf65d46db7b2618', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '336b3ef8a8744bf3b86854b32000bda6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:32.604847 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-9edd2245-51fb-4592-8824-6ff2f88806af tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] 10.222.0.16 "GET /compute/v2.1/servers/f00bea08-4180-45d6-ba01-4b104f5615d6" status: 200 len: 1466 microversion: 2.1 time: 0.185193 Jul 16 21:25:32.605946 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1805/3618] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:25:32 2022] GET /compute/v2.1/servers/f00bea08-4180-45d6-ba01-4b104f5615d6 => generated 1466 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:25:32.625397 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-9fd2ecae-816f-45a8-a466-8ac614f6a78c tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Calling method '.version_select at 0x7fdd9cd91dc0>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:32.675729 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-9fd2ecae-816f-45a8-a466-8ac614f6a78c tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] 10.222.0.16 "GET /compute/v2.1/images/1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9" status: 200 len: 784 microversion: 2.1 time: 0.054354 Jul 16 21:25:32.676330 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1814/3619] 10.222.0.16 () {58 vars in 1232 bytes} [Sat Jul 16 21:25:32 2022] GET /compute/v2.1/images/1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9 => generated 784 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:25:32.688427 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-6fd34229-e377-4235-b8a4-572e253195f1 tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:32.691376 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_api:SELECT=96,nova_api:DELETE=1,nova_api:INSERT=3 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:25:32.697713 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-6fd34229-e377-4235-b8a4-572e253195f1 tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] 10.222.0.16 "GET /compute/v2.1/flavors/452" status: 200 len: 370 microversion: 2.1 time: 0.013165 Jul 16 21:25:32.698278 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1806/3620] 10.222.0.16 () {58 vars in 1136 bytes} [Sat Jul 16 21:25:32 2022] GET /compute/v2.1/flavors/452 => generated 370 bytes in 14 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:25:32.709052 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-43a03c72-872d-408d-8c18-8214e96396f2 tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Calling method '.version_select at 0x7fdd9d328670>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:32.763959 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-4ac62505-0b84-4b78-8282-f25df9e48c32 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:32.765092 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-4ac62505-0b84-4b78-8282-f25df9e48c32 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] [instance: 4eb3d8d6-79ae-4109-a5a7-835c459d1719] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:25:32.774699 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4ac62505-0b84-4b78-8282-f25df9e48c32 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:32.775292 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4ac62505-0b84-4b78-8282-f25df9e48c32 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:32.776426 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4ac62505-0b84-4b78-8282-f25df9e48c32 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:32.776942 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4ac62505-0b84-4b78-8282-f25df9e48c32 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:33.130784 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-4ac62505-0b84-4b78-8282-f25df9e48c32 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a23fafbab18d41ee82b84bbdf94cf0fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfc2e42a0df346f4adef152ba2e3fa71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:33.141164 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-4ac62505-0b84-4b78-8282-f25df9e48c32 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'a23fafbab18d41ee82b84bbdf94cf0fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfc2e42a0df346f4adef152ba2e3fa71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:33.142864 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-4ac62505-0b84-4b78-8282-f25df9e48c32 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'a23fafbab18d41ee82b84bbdf94cf0fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfc2e42a0df346f4adef152ba2e3fa71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:33.144282 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-4ac62505-0b84-4b78-8282-f25df9e48c32 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] 10.222.0.16 "GET /compute/v2.1/servers/4eb3d8d6-79ae-4109-a5a7-835c459d1719" status: 200 len: 1874 microversion: 2.32 time: 0.386005 Jul 16 21:25:33.145104 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1807/3621] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:25:32 2022] GET /compute/v2.1/servers/4eb3d8d6-79ae-4109-a5a7-835c459d1719 => generated 1874 bytes in 388 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 16 21:25:33.151537 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-a40dbc98-96e3-4539-91e3-47fa28affa23 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:33.159995 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-a40dbc98-96e3-4539-91e3-47fa28affa23 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: 05358d0e-ed2f-443c-835c-f77cd929289e] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:25:33.165503 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a40dbc98-96e3-4539-91e3-47fa28affa23 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:33.166934 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a40dbc98-96e3-4539-91e3-47fa28affa23 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:33.168294 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a40dbc98-96e3-4539-91e3-47fa28affa23 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:33.168947 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a40dbc98-96e3-4539-91e3-47fa28affa23 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:33.398103 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-a40dbc98-96e3-4539-91e3-47fa28affa23 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '027dcc48e4fd4c36a5004096ad5e43f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dad823e240149edb30e592380249a13', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:33.408091 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-a40dbc98-96e3-4539-91e3-47fa28affa23 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "GET /compute/v2.1/servers/05358d0e-ed2f-443c-835c-f77cd929289e" status: 200 len: 1490 microversion: 2.1 time: 0.261825 Jul 16 21:25:33.408828 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1808/3622] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:25:33 2022] GET /compute/v2.1/servers/05358d0e-ed2f-443c-835c-f77cd929289e => generated 1490 bytes in 263 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:25:33.414786 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-cbd6189a-a71f-4e9e-9db3-212c0a753eda tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:33.415417 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-cbd6189a-a71f-4e9e-9db3-212c0a753eda tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 00c8a753-5bf0-448c-8107-3946aa9bf7ae] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:25:33.422490 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cbd6189a-a71f-4e9e-9db3-212c0a753eda tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:33.423150 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cbd6189a-a71f-4e9e-9db3-212c0a753eda tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:33.424277 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cbd6189a-a71f-4e9e-9db3-212c0a753eda tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:33.425430 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cbd6189a-a71f-4e9e-9db3-212c0a753eda tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:34.078284 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-cbd6189a-a71f-4e9e-9db3-212c0a753eda tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f2e974ab5aa48dfb4b00f810dcedb63', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd31be8475a6a49d6bfe8132dd050c6c0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:34.088103 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-cbd6189a-a71f-4e9e-9db3-212c0a753eda tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "GET /compute/v2.1/servers/00c8a753-5bf0-448c-8107-3946aa9bf7ae" status: 200 len: 1314 microversion: 2.1 time: 0.677966 Jul 16 21:25:34.089405 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1809/3623] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:25:33 2022] GET /compute/v2.1/servers/00c8a753-5bf0-448c-8107-3946aa9bf7ae => generated 1314 bytes in 679 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:25:34.094270 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-321dc05d-4de6-4454-91b8-730945a1cfcd req-80de457d-1784-4dae-b633-3001d63963b0 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "00c8a753-5bf0-448c-8107-3946aa9bf7ae", "name": "network-vif-plugged", "status": "completed", "tag": "bb33931c-0e67-4fab-ae33-70ed7930c4be"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:25:34.104062 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-321dc05d-4de6-4454-91b8-730945a1cfcd req-80de457d-1784-4dae-b633-3001d63963b0 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:34.104524 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-321dc05d-4de6-4454-91b8-730945a1cfcd req-80de457d-1784-4dae-b633-3001d63963b0 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:34.117332 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-321dc05d-4de6-4454-91b8-730945a1cfcd req-80de457d-1784-4dae-b633-3001d63963b0 service nova] Creating event network-vif-plugged:bb33931c-0e67-4fab-ae33-70ed7930c4be for instance 00c8a753-5bf0-448c-8107-3946aa9bf7ae on nch2-master-1 Jul 16 21:25:34.122307 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-321dc05d-4de6-4454-91b8-730945a1cfcd req-80de457d-1784-4dae-b633-3001d63963b0 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.031708 Jul 16 21:25:34.122997 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1810/3624] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:25:34 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:25:34.140154 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-c46db920-7445-4587-928d-dc5361bc5d87 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Action: 'create', calling method: .version_select at 0x7fdd9c3468b0>, body: {"keypair": {"name": "tempest-keypair-1244213655", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBJA5fh1xceTeh3q7G508CyPqNigvjMfiNm/P6NP0mJbsKDKvMCuu3o8oH+/dBWrdPS4YCAjfHVLp3bEst7U752xI4kUwAScN4SrwNkQSlzstnl/je2Lc6ErrmYHz8m+W+g=="}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:25:34.156598 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-c46db920-7445-4587-928d-dc5361bc5d87 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Getting quotas for project da56fc78f06142aba80ca4c6e3d60475. Resources: {'key_pairs'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:25:34.165866 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-c46db920-7445-4587-928d-dc5361bc5d87 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Getting quotas for user bcc9aa2a091c490599f9a9c41163e2eb and project da56fc78f06142aba80ca4c6e3d60475. Resources: {'key_pairs'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:25:34.285585 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-c46db920-7445-4587-928d-dc5361bc5d87 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] 10.222.0.16 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.161095 Jul 16 21:25:34.286671 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1811/3625] 10.222.0.16 () {60 vars in 1158 bytes} [Sat Jul 16 21:25:34 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:25:34.292569 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-c0c0aaf7-16b7-4d0b-84d4-a409aaa5388f tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServersTestJSON-server-1313984590", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452", "networks": [{"uuid": "2202d4ad-313c-4607-b9fc-eb98bac9afd0"}]}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:25:34.297939 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-43a03c72-872d-408d-8c18-8214e96396f2 tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] 10.222.0.16 "DELETE /compute/v2.1/images/706863d6-a902-4b5c-abc5-1e3e8a4c0e7a" status: 204 len: 0 microversion: 2.1 time: 1.592700 Jul 16 21:25:34.298690 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1815/3626] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:25:32 2022] DELETE /compute/v2.1/images/706863d6-a902-4b5c-abc5-1e3e8a4c0e7a => generated 0 bytes in 1595 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 16 21:25:34.327101 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-1ad0906d-e280-4361-b7b3-44ff3b7dceec tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:34.328133 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-1ad0906d-e280-4361-b7b3-44ff3b7dceec tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] [instance: 2fad683a-f21f-4a9c-8323-e9380ef7f9ef] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:25:34.337954 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1ad0906d-e280-4361-b7b3-44ff3b7dceec tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:34.337954 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1ad0906d-e280-4361-b7b3-44ff3b7dceec tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:34.337954 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1ad0906d-e280-4361-b7b3-44ff3b7dceec tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:34.338420 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1ad0906d-e280-4361-b7b3-44ff3b7dceec tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:34.388069 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.network.neutron [None req-c0c0aaf7-16b7-4d0b-84d4-a409aaa5388f tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] validate_networks() for [('2202d4ad-313c-4607-b9fc-eb98bac9afd0', None, None, None, None, None)] {{(pid=138681) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 16 21:25:34.417273 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-1ad0906d-e280-4361-b7b3-44ff3b7dceec tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] 10.222.0.16 "GET /compute/v2.1/servers/2fad683a-f21f-4a9c-8323-e9380ef7f9ef/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.116764 Jul 16 21:25:34.422869 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1816/3627] 10.222.0.16 () {58 vars in 1301 bytes} [Sat Jul 16 21:25:34 2022] GET /compute/v2.1/servers/2fad683a-f21f-4a9c-8323-e9380ef7f9ef/os-volume_attachments => generated 197 bytes in 123 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:25:34.447821 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-9d3b1b83-278a-4b29-a532-f359a19798d8 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:34.449619 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-9d3b1b83-278a-4b29-a532-f359a19798d8 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] [instance: 4eb3d8d6-79ae-4109-a5a7-835c459d1719] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:25:34.462173 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9d3b1b83-278a-4b29-a532-f359a19798d8 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:34.465092 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9d3b1b83-278a-4b29-a532-f359a19798d8 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:34.466245 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9d3b1b83-278a-4b29-a532-f359a19798d8 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:34.466706 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9d3b1b83-278a-4b29-a532-f359a19798d8 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:34.900219 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-9d3b1b83-278a-4b29-a532-f359a19798d8 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a23fafbab18d41ee82b84bbdf94cf0fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfc2e42a0df346f4adef152ba2e3fa71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:34.909383 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-9d3b1b83-278a-4b29-a532-f359a19798d8 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'a23fafbab18d41ee82b84bbdf94cf0fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfc2e42a0df346f4adef152ba2e3fa71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:34.910899 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-9d3b1b83-278a-4b29-a532-f359a19798d8 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'a23fafbab18d41ee82b84bbdf94cf0fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfc2e42a0df346f4adef152ba2e3fa71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:34.912138 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-9d3b1b83-278a-4b29-a532-f359a19798d8 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] 10.222.0.16 "GET /compute/v2.1/servers/4eb3d8d6-79ae-4109-a5a7-835c459d1719" status: 200 len: 1874 microversion: 2.32 time: 0.480191 Jul 16 21:25:34.912702 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1817/3628] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:25:34 2022] GET /compute/v2.1/servers/4eb3d8d6-79ae-4109-a5a7-835c459d1719 => generated 1874 bytes in 489 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:25:34.918743 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-a5fcf8cd-b84e-45d2-8a13-679a37b2c542 tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Calling method '.version_select at 0x7fdda2bc8af0>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:34.942370 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-a5fcf8cd-b84e-45d2-8a13-679a37b2c542 tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] HTTP exception thrown: Image not found. Jul 16 21:25:34.950191 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-a5fcf8cd-b84e-45d2-8a13-679a37b2c542 tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Request method failure captured: Jul 16 21:25:34.950191 ncdv-master-1 devstack@n-api.service[138680]: request: GET /compute/v2.1/images/706863d6-a902-4b5c-abc5-1e3e8a4c0e7a HTTP/1.1 Jul 16 21:25:34.950191 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:25:34.950191 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:25:34.950191 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:25:34.950191 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 0 Jul 16 21:25:34.950191 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:25:34.950191 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:25:34.950191 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:25:34.950191 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:25:34.950191 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0yyeoZK6r42ZrgYKqzqjzfWlSpAUSwogZvgrU6vI5Ma1BzJUf43iC9DYxchs8EE1qwBvPIbjkNWL7_t3fG2X51Xcxn7MHwhtCU8FoNCaqNnP_KMgNcUFs2fhiEN-sCwj_YUQi35-Y6CAIeyHbGRaJXNXIlr4pCnQFYcZd01OxOnAGvU Jul 16 21:25:34.950191 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:25:34.950191 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:25:34.950191 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:25:34.950191 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:25:34.950191 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:25:34.950191 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:25:34.950191 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: 336b3ef8a8744bf3b86854b32000bda6 Jul 16 21:25:34.950191 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-ImagesOneServerTestJSON-741929816 Jul 16 21:25:34.950191 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:25:34.950191 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:25:34.950191 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/336b3ef8a8744bf3b86854b32000bda6"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/336b3ef8a8744bf3b86854b32000bda6"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_336b3ef8a8744bf3b86854b32000bda6"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/336b3ef8a8744bf3b86854b32000bda6"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:25:34.950191 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-ImagesOneServerTestJSON-741929816 Jul 16 21:25:34.950191 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: 336b3ef8a8744bf3b86854b32000bda6 Jul 16 21:25:34.950191 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-ImagesOneServerTestJSON-741929816 Jul 16 21:25:34.950191 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-ImagesOneServerTestJSON-741929816-project Jul 16 21:25:34.950191 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:25:34.950191 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:25:34.950191 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: 3b0b46c0c1f74adaaaf65d46db7b2618 Jul 16 21:25:34.950191 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-ImagesOneServerTestJSON-741929816-project Jul 16 21:25:34.950191 ncdv-master-1 devstack@n-api.service[138680]: method: Jul 16 21:25:34.950191 ncdv-master-1 devstack@n-api.service[138680]: exception: Image not found. Jul 16 21:25:34.950191 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': '706863d6-a902-4b5c-abc5-1e3e8a4c0e7a'} Jul 16 21:25:34.950191 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:25:34.950191 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:25:34.950191 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 285, in show Jul 16 21:25:34.950191 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi image = self._client.call(context, 2, 'get', args=(image_id,)) Jul 16 21:25:34.950191 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 191, in call Jul 16 21:25:34.950191 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi result = getattr(controller, method)(*args, **kwargs) Jul 16 21:25:34.950191 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/v2/images.py", line 197, in get Jul 16 21:25:34.950191 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return self._get(image_id) Jul 16 21:25:34.950191 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/utils.py", line 670, in inner Jul 16 21:25:34.950191 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return RequestIdProxy(wrapped(*args, **kwargs)) Jul 16 21:25:34.950191 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/v2/images.py", line 190, in _get Jul 16 21:25:34.950191 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi resp, body = self.http_client.get(url, headers=header) Jul 16 21:25:34.950191 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/adapter.py", line 395, in get Jul 16 21:25:34.950191 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return self.request(url, 'GET', **kwargs) Jul 16 21:25:34.950191 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 380, in request Jul 16 21:25:34.950191 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return self._handle_response(resp) Jul 16 21:25:34.950191 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 120, in _handle_response Jul 16 21:25:34.950191 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.from_response(resp, resp.content) Jul 16 21:25:34.950191 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi glanceclient.exc.HTTPNotFound: HTTP 404 Not Found: No image found with ID 706863d6-a902-4b5c-abc5-1e3e8a4c0e7a Jul 16 21:25:34.950191 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:25:34.950191 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:25:34.950191 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:25:34.950191 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:25:34.950191 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/images.py", line 86, in show Jul 16 21:25:34.950191 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi image = self._image_api.get(context, id) Jul 16 21:25:34.950191 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 1205, in get Jul 16 21:25:34.950191 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return session.show(context, image_id, Jul 16 21:25:34.950191 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 287, in show Jul 16 21:25:34.950191 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi _reraise_translated_image_exception(image_id) Jul 16 21:25:34.950191 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 1031, in _reraise_translated_image_exception Jul 16 21:25:34.950191 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise new_exc.with_traceback(exc_trace) Jul 16 21:25:34.955995 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 285, in show Jul 16 21:25:34.955995 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi image = self._client.call(context, 2, 'get', args=(image_id,)) Jul 16 21:25:34.955995 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 191, in call Jul 16 21:25:34.955995 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi result = getattr(controller, method)(*args, **kwargs) Jul 16 21:25:34.955995 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/v2/images.py", line 197, in get Jul 16 21:25:34.955995 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return self._get(image_id) Jul 16 21:25:34.955995 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/utils.py", line 670, in inner Jul 16 21:25:34.955995 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return RequestIdProxy(wrapped(*args, **kwargs)) Jul 16 21:25:34.955995 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/v2/images.py", line 190, in _get Jul 16 21:25:34.955995 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi resp, body = self.http_client.get(url, headers=header) Jul 16 21:25:34.955995 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/adapter.py", line 395, in get Jul 16 21:25:34.955995 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return self.request(url, 'GET', **kwargs) Jul 16 21:25:34.955995 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 380, in request Jul 16 21:25:34.955995 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return self._handle_response(resp) Jul 16 21:25:34.955995 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 120, in _handle_response Jul 16 21:25:34.955995 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.from_response(resp, resp.content) Jul 16 21:25:34.955995 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.ImageNotFound: Image 706863d6-a902-4b5c-abc5-1e3e8a4c0e7a could not be found. Jul 16 21:25:34.955995 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:25:34.955995 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:25:34.955995 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:25:34.955995 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:25:34.955995 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:25:34.955995 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:25:34.955995 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:25:34.955995 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:25:34.955995 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Jul 16 21:25:34.955995 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Jul 16 21:25:34.955995 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:25:34.955995 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:25:34.955995 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/images.py", line 89, in show Jul 16 21:25:34.955995 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=explanation) Jul 16 21:25:34.955995 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Image not found. Jul 16 21:25:34.955995 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:25:34.955995 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:25:34.955995 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:25:34.955995 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:25:34.955995 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:25:34.955995 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:25:34.955995 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:25:34.955995 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:25:34.955995 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Image not found. Jul 16 21:25:34.955995 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:25:34.955995 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-a5fcf8cd-b84e-45d2-8a13-679a37b2c542 tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Returning 404 to user: Image not found. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:25:34.955995 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-a5fcf8cd-b84e-45d2-8a13-679a37b2c542 tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] 10.222.0.16 "GET /compute/v2.1/images/706863d6-a902-4b5c-abc5-1e3e8a4c0e7a" status: 404 len: 62 microversion: 2.1 time: 0.037388 Jul 16 21:25:34.955995 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1818/3629] 10.222.0.16 () {58 vars in 1232 bytes} [Sat Jul 16 21:25:34 2022] GET /compute/v2.1/images/706863d6-a902-4b5c-abc5-1e3e8a4c0e7a => generated 62 bytes in 39 msecs (HTTP/1.1 404) 9 headers in 378 bytes (1 switches on core 0) Jul 16 21:25:34.962813 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-52a79522-096e-4c83-988e-d47dbfbc44ea tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:34.963571 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-52a79522-096e-4c83-988e-d47dbfbc44ea tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] [instance: 2fad683a-f21f-4a9c-8323-e9380ef7f9ef] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:25:34.966404 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_api:SELECT=96,nova_api:UPDATE=1,nova_api:INSERT=3 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:25:34.971883 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-52a79522-096e-4c83-988e-d47dbfbc44ea tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:34.972346 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-52a79522-096e-4c83-988e-d47dbfbc44ea tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:34.973137 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-52a79522-096e-4c83-988e-d47dbfbc44ea tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:34.973583 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-52a79522-096e-4c83-988e-d47dbfbc44ea tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:35.008039 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-52a79522-096e-4c83-988e-d47dbfbc44ea tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] 10.222.0.16 "GET /compute/v2.1/servers/2fad683a-f21f-4a9c-8323-e9380ef7f9ef/os-volume_attachments/7b097a34-dea9-4828-bbfe-ec7905fbb026" status: 200 len: 194 microversion: 2.1 time: 0.054420 Jul 16 21:25:35.008662 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1819/3630] 10.222.0.16 () {58 vars in 1412 bytes} [Sat Jul 16 21:25:34 2022] GET /compute/v2.1/servers/2fad683a-f21f-4a9c-8323-e9380ef7f9ef/os-volume_attachments/7b097a34-dea9-4828-bbfe-ec7905fbb026 => generated 194 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:25:35.014255 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-1e211af2-a524-48f6-8199-391cfe0bcb05 tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Action: 'action', calling method: >, body: {"createImage": {"name": "tempest-\u20a1-606375136"}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:25:35.016437 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-1e211af2-a524-48f6-8199-391cfe0bcb05 tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] [instance: f00bea08-4180-45d6-ba01-4b104f5615d6] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:25:35.022742 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1e211af2-a524-48f6-8199-391cfe0bcb05 tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:35.023283 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1e211af2-a524-48f6-8199-391cfe0bcb05 tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:35.026102 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1e211af2-a524-48f6-8199-391cfe0bcb05 tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:35.026102 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1e211af2-a524-48f6-8199-391cfe0bcb05 tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:35.375654 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-c0c0aaf7-16b7-4d0b-84d4-a409aaa5388f tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Flavor limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 16 21:25:35.376175 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-c0c0aaf7-16b7-4d0b-84d4-a409aaa5388f tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Image limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 16 21:25:35.376540 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-c0c0aaf7-16b7-4d0b-84d4-a409aaa5388f tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Flavor pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 16 21:25:35.376953 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-c0c0aaf7-16b7-4d0b-84d4-a409aaa5388f tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Image pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 16 21:25:35.377368 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-c0c0aaf7-16b7-4d0b-84d4-a409aaa5388f tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 16 21:25:35.396202 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c0c0aaf7-16b7-4d0b-84d4-a409aaa5388f tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:35.396858 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c0c0aaf7-16b7-4d0b-84d4-a409aaa5388f tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:35.400582 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c0c0aaf7-16b7-4d0b-84d4-a409aaa5388f tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:35.405160 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c0c0aaf7-16b7-4d0b-84d4-a409aaa5388f tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:35.422800 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-c0c0aaf7-16b7-4d0b-84d4-a409aaa5388f tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Getting quotas for project 1dad823e240149edb30e592380249a13. Resources: {'ram', 'instances', 'cores'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:25:35.429458 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-c0c0aaf7-16b7-4d0b-84d4-a409aaa5388f tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Getting quotas for user 027dcc48e4fd4c36a5004096ad5e43f8 and project 1dad823e240149edb30e592380249a13. Resources: {'ram', 'instances', 'cores'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:25:35.479093 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-c0c0aaf7-16b7-4d0b-84d4-a409aaa5388f tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Going to run 1 instances... {{(pid=138681) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 16 21:25:35.485469 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-c0c0aaf7-16b7-4d0b-84d4-a409aaa5388f tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: c20f3279-68ae-4b04-8b87-cc16e5f73f1e] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138681) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 16 21:25:35.718295 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-1e211af2-a524-48f6-8199-391cfe0bcb05 tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] 10.222.0.16 "POST /compute/v2.1/servers/f00bea08-4180-45d6-ba01-4b104f5615d6/action" status: 202 len: 0 microversion: 2.1 time: 0.707913 Jul 16 21:25:35.718906 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-c0c0aaf7-16b7-4d0b-84d4-a409aaa5388f tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 1.430340 Jul 16 21:25:35.724166 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1812/3631] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 21:25:34 2022] POST /compute/v2.1/servers => generated 380 bytes in 1432 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 16 21:25:35.729897 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-6e71483a-ba78-4af7-8f1f-3aff529731c3 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:35.731209 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-6e71483a-ba78-4af7-8f1f-3aff529731c3 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 00c8a753-5bf0-448c-8107-3946aa9bf7ae] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:25:35.732169 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1820/3632] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 21:25:35 2022] POST /compute/v2.1/servers/f00bea08-4180-45d6-ba01-4b104f5615d6/action => generated 0 bytes in 723 msecs (HTTP/1.1 202) 10 headers in 450 bytes (1 switches on core 0) Jul 16 21:25:35.741797 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6e71483a-ba78-4af7-8f1f-3aff529731c3 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:35.742533 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6e71483a-ba78-4af7-8f1f-3aff529731c3 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:35.743746 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6e71483a-ba78-4af7-8f1f-3aff529731c3 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:35.744644 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6e71483a-ba78-4af7-8f1f-3aff529731c3 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:35.780995 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-84981154-775d-4e0f-b816-4d686e4c6150 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:35.781710 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-84981154-775d-4e0f-b816-4d686e4c6150 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: c20f3279-68ae-4b04-8b87-cc16e5f73f1e] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:25:35.971191 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-84981154-775d-4e0f-b816-4d686e4c6150 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '027dcc48e4fd4c36a5004096ad5e43f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dad823e240149edb30e592380249a13', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:35.985953 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-84981154-775d-4e0f-b816-4d686e4c6150 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "GET /compute/v2.1/servers/c20f3279-68ae-4b04-8b87-cc16e5f73f1e" status: 200 len: 1209 microversion: 2.1 time: 0.251926 Jul 16 21:25:35.999607 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1821/3633] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:25:35 2022] GET /compute/v2.1/servers/c20f3279-68ae-4b04-8b87-cc16e5f73f1e => generated 1209 bytes in 264 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:25:36.014090 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-1ac5c924-a2a7-4182-95ce-8db86df2f5ad tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Calling method '.version_select at 0x7fdd9cc3fca0>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:36.070769 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-1ac5c924-a2a7-4182-95ce-8db86df2f5ad tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] 10.222.0.16 "GET /compute/v2.1/images/1f5a232b-6451-422a-9c8c-7265f8b05ddf" status: 200 len: 1426 microversion: 2.1 time: 0.067927 Jul 16 21:25:36.071919 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1822/3634] 10.222.0.16 () {58 vars in 1232 bytes} [Sat Jul 16 21:25:35 2022] GET /compute/v2.1/images/1f5a232b-6451-422a-9c8c-7265f8b05ddf => generated 1426 bytes in 72 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:25:36.087068 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-6594651e-9320-4a56-a36c-1d9fa4f459c6 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:36.089796 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-6594651e-9320-4a56-a36c-1d9fa4f459c6 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] [instance: 4eb3d8d6-79ae-4109-a5a7-835c459d1719] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:25:36.107494 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6594651e-9320-4a56-a36c-1d9fa4f459c6 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:36.108566 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6594651e-9320-4a56-a36c-1d9fa4f459c6 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:36.119210 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6594651e-9320-4a56-a36c-1d9fa4f459c6 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:36.124530 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6594651e-9320-4a56-a36c-1d9fa4f459c6 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:36.174670 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-6e71483a-ba78-4af7-8f1f-3aff529731c3 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f2e974ab5aa48dfb4b00f810dcedb63', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd31be8475a6a49d6bfe8132dd050c6c0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:36.185833 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-6e71483a-ba78-4af7-8f1f-3aff529731c3 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "GET /compute/v2.1/servers/00c8a753-5bf0-448c-8107-3946aa9bf7ae" status: 200 len: 1314 microversion: 2.1 time: 0.464503 Jul 16 21:25:36.186423 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1813/3635] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:25:35 2022] GET /compute/v2.1/servers/00c8a753-5bf0-448c-8107-3946aa9bf7ae => generated 1314 bytes in 466 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:25:36.190795 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-ad1f8f78-c7b6-4f2a-a62b-deb7636866c1 req-55af4091-acd4-492f-a493-34c54cc65db6 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "f00bea08-4180-45d6-ba01-4b104f5615d6", "name": "network-vif-unplugged", "status": "completed", "tag": "541fa980-4b14-4bc2-af0f-d4bf299e66ca"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:25:36.204988 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-ad1f8f78-c7b6-4f2a-a62b-deb7636866c1 req-55af4091-acd4-492f-a493-34c54cc65db6 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:36.206616 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-ad1f8f78-c7b6-4f2a-a62b-deb7636866c1 req-55af4091-acd4-492f-a493-34c54cc65db6 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:36.226052 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-ad1f8f78-c7b6-4f2a-a62b-deb7636866c1 req-55af4091-acd4-492f-a493-34c54cc65db6 service nova] Creating event network-vif-unplugged:541fa980-4b14-4bc2-af0f-d4bf299e66ca for instance f00bea08-4180-45d6-ba01-4b104f5615d6 on nch2-master-1 Jul 16 21:25:36.232540 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-ad1f8f78-c7b6-4f2a-a62b-deb7636866c1 req-55af4091-acd4-492f-a493-34c54cc65db6 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.044585 Jul 16 21:25:36.233307 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1814/3636] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:25:36 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:25:36.467653 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-6594651e-9320-4a56-a36c-1d9fa4f459c6 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a23fafbab18d41ee82b84bbdf94cf0fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfc2e42a0df346f4adef152ba2e3fa71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:36.482531 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-6594651e-9320-4a56-a36c-1d9fa4f459c6 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'a23fafbab18d41ee82b84bbdf94cf0fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfc2e42a0df346f4adef152ba2e3fa71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:36.484522 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-6594651e-9320-4a56-a36c-1d9fa4f459c6 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'a23fafbab18d41ee82b84bbdf94cf0fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfc2e42a0df346f4adef152ba2e3fa71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:36.486781 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-6594651e-9320-4a56-a36c-1d9fa4f459c6 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] 10.222.0.16 "GET /compute/v2.1/servers/4eb3d8d6-79ae-4109-a5a7-835c459d1719" status: 200 len: 1874 microversion: 2.32 time: 0.413445 Jul 16 21:25:36.487624 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1823/3637] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:25:36 2022] GET /compute/v2.1/servers/4eb3d8d6-79ae-4109-a5a7-835c459d1719 => generated 1874 bytes in 415 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:25:37.005684 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-de6af997-8f52-466a-bead-354585309ea6 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:37.007464 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-de6af997-8f52-466a-bead-354585309ea6 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: c20f3279-68ae-4b04-8b87-cc16e5f73f1e] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:25:37.014519 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-de6af997-8f52-466a-bead-354585309ea6 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:37.015262 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-de6af997-8f52-466a-bead-354585309ea6 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:37.016040 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-de6af997-8f52-466a-bead-354585309ea6 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:37.016659 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-de6af997-8f52-466a-bead-354585309ea6 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:37.085950 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-2f7dbf88-a841-4ecb-a1ef-f0b79468ca6a tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Calling method '.version_select at 0x7fdda2bc8b80>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:37.104319 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-de6af997-8f52-466a-bead-354585309ea6 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '027dcc48e4fd4c36a5004096ad5e43f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dad823e240149edb30e592380249a13', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:37.121352 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-de6af997-8f52-466a-bead-354585309ea6 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "GET /compute/v2.1/servers/c20f3279-68ae-4b04-8b87-cc16e5f73f1e" status: 200 len: 1213 microversion: 2.1 time: 0.119857 Jul 16 21:25:37.122766 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1815/3638] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:25:36 2022] GET /compute/v2.1/servers/c20f3279-68ae-4b04-8b87-cc16e5f73f1e => generated 1213 bytes in 122 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:25:37.152770 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-2f7dbf88-a841-4ecb-a1ef-f0b79468ca6a tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] 10.222.0.16 "GET /compute/v2.1/images/1f5a232b-6451-422a-9c8c-7265f8b05ddf" status: 200 len: 1426 microversion: 2.1 time: 0.070959 Jul 16 21:25:37.153371 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1824/3639] 10.222.0.16 () {58 vars in 1232 bytes} [Sat Jul 16 21:25:37 2022] GET /compute/v2.1/images/1f5a232b-6451-422a-9c8c-7265f8b05ddf => generated 1426 bytes in 73 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:25:37.208939 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-12293b43-a069-46a2-a370-e1553d3125b3 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:37.209971 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-12293b43-a069-46a2-a370-e1553d3125b3 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 00c8a753-5bf0-448c-8107-3946aa9bf7ae] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:25:37.219478 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-12293b43-a069-46a2-a370-e1553d3125b3 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:37.227526 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-12293b43-a069-46a2-a370-e1553d3125b3 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:37.230286 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-12293b43-a069-46a2-a370-e1553d3125b3 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:37.230816 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-12293b43-a069-46a2-a370-e1553d3125b3 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:37.406129 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-12293b43-a069-46a2-a370-e1553d3125b3 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f2e974ab5aa48dfb4b00f810dcedb63', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd31be8475a6a49d6bfe8132dd050c6c0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:37.415754 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-12293b43-a069-46a2-a370-e1553d3125b3 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "GET /compute/v2.1/servers/00c8a753-5bf0-448c-8107-3946aa9bf7ae" status: 200 len: 1501 microversion: 2.1 time: 0.215286 Jul 16 21:25:37.416665 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1816/3640] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:25:37 2022] GET /compute/v2.1/servers/00c8a753-5bf0-448c-8107-3946aa9bf7ae => generated 1501 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:25:37.435160 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-9abc84fa-7383-468b-8d35-fcef007150a9 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Action: 'action', calling method: >, body: {"os-stop": {}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:25:37.442331 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9abc84fa-7383-468b-8d35-fcef007150a9 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:37.442785 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9abc84fa-7383-468b-8d35-fcef007150a9 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:37.465607 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-9abc84fa-7383-468b-8d35-fcef007150a9 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 00c8a753-5bf0-448c-8107-3946aa9bf7ae] Going to try to stop instance {{(pid=138680) force_stop /opt/stack/nova/nova/compute/api.py:2707}} Jul 16 21:25:37.506020 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-5dc7c24f-0440-46e0-b113-9aa0f9cd1029 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:37.507718 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-5dc7c24f-0440-46e0-b113-9aa0f9cd1029 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] [instance: 4eb3d8d6-79ae-4109-a5a7-835c459d1719] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:25:37.515508 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5dc7c24f-0440-46e0-b113-9aa0f9cd1029 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:37.521598 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5dc7c24f-0440-46e0-b113-9aa0f9cd1029 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:37.521598 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5dc7c24f-0440-46e0-b113-9aa0f9cd1029 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:37.521598 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5dc7c24f-0440-46e0-b113-9aa0f9cd1029 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:37.669244 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-9abc84fa-7383-468b-8d35-fcef007150a9 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "POST /compute/v2.1/servers/00c8a753-5bf0-448c-8107-3946aa9bf7ae/action" status: 202 len: 0 microversion: 2.1 time: 0.238727 Jul 16 21:25:37.670253 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1825/3641] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 21:25:37 2022] POST /compute/v2.1/servers/00c8a753-5bf0-448c-8107-3946aa9bf7ae/action => generated 0 bytes in 242 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 21:25:37.677017 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-fbcc172e-bf60-458f-bba8-72371f873403 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerRescueNegativeTestJSON-server-372228191", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "61f22a51-464f-433b-a3f7-919d551a0f2d"}], "security_groups": [{"name": "tempest-securitygroup--1568032478"}], "key_name": "tempest-keypair-1244213655", "user_data": "}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:25:37.723215 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-5dc7c24f-0440-46e0-b113-9aa0f9cd1029 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a23fafbab18d41ee82b84bbdf94cf0fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfc2e42a0df346f4adef152ba2e3fa71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:37.733656 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-5dc7c24f-0440-46e0-b113-9aa0f9cd1029 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'a23fafbab18d41ee82b84bbdf94cf0fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfc2e42a0df346f4adef152ba2e3fa71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:37.735279 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-5dc7c24f-0440-46e0-b113-9aa0f9cd1029 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'a23fafbab18d41ee82b84bbdf94cf0fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfc2e42a0df346f4adef152ba2e3fa71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:37.736518 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-5dc7c24f-0440-46e0-b113-9aa0f9cd1029 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] 10.222.0.16 "GET /compute/v2.1/servers/4eb3d8d6-79ae-4109-a5a7-835c459d1719" status: 200 len: 1874 microversion: 2.32 time: 0.234658 Jul 16 21:25:37.737028 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1817/3642] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:25:37 2022] GET /compute/v2.1/servers/4eb3d8d6-79ae-4109-a5a7-835c459d1719 => generated 1874 bytes in 236 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 16 21:25:37.744678 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-462c649b-129f-4f7a-bb44-615d1554255b tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:37.745304 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-462c649b-129f-4f7a-bb44-615d1554255b tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 00c8a753-5bf0-448c-8107-3946aa9bf7ae] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:25:37.751800 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-462c649b-129f-4f7a-bb44-615d1554255b tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:37.752255 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-462c649b-129f-4f7a-bb44-615d1554255b tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:37.753023 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-462c649b-129f-4f7a-bb44-615d1554255b tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:37.753463 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-462c649b-129f-4f7a-bb44-615d1554255b tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:37.834108 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.network.neutron [None req-fbcc172e-bf60-458f-bba8-72371f873403 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] validate_networks() for [('61f22a51-464f-433b-a3f7-919d551a0f2d', None, None, None, None, None)] {{(pid=138680) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 16 21:25:38.068924 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-462c649b-129f-4f7a-bb44-615d1554255b tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f2e974ab5aa48dfb4b00f810dcedb63', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd31be8475a6a49d6bfe8132dd050c6c0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:38.077375 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-462c649b-129f-4f7a-bb44-615d1554255b tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "GET /compute/v2.1/servers/00c8a753-5bf0-448c-8107-3946aa9bf7ae" status: 200 len: 1511 microversion: 2.1 time: 0.339121 Jul 16 21:25:38.078255 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1818/3643] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:25:37 2022] GET /compute/v2.1/servers/00c8a753-5bf0-448c-8107-3946aa9bf7ae => generated 1511 bytes in 340 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:25:38.089892 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-402e51e8-40f2-41ce-9962-edcf23ffefed tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "9262d444-5441-481e-aaac-410ded7b62bd"}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:25:38.092501 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-402e51e8-40f2-41ce-9962-edcf23ffefed tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] [instance: 2fad683a-f21f-4a9c-8323-e9380ef7f9ef] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:25:38.100862 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-402e51e8-40f2-41ce-9962-edcf23ffefed tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:38.102179 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-402e51e8-40f2-41ce-9962-edcf23ffefed tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:38.103202 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-402e51e8-40f2-41ce-9962-edcf23ffefed tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:38.103653 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-402e51e8-40f2-41ce-9962-edcf23ffefed tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:38.529498 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-fbcc172e-bf60-458f-bba8-72371f873403 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Flavor limits 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 16 21:25:38.530018 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-fbcc172e-bf60-458f-bba8-72371f873403 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Image limits 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 16 21:25:38.530892 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-fbcc172e-bf60-458f-bba8-72371f873403 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Flavor pref 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 16 21:25:38.531434 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-fbcc172e-bf60-458f-bba8-72371f873403 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Image pref 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 16 21:25:38.531882 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-fbcc172e-bf60-458f-bba8-72371f873403 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 16 21:25:38.544185 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-fbcc172e-bf60-458f-bba8-72371f873403 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:38.545675 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-fbcc172e-bf60-458f-bba8-72371f873403 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:38.545675 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-fbcc172e-bf60-458f-bba8-72371f873403 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:38.545903 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-fbcc172e-bf60-458f-bba8-72371f873403 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:38.568749 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-fbcc172e-bf60-458f-bba8-72371f873403 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Getting quotas for project da56fc78f06142aba80ca4c6e3d60475. Resources: {'ram', 'instances', 'cores'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:25:38.575567 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-fbcc172e-bf60-458f-bba8-72371f873403 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Getting quotas for user bcc9aa2a091c490599f9a9c41163e2eb and project da56fc78f06142aba80ca4c6e3d60475. Resources: {'ram', 'instances', 'cores'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:25:38.595652 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-fbcc172e-bf60-458f-bba8-72371f873403 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Going to run 1 instances... {{(pid=138680) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 16 21:25:38.598168 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-fbcc172e-bf60-458f-bba8-72371f873403 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] [instance: c2e32bd9-9d80-427b-bb6d-6274fd48173a] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138680) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 16 21:25:38.799831 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-fbcc172e-bf60-458f-bba8-72371f873403 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 409 microversion: 2.1 time: 1.128500 Jul 16 21:25:38.801948 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1826/3644] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 21:25:37 2022] POST /compute/v2.1/servers => generated 409 bytes in 1131 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 16 21:25:38.807924 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-05ec2f4b-e0ea-4f2e-b419-a53912553b62 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:38.809587 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-05ec2f4b-e0ea-4f2e-b419-a53912553b62 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: c20f3279-68ae-4b04-8b87-cc16e5f73f1e] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:25:38.820337 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-05ec2f4b-e0ea-4f2e-b419-a53912553b62 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:38.820866 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-05ec2f4b-e0ea-4f2e-b419-a53912553b62 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:38.821785 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-05ec2f4b-e0ea-4f2e-b419-a53912553b62 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:38.822265 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-05ec2f4b-e0ea-4f2e-b419-a53912553b62 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:38.946318 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-05ec2f4b-e0ea-4f2e-b419-a53912553b62 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '027dcc48e4fd4c36a5004096ad5e43f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dad823e240149edb30e592380249a13', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:38.954929 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-05ec2f4b-e0ea-4f2e-b419-a53912553b62 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "GET /compute/v2.1/servers/c20f3279-68ae-4b04-8b87-cc16e5f73f1e" status: 200 len: 1261 microversion: 2.1 time: 0.151496 Jul 16 21:25:38.961599 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1827/3645] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:25:38 2022] GET /compute/v2.1/servers/c20f3279-68ae-4b04-8b87-cc16e5f73f1e => generated 1261 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:25:38.969726 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-6e978e7a-31fc-4a7b-b65d-46bc11c25d9a tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Calling method '.version_select at 0x7fdd9cdd6dc0>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:39.038490 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-6e978e7a-31fc-4a7b-b65d-46bc11c25d9a tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] 10.222.0.16 "GET /compute/v2.1/images/1f5a232b-6451-422a-9c8c-7265f8b05ddf" status: 200 len: 1426 microversion: 2.1 time: 0.073063 Jul 16 21:25:39.040735 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1828/3646] 10.222.0.16 () {58 vars in 1232 bytes} [Sat Jul 16 21:25:38 2022] GET /compute/v2.1/images/1f5a232b-6451-422a-9c8c-7265f8b05ddf => generated 1426 bytes in 77 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:25:39.058611 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-9f64ab1c-bccb-468f-b292-cb3a9d36c55d tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:39.059278 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-9f64ab1c-bccb-468f-b292-cb3a9d36c55d tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] [instance: 4eb3d8d6-79ae-4109-a5a7-835c459d1719] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:25:39.068440 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9f64ab1c-bccb-468f-b292-cb3a9d36c55d tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:39.068440 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9f64ab1c-bccb-468f-b292-cb3a9d36c55d tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:39.076676 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9f64ab1c-bccb-468f-b292-cb3a9d36c55d tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:39.076676 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9f64ab1c-bccb-468f-b292-cb3a9d36c55d tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:39.426346 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-402e51e8-40f2-41ce-9962-edcf23ffefed tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] 10.222.0.16 "POST /compute/v2.1/servers/2fad683a-f21f-4a9c-8323-e9380ef7f9ef/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 1.346590 Jul 16 21:25:39.428027 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1819/3647] 10.222.0.16 () {60 vars in 1322 bytes} [Sat Jul 16 21:25:38 2022] POST /compute/v2.1/servers/2fad683a-f21f-4a9c-8323-e9380ef7f9ef/os-volume_attachments => generated 194 bytes in 1349 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:25:39.436390 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-039e2677-d5cd-4d53-9a8d-64a6614056b1 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:39.437398 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-039e2677-d5cd-4d53-9a8d-64a6614056b1 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] [instance: c2e32bd9-9d80-427b-bb6d-6274fd48173a] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:25:39.468335 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-9f64ab1c-bccb-468f-b292-cb3a9d36c55d tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a23fafbab18d41ee82b84bbdf94cf0fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfc2e42a0df346f4adef152ba2e3fa71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:39.484830 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-9f64ab1c-bccb-468f-b292-cb3a9d36c55d tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'a23fafbab18d41ee82b84bbdf94cf0fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfc2e42a0df346f4adef152ba2e3fa71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:39.489727 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-9f64ab1c-bccb-468f-b292-cb3a9d36c55d tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'a23fafbab18d41ee82b84bbdf94cf0fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfc2e42a0df346f4adef152ba2e3fa71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:39.491880 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-9f64ab1c-bccb-468f-b292-cb3a9d36c55d tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] 10.222.0.16 "GET /compute/v2.1/servers/4eb3d8d6-79ae-4109-a5a7-835c459d1719" status: 200 len: 1874 microversion: 2.32 time: 0.447567 Jul 16 21:25:39.495530 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1829/3648] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:25:39 2022] GET /compute/v2.1/servers/4eb3d8d6-79ae-4109-a5a7-835c459d1719 => generated 1874 bytes in 454 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:25:39.508222 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-738d85f8-679e-4aa1-99f5-331b0bfa7919 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:39.509042 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-738d85f8-679e-4aa1-99f5-331b0bfa7919 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 00c8a753-5bf0-448c-8107-3946aa9bf7ae] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:25:39.520441 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-738d85f8-679e-4aa1-99f5-331b0bfa7919 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:39.520809 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-738d85f8-679e-4aa1-99f5-331b0bfa7919 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:39.521625 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-738d85f8-679e-4aa1-99f5-331b0bfa7919 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:39.522651 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-738d85f8-679e-4aa1-99f5-331b0bfa7919 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:39.583368 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-039e2677-d5cd-4d53-9a8d-64a6614056b1 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcc9aa2a091c490599f9a9c41163e2eb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da56fc78f06142aba80ca4c6e3d60475', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:39.592559 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-039e2677-d5cd-4d53-9a8d-64a6614056b1 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] 10.222.0.16 "GET /compute/v2.1/servers/c2e32bd9-9d80-427b-bb6d-6274fd48173a" status: 200 len: 1245 microversion: 2.1 time: 0.163068 Jul 16 21:25:39.593145 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1820/3649] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:25:39 2022] GET /compute/v2.1/servers/c2e32bd9-9d80-427b-bb6d-6274fd48173a => generated 1245 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:25:39.799268 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-738d85f8-679e-4aa1-99f5-331b0bfa7919 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f2e974ab5aa48dfb4b00f810dcedb63', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd31be8475a6a49d6bfe8132dd050c6c0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:39.811180 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-738d85f8-679e-4aa1-99f5-331b0bfa7919 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "GET /compute/v2.1/servers/00c8a753-5bf0-448c-8107-3946aa9bf7ae" status: 200 len: 1511 microversion: 2.1 time: 0.313764 Jul 16 21:25:39.812573 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1830/3650] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:25:39 2022] GET /compute/v2.1/servers/00c8a753-5bf0-448c-8107-3946aa9bf7ae => generated 1511 bytes in 316 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:25:39.977428 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-18ab2f6b-a4a0-43f9-9e32-780c2e7ae207 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:39.978156 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-18ab2f6b-a4a0-43f9-9e32-780c2e7ae207 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: c20f3279-68ae-4b04-8b87-cc16e5f73f1e] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:25:39.992381 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-18ab2f6b-a4a0-43f9-9e32-780c2e7ae207 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:39.992838 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-18ab2f6b-a4a0-43f9-9e32-780c2e7ae207 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:39.993622 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-18ab2f6b-a4a0-43f9-9e32-780c2e7ae207 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:39.994063 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-18ab2f6b-a4a0-43f9-9e32-780c2e7ae207 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:40.055059 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-dcf39ece-a5b7-4bfa-83bf-28d67f9bcac0 tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Calling method '.version_select at 0x7fdd9d34b3a0>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:40.088679 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-18ab2f6b-a4a0-43f9-9e32-780c2e7ae207 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '027dcc48e4fd4c36a5004096ad5e43f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dad823e240149edb30e592380249a13', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:40.097368 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-18ab2f6b-a4a0-43f9-9e32-780c2e7ae207 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "GET /compute/v2.1/servers/c20f3279-68ae-4b04-8b87-cc16e5f73f1e" status: 200 len: 1267 microversion: 2.1 time: 0.123531 Jul 16 21:25:40.098480 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1821/3651] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:25:39 2022] GET /compute/v2.1/servers/c20f3279-68ae-4b04-8b87-cc16e5f73f1e => generated 1267 bytes in 126 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:25:40.127192 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-dcf39ece-a5b7-4bfa-83bf-28d67f9bcac0 tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] 10.222.0.16 "GET /compute/v2.1/images/1f5a232b-6451-422a-9c8c-7265f8b05ddf" status: 200 len: 1426 microversion: 2.1 time: 0.077095 Jul 16 21:25:40.128329 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1831/3652] 10.222.0.16 () {58 vars in 1232 bytes} [Sat Jul 16 21:25:40 2022] GET /compute/v2.1/images/1f5a232b-6451-422a-9c8c-7265f8b05ddf => generated 1426 bytes in 79 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:25:40.515448 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-11b994c6-095e-4f37-83c7-29fef0251c96 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:40.517300 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-11b994c6-095e-4f37-83c7-29fef0251c96 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] [instance: 4eb3d8d6-79ae-4109-a5a7-835c459d1719] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:25:40.529079 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-11b994c6-095e-4f37-83c7-29fef0251c96 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:40.532583 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-11b994c6-095e-4f37-83c7-29fef0251c96 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:40.533673 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-11b994c6-095e-4f37-83c7-29fef0251c96 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:40.534453 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-11b994c6-095e-4f37-83c7-29fef0251c96 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:40.612441 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-0463ea03-250f-4022-994a-feb0089f4fd1 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:40.613305 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-0463ea03-250f-4022-994a-feb0089f4fd1 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] [instance: c2e32bd9-9d80-427b-bb6d-6274fd48173a] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:25:40.624739 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0463ea03-250f-4022-994a-feb0089f4fd1 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:40.625501 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0463ea03-250f-4022-994a-feb0089f4fd1 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:40.626530 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0463ea03-250f-4022-994a-feb0089f4fd1 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:40.627205 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0463ea03-250f-4022-994a-feb0089f4fd1 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:40.962308 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-0463ea03-250f-4022-994a-feb0089f4fd1 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcc9aa2a091c490599f9a9c41163e2eb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da56fc78f06142aba80ca4c6e3d60475', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:40.981134 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=96,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:25:40.986837 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-0463ea03-250f-4022-994a-feb0089f4fd1 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] 10.222.0.16 "GET /compute/v2.1/servers/c2e32bd9-9d80-427b-bb6d-6274fd48173a" status: 200 len: 1249 microversion: 2.1 time: 0.379688 Jul 16 21:25:41.027640 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1832/3653] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:25:40 2022] GET /compute/v2.1/servers/c2e32bd9-9d80-427b-bb6d-6274fd48173a => generated 1249 bytes in 421 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:25:41.044026 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-6aaa6671-8e83-4430-98c5-f287d5111bd5 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:41.045636 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-6aaa6671-8e83-4430-98c5-f287d5111bd5 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 00c8a753-5bf0-448c-8107-3946aa9bf7ae] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:25:41.064957 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6aaa6671-8e83-4430-98c5-f287d5111bd5 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:41.069332 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6aaa6671-8e83-4430-98c5-f287d5111bd5 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:41.071895 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6aaa6671-8e83-4430-98c5-f287d5111bd5 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:41.071895 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6aaa6671-8e83-4430-98c5-f287d5111bd5 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:41.249468 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-11b994c6-095e-4f37-83c7-29fef0251c96 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a23fafbab18d41ee82b84bbdf94cf0fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfc2e42a0df346f4adef152ba2e3fa71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:41.263255 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-11b994c6-095e-4f37-83c7-29fef0251c96 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'a23fafbab18d41ee82b84bbdf94cf0fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfc2e42a0df346f4adef152ba2e3fa71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:41.265107 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-11b994c6-095e-4f37-83c7-29fef0251c96 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'a23fafbab18d41ee82b84bbdf94cf0fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfc2e42a0df346f4adef152ba2e3fa71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:41.266602 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-11b994c6-095e-4f37-83c7-29fef0251c96 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] 10.222.0.16 "GET /compute/v2.1/servers/4eb3d8d6-79ae-4109-a5a7-835c459d1719" status: 200 len: 1874 microversion: 2.32 time: 0.757354 Jul 16 21:25:41.268225 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1822/3654] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:25:40 2022] GET /compute/v2.1/servers/4eb3d8d6-79ae-4109-a5a7-835c459d1719 => generated 1874 bytes in 760 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:25:41.273659 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-9268bdd8-2082-4bee-9108-b46c6f8ea5f3 req-47f84ae8-e68c-4c92-b1dd-f1c8b447c23b service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "f00bea08-4180-45d6-ba01-4b104f5615d6", "name": "network-vif-plugged", "status": "completed", "tag": "541fa980-4b14-4bc2-af0f-d4bf299e66ca"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:25:41.288929 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-9268bdd8-2082-4bee-9108-b46c6f8ea5f3 req-47f84ae8-e68c-4c92-b1dd-f1c8b447c23b service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:41.289654 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-9268bdd8-2082-4bee-9108-b46c6f8ea5f3 req-47f84ae8-e68c-4c92-b1dd-f1c8b447c23b service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:41.304800 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-9268bdd8-2082-4bee-9108-b46c6f8ea5f3 req-47f84ae8-e68c-4c92-b1dd-f1c8b447c23b service nova] Creating event network-vif-plugged:541fa980-4b14-4bc2-af0f-d4bf299e66ca for instance f00bea08-4180-45d6-ba01-4b104f5615d6 on nch2-master-1 Jul 16 21:25:41.312270 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-9268bdd8-2082-4bee-9108-b46c6f8ea5f3 req-47f84ae8-e68c-4c92-b1dd-f1c8b447c23b service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.042234 Jul 16 21:25:41.313434 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1823/3655] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:25:41 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:25:41.322247 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-10c1fcfd-2036-41d9-87ef-3a416b7699fd tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:41.323585 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-10c1fcfd-2036-41d9-87ef-3a416b7699fd tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: c20f3279-68ae-4b04-8b87-cc16e5f73f1e] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:25:41.333232 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-10c1fcfd-2036-41d9-87ef-3a416b7699fd tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:41.335394 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-10c1fcfd-2036-41d9-87ef-3a416b7699fd tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:41.335394 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-10c1fcfd-2036-41d9-87ef-3a416b7699fd tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:41.336043 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-10c1fcfd-2036-41d9-87ef-3a416b7699fd tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:41.373085 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:25:42.094864 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-10c1fcfd-2036-41d9-87ef-3a416b7699fd tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '027dcc48e4fd4c36a5004096ad5e43f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dad823e240149edb30e592380249a13', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:42.118967 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-6aaa6671-8e83-4430-98c5-f287d5111bd5 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f2e974ab5aa48dfb4b00f810dcedb63', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd31be8475a6a49d6bfe8132dd050c6c0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:42.130852 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-10c1fcfd-2036-41d9-87ef-3a416b7699fd tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "GET /compute/v2.1/servers/c20f3279-68ae-4b04-8b87-cc16e5f73f1e" status: 200 len: 1309 microversion: 2.1 time: 0.815006 Jul 16 21:25:42.132456 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1824/3656] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:25:41 2022] GET /compute/v2.1/servers/c20f3279-68ae-4b04-8b87-cc16e5f73f1e => generated 1309 bytes in 818 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:25:42.134876 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-6aaa6671-8e83-4430-98c5-f287d5111bd5 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "GET /compute/v2.1/servers/00c8a753-5bf0-448c-8107-3946aa9bf7ae" status: 200 len: 1511 microversion: 2.1 time: 1.101521 Jul 16 21:25:42.136054 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1833/3657] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:25:41 2022] GET /compute/v2.1/servers/00c8a753-5bf0-448c-8107-3946aa9bf7ae => generated 1511 bytes in 1108 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:25:42.143147 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-3138bd95-593e-4554-8508-099f62d7f3fa tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:42.151249 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-3138bd95-593e-4554-8508-099f62d7f3fa tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] [instance: c2e32bd9-9d80-427b-bb6d-6274fd48173a] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:25:42.151604 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-8962280d-b7bb-48a9-9986-025d6743b431 tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Calling method '.version_select at 0x7fdda2a18790>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:42.155325 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3138bd95-593e-4554-8508-099f62d7f3fa tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:42.156066 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3138bd95-593e-4554-8508-099f62d7f3fa tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:42.158729 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3138bd95-593e-4554-8508-099f62d7f3fa tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:42.159956 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3138bd95-593e-4554-8508-099f62d7f3fa tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:42.236013 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-8962280d-b7bb-48a9-9986-025d6743b431 tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] 10.222.0.16 "GET /compute/v2.1/images/1f5a232b-6451-422a-9c8c-7265f8b05ddf" status: 200 len: 1426 microversion: 2.1 time: 0.100890 Jul 16 21:25:42.236601 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1825/3658] 10.222.0.16 () {58 vars in 1232 bytes} [Sat Jul 16 21:25:42 2022] GET /compute/v2.1/images/1f5a232b-6451-422a-9c8c-7265f8b05ddf => generated 1426 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:25:42.276589 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-3138bd95-593e-4554-8508-099f62d7f3fa tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcc9aa2a091c490599f9a9c41163e2eb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da56fc78f06142aba80ca4c6e3d60475', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:42.300395 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-7279c685-2f21-40b0-b505-6c3e563072e8 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:42.304150 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-7279c685-2f21-40b0-b505-6c3e563072e8 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] [instance: 4eb3d8d6-79ae-4109-a5a7-835c459d1719] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:25:42.310837 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-3138bd95-593e-4554-8508-099f62d7f3fa tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] 10.222.0.16 "GET /compute/v2.1/servers/c2e32bd9-9d80-427b-bb6d-6274fd48173a" status: 200 len: 1297 microversion: 2.1 time: 0.173485 Jul 16 21:25:42.315375 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1834/3659] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:25:42 2022] GET /compute/v2.1/servers/c2e32bd9-9d80-427b-bb6d-6274fd48173a => generated 1297 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:25:42.325503 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7279c685-2f21-40b0-b505-6c3e563072e8 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:42.325503 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7279c685-2f21-40b0-b505-6c3e563072e8 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:42.325719 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7279c685-2f21-40b0-b505-6c3e563072e8 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:42.326043 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7279c685-2f21-40b0-b505-6c3e563072e8 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:42.868452 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-7279c685-2f21-40b0-b505-6c3e563072e8 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a23fafbab18d41ee82b84bbdf94cf0fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfc2e42a0df346f4adef152ba2e3fa71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:42.879706 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-7279c685-2f21-40b0-b505-6c3e563072e8 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'a23fafbab18d41ee82b84bbdf94cf0fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfc2e42a0df346f4adef152ba2e3fa71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:42.882483 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-7279c685-2f21-40b0-b505-6c3e563072e8 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'a23fafbab18d41ee82b84bbdf94cf0fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfc2e42a0df346f4adef152ba2e3fa71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:42.884170 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-7279c685-2f21-40b0-b505-6c3e563072e8 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] 10.222.0.16 "GET /compute/v2.1/servers/4eb3d8d6-79ae-4109-a5a7-835c459d1719" status: 200 len: 1874 microversion: 2.32 time: 0.590623 Jul 16 21:25:42.885010 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1826/3660] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:25:42 2022] GET /compute/v2.1/servers/4eb3d8d6-79ae-4109-a5a7-835c459d1719 => generated 1874 bytes in 592 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:25:43.149662 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-ff3ceb3c-ad67-4090-a296-289bfa795b8d tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:43.150488 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-ff3ceb3c-ad67-4090-a296-289bfa795b8d tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: c20f3279-68ae-4b04-8b87-cc16e5f73f1e] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:25:43.166595 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-1baeb894-807e-4d4f-a7b5-feda86f127fc tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:43.167376 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-1baeb894-807e-4d4f-a7b5-feda86f127fc tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 00c8a753-5bf0-448c-8107-3946aa9bf7ae] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:25:43.180217 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ff3ceb3c-ad67-4090-a296-289bfa795b8d tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:43.180667 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ff3ceb3c-ad67-4090-a296-289bfa795b8d tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:43.185748 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1baeb894-807e-4d4f-a7b5-feda86f127fc tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:43.188603 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1baeb894-807e-4d4f-a7b5-feda86f127fc tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:43.189442 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1baeb894-807e-4d4f-a7b5-feda86f127fc tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:43.189894 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1baeb894-807e-4d4f-a7b5-feda86f127fc tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:43.194282 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ff3ceb3c-ad67-4090-a296-289bfa795b8d tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:43.194282 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ff3ceb3c-ad67-4090-a296-289bfa795b8d tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:43.692985 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-1baeb894-807e-4d4f-a7b5-feda86f127fc tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f2e974ab5aa48dfb4b00f810dcedb63', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd31be8475a6a49d6bfe8132dd050c6c0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:43.702576 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-1baeb894-807e-4d4f-a7b5-feda86f127fc tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "GET /compute/v2.1/servers/00c8a753-5bf0-448c-8107-3946aa9bf7ae" status: 200 len: 1511 microversion: 2.1 time: 0.540673 Jul 16 21:25:43.715691 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1827/3661] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:25:43 2022] GET /compute/v2.1/servers/00c8a753-5bf0-448c-8107-3946aa9bf7ae => generated 1511 bytes in 554 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:25:43.721078 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-aa2a6f6a-8e7c-4b84-9cfb-951def03b2ec tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Calling method '.version_select at 0x7fdd9c2e1ee0>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:43.752785 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-ff3ceb3c-ad67-4090-a296-289bfa795b8d tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '027dcc48e4fd4c36a5004096ad5e43f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dad823e240149edb30e592380249a13', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:43.764225 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-ff3ceb3c-ad67-4090-a296-289bfa795b8d tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "GET /compute/v2.1/servers/c20f3279-68ae-4b04-8b87-cc16e5f73f1e" status: 200 len: 1309 microversion: 2.1 time: 0.617307 Jul 16 21:25:43.766000 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1835/3662] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:25:43 2022] GET /compute/v2.1/servers/c20f3279-68ae-4b04-8b87-cc16e5f73f1e => generated 1309 bytes in 621 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:25:43.774039 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-deb54b62-04a8-4ee2-86be-42626c52583c tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:43.775550 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-deb54b62-04a8-4ee2-86be-42626c52583c tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] [instance: c2e32bd9-9d80-427b-bb6d-6274fd48173a] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:25:43.785971 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-deb54b62-04a8-4ee2-86be-42626c52583c tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:43.786825 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-deb54b62-04a8-4ee2-86be-42626c52583c tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:43.788716 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-deb54b62-04a8-4ee2-86be-42626c52583c tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:43.789351 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-deb54b62-04a8-4ee2-86be-42626c52583c tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:43.815975 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-aa2a6f6a-8e7c-4b84-9cfb-951def03b2ec tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] 10.222.0.16 "GET /compute/v2.1/images/1f5a232b-6451-422a-9c8c-7265f8b05ddf" status: 200 len: 1426 microversion: 2.1 time: 0.098781 Jul 16 21:25:43.817535 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1828/3663] 10.222.0.16 () {58 vars in 1232 bytes} [Sat Jul 16 21:25:43 2022] GET /compute/v2.1/images/1f5a232b-6451-422a-9c8c-7265f8b05ddf => generated 1426 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:25:43.824510 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-e11e3b7d-c4e1-474d-a401-db1db71903f9 req-2dedc4ef-0451-4047-9cf0-2ba4c91e52c1 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "c20f3279-68ae-4b04-8b87-cc16e5f73f1e", "tag": "64f8ba51-ed8e-4f82-b26d-ac00549f8cdf"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:25:43.837848 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-e11e3b7d-c4e1-474d-a401-db1db71903f9 req-2dedc4ef-0451-4047-9cf0-2ba4c91e52c1 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:43.838634 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-e11e3b7d-c4e1-474d-a401-db1db71903f9 req-2dedc4ef-0451-4047-9cf0-2ba4c91e52c1 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:43.854580 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-e11e3b7d-c4e1-474d-a401-db1db71903f9 req-2dedc4ef-0451-4047-9cf0-2ba4c91e52c1 service nova] Creating event network-changed:64f8ba51-ed8e-4f82-b26d-ac00549f8cdf for instance c20f3279-68ae-4b04-8b87-cc16e5f73f1e on nch1-master-1 Jul 16 21:25:43.864101 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-e11e3b7d-c4e1-474d-a401-db1db71903f9 req-2dedc4ef-0451-4047-9cf0-2ba4c91e52c1 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.042118 Jul 16 21:25:43.866530 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1829/3664] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:25:43 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:25:43.913703 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-3c5842d5-26c6-4f18-a5c7-5fbc1274e99f tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:43.926080 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-3c5842d5-26c6-4f18-a5c7-5fbc1274e99f tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] [instance: 4eb3d8d6-79ae-4109-a5a7-835c459d1719] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:25:43.947820 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3c5842d5-26c6-4f18-a5c7-5fbc1274e99f tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:43.950207 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3c5842d5-26c6-4f18-a5c7-5fbc1274e99f tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:43.953639 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3c5842d5-26c6-4f18-a5c7-5fbc1274e99f tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:43.956030 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3c5842d5-26c6-4f18-a5c7-5fbc1274e99f tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:43.970617 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-deb54b62-04a8-4ee2-86be-42626c52583c tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcc9aa2a091c490599f9a9c41163e2eb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da56fc78f06142aba80ca4c6e3d60475', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:43.987717 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-deb54b62-04a8-4ee2-86be-42626c52583c tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] 10.222.0.16 "GET /compute/v2.1/servers/c2e32bd9-9d80-427b-bb6d-6274fd48173a" status: 200 len: 1303 microversion: 2.1 time: 0.220376 Jul 16 21:25:43.997985 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1836/3665] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:25:43 2022] GET /compute/v2.1/servers/c2e32bd9-9d80-427b-bb6d-6274fd48173a => generated 1303 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:25:44.518326 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-3c5842d5-26c6-4f18-a5c7-5fbc1274e99f tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a23fafbab18d41ee82b84bbdf94cf0fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfc2e42a0df346f4adef152ba2e3fa71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:44.537294 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-3c5842d5-26c6-4f18-a5c7-5fbc1274e99f tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'a23fafbab18d41ee82b84bbdf94cf0fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfc2e42a0df346f4adef152ba2e3fa71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:44.545523 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-3c5842d5-26c6-4f18-a5c7-5fbc1274e99f tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'a23fafbab18d41ee82b84bbdf94cf0fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfc2e42a0df346f4adef152ba2e3fa71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:44.546677 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-3c5842d5-26c6-4f18-a5c7-5fbc1274e99f tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] 10.222.0.16 "GET /compute/v2.1/servers/4eb3d8d6-79ae-4109-a5a7-835c459d1719" status: 200 len: 1874 microversion: 2.32 time: 0.645314 Jul 16 21:25:44.549105 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1830/3666] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:25:43 2022] GET /compute/v2.1/servers/4eb3d8d6-79ae-4109-a5a7-835c459d1719 => generated 1874 bytes in 648 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:25:44.721478 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-c845ea25-b97f-47ba-8e7a-2ff6b5beaf91 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:44.723227 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-c845ea25-b97f-47ba-8e7a-2ff6b5beaf91 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 00c8a753-5bf0-448c-8107-3946aa9bf7ae] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:25:44.741207 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c845ea25-b97f-47ba-8e7a-2ff6b5beaf91 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:44.747961 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c845ea25-b97f-47ba-8e7a-2ff6b5beaf91 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:44.751148 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c845ea25-b97f-47ba-8e7a-2ff6b5beaf91 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:44.753089 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c845ea25-b97f-47ba-8e7a-2ff6b5beaf91 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:44.784288 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-78f7eb7e-42ef-42ed-a721-998021eaf0bd tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:44.788659 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-78f7eb7e-42ef-42ed-a721-998021eaf0bd tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: c20f3279-68ae-4b04-8b87-cc16e5f73f1e] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:25:44.800801 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-78f7eb7e-42ef-42ed-a721-998021eaf0bd tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:44.802789 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-78f7eb7e-42ef-42ed-a721-998021eaf0bd tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:44.805146 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-78f7eb7e-42ef-42ed-a721-998021eaf0bd tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:44.806393 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-78f7eb7e-42ef-42ed-a721-998021eaf0bd tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:45.314907 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-78f7eb7e-42ef-42ed-a721-998021eaf0bd tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '027dcc48e4fd4c36a5004096ad5e43f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dad823e240149edb30e592380249a13', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:45.333157 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-c845ea25-b97f-47ba-8e7a-2ff6b5beaf91 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f2e974ab5aa48dfb4b00f810dcedb63', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd31be8475a6a49d6bfe8132dd050c6c0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:45.338833 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-78f7eb7e-42ef-42ed-a721-998021eaf0bd tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "GET /compute/v2.1/servers/c20f3279-68ae-4b04-8b87-cc16e5f73f1e" status: 200 len: 1309 microversion: 2.1 time: 0.559797 Jul 16 21:25:45.339805 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1831/3667] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:25:44 2022] GET /compute/v2.1/servers/c20f3279-68ae-4b04-8b87-cc16e5f73f1e => generated 1309 bytes in 562 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:25:45.346386 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-17e8019c-b090-45b7-89e3-8f97301b88fb tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:45.347729 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-17e8019c-b090-45b7-89e3-8f97301b88fb tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] [instance: 2fad683a-f21f-4a9c-8323-e9380ef7f9ef] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:25:45.356945 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-17e8019c-b090-45b7-89e3-8f97301b88fb tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:45.357840 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-17e8019c-b090-45b7-89e3-8f97301b88fb tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:45.359071 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-17e8019c-b090-45b7-89e3-8f97301b88fb tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:45.359768 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-17e8019c-b090-45b7-89e3-8f97301b88fb tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:45.364708 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-c845ea25-b97f-47ba-8e7a-2ff6b5beaf91 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "GET /compute/v2.1/servers/00c8a753-5bf0-448c-8107-3946aa9bf7ae" status: 200 len: 1511 microversion: 2.1 time: 0.640504 Jul 16 21:25:45.365678 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1837/3668] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:25:44 2022] GET /compute/v2.1/servers/00c8a753-5bf0-448c-8107-3946aa9bf7ae => generated 1511 bytes in 650 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:25:45.393618 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-e92a686d-5574-4a24-9041-9af48bb204fa tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Calling method '.version_select at 0x7fdd9cdf5f70>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:45.412032 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell0:SELECT=2 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:25:45.440633 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-17e8019c-b090-45b7-89e3-8f97301b88fb tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] 10.222.0.16 "GET /compute/v2.1/servers/2fad683a-f21f-4a9c-8323-e9380ef7f9ef/os-volume_attachments" status: 200 len: 371 microversion: 2.1 time: 0.078409 Jul 16 21:25:45.441582 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1832/3669] 10.222.0.16 () {58 vars in 1301 bytes} [Sat Jul 16 21:25:45 2022] GET /compute/v2.1/servers/2fad683a-f21f-4a9c-8323-e9380ef7f9ef/os-volume_attachments => generated 371 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:25:45.452312 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-988e75e6-54de-4312-8b8e-9e2cf9ccbde4 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:45.457994 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-988e75e6-54de-4312-8b8e-9e2cf9ccbde4 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] [instance: c2e32bd9-9d80-427b-bb6d-6274fd48173a] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:25:45.464892 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-988e75e6-54de-4312-8b8e-9e2cf9ccbde4 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:45.465561 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-988e75e6-54de-4312-8b8e-9e2cf9ccbde4 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:45.466635 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-988e75e6-54de-4312-8b8e-9e2cf9ccbde4 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:45.467430 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-988e75e6-54de-4312-8b8e-9e2cf9ccbde4 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:45.485507 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-e92a686d-5574-4a24-9041-9af48bb204fa tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] 10.222.0.16 "GET /compute/v2.1/images/1f5a232b-6451-422a-9c8c-7265f8b05ddf" status: 200 len: 1426 microversion: 2.1 time: 0.115908 Jul 16 21:25:45.487108 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1838/3670] 10.222.0.16 () {58 vars in 1232 bytes} [Sat Jul 16 21:25:45 2022] GET /compute/v2.1/images/1f5a232b-6451-422a-9c8c-7265f8b05ddf => generated 1426 bytes in 117 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:25:45.571384 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-ad63605f-b8d5-49b6-9c5b-3216f4f8a15d tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:45.571947 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-ad63605f-b8d5-49b6-9c5b-3216f4f8a15d tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] [instance: 4eb3d8d6-79ae-4109-a5a7-835c459d1719] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:25:45.580522 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ad63605f-b8d5-49b6-9c5b-3216f4f8a15d tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:45.581030 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ad63605f-b8d5-49b6-9c5b-3216f4f8a15d tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:45.582767 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ad63605f-b8d5-49b6-9c5b-3216f4f8a15d tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:45.582767 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ad63605f-b8d5-49b6-9c5b-3216f4f8a15d tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:46.190557 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-ad63605f-b8d5-49b6-9c5b-3216f4f8a15d tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a23fafbab18d41ee82b84bbdf94cf0fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfc2e42a0df346f4adef152ba2e3fa71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:46.195082 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-988e75e6-54de-4312-8b8e-9e2cf9ccbde4 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcc9aa2a091c490599f9a9c41163e2eb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da56fc78f06142aba80ca4c6e3d60475', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:46.204229 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-ad63605f-b8d5-49b6-9c5b-3216f4f8a15d tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'a23fafbab18d41ee82b84bbdf94cf0fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfc2e42a0df346f4adef152ba2e3fa71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:46.207542 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-988e75e6-54de-4312-8b8e-9e2cf9ccbde4 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] 10.222.0.16 "GET /compute/v2.1/servers/c2e32bd9-9d80-427b-bb6d-6274fd48173a" status: 200 len: 1371 microversion: 2.1 time: 0.759846 Jul 16 21:25:46.210711 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1833/3671] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:25:45 2022] GET /compute/v2.1/servers/c2e32bd9-9d80-427b-bb6d-6274fd48173a => generated 1371 bytes in 762 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:25:46.216822 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-827bf8f8-8409-4ebb-b383-71534595dee7 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:46.217736 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-827bf8f8-8409-4ebb-b383-71534595dee7 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] [instance: 2fad683a-f21f-4a9c-8323-e9380ef7f9ef] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:25:46.227777 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-827bf8f8-8409-4ebb-b383-71534595dee7 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:46.244635 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-ad63605f-b8d5-49b6-9c5b-3216f4f8a15d tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'a23fafbab18d41ee82b84bbdf94cf0fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfc2e42a0df346f4adef152ba2e3fa71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:46.244854 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-827bf8f8-8409-4ebb-b383-71534595dee7 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:46.245053 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-ad63605f-b8d5-49b6-9c5b-3216f4f8a15d tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] 10.222.0.16 "GET /compute/v2.1/servers/4eb3d8d6-79ae-4109-a5a7-835c459d1719" status: 200 len: 1874 microversion: 2.32 time: 0.664813 Jul 16 21:25:46.245232 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-827bf8f8-8409-4ebb-b383-71534595dee7 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:46.245232 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-827bf8f8-8409-4ebb-b383-71534595dee7 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:46.245508 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1839/3672] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:25:45 2022] GET /compute/v2.1/servers/4eb3d8d6-79ae-4109-a5a7-835c459d1719 => generated 1874 bytes in 666 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:25:46.294282 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-827bf8f8-8409-4ebb-b383-71534595dee7 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] 10.222.0.16 "GET /compute/v2.1/servers/2fad683a-f21f-4a9c-8323-e9380ef7f9ef/os-volume_attachments/7b097a34-dea9-4828-bbfe-ec7905fbb026" status: 200 len: 194 microversion: 2.1 time: 0.083482 Jul 16 21:25:46.295941 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1834/3673] 10.222.0.16 () {58 vars in 1412 bytes} [Sat Jul 16 21:25:46 2022] GET /compute/v2.1/servers/2fad683a-f21f-4a9c-8323-e9380ef7f9ef/os-volume_attachments/7b097a34-dea9-4828-bbfe-ec7905fbb026 => generated 194 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:25:46.311615 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-5d724e08-6e2b-4101-91b2-3d3dea7ebcd0 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:46.312135 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-5d724e08-6e2b-4101-91b2-3d3dea7ebcd0 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] [instance: 2fad683a-f21f-4a9c-8323-e9380ef7f9ef] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:25:46.321150 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5d724e08-6e2b-4101-91b2-3d3dea7ebcd0 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:46.321618 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5d724e08-6e2b-4101-91b2-3d3dea7ebcd0 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:46.322431 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5d724e08-6e2b-4101-91b2-3d3dea7ebcd0 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:46.322873 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5d724e08-6e2b-4101-91b2-3d3dea7ebcd0 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:46.357692 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-c4aeb774-3a89-48fa-9767-6a1ba4efaf25 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:46.358314 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-c4aeb774-3a89-48fa-9767-6a1ba4efaf25 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: c20f3279-68ae-4b04-8b87-cc16e5f73f1e] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:25:46.410623 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c4aeb774-3a89-48fa-9767-6a1ba4efaf25 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:46.411113 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c4aeb774-3a89-48fa-9767-6a1ba4efaf25 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:46.411915 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c4aeb774-3a89-48fa-9767-6a1ba4efaf25 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:46.412400 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c4aeb774-3a89-48fa-9767-6a1ba4efaf25 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:46.463497 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.objects.instance [None req-5d724e08-6e2b-4101-91b2-3d3dea7ebcd0 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lazy-loading 'services' on Instance uuid 2fad683a-f21f-4a9c-8323-e9380ef7f9ef {{(pid=138680) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:25:46.739747 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-5d724e08-6e2b-4101-91b2-3d3dea7ebcd0 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] 10.222.0.16 "DELETE /compute/v2.1/servers/2fad683a-f21f-4a9c-8323-e9380ef7f9ef/os-volume_attachments/7b097a34-dea9-4828-bbfe-ec7905fbb026" status: 202 len: 0 microversion: 2.1 time: 0.437428 Jul 16 21:25:46.745429 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1840/3674] 10.222.0.16 () {58 vars in 1415 bytes} [Sat Jul 16 21:25:46 2022] DELETE /compute/v2.1/servers/2fad683a-f21f-4a9c-8323-e9380ef7f9ef/os-volume_attachments/7b097a34-dea9-4828-bbfe-ec7905fbb026 => generated 0 bytes in 444 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 21:25:46.757946 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-947a9488-b2ad-4dcf-a328-94f458c0c175 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:46.760638 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-947a9488-b2ad-4dcf-a328-94f458c0c175 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 00c8a753-5bf0-448c-8107-3946aa9bf7ae] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:25:46.776929 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-947a9488-b2ad-4dcf-a328-94f458c0c175 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:46.777453 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-947a9488-b2ad-4dcf-a328-94f458c0c175 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:46.779732 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-947a9488-b2ad-4dcf-a328-94f458c0c175 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:46.780247 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-947a9488-b2ad-4dcf-a328-94f458c0c175 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:46.837287 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-c4aeb774-3a89-48fa-9767-6a1ba4efaf25 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '027dcc48e4fd4c36a5004096ad5e43f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dad823e240149edb30e592380249a13', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:46.852854 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-c4aeb774-3a89-48fa-9767-6a1ba4efaf25 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "GET /compute/v2.1/servers/c20f3279-68ae-4b04-8b87-cc16e5f73f1e" status: 200 len: 1309 microversion: 2.1 time: 0.499787 Jul 16 21:25:46.855305 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1835/3675] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:25:46 2022] GET /compute/v2.1/servers/c20f3279-68ae-4b04-8b87-cc16e5f73f1e => generated 1309 bytes in 503 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:25:46.874859 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-db614bbd-0390-43f8-95fc-634e10eb0f42 tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Calling method '.version_select at 0x7fdd9c57d790>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:46.961014 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-db614bbd-0390-43f8-95fc-634e10eb0f42 tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] 10.222.0.16 "GET /compute/v2.1/images/1f5a232b-6451-422a-9c8c-7265f8b05ddf" status: 200 len: 1426 microversion: 2.1 time: 0.104617 Jul 16 21:25:46.970072 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1836/3676] 10.222.0.16 () {58 vars in 1232 bytes} [Sat Jul 16 21:25:46 2022] GET /compute/v2.1/images/1f5a232b-6451-422a-9c8c-7265f8b05ddf => generated 1426 bytes in 114 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:25:47.060084 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-947a9488-b2ad-4dcf-a328-94f458c0c175 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f2e974ab5aa48dfb4b00f810dcedb63', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd31be8475a6a49d6bfe8132dd050c6c0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:47.076906 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-947a9488-b2ad-4dcf-a328-94f458c0c175 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "GET /compute/v2.1/servers/00c8a753-5bf0-448c-8107-3946aa9bf7ae" status: 200 len: 1511 microversion: 2.1 time: 0.330017 Jul 16 21:25:47.078780 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1841/3677] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:25:46 2022] GET /compute/v2.1/servers/00c8a753-5bf0-448c-8107-3946aa9bf7ae => generated 1511 bytes in 333 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:25:47.230192 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-ca7b286e-7c6b-4beb-9a94-cf19a7217a5b tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:47.230192 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-ca7b286e-7c6b-4beb-9a94-cf19a7217a5b tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] [instance: c2e32bd9-9d80-427b-bb6d-6274fd48173a] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:25:47.239467 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ca7b286e-7c6b-4beb-9a94-cf19a7217a5b tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:47.241258 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ca7b286e-7c6b-4beb-9a94-cf19a7217a5b tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:47.242535 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ca7b286e-7c6b-4beb-9a94-cf19a7217a5b tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:47.243184 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ca7b286e-7c6b-4beb-9a94-cf19a7217a5b tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:47.254412 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-da30ff3b-fe57-4094-8a76-a75ba75a66a9 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:47.256274 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-da30ff3b-fe57-4094-8a76-a75ba75a66a9 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] [instance: 4eb3d8d6-79ae-4109-a5a7-835c459d1719] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:25:47.270301 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-da30ff3b-fe57-4094-8a76-a75ba75a66a9 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:47.274034 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-da30ff3b-fe57-4094-8a76-a75ba75a66a9 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:47.279285 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-da30ff3b-fe57-4094-8a76-a75ba75a66a9 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:47.280013 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-da30ff3b-fe57-4094-8a76-a75ba75a66a9 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:47.619174 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-ca7b286e-7c6b-4beb-9a94-cf19a7217a5b tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcc9aa2a091c490599f9a9c41163e2eb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da56fc78f06142aba80ca4c6e3d60475', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:47.630246 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-ca7b286e-7c6b-4beb-9a94-cf19a7217a5b tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] 10.222.0.16 "GET /compute/v2.1/servers/c2e32bd9-9d80-427b-bb6d-6274fd48173a" status: 200 len: 1371 microversion: 2.1 time: 0.406718 Jul 16 21:25:47.634043 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1837/3678] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:25:47 2022] GET /compute/v2.1/servers/c2e32bd9-9d80-427b-bb6d-6274fd48173a => generated 1371 bytes in 412 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:25:47.672928 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-da30ff3b-fe57-4094-8a76-a75ba75a66a9 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a23fafbab18d41ee82b84bbdf94cf0fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfc2e42a0df346f4adef152ba2e3fa71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:47.692556 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-07d68f35-4f39-4d9a-9c7e-5fe5dbd399b6 req-1369559b-043c-4d49-86f7-4e47ce54ba03 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "c2e32bd9-9d80-427b-bb6d-6274fd48173a", "tag": "347e5d31-8efd-4cb0-82eb-3b8a6e86e0cd"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:25:47.702431 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-da30ff3b-fe57-4094-8a76-a75ba75a66a9 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'a23fafbab18d41ee82b84bbdf94cf0fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfc2e42a0df346f4adef152ba2e3fa71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:47.707059 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-da30ff3b-fe57-4094-8a76-a75ba75a66a9 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'a23fafbab18d41ee82b84bbdf94cf0fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfc2e42a0df346f4adef152ba2e3fa71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:47.712974 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-da30ff3b-fe57-4094-8a76-a75ba75a66a9 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] 10.222.0.16 "GET /compute/v2.1/servers/4eb3d8d6-79ae-4109-a5a7-835c459d1719" status: 200 len: 1874 microversion: 2.32 time: 0.464233 Jul 16 21:25:47.723161 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1842/3679] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:25:47 2022] GET /compute/v2.1/servers/4eb3d8d6-79ae-4109-a5a7-835c459d1719 => generated 1874 bytes in 476 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:25:47.736757 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-07d68f35-4f39-4d9a-9c7e-5fe5dbd399b6 req-1369559b-043c-4d49-86f7-4e47ce54ba03 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:47.741371 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-07d68f35-4f39-4d9a-9c7e-5fe5dbd399b6 req-1369559b-043c-4d49-86f7-4e47ce54ba03 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:47.769345 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-07d68f35-4f39-4d9a-9c7e-5fe5dbd399b6 req-1369559b-043c-4d49-86f7-4e47ce54ba03 service nova] Creating event network-changed:347e5d31-8efd-4cb0-82eb-3b8a6e86e0cd for instance c2e32bd9-9d80-427b-bb6d-6274fd48173a on nch1-master-1 Jul 16 21:25:47.796158 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-07d68f35-4f39-4d9a-9c7e-5fe5dbd399b6 req-1369559b-043c-4d49-86f7-4e47ce54ba03 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.095664 Jul 16 21:25:47.799056 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1838/3680] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:25:47 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 110 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:25:47.880990 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-584af9a0-c867-41be-ae63-581a0f46e04b tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:47.880990 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-584af9a0-c867-41be-ae63-581a0f46e04b tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: c20f3279-68ae-4b04-8b87-cc16e5f73f1e] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:25:47.891677 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-584af9a0-c867-41be-ae63-581a0f46e04b tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:47.901959 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-584af9a0-c867-41be-ae63-581a0f46e04b tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:47.903125 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-584af9a0-c867-41be-ae63-581a0f46e04b tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:47.906325 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-584af9a0-c867-41be-ae63-581a0f46e04b tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:47.986846 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-e9a4129f-7037-4827-961b-6d1b388742cf tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Calling method '.version_select at 0x7fdd9c57d700>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:48.085975 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-e9a4129f-7037-4827-961b-6d1b388742cf tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] 10.222.0.16 "GET /compute/v2.1/images/1f5a232b-6451-422a-9c8c-7265f8b05ddf" status: 200 len: 1426 microversion: 2.1 time: 0.115408 Jul 16 21:25:48.093030 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1839/3681] 10.222.0.16 () {58 vars in 1232 bytes} [Sat Jul 16 21:25:47 2022] GET /compute/v2.1/images/1f5a232b-6451-422a-9c8c-7265f8b05ddf => generated 1426 bytes in 118 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:25:48.107948 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-03d9bf2f-7dc6-4353-b81c-1b08b1382030 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:48.110497 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-03d9bf2f-7dc6-4353-b81c-1b08b1382030 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 00c8a753-5bf0-448c-8107-3946aa9bf7ae] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:25:48.121872 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-03d9bf2f-7dc6-4353-b81c-1b08b1382030 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:48.122876 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-03d9bf2f-7dc6-4353-b81c-1b08b1382030 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:48.124076 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-03d9bf2f-7dc6-4353-b81c-1b08b1382030 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:48.125697 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-03d9bf2f-7dc6-4353-b81c-1b08b1382030 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:48.386713 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-03d9bf2f-7dc6-4353-b81c-1b08b1382030 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f2e974ab5aa48dfb4b00f810dcedb63', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd31be8475a6a49d6bfe8132dd050c6c0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:48.400796 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-03d9bf2f-7dc6-4353-b81c-1b08b1382030 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "GET /compute/v2.1/servers/00c8a753-5bf0-448c-8107-3946aa9bf7ae" status: 200 len: 1511 microversion: 2.1 time: 0.300626 Jul 16 21:25:48.415629 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-584af9a0-c867-41be-ae63-581a0f46e04b tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '027dcc48e4fd4c36a5004096ad5e43f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dad823e240149edb30e592380249a13', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:48.416201 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1840/3682] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:25:48 2022] GET /compute/v2.1/servers/00c8a753-5bf0-448c-8107-3946aa9bf7ae => generated 1511 bytes in 320 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:25:48.421187 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-584af9a0-c867-41be-ae63-581a0f46e04b tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "GET /compute/v2.1/servers/c20f3279-68ae-4b04-8b87-cc16e5f73f1e" status: 200 len: 1309 microversion: 2.1 time: 0.552183 Jul 16 21:25:48.422251 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1843/3683] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:25:47 2022] GET /compute/v2.1/servers/c20f3279-68ae-4b04-8b87-cc16e5f73f1e => generated 1309 bytes in 554 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:25:48.651310 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-c3e5b53c-e6b9-41c5-9fad-887acb1667cc tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:48.652044 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-c3e5b53c-e6b9-41c5-9fad-887acb1667cc tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] [instance: c2e32bd9-9d80-427b-bb6d-6274fd48173a] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:25:48.670318 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c3e5b53c-e6b9-41c5-9fad-887acb1667cc tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:48.672120 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c3e5b53c-e6b9-41c5-9fad-887acb1667cc tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:48.673362 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c3e5b53c-e6b9-41c5-9fad-887acb1667cc tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:48.676705 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c3e5b53c-e6b9-41c5-9fad-887acb1667cc tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:48.736309 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-e7c6b292-9b2d-4fa0-a208-a3799a000ba9 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:48.736928 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-e7c6b292-9b2d-4fa0-a208-a3799a000ba9 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] [instance: 4eb3d8d6-79ae-4109-a5a7-835c459d1719] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:25:48.759967 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell0:SELECT=2 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:25:48.764425 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell0:SELECT=8 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:25:48.765127 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e7c6b292-9b2d-4fa0-a208-a3799a000ba9 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:48.765773 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e7c6b292-9b2d-4fa0-a208-a3799a000ba9 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:48.766963 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e7c6b292-9b2d-4fa0-a208-a3799a000ba9 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:48.768195 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e7c6b292-9b2d-4fa0-a208-a3799a000ba9 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:49.592776 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell0:SELECT=12 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:25:49.656205 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-c3e5b53c-e6b9-41c5-9fad-887acb1667cc tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcc9aa2a091c490599f9a9c41163e2eb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da56fc78f06142aba80ca4c6e3d60475', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:49.677565 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-c3e5b53c-e6b9-41c5-9fad-887acb1667cc tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] 10.222.0.16 "GET /compute/v2.1/servers/c2e32bd9-9d80-427b-bb6d-6274fd48173a" status: 200 len: 1371 microversion: 2.1 time: 1.027764 Jul 16 21:25:49.677565 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1841/3684] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:25:48 2022] GET /compute/v2.1/servers/c2e32bd9-9d80-427b-bb6d-6274fd48173a => generated 1371 bytes in 1032 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:25:49.684362 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-568ccb8b-b74f-4b47-80e5-df8ba445b840 tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Calling method '.version_select at 0x7fdda2a18790>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:49.772831 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-568ccb8b-b74f-4b47-80e5-df8ba445b840 tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] 10.222.0.16 "GET /compute/v2.1/images/1f5a232b-6451-422a-9c8c-7265f8b05ddf" status: 200 len: 1426 microversion: 2.1 time: 0.093282 Jul 16 21:25:49.777474 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1842/3685] 10.222.0.16 () {58 vars in 1232 bytes} [Sat Jul 16 21:25:49 2022] GET /compute/v2.1/images/1f5a232b-6451-422a-9c8c-7265f8b05ddf => generated 1426 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:25:49.788230 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-e47dfe41-220d-4d4a-bb8c-e87623a4a599 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:49.793666 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-e47dfe41-220d-4d4a-bb8c-e87623a4a599 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] [instance: 2fad683a-f21f-4a9c-8323-e9380ef7f9ef] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:25:49.811966 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e47dfe41-220d-4d4a-bb8c-e87623a4a599 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:49.813438 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e47dfe41-220d-4d4a-bb8c-e87623a4a599 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:49.816506 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e47dfe41-220d-4d4a-bb8c-e87623a4a599 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:49.820421 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e47dfe41-220d-4d4a-bb8c-e87623a4a599 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:49.909167 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-e7c6b292-9b2d-4fa0-a208-a3799a000ba9 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a23fafbab18d41ee82b84bbdf94cf0fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfc2e42a0df346f4adef152ba2e3fa71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:49.917474 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-e47dfe41-220d-4d4a-bb8c-e87623a4a599 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] 10.222.0.16 "GET /compute/v2.1/servers/2fad683a-f21f-4a9c-8323-e9380ef7f9ef/os-volume_attachments/9262d444-5441-481e-aaac-410ded7b62bd" status: 200 len: 194 microversion: 2.1 time: 0.138627 Jul 16 21:25:49.923467 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1843/3686] 10.222.0.16 () {58 vars in 1412 bytes} [Sat Jul 16 21:25:49 2022] GET /compute/v2.1/servers/2fad683a-f21f-4a9c-8323-e9380ef7f9ef/os-volume_attachments/9262d444-5441-481e-aaac-410ded7b62bd => generated 194 bytes in 144 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:25:49.934033 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-e7c6b292-9b2d-4fa0-a208-a3799a000ba9 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'a23fafbab18d41ee82b84bbdf94cf0fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfc2e42a0df346f4adef152ba2e3fa71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:49.936817 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-e7c6b292-9b2d-4fa0-a208-a3799a000ba9 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'a23fafbab18d41ee82b84bbdf94cf0fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfc2e42a0df346f4adef152ba2e3fa71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:49.939730 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-4c1b72a9-4784-49c3-8ddf-9f611b54b8e6 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:49.940843 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-4c1b72a9-4784-49c3-8ddf-9f611b54b8e6 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 00c8a753-5bf0-448c-8107-3946aa9bf7ae] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:25:49.947527 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-e7c6b292-9b2d-4fa0-a208-a3799a000ba9 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] 10.222.0.16 "GET /compute/v2.1/servers/4eb3d8d6-79ae-4109-a5a7-835c459d1719" status: 200 len: 1874 microversion: 2.32 time: 1.208872 Jul 16 21:25:49.950110 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1844/3687] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:25:48 2022] GET /compute/v2.1/servers/4eb3d8d6-79ae-4109-a5a7-835c459d1719 => generated 1874 bytes in 1217 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 16 21:25:49.950370 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4c1b72a9-4784-49c3-8ddf-9f611b54b8e6 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:49.951442 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4c1b72a9-4784-49c3-8ddf-9f611b54b8e6 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:49.952471 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4c1b72a9-4784-49c3-8ddf-9f611b54b8e6 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:49.953217 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4c1b72a9-4784-49c3-8ddf-9f611b54b8e6 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:49.987084 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-866cc173-81df-493a-8f04-9fd6cb30ac20 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:49.995808 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-866cc173-81df-493a-8f04-9fd6cb30ac20 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: c20f3279-68ae-4b04-8b87-cc16e5f73f1e] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:25:50.012395 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-866cc173-81df-493a-8f04-9fd6cb30ac20 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:50.012919 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-866cc173-81df-493a-8f04-9fd6cb30ac20 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:50.014237 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-866cc173-81df-493a-8f04-9fd6cb30ac20 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:50.014967 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-866cc173-81df-493a-8f04-9fd6cb30ac20 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:50.359339 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-866cc173-81df-493a-8f04-9fd6cb30ac20 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '027dcc48e4fd4c36a5004096ad5e43f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dad823e240149edb30e592380249a13', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:50.371436 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-4c1b72a9-4784-49c3-8ddf-9f611b54b8e6 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f2e974ab5aa48dfb4b00f810dcedb63', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd31be8475a6a49d6bfe8132dd050c6c0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:50.372615 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-866cc173-81df-493a-8f04-9fd6cb30ac20 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "GET /compute/v2.1/servers/c20f3279-68ae-4b04-8b87-cc16e5f73f1e" status: 200 len: 1309 microversion: 2.1 time: 0.405846 Jul 16 21:25:50.374061 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1845/3688] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:25:49 2022] GET /compute/v2.1/servers/c20f3279-68ae-4b04-8b87-cc16e5f73f1e => generated 1309 bytes in 425 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:25:50.379261 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-2c5c60fb-c4f2-4513-842b-39d59aa5dd06 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:50.380072 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-2c5c60fb-c4f2-4513-842b-39d59aa5dd06 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] [instance: 2fad683a-f21f-4a9c-8323-e9380ef7f9ef] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:25:50.388725 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2c5c60fb-c4f2-4513-842b-39d59aa5dd06 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:50.389625 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2c5c60fb-c4f2-4513-842b-39d59aa5dd06 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:50.390019 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-4c1b72a9-4784-49c3-8ddf-9f611b54b8e6 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "GET /compute/v2.1/servers/00c8a753-5bf0-448c-8107-3946aa9bf7ae" status: 200 len: 1511 microversion: 2.1 time: 0.465696 Jul 16 21:25:50.390612 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1844/3689] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:25:49 2022] GET /compute/v2.1/servers/00c8a753-5bf0-448c-8107-3946aa9bf7ae => generated 1511 bytes in 467 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:25:50.391182 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2c5c60fb-c4f2-4513-842b-39d59aa5dd06 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:50.391879 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2c5c60fb-c4f2-4513-842b-39d59aa5dd06 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:50.552475 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.objects.instance [None req-2c5c60fb-c4f2-4513-842b-39d59aa5dd06 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lazy-loading 'services' on Instance uuid 2fad683a-f21f-4a9c-8323-e9380ef7f9ef {{(pid=138680) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:25:50.695861 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-1af47203-9e18-421a-9f33-681b20244bf6 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:50.697270 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-1af47203-9e18-421a-9f33-681b20244bf6 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] [instance: c2e32bd9-9d80-427b-bb6d-6274fd48173a] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:25:50.707927 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1af47203-9e18-421a-9f33-681b20244bf6 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:50.708781 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1af47203-9e18-421a-9f33-681b20244bf6 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:50.711254 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1af47203-9e18-421a-9f33-681b20244bf6 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:50.712091 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1af47203-9e18-421a-9f33-681b20244bf6 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:50.808265 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-2c5c60fb-c4f2-4513-842b-39d59aa5dd06 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] 10.222.0.16 "DELETE /compute/v2.1/servers/2fad683a-f21f-4a9c-8323-e9380ef7f9ef/os-volume_attachments/9262d444-5441-481e-aaac-410ded7b62bd" status: 202 len: 0 microversion: 2.1 time: 0.432630 Jul 16 21:25:50.808936 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1846/3690] 10.222.0.16 () {58 vars in 1415 bytes} [Sat Jul 16 21:25:50 2022] DELETE /compute/v2.1/servers/2fad683a-f21f-4a9c-8323-e9380ef7f9ef/os-volume_attachments/9262d444-5441-481e-aaac-410ded7b62bd => generated 0 bytes in 434 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 21:25:50.819114 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-548d0124-b5b1-43db-80ef-85bd80f3b9e8 tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Calling method '.version_select at 0x7fdda2b028b0>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:50.881681 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-548d0124-b5b1-43db-80ef-85bd80f3b9e8 tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] 10.222.0.16 "GET /compute/v2.1/images/1f5a232b-6451-422a-9c8c-7265f8b05ddf" status: 200 len: 1426 microversion: 2.1 time: 0.066224 Jul 16 21:25:50.882299 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1847/3691] 10.222.0.16 () {58 vars in 1232 bytes} [Sat Jul 16 21:25:50 2022] GET /compute/v2.1/images/1f5a232b-6451-422a-9c8c-7265f8b05ddf => generated 1426 bytes in 69 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:25:50.967460 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-109c4cdf-970a-4a88-9073-07ff632560cb tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:50.968728 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-109c4cdf-970a-4a88-9073-07ff632560cb tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] [instance: 4eb3d8d6-79ae-4109-a5a7-835c459d1719] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:25:50.971048 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-1af47203-9e18-421a-9f33-681b20244bf6 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcc9aa2a091c490599f9a9c41163e2eb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da56fc78f06142aba80ca4c6e3d60475', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:50.978338 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-109c4cdf-970a-4a88-9073-07ff632560cb tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:50.978796 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-109c4cdf-970a-4a88-9073-07ff632560cb tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:50.979844 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-109c4cdf-970a-4a88-9073-07ff632560cb tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:50.980206 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-1af47203-9e18-421a-9f33-681b20244bf6 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] 10.222.0.16 "GET /compute/v2.1/servers/c2e32bd9-9d80-427b-bb6d-6274fd48173a" status: 200 len: 1371 microversion: 2.1 time: 0.289128 Jul 16 21:25:50.980813 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1845/3692] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:25:50 2022] GET /compute/v2.1/servers/c2e32bd9-9d80-427b-bb6d-6274fd48173a => generated 1371 bytes in 291 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:25:50.984079 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-109c4cdf-970a-4a88-9073-07ff632560cb tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:51.270357 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-109c4cdf-970a-4a88-9073-07ff632560cb tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a23fafbab18d41ee82b84bbdf94cf0fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfc2e42a0df346f4adef152ba2e3fa71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:51.358735 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-109c4cdf-970a-4a88-9073-07ff632560cb tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'a23fafbab18d41ee82b84bbdf94cf0fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfc2e42a0df346f4adef152ba2e3fa71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:51.358735 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-109c4cdf-970a-4a88-9073-07ff632560cb tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'a23fafbab18d41ee82b84bbdf94cf0fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfc2e42a0df346f4adef152ba2e3fa71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:51.358735 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-109c4cdf-970a-4a88-9073-07ff632560cb tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] 10.222.0.16 "GET /compute/v2.1/servers/4eb3d8d6-79ae-4109-a5a7-835c459d1719" status: 200 len: 1874 microversion: 2.32 time: 0.322482 Jul 16 21:25:51.358735 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1848/3693] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:25:50 2022] GET /compute/v2.1/servers/4eb3d8d6-79ae-4109-a5a7-835c459d1719 => generated 1874 bytes in 324 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 16 21:25:51.389741 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-dfc3cfe1-86d7-4286-9932-efc6d25d3db1 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:51.390475 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-dfc3cfe1-86d7-4286-9932-efc6d25d3db1 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: c20f3279-68ae-4b04-8b87-cc16e5f73f1e] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:25:51.397908 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-dfc3cfe1-86d7-4286-9932-efc6d25d3db1 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:51.398367 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-dfc3cfe1-86d7-4286-9932-efc6d25d3db1 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:51.399188 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-dfc3cfe1-86d7-4286-9932-efc6d25d3db1 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:51.399888 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-dfc3cfe1-86d7-4286-9932-efc6d25d3db1 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:51.409134 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-c2b054f0-0d0f-48e8-b79c-d18093c8ba5f tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:51.409756 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-c2b054f0-0d0f-48e8-b79c-d18093c8ba5f tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 00c8a753-5bf0-448c-8107-3946aa9bf7ae] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:25:51.417427 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c2b054f0-0d0f-48e8-b79c-d18093c8ba5f tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:51.417898 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c2b054f0-0d0f-48e8-b79c-d18093c8ba5f tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:51.418688 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c2b054f0-0d0f-48e8-b79c-d18093c8ba5f tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:51.419168 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c2b054f0-0d0f-48e8-b79c-d18093c8ba5f tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:51.673683 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-dfc3cfe1-86d7-4286-9932-efc6d25d3db1 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '027dcc48e4fd4c36a5004096ad5e43f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dad823e240149edb30e592380249a13', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:51.682837 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-dfc3cfe1-86d7-4286-9932-efc6d25d3db1 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "GET /compute/v2.1/servers/c20f3279-68ae-4b04-8b87-cc16e5f73f1e" status: 200 len: 1309 microversion: 2.1 time: 0.296907 Jul 16 21:25:51.687708 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1846/3694] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:25:51 2022] GET /compute/v2.1/servers/c20f3279-68ae-4b04-8b87-cc16e5f73f1e => generated 1309 bytes in 303 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:25:51.697431 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-c2b054f0-0d0f-48e8-b79c-d18093c8ba5f tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f2e974ab5aa48dfb4b00f810dcedb63', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd31be8475a6a49d6bfe8132dd050c6c0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:51.711752 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-c2b054f0-0d0f-48e8-b79c-d18093c8ba5f tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "GET /compute/v2.1/servers/00c8a753-5bf0-448c-8107-3946aa9bf7ae" status: 200 len: 1511 microversion: 2.1 time: 0.306194 Jul 16 21:25:51.712941 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1849/3695] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:25:51 2022] GET /compute/v2.1/servers/00c8a753-5bf0-448c-8107-3946aa9bf7ae => generated 1511 bytes in 309 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:25:51.894960 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-d8a22b76-d8b5-4ba0-9e38-27116495c81c tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Calling method '.version_select at 0x7fdd9c2f10d0>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:51.979794 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-d8a22b76-d8b5-4ba0-9e38-27116495c81c tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] 10.222.0.16 "GET /compute/v2.1/images/1f5a232b-6451-422a-9c8c-7265f8b05ddf" status: 200 len: 1426 microversion: 2.1 time: 0.088608 Jul 16 21:25:51.980373 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1847/3696] 10.222.0.16 () {58 vars in 1232 bytes} [Sat Jul 16 21:25:51 2022] GET /compute/v2.1/images/1f5a232b-6451-422a-9c8c-7265f8b05ddf => generated 1426 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:25:52.000468 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-6b81aa4f-817f-4ce9-b3a0-e3340f337641 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:52.001177 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-6b81aa4f-817f-4ce9-b3a0-e3340f337641 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] [instance: c2e32bd9-9d80-427b-bb6d-6274fd48173a] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:25:52.014100 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6b81aa4f-817f-4ce9-b3a0-e3340f337641 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:52.016899 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6b81aa4f-817f-4ce9-b3a0-e3340f337641 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:52.017985 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6b81aa4f-817f-4ce9-b3a0-e3340f337641 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:52.021358 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6b81aa4f-817f-4ce9-b3a0-e3340f337641 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:52.036588 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=98,nova_cell1:INSERT=2 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:25:52.233071 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-6b81aa4f-817f-4ce9-b3a0-e3340f337641 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcc9aa2a091c490599f9a9c41163e2eb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da56fc78f06142aba80ca4c6e3d60475', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:52.244011 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-6b81aa4f-817f-4ce9-b3a0-e3340f337641 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] 10.222.0.16 "GET /compute/v2.1/servers/c2e32bd9-9d80-427b-bb6d-6274fd48173a" status: 200 len: 1371 microversion: 2.1 time: 0.249789 Jul 16 21:25:52.249594 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1850/3697] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:25:51 2022] GET /compute/v2.1/servers/c2e32bd9-9d80-427b-bb6d-6274fd48173a => generated 1371 bytes in 256 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:25:52.376255 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-82bd0f1d-9790-4ef6-a104-914befef9fc8 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:52.377668 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-82bd0f1d-9790-4ef6-a104-914befef9fc8 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] [instance: 4eb3d8d6-79ae-4109-a5a7-835c459d1719] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:25:52.385660 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-82bd0f1d-9790-4ef6-a104-914befef9fc8 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:52.386197 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-82bd0f1d-9790-4ef6-a104-914befef9fc8 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:52.387065 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-82bd0f1d-9790-4ef6-a104-914befef9fc8 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:52.387739 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-82bd0f1d-9790-4ef6-a104-914befef9fc8 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:52.392810 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=100 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:25:52.635252 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-82bd0f1d-9790-4ef6-a104-914befef9fc8 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a23fafbab18d41ee82b84bbdf94cf0fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfc2e42a0df346f4adef152ba2e3fa71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:52.650050 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-82bd0f1d-9790-4ef6-a104-914befef9fc8 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'a23fafbab18d41ee82b84bbdf94cf0fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfc2e42a0df346f4adef152ba2e3fa71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:52.651611 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-82bd0f1d-9790-4ef6-a104-914befef9fc8 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'a23fafbab18d41ee82b84bbdf94cf0fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfc2e42a0df346f4adef152ba2e3fa71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:52.652801 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-82bd0f1d-9790-4ef6-a104-914befef9fc8 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] 10.222.0.16 "GET /compute/v2.1/servers/4eb3d8d6-79ae-4109-a5a7-835c459d1719" status: 200 len: 1874 microversion: 2.32 time: 0.280769 Jul 16 21:25:52.653374 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1848/3698] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:25:52 2022] GET /compute/v2.1/servers/4eb3d8d6-79ae-4109-a5a7-835c459d1719 => generated 1874 bytes in 282 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:25:52.707532 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-d4564e31-90ac-4ed1-ba54-7607fb95a556 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:52.709698 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-d4564e31-90ac-4ed1-ba54-7607fb95a556 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: c20f3279-68ae-4b04-8b87-cc16e5f73f1e] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:25:52.717116 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d4564e31-90ac-4ed1-ba54-7607fb95a556 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:52.717822 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d4564e31-90ac-4ed1-ba54-7607fb95a556 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:52.718813 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d4564e31-90ac-4ed1-ba54-7607fb95a556 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:52.719476 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d4564e31-90ac-4ed1-ba54-7607fb95a556 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:52.735180 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-2589b44e-45c4-4a78-bdf1-19812c6080d7 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:52.737410 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-2589b44e-45c4-4a78-bdf1-19812c6080d7 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 00c8a753-5bf0-448c-8107-3946aa9bf7ae] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:25:52.748231 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_api:SELECT=96,nova_api:INSERT=4 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:25:52.754151 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2589b44e-45c4-4a78-bdf1-19812c6080d7 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:52.754915 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2589b44e-45c4-4a78-bdf1-19812c6080d7 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:52.758567 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2589b44e-45c4-4a78-bdf1-19812c6080d7 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:52.763293 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2589b44e-45c4-4a78-bdf1-19812c6080d7 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:53.065407 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-d4564e31-90ac-4ed1-ba54-7607fb95a556 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '027dcc48e4fd4c36a5004096ad5e43f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dad823e240149edb30e592380249a13', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:53.077492 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-d4564e31-90ac-4ed1-ba54-7607fb95a556 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "GET /compute/v2.1/servers/c20f3279-68ae-4b04-8b87-cc16e5f73f1e" status: 200 len: 1309 microversion: 2.1 time: 0.372281 Jul 16 21:25:53.080246 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1851/3699] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:25:52 2022] GET /compute/v2.1/servers/c20f3279-68ae-4b04-8b87-cc16e5f73f1e => generated 1309 bytes in 378 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:25:53.092751 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-3826ae1c-d360-4648-addb-db4f23202887 tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Calling method '.version_select at 0x7fdda2bc5940>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:53.167643 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-2589b44e-45c4-4a78-bdf1-19812c6080d7 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f2e974ab5aa48dfb4b00f810dcedb63', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd31be8475a6a49d6bfe8132dd050c6c0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:53.171531 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-3826ae1c-d360-4648-addb-db4f23202887 tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] 10.222.0.16 "GET /compute/v2.1/images/1f5a232b-6451-422a-9c8c-7265f8b05ddf" status: 200 len: 1426 microversion: 2.1 time: 0.089300 Jul 16 21:25:53.172313 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1852/3700] 10.222.0.16 () {58 vars in 1232 bytes} [Sat Jul 16 21:25:53 2022] GET /compute/v2.1/images/1f5a232b-6451-422a-9c8c-7265f8b05ddf => generated 1426 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:25:53.182874 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-2589b44e-45c4-4a78-bdf1-19812c6080d7 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "GET /compute/v2.1/servers/00c8a753-5bf0-448c-8107-3946aa9bf7ae" status: 200 len: 1511 microversion: 2.1 time: 0.456856 Jul 16 21:25:53.184735 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1849/3701] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:25:52 2022] GET /compute/v2.1/servers/00c8a753-5bf0-448c-8107-3946aa9bf7ae => generated 1511 bytes in 460 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:25:53.248644 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-30e73a63-b920-4906-a776-ba375a410bc8 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:53.248644 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-30e73a63-b920-4906-a776-ba375a410bc8 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] [instance: 2fad683a-f21f-4a9c-8323-e9380ef7f9ef] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:25:53.267794 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-30e73a63-b920-4906-a776-ba375a410bc8 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:53.268369 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-30e73a63-b920-4906-a776-ba375a410bc8 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:53.270010 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-30e73a63-b920-4906-a776-ba375a410bc8 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:53.270522 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-30e73a63-b920-4906-a776-ba375a410bc8 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:53.273727 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-5cd82277-ea1e-4f22-9d67-3c72f9ff707d tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:53.274786 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-5cd82277-ea1e-4f22-9d67-3c72f9ff707d tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] [instance: c2e32bd9-9d80-427b-bb6d-6274fd48173a] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:25:53.284732 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5cd82277-ea1e-4f22-9d67-3c72f9ff707d tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:53.287644 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5cd82277-ea1e-4f22-9d67-3c72f9ff707d tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:53.288728 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5cd82277-ea1e-4f22-9d67-3c72f9ff707d tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:53.301855 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5cd82277-ea1e-4f22-9d67-3c72f9ff707d tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.013s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:53.316806 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-30e73a63-b920-4906-a776-ba375a410bc8 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] 10.222.0.16 "GET /compute/v2.1/servers/2fad683a-f21f-4a9c-8323-e9380ef7f9ef/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.074509 Jul 16 21:25:53.318858 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1853/3702] 10.222.0.16 () {58 vars in 1301 bytes} [Sat Jul 16 21:25:53 2022] GET /compute/v2.1/servers/2fad683a-f21f-4a9c-8323-e9380ef7f9ef/os-volume_attachments => generated 25 bytes in 77 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Jul 16 21:25:53.549942 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-70379844-03f0-4af2-a3e8-79122c9364d4 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:53.550990 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-70379844-03f0-4af2-a3e8-79122c9364d4 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] [instance: 2fad683a-f21f-4a9c-8323-e9380ef7f9ef] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:25:53.562614 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-70379844-03f0-4af2-a3e8-79122c9364d4 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:53.563335 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-70379844-03f0-4af2-a3e8-79122c9364d4 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:53.566788 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-70379844-03f0-4af2-a3e8-79122c9364d4 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:53.567564 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-70379844-03f0-4af2-a3e8-79122c9364d4 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:53.585970 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-5cd82277-ea1e-4f22-9d67-3c72f9ff707d tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcc9aa2a091c490599f9a9c41163e2eb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da56fc78f06142aba80ca4c6e3d60475', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:53.596789 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-5cd82277-ea1e-4f22-9d67-3c72f9ff707d tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] 10.222.0.16 "GET /compute/v2.1/servers/c2e32bd9-9d80-427b-bb6d-6274fd48173a" status: 200 len: 1371 microversion: 2.1 time: 0.332454 Jul 16 21:25:53.598918 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1850/3703] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:25:53 2022] GET /compute/v2.1/servers/c2e32bd9-9d80-427b-bb6d-6274fd48173a => generated 1371 bytes in 338 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:25:53.643767 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-70379844-03f0-4af2-a3e8-79122c9364d4 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] 10.222.0.16 "GET /compute/v2.1/servers/2fad683a-f21f-4a9c-8323-e9380ef7f9ef/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.098421 Jul 16 21:25:53.645384 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1854/3704] 10.222.0.16 () {58 vars in 1301 bytes} [Sat Jul 16 21:25:53 2022] GET /compute/v2.1/servers/2fad683a-f21f-4a9c-8323-e9380ef7f9ef/os-volume_attachments => generated 25 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Jul 16 21:25:53.679711 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-d83f76a8-0733-4af7-b9a6-d7fca9a83f3d tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:53.679899 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-d83f76a8-0733-4af7-b9a6-d7fca9a83f3d tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] [instance: 4eb3d8d6-79ae-4109-a5a7-835c459d1719] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:25:53.707215 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d83f76a8-0733-4af7-b9a6-d7fca9a83f3d tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:53.709551 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d83f76a8-0733-4af7-b9a6-d7fca9a83f3d tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:53.715270 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d83f76a8-0733-4af7-b9a6-d7fca9a83f3d tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:53.717077 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d83f76a8-0733-4af7-b9a6-d7fca9a83f3d tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:53.760900 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-15887a6e-7475-4cc0-80ea-254593b0e16f tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:53.761475 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-15887a6e-7475-4cc0-80ea-254593b0e16f tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] [instance: 2fad683a-f21f-4a9c-8323-e9380ef7f9ef] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:25:53.770715 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-15887a6e-7475-4cc0-80ea-254593b0e16f tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:53.771221 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-15887a6e-7475-4cc0-80ea-254593b0e16f tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:53.773407 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-15887a6e-7475-4cc0-80ea-254593b0e16f tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:53.774159 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-15887a6e-7475-4cc0-80ea-254593b0e16f tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:53.839407 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-15887a6e-7475-4cc0-80ea-254593b0e16f tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] [instance: 2fad683a-f21f-4a9c-8323-e9380ef7f9ef] Going to try to terminate instance {{(pid=138680) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 16 21:25:54.064419 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-d83f76a8-0733-4af7-b9a6-d7fca9a83f3d tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a23fafbab18d41ee82b84bbdf94cf0fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfc2e42a0df346f4adef152ba2e3fa71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:54.077384 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-d83f76a8-0733-4af7-b9a6-d7fca9a83f3d tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'a23fafbab18d41ee82b84bbdf94cf0fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfc2e42a0df346f4adef152ba2e3fa71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:54.082221 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-d83f76a8-0733-4af7-b9a6-d7fca9a83f3d tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'a23fafbab18d41ee82b84bbdf94cf0fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfc2e42a0df346f4adef152ba2e3fa71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:54.082221 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-d83f76a8-0733-4af7-b9a6-d7fca9a83f3d tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] 10.222.0.16 "GET /compute/v2.1/servers/4eb3d8d6-79ae-4109-a5a7-835c459d1719" status: 200 len: 1874 microversion: 2.32 time: 0.410485 Jul 16 21:25:54.082221 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1851/3705] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:25:53 2022] GET /compute/v2.1/servers/4eb3d8d6-79ae-4109-a5a7-835c459d1719 => generated 1874 bytes in 412 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:25:54.099760 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-39830d63-caa2-4699-aa3b-0a7a8cf5821f tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:54.100380 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-39830d63-caa2-4699-aa3b-0a7a8cf5821f tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: c20f3279-68ae-4b04-8b87-cc16e5f73f1e] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:25:54.115192 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-39830d63-caa2-4699-aa3b-0a7a8cf5821f tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:54.116271 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-39830d63-caa2-4699-aa3b-0a7a8cf5821f tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:54.117146 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-39830d63-caa2-4699-aa3b-0a7a8cf5821f tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:54.117596 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-39830d63-caa2-4699-aa3b-0a7a8cf5821f tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:54.203496 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-15887a6e-7475-4cc0-80ea-254593b0e16f tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] 10.222.0.16 "DELETE /compute/v2.1/servers/2fad683a-f21f-4a9c-8323-e9380ef7f9ef" status: 204 len: 0 microversion: 2.1 time: 0.447031 Jul 16 21:25:54.205083 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1855/3706] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:25:53 2022] DELETE /compute/v2.1/servers/2fad683a-f21f-4a9c-8323-e9380ef7f9ef => generated 0 bytes in 449 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 16 21:25:54.216360 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-e727111f-96de-46ff-81ab-00d24dce30cb tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Calling method '.version_select at 0x7fdd9d434940>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:54.306218 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-e727111f-96de-46ff-81ab-00d24dce30cb tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] 10.222.0.16 "GET /compute/v2.1/images/1f5a232b-6451-422a-9c8c-7265f8b05ddf" status: 200 len: 1426 microversion: 2.1 time: 0.099339 Jul 16 21:25:54.312785 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1856/3707] 10.222.0.16 () {58 vars in 1232 bytes} [Sat Jul 16 21:25:54 2022] GET /compute/v2.1/images/1f5a232b-6451-422a-9c8c-7265f8b05ddf => generated 1426 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:25:54.316089 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-eb5479a1-d2e2-4509-8a9e-506e4cd0a4e0 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:54.319960 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-eb5479a1-d2e2-4509-8a9e-506e4cd0a4e0 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 00c8a753-5bf0-448c-8107-3946aa9bf7ae] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:25:54.328020 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-eb5479a1-d2e2-4509-8a9e-506e4cd0a4e0 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:54.328602 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-eb5479a1-d2e2-4509-8a9e-506e4cd0a4e0 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:54.329473 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-eb5479a1-d2e2-4509-8a9e-506e4cd0a4e0 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:54.330000 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-eb5479a1-d2e2-4509-8a9e-506e4cd0a4e0 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:54.371308 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-39830d63-caa2-4699-aa3b-0a7a8cf5821f tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '027dcc48e4fd4c36a5004096ad5e43f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dad823e240149edb30e592380249a13', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:54.392859 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-39830d63-caa2-4699-aa3b-0a7a8cf5821f tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "GET /compute/v2.1/servers/c20f3279-68ae-4b04-8b87-cc16e5f73f1e" status: 200 len: 1309 microversion: 2.1 time: 0.297959 Jul 16 21:25:54.393733 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1852/3708] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:25:54 2022] GET /compute/v2.1/servers/c20f3279-68ae-4b04-8b87-cc16e5f73f1e => generated 1309 bytes in 300 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:25:54.399596 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-e4de08d1-1786-476c-9b84-6fc7484c1736 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:54.400635 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-e4de08d1-1786-476c-9b84-6fc7484c1736 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] [instance: 2fad683a-f21f-4a9c-8323-e9380ef7f9ef] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:25:54.414498 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e4de08d1-1786-476c-9b84-6fc7484c1736 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:54.415713 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e4de08d1-1786-476c-9b84-6fc7484c1736 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:54.416650 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e4de08d1-1786-476c-9b84-6fc7484c1736 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:54.421190 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e4de08d1-1786-476c-9b84-6fc7484c1736 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:54.659780 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-eb5479a1-d2e2-4509-8a9e-506e4cd0a4e0 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f2e974ab5aa48dfb4b00f810dcedb63', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd31be8475a6a49d6bfe8132dd050c6c0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:54.680401 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-eb5479a1-d2e2-4509-8a9e-506e4cd0a4e0 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "GET /compute/v2.1/servers/00c8a753-5bf0-448c-8107-3946aa9bf7ae" status: 200 len: 1511 microversion: 2.1 time: 0.372253 Jul 16 21:25:54.689620 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1857/3709] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:25:54 2022] GET /compute/v2.1/servers/00c8a753-5bf0-448c-8107-3946aa9bf7ae => generated 1511 bytes in 374 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:25:54.689620 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-e97fa6f4-b84f-4bd3-9339-26505bcfb1fc req-536f5bd8-dcd7-4766-884c-898deae939f0 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "4eb3d8d6-79ae-4109-a5a7-835c459d1719", "name": "network-vif-plugged", "status": "completed", "tag": "f55f6499-e005-4119-ad39-6c8db2fb1c53"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:25:54.703915 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-e4de08d1-1786-476c-9b84-6fc7484c1736 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bba77f6ef42547f1aa8480e27868bd4a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '22ca92a86e6c40d59071e58ce4085568', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:54.712628 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-e4de08d1-1786-476c-9b84-6fc7484c1736 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] 10.222.0.16 "GET /compute/v2.1/servers/2fad683a-f21f-4a9c-8323-e9380ef7f9ef" status: 200 len: 1672 microversion: 2.1 time: 0.317590 Jul 16 21:25:54.713212 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1853/3710] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:25:54 2022] GET /compute/v2.1/servers/2fad683a-f21f-4a9c-8323-e9380ef7f9ef => generated 1672 bytes in 319 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:25:54.717563 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-e97fa6f4-b84f-4bd3-9339-26505bcfb1fc req-536f5bd8-dcd7-4766-884c-898deae939f0 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:54.718029 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-e97fa6f4-b84f-4bd3-9339-26505bcfb1fc req-536f5bd8-dcd7-4766-884c-898deae939f0 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:54.728509 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-37863919-6541-48ae-ad5a-36ca334b42c4 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:54.729135 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-37863919-6541-48ae-ad5a-36ca334b42c4 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] [instance: c2e32bd9-9d80-427b-bb6d-6274fd48173a] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:25:54.745782 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-37863919-6541-48ae-ad5a-36ca334b42c4 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:54.746251 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-37863919-6541-48ae-ad5a-36ca334b42c4 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:54.747069 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-37863919-6541-48ae-ad5a-36ca334b42c4 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:54.747518 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-37863919-6541-48ae-ad5a-36ca334b42c4 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:54.752193 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-e97fa6f4-b84f-4bd3-9339-26505bcfb1fc req-536f5bd8-dcd7-4766-884c-898deae939f0 service nova] Creating event network-vif-plugged:f55f6499-e005-4119-ad39-6c8db2fb1c53 for instance 4eb3d8d6-79ae-4109-a5a7-835c459d1719 on nch1-master-1 Jul 16 21:25:54.763875 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-e97fa6f4-b84f-4bd3-9339-26505bcfb1fc req-536f5bd8-dcd7-4766-884c-898deae939f0 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.080299 Jul 16 21:25:54.764394 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1858/3711] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:25:54 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:25:54.942676 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-37863919-6541-48ae-ad5a-36ca334b42c4 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcc9aa2a091c490599f9a9c41163e2eb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da56fc78f06142aba80ca4c6e3d60475', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:54.952243 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-37863919-6541-48ae-ad5a-36ca334b42c4 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] 10.222.0.16 "GET /compute/v2.1/servers/c2e32bd9-9d80-427b-bb6d-6274fd48173a" status: 200 len: 1371 microversion: 2.1 time: 0.228210 Jul 16 21:25:54.967406 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1854/3712] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:25:54 2022] GET /compute/v2.1/servers/c2e32bd9-9d80-427b-bb6d-6274fd48173a => generated 1371 bytes in 244 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:25:55.106796 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-71c290ce-0f03-42ad-9f51-6ad8e4420dbe tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:55.112034 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-71c290ce-0f03-42ad-9f51-6ad8e4420dbe tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] [instance: 4eb3d8d6-79ae-4109-a5a7-835c459d1719] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:25:55.125932 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-71c290ce-0f03-42ad-9f51-6ad8e4420dbe tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:55.126664 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-71c290ce-0f03-42ad-9f51-6ad8e4420dbe tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:55.127908 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-71c290ce-0f03-42ad-9f51-6ad8e4420dbe tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:55.128528 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-71c290ce-0f03-42ad-9f51-6ad8e4420dbe tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:55.321474 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-c2b5d4f3-2631-4668-9aa4-c316942a04e2 tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Calling method '.version_select at 0x7fdd9d257700>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:55.399159 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-c2b5d4f3-2631-4668-9aa4-c316942a04e2 tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] 10.222.0.16 "GET /compute/v2.1/images/1f5a232b-6451-422a-9c8c-7265f8b05ddf" status: 200 len: 1426 microversion: 2.1 time: 0.082139 Jul 16 21:25:55.400429 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1855/3713] 10.222.0.16 () {58 vars in 1232 bytes} [Sat Jul 16 21:25:55 2022] GET /compute/v2.1/images/1f5a232b-6451-422a-9c8c-7265f8b05ddf => generated 1426 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:25:55.410929 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-78ea4e4b-d47b-4d03-a8a3-65cfd7491d2c tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:55.412359 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-78ea4e4b-d47b-4d03-a8a3-65cfd7491d2c tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: c20f3279-68ae-4b04-8b87-cc16e5f73f1e] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:25:55.420737 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-78ea4e4b-d47b-4d03-a8a3-65cfd7491d2c tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:55.421574 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-78ea4e4b-d47b-4d03-a8a3-65cfd7491d2c tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:55.423196 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-78ea4e4b-d47b-4d03-a8a3-65cfd7491d2c tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:55.423954 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-78ea4e4b-d47b-4d03-a8a3-65cfd7491d2c tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:55.432881 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-71c290ce-0f03-42ad-9f51-6ad8e4420dbe tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a23fafbab18d41ee82b84bbdf94cf0fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfc2e42a0df346f4adef152ba2e3fa71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:55.461481 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-71c290ce-0f03-42ad-9f51-6ad8e4420dbe tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'a23fafbab18d41ee82b84bbdf94cf0fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfc2e42a0df346f4adef152ba2e3fa71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:55.471895 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-71c290ce-0f03-42ad-9f51-6ad8e4420dbe tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'a23fafbab18d41ee82b84bbdf94cf0fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfc2e42a0df346f4adef152ba2e3fa71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:55.482591 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-71c290ce-0f03-42ad-9f51-6ad8e4420dbe tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] 10.222.0.16 "GET /compute/v2.1/servers/4eb3d8d6-79ae-4109-a5a7-835c459d1719" status: 200 len: 1874 microversion: 2.32 time: 0.384201 Jul 16 21:25:55.485418 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1859/3714] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:25:55 2022] GET /compute/v2.1/servers/4eb3d8d6-79ae-4109-a5a7-835c459d1719 => generated 1874 bytes in 389 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:25:55.659946 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-78ea4e4b-d47b-4d03-a8a3-65cfd7491d2c tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '027dcc48e4fd4c36a5004096ad5e43f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dad823e240149edb30e592380249a13', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:55.670009 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-78ea4e4b-d47b-4d03-a8a3-65cfd7491d2c tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "GET /compute/v2.1/servers/c20f3279-68ae-4b04-8b87-cc16e5f73f1e" status: 200 len: 1309 microversion: 2.1 time: 0.263797 Jul 16 21:25:55.670755 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1856/3715] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:25:55 2022] GET /compute/v2.1/servers/c20f3279-68ae-4b04-8b87-cc16e5f73f1e => generated 1309 bytes in 265 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:25:55.698076 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-02a6722c-7853-42ec-98c7-d4f46c9a54a3 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:55.698983 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-02a6722c-7853-42ec-98c7-d4f46c9a54a3 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 00c8a753-5bf0-448c-8107-3946aa9bf7ae] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:25:55.709340 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-02a6722c-7853-42ec-98c7-d4f46c9a54a3 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:55.710303 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-02a6722c-7853-42ec-98c7-d4f46c9a54a3 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:55.712027 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-02a6722c-7853-42ec-98c7-d4f46c9a54a3 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:55.713143 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-02a6722c-7853-42ec-98c7-d4f46c9a54a3 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:55.733699 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-9d315c94-7c1e-4e3f-ab3a-ecefe82f6fac tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:55.734538 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-9d315c94-7c1e-4e3f-ab3a-ecefe82f6fac tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] [instance: 2fad683a-f21f-4a9c-8323-e9380ef7f9ef] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:25:55.744042 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9d315c94-7c1e-4e3f-ab3a-ecefe82f6fac tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:55.749291 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9d315c94-7c1e-4e3f-ab3a-ecefe82f6fac tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:55.751985 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9d315c94-7c1e-4e3f-ab3a-ecefe82f6fac tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:55.756583 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9d315c94-7c1e-4e3f-ab3a-ecefe82f6fac tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:56.159192 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-02a6722c-7853-42ec-98c7-d4f46c9a54a3 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f2e974ab5aa48dfb4b00f810dcedb63', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd31be8475a6a49d6bfe8132dd050c6c0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:56.169355 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-02a6722c-7853-42ec-98c7-d4f46c9a54a3 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "GET /compute/v2.1/servers/00c8a753-5bf0-448c-8107-3946aa9bf7ae" status: 200 len: 1511 microversion: 2.1 time: 0.475896 Jul 16 21:25:56.170013 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1860/3716] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:25:55 2022] GET /compute/v2.1/servers/00c8a753-5bf0-448c-8107-3946aa9bf7ae => generated 1511 bytes in 477 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:25:56.174649 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-2b9c7557-7187-4a43-a6d9-c45671a981bd tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:56.175474 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-2b9c7557-7187-4a43-a6d9-c45671a981bd tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] [instance: c2e32bd9-9d80-427b-bb6d-6274fd48173a] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:25:56.188440 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2b9c7557-7187-4a43-a6d9-c45671a981bd tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:56.188905 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2b9c7557-7187-4a43-a6d9-c45671a981bd tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:56.189695 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2b9c7557-7187-4a43-a6d9-c45671a981bd tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:56.190343 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2b9c7557-7187-4a43-a6d9-c45671a981bd tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:56.227878 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-9d315c94-7c1e-4e3f-ab3a-ecefe82f6fac tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bba77f6ef42547f1aa8480e27868bd4a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '22ca92a86e6c40d59071e58ce4085568', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:56.237235 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-9d315c94-7c1e-4e3f-ab3a-ecefe82f6fac tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] 10.222.0.16 "GET /compute/v2.1/servers/2fad683a-f21f-4a9c-8323-e9380ef7f9ef" status: 200 len: 1672 microversion: 2.1 time: 0.510709 Jul 16 21:25:56.238101 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1857/3717] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:25:55 2022] GET /compute/v2.1/servers/2fad683a-f21f-4a9c-8323-e9380ef7f9ef => generated 1672 bytes in 512 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:25:56.414656 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-2429a82c-12d6-4732-b95b-cbb06432b5ce tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Calling method '.version_select at 0x7fdd9d257700>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:56.420463 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-2b9c7557-7187-4a43-a6d9-c45671a981bd tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcc9aa2a091c490599f9a9c41163e2eb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da56fc78f06142aba80ca4c6e3d60475', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:56.430837 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-2b9c7557-7187-4a43-a6d9-c45671a981bd tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] 10.222.0.16 "GET /compute/v2.1/servers/c2e32bd9-9d80-427b-bb6d-6274fd48173a" status: 200 len: 1371 microversion: 2.1 time: 0.259755 Jul 16 21:25:56.431456 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1861/3718] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:25:56 2022] GET /compute/v2.1/servers/c2e32bd9-9d80-427b-bb6d-6274fd48173a => generated 1371 bytes in 261 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:25:56.488908 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-2429a82c-12d6-4732-b95b-cbb06432b5ce tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] 10.222.0.16 "GET /compute/v2.1/images/1f5a232b-6451-422a-9c8c-7265f8b05ddf" status: 200 len: 1426 microversion: 2.1 time: 0.079303 Jul 16 21:25:56.490743 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1858/3719] 10.222.0.16 () {58 vars in 1232 bytes} [Sat Jul 16 21:25:56 2022] GET /compute/v2.1/images/1f5a232b-6451-422a-9c8c-7265f8b05ddf => generated 1426 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:25:56.504059 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-2c8b4b55-c784-4f2b-ae1f-57cad05017bb tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:56.505451 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-2c8b4b55-c784-4f2b-ae1f-57cad05017bb tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] [instance: 4eb3d8d6-79ae-4109-a5a7-835c459d1719] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:25:56.513454 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2c8b4b55-c784-4f2b-ae1f-57cad05017bb tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:56.514269 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2c8b4b55-c784-4f2b-ae1f-57cad05017bb tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:56.515371 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2c8b4b55-c784-4f2b-ae1f-57cad05017bb tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:56.516215 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2c8b4b55-c784-4f2b-ae1f-57cad05017bb tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:56.688802 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-f57248ae-dae9-43ee-8da1-249f02163ca6 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:56.689624 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-f57248ae-dae9-43ee-8da1-249f02163ca6 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: c20f3279-68ae-4b04-8b87-cc16e5f73f1e] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:25:56.700126 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f57248ae-dae9-43ee-8da1-249f02163ca6 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:56.703177 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f57248ae-dae9-43ee-8da1-249f02163ca6 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:56.704150 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f57248ae-dae9-43ee-8da1-249f02163ca6 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:56.706561 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f57248ae-dae9-43ee-8da1-249f02163ca6 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:57.160349 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-2c8b4b55-c784-4f2b-ae1f-57cad05017bb tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a23fafbab18d41ee82b84bbdf94cf0fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfc2e42a0df346f4adef152ba2e3fa71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:57.173501 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-2c8b4b55-c784-4f2b-ae1f-57cad05017bb tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'a23fafbab18d41ee82b84bbdf94cf0fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfc2e42a0df346f4adef152ba2e3fa71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:57.175296 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-2c8b4b55-c784-4f2b-ae1f-57cad05017bb tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'a23fafbab18d41ee82b84bbdf94cf0fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfc2e42a0df346f4adef152ba2e3fa71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:57.176732 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-2c8b4b55-c784-4f2b-ae1f-57cad05017bb tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] 10.222.0.16 "GET /compute/v2.1/servers/4eb3d8d6-79ae-4109-a5a7-835c459d1719" status: 200 len: 1874 microversion: 2.32 time: 0.676932 Jul 16 21:25:57.189255 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1862/3720] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:25:56 2022] GET /compute/v2.1/servers/4eb3d8d6-79ae-4109-a5a7-835c459d1719 => generated 1874 bytes in 691 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:25:57.195601 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-25539f07-adee-4bfe-ac7d-8aefefa54498 req-5f70d2a2-d048-46e9-867d-834dd7e43ae8 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "4eb3d8d6-79ae-4109-a5a7-835c459d1719", "name": "network-vif-plugged", "status": "completed", "tag": "178c5641-50ae-4f11-97c8-41f1bfa01abe"}, {"server_uuid": "4eb3d8d6-79ae-4109-a5a7-835c459d1719", "name": "network-vif-plugged", "status": "completed", "tag": "0ae97a58-1fb7-46af-ad9b-e1ebb2af5923"}, {"server_uuid": "4eb3d8d6-79ae-4109-a5a7-835c459d1719", "name": "network-vif-plugged", "status": "completed", "tag": "fb878298-9eb7-4cd5-96d9-305373704495"}, {"server_uuid": "4eb3d8d6-79ae-4109-a5a7-835c459d1719", "name": "network-vif-plugged", "status": "completed", "tag": "02eaaf37-d146-4778-a757-4ba313f73f8b"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:25:57.204950 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-25539f07-adee-4bfe-ac7d-8aefefa54498 req-5f70d2a2-d048-46e9-867d-834dd7e43ae8 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:57.205669 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-25539f07-adee-4bfe-ac7d-8aefefa54498 req-5f70d2a2-d048-46e9-867d-834dd7e43ae8 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:57.222897 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-25539f07-adee-4bfe-ac7d-8aefefa54498 req-5f70d2a2-d048-46e9-867d-834dd7e43ae8 service nova] Creating event network-vif-plugged:178c5641-50ae-4f11-97c8-41f1bfa01abe for instance 4eb3d8d6-79ae-4109-a5a7-835c459d1719 on nch1-master-1 Jul 16 21:25:57.223769 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-25539f07-adee-4bfe-ac7d-8aefefa54498 req-5f70d2a2-d048-46e9-867d-834dd7e43ae8 service nova] Creating event network-vif-plugged:0ae97a58-1fb7-46af-ad9b-e1ebb2af5923 for instance 4eb3d8d6-79ae-4109-a5a7-835c459d1719 on nch1-master-1 Jul 16 21:25:57.225514 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-25539f07-adee-4bfe-ac7d-8aefefa54498 req-5f70d2a2-d048-46e9-867d-834dd7e43ae8 service nova] Creating event network-vif-plugged:fb878298-9eb7-4cd5-96d9-305373704495 for instance 4eb3d8d6-79ae-4109-a5a7-835c459d1719 on nch1-master-1 Jul 16 21:25:57.226210 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-25539f07-adee-4bfe-ac7d-8aefefa54498 req-5f70d2a2-d048-46e9-867d-834dd7e43ae8 service nova] Creating event network-vif-plugged:02eaaf37-d146-4778-a757-4ba313f73f8b for instance 4eb3d8d6-79ae-4109-a5a7-835c459d1719 on nch1-master-1 Jul 16 21:25:57.226680 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-f57248ae-dae9-43ee-8da1-249f02163ca6 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '027dcc48e4fd4c36a5004096ad5e43f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dad823e240149edb30e592380249a13', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:57.235454 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-25539f07-adee-4bfe-ac7d-8aefefa54498 req-5f70d2a2-d048-46e9-867d-834dd7e43ae8 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 696 microversion: 2.1 time: 0.044580 Jul 16 21:25:57.236869 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1863/3721] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:25:57 2022] POST /compute/v2.1/os-server-external-events => generated 696 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:25:57.237312 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-f57248ae-dae9-43ee-8da1-249f02163ca6 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "GET /compute/v2.1/servers/c20f3279-68ae-4b04-8b87-cc16e5f73f1e" status: 200 len: 1309 microversion: 2.1 time: 0.551172 Jul 16 21:25:57.243747 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-e73f5f35-febe-4d85-b490-3613024ff89d tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:57.245808 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-e73f5f35-febe-4d85-b490-3613024ff89d tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 00c8a753-5bf0-448c-8107-3946aa9bf7ae] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:25:57.249524 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1859/3722] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:25:56 2022] GET /compute/v2.1/servers/c20f3279-68ae-4b04-8b87-cc16e5f73f1e => generated 1309 bytes in 566 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:25:57.253144 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e73f5f35-febe-4d85-b490-3613024ff89d tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:57.253597 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e73f5f35-febe-4d85-b490-3613024ff89d tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:57.254365 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e73f5f35-febe-4d85-b490-3613024ff89d tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:57.254977 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e73f5f35-febe-4d85-b490-3613024ff89d tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:57.258954 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-dc9948a4-98fa-46ab-a36e-ebe6585de30a tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:57.259700 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-dc9948a4-98fa-46ab-a36e-ebe6585de30a tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] [instance: 2fad683a-f21f-4a9c-8323-e9380ef7f9ef] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:25:57.274137 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-dc9948a4-98fa-46ab-a36e-ebe6585de30a tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:57.274706 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-dc9948a4-98fa-46ab-a36e-ebe6585de30a tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:57.275601 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-dc9948a4-98fa-46ab-a36e-ebe6585de30a tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:57.276764 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-dc9948a4-98fa-46ab-a36e-ebe6585de30a tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:57.495382 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-dc9948a4-98fa-46ab-a36e-ebe6585de30a tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bba77f6ef42547f1aa8480e27868bd4a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '22ca92a86e6c40d59071e58ce4085568', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:57.503167 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-dc9948a4-98fa-46ab-a36e-ebe6585de30a tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] 10.222.0.16 "GET /compute/v2.1/servers/2fad683a-f21f-4a9c-8323-e9380ef7f9ef" status: 200 len: 1605 microversion: 2.1 time: 0.248589 Jul 16 21:25:57.513804 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1860/3723] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:25:57 2022] GET /compute/v2.1/servers/2fad683a-f21f-4a9c-8323-e9380ef7f9ef => generated 1605 bytes in 260 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:25:57.527138 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-8c957e91-bffa-46de-8fbb-1b67d187d7c0 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:57.527138 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-8c957e91-bffa-46de-8fbb-1b67d187d7c0 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] [instance: c2e32bd9-9d80-427b-bb6d-6274fd48173a] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:25:57.538368 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8c957e91-bffa-46de-8fbb-1b67d187d7c0 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:57.538667 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8c957e91-bffa-46de-8fbb-1b67d187d7c0 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:57.540931 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8c957e91-bffa-46de-8fbb-1b67d187d7c0 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:57.541384 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8c957e91-bffa-46de-8fbb-1b67d187d7c0 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:57.575212 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-e73f5f35-febe-4d85-b490-3613024ff89d tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f2e974ab5aa48dfb4b00f810dcedb63', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd31be8475a6a49d6bfe8132dd050c6c0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:57.583050 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-e73f5f35-febe-4d85-b490-3613024ff89d tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "GET /compute/v2.1/servers/00c8a753-5bf0-448c-8107-3946aa9bf7ae" status: 200 len: 1511 microversion: 2.1 time: 0.344554 Jul 16 21:25:57.589146 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1864/3724] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:25:57 2022] GET /compute/v2.1/servers/00c8a753-5bf0-448c-8107-3946aa9bf7ae => generated 1511 bytes in 346 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:25:57.596684 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-eb79e3ab-08ca-4b9e-8487-7775b3a63e4d tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Calling method '.version_select at 0x7fdda2b393a0>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:57.658374 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-eb79e3ab-08ca-4b9e-8487-7775b3a63e4d tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] 10.222.0.16 "GET /compute/v2.1/images/1f5a232b-6451-422a-9c8c-7265f8b05ddf" status: 200 len: 1426 microversion: 2.1 time: 0.073153 Jul 16 21:25:57.659293 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1865/3725] 10.222.0.16 () {58 vars in 1232 bytes} [Sat Jul 16 21:25:57 2022] GET /compute/v2.1/images/1f5a232b-6451-422a-9c8c-7265f8b05ddf => generated 1426 bytes in 75 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:25:57.794917 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-8c957e91-bffa-46de-8fbb-1b67d187d7c0 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcc9aa2a091c490599f9a9c41163e2eb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da56fc78f06142aba80ca4c6e3d60475', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:57.803589 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-8c957e91-bffa-46de-8fbb-1b67d187d7c0 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] 10.222.0.16 "GET /compute/v2.1/servers/c2e32bd9-9d80-427b-bb6d-6274fd48173a" status: 200 len: 1371 microversion: 2.1 time: 0.286475 Jul 16 21:25:57.815999 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1861/3726] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:25:57 2022] GET /compute/v2.1/servers/c2e32bd9-9d80-427b-bb6d-6274fd48173a => generated 1371 bytes in 292 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:25:57.977813 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-c57bd0d9-cfc0-4f61-8f62-15fba6d36bef req-ed635eca-43bc-4300-8396-ef62b5f87186 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "2fad683a-f21f-4a9c-8323-e9380ef7f9ef", "name": "network-vif-deleted", "tag": "aff74b78-59ed-4a31-9d5b-3c3e6aa5d30c"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:25:57.999390 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_api:SELECT=96,nova_api:INSERT=3,nova_api:UPDATE=1 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:25:58.032852 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-c57bd0d9-cfc0-4f61-8f62-15fba6d36bef req-ed635eca-43bc-4300-8396-ef62b5f87186 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:58.036934 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-c57bd0d9-cfc0-4f61-8f62-15fba6d36bef req-ed635eca-43bc-4300-8396-ef62b5f87186 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:58.065381 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-c57bd0d9-cfc0-4f61-8f62-15fba6d36bef req-ed635eca-43bc-4300-8396-ef62b5f87186 service nova] Creating event network-vif-deleted:aff74b78-59ed-4a31-9d5b-3c3e6aa5d30c for instance 2fad683a-f21f-4a9c-8323-e9380ef7f9ef on nch1-master-1 Jul 16 21:25:58.078883 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-c57bd0d9-cfc0-4f61-8f62-15fba6d36bef req-ed635eca-43bc-4300-8396-ef62b5f87186 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.106434 Jul 16 21:25:58.079930 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1866/3727] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:25:57 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 109 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:25:58.197221 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-3236d551-4f73-4dc3-9c7e-7696af9c5fa0 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:58.197849 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-3236d551-4f73-4dc3-9c7e-7696af9c5fa0 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] [instance: 4eb3d8d6-79ae-4109-a5a7-835c459d1719] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:25:58.231439 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3236d551-4f73-4dc3-9c7e-7696af9c5fa0 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:58.233941 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3236d551-4f73-4dc3-9c7e-7696af9c5fa0 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:58.238561 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3236d551-4f73-4dc3-9c7e-7696af9c5fa0 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:58.239223 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3236d551-4f73-4dc3-9c7e-7696af9c5fa0 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:58.257894 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-9bd84a83-7d27-4aa8-a3f0-a3fb0171178f tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:58.258884 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-9bd84a83-7d27-4aa8-a3f0-a3fb0171178f tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: c20f3279-68ae-4b04-8b87-cc16e5f73f1e] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:25:58.284331 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9bd84a83-7d27-4aa8-a3f0-a3fb0171178f tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:58.287599 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9bd84a83-7d27-4aa8-a3f0-a3fb0171178f tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:58.288834 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9bd84a83-7d27-4aa8-a3f0-a3fb0171178f tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:58.292443 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9bd84a83-7d27-4aa8-a3f0-a3fb0171178f tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:58.712055 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-3236d551-4f73-4dc3-9c7e-7696af9c5fa0 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a23fafbab18d41ee82b84bbdf94cf0fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfc2e42a0df346f4adef152ba2e3fa71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:58.730293 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-3236d551-4f73-4dc3-9c7e-7696af9c5fa0 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'a23fafbab18d41ee82b84bbdf94cf0fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfc2e42a0df346f4adef152ba2e3fa71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:58.734624 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-3236d551-4f73-4dc3-9c7e-7696af9c5fa0 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'a23fafbab18d41ee82b84bbdf94cf0fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfc2e42a0df346f4adef152ba2e3fa71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:58.736497 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-3236d551-4f73-4dc3-9c7e-7696af9c5fa0 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] 10.222.0.16 "GET /compute/v2.1/servers/4eb3d8d6-79ae-4109-a5a7-835c459d1719" status: 200 len: 1874 microversion: 2.32 time: 0.543536 Jul 16 21:25:58.743821 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1862/3728] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:25:58 2022] GET /compute/v2.1/servers/4eb3d8d6-79ae-4109-a5a7-835c459d1719 => generated 1874 bytes in 545 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:25:58.750530 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-4c09bb74-de35-435e-850d-5fcec7e7278e tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:58.755996 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-4c09bb74-de35-435e-850d-5fcec7e7278e tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] [instance: 2fad683a-f21f-4a9c-8323-e9380ef7f9ef] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:25:58.773030 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4c09bb74-de35-435e-850d-5fcec7e7278e tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:58.773489 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4c09bb74-de35-435e-850d-5fcec7e7278e tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:58.774278 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4c09bb74-de35-435e-850d-5fcec7e7278e tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:58.774563 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-9bd84a83-7d27-4aa8-a3f0-a3fb0171178f tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '027dcc48e4fd4c36a5004096ad5e43f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dad823e240149edb30e592380249a13', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:58.775045 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4c09bb74-de35-435e-850d-5fcec7e7278e tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:58.789450 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-9bd84a83-7d27-4aa8-a3f0-a3fb0171178f tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "GET /compute/v2.1/servers/c20f3279-68ae-4b04-8b87-cc16e5f73f1e" status: 200 len: 1309 microversion: 2.1 time: 0.537361 Jul 16 21:25:58.792003 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1867/3729] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:25:58 2022] GET /compute/v2.1/servers/c20f3279-68ae-4b04-8b87-cc16e5f73f1e => generated 1309 bytes in 542 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:25:58.806549 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-ed342fd8-a6ae-4400-a09d-d0801df18710 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:58.807211 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-ed342fd8-a6ae-4400-a09d-d0801df18710 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 00c8a753-5bf0-448c-8107-3946aa9bf7ae] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:25:58.821919 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ed342fd8-a6ae-4400-a09d-d0801df18710 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:58.822412 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ed342fd8-a6ae-4400-a09d-d0801df18710 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:58.823249 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ed342fd8-a6ae-4400-a09d-d0801df18710 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:58.825500 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ed342fd8-a6ae-4400-a09d-d0801df18710 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:58.850816 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4c09bb74-de35-435e-850d-5fcec7e7278e tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:58.851310 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4c09bb74-de35-435e-850d-5fcec7e7278e tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:58.851792 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.objects.instance [None req-4c09bb74-de35-435e-850d-5fcec7e7278e tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lazy-loading 'fault' on Instance uuid 2fad683a-f21f-4a9c-8323-e9380ef7f9ef {{(pid=138681) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:25:58.978919 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-4c09bb74-de35-435e-850d-5fcec7e7278e tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bba77f6ef42547f1aa8480e27868bd4a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '22ca92a86e6c40d59071e58ce4085568', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:58.987362 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-4c09bb74-de35-435e-850d-5fcec7e7278e tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] 10.222.0.16 "GET /compute/v2.1/servers/2fad683a-f21f-4a9c-8323-e9380ef7f9ef" status: 200 len: 1327 microversion: 2.1 time: 0.242024 Jul 16 21:25:58.988222 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1863/3730] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:25:58 2022] GET /compute/v2.1/servers/2fad683a-f21f-4a9c-8323-e9380ef7f9ef => generated 1327 bytes in 244 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:25:58.994413 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-9f095000-63ed-4394-9858-b9c6af838462 tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Calling method '.version_select at 0x7fdd9c2d5160>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:59.066119 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-9f095000-63ed-4394-9858-b9c6af838462 tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] 10.222.0.16 "GET /compute/v2.1/images/1f5a232b-6451-422a-9c8c-7265f8b05ddf" status: 200 len: 1435 microversion: 2.1 time: 0.076328 Jul 16 21:25:59.067419 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1864/3731] 10.222.0.16 () {58 vars in 1232 bytes} [Sat Jul 16 21:25:58 2022] GET /compute/v2.1/images/1f5a232b-6451-422a-9c8c-7265f8b05ddf => generated 1435 bytes in 78 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:25:59.077624 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-50e834a0-151c-4cd9-a174-45164e5619c6 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:59.079163 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-50e834a0-151c-4cd9-a174-45164e5619c6 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] [instance: c2e32bd9-9d80-427b-bb6d-6274fd48173a] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:25:59.105439 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-50e834a0-151c-4cd9-a174-45164e5619c6 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:59.105439 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-50e834a0-151c-4cd9-a174-45164e5619c6 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:59.105439 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-50e834a0-151c-4cd9-a174-45164e5619c6 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:59.105439 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-50e834a0-151c-4cd9-a174-45164e5619c6 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:59.108803 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-ed342fd8-a6ae-4400-a09d-d0801df18710 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f2e974ab5aa48dfb4b00f810dcedb63', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd31be8475a6a49d6bfe8132dd050c6c0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:59.127122 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-ed342fd8-a6ae-4400-a09d-d0801df18710 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "GET /compute/v2.1/servers/00c8a753-5bf0-448c-8107-3946aa9bf7ae" status: 200 len: 1511 microversion: 2.1 time: 0.328949 Jul 16 21:25:59.127122 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1868/3732] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:25:58 2022] GET /compute/v2.1/servers/00c8a753-5bf0-448c-8107-3946aa9bf7ae => generated 1511 bytes in 334 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:25:59.143517 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-bc23fb83-e59d-4ebb-87c9-438c6613a88f tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Calling method '.version_select at 0x7fdda2b2b9d0>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:59.231313 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-bc23fb83-e59d-4ebb-87c9-438c6613a88f tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] 10.222.0.16 "GET /compute/v2.1/images/1f5a232b-6451-422a-9c8c-7265f8b05ddf" status: 200 len: 1435 microversion: 2.1 time: 0.102370 Jul 16 21:25:59.232886 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1869/3733] 10.222.0.16 () {58 vars in 1232 bytes} [Sat Jul 16 21:25:59 2022] GET /compute/v2.1/images/1f5a232b-6451-422a-9c8c-7265f8b05ddf => generated 1435 bytes in 106 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:25:59.250101 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-61d8c0ad-9531-447b-a556-a57c69b12d69 tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:59.251001 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-61d8c0ad-9531-447b-a556-a57c69b12d69 tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] [instance: f00bea08-4180-45d6-ba01-4b104f5615d6] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:25:59.261370 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-61d8c0ad-9531-447b-a556-a57c69b12d69 tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:59.263899 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-61d8c0ad-9531-447b-a556-a57c69b12d69 tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:59.263899 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-61d8c0ad-9531-447b-a556-a57c69b12d69 tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:59.263899 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-61d8c0ad-9531-447b-a556-a57c69b12d69 tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:59.434946 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-50e834a0-151c-4cd9-a174-45164e5619c6 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcc9aa2a091c490599f9a9c41163e2eb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da56fc78f06142aba80ca4c6e3d60475', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:59.448064 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-50e834a0-151c-4cd9-a174-45164e5619c6 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] 10.222.0.16 "GET /compute/v2.1/servers/c2e32bd9-9d80-427b-bb6d-6274fd48173a" status: 200 len: 1371 microversion: 2.1 time: 0.376699 Jul 16 21:25:59.457890 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1865/3734] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:25:59 2022] GET /compute/v2.1/servers/c2e32bd9-9d80-427b-bb6d-6274fd48173a => generated 1371 bytes in 381 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:25:59.466474 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-af629da4-9f66-4b27-ab41-d996d3cc8ebd req-3341d823-044a-4a78-9f8b-d34216bf7655 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "4eb3d8d6-79ae-4109-a5a7-835c459d1719", "name": "network-vif-plugged", "status": "completed", "tag": "3581de64-9f10-4ee1-9fbc-52628f1ddcfa"}, {"server_uuid": "4eb3d8d6-79ae-4109-a5a7-835c459d1719", "name": "network-vif-plugged", "status": "completed", "tag": "b73090b3-6ab1-4f2c-b09d-274c9f89f289"}, {"server_uuid": "c2e32bd9-9d80-427b-bb6d-6274fd48173a", "name": "network-vif-plugged", "status": "completed", "tag": "347e5d31-8efd-4cb0-82eb-3b8a6e86e0cd"}, {"server_uuid": "c20f3279-68ae-4b04-8b87-cc16e5f73f1e", "name": "network-vif-plugged", "status": "completed", "tag": "64f8ba51-ed8e-4f82-b26d-ac00549f8cdf"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:25:59.488724 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-af629da4-9f66-4b27-ab41-d996d3cc8ebd req-3341d823-044a-4a78-9f8b-d34216bf7655 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:59.489246 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-af629da4-9f66-4b27-ab41-d996d3cc8ebd req-3341d823-044a-4a78-9f8b-d34216bf7655 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:59.514120 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-af629da4-9f66-4b27-ab41-d996d3cc8ebd req-3341d823-044a-4a78-9f8b-d34216bf7655 service nova] Creating event network-vif-plugged:3581de64-9f10-4ee1-9fbc-52628f1ddcfa for instance 4eb3d8d6-79ae-4109-a5a7-835c459d1719 on nch1-master-1 Jul 16 21:25:59.515315 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-af629da4-9f66-4b27-ab41-d996d3cc8ebd req-3341d823-044a-4a78-9f8b-d34216bf7655 service nova] Creating event network-vif-plugged:b73090b3-6ab1-4f2c-b09d-274c9f89f289 for instance 4eb3d8d6-79ae-4109-a5a7-835c459d1719 on nch1-master-1 Jul 16 21:25:59.517210 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-af629da4-9f66-4b27-ab41-d996d3cc8ebd req-3341d823-044a-4a78-9f8b-d34216bf7655 service nova] Creating event network-vif-plugged:347e5d31-8efd-4cb0-82eb-3b8a6e86e0cd for instance c2e32bd9-9d80-427b-bb6d-6274fd48173a on nch1-master-1 Jul 16 21:25:59.521589 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-61d8c0ad-9531-447b-a556-a57c69b12d69 tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3b0b46c0c1f74adaaaf65d46db7b2618', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '336b3ef8a8744bf3b86854b32000bda6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:25:59.536564 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-af629da4-9f66-4b27-ab41-d996d3cc8ebd req-3341d823-044a-4a78-9f8b-d34216bf7655 service nova] Creating event network-vif-plugged:64f8ba51-ed8e-4f82-b26d-ac00549f8cdf for instance c20f3279-68ae-4b04-8b87-cc16e5f73f1e on nch1-master-1 Jul 16 21:25:59.540829 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-61d8c0ad-9531-447b-a556-a57c69b12d69 tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] 10.222.0.16 "GET /compute/v2.1/servers/f00bea08-4180-45d6-ba01-4b104f5615d6" status: 200 len: 1466 microversion: 2.1 time: 0.297042 Jul 16 21:25:59.545036 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1870/3735] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:25:59 2022] GET /compute/v2.1/servers/f00bea08-4180-45d6-ba01-4b104f5615d6 => generated 1466 bytes in 298 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:25:59.550215 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-af629da4-9f66-4b27-ab41-d996d3cc8ebd req-3341d823-044a-4a78-9f8b-d34216bf7655 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 696 microversion: 2.1 time: 0.100309 Jul 16 21:25:59.551551 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1866/3736] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:25:59 2022] POST /compute/v2.1/os-server-external-events => generated 696 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:25:59.563083 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-2c86cc32-b2f3-4dae-ad05-f1c91e668d4f tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Calling method '.version_select at 0x7fdda2b2b550>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:59.756776 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-80474db1-66df-4651-aaf4-c8ad3ca1e80d tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:25:59.758386 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-80474db1-66df-4651-aaf4-c8ad3ca1e80d tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] [instance: 4eb3d8d6-79ae-4109-a5a7-835c459d1719] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:25:59.769950 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-80474db1-66df-4651-aaf4-c8ad3ca1e80d tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:59.770934 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-80474db1-66df-4651-aaf4-c8ad3ca1e80d tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:25:59.772217 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-80474db1-66df-4651-aaf4-c8ad3ca1e80d tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:25:59.772815 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-80474db1-66df-4651-aaf4-c8ad3ca1e80d tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:00.050091 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-80474db1-66df-4651-aaf4-c8ad3ca1e80d tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a23fafbab18d41ee82b84bbdf94cf0fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfc2e42a0df346f4adef152ba2e3fa71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:26:00.059354 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-80474db1-66df-4651-aaf4-c8ad3ca1e80d tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'a23fafbab18d41ee82b84bbdf94cf0fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfc2e42a0df346f4adef152ba2e3fa71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:26:00.060913 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-80474db1-66df-4651-aaf4-c8ad3ca1e80d tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'a23fafbab18d41ee82b84bbdf94cf0fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfc2e42a0df346f4adef152ba2e3fa71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:26:00.062193 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-80474db1-66df-4651-aaf4-c8ad3ca1e80d tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] 10.222.0.16 "GET /compute/v2.1/servers/4eb3d8d6-79ae-4109-a5a7-835c459d1719" status: 200 len: 1874 microversion: 2.32 time: 0.310786 Jul 16 21:26:00.062764 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1867/3737] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:25:59 2022] GET /compute/v2.1/servers/4eb3d8d6-79ae-4109-a5a7-835c459d1719 => generated 1874 bytes in 312 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 16 21:26:00.067338 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-d8dad107-5425-4165-a6a9-9e5cb34bc28c tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:26:00.068577 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-d8dad107-5425-4165-a6a9-9e5cb34bc28c tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: c20f3279-68ae-4b04-8b87-cc16e5f73f1e] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:26:00.075074 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d8dad107-5425-4165-a6a9-9e5cb34bc28c tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:00.075425 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d8dad107-5425-4165-a6a9-9e5cb34bc28c tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:00.076395 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d8dad107-5425-4165-a6a9-9e5cb34bc28c tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:00.076841 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d8dad107-5425-4165-a6a9-9e5cb34bc28c tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:00.081198 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=100 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:26:00.229116 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-d8dad107-5425-4165-a6a9-9e5cb34bc28c tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '027dcc48e4fd4c36a5004096ad5e43f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dad823e240149edb30e592380249a13', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:26:00.236566 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-d8dad107-5425-4165-a6a9-9e5cb34bc28c tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "GET /compute/v2.1/servers/c20f3279-68ae-4b04-8b87-cc16e5f73f1e" status: 200 len: 1309 microversion: 2.1 time: 0.172818 Jul 16 21:26:00.237130 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1868/3738] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:26:00 2022] GET /compute/v2.1/servers/c20f3279-68ae-4b04-8b87-cc16e5f73f1e => generated 1309 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:26:00.417986 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-3f54ab46-f669-46c6-8279-929953902fa2 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:26:00.418609 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-3f54ab46-f669-46c6-8279-929953902fa2 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] [instance: 2fad683a-f21f-4a9c-8323-e9380ef7f9ef] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:26:00.425434 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3f54ab46-f669-46c6-8279-929953902fa2 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:00.427816 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3f54ab46-f669-46c6-8279-929953902fa2 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:00.427816 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3f54ab46-f669-46c6-8279-929953902fa2 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:00.427816 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3f54ab46-f669-46c6-8279-929953902fa2 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:00.444131 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-3f54ab46-f669-46c6-8279-929953902fa2 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] HTTP exception thrown: Instance 2fad683a-f21f-4a9c-8323-e9380ef7f9ef could not be found. Jul 16 21:26:00.445781 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-3f54ab46-f669-46c6-8279-929953902fa2 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Request method failure captured: Jul 16 21:26:00.445781 ncdv-master-1 devstack@n-api.service[138681]: request: GET /compute/v2.1/servers/2fad683a-f21f-4a9c-8323-e9380ef7f9ef HTTP/1.1 Jul 16 21:26:00.445781 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:26:00.445781 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:26:00.445781 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:26:00.445781 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 21:26:00.445781 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:26:00.445781 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:26:00.445781 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:26:00.445781 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:26:00.445781 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0yu6GvklgrJz64l4jI_9XrerhdTTPfYTDqcboWClQTVUOsFR3X72A-yD-xnMKndBGdnSSARa9ZpL_5E_A7dDhsSNRAFKwYpwFmXgR867L_9gOcd2pdrdnzmP_tHWcFxJFbkFQsl_MPEJMXgACkcSj5Ps4CNmUG4pTC9v8iTd4ac5zTs Jul 16 21:26:00.445781 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:26:00.445781 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:26:00.445781 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:26:00.445781 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:26:00.445781 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:26:00.445781 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:26:00.445781 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: 22ca92a86e6c40d59071e58ce4085568 Jul 16 21:26:00.445781 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-AttachVolumeTestJSON-1168541016 Jul 16 21:26:00.445781 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:26:00.445781 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:26:00.445781 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/22ca92a86e6c40d59071e58ce4085568"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/22ca92a86e6c40d59071e58ce4085568"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_22ca92a86e6c40d59071e58ce4085568"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/22ca92a86e6c40d59071e58ce4085568"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:26:00.445781 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-AttachVolumeTestJSON-1168541016 Jul 16 21:26:00.445781 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: 22ca92a86e6c40d59071e58ce4085568 Jul 16 21:26:00.445781 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-AttachVolumeTestJSON-1168541016 Jul 16 21:26:00.445781 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-AttachVolumeTestJSON-1168541016-project Jul 16 21:26:00.445781 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:26:00.445781 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:26:00.445781 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: bba77f6ef42547f1aa8480e27868bd4a Jul 16 21:26:00.445781 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-AttachVolumeTestJSON-1168541016-project Jul 16 21:26:00.445781 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:26:00.445781 ncdv-master-1 devstack@n-api.service[138681]: exception: Instance 2fad683a-f21f-4a9c-8323-e9380ef7f9ef could not be found. Jul 16 21:26:00.445781 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': '2fad683a-f21f-4a9c-8323-e9380ef7f9ef'} Jul 16 21:26:00.445781 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:26:00.445781 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:26:00.445781 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:26:00.445781 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:26:00.445781 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:26:00.445781 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:26:00.445781 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:26:00.445781 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:26:00.445781 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:26:00.445781 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:26:00.445781 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 2fad683a-f21f-4a9c-8323-e9380ef7f9ef could not be found. Jul 16 21:26:00.445781 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:26:00.445781 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:26:00.445781 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:26:00.445781 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:26:00.445781 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:26:00.445781 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:26:00.445781 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:26:00.445781 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:26:00.445781 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:26:00.445781 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:26:00.445781 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:26:00.445781 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:26:00.445781 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:26:00.445781 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:26:00.445781 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:26:00.445781 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:26:00.445781 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 2fad683a-f21f-4a9c-8323-e9380ef7f9ef could not be found. Jul 16 21:26:00.445781 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:26:00.445781 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:26:00.445781 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:26:00.451038 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:26:00.451038 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:26:00.451038 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:26:00.451038 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:26:00.451038 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:26:00.451038 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 2fad683a-f21f-4a9c-8323-e9380ef7f9ef could not be found. Jul 16 21:26:00.451038 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:26:00.451038 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-3f54ab46-f669-46c6-8279-929953902fa2 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Returning 404 to user: Instance 2fad683a-f21f-4a9c-8323-e9380ef7f9ef could not be found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:26:00.451038 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-3f54ab46-f669-46c6-8279-929953902fa2 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] 10.222.0.16 "GET /compute/v2.1/servers/2fad683a-f21f-4a9c-8323-e9380ef7f9ef" status: 404 len: 111 microversion: 2.1 time: 0.209135 Jul 16 21:26:00.451038 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1869/3739] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:26:00 2022] GET /compute/v2.1/servers/2fad683a-f21f-4a9c-8323-e9380ef7f9ef => generated 111 bytes in 210 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:26:00.456980 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-5985b31d-a834-4a33-a73f-bfa9ec359e0e tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:26:00.456980 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-5985b31d-a834-4a33-a73f-bfa9ec359e0e tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 00c8a753-5bf0-448c-8107-3946aa9bf7ae] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:26:00.462367 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5985b31d-a834-4a33-a73f-bfa9ec359e0e tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:00.462856 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5985b31d-a834-4a33-a73f-bfa9ec359e0e tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:00.463650 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5985b31d-a834-4a33-a73f-bfa9ec359e0e tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:00.464098 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5985b31d-a834-4a33-a73f-bfa9ec359e0e tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:00.605988 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-5985b31d-a834-4a33-a73f-bfa9ec359e0e tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f2e974ab5aa48dfb4b00f810dcedb63', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd31be8475a6a49d6bfe8132dd050c6c0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:26:00.615537 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-5985b31d-a834-4a33-a73f-bfa9ec359e0e tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "GET /compute/v2.1/servers/00c8a753-5bf0-448c-8107-3946aa9bf7ae" status: 200 len: 1511 microversion: 2.1 time: 0.162246 Jul 16 21:26:00.615990 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1870/3740] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:26:00 2022] GET /compute/v2.1/servers/00c8a753-5bf0-448c-8107-3946aa9bf7ae => generated 1511 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:26:00.621074 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-0b1612eb-d610-43f4-b259-ab397113032b tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Calling method '.version_select at 0x7fdd9c2e1700>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:26:00.774684 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-0b1612eb-d610-43f4-b259-ab397113032b tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] 10.222.0.16 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-415855574" status: 202 len: 0 microversion: 2.1 time: 0.157371 Jul 16 21:26:00.777583 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1871/3741] 10.222.0.16 () {58 vars in 1217 bytes} [Sat Jul 16 21:26:00 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-415855574 => generated 0 bytes in 159 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 21:26:00.779969 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-12fa32c4-7644-4e0a-aa0d-6176bc359fbe tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:26:00.780589 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-12fa32c4-7644-4e0a-aa0d-6176bc359fbe tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] [instance: c2e32bd9-9d80-427b-bb6d-6274fd48173a] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:26:00.787909 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-12fa32c4-7644-4e0a-aa0d-6176bc359fbe tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:00.788438 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-12fa32c4-7644-4e0a-aa0d-6176bc359fbe tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:00.789385 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-12fa32c4-7644-4e0a-aa0d-6176bc359fbe tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:00.789902 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-12fa32c4-7644-4e0a-aa0d-6176bc359fbe tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:00.936250 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-12fa32c4-7644-4e0a-aa0d-6176bc359fbe tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcc9aa2a091c490599f9a9c41163e2eb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da56fc78f06142aba80ca4c6e3d60475', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:26:00.945136 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-12fa32c4-7644-4e0a-aa0d-6176bc359fbe tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] 10.222.0.16 "GET /compute/v2.1/servers/c2e32bd9-9d80-427b-bb6d-6274fd48173a" status: 200 len: 1371 microversion: 2.1 time: 0.168863 Jul 16 21:26:00.945712 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1872/3742] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:26:00 2022] GET /compute/v2.1/servers/c2e32bd9-9d80-427b-bb6d-6274fd48173a => generated 1371 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:26:01.029352 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-2c86cc32-b2f3-4dae-ad05-f1c91e668d4f tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] 10.222.0.16 "DELETE /compute/v2.1/images/1f5a232b-6451-422a-9c8c-7265f8b05ddf" status: 204 len: 0 microversion: 2.1 time: 1.471242 Jul 16 21:26:01.030050 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1871/3743] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:25:59 2022] DELETE /compute/v2.1/images/1f5a232b-6451-422a-9c8c-7265f8b05ddf => generated 0 bytes in 1473 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Jul 16 21:26:01.038621 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-cb436b48-f695-4792-9802-d04e3159296c tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Calling method '.version_select at 0x7fdd9c298160>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:26:01.082131 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-c6408472-b498-4712-a604-a754f4f30e05 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:26:01.082906 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-cb436b48-f695-4792-9802-d04e3159296c tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] HTTP exception thrown: Image not found. Jul 16 21:26:01.083371 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-c6408472-b498-4712-a604-a754f4f30e05 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] [instance: 4eb3d8d6-79ae-4109-a5a7-835c459d1719] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:26:01.085010 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-cb436b48-f695-4792-9802-d04e3159296c tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Request method failure captured: Jul 16 21:26:01.085010 ncdv-master-1 devstack@n-api.service[138681]: request: DELETE /compute/v2.1/images/706863d6-a902-4b5c-abc5-1e3e8a4c0e7a HTTP/1.1 Jul 16 21:26:01.085010 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:26:01.085010 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:26:01.085010 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:26:01.085010 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 21:26:01.085010 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:26:01.085010 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:26:01.085010 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:26:01.085010 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:26:01.085010 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0yyeoZK6r42ZrgYKqzqjzfWlSpAUSwogZvgrU6vI5Ma1BzJUf43iC9DYxchs8EE1qwBvPIbjkNWL7_t3fG2X51Xcxn7MHwhtCU8FoNCaqNnP_KMgNcUFs2fhiEN-sCwj_YUQi35-Y6CAIeyHbGRaJXNXIlr4pCnQFYcZd01OxOnAGvU Jul 16 21:26:01.085010 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:26:01.085010 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:26:01.085010 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:26:01.085010 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:26:01.085010 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:26:01.085010 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:26:01.085010 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: 336b3ef8a8744bf3b86854b32000bda6 Jul 16 21:26:01.085010 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-ImagesOneServerTestJSON-741929816 Jul 16 21:26:01.085010 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:26:01.085010 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:26:01.085010 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/336b3ef8a8744bf3b86854b32000bda6"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/336b3ef8a8744bf3b86854b32000bda6"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_336b3ef8a8744bf3b86854b32000bda6"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/336b3ef8a8744bf3b86854b32000bda6"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:26:01.085010 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-ImagesOneServerTestJSON-741929816 Jul 16 21:26:01.085010 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: 336b3ef8a8744bf3b86854b32000bda6 Jul 16 21:26:01.085010 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-ImagesOneServerTestJSON-741929816 Jul 16 21:26:01.085010 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-ImagesOneServerTestJSON-741929816-project Jul 16 21:26:01.085010 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:26:01.085010 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:26:01.085010 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: 3b0b46c0c1f74adaaaf65d46db7b2618 Jul 16 21:26:01.085010 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-ImagesOneServerTestJSON-741929816-project Jul 16 21:26:01.085010 ncdv-master-1 devstack@n-api.service[138681]: method: Jul 16 21:26:01.085010 ncdv-master-1 devstack@n-api.service[138681]: exception: Image not found. Jul 16 21:26:01.085010 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': '706863d6-a902-4b5c-abc5-1e3e8a4c0e7a'} Jul 16 21:26:01.085010 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:26:01.085010 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:26:01.085010 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 724, in delete Jul 16 21:26:01.085010 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi self._client.call(context, 2, 'delete', args=(image_id,)) Jul 16 21:26:01.085010 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 191, in call Jul 16 21:26:01.085010 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi result = getattr(controller, method)(*args, **kwargs) Jul 16 21:26:01.085010 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/utils.py", line 670, in inner Jul 16 21:26:01.085010 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return RequestIdProxy(wrapped(*args, **kwargs)) Jul 16 21:26:01.085010 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/v2/images.py", line 386, in delete Jul 16 21:26:01.085010 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi resp, body = self.http_client.delete(url) Jul 16 21:26:01.085010 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/adapter.py", line 410, in delete Jul 16 21:26:01.085010 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return self.request(url, 'DELETE', **kwargs) Jul 16 21:26:01.085010 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 380, in request Jul 16 21:26:01.085010 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return self._handle_response(resp) Jul 16 21:26:01.085010 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 120, in _handle_response Jul 16 21:26:01.085010 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.from_response(resp, resp.content) Jul 16 21:26:01.085010 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi glanceclient.exc.HTTPNotFound: HTTP 404 Not Found: Failed to find image 706863d6-a902-4b5c-abc5-1e3e8a4c0e7a to delete Jul 16 21:26:01.085010 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:26:01.085010 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:26:01.085010 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:26:01.085010 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:26:01.085010 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/images.py", line 104, in delete Jul 16 21:26:01.085010 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi self._image_api.delete(context, id) Jul 16 21:26:01.085010 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 1255, in delete Jul 16 21:26:01.085010 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return session.delete(context, image_id) Jul 16 21:26:01.085010 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 726, in delete Jul 16 21:26:01.085010 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.ImageNotFound(image_id=image_id) Jul 16 21:26:01.085010 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.ImageNotFound: Image 706863d6-a902-4b5c-abc5-1e3e8a4c0e7a could not be found. Jul 16 21:26:01.085010 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:26:01.085010 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:26:01.085010 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:26:01.085010 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:26:01.090462 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:26:01.090462 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:26:01.090462 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:26:01.090462 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:26:01.090462 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Jul 16 21:26:01.090462 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Jul 16 21:26:01.090462 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:26:01.090462 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:26:01.090462 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/images.py", line 107, in delete Jul 16 21:26:01.090462 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=explanation) Jul 16 21:26:01.090462 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Image not found. Jul 16 21:26:01.090462 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:26:01.090462 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:26:01.090462 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:26:01.090462 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:26:01.090462 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:26:01.090462 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:26:01.090462 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:26:01.090462 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:26:01.090462 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Image not found. Jul 16 21:26:01.090462 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:26:01.090462 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-cb436b48-f695-4792-9802-d04e3159296c tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Returning 404 to user: Image not found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:26:01.090462 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-cb436b48-f695-4792-9802-d04e3159296c tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] 10.222.0.16 "DELETE /compute/v2.1/images/706863d6-a902-4b5c-abc5-1e3e8a4c0e7a" status: 404 len: 62 microversion: 2.1 time: 0.053027 Jul 16 21:26:01.090462 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1873/3744] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:26:01 2022] DELETE /compute/v2.1/images/706863d6-a902-4b5c-abc5-1e3e8a4c0e7a => generated 62 bytes in 54 msecs (HTTP/1.1 404) 9 headers in 378 bytes (1 switches on core 0) Jul 16 21:26:01.098580 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-32cfef9a-2539-4213-9451-c0e6b0f4f688 tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:26:01.099318 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c6408472-b498-4712-a604-a754f4f30e05 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:01.099563 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-32cfef9a-2539-4213-9451-c0e6b0f4f688 tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] [instance: f00bea08-4180-45d6-ba01-4b104f5615d6] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:26:01.099782 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c6408472-b498-4712-a604-a754f4f30e05 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:01.100758 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c6408472-b498-4712-a604-a754f4f30e05 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:01.101387 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c6408472-b498-4712-a604-a754f4f30e05 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:01.111561 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-32cfef9a-2539-4213-9451-c0e6b0f4f688 tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:01.112409 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-32cfef9a-2539-4213-9451-c0e6b0f4f688 tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:01.113444 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-32cfef9a-2539-4213-9451-c0e6b0f4f688 tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:01.114632 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-32cfef9a-2539-4213-9451-c0e6b0f4f688 tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:01.128448 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:26:01.153920 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-32cfef9a-2539-4213-9451-c0e6b0f4f688 tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] [instance: f00bea08-4180-45d6-ba01-4b104f5615d6] Going to try to terminate instance {{(pid=138681) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 16 21:26:01.522111 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-32cfef9a-2539-4213-9451-c0e6b0f4f688 tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] 10.222.0.16 "DELETE /compute/v2.1/servers/f00bea08-4180-45d6-ba01-4b104f5615d6" status: 204 len: 0 microversion: 2.1 time: 0.428372 Jul 16 21:26:01.522718 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1874/3745] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:26:01 2022] DELETE /compute/v2.1/servers/f00bea08-4180-45d6-ba01-4b104f5615d6 => generated 0 bytes in 431 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 16 21:26:01.529775 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-21e934c7-aa19-4581-af29-4ab76f821a87 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:26:01.530488 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-21e934c7-aa19-4581-af29-4ab76f821a87 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: c20f3279-68ae-4b04-8b87-cc16e5f73f1e] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:26:01.537279 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-21e934c7-aa19-4581-af29-4ab76f821a87 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:01.538205 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-21e934c7-aa19-4581-af29-4ab76f821a87 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:01.538559 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-21e934c7-aa19-4581-af29-4ab76f821a87 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:01.538996 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-21e934c7-aa19-4581-af29-4ab76f821a87 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:01.806098 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-c6408472-b498-4712-a604-a754f4f30e05 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a23fafbab18d41ee82b84bbdf94cf0fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfc2e42a0df346f4adef152ba2e3fa71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:26:01.820033 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-c6408472-b498-4712-a604-a754f4f30e05 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'a23fafbab18d41ee82b84bbdf94cf0fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfc2e42a0df346f4adef152ba2e3fa71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:26:01.821936 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-c6408472-b498-4712-a604-a754f4f30e05 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'a23fafbab18d41ee82b84bbdf94cf0fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfc2e42a0df346f4adef152ba2e3fa71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:26:01.823247 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-c6408472-b498-4712-a604-a754f4f30e05 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] 10.222.0.16 "GET /compute/v2.1/servers/4eb3d8d6-79ae-4109-a5a7-835c459d1719" status: 200 len: 1874 microversion: 2.32 time: 0.745333 Jul 16 21:26:01.824441 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1872/3746] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:26:01 2022] GET /compute/v2.1/servers/4eb3d8d6-79ae-4109-a5a7-835c459d1719 => generated 1874 bytes in 747 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 16 21:26:01.830995 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-12f4f76f-0c00-4756-9bc7-ce5d62fb4165 tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:26:01.831725 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-12f4f76f-0c00-4756-9bc7-ce5d62fb4165 tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] [instance: f00bea08-4180-45d6-ba01-4b104f5615d6] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:26:01.832341 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-21e934c7-aa19-4581-af29-4ab76f821a87 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '027dcc48e4fd4c36a5004096ad5e43f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dad823e240149edb30e592380249a13', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:26:01.841595 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-12f4f76f-0c00-4756-9bc7-ce5d62fb4165 tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:01.841595 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-12f4f76f-0c00-4756-9bc7-ce5d62fb4165 tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:01.841595 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-12f4f76f-0c00-4756-9bc7-ce5d62fb4165 tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:01.841595 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-12f4f76f-0c00-4756-9bc7-ce5d62fb4165 tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:01.843633 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-21e934c7-aa19-4581-af29-4ab76f821a87 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "GET /compute/v2.1/servers/c20f3279-68ae-4b04-8b87-cc16e5f73f1e" status: 200 len: 1309 microversion: 2.1 time: 0.319611 Jul 16 21:26:01.844599 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1875/3747] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:26:01 2022] GET /compute/v2.1/servers/c20f3279-68ae-4b04-8b87-cc16e5f73f1e => generated 1309 bytes in 321 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:26:01.850471 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-a3351fad-2d27-4641-bec3-bdf952779266 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:26:01.851175 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-a3351fad-2d27-4641-bec3-bdf952779266 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 00c8a753-5bf0-448c-8107-3946aa9bf7ae] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:26:01.857611 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a3351fad-2d27-4641-bec3-bdf952779266 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:01.858454 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a3351fad-2d27-4641-bec3-bdf952779266 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:01.859076 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a3351fad-2d27-4641-bec3-bdf952779266 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:01.859593 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a3351fad-2d27-4641-bec3-bdf952779266 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:02.131668 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-12f4f76f-0c00-4756-9bc7-ce5d62fb4165 tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3b0b46c0c1f74adaaaf65d46db7b2618', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '336b3ef8a8744bf3b86854b32000bda6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:26:02.139533 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-12f4f76f-0c00-4756-9bc7-ce5d62fb4165 tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] 10.222.0.16 "GET /compute/v2.1/servers/f00bea08-4180-45d6-ba01-4b104f5615d6" status: 200 len: 1472 microversion: 2.1 time: 0.313749 Jul 16 21:26:02.140392 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1873/3748] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:26:01 2022] GET /compute/v2.1/servers/f00bea08-4180-45d6-ba01-4b104f5615d6 => generated 1472 bytes in 315 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:26:02.145330 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-f3f55181-66b0-4aff-9362-fcd7a76f5a9a tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:26:02.145956 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-f3f55181-66b0-4aff-9362-fcd7a76f5a9a tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] [instance: c2e32bd9-9d80-427b-bb6d-6274fd48173a] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:26:02.152695 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f3f55181-66b0-4aff-9362-fcd7a76f5a9a tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:02.153348 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f3f55181-66b0-4aff-9362-fcd7a76f5a9a tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:02.154327 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f3f55181-66b0-4aff-9362-fcd7a76f5a9a tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:02.154778 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f3f55181-66b0-4aff-9362-fcd7a76f5a9a tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:02.160569 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-a3351fad-2d27-4641-bec3-bdf952779266 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f2e974ab5aa48dfb4b00f810dcedb63', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd31be8475a6a49d6bfe8132dd050c6c0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:26:02.169344 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-a3351fad-2d27-4641-bec3-bdf952779266 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "GET /compute/v2.1/servers/00c8a753-5bf0-448c-8107-3946aa9bf7ae" status: 200 len: 1511 microversion: 2.1 time: 0.323566 Jul 16 21:26:02.170187 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1876/3749] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:26:01 2022] GET /compute/v2.1/servers/00c8a753-5bf0-448c-8107-3946aa9bf7ae => generated 1511 bytes in 325 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:26:02.354587 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-f3f55181-66b0-4aff-9362-fcd7a76f5a9a tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcc9aa2a091c490599f9a9c41163e2eb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da56fc78f06142aba80ca4c6e3d60475', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:26:02.393254 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-f3f55181-66b0-4aff-9362-fcd7a76f5a9a tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] 10.222.0.16 "GET /compute/v2.1/servers/c2e32bd9-9d80-427b-bb6d-6274fd48173a" status: 200 len: 1565 microversion: 2.1 time: 0.249005 Jul 16 21:26:02.407298 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1874/3750] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:26:02 2022] GET /compute/v2.1/servers/c2e32bd9-9d80-427b-bb6d-6274fd48173a => generated 1565 bytes in 254 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:26:02.412528 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-710b0980-db32-400e-ae8c-d0ce5b8d6249 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:26:02.413728 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-710b0980-db32-400e-ae8c-d0ce5b8d6249 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] [instance: c2e32bd9-9d80-427b-bb6d-6274fd48173a] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:26:02.425814 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-710b0980-db32-400e-ae8c-d0ce5b8d6249 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:02.426805 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-710b0980-db32-400e-ae8c-d0ce5b8d6249 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:02.428637 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-710b0980-db32-400e-ae8c-d0ce5b8d6249 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:02.435639 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-710b0980-db32-400e-ae8c-d0ce5b8d6249 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:02.549913 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-710b0980-db32-400e-ae8c-d0ce5b8d6249 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] 10.222.0.16 "GET /compute/v2.1/servers/c2e32bd9-9d80-427b-bb6d-6274fd48173a/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.142580 Jul 16 21:26:02.550707 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1877/3751] 10.222.0.16 () {58 vars in 1274 bytes} [Sat Jul 16 21:26:02 2022] GET /compute/v2.1/servers/c2e32bd9-9d80-427b-bb6d-6274fd48173a/os-interface => generated 284 bytes in 144 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 16 21:26:02.611389 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-52582936-1e3d-4978-a17b-8f0568b90c85 req-1a3ed95a-085b-4667-874b-69d8834479b2 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "f00bea08-4180-45d6-ba01-4b104f5615d6", "name": "network-vif-unplugged", "status": "completed", "tag": "541fa980-4b14-4bc2-af0f-d4bf299e66ca"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:26:02.620694 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-52582936-1e3d-4978-a17b-8f0568b90c85 req-1a3ed95a-085b-4667-874b-69d8834479b2 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:02.621288 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-52582936-1e3d-4978-a17b-8f0568b90c85 req-1a3ed95a-085b-4667-874b-69d8834479b2 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:02.636756 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-52582936-1e3d-4978-a17b-8f0568b90c85 req-1a3ed95a-085b-4667-874b-69d8834479b2 service nova] Creating event network-vif-unplugged:541fa980-4b14-4bc2-af0f-d4bf299e66ca for instance f00bea08-4180-45d6-ba01-4b104f5615d6 on nch2-master-1 Jul 16 21:26:02.642936 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-52582936-1e3d-4978-a17b-8f0568b90c85 req-1a3ed95a-085b-4667-874b-69d8834479b2 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.035999 Jul 16 21:26:02.643706 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1875/3752] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:26:02 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:26:02.846931 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-9c04b65c-fc77-40e8-a5bd-257a91ce3cc2 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:26:02.847772 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-9c04b65c-fc77-40e8-a5bd-257a91ce3cc2 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] [instance: 4eb3d8d6-79ae-4109-a5a7-835c459d1719] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:26:02.868077 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9c04b65c-fc77-40e8-a5bd-257a91ce3cc2 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:02.868685 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9c04b65c-fc77-40e8-a5bd-257a91ce3cc2 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:02.870098 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-369b91ac-0b77-4995-a62a-0a833c44fe4e tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:26:02.870782 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-369b91ac-0b77-4995-a62a-0a833c44fe4e tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: c20f3279-68ae-4b04-8b87-cc16e5f73f1e] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:26:02.871647 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9c04b65c-fc77-40e8-a5bd-257a91ce3cc2 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:02.872922 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9c04b65c-fc77-40e8-a5bd-257a91ce3cc2 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:02.884447 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-369b91ac-0b77-4995-a62a-0a833c44fe4e tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:02.885230 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-369b91ac-0b77-4995-a62a-0a833c44fe4e tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:02.886343 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-369b91ac-0b77-4995-a62a-0a833c44fe4e tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:02.887000 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-369b91ac-0b77-4995-a62a-0a833c44fe4e tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:03.275983 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-369b91ac-0b77-4995-a62a-0a833c44fe4e tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '027dcc48e4fd4c36a5004096ad5e43f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dad823e240149edb30e592380249a13', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:26:03.285869 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-369b91ac-0b77-4995-a62a-0a833c44fe4e tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "GET /compute/v2.1/servers/c20f3279-68ae-4b04-8b87-cc16e5f73f1e" status: 200 len: 1489 microversion: 2.1 time: 0.427892 Jul 16 21:26:03.286486 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1876/3753] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:26:02 2022] GET /compute/v2.1/servers/c20f3279-68ae-4b04-8b87-cc16e5f73f1e => generated 1489 bytes in 430 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:26:03.291629 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-08fe9f9e-15fd-4bf1-adb5-89f5e2d6f9a0 tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:26:03.292249 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-08fe9f9e-15fd-4bf1-adb5-89f5e2d6f9a0 tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] [instance: f00bea08-4180-45d6-ba01-4b104f5615d6] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:26:03.294056 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-9c04b65c-fc77-40e8-a5bd-257a91ce3cc2 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a23fafbab18d41ee82b84bbdf94cf0fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfc2e42a0df346f4adef152ba2e3fa71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:26:03.299311 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-08fe9f9e-15fd-4bf1-adb5-89f5e2d6f9a0 tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:03.299843 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-08fe9f9e-15fd-4bf1-adb5-89f5e2d6f9a0 tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:03.300657 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-08fe9f9e-15fd-4bf1-adb5-89f5e2d6f9a0 tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:03.301261 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-08fe9f9e-15fd-4bf1-adb5-89f5e2d6f9a0 tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:03.310844 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-9c04b65c-fc77-40e8-a5bd-257a91ce3cc2 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'a23fafbab18d41ee82b84bbdf94cf0fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfc2e42a0df346f4adef152ba2e3fa71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:26:03.312410 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-9c04b65c-fc77-40e8-a5bd-257a91ce3cc2 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'a23fafbab18d41ee82b84bbdf94cf0fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfc2e42a0df346f4adef152ba2e3fa71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:26:03.313598 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-9c04b65c-fc77-40e8-a5bd-257a91ce3cc2 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] 10.222.0.16 "GET /compute/v2.1/servers/4eb3d8d6-79ae-4109-a5a7-835c459d1719" status: 200 len: 1874 microversion: 2.32 time: 0.472567 Jul 16 21:26:03.314161 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1878/3754] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:26:02 2022] GET /compute/v2.1/servers/4eb3d8d6-79ae-4109-a5a7-835c459d1719 => generated 1874 bytes in 474 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:26:03.319175 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-faba8ec5-123e-4751-b526-490708a84591 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:26:03.319824 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-faba8ec5-123e-4751-b526-490708a84591 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 00c8a753-5bf0-448c-8107-3946aa9bf7ae] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:26:03.340236 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-faba8ec5-123e-4751-b526-490708a84591 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:03.341168 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-faba8ec5-123e-4751-b526-490708a84591 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:03.342025 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-faba8ec5-123e-4751-b526-490708a84591 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:03.342546 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-faba8ec5-123e-4751-b526-490708a84591 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:03.610754 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-faba8ec5-123e-4751-b526-490708a84591 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f2e974ab5aa48dfb4b00f810dcedb63', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd31be8475a6a49d6bfe8132dd050c6c0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:26:03.614888 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-08fe9f9e-15fd-4bf1-adb5-89f5e2d6f9a0 tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3b0b46c0c1f74adaaaf65d46db7b2618', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '336b3ef8a8744bf3b86854b32000bda6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:26:03.621184 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-faba8ec5-123e-4751-b526-490708a84591 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "GET /compute/v2.1/servers/00c8a753-5bf0-448c-8107-3946aa9bf7ae" status: 200 len: 1511 microversion: 2.1 time: 0.305841 Jul 16 21:26:03.621975 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1879/3755] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:26:03 2022] GET /compute/v2.1/servers/00c8a753-5bf0-448c-8107-3946aa9bf7ae => generated 1511 bytes in 307 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:26:03.631830 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-08fe9f9e-15fd-4bf1-adb5-89f5e2d6f9a0 tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] 10.222.0.16 "GET /compute/v2.1/servers/f00bea08-4180-45d6-ba01-4b104f5615d6" status: 200 len: 1472 microversion: 2.1 time: 0.339061 Jul 16 21:26:03.638470 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1877/3756] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:26:03 2022] GET /compute/v2.1/servers/f00bea08-4180-45d6-ba01-4b104f5615d6 => generated 1472 bytes in 351 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:26:03.639299 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-3921e08f-05f1-4864-afe8-395d864877e2 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:26:03.642039 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-3921e08f-05f1-4864-afe8-395d864877e2 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: 05358d0e-ed2f-443c-835c-f77cd929289e] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:26:03.648560 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3921e08f-05f1-4864-afe8-395d864877e2 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:03.649151 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3921e08f-05f1-4864-afe8-395d864877e2 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:03.650548 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3921e08f-05f1-4864-afe8-395d864877e2 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:03.651341 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3921e08f-05f1-4864-afe8-395d864877e2 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:04.110541 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-3921e08f-05f1-4864-afe8-395d864877e2 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '027dcc48e4fd4c36a5004096ad5e43f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dad823e240149edb30e592380249a13', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:26:04.121972 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-3921e08f-05f1-4864-afe8-395d864877e2 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "GET /compute/v2.1/servers/05358d0e-ed2f-443c-835c-f77cd929289e" status: 200 len: 1490 microversion: 2.1 time: 0.487345 Jul 16 21:26:04.122984 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1880/3757] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:26:03 2022] GET /compute/v2.1/servers/05358d0e-ed2f-443c-835c-f77cd929289e => generated 1490 bytes in 490 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:26:04.128337 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-8b108842-548d-4535-ac52-f16893a343cf req-97b6461c-b173-43d8-a6d4-d85e33bd140b service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "c2e32bd9-9d80-427b-bb6d-6274fd48173a", "tag": "347e5d31-8efd-4cb0-82eb-3b8a6e86e0cd"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:26:04.139788 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-8b108842-548d-4535-ac52-f16893a343cf req-97b6461c-b173-43d8-a6d4-d85e33bd140b service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:04.144224 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-e4939802-ccca-4a8a-a200-f3d2bf0e496c tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:26:04.145177 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-8b108842-548d-4535-ac52-f16893a343cf req-97b6461c-b173-43d8-a6d4-d85e33bd140b service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:04.146174 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-e4939802-ccca-4a8a-a200-f3d2bf0e496c tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: c20f3279-68ae-4b04-8b87-cc16e5f73f1e] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:26:04.160615 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e4939802-ccca-4a8a-a200-f3d2bf0e496c tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:04.165594 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e4939802-ccca-4a8a-a200-f3d2bf0e496c tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:04.166683 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e4939802-ccca-4a8a-a200-f3d2bf0e496c tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:04.167554 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e4939802-ccca-4a8a-a200-f3d2bf0e496c tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:04.174704 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-8b108842-548d-4535-ac52-f16893a343cf req-97b6461c-b173-43d8-a6d4-d85e33bd140b service nova] Creating event network-changed:347e5d31-8efd-4cb0-82eb-3b8a6e86e0cd for instance c2e32bd9-9d80-427b-bb6d-6274fd48173a on nch1-master-1 Jul 16 21:26:04.193484 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-8b108842-548d-4535-ac52-f16893a343cf req-97b6461c-b173-43d8-a6d4-d85e33bd140b service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.068258 Jul 16 21:26:04.194858 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1878/3758] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:26:04 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 71 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:26:04.337248 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-25bea1cd-d098-4499-85af-51abd2376ecd tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:26:04.339187 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-25bea1cd-d098-4499-85af-51abd2376ecd tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] [instance: 4eb3d8d6-79ae-4109-a5a7-835c459d1719] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:26:04.347423 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-25bea1cd-d098-4499-85af-51abd2376ecd tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:04.348662 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-25bea1cd-d098-4499-85af-51abd2376ecd tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:04.354110 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-25bea1cd-d098-4499-85af-51abd2376ecd tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:04.354733 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-25bea1cd-d098-4499-85af-51abd2376ecd tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:04.782552 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-e4939802-ccca-4a8a-a200-f3d2bf0e496c tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '027dcc48e4fd4c36a5004096ad5e43f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dad823e240149edb30e592380249a13', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:26:04.791894 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-e4939802-ccca-4a8a-a200-f3d2bf0e496c tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "GET /compute/v2.1/servers/c20f3279-68ae-4b04-8b87-cc16e5f73f1e" status: 200 len: 1489 microversion: 2.1 time: 0.654494 Jul 16 21:26:04.792888 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1881/3759] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:26:04 2022] GET /compute/v2.1/servers/c20f3279-68ae-4b04-8b87-cc16e5f73f1e => generated 1489 bytes in 656 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:26:04.798274 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-72bea4cd-2411-40a9-b5c1-d3d033bb2b17 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:26:04.798883 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-72bea4cd-2411-40a9-b5c1-d3d033bb2b17 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 00c8a753-5bf0-448c-8107-3946aa9bf7ae] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:26:04.839335 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-72bea4cd-2411-40a9-b5c1-d3d033bb2b17 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:04.839894 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-72bea4cd-2411-40a9-b5c1-d3d033bb2b17 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:04.842395 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-72bea4cd-2411-40a9-b5c1-d3d033bb2b17 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:04.843910 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-72bea4cd-2411-40a9-b5c1-d3d033bb2b17 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:05.024622 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-25bea1cd-d098-4499-85af-51abd2376ecd tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a23fafbab18d41ee82b84bbdf94cf0fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfc2e42a0df346f4adef152ba2e3fa71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:26:05.053811 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-25bea1cd-d098-4499-85af-51abd2376ecd tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'a23fafbab18d41ee82b84bbdf94cf0fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfc2e42a0df346f4adef152ba2e3fa71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:26:05.055884 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-25bea1cd-d098-4499-85af-51abd2376ecd tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'a23fafbab18d41ee82b84bbdf94cf0fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfc2e42a0df346f4adef152ba2e3fa71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:26:05.057268 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-25bea1cd-d098-4499-85af-51abd2376ecd tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] 10.222.0.16 "GET /compute/v2.1/servers/4eb3d8d6-79ae-4109-a5a7-835c459d1719" status: 200 len: 2824 microversion: 2.32 time: 0.726800 Jul 16 21:26:05.057794 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1879/3760] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:26:04 2022] GET /compute/v2.1/servers/4eb3d8d6-79ae-4109-a5a7-835c459d1719 => generated 2824 bytes in 728 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:26:05.078390 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-579da488-41a2-4e6a-91a6-e487849f6d4c tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:26:05.082228 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-579da488-41a2-4e6a-91a6-e487849f6d4c tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] [instance: f00bea08-4180-45d6-ba01-4b104f5615d6] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:26:05.092631 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-579da488-41a2-4e6a-91a6-e487849f6d4c tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:05.093474 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-579da488-41a2-4e6a-91a6-e487849f6d4c tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:05.094326 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-579da488-41a2-4e6a-91a6-e487849f6d4c tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:05.094929 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-579da488-41a2-4e6a-91a6-e487849f6d4c tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:05.366674 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-72bea4cd-2411-40a9-b5c1-d3d033bb2b17 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f2e974ab5aa48dfb4b00f810dcedb63', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd31be8475a6a49d6bfe8132dd050c6c0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:26:05.381540 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-72bea4cd-2411-40a9-b5c1-d3d033bb2b17 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "GET /compute/v2.1/servers/00c8a753-5bf0-448c-8107-3946aa9bf7ae" status: 200 len: 1511 microversion: 2.1 time: 0.587110 Jul 16 21:26:05.382252 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1882/3761] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:26:04 2022] GET /compute/v2.1/servers/00c8a753-5bf0-448c-8107-3946aa9bf7ae => generated 1511 bytes in 588 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:26:05.391718 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-8298f6ac-7db4-4c35-b536-cba471331a13 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:26:05.392310 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-8298f6ac-7db4-4c35-b536-cba471331a13 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: c20f3279-68ae-4b04-8b87-cc16e5f73f1e] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:26:05.398688 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8298f6ac-7db4-4c35-b536-cba471331a13 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:05.399187 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8298f6ac-7db4-4c35-b536-cba471331a13 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:05.399982 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8298f6ac-7db4-4c35-b536-cba471331a13 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:05.400438 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8298f6ac-7db4-4c35-b536-cba471331a13 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:05.429534 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-8298f6ac-7db4-4c35-b536-cba471331a13 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: c20f3279-68ae-4b04-8b87-cc16e5f73f1e] Going to try to terminate instance {{(pid=138681) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 16 21:26:05.553452 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-579da488-41a2-4e6a-91a6-e487849f6d4c tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3b0b46c0c1f74adaaaf65d46db7b2618', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '336b3ef8a8744bf3b86854b32000bda6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:26:05.563286 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-579da488-41a2-4e6a-91a6-e487849f6d4c tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] 10.222.0.16 "GET /compute/v2.1/servers/f00bea08-4180-45d6-ba01-4b104f5615d6" status: 200 len: 1472 microversion: 2.1 time: 0.498809 Jul 16 21:26:05.564511 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1880/3762] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:26:05 2022] GET /compute/v2.1/servers/f00bea08-4180-45d6-ba01-4b104f5615d6 => generated 1472 bytes in 501 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:26:05.582508 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-8d82d668-daa7-4af8-9171-d1bc26a57a54 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:26:05.582508 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-8d82d668-daa7-4af8-9171-d1bc26a57a54 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] [instance: 4eb3d8d6-79ae-4109-a5a7-835c459d1719] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:26:05.586806 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8d82d668-daa7-4af8-9171-d1bc26a57a54 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:05.587479 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8d82d668-daa7-4af8-9171-d1bc26a57a54 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:05.588434 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8d82d668-daa7-4af8-9171-d1bc26a57a54 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:05.589043 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8d82d668-daa7-4af8-9171-d1bc26a57a54 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:05.817276 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-8d82d668-daa7-4af8-9171-d1bc26a57a54 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] 10.222.0.16 "GET /compute/v2.1/servers/4eb3d8d6-79ae-4109-a5a7-835c459d1719/os-interface" status: 200 len: 1829 microversion: 2.32 time: 0.251266 Jul 16 21:26:05.828839 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1881/3763] 10.222.0.16 () {60 vars in 1315 bytes} [Sat Jul 16 21:26:05 2022] GET /compute/v2.1/servers/4eb3d8d6-79ae-4109-a5a7-835c459d1719/os-interface => generated 1829 bytes in 264 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:26:05.834796 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-056b1dc6-9577-4afc-a725-e686413959aa req-c2a30b84-bf2b-422f-9e8d-dfbb577ae765 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "f00bea08-4180-45d6-ba01-4b104f5615d6", "name": "network-vif-deleted", "tag": "541fa980-4b14-4bc2-af0f-d4bf299e66ca"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:26:05.846184 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-056b1dc6-9577-4afc-a725-e686413959aa req-c2a30b84-bf2b-422f-9e8d-dfbb577ae765 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:05.846671 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-056b1dc6-9577-4afc-a725-e686413959aa req-c2a30b84-bf2b-422f-9e8d-dfbb577ae765 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:05.877980 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-056b1dc6-9577-4afc-a725-e686413959aa req-c2a30b84-bf2b-422f-9e8d-dfbb577ae765 service nova] Creating event network-vif-deleted:541fa980-4b14-4bc2-af0f-d4bf299e66ca for instance f00bea08-4180-45d6-ba01-4b104f5615d6 on nch2-master-1 Jul 16 21:26:05.885936 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-056b1dc6-9577-4afc-a725-e686413959aa req-c2a30b84-bf2b-422f-9e8d-dfbb577ae765 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.055340 Jul 16 21:26:05.886716 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1882/3764] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:26:05 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:26:05.918367 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-8298f6ac-7db4-4c35-b536-cba471331a13 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "DELETE /compute/v2.1/servers/c20f3279-68ae-4b04-8b87-cc16e5f73f1e" status: 204 len: 0 microversion: 2.1 time: 0.534274 Jul 16 21:26:05.919576 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1883/3765] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:26:05 2022] DELETE /compute/v2.1/servers/c20f3279-68ae-4b04-8b87-cc16e5f73f1e => generated 0 bytes in 537 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 16 21:26:05.932231 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-3340047a-e3a0-4f55-86c9-c07d297bcebd tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:26:05.935487 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-3340047a-e3a0-4f55-86c9-c07d297bcebd tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: c20f3279-68ae-4b04-8b87-cc16e5f73f1e] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:26:05.954525 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3340047a-e3a0-4f55-86c9-c07d297bcebd tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:05.954984 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3340047a-e3a0-4f55-86c9-c07d297bcebd tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:05.955801 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3340047a-e3a0-4f55-86c9-c07d297bcebd tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:05.956272 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3340047a-e3a0-4f55-86c9-c07d297bcebd tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:06.399155 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-495bea15-77c9-4bc9-b0bb-cb73aaef3156 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:26:06.400574 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-495bea15-77c9-4bc9-b0bb-cb73aaef3156 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 00c8a753-5bf0-448c-8107-3946aa9bf7ae] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:26:06.407995 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-495bea15-77c9-4bc9-b0bb-cb73aaef3156 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:06.408462 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-495bea15-77c9-4bc9-b0bb-cb73aaef3156 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:06.409282 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-495bea15-77c9-4bc9-b0bb-cb73aaef3156 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:06.409754 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-495bea15-77c9-4bc9-b0bb-cb73aaef3156 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:06.428245 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=96,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:26:06.593085 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-3340047a-e3a0-4f55-86c9-c07d297bcebd tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '027dcc48e4fd4c36a5004096ad5e43f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dad823e240149edb30e592380249a13', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:26:06.603882 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-3340047a-e3a0-4f55-86c9-c07d297bcebd tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "GET /compute/v2.1/servers/c20f3279-68ae-4b04-8b87-cc16e5f73f1e" status: 200 len: 1495 microversion: 2.1 time: 0.679547 Jul 16 21:26:06.604549 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1883/3766] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:26:05 2022] GET /compute/v2.1/servers/c20f3279-68ae-4b04-8b87-cc16e5f73f1e => generated 1495 bytes in 681 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:26:06.609626 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-8697e646-5939-411d-a17e-0b5b5c98d1b7 tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:26:06.610525 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-8697e646-5939-411d-a17e-0b5b5c98d1b7 tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] [instance: f00bea08-4180-45d6-ba01-4b104f5615d6] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:26:06.617347 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8697e646-5939-411d-a17e-0b5b5c98d1b7 tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:06.617828 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8697e646-5939-411d-a17e-0b5b5c98d1b7 tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:06.618643 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8697e646-5939-411d-a17e-0b5b5c98d1b7 tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:06.619154 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8697e646-5939-411d-a17e-0b5b5c98d1b7 tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:06.654146 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8697e646-5939-411d-a17e-0b5b5c98d1b7 tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:06.654607 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8697e646-5939-411d-a17e-0b5b5c98d1b7 tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:06.655089 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.objects.instance [None req-8697e646-5939-411d-a17e-0b5b5c98d1b7 tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Lazy-loading 'fault' on Instance uuid f00bea08-4180-45d6-ba01-4b104f5615d6 {{(pid=138680) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:26:06.725637 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-8697e646-5939-411d-a17e-0b5b5c98d1b7 tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3b0b46c0c1f74adaaaf65d46db7b2618', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '336b3ef8a8744bf3b86854b32000bda6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:26:06.733319 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-8697e646-5939-411d-a17e-0b5b5c98d1b7 tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] 10.222.0.16 "GET /compute/v2.1/servers/f00bea08-4180-45d6-ba01-4b104f5615d6" status: 200 len: 1306 microversion: 2.1 time: 0.127766 Jul 16 21:26:06.733904 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1884/3767] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:26:06 2022] GET /compute/v2.1/servers/f00bea08-4180-45d6-ba01-4b104f5615d6 => generated 1306 bytes in 129 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:26:06.801649 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-495bea15-77c9-4bc9-b0bb-cb73aaef3156 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f2e974ab5aa48dfb4b00f810dcedb63', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd31be8475a6a49d6bfe8132dd050c6c0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:26:06.801649 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-495bea15-77c9-4bc9-b0bb-cb73aaef3156 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "GET /compute/v2.1/servers/00c8a753-5bf0-448c-8107-3946aa9bf7ae" status: 200 len: 1511 microversion: 2.1 time: 0.405586 Jul 16 21:26:06.801649 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1884/3768] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:26:06 2022] GET /compute/v2.1/servers/00c8a753-5bf0-448c-8107-3946aa9bf7ae => generated 1511 bytes in 407 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:26:07.627301 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-00ac7dac-7b78-4ba4-a627-c34708c862cb tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:26:07.627301 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-00ac7dac-7b78-4ba4-a627-c34708c862cb tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: c20f3279-68ae-4b04-8b87-cc16e5f73f1e] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:26:07.643680 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-00ac7dac-7b78-4ba4-a627-c34708c862cb tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:07.644271 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-00ac7dac-7b78-4ba4-a627-c34708c862cb tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:07.645426 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-00ac7dac-7b78-4ba4-a627-c34708c862cb tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:07.646550 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-00ac7dac-7b78-4ba4-a627-c34708c862cb tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:07.752954 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-f9f28c38-41d7-4b87-9fff-96ff92f142b6 tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:26:07.753712 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-f9f28c38-41d7-4b87-9fff-96ff92f142b6 tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] [instance: f00bea08-4180-45d6-ba01-4b104f5615d6] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:26:07.774361 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f9f28c38-41d7-4b87-9fff-96ff92f142b6 tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:07.775247 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f9f28c38-41d7-4b87-9fff-96ff92f142b6 tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:07.777488 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f9f28c38-41d7-4b87-9fff-96ff92f142b6 tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:07.778068 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f9f28c38-41d7-4b87-9fff-96ff92f142b6 tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:07.809000 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-f9f28c38-41d7-4b87-9fff-96ff92f142b6 tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] HTTP exception thrown: Instance f00bea08-4180-45d6-ba01-4b104f5615d6 could not be found. Jul 16 21:26:07.811168 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-f9f28c38-41d7-4b87-9fff-96ff92f142b6 tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Request method failure captured: Jul 16 21:26:07.811168 ncdv-master-1 devstack@n-api.service[138681]: request: GET /compute/v2.1/servers/f00bea08-4180-45d6-ba01-4b104f5615d6 HTTP/1.1 Jul 16 21:26:07.811168 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:26:07.811168 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:26:07.811168 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:26:07.811168 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 21:26:07.811168 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:26:07.811168 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:26:07.811168 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:26:07.811168 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:26:07.811168 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0yyeoZK6r42ZrgYKqzqjzfWlSpAUSwogZvgrU6vI5Ma1BzJUf43iC9DYxchs8EE1qwBvPIbjkNWL7_t3fG2X51Xcxn7MHwhtCU8FoNCaqNnP_KMgNcUFs2fhiEN-sCwj_YUQi35-Y6CAIeyHbGRaJXNXIlr4pCnQFYcZd01OxOnAGvU Jul 16 21:26:07.811168 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:26:07.811168 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:26:07.811168 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:26:07.811168 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:26:07.811168 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:26:07.811168 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:26:07.811168 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: 336b3ef8a8744bf3b86854b32000bda6 Jul 16 21:26:07.811168 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-ImagesOneServerTestJSON-741929816 Jul 16 21:26:07.811168 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:26:07.811168 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:26:07.811168 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/336b3ef8a8744bf3b86854b32000bda6"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/336b3ef8a8744bf3b86854b32000bda6"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_336b3ef8a8744bf3b86854b32000bda6"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/336b3ef8a8744bf3b86854b32000bda6"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:26:07.811168 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-ImagesOneServerTestJSON-741929816 Jul 16 21:26:07.811168 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: 336b3ef8a8744bf3b86854b32000bda6 Jul 16 21:26:07.811168 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-ImagesOneServerTestJSON-741929816 Jul 16 21:26:07.811168 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-ImagesOneServerTestJSON-741929816-project Jul 16 21:26:07.811168 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:26:07.811168 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:26:07.811168 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: 3b0b46c0c1f74adaaaf65d46db7b2618 Jul 16 21:26:07.811168 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-ImagesOneServerTestJSON-741929816-project Jul 16 21:26:07.811168 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:26:07.811168 ncdv-master-1 devstack@n-api.service[138681]: exception: Instance f00bea08-4180-45d6-ba01-4b104f5615d6 could not be found. Jul 16 21:26:07.811168 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': 'f00bea08-4180-45d6-ba01-4b104f5615d6'} Jul 16 21:26:07.811168 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:26:07.811168 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:26:07.811168 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:26:07.811168 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:26:07.811168 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:26:07.811168 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:26:07.811168 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:26:07.811168 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:26:07.811168 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:26:07.811168 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:26:07.811168 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance f00bea08-4180-45d6-ba01-4b104f5615d6 could not be found. Jul 16 21:26:07.811168 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:26:07.811168 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:26:07.811168 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:26:07.811168 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:26:07.811168 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:26:07.811168 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:26:07.811168 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:26:07.811168 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:26:07.811168 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:26:07.811168 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:26:07.811168 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:26:07.811168 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:26:07.811168 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:26:07.811168 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:26:07.811168 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:26:07.811168 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:26:07.811168 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance f00bea08-4180-45d6-ba01-4b104f5615d6 could not be found. Jul 16 21:26:07.811168 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:26:07.811168 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:26:07.811168 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:26:07.814705 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:26:07.814705 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:26:07.814705 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:26:07.814705 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:26:07.814705 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:26:07.814705 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance f00bea08-4180-45d6-ba01-4b104f5615d6 could not be found. Jul 16 21:26:07.814705 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:26:07.815881 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-f9f28c38-41d7-4b87-9fff-96ff92f142b6 tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] Returning 404 to user: Instance f00bea08-4180-45d6-ba01-4b104f5615d6 could not be found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:26:07.816790 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-f9f28c38-41d7-4b87-9fff-96ff92f142b6 tempest-ImagesOneServerTestJSON-741929816 tempest-ImagesOneServerTestJSON-741929816-project] 10.222.0.16 "GET /compute/v2.1/servers/f00bea08-4180-45d6-ba01-4b104f5615d6" status: 404 len: 111 microversion: 2.1 time: 0.069737 Jul 16 21:26:07.817713 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1885/3769] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:26:07 2022] GET /compute/v2.1/servers/f00bea08-4180-45d6-ba01-4b104f5615d6 => generated 111 bytes in 71 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Jul 16 21:26:07.826757 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-085c5992-d1a7-4a58-a6b0-3e32da63427b tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:26:07.827756 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-085c5992-d1a7-4a58-a6b0-3e32da63427b tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 00c8a753-5bf0-448c-8107-3946aa9bf7ae] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:26:07.835780 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-085c5992-d1a7-4a58-a6b0-3e32da63427b tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:07.836237 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-085c5992-d1a7-4a58-a6b0-3e32da63427b tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:07.837043 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-085c5992-d1a7-4a58-a6b0-3e32da63427b tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:07.837554 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-085c5992-d1a7-4a58-a6b0-3e32da63427b tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:08.094393 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-00ac7dac-7b78-4ba4-a627-c34708c862cb tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '027dcc48e4fd4c36a5004096ad5e43f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dad823e240149edb30e592380249a13', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:26:08.103497 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-00ac7dac-7b78-4ba4-a627-c34708c862cb tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "GET /compute/v2.1/servers/c20f3279-68ae-4b04-8b87-cc16e5f73f1e" status: 200 len: 1495 microversion: 2.1 time: 0.483935 Jul 16 21:26:08.116618 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1885/3770] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:26:07 2022] GET /compute/v2.1/servers/c20f3279-68ae-4b04-8b87-cc16e5f73f1e => generated 1495 bytes in 493 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:26:08.118935 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-f23d3c51-e20c-4d97-8c67-5dcdf1fdd3c0 req-80746dad-4892-4ecc-91a6-cc7dcf723e36 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "4eb3d8d6-79ae-4109-a5a7-835c459d1719", "tag": "fb878298-9eb7-4cd5-96d9-305373704495"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:26:08.130504 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-f23d3c51-e20c-4d97-8c67-5dcdf1fdd3c0 req-80746dad-4892-4ecc-91a6-cc7dcf723e36 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:08.131453 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-f23d3c51-e20c-4d97-8c67-5dcdf1fdd3c0 req-80746dad-4892-4ecc-91a6-cc7dcf723e36 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:08.138498 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-085c5992-d1a7-4a58-a6b0-3e32da63427b tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f2e974ab5aa48dfb4b00f810dcedb63', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd31be8475a6a49d6bfe8132dd050c6c0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:26:08.148089 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-f23d3c51-e20c-4d97-8c67-5dcdf1fdd3c0 req-80746dad-4892-4ecc-91a6-cc7dcf723e36 service nova] Creating event network-changed:fb878298-9eb7-4cd5-96d9-305373704495 for instance 4eb3d8d6-79ae-4109-a5a7-835c459d1719 on nch1-master-1 Jul 16 21:26:08.149484 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-085c5992-d1a7-4a58-a6b0-3e32da63427b tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "GET /compute/v2.1/servers/00c8a753-5bf0-448c-8107-3946aa9bf7ae" status: 200 len: 1511 microversion: 2.1 time: 0.326157 Jul 16 21:26:08.150253 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1886/3771] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:26:07 2022] GET /compute/v2.1/servers/00c8a753-5bf0-448c-8107-3946aa9bf7ae => generated 1511 bytes in 329 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:26:08.158126 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-f23d3c51-e20c-4d97-8c67-5dcdf1fdd3c0 req-80746dad-4892-4ecc-91a6-cc7dcf723e36 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.044484 Jul 16 21:26:08.158977 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1886/3772] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:26:08 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:26:09.125394 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-c968b63f-702c-45d5-b3a5-51c9c77d574c tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:26:09.126740 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-c968b63f-702c-45d5-b3a5-51c9c77d574c tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: c20f3279-68ae-4b04-8b87-cc16e5f73f1e] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:26:09.141543 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c968b63f-702c-45d5-b3a5-51c9c77d574c tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:09.142036 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c968b63f-702c-45d5-b3a5-51c9c77d574c tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:09.142818 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c968b63f-702c-45d5-b3a5-51c9c77d574c tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:09.143379 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c968b63f-702c-45d5-b3a5-51c9c77d574c tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:09.169320 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-44c1cdb7-f462-4261-81f9-abfa5f8cfc7f tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:26:09.170089 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-44c1cdb7-f462-4261-81f9-abfa5f8cfc7f tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 00c8a753-5bf0-448c-8107-3946aa9bf7ae] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:26:09.191059 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-44c1cdb7-f462-4261-81f9-abfa5f8cfc7f tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:09.191740 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-44c1cdb7-f462-4261-81f9-abfa5f8cfc7f tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:09.193194 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-44c1cdb7-f462-4261-81f9-abfa5f8cfc7f tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:09.195964 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-44c1cdb7-f462-4261-81f9-abfa5f8cfc7f tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:09.235109 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c968b63f-702c-45d5-b3a5-51c9c77d574c tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:09.240550 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c968b63f-702c-45d5-b3a5-51c9c77d574c tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:09.240550 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.objects.instance [None req-c968b63f-702c-45d5-b3a5-51c9c77d574c tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lazy-loading 'fault' on Instance uuid c20f3279-68ae-4b04-8b87-cc16e5f73f1e {{(pid=138681) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:26:09.360732 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-c968b63f-702c-45d5-b3a5-51c9c77d574c tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '027dcc48e4fd4c36a5004096ad5e43f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dad823e240149edb30e592380249a13', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:26:09.373633 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-c968b63f-702c-45d5-b3a5-51c9c77d574c tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "GET /compute/v2.1/servers/c20f3279-68ae-4b04-8b87-cc16e5f73f1e" status: 200 len: 1299 microversion: 2.1 time: 0.253749 Jul 16 21:26:09.374238 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1887/3773] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:26:09 2022] GET /compute/v2.1/servers/c20f3279-68ae-4b04-8b87-cc16e5f73f1e => generated 1299 bytes in 255 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:26:09.442774 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-44c1cdb7-f462-4261-81f9-abfa5f8cfc7f tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f2e974ab5aa48dfb4b00f810dcedb63', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd31be8475a6a49d6bfe8132dd050c6c0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:26:09.453240 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-44c1cdb7-f462-4261-81f9-abfa5f8cfc7f tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "GET /compute/v2.1/servers/00c8a753-5bf0-448c-8107-3946aa9bf7ae" status: 200 len: 1488 microversion: 2.1 time: 0.288999 Jul 16 21:26:09.453818 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1887/3774] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:26:09 2022] GET /compute/v2.1/servers/00c8a753-5bf0-448c-8107-3946aa9bf7ae => generated 1488 bytes in 291 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:26:09.473313 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-e2130039-3238-4c02-a73a-191b60775795 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:26:09.474645 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-e2130039-3238-4c02-a73a-191b60775795 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 00c8a753-5bf0-448c-8107-3946aa9bf7ae] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:26:09.481084 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e2130039-3238-4c02-a73a-191b60775795 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:09.481538 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e2130039-3238-4c02-a73a-191b60775795 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:09.482312 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e2130039-3238-4c02-a73a-191b60775795 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:09.482779 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e2130039-3238-4c02-a73a-191b60775795 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:09.508929 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-e2130039-3238-4c02-a73a-191b60775795 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 00c8a753-5bf0-448c-8107-3946aa9bf7ae] Going to try to terminate instance {{(pid=138681) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 16 21:26:09.934822 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-e2130039-3238-4c02-a73a-191b60775795 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "DELETE /compute/v2.1/servers/00c8a753-5bf0-448c-8107-3946aa9bf7ae" status: 204 len: 0 microversion: 2.1 time: 0.465543 Jul 16 21:26:09.935407 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1888/3775] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:26:09 2022] DELETE /compute/v2.1/servers/00c8a753-5bf0-448c-8107-3946aa9bf7ae => generated 0 bytes in 468 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 16 21:26:09.946099 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-055b58c9-d8bb-4c22-9377-1fd1ac0756f6 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:26:09.946721 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-055b58c9-d8bb-4c22-9377-1fd1ac0756f6 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 00c8a753-5bf0-448c-8107-3946aa9bf7ae] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:26:09.955211 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-055b58c9-d8bb-4c22-9377-1fd1ac0756f6 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:09.955665 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-055b58c9-d8bb-4c22-9377-1fd1ac0756f6 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:09.956475 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-055b58c9-d8bb-4c22-9377-1fd1ac0756f6 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:09.959772 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-055b58c9-d8bb-4c22-9377-1fd1ac0756f6 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:10.120963 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-055b58c9-d8bb-4c22-9377-1fd1ac0756f6 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f2e974ab5aa48dfb4b00f810dcedb63', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd31be8475a6a49d6bfe8132dd050c6c0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:26:10.135470 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-055b58c9-d8bb-4c22-9377-1fd1ac0756f6 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "GET /compute/v2.1/servers/00c8a753-5bf0-448c-8107-3946aa9bf7ae" status: 200 len: 1494 microversion: 2.1 time: 0.195618 Jul 16 21:26:10.136067 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1888/3776] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:26:09 2022] GET /compute/v2.1/servers/00c8a753-5bf0-448c-8107-3946aa9bf7ae => generated 1494 bytes in 197 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:26:10.174890 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-a62eea75-28df-4c2f-a6f2-b50b2c053d58 req-0b3a1131-10c5-4e98-a3f0-76f107d6b2c8 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "c20f3279-68ae-4b04-8b87-cc16e5f73f1e", "name": "network-vif-deleted", "tag": "64f8ba51-ed8e-4f82-b26d-ac00549f8cdf"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:26:10.186259 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-a62eea75-28df-4c2f-a6f2-b50b2c053d58 req-0b3a1131-10c5-4e98-a3f0-76f107d6b2c8 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:10.186633 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-a62eea75-28df-4c2f-a6f2-b50b2c053d58 req-0b3a1131-10c5-4e98-a3f0-76f107d6b2c8 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:10.200936 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-a62eea75-28df-4c2f-a6f2-b50b2c053d58 req-0b3a1131-10c5-4e98-a3f0-76f107d6b2c8 service nova] Creating event network-vif-deleted:64f8ba51-ed8e-4f82-b26d-ac00549f8cdf for instance c20f3279-68ae-4b04-8b87-cc16e5f73f1e on nch1-master-1 Jul 16 21:26:10.206433 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-a62eea75-28df-4c2f-a6f2-b50b2c053d58 req-0b3a1131-10c5-4e98-a3f0-76f107d6b2c8 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.036137 Jul 16 21:26:10.207335 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1889/3777] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:26:10 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 16 21:26:10.391787 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-3e4102cf-1834-4481-9fe9-7bee54cf9948 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:26:10.393022 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-3e4102cf-1834-4481-9fe9-7bee54cf9948 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: c20f3279-68ae-4b04-8b87-cc16e5f73f1e] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:26:10.405224 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3e4102cf-1834-4481-9fe9-7bee54cf9948 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:10.405224 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3e4102cf-1834-4481-9fe9-7bee54cf9948 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:10.405224 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3e4102cf-1834-4481-9fe9-7bee54cf9948 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:10.405224 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3e4102cf-1834-4481-9fe9-7bee54cf9948 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:10.418670 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-3e4102cf-1834-4481-9fe9-7bee54cf9948 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] HTTP exception thrown: Instance c20f3279-68ae-4b04-8b87-cc16e5f73f1e could not be found. Jul 16 21:26:10.420492 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-3e4102cf-1834-4481-9fe9-7bee54cf9948 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Request method failure captured: Jul 16 21:26:10.420492 ncdv-master-1 devstack@n-api.service[138680]: request: GET /compute/v2.1/servers/c20f3279-68ae-4b04-8b87-cc16e5f73f1e HTTP/1.1 Jul 16 21:26:10.420492 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:26:10.420492 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:26:10.420492 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:26:10.420492 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 0 Jul 16 21:26:10.420492 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:26:10.420492 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:26:10.420492 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:26:10.420492 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:26:10.420492 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0yxS9NCWgI1-O55Z_AF16JzmczB_p-s4P30aJMz-9zu9VxcHuIV00tsfB9hOBtHTZ_sI8GPeOOvUrqYZLgOBzCbOMr4ZO506_XIRjMCtHQ0umzvcB9V9iFSNFoiIe_b2jKm-OyHj1IB8YVBCAcwDKANlsCw5UaAoIABK4dLLFjaVpEo Jul 16 21:26:10.420492 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:26:10.420492 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:26:10.420492 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:26:10.420492 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:26:10.420492 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:26:10.420492 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:26:10.420492 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: 1dad823e240149edb30e592380249a13 Jul 16 21:26:10.420492 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-ServersTestJSON-1529284206 Jul 16 21:26:10.420492 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:26:10.420492 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:26:10.420492 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/1dad823e240149edb30e592380249a13"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/1dad823e240149edb30e592380249a13"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_1dad823e240149edb30e592380249a13"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/1dad823e240149edb30e592380249a13"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:26:10.420492 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-ServersTestJSON-1529284206 Jul 16 21:26:10.420492 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: 1dad823e240149edb30e592380249a13 Jul 16 21:26:10.420492 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-ServersTestJSON-1529284206 Jul 16 21:26:10.420492 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-ServersTestJSON-1529284206-project Jul 16 21:26:10.420492 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:26:10.420492 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:26:10.420492 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: 027dcc48e4fd4c36a5004096ad5e43f8 Jul 16 21:26:10.420492 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-ServersTestJSON-1529284206-project Jul 16 21:26:10.420492 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 21:26:10.420492 ncdv-master-1 devstack@n-api.service[138680]: exception: Instance c20f3279-68ae-4b04-8b87-cc16e5f73f1e could not be found. Jul 16 21:26:10.420492 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': 'c20f3279-68ae-4b04-8b87-cc16e5f73f1e'} Jul 16 21:26:10.420492 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:26:10.420492 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:26:10.420492 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:26:10.420492 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:26:10.420492 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:26:10.420492 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:26:10.420492 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:26:10.420492 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:26:10.420492 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:26:10.420492 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:26:10.420492 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance c20f3279-68ae-4b04-8b87-cc16e5f73f1e could not be found. Jul 16 21:26:10.420492 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:26:10.420492 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:26:10.420492 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:26:10.420492 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:26:10.420492 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:26:10.420492 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:26:10.420492 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:26:10.420492 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:26:10.420492 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:26:10.420492 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:26:10.420492 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:26:10.420492 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:26:10.420492 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:26:10.420492 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:26:10.420492 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:26:10.420492 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:26:10.420492 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance c20f3279-68ae-4b04-8b87-cc16e5f73f1e could not be found. Jul 16 21:26:10.420492 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:26:10.420492 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:26:10.420492 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:26:10.420492 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:26:10.423875 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:26:10.423875 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:26:10.423875 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:26:10.423875 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:26:10.423875 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance c20f3279-68ae-4b04-8b87-cc16e5f73f1e could not be found. Jul 16 21:26:10.423875 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:26:10.423875 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-3e4102cf-1834-4481-9fe9-7bee54cf9948 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Returning 404 to user: Instance c20f3279-68ae-4b04-8b87-cc16e5f73f1e could not be found. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:26:10.423875 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-3e4102cf-1834-4481-9fe9-7bee54cf9948 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "GET /compute/v2.1/servers/c20f3279-68ae-4b04-8b87-cc16e5f73f1e" status: 404 len: 111 microversion: 2.1 time: 0.034198 Jul 16 21:26:10.423875 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1889/3778] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:26:10 2022] GET /compute/v2.1/servers/c20f3279-68ae-4b04-8b87-cc16e5f73f1e => generated 111 bytes in 35 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:26:10.432575 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-ec9c57f2-6504-4cb8-b3d0-1003bfec6389 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:26:10.433247 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-ec9c57f2-6504-4cb8-b3d0-1003bfec6389 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: 05358d0e-ed2f-443c-835c-f77cd929289e] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:26:10.440503 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ec9c57f2-6504-4cb8-b3d0-1003bfec6389 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:10.440899 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ec9c57f2-6504-4cb8-b3d0-1003bfec6389 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:10.441722 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ec9c57f2-6504-4cb8-b3d0-1003bfec6389 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:10.442171 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ec9c57f2-6504-4cb8-b3d0-1003bfec6389 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:10.472723 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-ec9c57f2-6504-4cb8-b3d0-1003bfec6389 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: 05358d0e-ed2f-443c-835c-f77cd929289e] Going to try to terminate instance {{(pid=138681) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 16 21:26:10.661634 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_api:SELECT=96,nova_api:DELETE=1,nova_api:UPDATE=3 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:26:10.751491 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-ec9c57f2-6504-4cb8-b3d0-1003bfec6389 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "DELETE /compute/v2.1/servers/05358d0e-ed2f-443c-835c-f77cd929289e" status: 204 len: 0 microversion: 2.1 time: 0.323919 Jul 16 21:26:10.752527 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1890/3779] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:26:10 2022] DELETE /compute/v2.1/servers/05358d0e-ed2f-443c-835c-f77cd929289e => generated 0 bytes in 326 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 16 21:26:10.761787 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-c2902ab7-eb3d-4f97-afc0-a22c674be2d1 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:26:10.762408 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-c2902ab7-eb3d-4f97-afc0-a22c674be2d1 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: 05358d0e-ed2f-443c-835c-f77cd929289e] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:26:10.773051 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c2902ab7-eb3d-4f97-afc0-a22c674be2d1 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:10.773654 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c2902ab7-eb3d-4f97-afc0-a22c674be2d1 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:10.774281 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c2902ab7-eb3d-4f97-afc0-a22c674be2d1 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:10.774728 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c2902ab7-eb3d-4f97-afc0-a22c674be2d1 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:10.931932 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-c2902ab7-eb3d-4f97-afc0-a22c674be2d1 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '027dcc48e4fd4c36a5004096ad5e43f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dad823e240149edb30e592380249a13', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:26:10.944798 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-c2902ab7-eb3d-4f97-afc0-a22c674be2d1 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "GET /compute/v2.1/servers/05358d0e-ed2f-443c-835c-f77cd929289e" status: 200 len: 1496 microversion: 2.1 time: 0.186990 Jul 16 21:26:10.945379 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1890/3780] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:26:10 2022] GET /compute/v2.1/servers/05358d0e-ed2f-443c-835c-f77cd929289e => generated 1496 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:26:11.153297 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-51fa2a28-76e0-4d07-b457-1b8fdf366e64 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:26:11.154005 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-51fa2a28-76e0-4d07-b457-1b8fdf366e64 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 00c8a753-5bf0-448c-8107-3946aa9bf7ae] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:26:11.159941 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-51fa2a28-76e0-4d07-b457-1b8fdf366e64 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:11.160390 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-51fa2a28-76e0-4d07-b457-1b8fdf366e64 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:11.161651 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-51fa2a28-76e0-4d07-b457-1b8fdf366e64 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:11.162350 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-51fa2a28-76e0-4d07-b457-1b8fdf366e64 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:11.354656 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-51fa2a28-76e0-4d07-b457-1b8fdf366e64 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f2e974ab5aa48dfb4b00f810dcedb63', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd31be8475a6a49d6bfe8132dd050c6c0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:26:11.371842 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-51fa2a28-76e0-4d07-b457-1b8fdf366e64 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "GET /compute/v2.1/servers/00c8a753-5bf0-448c-8107-3946aa9bf7ae" status: 200 len: 1494 microversion: 2.1 time: 0.222585 Jul 16 21:26:11.372463 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1891/3781] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:26:11 2022] GET /compute/v2.1/servers/00c8a753-5bf0-448c-8107-3946aa9bf7ae => generated 1494 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:26:11.963288 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-be313599-4da9-473c-b9e5-7dca7a343f18 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:26:11.963905 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-be313599-4da9-473c-b9e5-7dca7a343f18 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: 05358d0e-ed2f-443c-835c-f77cd929289e] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:26:11.971515 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-be313599-4da9-473c-b9e5-7dca7a343f18 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:11.972100 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-be313599-4da9-473c-b9e5-7dca7a343f18 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:11.972951 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-be313599-4da9-473c-b9e5-7dca7a343f18 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:11.973524 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-be313599-4da9-473c-b9e5-7dca7a343f18 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:12.130869 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-be313599-4da9-473c-b9e5-7dca7a343f18 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '027dcc48e4fd4c36a5004096ad5e43f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dad823e240149edb30e592380249a13', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:26:12.138870 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-be313599-4da9-473c-b9e5-7dca7a343f18 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "GET /compute/v2.1/servers/05358d0e-ed2f-443c-835c-f77cd929289e" status: 200 len: 1496 microversion: 2.1 time: 0.180741 Jul 16 21:26:12.139491 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1891/3782] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:26:11 2022] GET /compute/v2.1/servers/05358d0e-ed2f-443c-835c-f77cd929289e => generated 1496 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:26:12.389892 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-d439b274-ed3d-451c-9ffc-ae70fd549634 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:26:12.391112 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-d439b274-ed3d-451c-9ffc-ae70fd549634 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 00c8a753-5bf0-448c-8107-3946aa9bf7ae] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:26:12.399566 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d439b274-ed3d-451c-9ffc-ae70fd549634 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:12.400027 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d439b274-ed3d-451c-9ffc-ae70fd549634 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:12.400811 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d439b274-ed3d-451c-9ffc-ae70fd549634 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:12.401258 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d439b274-ed3d-451c-9ffc-ae70fd549634 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:12.780905 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-d439b274-ed3d-451c-9ffc-ae70fd549634 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f2e974ab5aa48dfb4b00f810dcedb63', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd31be8475a6a49d6bfe8132dd050c6c0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:26:12.793992 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-d439b274-ed3d-451c-9ffc-ae70fd549634 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "GET /compute/v2.1/servers/00c8a753-5bf0-448c-8107-3946aa9bf7ae" status: 200 len: 1494 microversion: 2.1 time: 0.408262 Jul 16 21:26:12.794845 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1892/3783] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:26:12 2022] GET /compute/v2.1/servers/00c8a753-5bf0-448c-8107-3946aa9bf7ae => generated 1494 bytes in 410 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:26:13.161639 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-2d60e40f-84a4-4036-b50a-e4063a0cb4b1 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:26:13.162416 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-2d60e40f-84a4-4036-b50a-e4063a0cb4b1 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: 05358d0e-ed2f-443c-835c-f77cd929289e] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:26:13.178811 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2d60e40f-84a4-4036-b50a-e4063a0cb4b1 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:13.180222 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2d60e40f-84a4-4036-b50a-e4063a0cb4b1 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:13.182893 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2d60e40f-84a4-4036-b50a-e4063a0cb4b1 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:13.184074 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2d60e40f-84a4-4036-b50a-e4063a0cb4b1 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:13.214086 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=100 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:26:13.812395 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-68c7ef67-9174-40b8-bd22-23dab238db41 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:26:13.813929 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-68c7ef67-9174-40b8-bd22-23dab238db41 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 00c8a753-5bf0-448c-8107-3946aa9bf7ae] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:26:13.842346 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-68c7ef67-9174-40b8-bd22-23dab238db41 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:13.842809 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-68c7ef67-9174-40b8-bd22-23dab238db41 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:13.843618 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-68c7ef67-9174-40b8-bd22-23dab238db41 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:13.844062 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-68c7ef67-9174-40b8-bd22-23dab238db41 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:14.005365 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-2d60e40f-84a4-4036-b50a-e4063a0cb4b1 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '027dcc48e4fd4c36a5004096ad5e43f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dad823e240149edb30e592380249a13', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:26:14.042839 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-2d60e40f-84a4-4036-b50a-e4063a0cb4b1 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "GET /compute/v2.1/servers/05358d0e-ed2f-443c-835c-f77cd929289e" status: 200 len: 1496 microversion: 2.1 time: 0.885172 Jul 16 21:26:14.058208 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1892/3784] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:26:13 2022] GET /compute/v2.1/servers/05358d0e-ed2f-443c-835c-f77cd929289e => generated 1496 bytes in 891 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:26:14.273415 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-68c7ef67-9174-40b8-bd22-23dab238db41 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f2e974ab5aa48dfb4b00f810dcedb63', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd31be8475a6a49d6bfe8132dd050c6c0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:26:14.294936 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-68c7ef67-9174-40b8-bd22-23dab238db41 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "GET /compute/v2.1/servers/00c8a753-5bf0-448c-8107-3946aa9bf7ae" status: 200 len: 1494 microversion: 2.1 time: 0.476141 Jul 16 21:26:14.295508 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1893/3785] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:26:13 2022] GET /compute/v2.1/servers/00c8a753-5bf0-448c-8107-3946aa9bf7ae => generated 1494 bytes in 488 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:26:14.444535 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-5fdf15f4-720a-4b95-8db0-1e04412c3852 req-8c062889-3479-4404-ae89-99dbdad36d77 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "00c8a753-5bf0-448c-8107-3946aa9bf7ae", "name": "network-vif-deleted", "tag": "bb33931c-0e67-4fab-ae33-70ed7930c4be"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:26:14.458190 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-5fdf15f4-720a-4b95-8db0-1e04412c3852 req-8c062889-3479-4404-ae89-99dbdad36d77 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:14.459242 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-5fdf15f4-720a-4b95-8db0-1e04412c3852 req-8c062889-3479-4404-ae89-99dbdad36d77 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:14.482330 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-5fdf15f4-720a-4b95-8db0-1e04412c3852 req-8c062889-3479-4404-ae89-99dbdad36d77 service nova] Creating event network-vif-deleted:bb33931c-0e67-4fab-ae33-70ed7930c4be for instance 00c8a753-5bf0-448c-8107-3946aa9bf7ae on nch2-master-1 Jul 16 21:26:14.488490 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-5fdf15f4-720a-4b95-8db0-1e04412c3852 req-8c062889-3479-4404-ae89-99dbdad36d77 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.049238 Jul 16 21:26:14.490883 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1893/3786] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:26:14 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:26:15.062311 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-4143f30f-4af4-490e-bf19-1172c0308788 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:26:15.062995 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-4143f30f-4af4-490e-bf19-1172c0308788 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: 05358d0e-ed2f-443c-835c-f77cd929289e] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:26:15.072289 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4143f30f-4af4-490e-bf19-1172c0308788 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:15.072785 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4143f30f-4af4-490e-bf19-1172c0308788 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:15.073661 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4143f30f-4af4-490e-bf19-1172c0308788 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:15.074191 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4143f30f-4af4-490e-bf19-1172c0308788 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:15.157710 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-4143f30f-4af4-490e-bf19-1172c0308788 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '027dcc48e4fd4c36a5004096ad5e43f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dad823e240149edb30e592380249a13', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:26:15.170032 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-4143f30f-4af4-490e-bf19-1172c0308788 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "GET /compute/v2.1/servers/05358d0e-ed2f-443c-835c-f77cd929289e" status: 200 len: 1454 microversion: 2.1 time: 0.111895 Jul 16 21:26:15.172096 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1894/3787] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:26:15 2022] GET /compute/v2.1/servers/05358d0e-ed2f-443c-835c-f77cd929289e => generated 1454 bytes in 114 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:26:15.303753 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-00c97907-9d60-4d38-af25-5da6305f933c tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:26:15.304516 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-00c97907-9d60-4d38-af25-5da6305f933c tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 00c8a753-5bf0-448c-8107-3946aa9bf7ae] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:26:15.311889 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-00c97907-9d60-4d38-af25-5da6305f933c tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:15.312431 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-00c97907-9d60-4d38-af25-5da6305f933c tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:15.313733 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-00c97907-9d60-4d38-af25-5da6305f933c tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:15.314255 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-00c97907-9d60-4d38-af25-5da6305f933c tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:15.369033 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-00c97907-9d60-4d38-af25-5da6305f933c tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:15.370198 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-00c97907-9d60-4d38-af25-5da6305f933c tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:15.370725 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.objects.instance [None req-00c97907-9d60-4d38-af25-5da6305f933c tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lazy-loading 'fault' on Instance uuid 00c8a753-5bf0-448c-8107-3946aa9bf7ae {{(pid=138680) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:26:15.469901 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-00c97907-9d60-4d38-af25-5da6305f933c tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f2e974ab5aa48dfb4b00f810dcedb63', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd31be8475a6a49d6bfe8132dd050c6c0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:26:15.481889 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-00c97907-9d60-4d38-af25-5da6305f933c tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "GET /compute/v2.1/servers/00c8a753-5bf0-448c-8107-3946aa9bf7ae" status: 200 len: 1304 microversion: 2.1 time: 0.182755 Jul 16 21:26:15.482973 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1894/3788] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:26:15 2022] GET /compute/v2.1/servers/00c8a753-5bf0-448c-8107-3946aa9bf7ae => generated 1304 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:26:16.015413 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-0262e6c6-2897-4d69-a6af-32953493fb4a tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:26:16.015913 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-0262e6c6-2897-4d69-a6af-32953493fb4a tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] [instance: 2fad683a-f21f-4a9c-8323-e9380ef7f9ef] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:26:16.022432 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0262e6c6-2897-4d69-a6af-32953493fb4a tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:16.023328 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0262e6c6-2897-4d69-a6af-32953493fb4a tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:16.024106 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0262e6c6-2897-4d69-a6af-32953493fb4a tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:16.024546 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0262e6c6-2897-4d69-a6af-32953493fb4a tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:16.038822 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-0262e6c6-2897-4d69-a6af-32953493fb4a tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] HTTP exception thrown: Instance 2fad683a-f21f-4a9c-8323-e9380ef7f9ef could not be found. Jul 16 21:26:16.041039 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-0262e6c6-2897-4d69-a6af-32953493fb4a tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Request method failure captured: Jul 16 21:26:16.041039 ncdv-master-1 devstack@n-api.service[138681]: request: DELETE /compute/v2.1/servers/2fad683a-f21f-4a9c-8323-e9380ef7f9ef HTTP/1.1 Jul 16 21:26:16.041039 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:26:16.041039 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:26:16.041039 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:26:16.041039 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 21:26:16.041039 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:26:16.041039 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:26:16.041039 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:26:16.041039 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:26:16.041039 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0yu6GvklgrJz64l4jI_9XrerhdTTPfYTDqcboWClQTVUOsFR3X72A-yD-xnMKndBGdnSSARa9ZpL_5E_A7dDhsSNRAFKwYpwFmXgR867L_9gOcd2pdrdnzmP_tHWcFxJFbkFQsl_MPEJMXgACkcSj5Ps4CNmUG4pTC9v8iTd4ac5zTs Jul 16 21:26:16.041039 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:26:16.041039 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:26:16.041039 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:26:16.041039 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:26:16.041039 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:26:16.041039 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:26:16.041039 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: 22ca92a86e6c40d59071e58ce4085568 Jul 16 21:26:16.041039 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-AttachVolumeTestJSON-1168541016 Jul 16 21:26:16.041039 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:26:16.041039 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:26:16.041039 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/22ca92a86e6c40d59071e58ce4085568"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/22ca92a86e6c40d59071e58ce4085568"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_22ca92a86e6c40d59071e58ce4085568"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/22ca92a86e6c40d59071e58ce4085568"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:26:16.041039 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-AttachVolumeTestJSON-1168541016 Jul 16 21:26:16.041039 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: 22ca92a86e6c40d59071e58ce4085568 Jul 16 21:26:16.041039 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-AttachVolumeTestJSON-1168541016 Jul 16 21:26:16.041039 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-AttachVolumeTestJSON-1168541016-project Jul 16 21:26:16.041039 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:26:16.041039 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:26:16.041039 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: bba77f6ef42547f1aa8480e27868bd4a Jul 16 21:26:16.041039 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-AttachVolumeTestJSON-1168541016-project Jul 16 21:26:16.041039 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:26:16.041039 ncdv-master-1 devstack@n-api.service[138681]: exception: Instance 2fad683a-f21f-4a9c-8323-e9380ef7f9ef could not be found. Jul 16 21:26:16.041039 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': '2fad683a-f21f-4a9c-8323-e9380ef7f9ef'} Jul 16 21:26:16.041039 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:26:16.041039 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:26:16.041039 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:26:16.041039 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:26:16.041039 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:26:16.041039 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:26:16.041039 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:26:16.041039 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:26:16.041039 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:26:16.041039 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:26:16.041039 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 2fad683a-f21f-4a9c-8323-e9380ef7f9ef could not be found. Jul 16 21:26:16.041039 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:26:16.041039 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:26:16.041039 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:26:16.041039 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:26:16.041039 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:26:16.041039 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:26:16.041039 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:26:16.041039 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:26:16.041039 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:26:16.041039 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:26:16.041039 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Jul 16 21:26:16.041039 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Jul 16 21:26:16.041039 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Jul 16 21:26:16.041039 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Jul 16 21:26:16.041039 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:26:16.041039 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:26:16.041039 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:26:16.041039 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:26:16.049922 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 2fad683a-f21f-4a9c-8323-e9380ef7f9ef could not be found. Jul 16 21:26:16.049922 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:26:16.049922 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:26:16.049922 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:26:16.049922 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:26:16.049922 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:26:16.049922 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:26:16.049922 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:26:16.049922 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:26:16.049922 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 2fad683a-f21f-4a9c-8323-e9380ef7f9ef could not be found. Jul 16 21:26:16.049922 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:26:16.049922 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-0262e6c6-2897-4d69-a6af-32953493fb4a tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Returning 404 to user: Instance 2fad683a-f21f-4a9c-8323-e9380ef7f9ef could not be found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:26:16.049922 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-0262e6c6-2897-4d69-a6af-32953493fb4a tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] 10.222.0.16 "DELETE /compute/v2.1/servers/2fad683a-f21f-4a9c-8323-e9380ef7f9ef" status: 404 len: 111 microversion: 2.1 time: 0.031277 Jul 16 21:26:16.049922 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1895/3789] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:26:16 2022] DELETE /compute/v2.1/servers/2fad683a-f21f-4a9c-8323-e9380ef7f9ef => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:26:16.057444 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-f6fcdb65-e219-4d2a-8651-6785f5b561d4 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:26:16.058149 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-f6fcdb65-e219-4d2a-8651-6785f5b561d4 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] [instance: 2fad683a-f21f-4a9c-8323-e9380ef7f9ef] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:26:16.064631 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f6fcdb65-e219-4d2a-8651-6785f5b561d4 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:16.065256 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f6fcdb65-e219-4d2a-8651-6785f5b561d4 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:16.066161 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f6fcdb65-e219-4d2a-8651-6785f5b561d4 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:16.066610 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f6fcdb65-e219-4d2a-8651-6785f5b561d4 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:16.084413 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-f6fcdb65-e219-4d2a-8651-6785f5b561d4 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] HTTP exception thrown: Instance 2fad683a-f21f-4a9c-8323-e9380ef7f9ef could not be found. Jul 16 21:26:16.086426 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-f6fcdb65-e219-4d2a-8651-6785f5b561d4 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Request method failure captured: Jul 16 21:26:16.086426 ncdv-master-1 devstack@n-api.service[138680]: request: GET /compute/v2.1/servers/2fad683a-f21f-4a9c-8323-e9380ef7f9ef HTTP/1.1 Jul 16 21:26:16.086426 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:26:16.086426 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:26:16.086426 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:26:16.086426 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 0 Jul 16 21:26:16.086426 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:26:16.086426 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:26:16.086426 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:26:16.086426 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:26:16.086426 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0yu6GvklgrJz64l4jI_9XrerhdTTPfYTDqcboWClQTVUOsFR3X72A-yD-xnMKndBGdnSSARa9ZpL_5E_A7dDhsSNRAFKwYpwFmXgR867L_9gOcd2pdrdnzmP_tHWcFxJFbkFQsl_MPEJMXgACkcSj5Ps4CNmUG4pTC9v8iTd4ac5zTs Jul 16 21:26:16.086426 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:26:16.086426 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:26:16.086426 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:26:16.086426 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:26:16.086426 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:26:16.086426 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:26:16.086426 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: 22ca92a86e6c40d59071e58ce4085568 Jul 16 21:26:16.086426 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-AttachVolumeTestJSON-1168541016 Jul 16 21:26:16.086426 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:26:16.086426 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:26:16.086426 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/22ca92a86e6c40d59071e58ce4085568"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/22ca92a86e6c40d59071e58ce4085568"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_22ca92a86e6c40d59071e58ce4085568"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/22ca92a86e6c40d59071e58ce4085568"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:26:16.086426 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-AttachVolumeTestJSON-1168541016 Jul 16 21:26:16.086426 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: 22ca92a86e6c40d59071e58ce4085568 Jul 16 21:26:16.086426 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-AttachVolumeTestJSON-1168541016 Jul 16 21:26:16.086426 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-AttachVolumeTestJSON-1168541016-project Jul 16 21:26:16.086426 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:26:16.086426 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:26:16.086426 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: bba77f6ef42547f1aa8480e27868bd4a Jul 16 21:26:16.086426 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-AttachVolumeTestJSON-1168541016-project Jul 16 21:26:16.086426 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 21:26:16.086426 ncdv-master-1 devstack@n-api.service[138680]: exception: Instance 2fad683a-f21f-4a9c-8323-e9380ef7f9ef could not be found. Jul 16 21:26:16.086426 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': '2fad683a-f21f-4a9c-8323-e9380ef7f9ef'} Jul 16 21:26:16.086426 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:26:16.086426 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:26:16.086426 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:26:16.086426 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:26:16.086426 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:26:16.086426 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:26:16.086426 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:26:16.086426 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:26:16.086426 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:26:16.086426 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:26:16.086426 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 2fad683a-f21f-4a9c-8323-e9380ef7f9ef could not be found. Jul 16 21:26:16.086426 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:26:16.086426 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:26:16.086426 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:26:16.086426 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:26:16.086426 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:26:16.086426 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:26:16.086426 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:26:16.086426 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:26:16.086426 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:26:16.086426 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:26:16.086426 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:26:16.086426 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:26:16.086426 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:26:16.086426 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:26:16.086426 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:26:16.086426 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:26:16.086426 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 2fad683a-f21f-4a9c-8323-e9380ef7f9ef could not be found. Jul 16 21:26:16.086426 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:26:16.086426 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:26:16.086426 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:26:16.093362 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:26:16.093362 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:26:16.093362 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:26:16.093362 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:26:16.093362 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:26:16.093362 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 2fad683a-f21f-4a9c-8323-e9380ef7f9ef could not be found. Jul 16 21:26:16.093362 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:26:16.093362 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-f6fcdb65-e219-4d2a-8651-6785f5b561d4 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Returning 404 to user: Instance 2fad683a-f21f-4a9c-8323-e9380ef7f9ef could not be found. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:26:16.093362 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-f6fcdb65-e219-4d2a-8651-6785f5b561d4 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] 10.222.0.16 "GET /compute/v2.1/servers/2fad683a-f21f-4a9c-8323-e9380ef7f9ef" status: 404 len: 111 microversion: 2.1 time: 0.040282 Jul 16 21:26:16.093362 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1895/3790] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:26:16 2022] GET /compute/v2.1/servers/2fad683a-f21f-4a9c-8323-e9380ef7f9ef => generated 111 bytes in 42 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:26:16.188694 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-a66cbded-f654-49c5-b935-d9b1086a9a1d tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:26:16.189220 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-a66cbded-f654-49c5-b935-d9b1086a9a1d tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: 05358d0e-ed2f-443c-835c-f77cd929289e] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:26:16.201691 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a66cbded-f654-49c5-b935-d9b1086a9a1d tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:16.202628 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a66cbded-f654-49c5-b935-d9b1086a9a1d tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:16.203854 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a66cbded-f654-49c5-b935-d9b1086a9a1d tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:16.204921 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a66cbded-f654-49c5-b935-d9b1086a9a1d tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:16.257234 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a66cbded-f654-49c5-b935-d9b1086a9a1d tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:16.257864 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a66cbded-f654-49c5-b935-d9b1086a9a1d tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:16.258399 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.objects.instance [None req-a66cbded-f654-49c5-b935-d9b1086a9a1d tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lazy-loading 'fault' on Instance uuid 05358d0e-ed2f-443c-835c-f77cd929289e {{(pid=138681) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:26:16.371589 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-a66cbded-f654-49c5-b935-d9b1086a9a1d tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '027dcc48e4fd4c36a5004096ad5e43f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dad823e240149edb30e592380249a13', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:26:16.385750 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-a66cbded-f654-49c5-b935-d9b1086a9a1d tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "GET /compute/v2.1/servers/05358d0e-ed2f-443c-835c-f77cd929289e" status: 200 len: 1299 microversion: 2.1 time: 0.201001 Jul 16 21:26:16.393474 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1896/3791] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:26:16 2022] GET /compute/v2.1/servers/05358d0e-ed2f-443c-835c-f77cd929289e => generated 1299 bytes in 209 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:26:16.502909 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-562729c9-fea2-4d4b-85c9-b87eaed15a4c tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:26:16.503900 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-562729c9-fea2-4d4b-85c9-b87eaed15a4c tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 00c8a753-5bf0-448c-8107-3946aa9bf7ae] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:26:16.515329 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-562729c9-fea2-4d4b-85c9-b87eaed15a4c tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:16.522124 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-562729c9-fea2-4d4b-85c9-b87eaed15a4c tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:16.525055 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-562729c9-fea2-4d4b-85c9-b87eaed15a4c tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:16.525649 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-562729c9-fea2-4d4b-85c9-b87eaed15a4c tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:16.533748 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-70c29443-da23-45bf-b345-6ed1be619ae0 req-b3d228a8-58f2-4c79-88bb-8e38fba73cfc service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "05358d0e-ed2f-443c-835c-f77cd929289e", "name": "network-vif-deleted", "tag": "9675fc80-e092-4333-8050-341c379735a0"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:26:16.565557 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-70c29443-da23-45bf-b345-6ed1be619ae0 req-b3d228a8-58f2-4c79-88bb-8e38fba73cfc service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:16.566635 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-70c29443-da23-45bf-b345-6ed1be619ae0 req-b3d228a8-58f2-4c79-88bb-8e38fba73cfc service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:16.569407 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-562729c9-fea2-4d4b-85c9-b87eaed15a4c tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] HTTP exception thrown: Instance 00c8a753-5bf0-448c-8107-3946aa9bf7ae could not be found. Jul 16 21:26:16.572203 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-562729c9-fea2-4d4b-85c9-b87eaed15a4c tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Request method failure captured: Jul 16 21:26:16.572203 ncdv-master-1 devstack@n-api.service[138680]: request: GET /compute/v2.1/servers/00c8a753-5bf0-448c-8107-3946aa9bf7ae HTTP/1.1 Jul 16 21:26:16.572203 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:26:16.572203 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:26:16.572203 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:26:16.572203 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 0 Jul 16 21:26:16.572203 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:26:16.572203 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:26:16.572203 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:26:16.572203 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:26:16.572203 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0yvs4CI48W3ZCHoQ8huRZwHvqHxAdwrwrvk1v_ETBPx6FU4DdiPUjkjq4hN_4g_fHLAB_XZ9-AAN8Pyvy0KON2X25WlmyilO65MDHbIW9Xy9v9USVPAu_Ce-ze5X9UIHyned0AUneE4UPGiLLIkoXeoS5ekiq1pi209pud2lyUAuHgw Jul 16 21:26:16.572203 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:26:16.572203 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:26:16.572203 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:26:16.572203 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:26:16.572203 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:26:16.572203 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:26:16.572203 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: d31be8475a6a49d6bfe8132dd050c6c0 Jul 16 21:26:16.572203 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-DeleteServersTestJSON-1541892181 Jul 16 21:26:16.572203 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:26:16.572203 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:26:16.572203 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/d31be8475a6a49d6bfe8132dd050c6c0"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/d31be8475a6a49d6bfe8132dd050c6c0"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_d31be8475a6a49d6bfe8132dd050c6c0"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/d31be8475a6a49d6bfe8132dd050c6c0"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:26:16.572203 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-DeleteServersTestJSON-1541892181 Jul 16 21:26:16.572203 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: d31be8475a6a49d6bfe8132dd050c6c0 Jul 16 21:26:16.572203 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-DeleteServersTestJSON-1541892181 Jul 16 21:26:16.572203 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-DeleteServersTestJSON-1541892181-project Jul 16 21:26:16.572203 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:26:16.572203 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:26:16.572203 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: 6f2e974ab5aa48dfb4b00f810dcedb63 Jul 16 21:26:16.572203 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-DeleteServersTestJSON-1541892181-project Jul 16 21:26:16.572203 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 21:26:16.572203 ncdv-master-1 devstack@n-api.service[138680]: exception: Instance 00c8a753-5bf0-448c-8107-3946aa9bf7ae could not be found. Jul 16 21:26:16.572203 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': '00c8a753-5bf0-448c-8107-3946aa9bf7ae'} Jul 16 21:26:16.572203 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:26:16.572203 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:26:16.572203 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:26:16.572203 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:26:16.572203 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:26:16.572203 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:26:16.572203 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:26:16.572203 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:26:16.572203 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:26:16.572203 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:26:16.572203 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 00c8a753-5bf0-448c-8107-3946aa9bf7ae could not be found. Jul 16 21:26:16.572203 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:26:16.572203 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:26:16.572203 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:26:16.572203 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:26:16.572203 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:26:16.572203 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:26:16.572203 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:26:16.572203 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:26:16.572203 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:26:16.572203 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:26:16.572203 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:26:16.572203 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:26:16.572203 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:26:16.572203 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:26:16.572203 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:26:16.572203 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:26:16.572203 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 00c8a753-5bf0-448c-8107-3946aa9bf7ae could not be found. Jul 16 21:26:16.572203 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:26:16.572203 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:26:16.572203 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:26:16.575691 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:26:16.575691 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:26:16.575691 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:26:16.575691 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:26:16.575691 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:26:16.575691 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 00c8a753-5bf0-448c-8107-3946aa9bf7ae could not be found. Jul 16 21:26:16.575691 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:26:16.580835 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-562729c9-fea2-4d4b-85c9-b87eaed15a4c tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Returning 404 to user: Instance 00c8a753-5bf0-448c-8107-3946aa9bf7ae could not be found. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:26:16.583480 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-562729c9-fea2-4d4b-85c9-b87eaed15a4c tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "GET /compute/v2.1/servers/00c8a753-5bf0-448c-8107-3946aa9bf7ae" status: 404 len: 111 microversion: 2.1 time: 0.083584 Jul 16 21:26:16.588192 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-70c29443-da23-45bf-b345-6ed1be619ae0 req-b3d228a8-58f2-4c79-88bb-8e38fba73cfc service nova] Creating event network-vif-deleted:9675fc80-e092-4333-8050-341c379735a0 for instance 05358d0e-ed2f-443c-835c-f77cd929289e on nch2-master-1 Jul 16 21:26:16.590589 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1896/3792] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:26:16 2022] GET /compute/v2.1/servers/00c8a753-5bf0-448c-8107-3946aa9bf7ae => generated 111 bytes in 91 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:26:16.595908 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-70c29443-da23-45bf-b345-6ed1be619ae0 req-b3d228a8-58f2-4c79-88bb-8e38fba73cfc service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.069233 Jul 16 21:26:16.595908 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1897/3793] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:26:16 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 72 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 16 21:26:16.781848 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-662ddbd9-9ca3-44ff-bf75-7eecede38e9a tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-DeleteServersTestJSON-server-707543060", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452", "networks": [{"uuid": "fc3b835d-0a7f-4948-9fcb-1edfd6b9a8df"}]}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:26:16.905033 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.network.neutron [None req-662ddbd9-9ca3-44ff-bf75-7eecede38e9a tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] validate_networks() for [('fc3b835d-0a7f-4948-9fcb-1edfd6b9a8df', None, None, None, None, None)] {{(pid=138680) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 16 21:26:17.412214 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-45a8b5f9-1815-4f8d-9009-538ba77e0deb tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:26:17.412925 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-45a8b5f9-1815-4f8d-9009-538ba77e0deb tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: 05358d0e-ed2f-443c-835c-f77cd929289e] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:26:17.423252 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-45a8b5f9-1815-4f8d-9009-538ba77e0deb tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:17.423710 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-45a8b5f9-1815-4f8d-9009-538ba77e0deb tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:17.424700 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-45a8b5f9-1815-4f8d-9009-538ba77e0deb tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:17.425353 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-45a8b5f9-1815-4f8d-9009-538ba77e0deb tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:17.443320 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-45a8b5f9-1815-4f8d-9009-538ba77e0deb tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] HTTP exception thrown: Instance 05358d0e-ed2f-443c-835c-f77cd929289e could not be found. Jul 16 21:26:17.445225 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-45a8b5f9-1815-4f8d-9009-538ba77e0deb tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Request method failure captured: Jul 16 21:26:17.445225 ncdv-master-1 devstack@n-api.service[138681]: request: GET /compute/v2.1/servers/05358d0e-ed2f-443c-835c-f77cd929289e HTTP/1.1 Jul 16 21:26:17.445225 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:26:17.445225 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:26:17.445225 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:26:17.445225 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 21:26:17.445225 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:26:17.445225 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:26:17.445225 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:26:17.445225 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:26:17.445225 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0yxS9NCWgI1-O55Z_AF16JzmczB_p-s4P30aJMz-9zu9VxcHuIV00tsfB9hOBtHTZ_sI8GPeOOvUrqYZLgOBzCbOMr4ZO506_XIRjMCtHQ0umzvcB9V9iFSNFoiIe_b2jKm-OyHj1IB8YVBCAcwDKANlsCw5UaAoIABK4dLLFjaVpEo Jul 16 21:26:17.445225 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:26:17.445225 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:26:17.445225 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:26:17.445225 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:26:17.445225 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:26:17.445225 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:26:17.445225 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: 1dad823e240149edb30e592380249a13 Jul 16 21:26:17.445225 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-ServersTestJSON-1529284206 Jul 16 21:26:17.445225 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:26:17.445225 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:26:17.445225 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/1dad823e240149edb30e592380249a13"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/1dad823e240149edb30e592380249a13"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_1dad823e240149edb30e592380249a13"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/1dad823e240149edb30e592380249a13"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:26:17.445225 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-ServersTestJSON-1529284206 Jul 16 21:26:17.445225 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: 1dad823e240149edb30e592380249a13 Jul 16 21:26:17.445225 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-ServersTestJSON-1529284206 Jul 16 21:26:17.445225 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-ServersTestJSON-1529284206-project Jul 16 21:26:17.445225 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:26:17.445225 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:26:17.445225 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: 027dcc48e4fd4c36a5004096ad5e43f8 Jul 16 21:26:17.445225 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-ServersTestJSON-1529284206-project Jul 16 21:26:17.445225 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:26:17.445225 ncdv-master-1 devstack@n-api.service[138681]: exception: Instance 05358d0e-ed2f-443c-835c-f77cd929289e could not be found. Jul 16 21:26:17.445225 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': '05358d0e-ed2f-443c-835c-f77cd929289e'} Jul 16 21:26:17.445225 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:26:17.445225 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:26:17.445225 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:26:17.445225 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:26:17.445225 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:26:17.445225 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:26:17.445225 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:26:17.445225 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:26:17.445225 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:26:17.445225 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:26:17.445225 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 05358d0e-ed2f-443c-835c-f77cd929289e could not be found. Jul 16 21:26:17.445225 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:26:17.445225 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:26:17.445225 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:26:17.445225 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:26:17.445225 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:26:17.445225 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:26:17.445225 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:26:17.445225 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:26:17.445225 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:26:17.445225 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:26:17.445225 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:26:17.445225 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:26:17.445225 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:26:17.445225 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:26:17.445225 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:26:17.445225 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:26:17.445225 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 05358d0e-ed2f-443c-835c-f77cd929289e could not be found. Jul 16 21:26:17.445225 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:26:17.445225 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:26:17.445225 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:26:17.445225 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:26:17.448464 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:26:17.448464 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:26:17.448464 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:26:17.448464 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:26:17.448464 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 05358d0e-ed2f-443c-835c-f77cd929289e could not be found. Jul 16 21:26:17.448464 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:26:17.450995 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-45a8b5f9-1815-4f8d-9009-538ba77e0deb tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Returning 404 to user: Instance 05358d0e-ed2f-443c-835c-f77cd929289e could not be found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:26:17.452201 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-45a8b5f9-1815-4f8d-9009-538ba77e0deb tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "GET /compute/v2.1/servers/05358d0e-ed2f-443c-835c-f77cd929289e" status: 404 len: 111 microversion: 2.1 time: 0.044984 Jul 16 21:26:17.453145 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1898/3794] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:26:17 2022] GET /compute/v2.1/servers/05358d0e-ed2f-443c-835c-f77cd929289e => generated 111 bytes in 47 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Jul 16 21:26:17.606261 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-9b2bb9b1-a1fd-4a27-86d0-a353bb0c4cee tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServersTestJSON-server-193220897", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452", "networks": [{"uuid": "2202d4ad-313c-4607-b9fc-eb98bac9afd0"}]}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:26:17.686449 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.network.neutron [None req-9b2bb9b1-a1fd-4a27-86d0-a353bb0c4cee tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] validate_networks() for [('2202d4ad-313c-4607-b9fc-eb98bac9afd0', None, None, None, None, None)] {{(pid=138681) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 16 21:26:17.903197 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-662ddbd9-9ca3-44ff-bf75-7eecede38e9a tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Flavor limits 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 16 21:26:17.903197 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-662ddbd9-9ca3-44ff-bf75-7eecede38e9a tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Image limits 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 16 21:26:17.903197 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-662ddbd9-9ca3-44ff-bf75-7eecede38e9a tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Flavor pref 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 16 21:26:17.903197 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-662ddbd9-9ca3-44ff-bf75-7eecede38e9a tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Image pref 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 16 21:26:17.903792 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-662ddbd9-9ca3-44ff-bf75-7eecede38e9a tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 16 21:26:17.946927 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-662ddbd9-9ca3-44ff-bf75-7eecede38e9a tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:17.947749 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-662ddbd9-9ca3-44ff-bf75-7eecede38e9a tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:17.948613 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-662ddbd9-9ca3-44ff-bf75-7eecede38e9a tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:17.951385 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-662ddbd9-9ca3-44ff-bf75-7eecede38e9a tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:17.996119 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-662ddbd9-9ca3-44ff-bf75-7eecede38e9a tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Getting quotas for project d31be8475a6a49d6bfe8132dd050c6c0. Resources: {'ram', 'instances', 'cores'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:26:18.012196 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-662ddbd9-9ca3-44ff-bf75-7eecede38e9a tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Getting quotas for user 6f2e974ab5aa48dfb4b00f810dcedb63 and project d31be8475a6a49d6bfe8132dd050c6c0. Resources: {'ram', 'instances', 'cores'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:26:18.046908 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-662ddbd9-9ca3-44ff-bf75-7eecede38e9a tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Going to run 1 instances... {{(pid=138680) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 16 21:26:18.060259 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-662ddbd9-9ca3-44ff-bf75-7eecede38e9a tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 6ba66466-0da7-4e8d-8ddd-b46f582713fc] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138680) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 16 21:26:18.212414 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-662ddbd9-9ca3-44ff-bf75-7eecede38e9a tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 1.443130 Jul 16 21:26:18.217228 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1897/3795] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 21:26:16 2022] POST /compute/v2.1/servers => generated 380 bytes in 1448 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 16 21:26:18.226393 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-81592659-6d27-462c-8826-734a49f07b5c tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:26:18.241030 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-81592659-6d27-462c-8826-734a49f07b5c tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 6ba66466-0da7-4e8d-8ddd-b46f582713fc] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:26:18.411285 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-81592659-6d27-462c-8826-734a49f07b5c tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f2e974ab5aa48dfb4b00f810dcedb63', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd31be8475a6a49d6bfe8132dd050c6c0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:26:18.419326 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-81592659-6d27-462c-8826-734a49f07b5c tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "GET /compute/v2.1/servers/6ba66466-0da7-4e8d-8ddd-b46f582713fc" status: 200 len: 1214 microversion: 2.1 time: 0.197408 Jul 16 21:26:18.429483 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1898/3796] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:26:18 2022] GET /compute/v2.1/servers/6ba66466-0da7-4e8d-8ddd-b46f582713fc => generated 1214 bytes in 199 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:26:18.878680 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-9b2bb9b1-a1fd-4a27-86d0-a353bb0c4cee tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Flavor limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 16 21:26:18.879672 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-9b2bb9b1-a1fd-4a27-86d0-a353bb0c4cee tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Image limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 16 21:26:18.885875 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-9b2bb9b1-a1fd-4a27-86d0-a353bb0c4cee tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Flavor pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 16 21:26:18.885875 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-9b2bb9b1-a1fd-4a27-86d0-a353bb0c4cee tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Image pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 16 21:26:18.885875 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-9b2bb9b1-a1fd-4a27-86d0-a353bb0c4cee tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 16 21:26:18.900045 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9b2bb9b1-a1fd-4a27-86d0-a353bb0c4cee tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:18.900507 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9b2bb9b1-a1fd-4a27-86d0-a353bb0c4cee tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:18.901468 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9b2bb9b1-a1fd-4a27-86d0-a353bb0c4cee tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:18.901780 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9b2bb9b1-a1fd-4a27-86d0-a353bb0c4cee tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:18.958580 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-9b2bb9b1-a1fd-4a27-86d0-a353bb0c4cee tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Getting quotas for project 1dad823e240149edb30e592380249a13. Resources: {'ram', 'instances', 'cores'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:26:18.965546 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-9b2bb9b1-a1fd-4a27-86d0-a353bb0c4cee tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Getting quotas for user 027dcc48e4fd4c36a5004096ad5e43f8 and project 1dad823e240149edb30e592380249a13. Resources: {'ram', 'instances', 'cores'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:26:18.981109 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-9b2bb9b1-a1fd-4a27-86d0-a353bb0c4cee tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Going to run 1 instances... {{(pid=138681) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 16 21:26:18.983653 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-9b2bb9b1-a1fd-4a27-86d0-a353bb0c4cee tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: 6dcb4587-0e15-4a2d-b8f0-70b459fb6664] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138681) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 16 21:26:19.143901 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-9b2bb9b1-a1fd-4a27-86d0-a353bb0c4cee tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 1.537703 Jul 16 21:26:19.146134 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1899/3797] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 21:26:17 2022] POST /compute/v2.1/servers => generated 380 bytes in 1545 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 16 21:26:19.156934 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-5971d028-1d78-4900-8474-a8cbc14b1e41 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:26:19.157577 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-5971d028-1d78-4900-8474-a8cbc14b1e41 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: 6dcb4587-0e15-4a2d-b8f0-70b459fb6664] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:26:19.280140 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-5971d028-1d78-4900-8474-a8cbc14b1e41 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '027dcc48e4fd4c36a5004096ad5e43f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dad823e240149edb30e592380249a13', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:26:19.292014 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-5971d028-1d78-4900-8474-a8cbc14b1e41 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "GET /compute/v2.1/servers/6dcb4587-0e15-4a2d-b8f0-70b459fb6664" status: 200 len: 1208 microversion: 2.1 time: 0.139708 Jul 16 21:26:19.292571 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1899/3798] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:26:19 2022] GET /compute/v2.1/servers/6dcb4587-0e15-4a2d-b8f0-70b459fb6664 => generated 1208 bytes in 141 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:26:19.459863 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-750c6fa0-6c66-44e0-adea-761112796e9e tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:26:19.474930 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-750c6fa0-6c66-44e0-adea-761112796e9e tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 6ba66466-0da7-4e8d-8ddd-b46f582713fc] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:26:19.925689 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-750c6fa0-6c66-44e0-adea-761112796e9e tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f2e974ab5aa48dfb4b00f810dcedb63', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd31be8475a6a49d6bfe8132dd050c6c0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:26:19.942844 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-750c6fa0-6c66-44e0-adea-761112796e9e tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "GET /compute/v2.1/servers/6ba66466-0da7-4e8d-8ddd-b46f582713fc" status: 200 len: 1214 microversion: 2.1 time: 0.505393 Jul 16 21:26:19.951064 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1900/3799] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:26:19 2022] GET /compute/v2.1/servers/6ba66466-0da7-4e8d-8ddd-b46f582713fc => generated 1214 bytes in 511 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:26:20.309965 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-b07ed3ac-9abd-4560-bfdd-0c04464b29f1 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:26:20.310691 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-b07ed3ac-9abd-4560-bfdd-0c04464b29f1 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: 6dcb4587-0e15-4a2d-b8f0-70b459fb6664] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:26:20.387278 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-b07ed3ac-9abd-4560-bfdd-0c04464b29f1 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '027dcc48e4fd4c36a5004096ad5e43f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dad823e240149edb30e592380249a13', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:26:20.395659 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-b07ed3ac-9abd-4560-bfdd-0c04464b29f1 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "GET /compute/v2.1/servers/6dcb4587-0e15-4a2d-b8f0-70b459fb6664" status: 200 len: 1208 microversion: 2.1 time: 0.089796 Jul 16 21:26:20.396598 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1901/3800] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:26:20 2022] GET /compute/v2.1/servers/6dcb4587-0e15-4a2d-b8f0-70b459fb6664 => generated 1208 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:26:20.471363 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-39988d9c-e80c-4e05-8cec-5fcf8193749d tempest-InstanceActionsNegativeTestJSON-904555896 tempest-InstanceActionsNegativeTestJSON-904555896-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-InstanceActionsNegativeTestJSON-server-580687670", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452", "networks": [{"uuid": "c883c8ba-56ba-4c22-87b3-83daa5f4b762"}]}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:26:20.531329 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.network.neutron [None req-39988d9c-e80c-4e05-8cec-5fcf8193749d tempest-InstanceActionsNegativeTestJSON-904555896 tempest-InstanceActionsNegativeTestJSON-904555896-project] validate_networks() for [('c883c8ba-56ba-4c22-87b3-83daa5f4b762', None, None, None, None, None)] {{(pid=138680) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 16 21:26:20.717904 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-e283674d-ac67-45db-826c-e493aea56c95 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:26:20.718408 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-e283674d-ac67-45db-826c-e493aea56c95 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] [instance: f114e124-2903-4f30-b055-9f493e3e9e6c] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:26:20.727525 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e283674d-ac67-45db-826c-e493aea56c95 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:20.731478 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e283674d-ac67-45db-826c-e493aea56c95 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:20.732701 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e283674d-ac67-45db-826c-e493aea56c95 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:20.733330 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e283674d-ac67-45db-826c-e493aea56c95 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:20.760380 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-e283674d-ac67-45db-826c-e493aea56c95 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] HTTP exception thrown: Instance f114e124-2903-4f30-b055-9f493e3e9e6c could not be found. Jul 16 21:26:20.762000 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-e283674d-ac67-45db-826c-e493aea56c95 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Request method failure captured: Jul 16 21:26:20.762000 ncdv-master-1 devstack@n-api.service[138681]: request: DELETE /compute/v2.1/servers/f114e124-2903-4f30-b055-9f493e3e9e6c HTTP/1.1 Jul 16 21:26:20.762000 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:26:20.762000 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:26:20.762000 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:26:20.762000 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 21:26:20.762000 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:26:20.762000 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:26:20.762000 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:26:20.762000 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:26:20.762000 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0yu6GvklgrJz64l4jI_9XrerhdTTPfYTDqcboWClQTVUOsFR3X72A-yD-xnMKndBGdnSSARa9ZpL_5E_A7dDhsSNRAFKwYpwFmXgR867L_9gOcd2pdrdnzmP_tHWcFxJFbkFQsl_MPEJMXgACkcSj5Ps4CNmUG4pTC9v8iTd4ac5zTs Jul 16 21:26:20.762000 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:26:20.762000 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:26:20.762000 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:26:20.762000 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:26:20.762000 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:26:20.762000 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:26:20.762000 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: 22ca92a86e6c40d59071e58ce4085568 Jul 16 21:26:20.762000 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-AttachVolumeTestJSON-1168541016 Jul 16 21:26:20.762000 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:26:20.762000 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:26:20.762000 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/22ca92a86e6c40d59071e58ce4085568"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/22ca92a86e6c40d59071e58ce4085568"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_22ca92a86e6c40d59071e58ce4085568"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/22ca92a86e6c40d59071e58ce4085568"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:26:20.762000 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-AttachVolumeTestJSON-1168541016 Jul 16 21:26:20.762000 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: 22ca92a86e6c40d59071e58ce4085568 Jul 16 21:26:20.762000 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-AttachVolumeTestJSON-1168541016 Jul 16 21:26:20.762000 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-AttachVolumeTestJSON-1168541016-project Jul 16 21:26:20.762000 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:26:20.762000 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:26:20.762000 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: bba77f6ef42547f1aa8480e27868bd4a Jul 16 21:26:20.762000 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-AttachVolumeTestJSON-1168541016-project Jul 16 21:26:20.762000 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:26:20.762000 ncdv-master-1 devstack@n-api.service[138681]: exception: Instance f114e124-2903-4f30-b055-9f493e3e9e6c could not be found. Jul 16 21:26:20.762000 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': 'f114e124-2903-4f30-b055-9f493e3e9e6c'} Jul 16 21:26:20.762000 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:26:20.762000 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:26:20.762000 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:26:20.762000 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:26:20.762000 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:26:20.762000 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:26:20.762000 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:26:20.762000 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:26:20.762000 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:26:20.762000 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:26:20.762000 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance f114e124-2903-4f30-b055-9f493e3e9e6c could not be found. Jul 16 21:26:20.762000 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:26:20.762000 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:26:20.762000 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:26:20.762000 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:26:20.762000 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:26:20.762000 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:26:20.762000 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:26:20.762000 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:26:20.762000 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:26:20.762000 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:26:20.762000 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Jul 16 21:26:20.762000 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Jul 16 21:26:20.762000 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Jul 16 21:26:20.762000 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Jul 16 21:26:20.762000 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:26:20.762000 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:26:20.762000 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:26:20.762000 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:26:20.771598 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance f114e124-2903-4f30-b055-9f493e3e9e6c could not be found. Jul 16 21:26:20.771598 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:26:20.771598 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:26:20.771598 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:26:20.771598 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:26:20.771598 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:26:20.771598 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:26:20.771598 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:26:20.771598 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:26:20.771598 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance f114e124-2903-4f30-b055-9f493e3e9e6c could not be found. Jul 16 21:26:20.771598 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:26:20.771598 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-e283674d-ac67-45db-826c-e493aea56c95 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Returning 404 to user: Instance f114e124-2903-4f30-b055-9f493e3e9e6c could not be found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:26:20.771598 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-e283674d-ac67-45db-826c-e493aea56c95 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] 10.222.0.16 "DELETE /compute/v2.1/servers/f114e124-2903-4f30-b055-9f493e3e9e6c" status: 404 len: 111 microversion: 2.1 time: 0.049560 Jul 16 21:26:20.771598 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1902/3801] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:26:20 2022] DELETE /compute/v2.1/servers/f114e124-2903-4f30-b055-9f493e3e9e6c => generated 111 bytes in 51 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Jul 16 21:26:20.774214 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-f75c4581-9722-474b-9e7c-52c4ff6e5ce4 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:26:20.774826 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-f75c4581-9722-474b-9e7c-52c4ff6e5ce4 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] [instance: f114e124-2903-4f30-b055-9f493e3e9e6c] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:26:20.781178 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f75c4581-9722-474b-9e7c-52c4ff6e5ce4 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:20.781632 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f75c4581-9722-474b-9e7c-52c4ff6e5ce4 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:20.782514 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f75c4581-9722-474b-9e7c-52c4ff6e5ce4 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:20.783151 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f75c4581-9722-474b-9e7c-52c4ff6e5ce4 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:20.809969 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-f75c4581-9722-474b-9e7c-52c4ff6e5ce4 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] HTTP exception thrown: Instance f114e124-2903-4f30-b055-9f493e3e9e6c could not be found. Jul 16 21:26:20.811920 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-f75c4581-9722-474b-9e7c-52c4ff6e5ce4 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Request method failure captured: Jul 16 21:26:20.811920 ncdv-master-1 devstack@n-api.service[138681]: request: GET /compute/v2.1/servers/f114e124-2903-4f30-b055-9f493e3e9e6c HTTP/1.1 Jul 16 21:26:20.811920 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:26:20.811920 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:26:20.811920 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:26:20.811920 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 21:26:20.811920 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:26:20.811920 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:26:20.811920 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:26:20.811920 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:26:20.811920 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0yu6GvklgrJz64l4jI_9XrerhdTTPfYTDqcboWClQTVUOsFR3X72A-yD-xnMKndBGdnSSARa9ZpL_5E_A7dDhsSNRAFKwYpwFmXgR867L_9gOcd2pdrdnzmP_tHWcFxJFbkFQsl_MPEJMXgACkcSj5Ps4CNmUG4pTC9v8iTd4ac5zTs Jul 16 21:26:20.811920 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:26:20.811920 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:26:20.811920 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:26:20.811920 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:26:20.811920 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:26:20.811920 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:26:20.811920 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: 22ca92a86e6c40d59071e58ce4085568 Jul 16 21:26:20.811920 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-AttachVolumeTestJSON-1168541016 Jul 16 21:26:20.811920 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:26:20.811920 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:26:20.811920 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/22ca92a86e6c40d59071e58ce4085568"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/22ca92a86e6c40d59071e58ce4085568"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_22ca92a86e6c40d59071e58ce4085568"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/22ca92a86e6c40d59071e58ce4085568"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:26:20.811920 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-AttachVolumeTestJSON-1168541016 Jul 16 21:26:20.811920 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: 22ca92a86e6c40d59071e58ce4085568 Jul 16 21:26:20.811920 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-AttachVolumeTestJSON-1168541016 Jul 16 21:26:20.811920 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-AttachVolumeTestJSON-1168541016-project Jul 16 21:26:20.811920 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:26:20.811920 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:26:20.811920 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: bba77f6ef42547f1aa8480e27868bd4a Jul 16 21:26:20.811920 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-AttachVolumeTestJSON-1168541016-project Jul 16 21:26:20.811920 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:26:20.811920 ncdv-master-1 devstack@n-api.service[138681]: exception: Instance f114e124-2903-4f30-b055-9f493e3e9e6c could not be found. Jul 16 21:26:20.811920 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': 'f114e124-2903-4f30-b055-9f493e3e9e6c'} Jul 16 21:26:20.811920 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:26:20.811920 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:26:20.811920 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:26:20.811920 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:26:20.811920 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:26:20.811920 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:26:20.811920 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:26:20.811920 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:26:20.811920 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:26:20.811920 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:26:20.811920 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance f114e124-2903-4f30-b055-9f493e3e9e6c could not be found. Jul 16 21:26:20.811920 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:26:20.811920 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:26:20.811920 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:26:20.811920 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:26:20.811920 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:26:20.811920 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:26:20.811920 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:26:20.811920 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:26:20.811920 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:26:20.811920 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:26:20.811920 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:26:20.811920 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:26:20.811920 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:26:20.811920 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:26:20.811920 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:26:20.811920 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:26:20.811920 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance f114e124-2903-4f30-b055-9f493e3e9e6c could not be found. Jul 16 21:26:20.811920 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:26:20.811920 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:26:20.811920 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:26:20.815742 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:26:20.815742 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:26:20.815742 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:26:20.815742 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:26:20.815742 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:26:20.815742 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance f114e124-2903-4f30-b055-9f493e3e9e6c could not be found. Jul 16 21:26:20.815742 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:26:20.818298 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-f75c4581-9722-474b-9e7c-52c4ff6e5ce4 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] Returning 404 to user: Instance f114e124-2903-4f30-b055-9f493e3e9e6c could not be found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:26:20.819855 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-f75c4581-9722-474b-9e7c-52c4ff6e5ce4 tempest-AttachVolumeTestJSON-1168541016 tempest-AttachVolumeTestJSON-1168541016-project] 10.222.0.16 "GET /compute/v2.1/servers/f114e124-2903-4f30-b055-9f493e3e9e6c" status: 404 len: 111 microversion: 2.1 time: 0.050879 Jul 16 21:26:20.840262 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1903/3802] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:26:20 2022] GET /compute/v2.1/servers/f114e124-2903-4f30-b055-9f493e3e9e6c => generated 111 bytes in 57 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:26:20.967503 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-a1dbc064-72d1-47a2-badb-01e26183fcb2 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:26:20.968122 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-a1dbc064-72d1-47a2-badb-01e26183fcb2 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 6ba66466-0da7-4e8d-8ddd-b46f582713fc] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:26:20.974611 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a1dbc064-72d1-47a2-badb-01e26183fcb2 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:20.977107 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a1dbc064-72d1-47a2-badb-01e26183fcb2 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:20.977107 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a1dbc064-72d1-47a2-badb-01e26183fcb2 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:20.977107 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a1dbc064-72d1-47a2-badb-01e26183fcb2 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:21.006465 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=96,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:26:21.100886 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-a1dbc064-72d1-47a2-badb-01e26183fcb2 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f2e974ab5aa48dfb4b00f810dcedb63', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd31be8475a6a49d6bfe8132dd050c6c0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:26:21.122672 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-a1dbc064-72d1-47a2-badb-01e26183fcb2 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "GET /compute/v2.1/servers/6ba66466-0da7-4e8d-8ddd-b46f582713fc" status: 200 len: 1266 microversion: 2.1 time: 0.146729 Jul 16 21:26:21.122672 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1904/3803] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:26:20 2022] GET /compute/v2.1/servers/6ba66466-0da7-4e8d-8ddd-b46f582713fc => generated 1266 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:26:21.171383 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-39988d9c-e80c-4e05-8cec-5fcf8193749d tempest-InstanceActionsNegativeTestJSON-904555896 tempest-InstanceActionsNegativeTestJSON-904555896-project] Flavor limits 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 16 21:26:21.171823 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-39988d9c-e80c-4e05-8cec-5fcf8193749d tempest-InstanceActionsNegativeTestJSON-904555896 tempest-InstanceActionsNegativeTestJSON-904555896-project] Image limits 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 16 21:26:21.172317 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-39988d9c-e80c-4e05-8cec-5fcf8193749d tempest-InstanceActionsNegativeTestJSON-904555896 tempest-InstanceActionsNegativeTestJSON-904555896-project] Flavor pref 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 16 21:26:21.172738 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-39988d9c-e80c-4e05-8cec-5fcf8193749d tempest-InstanceActionsNegativeTestJSON-904555896 tempest-InstanceActionsNegativeTestJSON-904555896-project] Image pref 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 16 21:26:21.173182 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-39988d9c-e80c-4e05-8cec-5fcf8193749d tempest-InstanceActionsNegativeTestJSON-904555896 tempest-InstanceActionsNegativeTestJSON-904555896-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 16 21:26:21.184838 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-39988d9c-e80c-4e05-8cec-5fcf8193749d tempest-InstanceActionsNegativeTestJSON-904555896 tempest-InstanceActionsNegativeTestJSON-904555896-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:21.185303 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-39988d9c-e80c-4e05-8cec-5fcf8193749d tempest-InstanceActionsNegativeTestJSON-904555896 tempest-InstanceActionsNegativeTestJSON-904555896-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:21.186184 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-39988d9c-e80c-4e05-8cec-5fcf8193749d tempest-InstanceActionsNegativeTestJSON-904555896 tempest-InstanceActionsNegativeTestJSON-904555896-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:21.186636 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-39988d9c-e80c-4e05-8cec-5fcf8193749d tempest-InstanceActionsNegativeTestJSON-904555896 tempest-InstanceActionsNegativeTestJSON-904555896-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:21.209111 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-39988d9c-e80c-4e05-8cec-5fcf8193749d tempest-InstanceActionsNegativeTestJSON-904555896 tempest-InstanceActionsNegativeTestJSON-904555896-project] Getting quotas for project 8f2736abb6094c1f83a7aa3efc772506. Resources: {'ram', 'instances', 'cores'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:26:21.215397 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-39988d9c-e80c-4e05-8cec-5fcf8193749d tempest-InstanceActionsNegativeTestJSON-904555896 tempest-InstanceActionsNegativeTestJSON-904555896-project] Getting quotas for user 1963f7b0ee0244b2b4f5fc1d8b34f858 and project 8f2736abb6094c1f83a7aa3efc772506. Resources: {'ram', 'instances', 'cores'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:26:21.231948 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-39988d9c-e80c-4e05-8cec-5fcf8193749d tempest-InstanceActionsNegativeTestJSON-904555896 tempest-InstanceActionsNegativeTestJSON-904555896-project] Going to run 1 instances... {{(pid=138680) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 16 21:26:21.234193 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-39988d9c-e80c-4e05-8cec-5fcf8193749d tempest-InstanceActionsNegativeTestJSON-904555896 tempest-InstanceActionsNegativeTestJSON-904555896-project] [instance: ae4b76bc-e7bd-45ff-8716-361dcbbfaa1d] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138680) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 16 21:26:21.308635 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_api:SELECT=95,nova_api:INSERT=5 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:26:21.414217 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-f4781b6e-64cd-4121-98fd-30d5e385e09d tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:26:21.415228 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-f4781b6e-64cd-4121-98fd-30d5e385e09d tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: 6dcb4587-0e15-4a2d-b8f0-70b459fb6664] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:26:21.428470 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f4781b6e-64cd-4121-98fd-30d5e385e09d tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:21.432448 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f4781b6e-64cd-4121-98fd-30d5e385e09d tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:21.433474 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f4781b6e-64cd-4121-98fd-30d5e385e09d tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:21.434746 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f4781b6e-64cd-4121-98fd-30d5e385e09d tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:21.496460 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-39988d9c-e80c-4e05-8cec-5fcf8193749d tempest-InstanceActionsNegativeTestJSON-904555896 tempest-InstanceActionsNegativeTestJSON-904555896-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 1.202351 Jul 16 21:26:21.502133 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1900/3804] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 21:26:20 2022] POST /compute/v2.1/servers => generated 380 bytes in 1209 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 16 21:26:21.512176 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-9b400256-dc4a-466d-9a50-4befa728e413 tempest-InstanceActionsNegativeTestJSON-904555896 tempest-InstanceActionsNegativeTestJSON-904555896-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:26:21.528667 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-9b400256-dc4a-466d-9a50-4befa728e413 tempest-InstanceActionsNegativeTestJSON-904555896 tempest-InstanceActionsNegativeTestJSON-904555896-project] [instance: ae4b76bc-e7bd-45ff-8716-361dcbbfaa1d] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:26:21.564998 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-f4781b6e-64cd-4121-98fd-30d5e385e09d tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '027dcc48e4fd4c36a5004096ad5e43f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dad823e240149edb30e592380249a13', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:26:21.581064 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-f4781b6e-64cd-4121-98fd-30d5e385e09d tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "GET /compute/v2.1/servers/6dcb4587-0e15-4a2d-b8f0-70b459fb6664" status: 200 len: 1204 microversion: 2.1 time: 0.170542 Jul 16 21:26:21.581966 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1905/3805] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:26:21 2022] GET /compute/v2.1/servers/6dcb4587-0e15-4a2d-b8f0-70b459fb6664 => generated 1204 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:26:21.657148 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-9b400256-dc4a-466d-9a50-4befa728e413 tempest-InstanceActionsNegativeTestJSON-904555896 tempest-InstanceActionsNegativeTestJSON-904555896-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1963f7b0ee0244b2b4f5fc1d8b34f858', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8f2736abb6094c1f83a7aa3efc772506', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:26:21.668424 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-9b400256-dc4a-466d-9a50-4befa728e413 tempest-InstanceActionsNegativeTestJSON-904555896 tempest-InstanceActionsNegativeTestJSON-904555896-project] 10.222.0.16 "GET /compute/v2.1/servers/ae4b76bc-e7bd-45ff-8716-361dcbbfaa1d" status: 200 len: 1224 microversion: 2.1 time: 0.160183 Jul 16 21:26:21.669377 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1901/3806] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:26:21 2022] GET /compute/v2.1/servers/ae4b76bc-e7bd-45ff-8716-361dcbbfaa1d => generated 1224 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:26:22.136207 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-40efa09b-add7-4d07-b0d2-97b0457f9abd tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:26:22.136207 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-40efa09b-add7-4d07-b0d2-97b0457f9abd tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 6ba66466-0da7-4e8d-8ddd-b46f582713fc] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:26:22.145460 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-40efa09b-add7-4d07-b0d2-97b0457f9abd tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:22.147743 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-40efa09b-add7-4d07-b0d2-97b0457f9abd tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:22.148794 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-40efa09b-add7-4d07-b0d2-97b0457f9abd tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:22.152807 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-40efa09b-add7-4d07-b0d2-97b0457f9abd tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:22.330953 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-40efa09b-add7-4d07-b0d2-97b0457f9abd tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f2e974ab5aa48dfb4b00f810dcedb63', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd31be8475a6a49d6bfe8132dd050c6c0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:26:22.361067 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-40efa09b-add7-4d07-b0d2-97b0457f9abd tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "GET /compute/v2.1/servers/6ba66466-0da7-4e8d-8ddd-b46f582713fc" status: 200 len: 1272 microversion: 2.1 time: 0.233461 Jul 16 21:26:22.370049 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1906/3807] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:26:22 2022] GET /compute/v2.1/servers/6ba66466-0da7-4e8d-8ddd-b46f582713fc => generated 1272 bytes in 245 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:26:22.604808 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-e322524d-23a0-4254-b1f3-6fef31b61fb0 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:26:22.605484 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-e322524d-23a0-4254-b1f3-6fef31b61fb0 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: 6dcb4587-0e15-4a2d-b8f0-70b459fb6664] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:26:22.620443 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e322524d-23a0-4254-b1f3-6fef31b61fb0 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:22.620909 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e322524d-23a0-4254-b1f3-6fef31b61fb0 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:22.624668 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e322524d-23a0-4254-b1f3-6fef31b61fb0 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:22.625430 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e322524d-23a0-4254-b1f3-6fef31b61fb0 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:22.687828 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-50be9068-949f-4bdd-9b02-b25b3aca0c18 tempest-InstanceActionsNegativeTestJSON-904555896 tempest-InstanceActionsNegativeTestJSON-904555896-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:26:22.693764 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-50be9068-949f-4bdd-9b02-b25b3aca0c18 tempest-InstanceActionsNegativeTestJSON-904555896 tempest-InstanceActionsNegativeTestJSON-904555896-project] [instance: ae4b76bc-e7bd-45ff-8716-361dcbbfaa1d] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:26:22.913132 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-e322524d-23a0-4254-b1f3-6fef31b61fb0 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '027dcc48e4fd4c36a5004096ad5e43f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dad823e240149edb30e592380249a13', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:26:22.948174 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-50be9068-949f-4bdd-9b02-b25b3aca0c18 tempest-InstanceActionsNegativeTestJSON-904555896 tempest-InstanceActionsNegativeTestJSON-904555896-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1963f7b0ee0244b2b4f5fc1d8b34f858', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8f2736abb6094c1f83a7aa3efc772506', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:26:22.949117 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-e322524d-23a0-4254-b1f3-6fef31b61fb0 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "GET /compute/v2.1/servers/6dcb4587-0e15-4a2d-b8f0-70b459fb6664" status: 200 len: 1268 microversion: 2.1 time: 0.352556 Jul 16 21:26:22.951312 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1902/3808] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:26:22 2022] GET /compute/v2.1/servers/6dcb4587-0e15-4a2d-b8f0-70b459fb6664 => generated 1268 bytes in 355 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:26:22.969224 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-50be9068-949f-4bdd-9b02-b25b3aca0c18 tempest-InstanceActionsNegativeTestJSON-904555896 tempest-InstanceActionsNegativeTestJSON-904555896-project] 10.222.0.16 "GET /compute/v2.1/servers/ae4b76bc-e7bd-45ff-8716-361dcbbfaa1d" status: 200 len: 1224 microversion: 2.1 time: 0.287289 Jul 16 21:26:22.971360 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1907/3809] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:26:22 2022] GET /compute/v2.1/servers/ae4b76bc-e7bd-45ff-8716-361dcbbfaa1d => generated 1224 bytes in 290 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:26:23.389731 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-00fb6e9c-8ae8-4578-a2f1-1246e6f563e2 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:26:23.391181 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-00fb6e9c-8ae8-4578-a2f1-1246e6f563e2 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 6ba66466-0da7-4e8d-8ddd-b46f582713fc] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:26:23.407396 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-00fb6e9c-8ae8-4578-a2f1-1246e6f563e2 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:23.409859 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-00fb6e9c-8ae8-4578-a2f1-1246e6f563e2 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:23.412118 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-00fb6e9c-8ae8-4578-a2f1-1246e6f563e2 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:23.414880 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-00fb6e9c-8ae8-4578-a2f1-1246e6f563e2 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:23.925043 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-00fb6e9c-8ae8-4578-a2f1-1246e6f563e2 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f2e974ab5aa48dfb4b00f810dcedb63', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd31be8475a6a49d6bfe8132dd050c6c0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:26:23.940716 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-00fb6e9c-8ae8-4578-a2f1-1246e6f563e2 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "GET /compute/v2.1/servers/6ba66466-0da7-4e8d-8ddd-b46f582713fc" status: 200 len: 1314 microversion: 2.1 time: 0.557914 Jul 16 21:26:23.948960 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1903/3810] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:26:23 2022] GET /compute/v2.1/servers/6ba66466-0da7-4e8d-8ddd-b46f582713fc => generated 1314 bytes in 567 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:26:23.968596 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-4ffc626f-5723-4651-b6c9-54e5a5e3e1e8 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:26:23.969215 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-4ffc626f-5723-4651-b6c9-54e5a5e3e1e8 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: 6dcb4587-0e15-4a2d-b8f0-70b459fb6664] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:26:23.992483 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-d630b5e5-b704-4fab-92fd-43776ea6b1cf tempest-InstanceActionsNegativeTestJSON-904555896 tempest-InstanceActionsNegativeTestJSON-904555896-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:26:23.993238 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-d630b5e5-b704-4fab-92fd-43776ea6b1cf tempest-InstanceActionsNegativeTestJSON-904555896 tempest-InstanceActionsNegativeTestJSON-904555896-project] [instance: ae4b76bc-e7bd-45ff-8716-361dcbbfaa1d] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:26:23.995306 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4ffc626f-5723-4651-b6c9-54e5a5e3e1e8 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:23.995985 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4ffc626f-5723-4651-b6c9-54e5a5e3e1e8 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:23.997456 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4ffc626f-5723-4651-b6c9-54e5a5e3e1e8 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:23.998104 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4ffc626f-5723-4651-b6c9-54e5a5e3e1e8 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:24.007502 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d630b5e5-b704-4fab-92fd-43776ea6b1cf tempest-InstanceActionsNegativeTestJSON-904555896 tempest-InstanceActionsNegativeTestJSON-904555896-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:24.008404 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d630b5e5-b704-4fab-92fd-43776ea6b1cf tempest-InstanceActionsNegativeTestJSON-904555896 tempest-InstanceActionsNegativeTestJSON-904555896-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:24.009387 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d630b5e5-b704-4fab-92fd-43776ea6b1cf tempest-InstanceActionsNegativeTestJSON-904555896 tempest-InstanceActionsNegativeTestJSON-904555896-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:24.012703 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d630b5e5-b704-4fab-92fd-43776ea6b1cf tempest-InstanceActionsNegativeTestJSON-904555896 tempest-InstanceActionsNegativeTestJSON-904555896-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:24.170938 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-4ffc626f-5723-4651-b6c9-54e5a5e3e1e8 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '027dcc48e4fd4c36a5004096ad5e43f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dad823e240149edb30e592380249a13', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:26:24.181143 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-4ffc626f-5723-4651-b6c9-54e5a5e3e1e8 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "GET /compute/v2.1/servers/6dcb4587-0e15-4a2d-b8f0-70b459fb6664" status: 200 len: 1266 microversion: 2.1 time: 0.216694 Jul 16 21:26:24.183329 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1908/3811] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:26:23 2022] GET /compute/v2.1/servers/6dcb4587-0e15-4a2d-b8f0-70b459fb6664 => generated 1266 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:26:24.202182 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-d630b5e5-b704-4fab-92fd-43776ea6b1cf tempest-InstanceActionsNegativeTestJSON-904555896 tempest-InstanceActionsNegativeTestJSON-904555896-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1963f7b0ee0244b2b4f5fc1d8b34f858', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8f2736abb6094c1f83a7aa3efc772506', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:26:24.212588 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-d630b5e5-b704-4fab-92fd-43776ea6b1cf tempest-InstanceActionsNegativeTestJSON-904555896 tempest-InstanceActionsNegativeTestJSON-904555896-project] 10.222.0.16 "GET /compute/v2.1/servers/ae4b76bc-e7bd-45ff-8716-361dcbbfaa1d" status: 200 len: 1220 microversion: 2.1 time: 0.224700 Jul 16 21:26:24.213711 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1904/3812] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:26:23 2022] GET /compute/v2.1/servers/ae4b76bc-e7bd-45ff-8716-361dcbbfaa1d => generated 1220 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:26:24.962636 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-64c2b3d5-49e7-4095-9d2f-c7482ae82562 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:26:24.963593 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-64c2b3d5-49e7-4095-9d2f-c7482ae82562 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 6ba66466-0da7-4e8d-8ddd-b46f582713fc] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:26:24.987578 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-64c2b3d5-49e7-4095-9d2f-c7482ae82562 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:24.990501 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-64c2b3d5-49e7-4095-9d2f-c7482ae82562 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:24.993741 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-64c2b3d5-49e7-4095-9d2f-c7482ae82562 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:24.995181 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-64c2b3d5-49e7-4095-9d2f-c7482ae82562 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:25.201064 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-ab8413ed-e955-461c-8730-b3730ed0e6b2 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:26:25.201748 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-ab8413ed-e955-461c-8730-b3730ed0e6b2 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: 6dcb4587-0e15-4a2d-b8f0-70b459fb6664] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:26:25.218291 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ab8413ed-e955-461c-8730-b3730ed0e6b2 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:25.225532 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ab8413ed-e955-461c-8730-b3730ed0e6b2 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:25.227932 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ab8413ed-e955-461c-8730-b3730ed0e6b2 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:25.228575 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ab8413ed-e955-461c-8730-b3730ed0e6b2 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:25.697455 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-64c2b3d5-49e7-4095-9d2f-c7482ae82562 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f2e974ab5aa48dfb4b00f810dcedb63', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd31be8475a6a49d6bfe8132dd050c6c0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:26:25.705270 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-64c2b3d5-49e7-4095-9d2f-c7482ae82562 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "GET /compute/v2.1/servers/6ba66466-0da7-4e8d-8ddd-b46f582713fc" status: 200 len: 1314 microversion: 2.1 time: 0.748871 Jul 16 21:26:25.705849 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1909/3813] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:26:24 2022] GET /compute/v2.1/servers/6ba66466-0da7-4e8d-8ddd-b46f582713fc => generated 1314 bytes in 750 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:26:25.721706 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-5f79b133-2187-474f-8f6c-68e3b7848753 tempest-InstanceActionsNegativeTestJSON-904555896 tempest-InstanceActionsNegativeTestJSON-904555896-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:26:25.722329 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-5f79b133-2187-474f-8f6c-68e3b7848753 tempest-InstanceActionsNegativeTestJSON-904555896 tempest-InstanceActionsNegativeTestJSON-904555896-project] [instance: ae4b76bc-e7bd-45ff-8716-361dcbbfaa1d] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:26:25.753445 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5f79b133-2187-474f-8f6c-68e3b7848753 tempest-InstanceActionsNegativeTestJSON-904555896 tempest-InstanceActionsNegativeTestJSON-904555896-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:25.754006 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5f79b133-2187-474f-8f6c-68e3b7848753 tempest-InstanceActionsNegativeTestJSON-904555896 tempest-InstanceActionsNegativeTestJSON-904555896-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:25.754982 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5f79b133-2187-474f-8f6c-68e3b7848753 tempest-InstanceActionsNegativeTestJSON-904555896 tempest-InstanceActionsNegativeTestJSON-904555896-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:25.755456 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5f79b133-2187-474f-8f6c-68e3b7848753 tempest-InstanceActionsNegativeTestJSON-904555896 tempest-InstanceActionsNegativeTestJSON-904555896-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:25.991839 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-5f79b133-2187-474f-8f6c-68e3b7848753 tempest-InstanceActionsNegativeTestJSON-904555896 tempest-InstanceActionsNegativeTestJSON-904555896-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1963f7b0ee0244b2b4f5fc1d8b34f858', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8f2736abb6094c1f83a7aa3efc772506', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:26:26.009770 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-5f79b133-2187-474f-8f6c-68e3b7848753 tempest-InstanceActionsNegativeTestJSON-904555896 tempest-InstanceActionsNegativeTestJSON-904555896-project] 10.222.0.16 "GET /compute/v2.1/servers/ae4b76bc-e7bd-45ff-8716-361dcbbfaa1d" status: 200 len: 1282 microversion: 2.1 time: 0.298943 Jul 16 21:26:26.021912 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1910/3814] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:26:25 2022] GET /compute/v2.1/servers/ae4b76bc-e7bd-45ff-8716-361dcbbfaa1d => generated 1282 bytes in 303 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:26:26.028176 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-9d34200e-de27-4682-aee9-c14ba6c60389 req-46aec536-9dd9-48f6-8933-8335e17392b4 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "6ba66466-0da7-4e8d-8ddd-b46f582713fc", "tag": "20de36ec-25f9-4437-b0d2-08b6125425c3"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:26:26.042710 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-9d34200e-de27-4682-aee9-c14ba6c60389 req-46aec536-9dd9-48f6-8933-8335e17392b4 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:26.043639 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-9d34200e-de27-4682-aee9-c14ba6c60389 req-46aec536-9dd9-48f6-8933-8335e17392b4 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:26.056295 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-9d34200e-de27-4682-aee9-c14ba6c60389 req-46aec536-9dd9-48f6-8933-8335e17392b4 service nova] Creating event network-changed:20de36ec-25f9-4437-b0d2-08b6125425c3 for instance 6ba66466-0da7-4e8d-8ddd-b46f582713fc on nch2-master-1 Jul 16 21:26:26.061286 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-9d34200e-de27-4682-aee9-c14ba6c60389 req-46aec536-9dd9-48f6-8933-8335e17392b4 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.046851 Jul 16 21:26:26.062389 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1911/3815] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:26:26 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:26:26.068405 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-902476ec-d59f-4052-9e97-7eb934b45ed5 tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesBackupsTest-instance-1418814472", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452"}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:26:26.105418 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-ab8413ed-e955-461c-8730-b3730ed0e6b2 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '027dcc48e4fd4c36a5004096ad5e43f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dad823e240149edb30e592380249a13', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:26:26.114344 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-ab8413ed-e955-461c-8730-b3730ed0e6b2 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "GET /compute/v2.1/servers/6dcb4587-0e15-4a2d-b8f0-70b459fb6664" status: 200 len: 1308 microversion: 2.1 time: 0.917311 Jul 16 21:26:26.130447 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1905/3816] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:26:25 2022] GET /compute/v2.1/servers/6dcb4587-0e15-4a2d-b8f0-70b459fb6664 => generated 1308 bytes in 934 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:26:26.144070 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.network.neutron [None req-902476ec-d59f-4052-9e97-7eb934b45ed5 tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] validate_networks() for None {{(pid=138681) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 16 21:26:26.638156 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-902476ec-d59f-4052-9e97-7eb934b45ed5 tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] Flavor limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 16 21:26:26.638584 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-902476ec-d59f-4052-9e97-7eb934b45ed5 tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] Image limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 16 21:26:26.640050 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-902476ec-d59f-4052-9e97-7eb934b45ed5 tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] Flavor pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 16 21:26:26.640144 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-902476ec-d59f-4052-9e97-7eb934b45ed5 tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] Image pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 16 21:26:26.640539 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-902476ec-d59f-4052-9e97-7eb934b45ed5 tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 16 21:26:26.658310 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-902476ec-d59f-4052-9e97-7eb934b45ed5 tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:26.658753 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-902476ec-d59f-4052-9e97-7eb934b45ed5 tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:26.659611 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-902476ec-d59f-4052-9e97-7eb934b45ed5 tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:26.660051 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-902476ec-d59f-4052-9e97-7eb934b45ed5 tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:26.686389 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-902476ec-d59f-4052-9e97-7eb934b45ed5 tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] Getting quotas for project 7d5a798ee507429bb70566356229535d. Resources: {'ram', 'instances', 'cores'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:26:26.699197 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-902476ec-d59f-4052-9e97-7eb934b45ed5 tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] Getting quotas for user bb1f6532421745a4b0de70356748a647 and project 7d5a798ee507429bb70566356229535d. Resources: {'ram', 'instances', 'cores'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:26:26.722336 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-902476ec-d59f-4052-9e97-7eb934b45ed5 tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] Going to run 1 instances... {{(pid=138681) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 16 21:26:26.726329 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-902476ec-d59f-4052-9e97-7eb934b45ed5 tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] [instance: 9a184cb7-2548-41d5-8d37-8e72dfb3d75a] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138681) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 16 21:26:26.737262 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-8b47dfb9-5c65-4ef1-aec3-cdb486c85715 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:26:26.738980 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-8b47dfb9-5c65-4ef1-aec3-cdb486c85715 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 6ba66466-0da7-4e8d-8ddd-b46f582713fc] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:26:26.768304 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8b47dfb9-5c65-4ef1-aec3-cdb486c85715 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:26.768846 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8b47dfb9-5c65-4ef1-aec3-cdb486c85715 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:26.770268 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8b47dfb9-5c65-4ef1-aec3-cdb486c85715 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:26.771329 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8b47dfb9-5c65-4ef1-aec3-cdb486c85715 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:26.937718 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-902476ec-d59f-4052-9e97-7eb934b45ed5 tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.873661 Jul 16 21:26:26.938935 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1912/3817] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 21:26:26 2022] POST /compute/v2.1/servers => generated 380 bytes in 876 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 16 21:26:26.952642 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-ac3beb08-74fd-466f-9d76-1dcc99c16d48 tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:26:26.955819 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-ac3beb08-74fd-466f-9d76-1dcc99c16d48 tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] [instance: 9a184cb7-2548-41d5-8d37-8e72dfb3d75a] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:26:27.508489 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-ac3beb08-74fd-466f-9d76-1dcc99c16d48 tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb1f6532421745a4b0de70356748a647', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7d5a798ee507429bb70566356229535d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:26:27.534357 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-ac3beb08-74fd-466f-9d76-1dcc99c16d48 tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] 10.222.0.16 "GET /compute/v2.1/servers/9a184cb7-2548-41d5-8d37-8e72dfb3d75a" status: 200 len: 1214 microversion: 2.1 time: 0.577274 Jul 16 21:26:27.534357 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1913/3818] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:26:26 2022] GET /compute/v2.1/servers/9a184cb7-2548-41d5-8d37-8e72dfb3d75a => generated 1214 bytes in 579 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:26:27.548411 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-d521d4bf-7660-473a-aa57-ead0fb7450a1 tempest-InstanceActionsNegativeTestJSON-904555896 tempest-InstanceActionsNegativeTestJSON-904555896-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:26:27.549141 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-d521d4bf-7660-473a-aa57-ead0fb7450a1 tempest-InstanceActionsNegativeTestJSON-904555896 tempest-InstanceActionsNegativeTestJSON-904555896-project] [instance: ae4b76bc-e7bd-45ff-8716-361dcbbfaa1d] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:26:27.559687 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d521d4bf-7660-473a-aa57-ead0fb7450a1 tempest-InstanceActionsNegativeTestJSON-904555896 tempest-InstanceActionsNegativeTestJSON-904555896-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:27.560267 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d521d4bf-7660-473a-aa57-ead0fb7450a1 tempest-InstanceActionsNegativeTestJSON-904555896 tempest-InstanceActionsNegativeTestJSON-904555896-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:27.561148 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d521d4bf-7660-473a-aa57-ead0fb7450a1 tempest-InstanceActionsNegativeTestJSON-904555896 tempest-InstanceActionsNegativeTestJSON-904555896-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:27.561677 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d521d4bf-7660-473a-aa57-ead0fb7450a1 tempest-InstanceActionsNegativeTestJSON-904555896 tempest-InstanceActionsNegativeTestJSON-904555896-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:27.729705 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-8b47dfb9-5c65-4ef1-aec3-cdb486c85715 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f2e974ab5aa48dfb4b00f810dcedb63', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd31be8475a6a49d6bfe8132dd050c6c0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:26:27.740758 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-8b47dfb9-5c65-4ef1-aec3-cdb486c85715 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "GET /compute/v2.1/servers/6ba66466-0da7-4e8d-8ddd-b46f582713fc" status: 200 len: 1314 microversion: 2.1 time: 1.020148 Jul 16 21:26:27.741427 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1906/3819] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:26:26 2022] GET /compute/v2.1/servers/6ba66466-0da7-4e8d-8ddd-b46f582713fc => generated 1314 bytes in 1022 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:26:27.746861 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-0926262b-3f30-43d5-a258-e71b6a6be00e tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:26:27.747798 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-0926262b-3f30-43d5-a258-e71b6a6be00e tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: 6dcb4587-0e15-4a2d-b8f0-70b459fb6664] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:26:27.756776 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0926262b-3f30-43d5-a258-e71b6a6be00e tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:27.757235 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0926262b-3f30-43d5-a258-e71b6a6be00e tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:27.758072 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0926262b-3f30-43d5-a258-e71b6a6be00e tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:27.759245 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0926262b-3f30-43d5-a258-e71b6a6be00e tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:27.799426 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-d521d4bf-7660-473a-aa57-ead0fb7450a1 tempest-InstanceActionsNegativeTestJSON-904555896 tempest-InstanceActionsNegativeTestJSON-904555896-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1963f7b0ee0244b2b4f5fc1d8b34f858', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8f2736abb6094c1f83a7aa3efc772506', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:26:27.804616 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-d521d4bf-7660-473a-aa57-ead0fb7450a1 tempest-InstanceActionsNegativeTestJSON-904555896 tempest-InstanceActionsNegativeTestJSON-904555896-project] 10.222.0.16 "GET /compute/v2.1/servers/ae4b76bc-e7bd-45ff-8716-361dcbbfaa1d" status: 200 len: 1324 microversion: 2.1 time: 0.261725 Jul 16 21:26:27.805522 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1914/3820] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:26:27 2022] GET /compute/v2.1/servers/ae4b76bc-e7bd-45ff-8716-361dcbbfaa1d => generated 1324 bytes in 280 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:26:27.969647 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-0926262b-3f30-43d5-a258-e71b6a6be00e tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '027dcc48e4fd4c36a5004096ad5e43f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dad823e240149edb30e592380249a13', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:26:27.991174 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-0926262b-3f30-43d5-a258-e71b6a6be00e tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "GET /compute/v2.1/servers/6dcb4587-0e15-4a2d-b8f0-70b459fb6664" status: 200 len: 1308 microversion: 2.1 time: 0.248506 Jul 16 21:26:27.992298 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1907/3821] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:26:27 2022] GET /compute/v2.1/servers/6dcb4587-0e15-4a2d-b8f0-70b459fb6664 => generated 1308 bytes in 250 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:26:28.082154 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-767f303d-2890-4e71-a327-c59bbf51cc1f req-78cef5f8-f222-4046-ab72-737b8e478868 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "6dcb4587-0e15-4a2d-b8f0-70b459fb6664", "tag": "eda9d853-12d2-4057-a4a9-4683ca97fd9d"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:26:28.092099 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-767f303d-2890-4e71-a327-c59bbf51cc1f req-78cef5f8-f222-4046-ab72-737b8e478868 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:28.094748 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-767f303d-2890-4e71-a327-c59bbf51cc1f req-78cef5f8-f222-4046-ab72-737b8e478868 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:28.116513 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-767f303d-2890-4e71-a327-c59bbf51cc1f req-78cef5f8-f222-4046-ab72-737b8e478868 service nova] Creating event network-changed:eda9d853-12d2-4057-a4a9-4683ca97fd9d for instance 6dcb4587-0e15-4a2d-b8f0-70b459fb6664 on nch2-master-1 Jul 16 21:26:28.124181 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-767f303d-2890-4e71-a327-c59bbf51cc1f req-78cef5f8-f222-4046-ab72-737b8e478868 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.045594 Jul 16 21:26:28.125786 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1915/3822] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:26:28 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:26:28.543022 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-8b6eca1e-a17b-4ef3-8cf1-9283081afb9b tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:26:28.543649 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-8b6eca1e-a17b-4ef3-8cf1-9283081afb9b tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] [instance: 9a184cb7-2548-41d5-8d37-8e72dfb3d75a] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:26:28.555800 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8b6eca1e-a17b-4ef3-8cf1-9283081afb9b tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:28.556659 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8b6eca1e-a17b-4ef3-8cf1-9283081afb9b tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:28.558077 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8b6eca1e-a17b-4ef3-8cf1-9283081afb9b tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:28.558537 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8b6eca1e-a17b-4ef3-8cf1-9283081afb9b tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:28.641755 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-8b6eca1e-a17b-4ef3-8cf1-9283081afb9b tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb1f6532421745a4b0de70356748a647', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7d5a798ee507429bb70566356229535d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:26:28.650837 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-8b6eca1e-a17b-4ef3-8cf1-9283081afb9b tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] 10.222.0.16 "GET /compute/v2.1/servers/9a184cb7-2548-41d5-8d37-8e72dfb3d75a" status: 200 len: 1218 microversion: 2.1 time: 0.112567 Jul 16 21:26:28.651620 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1908/3823] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:26:28 2022] GET /compute/v2.1/servers/9a184cb7-2548-41d5-8d37-8e72dfb3d75a => generated 1218 bytes in 114 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:26:28.759714 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-65e46be6-37e4-4a3b-a970-8d482f50ed46 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:26:28.761151 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-65e46be6-37e4-4a3b-a970-8d482f50ed46 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 6ba66466-0da7-4e8d-8ddd-b46f582713fc] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:26:28.768680 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-65e46be6-37e4-4a3b-a970-8d482f50ed46 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:28.769450 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-65e46be6-37e4-4a3b-a970-8d482f50ed46 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:28.772011 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-65e46be6-37e4-4a3b-a970-8d482f50ed46 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:28.772676 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-65e46be6-37e4-4a3b-a970-8d482f50ed46 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:28.823950 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-b30acbfe-74af-4811-8ccc-df01bce4956d tempest-InstanceActionsNegativeTestJSON-904555896 tempest-InstanceActionsNegativeTestJSON-904555896-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:26:28.824642 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-b30acbfe-74af-4811-8ccc-df01bce4956d tempest-InstanceActionsNegativeTestJSON-904555896 tempest-InstanceActionsNegativeTestJSON-904555896-project] [instance: ae4b76bc-e7bd-45ff-8716-361dcbbfaa1d] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:26:28.840903 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b30acbfe-74af-4811-8ccc-df01bce4956d tempest-InstanceActionsNegativeTestJSON-904555896 tempest-InstanceActionsNegativeTestJSON-904555896-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:28.842780 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b30acbfe-74af-4811-8ccc-df01bce4956d tempest-InstanceActionsNegativeTestJSON-904555896 tempest-InstanceActionsNegativeTestJSON-904555896-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:28.844166 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b30acbfe-74af-4811-8ccc-df01bce4956d tempest-InstanceActionsNegativeTestJSON-904555896 tempest-InstanceActionsNegativeTestJSON-904555896-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:28.845681 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b30acbfe-74af-4811-8ccc-df01bce4956d tempest-InstanceActionsNegativeTestJSON-904555896 tempest-InstanceActionsNegativeTestJSON-904555896-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:29.276193 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-65e46be6-37e4-4a3b-a970-8d482f50ed46 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f2e974ab5aa48dfb4b00f810dcedb63', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd31be8475a6a49d6bfe8132dd050c6c0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:26:29.292312 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-65e46be6-37e4-4a3b-a970-8d482f50ed46 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "GET /compute/v2.1/servers/6ba66466-0da7-4e8d-8ddd-b46f582713fc" status: 200 len: 1314 microversion: 2.1 time: 0.530610 Jul 16 21:26:29.293108 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1916/3824] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:26:28 2022] GET /compute/v2.1/servers/6ba66466-0da7-4e8d-8ddd-b46f582713fc => generated 1314 bytes in 540 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:26:29.301898 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-4046d5e8-9ca6-43eb-9d52-50cf29372f12 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:26:29.302921 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-4046d5e8-9ca6-43eb-9d52-50cf29372f12 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: 6dcb4587-0e15-4a2d-b8f0-70b459fb6664] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:26:29.312471 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4046d5e8-9ca6-43eb-9d52-50cf29372f12 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:29.312942 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4046d5e8-9ca6-43eb-9d52-50cf29372f12 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:29.314075 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4046d5e8-9ca6-43eb-9d52-50cf29372f12 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:29.314584 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4046d5e8-9ca6-43eb-9d52-50cf29372f12 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:29.461488 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-b30acbfe-74af-4811-8ccc-df01bce4956d tempest-InstanceActionsNegativeTestJSON-904555896 tempest-InstanceActionsNegativeTestJSON-904555896-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1963f7b0ee0244b2b4f5fc1d8b34f858', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8f2736abb6094c1f83a7aa3efc772506', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:26:29.484658 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-b30acbfe-74af-4811-8ccc-df01bce4956d tempest-InstanceActionsNegativeTestJSON-904555896 tempest-InstanceActionsNegativeTestJSON-904555896-project] 10.222.0.16 "GET /compute/v2.1/servers/ae4b76bc-e7bd-45ff-8716-361dcbbfaa1d" status: 200 len: 1324 microversion: 2.1 time: 0.665899 Jul 16 21:26:29.485845 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1909/3825] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:26:28 2022] GET /compute/v2.1/servers/ae4b76bc-e7bd-45ff-8716-361dcbbfaa1d => generated 1324 bytes in 668 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:26:29.511606 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-a0fbefb1-3734-428f-8864-c4ea825a54e5 req-46e0004d-40b6-4af5-adba-5303a08e2ad7 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "ae4b76bc-e7bd-45ff-8716-361dcbbfaa1d", "tag": "95067406-61b5-4fed-bcd2-609bccc3cb11"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:26:29.525674 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-a0fbefb1-3734-428f-8864-c4ea825a54e5 req-46e0004d-40b6-4af5-adba-5303a08e2ad7 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:29.528324 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-a0fbefb1-3734-428f-8864-c4ea825a54e5 req-46e0004d-40b6-4af5-adba-5303a08e2ad7 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:29.550599 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-a0fbefb1-3734-428f-8864-c4ea825a54e5 req-46e0004d-40b6-4af5-adba-5303a08e2ad7 service nova] Creating event network-changed:95067406-61b5-4fed-bcd2-609bccc3cb11 for instance ae4b76bc-e7bd-45ff-8716-361dcbbfaa1d on nch2-master-1 Jul 16 21:26:29.555701 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-4046d5e8-9ca6-43eb-9d52-50cf29372f12 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '027dcc48e4fd4c36a5004096ad5e43f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dad823e240149edb30e592380249a13', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:26:29.563711 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-4046d5e8-9ca6-43eb-9d52-50cf29372f12 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "GET /compute/v2.1/servers/6dcb4587-0e15-4a2d-b8f0-70b459fb6664" status: 200 len: 1308 microversion: 2.1 time: 0.269459 Jul 16 21:26:29.564526 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1917/3826] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:26:29 2022] GET /compute/v2.1/servers/6dcb4587-0e15-4a2d-b8f0-70b459fb6664 => generated 1308 bytes in 271 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:26:29.565860 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-a0fbefb1-3734-428f-8864-c4ea825a54e5 req-46e0004d-40b6-4af5-adba-5303a08e2ad7 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.077821 Jul 16 21:26:29.567124 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1910/3827] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:26:29 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 80 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:26:29.668955 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-86bd77b6-23aa-4bfb-a5e4-c20afe7318fe tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:26:29.670179 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-86bd77b6-23aa-4bfb-a5e4-c20afe7318fe tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] [instance: 9a184cb7-2548-41d5-8d37-8e72dfb3d75a] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:26:29.679790 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-86bd77b6-23aa-4bfb-a5e4-c20afe7318fe tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:29.684655 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-86bd77b6-23aa-4bfb-a5e4-c20afe7318fe tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:29.684655 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-86bd77b6-23aa-4bfb-a5e4-c20afe7318fe tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:29.684655 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-86bd77b6-23aa-4bfb-a5e4-c20afe7318fe tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:29.811948 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-86bd77b6-23aa-4bfb-a5e4-c20afe7318fe tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb1f6532421745a4b0de70356748a647', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7d5a798ee507429bb70566356229535d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:26:29.826760 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-86bd77b6-23aa-4bfb-a5e4-c20afe7318fe tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] 10.222.0.16 "GET /compute/v2.1/servers/9a184cb7-2548-41d5-8d37-8e72dfb3d75a" status: 200 len: 1266 microversion: 2.1 time: 0.161583 Jul 16 21:26:29.827914 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1918/3828] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:26:29 2022] GET /compute/v2.1/servers/9a184cb7-2548-41d5-8d37-8e72dfb3d75a => generated 1266 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:26:30.310275 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-324898aa-de96-47e4-bcea-b64fd9bfb347 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:26:30.311028 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-324898aa-de96-47e4-bcea-b64fd9bfb347 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 6ba66466-0da7-4e8d-8ddd-b46f582713fc] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:26:30.318569 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-324898aa-de96-47e4-bcea-b64fd9bfb347 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:30.319079 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-324898aa-de96-47e4-bcea-b64fd9bfb347 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:30.319848 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-324898aa-de96-47e4-bcea-b64fd9bfb347 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:30.320291 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-324898aa-de96-47e4-bcea-b64fd9bfb347 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:30.502936 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-10d05f22-21e5-47d4-9ec9-d1a913527266 tempest-InstanceActionsNegativeTestJSON-904555896 tempest-InstanceActionsNegativeTestJSON-904555896-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:26:30.503579 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-10d05f22-21e5-47d4-9ec9-d1a913527266 tempest-InstanceActionsNegativeTestJSON-904555896 tempest-InstanceActionsNegativeTestJSON-904555896-project] [instance: ae4b76bc-e7bd-45ff-8716-361dcbbfaa1d] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:26:30.505603 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-324898aa-de96-47e4-bcea-b64fd9bfb347 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f2e974ab5aa48dfb4b00f810dcedb63', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd31be8475a6a49d6bfe8132dd050c6c0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:26:30.506607 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_api:SELECT=93,nova_api:UPDATE=1,nova_api:INSERT=6 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:26:30.511562 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-10d05f22-21e5-47d4-9ec9-d1a913527266 tempest-InstanceActionsNegativeTestJSON-904555896 tempest-InstanceActionsNegativeTestJSON-904555896-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:30.512016 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-10d05f22-21e5-47d4-9ec9-d1a913527266 tempest-InstanceActionsNegativeTestJSON-904555896 tempest-InstanceActionsNegativeTestJSON-904555896-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:30.512807 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-10d05f22-21e5-47d4-9ec9-d1a913527266 tempest-InstanceActionsNegativeTestJSON-904555896 tempest-InstanceActionsNegativeTestJSON-904555896-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:30.513244 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-10d05f22-21e5-47d4-9ec9-d1a913527266 tempest-InstanceActionsNegativeTestJSON-904555896 tempest-InstanceActionsNegativeTestJSON-904555896-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:30.513861 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-324898aa-de96-47e4-bcea-b64fd9bfb347 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "GET /compute/v2.1/servers/6ba66466-0da7-4e8d-8ddd-b46f582713fc" status: 200 len: 1314 microversion: 2.1 time: 0.207834 Jul 16 21:26:30.514431 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1911/3829] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:26:30 2022] GET /compute/v2.1/servers/6ba66466-0da7-4e8d-8ddd-b46f582713fc => generated 1314 bytes in 209 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:26:30.581572 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-88577171-13cc-440e-bfc0-0e86c12f838a tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:26:30.583024 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-88577171-13cc-440e-bfc0-0e86c12f838a tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: 6dcb4587-0e15-4a2d-b8f0-70b459fb6664] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:26:30.593944 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-88577171-13cc-440e-bfc0-0e86c12f838a tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:30.600519 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-88577171-13cc-440e-bfc0-0e86c12f838a tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:30.600519 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-88577171-13cc-440e-bfc0-0e86c12f838a tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:30.600519 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-88577171-13cc-440e-bfc0-0e86c12f838a tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:30.854006 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-10d05f22-21e5-47d4-9ec9-d1a913527266 tempest-InstanceActionsNegativeTestJSON-904555896 tempest-InstanceActionsNegativeTestJSON-904555896-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1963f7b0ee0244b2b4f5fc1d8b34f858', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8f2736abb6094c1f83a7aa3efc772506', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:26:30.861781 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-10d05f22-21e5-47d4-9ec9-d1a913527266 tempest-InstanceActionsNegativeTestJSON-904555896 tempest-InstanceActionsNegativeTestJSON-904555896-project] 10.222.0.16 "GET /compute/v2.1/servers/ae4b76bc-e7bd-45ff-8716-361dcbbfaa1d" status: 200 len: 1324 microversion: 2.1 time: 0.361697 Jul 16 21:26:30.862517 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1919/3830] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:26:30 2022] GET /compute/v2.1/servers/ae4b76bc-e7bd-45ff-8716-361dcbbfaa1d => generated 1324 bytes in 364 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:26:30.869232 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-e7482e19-9fda-42dd-9e93-fce871b84f11 tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:26:30.869976 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-e7482e19-9fda-42dd-9e93-fce871b84f11 tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] [instance: 9a184cb7-2548-41d5-8d37-8e72dfb3d75a] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:26:30.878620 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e7482e19-9fda-42dd-9e93-fce871b84f11 tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:30.879481 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e7482e19-9fda-42dd-9e93-fce871b84f11 tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:30.880839 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e7482e19-9fda-42dd-9e93-fce871b84f11 tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:30.881490 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e7482e19-9fda-42dd-9e93-fce871b84f11 tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:30.911957 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-88577171-13cc-440e-bfc0-0e86c12f838a tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '027dcc48e4fd4c36a5004096ad5e43f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dad823e240149edb30e592380249a13', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:26:30.929335 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-88577171-13cc-440e-bfc0-0e86c12f838a tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "GET /compute/v2.1/servers/6dcb4587-0e15-4a2d-b8f0-70b459fb6664" status: 200 len: 1308 microversion: 2.1 time: 0.351633 Jul 16 21:26:30.930060 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1912/3831] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:26:30 2022] GET /compute/v2.1/servers/6dcb4587-0e15-4a2d-b8f0-70b459fb6664 => generated 1308 bytes in 353 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:26:31.003503 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-e7482e19-9fda-42dd-9e93-fce871b84f11 tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb1f6532421745a4b0de70356748a647', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7d5a798ee507429bb70566356229535d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:26:31.013103 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-e7482e19-9fda-42dd-9e93-fce871b84f11 tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] 10.222.0.16 "GET /compute/v2.1/servers/9a184cb7-2548-41d5-8d37-8e72dfb3d75a" status: 200 len: 1272 microversion: 2.1 time: 0.149406 Jul 16 21:26:31.013748 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1920/3832] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:26:30 2022] GET /compute/v2.1/servers/9a184cb7-2548-41d5-8d37-8e72dfb3d75a => generated 1272 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:26:31.167131 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-20346caa-135b-46ad-a060-047ee327e335 tempest-VolumesNegativeTest-1268631578 tempest-VolumesNegativeTest-1268631578-project] Action: 'create', calling method: .version_select at 0x7fdd9c6a51f0>, body: {"volume": {"size": "#$%", "display_name": "tempest-VolumesNegativeTest-Volume-114135835", "metadata": {"Type": "work"}}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:26:31.170385 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-20346caa-135b-46ad-a060-047ee327e335 tempest-VolumesNegativeTest-1268631578 tempest-VolumesNegativeTest-1268631578-project] Request method failure captured: Jul 16 21:26:31.170385 ncdv-master-1 devstack@n-api.service[138680]: request: POST /compute/v2.1/os-volumes HTTP/1.1 Jul 16 21:26:31.170385 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:26:31.170385 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:26:31.170385 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:26:31.170385 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 121 Jul 16 21:26:31.170385 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:26:31.170385 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:26:31.170385 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:26:31.170385 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:26:31.170385 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0y0GbVcW-OaD2AM1uORhz5Gir-nJwmrhrYfNCEQad3KpHabIrQkEWR1x5iwMRkEbg-QI7gc6Nmd8QnPb7vXqjcziI-ZV_sbbOy7xiaMpmdx5UVwno6cuUGXOHpPaBk_lgsdouZoixX3XjVb0aNfKZbW4BKs5oG0fnIa7v6UjhrpkxB0 Jul 16 21:26:31.170385 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:26:31.170385 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:26:31.170385 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:26:31.170385 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:26:31.170385 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:26:31.170385 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:26:31.170385 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: 856b5fd8e4544b028f50cfdcaa39181a Jul 16 21:26:31.170385 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-VolumesNegativeTest-1268631578 Jul 16 21:26:31.170385 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:26:31.170385 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:26:31.170385 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/856b5fd8e4544b028f50cfdcaa39181a"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/856b5fd8e4544b028f50cfdcaa39181a"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_856b5fd8e4544b028f50cfdcaa39181a"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/856b5fd8e4544b028f50cfdcaa39181a"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:26:31.170385 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-VolumesNegativeTest-1268631578 Jul 16 21:26:31.170385 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: 856b5fd8e4544b028f50cfdcaa39181a Jul 16 21:26:31.170385 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-VolumesNegativeTest-1268631578 Jul 16 21:26:31.170385 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-VolumesNegativeTest-1268631578-project Jul 16 21:26:31.170385 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:26:31.170385 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:26:31.170385 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: 4bca4052ad6940e78c6b9a04748d1ff4 Jul 16 21:26:31.170385 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-VolumesNegativeTest-1268631578-project Jul 16 21:26:31.170385 ncdv-master-1 devstack@n-api.service[138680]: Jul 16 21:26:31.170385 ncdv-master-1 devstack@n-api.service[138680]: {"volume": {"size": "#$%", "display_name": "tempest-VolumesNegativeTest-Volume-114135835", "metadata": {"Type": "work"}}} Jul 16 21:26:31.170385 ncdv-master-1 devstack@n-api.service[138680]: method: Jul 16 21:26:31.170385 ncdv-master-1 devstack@n-api.service[138680]: exception: Invalid input for field/attribute size. Value: #$%. '#$%' does not match '^[0-9]+$' Jul 16 21:26:31.170385 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'body': {'volume': {'size': '#$%', 'display_name': 'tempest-VolumesNegativeTest-Volume-114135835', 'metadata': {'Type': 'work'}}}} Jul 16 21:26:31.170385 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:26:31.170385 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:26:31.170385 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 279, in validate Jul 16 21:26:31.170385 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Jul 16 21:26:31.170385 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/lib/python3/dist-packages/jsonschema/validators.py", line 353, in validate Jul 16 21:26:31.170385 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise error Jul 16 21:26:31.170385 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: '#$%' does not match '^[0-9]+$' Jul 16 21:26:31.170385 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:26:31.170385 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Failed validating 'pattern' in schema['properties']['volume']['properties']['size']: Jul 16 21:26:31.170385 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi {'minimum': 1, 'pattern': '^[0-9]+$', 'type': ['integer', 'string']} Jul 16 21:26:31.170385 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:26:31.170385 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi On instance['volume']: Jul 16 21:26:31.170385 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi '#$%' Jul 16 21:26:31.170385 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:26:31.170385 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:26:31.170385 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:26:31.170385 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:26:31.170385 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:26:31.170385 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:26:31.170385 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:26:31.170385 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:26:31.170385 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Jul 16 21:26:31.170385 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Jul 16 21:26:31.170385 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:26:31.170385 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:26:31.170385 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Jul 16 21:26:31.170385 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Jul 16 21:26:31.170385 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Jul 16 21:26:31.170385 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Jul 16 21:26:31.170385 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 308, in validate Jul 16 21:26:31.170385 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Jul 16 21:26:31.174977 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute size. Value: #$%. '#$%' does not match '^[0-9]+$' Jul 16 21:26:31.174977 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:26:31.174977 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:26:31.174977 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:26:31.174977 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:26:31.174977 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:26:31.174977 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:26:31.174977 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Jul 16 21:26:31.174977 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Jul 16 21:26:31.174977 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute size. Value: #$%. '#$%' does not match '^[0-9]+$' Jul 16 21:26:31.174977 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:26:31.183344 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-20346caa-135b-46ad-a060-047ee327e335 tempest-VolumesNegativeTest-1268631578 tempest-VolumesNegativeTest-1268631578-project] Returning 400 to user: Invalid input for field/attribute size. Value: #$%. '#$%' does not match '^[0-9]+$' {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:26:31.183344 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-20346caa-135b-46ad-a060-047ee327e335 tempest-VolumesNegativeTest-1268631578 tempest-VolumesNegativeTest-1268631578-project] 10.222.0.16 "POST /compute/v2.1/os-volumes" status: 400 len: 127 microversion: 2.1 time: 0.194630 Jul 16 21:26:31.183344 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1913/3833] 10.222.0.16 () {60 vars in 1155 bytes} [Sat Jul 16 21:26:30 2022] POST /compute/v2.1/os-volumes => generated 127 bytes in 196 msecs (HTTP/1.1 400) 9 headers in 381 bytes (2 switches on core 0) Jul 16 21:26:31.193330 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-3d9f00e7-5fc3-42bb-9c7f-ad9d0d350a8a tempest-VolumesNegativeTest-1268631578 tempest-VolumesNegativeTest-1268631578-project] Action: 'create', calling method: .version_select at 0x7fdd9c26f160>, body: {"volume": {"size": "0", "display_name": "tempest-VolumesNegativeTest-Volume-987330755", "metadata": {"Type": "work"}}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:26:31.198482 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-3d9f00e7-5fc3-42bb-9c7f-ad9d0d350a8a tempest-VolumesNegativeTest-1268631578 tempest-VolumesNegativeTest-1268631578-project] Request method failure captured: Jul 16 21:26:31.198482 ncdv-master-1 devstack@n-api.service[138681]: request: POST /compute/v2.1/os-volumes HTTP/1.1 Jul 16 21:26:31.198482 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:26:31.198482 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:26:31.198482 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:26:31.198482 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 119 Jul 16 21:26:31.198482 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:26:31.198482 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:26:31.198482 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:26:31.198482 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:26:31.198482 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0y0GbVcW-OaD2AM1uORhz5Gir-nJwmrhrYfNCEQad3KpHabIrQkEWR1x5iwMRkEbg-QI7gc6Nmd8QnPb7vXqjcziI-ZV_sbbOy7xiaMpmdx5UVwno6cuUGXOHpPaBk_lgsdouZoixX3XjVb0aNfKZbW4BKs5oG0fnIa7v6UjhrpkxB0 Jul 16 21:26:31.198482 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:26:31.198482 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:26:31.198482 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:26:31.198482 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:26:31.198482 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:26:31.198482 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:26:31.198482 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: 856b5fd8e4544b028f50cfdcaa39181a Jul 16 21:26:31.198482 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-VolumesNegativeTest-1268631578 Jul 16 21:26:31.198482 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:26:31.198482 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:26:31.198482 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/856b5fd8e4544b028f50cfdcaa39181a"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/856b5fd8e4544b028f50cfdcaa39181a"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_856b5fd8e4544b028f50cfdcaa39181a"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/856b5fd8e4544b028f50cfdcaa39181a"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:26:31.198482 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-VolumesNegativeTest-1268631578 Jul 16 21:26:31.198482 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: 856b5fd8e4544b028f50cfdcaa39181a Jul 16 21:26:31.198482 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-VolumesNegativeTest-1268631578 Jul 16 21:26:31.198482 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-VolumesNegativeTest-1268631578-project Jul 16 21:26:31.198482 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:26:31.198482 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:26:31.198482 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: 4bca4052ad6940e78c6b9a04748d1ff4 Jul 16 21:26:31.198482 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-VolumesNegativeTest-1268631578-project Jul 16 21:26:31.198482 ncdv-master-1 devstack@n-api.service[138681]: Jul 16 21:26:31.198482 ncdv-master-1 devstack@n-api.service[138681]: {"volume": {"size": "0", "display_name": "tempest-VolumesNegativeTest-Volume-987330755", "metadata": {"Type": "work"}}} Jul 16 21:26:31.198482 ncdv-master-1 devstack@n-api.service[138681]: method: Jul 16 21:26:31.198482 ncdv-master-1 devstack@n-api.service[138681]: exception: Invalid input for field/attribute size. Value: 0. 0 is less than the minimum of 1 Jul 16 21:26:31.198482 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'body': {'volume': {'size': '0', 'display_name': 'tempest-VolumesNegativeTest-Volume-987330755', 'metadata': {'Type': 'work'}}}} Jul 16 21:26:31.198482 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:26:31.198482 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:26:31.198482 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 279, in validate Jul 16 21:26:31.198482 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Jul 16 21:26:31.198482 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/lib/python3/dist-packages/jsonschema/validators.py", line 353, in validate Jul 16 21:26:31.198482 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise error Jul 16 21:26:31.198482 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: 0 is less than the minimum of 1 Jul 16 21:26:31.198482 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:26:31.198482 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Failed validating 'minimum' in schema['properties']['volume']['properties']['size']: Jul 16 21:26:31.198482 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi {'minimum': 1, 'pattern': '^[0-9]+$', 'type': ['integer', 'string']} Jul 16 21:26:31.198482 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:26:31.198482 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi On instance['volume']: Jul 16 21:26:31.198482 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi '0' Jul 16 21:26:31.198482 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:26:31.198482 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:26:31.198482 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:26:31.198482 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:26:31.198482 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:26:31.198482 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:26:31.198482 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:26:31.198482 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:26:31.198482 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Jul 16 21:26:31.198482 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Jul 16 21:26:31.198482 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:26:31.198482 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:26:31.198482 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Jul 16 21:26:31.198482 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Jul 16 21:26:31.198482 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Jul 16 21:26:31.198482 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Jul 16 21:26:31.198482 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 308, in validate Jul 16 21:26:31.198482 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Jul 16 21:26:31.209579 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute size. Value: 0. 0 is less than the minimum of 1 Jul 16 21:26:31.209579 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:26:31.209579 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:26:31.209579 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:26:31.209579 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:26:31.209579 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:26:31.209579 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:26:31.209579 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Jul 16 21:26:31.209579 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Jul 16 21:26:31.209579 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute size. Value: 0. 0 is less than the minimum of 1 Jul 16 21:26:31.209579 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:26:31.209579 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-3d9f00e7-5fc3-42bb-9c7f-ad9d0d350a8a tempest-VolumesNegativeTest-1268631578 tempest-VolumesNegativeTest-1268631578-project] Returning 400 to user: Invalid input for field/attribute size. Value: 0. 0 is less than the minimum of 1 {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:26:31.209579 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-3d9f00e7-5fc3-42bb-9c7f-ad9d0d350a8a tempest-VolumesNegativeTest-1268631578 tempest-VolumesNegativeTest-1268631578-project] 10.222.0.16 "POST /compute/v2.1/os-volumes" status: 400 len: 125 microversion: 2.1 time: 0.014693 Jul 16 21:26:31.209579 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1921/3834] 10.222.0.16 () {60 vars in 1155 bytes} [Sat Jul 16 21:26:31 2022] POST /compute/v2.1/os-volumes => generated 125 bytes in 17 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Jul 16 21:26:31.212714 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell0:SELECT=4 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:26:31.217153 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-79d99b4e-1a87-4de1-a360-c5712c2753b5 tempest-VolumesNegativeTest-1268631578 tempest-VolumesNegativeTest-1268631578-project] Action: 'create', calling method: .version_select at 0x7fdda2b2b430>, body: {"volume": {"size": "", "display_name": "tempest-VolumesNegativeTest-Volume-1699736609", "metadata": {"Type": "work"}}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:26:31.219549 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-79d99b4e-1a87-4de1-a360-c5712c2753b5 tempest-VolumesNegativeTest-1268631578 tempest-VolumesNegativeTest-1268631578-project] Request method failure captured: Jul 16 21:26:31.219549 ncdv-master-1 devstack@n-api.service[138680]: request: POST /compute/v2.1/os-volumes HTTP/1.1 Jul 16 21:26:31.219549 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:26:31.219549 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:26:31.219549 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:26:31.219549 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 119 Jul 16 21:26:31.219549 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:26:31.219549 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:26:31.219549 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:26:31.219549 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:26:31.219549 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0y0GbVcW-OaD2AM1uORhz5Gir-nJwmrhrYfNCEQad3KpHabIrQkEWR1x5iwMRkEbg-QI7gc6Nmd8QnPb7vXqjcziI-ZV_sbbOy7xiaMpmdx5UVwno6cuUGXOHpPaBk_lgsdouZoixX3XjVb0aNfKZbW4BKs5oG0fnIa7v6UjhrpkxB0 Jul 16 21:26:31.219549 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:26:31.219549 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:26:31.219549 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:26:31.219549 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:26:31.219549 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:26:31.219549 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:26:31.219549 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: 856b5fd8e4544b028f50cfdcaa39181a Jul 16 21:26:31.219549 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-VolumesNegativeTest-1268631578 Jul 16 21:26:31.219549 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:26:31.219549 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:26:31.219549 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/856b5fd8e4544b028f50cfdcaa39181a"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/856b5fd8e4544b028f50cfdcaa39181a"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_856b5fd8e4544b028f50cfdcaa39181a"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/856b5fd8e4544b028f50cfdcaa39181a"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:26:31.219549 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-VolumesNegativeTest-1268631578 Jul 16 21:26:31.219549 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: 856b5fd8e4544b028f50cfdcaa39181a Jul 16 21:26:31.219549 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-VolumesNegativeTest-1268631578 Jul 16 21:26:31.219549 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-VolumesNegativeTest-1268631578-project Jul 16 21:26:31.219549 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:26:31.219549 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:26:31.219549 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: 4bca4052ad6940e78c6b9a04748d1ff4 Jul 16 21:26:31.219549 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-VolumesNegativeTest-1268631578-project Jul 16 21:26:31.219549 ncdv-master-1 devstack@n-api.service[138680]: Jul 16 21:26:31.219549 ncdv-master-1 devstack@n-api.service[138680]: {"volume": {"size": "", "display_name": "tempest-VolumesNegativeTest-Volume-1699736609", "metadata": {"Type": "work"}}} Jul 16 21:26:31.219549 ncdv-master-1 devstack@n-api.service[138680]: method: Jul 16 21:26:31.219549 ncdv-master-1 devstack@n-api.service[138680]: exception: Invalid input for field/attribute size. Value: . '' does not match '^[0-9]+$' Jul 16 21:26:31.219549 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'body': {'volume': {'size': '', 'display_name': 'tempest-VolumesNegativeTest-Volume-1699736609', 'metadata': {'Type': 'work'}}}} Jul 16 21:26:31.219549 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:26:31.219549 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:26:31.219549 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 279, in validate Jul 16 21:26:31.219549 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Jul 16 21:26:31.219549 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/lib/python3/dist-packages/jsonschema/validators.py", line 353, in validate Jul 16 21:26:31.219549 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise error Jul 16 21:26:31.219549 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: '' does not match '^[0-9]+$' Jul 16 21:26:31.219549 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:26:31.219549 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Failed validating 'pattern' in schema['properties']['volume']['properties']['size']: Jul 16 21:26:31.219549 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi {'minimum': 1, 'pattern': '^[0-9]+$', 'type': ['integer', 'string']} Jul 16 21:26:31.219549 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:26:31.219549 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi On instance['volume']: Jul 16 21:26:31.219549 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi '' Jul 16 21:26:31.219549 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:26:31.219549 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:26:31.219549 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:26:31.219549 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:26:31.219549 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:26:31.219549 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:26:31.219549 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:26:31.219549 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:26:31.219549 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Jul 16 21:26:31.219549 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Jul 16 21:26:31.219549 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:26:31.219549 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:26:31.219549 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Jul 16 21:26:31.219549 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Jul 16 21:26:31.219549 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Jul 16 21:26:31.219549 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Jul 16 21:26:31.219549 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 308, in validate Jul 16 21:26:31.219549 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Jul 16 21:26:31.229534 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute size. Value: . '' does not match '^[0-9]+$' Jul 16 21:26:31.229534 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:26:31.229534 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:26:31.229534 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:26:31.229534 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:26:31.229534 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:26:31.229534 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:26:31.229534 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Jul 16 21:26:31.229534 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Jul 16 21:26:31.229534 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute size. Value: . '' does not match '^[0-9]+$' Jul 16 21:26:31.229534 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:26:31.229534 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-79d99b4e-1a87-4de1-a360-c5712c2753b5 tempest-VolumesNegativeTest-1268631578 tempest-VolumesNegativeTest-1268631578-project] Returning 400 to user: Invalid input for field/attribute size. Value: . '' does not match '^[0-9]+$' {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:26:31.229534 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-79d99b4e-1a87-4de1-a360-c5712c2753b5 tempest-VolumesNegativeTest-1268631578 tempest-VolumesNegativeTest-1268631578-project] 10.222.0.16 "POST /compute/v2.1/os-volumes" status: 400 len: 121 microversion: 2.1 time: 0.009045 Jul 16 21:26:31.229534 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1914/3835] 10.222.0.16 () {60 vars in 1155 bytes} [Sat Jul 16 21:26:31 2022] POST /compute/v2.1/os-volumes => generated 121 bytes in 11 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Jul 16 21:26:31.234214 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-f64a7574-1d2d-499c-8612-32d3cc7b04f8 tempest-VolumesNegativeTest-1268631578 tempest-VolumesNegativeTest-1268631578-project] Calling method '.version_select at 0x7fdd9c27d310>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:26:31.279919 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-f64a7574-1d2d-499c-8612-32d3cc7b04f8 tempest-VolumesNegativeTest-1268631578 tempest-VolumesNegativeTest-1268631578-project] HTTP exception thrown: Volume tempest-invalid-182916821 could not be found. Jul 16 21:26:31.290084 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-f64a7574-1d2d-499c-8612-32d3cc7b04f8 tempest-VolumesNegativeTest-1268631578 tempest-VolumesNegativeTest-1268631578-project] Request method failure captured: Jul 16 21:26:31.290084 ncdv-master-1 devstack@n-api.service[138681]: request: DELETE /compute/v2.1/os-volumes/tempest-invalid-182916821 HTTP/1.1 Jul 16 21:26:31.290084 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:26:31.290084 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:26:31.290084 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:26:31.290084 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 21:26:31.290084 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:26:31.290084 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:26:31.290084 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:26:31.290084 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:26:31.290084 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0y0GbVcW-OaD2AM1uORhz5Gir-nJwmrhrYfNCEQad3KpHabIrQkEWR1x5iwMRkEbg-QI7gc6Nmd8QnPb7vXqjcziI-ZV_sbbOy7xiaMpmdx5UVwno6cuUGXOHpPaBk_lgsdouZoixX3XjVb0aNfKZbW4BKs5oG0fnIa7v6UjhrpkxB0 Jul 16 21:26:31.290084 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:26:31.290084 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:26:31.290084 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:26:31.290084 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:26:31.290084 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:26:31.290084 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:26:31.290084 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: 856b5fd8e4544b028f50cfdcaa39181a Jul 16 21:26:31.290084 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-VolumesNegativeTest-1268631578 Jul 16 21:26:31.290084 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:26:31.290084 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:26:31.290084 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/856b5fd8e4544b028f50cfdcaa39181a"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/856b5fd8e4544b028f50cfdcaa39181a"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_856b5fd8e4544b028f50cfdcaa39181a"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/856b5fd8e4544b028f50cfdcaa39181a"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:26:31.290084 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-VolumesNegativeTest-1268631578 Jul 16 21:26:31.290084 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: 856b5fd8e4544b028f50cfdcaa39181a Jul 16 21:26:31.290084 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-VolumesNegativeTest-1268631578 Jul 16 21:26:31.290084 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-VolumesNegativeTest-1268631578-project Jul 16 21:26:31.290084 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:26:31.290084 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:26:31.290084 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: 4bca4052ad6940e78c6b9a04748d1ff4 Jul 16 21:26:31.290084 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-VolumesNegativeTest-1268631578-project Jul 16 21:26:31.290084 ncdv-master-1 devstack@n-api.service[138681]: method: Jul 16 21:26:31.290084 ncdv-master-1 devstack@n-api.service[138681]: exception: Volume tempest-invalid-182916821 could not be found. Jul 16 21:26:31.290084 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': 'tempest-invalid-182916821'} Jul 16 21:26:31.290084 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:26:31.290084 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:26:31.290084 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jul 16 21:26:31.290084 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jul 16 21:26:31.290084 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Jul 16 21:26:31.290084 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Jul 16 21:26:31.290084 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Jul 16 21:26:31.290084 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return self._delete(loc) Jul 16 21:26:31.290084 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Jul 16 21:26:31.290084 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Jul 16 21:26:31.290084 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Jul 16 21:26:31.290084 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Jul 16 21:26:31.290084 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jul 16 21:26:31.290084 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jul 16 21:26:31.290084 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jul 16 21:26:31.290084 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jul 16 21:26:31.290084 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume tempest-invalid-182916821 could not be found. (HTTP 404) (Request-ID: req-27081820-5d96-4207-a591-649bd6f8e708) Jul 16 21:26:31.290084 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:26:31.290084 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:26:31.290084 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:26:31.290084 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:26:31.290084 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 136, in delete Jul 16 21:26:31.290084 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi self.volume_api.delete(context, id) Jul 16 21:26:31.290084 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Jul 16 21:26:31.290084 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Jul 16 21:26:31.290084 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Jul 16 21:26:31.290084 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Jul 16 21:26:31.290084 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Jul 16 21:26:31.290084 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Jul 16 21:26:31.290084 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jul 16 21:26:31.290084 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jul 16 21:26:31.293692 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Jul 16 21:26:31.293692 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Jul 16 21:26:31.293692 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Jul 16 21:26:31.293692 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return self._delete(loc) Jul 16 21:26:31.293692 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Jul 16 21:26:31.293692 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Jul 16 21:26:31.293692 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Jul 16 21:26:31.293692 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Jul 16 21:26:31.293692 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jul 16 21:26:31.293692 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jul 16 21:26:31.293692 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jul 16 21:26:31.293692 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jul 16 21:26:31.293692 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume tempest-invalid-182916821 could not be found. Jul 16 21:26:31.293692 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:26:31.293692 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:26:31.293692 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:26:31.293692 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:26:31.293692 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:26:31.293692 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:26:31.293692 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:26:31.293692 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:26:31.293692 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Jul 16 21:26:31.293692 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Jul 16 21:26:31.293692 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:26:31.293692 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:26:31.293692 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 140, in delete Jul 16 21:26:31.293692 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:26:31.293692 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume tempest-invalid-182916821 could not be found. Jul 16 21:26:31.293692 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:26:31.293692 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:26:31.293692 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:26:31.293692 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:26:31.293692 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:26:31.293692 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:26:31.293692 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:26:31.293692 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:26:31.293692 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume tempest-invalid-182916821 could not be found. Jul 16 21:26:31.293692 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:26:31.299082 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-f64a7574-1d2d-499c-8612-32d3cc7b04f8 tempest-VolumesNegativeTest-1268631578 tempest-VolumesNegativeTest-1268631578-project] Returning 404 to user: Volume tempest-invalid-182916821 could not be found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:26:31.300143 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-f64a7574-1d2d-499c-8612-32d3cc7b04f8 tempest-VolumesNegativeTest-1268631578 tempest-VolumesNegativeTest-1268631578-project] 10.222.0.16 "DELETE /compute/v2.1/os-volumes/tempest-invalid-182916821" status: 404 len: 98 microversion: 2.1 time: 0.069783 Jul 16 21:26:31.305277 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1922/3836] 10.222.0.16 () {58 vars in 1214 bytes} [Sat Jul 16 21:26:31 2022] DELETE /compute/v2.1/os-volumes/tempest-invalid-182916821 => generated 98 bytes in 72 msecs (HTTP/1.1 404) 9 headers in 378 bytes (1 switches on core 0) Jul 16 21:26:31.312456 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-efa50d34-f681-4dc4-aa2a-05b7acf4f58b tempest-VolumesNegativeTest-1268631578 tempest-VolumesNegativeTest-1268631578-project] 10.222.0.16 "DELETE /compute/v2.1/os-volumes/" status: 404 len: 112 microversion: - time: 0.005694 Jul 16 21:26:31.313046 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1915/3837] 10.222.0.16 () {58 vars in 1139 bytes} [Sat Jul 16 21:26:31 2022] DELETE /compute/v2.1/os-volumes/ => generated 112 bytes in 7 msecs (HTTP/1.1 404) 5 headers in 228 bytes (1 switches on core 0) Jul 16 21:26:31.325151 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-68d665c6-07fd-4b48-ba81-01b67e646d2b tempest-VolumesNegativeTest-1268631578 tempest-VolumesNegativeTest-1268631578-project] 10.222.0.16 "GET /compute/v2.1/os-volumes/" status: 404 len: 112 microversion: - time: 0.005291 Jul 16 21:26:31.326596 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1923/3838] 10.222.0.16 () {58 vars in 1136 bytes} [Sat Jul 16 21:26:31 2022] GET /compute/v2.1/os-volumes/ => generated 112 bytes in 9 msecs (HTTP/1.1 404) 5 headers in 228 bytes (1 switches on core 0) Jul 16 21:26:31.340292 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-ab18a453-bf31-4e8c-a07f-3de45158a857 tempest-VolumesNegativeTest-1268631578 tempest-VolumesNegativeTest-1268631578-project] Calling method '.version_select at 0x7fdd9c6a5b80>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:26:31.372814 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-ab18a453-bf31-4e8c-a07f-3de45158a857 tempest-VolumesNegativeTest-1268631578 tempest-VolumesNegativeTest-1268631578-project] HTTP exception thrown: Volume c145f15e-ffc8-4418-a058-968262bde1e9 could not be found. Jul 16 21:26:31.374952 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-ab18a453-bf31-4e8c-a07f-3de45158a857 tempest-VolumesNegativeTest-1268631578 tempest-VolumesNegativeTest-1268631578-project] Request method failure captured: Jul 16 21:26:31.374952 ncdv-master-1 devstack@n-api.service[138680]: request: DELETE /compute/v2.1/os-volumes/c145f15e-ffc8-4418-a058-968262bde1e9 HTTP/1.1 Jul 16 21:26:31.374952 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:26:31.374952 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:26:31.374952 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:26:31.374952 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 0 Jul 16 21:26:31.374952 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:26:31.374952 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:26:31.374952 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:26:31.374952 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:26:31.374952 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0y0GbVcW-OaD2AM1uORhz5Gir-nJwmrhrYfNCEQad3KpHabIrQkEWR1x5iwMRkEbg-QI7gc6Nmd8QnPb7vXqjcziI-ZV_sbbOy7xiaMpmdx5UVwno6cuUGXOHpPaBk_lgsdouZoixX3XjVb0aNfKZbW4BKs5oG0fnIa7v6UjhrpkxB0 Jul 16 21:26:31.374952 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:26:31.374952 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:26:31.374952 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:26:31.374952 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:26:31.374952 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:26:31.374952 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:26:31.374952 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: 856b5fd8e4544b028f50cfdcaa39181a Jul 16 21:26:31.374952 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-VolumesNegativeTest-1268631578 Jul 16 21:26:31.374952 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:26:31.374952 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:26:31.374952 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/856b5fd8e4544b028f50cfdcaa39181a"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/856b5fd8e4544b028f50cfdcaa39181a"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_856b5fd8e4544b028f50cfdcaa39181a"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/856b5fd8e4544b028f50cfdcaa39181a"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:26:31.374952 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-VolumesNegativeTest-1268631578 Jul 16 21:26:31.374952 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: 856b5fd8e4544b028f50cfdcaa39181a Jul 16 21:26:31.374952 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-VolumesNegativeTest-1268631578 Jul 16 21:26:31.374952 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-VolumesNegativeTest-1268631578-project Jul 16 21:26:31.374952 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:26:31.374952 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:26:31.374952 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: 4bca4052ad6940e78c6b9a04748d1ff4 Jul 16 21:26:31.374952 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-VolumesNegativeTest-1268631578-project Jul 16 21:26:31.374952 ncdv-master-1 devstack@n-api.service[138680]: method: Jul 16 21:26:31.374952 ncdv-master-1 devstack@n-api.service[138680]: exception: Volume c145f15e-ffc8-4418-a058-968262bde1e9 could not be found. Jul 16 21:26:31.374952 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': 'c145f15e-ffc8-4418-a058-968262bde1e9'} Jul 16 21:26:31.374952 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:26:31.374952 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:26:31.374952 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jul 16 21:26:31.374952 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jul 16 21:26:31.374952 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Jul 16 21:26:31.374952 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Jul 16 21:26:31.374952 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Jul 16 21:26:31.374952 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return self._delete(loc) Jul 16 21:26:31.374952 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Jul 16 21:26:31.374952 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Jul 16 21:26:31.374952 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Jul 16 21:26:31.374952 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Jul 16 21:26:31.374952 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jul 16 21:26:31.374952 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jul 16 21:26:31.374952 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jul 16 21:26:31.374952 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jul 16 21:26:31.374952 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume c145f15e-ffc8-4418-a058-968262bde1e9 could not be found. (HTTP 404) (Request-ID: req-10029f87-6407-4cad-9f90-b3d0b5dc6932) Jul 16 21:26:31.374952 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:26:31.374952 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:26:31.374952 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:26:31.374952 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:26:31.374952 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 136, in delete Jul 16 21:26:31.374952 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi self.volume_api.delete(context, id) Jul 16 21:26:31.374952 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Jul 16 21:26:31.374952 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Jul 16 21:26:31.374952 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Jul 16 21:26:31.374952 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Jul 16 21:26:31.374952 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Jul 16 21:26:31.374952 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Jul 16 21:26:31.374952 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jul 16 21:26:31.378213 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jul 16 21:26:31.378213 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Jul 16 21:26:31.378213 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Jul 16 21:26:31.378213 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Jul 16 21:26:31.378213 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return self._delete(loc) Jul 16 21:26:31.378213 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Jul 16 21:26:31.378213 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Jul 16 21:26:31.378213 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Jul 16 21:26:31.378213 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Jul 16 21:26:31.378213 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jul 16 21:26:31.378213 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jul 16 21:26:31.378213 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jul 16 21:26:31.378213 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jul 16 21:26:31.378213 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume c145f15e-ffc8-4418-a058-968262bde1e9 could not be found. Jul 16 21:26:31.378213 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:26:31.378213 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:26:31.378213 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:26:31.378213 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:26:31.378213 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:26:31.378213 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:26:31.378213 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:26:31.378213 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:26:31.378213 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Jul 16 21:26:31.378213 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Jul 16 21:26:31.378213 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:26:31.378213 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:26:31.378213 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 140, in delete Jul 16 21:26:31.378213 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:26:31.378213 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume c145f15e-ffc8-4418-a058-968262bde1e9 could not be found. Jul 16 21:26:31.378213 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:26:31.378213 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:26:31.378213 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:26:31.378213 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:26:31.378213 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:26:31.378213 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:26:31.378213 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:26:31.378213 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:26:31.378213 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume c145f15e-ffc8-4418-a058-968262bde1e9 could not be found. Jul 16 21:26:31.378213 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:26:31.378213 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-ab18a453-bf31-4e8c-a07f-3de45158a857 tempest-VolumesNegativeTest-1268631578 tempest-VolumesNegativeTest-1268631578-project] Returning 404 to user: Volume c145f15e-ffc8-4418-a058-968262bde1e9 could not be found. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:26:31.378213 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-ab18a453-bf31-4e8c-a07f-3de45158a857 tempest-VolumesNegativeTest-1268631578 tempest-VolumesNegativeTest-1268631578-project] 10.222.0.16 "DELETE /compute/v2.1/os-volumes/c145f15e-ffc8-4418-a058-968262bde1e9" status: 404 len: 109 microversion: 2.1 time: 0.043893 Jul 16 21:26:31.378213 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1916/3839] 10.222.0.16 () {58 vars in 1247 bytes} [Sat Jul 16 21:26:31 2022] DELETE /compute/v2.1/os-volumes/c145f15e-ffc8-4418-a058-968262bde1e9 => generated 109 bytes in 46 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Jul 16 21:26:31.388196 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-cc193a5f-7d19-4cad-823d-7ce9f49de7ef tempest-VolumesNegativeTest-1268631578 tempest-VolumesNegativeTest-1268631578-project] Calling method '.version_select at 0x7fdd9c26fe50>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:26:31.422667 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-cc193a5f-7d19-4cad-823d-7ce9f49de7ef tempest-VolumesNegativeTest-1268631578 tempest-VolumesNegativeTest-1268631578-project] HTTP exception thrown: Volume e0c4b416-d68b-450e-959d-9096e4fe4b5a could not be found. Jul 16 21:26:31.424650 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-cc193a5f-7d19-4cad-823d-7ce9f49de7ef tempest-VolumesNegativeTest-1268631578 tempest-VolumesNegativeTest-1268631578-project] Request method failure captured: Jul 16 21:26:31.424650 ncdv-master-1 devstack@n-api.service[138681]: request: GET /compute/v2.1/os-volumes/e0c4b416-d68b-450e-959d-9096e4fe4b5a HTTP/1.1 Jul 16 21:26:31.424650 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:26:31.424650 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:26:31.424650 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:26:31.424650 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 21:26:31.424650 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:26:31.424650 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:26:31.424650 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:26:31.424650 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:26:31.424650 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0y0GbVcW-OaD2AM1uORhz5Gir-nJwmrhrYfNCEQad3KpHabIrQkEWR1x5iwMRkEbg-QI7gc6Nmd8QnPb7vXqjcziI-ZV_sbbOy7xiaMpmdx5UVwno6cuUGXOHpPaBk_lgsdouZoixX3XjVb0aNfKZbW4BKs5oG0fnIa7v6UjhrpkxB0 Jul 16 21:26:31.424650 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:26:31.424650 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:26:31.424650 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:26:31.424650 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:26:31.424650 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:26:31.424650 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:26:31.424650 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: 856b5fd8e4544b028f50cfdcaa39181a Jul 16 21:26:31.424650 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-VolumesNegativeTest-1268631578 Jul 16 21:26:31.424650 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:26:31.424650 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:26:31.424650 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/856b5fd8e4544b028f50cfdcaa39181a"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/856b5fd8e4544b028f50cfdcaa39181a"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_856b5fd8e4544b028f50cfdcaa39181a"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/856b5fd8e4544b028f50cfdcaa39181a"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:26:31.424650 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-VolumesNegativeTest-1268631578 Jul 16 21:26:31.424650 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: 856b5fd8e4544b028f50cfdcaa39181a Jul 16 21:26:31.424650 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-VolumesNegativeTest-1268631578 Jul 16 21:26:31.424650 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-VolumesNegativeTest-1268631578-project Jul 16 21:26:31.424650 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:26:31.424650 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:26:31.424650 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: 4bca4052ad6940e78c6b9a04748d1ff4 Jul 16 21:26:31.424650 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-VolumesNegativeTest-1268631578-project Jul 16 21:26:31.424650 ncdv-master-1 devstack@n-api.service[138681]: method: Jul 16 21:26:31.424650 ncdv-master-1 devstack@n-api.service[138681]: exception: Volume e0c4b416-d68b-450e-959d-9096e4fe4b5a could not be found. Jul 16 21:26:31.424650 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': 'e0c4b416-d68b-450e-959d-9096e4fe4b5a'} Jul 16 21:26:31.424650 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:26:31.424650 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:26:31.424650 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jul 16 21:26:31.424650 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jul 16 21:26:31.424650 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Jul 16 21:26:31.424650 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi item = cinderclient( Jul 16 21:26:31.424650 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Jul 16 21:26:31.424650 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Jul 16 21:26:31.424650 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Jul 16 21:26:31.424650 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Jul 16 21:26:31.424650 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Jul 16 21:26:31.424650 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Jul 16 21:26:31.424650 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jul 16 21:26:31.424650 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jul 16 21:26:31.424650 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jul 16 21:26:31.424650 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jul 16 21:26:31.424650 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume e0c4b416-d68b-450e-959d-9096e4fe4b5a could not be found. (HTTP 404) (Request-ID: req-f335abd7-51db-4fe3-889d-8d90ff55fb00) Jul 16 21:26:31.424650 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:26:31.424650 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:26:31.424650 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:26:31.424650 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:26:31.424650 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 120, in show Jul 16 21:26:31.424650 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get(context, id) Jul 16 21:26:31.424650 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Jul 16 21:26:31.424650 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Jul 16 21:26:31.424650 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Jul 16 21:26:31.424650 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Jul 16 21:26:31.424650 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Jul 16 21:26:31.424650 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Jul 16 21:26:31.424650 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jul 16 21:26:31.428631 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jul 16 21:26:31.428631 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Jul 16 21:26:31.428631 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi item = cinderclient( Jul 16 21:26:31.428631 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Jul 16 21:26:31.428631 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Jul 16 21:26:31.428631 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Jul 16 21:26:31.428631 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Jul 16 21:26:31.428631 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Jul 16 21:26:31.428631 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Jul 16 21:26:31.428631 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jul 16 21:26:31.428631 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jul 16 21:26:31.428631 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jul 16 21:26:31.428631 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jul 16 21:26:31.428631 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume e0c4b416-d68b-450e-959d-9096e4fe4b5a could not be found. Jul 16 21:26:31.428631 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:26:31.428631 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:26:31.428631 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:26:31.428631 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:26:31.428631 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:26:31.428631 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:26:31.428631 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:26:31.428631 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:26:31.428631 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Jul 16 21:26:31.428631 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Jul 16 21:26:31.428631 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:26:31.428631 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:26:31.428631 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 122, in show Jul 16 21:26:31.428631 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:26:31.428631 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume e0c4b416-d68b-450e-959d-9096e4fe4b5a could not be found. Jul 16 21:26:31.428631 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:26:31.428631 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:26:31.428631 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:26:31.428631 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:26:31.428631 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:26:31.428631 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:26:31.428631 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:26:31.428631 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:26:31.428631 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume e0c4b416-d68b-450e-959d-9096e4fe4b5a could not be found. Jul 16 21:26:31.428631 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:26:31.428631 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-cc193a5f-7d19-4cad-823d-7ce9f49de7ef tempest-VolumesNegativeTest-1268631578 tempest-VolumesNegativeTest-1268631578-project] Returning 404 to user: Volume e0c4b416-d68b-450e-959d-9096e4fe4b5a could not be found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:26:31.428631 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-cc193a5f-7d19-4cad-823d-7ce9f49de7ef tempest-VolumesNegativeTest-1268631578 tempest-VolumesNegativeTest-1268631578-project] 10.222.0.16 "GET /compute/v2.1/os-volumes/e0c4b416-d68b-450e-959d-9096e4fe4b5a" status: 404 len: 109 microversion: 2.1 time: 0.041780 Jul 16 21:26:31.428631 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1924/3840] 10.222.0.16 () {58 vars in 1244 bytes} [Sat Jul 16 21:26:31 2022] GET /compute/v2.1/os-volumes/e0c4b416-d68b-450e-959d-9096e4fe4b5a => generated 109 bytes in 43 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:26:31.539985 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-d07ae113-e988-4c3c-b7d0-36d08cc5a87c tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:26:31.541694 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-d07ae113-e988-4c3c-b7d0-36d08cc5a87c tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 6ba66466-0da7-4e8d-8ddd-b46f582713fc] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:26:31.549005 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d07ae113-e988-4c3c-b7d0-36d08cc5a87c tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:31.549459 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d07ae113-e988-4c3c-b7d0-36d08cc5a87c tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:31.550267 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d07ae113-e988-4c3c-b7d0-36d08cc5a87c tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:31.551601 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d07ae113-e988-4c3c-b7d0-36d08cc5a87c tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:31.663644 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell0:SELECT=18 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:26:31.855319 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-d07ae113-e988-4c3c-b7d0-36d08cc5a87c tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f2e974ab5aa48dfb4b00f810dcedb63', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd31be8475a6a49d6bfe8132dd050c6c0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:26:31.871725 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-d07ae113-e988-4c3c-b7d0-36d08cc5a87c tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "GET /compute/v2.1/servers/6ba66466-0da7-4e8d-8ddd-b46f582713fc" status: 200 len: 1314 microversion: 2.1 time: 0.336077 Jul 16 21:26:31.882023 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1917/3841] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:26:31 2022] GET /compute/v2.1/servers/6ba66466-0da7-4e8d-8ddd-b46f582713fc => generated 1314 bytes in 345 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:26:31.883964 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-de3bcf1a-432c-4ab7-96d5-596ce046c1ac tempest-InstanceActionsNegativeTestJSON-904555896 tempest-InstanceActionsNegativeTestJSON-904555896-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:26:31.886476 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-de3bcf1a-432c-4ab7-96d5-596ce046c1ac tempest-InstanceActionsNegativeTestJSON-904555896 tempest-InstanceActionsNegativeTestJSON-904555896-project] [instance: ae4b76bc-e7bd-45ff-8716-361dcbbfaa1d] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:26:31.900949 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-de3bcf1a-432c-4ab7-96d5-596ce046c1ac tempest-InstanceActionsNegativeTestJSON-904555896 tempest-InstanceActionsNegativeTestJSON-904555896-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:31.901647 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-de3bcf1a-432c-4ab7-96d5-596ce046c1ac tempest-InstanceActionsNegativeTestJSON-904555896 tempest-InstanceActionsNegativeTestJSON-904555896-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:31.902585 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-de3bcf1a-432c-4ab7-96d5-596ce046c1ac tempest-InstanceActionsNegativeTestJSON-904555896 tempest-InstanceActionsNegativeTestJSON-904555896-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:31.903270 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-de3bcf1a-432c-4ab7-96d5-596ce046c1ac tempest-InstanceActionsNegativeTestJSON-904555896 tempest-InstanceActionsNegativeTestJSON-904555896-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:31.947224 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-f9b98564-e810-4d12-befe-eba95ad1e5f2 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:26:31.947987 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-f9b98564-e810-4d12-befe-eba95ad1e5f2 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: 6dcb4587-0e15-4a2d-b8f0-70b459fb6664] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:26:31.959282 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f9b98564-e810-4d12-befe-eba95ad1e5f2 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:31.965791 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f9b98564-e810-4d12-befe-eba95ad1e5f2 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:31.965791 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f9b98564-e810-4d12-befe-eba95ad1e5f2 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:31.965791 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f9b98564-e810-4d12-befe-eba95ad1e5f2 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:32.306904 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-f9b98564-e810-4d12-befe-eba95ad1e5f2 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '027dcc48e4fd4c36a5004096ad5e43f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dad823e240149edb30e592380249a13', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:26:32.309643 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-de3bcf1a-432c-4ab7-96d5-596ce046c1ac tempest-InstanceActionsNegativeTestJSON-904555896 tempest-InstanceActionsNegativeTestJSON-904555896-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1963f7b0ee0244b2b4f5fc1d8b34f858', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8f2736abb6094c1f83a7aa3efc772506', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:26:32.314860 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-f9b98564-e810-4d12-befe-eba95ad1e5f2 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "GET /compute/v2.1/servers/6dcb4587-0e15-4a2d-b8f0-70b459fb6664" status: 200 len: 1308 microversion: 2.1 time: 0.372078 Jul 16 21:26:32.315477 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1918/3842] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:26:31 2022] GET /compute/v2.1/servers/6dcb4587-0e15-4a2d-b8f0-70b459fb6664 => generated 1308 bytes in 373 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:26:32.318816 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-de3bcf1a-432c-4ab7-96d5-596ce046c1ac tempest-InstanceActionsNegativeTestJSON-904555896 tempest-InstanceActionsNegativeTestJSON-904555896-project] 10.222.0.16 "GET /compute/v2.1/servers/ae4b76bc-e7bd-45ff-8716-361dcbbfaa1d" status: 200 len: 1324 microversion: 2.1 time: 0.441068 Jul 16 21:26:32.319496 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1925/3843] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:26:31 2022] GET /compute/v2.1/servers/ae4b76bc-e7bd-45ff-8716-361dcbbfaa1d => generated 1324 bytes in 442 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:26:32.320996 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-a4fd63dc-bb63-46de-b119-2d44b3be835e tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:26:32.321601 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-a4fd63dc-bb63-46de-b119-2d44b3be835e tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] [instance: 9a184cb7-2548-41d5-8d37-8e72dfb3d75a] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:26:32.328695 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a4fd63dc-bb63-46de-b119-2d44b3be835e tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:32.329455 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a4fd63dc-bb63-46de-b119-2d44b3be835e tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:32.330717 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a4fd63dc-bb63-46de-b119-2d44b3be835e tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:32.331300 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a4fd63dc-bb63-46de-b119-2d44b3be835e tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:32.349348 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=100 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:26:32.566183 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-a4fd63dc-bb63-46de-b119-2d44b3be835e tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb1f6532421745a4b0de70356748a647', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7d5a798ee507429bb70566356229535d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:26:32.577732 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-a4fd63dc-bb63-46de-b119-2d44b3be835e tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] 10.222.0.16 "GET /compute/v2.1/servers/9a184cb7-2548-41d5-8d37-8e72dfb3d75a" status: 200 len: 1314 microversion: 2.1 time: 0.260975 Jul 16 21:26:32.589812 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1919/3844] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:26:32 2022] GET /compute/v2.1/servers/9a184cb7-2548-41d5-8d37-8e72dfb3d75a => generated 1314 bytes in 273 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:26:32.899053 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-738e893a-f4c6-4fb4-99c3-fc998f52ecc0 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:26:32.899821 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-738e893a-f4c6-4fb4-99c3-fc998f52ecc0 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 6ba66466-0da7-4e8d-8ddd-b46f582713fc] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:26:32.909808 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-738e893a-f4c6-4fb4-99c3-fc998f52ecc0 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:32.911675 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-738e893a-f4c6-4fb4-99c3-fc998f52ecc0 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:32.912749 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-738e893a-f4c6-4fb4-99c3-fc998f52ecc0 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:32.913309 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-738e893a-f4c6-4fb4-99c3-fc998f52ecc0 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:32.963752 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-e5cec7e8-a401-48b5-b206-32f7e5ebbe7b req-6d4d5177-b7a0-476d-baea-b7c1a83ed903 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "6ba66466-0da7-4e8d-8ddd-b46f582713fc", "name": "network-vif-plugged", "status": "completed", "tag": "20de36ec-25f9-4437-b0d2-08b6125425c3"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:26:32.974191 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-e5cec7e8-a401-48b5-b206-32f7e5ebbe7b req-6d4d5177-b7a0-476d-baea-b7c1a83ed903 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:32.974646 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-e5cec7e8-a401-48b5-b206-32f7e5ebbe7b req-6d4d5177-b7a0-476d-baea-b7c1a83ed903 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:32.987709 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-e5cec7e8-a401-48b5-b206-32f7e5ebbe7b req-6d4d5177-b7a0-476d-baea-b7c1a83ed903 service nova] Creating event network-vif-plugged:20de36ec-25f9-4437-b0d2-08b6125425c3 for instance 6ba66466-0da7-4e8d-8ddd-b46f582713fc on nch2-master-1 Jul 16 21:26:32.993493 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-e5cec7e8-a401-48b5-b206-32f7e5ebbe7b req-6d4d5177-b7a0-476d-baea-b7c1a83ed903 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.034256 Jul 16 21:26:32.994126 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1920/3845] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:26:32 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 16 21:26:33.334161 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-9835a30f-4eef-41cc-8012-954c14bb44f0 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:26:33.335718 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-9835a30f-4eef-41cc-8012-954c14bb44f0 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: 6dcb4587-0e15-4a2d-b8f0-70b459fb6664] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:26:33.343885 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9835a30f-4eef-41cc-8012-954c14bb44f0 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:33.344482 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9835a30f-4eef-41cc-8012-954c14bb44f0 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:33.345316 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9835a30f-4eef-41cc-8012-954c14bb44f0 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:33.345996 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9835a30f-4eef-41cc-8012-954c14bb44f0 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:33.621503 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-738e893a-f4c6-4fb4-99c3-fc998f52ecc0 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f2e974ab5aa48dfb4b00f810dcedb63', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd31be8475a6a49d6bfe8132dd050c6c0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:26:33.630199 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-738e893a-f4c6-4fb4-99c3-fc998f52ecc0 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "GET /compute/v2.1/servers/6ba66466-0da7-4e8d-8ddd-b46f582713fc" status: 200 len: 1314 microversion: 2.1 time: 0.736052 Jul 16 21:26:33.630790 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1926/3846] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:26:32 2022] GET /compute/v2.1/servers/6ba66466-0da7-4e8d-8ddd-b46f582713fc => generated 1314 bytes in 738 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:26:33.635902 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-6e1aacbd-0dd2-4526-a9f3-0ac16163b6a5 tempest-InstanceActionsNegativeTestJSON-904555896 tempest-InstanceActionsNegativeTestJSON-904555896-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:26:33.636518 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-6e1aacbd-0dd2-4526-a9f3-0ac16163b6a5 tempest-InstanceActionsNegativeTestJSON-904555896 tempest-InstanceActionsNegativeTestJSON-904555896-project] [instance: ae4b76bc-e7bd-45ff-8716-361dcbbfaa1d] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:26:33.642522 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6e1aacbd-0dd2-4526-a9f3-0ac16163b6a5 tempest-InstanceActionsNegativeTestJSON-904555896 tempest-InstanceActionsNegativeTestJSON-904555896-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:33.643162 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6e1aacbd-0dd2-4526-a9f3-0ac16163b6a5 tempest-InstanceActionsNegativeTestJSON-904555896 tempest-InstanceActionsNegativeTestJSON-904555896-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:33.643938 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6e1aacbd-0dd2-4526-a9f3-0ac16163b6a5 tempest-InstanceActionsNegativeTestJSON-904555896 tempest-InstanceActionsNegativeTestJSON-904555896-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:33.644397 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6e1aacbd-0dd2-4526-a9f3-0ac16163b6a5 tempest-InstanceActionsNegativeTestJSON-904555896 tempest-InstanceActionsNegativeTestJSON-904555896-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:33.684764 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-9835a30f-4eef-41cc-8012-954c14bb44f0 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '027dcc48e4fd4c36a5004096ad5e43f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dad823e240149edb30e592380249a13', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:26:33.694876 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-9835a30f-4eef-41cc-8012-954c14bb44f0 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "GET /compute/v2.1/servers/6dcb4587-0e15-4a2d-b8f0-70b459fb6664" status: 200 len: 1308 microversion: 2.1 time: 0.365645 Jul 16 21:26:33.705757 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1921/3847] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:26:33 2022] GET /compute/v2.1/servers/6dcb4587-0e15-4a2d-b8f0-70b459fb6664 => generated 1308 bytes in 377 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:26:33.711836 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-132a6aa6-c0a2-4ddc-9dae-21626b3a8fe0 tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:26:33.712505 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-132a6aa6-c0a2-4ddc-9dae-21626b3a8fe0 tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] [instance: 9a184cb7-2548-41d5-8d37-8e72dfb3d75a] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:26:33.721338 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-132a6aa6-c0a2-4ddc-9dae-21626b3a8fe0 tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:33.723174 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-132a6aa6-c0a2-4ddc-9dae-21626b3a8fe0 tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:33.723636 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-132a6aa6-c0a2-4ddc-9dae-21626b3a8fe0 tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:33.724229 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-132a6aa6-c0a2-4ddc-9dae-21626b3a8fe0 tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:33.965690 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-6e1aacbd-0dd2-4526-a9f3-0ac16163b6a5 tempest-InstanceActionsNegativeTestJSON-904555896 tempest-InstanceActionsNegativeTestJSON-904555896-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1963f7b0ee0244b2b4f5fc1d8b34f858', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8f2736abb6094c1f83a7aa3efc772506', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:26:33.978934 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-6e1aacbd-0dd2-4526-a9f3-0ac16163b6a5 tempest-InstanceActionsNegativeTestJSON-904555896 tempest-InstanceActionsNegativeTestJSON-904555896-project] 10.222.0.16 "GET /compute/v2.1/servers/ae4b76bc-e7bd-45ff-8716-361dcbbfaa1d" status: 200 len: 1324 microversion: 2.1 time: 0.346455 Jul 16 21:26:33.980066 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1927/3848] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:26:33 2022] GET /compute/v2.1/servers/ae4b76bc-e7bd-45ff-8716-361dcbbfaa1d => generated 1324 bytes in 349 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:26:33.986619 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-7fa4c8bd-b4be-48cd-8313-37dd96463842 req-038eb7d6-1c1b-4ede-aeb5-6387c3d9130b service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "9a184cb7-2548-41d5-8d37-8e72dfb3d75a", "tag": "80734127-7d61-49c9-8895-1044cdd4e336"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:26:34.038456 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-7fa4c8bd-b4be-48cd-8313-37dd96463842 req-038eb7d6-1c1b-4ede-aeb5-6387c3d9130b service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:34.038456 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-7fa4c8bd-b4be-48cd-8313-37dd96463842 req-038eb7d6-1c1b-4ede-aeb5-6387c3d9130b service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:34.038456 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-7fa4c8bd-b4be-48cd-8313-37dd96463842 req-038eb7d6-1c1b-4ede-aeb5-6387c3d9130b service nova] Creating event network-changed:80734127-7d61-49c9-8895-1044cdd4e336 for instance 9a184cb7-2548-41d5-8d37-8e72dfb3d75a on nch1-master-1 Jul 16 21:26:34.042319 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-7fa4c8bd-b4be-48cd-8313-37dd96463842 req-038eb7d6-1c1b-4ede-aeb5-6387c3d9130b service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.060230 Jul 16 21:26:34.043538 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1928/3849] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:26:33 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 63 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:26:34.049334 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-132a6aa6-c0a2-4ddc-9dae-21626b3a8fe0 tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb1f6532421745a4b0de70356748a647', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7d5a798ee507429bb70566356229535d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:26:34.058943 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-132a6aa6-c0a2-4ddc-9dae-21626b3a8fe0 tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] 10.222.0.16 "GET /compute/v2.1/servers/9a184cb7-2548-41d5-8d37-8e72dfb3d75a" status: 200 len: 1314 microversion: 2.1 time: 0.352012 Jul 16 21:26:34.059590 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1922/3850] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:26:33 2022] GET /compute/v2.1/servers/9a184cb7-2548-41d5-8d37-8e72dfb3d75a => generated 1314 bytes in 353 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:26:34.649561 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-50249911-02dd-4782-b561-a4a34b150891 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:26:34.650284 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-50249911-02dd-4782-b561-a4a34b150891 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 6ba66466-0da7-4e8d-8ddd-b46f582713fc] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:26:34.657539 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-50249911-02dd-4782-b561-a4a34b150891 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:34.658251 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-50249911-02dd-4782-b561-a4a34b150891 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:34.659245 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-50249911-02dd-4782-b561-a4a34b150891 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:34.659835 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-50249911-02dd-4782-b561-a4a34b150891 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:34.714744 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-f15f568e-b19c-4ffc-adfe-49ce4d18b2a0 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:26:34.715455 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-f15f568e-b19c-4ffc-adfe-49ce4d18b2a0 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: 6dcb4587-0e15-4a2d-b8f0-70b459fb6664] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:26:34.728001 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f15f568e-b19c-4ffc-adfe-49ce4d18b2a0 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:34.728455 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f15f568e-b19c-4ffc-adfe-49ce4d18b2a0 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:34.731446 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f15f568e-b19c-4ffc-adfe-49ce4d18b2a0 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:34.731903 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f15f568e-b19c-4ffc-adfe-49ce4d18b2a0 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:34.897985 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-50249911-02dd-4782-b561-a4a34b150891 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f2e974ab5aa48dfb4b00f810dcedb63', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd31be8475a6a49d6bfe8132dd050c6c0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:26:34.902581 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=100 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:26:34.913424 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-50249911-02dd-4782-b561-a4a34b150891 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "GET /compute/v2.1/servers/6ba66466-0da7-4e8d-8ddd-b46f582713fc" status: 200 len: 1314 microversion: 2.1 time: 0.267337 Jul 16 21:26:34.914225 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1929/3851] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:26:34 2022] GET /compute/v2.1/servers/6ba66466-0da7-4e8d-8ddd-b46f582713fc => generated 1314 bytes in 272 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:26:34.975326 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-f15f568e-b19c-4ffc-adfe-49ce4d18b2a0 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '027dcc48e4fd4c36a5004096ad5e43f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dad823e240149edb30e592380249a13', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:26:34.987333 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-f15f568e-b19c-4ffc-adfe-49ce4d18b2a0 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "GET /compute/v2.1/servers/6dcb4587-0e15-4a2d-b8f0-70b459fb6664" status: 200 len: 1308 microversion: 2.1 time: 0.278064 Jul 16 21:26:34.988193 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1923/3852] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:26:34 2022] GET /compute/v2.1/servers/6dcb4587-0e15-4a2d-b8f0-70b459fb6664 => generated 1308 bytes in 280 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:26:35.006849 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-647dfef3-701e-423c-8f7f-6fca0f731238 tempest-InstanceActionsNegativeTestJSON-904555896 tempest-InstanceActionsNegativeTestJSON-904555896-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:26:35.007610 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-647dfef3-701e-423c-8f7f-6fca0f731238 tempest-InstanceActionsNegativeTestJSON-904555896 tempest-InstanceActionsNegativeTestJSON-904555896-project] [instance: ae4b76bc-e7bd-45ff-8716-361dcbbfaa1d] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:26:35.016541 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-647dfef3-701e-423c-8f7f-6fca0f731238 tempest-InstanceActionsNegativeTestJSON-904555896 tempest-InstanceActionsNegativeTestJSON-904555896-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:35.017053 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-647dfef3-701e-423c-8f7f-6fca0f731238 tempest-InstanceActionsNegativeTestJSON-904555896 tempest-InstanceActionsNegativeTestJSON-904555896-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:35.018278 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-647dfef3-701e-423c-8f7f-6fca0f731238 tempest-InstanceActionsNegativeTestJSON-904555896 tempest-InstanceActionsNegativeTestJSON-904555896-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:35.018556 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-647dfef3-701e-423c-8f7f-6fca0f731238 tempest-InstanceActionsNegativeTestJSON-904555896 tempest-InstanceActionsNegativeTestJSON-904555896-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:35.077354 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-7d00c7b3-7344-4ce1-b525-b4abd03d1feb req-7924b3b2-31d5-44d7-988f-618fdf60bc86 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "ae4b76bc-e7bd-45ff-8716-361dcbbfaa1d", "name": "network-vif-plugged", "status": "completed", "tag": "95067406-61b5-4fed-bcd2-609bccc3cb11"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:26:35.092305 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-7d00c7b3-7344-4ce1-b525-b4abd03d1feb req-7924b3b2-31d5-44d7-988f-618fdf60bc86 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:35.092967 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-7d00c7b3-7344-4ce1-b525-b4abd03d1feb req-7924b3b2-31d5-44d7-988f-618fdf60bc86 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:35.112449 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-7d00c7b3-7344-4ce1-b525-b4abd03d1feb req-7924b3b2-31d5-44d7-988f-618fdf60bc86 service nova] Creating event network-vif-plugged:95067406-61b5-4fed-bcd2-609bccc3cb11 for instance ae4b76bc-e7bd-45ff-8716-361dcbbfaa1d on nch2-master-1 Jul 16 21:26:35.118716 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-7d00c7b3-7344-4ce1-b525-b4abd03d1feb req-7924b3b2-31d5-44d7-988f-618fdf60bc86 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.045554 Jul 16 21:26:35.119627 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1924/3853] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:26:35 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 16 21:26:35.124962 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-22030219-8090-45bf-955e-370039a46992 tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:26:35.125863 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-22030219-8090-45bf-955e-370039a46992 tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] [instance: 9a184cb7-2548-41d5-8d37-8e72dfb3d75a] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:26:35.148991 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-22030219-8090-45bf-955e-370039a46992 tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:35.149670 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-22030219-8090-45bf-955e-370039a46992 tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:35.150695 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-22030219-8090-45bf-955e-370039a46992 tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:35.151343 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-22030219-8090-45bf-955e-370039a46992 tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:35.275394 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-647dfef3-701e-423c-8f7f-6fca0f731238 tempest-InstanceActionsNegativeTestJSON-904555896 tempest-InstanceActionsNegativeTestJSON-904555896-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1963f7b0ee0244b2b4f5fc1d8b34f858', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8f2736abb6094c1f83a7aa3efc772506', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:26:35.287854 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-647dfef3-701e-423c-8f7f-6fca0f731238 tempest-InstanceActionsNegativeTestJSON-904555896 tempest-InstanceActionsNegativeTestJSON-904555896-project] 10.222.0.16 "GET /compute/v2.1/servers/ae4b76bc-e7bd-45ff-8716-361dcbbfaa1d" status: 200 len: 1324 microversion: 2.1 time: 0.287357 Jul 16 21:26:35.288443 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1930/3854] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:26:34 2022] GET /compute/v2.1/servers/ae4b76bc-e7bd-45ff-8716-361dcbbfaa1d => generated 1324 bytes in 290 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:26:35.380942 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-22030219-8090-45bf-955e-370039a46992 tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb1f6532421745a4b0de70356748a647', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7d5a798ee507429bb70566356229535d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:26:35.389024 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-22030219-8090-45bf-955e-370039a46992 tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] 10.222.0.16 "GET /compute/v2.1/servers/9a184cb7-2548-41d5-8d37-8e72dfb3d75a" status: 200 len: 1314 microversion: 2.1 time: 0.268076 Jul 16 21:26:35.389605 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1925/3855] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:26:35 2022] GET /compute/v2.1/servers/9a184cb7-2548-41d5-8d37-8e72dfb3d75a => generated 1314 bytes in 269 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:26:35.930582 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-3ba5d0df-bb88-4b4e-94c8-b65fc21b93aa tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:26:35.931192 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-3ba5d0df-bb88-4b4e-94c8-b65fc21b93aa tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 6ba66466-0da7-4e8d-8ddd-b46f582713fc] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:26:35.938469 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3ba5d0df-bb88-4b4e-94c8-b65fc21b93aa tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:35.939028 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3ba5d0df-bb88-4b4e-94c8-b65fc21b93aa tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:35.939846 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3ba5d0df-bb88-4b4e-94c8-b65fc21b93aa tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:35.940592 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3ba5d0df-bb88-4b4e-94c8-b65fc21b93aa tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:36.006615 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-318bbefc-a6b6-4d7d-a14d-044d7ee9cd32 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:26:36.008003 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-318bbefc-a6b6-4d7d-a14d-044d7ee9cd32 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: 6dcb4587-0e15-4a2d-b8f0-70b459fb6664] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:26:36.016787 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-318bbefc-a6b6-4d7d-a14d-044d7ee9cd32 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:36.017474 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-318bbefc-a6b6-4d7d-a14d-044d7ee9cd32 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:36.018498 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-318bbefc-a6b6-4d7d-a14d-044d7ee9cd32 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:36.019764 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-318bbefc-a6b6-4d7d-a14d-044d7ee9cd32 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:36.154187 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-3ba5d0df-bb88-4b4e-94c8-b65fc21b93aa tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f2e974ab5aa48dfb4b00f810dcedb63', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd31be8475a6a49d6bfe8132dd050c6c0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:26:36.168576 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-3ba5d0df-bb88-4b4e-94c8-b65fc21b93aa tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "GET /compute/v2.1/servers/6ba66466-0da7-4e8d-8ddd-b46f582713fc" status: 200 len: 1501 microversion: 2.1 time: 0.242794 Jul 16 21:26:36.169160 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1931/3856] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:26:35 2022] GET /compute/v2.1/servers/6ba66466-0da7-4e8d-8ddd-b46f582713fc => generated 1501 bytes in 244 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:26:36.187231 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-43181ced-202c-42f5-811e-fa2007481f61 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Action: 'action', calling method: >, body: {"suspend": {}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:26:36.190828 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-318bbefc-a6b6-4d7d-a14d-044d7ee9cd32 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '027dcc48e4fd4c36a5004096ad5e43f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dad823e240149edb30e592380249a13', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:26:36.195149 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-43181ced-202c-42f5-811e-fa2007481f61 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 6ba66466-0da7-4e8d-8ddd-b46f582713fc] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:26:36.201734 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-318bbefc-a6b6-4d7d-a14d-044d7ee9cd32 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "GET /compute/v2.1/servers/6dcb4587-0e15-4a2d-b8f0-70b459fb6664" status: 200 len: 1308 microversion: 2.1 time: 0.199810 Jul 16 21:26:36.202313 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1926/3857] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:26:36 2022] GET /compute/v2.1/servers/6dcb4587-0e15-4a2d-b8f0-70b459fb6664 => generated 1308 bytes in 201 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:26:36.206451 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-43181ced-202c-42f5-811e-fa2007481f61 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:36.207645 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-43181ced-202c-42f5-811e-fa2007481f61 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:36.208568 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-43181ced-202c-42f5-811e-fa2007481f61 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:36.209126 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-43181ced-202c-42f5-811e-fa2007481f61 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:36.234210 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.objects.instance [None req-43181ced-202c-42f5-811e-fa2007481f61 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lazy-loading 'flavor' on Instance uuid 6ba66466-0da7-4e8d-8ddd-b46f582713fc {{(pid=138681) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:26:36.304645 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-337d610d-90d8-44f3-922b-bfdec136edf9 tempest-InstanceActionsNegativeTestJSON-904555896 tempest-InstanceActionsNegativeTestJSON-904555896-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:26:36.305265 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-337d610d-90d8-44f3-922b-bfdec136edf9 tempest-InstanceActionsNegativeTestJSON-904555896 tempest-InstanceActionsNegativeTestJSON-904555896-project] [instance: ae4b76bc-e7bd-45ff-8716-361dcbbfaa1d] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:26:36.311766 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-337d610d-90d8-44f3-922b-bfdec136edf9 tempest-InstanceActionsNegativeTestJSON-904555896 tempest-InstanceActionsNegativeTestJSON-904555896-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:36.312218 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-337d610d-90d8-44f3-922b-bfdec136edf9 tempest-InstanceActionsNegativeTestJSON-904555896 tempest-InstanceActionsNegativeTestJSON-904555896-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:36.313139 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-337d610d-90d8-44f3-922b-bfdec136edf9 tempest-InstanceActionsNegativeTestJSON-904555896 tempest-InstanceActionsNegativeTestJSON-904555896-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:36.313594 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-337d610d-90d8-44f3-922b-bfdec136edf9 tempest-InstanceActionsNegativeTestJSON-904555896 tempest-InstanceActionsNegativeTestJSON-904555896-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:36.455364 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-337d610d-90d8-44f3-922b-bfdec136edf9 tempest-InstanceActionsNegativeTestJSON-904555896 tempest-InstanceActionsNegativeTestJSON-904555896-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1963f7b0ee0244b2b4f5fc1d8b34f858', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8f2736abb6094c1f83a7aa3efc772506', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:26:36.463773 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-337d610d-90d8-44f3-922b-bfdec136edf9 tempest-InstanceActionsNegativeTestJSON-904555896 tempest-InstanceActionsNegativeTestJSON-904555896-project] 10.222.0.16 "GET /compute/v2.1/servers/ae4b76bc-e7bd-45ff-8716-361dcbbfaa1d" status: 200 len: 1324 microversion: 2.1 time: 0.162966 Jul 16 21:26:36.466394 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1927/3858] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:26:36 2022] GET /compute/v2.1/servers/ae4b76bc-e7bd-45ff-8716-361dcbbfaa1d => generated 1324 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:26:36.468835 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-2e4505b2-96e6-4993-be75-fdaeb3719af2 tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:26:36.469442 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-2e4505b2-96e6-4993-be75-fdaeb3719af2 tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] [instance: 9a184cb7-2548-41d5-8d37-8e72dfb3d75a] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:26:36.476202 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2e4505b2-96e6-4993-be75-fdaeb3719af2 tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:36.476652 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2e4505b2-96e6-4993-be75-fdaeb3719af2 tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:36.477411 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2e4505b2-96e6-4993-be75-fdaeb3719af2 tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:36.477848 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2e4505b2-96e6-4993-be75-fdaeb3719af2 tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:36.642206 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-2e4505b2-96e6-4993-be75-fdaeb3719af2 tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb1f6532421745a4b0de70356748a647', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7d5a798ee507429bb70566356229535d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:26:36.651793 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-2e4505b2-96e6-4993-be75-fdaeb3719af2 tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] 10.222.0.16 "GET /compute/v2.1/servers/9a184cb7-2548-41d5-8d37-8e72dfb3d75a" status: 200 len: 1314 microversion: 2.1 time: 0.186407 Jul 16 21:26:36.655813 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1928/3859] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:26:36 2022] GET /compute/v2.1/servers/9a184cb7-2548-41d5-8d37-8e72dfb3d75a => generated 1314 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:26:36.672253 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell0:SELECT=4 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:26:36.778574 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-43181ced-202c-42f5-811e-fa2007481f61 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "POST /compute/v2.1/servers/6ba66466-0da7-4e8d-8ddd-b46f582713fc/action" status: 202 len: 0 microversion: 2.1 time: 0.596532 Jul 16 21:26:36.779347 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1932/3860] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 21:26:36 2022] POST /compute/v2.1/servers/6ba66466-0da7-4e8d-8ddd-b46f582713fc/action => generated 0 bytes in 599 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 21:26:36.788394 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-dbf93504-d807-4cd2-a646-16065273201a tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:26:36.794851 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-dbf93504-d807-4cd2-a646-16065273201a tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 6ba66466-0da7-4e8d-8ddd-b46f582713fc] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:26:36.796224 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-dbf93504-d807-4cd2-a646-16065273201a tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:36.796718 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-dbf93504-d807-4cd2-a646-16065273201a tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:36.797512 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-dbf93504-d807-4cd2-a646-16065273201a tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:36.798138 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-dbf93504-d807-4cd2-a646-16065273201a tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:36.947778 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-dbf93504-d807-4cd2-a646-16065273201a tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f2e974ab5aa48dfb4b00f810dcedb63', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd31be8475a6a49d6bfe8132dd050c6c0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:26:36.955870 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-dbf93504-d807-4cd2-a646-16065273201a tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "GET /compute/v2.1/servers/6ba66466-0da7-4e8d-8ddd-b46f582713fc" status: 200 len: 1509 microversion: 2.1 time: 0.172423 Jul 16 21:26:36.958753 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1929/3861] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:26:36 2022] GET /compute/v2.1/servers/6ba66466-0da7-4e8d-8ddd-b46f582713fc => generated 1509 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:26:37.155934 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-c375a1f5-b9e0-4424-816e-91b68c7a94f3 req-c3337519-5528-4e67-8bef-8d8b505f82af service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "6dcb4587-0e15-4a2d-b8f0-70b459fb6664", "name": "network-vif-plugged", "status": "completed", "tag": "eda9d853-12d2-4057-a4a9-4683ca97fd9d"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:26:37.164355 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-c375a1f5-b9e0-4424-816e-91b68c7a94f3 req-c3337519-5528-4e67-8bef-8d8b505f82af service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:37.164803 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-c375a1f5-b9e0-4424-816e-91b68c7a94f3 req-c3337519-5528-4e67-8bef-8d8b505f82af service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:37.176534 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-c375a1f5-b9e0-4424-816e-91b68c7a94f3 req-c3337519-5528-4e67-8bef-8d8b505f82af service nova] Creating event network-vif-plugged:eda9d853-12d2-4057-a4a9-4683ca97fd9d for instance 6dcb4587-0e15-4a2d-b8f0-70b459fb6664 on nch2-master-1 Jul 16 21:26:37.181259 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-c375a1f5-b9e0-4424-816e-91b68c7a94f3 req-c3337519-5528-4e67-8bef-8d8b505f82af service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.030456 Jul 16 21:26:37.181867 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1933/3862] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:26:37 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 16 21:26:37.219690 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-78f30594-9898-488a-b8fa-765accfed983 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:26:37.220317 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-78f30594-9898-488a-b8fa-765accfed983 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: 6dcb4587-0e15-4a2d-b8f0-70b459fb6664] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:26:37.228896 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-78f30594-9898-488a-b8fa-765accfed983 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:37.229401 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-78f30594-9898-488a-b8fa-765accfed983 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:37.231044 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-78f30594-9898-488a-b8fa-765accfed983 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:37.231642 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-78f30594-9898-488a-b8fa-765accfed983 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:37.473193 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-78f30594-9898-488a-b8fa-765accfed983 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '027dcc48e4fd4c36a5004096ad5e43f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dad823e240149edb30e592380249a13', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:26:37.481633 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-2d4d1e30-ba5b-4bf6-befe-553712e2cbe8 tempest-InstanceActionsNegativeTestJSON-904555896 tempest-InstanceActionsNegativeTestJSON-904555896-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:26:37.482339 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-2d4d1e30-ba5b-4bf6-befe-553712e2cbe8 tempest-InstanceActionsNegativeTestJSON-904555896 tempest-InstanceActionsNegativeTestJSON-904555896-project] [instance: ae4b76bc-e7bd-45ff-8716-361dcbbfaa1d] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:26:37.483351 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-78f30594-9898-488a-b8fa-765accfed983 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "GET /compute/v2.1/servers/6dcb4587-0e15-4a2d-b8f0-70b459fb6664" status: 200 len: 1308 microversion: 2.1 time: 0.267515 Jul 16 21:26:37.483946 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1930/3863] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:26:37 2022] GET /compute/v2.1/servers/6dcb4587-0e15-4a2d-b8f0-70b459fb6664 => generated 1308 bytes in 269 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:26:37.489772 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2d4d1e30-ba5b-4bf6-befe-553712e2cbe8 tempest-InstanceActionsNegativeTestJSON-904555896 tempest-InstanceActionsNegativeTestJSON-904555896-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:37.496909 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2d4d1e30-ba5b-4bf6-befe-553712e2cbe8 tempest-InstanceActionsNegativeTestJSON-904555896 tempest-InstanceActionsNegativeTestJSON-904555896-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:37.504528 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2d4d1e30-ba5b-4bf6-befe-553712e2cbe8 tempest-InstanceActionsNegativeTestJSON-904555896 tempest-InstanceActionsNegativeTestJSON-904555896-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:37.504997 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2d4d1e30-ba5b-4bf6-befe-553712e2cbe8 tempest-InstanceActionsNegativeTestJSON-904555896 tempest-InstanceActionsNegativeTestJSON-904555896-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:37.522675 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell0:SELECT=20 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:26:37.649786 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-2d4d1e30-ba5b-4bf6-befe-553712e2cbe8 tempest-InstanceActionsNegativeTestJSON-904555896 tempest-InstanceActionsNegativeTestJSON-904555896-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1963f7b0ee0244b2b4f5fc1d8b34f858', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8f2736abb6094c1f83a7aa3efc772506', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:26:37.661050 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-2d4d1e30-ba5b-4bf6-befe-553712e2cbe8 tempest-InstanceActionsNegativeTestJSON-904555896 tempest-InstanceActionsNegativeTestJSON-904555896-project] 10.222.0.16 "GET /compute/v2.1/servers/ae4b76bc-e7bd-45ff-8716-361dcbbfaa1d" status: 200 len: 1521 microversion: 2.1 time: 0.183180 Jul 16 21:26:37.662440 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1934/3864] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:26:37 2022] GET /compute/v2.1/servers/ae4b76bc-e7bd-45ff-8716-361dcbbfaa1d => generated 1521 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:26:37.680206 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-434a6264-d54f-4529-a0b6-dd5a2afb8aa0 tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:26:37.680830 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-434a6264-d54f-4529-a0b6-dd5a2afb8aa0 tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] [instance: 9a184cb7-2548-41d5-8d37-8e72dfb3d75a] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:26:37.681444 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-587e7e4b-d51a-4399-a5e5-32952b381198 tempest-InstanceActionsNegativeTestJSON-904555896 tempest-InstanceActionsNegativeTestJSON-904555896-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:26:37.682035 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-587e7e4b-d51a-4399-a5e5-32952b381198 tempest-InstanceActionsNegativeTestJSON-904555896 tempest-InstanceActionsNegativeTestJSON-904555896-project] [instance: ae4b76bc-e7bd-45ff-8716-361dcbbfaa1d] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:26:37.687495 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-434a6264-d54f-4529-a0b6-dd5a2afb8aa0 tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:37.687957 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-434a6264-d54f-4529-a0b6-dd5a2afb8aa0 tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:37.688736 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-434a6264-d54f-4529-a0b6-dd5a2afb8aa0 tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:37.689114 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-587e7e4b-d51a-4399-a5e5-32952b381198 tempest-InstanceActionsNegativeTestJSON-904555896 tempest-InstanceActionsNegativeTestJSON-904555896-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:37.689240 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-434a6264-d54f-4529-a0b6-dd5a2afb8aa0 tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:37.689699 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-587e7e4b-d51a-4399-a5e5-32952b381198 tempest-InstanceActionsNegativeTestJSON-904555896 tempest-InstanceActionsNegativeTestJSON-904555896-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:37.690568 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-587e7e4b-d51a-4399-a5e5-32952b381198 tempest-InstanceActionsNegativeTestJSON-904555896 tempest-InstanceActionsNegativeTestJSON-904555896-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:37.690985 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-587e7e4b-d51a-4399-a5e5-32952b381198 tempest-InstanceActionsNegativeTestJSON-904555896 tempest-InstanceActionsNegativeTestJSON-904555896-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:37.729589 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-587e7e4b-d51a-4399-a5e5-32952b381198 tempest-InstanceActionsNegativeTestJSON-904555896 tempest-InstanceActionsNegativeTestJSON-904555896-project] HTTP exception thrown: Action 999 not found Jul 16 21:26:37.735960 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-587e7e4b-d51a-4399-a5e5-32952b381198 tempest-InstanceActionsNegativeTestJSON-904555896 tempest-InstanceActionsNegativeTestJSON-904555896-project] Request method failure captured: Jul 16 21:26:37.735960 ncdv-master-1 devstack@n-api.service[138681]: request: GET /compute/v2.1/servers/ae4b76bc-e7bd-45ff-8716-361dcbbfaa1d/os-instance-actions/999 HTTP/1.1 Jul 16 21:26:37.735960 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:26:37.735960 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:26:37.735960 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:26:37.735960 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 21:26:37.735960 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:26:37.735960 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:26:37.735960 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:26:37.735960 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:26:37.735960 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0yz8SF1pR_GCLI7xzvIfi_44OK1oBNBDHcBAfLnClU4mtBmY7-QyaipWO1AuqDL5KeIPqXAc56Z1uHiyiWn1sNMEZ7XgCxajnRdgeBkiSesgyWJbY-4VfDxL27uUaG2kAByEz6AQ-BESNileCOHjHH9LY0Rspt_PQAFMIxOsPVQt4JY Jul 16 21:26:37.735960 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:26:37.735960 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:26:37.735960 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:26:37.735960 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:26:37.735960 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:26:37.735960 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:26:37.735960 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: 8f2736abb6094c1f83a7aa3efc772506 Jul 16 21:26:37.735960 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-InstanceActionsNegativeTestJSON-904555896 Jul 16 21:26:37.735960 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:26:37.735960 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:26:37.735960 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/8f2736abb6094c1f83a7aa3efc772506"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/8f2736abb6094c1f83a7aa3efc772506"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_8f2736abb6094c1f83a7aa3efc772506"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/8f2736abb6094c1f83a7aa3efc772506"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:26:37.735960 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-InstanceActionsNegativeTestJSON-904555896 Jul 16 21:26:37.735960 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: 8f2736abb6094c1f83a7aa3efc772506 Jul 16 21:26:37.735960 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-InstanceActionsNegativeTestJSON-904555896 Jul 16 21:26:37.735960 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-InstanceActionsNegativeTestJSON-904555896-project Jul 16 21:26:37.735960 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:26:37.735960 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:26:37.735960 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: 1963f7b0ee0244b2b4f5fc1d8b34f858 Jul 16 21:26:37.735960 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-InstanceActionsNegativeTestJSON-904555896-project Jul 16 21:26:37.735960 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:26:37.735960 ncdv-master-1 devstack@n-api.service[138681]: exception: Action 999 not found Jul 16 21:26:37.735960 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'server_id': 'ae4b76bc-e7bd-45ff-8716-361dcbbfaa1d', 'id': '999'} Jul 16 21:26:37.735960 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:26:37.735960 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:26:37.735960 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:26:37.735960 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:26:37.735960 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:26:37.735960 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:26:37.735960 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:26:37.735960 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:26:37.735960 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/instance_actions.py", line 152, in show Jul 16 21:26:37.735960 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=msg) Jul 16 21:26:37.735960 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Action 999 not found Jul 16 21:26:37.735960 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:26:37.735960 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:26:37.735960 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:26:37.735960 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:26:37.735960 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:26:37.735960 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:26:37.735960 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:26:37.735960 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:26:37.735960 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Action 999 not found Jul 16 21:26:37.735960 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:26:37.740689 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-587e7e4b-d51a-4399-a5e5-32952b381198 tempest-InstanceActionsNegativeTestJSON-904555896 tempest-InstanceActionsNegativeTestJSON-904555896-project] Returning 404 to user: Action 999 not found {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:26:37.740986 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-587e7e4b-d51a-4399-a5e5-32952b381198 tempest-InstanceActionsNegativeTestJSON-904555896 tempest-InstanceActionsNegativeTestJSON-904555896-project] 10.222.0.16 "GET /compute/v2.1/servers/ae4b76bc-e7bd-45ff-8716-361dcbbfaa1d/os-instance-actions/999" status: 404 len: 66 microversion: 2.1 time: 0.063194 Jul 16 21:26:37.741671 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1935/3865] 10.222.0.16 () {58 vars in 1307 bytes} [Sat Jul 16 21:26:37 2022] GET /compute/v2.1/servers/ae4b76bc-e7bd-45ff-8716-361dcbbfaa1d/os-instance-actions/999 => generated 66 bytes in 66 msecs (HTTP/1.1 404) 9 headers in 378 bytes (2 switches on core 0) Jul 16 21:26:37.752470 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-61af9b13-a09b-4fc7-bb19-b0261e952e14 tempest-InstanceActionsNegativeTestJSON-904555896 tempest-InstanceActionsNegativeTestJSON-904555896-project] Calling method '.version_select at 0x7fdd9c24c9d0>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:26:37.753095 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-61af9b13-a09b-4fc7-bb19-b0261e952e14 tempest-InstanceActionsNegativeTestJSON-904555896 tempest-InstanceActionsNegativeTestJSON-904555896-project] [instance: 43386970-b64b-4ba2-8a8d-1eda8ff4216a] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:26:37.759907 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-61af9b13-a09b-4fc7-bb19-b0261e952e14 tempest-InstanceActionsNegativeTestJSON-904555896 tempest-InstanceActionsNegativeTestJSON-904555896-project] HTTP exception thrown: Instance 43386970-b64b-4ba2-8a8d-1eda8ff4216a could not be found. Jul 16 21:26:37.761759 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-61af9b13-a09b-4fc7-bb19-b0261e952e14 tempest-InstanceActionsNegativeTestJSON-904555896 tempest-InstanceActionsNegativeTestJSON-904555896-project] Request method failure captured: Jul 16 21:26:37.761759 ncdv-master-1 devstack@n-api.service[138681]: request: GET /compute/v2.1/servers/43386970-b64b-4ba2-8a8d-1eda8ff4216a/os-instance-actions HTTP/1.1 Jul 16 21:26:37.761759 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:26:37.761759 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:26:37.761759 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:26:37.761759 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 21:26:37.761759 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:26:37.761759 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:26:37.761759 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:26:37.761759 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:26:37.761759 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0yz8SF1pR_GCLI7xzvIfi_44OK1oBNBDHcBAfLnClU4mtBmY7-QyaipWO1AuqDL5KeIPqXAc56Z1uHiyiWn1sNMEZ7XgCxajnRdgeBkiSesgyWJbY-4VfDxL27uUaG2kAByEz6AQ-BESNileCOHjHH9LY0Rspt_PQAFMIxOsPVQt4JY Jul 16 21:26:37.761759 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:26:37.761759 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:26:37.761759 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:26:37.761759 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:26:37.761759 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:26:37.761759 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:26:37.761759 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: 8f2736abb6094c1f83a7aa3efc772506 Jul 16 21:26:37.761759 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-InstanceActionsNegativeTestJSON-904555896 Jul 16 21:26:37.761759 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:26:37.761759 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:26:37.761759 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/8f2736abb6094c1f83a7aa3efc772506"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/8f2736abb6094c1f83a7aa3efc772506"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_8f2736abb6094c1f83a7aa3efc772506"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/8f2736abb6094c1f83a7aa3efc772506"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:26:37.761759 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-InstanceActionsNegativeTestJSON-904555896 Jul 16 21:26:37.761759 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: 8f2736abb6094c1f83a7aa3efc772506 Jul 16 21:26:37.761759 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-InstanceActionsNegativeTestJSON-904555896 Jul 16 21:26:37.761759 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-InstanceActionsNegativeTestJSON-904555896-project Jul 16 21:26:37.761759 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:26:37.761759 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:26:37.761759 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: 1963f7b0ee0244b2b4f5fc1d8b34f858 Jul 16 21:26:37.761759 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-InstanceActionsNegativeTestJSON-904555896-project Jul 16 21:26:37.761759 ncdv-master-1 devstack@n-api.service[138681]: method: Jul 16 21:26:37.761759 ncdv-master-1 devstack@n-api.service[138681]: exception: Instance 43386970-b64b-4ba2-8a8d-1eda8ff4216a could not be found. Jul 16 21:26:37.761759 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'server_id': '43386970-b64b-4ba2-8a8d-1eda8ff4216a'} Jul 16 21:26:37.761759 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:26:37.761759 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:26:37.761759 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:26:37.761759 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:26:37.761759 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:26:37.761759 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:26:37.761759 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2898, in _get_instance Jul 16 21:26:37.761759 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=instance_uuid) Jul 16 21:26:37.761759 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 43386970-b64b-4ba2-8a8d-1eda8ff4216a could not be found. Jul 16 21:26:37.761759 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:26:37.761759 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:26:37.761759 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:26:37.761759 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:26:37.761759 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:26:37.761759 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:26:37.761759 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:26:37.761759 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:26:37.761759 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Jul 16 21:26:37.761759 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Jul 16 21:26:37.761759 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:26:37.761759 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:26:37.761759 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/instance_actions.py", line 89, in index Jul 16 21:26:37.761759 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance(req, context, server_id) Jul 16 21:26:37.761759 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Jul 16 21:26:37.761759 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Jul 16 21:26:37.761759 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/instance_actions.py", line 77, in _get_instance Jul 16 21:26:37.761759 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return common.get_instance(self.compute_api, context, server_id) Jul 16 21:26:37.761759 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:26:37.761759 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:26:37.765911 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 43386970-b64b-4ba2-8a8d-1eda8ff4216a could not be found. Jul 16 21:26:37.765911 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:26:37.765911 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:26:37.765911 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:26:37.765911 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:26:37.765911 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:26:37.765911 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:26:37.765911 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:26:37.765911 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:26:37.765911 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 43386970-b64b-4ba2-8a8d-1eda8ff4216a could not be found. Jul 16 21:26:37.765911 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:26:37.765911 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-61af9b13-a09b-4fc7-bb19-b0261e952e14 tempest-InstanceActionsNegativeTestJSON-904555896 tempest-InstanceActionsNegativeTestJSON-904555896-project] Returning 404 to user: Instance 43386970-b64b-4ba2-8a8d-1eda8ff4216a could not be found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:26:37.765911 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-61af9b13-a09b-4fc7-bb19-b0261e952e14 tempest-InstanceActionsNegativeTestJSON-904555896 tempest-InstanceActionsNegativeTestJSON-904555896-project] 10.222.0.16 "GET /compute/v2.1/servers/43386970-b64b-4ba2-8a8d-1eda8ff4216a/os-instance-actions" status: 404 len: 111 microversion: 2.1 time: 0.015455 Jul 16 21:26:37.765911 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1936/3866] 10.222.0.16 () {58 vars in 1295 bytes} [Sat Jul 16 21:26:37 2022] GET /compute/v2.1/servers/43386970-b64b-4ba2-8a8d-1eda8ff4216a/os-instance-actions => generated 111 bytes in 17 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Jul 16 21:26:37.776382 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-abbc6137-dd9b-4f3e-a736-a50660439805 tempest-InstanceActionsNegativeTestJSON-904555896 tempest-InstanceActionsNegativeTestJSON-904555896-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:26:37.778283 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-abbc6137-dd9b-4f3e-a736-a50660439805 tempest-InstanceActionsNegativeTestJSON-904555896 tempest-InstanceActionsNegativeTestJSON-904555896-project] [instance: ae4b76bc-e7bd-45ff-8716-361dcbbfaa1d] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:26:37.787988 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-abbc6137-dd9b-4f3e-a736-a50660439805 tempest-InstanceActionsNegativeTestJSON-904555896 tempest-InstanceActionsNegativeTestJSON-904555896-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:37.792537 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-abbc6137-dd9b-4f3e-a736-a50660439805 tempest-InstanceActionsNegativeTestJSON-904555896 tempest-InstanceActionsNegativeTestJSON-904555896-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:37.792537 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-abbc6137-dd9b-4f3e-a736-a50660439805 tempest-InstanceActionsNegativeTestJSON-904555896 tempest-InstanceActionsNegativeTestJSON-904555896-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:37.792537 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-abbc6137-dd9b-4f3e-a736-a50660439805 tempest-InstanceActionsNegativeTestJSON-904555896 tempest-InstanceActionsNegativeTestJSON-904555896-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:37.818267 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-abbc6137-dd9b-4f3e-a736-a50660439805 tempest-InstanceActionsNegativeTestJSON-904555896 tempest-InstanceActionsNegativeTestJSON-904555896-project] [instance: ae4b76bc-e7bd-45ff-8716-361dcbbfaa1d] Going to try to terminate instance {{(pid=138681) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 16 21:26:37.852404 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-434a6264-d54f-4529-a0b6-dd5a2afb8aa0 tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb1f6532421745a4b0de70356748a647', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7d5a798ee507429bb70566356229535d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:26:37.860142 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-434a6264-d54f-4529-a0b6-dd5a2afb8aa0 tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] 10.222.0.16 "GET /compute/v2.1/servers/9a184cb7-2548-41d5-8d37-8e72dfb3d75a" status: 200 len: 1314 microversion: 2.1 time: 0.186703 Jul 16 21:26:37.870985 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1931/3867] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:26:37 2022] GET /compute/v2.1/servers/9a184cb7-2548-41d5-8d37-8e72dfb3d75a => generated 1314 bytes in 198 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:26:37.986357 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-08c28bdf-97ce-4724-ae16-8ee5714270d5 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:26:37.986981 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-08c28bdf-97ce-4724-ae16-8ee5714270d5 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 6ba66466-0da7-4e8d-8ddd-b46f582713fc] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:26:37.994182 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-08c28bdf-97ce-4724-ae16-8ee5714270d5 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:37.994645 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-08c28bdf-97ce-4724-ae16-8ee5714270d5 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:37.995456 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-08c28bdf-97ce-4724-ae16-8ee5714270d5 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:37.995904 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-08c28bdf-97ce-4724-ae16-8ee5714270d5 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:38.147647 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-08c28bdf-97ce-4724-ae16-8ee5714270d5 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f2e974ab5aa48dfb4b00f810dcedb63', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd31be8475a6a49d6bfe8132dd050c6c0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:26:38.148586 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-08c28bdf-97ce-4724-ae16-8ee5714270d5 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "GET /compute/v2.1/servers/6ba66466-0da7-4e8d-8ddd-b46f582713fc" status: 200 len: 1492 microversion: 2.1 time: 0.166688 Jul 16 21:26:38.149386 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1932/3868] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:26:37 2022] GET /compute/v2.1/servers/6ba66466-0da7-4e8d-8ddd-b46f582713fc => generated 1492 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:26:38.165827 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-5adaa299-4042-4049-a7ad-a8e9a1bf6da6 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:26:38.171724 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-5adaa299-4042-4049-a7ad-a8e9a1bf6da6 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 6ba66466-0da7-4e8d-8ddd-b46f582713fc] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:26:38.172653 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5adaa299-4042-4049-a7ad-a8e9a1bf6da6 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:38.173111 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5adaa299-4042-4049-a7ad-a8e9a1bf6da6 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:38.174057 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5adaa299-4042-4049-a7ad-a8e9a1bf6da6 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:38.174639 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5adaa299-4042-4049-a7ad-a8e9a1bf6da6 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:38.195728 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-abbc6137-dd9b-4f3e-a736-a50660439805 tempest-InstanceActionsNegativeTestJSON-904555896 tempest-InstanceActionsNegativeTestJSON-904555896-project] 10.222.0.16 "DELETE /compute/v2.1/servers/ae4b76bc-e7bd-45ff-8716-361dcbbfaa1d" status: 204 len: 0 microversion: 2.1 time: 0.424502 Jul 16 21:26:38.200586 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1937/3869] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:26:37 2022] DELETE /compute/v2.1/servers/ae4b76bc-e7bd-45ff-8716-361dcbbfaa1d => generated 0 bytes in 427 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Jul 16 21:26:38.203674 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-5adaa299-4042-4049-a7ad-a8e9a1bf6da6 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 6ba66466-0da7-4e8d-8ddd-b46f582713fc] Going to try to terminate instance {{(pid=138680) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 16 21:26:38.206014 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-e0b38899-b08f-45a8-bd96-0cf958384abb tempest-InstanceActionsNegativeTestJSON-904555896 tempest-InstanceActionsNegativeTestJSON-904555896-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:26:38.206955 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-e0b38899-b08f-45a8-bd96-0cf958384abb tempest-InstanceActionsNegativeTestJSON-904555896 tempest-InstanceActionsNegativeTestJSON-904555896-project] [instance: ae4b76bc-e7bd-45ff-8716-361dcbbfaa1d] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:26:38.213120 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e0b38899-b08f-45a8-bd96-0cf958384abb tempest-InstanceActionsNegativeTestJSON-904555896 tempest-InstanceActionsNegativeTestJSON-904555896-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:38.213798 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e0b38899-b08f-45a8-bd96-0cf958384abb tempest-InstanceActionsNegativeTestJSON-904555896 tempest-InstanceActionsNegativeTestJSON-904555896-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:38.214630 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e0b38899-b08f-45a8-bd96-0cf958384abb tempest-InstanceActionsNegativeTestJSON-904555896 tempest-InstanceActionsNegativeTestJSON-904555896-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:38.215104 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e0b38899-b08f-45a8-bd96-0cf958384abb tempest-InstanceActionsNegativeTestJSON-904555896 tempest-InstanceActionsNegativeTestJSON-904555896-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:38.369177 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-e0b38899-b08f-45a8-bd96-0cf958384abb tempest-InstanceActionsNegativeTestJSON-904555896 tempest-InstanceActionsNegativeTestJSON-904555896-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1963f7b0ee0244b2b4f5fc1d8b34f858', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8f2736abb6094c1f83a7aa3efc772506', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:26:38.377714 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-e0b38899-b08f-45a8-bd96-0cf958384abb tempest-InstanceActionsNegativeTestJSON-904555896 tempest-InstanceActionsNegativeTestJSON-904555896-project] 10.222.0.16 "GET /compute/v2.1/servers/ae4b76bc-e7bd-45ff-8716-361dcbbfaa1d" status: 200 len: 1527 microversion: 2.1 time: 0.175841 Jul 16 21:26:38.378517 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1938/3870] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:26:38 2022] GET /compute/v2.1/servers/ae4b76bc-e7bd-45ff-8716-361dcbbfaa1d => generated 1527 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:26:38.501974 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-94e23478-899c-4e89-af7b-df8cdc92c3a3 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:26:38.502584 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-94e23478-899c-4e89-af7b-df8cdc92c3a3 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: 6dcb4587-0e15-4a2d-b8f0-70b459fb6664] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:26:38.512246 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-94e23478-899c-4e89-af7b-df8cdc92c3a3 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:38.512246 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-94e23478-899c-4e89-af7b-df8cdc92c3a3 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:38.512246 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-94e23478-899c-4e89-af7b-df8cdc92c3a3 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:38.512246 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-94e23478-899c-4e89-af7b-df8cdc92c3a3 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:38.635623 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-5adaa299-4042-4049-a7ad-a8e9a1bf6da6 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "DELETE /compute/v2.1/servers/6ba66466-0da7-4e8d-8ddd-b46f582713fc" status: 204 len: 0 microversion: 2.1 time: 0.473270 Jul 16 21:26:38.636331 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1933/3871] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:26:38 2022] DELETE /compute/v2.1/servers/6ba66466-0da7-4e8d-8ddd-b46f582713fc => generated 0 bytes in 475 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Jul 16 21:26:38.651091 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-dbda270f-ddfa-49fc-8cd1-3b97f942081b tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:26:38.651091 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-dbda270f-ddfa-49fc-8cd1-3b97f942081b tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 6ba66466-0da7-4e8d-8ddd-b46f582713fc] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:26:38.656771 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-dbda270f-ddfa-49fc-8cd1-3b97f942081b tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:38.657232 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-dbda270f-ddfa-49fc-8cd1-3b97f942081b tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:38.658077 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-dbda270f-ddfa-49fc-8cd1-3b97f942081b tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:38.658438 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-dbda270f-ddfa-49fc-8cd1-3b97f942081b tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:38.664085 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-94e23478-899c-4e89-af7b-df8cdc92c3a3 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '027dcc48e4fd4c36a5004096ad5e43f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dad823e240149edb30e592380249a13', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:26:38.674126 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-94e23478-899c-4e89-af7b-df8cdc92c3a3 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "GET /compute/v2.1/servers/6dcb4587-0e15-4a2d-b8f0-70b459fb6664" status: 200 len: 1308 microversion: 2.1 time: 0.176023 Jul 16 21:26:38.675269 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1939/3872] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:26:38 2022] GET /compute/v2.1/servers/6dcb4587-0e15-4a2d-b8f0-70b459fb6664 => generated 1308 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:26:38.816562 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-dbda270f-ddfa-49fc-8cd1-3b97f942081b tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f2e974ab5aa48dfb4b00f810dcedb63', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd31be8475a6a49d6bfe8132dd050c6c0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:26:38.839711 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-dbda270f-ddfa-49fc-8cd1-3b97f942081b tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "GET /compute/v2.1/servers/6ba66466-0da7-4e8d-8ddd-b46f582713fc" status: 200 len: 1498 microversion: 2.1 time: 0.195251 Jul 16 21:26:38.839711 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1934/3873] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:26:38 2022] GET /compute/v2.1/servers/6ba66466-0da7-4e8d-8ddd-b46f582713fc => generated 1498 bytes in 197 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:26:38.877839 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-947aee32-9b55-4206-9a5d-95ad4436e5d7 tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:26:38.878566 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-947aee32-9b55-4206-9a5d-95ad4436e5d7 tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] [instance: 9a184cb7-2548-41d5-8d37-8e72dfb3d75a] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:26:38.885783 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-947aee32-9b55-4206-9a5d-95ad4436e5d7 tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:38.887033 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-947aee32-9b55-4206-9a5d-95ad4436e5d7 tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:38.887033 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-947aee32-9b55-4206-9a5d-95ad4436e5d7 tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:38.887332 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-947aee32-9b55-4206-9a5d-95ad4436e5d7 tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:39.033548 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-947aee32-9b55-4206-9a5d-95ad4436e5d7 tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb1f6532421745a4b0de70356748a647', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7d5a798ee507429bb70566356229535d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:26:39.042046 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-947aee32-9b55-4206-9a5d-95ad4436e5d7 tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] 10.222.0.16 "GET /compute/v2.1/servers/9a184cb7-2548-41d5-8d37-8e72dfb3d75a" status: 200 len: 1314 microversion: 2.1 time: 0.168153 Jul 16 21:26:39.042631 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1940/3874] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:26:38 2022] GET /compute/v2.1/servers/9a184cb7-2548-41d5-8d37-8e72dfb3d75a => generated 1314 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:26:39.397198 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-69839b52-38fe-41df-ba45-64358570926a tempest-InstanceActionsNegativeTestJSON-904555896 tempest-InstanceActionsNegativeTestJSON-904555896-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:26:39.398361 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-69839b52-38fe-41df-ba45-64358570926a tempest-InstanceActionsNegativeTestJSON-904555896 tempest-InstanceActionsNegativeTestJSON-904555896-project] [instance: ae4b76bc-e7bd-45ff-8716-361dcbbfaa1d] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:26:39.404836 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-69839b52-38fe-41df-ba45-64358570926a tempest-InstanceActionsNegativeTestJSON-904555896 tempest-InstanceActionsNegativeTestJSON-904555896-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:39.405303 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-69839b52-38fe-41df-ba45-64358570926a tempest-InstanceActionsNegativeTestJSON-904555896 tempest-InstanceActionsNegativeTestJSON-904555896-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:39.406080 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-69839b52-38fe-41df-ba45-64358570926a tempest-InstanceActionsNegativeTestJSON-904555896 tempest-InstanceActionsNegativeTestJSON-904555896-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:39.406561 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-69839b52-38fe-41df-ba45-64358570926a tempest-InstanceActionsNegativeTestJSON-904555896 tempest-InstanceActionsNegativeTestJSON-904555896-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:39.550586 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-69839b52-38fe-41df-ba45-64358570926a tempest-InstanceActionsNegativeTestJSON-904555896 tempest-InstanceActionsNegativeTestJSON-904555896-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1963f7b0ee0244b2b4f5fc1d8b34f858', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8f2736abb6094c1f83a7aa3efc772506', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:26:39.553766 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:26:39.560173 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-69839b52-38fe-41df-ba45-64358570926a tempest-InstanceActionsNegativeTestJSON-904555896 tempest-InstanceActionsNegativeTestJSON-904555896-project] 10.222.0.16 "GET /compute/v2.1/servers/ae4b76bc-e7bd-45ff-8716-361dcbbfaa1d" status: 200 len: 1527 microversion: 2.1 time: 0.167470 Jul 16 21:26:39.561135 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1935/3875] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:26:39 2022] GET /compute/v2.1/servers/ae4b76bc-e7bd-45ff-8716-361dcbbfaa1d => generated 1527 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:26:39.692339 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-00d5ac55-2dde-45eb-b459-7a6588765a42 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:26:39.693639 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-00d5ac55-2dde-45eb-b459-7a6588765a42 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: 6dcb4587-0e15-4a2d-b8f0-70b459fb6664] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:26:39.700861 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-00d5ac55-2dde-45eb-b459-7a6588765a42 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:39.701314 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-00d5ac55-2dde-45eb-b459-7a6588765a42 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:39.702084 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-00d5ac55-2dde-45eb-b459-7a6588765a42 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:39.702524 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-00d5ac55-2dde-45eb-b459-7a6588765a42 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:39.847828 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-00d5ac55-2dde-45eb-b459-7a6588765a42 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '027dcc48e4fd4c36a5004096ad5e43f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dad823e240149edb30e592380249a13', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:26:39.855869 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-b5b6a5a4-ef5e-43a5-bebc-a98d7096a92a tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:26:39.856634 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-00d5ac55-2dde-45eb-b459-7a6588765a42 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "GET /compute/v2.1/servers/6dcb4587-0e15-4a2d-b8f0-70b459fb6664" status: 200 len: 1489 microversion: 2.1 time: 0.168130 Jul 16 21:26:39.857204 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1941/3876] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:26:39 2022] GET /compute/v2.1/servers/6dcb4587-0e15-4a2d-b8f0-70b459fb6664 => generated 1489 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:26:39.867687 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-b5b6a5a4-ef5e-43a5-bebc-a98d7096a92a tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 6ba66466-0da7-4e8d-8ddd-b46f582713fc] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:26:39.875864 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b5b6a5a4-ef5e-43a5-bebc-a98d7096a92a tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:39.876768 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b5b6a5a4-ef5e-43a5-bebc-a98d7096a92a tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:39.877322 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-2a4a34a1-a8e4-4e38-a7ff-998844197694 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Action: 'update', calling method: >, body: {"server": {"accessIPv4": "1.1.1.1", "accessIPv6": "::babe:202:202"}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:26:39.877706 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b5b6a5a4-ef5e-43a5-bebc-a98d7096a92a tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:39.878386 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b5b6a5a4-ef5e-43a5-bebc-a98d7096a92a tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:39.878837 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-2a4a34a1-a8e4-4e38-a7ff-998844197694 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: 6dcb4587-0e15-4a2d-b8f0-70b459fb6664] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:26:39.885442 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2a4a34a1-a8e4-4e38-a7ff-998844197694 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:39.885896 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2a4a34a1-a8e4-4e38-a7ff-998844197694 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:39.889948 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2a4a34a1-a8e4-4e38-a7ff-998844197694 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:39.889948 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2a4a34a1-a8e4-4e38-a7ff-998844197694 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:40.024035 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-2a4a34a1-a8e4-4e38-a7ff-998844197694 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "PUT /compute/v2.1/servers/6dcb4587-0e15-4a2d-b8f0-70b459fb6664" status: 200 len: 1084 microversion: 2.1 time: 0.150511 Jul 16 21:26:40.024820 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1942/3877] 10.222.0.16 () {60 vars in 1255 bytes} [Sat Jul 16 21:26:39 2022] PUT /compute/v2.1/servers/6dcb4587-0e15-4a2d-b8f0-70b459fb6664 => generated 1084 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:26:40.025509 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-b5b6a5a4-ef5e-43a5-bebc-a98d7096a92a tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f2e974ab5aa48dfb4b00f810dcedb63', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd31be8475a6a49d6bfe8132dd050c6c0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:26:40.037944 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-b5b6a5a4-ef5e-43a5-bebc-a98d7096a92a tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "GET /compute/v2.1/servers/6ba66466-0da7-4e8d-8ddd-b46f582713fc" status: 200 len: 1498 microversion: 2.1 time: 0.182703 Jul 16 21:26:40.039100 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-b5ace721-b9df-48e1-b8eb-b40d0a0a98a9 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:26:40.039902 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-b5ace721-b9df-48e1-b8eb-b40d0a0a98a9 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: 6dcb4587-0e15-4a2d-b8f0-70b459fb6664] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:26:40.046012 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b5ace721-b9df-48e1-b8eb-b40d0a0a98a9 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:40.046495 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b5ace721-b9df-48e1-b8eb-b40d0a0a98a9 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:40.047462 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b5ace721-b9df-48e1-b8eb-b40d0a0a98a9 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:40.047898 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b5ace721-b9df-48e1-b8eb-b40d0a0a98a9 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:40.048635 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1936/3878] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:26:39 2022] GET /compute/v2.1/servers/6ba66466-0da7-4e8d-8ddd-b46f582713fc => generated 1498 bytes in 197 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:26:40.068456 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-f94abcc8-b956-4ebc-9125-317ce157176a tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:26:40.069253 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-f94abcc8-b956-4ebc-9125-317ce157176a tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] [instance: 9a184cb7-2548-41d5-8d37-8e72dfb3d75a] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:26:40.075880 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f94abcc8-b956-4ebc-9125-317ce157176a tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:40.076536 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f94abcc8-b956-4ebc-9125-317ce157176a tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:40.077314 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f94abcc8-b956-4ebc-9125-317ce157176a tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:40.077762 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f94abcc8-b956-4ebc-9125-317ce157176a tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:40.254685 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-b5ace721-b9df-48e1-b8eb-b40d0a0a98a9 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '027dcc48e4fd4c36a5004096ad5e43f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dad823e240149edb30e592380249a13', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:26:40.267560 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-b5ace721-b9df-48e1-b8eb-b40d0a0a98a9 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "GET /compute/v2.1/servers/6dcb4587-0e15-4a2d-b8f0-70b459fb6664" status: 200 len: 1510 microversion: 2.1 time: 0.232983 Jul 16 21:26:40.268784 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1943/3879] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:26:40 2022] GET /compute/v2.1/servers/6dcb4587-0e15-4a2d-b8f0-70b459fb6664 => generated 1510 bytes in 235 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:26:40.289186 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-2ef0d869-d41b-40b4-96e3-5459d92a876e tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:26:40.289186 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-2ef0d869-d41b-40b4-96e3-5459d92a876e tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: 6dcb4587-0e15-4a2d-b8f0-70b459fb6664] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:26:40.297547 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-f94abcc8-b956-4ebc-9125-317ce157176a tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb1f6532421745a4b0de70356748a647', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7d5a798ee507429bb70566356229535d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:26:40.304411 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2ef0d869-d41b-40b4-96e3-5459d92a876e tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:40.304411 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2ef0d869-d41b-40b4-96e3-5459d92a876e tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:40.305102 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2ef0d869-d41b-40b4-96e3-5459d92a876e tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:40.305650 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2ef0d869-d41b-40b4-96e3-5459d92a876e tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:40.311309 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=94,nova_cell1:UPDATE=4,nova_cell1:INSERT=2 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:26:40.318131 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-f94abcc8-b956-4ebc-9125-317ce157176a tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] 10.222.0.16 "GET /compute/v2.1/servers/9a184cb7-2548-41d5-8d37-8e72dfb3d75a" status: 200 len: 1314 microversion: 2.1 time: 0.253765 Jul 16 21:26:40.319536 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1937/3880] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:26:40 2022] GET /compute/v2.1/servers/9a184cb7-2548-41d5-8d37-8e72dfb3d75a => generated 1314 bytes in 256 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:26:40.394896 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-c5000ad7-7b94-4ba9-a158-c118003e6393 req-ef8a8941-35da-4024-b3c5-e3e392334d97 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "9a184cb7-2548-41d5-8d37-8e72dfb3d75a", "name": "network-vif-plugged", "status": "completed", "tag": "80734127-7d61-49c9-8895-1044cdd4e336"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:26:40.405264 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-c5000ad7-7b94-4ba9-a158-c118003e6393 req-ef8a8941-35da-4024-b3c5-e3e392334d97 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:40.406018 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-c5000ad7-7b94-4ba9-a158-c118003e6393 req-ef8a8941-35da-4024-b3c5-e3e392334d97 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:40.421709 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-c5000ad7-7b94-4ba9-a158-c118003e6393 req-ef8a8941-35da-4024-b3c5-e3e392334d97 service nova] Creating event network-vif-plugged:80734127-7d61-49c9-8895-1044cdd4e336 for instance 9a184cb7-2548-41d5-8d37-8e72dfb3d75a on nch1-master-1 Jul 16 21:26:40.427488 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-c5000ad7-7b94-4ba9-a158-c118003e6393 req-ef8a8941-35da-4024-b3c5-e3e392334d97 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.035821 Jul 16 21:26:40.428795 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1938/3881] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:26:40 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 16 21:26:40.457027 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-2ef0d869-d41b-40b4-96e3-5459d92a876e tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '027dcc48e4fd4c36a5004096ad5e43f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dad823e240149edb30e592380249a13', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:26:40.465419 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-2ef0d869-d41b-40b4-96e3-5459d92a876e tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "GET /compute/v2.1/servers/6dcb4587-0e15-4a2d-b8f0-70b459fb6664" status: 200 len: 1510 microversion: 2.1 time: 0.184296 Jul 16 21:26:40.481006 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1944/3882] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:26:40 2022] GET /compute/v2.1/servers/6dcb4587-0e15-4a2d-b8f0-70b459fb6664 => generated 1510 bytes in 201 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:26:40.501139 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-94f42f52-57fd-47f3-8961-0597026521d4 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:26:40.502119 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-94f42f52-57fd-47f3-8961-0597026521d4 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: 6dcb4587-0e15-4a2d-b8f0-70b459fb6664] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:26:40.512281 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-94f42f52-57fd-47f3-8961-0597026521d4 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:40.512911 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-94f42f52-57fd-47f3-8961-0597026521d4 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:40.514115 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-94f42f52-57fd-47f3-8961-0597026521d4 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:40.514885 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-94f42f52-57fd-47f3-8961-0597026521d4 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:40.539425 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-6b3c1cae-b826-4d06-a524-ef81f17d0fad tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "685a896a-62d9-4253-ae50-c46b14a0daf6"}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:26:40.540122 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-6b3c1cae-b826-4d06-a524-ef81f17d0fad tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] [instance: c2e32bd9-9d80-427b-bb6d-6274fd48173a] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:26:40.546361 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6b3c1cae-b826-4d06-a524-ef81f17d0fad tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:40.546920 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6b3c1cae-b826-4d06-a524-ef81f17d0fad tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:40.547861 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6b3c1cae-b826-4d06-a524-ef81f17d0fad tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:40.548377 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6b3c1cae-b826-4d06-a524-ef81f17d0fad tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:40.561037 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-94f42f52-57fd-47f3-8961-0597026521d4 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: 6dcb4587-0e15-4a2d-b8f0-70b459fb6664] Going to try to terminate instance {{(pid=138680) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 16 21:26:40.961475 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-94f42f52-57fd-47f3-8961-0597026521d4 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "DELETE /compute/v2.1/servers/6dcb4587-0e15-4a2d-b8f0-70b459fb6664" status: 204 len: 0 microversion: 2.1 time: 0.478478 Jul 16 21:26:40.962366 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1939/3883] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:26:40 2022] DELETE /compute/v2.1/servers/6dcb4587-0e15-4a2d-b8f0-70b459fb6664 => generated 0 bytes in 482 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 16 21:26:40.967796 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-e1012685-fbdd-491f-8b7c-9d2b594a3ecd tempest-InstanceActionsNegativeTestJSON-904555896 tempest-InstanceActionsNegativeTestJSON-904555896-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:26:40.968711 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-e1012685-fbdd-491f-8b7c-9d2b594a3ecd tempest-InstanceActionsNegativeTestJSON-904555896 tempest-InstanceActionsNegativeTestJSON-904555896-project] [instance: ae4b76bc-e7bd-45ff-8716-361dcbbfaa1d] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:26:40.975921 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e1012685-fbdd-491f-8b7c-9d2b594a3ecd tempest-InstanceActionsNegativeTestJSON-904555896 tempest-InstanceActionsNegativeTestJSON-904555896-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:40.976576 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e1012685-fbdd-491f-8b7c-9d2b594a3ecd tempest-InstanceActionsNegativeTestJSON-904555896 tempest-InstanceActionsNegativeTestJSON-904555896-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:40.977503 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e1012685-fbdd-491f-8b7c-9d2b594a3ecd tempest-InstanceActionsNegativeTestJSON-904555896 tempest-InstanceActionsNegativeTestJSON-904555896-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:40.978092 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e1012685-fbdd-491f-8b7c-9d2b594a3ecd tempest-InstanceActionsNegativeTestJSON-904555896 tempest-InstanceActionsNegativeTestJSON-904555896-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:41.153070 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-e1012685-fbdd-491f-8b7c-9d2b594a3ecd tempest-InstanceActionsNegativeTestJSON-904555896 tempest-InstanceActionsNegativeTestJSON-904555896-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1963f7b0ee0244b2b4f5fc1d8b34f858', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8f2736abb6094c1f83a7aa3efc772506', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:26:41.161837 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-e1012685-fbdd-491f-8b7c-9d2b594a3ecd tempest-InstanceActionsNegativeTestJSON-904555896 tempest-InstanceActionsNegativeTestJSON-904555896-project] 10.222.0.16 "GET /compute/v2.1/servers/ae4b76bc-e7bd-45ff-8716-361dcbbfaa1d" status: 200 len: 1527 microversion: 2.1 time: 0.197898 Jul 16 21:26:41.162444 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1940/3884] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:26:40 2022] GET /compute/v2.1/servers/ae4b76bc-e7bd-45ff-8716-361dcbbfaa1d => generated 1527 bytes in 200 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:26:41.166869 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-6b99d4cd-a02c-4ee8-a299-7e5cffd81079 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:26:41.167500 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-6b99d4cd-a02c-4ee8-a299-7e5cffd81079 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: 6dcb4587-0e15-4a2d-b8f0-70b459fb6664] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:26:41.174470 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6b99d4cd-a02c-4ee8-a299-7e5cffd81079 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:41.174924 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6b99d4cd-a02c-4ee8-a299-7e5cffd81079 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:41.175784 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6b99d4cd-a02c-4ee8-a299-7e5cffd81079 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:41.177013 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6b99d4cd-a02c-4ee8-a299-7e5cffd81079 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:41.312828 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-6b99d4cd-a02c-4ee8-a299-7e5cffd81079 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '027dcc48e4fd4c36a5004096ad5e43f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dad823e240149edb30e592380249a13', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:26:41.320886 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-6b99d4cd-a02c-4ee8-a299-7e5cffd81079 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "GET /compute/v2.1/servers/6dcb4587-0e15-4a2d-b8f0-70b459fb6664" status: 200 len: 1516 microversion: 2.1 time: 0.157410 Jul 16 21:26:41.321409 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1941/3885] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:26:41 2022] GET /compute/v2.1/servers/6dcb4587-0e15-4a2d-b8f0-70b459fb6664 => generated 1516 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:26:41.325904 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-942242aa-f2b0-4b6e-bd4a-7cf533c30516 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:26:41.326515 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-942242aa-f2b0-4b6e-bd4a-7cf533c30516 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 6ba66466-0da7-4e8d-8ddd-b46f582713fc] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:26:41.332997 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-942242aa-f2b0-4b6e-bd4a-7cf533c30516 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:41.334601 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-942242aa-f2b0-4b6e-bd4a-7cf533c30516 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:41.334601 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-942242aa-f2b0-4b6e-bd4a-7cf533c30516 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:41.335072 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-942242aa-f2b0-4b6e-bd4a-7cf533c30516 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:41.481608 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-942242aa-f2b0-4b6e-bd4a-7cf533c30516 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f2e974ab5aa48dfb4b00f810dcedb63', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd31be8475a6a49d6bfe8132dd050c6c0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:26:41.489625 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-942242aa-f2b0-4b6e-bd4a-7cf533c30516 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "GET /compute/v2.1/servers/6ba66466-0da7-4e8d-8ddd-b46f582713fc" status: 200 len: 1498 microversion: 2.1 time: 0.167174 Jul 16 21:26:41.490209 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1942/3886] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:26:41 2022] GET /compute/v2.1/servers/6ba66466-0da7-4e8d-8ddd-b46f582713fc => generated 1498 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:26:41.494754 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-86e56d36-a78e-4f93-90bf-d2a55ecf6754 tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:26:41.495425 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-86e56d36-a78e-4f93-90bf-d2a55ecf6754 tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] [instance: 9a184cb7-2548-41d5-8d37-8e72dfb3d75a] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:26:41.501704 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-86e56d36-a78e-4f93-90bf-d2a55ecf6754 tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:41.502165 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-86e56d36-a78e-4f93-90bf-d2a55ecf6754 tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:41.502934 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-86e56d36-a78e-4f93-90bf-d2a55ecf6754 tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:41.503415 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-86e56d36-a78e-4f93-90bf-d2a55ecf6754 tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:41.650143 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-86e56d36-a78e-4f93-90bf-d2a55ecf6754 tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb1f6532421745a4b0de70356748a647', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7d5a798ee507429bb70566356229535d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:26:41.657950 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-86e56d36-a78e-4f93-90bf-d2a55ecf6754 tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] 10.222.0.16 "GET /compute/v2.1/servers/9a184cb7-2548-41d5-8d37-8e72dfb3d75a" status: 200 len: 1314 microversion: 2.1 time: 0.166728 Jul 16 21:26:41.658533 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1943/3887] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:26:41 2022] GET /compute/v2.1/servers/9a184cb7-2548-41d5-8d37-8e72dfb3d75a => generated 1314 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:26:41.793673 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-6b3c1cae-b826-4d06-a524-ef81f17d0fad tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] 10.222.0.16 "POST /compute/v2.1/servers/c2e32bd9-9d80-427b-bb6d-6274fd48173a/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 1.259365 Jul 16 21:26:41.794505 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1945/3888] 10.222.0.16 () {60 vars in 1322 bytes} [Sat Jul 16 21:26:40 2022] POST /compute/v2.1/servers/c2e32bd9-9d80-427b-bb6d-6274fd48173a/os-volume_attachments => generated 194 bytes in 1261 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:26:42.180336 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-9adf763d-7f59-4f26-b7c6-223ec1304da3 tempest-InstanceActionsNegativeTestJSON-904555896 tempest-InstanceActionsNegativeTestJSON-904555896-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:26:42.181893 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-9adf763d-7f59-4f26-b7c6-223ec1304da3 tempest-InstanceActionsNegativeTestJSON-904555896 tempest-InstanceActionsNegativeTestJSON-904555896-project] [instance: ae4b76bc-e7bd-45ff-8716-361dcbbfaa1d] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:26:42.189577 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9adf763d-7f59-4f26-b7c6-223ec1304da3 tempest-InstanceActionsNegativeTestJSON-904555896 tempest-InstanceActionsNegativeTestJSON-904555896-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:42.190171 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9adf763d-7f59-4f26-b7c6-223ec1304da3 tempest-InstanceActionsNegativeTestJSON-904555896 tempest-InstanceActionsNegativeTestJSON-904555896-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:42.191253 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9adf763d-7f59-4f26-b7c6-223ec1304da3 tempest-InstanceActionsNegativeTestJSON-904555896 tempest-InstanceActionsNegativeTestJSON-904555896-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:42.192487 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9adf763d-7f59-4f26-b7c6-223ec1304da3 tempest-InstanceActionsNegativeTestJSON-904555896 tempest-InstanceActionsNegativeTestJSON-904555896-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:42.339758 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-61af55a4-feea-48eb-b5a7-697436b57134 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:26:42.340470 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-61af55a4-feea-48eb-b5a7-697436b57134 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: 6dcb4587-0e15-4a2d-b8f0-70b459fb6664] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:26:42.349301 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-61af55a4-feea-48eb-b5a7-697436b57134 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:42.349975 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-61af55a4-feea-48eb-b5a7-697436b57134 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:42.350930 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-61af55a4-feea-48eb-b5a7-697436b57134 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:42.351556 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-61af55a4-feea-48eb-b5a7-697436b57134 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:42.493254 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-9adf763d-7f59-4f26-b7c6-223ec1304da3 tempest-InstanceActionsNegativeTestJSON-904555896 tempest-InstanceActionsNegativeTestJSON-904555896-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1963f7b0ee0244b2b4f5fc1d8b34f858', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8f2736abb6094c1f83a7aa3efc772506', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:26:42.509399 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-9adf763d-7f59-4f26-b7c6-223ec1304da3 tempest-InstanceActionsNegativeTestJSON-904555896 tempest-InstanceActionsNegativeTestJSON-904555896-project] 10.222.0.16 "GET /compute/v2.1/servers/ae4b76bc-e7bd-45ff-8716-361dcbbfaa1d" status: 200 len: 1527 microversion: 2.1 time: 0.334892 Jul 16 21:26:42.510363 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1944/3889] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:26:42 2022] GET /compute/v2.1/servers/ae4b76bc-e7bd-45ff-8716-361dcbbfaa1d => generated 1527 bytes in 337 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:26:42.515453 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-beee49cc-8024-4618-b5a5-57b475f67783 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:26:42.516402 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-beee49cc-8024-4618-b5a5-57b475f67783 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 6ba66466-0da7-4e8d-8ddd-b46f582713fc] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:26:42.549959 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-beee49cc-8024-4618-b5a5-57b475f67783 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:42.551239 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-beee49cc-8024-4618-b5a5-57b475f67783 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:42.553765 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-beee49cc-8024-4618-b5a5-57b475f67783 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:42.554573 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-beee49cc-8024-4618-b5a5-57b475f67783 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:42.651874 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-61af55a4-feea-48eb-b5a7-697436b57134 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '027dcc48e4fd4c36a5004096ad5e43f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dad823e240149edb30e592380249a13', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:26:42.668971 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-61af55a4-feea-48eb-b5a7-697436b57134 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "GET /compute/v2.1/servers/6dcb4587-0e15-4a2d-b8f0-70b459fb6664" status: 200 len: 1516 microversion: 2.1 time: 0.332299 Jul 16 21:26:42.671249 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1946/3890] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:26:42 2022] GET /compute/v2.1/servers/6dcb4587-0e15-4a2d-b8f0-70b459fb6664 => generated 1516 bytes in 337 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:26:42.690136 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-1a29c4cf-a97f-4cdb-81a0-3d70d61b2b34 tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:26:42.692682 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-1a29c4cf-a97f-4cdb-81a0-3d70d61b2b34 tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] [instance: 9a184cb7-2548-41d5-8d37-8e72dfb3d75a] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:26:42.724427 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1a29c4cf-a97f-4cdb-81a0-3d70d61b2b34 tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:42.725570 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1a29c4cf-a97f-4cdb-81a0-3d70d61b2b34 tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:42.727996 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1a29c4cf-a97f-4cdb-81a0-3d70d61b2b34 tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:42.729924 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1a29c4cf-a97f-4cdb-81a0-3d70d61b2b34 tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:43.262904 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-beee49cc-8024-4618-b5a5-57b475f67783 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f2e974ab5aa48dfb4b00f810dcedb63', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd31be8475a6a49d6bfe8132dd050c6c0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:26:43.273214 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-beee49cc-8024-4618-b5a5-57b475f67783 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "GET /compute/v2.1/servers/6ba66466-0da7-4e8d-8ddd-b46f582713fc" status: 200 len: 1498 microversion: 2.1 time: 0.761523 Jul 16 21:26:43.273987 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1945/3891] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:26:42 2022] GET /compute/v2.1/servers/6ba66466-0da7-4e8d-8ddd-b46f582713fc => generated 1498 bytes in 763 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:26:43.284361 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-eb54f8e9-8461-441e-8996-8877ef26aa6d req-c458aa71-6d2e-40dd-85ea-56e009383744 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "ae4b76bc-e7bd-45ff-8716-361dcbbfaa1d", "name": "network-vif-unplugged", "status": "completed", "tag": "95067406-61b5-4fed-bcd2-609bccc3cb11"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:26:43.293810 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-1a29c4cf-a97f-4cdb-81a0-3d70d61b2b34 tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb1f6532421745a4b0de70356748a647', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7d5a798ee507429bb70566356229535d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:26:43.303528 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-eb54f8e9-8461-441e-8996-8877ef26aa6d req-c458aa71-6d2e-40dd-85ea-56e009383744 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:43.303992 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-eb54f8e9-8461-441e-8996-8877ef26aa6d req-c458aa71-6d2e-40dd-85ea-56e009383744 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:43.306745 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-1a29c4cf-a97f-4cdb-81a0-3d70d61b2b34 tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] 10.222.0.16 "GET /compute/v2.1/servers/9a184cb7-2548-41d5-8d37-8e72dfb3d75a" status: 200 len: 1463 microversion: 2.1 time: 0.633202 Jul 16 21:26:43.323099 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1947/3892] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:26:42 2022] GET /compute/v2.1/servers/9a184cb7-2548-41d5-8d37-8e72dfb3d75a => generated 1463 bytes in 650 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:26:43.333869 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-eb54f8e9-8461-441e-8996-8877ef26aa6d req-c458aa71-6d2e-40dd-85ea-56e009383744 service nova] Creating event network-vif-unplugged:95067406-61b5-4fed-bcd2-609bccc3cb11 for instance ae4b76bc-e7bd-45ff-8716-361dcbbfaa1d on nch2-master-1 Jul 16 21:26:43.339647 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-053bb311-8161-4825-8ca4-a947337e5cfc tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "be0c4ae4-0277-4cb8-92f2-b87b2a58c6b6", "device": "/dev/sdb"}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:26:43.341235 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-053bb311-8161-4825-8ca4-a947337e5cfc tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] [instance: 9a184cb7-2548-41d5-8d37-8e72dfb3d75a] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:26:43.350214 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-eb54f8e9-8461-441e-8996-8877ef26aa6d req-c458aa71-6d2e-40dd-85ea-56e009383744 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.071155 Jul 16 21:26:43.350214 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1946/3893] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:26:43 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 73 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:26:43.360766 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-053bb311-8161-4825-8ca4-a947337e5cfc tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:43.361220 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-053bb311-8161-4825-8ca4-a947337e5cfc tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:43.369124 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-053bb311-8161-4825-8ca4-a947337e5cfc tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:43.369585 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-053bb311-8161-4825-8ca4-a947337e5cfc tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:43.440042 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-54b4b186-856b-46e1-adfe-30475eaef73e req-f5afa1b3-b485-405b-8860-7722d6fcedb3 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "6dcb4587-0e15-4a2d-b8f0-70b459fb6664", "name": "network-vif-deleted", "tag": "eda9d853-12d2-4057-a4a9-4683ca97fd9d"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:26:43.452923 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_api:INSERT=1,nova_api:SELECT=97,nova_api:UPDATE=2 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:26:43.478561 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-54b4b186-856b-46e1-adfe-30475eaef73e req-f5afa1b3-b485-405b-8860-7722d6fcedb3 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:43.480556 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-54b4b186-856b-46e1-adfe-30475eaef73e req-f5afa1b3-b485-405b-8860-7722d6fcedb3 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:43.497442 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-54b4b186-856b-46e1-adfe-30475eaef73e req-f5afa1b3-b485-405b-8860-7722d6fcedb3 service nova] Creating event network-vif-deleted:eda9d853-12d2-4057-a4a9-4683ca97fd9d for instance 6dcb4587-0e15-4a2d-b8f0-70b459fb6664 on nch2-master-1 Jul 16 21:26:43.505344 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-54b4b186-856b-46e1-adfe-30475eaef73e req-f5afa1b3-b485-405b-8860-7722d6fcedb3 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.071861 Jul 16 21:26:43.506730 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1947/3894] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:26:43 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 75 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:26:43.516003 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-58306239-f45e-44c2-9232-a4a93b652032 req-168e9822-f00d-4767-b89e-b7bd4842d5e1 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "6ba66466-0da7-4e8d-8ddd-b46f582713fc", "name": "network-vif-deleted", "tag": "20de36ec-25f9-4437-b0d2-08b6125425c3"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:26:43.530860 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-58306239-f45e-44c2-9232-a4a93b652032 req-168e9822-f00d-4767-b89e-b7bd4842d5e1 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:43.532955 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-58306239-f45e-44c2-9232-a4a93b652032 req-168e9822-f00d-4767-b89e-b7bd4842d5e1 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:43.556681 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-58306239-f45e-44c2-9232-a4a93b652032 req-168e9822-f00d-4767-b89e-b7bd4842d5e1 service nova] Creating event network-vif-deleted:20de36ec-25f9-4437-b0d2-08b6125425c3 for instance 6ba66466-0da7-4e8d-8ddd-b46f582713fc on nch2-master-1 Jul 16 21:26:43.570762 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-58306239-f45e-44c2-9232-a4a93b652032 req-168e9822-f00d-4767-b89e-b7bd4842d5e1 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.062148 Jul 16 21:26:43.572360 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1948/3895] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:26:43 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 65 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:26:43.577860 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-bb5e2b1b-6653-489e-acbe-93f99e0c1266 tempest-InstanceActionsNegativeTestJSON-904555896 tempest-InstanceActionsNegativeTestJSON-904555896-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:26:43.579418 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-bb5e2b1b-6653-489e-acbe-93f99e0c1266 tempest-InstanceActionsNegativeTestJSON-904555896 tempest-InstanceActionsNegativeTestJSON-904555896-project] [instance: ae4b76bc-e7bd-45ff-8716-361dcbbfaa1d] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:26:43.594492 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-bb5e2b1b-6653-489e-acbe-93f99e0c1266 tempest-InstanceActionsNegativeTestJSON-904555896 tempest-InstanceActionsNegativeTestJSON-904555896-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:43.605123 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-bb5e2b1b-6653-489e-acbe-93f99e0c1266 tempest-InstanceActionsNegativeTestJSON-904555896 tempest-InstanceActionsNegativeTestJSON-904555896-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.016s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:43.606624 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-bb5e2b1b-6653-489e-acbe-93f99e0c1266 tempest-InstanceActionsNegativeTestJSON-904555896 tempest-InstanceActionsNegativeTestJSON-904555896-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:43.608355 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-bb5e2b1b-6653-489e-acbe-93f99e0c1266 tempest-InstanceActionsNegativeTestJSON-904555896 tempest-InstanceActionsNegativeTestJSON-904555896-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:43.822363 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-bb5e2b1b-6653-489e-acbe-93f99e0c1266 tempest-InstanceActionsNegativeTestJSON-904555896 tempest-InstanceActionsNegativeTestJSON-904555896-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1963f7b0ee0244b2b4f5fc1d8b34f858', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8f2736abb6094c1f83a7aa3efc772506', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:26:43.830619 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-bb5e2b1b-6653-489e-acbe-93f99e0c1266 tempest-InstanceActionsNegativeTestJSON-904555896 tempest-InstanceActionsNegativeTestJSON-904555896-project] 10.222.0.16 "GET /compute/v2.1/servers/ae4b76bc-e7bd-45ff-8716-361dcbbfaa1d" status: 200 len: 1527 microversion: 2.1 time: 0.256792 Jul 16 21:26:43.831290 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1949/3896] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:26:43 2022] GET /compute/v2.1/servers/ae4b76bc-e7bd-45ff-8716-361dcbbfaa1d => generated 1527 bytes in 258 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:26:43.836926 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-24ff0425-1178-49cb-a693-c4c322d43a71 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:26:43.837532 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-24ff0425-1178-49cb-a693-c4c322d43a71 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: 6dcb4587-0e15-4a2d-b8f0-70b459fb6664] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:26:43.843815 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-24ff0425-1178-49cb-a693-c4c322d43a71 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:43.847888 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-24ff0425-1178-49cb-a693-c4c322d43a71 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:43.847888 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-24ff0425-1178-49cb-a693-c4c322d43a71 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:43.847888 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-24ff0425-1178-49cb-a693-c4c322d43a71 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:43.885467 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-24ff0425-1178-49cb-a693-c4c322d43a71 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:43.885926 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-24ff0425-1178-49cb-a693-c4c322d43a71 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:43.886386 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.objects.instance [None req-24ff0425-1178-49cb-a693-c4c322d43a71 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lazy-loading 'fault' on Instance uuid 6dcb4587-0e15-4a2d-b8f0-70b459fb6664 {{(pid=138680) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:26:43.998302 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-24ff0425-1178-49cb-a693-c4c322d43a71 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '027dcc48e4fd4c36a5004096ad5e43f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dad823e240149edb30e592380249a13', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:26:44.009079 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-24ff0425-1178-49cb-a693-c4c322d43a71 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "GET /compute/v2.1/servers/6dcb4587-0e15-4a2d-b8f0-70b459fb6664" status: 200 len: 1319 microversion: 2.1 time: 0.176628 Jul 16 21:26:44.010387 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1950/3897] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:26:43 2022] GET /compute/v2.1/servers/6dcb4587-0e15-4a2d-b8f0-70b459fb6664 => generated 1319 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:26:44.291202 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-ae79c2cc-a26b-4104-a3bc-87495d2856ad tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:26:44.291856 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-ae79c2cc-a26b-4104-a3bc-87495d2856ad tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 6ba66466-0da7-4e8d-8ddd-b46f582713fc] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:26:44.300105 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ae79c2cc-a26b-4104-a3bc-87495d2856ad tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:44.300105 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ae79c2cc-a26b-4104-a3bc-87495d2856ad tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:44.300105 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ae79c2cc-a26b-4104-a3bc-87495d2856ad tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:44.300570 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ae79c2cc-a26b-4104-a3bc-87495d2856ad tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:44.336363 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ae79c2cc-a26b-4104-a3bc-87495d2856ad tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:44.336829 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ae79c2cc-a26b-4104-a3bc-87495d2856ad tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:44.337288 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.objects.instance [None req-ae79c2cc-a26b-4104-a3bc-87495d2856ad tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lazy-loading 'fault' on Instance uuid 6ba66466-0da7-4e8d-8ddd-b46f582713fc {{(pid=138680) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:26:44.378919 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-ae79c2cc-a26b-4104-a3bc-87495d2856ad tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f2e974ab5aa48dfb4b00f810dcedb63', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd31be8475a6a49d6bfe8132dd050c6c0', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:26:44.386564 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-ae79c2cc-a26b-4104-a3bc-87495d2856ad tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "GET /compute/v2.1/servers/6ba66466-0da7-4e8d-8ddd-b46f582713fc" status: 200 len: 1304 microversion: 2.1 time: 0.099475 Jul 16 21:26:44.387180 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1951/3898] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:26:44 2022] GET /compute/v2.1/servers/6ba66466-0da7-4e8d-8ddd-b46f582713fc => generated 1304 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:26:44.561883 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-053bb311-8161-4825-8ca4-a947337e5cfc tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] 10.222.0.16 "POST /compute/v2.1/servers/9a184cb7-2548-41d5-8d37-8e72dfb3d75a/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 1.235115 Jul 16 21:26:44.562859 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1948/3899] 10.222.0.16 () {60 vars in 1322 bytes} [Sat Jul 16 21:26:43 2022] POST /compute/v2.1/servers/9a184cb7-2548-41d5-8d37-8e72dfb3d75a/os-volume_attachments => generated 194 bytes in 1237 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:26:44.849475 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-a59fa6a6-5a80-4462-9bb5-2a43c21f96d8 tempest-InstanceActionsNegativeTestJSON-904555896 tempest-InstanceActionsNegativeTestJSON-904555896-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:26:44.850045 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-a59fa6a6-5a80-4462-9bb5-2a43c21f96d8 tempest-InstanceActionsNegativeTestJSON-904555896 tempest-InstanceActionsNegativeTestJSON-904555896-project] [instance: ae4b76bc-e7bd-45ff-8716-361dcbbfaa1d] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:26:44.859058 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a59fa6a6-5a80-4462-9bb5-2a43c21f96d8 tempest-InstanceActionsNegativeTestJSON-904555896 tempest-InstanceActionsNegativeTestJSON-904555896-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:44.859526 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a59fa6a6-5a80-4462-9bb5-2a43c21f96d8 tempest-InstanceActionsNegativeTestJSON-904555896 tempest-InstanceActionsNegativeTestJSON-904555896-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:44.860306 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a59fa6a6-5a80-4462-9bb5-2a43c21f96d8 tempest-InstanceActionsNegativeTestJSON-904555896 tempest-InstanceActionsNegativeTestJSON-904555896-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:44.860959 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a59fa6a6-5a80-4462-9bb5-2a43c21f96d8 tempest-InstanceActionsNegativeTestJSON-904555896 tempest-InstanceActionsNegativeTestJSON-904555896-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:44.897421 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a59fa6a6-5a80-4462-9bb5-2a43c21f96d8 tempest-InstanceActionsNegativeTestJSON-904555896 tempest-InstanceActionsNegativeTestJSON-904555896-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:44.897884 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a59fa6a6-5a80-4462-9bb5-2a43c21f96d8 tempest-InstanceActionsNegativeTestJSON-904555896 tempest-InstanceActionsNegativeTestJSON-904555896-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:44.898348 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.objects.instance [None req-a59fa6a6-5a80-4462-9bb5-2a43c21f96d8 tempest-InstanceActionsNegativeTestJSON-904555896 tempest-InstanceActionsNegativeTestJSON-904555896-project] Lazy-loading 'fault' on Instance uuid ae4b76bc-e7bd-45ff-8716-361dcbbfaa1d {{(pid=138680) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:26:44.901380 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:26:44.944476 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-a59fa6a6-5a80-4462-9bb5-2a43c21f96d8 tempest-InstanceActionsNegativeTestJSON-904555896 tempest-InstanceActionsNegativeTestJSON-904555896-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1963f7b0ee0244b2b4f5fc1d8b34f858', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8f2736abb6094c1f83a7aa3efc772506', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:26:44.952361 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-a59fa6a6-5a80-4462-9bb5-2a43c21f96d8 tempest-InstanceActionsNegativeTestJSON-904555896 tempest-InstanceActionsNegativeTestJSON-904555896-project] 10.222.0.16 "GET /compute/v2.1/servers/ae4b76bc-e7bd-45ff-8716-361dcbbfaa1d" status: 200 len: 1314 microversion: 2.1 time: 0.107509 Jul 16 21:26:44.954116 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1952/3900] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:26:44 2022] GET /compute/v2.1/servers/ae4b76bc-e7bd-45ff-8716-361dcbbfaa1d => generated 1314 bytes in 110 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:26:45.030555 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-fee5114a-d1ff-46e1-96ff-ce47356b3c40 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:26:45.031330 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-fee5114a-d1ff-46e1-96ff-ce47356b3c40 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: 6dcb4587-0e15-4a2d-b8f0-70b459fb6664] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:26:45.038497 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fee5114a-d1ff-46e1-96ff-ce47356b3c40 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:45.038955 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fee5114a-d1ff-46e1-96ff-ce47356b3c40 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:45.039760 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fee5114a-d1ff-46e1-96ff-ce47356b3c40 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:45.040200 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fee5114a-d1ff-46e1-96ff-ce47356b3c40 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:45.054381 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-fee5114a-d1ff-46e1-96ff-ce47356b3c40 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] HTTP exception thrown: Instance 6dcb4587-0e15-4a2d-b8f0-70b459fb6664 could not be found. Jul 16 21:26:45.056114 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-fee5114a-d1ff-46e1-96ff-ce47356b3c40 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Request method failure captured: Jul 16 21:26:45.056114 ncdv-master-1 devstack@n-api.service[138681]: request: GET /compute/v2.1/servers/6dcb4587-0e15-4a2d-b8f0-70b459fb6664 HTTP/1.1 Jul 16 21:26:45.056114 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:26:45.056114 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:26:45.056114 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:26:45.056114 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 21:26:45.056114 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:26:45.056114 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:26:45.056114 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:26:45.056114 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:26:45.056114 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0yxS9NCWgI1-O55Z_AF16JzmczB_p-s4P30aJMz-9zu9VxcHuIV00tsfB9hOBtHTZ_sI8GPeOOvUrqYZLgOBzCbOMr4ZO506_XIRjMCtHQ0umzvcB9V9iFSNFoiIe_b2jKm-OyHj1IB8YVBCAcwDKANlsCw5UaAoIABK4dLLFjaVpEo Jul 16 21:26:45.056114 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:26:45.056114 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:26:45.056114 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:26:45.056114 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:26:45.056114 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:26:45.056114 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:26:45.056114 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: 1dad823e240149edb30e592380249a13 Jul 16 21:26:45.056114 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-ServersTestJSON-1529284206 Jul 16 21:26:45.056114 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:26:45.056114 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:26:45.056114 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/1dad823e240149edb30e592380249a13"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/1dad823e240149edb30e592380249a13"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_1dad823e240149edb30e592380249a13"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/1dad823e240149edb30e592380249a13"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:26:45.056114 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-ServersTestJSON-1529284206 Jul 16 21:26:45.056114 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: 1dad823e240149edb30e592380249a13 Jul 16 21:26:45.056114 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-ServersTestJSON-1529284206 Jul 16 21:26:45.056114 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-ServersTestJSON-1529284206-project Jul 16 21:26:45.056114 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:26:45.056114 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:26:45.056114 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: 027dcc48e4fd4c36a5004096ad5e43f8 Jul 16 21:26:45.056114 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-ServersTestJSON-1529284206-project Jul 16 21:26:45.056114 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:26:45.056114 ncdv-master-1 devstack@n-api.service[138681]: exception: Instance 6dcb4587-0e15-4a2d-b8f0-70b459fb6664 could not be found. Jul 16 21:26:45.056114 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': '6dcb4587-0e15-4a2d-b8f0-70b459fb6664'} Jul 16 21:26:45.056114 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:26:45.056114 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:26:45.056114 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:26:45.056114 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:26:45.056114 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:26:45.056114 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:26:45.056114 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:26:45.056114 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:26:45.056114 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:26:45.056114 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:26:45.056114 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 6dcb4587-0e15-4a2d-b8f0-70b459fb6664 could not be found. Jul 16 21:26:45.056114 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:26:45.056114 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:26:45.056114 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:26:45.056114 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:26:45.056114 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:26:45.056114 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:26:45.056114 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:26:45.056114 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:26:45.056114 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:26:45.056114 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:26:45.056114 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:26:45.056114 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:26:45.056114 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:26:45.056114 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:26:45.056114 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:26:45.056114 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:26:45.056114 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 6dcb4587-0e15-4a2d-b8f0-70b459fb6664 could not be found. Jul 16 21:26:45.056114 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:26:45.056114 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:26:45.056114 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:26:45.056114 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:26:45.059898 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:26:45.059898 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:26:45.059898 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:26:45.059898 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:26:45.059898 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 6dcb4587-0e15-4a2d-b8f0-70b459fb6664 could not be found. Jul 16 21:26:45.059898 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:26:45.059898 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-fee5114a-d1ff-46e1-96ff-ce47356b3c40 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Returning 404 to user: Instance 6dcb4587-0e15-4a2d-b8f0-70b459fb6664 could not be found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:26:45.059898 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-fee5114a-d1ff-46e1-96ff-ce47356b3c40 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "GET /compute/v2.1/servers/6dcb4587-0e15-4a2d-b8f0-70b459fb6664" status: 404 len: 111 microversion: 2.1 time: 0.032150 Jul 16 21:26:45.068710 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1949/3901] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:26:45 2022] GET /compute/v2.1/servers/6dcb4587-0e15-4a2d-b8f0-70b459fb6664 => generated 111 bytes in 37 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:26:45.192923 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-4f64e527-9405-4de3-bc3d-e8f3d72b566f tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServersTestJSON-server-866371299", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452", "networks": [{"uuid": "2202d4ad-313c-4607-b9fc-eb98bac9afd0"}]}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:26:45.248501 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.network.neutron [None req-4f64e527-9405-4de3-bc3d-e8f3d72b566f tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] validate_networks() for [('2202d4ad-313c-4607-b9fc-eb98bac9afd0', None, None, None, None, None)] {{(pid=138680) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 16 21:26:45.407890 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-cb01f340-0c3d-477a-aa44-7f6824c55c64 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:26:45.411179 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-cb01f340-0c3d-477a-aa44-7f6824c55c64 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 6ba66466-0da7-4e8d-8ddd-b46f582713fc] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:26:45.423768 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cb01f340-0c3d-477a-aa44-7f6824c55c64 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:45.424225 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cb01f340-0c3d-477a-aa44-7f6824c55c64 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:45.425008 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cb01f340-0c3d-477a-aa44-7f6824c55c64 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:45.429580 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cb01f340-0c3d-477a-aa44-7f6824c55c64 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:45.466874 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-cb01f340-0c3d-477a-aa44-7f6824c55c64 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] HTTP exception thrown: Instance 6ba66466-0da7-4e8d-8ddd-b46f582713fc could not be found. Jul 16 21:26:45.468871 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-cb01f340-0c3d-477a-aa44-7f6824c55c64 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Request method failure captured: Jul 16 21:26:45.468871 ncdv-master-1 devstack@n-api.service[138681]: request: GET /compute/v2.1/servers/6ba66466-0da7-4e8d-8ddd-b46f582713fc HTTP/1.1 Jul 16 21:26:45.468871 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:26:45.468871 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:26:45.468871 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:26:45.468871 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 21:26:45.468871 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:26:45.468871 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:26:45.468871 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:26:45.468871 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:26:45.468871 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0yvs4CI48W3ZCHoQ8huRZwHvqHxAdwrwrvk1v_ETBPx6FU4DdiPUjkjq4hN_4g_fHLAB_XZ9-AAN8Pyvy0KON2X25WlmyilO65MDHbIW9Xy9v9USVPAu_Ce-ze5X9UIHyned0AUneE4UPGiLLIkoXeoS5ekiq1pi209pud2lyUAuHgw Jul 16 21:26:45.468871 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:26:45.468871 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:26:45.468871 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:26:45.468871 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:26:45.468871 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:26:45.468871 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:26:45.468871 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: d31be8475a6a49d6bfe8132dd050c6c0 Jul 16 21:26:45.468871 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-DeleteServersTestJSON-1541892181 Jul 16 21:26:45.468871 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:26:45.468871 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:26:45.468871 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/d31be8475a6a49d6bfe8132dd050c6c0"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/d31be8475a6a49d6bfe8132dd050c6c0"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_d31be8475a6a49d6bfe8132dd050c6c0"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/d31be8475a6a49d6bfe8132dd050c6c0"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:26:45.468871 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-DeleteServersTestJSON-1541892181 Jul 16 21:26:45.468871 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: d31be8475a6a49d6bfe8132dd050c6c0 Jul 16 21:26:45.468871 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-DeleteServersTestJSON-1541892181 Jul 16 21:26:45.468871 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-DeleteServersTestJSON-1541892181-project Jul 16 21:26:45.468871 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:26:45.468871 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:26:45.468871 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: 6f2e974ab5aa48dfb4b00f810dcedb63 Jul 16 21:26:45.468871 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-DeleteServersTestJSON-1541892181-project Jul 16 21:26:45.468871 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:26:45.468871 ncdv-master-1 devstack@n-api.service[138681]: exception: Instance 6ba66466-0da7-4e8d-8ddd-b46f582713fc could not be found. Jul 16 21:26:45.468871 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': '6ba66466-0da7-4e8d-8ddd-b46f582713fc'} Jul 16 21:26:45.468871 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:26:45.468871 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:26:45.468871 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:26:45.468871 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:26:45.468871 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:26:45.468871 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:26:45.468871 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:26:45.468871 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:26:45.468871 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:26:45.468871 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:26:45.468871 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 6ba66466-0da7-4e8d-8ddd-b46f582713fc could not be found. Jul 16 21:26:45.468871 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:26:45.468871 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:26:45.468871 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:26:45.468871 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:26:45.468871 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:26:45.468871 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:26:45.468871 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:26:45.468871 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:26:45.468871 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:26:45.468871 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:26:45.468871 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:26:45.468871 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:26:45.468871 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:26:45.468871 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:26:45.468871 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:26:45.468871 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:26:45.468871 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 6ba66466-0da7-4e8d-8ddd-b46f582713fc could not be found. Jul 16 21:26:45.468871 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:26:45.468871 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:26:45.468871 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:26:45.473985 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:26:45.473985 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:26:45.473985 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:26:45.473985 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:26:45.473985 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:26:45.473985 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 6ba66466-0da7-4e8d-8ddd-b46f582713fc could not be found. Jul 16 21:26:45.473985 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:26:45.475196 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-cb01f340-0c3d-477a-aa44-7f6824c55c64 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Returning 404 to user: Instance 6ba66466-0da7-4e8d-8ddd-b46f582713fc could not be found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:26:45.476075 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-cb01f340-0c3d-477a-aa44-7f6824c55c64 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "GET /compute/v2.1/servers/6ba66466-0da7-4e8d-8ddd-b46f582713fc" status: 404 len: 111 microversion: 2.1 time: 0.075492 Jul 16 21:26:45.480796 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1950/3902] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:26:45 2022] GET /compute/v2.1/servers/6ba66466-0da7-4e8d-8ddd-b46f582713fc => generated 111 bytes in 81 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Jul 16 21:26:45.496824 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-62d44105-53a8-4058-acb4-8e2542f74c0e tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:26:45.498442 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-62d44105-53a8-4058-acb4-8e2542f74c0e tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 6ba66466-0da7-4e8d-8ddd-b46f582713fc] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:26:45.506210 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-62d44105-53a8-4058-acb4-8e2542f74c0e tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:45.506924 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-62d44105-53a8-4058-acb4-8e2542f74c0e tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:45.507887 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-62d44105-53a8-4058-acb4-8e2542f74c0e tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:45.508331 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-62d44105-53a8-4058-acb4-8e2542f74c0e tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:45.532961 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-62d44105-53a8-4058-acb4-8e2542f74c0e tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] HTTP exception thrown: Instance 6ba66466-0da7-4e8d-8ddd-b46f582713fc could not be found. Jul 16 21:26:45.534753 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-62d44105-53a8-4058-acb4-8e2542f74c0e tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Request method failure captured: Jul 16 21:26:45.534753 ncdv-master-1 devstack@n-api.service[138681]: request: DELETE /compute/v2.1/servers/6ba66466-0da7-4e8d-8ddd-b46f582713fc HTTP/1.1 Jul 16 21:26:45.534753 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:26:45.534753 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:26:45.534753 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:26:45.534753 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 21:26:45.534753 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:26:45.534753 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:26:45.534753 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:26:45.534753 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:26:45.534753 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0yvs4CI48W3ZCHoQ8huRZwHvqHxAdwrwrvk1v_ETBPx6FU4DdiPUjkjq4hN_4g_fHLAB_XZ9-AAN8Pyvy0KON2X25WlmyilO65MDHbIW9Xy9v9USVPAu_Ce-ze5X9UIHyned0AUneE4UPGiLLIkoXeoS5ekiq1pi209pud2lyUAuHgw Jul 16 21:26:45.534753 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:26:45.534753 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:26:45.534753 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:26:45.534753 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:26:45.534753 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:26:45.534753 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:26:45.534753 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: d31be8475a6a49d6bfe8132dd050c6c0 Jul 16 21:26:45.534753 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-DeleteServersTestJSON-1541892181 Jul 16 21:26:45.534753 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:26:45.534753 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:26:45.534753 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/d31be8475a6a49d6bfe8132dd050c6c0"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/d31be8475a6a49d6bfe8132dd050c6c0"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_d31be8475a6a49d6bfe8132dd050c6c0"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/d31be8475a6a49d6bfe8132dd050c6c0"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:26:45.534753 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-DeleteServersTestJSON-1541892181 Jul 16 21:26:45.534753 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: d31be8475a6a49d6bfe8132dd050c6c0 Jul 16 21:26:45.534753 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-DeleteServersTestJSON-1541892181 Jul 16 21:26:45.534753 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-DeleteServersTestJSON-1541892181-project Jul 16 21:26:45.534753 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:26:45.534753 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:26:45.534753 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: 6f2e974ab5aa48dfb4b00f810dcedb63 Jul 16 21:26:45.534753 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-DeleteServersTestJSON-1541892181-project Jul 16 21:26:45.534753 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:26:45.534753 ncdv-master-1 devstack@n-api.service[138681]: exception: Instance 6ba66466-0da7-4e8d-8ddd-b46f582713fc could not be found. Jul 16 21:26:45.534753 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': '6ba66466-0da7-4e8d-8ddd-b46f582713fc'} Jul 16 21:26:45.534753 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:26:45.534753 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:26:45.534753 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:26:45.534753 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:26:45.534753 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:26:45.534753 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:26:45.534753 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:26:45.534753 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:26:45.534753 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:26:45.534753 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:26:45.534753 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 6ba66466-0da7-4e8d-8ddd-b46f582713fc could not be found. Jul 16 21:26:45.534753 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:26:45.534753 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:26:45.534753 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:26:45.534753 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:26:45.534753 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:26:45.534753 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:26:45.534753 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:26:45.534753 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:26:45.534753 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:26:45.534753 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:26:45.534753 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Jul 16 21:26:45.534753 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Jul 16 21:26:45.534753 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Jul 16 21:26:45.534753 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Jul 16 21:26:45.534753 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:26:45.534753 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:26:45.534753 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:26:45.534753 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:26:45.545164 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 6ba66466-0da7-4e8d-8ddd-b46f582713fc could not be found. Jul 16 21:26:45.545164 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:26:45.545164 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:26:45.545164 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:26:45.545164 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:26:45.545164 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:26:45.545164 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:26:45.545164 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:26:45.545164 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:26:45.545164 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 6ba66466-0da7-4e8d-8ddd-b46f582713fc could not be found. Jul 16 21:26:45.545164 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:26:45.545164 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-62d44105-53a8-4058-acb4-8e2542f74c0e tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Returning 404 to user: Instance 6ba66466-0da7-4e8d-8ddd-b46f582713fc could not be found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:26:45.545164 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-62d44105-53a8-4058-acb4-8e2542f74c0e tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "DELETE /compute/v2.1/servers/6ba66466-0da7-4e8d-8ddd-b46f582713fc" status: 404 len: 111 microversion: 2.1 time: 0.047554 Jul 16 21:26:45.545164 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1951/3903] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:26:45 2022] DELETE /compute/v2.1/servers/6ba66466-0da7-4e8d-8ddd-b46f582713fc => generated 111 bytes in 50 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:26:45.552868 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-de5b838e-f1db-4c14-859c-23568014d017 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:26:45.554056 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-de5b838e-f1db-4c14-859c-23568014d017 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 6ba66466-0da7-4e8d-8ddd-b46f582713fc] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:26:45.561057 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-de5b838e-f1db-4c14-859c-23568014d017 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:45.561724 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-de5b838e-f1db-4c14-859c-23568014d017 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:45.562540 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-de5b838e-f1db-4c14-859c-23568014d017 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:45.563112 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-de5b838e-f1db-4c14-859c-23568014d017 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:45.577943 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-de5b838e-f1db-4c14-859c-23568014d017 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] HTTP exception thrown: Instance 6ba66466-0da7-4e8d-8ddd-b46f582713fc could not be found. Jul 16 21:26:45.580278 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-de5b838e-f1db-4c14-859c-23568014d017 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Request method failure captured: Jul 16 21:26:45.580278 ncdv-master-1 devstack@n-api.service[138681]: request: GET /compute/v2.1/servers/6ba66466-0da7-4e8d-8ddd-b46f582713fc HTTP/1.1 Jul 16 21:26:45.580278 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:26:45.580278 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:26:45.580278 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:26:45.580278 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 21:26:45.580278 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:26:45.580278 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:26:45.580278 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:26:45.580278 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:26:45.580278 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0yvs4CI48W3ZCHoQ8huRZwHvqHxAdwrwrvk1v_ETBPx6FU4DdiPUjkjq4hN_4g_fHLAB_XZ9-AAN8Pyvy0KON2X25WlmyilO65MDHbIW9Xy9v9USVPAu_Ce-ze5X9UIHyned0AUneE4UPGiLLIkoXeoS5ekiq1pi209pud2lyUAuHgw Jul 16 21:26:45.580278 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:26:45.580278 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:26:45.580278 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:26:45.580278 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:26:45.580278 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:26:45.580278 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:26:45.580278 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: d31be8475a6a49d6bfe8132dd050c6c0 Jul 16 21:26:45.580278 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-DeleteServersTestJSON-1541892181 Jul 16 21:26:45.580278 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:26:45.580278 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:26:45.580278 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/d31be8475a6a49d6bfe8132dd050c6c0"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/d31be8475a6a49d6bfe8132dd050c6c0"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_d31be8475a6a49d6bfe8132dd050c6c0"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/d31be8475a6a49d6bfe8132dd050c6c0"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:26:45.580278 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-DeleteServersTestJSON-1541892181 Jul 16 21:26:45.580278 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: d31be8475a6a49d6bfe8132dd050c6c0 Jul 16 21:26:45.580278 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-DeleteServersTestJSON-1541892181 Jul 16 21:26:45.580278 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-DeleteServersTestJSON-1541892181-project Jul 16 21:26:45.580278 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:26:45.580278 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:26:45.580278 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: 6f2e974ab5aa48dfb4b00f810dcedb63 Jul 16 21:26:45.580278 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-DeleteServersTestJSON-1541892181-project Jul 16 21:26:45.580278 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:26:45.580278 ncdv-master-1 devstack@n-api.service[138681]: exception: Instance 6ba66466-0da7-4e8d-8ddd-b46f582713fc could not be found. Jul 16 21:26:45.580278 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': '6ba66466-0da7-4e8d-8ddd-b46f582713fc'} Jul 16 21:26:45.580278 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:26:45.580278 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:26:45.580278 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:26:45.580278 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:26:45.580278 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:26:45.580278 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:26:45.580278 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:26:45.580278 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:26:45.580278 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:26:45.580278 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:26:45.580278 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 6ba66466-0da7-4e8d-8ddd-b46f582713fc could not be found. Jul 16 21:26:45.580278 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:26:45.580278 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:26:45.580278 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:26:45.580278 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:26:45.580278 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:26:45.580278 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:26:45.580278 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:26:45.580278 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:26:45.580278 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:26:45.580278 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:26:45.580278 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:26:45.580278 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:26:45.580278 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:26:45.580278 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:26:45.580278 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:26:45.580278 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:26:45.580278 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 6ba66466-0da7-4e8d-8ddd-b46f582713fc could not be found. Jul 16 21:26:45.580278 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:26:45.580278 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:26:45.580278 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:26:45.586763 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:26:45.586763 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:26:45.586763 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:26:45.586763 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:26:45.586763 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:26:45.586763 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 6ba66466-0da7-4e8d-8ddd-b46f582713fc could not be found. Jul 16 21:26:45.586763 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:26:45.586763 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-de5b838e-f1db-4c14-859c-23568014d017 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Returning 404 to user: Instance 6ba66466-0da7-4e8d-8ddd-b46f582713fc could not be found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:26:45.586763 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-de5b838e-f1db-4c14-859c-23568014d017 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "GET /compute/v2.1/servers/6ba66466-0da7-4e8d-8ddd-b46f582713fc" status: 404 len: 111 microversion: 2.1 time: 0.036355 Jul 16 21:26:45.586763 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1952/3904] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:26:45 2022] GET /compute/v2.1/servers/6ba66466-0da7-4e8d-8ddd-b46f582713fc => generated 111 bytes in 37 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:26:45.592491 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-8644810f-daa1-418b-8d2f-5b3d609f0e6e tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:26:45.593058 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-8644810f-daa1-418b-8d2f-5b3d609f0e6e tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 00c8a753-5bf0-448c-8107-3946aa9bf7ae] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:26:45.601793 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8644810f-daa1-418b-8d2f-5b3d609f0e6e tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:45.602175 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8644810f-daa1-418b-8d2f-5b3d609f0e6e tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:45.602965 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8644810f-daa1-418b-8d2f-5b3d609f0e6e tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:45.603457 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8644810f-daa1-418b-8d2f-5b3d609f0e6e tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:45.620541 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-8644810f-daa1-418b-8d2f-5b3d609f0e6e tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] HTTP exception thrown: Instance 00c8a753-5bf0-448c-8107-3946aa9bf7ae could not be found. Jul 16 21:26:45.622206 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-8644810f-daa1-418b-8d2f-5b3d609f0e6e tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Request method failure captured: Jul 16 21:26:45.622206 ncdv-master-1 devstack@n-api.service[138681]: request: DELETE /compute/v2.1/servers/00c8a753-5bf0-448c-8107-3946aa9bf7ae HTTP/1.1 Jul 16 21:26:45.622206 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:26:45.622206 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:26:45.622206 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:26:45.622206 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 21:26:45.622206 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:26:45.622206 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:26:45.622206 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:26:45.622206 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:26:45.622206 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0yvs4CI48W3ZCHoQ8huRZwHvqHxAdwrwrvk1v_ETBPx6FU4DdiPUjkjq4hN_4g_fHLAB_XZ9-AAN8Pyvy0KON2X25WlmyilO65MDHbIW9Xy9v9USVPAu_Ce-ze5X9UIHyned0AUneE4UPGiLLIkoXeoS5ekiq1pi209pud2lyUAuHgw Jul 16 21:26:45.622206 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:26:45.622206 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:26:45.622206 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:26:45.622206 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:26:45.622206 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:26:45.622206 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:26:45.622206 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: d31be8475a6a49d6bfe8132dd050c6c0 Jul 16 21:26:45.622206 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-DeleteServersTestJSON-1541892181 Jul 16 21:26:45.622206 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:26:45.622206 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:26:45.622206 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/d31be8475a6a49d6bfe8132dd050c6c0"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/d31be8475a6a49d6bfe8132dd050c6c0"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_d31be8475a6a49d6bfe8132dd050c6c0"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/d31be8475a6a49d6bfe8132dd050c6c0"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:26:45.622206 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-DeleteServersTestJSON-1541892181 Jul 16 21:26:45.622206 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: d31be8475a6a49d6bfe8132dd050c6c0 Jul 16 21:26:45.622206 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-DeleteServersTestJSON-1541892181 Jul 16 21:26:45.622206 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-DeleteServersTestJSON-1541892181-project Jul 16 21:26:45.622206 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:26:45.622206 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:26:45.622206 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: 6f2e974ab5aa48dfb4b00f810dcedb63 Jul 16 21:26:45.622206 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-DeleteServersTestJSON-1541892181-project Jul 16 21:26:45.622206 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:26:45.622206 ncdv-master-1 devstack@n-api.service[138681]: exception: Instance 00c8a753-5bf0-448c-8107-3946aa9bf7ae could not be found. Jul 16 21:26:45.622206 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': '00c8a753-5bf0-448c-8107-3946aa9bf7ae'} Jul 16 21:26:45.622206 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:26:45.622206 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:26:45.622206 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:26:45.622206 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:26:45.622206 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:26:45.622206 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:26:45.622206 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:26:45.622206 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:26:45.622206 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:26:45.622206 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:26:45.622206 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 00c8a753-5bf0-448c-8107-3946aa9bf7ae could not be found. Jul 16 21:26:45.622206 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:26:45.622206 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:26:45.622206 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:26:45.622206 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:26:45.622206 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:26:45.622206 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:26:45.622206 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:26:45.622206 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:26:45.622206 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:26:45.622206 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:26:45.622206 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Jul 16 21:26:45.622206 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Jul 16 21:26:45.622206 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Jul 16 21:26:45.622206 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Jul 16 21:26:45.622206 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:26:45.622206 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:26:45.622206 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:26:45.622206 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:26:45.627288 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 00c8a753-5bf0-448c-8107-3946aa9bf7ae could not be found. Jul 16 21:26:45.627288 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:26:45.627288 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:26:45.627288 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:26:45.627288 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:26:45.627288 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:26:45.627288 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:26:45.627288 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:26:45.627288 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:26:45.627288 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 00c8a753-5bf0-448c-8107-3946aa9bf7ae could not be found. Jul 16 21:26:45.627288 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:26:45.627288 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-8644810f-daa1-418b-8d2f-5b3d609f0e6e tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Returning 404 to user: Instance 00c8a753-5bf0-448c-8107-3946aa9bf7ae could not be found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:26:45.627288 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-8644810f-daa1-418b-8d2f-5b3d609f0e6e tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "DELETE /compute/v2.1/servers/00c8a753-5bf0-448c-8107-3946aa9bf7ae" status: 404 len: 111 microversion: 2.1 time: 0.034815 Jul 16 21:26:45.634345 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1953/3905] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:26:45 2022] DELETE /compute/v2.1/servers/00c8a753-5bf0-448c-8107-3946aa9bf7ae => generated 111 bytes in 39 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Jul 16 21:26:45.634345 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-98cd2da5-d1a3-4b74-8f18-7f5e1d71cf49 req-f386ddba-3db7-41a6-ab22-1d788f346211 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "ae4b76bc-e7bd-45ff-8716-361dcbbfaa1d", "name": "network-vif-deleted", "tag": "95067406-61b5-4fed-bcd2-609bccc3cb11"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:26:45.643904 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-98cd2da5-d1a3-4b74-8f18-7f5e1d71cf49 req-f386ddba-3db7-41a6-ab22-1d788f346211 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:45.644358 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-98cd2da5-d1a3-4b74-8f18-7f5e1d71cf49 req-f386ddba-3db7-41a6-ab22-1d788f346211 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:45.668401 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-98cd2da5-d1a3-4b74-8f18-7f5e1d71cf49 req-f386ddba-3db7-41a6-ab22-1d788f346211 service nova] Creating event network-vif-deleted:95067406-61b5-4fed-bcd2-609bccc3cb11 for instance ae4b76bc-e7bd-45ff-8716-361dcbbfaa1d on nch2-master-1 Jul 16 21:26:45.673954 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-98cd2da5-d1a3-4b74-8f18-7f5e1d71cf49 req-f386ddba-3db7-41a6-ab22-1d788f346211 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.045151 Jul 16 21:26:45.674892 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1954/3906] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:26:45 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:26:45.681981 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-336aea2a-3cc1-413c-ad99-d62bdc4b74ea tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:26:45.689621 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-336aea2a-3cc1-413c-ad99-d62bdc4b74ea tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 00c8a753-5bf0-448c-8107-3946aa9bf7ae] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:26:45.690975 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-336aea2a-3cc1-413c-ad99-d62bdc4b74ea tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:45.691466 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-336aea2a-3cc1-413c-ad99-d62bdc4b74ea tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:45.692806 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-336aea2a-3cc1-413c-ad99-d62bdc4b74ea tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:45.693248 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-336aea2a-3cc1-413c-ad99-d62bdc4b74ea tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:45.708751 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-336aea2a-3cc1-413c-ad99-d62bdc4b74ea tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] HTTP exception thrown: Instance 00c8a753-5bf0-448c-8107-3946aa9bf7ae could not be found. Jul 16 21:26:45.710795 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-336aea2a-3cc1-413c-ad99-d62bdc4b74ea tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Request method failure captured: Jul 16 21:26:45.710795 ncdv-master-1 devstack@n-api.service[138681]: request: GET /compute/v2.1/servers/00c8a753-5bf0-448c-8107-3946aa9bf7ae HTTP/1.1 Jul 16 21:26:45.710795 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:26:45.710795 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:26:45.710795 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:26:45.710795 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 21:26:45.710795 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:26:45.710795 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:26:45.710795 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:26:45.710795 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:26:45.710795 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0yvs4CI48W3ZCHoQ8huRZwHvqHxAdwrwrvk1v_ETBPx6FU4DdiPUjkjq4hN_4g_fHLAB_XZ9-AAN8Pyvy0KON2X25WlmyilO65MDHbIW9Xy9v9USVPAu_Ce-ze5X9UIHyned0AUneE4UPGiLLIkoXeoS5ekiq1pi209pud2lyUAuHgw Jul 16 21:26:45.710795 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:26:45.710795 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:26:45.710795 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:26:45.710795 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:26:45.710795 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:26:45.710795 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:26:45.710795 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: d31be8475a6a49d6bfe8132dd050c6c0 Jul 16 21:26:45.710795 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-DeleteServersTestJSON-1541892181 Jul 16 21:26:45.710795 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:26:45.710795 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:26:45.710795 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/d31be8475a6a49d6bfe8132dd050c6c0"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/d31be8475a6a49d6bfe8132dd050c6c0"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_d31be8475a6a49d6bfe8132dd050c6c0"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/d31be8475a6a49d6bfe8132dd050c6c0"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:26:45.710795 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-DeleteServersTestJSON-1541892181 Jul 16 21:26:45.710795 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: d31be8475a6a49d6bfe8132dd050c6c0 Jul 16 21:26:45.710795 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-DeleteServersTestJSON-1541892181 Jul 16 21:26:45.710795 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-DeleteServersTestJSON-1541892181-project Jul 16 21:26:45.710795 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:26:45.710795 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:26:45.710795 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: 6f2e974ab5aa48dfb4b00f810dcedb63 Jul 16 21:26:45.710795 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-DeleteServersTestJSON-1541892181-project Jul 16 21:26:45.710795 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:26:45.710795 ncdv-master-1 devstack@n-api.service[138681]: exception: Instance 00c8a753-5bf0-448c-8107-3946aa9bf7ae could not be found. Jul 16 21:26:45.710795 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': '00c8a753-5bf0-448c-8107-3946aa9bf7ae'} Jul 16 21:26:45.710795 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:26:45.710795 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:26:45.710795 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:26:45.710795 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:26:45.710795 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:26:45.710795 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:26:45.710795 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:26:45.710795 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:26:45.710795 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:26:45.710795 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:26:45.710795 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 00c8a753-5bf0-448c-8107-3946aa9bf7ae could not be found. Jul 16 21:26:45.710795 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:26:45.710795 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:26:45.710795 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:26:45.710795 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:26:45.710795 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:26:45.710795 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:26:45.710795 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:26:45.710795 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:26:45.710795 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:26:45.710795 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:26:45.710795 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:26:45.710795 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:26:45.710795 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:26:45.710795 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:26:45.710795 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:26:45.710795 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:26:45.710795 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 00c8a753-5bf0-448c-8107-3946aa9bf7ae could not be found. Jul 16 21:26:45.710795 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:26:45.710795 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:26:45.710795 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:26:45.719919 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:26:45.719919 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:26:45.719919 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:26:45.719919 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:26:45.719919 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:26:45.719919 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 00c8a753-5bf0-448c-8107-3946aa9bf7ae could not be found. Jul 16 21:26:45.719919 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:26:45.719919 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-336aea2a-3cc1-413c-ad99-d62bdc4b74ea tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Returning 404 to user: Instance 00c8a753-5bf0-448c-8107-3946aa9bf7ae could not be found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:26:45.719919 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-336aea2a-3cc1-413c-ad99-d62bdc4b74ea tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "GET /compute/v2.1/servers/00c8a753-5bf0-448c-8107-3946aa9bf7ae" status: 404 len: 111 microversion: 2.1 time: 0.036361 Jul 16 21:26:45.721029 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1955/3907] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:26:45 2022] GET /compute/v2.1/servers/00c8a753-5bf0-448c-8107-3946aa9bf7ae => generated 111 bytes in 45 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:26:45.726935 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-659d17e6-2b22-46a4-b91f-9ee1c6c67f4f tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:26:45.727468 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-659d17e6-2b22-46a4-b91f-9ee1c6c67f4f tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 8e6eab25-f25e-46a2-9c72-9424c872fb84] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:26:45.733933 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-659d17e6-2b22-46a4-b91f-9ee1c6c67f4f tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:45.734386 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-659d17e6-2b22-46a4-b91f-9ee1c6c67f4f tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:45.735206 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-659d17e6-2b22-46a4-b91f-9ee1c6c67f4f tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:45.735647 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-659d17e6-2b22-46a4-b91f-9ee1c6c67f4f tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:45.752648 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-659d17e6-2b22-46a4-b91f-9ee1c6c67f4f tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] HTTP exception thrown: Instance 8e6eab25-f25e-46a2-9c72-9424c872fb84 could not be found. Jul 16 21:26:45.757355 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-659d17e6-2b22-46a4-b91f-9ee1c6c67f4f tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Request method failure captured: Jul 16 21:26:45.757355 ncdv-master-1 devstack@n-api.service[138681]: request: DELETE /compute/v2.1/servers/8e6eab25-f25e-46a2-9c72-9424c872fb84 HTTP/1.1 Jul 16 21:26:45.757355 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:26:45.757355 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:26:45.757355 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:26:45.757355 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 21:26:45.757355 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:26:45.757355 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:26:45.757355 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:26:45.757355 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:26:45.757355 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0yvs4CI48W3ZCHoQ8huRZwHvqHxAdwrwrvk1v_ETBPx6FU4DdiPUjkjq4hN_4g_fHLAB_XZ9-AAN8Pyvy0KON2X25WlmyilO65MDHbIW9Xy9v9USVPAu_Ce-ze5X9UIHyned0AUneE4UPGiLLIkoXeoS5ekiq1pi209pud2lyUAuHgw Jul 16 21:26:45.757355 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:26:45.757355 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:26:45.757355 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:26:45.757355 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:26:45.757355 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:26:45.757355 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:26:45.757355 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: d31be8475a6a49d6bfe8132dd050c6c0 Jul 16 21:26:45.757355 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-DeleteServersTestJSON-1541892181 Jul 16 21:26:45.757355 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:26:45.757355 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:26:45.757355 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/d31be8475a6a49d6bfe8132dd050c6c0"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/d31be8475a6a49d6bfe8132dd050c6c0"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_d31be8475a6a49d6bfe8132dd050c6c0"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/d31be8475a6a49d6bfe8132dd050c6c0"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:26:45.757355 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-DeleteServersTestJSON-1541892181 Jul 16 21:26:45.757355 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: d31be8475a6a49d6bfe8132dd050c6c0 Jul 16 21:26:45.757355 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-DeleteServersTestJSON-1541892181 Jul 16 21:26:45.757355 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-DeleteServersTestJSON-1541892181-project Jul 16 21:26:45.757355 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:26:45.757355 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:26:45.757355 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: 6f2e974ab5aa48dfb4b00f810dcedb63 Jul 16 21:26:45.757355 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-DeleteServersTestJSON-1541892181-project Jul 16 21:26:45.757355 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:26:45.757355 ncdv-master-1 devstack@n-api.service[138681]: exception: Instance 8e6eab25-f25e-46a2-9c72-9424c872fb84 could not be found. Jul 16 21:26:45.757355 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': '8e6eab25-f25e-46a2-9c72-9424c872fb84'} Jul 16 21:26:45.757355 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:26:45.757355 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:26:45.757355 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:26:45.757355 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:26:45.757355 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:26:45.757355 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:26:45.757355 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:26:45.757355 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:26:45.757355 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:26:45.757355 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:26:45.757355 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 8e6eab25-f25e-46a2-9c72-9424c872fb84 could not be found. Jul 16 21:26:45.757355 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:26:45.757355 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:26:45.757355 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:26:45.757355 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:26:45.757355 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:26:45.757355 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:26:45.757355 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:26:45.757355 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:26:45.757355 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:26:45.757355 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:26:45.757355 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Jul 16 21:26:45.757355 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Jul 16 21:26:45.757355 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Jul 16 21:26:45.757355 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Jul 16 21:26:45.757355 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:26:45.757355 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:26:45.757355 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:26:45.757355 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:26:45.766413 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 8e6eab25-f25e-46a2-9c72-9424c872fb84 could not be found. Jul 16 21:26:45.766413 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:26:45.766413 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:26:45.766413 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:26:45.766413 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:26:45.766413 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:26:45.766413 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:26:45.766413 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:26:45.766413 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:26:45.766413 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 8e6eab25-f25e-46a2-9c72-9424c872fb84 could not be found. Jul 16 21:26:45.766413 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:26:45.766413 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-659d17e6-2b22-46a4-b91f-9ee1c6c67f4f tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Returning 404 to user: Instance 8e6eab25-f25e-46a2-9c72-9424c872fb84 could not be found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:26:45.766413 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-659d17e6-2b22-46a4-b91f-9ee1c6c67f4f tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "DELETE /compute/v2.1/servers/8e6eab25-f25e-46a2-9c72-9424c872fb84" status: 404 len: 111 microversion: 2.1 time: 0.037517 Jul 16 21:26:45.767513 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1956/3908] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:26:45 2022] DELETE /compute/v2.1/servers/8e6eab25-f25e-46a2-9c72-9424c872fb84 => generated 111 bytes in 46 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:26:45.772692 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-30f5b597-c413-49bc-aa8c-bb0af20c40a7 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:26:45.773489 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-30f5b597-c413-49bc-aa8c-bb0af20c40a7 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 8e6eab25-f25e-46a2-9c72-9424c872fb84] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:26:45.780954 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-30f5b597-c413-49bc-aa8c-bb0af20c40a7 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:45.781623 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-30f5b597-c413-49bc-aa8c-bb0af20c40a7 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:45.782063 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-4f64e527-9405-4de3-bc3d-e8f3d72b566f tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Flavor limits 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 16 21:26:45.782600 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-4f64e527-9405-4de3-bc3d-e8f3d72b566f tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Image limits 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 16 21:26:45.783401 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-4f64e527-9405-4de3-bc3d-e8f3d72b566f tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Flavor pref 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 16 21:26:45.783767 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-30f5b597-c413-49bc-aa8c-bb0af20c40a7 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:45.784222 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-30f5b597-c413-49bc-aa8c-bb0af20c40a7 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:45.784605 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-4f64e527-9405-4de3-bc3d-e8f3d72b566f tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Image pref 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 16 21:26:45.785211 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-4f64e527-9405-4de3-bc3d-e8f3d72b566f tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 16 21:26:45.798895 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4f64e527-9405-4de3-bc3d-e8f3d72b566f tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:45.799784 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4f64e527-9405-4de3-bc3d-e8f3d72b566f tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:45.800853 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4f64e527-9405-4de3-bc3d-e8f3d72b566f tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:45.801674 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-30f5b597-c413-49bc-aa8c-bb0af20c40a7 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] HTTP exception thrown: Instance 8e6eab25-f25e-46a2-9c72-9424c872fb84 could not be found. Jul 16 21:26:45.802246 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4f64e527-9405-4de3-bc3d-e8f3d72b566f tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:45.803382 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-30f5b597-c413-49bc-aa8c-bb0af20c40a7 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Request method failure captured: Jul 16 21:26:45.803382 ncdv-master-1 devstack@n-api.service[138681]: request: GET /compute/v2.1/servers/8e6eab25-f25e-46a2-9c72-9424c872fb84 HTTP/1.1 Jul 16 21:26:45.803382 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:26:45.803382 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:26:45.803382 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:26:45.803382 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 21:26:45.803382 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:26:45.803382 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:26:45.803382 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:26:45.803382 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:26:45.803382 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0yvs4CI48W3ZCHoQ8huRZwHvqHxAdwrwrvk1v_ETBPx6FU4DdiPUjkjq4hN_4g_fHLAB_XZ9-AAN8Pyvy0KON2X25WlmyilO65MDHbIW9Xy9v9USVPAu_Ce-ze5X9UIHyned0AUneE4UPGiLLIkoXeoS5ekiq1pi209pud2lyUAuHgw Jul 16 21:26:45.803382 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:26:45.803382 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:26:45.803382 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:26:45.803382 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:26:45.803382 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:26:45.803382 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:26:45.803382 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: d31be8475a6a49d6bfe8132dd050c6c0 Jul 16 21:26:45.803382 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-DeleteServersTestJSON-1541892181 Jul 16 21:26:45.803382 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:26:45.803382 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:26:45.803382 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/d31be8475a6a49d6bfe8132dd050c6c0"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/d31be8475a6a49d6bfe8132dd050c6c0"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_d31be8475a6a49d6bfe8132dd050c6c0"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/d31be8475a6a49d6bfe8132dd050c6c0"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:26:45.803382 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-DeleteServersTestJSON-1541892181 Jul 16 21:26:45.803382 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: d31be8475a6a49d6bfe8132dd050c6c0 Jul 16 21:26:45.803382 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-DeleteServersTestJSON-1541892181 Jul 16 21:26:45.803382 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-DeleteServersTestJSON-1541892181-project Jul 16 21:26:45.803382 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:26:45.803382 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:26:45.803382 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: 6f2e974ab5aa48dfb4b00f810dcedb63 Jul 16 21:26:45.803382 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-DeleteServersTestJSON-1541892181-project Jul 16 21:26:45.803382 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:26:45.803382 ncdv-master-1 devstack@n-api.service[138681]: exception: Instance 8e6eab25-f25e-46a2-9c72-9424c872fb84 could not be found. Jul 16 21:26:45.803382 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': '8e6eab25-f25e-46a2-9c72-9424c872fb84'} Jul 16 21:26:45.803382 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:26:45.803382 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:26:45.803382 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:26:45.803382 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:26:45.803382 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:26:45.803382 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:26:45.803382 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:26:45.803382 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:26:45.803382 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:26:45.803382 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:26:45.803382 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 8e6eab25-f25e-46a2-9c72-9424c872fb84 could not be found. Jul 16 21:26:45.803382 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:26:45.803382 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:26:45.803382 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:26:45.803382 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:26:45.803382 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:26:45.803382 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:26:45.803382 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:26:45.803382 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:26:45.803382 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:26:45.803382 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:26:45.803382 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:26:45.803382 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:26:45.803382 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:26:45.803382 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:26:45.803382 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:26:45.803382 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:26:45.803382 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 8e6eab25-f25e-46a2-9c72-9424c872fb84 could not be found. Jul 16 21:26:45.803382 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:26:45.803382 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:26:45.803382 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:26:45.806835 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:26:45.806835 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:26:45.806835 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:26:45.806835 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:26:45.806835 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:26:45.806835 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 8e6eab25-f25e-46a2-9c72-9424c872fb84 could not be found. Jul 16 21:26:45.806835 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:26:45.806835 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-30f5b597-c413-49bc-aa8c-bb0af20c40a7 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Returning 404 to user: Instance 8e6eab25-f25e-46a2-9c72-9424c872fb84 could not be found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:26:45.806835 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-30f5b597-c413-49bc-aa8c-bb0af20c40a7 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "GET /compute/v2.1/servers/8e6eab25-f25e-46a2-9c72-9424c872fb84" status: 404 len: 111 microversion: 2.1 time: 0.036273 Jul 16 21:26:45.806835 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1957/3909] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:26:45 2022] GET /compute/v2.1/servers/8e6eab25-f25e-46a2-9c72-9424c872fb84 => generated 111 bytes in 37 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:26:45.817847 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-39839a3f-0218-404a-8f1e-6d2ecb909e0a tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:26:45.818451 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-39839a3f-0218-404a-8f1e-6d2ecb909e0a tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 1cb418f7-6aea-42e6-88a2-007564d925c4] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:26:45.824966 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-39839a3f-0218-404a-8f1e-6d2ecb909e0a tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:45.825669 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-39839a3f-0218-404a-8f1e-6d2ecb909e0a tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:45.829199 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-4f64e527-9405-4de3-bc3d-e8f3d72b566f tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Getting quotas for project 1dad823e240149edb30e592380249a13. Resources: {'ram', 'instances', 'cores'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:26:45.829326 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-39839a3f-0218-404a-8f1e-6d2ecb909e0a tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:45.829326 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-39839a3f-0218-404a-8f1e-6d2ecb909e0a tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:45.832408 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-4f64e527-9405-4de3-bc3d-e8f3d72b566f tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Getting quotas for user 027dcc48e4fd4c36a5004096ad5e43f8 and project 1dad823e240149edb30e592380249a13. Resources: {'ram', 'instances', 'cores'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:26:45.841695 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-39839a3f-0218-404a-8f1e-6d2ecb909e0a tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] HTTP exception thrown: Instance 1cb418f7-6aea-42e6-88a2-007564d925c4 could not be found. Jul 16 21:26:45.843279 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-39839a3f-0218-404a-8f1e-6d2ecb909e0a tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Request method failure captured: Jul 16 21:26:45.843279 ncdv-master-1 devstack@n-api.service[138681]: request: DELETE /compute/v2.1/servers/1cb418f7-6aea-42e6-88a2-007564d925c4 HTTP/1.1 Jul 16 21:26:45.843279 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:26:45.843279 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:26:45.843279 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:26:45.843279 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 21:26:45.843279 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:26:45.843279 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:26:45.843279 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:26:45.843279 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:26:45.843279 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0yvs4CI48W3ZCHoQ8huRZwHvqHxAdwrwrvk1v_ETBPx6FU4DdiPUjkjq4hN_4g_fHLAB_XZ9-AAN8Pyvy0KON2X25WlmyilO65MDHbIW9Xy9v9USVPAu_Ce-ze5X9UIHyned0AUneE4UPGiLLIkoXeoS5ekiq1pi209pud2lyUAuHgw Jul 16 21:26:45.843279 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:26:45.843279 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:26:45.843279 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:26:45.843279 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:26:45.843279 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:26:45.843279 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:26:45.843279 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: d31be8475a6a49d6bfe8132dd050c6c0 Jul 16 21:26:45.843279 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-DeleteServersTestJSON-1541892181 Jul 16 21:26:45.843279 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:26:45.843279 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:26:45.843279 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/d31be8475a6a49d6bfe8132dd050c6c0"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/d31be8475a6a49d6bfe8132dd050c6c0"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_d31be8475a6a49d6bfe8132dd050c6c0"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/d31be8475a6a49d6bfe8132dd050c6c0"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:26:45.843279 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-DeleteServersTestJSON-1541892181 Jul 16 21:26:45.843279 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: d31be8475a6a49d6bfe8132dd050c6c0 Jul 16 21:26:45.843279 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-DeleteServersTestJSON-1541892181 Jul 16 21:26:45.843279 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-DeleteServersTestJSON-1541892181-project Jul 16 21:26:45.843279 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:26:45.843279 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:26:45.843279 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: 6f2e974ab5aa48dfb4b00f810dcedb63 Jul 16 21:26:45.843279 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-DeleteServersTestJSON-1541892181-project Jul 16 21:26:45.843279 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:26:45.843279 ncdv-master-1 devstack@n-api.service[138681]: exception: Instance 1cb418f7-6aea-42e6-88a2-007564d925c4 could not be found. Jul 16 21:26:45.843279 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': '1cb418f7-6aea-42e6-88a2-007564d925c4'} Jul 16 21:26:45.843279 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:26:45.843279 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:26:45.843279 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:26:45.843279 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:26:45.843279 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:26:45.843279 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:26:45.843279 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:26:45.843279 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:26:45.843279 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:26:45.843279 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:26:45.843279 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 1cb418f7-6aea-42e6-88a2-007564d925c4 could not be found. Jul 16 21:26:45.843279 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:26:45.843279 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:26:45.843279 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:26:45.843279 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:26:45.843279 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:26:45.843279 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:26:45.843279 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:26:45.843279 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:26:45.843279 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:26:45.843279 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:26:45.843279 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Jul 16 21:26:45.843279 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Jul 16 21:26:45.843279 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Jul 16 21:26:45.843279 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Jul 16 21:26:45.843279 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:26:45.843279 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:26:45.843279 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:26:45.843279 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:26:45.846648 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 1cb418f7-6aea-42e6-88a2-007564d925c4 could not be found. Jul 16 21:26:45.846648 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:26:45.846648 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:26:45.846648 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:26:45.846648 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:26:45.846648 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:26:45.846648 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:26:45.846648 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:26:45.846648 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:26:45.846648 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 1cb418f7-6aea-42e6-88a2-007564d925c4 could not be found. Jul 16 21:26:45.846648 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:26:45.846648 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-39839a3f-0218-404a-8f1e-6d2ecb909e0a tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Returning 404 to user: Instance 1cb418f7-6aea-42e6-88a2-007564d925c4 could not be found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:26:45.846648 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-39839a3f-0218-404a-8f1e-6d2ecb909e0a tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "DELETE /compute/v2.1/servers/1cb418f7-6aea-42e6-88a2-007564d925c4" status: 404 len: 111 microversion: 2.1 time: 0.032803 Jul 16 21:26:45.846648 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1958/3910] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:26:45 2022] DELETE /compute/v2.1/servers/1cb418f7-6aea-42e6-88a2-007564d925c4 => generated 111 bytes in 34 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:26:45.850127 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-4f64e527-9405-4de3-bc3d-e8f3d72b566f tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Going to run 1 instances... {{(pid=138680) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 16 21:26:45.852736 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-4f64e527-9405-4de3-bc3d-e8f3d72b566f tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: a45c5bbd-ab66-426a-8e31-f471ce68606f] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138680) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 16 21:26:45.853172 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-161e9700-9e06-4074-a4a5-67eb8e35f4c8 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:26:45.853779 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-161e9700-9e06-4074-a4a5-67eb8e35f4c8 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 1cb418f7-6aea-42e6-88a2-007564d925c4] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:26:45.861160 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-161e9700-9e06-4074-a4a5-67eb8e35f4c8 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:45.861921 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-161e9700-9e06-4074-a4a5-67eb8e35f4c8 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:45.865278 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-161e9700-9e06-4074-a4a5-67eb8e35f4c8 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:45.865921 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-161e9700-9e06-4074-a4a5-67eb8e35f4c8 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:45.888859 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-161e9700-9e06-4074-a4a5-67eb8e35f4c8 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] HTTP exception thrown: Instance 1cb418f7-6aea-42e6-88a2-007564d925c4 could not be found. Jul 16 21:26:45.890612 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-161e9700-9e06-4074-a4a5-67eb8e35f4c8 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Request method failure captured: Jul 16 21:26:45.890612 ncdv-master-1 devstack@n-api.service[138681]: request: GET /compute/v2.1/servers/1cb418f7-6aea-42e6-88a2-007564d925c4 HTTP/1.1 Jul 16 21:26:45.890612 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:26:45.890612 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:26:45.890612 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:26:45.890612 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 21:26:45.890612 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:26:45.890612 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:26:45.890612 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:26:45.890612 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:26:45.890612 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0yvs4CI48W3ZCHoQ8huRZwHvqHxAdwrwrvk1v_ETBPx6FU4DdiPUjkjq4hN_4g_fHLAB_XZ9-AAN8Pyvy0KON2X25WlmyilO65MDHbIW9Xy9v9USVPAu_Ce-ze5X9UIHyned0AUneE4UPGiLLIkoXeoS5ekiq1pi209pud2lyUAuHgw Jul 16 21:26:45.890612 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:26:45.890612 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:26:45.890612 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:26:45.890612 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:26:45.890612 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:26:45.890612 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:26:45.890612 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: d31be8475a6a49d6bfe8132dd050c6c0 Jul 16 21:26:45.890612 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-DeleteServersTestJSON-1541892181 Jul 16 21:26:45.890612 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:26:45.890612 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:26:45.890612 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/d31be8475a6a49d6bfe8132dd050c6c0"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/d31be8475a6a49d6bfe8132dd050c6c0"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_d31be8475a6a49d6bfe8132dd050c6c0"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/d31be8475a6a49d6bfe8132dd050c6c0"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:26:45.890612 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-DeleteServersTestJSON-1541892181 Jul 16 21:26:45.890612 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: d31be8475a6a49d6bfe8132dd050c6c0 Jul 16 21:26:45.890612 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-DeleteServersTestJSON-1541892181 Jul 16 21:26:45.890612 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-DeleteServersTestJSON-1541892181-project Jul 16 21:26:45.890612 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:26:45.890612 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:26:45.890612 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: 6f2e974ab5aa48dfb4b00f810dcedb63 Jul 16 21:26:45.890612 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-DeleteServersTestJSON-1541892181-project Jul 16 21:26:45.890612 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:26:45.890612 ncdv-master-1 devstack@n-api.service[138681]: exception: Instance 1cb418f7-6aea-42e6-88a2-007564d925c4 could not be found. Jul 16 21:26:45.890612 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': '1cb418f7-6aea-42e6-88a2-007564d925c4'} Jul 16 21:26:45.890612 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:26:45.890612 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:26:45.890612 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:26:45.890612 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:26:45.890612 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:26:45.890612 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:26:45.890612 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:26:45.890612 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:26:45.890612 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:26:45.890612 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:26:45.890612 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 1cb418f7-6aea-42e6-88a2-007564d925c4 could not be found. Jul 16 21:26:45.890612 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:26:45.890612 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:26:45.890612 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:26:45.890612 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:26:45.890612 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:26:45.890612 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:26:45.890612 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:26:45.890612 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:26:45.890612 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:26:45.890612 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:26:45.890612 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:26:45.890612 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:26:45.890612 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:26:45.890612 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:26:45.890612 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:26:45.890612 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:26:45.890612 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 1cb418f7-6aea-42e6-88a2-007564d925c4 could not be found. Jul 16 21:26:45.890612 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:26:45.890612 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:26:45.890612 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:26:45.895297 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:26:45.895297 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:26:45.895297 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:26:45.895297 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:26:45.895297 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:26:45.895297 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 1cb418f7-6aea-42e6-88a2-007564d925c4 could not be found. Jul 16 21:26:45.895297 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:26:45.896846 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-161e9700-9e06-4074-a4a5-67eb8e35f4c8 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Returning 404 to user: Instance 1cb418f7-6aea-42e6-88a2-007564d925c4 could not be found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:26:45.897753 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-161e9700-9e06-4074-a4a5-67eb8e35f4c8 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "GET /compute/v2.1/servers/1cb418f7-6aea-42e6-88a2-007564d925c4" status: 404 len: 111 microversion: 2.1 time: 0.048238 Jul 16 21:26:45.898350 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1959/3911] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:26:45 2022] GET /compute/v2.1/servers/1cb418f7-6aea-42e6-88a2-007564d925c4 => generated 111 bytes in 49 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Jul 16 21:26:45.916800 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-5db33b59-5360-446a-af18-2de88830a64b tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:26:45.917313 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-5db33b59-5360-446a-af18-2de88830a64b tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: e5d606fa-6108-494c-846f-d40e3273cc64] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:26:45.927641 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5db33b59-5360-446a-af18-2de88830a64b tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:45.928098 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5db33b59-5360-446a-af18-2de88830a64b tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:45.932490 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5db33b59-5360-446a-af18-2de88830a64b tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:45.932490 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5db33b59-5360-446a-af18-2de88830a64b tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:45.947916 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-5db33b59-5360-446a-af18-2de88830a64b tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] HTTP exception thrown: Instance e5d606fa-6108-494c-846f-d40e3273cc64 could not be found. Jul 16 21:26:45.949503 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-5db33b59-5360-446a-af18-2de88830a64b tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Request method failure captured: Jul 16 21:26:45.949503 ncdv-master-1 devstack@n-api.service[138681]: request: DELETE /compute/v2.1/servers/e5d606fa-6108-494c-846f-d40e3273cc64 HTTP/1.1 Jul 16 21:26:45.949503 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:26:45.949503 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:26:45.949503 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:26:45.949503 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 21:26:45.949503 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:26:45.949503 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:26:45.949503 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:26:45.949503 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:26:45.949503 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0yvs4CI48W3ZCHoQ8huRZwHvqHxAdwrwrvk1v_ETBPx6FU4DdiPUjkjq4hN_4g_fHLAB_XZ9-AAN8Pyvy0KON2X25WlmyilO65MDHbIW9Xy9v9USVPAu_Ce-ze5X9UIHyned0AUneE4UPGiLLIkoXeoS5ekiq1pi209pud2lyUAuHgw Jul 16 21:26:45.949503 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:26:45.949503 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:26:45.949503 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:26:45.949503 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:26:45.949503 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:26:45.949503 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:26:45.949503 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: d31be8475a6a49d6bfe8132dd050c6c0 Jul 16 21:26:45.949503 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-DeleteServersTestJSON-1541892181 Jul 16 21:26:45.949503 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:26:45.949503 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:26:45.949503 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/d31be8475a6a49d6bfe8132dd050c6c0"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/d31be8475a6a49d6bfe8132dd050c6c0"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_d31be8475a6a49d6bfe8132dd050c6c0"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/d31be8475a6a49d6bfe8132dd050c6c0"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:26:45.949503 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-DeleteServersTestJSON-1541892181 Jul 16 21:26:45.949503 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: d31be8475a6a49d6bfe8132dd050c6c0 Jul 16 21:26:45.949503 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-DeleteServersTestJSON-1541892181 Jul 16 21:26:45.949503 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-DeleteServersTestJSON-1541892181-project Jul 16 21:26:45.949503 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:26:45.949503 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:26:45.949503 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: 6f2e974ab5aa48dfb4b00f810dcedb63 Jul 16 21:26:45.949503 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-DeleteServersTestJSON-1541892181-project Jul 16 21:26:45.949503 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:26:45.949503 ncdv-master-1 devstack@n-api.service[138681]: exception: Instance e5d606fa-6108-494c-846f-d40e3273cc64 could not be found. Jul 16 21:26:45.949503 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': 'e5d606fa-6108-494c-846f-d40e3273cc64'} Jul 16 21:26:45.949503 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:26:45.949503 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:26:45.949503 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:26:45.949503 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:26:45.949503 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:26:45.949503 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:26:45.949503 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:26:45.949503 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:26:45.949503 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:26:45.949503 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:26:45.949503 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance e5d606fa-6108-494c-846f-d40e3273cc64 could not be found. Jul 16 21:26:45.949503 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:26:45.949503 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:26:45.949503 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:26:45.949503 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:26:45.949503 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:26:45.949503 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:26:45.949503 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:26:45.949503 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:26:45.949503 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:26:45.949503 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:26:45.949503 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Jul 16 21:26:45.949503 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Jul 16 21:26:45.949503 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Jul 16 21:26:45.949503 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Jul 16 21:26:45.949503 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:26:45.949503 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:26:45.949503 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:26:45.949503 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:26:45.953182 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance e5d606fa-6108-494c-846f-d40e3273cc64 could not be found. Jul 16 21:26:45.953182 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:26:45.953182 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:26:45.953182 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:26:45.953182 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:26:45.953182 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:26:45.953182 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:26:45.953182 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:26:45.953182 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:26:45.953182 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance e5d606fa-6108-494c-846f-d40e3273cc64 could not be found. Jul 16 21:26:45.953182 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:26:45.953182 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-5db33b59-5360-446a-af18-2de88830a64b tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Returning 404 to user: Instance e5d606fa-6108-494c-846f-d40e3273cc64 could not be found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:26:45.953182 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-5db33b59-5360-446a-af18-2de88830a64b tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "DELETE /compute/v2.1/servers/e5d606fa-6108-494c-846f-d40e3273cc64" status: 404 len: 111 microversion: 2.1 time: 0.041467 Jul 16 21:26:45.953182 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1960/3912] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:26:45 2022] DELETE /compute/v2.1/servers/e5d606fa-6108-494c-846f-d40e3273cc64 => generated 111 bytes in 43 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:26:45.962015 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-05f224be-26df-45df-a869-54459459229f tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:26:45.963132 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-05f224be-26df-45df-a869-54459459229f tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: e5d606fa-6108-494c-846f-d40e3273cc64] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:26:45.969513 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-05f224be-26df-45df-a869-54459459229f tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:45.970039 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-05f224be-26df-45df-a869-54459459229f tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:45.970915 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-05f224be-26df-45df-a869-54459459229f tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:45.971307 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-05f224be-26df-45df-a869-54459459229f tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:45.997621 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-05f224be-26df-45df-a869-54459459229f tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] HTTP exception thrown: Instance e5d606fa-6108-494c-846f-d40e3273cc64 could not be found. Jul 16 21:26:45.999105 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-4f64e527-9405-4de3-bc3d-e8f3d72b566f tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.815997 Jul 16 21:26:45.999522 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-05f224be-26df-45df-a869-54459459229f tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Request method failure captured: Jul 16 21:26:45.999522 ncdv-master-1 devstack@n-api.service[138681]: request: GET /compute/v2.1/servers/e5d606fa-6108-494c-846f-d40e3273cc64 HTTP/1.1 Jul 16 21:26:45.999522 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:26:45.999522 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:26:45.999522 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:26:45.999522 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 21:26:45.999522 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:26:45.999522 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:26:45.999522 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:26:45.999522 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:26:45.999522 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0yvs4CI48W3ZCHoQ8huRZwHvqHxAdwrwrvk1v_ETBPx6FU4DdiPUjkjq4hN_4g_fHLAB_XZ9-AAN8Pyvy0KON2X25WlmyilO65MDHbIW9Xy9v9USVPAu_Ce-ze5X9UIHyned0AUneE4UPGiLLIkoXeoS5ekiq1pi209pud2lyUAuHgw Jul 16 21:26:45.999522 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:26:45.999522 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:26:45.999522 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:26:45.999522 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:26:45.999522 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:26:45.999522 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:26:45.999522 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: d31be8475a6a49d6bfe8132dd050c6c0 Jul 16 21:26:45.999522 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-DeleteServersTestJSON-1541892181 Jul 16 21:26:45.999522 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:26:45.999522 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:26:45.999522 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/d31be8475a6a49d6bfe8132dd050c6c0"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/d31be8475a6a49d6bfe8132dd050c6c0"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_d31be8475a6a49d6bfe8132dd050c6c0"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/d31be8475a6a49d6bfe8132dd050c6c0"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:26:45.999522 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-DeleteServersTestJSON-1541892181 Jul 16 21:26:45.999522 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: d31be8475a6a49d6bfe8132dd050c6c0 Jul 16 21:26:45.999522 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-DeleteServersTestJSON-1541892181 Jul 16 21:26:45.999522 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-DeleteServersTestJSON-1541892181-project Jul 16 21:26:45.999522 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:26:45.999522 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:26:45.999522 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: 6f2e974ab5aa48dfb4b00f810dcedb63 Jul 16 21:26:45.999522 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-DeleteServersTestJSON-1541892181-project Jul 16 21:26:45.999522 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:26:45.999522 ncdv-master-1 devstack@n-api.service[138681]: exception: Instance e5d606fa-6108-494c-846f-d40e3273cc64 could not be found. Jul 16 21:26:45.999522 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': 'e5d606fa-6108-494c-846f-d40e3273cc64'} Jul 16 21:26:45.999522 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:26:45.999522 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:26:45.999522 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:26:45.999522 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:26:45.999522 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:26:45.999522 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:26:45.999522 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:26:45.999522 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:26:45.999522 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:26:45.999522 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:26:45.999522 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance e5d606fa-6108-494c-846f-d40e3273cc64 could not be found. Jul 16 21:26:45.999522 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:26:45.999522 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:26:45.999522 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:26:45.999522 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:26:45.999522 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:26:45.999522 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:26:45.999522 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:26:45.999522 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:26:45.999522 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:26:45.999522 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:26:45.999522 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:26:45.999522 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:26:45.999522 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:26:45.999522 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:26:45.999522 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:26:45.999522 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:26:45.999522 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance e5d606fa-6108-494c-846f-d40e3273cc64 could not be found. Jul 16 21:26:45.999522 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:26:45.999522 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:26:45.999522 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:26:46.003519 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:26:46.003519 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:26:46.003519 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:26:46.003519 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:26:46.003519 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:26:46.003519 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance e5d606fa-6108-494c-846f-d40e3273cc64 could not be found. Jul 16 21:26:46.003519 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:26:46.003519 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-05f224be-26df-45df-a869-54459459229f tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Returning 404 to user: Instance e5d606fa-6108-494c-846f-d40e3273cc64 could not be found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:26:46.003519 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-05f224be-26df-45df-a869-54459459229f tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "GET /compute/v2.1/servers/e5d606fa-6108-494c-846f-d40e3273cc64" status: 404 len: 111 microversion: 2.1 time: 0.045482 Jul 16 21:26:46.028387 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1953/3913] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 21:26:45 2022] POST /compute/v2.1/servers => generated 380 bytes in 822 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 16 21:26:46.041507 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1961/3914] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:26:45 2022] GET /compute/v2.1/servers/e5d606fa-6108-494c-846f-d40e3273cc64 => generated 111 bytes in 84 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:26:46.042388 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-c5b12b6b-0c10-442e-aa40-0f27622a7d8c tempest-InstanceActionsNegativeTestJSON-904555896 tempest-InstanceActionsNegativeTestJSON-904555896-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:26:46.043228 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-c5b12b6b-0c10-442e-aa40-0f27622a7d8c tempest-InstanceActionsNegativeTestJSON-904555896 tempest-InstanceActionsNegativeTestJSON-904555896-project] [instance: ae4b76bc-e7bd-45ff-8716-361dcbbfaa1d] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:26:46.058252 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-15f7becc-acb0-41c4-8896-eb68640bde26 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:26:46.060386 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-15f7becc-acb0-41c4-8896-eb68640bde26 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: a45c5bbd-ab66-426a-8e31-f471ce68606f] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:26:46.082808 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c5b12b6b-0c10-442e-aa40-0f27622a7d8c tempest-InstanceActionsNegativeTestJSON-904555896 tempest-InstanceActionsNegativeTestJSON-904555896-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:46.083339 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c5b12b6b-0c10-442e-aa40-0f27622a7d8c tempest-InstanceActionsNegativeTestJSON-904555896 tempest-InstanceActionsNegativeTestJSON-904555896-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:46.084167 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c5b12b6b-0c10-442e-aa40-0f27622a7d8c tempest-InstanceActionsNegativeTestJSON-904555896 tempest-InstanceActionsNegativeTestJSON-904555896-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:46.084778 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c5b12b6b-0c10-442e-aa40-0f27622a7d8c tempest-InstanceActionsNegativeTestJSON-904555896 tempest-InstanceActionsNegativeTestJSON-904555896-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:46.119641 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-c5b12b6b-0c10-442e-aa40-0f27622a7d8c tempest-InstanceActionsNegativeTestJSON-904555896 tempest-InstanceActionsNegativeTestJSON-904555896-project] HTTP exception thrown: Instance ae4b76bc-e7bd-45ff-8716-361dcbbfaa1d could not be found. Jul 16 21:26:46.121802 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-c5b12b6b-0c10-442e-aa40-0f27622a7d8c tempest-InstanceActionsNegativeTestJSON-904555896 tempest-InstanceActionsNegativeTestJSON-904555896-project] Request method failure captured: Jul 16 21:26:46.121802 ncdv-master-1 devstack@n-api.service[138680]: request: GET /compute/v2.1/servers/ae4b76bc-e7bd-45ff-8716-361dcbbfaa1d HTTP/1.1 Jul 16 21:26:46.121802 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:26:46.121802 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:26:46.121802 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:26:46.121802 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 0 Jul 16 21:26:46.121802 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:26:46.121802 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:26:46.121802 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:26:46.121802 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:26:46.121802 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0yz8SF1pR_GCLI7xzvIfi_44OK1oBNBDHcBAfLnClU4mtBmY7-QyaipWO1AuqDL5KeIPqXAc56Z1uHiyiWn1sNMEZ7XgCxajnRdgeBkiSesgyWJbY-4VfDxL27uUaG2kAByEz6AQ-BESNileCOHjHH9LY0Rspt_PQAFMIxOsPVQt4JY Jul 16 21:26:46.121802 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:26:46.121802 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:26:46.121802 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:26:46.121802 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:26:46.121802 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:26:46.121802 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:26:46.121802 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: 8f2736abb6094c1f83a7aa3efc772506 Jul 16 21:26:46.121802 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-InstanceActionsNegativeTestJSON-904555896 Jul 16 21:26:46.121802 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:26:46.121802 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:26:46.121802 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/8f2736abb6094c1f83a7aa3efc772506"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/8f2736abb6094c1f83a7aa3efc772506"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_8f2736abb6094c1f83a7aa3efc772506"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/8f2736abb6094c1f83a7aa3efc772506"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:26:46.121802 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-InstanceActionsNegativeTestJSON-904555896 Jul 16 21:26:46.121802 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: 8f2736abb6094c1f83a7aa3efc772506 Jul 16 21:26:46.121802 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-InstanceActionsNegativeTestJSON-904555896 Jul 16 21:26:46.121802 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-InstanceActionsNegativeTestJSON-904555896-project Jul 16 21:26:46.121802 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:26:46.121802 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:26:46.121802 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: 1963f7b0ee0244b2b4f5fc1d8b34f858 Jul 16 21:26:46.121802 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-InstanceActionsNegativeTestJSON-904555896-project Jul 16 21:26:46.121802 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 21:26:46.121802 ncdv-master-1 devstack@n-api.service[138680]: exception: Instance ae4b76bc-e7bd-45ff-8716-361dcbbfaa1d could not be found. Jul 16 21:26:46.121802 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': 'ae4b76bc-e7bd-45ff-8716-361dcbbfaa1d'} Jul 16 21:26:46.121802 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:26:46.121802 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:26:46.121802 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:26:46.121802 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:26:46.121802 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:26:46.121802 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:26:46.121802 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:26:46.121802 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:26:46.121802 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:26:46.121802 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:26:46.121802 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance ae4b76bc-e7bd-45ff-8716-361dcbbfaa1d could not be found. Jul 16 21:26:46.121802 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:26:46.121802 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:26:46.121802 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:26:46.121802 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:26:46.121802 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:26:46.121802 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:26:46.121802 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:26:46.121802 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:26:46.121802 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:26:46.121802 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:26:46.121802 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:26:46.121802 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:26:46.121802 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:26:46.121802 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:26:46.121802 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:26:46.121802 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:26:46.121802 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance ae4b76bc-e7bd-45ff-8716-361dcbbfaa1d could not be found. Jul 16 21:26:46.121802 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:26:46.121802 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:26:46.126666 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:26:46.126666 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:26:46.126666 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:26:46.126666 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:26:46.126666 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:26:46.126666 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:26:46.126666 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance ae4b76bc-e7bd-45ff-8716-361dcbbfaa1d could not be found. Jul 16 21:26:46.126666 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:26:46.128137 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-c5b12b6b-0c10-442e-aa40-0f27622a7d8c tempest-InstanceActionsNegativeTestJSON-904555896 tempest-InstanceActionsNegativeTestJSON-904555896-project] Returning 404 to user: Instance ae4b76bc-e7bd-45ff-8716-361dcbbfaa1d could not be found. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:26:46.129187 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-c5b12b6b-0c10-442e-aa40-0f27622a7d8c tempest-InstanceActionsNegativeTestJSON-904555896 tempest-InstanceActionsNegativeTestJSON-904555896-project] 10.222.0.16 "GET /compute/v2.1/servers/ae4b76bc-e7bd-45ff-8716-361dcbbfaa1d" status: 404 len: 111 microversion: 2.1 time: 0.099480 Jul 16 21:26:46.130348 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1954/3915] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:26:46 2022] GET /compute/v2.1/servers/ae4b76bc-e7bd-45ff-8716-361dcbbfaa1d => generated 111 bytes in 101 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:26:46.139795 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-15f7becc-acb0-41c4-8896-eb68640bde26 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '027dcc48e4fd4c36a5004096ad5e43f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dad823e240149edb30e592380249a13', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:26:46.148662 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-15f7becc-acb0-41c4-8896-eb68640bde26 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "GET /compute/v2.1/servers/a45c5bbd-ab66-426a-8e31-f471ce68606f" status: 200 len: 1208 microversion: 2.1 time: 0.103043 Jul 16 21:26:46.151234 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1962/3916] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:26:46 2022] GET /compute/v2.1/servers/a45c5bbd-ab66-426a-8e31-f471ce68606f => generated 1208 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:26:46.531185 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-a1e97fdf-d4da-45a7-bcc5-acd397cd2554 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Action: 'action', calling method: >, body: {"rescue": {"adminPass": "***"}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:26:46.532220 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-a1e97fdf-d4da-45a7-bcc5-acd397cd2554 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] [instance: c2e32bd9-9d80-427b-bb6d-6274fd48173a] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:26:46.546573 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a1e97fdf-d4da-45a7-bcc5-acd397cd2554 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:46.547342 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a1e97fdf-d4da-45a7-bcc5-acd397cd2554 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:46.549258 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a1e97fdf-d4da-45a7-bcc5-acd397cd2554 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:46.550294 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a1e97fdf-d4da-45a7-bcc5-acd397cd2554 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:46.601945 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.objects.instance [None req-a1e97fdf-d4da-45a7-bcc5-acd397cd2554 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lazy-loading 'flavor' on Instance uuid c2e32bd9-9d80-427b-bb6d-6274fd48173a {{(pid=138680) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:26:46.976830 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-a1e97fdf-d4da-45a7-bcc5-acd397cd2554 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] 10.222.0.16 "POST /compute/v2.1/servers/c2e32bd9-9d80-427b-bb6d-6274fd48173a/action" status: 200 len: 32 microversion: 2.1 time: 0.449541 Jul 16 21:26:46.982030 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1955/3917] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 21:26:46 2022] POST /compute/v2.1/servers/c2e32bd9-9d80-427b-bb6d-6274fd48173a/action => generated 32 bytes in 456 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Jul 16 21:26:46.987391 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-e4e9236b-f926-4915-a7ae-d43372d548ee tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:26:46.988197 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-e4e9236b-f926-4915-a7ae-d43372d548ee tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] [instance: c2e32bd9-9d80-427b-bb6d-6274fd48173a] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:26:47.000936 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e4e9236b-f926-4915-a7ae-d43372d548ee tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:47.001716 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e4e9236b-f926-4915-a7ae-d43372d548ee tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:47.003056 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e4e9236b-f926-4915-a7ae-d43372d548ee tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:47.004813 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e4e9236b-f926-4915-a7ae-d43372d548ee tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:47.167866 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-8d87b608-90d1-4691-9e88-eecbecdb5442 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:26:47.168893 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-8d87b608-90d1-4691-9e88-eecbecdb5442 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: a45c5bbd-ab66-426a-8e31-f471ce68606f] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:26:47.176623 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8d87b608-90d1-4691-9e88-eecbecdb5442 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:47.177443 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8d87b608-90d1-4691-9e88-eecbecdb5442 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:47.178450 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8d87b608-90d1-4691-9e88-eecbecdb5442 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:47.179066 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8d87b608-90d1-4691-9e88-eecbecdb5442 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:47.308102 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-8d87b608-90d1-4691-9e88-eecbecdb5442 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '027dcc48e4fd4c36a5004096ad5e43f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dad823e240149edb30e592380249a13', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:26:47.316280 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-e4e9236b-f926-4915-a7ae-d43372d548ee tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcc9aa2a091c490599f9a9c41163e2eb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da56fc78f06142aba80ca4c6e3d60475', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:26:47.328959 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-e4e9236b-f926-4915-a7ae-d43372d548ee tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] 10.222.0.16 "GET /compute/v2.1/servers/c2e32bd9-9d80-427b-bb6d-6274fd48173a" status: 200 len: 1735 microversion: 2.1 time: 0.345956 Jul 16 21:26:47.340856 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1963/3918] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:26:46 2022] GET /compute/v2.1/servers/c2e32bd9-9d80-427b-bb6d-6274fd48173a => generated 1735 bytes in 347 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:26:47.351365 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-8d87b608-90d1-4691-9e88-eecbecdb5442 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "GET /compute/v2.1/servers/a45c5bbd-ab66-426a-8e31-f471ce68606f" status: 200 len: 1212 microversion: 2.1 time: 0.166125 Jul 16 21:26:47.352532 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1956/3919] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:26:47 2022] GET /compute/v2.1/servers/a45c5bbd-ab66-426a-8e31-f471ce68606f => generated 1212 bytes in 191 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:26:48.352506 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-d3791837-a8c9-4fc6-a14b-c875935b9c72 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:26:48.353882 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-d3791837-a8c9-4fc6-a14b-c875935b9c72 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] [instance: c2e32bd9-9d80-427b-bb6d-6274fd48173a] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:26:48.371250 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-a83f4ced-9001-4c1b-b762-51a2296507ea tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:26:48.371871 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-a83f4ced-9001-4c1b-b762-51a2296507ea tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: a45c5bbd-ab66-426a-8e31-f471ce68606f] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:26:48.372833 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d3791837-a8c9-4fc6-a14b-c875935b9c72 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:48.373725 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d3791837-a8c9-4fc6-a14b-c875935b9c72 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:48.376630 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d3791837-a8c9-4fc6-a14b-c875935b9c72 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:48.377289 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d3791837-a8c9-4fc6-a14b-c875935b9c72 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:48.383194 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a83f4ced-9001-4c1b-b762-51a2296507ea tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:48.386184 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a83f4ced-9001-4c1b-b762-51a2296507ea tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:48.388643 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a83f4ced-9001-4c1b-b762-51a2296507ea tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:48.391692 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a83f4ced-9001-4c1b-b762-51a2296507ea tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:48.544704 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-a83f4ced-9001-4c1b-b762-51a2296507ea tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '027dcc48e4fd4c36a5004096ad5e43f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dad823e240149edb30e592380249a13', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:26:48.557235 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-a83f4ced-9001-4c1b-b762-51a2296507ea tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "GET /compute/v2.1/servers/a45c5bbd-ab66-426a-8e31-f471ce68606f" status: 200 len: 1260 microversion: 2.1 time: 0.189902 Jul 16 21:26:48.558145 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1957/3920] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:26:48 2022] GET /compute/v2.1/servers/a45c5bbd-ab66-426a-8e31-f471ce68606f => generated 1260 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:26:49.194383 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-d3791837-a8c9-4fc6-a14b-c875935b9c72 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcc9aa2a091c490599f9a9c41163e2eb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da56fc78f06142aba80ca4c6e3d60475', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:26:49.219245 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-d3791837-a8c9-4fc6-a14b-c875935b9c72 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] 10.222.0.16 "GET /compute/v2.1/servers/c2e32bd9-9d80-427b-bb6d-6274fd48173a" status: 200 len: 1735 microversion: 2.1 time: 0.873233 Jul 16 21:26:49.220526 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1964/3921] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:26:48 2022] GET /compute/v2.1/servers/c2e32bd9-9d80-427b-bb6d-6274fd48173a => generated 1735 bytes in 875 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:26:49.575643 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-7bd8d85e-9c21-407c-a6d4-330cd437e2f4 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:26:49.576326 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-7bd8d85e-9c21-407c-a6d4-330cd437e2f4 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: a45c5bbd-ab66-426a-8e31-f471ce68606f] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:26:49.584297 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7bd8d85e-9c21-407c-a6d4-330cd437e2f4 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:49.584749 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7bd8d85e-9c21-407c-a6d4-330cd437e2f4 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:49.585535 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7bd8d85e-9c21-407c-a6d4-330cd437e2f4 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:49.585984 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7bd8d85e-9c21-407c-a6d4-330cd437e2f4 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:49.905563 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-7bd8d85e-9c21-407c-a6d4-330cd437e2f4 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '027dcc48e4fd4c36a5004096ad5e43f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dad823e240149edb30e592380249a13', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:26:49.917970 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-7bd8d85e-9c21-407c-a6d4-330cd437e2f4 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "GET /compute/v2.1/servers/a45c5bbd-ab66-426a-8e31-f471ce68606f" status: 200 len: 1266 microversion: 2.1 time: 0.344284 Jul 16 21:26:49.919179 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1958/3922] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:26:49 2022] GET /compute/v2.1/servers/a45c5bbd-ab66-426a-8e31-f471ce68606f => generated 1266 bytes in 348 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:26:50.239700 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-63c40688-5720-496a-8a65-190e3bfbcf70 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:26:50.240326 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-63c40688-5720-496a-8a65-190e3bfbcf70 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] [instance: c2e32bd9-9d80-427b-bb6d-6274fd48173a] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:26:50.252959 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-63c40688-5720-496a-8a65-190e3bfbcf70 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:50.253411 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-63c40688-5720-496a-8a65-190e3bfbcf70 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:50.257117 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-63c40688-5720-496a-8a65-190e3bfbcf70 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:50.257592 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-63c40688-5720-496a-8a65-190e3bfbcf70 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:50.575611 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-63c40688-5720-496a-8a65-190e3bfbcf70 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcc9aa2a091c490599f9a9c41163e2eb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da56fc78f06142aba80ca4c6e3d60475', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:26:50.594530 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-63c40688-5720-496a-8a65-190e3bfbcf70 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] 10.222.0.16 "GET /compute/v2.1/servers/c2e32bd9-9d80-427b-bb6d-6274fd48173a" status: 200 len: 1735 microversion: 2.1 time: 0.360599 Jul 16 21:26:50.597292 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1965/3923] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:26:50 2022] GET /compute/v2.1/servers/c2e32bd9-9d80-427b-bb6d-6274fd48173a => generated 1735 bytes in 364 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:26:50.937708 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-d241bab0-d99c-4a18-a71a-ba256d53bebf tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:26:50.938980 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-d241bab0-d99c-4a18-a71a-ba256d53bebf tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: a45c5bbd-ab66-426a-8e31-f471ce68606f] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:26:50.947441 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d241bab0-d99c-4a18-a71a-ba256d53bebf tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:50.948210 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d241bab0-d99c-4a18-a71a-ba256d53bebf tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:50.949171 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d241bab0-d99c-4a18-a71a-ba256d53bebf tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:50.949791 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d241bab0-d99c-4a18-a71a-ba256d53bebf tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:51.168007 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-d241bab0-d99c-4a18-a71a-ba256d53bebf tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '027dcc48e4fd4c36a5004096ad5e43f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dad823e240149edb30e592380249a13', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:26:51.179147 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-d241bab0-d99c-4a18-a71a-ba256d53bebf tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "GET /compute/v2.1/servers/a45c5bbd-ab66-426a-8e31-f471ce68606f" status: 200 len: 1308 microversion: 2.1 time: 0.244994 Jul 16 21:26:51.180788 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1959/3924] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:26:50 2022] GET /compute/v2.1/servers/a45c5bbd-ab66-426a-8e31-f471ce68606f => generated 1308 bytes in 248 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:26:51.616104 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-82de99c4-ba16-4d59-aa47-724e33dca070 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:26:51.616665 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-82de99c4-ba16-4d59-aa47-724e33dca070 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] [instance: c2e32bd9-9d80-427b-bb6d-6274fd48173a] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:26:51.624394 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-82de99c4-ba16-4d59-aa47-724e33dca070 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:51.624928 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-82de99c4-ba16-4d59-aa47-724e33dca070 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:51.625980 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-82de99c4-ba16-4d59-aa47-724e33dca070 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:51.626640 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-82de99c4-ba16-4d59-aa47-724e33dca070 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:51.743959 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-fcfae375-db79-423e-ab1e-fdb90019d0fa req-7e41ae10-faec-4130-a2d1-b579a734a7c5 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "a45c5bbd-ab66-426a-8e31-f471ce68606f", "tag": "ccf9712c-2c5a-485b-bbdc-293d0cd69551"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:26:51.768452 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-fcfae375-db79-423e-ab1e-fdb90019d0fa req-7e41ae10-faec-4130-a2d1-b579a734a7c5 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:51.769269 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-fcfae375-db79-423e-ab1e-fdb90019d0fa req-7e41ae10-faec-4130-a2d1-b579a734a7c5 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:51.786110 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-fcfae375-db79-423e-ab1e-fdb90019d0fa req-7e41ae10-faec-4130-a2d1-b579a734a7c5 service nova] Creating event network-changed:ccf9712c-2c5a-485b-bbdc-293d0cd69551 for instance a45c5bbd-ab66-426a-8e31-f471ce68606f on nch2-master-1 Jul 16 21:26:51.793947 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-fcfae375-db79-423e-ab1e-fdb90019d0fa req-7e41ae10-faec-4130-a2d1-b579a734a7c5 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.054139 Jul 16 21:26:51.796903 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1960/3925] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:26:51 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 16 21:26:51.918422 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-82de99c4-ba16-4d59-aa47-724e33dca070 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcc9aa2a091c490599f9a9c41163e2eb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da56fc78f06142aba80ca4c6e3d60475', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:26:51.928509 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-82de99c4-ba16-4d59-aa47-724e33dca070 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] 10.222.0.16 "GET /compute/v2.1/servers/c2e32bd9-9d80-427b-bb6d-6274fd48173a" status: 200 len: 1735 microversion: 2.1 time: 0.317260 Jul 16 21:26:51.929803 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1966/3926] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:26:51 2022] GET /compute/v2.1/servers/c2e32bd9-9d80-427b-bb6d-6274fd48173a => generated 1735 bytes in 319 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:26:52.201984 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-0b9f54ae-cb1f-4a3b-8117-e4d9133b6a9a tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:26:52.203214 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-0b9f54ae-cb1f-4a3b-8117-e4d9133b6a9a tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: a45c5bbd-ab66-426a-8e31-f471ce68606f] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:26:52.214657 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0b9f54ae-cb1f-4a3b-8117-e4d9133b6a9a tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:52.215398 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0b9f54ae-cb1f-4a3b-8117-e4d9133b6a9a tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:52.222637 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0b9f54ae-cb1f-4a3b-8117-e4d9133b6a9a tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:52.222637 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0b9f54ae-cb1f-4a3b-8117-e4d9133b6a9a tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:52.444328 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-0b9f54ae-cb1f-4a3b-8117-e4d9133b6a9a tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '027dcc48e4fd4c36a5004096ad5e43f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dad823e240149edb30e592380249a13', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:26:52.452045 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-0b9f54ae-cb1f-4a3b-8117-e4d9133b6a9a tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "GET /compute/v2.1/servers/a45c5bbd-ab66-426a-8e31-f471ce68606f" status: 200 len: 1308 microversion: 2.1 time: 0.257142 Jul 16 21:26:52.452631 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1961/3927] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:26:52 2022] GET /compute/v2.1/servers/a45c5bbd-ab66-426a-8e31-f471ce68606f => generated 1308 bytes in 259 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:26:52.947900 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-be7fd50a-4713-4e1f-a9af-4170b48c832b tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:26:52.948528 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-be7fd50a-4713-4e1f-a9af-4170b48c832b tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] [instance: c2e32bd9-9d80-427b-bb6d-6274fd48173a] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:26:52.957483 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-be7fd50a-4713-4e1f-a9af-4170b48c832b tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:52.958708 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-be7fd50a-4713-4e1f-a9af-4170b48c832b tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:52.961775 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-be7fd50a-4713-4e1f-a9af-4170b48c832b tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:52.961775 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-be7fd50a-4713-4e1f-a9af-4170b48c832b tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:53.136464 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-be7fd50a-4713-4e1f-a9af-4170b48c832b tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcc9aa2a091c490599f9a9c41163e2eb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da56fc78f06142aba80ca4c6e3d60475', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:26:53.139395 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=94,nova_cell1:UPDATE=4,nova_cell1:INSERT=2 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:26:53.148091 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-be7fd50a-4713-4e1f-a9af-4170b48c832b tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] 10.222.0.16 "GET /compute/v2.1/servers/c2e32bd9-9d80-427b-bb6d-6274fd48173a" status: 200 len: 1735 microversion: 2.1 time: 0.204000 Jul 16 21:26:53.149487 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1967/3928] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:26:52 2022] GET /compute/v2.1/servers/c2e32bd9-9d80-427b-bb6d-6274fd48173a => generated 1735 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:26:53.473502 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-188e53a8-2732-4ef0-b802-eb81e057cff8 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:26:53.473938 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-188e53a8-2732-4ef0-b802-eb81e057cff8 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: a45c5bbd-ab66-426a-8e31-f471ce68606f] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:26:53.482037 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-188e53a8-2732-4ef0-b802-eb81e057cff8 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:53.482730 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-188e53a8-2732-4ef0-b802-eb81e057cff8 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:53.483648 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-188e53a8-2732-4ef0-b802-eb81e057cff8 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:53.484315 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-188e53a8-2732-4ef0-b802-eb81e057cff8 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:53.647191 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-188e53a8-2732-4ef0-b802-eb81e057cff8 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '027dcc48e4fd4c36a5004096ad5e43f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dad823e240149edb30e592380249a13', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:26:53.656660 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-188e53a8-2732-4ef0-b802-eb81e057cff8 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "GET /compute/v2.1/servers/a45c5bbd-ab66-426a-8e31-f471ce68606f" status: 200 len: 1308 microversion: 2.1 time: 0.190630 Jul 16 21:26:53.657809 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1962/3929] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:26:53 2022] GET /compute/v2.1/servers/a45c5bbd-ab66-426a-8e31-f471ce68606f => generated 1308 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:26:54.166001 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-19debacd-67dc-4e57-a94c-79d0fedc71da tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:26:54.166635 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-19debacd-67dc-4e57-a94c-79d0fedc71da tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] [instance: c2e32bd9-9d80-427b-bb6d-6274fd48173a] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:26:54.174641 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-19debacd-67dc-4e57-a94c-79d0fedc71da tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:54.175346 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-19debacd-67dc-4e57-a94c-79d0fedc71da tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:54.176681 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-19debacd-67dc-4e57-a94c-79d0fedc71da tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:54.177130 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-19debacd-67dc-4e57-a94c-79d0fedc71da tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:54.512135 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-19debacd-67dc-4e57-a94c-79d0fedc71da tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcc9aa2a091c490599f9a9c41163e2eb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da56fc78f06142aba80ca4c6e3d60475', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:26:54.522994 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-19debacd-67dc-4e57-a94c-79d0fedc71da tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] 10.222.0.16 "GET /compute/v2.1/servers/c2e32bd9-9d80-427b-bb6d-6274fd48173a" status: 200 len: 1735 microversion: 2.1 time: 0.360668 Jul 16 21:26:54.525780 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1968/3930] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:26:54 2022] GET /compute/v2.1/servers/c2e32bd9-9d80-427b-bb6d-6274fd48173a => generated 1735 bytes in 364 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:26:54.675178 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-51a5c787-b382-416d-bed8-5883b2758d76 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:26:54.676426 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-51a5c787-b382-416d-bed8-5883b2758d76 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: a45c5bbd-ab66-426a-8e31-f471ce68606f] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:26:54.683698 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-51a5c787-b382-416d-bed8-5883b2758d76 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:54.684161 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-51a5c787-b382-416d-bed8-5883b2758d76 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:54.684947 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-51a5c787-b382-416d-bed8-5883b2758d76 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:54.685397 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-51a5c787-b382-416d-bed8-5883b2758d76 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:54.928653 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-51a5c787-b382-416d-bed8-5883b2758d76 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '027dcc48e4fd4c36a5004096ad5e43f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dad823e240149edb30e592380249a13', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:26:54.936684 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-51a5c787-b382-416d-bed8-5883b2758d76 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "GET /compute/v2.1/servers/a45c5bbd-ab66-426a-8e31-f471ce68606f" status: 200 len: 1308 microversion: 2.1 time: 0.266125 Jul 16 21:26:54.939760 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1963/3931] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:26:54 2022] GET /compute/v2.1/servers/a45c5bbd-ab66-426a-8e31-f471ce68606f => generated 1308 bytes in 270 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:26:55.448496 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-7823d2c5-f9ed-4e80-9349-a3b44740feee tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:26:55.449518 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-7823d2c5-f9ed-4e80-9349-a3b44740feee tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 32df0911-cfa6-4410-840a-4098a9fef0b4] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:26:55.462354 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_api:SELECT=99,nova_api:UPDATE=1 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:26:55.466493 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7823d2c5-f9ed-4e80-9349-a3b44740feee tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:55.469164 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7823d2c5-f9ed-4e80-9349-a3b44740feee tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:55.470134 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7823d2c5-f9ed-4e80-9349-a3b44740feee tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:55.473195 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7823d2c5-f9ed-4e80-9349-a3b44740feee tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:55.501627 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-7823d2c5-f9ed-4e80-9349-a3b44740feee tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] HTTP exception thrown: Instance 32df0911-cfa6-4410-840a-4098a9fef0b4 could not be found. Jul 16 21:26:55.505446 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-7823d2c5-f9ed-4e80-9349-a3b44740feee tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Request method failure captured: Jul 16 21:26:55.505446 ncdv-master-1 devstack@n-api.service[138681]: request: DELETE /compute/v2.1/servers/32df0911-cfa6-4410-840a-4098a9fef0b4 HTTP/1.1 Jul 16 21:26:55.505446 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:26:55.505446 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:26:55.505446 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:26:55.505446 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 21:26:55.505446 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:26:55.505446 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:26:55.505446 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:26:55.505446 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:26:55.505446 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0yvs4CI48W3ZCHoQ8huRZwHvqHxAdwrwrvk1v_ETBPx6FU4DdiPUjkjq4hN_4g_fHLAB_XZ9-AAN8Pyvy0KON2X25WlmyilO65MDHbIW9Xy9v9USVPAu_Ce-ze5X9UIHyned0AUneE4UPGiLLIkoXeoS5ekiq1pi209pud2lyUAuHgw Jul 16 21:26:55.505446 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:26:55.505446 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:26:55.505446 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:26:55.505446 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:26:55.505446 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:26:55.505446 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:26:55.505446 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: d31be8475a6a49d6bfe8132dd050c6c0 Jul 16 21:26:55.505446 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-DeleteServersTestJSON-1541892181 Jul 16 21:26:55.505446 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:26:55.505446 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:26:55.505446 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/d31be8475a6a49d6bfe8132dd050c6c0"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/d31be8475a6a49d6bfe8132dd050c6c0"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_d31be8475a6a49d6bfe8132dd050c6c0"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/d31be8475a6a49d6bfe8132dd050c6c0"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:26:55.505446 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-DeleteServersTestJSON-1541892181 Jul 16 21:26:55.505446 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: d31be8475a6a49d6bfe8132dd050c6c0 Jul 16 21:26:55.505446 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-DeleteServersTestJSON-1541892181 Jul 16 21:26:55.505446 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-DeleteServersTestJSON-1541892181-project Jul 16 21:26:55.505446 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:26:55.505446 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:26:55.505446 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: 6f2e974ab5aa48dfb4b00f810dcedb63 Jul 16 21:26:55.505446 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-DeleteServersTestJSON-1541892181-project Jul 16 21:26:55.505446 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:26:55.505446 ncdv-master-1 devstack@n-api.service[138681]: exception: Instance 32df0911-cfa6-4410-840a-4098a9fef0b4 could not be found. Jul 16 21:26:55.505446 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': '32df0911-cfa6-4410-840a-4098a9fef0b4'} Jul 16 21:26:55.505446 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:26:55.505446 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:26:55.505446 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:26:55.505446 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:26:55.505446 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:26:55.505446 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:26:55.505446 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:26:55.505446 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:26:55.505446 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:26:55.505446 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:26:55.505446 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 32df0911-cfa6-4410-840a-4098a9fef0b4 could not be found. Jul 16 21:26:55.505446 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:26:55.505446 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:26:55.505446 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:26:55.505446 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:26:55.505446 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:26:55.505446 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:26:55.505446 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:26:55.505446 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:26:55.505446 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:26:55.505446 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:26:55.505446 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Jul 16 21:26:55.505446 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Jul 16 21:26:55.505446 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Jul 16 21:26:55.505446 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Jul 16 21:26:55.505446 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:26:55.505446 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:26:55.505446 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:26:55.505446 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:26:55.510018 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 32df0911-cfa6-4410-840a-4098a9fef0b4 could not be found. Jul 16 21:26:55.510018 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:26:55.510018 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:26:55.510018 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:26:55.510018 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:26:55.510018 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:26:55.510018 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:26:55.510018 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:26:55.510018 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:26:55.510018 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 32df0911-cfa6-4410-840a-4098a9fef0b4 could not be found. Jul 16 21:26:55.510018 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:26:55.511550 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-7823d2c5-f9ed-4e80-9349-a3b44740feee tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Returning 404 to user: Instance 32df0911-cfa6-4410-840a-4098a9fef0b4 could not be found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:26:55.513291 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-7823d2c5-f9ed-4e80-9349-a3b44740feee tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "DELETE /compute/v2.1/servers/32df0911-cfa6-4410-840a-4098a9fef0b4" status: 404 len: 111 microversion: 2.1 time: 0.068021 Jul 16 21:26:55.522965 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-8d7ec248-9307-4693-b448-0af6a53a3f6b tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:26:55.523830 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-8d7ec248-9307-4693-b448-0af6a53a3f6b tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: 32df0911-cfa6-4410-840a-4098a9fef0b4] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:26:55.527361 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1969/3932] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:26:55 2022] DELETE /compute/v2.1/servers/32df0911-cfa6-4410-840a-4098a9fef0b4 => generated 111 bytes in 83 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:26:55.530445 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8d7ec248-9307-4693-b448-0af6a53a3f6b tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:55.545096 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-d40fb58e-3d99-40b2-83a5-ad033c230ba8 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:26:55.545918 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-d40fb58e-3d99-40b2-83a5-ad033c230ba8 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] [instance: c2e32bd9-9d80-427b-bb6d-6274fd48173a] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:26:55.552128 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8d7ec248-9307-4693-b448-0af6a53a3f6b tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.022s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:55.552980 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8d7ec248-9307-4693-b448-0af6a53a3f6b tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:55.553426 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8d7ec248-9307-4693-b448-0af6a53a3f6b tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:55.555136 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d40fb58e-3d99-40b2-83a5-ad033c230ba8 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:55.556878 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d40fb58e-3d99-40b2-83a5-ad033c230ba8 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:55.558481 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d40fb58e-3d99-40b2-83a5-ad033c230ba8 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:55.559100 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d40fb58e-3d99-40b2-83a5-ad033c230ba8 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:55.583281 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-8d7ec248-9307-4693-b448-0af6a53a3f6b tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] HTTP exception thrown: Instance 32df0911-cfa6-4410-840a-4098a9fef0b4 could not be found. Jul 16 21:26:55.585259 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-8d7ec248-9307-4693-b448-0af6a53a3f6b tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Request method failure captured: Jul 16 21:26:55.585259 ncdv-master-1 devstack@n-api.service[138680]: request: GET /compute/v2.1/servers/32df0911-cfa6-4410-840a-4098a9fef0b4 HTTP/1.1 Jul 16 21:26:55.585259 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:26:55.585259 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:26:55.585259 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:26:55.585259 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 0 Jul 16 21:26:55.585259 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:26:55.585259 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:26:55.585259 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:26:55.585259 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:26:55.585259 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0yvs4CI48W3ZCHoQ8huRZwHvqHxAdwrwrvk1v_ETBPx6FU4DdiPUjkjq4hN_4g_fHLAB_XZ9-AAN8Pyvy0KON2X25WlmyilO65MDHbIW9Xy9v9USVPAu_Ce-ze5X9UIHyned0AUneE4UPGiLLIkoXeoS5ekiq1pi209pud2lyUAuHgw Jul 16 21:26:55.585259 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:26:55.585259 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:26:55.585259 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:26:55.585259 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:26:55.585259 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:26:55.585259 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:26:55.585259 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: d31be8475a6a49d6bfe8132dd050c6c0 Jul 16 21:26:55.585259 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-DeleteServersTestJSON-1541892181 Jul 16 21:26:55.585259 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:26:55.585259 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:26:55.585259 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/d31be8475a6a49d6bfe8132dd050c6c0"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/d31be8475a6a49d6bfe8132dd050c6c0"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_d31be8475a6a49d6bfe8132dd050c6c0"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/d31be8475a6a49d6bfe8132dd050c6c0"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:26:55.585259 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-DeleteServersTestJSON-1541892181 Jul 16 21:26:55.585259 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: d31be8475a6a49d6bfe8132dd050c6c0 Jul 16 21:26:55.585259 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-DeleteServersTestJSON-1541892181 Jul 16 21:26:55.585259 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-DeleteServersTestJSON-1541892181-project Jul 16 21:26:55.585259 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:26:55.585259 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:26:55.585259 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: 6f2e974ab5aa48dfb4b00f810dcedb63 Jul 16 21:26:55.585259 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-DeleteServersTestJSON-1541892181-project Jul 16 21:26:55.585259 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 21:26:55.585259 ncdv-master-1 devstack@n-api.service[138680]: exception: Instance 32df0911-cfa6-4410-840a-4098a9fef0b4 could not be found. Jul 16 21:26:55.585259 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': '32df0911-cfa6-4410-840a-4098a9fef0b4'} Jul 16 21:26:55.585259 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:26:55.585259 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:26:55.585259 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:26:55.585259 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:26:55.585259 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:26:55.585259 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:26:55.585259 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:26:55.585259 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:26:55.585259 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:26:55.585259 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:26:55.585259 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 32df0911-cfa6-4410-840a-4098a9fef0b4 could not be found. Jul 16 21:26:55.585259 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:26:55.585259 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:26:55.585259 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:26:55.585259 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:26:55.585259 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:26:55.585259 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:26:55.585259 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:26:55.585259 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:26:55.585259 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:26:55.585259 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:26:55.585259 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:26:55.585259 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:26:55.585259 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:26:55.585259 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:26:55.585259 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:26:55.585259 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:26:55.585259 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 32df0911-cfa6-4410-840a-4098a9fef0b4 could not be found. Jul 16 21:26:55.585259 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:26:55.585259 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:26:55.585259 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:26:55.588846 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:26:55.588846 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:26:55.588846 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:26:55.588846 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:26:55.588846 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:26:55.588846 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 32df0911-cfa6-4410-840a-4098a9fef0b4 could not be found. Jul 16 21:26:55.588846 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:26:55.588846 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-8d7ec248-9307-4693-b448-0af6a53a3f6b tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Returning 404 to user: Instance 32df0911-cfa6-4410-840a-4098a9fef0b4 could not be found. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:26:55.588846 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-8d7ec248-9307-4693-b448-0af6a53a3f6b tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "GET /compute/v2.1/servers/32df0911-cfa6-4410-840a-4098a9fef0b4" status: 404 len: 111 microversion: 2.1 time: 0.069001 Jul 16 21:26:55.591843 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1964/3933] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:26:55 2022] GET /compute/v2.1/servers/32df0911-cfa6-4410-840a-4098a9fef0b4 => generated 111 bytes in 74 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:26:55.816143 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell0:SELECT=2 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:26:55.830532 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-d40fb58e-3d99-40b2-83a5-ad033c230ba8 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcc9aa2a091c490599f9a9c41163e2eb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da56fc78f06142aba80ca4c6e3d60475', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:26:55.839740 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell0:SELECT=6 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:26:55.844207 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-d40fb58e-3d99-40b2-83a5-ad033c230ba8 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] 10.222.0.16 "GET /compute/v2.1/servers/c2e32bd9-9d80-427b-bb6d-6274fd48173a" status: 200 len: 1735 microversion: 2.1 time: 0.307693 Jul 16 21:26:55.853185 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1970/3934] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:26:55 2022] GET /compute/v2.1/servers/c2e32bd9-9d80-427b-bb6d-6274fd48173a => generated 1735 bytes in 316 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:26:55.858691 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-4db42ff3-9cf5-46ee-8d66-2bee70b1e331 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:26:55.859244 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-4db42ff3-9cf5-46ee-8d66-2bee70b1e331 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: a4f887a4-84e8-4b14-beca-e6ad747ba616] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:26:55.877494 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4db42ff3-9cf5-46ee-8d66-2bee70b1e331 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:55.880702 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4db42ff3-9cf5-46ee-8d66-2bee70b1e331 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.010s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:55.880702 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4db42ff3-9cf5-46ee-8d66-2bee70b1e331 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:55.880702 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4db42ff3-9cf5-46ee-8d66-2bee70b1e331 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:55.958685 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-4db42ff3-9cf5-46ee-8d66-2bee70b1e331 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] HTTP exception thrown: Instance a4f887a4-84e8-4b14-beca-e6ad747ba616 could not be found. Jul 16 21:26:55.960623 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-4db42ff3-9cf5-46ee-8d66-2bee70b1e331 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Request method failure captured: Jul 16 21:26:55.960623 ncdv-master-1 devstack@n-api.service[138681]: request: DELETE /compute/v2.1/servers/a4f887a4-84e8-4b14-beca-e6ad747ba616 HTTP/1.1 Jul 16 21:26:55.960623 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:26:55.960623 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:26:55.960623 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:26:55.960623 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 21:26:55.960623 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:26:55.960623 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:26:55.960623 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:26:55.960623 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:26:55.960623 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0yvs4CI48W3ZCHoQ8huRZwHvqHxAdwrwrvk1v_ETBPx6FU4DdiPUjkjq4hN_4g_fHLAB_XZ9-AAN8Pyvy0KON2X25WlmyilO65MDHbIW9Xy9v9USVPAu_Ce-ze5X9UIHyned0AUneE4UPGiLLIkoXeoS5ekiq1pi209pud2lyUAuHgw Jul 16 21:26:55.960623 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:26:55.960623 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:26:55.960623 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:26:55.960623 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:26:55.960623 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:26:55.960623 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:26:55.960623 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: d31be8475a6a49d6bfe8132dd050c6c0 Jul 16 21:26:55.960623 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-DeleteServersTestJSON-1541892181 Jul 16 21:26:55.960623 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:26:55.960623 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:26:55.960623 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/d31be8475a6a49d6bfe8132dd050c6c0"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/d31be8475a6a49d6bfe8132dd050c6c0"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_d31be8475a6a49d6bfe8132dd050c6c0"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/d31be8475a6a49d6bfe8132dd050c6c0"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:26:55.960623 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-DeleteServersTestJSON-1541892181 Jul 16 21:26:55.960623 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: d31be8475a6a49d6bfe8132dd050c6c0 Jul 16 21:26:55.960623 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-DeleteServersTestJSON-1541892181 Jul 16 21:26:55.960623 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-DeleteServersTestJSON-1541892181-project Jul 16 21:26:55.960623 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:26:55.960623 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:26:55.960623 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: 6f2e974ab5aa48dfb4b00f810dcedb63 Jul 16 21:26:55.960623 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-DeleteServersTestJSON-1541892181-project Jul 16 21:26:55.960623 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:26:55.960623 ncdv-master-1 devstack@n-api.service[138681]: exception: Instance a4f887a4-84e8-4b14-beca-e6ad747ba616 could not be found. Jul 16 21:26:55.960623 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': 'a4f887a4-84e8-4b14-beca-e6ad747ba616'} Jul 16 21:26:55.960623 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:26:55.960623 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:26:55.960623 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:26:55.960623 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:26:55.960623 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:26:55.960623 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:26:55.960623 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:26:55.960623 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:26:55.960623 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:26:55.960623 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:26:55.960623 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance a4f887a4-84e8-4b14-beca-e6ad747ba616 could not be found. Jul 16 21:26:55.960623 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:26:55.960623 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:26:55.960623 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:26:55.960623 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:26:55.960623 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:26:55.960623 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:26:55.960623 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:26:55.960623 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:26:55.960623 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:26:55.960623 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:26:55.960623 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Jul 16 21:26:55.960623 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Jul 16 21:26:55.960623 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Jul 16 21:26:55.960623 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Jul 16 21:26:55.960623 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:26:55.960623 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:26:55.960623 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:26:55.960623 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:26:55.970607 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance a4f887a4-84e8-4b14-beca-e6ad747ba616 could not be found. Jul 16 21:26:55.970607 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:26:55.970607 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:26:55.970607 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:26:55.970607 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:26:55.970607 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:26:55.970607 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:26:55.970607 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:26:55.970607 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:26:55.970607 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance a4f887a4-84e8-4b14-beca-e6ad747ba616 could not be found. Jul 16 21:26:55.970607 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:26:55.970607 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-4db42ff3-9cf5-46ee-8d66-2bee70b1e331 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Returning 404 to user: Instance a4f887a4-84e8-4b14-beca-e6ad747ba616 could not be found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:26:55.970607 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-4db42ff3-9cf5-46ee-8d66-2bee70b1e331 tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "DELETE /compute/v2.1/servers/a4f887a4-84e8-4b14-beca-e6ad747ba616" status: 404 len: 111 microversion: 2.1 time: 0.111399 Jul 16 21:26:55.970607 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1971/3935] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:26:55 2022] DELETE /compute/v2.1/servers/a4f887a4-84e8-4b14-beca-e6ad747ba616 => generated 111 bytes in 113 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:26:55.979470 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-13f8725a-1c15-42e6-9d89-1cf7a29beba7 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:26:55.982218 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-13f8725a-1c15-42e6-9d89-1cf7a29beba7 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: a45c5bbd-ab66-426a-8e31-f471ce68606f] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:26:55.996201 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-13f8725a-1c15-42e6-9d89-1cf7a29beba7 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:56.004799 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-13f8725a-1c15-42e6-9d89-1cf7a29beba7 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:56.004799 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-13f8725a-1c15-42e6-9d89-1cf7a29beba7 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:56.004799 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-13f8725a-1c15-42e6-9d89-1cf7a29beba7 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:56.144668 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell0:SELECT=2 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:26:56.154826 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-27de9325-8a22-495c-9793-b287fb007673 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ListServerFiltersTestJSON-instance-1583187080", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452", "networks": [{"uuid": "87929b0c-a3bd-4f87-8248-ab83363a54b1"}]}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:26:56.213497 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.network.neutron [None req-27de9325-8a22-495c-9793-b287fb007673 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] validate_networks() for [('87929b0c-a3bd-4f87-8248-ab83363a54b1', None, None, None, None, None)] {{(pid=138680) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 16 21:26:56.214762 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-13f8725a-1c15-42e6-9d89-1cf7a29beba7 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '027dcc48e4fd4c36a5004096ad5e43f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dad823e240149edb30e592380249a13', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:26:56.223804 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-13f8725a-1c15-42e6-9d89-1cf7a29beba7 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "GET /compute/v2.1/servers/a45c5bbd-ab66-426a-8e31-f471ce68606f" status: 200 len: 1308 microversion: 2.1 time: 0.255514 Jul 16 21:26:56.224476 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1972/3936] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:26:55 2022] GET /compute/v2.1/servers/a45c5bbd-ab66-426a-8e31-f471ce68606f => generated 1308 bytes in 257 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:26:56.236908 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-f0735c29-74db-401d-a43c-7ca34da6f3cb tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:26:56.239367 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-f0735c29-74db-401d-a43c-7ca34da6f3cb tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] [instance: a4f887a4-84e8-4b14-beca-e6ad747ba616] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:26:56.252441 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f0735c29-74db-401d-a43c-7ca34da6f3cb tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:56.253083 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f0735c29-74db-401d-a43c-7ca34da6f3cb tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:56.254517 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f0735c29-74db-401d-a43c-7ca34da6f3cb tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:56.255217 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f0735c29-74db-401d-a43c-7ca34da6f3cb tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:56.283371 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-f0735c29-74db-401d-a43c-7ca34da6f3cb tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] HTTP exception thrown: Instance a4f887a4-84e8-4b14-beca-e6ad747ba616 could not be found. Jul 16 21:26:56.285000 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-f0735c29-74db-401d-a43c-7ca34da6f3cb tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Request method failure captured: Jul 16 21:26:56.285000 ncdv-master-1 devstack@n-api.service[138681]: request: GET /compute/v2.1/servers/a4f887a4-84e8-4b14-beca-e6ad747ba616 HTTP/1.1 Jul 16 21:26:56.285000 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:26:56.285000 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:26:56.285000 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:26:56.285000 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 21:26:56.285000 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:26:56.285000 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:26:56.285000 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:26:56.285000 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:26:56.285000 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0yvs4CI48W3ZCHoQ8huRZwHvqHxAdwrwrvk1v_ETBPx6FU4DdiPUjkjq4hN_4g_fHLAB_XZ9-AAN8Pyvy0KON2X25WlmyilO65MDHbIW9Xy9v9USVPAu_Ce-ze5X9UIHyned0AUneE4UPGiLLIkoXeoS5ekiq1pi209pud2lyUAuHgw Jul 16 21:26:56.285000 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:26:56.285000 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:26:56.285000 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:26:56.285000 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:26:56.285000 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:26:56.285000 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:26:56.285000 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: d31be8475a6a49d6bfe8132dd050c6c0 Jul 16 21:26:56.285000 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-DeleteServersTestJSON-1541892181 Jul 16 21:26:56.285000 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:26:56.285000 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:26:56.285000 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/d31be8475a6a49d6bfe8132dd050c6c0"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/d31be8475a6a49d6bfe8132dd050c6c0"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_d31be8475a6a49d6bfe8132dd050c6c0"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/d31be8475a6a49d6bfe8132dd050c6c0"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:26:56.285000 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-DeleteServersTestJSON-1541892181 Jul 16 21:26:56.285000 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: d31be8475a6a49d6bfe8132dd050c6c0 Jul 16 21:26:56.285000 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-DeleteServersTestJSON-1541892181 Jul 16 21:26:56.285000 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-DeleteServersTestJSON-1541892181-project Jul 16 21:26:56.285000 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:26:56.285000 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:26:56.285000 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: 6f2e974ab5aa48dfb4b00f810dcedb63 Jul 16 21:26:56.285000 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-DeleteServersTestJSON-1541892181-project Jul 16 21:26:56.285000 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:26:56.285000 ncdv-master-1 devstack@n-api.service[138681]: exception: Instance a4f887a4-84e8-4b14-beca-e6ad747ba616 could not be found. Jul 16 21:26:56.285000 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': 'a4f887a4-84e8-4b14-beca-e6ad747ba616'} Jul 16 21:26:56.285000 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:26:56.285000 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:26:56.285000 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:26:56.285000 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:26:56.285000 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:26:56.285000 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:26:56.285000 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:26:56.285000 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:26:56.285000 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:26:56.285000 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:26:56.285000 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance a4f887a4-84e8-4b14-beca-e6ad747ba616 could not be found. Jul 16 21:26:56.285000 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:26:56.285000 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:26:56.285000 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:26:56.285000 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:26:56.285000 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:26:56.285000 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:26:56.285000 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:26:56.285000 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:26:56.285000 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:26:56.285000 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:26:56.285000 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:26:56.285000 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:26:56.285000 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:26:56.285000 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:26:56.285000 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:26:56.285000 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:26:56.285000 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance a4f887a4-84e8-4b14-beca-e6ad747ba616 could not be found. Jul 16 21:26:56.285000 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:26:56.285000 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:26:56.285000 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:26:56.293649 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:26:56.293649 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:26:56.293649 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:26:56.293649 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:26:56.293649 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:26:56.293649 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance a4f887a4-84e8-4b14-beca-e6ad747ba616 could not be found. Jul 16 21:26:56.293649 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:26:56.293649 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-f0735c29-74db-401d-a43c-7ca34da6f3cb tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] Returning 404 to user: Instance a4f887a4-84e8-4b14-beca-e6ad747ba616 could not be found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:26:56.293649 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-f0735c29-74db-401d-a43c-7ca34da6f3cb tempest-DeleteServersTestJSON-1541892181 tempest-DeleteServersTestJSON-1541892181-project] 10.222.0.16 "GET /compute/v2.1/servers/a4f887a4-84e8-4b14-beca-e6ad747ba616" status: 404 len: 111 microversion: 2.1 time: 0.060794 Jul 16 21:26:56.293649 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1973/3937] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:26:56 2022] GET /compute/v2.1/servers/a4f887a4-84e8-4b14-beca-e6ad747ba616 => generated 111 bytes in 62 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:26:56.829912 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-27de9325-8a22-495c-9793-b287fb007673 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Flavor limits 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 16 21:26:56.831114 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-27de9325-8a22-495c-9793-b287fb007673 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Image limits 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 16 21:26:56.833684 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-27de9325-8a22-495c-9793-b287fb007673 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Flavor pref 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 16 21:26:56.834354 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-27de9325-8a22-495c-9793-b287fb007673 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Image pref 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 16 21:26:56.835023 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-27de9325-8a22-495c-9793-b287fb007673 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 16 21:26:56.847464 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-27de9325-8a22-495c-9793-b287fb007673 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:56.849592 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-27de9325-8a22-495c-9793-b287fb007673 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:56.850606 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-27de9325-8a22-495c-9793-b287fb007673 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:56.851421 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-27de9325-8a22-495c-9793-b287fb007673 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:56.865646 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-62431e16-03bd-41fa-947f-ba3757a1b090 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:26:56.866322 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-62431e16-03bd-41fa-947f-ba3757a1b090 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] [instance: c2e32bd9-9d80-427b-bb6d-6274fd48173a] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:26:56.881938 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-62431e16-03bd-41fa-947f-ba3757a1b090 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:56.883692 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-62431e16-03bd-41fa-947f-ba3757a1b090 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:56.885509 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-62431e16-03bd-41fa-947f-ba3757a1b090 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:56.885967 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-62431e16-03bd-41fa-947f-ba3757a1b090 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:56.907308 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-27de9325-8a22-495c-9793-b287fb007673 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Getting quotas for project 3b214f5044c7445592e177835a5d18fc. Resources: {'ram', 'instances', 'cores'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:26:56.915764 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-27de9325-8a22-495c-9793-b287fb007673 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Getting quotas for user 029c76a0ba614b5880128c4309902c67 and project 3b214f5044c7445592e177835a5d18fc. Resources: {'ram', 'instances', 'cores'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:26:56.934416 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-27de9325-8a22-495c-9793-b287fb007673 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Going to run 1 instances... {{(pid=138680) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 16 21:26:56.937075 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-27de9325-8a22-495c-9793-b287fb007673 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] [instance: cf38328b-40fb-4537-b524-aaeb0a0abfb4] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138680) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 16 21:26:57.069989 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-27de9325-8a22-495c-9793-b287fb007673 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 1.476707 Jul 16 21:26:57.071258 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1965/3938] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 21:26:55 2022] POST /compute/v2.1/servers => generated 380 bytes in 1479 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 16 21:26:57.091136 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-6b5da4ea-ec43-4a24-8a6d-3913296635b7 req-e81ce4ea-70a3-4f60-ac98-8fd866eef07f service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "a45c5bbd-ab66-426a-8e31-f471ce68606f", "name": "network-vif-plugged", "status": "completed", "tag": "ccf9712c-2c5a-485b-bbdc-293d0cd69551"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:26:57.141771 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-6b5da4ea-ec43-4a24-8a6d-3913296635b7 req-e81ce4ea-70a3-4f60-ac98-8fd866eef07f service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:57.142283 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-6b5da4ea-ec43-4a24-8a6d-3913296635b7 req-e81ce4ea-70a3-4f60-ac98-8fd866eef07f service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.022s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:57.179293 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-6b5da4ea-ec43-4a24-8a6d-3913296635b7 req-e81ce4ea-70a3-4f60-ac98-8fd866eef07f service nova] Creating event network-vif-plugged:ccf9712c-2c5a-485b-bbdc-293d0cd69551 for instance a45c5bbd-ab66-426a-8e31-f471ce68606f on nch2-master-1 Jul 16 21:26:57.188280 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-62431e16-03bd-41fa-947f-ba3757a1b090 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcc9aa2a091c490599f9a9c41163e2eb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da56fc78f06142aba80ca4c6e3d60475', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:26:57.193491 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-6b5da4ea-ec43-4a24-8a6d-3913296635b7 req-e81ce4ea-70a3-4f60-ac98-8fd866eef07f service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.120735 Jul 16 21:26:57.199777 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1966/3939] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:26:57 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 128 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:26:57.201028 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-62431e16-03bd-41fa-947f-ba3757a1b090 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] 10.222.0.16 "GET /compute/v2.1/servers/c2e32bd9-9d80-427b-bb6d-6274fd48173a" status: 200 len: 1735 microversion: 2.1 time: 0.341083 Jul 16 21:26:57.202678 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1974/3940] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:26:56 2022] GET /compute/v2.1/servers/c2e32bd9-9d80-427b-bb6d-6274fd48173a => generated 1735 bytes in 343 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:26:57.242028 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-2b336462-dd9f-4ba9-87be-295b55b252b1 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:26:57.243195 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-2b336462-dd9f-4ba9-87be-295b55b252b1 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: a45c5bbd-ab66-426a-8e31-f471ce68606f] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:26:57.256658 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2b336462-dd9f-4ba9-87be-295b55b252b1 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:57.258061 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2b336462-dd9f-4ba9-87be-295b55b252b1 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:57.259071 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2b336462-dd9f-4ba9-87be-295b55b252b1 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:57.261656 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2b336462-dd9f-4ba9-87be-295b55b252b1 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:57.278253 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-547cce8e-74ab-4795-9117-e60e04cbd418 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ListServerFiltersTestJSON-instance-733651621", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452", "networks": [{"uuid": "87929b0c-a3bd-4f87-8248-ab83363a54b1"}]}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:26:57.422589 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.network.neutron [None req-547cce8e-74ab-4795-9117-e60e04cbd418 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] validate_networks() for [('87929b0c-a3bd-4f87-8248-ab83363a54b1', None, None, None, None, None)] {{(pid=138680) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 16 21:26:57.862538 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-2b336462-dd9f-4ba9-87be-295b55b252b1 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '027dcc48e4fd4c36a5004096ad5e43f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dad823e240149edb30e592380249a13', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:26:57.874788 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-2b336462-dd9f-4ba9-87be-295b55b252b1 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "GET /compute/v2.1/servers/a45c5bbd-ab66-426a-8e31-f471ce68606f" status: 200 len: 1308 microversion: 2.1 time: 0.637499 Jul 16 21:26:57.886605 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1975/3941] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:26:57 2022] GET /compute/v2.1/servers/a45c5bbd-ab66-426a-8e31-f471ce68606f => generated 1308 bytes in 650 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:26:58.219839 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-7a3f363d-b5de-42b3-b951-ba85b93abe77 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:26:58.220461 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-7a3f363d-b5de-42b3-b951-ba85b93abe77 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] [instance: c2e32bd9-9d80-427b-bb6d-6274fd48173a] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:26:58.231160 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7a3f363d-b5de-42b3-b951-ba85b93abe77 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:58.231614 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7a3f363d-b5de-42b3-b951-ba85b93abe77 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:58.232405 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7a3f363d-b5de-42b3-b951-ba85b93abe77 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:58.232851 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7a3f363d-b5de-42b3-b951-ba85b93abe77 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:58.435321 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-547cce8e-74ab-4795-9117-e60e04cbd418 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Flavor limits 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 16 21:26:58.436398 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-547cce8e-74ab-4795-9117-e60e04cbd418 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Image limits 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 16 21:26:58.437114 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-547cce8e-74ab-4795-9117-e60e04cbd418 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Flavor pref 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 16 21:26:58.437735 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-547cce8e-74ab-4795-9117-e60e04cbd418 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Image pref 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 16 21:26:58.438499 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-547cce8e-74ab-4795-9117-e60e04cbd418 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 16 21:26:58.459890 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-547cce8e-74ab-4795-9117-e60e04cbd418 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:58.460687 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-547cce8e-74ab-4795-9117-e60e04cbd418 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:58.461610 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-547cce8e-74ab-4795-9117-e60e04cbd418 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:58.463397 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-547cce8e-74ab-4795-9117-e60e04cbd418 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:58.475160 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-7a3f363d-b5de-42b3-b951-ba85b93abe77 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcc9aa2a091c490599f9a9c41163e2eb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da56fc78f06142aba80ca4c6e3d60475', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:26:58.485271 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-547cce8e-74ab-4795-9117-e60e04cbd418 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Getting quotas for project 3b214f5044c7445592e177835a5d18fc. Resources: {'ram', 'instances', 'cores'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:26:58.487072 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-7a3f363d-b5de-42b3-b951-ba85b93abe77 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] 10.222.0.16 "GET /compute/v2.1/servers/c2e32bd9-9d80-427b-bb6d-6274fd48173a" status: 200 len: 1735 microversion: 2.1 time: 0.269676 Jul 16 21:26:58.489312 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1976/3942] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:26:58 2022] GET /compute/v2.1/servers/c2e32bd9-9d80-427b-bb6d-6274fd48173a => generated 1735 bytes in 274 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:26:58.504295 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-547cce8e-74ab-4795-9117-e60e04cbd418 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Getting quotas for user 029c76a0ba614b5880128c4309902c67 and project 3b214f5044c7445592e177835a5d18fc. Resources: {'ram', 'instances', 'cores'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:26:58.518841 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-547cce8e-74ab-4795-9117-e60e04cbd418 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Going to run 1 instances... {{(pid=138680) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 16 21:26:58.523589 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-547cce8e-74ab-4795-9117-e60e04cbd418 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] [instance: b01533f3-9a31-4246-b3e7-f12f638e8e89] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138680) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 16 21:26:58.680979 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-547cce8e-74ab-4795-9117-e60e04cbd418 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 1.412081 Jul 16 21:26:58.681538 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1967/3943] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 21:26:57 2022] POST /compute/v2.1/servers => generated 380 bytes in 1414 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Jul 16 21:26:58.785460 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-4d89632f-4960-4192-8259-a2644018bc1b tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ListServerFiltersTestJSON-instance-644539345", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "451", "networks": [{"uuid": "87929b0c-a3bd-4f87-8248-ab83363a54b1"}]}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:26:58.877840 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.network.neutron [None req-4d89632f-4960-4192-8259-a2644018bc1b tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] validate_networks() for [('87929b0c-a3bd-4f87-8248-ab83363a54b1', None, None, None, None, None)] {{(pid=138681) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 16 21:26:58.902973 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-0c12cf6c-5e8b-4939-a782-9111c5d9a01a tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:26:58.910437 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-0c12cf6c-5e8b-4939-a782-9111c5d9a01a tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: a45c5bbd-ab66-426a-8e31-f471ce68606f] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:26:58.947706 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0c12cf6c-5e8b-4939-a782-9111c5d9a01a tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.013s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:58.950032 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0c12cf6c-5e8b-4939-a782-9111c5d9a01a tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:58.955969 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0c12cf6c-5e8b-4939-a782-9111c5d9a01a tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:58.956561 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0c12cf6c-5e8b-4939-a782-9111c5d9a01a tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:59.472436 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-0c12cf6c-5e8b-4939-a782-9111c5d9a01a tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '027dcc48e4fd4c36a5004096ad5e43f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dad823e240149edb30e592380249a13', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:26:59.483361 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-0c12cf6c-5e8b-4939-a782-9111c5d9a01a tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "GET /compute/v2.1/servers/a45c5bbd-ab66-426a-8e31-f471ce68606f" status: 200 len: 1308 microversion: 2.1 time: 0.592783 Jul 16 21:26:59.484014 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1968/3944] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:26:58 2022] GET /compute/v2.1/servers/a45c5bbd-ab66-426a-8e31-f471ce68606f => generated 1308 bytes in 594 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:26:59.512328 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-d81f83c7-9989-4e84-b4fd-3681b2a98cda tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:26:59.513313 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-d81f83c7-9989-4e84-b4fd-3681b2a98cda tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] [instance: c2e32bd9-9d80-427b-bb6d-6274fd48173a] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:26:59.520544 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d81f83c7-9989-4e84-b4fd-3681b2a98cda tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:59.521229 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d81f83c7-9989-4e84-b4fd-3681b2a98cda tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:59.522194 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d81f83c7-9989-4e84-b4fd-3681b2a98cda tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:59.522783 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d81f83c7-9989-4e84-b4fd-3681b2a98cda tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:59.805568 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-d81f83c7-9989-4e84-b4fd-3681b2a98cda tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcc9aa2a091c490599f9a9c41163e2eb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da56fc78f06142aba80ca4c6e3d60475', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:26:59.820227 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-d81f83c7-9989-4e84-b4fd-3681b2a98cda tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] 10.222.0.16 "GET /compute/v2.1/servers/c2e32bd9-9d80-427b-bb6d-6274fd48173a" status: 200 len: 1735 microversion: 2.1 time: 0.307689 Jul 16 21:26:59.822730 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1969/3945] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:26:59 2022] GET /compute/v2.1/servers/c2e32bd9-9d80-427b-bb6d-6274fd48173a => generated 1735 bytes in 315 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:26:59.866079 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-4d89632f-4960-4192-8259-a2644018bc1b tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Flavor limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 16 21:26:59.867676 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-4d89632f-4960-4192-8259-a2644018bc1b tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Image limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 16 21:26:59.868374 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-4d89632f-4960-4192-8259-a2644018bc1b tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Flavor pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 16 21:26:59.869033 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-4d89632f-4960-4192-8259-a2644018bc1b tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Image pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 16 21:26:59.870468 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-4d89632f-4960-4192-8259-a2644018bc1b tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 16 21:26:59.887804 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4d89632f-4960-4192-8259-a2644018bc1b tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:59.889041 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4d89632f-4960-4192-8259-a2644018bc1b tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:59.890037 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4d89632f-4960-4192-8259-a2644018bc1b tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:26:59.890807 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4d89632f-4960-4192-8259-a2644018bc1b tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:26:59.915988 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-4d89632f-4960-4192-8259-a2644018bc1b tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Getting quotas for project 3b214f5044c7445592e177835a5d18fc. Resources: {'ram', 'instances', 'cores'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:26:59.922503 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-4d89632f-4960-4192-8259-a2644018bc1b tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Getting quotas for user 029c76a0ba614b5880128c4309902c67 and project 3b214f5044c7445592e177835a5d18fc. Resources: {'ram', 'instances', 'cores'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:26:59.937198 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-4d89632f-4960-4192-8259-a2644018bc1b tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Going to run 1 instances... {{(pid=138681) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 16 21:26:59.940178 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-4d89632f-4960-4192-8259-a2644018bc1b tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] [instance: fbb7fab1-a4d5-48e2-9ace-fc65fad094ce] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138681) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 16 21:27:00.096409 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-4d89632f-4960-4192-8259-a2644018bc1b tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 1.314412 Jul 16 21:27:00.097868 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1977/3946] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 21:26:58 2022] POST /compute/v2.1/servers => generated 380 bytes in 1317 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 16 21:27:00.119395 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-ef85679c-49b6-465e-a164-2f2567442345 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:27:00.121405 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-ef85679c-49b6-465e-a164-2f2567442345 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] [instance: fbb7fab1-a4d5-48e2-9ace-fc65fad094ce] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:27:00.313416 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-ef85679c-49b6-465e-a164-2f2567442345 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '029c76a0ba614b5880128c4309902c67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3b214f5044c7445592e177835a5d18fc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:27:00.320993 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-ef85679c-49b6-465e-a164-2f2567442345 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] 10.222.0.16 "GET /compute/v2.1/servers/fbb7fab1-a4d5-48e2-9ace-fc65fad094ce" status: 200 len: 1220 microversion: 2.1 time: 0.214561 Jul 16 21:27:00.346197 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1970/3947] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:27:00 2022] GET /compute/v2.1/servers/fbb7fab1-a4d5-48e2-9ace-fc65fad094ce => generated 1220 bytes in 239 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:27:00.505438 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-b0945807-361c-4034-8c9c-6dda3604d7cb tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:27:00.506144 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-b0945807-361c-4034-8c9c-6dda3604d7cb tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: a45c5bbd-ab66-426a-8e31-f471ce68606f] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:27:00.548173 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b0945807-361c-4034-8c9c-6dda3604d7cb tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:00.554374 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b0945807-361c-4034-8c9c-6dda3604d7cb tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:00.556896 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b0945807-361c-4034-8c9c-6dda3604d7cb tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:00.559627 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b0945807-361c-4034-8c9c-6dda3604d7cb tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:00.863566 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-bcdc5c08-af14-4528-b9c6-f4a3090492cb tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:27:00.866721 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-bcdc5c08-af14-4528-b9c6-f4a3090492cb tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] [instance: c2e32bd9-9d80-427b-bb6d-6274fd48173a] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:27:00.909377 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-bcdc5c08-af14-4528-b9c6-f4a3090492cb tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:00.911453 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-bcdc5c08-af14-4528-b9c6-f4a3090492cb tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:00.913548 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-bcdc5c08-af14-4528-b9c6-f4a3090492cb tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:00.914010 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-bcdc5c08-af14-4528-b9c6-f4a3090492cb tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:01.370805 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-b0945807-361c-4034-8c9c-6dda3604d7cb tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '027dcc48e4fd4c36a5004096ad5e43f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dad823e240149edb30e592380249a13', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:27:01.402504 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-b0945807-361c-4034-8c9c-6dda3604d7cb tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "GET /compute/v2.1/servers/a45c5bbd-ab66-426a-8e31-f471ce68606f" status: 200 len: 1488 microversion: 2.1 time: 0.888663 Jul 16 21:27:01.402504 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1978/3948] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:27:00 2022] GET /compute/v2.1/servers/a45c5bbd-ab66-426a-8e31-f471ce68606f => generated 1488 bytes in 892 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:27:01.402504 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-50aebcce-a5f9-44e8-ac9a-57cdde3cb426 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:27:01.402504 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-50aebcce-a5f9-44e8-ac9a-57cdde3cb426 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] [instance: fbb7fab1-a4d5-48e2-9ace-fc65fad094ce] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:27:01.434571 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-bcdc5c08-af14-4528-b9c6-f4a3090492cb tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcc9aa2a091c490599f9a9c41163e2eb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da56fc78f06142aba80ca4c6e3d60475', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:27:01.473778 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-bcdc5c08-af14-4528-b9c6-f4a3090492cb tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] 10.222.0.16 "GET /compute/v2.1/servers/c2e32bd9-9d80-427b-bb6d-6274fd48173a" status: 200 len: 1735 microversion: 2.1 time: 0.630646 Jul 16 21:27:01.474415 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1971/3949] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:27:00 2022] GET /compute/v2.1/servers/c2e32bd9-9d80-427b-bb6d-6274fd48173a => generated 1735 bytes in 637 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:27:01.481678 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-f80207bd-aa4d-4c00-9010-47b00056585a tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Action: 'update', calling method: >, body: {"server": {"name": "tempest-\u00cd\u00f1st\u00e1\u00f1c\u00e9-1582271078"}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:27:01.489600 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-f80207bd-aa4d-4c00-9010-47b00056585a tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: a45c5bbd-ab66-426a-8e31-f471ce68606f] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:27:01.493354 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_api:SELECT=91,nova_api:INSERT=9 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:27:01.505487 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f80207bd-aa4d-4c00-9010-47b00056585a tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:01.508392 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-50aebcce-a5f9-44e8-ac9a-57cdde3cb426 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '029c76a0ba614b5880128c4309902c67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3b214f5044c7445592e177835a5d18fc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:27:01.510522 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f80207bd-aa4d-4c00-9010-47b00056585a tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:01.511439 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f80207bd-aa4d-4c00-9010-47b00056585a tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:01.511927 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f80207bd-aa4d-4c00-9010-47b00056585a tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:01.516084 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-50aebcce-a5f9-44e8-ac9a-57cdde3cb426 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] 10.222.0.16 "GET /compute/v2.1/servers/fbb7fab1-a4d5-48e2-9ace-fc65fad094ce" status: 200 len: 1220 microversion: 2.1 time: 0.123995 Jul 16 21:27:01.526105 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1979/3950] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:27:01 2022] GET /compute/v2.1/servers/fbb7fab1-a4d5-48e2-9ace-fc65fad094ce => generated 1220 bytes in 125 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:27:01.561223 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:27:01.717047 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-f80207bd-aa4d-4c00-9010-47b00056585a tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "PUT /compute/v2.1/servers/a45c5bbd-ab66-426a-8e31-f471ce68606f" status: 200 len: 1074 microversion: 2.1 time: 0.241060 Jul 16 21:27:01.718010 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1972/3951] 10.222.0.16 () {60 vars in 1255 bytes} [Sat Jul 16 21:27:01 2022] PUT /compute/v2.1/servers/a45c5bbd-ab66-426a-8e31-f471ce68606f => generated 1074 bytes in 243 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:27:01.734125 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-5204fca1-3d3e-4cc8-bab5-9d1ca8d8a3ed tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:27:01.734717 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-5204fca1-3d3e-4cc8-bab5-9d1ca8d8a3ed tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: a45c5bbd-ab66-426a-8e31-f471ce68606f] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:27:01.755096 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5204fca1-3d3e-4cc8-bab5-9d1ca8d8a3ed tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:01.755654 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5204fca1-3d3e-4cc8-bab5-9d1ca8d8a3ed tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:01.764935 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5204fca1-3d3e-4cc8-bab5-9d1ca8d8a3ed tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:01.766795 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5204fca1-3d3e-4cc8-bab5-9d1ca8d8a3ed tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:02.492960 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-329ba747-4c15-4933-9e17-2250b04a976b tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:27:02.493591 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-329ba747-4c15-4933-9e17-2250b04a976b tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] [instance: c2e32bd9-9d80-427b-bb6d-6274fd48173a] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:27:02.502868 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-329ba747-4c15-4933-9e17-2250b04a976b tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:02.503968 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-329ba747-4c15-4933-9e17-2250b04a976b tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:02.504794 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-329ba747-4c15-4933-9e17-2250b04a976b tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:02.505247 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-329ba747-4c15-4933-9e17-2250b04a976b tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:02.788930 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-5204fca1-3d3e-4cc8-bab5-9d1ca8d8a3ed tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '027dcc48e4fd4c36a5004096ad5e43f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dad823e240149edb30e592380249a13', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:27:02.799040 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-5204fca1-3d3e-4cc8-bab5-9d1ca8d8a3ed tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "GET /compute/v2.1/servers/a45c5bbd-ab66-426a-8e31-f471ce68606f" status: 200 len: 1500 microversion: 2.1 time: 1.069468 Jul 16 21:27:02.799908 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1980/3952] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:27:01 2022] GET /compute/v2.1/servers/a45c5bbd-ab66-426a-8e31-f471ce68606f => generated 1500 bytes in 1071 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:27:02.819175 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-7f5a0ba8-7719-4f5b-b71e-cd3ff66833f7 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:27:02.832599 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-7f5a0ba8-7719-4f5b-b71e-cd3ff66833f7 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] [instance: fbb7fab1-a4d5-48e2-9ace-fc65fad094ce] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:27:02.844309 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7f5a0ba8-7719-4f5b-b71e-cd3ff66833f7 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:02.845025 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7f5a0ba8-7719-4f5b-b71e-cd3ff66833f7 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:02.847620 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7f5a0ba8-7719-4f5b-b71e-cd3ff66833f7 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:02.848280 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7f5a0ba8-7719-4f5b-b71e-cd3ff66833f7 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:03.003152 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-7f5a0ba8-7719-4f5b-b71e-cd3ff66833f7 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '029c76a0ba614b5880128c4309902c67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3b214f5044c7445592e177835a5d18fc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:27:03.012119 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-7f5a0ba8-7719-4f5b-b71e-cd3ff66833f7 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] 10.222.0.16 "GET /compute/v2.1/servers/fbb7fab1-a4d5-48e2-9ace-fc65fad094ce" status: 200 len: 1272 microversion: 2.1 time: 0.210534 Jul 16 21:27:03.012677 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1981/3953] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:27:02 2022] GET /compute/v2.1/servers/fbb7fab1-a4d5-48e2-9ace-fc65fad094ce => generated 1272 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:27:03.027807 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-c43de7c6-f072-479a-b781-447c38288dc3 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:27:03.028672 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-c43de7c6-f072-479a-b781-447c38288dc3 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: a45c5bbd-ab66-426a-8e31-f471ce68606f] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:27:03.038211 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c43de7c6-f072-479a-b781-447c38288dc3 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:03.043880 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c43de7c6-f072-479a-b781-447c38288dc3 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:03.043880 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c43de7c6-f072-479a-b781-447c38288dc3 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:03.043880 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c43de7c6-f072-479a-b781-447c38288dc3 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:03.058311 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-329ba747-4c15-4933-9e17-2250b04a976b tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcc9aa2a091c490599f9a9c41163e2eb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da56fc78f06142aba80ca4c6e3d60475', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:27:03.072433 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-329ba747-4c15-4933-9e17-2250b04a976b tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] 10.222.0.16 "GET /compute/v2.1/servers/c2e32bd9-9d80-427b-bb6d-6274fd48173a" status: 200 len: 1715 microversion: 2.1 time: 0.584724 Jul 16 21:27:03.107421 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1973/3954] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:27:02 2022] GET /compute/v2.1/servers/c2e32bd9-9d80-427b-bb6d-6274fd48173a => generated 1715 bytes in 620 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:27:03.263265 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-293dc0b7-2b90-4ed9-a3ae-32cec4365e6d tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:27:03.263780 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-293dc0b7-2b90-4ed9-a3ae-32cec4365e6d tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] [instance: c2e32bd9-9d80-427b-bb6d-6274fd48173a] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:27:03.275109 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-293dc0b7-2b90-4ed9-a3ae-32cec4365e6d tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:03.276192 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-293dc0b7-2b90-4ed9-a3ae-32cec4365e6d tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:03.278596 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-293dc0b7-2b90-4ed9-a3ae-32cec4365e6d tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:03.279284 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-293dc0b7-2b90-4ed9-a3ae-32cec4365e6d tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:03.395534 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-293dc0b7-2b90-4ed9-a3ae-32cec4365e6d tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] HTTP exception thrown: Cannot 'detach_volume' instance c2e32bd9-9d80-427b-bb6d-6274fd48173a while it is in vm_state rescued Jul 16 21:27:03.397919 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-293dc0b7-2b90-4ed9-a3ae-32cec4365e6d tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Request method failure captured: Jul 16 21:27:03.397919 ncdv-master-1 devstack@n-api.service[138680]: request: DELETE /compute/v2.1/servers/c2e32bd9-9d80-427b-bb6d-6274fd48173a/os-volume_attachments/685a896a-62d9-4253-ae50-c46b14a0daf6 HTTP/1.1 Jul 16 21:27:03.397919 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:27:03.397919 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:27:03.397919 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:27:03.397919 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 0 Jul 16 21:27:03.397919 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:27:03.397919 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:27:03.397919 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:27:03.397919 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:27:03.397919 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0yxlBvXdj6P284bimRWVNOZAMmAVjIOasTJ1njzyaS12Nm5AKzVbxoKHi72FFVSaWe28lVKwWFtzjqUTy3NQWDNW-obFHfOCoFCCR4GNQ-_ruXI6NnMsKND6oz7nIdWgqUgB4Gac48-MGLdaneUoqvnHJrpsJjhWmKgGoKbdR8M7y0I Jul 16 21:27:03.397919 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:27:03.397919 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:27:03.397919 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:27:03.397919 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:27:03.397919 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:27:03.397919 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:27:03.397919 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: da56fc78f06142aba80ca4c6e3d60475 Jul 16 21:27:03.397919 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-439221844 Jul 16 21:27:03.397919 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:27:03.397919 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:27:03.397919 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/da56fc78f06142aba80ca4c6e3d60475"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/da56fc78f06142aba80ca4c6e3d60475"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_da56fc78f06142aba80ca4c6e3d60475"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/da56fc78f06142aba80ca4c6e3d60475"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:27:03.397919 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-ServerRescueNegativeTestJSON-439221844 Jul 16 21:27:03.397919 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: da56fc78f06142aba80ca4c6e3d60475 Jul 16 21:27:03.397919 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-439221844 Jul 16 21:27:03.397919 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-ServerRescueNegativeTestJSON-439221844-project Jul 16 21:27:03.397919 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:27:03.397919 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:27:03.397919 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: bcc9aa2a091c490599f9a9c41163e2eb Jul 16 21:27:03.397919 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-ServerRescueNegativeTestJSON-439221844-project Jul 16 21:27:03.397919 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 21:27:03.397919 ncdv-master-1 devstack@n-api.service[138680]: exception: Cannot 'detach_volume' instance c2e32bd9-9d80-427b-bb6d-6274fd48173a while it is in vm_state rescued Jul 16 21:27:03.397919 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'server_id': 'c2e32bd9-9d80-427b-bb6d-6274fd48173a', 'id': '685a896a-62d9-4253-ae50-c46b14a0daf6'} Jul 16 21:27:03.397919 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:27:03.397919 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:27:03.397919 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Jul 16 21:27:03.397919 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Jul 16 21:27:03.397919 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner Jul 16 21:27:03.397919 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Jul 16 21:27:03.397919 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 145, in inner Jul 16 21:27:03.397919 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.InstanceInvalidState( Jul 16 21:27:03.397919 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.InstanceInvalidState: Instance c2e32bd9-9d80-427b-bb6d-6274fd48173a in vm_state rescued. Cannot detach_volume while the instance is in this state. Jul 16 21:27:03.397919 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:27:03.397919 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:27:03.397919 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:27:03.397919 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:27:03.397919 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:27:03.397919 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:27:03.397919 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:27:03.397919 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:27:03.397919 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:27:03.397919 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:27:03.397919 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 573, in delete Jul 16 21:27:03.397919 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi common.raise_http_conflict_for_instance_invalid_state(state_error, Jul 16 21:27:03.397919 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 347, in raise_http_conflict_for_instance_invalid_state Jul 16 21:27:03.397919 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPConflict(explanation=msg) Jul 16 21:27:03.397919 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPConflict: Cannot 'detach_volume' instance c2e32bd9-9d80-427b-bb6d-6274fd48173a while it is in vm_state rescued Jul 16 21:27:03.397919 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:27:03.397919 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:27:03.397919 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:27:03.397919 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:27:03.401330 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:27:03.401330 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:27:03.401330 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:27:03.401330 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:27:03.401330 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Cannot 'detach_volume' instance c2e32bd9-9d80-427b-bb6d-6274fd48173a while it is in vm_state rescued Jul 16 21:27:03.401330 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:27:03.402974 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-293dc0b7-2b90-4ed9-a3ae-32cec4365e6d tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Returning 409 to user: Cannot 'detach_volume' instance c2e32bd9-9d80-427b-bb6d-6274fd48173a while it is in vm_state rescued {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:27:03.404649 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-293dc0b7-2b90-4ed9-a3ae-32cec4365e6d tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] 10.222.0.16 "DELETE /compute/v2.1/servers/c2e32bd9-9d80-427b-bb6d-6274fd48173a/os-volume_attachments/685a896a-62d9-4253-ae50-c46b14a0daf6" status: 409 len: 152 microversion: 2.1 time: 0.148976 Jul 16 21:27:03.406038 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1974/3955] 10.222.0.16 () {58 vars in 1415 bytes} [Sat Jul 16 21:27:03 2022] DELETE /compute/v2.1/servers/c2e32bd9-9d80-427b-bb6d-6274fd48173a/os-volume_attachments/685a896a-62d9-4253-ae50-c46b14a0daf6 => generated 152 bytes in 153 msecs (HTTP/1.1 409) 9 headers in 378 bytes (1 switches on core 0) Jul 16 21:27:03.417513 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-44263254-8181-4438-bd40-128d96a955c8 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Action: 'action', calling method: >, body: {"unrescue": {}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:27:03.418072 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-44263254-8181-4438-bd40-128d96a955c8 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] [instance: c2e32bd9-9d80-427b-bb6d-6274fd48173a] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:27:03.427862 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-44263254-8181-4438-bd40-128d96a955c8 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:03.428357 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-44263254-8181-4438-bd40-128d96a955c8 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:03.429390 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-44263254-8181-4438-bd40-128d96a955c8 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:03.429845 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-44263254-8181-4438-bd40-128d96a955c8 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:03.447327 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-c43de7c6-f072-479a-b781-447c38288dc3 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '027dcc48e4fd4c36a5004096ad5e43f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dad823e240149edb30e592380249a13', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:27:03.456367 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-c43de7c6-f072-479a-b781-447c38288dc3 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "GET /compute/v2.1/servers/a45c5bbd-ab66-426a-8e31-f471ce68606f" status: 200 len: 1500 microversion: 2.1 time: 0.442141 Jul 16 21:27:03.463111 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1982/3956] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:27:03 2022] GET /compute/v2.1/servers/a45c5bbd-ab66-426a-8e31-f471ce68606f => generated 1500 bytes in 443 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:27:03.478570 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-cddc362d-62b7-4d01-b4f7-01442ed51ebe tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Action: 'action', calling method: >, body: {"os-stop": {}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:27:03.486888 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cddc362d-62b7-4d01-b4f7-01442ed51ebe tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:03.488421 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cddc362d-62b7-4d01-b4f7-01442ed51ebe tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:03.523632 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-cddc362d-62b7-4d01-b4f7-01442ed51ebe tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: a45c5bbd-ab66-426a-8e31-f471ce68606f] Going to try to stop instance {{(pid=138681) force_stop /opt/stack/nova/nova/compute/api.py:2707}} Jul 16 21:27:03.818740 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-44263254-8181-4438-bd40-128d96a955c8 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] 10.222.0.16 "POST /compute/v2.1/servers/c2e32bd9-9d80-427b-bb6d-6274fd48173a/action" status: 202 len: 0 microversion: 2.1 time: 0.407469 Jul 16 21:27:03.824057 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1975/3957] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 21:27:03 2022] POST /compute/v2.1/servers/c2e32bd9-9d80-427b-bb6d-6274fd48173a/action => generated 0 bytes in 414 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 21:27:03.833356 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-f2981d23-cad0-44f3-ac5e-587e2e2a07c1 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:27:03.834294 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-f2981d23-cad0-44f3-ac5e-587e2e2a07c1 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] [instance: c2e32bd9-9d80-427b-bb6d-6274fd48173a] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:27:03.841068 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f2981d23-cad0-44f3-ac5e-587e2e2a07c1 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:03.841548 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f2981d23-cad0-44f3-ac5e-587e2e2a07c1 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:03.842607 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f2981d23-cad0-44f3-ac5e-587e2e2a07c1 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:03.843157 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f2981d23-cad0-44f3-ac5e-587e2e2a07c1 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:03.873432 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-cddc362d-62b7-4d01-b4f7-01442ed51ebe tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "POST /compute/v2.1/servers/a45c5bbd-ab66-426a-8e31-f471ce68606f/action" status: 202 len: 0 microversion: 2.1 time: 0.402131 Jul 16 21:27:03.874107 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1983/3958] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 21:27:03 2022] POST /compute/v2.1/servers/a45c5bbd-ab66-426a-8e31-f471ce68606f/action => generated 0 bytes in 404 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 21:27:03.885085 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-7ee04e53-c933-4a3f-9167-e968215bb47a tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:27:03.886074 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-7ee04e53-c933-4a3f-9167-e968215bb47a tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: a45c5bbd-ab66-426a-8e31-f471ce68606f] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:27:03.902978 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7ee04e53-c933-4a3f-9167-e968215bb47a tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:03.903759 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7ee04e53-c933-4a3f-9167-e968215bb47a tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:03.904768 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7ee04e53-c933-4a3f-9167-e968215bb47a tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:03.905558 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7ee04e53-c933-4a3f-9167-e968215bb47a tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:04.400139 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-f2981d23-cad0-44f3-ac5e-587e2e2a07c1 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcc9aa2a091c490599f9a9c41163e2eb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da56fc78f06142aba80ca4c6e3d60475', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:27:04.422984 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-7ee04e53-c933-4a3f-9167-e968215bb47a tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '027dcc48e4fd4c36a5004096ad5e43f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dad823e240149edb30e592380249a13', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:27:04.424702 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-f2981d23-cad0-44f3-ac5e-587e2e2a07c1 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] 10.222.0.16 "GET /compute/v2.1/servers/c2e32bd9-9d80-427b-bb6d-6274fd48173a" status: 200 len: 1723 microversion: 2.1 time: 0.599073 Jul 16 21:27:04.435058 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-7ee04e53-c933-4a3f-9167-e968215bb47a tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "GET /compute/v2.1/servers/a45c5bbd-ab66-426a-8e31-f471ce68606f" status: 200 len: 1510 microversion: 2.1 time: 0.556224 Jul 16 21:27:04.435858 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1984/3959] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:27:03 2022] GET /compute/v2.1/servers/a45c5bbd-ab66-426a-8e31-f471ce68606f => generated 1510 bytes in 558 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:27:04.441718 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1976/3960] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:27:03 2022] GET /compute/v2.1/servers/c2e32bd9-9d80-427b-bb6d-6274fd48173a => generated 1723 bytes in 617 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:27:04.449377 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-5afc20d1-0adf-435c-91de-df46fa97cb3b tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:27:04.450194 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-5afc20d1-0adf-435c-91de-df46fa97cb3b tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] [instance: fbb7fab1-a4d5-48e2-9ace-fc65fad094ce] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:27:04.466998 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5afc20d1-0adf-435c-91de-df46fa97cb3b tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:04.467258 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5afc20d1-0adf-435c-91de-df46fa97cb3b tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:04.467861 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5afc20d1-0adf-435c-91de-df46fa97cb3b tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:04.468877 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5afc20d1-0adf-435c-91de-df46fa97cb3b tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:04.651428 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-5afc20d1-0adf-435c-91de-df46fa97cb3b tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '029c76a0ba614b5880128c4309902c67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3b214f5044c7445592e177835a5d18fc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:27:04.661377 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-5afc20d1-0adf-435c-91de-df46fa97cb3b tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] 10.222.0.16 "GET /compute/v2.1/servers/fbb7fab1-a4d5-48e2-9ace-fc65fad094ce" status: 200 len: 1278 microversion: 2.1 time: 0.224269 Jul 16 21:27:04.662454 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1985/3961] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:27:04 2022] GET /compute/v2.1/servers/fbb7fab1-a4d5-48e2-9ace-fc65fad094ce => generated 1278 bytes in 226 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:27:05.453473 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-147bea8a-18b2-4ebb-9c1a-1da7787cf1c4 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:27:05.454100 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-1afc0150-84fa-4f15-be08-cb95c5accb43 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:27:05.454452 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-1afc0150-84fa-4f15-be08-cb95c5accb43 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] [instance: c2e32bd9-9d80-427b-bb6d-6274fd48173a] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:27:05.455398 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-147bea8a-18b2-4ebb-9c1a-1da7787cf1c4 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: a45c5bbd-ab66-426a-8e31-f471ce68606f] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:27:05.468193 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1afc0150-84fa-4f15-be08-cb95c5accb43 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:05.469556 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1afc0150-84fa-4f15-be08-cb95c5accb43 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:05.470732 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-147bea8a-18b2-4ebb-9c1a-1da7787cf1c4 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:05.471249 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-147bea8a-18b2-4ebb-9c1a-1da7787cf1c4 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:05.471421 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1afc0150-84fa-4f15-be08-cb95c5accb43 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:05.472691 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-147bea8a-18b2-4ebb-9c1a-1da7787cf1c4 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:05.473175 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1afc0150-84fa-4f15-be08-cb95c5accb43 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:05.473298 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-147bea8a-18b2-4ebb-9c1a-1da7787cf1c4 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:05.874142 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-147bea8a-18b2-4ebb-9c1a-1da7787cf1c4 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '027dcc48e4fd4c36a5004096ad5e43f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dad823e240149edb30e592380249a13', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:27:05.898900 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-147bea8a-18b2-4ebb-9c1a-1da7787cf1c4 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "GET /compute/v2.1/servers/a45c5bbd-ab66-426a-8e31-f471ce68606f" status: 200 len: 1510 microversion: 2.1 time: 0.448637 Jul 16 21:27:05.899692 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1986/3962] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:27:05 2022] GET /compute/v2.1/servers/a45c5bbd-ab66-426a-8e31-f471ce68606f => generated 1510 bytes in 451 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:27:05.912118 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-4c827bb5-cbc1-4872-bc3f-cb880f16fc09 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:27:05.914375 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-4c827bb5-cbc1-4872-bc3f-cb880f16fc09 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] [instance: fbb7fab1-a4d5-48e2-9ace-fc65fad094ce] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:27:05.923395 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4c827bb5-cbc1-4872-bc3f-cb880f16fc09 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:05.924083 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4c827bb5-cbc1-4872-bc3f-cb880f16fc09 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:05.925352 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4c827bb5-cbc1-4872-bc3f-cb880f16fc09 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:05.925999 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4c827bb5-cbc1-4872-bc3f-cb880f16fc09 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:05.944773 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-1afc0150-84fa-4f15-be08-cb95c5accb43 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcc9aa2a091c490599f9a9c41163e2eb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da56fc78f06142aba80ca4c6e3d60475', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:27:05.954800 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-1afc0150-84fa-4f15-be08-cb95c5accb43 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] 10.222.0.16 "GET /compute/v2.1/servers/c2e32bd9-9d80-427b-bb6d-6274fd48173a" status: 200 len: 1723 microversion: 2.1 time: 0.509526 Jul 16 21:27:05.966815 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1977/3963] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:27:05 2022] GET /compute/v2.1/servers/c2e32bd9-9d80-427b-bb6d-6274fd48173a => generated 1723 bytes in 522 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:27:05.967613 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:27:06.234092 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-4c827bb5-cbc1-4872-bc3f-cb880f16fc09 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '029c76a0ba614b5880128c4309902c67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3b214f5044c7445592e177835a5d18fc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:27:06.244260 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-4c827bb5-cbc1-4872-bc3f-cb880f16fc09 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] 10.222.0.16 "GET /compute/v2.1/servers/fbb7fab1-a4d5-48e2-9ace-fc65fad094ce" status: 200 len: 1320 microversion: 2.1 time: 0.342858 Jul 16 21:27:06.252847 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1987/3964] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:27:05 2022] GET /compute/v2.1/servers/fbb7fab1-a4d5-48e2-9ace-fc65fad094ce => generated 1320 bytes in 344 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:27:06.804789 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-a7dcb869-24e5-43b0-afe6-3754321f2d1b req-ef1cd2e7-8eb9-4a52-aa23-981e76bdfeaf service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "cf38328b-40fb-4537-b524-aaeb0a0abfb4", "tag": "f08f43fd-08c2-46ba-aec3-1900562b601f"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:27:06.828892 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-a7dcb869-24e5-43b0-afe6-3754321f2d1b req-ef1cd2e7-8eb9-4a52-aa23-981e76bdfeaf service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:06.830869 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-a7dcb869-24e5-43b0-afe6-3754321f2d1b req-ef1cd2e7-8eb9-4a52-aa23-981e76bdfeaf service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:06.894427 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-a7dcb869-24e5-43b0-afe6-3754321f2d1b req-ef1cd2e7-8eb9-4a52-aa23-981e76bdfeaf service nova] Creating event network-changed:f08f43fd-08c2-46ba-aec3-1900562b601f for instance cf38328b-40fb-4537-b524-aaeb0a0abfb4 on nch2-master-1 Jul 16 21:27:06.903275 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-a7dcb869-24e5-43b0-afe6-3754321f2d1b req-ef1cd2e7-8eb9-4a52-aa23-981e76bdfeaf service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.107872 Jul 16 21:27:06.905217 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1978/3965] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:27:06 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 111 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:27:06.924305 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-3999ceb5-9e1f-47f0-a528-2b99a190e376 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:27:06.927111 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-3999ceb5-9e1f-47f0-a528-2b99a190e376 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: a45c5bbd-ab66-426a-8e31-f471ce68606f] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:27:06.938467 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3999ceb5-9e1f-47f0-a528-2b99a190e376 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:06.939276 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3999ceb5-9e1f-47f0-a528-2b99a190e376 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:06.942427 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3999ceb5-9e1f-47f0-a528-2b99a190e376 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:06.943481 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3999ceb5-9e1f-47f0-a528-2b99a190e376 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:06.979931 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-233ab86b-fcd7-46c9-a7c2-4fac8c497f23 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:27:06.980736 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-233ab86b-fcd7-46c9-a7c2-4fac8c497f23 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] [instance: c2e32bd9-9d80-427b-bb6d-6274fd48173a] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:27:06.996867 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-233ab86b-fcd7-46c9-a7c2-4fac8c497f23 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:06.997327 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-233ab86b-fcd7-46c9-a7c2-4fac8c497f23 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:07.002529 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-233ab86b-fcd7-46c9-a7c2-4fac8c497f23 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:07.003237 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-233ab86b-fcd7-46c9-a7c2-4fac8c497f23 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:07.522325 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-3999ceb5-9e1f-47f0-a528-2b99a190e376 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '027dcc48e4fd4c36a5004096ad5e43f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dad823e240149edb30e592380249a13', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:27:07.539151 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-3999ceb5-9e1f-47f0-a528-2b99a190e376 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "GET /compute/v2.1/servers/a45c5bbd-ab66-426a-8e31-f471ce68606f" status: 200 len: 1510 microversion: 2.1 time: 0.619244 Jul 16 21:27:07.539151 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1988/3966] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:27:06 2022] GET /compute/v2.1/servers/a45c5bbd-ab66-426a-8e31-f471ce68606f => generated 1510 bytes in 625 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:27:07.551761 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-fcf5624d-d2b4-4b28-bbbb-1c871f0aee4e tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:27:07.552661 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-fcf5624d-d2b4-4b28-bbbb-1c871f0aee4e tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] [instance: fbb7fab1-a4d5-48e2-9ace-fc65fad094ce] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:27:07.560503 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fcf5624d-d2b4-4b28-bbbb-1c871f0aee4e tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:07.561265 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fcf5624d-d2b4-4b28-bbbb-1c871f0aee4e tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:07.562526 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fcf5624d-d2b4-4b28-bbbb-1c871f0aee4e tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:07.563287 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fcf5624d-d2b4-4b28-bbbb-1c871f0aee4e tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:07.636065 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-233ab86b-fcd7-46c9-a7c2-4fac8c497f23 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcc9aa2a091c490599f9a9c41163e2eb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da56fc78f06142aba80ca4c6e3d60475', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:27:07.644892 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-233ab86b-fcd7-46c9-a7c2-4fac8c497f23 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] 10.222.0.16 "GET /compute/v2.1/servers/c2e32bd9-9d80-427b-bb6d-6274fd48173a" status: 200 len: 1723 microversion: 2.1 time: 0.674972 Jul 16 21:27:07.649204 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1979/3967] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:27:06 2022] GET /compute/v2.1/servers/c2e32bd9-9d80-427b-bb6d-6274fd48173a => generated 1723 bytes in 677 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:27:07.788422 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-4b0f41ac-4d66-4dfe-b605-79bb4f68b50e req-94c229cc-74b2-4291-8f11-9cedd57a474a service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "fbb7fab1-a4d5-48e2-9ace-fc65fad094ce", "tag": "2ae39666-a3c4-4738-a6cb-db9cc84fdcd6"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:27:07.807110 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-4b0f41ac-4d66-4dfe-b605-79bb4f68b50e req-94c229cc-74b2-4291-8f11-9cedd57a474a service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:07.807719 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-4b0f41ac-4d66-4dfe-b605-79bb4f68b50e req-94c229cc-74b2-4291-8f11-9cedd57a474a service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:07.824367 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-4b0f41ac-4d66-4dfe-b605-79bb4f68b50e req-94c229cc-74b2-4291-8f11-9cedd57a474a service nova] Creating event network-changed:2ae39666-a3c4-4738-a6cb-db9cc84fdcd6 for instance fbb7fab1-a4d5-48e2-9ace-fc65fad094ce on nch2-master-1 Jul 16 21:27:07.832057 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-4b0f41ac-4d66-4dfe-b605-79bb4f68b50e req-94c229cc-74b2-4291-8f11-9cedd57a474a service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.049031 Jul 16 21:27:07.834275 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1980/3968] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:27:07 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 16 21:27:07.912665 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-fcf5624d-d2b4-4b28-bbbb-1c871f0aee4e tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '029c76a0ba614b5880128c4309902c67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3b214f5044c7445592e177835a5d18fc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:27:07.920780 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-fcf5624d-d2b4-4b28-bbbb-1c871f0aee4e tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] 10.222.0.16 "GET /compute/v2.1/servers/fbb7fab1-a4d5-48e2-9ace-fc65fad094ce" status: 200 len: 1320 microversion: 2.1 time: 0.382155 Jul 16 21:27:07.927547 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1989/3969] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:27:07 2022] GET /compute/v2.1/servers/fbb7fab1-a4d5-48e2-9ace-fc65fad094ce => generated 1320 bytes in 389 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:27:08.556406 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-954d8500-1810-4fe8-922c-22060d1fac43 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:27:08.556406 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-954d8500-1810-4fe8-922c-22060d1fac43 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: a45c5bbd-ab66-426a-8e31-f471ce68606f] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:27:08.558208 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell0:SELECT=4 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:27:08.575637 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-954d8500-1810-4fe8-922c-22060d1fac43 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:08.576306 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-954d8500-1810-4fe8-922c-22060d1fac43 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:08.577429 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-954d8500-1810-4fe8-922c-22060d1fac43 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:08.578006 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-954d8500-1810-4fe8-922c-22060d1fac43 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:08.662815 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-e8b7c0ef-991b-4bc8-a5f3-047a74388a01 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:27:08.663467 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-e8b7c0ef-991b-4bc8-a5f3-047a74388a01 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] [instance: c2e32bd9-9d80-427b-bb6d-6274fd48173a] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:27:08.678822 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e8b7c0ef-991b-4bc8-a5f3-047a74388a01 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:08.679589 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e8b7c0ef-991b-4bc8-a5f3-047a74388a01 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:08.680621 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e8b7c0ef-991b-4bc8-a5f3-047a74388a01 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:08.681212 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e8b7c0ef-991b-4bc8-a5f3-047a74388a01 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:08.858447 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-954d8500-1810-4fe8-922c-22060d1fac43 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '027dcc48e4fd4c36a5004096ad5e43f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dad823e240149edb30e592380249a13', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:27:08.878533 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-954d8500-1810-4fe8-922c-22060d1fac43 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "GET /compute/v2.1/servers/a45c5bbd-ab66-426a-8e31-f471ce68606f" status: 200 len: 1510 microversion: 2.1 time: 0.328256 Jul 16 21:27:08.879485 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1981/3970] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:27:08 2022] GET /compute/v2.1/servers/a45c5bbd-ab66-426a-8e31-f471ce68606f => generated 1510 bytes in 330 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:27:08.899977 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-e8b7c0ef-991b-4bc8-a5f3-047a74388a01 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcc9aa2a091c490599f9a9c41163e2eb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da56fc78f06142aba80ca4c6e3d60475', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:27:08.908122 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-e8b7c0ef-991b-4bc8-a5f3-047a74388a01 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] 10.222.0.16 "GET /compute/v2.1/servers/c2e32bd9-9d80-427b-bb6d-6274fd48173a" status: 200 len: 1729 microversion: 2.1 time: 0.249936 Jul 16 21:27:08.908708 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1990/3971] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:27:08 2022] GET /compute/v2.1/servers/c2e32bd9-9d80-427b-bb6d-6274fd48173a => generated 1729 bytes in 251 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:27:08.923828 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-18a22b22-232a-432b-9bfa-dfef820cb4fc req-cb5c89e3-22f6-4fdb-9098-52052f946ee3 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "b01533f3-9a31-4246-b3e7-f12f638e8e89", "tag": "3ce77aec-ed61-42cc-8170-868f6c10b522"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:27:08.925841 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-41df5497-be15-4717-a613-8bc8b06f81bf tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:27:08.926440 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-41df5497-be15-4717-a613-8bc8b06f81bf tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] [instance: c2e32bd9-9d80-427b-bb6d-6274fd48173a] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:27:08.933559 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-18a22b22-232a-432b-9bfa-dfef820cb4fc req-cb5c89e3-22f6-4fdb-9098-52052f946ee3 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:08.933976 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-41df5497-be15-4717-a613-8bc8b06f81bf tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:08.934442 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-18a22b22-232a-432b-9bfa-dfef820cb4fc req-cb5c89e3-22f6-4fdb-9098-52052f946ee3 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:08.934696 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-41df5497-be15-4717-a613-8bc8b06f81bf tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:08.935694 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-41df5497-be15-4717-a613-8bc8b06f81bf tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:08.939859 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-41df5497-be15-4717-a613-8bc8b06f81bf tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:08.948069 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-18a22b22-232a-432b-9bfa-dfef820cb4fc req-cb5c89e3-22f6-4fdb-9098-52052f946ee3 service nova] Creating event network-changed:3ce77aec-ed61-42cc-8170-868f6c10b522 for instance b01533f3-9a31-4246-b3e7-f12f638e8e89 on nch2-master-1 Jul 16 21:27:08.953845 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-18a22b22-232a-432b-9bfa-dfef820cb4fc req-cb5c89e3-22f6-4fdb-9098-52052f946ee3 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.034265 Jul 16 21:27:08.955413 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1982/3972] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:27:08 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:27:08.962823 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-4b3cdce7-14bc-434c-9bdb-d32227445de0 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:27:08.963202 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-4b3cdce7-14bc-434c-9bdb-d32227445de0 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] [instance: fbb7fab1-a4d5-48e2-9ace-fc65fad094ce] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:27:08.972390 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4b3cdce7-14bc-434c-9bdb-d32227445de0 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:08.972390 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4b3cdce7-14bc-434c-9bdb-d32227445de0 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:08.972390 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4b3cdce7-14bc-434c-9bdb-d32227445de0 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:08.972390 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4b3cdce7-14bc-434c-9bdb-d32227445de0 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:09.068431 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-41df5497-be15-4717-a613-8bc8b06f81bf tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] 10.222.0.16 "GET /compute/v2.1/servers/c2e32bd9-9d80-427b-bb6d-6274fd48173a/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.147021 Jul 16 21:27:09.069038 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1991/3973] 10.222.0.16 () {58 vars in 1274 bytes} [Sat Jul 16 21:27:08 2022] GET /compute/v2.1/servers/c2e32bd9-9d80-427b-bb6d-6274fd48173a/os-interface => generated 284 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:27:09.159386 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-4b3cdce7-14bc-434c-9bdb-d32227445de0 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '029c76a0ba614b5880128c4309902c67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3b214f5044c7445592e177835a5d18fc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:27:09.169924 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-4b3cdce7-14bc-434c-9bdb-d32227445de0 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] 10.222.0.16 "GET /compute/v2.1/servers/fbb7fab1-a4d5-48e2-9ace-fc65fad094ce" status: 200 len: 1320 microversion: 2.1 time: 0.212552 Jul 16 21:27:09.171570 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1983/3974] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:27:08 2022] GET /compute/v2.1/servers/fbb7fab1-a4d5-48e2-9ace-fc65fad094ce => generated 1320 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:27:09.897048 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-3ce13ffb-c679-4bf9-aaa3-7d32d620ead9 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:27:09.898330 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-3ce13ffb-c679-4bf9-aaa3-7d32d620ead9 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: a45c5bbd-ab66-426a-8e31-f471ce68606f] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:27:09.901512 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell0:SELECT=2 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:27:09.905170 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3ce13ffb-c679-4bf9-aaa3-7d32d620ead9 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:09.905633 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3ce13ffb-c679-4bf9-aaa3-7d32d620ead9 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:09.906419 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3ce13ffb-c679-4bf9-aaa3-7d32d620ead9 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:09.906890 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3ce13ffb-c679-4bf9-aaa3-7d32d620ead9 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:10.191325 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-353cbe64-65a3-43b2-9cfc-8a26679e712a tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:27:10.192128 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-353cbe64-65a3-43b2-9cfc-8a26679e712a tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] [instance: fbb7fab1-a4d5-48e2-9ace-fc65fad094ce] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:27:10.202002 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-353cbe64-65a3-43b2-9cfc-8a26679e712a tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:10.202718 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-353cbe64-65a3-43b2-9cfc-8a26679e712a tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:10.204535 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-353cbe64-65a3-43b2-9cfc-8a26679e712a tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:10.205224 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-353cbe64-65a3-43b2-9cfc-8a26679e712a tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:10.320146 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell0:SELECT=14 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:27:10.510765 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-3ce13ffb-c679-4bf9-aaa3-7d32d620ead9 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '027dcc48e4fd4c36a5004096ad5e43f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dad823e240149edb30e592380249a13', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:27:10.551284 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-3ce13ffb-c679-4bf9-aaa3-7d32d620ead9 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "GET /compute/v2.1/servers/a45c5bbd-ab66-426a-8e31-f471ce68606f" status: 200 len: 1510 microversion: 2.1 time: 0.659208 Jul 16 21:27:10.554143 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1992/3975] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:27:09 2022] GET /compute/v2.1/servers/a45c5bbd-ab66-426a-8e31-f471ce68606f => generated 1510 bytes in 663 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:27:10.566936 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-b774fb70-f5c7-460c-85d8-97045058d6ed req-3798fdb0-d32c-441f-9980-d3bba2ce5a78 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "c2e32bd9-9d80-427b-bb6d-6274fd48173a", "tag": "347e5d31-8efd-4cb0-82eb-3b8a6e86e0cd"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:27:10.600586 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-b774fb70-f5c7-460c-85d8-97045058d6ed req-3798fdb0-d32c-441f-9980-d3bba2ce5a78 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:10.605349 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-b774fb70-f5c7-460c-85d8-97045058d6ed req-3798fdb0-d32c-441f-9980-d3bba2ce5a78 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:10.652777 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-b774fb70-f5c7-460c-85d8-97045058d6ed req-3798fdb0-d32c-441f-9980-d3bba2ce5a78 service nova] Creating event network-changed:347e5d31-8efd-4cb0-82eb-3b8a6e86e0cd for instance c2e32bd9-9d80-427b-bb6d-6274fd48173a on nch1-master-1 Jul 16 21:27:10.671887 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-b774fb70-f5c7-460c-85d8-97045058d6ed req-3798fdb0-d32c-441f-9980-d3bba2ce5a78 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.109527 Jul 16 21:27:10.678877 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-353cbe64-65a3-43b2-9cfc-8a26679e712a tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '029c76a0ba614b5880128c4309902c67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3b214f5044c7445592e177835a5d18fc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:27:10.693074 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1993/3976] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:27:10 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 134 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 16 21:27:10.702142 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-353cbe64-65a3-43b2-9cfc-8a26679e712a tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] 10.222.0.16 "GET /compute/v2.1/servers/fbb7fab1-a4d5-48e2-9ace-fc65fad094ce" status: 200 len: 1320 microversion: 2.1 time: 0.514855 Jul 16 21:27:10.704893 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1984/3977] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:27:10 2022] GET /compute/v2.1/servers/fbb7fab1-a4d5-48e2-9ace-fc65fad094ce => generated 1320 bytes in 518 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:27:11.569771 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-a65d979b-a6ea-471a-a8ab-2341b74bbe31 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:27:11.571905 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-a65d979b-a6ea-471a-a8ab-2341b74bbe31 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: a45c5bbd-ab66-426a-8e31-f471ce68606f] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:27:11.578685 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell0:SELECT=8 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:27:11.584478 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a65d979b-a6ea-471a-a8ab-2341b74bbe31 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:11.585282 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a65d979b-a6ea-471a-a8ab-2341b74bbe31 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:11.586233 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a65d979b-a6ea-471a-a8ab-2341b74bbe31 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:11.586891 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a65d979b-a6ea-471a-a8ab-2341b74bbe31 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:11.721362 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-a3799cd9-2ba4-40b6-9482-16f06f6df71c tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:27:11.722632 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-a3799cd9-2ba4-40b6-9482-16f06f6df71c tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] [instance: fbb7fab1-a4d5-48e2-9ace-fc65fad094ce] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:27:11.732333 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a3799cd9-2ba4-40b6-9482-16f06f6df71c tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:11.733024 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a3799cd9-2ba4-40b6-9482-16f06f6df71c tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:11.734026 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a3799cd9-2ba4-40b6-9482-16f06f6df71c tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:11.734803 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a3799cd9-2ba4-40b6-9482-16f06f6df71c tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:12.116891 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-a65d979b-a6ea-471a-a8ab-2341b74bbe31 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '027dcc48e4fd4c36a5004096ad5e43f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dad823e240149edb30e592380249a13', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:27:12.132243 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-a65d979b-a6ea-471a-a8ab-2341b74bbe31 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "GET /compute/v2.1/servers/a45c5bbd-ab66-426a-8e31-f471ce68606f" status: 200 len: 1510 microversion: 2.1 time: 0.565313 Jul 16 21:27:12.132243 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1994/3978] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:27:11 2022] GET /compute/v2.1/servers/a45c5bbd-ab66-426a-8e31-f471ce68606f => generated 1510 bytes in 568 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:27:12.136896 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-a3799cd9-2ba4-40b6-9482-16f06f6df71c tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '029c76a0ba614b5880128c4309902c67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3b214f5044c7445592e177835a5d18fc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:27:12.147696 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-a3799cd9-2ba4-40b6-9482-16f06f6df71c tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] 10.222.0.16 "GET /compute/v2.1/servers/fbb7fab1-a4d5-48e2-9ace-fc65fad094ce" status: 200 len: 1320 microversion: 2.1 time: 0.430471 Jul 16 21:27:12.158447 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1985/3979] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:27:11 2022] GET /compute/v2.1/servers/fbb7fab1-a4d5-48e2-9ace-fc65fad094ce => generated 1320 bytes in 436 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:27:12.709449 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-a0ac98db-85b9-4436-ac2d-431b68aad7a4 req-9ac4ef9d-988e-4109-b56f-fb1ee8e87dcc service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "c2e32bd9-9d80-427b-bb6d-6274fd48173a", "tag": "347e5d31-8efd-4cb0-82eb-3b8a6e86e0cd"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:27:12.722868 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-a0ac98db-85b9-4436-ac2d-431b68aad7a4 req-9ac4ef9d-988e-4109-b56f-fb1ee8e87dcc service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:12.723556 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-a0ac98db-85b9-4436-ac2d-431b68aad7a4 req-9ac4ef9d-988e-4109-b56f-fb1ee8e87dcc service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:12.741296 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-a0ac98db-85b9-4436-ac2d-431b68aad7a4 req-9ac4ef9d-988e-4109-b56f-fb1ee8e87dcc service nova] Creating event network-changed:347e5d31-8efd-4cb0-82eb-3b8a6e86e0cd for instance c2e32bd9-9d80-427b-bb6d-6274fd48173a on nch1-master-1 Jul 16 21:27:12.746390 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-a0ac98db-85b9-4436-ac2d-431b68aad7a4 req-9ac4ef9d-988e-4109-b56f-fb1ee8e87dcc service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.041515 Jul 16 21:27:12.747247 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1995/3980] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:27:12 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:27:13.154964 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-6688aaff-b160-4085-bfc1-e2a14a8da8f6 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:27:13.157352 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-6688aaff-b160-4085-bfc1-e2a14a8da8f6 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: a45c5bbd-ab66-426a-8e31-f471ce68606f] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:27:13.168390 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-774adade-7087-47d8-8d75-7a91e7b5803e tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:27:13.169252 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6688aaff-b160-4085-bfc1-e2a14a8da8f6 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:13.169394 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-774adade-7087-47d8-8d75-7a91e7b5803e tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] [instance: fbb7fab1-a4d5-48e2-9ace-fc65fad094ce] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:27:13.170191 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6688aaff-b160-4085-bfc1-e2a14a8da8f6 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:13.171719 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6688aaff-b160-4085-bfc1-e2a14a8da8f6 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:13.172478 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6688aaff-b160-4085-bfc1-e2a14a8da8f6 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:13.189833 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-774adade-7087-47d8-8d75-7a91e7b5803e tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:13.190565 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-774adade-7087-47d8-8d75-7a91e7b5803e tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:13.191465 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-774adade-7087-47d8-8d75-7a91e7b5803e tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:13.192308 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-774adade-7087-47d8-8d75-7a91e7b5803e tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:13.838106 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-774adade-7087-47d8-8d75-7a91e7b5803e tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '029c76a0ba614b5880128c4309902c67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3b214f5044c7445592e177835a5d18fc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:27:13.844261 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-6688aaff-b160-4085-bfc1-e2a14a8da8f6 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '027dcc48e4fd4c36a5004096ad5e43f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dad823e240149edb30e592380249a13', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:27:13.849578 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-774adade-7087-47d8-8d75-7a91e7b5803e tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] 10.222.0.16 "GET /compute/v2.1/servers/fbb7fab1-a4d5-48e2-9ace-fc65fad094ce" status: 200 len: 1320 microversion: 2.1 time: 0.686612 Jul 16 21:27:13.850529 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1996/3981] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:27:13 2022] GET /compute/v2.1/servers/fbb7fab1-a4d5-48e2-9ace-fc65fad094ce => generated 1320 bytes in 689 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:27:13.859710 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-6688aaff-b160-4085-bfc1-e2a14a8da8f6 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "GET /compute/v2.1/servers/a45c5bbd-ab66-426a-8e31-f471ce68606f" status: 200 len: 1510 microversion: 2.1 time: 0.715176 Jul 16 21:27:13.861806 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1986/3982] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:27:13 2022] GET /compute/v2.1/servers/a45c5bbd-ab66-426a-8e31-f471ce68606f => generated 1510 bytes in 719 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:27:13.872074 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-b3f9d2d3-a555-4c40-a2fd-d3c24d19f299 req-eda09858-891c-41d4-887e-ae6150f3d0d2 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "cf38328b-40fb-4537-b524-aaeb0a0abfb4", "name": "network-vif-plugged", "status": "completed", "tag": "f08f43fd-08c2-46ba-aec3-1900562b601f"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:27:13.879096 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-b3f9d2d3-a555-4c40-a2fd-d3c24d19f299 req-eda09858-891c-41d4-887e-ae6150f3d0d2 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:13.879730 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-b3f9d2d3-a555-4c40-a2fd-d3c24d19f299 req-eda09858-891c-41d4-887e-ae6150f3d0d2 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:13.892697 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-b3f9d2d3-a555-4c40-a2fd-d3c24d19f299 req-eda09858-891c-41d4-887e-ae6150f3d0d2 service nova] Creating event network-vif-plugged:f08f43fd-08c2-46ba-aec3-1900562b601f for instance cf38328b-40fb-4537-b524-aaeb0a0abfb4 on nch2-master-1 Jul 16 21:27:13.900156 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-b3f9d2d3-a555-4c40-a2fd-d3c24d19f299 req-eda09858-891c-41d4-887e-ae6150f3d0d2 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.048187 Jul 16 21:27:13.901604 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1997/3983] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:27:13 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:27:14.866019 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-0432f875-ef01-432f-8d99-e972f190ca92 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:27:14.866709 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-0432f875-ef01-432f-8d99-e972f190ca92 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] [instance: fbb7fab1-a4d5-48e2-9ace-fc65fad094ce] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:27:14.874074 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0432f875-ef01-432f-8d99-e972f190ca92 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:14.874381 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0432f875-ef01-432f-8d99-e972f190ca92 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:14.883987 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-ff52ec71-450d-469e-bf98-0e5ae11f3aa4 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:27:14.885307 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-ff52ec71-450d-469e-bf98-0e5ae11f3aa4 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: a45c5bbd-ab66-426a-8e31-f471ce68606f] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:27:14.887659 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0432f875-ef01-432f-8d99-e972f190ca92 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:14.888618 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0432f875-ef01-432f-8d99-e972f190ca92 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.013s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:14.898956 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ff52ec71-450d-469e-bf98-0e5ae11f3aa4 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:14.899441 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ff52ec71-450d-469e-bf98-0e5ae11f3aa4 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:14.900219 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ff52ec71-450d-469e-bf98-0e5ae11f3aa4 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:14.913323 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ff52ec71-450d-469e-bf98-0e5ae11f3aa4 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:15.238178 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-ff52ec71-450d-469e-bf98-0e5ae11f3aa4 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '027dcc48e4fd4c36a5004096ad5e43f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dad823e240149edb30e592380249a13', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:27:15.243793 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-0432f875-ef01-432f-8d99-e972f190ca92 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '029c76a0ba614b5880128c4309902c67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3b214f5044c7445592e177835a5d18fc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:27:15.248592 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-ff52ec71-450d-469e-bf98-0e5ae11f3aa4 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "GET /compute/v2.1/servers/a45c5bbd-ab66-426a-8e31-f471ce68606f" status: 200 len: 1510 microversion: 2.1 time: 0.371778 Jul 16 21:27:15.249547 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1998/3984] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:27:14 2022] GET /compute/v2.1/servers/a45c5bbd-ab66-426a-8e31-f471ce68606f => generated 1510 bytes in 374 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:27:15.254911 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-0432f875-ef01-432f-8d99-e972f190ca92 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] 10.222.0.16 "GET /compute/v2.1/servers/fbb7fab1-a4d5-48e2-9ace-fc65fad094ce" status: 200 len: 1320 microversion: 2.1 time: 0.392875 Jul 16 21:27:15.263189 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1987/3985] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:27:14 2022] GET /compute/v2.1/servers/fbb7fab1-a4d5-48e2-9ace-fc65fad094ce => generated 1320 bytes in 394 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:27:16.248830 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-73d2d420-0b85-4727-bf19-5623c6cdde3b req-b99db019-f55f-4aa1-abcf-b3191664c265 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "b01533f3-9a31-4246-b3e7-f12f638e8e89", "name": "network-vif-plugged", "status": "completed", "tag": "3ce77aec-ed61-42cc-8170-868f6c10b522"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:27:16.265856 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-73d2d420-0b85-4727-bf19-5623c6cdde3b req-b99db019-f55f-4aa1-abcf-b3191664c265 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:16.266474 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-73d2d420-0b85-4727-bf19-5623c6cdde3b req-b99db019-f55f-4aa1-abcf-b3191664c265 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:16.274597 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-d85bdf07-c3b4-4c10-b5b3-d56123979947 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:27:16.275838 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-d85bdf07-c3b4-4c10-b5b3-d56123979947 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: a45c5bbd-ab66-426a-8e31-f471ce68606f] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:27:16.292995 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d85bdf07-c3b4-4c10-b5b3-d56123979947 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:16.293696 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d85bdf07-c3b4-4c10-b5b3-d56123979947 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:16.294496 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d85bdf07-c3b4-4c10-b5b3-d56123979947 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:16.297817 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d85bdf07-c3b4-4c10-b5b3-d56123979947 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:16.308051 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-73d2d420-0b85-4727-bf19-5623c6cdde3b req-b99db019-f55f-4aa1-abcf-b3191664c265 service nova] Creating event network-vif-plugged:3ce77aec-ed61-42cc-8170-868f6c10b522 for instance b01533f3-9a31-4246-b3e7-f12f638e8e89 on nch2-master-1 Jul 16 21:27:16.316216 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-73d2d420-0b85-4727-bf19-5623c6cdde3b req-b99db019-f55f-4aa1-abcf-b3191664c265 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.070492 Jul 16 21:27:16.318940 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 1999/3986] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:27:16 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 74 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:27:16.328646 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-a04680b6-92a5-43fe-a9cc-05959890762f tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:27:16.330519 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-a04680b6-92a5-43fe-a9cc-05959890762f tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] [instance: fbb7fab1-a4d5-48e2-9ace-fc65fad094ce] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:27:16.342115 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a04680b6-92a5-43fe-a9cc-05959890762f tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:16.342869 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a04680b6-92a5-43fe-a9cc-05959890762f tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:16.343926 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a04680b6-92a5-43fe-a9cc-05959890762f tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:16.344630 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a04680b6-92a5-43fe-a9cc-05959890762f tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:17.212870 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-a04680b6-92a5-43fe-a9cc-05959890762f tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '029c76a0ba614b5880128c4309902c67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3b214f5044c7445592e177835a5d18fc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:27:17.220928 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-a04680b6-92a5-43fe-a9cc-05959890762f tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] 10.222.0.16 "GET /compute/v2.1/servers/fbb7fab1-a4d5-48e2-9ace-fc65fad094ce" status: 200 len: 1320 microversion: 2.1 time: 0.900256 Jul 16 21:27:17.221174 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2000/3987] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:27:16 2022] GET /compute/v2.1/servers/fbb7fab1-a4d5-48e2-9ace-fc65fad094ce => generated 1320 bytes in 902 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:27:17.225257 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-d85bdf07-c3b4-4c10-b5b3-d56123979947 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '027dcc48e4fd4c36a5004096ad5e43f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dad823e240149edb30e592380249a13', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:27:17.235472 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-d85bdf07-c3b4-4c10-b5b3-d56123979947 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "GET /compute/v2.1/servers/a45c5bbd-ab66-426a-8e31-f471ce68606f" status: 200 len: 1510 microversion: 2.1 time: 0.971927 Jul 16 21:27:17.236402 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1988/3988] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:27:16 2022] GET /compute/v2.1/servers/a45c5bbd-ab66-426a-8e31-f471ce68606f => generated 1510 bytes in 974 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:27:18.160888 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-3cab0e7f-2b34-4f7b-b098-1b9a95589597 tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-InstanceActionsV221TestJSON-server-1087581754", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452", "networks": [{"uuid": "5fcdc232-7546-4c86-acd1-c9db59279307"}]}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:27:18.217423 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.network.neutron [None req-3cab0e7f-2b34-4f7b-b098-1b9a95589597 tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] validate_networks() for [('5fcdc232-7546-4c86-acd1-c9db59279307', None, None, None, None, None)] {{(pid=138681) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 16 21:27:18.243203 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-b902d149-2d05-4bed-834d-167e3683028f tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:27:18.244497 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-b902d149-2d05-4bed-834d-167e3683028f tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] [instance: fbb7fab1-a4d5-48e2-9ace-fc65fad094ce] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:27:18.271748 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b902d149-2d05-4bed-834d-167e3683028f tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:18.275065 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b902d149-2d05-4bed-834d-167e3683028f tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:18.276039 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b902d149-2d05-4bed-834d-167e3683028f tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:18.276793 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b902d149-2d05-4bed-834d-167e3683028f tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:18.534556 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-b902d149-2d05-4bed-834d-167e3683028f tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '029c76a0ba614b5880128c4309902c67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3b214f5044c7445592e177835a5d18fc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:27:18.536944 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:27:18.543472 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-b902d149-2d05-4bed-834d-167e3683028f tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] 10.222.0.16 "GET /compute/v2.1/servers/fbb7fab1-a4d5-48e2-9ace-fc65fad094ce" status: 200 len: 1320 microversion: 2.1 time: 0.309628 Jul 16 21:27:18.544422 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1989/3989] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:27:18 2022] GET /compute/v2.1/servers/fbb7fab1-a4d5-48e2-9ace-fc65fad094ce => generated 1320 bytes in 311 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:27:18.550325 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-d8803bce-7296-4b40-a9eb-8d3fdc9b5e67 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:27:18.552632 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-d8803bce-7296-4b40-a9eb-8d3fdc9b5e67 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: a45c5bbd-ab66-426a-8e31-f471ce68606f] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:27:18.560881 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d8803bce-7296-4b40-a9eb-8d3fdc9b5e67 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:18.561321 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d8803bce-7296-4b40-a9eb-8d3fdc9b5e67 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:18.562093 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d8803bce-7296-4b40-a9eb-8d3fdc9b5e67 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:18.562548 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d8803bce-7296-4b40-a9eb-8d3fdc9b5e67 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:18.770769 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-d8803bce-7296-4b40-a9eb-8d3fdc9b5e67 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '027dcc48e4fd4c36a5004096ad5e43f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dad823e240149edb30e592380249a13', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:27:18.787476 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-d8803bce-7296-4b40-a9eb-8d3fdc9b5e67 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "GET /compute/v2.1/servers/a45c5bbd-ab66-426a-8e31-f471ce68606f" status: 200 len: 1510 microversion: 2.1 time: 0.241696 Jul 16 21:27:18.788282 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1990/3990] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:27:18 2022] GET /compute/v2.1/servers/a45c5bbd-ab66-426a-8e31-f471ce68606f => generated 1510 bytes in 243 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:27:18.794047 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-dad8c7df-8922-46b7-a38c-1be059d71afd req-0797c0f5-69f1-4454-a73b-75185daeb576 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "fbb7fab1-a4d5-48e2-9ace-fc65fad094ce", "name": "network-vif-plugged", "status": "completed", "tag": "2ae39666-a3c4-4738-a6cb-db9cc84fdcd6"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:27:18.802857 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-dad8c7df-8922-46b7-a38c-1be059d71afd req-0797c0f5-69f1-4454-a73b-75185daeb576 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:18.803698 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-dad8c7df-8922-46b7-a38c-1be059d71afd req-0797c0f5-69f1-4454-a73b-75185daeb576 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:18.837033 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-dad8c7df-8922-46b7-a38c-1be059d71afd req-0797c0f5-69f1-4454-a73b-75185daeb576 service nova] Creating event network-vif-plugged:2ae39666-a3c4-4738-a6cb-db9cc84fdcd6 for instance fbb7fab1-a4d5-48e2-9ace-fc65fad094ce on nch2-master-1 Jul 16 21:27:18.846108 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-dad8c7df-8922-46b7-a38c-1be059d71afd req-0797c0f5-69f1-4454-a73b-75185daeb576 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.053896 Jul 16 21:27:18.849227 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1991/3991] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:27:18 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 60 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:27:18.880883 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-3cab0e7f-2b34-4f7b-b098-1b9a95589597 tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] Flavor limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 16 21:27:18.881359 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-3cab0e7f-2b34-4f7b-b098-1b9a95589597 tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] Image limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 16 21:27:18.881872 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-3cab0e7f-2b34-4f7b-b098-1b9a95589597 tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] Flavor pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 16 21:27:18.882306 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-3cab0e7f-2b34-4f7b-b098-1b9a95589597 tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] Image pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 16 21:27:18.882715 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-3cab0e7f-2b34-4f7b-b098-1b9a95589597 tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 16 21:27:18.917371 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3cab0e7f-2b34-4f7b-b098-1b9a95589597 tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:18.917371 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3cab0e7f-2b34-4f7b-b098-1b9a95589597 tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:18.917371 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3cab0e7f-2b34-4f7b-b098-1b9a95589597 tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:18.917371 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3cab0e7f-2b34-4f7b-b098-1b9a95589597 tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:18.961892 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-3cab0e7f-2b34-4f7b-b098-1b9a95589597 tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] Getting quotas for project aef73d309b204eeebfc2ce69b4e4a41b. Resources: {'ram', 'instances', 'cores'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:27:18.983728 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-3cab0e7f-2b34-4f7b-b098-1b9a95589597 tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] Getting quotas for user 45315c12d53349ae8cd245cb138b8dcd and project aef73d309b204eeebfc2ce69b4e4a41b. Resources: {'ram', 'instances', 'cores'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:27:19.010079 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-3cab0e7f-2b34-4f7b-b098-1b9a95589597 tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] Going to run 1 instances... {{(pid=138681) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 16 21:27:19.010079 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-3cab0e7f-2b34-4f7b-b098-1b9a95589597 tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] [instance: fbc2a45b-c662-475e-936d-70697843ce90] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138681) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 16 21:27:19.214549 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-3cab0e7f-2b34-4f7b-b098-1b9a95589597 tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.21 time: 1.189159 Jul 16 21:27:19.217151 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2001/3992] 10.222.0.16 () {62 vars in 1187 bytes} [Sat Jul 16 21:27:18 2022] POST /compute/v2.1/servers => generated 380 bytes in 1192 msecs (HTTP/1.1 202) 10 headers in 453 bytes (1 switches on core 0) Jul 16 21:27:19.230063 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-2e88a94e-7da0-45f3-9422-f020a0433570 tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:27:19.231177 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-2e88a94e-7da0-45f3-9422-f020a0433570 tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] [instance: fbc2a45b-c662-475e-936d-70697843ce90] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:27:19.305515 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-2e88a94e-7da0-45f3-9422-f020a0433570 tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '45315c12d53349ae8cd245cb138b8dcd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'aef73d309b204eeebfc2ce69b4e4a41b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:27:19.314353 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-2e88a94e-7da0-45f3-9422-f020a0433570 tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '45315c12d53349ae8cd245cb138b8dcd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'aef73d309b204eeebfc2ce69b4e4a41b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:27:19.316537 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-2e88a94e-7da0-45f3-9422-f020a0433570 tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '45315c12d53349ae8cd245cb138b8dcd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'aef73d309b204eeebfc2ce69b4e4a41b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:27:19.317902 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-2e88a94e-7da0-45f3-9422-f020a0433570 tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] 10.222.0.16 "GET /compute/v2.1/servers/fbc2a45b-c662-475e-936d-70697843ce90" status: 200 len: 1259 microversion: 2.21 time: 0.092916 Jul 16 21:27:19.318478 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1992/3993] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:27:19 2022] GET /compute/v2.1/servers/fbc2a45b-c662-475e-936d-70697843ce90 => generated 1259 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 16 21:27:19.561247 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-3d3aaf13-23ef-40db-8f81-78e553953e30 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:27:19.561874 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-3d3aaf13-23ef-40db-8f81-78e553953e30 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] [instance: fbb7fab1-a4d5-48e2-9ace-fc65fad094ce] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:27:19.569452 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3d3aaf13-23ef-40db-8f81-78e553953e30 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:19.570273 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3d3aaf13-23ef-40db-8f81-78e553953e30 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:19.575044 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3d3aaf13-23ef-40db-8f81-78e553953e30 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:19.575044 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3d3aaf13-23ef-40db-8f81-78e553953e30 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:19.736658 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-3d3aaf13-23ef-40db-8f81-78e553953e30 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '029c76a0ba614b5880128c4309902c67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3b214f5044c7445592e177835a5d18fc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:27:19.746113 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-3d3aaf13-23ef-40db-8f81-78e553953e30 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] 10.222.0.16 "GET /compute/v2.1/servers/fbb7fab1-a4d5-48e2-9ace-fc65fad094ce" status: 200 len: 1320 microversion: 2.1 time: 0.188924 Jul 16 21:27:19.746757 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2002/3994] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:27:19 2022] GET /compute/v2.1/servers/fbb7fab1-a4d5-48e2-9ace-fc65fad094ce => generated 1320 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:27:19.805831 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-5aad9ca9-3ca7-490c-8cfa-75945a13313b tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:27:19.806458 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-5aad9ca9-3ca7-490c-8cfa-75945a13313b tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: a45c5bbd-ab66-426a-8e31-f471ce68606f] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:27:19.814702 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5aad9ca9-3ca7-490c-8cfa-75945a13313b tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:19.815994 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5aad9ca9-3ca7-490c-8cfa-75945a13313b tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:19.816789 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5aad9ca9-3ca7-490c-8cfa-75945a13313b tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:19.817237 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5aad9ca9-3ca7-490c-8cfa-75945a13313b tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:20.026737 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-5aad9ca9-3ca7-490c-8cfa-75945a13313b tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '027dcc48e4fd4c36a5004096ad5e43f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dad823e240149edb30e592380249a13', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:27:20.052926 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-5aad9ca9-3ca7-490c-8cfa-75945a13313b tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "GET /compute/v2.1/servers/a45c5bbd-ab66-426a-8e31-f471ce68606f" status: 200 len: 1510 microversion: 2.1 time: 0.243438 Jul 16 21:27:20.052926 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1993/3995] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:27:19 2022] GET /compute/v2.1/servers/a45c5bbd-ab66-426a-8e31-f471ce68606f => generated 1510 bytes in 249 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:27:20.347900 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-37d3086f-4e7e-45d3-9502-b0222e6b418d tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:27:20.347900 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-37d3086f-4e7e-45d3-9502-b0222e6b418d tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] [instance: fbc2a45b-c662-475e-936d-70697843ce90] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:27:20.355607 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_api:SELECT=94,nova_api:INSERT=6 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:27:20.407470 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-37d3086f-4e7e-45d3-9502-b0222e6b418d tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '45315c12d53349ae8cd245cb138b8dcd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'aef73d309b204eeebfc2ce69b4e4a41b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:27:20.416191 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-37d3086f-4e7e-45d3-9502-b0222e6b418d tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '45315c12d53349ae8cd245cb138b8dcd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'aef73d309b204eeebfc2ce69b4e4a41b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:27:20.417801 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-37d3086f-4e7e-45d3-9502-b0222e6b418d tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '45315c12d53349ae8cd245cb138b8dcd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'aef73d309b204eeebfc2ce69b4e4a41b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:27:20.418924 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-37d3086f-4e7e-45d3-9502-b0222e6b418d tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] 10.222.0.16 "GET /compute/v2.1/servers/fbc2a45b-c662-475e-936d-70697843ce90" status: 200 len: 1259 microversion: 2.21 time: 0.084298 Jul 16 21:27:20.419753 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2003/3996] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:27:20 2022] GET /compute/v2.1/servers/fbc2a45b-c662-475e-936d-70697843ce90 => generated 1259 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:27:20.769118 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-4616c691-3a56-40e1-87ec-cd8dddf80baf tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:27:20.769741 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-4616c691-3a56-40e1-87ec-cd8dddf80baf tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] [instance: fbb7fab1-a4d5-48e2-9ace-fc65fad094ce] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:27:20.777765 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4616c691-3a56-40e1-87ec-cd8dddf80baf tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:20.778230 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4616c691-3a56-40e1-87ec-cd8dddf80baf tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:20.779045 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4616c691-3a56-40e1-87ec-cd8dddf80baf tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:20.779544 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4616c691-3a56-40e1-87ec-cd8dddf80baf tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:20.954160 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-4616c691-3a56-40e1-87ec-cd8dddf80baf tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '029c76a0ba614b5880128c4309902c67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3b214f5044c7445592e177835a5d18fc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:27:20.964096 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-4616c691-3a56-40e1-87ec-cd8dddf80baf tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] 10.222.0.16 "GET /compute/v2.1/servers/fbb7fab1-a4d5-48e2-9ace-fc65fad094ce" status: 200 len: 1510 microversion: 2.1 time: 0.198762 Jul 16 21:27:20.964705 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1994/3997] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:27:20 2022] GET /compute/v2.1/servers/fbb7fab1-a4d5-48e2-9ace-fc65fad094ce => generated 1510 bytes in 200 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:27:20.982107 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-97dd6d0e-9759-4f60-b211-9d93572c37dc tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:27:20.982820 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-97dd6d0e-9759-4f60-b211-9d93572c37dc tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] [instance: cf38328b-40fb-4537-b524-aaeb0a0abfb4] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:27:20.990974 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-97dd6d0e-9759-4f60-b211-9d93572c37dc tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:20.992834 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-97dd6d0e-9759-4f60-b211-9d93572c37dc tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:20.993853 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-97dd6d0e-9759-4f60-b211-9d93572c37dc tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:20.994587 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-97dd6d0e-9759-4f60-b211-9d93572c37dc tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:21.067319 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-14cbaef4-075a-4ced-b05f-58e88301e474 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:27:21.068008 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-14cbaef4-075a-4ced-b05f-58e88301e474 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: a45c5bbd-ab66-426a-8e31-f471ce68606f] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:27:21.076606 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-14cbaef4-075a-4ced-b05f-58e88301e474 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:21.077424 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-14cbaef4-075a-4ced-b05f-58e88301e474 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:21.080094 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-14cbaef4-075a-4ced-b05f-58e88301e474 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:21.082660 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-14cbaef4-075a-4ced-b05f-58e88301e474 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:21.255444 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-97dd6d0e-9759-4f60-b211-9d93572c37dc tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '029c76a0ba614b5880128c4309902c67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3b214f5044c7445592e177835a5d18fc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:27:21.265598 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-97dd6d0e-9759-4f60-b211-9d93572c37dc tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] 10.222.0.16 "GET /compute/v2.1/servers/cf38328b-40fb-4537-b524-aaeb0a0abfb4" status: 200 len: 1511 microversion: 2.1 time: 0.287012 Jul 16 21:27:21.266197 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2004/3998] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:27:20 2022] GET /compute/v2.1/servers/cf38328b-40fb-4537-b524-aaeb0a0abfb4 => generated 1511 bytes in 289 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:27:21.283404 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-a6578ba1-ed1c-4afd-877f-4611cac4faa3 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:27:21.285198 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-a6578ba1-ed1c-4afd-877f-4611cac4faa3 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] [instance: b01533f3-9a31-4246-b3e7-f12f638e8e89] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:27:21.290015 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-14cbaef4-075a-4ced-b05f-58e88301e474 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '027dcc48e4fd4c36a5004096ad5e43f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dad823e240149edb30e592380249a13', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:27:21.293036 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a6578ba1-ed1c-4afd-877f-4611cac4faa3 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:21.293673 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a6578ba1-ed1c-4afd-877f-4611cac4faa3 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:21.294916 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a6578ba1-ed1c-4afd-877f-4611cac4faa3 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:21.295999 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a6578ba1-ed1c-4afd-877f-4611cac4faa3 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:21.299882 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-14cbaef4-075a-4ced-b05f-58e88301e474 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "GET /compute/v2.1/servers/a45c5bbd-ab66-426a-8e31-f471ce68606f" status: 200 len: 1510 microversion: 2.1 time: 0.238304 Jul 16 21:27:21.300937 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1995/3999] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:27:21 2022] GET /compute/v2.1/servers/a45c5bbd-ab66-426a-8e31-f471ce68606f => generated 1510 bytes in 240 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:27:21.438965 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-aea26c2f-f193-497e-8ff0-5667814d3946 tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:27:21.440428 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-aea26c2f-f193-497e-8ff0-5667814d3946 tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] [instance: fbc2a45b-c662-475e-936d-70697843ce90] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:27:21.468241 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-aea26c2f-f193-497e-8ff0-5667814d3946 tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:21.469156 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-aea26c2f-f193-497e-8ff0-5667814d3946 tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:21.472429 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-aea26c2f-f193-497e-8ff0-5667814d3946 tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:21.473219 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-a6578ba1-ed1c-4afd-877f-4611cac4faa3 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '029c76a0ba614b5880128c4309902c67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3b214f5044c7445592e177835a5d18fc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:27:21.473836 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-aea26c2f-f193-497e-8ff0-5667814d3946 tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:21.481356 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-a6578ba1-ed1c-4afd-877f-4611cac4faa3 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] 10.222.0.16 "GET /compute/v2.1/servers/b01533f3-9a31-4246-b3e7-f12f638e8e89" status: 200 len: 1511 microversion: 2.1 time: 0.202618 Jul 16 21:27:21.481963 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2005/4000] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:27:21 2022] GET /compute/v2.1/servers/b01533f3-9a31-4246-b3e7-f12f638e8e89 => generated 1511 bytes in 204 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:27:21.509429 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-87b2718f-922c-464d-93e7-a6a96171b7d3 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:27:21.521925 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-87b2718f-922c-464d-93e7-a6a96171b7d3 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Policy check for os_compute_api:servers:allow_all_filters failed with credentials {'is_admin': False, 'user_id': '029c76a0ba614b5880128c4309902c67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3b214f5044c7445592e177835a5d18fc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:27:21.522903 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-87b2718f-922c-464d-93e7-a6a96171b7d3 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Searching by: {'flavor': '451', 'deleted': False, 'project_id': '3b214f5044c7445592e177835a5d18fc'} {{(pid=138681) get_all /opt/stack/nova/nova/compute/api.py:2962}} Jul 16 21:27:21.552431 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-87b2718f-922c-464d-93e7-a6a96171b7d3 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:21.553730 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-87b2718f-922c-464d-93e7-a6a96171b7d3 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:21.554644 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-87b2718f-922c-464d-93e7-a6a96171b7d3 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:21.555572 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-87b2718f-922c-464d-93e7-a6a96171b7d3 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:21.571681 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-aea26c2f-f193-497e-8ff0-5667814d3946 tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '45315c12d53349ae8cd245cb138b8dcd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'aef73d309b204eeebfc2ce69b4e4a41b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:27:21.587102 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-aea26c2f-f193-497e-8ff0-5667814d3946 tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '45315c12d53349ae8cd245cb138b8dcd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'aef73d309b204eeebfc2ce69b4e4a41b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:27:21.588650 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-aea26c2f-f193-497e-8ff0-5667814d3946 tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '45315c12d53349ae8cd245cb138b8dcd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'aef73d309b204eeebfc2ce69b4e4a41b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:27:21.589777 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-aea26c2f-f193-497e-8ff0-5667814d3946 tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] 10.222.0.16 "GET /compute/v2.1/servers/fbc2a45b-c662-475e-936d-70697843ce90" status: 200 len: 1255 microversion: 2.21 time: 0.156015 Jul 16 21:27:21.590499 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1996/4001] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:27:21 2022] GET /compute/v2.1/servers/fbc2a45b-c662-475e-936d-70697843ce90 => generated 1255 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:27:21.715093 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.multi_cell_list [None req-87b2718f-922c-464d-93e7-a6a96171b7d3 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Listed batch of 1 results from cell out of 1000 limit. Returned 1 total so far. {{(pid=138681) do_query /opt/stack/nova/nova/compute/multi_cell_list.py:373}} Jul 16 21:27:21.793771 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-87b2718f-922c-464d-93e7-a6a96171b7d3 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '029c76a0ba614b5880128c4309902c67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3b214f5044c7445592e177835a5d18fc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:27:21.801319 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-87b2718f-922c-464d-93e7-a6a96171b7d3 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:21.802174 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-87b2718f-922c-464d-93e7-a6a96171b7d3 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:21.947545 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-87b2718f-922c-464d-93e7-a6a96171b7d3 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] 10.222.0.16 "GET /compute/v2.1/servers/detail?flavor=451" status: 200 len: 1513 microversion: 2.1 time: 0.446039 Jul 16 21:27:21.948169 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2006/4002] 10.222.0.16 () {58 vars in 1166 bytes} [Sat Jul 16 21:27:21 2022] GET /compute/v2.1/servers/detail?flavor=451 => generated 1513 bytes in 451 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:27:21.965793 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-06868fe4-89ce-44e8-9d8f-7163c30a21de tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:27:21.969743 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-06868fe4-89ce-44e8-9d8f-7163c30a21de tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Policy check for os_compute_api:servers:allow_all_filters failed with credentials {'is_admin': False, 'user_id': '029c76a0ba614b5880128c4309902c67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3b214f5044c7445592e177835a5d18fc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:27:21.970654 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-06868fe4-89ce-44e8-9d8f-7163c30a21de tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Searching by: {'name': 'tempest-ListServerFiltersTestJSON-instance-1583187080', 'deleted': False, 'project_id': '3b214f5044c7445592e177835a5d18fc'} {{(pid=138680) get_all /opt/stack/nova/nova/compute/api.py:2962}} Jul 16 21:27:21.977817 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-06868fe4-89ce-44e8-9d8f-7163c30a21de tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:21.978291 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-06868fe4-89ce-44e8-9d8f-7163c30a21de tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:21.979104 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-06868fe4-89ce-44e8-9d8f-7163c30a21de tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:21.979546 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-06868fe4-89ce-44e8-9d8f-7163c30a21de tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:22.081552 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.multi_cell_list [None req-06868fe4-89ce-44e8-9d8f-7163c30a21de tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Listed batch of 1 results from cell out of 1000 limit. Returned 1 total so far. {{(pid=138680) do_query /opt/stack/nova/nova/compute/multi_cell_list.py:373}} Jul 16 21:27:22.153177 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-06868fe4-89ce-44e8-9d8f-7163c30a21de tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '029c76a0ba614b5880128c4309902c67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3b214f5044c7445592e177835a5d18fc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:27:22.159585 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-06868fe4-89ce-44e8-9d8f-7163c30a21de tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:22.160036 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-06868fe4-89ce-44e8-9d8f-7163c30a21de tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:22.306312 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-06868fe4-89ce-44e8-9d8f-7163c30a21de tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] 10.222.0.16 "GET /compute/v2.1/servers/detail?name=tempest-ListServerFiltersTestJSON-instance-1583187080" status: 200 len: 1514 microversion: 2.1 time: 0.344420 Jul 16 21:27:22.307220 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1997/4003] 10.222.0.16 () {58 vars in 1262 bytes} [Sat Jul 16 21:27:21 2022] GET /compute/v2.1/servers/detail?name=tempest-ListServerFiltersTestJSON-instance-1583187080 => generated 1514 bytes in 346 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:27:22.323786 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-9016966e-9cd8-450d-b489-d38295300140 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:27:22.324448 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-9016966e-9cd8-450d-b489-d38295300140 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: a45c5bbd-ab66-426a-8e31-f471ce68606f] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:27:22.328097 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-7c020a53-3656-42e6-9d61-b8f2ba53afa7 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:27:22.331295 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9016966e-9cd8-450d-b489-d38295300140 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:22.331731 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-7c020a53-3656-42e6-9d61-b8f2ba53afa7 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Policy check for os_compute_api:servers:allow_all_filters failed with credentials {'is_admin': False, 'user_id': '029c76a0ba614b5880128c4309902c67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3b214f5044c7445592e177835a5d18fc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:27:22.331909 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9016966e-9cd8-450d-b489-d38295300140 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:22.332542 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-7c020a53-3656-42e6-9d61-b8f2ba53afa7 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Searching by: {'vm_state': ['active'], 'deleted': False, 'project_id': '3b214f5044c7445592e177835a5d18fc'} {{(pid=138680) get_all /opt/stack/nova/nova/compute/api.py:2962}} Jul 16 21:27:22.332886 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9016966e-9cd8-450d-b489-d38295300140 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:22.333403 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9016966e-9cd8-450d-b489-d38295300140 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:22.338163 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7c020a53-3656-42e6-9d61-b8f2ba53afa7 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:22.338616 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7c020a53-3656-42e6-9d61-b8f2ba53afa7 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:22.339636 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7c020a53-3656-42e6-9d61-b8f2ba53afa7 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:22.340079 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7c020a53-3656-42e6-9d61-b8f2ba53afa7 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:22.507308 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.multi_cell_list [None req-7c020a53-3656-42e6-9d61-b8f2ba53afa7 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Listed batch of 3 results from cell out of 1000 limit. Returned 3 total so far. {{(pid=138680) do_query /opt/stack/nova/nova/compute/multi_cell_list.py:373}} Jul 16 21:27:22.607886 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-7c020a53-3656-42e6-9d61-b8f2ba53afa7 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '029c76a0ba614b5880128c4309902c67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3b214f5044c7445592e177835a5d18fc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:27:22.616880 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7c020a53-3656-42e6-9d61-b8f2ba53afa7 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:22.617372 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7c020a53-3656-42e6-9d61-b8f2ba53afa7 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:22.640555 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-9016966e-9cd8-450d-b489-d38295300140 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '027dcc48e4fd4c36a5004096ad5e43f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dad823e240149edb30e592380249a13', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:27:22.642791 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=100 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:27:22.655456 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-9016966e-9cd8-450d-b489-d38295300140 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "GET /compute/v2.1/servers/a45c5bbd-ab66-426a-8e31-f471ce68606f" status: 200 len: 1510 microversion: 2.1 time: 0.336341 Jul 16 21:27:22.655998 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2007/4004] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:27:22 2022] GET /compute/v2.1/servers/a45c5bbd-ab66-426a-8e31-f471ce68606f => generated 1510 bytes in 338 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:27:22.665608 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-212cb7b3-be0a-4605-a571-d4bdaaf84357 tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:27:22.665608 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-212cb7b3-be0a-4605-a571-d4bdaaf84357 tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] [instance: fbc2a45b-c662-475e-936d-70697843ce90] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:27:22.670098 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-212cb7b3-be0a-4605-a571-d4bdaaf84357 tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:22.670552 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-212cb7b3-be0a-4605-a571-d4bdaaf84357 tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:22.671597 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-212cb7b3-be0a-4605-a571-d4bdaaf84357 tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:22.672085 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-212cb7b3-be0a-4605-a571-d4bdaaf84357 tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:22.766097 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-212cb7b3-be0a-4605-a571-d4bdaaf84357 tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '45315c12d53349ae8cd245cb138b8dcd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'aef73d309b204eeebfc2ce69b4e4a41b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:27:22.774700 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-212cb7b3-be0a-4605-a571-d4bdaaf84357 tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '45315c12d53349ae8cd245cb138b8dcd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'aef73d309b204eeebfc2ce69b4e4a41b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:27:22.776241 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-212cb7b3-be0a-4605-a571-d4bdaaf84357 tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '45315c12d53349ae8cd245cb138b8dcd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'aef73d309b204eeebfc2ce69b4e4a41b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:27:22.777345 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-212cb7b3-be0a-4605-a571-d4bdaaf84357 tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] 10.222.0.16 "GET /compute/v2.1/servers/fbc2a45b-c662-475e-936d-70697843ce90" status: 200 len: 1329 microversion: 2.21 time: 0.120290 Jul 16 21:27:22.777914 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2008/4005] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:27:22 2022] GET /compute/v2.1/servers/fbc2a45b-c662-475e-936d-70697843ce90 => generated 1329 bytes in 121 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:27:22.817023 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-7c020a53-3656-42e6-9d61-b8f2ba53afa7 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] 10.222.0.16 "GET /compute/v2.1/servers/detail?status=active" status: 200 len: 4515 microversion: 2.1 time: 0.492208 Jul 16 21:27:22.831284 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1998/4006] 10.222.0.16 () {58 vars in 1172 bytes} [Sat Jul 16 21:27:22 2022] GET /compute/v2.1/servers/detail?status=active => generated 4515 bytes in 506 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:27:22.840756 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-69193f89-6679-4884-8ced-e1f80a94c704 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:27:22.844908 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-69193f89-6679-4884-8ced-e1f80a94c704 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Policy check for os_compute_api:servers:allow_all_filters failed with credentials {'is_admin': False, 'user_id': '029c76a0ba614b5880128c4309902c67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3b214f5044c7445592e177835a5d18fc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:27:22.849746 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.compute.servers [None req-69193f89-6679-4884-8ced-e1f80a94c704 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Removing options 'limit' from query {{(pid=138681) remove_invalid_options /opt/stack/nova/nova/api/openstack/compute/servers.py:1487}} Jul 16 21:27:22.849746 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-69193f89-6679-4884-8ced-e1f80a94c704 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Searching by: {'deleted': False, 'project_id': '3b214f5044c7445592e177835a5d18fc'} {{(pid=138681) get_all /opt/stack/nova/nova/compute/api.py:2962}} Jul 16 21:27:22.857518 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-69193f89-6679-4884-8ced-e1f80a94c704 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:22.858168 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-69193f89-6679-4884-8ced-e1f80a94c704 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:22.859174 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-69193f89-6679-4884-8ced-e1f80a94c704 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:22.859801 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-69193f89-6679-4884-8ced-e1f80a94c704 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:22.905428 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-69193f89-6679-4884-8ced-e1f80a94c704 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '029c76a0ba614b5880128c4309902c67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3b214f5044c7445592e177835a5d18fc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:27:22.913996 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-69193f89-6679-4884-8ced-e1f80a94c704 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:22.915053 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-69193f89-6679-4884-8ced-e1f80a94c704 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:23.082147 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-69193f89-6679-4884-8ced-e1f80a94c704 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] 10.222.0.16 "GET /compute/v2.1/servers/detail?limit=1" status: 200 len: 1659 microversion: 2.1 time: 0.245175 Jul 16 21:27:23.084010 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2009/4007] 10.222.0.16 () {58 vars in 1160 bytes} [Sat Jul 16 21:27:22 2022] GET /compute/v2.1/servers/detail?limit=1 => generated 1659 bytes in 247 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:27:23.102268 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-159cd10f-8e2a-4b5e-84ee-7dff9e6d1c18 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:27:23.107606 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-159cd10f-8e2a-4b5e-84ee-7dff9e6d1c18 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Policy check for os_compute_api:servers:allow_all_filters failed with credentials {'is_admin': False, 'user_id': '029c76a0ba614b5880128c4309902c67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3b214f5044c7445592e177835a5d18fc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:27:23.108870 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-159cd10f-8e2a-4b5e-84ee-7dff9e6d1c18 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Searching by: {'vm_state': ['active'], 'deleted': False, 'project_id': '3b214f5044c7445592e177835a5d18fc'} {{(pid=138680) get_all /opt/stack/nova/nova/compute/api.py:2962}} Jul 16 21:27:23.127678 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-159cd10f-8e2a-4b5e-84ee-7dff9e6d1c18 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:23.128705 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-159cd10f-8e2a-4b5e-84ee-7dff9e6d1c18 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:23.129818 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-159cd10f-8e2a-4b5e-84ee-7dff9e6d1c18 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:23.130681 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-159cd10f-8e2a-4b5e-84ee-7dff9e6d1c18 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:23.242920 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.multi_cell_list [None req-159cd10f-8e2a-4b5e-84ee-7dff9e6d1c18 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Listed batch of 3 results from cell out of 1000 limit. Returned 3 total so far. {{(pid=138680) do_query /opt/stack/nova/nova/compute/multi_cell_list.py:373}} Jul 16 21:27:23.303225 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-159cd10f-8e2a-4b5e-84ee-7dff9e6d1c18 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] 10.222.0.16 "GET /compute/v2.1/servers?status=active" status: 200 len: 1010 microversion: 2.1 time: 0.205614 Jul 16 21:27:23.303832 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 1999/4008] 10.222.0.16 () {58 vars in 1151 bytes} [Sat Jul 16 21:27:23 2022] GET /compute/v2.1/servers?status=active => generated 1010 bytes in 207 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:27:23.315458 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-d5f4fe4c-8dda-47ff-a26b-409161faf28b tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:27:23.318882 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-d5f4fe4c-8dda-47ff-a26b-409161faf28b tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Policy check for os_compute_api:servers:allow_all_filters failed with credentials {'is_admin': False, 'user_id': '029c76a0ba614b5880128c4309902c67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3b214f5044c7445592e177835a5d18fc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:27:23.319385 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.compute.servers [None req-d5f4fe4c-8dda-47ff-a26b-409161faf28b tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Removing options 'limit' from query {{(pid=138681) remove_invalid_options /opt/stack/nova/nova/api/openstack/compute/servers.py:1487}} Jul 16 21:27:23.320046 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-d5f4fe4c-8dda-47ff-a26b-409161faf28b tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Searching by: {'deleted': False, 'project_id': '3b214f5044c7445592e177835a5d18fc'} {{(pid=138681) get_all /opt/stack/nova/nova/compute/api.py:2962}} Jul 16 21:27:23.325405 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d5f4fe4c-8dda-47ff-a26b-409161faf28b tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:23.325841 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d5f4fe4c-8dda-47ff-a26b-409161faf28b tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:23.326702 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d5f4fe4c-8dda-47ff-a26b-409161faf28b tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:23.327179 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d5f4fe4c-8dda-47ff-a26b-409161faf28b tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:23.354283 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.multi_cell_list [None req-d5f4fe4c-8dda-47ff-a26b-409161faf28b tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Listed batch of 3 results from cell out of 1000 limit. Returned 3 total so far. {{(pid=138681) do_query /opt/stack/nova/nova/compute/multi_cell_list.py:373}} Jul 16 21:27:23.382828 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-d5f4fe4c-8dda-47ff-a26b-409161faf28b tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] 10.222.0.16 "GET /compute/v2.1/servers?limit=100000" status: 200 len: 1010 microversion: 2.1 time: 0.071256 Jul 16 21:27:23.388767 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2010/4009] 10.222.0.16 () {58 vars in 1149 bytes} [Sat Jul 16 21:27:23 2022] GET /compute/v2.1/servers?limit=100000 => generated 1010 bytes in 73 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:27:23.394484 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-0058b661-da4b-4331-9b74-2df632e4f173 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:27:23.397773 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-0058b661-da4b-4331-9b74-2df632e4f173 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Policy check for os_compute_api:servers:allow_all_filters failed with credentials {'is_admin': False, 'user_id': '029c76a0ba614b5880128c4309902c67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3b214f5044c7445592e177835a5d18fc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:27:23.398471 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-0058b661-da4b-4331-9b74-2df632e4f173 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Searching by: {'deleted': False, 'project_id': '3b214f5044c7445592e177835a5d18fc'} {{(pid=138680) get_all /opt/stack/nova/nova/compute/api.py:2962}} Jul 16 21:27:23.404957 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0058b661-da4b-4331-9b74-2df632e4f173 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:23.405414 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0058b661-da4b-4331-9b74-2df632e4f173 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:23.406645 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0058b661-da4b-4331-9b74-2df632e4f173 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:23.407123 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0058b661-da4b-4331-9b74-2df632e4f173 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:23.461915 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.multi_cell_list [None req-0058b661-da4b-4331-9b74-2df632e4f173 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Listed batch of 3 results from cell out of 1000 limit. Returned 3 total so far. {{(pid=138680) do_query /opt/stack/nova/nova/compute/multi_cell_list.py:373}} Jul 16 21:27:23.476297 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-ec265e51-f6db-41b4-aaff-c1da66b36a4e tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:27:23.476955 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-ec265e51-f6db-41b4-aaff-c1da66b36a4e tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] [instance: 9a184cb7-2548-41d5-8d37-8e72dfb3d75a] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:27:23.485935 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ec265e51-f6db-41b4-aaff-c1da66b36a4e tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:23.487129 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ec265e51-f6db-41b4-aaff-c1da66b36a4e tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:23.488148 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ec265e51-f6db-41b4-aaff-c1da66b36a4e tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:23.488850 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ec265e51-f6db-41b4-aaff-c1da66b36a4e tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:23.514126 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-0058b661-da4b-4331-9b74-2df632e4f173 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] 10.222.0.16 "GET /compute/v2.1/servers" status: 200 len: 1010 microversion: 2.1 time: 0.123381 Jul 16 21:27:23.514753 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2000/4010] 10.222.0.16 () {58 vars in 1124 bytes} [Sat Jul 16 21:27:23 2022] GET /compute/v2.1/servers => generated 1010 bytes in 125 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:27:23.530343 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-a2ee7ab3-4a9b-4943-81fd-aa2016770956 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:27:23.535462 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-a2ee7ab3-4a9b-4943-81fd-aa2016770956 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Policy check for os_compute_api:servers:allow_all_filters failed with credentials {'is_admin': False, 'user_id': '029c76a0ba614b5880128c4309902c67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3b214f5044c7445592e177835a5d18fc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:27:23.536642 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-a2ee7ab3-4a9b-4943-81fd-aa2016770956 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Searching by: {'flavor': '451', 'deleted': False, 'project_id': '3b214f5044c7445592e177835a5d18fc'} {{(pid=138680) get_all /opt/stack/nova/nova/compute/api.py:2962}} Jul 16 21:27:23.550871 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a2ee7ab3-4a9b-4943-81fd-aa2016770956 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:23.551383 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a2ee7ab3-4a9b-4943-81fd-aa2016770956 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:23.552190 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a2ee7ab3-4a9b-4943-81fd-aa2016770956 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:23.552656 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a2ee7ab3-4a9b-4943-81fd-aa2016770956 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:23.605107 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.objects.instance [None req-ec265e51-f6db-41b4-aaff-c1da66b36a4e tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] Lazy-loading 'services' on Instance uuid 9a184cb7-2548-41d5-8d37-8e72dfb3d75a {{(pid=138681) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:27:23.677879 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.multi_cell_list [None req-a2ee7ab3-4a9b-4943-81fd-aa2016770956 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Listed batch of 1 results from cell out of 1000 limit. Returned 1 total so far. {{(pid=138680) do_query /opt/stack/nova/nova/compute/multi_cell_list.py:373}} Jul 16 21:27:23.759603 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-a2ee7ab3-4a9b-4943-81fd-aa2016770956 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] 10.222.0.16 "GET /compute/v2.1/servers?flavor=451" status: 200 len: 345 microversion: 2.1 time: 0.236583 Jul 16 21:27:23.760297 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2001/4011] 10.222.0.16 () {58 vars in 1145 bytes} [Sat Jul 16 21:27:23 2022] GET /compute/v2.1/servers?flavor=451 => generated 345 bytes in 238 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:27:23.772773 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-240dce0e-a356-4aea-8f97-b90880176473 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:27:23.773497 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-240dce0e-a356-4aea-8f97-b90880176473 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: a45c5bbd-ab66-426a-8e31-f471ce68606f] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:27:23.779761 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-240dce0e-a356-4aea-8f97-b90880176473 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:23.780344 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-240dce0e-a356-4aea-8f97-b90880176473 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:23.781211 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-240dce0e-a356-4aea-8f97-b90880176473 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:23.781721 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-240dce0e-a356-4aea-8f97-b90880176473 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:23.907550 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-ec265e51-f6db-41b4-aaff-c1da66b36a4e tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] 10.222.0.16 "DELETE /compute/v2.1/servers/9a184cb7-2548-41d5-8d37-8e72dfb3d75a/os-volume_attachments/be0c4ae4-0277-4cb8-92f2-b87b2a58c6b6" status: 202 len: 0 microversion: 2.1 time: 0.440670 Jul 16 21:27:23.908318 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2011/4012] 10.222.0.16 () {58 vars in 1415 bytes} [Sat Jul 16 21:27:23 2022] DELETE /compute/v2.1/servers/9a184cb7-2548-41d5-8d37-8e72dfb3d75a/os-volume_attachments/be0c4ae4-0277-4cb8-92f2-b87b2a58c6b6 => generated 0 bytes in 443 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 21:27:23.918904 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-94177f96-bab0-42ee-857e-b4e493a65ae3 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:27:23.923042 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-94177f96-bab0-42ee-857e-b4e493a65ae3 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Policy check for os_compute_api:servers:allow_all_filters failed with credentials {'is_admin': False, 'user_id': '029c76a0ba614b5880128c4309902c67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3b214f5044c7445592e177835a5d18fc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:27:23.926006 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.compute.servers [None req-94177f96-bab0-42ee-857e-b4e493a65ae3 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Removing options 'limit' from query {{(pid=138681) remove_invalid_options /opt/stack/nova/nova/api/openstack/compute/servers.py:1487}} Jul 16 21:27:23.926794 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-94177f96-bab0-42ee-857e-b4e493a65ae3 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Searching by: {'deleted': False, 'project_id': '3b214f5044c7445592e177835a5d18fc'} {{(pid=138681) get_all /opt/stack/nova/nova/compute/api.py:2962}} Jul 16 21:27:23.933873 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-94177f96-bab0-42ee-857e-b4e493a65ae3 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:23.934397 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-94177f96-bab0-42ee-857e-b4e493a65ae3 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:23.935599 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-94177f96-bab0-42ee-857e-b4e493a65ae3 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:23.936044 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-94177f96-bab0-42ee-857e-b4e493a65ae3 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:23.973292 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-94177f96-bab0-42ee-857e-b4e493a65ae3 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] 10.222.0.16 "GET /compute/v2.1/servers?limit=1" status: 200 len: 484 microversion: 2.1 time: 0.060496 Jul 16 21:27:23.975929 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2012/4013] 10.222.0.16 () {58 vars in 1139 bytes} [Sat Jul 16 21:27:23 2022] GET /compute/v2.1/servers?limit=1 => generated 484 bytes in 67 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:27:23.981706 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-47c30ecf-87a2-42ad-b4aa-5c2d6303f3ba tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:27:23.982963 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-47c30ecf-87a2-42ad-b4aa-5c2d6303f3ba tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] [instance: fbc2a45b-c662-475e-936d-70697843ce90] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:27:23.992674 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-47c30ecf-87a2-42ad-b4aa-5c2d6303f3ba tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:23.993176 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-47c30ecf-87a2-42ad-b4aa-5c2d6303f3ba tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:23.994078 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-47c30ecf-87a2-42ad-b4aa-5c2d6303f3ba tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:23.994653 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-47c30ecf-87a2-42ad-b4aa-5c2d6303f3ba tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:24.102633 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-240dce0e-a356-4aea-8f97-b90880176473 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '027dcc48e4fd4c36a5004096ad5e43f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dad823e240149edb30e592380249a13', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:27:24.112931 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-240dce0e-a356-4aea-8f97-b90880176473 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "GET /compute/v2.1/servers/a45c5bbd-ab66-426a-8e31-f471ce68606f" status: 200 len: 1510 microversion: 2.1 time: 0.351164 Jul 16 21:27:24.113602 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2002/4014] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:27:23 2022] GET /compute/v2.1/servers/a45c5bbd-ab66-426a-8e31-f471ce68606f => generated 1510 bytes in 353 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:27:24.118974 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-5e67ec8a-d9a6-48f9-b9d2-d04f65b78967 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:27:24.122323 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-5e67ec8a-d9a6-48f9-b9d2-d04f65b78967 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Policy check for os_compute_api:servers:allow_all_filters failed with credentials {'is_admin': False, 'user_id': '029c76a0ba614b5880128c4309902c67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3b214f5044c7445592e177835a5d18fc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:27:24.122989 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-5e67ec8a-d9a6-48f9-b9d2-d04f65b78967 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Searching by: {'name': 'tempest-ListServerFiltersTestJSON-instance-1583187080', 'deleted': False, 'project_id': '3b214f5044c7445592e177835a5d18fc'} {{(pid=138680) get_all /opt/stack/nova/nova/compute/api.py:2962}} Jul 16 21:27:24.129446 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5e67ec8a-d9a6-48f9-b9d2-d04f65b78967 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:24.129902 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5e67ec8a-d9a6-48f9-b9d2-d04f65b78967 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:24.137402 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5e67ec8a-d9a6-48f9-b9d2-d04f65b78967 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:24.137402 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5e67ec8a-d9a6-48f9-b9d2-d04f65b78967 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:24.225908 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.multi_cell_list [None req-5e67ec8a-d9a6-48f9-b9d2-d04f65b78967 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Listed batch of 1 results from cell out of 1000 limit. Returned 1 total so far. {{(pid=138680) do_query /opt/stack/nova/nova/compute/multi_cell_list.py:373}} Jul 16 21:27:24.244925 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-47c30ecf-87a2-42ad-b4aa-5c2d6303f3ba tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '45315c12d53349ae8cd245cb138b8dcd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'aef73d309b204eeebfc2ce69b4e4a41b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:27:24.259887 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-47c30ecf-87a2-42ad-b4aa-5c2d6303f3ba tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '45315c12d53349ae8cd245cb138b8dcd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'aef73d309b204eeebfc2ce69b4e4a41b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:27:24.261838 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-47c30ecf-87a2-42ad-b4aa-5c2d6303f3ba tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '45315c12d53349ae8cd245cb138b8dcd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'aef73d309b204eeebfc2ce69b4e4a41b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:27:24.263116 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-47c30ecf-87a2-42ad-b4aa-5c2d6303f3ba tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] 10.222.0.16 "GET /compute/v2.1/servers/fbc2a45b-c662-475e-936d-70697843ce90" status: 200 len: 1359 microversion: 2.21 time: 0.285915 Jul 16 21:27:24.263697 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2013/4015] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:27:23 2022] GET /compute/v2.1/servers/fbc2a45b-c662-475e-936d-70697843ce90 => generated 1359 bytes in 287 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:27:24.288355 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-5e67ec8a-d9a6-48f9-b9d2-d04f65b78967 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] 10.222.0.16 "GET /compute/v2.1/servers?name=tempest-ListServerFiltersTestJSON-instance-1583187080" status: 200 len: 346 microversion: 2.1 time: 0.173624 Jul 16 21:27:24.288951 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2003/4016] 10.222.0.16 () {58 vars in 1241 bytes} [Sat Jul 16 21:27:24 2022] GET /compute/v2.1/servers?name=tempest-ListServerFiltersTestJSON-instance-1583187080 => generated 346 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:27:24.301155 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-757e5b8b-c2df-4490-9ee7-ca9c9444350f tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Action: 'action', calling method: >, body: {"os-stop": {}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:27:24.307858 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-757e5b8b-c2df-4490-9ee7-ca9c9444350f tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:24.308351 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-757e5b8b-c2df-4490-9ee7-ca9c9444350f tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:24.334543 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-757e5b8b-c2df-4490-9ee7-ca9c9444350f tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] [instance: cf38328b-40fb-4537-b524-aaeb0a0abfb4] Going to try to stop instance {{(pid=138681) force_stop /opt/stack/nova/nova/compute/api.py:2707}} Jul 16 21:27:24.577396 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-757e5b8b-c2df-4490-9ee7-ca9c9444350f tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] 10.222.0.16 "POST /compute/v2.1/servers/cf38328b-40fb-4537-b524-aaeb0a0abfb4/action" status: 202 len: 0 microversion: 2.1 time: 0.279987 Jul 16 21:27:24.578607 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2014/4017] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 21:27:24 2022] POST /compute/v2.1/servers/cf38328b-40fb-4537-b524-aaeb0a0abfb4/action => generated 0 bytes in 283 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Jul 16 21:27:24.588090 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-9410eadd-cb34-4a38-96b4-c9ddf25e42bb tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:27:24.589004 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-9410eadd-cb34-4a38-96b4-c9ddf25e42bb tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] [instance: cf38328b-40fb-4537-b524-aaeb0a0abfb4] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:27:24.597613 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9410eadd-cb34-4a38-96b4-c9ddf25e42bb tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:24.598503 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9410eadd-cb34-4a38-96b4-c9ddf25e42bb tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:24.599532 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9410eadd-cb34-4a38-96b4-c9ddf25e42bb tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:24.600815 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9410eadd-cb34-4a38-96b4-c9ddf25e42bb tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:24.843711 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-2babc49f-3153-4d01-9dab-aed275ce0297 req-ac249e51-eeff-49a5-9503-96b27f63f0c9 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "fbc2a45b-c662-475e-936d-70697843ce90", "tag": "6228d01b-0897-4125-96b9-bc789787346e"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:27:24.884257 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-2babc49f-3153-4d01-9dab-aed275ce0297 req-ac249e51-eeff-49a5-9503-96b27f63f0c9 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:24.891772 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-2babc49f-3153-4d01-9dab-aed275ce0297 req-ac249e51-eeff-49a5-9503-96b27f63f0c9 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:24.909700 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-2babc49f-3153-4d01-9dab-aed275ce0297 req-ac249e51-eeff-49a5-9503-96b27f63f0c9 service nova] Creating event network-changed:6228d01b-0897-4125-96b9-bc789787346e for instance fbc2a45b-c662-475e-936d-70697843ce90 on nch1-master-1 Jul 16 21:27:24.915254 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-2babc49f-3153-4d01-9dab-aed275ce0297 req-ac249e51-eeff-49a5-9503-96b27f63f0c9 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.076239 Jul 16 21:27:24.918227 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2015/4018] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:27:24 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 78 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:27:24.930467 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-9410eadd-cb34-4a38-96b4-c9ddf25e42bb tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '029c76a0ba614b5880128c4309902c67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3b214f5044c7445592e177835a5d18fc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:27:24.940310 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-9410eadd-cb34-4a38-96b4-c9ddf25e42bb tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] 10.222.0.16 "GET /compute/v2.1/servers/cf38328b-40fb-4537-b524-aaeb0a0abfb4" status: 200 len: 1521 microversion: 2.1 time: 0.355993 Jul 16 21:27:24.941315 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2004/4019] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:27:24 2022] GET /compute/v2.1/servers/cf38328b-40fb-4537-b524-aaeb0a0abfb4 => generated 1521 bytes in 358 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:27:25.044417 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-57b4c600-d0dd-47f0-93cd-c7e166985a47 tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:27:25.044417 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-57b4c600-d0dd-47f0-93cd-c7e166985a47 tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] [instance: 9a184cb7-2548-41d5-8d37-8e72dfb3d75a] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:27:25.072061 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-57b4c600-d0dd-47f0-93cd-c7e166985a47 tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:25.072061 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-57b4c600-d0dd-47f0-93cd-c7e166985a47 tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:25.072061 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-57b4c600-d0dd-47f0-93cd-c7e166985a47 tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:25.073424 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-57b4c600-d0dd-47f0-93cd-c7e166985a47 tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:25.125099 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-57b4c600-d0dd-47f0-93cd-c7e166985a47 tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] [instance: 9a184cb7-2548-41d5-8d37-8e72dfb3d75a] Going to try to terminate instance {{(pid=138681) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 16 21:27:25.130501 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-630c0462-6475-4edd-9888-b02de0b9306e tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:27:25.132045 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-630c0462-6475-4edd-9888-b02de0b9306e tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: a45c5bbd-ab66-426a-8e31-f471ce68606f] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:27:25.140194 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-630c0462-6475-4edd-9888-b02de0b9306e tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:25.140682 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-630c0462-6475-4edd-9888-b02de0b9306e tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:25.143623 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-630c0462-6475-4edd-9888-b02de0b9306e tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:25.143623 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-630c0462-6475-4edd-9888-b02de0b9306e tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:25.172861 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=100 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:27:25.388196 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-630c0462-6475-4edd-9888-b02de0b9306e tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '027dcc48e4fd4c36a5004096ad5e43f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dad823e240149edb30e592380249a13', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:27:25.396465 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-630c0462-6475-4edd-9888-b02de0b9306e tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "GET /compute/v2.1/servers/a45c5bbd-ab66-426a-8e31-f471ce68606f" status: 200 len: 1510 microversion: 2.1 time: 0.270298 Jul 16 21:27:25.397048 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2005/4020] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:27:25 2022] GET /compute/v2.1/servers/a45c5bbd-ab66-426a-8e31-f471ce68606f => generated 1510 bytes in 272 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:27:25.408956 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-6d65e6d0-30f7-4022-96ec-b80ad24e05f5 tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:27:25.409911 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-6d65e6d0-30f7-4022-96ec-b80ad24e05f5 tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] [instance: fbc2a45b-c662-475e-936d-70697843ce90] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:27:25.418772 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6d65e6d0-30f7-4022-96ec-b80ad24e05f5 tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:25.419319 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6d65e6d0-30f7-4022-96ec-b80ad24e05f5 tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:25.420603 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6d65e6d0-30f7-4022-96ec-b80ad24e05f5 tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:25.421217 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6d65e6d0-30f7-4022-96ec-b80ad24e05f5 tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:25.463107 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-57b4c600-d0dd-47f0-93cd-c7e166985a47 tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] 10.222.0.16 "DELETE /compute/v2.1/servers/9a184cb7-2548-41d5-8d37-8e72dfb3d75a" status: 204 len: 0 microversion: 2.1 time: 0.426585 Jul 16 21:27:25.463808 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2016/4021] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:27:25 2022] DELETE /compute/v2.1/servers/9a184cb7-2548-41d5-8d37-8e72dfb3d75a => generated 0 bytes in 429 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 16 21:27:25.473417 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-0d381b43-e6dd-499f-9403-d0819dc99de3 tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:27:25.473992 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-0d381b43-e6dd-499f-9403-d0819dc99de3 tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] [instance: 9a184cb7-2548-41d5-8d37-8e72dfb3d75a] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:27:25.480662 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0d381b43-e6dd-499f-9403-d0819dc99de3 tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:25.481112 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0d381b43-e6dd-499f-9403-d0819dc99de3 tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:25.481968 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0d381b43-e6dd-499f-9403-d0819dc99de3 tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:25.482335 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0d381b43-e6dd-499f-9403-d0819dc99de3 tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:25.641806 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-6d65e6d0-30f7-4022-96ec-b80ad24e05f5 tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '45315c12d53349ae8cd245cb138b8dcd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'aef73d309b204eeebfc2ce69b4e4a41b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:27:25.651626 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-6d65e6d0-30f7-4022-96ec-b80ad24e05f5 tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '45315c12d53349ae8cd245cb138b8dcd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'aef73d309b204eeebfc2ce69b4e4a41b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:27:25.653232 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-6d65e6d0-30f7-4022-96ec-b80ad24e05f5 tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '45315c12d53349ae8cd245cb138b8dcd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'aef73d309b204eeebfc2ce69b4e4a41b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:27:25.654438 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-6d65e6d0-30f7-4022-96ec-b80ad24e05f5 tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] 10.222.0.16 "GET /compute/v2.1/servers/fbc2a45b-c662-475e-936d-70697843ce90" status: 200 len: 1359 microversion: 2.21 time: 0.256305 Jul 16 21:27:25.655255 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2006/4022] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:27:25 2022] GET /compute/v2.1/servers/fbc2a45b-c662-475e-936d-70697843ce90 => generated 1359 bytes in 258 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:27:25.667507 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-0d381b43-e6dd-499f-9403-d0819dc99de3 tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb1f6532421745a4b0de70356748a647', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7d5a798ee507429bb70566356229535d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:27:25.675511 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-0d381b43-e6dd-499f-9403-d0819dc99de3 tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] 10.222.0.16 "GET /compute/v2.1/servers/9a184cb7-2548-41d5-8d37-8e72dfb3d75a" status: 200 len: 1469 microversion: 2.1 time: 0.207129 Jul 16 21:27:25.676082 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2017/4023] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:27:25 2022] GET /compute/v2.1/servers/9a184cb7-2548-41d5-8d37-8e72dfb3d75a => generated 1469 bytes in 208 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:27:25.958252 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-b89641fb-ce70-47a5-b1cf-34a155a99292 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:27:25.958862 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-b89641fb-ce70-47a5-b1cf-34a155a99292 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] [instance: cf38328b-40fb-4537-b524-aaeb0a0abfb4] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:27:25.965132 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b89641fb-ce70-47a5-b1cf-34a155a99292 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:25.965583 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b89641fb-ce70-47a5-b1cf-34a155a99292 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:25.966343 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b89641fb-ce70-47a5-b1cf-34a155a99292 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:25.966804 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b89641fb-ce70-47a5-b1cf-34a155a99292 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:26.105701 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-b89641fb-ce70-47a5-b1cf-34a155a99292 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '029c76a0ba614b5880128c4309902c67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3b214f5044c7445592e177835a5d18fc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:27:26.113915 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-b89641fb-ce70-47a5-b1cf-34a155a99292 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] 10.222.0.16 "GET /compute/v2.1/servers/cf38328b-40fb-4537-b524-aaeb0a0abfb4" status: 200 len: 1521 microversion: 2.1 time: 0.160231 Jul 16 21:27:26.115696 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2007/4024] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:27:25 2022] GET /compute/v2.1/servers/cf38328b-40fb-4537-b524-aaeb0a0abfb4 => generated 1521 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:27:26.414318 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-1999ab57-b59a-4283-82c6-ddeda5f4316c tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:27:26.415093 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-1999ab57-b59a-4283-82c6-ddeda5f4316c tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: a45c5bbd-ab66-426a-8e31-f471ce68606f] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:27:26.422335 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1999ab57-b59a-4283-82c6-ddeda5f4316c tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:26.422791 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1999ab57-b59a-4283-82c6-ddeda5f4316c tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:26.423600 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1999ab57-b59a-4283-82c6-ddeda5f4316c tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:26.427389 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1999ab57-b59a-4283-82c6-ddeda5f4316c tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:26.570329 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-1999ab57-b59a-4283-82c6-ddeda5f4316c tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '027dcc48e4fd4c36a5004096ad5e43f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dad823e240149edb30e592380249a13', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:27:26.577998 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-1999ab57-b59a-4283-82c6-ddeda5f4316c tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "GET /compute/v2.1/servers/a45c5bbd-ab66-426a-8e31-f471ce68606f" status: 200 len: 1510 microversion: 2.1 time: 0.167627 Jul 16 21:27:26.578574 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2018/4025] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:27:26 2022] GET /compute/v2.1/servers/a45c5bbd-ab66-426a-8e31-f471ce68606f => generated 1510 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:27:26.673728 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-ea7bb4ab-8632-47ab-966e-c8b057262fdf tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:27:26.675185 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-ea7bb4ab-8632-47ab-966e-c8b057262fdf tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] [instance: fbc2a45b-c662-475e-936d-70697843ce90] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:27:26.682434 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ea7bb4ab-8632-47ab-966e-c8b057262fdf tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:26.682917 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ea7bb4ab-8632-47ab-966e-c8b057262fdf tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:26.683927 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ea7bb4ab-8632-47ab-966e-c8b057262fdf tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:26.684376 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ea7bb4ab-8632-47ab-966e-c8b057262fdf tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:26.693001 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-e4e64305-ef51-4368-837c-4228f89e58d5 tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:27:26.693624 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-e4e64305-ef51-4368-837c-4228f89e58d5 tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] [instance: 9a184cb7-2548-41d5-8d37-8e72dfb3d75a] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:27:26.700039 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e4e64305-ef51-4368-837c-4228f89e58d5 tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:26.700519 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e4e64305-ef51-4368-837c-4228f89e58d5 tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:26.701293 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e4e64305-ef51-4368-837c-4228f89e58d5 tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:26.701736 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e4e64305-ef51-4368-837c-4228f89e58d5 tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:26.949267 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-ea7bb4ab-8632-47ab-966e-c8b057262fdf tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '45315c12d53349ae8cd245cb138b8dcd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'aef73d309b204eeebfc2ce69b4e4a41b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:27:26.961127 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-ea7bb4ab-8632-47ab-966e-c8b057262fdf tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '45315c12d53349ae8cd245cb138b8dcd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'aef73d309b204eeebfc2ce69b4e4a41b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:27:26.963606 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-ea7bb4ab-8632-47ab-966e-c8b057262fdf tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '45315c12d53349ae8cd245cb138b8dcd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'aef73d309b204eeebfc2ce69b4e4a41b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:27:26.969063 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-ea7bb4ab-8632-47ab-966e-c8b057262fdf tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] 10.222.0.16 "GET /compute/v2.1/servers/fbc2a45b-c662-475e-936d-70697843ce90" status: 200 len: 1359 microversion: 2.21 time: 0.297252 Jul 16 21:27:26.970444 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2008/4026] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:27:26 2022] GET /compute/v2.1/servers/fbc2a45b-c662-475e-936d-70697843ce90 => generated 1359 bytes in 301 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:27:26.983583 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-e4e64305-ef51-4368-837c-4228f89e58d5 tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb1f6532421745a4b0de70356748a647', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7d5a798ee507429bb70566356229535d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:27:26.991920 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-e4e64305-ef51-4368-837c-4228f89e58d5 tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] 10.222.0.16 "GET /compute/v2.1/servers/9a184cb7-2548-41d5-8d37-8e72dfb3d75a" status: 200 len: 1469 microversion: 2.1 time: 0.302966 Jul 16 21:27:26.992812 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2019/4027] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:27:26 2022] GET /compute/v2.1/servers/9a184cb7-2548-41d5-8d37-8e72dfb3d75a => generated 1469 bytes in 304 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:27:27.133112 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-3c03ef87-07a7-4afa-b99e-1a0b2d121e06 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:27:27.133739 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-3c03ef87-07a7-4afa-b99e-1a0b2d121e06 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] [instance: cf38328b-40fb-4537-b524-aaeb0a0abfb4] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:27:27.140431 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3c03ef87-07a7-4afa-b99e-1a0b2d121e06 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:27.140809 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3c03ef87-07a7-4afa-b99e-1a0b2d121e06 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:27.141598 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3c03ef87-07a7-4afa-b99e-1a0b2d121e06 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:27.142039 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3c03ef87-07a7-4afa-b99e-1a0b2d121e06 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:27.296809 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-3c03ef87-07a7-4afa-b99e-1a0b2d121e06 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '029c76a0ba614b5880128c4309902c67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3b214f5044c7445592e177835a5d18fc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:27:27.304859 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-3c03ef87-07a7-4afa-b99e-1a0b2d121e06 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] 10.222.0.16 "GET /compute/v2.1/servers/cf38328b-40fb-4537-b524-aaeb0a0abfb4" status: 200 len: 1521 microversion: 2.1 time: 0.177139 Jul 16 21:27:27.305450 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2009/4028] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:27:27 2022] GET /compute/v2.1/servers/cf38328b-40fb-4537-b524-aaeb0a0abfb4 => generated 1521 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:27:27.596415 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-52ec6997-631b-4051-b502-9da5e3aacaed tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:27:27.597192 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-52ec6997-631b-4051-b502-9da5e3aacaed tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: a45c5bbd-ab66-426a-8e31-f471ce68606f] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:27:27.606167 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-52ec6997-631b-4051-b502-9da5e3aacaed tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:27.606693 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-52ec6997-631b-4051-b502-9da5e3aacaed tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:27.607509 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-52ec6997-631b-4051-b502-9da5e3aacaed tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:27.607952 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-52ec6997-631b-4051-b502-9da5e3aacaed tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:27.796751 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-52ec6997-631b-4051-b502-9da5e3aacaed tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '027dcc48e4fd4c36a5004096ad5e43f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dad823e240149edb30e592380249a13', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:27:27.806352 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-52ec6997-631b-4051-b502-9da5e3aacaed tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "GET /compute/v2.1/servers/a45c5bbd-ab66-426a-8e31-f471ce68606f" status: 200 len: 1510 microversion: 2.1 time: 0.214887 Jul 16 21:27:27.807044 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2020/4029] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:27:27 2022] GET /compute/v2.1/servers/a45c5bbd-ab66-426a-8e31-f471ce68606f => generated 1510 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:27:27.992802 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-abd401b5-a221-4d50-8426-5d609cc0df18 tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:27:27.995445 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-abd401b5-a221-4d50-8426-5d609cc0df18 tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] [instance: fbc2a45b-c662-475e-936d-70697843ce90] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:27:28.009091 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-abd401b5-a221-4d50-8426-5d609cc0df18 tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:28.009967 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-abd401b5-a221-4d50-8426-5d609cc0df18 tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:28.010084 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-4546a2f0-5383-455d-9948-691ac9378d32 tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:27:28.010638 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-4546a2f0-5383-455d-9948-691ac9378d32 tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] [instance: 9a184cb7-2548-41d5-8d37-8e72dfb3d75a] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:27:28.011633 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-abd401b5-a221-4d50-8426-5d609cc0df18 tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:28.012190 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-abd401b5-a221-4d50-8426-5d609cc0df18 tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:28.018311 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4546a2f0-5383-455d-9948-691ac9378d32 tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:28.019086 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4546a2f0-5383-455d-9948-691ac9378d32 tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:28.020143 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4546a2f0-5383-455d-9948-691ac9378d32 tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:28.020797 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4546a2f0-5383-455d-9948-691ac9378d32 tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:28.398591 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-abd401b5-a221-4d50-8426-5d609cc0df18 tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '45315c12d53349ae8cd245cb138b8dcd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'aef73d309b204eeebfc2ce69b4e4a41b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:27:28.414835 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-4546a2f0-5383-455d-9948-691ac9378d32 tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb1f6532421745a4b0de70356748a647', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7d5a798ee507429bb70566356229535d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:27:28.418143 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-abd401b5-a221-4d50-8426-5d609cc0df18 tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '45315c12d53349ae8cd245cb138b8dcd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'aef73d309b204eeebfc2ce69b4e4a41b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:27:28.420995 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-abd401b5-a221-4d50-8426-5d609cc0df18 tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '45315c12d53349ae8cd245cb138b8dcd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'aef73d309b204eeebfc2ce69b4e4a41b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:27:28.422872 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-abd401b5-a221-4d50-8426-5d609cc0df18 tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] 10.222.0.16 "GET /compute/v2.1/servers/fbc2a45b-c662-475e-936d-70697843ce90" status: 200 len: 1359 microversion: 2.21 time: 0.437227 Jul 16 21:27:28.428033 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2010/4030] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:27:27 2022] GET /compute/v2.1/servers/fbc2a45b-c662-475e-936d-70697843ce90 => generated 1359 bytes in 443 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:27:28.431475 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-4546a2f0-5383-455d-9948-691ac9378d32 tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] 10.222.0.16 "GET /compute/v2.1/servers/9a184cb7-2548-41d5-8d37-8e72dfb3d75a" status: 200 len: 1469 microversion: 2.1 time: 0.422962 Jul 16 21:27:28.443160 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2021/4031] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:27:28 2022] GET /compute/v2.1/servers/9a184cb7-2548-41d5-8d37-8e72dfb3d75a => generated 1469 bytes in 438 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:27:28.445799 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-4c8e512f-1061-40ab-8356-32d3b540cb39 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:27:28.446668 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-4c8e512f-1061-40ab-8356-32d3b540cb39 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] [instance: cf38328b-40fb-4537-b524-aaeb0a0abfb4] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:27:28.458126 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4c8e512f-1061-40ab-8356-32d3b540cb39 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:28.461865 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4c8e512f-1061-40ab-8356-32d3b540cb39 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:28.464327 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4c8e512f-1061-40ab-8356-32d3b540cb39 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:28.465280 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4c8e512f-1061-40ab-8356-32d3b540cb39 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:28.807661 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-4c8e512f-1061-40ab-8356-32d3b540cb39 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '029c76a0ba614b5880128c4309902c67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3b214f5044c7445592e177835a5d18fc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:27:28.818077 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-4c8e512f-1061-40ab-8356-32d3b540cb39 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] 10.222.0.16 "GET /compute/v2.1/servers/cf38328b-40fb-4537-b524-aaeb0a0abfb4" status: 200 len: 1521 microversion: 2.1 time: 0.387782 Jul 16 21:27:28.824103 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2011/4032] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:27:28 2022] GET /compute/v2.1/servers/cf38328b-40fb-4537-b524-aaeb0a0abfb4 => generated 1521 bytes in 395 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:27:28.826118 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-446175bd-9f5f-4d41-ba68-f1c040dc9917 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:27:28.826840 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-446175bd-9f5f-4d41-ba68-f1c040dc9917 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: a45c5bbd-ab66-426a-8e31-f471ce68606f] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:27:28.836680 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-446175bd-9f5f-4d41-ba68-f1c040dc9917 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:28.839745 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-446175bd-9f5f-4d41-ba68-f1c040dc9917 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:28.839745 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-446175bd-9f5f-4d41-ba68-f1c040dc9917 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:28.839745 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-446175bd-9f5f-4d41-ba68-f1c040dc9917 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:28.856742 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=95,nova_cell1:INSERT=3,nova_cell1:UPDATE=2 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:27:29.418060 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-df807033-b69b-41c6-afdf-1e69b8c9e989 req-792b1800-a36c-4d50-b2b0-54b950e0fa46 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "9a184cb7-2548-41d5-8d37-8e72dfb3d75a", "name": "network-vif-deleted", "tag": "80734127-7d61-49c9-8895-1044cdd4e336"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:27:29.433611 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-df807033-b69b-41c6-afdf-1e69b8c9e989 req-792b1800-a36c-4d50-b2b0-54b950e0fa46 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:29.434293 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-df807033-b69b-41c6-afdf-1e69b8c9e989 req-792b1800-a36c-4d50-b2b0-54b950e0fa46 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:29.446313 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-df807033-b69b-41c6-afdf-1e69b8c9e989 req-792b1800-a36c-4d50-b2b0-54b950e0fa46 service nova] Creating event network-vif-deleted:80734127-7d61-49c9-8895-1044cdd4e336 for instance 9a184cb7-2548-41d5-8d37-8e72dfb3d75a on nch1-master-1 Jul 16 21:27:29.452211 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-df807033-b69b-41c6-afdf-1e69b8c9e989 req-792b1800-a36c-4d50-b2b0-54b950e0fa46 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.037727 Jul 16 21:27:29.453328 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2012/4033] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:27:29 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:27:29.460693 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-7dd5807c-36e0-4b64-acff-10dc77691df1 tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:27:29.463360 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-446175bd-9f5f-4d41-ba68-f1c040dc9917 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '027dcc48e4fd4c36a5004096ad5e43f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dad823e240149edb30e592380249a13', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:27:29.464556 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-7dd5807c-36e0-4b64-acff-10dc77691df1 tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] [instance: fbc2a45b-c662-475e-936d-70697843ce90] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:27:29.472517 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7dd5807c-36e0-4b64-acff-10dc77691df1 tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:29.473212 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7dd5807c-36e0-4b64-acff-10dc77691df1 tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:29.474027 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7dd5807c-36e0-4b64-acff-10dc77691df1 tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:29.474750 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7dd5807c-36e0-4b64-acff-10dc77691df1 tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:29.474863 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-446175bd-9f5f-4d41-ba68-f1c040dc9917 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "GET /compute/v2.1/servers/a45c5bbd-ab66-426a-8e31-f471ce68606f" status: 200 len: 1510 microversion: 2.1 time: 0.653854 Jul 16 21:27:29.475625 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2022/4034] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:27:28 2022] GET /compute/v2.1/servers/a45c5bbd-ab66-426a-8e31-f471ce68606f => generated 1510 bytes in 656 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:27:29.484492 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-f36c469f-37a1-4236-be41-1905760e05f2 tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:27:29.484492 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-f36c469f-37a1-4236-be41-1905760e05f2 tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] [instance: 9a184cb7-2548-41d5-8d37-8e72dfb3d75a] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:27:29.488595 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f36c469f-37a1-4236-be41-1905760e05f2 tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:29.489061 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f36c469f-37a1-4236-be41-1905760e05f2 tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:29.489845 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f36c469f-37a1-4236-be41-1905760e05f2 tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:29.490286 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f36c469f-37a1-4236-be41-1905760e05f2 tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:29.578560 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-f36c469f-37a1-4236-be41-1905760e05f2 tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb1f6532421745a4b0de70356748a647', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7d5a798ee507429bb70566356229535d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:27:29.592326 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-f36c469f-37a1-4236-be41-1905760e05f2 tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] 10.222.0.16 "GET /compute/v2.1/servers/9a184cb7-2548-41d5-8d37-8e72dfb3d75a" status: 200 len: 1427 microversion: 2.1 time: 0.115585 Jul 16 21:27:29.593178 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2023/4035] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:27:29 2022] GET /compute/v2.1/servers/9a184cb7-2548-41d5-8d37-8e72dfb3d75a => generated 1427 bytes in 117 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:27:29.655162 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-7dd5807c-36e0-4b64-acff-10dc77691df1 tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '45315c12d53349ae8cd245cb138b8dcd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'aef73d309b204eeebfc2ce69b4e4a41b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:27:29.675235 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-7dd5807c-36e0-4b64-acff-10dc77691df1 tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '45315c12d53349ae8cd245cb138b8dcd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'aef73d309b204eeebfc2ce69b4e4a41b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:27:29.677818 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-7dd5807c-36e0-4b64-acff-10dc77691df1 tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '45315c12d53349ae8cd245cb138b8dcd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'aef73d309b204eeebfc2ce69b4e4a41b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:27:29.679330 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-7dd5807c-36e0-4b64-acff-10dc77691df1 tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] 10.222.0.16 "GET /compute/v2.1/servers/fbc2a45b-c662-475e-936d-70697843ce90" status: 200 len: 1359 microversion: 2.21 time: 0.224544 Jul 16 21:27:29.679983 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2013/4036] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:27:29 2022] GET /compute/v2.1/servers/fbc2a45b-c662-475e-936d-70697843ce90 => generated 1359 bytes in 226 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:27:29.845857 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-926aa264-c067-4a03-ac6e-3deb7d13d40d tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:27:29.846643 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-926aa264-c067-4a03-ac6e-3deb7d13d40d tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] [instance: cf38328b-40fb-4537-b524-aaeb0a0abfb4] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:27:29.862480 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-926aa264-c067-4a03-ac6e-3deb7d13d40d tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:29.862942 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-926aa264-c067-4a03-ac6e-3deb7d13d40d tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:29.863869 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-926aa264-c067-4a03-ac6e-3deb7d13d40d tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:29.864406 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-926aa264-c067-4a03-ac6e-3deb7d13d40d tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:30.037480 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-926aa264-c067-4a03-ac6e-3deb7d13d40d tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '029c76a0ba614b5880128c4309902c67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3b214f5044c7445592e177835a5d18fc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:27:30.045745 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-926aa264-c067-4a03-ac6e-3deb7d13d40d tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] 10.222.0.16 "GET /compute/v2.1/servers/cf38328b-40fb-4537-b524-aaeb0a0abfb4" status: 200 len: 1521 microversion: 2.1 time: 0.208258 Jul 16 21:27:30.046575 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2024/4037] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:27:29 2022] GET /compute/v2.1/servers/cf38328b-40fb-4537-b524-aaeb0a0abfb4 => generated 1521 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:27:30.492826 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-cb8fae78-7fbf-4597-9166-31722243ea59 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:27:30.493441 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-cb8fae78-7fbf-4597-9166-31722243ea59 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: a45c5bbd-ab66-426a-8e31-f471ce68606f] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:27:30.501113 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cb8fae78-7fbf-4597-9166-31722243ea59 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:30.501579 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cb8fae78-7fbf-4597-9166-31722243ea59 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:30.502426 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cb8fae78-7fbf-4597-9166-31722243ea59 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:30.502869 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cb8fae78-7fbf-4597-9166-31722243ea59 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:30.616892 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-bea91234-8f19-4957-9a80-83a2dd6c7b2f tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:27:30.617556 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-bea91234-8f19-4957-9a80-83a2dd6c7b2f tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] [instance: 9a184cb7-2548-41d5-8d37-8e72dfb3d75a] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:27:30.626353 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-bea91234-8f19-4957-9a80-83a2dd6c7b2f tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:30.627519 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-bea91234-8f19-4957-9a80-83a2dd6c7b2f tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:30.628323 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-bea91234-8f19-4957-9a80-83a2dd6c7b2f tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:30.628846 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-bea91234-8f19-4957-9a80-83a2dd6c7b2f tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:30.671210 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-bea91234-8f19-4957-9a80-83a2dd6c7b2f tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:30.672002 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-bea91234-8f19-4957-9a80-83a2dd6c7b2f tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:30.672677 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.objects.instance [None req-bea91234-8f19-4957-9a80-83a2dd6c7b2f tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] Lazy-loading 'fault' on Instance uuid 9a184cb7-2548-41d5-8d37-8e72dfb3d75a {{(pid=138681) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:27:30.756094 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-bea91234-8f19-4957-9a80-83a2dd6c7b2f tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bb1f6532421745a4b0de70356748a647', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7d5a798ee507429bb70566356229535d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:27:30.772475 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-bea91234-8f19-4957-9a80-83a2dd6c7b2f tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] 10.222.0.16 "GET /compute/v2.1/servers/9a184cb7-2548-41d5-8d37-8e72dfb3d75a" status: 200 len: 1304 microversion: 2.1 time: 0.165766 Jul 16 21:27:30.785868 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2025/4038] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:27:30 2022] GET /compute/v2.1/servers/9a184cb7-2548-41d5-8d37-8e72dfb3d75a => generated 1304 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:27:30.791410 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-e9b9b4f8-dfb5-497e-a9e3-66eae4bd45fa tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:27:30.791962 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-cb8fae78-7fbf-4597-9166-31722243ea59 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '027dcc48e4fd4c36a5004096ad5e43f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dad823e240149edb30e592380249a13', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:27:30.792520 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-e9b9b4f8-dfb5-497e-a9e3-66eae4bd45fa tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] [instance: fbc2a45b-c662-475e-936d-70697843ce90] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:27:30.800410 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e9b9b4f8-dfb5-497e-a9e3-66eae4bd45fa tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:30.801129 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e9b9b4f8-dfb5-497e-a9e3-66eae4bd45fa tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:30.802066 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e9b9b4f8-dfb5-497e-a9e3-66eae4bd45fa tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:30.802732 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e9b9b4f8-dfb5-497e-a9e3-66eae4bd45fa tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:30.805565 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-cb8fae78-7fbf-4597-9166-31722243ea59 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "GET /compute/v2.1/servers/a45c5bbd-ab66-426a-8e31-f471ce68606f" status: 200 len: 1510 microversion: 2.1 time: 0.315633 Jul 16 21:27:30.806997 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2014/4039] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:27:30 2022] GET /compute/v2.1/servers/a45c5bbd-ab66-426a-8e31-f471ce68606f => generated 1510 bytes in 318 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:27:31.014144 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-e9b9b4f8-dfb5-497e-a9e3-66eae4bd45fa tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '45315c12d53349ae8cd245cb138b8dcd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'aef73d309b204eeebfc2ce69b4e4a41b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:27:31.026655 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-e9b9b4f8-dfb5-497e-a9e3-66eae4bd45fa tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '45315c12d53349ae8cd245cb138b8dcd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'aef73d309b204eeebfc2ce69b4e4a41b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:27:31.028475 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-e9b9b4f8-dfb5-497e-a9e3-66eae4bd45fa tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '45315c12d53349ae8cd245cb138b8dcd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'aef73d309b204eeebfc2ce69b4e4a41b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:27:31.029897 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-e9b9b4f8-dfb5-497e-a9e3-66eae4bd45fa tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] 10.222.0.16 "GET /compute/v2.1/servers/fbc2a45b-c662-475e-936d-70697843ce90" status: 200 len: 1359 microversion: 2.21 time: 0.242827 Jul 16 21:27:31.030584 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2026/4040] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:27:30 2022] GET /compute/v2.1/servers/fbc2a45b-c662-475e-936d-70697843ce90 => generated 1359 bytes in 244 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:27:31.063594 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-ad40c46a-6ec9-4462-8750-464819ea4817 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:27:31.064213 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-ad40c46a-6ec9-4462-8750-464819ea4817 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] [instance: cf38328b-40fb-4537-b524-aaeb0a0abfb4] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:27:31.071197 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ad40c46a-6ec9-4462-8750-464819ea4817 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:31.076773 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ad40c46a-6ec9-4462-8750-464819ea4817 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:31.076773 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ad40c46a-6ec9-4462-8750-464819ea4817 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:31.076773 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ad40c46a-6ec9-4462-8750-464819ea4817 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:31.295312 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-ad40c46a-6ec9-4462-8750-464819ea4817 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '029c76a0ba614b5880128c4309902c67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3b214f5044c7445592e177835a5d18fc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:27:31.308034 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-ad40c46a-6ec9-4462-8750-464819ea4817 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] 10.222.0.16 "GET /compute/v2.1/servers/cf38328b-40fb-4537-b524-aaeb0a0abfb4" status: 200 len: 1521 microversion: 2.1 time: 0.248115 Jul 16 21:27:31.308680 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2015/4041] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:27:31 2022] GET /compute/v2.1/servers/cf38328b-40fb-4537-b524-aaeb0a0abfb4 => generated 1521 bytes in 250 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:27:31.791861 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-e8b83250-fb1d-4ebc-bb9c-5cf4a0d535f6 tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:27:31.792477 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-e8b83250-fb1d-4ebc-bb9c-5cf4a0d535f6 tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] [instance: 9a184cb7-2548-41d5-8d37-8e72dfb3d75a] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:27:31.798970 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e8b83250-fb1d-4ebc-bb9c-5cf4a0d535f6 tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:31.799621 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e8b83250-fb1d-4ebc-bb9c-5cf4a0d535f6 tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:31.800393 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e8b83250-fb1d-4ebc-bb9c-5cf4a0d535f6 tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:31.800833 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e8b83250-fb1d-4ebc-bb9c-5cf4a0d535f6 tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:31.816401 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-e8b83250-fb1d-4ebc-bb9c-5cf4a0d535f6 tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] HTTP exception thrown: Instance 9a184cb7-2548-41d5-8d37-8e72dfb3d75a could not be found. Jul 16 21:27:31.818068 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-e8b83250-fb1d-4ebc-bb9c-5cf4a0d535f6 tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] Request method failure captured: Jul 16 21:27:31.818068 ncdv-master-1 devstack@n-api.service[138681]: request: GET /compute/v2.1/servers/9a184cb7-2548-41d5-8d37-8e72dfb3d75a HTTP/1.1 Jul 16 21:27:31.818068 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:27:31.818068 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:27:31.818068 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:27:31.818068 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 21:27:31.818068 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:27:31.818068 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:27:31.818068 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:27:31.818068 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:27:31.818068 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0yz71xK8z5O3nmhJc355Mb7UtRl2x9ZQFLMf0ZlnaKg2uxaNptCUkED88rNZw-Fy6iCDHIoropz_UzOaIG2p1kFWfYUn1b3GysZS0OG0aq3SYtXMcrT3LHDd3np0MY_PA_Ey2GdNf4bn6wVzUdb2__sQlpH0aU1VPTOwtRcKfc_zjWk Jul 16 21:27:31.818068 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:27:31.818068 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:27:31.818068 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:27:31.818068 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:27:31.818068 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:27:31.818068 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:27:31.818068 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: 7d5a798ee507429bb70566356229535d Jul 16 21:27:31.818068 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-VolumesBackupsTest-1570956956 Jul 16 21:27:31.818068 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:27:31.818068 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:27:31.818068 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/7d5a798ee507429bb70566356229535d"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/7d5a798ee507429bb70566356229535d"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_7d5a798ee507429bb70566356229535d"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/7d5a798ee507429bb70566356229535d"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:27:31.818068 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-VolumesBackupsTest-1570956956 Jul 16 21:27:31.818068 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: 7d5a798ee507429bb70566356229535d Jul 16 21:27:31.818068 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-VolumesBackupsTest-1570956956 Jul 16 21:27:31.818068 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-VolumesBackupsTest-1570956956-project Jul 16 21:27:31.818068 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:27:31.818068 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:27:31.818068 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: bb1f6532421745a4b0de70356748a647 Jul 16 21:27:31.818068 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-VolumesBackupsTest-1570956956-project Jul 16 21:27:31.818068 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:27:31.818068 ncdv-master-1 devstack@n-api.service[138681]: exception: Instance 9a184cb7-2548-41d5-8d37-8e72dfb3d75a could not be found. Jul 16 21:27:31.818068 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': '9a184cb7-2548-41d5-8d37-8e72dfb3d75a'} Jul 16 21:27:31.818068 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:27:31.818068 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:27:31.818068 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:27:31.818068 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:27:31.818068 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:27:31.818068 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:27:31.818068 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:27:31.818068 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:27:31.818068 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:27:31.818068 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:27:31.818068 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 9a184cb7-2548-41d5-8d37-8e72dfb3d75a could not be found. Jul 16 21:27:31.818068 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:27:31.818068 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:27:31.818068 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:27:31.818068 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:27:31.818068 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:27:31.818068 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:27:31.818068 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:27:31.818068 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:27:31.818068 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:27:31.818068 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:27:31.818068 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:27:31.818068 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:27:31.818068 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:27:31.818068 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:27:31.818068 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:27:31.818068 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:27:31.818068 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 9a184cb7-2548-41d5-8d37-8e72dfb3d75a could not be found. Jul 16 21:27:31.818068 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:27:31.818068 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:27:31.818068 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:27:31.821791 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:27:31.821791 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:27:31.821791 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:27:31.821791 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:27:31.821791 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:27:31.821791 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 9a184cb7-2548-41d5-8d37-8e72dfb3d75a could not be found. Jul 16 21:27:31.821791 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:27:31.821791 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-e8b83250-fb1d-4ebc-bb9c-5cf4a0d535f6 tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] Returning 404 to user: Instance 9a184cb7-2548-41d5-8d37-8e72dfb3d75a could not be found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:27:31.821791 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-e8b83250-fb1d-4ebc-bb9c-5cf4a0d535f6 tempest-VolumesBackupsTest-1570956956 tempest-VolumesBackupsTest-1570956956-project] 10.222.0.16 "GET /compute/v2.1/servers/9a184cb7-2548-41d5-8d37-8e72dfb3d75a" status: 404 len: 111 microversion: 2.1 time: 0.032492 Jul 16 21:27:31.821791 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2027/4042] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:27:31 2022] GET /compute/v2.1/servers/9a184cb7-2548-41d5-8d37-8e72dfb3d75a => generated 111 bytes in 34 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:27:31.824612 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-7102d8d9-36e6-4601-9cc8-5dfc60ab34cc tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:27:31.825240 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-7102d8d9-36e6-4601-9cc8-5dfc60ab34cc tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: a45c5bbd-ab66-426a-8e31-f471ce68606f] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:27:31.832747 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7102d8d9-36e6-4601-9cc8-5dfc60ab34cc tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:31.833204 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7102d8d9-36e6-4601-9cc8-5dfc60ab34cc tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:31.834120 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7102d8d9-36e6-4601-9cc8-5dfc60ab34cc tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:31.834567 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7102d8d9-36e6-4601-9cc8-5dfc60ab34cc tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:32.051429 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-e676d820-9932-459d-a515-4cd435598325 tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:27:32.053417 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-e676d820-9932-459d-a515-4cd435598325 tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] [instance: fbc2a45b-c662-475e-936d-70697843ce90] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:27:32.054598 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-7102d8d9-36e6-4601-9cc8-5dfc60ab34cc tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '027dcc48e4fd4c36a5004096ad5e43f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dad823e240149edb30e592380249a13', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:27:32.062550 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e676d820-9932-459d-a515-4cd435598325 tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:32.062940 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-7102d8d9-36e6-4601-9cc8-5dfc60ab34cc tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "GET /compute/v2.1/servers/a45c5bbd-ab66-426a-8e31-f471ce68606f" status: 200 len: 1510 microversion: 2.1 time: 0.243745 Jul 16 21:27:32.063519 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e676d820-9932-459d-a515-4cd435598325 tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:32.065012 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e676d820-9932-459d-a515-4cd435598325 tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:32.065689 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e676d820-9932-459d-a515-4cd435598325 tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:32.073465 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2016/4043] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:27:31 2022] GET /compute/v2.1/servers/a45c5bbd-ab66-426a-8e31-f471ce68606f => generated 1510 bytes in 255 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:27:32.238069 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-e676d820-9932-459d-a515-4cd435598325 tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '45315c12d53349ae8cd245cb138b8dcd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'aef73d309b204eeebfc2ce69b4e4a41b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:27:32.249821 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-e676d820-9932-459d-a515-4cd435598325 tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '45315c12d53349ae8cd245cb138b8dcd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'aef73d309b204eeebfc2ce69b4e4a41b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:27:32.251469 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-e676d820-9932-459d-a515-4cd435598325 tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '45315c12d53349ae8cd245cb138b8dcd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'aef73d309b204eeebfc2ce69b4e4a41b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:27:32.253037 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-e676d820-9932-459d-a515-4cd435598325 tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] 10.222.0.16 "GET /compute/v2.1/servers/fbc2a45b-c662-475e-936d-70697843ce90" status: 200 len: 1359 microversion: 2.21 time: 0.207140 Jul 16 21:27:32.253959 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2028/4044] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:27:32 2022] GET /compute/v2.1/servers/fbc2a45b-c662-475e-936d-70697843ce90 => generated 1359 bytes in 209 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:27:32.325721 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-cd2167d4-88b2-4315-ad1c-be1ec52138b6 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:27:32.326339 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-cd2167d4-88b2-4315-ad1c-be1ec52138b6 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] [instance: cf38328b-40fb-4537-b524-aaeb0a0abfb4] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:27:32.339469 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cd2167d4-88b2-4315-ad1c-be1ec52138b6 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:32.340329 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cd2167d4-88b2-4315-ad1c-be1ec52138b6 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:32.341319 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cd2167d4-88b2-4315-ad1c-be1ec52138b6 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:32.341905 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cd2167d4-88b2-4315-ad1c-be1ec52138b6 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:32.517054 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-cd2167d4-88b2-4315-ad1c-be1ec52138b6 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '029c76a0ba614b5880128c4309902c67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3b214f5044c7445592e177835a5d18fc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:27:32.525835 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-cd2167d4-88b2-4315-ad1c-be1ec52138b6 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] 10.222.0.16 "GET /compute/v2.1/servers/cf38328b-40fb-4537-b524-aaeb0a0abfb4" status: 200 len: 1521 microversion: 2.1 time: 0.203934 Jul 16 21:27:32.526427 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2017/4045] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:27:32 2022] GET /compute/v2.1/servers/cf38328b-40fb-4537-b524-aaeb0a0abfb4 => generated 1521 bytes in 205 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:27:32.760085 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-8e64bce0-740c-4dd6-bdaa-181222952255 req-3162f5b6-88ba-4446-99ec-bea2c4b18840 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "fbc2a45b-c662-475e-936d-70697843ce90", "name": "network-vif-plugged", "status": "completed", "tag": "6228d01b-0897-4125-96b9-bc789787346e"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:27:32.766977 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-8e64bce0-740c-4dd6-bdaa-181222952255 req-3162f5b6-88ba-4446-99ec-bea2c4b18840 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:32.767507 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-8e64bce0-740c-4dd6-bdaa-181222952255 req-3162f5b6-88ba-4446-99ec-bea2c4b18840 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:32.785477 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-8e64bce0-740c-4dd6-bdaa-181222952255 req-3162f5b6-88ba-4446-99ec-bea2c4b18840 service nova] Creating event network-vif-plugged:6228d01b-0897-4125-96b9-bc789787346e for instance fbc2a45b-c662-475e-936d-70697843ce90 on nch1-master-1 Jul 16 21:27:32.790816 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-8e64bce0-740c-4dd6-bdaa-181222952255 req-3162f5b6-88ba-4446-99ec-bea2c4b18840 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.036034 Jul 16 21:27:32.791462 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2029/4046] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:27:32 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:27:33.080843 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-00955726-5472-4b9e-a44d-48c763c71751 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:27:33.082051 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-00955726-5472-4b9e-a44d-48c763c71751 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: a45c5bbd-ab66-426a-8e31-f471ce68606f] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:27:33.087834 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell0:SELECT=6 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:27:33.089946 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_api:SELECT=100 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:27:33.091477 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-00955726-5472-4b9e-a44d-48c763c71751 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:33.091934 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-00955726-5472-4b9e-a44d-48c763c71751 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:33.092722 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-00955726-5472-4b9e-a44d-48c763c71751 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:33.093160 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-00955726-5472-4b9e-a44d-48c763c71751 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:33.237696 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-00955726-5472-4b9e-a44d-48c763c71751 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '027dcc48e4fd4c36a5004096ad5e43f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dad823e240149edb30e592380249a13', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:27:33.249831 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-00955726-5472-4b9e-a44d-48c763c71751 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "GET /compute/v2.1/servers/a45c5bbd-ab66-426a-8e31-f471ce68606f" status: 200 len: 1510 microversion: 2.1 time: 0.173360 Jul 16 21:27:33.251097 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2018/4047] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:27:33 2022] GET /compute/v2.1/servers/a45c5bbd-ab66-426a-8e31-f471ce68606f => generated 1510 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:27:33.272917 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-32e5b152-b63d-46dd-8941-779942264081 tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:27:33.273741 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-32e5b152-b63d-46dd-8941-779942264081 tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] [instance: fbc2a45b-c662-475e-936d-70697843ce90] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:27:33.279932 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell0:SELECT=12 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:27:33.282788 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-32e5b152-b63d-46dd-8941-779942264081 tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:33.284011 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-32e5b152-b63d-46dd-8941-779942264081 tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:33.285033 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-32e5b152-b63d-46dd-8941-779942264081 tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:33.285551 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-32e5b152-b63d-46dd-8941-779942264081 tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:33.433636 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-32e5b152-b63d-46dd-8941-779942264081 tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '45315c12d53349ae8cd245cb138b8dcd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'aef73d309b204eeebfc2ce69b4e4a41b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:27:33.442917 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-32e5b152-b63d-46dd-8941-779942264081 tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '45315c12d53349ae8cd245cb138b8dcd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'aef73d309b204eeebfc2ce69b4e4a41b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:27:33.444476 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-32e5b152-b63d-46dd-8941-779942264081 tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '45315c12d53349ae8cd245cb138b8dcd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'aef73d309b204eeebfc2ce69b4e4a41b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:27:33.445635 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-32e5b152-b63d-46dd-8941-779942264081 tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] 10.222.0.16 "GET /compute/v2.1/servers/fbc2a45b-c662-475e-936d-70697843ce90" status: 200 len: 1359 microversion: 2.21 time: 0.178113 Jul 16 21:27:33.446207 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2030/4048] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:27:33 2022] GET /compute/v2.1/servers/fbc2a45b-c662-475e-936d-70697843ce90 => generated 1359 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:27:33.543709 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-3b08db3d-5450-4b14-b95a-12992601ab30 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:27:33.544313 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-3b08db3d-5450-4b14-b95a-12992601ab30 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] [instance: cf38328b-40fb-4537-b524-aaeb0a0abfb4] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:27:33.551853 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3b08db3d-5450-4b14-b95a-12992601ab30 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:33.552668 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3b08db3d-5450-4b14-b95a-12992601ab30 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:33.553642 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3b08db3d-5450-4b14-b95a-12992601ab30 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:33.554301 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3b08db3d-5450-4b14-b95a-12992601ab30 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:33.773504 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-3b08db3d-5450-4b14-b95a-12992601ab30 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '029c76a0ba614b5880128c4309902c67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3b214f5044c7445592e177835a5d18fc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:27:33.782575 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-3b08db3d-5450-4b14-b95a-12992601ab30 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] 10.222.0.16 "GET /compute/v2.1/servers/cf38328b-40fb-4537-b524-aaeb0a0abfb4" status: 200 len: 1521 microversion: 2.1 time: 0.242794 Jul 16 21:27:33.784206 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2019/4049] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:27:33 2022] GET /compute/v2.1/servers/cf38328b-40fb-4537-b524-aaeb0a0abfb4 => generated 1521 bytes in 245 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:27:34.269070 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-1279ed17-b79b-4288-b133-ba0917f5d486 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:27:34.271890 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-1279ed17-b79b-4288-b133-ba0917f5d486 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: a45c5bbd-ab66-426a-8e31-f471ce68606f] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:27:34.274212 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell0:SELECT=10 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:27:34.297769 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1279ed17-b79b-4288-b133-ba0917f5d486 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:34.298233 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1279ed17-b79b-4288-b133-ba0917f5d486 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:34.299276 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1279ed17-b79b-4288-b133-ba0917f5d486 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:34.299928 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1279ed17-b79b-4288-b133-ba0917f5d486 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:34.464850 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-cbeb35d8-c9ad-46bc-a227-6ed64412facf tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:27:34.465495 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-cbeb35d8-c9ad-46bc-a227-6ed64412facf tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] [instance: fbc2a45b-c662-475e-936d-70697843ce90] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:27:34.481159 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell0:SELECT=12 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:27:34.485790 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cbeb35d8-c9ad-46bc-a227-6ed64412facf tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:34.486949 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cbeb35d8-c9ad-46bc-a227-6ed64412facf tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:34.488694 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cbeb35d8-c9ad-46bc-a227-6ed64412facf tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:34.490318 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cbeb35d8-c9ad-46bc-a227-6ed64412facf tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:34.565981 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-1279ed17-b79b-4288-b133-ba0917f5d486 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '027dcc48e4fd4c36a5004096ad5e43f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dad823e240149edb30e592380249a13', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:27:34.585524 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-1279ed17-b79b-4288-b133-ba0917f5d486 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "GET /compute/v2.1/servers/a45c5bbd-ab66-426a-8e31-f471ce68606f" status: 200 len: 1510 microversion: 2.1 time: 0.321800 Jul 16 21:27:34.589069 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2031/4050] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:27:34 2022] GET /compute/v2.1/servers/a45c5bbd-ab66-426a-8e31-f471ce68606f => generated 1510 bytes in 326 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:27:34.670270 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-cbeb35d8-c9ad-46bc-a227-6ed64412facf tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '45315c12d53349ae8cd245cb138b8dcd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'aef73d309b204eeebfc2ce69b4e4a41b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:27:34.679280 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-cbeb35d8-c9ad-46bc-a227-6ed64412facf tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '45315c12d53349ae8cd245cb138b8dcd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'aef73d309b204eeebfc2ce69b4e4a41b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:27:34.680794 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-cbeb35d8-c9ad-46bc-a227-6ed64412facf tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '45315c12d53349ae8cd245cb138b8dcd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'aef73d309b204eeebfc2ce69b4e4a41b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:27:34.681956 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-cbeb35d8-c9ad-46bc-a227-6ed64412facf tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] 10.222.0.16 "GET /compute/v2.1/servers/fbc2a45b-c662-475e-936d-70697843ce90" status: 200 len: 1552 microversion: 2.21 time: 0.221578 Jul 16 21:27:34.682521 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2020/4051] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:27:34 2022] GET /compute/v2.1/servers/fbc2a45b-c662-475e-936d-70697843ce90 => generated 1552 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 16 21:27:34.700725 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-bcd6cdeb-840a-4af2-b75a-9eb79d5d73bb tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:27:34.701232 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-bcd6cdeb-840a-4af2-b75a-9eb79d5d73bb tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] [instance: fbc2a45b-c662-475e-936d-70697843ce90] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:27:34.707506 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-bcd6cdeb-840a-4af2-b75a-9eb79d5d73bb tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:34.707967 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-bcd6cdeb-840a-4af2-b75a-9eb79d5d73bb tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:34.708749 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-bcd6cdeb-840a-4af2-b75a-9eb79d5d73bb tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:34.709192 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-bcd6cdeb-840a-4af2-b75a-9eb79d5d73bb tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:34.757130 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-bcd6cdeb-840a-4af2-b75a-9eb79d5d73bb tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] [instance: fbc2a45b-c662-475e-936d-70697843ce90] Going to try to terminate instance {{(pid=138681) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 16 21:27:34.801580 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-2d70aa1c-2043-40d5-bdd3-8dbea14ec1fc tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:27:34.802322 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-2d70aa1c-2043-40d5-bdd3-8dbea14ec1fc tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] [instance: cf38328b-40fb-4537-b524-aaeb0a0abfb4] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:27:34.808622 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2d70aa1c-2043-40d5-bdd3-8dbea14ec1fc tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:34.809105 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2d70aa1c-2043-40d5-bdd3-8dbea14ec1fc tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:34.810056 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2d70aa1c-2043-40d5-bdd3-8dbea14ec1fc tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:34.810502 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2d70aa1c-2043-40d5-bdd3-8dbea14ec1fc tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:34.984948 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-2d70aa1c-2043-40d5-bdd3-8dbea14ec1fc tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '029c76a0ba614b5880128c4309902c67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3b214f5044c7445592e177835a5d18fc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:27:34.994577 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-2d70aa1c-2043-40d5-bdd3-8dbea14ec1fc tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] 10.222.0.16 "GET /compute/v2.1/servers/cf38328b-40fb-4537-b524-aaeb0a0abfb4" status: 200 len: 1521 microversion: 2.1 time: 0.197811 Jul 16 21:27:34.995858 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2021/4052] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:27:34 2022] GET /compute/v2.1/servers/cf38328b-40fb-4537-b524-aaeb0a0abfb4 => generated 1521 bytes in 200 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:27:35.117759 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-bcd6cdeb-840a-4af2-b75a-9eb79d5d73bb tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] 10.222.0.16 "DELETE /compute/v2.1/servers/fbc2a45b-c662-475e-936d-70697843ce90" status: 204 len: 0 microversion: 2.21 time: 0.419056 Jul 16 21:27:35.123096 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2032/4053] 10.222.0.16 () {60 vars in 1279 bytes} [Sat Jul 16 21:27:34 2022] DELETE /compute/v2.1/servers/fbc2a45b-c662-475e-936d-70697843ce90 => generated 0 bytes in 423 msecs (HTTP/1.1 204) 9 headers in 365 bytes (2 switches on core 0) Jul 16 21:27:35.136114 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-ee980105-ea32-4a6b-beb2-ba3226a833c9 tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:27:35.137154 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-ee980105-ea32-4a6b-beb2-ba3226a833c9 tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] [instance: fbc2a45b-c662-475e-936d-70697843ce90] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:27:35.148577 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ee980105-ea32-4a6b-beb2-ba3226a833c9 tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:35.149225 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ee980105-ea32-4a6b-beb2-ba3226a833c9 tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:35.150176 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ee980105-ea32-4a6b-beb2-ba3226a833c9 tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:35.150775 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ee980105-ea32-4a6b-beb2-ba3226a833c9 tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:35.380849 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-ee980105-ea32-4a6b-beb2-ba3226a833c9 tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '45315c12d53349ae8cd245cb138b8dcd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'aef73d309b204eeebfc2ce69b4e4a41b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:27:35.393455 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=100 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:27:35.404217 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-ee980105-ea32-4a6b-beb2-ba3226a833c9 tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '45315c12d53349ae8cd245cb138b8dcd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'aef73d309b204eeebfc2ce69b4e4a41b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:27:35.407035 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-ee980105-ea32-4a6b-beb2-ba3226a833c9 tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '45315c12d53349ae8cd245cb138b8dcd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'aef73d309b204eeebfc2ce69b4e4a41b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:27:35.409296 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-ee980105-ea32-4a6b-beb2-ba3226a833c9 tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] 10.222.0.16 "GET /compute/v2.1/servers/fbc2a45b-c662-475e-936d-70697843ce90" status: 200 len: 1558 microversion: 2.21 time: 0.284366 Jul 16 21:27:35.411041 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2022/4054] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:27:35 2022] GET /compute/v2.1/servers/fbc2a45b-c662-475e-936d-70697843ce90 => generated 1558 bytes in 287 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:27:35.606906 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-0e5d3dae-f03d-4298-8fe3-da9ecef47886 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:27:35.610323 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-0e5d3dae-f03d-4298-8fe3-da9ecef47886 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: a45c5bbd-ab66-426a-8e31-f471ce68606f] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:27:35.622071 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0e5d3dae-f03d-4298-8fe3-da9ecef47886 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:35.622536 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0e5d3dae-f03d-4298-8fe3-da9ecef47886 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:35.623466 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0e5d3dae-f03d-4298-8fe3-da9ecef47886 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:35.624000 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0e5d3dae-f03d-4298-8fe3-da9ecef47886 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:35.914658 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-0e5d3dae-f03d-4298-8fe3-da9ecef47886 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '027dcc48e4fd4c36a5004096ad5e43f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dad823e240149edb30e592380249a13', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:27:35.922346 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-0e5d3dae-f03d-4298-8fe3-da9ecef47886 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "GET /compute/v2.1/servers/a45c5bbd-ab66-426a-8e31-f471ce68606f" status: 200 len: 1487 microversion: 2.1 time: 0.320338 Jul 16 21:27:35.922934 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2033/4055] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:27:35 2022] GET /compute/v2.1/servers/a45c5bbd-ab66-426a-8e31-f471ce68606f => generated 1487 bytes in 322 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:27:35.938808 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-efe0b3b4-8c45-4cd1-934a-dbc99c7916d3 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Action: 'update', calling method: >, body: {"server": {"name": "tempest-\u00cd\u00f1st\u00e1\u00f1c\u00e9-1970453336"}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:27:35.940750 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-efe0b3b4-8c45-4cd1-934a-dbc99c7916d3 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: a45c5bbd-ab66-426a-8e31-f471ce68606f] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:27:35.948781 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-efe0b3b4-8c45-4cd1-934a-dbc99c7916d3 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:35.949362 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-efe0b3b4-8c45-4cd1-934a-dbc99c7916d3 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:35.950255 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-efe0b3b4-8c45-4cd1-934a-dbc99c7916d3 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:35.950798 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-efe0b3b4-8c45-4cd1-934a-dbc99c7916d3 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:36.012380 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-8f5e7a1b-07de-4bae-ba05-0f0f9800f642 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:27:36.012954 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-8f5e7a1b-07de-4bae-ba05-0f0f9800f642 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] [instance: cf38328b-40fb-4537-b524-aaeb0a0abfb4] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:27:36.019649 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8f5e7a1b-07de-4bae-ba05-0f0f9800f642 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:36.020108 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8f5e7a1b-07de-4bae-ba05-0f0f9800f642 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:36.020893 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8f5e7a1b-07de-4bae-ba05-0f0f9800f642 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:36.021336 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8f5e7a1b-07de-4bae-ba05-0f0f9800f642 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:36.101686 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-efe0b3b4-8c45-4cd1-934a-dbc99c7916d3 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "PUT /compute/v2.1/servers/a45c5bbd-ab66-426a-8e31-f471ce68606f" status: 200 len: 1060 microversion: 2.1 time: 0.166662 Jul 16 21:27:36.102439 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2023/4056] 10.222.0.16 () {60 vars in 1255 bytes} [Sat Jul 16 21:27:35 2022] PUT /compute/v2.1/servers/a45c5bbd-ab66-426a-8e31-f471ce68606f => generated 1060 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:27:36.122989 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-7e6d2faf-a8ba-46c6-9495-6b8e4d69733a tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:27:36.123970 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-7e6d2faf-a8ba-46c6-9495-6b8e4d69733a tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: a45c5bbd-ab66-426a-8e31-f471ce68606f] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:27:36.132552 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7e6d2faf-a8ba-46c6-9495-6b8e4d69733a tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:36.132552 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7e6d2faf-a8ba-46c6-9495-6b8e4d69733a tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:36.137842 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7e6d2faf-a8ba-46c6-9495-6b8e4d69733a tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:36.138399 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7e6d2faf-a8ba-46c6-9495-6b8e4d69733a tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:36.306016 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-8f5e7a1b-07de-4bae-ba05-0f0f9800f642 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '029c76a0ba614b5880128c4309902c67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3b214f5044c7445592e177835a5d18fc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:27:36.315400 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-8f5e7a1b-07de-4bae-ba05-0f0f9800f642 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] 10.222.0.16 "GET /compute/v2.1/servers/cf38328b-40fb-4537-b524-aaeb0a0abfb4" status: 200 len: 1521 microversion: 2.1 time: 0.306133 Jul 16 21:27:36.325743 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2034/4057] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:27:36 2022] GET /compute/v2.1/servers/cf38328b-40fb-4537-b524-aaeb0a0abfb4 => generated 1521 bytes in 309 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:27:36.384026 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-7e6d2faf-a8ba-46c6-9495-6b8e4d69733a tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '027dcc48e4fd4c36a5004096ad5e43f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dad823e240149edb30e592380249a13', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:27:36.399053 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-7e6d2faf-a8ba-46c6-9495-6b8e4d69733a tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "GET /compute/v2.1/servers/a45c5bbd-ab66-426a-8e31-f471ce68606f" status: 200 len: 1487 microversion: 2.1 time: 0.279674 Jul 16 21:27:36.411213 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2024/4058] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:27:36 2022] GET /compute/v2.1/servers/a45c5bbd-ab66-426a-8e31-f471ce68606f => generated 1487 bytes in 282 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:27:36.422777 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-4a36d148-59e0-4ea6-a9c4-3ded063c8a12 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:27:36.423470 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-4a36d148-59e0-4ea6-a9c4-3ded063c8a12 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: a45c5bbd-ab66-426a-8e31-f471ce68606f] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:27:36.435862 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-1f3cc019-f2b1-4481-b528-befe3f1cd348 tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:27:36.437476 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-1f3cc019-f2b1-4481-b528-befe3f1cd348 tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] [instance: fbc2a45b-c662-475e-936d-70697843ce90] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:27:36.443973 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4a36d148-59e0-4ea6-a9c4-3ded063c8a12 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:36.444724 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4a36d148-59e0-4ea6-a9c4-3ded063c8a12 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:36.446193 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4a36d148-59e0-4ea6-a9c4-3ded063c8a12 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:36.447187 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4a36d148-59e0-4ea6-a9c4-3ded063c8a12 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:36.449470 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1f3cc019-f2b1-4481-b528-befe3f1cd348 tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:36.450245 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1f3cc019-f2b1-4481-b528-befe3f1cd348 tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:36.460538 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1f3cc019-f2b1-4481-b528-befe3f1cd348 tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:36.461275 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1f3cc019-f2b1-4481-b528-befe3f1cd348 tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:36.801878 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-4a36d148-59e0-4ea6-a9c4-3ded063c8a12 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '027dcc48e4fd4c36a5004096ad5e43f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dad823e240149edb30e592380249a13', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:27:36.814196 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-4a36d148-59e0-4ea6-a9c4-3ded063c8a12 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "GET /compute/v2.1/servers/a45c5bbd-ab66-426a-8e31-f471ce68606f" status: 200 len: 1487 microversion: 2.1 time: 0.396590 Jul 16 21:27:36.815054 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2035/4059] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:27:36 2022] GET /compute/v2.1/servers/a45c5bbd-ab66-426a-8e31-f471ce68606f => generated 1487 bytes in 398 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:27:36.831122 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-13f5c303-6b37-4daa-be9e-06f8e56503f3 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:27:36.831892 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-13f5c303-6b37-4daa-be9e-06f8e56503f3 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: a45c5bbd-ab66-426a-8e31-f471ce68606f] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:27:36.838598 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-13f5c303-6b37-4daa-be9e-06f8e56503f3 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:36.839161 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-13f5c303-6b37-4daa-be9e-06f8e56503f3 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:36.839950 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-13f5c303-6b37-4daa-be9e-06f8e56503f3 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:36.840577 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-13f5c303-6b37-4daa-be9e-06f8e56503f3 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:36.866242 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-1f3cc019-f2b1-4481-b528-befe3f1cd348 tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '45315c12d53349ae8cd245cb138b8dcd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'aef73d309b204eeebfc2ce69b4e4a41b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:27:36.873128 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-13f5c303-6b37-4daa-be9e-06f8e56503f3 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: a45c5bbd-ab66-426a-8e31-f471ce68606f] Going to try to terminate instance {{(pid=138681) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 16 21:27:36.881151 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-1f3cc019-f2b1-4481-b528-befe3f1cd348 tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '45315c12d53349ae8cd245cb138b8dcd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'aef73d309b204eeebfc2ce69b4e4a41b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:27:36.882750 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-1f3cc019-f2b1-4481-b528-befe3f1cd348 tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '45315c12d53349ae8cd245cb138b8dcd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'aef73d309b204eeebfc2ce69b4e4a41b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:27:36.883981 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-1f3cc019-f2b1-4481-b528-befe3f1cd348 tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] 10.222.0.16 "GET /compute/v2.1/servers/fbc2a45b-c662-475e-936d-70697843ce90" status: 200 len: 1558 microversion: 2.21 time: 0.456733 Jul 16 21:27:36.895216 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2025/4060] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:27:36 2022] GET /compute/v2.1/servers/fbc2a45b-c662-475e-936d-70697843ce90 => generated 1558 bytes in 459 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:27:37.028472 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:27:37.207137 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-c5b786e6-724a-45b2-8f3c-5f426fdc7f74 tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesNegativeTest-instance-142481664", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452"}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:27:37.246111 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-13f5c303-6b37-4daa-be9e-06f8e56503f3 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "DELETE /compute/v2.1/servers/a45c5bbd-ab66-426a-8e31-f471ce68606f" status: 204 len: 0 microversion: 2.1 time: 0.418570 Jul 16 21:27:37.247905 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2036/4061] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:27:36 2022] DELETE /compute/v2.1/servers/a45c5bbd-ab66-426a-8e31-f471ce68606f => generated 0 bytes in 422 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 16 21:27:37.260948 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-e6080b13-7e9e-4556-bf1f-4a2b13842a5d tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:27:37.261687 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-e6080b13-7e9e-4556-bf1f-4a2b13842a5d tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: a45c5bbd-ab66-426a-8e31-f471ce68606f] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:27:37.273055 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.network.neutron [None req-c5b786e6-724a-45b2-8f3c-5f426fdc7f74 tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] validate_networks() for None {{(pid=138680) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 16 21:27:37.279311 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e6080b13-7e9e-4556-bf1f-4a2b13842a5d tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:37.280296 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e6080b13-7e9e-4556-bf1f-4a2b13842a5d tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:37.281991 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e6080b13-7e9e-4556-bf1f-4a2b13842a5d tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:37.281991 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e6080b13-7e9e-4556-bf1f-4a2b13842a5d tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:37.627267 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-e6080b13-7e9e-4556-bf1f-4a2b13842a5d tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '027dcc48e4fd4c36a5004096ad5e43f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dad823e240149edb30e592380249a13', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:27:37.634986 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-e6080b13-7e9e-4556-bf1f-4a2b13842a5d tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "GET /compute/v2.1/servers/a45c5bbd-ab66-426a-8e31-f471ce68606f" status: 200 len: 1493 microversion: 2.1 time: 0.379605 Jul 16 21:27:37.646168 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2037/4062] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:27:37 2022] GET /compute/v2.1/servers/a45c5bbd-ab66-426a-8e31-f471ce68606f => generated 1493 bytes in 383 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:27:37.650324 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-bb86621c-f3eb-47a1-9234-d6f0ce7ae757 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:27:37.650957 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-bb86621c-f3eb-47a1-9234-d6f0ce7ae757 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] [instance: cf38328b-40fb-4537-b524-aaeb0a0abfb4] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:27:37.663105 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-bb86621c-f3eb-47a1-9234-d6f0ce7ae757 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:37.663569 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-bb86621c-f3eb-47a1-9234-d6f0ce7ae757 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:37.664358 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-bb86621c-f3eb-47a1-9234-d6f0ce7ae757 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:37.664801 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-bb86621c-f3eb-47a1-9234-d6f0ce7ae757 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:37.749145 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-c5b786e6-724a-45b2-8f3c-5f426fdc7f74 tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] Flavor limits 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 16 21:27:37.749932 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-c5b786e6-724a-45b2-8f3c-5f426fdc7f74 tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] Image limits 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 16 21:27:37.750688 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-c5b786e6-724a-45b2-8f3c-5f426fdc7f74 tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] Flavor pref 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 16 21:27:37.751560 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-c5b786e6-724a-45b2-8f3c-5f426fdc7f74 tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] Image pref 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 16 21:27:37.752747 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-c5b786e6-724a-45b2-8f3c-5f426fdc7f74 tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 16 21:27:37.776549 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c5b786e6-724a-45b2-8f3c-5f426fdc7f74 tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:37.777271 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c5b786e6-724a-45b2-8f3c-5f426fdc7f74 tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:37.778319 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c5b786e6-724a-45b2-8f3c-5f426fdc7f74 tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:37.779055 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c5b786e6-724a-45b2-8f3c-5f426fdc7f74 tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:37.802979 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-c5b786e6-724a-45b2-8f3c-5f426fdc7f74 tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] Getting quotas for project 5bfd328e0f1446369531a23d15387cd6. Resources: {'ram', 'instances', 'cores'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:27:37.812702 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-c5b786e6-724a-45b2-8f3c-5f426fdc7f74 tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] Getting quotas for user 6771589864f442cbbfc8abf018abbd09 and project 5bfd328e0f1446369531a23d15387cd6. Resources: {'ram', 'instances', 'cores'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:27:37.834524 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-c5b786e6-724a-45b2-8f3c-5f426fdc7f74 tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] Going to run 1 instances... {{(pid=138680) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 16 21:27:37.837271 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-c5b786e6-724a-45b2-8f3c-5f426fdc7f74 tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] [instance: 0b67ad87-5ea8-4837-90f8-b3bc99134f3b] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138680) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 16 21:27:37.906669 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-bb86621c-f3eb-47a1-9234-d6f0ce7ae757 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '029c76a0ba614b5880128c4309902c67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3b214f5044c7445592e177835a5d18fc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:27:37.915055 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-bb86621c-f3eb-47a1-9234-d6f0ce7ae757 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] 10.222.0.16 "GET /compute/v2.1/servers/cf38328b-40fb-4537-b524-aaeb0a0abfb4" status: 200 len: 1521 microversion: 2.1 time: 0.276104 Jul 16 21:27:37.915772 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2038/4063] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:27:37 2022] GET /compute/v2.1/servers/cf38328b-40fb-4537-b524-aaeb0a0abfb4 => generated 1521 bytes in 277 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:27:37.938143 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-1d85b181-23fa-459b-8427-18dde2f94f1f req-83a4ac20-eae2-496a-8ebc-08384edc21f3 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "fbc2a45b-c662-475e-936d-70697843ce90", "name": "network-vif-deleted", "tag": "6228d01b-0897-4125-96b9-bc789787346e"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:27:37.946852 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-1d85b181-23fa-459b-8427-18dde2f94f1f req-83a4ac20-eae2-496a-8ebc-08384edc21f3 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:37.947351 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-1d85b181-23fa-459b-8427-18dde2f94f1f req-83a4ac20-eae2-496a-8ebc-08384edc21f3 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:37.965733 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-1d85b181-23fa-459b-8427-18dde2f94f1f req-83a4ac20-eae2-496a-8ebc-08384edc21f3 service nova] Creating event network-vif-deleted:6228d01b-0897-4125-96b9-bc789787346e for instance fbc2a45b-c662-475e-936d-70697843ce90 on nch1-master-1 Jul 16 21:27:37.970577 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-1d85b181-23fa-459b-8427-18dde2f94f1f req-83a4ac20-eae2-496a-8ebc-08384edc21f3 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.037948 Jul 16 21:27:37.971318 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2039/4064] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:27:37 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:27:37.978504 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-ad034676-79c3-4aae-ba2b-fdf68f315bd3 tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:27:37.979238 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-ad034676-79c3-4aae-ba2b-fdf68f315bd3 tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] [instance: fbc2a45b-c662-475e-936d-70697843ce90] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:27:37.981240 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_api:SELECT=97,nova_api:UPDATE=3 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:27:37.997200 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-c5b786e6-724a-45b2-8f3c-5f426fdc7f74 tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.794597 Jul 16 21:27:37.998519 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2026/4065] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 21:27:37 2022] POST /compute/v2.1/servers => generated 380 bytes in 796 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Jul 16 21:27:37.998855 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ad034676-79c3-4aae-ba2b-fdf68f315bd3 tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:37.999341 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ad034676-79c3-4aae-ba2b-fdf68f315bd3 tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:38.000121 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ad034676-79c3-4aae-ba2b-fdf68f315bd3 tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:38.000570 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ad034676-79c3-4aae-ba2b-fdf68f315bd3 tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:38.009889 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-2e1b28e2-e8aa-4466-af18-a3863da368f2 tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:27:38.010505 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-2e1b28e2-e8aa-4466-af18-a3863da368f2 tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] [instance: 0b67ad87-5ea8-4837-90f8-b3bc99134f3b] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:27:38.098210 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-2e1b28e2-e8aa-4466-af18-a3863da368f2 tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6771589864f442cbbfc8abf018abbd09', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bfd328e0f1446369531a23d15387cd6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:27:38.101178 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ad034676-79c3-4aae-ba2b-fdf68f315bd3 tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:38.102258 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ad034676-79c3-4aae-ba2b-fdf68f315bd3 tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:38.103438 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.objects.instance [None req-ad034676-79c3-4aae-ba2b-fdf68f315bd3 tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] Lazy-loading 'fault' on Instance uuid fbc2a45b-c662-475e-936d-70697843ce90 {{(pid=138681) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:27:38.109980 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-2e1b28e2-e8aa-4466-af18-a3863da368f2 tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] 10.222.0.16 "GET /compute/v2.1/servers/0b67ad87-5ea8-4837-90f8-b3bc99134f3b" status: 200 len: 1214 microversion: 2.1 time: 0.104999 Jul 16 21:27:38.125123 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2027/4066] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:27:38 2022] GET /compute/v2.1/servers/0b67ad87-5ea8-4837-90f8-b3bc99134f3b => generated 1214 bytes in 120 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:27:38.160431 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-ad034676-79c3-4aae-ba2b-fdf68f315bd3 tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '45315c12d53349ae8cd245cb138b8dcd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'aef73d309b204eeebfc2ce69b4e4a41b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:27:38.169597 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-ad034676-79c3-4aae-ba2b-fdf68f315bd3 tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '45315c12d53349ae8cd245cb138b8dcd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'aef73d309b204eeebfc2ce69b4e4a41b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:27:38.171315 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-ad034676-79c3-4aae-ba2b-fdf68f315bd3 tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '45315c12d53349ae8cd245cb138b8dcd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'aef73d309b204eeebfc2ce69b4e4a41b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:27:38.172939 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-ad034676-79c3-4aae-ba2b-fdf68f315bd3 tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] 10.222.0.16 "GET /compute/v2.1/servers/fbc2a45b-c662-475e-936d-70697843ce90" status: 200 len: 1349 microversion: 2.21 time: 0.198420 Jul 16 21:27:38.174670 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2040/4067] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:27:37 2022] GET /compute/v2.1/servers/fbc2a45b-c662-475e-936d-70697843ce90 => generated 1349 bytes in 203 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:27:38.654529 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-255c1148-1aba-46e3-95cc-a33337fd6a87 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:27:38.655991 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-255c1148-1aba-46e3-95cc-a33337fd6a87 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: a45c5bbd-ab66-426a-8e31-f471ce68606f] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:27:38.681833 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-255c1148-1aba-46e3-95cc-a33337fd6a87 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:38.682525 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-255c1148-1aba-46e3-95cc-a33337fd6a87 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.009s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:38.683325 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-255c1148-1aba-46e3-95cc-a33337fd6a87 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:38.683771 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-255c1148-1aba-46e3-95cc-a33337fd6a87 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:38.932304 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-5ba9fd7a-0f39-4a1c-b720-b9e83ff3cc76 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:27:38.933596 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-5ba9fd7a-0f39-4a1c-b720-b9e83ff3cc76 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] [instance: cf38328b-40fb-4537-b524-aaeb0a0abfb4] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:27:38.948065 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5ba9fd7a-0f39-4a1c-b720-b9e83ff3cc76 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:38.948523 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5ba9fd7a-0f39-4a1c-b720-b9e83ff3cc76 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:38.949316 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5ba9fd7a-0f39-4a1c-b720-b9e83ff3cc76 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:38.951539 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5ba9fd7a-0f39-4a1c-b720-b9e83ff3cc76 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:39.420184 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-255c1148-1aba-46e3-95cc-a33337fd6a87 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '027dcc48e4fd4c36a5004096ad5e43f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dad823e240149edb30e592380249a13', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:27:39.433751 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-255c1148-1aba-46e3-95cc-a33337fd6a87 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "GET /compute/v2.1/servers/a45c5bbd-ab66-426a-8e31-f471ce68606f" status: 200 len: 1493 microversion: 2.1 time: 0.783862 Jul 16 21:27:39.434577 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2028/4068] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:27:38 2022] GET /compute/v2.1/servers/a45c5bbd-ab66-426a-8e31-f471ce68606f => generated 1493 bytes in 785 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:27:39.448132 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-062b9477-60fa-40d3-8c54-2a589795f9ed tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:27:39.448934 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-062b9477-60fa-40d3-8c54-2a589795f9ed tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] [instance: 0b67ad87-5ea8-4837-90f8-b3bc99134f3b] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:27:39.464403 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-062b9477-60fa-40d3-8c54-2a589795f9ed tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:39.464873 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-062b9477-60fa-40d3-8c54-2a589795f9ed tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:39.466638 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-062b9477-60fa-40d3-8c54-2a589795f9ed tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:39.467388 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-062b9477-60fa-40d3-8c54-2a589795f9ed tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:39.573281 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-062b9477-60fa-40d3-8c54-2a589795f9ed tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6771589864f442cbbfc8abf018abbd09', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bfd328e0f1446369531a23d15387cd6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:27:39.590113 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-062b9477-60fa-40d3-8c54-2a589795f9ed tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] 10.222.0.16 "GET /compute/v2.1/servers/0b67ad87-5ea8-4837-90f8-b3bc99134f3b" status: 200 len: 1218 microversion: 2.1 time: 0.153574 Jul 16 21:27:39.603256 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2029/4069] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:27:39 2022] GET /compute/v2.1/servers/0b67ad87-5ea8-4837-90f8-b3bc99134f3b => generated 1218 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:27:39.610733 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-2c074c5a-cff0-4cf8-ba81-936c4ae60432 tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:27:39.613823 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-2c074c5a-cff0-4cf8-ba81-936c4ae60432 tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] [instance: fbc2a45b-c662-475e-936d-70697843ce90] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:27:39.625026 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2c074c5a-cff0-4cf8-ba81-936c4ae60432 tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:39.626188 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2c074c5a-cff0-4cf8-ba81-936c4ae60432 tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:39.627746 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2c074c5a-cff0-4cf8-ba81-936c4ae60432 tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:39.628346 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2c074c5a-cff0-4cf8-ba81-936c4ae60432 tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:39.641974 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-5ba9fd7a-0f39-4a1c-b720-b9e83ff3cc76 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '029c76a0ba614b5880128c4309902c67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3b214f5044c7445592e177835a5d18fc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:27:39.651861 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-5ba9fd7a-0f39-4a1c-b720-b9e83ff3cc76 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] 10.222.0.16 "GET /compute/v2.1/servers/cf38328b-40fb-4537-b524-aaeb0a0abfb4" status: 200 len: 1521 microversion: 2.1 time: 0.723404 Jul 16 21:27:39.652444 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2041/4070] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:27:38 2022] GET /compute/v2.1/servers/cf38328b-40fb-4537-b524-aaeb0a0abfb4 => generated 1521 bytes in 725 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:27:39.927814 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-2c074c5a-cff0-4cf8-ba81-936c4ae60432 tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] HTTP exception thrown: Instance fbc2a45b-c662-475e-936d-70697843ce90 could not be found. Jul 16 21:27:39.929624 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-2c074c5a-cff0-4cf8-ba81-936c4ae60432 tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] Request method failure captured: Jul 16 21:27:39.929624 ncdv-master-1 devstack@n-api.service[138680]: request: GET /compute/v2.1/servers/fbc2a45b-c662-475e-936d-70697843ce90 HTTP/1.1 Jul 16 21:27:39.929624 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:27:39.929624 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:27:39.929624 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:27:39.929624 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 0 Jul 16 21:27:39.929624 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:27:39.929624 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:27:39.929624 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:27:39.929624 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:27:39.929624 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0y01jW-x0L-Sd78FjaoQK7nWLHZrY9iauVak2_chXqEiHo-otBkgek5tKlzjw3_euqEl1jpvY0MgMSnEgmw4vbXofM2NI9-F6RB_felcWzQE6FI3lRy7P59e8EY-5fMv8Relzp4452QVaTGO4Sfzh3LJE93yxFMtAW5S3_AqXD-ttPY Jul 16 21:27:39.929624 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:27:39.929624 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:27:39.929624 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:27:39.929624 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:27:39.929624 ncdv-master-1 devstack@n-api.service[138680]: X-Openstack-Nova-Api-Version: 2.21 Jul 16 21:27:39.929624 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:27:39.929624 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:27:39.929624 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: aef73d309b204eeebfc2ce69b4e4a41b Jul 16 21:27:39.929624 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-InstanceActionsV221TestJSON-1929810395 Jul 16 21:27:39.929624 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:27:39.929624 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:27:39.929624 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/aef73d309b204eeebfc2ce69b4e4a41b"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/aef73d309b204eeebfc2ce69b4e4a41b"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_aef73d309b204eeebfc2ce69b4e4a41b"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/aef73d309b204eeebfc2ce69b4e4a41b"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:27:39.929624 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-InstanceActionsV221TestJSON-1929810395 Jul 16 21:27:39.929624 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: aef73d309b204eeebfc2ce69b4e4a41b Jul 16 21:27:39.929624 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-InstanceActionsV221TestJSON-1929810395 Jul 16 21:27:39.929624 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-InstanceActionsV221TestJSON-1929810395-project Jul 16 21:27:39.929624 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:27:39.929624 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:27:39.929624 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: 45315c12d53349ae8cd245cb138b8dcd Jul 16 21:27:39.929624 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-InstanceActionsV221TestJSON-1929810395-project Jul 16 21:27:39.929624 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 21:27:39.929624 ncdv-master-1 devstack@n-api.service[138680]: exception: Instance fbc2a45b-c662-475e-936d-70697843ce90 could not be found. Jul 16 21:27:39.929624 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': 'fbc2a45b-c662-475e-936d-70697843ce90'} Jul 16 21:27:39.929624 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:27:39.929624 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:27:39.929624 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:27:39.929624 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:27:39.929624 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:27:39.929624 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:27:39.929624 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:27:39.929624 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:27:39.929624 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:27:39.929624 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:27:39.929624 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance fbc2a45b-c662-475e-936d-70697843ce90 could not be found. Jul 16 21:27:39.929624 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:27:39.929624 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:27:39.929624 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:27:39.929624 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:27:39.929624 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:27:39.929624 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:27:39.929624 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:27:39.929624 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:27:39.929624 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:27:39.929624 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:27:39.929624 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:27:39.929624 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:27:39.929624 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:27:39.929624 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:27:39.929624 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:27:39.929624 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:27:39.929624 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance fbc2a45b-c662-475e-936d-70697843ce90 could not be found. Jul 16 21:27:39.929624 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:27:39.929624 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:27:39.937969 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:27:39.937969 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:27:39.937969 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:27:39.937969 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:27:39.937969 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:27:39.937969 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:27:39.937969 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance fbc2a45b-c662-475e-936d-70697843ce90 could not be found. Jul 16 21:27:39.937969 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:27:39.937969 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-2c074c5a-cff0-4cf8-ba81-936c4ae60432 tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] Returning 404 to user: Instance fbc2a45b-c662-475e-936d-70697843ce90 could not be found. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:27:39.937969 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-2c074c5a-cff0-4cf8-ba81-936c4ae60432 tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] 10.222.0.16 "GET /compute/v2.1/servers/fbc2a45b-c662-475e-936d-70697843ce90" status: 404 len: 111 microversion: 2.21 time: 0.326452 Jul 16 21:27:39.937969 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2030/4071] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:27:39 2022] GET /compute/v2.1/servers/fbc2a45b-c662-475e-936d-70697843ce90 => generated 111 bytes in 328 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Jul 16 21:27:39.940731 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-0d954d1b-808b-4ed5-97ff-1c1027ca1be7 tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] Calling method '.version_select at 0x7fdda2a58280>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:27:39.941314 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-0d954d1b-808b-4ed5-97ff-1c1027ca1be7 tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] [instance: fbc2a45b-c662-475e-936d-70697843ce90] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:27:39.948359 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0d954d1b-808b-4ed5-97ff-1c1027ca1be7 tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:39.949185 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0d954d1b-808b-4ed5-97ff-1c1027ca1be7 tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:39.950119 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0d954d1b-808b-4ed5-97ff-1c1027ca1be7 tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:39.950707 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0d954d1b-808b-4ed5-97ff-1c1027ca1be7 tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:40.053460 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-0d954d1b-808b-4ed5-97ff-1c1027ca1be7 tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] 10.222.0.16 "GET /compute/v2.1/servers/fbc2a45b-c662-475e-936d-70697843ce90/os-instance-actions" status: 200 len: 611 microversion: 2.21 time: 0.116481 Jul 16 21:27:40.054423 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-ec9f0328-9653-4bd6-b5fd-86f20803be2b req-916346ec-6ba1-4d4e-930d-f130cf387630 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "a45c5bbd-ab66-426a-8e31-f471ce68606f", "name": "network-vif-deleted", "tag": "ccf9712c-2c5a-485b-bbdc-293d0cd69551"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:27:40.056144 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2042/4072] 10.222.0.16 () {60 vars in 1336 bytes} [Sat Jul 16 21:27:39 2022] GET /compute/v2.1/servers/fbc2a45b-c662-475e-936d-70697843ce90/os-instance-actions => generated 611 bytes in 120 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 16 21:27:40.084439 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-7abebea7-d5ee-4b5c-bc00-20b67c960027 tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:27:40.085179 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-7abebea7-d5ee-4b5c-bc00-20b67c960027 tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] [instance: fbc2a45b-c662-475e-936d-70697843ce90] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:27:40.094945 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-ec9f0328-9653-4bd6-b5fd-86f20803be2b req-916346ec-6ba1-4d4e-930d-f130cf387630 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:40.095923 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-ec9f0328-9653-4bd6-b5fd-86f20803be2b req-916346ec-6ba1-4d4e-930d-f130cf387630 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:40.109841 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-ec9f0328-9653-4bd6-b5fd-86f20803be2b req-916346ec-6ba1-4d4e-930d-f130cf387630 service nova] Creating event network-vif-deleted:ccf9712c-2c5a-485b-bbdc-293d0cd69551 for instance a45c5bbd-ab66-426a-8e31-f471ce68606f on nch2-master-1 Jul 16 21:27:40.131864 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-ec9f0328-9653-4bd6-b5fd-86f20803be2b req-916346ec-6ba1-4d4e-930d-f130cf387630 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.080810 Jul 16 21:27:40.137918 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2031/4073] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:27:40 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:27:40.138153 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7abebea7-d5ee-4b5c-bc00-20b67c960027 tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:40.138153 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7abebea7-d5ee-4b5c-bc00-20b67c960027 tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:40.140380 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7abebea7-d5ee-4b5c-bc00-20b67c960027 tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:40.140843 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7abebea7-d5ee-4b5c-bc00-20b67c960027 tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:40.175485 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-7abebea7-d5ee-4b5c-bc00-20b67c960027 tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] HTTP exception thrown: Instance fbc2a45b-c662-475e-936d-70697843ce90 could not be found. Jul 16 21:27:40.178278 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-7abebea7-d5ee-4b5c-bc00-20b67c960027 tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] Request method failure captured: Jul 16 21:27:40.178278 ncdv-master-1 devstack@n-api.service[138681]: request: DELETE /compute/v2.1/servers/fbc2a45b-c662-475e-936d-70697843ce90 HTTP/1.1 Jul 16 21:27:40.178278 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:27:40.178278 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:27:40.178278 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:27:40.178278 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 21:27:40.178278 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:27:40.178278 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:27:40.178278 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:27:40.178278 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:27:40.178278 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0y01jW-x0L-Sd78FjaoQK7nWLHZrY9iauVak2_chXqEiHo-otBkgek5tKlzjw3_euqEl1jpvY0MgMSnEgmw4vbXofM2NI9-F6RB_felcWzQE6FI3lRy7P59e8EY-5fMv8Relzp4452QVaTGO4Sfzh3LJE93yxFMtAW5S3_AqXD-ttPY Jul 16 21:27:40.178278 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:27:40.178278 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:27:40.178278 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:27:40.178278 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:27:40.178278 ncdv-master-1 devstack@n-api.service[138681]: X-Openstack-Nova-Api-Version: 2.21 Jul 16 21:27:40.178278 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:27:40.178278 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:27:40.178278 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: aef73d309b204eeebfc2ce69b4e4a41b Jul 16 21:27:40.178278 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-InstanceActionsV221TestJSON-1929810395 Jul 16 21:27:40.178278 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:27:40.178278 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:27:40.178278 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/aef73d309b204eeebfc2ce69b4e4a41b"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/aef73d309b204eeebfc2ce69b4e4a41b"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_aef73d309b204eeebfc2ce69b4e4a41b"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/aef73d309b204eeebfc2ce69b4e4a41b"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:27:40.178278 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-InstanceActionsV221TestJSON-1929810395 Jul 16 21:27:40.178278 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: aef73d309b204eeebfc2ce69b4e4a41b Jul 16 21:27:40.178278 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-InstanceActionsV221TestJSON-1929810395 Jul 16 21:27:40.178278 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-InstanceActionsV221TestJSON-1929810395-project Jul 16 21:27:40.178278 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:27:40.178278 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:27:40.178278 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: 45315c12d53349ae8cd245cb138b8dcd Jul 16 21:27:40.178278 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-InstanceActionsV221TestJSON-1929810395-project Jul 16 21:27:40.178278 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:27:40.178278 ncdv-master-1 devstack@n-api.service[138681]: exception: Instance fbc2a45b-c662-475e-936d-70697843ce90 could not be found. Jul 16 21:27:40.178278 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': 'fbc2a45b-c662-475e-936d-70697843ce90'} Jul 16 21:27:40.178278 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:27:40.178278 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:27:40.178278 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:27:40.178278 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:27:40.178278 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:27:40.178278 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:27:40.178278 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:27:40.178278 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:27:40.178278 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:27:40.178278 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:27:40.178278 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance fbc2a45b-c662-475e-936d-70697843ce90 could not be found. Jul 16 21:27:40.178278 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:27:40.178278 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:27:40.178278 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:27:40.178278 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:27:40.178278 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:27:40.178278 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:27:40.178278 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:27:40.178278 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:27:40.178278 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:27:40.178278 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:27:40.178278 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Jul 16 21:27:40.178278 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Jul 16 21:27:40.178278 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Jul 16 21:27:40.178278 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Jul 16 21:27:40.178278 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:27:40.178278 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:27:40.178278 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:27:40.178278 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:27:40.182139 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance fbc2a45b-c662-475e-936d-70697843ce90 could not be found. Jul 16 21:27:40.182139 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:27:40.182139 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:27:40.182139 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:27:40.182139 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:27:40.182139 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:27:40.182139 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:27:40.182139 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:27:40.182139 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:27:40.182139 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance fbc2a45b-c662-475e-936d-70697843ce90 could not be found. Jul 16 21:27:40.182139 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:27:40.186290 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-7abebea7-d5ee-4b5c-bc00-20b67c960027 tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] Returning 404 to user: Instance fbc2a45b-c662-475e-936d-70697843ce90 could not be found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:27:40.187273 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-7abebea7-d5ee-4b5c-bc00-20b67c960027 tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] 10.222.0.16 "DELETE /compute/v2.1/servers/fbc2a45b-c662-475e-936d-70697843ce90" status: 404 len: 111 microversion: 2.21 time: 0.120283 Jul 16 21:27:40.188982 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2043/4074] 10.222.0.16 () {60 vars in 1279 bytes} [Sat Jul 16 21:27:40 2022] DELETE /compute/v2.1/servers/fbc2a45b-c662-475e-936d-70697843ce90 => generated 111 bytes in 123 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Jul 16 21:27:40.198330 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-e63037b9-eace-4b07-899f-421d0bf32f93 tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:27:40.200901 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-e63037b9-eace-4b07-899f-421d0bf32f93 tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] [instance: fbc2a45b-c662-475e-936d-70697843ce90] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:27:40.215288 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e63037b9-eace-4b07-899f-421d0bf32f93 tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:40.215940 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e63037b9-eace-4b07-899f-421d0bf32f93 tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:40.217365 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e63037b9-eace-4b07-899f-421d0bf32f93 tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:40.218086 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e63037b9-eace-4b07-899f-421d0bf32f93 tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:40.239948 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-e63037b9-eace-4b07-899f-421d0bf32f93 tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] HTTP exception thrown: Instance fbc2a45b-c662-475e-936d-70697843ce90 could not be found. Jul 16 21:27:40.242398 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-e63037b9-eace-4b07-899f-421d0bf32f93 tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] Request method failure captured: Jul 16 21:27:40.242398 ncdv-master-1 devstack@n-api.service[138680]: request: GET /compute/v2.1/servers/fbc2a45b-c662-475e-936d-70697843ce90 HTTP/1.1 Jul 16 21:27:40.242398 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:27:40.242398 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:27:40.242398 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:27:40.242398 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 0 Jul 16 21:27:40.242398 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:27:40.242398 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:27:40.242398 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:27:40.242398 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:27:40.242398 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0y01jW-x0L-Sd78FjaoQK7nWLHZrY9iauVak2_chXqEiHo-otBkgek5tKlzjw3_euqEl1jpvY0MgMSnEgmw4vbXofM2NI9-F6RB_felcWzQE6FI3lRy7P59e8EY-5fMv8Relzp4452QVaTGO4Sfzh3LJE93yxFMtAW5S3_AqXD-ttPY Jul 16 21:27:40.242398 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:27:40.242398 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:27:40.242398 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:27:40.242398 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:27:40.242398 ncdv-master-1 devstack@n-api.service[138680]: X-Openstack-Nova-Api-Version: 2.21 Jul 16 21:27:40.242398 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:27:40.242398 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:27:40.242398 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: aef73d309b204eeebfc2ce69b4e4a41b Jul 16 21:27:40.242398 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-InstanceActionsV221TestJSON-1929810395 Jul 16 21:27:40.242398 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:27:40.242398 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:27:40.242398 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/aef73d309b204eeebfc2ce69b4e4a41b"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/aef73d309b204eeebfc2ce69b4e4a41b"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_aef73d309b204eeebfc2ce69b4e4a41b"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/aef73d309b204eeebfc2ce69b4e4a41b"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:27:40.242398 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-InstanceActionsV221TestJSON-1929810395 Jul 16 21:27:40.242398 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: aef73d309b204eeebfc2ce69b4e4a41b Jul 16 21:27:40.242398 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-InstanceActionsV221TestJSON-1929810395 Jul 16 21:27:40.242398 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-InstanceActionsV221TestJSON-1929810395-project Jul 16 21:27:40.242398 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:27:40.242398 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:27:40.242398 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: 45315c12d53349ae8cd245cb138b8dcd Jul 16 21:27:40.242398 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-InstanceActionsV221TestJSON-1929810395-project Jul 16 21:27:40.242398 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 21:27:40.242398 ncdv-master-1 devstack@n-api.service[138680]: exception: Instance fbc2a45b-c662-475e-936d-70697843ce90 could not be found. Jul 16 21:27:40.242398 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': 'fbc2a45b-c662-475e-936d-70697843ce90'} Jul 16 21:27:40.242398 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:27:40.242398 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:27:40.242398 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:27:40.242398 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:27:40.242398 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:27:40.242398 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:27:40.242398 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:27:40.242398 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:27:40.242398 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:27:40.242398 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:27:40.242398 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance fbc2a45b-c662-475e-936d-70697843ce90 could not be found. Jul 16 21:27:40.242398 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:27:40.242398 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:27:40.242398 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:27:40.242398 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:27:40.242398 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:27:40.242398 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:27:40.242398 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:27:40.242398 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:27:40.242398 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:27:40.242398 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:27:40.242398 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:27:40.242398 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:27:40.242398 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:27:40.242398 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:27:40.242398 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:27:40.242398 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:27:40.242398 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance fbc2a45b-c662-475e-936d-70697843ce90 could not be found. Jul 16 21:27:40.242398 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:27:40.242398 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:27:40.246238 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:27:40.246238 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:27:40.246238 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:27:40.246238 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:27:40.246238 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:27:40.246238 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:27:40.246238 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance fbc2a45b-c662-475e-936d-70697843ce90 could not be found. Jul 16 21:27:40.246238 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:27:40.246238 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-e63037b9-eace-4b07-899f-421d0bf32f93 tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] Returning 404 to user: Instance fbc2a45b-c662-475e-936d-70697843ce90 could not be found. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:27:40.246238 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-e63037b9-eace-4b07-899f-421d0bf32f93 tempest-InstanceActionsV221TestJSON-1929810395 tempest-InstanceActionsV221TestJSON-1929810395-project] 10.222.0.16 "GET /compute/v2.1/servers/fbc2a45b-c662-475e-936d-70697843ce90" status: 404 len: 111 microversion: 2.21 time: 0.050207 Jul 16 21:27:40.264832 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2032/4075] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:27:40 2022] GET /compute/v2.1/servers/fbc2a45b-c662-475e-936d-70697843ce90 => generated 111 bytes in 65 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Jul 16 21:27:40.459591 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-1e1cd515-176b-42ed-80c4-0f0db9e41a3f tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:27:40.460284 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-1e1cd515-176b-42ed-80c4-0f0db9e41a3f tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: a45c5bbd-ab66-426a-8e31-f471ce68606f] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:27:40.473621 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1e1cd515-176b-42ed-80c4-0f0db9e41a3f tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:40.474970 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1e1cd515-176b-42ed-80c4-0f0db9e41a3f tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:40.478589 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1e1cd515-176b-42ed-80c4-0f0db9e41a3f tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:40.479332 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1e1cd515-176b-42ed-80c4-0f0db9e41a3f tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:40.590193 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-1e1cd515-176b-42ed-80c4-0f0db9e41a3f tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '027dcc48e4fd4c36a5004096ad5e43f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dad823e240149edb30e592380249a13', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:27:40.602522 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-1e1cd515-176b-42ed-80c4-0f0db9e41a3f tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "GET /compute/v2.1/servers/a45c5bbd-ab66-426a-8e31-f471ce68606f" status: 200 len: 1292 microversion: 2.1 time: 0.153371 Jul 16 21:27:40.602886 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2044/4076] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:27:40 2022] GET /compute/v2.1/servers/a45c5bbd-ab66-426a-8e31-f471ce68606f => generated 1292 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:27:40.613302 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-ccacce06-3ad3-4e87-8316-97d479f7eb1f tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:27:40.613302 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-ccacce06-3ad3-4e87-8316-97d479f7eb1f tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] [instance: 0b67ad87-5ea8-4837-90f8-b3bc99134f3b] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:27:40.625549 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ccacce06-3ad3-4e87-8316-97d479f7eb1f tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:40.631318 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ccacce06-3ad3-4e87-8316-97d479f7eb1f tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:40.658638 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ccacce06-3ad3-4e87-8316-97d479f7eb1f tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:40.659187 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ccacce06-3ad3-4e87-8316-97d479f7eb1f tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:40.678299 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-7a258fb8-ef7c-4436-9f60-9ffd40710a88 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:27:40.678967 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-7a258fb8-ef7c-4436-9f60-9ffd40710a88 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] [instance: cf38328b-40fb-4537-b524-aaeb0a0abfb4] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:27:40.708622 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7a258fb8-ef7c-4436-9f60-9ffd40710a88 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:40.709092 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7a258fb8-ef7c-4436-9f60-9ffd40710a88 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:40.709896 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7a258fb8-ef7c-4436-9f60-9ffd40710a88 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:40.710337 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7a258fb8-ef7c-4436-9f60-9ffd40710a88 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:40.775261 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-ccacce06-3ad3-4e87-8316-97d479f7eb1f tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6771589864f442cbbfc8abf018abbd09', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bfd328e0f1446369531a23d15387cd6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:27:40.789660 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-ccacce06-3ad3-4e87-8316-97d479f7eb1f tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] 10.222.0.16 "GET /compute/v2.1/servers/0b67ad87-5ea8-4837-90f8-b3bc99134f3b" status: 200 len: 1266 microversion: 2.1 time: 0.185449 Jul 16 21:27:40.790815 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2033/4077] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:27:40 2022] GET /compute/v2.1/servers/0b67ad87-5ea8-4837-90f8-b3bc99134f3b => generated 1266 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:27:41.239657 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-7a258fb8-ef7c-4436-9f60-9ffd40710a88 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '029c76a0ba614b5880128c4309902c67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3b214f5044c7445592e177835a5d18fc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:27:41.248683 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-7a258fb8-ef7c-4436-9f60-9ffd40710a88 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] 10.222.0.16 "GET /compute/v2.1/servers/cf38328b-40fb-4537-b524-aaeb0a0abfb4" status: 200 len: 1521 microversion: 2.1 time: 0.580784 Jul 16 21:27:41.261835 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2045/4078] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:27:40 2022] GET /compute/v2.1/servers/cf38328b-40fb-4537-b524-aaeb0a0abfb4 => generated 1521 bytes in 596 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:27:41.623700 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-9f634ca0-b8f4-4fb6-ad8e-59aa3e2d422e tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:27:41.626064 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-9f634ca0-b8f4-4fb6-ad8e-59aa3e2d422e tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: a45c5bbd-ab66-426a-8e31-f471ce68606f] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:27:41.645175 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9f634ca0-b8f4-4fb6-ad8e-59aa3e2d422e tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:41.648480 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9f634ca0-b8f4-4fb6-ad8e-59aa3e2d422e tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:41.650872 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9f634ca0-b8f4-4fb6-ad8e-59aa3e2d422e tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:41.652153 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9f634ca0-b8f4-4fb6-ad8e-59aa3e2d422e tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:41.694785 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-9f634ca0-b8f4-4fb6-ad8e-59aa3e2d422e tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] HTTP exception thrown: Instance a45c5bbd-ab66-426a-8e31-f471ce68606f could not be found. Jul 16 21:27:41.697004 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-9f634ca0-b8f4-4fb6-ad8e-59aa3e2d422e tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Request method failure captured: Jul 16 21:27:41.697004 ncdv-master-1 devstack@n-api.service[138680]: request: GET /compute/v2.1/servers/a45c5bbd-ab66-426a-8e31-f471ce68606f HTTP/1.1 Jul 16 21:27:41.697004 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:27:41.697004 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:27:41.697004 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:27:41.697004 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 0 Jul 16 21:27:41.697004 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:27:41.697004 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:27:41.697004 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:27:41.697004 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:27:41.697004 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0yxS9NCWgI1-O55Z_AF16JzmczB_p-s4P30aJMz-9zu9VxcHuIV00tsfB9hOBtHTZ_sI8GPeOOvUrqYZLgOBzCbOMr4ZO506_XIRjMCtHQ0umzvcB9V9iFSNFoiIe_b2jKm-OyHj1IB8YVBCAcwDKANlsCw5UaAoIABK4dLLFjaVpEo Jul 16 21:27:41.697004 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:27:41.697004 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:27:41.697004 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:27:41.697004 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:27:41.697004 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:27:41.697004 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:27:41.697004 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: 1dad823e240149edb30e592380249a13 Jul 16 21:27:41.697004 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-ServersTestJSON-1529284206 Jul 16 21:27:41.697004 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:27:41.697004 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:27:41.697004 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/1dad823e240149edb30e592380249a13"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/1dad823e240149edb30e592380249a13"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_1dad823e240149edb30e592380249a13"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/1dad823e240149edb30e592380249a13"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:27:41.697004 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-ServersTestJSON-1529284206 Jul 16 21:27:41.697004 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: 1dad823e240149edb30e592380249a13 Jul 16 21:27:41.697004 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-ServersTestJSON-1529284206 Jul 16 21:27:41.697004 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-ServersTestJSON-1529284206-project Jul 16 21:27:41.697004 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:27:41.697004 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:27:41.697004 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: 027dcc48e4fd4c36a5004096ad5e43f8 Jul 16 21:27:41.697004 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-ServersTestJSON-1529284206-project Jul 16 21:27:41.697004 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 21:27:41.697004 ncdv-master-1 devstack@n-api.service[138680]: exception: Instance a45c5bbd-ab66-426a-8e31-f471ce68606f could not be found. Jul 16 21:27:41.697004 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': 'a45c5bbd-ab66-426a-8e31-f471ce68606f'} Jul 16 21:27:41.697004 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:27:41.697004 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:27:41.697004 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:27:41.697004 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:27:41.697004 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:27:41.697004 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:27:41.697004 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:27:41.697004 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:27:41.697004 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:27:41.697004 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:27:41.697004 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance a45c5bbd-ab66-426a-8e31-f471ce68606f could not be found. Jul 16 21:27:41.697004 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:27:41.697004 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:27:41.697004 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:27:41.697004 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:27:41.697004 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:27:41.697004 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:27:41.697004 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:27:41.697004 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:27:41.697004 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:27:41.697004 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:27:41.697004 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:27:41.697004 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:27:41.697004 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:27:41.697004 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:27:41.697004 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:27:41.697004 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:27:41.697004 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance a45c5bbd-ab66-426a-8e31-f471ce68606f could not be found. Jul 16 21:27:41.697004 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:27:41.697004 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:27:41.697004 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:27:41.697004 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:27:41.701324 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:27:41.701324 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:27:41.701324 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:27:41.701324 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:27:41.701324 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance a45c5bbd-ab66-426a-8e31-f471ce68606f could not be found. Jul 16 21:27:41.701324 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:27:41.701324 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-9f634ca0-b8f4-4fb6-ad8e-59aa3e2d422e tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Returning 404 to user: Instance a45c5bbd-ab66-426a-8e31-f471ce68606f could not be found. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:27:41.701324 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-9f634ca0-b8f4-4fb6-ad8e-59aa3e2d422e tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "GET /compute/v2.1/servers/a45c5bbd-ab66-426a-8e31-f471ce68606f" status: 404 len: 111 microversion: 2.1 time: 0.081509 Jul 16 21:27:41.701324 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2034/4079] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:27:41 2022] GET /compute/v2.1/servers/a45c5bbd-ab66-426a-8e31-f471ce68606f => generated 111 bytes in 83 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:27:41.721733 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-5afcd95f-8705-4192-bf11-6a97d4503759 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:27:41.722512 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-5afcd95f-8705-4192-bf11-6a97d4503759 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: a45c5bbd-ab66-426a-8e31-f471ce68606f] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:27:41.729153 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5afcd95f-8705-4192-bf11-6a97d4503759 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:41.733112 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5afcd95f-8705-4192-bf11-6a97d4503759 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:41.733112 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5afcd95f-8705-4192-bf11-6a97d4503759 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:41.733112 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5afcd95f-8705-4192-bf11-6a97d4503759 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:41.748818 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-5afcd95f-8705-4192-bf11-6a97d4503759 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] HTTP exception thrown: Instance a45c5bbd-ab66-426a-8e31-f471ce68606f could not be found. Jul 16 21:27:41.750878 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-5afcd95f-8705-4192-bf11-6a97d4503759 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Request method failure captured: Jul 16 21:27:41.750878 ncdv-master-1 devstack@n-api.service[138681]: request: DELETE /compute/v2.1/servers/a45c5bbd-ab66-426a-8e31-f471ce68606f HTTP/1.1 Jul 16 21:27:41.750878 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:27:41.750878 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:27:41.750878 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:27:41.750878 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 21:27:41.750878 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:27:41.750878 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:27:41.750878 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:27:41.750878 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:27:41.750878 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0yxS9NCWgI1-O55Z_AF16JzmczB_p-s4P30aJMz-9zu9VxcHuIV00tsfB9hOBtHTZ_sI8GPeOOvUrqYZLgOBzCbOMr4ZO506_XIRjMCtHQ0umzvcB9V9iFSNFoiIe_b2jKm-OyHj1IB8YVBCAcwDKANlsCw5UaAoIABK4dLLFjaVpEo Jul 16 21:27:41.750878 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:27:41.750878 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:27:41.750878 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:27:41.750878 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:27:41.750878 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:27:41.750878 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:27:41.750878 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: 1dad823e240149edb30e592380249a13 Jul 16 21:27:41.750878 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-ServersTestJSON-1529284206 Jul 16 21:27:41.750878 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:27:41.750878 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:27:41.750878 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/1dad823e240149edb30e592380249a13"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/1dad823e240149edb30e592380249a13"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_1dad823e240149edb30e592380249a13"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/1dad823e240149edb30e592380249a13"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:27:41.750878 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-ServersTestJSON-1529284206 Jul 16 21:27:41.750878 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: 1dad823e240149edb30e592380249a13 Jul 16 21:27:41.750878 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-ServersTestJSON-1529284206 Jul 16 21:27:41.750878 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-ServersTestJSON-1529284206-project Jul 16 21:27:41.750878 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:27:41.750878 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:27:41.750878 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: 027dcc48e4fd4c36a5004096ad5e43f8 Jul 16 21:27:41.750878 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-ServersTestJSON-1529284206-project Jul 16 21:27:41.750878 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:27:41.750878 ncdv-master-1 devstack@n-api.service[138681]: exception: Instance a45c5bbd-ab66-426a-8e31-f471ce68606f could not be found. Jul 16 21:27:41.750878 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': 'a45c5bbd-ab66-426a-8e31-f471ce68606f'} Jul 16 21:27:41.750878 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:27:41.750878 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:27:41.750878 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:27:41.750878 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:27:41.750878 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:27:41.750878 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:27:41.750878 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:27:41.750878 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:27:41.750878 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:27:41.750878 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:27:41.750878 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance a45c5bbd-ab66-426a-8e31-f471ce68606f could not be found. Jul 16 21:27:41.750878 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:27:41.750878 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:27:41.750878 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:27:41.750878 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:27:41.750878 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:27:41.750878 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:27:41.750878 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:27:41.750878 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:27:41.750878 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:27:41.750878 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:27:41.750878 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Jul 16 21:27:41.750878 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Jul 16 21:27:41.750878 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Jul 16 21:27:41.750878 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Jul 16 21:27:41.750878 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:27:41.750878 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:27:41.750878 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:27:41.750878 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:27:41.750878 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance a45c5bbd-ab66-426a-8e31-f471ce68606f could not be found. Jul 16 21:27:41.758881 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:27:41.758881 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:27:41.758881 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:27:41.758881 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:27:41.758881 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:27:41.758881 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:27:41.758881 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:27:41.758881 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:27:41.758881 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance a45c5bbd-ab66-426a-8e31-f471ce68606f could not be found. Jul 16 21:27:41.758881 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:27:41.758881 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-5afcd95f-8705-4192-bf11-6a97d4503759 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Returning 404 to user: Instance a45c5bbd-ab66-426a-8e31-f471ce68606f could not be found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:27:41.758881 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-5afcd95f-8705-4192-bf11-6a97d4503759 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "DELETE /compute/v2.1/servers/a45c5bbd-ab66-426a-8e31-f471ce68606f" status: 404 len: 111 microversion: 2.1 time: 0.036908 Jul 16 21:27:41.758881 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2046/4080] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:27:41 2022] DELETE /compute/v2.1/servers/a45c5bbd-ab66-426a-8e31-f471ce68606f => generated 111 bytes in 38 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Jul 16 21:27:41.762696 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-d85828bc-4f73-45a3-aff9-64badff2e486 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:27:41.763408 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-d85828bc-4f73-45a3-aff9-64badff2e486 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: a45c5bbd-ab66-426a-8e31-f471ce68606f] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:27:41.771624 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d85828bc-4f73-45a3-aff9-64badff2e486 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:41.772085 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d85828bc-4f73-45a3-aff9-64badff2e486 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:41.773019 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d85828bc-4f73-45a3-aff9-64badff2e486 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:41.773650 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d85828bc-4f73-45a3-aff9-64badff2e486 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:41.790227 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-d85828bc-4f73-45a3-aff9-64badff2e486 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] HTTP exception thrown: Instance a45c5bbd-ab66-426a-8e31-f471ce68606f could not be found. Jul 16 21:27:41.792244 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-d85828bc-4f73-45a3-aff9-64badff2e486 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Request method failure captured: Jul 16 21:27:41.792244 ncdv-master-1 devstack@n-api.service[138680]: request: GET /compute/v2.1/servers/a45c5bbd-ab66-426a-8e31-f471ce68606f HTTP/1.1 Jul 16 21:27:41.792244 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:27:41.792244 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:27:41.792244 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:27:41.792244 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 0 Jul 16 21:27:41.792244 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:27:41.792244 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:27:41.792244 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:27:41.792244 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:27:41.792244 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0yxS9NCWgI1-O55Z_AF16JzmczB_p-s4P30aJMz-9zu9VxcHuIV00tsfB9hOBtHTZ_sI8GPeOOvUrqYZLgOBzCbOMr4ZO506_XIRjMCtHQ0umzvcB9V9iFSNFoiIe_b2jKm-OyHj1IB8YVBCAcwDKANlsCw5UaAoIABK4dLLFjaVpEo Jul 16 21:27:41.792244 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:27:41.792244 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:27:41.792244 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:27:41.792244 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:27:41.792244 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:27:41.792244 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:27:41.792244 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: 1dad823e240149edb30e592380249a13 Jul 16 21:27:41.792244 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-ServersTestJSON-1529284206 Jul 16 21:27:41.792244 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:27:41.792244 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:27:41.792244 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/1dad823e240149edb30e592380249a13"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/1dad823e240149edb30e592380249a13"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_1dad823e240149edb30e592380249a13"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/1dad823e240149edb30e592380249a13"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:27:41.792244 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-ServersTestJSON-1529284206 Jul 16 21:27:41.792244 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: 1dad823e240149edb30e592380249a13 Jul 16 21:27:41.792244 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-ServersTestJSON-1529284206 Jul 16 21:27:41.792244 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-ServersTestJSON-1529284206-project Jul 16 21:27:41.792244 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:27:41.792244 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:27:41.792244 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: 027dcc48e4fd4c36a5004096ad5e43f8 Jul 16 21:27:41.792244 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-ServersTestJSON-1529284206-project Jul 16 21:27:41.792244 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 21:27:41.792244 ncdv-master-1 devstack@n-api.service[138680]: exception: Instance a45c5bbd-ab66-426a-8e31-f471ce68606f could not be found. Jul 16 21:27:41.792244 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': 'a45c5bbd-ab66-426a-8e31-f471ce68606f'} Jul 16 21:27:41.792244 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:27:41.792244 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:27:41.792244 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:27:41.792244 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:27:41.792244 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:27:41.792244 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:27:41.792244 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:27:41.792244 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:27:41.792244 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:27:41.792244 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:27:41.792244 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance a45c5bbd-ab66-426a-8e31-f471ce68606f could not be found. Jul 16 21:27:41.792244 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:27:41.792244 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:27:41.792244 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:27:41.792244 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:27:41.792244 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:27:41.792244 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:27:41.792244 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:27:41.792244 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:27:41.792244 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:27:41.792244 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:27:41.792244 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:27:41.792244 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:27:41.792244 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:27:41.792244 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:27:41.792244 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:27:41.792244 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:27:41.792244 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance a45c5bbd-ab66-426a-8e31-f471ce68606f could not be found. Jul 16 21:27:41.792244 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:27:41.792244 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:27:41.792244 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:27:41.792244 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:27:41.808483 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:27:41.808483 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:27:41.808483 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:27:41.808483 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:27:41.808483 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance a45c5bbd-ab66-426a-8e31-f471ce68606f could not be found. Jul 16 21:27:41.808483 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:27:41.809008 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-ce639c4a-7294-43f5-b255-1c060f1039cf tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:27:41.809639 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-ce639c4a-7294-43f5-b255-1c060f1039cf tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] [instance: 0b67ad87-5ea8-4837-90f8-b3bc99134f3b] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:27:41.817969 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ce639c4a-7294-43f5-b255-1c060f1039cf tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:41.818436 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ce639c4a-7294-43f5-b255-1c060f1039cf tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:41.819458 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ce639c4a-7294-43f5-b255-1c060f1039cf tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:41.820175 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ce639c4a-7294-43f5-b255-1c060f1039cf tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:41.822439 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-d85828bc-4f73-45a3-aff9-64badff2e486 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Returning 404 to user: Instance a45c5bbd-ab66-426a-8e31-f471ce68606f could not be found. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:27:41.826145 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-d85828bc-4f73-45a3-aff9-64badff2e486 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "GET /compute/v2.1/servers/a45c5bbd-ab66-426a-8e31-f471ce68606f" status: 404 len: 111 microversion: 2.1 time: 0.064897 Jul 16 21:27:41.826145 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2035/4081] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:27:41 2022] GET /compute/v2.1/servers/a45c5bbd-ab66-426a-8e31-f471ce68606f => generated 111 bytes in 66 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:27:41.834744 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-263cfdd2-3762-4b20-9220-c3741d6226bd tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:27:41.835551 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-263cfdd2-3762-4b20-9220-c3741d6226bd tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: 6dcb4587-0e15-4a2d-b8f0-70b459fb6664] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:27:41.845663 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-263cfdd2-3762-4b20-9220-c3741d6226bd tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:41.849189 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-263cfdd2-3762-4b20-9220-c3741d6226bd tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:41.849189 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-263cfdd2-3762-4b20-9220-c3741d6226bd tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:41.849189 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-263cfdd2-3762-4b20-9220-c3741d6226bd tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:41.867412 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-263cfdd2-3762-4b20-9220-c3741d6226bd tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] HTTP exception thrown: Instance 6dcb4587-0e15-4a2d-b8f0-70b459fb6664 could not be found. Jul 16 21:27:41.874815 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-263cfdd2-3762-4b20-9220-c3741d6226bd tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Request method failure captured: Jul 16 21:27:41.874815 ncdv-master-1 devstack@n-api.service[138680]: request: DELETE /compute/v2.1/servers/6dcb4587-0e15-4a2d-b8f0-70b459fb6664 HTTP/1.1 Jul 16 21:27:41.874815 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:27:41.874815 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:27:41.874815 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:27:41.874815 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 0 Jul 16 21:27:41.874815 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:27:41.874815 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:27:41.874815 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:27:41.874815 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:27:41.874815 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0yxS9NCWgI1-O55Z_AF16JzmczB_p-s4P30aJMz-9zu9VxcHuIV00tsfB9hOBtHTZ_sI8GPeOOvUrqYZLgOBzCbOMr4ZO506_XIRjMCtHQ0umzvcB9V9iFSNFoiIe_b2jKm-OyHj1IB8YVBCAcwDKANlsCw5UaAoIABK4dLLFjaVpEo Jul 16 21:27:41.874815 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:27:41.874815 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:27:41.874815 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:27:41.874815 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:27:41.874815 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:27:41.874815 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:27:41.874815 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: 1dad823e240149edb30e592380249a13 Jul 16 21:27:41.874815 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-ServersTestJSON-1529284206 Jul 16 21:27:41.874815 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:27:41.874815 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:27:41.874815 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/1dad823e240149edb30e592380249a13"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/1dad823e240149edb30e592380249a13"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_1dad823e240149edb30e592380249a13"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/1dad823e240149edb30e592380249a13"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:27:41.874815 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-ServersTestJSON-1529284206 Jul 16 21:27:41.874815 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: 1dad823e240149edb30e592380249a13 Jul 16 21:27:41.874815 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-ServersTestJSON-1529284206 Jul 16 21:27:41.874815 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-ServersTestJSON-1529284206-project Jul 16 21:27:41.874815 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:27:41.874815 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:27:41.874815 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: 027dcc48e4fd4c36a5004096ad5e43f8 Jul 16 21:27:41.874815 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-ServersTestJSON-1529284206-project Jul 16 21:27:41.874815 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 21:27:41.874815 ncdv-master-1 devstack@n-api.service[138680]: exception: Instance 6dcb4587-0e15-4a2d-b8f0-70b459fb6664 could not be found. Jul 16 21:27:41.874815 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': '6dcb4587-0e15-4a2d-b8f0-70b459fb6664'} Jul 16 21:27:41.874815 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:27:41.874815 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:27:41.874815 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:27:41.874815 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:27:41.874815 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:27:41.874815 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:27:41.874815 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:27:41.874815 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:27:41.874815 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:27:41.874815 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:27:41.874815 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 6dcb4587-0e15-4a2d-b8f0-70b459fb6664 could not be found. Jul 16 21:27:41.874815 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:27:41.874815 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:27:41.874815 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:27:41.874815 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:27:41.874815 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:27:41.874815 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:27:41.874815 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:27:41.874815 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:27:41.874815 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:27:41.874815 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:27:41.874815 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Jul 16 21:27:41.874815 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Jul 16 21:27:41.874815 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Jul 16 21:27:41.874815 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Jul 16 21:27:41.874815 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:27:41.874815 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:27:41.874815 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:27:41.874815 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:27:41.874815 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 6dcb4587-0e15-4a2d-b8f0-70b459fb6664 could not be found. Jul 16 21:27:41.880070 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:27:41.880070 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:27:41.880070 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:27:41.880070 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:27:41.880070 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:27:41.880070 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:27:41.880070 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:27:41.880070 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:27:41.880070 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 6dcb4587-0e15-4a2d-b8f0-70b459fb6664 could not be found. Jul 16 21:27:41.880070 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:27:41.880070 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-263cfdd2-3762-4b20-9220-c3741d6226bd tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Returning 404 to user: Instance 6dcb4587-0e15-4a2d-b8f0-70b459fb6664 could not be found. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:27:41.880070 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-263cfdd2-3762-4b20-9220-c3741d6226bd tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "DELETE /compute/v2.1/servers/6dcb4587-0e15-4a2d-b8f0-70b459fb6664" status: 404 len: 111 microversion: 2.1 time: 0.040419 Jul 16 21:27:41.880070 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2036/4082] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:27:41 2022] DELETE /compute/v2.1/servers/6dcb4587-0e15-4a2d-b8f0-70b459fb6664 => generated 111 bytes in 42 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:27:41.887573 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-615f6a5c-094b-485d-a760-47c083c39e64 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:27:41.888588 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-615f6a5c-094b-485d-a760-47c083c39e64 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: 6dcb4587-0e15-4a2d-b8f0-70b459fb6664] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:27:41.896120 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-615f6a5c-094b-485d-a760-47c083c39e64 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:41.896586 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-615f6a5c-094b-485d-a760-47c083c39e64 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:41.897373 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-615f6a5c-094b-485d-a760-47c083c39e64 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:41.897822 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-615f6a5c-094b-485d-a760-47c083c39e64 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:41.907960 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-ce639c4a-7294-43f5-b255-1c060f1039cf tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6771589864f442cbbfc8abf018abbd09', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bfd328e0f1446369531a23d15387cd6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:27:41.916492 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-615f6a5c-094b-485d-a760-47c083c39e64 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] HTTP exception thrown: Instance 6dcb4587-0e15-4a2d-b8f0-70b459fb6664 could not be found. Jul 16 21:27:41.916492 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-615f6a5c-094b-485d-a760-47c083c39e64 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Request method failure captured: Jul 16 21:27:41.916492 ncdv-master-1 devstack@n-api.service[138680]: request: GET /compute/v2.1/servers/6dcb4587-0e15-4a2d-b8f0-70b459fb6664 HTTP/1.1 Jul 16 21:27:41.916492 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:27:41.916492 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:27:41.916492 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:27:41.916492 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 0 Jul 16 21:27:41.916492 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:27:41.916492 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:27:41.916492 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:27:41.916492 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:27:41.916492 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0yxS9NCWgI1-O55Z_AF16JzmczB_p-s4P30aJMz-9zu9VxcHuIV00tsfB9hOBtHTZ_sI8GPeOOvUrqYZLgOBzCbOMr4ZO506_XIRjMCtHQ0umzvcB9V9iFSNFoiIe_b2jKm-OyHj1IB8YVBCAcwDKANlsCw5UaAoIABK4dLLFjaVpEo Jul 16 21:27:41.916492 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:27:41.916492 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:27:41.916492 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:27:41.916492 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:27:41.916492 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:27:41.916492 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:27:41.916492 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: 1dad823e240149edb30e592380249a13 Jul 16 21:27:41.916492 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-ServersTestJSON-1529284206 Jul 16 21:27:41.916492 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:27:41.916492 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:27:41.916492 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/1dad823e240149edb30e592380249a13"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/1dad823e240149edb30e592380249a13"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_1dad823e240149edb30e592380249a13"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/1dad823e240149edb30e592380249a13"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:27:41.916492 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-ServersTestJSON-1529284206 Jul 16 21:27:41.916492 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: 1dad823e240149edb30e592380249a13 Jul 16 21:27:41.916492 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-ServersTestJSON-1529284206 Jul 16 21:27:41.916492 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-ServersTestJSON-1529284206-project Jul 16 21:27:41.916492 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:27:41.916492 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:27:41.916492 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: 027dcc48e4fd4c36a5004096ad5e43f8 Jul 16 21:27:41.916492 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-ServersTestJSON-1529284206-project Jul 16 21:27:41.916492 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 21:27:41.916492 ncdv-master-1 devstack@n-api.service[138680]: exception: Instance 6dcb4587-0e15-4a2d-b8f0-70b459fb6664 could not be found. Jul 16 21:27:41.916492 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': '6dcb4587-0e15-4a2d-b8f0-70b459fb6664'} Jul 16 21:27:41.916492 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:27:41.916492 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:27:41.916492 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:27:41.916492 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:27:41.916492 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:27:41.916492 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:27:41.916492 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:27:41.916492 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:27:41.916492 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:27:41.916492 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:27:41.916492 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 6dcb4587-0e15-4a2d-b8f0-70b459fb6664 could not be found. Jul 16 21:27:41.916492 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:27:41.916492 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:27:41.916492 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:27:41.916492 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:27:41.916492 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:27:41.916492 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:27:41.916492 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:27:41.916492 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:27:41.916492 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:27:41.916492 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:27:41.916492 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:27:41.916492 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:27:41.916492 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:27:41.916492 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:27:41.916492 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:27:41.916492 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:27:41.920390 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 6dcb4587-0e15-4a2d-b8f0-70b459fb6664 could not be found. Jul 16 21:27:41.920390 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:27:41.920390 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:27:41.920390 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:27:41.920390 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:27:41.920390 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:27:41.920390 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:27:41.920390 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:27:41.920390 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:27:41.920390 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 6dcb4587-0e15-4a2d-b8f0-70b459fb6664 could not be found. Jul 16 21:27:41.920390 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:27:41.920390 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-615f6a5c-094b-485d-a760-47c083c39e64 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Returning 404 to user: Instance 6dcb4587-0e15-4a2d-b8f0-70b459fb6664 could not be found. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:27:41.920390 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-615f6a5c-094b-485d-a760-47c083c39e64 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "GET /compute/v2.1/servers/6dcb4587-0e15-4a2d-b8f0-70b459fb6664" status: 404 len: 111 microversion: 2.1 time: 0.034152 Jul 16 21:27:41.920390 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2037/4083] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:27:41 2022] GET /compute/v2.1/servers/6dcb4587-0e15-4a2d-b8f0-70b459fb6664 => generated 111 bytes in 39 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:27:41.925515 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-ff1133cb-9e65-4225-9b1a-a8ce5f6c2c02 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:27:41.926022 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-ff1133cb-9e65-4225-9b1a-a8ce5f6c2c02 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: c20f3279-68ae-4b04-8b87-cc16e5f73f1e] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:27:41.933269 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ff1133cb-9e65-4225-9b1a-a8ce5f6c2c02 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:41.934167 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ff1133cb-9e65-4225-9b1a-a8ce5f6c2c02 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:41.935250 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ff1133cb-9e65-4225-9b1a-a8ce5f6c2c02 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:41.935864 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ff1133cb-9e65-4225-9b1a-a8ce5f6c2c02 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:41.941140 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-ce639c4a-7294-43f5-b255-1c060f1039cf tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] 10.222.0.16 "GET /compute/v2.1/servers/0b67ad87-5ea8-4837-90f8-b3bc99134f3b" status: 200 len: 1272 microversion: 2.1 time: 0.127927 Jul 16 21:27:41.945117 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2047/4084] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:27:41 2022] GET /compute/v2.1/servers/0b67ad87-5ea8-4837-90f8-b3bc99134f3b => generated 1272 bytes in 141 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:27:41.952592 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-ff1133cb-9e65-4225-9b1a-a8ce5f6c2c02 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] HTTP exception thrown: Instance c20f3279-68ae-4b04-8b87-cc16e5f73f1e could not be found. Jul 16 21:27:41.955216 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-ff1133cb-9e65-4225-9b1a-a8ce5f6c2c02 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Request method failure captured: Jul 16 21:27:41.955216 ncdv-master-1 devstack@n-api.service[138680]: request: DELETE /compute/v2.1/servers/c20f3279-68ae-4b04-8b87-cc16e5f73f1e HTTP/1.1 Jul 16 21:27:41.955216 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:27:41.955216 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:27:41.955216 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:27:41.955216 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 0 Jul 16 21:27:41.955216 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:27:41.955216 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:27:41.955216 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:27:41.955216 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:27:41.955216 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0yxS9NCWgI1-O55Z_AF16JzmczB_p-s4P30aJMz-9zu9VxcHuIV00tsfB9hOBtHTZ_sI8GPeOOvUrqYZLgOBzCbOMr4ZO506_XIRjMCtHQ0umzvcB9V9iFSNFoiIe_b2jKm-OyHj1IB8YVBCAcwDKANlsCw5UaAoIABK4dLLFjaVpEo Jul 16 21:27:41.955216 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:27:41.955216 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:27:41.955216 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:27:41.955216 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:27:41.955216 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:27:41.955216 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:27:41.955216 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: 1dad823e240149edb30e592380249a13 Jul 16 21:27:41.955216 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-ServersTestJSON-1529284206 Jul 16 21:27:41.955216 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:27:41.955216 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:27:41.955216 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/1dad823e240149edb30e592380249a13"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/1dad823e240149edb30e592380249a13"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_1dad823e240149edb30e592380249a13"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/1dad823e240149edb30e592380249a13"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:27:41.955216 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-ServersTestJSON-1529284206 Jul 16 21:27:41.955216 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: 1dad823e240149edb30e592380249a13 Jul 16 21:27:41.955216 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-ServersTestJSON-1529284206 Jul 16 21:27:41.955216 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-ServersTestJSON-1529284206-project Jul 16 21:27:41.955216 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:27:41.955216 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:27:41.955216 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: 027dcc48e4fd4c36a5004096ad5e43f8 Jul 16 21:27:41.955216 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-ServersTestJSON-1529284206-project Jul 16 21:27:41.955216 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 21:27:41.955216 ncdv-master-1 devstack@n-api.service[138680]: exception: Instance c20f3279-68ae-4b04-8b87-cc16e5f73f1e could not be found. Jul 16 21:27:41.955216 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': 'c20f3279-68ae-4b04-8b87-cc16e5f73f1e'} Jul 16 21:27:41.955216 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:27:41.955216 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:27:41.955216 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:27:41.955216 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:27:41.955216 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:27:41.955216 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:27:41.955216 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:27:41.955216 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:27:41.955216 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:27:41.955216 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:27:41.955216 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance c20f3279-68ae-4b04-8b87-cc16e5f73f1e could not be found. Jul 16 21:27:41.955216 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:27:41.955216 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:27:41.955216 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:27:41.955216 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:27:41.955216 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:27:41.955216 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:27:41.955216 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:27:41.955216 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:27:41.955216 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:27:41.955216 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:27:41.955216 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Jul 16 21:27:41.955216 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Jul 16 21:27:41.955216 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Jul 16 21:27:41.955216 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Jul 16 21:27:41.955216 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:27:41.955216 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:27:41.955216 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:27:41.955216 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:27:41.955216 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance c20f3279-68ae-4b04-8b87-cc16e5f73f1e could not be found. Jul 16 21:27:41.959458 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:27:41.959458 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:27:41.959458 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:27:41.959458 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:27:41.959458 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:27:41.959458 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:27:41.959458 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:27:41.959458 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:27:41.959458 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance c20f3279-68ae-4b04-8b87-cc16e5f73f1e could not be found. Jul 16 21:27:41.959458 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:27:41.960825 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-ff1133cb-9e65-4225-9b1a-a8ce5f6c2c02 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Returning 404 to user: Instance c20f3279-68ae-4b04-8b87-cc16e5f73f1e could not be found. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:27:41.961710 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-ff1133cb-9e65-4225-9b1a-a8ce5f6c2c02 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "DELETE /compute/v2.1/servers/c20f3279-68ae-4b04-8b87-cc16e5f73f1e" status: 404 len: 111 microversion: 2.1 time: 0.040361 Jul 16 21:27:41.965562 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2038/4085] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:27:41 2022] DELETE /compute/v2.1/servers/c20f3279-68ae-4b04-8b87-cc16e5f73f1e => generated 111 bytes in 44 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:27:41.971530 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-6fff83b0-cd1c-43e7-9547-fd9fda754e14 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:27:41.973719 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-6fff83b0-cd1c-43e7-9547-fd9fda754e14 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: c20f3279-68ae-4b04-8b87-cc16e5f73f1e] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:27:41.980928 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6fff83b0-cd1c-43e7-9547-fd9fda754e14 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:41.981857 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6fff83b0-cd1c-43e7-9547-fd9fda754e14 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:41.983926 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6fff83b0-cd1c-43e7-9547-fd9fda754e14 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:41.984525 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6fff83b0-cd1c-43e7-9547-fd9fda754e14 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:42.010984 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-6fff83b0-cd1c-43e7-9547-fd9fda754e14 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] HTTP exception thrown: Instance c20f3279-68ae-4b04-8b87-cc16e5f73f1e could not be found. Jul 16 21:27:42.012745 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-6fff83b0-cd1c-43e7-9547-fd9fda754e14 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Request method failure captured: Jul 16 21:27:42.012745 ncdv-master-1 devstack@n-api.service[138681]: request: GET /compute/v2.1/servers/c20f3279-68ae-4b04-8b87-cc16e5f73f1e HTTP/1.1 Jul 16 21:27:42.012745 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:27:42.012745 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:27:42.012745 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:27:42.012745 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 21:27:42.012745 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:27:42.012745 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:27:42.012745 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:27:42.012745 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:27:42.012745 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0yxS9NCWgI1-O55Z_AF16JzmczB_p-s4P30aJMz-9zu9VxcHuIV00tsfB9hOBtHTZ_sI8GPeOOvUrqYZLgOBzCbOMr4ZO506_XIRjMCtHQ0umzvcB9V9iFSNFoiIe_b2jKm-OyHj1IB8YVBCAcwDKANlsCw5UaAoIABK4dLLFjaVpEo Jul 16 21:27:42.012745 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:27:42.012745 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:27:42.012745 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:27:42.012745 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:27:42.012745 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:27:42.012745 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:27:42.012745 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: 1dad823e240149edb30e592380249a13 Jul 16 21:27:42.012745 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-ServersTestJSON-1529284206 Jul 16 21:27:42.012745 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:27:42.012745 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:27:42.012745 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/1dad823e240149edb30e592380249a13"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/1dad823e240149edb30e592380249a13"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_1dad823e240149edb30e592380249a13"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/1dad823e240149edb30e592380249a13"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:27:42.012745 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-ServersTestJSON-1529284206 Jul 16 21:27:42.012745 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: 1dad823e240149edb30e592380249a13 Jul 16 21:27:42.012745 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-ServersTestJSON-1529284206 Jul 16 21:27:42.012745 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-ServersTestJSON-1529284206-project Jul 16 21:27:42.012745 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:27:42.012745 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:27:42.012745 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: 027dcc48e4fd4c36a5004096ad5e43f8 Jul 16 21:27:42.012745 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-ServersTestJSON-1529284206-project Jul 16 21:27:42.012745 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:27:42.012745 ncdv-master-1 devstack@n-api.service[138681]: exception: Instance c20f3279-68ae-4b04-8b87-cc16e5f73f1e could not be found. Jul 16 21:27:42.012745 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': 'c20f3279-68ae-4b04-8b87-cc16e5f73f1e'} Jul 16 21:27:42.012745 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:27:42.012745 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:27:42.012745 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:27:42.012745 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:27:42.012745 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:27:42.012745 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:27:42.012745 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:27:42.012745 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:27:42.012745 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:27:42.012745 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:27:42.012745 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance c20f3279-68ae-4b04-8b87-cc16e5f73f1e could not be found. Jul 16 21:27:42.012745 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:27:42.012745 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:27:42.012745 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:27:42.012745 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:27:42.012745 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:27:42.012745 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:27:42.012745 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:27:42.012745 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:27:42.012745 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:27:42.012745 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:27:42.012745 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:27:42.012745 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:27:42.012745 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:27:42.012745 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:27:42.012745 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:27:42.012745 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:27:42.012745 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance c20f3279-68ae-4b04-8b87-cc16e5f73f1e could not be found. Jul 16 21:27:42.012745 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:27:42.012745 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:27:42.012745 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:27:42.012745 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:27:42.019485 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:27:42.019485 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:27:42.019485 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:27:42.019485 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:27:42.019485 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance c20f3279-68ae-4b04-8b87-cc16e5f73f1e could not be found. Jul 16 21:27:42.019485 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:27:42.019485 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-6fff83b0-cd1c-43e7-9547-fd9fda754e14 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Returning 404 to user: Instance c20f3279-68ae-4b04-8b87-cc16e5f73f1e could not be found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:27:42.019485 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-6fff83b0-cd1c-43e7-9547-fd9fda754e14 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "GET /compute/v2.1/servers/c20f3279-68ae-4b04-8b87-cc16e5f73f1e" status: 404 len: 111 microversion: 2.1 time: 0.051520 Jul 16 21:27:42.028685 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-0b53896a-298c-4001-9ff3-bb8697d7c1b7 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:27:42.029265 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-0b53896a-298c-4001-9ff3-bb8697d7c1b7 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: 05358d0e-ed2f-443c-835c-f77cd929289e] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:27:42.034418 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2048/4086] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:27:41 2022] GET /compute/v2.1/servers/c20f3279-68ae-4b04-8b87-cc16e5f73f1e => generated 111 bytes in 67 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Jul 16 21:27:42.037769 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0b53896a-298c-4001-9ff3-bb8697d7c1b7 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:42.038480 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0b53896a-298c-4001-9ff3-bb8697d7c1b7 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:42.039304 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0b53896a-298c-4001-9ff3-bb8697d7c1b7 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:42.043797 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0b53896a-298c-4001-9ff3-bb8697d7c1b7 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:42.057165 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-0b53896a-298c-4001-9ff3-bb8697d7c1b7 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] HTTP exception thrown: Instance 05358d0e-ed2f-443c-835c-f77cd929289e could not be found. Jul 16 21:27:42.059052 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-0b53896a-298c-4001-9ff3-bb8697d7c1b7 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Request method failure captured: Jul 16 21:27:42.059052 ncdv-master-1 devstack@n-api.service[138680]: request: DELETE /compute/v2.1/servers/05358d0e-ed2f-443c-835c-f77cd929289e HTTP/1.1 Jul 16 21:27:42.059052 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:27:42.059052 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:27:42.059052 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:27:42.059052 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 0 Jul 16 21:27:42.059052 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:27:42.059052 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:27:42.059052 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:27:42.059052 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:27:42.059052 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0yxS9NCWgI1-O55Z_AF16JzmczB_p-s4P30aJMz-9zu9VxcHuIV00tsfB9hOBtHTZ_sI8GPeOOvUrqYZLgOBzCbOMr4ZO506_XIRjMCtHQ0umzvcB9V9iFSNFoiIe_b2jKm-OyHj1IB8YVBCAcwDKANlsCw5UaAoIABK4dLLFjaVpEo Jul 16 21:27:42.059052 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:27:42.059052 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:27:42.059052 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:27:42.059052 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:27:42.059052 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:27:42.059052 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:27:42.059052 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: 1dad823e240149edb30e592380249a13 Jul 16 21:27:42.059052 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-ServersTestJSON-1529284206 Jul 16 21:27:42.059052 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:27:42.059052 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:27:42.059052 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/1dad823e240149edb30e592380249a13"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/1dad823e240149edb30e592380249a13"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_1dad823e240149edb30e592380249a13"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/1dad823e240149edb30e592380249a13"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:27:42.059052 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-ServersTestJSON-1529284206 Jul 16 21:27:42.059052 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: 1dad823e240149edb30e592380249a13 Jul 16 21:27:42.059052 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-ServersTestJSON-1529284206 Jul 16 21:27:42.059052 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-ServersTestJSON-1529284206-project Jul 16 21:27:42.059052 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:27:42.059052 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:27:42.059052 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: 027dcc48e4fd4c36a5004096ad5e43f8 Jul 16 21:27:42.059052 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-ServersTestJSON-1529284206-project Jul 16 21:27:42.059052 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 21:27:42.059052 ncdv-master-1 devstack@n-api.service[138680]: exception: Instance 05358d0e-ed2f-443c-835c-f77cd929289e could not be found. Jul 16 21:27:42.059052 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': '05358d0e-ed2f-443c-835c-f77cd929289e'} Jul 16 21:27:42.059052 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:27:42.059052 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:27:42.059052 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:27:42.059052 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:27:42.059052 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:27:42.059052 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:27:42.059052 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:27:42.059052 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:27:42.059052 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:27:42.059052 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:27:42.059052 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 05358d0e-ed2f-443c-835c-f77cd929289e could not be found. Jul 16 21:27:42.059052 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:27:42.059052 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:27:42.059052 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:27:42.059052 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:27:42.059052 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:27:42.059052 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:27:42.059052 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:27:42.059052 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:27:42.059052 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:27:42.059052 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:27:42.059052 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Jul 16 21:27:42.059052 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Jul 16 21:27:42.059052 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Jul 16 21:27:42.059052 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Jul 16 21:27:42.059052 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:27:42.059052 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:27:42.059052 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:27:42.059052 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:27:42.059052 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 05358d0e-ed2f-443c-835c-f77cd929289e could not be found. Jul 16 21:27:42.063442 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:27:42.063442 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:27:42.063442 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:27:42.063442 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:27:42.063442 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:27:42.063442 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:27:42.063442 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:27:42.063442 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:27:42.063442 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 05358d0e-ed2f-443c-835c-f77cd929289e could not be found. Jul 16 21:27:42.063442 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:27:42.063442 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-0b53896a-298c-4001-9ff3-bb8697d7c1b7 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Returning 404 to user: Instance 05358d0e-ed2f-443c-835c-f77cd929289e could not be found. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:27:42.063442 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-0b53896a-298c-4001-9ff3-bb8697d7c1b7 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "DELETE /compute/v2.1/servers/05358d0e-ed2f-443c-835c-f77cd929289e" status: 404 len: 111 microversion: 2.1 time: 0.036474 Jul 16 21:27:42.075416 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-0904d953-4584-4a9c-ad7b-47b24dfaf1f9 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:27:42.076120 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-0904d953-4584-4a9c-ad7b-47b24dfaf1f9 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: 05358d0e-ed2f-443c-835c-f77cd929289e] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:27:42.081249 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2039/4087] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:27:42 2022] DELETE /compute/v2.1/servers/05358d0e-ed2f-443c-835c-f77cd929289e => generated 111 bytes in 56 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:27:42.082937 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0904d953-4584-4a9c-ad7b-47b24dfaf1f9 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:42.083359 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0904d953-4584-4a9c-ad7b-47b24dfaf1f9 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:42.084296 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0904d953-4584-4a9c-ad7b-47b24dfaf1f9 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:42.090739 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0904d953-4584-4a9c-ad7b-47b24dfaf1f9 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:42.116634 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-0904d953-4584-4a9c-ad7b-47b24dfaf1f9 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] HTTP exception thrown: Instance 05358d0e-ed2f-443c-835c-f77cd929289e could not be found. Jul 16 21:27:42.119218 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-0904d953-4584-4a9c-ad7b-47b24dfaf1f9 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Request method failure captured: Jul 16 21:27:42.119218 ncdv-master-1 devstack@n-api.service[138681]: request: GET /compute/v2.1/servers/05358d0e-ed2f-443c-835c-f77cd929289e HTTP/1.1 Jul 16 21:27:42.119218 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:27:42.119218 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:27:42.119218 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:27:42.119218 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 21:27:42.119218 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:27:42.119218 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:27:42.119218 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:27:42.119218 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:27:42.119218 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0yxS9NCWgI1-O55Z_AF16JzmczB_p-s4P30aJMz-9zu9VxcHuIV00tsfB9hOBtHTZ_sI8GPeOOvUrqYZLgOBzCbOMr4ZO506_XIRjMCtHQ0umzvcB9V9iFSNFoiIe_b2jKm-OyHj1IB8YVBCAcwDKANlsCw5UaAoIABK4dLLFjaVpEo Jul 16 21:27:42.119218 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:27:42.119218 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:27:42.119218 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:27:42.119218 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:27:42.119218 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:27:42.119218 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:27:42.119218 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: 1dad823e240149edb30e592380249a13 Jul 16 21:27:42.119218 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-ServersTestJSON-1529284206 Jul 16 21:27:42.119218 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:27:42.119218 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:27:42.119218 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/1dad823e240149edb30e592380249a13"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/1dad823e240149edb30e592380249a13"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_1dad823e240149edb30e592380249a13"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/1dad823e240149edb30e592380249a13"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:27:42.119218 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-ServersTestJSON-1529284206 Jul 16 21:27:42.119218 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: 1dad823e240149edb30e592380249a13 Jul 16 21:27:42.119218 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-ServersTestJSON-1529284206 Jul 16 21:27:42.119218 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-ServersTestJSON-1529284206-project Jul 16 21:27:42.119218 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:27:42.119218 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:27:42.119218 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: 027dcc48e4fd4c36a5004096ad5e43f8 Jul 16 21:27:42.119218 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-ServersTestJSON-1529284206-project Jul 16 21:27:42.119218 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:27:42.119218 ncdv-master-1 devstack@n-api.service[138681]: exception: Instance 05358d0e-ed2f-443c-835c-f77cd929289e could not be found. Jul 16 21:27:42.119218 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': '05358d0e-ed2f-443c-835c-f77cd929289e'} Jul 16 21:27:42.119218 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:27:42.119218 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:27:42.119218 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:27:42.119218 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:27:42.119218 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:27:42.119218 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:27:42.119218 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:27:42.119218 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:27:42.119218 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:27:42.119218 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:27:42.119218 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 05358d0e-ed2f-443c-835c-f77cd929289e could not be found. Jul 16 21:27:42.119218 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:27:42.119218 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:27:42.119218 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:27:42.119218 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:27:42.119218 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:27:42.119218 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:27:42.119218 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:27:42.119218 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:27:42.119218 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:27:42.119218 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:27:42.119218 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:27:42.119218 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:27:42.119218 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:27:42.119218 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:27:42.119218 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:27:42.119218 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:27:42.119218 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 05358d0e-ed2f-443c-835c-f77cd929289e could not be found. Jul 16 21:27:42.119218 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:27:42.119218 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:27:42.119218 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:27:42.119218 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:27:42.125105 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:27:42.125105 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:27:42.125105 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:27:42.125105 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:27:42.125105 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 05358d0e-ed2f-443c-835c-f77cd929289e could not be found. Jul 16 21:27:42.125105 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:27:42.128308 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-0904d953-4584-4a9c-ad7b-47b24dfaf1f9 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Returning 404 to user: Instance 05358d0e-ed2f-443c-835c-f77cd929289e could not be found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:27:42.130550 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-0904d953-4584-4a9c-ad7b-47b24dfaf1f9 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "GET /compute/v2.1/servers/05358d0e-ed2f-443c-835c-f77cd929289e" status: 404 len: 111 microversion: 2.1 time: 0.063445 Jul 16 21:27:42.135159 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2049/4088] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:27:42 2022] GET /compute/v2.1/servers/05358d0e-ed2f-443c-835c-f77cd929289e => generated 111 bytes in 68 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:27:42.145170 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-59b96b87-7d3c-4c2b-a34f-7c413df21a94 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:27:42.145170 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-59b96b87-7d3c-4c2b-a34f-7c413df21a94 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: c9901d0a-8a17-4e6c-ba98-6bd587d410e0] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:27:42.152239 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-59b96b87-7d3c-4c2b-a34f-7c413df21a94 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:42.152716 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-59b96b87-7d3c-4c2b-a34f-7c413df21a94 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:42.153798 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-59b96b87-7d3c-4c2b-a34f-7c413df21a94 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:42.154263 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-59b96b87-7d3c-4c2b-a34f-7c413df21a94 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:42.173805 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-59b96b87-7d3c-4c2b-a34f-7c413df21a94 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] HTTP exception thrown: Instance c9901d0a-8a17-4e6c-ba98-6bd587d410e0 could not be found. Jul 16 21:27:42.173805 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-59b96b87-7d3c-4c2b-a34f-7c413df21a94 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Request method failure captured: Jul 16 21:27:42.173805 ncdv-master-1 devstack@n-api.service[138680]: request: DELETE /compute/v2.1/servers/c9901d0a-8a17-4e6c-ba98-6bd587d410e0 HTTP/1.1 Jul 16 21:27:42.173805 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:27:42.173805 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:27:42.173805 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:27:42.173805 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 0 Jul 16 21:27:42.173805 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:27:42.173805 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:27:42.173805 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:27:42.173805 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:27:42.173805 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0yxS9NCWgI1-O55Z_AF16JzmczB_p-s4P30aJMz-9zu9VxcHuIV00tsfB9hOBtHTZ_sI8GPeOOvUrqYZLgOBzCbOMr4ZO506_XIRjMCtHQ0umzvcB9V9iFSNFoiIe_b2jKm-OyHj1IB8YVBCAcwDKANlsCw5UaAoIABK4dLLFjaVpEo Jul 16 21:27:42.173805 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:27:42.173805 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:27:42.173805 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:27:42.173805 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:27:42.173805 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:27:42.173805 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:27:42.173805 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: 1dad823e240149edb30e592380249a13 Jul 16 21:27:42.173805 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-ServersTestJSON-1529284206 Jul 16 21:27:42.173805 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:27:42.173805 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:27:42.173805 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/1dad823e240149edb30e592380249a13"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/1dad823e240149edb30e592380249a13"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_1dad823e240149edb30e592380249a13"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/1dad823e240149edb30e592380249a13"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:27:42.173805 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-ServersTestJSON-1529284206 Jul 16 21:27:42.173805 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: 1dad823e240149edb30e592380249a13 Jul 16 21:27:42.173805 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-ServersTestJSON-1529284206 Jul 16 21:27:42.173805 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-ServersTestJSON-1529284206-project Jul 16 21:27:42.173805 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:27:42.173805 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:27:42.173805 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: 027dcc48e4fd4c36a5004096ad5e43f8 Jul 16 21:27:42.173805 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-ServersTestJSON-1529284206-project Jul 16 21:27:42.173805 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 21:27:42.173805 ncdv-master-1 devstack@n-api.service[138680]: exception: Instance c9901d0a-8a17-4e6c-ba98-6bd587d410e0 could not be found. Jul 16 21:27:42.173805 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': 'c9901d0a-8a17-4e6c-ba98-6bd587d410e0'} Jul 16 21:27:42.173805 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:27:42.173805 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:27:42.173805 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:27:42.173805 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:27:42.173805 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:27:42.173805 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:27:42.173805 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:27:42.173805 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:27:42.173805 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:27:42.173805 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:27:42.173805 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance c9901d0a-8a17-4e6c-ba98-6bd587d410e0 could not be found. Jul 16 21:27:42.173805 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:27:42.173805 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:27:42.173805 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:27:42.173805 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:27:42.173805 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:27:42.173805 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:27:42.173805 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:27:42.173805 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:27:42.173805 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:27:42.173805 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:27:42.173805 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Jul 16 21:27:42.173805 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Jul 16 21:27:42.173805 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Jul 16 21:27:42.173805 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Jul 16 21:27:42.173805 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:27:42.173805 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:27:42.177636 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:27:42.177636 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:27:42.177636 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance c9901d0a-8a17-4e6c-ba98-6bd587d410e0 could not be found. Jul 16 21:27:42.177636 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:27:42.177636 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:27:42.177636 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:27:42.177636 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:27:42.177636 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:27:42.177636 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:27:42.177636 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:27:42.177636 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:27:42.177636 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance c9901d0a-8a17-4e6c-ba98-6bd587d410e0 could not be found. Jul 16 21:27:42.177636 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:27:42.177636 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-59b96b87-7d3c-4c2b-a34f-7c413df21a94 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Returning 404 to user: Instance c9901d0a-8a17-4e6c-ba98-6bd587d410e0 could not be found. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:27:42.177636 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-59b96b87-7d3c-4c2b-a34f-7c413df21a94 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "DELETE /compute/v2.1/servers/c9901d0a-8a17-4e6c-ba98-6bd587d410e0" status: 404 len: 111 microversion: 2.1 time: 0.036163 Jul 16 21:27:42.177636 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2040/4089] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:27:42 2022] DELETE /compute/v2.1/servers/c9901d0a-8a17-4e6c-ba98-6bd587d410e0 => generated 111 bytes in 37 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:27:42.185899 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-e7a4d403-2fd9-4575-9037-5a5a062c1a21 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:27:42.186632 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-e7a4d403-2fd9-4575-9037-5a5a062c1a21 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: c9901d0a-8a17-4e6c-ba98-6bd587d410e0] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:27:42.196743 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e7a4d403-2fd9-4575-9037-5a5a062c1a21 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:42.197408 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e7a4d403-2fd9-4575-9037-5a5a062c1a21 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:42.203182 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e7a4d403-2fd9-4575-9037-5a5a062c1a21 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:42.203182 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e7a4d403-2fd9-4575-9037-5a5a062c1a21 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:42.218074 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-e7a4d403-2fd9-4575-9037-5a5a062c1a21 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] HTTP exception thrown: Instance c9901d0a-8a17-4e6c-ba98-6bd587d410e0 could not be found. Jul 16 21:27:42.219729 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-e7a4d403-2fd9-4575-9037-5a5a062c1a21 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Request method failure captured: Jul 16 21:27:42.219729 ncdv-master-1 devstack@n-api.service[138681]: request: GET /compute/v2.1/servers/c9901d0a-8a17-4e6c-ba98-6bd587d410e0 HTTP/1.1 Jul 16 21:27:42.219729 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:27:42.219729 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:27:42.219729 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:27:42.219729 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 21:27:42.219729 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:27:42.219729 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:27:42.219729 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:27:42.219729 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:27:42.219729 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0yxS9NCWgI1-O55Z_AF16JzmczB_p-s4P30aJMz-9zu9VxcHuIV00tsfB9hOBtHTZ_sI8GPeOOvUrqYZLgOBzCbOMr4ZO506_XIRjMCtHQ0umzvcB9V9iFSNFoiIe_b2jKm-OyHj1IB8YVBCAcwDKANlsCw5UaAoIABK4dLLFjaVpEo Jul 16 21:27:42.219729 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:27:42.219729 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:27:42.219729 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:27:42.219729 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:27:42.219729 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:27:42.219729 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:27:42.219729 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: 1dad823e240149edb30e592380249a13 Jul 16 21:27:42.219729 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-ServersTestJSON-1529284206 Jul 16 21:27:42.219729 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:27:42.219729 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:27:42.219729 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/1dad823e240149edb30e592380249a13"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/1dad823e240149edb30e592380249a13"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_1dad823e240149edb30e592380249a13"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/1dad823e240149edb30e592380249a13"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:27:42.219729 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-ServersTestJSON-1529284206 Jul 16 21:27:42.219729 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: 1dad823e240149edb30e592380249a13 Jul 16 21:27:42.219729 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-ServersTestJSON-1529284206 Jul 16 21:27:42.219729 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-ServersTestJSON-1529284206-project Jul 16 21:27:42.219729 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:27:42.219729 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:27:42.219729 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: 027dcc48e4fd4c36a5004096ad5e43f8 Jul 16 21:27:42.219729 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-ServersTestJSON-1529284206-project Jul 16 21:27:42.219729 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:27:42.219729 ncdv-master-1 devstack@n-api.service[138681]: exception: Instance c9901d0a-8a17-4e6c-ba98-6bd587d410e0 could not be found. Jul 16 21:27:42.219729 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': 'c9901d0a-8a17-4e6c-ba98-6bd587d410e0'} Jul 16 21:27:42.219729 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:27:42.219729 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:27:42.219729 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:27:42.219729 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:27:42.219729 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:27:42.219729 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:27:42.219729 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:27:42.219729 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:27:42.219729 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:27:42.219729 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:27:42.219729 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance c9901d0a-8a17-4e6c-ba98-6bd587d410e0 could not be found. Jul 16 21:27:42.219729 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:27:42.219729 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:27:42.219729 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:27:42.219729 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:27:42.219729 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:27:42.219729 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:27:42.219729 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:27:42.219729 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:27:42.219729 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:27:42.219729 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:27:42.219729 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:27:42.219729 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:27:42.219729 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:27:42.219729 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:27:42.219729 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:27:42.219729 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:27:42.219729 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance c9901d0a-8a17-4e6c-ba98-6bd587d410e0 could not be found. Jul 16 21:27:42.219729 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:27:42.219729 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:27:42.219729 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:27:42.219729 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:27:42.223833 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:27:42.223833 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:27:42.223833 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:27:42.223833 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:27:42.223833 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance c9901d0a-8a17-4e6c-ba98-6bd587d410e0 could not be found. Jul 16 21:27:42.223833 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:27:42.223833 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-e7a4d403-2fd9-4575-9037-5a5a062c1a21 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Returning 404 to user: Instance c9901d0a-8a17-4e6c-ba98-6bd587d410e0 could not be found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:27:42.223833 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-e7a4d403-2fd9-4575-9037-5a5a062c1a21 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "GET /compute/v2.1/servers/c9901d0a-8a17-4e6c-ba98-6bd587d410e0" status: 404 len: 111 microversion: 2.1 time: 0.042122 Jul 16 21:27:42.232095 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-f3e1c735-10c5-4d16-ab97-44a4c59cdb76 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:27:42.232684 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-f3e1c735-10c5-4d16-ab97-44a4c59cdb76 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: 083a6d48-6623-437b-8649-92a693c68ebf] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:27:42.238857 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2050/4090] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:27:42 2022] GET /compute/v2.1/servers/c9901d0a-8a17-4e6c-ba98-6bd587d410e0 => generated 111 bytes in 57 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:27:42.241086 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f3e1c735-10c5-4d16-ab97-44a4c59cdb76 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:42.241553 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f3e1c735-10c5-4d16-ab97-44a4c59cdb76 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:42.242367 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f3e1c735-10c5-4d16-ab97-44a4c59cdb76 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:42.242818 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f3e1c735-10c5-4d16-ab97-44a4c59cdb76 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:42.271069 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-c062e070-914e-4ed6-95f3-6cfc48e29a99 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:27:42.272411 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-c062e070-914e-4ed6-95f3-6cfc48e29a99 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] [instance: cf38328b-40fb-4537-b524-aaeb0a0abfb4] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:27:42.277270 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-f3e1c735-10c5-4d16-ab97-44a4c59cdb76 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] HTTP exception thrown: Instance 083a6d48-6623-437b-8649-92a693c68ebf could not be found. Jul 16 21:27:42.280116 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c062e070-914e-4ed6-95f3-6cfc48e29a99 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:42.280850 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c062e070-914e-4ed6-95f3-6cfc48e29a99 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:42.282153 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c062e070-914e-4ed6-95f3-6cfc48e29a99 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:42.283505 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c062e070-914e-4ed6-95f3-6cfc48e29a99 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:42.285531 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-f3e1c735-10c5-4d16-ab97-44a4c59cdb76 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Request method failure captured: Jul 16 21:27:42.285531 ncdv-master-1 devstack@n-api.service[138680]: request: DELETE /compute/v2.1/servers/083a6d48-6623-437b-8649-92a693c68ebf HTTP/1.1 Jul 16 21:27:42.285531 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:27:42.285531 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:27:42.285531 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:27:42.285531 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 0 Jul 16 21:27:42.285531 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:27:42.285531 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:27:42.285531 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:27:42.285531 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:27:42.285531 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0yxS9NCWgI1-O55Z_AF16JzmczB_p-s4P30aJMz-9zu9VxcHuIV00tsfB9hOBtHTZ_sI8GPeOOvUrqYZLgOBzCbOMr4ZO506_XIRjMCtHQ0umzvcB9V9iFSNFoiIe_b2jKm-OyHj1IB8YVBCAcwDKANlsCw5UaAoIABK4dLLFjaVpEo Jul 16 21:27:42.285531 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:27:42.285531 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:27:42.285531 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:27:42.285531 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:27:42.285531 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:27:42.285531 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:27:42.285531 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: 1dad823e240149edb30e592380249a13 Jul 16 21:27:42.285531 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-ServersTestJSON-1529284206 Jul 16 21:27:42.285531 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:27:42.285531 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:27:42.285531 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/1dad823e240149edb30e592380249a13"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/1dad823e240149edb30e592380249a13"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_1dad823e240149edb30e592380249a13"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/1dad823e240149edb30e592380249a13"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:27:42.285531 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-ServersTestJSON-1529284206 Jul 16 21:27:42.285531 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: 1dad823e240149edb30e592380249a13 Jul 16 21:27:42.285531 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-ServersTestJSON-1529284206 Jul 16 21:27:42.285531 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-ServersTestJSON-1529284206-project Jul 16 21:27:42.285531 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:27:42.285531 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:27:42.285531 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: 027dcc48e4fd4c36a5004096ad5e43f8 Jul 16 21:27:42.285531 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-ServersTestJSON-1529284206-project Jul 16 21:27:42.285531 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 21:27:42.285531 ncdv-master-1 devstack@n-api.service[138680]: exception: Instance 083a6d48-6623-437b-8649-92a693c68ebf could not be found. Jul 16 21:27:42.285531 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': '083a6d48-6623-437b-8649-92a693c68ebf'} Jul 16 21:27:42.285531 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:27:42.285531 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:27:42.285531 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:27:42.285531 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:27:42.285531 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:27:42.285531 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:27:42.285531 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:27:42.285531 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:27:42.285531 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:27:42.285531 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:27:42.285531 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 083a6d48-6623-437b-8649-92a693c68ebf could not be found. Jul 16 21:27:42.285531 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:27:42.285531 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:27:42.285531 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:27:42.285531 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:27:42.285531 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:27:42.285531 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:27:42.285531 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:27:42.285531 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:27:42.285531 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:27:42.285531 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:27:42.285531 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Jul 16 21:27:42.285531 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Jul 16 21:27:42.285531 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Jul 16 21:27:42.285531 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Jul 16 21:27:42.285531 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:27:42.285531 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:27:42.285531 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:27:42.285531 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:27:42.285531 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 083a6d48-6623-437b-8649-92a693c68ebf could not be found. Jul 16 21:27:42.295988 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:27:42.295988 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:27:42.295988 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:27:42.295988 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:27:42.295988 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:27:42.295988 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:27:42.295988 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:27:42.295988 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:27:42.295988 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 083a6d48-6623-437b-8649-92a693c68ebf could not be found. Jul 16 21:27:42.295988 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:27:42.295988 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-f3e1c735-10c5-4d16-ab97-44a4c59cdb76 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Returning 404 to user: Instance 083a6d48-6623-437b-8649-92a693c68ebf could not be found. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:27:42.295988 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-f3e1c735-10c5-4d16-ab97-44a4c59cdb76 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "DELETE /compute/v2.1/servers/083a6d48-6623-437b-8649-92a693c68ebf" status: 404 len: 111 microversion: 2.1 time: 0.059599 Jul 16 21:27:42.295988 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2041/4091] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:27:42 2022] DELETE /compute/v2.1/servers/083a6d48-6623-437b-8649-92a693c68ebf => generated 111 bytes in 68 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:27:42.319921 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-28c53504-3061-40b0-8140-443c2398da35 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:27:42.324703 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-28c53504-3061-40b0-8140-443c2398da35 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: 083a6d48-6623-437b-8649-92a693c68ebf] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:27:42.341857 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-28c53504-3061-40b0-8140-443c2398da35 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:42.342280 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-28c53504-3061-40b0-8140-443c2398da35 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:42.349899 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-28c53504-3061-40b0-8140-443c2398da35 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:42.349899 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-28c53504-3061-40b0-8140-443c2398da35 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:42.397266 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-28c53504-3061-40b0-8140-443c2398da35 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] HTTP exception thrown: Instance 083a6d48-6623-437b-8649-92a693c68ebf could not be found. Jul 16 21:27:42.399129 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-28c53504-3061-40b0-8140-443c2398da35 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Request method failure captured: Jul 16 21:27:42.399129 ncdv-master-1 devstack@n-api.service[138680]: request: GET /compute/v2.1/servers/083a6d48-6623-437b-8649-92a693c68ebf HTTP/1.1 Jul 16 21:27:42.399129 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:27:42.399129 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:27:42.399129 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:27:42.399129 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 0 Jul 16 21:27:42.399129 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:27:42.399129 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:27:42.399129 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:27:42.399129 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:27:42.399129 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0yxS9NCWgI1-O55Z_AF16JzmczB_p-s4P30aJMz-9zu9VxcHuIV00tsfB9hOBtHTZ_sI8GPeOOvUrqYZLgOBzCbOMr4ZO506_XIRjMCtHQ0umzvcB9V9iFSNFoiIe_b2jKm-OyHj1IB8YVBCAcwDKANlsCw5UaAoIABK4dLLFjaVpEo Jul 16 21:27:42.399129 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:27:42.399129 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:27:42.399129 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:27:42.399129 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:27:42.399129 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:27:42.399129 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:27:42.399129 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: 1dad823e240149edb30e592380249a13 Jul 16 21:27:42.399129 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-ServersTestJSON-1529284206 Jul 16 21:27:42.399129 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:27:42.399129 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:27:42.399129 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/1dad823e240149edb30e592380249a13"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/1dad823e240149edb30e592380249a13"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_1dad823e240149edb30e592380249a13"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/1dad823e240149edb30e592380249a13"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:27:42.399129 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-ServersTestJSON-1529284206 Jul 16 21:27:42.399129 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: 1dad823e240149edb30e592380249a13 Jul 16 21:27:42.399129 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-ServersTestJSON-1529284206 Jul 16 21:27:42.399129 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-ServersTestJSON-1529284206-project Jul 16 21:27:42.399129 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:27:42.399129 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:27:42.399129 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: 027dcc48e4fd4c36a5004096ad5e43f8 Jul 16 21:27:42.399129 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-ServersTestJSON-1529284206-project Jul 16 21:27:42.399129 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 21:27:42.399129 ncdv-master-1 devstack@n-api.service[138680]: exception: Instance 083a6d48-6623-437b-8649-92a693c68ebf could not be found. Jul 16 21:27:42.399129 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': '083a6d48-6623-437b-8649-92a693c68ebf'} Jul 16 21:27:42.399129 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:27:42.399129 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:27:42.399129 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:27:42.399129 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:27:42.399129 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:27:42.399129 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:27:42.399129 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:27:42.399129 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:27:42.399129 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:27:42.399129 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:27:42.399129 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 083a6d48-6623-437b-8649-92a693c68ebf could not be found. Jul 16 21:27:42.399129 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:27:42.399129 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:27:42.399129 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:27:42.399129 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:27:42.399129 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:27:42.399129 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:27:42.399129 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:27:42.399129 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:27:42.399129 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:27:42.399129 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:27:42.399129 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:27:42.399129 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:27:42.399129 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:27:42.399129 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:27:42.399129 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:27:42.399129 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:27:42.399129 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 083a6d48-6623-437b-8649-92a693c68ebf could not be found. Jul 16 21:27:42.399129 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:27:42.399129 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:27:42.399129 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:27:42.399129 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:27:42.402836 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:27:42.402836 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:27:42.402836 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:27:42.402836 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:27:42.402836 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 083a6d48-6623-437b-8649-92a693c68ebf could not be found. Jul 16 21:27:42.402836 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:27:42.402836 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-28c53504-3061-40b0-8140-443c2398da35 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Returning 404 to user: Instance 083a6d48-6623-437b-8649-92a693c68ebf could not be found. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:27:42.403715 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-28c53504-3061-40b0-8140-443c2398da35 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "GET /compute/v2.1/servers/083a6d48-6623-437b-8649-92a693c68ebf" status: 404 len: 111 microversion: 2.1 time: 0.107394 Jul 16 21:27:42.408302 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2042/4092] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:27:42 2022] GET /compute/v2.1/servers/083a6d48-6623-437b-8649-92a693c68ebf => generated 111 bytes in 112 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:27:42.419602 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-5e7d026e-654c-4d5d-8442-8bfdf4dc8321 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:27:42.422948 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-5e7d026e-654c-4d5d-8442-8bfdf4dc8321 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: b442581d-3a9f-4a24-9fae-c2abf958b2bd] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:27:42.435457 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5e7d026e-654c-4d5d-8442-8bfdf4dc8321 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:42.438561 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5e7d026e-654c-4d5d-8442-8bfdf4dc8321 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:42.440071 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5e7d026e-654c-4d5d-8442-8bfdf4dc8321 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:42.440650 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5e7d026e-654c-4d5d-8442-8bfdf4dc8321 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:42.473127 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-5e7d026e-654c-4d5d-8442-8bfdf4dc8321 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] HTTP exception thrown: Instance b442581d-3a9f-4a24-9fae-c2abf958b2bd could not be found. Jul 16 21:27:42.474717 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-5e7d026e-654c-4d5d-8442-8bfdf4dc8321 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Request method failure captured: Jul 16 21:27:42.474717 ncdv-master-1 devstack@n-api.service[138680]: request: DELETE /compute/v2.1/servers/b442581d-3a9f-4a24-9fae-c2abf958b2bd HTTP/1.1 Jul 16 21:27:42.474717 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:27:42.474717 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:27:42.474717 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:27:42.474717 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 0 Jul 16 21:27:42.474717 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:27:42.474717 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:27:42.474717 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:27:42.474717 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:27:42.474717 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0yxS9NCWgI1-O55Z_AF16JzmczB_p-s4P30aJMz-9zu9VxcHuIV00tsfB9hOBtHTZ_sI8GPeOOvUrqYZLgOBzCbOMr4ZO506_XIRjMCtHQ0umzvcB9V9iFSNFoiIe_b2jKm-OyHj1IB8YVBCAcwDKANlsCw5UaAoIABK4dLLFjaVpEo Jul 16 21:27:42.474717 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:27:42.474717 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:27:42.474717 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:27:42.474717 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:27:42.474717 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:27:42.474717 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:27:42.474717 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: 1dad823e240149edb30e592380249a13 Jul 16 21:27:42.474717 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-ServersTestJSON-1529284206 Jul 16 21:27:42.474717 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:27:42.474717 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:27:42.474717 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/1dad823e240149edb30e592380249a13"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/1dad823e240149edb30e592380249a13"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_1dad823e240149edb30e592380249a13"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/1dad823e240149edb30e592380249a13"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:27:42.474717 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-ServersTestJSON-1529284206 Jul 16 21:27:42.474717 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: 1dad823e240149edb30e592380249a13 Jul 16 21:27:42.474717 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-ServersTestJSON-1529284206 Jul 16 21:27:42.474717 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-ServersTestJSON-1529284206-project Jul 16 21:27:42.474717 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:27:42.474717 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:27:42.474717 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: 027dcc48e4fd4c36a5004096ad5e43f8 Jul 16 21:27:42.474717 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-ServersTestJSON-1529284206-project Jul 16 21:27:42.474717 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 21:27:42.474717 ncdv-master-1 devstack@n-api.service[138680]: exception: Instance b442581d-3a9f-4a24-9fae-c2abf958b2bd could not be found. Jul 16 21:27:42.474717 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': 'b442581d-3a9f-4a24-9fae-c2abf958b2bd'} Jul 16 21:27:42.474717 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:27:42.474717 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:27:42.474717 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:27:42.474717 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:27:42.474717 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:27:42.474717 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:27:42.474717 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:27:42.474717 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:27:42.474717 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:27:42.474717 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:27:42.474717 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance b442581d-3a9f-4a24-9fae-c2abf958b2bd could not be found. Jul 16 21:27:42.474717 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:27:42.474717 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:27:42.474717 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:27:42.474717 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:27:42.474717 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:27:42.474717 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:27:42.474717 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:27:42.474717 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:27:42.474717 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:27:42.474717 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:27:42.474717 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Jul 16 21:27:42.474717 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Jul 16 21:27:42.474717 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Jul 16 21:27:42.474717 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Jul 16 21:27:42.474717 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:27:42.474717 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:27:42.474717 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:27:42.474717 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:27:42.474717 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance b442581d-3a9f-4a24-9fae-c2abf958b2bd could not be found. Jul 16 21:27:42.481571 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:27:42.481571 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:27:42.481571 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:27:42.481571 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:27:42.481571 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:27:42.481571 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:27:42.481571 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:27:42.481571 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:27:42.481571 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance b442581d-3a9f-4a24-9fae-c2abf958b2bd could not be found. Jul 16 21:27:42.481571 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:27:42.488282 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-5e7d026e-654c-4d5d-8442-8bfdf4dc8321 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Returning 404 to user: Instance b442581d-3a9f-4a24-9fae-c2abf958b2bd could not be found. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:27:42.489178 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-5e7d026e-654c-4d5d-8442-8bfdf4dc8321 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "DELETE /compute/v2.1/servers/b442581d-3a9f-4a24-9fae-c2abf958b2bd" status: 404 len: 111 microversion: 2.1 time: 0.079162 Jul 16 21:27:42.491970 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2043/4093] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:27:42 2022] DELETE /compute/v2.1/servers/b442581d-3a9f-4a24-9fae-c2abf958b2bd => generated 111 bytes in 81 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:27:42.497731 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-ca2318e1-ed5b-46f6-b925-cb5c373f4cc6 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:27:42.498523 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-ca2318e1-ed5b-46f6-b925-cb5c373f4cc6 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] [instance: c2e32bd9-9d80-427b-bb6d-6274fd48173a] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:27:42.506936 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ca2318e1-ed5b-46f6-b925-cb5c373f4cc6 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:42.508612 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ca2318e1-ed5b-46f6-b925-cb5c373f4cc6 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:42.509620 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ca2318e1-ed5b-46f6-b925-cb5c373f4cc6 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:42.510884 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ca2318e1-ed5b-46f6-b925-cb5c373f4cc6 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:42.645734 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.objects.instance [None req-ca2318e1-ed5b-46f6-b925-cb5c373f4cc6 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lazy-loading 'services' on Instance uuid c2e32bd9-9d80-427b-bb6d-6274fd48173a {{(pid=138680) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:27:42.758143 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-c062e070-914e-4ed6-95f3-6cfc48e29a99 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '029c76a0ba614b5880128c4309902c67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3b214f5044c7445592e177835a5d18fc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:27:42.769566 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-c062e070-914e-4ed6-95f3-6cfc48e29a99 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] 10.222.0.16 "GET /compute/v2.1/servers/cf38328b-40fb-4537-b524-aaeb0a0abfb4" status: 200 len: 1521 microversion: 2.1 time: 0.503139 Jul 16 21:27:42.771356 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2051/4094] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:27:42 2022] GET /compute/v2.1/servers/cf38328b-40fb-4537-b524-aaeb0a0abfb4 => generated 1521 bytes in 506 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:27:42.793500 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-fd55f7f8-a479-4a74-a479-65eb515a208d tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:27:42.795307 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-fd55f7f8-a479-4a74-a479-65eb515a208d tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: b442581d-3a9f-4a24-9fae-c2abf958b2bd] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:27:42.808332 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fd55f7f8-a479-4a74-a479-65eb515a208d tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:42.809394 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fd55f7f8-a479-4a74-a479-65eb515a208d tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:42.811683 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fd55f7f8-a479-4a74-a479-65eb515a208d tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:42.813368 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fd55f7f8-a479-4a74-a479-65eb515a208d tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:42.847111 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-fd55f7f8-a479-4a74-a479-65eb515a208d tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] HTTP exception thrown: Instance b442581d-3a9f-4a24-9fae-c2abf958b2bd could not be found. Jul 16 21:27:42.851727 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-fd55f7f8-a479-4a74-a479-65eb515a208d tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Request method failure captured: Jul 16 21:27:42.851727 ncdv-master-1 devstack@n-api.service[138681]: request: GET /compute/v2.1/servers/b442581d-3a9f-4a24-9fae-c2abf958b2bd HTTP/1.1 Jul 16 21:27:42.851727 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:27:42.851727 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:27:42.851727 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:27:42.851727 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 21:27:42.851727 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:27:42.851727 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:27:42.851727 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:27:42.851727 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:27:42.851727 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0yxS9NCWgI1-O55Z_AF16JzmczB_p-s4P30aJMz-9zu9VxcHuIV00tsfB9hOBtHTZ_sI8GPeOOvUrqYZLgOBzCbOMr4ZO506_XIRjMCtHQ0umzvcB9V9iFSNFoiIe_b2jKm-OyHj1IB8YVBCAcwDKANlsCw5UaAoIABK4dLLFjaVpEo Jul 16 21:27:42.851727 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:27:42.851727 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:27:42.851727 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:27:42.851727 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:27:42.851727 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:27:42.851727 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:27:42.851727 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: 1dad823e240149edb30e592380249a13 Jul 16 21:27:42.851727 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-ServersTestJSON-1529284206 Jul 16 21:27:42.851727 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:27:42.851727 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:27:42.851727 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/1dad823e240149edb30e592380249a13"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/1dad823e240149edb30e592380249a13"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_1dad823e240149edb30e592380249a13"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/1dad823e240149edb30e592380249a13"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:27:42.851727 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-ServersTestJSON-1529284206 Jul 16 21:27:42.851727 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: 1dad823e240149edb30e592380249a13 Jul 16 21:27:42.851727 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-ServersTestJSON-1529284206 Jul 16 21:27:42.851727 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-ServersTestJSON-1529284206-project Jul 16 21:27:42.851727 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:27:42.851727 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:27:42.851727 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: 027dcc48e4fd4c36a5004096ad5e43f8 Jul 16 21:27:42.851727 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-ServersTestJSON-1529284206-project Jul 16 21:27:42.851727 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:27:42.851727 ncdv-master-1 devstack@n-api.service[138681]: exception: Instance b442581d-3a9f-4a24-9fae-c2abf958b2bd could not be found. Jul 16 21:27:42.851727 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': 'b442581d-3a9f-4a24-9fae-c2abf958b2bd'} Jul 16 21:27:42.851727 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:27:42.851727 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:27:42.851727 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:27:42.851727 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:27:42.851727 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:27:42.851727 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:27:42.851727 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:27:42.851727 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:27:42.851727 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:27:42.851727 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:27:42.851727 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance b442581d-3a9f-4a24-9fae-c2abf958b2bd could not be found. Jul 16 21:27:42.851727 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:27:42.851727 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:27:42.851727 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:27:42.851727 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:27:42.851727 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:27:42.851727 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:27:42.851727 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:27:42.851727 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:27:42.851727 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:27:42.851727 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:27:42.851727 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:27:42.851727 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:27:42.851727 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:27:42.851727 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:27:42.851727 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:27:42.851727 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:27:42.851727 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance b442581d-3a9f-4a24-9fae-c2abf958b2bd could not be found. Jul 16 21:27:42.851727 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:27:42.851727 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:27:42.851727 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:27:42.851727 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:27:42.855945 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:27:42.855945 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:27:42.855945 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:27:42.855945 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:27:42.855945 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance b442581d-3a9f-4a24-9fae-c2abf958b2bd could not be found. Jul 16 21:27:42.855945 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:27:42.855945 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-fd55f7f8-a479-4a74-a479-65eb515a208d tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Returning 404 to user: Instance b442581d-3a9f-4a24-9fae-c2abf958b2bd could not be found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:27:42.855945 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-fd55f7f8-a479-4a74-a479-65eb515a208d tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "GET /compute/v2.1/servers/b442581d-3a9f-4a24-9fae-c2abf958b2bd" status: 404 len: 111 microversion: 2.1 time: 0.070984 Jul 16 21:27:42.858428 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2052/4095] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:27:42 2022] GET /compute/v2.1/servers/b442581d-3a9f-4a24-9fae-c2abf958b2bd => generated 111 bytes in 87 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:27:42.868571 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-ca2318e1-ed5b-46f6-b925-cb5c373f4cc6 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] 10.222.0.16 "DELETE /compute/v2.1/servers/c2e32bd9-9d80-427b-bb6d-6274fd48173a/os-volume_attachments/685a896a-62d9-4253-ae50-c46b14a0daf6" status: 202 len: 0 microversion: 2.1 time: 0.375021 Jul 16 21:27:42.868810 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-ba62b629-8e32-4094-aafd-2b65416028dc tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:27:42.870362 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2044/4096] 10.222.0.16 () {58 vars in 1415 bytes} [Sat Jul 16 21:27:42 2022] DELETE /compute/v2.1/servers/c2e32bd9-9d80-427b-bb6d-6274fd48173a/os-volume_attachments/685a896a-62d9-4253-ae50-c46b14a0daf6 => generated 0 bytes in 378 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 21:27:42.870909 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-ba62b629-8e32-4094-aafd-2b65416028dc tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: e6774019-149e-4804-b0ab-e3fcd137851e] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:27:42.883385 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-c982f5c0-055b-4945-8336-48d8d7077e94 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:27:42.886695 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-c982f5c0-055b-4945-8336-48d8d7077e94 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] [instance: c2e32bd9-9d80-427b-bb6d-6274fd48173a] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:27:42.894935 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c982f5c0-055b-4945-8336-48d8d7077e94 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:42.895663 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c982f5c0-055b-4945-8336-48d8d7077e94 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:42.896600 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c982f5c0-055b-4945-8336-48d8d7077e94 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:42.897095 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c982f5c0-055b-4945-8336-48d8d7077e94 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:42.910715 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ba62b629-8e32-4094-aafd-2b65416028dc tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:42.912699 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ba62b629-8e32-4094-aafd-2b65416028dc tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:42.913775 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ba62b629-8e32-4094-aafd-2b65416028dc tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:42.919941 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ba62b629-8e32-4094-aafd-2b65416028dc tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:42.949243 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-c982f5c0-055b-4945-8336-48d8d7077e94 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] 10.222.0.16 "GET /compute/v2.1/servers/c2e32bd9-9d80-427b-bb6d-6274fd48173a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.073329 Jul 16 21:27:42.950688 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2045/4097] 10.222.0.16 () {58 vars in 1301 bytes} [Sat Jul 16 21:27:42 2022] GET /compute/v2.1/servers/c2e32bd9-9d80-427b-bb6d-6274fd48173a/os-volume_attachments => generated 197 bytes in 76 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:27:42.963859 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-1d1dbedc-9ec7-4cd5-b904-5d2b6e429b79 tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:27:42.972294 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-1d1dbedc-9ec7-4cd5-b904-5d2b6e429b79 tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] [instance: 0b67ad87-5ea8-4837-90f8-b3bc99134f3b] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:27:42.983279 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1d1dbedc-9ec7-4cd5-b904-5d2b6e429b79 tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:42.983738 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1d1dbedc-9ec7-4cd5-b904-5d2b6e429b79 tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:42.998623 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1d1dbedc-9ec7-4cd5-b904-5d2b6e429b79 tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:43.000998 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1d1dbedc-9ec7-4cd5-b904-5d2b6e429b79 tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.015s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:43.008875 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-ba62b629-8e32-4094-aafd-2b65416028dc tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: e6774019-149e-4804-b0ab-e3fcd137851e] Going to try to terminate instance {{(pid=138681) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 16 21:27:43.165413 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-1d1dbedc-9ec7-4cd5-b904-5d2b6e429b79 tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6771589864f442cbbfc8abf018abbd09', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bfd328e0f1446369531a23d15387cd6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:27:43.175819 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-1d1dbedc-9ec7-4cd5-b904-5d2b6e429b79 tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] 10.222.0.16 "GET /compute/v2.1/servers/0b67ad87-5ea8-4837-90f8-b3bc99134f3b" status: 200 len: 1272 microversion: 2.1 time: 0.216595 Jul 16 21:27:43.181700 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2046/4098] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:27:42 2022] GET /compute/v2.1/servers/0b67ad87-5ea8-4837-90f8-b3bc99134f3b => generated 1272 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:27:43.472090 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-ba62b629-8e32-4094-aafd-2b65416028dc tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "DELETE /compute/v2.1/servers/e6774019-149e-4804-b0ab-e3fcd137851e" status: 204 len: 0 microversion: 2.1 time: 0.607108 Jul 16 21:27:43.473693 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2053/4099] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:27:42 2022] DELETE /compute/v2.1/servers/e6774019-149e-4804-b0ab-e3fcd137851e => generated 0 bytes in 610 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Jul 16 21:27:43.481482 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-b76ff75d-945b-4102-8ad8-6c5ed9ed52ca tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:27:43.482401 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-b76ff75d-945b-4102-8ad8-6c5ed9ed52ca tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: e6774019-149e-4804-b0ab-e3fcd137851e] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:27:43.496718 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b76ff75d-945b-4102-8ad8-6c5ed9ed52ca tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:43.499102 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b76ff75d-945b-4102-8ad8-6c5ed9ed52ca tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:43.505818 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b76ff75d-945b-4102-8ad8-6c5ed9ed52ca tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:43.506790 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b76ff75d-945b-4102-8ad8-6c5ed9ed52ca tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:43.792068 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-89f812b7-56d7-42e5-96e3-466b35220713 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:27:43.793121 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-89f812b7-56d7-42e5-96e3-466b35220713 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] [instance: cf38328b-40fb-4537-b524-aaeb0a0abfb4] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:27:43.811116 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-89f812b7-56d7-42e5-96e3-466b35220713 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:43.814611 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-89f812b7-56d7-42e5-96e3-466b35220713 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:43.888277 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-89f812b7-56d7-42e5-96e3-466b35220713 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:43.896611 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-b76ff75d-945b-4102-8ad8-6c5ed9ed52ca tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '027dcc48e4fd4c36a5004096ad5e43f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dad823e240149edb30e592380249a13', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:27:43.902992 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-89f812b7-56d7-42e5-96e3-466b35220713 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.015s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:43.922274 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-b76ff75d-945b-4102-8ad8-6c5ed9ed52ca tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "GET /compute/v2.1/servers/e6774019-149e-4804-b0ab-e3fcd137851e" status: 200 len: 1479 microversion: 2.1 time: 0.443082 Jul 16 21:27:43.928936 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2047/4100] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:27:43 2022] GET /compute/v2.1/servers/e6774019-149e-4804-b0ab-e3fcd137851e => generated 1479 bytes in 452 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:27:43.963106 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-ad99f44d-eef6-4a1a-8ab5-1b293c607a7c tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:27:43.964673 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-ad99f44d-eef6-4a1a-8ab5-1b293c607a7c tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] [instance: c2e32bd9-9d80-427b-bb6d-6274fd48173a] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:27:43.978190 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ad99f44d-eef6-4a1a-8ab5-1b293c607a7c tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:43.988148 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ad99f44d-eef6-4a1a-8ab5-1b293c607a7c tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.010s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:43.989111 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ad99f44d-eef6-4a1a-8ab5-1b293c607a7c tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:43.991977 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ad99f44d-eef6-4a1a-8ab5-1b293c607a7c tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:44.095843 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:27:44.128238 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-ad99f44d-eef6-4a1a-8ab5-1b293c607a7c tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] 10.222.0.16 "GET /compute/v2.1/servers/c2e32bd9-9d80-427b-bb6d-6274fd48173a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.167663 Jul 16 21:27:44.129127 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2048/4101] 10.222.0.16 () {58 vars in 1301 bytes} [Sat Jul 16 21:27:43 2022] GET /compute/v2.1/servers/c2e32bd9-9d80-427b-bb6d-6274fd48173a/os-volume_attachments => generated 197 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 16 21:27:44.192150 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-d64a1878-bacf-4abe-a876-17f651e9c625 tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:27:44.193265 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-d64a1878-bacf-4abe-a876-17f651e9c625 tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] [instance: 0b67ad87-5ea8-4837-90f8-b3bc99134f3b] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:27:44.205878 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d64a1878-bacf-4abe-a876-17f651e9c625 tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:44.207562 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d64a1878-bacf-4abe-a876-17f651e9c625 tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:44.208786 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d64a1878-bacf-4abe-a876-17f651e9c625 tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:44.209515 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d64a1878-bacf-4abe-a876-17f651e9c625 tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:44.433410 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-89f812b7-56d7-42e5-96e3-466b35220713 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '029c76a0ba614b5880128c4309902c67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3b214f5044c7445592e177835a5d18fc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:27:44.452990 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-89f812b7-56d7-42e5-96e3-466b35220713 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] 10.222.0.16 "GET /compute/v2.1/servers/cf38328b-40fb-4537-b524-aaeb0a0abfb4" status: 200 len: 1521 microversion: 2.1 time: 0.656634 Jul 16 21:27:44.452990 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2054/4102] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:27:43 2022] GET /compute/v2.1/servers/cf38328b-40fb-4537-b524-aaeb0a0abfb4 => generated 1521 bytes in 660 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:27:44.561896 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-d64a1878-bacf-4abe-a876-17f651e9c625 tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6771589864f442cbbfc8abf018abbd09', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bfd328e0f1446369531a23d15387cd6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:27:44.570463 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-d64a1878-bacf-4abe-a876-17f651e9c625 tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] 10.222.0.16 "GET /compute/v2.1/servers/0b67ad87-5ea8-4837-90f8-b3bc99134f3b" status: 200 len: 1314 microversion: 2.1 time: 0.382402 Jul 16 21:27:44.571122 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2049/4103] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:27:44 2022] GET /compute/v2.1/servers/0b67ad87-5ea8-4837-90f8-b3bc99134f3b => generated 1314 bytes in 384 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:27:44.946375 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-8ff7e600-f874-4be3-9085-f084949f242e tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:27:44.947087 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-8ff7e600-f874-4be3-9085-f084949f242e tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: e6774019-149e-4804-b0ab-e3fcd137851e] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:27:44.955903 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8ff7e600-f874-4be3-9085-f084949f242e tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:44.956603 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8ff7e600-f874-4be3-9085-f084949f242e tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:44.957653 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8ff7e600-f874-4be3-9085-f084949f242e tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:44.958241 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8ff7e600-f874-4be3-9085-f084949f242e tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:45.140816 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-7e60b462-098a-4aaf-be26-4b95db38665d tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:27:45.142405 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-7e60b462-098a-4aaf-be26-4b95db38665d tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] [instance: c2e32bd9-9d80-427b-bb6d-6274fd48173a] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:27:45.157970 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7e60b462-098a-4aaf-be26-4b95db38665d tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:45.158823 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7e60b462-098a-4aaf-be26-4b95db38665d tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:45.160007 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7e60b462-098a-4aaf-be26-4b95db38665d tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:45.161657 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7e60b462-098a-4aaf-be26-4b95db38665d tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:45.278589 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-7e60b462-098a-4aaf-be26-4b95db38665d tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] 10.222.0.16 "GET /compute/v2.1/servers/c2e32bd9-9d80-427b-bb6d-6274fd48173a/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.120501 Jul 16 21:27:45.279929 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2050/4104] 10.222.0.16 () {58 vars in 1301 bytes} [Sat Jul 16 21:27:45 2022] GET /compute/v2.1/servers/c2e32bd9-9d80-427b-bb6d-6274fd48173a/os-volume_attachments => generated 25 bytes in 144 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Jul 16 21:27:45.425609 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-69c71ff3-f66b-4372-8a66-4b915095ea1b tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Action: 'action', calling method: >, body: {"reboot": {"type": "HARD"}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:27:45.428996 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-69c71ff3-f66b-4372-8a66-4b915095ea1b tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] [instance: 867b2ab0-9301-414b-b045-85057d642971] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:27:45.438897 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-69c71ff3-f66b-4372-8a66-4b915095ea1b tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:45.443722 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-69c71ff3-f66b-4372-8a66-4b915095ea1b tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:45.449907 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-69c71ff3-f66b-4372-8a66-4b915095ea1b tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:45.450468 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-69c71ff3-f66b-4372-8a66-4b915095ea1b tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:45.541083 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-69c71ff3-f66b-4372-8a66-4b915095ea1b tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] HTTP exception thrown: Cannot 'reboot' instance 867b2ab0-9301-414b-b045-85057d642971 while it is in vm_state rescued Jul 16 21:27:45.543651 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-69c71ff3-f66b-4372-8a66-4b915095ea1b tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Request method failure captured: Jul 16 21:27:45.543651 ncdv-master-1 devstack@n-api.service[138680]: request: POST /compute/v2.1/servers/867b2ab0-9301-414b-b045-85057d642971/action HTTP/1.1 Jul 16 21:27:45.543651 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:27:45.543651 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:27:45.543651 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:27:45.543651 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 28 Jul 16 21:27:45.543651 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:27:45.543651 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:27:45.543651 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:27:45.543651 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:27:45.543651 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0yxlBvXdj6P284bimRWVNOZAMmAVjIOasTJ1njzyaS12Nm5AKzVbxoKHi72FFVSaWe28lVKwWFtzjqUTy3NQWDNW-obFHfOCoFCCR4GNQ-_ruXI6NnMsKND6oz7nIdWgqUgB4Gac48-MGLdaneUoqvnHJrpsJjhWmKgGoKbdR8M7y0I Jul 16 21:27:45.543651 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:27:45.543651 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:27:45.543651 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:27:45.543651 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:27:45.543651 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:27:45.543651 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:27:45.543651 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: da56fc78f06142aba80ca4c6e3d60475 Jul 16 21:27:45.543651 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-439221844 Jul 16 21:27:45.543651 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:27:45.543651 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:27:45.543651 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/da56fc78f06142aba80ca4c6e3d60475"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/da56fc78f06142aba80ca4c6e3d60475"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_da56fc78f06142aba80ca4c6e3d60475"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/da56fc78f06142aba80ca4c6e3d60475"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:27:45.543651 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-ServerRescueNegativeTestJSON-439221844 Jul 16 21:27:45.543651 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: da56fc78f06142aba80ca4c6e3d60475 Jul 16 21:27:45.543651 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-439221844 Jul 16 21:27:45.543651 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-ServerRescueNegativeTestJSON-439221844-project Jul 16 21:27:45.543651 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:27:45.543651 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:27:45.543651 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: bcc9aa2a091c490599f9a9c41163e2eb Jul 16 21:27:45.543651 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-ServerRescueNegativeTestJSON-439221844-project Jul 16 21:27:45.543651 ncdv-master-1 devstack@n-api.service[138680]: Jul 16 21:27:45.543651 ncdv-master-1 devstack@n-api.service[138680]: {"reboot": {"type": "HARD"}} Jul 16 21:27:45.543651 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 21:27:45.543651 ncdv-master-1 devstack@n-api.service[138680]: exception: Cannot 'reboot' instance 867b2ab0-9301-414b-b045-85057d642971 while it is in vm_state rescued Jul 16 21:27:45.543651 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': '867b2ab0-9301-414b-b045-85057d642971', 'body': {'reboot': {'type': 'HARD'}}} Jul 16 21:27:45.543651 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:27:45.543651 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:27:45.543651 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1036, in _action_reboot Jul 16 21:27:45.543651 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi self.compute_api.reboot(context, instance, reboot_type) Jul 16 21:27:45.543651 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner Jul 16 21:27:45.543651 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Jul 16 21:27:45.543651 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 3497, in reboot Jul 16 21:27:45.543651 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi self._hard_reboot(context, instance) Jul 16 21:27:45.543651 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 145, in inner Jul 16 21:27:45.543651 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.InstanceInvalidState( Jul 16 21:27:45.543651 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.InstanceInvalidState: Instance 867b2ab0-9301-414b-b045-85057d642971 in vm_state rescued. Cannot _hard_reboot while the instance is in this state. Jul 16 21:27:45.543651 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:27:45.543651 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:27:45.543651 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:27:45.543651 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:27:45.543651 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:27:45.543651 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:27:45.543651 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:27:45.543651 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:27:45.543651 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:27:45.543651 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:27:45.543651 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 16 21:27:45.543651 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 16 21:27:45.543651 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1040, in _action_reboot Jul 16 21:27:45.543651 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi common.raise_http_conflict_for_instance_invalid_state(state_error, Jul 16 21:27:45.543651 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 347, in raise_http_conflict_for_instance_invalid_state Jul 16 21:27:45.543651 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPConflict(explanation=msg) Jul 16 21:27:45.547912 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPConflict: Cannot 'reboot' instance 867b2ab0-9301-414b-b045-85057d642971 while it is in vm_state rescued Jul 16 21:27:45.547912 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:27:45.547912 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:27:45.547912 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:27:45.547912 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:27:45.547912 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:27:45.547912 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:27:45.547912 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:27:45.547912 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:27:45.547912 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Cannot 'reboot' instance 867b2ab0-9301-414b-b045-85057d642971 while it is in vm_state rescued Jul 16 21:27:45.547912 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:27:45.550117 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-69c71ff3-f66b-4372-8a66-4b915095ea1b tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Returning 409 to user: Cannot 'reboot' instance 867b2ab0-9301-414b-b045-85057d642971 while it is in vm_state rescued {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:27:45.557712 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-69c71ff3-f66b-4372-8a66-4b915095ea1b tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] 10.222.0.16 "POST /compute/v2.1/servers/867b2ab0-9301-414b-b045-85057d642971/action" status: 409 len: 145 microversion: 2.1 time: 0.135192 Jul 16 21:27:45.558489 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2051/4105] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 21:27:45 2022] POST /compute/v2.1/servers/867b2ab0-9301-414b-b045-85057d642971/action => generated 145 bytes in 143 msecs (HTTP/1.1 409) 9 headers in 378 bytes (2 switches on core 0) Jul 16 21:27:45.567334 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-56b570ba-715f-417a-b5e3-c324a1d7e616 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:27:45.568904 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-56b570ba-715f-417a-b5e3-c324a1d7e616 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] [instance: cf38328b-40fb-4537-b524-aaeb0a0abfb4] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:27:45.578091 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-56b570ba-715f-417a-b5e3-c324a1d7e616 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:45.579388 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-56b570ba-715f-417a-b5e3-c324a1d7e616 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:45.582028 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-56b570ba-715f-417a-b5e3-c324a1d7e616 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:45.582651 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-56b570ba-715f-417a-b5e3-c324a1d7e616 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:45.773913 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-8ff7e600-f874-4be3-9085-f084949f242e tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '027dcc48e4fd4c36a5004096ad5e43f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dad823e240149edb30e592380249a13', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:27:45.776489 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=97,nova_cell1:INSERT=2,nova_cell1:UPDATE=1 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:27:45.784387 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-8ff7e600-f874-4be3-9085-f084949f242e tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "GET /compute/v2.1/servers/e6774019-149e-4804-b0ab-e3fcd137851e" status: 200 len: 1479 microversion: 2.1 time: 0.842271 Jul 16 21:27:45.785218 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2055/4106] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:27:44 2022] GET /compute/v2.1/servers/e6774019-149e-4804-b0ab-e3fcd137851e => generated 1479 bytes in 844 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:27:45.794242 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-f53fcb40-ccba-4734-917b-7838a4aad37b tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Action: 'action', calling method: >, body: {"rebuild": {"imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9"}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:27:45.796211 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-f53fcb40-ccba-4734-917b-7838a4aad37b tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] [instance: 867b2ab0-9301-414b-b045-85057d642971] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:27:45.803835 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f53fcb40-ccba-4734-917b-7838a4aad37b tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:45.809735 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f53fcb40-ccba-4734-917b-7838a4aad37b tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:45.809735 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f53fcb40-ccba-4734-917b-7838a4aad37b tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:45.809735 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f53fcb40-ccba-4734-917b-7838a4aad37b tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:45.859123 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-f53fcb40-ccba-4734-917b-7838a4aad37b tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] HTTP exception thrown: Cannot 'rebuild' instance 867b2ab0-9301-414b-b045-85057d642971 while it is in vm_state rescued Jul 16 21:27:45.859123 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-f53fcb40-ccba-4734-917b-7838a4aad37b tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Request method failure captured: Jul 16 21:27:45.859123 ncdv-master-1 devstack@n-api.service[138681]: request: POST /compute/v2.1/servers/867b2ab0-9301-414b-b045-85057d642971/action HTTP/1.1 Jul 16 21:27:45.859123 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:27:45.859123 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:27:45.859123 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:27:45.859123 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 65 Jul 16 21:27:45.859123 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:27:45.859123 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:27:45.859123 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:27:45.859123 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:27:45.859123 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0yxlBvXdj6P284bimRWVNOZAMmAVjIOasTJ1njzyaS12Nm5AKzVbxoKHi72FFVSaWe28lVKwWFtzjqUTy3NQWDNW-obFHfOCoFCCR4GNQ-_ruXI6NnMsKND6oz7nIdWgqUgB4Gac48-MGLdaneUoqvnHJrpsJjhWmKgGoKbdR8M7y0I Jul 16 21:27:45.859123 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:27:45.859123 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:27:45.859123 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:27:45.859123 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:27:45.859123 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:27:45.859123 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:27:45.859123 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: da56fc78f06142aba80ca4c6e3d60475 Jul 16 21:27:45.859123 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-439221844 Jul 16 21:27:45.859123 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:27:45.859123 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:27:45.859123 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/da56fc78f06142aba80ca4c6e3d60475"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/da56fc78f06142aba80ca4c6e3d60475"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_da56fc78f06142aba80ca4c6e3d60475"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/da56fc78f06142aba80ca4c6e3d60475"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:27:45.859123 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-ServerRescueNegativeTestJSON-439221844 Jul 16 21:27:45.859123 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: da56fc78f06142aba80ca4c6e3d60475 Jul 16 21:27:45.859123 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-439221844 Jul 16 21:27:45.859123 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-ServerRescueNegativeTestJSON-439221844-project Jul 16 21:27:45.859123 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:27:45.859123 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:27:45.859123 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: bcc9aa2a091c490599f9a9c41163e2eb Jul 16 21:27:45.859123 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-ServerRescueNegativeTestJSON-439221844-project Jul 16 21:27:45.859123 ncdv-master-1 devstack@n-api.service[138681]: Jul 16 21:27:45.859123 ncdv-master-1 devstack@n-api.service[138681]: {"rebuild": {"imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9"}} Jul 16 21:27:45.859123 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:27:45.859123 ncdv-master-1 devstack@n-api.service[138681]: exception: Cannot 'rebuild' instance 867b2ab0-9301-414b-b045-85057d642971 while it is in vm_state rescued Jul 16 21:27:45.859123 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': '867b2ab0-9301-414b-b045-85057d642971', 'body': {'rebuild': {'imageRef': '1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9'}}} Jul 16 21:27:45.859123 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:27:45.859123 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:27:45.859123 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1220, in _action_rebuild Jul 16 21:27:45.859123 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi self.compute_api.rebuild(context, Jul 16 21:27:45.859123 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 275, in inner Jul 16 21:27:45.859123 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Jul 16 21:27:45.859123 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 343, in wrapper Jul 16 21:27:45.859123 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return func(self, context, instance, *args, **kwargs) Jul 16 21:27:45.859123 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner Jul 16 21:27:45.859123 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Jul 16 21:27:45.859123 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 145, in inner Jul 16 21:27:45.859123 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.InstanceInvalidState( Jul 16 21:27:45.859123 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.InstanceInvalidState: Instance 867b2ab0-9301-414b-b045-85057d642971 in vm_state rescued. Cannot rebuild while the instance is in this state. Jul 16 21:27:45.859123 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:27:45.859123 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:27:45.859123 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:27:45.859123 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:27:45.859123 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:27:45.859123 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:27:45.859123 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:27:45.859123 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:27:45.859123 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:27:45.859123 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:27:45.859123 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 16 21:27:45.859123 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 16 21:27:45.864245 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 16 21:27:45.864245 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 16 21:27:45.864245 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 16 21:27:45.864245 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 16 21:27:45.864245 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi [Previous line repeated 4 more times] Jul 16 21:27:45.864245 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1228, in _action_rebuild Jul 16 21:27:45.864245 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi common.raise_http_conflict_for_instance_invalid_state(state_error, Jul 16 21:27:45.864245 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 347, in raise_http_conflict_for_instance_invalid_state Jul 16 21:27:45.864245 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPConflict(explanation=msg) Jul 16 21:27:45.864245 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPConflict: Cannot 'rebuild' instance 867b2ab0-9301-414b-b045-85057d642971 while it is in vm_state rescued Jul 16 21:27:45.864245 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:27:45.864245 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:27:45.864245 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:27:45.864245 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:27:45.864245 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:27:45.864245 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:27:45.864245 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:27:45.864245 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:27:45.864245 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Cannot 'rebuild' instance 867b2ab0-9301-414b-b045-85057d642971 while it is in vm_state rescued Jul 16 21:27:45.864245 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:27:45.864245 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-f53fcb40-ccba-4734-917b-7838a4aad37b tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Returning 409 to user: Cannot 'rebuild' instance 867b2ab0-9301-414b-b045-85057d642971 while it is in vm_state rescued {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:27:45.864245 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-f53fcb40-ccba-4734-917b-7838a4aad37b tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] 10.222.0.16 "POST /compute/v2.1/servers/867b2ab0-9301-414b-b045-85057d642971/action" status: 409 len: 146 microversion: 2.1 time: 0.068053 Jul 16 21:27:45.864245 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2056/4107] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 21:27:45 2022] POST /compute/v2.1/servers/867b2ab0-9301-414b-b045-85057d642971/action => generated 146 bytes in 69 msecs (HTTP/1.1 409) 9 headers in 378 bytes (1 switches on core 0) Jul 16 21:27:45.864245 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-f355a62c-dc98-4146-8a6e-c8ec3d6435e3 tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:27:45.864245 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-f355a62c-dc98-4146-8a6e-c8ec3d6435e3 tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] [instance: 0b67ad87-5ea8-4837-90f8-b3bc99134f3b] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:27:45.876733 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f355a62c-dc98-4146-8a6e-c8ec3d6435e3 tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:45.877409 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f355a62c-dc98-4146-8a6e-c8ec3d6435e3 tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:45.878202 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f355a62c-dc98-4146-8a6e-c8ec3d6435e3 tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:45.878651 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f355a62c-dc98-4146-8a6e-c8ec3d6435e3 tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:46.557325 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-56b570ba-715f-417a-b5e3-c324a1d7e616 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '029c76a0ba614b5880128c4309902c67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3b214f5044c7445592e177835a5d18fc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:27:46.571367 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-56b570ba-715f-417a-b5e3-c324a1d7e616 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] 10.222.0.16 "GET /compute/v2.1/servers/cf38328b-40fb-4537-b524-aaeb0a0abfb4" status: 200 len: 1498 microversion: 2.1 time: 1.011068 Jul 16 21:27:46.582888 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2052/4108] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:27:45 2022] GET /compute/v2.1/servers/cf38328b-40fb-4537-b524-aaeb0a0abfb4 => generated 1498 bytes in 1013 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:27:46.587801 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-1f0486f9-4cca-4097-976a-3388c0a7ec9e tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:27:46.588494 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-1f0486f9-4cca-4097-976a-3388c0a7ec9e tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] [instance: c2e32bd9-9d80-427b-bb6d-6274fd48173a] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:27:46.596675 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1f0486f9-4cca-4097-976a-3388c0a7ec9e tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:46.597254 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1f0486f9-4cca-4097-976a-3388c0a7ec9e tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:46.598984 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1f0486f9-4cca-4097-976a-3388c0a7ec9e tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:46.599564 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1f0486f9-4cca-4097-976a-3388c0a7ec9e tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:46.632719 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-1f0486f9-4cca-4097-976a-3388c0a7ec9e tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] [instance: c2e32bd9-9d80-427b-bb6d-6274fd48173a] Going to try to terminate instance {{(pid=138680) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 16 21:27:46.658709 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-f355a62c-dc98-4146-8a6e-c8ec3d6435e3 tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6771589864f442cbbfc8abf018abbd09', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bfd328e0f1446369531a23d15387cd6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:27:46.674485 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-f355a62c-dc98-4146-8a6e-c8ec3d6435e3 tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] 10.222.0.16 "GET /compute/v2.1/servers/0b67ad87-5ea8-4837-90f8-b3bc99134f3b" status: 200 len: 1314 microversion: 2.1 time: 0.817209 Jul 16 21:27:46.677297 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2057/4109] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:27:45 2022] GET /compute/v2.1/servers/0b67ad87-5ea8-4837-90f8-b3bc99134f3b => generated 1314 bytes in 821 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:27:46.683770 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-6aea28f8-3e77-4595-b81f-f9c959beacc5 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:27:46.688304 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-6aea28f8-3e77-4595-b81f-f9c959beacc5 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Policy check for os_compute_api:servers:allow_all_filters failed with credentials {'is_admin': False, 'user_id': '029c76a0ba614b5880128c4309902c67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3b214f5044c7445592e177835a5d18fc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:27:46.690501 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-6aea28f8-3e77-4595-b81f-f9c959beacc5 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Searching by: {'vm_state': ['stopped'], 'deleted': False, 'project_id': '3b214f5044c7445592e177835a5d18fc'} {{(pid=138681) get_all /opt/stack/nova/nova/compute/api.py:2962}} Jul 16 21:27:46.702540 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6aea28f8-3e77-4595-b81f-f9c959beacc5 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:46.703103 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6aea28f8-3e77-4595-b81f-f9c959beacc5 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:46.711830 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6aea28f8-3e77-4595-b81f-f9c959beacc5 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:46.718606 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6aea28f8-3e77-4595-b81f-f9c959beacc5 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:46.755453 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.multi_cell_list [None req-6aea28f8-3e77-4595-b81f-f9c959beacc5 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Listed batch of 1 results from cell out of 1000 limit. Returned 1 total so far. {{(pid=138681) do_query /opt/stack/nova/nova/compute/multi_cell_list.py:373}} Jul 16 21:27:46.815922 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-6aea28f8-3e77-4595-b81f-f9c959beacc5 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] 10.222.0.16 "GET /compute/v2.1/servers?status=shutoff" status: 200 len: 346 microversion: 2.1 time: 0.136921 Jul 16 21:27:46.816546 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2058/4110] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Jul 16 21:27:46 2022] GET /compute/v2.1/servers?status=shutoff => generated 346 bytes in 139 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:27:46.835141 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-3774efcb-490c-45e4-8fab-6c94b09df5f0 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:27:46.835783 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-3774efcb-490c-45e4-8fab-6c94b09df5f0 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: e6774019-149e-4804-b0ab-e3fcd137851e] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:27:46.848338 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3774efcb-490c-45e4-8fab-6c94b09df5f0 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:46.848841 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3774efcb-490c-45e4-8fab-6c94b09df5f0 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:46.851643 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3774efcb-490c-45e4-8fab-6c94b09df5f0 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:46.852392 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3774efcb-490c-45e4-8fab-6c94b09df5f0 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:47.039565 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-1f0486f9-4cca-4097-976a-3388c0a7ec9e tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] 10.222.0.16 "DELETE /compute/v2.1/servers/c2e32bd9-9d80-427b-bb6d-6274fd48173a" status: 204 len: 0 microversion: 2.1 time: 0.456714 Jul 16 21:27:47.043216 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2053/4111] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:27:46 2022] DELETE /compute/v2.1/servers/c2e32bd9-9d80-427b-bb6d-6274fd48173a => generated 0 bytes in 468 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 16 21:27:47.046030 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-485b0e49-f8e1-48df-afb4-969bd3266f12 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Action: 'action', calling method: >, body: {"os-start": {}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:27:47.067689 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-485b0e49-f8e1-48df-afb4-969bd3266f12 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:47.068365 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-485b0e49-f8e1-48df-afb4-969bd3266f12 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:47.117570 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-485b0e49-f8e1-48df-afb4-969bd3266f12 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] [instance: cf38328b-40fb-4537-b524-aaeb0a0abfb4] Going to try to start instance {{(pid=138680) start /opt/stack/nova/nova/compute/api.py:2730}} Jul 16 21:27:47.368522 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-485b0e49-f8e1-48df-afb4-969bd3266f12 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] 10.222.0.16 "POST /compute/v2.1/servers/cf38328b-40fb-4537-b524-aaeb0a0abfb4/action" status: 202 len: 0 microversion: 2.1 time: 0.326726 Jul 16 21:27:47.369206 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2054/4112] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 21:27:47 2022] POST /compute/v2.1/servers/cf38328b-40fb-4537-b524-aaeb0a0abfb4/action => generated 0 bytes in 328 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 21:27:47.374848 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-f8b76a32-45df-4cf3-bf19-80ddfd0036e9 req-15269fe0-2fa1-476e-b583-4cf1931eeb24 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "0b67ad87-5ea8-4837-90f8-b3bc99134f3b", "tag": "40d2e6fe-dc7b-4001-b1c2-8b9d92cbd577"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:27:47.386267 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-f8b76a32-45df-4cf3-bf19-80ddfd0036e9 req-15269fe0-2fa1-476e-b583-4cf1931eeb24 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:47.387145 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-f8b76a32-45df-4cf3-bf19-80ddfd0036e9 req-15269fe0-2fa1-476e-b583-4cf1931eeb24 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:47.404093 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-f8b76a32-45df-4cf3-bf19-80ddfd0036e9 req-15269fe0-2fa1-476e-b583-4cf1931eeb24 service nova] Creating event network-changed:40d2e6fe-dc7b-4001-b1c2-8b9d92cbd577 for instance 0b67ad87-5ea8-4837-90f8-b3bc99134f3b on nch1-master-1 Jul 16 21:27:47.409579 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-f8b76a32-45df-4cf3-bf19-80ddfd0036e9 req-15269fe0-2fa1-476e-b583-4cf1931eeb24 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.038913 Jul 16 21:27:47.409752 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-3774efcb-490c-45e4-8fab-6c94b09df5f0 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '027dcc48e4fd4c36a5004096ad5e43f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dad823e240149edb30e592380249a13', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:27:47.410180 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2055/4113] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:27:47 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:27:47.419299 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-3774efcb-490c-45e4-8fab-6c94b09df5f0 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "GET /compute/v2.1/servers/e6774019-149e-4804-b0ab-e3fcd137851e" status: 200 len: 1479 microversion: 2.1 time: 0.588475 Jul 16 21:27:47.421365 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-f449828d-cc6a-4a66-b942-d4fd7dc37274 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:27:47.422272 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-f449828d-cc6a-4a66-b942-d4fd7dc37274 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] [instance: c2e32bd9-9d80-427b-bb6d-6274fd48173a] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:27:47.429858 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f449828d-cc6a-4a66-b942-d4fd7dc37274 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:47.430024 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2059/4114] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:27:46 2022] GET /compute/v2.1/servers/e6774019-149e-4804-b0ab-e3fcd137851e => generated 1479 bytes in 613 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:27:47.430459 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f449828d-cc6a-4a66-b942-d4fd7dc37274 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:47.431662 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f449828d-cc6a-4a66-b942-d4fd7dc37274 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:47.432300 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f449828d-cc6a-4a66-b942-d4fd7dc37274 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:47.434786 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-c9689a5d-89b7-4ce9-bf97-27b74f4fb75e tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:27:47.435624 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-c9689a5d-89b7-4ce9-bf97-27b74f4fb75e tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] [instance: cf38328b-40fb-4537-b524-aaeb0a0abfb4] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:27:47.442368 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c9689a5d-89b7-4ce9-bf97-27b74f4fb75e tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:47.443578 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c9689a5d-89b7-4ce9-bf97-27b74f4fb75e tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:47.448341 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c9689a5d-89b7-4ce9-bf97-27b74f4fb75e tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:47.448341 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c9689a5d-89b7-4ce9-bf97-27b74f4fb75e tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:47.788988 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell0:SELECT=2 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:27:47.841773 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-c9689a5d-89b7-4ce9-bf97-27b74f4fb75e tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '029c76a0ba614b5880128c4309902c67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3b214f5044c7445592e177835a5d18fc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:27:47.860509 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-c9689a5d-89b7-4ce9-bf97-27b74f4fb75e tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] 10.222.0.16 "GET /compute/v2.1/servers/cf38328b-40fb-4537-b524-aaeb0a0abfb4" status: 200 len: 1507 microversion: 2.1 time: 0.427425 Jul 16 21:27:47.871293 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2060/4115] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:27:47 2022] GET /compute/v2.1/servers/cf38328b-40fb-4537-b524-aaeb0a0abfb4 => generated 1507 bytes in 433 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:27:47.873950 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-318ad27c-4b18-4225-b9ac-e51f89b5a954 tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:27:47.877315 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-318ad27c-4b18-4225-b9ac-e51f89b5a954 tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] [instance: 0b67ad87-5ea8-4837-90f8-b3bc99134f3b] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:27:47.903961 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-318ad27c-4b18-4225-b9ac-e51f89b5a954 tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:47.904889 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-318ad27c-4b18-4225-b9ac-e51f89b5a954 tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:47.906155 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-318ad27c-4b18-4225-b9ac-e51f89b5a954 tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:47.906809 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-318ad27c-4b18-4225-b9ac-e51f89b5a954 tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:47.926365 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-f449828d-cc6a-4a66-b942-d4fd7dc37274 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcc9aa2a091c490599f9a9c41163e2eb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da56fc78f06142aba80ca4c6e3d60475', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:27:47.938249 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-f449828d-cc6a-4a66-b942-d4fd7dc37274 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] 10.222.0.16 "GET /compute/v2.1/servers/c2e32bd9-9d80-427b-bb6d-6274fd48173a" status: 200 len: 1689 microversion: 2.1 time: 0.526535 Jul 16 21:27:47.942647 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2056/4116] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:27:47 2022] GET /compute/v2.1/servers/c2e32bd9-9d80-427b-bb6d-6274fd48173a => generated 1689 bytes in 532 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:27:48.428115 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-318ad27c-4b18-4225-b9ac-e51f89b5a954 tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6771589864f442cbbfc8abf018abbd09', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bfd328e0f1446369531a23d15387cd6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:27:48.436650 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-76f88b48-65df-4007-a688-f9a975bb7c76 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:27:48.437443 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-76f88b48-65df-4007-a688-f9a975bb7c76 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: e6774019-149e-4804-b0ab-e3fcd137851e] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:27:48.440721 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell0:SELECT=8 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:27:48.448288 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_api:SELECT=96,nova_api:INSERT=3,nova_api:UPDATE=1 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:27:48.452307 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-76f88b48-65df-4007-a688-f9a975bb7c76 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:48.452728 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-318ad27c-4b18-4225-b9ac-e51f89b5a954 tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] 10.222.0.16 "GET /compute/v2.1/servers/0b67ad87-5ea8-4837-90f8-b3bc99134f3b" status: 200 len: 1314 microversion: 2.1 time: 0.580989 Jul 16 21:27:48.462751 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2061/4117] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:27:47 2022] GET /compute/v2.1/servers/0b67ad87-5ea8-4837-90f8-b3bc99134f3b => generated 1314 bytes in 599 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:27:48.463674 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-76f88b48-65df-4007-a688-f9a975bb7c76 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.011s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:48.464690 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-76f88b48-65df-4007-a688-f9a975bb7c76 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:48.465283 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-76f88b48-65df-4007-a688-f9a975bb7c76 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:48.522672 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-76f88b48-65df-4007-a688-f9a975bb7c76 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:48.522672 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-76f88b48-65df-4007-a688-f9a975bb7c76 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:48.522672 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.objects.instance [None req-76f88b48-65df-4007-a688-f9a975bb7c76 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lazy-loading 'fault' on Instance uuid e6774019-149e-4804-b0ab-e3fcd137851e {{(pid=138680) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:27:48.638376 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-76f88b48-65df-4007-a688-f9a975bb7c76 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '027dcc48e4fd4c36a5004096ad5e43f8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1dad823e240149edb30e592380249a13', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:27:48.661621 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-76f88b48-65df-4007-a688-f9a975bb7c76 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "GET /compute/v2.1/servers/e6774019-149e-4804-b0ab-e3fcd137851e" status: 200 len: 1282 microversion: 2.1 time: 0.215359 Jul 16 21:27:48.671194 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2057/4118] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:27:48 2022] GET /compute/v2.1/servers/e6774019-149e-4804-b0ab-e3fcd137851e => generated 1282 bytes in 239 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:27:48.881059 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-b74b0478-c3d1-47a1-adbc-235d0a0ac3ba tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:27:48.881692 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-b74b0478-c3d1-47a1-adbc-235d0a0ac3ba tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] [instance: cf38328b-40fb-4537-b524-aaeb0a0abfb4] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:27:48.888462 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b74b0478-c3d1-47a1-adbc-235d0a0ac3ba tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:48.888922 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b74b0478-c3d1-47a1-adbc-235d0a0ac3ba tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:48.889766 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b74b0478-c3d1-47a1-adbc-235d0a0ac3ba tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:48.890400 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b74b0478-c3d1-47a1-adbc-235d0a0ac3ba tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:48.960292 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-f4c56f3c-c491-4826-8c97-52756ce36499 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:27:48.960920 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-f4c56f3c-c491-4826-8c97-52756ce36499 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] [instance: c2e32bd9-9d80-427b-bb6d-6274fd48173a] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:27:48.972313 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f4c56f3c-c491-4826-8c97-52756ce36499 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:48.972793 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f4c56f3c-c491-4826-8c97-52756ce36499 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:48.973598 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f4c56f3c-c491-4826-8c97-52756ce36499 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:48.974050 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f4c56f3c-c491-4826-8c97-52756ce36499 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:49.203194 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-b74b0478-c3d1-47a1-adbc-235d0a0ac3ba tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '029c76a0ba614b5880128c4309902c67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3b214f5044c7445592e177835a5d18fc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:27:49.214263 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-b74b0478-c3d1-47a1-adbc-235d0a0ac3ba tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] 10.222.0.16 "GET /compute/v2.1/servers/cf38328b-40fb-4537-b524-aaeb0a0abfb4" status: 200 len: 1507 microversion: 2.1 time: 0.338282 Jul 16 21:27:49.226221 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2062/4119] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:27:48 2022] GET /compute/v2.1/servers/cf38328b-40fb-4537-b524-aaeb0a0abfb4 => generated 1507 bytes in 351 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:27:49.256008 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-f4c56f3c-c491-4826-8c97-52756ce36499 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcc9aa2a091c490599f9a9c41163e2eb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da56fc78f06142aba80ca4c6e3d60475', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:27:49.274264 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-f4c56f3c-c491-4826-8c97-52756ce36499 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] 10.222.0.16 "GET /compute/v2.1/servers/c2e32bd9-9d80-427b-bb6d-6274fd48173a" status: 200 len: 1689 microversion: 2.1 time: 0.317444 Jul 16 21:27:49.280128 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2058/4120] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:27:48 2022] GET /compute/v2.1/servers/c2e32bd9-9d80-427b-bb6d-6274fd48173a => generated 1689 bytes in 325 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:27:49.435930 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-c3f57ef5-d9f2-4b9f-adfc-eec2b1bc04e7 req-1f1c1ed7-ec36-4ae8-bf38-ebb901995576 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "e6774019-149e-4804-b0ab-e3fcd137851e", "name": "network-vif-deleted", "tag": "7893417f-7147-4ca4-88d9-d55421fdc8d5"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:27:49.454896 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-c3f57ef5-d9f2-4b9f-adfc-eec2b1bc04e7 req-1f1c1ed7-ec36-4ae8-bf38-ebb901995576 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:49.454896 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-c3f57ef5-d9f2-4b9f-adfc-eec2b1bc04e7 req-1f1c1ed7-ec36-4ae8-bf38-ebb901995576 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:49.462428 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-c3f57ef5-d9f2-4b9f-adfc-eec2b1bc04e7 req-1f1c1ed7-ec36-4ae8-bf38-ebb901995576 service nova] Creating event network-vif-deleted:7893417f-7147-4ca4-88d9-d55421fdc8d5 for instance e6774019-149e-4804-b0ab-e3fcd137851e on nch2-master-1 Jul 16 21:27:49.467844 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-c3f57ef5-d9f2-4b9f-adfc-eec2b1bc04e7 req-1f1c1ed7-ec36-4ae8-bf38-ebb901995576 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.036168 Jul 16 21:27:49.470989 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2063/4121] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:27:49 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:27:49.473685 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-8d558855-411d-41c5-a4ca-a105b694b498 tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:27:49.474465 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-8d558855-411d-41c5-a4ca-a105b694b498 tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] [instance: 0b67ad87-5ea8-4837-90f8-b3bc99134f3b] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:27:49.488470 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8d558855-411d-41c5-a4ca-a105b694b498 tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:49.488470 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8d558855-411d-41c5-a4ca-a105b694b498 tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:49.488470 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8d558855-411d-41c5-a4ca-a105b694b498 tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:49.488470 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8d558855-411d-41c5-a4ca-a105b694b498 tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:49.688296 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-2e3756e4-b8cb-4752-956c-1820e50df923 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:27:49.688296 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-2e3756e4-b8cb-4752-956c-1820e50df923 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] [instance: e6774019-149e-4804-b0ab-e3fcd137851e] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:27:49.702387 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2e3756e4-b8cb-4752-956c-1820e50df923 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:49.702988 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2e3756e4-b8cb-4752-956c-1820e50df923 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:49.703908 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2e3756e4-b8cb-4752-956c-1820e50df923 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:49.705019 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2e3756e4-b8cb-4752-956c-1820e50df923 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:49.722519 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-2e3756e4-b8cb-4752-956c-1820e50df923 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] HTTP exception thrown: Instance e6774019-149e-4804-b0ab-e3fcd137851e could not be found. Jul 16 21:27:49.724403 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-2e3756e4-b8cb-4752-956c-1820e50df923 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Request method failure captured: Jul 16 21:27:49.724403 ncdv-master-1 devstack@n-api.service[138681]: request: GET /compute/v2.1/servers/e6774019-149e-4804-b0ab-e3fcd137851e HTTP/1.1 Jul 16 21:27:49.724403 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:27:49.724403 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:27:49.724403 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:27:49.724403 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 21:27:49.724403 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:27:49.724403 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:27:49.724403 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:27:49.724403 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:27:49.724403 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0yxS9NCWgI1-O55Z_AF16JzmczB_p-s4P30aJMz-9zu9VxcHuIV00tsfB9hOBtHTZ_sI8GPeOOvUrqYZLgOBzCbOMr4ZO506_XIRjMCtHQ0umzvcB9V9iFSNFoiIe_b2jKm-OyHj1IB8YVBCAcwDKANlsCw5UaAoIABK4dLLFjaVpEo Jul 16 21:27:49.724403 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:27:49.724403 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:27:49.724403 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:27:49.724403 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:27:49.724403 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:27:49.724403 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:27:49.724403 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: 1dad823e240149edb30e592380249a13 Jul 16 21:27:49.724403 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-ServersTestJSON-1529284206 Jul 16 21:27:49.724403 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:27:49.724403 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:27:49.724403 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/1dad823e240149edb30e592380249a13"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/1dad823e240149edb30e592380249a13"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_1dad823e240149edb30e592380249a13"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/1dad823e240149edb30e592380249a13"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:27:49.724403 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-ServersTestJSON-1529284206 Jul 16 21:27:49.724403 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: 1dad823e240149edb30e592380249a13 Jul 16 21:27:49.724403 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-ServersTestJSON-1529284206 Jul 16 21:27:49.724403 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-ServersTestJSON-1529284206-project Jul 16 21:27:49.724403 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:27:49.724403 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:27:49.724403 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: 027dcc48e4fd4c36a5004096ad5e43f8 Jul 16 21:27:49.724403 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-ServersTestJSON-1529284206-project Jul 16 21:27:49.724403 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:27:49.724403 ncdv-master-1 devstack@n-api.service[138681]: exception: Instance e6774019-149e-4804-b0ab-e3fcd137851e could not be found. Jul 16 21:27:49.724403 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': 'e6774019-149e-4804-b0ab-e3fcd137851e'} Jul 16 21:27:49.724403 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:27:49.724403 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:27:49.724403 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:27:49.724403 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:27:49.724403 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:27:49.724403 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:27:49.724403 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:27:49.724403 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:27:49.724403 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:27:49.724403 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:27:49.724403 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance e6774019-149e-4804-b0ab-e3fcd137851e could not be found. Jul 16 21:27:49.724403 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:27:49.724403 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:27:49.724403 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:27:49.724403 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:27:49.724403 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:27:49.724403 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:27:49.724403 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:27:49.724403 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:27:49.724403 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:27:49.724403 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:27:49.724403 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:27:49.724403 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:27:49.724403 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:27:49.724403 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:27:49.724403 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:27:49.724403 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:27:49.724403 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance e6774019-149e-4804-b0ab-e3fcd137851e could not be found. Jul 16 21:27:49.724403 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:27:49.724403 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:27:49.724403 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:27:49.724403 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:27:49.729294 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:27:49.729294 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:27:49.729294 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:27:49.729294 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:27:49.729294 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance e6774019-149e-4804-b0ab-e3fcd137851e could not be found. Jul 16 21:27:49.729294 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:27:49.729702 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-2e3756e4-b8cb-4752-956c-1820e50df923 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] Returning 404 to user: Instance e6774019-149e-4804-b0ab-e3fcd137851e could not be found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:27:49.730406 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-2e3756e4-b8cb-4752-956c-1820e50df923 tempest-ServersTestJSON-1529284206 tempest-ServersTestJSON-1529284206-project] 10.222.0.16 "GET /compute/v2.1/servers/e6774019-149e-4804-b0ab-e3fcd137851e" status: 404 len: 111 microversion: 2.1 time: 0.053184 Jul 16 21:27:49.730989 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2064/4122] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:27:49 2022] GET /compute/v2.1/servers/e6774019-149e-4804-b0ab-e3fcd137851e => generated 111 bytes in 56 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:27:49.788594 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-8d558855-411d-41c5-a4ca-a105b694b498 tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6771589864f442cbbfc8abf018abbd09', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bfd328e0f1446369531a23d15387cd6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:27:49.800354 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-8d558855-411d-41c5-a4ca-a105b694b498 tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] 10.222.0.16 "GET /compute/v2.1/servers/0b67ad87-5ea8-4837-90f8-b3bc99134f3b" status: 200 len: 1314 microversion: 2.1 time: 0.333665 Jul 16 21:27:49.801463 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2059/4123] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:27:49 2022] GET /compute/v2.1/servers/0b67ad87-5ea8-4837-90f8-b3bc99134f3b => generated 1314 bytes in 336 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:27:50.234128 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-7f1fd682-f819-4597-a5f3-1bef982a3c8a tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:27:50.234774 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-7f1fd682-f819-4597-a5f3-1bef982a3c8a tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] [instance: cf38328b-40fb-4537-b524-aaeb0a0abfb4] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:27:50.242592 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7f1fd682-f819-4597-a5f3-1bef982a3c8a tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:50.254704 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7f1fd682-f819-4597-a5f3-1bef982a3c8a tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.012s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:50.261019 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7f1fd682-f819-4597-a5f3-1bef982a3c8a tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:50.261693 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7f1fd682-f819-4597-a5f3-1bef982a3c8a tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:50.293623 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-0fa7fe56-dbd5-4261-bf2b-473bc4a74083 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:27:50.294185 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-0fa7fe56-dbd5-4261-bf2b-473bc4a74083 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] [instance: c2e32bd9-9d80-427b-bb6d-6274fd48173a] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:27:50.304481 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0fa7fe56-dbd5-4261-bf2b-473bc4a74083 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:50.307130 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0fa7fe56-dbd5-4261-bf2b-473bc4a74083 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:50.308217 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0fa7fe56-dbd5-4261-bf2b-473bc4a74083 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:50.312433 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0fa7fe56-dbd5-4261-bf2b-473bc4a74083 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:50.974788 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-7f1fd682-f819-4597-a5f3-1bef982a3c8a tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '029c76a0ba614b5880128c4309902c67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3b214f5044c7445592e177835a5d18fc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:27:50.987061 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-7f1fd682-f819-4597-a5f3-1bef982a3c8a tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] 10.222.0.16 "GET /compute/v2.1/servers/cf38328b-40fb-4537-b524-aaeb0a0abfb4" status: 200 len: 1511 microversion: 2.1 time: 0.757372 Jul 16 21:27:50.988394 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2065/4124] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:27:50 2022] GET /compute/v2.1/servers/cf38328b-40fb-4537-b524-aaeb0a0abfb4 => generated 1511 bytes in 759 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:27:50.993838 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-15ecaa30-7129-4a47-bfcb-3fd9ba44fff5 tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:27:50.994624 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-15ecaa30-7129-4a47-bfcb-3fd9ba44fff5 tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] [instance: 0b67ad87-5ea8-4837-90f8-b3bc99134f3b] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:27:51.008877 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-15ecaa30-7129-4a47-bfcb-3fd9ba44fff5 tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:51.011685 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-15ecaa30-7129-4a47-bfcb-3fd9ba44fff5 tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:51.013149 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-15ecaa30-7129-4a47-bfcb-3fd9ba44fff5 tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:51.013754 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-15ecaa30-7129-4a47-bfcb-3fd9ba44fff5 tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:51.053031 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-0fa7fe56-dbd5-4261-bf2b-473bc4a74083 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcc9aa2a091c490599f9a9c41163e2eb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da56fc78f06142aba80ca4c6e3d60475', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:27:51.067992 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-0fa7fe56-dbd5-4261-bf2b-473bc4a74083 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] 10.222.0.16 "GET /compute/v2.1/servers/c2e32bd9-9d80-427b-bb6d-6274fd48173a" status: 200 len: 1689 microversion: 2.1 time: 0.775157 Jul 16 21:27:51.068620 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2060/4125] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:27:50 2022] GET /compute/v2.1/servers/c2e32bd9-9d80-427b-bb6d-6274fd48173a => generated 1689 bytes in 781 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:27:51.084650 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-ec4af75b-c4d6-46d5-be43-45cee2056b32 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:27:51.096078 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-ec4af75b-c4d6-46d5-be43-45cee2056b32 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Policy check for os_compute_api:servers:allow_all_filters failed with credentials {'is_admin': False, 'user_id': '029c76a0ba614b5880128c4309902c67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3b214f5044c7445592e177835a5d18fc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:27:51.096927 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.compute.servers [None req-ec4af75b-c4d6-46d5-be43-45cee2056b32 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Removing options 'limit' from query {{(pid=138680) remove_invalid_options /opt/stack/nova/nova/api/openstack/compute/servers.py:1487}} Jul 16 21:27:51.101535 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-ec4af75b-c4d6-46d5-be43-45cee2056b32 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Searching by: {'deleted': False, 'project_id': '3b214f5044c7445592e177835a5d18fc'} {{(pid=138680) get_all /opt/stack/nova/nova/compute/api.py:2962}} Jul 16 21:27:51.103077 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ec4af75b-c4d6-46d5-be43-45cee2056b32 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:51.106649 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ec4af75b-c4d6-46d5-be43-45cee2056b32 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:51.107719 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ec4af75b-c4d6-46d5-be43-45cee2056b32 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:51.112328 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ec4af75b-c4d6-46d5-be43-45cee2056b32 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:51.130649 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-ec4af75b-c4d6-46d5-be43-45cee2056b32 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] 10.222.0.16 "GET /compute/v2.1/servers?limit=0" status: 200 len: 15 microversion: 2.1 time: 0.043547 Jul 16 21:27:51.130649 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2061/4126] 10.222.0.16 () {58 vars in 1139 bytes} [Sat Jul 16 21:27:51 2022] GET /compute/v2.1/servers?limit=0 => generated 15 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Jul 16 21:27:51.140692 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-0315d0b3-12ad-4fe1-a6c4-9dc09d2a96af tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:27:51.141568 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-0315d0b3-12ad-4fe1-a6c4-9dc09d2a96af tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] [instance: cf38328b-40fb-4537-b524-aaeb0a0abfb4] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:27:51.153676 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0315d0b3-12ad-4fe1-a6c4-9dc09d2a96af tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:51.156226 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0315d0b3-12ad-4fe1-a6c4-9dc09d2a96af tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:51.164308 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0315d0b3-12ad-4fe1-a6c4-9dc09d2a96af tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:51.164308 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0315d0b3-12ad-4fe1-a6c4-9dc09d2a96af tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:52.091772 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-15ecaa30-7129-4a47-bfcb-3fd9ba44fff5 tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6771589864f442cbbfc8abf018abbd09', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bfd328e0f1446369531a23d15387cd6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:27:52.100279 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-15ecaa30-7129-4a47-bfcb-3fd9ba44fff5 tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] 10.222.0.16 "GET /compute/v2.1/servers/0b67ad87-5ea8-4837-90f8-b3bc99134f3b" status: 200 len: 1314 microversion: 2.1 time: 1.110575 Jul 16 21:27:52.100872 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2066/4127] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:27:50 2022] GET /compute/v2.1/servers/0b67ad87-5ea8-4837-90f8-b3bc99134f3b => generated 1314 bytes in 1112 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:27:52.108385 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-3349a758-0d19-4132-8c3a-f9430e454eee tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:27:52.114531 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-3349a758-0d19-4132-8c3a-f9430e454eee tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] [instance: c2e32bd9-9d80-427b-bb6d-6274fd48173a] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:27:52.133021 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3349a758-0d19-4132-8c3a-f9430e454eee tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:52.133483 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3349a758-0d19-4132-8c3a-f9430e454eee tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:52.134667 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3349a758-0d19-4132-8c3a-f9430e454eee tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:52.135301 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3349a758-0d19-4132-8c3a-f9430e454eee tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:52.197158 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-0315d0b3-12ad-4fe1-a6c4-9dc09d2a96af tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '029c76a0ba614b5880128c4309902c67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3b214f5044c7445592e177835a5d18fc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:27:52.214712 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-0315d0b3-12ad-4fe1-a6c4-9dc09d2a96af tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] 10.222.0.16 "GET /compute/v2.1/servers/cf38328b-40fb-4537-b524-aaeb0a0abfb4" status: 200 len: 1511 microversion: 2.1 time: 1.078582 Jul 16 21:27:52.216363 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2062/4128] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:27:51 2022] GET /compute/v2.1/servers/cf38328b-40fb-4537-b524-aaeb0a0abfb4 => generated 1511 bytes in 1081 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:27:52.236216 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-c8cd08ab-b9fd-4566-9adf-04bbb1acce75 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:27:52.238170 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-c8cd08ab-b9fd-4566-9adf-04bbb1acce75 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] [instance: b01533f3-9a31-4246-b3e7-f12f638e8e89] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:27:52.250312 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c8cd08ab-b9fd-4566-9adf-04bbb1acce75 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:52.251882 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c8cd08ab-b9fd-4566-9adf-04bbb1acce75 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:52.253030 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c8cd08ab-b9fd-4566-9adf-04bbb1acce75 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:52.253751 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c8cd08ab-b9fd-4566-9adf-04bbb1acce75 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:52.481468 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-3349a758-0d19-4132-8c3a-f9430e454eee tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcc9aa2a091c490599f9a9c41163e2eb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da56fc78f06142aba80ca4c6e3d60475', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:27:52.490294 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-3349a758-0d19-4132-8c3a-f9430e454eee tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] 10.222.0.16 "GET /compute/v2.1/servers/c2e32bd9-9d80-427b-bb6d-6274fd48173a" status: 200 len: 1689 microversion: 2.1 time: 0.388380 Jul 16 21:27:52.491332 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2067/4129] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:27:52 2022] GET /compute/v2.1/servers/c2e32bd9-9d80-427b-bb6d-6274fd48173a => generated 1689 bytes in 390 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:27:52.632432 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-c8cd08ab-b9fd-4566-9adf-04bbb1acce75 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '029c76a0ba614b5880128c4309902c67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3b214f5044c7445592e177835a5d18fc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:27:52.640333 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-c8cd08ab-b9fd-4566-9adf-04bbb1acce75 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] 10.222.0.16 "GET /compute/v2.1/servers/b01533f3-9a31-4246-b3e7-f12f638e8e89" status: 200 len: 1511 microversion: 2.1 time: 0.411644 Jul 16 21:27:52.641187 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2063/4130] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:27:52 2022] GET /compute/v2.1/servers/b01533f3-9a31-4246-b3e7-f12f638e8e89 => generated 1511 bytes in 413 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:27:52.659625 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-9ba84f86-c699-4d31-a1d9-3f5289cc8a96 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:27:52.660295 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-9ba84f86-c699-4d31-a1d9-3f5289cc8a96 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] [instance: fbb7fab1-a4d5-48e2-9ace-fc65fad094ce] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:27:52.666819 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9ba84f86-c699-4d31-a1d9-3f5289cc8a96 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:52.667535 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9ba84f86-c699-4d31-a1d9-3f5289cc8a96 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:52.668545 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9ba84f86-c699-4d31-a1d9-3f5289cc8a96 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:52.669181 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9ba84f86-c699-4d31-a1d9-3f5289cc8a96 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:53.125152 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-f99ddb49-7623-4662-bcab-18abeeba7ade tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:27:53.129423 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-f99ddb49-7623-4662-bcab-18abeeba7ade tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] [instance: 0b67ad87-5ea8-4837-90f8-b3bc99134f3b] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:27:53.150432 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f99ddb49-7623-4662-bcab-18abeeba7ade tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:53.151336 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f99ddb49-7623-4662-bcab-18abeeba7ade tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:53.154318 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f99ddb49-7623-4662-bcab-18abeeba7ade tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:53.154839 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f99ddb49-7623-4662-bcab-18abeeba7ade tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:53.165823 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=96,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:27:53.438663 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-9ba84f86-c699-4d31-a1d9-3f5289cc8a96 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '029c76a0ba614b5880128c4309902c67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3b214f5044c7445592e177835a5d18fc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:27:53.447854 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-9ba84f86-c699-4d31-a1d9-3f5289cc8a96 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] 10.222.0.16 "GET /compute/v2.1/servers/fbb7fab1-a4d5-48e2-9ace-fc65fad094ce" status: 200 len: 1510 microversion: 2.1 time: 0.793934 Jul 16 21:27:53.448575 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2068/4131] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:27:52 2022] GET /compute/v2.1/servers/fbb7fab1-a4d5-48e2-9ace-fc65fad094ce => generated 1510 bytes in 795 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:27:53.605165 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-1081c5b6-02af-4021-8129-0729bbc2c63e tempest-ComputeTenantNetworksTest-947448628 tempest-ComputeTenantNetworksTest-947448628-project] Calling method '.version_select at 0x7fdda2ad31f0>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:27:53.752278 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-f99ddb49-7623-4662-bcab-18abeeba7ade tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6771589864f442cbbfc8abf018abbd09', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bfd328e0f1446369531a23d15387cd6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:27:53.767669 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-f99ddb49-7623-4662-bcab-18abeeba7ade tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] 10.222.0.16 "GET /compute/v2.1/servers/0b67ad87-5ea8-4837-90f8-b3bc99134f3b" status: 200 len: 1314 microversion: 2.1 time: 0.652820 Jul 16 21:27:53.768272 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2064/4132] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:27:53 2022] GET /compute/v2.1/servers/0b67ad87-5ea8-4837-90f8-b3bc99134f3b => generated 1314 bytes in 654 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:27:53.772987 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-7f0ea789-e7cb-4e24-9722-29cbd36c0b0c tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:27:53.789167 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-7f0ea789-e7cb-4e24-9722-29cbd36c0b0c tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Policy check for os_compute_api:servers:allow_all_filters failed with credentials {'is_admin': False, 'user_id': '029c76a0ba614b5880128c4309902c67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3b214f5044c7445592e177835a5d18fc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:27:53.789873 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-7f0ea789-e7cb-4e24-9722-29cbd36c0b0c tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Searching by: {'ip': '172.20.0.10', 'deleted': False, 'project_id': '3b214f5044c7445592e177835a5d18fc'} {{(pid=138680) get_all /opt/stack/nova/nova/compute/api.py:2962}} Jul 16 21:27:54.152302 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7f0ea789-e7cb-4e24-9722-29cbd36c0b0c tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:54.152766 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7f0ea789-e7cb-4e24-9722-29cbd36c0b0c tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:54.153716 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7f0ea789-e7cb-4e24-9722-29cbd36c0b0c tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:54.154405 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7f0ea789-e7cb-4e24-9722-29cbd36c0b0c tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:54.252474 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-1081c5b6-02af-4021-8129-0729bbc2c63e tempest-ComputeTenantNetworksTest-947448628 tempest-ComputeTenantNetworksTest-947448628-project] 10.222.0.16 "GET /compute/v2.1/os-tenant-networks" status: 200 len: 309 microversion: 2.1 time: 0.802124 Jul 16 21:27:54.252626 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2069/4133] 10.222.0.16 () {58 vars in 1157 bytes} [Sat Jul 16 21:27:53 2022] GET /compute/v2.1/os-tenant-networks => generated 309 bytes in 803 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:27:54.263058 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-5808a9b5-8e17-406b-bd04-73c62472949c tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:27:54.263696 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-5808a9b5-8e17-406b-bd04-73c62472949c tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] [instance: c2e32bd9-9d80-427b-bb6d-6274fd48173a] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:27:54.288976 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5808a9b5-8e17-406b-bd04-73c62472949c tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:54.289795 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5808a9b5-8e17-406b-bd04-73c62472949c tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:54.291980 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5808a9b5-8e17-406b-bd04-73c62472949c tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:54.298322 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5808a9b5-8e17-406b-bd04-73c62472949c tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:54.328532 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.multi_cell_list [None req-7f0ea789-e7cb-4e24-9722-29cbd36c0b0c tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Listed batch of 1 results from cell out of 1000 limit. Returned 1 total so far. {{(pid=138680) do_query /opt/stack/nova/nova/compute/multi_cell_list.py:373}} Jul 16 21:27:54.454944 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-7f0ea789-e7cb-4e24-9722-29cbd36c0b0c tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] 10.222.0.16 "GET /compute/v2.1/servers?ip=172.20.0.10" status: 200 len: 345 microversion: 2.1 time: 0.685561 Jul 16 21:27:54.455916 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2065/4134] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Jul 16 21:27:53 2022] GET /compute/v2.1/servers?ip=172.20.0.10 => generated 345 bytes in 687 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:27:54.476209 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-9e60448b-2ec0-406a-90e2-0fb52f7fd744 tempest-ComputeTenantNetworksTest-947448628 tempest-ComputeTenantNetworksTest-947448628-project] Calling method '.version_select at 0x7fdd9d777940>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:27:54.498446 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-5808a9b5-8e17-406b-bd04-73c62472949c tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcc9aa2a091c490599f9a9c41163e2eb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da56fc78f06142aba80ca4c6e3d60475', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:27:54.509105 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-5808a9b5-8e17-406b-bd04-73c62472949c tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] 10.222.0.16 "GET /compute/v2.1/servers/c2e32bd9-9d80-427b-bb6d-6274fd48173a" status: 200 len: 1621 microversion: 2.1 time: 0.249995 Jul 16 21:27:54.521537 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2070/4135] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:27:54 2022] GET /compute/v2.1/servers/c2e32bd9-9d80-427b-bb6d-6274fd48173a => generated 1621 bytes in 268 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:27:54.536392 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-f7c442c2-5873-46a6-927b-ef40ddc0a65e req-0cfb84ae-d992-457b-9fd3-e758acb0e741 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "c2e32bd9-9d80-427b-bb6d-6274fd48173a", "name": "network-vif-deleted", "tag": "347e5d31-8efd-4cb0-82eb-3b8a6e86e0cd"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:27:54.541387 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-f7c442c2-5873-46a6-927b-ef40ddc0a65e req-0cfb84ae-d992-457b-9fd3-e758acb0e741 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:54.541922 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-f7c442c2-5873-46a6-927b-ef40ddc0a65e req-0cfb84ae-d992-457b-9fd3-e758acb0e741 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:54.555730 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-f7c442c2-5873-46a6-927b-ef40ddc0a65e req-0cfb84ae-d992-457b-9fd3-e758acb0e741 service nova] Creating event network-vif-deleted:347e5d31-8efd-4cb0-82eb-3b8a6e86e0cd for instance c2e32bd9-9d80-427b-bb6d-6274fd48173a on nch1-master-1 Jul 16 21:27:54.563954 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-f7c442c2-5873-46a6-927b-ef40ddc0a65e req-0cfb84ae-d992-457b-9fd3-e758acb0e741 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.039272 Jul 16 21:27:54.564591 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2071/4136] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:27:54 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:27:54.575296 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-8cd66b4c-61ca-4de2-b4e0-40fed706a538 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:27:54.579467 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-8cd66b4c-61ca-4de2-b4e0-40fed706a538 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] [instance: cf38328b-40fb-4537-b524-aaeb0a0abfb4] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:27:54.596537 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8cd66b4c-61ca-4de2-b4e0-40fed706a538 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:54.599226 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8cd66b4c-61ca-4de2-b4e0-40fed706a538 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:54.600152 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8cd66b4c-61ca-4de2-b4e0-40fed706a538 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:54.600562 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8cd66b4c-61ca-4de2-b4e0-40fed706a538 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:54.752705 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-9e60448b-2ec0-406a-90e2-0fb52f7fd744 tempest-ComputeTenantNetworksTest-947448628 tempest-ComputeTenantNetworksTest-947448628-project] 10.222.0.16 "GET /compute/v2.1/os-tenant-networks/81800cbc-1727-4b80-b68e-30daed6f87c1" status: 200 len: 140 microversion: 2.1 time: 0.288323 Jul 16 21:27:54.753560 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2066/4137] 10.222.0.16 () {58 vars in 1268 bytes} [Sat Jul 16 21:27:54 2022] GET /compute/v2.1/os-tenant-networks/81800cbc-1727-4b80-b68e-30daed6f87c1 => generated 140 bytes in 290 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:27:54.788560 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-e68a26c8-2d8a-4835-a501-165028b9ddcf tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:27:54.789371 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-e68a26c8-2d8a-4835-a501-165028b9ddcf tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] [instance: 0b67ad87-5ea8-4837-90f8-b3bc99134f3b] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:27:54.805902 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e68a26c8-2d8a-4835-a501-165028b9ddcf tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:54.806638 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e68a26c8-2d8a-4835-a501-165028b9ddcf tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:54.826947 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e68a26c8-2d8a-4835-a501-165028b9ddcf tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:54.827785 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e68a26c8-2d8a-4835-a501-165028b9ddcf tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:55.202128 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-8cd66b4c-61ca-4de2-b4e0-40fed706a538 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '029c76a0ba614b5880128c4309902c67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3b214f5044c7445592e177835a5d18fc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:27:55.221236 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-8cd66b4c-61ca-4de2-b4e0-40fed706a538 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] 10.222.0.16 "GET /compute/v2.1/servers/cf38328b-40fb-4537-b524-aaeb0a0abfb4" status: 200 len: 1511 microversion: 2.1 time: 0.644342 Jul 16 21:27:55.221968 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2072/4138] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:27:54 2022] GET /compute/v2.1/servers/cf38328b-40fb-4537-b524-aaeb0a0abfb4 => generated 1511 bytes in 657 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:27:55.239632 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-4aeca774-c3fd-4c73-ad67-1d17d6712c94 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:27:55.257050 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-4aeca774-c3fd-4c73-ad67-1d17d6712c94 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] [instance: b01533f3-9a31-4246-b3e7-f12f638e8e89] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:27:55.271406 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4aeca774-c3fd-4c73-ad67-1d17d6712c94 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:55.272333 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4aeca774-c3fd-4c73-ad67-1d17d6712c94 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:55.273635 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4aeca774-c3fd-4c73-ad67-1d17d6712c94 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:55.274379 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4aeca774-c3fd-4c73-ad67-1d17d6712c94 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:55.346655 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-e68a26c8-2d8a-4835-a501-165028b9ddcf tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6771589864f442cbbfc8abf018abbd09', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bfd328e0f1446369531a23d15387cd6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:27:55.371795 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-e68a26c8-2d8a-4835-a501-165028b9ddcf tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] 10.222.0.16 "GET /compute/v2.1/servers/0b67ad87-5ea8-4837-90f8-b3bc99134f3b" status: 200 len: 1314 microversion: 2.1 time: 0.584206 Jul 16 21:27:55.374259 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2067/4139] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:27:54 2022] GET /compute/v2.1/servers/0b67ad87-5ea8-4837-90f8-b3bc99134f3b => generated 1314 bytes in 590 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:27:55.530660 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-226217ac-6dde-4dfb-9a87-d36f8de2c9f6 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:27:55.532588 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-226217ac-6dde-4dfb-9a87-d36f8de2c9f6 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] [instance: c2e32bd9-9d80-427b-bb6d-6274fd48173a] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:27:55.541198 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-226217ac-6dde-4dfb-9a87-d36f8de2c9f6 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:55.541660 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-226217ac-6dde-4dfb-9a87-d36f8de2c9f6 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:55.542457 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-226217ac-6dde-4dfb-9a87-d36f8de2c9f6 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:55.543050 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-226217ac-6dde-4dfb-9a87-d36f8de2c9f6 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:55.609669 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-226217ac-6dde-4dfb-9a87-d36f8de2c9f6 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] HTTP exception thrown: Instance c2e32bd9-9d80-427b-bb6d-6274fd48173a could not be found. Jul 16 21:27:55.620437 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-226217ac-6dde-4dfb-9a87-d36f8de2c9f6 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Request method failure captured: Jul 16 21:27:55.620437 ncdv-master-1 devstack@n-api.service[138680]: request: GET /compute/v2.1/servers/c2e32bd9-9d80-427b-bb6d-6274fd48173a HTTP/1.1 Jul 16 21:27:55.620437 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:27:55.620437 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:27:55.620437 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:27:55.620437 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 0 Jul 16 21:27:55.620437 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:27:55.620437 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:27:55.620437 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:27:55.620437 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:27:55.620437 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0yxlBvXdj6P284bimRWVNOZAMmAVjIOasTJ1njzyaS12Nm5AKzVbxoKHi72FFVSaWe28lVKwWFtzjqUTy3NQWDNW-obFHfOCoFCCR4GNQ-_ruXI6NnMsKND6oz7nIdWgqUgB4Gac48-MGLdaneUoqvnHJrpsJjhWmKgGoKbdR8M7y0I Jul 16 21:27:55.620437 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:27:55.620437 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:27:55.620437 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:27:55.620437 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:27:55.620437 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:27:55.620437 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:27:55.620437 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: da56fc78f06142aba80ca4c6e3d60475 Jul 16 21:27:55.620437 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-439221844 Jul 16 21:27:55.620437 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:27:55.620437 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:27:55.620437 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/da56fc78f06142aba80ca4c6e3d60475"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/da56fc78f06142aba80ca4c6e3d60475"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_da56fc78f06142aba80ca4c6e3d60475"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/da56fc78f06142aba80ca4c6e3d60475"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:27:55.620437 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-ServerRescueNegativeTestJSON-439221844 Jul 16 21:27:55.620437 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: da56fc78f06142aba80ca4c6e3d60475 Jul 16 21:27:55.620437 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-439221844 Jul 16 21:27:55.620437 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-ServerRescueNegativeTestJSON-439221844-project Jul 16 21:27:55.620437 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:27:55.620437 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:27:55.620437 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: bcc9aa2a091c490599f9a9c41163e2eb Jul 16 21:27:55.620437 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-ServerRescueNegativeTestJSON-439221844-project Jul 16 21:27:55.620437 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 21:27:55.620437 ncdv-master-1 devstack@n-api.service[138680]: exception: Instance c2e32bd9-9d80-427b-bb6d-6274fd48173a could not be found. Jul 16 21:27:55.620437 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': 'c2e32bd9-9d80-427b-bb6d-6274fd48173a'} Jul 16 21:27:55.620437 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:27:55.620437 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:27:55.620437 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:27:55.620437 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:27:55.620437 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:27:55.620437 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:27:55.620437 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:27:55.620437 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:27:55.620437 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:27:55.620437 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:27:55.620437 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance c2e32bd9-9d80-427b-bb6d-6274fd48173a could not be found. Jul 16 21:27:55.620437 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:27:55.620437 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:27:55.620437 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:27:55.620437 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:27:55.620437 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:27:55.620437 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:27:55.620437 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:27:55.620437 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:27:55.620437 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:27:55.620437 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:27:55.620437 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:27:55.620437 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:27:55.620437 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:27:55.620437 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:27:55.620437 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:27:55.620437 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:27:55.620437 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance c2e32bd9-9d80-427b-bb6d-6274fd48173a could not be found. Jul 16 21:27:55.620437 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:27:55.620437 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:27:55.620437 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:27:55.624741 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:27:55.624741 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:27:55.624741 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:27:55.624741 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:27:55.624741 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:27:55.624741 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance c2e32bd9-9d80-427b-bb6d-6274fd48173a could not be found. Jul 16 21:27:55.624741 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:27:55.624741 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-226217ac-6dde-4dfb-9a87-d36f8de2c9f6 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Returning 404 to user: Instance c2e32bd9-9d80-427b-bb6d-6274fd48173a could not be found. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:27:55.624741 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-226217ac-6dde-4dfb-9a87-d36f8de2c9f6 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] 10.222.0.16 "GET /compute/v2.1/servers/c2e32bd9-9d80-427b-bb6d-6274fd48173a" status: 404 len: 111 microversion: 2.1 time: 0.098797 Jul 16 21:27:55.624741 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2068/4140] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:27:55 2022] GET /compute/v2.1/servers/c2e32bd9-9d80-427b-bb6d-6274fd48173a => generated 111 bytes in 100 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Jul 16 21:27:55.640816 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-6267267f-15b8-458f-a935-9c827c54f1c1 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Calling method '.version_select at 0x7fdd9c0c4af0>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:27:55.780722 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-6267267f-15b8-458f-a935-9c827c54f1c1 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] 10.222.0.16 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1244213655" status: 202 len: 0 microversion: 2.1 time: 0.145478 Jul 16 21:27:55.781778 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2069/4141] 10.222.0.16 () {58 vars in 1220 bytes} [Sat Jul 16 21:27:55 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1244213655 => generated 0 bytes in 148 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Jul 16 21:27:56.184882 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-4aeca774-c3fd-4c73-ad67-1d17d6712c94 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '029c76a0ba614b5880128c4309902c67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3b214f5044c7445592e177835a5d18fc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:27:56.203532 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-4aeca774-c3fd-4c73-ad67-1d17d6712c94 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] 10.222.0.16 "GET /compute/v2.1/servers/b01533f3-9a31-4246-b3e7-f12f638e8e89" status: 200 len: 1511 microversion: 2.1 time: 0.967391 Jul 16 21:27:56.206350 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2073/4142] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:27:55 2022] GET /compute/v2.1/servers/b01533f3-9a31-4246-b3e7-f12f638e8e89 => generated 1511 bytes in 972 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:27:56.228067 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-5be5325f-3544-455b-8d29-243e01308feb tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:27:56.229124 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-5be5325f-3544-455b-8d29-243e01308feb tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] [instance: fbb7fab1-a4d5-48e2-9ace-fc65fad094ce] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:27:56.241515 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5be5325f-3544-455b-8d29-243e01308feb tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:56.242425 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5be5325f-3544-455b-8d29-243e01308feb tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:56.244621 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5be5325f-3544-455b-8d29-243e01308feb tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:56.246556 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5be5325f-3544-455b-8d29-243e01308feb tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:56.389814 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-fdf09223-fa7c-45f0-be81-ad8ad4fdec51 tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:27:56.391827 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-fdf09223-fa7c-45f0-be81-ad8ad4fdec51 tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] [instance: 0b67ad87-5ea8-4837-90f8-b3bc99134f3b] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:27:56.404530 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fdf09223-fa7c-45f0-be81-ad8ad4fdec51 tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:56.405322 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fdf09223-fa7c-45f0-be81-ad8ad4fdec51 tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:56.406323 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fdf09223-fa7c-45f0-be81-ad8ad4fdec51 tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:56.407727 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fdf09223-fa7c-45f0-be81-ad8ad4fdec51 tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:56.725178 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell0:SELECT=2 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:27:56.754040 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-5be5325f-3544-455b-8d29-243e01308feb tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '029c76a0ba614b5880128c4309902c67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3b214f5044c7445592e177835a5d18fc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:27:56.768798 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-5be5325f-3544-455b-8d29-243e01308feb tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] 10.222.0.16 "GET /compute/v2.1/servers/fbb7fab1-a4d5-48e2-9ace-fc65fad094ce" status: 200 len: 1510 microversion: 2.1 time: 0.550957 Jul 16 21:27:56.769675 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2070/4143] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:27:56 2022] GET /compute/v2.1/servers/fbb7fab1-a4d5-48e2-9ace-fc65fad094ce => generated 1510 bytes in 553 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:27:56.787453 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-62da6452-23b0-4582-827a-c771bbd36506 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:27:56.792954 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-62da6452-23b0-4582-827a-c771bbd36506 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Policy check for os_compute_api:servers:allow_all_filters failed with credentials {'is_admin': False, 'user_id': '029c76a0ba614b5880128c4309902c67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3b214f5044c7445592e177835a5d18fc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:27:56.804216 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-62da6452-23b0-4582-827a-c771bbd36506 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Searching by: {'deleted': False, 'project_id': '3b214f5044c7445592e177835a5d18fc'} {{(pid=138680) get_all /opt/stack/nova/nova/compute/api.py:2962}} Jul 16 21:27:56.813229 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-62da6452-23b0-4582-827a-c771bbd36506 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:56.819841 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-62da6452-23b0-4582-827a-c771bbd36506 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:56.821275 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-62da6452-23b0-4582-827a-c771bbd36506 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:56.821959 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-62da6452-23b0-4582-827a-c771bbd36506 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:56.882334 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-fdf09223-fa7c-45f0-be81-ad8ad4fdec51 tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6771589864f442cbbfc8abf018abbd09', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bfd328e0f1446369531a23d15387cd6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:27:56.886442 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.multi_cell_list [None req-62da6452-23b0-4582-827a-c771bbd36506 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Listed batch of 3 results from cell out of 1000 limit. Returned 3 total so far. {{(pid=138680) do_query /opt/stack/nova/nova/compute/multi_cell_list.py:373}} Jul 16 21:27:56.898494 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-fdf09223-fa7c-45f0-be81-ad8ad4fdec51 tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] 10.222.0.16 "GET /compute/v2.1/servers/0b67ad87-5ea8-4837-90f8-b3bc99134f3b" status: 200 len: 1314 microversion: 2.1 time: 0.512567 Jul 16 21:27:56.909649 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2074/4144] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:27:56 2022] GET /compute/v2.1/servers/0b67ad87-5ea8-4837-90f8-b3bc99134f3b => generated 1314 bytes in 524 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:27:56.972117 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-62da6452-23b0-4582-827a-c771bbd36506 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '029c76a0ba614b5880128c4309902c67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3b214f5044c7445592e177835a5d18fc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:27:56.980726 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-62da6452-23b0-4582-827a-c771bbd36506 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:56.981359 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-62da6452-23b0-4582-827a-c771bbd36506 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:57.360021 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-62da6452-23b0-4582-827a-c771bbd36506 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] 10.222.0.16 "GET /compute/v2.1/servers/detail" status: 200 len: 4515 microversion: 2.1 time: 0.578413 Jul 16 21:27:57.363102 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2071/4145] 10.222.0.16 () {58 vars in 1145 bytes} [Sat Jul 16 21:27:56 2022] GET /compute/v2.1/servers/detail => generated 4515 bytes in 582 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:27:57.384317 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-7dcc824d-258c-49c8-847d-2a39556dce10 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:27:57.389226 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-7dcc824d-258c-49c8-847d-2a39556dce10 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Policy check for os_compute_api:servers:allow_all_filters failed with credentials {'is_admin': False, 'user_id': '029c76a0ba614b5880128c4309902c67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3b214f5044c7445592e177835a5d18fc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:27:57.390020 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-7dcc824d-258c-49c8-847d-2a39556dce10 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Searching by: {'ip': '172.20.0.', 'deleted': False, 'project_id': '3b214f5044c7445592e177835a5d18fc'} {{(pid=138681) get_all /opt/stack/nova/nova/compute/api.py:2962}} Jul 16 21:27:57.634561 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7dcc824d-258c-49c8-847d-2a39556dce10 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:57.635836 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7dcc824d-258c-49c8-847d-2a39556dce10 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:57.636838 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7dcc824d-258c-49c8-847d-2a39556dce10 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:57.637446 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7dcc824d-258c-49c8-847d-2a39556dce10 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:57.802988 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=100 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:27:57.807528 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.multi_cell_list [None req-7dcc824d-258c-49c8-847d-2a39556dce10 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Listed batch of 3 results from cell out of 1000 limit. Returned 3 total so far. {{(pid=138681) do_query /opt/stack/nova/nova/compute/multi_cell_list.py:373}} Jul 16 21:27:57.882483 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-7dcc824d-258c-49c8-847d-2a39556dce10 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] 10.222.0.16 "GET /compute/v2.1/servers?ip=172.20.0." status: 200 len: 1010 microversion: 2.1 time: 0.501793 Jul 16 21:27:57.883466 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2075/4146] 10.222.0.16 () {58 vars in 1149 bytes} [Sat Jul 16 21:27:57 2022] GET /compute/v2.1/servers?ip=172.20.0. => generated 1010 bytes in 504 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:27:57.902368 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-5172a761-ebd4-4d83-b262-730ad0fa67e2 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:27:57.906381 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-5172a761-ebd4-4d83-b262-730ad0fa67e2 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Policy check for os_compute_api:servers:allow_all_filters failed with credentials {'is_admin': False, 'user_id': '029c76a0ba614b5880128c4309902c67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3b214f5044c7445592e177835a5d18fc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:27:57.907573 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-5172a761-ebd4-4d83-b262-730ad0fa67e2 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Searching by: {'name': '^.*\\-instance\\-[0-9]+$', 'deleted': False, 'project_id': '3b214f5044c7445592e177835a5d18fc'} {{(pid=138680) get_all /opt/stack/nova/nova/compute/api.py:2962}} Jul 16 21:27:57.923969 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5172a761-ebd4-4d83-b262-730ad0fa67e2 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:57.925007 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5172a761-ebd4-4d83-b262-730ad0fa67e2 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:57.926225 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5172a761-ebd4-4d83-b262-730ad0fa67e2 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:57.929752 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5172a761-ebd4-4d83-b262-730ad0fa67e2 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:57.932305 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-ef0a40ff-4056-4ffa-bc01-7b66debb00e1 tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:27:57.933237 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-ef0a40ff-4056-4ffa-bc01-7b66debb00e1 tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] [instance: 0b67ad87-5ea8-4837-90f8-b3bc99134f3b] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:27:57.947171 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ef0a40ff-4056-4ffa-bc01-7b66debb00e1 tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:57.947171 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ef0a40ff-4056-4ffa-bc01-7b66debb00e1 tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:57.947371 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ef0a40ff-4056-4ffa-bc01-7b66debb00e1 tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:57.947774 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ef0a40ff-4056-4ffa-bc01-7b66debb00e1 tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:58.128003 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.multi_cell_list [None req-5172a761-ebd4-4d83-b262-730ad0fa67e2 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Listed batch of 3 results from cell out of 1000 limit. Returned 3 total so far. {{(pid=138680) do_query /opt/stack/nova/nova/compute/multi_cell_list.py:373}} Jul 16 21:27:58.203388 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-5172a761-ebd4-4d83-b262-730ad0fa67e2 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] 10.222.0.16 "GET /compute/v2.1/servers?name=%5E.%2A%5C-instance%5C-%5B0-9%5D%2B%24" status: 200 len: 1010 microversion: 2.1 time: 0.310079 Jul 16 21:27:58.210051 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2072/4147] 10.222.0.16 () {58 vars in 1211 bytes} [Sat Jul 16 21:27:57 2022] GET /compute/v2.1/servers?name=%5E.%2A%5C-instance%5C-%5B0-9%5D%2B%24 => generated 1010 bytes in 319 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:27:58.215486 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-48ac9120-293c-423b-9498-d2ebe8151e99 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:27:58.220025 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-48ac9120-293c-423b-9498-d2ebe8151e99 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Policy check for os_compute_api:servers:allow_all_filters failed with credentials {'is_admin': False, 'user_id': '029c76a0ba614b5880128c4309902c67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3b214f5044c7445592e177835a5d18fc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:27:58.221124 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-48ac9120-293c-423b-9498-d2ebe8151e99 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Searching by: {'name': '^.*\\-instance\\-.*$', 'deleted': False, 'project_id': '3b214f5044c7445592e177835a5d18fc'} {{(pid=138680) get_all /opt/stack/nova/nova/compute/api.py:2962}} Jul 16 21:27:58.242722 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-48ac9120-293c-423b-9498-d2ebe8151e99 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:58.243588 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-48ac9120-293c-423b-9498-d2ebe8151e99 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.012s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:58.248819 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-48ac9120-293c-423b-9498-d2ebe8151e99 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:58.249541 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-48ac9120-293c-423b-9498-d2ebe8151e99 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:58.321412 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-ef0a40ff-4056-4ffa-bc01-7b66debb00e1 tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6771589864f442cbbfc8abf018abbd09', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bfd328e0f1446369531a23d15387cd6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:27:58.330475 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-ef0a40ff-4056-4ffa-bc01-7b66debb00e1 tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] 10.222.0.16 "GET /compute/v2.1/servers/0b67ad87-5ea8-4837-90f8-b3bc99134f3b" status: 200 len: 1314 microversion: 2.1 time: 0.405329 Jul 16 21:27:58.332963 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2076/4148] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:27:57 2022] GET /compute/v2.1/servers/0b67ad87-5ea8-4837-90f8-b3bc99134f3b => generated 1314 bytes in 408 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:27:58.375233 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.multi_cell_list [None req-48ac9120-293c-423b-9498-d2ebe8151e99 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Listed batch of 3 results from cell out of 1000 limit. Returned 3 total so far. {{(pid=138680) do_query /opt/stack/nova/nova/compute/multi_cell_list.py:373}} Jul 16 21:27:58.410376 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-ef0a40ff-4056-4ffa-bc01-7b66debb00e1 tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:27:58.412683 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-ef0a40ff-4056-4ffa-bc01-7b66debb00e1 tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] 10.222.0.16 "GET /compute//" status: 200 len: 377 microversion: - time: 0.002929 Jul 16 21:27:58.413601 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2077/4149] 10.222.0.16 () {56 vars in 885 bytes} [Sat Jul 16 21:27:58 2022] GET /compute/ => generated 377 bytes in 4 msecs (HTTP/1.1 200) 3 headers in 91 bytes (1 switches on core 0) Jul 16 21:27:58.422208 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-ef0a40ff-4056-4ffa-bc01-7b66debb00e1 tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:27:58.424031 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-ef0a40ff-4056-4ffa-bc01-7b66debb00e1 tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] 10.222.0.16 "GET /compute/v2/" status: 200 len: 379 microversion: 2.1 time: 0.002451 Jul 16 21:27:58.424743 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2078/4150] 10.222.0.16 () {54 vars in 1034 bytes} [Sat Jul 16 21:27:58 2022] GET /compute/v2/ => generated 379 bytes in 4 msecs (HTTP/1.1 200) 5 headers in 221 bytes (1 switches on core 0) Jul 16 21:27:58.431648 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-ef0a40ff-4056-4ffa-bc01-7b66debb00e1 tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:27:58.433026 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-ef0a40ff-4056-4ffa-bc01-7b66debb00e1 tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] 10.222.0.16 "GET /compute/v2.1/" status: 200 len: 388 microversion: 2.1 time: 0.002620 Jul 16 21:27:58.433931 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2079/4151] 10.222.0.16 () {54 vars in 1040 bytes} [Sat Jul 16 21:27:58 2022] GET /compute/v2.1/ => generated 388 bytes in 4 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 16 21:27:58.441847 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-ef0a40ff-4056-4ffa-bc01-7b66debb00e1 tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:27:58.442999 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-ef0a40ff-4056-4ffa-bc01-7b66debb00e1 tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] 10.222.0.16 "GET /compute//" status: 200 len: 377 microversion: - time: 0.002173 Jul 16 21:27:58.443908 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2080/4152] 10.222.0.16 () {56 vars in 885 bytes} [Sat Jul 16 21:27:58 2022] GET /compute/ => generated 377 bytes in 3 msecs (HTTP/1.1 200) 3 headers in 91 bytes (1 switches on core 0) Jul 16 21:27:58.446044 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-48ac9120-293c-423b-9498-d2ebe8151e99 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] 10.222.0.16 "GET /compute/v2.1/servers?name=%5E.%2A%5C-instance%5C-.%2A%24" status: 200 len: 1010 microversion: 2.1 time: 0.234501 Jul 16 21:27:58.446736 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2073/4153] 10.222.0.16 () {58 vars in 1195 bytes} [Sat Jul 16 21:27:58 2022] GET /compute/v2.1/servers?name=%5E.%2A%5C-instance%5C-.%2A%24 => generated 1010 bytes in 236 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:27:58.464754 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-b1c7c7b5-af3e-4d4a-8a3e-d95ff55b99ff tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:27:58.468668 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-b1c7c7b5-af3e-4d4a-8a3e-d95ff55b99ff tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Policy check for os_compute_api:servers:allow_all_filters failed with credentials {'is_admin': False, 'user_id': '029c76a0ba614b5880128c4309902c67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3b214f5044c7445592e177835a5d18fc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:27:58.470071 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-b1c7c7b5-af3e-4d4a-8a3e-d95ff55b99ff tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Searching by: {'name': '1583187080', 'deleted': False, 'project_id': '3b214f5044c7445592e177835a5d18fc'} {{(pid=138681) get_all /opt/stack/nova/nova/compute/api.py:2962}} Jul 16 21:27:58.477599 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b1c7c7b5-af3e-4d4a-8a3e-d95ff55b99ff tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:58.477599 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b1c7c7b5-af3e-4d4a-8a3e-d95ff55b99ff tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:58.478310 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b1c7c7b5-af3e-4d4a-8a3e-d95ff55b99ff tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:58.478820 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b1c7c7b5-af3e-4d4a-8a3e-d95ff55b99ff tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:58.603633 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.multi_cell_list [None req-b1c7c7b5-af3e-4d4a-8a3e-d95ff55b99ff tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Listed batch of 1 results from cell out of 1000 limit. Returned 1 total so far. {{(pid=138681) do_query /opt/stack/nova/nova/compute/multi_cell_list.py:373}} Jul 16 21:27:58.674179 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-b1c7c7b5-af3e-4d4a-8a3e-d95ff55b99ff tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] 10.222.0.16 "GET /compute/v2.1/servers?name=1583187080" status: 200 len: 346 microversion: 2.1 time: 0.213236 Jul 16 21:27:58.674795 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2081/4154] 10.222.0.16 () {58 vars in 1155 bytes} [Sat Jul 16 21:27:58 2022] GET /compute/v2.1/servers?name=1583187080 => generated 346 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:27:58.686320 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-516e5abb-0a63-480b-88fa-9ec682057ffb tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:27:58.691341 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-516e5abb-0a63-480b-88fa-9ec682057ffb tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Policy check for os_compute_api:servers:allow_all_filters failed with credentials {'is_admin': False, 'user_id': '029c76a0ba614b5880128c4309902c67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3b214f5044c7445592e177835a5d18fc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:27:58.699370 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-516e5abb-0a63-480b-88fa-9ec682057ffb tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Searching by: {'name': '-instance', 'deleted': False, 'project_id': '3b214f5044c7445592e177835a5d18fc'} {{(pid=138680) get_all /opt/stack/nova/nova/compute/api.py:2962}} Jul 16 21:27:58.707410 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-516e5abb-0a63-480b-88fa-9ec682057ffb tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:58.708322 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-516e5abb-0a63-480b-88fa-9ec682057ffb tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:58.710828 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-516e5abb-0a63-480b-88fa-9ec682057ffb tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:58.711500 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-516e5abb-0a63-480b-88fa-9ec682057ffb tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:58.835814 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.multi_cell_list [None req-516e5abb-0a63-480b-88fa-9ec682057ffb tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Listed batch of 3 results from cell out of 1000 limit. Returned 3 total so far. {{(pid=138680) do_query /opt/stack/nova/nova/compute/multi_cell_list.py:373}} Jul 16 21:27:58.931572 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-516e5abb-0a63-480b-88fa-9ec682057ffb tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] 10.222.0.16 "GET /compute/v2.1/servers?name=-instance" status: 200 len: 1010 microversion: 2.1 time: 0.249161 Jul 16 21:27:58.933020 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2074/4155] 10.222.0.16 () {58 vars in 1153 bytes} [Sat Jul 16 21:27:58 2022] GET /compute/v2.1/servers?name=-instance => generated 1010 bytes in 251 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:27:58.944396 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-c765e18a-5e68-4b49-9f9e-4bdc0d2d5a3f tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:27:58.949125 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-c765e18a-5e68-4b49-9f9e-4bdc0d2d5a3f tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Policy check for os_compute_api:servers:allow_all_filters failed with credentials {'is_admin': False, 'user_id': '029c76a0ba614b5880128c4309902c67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3b214f5044c7445592e177835a5d18fc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:27:58.950124 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-c765e18a-5e68-4b49-9f9e-4bdc0d2d5a3f tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Searching by: {'name': 't-ListServerFiltersTestJSON-instance-158318708', 'deleted': False, 'project_id': '3b214f5044c7445592e177835a5d18fc'} {{(pid=138681) get_all /opt/stack/nova/nova/compute/api.py:2962}} Jul 16 21:27:58.955812 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-773490b2-f08b-448c-b6a2-5bf426dcb867 req-2208e822-8c06-496a-951b-4ffcda50e738 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "0b67ad87-5ea8-4837-90f8-b3bc99134f3b", "name": "network-vif-plugged", "status": "completed", "tag": "40d2e6fe-dc7b-4001-b1c2-8b9d92cbd577"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:27:58.969262 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c765e18a-5e68-4b49-9f9e-4bdc0d2d5a3f tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:58.969918 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c765e18a-5e68-4b49-9f9e-4bdc0d2d5a3f tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:58.970704 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-773490b2-f08b-448c-b6a2-5bf426dcb867 req-2208e822-8c06-496a-951b-4ffcda50e738 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:58.971190 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c765e18a-5e68-4b49-9f9e-4bdc0d2d5a3f tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:58.971310 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-773490b2-f08b-448c-b6a2-5bf426dcb867 req-2208e822-8c06-496a-951b-4ffcda50e738 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:58.971876 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c765e18a-5e68-4b49-9f9e-4bdc0d2d5a3f tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:58.990673 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-773490b2-f08b-448c-b6a2-5bf426dcb867 req-2208e822-8c06-496a-951b-4ffcda50e738 service nova] Creating event network-vif-plugged:40d2e6fe-dc7b-4001-b1c2-8b9d92cbd577 for instance 0b67ad87-5ea8-4837-90f8-b3bc99134f3b on nch1-master-1 Jul 16 21:27:58.994395 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-773490b2-f08b-448c-b6a2-5bf426dcb867 req-2208e822-8c06-496a-951b-4ffcda50e738 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.046567 Jul 16 21:27:58.996106 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2075/4156] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:27:58 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 16 21:27:59.131881 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.multi_cell_list [None req-c765e18a-5e68-4b49-9f9e-4bdc0d2d5a3f tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Listed batch of 1 results from cell out of 1000 limit. Returned 1 total so far. {{(pid=138681) do_query /opt/stack/nova/nova/compute/multi_cell_list.py:373}} Jul 16 21:27:59.211580 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-c765e18a-5e68-4b49-9f9e-4bdc0d2d5a3f tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] 10.222.0.16 "GET /compute/v2.1/servers?name=t-ListServerFiltersTestJSON-instance-158318708" status: 200 len: 346 microversion: 2.1 time: 0.270757 Jul 16 21:27:59.212566 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2082/4157] 10.222.0.16 () {58 vars in 1227 bytes} [Sat Jul 16 21:27:58 2022] GET /compute/v2.1/servers?name=t-ListServerFiltersTestJSON-instance-158318708 => generated 346 bytes in 272 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:27:59.229726 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-10008394-b209-4a7d-aa42-94c3acee7053 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:27:59.231340 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-10008394-b209-4a7d-aa42-94c3acee7053 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] [instance: fbb7fab1-a4d5-48e2-9ace-fc65fad094ce] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:27:59.243412 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-10008394-b209-4a7d-aa42-94c3acee7053 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:59.244049 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-10008394-b209-4a7d-aa42-94c3acee7053 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:59.249115 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-10008394-b209-4a7d-aa42-94c3acee7053 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:59.250122 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-10008394-b209-4a7d-aa42-94c3acee7053 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:59.300914 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-10008394-b209-4a7d-aa42-94c3acee7053 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] [instance: fbb7fab1-a4d5-48e2-9ace-fc65fad094ce] Going to try to terminate instance {{(pid=138680) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 16 21:27:59.357417 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-28fd2479-531a-4647-9d94-0ee3f6793127 tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:27:59.358269 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-28fd2479-531a-4647-9d94-0ee3f6793127 tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] [instance: 0b67ad87-5ea8-4837-90f8-b3bc99134f3b] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:27:59.378411 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-28fd2479-531a-4647-9d94-0ee3f6793127 tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:59.380413 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-28fd2479-531a-4647-9d94-0ee3f6793127 tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:59.382203 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-28fd2479-531a-4647-9d94-0ee3f6793127 tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:59.399849 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-28fd2479-531a-4647-9d94-0ee3f6793127 tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.016s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:59.710232 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-28fd2479-531a-4647-9d94-0ee3f6793127 tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6771589864f442cbbfc8abf018abbd09', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bfd328e0f1446369531a23d15387cd6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:27:59.722980 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-28fd2479-531a-4647-9d94-0ee3f6793127 tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] 10.222.0.16 "GET /compute/v2.1/servers/0b67ad87-5ea8-4837-90f8-b3bc99134f3b" status: 200 len: 1314 microversion: 2.1 time: 0.376924 Jul 16 21:27:59.724251 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2083/4158] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:27:59 2022] GET /compute/v2.1/servers/0b67ad87-5ea8-4837-90f8-b3bc99134f3b => generated 1314 bytes in 379 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:27:59.751854 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-10008394-b209-4a7d-aa42-94c3acee7053 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] 10.222.0.16 "DELETE /compute/v2.1/servers/fbb7fab1-a4d5-48e2-9ace-fc65fad094ce" status: 204 len: 0 microversion: 2.1 time: 0.528708 Jul 16 21:27:59.752540 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2076/4159] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:27:59 2022] DELETE /compute/v2.1/servers/fbb7fab1-a4d5-48e2-9ace-fc65fad094ce => generated 0 bytes in 531 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 16 21:27:59.761982 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-89c15501-cbc9-4569-ba3d-7501c44b3002 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:27:59.762691 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-89c15501-cbc9-4569-ba3d-7501c44b3002 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] [instance: fbb7fab1-a4d5-48e2-9ace-fc65fad094ce] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:27:59.770235 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-89c15501-cbc9-4569-ba3d-7501c44b3002 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:59.771079 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-89c15501-cbc9-4569-ba3d-7501c44b3002 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:27:59.772097 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-89c15501-cbc9-4569-ba3d-7501c44b3002 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:27:59.772541 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-89c15501-cbc9-4569-ba3d-7501c44b3002 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:00.160983 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-89c15501-cbc9-4569-ba3d-7501c44b3002 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '029c76a0ba614b5880128c4309902c67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3b214f5044c7445592e177835a5d18fc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:00.177584 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-89c15501-cbc9-4569-ba3d-7501c44b3002 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] 10.222.0.16 "GET /compute/v2.1/servers/fbb7fab1-a4d5-48e2-9ace-fc65fad094ce" status: 200 len: 1516 microversion: 2.1 time: 0.420547 Jul 16 21:28:00.178490 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2084/4160] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:27:59 2022] GET /compute/v2.1/servers/fbb7fab1-a4d5-48e2-9ace-fc65fad094ce => generated 1516 bytes in 422 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:28:00.746929 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-338518ec-1c66-4737-9b33-795f751d78ff tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:28:00.748580 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-338518ec-1c66-4737-9b33-795f751d78ff tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] [instance: 0b67ad87-5ea8-4837-90f8-b3bc99134f3b] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:28:00.771409 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-338518ec-1c66-4737-9b33-795f751d78ff tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:00.771922 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-338518ec-1c66-4737-9b33-795f751d78ff tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:00.776281 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-338518ec-1c66-4737-9b33-795f751d78ff tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:00.776281 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-338518ec-1c66-4737-9b33-795f751d78ff tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:01.116162 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-338518ec-1c66-4737-9b33-795f751d78ff tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6771589864f442cbbfc8abf018abbd09', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bfd328e0f1446369531a23d15387cd6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:01.127145 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-338518ec-1c66-4737-9b33-795f751d78ff tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] 10.222.0.16 "GET /compute/v2.1/servers/0b67ad87-5ea8-4837-90f8-b3bc99134f3b" status: 200 len: 1463 microversion: 2.1 time: 0.388290 Jul 16 21:28:01.130408 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2077/4161] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:28:00 2022] GET /compute/v2.1/servers/0b67ad87-5ea8-4837-90f8-b3bc99134f3b => generated 1463 bytes in 393 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:28:01.187467 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-b2da8d82-6d53-42c8-b048-4fedebed8a34 tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:28:01.196728 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-a534fa06-013e-4e7d-ae63-74d97cf15ade tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:28:01.197696 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-a534fa06-013e-4e7d-ae63-74d97cf15ade tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] [instance: fbb7fab1-a4d5-48e2-9ace-fc65fad094ce] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:28:01.199000 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-b2da8d82-6d53-42c8-b048-4fedebed8a34 tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] [instance: 0b67ad87-5ea8-4837-90f8-b3bc99134f3b] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:28:01.214141 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a534fa06-013e-4e7d-ae63-74d97cf15ade tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:01.214709 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a534fa06-013e-4e7d-ae63-74d97cf15ade tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:01.215578 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b2da8d82-6d53-42c8-b048-4fedebed8a34 tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:01.216598 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a534fa06-013e-4e7d-ae63-74d97cf15ade tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:01.217199 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b2da8d82-6d53-42c8-b048-4fedebed8a34 tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:01.217804 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a534fa06-013e-4e7d-ae63-74d97cf15ade tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:01.229199 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b2da8d82-6d53-42c8-b048-4fedebed8a34 tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:01.231129 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b2da8d82-6d53-42c8-b048-4fedebed8a34 tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:01.288641 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-b2da8d82-6d53-42c8-b048-4fedebed8a34 tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] [instance: 0b67ad87-5ea8-4837-90f8-b3bc99134f3b] Going to try to terminate instance {{(pid=138681) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 16 21:28:01.722216 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-a534fa06-013e-4e7d-ae63-74d97cf15ade tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '029c76a0ba614b5880128c4309902c67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3b214f5044c7445592e177835a5d18fc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:01.736651 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-a534fa06-013e-4e7d-ae63-74d97cf15ade tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] 10.222.0.16 "GET /compute/v2.1/servers/fbb7fab1-a4d5-48e2-9ace-fc65fad094ce" status: 200 len: 1516 microversion: 2.1 time: 0.544898 Jul 16 21:28:01.748139 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2078/4162] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:28:01 2022] GET /compute/v2.1/servers/fbb7fab1-a4d5-48e2-9ace-fc65fad094ce => generated 1516 bytes in 557 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:28:01.840268 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-b2da8d82-6d53-42c8-b048-4fedebed8a34 tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] 10.222.0.16 "DELETE /compute/v2.1/servers/0b67ad87-5ea8-4837-90f8-b3bc99134f3b" status: 204 len: 0 microversion: 2.1 time: 0.656902 Jul 16 21:28:01.841258 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2085/4163] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:28:01 2022] DELETE /compute/v2.1/servers/0b67ad87-5ea8-4837-90f8-b3bc99134f3b => generated 0 bytes in 659 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 16 21:28:01.851975 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-c2ccadfe-9b9a-4960-8791-031de4563729 tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:28:01.853019 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-c2ccadfe-9b9a-4960-8791-031de4563729 tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] [instance: 0b67ad87-5ea8-4837-90f8-b3bc99134f3b] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:28:01.887706 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c2ccadfe-9b9a-4960-8791-031de4563729 tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:01.887706 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c2ccadfe-9b9a-4960-8791-031de4563729 tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:01.888555 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c2ccadfe-9b9a-4960-8791-031de4563729 tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:01.889040 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c2ccadfe-9b9a-4960-8791-031de4563729 tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:02.134567 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-c2ccadfe-9b9a-4960-8791-031de4563729 tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6771589864f442cbbfc8abf018abbd09', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bfd328e0f1446369531a23d15387cd6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:02.146190 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-c2ccadfe-9b9a-4960-8791-031de4563729 tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] 10.222.0.16 "GET /compute/v2.1/servers/0b67ad87-5ea8-4837-90f8-b3bc99134f3b" status: 200 len: 1469 microversion: 2.1 time: 0.299360 Jul 16 21:28:02.148701 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2079/4164] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:28:01 2022] GET /compute/v2.1/servers/0b67ad87-5ea8-4837-90f8-b3bc99134f3b => generated 1469 bytes in 303 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:28:02.545889 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-70438327-ddab-48a7-9a14-df93b8d3df77 req-b6f07e55-e982-428f-a1b8-3dd9d100e585 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "fbb7fab1-a4d5-48e2-9ace-fc65fad094ce", "name": "network-vif-deleted", "tag": "2ae39666-a3c4-4738-a6cb-db9cc84fdcd6"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:28:02.555519 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-70438327-ddab-48a7-9a14-df93b8d3df77 req-b6f07e55-e982-428f-a1b8-3dd9d100e585 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:02.555894 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-70438327-ddab-48a7-9a14-df93b8d3df77 req-b6f07e55-e982-428f-a1b8-3dd9d100e585 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:02.570363 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-70438327-ddab-48a7-9a14-df93b8d3df77 req-b6f07e55-e982-428f-a1b8-3dd9d100e585 service nova] Creating event network-vif-deleted:2ae39666-a3c4-4738-a6cb-db9cc84fdcd6 for instance fbb7fab1-a4d5-48e2-9ace-fc65fad094ce on nch2-master-1 Jul 16 21:28:02.577409 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-70438327-ddab-48a7-9a14-df93b8d3df77 req-b6f07e55-e982-428f-a1b8-3dd9d100e585 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.035434 Jul 16 21:28:02.579246 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2086/4165] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:28:02 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:28:02.758496 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-5a88f50c-a0a1-44bf-afe7-e9abd805cdc9 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:28:02.759285 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-5a88f50c-a0a1-44bf-afe7-e9abd805cdc9 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] [instance: fbb7fab1-a4d5-48e2-9ace-fc65fad094ce] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:28:02.779081 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5a88f50c-a0a1-44bf-afe7-e9abd805cdc9 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:02.779613 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5a88f50c-a0a1-44bf-afe7-e9abd805cdc9 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:02.780586 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5a88f50c-a0a1-44bf-afe7-e9abd805cdc9 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:02.781105 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5a88f50c-a0a1-44bf-afe7-e9abd805cdc9 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:02.817856 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:28:02.957505 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-5a88f50c-a0a1-44bf-afe7-e9abd805cdc9 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '029c76a0ba614b5880128c4309902c67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3b214f5044c7445592e177835a5d18fc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:02.990260 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-5a88f50c-a0a1-44bf-afe7-e9abd805cdc9 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] 10.222.0.16 "GET /compute/v2.1/servers/fbb7fab1-a4d5-48e2-9ace-fc65fad094ce" status: 200 len: 1305 microversion: 2.1 time: 0.225914 Jul 16 21:28:02.993134 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2080/4166] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:28:02 2022] GET /compute/v2.1/servers/fbb7fab1-a4d5-48e2-9ace-fc65fad094ce => generated 1305 bytes in 244 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:28:03.173447 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-a0754b35-2b35-4fbc-9fd6-d15ed13a0da4 tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:28:03.175199 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-a0754b35-2b35-4fbc-9fd6-d15ed13a0da4 tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] [instance: 0b67ad87-5ea8-4837-90f8-b3bc99134f3b] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:28:03.185609 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a0754b35-2b35-4fbc-9fd6-d15ed13a0da4 tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:03.186244 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a0754b35-2b35-4fbc-9fd6-d15ed13a0da4 tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:03.187570 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a0754b35-2b35-4fbc-9fd6-d15ed13a0da4 tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:03.188886 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a0754b35-2b35-4fbc-9fd6-d15ed13a0da4 tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:03.727393 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-a0754b35-2b35-4fbc-9fd6-d15ed13a0da4 tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6771589864f442cbbfc8abf018abbd09', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bfd328e0f1446369531a23d15387cd6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:03.745969 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-a0754b35-2b35-4fbc-9fd6-d15ed13a0da4 tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] 10.222.0.16 "GET /compute/v2.1/servers/0b67ad87-5ea8-4837-90f8-b3bc99134f3b" status: 200 len: 1469 microversion: 2.1 time: 0.581355 Jul 16 21:28:03.751490 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2087/4167] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:28:03 2022] GET /compute/v2.1/servers/0b67ad87-5ea8-4837-90f8-b3bc99134f3b => generated 1469 bytes in 592 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:28:04.011289 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-0c7b8c87-3020-4dd8-986e-a0f3d8d88a0d tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:28:04.012564 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-0c7b8c87-3020-4dd8-986e-a0f3d8d88a0d tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] [instance: fbb7fab1-a4d5-48e2-9ace-fc65fad094ce] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:28:04.020962 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0c7b8c87-3020-4dd8-986e-a0f3d8d88a0d tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:04.021423 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0c7b8c87-3020-4dd8-986e-a0f3d8d88a0d tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:04.022250 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0c7b8c87-3020-4dd8-986e-a0f3d8d88a0d tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:04.022725 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0c7b8c87-3020-4dd8-986e-a0f3d8d88a0d tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:04.040035 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-0c7b8c87-3020-4dd8-986e-a0f3d8d88a0d tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] HTTP exception thrown: Instance fbb7fab1-a4d5-48e2-9ace-fc65fad094ce could not be found. Jul 16 21:28:04.042130 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-0c7b8c87-3020-4dd8-986e-a0f3d8d88a0d tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Request method failure captured: Jul 16 21:28:04.042130 ncdv-master-1 devstack@n-api.service[138680]: request: GET /compute/v2.1/servers/fbb7fab1-a4d5-48e2-9ace-fc65fad094ce HTTP/1.1 Jul 16 21:28:04.042130 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:28:04.042130 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:28:04.042130 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:28:04.042130 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 0 Jul 16 21:28:04.042130 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:28:04.042130 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:28:04.042130 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:28:04.042130 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:28:04.042130 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0y0f71twFMCVGLcyplWyRtyAM2CfOZwj4laMJ_sqbGEEyWtMp5KkAeIFFiT5aXWve6fmB41BoCEhDgyJ_zvQFNfWsoMy5Nd2qMpH4YpTCGyEpNT3rZaP4jGMzoknx7H-D8Y7484jvmo3TtoXuL69MLIEYajD2R1bB1NIQs0LYN96Amk Jul 16 21:28:04.042130 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:28:04.042130 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:28:04.042130 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:28:04.042130 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:28:04.042130 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:28:04.042130 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:28:04.042130 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: 3b214f5044c7445592e177835a5d18fc Jul 16 21:28:04.042130 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-ListServerFiltersTestJSON-921481021 Jul 16 21:28:04.042130 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:28:04.042130 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:28:04.042130 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/3b214f5044c7445592e177835a5d18fc"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/3b214f5044c7445592e177835a5d18fc"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_3b214f5044c7445592e177835a5d18fc"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/3b214f5044c7445592e177835a5d18fc"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:28:04.042130 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-ListServerFiltersTestJSON-921481021 Jul 16 21:28:04.042130 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: 3b214f5044c7445592e177835a5d18fc Jul 16 21:28:04.042130 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-ListServerFiltersTestJSON-921481021 Jul 16 21:28:04.042130 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-ListServerFiltersTestJSON-921481021-project Jul 16 21:28:04.042130 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:28:04.042130 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:28:04.042130 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: 029c76a0ba614b5880128c4309902c67 Jul 16 21:28:04.042130 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-ListServerFiltersTestJSON-921481021-project Jul 16 21:28:04.042130 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 21:28:04.042130 ncdv-master-1 devstack@n-api.service[138680]: exception: Instance fbb7fab1-a4d5-48e2-9ace-fc65fad094ce could not be found. Jul 16 21:28:04.042130 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': 'fbb7fab1-a4d5-48e2-9ace-fc65fad094ce'} Jul 16 21:28:04.042130 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:28:04.042130 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:28:04.042130 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:28:04.042130 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:28:04.042130 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:28:04.042130 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:28:04.042130 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:28:04.042130 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:28:04.042130 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:28:04.042130 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:28:04.042130 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance fbb7fab1-a4d5-48e2-9ace-fc65fad094ce could not be found. Jul 16 21:28:04.042130 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:28:04.042130 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:28:04.042130 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:28:04.042130 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:28:04.042130 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:28:04.042130 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:28:04.042130 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:28:04.042130 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:28:04.042130 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:28:04.042130 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:28:04.042130 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:28:04.042130 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:28:04.042130 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:28:04.042130 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:28:04.042130 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:28:04.042130 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:28:04.042130 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance fbb7fab1-a4d5-48e2-9ace-fc65fad094ce could not be found. Jul 16 21:28:04.042130 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:28:04.042130 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:28:04.042130 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:28:04.046905 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:28:04.046905 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:28:04.046905 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:28:04.046905 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:28:04.046905 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:28:04.046905 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance fbb7fab1-a4d5-48e2-9ace-fc65fad094ce could not be found. Jul 16 21:28:04.046905 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:28:04.048775 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-0c7b8c87-3020-4dd8-986e-a0f3d8d88a0d tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Returning 404 to user: Instance fbb7fab1-a4d5-48e2-9ace-fc65fad094ce could not be found. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:28:04.050050 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-0c7b8c87-3020-4dd8-986e-a0f3d8d88a0d tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] 10.222.0.16 "GET /compute/v2.1/servers/fbb7fab1-a4d5-48e2-9ace-fc65fad094ce" status: 404 len: 111 microversion: 2.1 time: 0.044317 Jul 16 21:28:04.063783 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-633c23a7-4e4f-4b83-b219-444bae998527 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:28:04.064303 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-633c23a7-4e4f-4b83-b219-444bae998527 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] [instance: b01533f3-9a31-4246-b3e7-f12f638e8e89] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:28:04.069441 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2081/4168] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:28:04 2022] GET /compute/v2.1/servers/fbb7fab1-a4d5-48e2-9ace-fc65fad094ce => generated 111 bytes in 64 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:28:04.071469 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-633c23a7-4e4f-4b83-b219-444bae998527 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:04.072105 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-633c23a7-4e4f-4b83-b219-444bae998527 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:04.073128 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-633c23a7-4e4f-4b83-b219-444bae998527 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:04.073940 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-633c23a7-4e4f-4b83-b219-444bae998527 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:04.122079 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-633c23a7-4e4f-4b83-b219-444bae998527 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] [instance: b01533f3-9a31-4246-b3e7-f12f638e8e89] Going to try to terminate instance {{(pid=138681) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 16 21:28:04.441277 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_api:SELECT=98,nova_api:UPDATE=2 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:28:04.590484 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-633c23a7-4e4f-4b83-b219-444bae998527 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] 10.222.0.16 "DELETE /compute/v2.1/servers/b01533f3-9a31-4246-b3e7-f12f638e8e89" status: 204 len: 0 microversion: 2.1 time: 0.534622 Jul 16 21:28:04.596270 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2088/4169] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:28:04 2022] DELETE /compute/v2.1/servers/b01533f3-9a31-4246-b3e7-f12f638e8e89 => generated 0 bytes in 536 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 16 21:28:04.612627 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-a89e6923-4887-40fb-918b-8348a986dda0 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:28:04.614177 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-a89e6923-4887-40fb-918b-8348a986dda0 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] [instance: b01533f3-9a31-4246-b3e7-f12f638e8e89] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:28:04.619874 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-ad14f9d2-1545-44d5-b322-6d7c6f439b47 req-aa5c0f10-291c-4f28-9537-721f67f8394d service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "0b67ad87-5ea8-4837-90f8-b3bc99134f3b", "name": "network-vif-deleted", "tag": "40d2e6fe-dc7b-4001-b1c2-8b9d92cbd577"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:28:04.650800 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a89e6923-4887-40fb-918b-8348a986dda0 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:04.651216 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a89e6923-4887-40fb-918b-8348a986dda0 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:04.652038 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a89e6923-4887-40fb-918b-8348a986dda0 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:04.652477 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a89e6923-4887-40fb-918b-8348a986dda0 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:04.661323 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-ad14f9d2-1545-44d5-b322-6d7c6f439b47 req-aa5c0f10-291c-4f28-9537-721f67f8394d service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:04.662286 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-ad14f9d2-1545-44d5-b322-6d7c6f439b47 req-aa5c0f10-291c-4f28-9537-721f67f8394d service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:04.686819 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-ad14f9d2-1545-44d5-b322-6d7c6f439b47 req-aa5c0f10-291c-4f28-9537-721f67f8394d service nova] Creating event network-vif-deleted:40d2e6fe-dc7b-4001-b1c2-8b9d92cbd577 for instance 0b67ad87-5ea8-4837-90f8-b3bc99134f3b on nch1-master-1 Jul 16 21:28:04.706280 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-ad14f9d2-1545-44d5-b322-6d7c6f439b47 req-aa5c0f10-291c-4f28-9537-721f67f8394d service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.099710 Jul 16 21:28:04.709365 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2089/4170] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:28:04 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 16 21:28:04.949139 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-b9c5c888-c89e-4f01-982f-657f668081f6 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerShowV254Test-server-940290877", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452", "networks": "none"}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:28:05.025821 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-a89e6923-4887-40fb-918b-8348a986dda0 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '029c76a0ba614b5880128c4309902c67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3b214f5044c7445592e177835a5d18fc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:05.050885 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-a89e6923-4887-40fb-918b-8348a986dda0 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] 10.222.0.16 "GET /compute/v2.1/servers/b01533f3-9a31-4246-b3e7-f12f638e8e89" status: 200 len: 1517 microversion: 2.1 time: 0.444596 Jul 16 21:28:05.062409 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2082/4171] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:28:04 2022] GET /compute/v2.1/servers/b01533f3-9a31-4246-b3e7-f12f638e8e89 => generated 1517 bytes in 463 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:28:05.068937 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-b9c5c888-c89e-4f01-982f-657f668081f6 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Flavor limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 16 21:28:05.069360 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-b9c5c888-c89e-4f01-982f-657f668081f6 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Image limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 16 21:28:05.069841 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-b9c5c888-c89e-4f01-982f-657f668081f6 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Flavor pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 16 21:28:05.070253 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-b9c5c888-c89e-4f01-982f-657f668081f6 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Image pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 16 21:28:05.070531 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-e2a58436-f310-44c3-a64b-7b0191f2758c tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:28:05.070643 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-b9c5c888-c89e-4f01-982f-657f668081f6 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 16 21:28:05.071339 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-e2a58436-f310-44c3-a64b-7b0191f2758c tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] [instance: 0b67ad87-5ea8-4837-90f8-b3bc99134f3b] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:28:05.080262 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e2a58436-f310-44c3-a64b-7b0191f2758c tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:05.080588 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e2a58436-f310-44c3-a64b-7b0191f2758c tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:05.081525 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e2a58436-f310-44c3-a64b-7b0191f2758c tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:05.082637 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e2a58436-f310-44c3-a64b-7b0191f2758c tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:05.096456 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b9c5c888-c89e-4f01-982f-657f668081f6 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:05.096915 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b9c5c888-c89e-4f01-982f-657f668081f6 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:05.097728 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b9c5c888-c89e-4f01-982f-657f668081f6 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:05.098169 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b9c5c888-c89e-4f01-982f-657f668081f6 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:05.133044 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e2a58436-f310-44c3-a64b-7b0191f2758c tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:05.133711 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e2a58436-f310-44c3-a64b-7b0191f2758c tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:05.136951 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.objects.instance [None req-e2a58436-f310-44c3-a64b-7b0191f2758c tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] Lazy-loading 'fault' on Instance uuid 0b67ad87-5ea8-4837-90f8-b3bc99134f3b {{(pid=138680) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:28:05.160397 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-b9c5c888-c89e-4f01-982f-657f668081f6 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Getting quotas for project 349811967f4b4d1a9ab6772602a205a6. Resources: {'ram', 'instances', 'cores'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:28:05.172706 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-b9c5c888-c89e-4f01-982f-657f668081f6 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Getting quotas for user 0f14ac04b7a44f17b946c5c8619554ad and project 349811967f4b4d1a9ab6772602a205a6. Resources: {'ram', 'instances', 'cores'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:28:05.201918 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-e2a58436-f310-44c3-a64b-7b0191f2758c tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6771589864f442cbbfc8abf018abbd09', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5bfd328e0f1446369531a23d15387cd6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:05.213887 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-e2a58436-f310-44c3-a64b-7b0191f2758c tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] 10.222.0.16 "GET /compute/v2.1/servers/0b67ad87-5ea8-4837-90f8-b3bc99134f3b" status: 200 len: 1304 microversion: 2.1 time: 0.146108 Jul 16 21:28:05.214685 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2083/4172] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:28:05 2022] GET /compute/v2.1/servers/0b67ad87-5ea8-4837-90f8-b3bc99134f3b => generated 1304 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:28:05.218247 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-b9c5c888-c89e-4f01-982f-657f668081f6 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Going to run 1 instances... {{(pid=138681) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 16 21:28:05.224538 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-b9c5c888-c89e-4f01-982f-657f668081f6 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] [instance: 93519c14-aee1-4da1-8cce-775e0ad268dc] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138681) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 16 21:28:05.395308 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-b9c5c888-c89e-4f01-982f-657f668081f6 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.54 time: 0.684299 Jul 16 21:28:05.397283 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2090/4173] 10.222.0.16 () {62 vars in 1187 bytes} [Sat Jul 16 21:28:04 2022] POST /compute/v2.1/servers => generated 380 bytes in 687 msecs (HTTP/1.1 202) 10 headers in 453 bytes (1 switches on core 0) Jul 16 21:28:05.407652 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-b5f523de-4340-481b-a914-1d99fe34408b tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:28:05.408496 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-b5f523de-4340-481b-a914-1d99fe34408b tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] [instance: 93519c14-aee1-4da1-8cce-775e0ad268dc] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:28:05.513833 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-b5f523de-4340-481b-a914-1d99fe34408b tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f14ac04b7a44f17b946c5c8619554ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '349811967f4b4d1a9ab6772602a205a6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:05.530181 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-b5f523de-4340-481b-a914-1d99fe34408b tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f14ac04b7a44f17b946c5c8619554ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '349811967f4b4d1a9ab6772602a205a6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:05.536144 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-b5f523de-4340-481b-a914-1d99fe34408b tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f14ac04b7a44f17b946c5c8619554ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '349811967f4b4d1a9ab6772602a205a6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:05.538977 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-b5f523de-4340-481b-a914-1d99fe34408b tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] 10.222.0.16 "GET /compute/v2.1/servers/93519c14-aee1-4da1-8cce-775e0ad268dc" status: 200 len: 1277 microversion: 2.54 time: 0.135127 Jul 16 21:28:05.539985 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2084/4174] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:28:05 2022] GET /compute/v2.1/servers/93519c14-aee1-4da1-8cce-775e0ad268dc => generated 1277 bytes in 137 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:28:06.055269 ncdv-master-1 devstack@n-api.service[138681]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jul 16 21:28:06.055852 ncdv-master-1 devstack@n-api.service[138681]: Sat Jul 16 21:28:06 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Jul 16 21:28:06.070361 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-7230582c-3270-4e50-bfdb-639156d296ca tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:28:06.071193 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-7230582c-3270-4e50-bfdb-639156d296ca tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] [instance: b01533f3-9a31-4246-b3e7-f12f638e8e89] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:28:06.087347 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7230582c-3270-4e50-bfdb-639156d296ca tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:06.088225 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7230582c-3270-4e50-bfdb-639156d296ca tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:06.089251 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7230582c-3270-4e50-bfdb-639156d296ca tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:06.089911 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7230582c-3270-4e50-bfdb-639156d296ca tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:06.233503 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-cb67a08f-02c1-4972-bf58-dcfcad37cb2e tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:28:06.234144 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-cb67a08f-02c1-4972-bf58-dcfcad37cb2e tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] [instance: 0b67ad87-5ea8-4837-90f8-b3bc99134f3b] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:28:06.275877 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cb67a08f-02c1-4972-bf58-dcfcad37cb2e tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:06.276794 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cb67a08f-02c1-4972-bf58-dcfcad37cb2e tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:06.277813 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cb67a08f-02c1-4972-bf58-dcfcad37cb2e tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:06.278410 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cb67a08f-02c1-4972-bf58-dcfcad37cb2e tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:06.309771 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-cb67a08f-02c1-4972-bf58-dcfcad37cb2e tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] HTTP exception thrown: Instance 0b67ad87-5ea8-4837-90f8-b3bc99134f3b could not be found. Jul 16 21:28:06.323521 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-cb67a08f-02c1-4972-bf58-dcfcad37cb2e tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] Request method failure captured: Jul 16 21:28:06.323521 ncdv-master-1 devstack@n-api.service[138680]: request: GET /compute/v2.1/servers/0b67ad87-5ea8-4837-90f8-b3bc99134f3b HTTP/1.1 Jul 16 21:28:06.323521 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:28:06.323521 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:28:06.323521 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:28:06.323521 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 0 Jul 16 21:28:06.323521 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:28:06.323521 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:28:06.323521 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:28:06.323521 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:28:06.323521 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0y1FkXtCIfJuzccZVdUTWn2sl-1VFIwunPAwwueaqcz4h3xKYB_R1BgfYk1EkrsjVTNJB4rZq7NTHLoVnoMbQ4Scg5hBty8ZInGFhxOBQhOHexnlAflVrVQoAUBVWqLJJxyeF1HByotMJepUPDDMIfho4bjwfOzMQddPFd1v3QdYatE Jul 16 21:28:06.323521 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:28:06.323521 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:28:06.323521 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:28:06.323521 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:28:06.323521 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:28:06.323521 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:28:06.323521 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: 5bfd328e0f1446369531a23d15387cd6 Jul 16 21:28:06.323521 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-VolumesNegativeTest-1381254418 Jul 16 21:28:06.323521 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:28:06.323521 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:28:06.323521 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/5bfd328e0f1446369531a23d15387cd6"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/5bfd328e0f1446369531a23d15387cd6"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_5bfd328e0f1446369531a23d15387cd6"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/5bfd328e0f1446369531a23d15387cd6"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:28:06.323521 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-VolumesNegativeTest-1381254418 Jul 16 21:28:06.323521 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: 5bfd328e0f1446369531a23d15387cd6 Jul 16 21:28:06.323521 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-VolumesNegativeTest-1381254418 Jul 16 21:28:06.323521 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-VolumesNegativeTest-1381254418-project Jul 16 21:28:06.323521 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:28:06.323521 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:28:06.323521 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: 6771589864f442cbbfc8abf018abbd09 Jul 16 21:28:06.323521 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-VolumesNegativeTest-1381254418-project Jul 16 21:28:06.323521 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 21:28:06.323521 ncdv-master-1 devstack@n-api.service[138680]: exception: Instance 0b67ad87-5ea8-4837-90f8-b3bc99134f3b could not be found. Jul 16 21:28:06.323521 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': '0b67ad87-5ea8-4837-90f8-b3bc99134f3b'} Jul 16 21:28:06.323521 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:28:06.323521 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:28:06.323521 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:28:06.323521 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:28:06.323521 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:28:06.323521 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:28:06.323521 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:28:06.323521 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:28:06.323521 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:28:06.323521 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:28:06.323521 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 0b67ad87-5ea8-4837-90f8-b3bc99134f3b could not be found. Jul 16 21:28:06.323521 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:28:06.323521 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:28:06.323521 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:28:06.323521 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:28:06.323521 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:28:06.323521 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:28:06.323521 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:28:06.323521 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:28:06.323521 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:28:06.323521 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:28:06.323521 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:28:06.323521 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:28:06.323521 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:28:06.323521 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:28:06.323521 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:28:06.323521 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:28:06.323521 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 0b67ad87-5ea8-4837-90f8-b3bc99134f3b could not be found. Jul 16 21:28:06.323521 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:28:06.323521 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:28:06.323521 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:28:06.333686 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:28:06.333686 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:28:06.333686 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:28:06.333686 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:28:06.333686 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:28:06.333686 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 0b67ad87-5ea8-4837-90f8-b3bc99134f3b could not be found. Jul 16 21:28:06.333686 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:28:06.333686 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-cb67a08f-02c1-4972-bf58-dcfcad37cb2e tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] Returning 404 to user: Instance 0b67ad87-5ea8-4837-90f8-b3bc99134f3b could not be found. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:28:06.333686 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-cb67a08f-02c1-4972-bf58-dcfcad37cb2e tempest-VolumesNegativeTest-1381254418 tempest-VolumesNegativeTest-1381254418-project] 10.222.0.16 "GET /compute/v2.1/servers/0b67ad87-5ea8-4837-90f8-b3bc99134f3b" status: 404 len: 111 microversion: 2.1 time: 0.095477 Jul 16 21:28:06.333686 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2085/4175] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:28:06 2022] GET /compute/v2.1/servers/0b67ad87-5ea8-4837-90f8-b3bc99134f3b => generated 111 bytes in 97 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Jul 16 21:28:06.558228 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-470d55fd-95e8-4cce-8155-125d3263112c tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:28:06.558864 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-470d55fd-95e8-4cce-8155-125d3263112c tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] [instance: 93519c14-aee1-4da1-8cce-775e0ad268dc] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:28:07.010443 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-470d55fd-95e8-4cce-8155-125d3263112c tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f14ac04b7a44f17b946c5c8619554ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '349811967f4b4d1a9ab6772602a205a6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:07.023652 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-470d55fd-95e8-4cce-8155-125d3263112c tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f14ac04b7a44f17b946c5c8619554ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '349811967f4b4d1a9ab6772602a205a6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:07.027814 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-470d55fd-95e8-4cce-8155-125d3263112c tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f14ac04b7a44f17b946c5c8619554ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '349811967f4b4d1a9ab6772602a205a6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:07.027814 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-470d55fd-95e8-4cce-8155-125d3263112c tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] 10.222.0.16 "GET /compute/v2.1/servers/93519c14-aee1-4da1-8cce-775e0ad268dc" status: 200 len: 1277 microversion: 2.54 time: 0.473553 Jul 16 21:28:07.028872 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2086/4176] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:28:06 2022] GET /compute/v2.1/servers/93519c14-aee1-4da1-8cce-775e0ad268dc => generated 1277 bytes in 476 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:28:07.554351 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-7230582c-3270-4e50-bfdb-639156d296ca tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '029c76a0ba614b5880128c4309902c67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3b214f5044c7445592e177835a5d18fc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:07.570925 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-7230582c-3270-4e50-bfdb-639156d296ca tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] 10.222.0.16 "GET /compute/v2.1/servers/b01533f3-9a31-4246-b3e7-f12f638e8e89" status: 200 len: 1517 microversion: 2.1 time: 1.505046 Jul 16 21:28:07.572250 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2091/4177] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:28:06 2022] GET /compute/v2.1/servers/b01533f3-9a31-4246-b3e7-f12f638e8e89 => generated 1517 bytes in 1507 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:28:08.049930 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-16989147-a408-4f66-bcd2-d1ea359a7f81 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:28:08.050608 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-16989147-a408-4f66-bcd2-d1ea359a7f81 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] [instance: 93519c14-aee1-4da1-8cce-775e0ad268dc] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:28:08.091597 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-16989147-a408-4f66-bcd2-d1ea359a7f81 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:08.109283 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-16989147-a408-4f66-bcd2-d1ea359a7f81 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:08.114869 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-16989147-a408-4f66-bcd2-d1ea359a7f81 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:08.121532 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-16989147-a408-4f66-bcd2-d1ea359a7f81 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:08.294951 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-16989147-a408-4f66-bcd2-d1ea359a7f81 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f14ac04b7a44f17b946c5c8619554ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '349811967f4b4d1a9ab6772602a205a6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:08.308740 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-16989147-a408-4f66-bcd2-d1ea359a7f81 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f14ac04b7a44f17b946c5c8619554ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '349811967f4b4d1a9ab6772602a205a6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:08.313426 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-16989147-a408-4f66-bcd2-d1ea359a7f81 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f14ac04b7a44f17b946c5c8619554ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '349811967f4b4d1a9ab6772602a205a6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:08.314444 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-16989147-a408-4f66-bcd2-d1ea359a7f81 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] 10.222.0.16 "GET /compute/v2.1/servers/93519c14-aee1-4da1-8cce-775e0ad268dc" status: 200 len: 1329 microversion: 2.54 time: 0.270098 Jul 16 21:28:08.315541 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2087/4178] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:28:08 2022] GET /compute/v2.1/servers/93519c14-aee1-4da1-8cce-775e0ad268dc => generated 1329 bytes in 272 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:28:08.553255 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-6d372098-9089-4acd-a808-56f277440621 req-15592a0d-2d87-474a-90be-b92d9b8f0d93 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "b01533f3-9a31-4246-b3e7-f12f638e8e89", "name": "network-vif-deleted", "tag": "3ce77aec-ed61-42cc-8170-868f6c10b522"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:28:08.569680 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-6d372098-9089-4acd-a808-56f277440621 req-15592a0d-2d87-474a-90be-b92d9b8f0d93 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:08.571365 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-6d372098-9089-4acd-a808-56f277440621 req-15592a0d-2d87-474a-90be-b92d9b8f0d93 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:08.591278 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-b843abd5-06bc-421d-adbb-f110cd492d70 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:28:08.591935 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-b843abd5-06bc-421d-adbb-f110cd492d70 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] [instance: b01533f3-9a31-4246-b3e7-f12f638e8e89] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:28:08.606459 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b843abd5-06bc-421d-adbb-f110cd492d70 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:08.611589 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b843abd5-06bc-421d-adbb-f110cd492d70 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:08.614216 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b843abd5-06bc-421d-adbb-f110cd492d70 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:08.614927 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b843abd5-06bc-421d-adbb-f110cd492d70 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:08.622423 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-6d372098-9089-4acd-a808-56f277440621 req-15592a0d-2d87-474a-90be-b92d9b8f0d93 service nova] Creating event network-vif-deleted:3ce77aec-ed61-42cc-8170-868f6c10b522 for instance b01533f3-9a31-4246-b3e7-f12f638e8e89 on nch2-master-1 Jul 16 21:28:08.638859 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-6d372098-9089-4acd-a808-56f277440621 req-15592a0d-2d87-474a-90be-b92d9b8f0d93 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.089058 Jul 16 21:28:08.639775 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2092/4179] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:28:08 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:28:08.770000 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-b843abd5-06bc-421d-adbb-f110cd492d70 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '029c76a0ba614b5880128c4309902c67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3b214f5044c7445592e177835a5d18fc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:08.772747 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell0:SELECT=10 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:28:08.783510 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-b843abd5-06bc-421d-adbb-f110cd492d70 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] 10.222.0.16 "GET /compute/v2.1/servers/b01533f3-9a31-4246-b3e7-f12f638e8e89" status: 200 len: 1475 microversion: 2.1 time: 0.197919 Jul 16 21:28:08.784423 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2088/4180] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:28:08 2022] GET /compute/v2.1/servers/b01533f3-9a31-4246-b3e7-f12f638e8e89 => generated 1475 bytes in 200 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:28:09.336897 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-5cef40f0-ce42-46ba-b0ae-ee9c1ecbf29f tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:28:09.337616 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-5cef40f0-ce42-46ba-b0ae-ee9c1ecbf29f tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] [instance: 93519c14-aee1-4da1-8cce-775e0ad268dc] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:28:09.375679 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5cef40f0-ce42-46ba-b0ae-ee9c1ecbf29f tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:09.375679 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5cef40f0-ce42-46ba-b0ae-ee9c1ecbf29f tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:09.375679 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5cef40f0-ce42-46ba-b0ae-ee9c1ecbf29f tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:09.375679 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5cef40f0-ce42-46ba-b0ae-ee9c1ecbf29f tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:09.466559 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-5cef40f0-ce42-46ba-b0ae-ee9c1ecbf29f tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f14ac04b7a44f17b946c5c8619554ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '349811967f4b4d1a9ab6772602a205a6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:09.477259 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-5cef40f0-ce42-46ba-b0ae-ee9c1ecbf29f tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f14ac04b7a44f17b946c5c8619554ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '349811967f4b4d1a9ab6772602a205a6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:09.478882 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-5cef40f0-ce42-46ba-b0ae-ee9c1ecbf29f tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f14ac04b7a44f17b946c5c8619554ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '349811967f4b4d1a9ab6772602a205a6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:09.480155 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-5cef40f0-ce42-46ba-b0ae-ee9c1ecbf29f tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] 10.222.0.16 "GET /compute/v2.1/servers/93519c14-aee1-4da1-8cce-775e0ad268dc" status: 200 len: 1347 microversion: 2.54 time: 0.149445 Jul 16 21:28:09.491078 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2093/4181] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:28:09 2022] GET /compute/v2.1/servers/93519c14-aee1-4da1-8cce-775e0ad268dc => generated 1347 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:28:09.807177 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-0573e088-9e12-405b-b45d-31c23d1c86a2 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:28:09.807849 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-0573e088-9e12-405b-b45d-31c23d1c86a2 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] [instance: b01533f3-9a31-4246-b3e7-f12f638e8e89] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:28:09.831456 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0573e088-9e12-405b-b45d-31c23d1c86a2 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:09.832335 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0573e088-9e12-405b-b45d-31c23d1c86a2 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:09.834430 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0573e088-9e12-405b-b45d-31c23d1c86a2 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:09.835399 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0573e088-9e12-405b-b45d-31c23d1c86a2 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:09.904786 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0573e088-9e12-405b-b45d-31c23d1c86a2 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:09.911667 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0573e088-9e12-405b-b45d-31c23d1c86a2 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:09.912954 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.objects.instance [None req-0573e088-9e12-405b-b45d-31c23d1c86a2 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lazy-loading 'fault' on Instance uuid b01533f3-9a31-4246-b3e7-f12f638e8e89 {{(pid=138680) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:28:10.051837 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-0573e088-9e12-405b-b45d-31c23d1c86a2 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '029c76a0ba614b5880128c4309902c67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3b214f5044c7445592e177835a5d18fc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:10.067429 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-0573e088-9e12-405b-b45d-31c23d1c86a2 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] 10.222.0.16 "GET /compute/v2.1/servers/b01533f3-9a31-4246-b3e7-f12f638e8e89" status: 200 len: 1310 microversion: 2.1 time: 0.265511 Jul 16 21:28:10.070146 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2089/4182] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:28:09 2022] GET /compute/v2.1/servers/b01533f3-9a31-4246-b3e7-f12f638e8e89 => generated 1310 bytes in 272 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:28:10.515436 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-5e77eb50-5ff5-4fc2-ad1d-b8f376f4b039 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:28:10.517272 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-5e77eb50-5ff5-4fc2-ad1d-b8f376f4b039 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] [instance: 93519c14-aee1-4da1-8cce-775e0ad268dc] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:28:10.528473 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5e77eb50-5ff5-4fc2-ad1d-b8f376f4b039 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:10.529262 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5e77eb50-5ff5-4fc2-ad1d-b8f376f4b039 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:10.530283 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5e77eb50-5ff5-4fc2-ad1d-b8f376f4b039 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:10.530909 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5e77eb50-5ff5-4fc2-ad1d-b8f376f4b039 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:10.549465 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=96,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:28:10.622773 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-5e77eb50-5ff5-4fc2-ad1d-b8f376f4b039 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f14ac04b7a44f17b946c5c8619554ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '349811967f4b4d1a9ab6772602a205a6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:10.636237 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-5e77eb50-5ff5-4fc2-ad1d-b8f376f4b039 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f14ac04b7a44f17b946c5c8619554ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '349811967f4b4d1a9ab6772602a205a6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:10.637931 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-5e77eb50-5ff5-4fc2-ad1d-b8f376f4b039 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f14ac04b7a44f17b946c5c8619554ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '349811967f4b4d1a9ab6772602a205a6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:10.643511 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-5e77eb50-5ff5-4fc2-ad1d-b8f376f4b039 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] 10.222.0.16 "GET /compute/v2.1/servers/93519c14-aee1-4da1-8cce-775e0ad268dc" status: 200 len: 1335 microversion: 2.54 time: 0.132113 Jul 16 21:28:10.647112 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2094/4183] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:28:10 2022] GET /compute/v2.1/servers/93519c14-aee1-4da1-8cce-775e0ad268dc => generated 1335 bytes in 136 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:28:11.088638 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-4f2815b5-37a8-4f3f-8fa5-ea20184306fb tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:28:11.090291 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-4f2815b5-37a8-4f3f-8fa5-ea20184306fb tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] [instance: b01533f3-9a31-4246-b3e7-f12f638e8e89] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:28:11.110865 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4f2815b5-37a8-4f3f-8fa5-ea20184306fb tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:11.111986 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4f2815b5-37a8-4f3f-8fa5-ea20184306fb tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:11.112371 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4f2815b5-37a8-4f3f-8fa5-ea20184306fb tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:11.112968 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4f2815b5-37a8-4f3f-8fa5-ea20184306fb tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:11.153446 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-4f2815b5-37a8-4f3f-8fa5-ea20184306fb tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] HTTP exception thrown: Instance b01533f3-9a31-4246-b3e7-f12f638e8e89 could not be found. Jul 16 21:28:11.155064 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-4f2815b5-37a8-4f3f-8fa5-ea20184306fb tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Request method failure captured: Jul 16 21:28:11.155064 ncdv-master-1 devstack@n-api.service[138680]: request: GET /compute/v2.1/servers/b01533f3-9a31-4246-b3e7-f12f638e8e89 HTTP/1.1 Jul 16 21:28:11.155064 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:28:11.155064 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:28:11.155064 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:28:11.155064 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 0 Jul 16 21:28:11.155064 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:28:11.155064 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:28:11.155064 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:28:11.155064 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:28:11.155064 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0y0f71twFMCVGLcyplWyRtyAM2CfOZwj4laMJ_sqbGEEyWtMp5KkAeIFFiT5aXWve6fmB41BoCEhDgyJ_zvQFNfWsoMy5Nd2qMpH4YpTCGyEpNT3rZaP4jGMzoknx7H-D8Y7484jvmo3TtoXuL69MLIEYajD2R1bB1NIQs0LYN96Amk Jul 16 21:28:11.155064 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:28:11.155064 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:28:11.155064 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:28:11.155064 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:28:11.155064 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:28:11.155064 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:28:11.155064 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: 3b214f5044c7445592e177835a5d18fc Jul 16 21:28:11.155064 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-ListServerFiltersTestJSON-921481021 Jul 16 21:28:11.155064 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:28:11.155064 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:28:11.155064 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/3b214f5044c7445592e177835a5d18fc"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/3b214f5044c7445592e177835a5d18fc"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_3b214f5044c7445592e177835a5d18fc"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/3b214f5044c7445592e177835a5d18fc"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:28:11.155064 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-ListServerFiltersTestJSON-921481021 Jul 16 21:28:11.155064 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: 3b214f5044c7445592e177835a5d18fc Jul 16 21:28:11.155064 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-ListServerFiltersTestJSON-921481021 Jul 16 21:28:11.155064 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-ListServerFiltersTestJSON-921481021-project Jul 16 21:28:11.155064 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:28:11.155064 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:28:11.155064 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: 029c76a0ba614b5880128c4309902c67 Jul 16 21:28:11.155064 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-ListServerFiltersTestJSON-921481021-project Jul 16 21:28:11.155064 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 21:28:11.155064 ncdv-master-1 devstack@n-api.service[138680]: exception: Instance b01533f3-9a31-4246-b3e7-f12f638e8e89 could not be found. Jul 16 21:28:11.155064 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': 'b01533f3-9a31-4246-b3e7-f12f638e8e89'} Jul 16 21:28:11.155064 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:28:11.155064 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:28:11.155064 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:28:11.155064 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:28:11.155064 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:28:11.155064 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:28:11.155064 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:28:11.155064 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:28:11.155064 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:28:11.155064 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:28:11.155064 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance b01533f3-9a31-4246-b3e7-f12f638e8e89 could not be found. Jul 16 21:28:11.155064 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:28:11.155064 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:28:11.155064 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:28:11.155064 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:28:11.155064 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:28:11.155064 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:28:11.155064 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:28:11.155064 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:28:11.155064 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:28:11.155064 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:28:11.155064 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:28:11.155064 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:28:11.155064 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:28:11.155064 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:28:11.155064 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:28:11.155064 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:28:11.155064 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance b01533f3-9a31-4246-b3e7-f12f638e8e89 could not be found. Jul 16 21:28:11.155064 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:28:11.155064 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:28:11.155064 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:28:11.164942 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:28:11.164942 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:28:11.164942 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:28:11.164942 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:28:11.164942 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:28:11.164942 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance b01533f3-9a31-4246-b3e7-f12f638e8e89 could not be found. Jul 16 21:28:11.164942 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:28:11.170099 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-4f2815b5-37a8-4f3f-8fa5-ea20184306fb tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Returning 404 to user: Instance b01533f3-9a31-4246-b3e7-f12f638e8e89 could not be found. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:28:11.173075 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-4f2815b5-37a8-4f3f-8fa5-ea20184306fb tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] 10.222.0.16 "GET /compute/v2.1/servers/b01533f3-9a31-4246-b3e7-f12f638e8e89" status: 404 len: 111 microversion: 2.1 time: 0.088971 Jul 16 21:28:11.174013 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2090/4184] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:28:11 2022] GET /compute/v2.1/servers/b01533f3-9a31-4246-b3e7-f12f638e8e89 => generated 111 bytes in 92 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:28:11.183773 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-ed699843-4454-457e-9f46-cc3aff0ab969 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:28:11.184996 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-ed699843-4454-457e-9f46-cc3aff0ab969 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] [instance: cf38328b-40fb-4537-b524-aaeb0a0abfb4] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:28:11.198387 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ed699843-4454-457e-9f46-cc3aff0ab969 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:11.199085 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ed699843-4454-457e-9f46-cc3aff0ab969 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:11.201868 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ed699843-4454-457e-9f46-cc3aff0ab969 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:11.202647 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ed699843-4454-457e-9f46-cc3aff0ab969 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:11.246015 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-ed699843-4454-457e-9f46-cc3aff0ab969 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] [instance: cf38328b-40fb-4537-b524-aaeb0a0abfb4] Going to try to terminate instance {{(pid=138681) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 16 21:28:11.648673 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-ed699843-4454-457e-9f46-cc3aff0ab969 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] 10.222.0.16 "DELETE /compute/v2.1/servers/cf38328b-40fb-4537-b524-aaeb0a0abfb4" status: 204 len: 0 microversion: 2.1 time: 0.469943 Jul 16 21:28:11.656673 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2095/4185] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:28:11 2022] DELETE /compute/v2.1/servers/cf38328b-40fb-4537-b524-aaeb0a0abfb4 => generated 0 bytes in 477 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Jul 16 21:28:11.663573 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-44f11f93-cc94-49e3-8da2-8290c841a28b tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:28:11.664667 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-44f11f93-cc94-49e3-8da2-8290c841a28b tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] [instance: cf38328b-40fb-4537-b524-aaeb0a0abfb4] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:28:11.678977 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-2f88c0b7-833e-4956-8297-92d1c7aaa6ec tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:28:11.685126 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-2f88c0b7-833e-4956-8297-92d1c7aaa6ec tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] [instance: 93519c14-aee1-4da1-8cce-775e0ad268dc] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:28:11.686183 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-44f11f93-cc94-49e3-8da2-8290c841a28b tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:11.686855 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-44f11f93-cc94-49e3-8da2-8290c841a28b tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:11.690207 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-44f11f93-cc94-49e3-8da2-8290c841a28b tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:11.690973 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-44f11f93-cc94-49e3-8da2-8290c841a28b tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:11.709799 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2f88c0b7-833e-4956-8297-92d1c7aaa6ec tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:11.715758 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2f88c0b7-833e-4956-8297-92d1c7aaa6ec tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:11.715758 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2f88c0b7-833e-4956-8297-92d1c7aaa6ec tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:11.715758 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2f88c0b7-833e-4956-8297-92d1c7aaa6ec tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:12.188699 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-2f88c0b7-833e-4956-8297-92d1c7aaa6ec tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f14ac04b7a44f17b946c5c8619554ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '349811967f4b4d1a9ab6772602a205a6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:12.199565 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-2f88c0b7-833e-4956-8297-92d1c7aaa6ec tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f14ac04b7a44f17b946c5c8619554ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '349811967f4b4d1a9ab6772602a205a6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:12.214768 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-2f88c0b7-833e-4956-8297-92d1c7aaa6ec tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f14ac04b7a44f17b946c5c8619554ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '349811967f4b4d1a9ab6772602a205a6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:12.217101 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-2f88c0b7-833e-4956-8297-92d1c7aaa6ec tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] 10.222.0.16 "GET /compute/v2.1/servers/93519c14-aee1-4da1-8cce-775e0ad268dc" status: 200 len: 1335 microversion: 2.54 time: 0.547878 Jul 16 21:28:12.225051 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2096/4186] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:28:11 2022] GET /compute/v2.1/servers/93519c14-aee1-4da1-8cce-775e0ad268dc => generated 1335 bytes in 557 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 16 21:28:12.426391 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-44f11f93-cc94-49e3-8da2-8290c841a28b tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '029c76a0ba614b5880128c4309902c67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3b214f5044c7445592e177835a5d18fc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:12.448612 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-44f11f93-cc94-49e3-8da2-8290c841a28b tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] 10.222.0.16 "GET /compute/v2.1/servers/cf38328b-40fb-4537-b524-aaeb0a0abfb4" status: 200 len: 1517 microversion: 2.1 time: 0.776069 Jul 16 21:28:12.448612 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2091/4187] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:28:11 2022] GET /compute/v2.1/servers/cf38328b-40fb-4537-b524-aaeb0a0abfb4 => generated 1517 bytes in 785 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:28:13.244182 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-e744a194-d379-44a6-a338-42299ed137e3 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:28:13.244847 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-e744a194-d379-44a6-a338-42299ed137e3 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] [instance: 93519c14-aee1-4da1-8cce-775e0ad268dc] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:28:13.280751 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e744a194-d379-44a6-a338-42299ed137e3 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:13.291653 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e744a194-d379-44a6-a338-42299ed137e3 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.011s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:13.296084 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e744a194-d379-44a6-a338-42299ed137e3 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:13.301763 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e744a194-d379-44a6-a338-42299ed137e3 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:13.465566 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-6bf30591-e9ed-4ece-9815-5500da4a43ee tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:28:13.466269 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-6bf30591-e9ed-4ece-9815-5500da4a43ee tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] [instance: cf38328b-40fb-4537-b524-aaeb0a0abfb4] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:28:13.488966 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6bf30591-e9ed-4ece-9815-5500da4a43ee tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:13.489426 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6bf30591-e9ed-4ece-9815-5500da4a43ee tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:13.490258 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6bf30591-e9ed-4ece-9815-5500da4a43ee tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:13.490701 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6bf30591-e9ed-4ece-9815-5500da4a43ee tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:13.537336 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-e744a194-d379-44a6-a338-42299ed137e3 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f14ac04b7a44f17b946c5c8619554ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '349811967f4b4d1a9ab6772602a205a6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:13.559631 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-e744a194-d379-44a6-a338-42299ed137e3 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f14ac04b7a44f17b946c5c8619554ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '349811967f4b4d1a9ab6772602a205a6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:13.563829 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-e744a194-d379-44a6-a338-42299ed137e3 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f14ac04b7a44f17b946c5c8619554ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '349811967f4b4d1a9ab6772602a205a6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:13.565049 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-e744a194-d379-44a6-a338-42299ed137e3 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] 10.222.0.16 "GET /compute/v2.1/servers/93519c14-aee1-4da1-8cce-775e0ad268dc" status: 200 len: 1335 microversion: 2.54 time: 0.325334 Jul 16 21:28:13.568260 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2097/4188] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:28:13 2022] GET /compute/v2.1/servers/93519c14-aee1-4da1-8cce-775e0ad268dc => generated 1335 bytes in 329 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:28:14.590018 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-3e884d31-ba05-4feb-93c3-c4519d3b058b tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:28:14.591456 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-3e884d31-ba05-4feb-93c3-c4519d3b058b tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] [instance: 93519c14-aee1-4da1-8cce-775e0ad268dc] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:28:14.622057 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3e884d31-ba05-4feb-93c3-c4519d3b058b tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:14.622991 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3e884d31-ba05-4feb-93c3-c4519d3b058b tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:14.624510 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3e884d31-ba05-4feb-93c3-c4519d3b058b tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:14.626686 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3e884d31-ba05-4feb-93c3-c4519d3b058b tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:14.916697 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-6bf30591-e9ed-4ece-9815-5500da4a43ee tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '029c76a0ba614b5880128c4309902c67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3b214f5044c7445592e177835a5d18fc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:14.930745 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-6bf30591-e9ed-4ece-9815-5500da4a43ee tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] 10.222.0.16 "GET /compute/v2.1/servers/cf38328b-40fb-4537-b524-aaeb0a0abfb4" status: 200 len: 1517 microversion: 2.1 time: 1.475837 Jul 16 21:28:14.938452 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2092/4189] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:28:13 2022] GET /compute/v2.1/servers/cf38328b-40fb-4537-b524-aaeb0a0abfb4 => generated 1517 bytes in 1488 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:28:15.030947 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-b9f44e91-dd21-4c85-8dfd-5331278eaa48 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Action: 'create', calling method: .version_select at 0x7fdd9dbf5160>, body: {"keypair": {"name": "tempest-keypair-545593029", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBAbK40a1XRHCR7B7wV+8KMQkOpVNNRKCp8Tz1bhwXZvKfyK42S/SBTkS2BxL3GBNY6MUPhcLRXj7ih4vsJb7h79Thxq+kNtI1YyHx1+gCCCV8fd/w8DZwOE2EkollE5G2Q=="}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:28:15.050392 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-b9f44e91-dd21-4c85-8dfd-5331278eaa48 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Getting quotas for project e451ee7bfd2d4ab58d30c33fef334b55. Resources: {'key_pairs'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:28:15.055723 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-b9f44e91-dd21-4c85-8dfd-5331278eaa48 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Getting quotas for user ca2c9e7e6d61414cbf2c7e17753bc0df and project e451ee7bfd2d4ab58d30c33fef334b55. Resources: {'key_pairs'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:28:15.060418 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-3e884d31-ba05-4feb-93c3-c4519d3b058b tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f14ac04b7a44f17b946c5c8619554ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '349811967f4b4d1a9ab6772602a205a6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:15.069969 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-3e884d31-ba05-4feb-93c3-c4519d3b058b tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f14ac04b7a44f17b946c5c8619554ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '349811967f4b4d1a9ab6772602a205a6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:15.072009 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-3e884d31-ba05-4feb-93c3-c4519d3b058b tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f14ac04b7a44f17b946c5c8619554ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '349811967f4b4d1a9ab6772602a205a6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:15.073372 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-3e884d31-ba05-4feb-93c3-c4519d3b058b tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] 10.222.0.16 "GET /compute/v2.1/servers/93519c14-aee1-4da1-8cce-775e0ad268dc" status: 200 len: 1356 microversion: 2.54 time: 0.487658 Jul 16 21:28:15.074364 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2098/4190] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:28:14 2022] GET /compute/v2.1/servers/93519c14-aee1-4da1-8cce-775e0ad268dc => generated 1356 bytes in 490 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:28:15.097287 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-0d97d5a0-f4e8-4767-a61f-a0ad396429bc tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Action: 'create', calling method: .version_select at 0x7fdd9c238c10>, body: {"keypair": {"name": "tempest-ServerShowV254Test-keypair-635440880", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBLf9DlnGJFnYw97vNV1ZhIe4DP3Jn7LeJFh/gZr0RaW8DRo5gO467lMr3K1W3y/SUuLMOpWfJ5Ca1Uy5f817x4a9gGCNzYOtzJ5e+9numJ7d1v75WE1P+6QiL7trmGemPQ=="}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:28:15.116083 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-0d97d5a0-f4e8-4767-a61f-a0ad396429bc tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Getting quotas for project 349811967f4b4d1a9ab6772602a205a6. Resources: {'key_pairs'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:28:15.122541 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-0d97d5a0-f4e8-4767-a61f-a0ad396429bc tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Getting quotas for user 0f14ac04b7a44f17b946c5c8619554ad and project 349811967f4b4d1a9ab6772602a205a6. Resources: {'key_pairs'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:28:15.124611 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell0:SELECT=8 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:28:15.183377 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-b9f44e91-dd21-4c85-8dfd-5331278eaa48 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] 10.222.0.16 "POST /compute/v2.1/os-keypairs" status: 201 len: 400 microversion: 2.20 time: 0.241864 Jul 16 21:28:15.184015 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2093/4191] 10.222.0.16 () {62 vars in 1199 bytes} [Sat Jul 16 21:28:14 2022] POST /compute/v2.1/os-keypairs => generated 400 bytes in 243 msecs (HTTP/1.1 201) 9 headers in 364 bytes (1 switches on core 0) Jul 16 21:28:15.229265 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-0d97d5a0-f4e8-4767-a61f-a0ad396429bc tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] 10.222.0.16 "POST /compute/v2.1/os-keypairs" status: 201 len: 419 microversion: 2.54 time: 0.137426 Jul 16 21:28:15.229880 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2099/4192] 10.222.0.16 () {62 vars in 1199 bytes} [Sat Jul 16 21:28:15 2022] POST /compute/v2.1/os-keypairs => generated 419 bytes in 139 msecs (HTTP/1.1 201) 9 headers in 364 bytes (1 switches on core 0) Jul 16 21:28:15.240945 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-f789b780-6312-4f08-ab3a-911b4d4aefdc tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Action: 'action', calling method: >, body: {"rebuild": {"key_name": "tempest-ServerShowV254Test-keypair-635440880", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9"}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:28:15.242754 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-f789b780-6312-4f08-ab3a-911b4d4aefdc tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] [instance: 93519c14-aee1-4da1-8cce-775e0ad268dc] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:28:15.249653 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f789b780-6312-4f08-ab3a-911b4d4aefdc tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:15.250869 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f789b780-6312-4f08-ab3a-911b4d4aefdc tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:15.251514 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f789b780-6312-4f08-ab3a-911b4d4aefdc tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:15.251958 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f789b780-6312-4f08-ab3a-911b4d4aefdc tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:15.280322 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_api:SELECT=97,nova_api:DELETE=1,nova_api:UPDATE=1,nova_api:INSERT=1 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:28:15.359061 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-f789b780-6312-4f08-ab3a-911b4d4aefdc tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Flavor limits 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 16 21:28:15.360688 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-f789b780-6312-4f08-ab3a-911b4d4aefdc tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Image limits 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 16 21:28:15.361529 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-f789b780-6312-4f08-ab3a-911b4d4aefdc tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Flavor pref 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 16 21:28:15.362115 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-f789b780-6312-4f08-ab3a-911b4d4aefdc tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Image pref 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 16 21:28:15.362685 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-f789b780-6312-4f08-ab3a-911b4d4aefdc tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 16 21:28:15.517280 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-3381643b-68a1-417e-acf4-7eeffd9bcb00 req-c4bebec8-f2d6-4c6e-b40e-3a6a20ec204d service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "cf38328b-40fb-4537-b524-aaeb0a0abfb4", "name": "network-vif-deleted", "tag": "f08f43fd-08c2-46ba-aec3-1900562b601f"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:28:15.549097 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-3381643b-68a1-417e-acf4-7eeffd9bcb00 req-c4bebec8-f2d6-4c6e-b40e-3a6a20ec204d service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:15.549097 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-3381643b-68a1-417e-acf4-7eeffd9bcb00 req-c4bebec8-f2d6-4c6e-b40e-3a6a20ec204d service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.013s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:15.563828 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-3381643b-68a1-417e-acf4-7eeffd9bcb00 req-c4bebec8-f2d6-4c6e-b40e-3a6a20ec204d service nova] Creating event network-vif-deleted:f08f43fd-08c2-46ba-aec3-1900562b601f for instance cf38328b-40fb-4537-b524-aaeb0a0abfb4 on nch2-master-1 Jul 16 21:28:15.570398 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-3381643b-68a1-417e-acf4-7eeffd9bcb00 req-c4bebec8-f2d6-4c6e-b40e-3a6a20ec204d service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.056849 Jul 16 21:28:15.571750 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2100/4193] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:28:15 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:28:15.728001 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-f789b780-6312-4f08-ab3a-911b4d4aefdc tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] [instance: 93519c14-aee1-4da1-8cce-775e0ad268dc] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:28:15.740096 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f789b780-6312-4f08-ab3a-911b4d4aefdc tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:15.741397 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f789b780-6312-4f08-ab3a-911b4d4aefdc tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:15.742397 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f789b780-6312-4f08-ab3a-911b4d4aefdc tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:15.742988 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f789b780-6312-4f08-ab3a-911b4d4aefdc tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:15.829734 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-f789b780-6312-4f08-ab3a-911b4d4aefdc tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] 10.222.0.16 "POST /compute/v2.1/servers/93519c14-aee1-4da1-8cce-775e0ad268dc/action" status: 202 len: 1138 microversion: 2.54 time: 0.592430 Jul 16 21:28:15.831278 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2094/4194] 10.222.0.16 () {62 vars in 1319 bytes} [Sat Jul 16 21:28:15 2022] POST /compute/v2.1/servers/93519c14-aee1-4da1-8cce-775e0ad268dc/action => generated 1138 bytes in 595 msecs (HTTP/1.1 202) 10 headers in 454 bytes (1 switches on core 0) Jul 16 21:28:15.856603 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-4442bc59-7478-4ab5-b557-d48703868ebe tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:28:15.864140 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-4442bc59-7478-4ab5-b557-d48703868ebe tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] [instance: 93519c14-aee1-4da1-8cce-775e0ad268dc] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:28:15.891627 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4442bc59-7478-4ab5-b557-d48703868ebe tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:15.892318 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4442bc59-7478-4ab5-b557-d48703868ebe tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:15.893221 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4442bc59-7478-4ab5-b557-d48703868ebe tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:15.893637 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4442bc59-7478-4ab5-b557-d48703868ebe tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:15.957900 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-acf6675d-783b-4eed-9263-4eebdf10673f tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:28:15.958718 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-acf6675d-783b-4eed-9263-4eebdf10673f tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] [instance: cf38328b-40fb-4537-b524-aaeb0a0abfb4] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:28:15.968291 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-acf6675d-783b-4eed-9263-4eebdf10673f tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:15.969368 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-acf6675d-783b-4eed-9263-4eebdf10673f tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:15.970474 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-acf6675d-783b-4eed-9263-4eebdf10673f tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:15.970993 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-acf6675d-783b-4eed-9263-4eebdf10673f tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:16.036604 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-acf6675d-783b-4eed-9263-4eebdf10673f tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:16.037225 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-acf6675d-783b-4eed-9263-4eebdf10673f tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:16.037806 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.objects.instance [None req-acf6675d-783b-4eed-9263-4eebdf10673f tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lazy-loading 'fault' on Instance uuid cf38328b-40fb-4537-b524-aaeb0a0abfb4 {{(pid=138680) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:28:16.143982 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-acf6675d-783b-4eed-9263-4eebdf10673f tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '029c76a0ba614b5880128c4309902c67', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3b214f5044c7445592e177835a5d18fc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:16.157693 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-acf6675d-783b-4eed-9263-4eebdf10673f tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] 10.222.0.16 "GET /compute/v2.1/servers/cf38328b-40fb-4537-b524-aaeb0a0abfb4" status: 200 len: 1311 microversion: 2.1 time: 0.206769 Jul 16 21:28:16.160855 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2095/4195] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:28:15 2022] GET /compute/v2.1/servers/cf38328b-40fb-4537-b524-aaeb0a0abfb4 => generated 1311 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:28:16.177555 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-4442bc59-7478-4ab5-b557-d48703868ebe tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f14ac04b7a44f17b946c5c8619554ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '349811967f4b4d1a9ab6772602a205a6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:16.186330 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-4442bc59-7478-4ab5-b557-d48703868ebe tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f14ac04b7a44f17b946c5c8619554ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '349811967f4b4d1a9ab6772602a205a6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:16.188820 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-4442bc59-7478-4ab5-b557-d48703868ebe tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f14ac04b7a44f17b946c5c8619554ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '349811967f4b4d1a9ab6772602a205a6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:16.190244 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-4442bc59-7478-4ab5-b557-d48703868ebe tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] 10.222.0.16 "GET /compute/v2.1/servers/93519c14-aee1-4da1-8cce-775e0ad268dc" status: 200 len: 1407 microversion: 2.54 time: 0.347713 Jul 16 21:28:16.191309 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2101/4196] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:28:15 2022] GET /compute/v2.1/servers/93519c14-aee1-4da1-8cce-775e0ad268dc => generated 1407 bytes in 350 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:28:17.186785 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-5e854388-055b-4920-b135-f215b8a93bb2 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:28:17.188581 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-5e854388-055b-4920-b135-f215b8a93bb2 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] [instance: cf38328b-40fb-4537-b524-aaeb0a0abfb4] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:28:17.205103 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5e854388-055b-4920-b135-f215b8a93bb2 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:17.205570 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5e854388-055b-4920-b135-f215b8a93bb2 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:17.206376 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5e854388-055b-4920-b135-f215b8a93bb2 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:17.206822 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5e854388-055b-4920-b135-f215b8a93bb2 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:17.211836 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-edc77d8a-4f99-47c6-9082-1872c417eb80 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:28:17.213243 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-edc77d8a-4f99-47c6-9082-1872c417eb80 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] [instance: 93519c14-aee1-4da1-8cce-775e0ad268dc] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:28:17.222388 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-edc77d8a-4f99-47c6-9082-1872c417eb80 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:17.223223 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-edc77d8a-4f99-47c6-9082-1872c417eb80 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:17.225956 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-edc77d8a-4f99-47c6-9082-1872c417eb80 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:17.226498 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-edc77d8a-4f99-47c6-9082-1872c417eb80 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:17.269610 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-5e854388-055b-4920-b135-f215b8a93bb2 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] HTTP exception thrown: Instance cf38328b-40fb-4537-b524-aaeb0a0abfb4 could not be found. Jul 16 21:28:17.271620 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-5e854388-055b-4920-b135-f215b8a93bb2 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Request method failure captured: Jul 16 21:28:17.271620 ncdv-master-1 devstack@n-api.service[138680]: request: GET /compute/v2.1/servers/cf38328b-40fb-4537-b524-aaeb0a0abfb4 HTTP/1.1 Jul 16 21:28:17.271620 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:28:17.271620 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:28:17.271620 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:28:17.271620 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 0 Jul 16 21:28:17.271620 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:28:17.271620 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:28:17.271620 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:28:17.271620 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:28:17.271620 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0y0f71twFMCVGLcyplWyRtyAM2CfOZwj4laMJ_sqbGEEyWtMp5KkAeIFFiT5aXWve6fmB41BoCEhDgyJ_zvQFNfWsoMy5Nd2qMpH4YpTCGyEpNT3rZaP4jGMzoknx7H-D8Y7484jvmo3TtoXuL69MLIEYajD2R1bB1NIQs0LYN96Amk Jul 16 21:28:17.271620 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:28:17.271620 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:28:17.271620 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:28:17.271620 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:28:17.271620 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:28:17.271620 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:28:17.271620 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: 3b214f5044c7445592e177835a5d18fc Jul 16 21:28:17.271620 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-ListServerFiltersTestJSON-921481021 Jul 16 21:28:17.271620 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:28:17.271620 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:28:17.271620 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/3b214f5044c7445592e177835a5d18fc"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/3b214f5044c7445592e177835a5d18fc"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_3b214f5044c7445592e177835a5d18fc"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/3b214f5044c7445592e177835a5d18fc"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:28:17.271620 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-ListServerFiltersTestJSON-921481021 Jul 16 21:28:17.271620 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: 3b214f5044c7445592e177835a5d18fc Jul 16 21:28:17.271620 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-ListServerFiltersTestJSON-921481021 Jul 16 21:28:17.271620 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-ListServerFiltersTestJSON-921481021-project Jul 16 21:28:17.271620 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:28:17.271620 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:28:17.271620 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: 029c76a0ba614b5880128c4309902c67 Jul 16 21:28:17.271620 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-ListServerFiltersTestJSON-921481021-project Jul 16 21:28:17.271620 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 21:28:17.271620 ncdv-master-1 devstack@n-api.service[138680]: exception: Instance cf38328b-40fb-4537-b524-aaeb0a0abfb4 could not be found. Jul 16 21:28:17.271620 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': 'cf38328b-40fb-4537-b524-aaeb0a0abfb4'} Jul 16 21:28:17.271620 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:28:17.271620 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:28:17.271620 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:28:17.271620 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:28:17.271620 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:28:17.271620 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:28:17.271620 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:28:17.271620 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:28:17.271620 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:28:17.271620 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:28:17.271620 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance cf38328b-40fb-4537-b524-aaeb0a0abfb4 could not be found. Jul 16 21:28:17.271620 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:28:17.271620 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:28:17.271620 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:28:17.271620 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:28:17.271620 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:28:17.271620 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:28:17.271620 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:28:17.271620 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:28:17.271620 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:28:17.271620 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:28:17.271620 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:28:17.271620 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:28:17.271620 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:28:17.271620 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:28:17.271620 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:28:17.271620 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:28:17.271620 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance cf38328b-40fb-4537-b524-aaeb0a0abfb4 could not be found. Jul 16 21:28:17.271620 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:28:17.271620 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:28:17.271620 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:28:17.275861 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:28:17.275861 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:28:17.275861 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:28:17.275861 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:28:17.275861 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:28:17.275861 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance cf38328b-40fb-4537-b524-aaeb0a0abfb4 could not be found. Jul 16 21:28:17.275861 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:28:17.283882 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-5e854388-055b-4920-b135-f215b8a93bb2 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] Returning 404 to user: Instance cf38328b-40fb-4537-b524-aaeb0a0abfb4 could not be found. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:28:17.285022 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-5e854388-055b-4920-b135-f215b8a93bb2 tempest-ListServerFiltersTestJSON-921481021 tempest-ListServerFiltersTestJSON-921481021-project] 10.222.0.16 "GET /compute/v2.1/servers/cf38328b-40fb-4537-b524-aaeb0a0abfb4" status: 404 len: 111 microversion: 2.1 time: 0.112260 Jul 16 21:28:17.285767 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2096/4197] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:28:17 2022] GET /compute/v2.1/servers/cf38328b-40fb-4537-b524-aaeb0a0abfb4 => generated 111 bytes in 114 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:28:17.378954 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-edc77d8a-4f99-47c6-9082-1872c417eb80 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f14ac04b7a44f17b946c5c8619554ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '349811967f4b4d1a9ab6772602a205a6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:17.404807 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-edc77d8a-4f99-47c6-9082-1872c417eb80 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f14ac04b7a44f17b946c5c8619554ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '349811967f4b4d1a9ab6772602a205a6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:17.408011 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-edc77d8a-4f99-47c6-9082-1872c417eb80 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f14ac04b7a44f17b946c5c8619554ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '349811967f4b4d1a9ab6772602a205a6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:17.410547 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-edc77d8a-4f99-47c6-9082-1872c417eb80 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] 10.222.0.16 "GET /compute/v2.1/servers/93519c14-aee1-4da1-8cce-775e0ad268dc" status: 200 len: 1407 microversion: 2.54 time: 0.204154 Jul 16 21:28:17.411162 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2102/4198] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:28:17 2022] GET /compute/v2.1/servers/93519c14-aee1-4da1-8cce-775e0ad268dc => generated 1407 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:28:18.437434 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-35c5a6b7-6a7f-4f2f-831e-e79a5016d4bd tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:28:18.437434 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-35c5a6b7-6a7f-4f2f-831e-e79a5016d4bd tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] [instance: 93519c14-aee1-4da1-8cce-775e0ad268dc] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:28:18.448357 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-35c5a6b7-6a7f-4f2f-831e-e79a5016d4bd tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:18.453988 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-35c5a6b7-6a7f-4f2f-831e-e79a5016d4bd tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:18.459110 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-35c5a6b7-6a7f-4f2f-831e-e79a5016d4bd tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:18.459810 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-35c5a6b7-6a7f-4f2f-831e-e79a5016d4bd tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:18.742981 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-35c5a6b7-6a7f-4f2f-831e-e79a5016d4bd tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f14ac04b7a44f17b946c5c8619554ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '349811967f4b4d1a9ab6772602a205a6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:18.758363 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-35c5a6b7-6a7f-4f2f-831e-e79a5016d4bd tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f14ac04b7a44f17b946c5c8619554ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '349811967f4b4d1a9ab6772602a205a6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:18.760584 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-35c5a6b7-6a7f-4f2f-831e-e79a5016d4bd tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f14ac04b7a44f17b946c5c8619554ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '349811967f4b4d1a9ab6772602a205a6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:18.762247 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-35c5a6b7-6a7f-4f2f-831e-e79a5016d4bd tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] 10.222.0.16 "GET /compute/v2.1/servers/93519c14-aee1-4da1-8cce-775e0ad268dc" status: 200 len: 1407 microversion: 2.54 time: 0.336708 Jul 16 21:28:18.763485 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2097/4199] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:28:18 2022] GET /compute/v2.1/servers/93519c14-aee1-4da1-8cce-775e0ad268dc => generated 1407 bytes in 338 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:28:19.784035 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-e4d9f692-fdab-40da-94e1-80b24d35f82a tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:28:19.790752 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-e4d9f692-fdab-40da-94e1-80b24d35f82a tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] [instance: 93519c14-aee1-4da1-8cce-775e0ad268dc] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:28:19.793350 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e4d9f692-fdab-40da-94e1-80b24d35f82a tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:19.793809 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e4d9f692-fdab-40da-94e1-80b24d35f82a tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:19.794598 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e4d9f692-fdab-40da-94e1-80b24d35f82a tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:19.795050 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e4d9f692-fdab-40da-94e1-80b24d35f82a tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:19.885974 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-5db7f178-68ed-4bdf-af71-c12f1c4f59f4 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeShelveTestJSON-server-185422108", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "051a9df4-f4db-45ac-8063-879ec2c58801"}], "security_groups": [{"name": "tempest-securitygroup--468373178"}], "key_name": "tempest-keypair-545593029", "user_data": "}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:28:20.004437 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-e4d9f692-fdab-40da-94e1-80b24d35f82a tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f14ac04b7a44f17b946c5c8619554ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '349811967f4b4d1a9ab6772602a205a6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:20.014295 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-e4d9f692-fdab-40da-94e1-80b24d35f82a tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f14ac04b7a44f17b946c5c8619554ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '349811967f4b4d1a9ab6772602a205a6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:20.017468 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-e4d9f692-fdab-40da-94e1-80b24d35f82a tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f14ac04b7a44f17b946c5c8619554ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '349811967f4b4d1a9ab6772602a205a6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:20.019193 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-e4d9f692-fdab-40da-94e1-80b24d35f82a tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] 10.222.0.16 "GET /compute/v2.1/servers/93519c14-aee1-4da1-8cce-775e0ad268dc" status: 200 len: 1407 microversion: 2.54 time: 0.240167 Jul 16 21:28:20.020260 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2103/4200] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:28:19 2022] GET /compute/v2.1/servers/93519c14-aee1-4da1-8cce-775e0ad268dc => generated 1407 bytes in 242 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:28:20.198986 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.network.neutron [None req-5db7f178-68ed-4bdf-af71-c12f1c4f59f4 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] validate_networks() for [('051a9df4-f4db-45ac-8063-879ec2c58801', None, None, None, None, None)] {{(pid=138680) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 16 21:28:21.054803 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-144284be-d9db-405a-9eec-590a94d72a73 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:28:21.055633 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-144284be-d9db-405a-9eec-590a94d72a73 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] [instance: 93519c14-aee1-4da1-8cce-775e0ad268dc] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:28:21.072246 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-144284be-d9db-405a-9eec-590a94d72a73 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:21.072949 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-144284be-d9db-405a-9eec-590a94d72a73 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:21.073796 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-144284be-d9db-405a-9eec-590a94d72a73 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:21.074338 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-144284be-d9db-405a-9eec-590a94d72a73 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:21.330171 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-144284be-d9db-405a-9eec-590a94d72a73 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f14ac04b7a44f17b946c5c8619554ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '349811967f4b4d1a9ab6772602a205a6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:21.347337 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-144284be-d9db-405a-9eec-590a94d72a73 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f14ac04b7a44f17b946c5c8619554ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '349811967f4b4d1a9ab6772602a205a6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:21.349275 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-144284be-d9db-405a-9eec-590a94d72a73 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f14ac04b7a44f17b946c5c8619554ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '349811967f4b4d1a9ab6772602a205a6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:21.350727 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-144284be-d9db-405a-9eec-590a94d72a73 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] 10.222.0.16 "GET /compute/v2.1/servers/93519c14-aee1-4da1-8cce-775e0ad268dc" status: 200 len: 1407 microversion: 2.54 time: 0.312399 Jul 16 21:28:21.351335 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2104/4201] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:28:21 2022] GET /compute/v2.1/servers/93519c14-aee1-4da1-8cce-775e0ad268dc => generated 1407 bytes in 314 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:28:21.520119 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-5db7f178-68ed-4bdf-af71-c12f1c4f59f4 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Flavor limits 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 16 21:28:21.520626 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-5db7f178-68ed-4bdf-af71-c12f1c4f59f4 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Image limits 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 16 21:28:21.521021 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-5db7f178-68ed-4bdf-af71-c12f1c4f59f4 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Flavor pref 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 16 21:28:21.521432 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-5db7f178-68ed-4bdf-af71-c12f1c4f59f4 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Image pref 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 16 21:28:21.521855 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-5db7f178-68ed-4bdf-af71-c12f1c4f59f4 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 16 21:28:21.529256 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell0:SELECT=11 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:28:21.534900 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5db7f178-68ed-4bdf-af71-c12f1c4f59f4 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:21.535557 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5db7f178-68ed-4bdf-af71-c12f1c4f59f4 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:21.536519 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5db7f178-68ed-4bdf-af71-c12f1c4f59f4 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:21.537215 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5db7f178-68ed-4bdf-af71-c12f1c4f59f4 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:21.557016 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-5db7f178-68ed-4bdf-af71-c12f1c4f59f4 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Getting quotas for project e451ee7bfd2d4ab58d30c33fef334b55. Resources: {'ram', 'instances', 'cores'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:28:21.562896 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-5db7f178-68ed-4bdf-af71-c12f1c4f59f4 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Getting quotas for user ca2c9e7e6d61414cbf2c7e17753bc0df and project e451ee7bfd2d4ab58d30c33fef334b55. Resources: {'ram', 'instances', 'cores'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:28:21.583966 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-5db7f178-68ed-4bdf-af71-c12f1c4f59f4 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Going to run 1 instances... {{(pid=138680) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 16 21:28:21.586317 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-5db7f178-68ed-4bdf-af71-c12f1c4f59f4 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] [instance: 08a33911-6e47-45fd-b499-155f199aacf8] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138680) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 16 21:28:21.798329 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-5db7f178-68ed-4bdf-af71-c12f1c4f59f4 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 401 microversion: 2.20 time: 1.917028 Jul 16 21:28:21.799911 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2098/4202] 10.222.0.16 () {62 vars in 1187 bytes} [Sat Jul 16 21:28:19 2022] POST /compute/v2.1/servers => generated 401 bytes in 1919 msecs (HTTP/1.1 202) 10 headers in 453 bytes (2 switches on core 0) Jul 16 21:28:21.809860 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-92666576-7126-4c9d-874d-bdb9d9136a41 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:28:21.810582 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-92666576-7126-4c9d-874d-bdb9d9136a41 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] [instance: 08a33911-6e47-45fd-b499-155f199aacf8] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:28:21.889169 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-92666576-7126-4c9d-874d-bdb9d9136a41 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:21.900580 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-92666576-7126-4c9d-874d-bdb9d9136a41 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:21.903353 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-92666576-7126-4c9d-874d-bdb9d9136a41 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:21.906374 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-92666576-7126-4c9d-874d-bdb9d9136a41 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] 10.222.0.16 "GET /compute/v2.1/servers/08a33911-6e47-45fd-b499-155f199aacf8" status: 200 len: 1280 microversion: 2.20 time: 0.098638 Jul 16 21:28:21.913591 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2105/4203] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:28:21 2022] GET /compute/v2.1/servers/08a33911-6e47-45fd-b499-155f199aacf8 => generated 1280 bytes in 108 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:28:22.371156 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-5bd33013-e905-4a44-89f6-8a9477ab7e4d tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:28:22.371787 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-5bd33013-e905-4a44-89f6-8a9477ab7e4d tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] [instance: 93519c14-aee1-4da1-8cce-775e0ad268dc] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:28:22.378916 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5bd33013-e905-4a44-89f6-8a9477ab7e4d tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:22.384227 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5bd33013-e905-4a44-89f6-8a9477ab7e4d tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:22.384227 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5bd33013-e905-4a44-89f6-8a9477ab7e4d tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:22.384227 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5bd33013-e905-4a44-89f6-8a9477ab7e4d tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:22.456165 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-5bd33013-e905-4a44-89f6-8a9477ab7e4d tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f14ac04b7a44f17b946c5c8619554ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '349811967f4b4d1a9ab6772602a205a6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:22.464322 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-5bd33013-e905-4a44-89f6-8a9477ab7e4d tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f14ac04b7a44f17b946c5c8619554ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '349811967f4b4d1a9ab6772602a205a6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:22.465924 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-5bd33013-e905-4a44-89f6-8a9477ab7e4d tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f14ac04b7a44f17b946c5c8619554ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '349811967f4b4d1a9ab6772602a205a6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:22.470068 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-5bd33013-e905-4a44-89f6-8a9477ab7e4d tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] 10.222.0.16 "GET /compute/v2.1/servers/93519c14-aee1-4da1-8cce-775e0ad268dc" status: 200 len: 1407 microversion: 2.54 time: 0.101280 Jul 16 21:28:22.470068 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2099/4204] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:28:22 2022] GET /compute/v2.1/servers/93519c14-aee1-4da1-8cce-775e0ad268dc => generated 1407 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:28:22.926252 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-fae7ca53-03e7-4c71-91ca-b69c41edfa97 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:28:22.927634 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-fae7ca53-03e7-4c71-91ca-b69c41edfa97 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] [instance: 08a33911-6e47-45fd-b499-155f199aacf8] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:28:23.026292 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-fae7ca53-03e7-4c71-91ca-b69c41edfa97 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:23.035239 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-fae7ca53-03e7-4c71-91ca-b69c41edfa97 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:23.037004 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-fae7ca53-03e7-4c71-91ca-b69c41edfa97 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:23.038175 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-fae7ca53-03e7-4c71-91ca-b69c41edfa97 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] 10.222.0.16 "GET /compute/v2.1/servers/08a33911-6e47-45fd-b499-155f199aacf8" status: 200 len: 1280 microversion: 2.20 time: 0.118006 Jul 16 21:28:23.039093 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2106/4205] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:28:22 2022] GET /compute/v2.1/servers/08a33911-6e47-45fd-b499-155f199aacf8 => generated 1280 bytes in 119 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:28:23.487038 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-493c42bc-00ae-4480-8056-e36b1c37448e tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:28:23.488316 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-493c42bc-00ae-4480-8056-e36b1c37448e tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] [instance: 93519c14-aee1-4da1-8cce-775e0ad268dc] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:28:23.496028 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-493c42bc-00ae-4480-8056-e36b1c37448e tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:23.496635 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-493c42bc-00ae-4480-8056-e36b1c37448e tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:23.497633 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-493c42bc-00ae-4480-8056-e36b1c37448e tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:23.498198 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-493c42bc-00ae-4480-8056-e36b1c37448e tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:23.503657 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:28:23.590704 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-493c42bc-00ae-4480-8056-e36b1c37448e tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f14ac04b7a44f17b946c5c8619554ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '349811967f4b4d1a9ab6772602a205a6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:23.600743 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-493c42bc-00ae-4480-8056-e36b1c37448e tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f14ac04b7a44f17b946c5c8619554ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '349811967f4b4d1a9ab6772602a205a6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:23.602694 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-493c42bc-00ae-4480-8056-e36b1c37448e tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f14ac04b7a44f17b946c5c8619554ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '349811967f4b4d1a9ab6772602a205a6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:23.604227 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-493c42bc-00ae-4480-8056-e36b1c37448e tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] 10.222.0.16 "GET /compute/v2.1/servers/93519c14-aee1-4da1-8cce-775e0ad268dc" status: 200 len: 1407 microversion: 2.54 time: 0.122461 Jul 16 21:28:23.605058 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2100/4206] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:28:23 2022] GET /compute/v2.1/servers/93519c14-aee1-4da1-8cce-775e0ad268dc => generated 1407 bytes in 124 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:28:24.057348 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-24fb82cf-5181-463e-aea6-789dd8912502 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:28:24.057974 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-24fb82cf-5181-463e-aea6-789dd8912502 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] [instance: 08a33911-6e47-45fd-b499-155f199aacf8] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:28:24.066159 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-24fb82cf-5181-463e-aea6-789dd8912502 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:24.066631 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-24fb82cf-5181-463e-aea6-789dd8912502 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:24.067538 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-24fb82cf-5181-463e-aea6-789dd8912502 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:24.068184 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-24fb82cf-5181-463e-aea6-789dd8912502 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:24.145216 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-24fb82cf-5181-463e-aea6-789dd8912502 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:24.153849 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-24fb82cf-5181-463e-aea6-789dd8912502 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:24.155678 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-24fb82cf-5181-463e-aea6-789dd8912502 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:24.157109 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-24fb82cf-5181-463e-aea6-789dd8912502 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] 10.222.0.16 "GET /compute/v2.1/servers/08a33911-6e47-45fd-b499-155f199aacf8" status: 200 len: 1332 microversion: 2.20 time: 0.103555 Jul 16 21:28:24.157748 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2107/4207] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:28:24 2022] GET /compute/v2.1/servers/08a33911-6e47-45fd-b499-155f199aacf8 => generated 1332 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:28:24.627704 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-deaf8426-fba0-4ccb-90c0-9ba805bc1034 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:28:24.628328 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-deaf8426-fba0-4ccb-90c0-9ba805bc1034 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] [instance: 93519c14-aee1-4da1-8cce-775e0ad268dc] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:28:24.636470 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-deaf8426-fba0-4ccb-90c0-9ba805bc1034 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:24.644752 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-deaf8426-fba0-4ccb-90c0-9ba805bc1034 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:24.649439 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-deaf8426-fba0-4ccb-90c0-9ba805bc1034 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:24.650792 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-deaf8426-fba0-4ccb-90c0-9ba805bc1034 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:24.773895 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-deaf8426-fba0-4ccb-90c0-9ba805bc1034 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f14ac04b7a44f17b946c5c8619554ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '349811967f4b4d1a9ab6772602a205a6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:24.809971 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-deaf8426-fba0-4ccb-90c0-9ba805bc1034 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f14ac04b7a44f17b946c5c8619554ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '349811967f4b4d1a9ab6772602a205a6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:24.825629 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-deaf8426-fba0-4ccb-90c0-9ba805bc1034 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f14ac04b7a44f17b946c5c8619554ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '349811967f4b4d1a9ab6772602a205a6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:24.827886 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-deaf8426-fba0-4ccb-90c0-9ba805bc1034 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] 10.222.0.16 "GET /compute/v2.1/servers/93519c14-aee1-4da1-8cce-775e0ad268dc" status: 200 len: 1407 microversion: 2.54 time: 0.203644 Jul 16 21:28:24.830121 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2101/4208] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:28:24 2022] GET /compute/v2.1/servers/93519c14-aee1-4da1-8cce-775e0ad268dc => generated 1407 bytes in 207 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:28:25.179117 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-87c9028c-454e-4afc-a8b9-5b6368ff44c4 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:28:25.179737 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-87c9028c-454e-4afc-a8b9-5b6368ff44c4 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] [instance: 08a33911-6e47-45fd-b499-155f199aacf8] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:28:25.186487 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-87c9028c-454e-4afc-a8b9-5b6368ff44c4 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:25.187110 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-87c9028c-454e-4afc-a8b9-5b6368ff44c4 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:25.187891 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-87c9028c-454e-4afc-a8b9-5b6368ff44c4 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:25.188328 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-87c9028c-454e-4afc-a8b9-5b6368ff44c4 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:25.328663 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-87c9028c-454e-4afc-a8b9-5b6368ff44c4 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:25.337693 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-87c9028c-454e-4afc-a8b9-5b6368ff44c4 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:25.339548 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-87c9028c-454e-4afc-a8b9-5b6368ff44c4 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:25.346829 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-87c9028c-454e-4afc-a8b9-5b6368ff44c4 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] 10.222.0.16 "GET /compute/v2.1/servers/08a33911-6e47-45fd-b499-155f199aacf8" status: 200 len: 1338 microversion: 2.20 time: 0.165918 Jul 16 21:28:25.346829 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2108/4209] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:28:25 2022] GET /compute/v2.1/servers/08a33911-6e47-45fd-b499-155f199aacf8 => generated 1338 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:28:25.849870 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-bbbe339f-d32e-4e0b-aceb-b20698d6e9f4 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:28:25.851193 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-bbbe339f-d32e-4e0b-aceb-b20698d6e9f4 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] [instance: 93519c14-aee1-4da1-8cce-775e0ad268dc] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:28:25.878969 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-bbbe339f-d32e-4e0b-aceb-b20698d6e9f4 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:25.879544 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-bbbe339f-d32e-4e0b-aceb-b20698d6e9f4 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:25.880356 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-bbbe339f-d32e-4e0b-aceb-b20698d6e9f4 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:25.880854 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-bbbe339f-d32e-4e0b-aceb-b20698d6e9f4 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:25.992096 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-072dd6eb-888d-4aea-bd4e-a216dd907ccb tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:28:25.992607 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-072dd6eb-888d-4aea-bd4e-a216dd907ccb tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] [instance: 867b2ab0-9301-414b-b045-85057d642971] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:28:26.026401 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-bbbe339f-d32e-4e0b-aceb-b20698d6e9f4 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f14ac04b7a44f17b946c5c8619554ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '349811967f4b4d1a9ab6772602a205a6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:26.036133 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-072dd6eb-888d-4aea-bd4e-a216dd907ccb tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.029s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:26.037233 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-072dd6eb-888d-4aea-bd4e-a216dd907ccb tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:26.041394 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-072dd6eb-888d-4aea-bd4e-a216dd907ccb tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:26.041394 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-072dd6eb-888d-4aea-bd4e-a216dd907ccb tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:26.041624 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-bbbe339f-d32e-4e0b-aceb-b20698d6e9f4 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f14ac04b7a44f17b946c5c8619554ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '349811967f4b4d1a9ab6772602a205a6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:26.042065 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-bbbe339f-d32e-4e0b-aceb-b20698d6e9f4 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f14ac04b7a44f17b946c5c8619554ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '349811967f4b4d1a9ab6772602a205a6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:26.043775 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-bbbe339f-d32e-4e0b-aceb-b20698d6e9f4 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] 10.222.0.16 "GET /compute/v2.1/servers/93519c14-aee1-4da1-8cce-775e0ad268dc" status: 200 len: 1407 microversion: 2.54 time: 0.199632 Jul 16 21:28:26.062627 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2102/4210] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:28:25 2022] GET /compute/v2.1/servers/93519c14-aee1-4da1-8cce-775e0ad268dc => generated 1407 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:28:26.147340 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-072dd6eb-888d-4aea-bd4e-a216dd907ccb tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] [instance: 867b2ab0-9301-414b-b045-85057d642971] Going to try to terminate instance {{(pid=138681) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 16 21:28:26.386265 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-a13d8711-49d0-4af4-88a7-7bce867903e8 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:28:26.387441 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-a13d8711-49d0-4af4-88a7-7bce867903e8 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] [instance: 08a33911-6e47-45fd-b499-155f199aacf8] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:28:26.403165 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a13d8711-49d0-4af4-88a7-7bce867903e8 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:26.407903 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a13d8711-49d0-4af4-88a7-7bce867903e8 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:26.409022 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a13d8711-49d0-4af4-88a7-7bce867903e8 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:26.409568 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a13d8711-49d0-4af4-88a7-7bce867903e8 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:26.635767 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-072dd6eb-888d-4aea-bd4e-a216dd907ccb tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] 10.222.0.16 "DELETE /compute/v2.1/servers/867b2ab0-9301-414b-b045-85057d642971" status: 204 len: 0 microversion: 2.1 time: 0.646101 Jul 16 21:28:26.635767 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2109/4211] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:28:25 2022] DELETE /compute/v2.1/servers/867b2ab0-9301-414b-b045-85057d642971 => generated 0 bytes in 651 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 16 21:28:26.645919 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-d90e5511-a030-4781-95c0-1d298ccf6642 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:28:26.646926 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-d90e5511-a030-4781-95c0-1d298ccf6642 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] [instance: 867b2ab0-9301-414b-b045-85057d642971] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:28:26.654060 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d90e5511-a030-4781-95c0-1d298ccf6642 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:26.658922 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d90e5511-a030-4781-95c0-1d298ccf6642 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:26.658922 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d90e5511-a030-4781-95c0-1d298ccf6642 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:26.658922 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d90e5511-a030-4781-95c0-1d298ccf6642 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:27.004495 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-a13d8711-49d0-4af4-88a7-7bce867903e8 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:27.022917 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-a13d8711-49d0-4af4-88a7-7bce867903e8 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:27.026407 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-a13d8711-49d0-4af4-88a7-7bce867903e8 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:27.035193 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-a13d8711-49d0-4af4-88a7-7bce867903e8 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] 10.222.0.16 "GET /compute/v2.1/servers/08a33911-6e47-45fd-b499-155f199aacf8" status: 200 len: 1405 microversion: 2.20 time: 0.677973 Jul 16 21:28:27.046830 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-d90e5511-a030-4781-95c0-1d298ccf6642 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcc9aa2a091c490599f9a9c41163e2eb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da56fc78f06142aba80ca4c6e3d60475', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:27.052687 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2103/4212] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:28:26 2022] GET /compute/v2.1/servers/08a33911-6e47-45fd-b499-155f199aacf8 => generated 1405 bytes in 681 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 16 21:28:27.055973 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-d90e5511-a030-4781-95c0-1d298ccf6642 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] 10.222.0.16 "GET /compute/v2.1/servers/867b2ab0-9301-414b-b045-85057d642971" status: 200 len: 1508 microversion: 2.1 time: 0.416991 Jul 16 21:28:27.058367 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2110/4213] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:28:26 2022] GET /compute/v2.1/servers/867b2ab0-9301-414b-b045-85057d642971 => generated 1508 bytes in 421 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:28:27.073327 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-97c92f1a-a8c3-4006-b6e8-18cb6177a015 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:28:27.073957 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-97c92f1a-a8c3-4006-b6e8-18cb6177a015 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] [instance: 93519c14-aee1-4da1-8cce-775e0ad268dc] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:28:27.081504 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-97c92f1a-a8c3-4006-b6e8-18cb6177a015 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:27.082022 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-97c92f1a-a8c3-4006-b6e8-18cb6177a015 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:27.083085 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-97c92f1a-a8c3-4006-b6e8-18cb6177a015 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:27.083727 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-97c92f1a-a8c3-4006-b6e8-18cb6177a015 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:27.238642 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-97c92f1a-a8c3-4006-b6e8-18cb6177a015 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f14ac04b7a44f17b946c5c8619554ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '349811967f4b4d1a9ab6772602a205a6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:27.249823 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-97c92f1a-a8c3-4006-b6e8-18cb6177a015 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f14ac04b7a44f17b946c5c8619554ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '349811967f4b4d1a9ab6772602a205a6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:27.255411 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-97c92f1a-a8c3-4006-b6e8-18cb6177a015 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f14ac04b7a44f17b946c5c8619554ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '349811967f4b4d1a9ab6772602a205a6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:27.259489 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-97c92f1a-a8c3-4006-b6e8-18cb6177a015 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] 10.222.0.16 "GET /compute/v2.1/servers/93519c14-aee1-4da1-8cce-775e0ad268dc" status: 200 len: 1407 microversion: 2.54 time: 0.193353 Jul 16 21:28:27.263248 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2104/4214] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:28:27 2022] GET /compute/v2.1/servers/93519c14-aee1-4da1-8cce-775e0ad268dc => generated 1407 bytes in 198 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:28:27.988254 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-6a06c2a2-af6f-46a7-89a0-5bb94b16c486 req-79ea7b3b-6de0-4ee9-8068-6959334bc4fc service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "08a33911-6e47-45fd-b499-155f199aacf8", "tag": "81e317e7-e9db-4ba9-b87d-9395b084ead7"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:28:28.033759 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-6a06c2a2-af6f-46a7-89a0-5bb94b16c486 req-79ea7b3b-6de0-4ee9-8068-6959334bc4fc service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:28.035269 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-6a06c2a2-af6f-46a7-89a0-5bb94b16c486 req-79ea7b3b-6de0-4ee9-8068-6959334bc4fc service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:28.054044 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-7e7a9c6d-2d84-433d-b8f7-861b7e9c2e37 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:28:28.055579 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-7e7a9c6d-2d84-433d-b8f7-861b7e9c2e37 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] [instance: 08a33911-6e47-45fd-b499-155f199aacf8] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:28:28.070683 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7e7a9c6d-2d84-433d-b8f7-861b7e9c2e37 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:28.076676 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7e7a9c6d-2d84-433d-b8f7-861b7e9c2e37 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:28.076676 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7e7a9c6d-2d84-433d-b8f7-861b7e9c2e37 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:28.076676 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7e7a9c6d-2d84-433d-b8f7-861b7e9c2e37 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:28.092412 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-6a06c2a2-af6f-46a7-89a0-5bb94b16c486 req-79ea7b3b-6de0-4ee9-8068-6959334bc4fc service nova] Creating event network-changed:81e317e7-e9db-4ba9-b87d-9395b084ead7 for instance 08a33911-6e47-45fd-b499-155f199aacf8 on nch1-master-1 Jul 16 21:28:28.101845 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-6a06c2a2-af6f-46a7-89a0-5bb94b16c486 req-79ea7b3b-6de0-4ee9-8068-6959334bc4fc service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.124592 Jul 16 21:28:28.103122 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2111/4215] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:28:27 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 127 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:28:28.110487 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-f43426af-1974-4908-ac62-1ebcfd4c502e tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:28:28.113727 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-f43426af-1974-4908-ac62-1ebcfd4c502e tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] [instance: 867b2ab0-9301-414b-b045-85057d642971] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:28:28.147537 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f43426af-1974-4908-ac62-1ebcfd4c502e tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:28.162228 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f43426af-1974-4908-ac62-1ebcfd4c502e tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:28.162228 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f43426af-1974-4908-ac62-1ebcfd4c502e tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:28.162228 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f43426af-1974-4908-ac62-1ebcfd4c502e tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:28.212793 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=96,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:28:28.636521 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-f43426af-1974-4908-ac62-1ebcfd4c502e tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcc9aa2a091c490599f9a9c41163e2eb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da56fc78f06142aba80ca4c6e3d60475', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:28.647641 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-f43426af-1974-4908-ac62-1ebcfd4c502e tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] 10.222.0.16 "GET /compute/v2.1/servers/867b2ab0-9301-414b-b045-85057d642971" status: 200 len: 1508 microversion: 2.1 time: 0.543013 Jul 16 21:28:28.648480 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2112/4216] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:28:28 2022] GET /compute/v2.1/servers/867b2ab0-9301-414b-b045-85057d642971 => generated 1508 bytes in 545 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:28:28.668513 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-d9e78309-8c84-4d3f-81cc-2bbadb9e31ea tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:28:28.669309 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-d9e78309-8c84-4d3f-81cc-2bbadb9e31ea tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] [instance: 93519c14-aee1-4da1-8cce-775e0ad268dc] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:28:28.677661 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d9e78309-8c84-4d3f-81cc-2bbadb9e31ea tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:28.678570 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d9e78309-8c84-4d3f-81cc-2bbadb9e31ea tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:28.679518 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d9e78309-8c84-4d3f-81cc-2bbadb9e31ea tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:28.680208 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d9e78309-8c84-4d3f-81cc-2bbadb9e31ea tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:29.241885 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-d9e78309-8c84-4d3f-81cc-2bbadb9e31ea tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f14ac04b7a44f17b946c5c8619554ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '349811967f4b4d1a9ab6772602a205a6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:29.243266 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-7e7a9c6d-2d84-433d-b8f7-861b7e9c2e37 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:29.248677 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-d9e78309-8c84-4d3f-81cc-2bbadb9e31ea tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f14ac04b7a44f17b946c5c8619554ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '349811967f4b4d1a9ab6772602a205a6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:29.251574 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-d9e78309-8c84-4d3f-81cc-2bbadb9e31ea tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f14ac04b7a44f17b946c5c8619554ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '349811967f4b4d1a9ab6772602a205a6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:29.254295 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-7e7a9c6d-2d84-433d-b8f7-861b7e9c2e37 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:29.256033 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-7e7a9c6d-2d84-433d-b8f7-861b7e9c2e37 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:29.258294 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-d9e78309-8c84-4d3f-81cc-2bbadb9e31ea tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] 10.222.0.16 "GET /compute/v2.1/servers/93519c14-aee1-4da1-8cce-775e0ad268dc" status: 200 len: 1407 microversion: 2.54 time: 0.593713 Jul 16 21:28:29.258811 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-7e7a9c6d-2d84-433d-b8f7-861b7e9c2e37 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] 10.222.0.16 "GET /compute/v2.1/servers/08a33911-6e47-45fd-b499-155f199aacf8" status: 200 len: 1405 microversion: 2.20 time: 1.208646 Jul 16 21:28:29.271546 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2105/4217] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:28:28 2022] GET /compute/v2.1/servers/08a33911-6e47-45fd-b499-155f199aacf8 => generated 1405 bytes in 1211 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 16 21:28:29.271639 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2113/4218] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:28:28 2022] GET /compute/v2.1/servers/93519c14-aee1-4da1-8cce-775e0ad268dc => generated 1407 bytes in 600 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:28:29.673023 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-1e0e60dc-af76-4ea7-9283-ac1e2f1aae33 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:28:29.673663 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-1e0e60dc-af76-4ea7-9283-ac1e2f1aae33 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] [instance: 867b2ab0-9301-414b-b045-85057d642971] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:28:29.691153 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1e0e60dc-af76-4ea7-9283-ac1e2f1aae33 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:29.692701 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1e0e60dc-af76-4ea7-9283-ac1e2f1aae33 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:29.693569 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1e0e60dc-af76-4ea7-9283-ac1e2f1aae33 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:29.694022 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1e0e60dc-af76-4ea7-9283-ac1e2f1aae33 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:30.281949 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-722b7008-4eeb-4c4b-a00b-6e5057236c1e tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:28:30.283144 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-722b7008-4eeb-4c4b-a00b-6e5057236c1e tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] [instance: 08a33911-6e47-45fd-b499-155f199aacf8] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:28:30.308926 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-722b7008-4eeb-4c4b-a00b-6e5057236c1e tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:30.309468 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-722b7008-4eeb-4c4b-a00b-6e5057236c1e tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:30.311809 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-722b7008-4eeb-4c4b-a00b-6e5057236c1e tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:30.312483 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-722b7008-4eeb-4c4b-a00b-6e5057236c1e tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:30.364459 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-1e0e60dc-af76-4ea7-9283-ac1e2f1aae33 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcc9aa2a091c490599f9a9c41163e2eb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da56fc78f06142aba80ca4c6e3d60475', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:30.377799 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-1e0e60dc-af76-4ea7-9283-ac1e2f1aae33 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] 10.222.0.16 "GET /compute/v2.1/servers/867b2ab0-9301-414b-b045-85057d642971" status: 200 len: 1508 microversion: 2.1 time: 0.709305 Jul 16 21:28:30.380365 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2106/4219] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:28:29 2022] GET /compute/v2.1/servers/867b2ab0-9301-414b-b045-85057d642971 => generated 1508 bytes in 712 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:28:30.394734 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-db07d39d-6026-4fa6-b7c2-04bcec74bb12 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:28:30.395438 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-db07d39d-6026-4fa6-b7c2-04bcec74bb12 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] [instance: 93519c14-aee1-4da1-8cce-775e0ad268dc] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:28:30.412124 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-db07d39d-6026-4fa6-b7c2-04bcec74bb12 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:30.412645 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-db07d39d-6026-4fa6-b7c2-04bcec74bb12 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:30.413597 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-db07d39d-6026-4fa6-b7c2-04bcec74bb12 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:30.414061 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-db07d39d-6026-4fa6-b7c2-04bcec74bb12 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:30.655944 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-db07d39d-6026-4fa6-b7c2-04bcec74bb12 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f14ac04b7a44f17b946c5c8619554ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '349811967f4b4d1a9ab6772602a205a6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:30.664647 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-db07d39d-6026-4fa6-b7c2-04bcec74bb12 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f14ac04b7a44f17b946c5c8619554ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '349811967f4b4d1a9ab6772602a205a6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:30.666280 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-db07d39d-6026-4fa6-b7c2-04bcec74bb12 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f14ac04b7a44f17b946c5c8619554ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '349811967f4b4d1a9ab6772602a205a6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:30.677360 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-db07d39d-6026-4fa6-b7c2-04bcec74bb12 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] 10.222.0.16 "GET /compute/v2.1/servers/93519c14-aee1-4da1-8cce-775e0ad268dc" status: 200 len: 1407 microversion: 2.54 time: 0.291133 Jul 16 21:28:30.678965 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2107/4220] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:28:30 2022] GET /compute/v2.1/servers/93519c14-aee1-4da1-8cce-775e0ad268dc => generated 1407 bytes in 298 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:28:30.775066 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-722b7008-4eeb-4c4b-a00b-6e5057236c1e tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:30.790167 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-722b7008-4eeb-4c4b-a00b-6e5057236c1e tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:30.791877 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-722b7008-4eeb-4c4b-a00b-6e5057236c1e tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:30.794226 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-722b7008-4eeb-4c4b-a00b-6e5057236c1e tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] 10.222.0.16 "GET /compute/v2.1/servers/08a33911-6e47-45fd-b499-155f199aacf8" status: 200 len: 1405 microversion: 2.20 time: 0.517558 Jul 16 21:28:30.799143 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2114/4221] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:28:30 2022] GET /compute/v2.1/servers/08a33911-6e47-45fd-b499-155f199aacf8 => generated 1405 bytes in 523 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 16 21:28:31.054558 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-815c5c5d-521a-4ada-9a76-669a34e6397a req-908a9dd5-72f1-4597-a2e2-b6fb0040de60 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "867b2ab0-9301-414b-b045-85057d642971", "name": "network-vif-deleted", "tag": "f3410245-1acc-427c-a9fb-aae6491051d9"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:28:31.066550 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-815c5c5d-521a-4ada-9a76-669a34e6397a req-908a9dd5-72f1-4597-a2e2-b6fb0040de60 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:31.067149 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-815c5c5d-521a-4ada-9a76-669a34e6397a req-908a9dd5-72f1-4597-a2e2-b6fb0040de60 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:31.080948 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-815c5c5d-521a-4ada-9a76-669a34e6397a req-908a9dd5-72f1-4597-a2e2-b6fb0040de60 service nova] Creating event network-vif-deleted:f3410245-1acc-427c-a9fb-aae6491051d9 for instance 867b2ab0-9301-414b-b045-85057d642971 on nch1-master-1 Jul 16 21:28:31.086668 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-815c5c5d-521a-4ada-9a76-669a34e6397a req-908a9dd5-72f1-4597-a2e2-b6fb0040de60 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.035773 Jul 16 21:28:31.087858 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2108/4222] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:28:31 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:28:31.400258 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-76091000-2eb3-474b-be91-0c0bc14369f4 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:28:31.401828 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-76091000-2eb3-474b-be91-0c0bc14369f4 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] [instance: 867b2ab0-9301-414b-b045-85057d642971] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:28:31.419409 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-76091000-2eb3-474b-be91-0c0bc14369f4 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:31.419966 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-76091000-2eb3-474b-be91-0c0bc14369f4 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:31.424206 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-76091000-2eb3-474b-be91-0c0bc14369f4 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:31.424909 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-76091000-2eb3-474b-be91-0c0bc14369f4 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:31.509500 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_api:SELECT=93,nova_api:UPDATE=3,nova_api:INSERT=4 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:28:31.516058 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-76091000-2eb3-474b-be91-0c0bc14369f4 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:31.519311 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-76091000-2eb3-474b-be91-0c0bc14369f4 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:31.520184 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.objects.instance [None req-76091000-2eb3-474b-be91-0c0bc14369f4 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lazy-loading 'fault' on Instance uuid 867b2ab0-9301-414b-b045-85057d642971 {{(pid=138681) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:28:31.697200 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-76091000-2eb3-474b-be91-0c0bc14369f4 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcc9aa2a091c490599f9a9c41163e2eb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da56fc78f06142aba80ca4c6e3d60475', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:31.703632 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-0251bacc-63ad-4549-977b-fb5e52f9a3c4 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:28:31.710309 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-0251bacc-63ad-4549-977b-fb5e52f9a3c4 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] [instance: 93519c14-aee1-4da1-8cce-775e0ad268dc] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:28:31.710309 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell0:SELECT=2 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:28:31.710625 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-76091000-2eb3-474b-be91-0c0bc14369f4 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] 10.222.0.16 "GET /compute/v2.1/servers/867b2ab0-9301-414b-b045-85057d642971" status: 200 len: 1312 microversion: 2.1 time: 0.313579 Jul 16 21:28:31.710625 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2115/4223] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:28:31 2022] GET /compute/v2.1/servers/867b2ab0-9301-414b-b045-85057d642971 => generated 1312 bytes in 317 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:28:31.718606 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0251bacc-63ad-4549-977b-fb5e52f9a3c4 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:31.720571 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0251bacc-63ad-4549-977b-fb5e52f9a3c4 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:31.721681 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0251bacc-63ad-4549-977b-fb5e52f9a3c4 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:31.722401 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0251bacc-63ad-4549-977b-fb5e52f9a3c4 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:31.777547 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell0:SELECT=5 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:28:31.817579 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-677e334e-37fd-4321-82a0-9c0dc5eb3069 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:28:31.818250 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-677e334e-37fd-4321-82a0-9c0dc5eb3069 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] [instance: 08a33911-6e47-45fd-b499-155f199aacf8] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:28:31.831105 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-677e334e-37fd-4321-82a0-9c0dc5eb3069 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:31.831742 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-677e334e-37fd-4321-82a0-9c0dc5eb3069 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:31.836046 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-677e334e-37fd-4321-82a0-9c0dc5eb3069 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:31.836694 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-677e334e-37fd-4321-82a0-9c0dc5eb3069 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:31.963906 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-0251bacc-63ad-4549-977b-fb5e52f9a3c4 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f14ac04b7a44f17b946c5c8619554ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '349811967f4b4d1a9ab6772602a205a6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:31.992350 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-0251bacc-63ad-4549-977b-fb5e52f9a3c4 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f14ac04b7a44f17b946c5c8619554ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '349811967f4b4d1a9ab6772602a205a6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:31.994777 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-0251bacc-63ad-4549-977b-fb5e52f9a3c4 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f14ac04b7a44f17b946c5c8619554ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '349811967f4b4d1a9ab6772602a205a6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:31.996420 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-0251bacc-63ad-4549-977b-fb5e52f9a3c4 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] 10.222.0.16 "GET /compute/v2.1/servers/93519c14-aee1-4da1-8cce-775e0ad268dc" status: 200 len: 1407 microversion: 2.54 time: 0.302187 Jul 16 21:28:31.997491 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2109/4224] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:28:31 2022] GET /compute/v2.1/servers/93519c14-aee1-4da1-8cce-775e0ad268dc => generated 1407 bytes in 304 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:28:32.092121 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-677e334e-37fd-4321-82a0-9c0dc5eb3069 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:32.105456 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-677e334e-37fd-4321-82a0-9c0dc5eb3069 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:32.112916 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-677e334e-37fd-4321-82a0-9c0dc5eb3069 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:32.115639 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-677e334e-37fd-4321-82a0-9c0dc5eb3069 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] 10.222.0.16 "GET /compute/v2.1/servers/08a33911-6e47-45fd-b499-155f199aacf8" status: 200 len: 1405 microversion: 2.20 time: 0.302655 Jul 16 21:28:32.123598 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2116/4225] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:28:31 2022] GET /compute/v2.1/servers/08a33911-6e47-45fd-b499-155f199aacf8 => generated 1405 bytes in 305 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 16 21:28:32.728197 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-143ab19c-8f85-401e-bf70-880819c2b7f7 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:28:32.729940 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-143ab19c-8f85-401e-bf70-880819c2b7f7 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] [instance: 867b2ab0-9301-414b-b045-85057d642971] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:28:32.750817 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-143ab19c-8f85-401e-bf70-880819c2b7f7 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:32.751688 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-143ab19c-8f85-401e-bf70-880819c2b7f7 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:32.763721 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-143ab19c-8f85-401e-bf70-880819c2b7f7 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:32.764445 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-143ab19c-8f85-401e-bf70-880819c2b7f7 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:32.912873 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-143ab19c-8f85-401e-bf70-880819c2b7f7 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:32.913624 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-143ab19c-8f85-401e-bf70-880819c2b7f7 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:32.914341 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.objects.instance [None req-143ab19c-8f85-401e-bf70-880819c2b7f7 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lazy-loading 'fault' on Instance uuid 867b2ab0-9301-414b-b045-85057d642971 {{(pid=138680) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:28:32.996527 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-143ab19c-8f85-401e-bf70-880819c2b7f7 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcc9aa2a091c490599f9a9c41163e2eb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da56fc78f06142aba80ca4c6e3d60475', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:33.009540 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-143ab19c-8f85-401e-bf70-880819c2b7f7 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] 10.222.0.16 "GET /compute/v2.1/servers/867b2ab0-9301-414b-b045-85057d642971" status: 200 len: 1312 microversion: 2.1 time: 0.287227 Jul 16 21:28:33.016246 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-530d52d0-d379-4947-95d2-4ce59a404d3d tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:28:33.017605 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-530d52d0-d379-4947-95d2-4ce59a404d3d tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] [instance: 93519c14-aee1-4da1-8cce-775e0ad268dc] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:28:33.036190 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2110/4226] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:28:32 2022] GET /compute/v2.1/servers/867b2ab0-9301-414b-b045-85057d642971 => generated 1312 bytes in 315 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:28:33.037550 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-530d52d0-d379-4947-95d2-4ce59a404d3d tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:33.039091 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-530d52d0-d379-4947-95d2-4ce59a404d3d tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:33.041078 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-530d52d0-d379-4947-95d2-4ce59a404d3d tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:33.042895 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-530d52d0-d379-4947-95d2-4ce59a404d3d tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:33.044781 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell0:SELECT=14 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:28:33.145626 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-a9630809-6ce4-4b6f-abe8-9f9a2e44902b tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:28:33.151153 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-a9630809-6ce4-4b6f-abe8-9f9a2e44902b tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] [instance: 08a33911-6e47-45fd-b499-155f199aacf8] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:28:33.164761 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a9630809-6ce4-4b6f-abe8-9f9a2e44902b tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:33.165402 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a9630809-6ce4-4b6f-abe8-9f9a2e44902b tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:33.174139 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a9630809-6ce4-4b6f-abe8-9f9a2e44902b tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:33.176216 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a9630809-6ce4-4b6f-abe8-9f9a2e44902b tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:33.268293 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-530d52d0-d379-4947-95d2-4ce59a404d3d tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f14ac04b7a44f17b946c5c8619554ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '349811967f4b4d1a9ab6772602a205a6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:33.292815 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-530d52d0-d379-4947-95d2-4ce59a404d3d tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f14ac04b7a44f17b946c5c8619554ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '349811967f4b4d1a9ab6772602a205a6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:33.296402 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-530d52d0-d379-4947-95d2-4ce59a404d3d tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f14ac04b7a44f17b946c5c8619554ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '349811967f4b4d1a9ab6772602a205a6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:33.297035 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-530d52d0-d379-4947-95d2-4ce59a404d3d tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] 10.222.0.16 "GET /compute/v2.1/servers/93519c14-aee1-4da1-8cce-775e0ad268dc" status: 200 len: 1407 microversion: 2.54 time: 0.285197 Jul 16 21:28:33.315152 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2117/4227] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:28:33 2022] GET /compute/v2.1/servers/93519c14-aee1-4da1-8cce-775e0ad268dc => generated 1407 bytes in 295 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:28:33.568767 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-a9630809-6ce4-4b6f-abe8-9f9a2e44902b tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:33.588944 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-a9630809-6ce4-4b6f-abe8-9f9a2e44902b tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:33.611164 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-a9630809-6ce4-4b6f-abe8-9f9a2e44902b tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:33.616685 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-a9630809-6ce4-4b6f-abe8-9f9a2e44902b tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] 10.222.0.16 "GET /compute/v2.1/servers/08a33911-6e47-45fd-b499-155f199aacf8" status: 200 len: 1405 microversion: 2.20 time: 0.474975 Jul 16 21:28:33.617895 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2111/4228] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:28:33 2022] GET /compute/v2.1/servers/08a33911-6e47-45fd-b499-155f199aacf8 => generated 1405 bytes in 477 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:28:34.060092 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-e1846026-8ab6-43ab-a6ef-fa1050b0d5e0 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:28:34.061369 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-e1846026-8ab6-43ab-a6ef-fa1050b0d5e0 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] [instance: 867b2ab0-9301-414b-b045-85057d642971] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:28:34.070506 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e1846026-8ab6-43ab-a6ef-fa1050b0d5e0 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:34.078264 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e1846026-8ab6-43ab-a6ef-fa1050b0d5e0 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:34.079393 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e1846026-8ab6-43ab-a6ef-fa1050b0d5e0 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:34.080017 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e1846026-8ab6-43ab-a6ef-fa1050b0d5e0 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:34.124081 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-e1846026-8ab6-43ab-a6ef-fa1050b0d5e0 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] HTTP exception thrown: Instance 867b2ab0-9301-414b-b045-85057d642971 could not be found. Jul 16 21:28:34.125881 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-e1846026-8ab6-43ab-a6ef-fa1050b0d5e0 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Request method failure captured: Jul 16 21:28:34.125881 ncdv-master-1 devstack@n-api.service[138681]: request: GET /compute/v2.1/servers/867b2ab0-9301-414b-b045-85057d642971 HTTP/1.1 Jul 16 21:28:34.125881 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:28:34.125881 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:28:34.125881 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:28:34.125881 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 21:28:34.125881 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:28:34.125881 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:28:34.125881 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:28:34.125881 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:28:34.125881 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0yxlBvXdj6P284bimRWVNOZAMmAVjIOasTJ1njzyaS12Nm5AKzVbxoKHi72FFVSaWe28lVKwWFtzjqUTy3NQWDNW-obFHfOCoFCCR4GNQ-_ruXI6NnMsKND6oz7nIdWgqUgB4Gac48-MGLdaneUoqvnHJrpsJjhWmKgGoKbdR8M7y0I Jul 16 21:28:34.125881 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:28:34.125881 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:28:34.125881 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:28:34.125881 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:28:34.125881 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:28:34.125881 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:28:34.125881 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: da56fc78f06142aba80ca4c6e3d60475 Jul 16 21:28:34.125881 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-439221844 Jul 16 21:28:34.125881 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:28:34.125881 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:28:34.125881 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/da56fc78f06142aba80ca4c6e3d60475"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/da56fc78f06142aba80ca4c6e3d60475"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_da56fc78f06142aba80ca4c6e3d60475"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/da56fc78f06142aba80ca4c6e3d60475"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:28:34.125881 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-ServerRescueNegativeTestJSON-439221844 Jul 16 21:28:34.125881 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: da56fc78f06142aba80ca4c6e3d60475 Jul 16 21:28:34.125881 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-439221844 Jul 16 21:28:34.125881 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-ServerRescueNegativeTestJSON-439221844-project Jul 16 21:28:34.125881 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:28:34.125881 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:28:34.125881 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: bcc9aa2a091c490599f9a9c41163e2eb Jul 16 21:28:34.125881 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-ServerRescueNegativeTestJSON-439221844-project Jul 16 21:28:34.125881 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:28:34.125881 ncdv-master-1 devstack@n-api.service[138681]: exception: Instance 867b2ab0-9301-414b-b045-85057d642971 could not be found. Jul 16 21:28:34.125881 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': '867b2ab0-9301-414b-b045-85057d642971'} Jul 16 21:28:34.125881 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:28:34.125881 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:28:34.125881 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:28:34.125881 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:28:34.125881 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:28:34.125881 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:28:34.125881 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:28:34.125881 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:28:34.125881 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:28:34.125881 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:28:34.125881 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 867b2ab0-9301-414b-b045-85057d642971 could not be found. Jul 16 21:28:34.125881 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:28:34.125881 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:28:34.125881 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:28:34.125881 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:28:34.125881 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:28:34.125881 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:28:34.125881 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:28:34.125881 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:28:34.125881 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:28:34.125881 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:28:34.125881 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:28:34.125881 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:28:34.125881 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:28:34.125881 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:28:34.125881 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:28:34.125881 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:28:34.125881 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 867b2ab0-9301-414b-b045-85057d642971 could not be found. Jul 16 21:28:34.125881 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:28:34.125881 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:28:34.125881 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:28:34.130261 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:28:34.130261 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:28:34.130261 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:28:34.130261 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:28:34.130261 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:28:34.130261 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 867b2ab0-9301-414b-b045-85057d642971 could not be found. Jul 16 21:28:34.130261 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:28:34.130261 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-e1846026-8ab6-43ab-a6ef-fa1050b0d5e0 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Returning 404 to user: Instance 867b2ab0-9301-414b-b045-85057d642971 could not be found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:28:34.130261 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-e1846026-8ab6-43ab-a6ef-fa1050b0d5e0 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] 10.222.0.16 "GET /compute/v2.1/servers/867b2ab0-9301-414b-b045-85057d642971" status: 404 len: 111 microversion: 2.1 time: 0.078276 Jul 16 21:28:34.132628 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2118/4229] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:28:34 2022] GET /compute/v2.1/servers/867b2ab0-9301-414b-b045-85057d642971 => generated 111 bytes in 84 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:28:34.137233 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-2d1a7c7f-e481-4c50-b0bc-b112703fb26d tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:28:34.137928 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-2d1a7c7f-e481-4c50-b0bc-b112703fb26d tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] [instance: 92fbfe5e-07b4-44ec-9729-1a9230f4c02c] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:28:34.161038 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2d1a7c7f-e481-4c50-b0bc-b112703fb26d tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:34.163856 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2d1a7c7f-e481-4c50-b0bc-b112703fb26d tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.010s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:34.164704 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2d1a7c7f-e481-4c50-b0bc-b112703fb26d tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:34.165151 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2d1a7c7f-e481-4c50-b0bc-b112703fb26d tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:34.229832 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-2d1a7c7f-e481-4c50-b0bc-b112703fb26d tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] [instance: 92fbfe5e-07b4-44ec-9729-1a9230f4c02c] Going to try to terminate instance {{(pid=138680) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 16 21:28:34.326689 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-b4903ddf-3f05-4319-81f9-05890626dc46 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:28:34.327991 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-b4903ddf-3f05-4319-81f9-05890626dc46 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] [instance: 93519c14-aee1-4da1-8cce-775e0ad268dc] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:28:34.337507 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b4903ddf-3f05-4319-81f9-05890626dc46 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:34.341753 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b4903ddf-3f05-4319-81f9-05890626dc46 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:34.344753 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b4903ddf-3f05-4319-81f9-05890626dc46 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:34.346463 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b4903ddf-3f05-4319-81f9-05890626dc46 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:34.506127 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-b4903ddf-3f05-4319-81f9-05890626dc46 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f14ac04b7a44f17b946c5c8619554ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '349811967f4b4d1a9ab6772602a205a6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:34.523225 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-b4903ddf-3f05-4319-81f9-05890626dc46 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f14ac04b7a44f17b946c5c8619554ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '349811967f4b4d1a9ab6772602a205a6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:34.526371 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-b4903ddf-3f05-4319-81f9-05890626dc46 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f14ac04b7a44f17b946c5c8619554ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '349811967f4b4d1a9ab6772602a205a6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:34.528937 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-b4903ddf-3f05-4319-81f9-05890626dc46 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] 10.222.0.16 "GET /compute/v2.1/servers/93519c14-aee1-4da1-8cce-775e0ad268dc" status: 200 len: 1407 microversion: 2.54 time: 0.206654 Jul 16 21:28:34.532145 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2119/4230] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:28:34 2022] GET /compute/v2.1/servers/93519c14-aee1-4da1-8cce-775e0ad268dc => generated 1407 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 16 21:28:34.712064 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-2818b792-4d04-4b09-b1b0-717e2844c074 tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerAddressesTestJSON-server-864406814", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452", "networks": [{"uuid": "fd421c2a-e253-4220-9f4d-936a1f612dbd"}]}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:28:34.753234 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-2d1a7c7f-e481-4c50-b0bc-b112703fb26d tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] 10.222.0.16 "DELETE /compute/v2.1/servers/92fbfe5e-07b4-44ec-9729-1a9230f4c02c" status: 204 len: 0 microversion: 2.1 time: 0.620285 Jul 16 21:28:34.753845 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2112/4231] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:28:34 2022] DELETE /compute/v2.1/servers/92fbfe5e-07b4-44ec-9729-1a9230f4c02c => generated 0 bytes in 622 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Jul 16 21:28:34.767314 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-9610e0cf-8b0c-4f14-a36e-e8c3da7ad015 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:28:34.768111 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-9610e0cf-8b0c-4f14-a36e-e8c3da7ad015 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] [instance: 08a33911-6e47-45fd-b499-155f199aacf8] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:28:34.787349 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9610e0cf-8b0c-4f14-a36e-e8c3da7ad015 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:34.787808 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9610e0cf-8b0c-4f14-a36e-e8c3da7ad015 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:34.790250 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9610e0cf-8b0c-4f14-a36e-e8c3da7ad015 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:34.790250 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9610e0cf-8b0c-4f14-a36e-e8c3da7ad015 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:34.824530 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.network.neutron [None req-2818b792-4d04-4b09-b1b0-717e2844c074 tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] validate_networks() for [('fd421c2a-e253-4220-9f4d-936a1f612dbd', None, None, None, None, None)] {{(pid=138681) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 16 21:28:35.504449 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-9610e0cf-8b0c-4f14-a36e-e8c3da7ad015 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:35.519042 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-9610e0cf-8b0c-4f14-a36e-e8c3da7ad015 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:35.524223 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-9610e0cf-8b0c-4f14-a36e-e8c3da7ad015 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:35.535880 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-9610e0cf-8b0c-4f14-a36e-e8c3da7ad015 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] 10.222.0.16 "GET /compute/v2.1/servers/08a33911-6e47-45fd-b499-155f199aacf8" status: 200 len: 1405 microversion: 2.20 time: 0.780398 Jul 16 21:28:35.551757 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2113/4232] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:28:34 2022] GET /compute/v2.1/servers/08a33911-6e47-45fd-b499-155f199aacf8 => generated 1405 bytes in 797 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:28:35.557520 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-f9cee0fc-a6d2-45df-ab7c-7b46a2a40fe7 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:28:35.558565 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-f9cee0fc-a6d2-45df-ab7c-7b46a2a40fe7 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] [instance: 92fbfe5e-07b4-44ec-9729-1a9230f4c02c] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:28:35.577524 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f9cee0fc-a6d2-45df-ab7c-7b46a2a40fe7 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:35.582145 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f9cee0fc-a6d2-45df-ab7c-7b46a2a40fe7 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:35.583903 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f9cee0fc-a6d2-45df-ab7c-7b46a2a40fe7 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:35.585720 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f9cee0fc-a6d2-45df-ab7c-7b46a2a40fe7 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:35.983728 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-f9cee0fc-a6d2-45df-ab7c-7b46a2a40fe7 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcc9aa2a091c490599f9a9c41163e2eb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da56fc78f06142aba80ca4c6e3d60475', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:35.997191 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-f9cee0fc-a6d2-45df-ab7c-7b46a2a40fe7 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] 10.222.0.16 "GET /compute/v2.1/servers/92fbfe5e-07b4-44ec-9729-1a9230f4c02c" status: 200 len: 1523 microversion: 2.1 time: 0.443894 Jul 16 21:28:36.000739 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2114/4233] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:28:35 2022] GET /compute/v2.1/servers/92fbfe5e-07b4-44ec-9729-1a9230f4c02c => generated 1523 bytes in 448 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:28:36.006751 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-dd11c853-7ff0-4c2c-9439-818c74f67971 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:28:36.022281 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-dd11c853-7ff0-4c2c-9439-818c74f67971 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] [instance: 93519c14-aee1-4da1-8cce-775e0ad268dc] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:28:36.031584 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-dd11c853-7ff0-4c2c-9439-818c74f67971 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:36.032861 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-dd11c853-7ff0-4c2c-9439-818c74f67971 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:36.034808 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-dd11c853-7ff0-4c2c-9439-818c74f67971 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:36.038661 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-dd11c853-7ff0-4c2c-9439-818c74f67971 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:36.083344 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:28:36.173873 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-dd11c853-7ff0-4c2c-9439-818c74f67971 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f14ac04b7a44f17b946c5c8619554ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '349811967f4b4d1a9ab6772602a205a6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:36.174355 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-2818b792-4d04-4b09-b1b0-717e2844c074 tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] Flavor limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 16 21:28:36.175374 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-2818b792-4d04-4b09-b1b0-717e2844c074 tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] Image limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 16 21:28:36.178641 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-2818b792-4d04-4b09-b1b0-717e2844c074 tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] Flavor pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 16 21:28:36.178641 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-2818b792-4d04-4b09-b1b0-717e2844c074 tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] Image pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 16 21:28:36.178641 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-2818b792-4d04-4b09-b1b0-717e2844c074 tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 16 21:28:36.187508 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-dd11c853-7ff0-4c2c-9439-818c74f67971 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f14ac04b7a44f17b946c5c8619554ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '349811967f4b4d1a9ab6772602a205a6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:36.189034 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2818b792-4d04-4b09-b1b0-717e2844c074 tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:36.189729 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2818b792-4d04-4b09-b1b0-717e2844c074 tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:36.190831 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-dd11c853-7ff0-4c2c-9439-818c74f67971 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f14ac04b7a44f17b946c5c8619554ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '349811967f4b4d1a9ab6772602a205a6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:36.190969 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2818b792-4d04-4b09-b1b0-717e2844c074 tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:36.191696 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2818b792-4d04-4b09-b1b0-717e2844c074 tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:36.193475 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-dd11c853-7ff0-4c2c-9439-818c74f67971 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] 10.222.0.16 "GET /compute/v2.1/servers/93519c14-aee1-4da1-8cce-775e0ad268dc" status: 200 len: 1407 microversion: 2.54 time: 0.190540 Jul 16 21:28:36.196417 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2115/4234] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:28:36 2022] GET /compute/v2.1/servers/93519c14-aee1-4da1-8cce-775e0ad268dc => generated 1407 bytes in 195 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:28:36.229837 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-2818b792-4d04-4b09-b1b0-717e2844c074 tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] Getting quotas for project 125617442dae45f58c179b625bcc799d. Resources: {'ram', 'instances', 'cores'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:28:36.239208 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-2818b792-4d04-4b09-b1b0-717e2844c074 tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] Getting quotas for user a27b3e9973c846c68f25f7981c995b73 and project 125617442dae45f58c179b625bcc799d. Resources: {'ram', 'instances', 'cores'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:28:36.264722 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-2818b792-4d04-4b09-b1b0-717e2844c074 tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] Going to run 1 instances... {{(pid=138681) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 16 21:28:36.267921 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-2818b792-4d04-4b09-b1b0-717e2844c074 tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] [instance: 138c70ad-acc1-4ce6-bb22-72d3d23175b1] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138681) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 16 21:28:36.432965 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-8e052402-c56c-43d6-8097-2cd299e62f68 req-39ba998f-5348-4715-9614-35cb123e135a service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "08a33911-6e47-45fd-b499-155f199aacf8", "name": "network-vif-plugged", "status": "completed", "tag": "81e317e7-e9db-4ba9-b87d-9395b084ead7"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:28:36.442488 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-8e052402-c56c-43d6-8097-2cd299e62f68 req-39ba998f-5348-4715-9614-35cb123e135a service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:36.442975 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-8e052402-c56c-43d6-8097-2cd299e62f68 req-39ba998f-5348-4715-9614-35cb123e135a service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:36.468389 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-8e052402-c56c-43d6-8097-2cd299e62f68 req-39ba998f-5348-4715-9614-35cb123e135a service nova] Creating event network-vif-plugged:81e317e7-e9db-4ba9-b87d-9395b084ead7 for instance 08a33911-6e47-45fd-b499-155f199aacf8 on nch1-master-1 Jul 16 21:28:36.480770 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-2818b792-4d04-4b09-b1b0-717e2844c074 tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 1.945876 Jul 16 21:28:36.483478 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-8e052402-c56c-43d6-8097-2cd299e62f68 req-39ba998f-5348-4715-9614-35cb123e135a service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.054256 Jul 16 21:28:36.484879 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2120/4236] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 21:28:34 2022] POST /compute/v2.1/servers => generated 380 bytes in 1952 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 16 21:28:36.485118 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2116/4236] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:28:36 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 16 21:28:36.499198 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-b965ae35-902c-42ad-8d3d-1c628c25477a tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:28:36.499198 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-b965ae35-902c-42ad-8d3d-1c628c25477a tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] [instance: 138c70ad-acc1-4ce6-bb22-72d3d23175b1] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:28:36.560392 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-1193ea4c-085e-4c66-9255-35fa278ed170 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:28:36.561019 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-1193ea4c-085e-4c66-9255-35fa278ed170 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] [instance: 08a33911-6e47-45fd-b499-155f199aacf8] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:28:36.576410 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1193ea4c-085e-4c66-9255-35fa278ed170 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:36.577580 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1193ea4c-085e-4c66-9255-35fa278ed170 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:36.578494 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1193ea4c-085e-4c66-9255-35fa278ed170 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:36.578932 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1193ea4c-085e-4c66-9255-35fa278ed170 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:36.743002 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-b965ae35-902c-42ad-8d3d-1c628c25477a tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a27b3e9973c846c68f25f7981c995b73', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '125617442dae45f58c179b625bcc799d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:36.764091 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-b965ae35-902c-42ad-8d3d-1c628c25477a tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] 10.222.0.16 "GET /compute/v2.1/servers/138c70ad-acc1-4ce6-bb22-72d3d23175b1" status: 200 len: 1216 microversion: 2.1 time: 0.273160 Jul 16 21:28:36.773030 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2117/4237] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:28:36 2022] GET /compute/v2.1/servers/138c70ad-acc1-4ce6-bb22-72d3d23175b1 => generated 1216 bytes in 278 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:28:36.910954 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-1193ea4c-085e-4c66-9255-35fa278ed170 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:36.920151 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-1193ea4c-085e-4c66-9255-35fa278ed170 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:36.922561 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-1193ea4c-085e-4c66-9255-35fa278ed170 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:36.923878 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-1193ea4c-085e-4c66-9255-35fa278ed170 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] 10.222.0.16 "GET /compute/v2.1/servers/08a33911-6e47-45fd-b499-155f199aacf8" status: 200 len: 1405 microversion: 2.20 time: 0.369174 Jul 16 21:28:36.924444 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2121/4238] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:28:36 2022] GET /compute/v2.1/servers/08a33911-6e47-45fd-b499-155f199aacf8 => generated 1405 bytes in 370 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:28:37.031809 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-23ed93c4-b93e-4a97-8491-0d1f93da7a2a tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:28:37.032352 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-23ed93c4-b93e-4a97-8491-0d1f93da7a2a tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] [instance: 92fbfe5e-07b4-44ec-9729-1a9230f4c02c] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:28:37.043213 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-23ed93c4-b93e-4a97-8491-0d1f93da7a2a tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:37.044315 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-23ed93c4-b93e-4a97-8491-0d1f93da7a2a tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:37.047724 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-23ed93c4-b93e-4a97-8491-0d1f93da7a2a tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:37.047724 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-23ed93c4-b93e-4a97-8491-0d1f93da7a2a tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:37.221656 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-03a68698-cc47-499a-be91-c0720c9b858a tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:28:37.222288 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-03a68698-cc47-499a-be91-c0720c9b858a tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] [instance: 93519c14-aee1-4da1-8cce-775e0ad268dc] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:28:37.234203 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-03a68698-cc47-499a-be91-c0720c9b858a tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:37.235219 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-03a68698-cc47-499a-be91-c0720c9b858a tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:37.236170 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-03a68698-cc47-499a-be91-c0720c9b858a tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:37.236746 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-03a68698-cc47-499a-be91-c0720c9b858a tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:37.253905 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-23ed93c4-b93e-4a97-8491-0d1f93da7a2a tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcc9aa2a091c490599f9a9c41163e2eb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da56fc78f06142aba80ca4c6e3d60475', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:37.262018 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-23ed93c4-b93e-4a97-8491-0d1f93da7a2a tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] 10.222.0.16 "GET /compute/v2.1/servers/92fbfe5e-07b4-44ec-9729-1a9230f4c02c" status: 200 len: 1523 microversion: 2.1 time: 0.248005 Jul 16 21:28:37.273048 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2118/4239] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:28:37 2022] GET /compute/v2.1/servers/92fbfe5e-07b4-44ec-9729-1a9230f4c02c => generated 1523 bytes in 260 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:28:37.323383 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-03a68698-cc47-499a-be91-c0720c9b858a tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f14ac04b7a44f17b946c5c8619554ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '349811967f4b4d1a9ab6772602a205a6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:37.333900 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-03a68698-cc47-499a-be91-c0720c9b858a tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f14ac04b7a44f17b946c5c8619554ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '349811967f4b4d1a9ab6772602a205a6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:37.335483 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-03a68698-cc47-499a-be91-c0720c9b858a tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f14ac04b7a44f17b946c5c8619554ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '349811967f4b4d1a9ab6772602a205a6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:37.336692 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-03a68698-cc47-499a-be91-c0720c9b858a tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] 10.222.0.16 "GET /compute/v2.1/servers/93519c14-aee1-4da1-8cce-775e0ad268dc" status: 200 len: 1407 microversion: 2.54 time: 0.120713 Jul 16 21:28:37.351165 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2122/4240] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:28:37 2022] GET /compute/v2.1/servers/93519c14-aee1-4da1-8cce-775e0ad268dc => generated 1407 bytes in 132 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:28:37.790311 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-fa06a261-ba21-4c2f-a5a1-b3cd3afc90b5 tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:28:37.791706 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-fa06a261-ba21-4c2f-a5a1-b3cd3afc90b5 tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] [instance: 138c70ad-acc1-4ce6-bb22-72d3d23175b1] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:28:37.874896 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-fa06a261-ba21-4c2f-a5a1-b3cd3afc90b5 tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a27b3e9973c846c68f25f7981c995b73', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '125617442dae45f58c179b625bcc799d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:37.880304 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-fa06a261-ba21-4c2f-a5a1-b3cd3afc90b5 tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] 10.222.0.16 "GET /compute/v2.1/servers/138c70ad-acc1-4ce6-bb22-72d3d23175b1" status: 200 len: 1216 microversion: 2.1 time: 0.101878 Jul 16 21:28:37.889599 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2119/4241] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:28:37 2022] GET /compute/v2.1/servers/138c70ad-acc1-4ce6-bb22-72d3d23175b1 => generated 1216 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:28:37.942472 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-2bea4f4f-f14b-4ddd-abee-41efaaf643be tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:28:37.943129 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-2bea4f4f-f14b-4ddd-abee-41efaaf643be tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] [instance: 08a33911-6e47-45fd-b499-155f199aacf8] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:28:37.949681 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2bea4f4f-f14b-4ddd-abee-41efaaf643be tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:37.950144 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2bea4f4f-f14b-4ddd-abee-41efaaf643be tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:37.950928 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2bea4f4f-f14b-4ddd-abee-41efaaf643be tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:37.951409 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2bea4f4f-f14b-4ddd-abee-41efaaf643be tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:38.282873 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-1d84735f-c4aa-431c-9f7b-127cffefcab8 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:28:38.283726 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-1d84735f-c4aa-431c-9f7b-127cffefcab8 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] [instance: 92fbfe5e-07b4-44ec-9729-1a9230f4c02c] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:28:38.300278 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-2bea4f4f-f14b-4ddd-abee-41efaaf643be tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:38.303065 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1d84735f-c4aa-431c-9f7b-127cffefcab8 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:38.304044 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1d84735f-c4aa-431c-9f7b-127cffefcab8 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:38.304895 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1d84735f-c4aa-431c-9f7b-127cffefcab8 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:38.305367 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1d84735f-c4aa-431c-9f7b-127cffefcab8 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:38.313163 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-2bea4f4f-f14b-4ddd-abee-41efaaf643be tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:38.321230 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-2bea4f4f-f14b-4ddd-abee-41efaaf643be tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:38.322798 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-2bea4f4f-f14b-4ddd-abee-41efaaf643be tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] 10.222.0.16 "GET /compute/v2.1/servers/08a33911-6e47-45fd-b499-155f199aacf8" status: 200 len: 1405 microversion: 2.20 time: 0.384359 Jul 16 21:28:38.336007 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2123/4242] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:28:37 2022] GET /compute/v2.1/servers/08a33911-6e47-45fd-b499-155f199aacf8 => generated 1405 bytes in 398 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:28:38.355768 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-511ae2b9-d29a-4f1a-9ead-7c51d972cf3f tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:28:38.356649 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-511ae2b9-d29a-4f1a-9ead-7c51d972cf3f tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] [instance: 93519c14-aee1-4da1-8cce-775e0ad268dc] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:28:38.370022 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-511ae2b9-d29a-4f1a-9ead-7c51d972cf3f tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:38.371713 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-511ae2b9-d29a-4f1a-9ead-7c51d972cf3f tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:38.374482 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-511ae2b9-d29a-4f1a-9ead-7c51d972cf3f tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:38.377026 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-511ae2b9-d29a-4f1a-9ead-7c51d972cf3f tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:38.565932 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-511ae2b9-d29a-4f1a-9ead-7c51d972cf3f tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f14ac04b7a44f17b946c5c8619554ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '349811967f4b4d1a9ab6772602a205a6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:38.579841 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-511ae2b9-d29a-4f1a-9ead-7c51d972cf3f tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f14ac04b7a44f17b946c5c8619554ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '349811967f4b4d1a9ab6772602a205a6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:38.584355 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-511ae2b9-d29a-4f1a-9ead-7c51d972cf3f tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f14ac04b7a44f17b946c5c8619554ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '349811967f4b4d1a9ab6772602a205a6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:38.585981 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-511ae2b9-d29a-4f1a-9ead-7c51d972cf3f tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] 10.222.0.16 "GET /compute/v2.1/servers/93519c14-aee1-4da1-8cce-775e0ad268dc" status: 200 len: 1407 microversion: 2.54 time: 0.234323 Jul 16 21:28:38.589687 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2124/4243] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:28:38 2022] GET /compute/v2.1/servers/93519c14-aee1-4da1-8cce-775e0ad268dc => generated 1407 bytes in 239 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:28:38.697557 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-1d84735f-c4aa-431c-9f7b-127cffefcab8 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcc9aa2a091c490599f9a9c41163e2eb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da56fc78f06142aba80ca4c6e3d60475', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:38.706527 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-1d84735f-c4aa-431c-9f7b-127cffefcab8 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] 10.222.0.16 "GET /compute/v2.1/servers/92fbfe5e-07b4-44ec-9729-1a9230f4c02c" status: 200 len: 1523 microversion: 2.1 time: 0.430150 Jul 16 21:28:38.707155 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2120/4244] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:28:38 2022] GET /compute/v2.1/servers/92fbfe5e-07b4-44ec-9729-1a9230f4c02c => generated 1523 bytes in 432 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:28:38.898359 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-27459784-1da1-4a3d-bc6f-1266689fee70 tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:28:38.898359 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-27459784-1da1-4a3d-bc6f-1266689fee70 tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] [instance: 138c70ad-acc1-4ce6-bb22-72d3d23175b1] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:28:38.913845 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-27459784-1da1-4a3d-bc6f-1266689fee70 tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:38.914304 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-27459784-1da1-4a3d-bc6f-1266689fee70 tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:38.915119 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-27459784-1da1-4a3d-bc6f-1266689fee70 tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:38.918480 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-27459784-1da1-4a3d-bc6f-1266689fee70 tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:39.021833 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-27459784-1da1-4a3d-bc6f-1266689fee70 tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a27b3e9973c846c68f25f7981c995b73', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '125617442dae45f58c179b625bcc799d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:39.031789 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-27459784-1da1-4a3d-bc6f-1266689fee70 tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] 10.222.0.16 "GET /compute/v2.1/servers/138c70ad-acc1-4ce6-bb22-72d3d23175b1" status: 200 len: 1212 microversion: 2.1 time: 0.138775 Jul 16 21:28:39.042265 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2125/4245] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:28:38 2022] GET /compute/v2.1/servers/138c70ad-acc1-4ce6-bb22-72d3d23175b1 => generated 1212 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:28:39.230072 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-e89e54b1-3bf6-4216-bf0c-09fd6c079030 req-3f031cd2-77b0-4e9c-9ce6-40282ad4c36f service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "92fbfe5e-07b4-44ec-9729-1a9230f4c02c", "name": "network-vif-deleted", "tag": "93df1436-7df4-45cf-b96b-37dafbf7a590"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:28:39.243719 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-e89e54b1-3bf6-4216-bf0c-09fd6c079030 req-3f031cd2-77b0-4e9c-9ce6-40282ad4c36f service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:39.244468 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-e89e54b1-3bf6-4216-bf0c-09fd6c079030 req-3f031cd2-77b0-4e9c-9ce6-40282ad4c36f service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:39.272944 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-e89e54b1-3bf6-4216-bf0c-09fd6c079030 req-3f031cd2-77b0-4e9c-9ce6-40282ad4c36f service nova] Creating event network-vif-deleted:93df1436-7df4-45cf-b96b-37dafbf7a590 for instance 92fbfe5e-07b4-44ec-9729-1a9230f4c02c on nch1-master-1 Jul 16 21:28:39.285516 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-e89e54b1-3bf6-4216-bf0c-09fd6c079030 req-3f031cd2-77b0-4e9c-9ce6-40282ad4c36f service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.059136 Jul 16 21:28:39.286597 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2121/4246] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:28:39 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 61 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:28:39.344955 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-8d0d5fd4-1d4f-4505-b4db-bf1ff0b197fc tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:28:39.346308 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-8d0d5fd4-1d4f-4505-b4db-bf1ff0b197fc tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] [instance: 08a33911-6e47-45fd-b499-155f199aacf8] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:28:39.357181 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8d0d5fd4-1d4f-4505-b4db-bf1ff0b197fc tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:39.358594 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8d0d5fd4-1d4f-4505-b4db-bf1ff0b197fc tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:39.360353 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8d0d5fd4-1d4f-4505-b4db-bf1ff0b197fc tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:39.361028 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8d0d5fd4-1d4f-4505-b4db-bf1ff0b197fc tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:39.610783 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-491a5b1c-68e5-4ee8-8819-795e51f69155 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:28:39.614870 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-491a5b1c-68e5-4ee8-8819-795e51f69155 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] [instance: 93519c14-aee1-4da1-8cce-775e0ad268dc] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:28:39.634539 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-491a5b1c-68e5-4ee8-8819-795e51f69155 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:39.639628 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-491a5b1c-68e5-4ee8-8819-795e51f69155 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:39.640748 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-491a5b1c-68e5-4ee8-8819-795e51f69155 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:39.641372 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-491a5b1c-68e5-4ee8-8819-795e51f69155 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:39.832166 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-8d0d5fd4-1d4f-4505-b4db-bf1ff0b197fc tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:39.871205 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-8d0d5fd4-1d4f-4505-b4db-bf1ff0b197fc tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:39.872760 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-8d0d5fd4-1d4f-4505-b4db-bf1ff0b197fc tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:39.876558 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-8d0d5fd4-1d4f-4505-b4db-bf1ff0b197fc tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] 10.222.0.16 "GET /compute/v2.1/servers/08a33911-6e47-45fd-b499-155f199aacf8" status: 200 len: 1598 microversion: 2.20 time: 0.534192 Jul 16 21:28:39.881098 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2126/4247] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:28:39 2022] GET /compute/v2.1/servers/08a33911-6e47-45fd-b499-155f199aacf8 => generated 1598 bytes in 542 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 16 21:28:39.886827 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-3f057af1-2c22-4963-94b5-41a81eea4cc8 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:28:39.896025 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-3f057af1-2c22-4963-94b5-41a81eea4cc8 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] [instance: 92fbfe5e-07b4-44ec-9729-1a9230f4c02c] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:28:39.910936 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3f057af1-2c22-4963-94b5-41a81eea4cc8 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:39.911722 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3f057af1-2c22-4963-94b5-41a81eea4cc8 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:39.912743 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3f057af1-2c22-4963-94b5-41a81eea4cc8 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:39.913406 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3f057af1-2c22-4963-94b5-41a81eea4cc8 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:39.925226 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-491a5b1c-68e5-4ee8-8819-795e51f69155 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f14ac04b7a44f17b946c5c8619554ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '349811967f4b4d1a9ab6772602a205a6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:39.935418 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-491a5b1c-68e5-4ee8-8819-795e51f69155 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f14ac04b7a44f17b946c5c8619554ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '349811967f4b4d1a9ab6772602a205a6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:39.937203 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-491a5b1c-68e5-4ee8-8819-795e51f69155 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f14ac04b7a44f17b946c5c8619554ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '349811967f4b4d1a9ab6772602a205a6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:39.940441 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-491a5b1c-68e5-4ee8-8819-795e51f69155 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] 10.222.0.16 "GET /compute/v2.1/servers/93519c14-aee1-4da1-8cce-775e0ad268dc" status: 200 len: 1407 microversion: 2.54 time: 0.334928 Jul 16 21:28:39.946641 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2122/4248] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:28:39 2022] GET /compute/v2.1/servers/93519c14-aee1-4da1-8cce-775e0ad268dc => generated 1407 bytes in 342 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:28:39.958532 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-c3ae327e-f2df-4615-b6ec-970fa7508a6f tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:28:39.963927 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-c3ae327e-f2df-4615-b6ec-970fa7508a6f tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] [instance: 08a33911-6e47-45fd-b499-155f199aacf8] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:28:39.985222 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c3ae327e-f2df-4615-b6ec-970fa7508a6f tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:39.985941 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c3ae327e-f2df-4615-b6ec-970fa7508a6f tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:39.986928 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c3ae327e-f2df-4615-b6ec-970fa7508a6f tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:39.987923 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c3ae327e-f2df-4615-b6ec-970fa7508a6f tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:39.999094 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3f057af1-2c22-4963-94b5-41a81eea4cc8 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:39.999710 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3f057af1-2c22-4963-94b5-41a81eea4cc8 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:40.000370 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.objects.instance [None req-3f057af1-2c22-4963-94b5-41a81eea4cc8 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lazy-loading 'fault' on Instance uuid 92fbfe5e-07b4-44ec-9729-1a9230f4c02c {{(pid=138681) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:28:40.121824 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-3f057af1-2c22-4963-94b5-41a81eea4cc8 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bcc9aa2a091c490599f9a9c41163e2eb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da56fc78f06142aba80ca4c6e3d60475', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:40.154148 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-3f057af1-2c22-4963-94b5-41a81eea4cc8 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] 10.222.0.16 "GET /compute/v2.1/servers/92fbfe5e-07b4-44ec-9729-1a9230f4c02c" status: 200 len: 1312 microversion: 2.1 time: 0.255219 Jul 16 21:28:40.154148 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2127/4249] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:28:39 2022] GET /compute/v2.1/servers/92fbfe5e-07b4-44ec-9729-1a9230f4c02c => generated 1312 bytes in 260 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:28:40.154148 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-5979d0ae-705e-4086-804b-4a535b31738e tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:28:40.154148 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-5979d0ae-705e-4086-804b-4a535b31738e tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] [instance: 138c70ad-acc1-4ce6-bb22-72d3d23175b1] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:28:40.174395 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5979d0ae-705e-4086-804b-4a535b31738e tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:40.179779 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5979d0ae-705e-4086-804b-4a535b31738e tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:40.181608 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5979d0ae-705e-4086-804b-4a535b31738e tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:40.183264 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5979d0ae-705e-4086-804b-4a535b31738e tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:40.278997 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-c3ae327e-f2df-4615-b6ec-970fa7508a6f tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] 10.222.0.16 "GET /compute/v2.1/servers/08a33911-6e47-45fd-b499-155f199aacf8/os-interface" status: 200 len: 285 microversion: 2.20 time: 0.330884 Jul 16 21:28:40.279617 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2123/4250] 10.222.0.16 () {60 vars in 1315 bytes} [Sat Jul 16 21:28:39 2022] GET /compute/v2.1/servers/08a33911-6e47-45fd-b499-155f199aacf8/os-interface => generated 285 bytes in 332 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 16 21:28:40.405129 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-5979d0ae-705e-4086-804b-4a535b31738e tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a27b3e9973c846c68f25f7981c995b73', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '125617442dae45f58c179b625bcc799d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:40.439536 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-5979d0ae-705e-4086-804b-4a535b31738e tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] 10.222.0.16 "GET /compute/v2.1/servers/138c70ad-acc1-4ce6-bb22-72d3d23175b1" status: 200 len: 1276 microversion: 2.1 time: 0.275477 Jul 16 21:28:40.439536 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2128/4251] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:28:40 2022] GET /compute/v2.1/servers/138c70ad-acc1-4ce6-bb22-72d3d23175b1 => generated 1276 bytes in 277 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:28:40.964476 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-a15290b4-d6fd-420b-b3b3-8d5d37542f30 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:28:40.965395 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-a15290b4-d6fd-420b-b3b3-8d5d37542f30 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] [instance: 93519c14-aee1-4da1-8cce-775e0ad268dc] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:28:40.976252 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a15290b4-d6fd-420b-b3b3-8d5d37542f30 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:40.980948 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a15290b4-d6fd-420b-b3b3-8d5d37542f30 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:40.982037 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a15290b4-d6fd-420b-b3b3-8d5d37542f30 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:40.982650 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a15290b4-d6fd-420b-b3b3-8d5d37542f30 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:41.160499 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-f19b8c8e-77ec-4bc3-871f-81d278cdcf76 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:28:41.162442 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-f19b8c8e-77ec-4bc3-871f-81d278cdcf76 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] [instance: 92fbfe5e-07b4-44ec-9729-1a9230f4c02c] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:28:41.174477 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f19b8c8e-77ec-4bc3-871f-81d278cdcf76 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:41.176906 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f19b8c8e-77ec-4bc3-871f-81d278cdcf76 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:41.176906 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f19b8c8e-77ec-4bc3-871f-81d278cdcf76 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:41.176906 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f19b8c8e-77ec-4bc3-871f-81d278cdcf76 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:41.247687 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-f19b8c8e-77ec-4bc3-871f-81d278cdcf76 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] HTTP exception thrown: Instance 92fbfe5e-07b4-44ec-9729-1a9230f4c02c could not be found. Jul 16 21:28:41.253834 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-f19b8c8e-77ec-4bc3-871f-81d278cdcf76 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Request method failure captured: Jul 16 21:28:41.253834 ncdv-master-1 devstack@n-api.service[138681]: request: GET /compute/v2.1/servers/92fbfe5e-07b4-44ec-9729-1a9230f4c02c HTTP/1.1 Jul 16 21:28:41.253834 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:28:41.253834 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:28:41.253834 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:28:41.253834 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 21:28:41.253834 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:28:41.253834 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:28:41.253834 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:28:41.253834 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:28:41.253834 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0yxlBvXdj6P284bimRWVNOZAMmAVjIOasTJ1njzyaS12Nm5AKzVbxoKHi72FFVSaWe28lVKwWFtzjqUTy3NQWDNW-obFHfOCoFCCR4GNQ-_ruXI6NnMsKND6oz7nIdWgqUgB4Gac48-MGLdaneUoqvnHJrpsJjhWmKgGoKbdR8M7y0I Jul 16 21:28:41.253834 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:28:41.253834 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:28:41.253834 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:28:41.253834 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:28:41.253834 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:28:41.253834 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:28:41.253834 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: da56fc78f06142aba80ca4c6e3d60475 Jul 16 21:28:41.253834 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-439221844 Jul 16 21:28:41.253834 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:28:41.253834 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:28:41.253834 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/da56fc78f06142aba80ca4c6e3d60475"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/da56fc78f06142aba80ca4c6e3d60475"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_da56fc78f06142aba80ca4c6e3d60475"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/da56fc78f06142aba80ca4c6e3d60475"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:28:41.253834 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-ServerRescueNegativeTestJSON-439221844 Jul 16 21:28:41.253834 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: da56fc78f06142aba80ca4c6e3d60475 Jul 16 21:28:41.253834 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-439221844 Jul 16 21:28:41.253834 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-ServerRescueNegativeTestJSON-439221844-project Jul 16 21:28:41.253834 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:28:41.253834 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:28:41.253834 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: bcc9aa2a091c490599f9a9c41163e2eb Jul 16 21:28:41.253834 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-ServerRescueNegativeTestJSON-439221844-project Jul 16 21:28:41.253834 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:28:41.253834 ncdv-master-1 devstack@n-api.service[138681]: exception: Instance 92fbfe5e-07b4-44ec-9729-1a9230f4c02c could not be found. Jul 16 21:28:41.253834 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': '92fbfe5e-07b4-44ec-9729-1a9230f4c02c'} Jul 16 21:28:41.253834 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:28:41.253834 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:28:41.253834 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:28:41.253834 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:28:41.253834 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:28:41.253834 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:28:41.253834 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:28:41.253834 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:28:41.253834 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:28:41.253834 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:28:41.253834 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 92fbfe5e-07b4-44ec-9729-1a9230f4c02c could not be found. Jul 16 21:28:41.253834 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:28:41.253834 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:28:41.253834 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:28:41.253834 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:28:41.253834 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:28:41.253834 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:28:41.253834 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:28:41.253834 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:28:41.253834 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:28:41.253834 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:28:41.253834 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:28:41.253834 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:28:41.253834 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:28:41.253834 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:28:41.253834 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:28:41.253834 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:28:41.253834 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 92fbfe5e-07b4-44ec-9729-1a9230f4c02c could not be found. Jul 16 21:28:41.253834 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:28:41.253834 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:28:41.253834 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:28:41.258163 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:28:41.258163 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:28:41.258163 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:28:41.258163 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:28:41.258163 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:28:41.258163 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 92fbfe5e-07b4-44ec-9729-1a9230f4c02c could not be found. Jul 16 21:28:41.258163 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:28:41.258163 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-f19b8c8e-77ec-4bc3-871f-81d278cdcf76 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] Returning 404 to user: Instance 92fbfe5e-07b4-44ec-9729-1a9230f4c02c could not be found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:28:41.258755 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-f19b8c8e-77ec-4bc3-871f-81d278cdcf76 tempest-ServerRescueNegativeTestJSON-439221844 tempest-ServerRescueNegativeTestJSON-439221844-project] 10.222.0.16 "GET /compute/v2.1/servers/92fbfe5e-07b4-44ec-9729-1a9230f4c02c" status: 404 len: 111 microversion: 2.1 time: 0.103285 Jul 16 21:28:41.262620 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2129/4252] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:28:41 2022] GET /compute/v2.1/servers/92fbfe5e-07b4-44ec-9729-1a9230f4c02c => generated 111 bytes in 110 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:28:41.283845 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-a15290b4-d6fd-420b-b3b3-8d5d37542f30 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f14ac04b7a44f17b946c5c8619554ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '349811967f4b4d1a9ab6772602a205a6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:41.299468 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-a15290b4-d6fd-420b-b3b3-8d5d37542f30 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f14ac04b7a44f17b946c5c8619554ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '349811967f4b4d1a9ab6772602a205a6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:41.302222 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-a15290b4-d6fd-420b-b3b3-8d5d37542f30 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f14ac04b7a44f17b946c5c8619554ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '349811967f4b4d1a9ab6772602a205a6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:41.303764 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-a15290b4-d6fd-420b-b3b3-8d5d37542f30 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] 10.222.0.16 "GET /compute/v2.1/servers/93519c14-aee1-4da1-8cce-775e0ad268dc" status: 200 len: 1407 microversion: 2.54 time: 0.344439 Jul 16 21:28:41.315599 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2124/4253] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:28:40 2022] GET /compute/v2.1/servers/93519c14-aee1-4da1-8cce-775e0ad268dc => generated 1407 bytes in 354 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:28:41.458549 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-e9a2e184-7ef4-4e5b-a30c-730068a7db96 tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:28:41.459244 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-e9a2e184-7ef4-4e5b-a30c-730068a7db96 tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] [instance: 138c70ad-acc1-4ce6-bb22-72d3d23175b1] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:28:41.473259 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e9a2e184-7ef4-4e5b-a30c-730068a7db96 tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:41.473720 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e9a2e184-7ef4-4e5b-a30c-730068a7db96 tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:41.477383 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e9a2e184-7ef4-4e5b-a30c-730068a7db96 tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:41.479081 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e9a2e184-7ef4-4e5b-a30c-730068a7db96 tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:41.492478 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=100 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:28:41.955978 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-e9a2e184-7ef4-4e5b-a30c-730068a7db96 tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a27b3e9973c846c68f25f7981c995b73', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '125617442dae45f58c179b625bcc799d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:41.975819 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-e9a2e184-7ef4-4e5b-a30c-730068a7db96 tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] 10.222.0.16 "GET /compute/v2.1/servers/138c70ad-acc1-4ce6-bb22-72d3d23175b1" status: 200 len: 1316 microversion: 2.1 time: 0.522293 Jul 16 21:28:41.977107 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2130/4254] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:28:41 2022] GET /compute/v2.1/servers/138c70ad-acc1-4ce6-bb22-72d3d23175b1 => generated 1316 bytes in 526 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:28:42.323259 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-392e70b6-d72e-4c72-bb39-fceb300e8229 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:28:42.323945 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-392e70b6-d72e-4c72-bb39-fceb300e8229 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] [instance: 93519c14-aee1-4da1-8cce-775e0ad268dc] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:28:42.337312 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-392e70b6-d72e-4c72-bb39-fceb300e8229 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:42.339805 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-392e70b6-d72e-4c72-bb39-fceb300e8229 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:42.340651 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-392e70b6-d72e-4c72-bb39-fceb300e8229 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:42.341115 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-392e70b6-d72e-4c72-bb39-fceb300e8229 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:42.441210 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-392e70b6-d72e-4c72-bb39-fceb300e8229 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f14ac04b7a44f17b946c5c8619554ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '349811967f4b4d1a9ab6772602a205a6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:42.451102 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-392e70b6-d72e-4c72-bb39-fceb300e8229 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f14ac04b7a44f17b946c5c8619554ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '349811967f4b4d1a9ab6772602a205a6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:42.454506 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-392e70b6-d72e-4c72-bb39-fceb300e8229 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f14ac04b7a44f17b946c5c8619554ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '349811967f4b4d1a9ab6772602a205a6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:42.457442 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-392e70b6-d72e-4c72-bb39-fceb300e8229 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] 10.222.0.16 "GET /compute/v2.1/servers/93519c14-aee1-4da1-8cce-775e0ad268dc" status: 200 len: 1407 microversion: 2.54 time: 0.138354 Jul 16 21:28:42.458854 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2125/4255] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:28:42 2022] GET /compute/v2.1/servers/93519c14-aee1-4da1-8cce-775e0ad268dc => generated 1407 bytes in 141 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:28:42.994099 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-d4934cf3-a4cd-43dd-8ade-4de50c6737a1 tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:28:42.994731 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-d4934cf3-a4cd-43dd-8ade-4de50c6737a1 tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] [instance: 138c70ad-acc1-4ce6-bb22-72d3d23175b1] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:28:43.015819 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d4934cf3-a4cd-43dd-8ade-4de50c6737a1 tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:43.016952 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d4934cf3-a4cd-43dd-8ade-4de50c6737a1 tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:43.018155 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d4934cf3-a4cd-43dd-8ade-4de50c6737a1 tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:43.019174 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d4934cf3-a4cd-43dd-8ade-4de50c6737a1 tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:43.225623 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-8a234cb4-73c1-4e33-8c52-93b50096727e req-23efef9a-fa39-4948-a806-ee1ab426fea1 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "08a33911-6e47-45fd-b499-155f199aacf8", "tag": "81e317e7-e9db-4ba9-b87d-9395b084ead7"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:28:43.238410 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-8a234cb4-73c1-4e33-8c52-93b50096727e req-23efef9a-fa39-4948-a806-ee1ab426fea1 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:43.240256 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-8a234cb4-73c1-4e33-8c52-93b50096727e req-23efef9a-fa39-4948-a806-ee1ab426fea1 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:43.259331 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-8a234cb4-73c1-4e33-8c52-93b50096727e req-23efef9a-fa39-4948-a806-ee1ab426fea1 service nova] Creating event network-changed:81e317e7-e9db-4ba9-b87d-9395b084ead7 for instance 08a33911-6e47-45fd-b499-155f199aacf8 on nch1-master-1 Jul 16 21:28:43.276829 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-8a234cb4-73c1-4e33-8c52-93b50096727e req-23efef9a-fa39-4948-a806-ee1ab426fea1 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.054746 Jul 16 21:28:43.277652 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2126/4256] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:28:43 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 16 21:28:43.488962 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-8e3b821f-e34a-4c07-8341-b836e0bcaeee tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:28:43.491269 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-8e3b821f-e34a-4c07-8341-b836e0bcaeee tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] [instance: 93519c14-aee1-4da1-8cce-775e0ad268dc] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:28:43.507442 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_api:SELECT=96,nova_api:INSERT=3,nova_api:UPDATE=1 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:28:43.512335 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8e3b821f-e34a-4c07-8341-b836e0bcaeee tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:43.516193 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8e3b821f-e34a-4c07-8341-b836e0bcaeee tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:43.517414 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8e3b821f-e34a-4c07-8341-b836e0bcaeee tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:43.520303 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8e3b821f-e34a-4c07-8341-b836e0bcaeee tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:43.775519 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-8e3b821f-e34a-4c07-8341-b836e0bcaeee tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f14ac04b7a44f17b946c5c8619554ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '349811967f4b4d1a9ab6772602a205a6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:43.788363 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-8e3b821f-e34a-4c07-8341-b836e0bcaeee tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f14ac04b7a44f17b946c5c8619554ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '349811967f4b4d1a9ab6772602a205a6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:43.791728 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-8e3b821f-e34a-4c07-8341-b836e0bcaeee tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f14ac04b7a44f17b946c5c8619554ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '349811967f4b4d1a9ab6772602a205a6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:43.793223 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-8e3b821f-e34a-4c07-8341-b836e0bcaeee tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] 10.222.0.16 "GET /compute/v2.1/servers/93519c14-aee1-4da1-8cce-775e0ad268dc" status: 200 len: 1407 microversion: 2.54 time: 0.316116 Jul 16 21:28:43.794328 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2127/4257] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:28:43 2022] GET /compute/v2.1/servers/93519c14-aee1-4da1-8cce-775e0ad268dc => generated 1407 bytes in 319 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 16 21:28:44.688787 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-d4934cf3-a4cd-43dd-8ade-4de50c6737a1 tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a27b3e9973c846c68f25f7981c995b73', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '125617442dae45f58c179b625bcc799d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:44.749254 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-d4934cf3-a4cd-43dd-8ade-4de50c6737a1 tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] 10.222.0.16 "GET /compute/v2.1/servers/138c70ad-acc1-4ce6-bb22-72d3d23175b1" status: 200 len: 1316 microversion: 2.1 time: 1.759145 Jul 16 21:28:44.749918 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2131/4258] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:28:42 2022] GET /compute/v2.1/servers/138c70ad-acc1-4ce6-bb22-72d3d23175b1 => generated 1316 bytes in 1761 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:28:44.838633 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-bb10b010-0024-4f52-9d97-a6cb4fecca08 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:28:44.841710 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-bb10b010-0024-4f52-9d97-a6cb4fecca08 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] [instance: 93519c14-aee1-4da1-8cce-775e0ad268dc] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:28:44.871444 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-bb10b010-0024-4f52-9d97-a6cb4fecca08 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:44.872300 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-bb10b010-0024-4f52-9d97-a6cb4fecca08 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:44.876181 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-bb10b010-0024-4f52-9d97-a6cb4fecca08 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:44.877418 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-bb10b010-0024-4f52-9d97-a6cb4fecca08 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:45.103633 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-bb10b010-0024-4f52-9d97-a6cb4fecca08 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f14ac04b7a44f17b946c5c8619554ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '349811967f4b4d1a9ab6772602a205a6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:45.118137 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-bb10b010-0024-4f52-9d97-a6cb4fecca08 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f14ac04b7a44f17b946c5c8619554ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '349811967f4b4d1a9ab6772602a205a6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:45.128186 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-bb10b010-0024-4f52-9d97-a6cb4fecca08 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f14ac04b7a44f17b946c5c8619554ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '349811967f4b4d1a9ab6772602a205a6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:45.129827 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-bb10b010-0024-4f52-9d97-a6cb4fecca08 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] 10.222.0.16 "GET /compute/v2.1/servers/93519c14-aee1-4da1-8cce-775e0ad268dc" status: 200 len: 1413 microversion: 2.54 time: 0.320273 Jul 16 21:28:45.130983 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2128/4259] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:28:44 2022] GET /compute/v2.1/servers/93519c14-aee1-4da1-8cce-775e0ad268dc => generated 1413 bytes in 323 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:28:45.566722 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-414aa644-79ba-48b0-bb19-9c5fc529e653 req-c5863621-e410-4b52-8c11-80a5e96c5dd3 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "138c70ad-acc1-4ce6-bb22-72d3d23175b1", "tag": "0bea62aa-549f-4465-97f0-73df9f0b2836"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:28:45.592291 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-414aa644-79ba-48b0-bb19-9c5fc529e653 req-c5863621-e410-4b52-8c11-80a5e96c5dd3 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:45.595868 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-414aa644-79ba-48b0-bb19-9c5fc529e653 req-c5863621-e410-4b52-8c11-80a5e96c5dd3 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:45.620709 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-414aa644-79ba-48b0-bb19-9c5fc529e653 req-c5863621-e410-4b52-8c11-80a5e96c5dd3 service nova] Creating event network-changed:0bea62aa-549f-4465-97f0-73df9f0b2836 for instance 138c70ad-acc1-4ce6-bb22-72d3d23175b1 on nch2-master-1 Jul 16 21:28:45.637057 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-414aa644-79ba-48b0-bb19-9c5fc529e653 req-c5863621-e410-4b52-8c11-80a5e96c5dd3 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.074903 Jul 16 21:28:45.638223 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2132/4260] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:28:45 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 77 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 16 21:28:45.770497 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-a44bf5fc-e05d-4e48-ad1b-8b56f4f66660 tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:28:45.771971 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-a44bf5fc-e05d-4e48-ad1b-8b56f4f66660 tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] [instance: 138c70ad-acc1-4ce6-bb22-72d3d23175b1] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:28:45.803528 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a44bf5fc-e05d-4e48-ad1b-8b56f4f66660 tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:45.805294 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a44bf5fc-e05d-4e48-ad1b-8b56f4f66660 tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:45.806303 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a44bf5fc-e05d-4e48-ad1b-8b56f4f66660 tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:45.806915 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a44bf5fc-e05d-4e48-ad1b-8b56f4f66660 tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:46.152480 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-0b90e4ef-3f8e-421e-b34d-03d3dab8dc8c tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:28:46.153191 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-0b90e4ef-3f8e-421e-b34d-03d3dab8dc8c tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] [instance: 93519c14-aee1-4da1-8cce-775e0ad268dc] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:28:46.177294 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0b90e4ef-3f8e-421e-b34d-03d3dab8dc8c tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:46.178002 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0b90e4ef-3f8e-421e-b34d-03d3dab8dc8c tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:46.179059 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0b90e4ef-3f8e-421e-b34d-03d3dab8dc8c tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:46.179695 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0b90e4ef-3f8e-421e-b34d-03d3dab8dc8c tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:46.243574 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell0:SELECT=2 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:28:46.262693 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell0:SELECT=6 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:28:46.372583 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-0b90e4ef-3f8e-421e-b34d-03d3dab8dc8c tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f14ac04b7a44f17b946c5c8619554ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '349811967f4b4d1a9ab6772602a205a6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:46.399088 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-0b90e4ef-3f8e-421e-b34d-03d3dab8dc8c tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f14ac04b7a44f17b946c5c8619554ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '349811967f4b4d1a9ab6772602a205a6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:46.406442 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-0b90e4ef-3f8e-421e-b34d-03d3dab8dc8c tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f14ac04b7a44f17b946c5c8619554ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '349811967f4b4d1a9ab6772602a205a6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:46.410549 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-0b90e4ef-3f8e-421e-b34d-03d3dab8dc8c tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] 10.222.0.16 "GET /compute/v2.1/servers/93519c14-aee1-4da1-8cce-775e0ad268dc" status: 200 len: 1413 microversion: 2.54 time: 0.265066 Jul 16 21:28:46.412048 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2133/4261] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:28:46 2022] GET /compute/v2.1/servers/93519c14-aee1-4da1-8cce-775e0ad268dc => generated 1413 bytes in 267 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:28:46.435573 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-a44bf5fc-e05d-4e48-ad1b-8b56f4f66660 tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a27b3e9973c846c68f25f7981c995b73', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '125617442dae45f58c179b625bcc799d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:46.453341 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-a44bf5fc-e05d-4e48-ad1b-8b56f4f66660 tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] 10.222.0.16 "GET /compute/v2.1/servers/138c70ad-acc1-4ce6-bb22-72d3d23175b1" status: 200 len: 1316 microversion: 2.1 time: 0.686894 Jul 16 21:28:46.463197 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2129/4262] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:28:45 2022] GET /compute/v2.1/servers/138c70ad-acc1-4ce6-bb22-72d3d23175b1 => generated 1316 bytes in 695 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:28:47.433684 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-fccc798e-a430-4d04-b479-f45c7c182f9f tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:28:47.434961 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-fccc798e-a430-4d04-b479-f45c7c182f9f tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] [instance: 93519c14-aee1-4da1-8cce-775e0ad268dc] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:28:47.446531 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fccc798e-a430-4d04-b479-f45c7c182f9f tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:47.447045 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fccc798e-a430-4d04-b479-f45c7c182f9f tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:47.447842 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fccc798e-a430-4d04-b479-f45c7c182f9f tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:47.448285 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fccc798e-a430-4d04-b479-f45c7c182f9f tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:47.470478 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-ab81868f-9d64-461f-ae3b-a3bbeac12705 tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:28:47.471314 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-ab81868f-9d64-461f-ae3b-a3bbeac12705 tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] [instance: 138c70ad-acc1-4ce6-bb22-72d3d23175b1] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:28:47.484650 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ab81868f-9d64-461f-ae3b-a3bbeac12705 tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:47.485001 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ab81868f-9d64-461f-ae3b-a3bbeac12705 tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:47.485786 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ab81868f-9d64-461f-ae3b-a3bbeac12705 tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:47.486219 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ab81868f-9d64-461f-ae3b-a3bbeac12705 tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:47.545323 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-fccc798e-a430-4d04-b479-f45c7c182f9f tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f14ac04b7a44f17b946c5c8619554ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '349811967f4b4d1a9ab6772602a205a6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:47.559149 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-fccc798e-a430-4d04-b479-f45c7c182f9f tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f14ac04b7a44f17b946c5c8619554ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '349811967f4b4d1a9ab6772602a205a6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:47.562110 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-fccc798e-a430-4d04-b479-f45c7c182f9f tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f14ac04b7a44f17b946c5c8619554ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '349811967f4b4d1a9ab6772602a205a6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:47.565691 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-fccc798e-a430-4d04-b479-f45c7c182f9f tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] 10.222.0.16 "GET /compute/v2.1/servers/93519c14-aee1-4da1-8cce-775e0ad268dc" status: 200 len: 1398 microversion: 2.54 time: 0.139315 Jul 16 21:28:47.566874 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2134/4263] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:28:47 2022] GET /compute/v2.1/servers/93519c14-aee1-4da1-8cce-775e0ad268dc => generated 1398 bytes in 141 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:28:47.602738 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-c89f6798-acfe-454a-af05-58269bc64143 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Calling method '.version_select at 0x7fdd9d2331f0>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:28:47.698801 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-ab81868f-9d64-461f-ae3b-a3bbeac12705 tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a27b3e9973c846c68f25f7981c995b73', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '125617442dae45f58c179b625bcc799d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:47.707808 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-ab81868f-9d64-461f-ae3b-a3bbeac12705 tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] 10.222.0.16 "GET /compute/v2.1/servers/138c70ad-acc1-4ce6-bb22-72d3d23175b1" status: 200 len: 1316 microversion: 2.1 time: 0.241899 Jul 16 21:28:47.708402 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2130/4264] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:28:47 2022] GET /compute/v2.1/servers/138c70ad-acc1-4ce6-bb22-72d3d23175b1 => generated 1316 bytes in 243 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:28:47.733901 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-c89f6798-acfe-454a-af05-58269bc64143 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] 10.222.0.16 "DELETE /compute/v2.1/os-keypairs/tempest-ServerShowV254Test-keypair-635440880" status: 204 len: 0 microversion: 2.54 time: 0.142167 Jul 16 21:28:47.736260 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2135/4265] 10.222.0.16 () {60 vars in 1315 bytes} [Sat Jul 16 21:28:47 2022] DELETE /compute/v2.1/os-keypairs/tempest-ServerShowV254Test-keypair-635440880 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Jul 16 21:28:47.751585 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-f7bdf4a4-cd2a-4de6-8ef3-ae5f203e415e tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:28:47.754506 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-f7bdf4a4-cd2a-4de6-8ef3-ae5f203e415e tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] [instance: 93519c14-aee1-4da1-8cce-775e0ad268dc] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:28:47.771076 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f7bdf4a4-cd2a-4de6-8ef3-ae5f203e415e tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:47.771765 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f7bdf4a4-cd2a-4de6-8ef3-ae5f203e415e tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:47.773150 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f7bdf4a4-cd2a-4de6-8ef3-ae5f203e415e tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:47.773838 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f7bdf4a4-cd2a-4de6-8ef3-ae5f203e415e tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:47.815344 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-f7bdf4a4-cd2a-4de6-8ef3-ae5f203e415e tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] [instance: 93519c14-aee1-4da1-8cce-775e0ad268dc] Going to try to terminate instance {{(pid=138680) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 16 21:28:47.881472 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell0:SELECT=4 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:28:48.180935 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-f7bdf4a4-cd2a-4de6-8ef3-ae5f203e415e tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] 10.222.0.16 "DELETE /compute/v2.1/servers/93519c14-aee1-4da1-8cce-775e0ad268dc" status: 204 len: 0 microversion: 2.54 time: 0.435233 Jul 16 21:28:48.181571 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2131/4266] 10.222.0.16 () {60 vars in 1279 bytes} [Sat Jul 16 21:28:47 2022] DELETE /compute/v2.1/servers/93519c14-aee1-4da1-8cce-775e0ad268dc => generated 0 bytes in 437 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Jul 16 21:28:48.192233 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-44297a1c-079b-41fa-a19e-10e87ee69401 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:28:48.192648 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-44297a1c-079b-41fa-a19e-10e87ee69401 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] [instance: 93519c14-aee1-4da1-8cce-775e0ad268dc] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:28:48.206743 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-44297a1c-079b-41fa-a19e-10e87ee69401 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:48.210044 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-44297a1c-079b-41fa-a19e-10e87ee69401 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:48.211328 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-44297a1c-079b-41fa-a19e-10e87ee69401 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:48.214598 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-44297a1c-079b-41fa-a19e-10e87ee69401 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:48.373028 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-44297a1c-079b-41fa-a19e-10e87ee69401 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f14ac04b7a44f17b946c5c8619554ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '349811967f4b4d1a9ab6772602a205a6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:48.382161 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-44297a1c-079b-41fa-a19e-10e87ee69401 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f14ac04b7a44f17b946c5c8619554ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '349811967f4b4d1a9ab6772602a205a6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:48.389791 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-44297a1c-079b-41fa-a19e-10e87ee69401 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f14ac04b7a44f17b946c5c8619554ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '349811967f4b4d1a9ab6772602a205a6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:48.395890 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-44297a1c-079b-41fa-a19e-10e87ee69401 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] 10.222.0.16 "GET /compute/v2.1/servers/93519c14-aee1-4da1-8cce-775e0ad268dc" status: 200 len: 1404 microversion: 2.54 time: 0.206481 Jul 16 21:28:48.398873 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2136/4267] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:28:48 2022] GET /compute/v2.1/servers/93519c14-aee1-4da1-8cce-775e0ad268dc => generated 1404 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:28:48.728827 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-049045f0-a5b9-4f82-936c-108def5d4027 tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:28:48.728827 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-049045f0-a5b9-4f82-936c-108def5d4027 tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] [instance: 138c70ad-acc1-4ce6-bb22-72d3d23175b1] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:28:48.736693 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-049045f0-a5b9-4f82-936c-108def5d4027 tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:48.737458 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-049045f0-a5b9-4f82-936c-108def5d4027 tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:48.738430 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-049045f0-a5b9-4f82-936c-108def5d4027 tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:48.739056 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-049045f0-a5b9-4f82-936c-108def5d4027 tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:49.155732 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-049045f0-a5b9-4f82-936c-108def5d4027 tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a27b3e9973c846c68f25f7981c995b73', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '125617442dae45f58c179b625bcc799d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:49.160455 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-049045f0-a5b9-4f82-936c-108def5d4027 tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] 10.222.0.16 "GET /compute/v2.1/servers/138c70ad-acc1-4ce6-bb22-72d3d23175b1" status: 200 len: 1316 microversion: 2.1 time: 0.438404 Jul 16 21:28:49.168592 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2132/4268] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:28:48 2022] GET /compute/v2.1/servers/138c70ad-acc1-4ce6-bb22-72d3d23175b1 => generated 1316 bytes in 447 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:28:49.425063 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-3ea9d88e-e43c-4e01-9591-4acabd7e87d4 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:28:49.426545 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-3ea9d88e-e43c-4e01-9591-4acabd7e87d4 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] [instance: 93519c14-aee1-4da1-8cce-775e0ad268dc] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:28:49.457058 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3ea9d88e-e43c-4e01-9591-4acabd7e87d4 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:49.457593 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3ea9d88e-e43c-4e01-9591-4acabd7e87d4 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.025s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:49.458451 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3ea9d88e-e43c-4e01-9591-4acabd7e87d4 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:49.458904 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3ea9d88e-e43c-4e01-9591-4acabd7e87d4 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:49.676818 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-3ea9d88e-e43c-4e01-9591-4acabd7e87d4 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f14ac04b7a44f17b946c5c8619554ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '349811967f4b4d1a9ab6772602a205a6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:49.688704 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-3ea9d88e-e43c-4e01-9591-4acabd7e87d4 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f14ac04b7a44f17b946c5c8619554ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '349811967f4b4d1a9ab6772602a205a6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:49.690567 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-3ea9d88e-e43c-4e01-9591-4acabd7e87d4 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f14ac04b7a44f17b946c5c8619554ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '349811967f4b4d1a9ab6772602a205a6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:49.715166 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-3ea9d88e-e43c-4e01-9591-4acabd7e87d4 tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] 10.222.0.16 "GET /compute/v2.1/servers/93519c14-aee1-4da1-8cce-775e0ad268dc" status: 200 len: 1404 microversion: 2.54 time: 0.288627 Jul 16 21:28:49.715955 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2137/4269] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:28:49 2022] GET /compute/v2.1/servers/93519c14-aee1-4da1-8cce-775e0ad268dc => generated 1404 bytes in 300 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:28:50.177235 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-9321b078-a566-47f1-97df-f09dd86adb9d tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:28:50.177859 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-9321b078-a566-47f1-97df-f09dd86adb9d tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] [instance: 138c70ad-acc1-4ce6-bb22-72d3d23175b1] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:28:50.197102 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9321b078-a566-47f1-97df-f09dd86adb9d tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:50.197832 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9321b078-a566-47f1-97df-f09dd86adb9d tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:50.198828 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9321b078-a566-47f1-97df-f09dd86adb9d tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:50.199461 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9321b078-a566-47f1-97df-f09dd86adb9d tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:50.567198 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-01603d01-64b6-446e-aaa5-70dcd0afe7b9 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Action: 'create', calling method: .version_select at 0x7fdd9d233040>, body: {"keypair": {"name": "tempest-TestNetworkBasicOps-691771975", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBLojjWBI/ZN5y1kO8yVqFkVrF4ynUPEdv423c+K2EwDXrB+uHyoZ8OxlVfFJKWj8cyiS01C6grFxT4qR6sDjmxI12qipnWQFa3f/T5Yg2A1/2MOAvq5eE99IUTv9OUZgnQ=="}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:28:50.604036 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-01603d01-64b6-446e-aaa5-70dcd0afe7b9 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Getting quotas for project bed7c94624484c7cb8abeb3bfc8757b9. Resources: {'key_pairs'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:28:50.623704 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-01603d01-64b6-446e-aaa5-70dcd0afe7b9 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Getting quotas for user 4ddfae3448704591b662de83b97a1876 and project bed7c94624484c7cb8abeb3bfc8757b9. Resources: {'key_pairs'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:28:50.813763 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-01603d01-64b6-446e-aaa5-70dcd0afe7b9 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] 10.222.0.16 "POST /compute/v2.1/os-keypairs" status: 200 len: 397 microversion: 2.1 time: 0.256601 Jul 16 21:28:50.816903 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2138/4270] 10.222.0.16 () {60 vars in 1158 bytes} [Sat Jul 16 21:28:50 2022] POST /compute/v2.1/os-keypairs => generated 397 bytes in 260 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:28:50.833711 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-4eda635e-df39-4b99-9dc2-6987f00b3a3c tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:28:50.835553 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-4eda635e-df39-4b99-9dc2-6987f00b3a3c tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] [instance: 93519c14-aee1-4da1-8cce-775e0ad268dc] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:28:50.846285 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4eda635e-df39-4b99-9dc2-6987f00b3a3c tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:50.846954 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4eda635e-df39-4b99-9dc2-6987f00b3a3c tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:50.848241 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4eda635e-df39-4b99-9dc2-6987f00b3a3c tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:50.848908 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4eda635e-df39-4b99-9dc2-6987f00b3a3c tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:50.882032 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-9321b078-a566-47f1-97df-f09dd86adb9d tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a27b3e9973c846c68f25f7981c995b73', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '125617442dae45f58c179b625bcc799d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:50.897840 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-9321b078-a566-47f1-97df-f09dd86adb9d tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] 10.222.0.16 "GET /compute/v2.1/servers/138c70ad-acc1-4ce6-bb22-72d3d23175b1" status: 200 len: 1316 microversion: 2.1 time: 0.724375 Jul 16 21:28:50.909320 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2133/4271] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:28:50 2022] GET /compute/v2.1/servers/138c70ad-acc1-4ce6-bb22-72d3d23175b1 => generated 1316 bytes in 736 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:28:51.059983 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-4eda635e-df39-4b99-9dc2-6987f00b3a3c tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f14ac04b7a44f17b946c5c8619554ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '349811967f4b4d1a9ab6772602a205a6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:51.074623 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-4eda635e-df39-4b99-9dc2-6987f00b3a3c tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '0f14ac04b7a44f17b946c5c8619554ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '349811967f4b4d1a9ab6772602a205a6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:51.084405 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-4eda635e-df39-4b99-9dc2-6987f00b3a3c tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '0f14ac04b7a44f17b946c5c8619554ad', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '349811967f4b4d1a9ab6772602a205a6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:51.087131 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-4eda635e-df39-4b99-9dc2-6987f00b3a3c tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] 10.222.0.16 "GET /compute/v2.1/servers/93519c14-aee1-4da1-8cce-775e0ad268dc" status: 200 len: 1404 microversion: 2.54 time: 0.267992 Jul 16 21:28:51.090505 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2139/4272] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:28:50 2022] GET /compute/v2.1/servers/93519c14-aee1-4da1-8cce-775e0ad268dc => generated 1404 bytes in 271 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:28:51.923549 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-8dc8398f-8de1-4412-9df5-42e0b0995a2b tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:28:51.924307 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-8dc8398f-8de1-4412-9df5-42e0b0995a2b tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] [instance: 138c70ad-acc1-4ce6-bb22-72d3d23175b1] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:28:51.933024 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8dc8398f-8de1-4412-9df5-42e0b0995a2b tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:51.933869 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8dc8398f-8de1-4412-9df5-42e0b0995a2b tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:51.934849 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8dc8398f-8de1-4412-9df5-42e0b0995a2b tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:51.936072 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8dc8398f-8de1-4412-9df5-42e0b0995a2b tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:52.109997 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-8f6d7ff0-dac4-454f-9006-85b7c0c8c7af tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:28:52.112278 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-8f6d7ff0-dac4-454f-9006-85b7c0c8c7af tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] [instance: 93519c14-aee1-4da1-8cce-775e0ad268dc] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:28:52.127439 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8f6d7ff0-dac4-454f-9006-85b7c0c8c7af tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:52.128067 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8f6d7ff0-dac4-454f-9006-85b7c0c8c7af tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:52.129446 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8f6d7ff0-dac4-454f-9006-85b7c0c8c7af tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:52.130722 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8f6d7ff0-dac4-454f-9006-85b7c0c8c7af tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:52.146659 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-8dc8398f-8de1-4412-9df5-42e0b0995a2b tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a27b3e9973c846c68f25f7981c995b73', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '125617442dae45f58c179b625bcc799d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:52.150929 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:28:52.183916 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-8dc8398f-8de1-4412-9df5-42e0b0995a2b tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] 10.222.0.16 "GET /compute/v2.1/servers/138c70ad-acc1-4ce6-bb22-72d3d23175b1" status: 200 len: 1316 microversion: 2.1 time: 0.267643 Jul 16 21:28:52.194707 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2134/4273] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:28:51 2022] GET /compute/v2.1/servers/138c70ad-acc1-4ce6-bb22-72d3d23175b1 => generated 1316 bytes in 269 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:28:52.207327 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-8f6d7ff0-dac4-454f-9006-85b7c0c8c7af tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] HTTP exception thrown: Instance 93519c14-aee1-4da1-8cce-775e0ad268dc could not be found. Jul 16 21:28:52.209035 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-8f6d7ff0-dac4-454f-9006-85b7c0c8c7af tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Request method failure captured: Jul 16 21:28:52.209035 ncdv-master-1 devstack@n-api.service[138681]: request: GET /compute/v2.1/servers/93519c14-aee1-4da1-8cce-775e0ad268dc HTTP/1.1 Jul 16 21:28:52.209035 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:28:52.209035 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:28:52.209035 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:28:52.209035 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 21:28:52.209035 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:28:52.209035 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:28:52.209035 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:28:52.209035 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:28:52.209035 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0y1kDL7SklYzBkYh011pA5B4_s3SWrigLfwkY4Z4-YkIZkYyxSRRXtbuIjdLouRi1XirbZ2gdQa2GiS775W_nb-TPwalw1Ngh6uliUSD1SHWoa5jkBrbxdHtPqiyM3PygImQDhhkgqbKtEn_QjUw7FNdIUq_ThLxLbbmGVIeAnSJy_A Jul 16 21:28:52.209035 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:28:52.209035 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:28:52.209035 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:28:52.209035 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:28:52.209035 ncdv-master-1 devstack@n-api.service[138681]: X-Openstack-Nova-Api-Version: 2.54 Jul 16 21:28:52.209035 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:28:52.209035 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:28:52.209035 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: 349811967f4b4d1a9ab6772602a205a6 Jul 16 21:28:52.209035 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-ServerShowV254Test-1871809174 Jul 16 21:28:52.209035 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:28:52.209035 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:28:52.209035 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/349811967f4b4d1a9ab6772602a205a6"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/349811967f4b4d1a9ab6772602a205a6"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_349811967f4b4d1a9ab6772602a205a6"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/349811967f4b4d1a9ab6772602a205a6"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:28:52.209035 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-ServerShowV254Test-1871809174 Jul 16 21:28:52.209035 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: 349811967f4b4d1a9ab6772602a205a6 Jul 16 21:28:52.209035 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-ServerShowV254Test-1871809174 Jul 16 21:28:52.209035 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-ServerShowV254Test-1871809174-project Jul 16 21:28:52.209035 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:28:52.209035 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:28:52.209035 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: 0f14ac04b7a44f17b946c5c8619554ad Jul 16 21:28:52.209035 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-ServerShowV254Test-1871809174-project Jul 16 21:28:52.209035 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:28:52.209035 ncdv-master-1 devstack@n-api.service[138681]: exception: Instance 93519c14-aee1-4da1-8cce-775e0ad268dc could not be found. Jul 16 21:28:52.209035 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': '93519c14-aee1-4da1-8cce-775e0ad268dc'} Jul 16 21:28:52.209035 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:28:52.209035 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:28:52.209035 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:28:52.209035 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:28:52.209035 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:28:52.209035 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:28:52.209035 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:28:52.209035 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:28:52.209035 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:28:52.209035 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:28:52.209035 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 93519c14-aee1-4da1-8cce-775e0ad268dc could not be found. Jul 16 21:28:52.209035 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:28:52.209035 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:28:52.209035 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:28:52.209035 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:28:52.209035 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:28:52.209035 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:28:52.209035 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:28:52.209035 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:28:52.209035 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:28:52.209035 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:28:52.209035 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:28:52.209035 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:28:52.209035 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:28:52.209035 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:28:52.209035 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:28:52.209035 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:28:52.209035 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 93519c14-aee1-4da1-8cce-775e0ad268dc could not be found. Jul 16 21:28:52.209035 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:28:52.209035 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:28:52.209035 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:28:52.212812 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:28:52.212812 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:28:52.212812 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:28:52.212812 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:28:52.212812 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:28:52.212812 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 93519c14-aee1-4da1-8cce-775e0ad268dc could not be found. Jul 16 21:28:52.212812 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:28:52.212812 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-8f6d7ff0-dac4-454f-9006-85b7c0c8c7af tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] Returning 404 to user: Instance 93519c14-aee1-4da1-8cce-775e0ad268dc could not be found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:28:52.217299 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-8f6d7ff0-dac4-454f-9006-85b7c0c8c7af tempest-ServerShowV254Test-1871809174 tempest-ServerShowV254Test-1871809174-project] 10.222.0.16 "GET /compute/v2.1/servers/93519c14-aee1-4da1-8cce-775e0ad268dc" status: 404 len: 111 microversion: 2.54 time: 0.115348 Jul 16 21:28:52.217897 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2140/4274] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:28:52 2022] GET /compute/v2.1/servers/93519c14-aee1-4da1-8cce-775e0ad268dc => generated 111 bytes in 117 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Jul 16 21:28:52.550280 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-e5cb7b8a-574c-4140-bb55-1dfc814197cf req-3493c9f8-5ddf-435d-b4b6-ddab8608618c service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "138c70ad-acc1-4ce6-bb22-72d3d23175b1", "name": "network-vif-plugged", "status": "completed", "tag": "0bea62aa-549f-4465-97f0-73df9f0b2836"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:28:52.567668 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-e5cb7b8a-574c-4140-bb55-1dfc814197cf req-3493c9f8-5ddf-435d-b4b6-ddab8608618c service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:52.568127 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-e5cb7b8a-574c-4140-bb55-1dfc814197cf req-3493c9f8-5ddf-435d-b4b6-ddab8608618c service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:52.586736 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-e5cb7b8a-574c-4140-bb55-1dfc814197cf req-3493c9f8-5ddf-435d-b4b6-ddab8608618c service nova] Creating event network-vif-plugged:0bea62aa-549f-4465-97f0-73df9f0b2836 for instance 138c70ad-acc1-4ce6-bb22-72d3d23175b1 on nch2-master-1 Jul 16 21:28:52.596538 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-e5cb7b8a-574c-4140-bb55-1dfc814197cf req-3493c9f8-5ddf-435d-b4b6-ddab8608618c service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.050253 Jul 16 21:28:52.597886 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2135/4275] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:28:52 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:28:53.205269 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-1aea82ac-9c86-42ba-83b7-4da90e991553 tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:28:53.206786 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-1aea82ac-9c86-42ba-83b7-4da90e991553 tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] [instance: 138c70ad-acc1-4ce6-bb22-72d3d23175b1] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:28:53.235484 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1aea82ac-9c86-42ba-83b7-4da90e991553 tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:53.238311 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1aea82ac-9c86-42ba-83b7-4da90e991553 tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:53.242922 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1aea82ac-9c86-42ba-83b7-4da90e991553 tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:53.246486 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1aea82ac-9c86-42ba-83b7-4da90e991553 tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:53.602899 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-1aea82ac-9c86-42ba-83b7-4da90e991553 tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a27b3e9973c846c68f25f7981c995b73', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '125617442dae45f58c179b625bcc799d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:53.614808 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-1aea82ac-9c86-42ba-83b7-4da90e991553 tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] 10.222.0.16 "GET /compute/v2.1/servers/138c70ad-acc1-4ce6-bb22-72d3d23175b1" status: 200 len: 1316 microversion: 2.1 time: 0.416873 Jul 16 21:28:53.616408 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2141/4276] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:28:53 2022] GET /compute/v2.1/servers/138c70ad-acc1-4ce6-bb22-72d3d23175b1 => generated 1316 bytes in 419 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:28:54.635804 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-b38f15d0-335b-452d-8edc-40fc6c040f6f tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:28:54.637114 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-b38f15d0-335b-452d-8edc-40fc6c040f6f tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] [instance: 138c70ad-acc1-4ce6-bb22-72d3d23175b1] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:28:54.655442 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b38f15d0-335b-452d-8edc-40fc6c040f6f tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:54.657290 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b38f15d0-335b-452d-8edc-40fc6c040f6f tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:54.659569 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b38f15d0-335b-452d-8edc-40fc6c040f6f tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:54.660028 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b38f15d0-335b-452d-8edc-40fc6c040f6f tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:54.933769 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-b38f15d0-335b-452d-8edc-40fc6c040f6f tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a27b3e9973c846c68f25f7981c995b73', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '125617442dae45f58c179b625bcc799d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:54.972529 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-b38f15d0-335b-452d-8edc-40fc6c040f6f tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] 10.222.0.16 "GET /compute/v2.1/servers/138c70ad-acc1-4ce6-bb22-72d3d23175b1" status: 200 len: 1316 microversion: 2.1 time: 0.319753 Jul 16 21:28:54.972529 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2136/4277] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:28:54 2022] GET /compute/v2.1/servers/138c70ad-acc1-4ce6-bb22-72d3d23175b1 => generated 1316 bytes in 322 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:28:55.335269 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-6d46d8b0-219b-4254-bb67-e116949ae187 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerShowV257Test-server-311705355", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452", "networks": "none"}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:28:55.408340 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-6d46d8b0-219b-4254-bb67-e116949ae187 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Flavor limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 16 21:28:55.408891 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-6d46d8b0-219b-4254-bb67-e116949ae187 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Image limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 16 21:28:55.409387 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-6d46d8b0-219b-4254-bb67-e116949ae187 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Flavor pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 16 21:28:55.409802 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-6d46d8b0-219b-4254-bb67-e116949ae187 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Image pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 16 21:28:55.410409 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-6d46d8b0-219b-4254-bb67-e116949ae187 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 16 21:28:55.421074 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6d46d8b0-219b-4254-bb67-e116949ae187 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:55.425865 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6d46d8b0-219b-4254-bb67-e116949ae187 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:55.425865 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6d46d8b0-219b-4254-bb67-e116949ae187 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:55.425865 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6d46d8b0-219b-4254-bb67-e116949ae187 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:55.440616 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-6d46d8b0-219b-4254-bb67-e116949ae187 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Getting quotas for project 3c6f089da37e4ff09784a2b42513ff36. Resources: {'ram', 'instances', 'cores'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:28:55.446005 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-6d46d8b0-219b-4254-bb67-e116949ae187 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Getting quotas for user af0cd3652c8d42bcb8af08852ff4f203 and project 3c6f089da37e4ff09784a2b42513ff36. Resources: {'ram', 'instances', 'cores'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:28:55.459953 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-6d46d8b0-219b-4254-bb67-e116949ae187 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Going to run 1 instances... {{(pid=138681) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 16 21:28:55.462248 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-6d46d8b0-219b-4254-bb67-e116949ae187 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] [instance: fed8328c-f9d3-4f4a-97c9-4919a94e6106] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138681) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 16 21:28:55.476266 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-3a373348-1838-4baa-b9fc-715b4126c2aa tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestNetworkBasicOps-server-1768154755", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452", "networks": [{"uuid": "2e8aed8a-8437-4cbf-af2c-f7c8a50f2c69"}], "key_name": "tempest-TestNetworkBasicOps-691771975", "security_groups": [{"name": "tempest-secgroup-smoke-2060645204"}]}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:28:55.606478 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-6d46d8b0-219b-4254-bb67-e116949ae187 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.57 time: 0.431250 Jul 16 21:28:55.607572 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2142/4278] 10.222.0.16 () {62 vars in 1187 bytes} [Sat Jul 16 21:28:55 2022] POST /compute/v2.1/servers => generated 380 bytes in 433 msecs (HTTP/1.1 202) 10 headers in 453 bytes (2 switches on core 0) Jul 16 21:28:55.618099 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-469a1fdf-d558-47ac-8485-19f3593a1be7 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:28:55.622504 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-469a1fdf-d558-47ac-8485-19f3593a1be7 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] [instance: fed8328c-f9d3-4f4a-97c9-4919a94e6106] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:28:55.646882 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_api:SELECT=92,nova_api:INSERT=7,nova_api:DELETE=1 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:28:55.725019 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-469a1fdf-d558-47ac-8485-19f3593a1be7 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'af0cd3652c8d42bcb8af08852ff4f203', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c6f089da37e4ff09784a2b42513ff36', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:55.735747 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-469a1fdf-d558-47ac-8485-19f3593a1be7 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'af0cd3652c8d42bcb8af08852ff4f203', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c6f089da37e4ff09784a2b42513ff36', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:55.740523 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-469a1fdf-d558-47ac-8485-19f3593a1be7 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'af0cd3652c8d42bcb8af08852ff4f203', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c6f089da37e4ff09784a2b42513ff36', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:55.742709 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-469a1fdf-d558-47ac-8485-19f3593a1be7 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] 10.222.0.16 "GET /compute/v2.1/servers/fed8328c-f9d3-4f4a-97c9-4919a94e6106" status: 200 len: 1277 microversion: 2.57 time: 0.128748 Jul 16 21:28:55.743761 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2143/4279] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:28:55 2022] GET /compute/v2.1/servers/fed8328c-f9d3-4f4a-97c9-4919a94e6106 => generated 1277 bytes in 130 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 16 21:28:55.754704 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.network.neutron [None req-3a373348-1838-4baa-b9fc-715b4126c2aa tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] validate_networks() for [('2e8aed8a-8437-4cbf-af2c-f7c8a50f2c69', None, None, None, None, None)] {{(pid=138680) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 16 21:28:55.941069 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-bd8c2a1c-589c-4eaa-8251-92cefce1f5d6 tempest-ExtensionsTest-277732453 tempest-ExtensionsTest-277732453-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:28:55.946542 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-bd8c2a1c-589c-4eaa-8251-92cefce1f5d6 tempest-ExtensionsTest-277732453 tempest-ExtensionsTest-277732453-project] 10.222.0.16 "GET /compute/v2.1/extensions/os-consoles" status: 200 len: 217 microversion: 2.1 time: 0.193878 Jul 16 21:28:55.951815 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2144/4280] 10.222.0.16 () {58 vars in 1169 bytes} [Sat Jul 16 21:28:55 2022] GET /compute/v2.1/extensions/os-consoles => generated 217 bytes in 200 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:28:55.957740 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-8b8e5045-1c24-430d-a0fb-b62e97a49bba tempest-ExtensionsTest-277732453 tempest-ExtensionsTest-277732453-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:28:55.960058 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-8b8e5045-1c24-430d-a0fb-b62e97a49bba tempest-ExtensionsTest-277732453 tempest-ExtensionsTest-277732453-project] 10.222.0.16 "GET /compute/v2.1/extensions" status: 200 len: 22700 microversion: 2.1 time: 0.006605 Jul 16 21:28:55.960761 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2145/4281] 10.222.0.16 () {58 vars in 1133 bytes} [Sat Jul 16 21:28:55 2022] GET /compute/v2.1/extensions => generated 22700 bytes in 8 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Jul 16 21:28:55.972893 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-d2a43697-590b-4ac0-b0ea-5b8d7bb49a52 tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:28:55.974005 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-d2a43697-590b-4ac0-b0ea-5b8d7bb49a52 tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] [instance: 138c70ad-acc1-4ce6-bb22-72d3d23175b1] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:28:55.986234 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d2a43697-590b-4ac0-b0ea-5b8d7bb49a52 tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:55.994403 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d2a43697-590b-4ac0-b0ea-5b8d7bb49a52 tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:55.996254 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d2a43697-590b-4ac0-b0ea-5b8d7bb49a52 tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:55.996899 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d2a43697-590b-4ac0-b0ea-5b8d7bb49a52 tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:56.383409 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-d2a43697-590b-4ac0-b0ea-5b8d7bb49a52 tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a27b3e9973c846c68f25f7981c995b73', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '125617442dae45f58c179b625bcc799d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:56.392079 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-d2a43697-590b-4ac0-b0ea-5b8d7bb49a52 tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] 10.222.0.16 "GET /compute/v2.1/servers/138c70ad-acc1-4ce6-bb22-72d3d23175b1" status: 200 len: 1505 microversion: 2.1 time: 0.423213 Jul 16 21:28:56.392658 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2146/4282] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:28:55 2022] GET /compute/v2.1/servers/138c70ad-acc1-4ce6-bb22-72d3d23175b1 => generated 1505 bytes in 424 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:28:56.415411 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-05323128-5b24-4793-bcda-8ac15e568d2d tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:28:56.415929 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-05323128-5b24-4793-bcda-8ac15e568d2d tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] [instance: 138c70ad-acc1-4ce6-bb22-72d3d23175b1] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:28:56.423000 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-05323128-5b24-4793-bcda-8ac15e568d2d tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:56.424243 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-05323128-5b24-4793-bcda-8ac15e568d2d tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:56.425260 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-05323128-5b24-4793-bcda-8ac15e568d2d tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:56.425930 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-05323128-5b24-4793-bcda-8ac15e568d2d tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:56.453034 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-05323128-5b24-4793-bcda-8ac15e568d2d tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] 10.222.0.16 "GET /compute/v2.1/servers/138c70ad-acc1-4ce6-bb22-72d3d23175b1/ips" status: 200 len: 109 microversion: 2.1 time: 0.041876 Jul 16 21:28:56.453948 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2147/4283] 10.222.0.16 () {58 vars in 1247 bytes} [Sat Jul 16 21:28:56 2022] GET /compute/v2.1/servers/138c70ad-acc1-4ce6-bb22-72d3d23175b1/ips => generated 109 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 16 21:28:56.465638 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-362affd2-6b74-42ec-8c50-71094a167f78 tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:28:56.466226 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-362affd2-6b74-42ec-8c50-71094a167f78 tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] [instance: 138c70ad-acc1-4ce6-bb22-72d3d23175b1] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:28:56.486983 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-362affd2-6b74-42ec-8c50-71094a167f78 tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:56.487471 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-362affd2-6b74-42ec-8c50-71094a167f78 tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:56.488266 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-362affd2-6b74-42ec-8c50-71094a167f78 tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:56.491469 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-362affd2-6b74-42ec-8c50-71094a167f78 tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:56.543038 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-362affd2-6b74-42ec-8c50-71094a167f78 tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] 10.222.0.16 "GET /compute/v2.1/servers/138c70ad-acc1-4ce6-bb22-72d3d23175b1/ips" status: 200 len: 109 microversion: 2.1 time: 0.081919 Jul 16 21:28:56.545067 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2148/4284] 10.222.0.16 () {58 vars in 1247 bytes} [Sat Jul 16 21:28:56 2022] GET /compute/v2.1/servers/138c70ad-acc1-4ce6-bb22-72d3d23175b1/ips => generated 109 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 16 21:28:56.557220 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-91326018-0203-4bdd-b089-fc4b0892323d tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:28:56.557826 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-91326018-0203-4bdd-b089-fc4b0892323d tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] [instance: 138c70ad-acc1-4ce6-bb22-72d3d23175b1] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:28:56.565719 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-91326018-0203-4bdd-b089-fc4b0892323d tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:56.566275 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-91326018-0203-4bdd-b089-fc4b0892323d tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:56.567107 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-91326018-0203-4bdd-b089-fc4b0892323d tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:56.567559 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-91326018-0203-4bdd-b089-fc4b0892323d tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:56.617881 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-91326018-0203-4bdd-b089-fc4b0892323d tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] 10.222.0.16 "GET /compute/v2.1/servers/138c70ad-acc1-4ce6-bb22-72d3d23175b1/ips/tempest-ServerAddressesTestJSON-1906142262-network" status: 200 len: 94 microversion: 2.1 time: 0.064425 Jul 16 21:28:56.622270 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2149/4285] 10.222.0.16 () {58 vars in 1400 bytes} [Sat Jul 16 21:28:56 2022] GET /compute/v2.1/servers/138c70ad-acc1-4ce6-bb22-72d3d23175b1/ips/tempest-ServerAddressesTestJSON-1906142262-network => generated 94 bytes in 66 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Jul 16 21:28:56.630336 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-b9769490-3bb1-480e-817a-d0f8ce1cc3d0 tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:28:56.630843 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-b9769490-3bb1-480e-817a-d0f8ce1cc3d0 tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] [instance: 138c70ad-acc1-4ce6-bb22-72d3d23175b1] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:28:56.642609 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b9769490-3bb1-480e-817a-d0f8ce1cc3d0 tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:56.647175 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b9769490-3bb1-480e-817a-d0f8ce1cc3d0 tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:56.648235 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b9769490-3bb1-480e-817a-d0f8ce1cc3d0 tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:56.648704 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b9769490-3bb1-480e-817a-d0f8ce1cc3d0 tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:56.692503 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-3a373348-1838-4baa-b9fc-715b4126c2aa tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Flavor limits 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 16 21:28:56.692943 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-3a373348-1838-4baa-b9fc-715b4126c2aa tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Image limits 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 16 21:28:56.693333 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-b9769490-3bb1-480e-817a-d0f8ce1cc3d0 tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] [instance: 138c70ad-acc1-4ce6-bb22-72d3d23175b1] Going to try to terminate instance {{(pid=138681) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 16 21:28:56.693454 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-3a373348-1838-4baa-b9fc-715b4126c2aa tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Flavor pref 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 16 21:28:56.693834 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-3a373348-1838-4baa-b9fc-715b4126c2aa tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Image pref 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 16 21:28:56.694280 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-3a373348-1838-4baa-b9fc-715b4126c2aa tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 16 21:28:56.711310 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3a373348-1838-4baa-b9fc-715b4126c2aa tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:56.712055 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3a373348-1838-4baa-b9fc-715b4126c2aa tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:56.713036 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3a373348-1838-4baa-b9fc-715b4126c2aa tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:56.713484 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3a373348-1838-4baa-b9fc-715b4126c2aa tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:56.736374 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-3a373348-1838-4baa-b9fc-715b4126c2aa tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Getting quotas for project bed7c94624484c7cb8abeb3bfc8757b9. Resources: {'ram', 'instances', 'cores'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:28:56.749594 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-3a373348-1838-4baa-b9fc-715b4126c2aa tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Getting quotas for user 4ddfae3448704591b662de83b97a1876 and project bed7c94624484c7cb8abeb3bfc8757b9. Resources: {'ram', 'instances', 'cores'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:28:56.765905 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-3a373348-1838-4baa-b9fc-715b4126c2aa tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Going to run 1 instances... {{(pid=138680) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 16 21:28:56.769252 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-3a373348-1838-4baa-b9fc-715b4126c2aa tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] [instance: 59a4c34e-3184-465b-b14c-0ad55192c8ce] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138680) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 16 21:28:56.926692 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-3a373348-1838-4baa-b9fc-715b4126c2aa tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 406 microversion: 2.1 time: 1.454262 Jul 16 21:28:56.927744 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2137/4286] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 21:28:55 2022] POST /compute/v2.1/servers => generated 406 bytes in 1456 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Jul 16 21:28:56.965208 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-c35ec596-1107-4a49-afbc-e7063c42e180 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:28:56.966183 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-c35ec596-1107-4a49-afbc-e7063c42e180 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] [instance: fed8328c-f9d3-4f4a-97c9-4919a94e6106] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:28:57.034984 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-c35ec596-1107-4a49-afbc-e7063c42e180 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'af0cd3652c8d42bcb8af08852ff4f203', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c6f089da37e4ff09784a2b42513ff36', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:57.050406 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-c35ec596-1107-4a49-afbc-e7063c42e180 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'af0cd3652c8d42bcb8af08852ff4f203', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c6f089da37e4ff09784a2b42513ff36', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:57.050406 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-c35ec596-1107-4a49-afbc-e7063c42e180 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'af0cd3652c8d42bcb8af08852ff4f203', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c6f089da37e4ff09784a2b42513ff36', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:57.050406 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-c35ec596-1107-4a49-afbc-e7063c42e180 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] 10.222.0.16 "GET /compute/v2.1/servers/fed8328c-f9d3-4f4a-97c9-4919a94e6106" status: 200 len: 1277 microversion: 2.57 time: 0.119039 Jul 16 21:28:57.050406 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2138/4287] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:28:56 2022] GET /compute/v2.1/servers/fed8328c-f9d3-4f4a-97c9-4919a94e6106 => generated 1277 bytes in 120 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:28:57.078089 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-17ffc1a0-3b14-444f-baa6-42b1f0d3e6cd tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:28:57.078794 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-17ffc1a0-3b14-444f-baa6-42b1f0d3e6cd tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] [instance: 59a4c34e-3184-465b-b14c-0ad55192c8ce] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:28:57.208814 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-b9769490-3bb1-480e-817a-d0f8ce1cc3d0 tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] 10.222.0.16 "DELETE /compute/v2.1/servers/138c70ad-acc1-4ce6-bb22-72d3d23175b1" status: 204 len: 0 microversion: 2.1 time: 0.582282 Jul 16 21:28:57.212551 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2150/4288] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:28:56 2022] DELETE /compute/v2.1/servers/138c70ad-acc1-4ce6-bb22-72d3d23175b1 => generated 0 bytes in 584 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Jul 16 21:28:57.222665 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-e4dbfa24-52ef-488f-aa12-91638038b48d tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:28:57.223652 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-e4dbfa24-52ef-488f-aa12-91638038b48d tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] [instance: 138c70ad-acc1-4ce6-bb22-72d3d23175b1] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:28:57.231595 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e4dbfa24-52ef-488f-aa12-91638038b48d tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:57.235518 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e4dbfa24-52ef-488f-aa12-91638038b48d tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:57.235518 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e4dbfa24-52ef-488f-aa12-91638038b48d tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:57.235518 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e4dbfa24-52ef-488f-aa12-91638038b48d tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:57.247101 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-17ffc1a0-3b14-444f-baa6-42b1f0d3e6cd tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ddfae3448704591b662de83b97a1876', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bed7c94624484c7cb8abeb3bfc8757b9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:57.262210 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-17ffc1a0-3b14-444f-baa6-42b1f0d3e6cd tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] 10.222.0.16 "GET /compute/v2.1/servers/59a4c34e-3184-465b-b14c-0ad55192c8ce" status: 200 len: 1248 microversion: 2.1 time: 0.196938 Jul 16 21:28:57.262821 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2139/4289] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:28:57 2022] GET /compute/v2.1/servers/59a4c34e-3184-465b-b14c-0ad55192c8ce => generated 1248 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:28:57.547025 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-e4dbfa24-52ef-488f-aa12-91638038b48d tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a27b3e9973c846c68f25f7981c995b73', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '125617442dae45f58c179b625bcc799d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:57.563067 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-e4dbfa24-52ef-488f-aa12-91638038b48d tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] 10.222.0.16 "GET /compute/v2.1/servers/138c70ad-acc1-4ce6-bb22-72d3d23175b1" status: 200 len: 1511 microversion: 2.1 time: 0.346706 Jul 16 21:28:57.572881 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2151/4290] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:28:57 2022] GET /compute/v2.1/servers/138c70ad-acc1-4ce6-bb22-72d3d23175b1 => generated 1511 bytes in 349 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:28:58.068425 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-42b2df02-ede5-4f60-b62c-ac715ca5b262 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:28:58.069363 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-42b2df02-ede5-4f60-b62c-ac715ca5b262 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] [instance: fed8328c-f9d3-4f4a-97c9-4919a94e6106] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:28:58.081114 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-42b2df02-ede5-4f60-b62c-ac715ca5b262 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:58.081566 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-42b2df02-ede5-4f60-b62c-ac715ca5b262 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:58.084985 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-42b2df02-ede5-4f60-b62c-ac715ca5b262 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:58.085624 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-42b2df02-ede5-4f60-b62c-ac715ca5b262 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:58.187254 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-42b2df02-ede5-4f60-b62c-ac715ca5b262 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'af0cd3652c8d42bcb8af08852ff4f203', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c6f089da37e4ff09784a2b42513ff36', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:58.202749 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-42b2df02-ede5-4f60-b62c-ac715ca5b262 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'af0cd3652c8d42bcb8af08852ff4f203', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c6f089da37e4ff09784a2b42513ff36', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:58.206434 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-42b2df02-ede5-4f60-b62c-ac715ca5b262 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'af0cd3652c8d42bcb8af08852ff4f203', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c6f089da37e4ff09784a2b42513ff36', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:58.207711 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-42b2df02-ede5-4f60-b62c-ac715ca5b262 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] 10.222.0.16 "GET /compute/v2.1/servers/fed8328c-f9d3-4f4a-97c9-4919a94e6106" status: 200 len: 1281 microversion: 2.57 time: 0.144031 Jul 16 21:28:58.208618 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2140/4291] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:28:58 2022] GET /compute/v2.1/servers/fed8328c-f9d3-4f4a-97c9-4919a94e6106 => generated 1281 bytes in 145 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:28:58.280738 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-d8d9941a-0fe7-470c-8eb5-fe83e2865e18 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:28:58.281361 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-d8d9941a-0fe7-470c-8eb5-fe83e2865e18 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] [instance: 59a4c34e-3184-465b-b14c-0ad55192c8ce] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:28:58.345241 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-d8d9941a-0fe7-470c-8eb5-fe83e2865e18 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ddfae3448704591b662de83b97a1876', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bed7c94624484c7cb8abeb3bfc8757b9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:58.352634 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-d8d9941a-0fe7-470c-8eb5-fe83e2865e18 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] 10.222.0.16 "GET /compute/v2.1/servers/59a4c34e-3184-465b-b14c-0ad55192c8ce" status: 200 len: 1248 microversion: 2.1 time: 0.076641 Jul 16 21:28:58.353268 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2152/4292] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:28:58 2022] GET /compute/v2.1/servers/59a4c34e-3184-465b-b14c-0ad55192c8ce => generated 1248 bytes in 78 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:28:58.580525 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-2d07a2d4-dc90-463f-afcb-2f00539b5c5a tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:28:58.581387 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-2d07a2d4-dc90-463f-afcb-2f00539b5c5a tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] [instance: 138c70ad-acc1-4ce6-bb22-72d3d23175b1] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:28:58.597540 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2d07a2d4-dc90-463f-afcb-2f00539b5c5a tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:58.602582 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2d07a2d4-dc90-463f-afcb-2f00539b5c5a tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.014s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:58.603966 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2d07a2d4-dc90-463f-afcb-2f00539b5c5a tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:58.606656 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2d07a2d4-dc90-463f-afcb-2f00539b5c5a tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:58.927226 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-2d07a2d4-dc90-463f-afcb-2f00539b5c5a tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a27b3e9973c846c68f25f7981c995b73', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '125617442dae45f58c179b625bcc799d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:58.936921 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-2d07a2d4-dc90-463f-afcb-2f00539b5c5a tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] 10.222.0.16 "GET /compute/v2.1/servers/138c70ad-acc1-4ce6-bb22-72d3d23175b1" status: 200 len: 1511 microversion: 2.1 time: 0.360615 Jul 16 21:28:58.950967 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2141/4293] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:28:58 2022] GET /compute/v2.1/servers/138c70ad-acc1-4ce6-bb22-72d3d23175b1 => generated 1511 bytes in 375 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:28:59.229931 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-ae14da10-f2a4-4bee-a9eb-9fe630d7f653 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:28:59.230528 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-ae14da10-f2a4-4bee-a9eb-9fe630d7f653 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] [instance: fed8328c-f9d3-4f4a-97c9-4919a94e6106] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:28:59.255981 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ae14da10-f2a4-4bee-a9eb-9fe630d7f653 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:59.256448 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ae14da10-f2a4-4bee-a9eb-9fe630d7f653 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:59.268240 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ae14da10-f2a4-4bee-a9eb-9fe630d7f653 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:59.269425 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ae14da10-f2a4-4bee-a9eb-9fe630d7f653 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.012s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:59.353069 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-ae14da10-f2a4-4bee-a9eb-9fe630d7f653 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'af0cd3652c8d42bcb8af08852ff4f203', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c6f089da37e4ff09784a2b42513ff36', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:59.362097 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-ae14da10-f2a4-4bee-a9eb-9fe630d7f653 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'af0cd3652c8d42bcb8af08852ff4f203', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c6f089da37e4ff09784a2b42513ff36', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:59.363684 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-ae14da10-f2a4-4bee-a9eb-9fe630d7f653 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'af0cd3652c8d42bcb8af08852ff4f203', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c6f089da37e4ff09784a2b42513ff36', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:59.368737 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-ae14da10-f2a4-4bee-a9eb-9fe630d7f653 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] 10.222.0.16 "GET /compute/v2.1/servers/fed8328c-f9d3-4f4a-97c9-4919a94e6106" status: 200 len: 1329 microversion: 2.57 time: 0.143699 Jul 16 21:28:59.369413 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2153/4294] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:28:59 2022] GET /compute/v2.1/servers/fed8328c-f9d3-4f4a-97c9-4919a94e6106 => generated 1329 bytes in 145 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:28:59.385212 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-7acd79f6-9ff0-434d-9b3d-d550684602c9 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:28:59.386948 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-7acd79f6-9ff0-434d-9b3d-d550684602c9 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] [instance: 59a4c34e-3184-465b-b14c-0ad55192c8ce] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:28:59.418650 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7acd79f6-9ff0-434d-9b3d-d550684602c9 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:59.419687 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7acd79f6-9ff0-434d-9b3d-d550684602c9 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:59.420857 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7acd79f6-9ff0-434d-9b3d-d550684602c9 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:59.421662 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7acd79f6-9ff0-434d-9b3d-d550684602c9 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:59.602449 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-7acd79f6-9ff0-434d-9b3d-d550684602c9 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ddfae3448704591b662de83b97a1876', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bed7c94624484c7cb8abeb3bfc8757b9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:28:59.626654 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-7acd79f6-9ff0-434d-9b3d-d550684602c9 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] 10.222.0.16 "GET /compute/v2.1/servers/59a4c34e-3184-465b-b14c-0ad55192c8ce" status: 200 len: 1244 microversion: 2.1 time: 0.258687 Jul 16 21:28:59.630666 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2142/4295] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:28:59 2022] GET /compute/v2.1/servers/59a4c34e-3184-465b-b14c-0ad55192c8ce => generated 1244 bytes in 264 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:28:59.767812 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-4d9333bd-a767-4b6e-bde2-0fa3af34d958 req-294676a7-808c-41b6-ba42-8a02dafc6e36 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "138c70ad-acc1-4ce6-bb22-72d3d23175b1", "name": "network-vif-deleted", "tag": "0bea62aa-549f-4465-97f0-73df9f0b2836"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:28:59.784181 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-4d9333bd-a767-4b6e-bde2-0fa3af34d958 req-294676a7-808c-41b6-ba42-8a02dafc6e36 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:59.784181 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-4d9333bd-a767-4b6e-bde2-0fa3af34d958 req-294676a7-808c-41b6-ba42-8a02dafc6e36 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:59.798040 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:28:59.805231 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-4d9333bd-a767-4b6e-bde2-0fa3af34d958 req-294676a7-808c-41b6-ba42-8a02dafc6e36 service nova] Creating event network-vif-deleted:0bea62aa-549f-4465-97f0-73df9f0b2836 for instance 138c70ad-acc1-4ce6-bb22-72d3d23175b1 on nch2-master-1 Jul 16 21:28:59.812842 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-4d9333bd-a767-4b6e-bde2-0fa3af34d958 req-294676a7-808c-41b6-ba42-8a02dafc6e36 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.330484 Jul 16 21:28:59.813999 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2154/4296] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:28:59 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 332 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 16 21:28:59.956572 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-9fd504fb-bf18-49d6-ac8e-a11c8ca08497 tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:28:59.957248 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-9fd504fb-bf18-49d6-ac8e-a11c8ca08497 tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] [instance: 138c70ad-acc1-4ce6-bb22-72d3d23175b1] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:28:59.964135 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9fd504fb-bf18-49d6-ac8e-a11c8ca08497 tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:59.964814 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9fd504fb-bf18-49d6-ac8e-a11c8ca08497 tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:28:59.965778 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9fd504fb-bf18-49d6-ac8e-a11c8ca08497 tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:28:59.966409 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9fd504fb-bf18-49d6-ac8e-a11c8ca08497 tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:00.040593 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-9fd504fb-bf18-49d6-ac8e-a11c8ca08497 tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a27b3e9973c846c68f25f7981c995b73', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '125617442dae45f58c179b625bcc799d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:00.049503 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-9fd504fb-bf18-49d6-ac8e-a11c8ca08497 tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] 10.222.0.16 "GET /compute/v2.1/servers/138c70ad-acc1-4ce6-bb22-72d3d23175b1" status: 200 len: 1301 microversion: 2.1 time: 0.096863 Jul 16 21:29:00.050023 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2143/4297] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:28:59 2022] GET /compute/v2.1/servers/138c70ad-acc1-4ce6-bb22-72d3d23175b1 => generated 1301 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:29:00.398135 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-07d7b7f0-0d5d-41e8-8b42-2993719dc9d4 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:29:00.399649 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-07d7b7f0-0d5d-41e8-8b42-2993719dc9d4 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] [instance: fed8328c-f9d3-4f4a-97c9-4919a94e6106] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:29:00.406651 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-07d7b7f0-0d5d-41e8-8b42-2993719dc9d4 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:00.407136 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-07d7b7f0-0d5d-41e8-8b42-2993719dc9d4 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:00.408198 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-07d7b7f0-0d5d-41e8-8b42-2993719dc9d4 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:00.408639 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-07d7b7f0-0d5d-41e8-8b42-2993719dc9d4 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:00.485398 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-07d7b7f0-0d5d-41e8-8b42-2993719dc9d4 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'af0cd3652c8d42bcb8af08852ff4f203', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c6f089da37e4ff09784a2b42513ff36', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:00.492055 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-07d7b7f0-0d5d-41e8-8b42-2993719dc9d4 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'af0cd3652c8d42bcb8af08852ff4f203', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c6f089da37e4ff09784a2b42513ff36', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:00.493579 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-07d7b7f0-0d5d-41e8-8b42-2993719dc9d4 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'af0cd3652c8d42bcb8af08852ff4f203', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c6f089da37e4ff09784a2b42513ff36', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:00.494725 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-07d7b7f0-0d5d-41e8-8b42-2993719dc9d4 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] 10.222.0.16 "GET /compute/v2.1/servers/fed8328c-f9d3-4f4a-97c9-4919a94e6106" status: 200 len: 1347 microversion: 2.57 time: 0.102115 Jul 16 21:29:00.495437 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2155/4298] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:29:00 2022] GET /compute/v2.1/servers/fed8328c-f9d3-4f4a-97c9-4919a94e6106 => generated 1347 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:29:00.647756 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-b3514fd6-cb6c-484e-830c-315c1e5726aa tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:29:00.648382 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-b3514fd6-cb6c-484e-830c-315c1e5726aa tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] [instance: 59a4c34e-3184-465b-b14c-0ad55192c8ce] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:29:00.655496 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b3514fd6-cb6c-484e-830c-315c1e5726aa tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:00.655957 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b3514fd6-cb6c-484e-830c-315c1e5726aa tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:00.656733 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b3514fd6-cb6c-484e-830c-315c1e5726aa tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:00.657184 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b3514fd6-cb6c-484e-830c-315c1e5726aa tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:00.723657 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-b3514fd6-cb6c-484e-830c-315c1e5726aa tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ddfae3448704591b662de83b97a1876', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bed7c94624484c7cb8abeb3bfc8757b9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:00.735601 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-b3514fd6-cb6c-484e-830c-315c1e5726aa tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] 10.222.0.16 "GET /compute/v2.1/servers/59a4c34e-3184-465b-b14c-0ad55192c8ce" status: 200 len: 1300 microversion: 2.1 time: 0.091663 Jul 16 21:29:00.736786 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2144/4299] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:29:00 2022] GET /compute/v2.1/servers/59a4c34e-3184-465b-b14c-0ad55192c8ce => generated 1300 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:29:01.066598 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-1406a9b2-2350-49c5-90c5-d8741041c679 tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:29:01.067274 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-1406a9b2-2350-49c5-90c5-d8741041c679 tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] [instance: 138c70ad-acc1-4ce6-bb22-72d3d23175b1] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:29:01.081398 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1406a9b2-2350-49c5-90c5-d8741041c679 tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:01.081799 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1406a9b2-2350-49c5-90c5-d8741041c679 tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:01.082591 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1406a9b2-2350-49c5-90c5-d8741041c679 tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:01.083085 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1406a9b2-2350-49c5-90c5-d8741041c679 tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:01.131939 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1406a9b2-2350-49c5-90c5-d8741041c679 tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:01.135211 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1406a9b2-2350-49c5-90c5-d8741041c679 tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:01.135211 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.objects.instance [None req-1406a9b2-2350-49c5-90c5-d8741041c679 tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] Lazy-loading 'fault' on Instance uuid 138c70ad-acc1-4ce6-bb22-72d3d23175b1 {{(pid=138681) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:29:01.217218 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-1406a9b2-2350-49c5-90c5-d8741041c679 tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a27b3e9973c846c68f25f7981c995b73', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '125617442dae45f58c179b625bcc799d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:01.225168 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-1406a9b2-2350-49c5-90c5-d8741041c679 tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] 10.222.0.16 "GET /compute/v2.1/servers/138c70ad-acc1-4ce6-bb22-72d3d23175b1" status: 200 len: 1306 microversion: 2.1 time: 0.162386 Jul 16 21:29:01.225747 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2156/4300] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:29:01 2022] GET /compute/v2.1/servers/138c70ad-acc1-4ce6-bb22-72d3d23175b1 => generated 1306 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:29:01.514231 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-056b9a28-64d5-4bfe-bcb9-d23c8fb2286a tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:29:01.514838 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-056b9a28-64d5-4bfe-bcb9-d23c8fb2286a tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] [instance: fed8328c-f9d3-4f4a-97c9-4919a94e6106] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:29:01.521079 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-056b9a28-64d5-4bfe-bcb9-d23c8fb2286a tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:01.521537 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-056b9a28-64d5-4bfe-bcb9-d23c8fb2286a tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:01.522312 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-056b9a28-64d5-4bfe-bcb9-d23c8fb2286a tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:01.522949 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-056b9a28-64d5-4bfe-bcb9-d23c8fb2286a tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:01.643658 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-056b9a28-64d5-4bfe-bcb9-d23c8fb2286a tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'af0cd3652c8d42bcb8af08852ff4f203', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c6f089da37e4ff09784a2b42513ff36', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:01.652738 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-056b9a28-64d5-4bfe-bcb9-d23c8fb2286a tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'af0cd3652c8d42bcb8af08852ff4f203', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c6f089da37e4ff09784a2b42513ff36', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:01.655288 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-056b9a28-64d5-4bfe-bcb9-d23c8fb2286a tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'af0cd3652c8d42bcb8af08852ff4f203', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c6f089da37e4ff09784a2b42513ff36', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:01.656474 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-056b9a28-64d5-4bfe-bcb9-d23c8fb2286a tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] 10.222.0.16 "GET /compute/v2.1/servers/fed8328c-f9d3-4f4a-97c9-4919a94e6106" status: 200 len: 1335 microversion: 2.57 time: 0.146435 Jul 16 21:29:01.657039 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2145/4301] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:29:01 2022] GET /compute/v2.1/servers/fed8328c-f9d3-4f4a-97c9-4919a94e6106 => generated 1335 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:29:01.753913 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-13ffc266-7d44-4904-aa81-1a4e080f5493 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:29:01.754563 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-13ffc266-7d44-4904-aa81-1a4e080f5493 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] [instance: 59a4c34e-3184-465b-b14c-0ad55192c8ce] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:29:01.761817 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-13ffc266-7d44-4904-aa81-1a4e080f5493 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:01.762278 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-13ffc266-7d44-4904-aa81-1a4e080f5493 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:01.763226 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-13ffc266-7d44-4904-aa81-1a4e080f5493 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:01.763720 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-13ffc266-7d44-4904-aa81-1a4e080f5493 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:01.852869 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-13ffc266-7d44-4904-aa81-1a4e080f5493 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ddfae3448704591b662de83b97a1876', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bed7c94624484c7cb8abeb3bfc8757b9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:01.880200 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-13ffc266-7d44-4904-aa81-1a4e080f5493 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] 10.222.0.16 "GET /compute/v2.1/servers/59a4c34e-3184-465b-b14c-0ad55192c8ce" status: 200 len: 1308 microversion: 2.1 time: 0.128762 Jul 16 21:29:01.882361 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2157/4302] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:29:01 2022] GET /compute/v2.1/servers/59a4c34e-3184-465b-b14c-0ad55192c8ce => generated 1308 bytes in 134 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:29:02.244035 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-3c9e3b9b-25fe-4f95-a49b-3d2fb28d29ab tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:29:02.244666 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-3c9e3b9b-25fe-4f95-a49b-3d2fb28d29ab tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] [instance: 138c70ad-acc1-4ce6-bb22-72d3d23175b1] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:29:02.263470 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3c9e3b9b-25fe-4f95-a49b-3d2fb28d29ab tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:02.263470 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3c9e3b9b-25fe-4f95-a49b-3d2fb28d29ab tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:02.263470 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3c9e3b9b-25fe-4f95-a49b-3d2fb28d29ab tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:02.263470 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3c9e3b9b-25fe-4f95-a49b-3d2fb28d29ab tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:02.275113 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-3c9e3b9b-25fe-4f95-a49b-3d2fb28d29ab tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] HTTP exception thrown: Instance 138c70ad-acc1-4ce6-bb22-72d3d23175b1 could not be found. Jul 16 21:29:02.277295 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-3c9e3b9b-25fe-4f95-a49b-3d2fb28d29ab tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] Request method failure captured: Jul 16 21:29:02.277295 ncdv-master-1 devstack@n-api.service[138680]: request: GET /compute/v2.1/servers/138c70ad-acc1-4ce6-bb22-72d3d23175b1 HTTP/1.1 Jul 16 21:29:02.277295 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:29:02.277295 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:29:02.277295 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:29:02.277295 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 0 Jul 16 21:29:02.277295 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:29:02.277295 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:29:02.277295 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:29:02.277295 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:29:02.277295 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0y2CoEwHlk6eSs-F1otvHV0KFPfcRHHNcZAvdN9MUhKKjFnAMQiqNixxIwnTiuIQ00NhBkHJFugHUQxSECX3e6pZg41mag9dghtb_Y3mdc0ZPHUPWL4EHD_RlJ7-9B9qpqWWr6BkZXdBEVYXkbfawdg2u44EHoL_8y-DAgEmugHfsZE Jul 16 21:29:02.277295 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:29:02.277295 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:29:02.277295 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:29:02.277295 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:29:02.277295 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:29:02.277295 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:29:02.277295 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: 125617442dae45f58c179b625bcc799d Jul 16 21:29:02.277295 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-ServerAddressesTestJSON-1664683043 Jul 16 21:29:02.277295 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:29:02.277295 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:29:02.277295 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/125617442dae45f58c179b625bcc799d"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/125617442dae45f58c179b625bcc799d"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_125617442dae45f58c179b625bcc799d"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/125617442dae45f58c179b625bcc799d"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:29:02.277295 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-ServerAddressesTestJSON-1664683043 Jul 16 21:29:02.277295 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: 125617442dae45f58c179b625bcc799d Jul 16 21:29:02.277295 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-ServerAddressesTestJSON-1664683043 Jul 16 21:29:02.277295 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-ServerAddressesTestJSON-1664683043-project Jul 16 21:29:02.277295 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:29:02.277295 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:29:02.277295 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: a27b3e9973c846c68f25f7981c995b73 Jul 16 21:29:02.277295 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-ServerAddressesTestJSON-1664683043-project Jul 16 21:29:02.277295 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 21:29:02.277295 ncdv-master-1 devstack@n-api.service[138680]: exception: Instance 138c70ad-acc1-4ce6-bb22-72d3d23175b1 could not be found. Jul 16 21:29:02.277295 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': '138c70ad-acc1-4ce6-bb22-72d3d23175b1'} Jul 16 21:29:02.277295 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:29:02.277295 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:29:02.277295 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:29:02.277295 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:29:02.277295 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:29:02.277295 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:29:02.277295 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:29:02.277295 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:29:02.277295 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:29:02.277295 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:29:02.277295 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 138c70ad-acc1-4ce6-bb22-72d3d23175b1 could not be found. Jul 16 21:29:02.277295 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:29:02.277295 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:29:02.277295 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:29:02.277295 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:29:02.277295 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:29:02.277295 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:29:02.277295 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:29:02.277295 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:29:02.277295 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:29:02.277295 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:29:02.277295 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:29:02.277295 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:29:02.277295 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:29:02.277295 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:29:02.277295 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:29:02.277295 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:29:02.277295 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 138c70ad-acc1-4ce6-bb22-72d3d23175b1 could not be found. Jul 16 21:29:02.277295 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:29:02.277295 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:29:02.277295 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:29:02.282243 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:29:02.282243 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:29:02.282243 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:29:02.282243 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:29:02.282243 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:29:02.282243 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 138c70ad-acc1-4ce6-bb22-72d3d23175b1 could not be found. Jul 16 21:29:02.282243 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:29:02.284213 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-3c9e3b9b-25fe-4f95-a49b-3d2fb28d29ab tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] Returning 404 to user: Instance 138c70ad-acc1-4ce6-bb22-72d3d23175b1 could not be found. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:29:02.285552 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-3c9e3b9b-25fe-4f95-a49b-3d2fb28d29ab tempest-ServerAddressesTestJSON-1664683043 tempest-ServerAddressesTestJSON-1664683043-project] 10.222.0.16 "GET /compute/v2.1/servers/138c70ad-acc1-4ce6-bb22-72d3d23175b1" status: 404 len: 111 microversion: 2.1 time: 0.045471 Jul 16 21:29:02.286151 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2146/4303] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:29:02 2022] GET /compute/v2.1/servers/138c70ad-acc1-4ce6-bb22-72d3d23175b1 => generated 111 bytes in 47 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:29:02.680736 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-c9d99bf2-fd5e-427e-aed8-bb73d19b78b0 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:29:02.681476 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-c9d99bf2-fd5e-427e-aed8-bb73d19b78b0 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] [instance: fed8328c-f9d3-4f4a-97c9-4919a94e6106] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:29:02.694490 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c9d99bf2-fd5e-427e-aed8-bb73d19b78b0 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:02.694490 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c9d99bf2-fd5e-427e-aed8-bb73d19b78b0 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:02.694490 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c9d99bf2-fd5e-427e-aed8-bb73d19b78b0 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:02.694490 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c9d99bf2-fd5e-427e-aed8-bb73d19b78b0 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:02.798419 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-c9d99bf2-fd5e-427e-aed8-bb73d19b78b0 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'af0cd3652c8d42bcb8af08852ff4f203', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c6f089da37e4ff09784a2b42513ff36', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:02.816135 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-c9d99bf2-fd5e-427e-aed8-bb73d19b78b0 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'af0cd3652c8d42bcb8af08852ff4f203', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c6f089da37e4ff09784a2b42513ff36', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:02.817665 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-c9d99bf2-fd5e-427e-aed8-bb73d19b78b0 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'af0cd3652c8d42bcb8af08852ff4f203', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c6f089da37e4ff09784a2b42513ff36', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:02.818857 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-c9d99bf2-fd5e-427e-aed8-bb73d19b78b0 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] 10.222.0.16 "GET /compute/v2.1/servers/fed8328c-f9d3-4f4a-97c9-4919a94e6106" status: 200 len: 1335 microversion: 2.57 time: 0.142346 Jul 16 21:29:02.819490 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2158/4304] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:29:02 2022] GET /compute/v2.1/servers/fed8328c-f9d3-4f4a-97c9-4919a94e6106 => generated 1335 bytes in 144 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:29:02.905929 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-1dc01a1f-8420-40df-832a-2efa39b715ae tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:29:02.906807 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-1dc01a1f-8420-40df-832a-2efa39b715ae tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] [instance: 59a4c34e-3184-465b-b14c-0ad55192c8ce] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:29:02.914517 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1dc01a1f-8420-40df-832a-2efa39b715ae tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:02.915203 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1dc01a1f-8420-40df-832a-2efa39b715ae tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:02.916354 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1dc01a1f-8420-40df-832a-2efa39b715ae tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:02.916862 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1dc01a1f-8420-40df-832a-2efa39b715ae tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:03.043233 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-1dc01a1f-8420-40df-832a-2efa39b715ae tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ddfae3448704591b662de83b97a1876', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bed7c94624484c7cb8abeb3bfc8757b9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:03.055352 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-1dc01a1f-8420-40df-832a-2efa39b715ae tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] 10.222.0.16 "GET /compute/v2.1/servers/59a4c34e-3184-465b-b14c-0ad55192c8ce" status: 200 len: 1306 microversion: 2.1 time: 0.158526 Jul 16 21:29:03.056163 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2147/4305] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:29:02 2022] GET /compute/v2.1/servers/59a4c34e-3184-465b-b14c-0ad55192c8ce => generated 1306 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:29:03.843020 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-2ec9405b-2534-42e1-abe6-cdfe9c026573 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:29:03.843643 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-2ec9405b-2534-42e1-abe6-cdfe9c026573 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] [instance: fed8328c-f9d3-4f4a-97c9-4919a94e6106] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:29:03.851500 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2ec9405b-2534-42e1-abe6-cdfe9c026573 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:03.854504 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2ec9405b-2534-42e1-abe6-cdfe9c026573 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:03.854504 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2ec9405b-2534-42e1-abe6-cdfe9c026573 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:03.854504 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2ec9405b-2534-42e1-abe6-cdfe9c026573 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:04.010962 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-2ec9405b-2534-42e1-abe6-cdfe9c026573 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'af0cd3652c8d42bcb8af08852ff4f203', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c6f089da37e4ff09784a2b42513ff36', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:04.022101 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-2ec9405b-2534-42e1-abe6-cdfe9c026573 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'af0cd3652c8d42bcb8af08852ff4f203', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c6f089da37e4ff09784a2b42513ff36', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:04.023953 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-2ec9405b-2534-42e1-abe6-cdfe9c026573 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'af0cd3652c8d42bcb8af08852ff4f203', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c6f089da37e4ff09784a2b42513ff36', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:04.025210 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-2ec9405b-2534-42e1-abe6-cdfe9c026573 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] 10.222.0.16 "GET /compute/v2.1/servers/fed8328c-f9d3-4f4a-97c9-4919a94e6106" status: 200 len: 1356 microversion: 2.57 time: 0.187089 Jul 16 21:29:04.026418 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2159/4306] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:29:03 2022] GET /compute/v2.1/servers/fed8328c-f9d3-4f4a-97c9-4919a94e6106 => generated 1356 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:29:04.045437 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-76efa57b-b317-4262-bf07-cff4cc7108ab tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Action: 'action', calling method: >, body: {"rebuild": {"user_data": "ZWNobyAiaGVsbG8gd29ybGQi", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9"}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:29:04.046933 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-76efa57b-b317-4262-bf07-cff4cc7108ab tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] [instance: fed8328c-f9d3-4f4a-97c9-4919a94e6106] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:29:04.054979 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-76efa57b-b317-4262-bf07-cff4cc7108ab tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:04.055654 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-76efa57b-b317-4262-bf07-cff4cc7108ab tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:04.056584 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-76efa57b-b317-4262-bf07-cff4cc7108ab tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:04.057405 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-76efa57b-b317-4262-bf07-cff4cc7108ab tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:04.079606 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-cac77685-87de-4c0b-ba4f-5bc81960c987 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:29:04.080526 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-cac77685-87de-4c0b-ba4f-5bc81960c987 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] [instance: 59a4c34e-3184-465b-b14c-0ad55192c8ce] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:29:04.088885 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cac77685-87de-4c0b-ba4f-5bc81960c987 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:04.090868 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cac77685-87de-4c0b-ba4f-5bc81960c987 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:04.092384 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cac77685-87de-4c0b-ba4f-5bc81960c987 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:04.093372 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cac77685-87de-4c0b-ba4f-5bc81960c987 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:04.183729 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-76efa57b-b317-4262-bf07-cff4cc7108ab tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Flavor limits 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 16 21:29:04.184215 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-76efa57b-b317-4262-bf07-cff4cc7108ab tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Image limits 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 16 21:29:04.186138 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-76efa57b-b317-4262-bf07-cff4cc7108ab tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Flavor pref 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 16 21:29:04.186804 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-76efa57b-b317-4262-bf07-cff4cc7108ab tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Image pref 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 16 21:29:04.187897 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-76efa57b-b317-4262-bf07-cff4cc7108ab tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 16 21:29:04.194525 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-cac77685-87de-4c0b-ba4f-5bc81960c987 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ddfae3448704591b662de83b97a1876', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bed7c94624484c7cb8abeb3bfc8757b9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:04.211396 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-cac77685-87de-4c0b-ba4f-5bc81960c987 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] 10.222.0.16 "GET /compute/v2.1/servers/59a4c34e-3184-465b-b14c-0ad55192c8ce" status: 200 len: 1306 microversion: 2.1 time: 0.137953 Jul 16 21:29:04.212538 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2160/4307] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:29:04 2022] GET /compute/v2.1/servers/59a4c34e-3184-465b-b14c-0ad55192c8ce => generated 1306 bytes in 140 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:29:04.509161 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-76efa57b-b317-4262-bf07-cff4cc7108ab tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] [instance: fed8328c-f9d3-4f4a-97c9-4919a94e6106] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:29:04.539187 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-76efa57b-b317-4262-bf07-cff4cc7108ab tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:04.539187 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-76efa57b-b317-4262-bf07-cff4cc7108ab tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:04.539187 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-76efa57b-b317-4262-bf07-cff4cc7108ab tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:04.539187 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-76efa57b-b317-4262-bf07-cff4cc7108ab tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:04.634995 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-76efa57b-b317-4262-bf07-cff4cc7108ab tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] 10.222.0.16 "POST /compute/v2.1/servers/fed8328c-f9d3-4f4a-97c9-4919a94e6106/action" status: 202 len: 1137 microversion: 2.57 time: 0.594250 Jul 16 21:29:04.637022 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2148/4308] 10.222.0.16 () {62 vars in 1319 bytes} [Sat Jul 16 21:29:04 2022] POST /compute/v2.1/servers/fed8328c-f9d3-4f4a-97c9-4919a94e6106/action => generated 1137 bytes in 597 msecs (HTTP/1.1 202) 10 headers in 454 bytes (1 switches on core 0) Jul 16 21:29:04.651857 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-e121e13d-7fef-41ac-923a-50f44c43de88 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:29:04.653895 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-e121e13d-7fef-41ac-923a-50f44c43de88 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] [instance: fed8328c-f9d3-4f4a-97c9-4919a94e6106] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:29:04.662296 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e121e13d-7fef-41ac-923a-50f44c43de88 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:04.662758 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e121e13d-7fef-41ac-923a-50f44c43de88 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:04.665341 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e121e13d-7fef-41ac-923a-50f44c43de88 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:04.665804 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e121e13d-7fef-41ac-923a-50f44c43de88 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:04.689875 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-a3a95002-8668-4015-94b7-58ccf6428867 tempest-VolumesTestJSON-1603190450 tempest-VolumesTestJSON-1603190450-project] Calling method '.version_select at 0x7fdda2aefa60>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:29:04.787523 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-a3a95002-8668-4015-94b7-58ccf6428867 tempest-VolumesTestJSON-1603190450 tempest-VolumesTestJSON-1603190450-project] 10.222.0.16 "GET /compute/v2.1/os-volumes/1aaa0726-b365-4d7a-8180-846786c1cfde" status: 200 len: 343 microversion: 2.1 time: 0.103463 Jul 16 21:29:04.788577 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2149/4309] 10.222.0.16 () {58 vars in 1244 bytes} [Sat Jul 16 21:29:04 2022] GET /compute/v2.1/os-volumes/1aaa0726-b365-4d7a-8180-846786c1cfde => generated 343 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:29:04.951960 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-e121e13d-7fef-41ac-923a-50f44c43de88 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'af0cd3652c8d42bcb8af08852ff4f203', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c6f089da37e4ff09784a2b42513ff36', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:04.973078 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-e121e13d-7fef-41ac-923a-50f44c43de88 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'af0cd3652c8d42bcb8af08852ff4f203', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c6f089da37e4ff09784a2b42513ff36', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:04.974670 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-e121e13d-7fef-41ac-923a-50f44c43de88 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'af0cd3652c8d42bcb8af08852ff4f203', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c6f089da37e4ff09784a2b42513ff36', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:04.983963 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-e121e13d-7fef-41ac-923a-50f44c43de88 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] 10.222.0.16 "GET /compute/v2.1/servers/fed8328c-f9d3-4f4a-97c9-4919a94e6106" status: 200 len: 1365 microversion: 2.57 time: 0.334600 Jul 16 21:29:04.987767 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2161/4310] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:29:04 2022] GET /compute/v2.1/servers/fed8328c-f9d3-4f4a-97c9-4919a94e6106 => generated 1365 bytes in 340 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:29:05.230439 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-379f45dd-994d-4dd4-ac76-1690b67d0d89 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:29:05.231185 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-379f45dd-994d-4dd4-ac76-1690b67d0d89 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] [instance: 59a4c34e-3184-465b-b14c-0ad55192c8ce] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:29:05.243413 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-379f45dd-994d-4dd4-ac76-1690b67d0d89 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:05.243887 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-379f45dd-994d-4dd4-ac76-1690b67d0d89 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:05.247910 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-379f45dd-994d-4dd4-ac76-1690b67d0d89 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:05.248677 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-379f45dd-994d-4dd4-ac76-1690b67d0d89 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:05.646984 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-379f45dd-994d-4dd4-ac76-1690b67d0d89 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ddfae3448704591b662de83b97a1876', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bed7c94624484c7cb8abeb3bfc8757b9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:05.663828 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-379f45dd-994d-4dd4-ac76-1690b67d0d89 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] 10.222.0.16 "GET /compute/v2.1/servers/59a4c34e-3184-465b-b14c-0ad55192c8ce" status: 200 len: 1374 microversion: 2.1 time: 0.437968 Jul 16 21:29:05.673250 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2150/4311] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:29:05 2022] GET /compute/v2.1/servers/59a4c34e-3184-465b-b14c-0ad55192c8ce => generated 1374 bytes in 448 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:29:06.008377 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-f41d556a-1083-4668-920c-ce2f2d73f9f5 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:29:06.011392 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-f41d556a-1083-4668-920c-ce2f2d73f9f5 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] [instance: fed8328c-f9d3-4f4a-97c9-4919a94e6106] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:29:06.013929 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell0:SELECT=2 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:29:06.024004 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f41d556a-1083-4668-920c-ce2f2d73f9f5 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:06.024466 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f41d556a-1083-4668-920c-ce2f2d73f9f5 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:06.025255 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f41d556a-1083-4668-920c-ce2f2d73f9f5 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:06.025705 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f41d556a-1083-4668-920c-ce2f2d73f9f5 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:06.282425 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-f41d556a-1083-4668-920c-ce2f2d73f9f5 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'af0cd3652c8d42bcb8af08852ff4f203', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c6f089da37e4ff09784a2b42513ff36', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:06.302103 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-f41d556a-1083-4668-920c-ce2f2d73f9f5 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'af0cd3652c8d42bcb8af08852ff4f203', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c6f089da37e4ff09784a2b42513ff36', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:06.305300 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-f41d556a-1083-4668-920c-ce2f2d73f9f5 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'af0cd3652c8d42bcb8af08852ff4f203', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c6f089da37e4ff09784a2b42513ff36', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:06.306878 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-f41d556a-1083-4668-920c-ce2f2d73f9f5 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] 10.222.0.16 "GET /compute/v2.1/servers/fed8328c-f9d3-4f4a-97c9-4919a94e6106" status: 200 len: 1365 microversion: 2.57 time: 0.306514 Jul 16 21:29:06.324984 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2162/4312] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:29:05 2022] GET /compute/v2.1/servers/fed8328c-f9d3-4f4a-97c9-4919a94e6106 => generated 1365 bytes in 325 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:29:06.696005 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-441dec2b-c4a2-4bb9-b420-1b155c4cc64c tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:29:06.697450 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-441dec2b-c4a2-4bb9-b420-1b155c4cc64c tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] [instance: 59a4c34e-3184-465b-b14c-0ad55192c8ce] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:29:06.716616 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-441dec2b-c4a2-4bb9-b420-1b155c4cc64c tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:06.717593 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-441dec2b-c4a2-4bb9-b420-1b155c4cc64c tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:06.718711 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-441dec2b-c4a2-4bb9-b420-1b155c4cc64c tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:06.720582 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-441dec2b-c4a2-4bb9-b420-1b155c4cc64c tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:06.723025 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell0:SELECT=2 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:29:07.170394 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-441dec2b-c4a2-4bb9-b420-1b155c4cc64c tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ddfae3448704591b662de83b97a1876', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bed7c94624484c7cb8abeb3bfc8757b9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:07.176465 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-441dec2b-c4a2-4bb9-b420-1b155c4cc64c tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] 10.222.0.16 "GET /compute/v2.1/servers/59a4c34e-3184-465b-b14c-0ad55192c8ce" status: 200 len: 1374 microversion: 2.1 time: 0.487088 Jul 16 21:29:07.177050 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2151/4313] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:29:06 2022] GET /compute/v2.1/servers/59a4c34e-3184-465b-b14c-0ad55192c8ce => generated 1374 bytes in 488 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:29:07.326176 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-0bd67503-9475-4bee-a196-129a68c2c569 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:29:07.326804 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-0bd67503-9475-4bee-a196-129a68c2c569 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] [instance: fed8328c-f9d3-4f4a-97c9-4919a94e6106] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:29:07.339495 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0bd67503-9475-4bee-a196-129a68c2c569 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:07.340497 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0bd67503-9475-4bee-a196-129a68c2c569 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:07.342145 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0bd67503-9475-4bee-a196-129a68c2c569 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:07.342980 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0bd67503-9475-4bee-a196-129a68c2c569 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:07.504042 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-0bd67503-9475-4bee-a196-129a68c2c569 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'af0cd3652c8d42bcb8af08852ff4f203', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c6f089da37e4ff09784a2b42513ff36', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:07.515507 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-0bd67503-9475-4bee-a196-129a68c2c569 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'af0cd3652c8d42bcb8af08852ff4f203', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c6f089da37e4ff09784a2b42513ff36', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:07.529115 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-0bd67503-9475-4bee-a196-129a68c2c569 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'af0cd3652c8d42bcb8af08852ff4f203', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c6f089da37e4ff09784a2b42513ff36', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:07.531947 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-0bd67503-9475-4bee-a196-129a68c2c569 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] 10.222.0.16 "GET /compute/v2.1/servers/fed8328c-f9d3-4f4a-97c9-4919a94e6106" status: 200 len: 1365 microversion: 2.57 time: 0.208474 Jul 16 21:29:07.531947 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2163/4314] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:29:07 2022] GET /compute/v2.1/servers/fed8328c-f9d3-4f4a-97c9-4919a94e6106 => generated 1365 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:29:07.603208 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-e31e5fac-c8db-4f40-8810-2a98c3b05960 req-45f8e1e3-7eb1-4b77-ae15-626839686051 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "59a4c34e-3184-465b-b14c-0ad55192c8ce", "tag": "cad17319-3e42-44f8-911a-cd258470adec"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:29:07.609692 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell0:SELECT=10 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:29:07.621636 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-e31e5fac-c8db-4f40-8810-2a98c3b05960 req-45f8e1e3-7eb1-4b77-ae15-626839686051 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:07.628149 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-e31e5fac-c8db-4f40-8810-2a98c3b05960 req-45f8e1e3-7eb1-4b77-ae15-626839686051 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:07.652236 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-e31e5fac-c8db-4f40-8810-2a98c3b05960 req-45f8e1e3-7eb1-4b77-ae15-626839686051 service nova] Creating event network-changed:cad17319-3e42-44f8-911a-cd258470adec for instance 59a4c34e-3184-465b-b14c-0ad55192c8ce on nch2-master-1 Jul 16 21:29:07.661133 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-e31e5fac-c8db-4f40-8810-2a98c3b05960 req-45f8e1e3-7eb1-4b77-ae15-626839686051 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.061544 Jul 16 21:29:07.663402 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2152/4315] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:29:07 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 65 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:29:08.193175 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-f1ef8d20-52f6-4de2-b868-597e4e2c38ec tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:29:08.193831 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-f1ef8d20-52f6-4de2-b868-597e4e2c38ec tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] [instance: 59a4c34e-3184-465b-b14c-0ad55192c8ce] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:29:08.206371 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f1ef8d20-52f6-4de2-b868-597e4e2c38ec tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:08.206848 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f1ef8d20-52f6-4de2-b868-597e4e2c38ec tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:08.207971 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f1ef8d20-52f6-4de2-b868-597e4e2c38ec tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:08.212484 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f1ef8d20-52f6-4de2-b868-597e4e2c38ec tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:08.349131 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell0:SELECT=14 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:29:08.481012 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-f1ef8d20-52f6-4de2-b868-597e4e2c38ec tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ddfae3448704591b662de83b97a1876', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bed7c94624484c7cb8abeb3bfc8757b9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:08.495779 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-f1ef8d20-52f6-4de2-b868-597e4e2c38ec tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] 10.222.0.16 "GET /compute/v2.1/servers/59a4c34e-3184-465b-b14c-0ad55192c8ce" status: 200 len: 1374 microversion: 2.1 time: 0.306382 Jul 16 21:29:08.496595 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2164/4316] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:29:08 2022] GET /compute/v2.1/servers/59a4c34e-3184-465b-b14c-0ad55192c8ce => generated 1374 bytes in 308 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:29:08.551569 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-2ee0c446-8036-41f5-9f15-9a0b6bba979c tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:29:08.552362 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-2ee0c446-8036-41f5-9f15-9a0b6bba979c tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] [instance: fed8328c-f9d3-4f4a-97c9-4919a94e6106] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:29:08.563121 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2ee0c446-8036-41f5-9f15-9a0b6bba979c tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:08.563710 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2ee0c446-8036-41f5-9f15-9a0b6bba979c tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:08.564733 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2ee0c446-8036-41f5-9f15-9a0b6bba979c tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:08.566184 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2ee0c446-8036-41f5-9f15-9a0b6bba979c tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:08.720047 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-2ee0c446-8036-41f5-9f15-9a0b6bba979c tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'af0cd3652c8d42bcb8af08852ff4f203', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c6f089da37e4ff09784a2b42513ff36', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:08.732635 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-2ee0c446-8036-41f5-9f15-9a0b6bba979c tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'af0cd3652c8d42bcb8af08852ff4f203', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c6f089da37e4ff09784a2b42513ff36', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:08.735828 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-2ee0c446-8036-41f5-9f15-9a0b6bba979c tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'af0cd3652c8d42bcb8af08852ff4f203', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c6f089da37e4ff09784a2b42513ff36', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:08.738348 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-2ee0c446-8036-41f5-9f15-9a0b6bba979c tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] 10.222.0.16 "GET /compute/v2.1/servers/fed8328c-f9d3-4f4a-97c9-4919a94e6106" status: 200 len: 1365 microversion: 2.57 time: 0.191572 Jul 16 21:29:08.740069 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2153/4317] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:29:08 2022] GET /compute/v2.1/servers/fed8328c-f9d3-4f4a-97c9-4919a94e6106 => generated 1365 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:29:09.003171 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-0323f01f-fdf9-4d0c-a895-d5f3290c8d22 tempest-VolumesTestJSON-1603190450 tempest-VolumesTestJSON-1603190450-project] Calling method '.version_select at 0x7fdd9d090430>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:29:09.155100 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-0323f01f-fdf9-4d0c-a895-d5f3290c8d22 tempest-VolumesTestJSON-1603190450 tempest-VolumesTestJSON-1603190450-project] 10.222.0.16 "GET /compute/v2.1/os-volumes/60002b45-60de-4193-9cc8-c089d98ce7ba" status: 200 len: 343 microversion: 2.1 time: 0.155859 Jul 16 21:29:09.155691 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2165/4318] 10.222.0.16 () {58 vars in 1244 bytes} [Sat Jul 16 21:29:08 2022] GET /compute/v2.1/os-volumes/60002b45-60de-4193-9cc8-c089d98ce7ba => generated 343 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:29:09.514915 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-537dcc86-840f-4ae1-9973-714c835ba93e tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:29:09.515579 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-537dcc86-840f-4ae1-9973-714c835ba93e tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] [instance: 59a4c34e-3184-465b-b14c-0ad55192c8ce] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:29:09.534275 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-537dcc86-840f-4ae1-9973-714c835ba93e tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:09.535160 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-537dcc86-840f-4ae1-9973-714c835ba93e tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:09.535995 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-537dcc86-840f-4ae1-9973-714c835ba93e tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:09.536439 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-537dcc86-840f-4ae1-9973-714c835ba93e tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:09.758504 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-c02902ba-9161-4a80-af31-03187440f4fa tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:29:09.759912 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-c02902ba-9161-4a80-af31-03187440f4fa tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] [instance: fed8328c-f9d3-4f4a-97c9-4919a94e6106] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:29:09.772881 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c02902ba-9161-4a80-af31-03187440f4fa tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:09.773284 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c02902ba-9161-4a80-af31-03187440f4fa tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:09.782514 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c02902ba-9161-4a80-af31-03187440f4fa tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:09.783054 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c02902ba-9161-4a80-af31-03187440f4fa tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:09.811119 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-537dcc86-840f-4ae1-9973-714c835ba93e tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ddfae3448704591b662de83b97a1876', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bed7c94624484c7cb8abeb3bfc8757b9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:09.818403 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-537dcc86-840f-4ae1-9973-714c835ba93e tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] 10.222.0.16 "GET /compute/v2.1/servers/59a4c34e-3184-465b-b14c-0ad55192c8ce" status: 200 len: 1374 microversion: 2.1 time: 0.308543 Jul 16 21:29:09.836425 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2154/4319] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:29:09 2022] GET /compute/v2.1/servers/59a4c34e-3184-465b-b14c-0ad55192c8ce => generated 1374 bytes in 327 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:29:09.920758 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-c02902ba-9161-4a80-af31-03187440f4fa tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'af0cd3652c8d42bcb8af08852ff4f203', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c6f089da37e4ff09784a2b42513ff36', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:09.959224 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-c02902ba-9161-4a80-af31-03187440f4fa tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'af0cd3652c8d42bcb8af08852ff4f203', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c6f089da37e4ff09784a2b42513ff36', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:09.961208 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-c02902ba-9161-4a80-af31-03187440f4fa tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'af0cd3652c8d42bcb8af08852ff4f203', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c6f089da37e4ff09784a2b42513ff36', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:09.963603 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-c02902ba-9161-4a80-af31-03187440f4fa tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] 10.222.0.16 "GET /compute/v2.1/servers/fed8328c-f9d3-4f4a-97c9-4919a94e6106" status: 200 len: 1365 microversion: 2.57 time: 0.209604 Jul 16 21:29:09.964686 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2166/4320] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:29:09 2022] GET /compute/v2.1/servers/fed8328c-f9d3-4f4a-97c9-4919a94e6106 => generated 1365 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:29:10.844811 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-0b2900c3-59bb-46d4-9942-e75a26a47473 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:29:10.845465 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-0b2900c3-59bb-46d4-9942-e75a26a47473 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] [instance: 59a4c34e-3184-465b-b14c-0ad55192c8ce] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:29:10.855538 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0b2900c3-59bb-46d4-9942-e75a26a47473 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:10.855914 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0b2900c3-59bb-46d4-9942-e75a26a47473 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:10.857046 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0b2900c3-59bb-46d4-9942-e75a26a47473 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:10.857791 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0b2900c3-59bb-46d4-9942-e75a26a47473 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:10.886272 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:29:10.999692 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-814eee37-2f8b-4f7a-9220-0d42565a6306 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:29:11.000979 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-814eee37-2f8b-4f7a-9220-0d42565a6306 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] [instance: fed8328c-f9d3-4f4a-97c9-4919a94e6106] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:29:11.019429 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-814eee37-2f8b-4f7a-9220-0d42565a6306 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:11.019844 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-814eee37-2f8b-4f7a-9220-0d42565a6306 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:11.027055 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-814eee37-2f8b-4f7a-9220-0d42565a6306 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:11.029660 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-814eee37-2f8b-4f7a-9220-0d42565a6306 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:11.083786 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-0b2900c3-59bb-46d4-9942-e75a26a47473 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ddfae3448704591b662de83b97a1876', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bed7c94624484c7cb8abeb3bfc8757b9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:11.098683 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-0b2900c3-59bb-46d4-9942-e75a26a47473 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] 10.222.0.16 "GET /compute/v2.1/servers/59a4c34e-3184-465b-b14c-0ad55192c8ce" status: 200 len: 1374 microversion: 2.1 time: 0.258739 Jul 16 21:29:11.109136 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2155/4321] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:29:10 2022] GET /compute/v2.1/servers/59a4c34e-3184-465b-b14c-0ad55192c8ce => generated 1374 bytes in 270 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:29:11.222137 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-814eee37-2f8b-4f7a-9220-0d42565a6306 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'af0cd3652c8d42bcb8af08852ff4f203', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c6f089da37e4ff09784a2b42513ff36', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:11.232121 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-814eee37-2f8b-4f7a-9220-0d42565a6306 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'af0cd3652c8d42bcb8af08852ff4f203', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c6f089da37e4ff09784a2b42513ff36', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:11.234245 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-814eee37-2f8b-4f7a-9220-0d42565a6306 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'af0cd3652c8d42bcb8af08852ff4f203', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c6f089da37e4ff09784a2b42513ff36', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:11.235761 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-814eee37-2f8b-4f7a-9220-0d42565a6306 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] 10.222.0.16 "GET /compute/v2.1/servers/fed8328c-f9d3-4f4a-97c9-4919a94e6106" status: 200 len: 1365 microversion: 2.57 time: 0.251486 Jul 16 21:29:11.236839 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2167/4322] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:29:10 2022] GET /compute/v2.1/servers/fed8328c-f9d3-4f4a-97c9-4919a94e6106 => generated 1365 bytes in 257 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:29:12.119395 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-1edeb444-09cc-403f-bcb6-4cf3b5195fad tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:29:12.120602 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-1edeb444-09cc-403f-bcb6-4cf3b5195fad tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] [instance: 59a4c34e-3184-465b-b14c-0ad55192c8ce] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:29:12.138690 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1edeb444-09cc-403f-bcb6-4cf3b5195fad tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:12.139740 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1edeb444-09cc-403f-bcb6-4cf3b5195fad tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:12.144033 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1edeb444-09cc-403f-bcb6-4cf3b5195fad tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:12.144894 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1edeb444-09cc-403f-bcb6-4cf3b5195fad tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:12.261394 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-9e916ff6-bc80-41c3-9e85-287f03231717 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:29:12.262411 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-9e916ff6-bc80-41c3-9e85-287f03231717 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] [instance: fed8328c-f9d3-4f4a-97c9-4919a94e6106] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:29:12.274006 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9e916ff6-bc80-41c3-9e85-287f03231717 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:12.274613 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9e916ff6-bc80-41c3-9e85-287f03231717 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:12.275889 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9e916ff6-bc80-41c3-9e85-287f03231717 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:12.276767 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9e916ff6-bc80-41c3-9e85-287f03231717 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:12.457772 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-9e916ff6-bc80-41c3-9e85-287f03231717 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'af0cd3652c8d42bcb8af08852ff4f203', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c6f089da37e4ff09784a2b42513ff36', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:12.472673 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-9e916ff6-bc80-41c3-9e85-287f03231717 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'af0cd3652c8d42bcb8af08852ff4f203', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c6f089da37e4ff09784a2b42513ff36', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:12.475412 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-9e916ff6-bc80-41c3-9e85-287f03231717 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'af0cd3652c8d42bcb8af08852ff4f203', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c6f089da37e4ff09784a2b42513ff36', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:12.477034 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-9e916ff6-bc80-41c3-9e85-287f03231717 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] 10.222.0.16 "GET /compute/v2.1/servers/fed8328c-f9d3-4f4a-97c9-4919a94e6106" status: 200 len: 1365 microversion: 2.57 time: 0.223289 Jul 16 21:29:12.478311 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2168/4323] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:29:12 2022] GET /compute/v2.1/servers/fed8328c-f9d3-4f4a-97c9-4919a94e6106 => generated 1365 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:29:12.533931 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-1edeb444-09cc-403f-bcb6-4cf3b5195fad tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ddfae3448704591b662de83b97a1876', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bed7c94624484c7cb8abeb3bfc8757b9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:12.543312 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-1edeb444-09cc-403f-bcb6-4cf3b5195fad tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] 10.222.0.16 "GET /compute/v2.1/servers/59a4c34e-3184-465b-b14c-0ad55192c8ce" status: 200 len: 1374 microversion: 2.1 time: 0.428490 Jul 16 21:29:12.549020 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2156/4324] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:29:12 2022] GET /compute/v2.1/servers/59a4c34e-3184-465b-b14c-0ad55192c8ce => generated 1374 bytes in 433 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:29:13.256602 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-8f0954a9-281c-4c8b-aa43-79351ae736e5 tempest-VolumesTestJSON-1603190450 tempest-VolumesTestJSON-1603190450-project] Calling method '.version_select at 0x7fdd9cf97b80>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:29:13.305542 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-8f0954a9-281c-4c8b-aa43-79351ae736e5 tempest-VolumesTestJSON-1603190450 tempest-VolumesTestJSON-1603190450-project] 10.222.0.16 "GET /compute/v2.1/os-volumes/3a0b5ef1-5d7b-48c3-bb2d-db5fc3f57aad" status: 200 len: 343 microversion: 2.1 time: 0.052712 Jul 16 21:29:13.306295 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2169/4325] 10.222.0.16 () {58 vars in 1244 bytes} [Sat Jul 16 21:29:13 2022] GET /compute/v2.1/os-volumes/3a0b5ef1-5d7b-48c3-bb2d-db5fc3f57aad => generated 343 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:29:13.326400 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-d14155f8-990b-44de-a50c-250945da9e19 tempest-VolumesTestJSON-1603190450 tempest-VolumesTestJSON-1603190450-project] Calling method '.version_select at 0x7fdda2b391f0>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:29:13.421519 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-d14155f8-990b-44de-a50c-250945da9e19 tempest-VolumesTestJSON-1603190450 tempest-VolumesTestJSON-1603190450-project] 10.222.0.16 "GET /compute/v2.1/os-volumes" status: 200 len: 1012 microversion: 2.1 time: 0.106999 Jul 16 21:29:13.422204 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2157/4326] 10.222.0.16 () {58 vars in 1133 bytes} [Sat Jul 16 21:29:13 2022] GET /compute/v2.1/os-volumes => generated 1012 bytes in 108 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:29:13.435254 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-a6588e8b-7e7a-4c55-936d-2da128a1fcd8 tempest-VolumesTestJSON-1603190450 tempest-VolumesTestJSON-1603190450-project] Calling method '.version_select at 0x7fdd9d086790>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:29:13.498709 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-917d4781-345f-4b50-9984-e5f52fcb3def tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:29:13.498709 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-917d4781-345f-4b50-9984-e5f52fcb3def tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] [instance: fed8328c-f9d3-4f4a-97c9-4919a94e6106] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:29:13.504677 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-917d4781-345f-4b50-9984-e5f52fcb3def tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:13.505600 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-917d4781-345f-4b50-9984-e5f52fcb3def tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:13.506476 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-917d4781-345f-4b50-9984-e5f52fcb3def tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:13.506995 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-917d4781-345f-4b50-9984-e5f52fcb3def tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:13.550538 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-a6588e8b-7e7a-4c55-936d-2da128a1fcd8 tempest-VolumesTestJSON-1603190450 tempest-VolumesTestJSON-1603190450-project] 10.222.0.16 "GET /compute/v2.1/os-volumes?limit=2" status: 200 len: 679 microversion: 2.1 time: 0.114120 Jul 16 21:29:13.551309 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2170/4327] 10.222.0.16 () {58 vars in 1148 bytes} [Sat Jul 16 21:29:13 2022] GET /compute/v2.1/os-volumes?limit=2 => generated 679 bytes in 121 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 16 21:29:13.562831 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-714ec135-4928-4081-bd66-27a27815c4dd tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:29:13.563881 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-714ec135-4928-4081-bd66-27a27815c4dd tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] [instance: 59a4c34e-3184-465b-b14c-0ad55192c8ce] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:29:13.570677 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-714ec135-4928-4081-bd66-27a27815c4dd tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:13.571573 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-714ec135-4928-4081-bd66-27a27815c4dd tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:13.572653 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-714ec135-4928-4081-bd66-27a27815c4dd tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:13.573619 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-714ec135-4928-4081-bd66-27a27815c4dd tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:13.603915 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-917d4781-345f-4b50-9984-e5f52fcb3def tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'af0cd3652c8d42bcb8af08852ff4f203', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c6f089da37e4ff09784a2b42513ff36', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:13.614178 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-917d4781-345f-4b50-9984-e5f52fcb3def tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'af0cd3652c8d42bcb8af08852ff4f203', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c6f089da37e4ff09784a2b42513ff36', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:13.615894 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-917d4781-345f-4b50-9984-e5f52fcb3def tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'af0cd3652c8d42bcb8af08852ff4f203', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c6f089da37e4ff09784a2b42513ff36', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:13.617098 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-917d4781-345f-4b50-9984-e5f52fcb3def tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] 10.222.0.16 "GET /compute/v2.1/servers/fed8328c-f9d3-4f4a-97c9-4919a94e6106" status: 200 len: 1365 microversion: 2.57 time: 0.125678 Jul 16 21:29:13.627604 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2158/4328] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:29:13 2022] GET /compute/v2.1/servers/fed8328c-f9d3-4f4a-97c9-4919a94e6106 => generated 1365 bytes in 127 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 16 21:29:13.638286 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-0134be2c-aeee-43ff-9904-0c71adc5603a tempest-VolumesTestJSON-1603190450 tempest-VolumesTestJSON-1603190450-project] Calling method '.version_select at 0x7fdd9d03e550>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:29:13.790478 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-0134be2c-aeee-43ff-9904-0c71adc5603a tempest-VolumesTestJSON-1603190450 tempest-VolumesTestJSON-1603190450-project] 10.222.0.16 "GET /compute/v2.1/os-volumes" status: 200 len: 1012 microversion: 2.1 time: 0.170761 Jul 16 21:29:13.791093 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2159/4329] 10.222.0.16 () {58 vars in 1133 bytes} [Sat Jul 16 21:29:13 2022] GET /compute/v2.1/os-volumes => generated 1012 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:29:13.803674 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-1094a8ce-6d0d-49a3-a307-0f6be5791ce6 tempest-VolumesTestJSON-1603190450 tempest-VolumesTestJSON-1603190450-project] Calling method '.version_select at 0x7fdd9cc3fee0>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:29:13.894779 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-714ec135-4928-4081-bd66-27a27815c4dd tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ddfae3448704591b662de83b97a1876', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bed7c94624484c7cb8abeb3bfc8757b9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:13.906983 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-714ec135-4928-4081-bd66-27a27815c4dd tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] 10.222.0.16 "GET /compute/v2.1/servers/59a4c34e-3184-465b-b14c-0ad55192c8ce" status: 200 len: 1374 microversion: 2.1 time: 0.348193 Jul 16 21:29:13.907603 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2171/4330] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:29:13 2022] GET /compute/v2.1/servers/59a4c34e-3184-465b-b14c-0ad55192c8ce => generated 1374 bytes in 350 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:29:13.910260 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-1094a8ce-6d0d-49a3-a307-0f6be5791ce6 tempest-VolumesTestJSON-1603190450 tempest-VolumesTestJSON-1603190450-project] 10.222.0.16 "GET /compute/v2.1/os-volumes?offset=1&limit=1" status: 200 len: 346 microversion: 2.1 time: 0.109253 Jul 16 21:29:13.910976 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2160/4331] 10.222.0.16 () {58 vars in 1166 bytes} [Sat Jul 16 21:29:13 2022] GET /compute/v2.1/os-volumes?offset=1&limit=1 => generated 346 bytes in 113 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 16 21:29:13.923023 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-3e621446-5f44-4524-a574-bf1959fd5481 tempest-VolumesTestJSON-1603190450 tempest-VolumesTestJSON-1603190450-project] Calling method '.version_select at 0x7fdd9d186310>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:29:14.020889 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-3e621446-5f44-4524-a574-bf1959fd5481 tempest-VolumesTestJSON-1603190450 tempest-VolumesTestJSON-1603190450-project] 10.222.0.16 "GET /compute/v2.1/os-volumes/detail?limit=2" status: 200 len: 679 microversion: 2.1 time: 0.102001 Jul 16 21:29:14.021569 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2172/4332] 10.222.0.16 () {58 vars in 1169 bytes} [Sat Jul 16 21:29:13 2022] GET /compute/v2.1/os-volumes/detail?limit=2 => generated 679 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:29:14.038453 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-bd1c151a-02db-4ac5-9fc7-3f8e7d178814 tempest-VolumesTestJSON-1603190450 tempest-VolumesTestJSON-1603190450-project] Calling method '.version_select at 0x7fdd9d3adaf0>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:29:14.125022 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-bd1c151a-02db-4ac5-9fc7-3f8e7d178814 tempest-VolumesTestJSON-1603190450 tempest-VolumesTestJSON-1603190450-project] 10.222.0.16 "GET /compute/v2.1/os-volumes/detail" status: 200 len: 1012 microversion: 2.1 time: 0.090635 Jul 16 21:29:14.126005 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2161/4333] 10.222.0.16 () {58 vars in 1154 bytes} [Sat Jul 16 21:29:14 2022] GET /compute/v2.1/os-volumes/detail => generated 1012 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:29:14.137451 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-72f4448e-9f61-4bb8-8c89-60764b3d7727 tempest-VolumesTestJSON-1603190450 tempest-VolumesTestJSON-1603190450-project] Calling method '.version_select at 0x7fdd9cf97d30>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:29:14.235973 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-72f4448e-9f61-4bb8-8c89-60764b3d7727 tempest-VolumesTestJSON-1603190450 tempest-VolumesTestJSON-1603190450-project] 10.222.0.16 "GET /compute/v2.1/os-volumes/detail?offset=1&limit=1" status: 200 len: 346 microversion: 2.1 time: 0.102373 Jul 16 21:29:14.236982 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2173/4334] 10.222.0.16 () {58 vars in 1187 bytes} [Sat Jul 16 21:29:14 2022] GET /compute/v2.1/os-volumes/detail?offset=1&limit=1 => generated 346 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:29:14.251654 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-2ae46eba-8e8a-49d2-a899-c3a0bcf772cd tempest-VolumesTestJSON-1603190450 tempest-VolumesTestJSON-1603190450-project] Calling method '.version_select at 0x7fdd9d3ad670>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:29:14.354472 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-2ae46eba-8e8a-49d2-a899-c3a0bcf772cd tempest-VolumesTestJSON-1603190450 tempest-VolumesTestJSON-1603190450-project] 10.222.0.16 "GET /compute/v2.1/os-volumes/detail" status: 200 len: 1012 microversion: 2.1 time: 0.108024 Jul 16 21:29:14.355137 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2162/4335] 10.222.0.16 () {58 vars in 1154 bytes} [Sat Jul 16 21:29:14 2022] GET /compute/v2.1/os-volumes/detail => generated 1012 bytes in 110 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:29:14.617029 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-a2245b08-759a-4985-8f7c-c3bf9a147cba req-1b3f5f3e-9ee8-4821-9794-180180cacf58 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "59a4c34e-3184-465b-b14c-0ad55192c8ce", "name": "network-vif-plugged", "status": "completed", "tag": "cad17319-3e42-44f8-911a-cd258470adec"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:29:14.646939 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-c3aa2947-cb5a-402a-837e-5a57b3e0109c tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:29:14.647593 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-c3aa2947-cb5a-402a-837e-5a57b3e0109c tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] [instance: fed8328c-f9d3-4f4a-97c9-4919a94e6106] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:29:14.654515 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-a2245b08-759a-4985-8f7c-c3bf9a147cba req-1b3f5f3e-9ee8-4821-9794-180180cacf58 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:14.654893 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-a2245b08-759a-4985-8f7c-c3bf9a147cba req-1b3f5f3e-9ee8-4821-9794-180180cacf58 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:14.671315 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c3aa2947-cb5a-402a-837e-5a57b3e0109c tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:14.671315 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c3aa2947-cb5a-402a-837e-5a57b3e0109c tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:14.678266 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c3aa2947-cb5a-402a-837e-5a57b3e0109c tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:14.678768 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c3aa2947-cb5a-402a-837e-5a57b3e0109c tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:14.696863 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-a2245b08-759a-4985-8f7c-c3bf9a147cba req-1b3f5f3e-9ee8-4821-9794-180180cacf58 service nova] Creating event network-vif-plugged:cad17319-3e42-44f8-911a-cd258470adec for instance 59a4c34e-3184-465b-b14c-0ad55192c8ce on nch2-master-1 Jul 16 21:29:14.713719 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-a2245b08-759a-4985-8f7c-c3bf9a147cba req-1b3f5f3e-9ee8-4821-9794-180180cacf58 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.276478 Jul 16 21:29:14.714546 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2174/4336] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:29:14 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 281 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 16 21:29:14.778825 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-c3aa2947-cb5a-402a-837e-5a57b3e0109c tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'af0cd3652c8d42bcb8af08852ff4f203', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c6f089da37e4ff09784a2b42513ff36', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:14.793434 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-c3aa2947-cb5a-402a-837e-5a57b3e0109c tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'af0cd3652c8d42bcb8af08852ff4f203', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c6f089da37e4ff09784a2b42513ff36', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:14.795514 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-c3aa2947-cb5a-402a-837e-5a57b3e0109c tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'af0cd3652c8d42bcb8af08852ff4f203', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c6f089da37e4ff09784a2b42513ff36', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:14.798689 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-c3aa2947-cb5a-402a-837e-5a57b3e0109c tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] 10.222.0.16 "GET /compute/v2.1/servers/fed8328c-f9d3-4f4a-97c9-4919a94e6106" status: 200 len: 1365 microversion: 2.57 time: 0.165775 Jul 16 21:29:14.800468 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2163/4337] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:29:14 2022] GET /compute/v2.1/servers/fed8328c-f9d3-4f4a-97c9-4919a94e6106 => generated 1365 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:29:14.926739 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-a3ba7688-bafb-4d2f-a89e-6669f4b93f11 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:29:14.927511 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-a3ba7688-bafb-4d2f-a89e-6669f4b93f11 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] [instance: 59a4c34e-3184-465b-b14c-0ad55192c8ce] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:29:14.935549 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a3ba7688-bafb-4d2f-a89e-6669f4b93f11 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:14.936253 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a3ba7688-bafb-4d2f-a89e-6669f4b93f11 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:14.937475 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a3ba7688-bafb-4d2f-a89e-6669f4b93f11 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:14.937990 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a3ba7688-bafb-4d2f-a89e-6669f4b93f11 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:15.211652 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-a3ba7688-bafb-4d2f-a89e-6669f4b93f11 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ddfae3448704591b662de83b97a1876', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bed7c94624484c7cb8abeb3bfc8757b9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:15.225994 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-a3ba7688-bafb-4d2f-a89e-6669f4b93f11 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] 10.222.0.16 "GET /compute/v2.1/servers/59a4c34e-3184-465b-b14c-0ad55192c8ce" status: 200 len: 1374 microversion: 2.1 time: 0.303804 Jul 16 21:29:15.231074 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2175/4338] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:29:14 2022] GET /compute/v2.1/servers/59a4c34e-3184-465b-b14c-0ad55192c8ce => generated 1374 bytes in 306 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:29:15.663961 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-407225f7-43fa-49f2-9620-1e1745454df9 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:29:15.664535 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-407225f7-43fa-49f2-9620-1e1745454df9 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] [instance: 08a33911-6e47-45fd-b499-155f199aacf8] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:29:15.671383 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-407225f7-43fa-49f2-9620-1e1745454df9 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:15.671845 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-407225f7-43fa-49f2-9620-1e1745454df9 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:15.672628 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-407225f7-43fa-49f2-9620-1e1745454df9 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:15.673075 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-407225f7-43fa-49f2-9620-1e1745454df9 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:15.699472 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-407225f7-43fa-49f2-9620-1e1745454df9 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] 10.222.0.16 "GET /compute/v2.1/servers/08a33911-6e47-45fd-b499-155f199aacf8/ips" status: 200 len: 153 microversion: 2.20 time: 0.039487 Jul 16 21:29:15.700636 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2164/4339] 10.222.0.16 () {60 vars in 1288 bytes} [Sat Jul 16 21:29:15 2022] GET /compute/v2.1/servers/08a33911-6e47-45fd-b499-155f199aacf8/ips => generated 153 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 16 21:29:15.819314 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-6eb84a1f-c1bd-4c03-a272-3468a3322daf tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:29:15.819931 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-6eb84a1f-c1bd-4c03-a272-3468a3322daf tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] [instance: fed8328c-f9d3-4f4a-97c9-4919a94e6106] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:29:15.840990 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6eb84a1f-c1bd-4c03-a272-3468a3322daf tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:15.841587 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6eb84a1f-c1bd-4c03-a272-3468a3322daf tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:15.842387 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6eb84a1f-c1bd-4c03-a272-3468a3322daf tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:15.842831 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6eb84a1f-c1bd-4c03-a272-3468a3322daf tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:15.983837 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-6eb84a1f-c1bd-4c03-a272-3468a3322daf tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'af0cd3652c8d42bcb8af08852ff4f203', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c6f089da37e4ff09784a2b42513ff36', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:15.991820 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=100 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:29:15.996252 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-6eb84a1f-c1bd-4c03-a272-3468a3322daf tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'af0cd3652c8d42bcb8af08852ff4f203', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c6f089da37e4ff09784a2b42513ff36', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:15.998900 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-6eb84a1f-c1bd-4c03-a272-3468a3322daf tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'af0cd3652c8d42bcb8af08852ff4f203', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c6f089da37e4ff09784a2b42513ff36', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:16.000491 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-6eb84a1f-c1bd-4c03-a272-3468a3322daf tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] 10.222.0.16 "GET /compute/v2.1/servers/fed8328c-f9d3-4f4a-97c9-4919a94e6106" status: 200 len: 1365 microversion: 2.57 time: 0.185588 Jul 16 21:29:16.001703 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2176/4340] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:29:15 2022] GET /compute/v2.1/servers/fed8328c-f9d3-4f4a-97c9-4919a94e6106 => generated 1365 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:29:16.244387 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-743e1a32-33bc-4351-b74b-dc40fdecac54 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:29:16.245018 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-743e1a32-33bc-4351-b74b-dc40fdecac54 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] [instance: 59a4c34e-3184-465b-b14c-0ad55192c8ce] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:29:16.257877 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-743e1a32-33bc-4351-b74b-dc40fdecac54 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:16.258552 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-743e1a32-33bc-4351-b74b-dc40fdecac54 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:16.261816 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-743e1a32-33bc-4351-b74b-dc40fdecac54 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:16.262804 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-743e1a32-33bc-4351-b74b-dc40fdecac54 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:16.447373 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-743e1a32-33bc-4351-b74b-dc40fdecac54 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ddfae3448704591b662de83b97a1876', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bed7c94624484c7cb8abeb3bfc8757b9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:16.456478 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-743e1a32-33bc-4351-b74b-dc40fdecac54 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] 10.222.0.16 "GET /compute/v2.1/servers/59a4c34e-3184-465b-b14c-0ad55192c8ce" status: 200 len: 1374 microversion: 2.1 time: 0.217241 Jul 16 21:29:16.457071 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2165/4341] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:29:16 2022] GET /compute/v2.1/servers/59a4c34e-3184-465b-b14c-0ad55192c8ce => generated 1374 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:29:17.019747 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-9fa374c1-eac4-49a7-a690-94f44c0bdb94 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:29:17.020364 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-9fa374c1-eac4-49a7-a690-94f44c0bdb94 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] [instance: fed8328c-f9d3-4f4a-97c9-4919a94e6106] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:29:17.028069 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9fa374c1-eac4-49a7-a690-94f44c0bdb94 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:17.028737 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9fa374c1-eac4-49a7-a690-94f44c0bdb94 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:17.029713 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9fa374c1-eac4-49a7-a690-94f44c0bdb94 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:17.030330 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9fa374c1-eac4-49a7-a690-94f44c0bdb94 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:17.130480 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-9fa374c1-eac4-49a7-a690-94f44c0bdb94 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'af0cd3652c8d42bcb8af08852ff4f203', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c6f089da37e4ff09784a2b42513ff36', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:17.145173 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-9fa374c1-eac4-49a7-a690-94f44c0bdb94 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'af0cd3652c8d42bcb8af08852ff4f203', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c6f089da37e4ff09784a2b42513ff36', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:17.146770 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-9fa374c1-eac4-49a7-a690-94f44c0bdb94 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'af0cd3652c8d42bcb8af08852ff4f203', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c6f089da37e4ff09784a2b42513ff36', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:17.148324 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-9fa374c1-eac4-49a7-a690-94f44c0bdb94 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] 10.222.0.16 "GET /compute/v2.1/servers/fed8328c-f9d3-4f4a-97c9-4919a94e6106" status: 200 len: 1365 microversion: 2.57 time: 0.132550 Jul 16 21:29:17.165388 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2177/4342] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:29:17 2022] GET /compute/v2.1/servers/fed8328c-f9d3-4f4a-97c9-4919a94e6106 => generated 1365 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:29:17.474031 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-ccfb31a9-172e-443e-89bc-a2324362cbb4 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:29:17.474848 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-ccfb31a9-172e-443e-89bc-a2324362cbb4 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] [instance: 59a4c34e-3184-465b-b14c-0ad55192c8ce] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:29:17.477929 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_api:SELECT=92,nova_api:UPDATE=1,nova_api:INSERT=3 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:29:17.484885 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ccfb31a9-172e-443e-89bc-a2324362cbb4 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:17.485444 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ccfb31a9-172e-443e-89bc-a2324362cbb4 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:17.486581 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ccfb31a9-172e-443e-89bc-a2324362cbb4 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:17.487213 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ccfb31a9-172e-443e-89bc-a2324362cbb4 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:17.668103 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-ccfb31a9-172e-443e-89bc-a2324362cbb4 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ddfae3448704591b662de83b97a1876', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bed7c94624484c7cb8abeb3bfc8757b9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:17.711066 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-ccfb31a9-172e-443e-89bc-a2324362cbb4 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] 10.222.0.16 "GET /compute/v2.1/servers/59a4c34e-3184-465b-b14c-0ad55192c8ce" status: 200 len: 1546 microversion: 2.1 time: 0.240257 Jul 16 21:29:17.735043 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-4e2961e1-c8ea-4172-bd1e-493edc04ae81 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:29:17.736789 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2166/4343] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:29:17 2022] GET /compute/v2.1/servers/59a4c34e-3184-465b-b14c-0ad55192c8ce => generated 1546 bytes in 267 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:29:17.739692 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-4e2961e1-c8ea-4172-bd1e-493edc04ae81 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] [instance: 59a4c34e-3184-465b-b14c-0ad55192c8ce] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:29:17.760716 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4e2961e1-c8ea-4172-bd1e-493edc04ae81 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:17.761176 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4e2961e1-c8ea-4172-bd1e-493edc04ae81 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:17.761972 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4e2961e1-c8ea-4172-bd1e-493edc04ae81 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:17.762974 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4e2961e1-c8ea-4172-bd1e-493edc04ae81 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:18.069203 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-4e2961e1-c8ea-4172-bd1e-493edc04ae81 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ddfae3448704591b662de83b97a1876', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bed7c94624484c7cb8abeb3bfc8757b9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:18.078247 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-4e2961e1-c8ea-4172-bd1e-493edc04ae81 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] 10.222.0.16 "GET /compute/v2.1/servers/59a4c34e-3184-465b-b14c-0ad55192c8ce" status: 200 len: 1546 microversion: 2.1 time: 0.347724 Jul 16 21:29:18.090903 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2178/4344] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:29:17 2022] GET /compute/v2.1/servers/59a4c34e-3184-465b-b14c-0ad55192c8ce => generated 1546 bytes in 361 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:29:18.171633 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-39d0c604-330a-4f74-8190-902023aa3b37 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:29:18.172549 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-39d0c604-330a-4f74-8190-902023aa3b37 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] [instance: fed8328c-f9d3-4f4a-97c9-4919a94e6106] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:29:18.187971 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-39d0c604-330a-4f74-8190-902023aa3b37 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:18.190346 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-39d0c604-330a-4f74-8190-902023aa3b37 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:18.191422 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-39d0c604-330a-4f74-8190-902023aa3b37 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:18.193676 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-39d0c604-330a-4f74-8190-902023aa3b37 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:18.347806 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-39d0c604-330a-4f74-8190-902023aa3b37 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'af0cd3652c8d42bcb8af08852ff4f203', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c6f089da37e4ff09784a2b42513ff36', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:18.366494 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-39d0c604-330a-4f74-8190-902023aa3b37 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'af0cd3652c8d42bcb8af08852ff4f203', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c6f089da37e4ff09784a2b42513ff36', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:18.368258 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-39d0c604-330a-4f74-8190-902023aa3b37 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'af0cd3652c8d42bcb8af08852ff4f203', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c6f089da37e4ff09784a2b42513ff36', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:18.369474 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-39d0c604-330a-4f74-8190-902023aa3b37 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] 10.222.0.16 "GET /compute/v2.1/servers/fed8328c-f9d3-4f4a-97c9-4919a94e6106" status: 200 len: 1365 microversion: 2.57 time: 0.205521 Jul 16 21:29:18.383537 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2167/4345] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:29:18 2022] GET /compute/v2.1/servers/fed8328c-f9d3-4f4a-97c9-4919a94e6106 => generated 1365 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:29:19.390290 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-943306b9-9a67-4f6d-9a39-7d2785c8d27b tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:29:19.390923 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-943306b9-9a67-4f6d-9a39-7d2785c8d27b tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] [instance: fed8328c-f9d3-4f4a-97c9-4919a94e6106] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:29:19.409233 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-943306b9-9a67-4f6d-9a39-7d2785c8d27b tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:19.410298 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-943306b9-9a67-4f6d-9a39-7d2785c8d27b tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:19.413376 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-943306b9-9a67-4f6d-9a39-7d2785c8d27b tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:19.414009 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-943306b9-9a67-4f6d-9a39-7d2785c8d27b tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:19.573033 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-943306b9-9a67-4f6d-9a39-7d2785c8d27b tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'af0cd3652c8d42bcb8af08852ff4f203', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c6f089da37e4ff09784a2b42513ff36', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:19.586975 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-943306b9-9a67-4f6d-9a39-7d2785c8d27b tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'af0cd3652c8d42bcb8af08852ff4f203', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c6f089da37e4ff09784a2b42513ff36', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:19.591122 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-943306b9-9a67-4f6d-9a39-7d2785c8d27b tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'af0cd3652c8d42bcb8af08852ff4f203', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c6f089da37e4ff09784a2b42513ff36', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:19.597987 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-943306b9-9a67-4f6d-9a39-7d2785c8d27b tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] 10.222.0.16 "GET /compute/v2.1/servers/fed8328c-f9d3-4f4a-97c9-4919a94e6106" status: 200 len: 1365 microversion: 2.57 time: 0.212636 Jul 16 21:29:19.598833 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2179/4346] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:29:19 2022] GET /compute/v2.1/servers/fed8328c-f9d3-4f4a-97c9-4919a94e6106 => generated 1365 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:29:20.352718 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-a239d28f-d3f0-4f2d-bc1d-127e8e6dcc89 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Action: 'action', calling method: >, body: {"shelve": {}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:29:20.353292 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-a239d28f-d3f0-4f2d-bc1d-127e8e6dcc89 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] [instance: 08a33911-6e47-45fd-b499-155f199aacf8] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:29:20.362769 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a239d28f-d3f0-4f2d-bc1d-127e8e6dcc89 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:20.363520 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a239d28f-d3f0-4f2d-bc1d-127e8e6dcc89 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:20.364349 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a239d28f-d3f0-4f2d-bc1d-127e8e6dcc89 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:20.364816 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a239d28f-d3f0-4f2d-bc1d-127e8e6dcc89 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:20.407814 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.objects.instance [None req-a239d28f-d3f0-4f2d-bc1d-127e8e6dcc89 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lazy-loading 'flavor' on Instance uuid 08a33911-6e47-45fd-b499-155f199aacf8 {{(pid=138680) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:29:20.620517 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-7e8ac8e9-03b5-4984-80f9-51b0bbc3cd06 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:29:20.621139 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-7e8ac8e9-03b5-4984-80f9-51b0bbc3cd06 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] [instance: fed8328c-f9d3-4f4a-97c9-4919a94e6106] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:29:20.632272 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7e8ac8e9-03b5-4984-80f9-51b0bbc3cd06 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:20.651816 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7e8ac8e9-03b5-4984-80f9-51b0bbc3cd06 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.019s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:20.652765 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7e8ac8e9-03b5-4984-80f9-51b0bbc3cd06 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:20.658779 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7e8ac8e9-03b5-4984-80f9-51b0bbc3cd06 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:20.805043 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-7e8ac8e9-03b5-4984-80f9-51b0bbc3cd06 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'af0cd3652c8d42bcb8af08852ff4f203', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c6f089da37e4ff09784a2b42513ff36', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:20.825586 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-7e8ac8e9-03b5-4984-80f9-51b0bbc3cd06 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'af0cd3652c8d42bcb8af08852ff4f203', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c6f089da37e4ff09784a2b42513ff36', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:20.826989 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-7e8ac8e9-03b5-4984-80f9-51b0bbc3cd06 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'af0cd3652c8d42bcb8af08852ff4f203', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c6f089da37e4ff09784a2b42513ff36', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:20.828228 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-7e8ac8e9-03b5-4984-80f9-51b0bbc3cd06 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] 10.222.0.16 "GET /compute/v2.1/servers/fed8328c-f9d3-4f4a-97c9-4919a94e6106" status: 200 len: 1365 microversion: 2.57 time: 0.211682 Jul 16 21:29:20.836380 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2180/4347] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:29:20 2022] GET /compute/v2.1/servers/fed8328c-f9d3-4f4a-97c9-4919a94e6106 => generated 1365 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 16 21:29:21.497882 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-a239d28f-d3f0-4f2d-bc1d-127e8e6dcc89 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] 10.222.0.16 "POST /compute/v2.1/servers/08a33911-6e47-45fd-b499-155f199aacf8/action" status: 202 len: 0 microversion: 2.20 time: 1.155833 Jul 16 21:29:21.500126 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2168/4348] 10.222.0.16 () {62 vars in 1318 bytes} [Sat Jul 16 21:29:20 2022] POST /compute/v2.1/servers/08a33911-6e47-45fd-b499-155f199aacf8/action => generated 0 bytes in 1159 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Jul 16 21:29:21.507258 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-90d5ed32-3782-484e-9afa-022ba1be444a tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:29:21.508091 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-90d5ed32-3782-484e-9afa-022ba1be444a tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] [instance: 08a33911-6e47-45fd-b499-155f199aacf8] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:29:21.518908 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-90d5ed32-3782-484e-9afa-022ba1be444a tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:21.519681 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-90d5ed32-3782-484e-9afa-022ba1be444a tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:21.523141 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-90d5ed32-3782-484e-9afa-022ba1be444a tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:21.526325 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-90d5ed32-3782-484e-9afa-022ba1be444a tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:21.865045 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-90d5ed32-3782-484e-9afa-022ba1be444a tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:21.875542 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-90d5ed32-3782-484e-9afa-022ba1be444a tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:21.877062 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-90d5ed32-3782-484e-9afa-022ba1be444a tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:21.886092 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-90d5ed32-3782-484e-9afa-022ba1be444a tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] 10.222.0.16 "GET /compute/v2.1/servers/08a33911-6e47-45fd-b499-155f199aacf8" status: 200 len: 1723 microversion: 2.20 time: 0.383000 Jul 16 21:29:21.888614 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2181/4349] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:29:21 2022] GET /compute/v2.1/servers/08a33911-6e47-45fd-b499-155f199aacf8 => generated 1723 bytes in 386 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 16 21:29:21.899661 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-83031e90-9450-44fb-bac2-f7189a759bf5 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:29:21.900742 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-83031e90-9450-44fb-bac2-f7189a759bf5 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] [instance: fed8328c-f9d3-4f4a-97c9-4919a94e6106] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:29:21.917169 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-83031e90-9450-44fb-bac2-f7189a759bf5 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:21.919144 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-83031e90-9450-44fb-bac2-f7189a759bf5 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:21.925635 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-83031e90-9450-44fb-bac2-f7189a759bf5 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:21.928216 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-83031e90-9450-44fb-bac2-f7189a759bf5 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:21.935488 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-3632da40-58c8-4370-90db-20bf32a0eb4f tempest-ServerGroup264TestJSON-1937790688 tempest-ServerGroup264TestJSON-1937790688-project] Action: 'create', calling method: .version_select at 0x7fdd9d03e3a0>, body: {"server_group": {"name": "server-group", "policy": "affinity"}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:29:21.955023 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-3632da40-58c8-4370-90db-20bf32a0eb4f tempest-ServerGroup264TestJSON-1937790688 tempest-ServerGroup264TestJSON-1937790688-project] Getting quotas for project 295deae6a1574b69985deb317a9a4973. Resources: {'server_groups'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:29:21.962280 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-3632da40-58c8-4370-90db-20bf32a0eb4f tempest-ServerGroup264TestJSON-1937790688 tempest-ServerGroup264TestJSON-1937790688-project] Getting quotas for user c88d6c32d91e4aedb4e5d1c2f6358df3 and project 295deae6a1574b69985deb317a9a4973. Resources: {'server_groups'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:29:22.110604 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-3632da40-58c8-4370-90db-20bf32a0eb4f tempest-ServerGroup264TestJSON-1937790688 tempest-ServerGroup264TestJSON-1937790688-project] Getting quotas for project 295deae6a1574b69985deb317a9a4973. Resources: {'server_groups'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:29:22.128388 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-3632da40-58c8-4370-90db-20bf32a0eb4f tempest-ServerGroup264TestJSON-1937790688 tempest-ServerGroup264TestJSON-1937790688-project] Getting quotas for user c88d6c32d91e4aedb4e5d1c2f6358df3 and project 295deae6a1574b69985deb317a9a4973. Resources: {'server_groups'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:29:22.145488 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-3632da40-58c8-4370-90db-20bf32a0eb4f tempest-ServerGroup264TestJSON-1937790688 tempest-ServerGroup264TestJSON-1937790688-project] 10.222.0.16 "POST /compute/v2.1/os-server-groups" status: 200 len: 235 microversion: 2.64 time: 0.299556 Jul 16 21:29:22.151123 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2169/4350] 10.222.0.16 () {62 vars in 1213 bytes} [Sat Jul 16 21:29:21 2022] POST /compute/v2.1/os-server-groups => generated 235 bytes in 303 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 16 21:29:22.158119 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-4019d576-b3ac-4bd6-b26d-81babf3ddfd1 tempest-ServerGroup264TestJSON-1937790688 tempest-ServerGroup264TestJSON-1937790688-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:29:22.170667 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-4019d576-b3ac-4bd6-b26d-81babf3ddfd1 tempest-ServerGroup264TestJSON-1937790688 tempest-ServerGroup264TestJSON-1937790688-project] 10.222.0.16 "GET /compute/v2.1/os-server-groups" status: 200 len: 238 microversion: 2.64 time: 0.017742 Jul 16 21:29:22.178348 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2170/4351] 10.222.0.16 () {60 vars in 1192 bytes} [Sat Jul 16 21:29:22 2022] GET /compute/v2.1/os-server-groups => generated 238 bytes in 26 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Jul 16 21:29:22.184209 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-dee95b79-870e-46c8-886b-d64dc5d176b2 tempest-ServerGroup264TestJSON-1937790688 tempest-ServerGroup264TestJSON-1937790688-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:29:22.194825 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-dee95b79-870e-46c8-886b-d64dc5d176b2 tempest-ServerGroup264TestJSON-1937790688 tempest-ServerGroup264TestJSON-1937790688-project] 10.222.0.16 "GET /compute/v2.1/os-server-groups/aa8ccb51-d5ed-41ce-8625-9b0cf5f2def6" status: 200 len: 235 microversion: 2.64 time: 0.014500 Jul 16 21:29:22.201065 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2171/4352] 10.222.0.16 () {60 vars in 1303 bytes} [Sat Jul 16 21:29:22 2022] GET /compute/v2.1/os-server-groups/aa8ccb51-d5ed-41ce-8625-9b0cf5f2def6 => generated 235 bytes in 22 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 16 21:29:22.206245 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-91c4db84-2ee2-4f31-bafa-55a2409c60b5 tempest-ServerGroup264TestJSON-1937790688 tempest-ServerGroup264TestJSON-1937790688-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:29:22.308474 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-91c4db84-2ee2-4f31-bafa-55a2409c60b5 tempest-ServerGroup264TestJSON-1937790688 tempest-ServerGroup264TestJSON-1937790688-project] 10.222.0.16 "DELETE /compute/v2.1/os-server-groups/aa8ccb51-d5ed-41ce-8625-9b0cf5f2def6" status: 204 len: 0 microversion: 2.64 time: 0.106261 Jul 16 21:29:22.311080 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2172/4353] 10.222.0.16 () {60 vars in 1306 bytes} [Sat Jul 16 21:29:22 2022] DELETE /compute/v2.1/os-server-groups/aa8ccb51-d5ed-41ce-8625-9b0cf5f2def6 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Jul 16 21:29:22.323254 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-11429cb1-caa2-4240-89d4-2e2307a8096c tempest-ServerGroup264TestJSON-1937790688 tempest-ServerGroup264TestJSON-1937790688-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:29:22.334566 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-11429cb1-caa2-4240-89d4-2e2307a8096c tempest-ServerGroup264TestJSON-1937790688 tempest-ServerGroup264TestJSON-1937790688-project] HTTP exception thrown: Instance group aa8ccb51-d5ed-41ce-8625-9b0cf5f2def6 could not be found. Jul 16 21:29:22.336740 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-11429cb1-caa2-4240-89d4-2e2307a8096c tempest-ServerGroup264TestJSON-1937790688 tempest-ServerGroup264TestJSON-1937790688-project] Request method failure captured: Jul 16 21:29:22.336740 ncdv-master-1 devstack@n-api.service[138680]: request: DELETE /compute/v2.1/os-server-groups/aa8ccb51-d5ed-41ce-8625-9b0cf5f2def6 HTTP/1.1 Jul 16 21:29:22.336740 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:29:22.336740 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:29:22.336740 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:29:22.336740 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 0 Jul 16 21:29:22.336740 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:29:22.336740 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:29:22.336740 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:29:22.336740 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:29:22.336740 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0y2xTdew4_6Wxk0IF0afVTI-Q3W6o0yI2S5mVFsZsbm69EvAaXpFmCGHAvbMWZrgnRUD0FLuzGAvvTuVi94-qJEwkY1lb3-eOR3B1Cuum6XNVla5I3ibPEFlKE2yOBEzkWhMSsxhmR3QdIkRCAzra0Lr8oKbtpiqbkKdMF5bjPOYYfw Jul 16 21:29:22.336740 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:29:22.336740 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:29:22.336740 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:29:22.336740 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:29:22.336740 ncdv-master-1 devstack@n-api.service[138680]: X-Openstack-Nova-Api-Version: 2.64 Jul 16 21:29:22.336740 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:29:22.336740 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:29:22.336740 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: 295deae6a1574b69985deb317a9a4973 Jul 16 21:29:22.336740 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-ServerGroup264TestJSON-1937790688 Jul 16 21:29:22.336740 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:29:22.336740 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:29:22.336740 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/295deae6a1574b69985deb317a9a4973"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/295deae6a1574b69985deb317a9a4973"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_295deae6a1574b69985deb317a9a4973"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/295deae6a1574b69985deb317a9a4973"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:29:22.336740 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-ServerGroup264TestJSON-1937790688 Jul 16 21:29:22.336740 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: 295deae6a1574b69985deb317a9a4973 Jul 16 21:29:22.336740 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-ServerGroup264TestJSON-1937790688 Jul 16 21:29:22.336740 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-ServerGroup264TestJSON-1937790688-project Jul 16 21:29:22.336740 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:29:22.336740 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:29:22.336740 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: c88d6c32d91e4aedb4e5d1c2f6358df3 Jul 16 21:29:22.336740 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-ServerGroup264TestJSON-1937790688-project Jul 16 21:29:22.336740 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 21:29:22.336740 ncdv-master-1 devstack@n-api.service[138680]: exception: Instance group aa8ccb51-d5ed-41ce-8625-9b0cf5f2def6 could not be found. Jul 16 21:29:22.336740 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': 'aa8ccb51-d5ed-41ce-8625-9b0cf5f2def6'} Jul 16 21:29:22.336740 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:29:22.336740 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:29:22.336740 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/server_groups.py", line 139, in delete Jul 16 21:29:22.336740 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi sg = objects.InstanceGroup.get_by_uuid(context, id) Jul 16 21:29:22.336740 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Jul 16 21:29:22.336740 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Jul 16 21:29:22.336740 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/instance_group.py", line 373, in get_by_uuid Jul 16 21:29:22.336740 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi db_group = cls._get_from_db_by_uuid(context, uuid) Jul 16 21:29:22.336740 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1010, in wrapper Jul 16 21:29:22.336740 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return fn(*args, **kwargs) Jul 16 21:29:22.336740 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/instance_group.py", line 228, in _get_from_db_by_uuid Jul 16 21:29:22.336740 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.InstanceGroupNotFound(group_uuid=uuid) Jul 16 21:29:22.336740 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.InstanceGroupNotFound: Instance group aa8ccb51-d5ed-41ce-8625-9b0cf5f2def6 could not be found. Jul 16 21:29:22.336740 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:29:22.336740 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:29:22.336740 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:29:22.336740 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:29:22.336740 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:29:22.336740 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:29:22.336740 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:29:22.336740 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:29:22.336740 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:29:22.336740 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:29:22.336740 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/server_groups.py", line 141, in delete Jul 16 21:29:22.336740 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:29:22.336740 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance group aa8ccb51-d5ed-41ce-8625-9b0cf5f2def6 could not be found. Jul 16 21:29:22.336740 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:29:22.336740 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:29:22.336740 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:29:22.336740 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:29:22.341912 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:29:22.341912 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:29:22.341912 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:29:22.341912 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:29:22.341912 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance group aa8ccb51-d5ed-41ce-8625-9b0cf5f2def6 could not be found. Jul 16 21:29:22.341912 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:29:22.344531 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-11429cb1-caa2-4240-89d4-2e2307a8096c tempest-ServerGroup264TestJSON-1937790688 tempest-ServerGroup264TestJSON-1937790688-project] Returning 404 to user: Instance group aa8ccb51-d5ed-41ce-8625-9b0cf5f2def6 could not be found. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:29:22.345748 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-11429cb1-caa2-4240-89d4-2e2307a8096c tempest-ServerGroup264TestJSON-1937790688 tempest-ServerGroup264TestJSON-1937790688-project] 10.222.0.16 "DELETE /compute/v2.1/os-server-groups/aa8ccb51-d5ed-41ce-8625-9b0cf5f2def6" status: 404 len: 117 microversion: 2.64 time: 0.029178 Jul 16 21:29:22.346878 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2173/4354] 10.222.0.16 () {60 vars in 1306 bytes} [Sat Jul 16 21:29:22 2022] DELETE /compute/v2.1/os-server-groups/aa8ccb51-d5ed-41ce-8625-9b0cf5f2def6 => generated 117 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 381 bytes (2 switches on core 0) Jul 16 21:29:22.469398 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-83031e90-9450-44fb-bac2-f7189a759bf5 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'af0cd3652c8d42bcb8af08852ff4f203', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c6f089da37e4ff09784a2b42513ff36', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:22.505961 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-83031e90-9450-44fb-bac2-f7189a759bf5 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'af0cd3652c8d42bcb8af08852ff4f203', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c6f089da37e4ff09784a2b42513ff36', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:22.514308 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-83031e90-9450-44fb-bac2-f7189a759bf5 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'af0cd3652c8d42bcb8af08852ff4f203', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c6f089da37e4ff09784a2b42513ff36', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:22.519967 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-83031e90-9450-44fb-bac2-f7189a759bf5 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] 10.222.0.16 "GET /compute/v2.1/servers/fed8328c-f9d3-4f4a-97c9-4919a94e6106" status: 200 len: 1365 microversion: 2.57 time: 0.630002 Jul 16 21:29:22.525583 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2182/4355] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:29:21 2022] GET /compute/v2.1/servers/fed8328c-f9d3-4f4a-97c9-4919a94e6106 => generated 1365 bytes in 636 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:29:22.726633 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-c1a86ea1-cd62-4a1c-ad47-8912eb938e4f req-27dc6a4e-12db-4631-8ee1-b09af4066912 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "59a4c34e-3184-465b-b14c-0ad55192c8ce", "tag": "cad17319-3e42-44f8-911a-cd258470adec"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:29:22.737467 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-c1a86ea1-cd62-4a1c-ad47-8912eb938e4f req-27dc6a4e-12db-4631-8ee1-b09af4066912 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:22.738168 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-c1a86ea1-cd62-4a1c-ad47-8912eb938e4f req-27dc6a4e-12db-4631-8ee1-b09af4066912 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:22.754230 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-c1a86ea1-cd62-4a1c-ad47-8912eb938e4f req-27dc6a4e-12db-4631-8ee1-b09af4066912 service nova] Creating event network-changed:cad17319-3e42-44f8-911a-cd258470adec for instance 59a4c34e-3184-465b-b14c-0ad55192c8ce on nch2-master-1 Jul 16 21:29:22.761305 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-c1a86ea1-cd62-4a1c-ad47-8912eb938e4f req-27dc6a4e-12db-4631-8ee1-b09af4066912 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.038100 Jul 16 21:29:22.768095 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2174/4356] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:29:22 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:29:22.905075 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-821beb08-1b38-4bdb-aeb7-f2ed4ae83ee9 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:29:22.905956 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-821beb08-1b38-4bdb-aeb7-f2ed4ae83ee9 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] [instance: 08a33911-6e47-45fd-b499-155f199aacf8] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:29:22.920086 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-821beb08-1b38-4bdb-aeb7-f2ed4ae83ee9 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:22.924381 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-821beb08-1b38-4bdb-aeb7-f2ed4ae83ee9 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:22.925637 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-821beb08-1b38-4bdb-aeb7-f2ed4ae83ee9 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:22.925745 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-821beb08-1b38-4bdb-aeb7-f2ed4ae83ee9 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:23.368375 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-821beb08-1b38-4bdb-aeb7-f2ed4ae83ee9 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:23.377981 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-821beb08-1b38-4bdb-aeb7-f2ed4ae83ee9 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:23.384269 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-821beb08-1b38-4bdb-aeb7-f2ed4ae83ee9 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:23.386705 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-821beb08-1b38-4bdb-aeb7-f2ed4ae83ee9 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] 10.222.0.16 "GET /compute/v2.1/servers/08a33911-6e47-45fd-b499-155f199aacf8" status: 200 len: 1723 microversion: 2.20 time: 0.485886 Jul 16 21:29:23.390770 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2183/4357] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:29:22 2022] GET /compute/v2.1/servers/08a33911-6e47-45fd-b499-155f199aacf8 => generated 1723 bytes in 490 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:29:23.547262 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-27f073ac-180b-40ef-93b4-008d8a2234ba tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:29:23.548361 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-27f073ac-180b-40ef-93b4-008d8a2234ba tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] [instance: fed8328c-f9d3-4f4a-97c9-4919a94e6106] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:29:23.562152 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-27f073ac-180b-40ef-93b4-008d8a2234ba tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:23.562936 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-27f073ac-180b-40ef-93b4-008d8a2234ba tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:23.564086 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-27f073ac-180b-40ef-93b4-008d8a2234ba tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:23.566618 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-27f073ac-180b-40ef-93b4-008d8a2234ba tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:23.745517 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-27f073ac-180b-40ef-93b4-008d8a2234ba tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'af0cd3652c8d42bcb8af08852ff4f203', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c6f089da37e4ff09784a2b42513ff36', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:23.756167 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-27f073ac-180b-40ef-93b4-008d8a2234ba tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'af0cd3652c8d42bcb8af08852ff4f203', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c6f089da37e4ff09784a2b42513ff36', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:23.757897 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-27f073ac-180b-40ef-93b4-008d8a2234ba tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'af0cd3652c8d42bcb8af08852ff4f203', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c6f089da37e4ff09784a2b42513ff36', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:23.759159 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-27f073ac-180b-40ef-93b4-008d8a2234ba tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] 10.222.0.16 "GET /compute/v2.1/servers/fed8328c-f9d3-4f4a-97c9-4919a94e6106" status: 200 len: 1365 microversion: 2.57 time: 0.217181 Jul 16 21:29:23.770945 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2175/4358] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:29:23 2022] GET /compute/v2.1/servers/fed8328c-f9d3-4f4a-97c9-4919a94e6106 => generated 1365 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:29:24.409670 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-78d20412-f0a7-4561-bb3f-2c920d316a36 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:29:24.410593 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-78d20412-f0a7-4561-bb3f-2c920d316a36 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] [instance: 08a33911-6e47-45fd-b499-155f199aacf8] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:29:24.426031 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-78d20412-f0a7-4561-bb3f-2c920d316a36 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:24.426499 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-78d20412-f0a7-4561-bb3f-2c920d316a36 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:24.427331 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-78d20412-f0a7-4561-bb3f-2c920d316a36 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:24.434668 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-78d20412-f0a7-4561-bb3f-2c920d316a36 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:24.759558 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-78d20412-f0a7-4561-bb3f-2c920d316a36 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:24.772011 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-78d20412-f0a7-4561-bb3f-2c920d316a36 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:24.778362 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-e5104c2d-4da4-4b8d-8401-0590e2d378f2 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:29:24.779129 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-e5104c2d-4da4-4b8d-8401-0590e2d378f2 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] [instance: fed8328c-f9d3-4f4a-97c9-4919a94e6106] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:29:24.787597 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-78d20412-f0a7-4561-bb3f-2c920d316a36 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:24.788910 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-78d20412-f0a7-4561-bb3f-2c920d316a36 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] 10.222.0.16 "GET /compute/v2.1/servers/08a33911-6e47-45fd-b499-155f199aacf8" status: 200 len: 1723 microversion: 2.20 time: 0.383175 Jul 16 21:29:24.799992 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2184/4359] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:29:24 2022] GET /compute/v2.1/servers/08a33911-6e47-45fd-b499-155f199aacf8 => generated 1723 bytes in 386 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:29:24.800185 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e5104c2d-4da4-4b8d-8401-0590e2d378f2 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:24.800185 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e5104c2d-4da4-4b8d-8401-0590e2d378f2 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:24.800185 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e5104c2d-4da4-4b8d-8401-0590e2d378f2 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:24.800185 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e5104c2d-4da4-4b8d-8401-0590e2d378f2 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:24.961290 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-e5104c2d-4da4-4b8d-8401-0590e2d378f2 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'af0cd3652c8d42bcb8af08852ff4f203', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c6f089da37e4ff09784a2b42513ff36', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:24.971765 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-e5104c2d-4da4-4b8d-8401-0590e2d378f2 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'af0cd3652c8d42bcb8af08852ff4f203', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c6f089da37e4ff09784a2b42513ff36', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:24.973419 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-e5104c2d-4da4-4b8d-8401-0590e2d378f2 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'af0cd3652c8d42bcb8af08852ff4f203', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c6f089da37e4ff09784a2b42513ff36', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:24.974739 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-e5104c2d-4da4-4b8d-8401-0590e2d378f2 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] 10.222.0.16 "GET /compute/v2.1/servers/fed8328c-f9d3-4f4a-97c9-4919a94e6106" status: 200 len: 1365 microversion: 2.57 time: 0.201002 Jul 16 21:29:24.989750 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2176/4360] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:29:24 2022] GET /compute/v2.1/servers/fed8328c-f9d3-4f4a-97c9-4919a94e6106 => generated 1365 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:29:25.807703 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-d803a632-0820-4587-9adc-f5798998d82b tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:29:25.808327 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-d803a632-0820-4587-9adc-f5798998d82b tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] [instance: 08a33911-6e47-45fd-b499-155f199aacf8] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:29:25.816071 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d803a632-0820-4587-9adc-f5798998d82b tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:25.816550 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d803a632-0820-4587-9adc-f5798998d82b tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:25.817338 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d803a632-0820-4587-9adc-f5798998d82b tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:25.817779 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d803a632-0820-4587-9adc-f5798998d82b tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:25.998202 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-88642725-0132-4127-bd4c-3c3c1e2e61b4 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:29:26.002356 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-88642725-0132-4127-bd4c-3c3c1e2e61b4 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] [instance: fed8328c-f9d3-4f4a-97c9-4919a94e6106] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:29:26.008173 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-88642725-0132-4127-bd4c-3c3c1e2e61b4 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:26.008752 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-88642725-0132-4127-bd4c-3c3c1e2e61b4 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:26.009921 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-88642725-0132-4127-bd4c-3c3c1e2e61b4 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:26.010499 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-88642725-0132-4127-bd4c-3c3c1e2e61b4 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:26.057064 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-d803a632-0820-4587-9adc-f5798998d82b tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:26.076642 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-d803a632-0820-4587-9adc-f5798998d82b tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:26.078533 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-d803a632-0820-4587-9adc-f5798998d82b tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:26.081584 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-d803a632-0820-4587-9adc-f5798998d82b tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] 10.222.0.16 "GET /compute/v2.1/servers/08a33911-6e47-45fd-b499-155f199aacf8" status: 200 len: 1723 microversion: 2.20 time: 0.276791 Jul 16 21:29:26.082329 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2185/4361] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:29:25 2022] GET /compute/v2.1/servers/08a33911-6e47-45fd-b499-155f199aacf8 => generated 1723 bytes in 279 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:29:26.152215 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-88642725-0132-4127-bd4c-3c3c1e2e61b4 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'af0cd3652c8d42bcb8af08852ff4f203', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c6f089da37e4ff09784a2b42513ff36', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:26.163860 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-88642725-0132-4127-bd4c-3c3c1e2e61b4 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'af0cd3652c8d42bcb8af08852ff4f203', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c6f089da37e4ff09784a2b42513ff36', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:26.165596 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-88642725-0132-4127-bd4c-3c3c1e2e61b4 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'af0cd3652c8d42bcb8af08852ff4f203', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c6f089da37e4ff09784a2b42513ff36', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:26.167029 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-88642725-0132-4127-bd4c-3c3c1e2e61b4 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] 10.222.0.16 "GET /compute/v2.1/servers/fed8328c-f9d3-4f4a-97c9-4919a94e6106" status: 200 len: 1365 microversion: 2.57 time: 0.177095 Jul 16 21:29:26.167773 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2177/4362] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:29:25 2022] GET /compute/v2.1/servers/fed8328c-f9d3-4f4a-97c9-4919a94e6106 => generated 1365 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:29:27.107311 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-40558105-8737-4b85-b80b-17b46a37654a tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:29:27.107311 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-40558105-8737-4b85-b80b-17b46a37654a tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] [instance: 08a33911-6e47-45fd-b499-155f199aacf8] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:29:27.122928 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-40558105-8737-4b85-b80b-17b46a37654a tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:27.123668 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-40558105-8737-4b85-b80b-17b46a37654a tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:27.131223 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-40558105-8737-4b85-b80b-17b46a37654a tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:27.131873 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-40558105-8737-4b85-b80b-17b46a37654a tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:27.188042 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-c92f4866-06f5-475b-a7c2-bf76797b2966 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:29:27.188634 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-c92f4866-06f5-475b-a7c2-bf76797b2966 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] [instance: fed8328c-f9d3-4f4a-97c9-4919a94e6106] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:29:27.204570 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c92f4866-06f5-475b-a7c2-bf76797b2966 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:27.205046 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c92f4866-06f5-475b-a7c2-bf76797b2966 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:27.205851 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c92f4866-06f5-475b-a7c2-bf76797b2966 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:27.206304 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c92f4866-06f5-475b-a7c2-bf76797b2966 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:27.371940 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-c92f4866-06f5-475b-a7c2-bf76797b2966 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'af0cd3652c8d42bcb8af08852ff4f203', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c6f089da37e4ff09784a2b42513ff36', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:27.382118 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-c92f4866-06f5-475b-a7c2-bf76797b2966 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'af0cd3652c8d42bcb8af08852ff4f203', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c6f089da37e4ff09784a2b42513ff36', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:27.384247 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-c92f4866-06f5-475b-a7c2-bf76797b2966 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'af0cd3652c8d42bcb8af08852ff4f203', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c6f089da37e4ff09784a2b42513ff36', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:27.385708 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-c92f4866-06f5-475b-a7c2-bf76797b2966 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] 10.222.0.16 "GET /compute/v2.1/servers/fed8328c-f9d3-4f4a-97c9-4919a94e6106" status: 200 len: 1365 microversion: 2.57 time: 0.202401 Jul 16 21:29:27.395782 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2178/4363] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:29:27 2022] GET /compute/v2.1/servers/fed8328c-f9d3-4f4a-97c9-4919a94e6106 => generated 1365 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 16 21:29:27.542519 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-40558105-8737-4b85-b80b-17b46a37654a tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:27.554208 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-40558105-8737-4b85-b80b-17b46a37654a tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:27.557159 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-40558105-8737-4b85-b80b-17b46a37654a tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:27.560222 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-40558105-8737-4b85-b80b-17b46a37654a tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] 10.222.0.16 "GET /compute/v2.1/servers/08a33911-6e47-45fd-b499-155f199aacf8" status: 200 len: 1723 microversion: 2.20 time: 0.462849 Jul 16 21:29:27.561004 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2186/4364] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:29:27 2022] GET /compute/v2.1/servers/08a33911-6e47-45fd-b499-155f199aacf8 => generated 1723 bytes in 466 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 16 21:29:28.409122 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-d8ed9ea5-6e1a-4005-ba49-50c29082b6e3 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:29:28.410806 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-d8ed9ea5-6e1a-4005-ba49-50c29082b6e3 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] [instance: fed8328c-f9d3-4f4a-97c9-4919a94e6106] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:29:28.423448 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d8ed9ea5-6e1a-4005-ba49-50c29082b6e3 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:28.424123 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d8ed9ea5-6e1a-4005-ba49-50c29082b6e3 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:28.425254 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d8ed9ea5-6e1a-4005-ba49-50c29082b6e3 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:28.425857 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d8ed9ea5-6e1a-4005-ba49-50c29082b6e3 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:28.534911 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-d8ed9ea5-6e1a-4005-ba49-50c29082b6e3 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'af0cd3652c8d42bcb8af08852ff4f203', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c6f089da37e4ff09784a2b42513ff36', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:28.547860 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-d8ed9ea5-6e1a-4005-ba49-50c29082b6e3 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'af0cd3652c8d42bcb8af08852ff4f203', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c6f089da37e4ff09784a2b42513ff36', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:28.549443 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-d8ed9ea5-6e1a-4005-ba49-50c29082b6e3 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'af0cd3652c8d42bcb8af08852ff4f203', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c6f089da37e4ff09784a2b42513ff36', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:28.555476 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-d8ed9ea5-6e1a-4005-ba49-50c29082b6e3 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] 10.222.0.16 "GET /compute/v2.1/servers/fed8328c-f9d3-4f4a-97c9-4919a94e6106" status: 200 len: 1365 microversion: 2.57 time: 0.145549 Jul 16 21:29:28.556109 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2179/4365] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:29:28 2022] GET /compute/v2.1/servers/fed8328c-f9d3-4f4a-97c9-4919a94e6106 => generated 1365 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:29:28.595524 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-228d75c7-4ca7-49a0-86ab-09679874eacb tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:29:28.596738 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-228d75c7-4ca7-49a0-86ab-09679874eacb tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] [instance: 08a33911-6e47-45fd-b499-155f199aacf8] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:29:28.601218 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_api:SELECT=83,nova_api:UPDATE=1 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:29:28.607950 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-228d75c7-4ca7-49a0-86ab-09679874eacb tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:28.608660 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-228d75c7-4ca7-49a0-86ab-09679874eacb tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:28.610899 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-228d75c7-4ca7-49a0-86ab-09679874eacb tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:28.610899 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-228d75c7-4ca7-49a0-86ab-09679874eacb tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:28.815066 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-228d75c7-4ca7-49a0-86ab-09679874eacb tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:28.839052 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-228d75c7-4ca7-49a0-86ab-09679874eacb tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:28.842464 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-228d75c7-4ca7-49a0-86ab-09679874eacb tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:28.845122 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-228d75c7-4ca7-49a0-86ab-09679874eacb tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] 10.222.0.16 "GET /compute/v2.1/servers/08a33911-6e47-45fd-b499-155f199aacf8" status: 200 len: 1723 microversion: 2.20 time: 0.267655 Jul 16 21:29:28.847056 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2187/4366] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:29:28 2022] GET /compute/v2.1/servers/08a33911-6e47-45fd-b499-155f199aacf8 => generated 1723 bytes in 270 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:29:29.576371 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-f7f22dda-14c0-4174-a69d-836e46e20bc4 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:29:29.578247 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-f7f22dda-14c0-4174-a69d-836e46e20bc4 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] [instance: fed8328c-f9d3-4f4a-97c9-4919a94e6106] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:29:29.585821 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f7f22dda-14c0-4174-a69d-836e46e20bc4 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:29.586367 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f7f22dda-14c0-4174-a69d-836e46e20bc4 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:29.587268 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f7f22dda-14c0-4174-a69d-836e46e20bc4 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:29.587998 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f7f22dda-14c0-4174-a69d-836e46e20bc4 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:29.739161 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-f7f22dda-14c0-4174-a69d-836e46e20bc4 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'af0cd3652c8d42bcb8af08852ff4f203', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c6f089da37e4ff09784a2b42513ff36', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:29.742231 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:29:29.767650 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-f7f22dda-14c0-4174-a69d-836e46e20bc4 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'af0cd3652c8d42bcb8af08852ff4f203', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c6f089da37e4ff09784a2b42513ff36', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:29.768862 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-f7f22dda-14c0-4174-a69d-836e46e20bc4 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'af0cd3652c8d42bcb8af08852ff4f203', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c6f089da37e4ff09784a2b42513ff36', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:29.770349 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-f7f22dda-14c0-4174-a69d-836e46e20bc4 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] 10.222.0.16 "GET /compute/v2.1/servers/fed8328c-f9d3-4f4a-97c9-4919a94e6106" status: 200 len: 1365 microversion: 2.57 time: 0.200539 Jul 16 21:29:29.771468 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2180/4367] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:29:29 2022] GET /compute/v2.1/servers/fed8328c-f9d3-4f4a-97c9-4919a94e6106 => generated 1365 bytes in 202 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:29:29.863820 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-074f4545-b959-4dc7-99b0-6b5e1cc58a6a tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:29:29.864568 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-074f4545-b959-4dc7-99b0-6b5e1cc58a6a tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] [instance: 08a33911-6e47-45fd-b499-155f199aacf8] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:29:29.873392 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-074f4545-b959-4dc7-99b0-6b5e1cc58a6a tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:29.873868 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-074f4545-b959-4dc7-99b0-6b5e1cc58a6a tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:29.874821 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-074f4545-b959-4dc7-99b0-6b5e1cc58a6a tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:29.875301 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-074f4545-b959-4dc7-99b0-6b5e1cc58a6a tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:30.102851 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-074f4545-b959-4dc7-99b0-6b5e1cc58a6a tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:30.112132 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-074f4545-b959-4dc7-99b0-6b5e1cc58a6a tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:30.114396 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-074f4545-b959-4dc7-99b0-6b5e1cc58a6a tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:30.116712 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-074f4545-b959-4dc7-99b0-6b5e1cc58a6a tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] 10.222.0.16 "GET /compute/v2.1/servers/08a33911-6e47-45fd-b499-155f199aacf8" status: 200 len: 1723 microversion: 2.20 time: 0.257345 Jul 16 21:29:30.118166 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2188/4368] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:29:29 2022] GET /compute/v2.1/servers/08a33911-6e47-45fd-b499-155f199aacf8 => generated 1723 bytes in 259 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:29:30.794859 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-ae6e0944-cf70-4485-887e-537d4f86c44c tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:29:30.795696 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-ae6e0944-cf70-4485-887e-537d4f86c44c tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] [instance: fed8328c-f9d3-4f4a-97c9-4919a94e6106] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:29:30.802867 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ae6e0944-cf70-4485-887e-537d4f86c44c tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:30.803443 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ae6e0944-cf70-4485-887e-537d4f86c44c tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:30.804385 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ae6e0944-cf70-4485-887e-537d4f86c44c tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:30.804907 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ae6e0944-cf70-4485-887e-537d4f86c44c tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:30.881333 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-ae6e0944-cf70-4485-887e-537d4f86c44c tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'af0cd3652c8d42bcb8af08852ff4f203', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c6f089da37e4ff09784a2b42513ff36', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:30.891132 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-ae6e0944-cf70-4485-887e-537d4f86c44c tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'af0cd3652c8d42bcb8af08852ff4f203', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c6f089da37e4ff09784a2b42513ff36', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:30.892952 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-ae6e0944-cf70-4485-887e-537d4f86c44c tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'af0cd3652c8d42bcb8af08852ff4f203', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c6f089da37e4ff09784a2b42513ff36', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:30.894138 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-ae6e0944-cf70-4485-887e-537d4f86c44c tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] 10.222.0.16 "GET /compute/v2.1/servers/fed8328c-f9d3-4f4a-97c9-4919a94e6106" status: 200 len: 1365 microversion: 2.57 time: 0.107268 Jul 16 21:29:30.894704 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2181/4369] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:29:30 2022] GET /compute/v2.1/servers/fed8328c-f9d3-4f4a-97c9-4919a94e6106 => generated 1365 bytes in 109 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:29:31.137551 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-c126aca9-61ea-43d3-b7fa-285c508cad8d tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:29:31.142739 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-c126aca9-61ea-43d3-b7fa-285c508cad8d tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] [instance: 08a33911-6e47-45fd-b499-155f199aacf8] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:29:31.149558 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c126aca9-61ea-43d3-b7fa-285c508cad8d tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:31.150017 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c126aca9-61ea-43d3-b7fa-285c508cad8d tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:31.150807 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c126aca9-61ea-43d3-b7fa-285c508cad8d tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:31.155033 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c126aca9-61ea-43d3-b7fa-285c508cad8d tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:31.385362 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-c126aca9-61ea-43d3-b7fa-285c508cad8d tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:31.398411 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-c126aca9-61ea-43d3-b7fa-285c508cad8d tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:31.402475 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-c126aca9-61ea-43d3-b7fa-285c508cad8d tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:31.408882 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-c126aca9-61ea-43d3-b7fa-285c508cad8d tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] 10.222.0.16 "GET /compute/v2.1/servers/08a33911-6e47-45fd-b499-155f199aacf8" status: 200 len: 1723 microversion: 2.20 time: 0.276494 Jul 16 21:29:31.409482 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2189/4370] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:29:31 2022] GET /compute/v2.1/servers/08a33911-6e47-45fd-b499-155f199aacf8 => generated 1723 bytes in 278 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 16 21:29:31.914975 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-300fcaca-ba89-42a5-bf08-ec532f240551 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:29:31.915559 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-300fcaca-ba89-42a5-bf08-ec532f240551 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] [instance: fed8328c-f9d3-4f4a-97c9-4919a94e6106] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:29:31.924832 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-300fcaca-ba89-42a5-bf08-ec532f240551 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:31.925264 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-300fcaca-ba89-42a5-bf08-ec532f240551 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:31.926056 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-300fcaca-ba89-42a5-bf08-ec532f240551 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:31.926505 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-300fcaca-ba89-42a5-bf08-ec532f240551 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:32.015366 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-300fcaca-ba89-42a5-bf08-ec532f240551 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'af0cd3652c8d42bcb8af08852ff4f203', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c6f089da37e4ff09784a2b42513ff36', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:32.025186 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-300fcaca-ba89-42a5-bf08-ec532f240551 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'af0cd3652c8d42bcb8af08852ff4f203', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c6f089da37e4ff09784a2b42513ff36', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:32.026944 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-300fcaca-ba89-42a5-bf08-ec532f240551 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'af0cd3652c8d42bcb8af08852ff4f203', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c6f089da37e4ff09784a2b42513ff36', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:32.028705 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-300fcaca-ba89-42a5-bf08-ec532f240551 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] 10.222.0.16 "GET /compute/v2.1/servers/fed8328c-f9d3-4f4a-97c9-4919a94e6106" status: 200 len: 1365 microversion: 2.57 time: 0.117484 Jul 16 21:29:32.029466 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2182/4371] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:29:31 2022] GET /compute/v2.1/servers/fed8328c-f9d3-4f4a-97c9-4919a94e6106 => generated 1365 bytes in 119 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:29:32.428257 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-ec3d0a9e-48c1-46e7-99f8-87fad1eeacd1 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:29:32.429020 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-ec3d0a9e-48c1-46e7-99f8-87fad1eeacd1 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] [instance: 08a33911-6e47-45fd-b499-155f199aacf8] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:29:32.437332 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ec3d0a9e-48c1-46e7-99f8-87fad1eeacd1 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:32.437857 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ec3d0a9e-48c1-46e7-99f8-87fad1eeacd1 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:32.438764 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ec3d0a9e-48c1-46e7-99f8-87fad1eeacd1 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:32.439289 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ec3d0a9e-48c1-46e7-99f8-87fad1eeacd1 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:32.587628 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-ec3d0a9e-48c1-46e7-99f8-87fad1eeacd1 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:32.599796 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-ec3d0a9e-48c1-46e7-99f8-87fad1eeacd1 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:32.601309 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-ec3d0a9e-48c1-46e7-99f8-87fad1eeacd1 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:32.603148 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-ec3d0a9e-48c1-46e7-99f8-87fad1eeacd1 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] 10.222.0.16 "GET /compute/v2.1/servers/08a33911-6e47-45fd-b499-155f199aacf8" status: 200 len: 1744 microversion: 2.20 time: 0.180049 Jul 16 21:29:32.608100 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2190/4372] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:29:32 2022] GET /compute/v2.1/servers/08a33911-6e47-45fd-b499-155f199aacf8 => generated 1744 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:29:33.049066 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-d7510293-1f16-4ca7-bc53-5423b8cb3dfb tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:29:33.049692 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-d7510293-1f16-4ca7-bc53-5423b8cb3dfb tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] [instance: fed8328c-f9d3-4f4a-97c9-4919a94e6106] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:29:33.057209 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell0:SELECT=8 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:29:33.060396 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d7510293-1f16-4ca7-bc53-5423b8cb3dfb tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:33.060863 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d7510293-1f16-4ca7-bc53-5423b8cb3dfb tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:33.061656 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d7510293-1f16-4ca7-bc53-5423b8cb3dfb tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:33.062228 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d7510293-1f16-4ca7-bc53-5423b8cb3dfb tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:33.161317 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-d7510293-1f16-4ca7-bc53-5423b8cb3dfb tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'af0cd3652c8d42bcb8af08852ff4f203', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c6f089da37e4ff09784a2b42513ff36', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:33.170743 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-d7510293-1f16-4ca7-bc53-5423b8cb3dfb tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'af0cd3652c8d42bcb8af08852ff4f203', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c6f089da37e4ff09784a2b42513ff36', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:33.172755 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-d7510293-1f16-4ca7-bc53-5423b8cb3dfb tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'af0cd3652c8d42bcb8af08852ff4f203', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c6f089da37e4ff09784a2b42513ff36', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:33.174434 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-d7510293-1f16-4ca7-bc53-5423b8cb3dfb tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] 10.222.0.16 "GET /compute/v2.1/servers/fed8328c-f9d3-4f4a-97c9-4919a94e6106" status: 200 len: 1365 microversion: 2.57 time: 0.130263 Jul 16 21:29:33.175641 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2183/4373] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:29:33 2022] GET /compute/v2.1/servers/fed8328c-f9d3-4f4a-97c9-4919a94e6106 => generated 1365 bytes in 132 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:29:33.625182 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-0624376c-cbe6-4769-8857-1d795b84a158 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:29:33.626727 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-0624376c-cbe6-4769-8857-1d795b84a158 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] [instance: 08a33911-6e47-45fd-b499-155f199aacf8] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:29:33.635145 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0624376c-cbe6-4769-8857-1d795b84a158 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:33.635934 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0624376c-cbe6-4769-8857-1d795b84a158 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:33.637586 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0624376c-cbe6-4769-8857-1d795b84a158 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:33.638228 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0624376c-cbe6-4769-8857-1d795b84a158 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:34.195901 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-fe84d205-d7d0-4d09-832c-acd40793dc07 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:29:34.197087 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-fe84d205-d7d0-4d09-832c-acd40793dc07 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] [instance: fed8328c-f9d3-4f4a-97c9-4919a94e6106] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:29:34.210055 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-fe84d205-d7d0-4d09-832c-acd40793dc07 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:34.210055 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-fe84d205-d7d0-4d09-832c-acd40793dc07 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:34.210055 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-fe84d205-d7d0-4d09-832c-acd40793dc07 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:34.210055 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-fe84d205-d7d0-4d09-832c-acd40793dc07 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:34.266661 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-0624376c-cbe6-4769-8857-1d795b84a158 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:34.293651 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-0624376c-cbe6-4769-8857-1d795b84a158 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:34.295878 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-0624376c-cbe6-4769-8857-1d795b84a158 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:34.302002 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-0624376c-cbe6-4769-8857-1d795b84a158 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] 10.222.0.16 "GET /compute/v2.1/servers/08a33911-6e47-45fd-b499-155f199aacf8" status: 200 len: 1744 microversion: 2.20 time: 0.679837 Jul 16 21:29:34.315804 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-fe84d205-d7d0-4d09-832c-acd40793dc07 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'af0cd3652c8d42bcb8af08852ff4f203', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c6f089da37e4ff09784a2b42513ff36', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:34.317138 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2191/4374] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:29:33 2022] GET /compute/v2.1/servers/08a33911-6e47-45fd-b499-155f199aacf8 => generated 1744 bytes in 698 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:29:34.334433 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-fe84d205-d7d0-4d09-832c-acd40793dc07 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'af0cd3652c8d42bcb8af08852ff4f203', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c6f089da37e4ff09784a2b42513ff36', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:34.336209 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-fe84d205-d7d0-4d09-832c-acd40793dc07 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'af0cd3652c8d42bcb8af08852ff4f203', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c6f089da37e4ff09784a2b42513ff36', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:34.337858 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-fe84d205-d7d0-4d09-832c-acd40793dc07 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] 10.222.0.16 "GET /compute/v2.1/servers/fed8328c-f9d3-4f4a-97c9-4919a94e6106" status: 200 len: 1371 microversion: 2.57 time: 0.146752 Jul 16 21:29:34.343034 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2184/4375] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:29:34 2022] GET /compute/v2.1/servers/fed8328c-f9d3-4f4a-97c9-4919a94e6106 => generated 1371 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:29:35.329118 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-22249039-49f8-4ace-953d-0813a5e7a544 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:29:35.330450 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-22249039-49f8-4ace-953d-0813a5e7a544 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] [instance: 08a33911-6e47-45fd-b499-155f199aacf8] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:29:35.345175 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-22249039-49f8-4ace-953d-0813a5e7a544 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:35.345534 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-22249039-49f8-4ace-953d-0813a5e7a544 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:35.346322 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-22249039-49f8-4ace-953d-0813a5e7a544 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:35.347713 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-22249039-49f8-4ace-953d-0813a5e7a544 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:35.368531 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-dd0757d8-cb8e-4bea-8fce-5217cedc9387 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:29:35.369329 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-dd0757d8-cb8e-4bea-8fce-5217cedc9387 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] [instance: fed8328c-f9d3-4f4a-97c9-4919a94e6106] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:29:35.380643 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-dd0757d8-cb8e-4bea-8fce-5217cedc9387 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:35.381567 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-dd0757d8-cb8e-4bea-8fce-5217cedc9387 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:35.382927 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-dd0757d8-cb8e-4bea-8fce-5217cedc9387 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:35.383628 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-dd0757d8-cb8e-4bea-8fce-5217cedc9387 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:35.633634 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-dd0757d8-cb8e-4bea-8fce-5217cedc9387 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'af0cd3652c8d42bcb8af08852ff4f203', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c6f089da37e4ff09784a2b42513ff36', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:35.659790 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-dd0757d8-cb8e-4bea-8fce-5217cedc9387 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'af0cd3652c8d42bcb8af08852ff4f203', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c6f089da37e4ff09784a2b42513ff36', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:35.666169 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-dd0757d8-cb8e-4bea-8fce-5217cedc9387 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'af0cd3652c8d42bcb8af08852ff4f203', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c6f089da37e4ff09784a2b42513ff36', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:35.671713 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-dd0757d8-cb8e-4bea-8fce-5217cedc9387 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] 10.222.0.16 "GET /compute/v2.1/servers/fed8328c-f9d3-4f4a-97c9-4919a94e6106" status: 200 len: 1371 microversion: 2.57 time: 0.309759 Jul 16 21:29:35.673460 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2185/4376] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:29:35 2022] GET /compute/v2.1/servers/fed8328c-f9d3-4f4a-97c9-4919a94e6106 => generated 1371 bytes in 312 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 16 21:29:35.945418 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-22249039-49f8-4ace-953d-0813a5e7a544 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:35.966125 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-22249039-49f8-4ace-953d-0813a5e7a544 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:35.968254 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-22249039-49f8-4ace-953d-0813a5e7a544 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:35.969672 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-22249039-49f8-4ace-953d-0813a5e7a544 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] 10.222.0.16 "GET /compute/v2.1/servers/08a33911-6e47-45fd-b499-155f199aacf8" status: 200 len: 1739 microversion: 2.20 time: 0.647810 Jul 16 21:29:35.971366 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2192/4377] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:29:35 2022] GET /compute/v2.1/servers/08a33911-6e47-45fd-b499-155f199aacf8 => generated 1739 bytes in 650 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:29:36.080057 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-bb49cd85-d014-431b-8d5e-1e22339d1de4 req-78420148-a4e1-4c63-ae1c-f6b9da7175c5 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "08a33911-6e47-45fd-b499-155f199aacf8", "name": "network-vif-plugged", "status": "completed", "tag": "81e317e7-e9db-4ba9-b87d-9395b084ead7"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:29:36.099122 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-bb49cd85-d014-431b-8d5e-1e22339d1de4 req-78420148-a4e1-4c63-ae1c-f6b9da7175c5 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:36.099122 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-bb49cd85-d014-431b-8d5e-1e22339d1de4 req-78420148-a4e1-4c63-ae1c-f6b9da7175c5 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:36.174645 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-bb49cd85-d014-431b-8d5e-1e22339d1de4 req-78420148-a4e1-4c63-ae1c-f6b9da7175c5 service nova] Creating event network-vif-plugged:81e317e7-e9db-4ba9-b87d-9395b084ead7 for instance 08a33911-6e47-45fd-b499-155f199aacf8 on nch1-master-1 Jul 16 21:29:36.188297 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-bb49cd85-d014-431b-8d5e-1e22339d1de4 req-78420148-a4e1-4c63-ae1c-f6b9da7175c5 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.108722 Jul 16 21:29:36.189274 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2186/4378] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:29:36 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 114 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:29:36.712319 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-6c732351-253d-4bba-a02a-ca3db8ab10c0 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:29:36.713750 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-6c732351-253d-4bba-a02a-ca3db8ab10c0 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] [instance: fed8328c-f9d3-4f4a-97c9-4919a94e6106] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:29:36.728139 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6c732351-253d-4bba-a02a-ca3db8ab10c0 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:36.728989 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6c732351-253d-4bba-a02a-ca3db8ab10c0 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:36.732644 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6c732351-253d-4bba-a02a-ca3db8ab10c0 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:36.733330 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6c732351-253d-4bba-a02a-ca3db8ab10c0 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:36.816921 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=100 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:29:36.872456 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-6c732351-253d-4bba-a02a-ca3db8ab10c0 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'af0cd3652c8d42bcb8af08852ff4f203', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c6f089da37e4ff09784a2b42513ff36', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:36.919900 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-6c732351-253d-4bba-a02a-ca3db8ab10c0 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'af0cd3652c8d42bcb8af08852ff4f203', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c6f089da37e4ff09784a2b42513ff36', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:36.919900 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-6c732351-253d-4bba-a02a-ca3db8ab10c0 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'af0cd3652c8d42bcb8af08852ff4f203', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c6f089da37e4ff09784a2b42513ff36', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:36.919900 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-6c732351-253d-4bba-a02a-ca3db8ab10c0 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] 10.222.0.16 "GET /compute/v2.1/servers/fed8328c-f9d3-4f4a-97c9-4919a94e6106" status: 200 len: 1356 microversion: 2.57 time: 0.185302 Jul 16 21:29:36.919900 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2193/4379] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:29:36 2022] GET /compute/v2.1/servers/fed8328c-f9d3-4f4a-97c9-4919a94e6106 => generated 1356 bytes in 197 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:29:36.920414 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-269bd769-60f1-46c4-8e76-013af827a91c tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:29:36.920414 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-269bd769-60f1-46c4-8e76-013af827a91c tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] [instance: fed8328c-f9d3-4f4a-97c9-4919a94e6106] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:29:36.928297 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-269bd769-60f1-46c4-8e76-013af827a91c tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:36.928985 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-269bd769-60f1-46c4-8e76-013af827a91c tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:36.930635 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-269bd769-60f1-46c4-8e76-013af827a91c tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:36.931685 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-269bd769-60f1-46c4-8e76-013af827a91c tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:36.966318 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-269bd769-60f1-46c4-8e76-013af827a91c tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] [instance: fed8328c-f9d3-4f4a-97c9-4919a94e6106] Going to try to terminate instance {{(pid=138680) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 16 21:29:36.991189 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-6fc534e3-ca11-44c6-8de3-8bc930159fe7 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:29:36.992145 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-6fc534e3-ca11-44c6-8de3-8bc930159fe7 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] [instance: 08a33911-6e47-45fd-b499-155f199aacf8] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:29:36.999160 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6fc534e3-ca11-44c6-8de3-8bc930159fe7 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:36.999893 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6fc534e3-ca11-44c6-8de3-8bc930159fe7 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:37.001594 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6fc534e3-ca11-44c6-8de3-8bc930159fe7 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:37.002242 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6fc534e3-ca11-44c6-8de3-8bc930159fe7 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:37.348229 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-6fc534e3-ca11-44c6-8de3-8bc930159fe7 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:37.359376 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-6fc534e3-ca11-44c6-8de3-8bc930159fe7 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:37.365441 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-6fc534e3-ca11-44c6-8de3-8bc930159fe7 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:37.371504 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-6fc534e3-ca11-44c6-8de3-8bc930159fe7 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] 10.222.0.16 "GET /compute/v2.1/servers/08a33911-6e47-45fd-b499-155f199aacf8" status: 200 len: 1739 microversion: 2.20 time: 0.384369 Jul 16 21:29:37.374623 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2194/4380] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:29:36 2022] GET /compute/v2.1/servers/08a33911-6e47-45fd-b499-155f199aacf8 => generated 1739 bytes in 390 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:29:37.562810 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-269bd769-60f1-46c4-8e76-013af827a91c tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] 10.222.0.16 "DELETE /compute/v2.1/servers/fed8328c-f9d3-4f4a-97c9-4919a94e6106" status: 204 len: 0 microversion: 2.57 time: 0.650505 Jul 16 21:29:37.563643 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2187/4381] 10.222.0.16 () {60 vars in 1279 bytes} [Sat Jul 16 21:29:36 2022] DELETE /compute/v2.1/servers/fed8328c-f9d3-4f4a-97c9-4919a94e6106 => generated 0 bytes in 652 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Jul 16 21:29:37.574320 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-0e1df288-133c-46e0-a8ab-c0adad0f815a tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:29:37.574320 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-0e1df288-133c-46e0-a8ab-c0adad0f815a tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] [instance: fed8328c-f9d3-4f4a-97c9-4919a94e6106] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:29:37.585857 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0e1df288-133c-46e0-a8ab-c0adad0f815a tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:37.586320 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0e1df288-133c-46e0-a8ab-c0adad0f815a tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:37.587140 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0e1df288-133c-46e0-a8ab-c0adad0f815a tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:37.587778 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0e1df288-133c-46e0-a8ab-c0adad0f815a tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:37.797989 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-0e1df288-133c-46e0-a8ab-c0adad0f815a tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'af0cd3652c8d42bcb8af08852ff4f203', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c6f089da37e4ff09784a2b42513ff36', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:37.814350 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-0e1df288-133c-46e0-a8ab-c0adad0f815a tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'af0cd3652c8d42bcb8af08852ff4f203', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c6f089da37e4ff09784a2b42513ff36', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:37.816139 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-0e1df288-133c-46e0-a8ab-c0adad0f815a tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'af0cd3652c8d42bcb8af08852ff4f203', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c6f089da37e4ff09784a2b42513ff36', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:37.817527 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-0e1df288-133c-46e0-a8ab-c0adad0f815a tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] 10.222.0.16 "GET /compute/v2.1/servers/fed8328c-f9d3-4f4a-97c9-4919a94e6106" status: 200 len: 1362 microversion: 2.57 time: 0.249887 Jul 16 21:29:37.849285 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2195/4382] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:29:37 2022] GET /compute/v2.1/servers/fed8328c-f9d3-4f4a-97c9-4919a94e6106 => generated 1362 bytes in 282 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:29:38.393918 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-edd726f9-345a-4d29-82f1-41aed4aeb4bb tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:29:38.394645 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-edd726f9-345a-4d29-82f1-41aed4aeb4bb tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] [instance: 08a33911-6e47-45fd-b499-155f199aacf8] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:29:38.405164 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-edd726f9-345a-4d29-82f1-41aed4aeb4bb tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:38.412725 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-edd726f9-345a-4d29-82f1-41aed4aeb4bb tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:38.412725 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-edd726f9-345a-4d29-82f1-41aed4aeb4bb tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:38.412725 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-edd726f9-345a-4d29-82f1-41aed4aeb4bb tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:38.581850 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-edd726f9-345a-4d29-82f1-41aed4aeb4bb tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:38.591635 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-edd726f9-345a-4d29-82f1-41aed4aeb4bb tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:38.593440 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-edd726f9-345a-4d29-82f1-41aed4aeb4bb tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:38.594856 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-edd726f9-345a-4d29-82f1-41aed4aeb4bb tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] 10.222.0.16 "GET /compute/v2.1/servers/08a33911-6e47-45fd-b499-155f199aacf8" status: 200 len: 1739 microversion: 2.20 time: 0.205188 Jul 16 21:29:38.599071 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2188/4383] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:29:38 2022] GET /compute/v2.1/servers/08a33911-6e47-45fd-b499-155f199aacf8 => generated 1739 bytes in 207 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:29:38.878421 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-f54b6e47-e303-4ee4-b56f-dfd30227f058 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:29:38.879210 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-f54b6e47-e303-4ee4-b56f-dfd30227f058 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] [instance: fed8328c-f9d3-4f4a-97c9-4919a94e6106] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:29:38.893437 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f54b6e47-e303-4ee4-b56f-dfd30227f058 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:38.894857 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f54b6e47-e303-4ee4-b56f-dfd30227f058 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:38.896996 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f54b6e47-e303-4ee4-b56f-dfd30227f058 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:38.900743 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f54b6e47-e303-4ee4-b56f-dfd30227f058 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:39.071436 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-f54b6e47-e303-4ee4-b56f-dfd30227f058 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'af0cd3652c8d42bcb8af08852ff4f203', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c6f089da37e4ff09784a2b42513ff36', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:39.084717 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-f54b6e47-e303-4ee4-b56f-dfd30227f058 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'af0cd3652c8d42bcb8af08852ff4f203', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c6f089da37e4ff09784a2b42513ff36', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:39.091358 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-f54b6e47-e303-4ee4-b56f-dfd30227f058 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'af0cd3652c8d42bcb8af08852ff4f203', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c6f089da37e4ff09784a2b42513ff36', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:39.092753 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-f54b6e47-e303-4ee4-b56f-dfd30227f058 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] 10.222.0.16 "GET /compute/v2.1/servers/fed8328c-f9d3-4f4a-97c9-4919a94e6106" status: 200 len: 1362 microversion: 2.57 time: 0.223637 Jul 16 21:29:39.112719 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2196/4384] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:29:38 2022] GET /compute/v2.1/servers/fed8328c-f9d3-4f4a-97c9-4919a94e6106 => generated 1362 bytes in 239 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:29:39.614065 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-c6598ab6-cf97-4902-8733-a79a08e07e39 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:29:39.615076 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-c6598ab6-cf97-4902-8733-a79a08e07e39 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] [instance: 08a33911-6e47-45fd-b499-155f199aacf8] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:29:39.626280 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c6598ab6-cf97-4902-8733-a79a08e07e39 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:39.630895 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c6598ab6-cf97-4902-8733-a79a08e07e39 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:39.631873 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c6598ab6-cf97-4902-8733-a79a08e07e39 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:39.633614 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c6598ab6-cf97-4902-8733-a79a08e07e39 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:39.958338 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-c6598ab6-cf97-4902-8733-a79a08e07e39 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:39.968520 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-c6598ab6-cf97-4902-8733-a79a08e07e39 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:39.977549 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-c6598ab6-cf97-4902-8733-a79a08e07e39 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:39.982889 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-c6598ab6-cf97-4902-8733-a79a08e07e39 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] 10.222.0.16 "GET /compute/v2.1/servers/08a33911-6e47-45fd-b499-155f199aacf8" status: 200 len: 1739 microversion: 2.20 time: 0.368993 Jul 16 21:29:39.982889 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2189/4385] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:29:39 2022] GET /compute/v2.1/servers/08a33911-6e47-45fd-b499-155f199aacf8 => generated 1739 bytes in 370 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:29:40.112881 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-88f317f8-8180-4eeb-be1c-0b89d1453a10 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:29:40.114159 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-88f317f8-8180-4eeb-be1c-0b89d1453a10 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] [instance: fed8328c-f9d3-4f4a-97c9-4919a94e6106] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:29:40.127877 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-88f317f8-8180-4eeb-be1c-0b89d1453a10 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:40.128754 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-88f317f8-8180-4eeb-be1c-0b89d1453a10 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:40.141808 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-88f317f8-8180-4eeb-be1c-0b89d1453a10 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:40.144897 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-88f317f8-8180-4eeb-be1c-0b89d1453a10 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.015s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:40.262948 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-88f317f8-8180-4eeb-be1c-0b89d1453a10 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:40.264607 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-88f317f8-8180-4eeb-be1c-0b89d1453a10 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:40.265299 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.objects.instance [None req-88f317f8-8180-4eeb-be1c-0b89d1453a10 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Lazy-loading 'fault' on Instance uuid fed8328c-f9d3-4f4a-97c9-4919a94e6106 {{(pid=138681) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:29:40.419777 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-88f317f8-8180-4eeb-be1c-0b89d1453a10 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'af0cd3652c8d42bcb8af08852ff4f203', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c6f089da37e4ff09784a2b42513ff36', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:40.431996 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-88f317f8-8180-4eeb-be1c-0b89d1453a10 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'af0cd3652c8d42bcb8af08852ff4f203', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c6f089da37e4ff09784a2b42513ff36', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:40.435053 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-88f317f8-8180-4eeb-be1c-0b89d1453a10 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'af0cd3652c8d42bcb8af08852ff4f203', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3c6f089da37e4ff09784a2b42513ff36', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:40.437338 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-88f317f8-8180-4eeb-be1c-0b89d1453a10 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] 10.222.0.16 "GET /compute/v2.1/servers/fed8328c-f9d3-4f4a-97c9-4919a94e6106" status: 200 len: 1367 microversion: 2.57 time: 0.328502 Jul 16 21:29:40.452580 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2197/4386] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:29:40 2022] GET /compute/v2.1/servers/fed8328c-f9d3-4f4a-97c9-4919a94e6106 => generated 1367 bytes in 345 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:29:40.998731 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-04a8253c-6ff8-4714-97f1-536bd27a8cce tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:29:40.999384 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-04a8253c-6ff8-4714-97f1-536bd27a8cce tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] [instance: 08a33911-6e47-45fd-b499-155f199aacf8] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:29:41.006766 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-04a8253c-6ff8-4714-97f1-536bd27a8cce tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:41.007494 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-04a8253c-6ff8-4714-97f1-536bd27a8cce tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:41.009124 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-04a8253c-6ff8-4714-97f1-536bd27a8cce tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:41.009807 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-04a8253c-6ff8-4714-97f1-536bd27a8cce tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:41.389034 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-04a8253c-6ff8-4714-97f1-536bd27a8cce tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:41.410040 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-04a8253c-6ff8-4714-97f1-536bd27a8cce tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:41.412646 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-04a8253c-6ff8-4714-97f1-536bd27a8cce tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:41.414933 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-04a8253c-6ff8-4714-97f1-536bd27a8cce tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] 10.222.0.16 "GET /compute/v2.1/servers/08a33911-6e47-45fd-b499-155f199aacf8" status: 200 len: 1739 microversion: 2.20 time: 0.419672 Jul 16 21:29:41.416114 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2190/4387] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:29:40 2022] GET /compute/v2.1/servers/08a33911-6e47-45fd-b499-155f199aacf8 => generated 1739 bytes in 422 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:29:41.461669 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-304bdf3e-9ecc-4883-85e8-6e2b472bde74 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:29:41.462302 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-304bdf3e-9ecc-4883-85e8-6e2b472bde74 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] [instance: fed8328c-f9d3-4f4a-97c9-4919a94e6106] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:29:41.491091 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-304bdf3e-9ecc-4883-85e8-6e2b472bde74 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:41.491986 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-304bdf3e-9ecc-4883-85e8-6e2b472bde74 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:41.493279 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-304bdf3e-9ecc-4883-85e8-6e2b472bde74 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:41.493893 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-304bdf3e-9ecc-4883-85e8-6e2b472bde74 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:41.564647 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-304bdf3e-9ecc-4883-85e8-6e2b472bde74 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] HTTP exception thrown: Instance fed8328c-f9d3-4f4a-97c9-4919a94e6106 could not be found. Jul 16 21:29:41.569594 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-304bdf3e-9ecc-4883-85e8-6e2b472bde74 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Request method failure captured: Jul 16 21:29:41.569594 ncdv-master-1 devstack@n-api.service[138681]: request: GET /compute/v2.1/servers/fed8328c-f9d3-4f4a-97c9-4919a94e6106 HTTP/1.1 Jul 16 21:29:41.569594 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:29:41.569594 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:29:41.569594 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:29:41.569594 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 21:29:41.569594 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:29:41.569594 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:29:41.569594 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:29:41.569594 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:29:41.569594 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0y2WNUqZlY8Bx7OE6Ksdt8NKfaxQFHqgSfQuQnJCeYpEDVIMOYS4l1nIXjrYAIqXzuJ2WbO2qc4tRfJbOxRECSuS_EvYzrUAv7hKLjOPhFML9EPZ7lxTX1vcu6Tvi1gdRKEDiwyOoy--r7kLFa24OVE-T-F6FRl6bUZ27kRhbxB-IJE Jul 16 21:29:41.569594 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:29:41.569594 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:29:41.569594 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:29:41.569594 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:29:41.569594 ncdv-master-1 devstack@n-api.service[138681]: X-Openstack-Nova-Api-Version: 2.57 Jul 16 21:29:41.569594 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:29:41.569594 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:29:41.569594 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: 3c6f089da37e4ff09784a2b42513ff36 Jul 16 21:29:41.569594 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-ServerShowV257Test-409263129 Jul 16 21:29:41.569594 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:29:41.569594 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:29:41.569594 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/3c6f089da37e4ff09784a2b42513ff36"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/3c6f089da37e4ff09784a2b42513ff36"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_3c6f089da37e4ff09784a2b42513ff36"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/3c6f089da37e4ff09784a2b42513ff36"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:29:41.569594 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-ServerShowV257Test-409263129 Jul 16 21:29:41.569594 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: 3c6f089da37e4ff09784a2b42513ff36 Jul 16 21:29:41.569594 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-ServerShowV257Test-409263129 Jul 16 21:29:41.569594 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-ServerShowV257Test-409263129-project Jul 16 21:29:41.569594 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:29:41.569594 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:29:41.569594 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: af0cd3652c8d42bcb8af08852ff4f203 Jul 16 21:29:41.569594 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-ServerShowV257Test-409263129-project Jul 16 21:29:41.569594 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:29:41.569594 ncdv-master-1 devstack@n-api.service[138681]: exception: Instance fed8328c-f9d3-4f4a-97c9-4919a94e6106 could not be found. Jul 16 21:29:41.569594 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': 'fed8328c-f9d3-4f4a-97c9-4919a94e6106'} Jul 16 21:29:41.569594 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:29:41.569594 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:29:41.569594 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:29:41.569594 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:29:41.569594 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:29:41.569594 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:29:41.569594 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:29:41.569594 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:29:41.569594 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:29:41.569594 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:29:41.569594 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance fed8328c-f9d3-4f4a-97c9-4919a94e6106 could not be found. Jul 16 21:29:41.569594 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:29:41.569594 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:29:41.569594 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:29:41.569594 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:29:41.569594 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:29:41.569594 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:29:41.569594 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:29:41.569594 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:29:41.569594 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:29:41.569594 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:29:41.569594 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:29:41.569594 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:29:41.569594 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:29:41.569594 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:29:41.569594 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:29:41.569594 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:29:41.569594 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance fed8328c-f9d3-4f4a-97c9-4919a94e6106 could not be found. Jul 16 21:29:41.569594 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:29:41.569594 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:29:41.569594 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:29:41.579751 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:29:41.579751 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:29:41.579751 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:29:41.579751 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:29:41.579751 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:29:41.579751 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance fed8328c-f9d3-4f4a-97c9-4919a94e6106 could not be found. Jul 16 21:29:41.579751 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:29:41.579751 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-304bdf3e-9ecc-4883-85e8-6e2b472bde74 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] Returning 404 to user: Instance fed8328c-f9d3-4f4a-97c9-4919a94e6106 could not be found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:29:41.582856 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-304bdf3e-9ecc-4883-85e8-6e2b472bde74 tempest-ServerShowV257Test-409263129 tempest-ServerShowV257Test-409263129-project] 10.222.0.16 "GET /compute/v2.1/servers/fed8328c-f9d3-4f4a-97c9-4919a94e6106" status: 404 len: 111 microversion: 2.57 time: 0.125689 Jul 16 21:29:41.585622 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2198/4388] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:29:41 2022] GET /compute/v2.1/servers/fed8328c-f9d3-4f4a-97c9-4919a94e6106 => generated 111 bytes in 129 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Jul 16 21:29:42.085713 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-5277e22e-c8e8-441c-b288-80962ce86425 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Action: 'action', calling method: >, body: {"os-getConsoleOutput": {}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:29:42.088158 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-5277e22e-c8e8-441c-b288-80962ce86425 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] [instance: 4eb3d8d6-79ae-4109-a5a7-835c459d1719] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:29:42.109451 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5277e22e-c8e8-441c-b288-80962ce86425 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:42.110001 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5277e22e-c8e8-441c-b288-80962ce86425 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:42.110850 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5277e22e-c8e8-441c-b288-80962ce86425 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:42.111402 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5277e22e-c8e8-441c-b288-80962ce86425 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:42.245323 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-5277e22e-c8e8-441c-b288-80962ce86425 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] 10.222.0.16 "POST /compute/v2.1/servers/4eb3d8d6-79ae-4109-a5a7-835c459d1719/action" status: 200 len: 24578 microversion: 2.32 time: 0.619600 Jul 16 21:29:42.246858 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2191/4389] 10.222.0.16 () {62 vars in 1318 bytes} [Sat Jul 16 21:29:41 2022] POST /compute/v2.1/servers/4eb3d8d6-79ae-4109-a5a7-835c459d1719/action => generated 24578 bytes in 623 msecs (HTTP/1.1 200) 9 headers in 361 bytes (1 switches on core 0) Jul 16 21:29:42.436925 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-fbd87d7e-5e0b-4041-93a0-5d014241d660 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:29:42.437716 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-fbd87d7e-5e0b-4041-93a0-5d014241d660 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] [instance: 08a33911-6e47-45fd-b499-155f199aacf8] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:29:42.463478 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-fbd87d7e-5e0b-4041-93a0-5d014241d660 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:42.463478 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-fbd87d7e-5e0b-4041-93a0-5d014241d660 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:42.464776 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-fbd87d7e-5e0b-4041-93a0-5d014241d660 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:42.465710 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-fbd87d7e-5e0b-4041-93a0-5d014241d660 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:42.706870 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-046c3b16-42c5-4769-b445-492c4b8ad334 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerRescueTestJSON-server-1307601716", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "}]}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:29:42.834526 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.network.neutron [None req-046c3b16-42c5-4769-b445-492c4b8ad334 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] validate_networks() for [('623041c3-6f6a-4dae-9458-468726d1421e', None, None, None, None, None)] {{(pid=138681) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 16 21:29:43.005487 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-fbd87d7e-5e0b-4041-93a0-5d014241d660 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:43.023379 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-fbd87d7e-5e0b-4041-93a0-5d014241d660 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:43.028690 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-fbd87d7e-5e0b-4041-93a0-5d014241d660 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:43.030546 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-fbd87d7e-5e0b-4041-93a0-5d014241d660 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] 10.222.0.16 "GET /compute/v2.1/servers/08a33911-6e47-45fd-b499-155f199aacf8" status: 200 len: 1739 microversion: 2.20 time: 0.600450 Jul 16 21:29:43.031171 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2192/4390] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:29:42 2022] GET /compute/v2.1/servers/08a33911-6e47-45fd-b499-155f199aacf8 => generated 1739 bytes in 602 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:29:43.383333 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-43d372bd-a391-4733-9b8d-e974722be3c7 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Action: 'action', calling method: >, body: {"os-getConsoleOutput": {}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:29:43.385266 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-43d372bd-a391-4733-9b8d-e974722be3c7 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] [instance: 59a4c34e-3184-465b-b14c-0ad55192c8ce] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:29:43.395904 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-43d372bd-a391-4733-9b8d-e974722be3c7 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:43.399562 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-43d372bd-a391-4733-9b8d-e974722be3c7 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:43.407674 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-43d372bd-a391-4733-9b8d-e974722be3c7 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:43.407674 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-43d372bd-a391-4733-9b8d-e974722be3c7 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:43.477079 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-43d372bd-a391-4733-9b8d-e974722be3c7 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] 10.222.0.16 "POST /compute/v2.1/servers/59a4c34e-3184-465b-b14c-0ad55192c8ce/action" status: 200 len: 24578 microversion: 2.1 time: 0.098035 Jul 16 21:29:43.491768 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2193/4391] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 21:29:43 2022] POST /compute/v2.1/servers/59a4c34e-3184-465b-b14c-0ad55192c8ce/action => generated 24578 bytes in 113 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 16 21:29:43.781496 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-046c3b16-42c5-4769-b445-492c4b8ad334 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Flavor limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 16 21:29:43.782588 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-046c3b16-42c5-4769-b445-492c4b8ad334 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Image limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 16 21:29:43.783303 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-046c3b16-42c5-4769-b445-492c4b8ad334 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Flavor pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 16 21:29:43.783887 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-046c3b16-42c5-4769-b445-492c4b8ad334 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Image pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 16 21:29:43.784498 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-046c3b16-42c5-4769-b445-492c4b8ad334 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 16 21:29:43.816478 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-046c3b16-42c5-4769-b445-492c4b8ad334 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:43.827186 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-046c3b16-42c5-4769-b445-492c4b8ad334 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.009s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:43.828335 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-046c3b16-42c5-4769-b445-492c4b8ad334 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:43.832085 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-046c3b16-42c5-4769-b445-492c4b8ad334 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:43.891879 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-046c3b16-42c5-4769-b445-492c4b8ad334 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Getting quotas for project 207ec2d1883c4a1ba89696e6e77cf3aa. Resources: {'ram', 'instances', 'cores'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:29:43.901402 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-046c3b16-42c5-4769-b445-492c4b8ad334 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Getting quotas for user da77ec3d0f04496d8c8df4025bba76d1 and project 207ec2d1883c4a1ba89696e6e77cf3aa. Resources: {'ram', 'instances', 'cores'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:29:43.930487 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-046c3b16-42c5-4769-b445-492c4b8ad334 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Going to run 1 instances... {{(pid=138681) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 16 21:29:43.933416 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-046c3b16-42c5-4769-b445-492c4b8ad334 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] [instance: 515fd3da-3965-4882-b9e8-100a7eea64d9] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138681) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 16 21:29:44.054402 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-81cd616b-9c90-44c3-b9c7-3fbb281d1fa0 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:29:44.055174 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-81cd616b-9c90-44c3-b9c7-3fbb281d1fa0 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] [instance: 08a33911-6e47-45fd-b499-155f199aacf8] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:29:44.075232 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-81cd616b-9c90-44c3-b9c7-3fbb281d1fa0 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:44.082440 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-81cd616b-9c90-44c3-b9c7-3fbb281d1fa0 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:44.088602 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-81cd616b-9c90-44c3-b9c7-3fbb281d1fa0 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:44.089197 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-81cd616b-9c90-44c3-b9c7-3fbb281d1fa0 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:44.179818 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-046c3b16-42c5-4769-b445-492c4b8ad334 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 383 microversion: 2.1 time: 1.886320 Jul 16 21:29:44.184527 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2199/4392] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 21:29:42 2022] POST /compute/v2.1/servers => generated 383 bytes in 1892 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 16 21:29:44.192070 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-90198122-91ec-4433-854d-fac9332c9bff tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:29:44.195884 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-90198122-91ec-4433-854d-fac9332c9bff tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] [instance: 515fd3da-3965-4882-b9e8-100a7eea64d9] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:29:44.453018 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-90198122-91ec-4433-854d-fac9332c9bff tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da77ec3d0f04496d8c8df4025bba76d1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '207ec2d1883c4a1ba89696e6e77cf3aa', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:44.463854 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-90198122-91ec-4433-854d-fac9332c9bff tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] 10.222.0.16 "GET /compute/v2.1/servers/515fd3da-3965-4882-b9e8-100a7eea64d9" status: 200 len: 1214 microversion: 2.1 time: 0.273431 Jul 16 21:29:44.464863 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2200/4393] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:29:44 2022] GET /compute/v2.1/servers/515fd3da-3965-4882-b9e8-100a7eea64d9 => generated 1214 bytes in 277 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:29:45.373152 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-81cd616b-9c90-44c3-b9c7-3fbb281d1fa0 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:45.412356 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-81cd616b-9c90-44c3-b9c7-3fbb281d1fa0 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:45.414190 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-81cd616b-9c90-44c3-b9c7-3fbb281d1fa0 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:45.431157 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-81cd616b-9c90-44c3-b9c7-3fbb281d1fa0 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] 10.222.0.16 "GET /compute/v2.1/servers/08a33911-6e47-45fd-b499-155f199aacf8" status: 200 len: 1739 microversion: 2.20 time: 1.365526 Jul 16 21:29:45.432431 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2194/4394] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:29:44 2022] GET /compute/v2.1/servers/08a33911-6e47-45fd-b499-155f199aacf8 => generated 1739 bytes in 1383 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:29:45.480694 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-2918c538-2ae1-4e23-b675-16e1692aadcd tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:29:45.481938 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-2918c538-2ae1-4e23-b675-16e1692aadcd tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] [instance: 515fd3da-3965-4882-b9e8-100a7eea64d9] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:29:45.659865 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-2918c538-2ae1-4e23-b675-16e1692aadcd tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da77ec3d0f04496d8c8df4025bba76d1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '207ec2d1883c4a1ba89696e6e77cf3aa', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:45.696503 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-2918c538-2ae1-4e23-b675-16e1692aadcd tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] 10.222.0.16 "GET /compute/v2.1/servers/515fd3da-3965-4882-b9e8-100a7eea64d9" status: 200 len: 1214 microversion: 2.1 time: 0.219607 Jul 16 21:29:45.723593 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2201/4395] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:29:45 2022] GET /compute/v2.1/servers/515fd3da-3965-4882-b9e8-100a7eea64d9 => generated 1214 bytes in 247 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:29:46.454658 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-f8f70bd8-15f0-447b-ae49-0f5e5189fa9e tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:29:46.456090 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-f8f70bd8-15f0-447b-ae49-0f5e5189fa9e tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] [instance: 08a33911-6e47-45fd-b499-155f199aacf8] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:29:46.466447 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f8f70bd8-15f0-447b-ae49-0f5e5189fa9e tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:46.467298 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f8f70bd8-15f0-447b-ae49-0f5e5189fa9e tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:46.468508 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f8f70bd8-15f0-447b-ae49-0f5e5189fa9e tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:46.469196 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f8f70bd8-15f0-447b-ae49-0f5e5189fa9e tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:46.724090 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-9d489fee-16d9-436a-8472-93169110cdf8 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:29:46.726554 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-9d489fee-16d9-436a-8472-93169110cdf8 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] [instance: 515fd3da-3965-4882-b9e8-100a7eea64d9] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:29:46.741733 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9d489fee-16d9-436a-8472-93169110cdf8 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:46.746722 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9d489fee-16d9-436a-8472-93169110cdf8 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:46.748159 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9d489fee-16d9-436a-8472-93169110cdf8 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:46.749510 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9d489fee-16d9-436a-8472-93169110cdf8 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:46.934712 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-9d489fee-16d9-436a-8472-93169110cdf8 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da77ec3d0f04496d8c8df4025bba76d1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '207ec2d1883c4a1ba89696e6e77cf3aa', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:46.941781 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-f8f70bd8-15f0-447b-ae49-0f5e5189fa9e tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:46.963504 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-f8f70bd8-15f0-447b-ae49-0f5e5189fa9e tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:46.966299 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-9d489fee-16d9-436a-8472-93169110cdf8 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] 10.222.0.16 "GET /compute/v2.1/servers/515fd3da-3965-4882-b9e8-100a7eea64d9" status: 200 len: 1210 microversion: 2.1 time: 0.245397 Jul 16 21:29:46.966299 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2202/4396] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:29:46 2022] GET /compute/v2.1/servers/515fd3da-3965-4882-b9e8-100a7eea64d9 => generated 1210 bytes in 253 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:29:46.980719 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-f8f70bd8-15f0-447b-ae49-0f5e5189fa9e tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:46.984731 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-f8f70bd8-15f0-447b-ae49-0f5e5189fa9e tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] 10.222.0.16 "GET /compute/v2.1/servers/08a33911-6e47-45fd-b499-155f199aacf8" status: 200 len: 1739 microversion: 2.20 time: 0.535363 Jul 16 21:29:46.987734 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2195/4397] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:29:46 2022] GET /compute/v2.1/servers/08a33911-6e47-45fd-b499-155f199aacf8 => generated 1739 bytes in 539 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:29:47.986778 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-0ce5eed6-abeb-44b0-a3a3-af328a7ade31 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:29:47.988055 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-0ce5eed6-abeb-44b0-a3a3-af328a7ade31 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] [instance: 515fd3da-3965-4882-b9e8-100a7eea64d9] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:29:48.004671 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-5e04e25f-8be2-470e-868f-41ce3def20a6 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:29:48.006040 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-5e04e25f-8be2-470e-868f-41ce3def20a6 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] [instance: 08a33911-6e47-45fd-b499-155f199aacf8] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:29:48.012240 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0ce5eed6-abeb-44b0-a3a3-af328a7ade31 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:48.012702 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0ce5eed6-abeb-44b0-a3a3-af328a7ade31 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:48.017329 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0ce5eed6-abeb-44b0-a3a3-af328a7ade31 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:48.019553 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5e04e25f-8be2-470e-868f-41ce3def20a6 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:48.019696 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0ce5eed6-abeb-44b0-a3a3-af328a7ade31 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:48.021521 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5e04e25f-8be2-470e-868f-41ce3def20a6 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:48.029444 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5e04e25f-8be2-470e-868f-41ce3def20a6 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:48.029444 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5e04e25f-8be2-470e-868f-41ce3def20a6 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:48.398956 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-0ce5eed6-abeb-44b0-a3a3-af328a7ade31 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da77ec3d0f04496d8c8df4025bba76d1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '207ec2d1883c4a1ba89696e6e77cf3aa', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:48.413770 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-0ce5eed6-abeb-44b0-a3a3-af328a7ade31 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] 10.222.0.16 "GET /compute/v2.1/servers/515fd3da-3965-4882-b9e8-100a7eea64d9" status: 200 len: 1284 microversion: 2.1 time: 0.435808 Jul 16 21:29:48.428451 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2203/4398] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:29:47 2022] GET /compute/v2.1/servers/515fd3da-3965-4882-b9e8-100a7eea64d9 => generated 1284 bytes in 451 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:29:48.525556 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-5e04e25f-8be2-470e-868f-41ce3def20a6 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:48.538685 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-5e04e25f-8be2-470e-868f-41ce3def20a6 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:48.540265 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-5e04e25f-8be2-470e-868f-41ce3def20a6 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:48.541470 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-5e04e25f-8be2-470e-868f-41ce3def20a6 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] 10.222.0.16 "GET /compute/v2.1/servers/08a33911-6e47-45fd-b499-155f199aacf8" status: 200 len: 1739 microversion: 2.20 time: 0.541372 Jul 16 21:29:48.559765 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2196/4399] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:29:47 2022] GET /compute/v2.1/servers/08a33911-6e47-45fd-b499-155f199aacf8 => generated 1739 bytes in 560 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:29:49.438464 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-2efa32c2-186f-40a5-b0d6-cf73810344ad tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:29:49.439761 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-2efa32c2-186f-40a5-b0d6-cf73810344ad tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] [instance: 515fd3da-3965-4882-b9e8-100a7eea64d9] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:29:49.469513 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2efa32c2-186f-40a5-b0d6-cf73810344ad tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:49.474599 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2efa32c2-186f-40a5-b0d6-cf73810344ad tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:49.475863 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2efa32c2-186f-40a5-b0d6-cf73810344ad tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:49.478541 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2efa32c2-186f-40a5-b0d6-cf73810344ad tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:49.561738 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-e350ecde-d993-4504-b922-8ac3c9f31509 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:29:49.562494 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-e350ecde-d993-4504-b922-8ac3c9f31509 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] [instance: 08a33911-6e47-45fd-b499-155f199aacf8] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:29:49.580610 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_api:SELECT=94,nova_api:INSERT=2,nova_api:DELETE=3,nova_api:UPDATE=1 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:29:49.590174 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e350ecde-d993-4504-b922-8ac3c9f31509 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:49.590982 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e350ecde-d993-4504-b922-8ac3c9f31509 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:49.591950 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e350ecde-d993-4504-b922-8ac3c9f31509 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:49.594433 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e350ecde-d993-4504-b922-8ac3c9f31509 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:49.619504 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:29:49.684651 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-2efa32c2-186f-40a5-b0d6-cf73810344ad tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da77ec3d0f04496d8c8df4025bba76d1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '207ec2d1883c4a1ba89696e6e77cf3aa', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:49.693518 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-2efa32c2-186f-40a5-b0d6-cf73810344ad tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] 10.222.0.16 "GET /compute/v2.1/servers/515fd3da-3965-4882-b9e8-100a7eea64d9" status: 200 len: 1272 microversion: 2.1 time: 0.258902 Jul 16 21:29:49.700135 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2204/4400] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:29:49 2022] GET /compute/v2.1/servers/515fd3da-3965-4882-b9e8-100a7eea64d9 => generated 1272 bytes in 267 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:29:50.225507 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-e350ecde-d993-4504-b922-8ac3c9f31509 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:50.241503 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-e350ecde-d993-4504-b922-8ac3c9f31509 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:50.243403 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-e350ecde-d993-4504-b922-8ac3c9f31509 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:50.245612 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-e350ecde-d993-4504-b922-8ac3c9f31509 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] 10.222.0.16 "GET /compute/v2.1/servers/08a33911-6e47-45fd-b499-155f199aacf8" status: 200 len: 1739 microversion: 2.20 time: 0.688116 Jul 16 21:29:50.258895 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2197/4401] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:29:49 2022] GET /compute/v2.1/servers/08a33911-6e47-45fd-b499-155f199aacf8 => generated 1739 bytes in 702 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:29:50.724217 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-a9a736c2-e86d-4bf2-b9f0-eb92d18e4e31 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:29:50.724217 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-a9a736c2-e86d-4bf2-b9f0-eb92d18e4e31 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] [instance: 515fd3da-3965-4882-b9e8-100a7eea64d9] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:29:50.738095 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a9a736c2-e86d-4bf2-b9f0-eb92d18e4e31 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:50.744351 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a9a736c2-e86d-4bf2-b9f0-eb92d18e4e31 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:50.747692 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a9a736c2-e86d-4bf2-b9f0-eb92d18e4e31 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:50.748236 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a9a736c2-e86d-4bf2-b9f0-eb92d18e4e31 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:51.266491 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-918ceca0-bd59-4fa8-b81e-723306ec352e tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:29:51.267146 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-918ceca0-bd59-4fa8-b81e-723306ec352e tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] [instance: 08a33911-6e47-45fd-b499-155f199aacf8] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:29:51.286374 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-918ceca0-bd59-4fa8-b81e-723306ec352e tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:51.292910 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-918ceca0-bd59-4fa8-b81e-723306ec352e tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:51.296455 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-918ceca0-bd59-4fa8-b81e-723306ec352e tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:51.299744 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-918ceca0-bd59-4fa8-b81e-723306ec352e tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:51.663791 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-a9a736c2-e86d-4bf2-b9f0-eb92d18e4e31 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da77ec3d0f04496d8c8df4025bba76d1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '207ec2d1883c4a1ba89696e6e77cf3aa', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:51.677734 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-a9a736c2-e86d-4bf2-b9f0-eb92d18e4e31 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] 10.222.0.16 "GET /compute/v2.1/servers/515fd3da-3965-4882-b9e8-100a7eea64d9" status: 200 len: 1314 microversion: 2.1 time: 0.962516 Jul 16 21:29:51.679640 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2205/4402] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:29:50 2022] GET /compute/v2.1/servers/515fd3da-3965-4882-b9e8-100a7eea64d9 => generated 1314 bytes in 967 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:29:51.819927 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-918ceca0-bd59-4fa8-b81e-723306ec352e tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:51.851636 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-918ceca0-bd59-4fa8-b81e-723306ec352e tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:51.851636 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-918ceca0-bd59-4fa8-b81e-723306ec352e tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:51.852447 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-918ceca0-bd59-4fa8-b81e-723306ec352e tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] 10.222.0.16 "GET /compute/v2.1/servers/08a33911-6e47-45fd-b499-155f199aacf8" status: 200 len: 1739 microversion: 2.20 time: 0.590409 Jul 16 21:29:51.853243 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2198/4403] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:29:51 2022] GET /compute/v2.1/servers/08a33911-6e47-45fd-b499-155f199aacf8 => generated 1739 bytes in 593 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:29:52.724194 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-ef0b0d55-67d8-475e-943e-7b29ab397804 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:29:52.724844 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-ef0b0d55-67d8-475e-943e-7b29ab397804 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] [instance: 515fd3da-3965-4882-b9e8-100a7eea64d9] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:29:52.756008 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ef0b0d55-67d8-475e-943e-7b29ab397804 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:52.756575 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ef0b0d55-67d8-475e-943e-7b29ab397804 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:52.760239 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ef0b0d55-67d8-475e-943e-7b29ab397804 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:52.761402 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ef0b0d55-67d8-475e-943e-7b29ab397804 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:52.872410 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-2f20ec0f-aa6f-4438-9c76-e0b119022a84 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:29:52.873038 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-2f20ec0f-aa6f-4438-9c76-e0b119022a84 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] [instance: 08a33911-6e47-45fd-b499-155f199aacf8] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:29:52.892651 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2f20ec0f-aa6f-4438-9c76-e0b119022a84 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:52.893119 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2f20ec0f-aa6f-4438-9c76-e0b119022a84 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:52.893920 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2f20ec0f-aa6f-4438-9c76-e0b119022a84 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:52.894370 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2f20ec0f-aa6f-4438-9c76-e0b119022a84 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:53.515459 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-ef0b0d55-67d8-475e-943e-7b29ab397804 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da77ec3d0f04496d8c8df4025bba76d1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '207ec2d1883c4a1ba89696e6e77cf3aa', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:53.529391 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-ef0b0d55-67d8-475e-943e-7b29ab397804 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] 10.222.0.16 "GET /compute/v2.1/servers/515fd3da-3965-4882-b9e8-100a7eea64d9" status: 200 len: 1314 microversion: 2.1 time: 0.836310 Jul 16 21:29:53.539254 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2206/4404] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:29:52 2022] GET /compute/v2.1/servers/515fd3da-3965-4882-b9e8-100a7eea64d9 => generated 1314 bytes in 847 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:29:53.547995 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-4b849da8-3983-47e5-9fdd-9d53da233375 req-e50ba892-c74e-4aba-b3a8-eda97f19c280 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "515fd3da-3965-4882-b9e8-100a7eea64d9", "tag": "f74e70f6-9bde-4b10-9973-eedb2194871b"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:29:53.560541 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-4b849da8-3983-47e5-9fdd-9d53da233375 req-e50ba892-c74e-4aba-b3a8-eda97f19c280 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:53.561343 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-4b849da8-3983-47e5-9fdd-9d53da233375 req-e50ba892-c74e-4aba-b3a8-eda97f19c280 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:53.588408 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-4b849da8-3983-47e5-9fdd-9d53da233375 req-e50ba892-c74e-4aba-b3a8-eda97f19c280 service nova] Creating event network-changed:f74e70f6-9bde-4b10-9973-eedb2194871b for instance 515fd3da-3965-4882-b9e8-100a7eea64d9 on nch1-master-1 Jul 16 21:29:53.604299 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-4b849da8-3983-47e5-9fdd-9d53da233375 req-e50ba892-c74e-4aba-b3a8-eda97f19c280 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.063341 Jul 16 21:29:53.613286 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-2f20ec0f-aa6f-4438-9c76-e0b119022a84 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:53.620208 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2207/4405] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:29:53 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 80 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:29:53.631965 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-2f20ec0f-aa6f-4438-9c76-e0b119022a84 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:53.633891 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-2f20ec0f-aa6f-4438-9c76-e0b119022a84 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:53.639259 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-2f20ec0f-aa6f-4438-9c76-e0b119022a84 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] 10.222.0.16 "GET /compute/v2.1/servers/08a33911-6e47-45fd-b499-155f199aacf8" status: 200 len: 1739 microversion: 2.20 time: 0.771256 Jul 16 21:29:53.649715 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2199/4406] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:29:52 2022] GET /compute/v2.1/servers/08a33911-6e47-45fd-b499-155f199aacf8 => generated 1739 bytes in 782 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 16 21:29:54.544497 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell0:SELECT=2 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:29:54.549039 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-f8cf7a80-3b92-4f0d-94e3-a9fdf9b76475 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:29:54.550142 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-f8cf7a80-3b92-4f0d-94e3-a9fdf9b76475 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] [instance: 515fd3da-3965-4882-b9e8-100a7eea64d9] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:29:54.569467 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f8cf7a80-3b92-4f0d-94e3-a9fdf9b76475 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:54.570391 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f8cf7a80-3b92-4f0d-94e3-a9fdf9b76475 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:54.571450 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f8cf7a80-3b92-4f0d-94e3-a9fdf9b76475 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:54.573339 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f8cf7a80-3b92-4f0d-94e3-a9fdf9b76475 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:54.676217 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-2428496e-3d21-4a54-ba01-d330b36df0cd tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:29:54.678003 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-2428496e-3d21-4a54-ba01-d330b36df0cd tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] [instance: 08a33911-6e47-45fd-b499-155f199aacf8] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:29:54.689901 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2428496e-3d21-4a54-ba01-d330b36df0cd tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:54.694753 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2428496e-3d21-4a54-ba01-d330b36df0cd tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:54.696153 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2428496e-3d21-4a54-ba01-d330b36df0cd tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:54.696821 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2428496e-3d21-4a54-ba01-d330b36df0cd tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:55.359791 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-f8cf7a80-3b92-4f0d-94e3-a9fdf9b76475 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da77ec3d0f04496d8c8df4025bba76d1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '207ec2d1883c4a1ba89696e6e77cf3aa', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:55.379238 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-f8cf7a80-3b92-4f0d-94e3-a9fdf9b76475 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] 10.222.0.16 "GET /compute/v2.1/servers/515fd3da-3965-4882-b9e8-100a7eea64d9" status: 200 len: 1314 microversion: 2.1 time: 0.836135 Jul 16 21:29:55.401547 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-2428496e-3d21-4a54-ba01-d330b36df0cd tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:55.410812 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2208/4407] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:29:54 2022] GET /compute/v2.1/servers/515fd3da-3965-4882-b9e8-100a7eea64d9 => generated 1314 bytes in 869 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:29:55.414087 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-2428496e-3d21-4a54-ba01-d330b36df0cd tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:55.417705 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-2428496e-3d21-4a54-ba01-d330b36df0cd tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:55.420496 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-2428496e-3d21-4a54-ba01-d330b36df0cd tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] 10.222.0.16 "GET /compute/v2.1/servers/08a33911-6e47-45fd-b499-155f199aacf8" status: 200 len: 1739 microversion: 2.20 time: 0.749586 Jul 16 21:29:55.421638 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2200/4408] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:29:54 2022] GET /compute/v2.1/servers/08a33911-6e47-45fd-b499-155f199aacf8 => generated 1739 bytes in 752 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:29:56.405283 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-d38b240f-ed9e-42f8-882c-88a498274f84 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:29:56.411781 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-d38b240f-ed9e-42f8-882c-88a498274f84 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] [instance: 515fd3da-3965-4882-b9e8-100a7eea64d9] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:29:56.414759 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell0:SELECT=10 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:29:56.430950 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d38b240f-ed9e-42f8-882c-88a498274f84 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:56.433593 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d38b240f-ed9e-42f8-882c-88a498274f84 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:56.442221 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-7181e229-17dd-4f07-8446-889c856fbc9b tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:29:56.443624 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-7181e229-17dd-4f07-8446-889c856fbc9b tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] [instance: 08a33911-6e47-45fd-b499-155f199aacf8] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:29:56.457689 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d38b240f-ed9e-42f8-882c-88a498274f84 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:56.466578 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d38b240f-ed9e-42f8-882c-88a498274f84 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:56.475602 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7181e229-17dd-4f07-8446-889c856fbc9b tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:56.478670 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7181e229-17dd-4f07-8446-889c856fbc9b tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:56.481026 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7181e229-17dd-4f07-8446-889c856fbc9b tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:56.485268 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7181e229-17dd-4f07-8446-889c856fbc9b tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:56.859171 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-d38b240f-ed9e-42f8-882c-88a498274f84 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da77ec3d0f04496d8c8df4025bba76d1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '207ec2d1883c4a1ba89696e6e77cf3aa', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:56.877355 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-d38b240f-ed9e-42f8-882c-88a498274f84 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] 10.222.0.16 "GET /compute/v2.1/servers/515fd3da-3965-4882-b9e8-100a7eea64d9" status: 200 len: 1314 microversion: 2.1 time: 0.477301 Jul 16 21:29:56.888956 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-7181e229-17dd-4f07-8446-889c856fbc9b tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:56.889728 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2209/4409] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:29:56 2022] GET /compute/v2.1/servers/515fd3da-3965-4882-b9e8-100a7eea64d9 => generated 1314 bytes in 491 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:29:56.918066 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-7181e229-17dd-4f07-8446-889c856fbc9b tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:56.926319 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-7181e229-17dd-4f07-8446-889c856fbc9b tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:56.935697 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-7181e229-17dd-4f07-8446-889c856fbc9b tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] 10.222.0.16 "GET /compute/v2.1/servers/08a33911-6e47-45fd-b499-155f199aacf8" status: 200 len: 1739 microversion: 2.20 time: 0.496068 Jul 16 21:29:56.938337 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2201/4410] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:29:56 2022] GET /compute/v2.1/servers/08a33911-6e47-45fd-b499-155f199aacf8 => generated 1739 bytes in 503 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 16 21:29:57.899103 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-135c7ac0-eb6c-48a3-918b-6a46ce1624be tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:29:57.900139 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-135c7ac0-eb6c-48a3-918b-6a46ce1624be tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] [instance: 515fd3da-3965-4882-b9e8-100a7eea64d9] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:29:57.915894 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-135c7ac0-eb6c-48a3-918b-6a46ce1624be tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:57.916599 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-135c7ac0-eb6c-48a3-918b-6a46ce1624be tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:57.917548 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-135c7ac0-eb6c-48a3-918b-6a46ce1624be tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:57.918492 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-135c7ac0-eb6c-48a3-918b-6a46ce1624be tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:57.955608 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-696de268-7fbe-4563-b839-63023eae8fa8 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:29:57.956287 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-696de268-7fbe-4563-b839-63023eae8fa8 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] [instance: 08a33911-6e47-45fd-b499-155f199aacf8] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:29:57.978826 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-696de268-7fbe-4563-b839-63023eae8fa8 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:57.980048 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-696de268-7fbe-4563-b839-63023eae8fa8 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:57.987059 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-696de268-7fbe-4563-b839-63023eae8fa8 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:29:57.987631 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-696de268-7fbe-4563-b839-63023eae8fa8 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:29:59.032106 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-696de268-7fbe-4563-b839-63023eae8fa8 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:59.038793 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-135c7ac0-eb6c-48a3-918b-6a46ce1624be tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da77ec3d0f04496d8c8df4025bba76d1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '207ec2d1883c4a1ba89696e6e77cf3aa', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:59.041266 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-696de268-7fbe-4563-b839-63023eae8fa8 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:59.048234 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-696de268-7fbe-4563-b839-63023eae8fa8 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:29:59.050232 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-135c7ac0-eb6c-48a3-918b-6a46ce1624be tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] 10.222.0.16 "GET /compute/v2.1/servers/515fd3da-3965-4882-b9e8-100a7eea64d9" status: 200 len: 1314 microversion: 2.1 time: 1.158362 Jul 16 21:29:59.056456 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2210/4411] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:29:57 2022] GET /compute/v2.1/servers/515fd3da-3965-4882-b9e8-100a7eea64d9 => generated 1314 bytes in 1165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:29:59.058328 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-135c7ac0-eb6c-48a3-918b-6a46ce1624be tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:29:59.059573 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-135c7ac0-eb6c-48a3-918b-6a46ce1624be tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] 10.222.0.16 "GET /compute//" status: 200 len: 377 microversion: - time: 0.002317 Jul 16 21:29:59.065682 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2211/4412] 10.222.0.16 () {56 vars in 885 bytes} [Sat Jul 16 21:29:59 2022] GET /compute/ => generated 377 bytes in 3 msecs (HTTP/1.1 200) 3 headers in 91 bytes (1 switches on core 0) Jul 16 21:29:59.065834 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-696de268-7fbe-4563-b839-63023eae8fa8 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] 10.222.0.16 "GET /compute/v2.1/servers/08a33911-6e47-45fd-b499-155f199aacf8" status: 200 len: 1739 microversion: 2.20 time: 1.098444 Jul 16 21:29:59.065834 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2202/4413] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:29:57 2022] GET /compute/v2.1/servers/08a33911-6e47-45fd-b499-155f199aacf8 => generated 1739 bytes in 1113 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 16 21:30:00.068029 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-ba2371df-175a-46bc-aeaf-ac9f94290431 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:30:00.069516 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-ba2371df-175a-46bc-aeaf-ac9f94290431 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] [instance: 515fd3da-3965-4882-b9e8-100a7eea64d9] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:30:00.081271 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ba2371df-175a-46bc-aeaf-ac9f94290431 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:30:00.084857 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ba2371df-175a-46bc-aeaf-ac9f94290431 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:30:00.085678 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ba2371df-175a-46bc-aeaf-ac9f94290431 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:30:00.086132 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ba2371df-175a-46bc-aeaf-ac9f94290431 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:30:00.089966 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-79ac94a6-4a6d-442c-accc-b2409934a506 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:30:00.091043 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-79ac94a6-4a6d-442c-accc-b2409934a506 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] [instance: 08a33911-6e47-45fd-b499-155f199aacf8] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:30:00.107926 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-79ac94a6-4a6d-442c-accc-b2409934a506 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:30:00.110869 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-79ac94a6-4a6d-442c-accc-b2409934a506 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:30:00.112030 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-79ac94a6-4a6d-442c-accc-b2409934a506 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:30:00.113518 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-79ac94a6-4a6d-442c-accc-b2409934a506 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:30:00.504099 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-79ac94a6-4a6d-442c-accc-b2409934a506 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:30:00.530923 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-ba2371df-175a-46bc-aeaf-ac9f94290431 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da77ec3d0f04496d8c8df4025bba76d1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '207ec2d1883c4a1ba89696e6e77cf3aa', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:30:00.539182 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-ba2371df-175a-46bc-aeaf-ac9f94290431 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] 10.222.0.16 "GET /compute/v2.1/servers/515fd3da-3965-4882-b9e8-100a7eea64d9" status: 200 len: 1314 microversion: 2.1 time: 0.475482 Jul 16 21:30:00.540114 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-79ac94a6-4a6d-442c-accc-b2409934a506 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:30:00.549916 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-79ac94a6-4a6d-442c-accc-b2409934a506 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:30:00.550847 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2212/4414] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:30:00 2022] GET /compute/v2.1/servers/515fd3da-3965-4882-b9e8-100a7eea64d9 => generated 1314 bytes in 488 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:30:00.556584 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-79ac94a6-4a6d-442c-accc-b2409934a506 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] 10.222.0.16 "GET /compute/v2.1/servers/08a33911-6e47-45fd-b499-155f199aacf8" status: 200 len: 1721 microversion: 2.20 time: 0.474482 Jul 16 21:30:00.560694 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2203/4415] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:30:00 2022] GET /compute/v2.1/servers/08a33911-6e47-45fd-b499-155f199aacf8 => generated 1721 bytes in 479 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:30:00.835486 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-5d729d10-f8fd-4f6c-accf-d523a358a7fb req-c92500d6-1a18-4796-b0b7-bb1a1b919001 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "515fd3da-3965-4882-b9e8-100a7eea64d9", "name": "network-vif-plugged", "status": "completed", "tag": "f74e70f6-9bde-4b10-9973-eedb2194871b"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:30:00.849001 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-5d729d10-f8fd-4f6c-accf-d523a358a7fb req-c92500d6-1a18-4796-b0b7-bb1a1b919001 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:30:00.851056 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-5d729d10-f8fd-4f6c-accf-d523a358a7fb req-c92500d6-1a18-4796-b0b7-bb1a1b919001 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:30:00.885042 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-5d729d10-f8fd-4f6c-accf-d523a358a7fb req-c92500d6-1a18-4796-b0b7-bb1a1b919001 service nova] Creating event network-vif-plugged:f74e70f6-9bde-4b10-9973-eedb2194871b for instance 515fd3da-3965-4882-b9e8-100a7eea64d9 on nch1-master-1 Jul 16 21:30:00.904256 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-5d729d10-f8fd-4f6c-accf-d523a358a7fb req-c92500d6-1a18-4796-b0b7-bb1a1b919001 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.072431 Jul 16 21:30:00.905261 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2213/4416] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:30:00 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 74 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:30:01.532240 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-eb0e0c3c-003d-4825-9397-761c411111b6 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Action: 'create', calling method: .version_select at 0x7fdd9cc35d30>, body: {"keypair": {"name": "tempest-TestNetworkBasicOps-913257510", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBI5oyxg9L1jtRsuwdsXwi1VpXoeg6B0F50yOcmsFhcUCE0zJBzKZYZ2YFzHcr33JaAyj+6PWi/qch0TM8FZmQiyQ3ivpGeeOLIJtMJLu/V5iPe7FzMgXcArth6C1n9QOIw=="}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:30:01.575953 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-eb0e0c3c-003d-4825-9397-761c411111b6 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Getting quotas for project bed7c94624484c7cb8abeb3bfc8757b9. Resources: {'key_pairs'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:30:01.595796 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-bbccda3f-15cc-4535-98fc-4683d4d26b9e tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:30:01.596625 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-eb0e0c3c-003d-4825-9397-761c411111b6 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Getting quotas for user 4ddfae3448704591b662de83b97a1876 and project bed7c94624484c7cb8abeb3bfc8757b9. Resources: {'key_pairs'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:30:01.612462 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-bbccda3f-15cc-4535-98fc-4683d4d26b9e tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] [instance: 515fd3da-3965-4882-b9e8-100a7eea64d9] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:30:01.650814 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_api:SELECT=69,nova_api:INSERT=3 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:30:01.654330 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-bbccda3f-15cc-4535-98fc-4683d4d26b9e tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:30:01.655619 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-bbccda3f-15cc-4535-98fc-4683d4d26b9e tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:30:01.656645 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-bbccda3f-15cc-4535-98fc-4683d4d26b9e tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:30:01.661997 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-bbccda3f-15cc-4535-98fc-4683d4d26b9e tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:30:01.729349 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-eb0e0c3c-003d-4825-9397-761c411111b6 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] 10.222.0.16 "POST /compute/v2.1/os-keypairs" status: 200 len: 397 microversion: 2.1 time: 0.201622 Jul 16 21:30:01.735186 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2204/4417] 10.222.0.16 () {60 vars in 1158 bytes} [Sat Jul 16 21:30:01 2022] POST /compute/v2.1/os-keypairs => generated 397 bytes in 203 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:30:01.735484 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-4208613d-55b0-4187-b4da-04f0e0813483 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:30:01.736116 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-4208613d-55b0-4187-b4da-04f0e0813483 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] [instance: 08a33911-6e47-45fd-b499-155f199aacf8] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:30:01.747510 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4208613d-55b0-4187-b4da-04f0e0813483 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:30:01.767781 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4208613d-55b0-4187-b4da-04f0e0813483 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:30:01.770179 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4208613d-55b0-4187-b4da-04f0e0813483 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:30:01.771400 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4208613d-55b0-4187-b4da-04f0e0813483 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:30:02.256666 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-bbccda3f-15cc-4535-98fc-4683d4d26b9e tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da77ec3d0f04496d8c8df4025bba76d1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '207ec2d1883c4a1ba89696e6e77cf3aa', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:30:02.266565 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-bbccda3f-15cc-4535-98fc-4683d4d26b9e tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] 10.222.0.16 "GET /compute/v2.1/servers/515fd3da-3965-4882-b9e8-100a7eea64d9" status: 200 len: 1314 microversion: 2.1 time: 0.681233 Jul 16 21:30:02.271387 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2214/4418] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:30:01 2022] GET /compute/v2.1/servers/515fd3da-3965-4882-b9e8-100a7eea64d9 => generated 1314 bytes in 687 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:30:02.938048 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-2df3a344-3289-467f-a144-f7e1e314d46d req-77f6d6b1-99bc-49ba-b1f1-a67da055520e service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "08a33911-6e47-45fd-b499-155f199aacf8", "name": "network-vif-unplugged", "status": "completed", "tag": "81e317e7-e9db-4ba9-b87d-9395b084ead7"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:30:02.956813 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-2df3a344-3289-467f-a144-f7e1e314d46d req-77f6d6b1-99bc-49ba-b1f1-a67da055520e service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:30:02.957534 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-2df3a344-3289-467f-a144-f7e1e314d46d req-77f6d6b1-99bc-49ba-b1f1-a67da055520e service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:30:02.979484 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.compute.server_external_events [req-2df3a344-3289-467f-a144-f7e1e314d46d req-77f6d6b1-99bc-49ba-b1f1-a67da055520e service nova] Unable to find a host for instance 08a33911-6e47-45fd-b499-155f199aacf8. Dropping event network-vif-unplugged {{(pid=138681) create /opt/stack/nova/nova/api/openstack/compute/server_external_events.py:143}} Jul 16 21:30:02.980578 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-2df3a344-3289-467f-a144-f7e1e314d46d req-77f6d6b1-99bc-49ba-b1f1-a67da055520e service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 207 len: 182 microversion: 2.1 time: 0.046408 Jul 16 21:30:02.982004 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2215/4419] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:30:02 2022] POST /compute/v2.1/os-server-external-events => generated 182 bytes in 48 msecs (HTTP/1.1 207) 9 headers in 367 bytes (2 switches on core 0) Jul 16 21:30:03.006982 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-4208613d-55b0-4187-b4da-04f0e0813483 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:30:03.020819 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-4208613d-55b0-4187-b4da-04f0e0813483 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:30:03.025493 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-4208613d-55b0-4187-b4da-04f0e0813483 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:30:03.028710 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-4208613d-55b0-4187-b4da-04f0e0813483 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] 10.222.0.16 "GET /compute/v2.1/servers/08a33911-6e47-45fd-b499-155f199aacf8" status: 200 len: 1724 microversion: 2.20 time: 1.297513 Jul 16 21:30:03.044811 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2205/4420] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:30:01 2022] GET /compute/v2.1/servers/08a33911-6e47-45fd-b499-155f199aacf8 => generated 1724 bytes in 1314 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:30:03.050412 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-edc2fa4c-559f-4c62-ba4d-fa2af62a5815 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "18e9c5fe-8f27-4cbf-bbe8-0e1f902e6813"}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:30:03.054343 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-edc2fa4c-559f-4c62-ba4d-fa2af62a5815 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] [instance: 08a33911-6e47-45fd-b499-155f199aacf8] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:30:03.067071 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-edc2fa4c-559f-4c62-ba4d-fa2af62a5815 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:30:03.070644 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-edc2fa4c-559f-4c62-ba4d-fa2af62a5815 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:30:03.073702 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-edc2fa4c-559f-4c62-ba4d-fa2af62a5815 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:30:03.075321 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-edc2fa4c-559f-4c62-ba4d-fa2af62a5815 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:30:03.141557 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=100 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:30:03.288352 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-54964e5a-b4de-4a11-ae81-11e9a32b946f tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:30:03.289577 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-54964e5a-b4de-4a11-ae81-11e9a32b946f tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] [instance: 515fd3da-3965-4882-b9e8-100a7eea64d9] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:30:03.300459 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-54964e5a-b4de-4a11-ae81-11e9a32b946f tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:30:03.301698 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-54964e5a-b4de-4a11-ae81-11e9a32b946f tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:30:03.309799 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-54964e5a-b4de-4a11-ae81-11e9a32b946f tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:30:03.311782 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-54964e5a-b4de-4a11-ae81-11e9a32b946f tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:30:04.196250 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-54964e5a-b4de-4a11-ae81-11e9a32b946f tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da77ec3d0f04496d8c8df4025bba76d1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '207ec2d1883c4a1ba89696e6e77cf3aa', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:30:04.211555 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-54964e5a-b4de-4a11-ae81-11e9a32b946f tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] 10.222.0.16 "GET /compute/v2.1/servers/515fd3da-3965-4882-b9e8-100a7eea64d9" status: 200 len: 1314 microversion: 2.1 time: 0.927112 Jul 16 21:30:04.212690 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2206/4421] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:30:03 2022] GET /compute/v2.1/servers/515fd3da-3965-4882-b9e8-100a7eea64d9 => generated 1314 bytes in 929 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:30:04.735673 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-edc2fa4c-559f-4c62-ba4d-fa2af62a5815 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] 10.222.0.16 "POST /compute/v2.1/servers/08a33911-6e47-45fd-b499-155f199aacf8/os-volume_attachments" status: 200 len: 188 microversion: 2.20 time: 1.689179 Jul 16 21:30:04.739924 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2216/4422] 10.222.0.16 () {62 vars in 1363 bytes} [Sat Jul 16 21:30:03 2022] POST /compute/v2.1/servers/08a33911-6e47-45fd-b499-155f199aacf8/os-volume_attachments => generated 188 bytes in 1691 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 16 21:30:05.029603 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-9815d312-f70d-4dd6-9156-faf56f2a9989 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Action: 'action', calling method: >, body: {"unshelve": null} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:30:05.030675 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-9815d312-f70d-4dd6-9156-faf56f2a9989 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] [instance: 08a33911-6e47-45fd-b499-155f199aacf8] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:30:05.064192 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9815d312-f70d-4dd6-9156-faf56f2a9989 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:30:05.067724 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9815d312-f70d-4dd6-9156-faf56f2a9989 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:30:05.077451 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9815d312-f70d-4dd6-9156-faf56f2a9989 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:30:05.078077 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9815d312-f70d-4dd6-9156-faf56f2a9989 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:30:05.240920 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-205573a2-98e6-4255-86f3-195a9f195f8f tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:30:05.241767 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-205573a2-98e6-4255-86f3-195a9f195f8f tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] [instance: 515fd3da-3965-4882-b9e8-100a7eea64d9] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:30:05.259213 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-205573a2-98e6-4255-86f3-195a9f195f8f tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:30:05.261500 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-205573a2-98e6-4255-86f3-195a9f195f8f tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:30:05.264091 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-205573a2-98e6-4255-86f3-195a9f195f8f tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:30:05.264600 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-205573a2-98e6-4255-86f3-195a9f195f8f tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:30:05.610893 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-9815d312-f70d-4dd6-9156-faf56f2a9989 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] 10.222.0.16 "POST /compute/v2.1/servers/08a33911-6e47-45fd-b499-155f199aacf8/action" status: 202 len: 0 microversion: 2.20 time: 0.586123 Jul 16 21:30:05.614649 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2207/4423] 10.222.0.16 () {62 vars in 1318 bytes} [Sat Jul 16 21:30:05 2022] POST /compute/v2.1/servers/08a33911-6e47-45fd-b499-155f199aacf8/action => generated 0 bytes in 590 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Jul 16 21:30:05.621445 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-485eab14-7e73-423c-a080-d37c77ec8209 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:30:05.622076 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-485eab14-7e73-423c-a080-d37c77ec8209 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] [instance: 08a33911-6e47-45fd-b499-155f199aacf8] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:30:05.641619 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-485eab14-7e73-423c-a080-d37c77ec8209 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:30:05.644311 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-485eab14-7e73-423c-a080-d37c77ec8209 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:30:05.645431 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-485eab14-7e73-423c-a080-d37c77ec8209 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:30:05.648336 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-485eab14-7e73-423c-a080-d37c77ec8209 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:30:06.190317 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-205573a2-98e6-4255-86f3-195a9f195f8f tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da77ec3d0f04496d8c8df4025bba76d1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '207ec2d1883c4a1ba89696e6e77cf3aa', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:30:06.200767 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-205573a2-98e6-4255-86f3-195a9f195f8f tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] 10.222.0.16 "GET /compute/v2.1/servers/515fd3da-3965-4882-b9e8-100a7eea64d9" status: 200 len: 1500 microversion: 2.1 time: 0.971866 Jul 16 21:30:06.219496 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2217/4424] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:30:05 2022] GET /compute/v2.1/servers/515fd3da-3965-4882-b9e8-100a7eea64d9 => generated 1500 bytes in 991 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:30:06.233372 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-0bb74946-1a7e-41f2-922b-ae01eeb11a83 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Action: 'action', calling method: >, body: {"rescue": {"adminPass": "***"}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:30:06.234481 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-0bb74946-1a7e-41f2-922b-ae01eeb11a83 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] [instance: 515fd3da-3965-4882-b9e8-100a7eea64d9] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:30:06.257644 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0bb74946-1a7e-41f2-922b-ae01eeb11a83 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:30:06.258102 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0bb74946-1a7e-41f2-922b-ae01eeb11a83 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:30:06.258901 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0bb74946-1a7e-41f2-922b-ae01eeb11a83 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:30:06.259458 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0bb74946-1a7e-41f2-922b-ae01eeb11a83 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:30:06.289737 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.objects.instance [None req-0bb74946-1a7e-41f2-922b-ae01eeb11a83 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lazy-loading 'flavor' on Instance uuid 515fd3da-3965-4882-b9e8-100a7eea64d9 {{(pid=138681) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:30:06.508288 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-485eab14-7e73-423c-a080-d37c77ec8209 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:30:06.524048 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-485eab14-7e73-423c-a080-d37c77ec8209 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:30:06.525633 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-485eab14-7e73-423c-a080-d37c77ec8209 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:30:06.526832 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-485eab14-7e73-423c-a080-d37c77ec8209 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] 10.222.0.16 "GET /compute/v2.1/servers/08a33911-6e47-45fd-b499-155f199aacf8" status: 200 len: 1750 microversion: 2.20 time: 0.911131 Jul 16 21:30:06.528773 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2208/4425] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:30:05 2022] GET /compute/v2.1/servers/08a33911-6e47-45fd-b499-155f199aacf8 => generated 1750 bytes in 914 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:30:06.715885 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-0bb74946-1a7e-41f2-922b-ae01eeb11a83 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] 10.222.0.16 "POST /compute/v2.1/servers/515fd3da-3965-4882-b9e8-100a7eea64d9/action" status: 200 len: 32 microversion: 2.1 time: 0.494106 Jul 16 21:30:06.715885 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2218/4426] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 21:30:06 2022] POST /compute/v2.1/servers/515fd3da-3965-4882-b9e8-100a7eea64d9/action => generated 32 bytes in 496 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Jul 16 21:30:06.725166 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-14e996c7-2bbf-49c9-bff9-b9b9252375b5 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:30:06.726442 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-14e996c7-2bbf-49c9-bff9-b9b9252375b5 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] [instance: 515fd3da-3965-4882-b9e8-100a7eea64d9] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:30:06.740343 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-14e996c7-2bbf-49c9-bff9-b9b9252375b5 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:30:06.740918 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-14e996c7-2bbf-49c9-bff9-b9b9252375b5 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:30:06.742400 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-14e996c7-2bbf-49c9-bff9-b9b9252375b5 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:30:06.745234 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-14e996c7-2bbf-49c9-bff9-b9b9252375b5 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:30:07.341749 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-14e996c7-2bbf-49c9-bff9-b9b9252375b5 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da77ec3d0f04496d8c8df4025bba76d1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '207ec2d1883c4a1ba89696e6e77cf3aa', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:30:07.349609 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-14e996c7-2bbf-49c9-bff9-b9b9252375b5 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] 10.222.0.16 "GET /compute/v2.1/servers/515fd3da-3965-4882-b9e8-100a7eea64d9" status: 200 len: 1506 microversion: 2.1 time: 0.628696 Jul 16 21:30:07.350946 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2209/4427] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:30:06 2022] GET /compute/v2.1/servers/515fd3da-3965-4882-b9e8-100a7eea64d9 => generated 1506 bytes in 631 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:30:07.547887 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-306101f9-f765-4cc5-b5aa-5677b902e956 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:30:07.548614 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-306101f9-f765-4cc5-b5aa-5677b902e956 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] [instance: 08a33911-6e47-45fd-b499-155f199aacf8] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:30:07.558047 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-306101f9-f765-4cc5-b5aa-5677b902e956 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:30:07.558718 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-306101f9-f765-4cc5-b5aa-5677b902e956 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:30:07.559539 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-306101f9-f765-4cc5-b5aa-5677b902e956 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:30:07.562609 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-306101f9-f765-4cc5-b5aa-5677b902e956 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:30:08.370669 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-70e60576-6aa3-4735-8ff8-670a158593a1 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:30:08.390324 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-70e60576-6aa3-4735-8ff8-670a158593a1 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] [instance: 515fd3da-3965-4882-b9e8-100a7eea64d9] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:30:08.390324 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-70e60576-6aa3-4735-8ff8-670a158593a1 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:30:08.390324 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-70e60576-6aa3-4735-8ff8-670a158593a1 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:30:08.390324 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-70e60576-6aa3-4735-8ff8-670a158593a1 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:30:08.390324 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-70e60576-6aa3-4735-8ff8-670a158593a1 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:30:08.422137 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-306101f9-f765-4cc5-b5aa-5677b902e956 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:30:08.436768 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-306101f9-f765-4cc5-b5aa-5677b902e956 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:30:08.441361 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-306101f9-f765-4cc5-b5aa-5677b902e956 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:30:08.442574 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-306101f9-f765-4cc5-b5aa-5677b902e956 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] 10.222.0.16 "GET /compute/v2.1/servers/08a33911-6e47-45fd-b499-155f199aacf8" status: 200 len: 1750 microversion: 2.20 time: 0.900015 Jul 16 21:30:08.447254 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2219/4428] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:30:07 2022] GET /compute/v2.1/servers/08a33911-6e47-45fd-b499-155f199aacf8 => generated 1750 bytes in 906 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:30:08.821648 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-306101f9-f765-4cc5-b5aa-5677b902e956 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:30:08.824164 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-306101f9-f765-4cc5-b5aa-5677b902e956 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] 10.222.0.16 "GET /compute//" status: 200 len: 377 microversion: - time: 0.003140 Jul 16 21:30:08.827466 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2220/4429] 10.222.0.16 () {56 vars in 885 bytes} [Sat Jul 16 21:30:08 2022] GET /compute/ => generated 377 bytes in 7 msecs (HTTP/1.1 200) 3 headers in 91 bytes (2 switches on core 0) Jul 16 21:30:09.070565 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-70e60576-6aa3-4735-8ff8-670a158593a1 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da77ec3d0f04496d8c8df4025bba76d1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '207ec2d1883c4a1ba89696e6e77cf3aa', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:30:09.083786 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-70e60576-6aa3-4735-8ff8-670a158593a1 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] 10.222.0.16 "GET /compute/v2.1/servers/515fd3da-3965-4882-b9e8-100a7eea64d9" status: 200 len: 1506 microversion: 2.1 time: 0.719088 Jul 16 21:30:09.084689 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2210/4430] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:30:08 2022] GET /compute/v2.1/servers/515fd3da-3965-4882-b9e8-100a7eea64d9 => generated 1506 bytes in 721 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:30:09.470332 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-9ccb8870-18fb-44e3-886c-4d868c09215d tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:30:09.474036 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-9ccb8870-18fb-44e3-886c-4d868c09215d tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] [instance: 08a33911-6e47-45fd-b499-155f199aacf8] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:30:09.505218 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9ccb8870-18fb-44e3-886c-4d868c09215d tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:30:09.507411 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9ccb8870-18fb-44e3-886c-4d868c09215d tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:30:09.508980 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9ccb8870-18fb-44e3-886c-4d868c09215d tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:30:09.511564 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9ccb8870-18fb-44e3-886c-4d868c09215d tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:30:09.567677 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-226ff859-8864-4380-9dd3-42db775be95e tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestNetworkBasicOps-server-1020540525", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452", "networks": [{"uuid": "76e68baa-9547-475e-b27b-ae0a556e4258"}], "key_name": "tempest-TestNetworkBasicOps-913257510", "security_groups": [{"name": "tempest-secgroup-smoke-2106229622"}]}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:30:09.961929 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.network.neutron [None req-226ff859-8864-4380-9dd3-42db775be95e tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] validate_networks() for [('76e68baa-9547-475e-b27b-ae0a556e4258', None, None, None, None, None)] {{(pid=138680) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 16 21:30:10.185433 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-9ccb8870-18fb-44e3-886c-4d868c09215d tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:30:10.200618 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-9ccb8870-18fb-44e3-886c-4d868c09215d tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:30:10.202472 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-9ccb8870-18fb-44e3-886c-4d868c09215d tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:30:10.204093 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-9ccb8870-18fb-44e3-886c-4d868c09215d tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] 10.222.0.16 "GET /compute/v2.1/servers/08a33911-6e47-45fd-b499-155f199aacf8" status: 200 len: 1752 microversion: 2.20 time: 0.743195 Jul 16 21:30:10.217619 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2221/4431] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:30:09 2022] GET /compute/v2.1/servers/08a33911-6e47-45fd-b499-155f199aacf8 => generated 1752 bytes in 757 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:30:10.224544 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-3cfc3111-47d3-4e47-bfaa-421207b6d91d tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:30:10.225958 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-3cfc3111-47d3-4e47-bfaa-421207b6d91d tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] [instance: 515fd3da-3965-4882-b9e8-100a7eea64d9] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:30:10.234496 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3cfc3111-47d3-4e47-bfaa-421207b6d91d tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:30:10.235420 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3cfc3111-47d3-4e47-bfaa-421207b6d91d tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:30:10.236429 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3cfc3111-47d3-4e47-bfaa-421207b6d91d tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:30:10.237181 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3cfc3111-47d3-4e47-bfaa-421207b6d91d tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:30:10.586575 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-3cfc3111-47d3-4e47-bfaa-421207b6d91d tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da77ec3d0f04496d8c8df4025bba76d1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '207ec2d1883c4a1ba89696e6e77cf3aa', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:30:10.593829 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-3cfc3111-47d3-4e47-bfaa-421207b6d91d tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] 10.222.0.16 "GET /compute/v2.1/servers/515fd3da-3965-4882-b9e8-100a7eea64d9" status: 200 len: 1506 microversion: 2.1 time: 0.374558 Jul 16 21:30:10.594699 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2222/4432] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:30:10 2022] GET /compute/v2.1/servers/515fd3da-3965-4882-b9e8-100a7eea64d9 => generated 1506 bytes in 376 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:30:11.025800 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-226ff859-8864-4380-9dd3-42db775be95e tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Flavor limits 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 16 21:30:11.026801 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-226ff859-8864-4380-9dd3-42db775be95e tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Image limits 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 16 21:30:11.027604 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-226ff859-8864-4380-9dd3-42db775be95e tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Flavor pref 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 16 21:30:11.028642 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-226ff859-8864-4380-9dd3-42db775be95e tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Image pref 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 16 21:30:11.029272 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-226ff859-8864-4380-9dd3-42db775be95e tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 16 21:30:11.042855 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-226ff859-8864-4380-9dd3-42db775be95e tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:30:11.044031 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-226ff859-8864-4380-9dd3-42db775be95e tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:30:11.046731 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-226ff859-8864-4380-9dd3-42db775be95e tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:30:11.046731 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-226ff859-8864-4380-9dd3-42db775be95e tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:30:11.079546 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-226ff859-8864-4380-9dd3-42db775be95e tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Getting quotas for project bed7c94624484c7cb8abeb3bfc8757b9. Resources: {'ram', 'instances', 'cores'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:30:11.089657 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-226ff859-8864-4380-9dd3-42db775be95e tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Getting quotas for user 4ddfae3448704591b662de83b97a1876 and project bed7c94624484c7cb8abeb3bfc8757b9. Resources: {'ram', 'instances', 'cores'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:30:11.112765 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-226ff859-8864-4380-9dd3-42db775be95e tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Going to run 1 instances... {{(pid=138680) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 16 21:30:11.117374 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-226ff859-8864-4380-9dd3-42db775be95e tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] [instance: d17892b8-b5bd-469e-9284-0abb3115daaa] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138680) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 16 21:30:11.231569 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-3406d820-12f2-4541-966a-b655e71e1346 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:30:11.233012 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-3406d820-12f2-4541-966a-b655e71e1346 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] [instance: 08a33911-6e47-45fd-b499-155f199aacf8] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:30:11.250355 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-226ff859-8864-4380-9dd3-42db775be95e tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 406 microversion: 2.1 time: 1.686402 Jul 16 21:30:11.253044 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2211/4433] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 21:30:09 2022] POST /compute/v2.1/servers => generated 406 bytes in 1690 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 16 21:30:11.253241 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3406d820-12f2-4541-966a-b655e71e1346 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:30:11.253694 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3406d820-12f2-4541-966a-b655e71e1346 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:30:11.254475 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3406d820-12f2-4541-966a-b655e71e1346 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:30:11.254949 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3406d820-12f2-4541-966a-b655e71e1346 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:30:11.274783 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-8fc22d26-b001-4a09-af55-d230270a5ff0 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:30:11.274783 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-8fc22d26-b001-4a09-af55-d230270a5ff0 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] [instance: d17892b8-b5bd-469e-9284-0abb3115daaa] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:30:11.424068 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-8fc22d26-b001-4a09-af55-d230270a5ff0 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ddfae3448704591b662de83b97a1876', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bed7c94624484c7cb8abeb3bfc8757b9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:30:11.433057 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-8fc22d26-b001-4a09-af55-d230270a5ff0 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] 10.222.0.16 "GET /compute/v2.1/servers/d17892b8-b5bd-469e-9284-0abb3115daaa" status: 200 len: 1248 microversion: 2.1 time: 0.174144 Jul 16 21:30:11.445475 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2212/4434] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:30:11 2022] GET /compute/v2.1/servers/d17892b8-b5bd-469e-9284-0abb3115daaa => generated 1248 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:30:11.616667 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-736e623b-0ee5-4ee9-afb1-94304bcc04ed tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:30:11.617520 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-736e623b-0ee5-4ee9-afb1-94304bcc04ed tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] [instance: 515fd3da-3965-4882-b9e8-100a7eea64d9] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:30:11.656511 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-736e623b-0ee5-4ee9-afb1-94304bcc04ed tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:30:11.668155 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-736e623b-0ee5-4ee9-afb1-94304bcc04ed tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.011s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:30:11.669849 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-736e623b-0ee5-4ee9-afb1-94304bcc04ed tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:30:11.675127 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-736e623b-0ee5-4ee9-afb1-94304bcc04ed tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:30:12.161864 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-3406d820-12f2-4541-966a-b655e71e1346 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:30:12.185155 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-3406d820-12f2-4541-966a-b655e71e1346 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:30:12.194294 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-3406d820-12f2-4541-966a-b655e71e1346 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:30:12.202186 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-3406d820-12f2-4541-966a-b655e71e1346 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] 10.222.0.16 "GET /compute/v2.1/servers/08a33911-6e47-45fd-b499-155f199aacf8" status: 200 len: 1752 microversion: 2.20 time: 0.980913 Jul 16 21:30:12.203513 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2223/4435] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:30:11 2022] GET /compute/v2.1/servers/08a33911-6e47-45fd-b499-155f199aacf8 => generated 1752 bytes in 984 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:30:12.452318 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-ba82c105-cbf9-4178-8d0b-1da3c164af8f tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:30:12.453664 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-ba82c105-cbf9-4178-8d0b-1da3c164af8f tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] [instance: d17892b8-b5bd-469e-9284-0abb3115daaa] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:30:12.850332 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-ba82c105-cbf9-4178-8d0b-1da3c164af8f tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ddfae3448704591b662de83b97a1876', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bed7c94624484c7cb8abeb3bfc8757b9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:30:12.874796 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-ba82c105-cbf9-4178-8d0b-1da3c164af8f tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] 10.222.0.16 "GET /compute/v2.1/servers/d17892b8-b5bd-469e-9284-0abb3115daaa" status: 200 len: 1248 microversion: 2.1 time: 0.426905 Jul 16 21:30:12.875414 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2224/4436] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:30:12 2022] GET /compute/v2.1/servers/d17892b8-b5bd-469e-9284-0abb3115daaa => generated 1248 bytes in 429 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:30:13.020630 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-736e623b-0ee5-4ee9-afb1-94304bcc04ed tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da77ec3d0f04496d8c8df4025bba76d1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '207ec2d1883c4a1ba89696e6e77cf3aa', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:30:13.033044 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-736e623b-0ee5-4ee9-afb1-94304bcc04ed tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] 10.222.0.16 "GET /compute/v2.1/servers/515fd3da-3965-4882-b9e8-100a7eea64d9" status: 200 len: 1506 microversion: 2.1 time: 1.425508 Jul 16 21:30:13.045219 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2213/4437] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:30:11 2022] GET /compute/v2.1/servers/515fd3da-3965-4882-b9e8-100a7eea64d9 => generated 1506 bytes in 1439 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:30:13.222078 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-71c40b5a-3dbf-48bd-becf-6ff0784ce1ae tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:30:13.223381 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-71c40b5a-3dbf-48bd-becf-6ff0784ce1ae tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] [instance: 08a33911-6e47-45fd-b499-155f199aacf8] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:30:13.231098 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-71c40b5a-3dbf-48bd-becf-6ff0784ce1ae tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:30:13.232696 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-71c40b5a-3dbf-48bd-becf-6ff0784ce1ae tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:30:13.233698 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-71c40b5a-3dbf-48bd-becf-6ff0784ce1ae tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:30:13.234473 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-71c40b5a-3dbf-48bd-becf-6ff0784ce1ae tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:30:13.582003 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-71c40b5a-3dbf-48bd-becf-6ff0784ce1ae tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:30:13.601350 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-71c40b5a-3dbf-48bd-becf-6ff0784ce1ae tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:30:13.604219 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-71c40b5a-3dbf-48bd-becf-6ff0784ce1ae tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:30:13.605915 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-71c40b5a-3dbf-48bd-becf-6ff0784ce1ae tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] 10.222.0.16 "GET /compute/v2.1/servers/08a33911-6e47-45fd-b499-155f199aacf8" status: 200 len: 1808 microversion: 2.20 time: 0.388439 Jul 16 21:30:13.607529 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2225/4438] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:30:13 2022] GET /compute/v2.1/servers/08a33911-6e47-45fd-b499-155f199aacf8 => generated 1808 bytes in 390 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:30:13.895178 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-6891ed7e-08c8-43dc-b862-174a25507b0b tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:30:13.901107 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-6891ed7e-08c8-43dc-b862-174a25507b0b tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] [instance: d17892b8-b5bd-469e-9284-0abb3115daaa] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:30:13.947918 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6891ed7e-08c8-43dc-b862-174a25507b0b tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:30:13.953379 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6891ed7e-08c8-43dc-b862-174a25507b0b tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:30:13.956726 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6891ed7e-08c8-43dc-b862-174a25507b0b tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:30:13.957562 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6891ed7e-08c8-43dc-b862-174a25507b0b tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:30:14.052538 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-76388a2a-f65d-4905-a6e2-a3a589e74fc3 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:30:14.053651 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-76388a2a-f65d-4905-a6e2-a3a589e74fc3 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] [instance: 515fd3da-3965-4882-b9e8-100a7eea64d9] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:30:14.065120 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-76388a2a-f65d-4905-a6e2-a3a589e74fc3 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:30:14.066193 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-76388a2a-f65d-4905-a6e2-a3a589e74fc3 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:30:14.067693 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-76388a2a-f65d-4905-a6e2-a3a589e74fc3 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:30:14.068361 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-76388a2a-f65d-4905-a6e2-a3a589e74fc3 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:30:14.203245 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-6891ed7e-08c8-43dc-b862-174a25507b0b tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ddfae3448704591b662de83b97a1876', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bed7c94624484c7cb8abeb3bfc8757b9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:30:14.214924 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-6891ed7e-08c8-43dc-b862-174a25507b0b tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] 10.222.0.16 "GET /compute/v2.1/servers/d17892b8-b5bd-469e-9284-0abb3115daaa" status: 200 len: 1300 microversion: 2.1 time: 0.323496 Jul 16 21:30:14.216578 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2214/4439] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:30:13 2022] GET /compute/v2.1/servers/d17892b8-b5bd-469e-9284-0abb3115daaa => generated 1300 bytes in 328 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:30:14.528194 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-76388a2a-f65d-4905-a6e2-a3a589e74fc3 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da77ec3d0f04496d8c8df4025bba76d1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '207ec2d1883c4a1ba89696e6e77cf3aa', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:30:14.554367 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-76388a2a-f65d-4905-a6e2-a3a589e74fc3 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] 10.222.0.16 "GET /compute/v2.1/servers/515fd3da-3965-4882-b9e8-100a7eea64d9" status: 200 len: 1506 microversion: 2.1 time: 0.500172 Jul 16 21:30:14.566618 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2226/4440] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:30:14 2022] GET /compute/v2.1/servers/515fd3da-3965-4882-b9e8-100a7eea64d9 => generated 1506 bytes in 509 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:30:14.626366 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-90a088fb-60ab-416d-8ba1-ce2efa02f4b2 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:30:14.627045 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-90a088fb-60ab-416d-8ba1-ce2efa02f4b2 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] [instance: 08a33911-6e47-45fd-b499-155f199aacf8] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:30:14.647040 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-90a088fb-60ab-416d-8ba1-ce2efa02f4b2 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:30:14.649343 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-90a088fb-60ab-416d-8ba1-ce2efa02f4b2 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:30:14.662169 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-90a088fb-60ab-416d-8ba1-ce2efa02f4b2 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:30:14.662169 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-90a088fb-60ab-416d-8ba1-ce2efa02f4b2 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:30:14.697692 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:30:14.957095 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-836d758a-c08b-46ad-b65e-b20d8b5f5b62 req-0cca4458-6e7f-4c61-9a35-267fca991de8 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "08a33911-6e47-45fd-b499-155f199aacf8", "tag": "81e317e7-e9db-4ba9-b87d-9395b084ead7"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:30:14.974619 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-836d758a-c08b-46ad-b65e-b20d8b5f5b62 req-0cca4458-6e7f-4c61-9a35-267fca991de8 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:30:14.975130 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-836d758a-c08b-46ad-b65e-b20d8b5f5b62 req-0cca4458-6e7f-4c61-9a35-267fca991de8 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:30:14.995598 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-836d758a-c08b-46ad-b65e-b20d8b5f5b62 req-0cca4458-6e7f-4c61-9a35-267fca991de8 service nova] Creating event network-changed:81e317e7-e9db-4ba9-b87d-9395b084ead7 for instance 08a33911-6e47-45fd-b499-155f199aacf8 on nch2-master-1 Jul 16 21:30:15.008785 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-836d758a-c08b-46ad-b65e-b20d8b5f5b62 req-0cca4458-6e7f-4c61-9a35-267fca991de8 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.055240 Jul 16 21:30:15.009919 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2227/4441] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:30:14 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 16 21:30:15.172640 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-90a088fb-60ab-416d-8ba1-ce2efa02f4b2 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:30:15.202527 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-90a088fb-60ab-416d-8ba1-ce2efa02f4b2 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:30:15.202527 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-90a088fb-60ab-416d-8ba1-ce2efa02f4b2 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:30:15.202527 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-90a088fb-60ab-416d-8ba1-ce2efa02f4b2 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] 10.222.0.16 "GET /compute/v2.1/servers/08a33911-6e47-45fd-b499-155f199aacf8" status: 200 len: 1808 microversion: 2.20 time: 0.568893 Jul 16 21:30:15.202527 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2215/4442] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:30:14 2022] GET /compute/v2.1/servers/08a33911-6e47-45fd-b499-155f199aacf8 => generated 1808 bytes in 572 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:30:15.234459 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-0ff2fd9b-c325-4096-9c20-4a090b452005 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:30:15.235748 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-0ff2fd9b-c325-4096-9c20-4a090b452005 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] [instance: d17892b8-b5bd-469e-9284-0abb3115daaa] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:30:15.255177 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0ff2fd9b-c325-4096-9c20-4a090b452005 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:30:15.258508 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0ff2fd9b-c325-4096-9c20-4a090b452005 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:30:15.260286 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0ff2fd9b-c325-4096-9c20-4a090b452005 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:30:15.261223 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0ff2fd9b-c325-4096-9c20-4a090b452005 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:30:15.385843 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-0ff2fd9b-c325-4096-9c20-4a090b452005 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ddfae3448704591b662de83b97a1876', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bed7c94624484c7cb8abeb3bfc8757b9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:30:15.400523 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-0ff2fd9b-c325-4096-9c20-4a090b452005 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] 10.222.0.16 "GET /compute/v2.1/servers/d17892b8-b5bd-469e-9284-0abb3115daaa" status: 200 len: 1306 microversion: 2.1 time: 0.169072 Jul 16 21:30:15.402269 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2228/4443] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:30:15 2022] GET /compute/v2.1/servers/d17892b8-b5bd-469e-9284-0abb3115daaa => generated 1306 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:30:15.575398 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-efe281fe-35ea-4d1b-ab93-7789331910cc tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:30:15.576072 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-efe281fe-35ea-4d1b-ab93-7789331910cc tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] [instance: 515fd3da-3965-4882-b9e8-100a7eea64d9] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:30:15.582835 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-efe281fe-35ea-4d1b-ab93-7789331910cc tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:30:15.583270 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-efe281fe-35ea-4d1b-ab93-7789331910cc tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:30:15.584106 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-efe281fe-35ea-4d1b-ab93-7789331910cc tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:30:15.584767 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-efe281fe-35ea-4d1b-ab93-7789331910cc tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:30:15.961589 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-efe281fe-35ea-4d1b-ab93-7789331910cc tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da77ec3d0f04496d8c8df4025bba76d1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '207ec2d1883c4a1ba89696e6e77cf3aa', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:30:15.970433 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-efe281fe-35ea-4d1b-ab93-7789331910cc tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] 10.222.0.16 "GET /compute/v2.1/servers/515fd3da-3965-4882-b9e8-100a7eea64d9" status: 200 len: 1506 microversion: 2.1 time: 0.400692 Jul 16 21:30:15.970889 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2216/4444] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:30:15 2022] GET /compute/v2.1/servers/515fd3da-3965-4882-b9e8-100a7eea64d9 => generated 1506 bytes in 402 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:30:16.244034 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-7737478f-02bf-4f47-b69c-d5cc68e6626e tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:30:16.252145 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-7737478f-02bf-4f47-b69c-d5cc68e6626e tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] [instance: 08a33911-6e47-45fd-b499-155f199aacf8] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:30:16.281024 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7737478f-02bf-4f47-b69c-d5cc68e6626e tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:30:16.282176 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7737478f-02bf-4f47-b69c-d5cc68e6626e tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:30:16.284168 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7737478f-02bf-4f47-b69c-d5cc68e6626e tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:30:16.285224 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7737478f-02bf-4f47-b69c-d5cc68e6626e tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:30:16.417917 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-2529528c-dc2a-4057-a02f-a084a6dda4de tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:30:16.418790 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-2529528c-dc2a-4057-a02f-a084a6dda4de tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] [instance: d17892b8-b5bd-469e-9284-0abb3115daaa] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:30:16.426252 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2529528c-dc2a-4057-a02f-a084a6dda4de tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:30:16.426888 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2529528c-dc2a-4057-a02f-a084a6dda4de tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:30:16.427913 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2529528c-dc2a-4057-a02f-a084a6dda4de tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:30:16.428519 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2529528c-dc2a-4057-a02f-a084a6dda4de tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:30:16.698317 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-7737478f-02bf-4f47-b69c-d5cc68e6626e tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:30:16.707059 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-7737478f-02bf-4f47-b69c-d5cc68e6626e tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:30:16.708823 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-7737478f-02bf-4f47-b69c-d5cc68e6626e tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:30:16.709997 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-7737478f-02bf-4f47-b69c-d5cc68e6626e tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] 10.222.0.16 "GET /compute/v2.1/servers/08a33911-6e47-45fd-b499-155f199aacf8" status: 200 len: 1808 microversion: 2.20 time: 0.492309 Jul 16 21:30:16.710569 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2229/4445] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:30:16 2022] GET /compute/v2.1/servers/08a33911-6e47-45fd-b499-155f199aacf8 => generated 1808 bytes in 494 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:30:16.798064 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-2529528c-dc2a-4057-a02f-a084a6dda4de tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ddfae3448704591b662de83b97a1876', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bed7c94624484c7cb8abeb3bfc8757b9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:30:16.812884 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-2529528c-dc2a-4057-a02f-a084a6dda4de tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] 10.222.0.16 "GET /compute/v2.1/servers/d17892b8-b5bd-469e-9284-0abb3115daaa" status: 200 len: 1374 microversion: 2.1 time: 0.398988 Jul 16 21:30:16.813695 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2217/4446] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:30:16 2022] GET /compute/v2.1/servers/d17892b8-b5bd-469e-9284-0abb3115daaa => generated 1374 bytes in 401 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:30:16.993016 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-0784d856-7be0-4ae6-95d2-74807af84d1f tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:30:16.993981 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-0784d856-7be0-4ae6-95d2-74807af84d1f tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] [instance: 515fd3da-3965-4882-b9e8-100a7eea64d9] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:30:17.001240 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0784d856-7be0-4ae6-95d2-74807af84d1f tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:30:17.004193 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0784d856-7be0-4ae6-95d2-74807af84d1f tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:30:17.005377 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0784d856-7be0-4ae6-95d2-74807af84d1f tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:30:17.007803 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0784d856-7be0-4ae6-95d2-74807af84d1f tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:30:17.733755 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-29e259f9-91b9-4e7c-a91b-87c10126f418 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:30:17.734211 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-29e259f9-91b9-4e7c-a91b-87c10126f418 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] [instance: 08a33911-6e47-45fd-b499-155f199aacf8] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:30:17.754584 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-0784d856-7be0-4ae6-95d2-74807af84d1f tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da77ec3d0f04496d8c8df4025bba76d1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '207ec2d1883c4a1ba89696e6e77cf3aa', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:30:17.757529 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=90,nova_cell1:INSERT=4,nova_cell1:UPDATE=6 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:30:17.764034 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-29e259f9-91b9-4e7c-a91b-87c10126f418 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:30:17.764296 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-29e259f9-91b9-4e7c-a91b-87c10126f418 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.013s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:30:17.765860 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-0784d856-7be0-4ae6-95d2-74807af84d1f tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] 10.222.0.16 "GET /compute/v2.1/servers/515fd3da-3965-4882-b9e8-100a7eea64d9" status: 200 len: 1506 microversion: 2.1 time: 0.781702 Jul 16 21:30:17.771256 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-29e259f9-91b9-4e7c-a91b-87c10126f418 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:30:17.773411 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-29e259f9-91b9-4e7c-a91b-87c10126f418 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:30:17.778794 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2230/4447] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:30:16 2022] GET /compute/v2.1/servers/515fd3da-3965-4882-b9e8-100a7eea64d9 => generated 1506 bytes in 795 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:30:17.831832 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-2098ed3a-a7da-494b-98ef-c9ef4848e623 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:30:17.833231 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-2098ed3a-a7da-494b-98ef-c9ef4848e623 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] [instance: d17892b8-b5bd-469e-9284-0abb3115daaa] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:30:17.849066 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2098ed3a-a7da-494b-98ef-c9ef4848e623 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:30:17.849066 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2098ed3a-a7da-494b-98ef-c9ef4848e623 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:30:17.856309 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2098ed3a-a7da-494b-98ef-c9ef4848e623 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:30:17.856309 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2098ed3a-a7da-494b-98ef-c9ef4848e623 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:30:18.044630 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-29e259f9-91b9-4e7c-a91b-87c10126f418 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:30:18.065450 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-29e259f9-91b9-4e7c-a91b-87c10126f418 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:30:18.067298 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-29e259f9-91b9-4e7c-a91b-87c10126f418 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:30:18.068658 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-29e259f9-91b9-4e7c-a91b-87c10126f418 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] 10.222.0.16 "GET /compute/v2.1/servers/08a33911-6e47-45fd-b499-155f199aacf8" status: 200 len: 1808 microversion: 2.20 time: 0.344752 Jul 16 21:30:18.069588 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2218/4448] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:30:17 2022] GET /compute/v2.1/servers/08a33911-6e47-45fd-b499-155f199aacf8 => generated 1808 bytes in 346 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:30:18.255398 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-2098ed3a-a7da-494b-98ef-c9ef4848e623 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ddfae3448704591b662de83b97a1876', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bed7c94624484c7cb8abeb3bfc8757b9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:30:18.263558 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-2098ed3a-a7da-494b-98ef-c9ef4848e623 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] 10.222.0.16 "GET /compute/v2.1/servers/d17892b8-b5bd-469e-9284-0abb3115daaa" status: 200 len: 1374 microversion: 2.1 time: 0.437508 Jul 16 21:30:18.273398 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2231/4449] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:30:17 2022] GET /compute/v2.1/servers/d17892b8-b5bd-469e-9284-0abb3115daaa => generated 1374 bytes in 448 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:30:18.795112 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-706c86bf-669e-4290-97d0-f0f64d5d598a tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:30:18.795744 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-706c86bf-669e-4290-97d0-f0f64d5d598a tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] [instance: 515fd3da-3965-4882-b9e8-100a7eea64d9] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:30:18.828670 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-706c86bf-669e-4290-97d0-f0f64d5d598a tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:30:18.829179 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-706c86bf-669e-4290-97d0-f0f64d5d598a tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:30:18.832904 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-706c86bf-669e-4290-97d0-f0f64d5d598a tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:30:18.833365 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-706c86bf-669e-4290-97d0-f0f64d5d598a tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:30:18.835985 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-94aea4b9-034f-4056-b684-deb0b98ffdaa req-221ea63d-7f94-4055-9a01-525df75a493c service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "d17892b8-b5bd-469e-9284-0abb3115daaa", "tag": "88f48838-7774-4261-8f33-ad28281a1d60"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:30:18.851284 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-94aea4b9-034f-4056-b684-deb0b98ffdaa req-221ea63d-7f94-4055-9a01-525df75a493c service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:30:18.851750 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-94aea4b9-034f-4056-b684-deb0b98ffdaa req-221ea63d-7f94-4055-9a01-525df75a493c service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:30:18.909236 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-94aea4b9-034f-4056-b684-deb0b98ffdaa req-221ea63d-7f94-4055-9a01-525df75a493c service nova] Creating event network-changed:88f48838-7774-4261-8f33-ad28281a1d60 for instance d17892b8-b5bd-469e-9284-0abb3115daaa on nch2-master-1 Jul 16 21:30:18.922947 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-94aea4b9-034f-4056-b684-deb0b98ffdaa req-221ea63d-7f94-4055-9a01-525df75a493c service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.091018 Jul 16 21:30:18.924252 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2232/4450] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:30:18 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:30:19.123925 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-beb6e9bd-82db-4ec5-8275-07f12b6b4f31 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:30:19.124968 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-beb6e9bd-82db-4ec5-8275-07f12b6b4f31 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] [instance: 08a33911-6e47-45fd-b499-155f199aacf8] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:30:19.154593 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-beb6e9bd-82db-4ec5-8275-07f12b6b4f31 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:30:19.155605 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-beb6e9bd-82db-4ec5-8275-07f12b6b4f31 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:30:19.159712 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-beb6e9bd-82db-4ec5-8275-07f12b6b4f31 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:30:19.163517 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-beb6e9bd-82db-4ec5-8275-07f12b6b4f31 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:30:19.375537 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-706c86bf-669e-4290-97d0-f0f64d5d598a tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da77ec3d0f04496d8c8df4025bba76d1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '207ec2d1883c4a1ba89696e6e77cf3aa', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:30:19.384357 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-706c86bf-669e-4290-97d0-f0f64d5d598a tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] 10.222.0.16 "GET /compute/v2.1/servers/515fd3da-3965-4882-b9e8-100a7eea64d9" status: 200 len: 1506 microversion: 2.1 time: 0.593735 Jul 16 21:30:19.384930 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2219/4451] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:30:18 2022] GET /compute/v2.1/servers/515fd3da-3965-4882-b9e8-100a7eea64d9 => generated 1506 bytes in 595 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:30:19.399398 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-a0169484-7765-4be0-ac23-39a377e5c66f tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:30:19.400740 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-a0169484-7765-4be0-ac23-39a377e5c66f tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] [instance: d17892b8-b5bd-469e-9284-0abb3115daaa] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:30:19.408190 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a0169484-7765-4be0-ac23-39a377e5c66f tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:30:19.408660 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a0169484-7765-4be0-ac23-39a377e5c66f tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:30:19.409849 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a0169484-7765-4be0-ac23-39a377e5c66f tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:30:19.410496 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a0169484-7765-4be0-ac23-39a377e5c66f tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:30:19.604254 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-beb6e9bd-82db-4ec5-8275-07f12b6b4f31 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:30:19.623684 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-beb6e9bd-82db-4ec5-8275-07f12b6b4f31 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:30:19.627033 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-beb6e9bd-82db-4ec5-8275-07f12b6b4f31 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:30:19.629414 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-beb6e9bd-82db-4ec5-8275-07f12b6b4f31 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] 10.222.0.16 "GET /compute/v2.1/servers/08a33911-6e47-45fd-b499-155f199aacf8" status: 200 len: 1808 microversion: 2.20 time: 0.542940 Jul 16 21:30:19.636503 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2233/4452] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:30:19 2022] GET /compute/v2.1/servers/08a33911-6e47-45fd-b499-155f199aacf8 => generated 1808 bytes in 553 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:30:19.713333 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-a0169484-7765-4be0-ac23-39a377e5c66f tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ddfae3448704591b662de83b97a1876', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bed7c94624484c7cb8abeb3bfc8757b9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:30:19.722032 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-a0169484-7765-4be0-ac23-39a377e5c66f tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] 10.222.0.16 "GET /compute/v2.1/servers/d17892b8-b5bd-469e-9284-0abb3115daaa" status: 200 len: 1374 microversion: 2.1 time: 0.326587 Jul 16 21:30:19.742876 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2220/4453] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:30:19 2022] GET /compute/v2.1/servers/d17892b8-b5bd-469e-9284-0abb3115daaa => generated 1374 bytes in 357 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:30:20.403478 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-421562a1-6909-4cfe-9688-dd47e2161957 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:30:20.404822 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-421562a1-6909-4cfe-9688-dd47e2161957 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] [instance: 515fd3da-3965-4882-b9e8-100a7eea64d9] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:30:20.417462 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-421562a1-6909-4cfe-9688-dd47e2161957 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:30:20.418420 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-421562a1-6909-4cfe-9688-dd47e2161957 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:30:20.419840 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-421562a1-6909-4cfe-9688-dd47e2161957 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:30:20.420537 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-421562a1-6909-4cfe-9688-dd47e2161957 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:30:20.657424 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-0028df79-e077-4591-80f8-70dae2be0b84 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:30:20.658646 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-0028df79-e077-4591-80f8-70dae2be0b84 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] [instance: 08a33911-6e47-45fd-b499-155f199aacf8] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:30:20.674386 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0028df79-e077-4591-80f8-70dae2be0b84 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:30:20.675704 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0028df79-e077-4591-80f8-70dae2be0b84 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:30:20.677832 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0028df79-e077-4591-80f8-70dae2be0b84 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:30:20.680644 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0028df79-e077-4591-80f8-70dae2be0b84 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:30:20.697631 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-421562a1-6909-4cfe-9688-dd47e2161957 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da77ec3d0f04496d8c8df4025bba76d1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '207ec2d1883c4a1ba89696e6e77cf3aa', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:30:20.710015 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-421562a1-6909-4cfe-9688-dd47e2161957 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] 10.222.0.16 "GET /compute/v2.1/servers/515fd3da-3965-4882-b9e8-100a7eea64d9" status: 200 len: 1506 microversion: 2.1 time: 0.308551 Jul 16 21:30:20.717240 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2234/4454] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:30:20 2022] GET /compute/v2.1/servers/515fd3da-3965-4882-b9e8-100a7eea64d9 => generated 1506 bytes in 318 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:30:20.752817 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-08055abf-aca3-4849-b006-e246cd3f8153 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:30:20.754507 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-08055abf-aca3-4849-b006-e246cd3f8153 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] [instance: d17892b8-b5bd-469e-9284-0abb3115daaa] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:30:20.779525 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-08055abf-aca3-4849-b006-e246cd3f8153 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:30:20.780540 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-08055abf-aca3-4849-b006-e246cd3f8153 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:30:20.781334 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-08055abf-aca3-4849-b006-e246cd3f8153 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:30:20.797823 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-08055abf-aca3-4849-b006-e246cd3f8153 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.012s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:30:21.064747 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell0:SELECT=2 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:30:21.074980 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-0028df79-e077-4591-80f8-70dae2be0b84 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:30:21.090289 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-08055abf-aca3-4849-b006-e246cd3f8153 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ddfae3448704591b662de83b97a1876', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bed7c94624484c7cb8abeb3bfc8757b9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:30:21.092466 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-0028df79-e077-4591-80f8-70dae2be0b84 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:30:21.094310 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-0028df79-e077-4591-80f8-70dae2be0b84 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:30:21.096482 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-0028df79-e077-4591-80f8-70dae2be0b84 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] 10.222.0.16 "GET /compute/v2.1/servers/08a33911-6e47-45fd-b499-155f199aacf8" status: 200 len: 1808 microversion: 2.20 time: 0.444008 Jul 16 21:30:21.110767 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2221/4455] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:30:20 2022] GET /compute/v2.1/servers/08a33911-6e47-45fd-b499-155f199aacf8 => generated 1808 bytes in 447 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 16 21:30:21.118966 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-08055abf-aca3-4849-b006-e246cd3f8153 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] 10.222.0.16 "GET /compute/v2.1/servers/d17892b8-b5bd-469e-9284-0abb3115daaa" status: 200 len: 1374 microversion: 2.1 time: 0.381168 Jul 16 21:30:21.125153 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2235/4456] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:30:20 2022] GET /compute/v2.1/servers/d17892b8-b5bd-469e-9284-0abb3115daaa => generated 1374 bytes in 384 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:30:21.729066 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-a842ef18-b308-43fe-995e-87dbbaf2f47d tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:30:21.729689 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-a842ef18-b308-43fe-995e-87dbbaf2f47d tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] [instance: 515fd3da-3965-4882-b9e8-100a7eea64d9] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:30:21.732131 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell0:SELECT=12 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:30:21.738341 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a842ef18-b308-43fe-995e-87dbbaf2f47d tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:30:21.738792 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a842ef18-b308-43fe-995e-87dbbaf2f47d tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:30:21.739632 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a842ef18-b308-43fe-995e-87dbbaf2f47d tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:30:21.740079 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a842ef18-b308-43fe-995e-87dbbaf2f47d tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:30:21.913609 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-a842ef18-b308-43fe-995e-87dbbaf2f47d tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da77ec3d0f04496d8c8df4025bba76d1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '207ec2d1883c4a1ba89696e6e77cf3aa', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:30:21.922583 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-a842ef18-b308-43fe-995e-87dbbaf2f47d tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] 10.222.0.16 "GET /compute/v2.1/servers/515fd3da-3965-4882-b9e8-100a7eea64d9" status: 200 len: 1506 microversion: 2.1 time: 0.197454 Jul 16 21:30:21.923284 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2222/4457] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:30:21 2022] GET /compute/v2.1/servers/515fd3da-3965-4882-b9e8-100a7eea64d9 => generated 1506 bytes in 199 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:30:22.119994 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-639a3c07-ede0-4683-9b30-3aac12c67d56 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:30:22.121976 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-639a3c07-ede0-4683-9b30-3aac12c67d56 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] [instance: 08a33911-6e47-45fd-b499-155f199aacf8] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:30:22.137154 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-639a3c07-ede0-4683-9b30-3aac12c67d56 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:30:22.140448 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-639a3c07-ede0-4683-9b30-3aac12c67d56 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:30:22.140448 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-639a3c07-ede0-4683-9b30-3aac12c67d56 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:30:22.140448 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-639a3c07-ede0-4683-9b30-3aac12c67d56 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:30:22.141685 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-c5376a0b-1cb3-4429-872c-9ad6f55dbb21 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:30:22.142440 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-c5376a0b-1cb3-4429-872c-9ad6f55dbb21 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] [instance: d17892b8-b5bd-469e-9284-0abb3115daaa] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:30:22.152385 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c5376a0b-1cb3-4429-872c-9ad6f55dbb21 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:30:22.154882 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c5376a0b-1cb3-4429-872c-9ad6f55dbb21 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:30:22.157166 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c5376a0b-1cb3-4429-872c-9ad6f55dbb21 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:30:22.159805 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c5376a0b-1cb3-4429-872c-9ad6f55dbb21 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:30:22.439627 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-639a3c07-ede0-4683-9b30-3aac12c67d56 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:30:22.452151 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-639a3c07-ede0-4683-9b30-3aac12c67d56 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:30:22.453685 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-639a3c07-ede0-4683-9b30-3aac12c67d56 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:30:22.464167 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-639a3c07-ede0-4683-9b30-3aac12c67d56 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] 10.222.0.16 "GET /compute/v2.1/servers/08a33911-6e47-45fd-b499-155f199aacf8" status: 200 len: 1808 microversion: 2.20 time: 0.350231 Jul 16 21:30:22.469312 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2236/4458] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:30:22 2022] GET /compute/v2.1/servers/08a33911-6e47-45fd-b499-155f199aacf8 => generated 1808 bytes in 356 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:30:22.563900 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-c5376a0b-1cb3-4429-872c-9ad6f55dbb21 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ddfae3448704591b662de83b97a1876', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bed7c94624484c7cb8abeb3bfc8757b9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:30:22.574893 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-c5376a0b-1cb3-4429-872c-9ad6f55dbb21 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] 10.222.0.16 "GET /compute/v2.1/servers/d17892b8-b5bd-469e-9284-0abb3115daaa" status: 200 len: 1374 microversion: 2.1 time: 0.437894 Jul 16 21:30:22.575885 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2223/4459] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:30:22 2022] GET /compute/v2.1/servers/d17892b8-b5bd-469e-9284-0abb3115daaa => generated 1374 bytes in 440 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:30:22.943051 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-3006f436-d424-4a9d-983b-d08e1338f015 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:30:22.943860 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-3006f436-d424-4a9d-983b-d08e1338f015 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] [instance: 515fd3da-3965-4882-b9e8-100a7eea64d9] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:30:22.948979 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell0:SELECT=2 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:30:22.965765 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3006f436-d424-4a9d-983b-d08e1338f015 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:30:22.973206 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3006f436-d424-4a9d-983b-d08e1338f015 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:30:22.975752 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3006f436-d424-4a9d-983b-d08e1338f015 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:30:22.977555 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3006f436-d424-4a9d-983b-d08e1338f015 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:30:23.219266 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-3006f436-d424-4a9d-983b-d08e1338f015 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da77ec3d0f04496d8c8df4025bba76d1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '207ec2d1883c4a1ba89696e6e77cf3aa', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:30:23.234942 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-3006f436-d424-4a9d-983b-d08e1338f015 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] 10.222.0.16 "GET /compute/v2.1/servers/515fd3da-3965-4882-b9e8-100a7eea64d9" status: 200 len: 1506 microversion: 2.1 time: 0.295534 Jul 16 21:30:23.236285 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2237/4460] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:30:22 2022] GET /compute/v2.1/servers/515fd3da-3965-4882-b9e8-100a7eea64d9 => generated 1506 bytes in 300 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:30:23.489117 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-5ecb98bc-01dc-46b5-b41a-f9576b867642 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:30:23.490738 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-5ecb98bc-01dc-46b5-b41a-f9576b867642 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] [instance: 08a33911-6e47-45fd-b499-155f199aacf8] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:30:23.496066 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_api:SELECT=79,nova_api:INSERT=4 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:30:23.507404 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5ecb98bc-01dc-46b5-b41a-f9576b867642 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:30:23.511077 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5ecb98bc-01dc-46b5-b41a-f9576b867642 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:30:23.512256 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5ecb98bc-01dc-46b5-b41a-f9576b867642 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:30:23.513862 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5ecb98bc-01dc-46b5-b41a-f9576b867642 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:30:23.594170 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-a61358b7-dc77-49f4-8bbd-71f5d6850466 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:30:23.598552 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-a61358b7-dc77-49f4-8bbd-71f5d6850466 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] [instance: d17892b8-b5bd-469e-9284-0abb3115daaa] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:30:23.607532 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a61358b7-dc77-49f4-8bbd-71f5d6850466 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:30:23.609788 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a61358b7-dc77-49f4-8bbd-71f5d6850466 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:30:23.611207 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a61358b7-dc77-49f4-8bbd-71f5d6850466 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:30:23.611870 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a61358b7-dc77-49f4-8bbd-71f5d6850466 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:30:23.928777 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-5ecb98bc-01dc-46b5-b41a-f9576b867642 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:30:23.948303 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-5ecb98bc-01dc-46b5-b41a-f9576b867642 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:30:23.951335 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-5ecb98bc-01dc-46b5-b41a-f9576b867642 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:30:23.963240 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-5ecb98bc-01dc-46b5-b41a-f9576b867642 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] 10.222.0.16 "GET /compute/v2.1/servers/08a33911-6e47-45fd-b499-155f199aacf8" status: 200 len: 1808 microversion: 2.20 time: 0.467993 Jul 16 21:30:23.963240 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2224/4461] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:30:23 2022] GET /compute/v2.1/servers/08a33911-6e47-45fd-b499-155f199aacf8 => generated 1808 bytes in 470 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:30:23.981829 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-a61358b7-dc77-49f4-8bbd-71f5d6850466 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ddfae3448704591b662de83b97a1876', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bed7c94624484c7cb8abeb3bfc8757b9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:30:23.990393 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-a61358b7-dc77-49f4-8bbd-71f5d6850466 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] 10.222.0.16 "GET /compute/v2.1/servers/d17892b8-b5bd-469e-9284-0abb3115daaa" status: 200 len: 1374 microversion: 2.1 time: 0.401173 Jul 16 21:30:24.000861 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2238/4462] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:30:23 2022] GET /compute/v2.1/servers/d17892b8-b5bd-469e-9284-0abb3115daaa => generated 1374 bytes in 412 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:30:24.276386 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-cda0c88c-f34f-4158-bbf8-e45e651b5c25 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:30:24.277216 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-cda0c88c-f34f-4158-bbf8-e45e651b5c25 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] [instance: 515fd3da-3965-4882-b9e8-100a7eea64d9] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:30:24.293834 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cda0c88c-f34f-4158-bbf8-e45e651b5c25 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:30:24.294750 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cda0c88c-f34f-4158-bbf8-e45e651b5c25 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:30:24.295822 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cda0c88c-f34f-4158-bbf8-e45e651b5c25 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:30:24.296743 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cda0c88c-f34f-4158-bbf8-e45e651b5c25 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:30:24.525627 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-cda0c88c-f34f-4158-bbf8-e45e651b5c25 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da77ec3d0f04496d8c8df4025bba76d1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '207ec2d1883c4a1ba89696e6e77cf3aa', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:30:24.539840 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-cda0c88c-f34f-4158-bbf8-e45e651b5c25 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] 10.222.0.16 "GET /compute/v2.1/servers/515fd3da-3965-4882-b9e8-100a7eea64d9" status: 200 len: 1506 microversion: 2.1 time: 0.268790 Jul 16 21:30:24.542672 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2225/4463] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:30:24 2022] GET /compute/v2.1/servers/515fd3da-3965-4882-b9e8-100a7eea64d9 => generated 1506 bytes in 273 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:30:24.975606 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-0d9f7042-0bf7-4cb2-9e84-7936a1995676 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:30:24.976280 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-0d9f7042-0bf7-4cb2-9e84-7936a1995676 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] [instance: 08a33911-6e47-45fd-b499-155f199aacf8] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:30:24.984185 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0d9f7042-0bf7-4cb2-9e84-7936a1995676 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:30:24.985464 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0d9f7042-0bf7-4cb2-9e84-7936a1995676 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:30:24.986441 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0d9f7042-0bf7-4cb2-9e84-7936a1995676 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:30:24.987103 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0d9f7042-0bf7-4cb2-9e84-7936a1995676 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:30:25.008469 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-951e627e-726b-410d-97c3-1eaaf7082bd8 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:30:25.009923 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-951e627e-726b-410d-97c3-1eaaf7082bd8 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] [instance: d17892b8-b5bd-469e-9284-0abb3115daaa] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:30:25.060820 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-951e627e-726b-410d-97c3-1eaaf7082bd8 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:30:25.064077 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-951e627e-726b-410d-97c3-1eaaf7082bd8 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:30:25.071517 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-951e627e-726b-410d-97c3-1eaaf7082bd8 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:30:25.071517 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-951e627e-726b-410d-97c3-1eaaf7082bd8 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:30:25.381858 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-0d9f7042-0bf7-4cb2-9e84-7936a1995676 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:30:25.391315 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-0d9f7042-0bf7-4cb2-9e84-7936a1995676 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:30:25.392990 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-0d9f7042-0bf7-4cb2-9e84-7936a1995676 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:30:25.394377 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-0d9f7042-0bf7-4cb2-9e84-7936a1995676 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] 10.222.0.16 "GET /compute/v2.1/servers/08a33911-6e47-45fd-b499-155f199aacf8" status: 200 len: 1808 microversion: 2.20 time: 0.425250 Jul 16 21:30:25.397816 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2239/4464] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:30:24 2022] GET /compute/v2.1/servers/08a33911-6e47-45fd-b499-155f199aacf8 => generated 1808 bytes in 427 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:30:25.422233 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-951e627e-726b-410d-97c3-1eaaf7082bd8 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ddfae3448704591b662de83b97a1876', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bed7c94624484c7cb8abeb3bfc8757b9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:30:25.430563 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-951e627e-726b-410d-97c3-1eaaf7082bd8 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] 10.222.0.16 "GET /compute/v2.1/servers/d17892b8-b5bd-469e-9284-0abb3115daaa" status: 200 len: 1374 microversion: 2.1 time: 0.426263 Jul 16 21:30:25.431237 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2226/4465] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:30:25 2022] GET /compute/v2.1/servers/d17892b8-b5bd-469e-9284-0abb3115daaa => generated 1374 bytes in 428 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:30:25.559792 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-63d3326c-752f-45fc-952f-5dbae365a54c tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:30:25.560414 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-63d3326c-752f-45fc-952f-5dbae365a54c tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] [instance: 515fd3da-3965-4882-b9e8-100a7eea64d9] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:30:25.566438 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-63d3326c-752f-45fc-952f-5dbae365a54c tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:30:25.566892 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-63d3326c-752f-45fc-952f-5dbae365a54c tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:30:25.567809 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-63d3326c-752f-45fc-952f-5dbae365a54c tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:30:25.568252 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-63d3326c-752f-45fc-952f-5dbae365a54c tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:30:25.718949 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-63d3326c-752f-45fc-952f-5dbae365a54c tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da77ec3d0f04496d8c8df4025bba76d1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '207ec2d1883c4a1ba89696e6e77cf3aa', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:30:25.731889 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-63d3326c-752f-45fc-952f-5dbae365a54c tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] 10.222.0.16 "GET /compute/v2.1/servers/515fd3da-3965-4882-b9e8-100a7eea64d9" status: 200 len: 1506 microversion: 2.1 time: 0.176783 Jul 16 21:30:25.732808 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2240/4466] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:30:25 2022] GET /compute/v2.1/servers/515fd3da-3965-4882-b9e8-100a7eea64d9 => generated 1506 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:30:26.413077 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-8d59cd0a-1b4f-4735-a4a1-ddd3f7ca6b9f tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:30:26.414358 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-8d59cd0a-1b4f-4735-a4a1-ddd3f7ca6b9f tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] [instance: 08a33911-6e47-45fd-b499-155f199aacf8] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:30:26.426291 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8d59cd0a-1b4f-4735-a4a1-ddd3f7ca6b9f tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:30:26.430195 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8d59cd0a-1b4f-4735-a4a1-ddd3f7ca6b9f tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:30:26.456014 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-b793c523-10ff-45c3-a52b-5ba63364746f tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:30:26.457421 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-b793c523-10ff-45c3-a52b-5ba63364746f tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] [instance: d17892b8-b5bd-469e-9284-0abb3115daaa] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:30:26.464250 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8d59cd0a-1b4f-4735-a4a1-ddd3f7ca6b9f tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:30:26.465864 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8d59cd0a-1b4f-4735-a4a1-ddd3f7ca6b9f tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.027s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:30:26.476701 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b793c523-10ff-45c3-a52b-5ba63364746f tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:30:26.476701 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b793c523-10ff-45c3-a52b-5ba63364746f tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:30:26.476701 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b793c523-10ff-45c3-a52b-5ba63364746f tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:30:26.476701 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b793c523-10ff-45c3-a52b-5ba63364746f tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:30:26.831793 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-8d59cd0a-1b4f-4735-a4a1-ddd3f7ca6b9f tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:30:26.847766 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-8d59cd0a-1b4f-4735-a4a1-ddd3f7ca6b9f tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:30:26.855713 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-8d59cd0a-1b4f-4735-a4a1-ddd3f7ca6b9f tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:30:26.857580 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-8d59cd0a-1b4f-4735-a4a1-ddd3f7ca6b9f tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] 10.222.0.16 "GET /compute/v2.1/servers/08a33911-6e47-45fd-b499-155f199aacf8" status: 200 len: 1808 microversion: 2.20 time: 0.448450 Jul 16 21:30:26.858400 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2227/4467] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:30:26 2022] GET /compute/v2.1/servers/08a33911-6e47-45fd-b499-155f199aacf8 => generated 1808 bytes in 450 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:30:26.872079 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-586fbb82-f9f9-483f-87dd-b01607be43b5 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:30:26.872079 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-586fbb82-f9f9-483f-87dd-b01607be43b5 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] [instance: 515fd3da-3965-4882-b9e8-100a7eea64d9] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:30:26.886044 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-b793c523-10ff-45c3-a52b-5ba63364746f tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ddfae3448704591b662de83b97a1876', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bed7c94624484c7cb8abeb3bfc8757b9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:30:26.895139 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-b793c523-10ff-45c3-a52b-5ba63364746f tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] 10.222.0.16 "GET /compute/v2.1/servers/d17892b8-b5bd-469e-9284-0abb3115daaa" status: 200 len: 1374 microversion: 2.1 time: 0.443026 Jul 16 21:30:26.899110 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-586fbb82-f9f9-483f-87dd-b01607be43b5 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:30:26.900213 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-586fbb82-f9f9-483f-87dd-b01607be43b5 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:30:26.912310 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-586fbb82-f9f9-483f-87dd-b01607be43b5 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:30:26.912635 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2241/4468] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:30:26 2022] GET /compute/v2.1/servers/d17892b8-b5bd-469e-9284-0abb3115daaa => generated 1374 bytes in 460 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:30:26.918987 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-0a383338-fd21-4224-b6e6-c3e157ee52de req-0a0fe60e-5152-4ecf-83fb-22030a21d7f6 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "d17892b8-b5bd-469e-9284-0abb3115daaa", "name": "network-vif-plugged", "status": "completed", "tag": "88f48838-7774-4261-8f33-ad28281a1d60"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:30:26.920004 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-586fbb82-f9f9-483f-87dd-b01607be43b5 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:30:26.950336 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-0a383338-fd21-4224-b6e6-c3e157ee52de req-0a0fe60e-5152-4ecf-83fb-22030a21d7f6 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:30:26.953051 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-0a383338-fd21-4224-b6e6-c3e157ee52de req-0a0fe60e-5152-4ecf-83fb-22030a21d7f6 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:30:27.000876 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-0a383338-fd21-4224-b6e6-c3e157ee52de req-0a0fe60e-5152-4ecf-83fb-22030a21d7f6 service nova] Creating event network-vif-plugged:88f48838-7774-4261-8f33-ad28281a1d60 for instance d17892b8-b5bd-469e-9284-0abb3115daaa on nch2-master-1 Jul 16 21:30:27.018863 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-0a383338-fd21-4224-b6e6-c3e157ee52de req-0a0fe60e-5152-4ecf-83fb-22030a21d7f6 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.104586 Jul 16 21:30:27.025846 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2242/4469] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:30:26 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 112 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:30:27.297818 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-586fbb82-f9f9-483f-87dd-b01607be43b5 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da77ec3d0f04496d8c8df4025bba76d1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '207ec2d1883c4a1ba89696e6e77cf3aa', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:30:27.319236 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-586fbb82-f9f9-483f-87dd-b01607be43b5 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] 10.222.0.16 "GET /compute/v2.1/servers/515fd3da-3965-4882-b9e8-100a7eea64d9" status: 200 len: 1506 microversion: 2.1 time: 0.452253 Jul 16 21:30:27.320217 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2228/4470] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:30:26 2022] GET /compute/v2.1/servers/515fd3da-3965-4882-b9e8-100a7eea64d9 => generated 1506 bytes in 461 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:30:27.897517 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-0c5fec81-9be3-4949-b569-2088a9b2ce87 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:30:27.900665 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-0c5fec81-9be3-4949-b569-2088a9b2ce87 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] [instance: 08a33911-6e47-45fd-b499-155f199aacf8] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:30:27.933703 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-4e4e9f2c-3494-488e-800e-78ce2938d68a tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:30:27.934752 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0c5fec81-9be3-4949-b569-2088a9b2ce87 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:30:27.935860 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0c5fec81-9be3-4949-b569-2088a9b2ce87 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:30:27.937369 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0c5fec81-9be3-4949-b569-2088a9b2ce87 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:30:27.942011 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-4e4e9f2c-3494-488e-800e-78ce2938d68a tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] [instance: d17892b8-b5bd-469e-9284-0abb3115daaa] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:30:27.943884 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0c5fec81-9be3-4949-b569-2088a9b2ce87 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:30:27.966536 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4e4e9f2c-3494-488e-800e-78ce2938d68a tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:30:27.968503 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4e4e9f2c-3494-488e-800e-78ce2938d68a tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.015s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:30:27.969787 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4e4e9f2c-3494-488e-800e-78ce2938d68a tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:30:27.971772 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4e4e9f2c-3494-488e-800e-78ce2938d68a tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:30:28.383131 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-0c5fec81-9be3-4949-b569-2088a9b2ce87 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:30:28.396558 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-0c5fec81-9be3-4949-b569-2088a9b2ce87 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:30:28.399266 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-0c5fec81-9be3-4949-b569-2088a9b2ce87 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:30:28.401693 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-0c5fec81-9be3-4949-b569-2088a9b2ce87 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] 10.222.0.16 "GET /compute/v2.1/servers/08a33911-6e47-45fd-b499-155f199aacf8" status: 200 len: 1808 microversion: 2.20 time: 0.525417 Jul 16 21:30:28.408235 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2243/4471] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:30:27 2022] GET /compute/v2.1/servers/08a33911-6e47-45fd-b499-155f199aacf8 => generated 1808 bytes in 534 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:30:28.423450 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-23f53e7b-cb4c-4503-bc59-56b9fb7fa2f5 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:30:28.424582 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-23f53e7b-cb4c-4503-bc59-56b9fb7fa2f5 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] [instance: 515fd3da-3965-4882-b9e8-100a7eea64d9] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:30:28.438344 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-4e4e9f2c-3494-488e-800e-78ce2938d68a tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ddfae3448704591b662de83b97a1876', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bed7c94624484c7cb8abeb3bfc8757b9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:30:28.445903 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-23f53e7b-cb4c-4503-bc59-56b9fb7fa2f5 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:30:28.447257 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-23f53e7b-cb4c-4503-bc59-56b9fb7fa2f5 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:30:28.448278 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-23f53e7b-cb4c-4503-bc59-56b9fb7fa2f5 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:30:28.448906 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-23f53e7b-cb4c-4503-bc59-56b9fb7fa2f5 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:30:28.460369 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-4e4e9f2c-3494-488e-800e-78ce2938d68a tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] 10.222.0.16 "GET /compute/v2.1/servers/d17892b8-b5bd-469e-9284-0abb3115daaa" status: 200 len: 1374 microversion: 2.1 time: 0.545420 Jul 16 21:30:28.475920 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2229/4472] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:30:27 2022] GET /compute/v2.1/servers/d17892b8-b5bd-469e-9284-0abb3115daaa => generated 1374 bytes in 560 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:30:28.743991 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-23f53e7b-cb4c-4503-bc59-56b9fb7fa2f5 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da77ec3d0f04496d8c8df4025bba76d1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '207ec2d1883c4a1ba89696e6e77cf3aa', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:30:28.753911 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-23f53e7b-cb4c-4503-bc59-56b9fb7fa2f5 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] 10.222.0.16 "GET /compute/v2.1/servers/515fd3da-3965-4882-b9e8-100a7eea64d9" status: 200 len: 1506 microversion: 2.1 time: 0.343151 Jul 16 21:30:28.770393 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2244/4473] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:30:28 2022] GET /compute/v2.1/servers/515fd3da-3965-4882-b9e8-100a7eea64d9 => generated 1506 bytes in 355 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:30:29.420720 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-ea32a2ac-8083-41fe-8d27-6551cd825d2b tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:30:29.424954 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-ea32a2ac-8083-41fe-8d27-6551cd825d2b tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] [instance: 08a33911-6e47-45fd-b499-155f199aacf8] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:30:29.444043 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ea32a2ac-8083-41fe-8d27-6551cd825d2b tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:30:29.444831 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ea32a2ac-8083-41fe-8d27-6551cd825d2b tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:30:29.447043 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ea32a2ac-8083-41fe-8d27-6551cd825d2b tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:30:29.449577 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ea32a2ac-8083-41fe-8d27-6551cd825d2b tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:30:29.480659 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-2305e8bb-2272-4803-ae34-d4c05fbe7f4b tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:30:29.482306 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-2305e8bb-2272-4803-ae34-d4c05fbe7f4b tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] [instance: d17892b8-b5bd-469e-9284-0abb3115daaa] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:30:29.496283 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2305e8bb-2272-4803-ae34-d4c05fbe7f4b tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:30:29.499035 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2305e8bb-2272-4803-ae34-d4c05fbe7f4b tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:30:29.500282 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2305e8bb-2272-4803-ae34-d4c05fbe7f4b tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:30:29.500957 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2305e8bb-2272-4803-ae34-d4c05fbe7f4b tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:30:29.773010 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-ea32a2ac-8083-41fe-8d27-6551cd825d2b tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:30:29.785044 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-ea32a2ac-8083-41fe-8d27-6551cd825d2b tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:30:29.787867 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-ea32a2ac-8083-41fe-8d27-6551cd825d2b tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:30:29.790720 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-ea32a2ac-8083-41fe-8d27-6551cd825d2b tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] 10.222.0.16 "GET /compute/v2.1/servers/08a33911-6e47-45fd-b499-155f199aacf8" status: 200 len: 1808 microversion: 2.20 time: 0.373967 Jul 16 21:30:29.792102 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2230/4474] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:30:29 2022] GET /compute/v2.1/servers/08a33911-6e47-45fd-b499-155f199aacf8 => generated 1808 bytes in 377 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:30:29.797415 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-e6dfe59c-3b9e-4817-a7cc-7d26d23da2f2 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:30:29.798447 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-e6dfe59c-3b9e-4817-a7cc-7d26d23da2f2 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] [instance: 515fd3da-3965-4882-b9e8-100a7eea64d9] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:30:29.806530 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e6dfe59c-3b9e-4817-a7cc-7d26d23da2f2 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:30:29.813708 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e6dfe59c-3b9e-4817-a7cc-7d26d23da2f2 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:30:29.813708 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e6dfe59c-3b9e-4817-a7cc-7d26d23da2f2 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:30:29.813708 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e6dfe59c-3b9e-4817-a7cc-7d26d23da2f2 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:30:29.827828 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-2305e8bb-2272-4803-ae34-d4c05fbe7f4b tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ddfae3448704591b662de83b97a1876', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bed7c94624484c7cb8abeb3bfc8757b9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:30:29.835532 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-2305e8bb-2272-4803-ae34-d4c05fbe7f4b tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] 10.222.0.16 "GET /compute/v2.1/servers/d17892b8-b5bd-469e-9284-0abb3115daaa" status: 200 len: 1374 microversion: 2.1 time: 0.359675 Jul 16 21:30:29.837537 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2245/4475] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:30:29 2022] GET /compute/v2.1/servers/d17892b8-b5bd-469e-9284-0abb3115daaa => generated 1374 bytes in 361 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:30:29.955535 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-e6dfe59c-3b9e-4817-a7cc-7d26d23da2f2 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da77ec3d0f04496d8c8df4025bba76d1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '207ec2d1883c4a1ba89696e6e77cf3aa', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:30:29.963802 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-e6dfe59c-3b9e-4817-a7cc-7d26d23da2f2 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] 10.222.0.16 "GET /compute/v2.1/servers/515fd3da-3965-4882-b9e8-100a7eea64d9" status: 200 len: 1506 microversion: 2.1 time: 0.170438 Jul 16 21:30:29.964395 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2231/4476] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:30:29 2022] GET /compute/v2.1/servers/515fd3da-3965-4882-b9e8-100a7eea64d9 => generated 1506 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:30:30.809906 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-2e3f37d1-2ef5-464f-9cae-80fde46b2146 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:30:30.811333 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-2e3f37d1-2ef5-464f-9cae-80fde46b2146 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] [instance: 08a33911-6e47-45fd-b499-155f199aacf8] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:30:30.820902 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2e3f37d1-2ef5-464f-9cae-80fde46b2146 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:30:30.822392 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2e3f37d1-2ef5-464f-9cae-80fde46b2146 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:30:30.823709 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2e3f37d1-2ef5-464f-9cae-80fde46b2146 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:30:30.824363 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2e3f37d1-2ef5-464f-9cae-80fde46b2146 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:30:30.857552 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-5e36852c-c2b1-426e-88d6-624d059c6901 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:30:30.859741 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-5e36852c-c2b1-426e-88d6-624d059c6901 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] [instance: d17892b8-b5bd-469e-9284-0abb3115daaa] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:30:30.876120 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5e36852c-c2b1-426e-88d6-624d059c6901 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:30:30.876856 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5e36852c-c2b1-426e-88d6-624d059c6901 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:30:30.877888 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5e36852c-c2b1-426e-88d6-624d059c6901 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:30:30.879034 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5e36852c-c2b1-426e-88d6-624d059c6901 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:30:30.888111 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=100 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:30:31.201589 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-2e3f37d1-2ef5-464f-9cae-80fde46b2146 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:30:31.221928 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-5e36852c-c2b1-426e-88d6-624d059c6901 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ddfae3448704591b662de83b97a1876', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bed7c94624484c7cb8abeb3bfc8757b9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:30:31.230269 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-2e3f37d1-2ef5-464f-9cae-80fde46b2146 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:30:31.234478 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-2e3f37d1-2ef5-464f-9cae-80fde46b2146 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:30:31.237100 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-5e36852c-c2b1-426e-88d6-624d059c6901 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] 10.222.0.16 "GET /compute/v2.1/servers/d17892b8-b5bd-469e-9284-0abb3115daaa" status: 200 len: 1546 microversion: 2.1 time: 0.388550 Jul 16 21:30:31.248642 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-2e3f37d1-2ef5-464f-9cae-80fde46b2146 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] 10.222.0.16 "GET /compute/v2.1/servers/08a33911-6e47-45fd-b499-155f199aacf8" status: 200 len: 1808 microversion: 2.20 time: 0.432365 Jul 16 21:30:31.248997 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2232/4477] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:30:30 2022] GET /compute/v2.1/servers/d17892b8-b5bd-469e-9284-0abb3115daaa => generated 1546 bytes in 390 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:30:31.248997 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-127841c4-99f9-4ab9-99a3-1cc07f143559 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:30:31.248997 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-127841c4-99f9-4ab9-99a3-1cc07f143559 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] [instance: 515fd3da-3965-4882-b9e8-100a7eea64d9] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:30:31.251615 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2246/4478] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:30:30 2022] GET /compute/v2.1/servers/08a33911-6e47-45fd-b499-155f199aacf8 => generated 1808 bytes in 447 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:30:31.256690 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-d7a7ccf1-e449-4280-acb7-ba399e1754d6 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:30:31.258052 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-d7a7ccf1-e449-4280-acb7-ba399e1754d6 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] [instance: d17892b8-b5bd-469e-9284-0abb3115daaa] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:30:31.265980 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d7a7ccf1-e449-4280-acb7-ba399e1754d6 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:30:31.266719 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d7a7ccf1-e449-4280-acb7-ba399e1754d6 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:30:31.267693 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d7a7ccf1-e449-4280-acb7-ba399e1754d6 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:30:31.268305 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d7a7ccf1-e449-4280-acb7-ba399e1754d6 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:30:31.270125 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-127841c4-99f9-4ab9-99a3-1cc07f143559 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:30:31.270606 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-127841c4-99f9-4ab9-99a3-1cc07f143559 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:30:31.271756 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-127841c4-99f9-4ab9-99a3-1cc07f143559 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:30:31.272259 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-127841c4-99f9-4ab9-99a3-1cc07f143559 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:30:31.557221 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-d7a7ccf1-e449-4280-acb7-ba399e1754d6 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ddfae3448704591b662de83b97a1876', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bed7c94624484c7cb8abeb3bfc8757b9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:30:31.580809 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-d7a7ccf1-e449-4280-acb7-ba399e1754d6 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] 10.222.0.16 "GET /compute/v2.1/servers/d17892b8-b5bd-469e-9284-0abb3115daaa" status: 200 len: 1546 microversion: 2.1 time: 0.327878 Jul 16 21:30:31.583709 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2247/4479] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:30:31 2022] GET /compute/v2.1/servers/d17892b8-b5bd-469e-9284-0abb3115daaa => generated 1546 bytes in 331 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:30:31.603177 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-127841c4-99f9-4ab9-99a3-1cc07f143559 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da77ec3d0f04496d8c8df4025bba76d1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '207ec2d1883c4a1ba89696e6e77cf3aa', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:30:31.617665 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-127841c4-99f9-4ab9-99a3-1cc07f143559 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] 10.222.0.16 "GET /compute/v2.1/servers/515fd3da-3965-4882-b9e8-100a7eea64d9" status: 200 len: 1506 microversion: 2.1 time: 0.377940 Jul 16 21:30:31.628147 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2233/4480] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:30:31 2022] GET /compute/v2.1/servers/515fd3da-3965-4882-b9e8-100a7eea64d9 => generated 1506 bytes in 389 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:30:32.270184 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-92c76760-4cfa-45a9-baeb-343e0425de06 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:30:32.270808 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-92c76760-4cfa-45a9-baeb-343e0425de06 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] [instance: 08a33911-6e47-45fd-b499-155f199aacf8] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:30:32.278933 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-92c76760-4cfa-45a9-baeb-343e0425de06 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:30:32.280257 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-92c76760-4cfa-45a9-baeb-343e0425de06 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:30:32.281918 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-92c76760-4cfa-45a9-baeb-343e0425de06 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:30:32.282573 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-92c76760-4cfa-45a9-baeb-343e0425de06 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:30:32.483486 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-92c76760-4cfa-45a9-baeb-343e0425de06 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:30:32.486432 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=100 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:30:32.497961 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-92c76760-4cfa-45a9-baeb-343e0425de06 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:30:32.503121 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-92c76760-4cfa-45a9-baeb-343e0425de06 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:30:32.504354 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-92c76760-4cfa-45a9-baeb-343e0425de06 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] 10.222.0.16 "GET /compute/v2.1/servers/08a33911-6e47-45fd-b499-155f199aacf8" status: 200 len: 1808 microversion: 2.20 time: 0.238184 Jul 16 21:30:32.505077 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2248/4481] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:30:32 2022] GET /compute/v2.1/servers/08a33911-6e47-45fd-b499-155f199aacf8 => generated 1808 bytes in 239 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:30:32.637221 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-947c2df2-8699-4072-a201-d4e089bf3a11 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:30:32.638139 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-947c2df2-8699-4072-a201-d4e089bf3a11 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] [instance: 515fd3da-3965-4882-b9e8-100a7eea64d9] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:30:32.650921 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-947c2df2-8699-4072-a201-d4e089bf3a11 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:30:32.651411 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-947c2df2-8699-4072-a201-d4e089bf3a11 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:30:32.652210 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-947c2df2-8699-4072-a201-d4e089bf3a11 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:30:32.653080 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-947c2df2-8699-4072-a201-d4e089bf3a11 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:30:32.848823 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-947c2df2-8699-4072-a201-d4e089bf3a11 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da77ec3d0f04496d8c8df4025bba76d1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '207ec2d1883c4a1ba89696e6e77cf3aa', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:30:32.863568 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-947c2df2-8699-4072-a201-d4e089bf3a11 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] 10.222.0.16 "GET /compute/v2.1/servers/515fd3da-3965-4882-b9e8-100a7eea64d9" status: 200 len: 1506 microversion: 2.1 time: 0.231906 Jul 16 21:30:32.878314 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2234/4482] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:30:32 2022] GET /compute/v2.1/servers/515fd3da-3965-4882-b9e8-100a7eea64d9 => generated 1506 bytes in 248 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:30:33.525345 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-f752fe02-ece0-4ea5-80f3-1cbf6ce5eacd tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:30:33.525977 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-f752fe02-ece0-4ea5-80f3-1cbf6ce5eacd tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] [instance: 08a33911-6e47-45fd-b499-155f199aacf8] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:30:33.534743 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_api:SELECT=70 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:30:33.537122 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f752fe02-ece0-4ea5-80f3-1cbf6ce5eacd tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:30:33.537802 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f752fe02-ece0-4ea5-80f3-1cbf6ce5eacd tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:30:33.538822 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f752fe02-ece0-4ea5-80f3-1cbf6ce5eacd tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:30:33.539441 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f752fe02-ece0-4ea5-80f3-1cbf6ce5eacd tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:30:33.882454 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-dab700ff-00ea-420b-9d1b-a86701ba6ea7 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:30:33.883200 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-dab700ff-00ea-420b-9d1b-a86701ba6ea7 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] [instance: 515fd3da-3965-4882-b9e8-100a7eea64d9] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:30:33.894434 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-dab700ff-00ea-420b-9d1b-a86701ba6ea7 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:30:33.902019 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-dab700ff-00ea-420b-9d1b-a86701ba6ea7 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:30:33.904020 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-dab700ff-00ea-420b-9d1b-a86701ba6ea7 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:30:33.907554 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-dab700ff-00ea-420b-9d1b-a86701ba6ea7 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:30:33.984459 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-f752fe02-ece0-4ea5-80f3-1cbf6ce5eacd tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:30:34.000614 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-f752fe02-ece0-4ea5-80f3-1cbf6ce5eacd tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:30:34.004233 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-f752fe02-ece0-4ea5-80f3-1cbf6ce5eacd tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:30:34.005908 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-f752fe02-ece0-4ea5-80f3-1cbf6ce5eacd tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] 10.222.0.16 "GET /compute/v2.1/servers/08a33911-6e47-45fd-b499-155f199aacf8" status: 200 len: 1808 microversion: 2.20 time: 0.485294 Jul 16 21:30:34.007355 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2249/4483] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:30:33 2022] GET /compute/v2.1/servers/08a33911-6e47-45fd-b499-155f199aacf8 => generated 1808 bytes in 488 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:30:34.262179 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-dab700ff-00ea-420b-9d1b-a86701ba6ea7 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da77ec3d0f04496d8c8df4025bba76d1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '207ec2d1883c4a1ba89696e6e77cf3aa', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:30:34.270816 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-dab700ff-00ea-420b-9d1b-a86701ba6ea7 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] 10.222.0.16 "GET /compute/v2.1/servers/515fd3da-3965-4882-b9e8-100a7eea64d9" status: 200 len: 1506 microversion: 2.1 time: 0.393643 Jul 16 21:30:34.285241 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2235/4484] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:30:33 2022] GET /compute/v2.1/servers/515fd3da-3965-4882-b9e8-100a7eea64d9 => generated 1506 bytes in 409 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:30:35.026439 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-a4c385b7-8381-4b38-97aa-15ea492a505a tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:30:35.032187 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-a4c385b7-8381-4b38-97aa-15ea492a505a tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] [instance: 08a33911-6e47-45fd-b499-155f199aacf8] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:30:35.045587 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a4c385b7-8381-4b38-97aa-15ea492a505a tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:30:35.046222 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a4c385b7-8381-4b38-97aa-15ea492a505a tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:30:35.047324 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a4c385b7-8381-4b38-97aa-15ea492a505a tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:30:35.047885 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a4c385b7-8381-4b38-97aa-15ea492a505a tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:30:35.290824 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-cccadbc0-8d86-42a3-8214-1976b852fbc8 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:30:35.292038 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-cccadbc0-8d86-42a3-8214-1976b852fbc8 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] [instance: 515fd3da-3965-4882-b9e8-100a7eea64d9] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:30:35.309005 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cccadbc0-8d86-42a3-8214-1976b852fbc8 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:30:35.309717 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cccadbc0-8d86-42a3-8214-1976b852fbc8 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:30:35.310740 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cccadbc0-8d86-42a3-8214-1976b852fbc8 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:30:35.311363 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cccadbc0-8d86-42a3-8214-1976b852fbc8 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:30:35.532739 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-a4c385b7-8381-4b38-97aa-15ea492a505a tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:30:35.542192 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-a4c385b7-8381-4b38-97aa-15ea492a505a tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:30:35.543941 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-a4c385b7-8381-4b38-97aa-15ea492a505a tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:30:35.545337 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-a4c385b7-8381-4b38-97aa-15ea492a505a tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] 10.222.0.16 "GET /compute/v2.1/servers/08a33911-6e47-45fd-b499-155f199aacf8" status: 200 len: 1808 microversion: 2.20 time: 0.524014 Jul 16 21:30:35.545979 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2250/4485] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:30:35 2022] GET /compute/v2.1/servers/08a33911-6e47-45fd-b499-155f199aacf8 => generated 1808 bytes in 525 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:30:35.753084 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-cccadbc0-8d86-42a3-8214-1976b852fbc8 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da77ec3d0f04496d8c8df4025bba76d1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '207ec2d1883c4a1ba89696e6e77cf3aa', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:30:35.761713 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-cccadbc0-8d86-42a3-8214-1976b852fbc8 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] 10.222.0.16 "GET /compute/v2.1/servers/515fd3da-3965-4882-b9e8-100a7eea64d9" status: 200 len: 1506 microversion: 2.1 time: 0.475499 Jul 16 21:30:35.762573 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2236/4486] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:30:35 2022] GET /compute/v2.1/servers/515fd3da-3965-4882-b9e8-100a7eea64d9 => generated 1506 bytes in 477 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:30:36.564215 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-ce6c4f50-a9bc-496b-8f4f-de2e07a79e54 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:30:36.564818 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-ce6c4f50-a9bc-496b-8f4f-de2e07a79e54 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] [instance: 08a33911-6e47-45fd-b499-155f199aacf8] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:30:36.572267 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ce6c4f50-a9bc-496b-8f4f-de2e07a79e54 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:30:36.572726 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ce6c4f50-a9bc-496b-8f4f-de2e07a79e54 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:30:36.573520 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ce6c4f50-a9bc-496b-8f4f-de2e07a79e54 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:30:36.573966 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ce6c4f50-a9bc-496b-8f4f-de2e07a79e54 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:30:36.780512 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-cb1e61b3-64e0-4462-9c67-33c2c2b5c542 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:30:36.781801 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-cb1e61b3-64e0-4462-9c67-33c2c2b5c542 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] [instance: 515fd3da-3965-4882-b9e8-100a7eea64d9] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:30:36.791325 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cb1e61b3-64e0-4462-9c67-33c2c2b5c542 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:30:36.792134 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cb1e61b3-64e0-4462-9c67-33c2c2b5c542 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:30:36.793205 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cb1e61b3-64e0-4462-9c67-33c2c2b5c542 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:30:36.793891 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cb1e61b3-64e0-4462-9c67-33c2c2b5c542 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:30:36.851931 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-ce6c4f50-a9bc-496b-8f4f-de2e07a79e54 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:30:36.873516 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-ce6c4f50-a9bc-496b-8f4f-de2e07a79e54 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:30:36.875874 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-ce6c4f50-a9bc-496b-8f4f-de2e07a79e54 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:30:36.880019 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-ce6c4f50-a9bc-496b-8f4f-de2e07a79e54 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] 10.222.0.16 "GET /compute/v2.1/servers/08a33911-6e47-45fd-b499-155f199aacf8" status: 200 len: 1808 microversion: 2.20 time: 0.319884 Jul 16 21:30:36.880606 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2251/4487] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:30:36 2022] GET /compute/v2.1/servers/08a33911-6e47-45fd-b499-155f199aacf8 => generated 1808 bytes in 321 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 16 21:30:37.133324 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-cb1e61b3-64e0-4462-9c67-33c2c2b5c542 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da77ec3d0f04496d8c8df4025bba76d1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '207ec2d1883c4a1ba89696e6e77cf3aa', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:30:37.144625 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-cb1e61b3-64e0-4462-9c67-33c2c2b5c542 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] 10.222.0.16 "GET /compute/v2.1/servers/515fd3da-3965-4882-b9e8-100a7eea64d9" status: 200 len: 1486 microversion: 2.1 time: 0.368810 Jul 16 21:30:37.145370 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2237/4488] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:30:36 2022] GET /compute/v2.1/servers/515fd3da-3965-4882-b9e8-100a7eea64d9 => generated 1486 bytes in 371 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:30:37.343114 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-820f8b84-1558-40af-a2a5-79d2863fc785 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerRescueTestJSON-server-1324307820", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "}]}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:30:37.405518 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.network.neutron [None req-820f8b84-1558-40af-a2a5-79d2863fc785 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] validate_networks() for [('623041c3-6f6a-4dae-9458-468726d1421e', None, None, None, None, None)] {{(pid=138681) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 16 21:30:37.898667 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-3b0c0db6-bd7f-4976-88fd-df46b3181d47 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:30:37.900551 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-3b0c0db6-bd7f-4976-88fd-df46b3181d47 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] [instance: 08a33911-6e47-45fd-b499-155f199aacf8] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:30:37.908086 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3b0c0db6-bd7f-4976-88fd-df46b3181d47 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:30:37.908872 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3b0c0db6-bd7f-4976-88fd-df46b3181d47 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:30:37.909801 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3b0c0db6-bd7f-4976-88fd-df46b3181d47 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:30:37.910434 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3b0c0db6-bd7f-4976-88fd-df46b3181d47 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:30:38.084463 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-3b0c0db6-bd7f-4976-88fd-df46b3181d47 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:30:38.095360 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-3b0c0db6-bd7f-4976-88fd-df46b3181d47 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:30:38.098170 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-3b0c0db6-bd7f-4976-88fd-df46b3181d47 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:30:38.100020 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-3b0c0db6-bd7f-4976-88fd-df46b3181d47 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] 10.222.0.16 "GET /compute/v2.1/servers/08a33911-6e47-45fd-b499-155f199aacf8" status: 200 len: 1808 microversion: 2.20 time: 0.205313 Jul 16 21:30:38.101299 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2238/4489] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:30:37 2022] GET /compute/v2.1/servers/08a33911-6e47-45fd-b499-155f199aacf8 => generated 1808 bytes in 207 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 16 21:30:38.631841 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-35859fa9-7d16-439e-924a-17a73b4153aa req-983e9be5-1ffe-4f2a-a1f4-1eddf55c5a1a service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "08a33911-6e47-45fd-b499-155f199aacf8", "name": "network-vif-plugged", "status": "completed", "tag": "81e317e7-e9db-4ba9-b87d-9395b084ead7"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:30:38.636712 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-820f8b84-1558-40af-a2a5-79d2863fc785 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Flavor limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 16 21:30:38.637196 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-820f8b84-1558-40af-a2a5-79d2863fc785 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Image limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 16 21:30:38.637810 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-820f8b84-1558-40af-a2a5-79d2863fc785 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Flavor pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 16 21:30:38.639313 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-820f8b84-1558-40af-a2a5-79d2863fc785 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Image pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 16 21:30:38.640004 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-820f8b84-1558-40af-a2a5-79d2863fc785 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 16 21:30:38.657582 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-35859fa9-7d16-439e-924a-17a73b4153aa req-983e9be5-1ffe-4f2a-a1f4-1eddf55c5a1a service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:30:38.657582 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-35859fa9-7d16-439e-924a-17a73b4153aa req-983e9be5-1ffe-4f2a-a1f4-1eddf55c5a1a service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:30:38.665793 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-820f8b84-1558-40af-a2a5-79d2863fc785 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:30:38.666524 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-820f8b84-1558-40af-a2a5-79d2863fc785 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:30:38.668717 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-820f8b84-1558-40af-a2a5-79d2863fc785 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:30:38.670398 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-820f8b84-1558-40af-a2a5-79d2863fc785 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:30:38.672411 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-35859fa9-7d16-439e-924a-17a73b4153aa req-983e9be5-1ffe-4f2a-a1f4-1eddf55c5a1a service nova] Creating event network-vif-plugged:81e317e7-e9db-4ba9-b87d-9395b084ead7 for instance 08a33911-6e47-45fd-b499-155f199aacf8 on nch2-master-1 Jul 16 21:30:38.700175 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-35859fa9-7d16-439e-924a-17a73b4153aa req-983e9be5-1ffe-4f2a-a1f4-1eddf55c5a1a service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.072048 Jul 16 21:30:38.701089 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2239/4490] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:30:38 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 74 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:30:38.786743 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-820f8b84-1558-40af-a2a5-79d2863fc785 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Getting quotas for project 207ec2d1883c4a1ba89696e6e77cf3aa. Resources: {'ram', 'instances', 'cores'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:30:38.800202 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-820f8b84-1558-40af-a2a5-79d2863fc785 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Getting quotas for user da77ec3d0f04496d8c8df4025bba76d1 and project 207ec2d1883c4a1ba89696e6e77cf3aa. Resources: {'ram', 'instances', 'cores'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:30:38.848035 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-820f8b84-1558-40af-a2a5-79d2863fc785 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Going to run 1 instances... {{(pid=138681) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 16 21:30:38.858937 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-820f8b84-1558-40af-a2a5-79d2863fc785 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] [instance: 4fa0664b-97f0-44ab-9823-a3a92c1bfdb8] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138681) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 16 21:30:39.054139 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-820f8b84-1558-40af-a2a5-79d2863fc785 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 383 microversion: 2.1 time: 1.716849 Jul 16 21:30:39.056672 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2252/4491] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 21:30:37 2022] POST /compute/v2.1/servers => generated 383 bytes in 1723 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 16 21:30:39.067814 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-c07fa200-a4d8-46db-9321-87228f41e3a3 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:30:39.069333 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-c07fa200-a4d8-46db-9321-87228f41e3a3 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] [instance: 4fa0664b-97f0-44ab-9823-a3a92c1bfdb8] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:30:39.120840 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-39fe8877-03e2-4101-bd94-55316591e5f7 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:30:39.122487 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-39fe8877-03e2-4101-bd94-55316591e5f7 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] [instance: 08a33911-6e47-45fd-b499-155f199aacf8] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:30:39.129821 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-39fe8877-03e2-4101-bd94-55316591e5f7 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:30:39.130576 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-39fe8877-03e2-4101-bd94-55316591e5f7 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:30:39.131749 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-39fe8877-03e2-4101-bd94-55316591e5f7 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:30:39.132988 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-39fe8877-03e2-4101-bd94-55316591e5f7 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:30:39.190011 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-c07fa200-a4d8-46db-9321-87228f41e3a3 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da77ec3d0f04496d8c8df4025bba76d1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '207ec2d1883c4a1ba89696e6e77cf3aa', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:30:39.204564 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-c07fa200-a4d8-46db-9321-87228f41e3a3 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] 10.222.0.16 "GET /compute/v2.1/servers/4fa0664b-97f0-44ab-9823-a3a92c1bfdb8" status: 200 len: 1214 microversion: 2.1 time: 0.138723 Jul 16 21:30:39.206442 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2240/4492] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:30:39 2022] GET /compute/v2.1/servers/4fa0664b-97f0-44ab-9823-a3a92c1bfdb8 => generated 1214 bytes in 144 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:30:40.045740 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-39fe8877-03e2-4101-bd94-55316591e5f7 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:30:40.061662 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-39fe8877-03e2-4101-bd94-55316591e5f7 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:30:40.063795 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-39fe8877-03e2-4101-bd94-55316591e5f7 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:30:40.065012 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-39fe8877-03e2-4101-bd94-55316591e5f7 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] 10.222.0.16 "GET /compute/v2.1/servers/08a33911-6e47-45fd-b499-155f199aacf8" status: 200 len: 1808 microversion: 2.20 time: 0.950277 Jul 16 21:30:40.082571 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2253/4493] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:30:39 2022] GET /compute/v2.1/servers/08a33911-6e47-45fd-b499-155f199aacf8 => generated 1808 bytes in 968 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:30:40.230854 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-c390a352-6a88-4fec-acef-752e830b0b9d tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:30:40.231566 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-c390a352-6a88-4fec-acef-752e830b0b9d tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] [instance: 4fa0664b-97f0-44ab-9823-a3a92c1bfdb8] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:30:40.385494 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-c390a352-6a88-4fec-acef-752e830b0b9d tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da77ec3d0f04496d8c8df4025bba76d1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '207ec2d1883c4a1ba89696e6e77cf3aa', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:30:40.394169 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-c390a352-6a88-4fec-acef-752e830b0b9d tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] 10.222.0.16 "GET /compute/v2.1/servers/4fa0664b-97f0-44ab-9823-a3a92c1bfdb8" status: 200 len: 1214 microversion: 2.1 time: 0.169132 Jul 16 21:30:40.395427 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2241/4494] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:30:40 2022] GET /compute/v2.1/servers/4fa0664b-97f0-44ab-9823-a3a92c1bfdb8 => generated 1214 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:30:41.091726 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-0b875970-fa7f-4309-ade8-51f6b9ecc2ad tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:30:41.091726 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-0b875970-fa7f-4309-ade8-51f6b9ecc2ad tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] [instance: 08a33911-6e47-45fd-b499-155f199aacf8] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:30:41.107245 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0b875970-fa7f-4309-ade8-51f6b9ecc2ad tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:30:41.108011 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0b875970-fa7f-4309-ade8-51f6b9ecc2ad tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:30:41.108936 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0b875970-fa7f-4309-ade8-51f6b9ecc2ad tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:30:41.112296 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0b875970-fa7f-4309-ade8-51f6b9ecc2ad tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:30:41.412568 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-2d7e393f-da58-4e5c-b37a-f1630407e937 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:30:41.413193 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-2d7e393f-da58-4e5c-b37a-f1630407e937 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] [instance: 4fa0664b-97f0-44ab-9823-a3a92c1bfdb8] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:30:41.424863 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2d7e393f-da58-4e5c-b37a-f1630407e937 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:30:41.425544 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2d7e393f-da58-4e5c-b37a-f1630407e937 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:30:41.427025 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2d7e393f-da58-4e5c-b37a-f1630407e937 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:30:41.427639 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2d7e393f-da58-4e5c-b37a-f1630407e937 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:30:41.462062 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-0b875970-fa7f-4309-ade8-51f6b9ecc2ad tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:30:41.473052 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-0b875970-fa7f-4309-ade8-51f6b9ecc2ad tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:30:41.485087 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-0b875970-fa7f-4309-ade8-51f6b9ecc2ad tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:30:41.490477 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-0b875970-fa7f-4309-ade8-51f6b9ecc2ad tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] 10.222.0.16 "GET /compute/v2.1/servers/08a33911-6e47-45fd-b499-155f199aacf8" status: 200 len: 1808 microversion: 2.20 time: 0.406227 Jul 16 21:30:41.491199 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2254/4495] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:30:41 2022] GET /compute/v2.1/servers/08a33911-6e47-45fd-b499-155f199aacf8 => generated 1808 bytes in 408 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:30:41.586241 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-2d7e393f-da58-4e5c-b37a-f1630407e937 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da77ec3d0f04496d8c8df4025bba76d1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '207ec2d1883c4a1ba89696e6e77cf3aa', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:30:41.598998 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-2d7e393f-da58-4e5c-b37a-f1630407e937 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] 10.222.0.16 "GET /compute/v2.1/servers/4fa0664b-97f0-44ab-9823-a3a92c1bfdb8" status: 200 len: 1210 microversion: 2.1 time: 0.190299 Jul 16 21:30:41.610390 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2242/4496] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:30:41 2022] GET /compute/v2.1/servers/4fa0664b-97f0-44ab-9823-a3a92c1bfdb8 => generated 1210 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:30:42.512744 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-1524c1d6-8c14-4e9b-835a-5bb1046025f7 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:30:42.518884 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-1524c1d6-8c14-4e9b-835a-5bb1046025f7 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] [instance: 08a33911-6e47-45fd-b499-155f199aacf8] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:30:42.533958 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1524c1d6-8c14-4e9b-835a-5bb1046025f7 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:30:42.534493 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1524c1d6-8c14-4e9b-835a-5bb1046025f7 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:30:42.535370 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1524c1d6-8c14-4e9b-835a-5bb1046025f7 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:30:42.535816 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1524c1d6-8c14-4e9b-835a-5bb1046025f7 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:30:42.621669 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-3123d8f6-8c74-4305-911e-1d1893470f42 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:30:42.628270 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-3123d8f6-8c74-4305-911e-1d1893470f42 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] [instance: 4fa0664b-97f0-44ab-9823-a3a92c1bfdb8] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:30:42.657815 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3123d8f6-8c74-4305-911e-1d1893470f42 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:30:42.663979 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3123d8f6-8c74-4305-911e-1d1893470f42 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:30:42.677812 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3123d8f6-8c74-4305-911e-1d1893470f42 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:30:42.677812 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3123d8f6-8c74-4305-911e-1d1893470f42 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:30:42.796331 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-3123d8f6-8c74-4305-911e-1d1893470f42 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da77ec3d0f04496d8c8df4025bba76d1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '207ec2d1883c4a1ba89696e6e77cf3aa', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:30:42.807252 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-3123d8f6-8c74-4305-911e-1d1893470f42 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] 10.222.0.16 "GET /compute/v2.1/servers/4fa0664b-97f0-44ab-9823-a3a92c1bfdb8" status: 200 len: 1284 microversion: 2.1 time: 0.192803 Jul 16 21:30:42.808563 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2243/4497] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:30:42 2022] GET /compute/v2.1/servers/4fa0664b-97f0-44ab-9823-a3a92c1bfdb8 => generated 1284 bytes in 195 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:30:42.894178 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-1524c1d6-8c14-4e9b-835a-5bb1046025f7 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:30:42.905687 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-1524c1d6-8c14-4e9b-835a-5bb1046025f7 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:30:42.907942 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-1524c1d6-8c14-4e9b-835a-5bb1046025f7 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:30:42.909498 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-1524c1d6-8c14-4e9b-835a-5bb1046025f7 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] 10.222.0.16 "GET /compute/v2.1/servers/08a33911-6e47-45fd-b499-155f199aacf8" status: 200 len: 1808 microversion: 2.20 time: 0.402040 Jul 16 21:30:42.910827 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2255/4498] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:30:42 2022] GET /compute/v2.1/servers/08a33911-6e47-45fd-b499-155f199aacf8 => generated 1808 bytes in 404 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:30:43.828565 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-9ef140de-8dab-40e9-a06e-708c82f7bfdd tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:30:43.829184 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-9ef140de-8dab-40e9-a06e-708c82f7bfdd tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] [instance: 4fa0664b-97f0-44ab-9823-a3a92c1bfdb8] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:30:43.842856 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9ef140de-8dab-40e9-a06e-708c82f7bfdd tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:30:43.843347 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9ef140de-8dab-40e9-a06e-708c82f7bfdd tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:30:43.844584 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9ef140de-8dab-40e9-a06e-708c82f7bfdd tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:30:43.845190 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9ef140de-8dab-40e9-a06e-708c82f7bfdd tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:30:43.932283 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-b833fd27-a97c-4aaf-918a-ae204a145b81 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:30:43.933876 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-b833fd27-a97c-4aaf-918a-ae204a145b81 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] [instance: 08a33911-6e47-45fd-b499-155f199aacf8] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:30:43.942330 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b833fd27-a97c-4aaf-918a-ae204a145b81 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:30:43.944221 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b833fd27-a97c-4aaf-918a-ae204a145b81 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:30:43.947827 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b833fd27-a97c-4aaf-918a-ae204a145b81 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:30:43.948866 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b833fd27-a97c-4aaf-918a-ae204a145b81 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:30:44.280255 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-9ef140de-8dab-40e9-a06e-708c82f7bfdd tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da77ec3d0f04496d8c8df4025bba76d1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '207ec2d1883c4a1ba89696e6e77cf3aa', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:30:44.290425 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-9ef140de-8dab-40e9-a06e-708c82f7bfdd tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] 10.222.0.16 "GET /compute/v2.1/servers/4fa0664b-97f0-44ab-9823-a3a92c1bfdb8" status: 200 len: 1314 microversion: 2.1 time: 0.464686 Jul 16 21:30:44.291267 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2244/4499] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:30:43 2022] GET /compute/v2.1/servers/4fa0664b-97f0-44ab-9823-a3a92c1bfdb8 => generated 1314 bytes in 467 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:30:44.294493 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-b833fd27-a97c-4aaf-918a-ae204a145b81 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:30:44.303924 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-b833fd27-a97c-4aaf-918a-ae204a145b81 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:30:44.305295 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-b833fd27-a97c-4aaf-918a-ae204a145b81 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:30:44.306783 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-b833fd27-a97c-4aaf-918a-ae204a145b81 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] 10.222.0.16 "GET /compute/v2.1/servers/08a33911-6e47-45fd-b499-155f199aacf8" status: 200 len: 1808 microversion: 2.20 time: 0.381190 Jul 16 21:30:44.308916 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2256/4500] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:30:43 2022] GET /compute/v2.1/servers/08a33911-6e47-45fd-b499-155f199aacf8 => generated 1808 bytes in 384 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:30:44.693885 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-c856c581-8c31-4bd3-b8b2-8d7a457c3d15 req-41523d14-970d-47cf-ad19-38c1fd1843f2 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "4fa0664b-97f0-44ab-9823-a3a92c1bfdb8", "tag": "cb0aa507-e5a6-4b6b-975c-b074af9c5bd6"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:30:44.711232 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-c856c581-8c31-4bd3-b8b2-8d7a457c3d15 req-41523d14-970d-47cf-ad19-38c1fd1843f2 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:30:44.711232 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-c856c581-8c31-4bd3-b8b2-8d7a457c3d15 req-41523d14-970d-47cf-ad19-38c1fd1843f2 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:30:44.732027 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-c856c581-8c31-4bd3-b8b2-8d7a457c3d15 req-41523d14-970d-47cf-ad19-38c1fd1843f2 service nova] Creating event network-changed:cb0aa507-e5a6-4b6b-975c-b074af9c5bd6 for instance 4fa0664b-97f0-44ab-9823-a3a92c1bfdb8 on nch1-master-1 Jul 16 21:30:44.736989 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-c856c581-8c31-4bd3-b8b2-8d7a457c3d15 req-41523d14-970d-47cf-ad19-38c1fd1843f2 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.047768 Jul 16 21:30:44.738114 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2245/4501] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:30:44 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:30:45.307505 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-5b127b05-c908-4191-a476-2b2c9ce61f76 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:30:45.308135 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-5b127b05-c908-4191-a476-2b2c9ce61f76 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] [instance: 4fa0664b-97f0-44ab-9823-a3a92c1bfdb8] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:30:45.320973 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5b127b05-c908-4191-a476-2b2c9ce61f76 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:30:45.321720 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5b127b05-c908-4191-a476-2b2c9ce61f76 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:30:45.322515 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5b127b05-c908-4191-a476-2b2c9ce61f76 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:30:45.322960 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5b127b05-c908-4191-a476-2b2c9ce61f76 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:30:45.333137 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-63f596bc-8174-408c-8224-57d2fa73d2bf tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:30:45.333916 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-63f596bc-8174-408c-8224-57d2fa73d2bf tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] [instance: 08a33911-6e47-45fd-b499-155f199aacf8] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:30:45.346934 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-63f596bc-8174-408c-8224-57d2fa73d2bf tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:30:45.349923 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-63f596bc-8174-408c-8224-57d2fa73d2bf tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:30:45.350898 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-63f596bc-8174-408c-8224-57d2fa73d2bf tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:30:45.351540 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-63f596bc-8174-408c-8224-57d2fa73d2bf tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:30:45.687504 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-5b127b05-c908-4191-a476-2b2c9ce61f76 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da77ec3d0f04496d8c8df4025bba76d1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '207ec2d1883c4a1ba89696e6e77cf3aa', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:30:45.708531 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-5b127b05-c908-4191-a476-2b2c9ce61f76 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] 10.222.0.16 "GET /compute/v2.1/servers/4fa0664b-97f0-44ab-9823-a3a92c1bfdb8" status: 200 len: 1314 microversion: 2.1 time: 0.404466 Jul 16 21:30:45.719529 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2257/4502] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:30:45 2022] GET /compute/v2.1/servers/4fa0664b-97f0-44ab-9823-a3a92c1bfdb8 => generated 1314 bytes in 407 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:30:45.789866 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-63f596bc-8174-408c-8224-57d2fa73d2bf tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:30:45.799655 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-63f596bc-8174-408c-8224-57d2fa73d2bf tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:30:45.801348 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-63f596bc-8174-408c-8224-57d2fa73d2bf tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:30:45.802651 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-63f596bc-8174-408c-8224-57d2fa73d2bf tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] 10.222.0.16 "GET /compute/v2.1/servers/08a33911-6e47-45fd-b499-155f199aacf8" status: 200 len: 1795 microversion: 2.20 time: 0.473726 Jul 16 21:30:45.803507 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2246/4503] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:30:45 2022] GET /compute/v2.1/servers/08a33911-6e47-45fd-b499-155f199aacf8 => generated 1795 bytes in 475 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 16 21:30:46.727573 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-3a5c416e-224b-4932-bac4-d5a8b5363ef3 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:30:46.728187 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-3a5c416e-224b-4932-bac4-d5a8b5363ef3 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] [instance: 4fa0664b-97f0-44ab-9823-a3a92c1bfdb8] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:30:46.735443 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3a5c416e-224b-4932-bac4-d5a8b5363ef3 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:30:46.735894 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3a5c416e-224b-4932-bac4-d5a8b5363ef3 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:30:46.736660 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3a5c416e-224b-4932-bac4-d5a8b5363ef3 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:30:46.737097 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3a5c416e-224b-4932-bac4-d5a8b5363ef3 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:30:46.878858 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-3a5c416e-224b-4932-bac4-d5a8b5363ef3 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da77ec3d0f04496d8c8df4025bba76d1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '207ec2d1883c4a1ba89696e6e77cf3aa', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:30:46.889376 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-3a5c416e-224b-4932-bac4-d5a8b5363ef3 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] 10.222.0.16 "GET /compute/v2.1/servers/4fa0664b-97f0-44ab-9823-a3a92c1bfdb8" status: 200 len: 1314 microversion: 2.1 time: 0.166007 Jul 16 21:30:46.890323 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2258/4504] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:30:46 2022] GET /compute/v2.1/servers/4fa0664b-97f0-44ab-9823-a3a92c1bfdb8 => generated 1314 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:30:47.906917 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-edd1ea5c-9e41-47b6-8466-28ffe6d6354e tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:30:47.907556 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-edd1ea5c-9e41-47b6-8466-28ffe6d6354e tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] [instance: 4fa0664b-97f0-44ab-9823-a3a92c1bfdb8] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:30:47.914585 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-edd1ea5c-9e41-47b6-8466-28ffe6d6354e tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:30:47.915063 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-edd1ea5c-9e41-47b6-8466-28ffe6d6354e tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:30:47.916193 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-edd1ea5c-9e41-47b6-8466-28ffe6d6354e tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:30:47.920661 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-edd1ea5c-9e41-47b6-8466-28ffe6d6354e tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:30:48.070139 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-edd1ea5c-9e41-47b6-8466-28ffe6d6354e tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da77ec3d0f04496d8c8df4025bba76d1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '207ec2d1883c4a1ba89696e6e77cf3aa', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:30:48.080135 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-edd1ea5c-9e41-47b6-8466-28ffe6d6354e tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] 10.222.0.16 "GET /compute/v2.1/servers/4fa0664b-97f0-44ab-9823-a3a92c1bfdb8" status: 200 len: 1314 microversion: 2.1 time: 0.177191 Jul 16 21:30:48.081316 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2247/4505] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:30:47 2022] GET /compute/v2.1/servers/4fa0664b-97f0-44ab-9823-a3a92c1bfdb8 => generated 1314 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:30:49.097381 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-5d7d3c80-d905-42b2-bdad-9d919d3729c1 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:30:49.098006 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-5d7d3c80-d905-42b2-bdad-9d919d3729c1 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] [instance: 4fa0664b-97f0-44ab-9823-a3a92c1bfdb8] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:30:49.104284 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell0:SELECT=2 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:30:49.105477 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell0:SELECT=6 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:30:49.107397 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5d7d3c80-d905-42b2-bdad-9d919d3729c1 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:30:49.108056 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5d7d3c80-d905-42b2-bdad-9d919d3729c1 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:30:49.109153 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5d7d3c80-d905-42b2-bdad-9d919d3729c1 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:30:49.110088 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5d7d3c80-d905-42b2-bdad-9d919d3729c1 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:30:49.278782 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-5d7d3c80-d905-42b2-bdad-9d919d3729c1 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da77ec3d0f04496d8c8df4025bba76d1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '207ec2d1883c4a1ba89696e6e77cf3aa', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:30:49.287192 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-5d7d3c80-d905-42b2-bdad-9d919d3729c1 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] 10.222.0.16 "GET /compute/v2.1/servers/4fa0664b-97f0-44ab-9823-a3a92c1bfdb8" status: 200 len: 1314 microversion: 2.1 time: 0.193785 Jul 16 21:30:49.288032 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2259/4506] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:30:49 2022] GET /compute/v2.1/servers/4fa0664b-97f0-44ab-9823-a3a92c1bfdb8 => generated 1314 bytes in 195 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:30:50.306857 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-f8026b5b-bbf7-4e51-880c-a96fee6fb673 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:30:50.308639 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-f8026b5b-bbf7-4e51-880c-a96fee6fb673 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] [instance: 4fa0664b-97f0-44ab-9823-a3a92c1bfdb8] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:30:50.319335 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f8026b5b-bbf7-4e51-880c-a96fee6fb673 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:30:50.320102 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f8026b5b-bbf7-4e51-880c-a96fee6fb673 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:30:50.321355 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f8026b5b-bbf7-4e51-880c-a96fee6fb673 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:30:50.322301 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f8026b5b-bbf7-4e51-880c-a96fee6fb673 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:30:50.390867 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell0:SELECT=4 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:30:50.516112 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-6a7f1395-159d-4df7-9750-2789e6594abb req-20094758-38b1-480f-baba-49c9be52d5c2 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "4fa0664b-97f0-44ab-9823-a3a92c1bfdb8", "name": "network-vif-plugged", "status": "completed", "tag": "cb0aa507-e5a6-4b6b-975c-b074af9c5bd6"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:30:50.535850 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-f8026b5b-bbf7-4e51-880c-a96fee6fb673 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da77ec3d0f04496d8c8df4025bba76d1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '207ec2d1883c4a1ba89696e6e77cf3aa', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:30:50.552848 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-f8026b5b-bbf7-4e51-880c-a96fee6fb673 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] 10.222.0.16 "GET /compute/v2.1/servers/4fa0664b-97f0-44ab-9823-a3a92c1bfdb8" status: 200 len: 1314 microversion: 2.1 time: 0.251943 Jul 16 21:30:50.555865 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2248/4507] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:30:50 2022] GET /compute/v2.1/servers/4fa0664b-97f0-44ab-9823-a3a92c1bfdb8 => generated 1314 bytes in 256 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:30:50.557001 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-6a7f1395-159d-4df7-9750-2789e6594abb req-20094758-38b1-480f-baba-49c9be52d5c2 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:30:50.557716 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-6a7f1395-159d-4df7-9750-2789e6594abb req-20094758-38b1-480f-baba-49c9be52d5c2 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:30:50.573996 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-6a7f1395-159d-4df7-9750-2789e6594abb req-20094758-38b1-480f-baba-49c9be52d5c2 service nova] Creating event network-vif-plugged:cb0aa507-e5a6-4b6b-975c-b074af9c5bd6 for instance 4fa0664b-97f0-44ab-9823-a3a92c1bfdb8 on nch1-master-1 Jul 16 21:30:50.583188 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-6a7f1395-159d-4df7-9750-2789e6594abb req-20094758-38b1-480f-baba-49c9be52d5c2 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.070029 Jul 16 21:30:50.583746 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2260/4508] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:30:50 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 73 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:30:51.573045 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-0a10a1cf-4ea8-4485-b91c-4accaf33add6 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:30:51.574335 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-0a10a1cf-4ea8-4485-b91c-4accaf33add6 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] [instance: 4fa0664b-97f0-44ab-9823-a3a92c1bfdb8] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:30:51.590266 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0a10a1cf-4ea8-4485-b91c-4accaf33add6 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:30:51.594372 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0a10a1cf-4ea8-4485-b91c-4accaf33add6 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:30:51.595292 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0a10a1cf-4ea8-4485-b91c-4accaf33add6 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:30:51.595648 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0a10a1cf-4ea8-4485-b91c-4accaf33add6 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:30:51.882349 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-0a10a1cf-4ea8-4485-b91c-4accaf33add6 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da77ec3d0f04496d8c8df4025bba76d1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '207ec2d1883c4a1ba89696e6e77cf3aa', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:30:51.891642 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-0a10a1cf-4ea8-4485-b91c-4accaf33add6 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] 10.222.0.16 "GET /compute/v2.1/servers/4fa0664b-97f0-44ab-9823-a3a92c1bfdb8" status: 200 len: 1314 microversion: 2.1 time: 0.323518 Jul 16 21:30:51.892241 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2249/4509] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:30:51 2022] GET /compute/v2.1/servers/4fa0664b-97f0-44ab-9823-a3a92c1bfdb8 => generated 1314 bytes in 325 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:30:52.910794 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-d4c67433-4abb-452c-b8e7-7fa49a3aed2f tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:30:52.913338 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-d4c67433-4abb-452c-b8e7-7fa49a3aed2f tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] [instance: 4fa0664b-97f0-44ab-9823-a3a92c1bfdb8] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:30:52.918559 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d4c67433-4abb-452c-b8e7-7fa49a3aed2f tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:30:52.920338 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d4c67433-4abb-452c-b8e7-7fa49a3aed2f tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:30:52.921158 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d4c67433-4abb-452c-b8e7-7fa49a3aed2f tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:30:52.922046 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d4c67433-4abb-452c-b8e7-7fa49a3aed2f tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:30:53.170131 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-d4c67433-4abb-452c-b8e7-7fa49a3aed2f tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da77ec3d0f04496d8c8df4025bba76d1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '207ec2d1883c4a1ba89696e6e77cf3aa', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:30:53.182545 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-d4c67433-4abb-452c-b8e7-7fa49a3aed2f tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] 10.222.0.16 "GET /compute/v2.1/servers/4fa0664b-97f0-44ab-9823-a3a92c1bfdb8" status: 200 len: 1500 microversion: 2.1 time: 0.277455 Jul 16 21:30:53.183768 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2261/4510] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:30:52 2022] GET /compute/v2.1/servers/4fa0664b-97f0-44ab-9823-a3a92c1bfdb8 => generated 1500 bytes in 279 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:30:53.207159 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-97a783bc-47a6-4303-b0d7-c43bd2c33357 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Action: 'action', calling method: >, body: {"rescue": {"adminPass": "***"}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:30:53.208433 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-97a783bc-47a6-4303-b0d7-c43bd2c33357 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] [instance: 4fa0664b-97f0-44ab-9823-a3a92c1bfdb8] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:30:53.224868 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-97a783bc-47a6-4303-b0d7-c43bd2c33357 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:30:53.225326 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-97a783bc-47a6-4303-b0d7-c43bd2c33357 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:30:53.227475 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-97a783bc-47a6-4303-b0d7-c43bd2c33357 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:30:53.228088 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-97a783bc-47a6-4303-b0d7-c43bd2c33357 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:30:53.262830 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.objects.instance [None req-97a783bc-47a6-4303-b0d7-c43bd2c33357 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lazy-loading 'flavor' on Instance uuid 4fa0664b-97f0-44ab-9823-a3a92c1bfdb8 {{(pid=138680) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:30:53.462866 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=99,nova_cell1:UPDATE=1 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:30:53.640787 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-97a783bc-47a6-4303-b0d7-c43bd2c33357 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] 10.222.0.16 "POST /compute/v2.1/servers/4fa0664b-97f0-44ab-9823-a3a92c1bfdb8/action" status: 200 len: 32 microversion: 2.1 time: 0.438471 Jul 16 21:30:53.645358 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2250/4511] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 21:30:53 2022] POST /compute/v2.1/servers/4fa0664b-97f0-44ab-9823-a3a92c1bfdb8/action => generated 32 bytes in 441 msecs (HTTP/1.1 200) 9 headers in 356 bytes (2 switches on core 0) Jul 16 21:30:53.652679 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-841c778d-4a8d-454c-ab66-c617f94fd431 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:30:53.653463 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-841c778d-4a8d-454c-ab66-c617f94fd431 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] [instance: 4fa0664b-97f0-44ab-9823-a3a92c1bfdb8] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:30:53.668116 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-841c778d-4a8d-454c-ab66-c617f94fd431 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:30:53.668570 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-841c778d-4a8d-454c-ab66-c617f94fd431 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:30:53.669341 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-841c778d-4a8d-454c-ab66-c617f94fd431 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:30:53.677063 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-841c778d-4a8d-454c-ab66-c617f94fd431 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:30:54.443059 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-841c778d-4a8d-454c-ab66-c617f94fd431 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da77ec3d0f04496d8c8df4025bba76d1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '207ec2d1883c4a1ba89696e6e77cf3aa', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:30:54.454550 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-841c778d-4a8d-454c-ab66-c617f94fd431 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] 10.222.0.16 "GET /compute/v2.1/servers/4fa0664b-97f0-44ab-9823-a3a92c1bfdb8" status: 200 len: 1506 microversion: 2.1 time: 0.807524 Jul 16 21:30:54.468186 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2262/4512] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:30:53 2022] GET /compute/v2.1/servers/4fa0664b-97f0-44ab-9823-a3a92c1bfdb8 => generated 1506 bytes in 812 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:30:55.478106 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-1a2174a5-7c3f-4ae9-b26d-ee598afb85c9 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:30:55.478976 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-1a2174a5-7c3f-4ae9-b26d-ee598afb85c9 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] [instance: 4fa0664b-97f0-44ab-9823-a3a92c1bfdb8] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:30:55.492612 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1a2174a5-7c3f-4ae9-b26d-ee598afb85c9 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:30:55.492612 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1a2174a5-7c3f-4ae9-b26d-ee598afb85c9 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:30:55.492960 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1a2174a5-7c3f-4ae9-b26d-ee598afb85c9 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:30:55.493840 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1a2174a5-7c3f-4ae9-b26d-ee598afb85c9 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:30:55.738234 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-1a2174a5-7c3f-4ae9-b26d-ee598afb85c9 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da77ec3d0f04496d8c8df4025bba76d1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '207ec2d1883c4a1ba89696e6e77cf3aa', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:30:55.747083 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-1a2174a5-7c3f-4ae9-b26d-ee598afb85c9 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] 10.222.0.16 "GET /compute/v2.1/servers/4fa0664b-97f0-44ab-9823-a3a92c1bfdb8" status: 200 len: 1506 microversion: 2.1 time: 0.275712 Jul 16 21:30:55.747847 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2251/4513] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:30:55 2022] GET /compute/v2.1/servers/4fa0664b-97f0-44ab-9823-a3a92c1bfdb8 => generated 1506 bytes in 277 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:30:56.763938 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-c9207833-2f4f-4d00-8753-68c81912c39d tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:30:56.764557 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-c9207833-2f4f-4d00-8753-68c81912c39d tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] [instance: 4fa0664b-97f0-44ab-9823-a3a92c1bfdb8] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:30:56.771731 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c9207833-2f4f-4d00-8753-68c81912c39d tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:30:56.772219 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c9207833-2f4f-4d00-8753-68c81912c39d tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:30:56.773550 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c9207833-2f4f-4d00-8753-68c81912c39d tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:30:56.774155 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c9207833-2f4f-4d00-8753-68c81912c39d tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:30:57.095552 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-c9207833-2f4f-4d00-8753-68c81912c39d tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da77ec3d0f04496d8c8df4025bba76d1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '207ec2d1883c4a1ba89696e6e77cf3aa', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:30:57.107915 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-c9207833-2f4f-4d00-8753-68c81912c39d tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] 10.222.0.16 "GET /compute/v2.1/servers/4fa0664b-97f0-44ab-9823-a3a92c1bfdb8" status: 200 len: 1506 microversion: 2.1 time: 0.347693 Jul 16 21:30:57.108883 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2263/4514] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:30:56 2022] GET /compute/v2.1/servers/4fa0664b-97f0-44ab-9823-a3a92c1bfdb8 => generated 1506 bytes in 349 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:30:58.126922 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-03f9a9d8-15ec-4bcf-a9ef-1e5afbedc3bc tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:30:58.127799 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-03f9a9d8-15ec-4bcf-a9ef-1e5afbedc3bc tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] [instance: 4fa0664b-97f0-44ab-9823-a3a92c1bfdb8] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:30:58.130563 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_api:SELECT=60 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:30:58.140020 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-03f9a9d8-15ec-4bcf-a9ef-1e5afbedc3bc tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:30:58.149597 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-03f9a9d8-15ec-4bcf-a9ef-1e5afbedc3bc tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:30:58.149597 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-03f9a9d8-15ec-4bcf-a9ef-1e5afbedc3bc tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:30:58.149597 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-03f9a9d8-15ec-4bcf-a9ef-1e5afbedc3bc tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:30:58.318070 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-03f9a9d8-15ec-4bcf-a9ef-1e5afbedc3bc tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da77ec3d0f04496d8c8df4025bba76d1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '207ec2d1883c4a1ba89696e6e77cf3aa', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:30:58.330084 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-03f9a9d8-15ec-4bcf-a9ef-1e5afbedc3bc tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] 10.222.0.16 "GET /compute/v2.1/servers/4fa0664b-97f0-44ab-9823-a3a92c1bfdb8" status: 200 len: 1506 microversion: 2.1 time: 0.208956 Jul 16 21:30:58.330983 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2252/4515] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:30:58 2022] GET /compute/v2.1/servers/4fa0664b-97f0-44ab-9823-a3a92c1bfdb8 => generated 1506 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:30:59.354258 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-70046237-682f-4446-a499-ab5e2938bfcf tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:30:59.355721 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-70046237-682f-4446-a499-ab5e2938bfcf tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] [instance: 4fa0664b-97f0-44ab-9823-a3a92c1bfdb8] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:30:59.368587 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-70046237-682f-4446-a499-ab5e2938bfcf tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:30:59.369329 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-70046237-682f-4446-a499-ab5e2938bfcf tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:30:59.370276 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-70046237-682f-4446-a499-ab5e2938bfcf tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:30:59.370874 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-70046237-682f-4446-a499-ab5e2938bfcf tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:30:59.674387 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-70046237-682f-4446-a499-ab5e2938bfcf tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da77ec3d0f04496d8c8df4025bba76d1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '207ec2d1883c4a1ba89696e6e77cf3aa', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:30:59.691087 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-70046237-682f-4446-a499-ab5e2938bfcf tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] 10.222.0.16 "GET /compute/v2.1/servers/4fa0664b-97f0-44ab-9823-a3a92c1bfdb8" status: 200 len: 1506 microversion: 2.1 time: 0.344763 Jul 16 21:30:59.695627 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2264/4516] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:30:59 2022] GET /compute/v2.1/servers/4fa0664b-97f0-44ab-9823-a3a92c1bfdb8 => generated 1506 bytes in 347 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:31:00.709223 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-1f401cb4-3b5f-4114-ba34-6ca40d0b933a tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:31:00.710049 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-1f401cb4-3b5f-4114-ba34-6ca40d0b933a tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] [instance: 4fa0664b-97f0-44ab-9823-a3a92c1bfdb8] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:31:00.726923 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1f401cb4-3b5f-4114-ba34-6ca40d0b933a tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:00.727799 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1f401cb4-3b5f-4114-ba34-6ca40d0b933a tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:00.728851 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1f401cb4-3b5f-4114-ba34-6ca40d0b933a tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:00.729500 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1f401cb4-3b5f-4114-ba34-6ca40d0b933a tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:01.059256 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-1f401cb4-3b5f-4114-ba34-6ca40d0b933a tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da77ec3d0f04496d8c8df4025bba76d1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '207ec2d1883c4a1ba89696e6e77cf3aa', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:31:01.072509 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-1f401cb4-3b5f-4114-ba34-6ca40d0b933a tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] 10.222.0.16 "GET /compute/v2.1/servers/4fa0664b-97f0-44ab-9823-a3a92c1bfdb8" status: 200 len: 1506 microversion: 2.1 time: 0.367598 Jul 16 21:31:01.075198 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2253/4517] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:31:00 2022] GET /compute/v2.1/servers/4fa0664b-97f0-44ab-9823-a3a92c1bfdb8 => generated 1506 bytes in 371 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:31:01.472295 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-756c0bc7-4fcc-4a77-aa06-a339778d585c tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:31:01.472762 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-756c0bc7-4fcc-4a77-aa06-a339778d585c tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] [instance: d17892b8-b5bd-469e-9284-0abb3115daaa] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:31:01.481669 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-756c0bc7-4fcc-4a77-aa06-a339778d585c tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:01.482135 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-756c0bc7-4fcc-4a77-aa06-a339778d585c tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:01.482953 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-756c0bc7-4fcc-4a77-aa06-a339778d585c tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:01.483426 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-756c0bc7-4fcc-4a77-aa06-a339778d585c tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:01.528804 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-756c0bc7-4fcc-4a77-aa06-a339778d585c tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] [instance: d17892b8-b5bd-469e-9284-0abb3115daaa] Going to try to terminate instance {{(pid=138680) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 16 21:31:01.665734 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-7897ff52-6a5e-4f3a-bc2d-29cb9ea6d99e tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestServerAdvancedOps-server-939533245", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452", "networks": [{"uuid": "1d12c7b7-a342-4dbb-ad81-b26fee6f76dc"}]}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:31:01.766762 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.network.neutron [None req-7897ff52-6a5e-4f3a-bc2d-29cb9ea6d99e tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] validate_networks() for [('1d12c7b7-a342-4dbb-ad81-b26fee6f76dc', None, None, None, None, None)] {{(pid=138681) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 16 21:31:01.999980 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-756c0bc7-4fcc-4a77-aa06-a339778d585c tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] 10.222.0.16 "DELETE /compute/v2.1/servers/d17892b8-b5bd-469e-9284-0abb3115daaa" status: 204 len: 0 microversion: 2.1 time: 0.531503 Jul 16 21:31:02.000584 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2254/4518] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:31:01 2022] DELETE /compute/v2.1/servers/d17892b8-b5bd-469e-9284-0abb3115daaa => generated 0 bytes in 533 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Jul 16 21:31:02.010587 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-5c8289b9-8dc7-4722-8823-e4048ca7d55a tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:31:02.011290 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-5c8289b9-8dc7-4722-8823-e4048ca7d55a tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] [instance: d17892b8-b5bd-469e-9284-0abb3115daaa] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:31:02.021669 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5c8289b9-8dc7-4722-8823-e4048ca7d55a tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:02.022505 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5c8289b9-8dc7-4722-8823-e4048ca7d55a tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:02.024552 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5c8289b9-8dc7-4722-8823-e4048ca7d55a tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:02.025187 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5c8289b9-8dc7-4722-8823-e4048ca7d55a tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:02.467748 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-5c8289b9-8dc7-4722-8823-e4048ca7d55a tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ddfae3448704591b662de83b97a1876', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bed7c94624484c7cb8abeb3bfc8757b9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:31:02.470068 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-7897ff52-6a5e-4f3a-bc2d-29cb9ea6d99e tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Flavor limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 16 21:31:02.470497 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-7897ff52-6a5e-4f3a-bc2d-29cb9ea6d99e tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Image limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 16 21:31:02.471100 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-7897ff52-6a5e-4f3a-bc2d-29cb9ea6d99e tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Flavor pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 16 21:31:02.471704 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-7897ff52-6a5e-4f3a-bc2d-29cb9ea6d99e tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Image pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 16 21:31:02.472291 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-7897ff52-6a5e-4f3a-bc2d-29cb9ea6d99e tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 16 21:31:02.476925 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-5c8289b9-8dc7-4722-8823-e4048ca7d55a tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] 10.222.0.16 "GET /compute/v2.1/servers/d17892b8-b5bd-469e-9284-0abb3115daaa" status: 200 len: 1552 microversion: 2.1 time: 0.471265 Jul 16 21:31:02.478371 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2255/4519] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:31:02 2022] GET /compute/v2.1/servers/d17892b8-b5bd-469e-9284-0abb3115daaa => generated 1552 bytes in 474 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:31:02.483966 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-260bea5d-23b5-4b0e-953c-600d793c992f tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:31:02.484822 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-260bea5d-23b5-4b0e-953c-600d793c992f tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] [instance: 4fa0664b-97f0-44ab-9823-a3a92c1bfdb8] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:31:02.494786 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7897ff52-6a5e-4f3a-bc2d-29cb9ea6d99e tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:02.495284 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7897ff52-6a5e-4f3a-bc2d-29cb9ea6d99e tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:02.496154 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7897ff52-6a5e-4f3a-bc2d-29cb9ea6d99e tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:02.496601 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7897ff52-6a5e-4f3a-bc2d-29cb9ea6d99e tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:02.499081 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-260bea5d-23b5-4b0e-953c-600d793c992f tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:02.499678 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-260bea5d-23b5-4b0e-953c-600d793c992f tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:02.501203 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-260bea5d-23b5-4b0e-953c-600d793c992f tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:02.503743 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-260bea5d-23b5-4b0e-953c-600d793c992f tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:02.517431 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-7897ff52-6a5e-4f3a-bc2d-29cb9ea6d99e tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Getting quotas for project 1e8c65e58fac42c4bd7f49eaa3a14115. Resources: {'ram', 'instances', 'cores'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:31:02.524972 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-7897ff52-6a5e-4f3a-bc2d-29cb9ea6d99e tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Getting quotas for user 1006989e743247bdb29a51b3a634afd9 and project 1e8c65e58fac42c4bd7f49eaa3a14115. Resources: {'ram', 'instances', 'cores'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:31:02.544278 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-7897ff52-6a5e-4f3a-bc2d-29cb9ea6d99e tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Going to run 1 instances... {{(pid=138681) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 16 21:31:02.546718 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-7897ff52-6a5e-4f3a-bc2d-29cb9ea6d99e tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] [instance: 3aa4786f-20b7-4123-a231-d3d1c3b91b3b] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138681) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 16 21:31:02.704657 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-7897ff52-6a5e-4f3a-bc2d-29cb9ea6d99e tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 1.328957 Jul 16 21:31:02.714849 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2265/4520] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 21:31:01 2022] POST /compute/v2.1/servers => generated 380 bytes in 1340 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 16 21:31:02.736858 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-9d98e0fa-5554-48e2-b44c-29a743735889 tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:31:02.737711 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-9d98e0fa-5554-48e2-b44c-29a743735889 tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] [instance: 3aa4786f-20b7-4123-a231-d3d1c3b91b3b] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:31:02.812205 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-260bea5d-23b5-4b0e-953c-600d793c992f tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da77ec3d0f04496d8c8df4025bba76d1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '207ec2d1883c4a1ba89696e6e77cf3aa', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:31:02.832979 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-260bea5d-23b5-4b0e-953c-600d793c992f tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] 10.222.0.16 "GET /compute/v2.1/servers/4fa0664b-97f0-44ab-9823-a3a92c1bfdb8" status: 200 len: 1506 microversion: 2.1 time: 0.353269 Jul 16 21:31:02.846500 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2256/4521] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:31:02 2022] GET /compute/v2.1/servers/4fa0664b-97f0-44ab-9823-a3a92c1bfdb8 => generated 1506 bytes in 367 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:31:02.885036 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-9d98e0fa-5554-48e2-b44c-29a743735889 tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1006989e743247bdb29a51b3a634afd9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e8c65e58fac42c4bd7f49eaa3a14115', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:31:02.893287 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-9d98e0fa-5554-48e2-b44c-29a743735889 tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] 10.222.0.16 "GET /compute/v2.1/servers/3aa4786f-20b7-4123-a231-d3d1c3b91b3b" status: 200 len: 1214 microversion: 2.1 time: 0.166686 Jul 16 21:31:02.893880 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2266/4522] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:31:02 2022] GET /compute/v2.1/servers/3aa4786f-20b7-4123-a231-d3d1c3b91b3b => generated 1214 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:31:03.498261 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-9ef41572-595d-4ed1-8803-737eeeb2eca9 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:31:03.499095 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-9ef41572-595d-4ed1-8803-737eeeb2eca9 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] [instance: d17892b8-b5bd-469e-9284-0abb3115daaa] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:31:03.509909 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9ef41572-595d-4ed1-8803-737eeeb2eca9 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:03.510776 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9ef41572-595d-4ed1-8803-737eeeb2eca9 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:03.514778 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9ef41572-595d-4ed1-8803-737eeeb2eca9 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:03.516633 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9ef41572-595d-4ed1-8803-737eeeb2eca9 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:03.849834 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-9630226e-8f72-4356-9b4f-501eba7e06e6 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:31:03.851714 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-9630226e-8f72-4356-9b4f-501eba7e06e6 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] [instance: 4fa0664b-97f0-44ab-9823-a3a92c1bfdb8] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:31:03.871216 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9630226e-8f72-4356-9b4f-501eba7e06e6 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:03.871216 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9630226e-8f72-4356-9b4f-501eba7e06e6 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:03.879521 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9630226e-8f72-4356-9b4f-501eba7e06e6 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:03.880249 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9630226e-8f72-4356-9b4f-501eba7e06e6 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:03.914787 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=92 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:31:04.286450 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-9ef41572-595d-4ed1-8803-737eeeb2eca9 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ddfae3448704591b662de83b97a1876', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bed7c94624484c7cb8abeb3bfc8757b9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:31:04.315240 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-9630226e-8f72-4356-9b4f-501eba7e06e6 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da77ec3d0f04496d8c8df4025bba76d1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '207ec2d1883c4a1ba89696e6e77cf3aa', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:31:04.324918 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-9630226e-8f72-4356-9b4f-501eba7e06e6 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] 10.222.0.16 "GET /compute/v2.1/servers/4fa0664b-97f0-44ab-9823-a3a92c1bfdb8" status: 200 len: 1506 microversion: 2.1 time: 0.478546 Jul 16 21:31:04.335842 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2267/4523] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:31:03 2022] GET /compute/v2.1/servers/4fa0664b-97f0-44ab-9823-a3a92c1bfdb8 => generated 1506 bytes in 490 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:31:04.340298 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-9ef41572-595d-4ed1-8803-737eeeb2eca9 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] 10.222.0.16 "GET /compute/v2.1/servers/d17892b8-b5bd-469e-9284-0abb3115daaa" status: 200 len: 1552 microversion: 2.1 time: 0.846711 Jul 16 21:31:04.341574 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-93a99c6f-03b9-4da1-a8e9-5fa3263a1585 tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:31:04.342415 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-93a99c6f-03b9-4da1-a8e9-5fa3263a1585 tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] [instance: 3aa4786f-20b7-4123-a231-d3d1c3b91b3b] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:31:04.343306 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2257/4524] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:31:03 2022] GET /compute/v2.1/servers/d17892b8-b5bd-469e-9284-0abb3115daaa => generated 1552 bytes in 852 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:31:04.426255 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-93a99c6f-03b9-4da1-a8e9-5fa3263a1585 tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1006989e743247bdb29a51b3a634afd9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e8c65e58fac42c4bd7f49eaa3a14115', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:31:04.436749 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-93a99c6f-03b9-4da1-a8e9-5fa3263a1585 tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] 10.222.0.16 "GET /compute/v2.1/servers/3aa4786f-20b7-4123-a231-d3d1c3b91b3b" status: 200 len: 1214 microversion: 2.1 time: 0.099354 Jul 16 21:31:04.437934 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2268/4525] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:31:04 2022] GET /compute/v2.1/servers/3aa4786f-20b7-4123-a231-d3d1c3b91b3b => generated 1214 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:31:05.018026 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-30a434c2-97e9-4dbb-b079-21f178304622 req-f305d718-0f6c-43e0-86f0-7252c726961d service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "d17892b8-b5bd-469e-9284-0abb3115daaa", "name": "network-vif-deleted", "tag": "88f48838-7774-4261-8f33-ad28281a1d60"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:31:05.058334 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-30a434c2-97e9-4dbb-b079-21f178304622 req-f305d718-0f6c-43e0-86f0-7252c726961d service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:05.059213 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-30a434c2-97e9-4dbb-b079-21f178304622 req-f305d718-0f6c-43e0-86f0-7252c726961d service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:05.074876 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-30a434c2-97e9-4dbb-b079-21f178304622 req-f305d718-0f6c-43e0-86f0-7252c726961d service nova] Creating event network-vif-deleted:88f48838-7774-4261-8f33-ad28281a1d60 for instance d17892b8-b5bd-469e-9284-0abb3115daaa on nch2-master-1 Jul 16 21:31:05.081325 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-30a434c2-97e9-4dbb-b079-21f178304622 req-f305d718-0f6c-43e0-86f0-7252c726961d service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.067515 Jul 16 21:31:05.082501 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2258/4526] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:31:05 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 70 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:31:05.342436 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-2ec3e883-d188-4046-b6f4-dec9f19294f1 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:31:05.343107 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-2ec3e883-d188-4046-b6f4-dec9f19294f1 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] [instance: 4fa0664b-97f0-44ab-9823-a3a92c1bfdb8] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:31:05.352537 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_api:SELECT=66,nova_api:INSERT=6 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:31:05.355373 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2ec3e883-d188-4046-b6f4-dec9f19294f1 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:05.355999 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2ec3e883-d188-4046-b6f4-dec9f19294f1 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:05.362992 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-4f20ce21-2868-4c78-b614-c5c8176caf3c tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:31:05.370058 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2ec3e883-d188-4046-b6f4-dec9f19294f1 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:05.370058 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2ec3e883-d188-4046-b6f4-dec9f19294f1 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:05.370308 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-4f20ce21-2868-4c78-b614-c5c8176caf3c tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] [instance: d17892b8-b5bd-469e-9284-0abb3115daaa] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:31:05.380792 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4f20ce21-2868-4c78-b614-c5c8176caf3c tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:05.385249 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4f20ce21-2868-4c78-b614-c5c8176caf3c tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:05.393002 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4f20ce21-2868-4c78-b614-c5c8176caf3c tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:05.393556 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4f20ce21-2868-4c78-b614-c5c8176caf3c tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:05.492377 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4f20ce21-2868-4c78-b614-c5c8176caf3c tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:05.492840 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4f20ce21-2868-4c78-b614-c5c8176caf3c tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:05.502280 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.objects.instance [None req-4f20ce21-2868-4c78-b614-c5c8176caf3c tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lazy-loading 'fault' on Instance uuid d17892b8-b5bd-469e-9284-0abb3115daaa {{(pid=138680) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:31:05.587552 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-4f20ce21-2868-4c78-b614-c5c8176caf3c tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ddfae3448704591b662de83b97a1876', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bed7c94624484c7cb8abeb3bfc8757b9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:31:05.592775 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-4f20ce21-2868-4c78-b614-c5c8176caf3c tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] 10.222.0.16 "GET /compute/v2.1/servers/d17892b8-b5bd-469e-9284-0abb3115daaa" status: 200 len: 1338 microversion: 2.1 time: 0.234738 Jul 16 21:31:05.593913 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2259/4527] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:31:05 2022] GET /compute/v2.1/servers/d17892b8-b5bd-469e-9284-0abb3115daaa => generated 1338 bytes in 237 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:31:05.598690 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-fce506b5-ca95-4353-a362-b7465cc16bb6 tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:31:05.599294 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-fce506b5-ca95-4353-a362-b7465cc16bb6 tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] [instance: 3aa4786f-20b7-4123-a231-d3d1c3b91b3b] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:31:05.606609 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-fce506b5-ca95-4353-a362-b7465cc16bb6 tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:05.607100 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-fce506b5-ca95-4353-a362-b7465cc16bb6 tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:05.608055 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-fce506b5-ca95-4353-a362-b7465cc16bb6 tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:05.608652 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-fce506b5-ca95-4353-a362-b7465cc16bb6 tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:05.619401 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-2ec3e883-d188-4046-b6f4-dec9f19294f1 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da77ec3d0f04496d8c8df4025bba76d1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '207ec2d1883c4a1ba89696e6e77cf3aa', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:31:05.630682 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-2ec3e883-d188-4046-b6f4-dec9f19294f1 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] 10.222.0.16 "GET /compute/v2.1/servers/4fa0664b-97f0-44ab-9823-a3a92c1bfdb8" status: 200 len: 1506 microversion: 2.1 time: 0.291549 Jul 16 21:31:05.631312 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2269/4528] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:31:05 2022] GET /compute/v2.1/servers/4fa0664b-97f0-44ab-9823-a3a92c1bfdb8 => generated 1506 bytes in 293 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:31:05.698158 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-fce506b5-ca95-4353-a362-b7465cc16bb6 tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1006989e743247bdb29a51b3a634afd9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e8c65e58fac42c4bd7f49eaa3a14115', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:31:05.711807 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-fce506b5-ca95-4353-a362-b7465cc16bb6 tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] 10.222.0.16 "GET /compute/v2.1/servers/3aa4786f-20b7-4123-a231-d3d1c3b91b3b" status: 200 len: 1266 microversion: 2.1 time: 0.116570 Jul 16 21:31:05.712498 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2260/4529] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:31:05 2022] GET /compute/v2.1/servers/3aa4786f-20b7-4123-a231-d3d1c3b91b3b => generated 1266 bytes in 118 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:31:06.614022 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-410c481e-504a-48b3-8865-54fdefbf568a tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:31:06.614808 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-410c481e-504a-48b3-8865-54fdefbf568a tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] [instance: d17892b8-b5bd-469e-9284-0abb3115daaa] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:31:06.625266 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-410c481e-504a-48b3-8865-54fdefbf568a tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:06.640186 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-410c481e-504a-48b3-8865-54fdefbf568a tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:06.641137 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-410c481e-504a-48b3-8865-54fdefbf568a tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:06.641637 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-410c481e-504a-48b3-8865-54fdefbf568a tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:06.649212 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-93c9b647-dd22-43f7-97c1-3754a61068c8 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:31:06.650568 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-93c9b647-dd22-43f7-97c1-3754a61068c8 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] [instance: 4fa0664b-97f0-44ab-9823-a3a92c1bfdb8] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:31:06.659923 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-93c9b647-dd22-43f7-97c1-3754a61068c8 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:06.660668 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-93c9b647-dd22-43f7-97c1-3754a61068c8 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:06.661725 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-93c9b647-dd22-43f7-97c1-3754a61068c8 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:06.662328 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-93c9b647-dd22-43f7-97c1-3754a61068c8 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:06.682486 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-410c481e-504a-48b3-8865-54fdefbf568a tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] HTTP exception thrown: Instance d17892b8-b5bd-469e-9284-0abb3115daaa could not be found. Jul 16 21:31:06.684342 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-410c481e-504a-48b3-8865-54fdefbf568a tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Request method failure captured: Jul 16 21:31:06.684342 ncdv-master-1 devstack@n-api.service[138681]: request: GET /compute/v2.1/servers/d17892b8-b5bd-469e-9284-0abb3115daaa HTTP/1.1 Jul 16 21:31:06.684342 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:31:06.684342 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:31:06.684342 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:31:06.684342 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 21:31:06.684342 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:31:06.684342 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:31:06.684342 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:31:06.684342 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:31:06.684342 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0y2AoaTmW9fzKpp4uGm7wex0cqFnrFlzY31Q3uQkOFuadyloOF8tCXPJOxztzHL0OnbEMWGJwuxCROg93CNNRADEFI3axz2E-JaVn1YaPI_h8JL-g89OY5Lk8ug8D0BLGsXQOPqIYrZnwwRF_WHMaANcXwS7qf8jAbwL7w-UUVnlf9c Jul 16 21:31:06.684342 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:31:06.684342 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:31:06.684342 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:31:06.684342 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:31:06.684342 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:31:06.684342 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:31:06.684342 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: bed7c94624484c7cb8abeb3bfc8757b9 Jul 16 21:31:06.684342 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-TestNetworkBasicOps-590553405 Jul 16 21:31:06.684342 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:31:06.684342 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:31:06.684342 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/bed7c94624484c7cb8abeb3bfc8757b9"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/bed7c94624484c7cb8abeb3bfc8757b9"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_bed7c94624484c7cb8abeb3bfc8757b9"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/bed7c94624484c7cb8abeb3bfc8757b9"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:31:06.684342 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-TestNetworkBasicOps-590553405 Jul 16 21:31:06.684342 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: bed7c94624484c7cb8abeb3bfc8757b9 Jul 16 21:31:06.684342 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-TestNetworkBasicOps-590553405 Jul 16 21:31:06.684342 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-TestNetworkBasicOps-590553405-project Jul 16 21:31:06.684342 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:31:06.684342 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:31:06.684342 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: 4ddfae3448704591b662de83b97a1876 Jul 16 21:31:06.684342 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-TestNetworkBasicOps-590553405-project Jul 16 21:31:06.684342 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:31:06.684342 ncdv-master-1 devstack@n-api.service[138681]: exception: Instance d17892b8-b5bd-469e-9284-0abb3115daaa could not be found. Jul 16 21:31:06.684342 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': 'd17892b8-b5bd-469e-9284-0abb3115daaa'} Jul 16 21:31:06.684342 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:31:06.684342 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:31:06.684342 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:31:06.684342 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:31:06.684342 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:31:06.684342 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:31:06.684342 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:31:06.684342 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:31:06.684342 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:31:06.684342 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:31:06.684342 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance d17892b8-b5bd-469e-9284-0abb3115daaa could not be found. Jul 16 21:31:06.684342 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:31:06.684342 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:31:06.684342 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:31:06.684342 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:31:06.684342 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:31:06.684342 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:31:06.684342 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:31:06.684342 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:31:06.684342 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:31:06.684342 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:31:06.684342 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:31:06.684342 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:31:06.684342 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:31:06.684342 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:31:06.684342 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:31:06.684342 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:31:06.684342 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance d17892b8-b5bd-469e-9284-0abb3115daaa could not be found. Jul 16 21:31:06.684342 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:31:06.684342 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:31:06.684342 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:31:06.689229 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:31:06.689229 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:31:06.689229 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:31:06.689229 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:31:06.689229 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:31:06.689229 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance d17892b8-b5bd-469e-9284-0abb3115daaa could not be found. Jul 16 21:31:06.689229 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:31:06.689229 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-410c481e-504a-48b3-8865-54fdefbf568a tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Returning 404 to user: Instance d17892b8-b5bd-469e-9284-0abb3115daaa could not be found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:31:06.689229 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-410c481e-504a-48b3-8865-54fdefbf568a tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] 10.222.0.16 "GET /compute/v2.1/servers/d17892b8-b5bd-469e-9284-0abb3115daaa" status: 404 len: 111 microversion: 2.1 time: 0.077018 Jul 16 21:31:06.689229 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2270/4530] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:31:06 2022] GET /compute/v2.1/servers/d17892b8-b5bd-469e-9284-0abb3115daaa => generated 111 bytes in 78 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:31:06.730423 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-9ae00d26-fdcb-4a5e-8bf1-35d3856cd522 tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:31:06.742699 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-9ae00d26-fdcb-4a5e-8bf1-35d3856cd522 tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] [instance: 3aa4786f-20b7-4123-a231-d3d1c3b91b3b] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:31:06.742699 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9ae00d26-fdcb-4a5e-8bf1-35d3856cd522 tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:06.742699 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9ae00d26-fdcb-4a5e-8bf1-35d3856cd522 tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:06.742699 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9ae00d26-fdcb-4a5e-8bf1-35d3856cd522 tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:06.742699 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9ae00d26-fdcb-4a5e-8bf1-35d3856cd522 tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:06.873756 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-9ae00d26-fdcb-4a5e-8bf1-35d3856cd522 tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1006989e743247bdb29a51b3a634afd9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e8c65e58fac42c4bd7f49eaa3a14115', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:31:06.884402 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-9ae00d26-fdcb-4a5e-8bf1-35d3856cd522 tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] 10.222.0.16 "GET /compute/v2.1/servers/3aa4786f-20b7-4123-a231-d3d1c3b91b3b" status: 200 len: 1272 microversion: 2.1 time: 0.159468 Jul 16 21:31:06.885865 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2271/4531] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:31:06 2022] GET /compute/v2.1/servers/3aa4786f-20b7-4123-a231-d3d1c3b91b3b => generated 1272 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:31:07.079292 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-93c9b647-dd22-43f7-97c1-3754a61068c8 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da77ec3d0f04496d8c8df4025bba76d1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '207ec2d1883c4a1ba89696e6e77cf3aa', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:31:07.092584 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-93c9b647-dd22-43f7-97c1-3754a61068c8 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] 10.222.0.16 "GET /compute/v2.1/servers/4fa0664b-97f0-44ab-9823-a3a92c1bfdb8" status: 200 len: 1506 microversion: 2.1 time: 0.447273 Jul 16 21:31:07.105514 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2261/4532] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:31:06 2022] GET /compute/v2.1/servers/4fa0664b-97f0-44ab-9823-a3a92c1bfdb8 => generated 1506 bytes in 461 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:31:07.285412 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-f427aacd-7f0f-48a7-9d9f-3dee80f1016a tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Calling method '.version_select at 0x7fdd9c307280>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:31:07.288931 ncdv-master-1 devstack@n-api.service[138681]: ERROR oslo.messaging._drivers.impl_rabbit [None req-f427aacd-7f0f-48a7-9d9f-3dee80f1016a tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] [0cc6ec90-aeaa-4138-9056-1296518c79df] AMQP server on 10.222.0.16:5672 is unreachable: [Errno 104] Connection reset by peer. Trying again in 1 seconds.: ConnectionResetError: [Errno 104] Connection reset by peer Jul 16 21:31:07.902469 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-11ae092d-2874-4bf9-aeed-e1c64cf300de tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:31:07.903112 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-11ae092d-2874-4bf9-aeed-e1c64cf300de tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] [instance: 3aa4786f-20b7-4123-a231-d3d1c3b91b3b] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:31:07.914625 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-11ae092d-2874-4bf9-aeed-e1c64cf300de tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:07.915107 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-11ae092d-2874-4bf9-aeed-e1c64cf300de tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:07.916283 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-11ae092d-2874-4bf9-aeed-e1c64cf300de tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:07.916881 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-11ae092d-2874-4bf9-aeed-e1c64cf300de tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:08.173705 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-11ae092d-2874-4bf9-aeed-e1c64cf300de tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1006989e743247bdb29a51b3a634afd9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e8c65e58fac42c4bd7f49eaa3a14115', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:31:08.195235 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-11ae092d-2874-4bf9-aeed-e1c64cf300de tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] 10.222.0.16 "GET /compute/v2.1/servers/3aa4786f-20b7-4123-a231-d3d1c3b91b3b" status: 200 len: 1314 microversion: 2.1 time: 0.295756 Jul 16 21:31:08.201706 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2262/4533] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:31:07 2022] GET /compute/v2.1/servers/3aa4786f-20b7-4123-a231-d3d1c3b91b3b => generated 1314 bytes in 299 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:31:08.206932 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-1906b293-541b-4457-9051-887f30a0704c tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:31:08.215954 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-1906b293-541b-4457-9051-887f30a0704c tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] [instance: 4fa0664b-97f0-44ab-9823-a3a92c1bfdb8] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:31:08.237698 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1906b293-541b-4457-9051-887f30a0704c tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:08.240466 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1906b293-541b-4457-9051-887f30a0704c tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:08.242036 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1906b293-541b-4457-9051-887f30a0704c tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:08.242857 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1906b293-541b-4457-9051-887f30a0704c tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:08.300617 ncdv-master-1 devstack@n-api.service[138681]: INFO oslo.messaging._drivers.impl_rabbit [None req-f427aacd-7f0f-48a7-9d9f-3dee80f1016a tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] [0cc6ec90-aeaa-4138-9056-1296518c79df] Reconnected to AMQP server on 10.222.0.16:5672 via [amqp] client with port 58266. Jul 16 21:31:08.536896 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-f427aacd-7f0f-48a7-9d9f-3dee80f1016a tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] 10.222.0.16 "DELETE /compute/v2.1/os-keypairs/tempest-TestNetworkBasicOps-913257510" status: 202 len: 0 microversion: 2.1 time: 1.254968 Jul 16 21:31:08.539000 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2272/4534] 10.222.0.16 () {58 vars in 1253 bytes} [Sat Jul 16 21:31:07 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestNetworkBasicOps-913257510 => generated 0 bytes in 1258 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 21:31:08.581897 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-1906b293-541b-4457-9051-887f30a0704c tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da77ec3d0f04496d8c8df4025bba76d1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '207ec2d1883c4a1ba89696e6e77cf3aa', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:31:08.617422 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-1906b293-541b-4457-9051-887f30a0704c tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] 10.222.0.16 "GET /compute/v2.1/servers/4fa0664b-97f0-44ab-9823-a3a92c1bfdb8" status: 200 len: 1506 microversion: 2.1 time: 0.418106 Jul 16 21:31:08.619142 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2263/4535] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:31:08 2022] GET /compute/v2.1/servers/4fa0664b-97f0-44ab-9823-a3a92c1bfdb8 => generated 1506 bytes in 422 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:31:09.134912 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-2f871b3f-92e1-4afb-a1be-44f50dfb05a2 req-39cd901f-30e8-4ea9-b211-6733fad40e0a service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "3aa4786f-20b7-4123-a231-d3d1c3b91b3b", "tag": "6c497884-33a6-412b-8db5-37cebffc7eca"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:31:09.203639 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-2f871b3f-92e1-4afb-a1be-44f50dfb05a2 req-39cd901f-30e8-4ea9-b211-6733fad40e0a service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:09.204508 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-2f871b3f-92e1-4afb-a1be-44f50dfb05a2 req-39cd901f-30e8-4ea9-b211-6733fad40e0a service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:09.230785 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-c8676447-72a6-4b99-b06a-a0d4af3482cd tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:31:09.236113 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-c8676447-72a6-4b99-b06a-a0d4af3482cd tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] [instance: 3aa4786f-20b7-4123-a231-d3d1c3b91b3b] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:31:09.267499 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-2f871b3f-92e1-4afb-a1be-44f50dfb05a2 req-39cd901f-30e8-4ea9-b211-6733fad40e0a service nova] Creating event network-changed:6c497884-33a6-412b-8db5-37cebffc7eca for instance 3aa4786f-20b7-4123-a231-d3d1c3b91b3b on nch2-master-1 Jul 16 21:31:09.269136 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c8676447-72a6-4b99-b06a-a0d4af3482cd tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:09.269941 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c8676447-72a6-4b99-b06a-a0d4af3482cd tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:09.280329 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c8676447-72a6-4b99-b06a-a0d4af3482cd tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:09.281195 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c8676447-72a6-4b99-b06a-a0d4af3482cd tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:09.287628 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-2f871b3f-92e1-4afb-a1be-44f50dfb05a2 req-39cd901f-30e8-4ea9-b211-6733fad40e0a service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.161383 Jul 16 21:31:09.289964 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2273/4536] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:31:09 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:31:09.657032 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-37387ef6-9270-42e7-a2da-f625d7404c8d tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:31:09.661762 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-37387ef6-9270-42e7-a2da-f625d7404c8d tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] [instance: 4fa0664b-97f0-44ab-9823-a3a92c1bfdb8] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:31:09.712820 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-37387ef6-9270-42e7-a2da-f625d7404c8d tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:09.716943 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-37387ef6-9270-42e7-a2da-f625d7404c8d tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:09.726913 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-37387ef6-9270-42e7-a2da-f625d7404c8d tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:09.727964 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-37387ef6-9270-42e7-a2da-f625d7404c8d tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:10.381244 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-c8676447-72a6-4b99-b06a-a0d4af3482cd tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1006989e743247bdb29a51b3a634afd9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e8c65e58fac42c4bd7f49eaa3a14115', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:31:10.392014 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-c8676447-72a6-4b99-b06a-a0d4af3482cd tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] 10.222.0.16 "GET /compute/v2.1/servers/3aa4786f-20b7-4123-a231-d3d1c3b91b3b" status: 200 len: 1314 microversion: 2.1 time: 1.169910 Jul 16 21:31:10.392691 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2264/4537] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:31:09 2022] GET /compute/v2.1/servers/3aa4786f-20b7-4123-a231-d3d1c3b91b3b => generated 1314 bytes in 1172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:31:10.485757 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-37387ef6-9270-42e7-a2da-f625d7404c8d tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da77ec3d0f04496d8c8df4025bba76d1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '207ec2d1883c4a1ba89696e6e77cf3aa', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:31:10.494018 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-37387ef6-9270-42e7-a2da-f625d7404c8d tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] 10.222.0.16 "GET /compute/v2.1/servers/4fa0664b-97f0-44ab-9823-a3a92c1bfdb8" status: 200 len: 1506 microversion: 2.1 time: 0.844995 Jul 16 21:31:10.495831 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2274/4538] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:31:09 2022] GET /compute/v2.1/servers/4fa0664b-97f0-44ab-9823-a3a92c1bfdb8 => generated 1506 bytes in 848 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:31:11.414054 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-cad876f9-2c11-4c04-b84d-d64fd3b1e0cf tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:31:11.414730 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-cad876f9-2c11-4c04-b84d-d64fd3b1e0cf tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] [instance: 3aa4786f-20b7-4123-a231-d3d1c3b91b3b] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:31:11.421884 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cad876f9-2c11-4c04-b84d-d64fd3b1e0cf tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:11.422503 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cad876f9-2c11-4c04-b84d-d64fd3b1e0cf tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:11.423572 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cad876f9-2c11-4c04-b84d-d64fd3b1e0cf tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:11.424214 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cad876f9-2c11-4c04-b84d-d64fd3b1e0cf tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:11.441309 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=96,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:31:11.514535 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-695cf3d3-5273-47cb-8ac4-23929a71dc79 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:31:11.514535 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-695cf3d3-5273-47cb-8ac4-23929a71dc79 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] [instance: 4fa0664b-97f0-44ab-9823-a3a92c1bfdb8] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:31:11.528222 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-695cf3d3-5273-47cb-8ac4-23929a71dc79 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:11.528822 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-695cf3d3-5273-47cb-8ac4-23929a71dc79 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:11.529829 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-695cf3d3-5273-47cb-8ac4-23929a71dc79 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:11.532429 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-695cf3d3-5273-47cb-8ac4-23929a71dc79 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:11.752261 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-cad876f9-2c11-4c04-b84d-d64fd3b1e0cf tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1006989e743247bdb29a51b3a634afd9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e8c65e58fac42c4bd7f49eaa3a14115', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:31:11.762011 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-cad876f9-2c11-4c04-b84d-d64fd3b1e0cf tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] 10.222.0.16 "GET /compute/v2.1/servers/3aa4786f-20b7-4123-a231-d3d1c3b91b3b" status: 200 len: 1314 microversion: 2.1 time: 0.352890 Jul 16 21:31:11.766389 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2265/4539] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:31:11 2022] GET /compute/v2.1/servers/3aa4786f-20b7-4123-a231-d3d1c3b91b3b => generated 1314 bytes in 358 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:31:11.890144 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-695cf3d3-5273-47cb-8ac4-23929a71dc79 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da77ec3d0f04496d8c8df4025bba76d1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '207ec2d1883c4a1ba89696e6e77cf3aa', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:31:11.898665 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-695cf3d3-5273-47cb-8ac4-23929a71dc79 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] 10.222.0.16 "GET /compute/v2.1/servers/4fa0664b-97f0-44ab-9823-a3a92c1bfdb8" status: 200 len: 1506 microversion: 2.1 time: 0.388862 Jul 16 21:31:11.899866 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2275/4540] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:31:11 2022] GET /compute/v2.1/servers/4fa0664b-97f0-44ab-9823-a3a92c1bfdb8 => generated 1506 bytes in 390 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:31:12.612584 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-bad75b62-9d47-4d8e-b20b-2b1cff03c835 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:31:12.613174 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-bad75b62-9d47-4d8e-b20b-2b1cff03c835 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] [instance: 59a4c34e-3184-465b-b14c-0ad55192c8ce] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:31:12.626623 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-bad75b62-9d47-4d8e-b20b-2b1cff03c835 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:12.630074 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-bad75b62-9d47-4d8e-b20b-2b1cff03c835 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:12.632270 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-bad75b62-9d47-4d8e-b20b-2b1cff03c835 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:12.633078 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-bad75b62-9d47-4d8e-b20b-2b1cff03c835 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:12.642902 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell0:SELECT=2 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:31:12.701112 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-bad75b62-9d47-4d8e-b20b-2b1cff03c835 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] [instance: 59a4c34e-3184-465b-b14c-0ad55192c8ce] Going to try to terminate instance {{(pid=138680) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 16 21:31:12.886176 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-5f8cf470-4b13-4d23-bb49-48653d8410ae req-22fa4d15-ea35-4042-b746-54ba6884d63e service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "59a4c34e-3184-465b-b14c-0ad55192c8ce", "tag": "cad17319-3e42-44f8-911a-cd258470adec"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:31:12.903251 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-5f8cf470-4b13-4d23-bb49-48653d8410ae req-22fa4d15-ea35-4042-b746-54ba6884d63e service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:12.903706 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-5f8cf470-4b13-4d23-bb49-48653d8410ae req-22fa4d15-ea35-4042-b746-54ba6884d63e service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:12.925700 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-5f8cf470-4b13-4d23-bb49-48653d8410ae req-22fa4d15-ea35-4042-b746-54ba6884d63e service nova] Creating event network-changed:cad17319-3e42-44f8-911a-cd258470adec for instance 59a4c34e-3184-465b-b14c-0ad55192c8ce on nch2-master-1 Jul 16 21:31:12.931506 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-5f8cf470-4b13-4d23-bb49-48653d8410ae req-22fa4d15-ea35-4042-b746-54ba6884d63e service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.298841 Jul 16 21:31:12.932251 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2276/4541] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:31:12 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 303 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:31:12.937936 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-eb2fb9dd-ca4a-477f-8c1b-c472d615aefb tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:31:12.943360 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-eb2fb9dd-ca4a-477f-8c1b-c472d615aefb tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] [instance: 3aa4786f-20b7-4123-a231-d3d1c3b91b3b] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:31:12.947297 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-eb2fb9dd-ca4a-477f-8c1b-c472d615aefb tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:12.947752 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-eb2fb9dd-ca4a-477f-8c1b-c472d615aefb tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:12.948737 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-eb2fb9dd-ca4a-477f-8c1b-c472d615aefb tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:12.949342 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-eb2fb9dd-ca4a-477f-8c1b-c472d615aefb tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:13.158183 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-bad75b62-9d47-4d8e-b20b-2b1cff03c835 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] 10.222.0.16 "DELETE /compute/v2.1/servers/59a4c34e-3184-465b-b14c-0ad55192c8ce" status: 204 len: 0 microversion: 2.1 time: 0.549814 Jul 16 21:31:13.158795 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2266/4542] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:31:12 2022] DELETE /compute/v2.1/servers/59a4c34e-3184-465b-b14c-0ad55192c8ce => generated 0 bytes in 551 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 16 21:31:13.166006 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-fd4c70ea-9f9c-4cf3-b767-1e290bdf8908 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:31:13.166779 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-fd4c70ea-9f9c-4cf3-b767-1e290bdf8908 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] [instance: 4fa0664b-97f0-44ab-9823-a3a92c1bfdb8] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:31:13.177995 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-fd4c70ea-9f9c-4cf3-b767-1e290bdf8908 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:13.179070 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-fd4c70ea-9f9c-4cf3-b767-1e290bdf8908 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:13.179830 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-fd4c70ea-9f9c-4cf3-b767-1e290bdf8908 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:13.180274 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-fd4c70ea-9f9c-4cf3-b767-1e290bdf8908 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:13.380648 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-eb2fb9dd-ca4a-477f-8c1b-c472d615aefb tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1006989e743247bdb29a51b3a634afd9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e8c65e58fac42c4bd7f49eaa3a14115', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:31:13.391853 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-eb2fb9dd-ca4a-477f-8c1b-c472d615aefb tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] 10.222.0.16 "GET /compute/v2.1/servers/3aa4786f-20b7-4123-a231-d3d1c3b91b3b" status: 200 len: 1314 microversion: 2.1 time: 0.457727 Jul 16 21:31:13.393061 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2277/4543] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:31:12 2022] GET /compute/v2.1/servers/3aa4786f-20b7-4123-a231-d3d1c3b91b3b => generated 1314 bytes in 460 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:31:13.399688 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-0d70e59d-24ab-4413-94c0-51e424e1c82a tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:31:13.400564 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-0d70e59d-24ab-4413-94c0-51e424e1c82a tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] [instance: 59a4c34e-3184-465b-b14c-0ad55192c8ce] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:31:13.409643 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0d70e59d-24ab-4413-94c0-51e424e1c82a tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:13.410325 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0d70e59d-24ab-4413-94c0-51e424e1c82a tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:13.411671 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0d70e59d-24ab-4413-94c0-51e424e1c82a tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:13.412863 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0d70e59d-24ab-4413-94c0-51e424e1c82a tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:13.627468 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-fd4c70ea-9f9c-4cf3-b767-1e290bdf8908 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da77ec3d0f04496d8c8df4025bba76d1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '207ec2d1883c4a1ba89696e6e77cf3aa', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:31:13.637799 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-fd4c70ea-9f9c-4cf3-b767-1e290bdf8908 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] 10.222.0.16 "GET /compute/v2.1/servers/4fa0664b-97f0-44ab-9823-a3a92c1bfdb8" status: 200 len: 1506 microversion: 2.1 time: 0.477188 Jul 16 21:31:13.638702 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2267/4544] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:31:13 2022] GET /compute/v2.1/servers/4fa0664b-97f0-44ab-9823-a3a92c1bfdb8 => generated 1506 bytes in 479 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:31:13.856888 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-0d70e59d-24ab-4413-94c0-51e424e1c82a tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ddfae3448704591b662de83b97a1876', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bed7c94624484c7cb8abeb3bfc8757b9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:31:13.875855 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-0d70e59d-24ab-4413-94c0-51e424e1c82a tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] 10.222.0.16 "GET /compute/v2.1/servers/59a4c34e-3184-465b-b14c-0ad55192c8ce" status: 200 len: 1671 microversion: 2.1 time: 0.481047 Jul 16 21:31:13.876453 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2278/4545] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:31:13 2022] GET /compute/v2.1/servers/59a4c34e-3184-465b-b14c-0ad55192c8ce => generated 1671 bytes in 483 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:31:14.411031 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-f5e7683f-03f2-429b-9bd4-2ebb569a1b34 tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:31:14.411957 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-f5e7683f-03f2-429b-9bd4-2ebb569a1b34 tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] [instance: 3aa4786f-20b7-4123-a231-d3d1c3b91b3b] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:31:14.426416 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f5e7683f-03f2-429b-9bd4-2ebb569a1b34 tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:14.429802 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f5e7683f-03f2-429b-9bd4-2ebb569a1b34 tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:14.431978 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f5e7683f-03f2-429b-9bd4-2ebb569a1b34 tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:14.434507 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f5e7683f-03f2-429b-9bd4-2ebb569a1b34 tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:14.657857 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-972155ce-b253-4dfa-9728-f9f1ca9230ce tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:31:14.658516 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-972155ce-b253-4dfa-9728-f9f1ca9230ce tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] [instance: 4fa0664b-97f0-44ab-9823-a3a92c1bfdb8] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:31:14.669311 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell0:SELECT=10 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:31:14.674071 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-972155ce-b253-4dfa-9728-f9f1ca9230ce tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:14.674710 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-972155ce-b253-4dfa-9728-f9f1ca9230ce tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:14.675880 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-972155ce-b253-4dfa-9728-f9f1ca9230ce tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:14.676432 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-972155ce-b253-4dfa-9728-f9f1ca9230ce tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:15.260289 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-f5e7683f-03f2-429b-9bd4-2ebb569a1b34 tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1006989e743247bdb29a51b3a634afd9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e8c65e58fac42c4bd7f49eaa3a14115', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:31:15.269457 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-f5e7683f-03f2-429b-9bd4-2ebb569a1b34 tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] 10.222.0.16 "GET /compute/v2.1/servers/3aa4786f-20b7-4123-a231-d3d1c3b91b3b" status: 200 len: 1314 microversion: 2.1 time: 0.863509 Jul 16 21:31:15.288298 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2268/4546] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:31:14 2022] GET /compute/v2.1/servers/3aa4786f-20b7-4123-a231-d3d1c3b91b3b => generated 1314 bytes in 874 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:31:15.288298 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-2c1fbaa9-4292-40b1-a3da-d02e15cce734 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:31:15.288298 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-2c1fbaa9-4292-40b1-a3da-d02e15cce734 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] [instance: 59a4c34e-3184-465b-b14c-0ad55192c8ce] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:31:15.302805 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2c1fbaa9-4292-40b1-a3da-d02e15cce734 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:15.303373 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2c1fbaa9-4292-40b1-a3da-d02e15cce734 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:15.306743 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2c1fbaa9-4292-40b1-a3da-d02e15cce734 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:15.307599 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2c1fbaa9-4292-40b1-a3da-d02e15cce734 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:15.605547 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-972155ce-b253-4dfa-9728-f9f1ca9230ce tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da77ec3d0f04496d8c8df4025bba76d1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '207ec2d1883c4a1ba89696e6e77cf3aa', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:31:15.613326 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-972155ce-b253-4dfa-9728-f9f1ca9230ce tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] 10.222.0.16 "GET /compute/v2.1/servers/4fa0664b-97f0-44ab-9823-a3a92c1bfdb8" status: 200 len: 1506 microversion: 2.1 time: 0.959815 Jul 16 21:31:15.624805 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2279/4547] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:31:14 2022] GET /compute/v2.1/servers/4fa0664b-97f0-44ab-9823-a3a92c1bfdb8 => generated 1506 bytes in 972 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:31:15.760303 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-2c1fbaa9-4292-40b1-a3da-d02e15cce734 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ddfae3448704591b662de83b97a1876', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bed7c94624484c7cb8abeb3bfc8757b9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:31:15.769090 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-2c1fbaa9-4292-40b1-a3da-d02e15cce734 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] 10.222.0.16 "GET /compute/v2.1/servers/59a4c34e-3184-465b-b14c-0ad55192c8ce" status: 200 len: 1671 microversion: 2.1 time: 0.488987 Jul 16 21:31:15.769781 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2269/4548] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:31:15 2022] GET /compute/v2.1/servers/59a4c34e-3184-465b-b14c-0ad55192c8ce => generated 1671 bytes in 490 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:31:16.144491 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-dd175581-de6e-495f-9cea-e938d150755f req-827c5fc9-c8da-4694-9a99-5fd43dcf13fa service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "59a4c34e-3184-465b-b14c-0ad55192c8ce", "name": "network-vif-deleted", "tag": "cad17319-3e42-44f8-911a-cd258470adec"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:31:16.154665 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-dd175581-de6e-495f-9cea-e938d150755f req-827c5fc9-c8da-4694-9a99-5fd43dcf13fa service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:16.155147 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-dd175581-de6e-495f-9cea-e938d150755f req-827c5fc9-c8da-4694-9a99-5fd43dcf13fa service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:16.168443 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-dd175581-de6e-495f-9cea-e938d150755f req-827c5fc9-c8da-4694-9a99-5fd43dcf13fa service nova] Creating event network-vif-deleted:cad17319-3e42-44f8-911a-cd258470adec for instance 59a4c34e-3184-465b-b14c-0ad55192c8ce on nch2-master-1 Jul 16 21:31:16.173839 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-dd175581-de6e-495f-9cea-e938d150755f req-827c5fc9-c8da-4694-9a99-5fd43dcf13fa service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.032991 Jul 16 21:31:16.174532 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2280/4549] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:31:16 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:31:16.299543 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-4724e9b7-696c-4f93-a45a-0b2a97aae827 tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:31:16.300086 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-4724e9b7-696c-4f93-a45a-0b2a97aae827 tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] [instance: 3aa4786f-20b7-4123-a231-d3d1c3b91b3b] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:31:16.307251 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4724e9b7-696c-4f93-a45a-0b2a97aae827 tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:16.307710 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4724e9b7-696c-4f93-a45a-0b2a97aae827 tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:16.308479 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4724e9b7-696c-4f93-a45a-0b2a97aae827 tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:16.308932 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4724e9b7-696c-4f93-a45a-0b2a97aae827 tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:16.632554 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-a0cf4eff-275c-4d8c-aa53-9a30433bc383 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:31:16.633348 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-a0cf4eff-275c-4d8c-aa53-9a30433bc383 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] [instance: 4fa0664b-97f0-44ab-9823-a3a92c1bfdb8] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:31:16.652031 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a0cf4eff-275c-4d8c-aa53-9a30433bc383 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:16.655739 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a0cf4eff-275c-4d8c-aa53-9a30433bc383 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:16.658150 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a0cf4eff-275c-4d8c-aa53-9a30433bc383 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:16.659436 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a0cf4eff-275c-4d8c-aa53-9a30433bc383 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:16.848136 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-4724e9b7-696c-4f93-a45a-0b2a97aae827 tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1006989e743247bdb29a51b3a634afd9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e8c65e58fac42c4bd7f49eaa3a14115', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:31:16.857162 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-4724e9b7-696c-4f93-a45a-0b2a97aae827 tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] 10.222.0.16 "GET /compute/v2.1/servers/3aa4786f-20b7-4123-a231-d3d1c3b91b3b" status: 200 len: 1314 microversion: 2.1 time: 0.564788 Jul 16 21:31:16.857759 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2270/4550] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:31:16 2022] GET /compute/v2.1/servers/3aa4786f-20b7-4123-a231-d3d1c3b91b3b => generated 1314 bytes in 566 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:31:16.876049 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-7e9bdf79-ac96-40c4-b13c-aaa2421e6be5 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:31:16.876970 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-7e9bdf79-ac96-40c4-b13c-aaa2421e6be5 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] [instance: 59a4c34e-3184-465b-b14c-0ad55192c8ce] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:31:16.897773 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7e9bdf79-ac96-40c4-b13c-aaa2421e6be5 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:16.898600 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7e9bdf79-ac96-40c4-b13c-aaa2421e6be5 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:16.902452 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7e9bdf79-ac96-40c4-b13c-aaa2421e6be5 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:16.904074 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7e9bdf79-ac96-40c4-b13c-aaa2421e6be5 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:16.949598 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-a0cf4eff-275c-4d8c-aa53-9a30433bc383 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da77ec3d0f04496d8c8df4025bba76d1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '207ec2d1883c4a1ba89696e6e77cf3aa', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:31:16.965751 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-a0cf4eff-275c-4d8c-aa53-9a30433bc383 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] 10.222.0.16 "GET /compute/v2.1/servers/4fa0664b-97f0-44ab-9823-a3a92c1bfdb8" status: 200 len: 1506 microversion: 2.1 time: 0.334384 Jul 16 21:31:16.965751 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2281/4551] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:31:16 2022] GET /compute/v2.1/servers/4fa0664b-97f0-44ab-9823-a3a92c1bfdb8 => generated 1506 bytes in 336 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:31:16.968769 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-a18c068b-c0a3-4ae8-b7f4-77099de634eb req-2186edce-acc1-4b21-ae28-6e1ff776875a service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "3aa4786f-20b7-4123-a231-d3d1c3b91b3b", "name": "network-vif-plugged", "status": "completed", "tag": "6c497884-33a6-412b-8db5-37cebffc7eca"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:31:16.980572 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-a18c068b-c0a3-4ae8-b7f4-77099de634eb req-2186edce-acc1-4b21-ae28-6e1ff776875a service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:16.981441 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-a18c068b-c0a3-4ae8-b7f4-77099de634eb req-2186edce-acc1-4b21-ae28-6e1ff776875a service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:16.984340 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7e9bdf79-ac96-40c4-b13c-aaa2421e6be5 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:16.984804 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7e9bdf79-ac96-40c4-b13c-aaa2421e6be5 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:16.985264 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.objects.instance [None req-7e9bdf79-ac96-40c4-b13c-aaa2421e6be5 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lazy-loading 'fault' on Instance uuid 59a4c34e-3184-465b-b14c-0ad55192c8ce {{(pid=138680) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:31:16.995598 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-a18c068b-c0a3-4ae8-b7f4-77099de634eb req-2186edce-acc1-4b21-ae28-6e1ff776875a service nova] Creating event network-vif-plugged:6c497884-33a6-412b-8db5-37cebffc7eca for instance 3aa4786f-20b7-4123-a231-d3d1c3b91b3b on nch2-master-1 Jul 16 21:31:17.000965 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-a18c068b-c0a3-4ae8-b7f4-77099de634eb req-2186edce-acc1-4b21-ae28-6e1ff776875a service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.036582 Jul 16 21:31:17.002026 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2282/4552] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:31:16 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:31:17.030168 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-7e9bdf79-ac96-40c4-b13c-aaa2421e6be5 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ddfae3448704591b662de83b97a1876', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bed7c94624484c7cb8abeb3bfc8757b9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:31:17.039246 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-7e9bdf79-ac96-40c4-b13c-aaa2421e6be5 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] 10.222.0.16 "GET /compute/v2.1/servers/59a4c34e-3184-465b-b14c-0ad55192c8ce" status: 200 len: 1338 microversion: 2.1 time: 0.166927 Jul 16 21:31:17.040331 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2271/4553] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:31:16 2022] GET /compute/v2.1/servers/59a4c34e-3184-465b-b14c-0ad55192c8ce => generated 1338 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:31:17.876060 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-743949bc-0c65-4b22-a625-5b236e54d2df tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:31:17.876966 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-743949bc-0c65-4b22-a625-5b236e54d2df tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] [instance: 3aa4786f-20b7-4123-a231-d3d1c3b91b3b] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:31:17.886860 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-743949bc-0c65-4b22-a625-5b236e54d2df tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:17.887379 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-743949bc-0c65-4b22-a625-5b236e54d2df tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:17.888697 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-743949bc-0c65-4b22-a625-5b236e54d2df tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:17.889280 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-743949bc-0c65-4b22-a625-5b236e54d2df tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:17.986488 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-8fb10bce-e75b-417e-ae82-e11f1215eb30 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:31:17.987765 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-8fb10bce-e75b-417e-ae82-e11f1215eb30 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] [instance: 4fa0664b-97f0-44ab-9823-a3a92c1bfdb8] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:31:17.998029 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8fb10bce-e75b-417e-ae82-e11f1215eb30 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:17.998823 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8fb10bce-e75b-417e-ae82-e11f1215eb30 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:17.999698 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8fb10bce-e75b-417e-ae82-e11f1215eb30 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:18.000445 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8fb10bce-e75b-417e-ae82-e11f1215eb30 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:18.106836 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-743949bc-0c65-4b22-a625-5b236e54d2df tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1006989e743247bdb29a51b3a634afd9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e8c65e58fac42c4bd7f49eaa3a14115', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:31:18.114830 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-743949bc-0c65-4b22-a625-5b236e54d2df tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] 10.222.0.16 "GET /compute/v2.1/servers/3aa4786f-20b7-4123-a231-d3d1c3b91b3b" status: 200 len: 1314 microversion: 2.1 time: 0.243441 Jul 16 21:31:18.115691 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2283/4554] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:31:17 2022] GET /compute/v2.1/servers/3aa4786f-20b7-4123-a231-d3d1c3b91b3b => generated 1314 bytes in 245 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:31:18.135796 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-bb992a04-7421-4abd-b1c5-b8ceaae27317 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:31:18.137666 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-bb992a04-7421-4abd-b1c5-b8ceaae27317 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] [instance: 59a4c34e-3184-465b-b14c-0ad55192c8ce] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:31:18.142792 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-bb992a04-7421-4abd-b1c5-b8ceaae27317 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:18.143288 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-bb992a04-7421-4abd-b1c5-b8ceaae27317 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:18.144064 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-bb992a04-7421-4abd-b1c5-b8ceaae27317 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:18.144505 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-bb992a04-7421-4abd-b1c5-b8ceaae27317 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:18.159159 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-bb992a04-7421-4abd-b1c5-b8ceaae27317 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] HTTP exception thrown: Instance 59a4c34e-3184-465b-b14c-0ad55192c8ce could not be found. Jul 16 21:31:18.160827 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-bb992a04-7421-4abd-b1c5-b8ceaae27317 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Request method failure captured: Jul 16 21:31:18.160827 ncdv-master-1 devstack@n-api.service[138681]: request: GET /compute/v2.1/servers/59a4c34e-3184-465b-b14c-0ad55192c8ce HTTP/1.1 Jul 16 21:31:18.160827 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:31:18.160827 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:31:18.160827 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:31:18.160827 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 21:31:18.160827 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:31:18.160827 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:31:18.160827 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:31:18.160827 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:31:18.160827 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0y2AoaTmW9fzKpp4uGm7wex0cqFnrFlzY31Q3uQkOFuadyloOF8tCXPJOxztzHL0OnbEMWGJwuxCROg93CNNRADEFI3axz2E-JaVn1YaPI_h8JL-g89OY5Lk8ug8D0BLGsXQOPqIYrZnwwRF_WHMaANcXwS7qf8jAbwL7w-UUVnlf9c Jul 16 21:31:18.160827 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:31:18.160827 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:31:18.160827 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:31:18.160827 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:31:18.160827 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:31:18.160827 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:31:18.160827 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: bed7c94624484c7cb8abeb3bfc8757b9 Jul 16 21:31:18.160827 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-TestNetworkBasicOps-590553405 Jul 16 21:31:18.160827 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:31:18.160827 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:31:18.160827 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/bed7c94624484c7cb8abeb3bfc8757b9"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/bed7c94624484c7cb8abeb3bfc8757b9"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_bed7c94624484c7cb8abeb3bfc8757b9"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/bed7c94624484c7cb8abeb3bfc8757b9"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:31:18.160827 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-TestNetworkBasicOps-590553405 Jul 16 21:31:18.160827 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: bed7c94624484c7cb8abeb3bfc8757b9 Jul 16 21:31:18.160827 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-TestNetworkBasicOps-590553405 Jul 16 21:31:18.160827 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-TestNetworkBasicOps-590553405-project Jul 16 21:31:18.160827 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:31:18.160827 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:31:18.160827 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: 4ddfae3448704591b662de83b97a1876 Jul 16 21:31:18.160827 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-TestNetworkBasicOps-590553405-project Jul 16 21:31:18.160827 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:31:18.160827 ncdv-master-1 devstack@n-api.service[138681]: exception: Instance 59a4c34e-3184-465b-b14c-0ad55192c8ce could not be found. Jul 16 21:31:18.160827 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': '59a4c34e-3184-465b-b14c-0ad55192c8ce'} Jul 16 21:31:18.160827 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:31:18.160827 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:31:18.160827 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:31:18.160827 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:31:18.160827 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:31:18.160827 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:31:18.160827 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:31:18.160827 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:31:18.160827 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:31:18.160827 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:31:18.160827 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 59a4c34e-3184-465b-b14c-0ad55192c8ce could not be found. Jul 16 21:31:18.160827 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:31:18.160827 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:31:18.160827 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:31:18.160827 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:31:18.160827 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:31:18.160827 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:31:18.160827 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:31:18.160827 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:31:18.160827 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:31:18.160827 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:31:18.160827 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:31:18.160827 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:31:18.160827 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:31:18.160827 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:31:18.160827 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:31:18.160827 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:31:18.160827 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 59a4c34e-3184-465b-b14c-0ad55192c8ce could not be found. Jul 16 21:31:18.160827 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:31:18.160827 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:31:18.160827 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:31:18.171795 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:31:18.171795 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:31:18.171795 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:31:18.171795 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:31:18.171795 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:31:18.171795 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 59a4c34e-3184-465b-b14c-0ad55192c8ce could not be found. Jul 16 21:31:18.171795 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:31:18.171795 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-bb992a04-7421-4abd-b1c5-b8ceaae27317 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Returning 404 to user: Instance 59a4c34e-3184-465b-b14c-0ad55192c8ce could not be found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:31:18.171795 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-bb992a04-7421-4abd-b1c5-b8ceaae27317 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] 10.222.0.16 "GET /compute/v2.1/servers/59a4c34e-3184-465b-b14c-0ad55192c8ce" status: 404 len: 111 microversion: 2.1 time: 0.035405 Jul 16 21:31:18.171795 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2284/4555] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:31:18 2022] GET /compute/v2.1/servers/59a4c34e-3184-465b-b14c-0ad55192c8ce => generated 111 bytes in 37 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:31:18.224861 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-8fb10bce-e75b-417e-ae82-e11f1215eb30 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da77ec3d0f04496d8c8df4025bba76d1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '207ec2d1883c4a1ba89696e6e77cf3aa', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:31:18.233528 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-8fb10bce-e75b-417e-ae82-e11f1215eb30 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] 10.222.0.16 "GET /compute/v2.1/servers/4fa0664b-97f0-44ab-9823-a3a92c1bfdb8" status: 200 len: 1506 microversion: 2.1 time: 0.258781 Jul 16 21:31:18.234235 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2272/4556] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:31:17 2022] GET /compute/v2.1/servers/4fa0664b-97f0-44ab-9823-a3a92c1bfdb8 => generated 1506 bytes in 260 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:31:18.515605 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-1474c12b-2bd7-4421-9223-218d0f0ebc0a tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Calling method '.version_select at 0x7fdd9c54ae50>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:31:18.675357 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-1474c12b-2bd7-4421-9223-218d0f0ebc0a tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] 10.222.0.16 "DELETE /compute/v2.1/os-keypairs/tempest-TestNetworkBasicOps-691771975" status: 202 len: 0 microversion: 2.1 time: 0.165475 Jul 16 21:31:18.681902 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2285/4557] 10.222.0.16 () {58 vars in 1253 bytes} [Sat Jul 16 21:31:18 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestNetworkBasicOps-691771975 => generated 0 bytes in 167 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 21:31:18.828393 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-76303bd8-7966-43e8-b8bb-4aba9c481199 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:31:18.832397 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-76303bd8-7966-43e8-b8bb-4aba9c481199 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] [instance: 08a33911-6e47-45fd-b499-155f199aacf8] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:31:18.842219 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-76303bd8-7966-43e8-b8bb-4aba9c481199 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:18.842815 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-76303bd8-7966-43e8-b8bb-4aba9c481199 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:18.844012 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-76303bd8-7966-43e8-b8bb-4aba9c481199 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:18.844609 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-76303bd8-7966-43e8-b8bb-4aba9c481199 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:18.886750 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-76303bd8-7966-43e8-b8bb-4aba9c481199 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] 10.222.0.16 "GET /compute/v2.1/servers/08a33911-6e47-45fd-b499-155f199aacf8/os-volume_attachments/18e9c5fe-8f27-4cbf-bbe8-0e1f902e6813" status: 200 len: 194 microversion: 2.20 time: 0.062945 Jul 16 21:31:18.888202 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2273/4558] 10.222.0.16 () {60 vars in 1453 bytes} [Sat Jul 16 21:31:18 2022] GET /compute/v2.1/servers/08a33911-6e47-45fd-b499-155f199aacf8/os-volume_attachments/18e9c5fe-8f27-4cbf-bbe8-0e1f902e6813 => generated 194 bytes in 65 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 16 21:31:18.976706 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-6132a5a0-1793-4887-82ef-846badb6d6a3 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:31:18.977319 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-6132a5a0-1793-4887-82ef-846badb6d6a3 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] [instance: 08a33911-6e47-45fd-b499-155f199aacf8] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:31:18.991986 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6132a5a0-1793-4887-82ef-846badb6d6a3 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:18.997448 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6132a5a0-1793-4887-82ef-846badb6d6a3 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:18.997448 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6132a5a0-1793-4887-82ef-846badb6d6a3 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:18.997448 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6132a5a0-1793-4887-82ef-846badb6d6a3 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:19.124682 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.objects.instance [None req-6132a5a0-1793-4887-82ef-846badb6d6a3 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lazy-loading 'services' on Instance uuid 08a33911-6e47-45fd-b499-155f199aacf8 {{(pid=138681) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:31:19.135376 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-9694cfc5-2b15-40ba-b9c7-a44573e78fc4 tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:31:19.136652 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-9694cfc5-2b15-40ba-b9c7-a44573e78fc4 tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] [instance: 3aa4786f-20b7-4123-a231-d3d1c3b91b3b] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:31:19.151109 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9694cfc5-2b15-40ba-b9c7-a44573e78fc4 tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:19.153489 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9694cfc5-2b15-40ba-b9c7-a44573e78fc4 tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:19.153489 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9694cfc5-2b15-40ba-b9c7-a44573e78fc4 tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:19.154124 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9694cfc5-2b15-40ba-b9c7-a44573e78fc4 tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:19.473490 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-9694cfc5-2b15-40ba-b9c7-a44573e78fc4 tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1006989e743247bdb29a51b3a634afd9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e8c65e58fac42c4bd7f49eaa3a14115', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:31:19.484949 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-9694cfc5-2b15-40ba-b9c7-a44573e78fc4 tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] 10.222.0.16 "GET /compute/v2.1/servers/3aa4786f-20b7-4123-a231-d3d1c3b91b3b" status: 200 len: 1501 microversion: 2.1 time: 0.356543 Jul 16 21:31:19.501318 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2274/4559] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:31:19 2022] GET /compute/v2.1/servers/3aa4786f-20b7-4123-a231-d3d1c3b91b3b => generated 1501 bytes in 373 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:31:19.508249 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-30b3d6c3-e94a-4d2b-ba11-2093491395f4 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:31:19.509277 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-30b3d6c3-e94a-4d2b-ba11-2093491395f4 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] [instance: 4fa0664b-97f0-44ab-9823-a3a92c1bfdb8] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:31:19.520701 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-30b3d6c3-e94a-4d2b-ba11-2093491395f4 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:19.520701 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-30b3d6c3-e94a-4d2b-ba11-2093491395f4 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:19.520701 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-30b3d6c3-e94a-4d2b-ba11-2093491395f4 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:19.520701 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-30b3d6c3-e94a-4d2b-ba11-2093491395f4 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:19.548567 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-6132a5a0-1793-4887-82ef-846badb6d6a3 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] 10.222.0.16 "DELETE /compute/v2.1/servers/08a33911-6e47-45fd-b499-155f199aacf8/os-volume_attachments/18e9c5fe-8f27-4cbf-bbe8-0e1f902e6813" status: 202 len: 0 microversion: 2.20 time: 0.576629 Jul 16 21:31:19.550073 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2286/4560] 10.222.0.16 () {60 vars in 1456 bytes} [Sat Jul 16 21:31:18 2022] DELETE /compute/v2.1/servers/08a33911-6e47-45fd-b499-155f199aacf8/os-volume_attachments/18e9c5fe-8f27-4cbf-bbe8-0e1f902e6813 => generated 0 bytes in 579 msecs (HTTP/1.1 202) 9 headers in 363 bytes (2 switches on core 0) Jul 16 21:31:19.558479 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-ee0dfc62-2099-47f8-baeb-a772185bbb6b tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:31:19.559414 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-ee0dfc62-2099-47f8-baeb-a772185bbb6b tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] [instance: 3aa4786f-20b7-4123-a231-d3d1c3b91b3b] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:31:19.570914 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ee0dfc62-2099-47f8-baeb-a772185bbb6b tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:19.571423 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ee0dfc62-2099-47f8-baeb-a772185bbb6b tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:19.572386 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ee0dfc62-2099-47f8-baeb-a772185bbb6b tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:19.572863 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ee0dfc62-2099-47f8-baeb-a772185bbb6b tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:19.875157 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-30b3d6c3-e94a-4d2b-ba11-2093491395f4 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da77ec3d0f04496d8c8df4025bba76d1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '207ec2d1883c4a1ba89696e6e77cf3aa', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:31:19.923806 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-30b3d6c3-e94a-4d2b-ba11-2093491395f4 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] 10.222.0.16 "GET /compute/v2.1/servers/4fa0664b-97f0-44ab-9823-a3a92c1bfdb8" status: 200 len: 1506 microversion: 2.1 time: 0.420819 Jul 16 21:31:19.925878 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2275/4561] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:31:19 2022] GET /compute/v2.1/servers/4fa0664b-97f0-44ab-9823-a3a92c1bfdb8 => generated 1506 bytes in 422 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:31:19.940379 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-7f8186a7-6a30-4370-a05a-5cc9ee6dc6a3 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:31:19.941867 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-7f8186a7-6a30-4370-a05a-5cc9ee6dc6a3 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] [instance: 08a33911-6e47-45fd-b499-155f199aacf8] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:31:19.953076 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7f8186a7-6a30-4370-a05a-5cc9ee6dc6a3 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:19.953076 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7f8186a7-6a30-4370-a05a-5cc9ee6dc6a3 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:19.953076 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7f8186a7-6a30-4370-a05a-5cc9ee6dc6a3 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:19.953076 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7f8186a7-6a30-4370-a05a-5cc9ee6dc6a3 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:20.001149 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-ee0dfc62-2099-47f8-baeb-a772185bbb6b tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1006989e743247bdb29a51b3a634afd9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e8c65e58fac42c4bd7f49eaa3a14115', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:31:20.012038 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-7f8186a7-6a30-4370-a05a-5cc9ee6dc6a3 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] 10.222.0.16 "GET /compute/v2.1/servers/08a33911-6e47-45fd-b499-155f199aacf8/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.074844 Jul 16 21:31:20.012887 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-ee0dfc62-2099-47f8-baeb-a772185bbb6b tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] 10.222.0.16 "GET /compute/v2.1/servers/3aa4786f-20b7-4123-a231-d3d1c3b91b3b" status: 200 len: 1501 microversion: 2.1 time: 0.459597 Jul 16 21:31:20.014036 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2276/4562] 10.222.0.16 () {60 vars in 1342 bytes} [Sat Jul 16 21:31:19 2022] GET /compute/v2.1/servers/08a33911-6e47-45fd-b499-155f199aacf8/os-volume_attachments => generated 197 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 16 21:31:20.024181 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2287/4563] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:31:19 2022] GET /compute/v2.1/servers/3aa4786f-20b7-4123-a231-d3d1c3b91b3b => generated 1501 bytes in 473 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:31:20.033342 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-dd70148e-bbfa-4a16-b849-8d7b99e91db7 tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Action: 'action', calling method: >, body: {"suspend": {}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:31:20.034082 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-dd70148e-bbfa-4a16-b849-8d7b99e91db7 tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] [instance: 3aa4786f-20b7-4123-a231-d3d1c3b91b3b] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:31:20.041930 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-dd70148e-bbfa-4a16-b849-8d7b99e91db7 tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:20.042650 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-dd70148e-bbfa-4a16-b849-8d7b99e91db7 tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:20.043782 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-dd70148e-bbfa-4a16-b849-8d7b99e91db7 tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:20.044445 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-dd70148e-bbfa-4a16-b849-8d7b99e91db7 tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:20.071627 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.objects.instance [None req-dd70148e-bbfa-4a16-b849-8d7b99e91db7 tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Lazy-loading 'flavor' on Instance uuid 3aa4786f-20b7-4123-a231-d3d1c3b91b3b {{(pid=138680) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:31:20.546562 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-dd70148e-bbfa-4a16-b849-8d7b99e91db7 tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] 10.222.0.16 "POST /compute/v2.1/servers/3aa4786f-20b7-4123-a231-d3d1c3b91b3b/action" status: 202 len: 0 microversion: 2.1 time: 0.517596 Jul 16 21:31:20.547365 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2277/4564] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 21:31:20 2022] POST /compute/v2.1/servers/3aa4786f-20b7-4123-a231-d3d1c3b91b3b/action => generated 0 bytes in 519 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 21:31:20.568758 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-45e2794b-2460-4d3d-bd94-f19bdbb8971f tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:31:20.569551 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-45e2794b-2460-4d3d-bd94-f19bdbb8971f tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] [instance: 3aa4786f-20b7-4123-a231-d3d1c3b91b3b] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:31:20.583230 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-45e2794b-2460-4d3d-bd94-f19bdbb8971f tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:20.585261 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-45e2794b-2460-4d3d-bd94-f19bdbb8971f tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:20.587572 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-45e2794b-2460-4d3d-bd94-f19bdbb8971f tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:20.605035 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-45e2794b-2460-4d3d-bd94-f19bdbb8971f tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:20.943098 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-00610313-699a-4dfa-87a8-c49b7c23665a tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:31:20.945610 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-00610313-699a-4dfa-87a8-c49b7c23665a tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] [instance: 4fa0664b-97f0-44ab-9823-a3a92c1bfdb8] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:31:20.972378 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-00610313-699a-4dfa-87a8-c49b7c23665a tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:20.973139 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-00610313-699a-4dfa-87a8-c49b7c23665a tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:20.977953 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-00610313-699a-4dfa-87a8-c49b7c23665a tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:20.984987 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-00610313-699a-4dfa-87a8-c49b7c23665a tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:21.050149 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=95,nova_cell1:UPDATE=3,nova_cell1:INSERT=2 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:31:21.182577 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-45e2794b-2460-4d3d-bd94-f19bdbb8971f tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1006989e743247bdb29a51b3a634afd9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e8c65e58fac42c4bd7f49eaa3a14115', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:31:21.215698 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-45e2794b-2460-4d3d-bd94-f19bdbb8971f tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] 10.222.0.16 "GET /compute/v2.1/servers/3aa4786f-20b7-4123-a231-d3d1c3b91b3b" status: 200 len: 1509 microversion: 2.1 time: 0.657259 Jul 16 21:31:21.217151 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2288/4565] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:31:20 2022] GET /compute/v2.1/servers/3aa4786f-20b7-4123-a231-d3d1c3b91b3b => generated 1509 bytes in 666 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:31:21.230658 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-b8e7f7f4-91cd-4cf0-9dca-00cd84219ca1 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:31:21.267457 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-b8e7f7f4-91cd-4cf0-9dca-00cd84219ca1 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] [instance: 08a33911-6e47-45fd-b499-155f199aacf8] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:31:21.287174 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b8e7f7f4-91cd-4cf0-9dca-00cd84219ca1 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:21.291805 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b8e7f7f4-91cd-4cf0-9dca-00cd84219ca1 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:21.292936 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b8e7f7f4-91cd-4cf0-9dca-00cd84219ca1 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:21.295035 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b8e7f7f4-91cd-4cf0-9dca-00cd84219ca1 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:21.337249 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-00610313-699a-4dfa-87a8-c49b7c23665a tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da77ec3d0f04496d8c8df4025bba76d1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '207ec2d1883c4a1ba89696e6e77cf3aa', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:31:21.345213 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-00610313-699a-4dfa-87a8-c49b7c23665a tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] 10.222.0.16 "GET /compute/v2.1/servers/4fa0664b-97f0-44ab-9823-a3a92c1bfdb8" status: 200 len: 1506 microversion: 2.1 time: 0.407112 Jul 16 21:31:21.345785 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2278/4566] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:31:20 2022] GET /compute/v2.1/servers/4fa0664b-97f0-44ab-9823-a3a92c1bfdb8 => generated 1506 bytes in 409 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:31:21.389832 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-b8e7f7f4-91cd-4cf0-9dca-00cd84219ca1 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] 10.222.0.16 "GET /compute/v2.1/servers/08a33911-6e47-45fd-b499-155f199aacf8/os-volume_attachments" status: 200 len: 25 microversion: 2.20 time: 0.170082 Jul 16 21:31:21.390818 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2289/4567] 10.222.0.16 () {60 vars in 1342 bytes} [Sat Jul 16 21:31:21 2022] GET /compute/v2.1/servers/08a33911-6e47-45fd-b499-155f199aacf8/os-volume_attachments => generated 25 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:31:21.473228 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-3875f9c0-2bad-42bc-ab89-0fbe39a60e88 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:31:21.473830 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-3875f9c0-2bad-42bc-ab89-0fbe39a60e88 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] [instance: 08a33911-6e47-45fd-b499-155f199aacf8] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:31:21.480575 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3875f9c0-2bad-42bc-ab89-0fbe39a60e88 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:21.481065 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3875f9c0-2bad-42bc-ab89-0fbe39a60e88 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:21.482363 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3875f9c0-2bad-42bc-ab89-0fbe39a60e88 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:21.483530 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3875f9c0-2bad-42bc-ab89-0fbe39a60e88 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:21.526934 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-3875f9c0-2bad-42bc-ab89-0fbe39a60e88 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] [instance: 08a33911-6e47-45fd-b499-155f199aacf8] Going to try to terminate instance {{(pid=138680) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 16 21:31:21.914554 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-3875f9c0-2bad-42bc-ab89-0fbe39a60e88 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] 10.222.0.16 "DELETE /compute/v2.1/servers/08a33911-6e47-45fd-b499-155f199aacf8" status: 204 len: 0 microversion: 2.20 time: 0.445223 Jul 16 21:31:21.922416 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2279/4568] 10.222.0.16 () {60 vars in 1279 bytes} [Sat Jul 16 21:31:21 2022] DELETE /compute/v2.1/servers/08a33911-6e47-45fd-b499-155f199aacf8 => generated 0 bytes in 454 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Jul 16 21:31:21.935203 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-f8097325-af01-41c5-9da5-24151b15dd20 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:31:21.936354 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-f8097325-af01-41c5-9da5-24151b15dd20 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] [instance: 08a33911-6e47-45fd-b499-155f199aacf8] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:31:21.946492 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f8097325-af01-41c5-9da5-24151b15dd20 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:21.946955 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f8097325-af01-41c5-9da5-24151b15dd20 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:21.947791 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f8097325-af01-41c5-9da5-24151b15dd20 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:21.948254 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f8097325-af01-41c5-9da5-24151b15dd20 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:21.951257 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=99,nova_cell1:INSERT=1 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:31:22.159267 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-f8097325-af01-41c5-9da5-24151b15dd20 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:31:22.176435 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-f8097325-af01-41c5-9da5-24151b15dd20 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:31:22.179000 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-f8097325-af01-41c5-9da5-24151b15dd20 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:31:22.180552 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-f8097325-af01-41c5-9da5-24151b15dd20 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] 10.222.0.16 "GET /compute/v2.1/servers/08a33911-6e47-45fd-b499-155f199aacf8" status: 200 len: 1723 microversion: 2.20 time: 0.250095 Jul 16 21:31:22.181616 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2290/4569] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:31:21 2022] GET /compute/v2.1/servers/08a33911-6e47-45fd-b499-155f199aacf8 => generated 1723 bytes in 252 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:31:22.237052 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-de7b0c4b-ce57-4ba7-880c-255aa969e5a9 tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:31:22.238529 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-de7b0c4b-ce57-4ba7-880c-255aa969e5a9 tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] [instance: 3aa4786f-20b7-4123-a231-d3d1c3b91b3b] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:31:22.255045 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-de7b0c4b-ce57-4ba7-880c-255aa969e5a9 tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:22.259675 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-de7b0c4b-ce57-4ba7-880c-255aa969e5a9 tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:22.259675 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-de7b0c4b-ce57-4ba7-880c-255aa969e5a9 tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:22.259675 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-de7b0c4b-ce57-4ba7-880c-255aa969e5a9 tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:22.361752 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-01e5a53d-ea4c-4350-9bb7-d5ada18bbfbe tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:31:22.362486 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-01e5a53d-ea4c-4350-9bb7-d5ada18bbfbe tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] [instance: 4fa0664b-97f0-44ab-9823-a3a92c1bfdb8] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:31:22.374337 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-01e5a53d-ea4c-4350-9bb7-d5ada18bbfbe tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:22.374794 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-01e5a53d-ea4c-4350-9bb7-d5ada18bbfbe tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:22.376820 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-01e5a53d-ea4c-4350-9bb7-d5ada18bbfbe tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:22.377471 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-01e5a53d-ea4c-4350-9bb7-d5ada18bbfbe tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:22.900663 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-de7b0c4b-ce57-4ba7-880c-255aa969e5a9 tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1006989e743247bdb29a51b3a634afd9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e8c65e58fac42c4bd7f49eaa3a14115', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:31:22.915730 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-de7b0c4b-ce57-4ba7-880c-255aa969e5a9 tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] 10.222.0.16 "GET /compute/v2.1/servers/3aa4786f-20b7-4123-a231-d3d1c3b91b3b" status: 200 len: 1492 microversion: 2.1 time: 0.681977 Jul 16 21:31:22.916795 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2280/4570] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:31:22 2022] GET /compute/v2.1/servers/3aa4786f-20b7-4123-a231-d3d1c3b91b3b => generated 1492 bytes in 687 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:31:22.922096 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-01e5a53d-ea4c-4350-9bb7-d5ada18bbfbe tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da77ec3d0f04496d8c8df4025bba76d1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '207ec2d1883c4a1ba89696e6e77cf3aa', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:31:22.938978 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-14728234-82fa-4c40-a7e8-23d91d972e43 tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Action: 'action', calling method: >, body: {"resume": {}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:31:22.939565 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-14728234-82fa-4c40-a7e8-23d91d972e43 tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] [instance: 3aa4786f-20b7-4123-a231-d3d1c3b91b3b] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:31:22.941067 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-01e5a53d-ea4c-4350-9bb7-d5ada18bbfbe tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] 10.222.0.16 "GET /compute/v2.1/servers/4fa0664b-97f0-44ab-9823-a3a92c1bfdb8" status: 200 len: 1506 microversion: 2.1 time: 0.582633 Jul 16 21:31:22.952739 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2291/4571] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:31:22 2022] GET /compute/v2.1/servers/4fa0664b-97f0-44ab-9823-a3a92c1bfdb8 => generated 1506 bytes in 595 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:31:22.978325 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-14728234-82fa-4c40-a7e8-23d91d972e43 tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:22.983543 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-14728234-82fa-4c40-a7e8-23d91d972e43 tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:22.988438 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-14728234-82fa-4c40-a7e8-23d91d972e43 tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:22.989089 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-14728234-82fa-4c40-a7e8-23d91d972e43 tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:23.201140 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-13a67015-4b09-48cc-a521-a03bf08bea3e tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:31:23.201942 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-13a67015-4b09-48cc-a521-a03bf08bea3e tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] [instance: 08a33911-6e47-45fd-b499-155f199aacf8] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:31:23.217454 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-13a67015-4b09-48cc-a521-a03bf08bea3e tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:23.217927 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-13a67015-4b09-48cc-a521-a03bf08bea3e tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:23.218736 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-13a67015-4b09-48cc-a521-a03bf08bea3e tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:23.222346 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-13a67015-4b09-48cc-a521-a03bf08bea3e tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:23.348977 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-14728234-82fa-4c40-a7e8-23d91d972e43 tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] 10.222.0.16 "POST /compute/v2.1/servers/3aa4786f-20b7-4123-a231-d3d1c3b91b3b/action" status: 202 len: 0 microversion: 2.1 time: 0.416105 Jul 16 21:31:23.355600 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2281/4572] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 21:31:22 2022] POST /compute/v2.1/servers/3aa4786f-20b7-4123-a231-d3d1c3b91b3b/action => generated 0 bytes in 424 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 21:31:23.373450 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-0a336203-60b6-4cd1-8d2c-4810d97f8510 tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:31:23.374308 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-0a336203-60b6-4cd1-8d2c-4810d97f8510 tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] [instance: 3aa4786f-20b7-4123-a231-d3d1c3b91b3b] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:31:23.415995 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0a336203-60b6-4cd1-8d2c-4810d97f8510 tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:23.416475 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0a336203-60b6-4cd1-8d2c-4810d97f8510 tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:23.417298 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0a336203-60b6-4cd1-8d2c-4810d97f8510 tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:23.420179 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0a336203-60b6-4cd1-8d2c-4810d97f8510 tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:24.635243 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-0a336203-60b6-4cd1-8d2c-4810d97f8510 tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1006989e743247bdb29a51b3a634afd9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e8c65e58fac42c4bd7f49eaa3a14115', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:31:24.645648 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-0a336203-60b6-4cd1-8d2c-4810d97f8510 tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] 10.222.0.16 "GET /compute/v2.1/servers/3aa4786f-20b7-4123-a231-d3d1c3b91b3b" status: 200 len: 1498 microversion: 2.1 time: 1.287954 Jul 16 21:31:24.646632 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2282/4573] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:31:23 2022] GET /compute/v2.1/servers/3aa4786f-20b7-4123-a231-d3d1c3b91b3b => generated 1498 bytes in 1290 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:31:24.662641 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-c6fef9f9-adbd-431f-a7d8-01c844d9555f tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:31:24.663695 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-c6fef9f9-adbd-431f-a7d8-01c844d9555f tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] [instance: 4fa0664b-97f0-44ab-9823-a3a92c1bfdb8] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:31:24.675132 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c6fef9f9-adbd-431f-a7d8-01c844d9555f tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:24.675862 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c6fef9f9-adbd-431f-a7d8-01c844d9555f tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:24.678731 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c6fef9f9-adbd-431f-a7d8-01c844d9555f tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:24.679786 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c6fef9f9-adbd-431f-a7d8-01c844d9555f tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:24.733088 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-13a67015-4b09-48cc-a521-a03bf08bea3e tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:31:24.743392 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-13a67015-4b09-48cc-a521-a03bf08bea3e tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:31:24.745186 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-13a67015-4b09-48cc-a521-a03bf08bea3e tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:31:24.746517 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-13a67015-4b09-48cc-a521-a03bf08bea3e tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] 10.222.0.16 "GET /compute/v2.1/servers/08a33911-6e47-45fd-b499-155f199aacf8" status: 200 len: 1723 microversion: 2.20 time: 1.550104 Jul 16 21:31:24.747506 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2292/4574] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:31:23 2022] GET /compute/v2.1/servers/08a33911-6e47-45fd-b499-155f199aacf8 => generated 1723 bytes in 1552 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:31:25.333256 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-c6fef9f9-adbd-431f-a7d8-01c844d9555f tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da77ec3d0f04496d8c8df4025bba76d1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '207ec2d1883c4a1ba89696e6e77cf3aa', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:31:25.344005 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-c6fef9f9-adbd-431f-a7d8-01c844d9555f tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] 10.222.0.16 "GET /compute/v2.1/servers/4fa0664b-97f0-44ab-9823-a3a92c1bfdb8" status: 200 len: 1486 microversion: 2.1 time: 0.695073 Jul 16 21:31:25.346952 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2283/4575] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:31:24 2022] GET /compute/v2.1/servers/4fa0664b-97f0-44ab-9823-a3a92c1bfdb8 => generated 1486 bytes in 699 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:31:25.370454 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-80303e63-72bd-4c74-9a95-b2a04b106e90 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Action: 'action', calling method: >, body: {"unrescue": {}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:31:25.371375 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-80303e63-72bd-4c74-9a95-b2a04b106e90 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] [instance: 4fa0664b-97f0-44ab-9823-a3a92c1bfdb8] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:31:25.387354 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-80303e63-72bd-4c74-9a95-b2a04b106e90 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:25.390622 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-80303e63-72bd-4c74-9a95-b2a04b106e90 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:25.391771 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-80303e63-72bd-4c74-9a95-b2a04b106e90 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:25.392422 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-80303e63-72bd-4c74-9a95-b2a04b106e90 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:25.663936 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-28eb724e-1158-467e-b8ba-734b0fbefafc tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:31:25.664563 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-28eb724e-1158-467e-b8ba-734b0fbefafc tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] [instance: 3aa4786f-20b7-4123-a231-d3d1c3b91b3b] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:31:25.671254 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-28eb724e-1158-467e-b8ba-734b0fbefafc tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:25.671807 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-28eb724e-1158-467e-b8ba-734b0fbefafc tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:25.672668 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-28eb724e-1158-467e-b8ba-734b0fbefafc tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:25.673483 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-28eb724e-1158-467e-b8ba-734b0fbefafc tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:25.684242 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-80303e63-72bd-4c74-9a95-b2a04b106e90 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] 10.222.0.16 "POST /compute/v2.1/servers/4fa0664b-97f0-44ab-9823-a3a92c1bfdb8/action" status: 202 len: 0 microversion: 2.1 time: 0.318383 Jul 16 21:31:25.685029 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2293/4576] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 21:31:25 2022] POST /compute/v2.1/servers/4fa0664b-97f0-44ab-9823-a3a92c1bfdb8/action => generated 0 bytes in 320 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 21:31:25.694044 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-471d8a9d-4255-4635-9363-bf02978a181f tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:31:25.695520 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-471d8a9d-4255-4635-9363-bf02978a181f tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] [instance: 4fa0664b-97f0-44ab-9823-a3a92c1bfdb8] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:31:25.705179 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-471d8a9d-4255-4635-9363-bf02978a181f tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:25.705737 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-471d8a9d-4255-4635-9363-bf02978a181f tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:25.707451 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-471d8a9d-4255-4635-9363-bf02978a181f tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:25.708100 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-471d8a9d-4255-4635-9363-bf02978a181f tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:26.708477 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-28eb724e-1158-467e-b8ba-734b0fbefafc tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1006989e743247bdb29a51b3a634afd9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e8c65e58fac42c4bd7f49eaa3a14115', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:31:26.719681 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-28eb724e-1158-467e-b8ba-734b0fbefafc tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] 10.222.0.16 "GET /compute/v2.1/servers/3aa4786f-20b7-4123-a231-d3d1c3b91b3b" status: 200 len: 1498 microversion: 2.1 time: 1.060280 Jul 16 21:31:26.720958 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2284/4577] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:31:25 2022] GET /compute/v2.1/servers/3aa4786f-20b7-4123-a231-d3d1c3b91b3b => generated 1498 bytes in 1062 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:31:26.729668 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-c392d33e-ab2e-4bff-8d22-c403e2dfa2e1 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:31:26.730752 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-c392d33e-ab2e-4bff-8d22-c403e2dfa2e1 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] [instance: 08a33911-6e47-45fd-b499-155f199aacf8] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:31:26.737906 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c392d33e-ab2e-4bff-8d22-c403e2dfa2e1 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:26.738532 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c392d33e-ab2e-4bff-8d22-c403e2dfa2e1 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:26.739587 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c392d33e-ab2e-4bff-8d22-c403e2dfa2e1 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:26.740802 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c392d33e-ab2e-4bff-8d22-c403e2dfa2e1 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:26.778192 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-471d8a9d-4255-4635-9363-bf02978a181f tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da77ec3d0f04496d8c8df4025bba76d1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '207ec2d1883c4a1ba89696e6e77cf3aa', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:31:26.791600 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-471d8a9d-4255-4635-9363-bf02978a181f tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] 10.222.0.16 "GET /compute/v2.1/servers/4fa0664b-97f0-44ab-9823-a3a92c1bfdb8" status: 200 len: 1494 microversion: 2.1 time: 1.101898 Jul 16 21:31:26.820142 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2294/4578] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:31:25 2022] GET /compute/v2.1/servers/4fa0664b-97f0-44ab-9823-a3a92c1bfdb8 => generated 1494 bytes in 1131 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:31:27.277725 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-c392d33e-ab2e-4bff-8d22-c403e2dfa2e1 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:31:27.277725 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-c392d33e-ab2e-4bff-8d22-c403e2dfa2e1 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:31:27.277725 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-c392d33e-ab2e-4bff-8d22-c403e2dfa2e1 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:31:27.277725 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-c392d33e-ab2e-4bff-8d22-c403e2dfa2e1 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] 10.222.0.16 "GET /compute/v2.1/servers/08a33911-6e47-45fd-b499-155f199aacf8" status: 200 len: 1723 microversion: 2.20 time: 0.512236 Jul 16 21:31:27.277725 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2285/4579] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:31:26 2022] GET /compute/v2.1/servers/08a33911-6e47-45fd-b499-155f199aacf8 => generated 1723 bytes in 514 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:31:27.743516 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-e45446de-b9b2-4890-9154-d444974520cf tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:31:27.744293 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-e45446de-b9b2-4890-9154-d444974520cf tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] [instance: 3aa4786f-20b7-4123-a231-d3d1c3b91b3b] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:31:27.757494 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e45446de-b9b2-4890-9154-d444974520cf tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:27.759549 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e45446de-b9b2-4890-9154-d444974520cf tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:27.761370 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e45446de-b9b2-4890-9154-d444974520cf tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:27.761370 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e45446de-b9b2-4890-9154-d444974520cf tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:27.826572 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-49f599d9-0df4-4604-baab-034dcf616fa9 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:31:27.828256 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-49f599d9-0df4-4604-baab-034dcf616fa9 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] [instance: 4fa0664b-97f0-44ab-9823-a3a92c1bfdb8] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:31:27.845091 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-49f599d9-0df4-4604-baab-034dcf616fa9 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:27.847384 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-49f599d9-0df4-4604-baab-034dcf616fa9 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:27.849389 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-49f599d9-0df4-4604-baab-034dcf616fa9 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:27.852326 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-49f599d9-0df4-4604-baab-034dcf616fa9 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:28.356222 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-e45446de-b9b2-4890-9154-d444974520cf tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1006989e743247bdb29a51b3a634afd9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e8c65e58fac42c4bd7f49eaa3a14115', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:31:28.369233 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-e45446de-b9b2-4890-9154-d444974520cf tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] 10.222.0.16 "GET /compute/v2.1/servers/3aa4786f-20b7-4123-a231-d3d1c3b91b3b" status: 200 len: 1501 microversion: 2.1 time: 0.629312 Jul 16 21:31:28.371268 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-49f599d9-0df4-4604-baab-034dcf616fa9 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da77ec3d0f04496d8c8df4025bba76d1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '207ec2d1883c4a1ba89696e6e77cf3aa', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:31:28.371791 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2295/4580] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:31:27 2022] GET /compute/v2.1/servers/3aa4786f-20b7-4123-a231-d3d1c3b91b3b => generated 1501 bytes in 633 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:31:28.382889 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-4ddd234d-c0d7-4c4a-b961-bcfd70307ce2 req-cd6adc7c-9b25-4a89-b02f-8102c88a27f8 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "08a33911-6e47-45fd-b499-155f199aacf8", "name": "network-vif-deleted", "tag": "81e317e7-e9db-4ba9-b87d-9395b084ead7"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:31:28.387437 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-49f599d9-0df4-4604-baab-034dcf616fa9 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] 10.222.0.16 "GET /compute/v2.1/servers/4fa0664b-97f0-44ab-9823-a3a92c1bfdb8" status: 200 len: 1494 microversion: 2.1 time: 0.572771 Jul 16 21:31:28.388684 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2286/4581] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:31:27 2022] GET /compute/v2.1/servers/4fa0664b-97f0-44ab-9823-a3a92c1bfdb8 => generated 1494 bytes in 575 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:31:28.396336 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-1d984b3c-cd69-4c32-9c3d-ce36cbf5fe32 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:31:28.397764 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-1d984b3c-cd69-4c32-9c3d-ce36cbf5fe32 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] [instance: 08a33911-6e47-45fd-b499-155f199aacf8] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:31:28.403732 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-4ddd234d-c0d7-4c4a-b961-bcfd70307ce2 req-cd6adc7c-9b25-4a89-b02f-8102c88a27f8 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:28.407107 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1d984b3c-cd69-4c32-9c3d-ce36cbf5fe32 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:28.407820 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1d984b3c-cd69-4c32-9c3d-ce36cbf5fe32 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:28.408773 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-4ddd234d-c0d7-4c4a-b961-bcfd70307ce2 req-cd6adc7c-9b25-4a89-b02f-8102c88a27f8 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:28.409167 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1d984b3c-cd69-4c32-9c3d-ce36cbf5fe32 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:28.410240 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1d984b3c-cd69-4c32-9c3d-ce36cbf5fe32 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:28.429234 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-4ddd234d-c0d7-4c4a-b961-bcfd70307ce2 req-cd6adc7c-9b25-4a89-b02f-8102c88a27f8 service nova] Creating event network-vif-deleted:81e317e7-e9db-4ba9-b87d-9395b084ead7 for instance 08a33911-6e47-45fd-b499-155f199aacf8 on nch2-master-1 Jul 16 21:31:28.435760 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-4ddd234d-c0d7-4c4a-b961-bcfd70307ce2 req-cd6adc7c-9b25-4a89-b02f-8102c88a27f8 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.061806 Jul 16 21:31:28.436715 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2296/4582] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:31:28 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 64 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:31:28.444555 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-5a75b5c7-6627-43d3-bc58-37baca08d1c0 tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Action: 'action', calling method: >, body: {"suspend": {}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:31:28.445449 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-5a75b5c7-6627-43d3-bc58-37baca08d1c0 tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] [instance: 3aa4786f-20b7-4123-a231-d3d1c3b91b3b] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:31:28.453585 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5a75b5c7-6627-43d3-bc58-37baca08d1c0 tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:28.454242 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1d984b3c-cd69-4c32-9c3d-ce36cbf5fe32 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:28.454802 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5a75b5c7-6627-43d3-bc58-37baca08d1c0 tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:28.455568 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1d984b3c-cd69-4c32-9c3d-ce36cbf5fe32 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:28.455895 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5a75b5c7-6627-43d3-bc58-37baca08d1c0 tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:28.456481 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5a75b5c7-6627-43d3-bc58-37baca08d1c0 tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:28.457045 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.objects.instance [None req-1d984b3c-cd69-4c32-9c3d-ce36cbf5fe32 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lazy-loading 'fault' on Instance uuid 08a33911-6e47-45fd-b499-155f199aacf8 {{(pid=138680) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:31:28.486688 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.objects.instance [None req-5a75b5c7-6627-43d3-bc58-37baca08d1c0 tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Lazy-loading 'flavor' on Instance uuid 3aa4786f-20b7-4123-a231-d3d1c3b91b3b {{(pid=138681) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:31:28.590341 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-1d984b3c-cd69-4c32-9c3d-ce36cbf5fe32 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:31:28.608328 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-1d984b3c-cd69-4c32-9c3d-ce36cbf5fe32 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:31:28.611211 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-1d984b3c-cd69-4c32-9c3d-ce36cbf5fe32 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:31:28.614982 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-1d984b3c-cd69-4c32-9c3d-ce36cbf5fe32 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] 10.222.0.16 "GET /compute/v2.1/servers/08a33911-6e47-45fd-b499-155f199aacf8" status: 200 len: 1370 microversion: 2.20 time: 0.224729 Jul 16 21:31:28.627143 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2287/4583] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:31:28 2022] GET /compute/v2.1/servers/08a33911-6e47-45fd-b499-155f199aacf8 => generated 1370 bytes in 238 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:31:28.957360 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-5a75b5c7-6627-43d3-bc58-37baca08d1c0 tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] 10.222.0.16 "POST /compute/v2.1/servers/3aa4786f-20b7-4123-a231-d3d1c3b91b3b/action" status: 202 len: 0 microversion: 2.1 time: 0.518868 Jul 16 21:31:28.960561 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2297/4584] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 21:31:28 2022] POST /compute/v2.1/servers/3aa4786f-20b7-4123-a231-d3d1c3b91b3b/action => generated 0 bytes in 523 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 21:31:28.969717 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-1ae0b57f-a279-47e1-a05d-73fbed62eec9 tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:31:28.971328 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-1ae0b57f-a279-47e1-a05d-73fbed62eec9 tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] [instance: 3aa4786f-20b7-4123-a231-d3d1c3b91b3b] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:31:28.995087 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1ae0b57f-a279-47e1-a05d-73fbed62eec9 tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:28.997283 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1ae0b57f-a279-47e1-a05d-73fbed62eec9 tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:29.000053 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1ae0b57f-a279-47e1-a05d-73fbed62eec9 tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:29.011473 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1ae0b57f-a279-47e1-a05d-73fbed62eec9 tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:29.406707 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-b245828d-3b28-41b3-85b0-140f76b5bef9 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:31:29.408291 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-b245828d-3b28-41b3-85b0-140f76b5bef9 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] [instance: 4fa0664b-97f0-44ab-9823-a3a92c1bfdb8] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:31:29.420325 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b245828d-3b28-41b3-85b0-140f76b5bef9 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:29.422802 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b245828d-3b28-41b3-85b0-140f76b5bef9 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:29.423856 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b245828d-3b28-41b3-85b0-140f76b5bef9 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:29.424509 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b245828d-3b28-41b3-85b0-140f76b5bef9 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:29.575777 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-1ae0b57f-a279-47e1-a05d-73fbed62eec9 tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1006989e743247bdb29a51b3a634afd9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e8c65e58fac42c4bd7f49eaa3a14115', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:31:29.581709 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-1ae0b57f-a279-47e1-a05d-73fbed62eec9 tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] 10.222.0.16 "GET /compute/v2.1/servers/3aa4786f-20b7-4123-a231-d3d1c3b91b3b" status: 200 len: 1509 microversion: 2.1 time: 0.617256 Jul 16 21:31:29.582308 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2288/4585] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:31:28 2022] GET /compute/v2.1/servers/3aa4786f-20b7-4123-a231-d3d1c3b91b3b => generated 1509 bytes in 619 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:31:29.649982 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-0e87a753-ff7c-43b4-8544-8acc14709aa3 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:31:29.653152 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-0e87a753-ff7c-43b4-8544-8acc14709aa3 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] [instance: 08a33911-6e47-45fd-b499-155f199aacf8] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:31:29.670431 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0e87a753-ff7c-43b4-8544-8acc14709aa3 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:29.673263 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0e87a753-ff7c-43b4-8544-8acc14709aa3 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:29.679700 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0e87a753-ff7c-43b4-8544-8acc14709aa3 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:29.680438 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0e87a753-ff7c-43b4-8544-8acc14709aa3 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:29.727606 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-0e87a753-ff7c-43b4-8544-8acc14709aa3 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] HTTP exception thrown: Instance 08a33911-6e47-45fd-b499-155f199aacf8 could not be found. Jul 16 21:31:29.731758 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-0e87a753-ff7c-43b4-8544-8acc14709aa3 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Request method failure captured: Jul 16 21:31:29.731758 ncdv-master-1 devstack@n-api.service[138680]: request: GET /compute/v2.1/servers/08a33911-6e47-45fd-b499-155f199aacf8 HTTP/1.1 Jul 16 21:31:29.731758 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:31:29.731758 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:31:29.731758 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:31:29.731758 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 0 Jul 16 21:31:29.731758 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:31:29.731758 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:31:29.731758 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:31:29.731758 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:31:29.731758 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0y1ucaZ1yKJJ3ZclTlid5u9GxaiMbQlzksI7M27rJ0B6rB8jj8I68KlCkH-lQUpdPSiKnnTfAiP8PzCbWLft6KLltxUTtp9U114w8i4pdCqez2cdl7YB-ynlGpqTgM-nH_ZWHIpWAB7cPaNN1ImROhBb6Fvg2kKIxL9C97eioGmF_M4 Jul 16 21:31:29.731758 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:31:29.731758 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:31:29.731758 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:31:29.731758 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:31:29.731758 ncdv-master-1 devstack@n-api.service[138680]: X-Openstack-Nova-Api-Version: 2.20 Jul 16 21:31:29.731758 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:31:29.731758 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:31:29.731758 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: e451ee7bfd2d4ab58d30c33fef334b55 Jul 16 21:31:29.731758 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-2044012735 Jul 16 21:31:29.731758 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:31:29.731758 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:31:29.731758 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/e451ee7bfd2d4ab58d30c33fef334b55"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/e451ee7bfd2d4ab58d30c33fef334b55"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_e451ee7bfd2d4ab58d30c33fef334b55"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/e451ee7bfd2d4ab58d30c33fef334b55"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:31:29.731758 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-AttachVolumeShelveTestJSON-2044012735 Jul 16 21:31:29.731758 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: e451ee7bfd2d4ab58d30c33fef334b55 Jul 16 21:31:29.731758 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-2044012735 Jul 16 21:31:29.731758 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-AttachVolumeShelveTestJSON-2044012735-project Jul 16 21:31:29.731758 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:31:29.731758 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:31:29.731758 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: ca2c9e7e6d61414cbf2c7e17753bc0df Jul 16 21:31:29.731758 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-AttachVolumeShelveTestJSON-2044012735-project Jul 16 21:31:29.731758 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 21:31:29.731758 ncdv-master-1 devstack@n-api.service[138680]: exception: Instance 08a33911-6e47-45fd-b499-155f199aacf8 could not be found. Jul 16 21:31:29.731758 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': '08a33911-6e47-45fd-b499-155f199aacf8'} Jul 16 21:31:29.731758 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:31:29.731758 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:31:29.731758 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:31:29.731758 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:31:29.731758 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:31:29.731758 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:31:29.731758 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:31:29.731758 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:31:29.731758 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:31:29.731758 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:31:29.731758 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 08a33911-6e47-45fd-b499-155f199aacf8 could not be found. Jul 16 21:31:29.731758 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:31:29.731758 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:31:29.731758 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:31:29.731758 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:31:29.731758 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:31:29.731758 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:31:29.731758 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:31:29.731758 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:31:29.731758 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:31:29.731758 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:31:29.731758 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:31:29.731758 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:31:29.731758 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:31:29.731758 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:31:29.731758 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:31:29.731758 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:31:29.731758 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 08a33911-6e47-45fd-b499-155f199aacf8 could not be found. Jul 16 21:31:29.731758 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:31:29.731758 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:31:29.738657 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:31:29.738657 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:31:29.738657 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:31:29.738657 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:31:29.738657 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:31:29.738657 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:31:29.738657 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 08a33911-6e47-45fd-b499-155f199aacf8 could not be found. Jul 16 21:31:29.738657 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:31:29.738657 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-0e87a753-ff7c-43b4-8544-8acc14709aa3 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Returning 404 to user: Instance 08a33911-6e47-45fd-b499-155f199aacf8 could not be found. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:31:29.738657 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-0e87a753-ff7c-43b4-8544-8acc14709aa3 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] 10.222.0.16 "GET /compute/v2.1/servers/08a33911-6e47-45fd-b499-155f199aacf8" status: 404 len: 111 microversion: 2.20 time: 0.088823 Jul 16 21:31:29.747384 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2289/4586] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:31:29 2022] GET /compute/v2.1/servers/08a33911-6e47-45fd-b499-155f199aacf8 => generated 111 bytes in 103 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Jul 16 21:31:29.755049 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-4933efaf-b231-40d6-9c1e-5f74a467a1ab tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Calling method '.version_select at 0x7fdd9d07aaf0>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:31:29.791214 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_api:SELECT=97,nova_api:UPDATE=3 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:31:29.878967 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-4933efaf-b231-40d6-9c1e-5f74a467a1ab tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] 10.222.0.16 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-545593029" status: 204 len: 0 microversion: 2.20 time: 0.130251 Jul 16 21:31:29.880035 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2290/4587] 10.222.0.16 () {60 vars in 1258 bytes} [Sat Jul 16 21:31:29 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-545593029 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Jul 16 21:31:30.013760 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-b245828d-3b28-41b3-85b0-140f76b5bef9 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da77ec3d0f04496d8c8df4025bba76d1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '207ec2d1883c4a1ba89696e6e77cf3aa', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:31:30.046779 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-b245828d-3b28-41b3-85b0-140f76b5bef9 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] 10.222.0.16 "GET /compute/v2.1/servers/4fa0664b-97f0-44ab-9823-a3a92c1bfdb8" status: 200 len: 1500 microversion: 2.1 time: 0.644016 Jul 16 21:31:30.047458 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2298/4588] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:31:29 2022] GET /compute/v2.1/servers/4fa0664b-97f0-44ab-9823-a3a92c1bfdb8 => generated 1500 bytes in 646 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:31:30.082718 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-90971e08-60ce-4b95-ba93-3c83a8b2cb72 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:31:30.083365 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-90971e08-60ce-4b95-ba93-3c83a8b2cb72 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] [instance: 4fa0664b-97f0-44ab-9823-a3a92c1bfdb8] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:31:30.100201 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-90971e08-60ce-4b95-ba93-3c83a8b2cb72 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:30.100887 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-90971e08-60ce-4b95-ba93-3c83a8b2cb72 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:30.101955 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-90971e08-60ce-4b95-ba93-3c83a8b2cb72 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:30.102798 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-90971e08-60ce-4b95-ba93-3c83a8b2cb72 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:30.131534 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-90971e08-60ce-4b95-ba93-3c83a8b2cb72 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] [instance: 4fa0664b-97f0-44ab-9823-a3a92c1bfdb8] Going to try to terminate instance {{(pid=138680) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 16 21:31:30.602012 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-90971e08-60ce-4b95-ba93-3c83a8b2cb72 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] 10.222.0.16 "DELETE /compute/v2.1/servers/4fa0664b-97f0-44ab-9823-a3a92c1bfdb8" status: 204 len: 0 microversion: 2.1 time: 0.535593 Jul 16 21:31:30.603170 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2291/4589] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:31:30 2022] DELETE /compute/v2.1/servers/4fa0664b-97f0-44ab-9823-a3a92c1bfdb8 => generated 0 bytes in 538 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 16 21:31:30.607766 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-3e50a5ab-5c33-458e-80e1-b0c36ea67ba0 tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:31:30.608342 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-3e50a5ab-5c33-458e-80e1-b0c36ea67ba0 tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] [instance: 3aa4786f-20b7-4123-a231-d3d1c3b91b3b] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:31:30.614264 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-0022ff5b-f89f-40cc-99c8-52e9e0864ea9 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:31:30.614994 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-0022ff5b-f89f-40cc-99c8-52e9e0864ea9 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] [instance: 4fa0664b-97f0-44ab-9823-a3a92c1bfdb8] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:31:30.618313 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3e50a5ab-5c33-458e-80e1-b0c36ea67ba0 tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:30.618842 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3e50a5ab-5c33-458e-80e1-b0c36ea67ba0 tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:30.619863 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3e50a5ab-5c33-458e-80e1-b0c36ea67ba0 tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:30.621471 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3e50a5ab-5c33-458e-80e1-b0c36ea67ba0 tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:30.629111 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0022ff5b-f89f-40cc-99c8-52e9e0864ea9 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:30.629588 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0022ff5b-f89f-40cc-99c8-52e9e0864ea9 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:30.630404 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0022ff5b-f89f-40cc-99c8-52e9e0864ea9 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:30.635833 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0022ff5b-f89f-40cc-99c8-52e9e0864ea9 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:30.641999 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=94,nova_cell1:UPDATE=3,nova_cell1:INSERT=3 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:31:31.416314 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-3e50a5ab-5c33-458e-80e1-b0c36ea67ba0 tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1006989e743247bdb29a51b3a634afd9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e8c65e58fac42c4bd7f49eaa3a14115', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:31:31.419962 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-0022ff5b-f89f-40cc-99c8-52e9e0864ea9 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da77ec3d0f04496d8c8df4025bba76d1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '207ec2d1883c4a1ba89696e6e77cf3aa', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:31:31.433980 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-0022ff5b-f89f-40cc-99c8-52e9e0864ea9 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] 10.222.0.16 "GET /compute/v2.1/servers/4fa0664b-97f0-44ab-9823-a3a92c1bfdb8" status: 200 len: 1506 microversion: 2.1 time: 0.826147 Jul 16 21:31:31.440262 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2292/4590] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:31:30 2022] GET /compute/v2.1/servers/4fa0664b-97f0-44ab-9823-a3a92c1bfdb8 => generated 1506 bytes in 828 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:31:31.440396 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-3e50a5ab-5c33-458e-80e1-b0c36ea67ba0 tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] 10.222.0.16 "GET /compute/v2.1/servers/3aa4786f-20b7-4123-a231-d3d1c3b91b3b" status: 200 len: 1492 microversion: 2.1 time: 0.838526 Jul 16 21:31:31.440396 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2299/4591] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:31:30 2022] GET /compute/v2.1/servers/3aa4786f-20b7-4123-a231-d3d1c3b91b3b => generated 1492 bytes in 841 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:31:31.455717 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-c74375ca-e209-4390-a7c2-5be8f92e51ef tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Action: 'action', calling method: >, body: {"resume": {}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:31:31.456324 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-c74375ca-e209-4390-a7c2-5be8f92e51ef tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] [instance: 3aa4786f-20b7-4123-a231-d3d1c3b91b3b] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:31:31.467054 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c74375ca-e209-4390-a7c2-5be8f92e51ef tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:31.467636 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c74375ca-e209-4390-a7c2-5be8f92e51ef tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:31.468864 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c74375ca-e209-4390-a7c2-5be8f92e51ef tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:31.469875 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c74375ca-e209-4390-a7c2-5be8f92e51ef tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:31.767124 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-c74375ca-e209-4390-a7c2-5be8f92e51ef tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] 10.222.0.16 "POST /compute/v2.1/servers/3aa4786f-20b7-4123-a231-d3d1c3b91b3b/action" status: 202 len: 0 microversion: 2.1 time: 0.317140 Jul 16 21:31:31.767885 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2293/4592] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 21:31:31 2022] POST /compute/v2.1/servers/3aa4786f-20b7-4123-a231-d3d1c3b91b3b/action => generated 0 bytes in 319 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 21:31:31.786563 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-1428887e-d4a6-4c50-97ea-dbe039d1d8b9 tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:31:31.788036 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-1428887e-d4a6-4c50-97ea-dbe039d1d8b9 tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] [instance: 3aa4786f-20b7-4123-a231-d3d1c3b91b3b] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:31:31.800439 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1428887e-d4a6-4c50-97ea-dbe039d1d8b9 tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:31.804456 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1428887e-d4a6-4c50-97ea-dbe039d1d8b9 tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:31.806514 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1428887e-d4a6-4c50-97ea-dbe039d1d8b9 tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:31.807185 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1428887e-d4a6-4c50-97ea-dbe039d1d8b9 tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:31.910850 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-e4894eca-5414-439b-bd80-86c1c56bc2e4 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Action: 'create', calling method: .version_select at 0x7fdd9c528b80>, body: {"keypair": {"name": "tempest-keypair-1464370033", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBOYigN4j6bz0IP7REpfITC5V16+gWkg98IR/f6DwgQT8tmnx+4CmpO8P7AaI5q9c9aQt8dV9Oer/R2E1rdqkUfWeNwORHHMHXr03Jj4Bg/gXSSkin/G/y8Hudvt/wv2EzQ=="}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:31:31.953345 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-e4894eca-5414-439b-bd80-86c1c56bc2e4 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Getting quotas for project e451ee7bfd2d4ab58d30c33fef334b55. Resources: {'key_pairs'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:31:31.977010 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-e4894eca-5414-439b-bd80-86c1c56bc2e4 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Getting quotas for user ca2c9e7e6d61414cbf2c7e17753bc0df and project e451ee7bfd2d4ab58d30c33fef334b55. Resources: {'key_pairs'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:31:32.172304 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-e4894eca-5414-439b-bd80-86c1c56bc2e4 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] 10.222.0.16 "POST /compute/v2.1/os-keypairs" status: 201 len: 401 microversion: 2.20 time: 0.267614 Jul 16 21:31:32.178374 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2294/4593] 10.222.0.16 () {62 vars in 1199 bytes} [Sat Jul 16 21:31:31 2022] POST /compute/v2.1/os-keypairs => generated 401 bytes in 274 msecs (HTTP/1.1 201) 9 headers in 364 bytes (2 switches on core 0) Jul 16 21:31:32.251903 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-1428887e-d4a6-4c50-97ea-dbe039d1d8b9 tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1006989e743247bdb29a51b3a634afd9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e8c65e58fac42c4bd7f49eaa3a14115', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:31:32.261745 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-1428887e-d4a6-4c50-97ea-dbe039d1d8b9 tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] 10.222.0.16 "GET /compute/v2.1/servers/3aa4786f-20b7-4123-a231-d3d1c3b91b3b" status: 200 len: 1498 microversion: 2.1 time: 0.488570 Jul 16 21:31:32.263534 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2300/4594] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:31:31 2022] GET /compute/v2.1/servers/3aa4786f-20b7-4123-a231-d3d1c3b91b3b => generated 1498 bytes in 492 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:31:32.451632 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-9e3c0a71-5bc0-4b34-ab66-6ed2bd9f935c tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:31:32.452384 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-9e3c0a71-5bc0-4b34-ab66-6ed2bd9f935c tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] [instance: 4fa0664b-97f0-44ab-9823-a3a92c1bfdb8] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:31:32.461574 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9e3c0a71-5bc0-4b34-ab66-6ed2bd9f935c tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:32.462386 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9e3c0a71-5bc0-4b34-ab66-6ed2bd9f935c tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:32.463238 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9e3c0a71-5bc0-4b34-ab66-6ed2bd9f935c tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:32.463693 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9e3c0a71-5bc0-4b34-ab66-6ed2bd9f935c tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:32.857565 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-9e3c0a71-5bc0-4b34-ab66-6ed2bd9f935c tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da77ec3d0f04496d8c8df4025bba76d1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '207ec2d1883c4a1ba89696e6e77cf3aa', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:31:32.870877 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-9e3c0a71-5bc0-4b34-ab66-6ed2bd9f935c tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] 10.222.0.16 "GET /compute/v2.1/servers/4fa0664b-97f0-44ab-9823-a3a92c1bfdb8" status: 200 len: 1506 microversion: 2.1 time: 0.423352 Jul 16 21:31:32.871692 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2295/4595] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:31:32 2022] GET /compute/v2.1/servers/4fa0664b-97f0-44ab-9823-a3a92c1bfdb8 => generated 1506 bytes in 425 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:31:33.280446 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-2487d0fe-9ccb-4f42-87ea-5b1fc18089d2 tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:31:33.281690 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-2487d0fe-9ccb-4f42-87ea-5b1fc18089d2 tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] [instance: 3aa4786f-20b7-4123-a231-d3d1c3b91b3b] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:31:33.301049 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2487d0fe-9ccb-4f42-87ea-5b1fc18089d2 tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:33.301609 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2487d0fe-9ccb-4f42-87ea-5b1fc18089d2 tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:33.303059 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2487d0fe-9ccb-4f42-87ea-5b1fc18089d2 tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:33.303601 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2487d0fe-9ccb-4f42-87ea-5b1fc18089d2 tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:33.884334 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-2487d0fe-9ccb-4f42-87ea-5b1fc18089d2 tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1006989e743247bdb29a51b3a634afd9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e8c65e58fac42c4bd7f49eaa3a14115', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:31:33.897558 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-7d4520e7-a9d8-46d9-9d4c-b0a9765ff218 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:31:33.898957 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-7d4520e7-a9d8-46d9-9d4c-b0a9765ff218 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] [instance: 4fa0664b-97f0-44ab-9823-a3a92c1bfdb8] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:31:33.909742 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7d4520e7-a9d8-46d9-9d4c-b0a9765ff218 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:33.911871 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7d4520e7-a9d8-46d9-9d4c-b0a9765ff218 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:33.912118 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-2487d0fe-9ccb-4f42-87ea-5b1fc18089d2 tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] 10.222.0.16 "GET /compute/v2.1/servers/3aa4786f-20b7-4123-a231-d3d1c3b91b3b" status: 200 len: 1498 microversion: 2.1 time: 0.631392 Jul 16 21:31:33.918312 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7d4520e7-a9d8-46d9-9d4c-b0a9765ff218 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:33.920894 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7d4520e7-a9d8-46d9-9d4c-b0a9765ff218 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:33.934853 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2301/4596] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:31:33 2022] GET /compute/v2.1/servers/3aa4786f-20b7-4123-a231-d3d1c3b91b3b => generated 1498 bytes in 659 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:31:34.398282 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-7d4520e7-a9d8-46d9-9d4c-b0a9765ff218 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da77ec3d0f04496d8c8df4025bba76d1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '207ec2d1883c4a1ba89696e6e77cf3aa', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:31:34.406478 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-7d4520e7-a9d8-46d9-9d4c-b0a9765ff218 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] 10.222.0.16 "GET /compute/v2.1/servers/4fa0664b-97f0-44ab-9823-a3a92c1bfdb8" status: 200 len: 1506 microversion: 2.1 time: 0.516432 Jul 16 21:31:34.417185 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2296/4597] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:31:33 2022] GET /compute/v2.1/servers/4fa0664b-97f0-44ab-9823-a3a92c1bfdb8 => generated 1506 bytes in 528 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:31:34.932420 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-c95681aa-ee86-434b-a270-c07ff337dc53 tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:31:34.934320 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-c95681aa-ee86-434b-a270-c07ff337dc53 tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] [instance: 3aa4786f-20b7-4123-a231-d3d1c3b91b3b] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:31:34.949125 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c95681aa-ee86-434b-a270-c07ff337dc53 tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:34.953024 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c95681aa-ee86-434b-a270-c07ff337dc53 tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:34.959513 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c95681aa-ee86-434b-a270-c07ff337dc53 tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:34.960044 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c95681aa-ee86-434b-a270-c07ff337dc53 tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:35.259692 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-c95681aa-ee86-434b-a270-c07ff337dc53 tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1006989e743247bdb29a51b3a634afd9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e8c65e58fac42c4bd7f49eaa3a14115', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:31:35.267788 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-c95681aa-ee86-434b-a270-c07ff337dc53 tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] 10.222.0.16 "GET /compute/v2.1/servers/3aa4786f-20b7-4123-a231-d3d1c3b91b3b" status: 200 len: 1501 microversion: 2.1 time: 0.342408 Jul 16 21:31:35.276792 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2302/4598] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:31:34 2022] GET /compute/v2.1/servers/3aa4786f-20b7-4123-a231-d3d1c3b91b3b => generated 1501 bytes in 352 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:31:35.295127 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-32f167f2-98ad-4037-b82c-8c8d7c98bdb6 tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:31:35.296939 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-32f167f2-98ad-4037-b82c-8c8d7c98bdb6 tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] [instance: 3aa4786f-20b7-4123-a231-d3d1c3b91b3b] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:31:35.331747 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-32f167f2-98ad-4037-b82c-8c8d7c98bdb6 tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:35.335359 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-32f167f2-98ad-4037-b82c-8c8d7c98bdb6 tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:35.336474 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-32f167f2-98ad-4037-b82c-8c8d7c98bdb6 tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:35.339386 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-32f167f2-98ad-4037-b82c-8c8d7c98bdb6 tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:35.381690 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-32f167f2-98ad-4037-b82c-8c8d7c98bdb6 tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] [instance: 3aa4786f-20b7-4123-a231-d3d1c3b91b3b] Going to try to terminate instance {{(pid=138680) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 16 21:31:35.431807 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-88f31a62-a98a-4f4e-b7a4-c6c05bcefa96 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:31:35.433220 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-88f31a62-a98a-4f4e-b7a4-c6c05bcefa96 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] [instance: 4fa0664b-97f0-44ab-9823-a3a92c1bfdb8] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:31:35.445314 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-88f31a62-a98a-4f4e-b7a4-c6c05bcefa96 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:35.446929 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-88f31a62-a98a-4f4e-b7a4-c6c05bcefa96 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:35.449585 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-88f31a62-a98a-4f4e-b7a4-c6c05bcefa96 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:35.450415 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-88f31a62-a98a-4f4e-b7a4-c6c05bcefa96 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:35.652584 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-88f31a62-a98a-4f4e-b7a4-c6c05bcefa96 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da77ec3d0f04496d8c8df4025bba76d1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '207ec2d1883c4a1ba89696e6e77cf3aa', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:31:35.668036 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-88f31a62-a98a-4f4e-b7a4-c6c05bcefa96 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] 10.222.0.16 "GET /compute/v2.1/servers/4fa0664b-97f0-44ab-9823-a3a92c1bfdb8" status: 200 len: 1464 microversion: 2.1 time: 0.247470 Jul 16 21:31:35.678592 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2303/4599] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:31:35 2022] GET /compute/v2.1/servers/4fa0664b-97f0-44ab-9823-a3a92c1bfdb8 => generated 1464 bytes in 259 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:31:35.689362 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-c6905715-7205-4cd7-a0e0-664504ca5acd req-081dda25-a1fe-4aa7-81d1-75687ba07b51 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "4fa0664b-97f0-44ab-9823-a3a92c1bfdb8", "name": "network-vif-deleted", "tag": "cb0aa507-e5a6-4b6b-975c-b074af9c5bd6"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:31:35.714690 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-c6905715-7205-4cd7-a0e0-664504ca5acd req-081dda25-a1fe-4aa7-81d1-75687ba07b51 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:35.715495 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-c6905715-7205-4cd7-a0e0-664504ca5acd req-081dda25-a1fe-4aa7-81d1-75687ba07b51 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:35.745293 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-c6905715-7205-4cd7-a0e0-664504ca5acd req-081dda25-a1fe-4aa7-81d1-75687ba07b51 service nova] Creating event network-vif-deleted:cb0aa507-e5a6-4b6b-975c-b074af9c5bd6 for instance 4fa0664b-97f0-44ab-9823-a3a92c1bfdb8 on nch1-master-1 Jul 16 21:31:35.756473 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-c6905715-7205-4cd7-a0e0-664504ca5acd req-081dda25-a1fe-4aa7-81d1-75687ba07b51 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.073350 Jul 16 21:31:35.761307 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2304/4600] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:31:35 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 78 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:31:35.939389 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-32f167f2-98ad-4037-b82c-8c8d7c98bdb6 tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] 10.222.0.16 "DELETE /compute/v2.1/servers/3aa4786f-20b7-4123-a231-d3d1c3b91b3b" status: 204 len: 0 microversion: 2.1 time: 0.638743 Jul 16 21:31:35.939389 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2297/4601] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:31:35 2022] DELETE /compute/v2.1/servers/3aa4786f-20b7-4123-a231-d3d1c3b91b3b => generated 0 bytes in 645 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 16 21:31:35.954012 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-c3b87628-dca7-4422-8ce3-0a6b76fd2281 tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:31:35.957509 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-c3b87628-dca7-4422-8ce3-0a6b76fd2281 tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] [instance: 3aa4786f-20b7-4123-a231-d3d1c3b91b3b] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:31:35.981070 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c3b87628-dca7-4422-8ce3-0a6b76fd2281 tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:35.982563 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c3b87628-dca7-4422-8ce3-0a6b76fd2281 tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:35.986306 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c3b87628-dca7-4422-8ce3-0a6b76fd2281 tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:35.986988 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c3b87628-dca7-4422-8ce3-0a6b76fd2281 tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:36.020158 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=95,nova_cell1:UPDATE=3,nova_cell1:INSERT=2 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:31:36.535401 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-c3b87628-dca7-4422-8ce3-0a6b76fd2281 tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1006989e743247bdb29a51b3a634afd9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e8c65e58fac42c4bd7f49eaa3a14115', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:31:36.541499 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-c3b87628-dca7-4422-8ce3-0a6b76fd2281 tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] 10.222.0.16 "GET /compute/v2.1/servers/3aa4786f-20b7-4123-a231-d3d1c3b91b3b" status: 200 len: 1507 microversion: 2.1 time: 0.600441 Jul 16 21:31:36.542100 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2305/4602] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:31:35 2022] GET /compute/v2.1/servers/3aa4786f-20b7-4123-a231-d3d1c3b91b3b => generated 1507 bytes in 602 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:31:36.688068 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-73501d84-8fdc-48e4-9041-7c86683c2f97 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:31:36.690048 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-73501d84-8fdc-48e4-9041-7c86683c2f97 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] [instance: 4fa0664b-97f0-44ab-9823-a3a92c1bfdb8] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:31:36.709940 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-73501d84-8fdc-48e4-9041-7c86683c2f97 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:36.711482 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-73501d84-8fdc-48e4-9041-7c86683c2f97 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:36.716473 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-73501d84-8fdc-48e4-9041-7c86683c2f97 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:36.716779 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-73501d84-8fdc-48e4-9041-7c86683c2f97 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:36.781430 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-73501d84-8fdc-48e4-9041-7c86683c2f97 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:36.782376 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-73501d84-8fdc-48e4-9041-7c86683c2f97 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:36.784217 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.objects.instance [None req-73501d84-8fdc-48e4-9041-7c86683c2f97 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lazy-loading 'fault' on Instance uuid 4fa0664b-97f0-44ab-9823-a3a92c1bfdb8 {{(pid=138680) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:31:36.923954 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-73501d84-8fdc-48e4-9041-7c86683c2f97 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da77ec3d0f04496d8c8df4025bba76d1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '207ec2d1883c4a1ba89696e6e77cf3aa', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:31:36.940999 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-73501d84-8fdc-48e4-9041-7c86683c2f97 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] 10.222.0.16 "GET /compute/v2.1/servers/4fa0664b-97f0-44ab-9823-a3a92c1bfdb8" status: 200 len: 1304 microversion: 2.1 time: 0.256371 Jul 16 21:31:36.949122 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2298/4603] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:31:36 2022] GET /compute/v2.1/servers/4fa0664b-97f0-44ab-9823-a3a92c1bfdb8 => generated 1304 bytes in 261 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:31:37.567352 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-54e89f60-520d-4e99-a3af-3c62bbeeb6ab tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:31:37.587696 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-54e89f60-520d-4e99-a3af-3c62bbeeb6ab tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] [instance: 3aa4786f-20b7-4123-a231-d3d1c3b91b3b] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:31:37.642967 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-54e89f60-520d-4e99-a3af-3c62bbeeb6ab tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:37.643496 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-54e89f60-520d-4e99-a3af-3c62bbeeb6ab tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:37.653848 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-54e89f60-520d-4e99-a3af-3c62bbeeb6ab tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:37.654938 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-54e89f60-520d-4e99-a3af-3c62bbeeb6ab tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:37.961097 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-a7f85ab1-b7dc-40c8-a0af-eba5d0dca195 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:31:37.961746 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-a7f85ab1-b7dc-40c8-a0af-eba5d0dca195 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] [instance: 4fa0664b-97f0-44ab-9823-a3a92c1bfdb8] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:31:37.983700 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a7f85ab1-b7dc-40c8-a0af-eba5d0dca195 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:37.985729 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a7f85ab1-b7dc-40c8-a0af-eba5d0dca195 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:37.989412 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a7f85ab1-b7dc-40c8-a0af-eba5d0dca195 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:37.991379 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a7f85ab1-b7dc-40c8-a0af-eba5d0dca195 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:38.028769 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-a7f85ab1-b7dc-40c8-a0af-eba5d0dca195 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] HTTP exception thrown: Instance 4fa0664b-97f0-44ab-9823-a3a92c1bfdb8 could not be found. Jul 16 21:31:38.034856 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-a7f85ab1-b7dc-40c8-a0af-eba5d0dca195 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Request method failure captured: Jul 16 21:31:38.034856 ncdv-master-1 devstack@n-api.service[138680]: request: GET /compute/v2.1/servers/4fa0664b-97f0-44ab-9823-a3a92c1bfdb8 HTTP/1.1 Jul 16 21:31:38.034856 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:31:38.034856 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:31:38.034856 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:31:38.034856 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 0 Jul 16 21:31:38.034856 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:31:38.034856 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:31:38.034856 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:31:38.034856 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:31:38.034856 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0y3FP3QHjhX3GJLGfDYK8yH2ipiwfxhuYVl72Yq3zCWqktw2oRz0Vp6uBJ43tQfjrJRsrBMy9qlPSpZpq79cCogyPwRKOOhs7ji6KNmqfVGRI8zyZDEnK4Uzqiwhpt1K21cJh5Tr6hBPIf-oCch2kUReB0byV1m5CleH4ahZz4ST7GM Jul 16 21:31:38.034856 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:31:38.034856 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:31:38.034856 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:31:38.034856 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:31:38.034856 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:31:38.034856 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:31:38.034856 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: 207ec2d1883c4a1ba89696e6e77cf3aa Jul 16 21:31:38.034856 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-ServerRescueTestJSON-975252187 Jul 16 21:31:38.034856 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:31:38.034856 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:31:38.034856 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/207ec2d1883c4a1ba89696e6e77cf3aa"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/207ec2d1883c4a1ba89696e6e77cf3aa"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_207ec2d1883c4a1ba89696e6e77cf3aa"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/207ec2d1883c4a1ba89696e6e77cf3aa"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:31:38.034856 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-ServerRescueTestJSON-975252187 Jul 16 21:31:38.034856 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: 207ec2d1883c4a1ba89696e6e77cf3aa Jul 16 21:31:38.034856 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-ServerRescueTestJSON-975252187 Jul 16 21:31:38.034856 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-ServerRescueTestJSON-975252187-project Jul 16 21:31:38.034856 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:31:38.034856 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:31:38.034856 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: da77ec3d0f04496d8c8df4025bba76d1 Jul 16 21:31:38.034856 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-ServerRescueTestJSON-975252187-project Jul 16 21:31:38.034856 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 21:31:38.034856 ncdv-master-1 devstack@n-api.service[138680]: exception: Instance 4fa0664b-97f0-44ab-9823-a3a92c1bfdb8 could not be found. Jul 16 21:31:38.034856 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': '4fa0664b-97f0-44ab-9823-a3a92c1bfdb8'} Jul 16 21:31:38.034856 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:31:38.034856 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:31:38.034856 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:31:38.034856 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:31:38.034856 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:31:38.034856 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:31:38.034856 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:31:38.034856 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:31:38.034856 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:31:38.034856 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:31:38.034856 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 4fa0664b-97f0-44ab-9823-a3a92c1bfdb8 could not be found. Jul 16 21:31:38.034856 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:31:38.034856 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:31:38.034856 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:31:38.034856 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:31:38.034856 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:31:38.034856 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:31:38.034856 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:31:38.034856 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:31:38.034856 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:31:38.034856 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:31:38.034856 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:31:38.034856 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:31:38.034856 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:31:38.034856 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:31:38.034856 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:31:38.034856 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:31:38.034856 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 4fa0664b-97f0-44ab-9823-a3a92c1bfdb8 could not be found. Jul 16 21:31:38.034856 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:31:38.034856 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:31:38.034856 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:31:38.042632 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:31:38.042632 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:31:38.042632 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:31:38.042632 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:31:38.042632 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:31:38.042632 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 4fa0664b-97f0-44ab-9823-a3a92c1bfdb8 could not be found. Jul 16 21:31:38.042632 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:31:38.047253 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-a7f85ab1-b7dc-40c8-a0af-eba5d0dca195 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Returning 404 to user: Instance 4fa0664b-97f0-44ab-9823-a3a92c1bfdb8 could not be found. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:31:38.051250 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-a7f85ab1-b7dc-40c8-a0af-eba5d0dca195 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] 10.222.0.16 "GET /compute/v2.1/servers/4fa0664b-97f0-44ab-9823-a3a92c1bfdb8" status: 404 len: 111 microversion: 2.1 time: 0.093717 Jul 16 21:31:38.054787 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2299/4604] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:31:37 2022] GET /compute/v2.1/servers/4fa0664b-97f0-44ab-9823-a3a92c1bfdb8 => generated 111 bytes in 100 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Jul 16 21:31:38.071464 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-50a7ed5d-eef1-4e4a-b04a-372f571f2d7c tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:31:38.074371 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-50a7ed5d-eef1-4e4a-b04a-372f571f2d7c tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] [instance: 515fd3da-3965-4882-b9e8-100a7eea64d9] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:31:38.101525 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-50a7ed5d-eef1-4e4a-b04a-372f571f2d7c tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:38.102077 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-50a7ed5d-eef1-4e4a-b04a-372f571f2d7c tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:38.102822 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-50a7ed5d-eef1-4e4a-b04a-372f571f2d7c tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:38.103295 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-50a7ed5d-eef1-4e4a-b04a-372f571f2d7c tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:38.145889 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-54e89f60-520d-4e99-a3af-3c62bbeeb6ab tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1006989e743247bdb29a51b3a634afd9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e8c65e58fac42c4bd7f49eaa3a14115', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:31:38.161411 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-54e89f60-520d-4e99-a3af-3c62bbeeb6ab tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] 10.222.0.16 "GET /compute/v2.1/servers/3aa4786f-20b7-4123-a231-d3d1c3b91b3b" status: 200 len: 1507 microversion: 2.1 time: 0.605198 Jul 16 21:31:38.169426 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-50a7ed5d-eef1-4e4a-b04a-372f571f2d7c tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] [instance: 515fd3da-3965-4882-b9e8-100a7eea64d9] Going to try to terminate instance {{(pid=138680) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 16 21:31:38.185462 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2306/4605] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:31:37 2022] GET /compute/v2.1/servers/3aa4786f-20b7-4123-a231-d3d1c3b91b3b => generated 1507 bytes in 630 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:31:38.524775 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-105a4e1f-f7d0-40a7-b203-ca77d97960f8 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeShelveTestJSON-server-1591514188", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "051a9df4-f4db-45ac-8063-879ec2c58801"}], "security_groups": [{"name": "tempest-securitygroup--76517446"}], "key_name": "tempest-keypair-1464370033", "user_data": "}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:31:38.548847 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_api:SELECT=78,nova_api:DELETE=2 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:31:38.625118 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-50a7ed5d-eef1-4e4a-b04a-372f571f2d7c tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] 10.222.0.16 "DELETE /compute/v2.1/servers/515fd3da-3965-4882-b9e8-100a7eea64d9" status: 204 len: 0 microversion: 2.1 time: 0.560933 Jul 16 21:31:38.625118 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2300/4606] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:31:38 2022] DELETE /compute/v2.1/servers/515fd3da-3965-4882-b9e8-100a7eea64d9 => generated 0 bytes in 564 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 16 21:31:38.632625 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-3f9969d0-407c-4879-a571-e663a51f58f1 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:31:38.633404 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-3f9969d0-407c-4879-a571-e663a51f58f1 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] [instance: 515fd3da-3965-4882-b9e8-100a7eea64d9] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:31:38.649603 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3f9969d0-407c-4879-a571-e663a51f58f1 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:38.650357 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3f9969d0-407c-4879-a571-e663a51f58f1 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:38.651345 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3f9969d0-407c-4879-a571-e663a51f58f1 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:38.651784 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3f9969d0-407c-4879-a571-e663a51f58f1 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:39.037481 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.network.neutron [None req-105a4e1f-f7d0-40a7-b203-ca77d97960f8 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] validate_networks() for [('051a9df4-f4db-45ac-8063-879ec2c58801', None, None, None, None, None)] {{(pid=138681) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 16 21:31:39.241326 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-3f9969d0-407c-4879-a571-e663a51f58f1 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da77ec3d0f04496d8c8df4025bba76d1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '207ec2d1883c4a1ba89696e6e77cf3aa', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:31:39.258318 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-3f9969d0-407c-4879-a571-e663a51f58f1 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] 10.222.0.16 "GET /compute/v2.1/servers/515fd3da-3965-4882-b9e8-100a7eea64d9" status: 200 len: 1492 microversion: 2.1 time: 0.630645 Jul 16 21:31:39.271843 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2301/4607] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:31:38 2022] GET /compute/v2.1/servers/515fd3da-3965-4882-b9e8-100a7eea64d9 => generated 1492 bytes in 645 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:31:39.280164 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-efa51b80-9428-45e4-8606-ee556657378b tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:31:39.282225 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-efa51b80-9428-45e4-8606-ee556657378b tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] [instance: 3aa4786f-20b7-4123-a231-d3d1c3b91b3b] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:31:39.298445 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-efa51b80-9428-45e4-8606-ee556657378b tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:39.298445 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-efa51b80-9428-45e4-8606-ee556657378b tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:39.298445 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-efa51b80-9428-45e4-8606-ee556657378b tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:39.298445 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-efa51b80-9428-45e4-8606-ee556657378b tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:40.084569 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-efa51b80-9428-45e4-8606-ee556657378b tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1006989e743247bdb29a51b3a634afd9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e8c65e58fac42c4bd7f49eaa3a14115', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:31:40.102785 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-efa51b80-9428-45e4-8606-ee556657378b tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] 10.222.0.16 "GET /compute/v2.1/servers/3aa4786f-20b7-4123-a231-d3d1c3b91b3b" status: 200 len: 1507 microversion: 2.1 time: 0.828579 Jul 16 21:31:40.106037 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2302/4608] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:31:39 2022] GET /compute/v2.1/servers/3aa4786f-20b7-4123-a231-d3d1c3b91b3b => generated 1507 bytes in 832 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:31:40.278626 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-4f3adb2d-2ba9-43d2-ae49-08a5f99401fa tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:31:40.284929 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-4f3adb2d-2ba9-43d2-ae49-08a5f99401fa tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] [instance: 515fd3da-3965-4882-b9e8-100a7eea64d9] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:31:40.307580 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4f3adb2d-2ba9-43d2-ae49-08a5f99401fa tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:40.308409 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4f3adb2d-2ba9-43d2-ae49-08a5f99401fa tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:40.309381 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4f3adb2d-2ba9-43d2-ae49-08a5f99401fa tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:40.310167 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4f3adb2d-2ba9-43d2-ae49-08a5f99401fa tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:41.279536 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-105a4e1f-f7d0-40a7-b203-ca77d97960f8 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Flavor limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 16 21:31:41.280019 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-105a4e1f-f7d0-40a7-b203-ca77d97960f8 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Image limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 16 21:31:41.280485 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-105a4e1f-f7d0-40a7-b203-ca77d97960f8 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Flavor pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 16 21:31:41.287509 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-105a4e1f-f7d0-40a7-b203-ca77d97960f8 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Image pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 16 21:31:41.292278 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-105a4e1f-f7d0-40a7-b203-ca77d97960f8 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 16 21:31:41.310224 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-105a4e1f-f7d0-40a7-b203-ca77d97960f8 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:41.313537 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-105a4e1f-f7d0-40a7-b203-ca77d97960f8 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:41.316257 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-105a4e1f-f7d0-40a7-b203-ca77d97960f8 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:41.320236 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-105a4e1f-f7d0-40a7-b203-ca77d97960f8 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:41.376125 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-105a4e1f-f7d0-40a7-b203-ca77d97960f8 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Getting quotas for project e451ee7bfd2d4ab58d30c33fef334b55. Resources: {'ram', 'instances', 'cores'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:31:41.384801 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-105a4e1f-f7d0-40a7-b203-ca77d97960f8 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Getting quotas for user ca2c9e7e6d61414cbf2c7e17753bc0df and project e451ee7bfd2d4ab58d30c33fef334b55. Resources: {'ram', 'instances', 'cores'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:31:41.396392 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-4f3adb2d-2ba9-43d2-ae49-08a5f99401fa tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da77ec3d0f04496d8c8df4025bba76d1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '207ec2d1883c4a1ba89696e6e77cf3aa', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:31:41.404518 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-105a4e1f-f7d0-40a7-b203-ca77d97960f8 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Going to run 1 instances... {{(pid=138681) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 16 21:31:41.408754 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-105a4e1f-f7d0-40a7-b203-ca77d97960f8 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] [instance: 8b77de54-4c8b-4ed5-84f5-68bf69f9809f] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138681) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 16 21:31:41.415198 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-4f3adb2d-2ba9-43d2-ae49-08a5f99401fa tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] 10.222.0.16 "GET /compute/v2.1/servers/515fd3da-3965-4882-b9e8-100a7eea64d9" status: 200 len: 1492 microversion: 2.1 time: 1.141619 Jul 16 21:31:41.427245 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2303/4609] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:31:40 2022] GET /compute/v2.1/servers/515fd3da-3965-4882-b9e8-100a7eea64d9 => generated 1492 bytes in 1155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:31:41.443587 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-10e63f07-c57b-4a24-9cb0-a4a18490cfb2 req-1fcbe6d4-053b-4366-b7ee-faddee8ba8c4 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "3aa4786f-20b7-4123-a231-d3d1c3b91b3b", "name": "network-vif-deleted", "tag": "6c497884-33a6-412b-8db5-37cebffc7eca"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:31:41.481537 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-10e63f07-c57b-4a24-9cb0-a4a18490cfb2 req-1fcbe6d4-053b-4366-b7ee-faddee8ba8c4 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:41.482008 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-10e63f07-c57b-4a24-9cb0-a4a18490cfb2 req-1fcbe6d4-053b-4366-b7ee-faddee8ba8c4 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:41.518375 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-10e63f07-c57b-4a24-9cb0-a4a18490cfb2 req-1fcbe6d4-053b-4366-b7ee-faddee8ba8c4 service nova] Creating event network-vif-deleted:6c497884-33a6-412b-8db5-37cebffc7eca for instance 3aa4786f-20b7-4123-a231-d3d1c3b91b3b on nch2-master-1 Jul 16 21:31:41.533419 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-10e63f07-c57b-4a24-9cb0-a4a18490cfb2 req-1fcbe6d4-053b-4366-b7ee-faddee8ba8c4 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.103098 Jul 16 21:31:41.535029 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2304/4610] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:31:41 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:31:41.566625 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-f0534c26-879d-457b-863f-7105f36c4922 tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:31:41.567422 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-f0534c26-879d-457b-863f-7105f36c4922 tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] [instance: 3aa4786f-20b7-4123-a231-d3d1c3b91b3b] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:31:41.608911 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f0534c26-879d-457b-863f-7105f36c4922 tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:41.609977 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f0534c26-879d-457b-863f-7105f36c4922 tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.010s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:41.612717 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f0534c26-879d-457b-863f-7105f36c4922 tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:41.615815 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f0534c26-879d-457b-863f-7105f36c4922 tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:41.646248 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-105a4e1f-f7d0-40a7-b203-ca77d97960f8 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 400 microversion: 2.20 time: 3.138466 Jul 16 21:31:41.648870 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2307/4611] 10.222.0.16 () {62 vars in 1187 bytes} [Sat Jul 16 21:31:38 2022] POST /compute/v2.1/servers => generated 400 bytes in 3142 msecs (HTTP/1.1 202) 10 headers in 453 bytes (2 switches on core 0) Jul 16 21:31:41.666248 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-6106ce65-8dea-401f-ab5f-1a0b5ded8241 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:31:41.669957 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-6106ce65-8dea-401f-ab5f-1a0b5ded8241 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] [instance: 8b77de54-4c8b-4ed5-84f5-68bf69f9809f] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:31:41.732340 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f0534c26-879d-457b-863f-7105f36c4922 tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:41.732797 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f0534c26-879d-457b-863f-7105f36c4922 tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:41.733264 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.objects.instance [None req-f0534c26-879d-457b-863f-7105f36c4922 tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Lazy-loading 'fault' on Instance uuid 3aa4786f-20b7-4123-a231-d3d1c3b91b3b {{(pid=138680) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:31:41.991268 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell0:SELECT=4 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:31:42.029042 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-f0534c26-879d-457b-863f-7105f36c4922 tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1006989e743247bdb29a51b3a634afd9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e8c65e58fac42c4bd7f49eaa3a14115', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:31:42.032358 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-6106ce65-8dea-401f-ab5f-1a0b5ded8241 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:31:42.044363 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-f0534c26-879d-457b-863f-7105f36c4922 tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] 10.222.0.16 "GET /compute/v2.1/servers/3aa4786f-20b7-4123-a231-d3d1c3b91b3b" status: 200 len: 1304 microversion: 2.1 time: 0.492572 Jul 16 21:31:42.046670 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2305/4612] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:31:41 2022] GET /compute/v2.1/servers/3aa4786f-20b7-4123-a231-d3d1c3b91b3b => generated 1304 bytes in 500 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:31:42.061050 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-6106ce65-8dea-401f-ab5f-1a0b5ded8241 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:31:42.066741 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-6106ce65-8dea-401f-ab5f-1a0b5ded8241 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:31:42.071902 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-6106ce65-8dea-401f-ab5f-1a0b5ded8241 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] 10.222.0.16 "GET /compute/v2.1/servers/8b77de54-4c8b-4ed5-84f5-68bf69f9809f" status: 200 len: 1282 microversion: 2.20 time: 0.413203 Jul 16 21:31:42.075640 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2308/4613] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:31:41 2022] GET /compute/v2.1/servers/8b77de54-4c8b-4ed5-84f5-68bf69f9809f => generated 1282 bytes in 422 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:31:42.438407 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-7a314350-98d6-487d-8bb7-172c2f87c2aa tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:31:42.439050 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-7a314350-98d6-487d-8bb7-172c2f87c2aa tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] [instance: 515fd3da-3965-4882-b9e8-100a7eea64d9] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:31:42.455251 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7a314350-98d6-487d-8bb7-172c2f87c2aa tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:42.455251 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7a314350-98d6-487d-8bb7-172c2f87c2aa tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:42.467180 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7a314350-98d6-487d-8bb7-172c2f87c2aa tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:42.470061 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7a314350-98d6-487d-8bb7-172c2f87c2aa tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.014s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:42.482660 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=94,nova_cell1:UPDATE=3,nova_cell1:INSERT=3 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:31:43.066452 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-b2532d67-ab29-4d36-abfb-b89d59ec75a0 tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:31:43.068043 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-b2532d67-ab29-4d36-abfb-b89d59ec75a0 tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] [instance: 3aa4786f-20b7-4123-a231-d3d1c3b91b3b] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:31:43.078682 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b2532d67-ab29-4d36-abfb-b89d59ec75a0 tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:43.079440 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b2532d67-ab29-4d36-abfb-b89d59ec75a0 tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:43.080565 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b2532d67-ab29-4d36-abfb-b89d59ec75a0 tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:43.081180 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b2532d67-ab29-4d36-abfb-b89d59ec75a0 tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:43.140068 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-b2532d67-ab29-4d36-abfb-b89d59ec75a0 tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] HTTP exception thrown: Instance 3aa4786f-20b7-4123-a231-d3d1c3b91b3b could not be found. Jul 16 21:31:43.143079 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-b2532d67-ab29-4d36-abfb-b89d59ec75a0 tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Request method failure captured: Jul 16 21:31:43.143079 ncdv-master-1 devstack@n-api.service[138681]: request: GET /compute/v2.1/servers/3aa4786f-20b7-4123-a231-d3d1c3b91b3b HTTP/1.1 Jul 16 21:31:43.143079 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:31:43.143079 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:31:43.143079 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:31:43.143079 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 21:31:43.143079 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:31:43.143079 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:31:43.143079 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:31:43.143079 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:31:43.143079 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0y4R7yrVjlImWtmbdZRW1I-Lovoq48zyFyvg6xRkr4l9HfqxZ94g47fVmxyDxKv7ZS3B2BsB7A3_JXNeN22x5vJKMyeoQ9bdJo2SIW4a68s1YBXnW7t36S8fAHh3BDTRQvMroKlhJt4zLviQny9SHeqhSyOB21pQz2EeYZEYDywGwg0 Jul 16 21:31:43.143079 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:31:43.143079 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:31:43.143079 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:31:43.143079 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:31:43.143079 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:31:43.143079 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:31:43.143079 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: 1e8c65e58fac42c4bd7f49eaa3a14115 Jul 16 21:31:43.143079 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-TestServerAdvancedOps-475632099 Jul 16 21:31:43.143079 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:31:43.143079 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:31:43.143079 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/1e8c65e58fac42c4bd7f49eaa3a14115"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/1e8c65e58fac42c4bd7f49eaa3a14115"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_1e8c65e58fac42c4bd7f49eaa3a14115"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/1e8c65e58fac42c4bd7f49eaa3a14115"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:31:43.143079 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-TestServerAdvancedOps-475632099 Jul 16 21:31:43.143079 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: 1e8c65e58fac42c4bd7f49eaa3a14115 Jul 16 21:31:43.143079 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-TestServerAdvancedOps-475632099 Jul 16 21:31:43.143079 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-TestServerAdvancedOps-475632099-project Jul 16 21:31:43.143079 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:31:43.143079 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:31:43.143079 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: 1006989e743247bdb29a51b3a634afd9 Jul 16 21:31:43.143079 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-TestServerAdvancedOps-475632099-project Jul 16 21:31:43.143079 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:31:43.143079 ncdv-master-1 devstack@n-api.service[138681]: exception: Instance 3aa4786f-20b7-4123-a231-d3d1c3b91b3b could not be found. Jul 16 21:31:43.143079 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': '3aa4786f-20b7-4123-a231-d3d1c3b91b3b'} Jul 16 21:31:43.143079 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:31:43.143079 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:31:43.143079 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:31:43.143079 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:31:43.143079 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:31:43.143079 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:31:43.143079 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:31:43.143079 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:31:43.143079 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:31:43.143079 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:31:43.143079 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 3aa4786f-20b7-4123-a231-d3d1c3b91b3b could not be found. Jul 16 21:31:43.143079 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:31:43.143079 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:31:43.143079 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:31:43.143079 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:31:43.143079 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:31:43.143079 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:31:43.143079 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:31:43.143079 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:31:43.143079 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:31:43.143079 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:31:43.143079 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:31:43.143079 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:31:43.143079 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:31:43.143079 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:31:43.143079 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:31:43.143079 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:31:43.143079 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 3aa4786f-20b7-4123-a231-d3d1c3b91b3b could not be found. Jul 16 21:31:43.143079 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:31:43.143079 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:31:43.143079 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:31:43.149040 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:31:43.149040 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:31:43.149040 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:31:43.149040 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:31:43.149040 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:31:43.149040 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 3aa4786f-20b7-4123-a231-d3d1c3b91b3b could not be found. Jul 16 21:31:43.149040 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:31:43.149040 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-b2532d67-ab29-4d36-abfb-b89d59ec75a0 tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] Returning 404 to user: Instance 3aa4786f-20b7-4123-a231-d3d1c3b91b3b could not be found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:31:43.149040 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-b2532d67-ab29-4d36-abfb-b89d59ec75a0 tempest-TestServerAdvancedOps-475632099 tempest-TestServerAdvancedOps-475632099-project] 10.222.0.16 "GET /compute/v2.1/servers/3aa4786f-20b7-4123-a231-d3d1c3b91b3b" status: 404 len: 111 microversion: 2.1 time: 0.082880 Jul 16 21:31:43.170593 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2309/4614] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:31:43 2022] GET /compute/v2.1/servers/3aa4786f-20b7-4123-a231-d3d1c3b91b3b => generated 111 bytes in 109 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:31:43.176666 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-43805017-5f24-4097-82f1-085fdf97d7f2 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:31:43.177708 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-43805017-5f24-4097-82f1-085fdf97d7f2 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] [instance: 8b77de54-4c8b-4ed5-84f5-68bf69f9809f] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:31:43.300153 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-7a314350-98d6-487d-8bb7-172c2f87c2aa tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da77ec3d0f04496d8c8df4025bba76d1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '207ec2d1883c4a1ba89696e6e77cf3aa', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:31:43.315760 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-7a314350-98d6-487d-8bb7-172c2f87c2aa tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] 10.222.0.16 "GET /compute/v2.1/servers/515fd3da-3965-4882-b9e8-100a7eea64d9" status: 200 len: 1492 microversion: 2.1 time: 0.884137 Jul 16 21:31:43.316368 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2306/4615] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:31:42 2022] GET /compute/v2.1/servers/515fd3da-3965-4882-b9e8-100a7eea64d9 => generated 1492 bytes in 886 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:31:43.451523 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-43805017-5f24-4097-82f1-085fdf97d7f2 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:31:43.470865 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-43805017-5f24-4097-82f1-085fdf97d7f2 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:31:43.472678 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-43805017-5f24-4097-82f1-085fdf97d7f2 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:31:43.477314 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-43805017-5f24-4097-82f1-085fdf97d7f2 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] 10.222.0.16 "GET /compute/v2.1/servers/8b77de54-4c8b-4ed5-84f5-68bf69f9809f" status: 200 len: 1282 microversion: 2.20 time: 0.305440 Jul 16 21:31:43.478144 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2310/4616] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:31:43 2022] GET /compute/v2.1/servers/8b77de54-4c8b-4ed5-84f5-68bf69f9809f => generated 1282 bytes in 307 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:31:44.339430 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-e4b9c2d5-4d58-4985-8e95-f666b6db1ff7 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:31:44.340071 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-e4b9c2d5-4d58-4985-8e95-f666b6db1ff7 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] [instance: 515fd3da-3965-4882-b9e8-100a7eea64d9] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:31:44.357916 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e4b9c2d5-4d58-4985-8e95-f666b6db1ff7 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:44.358388 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e4b9c2d5-4d58-4985-8e95-f666b6db1ff7 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:44.359228 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e4b9c2d5-4d58-4985-8e95-f666b6db1ff7 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:44.359677 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e4b9c2d5-4d58-4985-8e95-f666b6db1ff7 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:44.480955 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-e4b9c2d5-4d58-4985-8e95-f666b6db1ff7 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da77ec3d0f04496d8c8df4025bba76d1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '207ec2d1883c4a1ba89696e6e77cf3aa', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:31:44.499335 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-48253b9f-aa0b-43bc-8132-3630d0b33dab tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:31:44.500065 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-48253b9f-aa0b-43bc-8132-3630d0b33dab tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] [instance: 8b77de54-4c8b-4ed5-84f5-68bf69f9809f] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:31:44.511216 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-e4b9c2d5-4d58-4985-8e95-f666b6db1ff7 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] 10.222.0.16 "GET /compute/v2.1/servers/515fd3da-3965-4882-b9e8-100a7eea64d9" status: 200 len: 1450 microversion: 2.1 time: 0.181648 Jul 16 21:31:44.512709 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2307/4617] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:31:44 2022] GET /compute/v2.1/servers/515fd3da-3965-4882-b9e8-100a7eea64d9 => generated 1450 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:31:44.518805 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-48253b9f-aa0b-43bc-8132-3630d0b33dab tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:44.519618 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-48253b9f-aa0b-43bc-8132-3630d0b33dab tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:44.520692 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-48253b9f-aa0b-43bc-8132-3630d0b33dab tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:44.523931 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-48253b9f-aa0b-43bc-8132-3630d0b33dab tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:44.744032 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-db840028-4dcd-4d77-a44e-fe5fdd1787f7 req-4fdb0219-00f5-4606-8073-544ba8b57a85 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "515fd3da-3965-4882-b9e8-100a7eea64d9", "name": "network-vif-deleted", "tag": "f74e70f6-9bde-4b10-9973-eedb2194871b"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:31:44.765873 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-db840028-4dcd-4d77-a44e-fe5fdd1787f7 req-4fdb0219-00f5-4606-8073-544ba8b57a85 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:44.768472 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-db840028-4dcd-4d77-a44e-fe5fdd1787f7 req-4fdb0219-00f5-4606-8073-544ba8b57a85 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:44.800761 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-db840028-4dcd-4d77-a44e-fe5fdd1787f7 req-4fdb0219-00f5-4606-8073-544ba8b57a85 service nova] Creating event network-vif-deleted:f74e70f6-9bde-4b10-9973-eedb2194871b for instance 515fd3da-3965-4882-b9e8-100a7eea64d9 on nch1-master-1 Jul 16 21:31:44.828169 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-db840028-4dcd-4d77-a44e-fe5fdd1787f7 req-4fdb0219-00f5-4606-8073-544ba8b57a85 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.075876 Jul 16 21:31:44.831554 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2308/4618] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:31:44 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 16 21:31:45.353164 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-48253b9f-aa0b-43bc-8132-3630d0b33dab tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:31:45.372552 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-48253b9f-aa0b-43bc-8132-3630d0b33dab tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:31:45.374525 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-48253b9f-aa0b-43bc-8132-3630d0b33dab tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:31:45.375817 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-48253b9f-aa0b-43bc-8132-3630d0b33dab tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] 10.222.0.16 "GET /compute/v2.1/servers/8b77de54-4c8b-4ed5-84f5-68bf69f9809f" status: 200 len: 1334 microversion: 2.20 time: 0.881628 Jul 16 21:31:45.390280 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2311/4619] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:31:44 2022] GET /compute/v2.1/servers/8b77de54-4c8b-4ed5-84f5-68bf69f9809f => generated 1334 bytes in 894 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:31:45.531628 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-72195e0b-9d7e-477a-9d18-a3f8092021d1 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:31:45.533381 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-72195e0b-9d7e-477a-9d18-a3f8092021d1 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] [instance: 515fd3da-3965-4882-b9e8-100a7eea64d9] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:31:45.551120 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-72195e0b-9d7e-477a-9d18-a3f8092021d1 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:45.551828 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-72195e0b-9d7e-477a-9d18-a3f8092021d1 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:45.554632 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-72195e0b-9d7e-477a-9d18-a3f8092021d1 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:45.555682 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-72195e0b-9d7e-477a-9d18-a3f8092021d1 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:45.657356 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-72195e0b-9d7e-477a-9d18-a3f8092021d1 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:45.658710 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-72195e0b-9d7e-477a-9d18-a3f8092021d1 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:45.661317 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.objects.instance [None req-72195e0b-9d7e-477a-9d18-a3f8092021d1 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lazy-loading 'fault' on Instance uuid 515fd3da-3965-4882-b9e8-100a7eea64d9 {{(pid=138680) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:31:45.793060 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-72195e0b-9d7e-477a-9d18-a3f8092021d1 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'da77ec3d0f04496d8c8df4025bba76d1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '207ec2d1883c4a1ba89696e6e77cf3aa', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:31:45.810027 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-72195e0b-9d7e-477a-9d18-a3f8092021d1 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] 10.222.0.16 "GET /compute/v2.1/servers/515fd3da-3965-4882-b9e8-100a7eea64d9" status: 200 len: 1304 microversion: 2.1 time: 0.284355 Jul 16 21:31:45.821298 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2309/4620] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:31:45 2022] GET /compute/v2.1/servers/515fd3da-3965-4882-b9e8-100a7eea64d9 => generated 1304 bytes in 286 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:31:46.395250 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-109169d9-50a0-40f4-8b5e-20974e0d4b5e tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:31:46.396269 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-109169d9-50a0-40f4-8b5e-20974e0d4b5e tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] [instance: 8b77de54-4c8b-4ed5-84f5-68bf69f9809f] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:31:46.403293 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-109169d9-50a0-40f4-8b5e-20974e0d4b5e tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:46.403554 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-109169d9-50a0-40f4-8b5e-20974e0d4b5e tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:46.404791 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-109169d9-50a0-40f4-8b5e-20974e0d4b5e tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:46.405402 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-109169d9-50a0-40f4-8b5e-20974e0d4b5e tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:46.586615 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-109169d9-50a0-40f4-8b5e-20974e0d4b5e tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:31:46.601241 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-109169d9-50a0-40f4-8b5e-20974e0d4b5e tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:31:46.604389 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-109169d9-50a0-40f4-8b5e-20974e0d4b5e tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:31:46.612109 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-109169d9-50a0-40f4-8b5e-20974e0d4b5e tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] 10.222.0.16 "GET /compute/v2.1/servers/8b77de54-4c8b-4ed5-84f5-68bf69f9809f" status: 200 len: 1340 microversion: 2.20 time: 0.216377 Jul 16 21:31:46.612109 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2312/4621] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:31:46 2022] GET /compute/v2.1/servers/8b77de54-4c8b-4ed5-84f5-68bf69f9809f => generated 1340 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:31:46.828829 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-c58d4aad-8773-423f-bc24-8e11de1fb0e1 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:31:46.830316 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-c58d4aad-8773-423f-bc24-8e11de1fb0e1 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] [instance: 515fd3da-3965-4882-b9e8-100a7eea64d9] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:31:46.838536 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c58d4aad-8773-423f-bc24-8e11de1fb0e1 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:46.839373 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c58d4aad-8773-423f-bc24-8e11de1fb0e1 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:46.841780 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c58d4aad-8773-423f-bc24-8e11de1fb0e1 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:46.842473 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c58d4aad-8773-423f-bc24-8e11de1fb0e1 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:46.860392 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-c58d4aad-8773-423f-bc24-8e11de1fb0e1 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] HTTP exception thrown: Instance 515fd3da-3965-4882-b9e8-100a7eea64d9 could not be found. Jul 16 21:31:46.862567 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-c58d4aad-8773-423f-bc24-8e11de1fb0e1 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Request method failure captured: Jul 16 21:31:46.862567 ncdv-master-1 devstack@n-api.service[138680]: request: GET /compute/v2.1/servers/515fd3da-3965-4882-b9e8-100a7eea64d9 HTTP/1.1 Jul 16 21:31:46.862567 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:31:46.862567 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:31:46.862567 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:31:46.862567 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 0 Jul 16 21:31:46.862567 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:31:46.862567 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:31:46.862567 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:31:46.862567 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:31:46.862567 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0y3FP3QHjhX3GJLGfDYK8yH2ipiwfxhuYVl72Yq3zCWqktw2oRz0Vp6uBJ43tQfjrJRsrBMy9qlPSpZpq79cCogyPwRKOOhs7ji6KNmqfVGRI8zyZDEnK4Uzqiwhpt1K21cJh5Tr6hBPIf-oCch2kUReB0byV1m5CleH4ahZz4ST7GM Jul 16 21:31:46.862567 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:31:46.862567 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:31:46.862567 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:31:46.862567 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:31:46.862567 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:31:46.862567 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:31:46.862567 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: 207ec2d1883c4a1ba89696e6e77cf3aa Jul 16 21:31:46.862567 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-ServerRescueTestJSON-975252187 Jul 16 21:31:46.862567 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:31:46.862567 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:31:46.862567 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/207ec2d1883c4a1ba89696e6e77cf3aa"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/207ec2d1883c4a1ba89696e6e77cf3aa"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_207ec2d1883c4a1ba89696e6e77cf3aa"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/207ec2d1883c4a1ba89696e6e77cf3aa"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:31:46.862567 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-ServerRescueTestJSON-975252187 Jul 16 21:31:46.862567 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: 207ec2d1883c4a1ba89696e6e77cf3aa Jul 16 21:31:46.862567 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-ServerRescueTestJSON-975252187 Jul 16 21:31:46.862567 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-ServerRescueTestJSON-975252187-project Jul 16 21:31:46.862567 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:31:46.862567 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:31:46.862567 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: da77ec3d0f04496d8c8df4025bba76d1 Jul 16 21:31:46.862567 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-ServerRescueTestJSON-975252187-project Jul 16 21:31:46.862567 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 21:31:46.862567 ncdv-master-1 devstack@n-api.service[138680]: exception: Instance 515fd3da-3965-4882-b9e8-100a7eea64d9 could not be found. Jul 16 21:31:46.862567 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': '515fd3da-3965-4882-b9e8-100a7eea64d9'} Jul 16 21:31:46.862567 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:31:46.862567 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:31:46.862567 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:31:46.862567 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:31:46.862567 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:31:46.862567 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:31:46.862567 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:31:46.862567 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:31:46.862567 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:31:46.862567 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:31:46.862567 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 515fd3da-3965-4882-b9e8-100a7eea64d9 could not be found. Jul 16 21:31:46.862567 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:31:46.862567 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:31:46.862567 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:31:46.862567 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:31:46.862567 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:31:46.862567 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:31:46.862567 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:31:46.862567 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:31:46.862567 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:31:46.862567 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:31:46.862567 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:31:46.862567 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:31:46.862567 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:31:46.862567 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:31:46.862567 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:31:46.862567 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:31:46.862567 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 515fd3da-3965-4882-b9e8-100a7eea64d9 could not be found. Jul 16 21:31:46.862567 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:31:46.862567 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:31:46.862567 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:31:46.867246 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:31:46.867246 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:31:46.867246 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:31:46.867246 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:31:46.867246 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:31:46.867246 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 515fd3da-3965-4882-b9e8-100a7eea64d9 could not be found. Jul 16 21:31:46.867246 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:31:46.869933 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-c58d4aad-8773-423f-bc24-8e11de1fb0e1 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] Returning 404 to user: Instance 515fd3da-3965-4882-b9e8-100a7eea64d9 could not be found. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:31:46.871042 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-c58d4aad-8773-423f-bc24-8e11de1fb0e1 tempest-ServerRescueTestJSON-975252187 tempest-ServerRescueTestJSON-975252187-project] 10.222.0.16 "GET /compute/v2.1/servers/515fd3da-3965-4882-b9e8-100a7eea64d9" status: 404 len: 111 microversion: 2.1 time: 0.046605 Jul 16 21:31:46.872533 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2310/4622] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:31:46 2022] GET /compute/v2.1/servers/515fd3da-3965-4882-b9e8-100a7eea64d9 => generated 111 bytes in 49 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:31:47.626339 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-942e5575-686c-4295-9728-f5dbed62d7ff tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:31:47.627665 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-942e5575-686c-4295-9728-f5dbed62d7ff tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] [instance: 8b77de54-4c8b-4ed5-84f5-68bf69f9809f] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:31:47.636931 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-942e5575-686c-4295-9728-f5dbed62d7ff tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:47.637635 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-942e5575-686c-4295-9728-f5dbed62d7ff tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:47.649046 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-942e5575-686c-4295-9728-f5dbed62d7ff tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:47.649046 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-942e5575-686c-4295-9728-f5dbed62d7ff tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:47.797964 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-942e5575-686c-4295-9728-f5dbed62d7ff tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:31:47.809811 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-942e5575-686c-4295-9728-f5dbed62d7ff tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:31:47.812071 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-942e5575-686c-4295-9728-f5dbed62d7ff tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:31:47.813558 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-942e5575-686c-4295-9728-f5dbed62d7ff tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] 10.222.0.16 "GET /compute/v2.1/servers/8b77de54-4c8b-4ed5-84f5-68bf69f9809f" status: 200 len: 1340 microversion: 2.20 time: 0.191558 Jul 16 21:31:47.825202 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2313/4623] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:31:47 2022] GET /compute/v2.1/servers/8b77de54-4c8b-4ed5-84f5-68bf69f9809f => generated 1340 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:31:48.837160 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-1aa56de1-6273-453c-ac56-3a45962a70e9 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:31:48.837785 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-1aa56de1-6273-453c-ac56-3a45962a70e9 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] [instance: 8b77de54-4c8b-4ed5-84f5-68bf69f9809f] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:31:48.849560 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1aa56de1-6273-453c-ac56-3a45962a70e9 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:48.851581 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1aa56de1-6273-453c-ac56-3a45962a70e9 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:48.855608 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1aa56de1-6273-453c-ac56-3a45962a70e9 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:48.855608 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1aa56de1-6273-453c-ac56-3a45962a70e9 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:48.903751 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-f8b17629-4c42-4f14-9895-0e6b6895c16b tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Action: 'create', calling method: .version_select at 0x7fdd8ff494c0>, body: {"keypair": {"name": "tempest-TestNetworkBasicOps-520706742", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBBg7ZWDWuthVid8+yJBYa7vDmhXLn7EzocNxtLwA5GO5Sw7F6tDTpaggvqeoAjsdpSIUQ0L4DdGDGnjAbjIWA4drsrX9yuqvnKbtXVubXw9vHLJw2ZeT93kZn5G2TuBn4A=="}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:31:48.950463 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-f8b17629-4c42-4f14-9895-0e6b6895c16b tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Getting quotas for project bed7c94624484c7cb8abeb3bfc8757b9. Resources: {'key_pairs'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:31:48.972557 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-f8b17629-4c42-4f14-9895-0e6b6895c16b tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Getting quotas for user 4ddfae3448704591b662de83b97a1876 and project bed7c94624484c7cb8abeb3bfc8757b9. Resources: {'key_pairs'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:31:49.120139 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-f8b17629-4c42-4f14-9895-0e6b6895c16b tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] 10.222.0.16 "POST /compute/v2.1/os-keypairs" status: 200 len: 397 microversion: 2.1 time: 0.220365 Jul 16 21:31:49.121218 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2314/4624] 10.222.0.16 () {60 vars in 1158 bytes} [Sat Jul 16 21:31:48 2022] POST /compute/v2.1/os-keypairs => generated 397 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:31:49.850938 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-1aa56de1-6273-453c-ac56-3a45962a70e9 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:31:49.872346 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-1aa56de1-6273-453c-ac56-3a45962a70e9 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:31:49.876476 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-1aa56de1-6273-453c-ac56-3a45962a70e9 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:31:49.877437 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-1aa56de1-6273-453c-ac56-3a45962a70e9 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] 10.222.0.16 "GET /compute/v2.1/servers/8b77de54-4c8b-4ed5-84f5-68bf69f9809f" status: 200 len: 1406 microversion: 2.20 time: 1.045896 Jul 16 21:31:49.885983 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2311/4625] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:31:48 2022] GET /compute/v2.1/servers/8b77de54-4c8b-4ed5-84f5-68bf69f9809f => generated 1406 bytes in 1055 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:31:50.909915 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-10d3c8c1-7b4a-453a-a7e0-098bbf523373 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:31:50.912048 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-10d3c8c1-7b4a-453a-a7e0-098bbf523373 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] [instance: 8b77de54-4c8b-4ed5-84f5-68bf69f9809f] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:31:50.937254 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-10d3c8c1-7b4a-453a-a7e0-098bbf523373 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:50.940151 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-10d3c8c1-7b4a-453a-a7e0-098bbf523373 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:50.942118 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-10d3c8c1-7b4a-453a-a7e0-098bbf523373 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:50.947597 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-10d3c8c1-7b4a-453a-a7e0-098bbf523373 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:51.363770 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell0:SELECT=2 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:31:51.481949 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-10d3c8c1-7b4a-453a-a7e0-098bbf523373 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:31:51.503745 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-10d3c8c1-7b4a-453a-a7e0-098bbf523373 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:31:51.513539 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-10d3c8c1-7b4a-453a-a7e0-098bbf523373 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:31:51.514904 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-10d3c8c1-7b4a-453a-a7e0-098bbf523373 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] 10.222.0.16 "GET /compute/v2.1/servers/8b77de54-4c8b-4ed5-84f5-68bf69f9809f" status: 200 len: 1406 microversion: 2.20 time: 0.615304 Jul 16 21:31:51.526586 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2315/4626] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:31:50 2022] GET /compute/v2.1/servers/8b77de54-4c8b-4ed5-84f5-68bf69f9809f => generated 1406 bytes in 617 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:31:51.856649 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-6cfc157f-8f75-4645-903d-0c5b868cb722 req-4e621a9d-36dd-4f3b-84fa-ac77f59cb2e3 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "8b77de54-4c8b-4ed5-84f5-68bf69f9809f", "tag": "5310cc77-6413-4115-9f89-f9fab39070d1"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:31:51.878741 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-6cfc157f-8f75-4645-903d-0c5b868cb722 req-4e621a9d-36dd-4f3b-84fa-ac77f59cb2e3 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:51.879233 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-6cfc157f-8f75-4645-903d-0c5b868cb722 req-4e621a9d-36dd-4f3b-84fa-ac77f59cb2e3 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:51.908056 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-6cfc157f-8f75-4645-903d-0c5b868cb722 req-4e621a9d-36dd-4f3b-84fa-ac77f59cb2e3 service nova] Creating event network-changed:5310cc77-6413-4115-9f89-f9fab39070d1 for instance 8b77de54-4c8b-4ed5-84f5-68bf69f9809f on nch2-master-1 Jul 16 21:31:51.913927 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-6cfc157f-8f75-4645-903d-0c5b868cb722 req-4e621a9d-36dd-4f3b-84fa-ac77f59cb2e3 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.061119 Jul 16 21:31:51.916078 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2312/4627] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:31:51 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 64 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:31:52.534536 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-834f2910-9b6e-46d9-b01a-8076d22cdd75 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:31:52.535187 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-834f2910-9b6e-46d9-b01a-8076d22cdd75 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] [instance: 8b77de54-4c8b-4ed5-84f5-68bf69f9809f] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:31:52.544663 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-834f2910-9b6e-46d9-b01a-8076d22cdd75 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:52.545515 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-834f2910-9b6e-46d9-b01a-8076d22cdd75 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:52.547114 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-834f2910-9b6e-46d9-b01a-8076d22cdd75 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:52.547725 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-834f2910-9b6e-46d9-b01a-8076d22cdd75 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:53.310008 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-834f2910-9b6e-46d9-b01a-8076d22cdd75 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:31:53.331933 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-834f2910-9b6e-46d9-b01a-8076d22cdd75 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:31:53.331933 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-834f2910-9b6e-46d9-b01a-8076d22cdd75 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:31:53.331933 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-834f2910-9b6e-46d9-b01a-8076d22cdd75 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] 10.222.0.16 "GET /compute/v2.1/servers/8b77de54-4c8b-4ed5-84f5-68bf69f9809f" status: 200 len: 1406 microversion: 2.20 time: 0.798911 Jul 16 21:31:53.331933 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2316/4628] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:31:52 2022] GET /compute/v2.1/servers/8b77de54-4c8b-4ed5-84f5-68bf69f9809f => generated 1406 bytes in 800 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:31:54.350214 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-b8cbf97e-6d9c-46fe-afd2-6abd16548a75 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:31:54.351072 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-b8cbf97e-6d9c-46fe-afd2-6abd16548a75 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] [instance: 8b77de54-4c8b-4ed5-84f5-68bf69f9809f] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:31:54.358150 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b8cbf97e-6d9c-46fe-afd2-6abd16548a75 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:54.358938 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b8cbf97e-6d9c-46fe-afd2-6abd16548a75 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:54.359959 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b8cbf97e-6d9c-46fe-afd2-6abd16548a75 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:54.360477 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b8cbf97e-6d9c-46fe-afd2-6abd16548a75 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:54.546384 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-b8cbf97e-6d9c-46fe-afd2-6abd16548a75 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:31:54.566218 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-b8cbf97e-6d9c-46fe-afd2-6abd16548a75 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:31:54.570218 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-b8cbf97e-6d9c-46fe-afd2-6abd16548a75 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:31:54.571476 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-b8cbf97e-6d9c-46fe-afd2-6abd16548a75 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] 10.222.0.16 "GET /compute/v2.1/servers/8b77de54-4c8b-4ed5-84f5-68bf69f9809f" status: 200 len: 1406 microversion: 2.20 time: 0.226809 Jul 16 21:31:54.572024 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2313/4629] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:31:54 2022] GET /compute/v2.1/servers/8b77de54-4c8b-4ed5-84f5-68bf69f9809f => generated 1406 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:31:55.594188 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-33b46e48-8f30-4dc7-bf0e-45909ad5c0df tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:31:55.594971 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-33b46e48-8f30-4dc7-bf0e-45909ad5c0df tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] [instance: 8b77de54-4c8b-4ed5-84f5-68bf69f9809f] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:31:55.608004 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-33b46e48-8f30-4dc7-bf0e-45909ad5c0df tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:55.610014 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-33b46e48-8f30-4dc7-bf0e-45909ad5c0df tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:55.615304 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-33b46e48-8f30-4dc7-bf0e-45909ad5c0df tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:55.616018 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-33b46e48-8f30-4dc7-bf0e-45909ad5c0df tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:55.973092 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-33b46e48-8f30-4dc7-bf0e-45909ad5c0df tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:31:55.988388 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-33b46e48-8f30-4dc7-bf0e-45909ad5c0df tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:31:55.990261 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-33b46e48-8f30-4dc7-bf0e-45909ad5c0df tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:31:55.991482 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-33b46e48-8f30-4dc7-bf0e-45909ad5c0df tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] 10.222.0.16 "GET /compute/v2.1/servers/8b77de54-4c8b-4ed5-84f5-68bf69f9809f" status: 200 len: 1406 microversion: 2.20 time: 0.403264 Jul 16 21:31:55.992105 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2317/4630] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:31:55 2022] GET /compute/v2.1/servers/8b77de54-4c8b-4ed5-84f5-68bf69f9809f => generated 1406 bytes in 405 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:31:56.203110 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-ec11b6ec-2f2c-4f9b-8d91-fad968fba879 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestNetworkBasicOps-server-163847941", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452", "networks": [{"uuid": "638e6d76-c66f-4116-b5cc-8039708eeb98"}], "key_name": "tempest-TestNetworkBasicOps-520706742", "security_groups": [{"name": "tempest-secgroup-smoke-2114697557"}]}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:31:56.399568 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.network.neutron [None req-ec11b6ec-2f2c-4f9b-8d91-fad968fba879 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] validate_networks() for [('638e6d76-c66f-4116-b5cc-8039708eeb98', None, None, None, None, None)] {{(pid=138680) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 16 21:31:57.009725 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-528b3cdd-8c54-4092-8589-6aecf47086da tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:31:57.011018 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-528b3cdd-8c54-4092-8589-6aecf47086da tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] [instance: 8b77de54-4c8b-4ed5-84f5-68bf69f9809f] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:31:57.023817 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-528b3cdd-8c54-4092-8589-6aecf47086da tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:57.029926 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-528b3cdd-8c54-4092-8589-6aecf47086da tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:57.035141 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-528b3cdd-8c54-4092-8589-6aecf47086da tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:57.036452 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-528b3cdd-8c54-4092-8589-6aecf47086da tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:57.234542 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-ec11b6ec-2f2c-4f9b-8d91-fad968fba879 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Flavor limits 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 16 21:31:57.235712 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-ec11b6ec-2f2c-4f9b-8d91-fad968fba879 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Image limits 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 16 21:31:57.236274 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-ec11b6ec-2f2c-4f9b-8d91-fad968fba879 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Flavor pref 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 16 21:31:57.236702 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-ec11b6ec-2f2c-4f9b-8d91-fad968fba879 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Image pref 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 16 21:31:57.237130 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-ec11b6ec-2f2c-4f9b-8d91-fad968fba879 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 16 21:31:57.250538 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ec11b6ec-2f2c-4f9b-8d91-fad968fba879 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:57.251678 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ec11b6ec-2f2c-4f9b-8d91-fad968fba879 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:57.255215 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ec11b6ec-2f2c-4f9b-8d91-fad968fba879 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:57.255215 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ec11b6ec-2f2c-4f9b-8d91-fad968fba879 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:57.284061 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-ec11b6ec-2f2c-4f9b-8d91-fad968fba879 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Getting quotas for project bed7c94624484c7cb8abeb3bfc8757b9. Resources: {'ram', 'instances', 'cores'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:31:57.292327 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-ec11b6ec-2f2c-4f9b-8d91-fad968fba879 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Getting quotas for user 4ddfae3448704591b662de83b97a1876 and project bed7c94624484c7cb8abeb3bfc8757b9. Resources: {'ram', 'instances', 'cores'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:31:57.309720 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-ec11b6ec-2f2c-4f9b-8d91-fad968fba879 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Going to run 1 instances... {{(pid=138680) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 16 21:31:57.313595 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-ec11b6ec-2f2c-4f9b-8d91-fad968fba879 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] [instance: 88cf13fc-0fa3-4e1e-9d76-db4d93b49faf] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138680) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 16 21:31:57.340768 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_api:SELECT=93,nova_api:DELETE=1,nova_api:UPDATE=3,nova_api:INSERT=3 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:31:57.476029 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-ec11b6ec-2f2c-4f9b-8d91-fad968fba879 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 406 microversion: 2.1 time: 1.277447 Jul 16 21:31:57.477583 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2314/4631] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 21:31:56 2022] POST /compute/v2.1/servers => generated 406 bytes in 1280 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 16 21:31:57.487712 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-3b10b828-0fdc-4bb3-9e02-184140373c1b tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:31:57.490229 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-3b10b828-0fdc-4bb3-9e02-184140373c1b tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] [instance: 88cf13fc-0fa3-4e1e-9d76-db4d93b49faf] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:31:57.683989 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-3b10b828-0fdc-4bb3-9e02-184140373c1b tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ddfae3448704591b662de83b97a1876', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bed7c94624484c7cb8abeb3bfc8757b9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:31:57.699292 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-3b10b828-0fdc-4bb3-9e02-184140373c1b tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] 10.222.0.16 "GET /compute/v2.1/servers/88cf13fc-0fa3-4e1e-9d76-db4d93b49faf" status: 200 len: 1247 microversion: 2.1 time: 0.215377 Jul 16 21:31:57.710081 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2315/4632] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:31:57 2022] GET /compute/v2.1/servers/88cf13fc-0fa3-4e1e-9d76-db4d93b49faf => generated 1247 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:31:57.732212 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-528b3cdd-8c54-4092-8589-6aecf47086da tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:31:57.746281 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-528b3cdd-8c54-4092-8589-6aecf47086da tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:31:57.750371 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-528b3cdd-8c54-4092-8589-6aecf47086da tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:31:57.751567 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-528b3cdd-8c54-4092-8589-6aecf47086da tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] 10.222.0.16 "GET /compute/v2.1/servers/8b77de54-4c8b-4ed5-84f5-68bf69f9809f" status: 200 len: 1406 microversion: 2.20 time: 0.745813 Jul 16 21:31:57.767588 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2318/4633] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:31:57 2022] GET /compute/v2.1/servers/8b77de54-4c8b-4ed5-84f5-68bf69f9809f => generated 1406 bytes in 757 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:31:58.717049 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-731faa5e-7e53-4c15-a194-dcb2a5c8b981 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:31:58.719416 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-731faa5e-7e53-4c15-a194-dcb2a5c8b981 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] [instance: 88cf13fc-0fa3-4e1e-9d76-db4d93b49faf] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:31:58.769828 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-60a5c357-19fe-45f9-8bf6-2f946d3e705f tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:31:58.771532 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-60a5c357-19fe-45f9-8bf6-2f946d3e705f tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] [instance: 8b77de54-4c8b-4ed5-84f5-68bf69f9809f] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:31:58.786172 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-60a5c357-19fe-45f9-8bf6-2f946d3e705f tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:58.786642 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-60a5c357-19fe-45f9-8bf6-2f946d3e705f tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:58.787603 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-60a5c357-19fe-45f9-8bf6-2f946d3e705f tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:31:58.788051 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-60a5c357-19fe-45f9-8bf6-2f946d3e705f tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:31:58.903562 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-731faa5e-7e53-4c15-a194-dcb2a5c8b981 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ddfae3448704591b662de83b97a1876', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bed7c94624484c7cb8abeb3bfc8757b9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:31:58.915889 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-731faa5e-7e53-4c15-a194-dcb2a5c8b981 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] 10.222.0.16 "GET /compute/v2.1/servers/88cf13fc-0fa3-4e1e-9d76-db4d93b49faf" status: 200 len: 1247 microversion: 2.1 time: 0.201017 Jul 16 21:31:58.925264 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2316/4634] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:31:58 2022] GET /compute/v2.1/servers/88cf13fc-0fa3-4e1e-9d76-db4d93b49faf => generated 1247 bytes in 208 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:31:58.989838 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell0:SELECT=14 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:31:59.103643 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-60a5c357-19fe-45f9-8bf6-2f946d3e705f tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:31:59.127568 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-60a5c357-19fe-45f9-8bf6-2f946d3e705f tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:31:59.130564 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-60a5c357-19fe-45f9-8bf6-2f946d3e705f tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:31:59.132070 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-60a5c357-19fe-45f9-8bf6-2f946d3e705f tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] 10.222.0.16 "GET /compute/v2.1/servers/8b77de54-4c8b-4ed5-84f5-68bf69f9809f" status: 200 len: 1406 microversion: 2.20 time: 0.366116 Jul 16 21:31:59.133479 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2319/4635] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:31:58 2022] GET /compute/v2.1/servers/8b77de54-4c8b-4ed5-84f5-68bf69f9809f => generated 1406 bytes in 368 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:31:59.965264 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-86cb69f3-2ada-4dd0-aaad-35e6625f080f tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:31:59.968853 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-86cb69f3-2ada-4dd0-aaad-35e6625f080f tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] [instance: 88cf13fc-0fa3-4e1e-9d76-db4d93b49faf] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:32:00.018475 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-86cb69f3-2ada-4dd0-aaad-35e6625f080f tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:00.018965 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-86cb69f3-2ada-4dd0-aaad-35e6625f080f tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:00.023305 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-86cb69f3-2ada-4dd0-aaad-35e6625f080f tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:00.024498 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-86cb69f3-2ada-4dd0-aaad-35e6625f080f tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:00.151367 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-3d8e8d66-83bb-486a-8f98-f7fd59563ff3 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:32:00.155349 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-3d8e8d66-83bb-486a-8f98-f7fd59563ff3 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] [instance: 8b77de54-4c8b-4ed5-84f5-68bf69f9809f] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:32:00.164864 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3d8e8d66-83bb-486a-8f98-f7fd59563ff3 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:00.165908 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3d8e8d66-83bb-486a-8f98-f7fd59563ff3 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:00.167127 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3d8e8d66-83bb-486a-8f98-f7fd59563ff3 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:00.168642 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3d8e8d66-83bb-486a-8f98-f7fd59563ff3 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:00.503211 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-86cb69f3-2ada-4dd0-aaad-35e6625f080f tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ddfae3448704591b662de83b97a1876', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bed7c94624484c7cb8abeb3bfc8757b9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:32:00.512052 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-86cb69f3-2ada-4dd0-aaad-35e6625f080f tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] 10.222.0.16 "GET /compute/v2.1/servers/88cf13fc-0fa3-4e1e-9d76-db4d93b49faf" status: 200 len: 1299 microversion: 2.1 time: 0.573523 Jul 16 21:32:00.512804 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2317/4636] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:31:59 2022] GET /compute/v2.1/servers/88cf13fc-0fa3-4e1e-9d76-db4d93b49faf => generated 1299 bytes in 577 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:32:00.782418 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-3d8e8d66-83bb-486a-8f98-f7fd59563ff3 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:32:00.793582 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-3d8e8d66-83bb-486a-8f98-f7fd59563ff3 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:32:00.811141 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-3d8e8d66-83bb-486a-8f98-f7fd59563ff3 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:32:00.816041 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-3d8e8d66-83bb-486a-8f98-f7fd59563ff3 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] 10.222.0.16 "GET /compute/v2.1/servers/8b77de54-4c8b-4ed5-84f5-68bf69f9809f" status: 200 len: 1406 microversion: 2.20 time: 0.668425 Jul 16 21:32:00.818485 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2320/4637] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:32:00 2022] GET /compute/v2.1/servers/8b77de54-4c8b-4ed5-84f5-68bf69f9809f => generated 1406 bytes in 671 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:32:00.831403 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-b4ea44dc-0c9e-4cdd-8a0e-b83e42753aa8 req-a5016644-e650-49d7-99a7-99c02ce55046 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "8b77de54-4c8b-4ed5-84f5-68bf69f9809f", "name": "network-vif-plugged", "status": "completed", "tag": "5310cc77-6413-4115-9f89-f9fab39070d1"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:32:00.856705 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-b4ea44dc-0c9e-4cdd-8a0e-b83e42753aa8 req-a5016644-e650-49d7-99a7-99c02ce55046 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:00.859702 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-b4ea44dc-0c9e-4cdd-8a0e-b83e42753aa8 req-a5016644-e650-49d7-99a7-99c02ce55046 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:00.886654 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-b4ea44dc-0c9e-4cdd-8a0e-b83e42753aa8 req-a5016644-e650-49d7-99a7-99c02ce55046 service nova] Creating event network-vif-plugged:5310cc77-6413-4115-9f89-f9fab39070d1 for instance 8b77de54-4c8b-4ed5-84f5-68bf69f9809f on nch2-master-1 Jul 16 21:32:00.900457 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-b4ea44dc-0c9e-4cdd-8a0e-b83e42753aa8 req-a5016644-e650-49d7-99a7-99c02ce55046 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.072701 Jul 16 21:32:00.902187 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2318/4638] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:32:00 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 75 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:32:01.539377 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-5c823da9-89d2-4906-aa82-6bad1a1991d8 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:32:01.540704 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-5c823da9-89d2-4906-aa82-6bad1a1991d8 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] [instance: 88cf13fc-0fa3-4e1e-9d76-db4d93b49faf] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:32:01.556782 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5c823da9-89d2-4906-aa82-6bad1a1991d8 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:01.557303 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5c823da9-89d2-4906-aa82-6bad1a1991d8 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:01.558663 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5c823da9-89d2-4906-aa82-6bad1a1991d8 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:01.559447 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5c823da9-89d2-4906-aa82-6bad1a1991d8 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:01.722505 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-5c823da9-89d2-4906-aa82-6bad1a1991d8 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ddfae3448704591b662de83b97a1876', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bed7c94624484c7cb8abeb3bfc8757b9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:32:01.737642 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-5c823da9-89d2-4906-aa82-6bad1a1991d8 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] 10.222.0.16 "GET /compute/v2.1/servers/88cf13fc-0fa3-4e1e-9d76-db4d93b49faf" status: 200 len: 1305 microversion: 2.1 time: 0.204694 Jul 16 21:32:01.744268 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2321/4639] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:32:01 2022] GET /compute/v2.1/servers/88cf13fc-0fa3-4e1e-9d76-db4d93b49faf => generated 1305 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:32:01.835817 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-143c368f-e150-4e15-9b3f-4e056bca06dd tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:32:01.836530 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-143c368f-e150-4e15-9b3f-4e056bca06dd tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] [instance: 8b77de54-4c8b-4ed5-84f5-68bf69f9809f] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:32:01.846432 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-143c368f-e150-4e15-9b3f-4e056bca06dd tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:01.846897 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-143c368f-e150-4e15-9b3f-4e056bca06dd tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:01.847710 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-143c368f-e150-4e15-9b3f-4e056bca06dd tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:01.848764 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-143c368f-e150-4e15-9b3f-4e056bca06dd tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:02.542092 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-143c368f-e150-4e15-9b3f-4e056bca06dd tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:32:02.553905 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-143c368f-e150-4e15-9b3f-4e056bca06dd tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:32:02.556639 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-143c368f-e150-4e15-9b3f-4e056bca06dd tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:32:02.558497 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-143c368f-e150-4e15-9b3f-4e056bca06dd tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] 10.222.0.16 "GET /compute/v2.1/servers/8b77de54-4c8b-4ed5-84f5-68bf69f9809f" status: 200 len: 1406 microversion: 2.20 time: 0.727597 Jul 16 21:32:02.562672 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2319/4640] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:32:01 2022] GET /compute/v2.1/servers/8b77de54-4c8b-4ed5-84f5-68bf69f9809f => generated 1406 bytes in 732 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:32:02.764760 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-98292c76-9099-4896-bfeb-a99b3ac9b782 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:32:02.766138 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-98292c76-9099-4896-bfeb-a99b3ac9b782 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] [instance: 88cf13fc-0fa3-4e1e-9d76-db4d93b49faf] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:32:02.775438 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-98292c76-9099-4896-bfeb-a99b3ac9b782 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:02.777430 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-98292c76-9099-4896-bfeb-a99b3ac9b782 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:02.778408 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-98292c76-9099-4896-bfeb-a99b3ac9b782 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:02.779099 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-98292c76-9099-4896-bfeb-a99b3ac9b782 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:03.584650 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-83879cd8-d538-4c7a-8f5c-90fcd62b936f tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:32:03.586529 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-83879cd8-d538-4c7a-8f5c-90fcd62b936f tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] [instance: 8b77de54-4c8b-4ed5-84f5-68bf69f9809f] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:32:03.608543 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-83879cd8-d538-4c7a-8f5c-90fcd62b936f tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:03.610356 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-83879cd8-d538-4c7a-8f5c-90fcd62b936f tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:03.611713 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-83879cd8-d538-4c7a-8f5c-90fcd62b936f tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:03.615040 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-83879cd8-d538-4c7a-8f5c-90fcd62b936f tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:03.622508 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-98292c76-9099-4896-bfeb-a99b3ac9b782 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ddfae3448704591b662de83b97a1876', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bed7c94624484c7cb8abeb3bfc8757b9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:32:03.631918 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-98292c76-9099-4896-bfeb-a99b3ac9b782 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] 10.222.0.16 "GET /compute/v2.1/servers/88cf13fc-0fa3-4e1e-9d76-db4d93b49faf" status: 200 len: 1373 microversion: 2.1 time: 0.874459 Jul 16 21:32:03.636743 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2322/4641] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:32:02 2022] GET /compute/v2.1/servers/88cf13fc-0fa3-4e1e-9d76-db4d93b49faf => generated 1373 bytes in 880 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:32:04.105868 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-83879cd8-d538-4c7a-8f5c-90fcd62b936f tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:32:04.125444 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-83879cd8-d538-4c7a-8f5c-90fcd62b936f tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:32:04.131643 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-83879cd8-d538-4c7a-8f5c-90fcd62b936f tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:32:04.135921 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-83879cd8-d538-4c7a-8f5c-90fcd62b936f tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] 10.222.0.16 "GET /compute/v2.1/servers/8b77de54-4c8b-4ed5-84f5-68bf69f9809f" status: 200 len: 1598 microversion: 2.20 time: 0.559759 Jul 16 21:32:04.144435 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2320/4642] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:32:03 2022] GET /compute/v2.1/servers/8b77de54-4c8b-4ed5-84f5-68bf69f9809f => generated 1598 bytes in 569 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 16 21:32:04.163839 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-e7c09040-55fd-4d60-9d0c-a57eee0e9790 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:32:04.164935 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-e7c09040-55fd-4d60-9d0c-a57eee0e9790 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] [instance: 8b77de54-4c8b-4ed5-84f5-68bf69f9809f] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:32:04.181089 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e7c09040-55fd-4d60-9d0c-a57eee0e9790 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:04.184885 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e7c09040-55fd-4d60-9d0c-a57eee0e9790 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:04.186277 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e7c09040-55fd-4d60-9d0c-a57eee0e9790 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:04.187099 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e7c09040-55fd-4d60-9d0c-a57eee0e9790 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:04.450977 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-e7c09040-55fd-4d60-9d0c-a57eee0e9790 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] 10.222.0.16 "GET /compute/v2.1/servers/8b77de54-4c8b-4ed5-84f5-68bf69f9809f/os-interface" status: 200 len: 284 microversion: 2.20 time: 0.292344 Jul 16 21:32:04.452103 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2323/4643] 10.222.0.16 () {60 vars in 1315 bytes} [Sat Jul 16 21:32:04 2022] GET /compute/v2.1/servers/8b77de54-4c8b-4ed5-84f5-68bf69f9809f/os-interface => generated 284 bytes in 294 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 16 21:32:04.671682 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-0b801b77-390a-44b8-9307-d68564029967 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:32:04.672328 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-0b801b77-390a-44b8-9307-d68564029967 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] [instance: 88cf13fc-0fa3-4e1e-9d76-db4d93b49faf] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:32:04.699724 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0b801b77-390a-44b8-9307-d68564029967 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:04.702831 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0b801b77-390a-44b8-9307-d68564029967 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:04.703422 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0b801b77-390a-44b8-9307-d68564029967 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:04.703887 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0b801b77-390a-44b8-9307-d68564029967 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:06.226640 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-0b801b77-390a-44b8-9307-d68564029967 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ddfae3448704591b662de83b97a1876', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bed7c94624484c7cb8abeb3bfc8757b9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:32:06.241094 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-0b801b77-390a-44b8-9307-d68564029967 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] 10.222.0.16 "GET /compute/v2.1/servers/88cf13fc-0fa3-4e1e-9d76-db4d93b49faf" status: 200 len: 1373 microversion: 2.1 time: 1.586650 Jul 16 21:32:06.243883 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2321/4644] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:32:04 2022] GET /compute/v2.1/servers/88cf13fc-0fa3-4e1e-9d76-db4d93b49faf => generated 1373 bytes in 1590 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:32:07.264181 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-91e69f47-fd56-4878-aea2-6f45290e044e tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:32:07.264798 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-91e69f47-fd56-4878-aea2-6f45290e044e tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] [instance: 88cf13fc-0fa3-4e1e-9d76-db4d93b49faf] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:32:07.279265 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-91e69f47-fd56-4878-aea2-6f45290e044e tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:07.279756 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-91e69f47-fd56-4878-aea2-6f45290e044e tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:07.282013 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-91e69f47-fd56-4878-aea2-6f45290e044e tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:07.282013 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-91e69f47-fd56-4878-aea2-6f45290e044e tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:08.045400 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-91e69f47-fd56-4878-aea2-6f45290e044e tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ddfae3448704591b662de83b97a1876', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bed7c94624484c7cb8abeb3bfc8757b9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:32:08.056430 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-91e69f47-fd56-4878-aea2-6f45290e044e tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] 10.222.0.16 "GET /compute/v2.1/servers/88cf13fc-0fa3-4e1e-9d76-db4d93b49faf" status: 200 len: 1373 microversion: 2.1 time: 0.797036 Jul 16 21:32:08.070638 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2324/4645] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:32:07 2022] GET /compute/v2.1/servers/88cf13fc-0fa3-4e1e-9d76-db4d93b49faf => generated 1373 bytes in 812 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:32:09.087927 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-84a2e842-f639-4a33-a1a1-ccc69415f76c tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:32:09.089324 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-84a2e842-f639-4a33-a1a1-ccc69415f76c tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] [instance: 88cf13fc-0fa3-4e1e-9d76-db4d93b49faf] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:32:09.093496 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell0:SELECT=2 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:32:09.094420 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell0:SELECT=10 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:32:09.098494 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-84a2e842-f639-4a33-a1a1-ccc69415f76c tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:09.099103 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-84a2e842-f639-4a33-a1a1-ccc69415f76c tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:09.099988 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-84a2e842-f639-4a33-a1a1-ccc69415f76c tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:09.100500 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-84a2e842-f639-4a33-a1a1-ccc69415f76c tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:09.413383 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-b4c9b8e8-6710-4a54-894b-29f87c23d6ea req-c7cdc222-96bf-4b6e-9d50-612da747d276 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "88cf13fc-0fa3-4e1e-9d76-db4d93b49faf", "tag": "008d375b-5b42-4275-ac14-31bcaa9250a8"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:32:09.435760 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-b4c9b8e8-6710-4a54-894b-29f87c23d6ea req-c7cdc222-96bf-4b6e-9d50-612da747d276 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:09.436868 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-b4c9b8e8-6710-4a54-894b-29f87c23d6ea req-c7cdc222-96bf-4b6e-9d50-612da747d276 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:09.483248 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-b4c9b8e8-6710-4a54-894b-29f87c23d6ea req-c7cdc222-96bf-4b6e-9d50-612da747d276 service nova] Creating event network-changed:008d375b-5b42-4275-ac14-31bcaa9250a8 for instance 88cf13fc-0fa3-4e1e-9d76-db4d93b49faf on nch1-master-1 Jul 16 21:32:09.496399 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-b4c9b8e8-6710-4a54-894b-29f87c23d6ea req-c7cdc222-96bf-4b6e-9d50-612da747d276 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.087578 Jul 16 21:32:09.499333 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2325/4646] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:32:09 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:32:10.205464 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-84a2e842-f639-4a33-a1a1-ccc69415f76c tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ddfae3448704591b662de83b97a1876', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bed7c94624484c7cb8abeb3bfc8757b9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:32:10.219398 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-84a2e842-f639-4a33-a1a1-ccc69415f76c tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] 10.222.0.16 "GET /compute/v2.1/servers/88cf13fc-0fa3-4e1e-9d76-db4d93b49faf" status: 200 len: 1373 microversion: 2.1 time: 1.135505 Jul 16 21:32:10.221325 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2322/4647] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:32:09 2022] GET /compute/v2.1/servers/88cf13fc-0fa3-4e1e-9d76-db4d93b49faf => generated 1373 bytes in 1139 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:32:11.245811 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-28ae820b-0e7a-4406-a0dc-da5f2a8a0ceb tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:32:11.247115 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-28ae820b-0e7a-4406-a0dc-da5f2a8a0ceb tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] [instance: 88cf13fc-0fa3-4e1e-9d76-db4d93b49faf] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:32:11.257235 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_api:SELECT=64,nova_api:INSERT=4 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:32:11.298570 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-28ae820b-0e7a-4406-a0dc-da5f2a8a0ceb tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:11.299045 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-28ae820b-0e7a-4406-a0dc-da5f2a8a0ceb tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:11.299834 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-28ae820b-0e7a-4406-a0dc-da5f2a8a0ceb tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:11.300272 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-28ae820b-0e7a-4406-a0dc-da5f2a8a0ceb tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:11.384427 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=92 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:32:11.567891 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-24de0626-8b2e-48f5-8e9d-a1c32a722896 req-7379f10a-ffdc-4914-b3a0-6d18093ad24b service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "8b77de54-4c8b-4ed5-84f5-68bf69f9809f", "tag": "5310cc77-6413-4115-9f89-f9fab39070d1"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:32:11.586516 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-24de0626-8b2e-48f5-8e9d-a1c32a722896 req-7379f10a-ffdc-4914-b3a0-6d18093ad24b service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:11.589696 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-24de0626-8b2e-48f5-8e9d-a1c32a722896 req-7379f10a-ffdc-4914-b3a0-6d18093ad24b service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:11.626225 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-24de0626-8b2e-48f5-8e9d-a1c32a722896 req-7379f10a-ffdc-4914-b3a0-6d18093ad24b service nova] Creating event network-changed:5310cc77-6413-4115-9f89-f9fab39070d1 for instance 8b77de54-4c8b-4ed5-84f5-68bf69f9809f on nch2-master-1 Jul 16 21:32:11.632999 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-24de0626-8b2e-48f5-8e9d-a1c32a722896 req-7379f10a-ffdc-4914-b3a0-6d18093ad24b service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.073063 Jul 16 21:32:11.634249 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2323/4648] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:32:11 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 77 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 16 21:32:11.973409 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-28ae820b-0e7a-4406-a0dc-da5f2a8a0ceb tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ddfae3448704591b662de83b97a1876', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bed7c94624484c7cb8abeb3bfc8757b9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:32:11.990103 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-28ae820b-0e7a-4406-a0dc-da5f2a8a0ceb tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] 10.222.0.16 "GET /compute/v2.1/servers/88cf13fc-0fa3-4e1e-9d76-db4d93b49faf" status: 200 len: 1373 microversion: 2.1 time: 0.750716 Jul 16 21:32:12.009965 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2326/4649] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:32:11 2022] GET /compute/v2.1/servers/88cf13fc-0fa3-4e1e-9d76-db4d93b49faf => generated 1373 bytes in 772 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:32:13.010536 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-c18eb278-3f43-4ea7-9ecb-656f562b3ab3 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:32:13.012152 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-c18eb278-3f43-4ea7-9ecb-656f562b3ab3 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] [instance: 88cf13fc-0fa3-4e1e-9d76-db4d93b49faf] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:32:13.022011 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c18eb278-3f43-4ea7-9ecb-656f562b3ab3 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:13.022880 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c18eb278-3f43-4ea7-9ecb-656f562b3ab3 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:13.023984 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c18eb278-3f43-4ea7-9ecb-656f562b3ab3 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:13.024650 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c18eb278-3f43-4ea7-9ecb-656f562b3ab3 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:14.009305 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-c18eb278-3f43-4ea7-9ecb-656f562b3ab3 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ddfae3448704591b662de83b97a1876', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bed7c94624484c7cb8abeb3bfc8757b9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:32:14.023105 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-c18eb278-3f43-4ea7-9ecb-656f562b3ab3 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] 10.222.0.16 "GET /compute/v2.1/servers/88cf13fc-0fa3-4e1e-9d76-db4d93b49faf" status: 200 len: 1373 microversion: 2.1 time: 1.012479 Jul 16 21:32:14.034711 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2324/4650] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:32:13 2022] GET /compute/v2.1/servers/88cf13fc-0fa3-4e1e-9d76-db4d93b49faf => generated 1373 bytes in 1019 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:32:15.046619 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-01b7c333-2ef9-4698-9428-4f4ab7852fea tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:32:15.050487 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-01b7c333-2ef9-4698-9428-4f4ab7852fea tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] [instance: 88cf13fc-0fa3-4e1e-9d76-db4d93b49faf] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:32:15.071781 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-01b7c333-2ef9-4698-9428-4f4ab7852fea tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:15.073231 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-01b7c333-2ef9-4698-9428-4f4ab7852fea tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:15.078663 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-01b7c333-2ef9-4698-9428-4f4ab7852fea tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:15.079225 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-01b7c333-2ef9-4698-9428-4f4ab7852fea tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:15.767466 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-01b7c333-2ef9-4698-9428-4f4ab7852fea tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ddfae3448704591b662de83b97a1876', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bed7c94624484c7cb8abeb3bfc8757b9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:32:15.779533 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-01b7c333-2ef9-4698-9428-4f4ab7852fea tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] 10.222.0.16 "GET /compute/v2.1/servers/88cf13fc-0fa3-4e1e-9d76-db4d93b49faf" status: 200 len: 1373 microversion: 2.1 time: 0.739681 Jul 16 21:32:15.789608 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2327/4651] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:32:15 2022] GET /compute/v2.1/servers/88cf13fc-0fa3-4e1e-9d76-db4d93b49faf => generated 1373 bytes in 752 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:32:16.538486 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-5cd04467-04f5-4865-bd10-5aa8adc2a3c2 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerTagsTestJSON-server-1735404096", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452", "networks": [{"uuid": "a9c79e72-d481-43f0-a77b-94fcefdb5d7f"}]}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:32:16.616886 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.network.neutron [None req-5cd04467-04f5-4865-bd10-5aa8adc2a3c2 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] validate_networks() for [('a9c79e72-d481-43f0-a77b-94fcefdb5d7f', None, None, None, None, None)] {{(pid=138680) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 16 21:32:16.798738 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-b0d00bfc-24bf-43f9-ac2f-3a6a5e2e24d6 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:32:16.799385 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-b0d00bfc-24bf-43f9-ac2f-3a6a5e2e24d6 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] [instance: 88cf13fc-0fa3-4e1e-9d76-db4d93b49faf] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:32:16.817177 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b0d00bfc-24bf-43f9-ac2f-3a6a5e2e24d6 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:16.817177 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b0d00bfc-24bf-43f9-ac2f-3a6a5e2e24d6 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:16.822558 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b0d00bfc-24bf-43f9-ac2f-3a6a5e2e24d6 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:16.823043 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b0d00bfc-24bf-43f9-ac2f-3a6a5e2e24d6 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:17.378366 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-b0d00bfc-24bf-43f9-ac2f-3a6a5e2e24d6 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ddfae3448704591b662de83b97a1876', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bed7c94624484c7cb8abeb3bfc8757b9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:32:17.396746 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-b0d00bfc-24bf-43f9-ac2f-3a6a5e2e24d6 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] 10.222.0.16 "GET /compute/v2.1/servers/88cf13fc-0fa3-4e1e-9d76-db4d93b49faf" status: 200 len: 1373 microversion: 2.1 time: 0.601400 Jul 16 21:32:17.410146 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2328/4652] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:32:16 2022] GET /compute/v2.1/servers/88cf13fc-0fa3-4e1e-9d76-db4d93b49faf => generated 1373 bytes in 617 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:32:18.415347 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-6d96ff08-4a8a-4c28-88f0-678858f97bca tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:32:18.416072 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-6d96ff08-4a8a-4c28-88f0-678858f97bca tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] [instance: 88cf13fc-0fa3-4e1e-9d76-db4d93b49faf] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:32:18.422681 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6d96ff08-4a8a-4c28-88f0-678858f97bca tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:18.430503 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6d96ff08-4a8a-4c28-88f0-678858f97bca tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:18.431505 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6d96ff08-4a8a-4c28-88f0-678858f97bca tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:18.431988 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6d96ff08-4a8a-4c28-88f0-678858f97bca tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:18.814934 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-6d96ff08-4a8a-4c28-88f0-678858f97bca tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ddfae3448704591b662de83b97a1876', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bed7c94624484c7cb8abeb3bfc8757b9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:32:18.827733 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-6d96ff08-4a8a-4c28-88f0-678858f97bca tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] 10.222.0.16 "GET /compute/v2.1/servers/88cf13fc-0fa3-4e1e-9d76-db4d93b49faf" status: 200 len: 1373 microversion: 2.1 time: 0.417824 Jul 16 21:32:18.828644 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2329/4653] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:32:18 2022] GET /compute/v2.1/servers/88cf13fc-0fa3-4e1e-9d76-db4d93b49faf => generated 1373 bytes in 420 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:32:18.850587 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-5cd04467-04f5-4865-bd10-5aa8adc2a3c2 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Flavor limits 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 16 21:32:18.851251 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-5cd04467-04f5-4865-bd10-5aa8adc2a3c2 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Image limits 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 16 21:32:18.851581 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-5cd04467-04f5-4865-bd10-5aa8adc2a3c2 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Flavor pref 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 16 21:32:18.852964 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-5cd04467-04f5-4865-bd10-5aa8adc2a3c2 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Image pref 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 16 21:32:18.852964 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-5cd04467-04f5-4865-bd10-5aa8adc2a3c2 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 16 21:32:18.869458 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5cd04467-04f5-4865-bd10-5aa8adc2a3c2 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:18.873575 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5cd04467-04f5-4865-bd10-5aa8adc2a3c2 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:18.873575 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5cd04467-04f5-4865-bd10-5aa8adc2a3c2 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:18.873575 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5cd04467-04f5-4865-bd10-5aa8adc2a3c2 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:18.881650 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=80 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:32:18.895609 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-5cd04467-04f5-4865-bd10-5aa8adc2a3c2 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Getting quotas for project 367c69301e6a44419af5c2d28b616b80. Resources: {'ram', 'instances', 'cores'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:32:18.904925 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-5cd04467-04f5-4865-bd10-5aa8adc2a3c2 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Getting quotas for user 08abcba4284e434680ee4640f1541cf9 and project 367c69301e6a44419af5c2d28b616b80. Resources: {'ram', 'instances', 'cores'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:32:18.925618 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-5cd04467-04f5-4865-bd10-5aa8adc2a3c2 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Going to run 1 instances... {{(pid=138680) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 16 21:32:18.929146 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-5cd04467-04f5-4865-bd10-5aa8adc2a3c2 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] [instance: 63bbc7c6-1f26-4d17-bc3e-01f31f1fa939] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138680) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 16 21:32:19.086846 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-5cd04467-04f5-4865-bd10-5aa8adc2a3c2 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.26 time: 2.916151 Jul 16 21:32:19.088936 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2325/4654] 10.222.0.16 () {62 vars in 1187 bytes} [Sat Jul 16 21:32:16 2022] POST /compute/v2.1/servers => generated 380 bytes in 2919 msecs (HTTP/1.1 202) 10 headers in 453 bytes (1 switches on core 0) Jul 16 21:32:19.101011 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-6c5c7784-ac38-4d46-8893-9a584a794e69 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:32:19.104792 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-6c5c7784-ac38-4d46-8893-9a584a794e69 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] [instance: 63bbc7c6-1f26-4d17-bc3e-01f31f1fa939] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:32:19.274465 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-6c5c7784-ac38-4d46-8893-9a584a794e69 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '08abcba4284e434680ee4640f1541cf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '367c69301e6a44419af5c2d28b616b80', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:32:19.286558 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-6c5c7784-ac38-4d46-8893-9a584a794e69 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '08abcba4284e434680ee4640f1541cf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '367c69301e6a44419af5c2d28b616b80', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:32:19.290669 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-6c5c7784-ac38-4d46-8893-9a584a794e69 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '08abcba4284e434680ee4640f1541cf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '367c69301e6a44419af5c2d28b616b80', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:32:19.291951 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-6c5c7784-ac38-4d46-8893-9a584a794e69 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] 10.222.0.16 "GET /compute/v2.1/servers/63bbc7c6-1f26-4d17-bc3e-01f31f1fa939" status: 200 len: 1262 microversion: 2.26 time: 0.196923 Jul 16 21:32:19.295702 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2330/4655] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:32:19 2022] GET /compute/v2.1/servers/63bbc7c6-1f26-4d17-bc3e-01f31f1fa939 => generated 1262 bytes in 201 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:32:19.865743 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-0051989c-2a18-402b-a87b-1daa5b060f86 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:32:19.867819 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-0051989c-2a18-402b-a87b-1daa5b060f86 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] [instance: 88cf13fc-0fa3-4e1e-9d76-db4d93b49faf] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:32:19.888714 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0051989c-2a18-402b-a87b-1daa5b060f86 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:19.889184 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0051989c-2a18-402b-a87b-1daa5b060f86 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:19.890012 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0051989c-2a18-402b-a87b-1daa5b060f86 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:19.899434 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0051989c-2a18-402b-a87b-1daa5b060f86 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:20.330646 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-75c9b4fa-73a6-4502-9e48-465a8adfda01 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:32:20.332676 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-75c9b4fa-73a6-4502-9e48-465a8adfda01 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] [instance: 63bbc7c6-1f26-4d17-bc3e-01f31f1fa939] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:32:20.446691 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-0051989c-2a18-402b-a87b-1daa5b060f86 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ddfae3448704591b662de83b97a1876', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bed7c94624484c7cb8abeb3bfc8757b9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:32:20.461491 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-0051989c-2a18-402b-a87b-1daa5b060f86 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] 10.222.0.16 "GET /compute/v2.1/servers/88cf13fc-0fa3-4e1e-9d76-db4d93b49faf" status: 200 len: 1373 microversion: 2.1 time: 0.616791 Jul 16 21:32:20.461993 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2326/4656] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:32:19 2022] GET /compute/v2.1/servers/88cf13fc-0fa3-4e1e-9d76-db4d93b49faf => generated 1373 bytes in 618 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:32:20.505559 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-75c9b4fa-73a6-4502-9e48-465a8adfda01 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '08abcba4284e434680ee4640f1541cf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '367c69301e6a44419af5c2d28b616b80', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:32:20.513989 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-75c9b4fa-73a6-4502-9e48-465a8adfda01 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '08abcba4284e434680ee4640f1541cf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '367c69301e6a44419af5c2d28b616b80', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:32:20.516109 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-75c9b4fa-73a6-4502-9e48-465a8adfda01 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '08abcba4284e434680ee4640f1541cf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '367c69301e6a44419af5c2d28b616b80', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:32:20.521645 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-75c9b4fa-73a6-4502-9e48-465a8adfda01 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] 10.222.0.16 "GET /compute/v2.1/servers/63bbc7c6-1f26-4d17-bc3e-01f31f1fa939" status: 200 len: 1262 microversion: 2.26 time: 0.203984 Jul 16 21:32:20.522388 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2331/4657] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:32:20 2022] GET /compute/v2.1/servers/63bbc7c6-1f26-4d17-bc3e-01f31f1fa939 => generated 1262 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:32:21.481096 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-c9f657b5-bd09-4e88-997b-7917c68a9dad tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:32:21.481863 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-c9f657b5-bd09-4e88-997b-7917c68a9dad tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] [instance: 88cf13fc-0fa3-4e1e-9d76-db4d93b49faf] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:32:21.493065 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c9f657b5-bd09-4e88-997b-7917c68a9dad tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:21.493846 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c9f657b5-bd09-4e88-997b-7917c68a9dad tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:21.494990 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c9f657b5-bd09-4e88-997b-7917c68a9dad tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:21.496979 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c9f657b5-bd09-4e88-997b-7917c68a9dad tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:21.542031 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-e04fd64b-3930-4a6f-9b10-36a04297d2d0 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:32:21.542826 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-e04fd64b-3930-4a6f-9b10-36a04297d2d0 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] [instance: 63bbc7c6-1f26-4d17-bc3e-01f31f1fa939] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:32:21.557125 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e04fd64b-3930-4a6f-9b10-36a04297d2d0 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:21.557961 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e04fd64b-3930-4a6f-9b10-36a04297d2d0 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:21.563816 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e04fd64b-3930-4a6f-9b10-36a04297d2d0 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:21.566763 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e04fd64b-3930-4a6f-9b10-36a04297d2d0 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:22.039236 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-e04fd64b-3930-4a6f-9b10-36a04297d2d0 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '08abcba4284e434680ee4640f1541cf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '367c69301e6a44419af5c2d28b616b80', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:32:22.052152 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-e04fd64b-3930-4a6f-9b10-36a04297d2d0 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '08abcba4284e434680ee4640f1541cf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '367c69301e6a44419af5c2d28b616b80', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:32:22.053938 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-e04fd64b-3930-4a6f-9b10-36a04297d2d0 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '08abcba4284e434680ee4640f1541cf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '367c69301e6a44419af5c2d28b616b80', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:32:22.057016 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-e04fd64b-3930-4a6f-9b10-36a04297d2d0 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] 10.222.0.16 "GET /compute/v2.1/servers/63bbc7c6-1f26-4d17-bc3e-01f31f1fa939" status: 200 len: 1258 microversion: 2.26 time: 0.519606 Jul 16 21:32:22.059827 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2332/4658] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:32:21 2022] GET /compute/v2.1/servers/63bbc7c6-1f26-4d17-bc3e-01f31f1fa939 => generated 1258 bytes in 523 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:32:22.078662 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-70f670c6-0690-4a9f-bb63-111ead87c49d req-0db61915-0803-4b0f-b27b-6b51508332b0 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "88cf13fc-0fa3-4e1e-9d76-db4d93b49faf", "name": "network-vif-plugged", "status": "completed", "tag": "008d375b-5b42-4275-ac14-31bcaa9250a8"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:32:22.088548 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-70f670c6-0690-4a9f-bb63-111ead87c49d req-0db61915-0803-4b0f-b27b-6b51508332b0 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:22.090957 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-70f670c6-0690-4a9f-bb63-111ead87c49d req-0db61915-0803-4b0f-b27b-6b51508332b0 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:22.122783 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-70f670c6-0690-4a9f-bb63-111ead87c49d req-0db61915-0803-4b0f-b27b-6b51508332b0 service nova] Creating event network-vif-plugged:008d375b-5b42-4275-ac14-31bcaa9250a8 for instance 88cf13fc-0fa3-4e1e-9d76-db4d93b49faf on nch1-master-1 Jul 16 21:32:22.135341 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-70f670c6-0690-4a9f-bb63-111ead87c49d req-0db61915-0803-4b0f-b27b-6b51508332b0 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.070007 Jul 16 21:32:22.139289 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2333/4659] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:32:22 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 79 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:32:22.283789 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-c9f657b5-bd09-4e88-997b-7917c68a9dad tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ddfae3448704591b662de83b97a1876', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bed7c94624484c7cb8abeb3bfc8757b9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:32:22.304015 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-c9f657b5-bd09-4e88-997b-7917c68a9dad tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] 10.222.0.16 "GET /compute/v2.1/servers/88cf13fc-0fa3-4e1e-9d76-db4d93b49faf" status: 200 len: 1373 microversion: 2.1 time: 0.828218 Jul 16 21:32:22.308655 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2327/4660] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:32:21 2022] GET /compute/v2.1/servers/88cf13fc-0fa3-4e1e-9d76-db4d93b49faf => generated 1373 bytes in 833 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:32:23.090062 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-4e50eb65-7e50-49b9-a8fe-b0f8cc476e3a tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:32:23.091561 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-4e50eb65-7e50-49b9-a8fe-b0f8cc476e3a tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] [instance: 63bbc7c6-1f26-4d17-bc3e-01f31f1fa939] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:32:23.106324 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4e50eb65-7e50-49b9-a8fe-b0f8cc476e3a tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:23.107074 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4e50eb65-7e50-49b9-a8fe-b0f8cc476e3a tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:23.108049 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4e50eb65-7e50-49b9-a8fe-b0f8cc476e3a tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:23.109039 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4e50eb65-7e50-49b9-a8fe-b0f8cc476e3a tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:23.324381 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-76e2b23a-720b-4d37-a89d-a5f3aee4a279 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:32:23.325274 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-76e2b23a-720b-4d37-a89d-a5f3aee4a279 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] [instance: 88cf13fc-0fa3-4e1e-9d76-db4d93b49faf] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:32:23.336084 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-76e2b23a-720b-4d37-a89d-a5f3aee4a279 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:23.338123 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-76e2b23a-720b-4d37-a89d-a5f3aee4a279 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:23.344144 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-76e2b23a-720b-4d37-a89d-a5f3aee4a279 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:23.344996 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-76e2b23a-720b-4d37-a89d-a5f3aee4a279 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:23.361935 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-4e50eb65-7e50-49b9-a8fe-b0f8cc476e3a tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '08abcba4284e434680ee4640f1541cf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '367c69301e6a44419af5c2d28b616b80', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:32:23.377955 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-4e50eb65-7e50-49b9-a8fe-b0f8cc476e3a tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '08abcba4284e434680ee4640f1541cf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '367c69301e6a44419af5c2d28b616b80', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:32:23.414311 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-4e50eb65-7e50-49b9-a8fe-b0f8cc476e3a tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '08abcba4284e434680ee4640f1541cf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '367c69301e6a44419af5c2d28b616b80', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:32:23.418001 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-4e50eb65-7e50-49b9-a8fe-b0f8cc476e3a tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] 10.222.0.16 "GET /compute/v2.1/servers/63bbc7c6-1f26-4d17-bc3e-01f31f1fa939" status: 200 len: 1332 microversion: 2.26 time: 0.332974 Jul 16 21:32:23.422419 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2334/4661] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:32:23 2022] GET /compute/v2.1/servers/63bbc7c6-1f26-4d17-bc3e-01f31f1fa939 => generated 1332 bytes in 338 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:32:23.976788 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-76e2b23a-720b-4d37-a89d-a5f3aee4a279 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ddfae3448704591b662de83b97a1876', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bed7c94624484c7cb8abeb3bfc8757b9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:32:23.985543 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-76e2b23a-720b-4d37-a89d-a5f3aee4a279 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] 10.222.0.16 "GET /compute/v2.1/servers/88cf13fc-0fa3-4e1e-9d76-db4d93b49faf" status: 200 len: 1373 microversion: 2.1 time: 0.666781 Jul 16 21:32:23.986635 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2328/4662] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:32:23 2022] GET /compute/v2.1/servers/88cf13fc-0fa3-4e1e-9d76-db4d93b49faf => generated 1373 bytes in 668 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:32:24.440633 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-36561a59-8bf4-47d5-8e4c-29b209803fcb tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:32:24.441902 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-36561a59-8bf4-47d5-8e4c-29b209803fcb tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] [instance: 63bbc7c6-1f26-4d17-bc3e-01f31f1fa939] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:32:24.450787 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-36561a59-8bf4-47d5-8e4c-29b209803fcb tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:24.454078 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-36561a59-8bf4-47d5-8e4c-29b209803fcb tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:24.458054 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-36561a59-8bf4-47d5-8e4c-29b209803fcb tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:24.458669 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-36561a59-8bf4-47d5-8e4c-29b209803fcb tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:24.608396 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-36561a59-8bf4-47d5-8e4c-29b209803fcb tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '08abcba4284e434680ee4640f1541cf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '367c69301e6a44419af5c2d28b616b80', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:32:24.625951 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-36561a59-8bf4-47d5-8e4c-29b209803fcb tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '08abcba4284e434680ee4640f1541cf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '367c69301e6a44419af5c2d28b616b80', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:32:24.630233 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-36561a59-8bf4-47d5-8e4c-29b209803fcb tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '08abcba4284e434680ee4640f1541cf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '367c69301e6a44419af5c2d28b616b80', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:32:24.634631 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-36561a59-8bf4-47d5-8e4c-29b209803fcb tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] 10.222.0.16 "GET /compute/v2.1/servers/63bbc7c6-1f26-4d17-bc3e-01f31f1fa939" status: 200 len: 1320 microversion: 2.26 time: 0.195529 Jul 16 21:32:24.636582 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2335/4663] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:32:24 2022] GET /compute/v2.1/servers/63bbc7c6-1f26-4d17-bc3e-01f31f1fa939 => generated 1320 bytes in 201 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:32:25.003863 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-3678820d-7b17-41b2-b76e-907a26beee17 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:32:25.004692 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-3678820d-7b17-41b2-b76e-907a26beee17 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] [instance: 88cf13fc-0fa3-4e1e-9d76-db4d93b49faf] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:32:25.014176 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3678820d-7b17-41b2-b76e-907a26beee17 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:25.015447 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3678820d-7b17-41b2-b76e-907a26beee17 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:25.015447 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3678820d-7b17-41b2-b76e-907a26beee17 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:25.015866 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3678820d-7b17-41b2-b76e-907a26beee17 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:25.358133 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-3678820d-7b17-41b2-b76e-907a26beee17 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ddfae3448704591b662de83b97a1876', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bed7c94624484c7cb8abeb3bfc8757b9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:32:25.373361 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-3678820d-7b17-41b2-b76e-907a26beee17 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] 10.222.0.16 "GET /compute/v2.1/servers/88cf13fc-0fa3-4e1e-9d76-db4d93b49faf" status: 200 len: 1545 microversion: 2.1 time: 0.373975 Jul 16 21:32:25.374591 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2329/4664] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:32:24 2022] GET /compute/v2.1/servers/88cf13fc-0fa3-4e1e-9d76-db4d93b49faf => generated 1545 bytes in 376 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:32:25.392286 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-9862813f-e60b-4221-9296-af214cd78a26 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:32:25.393389 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-9862813f-e60b-4221-9296-af214cd78a26 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] [instance: 88cf13fc-0fa3-4e1e-9d76-db4d93b49faf] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:32:25.404428 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9862813f-e60b-4221-9296-af214cd78a26 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:25.404888 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9862813f-e60b-4221-9296-af214cd78a26 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:25.405682 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9862813f-e60b-4221-9296-af214cd78a26 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:25.406128 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9862813f-e60b-4221-9296-af214cd78a26 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:25.662403 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-ea9ee258-6daf-42d0-bc16-f1b14a727294 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:32:25.663061 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-ea9ee258-6daf-42d0-bc16-f1b14a727294 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] [instance: 63bbc7c6-1f26-4d17-bc3e-01f31f1fa939] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:32:25.670522 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ea9ee258-6daf-42d0-bc16-f1b14a727294 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:25.671477 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ea9ee258-6daf-42d0-bc16-f1b14a727294 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:25.672528 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ea9ee258-6daf-42d0-bc16-f1b14a727294 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:25.673271 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ea9ee258-6daf-42d0-bc16-f1b14a727294 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:25.744225 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-9862813f-e60b-4221-9296-af214cd78a26 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ddfae3448704591b662de83b97a1876', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bed7c94624484c7cb8abeb3bfc8757b9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:32:25.755169 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-9862813f-e60b-4221-9296-af214cd78a26 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] 10.222.0.16 "GET /compute/v2.1/servers/88cf13fc-0fa3-4e1e-9d76-db4d93b49faf" status: 200 len: 1545 microversion: 2.1 time: 0.367096 Jul 16 21:32:25.756751 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2336/4665] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:32:25 2022] GET /compute/v2.1/servers/88cf13fc-0fa3-4e1e-9d76-db4d93b49faf => generated 1545 bytes in 369 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:32:26.471168 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-ea9ee258-6daf-42d0-bc16-f1b14a727294 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '08abcba4284e434680ee4640f1541cf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '367c69301e6a44419af5c2d28b616b80', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:32:26.482847 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-ea9ee258-6daf-42d0-bc16-f1b14a727294 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '08abcba4284e434680ee4640f1541cf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '367c69301e6a44419af5c2d28b616b80', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:32:26.485564 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-ea9ee258-6daf-42d0-bc16-f1b14a727294 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '08abcba4284e434680ee4640f1541cf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '367c69301e6a44419af5c2d28b616b80', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:32:26.487148 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-ea9ee258-6daf-42d0-bc16-f1b14a727294 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] 10.222.0.16 "GET /compute/v2.1/servers/63bbc7c6-1f26-4d17-bc3e-01f31f1fa939" status: 200 len: 1362 microversion: 2.26 time: 0.829298 Jul 16 21:32:26.499246 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2330/4666] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:32:25 2022] GET /compute/v2.1/servers/63bbc7c6-1f26-4d17-bc3e-01f31f1fa939 => generated 1362 bytes in 842 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 16 21:32:27.510603 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-97a898fd-123d-408a-b9e2-406bd33d2521 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:32:27.511517 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-97a898fd-123d-408a-b9e2-406bd33d2521 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] [instance: 63bbc7c6-1f26-4d17-bc3e-01f31f1fa939] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:32:27.523336 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-97a898fd-123d-408a-b9e2-406bd33d2521 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:27.524455 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-97a898fd-123d-408a-b9e2-406bd33d2521 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:27.525515 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-97a898fd-123d-408a-b9e2-406bd33d2521 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:27.526148 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-97a898fd-123d-408a-b9e2-406bd33d2521 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:27.987132 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-97a898fd-123d-408a-b9e2-406bd33d2521 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '08abcba4284e434680ee4640f1541cf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '367c69301e6a44419af5c2d28b616b80', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:32:28.026371 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-97a898fd-123d-408a-b9e2-406bd33d2521 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '08abcba4284e434680ee4640f1541cf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '367c69301e6a44419af5c2d28b616b80', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:32:28.030236 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-97a898fd-123d-408a-b9e2-406bd33d2521 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '08abcba4284e434680ee4640f1541cf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '367c69301e6a44419af5c2d28b616b80', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:32:28.031501 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-97a898fd-123d-408a-b9e2-406bd33d2521 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] 10.222.0.16 "GET /compute/v2.1/servers/63bbc7c6-1f26-4d17-bc3e-01f31f1fa939" status: 200 len: 1362 microversion: 2.26 time: 0.526225 Jul 16 21:32:28.032072 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2337/4667] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:32:27 2022] GET /compute/v2.1/servers/63bbc7c6-1f26-4d17-bc3e-01f31f1fa939 => generated 1362 bytes in 528 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:32:28.146181 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-0d31e910-472a-4272-8f47-c6df6c145c15 req-a0a317a1-aaf9-49c4-bfc1-f71f335d8f63 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "63bbc7c6-1f26-4d17-bc3e-01f31f1fa939", "tag": "839ad50f-8742-447d-b523-395de732cb18"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:32:28.174886 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-0d31e910-472a-4272-8f47-c6df6c145c15 req-a0a317a1-aaf9-49c4-bfc1-f71f335d8f63 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:28.177019 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-0d31e910-472a-4272-8f47-c6df6c145c15 req-a0a317a1-aaf9-49c4-bfc1-f71f335d8f63 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.014s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:28.217482 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-0d31e910-472a-4272-8f47-c6df6c145c15 req-a0a317a1-aaf9-49c4-bfc1-f71f335d8f63 service nova] Creating event network-changed:839ad50f-8742-447d-b523-395de732cb18 for instance 63bbc7c6-1f26-4d17-bc3e-01f31f1fa939 on nch2-master-1 Jul 16 21:32:28.249388 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-0d31e910-472a-4272-8f47-c6df6c145c15 req-a0a317a1-aaf9-49c4-bfc1-f71f335d8f63 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.109795 Jul 16 21:32:28.250575 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2331/4668] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:32:28 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 112 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:32:28.701948 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-de819174-0ad1-422a-a228-6a58eeaf10a8 tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] Action: 'create', calling method: .version_select at 0x7fdd8ff4ea60>, body: {"keypair": {"name": "tempest-TestServerBasicOps-956155148", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBHNzYDYvUl4quNJG/ochzNNJKooXjf0uluygzUTUzi/celrsWujMMyhNOliL5OIdQL68pQNwWrbx+fxqVKf1s4ubrG7WRk8bw2QEXRub36d6rjgWzOsHncjjS03C3RaZEg=="}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:32:28.719529 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-de819174-0ad1-422a-a228-6a58eeaf10a8 tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] Getting quotas for project ce30584a7dfa4ab19de73aa8b18b113b. Resources: {'key_pairs'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:32:28.728242 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-de819174-0ad1-422a-a228-6a58eeaf10a8 tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] Getting quotas for user 4fa446573f7d42de816280280c3dec19 and project ce30584a7dfa4ab19de73aa8b18b113b. Resources: {'key_pairs'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:32:28.817781 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-de819174-0ad1-422a-a228-6a58eeaf10a8 tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] 10.222.0.16 "POST /compute/v2.1/os-keypairs" status: 200 len: 396 microversion: 2.1 time: 0.365499 Jul 16 21:32:28.818537 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2338/4669] 10.222.0.16 () {60 vars in 1158 bytes} [Sat Jul 16 21:32:28 2022] POST /compute/v2.1/os-keypairs => generated 396 bytes in 370 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:32:29.053305 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-569fabe3-3295-4d2b-a1be-3b2713e208df tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:32:29.054042 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-569fabe3-3295-4d2b-a1be-3b2713e208df tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] [instance: 63bbc7c6-1f26-4d17-bc3e-01f31f1fa939] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:32:29.056147 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell0:SELECT=2 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:32:29.056838 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell0:SELECT=6 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:32:29.082899 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-569fabe3-3295-4d2b-a1be-3b2713e208df tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:29.084960 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-569fabe3-3295-4d2b-a1be-3b2713e208df tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:29.086666 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-569fabe3-3295-4d2b-a1be-3b2713e208df tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:29.088515 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-569fabe3-3295-4d2b-a1be-3b2713e208df tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:29.374618 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-a34eca23-1050-4440-ac57-c595924ab8e8 req-9e547cca-7304-47e6-8885-43314f2a42bf service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "88cf13fc-0fa3-4e1e-9d76-db4d93b49faf", "tag": "008d375b-5b42-4275-ac14-31bcaa9250a8"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:32:29.383857 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-a34eca23-1050-4440-ac57-c595924ab8e8 req-9e547cca-7304-47e6-8885-43314f2a42bf service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:29.384878 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-a34eca23-1050-4440-ac57-c595924ab8e8 req-9e547cca-7304-47e6-8885-43314f2a42bf service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:29.397033 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-a34eca23-1050-4440-ac57-c595924ab8e8 req-9e547cca-7304-47e6-8885-43314f2a42bf service nova] Creating event network-changed:008d375b-5b42-4275-ac14-31bcaa9250a8 for instance 88cf13fc-0fa3-4e1e-9d76-db4d93b49faf on nch1-master-1 Jul 16 21:32:29.403399 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-a34eca23-1050-4440-ac57-c595924ab8e8 req-9e547cca-7304-47e6-8885-43314f2a42bf service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.031912 Jul 16 21:32:29.407387 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2339/4670] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:32:29 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 16 21:32:29.440918 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-569fabe3-3295-4d2b-a1be-3b2713e208df tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '08abcba4284e434680ee4640f1541cf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '367c69301e6a44419af5c2d28b616b80', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:32:29.450869 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-569fabe3-3295-4d2b-a1be-3b2713e208df tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '08abcba4284e434680ee4640f1541cf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '367c69301e6a44419af5c2d28b616b80', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:32:29.452782 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-569fabe3-3295-4d2b-a1be-3b2713e208df tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '08abcba4284e434680ee4640f1541cf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '367c69301e6a44419af5c2d28b616b80', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:32:29.454582 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-569fabe3-3295-4d2b-a1be-3b2713e208df tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] 10.222.0.16 "GET /compute/v2.1/servers/63bbc7c6-1f26-4d17-bc3e-01f31f1fa939" status: 200 len: 1362 microversion: 2.26 time: 0.407295 Jul 16 21:32:29.455692 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2332/4671] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:32:29 2022] GET /compute/v2.1/servers/63bbc7c6-1f26-4d17-bc3e-01f31f1fa939 => generated 1362 bytes in 409 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:32:30.478600 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-786220bc-7466-47ff-bf5b-b7ee1c2d5bb9 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:32:30.479936 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-786220bc-7466-47ff-bf5b-b7ee1c2d5bb9 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] [instance: 63bbc7c6-1f26-4d17-bc3e-01f31f1fa939] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:32:30.487891 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-786220bc-7466-47ff-bf5b-b7ee1c2d5bb9 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:30.488698 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-786220bc-7466-47ff-bf5b-b7ee1c2d5bb9 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:30.489665 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-786220bc-7466-47ff-bf5b-b7ee1c2d5bb9 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:30.490311 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-786220bc-7466-47ff-bf5b-b7ee1c2d5bb9 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:31.291509 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-786220bc-7466-47ff-bf5b-b7ee1c2d5bb9 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '08abcba4284e434680ee4640f1541cf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '367c69301e6a44419af5c2d28b616b80', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:32:31.326617 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-786220bc-7466-47ff-bf5b-b7ee1c2d5bb9 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '08abcba4284e434680ee4640f1541cf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '367c69301e6a44419af5c2d28b616b80', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:32:31.336561 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-786220bc-7466-47ff-bf5b-b7ee1c2d5bb9 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '08abcba4284e434680ee4640f1541cf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '367c69301e6a44419af5c2d28b616b80', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:32:31.343137 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-786220bc-7466-47ff-bf5b-b7ee1c2d5bb9 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] 10.222.0.16 "GET /compute/v2.1/servers/63bbc7c6-1f26-4d17-bc3e-01f31f1fa939" status: 200 len: 1362 microversion: 2.26 time: 0.872881 Jul 16 21:32:31.366034 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2340/4672] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:32:30 2022] GET /compute/v2.1/servers/63bbc7c6-1f26-4d17-bc3e-01f31f1fa939 => generated 1362 bytes in 893 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:32:32.370314 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-61e95253-fead-4123-b38f-fc27728e3afc tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:32:32.370945 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-61e95253-fead-4123-b38f-fc27728e3afc tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] [instance: 63bbc7c6-1f26-4d17-bc3e-01f31f1fa939] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:32:32.376031 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_api:INSERT=4,nova_api:SELECT=52 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:32:32.393153 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-61e95253-fead-4123-b38f-fc27728e3afc tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:32.394220 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-61e95253-fead-4123-b38f-fc27728e3afc tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:32.395240 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-61e95253-fead-4123-b38f-fc27728e3afc tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:32.395893 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-61e95253-fead-4123-b38f-fc27728e3afc tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:32.647352 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-61e95253-fead-4123-b38f-fc27728e3afc tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '08abcba4284e434680ee4640f1541cf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '367c69301e6a44419af5c2d28b616b80', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:32:32.656226 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-61e95253-fead-4123-b38f-fc27728e3afc tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '08abcba4284e434680ee4640f1541cf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '367c69301e6a44419af5c2d28b616b80', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:32:32.660750 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-61e95253-fead-4123-b38f-fc27728e3afc tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '08abcba4284e434680ee4640f1541cf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '367c69301e6a44419af5c2d28b616b80', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:32:32.662228 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-61e95253-fead-4123-b38f-fc27728e3afc tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] 10.222.0.16 "GET /compute/v2.1/servers/63bbc7c6-1f26-4d17-bc3e-01f31f1fa939" status: 200 len: 1362 microversion: 2.26 time: 0.296656 Jul 16 21:32:32.663209 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2333/4673] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:32:32 2022] GET /compute/v2.1/servers/63bbc7c6-1f26-4d17-bc3e-01f31f1fa939 => generated 1362 bytes in 298 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:32:33.680896 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-fbb378e4-f868-4276-822b-cfe2a1921a5c tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:32:33.681525 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-fbb378e4-f868-4276-822b-cfe2a1921a5c tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] [instance: 63bbc7c6-1f26-4d17-bc3e-01f31f1fa939] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:32:33.688431 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fbb378e4-f868-4276-822b-cfe2a1921a5c tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:33.688890 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fbb378e4-f868-4276-822b-cfe2a1921a5c tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:33.691475 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fbb378e4-f868-4276-822b-cfe2a1921a5c tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:33.691940 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fbb378e4-f868-4276-822b-cfe2a1921a5c tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:33.962711 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-fbb378e4-f868-4276-822b-cfe2a1921a5c tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '08abcba4284e434680ee4640f1541cf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '367c69301e6a44419af5c2d28b616b80', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:32:33.970980 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-fbb378e4-f868-4276-822b-cfe2a1921a5c tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '08abcba4284e434680ee4640f1541cf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '367c69301e6a44419af5c2d28b616b80', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:32:33.973081 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-fbb378e4-f868-4276-822b-cfe2a1921a5c tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '08abcba4284e434680ee4640f1541cf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '367c69301e6a44419af5c2d28b616b80', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:32:33.974496 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-fbb378e4-f868-4276-822b-cfe2a1921a5c tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] 10.222.0.16 "GET /compute/v2.1/servers/63bbc7c6-1f26-4d17-bc3e-01f31f1fa939" status: 200 len: 1362 microversion: 2.26 time: 0.297736 Jul 16 21:32:33.975086 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2341/4674] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:32:33 2022] GET /compute/v2.1/servers/63bbc7c6-1f26-4d17-bc3e-01f31f1fa939 => generated 1362 bytes in 299 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:32:34.228348 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-f23d926c-63e4-41d3-ab6b-756f87861ad1 tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestServerBasicOps-server-1186175461", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452", "key_name": "tempest-TestServerBasicOps-956155148", "security_groups": [{"name": "tempest-secgroup-smoke-431115029"}], "config_drive": true, "metadata": {"meta1": "data1", "meta2": "data2", "metaN": "dataN"}, "networks": [{"uuid": "3367c281-d532-4c5f-bcef-2b2cc8cd37fa"}]}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:32:34.395297 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.network.neutron [None req-f23d926c-63e4-41d3-ab6b-756f87861ad1 tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] validate_networks() for [('3367c281-d532-4c5f-bcef-2b2cc8cd37fa', None, None, None, None, None)] {{(pid=138680) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 16 21:32:34.856230 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-f23d926c-63e4-41d3-ab6b-756f87861ad1 tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] Getting quotas for project ce30584a7dfa4ab19de73aa8b18b113b. Resources: dict_keys(['metadata_items']) {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:32:34.865327 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-f23d926c-63e4-41d3-ab6b-756f87861ad1 tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] Getting quotas for user 4fa446573f7d42de816280280c3dec19 and project ce30584a7dfa4ab19de73aa8b18b113b. Resources: dict_keys(['metadata_items']) {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:32:34.898912 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-f23d926c-63e4-41d3-ab6b-756f87861ad1 tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] Flavor limits 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 16 21:32:34.899978 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-f23d926c-63e4-41d3-ab6b-756f87861ad1 tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] Image limits 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 16 21:32:34.900839 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-f23d926c-63e4-41d3-ab6b-756f87861ad1 tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] Flavor pref 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 16 21:32:34.901573 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-f23d926c-63e4-41d3-ab6b-756f87861ad1 tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] Image pref 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 16 21:32:34.902382 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-f23d926c-63e4-41d3-ab6b-756f87861ad1 tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 16 21:32:34.917817 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f23d926c-63e4-41d3-ab6b-756f87861ad1 tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:34.918476 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f23d926c-63e4-41d3-ab6b-756f87861ad1 tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:34.919910 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f23d926c-63e4-41d3-ab6b-756f87861ad1 tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:34.920762 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f23d926c-63e4-41d3-ab6b-756f87861ad1 tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:34.948126 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-f23d926c-63e4-41d3-ab6b-756f87861ad1 tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] Getting quotas for project ce30584a7dfa4ab19de73aa8b18b113b. Resources: {'ram', 'instances', 'cores'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:32:34.956634 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-f23d926c-63e4-41d3-ab6b-756f87861ad1 tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] Getting quotas for user 4fa446573f7d42de816280280c3dec19 and project ce30584a7dfa4ab19de73aa8b18b113b. Resources: {'ram', 'instances', 'cores'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:32:34.977972 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-f23d926c-63e4-41d3-ab6b-756f87861ad1 tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] Going to run 1 instances... {{(pid=138680) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 16 21:32:34.981583 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-f23d926c-63e4-41d3-ab6b-756f87861ad1 tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] [instance: 3c0f0855-7a9c-4d8a-95ac-3bf41d32d4e6] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138680) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 16 21:32:34.999598 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-709a06c0-0ba2-44f7-8cf1-dc6f99119d86 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:32:34.999598 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-709a06c0-0ba2-44f7-8cf1-dc6f99119d86 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] [instance: 63bbc7c6-1f26-4d17-bc3e-01f31f1fa939] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:32:35.024979 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-709a06c0-0ba2-44f7-8cf1-dc6f99119d86 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:35.028273 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-709a06c0-0ba2-44f7-8cf1-dc6f99119d86 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:35.028273 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-709a06c0-0ba2-44f7-8cf1-dc6f99119d86 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:35.029144 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-709a06c0-0ba2-44f7-8cf1-dc6f99119d86 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:35.212508 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-f23d926c-63e4-41d3-ab6b-756f87861ad1 tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 405 microversion: 2.1 time: 0.988779 Jul 16 21:32:35.217296 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2334/4675] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 21:32:34 2022] POST /compute/v2.1/servers => generated 405 bytes in 991 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 16 21:32:35.222578 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-e872232b-fb72-4a6d-bf00-93784bb18186 req-66b77ae8-2f84-489c-8124-b2d673c5e959 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "63bbc7c6-1f26-4d17-bc3e-01f31f1fa939", "name": "network-vif-plugged", "status": "completed", "tag": "839ad50f-8742-447d-b523-395de732cb18"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:32:35.235296 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-e872232b-fb72-4a6d-bf00-93784bb18186 req-66b77ae8-2f84-489c-8124-b2d673c5e959 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:35.239400 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-709a06c0-0ba2-44f7-8cf1-dc6f99119d86 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '08abcba4284e434680ee4640f1541cf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '367c69301e6a44419af5c2d28b616b80', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:32:35.244425 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-e872232b-fb72-4a6d-bf00-93784bb18186 req-66b77ae8-2f84-489c-8124-b2d673c5e959 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.009s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:35.254388 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-709a06c0-0ba2-44f7-8cf1-dc6f99119d86 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '08abcba4284e434680ee4640f1541cf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '367c69301e6a44419af5c2d28b616b80', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:32:35.262385 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-709a06c0-0ba2-44f7-8cf1-dc6f99119d86 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '08abcba4284e434680ee4640f1541cf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '367c69301e6a44419af5c2d28b616b80', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:32:35.262385 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-709a06c0-0ba2-44f7-8cf1-dc6f99119d86 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] 10.222.0.16 "GET /compute/v2.1/servers/63bbc7c6-1f26-4d17-bc3e-01f31f1fa939" status: 200 len: 1362 microversion: 2.26 time: 0.271722 Jul 16 21:32:35.262385 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2342/4676] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:32:34 2022] GET /compute/v2.1/servers/63bbc7c6-1f26-4d17-bc3e-01f31f1fa939 => generated 1362 bytes in 273 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:32:35.275847 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-e872232b-fb72-4a6d-bf00-93784bb18186 req-66b77ae8-2f84-489c-8124-b2d673c5e959 service nova] Creating event network-vif-plugged:839ad50f-8742-447d-b523-395de732cb18 for instance 63bbc7c6-1f26-4d17-bc3e-01f31f1fa939 on nch2-master-1 Jul 16 21:32:35.277416 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-8804582f-ca7e-4f75-931d-08f5ccea97b0 tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:32:35.278076 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-8804582f-ca7e-4f75-931d-08f5ccea97b0 tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] [instance: 3c0f0855-7a9c-4d8a-95ac-3bf41d32d4e6] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:32:35.285443 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-e872232b-fb72-4a6d-bf00-93784bb18186 req-66b77ae8-2f84-489c-8124-b2d673c5e959 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.069875 Jul 16 21:32:35.286827 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2335/4677] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:32:35 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 72 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:32:35.349906 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-8804582f-ca7e-4f75-931d-08f5ccea97b0 tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4fa446573f7d42de816280280c3dec19', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ce30584a7dfa4ab19de73aa8b18b113b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:32:35.358227 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-8804582f-ca7e-4f75-931d-08f5ccea97b0 tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] 10.222.0.16 "GET /compute/v2.1/servers/3c0f0855-7a9c-4d8a-95ac-3bf41d32d4e6" status: 200 len: 1302 microversion: 2.1 time: 0.094854 Jul 16 21:32:35.358826 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2343/4678] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:32:35 2022] GET /compute/v2.1/servers/3c0f0855-7a9c-4d8a-95ac-3bf41d32d4e6 => generated 1302 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:32:36.286832 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-e5e81594-d51a-46f6-8065-7391c3bd832f tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:32:36.288893 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-e5e81594-d51a-46f6-8065-7391c3bd832f tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] [instance: 63bbc7c6-1f26-4d17-bc3e-01f31f1fa939] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:32:36.309937 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e5e81594-d51a-46f6-8065-7391c3bd832f tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:36.311187 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e5e81594-d51a-46f6-8065-7391c3bd832f tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:36.315155 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e5e81594-d51a-46f6-8065-7391c3bd832f tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:36.316225 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e5e81594-d51a-46f6-8065-7391c3bd832f tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:36.383511 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-dbee42b2-50a4-4e3e-8876-38b8db58e61e tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:32:36.384152 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-dbee42b2-50a4-4e3e-8876-38b8db58e61e tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] [instance: 3c0f0855-7a9c-4d8a-95ac-3bf41d32d4e6] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:32:36.563474 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-dbee42b2-50a4-4e3e-8876-38b8db58e61e tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4fa446573f7d42de816280280c3dec19', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ce30584a7dfa4ab19de73aa8b18b113b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:32:36.577083 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-dbee42b2-50a4-4e3e-8876-38b8db58e61e tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] 10.222.0.16 "GET /compute/v2.1/servers/3c0f0855-7a9c-4d8a-95ac-3bf41d32d4e6" status: 200 len: 1302 microversion: 2.1 time: 0.204386 Jul 16 21:32:36.634946 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2344/4679] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:32:36 2022] GET /compute/v2.1/servers/3c0f0855-7a9c-4d8a-95ac-3bf41d32d4e6 => generated 1302 bytes in 264 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:32:36.827659 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-e5e81594-d51a-46f6-8065-7391c3bd832f tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '08abcba4284e434680ee4640f1541cf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '367c69301e6a44419af5c2d28b616b80', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:32:36.844425 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-e5e81594-d51a-46f6-8065-7391c3bd832f tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '08abcba4284e434680ee4640f1541cf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '367c69301e6a44419af5c2d28b616b80', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:32:36.852592 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-e5e81594-d51a-46f6-8065-7391c3bd832f tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '08abcba4284e434680ee4640f1541cf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '367c69301e6a44419af5c2d28b616b80', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:32:36.857350 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-e5e81594-d51a-46f6-8065-7391c3bd832f tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] 10.222.0.16 "GET /compute/v2.1/servers/63bbc7c6-1f26-4d17-bc3e-01f31f1fa939" status: 200 len: 1362 microversion: 2.26 time: 0.579364 Jul 16 21:32:36.862713 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2336/4680] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:32:36 2022] GET /compute/v2.1/servers/63bbc7c6-1f26-4d17-bc3e-01f31f1fa939 => generated 1362 bytes in 582 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:32:37.653190 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-141f29d6-dcce-498c-8e94-456e63f1d727 tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:32:37.656600 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-141f29d6-dcce-498c-8e94-456e63f1d727 tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] [instance: 3c0f0855-7a9c-4d8a-95ac-3bf41d32d4e6] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:32:37.676069 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-141f29d6-dcce-498c-8e94-456e63f1d727 tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:37.676622 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-141f29d6-dcce-498c-8e94-456e63f1d727 tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:37.677614 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-141f29d6-dcce-498c-8e94-456e63f1d727 tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:37.679231 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-141f29d6-dcce-498c-8e94-456e63f1d727 tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:37.796093 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-141f29d6-dcce-498c-8e94-456e63f1d727 tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4fa446573f7d42de816280280c3dec19', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ce30584a7dfa4ab19de73aa8b18b113b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:32:37.812019 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-141f29d6-dcce-498c-8e94-456e63f1d727 tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] 10.222.0.16 "GET /compute/v2.1/servers/3c0f0855-7a9c-4d8a-95ac-3bf41d32d4e6" status: 200 len: 1354 microversion: 2.1 time: 0.162860 Jul 16 21:32:37.820812 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2345/4681] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:32:37 2022] GET /compute/v2.1/servers/3c0f0855-7a9c-4d8a-95ac-3bf41d32d4e6 => generated 1354 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:32:37.881484 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-c72a76a6-4b25-4b35-9ce8-121e8b53fc3d tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:32:37.882741 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-c72a76a6-4b25-4b35-9ce8-121e8b53fc3d tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] [instance: 63bbc7c6-1f26-4d17-bc3e-01f31f1fa939] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:32:37.892804 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c72a76a6-4b25-4b35-9ce8-121e8b53fc3d tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:37.893554 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c72a76a6-4b25-4b35-9ce8-121e8b53fc3d tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:37.895623 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c72a76a6-4b25-4b35-9ce8-121e8b53fc3d tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:37.896376 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c72a76a6-4b25-4b35-9ce8-121e8b53fc3d tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:38.132680 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-c72a76a6-4b25-4b35-9ce8-121e8b53fc3d tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '08abcba4284e434680ee4640f1541cf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '367c69301e6a44419af5c2d28b616b80', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:32:38.149039 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-c72a76a6-4b25-4b35-9ce8-121e8b53fc3d tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '08abcba4284e434680ee4640f1541cf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '367c69301e6a44419af5c2d28b616b80', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:32:38.151084 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-c72a76a6-4b25-4b35-9ce8-121e8b53fc3d tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '08abcba4284e434680ee4640f1541cf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '367c69301e6a44419af5c2d28b616b80', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:32:38.152682 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-c72a76a6-4b25-4b35-9ce8-121e8b53fc3d tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] 10.222.0.16 "GET /compute/v2.1/servers/63bbc7c6-1f26-4d17-bc3e-01f31f1fa939" status: 200 len: 1546 microversion: 2.26 time: 0.279613 Jul 16 21:32:38.153767 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2337/4682] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:32:37 2022] GET /compute/v2.1/servers/63bbc7c6-1f26-4d17-bc3e-01f31f1fa939 => generated 1546 bytes in 281 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:32:38.176940 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-e53d111c-8d34-4f39-8eaf-62687bac6d30 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Calling method '.version_select at 0x7fdd8ff32a60>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:32:38.196177 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e53d111c-8d34-4f39-8eaf-62687bac6d30 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:38.199793 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e53d111c-8d34-4f39-8eaf-62687bac6d30 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:38.199793 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-e53d111c-8d34-4f39-8eaf-62687bac6d30 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] [instance: 63bbc7c6-1f26-4d17-bc3e-01f31f1fa939] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:32:38.207800 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e53d111c-8d34-4f39-8eaf-62687bac6d30 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:38.208774 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e53d111c-8d34-4f39-8eaf-62687bac6d30 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:38.209756 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e53d111c-8d34-4f39-8eaf-62687bac6d30 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:38.210523 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e53d111c-8d34-4f39-8eaf-62687bac6d30 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:38.247912 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e53d111c-8d34-4f39-8eaf-62687bac6d30 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:38.248482 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e53d111c-8d34-4f39-8eaf-62687bac6d30 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:38.267703 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e53d111c-8d34-4f39-8eaf-62687bac6d30 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:38.268975 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e53d111c-8d34-4f39-8eaf-62687bac6d30 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:38.417657 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.objects.instance [None req-e53d111c-8d34-4f39-8eaf-62687bac6d30 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lazy-loading 'flavor' on Instance uuid 63bbc7c6-1f26-4d17-bc3e-01f31f1fa939 {{(pid=138681) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:32:38.437635 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-e53d111c-8d34-4f39-8eaf-62687bac6d30 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] 10.222.0.16 "PUT /compute/v2.1/servers/63bbc7c6-1f26-4d17-bc3e-01f31f1fa939/tags/tempest-tag-1501147621" status: 201 len: 0 microversion: 2.26 time: 0.265403 Jul 16 21:32:38.443033 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2346/4683] 10.222.0.16 () {62 vars in 1379 bytes} [Sat Jul 16 21:32:38 2022] PUT /compute/v2.1/servers/63bbc7c6-1f26-4d17-bc3e-01f31f1fa939/tags/tempest-tag-1501147621 => generated 0 bytes in 274 msecs (HTTP/1.1 201) 10 headers in 486 bytes (2 switches on core 0) Jul 16 21:32:38.449418 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-ac5850f2-93b8-45f2-9de5-579100ef8d00 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Calling method '.version_select at 0x7fdd8fd99700>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:32:38.465649 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ac5850f2-93b8-45f2-9de5-579100ef8d00 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:38.465649 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ac5850f2-93b8-45f2-9de5-579100ef8d00 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:38.499599 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-ac5850f2-93b8-45f2-9de5-579100ef8d00 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] 10.222.0.16 "GET /compute/v2.1/servers/63bbc7c6-1f26-4d17-bc3e-01f31f1fa939/tags/tempest-tag-1501147621" status: 204 len: 0 microversion: 2.26 time: 0.055278 Jul 16 21:32:38.504554 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2338/4684] 10.222.0.16 () {60 vars in 1360 bytes} [Sat Jul 16 21:32:38 2022] GET /compute/v2.1/servers/63bbc7c6-1f26-4d17-bc3e-01f31f1fa939/tags/tempest-tag-1501147621 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Jul 16 21:32:38.511579 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-0c271cec-14a0-4b24-8cf2-0d21a5a25eb2 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Calling method '.version_select at 0x7fdd8fec7820>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:32:38.527723 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0c271cec-14a0-4b24-8cf2-0d21a5a25eb2 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:38.527723 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0c271cec-14a0-4b24-8cf2-0d21a5a25eb2 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:38.529222 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-0c271cec-14a0-4b24-8cf2-0d21a5a25eb2 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] [instance: 63bbc7c6-1f26-4d17-bc3e-01f31f1fa939] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:32:38.545853 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0c271cec-14a0-4b24-8cf2-0d21a5a25eb2 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:38.546621 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0c271cec-14a0-4b24-8cf2-0d21a5a25eb2 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:38.547887 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0c271cec-14a0-4b24-8cf2-0d21a5a25eb2 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:38.548527 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0c271cec-14a0-4b24-8cf2-0d21a5a25eb2 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:38.558951 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=97,nova_cell1:SAVEPOINT=1,nova_cell1:INSERT=1,nova_cell1:RELEASE=1 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:32:38.602473 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0c271cec-14a0-4b24-8cf2-0d21a5a25eb2 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:38.610159 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0c271cec-14a0-4b24-8cf2-0d21a5a25eb2 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:38.742883 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.objects.instance [None req-0c271cec-14a0-4b24-8cf2-0d21a5a25eb2 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lazy-loading 'flavor' on Instance uuid 63bbc7c6-1f26-4d17-bc3e-01f31f1fa939 {{(pid=138681) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:32:38.764442 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-0c271cec-14a0-4b24-8cf2-0d21a5a25eb2 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] 10.222.0.16 "DELETE /compute/v2.1/servers/63bbc7c6-1f26-4d17-bc3e-01f31f1fa939/tags" status: 204 len: 0 microversion: 2.26 time: 0.258250 Jul 16 21:32:38.766967 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2347/4685] 10.222.0.16 () {60 vars in 1294 bytes} [Sat Jul 16 21:32:38 2022] DELETE /compute/v2.1/servers/63bbc7c6-1f26-4d17-bc3e-01f31f1fa939/tags => generated 0 bytes in 260 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Jul 16 21:32:38.780129 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-6dd74c45-0a24-4498-9378-bfaf9fd37833 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Calling method '.version_select at 0x7fdd9d025af0>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:32:38.789394 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6dd74c45-0a24-4498-9378-bfaf9fd37833 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:38.790136 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6dd74c45-0a24-4498-9378-bfaf9fd37833 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:38.805087 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-6dd74c45-0a24-4498-9378-bfaf9fd37833 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] 10.222.0.16 "GET /compute/v2.1/servers/63bbc7c6-1f26-4d17-bc3e-01f31f1fa939/tags" status: 200 len: 12 microversion: 2.26 time: 0.028917 Jul 16 21:32:38.806166 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2339/4686] 10.222.0.16 () {60 vars in 1291 bytes} [Sat Jul 16 21:32:38 2022] GET /compute/v2.1/servers/63bbc7c6-1f26-4d17-bc3e-01f31f1fa939/tags => generated 12 bytes in 31 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:32:38.816930 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-b596f658-cd5f-4453-807d-cb2181b5cf79 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Calling method '.version_select at 0x7fdd8fec7ca0>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:32:38.830495 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b596f658-cd5f-4453-807d-cb2181b5cf79 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:38.831065 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b596f658-cd5f-4453-807d-cb2181b5cf79 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:38.831519 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-b596f658-cd5f-4453-807d-cb2181b5cf79 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] [instance: 63bbc7c6-1f26-4d17-bc3e-01f31f1fa939] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:32:38.834915 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-c14c36a7-3d1b-48ca-a99c-77129de9dc2d tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:32:38.838216 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-c14c36a7-3d1b-48ca-a99c-77129de9dc2d tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] [instance: 3c0f0855-7a9c-4d8a-95ac-3bf41d32d4e6] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:32:38.846452 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c14c36a7-3d1b-48ca-a99c-77129de9dc2d tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:38.846914 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c14c36a7-3d1b-48ca-a99c-77129de9dc2d tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:38.848926 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b596f658-cd5f-4453-807d-cb2181b5cf79 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:38.849397 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b596f658-cd5f-4453-807d-cb2181b5cf79 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:38.850189 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b596f658-cd5f-4453-807d-cb2181b5cf79 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:38.853238 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c14c36a7-3d1b-48ca-a99c-77129de9dc2d tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:38.853716 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c14c36a7-3d1b-48ca-a99c-77129de9dc2d tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:38.855959 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b596f658-cd5f-4453-807d-cb2181b5cf79 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:38.926008 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b596f658-cd5f-4453-807d-cb2181b5cf79 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:38.926609 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b596f658-cd5f-4453-807d-cb2181b5cf79 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:38.936890 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b596f658-cd5f-4453-807d-cb2181b5cf79 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:38.937357 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b596f658-cd5f-4453-807d-cb2181b5cf79 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:39.031423 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-c14c36a7-3d1b-48ca-a99c-77129de9dc2d tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4fa446573f7d42de816280280c3dec19', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ce30584a7dfa4ab19de73aa8b18b113b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:32:39.042804 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-c14c36a7-3d1b-48ca-a99c-77129de9dc2d tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] 10.222.0.16 "GET /compute/v2.1/servers/3c0f0855-7a9c-4d8a-95ac-3bf41d32d4e6" status: 200 len: 1372 microversion: 2.1 time: 0.217620 Jul 16 21:32:39.048632 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2340/4687] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:32:38 2022] GET /compute/v2.1/servers/3c0f0855-7a9c-4d8a-95ac-3bf41d32d4e6 => generated 1372 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:32:39.097050 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.objects.instance [None req-b596f658-cd5f-4453-807d-cb2181b5cf79 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lazy-loading 'flavor' on Instance uuid 63bbc7c6-1f26-4d17-bc3e-01f31f1fa939 {{(pid=138681) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:32:39.124190 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-b596f658-cd5f-4453-807d-cb2181b5cf79 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] 10.222.0.16 "PUT /compute/v2.1/servers/63bbc7c6-1f26-4d17-bc3e-01f31f1fa939/tags/tempest-tag-1820530876" status: 201 len: 0 microversion: 2.26 time: 0.307533 Jul 16 21:32:39.134370 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2348/4688] 10.222.0.16 () {62 vars in 1379 bytes} [Sat Jul 16 21:32:38 2022] PUT /compute/v2.1/servers/63bbc7c6-1f26-4d17-bc3e-01f31f1fa939/tags/tempest-tag-1820530876 => generated 0 bytes in 323 msecs (HTTP/1.1 201) 10 headers in 486 bytes (1 switches on core 0) Jul 16 21:32:39.140858 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-60f54992-01ee-404f-a565-c5507dc3bedc tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Calling method '.version_select at 0x7fdd8fd9a8b0>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:32:39.160838 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-60f54992-01ee-404f-a565-c5507dc3bedc tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:39.162677 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-60f54992-01ee-404f-a565-c5507dc3bedc tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:39.173985 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-60f54992-01ee-404f-a565-c5507dc3bedc tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] 10.222.0.16 "GET /compute/v2.1/servers/63bbc7c6-1f26-4d17-bc3e-01f31f1fa939/tags" status: 200 len: 36 microversion: 2.26 time: 0.039140 Jul 16 21:32:39.175522 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2341/4689] 10.222.0.16 () {60 vars in 1291 bytes} [Sat Jul 16 21:32:39 2022] GET /compute/v2.1/servers/63bbc7c6-1f26-4d17-bc3e-01f31f1fa939/tags => generated 36 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:32:39.188444 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-b3c7e282-2a58-47ee-a22e-ff6bae00ac02 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Calling method '.version_select at 0x7fdd8ff63550>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:32:39.200086 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b3c7e282-2a58-47ee-a22e-ff6bae00ac02 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:39.200643 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b3c7e282-2a58-47ee-a22e-ff6bae00ac02 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:39.201383 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-b3c7e282-2a58-47ee-a22e-ff6bae00ac02 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] [instance: 63bbc7c6-1f26-4d17-bc3e-01f31f1fa939] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:32:39.210124 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b3c7e282-2a58-47ee-a22e-ff6bae00ac02 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:39.210954 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b3c7e282-2a58-47ee-a22e-ff6bae00ac02 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:39.212000 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b3c7e282-2a58-47ee-a22e-ff6bae00ac02 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:39.213375 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b3c7e282-2a58-47ee-a22e-ff6bae00ac02 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:39.280116 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b3c7e282-2a58-47ee-a22e-ff6bae00ac02 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:39.280723 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b3c7e282-2a58-47ee-a22e-ff6bae00ac02 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:39.404541 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.objects.instance [None req-b3c7e282-2a58-47ee-a22e-ff6bae00ac02 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lazy-loading 'flavor' on Instance uuid 63bbc7c6-1f26-4d17-bc3e-01f31f1fa939 {{(pid=138681) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:32:39.421728 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-b3c7e282-2a58-47ee-a22e-ff6bae00ac02 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] 10.222.0.16 "DELETE /compute/v2.1/servers/63bbc7c6-1f26-4d17-bc3e-01f31f1fa939/tags/tempest-tag-1820530876" status: 204 len: 0 microversion: 2.26 time: 0.238608 Jul 16 21:32:39.422569 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2349/4690] 10.222.0.16 () {60 vars in 1363 bytes} [Sat Jul 16 21:32:39 2022] DELETE /compute/v2.1/servers/63bbc7c6-1f26-4d17-bc3e-01f31f1fa939/tags/tempest-tag-1820530876 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Jul 16 21:32:39.435931 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-50a3328f-33f3-4ebd-9dca-fb575e628b01 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Calling method '.version_select at 0x7fdd9d025af0>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:32:39.439610 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-50a3328f-33f3-4ebd-9dca-fb575e628b01 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:39.440365 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-50a3328f-33f3-4ebd-9dca-fb575e628b01 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:39.451946 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-50a3328f-33f3-4ebd-9dca-fb575e628b01 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] 10.222.0.16 "GET /compute/v2.1/servers/63bbc7c6-1f26-4d17-bc3e-01f31f1fa939/tags" status: 200 len: 12 microversion: 2.26 time: 0.024771 Jul 16 21:32:39.452605 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2342/4691] 10.222.0.16 () {60 vars in 1291 bytes} [Sat Jul 16 21:32:39 2022] GET /compute/v2.1/servers/63bbc7c6-1f26-4d17-bc3e-01f31f1fa939/tags => generated 12 bytes in 26 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:32:39.464656 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-e2d8441f-366c-42a3-b85e-745bdcffbb94 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Calling method '.version_select at 0x7fdd8ff32ca0>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:32:39.474649 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e2d8441f-366c-42a3-b85e-745bdcffbb94 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:39.475151 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e2d8441f-366c-42a3-b85e-745bdcffbb94 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:39.475845 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-e2d8441f-366c-42a3-b85e-745bdcffbb94 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] [instance: 63bbc7c6-1f26-4d17-bc3e-01f31f1fa939] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:32:39.482863 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e2d8441f-366c-42a3-b85e-745bdcffbb94 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:39.483356 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e2d8441f-366c-42a3-b85e-745bdcffbb94 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:39.484358 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e2d8441f-366c-42a3-b85e-745bdcffbb94 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:39.484958 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e2d8441f-366c-42a3-b85e-745bdcffbb94 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:39.516027 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e2d8441f-366c-42a3-b85e-745bdcffbb94 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:39.516510 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e2d8441f-366c-42a3-b85e-745bdcffbb94 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:39.535172 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.objects.instance [None req-e2d8441f-366c-42a3-b85e-745bdcffbb94 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lazy-loading 'flavor' on Instance uuid 63bbc7c6-1f26-4d17-bc3e-01f31f1fa939 {{(pid=138681) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:32:39.555796 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-e2d8441f-366c-42a3-b85e-745bdcffbb94 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] 10.222.0.16 "DELETE /compute/v2.1/servers/63bbc7c6-1f26-4d17-bc3e-01f31f1fa939/tags" status: 204 len: 0 microversion: 2.26 time: 0.096515 Jul 16 21:32:39.556746 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2350/4692] 10.222.0.16 () {60 vars in 1294 bytes} [Sat Jul 16 21:32:39 2022] DELETE /compute/v2.1/servers/63bbc7c6-1f26-4d17-bc3e-01f31f1fa939/tags => generated 0 bytes in 98 msecs (HTTP/1.1 204) 9 headers in 365 bytes (2 switches on core 0) Jul 16 21:32:39.569183 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-9cbbcdbf-1445-4a23-81f8-e0a0e2d767a5 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Calling method '.version_select at 0x7fdda2960ee0>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:32:39.579417 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9cbbcdbf-1445-4a23-81f8-e0a0e2d767a5 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:39.580692 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9cbbcdbf-1445-4a23-81f8-e0a0e2d767a5 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:39.582091 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-9cbbcdbf-1445-4a23-81f8-e0a0e2d767a5 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] [instance: 63bbc7c6-1f26-4d17-bc3e-01f31f1fa939] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:32:39.592230 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9cbbcdbf-1445-4a23-81f8-e0a0e2d767a5 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:39.593271 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9cbbcdbf-1445-4a23-81f8-e0a0e2d767a5 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:39.594338 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9cbbcdbf-1445-4a23-81f8-e0a0e2d767a5 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:39.595077 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9cbbcdbf-1445-4a23-81f8-e0a0e2d767a5 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:39.630094 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9cbbcdbf-1445-4a23-81f8-e0a0e2d767a5 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:39.631331 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9cbbcdbf-1445-4a23-81f8-e0a0e2d767a5 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:39.645203 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9cbbcdbf-1445-4a23-81f8-e0a0e2d767a5 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:39.649892 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9cbbcdbf-1445-4a23-81f8-e0a0e2d767a5 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:39.813215 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.objects.instance [None req-9cbbcdbf-1445-4a23-81f8-e0a0e2d767a5 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lazy-loading 'flavor' on Instance uuid 63bbc7c6-1f26-4d17-bc3e-01f31f1fa939 {{(pid=138680) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:32:39.852437 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-9cbbcdbf-1445-4a23-81f8-e0a0e2d767a5 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] 10.222.0.16 "PUT /compute/v2.1/servers/63bbc7c6-1f26-4d17-bc3e-01f31f1fa939/tags/tempest-tag-186653821" status: 201 len: 0 microversion: 2.26 time: 0.290356 Jul 16 21:32:39.853122 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2343/4693] 10.222.0.16 () {62 vars in 1376 bytes} [Sat Jul 16 21:32:39 2022] PUT /compute/v2.1/servers/63bbc7c6-1f26-4d17-bc3e-01f31f1fa939/tags/tempest-tag-186653821 => generated 0 bytes in 292 msecs (HTTP/1.1 201) 10 headers in 485 bytes (1 switches on core 0) Jul 16 21:32:39.865421 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-e7b818e7-27bb-4030-b91b-6d2349b5888c tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Calling method '.version_select at 0x7fdd8fec7ee0>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:32:39.904576 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e7b818e7-27bb-4030-b91b-6d2349b5888c tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:39.904576 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e7b818e7-27bb-4030-b91b-6d2349b5888c tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:39.904576 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-e7b818e7-27bb-4030-b91b-6d2349b5888c tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] [instance: 63bbc7c6-1f26-4d17-bc3e-01f31f1fa939] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:32:39.914469 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e7b818e7-27bb-4030-b91b-6d2349b5888c tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:39.915196 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e7b818e7-27bb-4030-b91b-6d2349b5888c tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:39.916713 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e7b818e7-27bb-4030-b91b-6d2349b5888c tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:39.917329 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e7b818e7-27bb-4030-b91b-6d2349b5888c tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:39.949904 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e7b818e7-27bb-4030-b91b-6d2349b5888c tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:39.950517 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e7b818e7-27bb-4030-b91b-6d2349b5888c tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:39.961342 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e7b818e7-27bb-4030-b91b-6d2349b5888c tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:39.961871 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e7b818e7-27bb-4030-b91b-6d2349b5888c tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:40.060453 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.objects.instance [None req-e7b818e7-27bb-4030-b91b-6d2349b5888c tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lazy-loading 'flavor' on Instance uuid 63bbc7c6-1f26-4d17-bc3e-01f31f1fa939 {{(pid=138681) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:32:40.062301 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-117907ed-2283-48fb-a66e-c80e270242df tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:32:40.062925 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-117907ed-2283-48fb-a66e-c80e270242df tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] [instance: 3c0f0855-7a9c-4d8a-95ac-3bf41d32d4e6] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:32:40.077470 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-117907ed-2283-48fb-a66e-c80e270242df tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:40.080258 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-117907ed-2283-48fb-a66e-c80e270242df tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:40.080258 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-117907ed-2283-48fb-a66e-c80e270242df tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:40.080511 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-e7b818e7-27bb-4030-b91b-6d2349b5888c tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] 10.222.0.16 "PUT /compute/v2.1/servers/63bbc7c6-1f26-4d17-bc3e-01f31f1fa939/tags/tempest-tag-1962676109" status: 201 len: 0 microversion: 2.26 time: 0.218242 Jul 16 21:32:40.081091 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-117907ed-2283-48fb-a66e-c80e270242df tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:40.081320 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2351/4694] 10.222.0.16 () {62 vars in 1379 bytes} [Sat Jul 16 21:32:39 2022] PUT /compute/v2.1/servers/63bbc7c6-1f26-4d17-bc3e-01f31f1fa939/tags/tempest-tag-1962676109 => generated 0 bytes in 220 msecs (HTTP/1.1 201) 10 headers in 486 bytes (1 switches on core 0) Jul 16 21:32:40.095246 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-be65d971-6143-4b40-86aa-694008f65f16 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Calling method '.version_select at 0x7fdd8ff69310>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:32:40.113039 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-be65d971-6143-4b40-86aa-694008f65f16 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:40.113039 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-be65d971-6143-4b40-86aa-694008f65f16 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:40.113039 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-be65d971-6143-4b40-86aa-694008f65f16 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] [instance: 63bbc7c6-1f26-4d17-bc3e-01f31f1fa939] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:32:40.113612 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=97,nova_cell1:SAVEPOINT=1,nova_cell1:INSERT=1,nova_cell1:RELEASE=1 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:32:40.115219 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-be65d971-6143-4b40-86aa-694008f65f16 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:40.115883 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-be65d971-6143-4b40-86aa-694008f65f16 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:40.116922 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-be65d971-6143-4b40-86aa-694008f65f16 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:40.117580 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-be65d971-6143-4b40-86aa-694008f65f16 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:40.146179 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-be65d971-6143-4b40-86aa-694008f65f16 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:40.147759 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-be65d971-6143-4b40-86aa-694008f65f16 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:40.245281 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.objects.instance [None req-be65d971-6143-4b40-86aa-694008f65f16 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lazy-loading 'flavor' on Instance uuid 63bbc7c6-1f26-4d17-bc3e-01f31f1fa939 {{(pid=138681) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:32:40.265003 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-be65d971-6143-4b40-86aa-694008f65f16 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] 10.222.0.16 "DELETE /compute/v2.1/servers/63bbc7c6-1f26-4d17-bc3e-01f31f1fa939/tags" status: 204 len: 0 microversion: 2.26 time: 0.174327 Jul 16 21:32:40.269955 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2352/4695] 10.222.0.16 () {60 vars in 1294 bytes} [Sat Jul 16 21:32:40 2022] DELETE /compute/v2.1/servers/63bbc7c6-1f26-4d17-bc3e-01f31f1fa939/tags => generated 0 bytes in 176 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Jul 16 21:32:40.274771 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-93eb7229-5baa-4486-bb6c-2574d56b962d tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Calling method '.version_select at 0x7fdd8ff091f0>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:32:40.283354 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-93eb7229-5baa-4486-bb6c-2574d56b962d tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:40.283847 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-93eb7229-5baa-4486-bb6c-2574d56b962d tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:40.295904 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-93eb7229-5baa-4486-bb6c-2574d56b962d tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] 10.222.0.16 "GET /compute/v2.1/servers/63bbc7c6-1f26-4d17-bc3e-01f31f1fa939/tags" status: 200 len: 12 microversion: 2.26 time: 0.025439 Jul 16 21:32:40.298391 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2353/4696] 10.222.0.16 () {60 vars in 1291 bytes} [Sat Jul 16 21:32:40 2022] GET /compute/v2.1/servers/63bbc7c6-1f26-4d17-bc3e-01f31f1fa939/tags => generated 12 bytes in 29 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:32:40.308095 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-6b8879a6-e36b-4ed0-9250-d7182b30d1d8 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Calling method '.version_select at 0x7fdd8ff69b80>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:32:40.318630 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6b8879a6-e36b-4ed0-9250-d7182b30d1d8 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:40.319270 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6b8879a6-e36b-4ed0-9250-d7182b30d1d8 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:40.319769 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-6b8879a6-e36b-4ed0-9250-d7182b30d1d8 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] [instance: 63bbc7c6-1f26-4d17-bc3e-01f31f1fa939] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:32:40.327255 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6b8879a6-e36b-4ed0-9250-d7182b30d1d8 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:40.327255 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6b8879a6-e36b-4ed0-9250-d7182b30d1d8 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:40.327965 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6b8879a6-e36b-4ed0-9250-d7182b30d1d8 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:40.328461 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6b8879a6-e36b-4ed0-9250-d7182b30d1d8 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:40.355623 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6b8879a6-e36b-4ed0-9250-d7182b30d1d8 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:40.356107 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6b8879a6-e36b-4ed0-9250-d7182b30d1d8 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:40.377189 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.objects.instance [None req-6b8879a6-e36b-4ed0-9250-d7182b30d1d8 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lazy-loading 'flavor' on Instance uuid 63bbc7c6-1f26-4d17-bc3e-01f31f1fa939 {{(pid=138681) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:32:40.405369 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-6b8879a6-e36b-4ed0-9250-d7182b30d1d8 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] 10.222.0.16 "DELETE /compute/v2.1/servers/63bbc7c6-1f26-4d17-bc3e-01f31f1fa939/tags" status: 204 len: 0 microversion: 2.26 time: 0.102169 Jul 16 21:32:40.421309 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2354/4697] 10.222.0.16 () {60 vars in 1294 bytes} [Sat Jul 16 21:32:40 2022] DELETE /compute/v2.1/servers/63bbc7c6-1f26-4d17-bc3e-01f31f1fa939/tags => generated 0 bytes in 106 msecs (HTTP/1.1 204) 9 headers in 365 bytes (2 switches on core 0) Jul 16 21:32:40.434454 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-4a2a6bd0-bac1-4a20-ba6d-56aa277785a2 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Calling method '.version_select at 0x7fdd8ff32ca0>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:32:40.449398 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4a2a6bd0-bac1-4a20-ba6d-56aa277785a2 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:40.450290 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4a2a6bd0-bac1-4a20-ba6d-56aa277785a2 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:40.451070 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-4a2a6bd0-bac1-4a20-ba6d-56aa277785a2 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] [instance: 63bbc7c6-1f26-4d17-bc3e-01f31f1fa939] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:32:40.460697 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4a2a6bd0-bac1-4a20-ba6d-56aa277785a2 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:40.462539 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4a2a6bd0-bac1-4a20-ba6d-56aa277785a2 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:40.463732 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4a2a6bd0-bac1-4a20-ba6d-56aa277785a2 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:40.464669 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4a2a6bd0-bac1-4a20-ba6d-56aa277785a2 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:40.506019 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4a2a6bd0-bac1-4a20-ba6d-56aa277785a2 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:40.506822 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4a2a6bd0-bac1-4a20-ba6d-56aa277785a2 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:40.519312 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4a2a6bd0-bac1-4a20-ba6d-56aa277785a2 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:40.519935 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4a2a6bd0-bac1-4a20-ba6d-56aa277785a2 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:40.647233 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.objects.instance [None req-4a2a6bd0-bac1-4a20-ba6d-56aa277785a2 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lazy-loading 'flavor' on Instance uuid 63bbc7c6-1f26-4d17-bc3e-01f31f1fa939 {{(pid=138681) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:32:40.668869 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-4a2a6bd0-bac1-4a20-ba6d-56aa277785a2 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] 10.222.0.16 "PUT /compute/v2.1/servers/63bbc7c6-1f26-4d17-bc3e-01f31f1fa939/tags/tempest-tag-291176182" status: 201 len: 0 microversion: 2.26 time: 0.246268 Jul 16 21:32:40.673928 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2355/4698] 10.222.0.16 () {62 vars in 1376 bytes} [Sat Jul 16 21:32:40 2022] PUT /compute/v2.1/servers/63bbc7c6-1f26-4d17-bc3e-01f31f1fa939/tags/tempest-tag-291176182 => generated 0 bytes in 248 msecs (HTTP/1.1 201) 10 headers in 485 bytes (1 switches on core 0) Jul 16 21:32:40.679843 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-87b99c6b-390d-4860-99a9-b0d44a04108b tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Calling method '.version_select at 0x7fdd8ff4ec10>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:32:40.691737 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-87b99c6b-390d-4860-99a9-b0d44a04108b tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:40.692205 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-87b99c6b-390d-4860-99a9-b0d44a04108b tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:40.692841 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-87b99c6b-390d-4860-99a9-b0d44a04108b tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] [instance: 63bbc7c6-1f26-4d17-bc3e-01f31f1fa939] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:32:40.698769 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-87b99c6b-390d-4860-99a9-b0d44a04108b tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:40.699298 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-87b99c6b-390d-4860-99a9-b0d44a04108b tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:40.700122 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-87b99c6b-390d-4860-99a9-b0d44a04108b tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:40.700715 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-87b99c6b-390d-4860-99a9-b0d44a04108b tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:40.736855 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-87b99c6b-390d-4860-99a9-b0d44a04108b tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:40.737302 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-87b99c6b-390d-4860-99a9-b0d44a04108b tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:40.739396 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-117907ed-2283-48fb-a66e-c80e270242df tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4fa446573f7d42de816280280c3dec19', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ce30584a7dfa4ab19de73aa8b18b113b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:32:40.747963 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-87b99c6b-390d-4860-99a9-b0d44a04108b tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:40.748103 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-87b99c6b-390d-4860-99a9-b0d44a04108b tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:40.764947 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-117907ed-2283-48fb-a66e-c80e270242df tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] 10.222.0.16 "GET /compute/v2.1/servers/3c0f0855-7a9c-4d8a-95ac-3bf41d32d4e6" status: 200 len: 1427 microversion: 2.1 time: 0.707034 Jul 16 21:32:40.775132 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2344/4699] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:32:40 2022] GET /compute/v2.1/servers/3c0f0855-7a9c-4d8a-95ac-3bf41d32d4e6 => generated 1427 bytes in 708 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:32:40.866139 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.objects.instance [None req-87b99c6b-390d-4860-99a9-b0d44a04108b tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lazy-loading 'flavor' on Instance uuid 63bbc7c6-1f26-4d17-bc3e-01f31f1fa939 {{(pid=138681) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:32:40.901168 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-87b99c6b-390d-4860-99a9-b0d44a04108b tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] 10.222.0.16 "PUT /compute/v2.1/servers/63bbc7c6-1f26-4d17-bc3e-01f31f1fa939/tags/tempest-tag-1807778091" status: 201 len: 0 microversion: 2.26 time: 0.224781 Jul 16 21:32:40.903072 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2356/4700] 10.222.0.16 () {62 vars in 1379 bytes} [Sat Jul 16 21:32:40 2022] PUT /compute/v2.1/servers/63bbc7c6-1f26-4d17-bc3e-01f31f1fa939/tags/tempest-tag-1807778091 => generated 0 bytes in 228 msecs (HTTP/1.1 201) 10 headers in 486 bytes (1 switches on core 0) Jul 16 21:32:40.912407 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-44d4c401-28f0-4d82-b216-56541e16d6f1 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Action: 'update_all', calling method: .version_select at 0x7fdd9c08e0d0>, body: {"tags": ["tempest-tag-80885220", "tempest-tag-287707306"]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:32:40.925113 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-44d4c401-28f0-4d82-b216-56541e16d6f1 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:40.926793 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-44d4c401-28f0-4d82-b216-56541e16d6f1 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:40.926793 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-44d4c401-28f0-4d82-b216-56541e16d6f1 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] [instance: 63bbc7c6-1f26-4d17-bc3e-01f31f1fa939] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:32:40.932452 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-44d4c401-28f0-4d82-b216-56541e16d6f1 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:40.932908 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-44d4c401-28f0-4d82-b216-56541e16d6f1 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:40.933695 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-44d4c401-28f0-4d82-b216-56541e16d6f1 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:40.934143 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-44d4c401-28f0-4d82-b216-56541e16d6f1 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:40.965526 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-44d4c401-28f0-4d82-b216-56541e16d6f1 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:40.965883 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-44d4c401-28f0-4d82-b216-56541e16d6f1 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:41.102238 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.objects.instance [None req-44d4c401-28f0-4d82-b216-56541e16d6f1 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lazy-loading 'flavor' on Instance uuid 63bbc7c6-1f26-4d17-bc3e-01f31f1fa939 {{(pid=138680) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:32:41.143727 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-44d4c401-28f0-4d82-b216-56541e16d6f1 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] 10.222.0.16 "PUT /compute/v2.1/servers/63bbc7c6-1f26-4d17-bc3e-01f31f1fa939/tags" status: 200 len: 59 microversion: 2.26 time: 0.235175 Jul 16 21:32:41.149365 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2345/4701] 10.222.0.16 () {62 vars in 1311 bytes} [Sat Jul 16 21:32:40 2022] PUT /compute/v2.1/servers/63bbc7c6-1f26-4d17-bc3e-01f31f1fa939/tags => generated 59 bytes in 242 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:32:41.154345 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-b429735a-16e0-4d4e-b44f-8089736d2f6d tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Calling method '.version_select at 0x7fdd8ff23940>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:32:41.160672 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_api:SELECT=99,nova_api:INSERT=1 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:32:41.201795 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b429735a-16e0-4d4e-b44f-8089736d2f6d tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:41.202050 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b429735a-16e0-4d4e-b44f-8089736d2f6d tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:41.223048 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-b429735a-16e0-4d4e-b44f-8089736d2f6d tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] 10.222.0.16 "GET /compute/v2.1/servers/63bbc7c6-1f26-4d17-bc3e-01f31f1fa939/tags" status: 200 len: 59 microversion: 2.26 time: 0.072810 Jul 16 21:32:41.235581 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2357/4702] 10.222.0.16 () {60 vars in 1291 bytes} [Sat Jul 16 21:32:41 2022] GET /compute/v2.1/servers/63bbc7c6-1f26-4d17-bc3e-01f31f1fa939/tags => generated 59 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:32:41.248463 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-b41c573b-fb2c-4fd3-976f-bce719be91e9 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Calling method '.version_select at 0x7fdd9d3ad670>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:32:41.268635 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b41c573b-fb2c-4fd3-976f-bce719be91e9 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:41.268635 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b41c573b-fb2c-4fd3-976f-bce719be91e9 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:41.271067 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-b41c573b-fb2c-4fd3-976f-bce719be91e9 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] [instance: 63bbc7c6-1f26-4d17-bc3e-01f31f1fa939] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:32:41.279068 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b41c573b-fb2c-4fd3-976f-bce719be91e9 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:41.279548 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b41c573b-fb2c-4fd3-976f-bce719be91e9 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:41.280469 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b41c573b-fb2c-4fd3-976f-bce719be91e9 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:41.280874 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b41c573b-fb2c-4fd3-976f-bce719be91e9 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:41.336940 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b41c573b-fb2c-4fd3-976f-bce719be91e9 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:41.340592 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b41c573b-fb2c-4fd3-976f-bce719be91e9 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:41.481567 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.objects.instance [None req-b41c573b-fb2c-4fd3-976f-bce719be91e9 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lazy-loading 'flavor' on Instance uuid 63bbc7c6-1f26-4d17-bc3e-01f31f1fa939 {{(pid=138680) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:32:41.503650 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-b41c573b-fb2c-4fd3-976f-bce719be91e9 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] 10.222.0.16 "DELETE /compute/v2.1/servers/63bbc7c6-1f26-4d17-bc3e-01f31f1fa939/tags" status: 204 len: 0 microversion: 2.26 time: 0.262374 Jul 16 21:32:41.504986 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2346/4703] 10.222.0.16 () {60 vars in 1294 bytes} [Sat Jul 16 21:32:41 2022] DELETE /compute/v2.1/servers/63bbc7c6-1f26-4d17-bc3e-01f31f1fa939/tags => generated 0 bytes in 266 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Jul 16 21:32:41.514785 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-7ce18111-7345-4c29-b0d0-93310f83c2a8 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:32:41.515397 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-7ce18111-7345-4c29-b0d0-93310f83c2a8 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] [instance: 63bbc7c6-1f26-4d17-bc3e-01f31f1fa939] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:32:41.527040 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7ce18111-7345-4c29-b0d0-93310f83c2a8 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:41.527480 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7ce18111-7345-4c29-b0d0-93310f83c2a8 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:41.528400 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7ce18111-7345-4c29-b0d0-93310f83c2a8 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:41.528746 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7ce18111-7345-4c29-b0d0-93310f83c2a8 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:41.555725 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-710da9e2-ad1a-4f0f-ac42-60ab0c2e09ba req-c72d26fb-d89e-43b4-9826-9639261a3a9f service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "3c0f0855-7a9c-4d8a-95ac-3bf41d32d4e6", "tag": "a9915b3b-bd00-4ade-884c-60255466029d"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:32:41.565401 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-710da9e2-ad1a-4f0f-ac42-60ab0c2e09ba req-c72d26fb-d89e-43b4-9826-9639261a3a9f service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:41.566018 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-710da9e2-ad1a-4f0f-ac42-60ab0c2e09ba req-c72d26fb-d89e-43b4-9826-9639261a3a9f service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:41.579882 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-7ce18111-7345-4c29-b0d0-93310f83c2a8 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] [instance: 63bbc7c6-1f26-4d17-bc3e-01f31f1fa939] Going to try to terminate instance {{(pid=138681) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 16 21:32:41.590425 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-710da9e2-ad1a-4f0f-ac42-60ab0c2e09ba req-c72d26fb-d89e-43b4-9826-9639261a3a9f service nova] Creating event network-changed:a9915b3b-bd00-4ade-884c-60255466029d for instance 3c0f0855-7a9c-4d8a-95ac-3bf41d32d4e6 on nch1-master-1 Jul 16 21:32:41.604891 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-710da9e2-ad1a-4f0f-ac42-60ab0c2e09ba req-c72d26fb-d89e-43b4-9826-9639261a3a9f service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.052918 Jul 16 21:32:41.605654 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2347/4704] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:32:41 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 16 21:32:41.786582 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-6c3e8121-e423-41c6-b5c3-894c057c80fc tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:32:41.787656 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-6c3e8121-e423-41c6-b5c3-894c057c80fc tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] [instance: 3c0f0855-7a9c-4d8a-95ac-3bf41d32d4e6] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:32:41.810779 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6c3e8121-e423-41c6-b5c3-894c057c80fc tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:41.811474 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6c3e8121-e423-41c6-b5c3-894c057c80fc tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.010s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:41.812372 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6c3e8121-e423-41c6-b5c3-894c057c80fc tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:41.812779 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6c3e8121-e423-41c6-b5c3-894c057c80fc tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:42.035586 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-7ce18111-7345-4c29-b0d0-93310f83c2a8 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] 10.222.0.16 "DELETE /compute/v2.1/servers/63bbc7c6-1f26-4d17-bc3e-01f31f1fa939" status: 204 len: 0 microversion: 2.26 time: 0.525149 Jul 16 21:32:42.038851 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2358/4705] 10.222.0.16 () {60 vars in 1279 bytes} [Sat Jul 16 21:32:41 2022] DELETE /compute/v2.1/servers/63bbc7c6-1f26-4d17-bc3e-01f31f1fa939 => generated 0 bytes in 529 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Jul 16 21:32:42.055867 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-50e0607c-3a6b-4497-8b02-4faa8f5c314d tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:32:42.056895 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-50e0607c-3a6b-4497-8b02-4faa8f5c314d tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] [instance: 63bbc7c6-1f26-4d17-bc3e-01f31f1fa939] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:32:42.057634 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-6c3e8121-e423-41c6-b5c3-894c057c80fc tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4fa446573f7d42de816280280c3dec19', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ce30584a7dfa4ab19de73aa8b18b113b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:32:42.065154 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-50e0607c-3a6b-4497-8b02-4faa8f5c314d tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:42.066306 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-50e0607c-3a6b-4497-8b02-4faa8f5c314d tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:42.070610 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-50e0607c-3a6b-4497-8b02-4faa8f5c314d tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:42.070610 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-50e0607c-3a6b-4497-8b02-4faa8f5c314d tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:42.070832 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-6c3e8121-e423-41c6-b5c3-894c057c80fc tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] 10.222.0.16 "GET /compute/v2.1/servers/3c0f0855-7a9c-4d8a-95ac-3bf41d32d4e6" status: 200 len: 1427 microversion: 2.1 time: 0.289083 Jul 16 21:32:42.070832 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2348/4706] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:32:41 2022] GET /compute/v2.1/servers/3c0f0855-7a9c-4d8a-95ac-3bf41d32d4e6 => generated 1427 bytes in 291 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:32:42.245289 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=86,nova_cell1:DELETE=5,nova_cell1:SAVEPOINT=3,nova_cell1:INSERT=3,nova_cell1:RELEASE=3 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:32:42.279627 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-50e0607c-3a6b-4497-8b02-4faa8f5c314d tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '08abcba4284e434680ee4640f1541cf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '367c69301e6a44419af5c2d28b616b80', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:32:42.293922 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-50e0607c-3a6b-4497-8b02-4faa8f5c314d tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '08abcba4284e434680ee4640f1541cf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '367c69301e6a44419af5c2d28b616b80', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:32:42.295602 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-50e0607c-3a6b-4497-8b02-4faa8f5c314d tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '08abcba4284e434680ee4640f1541cf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '367c69301e6a44419af5c2d28b616b80', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:32:42.297595 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-50e0607c-3a6b-4497-8b02-4faa8f5c314d tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] 10.222.0.16 "GET /compute/v2.1/servers/63bbc7c6-1f26-4d17-bc3e-01f31f1fa939" status: 200 len: 1552 microversion: 2.26 time: 0.253669 Jul 16 21:32:42.313918 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2359/4707] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:32:42 2022] GET /compute/v2.1/servers/63bbc7c6-1f26-4d17-bc3e-01f31f1fa939 => generated 1552 bytes in 267 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 16 21:32:42.876252 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-4a7cfeea-cbf0-4d62-9931-3faf4a991f10 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:32:42.878260 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-4a7cfeea-cbf0-4d62-9931-3faf4a991f10 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] [instance: 8b77de54-4c8b-4ed5-84f5-68bf69f9809f] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:32:42.888709 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4a7cfeea-cbf0-4d62-9931-3faf4a991f10 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:42.889174 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4a7cfeea-cbf0-4d62-9931-3faf4a991f10 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:42.889975 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4a7cfeea-cbf0-4d62-9931-3faf4a991f10 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:42.890418 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4a7cfeea-cbf0-4d62-9931-3faf4a991f10 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:42.925301 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-4a7cfeea-cbf0-4d62-9931-3faf4a991f10 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] 10.222.0.16 "GET /compute/v2.1/servers/8b77de54-4c8b-4ed5-84f5-68bf69f9809f/ips" status: 200 len: 152 microversion: 2.20 time: 0.052863 Jul 16 21:32:42.927512 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2349/4708] 10.222.0.16 () {60 vars in 1288 bytes} [Sat Jul 16 21:32:42 2022] GET /compute/v2.1/servers/8b77de54-4c8b-4ed5-84f5-68bf69f9809f/ips => generated 152 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 16 21:32:43.086016 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-853a0c68-b616-4395-be12-36ecc314499e tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:32:43.086740 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-853a0c68-b616-4395-be12-36ecc314499e tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] [instance: 3c0f0855-7a9c-4d8a-95ac-3bf41d32d4e6] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:32:43.094725 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-853a0c68-b616-4395-be12-36ecc314499e tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:43.095453 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-853a0c68-b616-4395-be12-36ecc314499e tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:43.097146 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-853a0c68-b616-4395-be12-36ecc314499e tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:43.098123 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-853a0c68-b616-4395-be12-36ecc314499e tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:43.246203 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-853a0c68-b616-4395-be12-36ecc314499e tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4fa446573f7d42de816280280c3dec19', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ce30584a7dfa4ab19de73aa8b18b113b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:32:43.255910 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-853a0c68-b616-4395-be12-36ecc314499e tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] 10.222.0.16 "GET /compute/v2.1/servers/3c0f0855-7a9c-4d8a-95ac-3bf41d32d4e6" status: 200 len: 1427 microversion: 2.1 time: 0.173993 Jul 16 21:32:43.257518 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2360/4709] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:32:43 2022] GET /compute/v2.1/servers/3c0f0855-7a9c-4d8a-95ac-3bf41d32d4e6 => generated 1427 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:32:43.318264 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-1ef13a58-841b-40a1-b7ee-312b183de6d4 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:32:43.322588 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-1ef13a58-841b-40a1-b7ee-312b183de6d4 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] [instance: 63bbc7c6-1f26-4d17-bc3e-01f31f1fa939] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:32:43.327037 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1ef13a58-841b-40a1-b7ee-312b183de6d4 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:43.328090 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1ef13a58-841b-40a1-b7ee-312b183de6d4 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:43.329084 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1ef13a58-841b-40a1-b7ee-312b183de6d4 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:43.329681 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1ef13a58-841b-40a1-b7ee-312b183de6d4 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:43.596936 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-1ef13a58-841b-40a1-b7ee-312b183de6d4 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '08abcba4284e434680ee4640f1541cf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '367c69301e6a44419af5c2d28b616b80', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:32:43.606284 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-1ef13a58-841b-40a1-b7ee-312b183de6d4 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '08abcba4284e434680ee4640f1541cf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '367c69301e6a44419af5c2d28b616b80', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:32:43.615598 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-1ef13a58-841b-40a1-b7ee-312b183de6d4 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '08abcba4284e434680ee4640f1541cf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '367c69301e6a44419af5c2d28b616b80', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:32:43.617320 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-1ef13a58-841b-40a1-b7ee-312b183de6d4 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] 10.222.0.16 "GET /compute/v2.1/servers/63bbc7c6-1f26-4d17-bc3e-01f31f1fa939" status: 200 len: 1552 microversion: 2.26 time: 0.303198 Jul 16 21:32:43.617976 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2350/4710] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:32:43 2022] GET /compute/v2.1/servers/63bbc7c6-1f26-4d17-bc3e-01f31f1fa939 => generated 1552 bytes in 305 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:32:44.280233 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-759789dc-c2c6-4b6c-9db2-7518b10f7304 tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:32:44.282051 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-759789dc-c2c6-4b6c-9db2-7518b10f7304 tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] [instance: 3c0f0855-7a9c-4d8a-95ac-3bf41d32d4e6] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:32:44.294074 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-759789dc-c2c6-4b6c-9db2-7518b10f7304 tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:44.294524 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-759789dc-c2c6-4b6c-9db2-7518b10f7304 tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:44.295432 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-759789dc-c2c6-4b6c-9db2-7518b10f7304 tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:44.295837 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-759789dc-c2c6-4b6c-9db2-7518b10f7304 tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:44.335346 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-6543d7ee-c841-4768-b05c-3208342895fb req-c2ce6ee4-ab02-4fc3-9129-8eeb6e1c4349 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "63bbc7c6-1f26-4d17-bc3e-01f31f1fa939", "name": "network-vif-deleted", "tag": "839ad50f-8742-447d-b523-395de732cb18"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:32:44.344942 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-6543d7ee-c841-4768-b05c-3208342895fb req-c2ce6ee4-ab02-4fc3-9129-8eeb6e1c4349 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:44.345401 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-6543d7ee-c841-4768-b05c-3208342895fb req-c2ce6ee4-ab02-4fc3-9129-8eeb6e1c4349 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:44.372264 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-6543d7ee-c841-4768-b05c-3208342895fb req-c2ce6ee4-ab02-4fc3-9129-8eeb6e1c4349 service nova] Creating event network-vif-deleted:839ad50f-8742-447d-b523-395de732cb18 for instance 63bbc7c6-1f26-4d17-bc3e-01f31f1fa939 on nch2-master-1 Jul 16 21:32:44.380371 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-6543d7ee-c841-4768-b05c-3208342895fb req-c2ce6ee4-ab02-4fc3-9129-8eeb6e1c4349 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.048122 Jul 16 21:32:44.389588 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2351/4711] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:32:44 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:32:44.519399 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-759789dc-c2c6-4b6c-9db2-7518b10f7304 tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4fa446573f7d42de816280280c3dec19', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ce30584a7dfa4ab19de73aa8b18b113b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:32:44.532589 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-759789dc-c2c6-4b6c-9db2-7518b10f7304 tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] 10.222.0.16 "GET /compute/v2.1/servers/3c0f0855-7a9c-4d8a-95ac-3bf41d32d4e6" status: 200 len: 1427 microversion: 2.1 time: 0.255838 Jul 16 21:32:44.533156 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2361/4712] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:32:44 2022] GET /compute/v2.1/servers/3c0f0855-7a9c-4d8a-95ac-3bf41d32d4e6 => generated 1427 bytes in 262 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:32:44.637651 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-7146bdaf-d962-4308-ba25-b4a72fc7b3c1 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:32:44.638629 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-7146bdaf-d962-4308-ba25-b4a72fc7b3c1 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] [instance: 63bbc7c6-1f26-4d17-bc3e-01f31f1fa939] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:32:44.651699 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7146bdaf-d962-4308-ba25-b4a72fc7b3c1 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:44.652455 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7146bdaf-d962-4308-ba25-b4a72fc7b3c1 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:44.653741 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7146bdaf-d962-4308-ba25-b4a72fc7b3c1 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:44.655246 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7146bdaf-d962-4308-ba25-b4a72fc7b3c1 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:44.800489 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-7146bdaf-d962-4308-ba25-b4a72fc7b3c1 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '08abcba4284e434680ee4640f1541cf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '367c69301e6a44419af5c2d28b616b80', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:32:44.820860 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-7146bdaf-d962-4308-ba25-b4a72fc7b3c1 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '08abcba4284e434680ee4640f1541cf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '367c69301e6a44419af5c2d28b616b80', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:32:44.825812 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-7146bdaf-d962-4308-ba25-b4a72fc7b3c1 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '08abcba4284e434680ee4640f1541cf9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '367c69301e6a44419af5c2d28b616b80', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:32:44.828242 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-7146bdaf-d962-4308-ba25-b4a72fc7b3c1 tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] 10.222.0.16 "GET /compute/v2.1/servers/63bbc7c6-1f26-4d17-bc3e-01f31f1fa939" status: 200 len: 1347 microversion: 2.26 time: 0.195090 Jul 16 21:32:44.831080 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2352/4713] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:32:44 2022] GET /compute/v2.1/servers/63bbc7c6-1f26-4d17-bc3e-01f31f1fa939 => generated 1347 bytes in 199 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:32:45.556777 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-fbb76001-1b9b-402a-af2b-59e0b0224f9b tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:32:45.558079 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-fbb76001-1b9b-402a-af2b-59e0b0224f9b tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] [instance: 3c0f0855-7a9c-4d8a-95ac-3bf41d32d4e6] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:32:45.579639 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fbb76001-1b9b-402a-af2b-59e0b0224f9b tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:45.591189 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fbb76001-1b9b-402a-af2b-59e0b0224f9b tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.016s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:45.592374 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fbb76001-1b9b-402a-af2b-59e0b0224f9b tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:45.593334 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fbb76001-1b9b-402a-af2b-59e0b0224f9b tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:45.845513 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell0:SELECT=2 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:32:45.846339 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell0:SELECT=10 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:32:45.852062 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-7351a80a-6755-4c7c-b13d-2e248e89c93a tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:32:45.852685 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-7351a80a-6755-4c7c-b13d-2e248e89c93a tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] [instance: 63bbc7c6-1f26-4d17-bc3e-01f31f1fa939] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:32:45.869898 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7351a80a-6755-4c7c-b13d-2e248e89c93a tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:45.871560 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7351a80a-6755-4c7c-b13d-2e248e89c93a tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:45.875187 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7351a80a-6755-4c7c-b13d-2e248e89c93a tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:45.876716 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7351a80a-6755-4c7c-b13d-2e248e89c93a tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:45.927289 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-7351a80a-6755-4c7c-b13d-2e248e89c93a tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] HTTP exception thrown: Instance 63bbc7c6-1f26-4d17-bc3e-01f31f1fa939 could not be found. Jul 16 21:32:45.931294 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-7351a80a-6755-4c7c-b13d-2e248e89c93a tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Request method failure captured: Jul 16 21:32:45.931294 ncdv-master-1 devstack@n-api.service[138680]: request: GET /compute/v2.1/servers/63bbc7c6-1f26-4d17-bc3e-01f31f1fa939 HTTP/1.1 Jul 16 21:32:45.931294 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:32:45.931294 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:32:45.931294 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:32:45.931294 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 0 Jul 16 21:32:45.931294 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:32:45.931294 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:32:45.931294 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:32:45.931294 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:32:45.931294 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0y5fD0nTk3vyoZDCXGbDE1MZGCAi8HpfI4bINJwimUPm7r0lFGrPAjAWJJDzKcrWWc0Pn5pYojsg3njw9CzAypKsBeMoesbe1870IizUvaVYL6w-PZUVvhrehp9T7BKCC1YXkOiyP8YRKb0kbH8EepGsjKB2qVq_E4f0bIH4hDbzMHo Jul 16 21:32:45.931294 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:32:45.931294 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:32:45.931294 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:32:45.931294 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:32:45.931294 ncdv-master-1 devstack@n-api.service[138680]: X-Openstack-Nova-Api-Version: 2.26 Jul 16 21:32:45.931294 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:32:45.931294 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:32:45.931294 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: 367c69301e6a44419af5c2d28b616b80 Jul 16 21:32:45.931294 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-ServerTagsTestJSON-911661735 Jul 16 21:32:45.931294 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:32:45.931294 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:32:45.931294 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/367c69301e6a44419af5c2d28b616b80"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/367c69301e6a44419af5c2d28b616b80"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_367c69301e6a44419af5c2d28b616b80"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/367c69301e6a44419af5c2d28b616b80"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:32:45.931294 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-ServerTagsTestJSON-911661735 Jul 16 21:32:45.931294 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: 367c69301e6a44419af5c2d28b616b80 Jul 16 21:32:45.931294 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-ServerTagsTestJSON-911661735 Jul 16 21:32:45.931294 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-ServerTagsTestJSON-911661735-project Jul 16 21:32:45.931294 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:32:45.931294 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:32:45.931294 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: 08abcba4284e434680ee4640f1541cf9 Jul 16 21:32:45.931294 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-ServerTagsTestJSON-911661735-project Jul 16 21:32:45.931294 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 21:32:45.931294 ncdv-master-1 devstack@n-api.service[138680]: exception: Instance 63bbc7c6-1f26-4d17-bc3e-01f31f1fa939 could not be found. Jul 16 21:32:45.931294 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': '63bbc7c6-1f26-4d17-bc3e-01f31f1fa939'} Jul 16 21:32:45.931294 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:32:45.931294 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:32:45.931294 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:32:45.931294 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:32:45.931294 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:32:45.931294 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:32:45.931294 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:32:45.931294 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:32:45.931294 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:32:45.931294 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:32:45.931294 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 63bbc7c6-1f26-4d17-bc3e-01f31f1fa939 could not be found. Jul 16 21:32:45.931294 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:32:45.931294 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:32:45.931294 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:32:45.931294 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:32:45.931294 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:32:45.931294 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:32:45.931294 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:32:45.931294 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:32:45.931294 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:32:45.931294 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:32:45.931294 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:32:45.931294 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:32:45.931294 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:32:45.931294 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:32:45.931294 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:32:45.931294 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:32:45.931294 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 63bbc7c6-1f26-4d17-bc3e-01f31f1fa939 could not be found. Jul 16 21:32:45.931294 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:32:45.931294 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:32:45.931294 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:32:45.939121 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:32:45.939121 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:32:45.939121 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:32:45.939121 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:32:45.939121 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:32:45.939121 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 63bbc7c6-1f26-4d17-bc3e-01f31f1fa939 could not be found. Jul 16 21:32:45.939121 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:32:45.939121 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-7351a80a-6755-4c7c-b13d-2e248e89c93a tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] Returning 404 to user: Instance 63bbc7c6-1f26-4d17-bc3e-01f31f1fa939 could not be found. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:32:45.939121 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-7351a80a-6755-4c7c-b13d-2e248e89c93a tempest-ServerTagsTestJSON-911661735 tempest-ServerTagsTestJSON-911661735-project] 10.222.0.16 "GET /compute/v2.1/servers/63bbc7c6-1f26-4d17-bc3e-01f31f1fa939" status: 404 len: 111 microversion: 2.26 time: 0.090459 Jul 16 21:32:45.939121 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2353/4714] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:32:45 2022] GET /compute/v2.1/servers/63bbc7c6-1f26-4d17-bc3e-01f31f1fa939 => generated 111 bytes in 94 msecs (HTTP/1.1 404) 9 headers in 381 bytes (2 switches on core 0) Jul 16 21:32:46.305950 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-fbb76001-1b9b-402a-af2b-59e0b0224f9b tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4fa446573f7d42de816280280c3dec19', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ce30584a7dfa4ab19de73aa8b18b113b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:32:46.338246 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-fbb76001-1b9b-402a-af2b-59e0b0224f9b tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] 10.222.0.16 "GET /compute/v2.1/servers/3c0f0855-7a9c-4d8a-95ac-3bf41d32d4e6" status: 200 len: 1427 microversion: 2.1 time: 0.785556 Jul 16 21:32:46.339466 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2362/4715] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:32:45 2022] GET /compute/v2.1/servers/3c0f0855-7a9c-4d8a-95ac-3bf41d32d4e6 => generated 1427 bytes in 788 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:32:46.655746 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-ad018059-da6f-4b91-87c9-2b810a1185aa tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Action: 'action', calling method: >, body: {"shelve": {}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:32:46.656305 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-ad018059-da6f-4b91-87c9-2b810a1185aa tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] [instance: 8b77de54-4c8b-4ed5-84f5-68bf69f9809f] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:32:46.663569 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ad018059-da6f-4b91-87c9-2b810a1185aa tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:46.664054 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ad018059-da6f-4b91-87c9-2b810a1185aa tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:46.665072 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ad018059-da6f-4b91-87c9-2b810a1185aa tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:46.665575 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ad018059-da6f-4b91-87c9-2b810a1185aa tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:46.690527 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.objects.instance [None req-ad018059-da6f-4b91-87c9-2b810a1185aa tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lazy-loading 'flavor' on Instance uuid 8b77de54-4c8b-4ed5-84f5-68bf69f9809f {{(pid=138680) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:32:46.989245 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-64a828f8-764e-4d8e-8f80-326c76824463 req-2893f573-c2ca-4da3-9b69-a3bab3bd23a5 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "3c0f0855-7a9c-4d8a-95ac-3bf41d32d4e6", "name": "network-vif-plugged", "status": "completed", "tag": "a9915b3b-bd00-4ade-884c-60255466029d"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:32:46.997459 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell0:SELECT=12 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:32:47.000673 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-64a828f8-764e-4d8e-8f80-326c76824463 req-2893f573-c2ca-4da3-9b69-a3bab3bd23a5 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:47.001847 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-64a828f8-764e-4d8e-8f80-326c76824463 req-2893f573-c2ca-4da3-9b69-a3bab3bd23a5 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:47.015838 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-64a828f8-764e-4d8e-8f80-326c76824463 req-2893f573-c2ca-4da3-9b69-a3bab3bd23a5 service nova] Creating event network-vif-plugged:a9915b3b-bd00-4ade-884c-60255466029d for instance 3c0f0855-7a9c-4d8a-95ac-3bf41d32d4e6 on nch1-master-1 Jul 16 21:32:47.022563 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-64a828f8-764e-4d8e-8f80-326c76824463 req-2893f573-c2ca-4da3-9b69-a3bab3bd23a5 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.039282 Jul 16 21:32:47.027385 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2363/4716] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:32:46 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 16 21:32:47.357209 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-629379ca-b8b3-47fa-ad5d-3a9cd8547890 tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:32:47.367259 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-629379ca-b8b3-47fa-ad5d-3a9cd8547890 tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] [instance: 3c0f0855-7a9c-4d8a-95ac-3bf41d32d4e6] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:32:47.375573 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-629379ca-b8b3-47fa-ad5d-3a9cd8547890 tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:47.376335 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-629379ca-b8b3-47fa-ad5d-3a9cd8547890 tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:47.377183 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-629379ca-b8b3-47fa-ad5d-3a9cd8547890 tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:47.378097 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-629379ca-b8b3-47fa-ad5d-3a9cd8547890 tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:47.402041 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-ad018059-da6f-4b91-87c9-2b810a1185aa tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] 10.222.0.16 "POST /compute/v2.1/servers/8b77de54-4c8b-4ed5-84f5-68bf69f9809f/action" status: 202 len: 0 microversion: 2.20 time: 0.750786 Jul 16 21:32:47.404847 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2354/4717] 10.222.0.16 () {62 vars in 1318 bytes} [Sat Jul 16 21:32:46 2022] POST /compute/v2.1/servers/8b77de54-4c8b-4ed5-84f5-68bf69f9809f/action => generated 0 bytes in 754 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Jul 16 21:32:47.428297 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-fb402dbc-fbb2-47ed-b59f-74107f3b3630 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:32:47.433206 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-fb402dbc-fbb2-47ed-b59f-74107f3b3630 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] [instance: 8b77de54-4c8b-4ed5-84f5-68bf69f9809f] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:32:47.447828 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-fb402dbc-fbb2-47ed-b59f-74107f3b3630 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:47.448292 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-fb402dbc-fbb2-47ed-b59f-74107f3b3630 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:47.449119 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-fb402dbc-fbb2-47ed-b59f-74107f3b3630 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:47.449567 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-fb402dbc-fbb2-47ed-b59f-74107f3b3630 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:47.914273 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-fb402dbc-fbb2-47ed-b59f-74107f3b3630 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:32:47.921921 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-629379ca-b8b3-47fa-ad5d-3a9cd8547890 tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4fa446573f7d42de816280280c3dec19', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ce30584a7dfa4ab19de73aa8b18b113b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:32:47.923289 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-fb402dbc-fbb2-47ed-b59f-74107f3b3630 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:32:47.925679 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-fb402dbc-fbb2-47ed-b59f-74107f3b3630 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:32:47.927607 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-fb402dbc-fbb2-47ed-b59f-74107f3b3630 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] 10.222.0.16 "GET /compute/v2.1/servers/8b77de54-4c8b-4ed5-84f5-68bf69f9809f" status: 200 len: 1723 microversion: 2.20 time: 0.512373 Jul 16 21:32:47.929172 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2355/4718] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:32:47 2022] GET /compute/v2.1/servers/8b77de54-4c8b-4ed5-84f5-68bf69f9809f => generated 1723 bytes in 514 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:32:47.941624 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-629379ca-b8b3-47fa-ad5d-3a9cd8547890 tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] 10.222.0.16 "GET /compute/v2.1/servers/3c0f0855-7a9c-4d8a-95ac-3bf41d32d4e6" status: 200 len: 1427 microversion: 2.1 time: 0.586943 Jul 16 21:32:47.943263 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2364/4719] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:32:47 2022] GET /compute/v2.1/servers/3c0f0855-7a9c-4d8a-95ac-3bf41d32d4e6 => generated 1427 bytes in 592 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:32:48.529169 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-086bc4d9-3261-40dd-aa56-0cecff0a8180 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Action: 'action', calling method: >, body: {"os-getConsoleOutput": {}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:32:48.530227 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-086bc4d9-3261-40dd-aa56-0cecff0a8180 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] [instance: 88cf13fc-0fa3-4e1e-9d76-db4d93b49faf] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:32:48.541659 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-086bc4d9-3261-40dd-aa56-0cecff0a8180 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:48.541659 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-086bc4d9-3261-40dd-aa56-0cecff0a8180 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:48.541659 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-086bc4d9-3261-40dd-aa56-0cecff0a8180 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:48.541659 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-086bc4d9-3261-40dd-aa56-0cecff0a8180 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:48.601915 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-086bc4d9-3261-40dd-aa56-0cecff0a8180 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] 10.222.0.16 "POST /compute/v2.1/servers/88cf13fc-0fa3-4e1e-9d76-db4d93b49faf/action" status: 200 len: 24571 microversion: 2.1 time: 0.075844 Jul 16 21:32:48.609818 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2356/4720] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 21:32:48 2022] POST /compute/v2.1/servers/88cf13fc-0fa3-4e1e-9d76-db4d93b49faf/action => generated 24571 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 16 21:32:48.949371 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-8e27a385-a434-419e-ad1a-6672565bf5ac tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:32:48.950672 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-8e27a385-a434-419e-ad1a-6672565bf5ac tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] [instance: 8b77de54-4c8b-4ed5-84f5-68bf69f9809f] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:32:48.960722 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-9b7ecb2e-e09f-4939-80b0-6893478fcb87 tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:32:48.961581 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8e27a385-a434-419e-ad1a-6672565bf5ac tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:48.962070 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-9b7ecb2e-e09f-4939-80b0-6893478fcb87 tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] [instance: 3c0f0855-7a9c-4d8a-95ac-3bf41d32d4e6] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:32:48.962218 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8e27a385-a434-419e-ad1a-6672565bf5ac tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:48.962987 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8e27a385-a434-419e-ad1a-6672565bf5ac tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:48.963763 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8e27a385-a434-419e-ad1a-6672565bf5ac tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:48.970138 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9b7ecb2e-e09f-4939-80b0-6893478fcb87 tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:48.970966 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9b7ecb2e-e09f-4939-80b0-6893478fcb87 tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:48.971952 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9b7ecb2e-e09f-4939-80b0-6893478fcb87 tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:48.972620 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9b7ecb2e-e09f-4939-80b0-6893478fcb87 tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:49.227254 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-8e27a385-a434-419e-ad1a-6672565bf5ac tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:32:49.236298 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-9b7ecb2e-e09f-4939-80b0-6893478fcb87 tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4fa446573f7d42de816280280c3dec19', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ce30584a7dfa4ab19de73aa8b18b113b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:32:49.244493 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-8e27a385-a434-419e-ad1a-6672565bf5ac tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:32:49.246070 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-8e27a385-a434-419e-ad1a-6672565bf5ac tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:32:49.247315 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-8e27a385-a434-419e-ad1a-6672565bf5ac tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] 10.222.0.16 "GET /compute/v2.1/servers/8b77de54-4c8b-4ed5-84f5-68bf69f9809f" status: 200 len: 1723 microversion: 2.20 time: 0.304659 Jul 16 21:32:49.267057 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-9b7ecb2e-e09f-4939-80b0-6893478fcb87 tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] 10.222.0.16 "GET /compute/v2.1/servers/3c0f0855-7a9c-4d8a-95ac-3bf41d32d4e6" status: 200 len: 1427 microversion: 2.1 time: 0.310555 Jul 16 21:32:49.267370 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2365/4721] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:32:48 2022] GET /compute/v2.1/servers/8b77de54-4c8b-4ed5-84f5-68bf69f9809f => generated 1723 bytes in 323 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:32:49.277533 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2357/4722] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:32:48 2022] GET /compute/v2.1/servers/3c0f0855-7a9c-4d8a-95ac-3bf41d32d4e6 => generated 1427 bytes in 323 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:32:50.266185 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-2813f9f2-43ae-4f8a-a97f-cdac597cb28a tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:32:50.268233 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-2813f9f2-43ae-4f8a-a97f-cdac597cb28a tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] [instance: 8b77de54-4c8b-4ed5-84f5-68bf69f9809f] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:32:50.275531 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2813f9f2-43ae-4f8a-a97f-cdac597cb28a tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:50.276109 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2813f9f2-43ae-4f8a-a97f-cdac597cb28a tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:50.277155 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2813f9f2-43ae-4f8a-a97f-cdac597cb28a tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:50.277841 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2813f9f2-43ae-4f8a-a97f-cdac597cb28a tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:50.297802 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-c68888e3-41e8-4796-91b9-59e2a75d35bf tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:32:50.298461 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-c68888e3-41e8-4796-91b9-59e2a75d35bf tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] [instance: 3c0f0855-7a9c-4d8a-95ac-3bf41d32d4e6] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:32:50.315524 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c68888e3-41e8-4796-91b9-59e2a75d35bf tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:50.317112 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c68888e3-41e8-4796-91b9-59e2a75d35bf tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:50.317112 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c68888e3-41e8-4796-91b9-59e2a75d35bf tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:50.317363 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c68888e3-41e8-4796-91b9-59e2a75d35bf tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:50.600908 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-2813f9f2-43ae-4f8a-a97f-cdac597cb28a tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:32:50.614745 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-2813f9f2-43ae-4f8a-a97f-cdac597cb28a tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:32:50.617576 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-2813f9f2-43ae-4f8a-a97f-cdac597cb28a tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:32:50.618867 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-2813f9f2-43ae-4f8a-a97f-cdac597cb28a tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] 10.222.0.16 "GET /compute/v2.1/servers/8b77de54-4c8b-4ed5-84f5-68bf69f9809f" status: 200 len: 1723 microversion: 2.20 time: 0.356631 Jul 16 21:32:50.619432 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2366/4723] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:32:50 2022] GET /compute/v2.1/servers/8b77de54-4c8b-4ed5-84f5-68bf69f9809f => generated 1723 bytes in 358 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:32:50.664794 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-c68888e3-41e8-4796-91b9-59e2a75d35bf tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4fa446573f7d42de816280280c3dec19', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ce30584a7dfa4ab19de73aa8b18b113b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:32:50.681006 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-c68888e3-41e8-4796-91b9-59e2a75d35bf tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] 10.222.0.16 "GET /compute/v2.1/servers/3c0f0855-7a9c-4d8a-95ac-3bf41d32d4e6" status: 200 len: 1607 microversion: 2.1 time: 0.399307 Jul 16 21:32:50.683144 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2358/4724] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:32:50 2022] GET /compute/v2.1/servers/3c0f0855-7a9c-4d8a-95ac-3bf41d32d4e6 => generated 1607 bytes in 402 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:32:50.703649 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-3e24cc2c-2b1a-4f9b-9766-e79c4e8ba995 tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:32:50.704693 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-3e24cc2c-2b1a-4f9b-9766-e79c4e8ba995 tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] [instance: 3c0f0855-7a9c-4d8a-95ac-3bf41d32d4e6] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:32:50.716175 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3e24cc2c-2b1a-4f9b-9766-e79c4e8ba995 tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:50.717596 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3e24cc2c-2b1a-4f9b-9766-e79c4e8ba995 tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:50.718548 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3e24cc2c-2b1a-4f9b-9766-e79c4e8ba995 tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:50.719370 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3e24cc2c-2b1a-4f9b-9766-e79c4e8ba995 tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:50.996492 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-3e24cc2c-2b1a-4f9b-9766-e79c4e8ba995 tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4fa446573f7d42de816280280c3dec19', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ce30584a7dfa4ab19de73aa8b18b113b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:32:51.005235 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-3e24cc2c-2b1a-4f9b-9766-e79c4e8ba995 tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] 10.222.0.16 "GET /compute/v2.1/servers/3c0f0855-7a9c-4d8a-95ac-3bf41d32d4e6" status: 200 len: 1607 microversion: 2.1 time: 0.307722 Jul 16 21:32:51.005819 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2367/4725] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:32:50 2022] GET /compute/v2.1/servers/3c0f0855-7a9c-4d8a-95ac-3bf41d32d4e6 => generated 1607 bytes in 309 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:32:51.022498 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-f58e212a-cd87-4f98-b7e5-92c48ff49f21 tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:32:51.031606 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-f58e212a-cd87-4f98-b7e5-92c48ff49f21 tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] [instance: 3c0f0855-7a9c-4d8a-95ac-3bf41d32d4e6] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:32:51.031606 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f58e212a-cd87-4f98-b7e5-92c48ff49f21 tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:51.032002 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f58e212a-cd87-4f98-b7e5-92c48ff49f21 tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:51.032650 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f58e212a-cd87-4f98-b7e5-92c48ff49f21 tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:51.033135 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f58e212a-cd87-4f98-b7e5-92c48ff49f21 tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:51.298330 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-f58e212a-cd87-4f98-b7e5-92c48ff49f21 tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4fa446573f7d42de816280280c3dec19', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ce30584a7dfa4ab19de73aa8b18b113b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:32:51.307406 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-f58e212a-cd87-4f98-b7e5-92c48ff49f21 tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] 10.222.0.16 "GET /compute/v2.1/servers/3c0f0855-7a9c-4d8a-95ac-3bf41d32d4e6" status: 200 len: 1607 microversion: 2.1 time: 0.289684 Jul 16 21:32:51.308621 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2359/4726] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:32:51 2022] GET /compute/v2.1/servers/3c0f0855-7a9c-4d8a-95ac-3bf41d32d4e6 => generated 1607 bytes in 292 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:32:51.640219 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-97fb5472-61ff-482f-bdc1-00e47dab6504 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:32:51.641820 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-97fb5472-61ff-482f-bdc1-00e47dab6504 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] [instance: 8b77de54-4c8b-4ed5-84f5-68bf69f9809f] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:32:51.652530 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-97fb5472-61ff-482f-bdc1-00e47dab6504 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:51.652991 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-97fb5472-61ff-482f-bdc1-00e47dab6504 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:51.653793 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-97fb5472-61ff-482f-bdc1-00e47dab6504 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:51.657964 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-97fb5472-61ff-482f-bdc1-00e47dab6504 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:52.021297 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-97fb5472-61ff-482f-bdc1-00e47dab6504 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:32:52.032598 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-97fb5472-61ff-482f-bdc1-00e47dab6504 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:32:52.035865 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-97fb5472-61ff-482f-bdc1-00e47dab6504 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:32:52.037415 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-97fb5472-61ff-482f-bdc1-00e47dab6504 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] 10.222.0.16 "GET /compute/v2.1/servers/8b77de54-4c8b-4ed5-84f5-68bf69f9809f" status: 200 len: 1723 microversion: 2.20 time: 0.403418 Jul 16 21:32:52.038454 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2368/4727] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:32:51 2022] GET /compute/v2.1/servers/8b77de54-4c8b-4ed5-84f5-68bf69f9809f => generated 1723 bytes in 405 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:32:53.056884 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-ff706084-0dc5-4736-a2dc-b4edb8298f15 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:32:53.057503 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-ff706084-0dc5-4736-a2dc-b4edb8298f15 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] [instance: 8b77de54-4c8b-4ed5-84f5-68bf69f9809f] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:32:53.068476 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ff706084-0dc5-4736-a2dc-b4edb8298f15 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:53.069292 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ff706084-0dc5-4736-a2dc-b4edb8298f15 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:53.070244 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ff706084-0dc5-4736-a2dc-b4edb8298f15 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:53.070698 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ff706084-0dc5-4736-a2dc-b4edb8298f15 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:53.087387 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=94,nova_cell1:DELETE=2,nova_cell1:INSERT=2,nova_cell1:UPDATE=2 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:32:53.258186 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-ff706084-0dc5-4736-a2dc-b4edb8298f15 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:32:53.270616 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-ff706084-0dc5-4736-a2dc-b4edb8298f15 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:32:53.272849 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-ff706084-0dc5-4736-a2dc-b4edb8298f15 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:32:53.274255 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-ff706084-0dc5-4736-a2dc-b4edb8298f15 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] 10.222.0.16 "GET /compute/v2.1/servers/8b77de54-4c8b-4ed5-84f5-68bf69f9809f" status: 200 len: 1723 microversion: 2.20 time: 0.221870 Jul 16 21:32:53.285020 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2360/4728] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:32:53 2022] GET /compute/v2.1/servers/8b77de54-4c8b-4ed5-84f5-68bf69f9809f => generated 1723 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:32:54.293020 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-bf92fb1f-24e0-4c07-a7b3-58e06e075463 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:32:54.293722 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-bf92fb1f-24e0-4c07-a7b3-58e06e075463 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] [instance: 8b77de54-4c8b-4ed5-84f5-68bf69f9809f] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:32:54.306843 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-bf92fb1f-24e0-4c07-a7b3-58e06e075463 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:54.309368 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-bf92fb1f-24e0-4c07-a7b3-58e06e075463 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:54.310834 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-bf92fb1f-24e0-4c07-a7b3-58e06e075463 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:54.311497 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-bf92fb1f-24e0-4c07-a7b3-58e06e075463 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:54.648971 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-bf92fb1f-24e0-4c07-a7b3-58e06e075463 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:32:54.662377 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-bf92fb1f-24e0-4c07-a7b3-58e06e075463 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:32:54.675843 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-bf92fb1f-24e0-4c07-a7b3-58e06e075463 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:32:54.675843 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-bf92fb1f-24e0-4c07-a7b3-58e06e075463 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] 10.222.0.16 "GET /compute/v2.1/servers/8b77de54-4c8b-4ed5-84f5-68bf69f9809f" status: 200 len: 1723 microversion: 2.20 time: 0.383073 Jul 16 21:32:54.685438 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2369/4729] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:32:54 2022] GET /compute/v2.1/servers/8b77de54-4c8b-4ed5-84f5-68bf69f9809f => generated 1723 bytes in 388 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:32:55.070226 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-38f058a5-e1a1-4dde-b67e-4f981e7cb424 req-b4f462ca-36e3-488c-b6a0-82052d335624 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "3c0f0855-7a9c-4d8a-95ac-3bf41d32d4e6", "tag": "a9915b3b-bd00-4ade-884c-60255466029d"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:32:55.093847 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-38f058a5-e1a1-4dde-b67e-4f981e7cb424 req-b4f462ca-36e3-488c-b6a0-82052d335624 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:55.094324 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-38f058a5-e1a1-4dde-b67e-4f981e7cb424 req-b4f462ca-36e3-488c-b6a0-82052d335624 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:55.119426 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-38f058a5-e1a1-4dde-b67e-4f981e7cb424 req-b4f462ca-36e3-488c-b6a0-82052d335624 service nova] Creating event network-changed:a9915b3b-bd00-4ade-884c-60255466029d for instance 3c0f0855-7a9c-4d8a-95ac-3bf41d32d4e6 on nch1-master-1 Jul 16 21:32:55.125644 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-38f058a5-e1a1-4dde-b67e-4f981e7cb424 req-b4f462ca-36e3-488c-b6a0-82052d335624 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.058939 Jul 16 21:32:55.126492 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2361/4730] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:32:55 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 61 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:32:55.698863 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-a0a5997a-70d6-493b-bca3-6f559bf3e546 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:32:55.698863 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-a0a5997a-70d6-493b-bca3-6f559bf3e546 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] [instance: 8b77de54-4c8b-4ed5-84f5-68bf69f9809f] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:32:55.703212 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a0a5997a-70d6-493b-bca3-6f559bf3e546 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:55.704100 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a0a5997a-70d6-493b-bca3-6f559bf3e546 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:55.706116 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a0a5997a-70d6-493b-bca3-6f559bf3e546 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:55.706896 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a0a5997a-70d6-493b-bca3-6f559bf3e546 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:55.712499 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=95,nova_cell1:SAVEPOINT=1,nova_cell1:INSERT=2,nova_cell1:RELEASE=1,nova_cell1:UPDATE=1 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:32:56.323489 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-a0a5997a-70d6-493b-bca3-6f559bf3e546 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:32:56.358719 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-a0a5997a-70d6-493b-bca3-6f559bf3e546 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:32:56.366745 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-a0a5997a-70d6-493b-bca3-6f559bf3e546 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:32:56.369165 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-a0a5997a-70d6-493b-bca3-6f559bf3e546 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] 10.222.0.16 "GET /compute/v2.1/servers/8b77de54-4c8b-4ed5-84f5-68bf69f9809f" status: 200 len: 1723 microversion: 2.20 time: 0.681205 Jul 16 21:32:56.381429 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2370/4731] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:32:55 2022] GET /compute/v2.1/servers/8b77de54-4c8b-4ed5-84f5-68bf69f9809f => generated 1723 bytes in 683 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:32:57.390487 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-16c821c0-7fec-41ee-9f69-ff23015302ab tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:32:57.391901 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-16c821c0-7fec-41ee-9f69-ff23015302ab tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] [instance: 8b77de54-4c8b-4ed5-84f5-68bf69f9809f] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:32:57.400668 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-16c821c0-7fec-41ee-9f69-ff23015302ab tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:57.406106 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-16c821c0-7fec-41ee-9f69-ff23015302ab tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:57.406106 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-16c821c0-7fec-41ee-9f69-ff23015302ab tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:57.406106 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-16c821c0-7fec-41ee-9f69-ff23015302ab tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:57.770643 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-16c821c0-7fec-41ee-9f69-ff23015302ab tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:32:57.787906 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-16c821c0-7fec-41ee-9f69-ff23015302ab tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:32:57.792266 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-16c821c0-7fec-41ee-9f69-ff23015302ab tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:32:57.797844 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-16c821c0-7fec-41ee-9f69-ff23015302ab tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] 10.222.0.16 "GET /compute/v2.1/servers/8b77de54-4c8b-4ed5-84f5-68bf69f9809f" status: 200 len: 1723 microversion: 2.20 time: 0.410743 Jul 16 21:32:57.798854 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2362/4732] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:32:57 2022] GET /compute/v2.1/servers/8b77de54-4c8b-4ed5-84f5-68bf69f9809f => generated 1723 bytes in 415 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:32:58.833550 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-4ecb7451-de20-4bae-a35c-220c9567241f tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:32:58.834195 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-4ecb7451-de20-4bae-a35c-220c9567241f tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] [instance: 8b77de54-4c8b-4ed5-84f5-68bf69f9809f] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:32:58.849253 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4ecb7451-de20-4bae-a35c-220c9567241f tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:58.850072 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4ecb7451-de20-4bae-a35c-220c9567241f tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:58.851081 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4ecb7451-de20-4bae-a35c-220c9567241f tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:58.855873 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4ecb7451-de20-4bae-a35c-220c9567241f tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:59.637265 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-4ecb7451-de20-4bae-a35c-220c9567241f tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:32:59.645434 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-4ecb7451-de20-4bae-a35c-220c9567241f tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:32:59.647497 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-4ecb7451-de20-4bae-a35c-220c9567241f tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:32:59.648797 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-4ecb7451-de20-4bae-a35c-220c9567241f tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] 10.222.0.16 "GET /compute/v2.1/servers/8b77de54-4c8b-4ed5-84f5-68bf69f9809f" status: 200 len: 1723 microversion: 2.20 time: 0.831557 Jul 16 21:32:59.651176 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2371/4733] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:32:58 2022] GET /compute/v2.1/servers/8b77de54-4c8b-4ed5-84f5-68bf69f9809f => generated 1723 bytes in 833 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 16 21:32:59.665889 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-935351b4-21b2-4781-b9cc-8fc536a8d3fd tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Action: 'action', calling method: >, body: {"os-getConsoleOutput": {}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:32:59.667541 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-935351b4-21b2-4781-b9cc-8fc536a8d3fd tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] [instance: 88cf13fc-0fa3-4e1e-9d76-db4d93b49faf] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:32:59.676134 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-935351b4-21b2-4781-b9cc-8fc536a8d3fd tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:59.676602 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-935351b4-21b2-4781-b9cc-8fc536a8d3fd tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:59.680394 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-935351b4-21b2-4781-b9cc-8fc536a8d3fd tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:32:59.680394 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-935351b4-21b2-4781-b9cc-8fc536a8d3fd tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:32:59.739938 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-935351b4-21b2-4781-b9cc-8fc536a8d3fd tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] 10.222.0.16 "POST /compute/v2.1/servers/88cf13fc-0fa3-4e1e-9d76-db4d93b49faf/action" status: 200 len: 32638 microversion: 2.1 time: 0.078501 Jul 16 21:32:59.746519 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2363/4734] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 21:32:59 2022] POST /compute/v2.1/servers/88cf13fc-0fa3-4e1e-9d76-db4d93b49faf/action => generated 32638 bytes in 80 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 16 21:33:00.669313 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-4fa87eda-29d3-4570-91ac-6d119508ecdb tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:33:00.677659 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-4fa87eda-29d3-4570-91ac-6d119508ecdb tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] [instance: 8b77de54-4c8b-4ed5-84f5-68bf69f9809f] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:33:00.695494 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4fa87eda-29d3-4570-91ac-6d119508ecdb tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:00.700627 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4fa87eda-29d3-4570-91ac-6d119508ecdb tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:00.700627 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4fa87eda-29d3-4570-91ac-6d119508ecdb tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:00.701351 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4fa87eda-29d3-4570-91ac-6d119508ecdb tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:01.116122 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-4fa87eda-29d3-4570-91ac-6d119508ecdb tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:33:01.126157 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-4fa87eda-29d3-4570-91ac-6d119508ecdb tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:33:01.128253 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-4fa87eda-29d3-4570-91ac-6d119508ecdb tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:33:01.129663 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-4fa87eda-29d3-4570-91ac-6d119508ecdb tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] 10.222.0.16 "GET /compute/v2.1/servers/8b77de54-4c8b-4ed5-84f5-68bf69f9809f" status: 200 len: 1723 microversion: 2.20 time: 0.465567 Jul 16 21:33:01.130470 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2372/4735] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:33:00 2022] GET /compute/v2.1/servers/8b77de54-4c8b-4ed5-84f5-68bf69f9809f => generated 1723 bytes in 467 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:33:01.914012 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-1a18b334-1faf-49c5-aa08-e851f411f424 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:33:01.914565 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-1a18b334-1faf-49c5-aa08-e851f411f424 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] [instance: 88cf13fc-0fa3-4e1e-9d76-db4d93b49faf] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:33:01.927989 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-d469469c-e831-4c93-89a0-fb431ef5bb2c req-04169517-146d-4a16-94ae-d51e24a0400a service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "88cf13fc-0fa3-4e1e-9d76-db4d93b49faf", "tag": "008d375b-5b42-4275-ac14-31bcaa9250a8"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:33:01.932377 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1a18b334-1faf-49c5-aa08-e851f411f424 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:01.932837 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1a18b334-1faf-49c5-aa08-e851f411f424 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:01.934142 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-d469469c-e831-4c93-89a0-fb431ef5bb2c req-04169517-146d-4a16-94ae-d51e24a0400a service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:01.934689 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-d469469c-e831-4c93-89a0-fb431ef5bb2c req-04169517-146d-4a16-94ae-d51e24a0400a service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:01.941325 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1a18b334-1faf-49c5-aa08-e851f411f424 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:01.942408 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1a18b334-1faf-49c5-aa08-e851f411f424 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:01.968256 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-d469469c-e831-4c93-89a0-fb431ef5bb2c req-04169517-146d-4a16-94ae-d51e24a0400a service nova] Creating event network-changed:008d375b-5b42-4275-ac14-31bcaa9250a8 for instance 88cf13fc-0fa3-4e1e-9d76-db4d93b49faf on nch1-master-1 Jul 16 21:33:01.983388 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-d469469c-e831-4c93-89a0-fb431ef5bb2c req-04169517-146d-4a16-94ae-d51e24a0400a service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.067858 Jul 16 21:33:01.984714 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2373/4736] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:33:01 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 70 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:33:02.039720 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-1a18b334-1faf-49c5-aa08-e851f411f424 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] [instance: 88cf13fc-0fa3-4e1e-9d76-db4d93b49faf] Going to try to terminate instance {{(pid=138680) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 16 21:33:02.149076 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-87da44a5-5beb-468e-9749-72bb1b3d9b29 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:33:02.150308 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-87da44a5-5beb-468e-9749-72bb1b3d9b29 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] [instance: 8b77de54-4c8b-4ed5-84f5-68bf69f9809f] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:33:02.163328 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-87da44a5-5beb-468e-9749-72bb1b3d9b29 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:02.165818 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-87da44a5-5beb-468e-9749-72bb1b3d9b29 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:02.166740 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-87da44a5-5beb-468e-9749-72bb1b3d9b29 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:02.168450 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-87da44a5-5beb-468e-9749-72bb1b3d9b29 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:02.494962 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-1a18b334-1faf-49c5-aa08-e851f411f424 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] 10.222.0.16 "DELETE /compute/v2.1/servers/88cf13fc-0fa3-4e1e-9d76-db4d93b49faf" status: 204 len: 0 microversion: 2.1 time: 0.584185 Jul 16 21:33:02.494962 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2364/4737] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:33:01 2022] DELETE /compute/v2.1/servers/88cf13fc-0fa3-4e1e-9d76-db4d93b49faf => generated 0 bytes in 586 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 16 21:33:02.507097 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-8c9fdb08-cd3d-4f9d-936e-2d28f62ee383 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:33:02.507825 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-8c9fdb08-cd3d-4f9d-936e-2d28f62ee383 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] [instance: 88cf13fc-0fa3-4e1e-9d76-db4d93b49faf] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:33:02.517480 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_api:SELECT=96,nova_api:INSERT=3,nova_api:UPDATE=1 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:33:02.531149 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8c9fdb08-cd3d-4f9d-936e-2d28f62ee383 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:02.536117 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8c9fdb08-cd3d-4f9d-936e-2d28f62ee383 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:02.537797 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8c9fdb08-cd3d-4f9d-936e-2d28f62ee383 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:02.538271 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8c9fdb08-cd3d-4f9d-936e-2d28f62ee383 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:03.003313 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-87da44a5-5beb-468e-9749-72bb1b3d9b29 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:33:03.015922 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-87da44a5-5beb-468e-9749-72bb1b3d9b29 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:33:03.019261 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-87da44a5-5beb-468e-9749-72bb1b3d9b29 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:33:03.021843 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-87da44a5-5beb-468e-9749-72bb1b3d9b29 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] 10.222.0.16 "GET /compute/v2.1/servers/8b77de54-4c8b-4ed5-84f5-68bf69f9809f" status: 200 len: 1723 microversion: 2.20 time: 0.877311 Jul 16 21:33:03.023182 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2374/4738] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:33:02 2022] GET /compute/v2.1/servers/8b77de54-4c8b-4ed5-84f5-68bf69f9809f => generated 1723 bytes in 880 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:33:03.090826 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-8c9fdb08-cd3d-4f9d-936e-2d28f62ee383 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ddfae3448704591b662de83b97a1876', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bed7c94624484c7cb8abeb3bfc8757b9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:33:03.104581 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-8c9fdb08-cd3d-4f9d-936e-2d28f62ee383 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] 10.222.0.16 "GET /compute/v2.1/servers/88cf13fc-0fa3-4e1e-9d76-db4d93b49faf" status: 200 len: 1669 microversion: 2.1 time: 0.605484 Jul 16 21:33:03.106296 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2365/4739] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:33:02 2022] GET /compute/v2.1/servers/88cf13fc-0fa3-4e1e-9d76-db4d93b49faf => generated 1669 bytes in 608 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:33:03.497114 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-f8f71a03-fb05-4329-a63b-259a7b18d98a tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Action: 'action', calling method: >, body: {"os-getConsoleOutput": {}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:33:03.498551 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-f8f71a03-fb05-4329-a63b-259a7b18d98a tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] [instance: 4eb3d8d6-79ae-4109-a5a7-835c459d1719] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:33:03.506974 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f8f71a03-fb05-4329-a63b-259a7b18d98a tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:03.507976 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f8f71a03-fb05-4329-a63b-259a7b18d98a tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:03.516100 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f8f71a03-fb05-4329-a63b-259a7b18d98a tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:03.516100 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f8f71a03-fb05-4329-a63b-259a7b18d98a tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:03.596813 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-f8f71a03-fb05-4329-a63b-259a7b18d98a tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] 10.222.0.16 "POST /compute/v2.1/servers/4eb3d8d6-79ae-4109-a5a7-835c459d1719/action" status: 200 len: 32474 microversion: 2.32 time: 0.100410 Jul 16 21:33:03.607518 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2375/4740] 10.222.0.16 () {62 vars in 1318 bytes} [Sat Jul 16 21:33:03 2022] POST /compute/v2.1/servers/4eb3d8d6-79ae-4109-a5a7-835c459d1719/action => generated 32474 bytes in 115 msecs (HTTP/1.1 200) 9 headers in 361 bytes (1 switches on core 0) Jul 16 21:33:03.612602 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-533078d4-40db-42a6-99c8-11dde78d7e6f tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:33:03.619617 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-533078d4-40db-42a6-99c8-11dde78d7e6f tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] [instance: 4eb3d8d6-79ae-4109-a5a7-835c459d1719] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:33:03.623396 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-533078d4-40db-42a6-99c8-11dde78d7e6f tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:03.626524 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-533078d4-40db-42a6-99c8-11dde78d7e6f tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:03.629633 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-533078d4-40db-42a6-99c8-11dde78d7e6f tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:03.630098 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-533078d4-40db-42a6-99c8-11dde78d7e6f tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:03.688904 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-533078d4-40db-42a6-99c8-11dde78d7e6f tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] [instance: 4eb3d8d6-79ae-4109-a5a7-835c459d1719] Going to try to terminate instance {{(pid=138680) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 16 21:33:04.043100 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-660c0076-96e0-4251-9804-c8b04ee4297a tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:33:04.047159 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-660c0076-96e0-4251-9804-c8b04ee4297a tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] [instance: 8b77de54-4c8b-4ed5-84f5-68bf69f9809f] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:33:04.054073 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-660c0076-96e0-4251-9804-c8b04ee4297a tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:04.054797 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-660c0076-96e0-4251-9804-c8b04ee4297a tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:04.055780 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-660c0076-96e0-4251-9804-c8b04ee4297a tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:04.056408 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-660c0076-96e0-4251-9804-c8b04ee4297a tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:04.098293 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-533078d4-40db-42a6-99c8-11dde78d7e6f tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] 10.222.0.16 "DELETE /compute/v2.1/servers/4eb3d8d6-79ae-4109-a5a7-835c459d1719" status: 204 len: 0 microversion: 2.32 time: 0.490990 Jul 16 21:33:04.099025 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2366/4741] 10.222.0.16 () {60 vars in 1279 bytes} [Sat Jul 16 21:33:03 2022] DELETE /compute/v2.1/servers/4eb3d8d6-79ae-4109-a5a7-835c459d1719 => generated 0 bytes in 493 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Jul 16 21:33:04.110673 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-749e56ab-f7dd-42ec-8b45-5333e747d987 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:33:04.115139 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-749e56ab-f7dd-42ec-8b45-5333e747d987 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] [instance: 4eb3d8d6-79ae-4109-a5a7-835c459d1719] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:33:04.128011 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-749e56ab-f7dd-42ec-8b45-5333e747d987 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:04.138287 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-749e56ab-f7dd-42ec-8b45-5333e747d987 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.010s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:04.139384 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-749e56ab-f7dd-42ec-8b45-5333e747d987 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:04.140047 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-749e56ab-f7dd-42ec-8b45-5333e747d987 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:04.854638 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-660c0076-96e0-4251-9804-c8b04ee4297a tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:33:04.867691 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-660c0076-96e0-4251-9804-c8b04ee4297a tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:33:04.872572 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-660c0076-96e0-4251-9804-c8b04ee4297a tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:33:04.876043 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-660c0076-96e0-4251-9804-c8b04ee4297a tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] 10.222.0.16 "GET /compute/v2.1/servers/8b77de54-4c8b-4ed5-84f5-68bf69f9809f" status: 200 len: 1744 microversion: 2.20 time: 0.837610 Jul 16 21:33:04.881642 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2376/4742] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:33:04 2022] GET /compute/v2.1/servers/8b77de54-4c8b-4ed5-84f5-68bf69f9809f => generated 1744 bytes in 839 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 16 21:33:04.883330 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-529f6599-0506-4294-8b63-92b3db767e90 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:33:04.883961 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-529f6599-0506-4294-8b63-92b3db767e90 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] [instance: 88cf13fc-0fa3-4e1e-9d76-db4d93b49faf] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:33:04.890133 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-529f6599-0506-4294-8b63-92b3db767e90 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:04.890582 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-529f6599-0506-4294-8b63-92b3db767e90 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:04.891397 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-529f6599-0506-4294-8b63-92b3db767e90 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:04.892381 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-529f6599-0506-4294-8b63-92b3db767e90 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:05.120949 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-749e56ab-f7dd-42ec-8b45-5333e747d987 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a23fafbab18d41ee82b84bbdf94cf0fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfc2e42a0df346f4adef152ba2e3fa71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:33:05.147032 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-749e56ab-f7dd-42ec-8b45-5333e747d987 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'a23fafbab18d41ee82b84bbdf94cf0fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfc2e42a0df346f4adef152ba2e3fa71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:33:05.148982 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-749e56ab-f7dd-42ec-8b45-5333e747d987 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'a23fafbab18d41ee82b84bbdf94cf0fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfc2e42a0df346f4adef152ba2e3fa71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:33:05.150695 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-749e56ab-f7dd-42ec-8b45-5333e747d987 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] 10.222.0.16 "GET /compute/v2.1/servers/4eb3d8d6-79ae-4109-a5a7-835c459d1719" status: 200 len: 2948 microversion: 2.32 time: 1.046587 Jul 16 21:33:05.151544 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2367/4743] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:33:04 2022] GET /compute/v2.1/servers/4eb3d8d6-79ae-4109-a5a7-835c459d1719 => generated 2948 bytes in 1048 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:33:05.412894 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-e5420884-c442-465a-a09c-1aaf2846716d tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServersNegativeTestJSON-server-735400319", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452", "networks": [{"uuid": "f14874ef-c1c1-4761-ab34-120b93719d08"}]}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:33:05.540988 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.network.neutron [None req-e5420884-c442-465a-a09c-1aaf2846716d tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] validate_networks() for [('f14874ef-c1c1-4761-ab34-120b93719d08', None, None, None, None, None)] {{(pid=138680) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 16 21:33:05.867630 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-529f6599-0506-4294-8b63-92b3db767e90 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ddfae3448704591b662de83b97a1876', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bed7c94624484c7cb8abeb3bfc8757b9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:33:05.890770 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-529f6599-0506-4294-8b63-92b3db767e90 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] 10.222.0.16 "GET /compute/v2.1/servers/88cf13fc-0fa3-4e1e-9d76-db4d93b49faf" status: 200 len: 1669 microversion: 2.1 time: 1.010793 Jul 16 21:33:05.894016 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2377/4744] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:33:04 2022] GET /compute/v2.1/servers/88cf13fc-0fa3-4e1e-9d76-db4d93b49faf => generated 1669 bytes in 1016 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:33:05.909532 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-0d311e9e-fee9-417f-9a6f-858155d5d464 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:33:05.911609 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-0d311e9e-fee9-417f-9a6f-858155d5d464 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] [instance: 8b77de54-4c8b-4ed5-84f5-68bf69f9809f] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:33:05.921061 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0d311e9e-fee9-417f-9a6f-858155d5d464 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:05.922346 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0d311e9e-fee9-417f-9a6f-858155d5d464 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:05.924897 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0d311e9e-fee9-417f-9a6f-858155d5d464 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:05.925833 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0d311e9e-fee9-417f-9a6f-858155d5d464 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:06.654861 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-0d311e9e-fee9-417f-9a6f-858155d5d464 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:33:06.670202 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-0d311e9e-fee9-417f-9a6f-858155d5d464 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:33:06.671928 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-0d311e9e-fee9-417f-9a6f-858155d5d464 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:33:06.690333 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-0d311e9e-fee9-417f-9a6f-858155d5d464 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] 10.222.0.16 "GET /compute/v2.1/servers/8b77de54-4c8b-4ed5-84f5-68bf69f9809f" status: 200 len: 1739 microversion: 2.20 time: 0.780506 Jul 16 21:33:06.697351 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2378/4745] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:33:05 2022] GET /compute/v2.1/servers/8b77de54-4c8b-4ed5-84f5-68bf69f9809f => generated 1739 bytes in 802 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:33:06.713646 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-657e75e1-25ee-4ecd-8b2b-85345f019653 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:33:06.715504 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-657e75e1-25ee-4ecd-8b2b-85345f019653 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] [instance: 4eb3d8d6-79ae-4109-a5a7-835c459d1719] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:33:06.742902 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-657e75e1-25ee-4ecd-8b2b-85345f019653 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:06.743456 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-657e75e1-25ee-4ecd-8b2b-85345f019653 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:06.744199 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-657e75e1-25ee-4ecd-8b2b-85345f019653 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:06.744647 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-657e75e1-25ee-4ecd-8b2b-85345f019653 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:07.926719 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-657e75e1-25ee-4ecd-8b2b-85345f019653 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a23fafbab18d41ee82b84bbdf94cf0fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfc2e42a0df346f4adef152ba2e3fa71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:33:07.941847 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-657e75e1-25ee-4ecd-8b2b-85345f019653 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'a23fafbab18d41ee82b84bbdf94cf0fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfc2e42a0df346f4adef152ba2e3fa71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:33:07.943606 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-657e75e1-25ee-4ecd-8b2b-85345f019653 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'a23fafbab18d41ee82b84bbdf94cf0fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfc2e42a0df346f4adef152ba2e3fa71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:33:07.945582 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-657e75e1-25ee-4ecd-8b2b-85345f019653 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] 10.222.0.16 "GET /compute/v2.1/servers/4eb3d8d6-79ae-4109-a5a7-835c459d1719" status: 200 len: 2948 microversion: 2.32 time: 1.245854 Jul 16 21:33:07.946277 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2379/4746] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:33:06 2022] GET /compute/v2.1/servers/4eb3d8d6-79ae-4109-a5a7-835c459d1719 => generated 2948 bytes in 1247 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:33:07.951252 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-0457ac6b-5ae3-40e3-855c-e327a3ac65a4 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:33:07.952409 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-0457ac6b-5ae3-40e3-855c-e327a3ac65a4 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] [instance: 88cf13fc-0fa3-4e1e-9d76-db4d93b49faf] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:33:07.959638 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0457ac6b-5ae3-40e3-855c-e327a3ac65a4 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:07.960344 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0457ac6b-5ae3-40e3-855c-e327a3ac65a4 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:07.961412 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0457ac6b-5ae3-40e3-855c-e327a3ac65a4 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:07.962325 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0457ac6b-5ae3-40e3-855c-e327a3ac65a4 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:08.119722 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-0457ac6b-5ae3-40e3-855c-e327a3ac65a4 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4ddfae3448704591b662de83b97a1876', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bed7c94624484c7cb8abeb3bfc8757b9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:33:08.139263 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-0457ac6b-5ae3-40e3-855c-e327a3ac65a4 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] 10.222.0.16 "GET /compute/v2.1/servers/88cf13fc-0fa3-4e1e-9d76-db4d93b49faf" status: 200 len: 1483 microversion: 2.1 time: 0.190200 Jul 16 21:33:08.154219 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2380/4747] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:33:07 2022] GET /compute/v2.1/servers/88cf13fc-0fa3-4e1e-9d76-db4d93b49faf => generated 1483 bytes in 207 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:33:08.179356 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-f5b57490-d59f-4761-a16a-4f70960c8fdc req-b9f50aa9-d0b2-4c33-88a1-c54649160fc3 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "8b77de54-4c8b-4ed5-84f5-68bf69f9809f", "name": "network-vif-plugged", "status": "completed", "tag": "5310cc77-6413-4115-9f89-f9fab39070d1"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:33:08.211061 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-f5b57490-d59f-4761-a16a-4f70960c8fdc req-b9f50aa9-d0b2-4c33-88a1-c54649160fc3 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:08.215351 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-f5b57490-d59f-4761-a16a-4f70960c8fdc req-b9f50aa9-d0b2-4c33-88a1-c54649160fc3 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:08.266874 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-f5b57490-d59f-4761-a16a-4f70960c8fdc req-b9f50aa9-d0b2-4c33-88a1-c54649160fc3 service nova] Creating event network-vif-plugged:5310cc77-6413-4115-9f89-f9fab39070d1 for instance 8b77de54-4c8b-4ed5-84f5-68bf69f9809f on nch2-master-1 Jul 16 21:33:08.279761 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-f5b57490-d59f-4761-a16a-4f70960c8fdc req-b9f50aa9-d0b2-4c33-88a1-c54649160fc3 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.122501 Jul 16 21:33:08.282239 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2381/4748] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:33:08 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 127 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:33:08.302979 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-da45ff5d-3815-49de-85a3-844ed49f616c tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:33:08.305898 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-da45ff5d-3815-49de-85a3-844ed49f616c tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] [instance: 8b77de54-4c8b-4ed5-84f5-68bf69f9809f] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:33:08.323872 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-da45ff5d-3815-49de-85a3-844ed49f616c tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:08.326042 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-da45ff5d-3815-49de-85a3-844ed49f616c tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:08.331133 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-da45ff5d-3815-49de-85a3-844ed49f616c tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:08.332007 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-da45ff5d-3815-49de-85a3-844ed49f616c tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:08.390827 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-e5420884-c442-465a-a09c-1aaf2846716d tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Flavor limits 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 16 21:33:08.391364 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-e5420884-c442-465a-a09c-1aaf2846716d tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Image limits 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 16 21:33:08.391988 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-e5420884-c442-465a-a09c-1aaf2846716d tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Flavor pref 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 16 21:33:08.392608 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-e5420884-c442-465a-a09c-1aaf2846716d tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Image pref 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 16 21:33:08.394111 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-e5420884-c442-465a-a09c-1aaf2846716d tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 16 21:33:08.434267 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e5420884-c442-465a-a09c-1aaf2846716d tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:08.434267 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e5420884-c442-465a-a09c-1aaf2846716d tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:08.434267 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e5420884-c442-465a-a09c-1aaf2846716d tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:08.434267 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e5420884-c442-465a-a09c-1aaf2846716d tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:08.499735 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-e5420884-c442-465a-a09c-1aaf2846716d tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Getting quotas for project da2248fa190342f0bfe9211cae1e403b. Resources: {'ram', 'instances', 'cores'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:33:08.529408 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-e5420884-c442-465a-a09c-1aaf2846716d tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Getting quotas for user 22e264440fff43cdb7be6702839af37f and project da2248fa190342f0bfe9211cae1e403b. Resources: {'ram', 'instances', 'cores'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:33:08.601182 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-e5420884-c442-465a-a09c-1aaf2846716d tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Going to run 1 instances... {{(pid=138680) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 16 21:33:08.615177 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-e5420884-c442-465a-a09c-1aaf2846716d tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: e4c7a9cb-e55d-4819-9885-2e72deffddb1] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138680) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 16 21:33:08.793839 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-da45ff5d-3815-49de-85a3-844ed49f616c tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:33:08.802881 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-da45ff5d-3815-49de-85a3-844ed49f616c tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:33:08.810498 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-da45ff5d-3815-49de-85a3-844ed49f616c tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:33:08.811975 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-da45ff5d-3815-49de-85a3-844ed49f616c tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] 10.222.0.16 "GET /compute/v2.1/servers/8b77de54-4c8b-4ed5-84f5-68bf69f9809f" status: 200 len: 1739 microversion: 2.20 time: 0.525987 Jul 16 21:33:08.813547 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2382/4749] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:33:08 2022] GET /compute/v2.1/servers/8b77de54-4c8b-4ed5-84f5-68bf69f9809f => generated 1739 bytes in 530 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:33:08.825273 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-5a71607e-baad-46de-8847-cdb0c305c7e1 req-0b298066-7587-475a-ab82-54751882f07e service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "88cf13fc-0fa3-4e1e-9d76-db4d93b49faf", "name": "network-vif-deleted", "tag": "008d375b-5b42-4275-ac14-31bcaa9250a8"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:33:08.835899 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-e5420884-c442-465a-a09c-1aaf2846716d tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 3.682230 Jul 16 21:33:08.836586 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2368/4750] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 21:33:05 2022] POST /compute/v2.1/servers => generated 380 bytes in 3684 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 16 21:33:08.858667 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-5a71607e-baad-46de-8847-cdb0c305c7e1 req-0b298066-7587-475a-ab82-54751882f07e service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:08.864318 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-5a71607e-baad-46de-8847-cdb0c305c7e1 req-0b298066-7587-475a-ab82-54751882f07e service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:08.874279 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-66733345-71c8-445f-8fe6-46fe38721985 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:33:08.875038 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-66733345-71c8-445f-8fe6-46fe38721985 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: e4c7a9cb-e55d-4819-9885-2e72deffddb1] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:33:08.917377 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-5a71607e-baad-46de-8847-cdb0c305c7e1 req-0b298066-7587-475a-ab82-54751882f07e service nova] Creating event network-vif-deleted:008d375b-5b42-4275-ac14-31bcaa9250a8 for instance 88cf13fc-0fa3-4e1e-9d76-db4d93b49faf on nch1-master-1 Jul 16 21:33:08.931350 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-5a71607e-baad-46de-8847-cdb0c305c7e1 req-0b298066-7587-475a-ab82-54751882f07e service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.116266 Jul 16 21:33:08.935901 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2383/4751] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:33:08 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 122 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:33:08.982184 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-9d7d3be5-99b1-45ec-80d2-1b196b05f3db tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:33:08.983147 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-9d7d3be5-99b1-45ec-80d2-1b196b05f3db tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] [instance: 4eb3d8d6-79ae-4109-a5a7-835c459d1719] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:33:09.001129 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9d7d3be5-99b1-45ec-80d2-1b196b05f3db tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:09.001652 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9d7d3be5-99b1-45ec-80d2-1b196b05f3db tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:09.002526 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9d7d3be5-99b1-45ec-80d2-1b196b05f3db tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:09.002981 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9d7d3be5-99b1-45ec-80d2-1b196b05f3db tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:09.500388 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-66733345-71c8-445f-8fe6-46fe38721985 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '22e264440fff43cdb7be6702839af37f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da2248fa190342f0bfe9211cae1e403b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:33:09.510044 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-66733345-71c8-445f-8fe6-46fe38721985 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1" status: 200 len: 1216 microversion: 2.1 time: 0.657177 Jul 16 21:33:09.515902 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2369/4752] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:33:08 2022] GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1 => generated 1216 bytes in 674 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:33:09.531818 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-43f711cf-cec5-44d0-babf-1110a788bab4 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:33:09.532964 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-43f711cf-cec5-44d0-babf-1110a788bab4 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] [instance: 88cf13fc-0fa3-4e1e-9d76-db4d93b49faf] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:33:09.542790 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-43f711cf-cec5-44d0-babf-1110a788bab4 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:09.548076 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-43f711cf-cec5-44d0-babf-1110a788bab4 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:09.550257 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-43f711cf-cec5-44d0-babf-1110a788bab4 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:09.551054 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-43f711cf-cec5-44d0-babf-1110a788bab4 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:09.588754 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-43f711cf-cec5-44d0-babf-1110a788bab4 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] HTTP exception thrown: Instance 88cf13fc-0fa3-4e1e-9d76-db4d93b49faf could not be found. Jul 16 21:33:09.590651 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-43f711cf-cec5-44d0-babf-1110a788bab4 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Request method failure captured: Jul 16 21:33:09.590651 ncdv-master-1 devstack@n-api.service[138680]: request: GET /compute/v2.1/servers/88cf13fc-0fa3-4e1e-9d76-db4d93b49faf HTTP/1.1 Jul 16 21:33:09.590651 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:33:09.590651 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:33:09.590651 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:33:09.590651 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 0 Jul 16 21:33:09.590651 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:33:09.590651 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:33:09.590651 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:33:09.590651 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:33:09.590651 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0y2AoaTmW9fzKpp4uGm7wex0cqFnrFlzY31Q3uQkOFuadyloOF8tCXPJOxztzHL0OnbEMWGJwuxCROg93CNNRADEFI3axz2E-JaVn1YaPI_h8JL-g89OY5Lk8ug8D0BLGsXQOPqIYrZnwwRF_WHMaANcXwS7qf8jAbwL7w-UUVnlf9c Jul 16 21:33:09.590651 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:33:09.590651 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:33:09.590651 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:33:09.590651 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:33:09.590651 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:33:09.590651 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:33:09.590651 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: bed7c94624484c7cb8abeb3bfc8757b9 Jul 16 21:33:09.590651 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-TestNetworkBasicOps-590553405 Jul 16 21:33:09.590651 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:33:09.590651 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:33:09.590651 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/bed7c94624484c7cb8abeb3bfc8757b9"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/bed7c94624484c7cb8abeb3bfc8757b9"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_bed7c94624484c7cb8abeb3bfc8757b9"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/bed7c94624484c7cb8abeb3bfc8757b9"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:33:09.590651 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-TestNetworkBasicOps-590553405 Jul 16 21:33:09.590651 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: bed7c94624484c7cb8abeb3bfc8757b9 Jul 16 21:33:09.590651 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-TestNetworkBasicOps-590553405 Jul 16 21:33:09.590651 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-TestNetworkBasicOps-590553405-project Jul 16 21:33:09.590651 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:33:09.590651 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:33:09.590651 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: 4ddfae3448704591b662de83b97a1876 Jul 16 21:33:09.590651 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-TestNetworkBasicOps-590553405-project Jul 16 21:33:09.590651 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 21:33:09.590651 ncdv-master-1 devstack@n-api.service[138680]: exception: Instance 88cf13fc-0fa3-4e1e-9d76-db4d93b49faf could not be found. Jul 16 21:33:09.590651 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': '88cf13fc-0fa3-4e1e-9d76-db4d93b49faf'} Jul 16 21:33:09.590651 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:33:09.590651 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:33:09.590651 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:33:09.590651 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:33:09.590651 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:33:09.590651 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:33:09.590651 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:33:09.590651 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:33:09.590651 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:33:09.590651 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:33:09.590651 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 88cf13fc-0fa3-4e1e-9d76-db4d93b49faf could not be found. Jul 16 21:33:09.590651 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:33:09.590651 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:33:09.590651 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:33:09.590651 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:33:09.590651 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:33:09.590651 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:33:09.590651 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:33:09.590651 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:33:09.590651 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:33:09.590651 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:33:09.590651 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:33:09.590651 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:33:09.590651 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:33:09.590651 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:33:09.590651 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:33:09.590651 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:33:09.590651 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 88cf13fc-0fa3-4e1e-9d76-db4d93b49faf could not be found. Jul 16 21:33:09.590651 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:33:09.590651 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:33:09.590651 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:33:09.601128 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:33:09.601128 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:33:09.601128 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:33:09.601128 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:33:09.601128 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:33:09.601128 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 88cf13fc-0fa3-4e1e-9d76-db4d93b49faf could not be found. Jul 16 21:33:09.601128 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:33:09.601128 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-43f711cf-cec5-44d0-babf-1110a788bab4 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Returning 404 to user: Instance 88cf13fc-0fa3-4e1e-9d76-db4d93b49faf could not be found. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:33:09.604258 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-43f711cf-cec5-44d0-babf-1110a788bab4 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] 10.222.0.16 "GET /compute/v2.1/servers/88cf13fc-0fa3-4e1e-9d76-db4d93b49faf" status: 404 len: 111 microversion: 2.1 time: 0.083832 Jul 16 21:33:09.604258 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2370/4753] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:33:09 2022] GET /compute/v2.1/servers/88cf13fc-0fa3-4e1e-9d76-db4d93b49faf => generated 111 bytes in 86 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:33:09.835940 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-2a842afd-249f-42c9-bd5d-2fa04b7ea84a tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:33:09.840727 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-2a842afd-249f-42c9-bd5d-2fa04b7ea84a tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] [instance: 8b77de54-4c8b-4ed5-84f5-68bf69f9809f] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:33:09.850183 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2a842afd-249f-42c9-bd5d-2fa04b7ea84a tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:09.864437 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2a842afd-249f-42c9-bd5d-2fa04b7ea84a tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.014s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:09.867897 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2a842afd-249f-42c9-bd5d-2fa04b7ea84a tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:09.871753 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2a842afd-249f-42c9-bd5d-2fa04b7ea84a tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:10.374410 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-9d7d3be5-99b1-45ec-80d2-1b196b05f3db tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a23fafbab18d41ee82b84bbdf94cf0fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfc2e42a0df346f4adef152ba2e3fa71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:33:10.393640 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-9d7d3be5-99b1-45ec-80d2-1b196b05f3db tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'a23fafbab18d41ee82b84bbdf94cf0fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfc2e42a0df346f4adef152ba2e3fa71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:33:10.403677 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-9d7d3be5-99b1-45ec-80d2-1b196b05f3db tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'a23fafbab18d41ee82b84bbdf94cf0fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfc2e42a0df346f4adef152ba2e3fa71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:33:10.405419 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-9d7d3be5-99b1-45ec-80d2-1b196b05f3db tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] 10.222.0.16 "GET /compute/v2.1/servers/4eb3d8d6-79ae-4109-a5a7-835c459d1719" status: 200 len: 2927 microversion: 2.32 time: 1.441103 Jul 16 21:33:10.409807 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2384/4754] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:33:08 2022] GET /compute/v2.1/servers/4eb3d8d6-79ae-4109-a5a7-835c459d1719 => generated 2927 bytes in 1446 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:33:10.442585 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-bbbed2c7-e6a5-4778-8a9a-b39f89282622 req-14f46710-3118-4de7-8e07-d7bc1218c6ea service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "4eb3d8d6-79ae-4109-a5a7-835c459d1719", "name": "network-vif-unplugged", "status": "completed", "tag": "0ae97a58-1fb7-46af-ad9b-e1ebb2af5923"}, {"server_uuid": "4eb3d8d6-79ae-4109-a5a7-835c459d1719", "name": "network-vif-unplugged", "status": "completed", "tag": "fb878298-9eb7-4cd5-96d9-305373704495"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:33:10.464194 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-bbbed2c7-e6a5-4778-8a9a-b39f89282622 req-14f46710-3118-4de7-8e07-d7bc1218c6ea service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:10.464810 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-bbbed2c7-e6a5-4778-8a9a-b39f89282622 req-14f46710-3118-4de7-8e07-d7bc1218c6ea service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:10.490383 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-bbbed2c7-e6a5-4778-8a9a-b39f89282622 req-14f46710-3118-4de7-8e07-d7bc1218c6ea service nova] Creating event network-vif-unplugged:0ae97a58-1fb7-46af-ad9b-e1ebb2af5923 for instance 4eb3d8d6-79ae-4109-a5a7-835c459d1719 on nch1-master-1 Jul 16 21:33:10.494532 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-bbbed2c7-e6a5-4778-8a9a-b39f89282622 req-14f46710-3118-4de7-8e07-d7bc1218c6ea service nova] Creating event network-vif-unplugged:fb878298-9eb7-4cd5-96d9-305373704495 for instance 4eb3d8d6-79ae-4109-a5a7-835c459d1719 on nch1-master-1 Jul 16 21:33:10.512805 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-bbbed2c7-e6a5-4778-8a9a-b39f89282622 req-14f46710-3118-4de7-8e07-d7bc1218c6ea service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 358 microversion: 2.1 time: 0.076167 Jul 16 21:33:10.514009 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2385/4755] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:33:10 2022] POST /compute/v2.1/os-server-external-events => generated 358 bytes in 78 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 16 21:33:10.528113 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-b411d63a-681b-4df4-afbc-4b0bebf1d615 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:33:10.528836 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-b411d63a-681b-4df4-afbc-4b0bebf1d615 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: e4c7a9cb-e55d-4819-9885-2e72deffddb1] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:33:10.601741 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-2a842afd-249f-42c9-bd5d-2fa04b7ea84a tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:33:10.612689 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-2a842afd-249f-42c9-bd5d-2fa04b7ea84a tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:33:10.621314 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-2a842afd-249f-42c9-bd5d-2fa04b7ea84a tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:33:10.621314 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-2a842afd-249f-42c9-bd5d-2fa04b7ea84a tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] 10.222.0.16 "GET /compute/v2.1/servers/8b77de54-4c8b-4ed5-84f5-68bf69f9809f" status: 200 len: 1739 microversion: 2.20 time: 0.790173 Jul 16 21:33:10.621314 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2371/4756] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:33:09 2022] GET /compute/v2.1/servers/8b77de54-4c8b-4ed5-84f5-68bf69f9809f => generated 1739 bytes in 791 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:33:10.796581 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-b411d63a-681b-4df4-afbc-4b0bebf1d615 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '22e264440fff43cdb7be6702839af37f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da2248fa190342f0bfe9211cae1e403b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:33:10.805792 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-b411d63a-681b-4df4-afbc-4b0bebf1d615 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1" status: 200 len: 1216 microversion: 2.1 time: 0.281743 Jul 16 21:33:10.806395 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2386/4757] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:33:10 2022] GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1 => generated 1216 bytes in 283 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:33:10.980925 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-5374f09e-ef35-4075-994d-290938a21d75 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] Calling method '.version_select at 0x7fdd8fd9aee0>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:33:11.142436 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-5374f09e-ef35-4075-994d-290938a21d75 tempest-TestNetworkBasicOps-590553405 tempest-TestNetworkBasicOps-590553405-project] 10.222.0.16 "DELETE /compute/v2.1/os-keypairs/tempest-TestNetworkBasicOps-520706742" status: 202 len: 0 microversion: 2.1 time: 0.165522 Jul 16 21:33:11.144074 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2372/4758] 10.222.0.16 () {58 vars in 1253 bytes} [Sat Jul 16 21:33:10 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestNetworkBasicOps-520706742 => generated 0 bytes in 168 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 21:33:11.434114 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-8662132e-53e7-4ba6-8e46-456e93559909 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:33:11.435454 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-8662132e-53e7-4ba6-8e46-456e93559909 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] [instance: 4eb3d8d6-79ae-4109-a5a7-835c459d1719] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:33:11.453358 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8662132e-53e7-4ba6-8e46-456e93559909 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:11.453985 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8662132e-53e7-4ba6-8e46-456e93559909 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:11.454914 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8662132e-53e7-4ba6-8e46-456e93559909 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:11.456558 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8662132e-53e7-4ba6-8e46-456e93559909 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:11.638326 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-5c37414f-2364-4ec7-be0b-baf4a783b1f1 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:33:11.639351 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-5c37414f-2364-4ec7-be0b-baf4a783b1f1 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] [instance: 8b77de54-4c8b-4ed5-84f5-68bf69f9809f] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:33:11.700787 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5c37414f-2364-4ec7-be0b-baf4a783b1f1 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:11.702564 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5c37414f-2364-4ec7-be0b-baf4a783b1f1 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:11.703534 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5c37414f-2364-4ec7-be0b-baf4a783b1f1 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:11.716846 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5c37414f-2364-4ec7-be0b-baf4a783b1f1 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:12.127548 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-8662132e-53e7-4ba6-8e46-456e93559909 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a23fafbab18d41ee82b84bbdf94cf0fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfc2e42a0df346f4adef152ba2e3fa71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:33:12.140779 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-8662132e-53e7-4ba6-8e46-456e93559909 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'a23fafbab18d41ee82b84bbdf94cf0fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfc2e42a0df346f4adef152ba2e3fa71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:33:12.146781 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-8662132e-53e7-4ba6-8e46-456e93559909 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'a23fafbab18d41ee82b84bbdf94cf0fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfc2e42a0df346f4adef152ba2e3fa71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:33:12.152985 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-8662132e-53e7-4ba6-8e46-456e93559909 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] 10.222.0.16 "GET /compute/v2.1/servers/4eb3d8d6-79ae-4109-a5a7-835c459d1719" status: 200 len: 2906 microversion: 2.32 time: 0.722759 Jul 16 21:33:12.156717 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2387/4759] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:33:11 2022] GET /compute/v2.1/servers/4eb3d8d6-79ae-4109-a5a7-835c459d1719 => generated 2906 bytes in 727 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:33:12.169119 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-7039fba3-491e-487a-9d2b-fc977bd87a0a tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:33:12.169857 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-7039fba3-491e-487a-9d2b-fc977bd87a0a tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: e4c7a9cb-e55d-4819-9885-2e72deffddb1] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:33:12.203186 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7039fba3-491e-487a-9d2b-fc977bd87a0a tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:12.204385 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7039fba3-491e-487a-9d2b-fc977bd87a0a tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:12.207060 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7039fba3-491e-487a-9d2b-fc977bd87a0a tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:12.210095 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7039fba3-491e-487a-9d2b-fc977bd87a0a tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:12.216359 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-5c37414f-2364-4ec7-be0b-baf4a783b1f1 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:33:12.235518 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-5c37414f-2364-4ec7-be0b-baf4a783b1f1 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:33:12.249104 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-5c37414f-2364-4ec7-be0b-baf4a783b1f1 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:33:12.250369 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-5c37414f-2364-4ec7-be0b-baf4a783b1f1 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] 10.222.0.16 "GET /compute/v2.1/servers/8b77de54-4c8b-4ed5-84f5-68bf69f9809f" status: 200 len: 1739 microversion: 2.20 time: 0.616660 Jul 16 21:33:12.255254 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2373/4760] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:33:11 2022] GET /compute/v2.1/servers/8b77de54-4c8b-4ed5-84f5-68bf69f9809f => generated 1739 bytes in 622 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:33:12.313754 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-7039fba3-491e-487a-9d2b-fc977bd87a0a tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '22e264440fff43cdb7be6702839af37f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da2248fa190342f0bfe9211cae1e403b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:33:12.325506 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-7039fba3-491e-487a-9d2b-fc977bd87a0a tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1" status: 200 len: 1268 microversion: 2.1 time: 0.165537 Jul 16 21:33:12.328140 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2388/4761] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:33:12 2022] GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1 => generated 1268 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:33:12.593104 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-74c94f5d-4ea0-4d1b-a42f-244b9174cc4f req-7d0c41b5-b177-41ea-bc09-200199e4c073 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "4eb3d8d6-79ae-4109-a5a7-835c459d1719", "name": "network-vif-unplugged", "status": "completed", "tag": "b73090b3-6ab1-4f2c-b09d-274c9f89f289"}, {"server_uuid": "4eb3d8d6-79ae-4109-a5a7-835c459d1719", "name": "network-vif-unplugged", "status": "completed", "tag": "f55f6499-e005-4119-ad39-6c8db2fb1c53"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:33:12.649617 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-74c94f5d-4ea0-4d1b-a42f-244b9174cc4f req-7d0c41b5-b177-41ea-bc09-200199e4c073 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:12.651596 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-74c94f5d-4ea0-4d1b-a42f-244b9174cc4f req-7d0c41b5-b177-41ea-bc09-200199e4c073 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:12.685141 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-74c94f5d-4ea0-4d1b-a42f-244b9174cc4f req-7d0c41b5-b177-41ea-bc09-200199e4c073 service nova] Creating event network-vif-unplugged:b73090b3-6ab1-4f2c-b09d-274c9f89f289 for instance 4eb3d8d6-79ae-4109-a5a7-835c459d1719 on nch1-master-1 Jul 16 21:33:12.688969 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-74c94f5d-4ea0-4d1b-a42f-244b9174cc4f req-7d0c41b5-b177-41ea-bc09-200199e4c073 service nova] Creating event network-vif-unplugged:f55f6499-e005-4119-ad39-6c8db2fb1c53 for instance 4eb3d8d6-79ae-4109-a5a7-835c459d1719 on nch1-master-1 Jul 16 21:33:12.707539 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-74c94f5d-4ea0-4d1b-a42f-244b9174cc4f req-7d0c41b5-b177-41ea-bc09-200199e4c073 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 358 microversion: 2.1 time: 0.127033 Jul 16 21:33:12.713688 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2374/4762] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:33:12 2022] POST /compute/v2.1/os-server-external-events => generated 358 bytes in 134 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:33:13.001770 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-837a741a-5e42-40be-99bf-b4482076ed13 req-9e044669-d6b6-428b-97ec-036f180b15b1 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "4eb3d8d6-79ae-4109-a5a7-835c459d1719", "name": "network-vif-deleted", "tag": "0ae97a58-1fb7-46af-ad9b-e1ebb2af5923"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:33:13.017164 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-837a741a-5e42-40be-99bf-b4482076ed13 req-9e044669-d6b6-428b-97ec-036f180b15b1 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:13.021317 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-837a741a-5e42-40be-99bf-b4482076ed13 req-9e044669-d6b6-428b-97ec-036f180b15b1 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:13.049940 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-837a741a-5e42-40be-99bf-b4482076ed13 req-9e044669-d6b6-428b-97ec-036f180b15b1 service nova] Creating event network-vif-deleted:0ae97a58-1fb7-46af-ad9b-e1ebb2af5923 for instance 4eb3d8d6-79ae-4109-a5a7-835c459d1719 on nch1-master-1 Jul 16 21:33:13.066306 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-837a741a-5e42-40be-99bf-b4482076ed13 req-9e044669-d6b6-428b-97ec-036f180b15b1 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.067466 Jul 16 21:33:13.067188 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2389/4763] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:33:12 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 72 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:33:13.186012 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-ee6a22e6-1b82-461d-894c-d67547d33b6c tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:33:13.187517 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-ee6a22e6-1b82-461d-894c-d67547d33b6c tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] [instance: 4eb3d8d6-79ae-4109-a5a7-835c459d1719] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:33:13.220153 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ee6a22e6-1b82-461d-894c-d67547d33b6c tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:13.220719 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ee6a22e6-1b82-461d-894c-d67547d33b6c tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:13.222227 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ee6a22e6-1b82-461d-894c-d67547d33b6c tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:13.222904 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ee6a22e6-1b82-461d-894c-d67547d33b6c tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:13.273799 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-aa9755b0-0c23-44cb-8841-2bbfa6fdfaae tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:33:13.274669 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-aa9755b0-0c23-44cb-8841-2bbfa6fdfaae tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] [instance: 8b77de54-4c8b-4ed5-84f5-68bf69f9809f] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:33:13.276941 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_api:SELECT=73,nova_api:UPDATE=1 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:33:13.300654 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-aa9755b0-0c23-44cb-8841-2bbfa6fdfaae tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:13.305703 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-aa9755b0-0c23-44cb-8841-2bbfa6fdfaae tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:13.310915 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-aa9755b0-0c23-44cb-8841-2bbfa6fdfaae tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:13.314118 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-aa9755b0-0c23-44cb-8841-2bbfa6fdfaae tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:13.758313 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-ee6a22e6-1b82-461d-894c-d67547d33b6c tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a23fafbab18d41ee82b84bbdf94cf0fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfc2e42a0df346f4adef152ba2e3fa71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:33:13.773278 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-ee6a22e6-1b82-461d-894c-d67547d33b6c tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'a23fafbab18d41ee82b84bbdf94cf0fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfc2e42a0df346f4adef152ba2e3fa71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:33:13.774876 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-ee6a22e6-1b82-461d-894c-d67547d33b6c tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'a23fafbab18d41ee82b84bbdf94cf0fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfc2e42a0df346f4adef152ba2e3fa71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:33:13.776181 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-ee6a22e6-1b82-461d-894c-d67547d33b6c tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] 10.222.0.16 "GET /compute/v2.1/servers/4eb3d8d6-79ae-4109-a5a7-835c459d1719" status: 200 len: 2860 microversion: 2.32 time: 0.595500 Jul 16 21:33:13.781098 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2375/4764] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:33:13 2022] GET /compute/v2.1/servers/4eb3d8d6-79ae-4109-a5a7-835c459d1719 => generated 2860 bytes in 601 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:33:13.809229 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-016a3911-3ae9-4ddb-8e5b-143be0dea0ce tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:33:13.810493 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-016a3911-3ae9-4ddb-8e5b-143be0dea0ce tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: e4c7a9cb-e55d-4819-9885-2e72deffddb1] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:33:13.827372 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-016a3911-3ae9-4ddb-8e5b-143be0dea0ce tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:13.829812 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-016a3911-3ae9-4ddb-8e5b-143be0dea0ce tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:13.833307 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-016a3911-3ae9-4ddb-8e5b-143be0dea0ce tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:13.834216 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-016a3911-3ae9-4ddb-8e5b-143be0dea0ce tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:13.981610 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-016a3911-3ae9-4ddb-8e5b-143be0dea0ce tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '22e264440fff43cdb7be6702839af37f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da2248fa190342f0bfe9211cae1e403b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:33:13.996657 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-016a3911-3ae9-4ddb-8e5b-143be0dea0ce tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1" status: 200 len: 1274 microversion: 2.1 time: 0.201519 Jul 16 21:33:13.997966 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2376/4765] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:33:13 2022] GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1 => generated 1274 bytes in 204 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:33:14.050743 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-aa9755b0-0c23-44cb-8841-2bbfa6fdfaae tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:33:14.066800 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-aa9755b0-0c23-44cb-8841-2bbfa6fdfaae tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:33:14.074543 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-aa9755b0-0c23-44cb-8841-2bbfa6fdfaae tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:33:14.082586 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-aa9755b0-0c23-44cb-8841-2bbfa6fdfaae tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] 10.222.0.16 "GET /compute/v2.1/servers/8b77de54-4c8b-4ed5-84f5-68bf69f9809f" status: 200 len: 1739 microversion: 2.20 time: 0.806675 Jul 16 21:33:14.095096 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2390/4766] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:33:13 2022] GET /compute/v2.1/servers/8b77de54-4c8b-4ed5-84f5-68bf69f9809f => generated 1739 bytes in 827 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:33:14.806563 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-ca6a963d-8ac0-4b20-9ea4-54bba077dd68 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:33:14.807911 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-ca6a963d-8ac0-4b20-9ea4-54bba077dd68 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] [instance: 4eb3d8d6-79ae-4109-a5a7-835c459d1719] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:33:14.825909 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ca6a963d-8ac0-4b20-9ea4-54bba077dd68 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:14.826699 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ca6a963d-8ac0-4b20-9ea4-54bba077dd68 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:14.827699 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ca6a963d-8ac0-4b20-9ea4-54bba077dd68 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:14.828305 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ca6a963d-8ac0-4b20-9ea4-54bba077dd68 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:14.914936 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-5a277a4d-f688-4d3b-9763-a852ea729621 req-49aa309e-19f9-4600-8cc8-757323f75405 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "4eb3d8d6-79ae-4109-a5a7-835c459d1719", "name": "network-vif-unplugged", "status": "completed", "tag": "3581de64-9f10-4ee1-9fbc-52628f1ddcfa"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:33:14.929203 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-5a277a4d-f688-4d3b-9763-a852ea729621 req-49aa309e-19f9-4600-8cc8-757323f75405 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:14.930750 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-5a277a4d-f688-4d3b-9763-a852ea729621 req-49aa309e-19f9-4600-8cc8-757323f75405 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:14.976927 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-5a277a4d-f688-4d3b-9763-a852ea729621 req-49aa309e-19f9-4600-8cc8-757323f75405 service nova] Creating event network-vif-unplugged:3581de64-9f10-4ee1-9fbc-52628f1ddcfa for instance 4eb3d8d6-79ae-4109-a5a7-835c459d1719 on nch1-master-1 Jul 16 21:33:14.985933 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-5a277a4d-f688-4d3b-9763-a852ea729621 req-49aa309e-19f9-4600-8cc8-757323f75405 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.078857 Jul 16 21:33:14.990180 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2391/4767] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:33:14 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:33:15.017626 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-f0e9d7d6-27e6-479c-bd2e-c66c07161071 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:33:15.018531 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-f0e9d7d6-27e6-479c-bd2e-c66c07161071 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: e4c7a9cb-e55d-4819-9885-2e72deffddb1] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:33:15.035719 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f0e9d7d6-27e6-479c-bd2e-c66c07161071 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:15.036893 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f0e9d7d6-27e6-479c-bd2e-c66c07161071 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:15.039503 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f0e9d7d6-27e6-479c-bd2e-c66c07161071 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:15.040033 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f0e9d7d6-27e6-479c-bd2e-c66c07161071 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:15.051744 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=100 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:33:15.255360 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-ca6a963d-8ac0-4b20-9ea4-54bba077dd68 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a23fafbab18d41ee82b84bbdf94cf0fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfc2e42a0df346f4adef152ba2e3fa71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:33:15.276264 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-ca6a963d-8ac0-4b20-9ea4-54bba077dd68 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'a23fafbab18d41ee82b84bbdf94cf0fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfc2e42a0df346f4adef152ba2e3fa71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:33:15.282766 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-ca6a963d-8ac0-4b20-9ea4-54bba077dd68 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'a23fafbab18d41ee82b84bbdf94cf0fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfc2e42a0df346f4adef152ba2e3fa71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:33:15.288220 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-ca6a963d-8ac0-4b20-9ea4-54bba077dd68 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] 10.222.0.16 "GET /compute/v2.1/servers/4eb3d8d6-79ae-4109-a5a7-835c459d1719" status: 200 len: 2746 microversion: 2.32 time: 0.484295 Jul 16 21:33:15.306231 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2377/4768] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:33:14 2022] GET /compute/v2.1/servers/4eb3d8d6-79ae-4109-a5a7-835c459d1719 => generated 2746 bytes in 505 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 16 21:33:15.622301 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-2796fd6e-1133-4a3f-a5db-cbe384e207df tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:33:15.623050 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-2796fd6e-1133-4a3f-a5db-cbe384e207df tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] [instance: 8b77de54-4c8b-4ed5-84f5-68bf69f9809f] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:33:15.649948 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2796fd6e-1133-4a3f-a5db-cbe384e207df tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:15.651888 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2796fd6e-1133-4a3f-a5db-cbe384e207df tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:15.654693 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2796fd6e-1133-4a3f-a5db-cbe384e207df tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:15.660156 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2796fd6e-1133-4a3f-a5db-cbe384e207df tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:16.356100 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-f0e9d7d6-27e6-479c-bd2e-c66c07161071 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '22e264440fff43cdb7be6702839af37f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da2248fa190342f0bfe9211cae1e403b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:33:16.398062 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-f0e9d7d6-27e6-479c-bd2e-c66c07161071 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1" status: 200 len: 1316 microversion: 2.1 time: 1.375408 Jul 16 21:33:16.399363 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2392/4769] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:33:15 2022] GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1 => generated 1316 bytes in 1388 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:33:16.420590 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-93abe988-94f8-43d2-89f4-5b46f2856bf3 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:33:16.422001 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-93abe988-94f8-43d2-89f4-5b46f2856bf3 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] [instance: 4eb3d8d6-79ae-4109-a5a7-835c459d1719] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:33:16.441413 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-93abe988-94f8-43d2-89f4-5b46f2856bf3 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:16.442327 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-93abe988-94f8-43d2-89f4-5b46f2856bf3 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:16.444294 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-93abe988-94f8-43d2-89f4-5b46f2856bf3 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:16.452270 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-93abe988-94f8-43d2-89f4-5b46f2856bf3 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:16.555048 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-2796fd6e-1133-4a3f-a5db-cbe384e207df tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:33:16.580512 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=96,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:33:16.593489 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-2796fd6e-1133-4a3f-a5db-cbe384e207df tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:33:16.603930 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-2796fd6e-1133-4a3f-a5db-cbe384e207df tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:33:16.607404 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-2796fd6e-1133-4a3f-a5db-cbe384e207df tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] 10.222.0.16 "GET /compute/v2.1/servers/8b77de54-4c8b-4ed5-84f5-68bf69f9809f" status: 200 len: 1739 microversion: 2.20 time: 1.299326 Jul 16 21:33:16.609399 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2378/4770] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:33:15 2022] GET /compute/v2.1/servers/8b77de54-4c8b-4ed5-84f5-68bf69f9809f => generated 1739 bytes in 1302 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:33:16.935430 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-1e661866-19a6-460d-aa43-b47364c09dc5 req-be21b992-521d-4686-b6fe-af767b6a0e37 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "4eb3d8d6-79ae-4109-a5a7-835c459d1719", "name": "network-vif-deleted", "tag": "fb878298-9eb7-4cd5-96d9-305373704495"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:33:16.954266 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-1e661866-19a6-460d-aa43-b47364c09dc5 req-be21b992-521d-4686-b6fe-af767b6a0e37 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:16.956908 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-1e661866-19a6-460d-aa43-b47364c09dc5 req-be21b992-521d-4686-b6fe-af767b6a0e37 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:17.015341 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-1e661866-19a6-460d-aa43-b47364c09dc5 req-be21b992-521d-4686-b6fe-af767b6a0e37 service nova] Creating event network-vif-deleted:fb878298-9eb7-4cd5-96d9-305373704495 for instance 4eb3d8d6-79ae-4109-a5a7-835c459d1719 on nch1-master-1 Jul 16 21:33:17.032821 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-1e661866-19a6-460d-aa43-b47364c09dc5 req-be21b992-521d-4686-b6fe-af767b6a0e37 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.100806 Jul 16 21:33:17.035320 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2379/4771] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:33:16 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 16 21:33:17.097696 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-93abe988-94f8-43d2-89f4-5b46f2856bf3 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a23fafbab18d41ee82b84bbdf94cf0fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfc2e42a0df346f4adef152ba2e3fa71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:33:17.118524 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-93abe988-94f8-43d2-89f4-5b46f2856bf3 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'a23fafbab18d41ee82b84bbdf94cf0fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfc2e42a0df346f4adef152ba2e3fa71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:33:17.123764 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-93abe988-94f8-43d2-89f4-5b46f2856bf3 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'a23fafbab18d41ee82b84bbdf94cf0fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfc2e42a0df346f4adef152ba2e3fa71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:33:17.128341 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-93abe988-94f8-43d2-89f4-5b46f2856bf3 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] 10.222.0.16 "GET /compute/v2.1/servers/4eb3d8d6-79ae-4109-a5a7-835c459d1719" status: 200 len: 2700 microversion: 2.32 time: 0.726591 Jul 16 21:33:17.147240 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2393/4772] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:33:16 2022] GET /compute/v2.1/servers/4eb3d8d6-79ae-4109-a5a7-835c459d1719 => generated 2700 bytes in 747 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:33:17.417813 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-ae40fd09-fc40-4157-b5b6-d306bdfacea5 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:33:17.418690 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-ae40fd09-fc40-4157-b5b6-d306bdfacea5 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: e4c7a9cb-e55d-4819-9885-2e72deffddb1] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:33:17.431106 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ae40fd09-fc40-4157-b5b6-d306bdfacea5 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:17.435597 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ae40fd09-fc40-4157-b5b6-d306bdfacea5 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:17.440274 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ae40fd09-fc40-4157-b5b6-d306bdfacea5 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:17.440274 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ae40fd09-fc40-4157-b5b6-d306bdfacea5 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:17.629217 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-1ecabf40-a22d-4e0b-a98a-49652cbe7599 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:33:17.630521 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-1ecabf40-a22d-4e0b-a98a-49652cbe7599 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] [instance: 8b77de54-4c8b-4ed5-84f5-68bf69f9809f] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:33:17.646796 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1ecabf40-a22d-4e0b-a98a-49652cbe7599 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:17.647593 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1ecabf40-a22d-4e0b-a98a-49652cbe7599 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:17.649807 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1ecabf40-a22d-4e0b-a98a-49652cbe7599 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:17.651894 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1ecabf40-a22d-4e0b-a98a-49652cbe7599 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:18.226546 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-ae40fd09-fc40-4157-b5b6-d306bdfacea5 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '22e264440fff43cdb7be6702839af37f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da2248fa190342f0bfe9211cae1e403b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:33:18.238708 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-ae40fd09-fc40-4157-b5b6-d306bdfacea5 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1" status: 200 len: 1316 microversion: 2.1 time: 0.825421 Jul 16 21:33:18.251694 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2380/4773] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:33:17 2022] GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1 => generated 1316 bytes in 839 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:33:18.266758 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-e7d4ee78-b8be-4c95-be87-12436ea67d30 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:33:18.273110 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-e7d4ee78-b8be-4c95-be87-12436ea67d30 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] [instance: 4eb3d8d6-79ae-4109-a5a7-835c459d1719] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:33:18.288560 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e7d4ee78-b8be-4c95-be87-12436ea67d30 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:18.290429 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e7d4ee78-b8be-4c95-be87-12436ea67d30 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:18.293327 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e7d4ee78-b8be-4c95-be87-12436ea67d30 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:18.294761 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e7d4ee78-b8be-4c95-be87-12436ea67d30 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:18.474315 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell0:SELECT=2 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:33:18.904268 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-1ecabf40-a22d-4e0b-a98a-49652cbe7599 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:33:18.923359 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-1ecabf40-a22d-4e0b-a98a-49652cbe7599 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:33:18.930409 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-1ecabf40-a22d-4e0b-a98a-49652cbe7599 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:33:18.937560 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-1ecabf40-a22d-4e0b-a98a-49652cbe7599 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] 10.222.0.16 "GET /compute/v2.1/servers/8b77de54-4c8b-4ed5-84f5-68bf69f9809f" status: 200 len: 1739 microversion: 2.20 time: 1.306458 Jul 16 21:33:18.950686 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2394/4774] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:33:17 2022] GET /compute/v2.1/servers/8b77de54-4c8b-4ed5-84f5-68bf69f9809f => generated 1739 bytes in 1327 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 16 21:33:19.258955 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-46ae4980-ef97-452c-89d8-da98226a696d tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:33:19.259626 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-46ae4980-ef97-452c-89d8-da98226a696d tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: e4c7a9cb-e55d-4819-9885-2e72deffddb1] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:33:19.273712 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-46ae4980-ef97-452c-89d8-da98226a696d tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:19.274261 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-46ae4980-ef97-452c-89d8-da98226a696d tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:19.275086 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-46ae4980-ef97-452c-89d8-da98226a696d tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:19.275541 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-46ae4980-ef97-452c-89d8-da98226a696d tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:19.389127 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-e7d4ee78-b8be-4c95-be87-12436ea67d30 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a23fafbab18d41ee82b84bbdf94cf0fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfc2e42a0df346f4adef152ba2e3fa71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:33:19.404384 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-e7d4ee78-b8be-4c95-be87-12436ea67d30 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'a23fafbab18d41ee82b84bbdf94cf0fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfc2e42a0df346f4adef152ba2e3fa71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:33:19.406211 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-e7d4ee78-b8be-4c95-be87-12436ea67d30 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'a23fafbab18d41ee82b84bbdf94cf0fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfc2e42a0df346f4adef152ba2e3fa71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:33:19.407994 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-e7d4ee78-b8be-4c95-be87-12436ea67d30 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] 10.222.0.16 "GET /compute/v2.1/servers/4eb3d8d6-79ae-4109-a5a7-835c459d1719" status: 200 len: 2414 microversion: 2.32 time: 1.154801 Jul 16 21:33:19.409361 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2381/4775] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:33:18 2022] GET /compute/v2.1/servers/4eb3d8d6-79ae-4109-a5a7-835c459d1719 => generated 2414 bytes in 1157 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:33:19.789025 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-46ae4980-ef97-452c-89d8-da98226a696d tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '22e264440fff43cdb7be6702839af37f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da2248fa190342f0bfe9211cae1e403b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:33:19.798683 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-46ae4980-ef97-452c-89d8-da98226a696d tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1" status: 200 len: 1316 microversion: 2.1 time: 0.544183 Jul 16 21:33:19.811252 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2395/4776] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:33:19 2022] GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1 => generated 1316 bytes in 558 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:33:19.957764 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-7621950e-4b79-4a4e-bc5f-6a0b5715d2ec tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:33:19.958306 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-7621950e-4b79-4a4e-bc5f-6a0b5715d2ec tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] [instance: 8b77de54-4c8b-4ed5-84f5-68bf69f9809f] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:33:19.969175 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell0:SELECT=8 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:33:19.976476 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7621950e-4b79-4a4e-bc5f-6a0b5715d2ec tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:19.980638 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7621950e-4b79-4a4e-bc5f-6a0b5715d2ec tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:19.980638 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7621950e-4b79-4a4e-bc5f-6a0b5715d2ec tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:19.980638 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7621950e-4b79-4a4e-bc5f-6a0b5715d2ec tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:20.271763 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-6d076ac6-abab-4f3d-8407-9f7fdb97b40f req-d2c2453c-a124-4fae-8ca0-737e26d76f31 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "e4c7a9cb-e55d-4819-9885-2e72deffddb1", "tag": "0b459b05-0d03-4317-8a9b-bae73b0eaddd"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:33:20.301557 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-6d076ac6-abab-4f3d-8407-9f7fdb97b40f req-d2c2453c-a124-4fae-8ca0-737e26d76f31 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:20.302375 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-6d076ac6-abab-4f3d-8407-9f7fdb97b40f req-d2c2453c-a124-4fae-8ca0-737e26d76f31 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:20.365501 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-6d076ac6-abab-4f3d-8407-9f7fdb97b40f req-d2c2453c-a124-4fae-8ca0-737e26d76f31 service nova] Creating event network-changed:0b459b05-0d03-4317-8a9b-bae73b0eaddd for instance e4c7a9cb-e55d-4819-9885-2e72deffddb1 on nch2-master-1 Jul 16 21:33:20.376714 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-6d076ac6-abab-4f3d-8407-9f7fdb97b40f req-d2c2453c-a124-4fae-8ca0-737e26d76f31 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.129308 Jul 16 21:33:20.396451 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2396/4777] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:33:20 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:33:20.436165 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-b056d5a1-776a-4107-b536-2f091a6fc8fd tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:33:20.436165 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-b056d5a1-776a-4107-b536-2f091a6fc8fd tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] [instance: 4eb3d8d6-79ae-4109-a5a7-835c459d1719] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:33:20.443363 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b056d5a1-776a-4107-b536-2f091a6fc8fd tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:20.444045 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b056d5a1-776a-4107-b536-2f091a6fc8fd tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:20.445015 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b056d5a1-776a-4107-b536-2f091a6fc8fd tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:20.448965 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b056d5a1-776a-4107-b536-2f091a6fc8fd tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:20.756415 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-7621950e-4b79-4a4e-bc5f-6a0b5715d2ec tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:33:20.774464 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-7621950e-4b79-4a4e-bc5f-6a0b5715d2ec tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:33:20.777192 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-7621950e-4b79-4a4e-bc5f-6a0b5715d2ec tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:33:20.778450 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-7621950e-4b79-4a4e-bc5f-6a0b5715d2ec tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] 10.222.0.16 "GET /compute/v2.1/servers/8b77de54-4c8b-4ed5-84f5-68bf69f9809f" status: 200 len: 1739 microversion: 2.20 time: 0.825229 Jul 16 21:33:20.781453 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2382/4778] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:33:19 2022] GET /compute/v2.1/servers/8b77de54-4c8b-4ed5-84f5-68bf69f9809f => generated 1739 bytes in 829 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:33:20.805966 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell0:SELECT=2 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:33:20.832320 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-23041a3f-1a6c-4e5e-8286-254e82db1bb2 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:33:20.839345 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-23041a3f-1a6c-4e5e-8286-254e82db1bb2 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: e4c7a9cb-e55d-4819-9885-2e72deffddb1] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:33:20.874015 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-23041a3f-1a6c-4e5e-8286-254e82db1bb2 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:20.878864 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-b056d5a1-776a-4107-b536-2f091a6fc8fd tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a23fafbab18d41ee82b84bbdf94cf0fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfc2e42a0df346f4adef152ba2e3fa71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:33:20.884198 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-23041a3f-1a6c-4e5e-8286-254e82db1bb2 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.021s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:20.887374 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-23041a3f-1a6c-4e5e-8286-254e82db1bb2 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:20.894411 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-b056d5a1-776a-4107-b536-2f091a6fc8fd tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'a23fafbab18d41ee82b84bbdf94cf0fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfc2e42a0df346f4adef152ba2e3fa71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:33:20.894681 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-23041a3f-1a6c-4e5e-8286-254e82db1bb2 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:20.915493 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-b056d5a1-776a-4107-b536-2f091a6fc8fd tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'a23fafbab18d41ee82b84bbdf94cf0fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfc2e42a0df346f4adef152ba2e3fa71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:33:20.917280 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-b056d5a1-776a-4107-b536-2f091a6fc8fd tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] 10.222.0.16 "GET /compute/v2.1/servers/4eb3d8d6-79ae-4109-a5a7-835c459d1719" status: 200 len: 2368 microversion: 2.32 time: 0.492476 Jul 16 21:33:20.922459 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2397/4779] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:33:20 2022] GET /compute/v2.1/servers/4eb3d8d6-79ae-4109-a5a7-835c459d1719 => generated 2368 bytes in 498 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:33:21.267576 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-23041a3f-1a6c-4e5e-8286-254e82db1bb2 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '22e264440fff43cdb7be6702839af37f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da2248fa190342f0bfe9211cae1e403b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:33:21.282768 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-23041a3f-1a6c-4e5e-8286-254e82db1bb2 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1" status: 200 len: 1316 microversion: 2.1 time: 0.468454 Jul 16 21:33:21.284931 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2383/4780] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:33:20 2022] GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1 => generated 1316 bytes in 471 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:33:21.532929 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-674ebd9b-f40c-40ae-9994-193b2d72126b req-9bff62b5-074a-4ce8-a4f7-ba4db2c13d8c service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "4eb3d8d6-79ae-4109-a5a7-835c459d1719", "name": "network-vif-deleted", "tag": "f55f6499-e005-4119-ad39-6c8db2fb1c53"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:33:21.564190 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-674ebd9b-f40c-40ae-9994-193b2d72126b req-9bff62b5-074a-4ce8-a4f7-ba4db2c13d8c service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:21.566940 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-674ebd9b-f40c-40ae-9994-193b2d72126b req-9bff62b5-074a-4ce8-a4f7-ba4db2c13d8c service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:21.591876 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-674ebd9b-f40c-40ae-9994-193b2d72126b req-9bff62b5-074a-4ce8-a4f7-ba4db2c13d8c service nova] Creating event network-vif-deleted:f55f6499-e005-4119-ad39-6c8db2fb1c53 for instance 4eb3d8d6-79ae-4109-a5a7-835c459d1719 on nch1-master-1 Jul 16 21:33:21.599387 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-674ebd9b-f40c-40ae-9994-193b2d72126b req-9bff62b5-074a-4ce8-a4f7-ba4db2c13d8c service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.074042 Jul 16 21:33:21.600801 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2398/4781] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:33:21 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 77 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:33:21.807777 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-54d60a93-6a25-44c8-b578-23bba49b9be5 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:33:21.812452 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-54d60a93-6a25-44c8-b578-23bba49b9be5 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] [instance: 8b77de54-4c8b-4ed5-84f5-68bf69f9809f] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:33:21.833913 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-54d60a93-6a25-44c8-b578-23bba49b9be5 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:21.834373 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-54d60a93-6a25-44c8-b578-23bba49b9be5 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:21.835557 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-54d60a93-6a25-44c8-b578-23bba49b9be5 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:21.836028 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-54d60a93-6a25-44c8-b578-23bba49b9be5 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:21.944677 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-eb8b93cd-d0f5-4fc3-a8ec-c2cd9420025b tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:33:21.945319 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-eb8b93cd-d0f5-4fc3-a8ec-c2cd9420025b tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] [instance: 4eb3d8d6-79ae-4109-a5a7-835c459d1719] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:33:21.957830 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-eb8b93cd-d0f5-4fc3-a8ec-c2cd9420025b tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:21.958281 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-eb8b93cd-d0f5-4fc3-a8ec-c2cd9420025b tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:21.959131 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-eb8b93cd-d0f5-4fc3-a8ec-c2cd9420025b tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:21.959573 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-eb8b93cd-d0f5-4fc3-a8ec-c2cd9420025b tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:22.519197 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-eb8b93cd-d0f5-4fc3-a8ec-c2cd9420025b tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a23fafbab18d41ee82b84bbdf94cf0fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfc2e42a0df346f4adef152ba2e3fa71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:33:22.529742 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-eb8b93cd-d0f5-4fc3-a8ec-c2cd9420025b tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'a23fafbab18d41ee82b84bbdf94cf0fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfc2e42a0df346f4adef152ba2e3fa71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:33:22.537798 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-eb8b93cd-d0f5-4fc3-a8ec-c2cd9420025b tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'a23fafbab18d41ee82b84bbdf94cf0fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfc2e42a0df346f4adef152ba2e3fa71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:33:22.542733 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-eb8b93cd-d0f5-4fc3-a8ec-c2cd9420025b tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] 10.222.0.16 "GET /compute/v2.1/servers/4eb3d8d6-79ae-4109-a5a7-835c459d1719" status: 200 len: 2255 microversion: 2.32 time: 0.604461 Jul 16 21:33:22.545079 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2399/4782] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:33:21 2022] GET /compute/v2.1/servers/4eb3d8d6-79ae-4109-a5a7-835c459d1719 => generated 2255 bytes in 608 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:33:22.558260 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-bc5fe3c9-1049-4944-9db0-1c71ee55a54b tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:33:22.561327 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-bc5fe3c9-1049-4944-9db0-1c71ee55a54b tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: e4c7a9cb-e55d-4819-9885-2e72deffddb1] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:33:22.576345 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-bc5fe3c9-1049-4944-9db0-1c71ee55a54b tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:22.576943 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-bc5fe3c9-1049-4944-9db0-1c71ee55a54b tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:22.577846 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-bc5fe3c9-1049-4944-9db0-1c71ee55a54b tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:22.578363 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-bc5fe3c9-1049-4944-9db0-1c71ee55a54b tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:22.601378 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-54d60a93-6a25-44c8-b578-23bba49b9be5 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:33:22.612326 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-54d60a93-6a25-44c8-b578-23bba49b9be5 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:33:22.613091 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-54d60a93-6a25-44c8-b578-23bba49b9be5 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:33:22.617128 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-54d60a93-6a25-44c8-b578-23bba49b9be5 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] 10.222.0.16 "GET /compute/v2.1/servers/8b77de54-4c8b-4ed5-84f5-68bf69f9809f" status: 200 len: 1739 microversion: 2.20 time: 0.814906 Jul 16 21:33:22.628807 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2384/4783] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:33:21 2022] GET /compute/v2.1/servers/8b77de54-4c8b-4ed5-84f5-68bf69f9809f => generated 1739 bytes in 830 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 16 21:33:22.929111 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-bc5fe3c9-1049-4944-9db0-1c71ee55a54b tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '22e264440fff43cdb7be6702839af37f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da2248fa190342f0bfe9211cae1e403b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:33:22.937137 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-bc5fe3c9-1049-4944-9db0-1c71ee55a54b tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1" status: 200 len: 1316 microversion: 2.1 time: 0.384462 Jul 16 21:33:22.938766 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2400/4784] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:33:22 2022] GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1 => generated 1316 bytes in 387 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:33:23.601112 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-4f7ed88e-7cf1-417a-8601-ac2ee27b76fd tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:33:23.601112 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-4f7ed88e-7cf1-417a-8601-ac2ee27b76fd tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] [instance: 4eb3d8d6-79ae-4109-a5a7-835c459d1719] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:33:23.601112 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4f7ed88e-7cf1-417a-8601-ac2ee27b76fd tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:23.601112 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4f7ed88e-7cf1-417a-8601-ac2ee27b76fd tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:23.601112 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4f7ed88e-7cf1-417a-8601-ac2ee27b76fd tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:23.601112 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4f7ed88e-7cf1-417a-8601-ac2ee27b76fd tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:23.636987 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-dcc2bd65-eee3-474a-b64d-dd0fae859670 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:33:23.638615 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-dcc2bd65-eee3-474a-b64d-dd0fae859670 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] [instance: 8b77de54-4c8b-4ed5-84f5-68bf69f9809f] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:33:23.648485 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-dcc2bd65-eee3-474a-b64d-dd0fae859670 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:23.649431 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-dcc2bd65-eee3-474a-b64d-dd0fae859670 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:23.650446 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-dcc2bd65-eee3-474a-b64d-dd0fae859670 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:23.651462 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-dcc2bd65-eee3-474a-b64d-dd0fae859670 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:23.936854 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-4f7ed88e-7cf1-417a-8601-ac2ee27b76fd tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a23fafbab18d41ee82b84bbdf94cf0fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfc2e42a0df346f4adef152ba2e3fa71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:33:23.964742 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-4f7ed88e-7cf1-417a-8601-ac2ee27b76fd tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'a23fafbab18d41ee82b84bbdf94cf0fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfc2e42a0df346f4adef152ba2e3fa71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:33:23.966734 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-4f7ed88e-7cf1-417a-8601-ac2ee27b76fd tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'a23fafbab18d41ee82b84bbdf94cf0fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfc2e42a0df346f4adef152ba2e3fa71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:33:23.968044 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-4f7ed88e-7cf1-417a-8601-ac2ee27b76fd tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] 10.222.0.16 "GET /compute/v2.1/servers/4eb3d8d6-79ae-4109-a5a7-835c459d1719" status: 200 len: 2209 microversion: 2.32 time: 0.407518 Jul 16 21:33:23.969863 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2385/4785] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:33:23 2022] GET /compute/v2.1/servers/4eb3d8d6-79ae-4109-a5a7-835c459d1719 => generated 2209 bytes in 410 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:33:23.985742 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-5c318b94-d044-4825-b206-7b2011392bd0 req-74d787ef-29df-4ae4-9f8b-422f25ef045b service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "4eb3d8d6-79ae-4109-a5a7-835c459d1719", "name": "network-vif-deleted", "tag": "b73090b3-6ab1-4f2c-b09d-274c9f89f289"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:33:23.996394 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-5c318b94-d044-4825-b206-7b2011392bd0 req-74d787ef-29df-4ae4-9f8b-422f25ef045b service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:23.997150 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-5c318b94-d044-4825-b206-7b2011392bd0 req-74d787ef-29df-4ae4-9f8b-422f25ef045b service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:24.005788 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-dcc2bd65-eee3-474a-b64d-dd0fae859670 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:33:24.017705 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-5c318b94-d044-4825-b206-7b2011392bd0 req-74d787ef-29df-4ae4-9f8b-422f25ef045b service nova] Creating event network-vif-deleted:b73090b3-6ab1-4f2c-b09d-274c9f89f289 for instance 4eb3d8d6-79ae-4109-a5a7-835c459d1719 on nch1-master-1 Jul 16 21:33:24.021579 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-dcc2bd65-eee3-474a-b64d-dd0fae859670 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:33:24.027591 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-dcc2bd65-eee3-474a-b64d-dd0fae859670 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:33:24.029154 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-dcc2bd65-eee3-474a-b64d-dd0fae859670 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] 10.222.0.16 "GET /compute/v2.1/servers/8b77de54-4c8b-4ed5-84f5-68bf69f9809f" status: 200 len: 1739 microversion: 2.20 time: 0.397063 Jul 16 21:33:24.029354 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-5c318b94-d044-4825-b206-7b2011392bd0 req-74d787ef-29df-4ae4-9f8b-422f25ef045b service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.057945 Jul 16 21:33:24.030179 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2386/4786] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:33:23 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 60 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:33:24.032824 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2401/4787] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:33:23 2022] GET /compute/v2.1/servers/8b77de54-4c8b-4ed5-84f5-68bf69f9809f => generated 1739 bytes in 401 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 16 21:33:24.041850 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-b845479a-76f6-431d-bb47-e3513c0f0597 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:33:24.044145 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-b845479a-76f6-431d-bb47-e3513c0f0597 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: e4c7a9cb-e55d-4819-9885-2e72deffddb1] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:33:24.052140 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b845479a-76f6-431d-bb47-e3513c0f0597 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:24.052608 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b845479a-76f6-431d-bb47-e3513c0f0597 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:24.053405 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b845479a-76f6-431d-bb47-e3513c0f0597 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:24.059573 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b845479a-76f6-431d-bb47-e3513c0f0597 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:24.357907 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-b845479a-76f6-431d-bb47-e3513c0f0597 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '22e264440fff43cdb7be6702839af37f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da2248fa190342f0bfe9211cae1e403b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:33:24.368547 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-b845479a-76f6-431d-bb47-e3513c0f0597 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1" status: 200 len: 1316 microversion: 2.1 time: 0.334879 Jul 16 21:33:24.378955 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2387/4788] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:33:24 2022] GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1 => generated 1316 bytes in 346 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:33:24.993827 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-237e8d1e-dcb3-4eda-8fd4-5f014bf8d285 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:33:24.993827 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-237e8d1e-dcb3-4eda-8fd4-5f014bf8d285 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] [instance: 4eb3d8d6-79ae-4109-a5a7-835c459d1719] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:33:25.004885 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-237e8d1e-dcb3-4eda-8fd4-5f014bf8d285 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:25.006004 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-237e8d1e-dcb3-4eda-8fd4-5f014bf8d285 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:25.006756 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-237e8d1e-dcb3-4eda-8fd4-5f014bf8d285 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:25.007295 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-237e8d1e-dcb3-4eda-8fd4-5f014bf8d285 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:25.060044 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-8f767734-a2d2-4efe-82b4-4d68bee08b4f tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:33:25.064055 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-8f767734-a2d2-4efe-82b4-4d68bee08b4f tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] [instance: 8b77de54-4c8b-4ed5-84f5-68bf69f9809f] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:33:25.086726 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8f767734-a2d2-4efe-82b4-4d68bee08b4f tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:25.087219 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8f767734-a2d2-4efe-82b4-4d68bee08b4f tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:25.088013 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8f767734-a2d2-4efe-82b4-4d68bee08b4f tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:25.088459 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8f767734-a2d2-4efe-82b4-4d68bee08b4f tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:25.573912 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-237e8d1e-dcb3-4eda-8fd4-5f014bf8d285 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a23fafbab18d41ee82b84bbdf94cf0fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfc2e42a0df346f4adef152ba2e3fa71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:33:25.583618 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-237e8d1e-dcb3-4eda-8fd4-5f014bf8d285 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'a23fafbab18d41ee82b84bbdf94cf0fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfc2e42a0df346f4adef152ba2e3fa71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:33:25.592647 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-237e8d1e-dcb3-4eda-8fd4-5f014bf8d285 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'a23fafbab18d41ee82b84bbdf94cf0fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfc2e42a0df346f4adef152ba2e3fa71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:33:25.594199 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-237e8d1e-dcb3-4eda-8fd4-5f014bf8d285 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] 10.222.0.16 "GET /compute/v2.1/servers/4eb3d8d6-79ae-4109-a5a7-835c459d1719" status: 200 len: 2052 microversion: 2.32 time: 0.608396 Jul 16 21:33:25.596812 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2402/4789] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:33:24 2022] GET /compute/v2.1/servers/4eb3d8d6-79ae-4109-a5a7-835c459d1719 => generated 2052 bytes in 612 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:33:25.614631 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-425b8f4c-772c-4583-b4f2-85528765e419 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:33:25.617929 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-425b8f4c-772c-4583-b4f2-85528765e419 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: e4c7a9cb-e55d-4819-9885-2e72deffddb1] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:33:25.631690 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-425b8f4c-772c-4583-b4f2-85528765e419 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:25.632342 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-425b8f4c-772c-4583-b4f2-85528765e419 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:25.634718 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-8f767734-a2d2-4efe-82b4-4d68bee08b4f tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:33:25.636003 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-425b8f4c-772c-4583-b4f2-85528765e419 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:25.636688 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-425b8f4c-772c-4583-b4f2-85528765e419 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:25.655667 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-8f767734-a2d2-4efe-82b4-4d68bee08b4f tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:33:25.667896 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-8f767734-a2d2-4efe-82b4-4d68bee08b4f tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:33:25.673507 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-8f767734-a2d2-4efe-82b4-4d68bee08b4f tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] 10.222.0.16 "GET /compute/v2.1/servers/8b77de54-4c8b-4ed5-84f5-68bf69f9809f" status: 200 len: 1739 microversion: 2.20 time: 0.624547 Jul 16 21:33:25.676438 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2388/4790] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:33:25 2022] GET /compute/v2.1/servers/8b77de54-4c8b-4ed5-84f5-68bf69f9809f => generated 1739 bytes in 630 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 16 21:33:26.028987 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-425b8f4c-772c-4583-b4f2-85528765e419 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '22e264440fff43cdb7be6702839af37f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da2248fa190342f0bfe9211cae1e403b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:33:26.038186 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-425b8f4c-772c-4583-b4f2-85528765e419 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1" status: 200 len: 1316 microversion: 2.1 time: 0.429938 Jul 16 21:33:26.052744 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2403/4791] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:33:25 2022] GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1 => generated 1316 bytes in 445 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:33:26.193925 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-80f570eb-4610-40df-826e-be5b49467031 req-147f646a-5ba6-434f-bfa1-85b143ae87c4 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "4eb3d8d6-79ae-4109-a5a7-835c459d1719", "name": "network-vif-deleted", "tag": "3581de64-9f10-4ee1-9fbc-52628f1ddcfa"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:33:26.205097 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-80f570eb-4610-40df-826e-be5b49467031 req-147f646a-5ba6-434f-bfa1-85b143ae87c4 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:26.206284 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-80f570eb-4610-40df-826e-be5b49467031 req-147f646a-5ba6-434f-bfa1-85b143ae87c4 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:26.224834 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-80f570eb-4610-40df-826e-be5b49467031 req-147f646a-5ba6-434f-bfa1-85b143ae87c4 service nova] Creating event network-vif-deleted:3581de64-9f10-4ee1-9fbc-52628f1ddcfa for instance 4eb3d8d6-79ae-4109-a5a7-835c459d1719 on nch1-master-1 Jul 16 21:33:26.231177 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-80f570eb-4610-40df-826e-be5b49467031 req-147f646a-5ba6-434f-bfa1-85b143ae87c4 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.042655 Jul 16 21:33:26.232494 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2389/4792] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:33:26 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:33:26.618064 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-13c79757-ff72-4b0f-9e5e-922461660ad5 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:33:26.619000 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-13c79757-ff72-4b0f-9e5e-922461660ad5 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] [instance: 4eb3d8d6-79ae-4109-a5a7-835c459d1719] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:33:26.631731 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-13c79757-ff72-4b0f-9e5e-922461660ad5 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:26.636954 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-13c79757-ff72-4b0f-9e5e-922461660ad5 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:26.645878 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-13c79757-ff72-4b0f-9e5e-922461660ad5 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:26.645878 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-13c79757-ff72-4b0f-9e5e-922461660ad5 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:26.695288 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-1ecd2b39-5e87-48a4-9d94-798eaa7aa5d7 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:33:26.696805 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-1ecd2b39-5e87-48a4-9d94-798eaa7aa5d7 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] [instance: 8b77de54-4c8b-4ed5-84f5-68bf69f9809f] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:33:26.736686 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1ecd2b39-5e87-48a4-9d94-798eaa7aa5d7 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:26.739564 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1ecd2b39-5e87-48a4-9d94-798eaa7aa5d7 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:26.740461 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1ecd2b39-5e87-48a4-9d94-798eaa7aa5d7 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:26.741118 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1ecd2b39-5e87-48a4-9d94-798eaa7aa5d7 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:26.872006 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-13c79757-ff72-4b0f-9e5e-922461660ad5 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a23fafbab18d41ee82b84bbdf94cf0fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfc2e42a0df346f4adef152ba2e3fa71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:33:26.888218 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-13c79757-ff72-4b0f-9e5e-922461660ad5 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'a23fafbab18d41ee82b84bbdf94cf0fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfc2e42a0df346f4adef152ba2e3fa71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:33:26.889957 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-13c79757-ff72-4b0f-9e5e-922461660ad5 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'a23fafbab18d41ee82b84bbdf94cf0fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfc2e42a0df346f4adef152ba2e3fa71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:33:26.895399 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-13c79757-ff72-4b0f-9e5e-922461660ad5 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] 10.222.0.16 "GET /compute/v2.1/servers/4eb3d8d6-79ae-4109-a5a7-835c459d1719" status: 200 len: 1604 microversion: 2.32 time: 0.279065 Jul 16 21:33:26.903180 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2404/4793] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:33:26 2022] GET /compute/v2.1/servers/4eb3d8d6-79ae-4109-a5a7-835c459d1719 => generated 1604 bytes in 291 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:33:27.066931 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-12899188-f32c-4064-b4a7-54699e90f699 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:33:27.068329 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-12899188-f32c-4064-b4a7-54699e90f699 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: e4c7a9cb-e55d-4819-9885-2e72deffddb1] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:33:27.088068 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-12899188-f32c-4064-b4a7-54699e90f699 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:27.088882 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-12899188-f32c-4064-b4a7-54699e90f699 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:27.090436 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-12899188-f32c-4064-b4a7-54699e90f699 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:27.092869 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-12899188-f32c-4064-b4a7-54699e90f699 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:27.141143 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-1ecd2b39-5e87-48a4-9d94-798eaa7aa5d7 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:33:27.154635 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-1ecd2b39-5e87-48a4-9d94-798eaa7aa5d7 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:33:27.156195 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-1ecd2b39-5e87-48a4-9d94-798eaa7aa5d7 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:33:27.157407 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-1ecd2b39-5e87-48a4-9d94-798eaa7aa5d7 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] 10.222.0.16 "GET /compute/v2.1/servers/8b77de54-4c8b-4ed5-84f5-68bf69f9809f" status: 200 len: 1739 microversion: 2.20 time: 0.467851 Jul 16 21:33:27.157979 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2390/4794] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:33:26 2022] GET /compute/v2.1/servers/8b77de54-4c8b-4ed5-84f5-68bf69f9809f => generated 1739 bytes in 469 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:33:27.430936 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-12899188-f32c-4064-b4a7-54699e90f699 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '22e264440fff43cdb7be6702839af37f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da2248fa190342f0bfe9211cae1e403b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:33:27.441915 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-12899188-f32c-4064-b4a7-54699e90f699 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1" status: 200 len: 1316 microversion: 2.1 time: 0.381195 Jul 16 21:33:27.444719 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2405/4795] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:33:27 2022] GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1 => generated 1316 bytes in 384 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:33:27.630714 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-79052a35-43ff-407b-a3ed-4556f8ff7dad tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:33:27.633129 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-79052a35-43ff-407b-a3ed-4556f8ff7dad tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] [instance: 3c0f0855-7a9c-4d8a-95ac-3bf41d32d4e6] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:33:27.653997 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-79052a35-43ff-407b-a3ed-4556f8ff7dad tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:27.655102 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-79052a35-43ff-407b-a3ed-4556f8ff7dad tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:27.659859 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-79052a35-43ff-407b-a3ed-4556f8ff7dad tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:27.662038 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-79052a35-43ff-407b-a3ed-4556f8ff7dad tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:27.719778 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-79052a35-43ff-407b-a3ed-4556f8ff7dad tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] 10.222.0.16 "GET /compute/v2.1/servers/3c0f0855-7a9c-4d8a-95ac-3bf41d32d4e6/os-server-password" status: 200 len: 116 microversion: 2.1 time: 0.098407 Jul 16 21:33:27.722300 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2391/4796] 10.222.0.16 () {58 vars in 1292 bytes} [Sat Jul 16 21:33:27 2022] GET /compute/v2.1/servers/3c0f0855-7a9c-4d8a-95ac-3bf41d32d4e6/os-server-password => generated 116 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:33:27.914776 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-31d2baa1-9905-4da4-ac6b-89d9f9983adb tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:33:27.915428 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-31d2baa1-9905-4da4-ac6b-89d9f9983adb tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] [instance: 4eb3d8d6-79ae-4109-a5a7-835c459d1719] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:33:27.926097 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-31d2baa1-9905-4da4-ac6b-89d9f9983adb tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:27.934729 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-31d2baa1-9905-4da4-ac6b-89d9f9983adb tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.009s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:27.941066 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-31d2baa1-9905-4da4-ac6b-89d9f9983adb tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:27.941852 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-31d2baa1-9905-4da4-ac6b-89d9f9983adb tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:27.983567 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-17d5e503-5a27-4206-8515-78486332bb90 req-7c128645-ada1-4489-838d-b77dd1f834ae service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "e4c7a9cb-e55d-4819-9885-2e72deffddb1", "name": "network-vif-plugged", "status": "completed", "tag": "0b459b05-0d03-4317-8a9b-bae73b0eaddd"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:33:28.006523 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-17d5e503-5a27-4206-8515-78486332bb90 req-7c128645-ada1-4489-838d-b77dd1f834ae service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:28.007202 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-17d5e503-5a27-4206-8515-78486332bb90 req-7c128645-ada1-4489-838d-b77dd1f834ae service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:28.024577 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-17d5e503-5a27-4206-8515-78486332bb90 req-7c128645-ada1-4489-838d-b77dd1f834ae service nova] Creating event network-vif-plugged:0b459b05-0d03-4317-8a9b-bae73b0eaddd for instance e4c7a9cb-e55d-4819-9885-2e72deffddb1 on nch2-master-1 Jul 16 21:33:28.034448 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-17d5e503-5a27-4206-8515-78486332bb90 req-7c128645-ada1-4489-838d-b77dd1f834ae service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.051713 Jul 16 21:33:28.034448 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2392/4797] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:33:27 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 16 21:33:28.060843 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-31d2baa1-9905-4da4-ac6b-89d9f9983adb tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:28.062254 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-31d2baa1-9905-4da4-ac6b-89d9f9983adb tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:28.064456 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.objects.instance [None req-31d2baa1-9905-4da4-ac6b-89d9f9983adb tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lazy-loading 'fault' on Instance uuid 4eb3d8d6-79ae-4109-a5a7-835c459d1719 {{(pid=138681) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:33:28.178425 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-8a6c9487-aff9-41a5-b564-ff76848ac433 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:33:28.179180 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-8a6c9487-aff9-41a5-b564-ff76848ac433 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] [instance: 8b77de54-4c8b-4ed5-84f5-68bf69f9809f] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:33:28.194218 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8a6c9487-aff9-41a5-b564-ff76848ac433 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:28.198821 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8a6c9487-aff9-41a5-b564-ff76848ac433 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:28.199806 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8a6c9487-aff9-41a5-b564-ff76848ac433 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:28.200260 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8a6c9487-aff9-41a5-b564-ff76848ac433 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:28.371048 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-31d2baa1-9905-4da4-ac6b-89d9f9983adb tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a23fafbab18d41ee82b84bbdf94cf0fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfc2e42a0df346f4adef152ba2e3fa71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:33:28.394578 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-31d2baa1-9905-4da4-ac6b-89d9f9983adb tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'a23fafbab18d41ee82b84bbdf94cf0fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfc2e42a0df346f4adef152ba2e3fa71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:33:28.400688 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-31d2baa1-9905-4da4-ac6b-89d9f9983adb tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'a23fafbab18d41ee82b84bbdf94cf0fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cfc2e42a0df346f4adef152ba2e3fa71', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:33:28.417450 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-31d2baa1-9905-4da4-ac6b-89d9f9983adb tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] 10.222.0.16 "GET /compute/v2.1/servers/4eb3d8d6-79ae-4109-a5a7-835c459d1719" status: 200 len: 1609 microversion: 2.32 time: 0.505264 Jul 16 21:33:28.424886 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2406/4798] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:33:27 2022] GET /compute/v2.1/servers/4eb3d8d6-79ae-4109-a5a7-835c459d1719 => generated 1609 bytes in 511 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:33:28.472693 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-cb0f03cc-0aca-4c88-88dc-fe1edef1cffd tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:33:28.473501 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-cb0f03cc-0aca-4c88-88dc-fe1edef1cffd tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: e4c7a9cb-e55d-4819-9885-2e72deffddb1] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:33:28.513832 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cb0f03cc-0aca-4c88-88dc-fe1edef1cffd tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:28.522396 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cb0f03cc-0aca-4c88-88dc-fe1edef1cffd tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:28.528194 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cb0f03cc-0aca-4c88-88dc-fe1edef1cffd tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:28.528803 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cb0f03cc-0aca-4c88-88dc-fe1edef1cffd tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:28.545996 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-8a6c9487-aff9-41a5-b564-ff76848ac433 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:33:28.574816 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-8a6c9487-aff9-41a5-b564-ff76848ac433 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:33:28.577649 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-8a6c9487-aff9-41a5-b564-ff76848ac433 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:33:28.585752 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-8a6c9487-aff9-41a5-b564-ff76848ac433 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] 10.222.0.16 "GET /compute/v2.1/servers/8b77de54-4c8b-4ed5-84f5-68bf69f9809f" status: 200 len: 1739 microversion: 2.20 time: 0.410476 Jul 16 21:33:28.586933 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2393/4799] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:33:28 2022] GET /compute/v2.1/servers/8b77de54-4c8b-4ed5-84f5-68bf69f9809f => generated 1739 bytes in 415 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:33:28.785521 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-cb0f03cc-0aca-4c88-88dc-fe1edef1cffd tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '22e264440fff43cdb7be6702839af37f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da2248fa190342f0bfe9211cae1e403b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:33:28.797104 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-cb0f03cc-0aca-4c88-88dc-fe1edef1cffd tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1" status: 200 len: 1316 microversion: 2.1 time: 0.329779 Jul 16 21:33:28.798051 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2407/4800] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:33:28 2022] GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1 => generated 1316 bytes in 332 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:33:29.446403 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-80e8832f-b6dc-4e5f-b03f-0702b4ea1a3a tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:33:29.450015 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-80e8832f-b6dc-4e5f-b03f-0702b4ea1a3a tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] [instance: 4eb3d8d6-79ae-4109-a5a7-835c459d1719] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:33:29.461150 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-80e8832f-b6dc-4e5f-b03f-0702b4ea1a3a tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:29.466788 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-80e8832f-b6dc-4e5f-b03f-0702b4ea1a3a tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:29.466788 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-80e8832f-b6dc-4e5f-b03f-0702b4ea1a3a tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:29.466788 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-80e8832f-b6dc-4e5f-b03f-0702b4ea1a3a tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:29.480631 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-80e8832f-b6dc-4e5f-b03f-0702b4ea1a3a tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] HTTP exception thrown: Instance 4eb3d8d6-79ae-4109-a5a7-835c459d1719 could not be found. Jul 16 21:33:29.482684 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-80e8832f-b6dc-4e5f-b03f-0702b4ea1a3a tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Request method failure captured: Jul 16 21:33:29.482684 ncdv-master-1 devstack@n-api.service[138680]: request: GET /compute/v2.1/servers/4eb3d8d6-79ae-4109-a5a7-835c459d1719 HTTP/1.1 Jul 16 21:33:29.482684 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:33:29.482684 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:33:29.482684 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:33:29.482684 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 0 Jul 16 21:33:29.482684 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:33:29.482684 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:33:29.482684 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:33:29.482684 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:33:29.482684 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0yxwGA0BjWaH4IFLXPRW6P0VOiNvUSeCxC3iPOH23aOhIAp0mlxPNlLQwYl7fhJ6I0RtLrdBFgHdA2BF2c3Y9mg2djFpaCdZN9JZ1uT3oWT2rVKHWQZX_rjqb_WiG7iquPb62h0SegXtyTFOr-BlTyLqfGBmbMEiet2UrEAsKGhG9hk Jul 16 21:33:29.482684 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:33:29.482684 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:33:29.482684 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:33:29.482684 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:33:29.482684 ncdv-master-1 devstack@n-api.service[138680]: X-Openstack-Nova-Api-Version: 2.32 Jul 16 21:33:29.482684 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:33:29.482684 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:33:29.482684 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: cfc2e42a0df346f4adef152ba2e3fa71 Jul 16 21:33:29.482684 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-TaggedBootDevicesTest-1806311997 Jul 16 21:33:29.482684 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:33:29.482684 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:33:29.482684 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/cfc2e42a0df346f4adef152ba2e3fa71"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/cfc2e42a0df346f4adef152ba2e3fa71"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_cfc2e42a0df346f4adef152ba2e3fa71"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/cfc2e42a0df346f4adef152ba2e3fa71"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:33:29.482684 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-TaggedBootDevicesTest-1806311997 Jul 16 21:33:29.482684 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: cfc2e42a0df346f4adef152ba2e3fa71 Jul 16 21:33:29.482684 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-TaggedBootDevicesTest-1806311997 Jul 16 21:33:29.482684 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-TaggedBootDevicesTest-1806311997-project Jul 16 21:33:29.482684 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:33:29.482684 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:33:29.482684 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: a23fafbab18d41ee82b84bbdf94cf0fe Jul 16 21:33:29.482684 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-TaggedBootDevicesTest-1806311997-project Jul 16 21:33:29.482684 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 21:33:29.482684 ncdv-master-1 devstack@n-api.service[138680]: exception: Instance 4eb3d8d6-79ae-4109-a5a7-835c459d1719 could not be found. Jul 16 21:33:29.482684 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': '4eb3d8d6-79ae-4109-a5a7-835c459d1719'} Jul 16 21:33:29.482684 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:33:29.482684 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:33:29.482684 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:33:29.482684 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:33:29.482684 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:33:29.482684 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:33:29.482684 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:33:29.482684 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:33:29.482684 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:33:29.482684 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:33:29.482684 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 4eb3d8d6-79ae-4109-a5a7-835c459d1719 could not be found. Jul 16 21:33:29.482684 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:33:29.482684 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:33:29.482684 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:33:29.482684 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:33:29.482684 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:33:29.482684 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:33:29.482684 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:33:29.482684 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:33:29.482684 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:33:29.482684 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:33:29.482684 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:33:29.482684 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:33:29.482684 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:33:29.482684 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:33:29.482684 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:33:29.482684 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:33:29.482684 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 4eb3d8d6-79ae-4109-a5a7-835c459d1719 could not be found. Jul 16 21:33:29.482684 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:33:29.482684 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:33:29.482684 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:33:29.491882 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:33:29.491882 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:33:29.491882 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:33:29.491882 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:33:29.491882 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:33:29.491882 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 4eb3d8d6-79ae-4109-a5a7-835c459d1719 could not be found. Jul 16 21:33:29.491882 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:33:29.494999 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-80e8832f-b6dc-4e5f-b03f-0702b4ea1a3a tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Returning 404 to user: Instance 4eb3d8d6-79ae-4109-a5a7-835c459d1719 could not be found. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:33:29.496193 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-80e8832f-b6dc-4e5f-b03f-0702b4ea1a3a tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] 10.222.0.16 "GET /compute/v2.1/servers/4eb3d8d6-79ae-4109-a5a7-835c459d1719" status: 404 len: 111 microversion: 2.32 time: 0.058265 Jul 16 21:33:29.497498 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2394/4801] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:33:29 2022] GET /compute/v2.1/servers/4eb3d8d6-79ae-4109-a5a7-835c459d1719 => generated 111 bytes in 60 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Jul 16 21:33:29.570041 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-ee68cd50-6488-487d-af1c-9c0e677f61f4 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] Calling method '.version_select at 0x7fdd9cf52c10>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:33:29.606167 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-94cedbf5-0d1e-460e-a9da-2e90766fcee2 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:33:29.607237 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-94cedbf5-0d1e-460e-a9da-2e90766fcee2 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] [instance: 8b77de54-4c8b-4ed5-84f5-68bf69f9809f] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:33:29.623382 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-94cedbf5-0d1e-460e-a9da-2e90766fcee2 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:29.624014 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-94cedbf5-0d1e-460e-a9da-2e90766fcee2 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:29.624926 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-94cedbf5-0d1e-460e-a9da-2e90766fcee2 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:29.625488 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-94cedbf5-0d1e-460e-a9da-2e90766fcee2 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:29.766837 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-ee68cd50-6488-487d-af1c-9c0e677f61f4 tempest-TaggedBootDevicesTest-1806311997 tempest-TaggedBootDevicesTest-1806311997-project] 10.222.0.16 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1847534343" status: 204 len: 0 microversion: 2.32 time: 0.201919 Jul 16 21:33:29.768904 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2408/4802] 10.222.0.16 () {60 vars in 1261 bytes} [Sat Jul 16 21:33:29 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1847534343 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Jul 16 21:33:29.816393 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-dedde80f-87aa-414e-9a50-e43eb8fb0cf8 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:33:29.817870 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-dedde80f-87aa-414e-9a50-e43eb8fb0cf8 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: e4c7a9cb-e55d-4819-9885-2e72deffddb1] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:33:29.846202 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-dedde80f-87aa-414e-9a50-e43eb8fb0cf8 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:29.846943 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-dedde80f-87aa-414e-9a50-e43eb8fb0cf8 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:29.847923 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-dedde80f-87aa-414e-9a50-e43eb8fb0cf8 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:29.995742 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-dedde80f-87aa-414e-9a50-e43eb8fb0cf8 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:30.064677 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-94cedbf5-0d1e-460e-a9da-2e90766fcee2 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:33:30.075867 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-94cedbf5-0d1e-460e-a9da-2e90766fcee2 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:33:30.077823 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-94cedbf5-0d1e-460e-a9da-2e90766fcee2 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:33:30.079198 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-94cedbf5-0d1e-460e-a9da-2e90766fcee2 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] 10.222.0.16 "GET /compute/v2.1/servers/8b77de54-4c8b-4ed5-84f5-68bf69f9809f" status: 200 len: 1739 microversion: 2.20 time: 0.477338 Jul 16 21:33:30.085707 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2395/4803] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:33:29 2022] GET /compute/v2.1/servers/8b77de54-4c8b-4ed5-84f5-68bf69f9809f => generated 1739 bytes in 485 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:33:30.165898 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-dedde80f-87aa-414e-9a50-e43eb8fb0cf8 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '22e264440fff43cdb7be6702839af37f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da2248fa190342f0bfe9211cae1e403b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:33:30.192486 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-dedde80f-87aa-414e-9a50-e43eb8fb0cf8 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1" status: 200 len: 1316 microversion: 2.1 time: 0.377954 Jul 16 21:33:30.193132 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2409/4804] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:33:29 2022] GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1 => generated 1316 bytes in 382 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:33:31.105724 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-7d761337-6d49-4aa1-820f-d841bb9961d8 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:33:31.107044 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-7d761337-6d49-4aa1-820f-d841bb9961d8 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] [instance: 8b77de54-4c8b-4ed5-84f5-68bf69f9809f] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:33:31.122444 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7d761337-6d49-4aa1-820f-d841bb9961d8 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:31.126403 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7d761337-6d49-4aa1-820f-d841bb9961d8 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:31.127153 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7d761337-6d49-4aa1-820f-d841bb9961d8 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:31.130805 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7d761337-6d49-4aa1-820f-d841bb9961d8 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:31.210828 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-6eb6ce18-f7fe-46dd-8746-f5dd3292985b tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:33:31.212247 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-6eb6ce18-f7fe-46dd-8746-f5dd3292985b tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: e4c7a9cb-e55d-4819-9885-2e72deffddb1] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:33:31.234721 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6eb6ce18-f7fe-46dd-8746-f5dd3292985b tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:31.244912 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6eb6ce18-f7fe-46dd-8746-f5dd3292985b tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.010s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:31.245789 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6eb6ce18-f7fe-46dd-8746-f5dd3292985b tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:31.246253 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6eb6ce18-f7fe-46dd-8746-f5dd3292985b tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:31.657085 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-7d761337-6d49-4aa1-820f-d841bb9961d8 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:33:31.675193 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-7d761337-6d49-4aa1-820f-d841bb9961d8 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:33:31.679532 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-7d761337-6d49-4aa1-820f-d841bb9961d8 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:33:31.680826 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-7d761337-6d49-4aa1-820f-d841bb9961d8 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] 10.222.0.16 "GET /compute/v2.1/servers/8b77de54-4c8b-4ed5-84f5-68bf69f9809f" status: 200 len: 1739 microversion: 2.20 time: 0.579881 Jul 16 21:33:31.680957 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-6eb6ce18-f7fe-46dd-8746-f5dd3292985b tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '22e264440fff43cdb7be6702839af37f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da2248fa190342f0bfe9211cae1e403b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:33:31.681405 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2396/4805] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:33:31 2022] GET /compute/v2.1/servers/8b77de54-4c8b-4ed5-84f5-68bf69f9809f => generated 1739 bytes in 581 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:33:31.696694 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=100 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:33:31.701600 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-6eb6ce18-f7fe-46dd-8746-f5dd3292985b tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1" status: 200 len: 1504 microversion: 2.1 time: 0.494836 Jul 16 21:33:31.702197 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2410/4806] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:33:31 2022] GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1 => generated 1504 bytes in 497 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:33:31.884359 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-76eef61c-875f-4f1b-8ce2-ac526177c388 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServersNegativeTestJSON-server-1522209065", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452", "networks": [{"uuid": "f14874ef-c1c1-4761-ab34-120b93719d08"}]}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:33:31.960715 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.network.neutron [None req-76eef61c-875f-4f1b-8ce2-ac526177c388 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] validate_networks() for [('f14874ef-c1c1-4761-ab34-120b93719d08', None, None, None, None, None)] {{(pid=138680) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 16 21:33:32.701086 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-75303223-61be-4c00-992c-fcd71e177ba8 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:33:32.701796 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-75303223-61be-4c00-992c-fcd71e177ba8 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] [instance: 8b77de54-4c8b-4ed5-84f5-68bf69f9809f] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:33:32.709864 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-75303223-61be-4c00-992c-fcd71e177ba8 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:32.710391 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-75303223-61be-4c00-992c-fcd71e177ba8 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:32.711452 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-75303223-61be-4c00-992c-fcd71e177ba8 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:32.711963 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-75303223-61be-4c00-992c-fcd71e177ba8 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:32.740382 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-76eef61c-875f-4f1b-8ce2-ac526177c388 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Flavor limits 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 16 21:33:32.744772 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-76eef61c-875f-4f1b-8ce2-ac526177c388 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Image limits 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 16 21:33:32.746197 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-76eef61c-875f-4f1b-8ce2-ac526177c388 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Flavor pref 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 16 21:33:32.746852 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-76eef61c-875f-4f1b-8ce2-ac526177c388 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Image pref 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 16 21:33:32.747503 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-76eef61c-875f-4f1b-8ce2-ac526177c388 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 16 21:33:32.764924 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-76eef61c-875f-4f1b-8ce2-ac526177c388 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:32.765717 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-76eef61c-875f-4f1b-8ce2-ac526177c388 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:32.773735 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-76eef61c-875f-4f1b-8ce2-ac526177c388 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:32.773735 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-76eef61c-875f-4f1b-8ce2-ac526177c388 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:32.805343 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-76eef61c-875f-4f1b-8ce2-ac526177c388 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Getting quotas for project da2248fa190342f0bfe9211cae1e403b. Resources: {'ram', 'instances', 'cores'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:33:32.812912 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-76eef61c-875f-4f1b-8ce2-ac526177c388 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Getting quotas for user 22e264440fff43cdb7be6702839af37f and project da2248fa190342f0bfe9211cae1e403b. Resources: {'ram', 'instances', 'cores'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:33:32.830994 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-76eef61c-875f-4f1b-8ce2-ac526177c388 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Going to run 1 instances... {{(pid=138680) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 16 21:33:32.833731 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-76eef61c-875f-4f1b-8ce2-ac526177c388 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: c77c9a95-ea71-40d7-b7a7-ded612d194da] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138680) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 16 21:33:32.847167 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_api:SELECT=95,nova_api:UPDATE=1,nova_api:INSERT=3,nova_api:DELETE=1 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:33:32.959151 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-76eef61c-875f-4f1b-8ce2-ac526177c388 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 1.092754 Jul 16 21:33:32.961253 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2397/4807] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 21:33:31 2022] POST /compute/v2.1/servers => generated 380 bytes in 1095 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 16 21:33:32.980043 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-75303223-61be-4c00-992c-fcd71e177ba8 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:33:32.987639 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-2679b19a-8c65-488d-874a-e65cffab4681 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:33:32.988388 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-2679b19a-8c65-488d-874a-e65cffab4681 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: c77c9a95-ea71-40d7-b7a7-ded612d194da] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:33:33.009560 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-75303223-61be-4c00-992c-fcd71e177ba8 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:33:33.016048 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-75303223-61be-4c00-992c-fcd71e177ba8 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:33:33.022621 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-75303223-61be-4c00-992c-fcd71e177ba8 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] 10.222.0.16 "GET /compute/v2.1/servers/8b77de54-4c8b-4ed5-84f5-68bf69f9809f" status: 200 len: 1739 microversion: 2.20 time: 0.325157 Jul 16 21:33:33.025532 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2411/4808] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:33:32 2022] GET /compute/v2.1/servers/8b77de54-4c8b-4ed5-84f5-68bf69f9809f => generated 1739 bytes in 329 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 16 21:33:33.150198 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-2679b19a-8c65-488d-874a-e65cffab4681 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '22e264440fff43cdb7be6702839af37f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da2248fa190342f0bfe9211cae1e403b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:33:33.161333 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-2679b19a-8c65-488d-874a-e65cffab4681 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "GET /compute/v2.1/servers/c77c9a95-ea71-40d7-b7a7-ded612d194da" status: 200 len: 1217 microversion: 2.1 time: 0.178030 Jul 16 21:33:33.162987 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2398/4809] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:33:32 2022] GET /compute/v2.1/servers/c77c9a95-ea71-40d7-b7a7-ded612d194da => generated 1217 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:33:34.043576 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-22adc2b1-1308-489c-b33e-e6bee83addef tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:33:34.044850 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-22adc2b1-1308-489c-b33e-e6bee83addef tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] [instance: 8b77de54-4c8b-4ed5-84f5-68bf69f9809f] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:33:34.052910 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-22adc2b1-1308-489c-b33e-e6bee83addef tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:34.053661 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-22adc2b1-1308-489c-b33e-e6bee83addef tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:34.055321 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-22adc2b1-1308-489c-b33e-e6bee83addef tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:34.055979 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-22adc2b1-1308-489c-b33e-e6bee83addef tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:34.184029 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-5d29cdf6-2dad-4af8-8526-4cfd6fe578fa tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:33:34.184807 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-5d29cdf6-2dad-4af8-8526-4cfd6fe578fa tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: c77c9a95-ea71-40d7-b7a7-ded612d194da] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:33:34.199657 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5d29cdf6-2dad-4af8-8526-4cfd6fe578fa tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:34.201030 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5d29cdf6-2dad-4af8-8526-4cfd6fe578fa tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:34.201831 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5d29cdf6-2dad-4af8-8526-4cfd6fe578fa tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:34.202355 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5d29cdf6-2dad-4af8-8526-4cfd6fe578fa tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:34.371144 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-5d29cdf6-2dad-4af8-8526-4cfd6fe578fa tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '22e264440fff43cdb7be6702839af37f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da2248fa190342f0bfe9211cae1e403b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:33:34.381336 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-5d29cdf6-2dad-4af8-8526-4cfd6fe578fa tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "GET /compute/v2.1/servers/c77c9a95-ea71-40d7-b7a7-ded612d194da" status: 200 len: 1221 microversion: 2.1 time: 0.200743 Jul 16 21:33:34.381614 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2399/4810] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:33:34 2022] GET /compute/v2.1/servers/c77c9a95-ea71-40d7-b7a7-ded612d194da => generated 1221 bytes in 202 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:33:34.462405 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-22adc2b1-1308-489c-b33e-e6bee83addef tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:33:34.479148 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-22adc2b1-1308-489c-b33e-e6bee83addef tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:33:34.479148 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-22adc2b1-1308-489c-b33e-e6bee83addef tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:33:34.479148 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-22adc2b1-1308-489c-b33e-e6bee83addef tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] 10.222.0.16 "GET /compute/v2.1/servers/8b77de54-4c8b-4ed5-84f5-68bf69f9809f" status: 200 len: 1721 microversion: 2.20 time: 0.435638 Jul 16 21:33:34.479148 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2412/4811] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:33:34 2022] GET /compute/v2.1/servers/8b77de54-4c8b-4ed5-84f5-68bf69f9809f => generated 1721 bytes in 438 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:33:35.397985 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-6739166a-a361-4da7-b637-1457e3a80e36 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:33:35.399265 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-6739166a-a361-4da7-b637-1457e3a80e36 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: c77c9a95-ea71-40d7-b7a7-ded612d194da] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:33:35.408144 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6739166a-a361-4da7-b637-1457e3a80e36 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:35.409518 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6739166a-a361-4da7-b637-1457e3a80e36 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:35.410486 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6739166a-a361-4da7-b637-1457e3a80e36 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:35.410984 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6739166a-a361-4da7-b637-1457e3a80e36 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:35.496481 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-3f36a708-18dd-4ad3-8b7f-ba2f5ffeac53 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:33:35.497827 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-3f36a708-18dd-4ad3-8b7f-ba2f5ffeac53 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] [instance: 8b77de54-4c8b-4ed5-84f5-68bf69f9809f] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:33:35.510598 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-6739166a-a361-4da7-b637-1457e3a80e36 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '22e264440fff43cdb7be6702839af37f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da2248fa190342f0bfe9211cae1e403b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:33:35.512415 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3f36a708-18dd-4ad3-8b7f-ba2f5ffeac53 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:35.512920 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3f36a708-18dd-4ad3-8b7f-ba2f5ffeac53 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:35.513358 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=100 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:33:35.516826 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3f36a708-18dd-4ad3-8b7f-ba2f5ffeac53 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:35.517300 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3f36a708-18dd-4ad3-8b7f-ba2f5ffeac53 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:35.535172 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-6739166a-a361-4da7-b637-1457e3a80e36 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "GET /compute/v2.1/servers/c77c9a95-ea71-40d7-b7a7-ded612d194da" status: 200 len: 1269 microversion: 2.1 time: 0.140784 Jul 16 21:33:35.540614 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2400/4812] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:33:35 2022] GET /compute/v2.1/servers/c77c9a95-ea71-40d7-b7a7-ded612d194da => generated 1269 bytes in 146 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:33:35.700524 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-3f36a708-18dd-4ad3-8b7f-ba2f5ffeac53 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:33:35.714086 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-3f36a708-18dd-4ad3-8b7f-ba2f5ffeac53 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:33:35.716184 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-3f36a708-18dd-4ad3-8b7f-ba2f5ffeac53 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:33:35.717701 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-3f36a708-18dd-4ad3-8b7f-ba2f5ffeac53 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] 10.222.0.16 "GET /compute/v2.1/servers/8b77de54-4c8b-4ed5-84f5-68bf69f9809f" status: 200 len: 1721 microversion: 2.20 time: 0.226734 Jul 16 21:33:35.718477 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2413/4813] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:33:35 2022] GET /compute/v2.1/servers/8b77de54-4c8b-4ed5-84f5-68bf69f9809f => generated 1721 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:33:36.563691 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-8d44679b-0ba7-4af9-b930-9f08e5e72678 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:33:36.568060 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-8d44679b-0ba7-4af9-b930-9f08e5e72678 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: c77c9a95-ea71-40d7-b7a7-ded612d194da] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:33:36.591903 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8d44679b-0ba7-4af9-b930-9f08e5e72678 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:36.592519 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8d44679b-0ba7-4af9-b930-9f08e5e72678 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:36.594026 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8d44679b-0ba7-4af9-b930-9f08e5e72678 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:36.594584 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8d44679b-0ba7-4af9-b930-9f08e5e72678 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:36.725729 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-8d44679b-0ba7-4af9-b930-9f08e5e72678 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '22e264440fff43cdb7be6702839af37f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da2248fa190342f0bfe9211cae1e403b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:33:36.738255 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-8d44679b-0ba7-4af9-b930-9f08e5e72678 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "GET /compute/v2.1/servers/c77c9a95-ea71-40d7-b7a7-ded612d194da" status: 200 len: 1275 microversion: 2.1 time: 0.184989 Jul 16 21:33:36.739668 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2401/4814] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:33:36 2022] GET /compute/v2.1/servers/c77c9a95-ea71-40d7-b7a7-ded612d194da => generated 1275 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:33:36.742528 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-2bbb3ea0-8275-4a3a-9868-439263403f08 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:33:36.743591 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-2bbb3ea0-8275-4a3a-9868-439263403f08 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] [instance: 8b77de54-4c8b-4ed5-84f5-68bf69f9809f] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:33:36.755319 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2bbb3ea0-8275-4a3a-9868-439263403f08 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:36.756425 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2bbb3ea0-8275-4a3a-9868-439263403f08 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:36.757625 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2bbb3ea0-8275-4a3a-9868-439263403f08 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:36.758305 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2bbb3ea0-8275-4a3a-9868-439263403f08 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:37.244909 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-2bbb3ea0-8275-4a3a-9868-439263403f08 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:33:37.271091 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-2bbb3ea0-8275-4a3a-9868-439263403f08 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:33:37.280481 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-2bbb3ea0-8275-4a3a-9868-439263403f08 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:33:37.285856 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-2bbb3ea0-8275-4a3a-9868-439263403f08 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] 10.222.0.16 "GET /compute/v2.1/servers/8b77de54-4c8b-4ed5-84f5-68bf69f9809f" status: 200 len: 1721 microversion: 2.20 time: 0.548750 Jul 16 21:33:37.288670 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2414/4815] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:33:36 2022] GET /compute/v2.1/servers/8b77de54-4c8b-4ed5-84f5-68bf69f9809f => generated 1721 bytes in 554 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 16 21:33:37.591455 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-2776297e-95e5-427c-9c2f-a174ec0187c3 req-af626321-c94b-40d4-a456-8d083ac6a13d service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "8b77de54-4c8b-4ed5-84f5-68bf69f9809f", "name": "network-vif-unplugged", "status": "completed", "tag": "5310cc77-6413-4115-9f89-f9fab39070d1"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:33:37.608773 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-2776297e-95e5-427c-9c2f-a174ec0187c3 req-af626321-c94b-40d4-a456-8d083ac6a13d service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:37.611218 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-2776297e-95e5-427c-9c2f-a174ec0187c3 req-af626321-c94b-40d4-a456-8d083ac6a13d service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:37.638376 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-2776297e-95e5-427c-9c2f-a174ec0187c3 req-af626321-c94b-40d4-a456-8d083ac6a13d service nova] Creating event network-vif-unplugged:5310cc77-6413-4115-9f89-f9fab39070d1 for instance 8b77de54-4c8b-4ed5-84f5-68bf69f9809f on nch2-master-1 Jul 16 21:33:37.649181 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-2776297e-95e5-427c-9c2f-a174ec0187c3 req-af626321-c94b-40d4-a456-8d083ac6a13d service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.067379 Jul 16 21:33:37.650036 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2402/4816] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:33:37 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 69 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:33:37.762406 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-7ffeb35a-3f00-4bf7-8ed0-7365ed5bb034 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:33:37.764080 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-7ffeb35a-3f00-4bf7-8ed0-7365ed5bb034 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: c77c9a95-ea71-40d7-b7a7-ded612d194da] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:33:37.775481 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7ffeb35a-3f00-4bf7-8ed0-7365ed5bb034 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:37.777180 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7ffeb35a-3f00-4bf7-8ed0-7365ed5bb034 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:37.782307 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7ffeb35a-3f00-4bf7-8ed0-7365ed5bb034 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:37.782307 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7ffeb35a-3f00-4bf7-8ed0-7365ed5bb034 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:38.178812 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-7ffeb35a-3f00-4bf7-8ed0-7365ed5bb034 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '22e264440fff43cdb7be6702839af37f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da2248fa190342f0bfe9211cae1e403b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:33:38.194225 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-7ffeb35a-3f00-4bf7-8ed0-7365ed5bb034 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "GET /compute/v2.1/servers/c77c9a95-ea71-40d7-b7a7-ded612d194da" status: 200 len: 1317 microversion: 2.1 time: 0.432639 Jul 16 21:33:38.212483 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2415/4817] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:33:37 2022] GET /compute/v2.1/servers/c77c9a95-ea71-40d7-b7a7-ded612d194da => generated 1317 bytes in 455 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:33:38.307291 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-320cbe53-7b28-41ba-93d6-d93647ae463a tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:33:38.308155 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-320cbe53-7b28-41ba-93d6-d93647ae463a tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] [instance: 8b77de54-4c8b-4ed5-84f5-68bf69f9809f] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:33:38.349329 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-320cbe53-7b28-41ba-93d6-d93647ae463a tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:38.349804 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-320cbe53-7b28-41ba-93d6-d93647ae463a tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:38.350620 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-320cbe53-7b28-41ba-93d6-d93647ae463a tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:38.357700 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-320cbe53-7b28-41ba-93d6-d93647ae463a tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:38.758438 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-320cbe53-7b28-41ba-93d6-d93647ae463a tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:33:38.779313 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-320cbe53-7b28-41ba-93d6-d93647ae463a tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:33:38.779313 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-320cbe53-7b28-41ba-93d6-d93647ae463a tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:33:38.779942 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-320cbe53-7b28-41ba-93d6-d93647ae463a tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] 10.222.0.16 "GET /compute/v2.1/servers/8b77de54-4c8b-4ed5-84f5-68bf69f9809f" status: 200 len: 1664 microversion: 2.20 time: 0.477869 Jul 16 21:33:38.790779 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2403/4818] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:33:38 2022] GET /compute/v2.1/servers/8b77de54-4c8b-4ed5-84f5-68bf69f9809f => generated 1664 bytes in 488 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:33:38.802743 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-1a2e2c8e-67f1-46d3-b2bf-ea78fe7f26bc tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "3ac2d5bc-2859-4330-aaae-14914ce01b8a"}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:33:38.804356 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-1a2e2c8e-67f1-46d3-b2bf-ea78fe7f26bc tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] [instance: 8b77de54-4c8b-4ed5-84f5-68bf69f9809f] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:33:38.812420 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1a2e2c8e-67f1-46d3-b2bf-ea78fe7f26bc tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:38.816168 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1a2e2c8e-67f1-46d3-b2bf-ea78fe7f26bc tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:38.816168 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1a2e2c8e-67f1-46d3-b2bf-ea78fe7f26bc tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:38.816168 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1a2e2c8e-67f1-46d3-b2bf-ea78fe7f26bc tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:39.220496 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-25b1b38d-fa87-4bba-be15-df4eb6086eeb tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:33:39.223220 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-25b1b38d-fa87-4bba-be15-df4eb6086eeb tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: c77c9a95-ea71-40d7-b7a7-ded612d194da] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:33:39.233755 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-25b1b38d-fa87-4bba-be15-df4eb6086eeb tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:39.234305 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-25b1b38d-fa87-4bba-be15-df4eb6086eeb tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:39.235236 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-25b1b38d-fa87-4bba-be15-df4eb6086eeb tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:39.235827 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-25b1b38d-fa87-4bba-be15-df4eb6086eeb tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:39.490433 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-25b1b38d-fa87-4bba-be15-df4eb6086eeb tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '22e264440fff43cdb7be6702839af37f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da2248fa190342f0bfe9211cae1e403b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:33:39.499050 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-25b1b38d-fa87-4bba-be15-df4eb6086eeb tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "GET /compute/v2.1/servers/c77c9a95-ea71-40d7-b7a7-ded612d194da" status: 200 len: 1317 microversion: 2.1 time: 0.284663 Jul 16 21:33:39.499652 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2404/4819] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:33:39 2022] GET /compute/v2.1/servers/c77c9a95-ea71-40d7-b7a7-ded612d194da => generated 1317 bytes in 286 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:33:39.766077 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-c2488a09-f03c-4dee-abd8-d222ef43eeb3 req-83f20db9-34b2-476b-b9c7-f14a0b2c6dd9 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "c77c9a95-ea71-40d7-b7a7-ded612d194da", "tag": "4c04a897-c81c-4123-9774-c657689d7d36"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:33:39.785380 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-c2488a09-f03c-4dee-abd8-d222ef43eeb3 req-83f20db9-34b2-476b-b9c7-f14a0b2c6dd9 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:39.787285 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-c2488a09-f03c-4dee-abd8-d222ef43eeb3 req-83f20db9-34b2-476b-b9c7-f14a0b2c6dd9 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:39.814623 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-c2488a09-f03c-4dee-abd8-d222ef43eeb3 req-83f20db9-34b2-476b-b9c7-f14a0b2c6dd9 service nova] Creating event network-changed:4c04a897-c81c-4123-9774-c657689d7d36 for instance c77c9a95-ea71-40d7-b7a7-ded612d194da on nch1-master-1 Jul 16 21:33:39.827409 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-c2488a09-f03c-4dee-abd8-d222ef43eeb3 req-83f20db9-34b2-476b-b9c7-f14a0b2c6dd9 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.065257 Jul 16 21:33:39.833038 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2405/4820] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:33:39 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 72 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 16 21:33:40.423995 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-1a2e2c8e-67f1-46d3-b2bf-ea78fe7f26bc tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] 10.222.0.16 "POST /compute/v2.1/servers/8b77de54-4c8b-4ed5-84f5-68bf69f9809f/os-volume_attachments" status: 200 len: 188 microversion: 2.20 time: 1.626522 Jul 16 21:33:40.425589 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2416/4821] 10.222.0.16 () {62 vars in 1363 bytes} [Sat Jul 16 21:33:38 2022] POST /compute/v2.1/servers/8b77de54-4c8b-4ed5-84f5-68bf69f9809f/os-volume_attachments => generated 188 bytes in 1629 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 16 21:33:40.517614 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-69f96ff2-9fc7-46c6-8bf6-ca68e109fba1 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:33:40.518516 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-69f96ff2-9fc7-46c6-8bf6-ca68e109fba1 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: c77c9a95-ea71-40d7-b7a7-ded612d194da] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:33:40.529188 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-69f96ff2-9fc7-46c6-8bf6-ca68e109fba1 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:40.529654 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-69f96ff2-9fc7-46c6-8bf6-ca68e109fba1 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:40.530437 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-69f96ff2-9fc7-46c6-8bf6-ca68e109fba1 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:40.530920 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-69f96ff2-9fc7-46c6-8bf6-ca68e109fba1 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:40.605723 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-1ed57a23-fa7b-43d1-8bbd-924f7156e668 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:33:40.606342 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-1ed57a23-fa7b-43d1-8bbd-924f7156e668 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] [instance: 8b77de54-4c8b-4ed5-84f5-68bf69f9809f] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:33:40.615171 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1ed57a23-fa7b-43d1-8bbd-924f7156e668 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:40.615730 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1ed57a23-fa7b-43d1-8bbd-924f7156e668 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:40.616803 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1ed57a23-fa7b-43d1-8bbd-924f7156e668 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:40.617338 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1ed57a23-fa7b-43d1-8bbd-924f7156e668 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:40.834690 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-69f96ff2-9fc7-46c6-8bf6-ca68e109fba1 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '22e264440fff43cdb7be6702839af37f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da2248fa190342f0bfe9211cae1e403b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:33:40.843354 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-69f96ff2-9fc7-46c6-8bf6-ca68e109fba1 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "GET /compute/v2.1/servers/c77c9a95-ea71-40d7-b7a7-ded612d194da" status: 200 len: 1317 microversion: 2.1 time: 0.330543 Jul 16 21:33:40.844801 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2406/4822] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:33:40 2022] GET /compute/v2.1/servers/c77c9a95-ea71-40d7-b7a7-ded612d194da => generated 1317 bytes in 333 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:33:41.505798 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-1ed57a23-fa7b-43d1-8bbd-924f7156e668 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] 10.222.0.16 "DELETE /compute/v2.1/servers/8b77de54-4c8b-4ed5-84f5-68bf69f9809f/os-volume_attachments/3ac2d5bc-2859-4330-aaae-14914ce01b8a" status: 202 len: 0 microversion: 2.20 time: 0.904472 Jul 16 21:33:41.509687 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2417/4823] 10.222.0.16 () {60 vars in 1456 bytes} [Sat Jul 16 21:33:40 2022] DELETE /compute/v2.1/servers/8b77de54-4c8b-4ed5-84f5-68bf69f9809f/os-volume_attachments/3ac2d5bc-2859-4330-aaae-14914ce01b8a => generated 0 bytes in 906 msecs (HTTP/1.1 202) 9 headers in 363 bytes (2 switches on core 0) Jul 16 21:33:41.567540 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-20f00d20-d267-4626-b414-d4881a3333f5 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Action: 'action', calling method: >, body: {"unshelve": null} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:33:41.568186 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-20f00d20-d267-4626-b414-d4881a3333f5 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] [instance: 8b77de54-4c8b-4ed5-84f5-68bf69f9809f] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:33:41.575002 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-20f00d20-d267-4626-b414-d4881a3333f5 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:41.575506 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-20f00d20-d267-4626-b414-d4881a3333f5 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:41.576283 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-20f00d20-d267-4626-b414-d4881a3333f5 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:41.576728 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-20f00d20-d267-4626-b414-d4881a3333f5 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:41.867768 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-4cc9fbaa-c06d-4432-8d5c-083f2762607c tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:33:41.869289 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-4cc9fbaa-c06d-4432-8d5c-083f2762607c tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: c77c9a95-ea71-40d7-b7a7-ded612d194da] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:33:41.876706 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4cc9fbaa-c06d-4432-8d5c-083f2762607c tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:41.877628 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4cc9fbaa-c06d-4432-8d5c-083f2762607c tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:41.878727 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4cc9fbaa-c06d-4432-8d5c-083f2762607c tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:41.879214 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4cc9fbaa-c06d-4432-8d5c-083f2762607c tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:41.910700 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-20f00d20-d267-4626-b414-d4881a3333f5 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] 10.222.0.16 "POST /compute/v2.1/servers/8b77de54-4c8b-4ed5-84f5-68bf69f9809f/action" status: 202 len: 0 microversion: 2.20 time: 0.347871 Jul 16 21:33:41.915535 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2407/4824] 10.222.0.16 () {62 vars in 1318 bytes} [Sat Jul 16 21:33:41 2022] POST /compute/v2.1/servers/8b77de54-4c8b-4ed5-84f5-68bf69f9809f/action => generated 0 bytes in 353 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Jul 16 21:33:41.926515 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-83b79027-701f-422a-8701-0715380a3758 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:33:41.929265 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-83b79027-701f-422a-8701-0715380a3758 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] [instance: 8b77de54-4c8b-4ed5-84f5-68bf69f9809f] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:33:41.947464 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-83b79027-701f-422a-8701-0715380a3758 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:41.948226 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-83b79027-701f-422a-8701-0715380a3758 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:41.949223 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-83b79027-701f-422a-8701-0715380a3758 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:41.949672 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-83b79027-701f-422a-8701-0715380a3758 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:42.194967 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-4cc9fbaa-c06d-4432-8d5c-083f2762607c tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '22e264440fff43cdb7be6702839af37f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da2248fa190342f0bfe9211cae1e403b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:33:42.204313 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-4cc9fbaa-c06d-4432-8d5c-083f2762607c tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "GET /compute/v2.1/servers/c77c9a95-ea71-40d7-b7a7-ded612d194da" status: 200 len: 1317 microversion: 2.1 time: 0.347092 Jul 16 21:33:42.205650 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2418/4825] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:33:41 2022] GET /compute/v2.1/servers/c77c9a95-ea71-40d7-b7a7-ded612d194da => generated 1317 bytes in 349 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:33:42.274402 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-83b79027-701f-422a-8701-0715380a3758 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:33:42.285970 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-83b79027-701f-422a-8701-0715380a3758 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:33:42.288127 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-83b79027-701f-422a-8701-0715380a3758 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:33:42.289707 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-83b79027-701f-422a-8701-0715380a3758 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] 10.222.0.16 "GET /compute/v2.1/servers/8b77de54-4c8b-4ed5-84f5-68bf69f9809f" status: 200 len: 1672 microversion: 2.20 time: 0.372874 Jul 16 21:33:42.301560 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2408/4826] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:33:41 2022] GET /compute/v2.1/servers/8b77de54-4c8b-4ed5-84f5-68bf69f9809f => generated 1672 bytes in 375 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:33:43.227476 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-de746607-719d-4bf4-81a5-a81171edcf80 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:33:43.230879 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-de746607-719d-4bf4-81a5-a81171edcf80 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: c77c9a95-ea71-40d7-b7a7-ded612d194da] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:33:43.237515 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-de746607-719d-4bf4-81a5-a81171edcf80 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:43.239578 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-de746607-719d-4bf4-81a5-a81171edcf80 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:43.240851 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-de746607-719d-4bf4-81a5-a81171edcf80 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:43.241459 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-de746607-719d-4bf4-81a5-a81171edcf80 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:43.314287 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-1fa7f1b4-701b-42cc-9312-7295a6ad923e tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:33:43.315059 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-1fa7f1b4-701b-42cc-9312-7295a6ad923e tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] [instance: 8b77de54-4c8b-4ed5-84f5-68bf69f9809f] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:33:43.320765 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell0:SELECT=2 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:33:43.321617 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell0:SELECT=8 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:33:43.333630 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1fa7f1b4-701b-42cc-9312-7295a6ad923e tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:43.333630 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1fa7f1b4-701b-42cc-9312-7295a6ad923e tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:43.342316 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1fa7f1b4-701b-42cc-9312-7295a6ad923e tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:43.342927 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1fa7f1b4-701b-42cc-9312-7295a6ad923e tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.009s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:43.587413 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-de746607-719d-4bf4-81a5-a81171edcf80 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '22e264440fff43cdb7be6702839af37f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da2248fa190342f0bfe9211cae1e403b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:33:43.614383 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-1fa7f1b4-701b-42cc-9312-7295a6ad923e tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:33:43.614532 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-de746607-719d-4bf4-81a5-a81171edcf80 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "GET /compute/v2.1/servers/c77c9a95-ea71-40d7-b7a7-ded612d194da" status: 200 len: 1317 microversion: 2.1 time: 0.381279 Jul 16 21:33:43.614532 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2419/4827] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:33:43 2022] GET /compute/v2.1/servers/c77c9a95-ea71-40d7-b7a7-ded612d194da => generated 1317 bytes in 383 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:33:43.614733 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-1fa7f1b4-701b-42cc-9312-7295a6ad923e tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:33:43.615735 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-1fa7f1b4-701b-42cc-9312-7295a6ad923e tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:33:43.617137 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-1fa7f1b4-701b-42cc-9312-7295a6ad923e tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] 10.222.0.16 "GET /compute/v2.1/servers/8b77de54-4c8b-4ed5-84f5-68bf69f9809f" status: 200 len: 1672 microversion: 2.20 time: 0.311567 Jul 16 21:33:43.619669 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2409/4828] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:33:43 2022] GET /compute/v2.1/servers/8b77de54-4c8b-4ed5-84f5-68bf69f9809f => generated 1672 bytes in 315 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:33:44.619681 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-189eeb52-e412-4217-a000-713afe911a59 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:33:44.620312 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-189eeb52-e412-4217-a000-713afe911a59 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: c77c9a95-ea71-40d7-b7a7-ded612d194da] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:33:44.636112 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-189eeb52-e412-4217-a000-713afe911a59 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:44.637427 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-189eeb52-e412-4217-a000-713afe911a59 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:44.638854 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-189eeb52-e412-4217-a000-713afe911a59 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:44.639491 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-189eeb52-e412-4217-a000-713afe911a59 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:44.640067 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-48a88126-0357-437b-b128-bde554d989da tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:33:44.641328 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-48a88126-0357-437b-b128-bde554d989da tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] [instance: 8b77de54-4c8b-4ed5-84f5-68bf69f9809f] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:33:44.672133 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-48a88126-0357-437b-b128-bde554d989da tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:44.672726 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-48a88126-0357-437b-b128-bde554d989da tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.010s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:44.673642 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-48a88126-0357-437b-b128-bde554d989da tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:44.674162 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-48a88126-0357-437b-b128-bde554d989da tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:45.095480 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-189eeb52-e412-4217-a000-713afe911a59 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '22e264440fff43cdb7be6702839af37f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da2248fa190342f0bfe9211cae1e403b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:33:45.106771 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-189eeb52-e412-4217-a000-713afe911a59 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "GET /compute/v2.1/servers/c77c9a95-ea71-40d7-b7a7-ded612d194da" status: 200 len: 1317 microversion: 2.1 time: 0.492231 Jul 16 21:33:45.108043 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2420/4829] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:33:44 2022] GET /compute/v2.1/servers/c77c9a95-ea71-40d7-b7a7-ded612d194da => generated 1317 bytes in 496 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:33:45.172683 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-48a88126-0357-437b-b128-bde554d989da tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:33:45.192578 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-48a88126-0357-437b-b128-bde554d989da tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:33:45.196563 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-48a88126-0357-437b-b128-bde554d989da tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:33:45.199471 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-48a88126-0357-437b-b128-bde554d989da tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] 10.222.0.16 "GET /compute/v2.1/servers/8b77de54-4c8b-4ed5-84f5-68bf69f9809f" status: 200 len: 1730 microversion: 2.20 time: 0.563172 Jul 16 21:33:45.202799 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2410/4830] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:33:44 2022] GET /compute/v2.1/servers/8b77de54-4c8b-4ed5-84f5-68bf69f9809f => generated 1730 bytes in 568 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 16 21:33:45.667298 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-be078a16-5267-42e3-93fb-e1c149ca4f16 req-ef2a2feb-6310-4303-b3ad-ea22f486cf4b service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "c77c9a95-ea71-40d7-b7a7-ded612d194da", "name": "network-vif-plugged", "status": "completed", "tag": "4c04a897-c81c-4123-9774-c657689d7d36"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:33:45.676958 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_api:SELECT=65,nova_api:DELETE=1 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:33:45.679638 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-be078a16-5267-42e3-93fb-e1c149ca4f16 req-ef2a2feb-6310-4303-b3ad-ea22f486cf4b service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:45.680342 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-be078a16-5267-42e3-93fb-e1c149ca4f16 req-ef2a2feb-6310-4303-b3ad-ea22f486cf4b service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:45.707111 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-be078a16-5267-42e3-93fb-e1c149ca4f16 req-ef2a2feb-6310-4303-b3ad-ea22f486cf4b service nova] Creating event network-vif-plugged:4c04a897-c81c-4123-9774-c657689d7d36 for instance c77c9a95-ea71-40d7-b7a7-ded612d194da on nch1-master-1 Jul 16 21:33:45.710973 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-be078a16-5267-42e3-93fb-e1c149ca4f16 req-ef2a2feb-6310-4303-b3ad-ea22f486cf4b service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.047401 Jul 16 21:33:45.711901 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2421/4831] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:33:45 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:33:46.124555 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-4a018c0b-a110-4f33-a63a-1acef871db9e tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:33:46.125175 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-4a018c0b-a110-4f33-a63a-1acef871db9e tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: c77c9a95-ea71-40d7-b7a7-ded612d194da] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:33:46.133203 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4a018c0b-a110-4f33-a63a-1acef871db9e tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:46.133996 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4a018c0b-a110-4f33-a63a-1acef871db9e tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:46.134955 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4a018c0b-a110-4f33-a63a-1acef871db9e tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:46.135588 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4a018c0b-a110-4f33-a63a-1acef871db9e tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:46.220685 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-0291440e-bd54-41a3-807e-5d68a12c43b2 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:33:46.222180 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-0291440e-bd54-41a3-807e-5d68a12c43b2 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] [instance: 8b77de54-4c8b-4ed5-84f5-68bf69f9809f] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:33:46.238047 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0291440e-bd54-41a3-807e-5d68a12c43b2 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:46.238736 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0291440e-bd54-41a3-807e-5d68a12c43b2 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:46.239757 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0291440e-bd54-41a3-807e-5d68a12c43b2 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:46.240705 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0291440e-bd54-41a3-807e-5d68a12c43b2 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:47.024959 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-0291440e-bd54-41a3-807e-5d68a12c43b2 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:33:47.040193 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-0291440e-bd54-41a3-807e-5d68a12c43b2 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:33:47.040751 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-0291440e-bd54-41a3-807e-5d68a12c43b2 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:33:47.044792 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-0291440e-bd54-41a3-807e-5d68a12c43b2 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] 10.222.0.16 "GET /compute/v2.1/servers/8b77de54-4c8b-4ed5-84f5-68bf69f9809f" status: 200 len: 1730 microversion: 2.20 time: 0.828604 Jul 16 21:33:47.047040 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2422/4832] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:33:46 2022] GET /compute/v2.1/servers/8b77de54-4c8b-4ed5-84f5-68bf69f9809f => generated 1730 bytes in 831 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 16 21:33:47.053449 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-4a018c0b-a110-4f33-a63a-1acef871db9e tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '22e264440fff43cdb7be6702839af37f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da2248fa190342f0bfe9211cae1e403b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:33:47.068130 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-4a018c0b-a110-4f33-a63a-1acef871db9e tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "GET /compute/v2.1/servers/c77c9a95-ea71-40d7-b7a7-ded612d194da" status: 200 len: 1317 microversion: 2.1 time: 0.944656 Jul 16 21:33:47.069706 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2411/4833] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:33:46 2022] GET /compute/v2.1/servers/c77c9a95-ea71-40d7-b7a7-ded612d194da => generated 1317 bytes in 949 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:33:48.068632 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-9be58a06-91b2-414b-bfb7-a1d73c3308ce tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:33:48.070105 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-9be58a06-91b2-414b-bfb7-a1d73c3308ce tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] [instance: 8b77de54-4c8b-4ed5-84f5-68bf69f9809f] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:33:48.086298 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-a541861c-4336-4be7-ab9c-c2a6543eb399 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:33:48.091134 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9be58a06-91b2-414b-bfb7-a1d73c3308ce tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:48.091627 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9be58a06-91b2-414b-bfb7-a1d73c3308ce tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:48.094767 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9be58a06-91b2-414b-bfb7-a1d73c3308ce tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:48.095427 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9be58a06-91b2-414b-bfb7-a1d73c3308ce tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:48.098376 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-a541861c-4336-4be7-ab9c-c2a6543eb399 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: c77c9a95-ea71-40d7-b7a7-ded612d194da] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:33:48.110674 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=87,nova_cell1:INSERT=5,nova_cell1:UPDATE=8 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:33:48.111796 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a541861c-4336-4be7-ab9c-c2a6543eb399 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:48.113178 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a541861c-4336-4be7-ab9c-c2a6543eb399 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:48.114492 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a541861c-4336-4be7-ab9c-c2a6543eb399 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:48.115155 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a541861c-4336-4be7-ab9c-c2a6543eb399 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:48.536408 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-9be58a06-91b2-414b-bfb7-a1d73c3308ce tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:33:48.548606 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-a541861c-4336-4be7-ab9c-c2a6543eb399 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '22e264440fff43cdb7be6702839af37f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da2248fa190342f0bfe9211cae1e403b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:33:48.561926 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-a541861c-4336-4be7-ab9c-c2a6543eb399 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "GET /compute/v2.1/servers/c77c9a95-ea71-40d7-b7a7-ded612d194da" status: 200 len: 1505 microversion: 2.1 time: 0.479679 Jul 16 21:33:48.596183 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-9be58a06-91b2-414b-bfb7-a1d73c3308ce tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:33:48.604149 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2412/4834] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:33:48 2022] GET /compute/v2.1/servers/c77c9a95-ea71-40d7-b7a7-ded612d194da => generated 1505 bytes in 515 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:33:48.613429 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-9be58a06-91b2-414b-bfb7-a1d73c3308ce tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:33:48.630540 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-706bbee8-72f6-4870-839f-32c5cdcd9e10 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:33:48.647267 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-9be58a06-91b2-414b-bfb7-a1d73c3308ce tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] 10.222.0.16 "GET /compute/v2.1/servers/8b77de54-4c8b-4ed5-84f5-68bf69f9809f" status: 200 len: 1730 microversion: 2.20 time: 0.569144 Jul 16 21:33:48.647267 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2423/4835] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:33:48 2022] GET /compute/v2.1/servers/8b77de54-4c8b-4ed5-84f5-68bf69f9809f => generated 1730 bytes in 571 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 16 21:33:48.647514 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-706bbee8-72f6-4870-839f-32c5cdcd9e10 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: c77c9a95-ea71-40d7-b7a7-ded612d194da] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:33:48.647514 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-706bbee8-72f6-4870-839f-32c5cdcd9e10 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:48.647514 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-706bbee8-72f6-4870-839f-32c5cdcd9e10 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:48.647514 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-706bbee8-72f6-4870-839f-32c5cdcd9e10 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:48.647514 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-706bbee8-72f6-4870-839f-32c5cdcd9e10 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:48.703700 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-706bbee8-72f6-4870-839f-32c5cdcd9e10 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: c77c9a95-ea71-40d7-b7a7-ded612d194da] Going to try to terminate instance {{(pid=138680) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 16 21:33:49.101134 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-706bbee8-72f6-4870-839f-32c5cdcd9e10 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "DELETE /compute/v2.1/servers/c77c9a95-ea71-40d7-b7a7-ded612d194da" status: 204 len: 0 microversion: 2.1 time: 0.496153 Jul 16 21:33:49.101822 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2413/4836] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:33:48 2022] DELETE /compute/v2.1/servers/c77c9a95-ea71-40d7-b7a7-ded612d194da => generated 0 bytes in 499 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 16 21:33:49.123622 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-014198fd-9b23-4f84-b0b3-b438f7bf57a2 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:33:49.131074 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-014198fd-9b23-4f84-b0b3-b438f7bf57a2 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: c77c9a95-ea71-40d7-b7a7-ded612d194da] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:33:49.140299 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-014198fd-9b23-4f84-b0b3-b438f7bf57a2 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:49.141050 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-014198fd-9b23-4f84-b0b3-b438f7bf57a2 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:49.148628 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-014198fd-9b23-4f84-b0b3-b438f7bf57a2 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:49.151066 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-014198fd-9b23-4f84-b0b3-b438f7bf57a2 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:49.474385 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-014198fd-9b23-4f84-b0b3-b438f7bf57a2 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '22e264440fff43cdb7be6702839af37f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da2248fa190342f0bfe9211cae1e403b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:33:49.483184 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-014198fd-9b23-4f84-b0b3-b438f7bf57a2 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "GET /compute/v2.1/servers/c77c9a95-ea71-40d7-b7a7-ded612d194da" status: 200 len: 1511 microversion: 2.1 time: 0.372903 Jul 16 21:33:49.494384 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2424/4837] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:33:49 2022] GET /compute/v2.1/servers/c77c9a95-ea71-40d7-b7a7-ded612d194da => generated 1511 bytes in 375 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:33:49.657463 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-abf580b7-6ab9-489c-93bb-de1538c4c116 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:33:49.665258 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-abf580b7-6ab9-489c-93bb-de1538c4c116 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] [instance: 8b77de54-4c8b-4ed5-84f5-68bf69f9809f] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:33:49.666694 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-abf580b7-6ab9-489c-93bb-de1538c4c116 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:49.667420 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-abf580b7-6ab9-489c-93bb-de1538c4c116 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:49.668428 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-abf580b7-6ab9-489c-93bb-de1538c4c116 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:49.669067 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-abf580b7-6ab9-489c-93bb-de1538c4c116 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:49.991170 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-abf580b7-6ab9-489c-93bb-de1538c4c116 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:33:50.006975 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-abf580b7-6ab9-489c-93bb-de1538c4c116 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:33:50.008935 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-abf580b7-6ab9-489c-93bb-de1538c4c116 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:33:50.010379 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-abf580b7-6ab9-489c-93bb-de1538c4c116 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] 10.222.0.16 "GET /compute/v2.1/servers/8b77de54-4c8b-4ed5-84f5-68bf69f9809f" status: 200 len: 1730 microversion: 2.20 time: 0.357505 Jul 16 21:33:50.011335 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2414/4838] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:33:49 2022] GET /compute/v2.1/servers/8b77de54-4c8b-4ed5-84f5-68bf69f9809f => generated 1730 bytes in 359 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:33:50.512169 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-21a161ec-e1a2-40ba-93b7-e223858c6589 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:33:50.518624 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-21a161ec-e1a2-40ba-93b7-e223858c6589 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: c77c9a95-ea71-40d7-b7a7-ded612d194da] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:33:50.537308 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-21a161ec-e1a2-40ba-93b7-e223858c6589 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:50.542074 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-21a161ec-e1a2-40ba-93b7-e223858c6589 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:50.542074 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-21a161ec-e1a2-40ba-93b7-e223858c6589 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:50.542074 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-21a161ec-e1a2-40ba-93b7-e223858c6589 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:50.850961 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-21a161ec-e1a2-40ba-93b7-e223858c6589 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '22e264440fff43cdb7be6702839af37f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da2248fa190342f0bfe9211cae1e403b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:33:50.873097 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-21a161ec-e1a2-40ba-93b7-e223858c6589 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "GET /compute/v2.1/servers/c77c9a95-ea71-40d7-b7a7-ded612d194da" status: 200 len: 1511 microversion: 2.1 time: 0.373216 Jul 16 21:33:50.897592 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2425/4839] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:33:50 2022] GET /compute/v2.1/servers/c77c9a95-ea71-40d7-b7a7-ded612d194da => generated 1511 bytes in 399 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:33:51.036170 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-211d3e1e-24e8-459b-b809-8b89e8a1c3ff tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:33:51.037944 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-211d3e1e-24e8-459b-b809-8b89e8a1c3ff tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] [instance: 8b77de54-4c8b-4ed5-84f5-68bf69f9809f] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:33:51.064327 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-211d3e1e-24e8-459b-b809-8b89e8a1c3ff tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:51.067363 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-211d3e1e-24e8-459b-b809-8b89e8a1c3ff tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:51.069680 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-211d3e1e-24e8-459b-b809-8b89e8a1c3ff tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:51.070844 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-211d3e1e-24e8-459b-b809-8b89e8a1c3ff tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:51.545233 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-211d3e1e-24e8-459b-b809-8b89e8a1c3ff tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:33:51.570195 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-211d3e1e-24e8-459b-b809-8b89e8a1c3ff tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:33:51.577588 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-211d3e1e-24e8-459b-b809-8b89e8a1c3ff tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:33:51.579467 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-211d3e1e-24e8-459b-b809-8b89e8a1c3ff tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] 10.222.0.16 "GET /compute/v2.1/servers/8b77de54-4c8b-4ed5-84f5-68bf69f9809f" status: 200 len: 1730 microversion: 2.20 time: 0.551015 Jul 16 21:33:51.582487 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2415/4840] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:33:51 2022] GET /compute/v2.1/servers/8b77de54-4c8b-4ed5-84f5-68bf69f9809f => generated 1730 bytes in 555 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:33:51.916833 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-c4ee3e84-328a-432f-8904-d84e25e9fc45 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:33:51.918244 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-c4ee3e84-328a-432f-8904-d84e25e9fc45 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: c77c9a95-ea71-40d7-b7a7-ded612d194da] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:33:51.926078 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c4ee3e84-328a-432f-8904-d84e25e9fc45 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:51.926992 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c4ee3e84-328a-432f-8904-d84e25e9fc45 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:51.928062 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c4ee3e84-328a-432f-8904-d84e25e9fc45 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:51.928720 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c4ee3e84-328a-432f-8904-d84e25e9fc45 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:52.159723 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-c4ee3e84-328a-432f-8904-d84e25e9fc45 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '22e264440fff43cdb7be6702839af37f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da2248fa190342f0bfe9211cae1e403b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:33:52.172948 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-c4ee3e84-328a-432f-8904-d84e25e9fc45 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "GET /compute/v2.1/servers/c77c9a95-ea71-40d7-b7a7-ded612d194da" status: 200 len: 1511 microversion: 2.1 time: 0.259964 Jul 16 21:33:52.172948 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2426/4841] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:33:51 2022] GET /compute/v2.1/servers/c77c9a95-ea71-40d7-b7a7-ded612d194da => generated 1511 bytes in 262 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:33:52.477185 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-777ca607-f969-4e8b-a024-3c0a0747af9d req-ef52fb71-9108-4520-9d4b-47b85c08940b service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "c77c9a95-ea71-40d7-b7a7-ded612d194da", "name": "network-vif-deleted", "tag": "4c04a897-c81c-4123-9774-c657689d7d36"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:33:52.528297 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-777ca607-f969-4e8b-a024-3c0a0747af9d req-ef52fb71-9108-4520-9d4b-47b85c08940b service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:52.532434 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-777ca607-f969-4e8b-a024-3c0a0747af9d req-ef52fb71-9108-4520-9d4b-47b85c08940b service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:52.559106 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=96,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:33:52.561584 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-777ca607-f969-4e8b-a024-3c0a0747af9d req-ef52fb71-9108-4520-9d4b-47b85c08940b service nova] Creating event network-vif-deleted:4c04a897-c81c-4123-9774-c657689d7d36 for instance c77c9a95-ea71-40d7-b7a7-ded612d194da on nch1-master-1 Jul 16 21:33:52.569966 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-777ca607-f969-4e8b-a024-3c0a0747af9d req-ef52fb71-9108-4520-9d4b-47b85c08940b service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.097269 Jul 16 21:33:52.580152 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2416/4842] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:33:52 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 109 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:33:52.606977 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-61d3f76e-13dd-4001-9e33-b3e6a42f31a7 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:33:52.608568 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-61d3f76e-13dd-4001-9e33-b3e6a42f31a7 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] [instance: 8b77de54-4c8b-4ed5-84f5-68bf69f9809f] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:33:52.616546 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-61d3f76e-13dd-4001-9e33-b3e6a42f31a7 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:52.617148 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-61d3f76e-13dd-4001-9e33-b3e6a42f31a7 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:52.618265 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-61d3f76e-13dd-4001-9e33-b3e6a42f31a7 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:52.619064 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-61d3f76e-13dd-4001-9e33-b3e6a42f31a7 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:52.977156 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-61d3f76e-13dd-4001-9e33-b3e6a42f31a7 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:33:52.999823 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-61d3f76e-13dd-4001-9e33-b3e6a42f31a7 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:33:53.001964 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-61d3f76e-13dd-4001-9e33-b3e6a42f31a7 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:33:53.004946 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-61d3f76e-13dd-4001-9e33-b3e6a42f31a7 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] 10.222.0.16 "GET /compute/v2.1/servers/8b77de54-4c8b-4ed5-84f5-68bf69f9809f" status: 200 len: 1730 microversion: 2.20 time: 0.402301 Jul 16 21:33:53.020957 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2427/4843] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:33:52 2022] GET /compute/v2.1/servers/8b77de54-4c8b-4ed5-84f5-68bf69f9809f => generated 1730 bytes in 416 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:33:53.190624 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-68f9ea49-9c8f-49ca-b963-ad3756d8fac8 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:33:53.191700 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-68f9ea49-9c8f-49ca-b963-ad3756d8fac8 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: c77c9a95-ea71-40d7-b7a7-ded612d194da] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:33:53.214881 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-68f9ea49-9c8f-49ca-b963-ad3756d8fac8 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:53.215379 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-68f9ea49-9c8f-49ca-b963-ad3756d8fac8 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:53.216216 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-68f9ea49-9c8f-49ca-b963-ad3756d8fac8 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:53.216664 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-68f9ea49-9c8f-49ca-b963-ad3756d8fac8 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:53.276303 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-68f9ea49-9c8f-49ca-b963-ad3756d8fac8 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:53.277316 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-68f9ea49-9c8f-49ca-b963-ad3756d8fac8 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:53.278076 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.objects.instance [None req-68f9ea49-9c8f-49ca-b963-ad3756d8fac8 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lazy-loading 'fault' on Instance uuid c77c9a95-ea71-40d7-b7a7-ded612d194da {{(pid=138680) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:33:53.828670 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-68f9ea49-9c8f-49ca-b963-ad3756d8fac8 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '22e264440fff43cdb7be6702839af37f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da2248fa190342f0bfe9211cae1e403b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:33:53.836001 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-68f9ea49-9c8f-49ca-b963-ad3756d8fac8 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "GET /compute/v2.1/servers/c77c9a95-ea71-40d7-b7a7-ded612d194da" status: 200 len: 1307 microversion: 2.1 time: 0.650453 Jul 16 21:33:53.849131 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2417/4844] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:33:53 2022] GET /compute/v2.1/servers/c77c9a95-ea71-40d7-b7a7-ded612d194da => generated 1307 bytes in 664 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:33:54.028027 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-7811fd26-f5a6-4650-a84a-6468f08e61c2 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:33:54.029235 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-7811fd26-f5a6-4650-a84a-6468f08e61c2 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] [instance: 8b77de54-4c8b-4ed5-84f5-68bf69f9809f] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:33:54.038091 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7811fd26-f5a6-4650-a84a-6468f08e61c2 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:54.038817 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7811fd26-f5a6-4650-a84a-6468f08e61c2 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:54.040388 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7811fd26-f5a6-4650-a84a-6468f08e61c2 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:54.041302 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7811fd26-f5a6-4650-a84a-6468f08e61c2 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:54.260578 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-7811fd26-f5a6-4650-a84a-6468f08e61c2 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:33:54.270013 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-7811fd26-f5a6-4650-a84a-6468f08e61c2 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:33:54.271828 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-7811fd26-f5a6-4650-a84a-6468f08e61c2 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:33:54.273466 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-7811fd26-f5a6-4650-a84a-6468f08e61c2 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] 10.222.0.16 "GET /compute/v2.1/servers/8b77de54-4c8b-4ed5-84f5-68bf69f9809f" status: 200 len: 1730 microversion: 2.20 time: 0.253217 Jul 16 21:33:54.285035 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2428/4845] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:33:54 2022] GET /compute/v2.1/servers/8b77de54-4c8b-4ed5-84f5-68bf69f9809f => generated 1730 bytes in 265 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:33:54.857058 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-31acb161-8343-4342-8903-eafe7de73caa tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:33:54.857963 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-31acb161-8343-4342-8903-eafe7de73caa tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: c77c9a95-ea71-40d7-b7a7-ded612d194da] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:33:54.874136 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-31acb161-8343-4342-8903-eafe7de73caa tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:54.874900 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-31acb161-8343-4342-8903-eafe7de73caa tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:54.876031 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-31acb161-8343-4342-8903-eafe7de73caa tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:54.877330 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-31acb161-8343-4342-8903-eafe7de73caa tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:54.922633 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-31acb161-8343-4342-8903-eafe7de73caa tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] HTTP exception thrown: Instance c77c9a95-ea71-40d7-b7a7-ded612d194da could not be found. Jul 16 21:33:54.927041 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-31acb161-8343-4342-8903-eafe7de73caa tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Request method failure captured: Jul 16 21:33:54.927041 ncdv-master-1 devstack@n-api.service[138680]: request: GET /compute/v2.1/servers/c77c9a95-ea71-40d7-b7a7-ded612d194da HTTP/1.1 Jul 16 21:33:54.927041 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:33:54.927041 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:33:54.927041 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:33:54.927041 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 0 Jul 16 21:33:54.927041 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:33:54.927041 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:33:54.927041 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:33:54.927041 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:33:54.927041 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0y6QJDVdPNcLbUnwwgOJV-QkddLQ7yqnzBNVp2CrOe5UQqjQOER3spl3PswEW82ISSx8FhmuIrEwFSG69Ze1pTOmWBWZ8PURIgVHbN3yBPA3U2S2TKkQkFwyNuvvV03MCr9U0QGJIjXVOkiqFEAy8q87U_okZgCdI-BFjBFgrCdfAEo Jul 16 21:33:54.927041 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:33:54.927041 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:33:54.927041 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:33:54.927041 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:33:54.927041 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:33:54.927041 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:33:54.927041 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: da2248fa190342f0bfe9211cae1e403b Jul 16 21:33:54.927041 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-ServersNegativeTestJSON-1102403437 Jul 16 21:33:54.927041 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:33:54.927041 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:33:54.927041 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/da2248fa190342f0bfe9211cae1e403b"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/da2248fa190342f0bfe9211cae1e403b"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_da2248fa190342f0bfe9211cae1e403b"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/da2248fa190342f0bfe9211cae1e403b"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:33:54.927041 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-ServersNegativeTestJSON-1102403437 Jul 16 21:33:54.927041 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: da2248fa190342f0bfe9211cae1e403b Jul 16 21:33:54.927041 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-ServersNegativeTestJSON-1102403437 Jul 16 21:33:54.927041 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-ServersNegativeTestJSON-1102403437-project Jul 16 21:33:54.927041 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:33:54.927041 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:33:54.927041 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: 22e264440fff43cdb7be6702839af37f Jul 16 21:33:54.927041 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-ServersNegativeTestJSON-1102403437-project Jul 16 21:33:54.927041 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 21:33:54.927041 ncdv-master-1 devstack@n-api.service[138680]: exception: Instance c77c9a95-ea71-40d7-b7a7-ded612d194da could not be found. Jul 16 21:33:54.927041 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': 'c77c9a95-ea71-40d7-b7a7-ded612d194da'} Jul 16 21:33:54.927041 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:33:54.927041 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:33:54.927041 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:33:54.927041 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:33:54.927041 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:33:54.927041 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:33:54.927041 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:33:54.927041 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:33:54.927041 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:33:54.927041 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:33:54.927041 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance c77c9a95-ea71-40d7-b7a7-ded612d194da could not be found. Jul 16 21:33:54.927041 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:33:54.927041 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:33:54.927041 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:33:54.927041 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:33:54.927041 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:33:54.927041 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:33:54.927041 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:33:54.927041 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:33:54.927041 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:33:54.927041 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:33:54.927041 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:33:54.927041 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:33:54.927041 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:33:54.927041 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:33:54.927041 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:33:54.927041 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:33:54.927041 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance c77c9a95-ea71-40d7-b7a7-ded612d194da could not be found. Jul 16 21:33:54.927041 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:33:54.927041 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:33:54.927041 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:33:54.932749 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:33:54.932749 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:33:54.932749 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:33:54.932749 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:33:54.932749 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:33:54.932749 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance c77c9a95-ea71-40d7-b7a7-ded612d194da could not be found. Jul 16 21:33:54.932749 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:33:54.937006 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-31acb161-8343-4342-8903-eafe7de73caa tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Returning 404 to user: Instance c77c9a95-ea71-40d7-b7a7-ded612d194da could not be found. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:33:54.941018 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-31acb161-8343-4342-8903-eafe7de73caa tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "GET /compute/v2.1/servers/c77c9a95-ea71-40d7-b7a7-ded612d194da" status: 404 len: 111 microversion: 2.1 time: 0.088328 Jul 16 21:33:54.944281 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2418/4846] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:33:54 2022] GET /compute/v2.1/servers/c77c9a95-ea71-40d7-b7a7-ded612d194da => generated 111 bytes in 93 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:33:54.952096 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-a5e08535-f44f-4402-82d5-522a01003259 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:33:54.953078 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-a5e08535-f44f-4402-82d5-522a01003259 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: e4c7a9cb-e55d-4819-9885-2e72deffddb1] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:33:54.972791 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a5e08535-f44f-4402-82d5-522a01003259 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:54.973750 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a5e08535-f44f-4402-82d5-522a01003259 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:54.974681 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a5e08535-f44f-4402-82d5-522a01003259 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:54.975336 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a5e08535-f44f-4402-82d5-522a01003259 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:55.294848 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-8a36d79d-b3b0-4276-ac38-ce40d1416975 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:33:55.295493 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-8a36d79d-b3b0-4276-ac38-ce40d1416975 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] [instance: 8b77de54-4c8b-4ed5-84f5-68bf69f9809f] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:33:55.306830 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8a36d79d-b3b0-4276-ac38-ce40d1416975 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:55.307957 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8a36d79d-b3b0-4276-ac38-ce40d1416975 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:55.308119 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8a36d79d-b3b0-4276-ac38-ce40d1416975 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:55.308527 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8a36d79d-b3b0-4276-ac38-ce40d1416975 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:55.319266 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-a5e08535-f44f-4402-82d5-522a01003259 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '22e264440fff43cdb7be6702839af37f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da2248fa190342f0bfe9211cae1e403b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:33:55.339892 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-a5e08535-f44f-4402-82d5-522a01003259 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1" status: 200 len: 1504 microversion: 2.1 time: 0.391894 Jul 16 21:33:55.344854 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2429/4847] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:33:54 2022] GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1 => generated 1504 bytes in 398 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:33:55.448661 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-469aef28-ea1e-467c-b3f6-2b07e1cb5810 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Action: 'create', calling method: >, body: {"server": {"name": 12345, "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452", "networks": [{"uuid": "f14874ef-c1c1-4761-ab34-120b93719d08"}]}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:33:55.451862 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-469aef28-ea1e-467c-b3f6-2b07e1cb5810 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Request method failure captured: Jul 16 21:33:55.451862 ncdv-master-1 devstack@n-api.service[138681]: request: POST /compute/v2.1/servers HTTP/1.1 Jul 16 21:33:55.451862 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:33:55.451862 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:33:55.451862 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:33:55.451862 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 163 Jul 16 21:33:55.451862 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:33:55.451862 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:33:55.451862 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:33:55.451862 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:33:55.451862 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0y6QJDVdPNcLbUnwwgOJV-QkddLQ7yqnzBNVp2CrOe5UQqjQOER3spl3PswEW82ISSx8FhmuIrEwFSG69Ze1pTOmWBWZ8PURIgVHbN3yBPA3U2S2TKkQkFwyNuvvV03MCr9U0QGJIjXVOkiqFEAy8q87U_okZgCdI-BFjBFgrCdfAEo Jul 16 21:33:55.451862 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:33:55.451862 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:33:55.451862 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:33:55.451862 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:33:55.451862 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:33:55.451862 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:33:55.451862 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: da2248fa190342f0bfe9211cae1e403b Jul 16 21:33:55.451862 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-ServersNegativeTestJSON-1102403437 Jul 16 21:33:55.451862 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:33:55.451862 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:33:55.451862 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/da2248fa190342f0bfe9211cae1e403b"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/da2248fa190342f0bfe9211cae1e403b"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_da2248fa190342f0bfe9211cae1e403b"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/da2248fa190342f0bfe9211cae1e403b"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:33:55.451862 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-ServersNegativeTestJSON-1102403437 Jul 16 21:33:55.451862 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: da2248fa190342f0bfe9211cae1e403b Jul 16 21:33:55.451862 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-ServersNegativeTestJSON-1102403437 Jul 16 21:33:55.451862 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-ServersNegativeTestJSON-1102403437-project Jul 16 21:33:55.451862 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:33:55.451862 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:33:55.451862 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: 22e264440fff43cdb7be6702839af37f Jul 16 21:33:55.451862 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-ServersNegativeTestJSON-1102403437-project Jul 16 21:33:55.451862 ncdv-master-1 devstack@n-api.service[138681]: Jul 16 21:33:55.451862 ncdv-master-1 devstack@n-api.service[138681]: {"server": {"name": 12345, "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452", "networks": [{"uuid": "f14874ef-c1c1-4761-ab34-120b93719d08"}]}} Jul 16 21:33:55.451862 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:33:55.451862 ncdv-master-1 devstack@n-api.service[138681]: exception: Invalid input for field/attribute name. Value: 12345. 12345 is not of type 'string' Jul 16 21:33:55.451862 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'body': {'server': {'name': 12345, 'imageRef': '1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9', 'flavorRef': '452', 'networks': [{'uuid': 'f14874ef-c1c1-4761-ab34-120b93719d08'}]}}} Jul 16 21:33:55.451862 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:33:55.451862 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:33:55.451862 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 279, in validate Jul 16 21:33:55.451862 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Jul 16 21:33:55.451862 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/lib/python3/dist-packages/jsonschema/validators.py", line 353, in validate Jul 16 21:33:55.451862 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise error Jul 16 21:33:55.451862 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: 12345 is not of type 'string' Jul 16 21:33:55.451862 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:33:55.451862 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Failed validating 'type' in schema['properties']['server']['properties']['name']: Jul 16 21:33:55.451862 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi {'format': 'name', 'maxLength': 255, 'minLength': 1, 'type': 'string'} Jul 16 21:33:55.451862 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:33:55.451862 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi On instance['server']: Jul 16 21:33:55.451862 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi 12345 Jul 16 21:33:55.451862 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:33:55.451862 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:33:55.451862 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:33:55.451862 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:33:55.451862 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:33:55.451862 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:33:55.451862 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:33:55.451862 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:33:55.451862 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:33:55.451862 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:33:55.451862 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 16 21:33:55.451862 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 16 21:33:55.451862 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Jul 16 21:33:55.451862 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Jul 16 21:33:55.451862 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Jul 16 21:33:55.451862 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Jul 16 21:33:55.451862 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 308, in validate Jul 16 21:33:55.458354 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Jul 16 21:33:55.458354 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute name. Value: 12345. 12345 is not of type 'string' Jul 16 21:33:55.458354 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:33:55.458354 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:33:55.458354 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:33:55.458354 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:33:55.458354 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:33:55.458354 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:33:55.458354 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Jul 16 21:33:55.458354 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Jul 16 21:33:55.458354 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute name. Value: 12345. 12345 is not of type 'string' Jul 16 21:33:55.458354 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:33:55.458354 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-469aef28-ea1e-467c-b3f6-2b07e1cb5810 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Returning 400 to user: Invalid input for field/attribute name. Value: 12345. 12345 is not of type 'string' {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:33:55.458354 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-469aef28-ea1e-467c-b3f6-2b07e1cb5810 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 400 len: 127 microversion: 2.1 time: 0.008784 Jul 16 21:33:55.461431 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2430/4848] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 21:33:55 2022] POST /compute/v2.1/servers => generated 127 bytes in 15 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Jul 16 21:33:55.468270 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-f869cd6e-5424-4d83-9bf8-28e9f404a478 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:33:55.468888 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-f869cd6e-5424-4d83-9bf8-28e9f404a478 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: e4c7a9cb-e55d-4819-9885-2e72deffddb1] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:33:55.484151 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f869cd6e-5424-4d83-9bf8-28e9f404a478 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:55.484602 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f869cd6e-5424-4d83-9bf8-28e9f404a478 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:55.491041 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f869cd6e-5424-4d83-9bf8-28e9f404a478 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:55.491041 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f869cd6e-5424-4d83-9bf8-28e9f404a478 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:55.773188 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-8a36d79d-b3b0-4276-ac38-ce40d1416975 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:33:55.790999 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-8a36d79d-b3b0-4276-ac38-ce40d1416975 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:33:55.792597 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-8a36d79d-b3b0-4276-ac38-ce40d1416975 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:33:55.793941 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-8a36d79d-b3b0-4276-ac38-ce40d1416975 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] 10.222.0.16 "GET /compute/v2.1/servers/8b77de54-4c8b-4ed5-84f5-68bf69f9809f" status: 200 len: 1730 microversion: 2.20 time: 0.503152 Jul 16 21:33:55.794978 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2419/4849] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:33:55 2022] GET /compute/v2.1/servers/8b77de54-4c8b-4ed5-84f5-68bf69f9809f => generated 1730 bytes in 505 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:33:55.808383 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-f869cd6e-5424-4d83-9bf8-28e9f404a478 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '22e264440fff43cdb7be6702839af37f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da2248fa190342f0bfe9211cae1e403b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:33:55.829537 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-f869cd6e-5424-4d83-9bf8-28e9f404a478 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1" status: 200 len: 1504 microversion: 2.1 time: 0.366548 Jul 16 21:33:55.830173 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2431/4850] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:33:55 2022] GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1 => generated 1504 bytes in 368 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:33:55.856135 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-03619712-0deb-4a07-8ec0-e40c37be64ce tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:33:55.856844 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-03619712-0deb-4a07-8ec0-e40c37be64ce tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: e4c7a9cb-e55d-4819-9885-2e72deffddb1] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:33:55.876177 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-03619712-0deb-4a07-8ec0-e40c37be64ce tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:55.878623 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-03619712-0deb-4a07-8ec0-e40c37be64ce tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:55.880436 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-03619712-0deb-4a07-8ec0-e40c37be64ce tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:55.881547 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-03619712-0deb-4a07-8ec0-e40c37be64ce tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:56.212884 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-03619712-0deb-4a07-8ec0-e40c37be64ce tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '22e264440fff43cdb7be6702839af37f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da2248fa190342f0bfe9211cae1e403b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:33:56.227785 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-03619712-0deb-4a07-8ec0-e40c37be64ce tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1" status: 200 len: 1504 microversion: 2.1 time: 0.380265 Jul 16 21:33:56.229070 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2420/4851] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:33:55 2022] GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1 => generated 1504 bytes in 383 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:33:56.812831 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-213abd82-a829-4388-b1d0-66160b2253be tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:33:56.814103 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-213abd82-a829-4388-b1d0-66160b2253be tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] [instance: 8b77de54-4c8b-4ed5-84f5-68bf69f9809f] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:33:56.848217 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-213abd82-a829-4388-b1d0-66160b2253be tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:56.848841 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-213abd82-a829-4388-b1d0-66160b2253be tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:56.850944 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-213abd82-a829-4388-b1d0-66160b2253be tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:56.851587 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-213abd82-a829-4388-b1d0-66160b2253be tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:57.172113 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-213abd82-a829-4388-b1d0-66160b2253be tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:33:57.181413 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-213abd82-a829-4388-b1d0-66160b2253be tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:33:57.185954 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-213abd82-a829-4388-b1d0-66160b2253be tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:33:57.189372 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-213abd82-a829-4388-b1d0-66160b2253be tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] 10.222.0.16 "GET /compute/v2.1/servers/8b77de54-4c8b-4ed5-84f5-68bf69f9809f" status: 200 len: 1730 microversion: 2.20 time: 0.381163 Jul 16 21:33:57.196498 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2432/4852] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:33:56 2022] GET /compute/v2.1/servers/8b77de54-4c8b-4ed5-84f5-68bf69f9809f => generated 1730 bytes in 385 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:33:58.216356 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-0fd7ef85-0a88-419a-9020-2ba6ecece80f tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:33:58.273261 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-0fd7ef85-0a88-419a-9020-2ba6ecece80f tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] [instance: 8b77de54-4c8b-4ed5-84f5-68bf69f9809f] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:33:58.273261 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0fd7ef85-0a88-419a-9020-2ba6ecece80f tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:58.273261 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0fd7ef85-0a88-419a-9020-2ba6ecece80f tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:58.273261 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0fd7ef85-0a88-419a-9020-2ba6ecece80f tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:58.273261 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0fd7ef85-0a88-419a-9020-2ba6ecece80f tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:58.392495 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-0fd7ef85-0a88-419a-9020-2ba6ecece80f tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:33:58.402862 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-0fd7ef85-0a88-419a-9020-2ba6ecece80f tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:33:58.404669 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-0fd7ef85-0a88-419a-9020-2ba6ecece80f tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:33:58.405944 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-0fd7ef85-0a88-419a-9020-2ba6ecece80f tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] 10.222.0.16 "GET /compute/v2.1/servers/8b77de54-4c8b-4ed5-84f5-68bf69f9809f" status: 200 len: 1730 microversion: 2.20 time: 0.196650 Jul 16 21:33:58.406556 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2421/4853] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:33:58 2022] GET /compute/v2.1/servers/8b77de54-4c8b-4ed5-84f5-68bf69f9809f => generated 1730 bytes in 198 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:33:59.436855 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-5711752c-cffb-4ea6-9f55-dc7006c76a8e tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:33:59.437482 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-5711752c-cffb-4ea6-9f55-dc7006c76a8e tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] [instance: 8b77de54-4c8b-4ed5-84f5-68bf69f9809f] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:33:59.448000 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5711752c-cffb-4ea6-9f55-dc7006c76a8e tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:59.448589 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5711752c-cffb-4ea6-9f55-dc7006c76a8e tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:59.449404 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5711752c-cffb-4ea6-9f55-dc7006c76a8e tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:59.449857 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5711752c-cffb-4ea6-9f55-dc7006c76a8e tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:59.493705 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-a173af95-9237-42ff-8a25-8ec87b0e7e8e tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServersNegativeTestJSON-server-581436318", "imageRef": "", "flavorRef": "452", "block_device_mapping_v2": [{"boot_index": "0", "uuid": "75978d6a-3102-4e0e-bde6-0442de5a8d0a", "source_type": "volume", "destination_type": "volume", "delete_on_termination": false}], "networks": [{"uuid": "f14874ef-c1c1-4761-ab34-120b93719d08"}]}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:33:59.568836 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-a173af95-9237-42ff-8a25-8ec87b0e7e8e tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] HTTP exception thrown: Block Device 75978d6a-3102-4e0e-bde6-0442de5a8d0a is not bootable. Jul 16 21:33:59.583717 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-a173af95-9237-42ff-8a25-8ec87b0e7e8e tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Request method failure captured: Jul 16 21:33:59.583717 ncdv-master-1 devstack@n-api.service[138680]: request: POST /compute/v2.1/servers HTTP/1.1 Jul 16 21:33:59.583717 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:33:59.583717 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:33:59.583717 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:33:59.583717 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 357 Jul 16 21:33:59.583717 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:33:59.583717 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:33:59.583717 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:33:59.583717 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:33:59.583717 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0y6QJDVdPNcLbUnwwgOJV-QkddLQ7yqnzBNVp2CrOe5UQqjQOER3spl3PswEW82ISSx8FhmuIrEwFSG69Ze1pTOmWBWZ8PURIgVHbN3yBPA3U2S2TKkQkFwyNuvvV03MCr9U0QGJIjXVOkiqFEAy8q87U_okZgCdI-BFjBFgrCdfAEo Jul 16 21:33:59.583717 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:33:59.583717 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:33:59.583717 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:33:59.583717 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:33:59.583717 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:33:59.583717 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:33:59.583717 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: da2248fa190342f0bfe9211cae1e403b Jul 16 21:33:59.583717 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-ServersNegativeTestJSON-1102403437 Jul 16 21:33:59.583717 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:33:59.583717 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:33:59.583717 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/da2248fa190342f0bfe9211cae1e403b"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/da2248fa190342f0bfe9211cae1e403b"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_da2248fa190342f0bfe9211cae1e403b"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/da2248fa190342f0bfe9211cae1e403b"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:33:59.583717 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-ServersNegativeTestJSON-1102403437 Jul 16 21:33:59.583717 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: da2248fa190342f0bfe9211cae1e403b Jul 16 21:33:59.583717 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-ServersNegativeTestJSON-1102403437 Jul 16 21:33:59.583717 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-ServersNegativeTestJSON-1102403437-project Jul 16 21:33:59.583717 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:33:59.583717 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:33:59.583717 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: 22e264440fff43cdb7be6702839af37f Jul 16 21:33:59.583717 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-ServersNegativeTestJSON-1102403437-project Jul 16 21:33:59.583717 ncdv-master-1 devstack@n-api.service[138680]: Jul 16 21:33:59.583717 ncdv-master-1 devstack@n-api.service[138680]: {"server": {"name": "tempest-ServersNegativeTestJSON-server-581436318", "imageRef": "", "flavorRef": "452", "block_device_mapping_v2": [{"boot_index": "0", "uuid": "75978d6a-3102-4e0e-bde6-0442de5a8d0a", "source_type": "volume", "destination_type": "volume", "delete_on_termination": false}], "networks": [{"uuid": "f14874ef-c1c1-4761-ab34-120b93719d08"}]}} Jul 16 21:33:59.583717 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 21:33:59.583717 ncdv-master-1 devstack@n-api.service[138680]: exception: Block Device 75978d6a-3102-4e0e-bde6-0442de5a8d0a is not bootable. Jul 16 21:33:59.583717 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'body': {'server': {'name': 'tempest-ServersNegativeTestJSON-server-581436318', 'imageRef': '', 'flavorRef': '452', 'block_device_mapping_v2': [{'boot_index': 0, 'source_type': 'volume', 'destination_type': 'volume', 'delete_on_termination': False, 'volume_id': '75978d6a-3102-4e0e-bde6-0442de5a8d0a'}], 'networks': [{'uuid': 'f14874ef-c1c1-4761-ab34-120b93719d08'}]}}} Jul 16 21:33:59.583717 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:33:59.583717 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:33:59.583717 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 784, in create Jul 16 21:33:59.583717 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instances, resv_id = self.compute_api.create( Jul 16 21:33:59.583717 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2149, in create Jul 16 21:33:59.583717 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return self._create_instance( Jul 16 21:33:59.583717 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 1622, in _create_instance Jul 16 21:33:59.583717 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi boot_meta = block_device.get_bdm_image_metadata( Jul 16 21:33:59.583717 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/block_device.py", line 662, in get_bdm_image_metadata Jul 16 21:33:59.583717 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.InvalidBDMVolumeNotBootable(id=volume_id) Jul 16 21:33:59.583717 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.InvalidBDMVolumeNotBootable: Block Device 75978d6a-3102-4e0e-bde6-0442de5a8d0a is not bootable. Jul 16 21:33:59.583717 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:33:59.583717 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:33:59.583717 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:33:59.583717 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:33:59.583717 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:33:59.583717 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:33:59.583717 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:33:59.583717 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:33:59.583717 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:33:59.583717 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:33:59.583717 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 16 21:33:59.583717 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 16 21:33:59.583717 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 16 21:33:59.588603 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 16 21:33:59.588603 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 16 21:33:59.588603 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 16 21:33:59.588603 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi [Previous line repeated 10 more times] Jul 16 21:33:59.588603 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 866, in create Jul 16 21:33:59.588603 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=error.format_message()) Jul 16 21:33:59.588603 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Block Device 75978d6a-3102-4e0e-bde6-0442de5a8d0a is not bootable. Jul 16 21:33:59.588603 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:33:59.588603 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:33:59.588603 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:33:59.588603 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:33:59.588603 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:33:59.588603 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:33:59.588603 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:33:59.588603 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:33:59.588603 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Block Device 75978d6a-3102-4e0e-bde6-0442de5a8d0a is not bootable. Jul 16 21:33:59.588603 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:33:59.588603 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-a173af95-9237-42ff-8a25-8ec87b0e7e8e tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Returning 400 to user: Block Device 75978d6a-3102-4e0e-bde6-0442de5a8d0a is not bootable. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:33:59.588603 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-a173af95-9237-42ff-8a25-8ec87b0e7e8e tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 400 len: 110 microversion: 2.1 time: 0.095673 Jul 16 21:33:59.588603 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2422/4854] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 21:33:59 2022] POST /compute/v2.1/servers => generated 110 bytes in 97 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Jul 16 21:33:59.594393 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-53e398ef-e3d9-49a1-afd6-6a861815f036 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:33:59.595107 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-53e398ef-e3d9-49a1-afd6-6a861815f036 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: e4c7a9cb-e55d-4819-9885-2e72deffddb1] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:33:59.602164 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-53e398ef-e3d9-49a1-afd6-6a861815f036 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:59.602616 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-53e398ef-e3d9-49a1-afd6-6a861815f036 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:59.603658 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-53e398ef-e3d9-49a1-afd6-6a861815f036 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:59.604094 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-53e398ef-e3d9-49a1-afd6-6a861815f036 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:59.625186 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-5711752c-cffb-4ea6-9f55-dc7006c76a8e tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:33:59.633741 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-5711752c-cffb-4ea6-9f55-dc7006c76a8e tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:33:59.635181 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-5711752c-cffb-4ea6-9f55-dc7006c76a8e tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:33:59.636368 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-5711752c-cffb-4ea6-9f55-dc7006c76a8e tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] 10.222.0.16 "GET /compute/v2.1/servers/8b77de54-4c8b-4ed5-84f5-68bf69f9809f" status: 200 len: 1730 microversion: 2.20 time: 0.214900 Jul 16 21:33:59.636935 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2433/4855] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:33:59 2022] GET /compute/v2.1/servers/8b77de54-4c8b-4ed5-84f5-68bf69f9809f => generated 1730 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:33:59.842302 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-53e398ef-e3d9-49a1-afd6-6a861815f036 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '22e264440fff43cdb7be6702839af37f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da2248fa190342f0bfe9211cae1e403b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:33:59.851814 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-53e398ef-e3d9-49a1-afd6-6a861815f036 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1" status: 200 len: 1504 microversion: 2.1 time: 0.261048 Jul 16 21:33:59.867279 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2423/4856] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:33:59 2022] GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1 => generated 1504 bytes in 276 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:33:59.871153 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-75f37520-234e-4c43-92de-82a96401513c tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:33:59.871815 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-75f37520-234e-4c43-92de-82a96401513c tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: e4c7a9cb-e55d-4819-9885-2e72deffddb1] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:33:59.879301 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-75f37520-234e-4c43-92de-82a96401513c tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:59.879897 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-75f37520-234e-4c43-92de-82a96401513c tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:33:59.880798 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-75f37520-234e-4c43-92de-82a96401513c tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:33:59.881347 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-75f37520-234e-4c43-92de-82a96401513c tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:00.114993 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-75f37520-234e-4c43-92de-82a96401513c tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '22e264440fff43cdb7be6702839af37f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da2248fa190342f0bfe9211cae1e403b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:34:00.124790 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-75f37520-234e-4c43-92de-82a96401513c tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1" status: 200 len: 1504 microversion: 2.1 time: 0.257533 Jul 16 21:34:00.125423 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2434/4857] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:33:59 2022] GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1 => generated 1504 bytes in 259 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:34:00.655118 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-e5b595a1-8e7b-4e88-a295-c3c09371e22b tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:34:00.656385 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-e5b595a1-8e7b-4e88-a295-c3c09371e22b tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] [instance: 8b77de54-4c8b-4ed5-84f5-68bf69f9809f] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:34:00.664801 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e5b595a1-8e7b-4e88-a295-c3c09371e22b tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:00.665900 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e5b595a1-8e7b-4e88-a295-c3c09371e22b tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:00.666943 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e5b595a1-8e7b-4e88-a295-c3c09371e22b tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:00.667620 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e5b595a1-8e7b-4e88-a295-c3c09371e22b tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:01.114362 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-e5b595a1-8e7b-4e88-a295-c3c09371e22b tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:34:01.125610 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-e5b595a1-8e7b-4e88-a295-c3c09371e22b tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:34:01.134058 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-e5b595a1-8e7b-4e88-a295-c3c09371e22b tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:34:01.134058 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-e5b595a1-8e7b-4e88-a295-c3c09371e22b tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] 10.222.0.16 "GET /compute/v2.1/servers/8b77de54-4c8b-4ed5-84f5-68bf69f9809f" status: 200 len: 1730 microversion: 2.20 time: 0.481694 Jul 16 21:34:01.135661 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2424/4858] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:34:00 2022] GET /compute/v2.1/servers/8b77de54-4c8b-4ed5-84f5-68bf69f9809f => generated 1730 bytes in 485 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 16 21:34:02.163235 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-88cc50a9-83ae-4d89-8024-497b348fc243 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:34:02.164478 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-88cc50a9-83ae-4d89-8024-497b348fc243 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] [instance: 8b77de54-4c8b-4ed5-84f5-68bf69f9809f] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:34:02.172884 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-88cc50a9-83ae-4d89-8024-497b348fc243 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:02.174011 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-88cc50a9-83ae-4d89-8024-497b348fc243 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:02.175586 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-88cc50a9-83ae-4d89-8024-497b348fc243 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:02.176208 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-88cc50a9-83ae-4d89-8024-497b348fc243 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:02.498440 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-88cc50a9-83ae-4d89-8024-497b348fc243 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:34:02.512921 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-88cc50a9-83ae-4d89-8024-497b348fc243 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:34:02.515829 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-88cc50a9-83ae-4d89-8024-497b348fc243 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:34:02.517952 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-88cc50a9-83ae-4d89-8024-497b348fc243 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] 10.222.0.16 "GET /compute/v2.1/servers/8b77de54-4c8b-4ed5-84f5-68bf69f9809f" status: 200 len: 1730 microversion: 2.20 time: 0.360555 Jul 16 21:34:02.519930 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2435/4859] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:34:02 2022] GET /compute/v2.1/servers/8b77de54-4c8b-4ed5-84f5-68bf69f9809f => generated 1730 bytes in 363 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:34:03.287524 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-b717e6a3-75bd-4071-be90-f078433c1a13 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServersNegativeTestJSON-server-1076295641", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452", "block_device_mapping_v2": [{"source_type": "image", "delete_on_termination": true, "boot_index": 0, "uuid": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "destination_type": "local"}, {"source_type": "volume", "uuid": "6067192a-2bc9-4dce-ac75-d0ecdcbbbd48", "destination_type": "invalid"}], "networks": [{"uuid": "f14874ef-c1c1-4761-ab34-120b93719d08"}]}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:34:03.290603 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-b717e6a3-75bd-4071-be90-f078433c1a13 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Request method failure captured: Jul 16 21:34:03.290603 ncdv-master-1 devstack@n-api.service[138680]: request: POST /compute/v2.1/servers HTTP/1.1 Jul 16 21:34:03.290603 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:34:03.290603 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:34:03.290603 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:34:03.290603 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 495 Jul 16 21:34:03.290603 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:34:03.290603 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:34:03.290603 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:34:03.290603 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:34:03.290603 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0y6QJDVdPNcLbUnwwgOJV-QkddLQ7yqnzBNVp2CrOe5UQqjQOER3spl3PswEW82ISSx8FhmuIrEwFSG69Ze1pTOmWBWZ8PURIgVHbN3yBPA3U2S2TKkQkFwyNuvvV03MCr9U0QGJIjXVOkiqFEAy8q87U_okZgCdI-BFjBFgrCdfAEo Jul 16 21:34:03.290603 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:34:03.290603 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:34:03.290603 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:34:03.290603 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:34:03.290603 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:34:03.290603 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:34:03.290603 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: da2248fa190342f0bfe9211cae1e403b Jul 16 21:34:03.290603 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-ServersNegativeTestJSON-1102403437 Jul 16 21:34:03.290603 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:34:03.290603 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:34:03.290603 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/da2248fa190342f0bfe9211cae1e403b"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/da2248fa190342f0bfe9211cae1e403b"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_da2248fa190342f0bfe9211cae1e403b"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/da2248fa190342f0bfe9211cae1e403b"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:34:03.290603 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-ServersNegativeTestJSON-1102403437 Jul 16 21:34:03.290603 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: da2248fa190342f0bfe9211cae1e403b Jul 16 21:34:03.290603 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-ServersNegativeTestJSON-1102403437 Jul 16 21:34:03.290603 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-ServersNegativeTestJSON-1102403437-project Jul 16 21:34:03.290603 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:34:03.290603 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:34:03.290603 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: 22e264440fff43cdb7be6702839af37f Jul 16 21:34:03.290603 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-ServersNegativeTestJSON-1102403437-project Jul 16 21:34:03.290603 ncdv-master-1 devstack@n-api.service[138680]: Jul 16 21:34:03.290603 ncdv-master-1 devstack@n-api.service[138680]: {"server": {"name": "tempest-ServersNegativeTestJSON-server-1076295641", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452", "block_device_mapping_v2": [{"source_type": "image", "delete_on_termination": true, "boot_index": 0, "uuid": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "destination_type": "local"}, {"source_type": "volume", "uuid": "6067192a-2bc9-4dce-ac75-d0ecdcbbbd48", "destination_type": "invalid"}], "networks": [{"uuid": "f14874ef-c1c1-4761-ab34-120b93719d08"}]}} Jul 16 21:34:03.290603 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 21:34:03.290603 ncdv-master-1 devstack@n-api.service[138680]: exception: Invalid input for field/attribute destination_type. Value: invalid. 'invalid' is not one of ['local', 'volume'] Jul 16 21:34:03.290603 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'body': {'server': {'name': 'tempest-ServersNegativeTestJSON-server-1076295641', 'imageRef': '1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9', 'flavorRef': '452', 'block_device_mapping_v2': [{'source_type': 'image', 'delete_on_termination': True, 'boot_index': 0, 'uuid': '1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9', 'destination_type': 'local'}, {'source_type': 'volume', 'uuid': '6067192a-2bc9-4dce-ac75-d0ecdcbbbd48', 'destination_type': 'invalid'}], 'networks': [{'uuid': 'f14874ef-c1c1-4761-ab34-120b93719d08'}]}}} Jul 16 21:34:03.290603 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:34:03.290603 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:34:03.290603 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 279, in validate Jul 16 21:34:03.290603 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Jul 16 21:34:03.290603 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/lib/python3/dist-packages/jsonschema/validators.py", line 353, in validate Jul 16 21:34:03.290603 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise error Jul 16 21:34:03.290603 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: 'invalid' is not one of ['local', 'volume'] Jul 16 21:34:03.290603 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:34:03.290603 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Failed validating 'enum' in schema['properties']['server']['properties']['block_device_mapping_v2']['items']['properties']['destination_type']: Jul 16 21:34:03.290603 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi {'enum': ['local', 'volume'], 'type': 'string'} Jul 16 21:34:03.290603 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:34:03.290603 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi On instance['server']['block_device_mapping_v2'][1]: Jul 16 21:34:03.290603 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi 'invalid' Jul 16 21:34:03.290603 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:34:03.290603 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:34:03.290603 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:34:03.290603 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:34:03.290603 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:34:03.290603 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:34:03.290603 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:34:03.290603 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:34:03.290603 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:34:03.290603 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:34:03.298299 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 16 21:34:03.298299 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 16 21:34:03.298299 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Jul 16 21:34:03.298299 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Jul 16 21:34:03.298299 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Jul 16 21:34:03.298299 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Jul 16 21:34:03.298299 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 308, in validate Jul 16 21:34:03.298299 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Jul 16 21:34:03.298299 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute destination_type. Value: invalid. 'invalid' is not one of ['local', 'volume'] Jul 16 21:34:03.298299 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:34:03.298299 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:34:03.298299 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:34:03.298299 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:34:03.298299 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:34:03.298299 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:34:03.298299 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Jul 16 21:34:03.298299 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Jul 16 21:34:03.298299 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute destination_type. Value: invalid. 'invalid' is not one of ['local', 'volume'] Jul 16 21:34:03.298299 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:34:03.298299 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-b717e6a3-75bd-4071-be90-f078433c1a13 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Returning 400 to user: Invalid input for field/attribute destination_type. Value: invalid. 'invalid' is not one of ['local', 'volume'] {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:34:03.298299 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-b717e6a3-75bd-4071-be90-f078433c1a13 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 400 len: 155 microversion: 2.1 time: 0.008896 Jul 16 21:34:03.301198 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-2f4a1e48-c131-491f-a994-2645a03111e0 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:34:03.302247 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-2f4a1e48-c131-491f-a994-2645a03111e0 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: e4c7a9cb-e55d-4819-9885-2e72deffddb1] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:34:03.303105 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2425/4860] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 21:34:03 2022] POST /compute/v2.1/servers => generated 155 bytes in 12 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Jul 16 21:34:03.311716 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2f4a1e48-c131-491f-a994-2645a03111e0 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:03.312178 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2f4a1e48-c131-491f-a994-2645a03111e0 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:03.312982 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2f4a1e48-c131-491f-a994-2645a03111e0 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:03.313435 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2f4a1e48-c131-491f-a994-2645a03111e0 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:03.540847 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-3dab9208-d64a-4367-a752-e51895635289 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:34:03.541643 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-3dab9208-d64a-4367-a752-e51895635289 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] [instance: 8b77de54-4c8b-4ed5-84f5-68bf69f9809f] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:34:03.564006 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3dab9208-d64a-4367-a752-e51895635289 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:03.569759 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3dab9208-d64a-4367-a752-e51895635289 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:03.570668 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3dab9208-d64a-4367-a752-e51895635289 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:03.571714 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3dab9208-d64a-4367-a752-e51895635289 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:03.776512 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-2f4a1e48-c131-491f-a994-2645a03111e0 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '22e264440fff43cdb7be6702839af37f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da2248fa190342f0bfe9211cae1e403b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:34:03.787316 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-2f4a1e48-c131-491f-a994-2645a03111e0 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1" status: 200 len: 1504 microversion: 2.1 time: 0.489953 Jul 16 21:34:03.788619 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2436/4861] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:34:03 2022] GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1 => generated 1504 bytes in 492 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:34:03.806109 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-dda9c67d-e74c-415b-bd80-5eeab5cdc713 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:34:03.806727 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-dda9c67d-e74c-415b-bd80-5eeab5cdc713 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: e4c7a9cb-e55d-4819-9885-2e72deffddb1] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:34:03.813345 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-dda9c67d-e74c-415b-bd80-5eeab5cdc713 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:03.814003 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-dda9c67d-e74c-415b-bd80-5eeab5cdc713 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:03.814827 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-dda9c67d-e74c-415b-bd80-5eeab5cdc713 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:03.815376 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-dda9c67d-e74c-415b-bd80-5eeab5cdc713 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:04.051858 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-3dab9208-d64a-4367-a752-e51895635289 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:34:04.067897 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-3dab9208-d64a-4367-a752-e51895635289 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:34:04.069604 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-3dab9208-d64a-4367-a752-e51895635289 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:34:04.075206 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-3dab9208-d64a-4367-a752-e51895635289 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] 10.222.0.16 "GET /compute/v2.1/servers/8b77de54-4c8b-4ed5-84f5-68bf69f9809f" status: 200 len: 1730 microversion: 2.20 time: 0.539919 Jul 16 21:34:04.086861 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2426/4862] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:34:03 2022] GET /compute/v2.1/servers/8b77de54-4c8b-4ed5-84f5-68bf69f9809f => generated 1730 bytes in 545 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:34:04.255217 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-dda9c67d-e74c-415b-bd80-5eeab5cdc713 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '22e264440fff43cdb7be6702839af37f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da2248fa190342f0bfe9211cae1e403b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:34:04.267147 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-dda9c67d-e74c-415b-bd80-5eeab5cdc713 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1" status: 200 len: 1504 microversion: 2.1 time: 0.465011 Jul 16 21:34:04.271746 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2437/4863] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:34:03 2022] GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1 => generated 1504 bytes in 470 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:34:04.410269 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-9e98cb8a-c0e4-4c82-a68b-ff0f987976fc tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServersNegativeTestJSON-server-1121808579", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452", "metadata": {"a": "bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb"}, "networks": [{"uuid": "f14874ef-c1c1-4761-ab34-120b93719d08"}]}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:34:04.418582 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-9e98cb8a-c0e4-4c82-a68b-ff0f987976fc tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Request method failure captured: Jul 16 21:34:04.418582 ncdv-master-1 devstack@n-api.service[138680]: request: POST /compute/v2.1/servers HTTP/1.1 Jul 16 21:34:04.418582 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:34:04.418582 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:34:04.418582 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:34:04.418582 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 492 Jul 16 21:34:04.418582 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:34:04.418582 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:34:04.418582 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:34:04.418582 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:34:04.418582 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0y6QJDVdPNcLbUnwwgOJV-QkddLQ7yqnzBNVp2CrOe5UQqjQOER3spl3PswEW82ISSx8FhmuIrEwFSG69Ze1pTOmWBWZ8PURIgVHbN3yBPA3U2S2TKkQkFwyNuvvV03MCr9U0QGJIjXVOkiqFEAy8q87U_okZgCdI-BFjBFgrCdfAEo Jul 16 21:34:04.418582 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:34:04.418582 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:34:04.418582 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:34:04.418582 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:34:04.418582 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:34:04.418582 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:34:04.418582 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: da2248fa190342f0bfe9211cae1e403b Jul 16 21:34:04.418582 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-ServersNegativeTestJSON-1102403437 Jul 16 21:34:04.418582 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:34:04.418582 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:34:04.418582 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/da2248fa190342f0bfe9211cae1e403b"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/da2248fa190342f0bfe9211cae1e403b"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_da2248fa190342f0bfe9211cae1e403b"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/da2248fa190342f0bfe9211cae1e403b"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:34:04.418582 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-ServersNegativeTestJSON-1102403437 Jul 16 21:34:04.418582 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: da2248fa190342f0bfe9211cae1e403b Jul 16 21:34:04.418582 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-ServersNegativeTestJSON-1102403437 Jul 16 21:34:04.418582 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-ServersNegativeTestJSON-1102403437-project Jul 16 21:34:04.418582 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:34:04.418582 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:34:04.418582 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: 22e264440fff43cdb7be6702839af37f Jul 16 21:34:04.418582 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-ServersNegativeTestJSON-1102403437-project Jul 16 21:34:04.418582 ncdv-master-1 devstack@n-api.service[138680]: Jul 16 21:34:04.418582 ncdv-master-1 devstack@n-api.service[138680]: {"server": {"name": "tempest-ServersNegativeTestJSON-server-1121808579", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452", "metadata": {"a": "bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb"}, "networks": [{"uuid": "f14874ef-c1c1-4761-ab34-120b93719d08"}]}} Jul 16 21:34:04.418582 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 21:34:04.418582 ncdv-master-1 devstack@n-api.service[138680]: exception: Invalid input for field/attribute a. Value: bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb. 'bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb' is too long Jul 16 21:34:04.418582 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'body': {'server': {'name': 'tempest-ServersNegativeTestJSON-server-1121808579', 'imageRef': '1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9', 'flavorRef': '452', 'metadata': {'a': 'bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb'}, 'networks': [{'uuid': 'f14874ef-c1c1-4761-ab34-120b93719d08'}]}}} Jul 16 21:34:04.418582 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:34:04.418582 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:34:04.418582 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 279, in validate Jul 16 21:34:04.418582 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Jul 16 21:34:04.418582 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/lib/python3/dist-packages/jsonschema/validators.py", line 353, in validate Jul 16 21:34:04.418582 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise error Jul 16 21:34:04.418582 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: 'bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb' is too long Jul 16 21:34:04.418582 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:34:04.418582 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Failed validating 'maxLength' in schema['properties']['server']['properties']['metadata']['patternProperties']['^[a-zA-Z0-9-_:. ]{1,255}$']: Jul 16 21:34:04.418582 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi {'maxLength': 255, 'type': 'string'} Jul 16 21:34:04.418582 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:34:04.418582 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi On instance['server']['metadata']: Jul 16 21:34:04.418582 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi 'bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb' Jul 16 21:34:04.441707 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:34:04.441707 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:34:04.441707 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:34:04.441707 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:34:04.441707 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:34:04.441707 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:34:04.441707 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:34:04.441707 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:34:04.441707 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:34:04.441707 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:34:04.441707 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 16 21:34:04.441707 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 16 21:34:04.441707 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Jul 16 21:34:04.441707 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Jul 16 21:34:04.441707 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Jul 16 21:34:04.441707 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Jul 16 21:34:04.441707 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 308, in validate Jul 16 21:34:04.441707 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Jul 16 21:34:04.441707 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute a. Value: bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb. 'bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb' is too long Jul 16 21:34:04.441707 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:34:04.441707 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:34:04.441707 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:34:04.441707 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:34:04.441707 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:34:04.441707 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:34:04.441707 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Jul 16 21:34:04.441707 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Jul 16 21:34:04.441707 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute a. Value: bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb. 'bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb' is too long Jul 16 21:34:04.441707 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:34:04.441707 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-9e98cb8a-c0e4-4c82-a68b-ff0f987976fc tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Returning 400 to user: Invalid input for field/attribute a. Value: bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb. 'bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb' is too long {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:34:04.441707 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-9e98cb8a-c0e4-4c82-a68b-ff0f987976fc tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 400 len: 624 microversion: 2.1 time: 0.015541 Jul 16 21:34:04.441707 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2427/4864] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 21:34:04 2022] POST /compute/v2.1/servers => generated 624 bytes in 19 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Jul 16 21:34:04.448455 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-501a7a40-4ac7-44be-9c35-62e8ab4ae18f tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:34:04.449124 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-501a7a40-4ac7-44be-9c35-62e8ab4ae18f tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: e4c7a9cb-e55d-4819-9885-2e72deffddb1] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:34:04.461621 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-501a7a40-4ac7-44be-9c35-62e8ab4ae18f tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:04.462088 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-501a7a40-4ac7-44be-9c35-62e8ab4ae18f tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:04.462899 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-501a7a40-4ac7-44be-9c35-62e8ab4ae18f tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:04.463448 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-501a7a40-4ac7-44be-9c35-62e8ab4ae18f tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:05.037401 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-501a7a40-4ac7-44be-9c35-62e8ab4ae18f tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '22e264440fff43cdb7be6702839af37f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da2248fa190342f0bfe9211cae1e403b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:34:05.047122 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-501a7a40-4ac7-44be-9c35-62e8ab4ae18f tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1" status: 200 len: 1504 microversion: 2.1 time: 0.608969 Jul 16 21:34:05.047742 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2438/4865] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:34:04 2022] GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1 => generated 1504 bytes in 610 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:34:05.075803 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-51bfa8fe-0b67-4ba2-b559-5f5d483a2e34 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:34:05.077033 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-51bfa8fe-0b67-4ba2-b559-5f5d483a2e34 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: e4c7a9cb-e55d-4819-9885-2e72deffddb1] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:34:05.095451 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-0cac7676-dd24-45e5-bb4f-a2140be26fc2 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:34:05.100462 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-0cac7676-dd24-45e5-bb4f-a2140be26fc2 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] [instance: 8b77de54-4c8b-4ed5-84f5-68bf69f9809f] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:34:05.116208 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-51bfa8fe-0b67-4ba2-b559-5f5d483a2e34 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:05.117678 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-51bfa8fe-0b67-4ba2-b559-5f5d483a2e34 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:05.118832 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0cac7676-dd24-45e5-bb4f-a2140be26fc2 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:05.120561 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-51bfa8fe-0b67-4ba2-b559-5f5d483a2e34 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:05.122252 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0cac7676-dd24-45e5-bb4f-a2140be26fc2 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:05.123313 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0cac7676-dd24-45e5-bb4f-a2140be26fc2 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:05.123516 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-51bfa8fe-0b67-4ba2-b559-5f5d483a2e34 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:05.129055 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0cac7676-dd24-45e5-bb4f-a2140be26fc2 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:05.556011 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-51bfa8fe-0b67-4ba2-b559-5f5d483a2e34 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '22e264440fff43cdb7be6702839af37f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da2248fa190342f0bfe9211cae1e403b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:34:05.574124 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-0cac7676-dd24-45e5-bb4f-a2140be26fc2 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:34:05.584041 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-0cac7676-dd24-45e5-bb4f-a2140be26fc2 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:34:05.585886 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-0cac7676-dd24-45e5-bb4f-a2140be26fc2 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:34:05.587518 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-0cac7676-dd24-45e5-bb4f-a2140be26fc2 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] 10.222.0.16 "GET /compute/v2.1/servers/8b77de54-4c8b-4ed5-84f5-68bf69f9809f" status: 200 len: 1730 microversion: 2.20 time: 0.497719 Jul 16 21:34:05.588496 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2439/4866] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:34:05 2022] GET /compute/v2.1/servers/8b77de54-4c8b-4ed5-84f5-68bf69f9809f => generated 1730 bytes in 499 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 16 21:34:05.599653 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-51bfa8fe-0b67-4ba2-b559-5f5d483a2e34 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1" status: 200 len: 1504 microversion: 2.1 time: 0.534771 Jul 16 21:34:05.600342 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2428/4867] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:34:05 2022] GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1 => generated 1504 bytes in 536 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:34:05.670911 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-193fd3b2-5c6e-43cc-b2ce-2771d385ee68 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Action: 'create', calling method: >, body: {"server": {"name": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452", "networks": [{"uuid": "f14874ef-c1c1-4761-ab34-120b93719d08"}]}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:34:05.673392 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-193fd3b2-5c6e-43cc-b2ce-2771d385ee68 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Request method failure captured: Jul 16 21:34:05.673392 ncdv-master-1 devstack@n-api.service[138681]: request: POST /compute/v2.1/servers HTTP/1.1 Jul 16 21:34:05.673392 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:34:05.673392 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:34:05.673392 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:34:05.673392 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 416 Jul 16 21:34:05.673392 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:34:05.673392 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:34:05.673392 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:34:05.673392 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:34:05.673392 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0y6QJDVdPNcLbUnwwgOJV-QkddLQ7yqnzBNVp2CrOe5UQqjQOER3spl3PswEW82ISSx8FhmuIrEwFSG69Ze1pTOmWBWZ8PURIgVHbN3yBPA3U2S2TKkQkFwyNuvvV03MCr9U0QGJIjXVOkiqFEAy8q87U_okZgCdI-BFjBFgrCdfAEo Jul 16 21:34:05.673392 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:34:05.673392 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:34:05.673392 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:34:05.673392 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:34:05.673392 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:34:05.673392 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:34:05.673392 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: da2248fa190342f0bfe9211cae1e403b Jul 16 21:34:05.673392 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-ServersNegativeTestJSON-1102403437 Jul 16 21:34:05.673392 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:34:05.673392 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:34:05.673392 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/da2248fa190342f0bfe9211cae1e403b"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/da2248fa190342f0bfe9211cae1e403b"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_da2248fa190342f0bfe9211cae1e403b"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/da2248fa190342f0bfe9211cae1e403b"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:34:05.673392 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-ServersNegativeTestJSON-1102403437 Jul 16 21:34:05.673392 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: da2248fa190342f0bfe9211cae1e403b Jul 16 21:34:05.673392 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-ServersNegativeTestJSON-1102403437 Jul 16 21:34:05.673392 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-ServersNegativeTestJSON-1102403437-project Jul 16 21:34:05.673392 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:34:05.673392 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:34:05.673392 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: 22e264440fff43cdb7be6702839af37f Jul 16 21:34:05.673392 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-ServersNegativeTestJSON-1102403437-project Jul 16 21:34:05.673392 ncdv-master-1 devstack@n-api.service[138681]: Jul 16 21:34:05.673392 ncdv-master-1 devstack@n-api.service[138681]: {"server": {"name": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452", "networks": [{"uuid": "f14874ef-c1c1-4761-ab34-120b93719d08"}]}} Jul 16 21:34:05.673392 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:34:05.673392 ncdv-master-1 devstack@n-api.service[138681]: exception: Invalid input for field/attribute name. Value: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa. 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' is too long Jul 16 21:34:05.673392 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'body': {'server': {'name': 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa', 'imageRef': '1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9', 'flavorRef': '452', 'networks': [{'uuid': 'f14874ef-c1c1-4761-ab34-120b93719d08'}]}}} Jul 16 21:34:05.673392 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:34:05.673392 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:34:05.673392 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 279, in validate Jul 16 21:34:05.673392 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Jul 16 21:34:05.673392 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/lib/python3/dist-packages/jsonschema/validators.py", line 353, in validate Jul 16 21:34:05.673392 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise error Jul 16 21:34:05.673392 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' is too long Jul 16 21:34:05.673392 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:34:05.673392 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Failed validating 'maxLength' in schema['properties']['server']['properties']['name']: Jul 16 21:34:05.673392 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi {'format': 'name', 'maxLength': 255, 'minLength': 1, 'type': 'string'} Jul 16 21:34:05.673392 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:34:05.673392 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi On instance['server']: Jul 16 21:34:05.673392 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' Jul 16 21:34:05.673392 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:34:05.673392 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:34:05.673392 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:34:05.682111 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:34:05.682111 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:34:05.682111 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:34:05.682111 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:34:05.682111 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:34:05.682111 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:34:05.682111 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:34:05.682111 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 16 21:34:05.682111 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 16 21:34:05.682111 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Jul 16 21:34:05.682111 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Jul 16 21:34:05.682111 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Jul 16 21:34:05.682111 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Jul 16 21:34:05.682111 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 308, in validate Jul 16 21:34:05.682111 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Jul 16 21:34:05.682111 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute name. Value: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa. 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' is too long Jul 16 21:34:05.682111 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:34:05.682111 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:34:05.682111 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:34:05.682111 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:34:05.682111 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:34:05.682111 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:34:05.682111 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Jul 16 21:34:05.682111 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Jul 16 21:34:05.682111 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute name. Value: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa. 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' is too long Jul 16 21:34:05.682111 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:34:05.682111 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-193fd3b2-5c6e-43cc-b2ce-2771d385ee68 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Returning 400 to user: Invalid input for field/attribute name. Value: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa. 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' is too long {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:34:05.682111 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-193fd3b2-5c6e-43cc-b2ce-2771d385ee68 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 400 len: 619 microversion: 2.1 time: 0.008012 Jul 16 21:34:05.687158 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-81f48357-3aa7-4863-a4e4-f4a45a1a236b tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:34:05.687158 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-81f48357-3aa7-4863-a4e4-f4a45a1a236b tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: e4c7a9cb-e55d-4819-9885-2e72deffddb1] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:34:05.691247 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2440/4868] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 21:34:05 2022] POST /compute/v2.1/servers => generated 619 bytes in 25 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Jul 16 21:34:05.694896 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-81f48357-3aa7-4863-a4e4-f4a45a1a236b tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:05.695388 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-81f48357-3aa7-4863-a4e4-f4a45a1a236b tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:05.696246 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-81f48357-3aa7-4863-a4e4-f4a45a1a236b tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:05.696690 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-81f48357-3aa7-4863-a4e4-f4a45a1a236b tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:05.840225 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-49302925-58a4-42bb-9312-427fa7986425 req-29a9322a-c94a-4c57-af5d-7e199d422a5a service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "8b77de54-4c8b-4ed5-84f5-68bf69f9809f", "name": "network-vif-plugged", "status": "completed", "tag": "5310cc77-6413-4115-9f89-f9fab39070d1"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:34:05.852822 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-49302925-58a4-42bb-9312-427fa7986425 req-29a9322a-c94a-4c57-af5d-7e199d422a5a service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:05.853785 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-49302925-58a4-42bb-9312-427fa7986425 req-29a9322a-c94a-4c57-af5d-7e199d422a5a service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:05.872367 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-49302925-58a4-42bb-9312-427fa7986425 req-29a9322a-c94a-4c57-af5d-7e199d422a5a service nova] Creating event network-vif-plugged:5310cc77-6413-4115-9f89-f9fab39070d1 for instance 8b77de54-4c8b-4ed5-84f5-68bf69f9809f on nch2-master-1 Jul 16 21:34:05.883743 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-49302925-58a4-42bb-9312-427fa7986425 req-29a9322a-c94a-4c57-af5d-7e199d422a5a service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.045701 Jul 16 21:34:05.886287 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2441/4869] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:34:05 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 16 21:34:06.104679 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-81f48357-3aa7-4863-a4e4-f4a45a1a236b tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '22e264440fff43cdb7be6702839af37f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da2248fa190342f0bfe9211cae1e403b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:34:06.127514 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-81f48357-3aa7-4863-a4e4-f4a45a1a236b tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1" status: 200 len: 1504 microversion: 2.1 time: 0.435301 Jul 16 21:34:06.135266 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2429/4870] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:34:05 2022] GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1 => generated 1504 bytes in 451 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:34:06.160727 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-038ac9f3-8c4e-49f3-b484-2579073b8fdd tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:34:06.160727 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-038ac9f3-8c4e-49f3-b484-2579073b8fdd tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: e4c7a9cb-e55d-4819-9885-2e72deffddb1] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:34:06.176459 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-038ac9f3-8c4e-49f3-b484-2579073b8fdd tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:06.176925 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-038ac9f3-8c4e-49f3-b484-2579073b8fdd tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:06.178189 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-038ac9f3-8c4e-49f3-b484-2579073b8fdd tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:06.178579 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-038ac9f3-8c4e-49f3-b484-2579073b8fdd tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:06.226212 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=100 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:34:06.606782 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-9dc1b18e-ae0b-4429-8f6b-7e774c8daefa tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:34:06.608676 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-9dc1b18e-ae0b-4429-8f6b-7e774c8daefa tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] [instance: 8b77de54-4c8b-4ed5-84f5-68bf69f9809f] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:34:06.620340 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9dc1b18e-ae0b-4429-8f6b-7e774c8daefa tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:06.622950 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9dc1b18e-ae0b-4429-8f6b-7e774c8daefa tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:06.623997 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9dc1b18e-ae0b-4429-8f6b-7e774c8daefa tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:06.625588 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9dc1b18e-ae0b-4429-8f6b-7e774c8daefa tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:06.810288 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-038ac9f3-8c4e-49f3-b484-2579073b8fdd tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '22e264440fff43cdb7be6702839af37f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da2248fa190342f0bfe9211cae1e403b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:34:06.832469 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-038ac9f3-8c4e-49f3-b484-2579073b8fdd tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1" status: 200 len: 1504 microversion: 2.1 time: 0.686420 Jul 16 21:34:06.833051 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2442/4871] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:34:06 2022] GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1 => generated 1504 bytes in 689 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:34:07.035344 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-fa10d90a-184b-4cd9-86ba-7363b15fff53 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServersNegativeTestJSON-server-150105035", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": -1, "networks": [{"uuid": "f14874ef-c1c1-4761-ab34-120b93719d08"}]}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:34:07.052350 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-9dc1b18e-ae0b-4429-8f6b-7e774c8daefa tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:34:07.054325 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-fa10d90a-184b-4cd9-86ba-7363b15fff53 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] HTTP exception thrown: Flavor -1 could not be found. Jul 16 21:34:07.061641 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-9dc1b18e-ae0b-4429-8f6b-7e774c8daefa tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:34:07.063393 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-9dc1b18e-ae0b-4429-8f6b-7e774c8daefa tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:34:07.064688 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-9dc1b18e-ae0b-4429-8f6b-7e774c8daefa tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] 10.222.0.16 "GET /compute/v2.1/servers/8b77de54-4c8b-4ed5-84f5-68bf69f9809f" status: 200 len: 1730 microversion: 2.20 time: 0.462943 Jul 16 21:34:07.066265 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2430/4872] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:34:06 2022] GET /compute/v2.1/servers/8b77de54-4c8b-4ed5-84f5-68bf69f9809f => generated 1730 bytes in 465 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 16 21:34:07.073519 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-fa10d90a-184b-4cd9-86ba-7363b15fff53 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Request method failure captured: Jul 16 21:34:07.073519 ncdv-master-1 devstack@n-api.service[138681]: request: POST /compute/v2.1/servers HTTP/1.1 Jul 16 21:34:07.073519 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:34:07.073519 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:34:07.073519 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:34:07.073519 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 205 Jul 16 21:34:07.073519 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:34:07.073519 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:34:07.073519 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:34:07.073519 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:34:07.073519 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0y6QJDVdPNcLbUnwwgOJV-QkddLQ7yqnzBNVp2CrOe5UQqjQOER3spl3PswEW82ISSx8FhmuIrEwFSG69Ze1pTOmWBWZ8PURIgVHbN3yBPA3U2S2TKkQkFwyNuvvV03MCr9U0QGJIjXVOkiqFEAy8q87U_okZgCdI-BFjBFgrCdfAEo Jul 16 21:34:07.073519 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:34:07.073519 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:34:07.073519 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:34:07.073519 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:34:07.073519 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:34:07.073519 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:34:07.073519 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: da2248fa190342f0bfe9211cae1e403b Jul 16 21:34:07.073519 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-ServersNegativeTestJSON-1102403437 Jul 16 21:34:07.073519 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:34:07.073519 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:34:07.073519 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/da2248fa190342f0bfe9211cae1e403b"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/da2248fa190342f0bfe9211cae1e403b"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_da2248fa190342f0bfe9211cae1e403b"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/da2248fa190342f0bfe9211cae1e403b"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:34:07.073519 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-ServersNegativeTestJSON-1102403437 Jul 16 21:34:07.073519 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: da2248fa190342f0bfe9211cae1e403b Jul 16 21:34:07.073519 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-ServersNegativeTestJSON-1102403437 Jul 16 21:34:07.073519 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-ServersNegativeTestJSON-1102403437-project Jul 16 21:34:07.073519 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:34:07.073519 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:34:07.073519 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: 22e264440fff43cdb7be6702839af37f Jul 16 21:34:07.073519 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-ServersNegativeTestJSON-1102403437-project Jul 16 21:34:07.073519 ncdv-master-1 devstack@n-api.service[138681]: Jul 16 21:34:07.073519 ncdv-master-1 devstack@n-api.service[138681]: {"server": {"name": "tempest-ServersNegativeTestJSON-server-150105035", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": -1, "networks": [{"uuid": "f14874ef-c1c1-4761-ab34-120b93719d08"}]}} Jul 16 21:34:07.073519 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:34:07.073519 ncdv-master-1 devstack@n-api.service[138681]: exception: Flavor -1 could not be found. Jul 16 21:34:07.073519 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'body': {'server': {'name': 'tempest-ServersNegativeTestJSON-server-150105035', 'imageRef': '1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9', 'flavorRef': -1, 'networks': [{'uuid': 'f14874ef-c1c1-4761-ab34-120b93719d08'}]}}} Jul 16 21:34:07.073519 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:34:07.073519 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:34:07.073519 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 778, in create Jul 16 21:34:07.073519 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id( Jul 16 21:34:07.073519 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Jul 16 21:34:07.073519 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Jul 16 21:34:07.073519 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Jul 16 21:34:07.073519 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Jul 16 21:34:07.073519 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Jul 16 21:34:07.073519 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Jul 16 21:34:07.073519 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Jul 16 21:34:07.073519 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:34:07.073519 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Jul 16 21:34:07.073519 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Jul 16 21:34:07.073519 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor -1 could not be found. Jul 16 21:34:07.073519 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:34:07.073519 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:34:07.073519 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:34:07.073519 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:34:07.073519 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:34:07.073519 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:34:07.073519 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:34:07.073519 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:34:07.073519 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:34:07.073519 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:34:07.073519 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 16 21:34:07.073519 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 16 21:34:07.078170 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 16 21:34:07.078170 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 16 21:34:07.078170 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 16 21:34:07.078170 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 16 21:34:07.078170 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi [Previous line repeated 10 more times] Jul 16 21:34:07.078170 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 866, in create Jul 16 21:34:07.078170 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=error.format_message()) Jul 16 21:34:07.078170 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Flavor -1 could not be found. Jul 16 21:34:07.078170 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:34:07.078170 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:34:07.078170 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:34:07.078170 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:34:07.078170 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:34:07.078170 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:34:07.078170 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:34:07.078170 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:34:07.078170 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor -1 could not be found. Jul 16 21:34:07.078170 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:34:07.086335 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-fa10d90a-184b-4cd9-86ba-7363b15fff53 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Returning 400 to user: Flavor -1 could not be found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:34:07.087993 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-fa10d90a-184b-4cd9-86ba-7363b15fff53 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 400 len: 73 microversion: 2.1 time: 0.059302 Jul 16 21:34:07.089168 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2443/4873] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 21:34:07 2022] POST /compute/v2.1/servers => generated 73 bytes in 61 msecs (HTTP/1.1 400) 9 headers in 380 bytes (1 switches on core 0) Jul 16 21:34:07.102453 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-54141056-0b30-4c6e-b1ae-a8fd5e0a9381 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:34:07.103050 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-54141056-0b30-4c6e-b1ae-a8fd5e0a9381 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: e4c7a9cb-e55d-4819-9885-2e72deffddb1] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:34:07.121200 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-54141056-0b30-4c6e-b1ae-a8fd5e0a9381 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:07.121670 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-54141056-0b30-4c6e-b1ae-a8fd5e0a9381 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:07.122462 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-54141056-0b30-4c6e-b1ae-a8fd5e0a9381 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:07.122917 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-54141056-0b30-4c6e-b1ae-a8fd5e0a9381 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:07.544745 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-54141056-0b30-4c6e-b1ae-a8fd5e0a9381 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '22e264440fff43cdb7be6702839af37f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da2248fa190342f0bfe9211cae1e403b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:34:07.559742 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-54141056-0b30-4c6e-b1ae-a8fd5e0a9381 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1" status: 200 len: 1504 microversion: 2.1 time: 0.461125 Jul 16 21:34:07.560597 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2431/4874] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:34:07 2022] GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1 => generated 1504 bytes in 467 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:34:07.592095 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-895f3db2-6405-4988-91be-c9b181e2aa89 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:34:07.592095 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-895f3db2-6405-4988-91be-c9b181e2aa89 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: e4c7a9cb-e55d-4819-9885-2e72deffddb1] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:34:07.595878 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-895f3db2-6405-4988-91be-c9b181e2aa89 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:07.597594 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-895f3db2-6405-4988-91be-c9b181e2aa89 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:07.598625 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-895f3db2-6405-4988-91be-c9b181e2aa89 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:07.599440 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-895f3db2-6405-4988-91be-c9b181e2aa89 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:07.879567 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-895f3db2-6405-4988-91be-c9b181e2aa89 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '22e264440fff43cdb7be6702839af37f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da2248fa190342f0bfe9211cae1e403b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:34:07.891247 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-895f3db2-6405-4988-91be-c9b181e2aa89 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1" status: 200 len: 1504 microversion: 2.1 time: 0.315314 Jul 16 21:34:07.892532 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2444/4875] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:34:07 2022] GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1 => generated 1504 bytes in 317 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:34:08.056928 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-0fe168b4-20a2-4266-aec9-371ff87bd6d9 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServersNegativeTestJSON-server-303704336", "imageRef": -1, "flavorRef": "452", "networks": [{"uuid": "f14874ef-c1c1-4761-ab34-120b93719d08"}]}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:34:08.059757 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-0fe168b4-20a2-4266-aec9-371ff87bd6d9 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Request method failure captured: Jul 16 21:34:08.059757 ncdv-master-1 devstack@n-api.service[138680]: request: POST /compute/v2.1/servers HTTP/1.1 Jul 16 21:34:08.059757 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:34:08.059757 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:34:08.059757 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:34:08.059757 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 172 Jul 16 21:34:08.059757 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:34:08.059757 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:34:08.059757 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:34:08.059757 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:34:08.059757 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0y6QJDVdPNcLbUnwwgOJV-QkddLQ7yqnzBNVp2CrOe5UQqjQOER3spl3PswEW82ISSx8FhmuIrEwFSG69Ze1pTOmWBWZ8PURIgVHbN3yBPA3U2S2TKkQkFwyNuvvV03MCr9U0QGJIjXVOkiqFEAy8q87U_okZgCdI-BFjBFgrCdfAEo Jul 16 21:34:08.059757 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:34:08.059757 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:34:08.059757 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:34:08.059757 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:34:08.059757 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:34:08.059757 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:34:08.059757 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: da2248fa190342f0bfe9211cae1e403b Jul 16 21:34:08.059757 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-ServersNegativeTestJSON-1102403437 Jul 16 21:34:08.059757 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:34:08.059757 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:34:08.059757 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/da2248fa190342f0bfe9211cae1e403b"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/da2248fa190342f0bfe9211cae1e403b"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_da2248fa190342f0bfe9211cae1e403b"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/da2248fa190342f0bfe9211cae1e403b"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:34:08.059757 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-ServersNegativeTestJSON-1102403437 Jul 16 21:34:08.059757 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: da2248fa190342f0bfe9211cae1e403b Jul 16 21:34:08.059757 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-ServersNegativeTestJSON-1102403437 Jul 16 21:34:08.059757 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-ServersNegativeTestJSON-1102403437-project Jul 16 21:34:08.059757 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:34:08.059757 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:34:08.059757 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: 22e264440fff43cdb7be6702839af37f Jul 16 21:34:08.059757 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-ServersNegativeTestJSON-1102403437-project Jul 16 21:34:08.059757 ncdv-master-1 devstack@n-api.service[138680]: Jul 16 21:34:08.059757 ncdv-master-1 devstack@n-api.service[138680]: {"server": {"name": "tempest-ServersNegativeTestJSON-server-303704336", "imageRef": -1, "flavorRef": "452", "networks": [{"uuid": "f14874ef-c1c1-4761-ab34-120b93719d08"}]}} Jul 16 21:34:08.059757 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 21:34:08.059757 ncdv-master-1 devstack@n-api.service[138680]: exception: Invalid input for field/attribute imageRef. Value: -1. -1 is not valid under any of the given schemas Jul 16 21:34:08.059757 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'body': {'server': {'name': 'tempest-ServersNegativeTestJSON-server-303704336', 'imageRef': -1, 'flavorRef': '452', 'networks': [{'uuid': 'f14874ef-c1c1-4761-ab34-120b93719d08'}]}}} Jul 16 21:34:08.059757 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:34:08.059757 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:34:08.059757 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 279, in validate Jul 16 21:34:08.059757 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Jul 16 21:34:08.059757 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/lib/python3/dist-packages/jsonschema/validators.py", line 353, in validate Jul 16 21:34:08.059757 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise error Jul 16 21:34:08.059757 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: -1 is not valid under any of the given schemas Jul 16 21:34:08.059757 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:34:08.059757 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Failed validating 'oneOf' in schema['properties']['server']['properties']['imageRef']: Jul 16 21:34:08.059757 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi {'oneOf': [{'format': 'uuid', 'type': 'string'}, Jul 16 21:34:08.059757 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi {'maxLength': 0, 'type': 'string'}]} Jul 16 21:34:08.059757 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:34:08.059757 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi On instance['server']: Jul 16 21:34:08.059757 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi -1 Jul 16 21:34:08.059757 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:34:08.059757 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:34:08.059757 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:34:08.059757 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:34:08.059757 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:34:08.059757 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:34:08.059757 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:34:08.059757 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:34:08.059757 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:34:08.059757 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:34:08.059757 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 16 21:34:08.059757 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 16 21:34:08.059757 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Jul 16 21:34:08.059757 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Jul 16 21:34:08.059757 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Jul 16 21:34:08.059757 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Jul 16 21:34:08.064926 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 308, in validate Jul 16 21:34:08.064926 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Jul 16 21:34:08.064926 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute imageRef. Value: -1. -1 is not valid under any of the given schemas Jul 16 21:34:08.064926 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:34:08.064926 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:34:08.064926 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:34:08.064926 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:34:08.064926 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:34:08.064926 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:34:08.064926 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Jul 16 21:34:08.064926 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Jul 16 21:34:08.064926 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute imageRef. Value: -1. -1 is not valid under any of the given schemas Jul 16 21:34:08.064926 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:34:08.064926 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-0fe168b4-20a2-4266-aec9-371ff87bd6d9 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Returning 400 to user: Invalid input for field/attribute imageRef. Value: -1. -1 is not valid under any of the given schemas {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:34:08.064926 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-0fe168b4-20a2-4266-aec9-371ff87bd6d9 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 400 len: 145 microversion: 2.1 time: 0.009290 Jul 16 21:34:08.064926 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2432/4876] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 21:34:08 2022] POST /compute/v2.1/servers => generated 145 bytes in 11 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Jul 16 21:34:08.072381 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-05a14962-2676-4279-a2e6-f1afaf877660 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:34:08.075673 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-05a14962-2676-4279-a2e6-f1afaf877660 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: e4c7a9cb-e55d-4819-9885-2e72deffddb1] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:34:08.088461 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-05a14962-2676-4279-a2e6-f1afaf877660 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:08.090156 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-05a14962-2676-4279-a2e6-f1afaf877660 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:08.093309 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-e94993a6-357e-41bf-b398-dd07ed2f632c tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:34:08.094365 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-05a14962-2676-4279-a2e6-f1afaf877660 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:08.099911 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-05a14962-2676-4279-a2e6-f1afaf877660 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:08.105556 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-e94993a6-357e-41bf-b398-dd07ed2f632c tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] [instance: 8b77de54-4c8b-4ed5-84f5-68bf69f9809f] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:34:08.119723 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_api:INSERT=3,nova_api:SELECT=76,nova_api:UPDATE=1 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:34:08.121895 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e94993a6-357e-41bf-b398-dd07ed2f632c tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:08.122932 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e94993a6-357e-41bf-b398-dd07ed2f632c tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:08.124059 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e94993a6-357e-41bf-b398-dd07ed2f632c tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:08.124586 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e94993a6-357e-41bf-b398-dd07ed2f632c tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:08.580674 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-e94993a6-357e-41bf-b398-dd07ed2f632c tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:34:08.581660 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-05a14962-2676-4279-a2e6-f1afaf877660 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '22e264440fff43cdb7be6702839af37f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da2248fa190342f0bfe9211cae1e403b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:34:08.592792 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-05a14962-2676-4279-a2e6-f1afaf877660 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1" status: 200 len: 1504 microversion: 2.1 time: 0.524924 Jul 16 21:34:08.598404 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2445/4877] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:34:08 2022] GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1 => generated 1504 bytes in 532 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:34:08.611699 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-e94993a6-357e-41bf-b398-dd07ed2f632c tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:34:08.615564 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-e94993a6-357e-41bf-b398-dd07ed2f632c tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:34:08.617113 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-e94993a6-357e-41bf-b398-dd07ed2f632c tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] 10.222.0.16 "GET /compute/v2.1/servers/8b77de54-4c8b-4ed5-84f5-68bf69f9809f" status: 200 len: 1730 microversion: 2.20 time: 0.536977 Jul 16 21:34:08.618147 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2433/4878] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:34:08 2022] GET /compute/v2.1/servers/8b77de54-4c8b-4ed5-84f5-68bf69f9809f => generated 1730 bytes in 538 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:34:08.618424 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-736d05f1-c686-43f9-85c9-75c96a14c4f8 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:34:08.619187 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-736d05f1-c686-43f9-85c9-75c96a14c4f8 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: e4c7a9cb-e55d-4819-9885-2e72deffddb1] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:34:08.628749 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-736d05f1-c686-43f9-85c9-75c96a14c4f8 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:08.629589 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-736d05f1-c686-43f9-85c9-75c96a14c4f8 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:08.630719 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-736d05f1-c686-43f9-85c9-75c96a14c4f8 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:08.631445 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-736d05f1-c686-43f9-85c9-75c96a14c4f8 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:09.256646 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-736d05f1-c686-43f9-85c9-75c96a14c4f8 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '22e264440fff43cdb7be6702839af37f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da2248fa190342f0bfe9211cae1e403b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:34:09.274065 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-736d05f1-c686-43f9-85c9-75c96a14c4f8 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1" status: 200 len: 1504 microversion: 2.1 time: 0.662436 Jul 16 21:34:09.277184 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2446/4879] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:34:08 2022] GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1 => generated 1504 bytes in 667 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:34:09.480153 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-03e639dd-8e14-45de-87e3-0bed1ebd9994 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServersNegativeTestJSON-server-1147860824", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452", "networks": [{"fixed_ip": "10.0.1.1", "uuid": "a-b-c-d-e-f-g-h-i-j"}]}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:34:09.484872 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-03e639dd-8e14-45de-87e3-0bed1ebd9994 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] HTTP exception thrown: Bad networks format: network uuid is not in proper format (a-b-c-d-e-f-g-h-i-j) Jul 16 21:34:09.487119 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-03e639dd-8e14-45de-87e3-0bed1ebd9994 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Request method failure captured: Jul 16 21:34:09.487119 ncdv-master-1 devstack@n-api.service[138680]: request: POST /compute/v2.1/servers HTTP/1.1 Jul 16 21:34:09.487119 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:34:09.487119 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:34:09.487119 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:34:09.487119 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 216 Jul 16 21:34:09.487119 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:34:09.487119 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:34:09.487119 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:34:09.487119 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:34:09.487119 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0y6QJDVdPNcLbUnwwgOJV-QkddLQ7yqnzBNVp2CrOe5UQqjQOER3spl3PswEW82ISSx8FhmuIrEwFSG69Ze1pTOmWBWZ8PURIgVHbN3yBPA3U2S2TKkQkFwyNuvvV03MCr9U0QGJIjXVOkiqFEAy8q87U_okZgCdI-BFjBFgrCdfAEo Jul 16 21:34:09.487119 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:34:09.487119 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:34:09.487119 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:34:09.487119 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:34:09.487119 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:34:09.487119 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:34:09.487119 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: da2248fa190342f0bfe9211cae1e403b Jul 16 21:34:09.487119 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-ServersNegativeTestJSON-1102403437 Jul 16 21:34:09.487119 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:34:09.487119 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:34:09.487119 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/da2248fa190342f0bfe9211cae1e403b"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/da2248fa190342f0bfe9211cae1e403b"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_da2248fa190342f0bfe9211cae1e403b"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/da2248fa190342f0bfe9211cae1e403b"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:34:09.487119 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-ServersNegativeTestJSON-1102403437 Jul 16 21:34:09.487119 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: da2248fa190342f0bfe9211cae1e403b Jul 16 21:34:09.487119 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-ServersNegativeTestJSON-1102403437 Jul 16 21:34:09.487119 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-ServersNegativeTestJSON-1102403437-project Jul 16 21:34:09.487119 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:34:09.487119 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:34:09.487119 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: 22e264440fff43cdb7be6702839af37f Jul 16 21:34:09.487119 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-ServersNegativeTestJSON-1102403437-project Jul 16 21:34:09.487119 ncdv-master-1 devstack@n-api.service[138680]: Jul 16 21:34:09.487119 ncdv-master-1 devstack@n-api.service[138680]: {"server": {"name": "tempest-ServersNegativeTestJSON-server-1147860824", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452", "networks": [{"fixed_ip": "10.0.1.1", "uuid": "a-b-c-d-e-f-g-h-i-j"}]}} Jul 16 21:34:09.487119 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 21:34:09.487119 ncdv-master-1 devstack@n-api.service[138680]: exception: Bad networks format: network uuid is not in proper format (a-b-c-d-e-f-g-h-i-j) Jul 16 21:34:09.487119 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'body': {'server': {'name': 'tempest-ServersNegativeTestJSON-server-1147860824', 'imageRef': '1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9', 'flavorRef': '452', 'networks': [{'fixed_ip': '10.0.1.1', 'uuid': 'a-b-c-d-e-f-g-h-i-j'}]}}} Jul 16 21:34:09.487119 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:34:09.487119 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:34:09.487119 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:34:09.487119 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:34:09.487119 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:34:09.487119 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:34:09.487119 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:34:09.487119 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:34:09.487119 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 16 21:34:09.487119 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 16 21:34:09.487119 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 16 21:34:09.487119 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 16 21:34:09.487119 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 16 21:34:09.487119 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 16 21:34:09.487119 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi [Previous line repeated 10 more times] Jul 16 21:34:09.487119 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 773, in create Jul 16 21:34:09.487119 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi self._process_networks_for_create( Jul 16 21:34:09.487119 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 547, in _process_networks_for_create Jul 16 21:34:09.487119 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi requested_networks = self._get_requested_networks( Jul 16 21:34:09.487119 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 447, in _get_requested_networks Jul 16 21:34:09.487119 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi self._validate_network_id( Jul 16 21:34:09.487119 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 396, in _validate_network_id Jul 16 21:34:09.487119 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=msg) Jul 16 21:34:09.487119 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Bad networks format: network uuid is not in proper format (a-b-c-d-e-f-g-h-i-j) Jul 16 21:34:09.487119 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:34:09.487119 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:34:09.487119 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:34:09.492965 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:34:09.492965 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:34:09.492965 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:34:09.492965 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:34:09.492965 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:34:09.492965 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Bad networks format: network uuid is not in proper format (a-b-c-d-e-f-g-h-i-j) Jul 16 21:34:09.492965 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:34:09.495304 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-03e639dd-8e14-45de-87e3-0bed1ebd9994 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Returning 400 to user: Bad networks format: network uuid is not in proper format (a-b-c-d-e-f-g-h-i-j) {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:34:09.496585 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-03e639dd-8e14-45de-87e3-0bed1ebd9994 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 400 len: 123 microversion: 2.1 time: 0.022595 Jul 16 21:34:09.497612 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2434/4880] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 21:34:09 2022] POST /compute/v2.1/servers => generated 123 bytes in 25 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Jul 16 21:34:09.507187 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-9f66ce61-56f7-4b74-ab08-4175a636cbbc tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:34:09.508146 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-9f66ce61-56f7-4b74-ab08-4175a636cbbc tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: e4c7a9cb-e55d-4819-9885-2e72deffddb1] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:34:09.522173 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9f66ce61-56f7-4b74-ab08-4175a636cbbc tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:09.524922 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9f66ce61-56f7-4b74-ab08-4175a636cbbc tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:09.526969 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9f66ce61-56f7-4b74-ab08-4175a636cbbc tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:09.528177 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9f66ce61-56f7-4b74-ab08-4175a636cbbc tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:09.642392 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-a6d8a86a-3f5e-454c-a210-dfcaa3596e18 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:34:09.644484 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-a6d8a86a-3f5e-454c-a210-dfcaa3596e18 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] [instance: 8b77de54-4c8b-4ed5-84f5-68bf69f9809f] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:34:09.666032 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a6d8a86a-3f5e-454c-a210-dfcaa3596e18 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:09.667673 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a6d8a86a-3f5e-454c-a210-dfcaa3596e18 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:09.676611 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a6d8a86a-3f5e-454c-a210-dfcaa3596e18 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:09.677756 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a6d8a86a-3f5e-454c-a210-dfcaa3596e18 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:10.308406 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-9f66ce61-56f7-4b74-ab08-4175a636cbbc tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '22e264440fff43cdb7be6702839af37f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da2248fa190342f0bfe9211cae1e403b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:34:10.320454 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-9f66ce61-56f7-4b74-ab08-4175a636cbbc tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1" status: 200 len: 1504 microversion: 2.1 time: 0.818537 Jul 16 21:34:10.321047 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2447/4881] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:34:09 2022] GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1 => generated 1504 bytes in 820 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:34:10.348791 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-e1d52ea7-321d-44b6-9317-3588f31db91d tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:34:10.350253 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-e1d52ea7-321d-44b6-9317-3588f31db91d tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: e4c7a9cb-e55d-4819-9885-2e72deffddb1] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:34:10.360886 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e1d52ea7-321d-44b6-9317-3588f31db91d tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:10.363246 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e1d52ea7-321d-44b6-9317-3588f31db91d tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:10.364476 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e1d52ea7-321d-44b6-9317-3588f31db91d tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:10.365156 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e1d52ea7-321d-44b6-9317-3588f31db91d tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:10.510357 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-a6d8a86a-3f5e-454c-a210-dfcaa3596e18 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:34:10.522616 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-a6d8a86a-3f5e-454c-a210-dfcaa3596e18 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:34:10.527669 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-a6d8a86a-3f5e-454c-a210-dfcaa3596e18 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:34:10.527669 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-a6d8a86a-3f5e-454c-a210-dfcaa3596e18 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] 10.222.0.16 "GET /compute/v2.1/servers/8b77de54-4c8b-4ed5-84f5-68bf69f9809f" status: 200 len: 1730 microversion: 2.20 time: 0.890088 Jul 16 21:34:10.535068 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2435/4882] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:34:09 2022] GET /compute/v2.1/servers/8b77de54-4c8b-4ed5-84f5-68bf69f9809f => generated 1730 bytes in 897 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 16 21:34:10.852571 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-e1d52ea7-321d-44b6-9317-3588f31db91d tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '22e264440fff43cdb7be6702839af37f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da2248fa190342f0bfe9211cae1e403b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:34:10.872321 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-e1d52ea7-321d-44b6-9317-3588f31db91d tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1" status: 200 len: 1504 microversion: 2.1 time: 0.530359 Jul 16 21:34:10.876672 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2448/4883] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:34:10 2022] GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1 => generated 1504 bytes in 536 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:34:10.972865 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-d33a5495-b58a-485c-b41c-f0ed0d3f5b35 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServersNegativeTestJSON-server-379989738", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452", "key_name": "tempest-key-1785776460", "networks": [{"uuid": "f14874ef-c1c1-4761-ab34-120b93719d08"}]}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:34:11.059339 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.network.neutron [None req-d33a5495-b58a-485c-b41c-f0ed0d3f5b35 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] validate_networks() for [('f14874ef-c1c1-4761-ab34-120b93719d08', None, None, None, None, None)] {{(pid=138680) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 16 21:34:11.380419 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-d33a5495-b58a-485c-b41c-f0ed0d3f5b35 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] HTTP exception thrown: Invalid key_name provided. Jul 16 21:34:11.388302 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-d33a5495-b58a-485c-b41c-f0ed0d3f5b35 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Request method failure captured: Jul 16 21:34:11.388302 ncdv-master-1 devstack@n-api.service[138680]: request: POST /compute/v2.1/servers HTTP/1.1 Jul 16 21:34:11.388302 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:34:11.388302 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:34:11.388302 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:34:11.388302 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 246 Jul 16 21:34:11.388302 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:34:11.388302 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:34:11.388302 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:34:11.388302 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:34:11.388302 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0y6QJDVdPNcLbUnwwgOJV-QkddLQ7yqnzBNVp2CrOe5UQqjQOER3spl3PswEW82ISSx8FhmuIrEwFSG69Ze1pTOmWBWZ8PURIgVHbN3yBPA3U2S2TKkQkFwyNuvvV03MCr9U0QGJIjXVOkiqFEAy8q87U_okZgCdI-BFjBFgrCdfAEo Jul 16 21:34:11.388302 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:34:11.388302 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:34:11.388302 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:34:11.388302 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:34:11.388302 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:34:11.388302 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:34:11.388302 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: da2248fa190342f0bfe9211cae1e403b Jul 16 21:34:11.388302 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-ServersNegativeTestJSON-1102403437 Jul 16 21:34:11.388302 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:34:11.388302 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:34:11.388302 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/da2248fa190342f0bfe9211cae1e403b"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/da2248fa190342f0bfe9211cae1e403b"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_da2248fa190342f0bfe9211cae1e403b"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/da2248fa190342f0bfe9211cae1e403b"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:34:11.388302 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-ServersNegativeTestJSON-1102403437 Jul 16 21:34:11.388302 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: da2248fa190342f0bfe9211cae1e403b Jul 16 21:34:11.388302 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-ServersNegativeTestJSON-1102403437 Jul 16 21:34:11.388302 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-ServersNegativeTestJSON-1102403437-project Jul 16 21:34:11.388302 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:34:11.388302 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:34:11.388302 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: 22e264440fff43cdb7be6702839af37f Jul 16 21:34:11.388302 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-ServersNegativeTestJSON-1102403437-project Jul 16 21:34:11.388302 ncdv-master-1 devstack@n-api.service[138680]: Jul 16 21:34:11.388302 ncdv-master-1 devstack@n-api.service[138680]: {"server": {"name": "tempest-ServersNegativeTestJSON-server-379989738", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452", "key_name": "tempest-key-1785776460", "networks": [{"uuid": "f14874ef-c1c1-4761-ab34-120b93719d08"}]}} Jul 16 21:34:11.388302 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 21:34:11.388302 ncdv-master-1 devstack@n-api.service[138680]: exception: Invalid key_name provided. Jul 16 21:34:11.388302 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'body': {'server': {'name': 'tempest-ServersNegativeTestJSON-server-379989738', 'imageRef': '1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9', 'flavorRef': '452', 'key_name': 'tempest-key-1785776460', 'networks': [{'uuid': 'f14874ef-c1c1-4761-ab34-120b93719d08'}]}}} Jul 16 21:34:11.388302 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:34:11.388302 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:34:11.388302 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 784, in create Jul 16 21:34:11.388302 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instances, resv_id = self.compute_api.create( Jul 16 21:34:11.388302 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2149, in create Jul 16 21:34:11.388302 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return self._create_instance( Jul 16 21:34:11.388302 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 1632, in _create_instance Jul 16 21:34:11.388302 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi ) = self._validate_and_build_base_options( Jul 16 21:34:11.388302 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 1075, in _validate_and_build_base_options Jul 16 21:34:11.388302 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi key_pair = objects.KeyPair.get_by_name(context, Jul 16 21:34:11.388302 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Jul 16 21:34:11.388302 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Jul 16 21:34:11.388302 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/keypair.py", line 144, in get_by_name Jul 16 21:34:11.388302 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi db_keypair = cls._get_from_db(context, user_id, name) Jul 16 21:34:11.388302 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/keypair.py", line 126, in _get_from_db Jul 16 21:34:11.388302 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return _get_from_db(context, user_id, name=name) Jul 16 21:34:11.388302 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1010, in wrapper Jul 16 21:34:11.388302 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return fn(*args, **kwargs) Jul 16 21:34:11.388302 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/keypair.py", line 40, in _get_from_db Jul 16 21:34:11.388302 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.KeypairNotFound(user_id=user_id, name=name) Jul 16 21:34:11.388302 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.KeypairNotFound: Keypair tempest-key-1785776460 not found for user 22e264440fff43cdb7be6702839af37f Jul 16 21:34:11.388302 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:34:11.388302 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:34:11.388302 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:34:11.388302 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:34:11.388302 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:34:11.394027 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:34:11.394027 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:34:11.394027 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:34:11.394027 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:34:11.394027 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:34:11.394027 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 16 21:34:11.394027 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 16 21:34:11.394027 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 16 21:34:11.394027 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 16 21:34:11.394027 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 16 21:34:11.394027 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 16 21:34:11.394027 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi [Previous line repeated 10 more times] Jul 16 21:34:11.394027 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 807, in create Jul 16 21:34:11.394027 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=msg) Jul 16 21:34:11.394027 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid key_name provided. Jul 16 21:34:11.394027 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:34:11.394027 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:34:11.394027 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:34:11.394027 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:34:11.394027 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:34:11.394027 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:34:11.394027 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:34:11.394027 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:34:11.394027 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid key_name provided. Jul 16 21:34:11.394027 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:34:11.394027 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-d33a5495-b58a-485c-b41c-f0ed0d3f5b35 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Returning 400 to user: Invalid key_name provided. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:34:11.394027 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-d33a5495-b58a-485c-b41c-f0ed0d3f5b35 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 400 len: 70 microversion: 2.1 time: 0.423134 Jul 16 21:34:11.394027 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2436/4884] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 21:34:10 2022] POST /compute/v2.1/servers => generated 70 bytes in 426 msecs (HTTP/1.1 400) 9 headers in 380 bytes (1 switches on core 0) Jul 16 21:34:11.401801 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-6729e1b7-3569-4fe6-bb83-eab349e40197 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:34:11.402499 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-6729e1b7-3569-4fe6-bb83-eab349e40197 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: e4c7a9cb-e55d-4819-9885-2e72deffddb1] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:34:11.408775 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6729e1b7-3569-4fe6-bb83-eab349e40197 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:11.409526 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6729e1b7-3569-4fe6-bb83-eab349e40197 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:11.410506 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6729e1b7-3569-4fe6-bb83-eab349e40197 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:11.410948 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6729e1b7-3569-4fe6-bb83-eab349e40197 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:11.552486 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-eb3b418b-6c19-4242-992c-9496ce3af6e5 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:34:11.561306 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-eb3b418b-6c19-4242-992c-9496ce3af6e5 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] [instance: 8b77de54-4c8b-4ed5-84f5-68bf69f9809f] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:34:11.577938 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-eb3b418b-6c19-4242-992c-9496ce3af6e5 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:11.578393 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-eb3b418b-6c19-4242-992c-9496ce3af6e5 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:11.579210 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-eb3b418b-6c19-4242-992c-9496ce3af6e5 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:11.579654 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-eb3b418b-6c19-4242-992c-9496ce3af6e5 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:11.658053 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-6729e1b7-3569-4fe6-bb83-eab349e40197 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '22e264440fff43cdb7be6702839af37f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da2248fa190342f0bfe9211cae1e403b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:34:11.681507 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-6729e1b7-3569-4fe6-bb83-eab349e40197 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1" status: 200 len: 1504 microversion: 2.1 time: 0.283619 Jul 16 21:34:11.682108 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2449/4885] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:34:11 2022] GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1 => generated 1504 bytes in 285 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:34:11.700389 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-ff6c5643-b480-4964-a567-8daace291a01 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:34:11.701013 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-ff6c5643-b480-4964-a567-8daace291a01 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: e4c7a9cb-e55d-4819-9885-2e72deffddb1] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:34:11.708166 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ff6c5643-b480-4964-a567-8daace291a01 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:11.708971 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ff6c5643-b480-4964-a567-8daace291a01 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:11.709762 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ff6c5643-b480-4964-a567-8daace291a01 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:11.710195 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ff6c5643-b480-4964-a567-8daace291a01 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:11.825515 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-eb3b418b-6c19-4242-992c-9496ce3af6e5 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:34:11.836402 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-eb3b418b-6c19-4242-992c-9496ce3af6e5 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:34:11.838118 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-eb3b418b-6c19-4242-992c-9496ce3af6e5 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:34:11.839339 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-eb3b418b-6c19-4242-992c-9496ce3af6e5 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] 10.222.0.16 "GET /compute/v2.1/servers/8b77de54-4c8b-4ed5-84f5-68bf69f9809f" status: 200 len: 1717 microversion: 2.20 time: 0.291841 Jul 16 21:34:11.855196 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2437/4886] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:34:11 2022] GET /compute/v2.1/servers/8b77de54-4c8b-4ed5-84f5-68bf69f9809f => generated 1717 bytes in 293 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 16 21:34:11.950719 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-ff6c5643-b480-4964-a567-8daace291a01 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '22e264440fff43cdb7be6702839af37f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da2248fa190342f0bfe9211cae1e403b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:34:11.961161 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-ff6c5643-b480-4964-a567-8daace291a01 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1" status: 200 len: 1504 microversion: 2.1 time: 0.264679 Jul 16 21:34:11.975040 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2450/4887] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:34:11 2022] GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1 => generated 1504 bytes in 266 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:34:12.035937 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-71456cd8-1501-498f-afe2-362b218449f3 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServersNegativeTestJSON-server-1044664193", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452", "security_groups": [{"name": "does_not_exist"}], "networks": [{"uuid": "f14874ef-c1c1-4761-ab34-120b93719d08"}]}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:34:12.176369 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.network.security_group_api [None req-71456cd8-1501-498f-afe2-362b218449f3 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Neutron security group does_not_exist not found {{(pid=138680) validate_name /opt/stack/nova/nova/network/security_group_api.py:74}} Jul 16 21:34:12.177431 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-71456cd8-1501-498f-afe2-362b218449f3 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] HTTP exception thrown: Unable to find security_group with name or id 'does_not_exist' Jul 16 21:34:12.208117 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-71456cd8-1501-498f-afe2-362b218449f3 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Request method failure captured: Jul 16 21:34:12.208117 ncdv-master-1 devstack@n-api.service[138680]: request: POST /compute/v2.1/servers HTTP/1.1 Jul 16 21:34:12.208117 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:34:12.208117 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:34:12.208117 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:34:12.208117 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 258 Jul 16 21:34:12.208117 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:34:12.208117 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:34:12.208117 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:34:12.208117 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:34:12.208117 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0y6QJDVdPNcLbUnwwgOJV-QkddLQ7yqnzBNVp2CrOe5UQqjQOER3spl3PswEW82ISSx8FhmuIrEwFSG69Ze1pTOmWBWZ8PURIgVHbN3yBPA3U2S2TKkQkFwyNuvvV03MCr9U0QGJIjXVOkiqFEAy8q87U_okZgCdI-BFjBFgrCdfAEo Jul 16 21:34:12.208117 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:34:12.208117 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:34:12.208117 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:34:12.208117 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:34:12.208117 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:34:12.208117 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:34:12.208117 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: da2248fa190342f0bfe9211cae1e403b Jul 16 21:34:12.208117 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-ServersNegativeTestJSON-1102403437 Jul 16 21:34:12.208117 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:34:12.208117 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:34:12.208117 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/da2248fa190342f0bfe9211cae1e403b"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/da2248fa190342f0bfe9211cae1e403b"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_da2248fa190342f0bfe9211cae1e403b"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/da2248fa190342f0bfe9211cae1e403b"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:34:12.208117 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-ServersNegativeTestJSON-1102403437 Jul 16 21:34:12.208117 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: da2248fa190342f0bfe9211cae1e403b Jul 16 21:34:12.208117 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-ServersNegativeTestJSON-1102403437 Jul 16 21:34:12.208117 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-ServersNegativeTestJSON-1102403437-project Jul 16 21:34:12.208117 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:34:12.208117 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:34:12.208117 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: 22e264440fff43cdb7be6702839af37f Jul 16 21:34:12.208117 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-ServersNegativeTestJSON-1102403437-project Jul 16 21:34:12.208117 ncdv-master-1 devstack@n-api.service[138680]: Jul 16 21:34:12.208117 ncdv-master-1 devstack@n-api.service[138680]: {"server": {"name": "tempest-ServersNegativeTestJSON-server-1044664193", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452", "security_groups": [{"name": "does_not_exist"}], "networks": [{"uuid": "f14874ef-c1c1-4761-ab34-120b93719d08"}]}} Jul 16 21:34:12.208117 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 21:34:12.208117 ncdv-master-1 devstack@n-api.service[138680]: exception: Unable to find security_group with name or id 'does_not_exist' Jul 16 21:34:12.208117 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'body': {'server': {'name': 'tempest-ServersNegativeTestJSON-server-1044664193', 'imageRef': '1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9', 'flavorRef': '452', 'security_groups': [{'name': 'does_not_exist'}], 'networks': [{'uuid': 'f14874ef-c1c1-4761-ab34-120b93719d08'}]}}} Jul 16 21:34:12.208117 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:34:12.208117 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:34:12.208117 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 476, in find_resource Jul 16 21:34:12.208117 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return self.find_resource_by_id(resource, name_or_id, Jul 16 21:34:12.208117 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 197, in wrapper Jul 16 21:34:12.208117 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Jul 16 21:34:12.208117 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 440, in find_resource_by_id Jul 16 21:34:12.208117 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exceptions.NotFound(message=not_found_message) Jul 16 21:34:12.208117 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi neutronclient.common.exceptions.NotFound: Unable to find security_group with id 'does_not_exist' Jul 16 21:34:12.208117 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:34:12.208117 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:34:12.208117 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:34:12.208117 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:34:12.208117 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 480, in find_resource Jul 16 21:34:12.208117 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return self._find_resource_by_name( Jul 16 21:34:12.208117 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 197, in wrapper Jul 16 21:34:12.208117 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Jul 16 21:34:12.208117 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 469, in _find_resource_by_name Jul 16 21:34:12.208117 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exceptions.NotFound(message=not_found_message) Jul 16 21:34:12.208117 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi neutronclient.common.exceptions.NotFound: Unable to find security_group with name 'does_not_exist' Jul 16 21:34:12.208117 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:34:12.208117 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:34:12.208117 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:34:12.208117 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:34:12.208117 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/security_group_api.py", line 68, in validate_name Jul 16 21:34:12.208117 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return neutronv20.find_resourceid_by_name_or_id( Jul 16 21:34:12.213211 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/neutron/v2_0/__init__.py", line 56, in find_resourceid_by_name_or_id Jul 16 21:34:12.213211 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return find_resource_by_name_or_id(client, resource, name_or_id, Jul 16 21:34:12.213211 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/neutron/v2_0/__init__.py", line 49, in find_resource_by_name_or_id Jul 16 21:34:12.213211 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return client.find_resource(resource, name_or_id, project_id, Jul 16 21:34:12.213211 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 197, in wrapper Jul 16 21:34:12.213211 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Jul 16 21:34:12.213211 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 488, in find_resource Jul 16 21:34:12.213211 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exceptions.NotFound( Jul 16 21:34:12.213211 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi neutronclient.common.exceptions.NotFound: Unable to find security_group with name or id 'does_not_exist' Jul 16 21:34:12.213211 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:34:12.213211 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:34:12.213211 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:34:12.213211 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:34:12.213211 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 784, in create Jul 16 21:34:12.213211 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instances, resv_id = self.compute_api.create( Jul 16 21:34:12.213211 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2149, in create Jul 16 21:34:12.213211 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return self._create_instance( Jul 16 21:34:12.213211 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 1632, in _create_instance Jul 16 21:34:12.213211 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi ) = self._validate_and_build_base_options( Jul 16 21:34:12.213211 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 1060, in _validate_and_build_base_options Jul 16 21:34:12.213211 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi security_groups = self._check_requested_secgroups( Jul 16 21:34:12.213211 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 507, in _check_requested_secgroups Jul 16 21:34:12.213211 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi secgroup_uuid = security_group_api.validate_name(context, secgroup) Jul 16 21:34:12.213211 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/security_group_api.py", line 75, in validate_name Jul 16 21:34:12.213211 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.SecurityGroupNotFound(str(e)) Jul 16 21:34:12.213211 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.SecurityGroupNotFound: Unable to find security_group with name or id 'does_not_exist' Jul 16 21:34:12.213211 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:34:12.213211 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:34:12.213211 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:34:12.213211 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:34:12.213211 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:34:12.213211 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:34:12.213211 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:34:12.213211 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:34:12.213211 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:34:12.213211 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:34:12.213211 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 16 21:34:12.213211 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 16 21:34:12.213211 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 16 21:34:12.213211 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 16 21:34:12.213211 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 16 21:34:12.213211 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 16 21:34:12.213211 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi [Previous line repeated 10 more times] Jul 16 21:34:12.213211 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 866, in create Jul 16 21:34:12.213211 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=error.format_message()) Jul 16 21:34:12.213211 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Unable to find security_group with name or id 'does_not_exist' Jul 16 21:34:12.213211 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:34:12.213211 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:34:12.213211 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:34:12.213211 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:34:12.213211 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:34:12.213211 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:34:12.213211 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:34:12.213211 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:34:12.213211 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Unable to find security_group with name or id 'does_not_exist' Jul 16 21:34:12.213211 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:34:12.213211 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-71456cd8-1501-498f-afe2-362b218449f3 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Returning 400 to user: Unable to find security_group with name or id 'does_not_exist' {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:34:12.217863 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-71456cd8-1501-498f-afe2-362b218449f3 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 400 len: 106 microversion: 2.1 time: 0.182460 Jul 16 21:34:12.217863 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2438/4888] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 21:34:12 2022] POST /compute/v2.1/servers => generated 106 bytes in 186 msecs (HTTP/1.1 400) 9 headers in 381 bytes (2 switches on core 0) Jul 16 21:34:12.223839 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-2b24b3e0-ea2f-4c15-a68e-7f642743c0c1 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:34:12.224500 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-2b24b3e0-ea2f-4c15-a68e-7f642743c0c1 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: e4c7a9cb-e55d-4819-9885-2e72deffddb1] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:34:12.232166 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2b24b3e0-ea2f-4c15-a68e-7f642743c0c1 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:12.232791 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2b24b3e0-ea2f-4c15-a68e-7f642743c0c1 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:12.233653 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2b24b3e0-ea2f-4c15-a68e-7f642743c0c1 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:12.234175 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2b24b3e0-ea2f-4c15-a68e-7f642743c0c1 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:12.475804 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-2b24b3e0-ea2f-4c15-a68e-7f642743c0c1 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '22e264440fff43cdb7be6702839af37f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da2248fa190342f0bfe9211cae1e403b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:34:12.493043 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-2b24b3e0-ea2f-4c15-a68e-7f642743c0c1 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1" status: 200 len: 1504 microversion: 2.1 time: 0.273222 Jul 16 21:34:12.494332 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2451/4889] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:34:12 2022] GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1 => generated 1504 bytes in 275 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:34:12.511481 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-ad0eec21-84bf-41dc-84c8-a75b2f950cdf tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:34:12.514097 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-ad0eec21-84bf-41dc-84c8-a75b2f950cdf tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: e4c7a9cb-e55d-4819-9885-2e72deffddb1] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:34:12.520698 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ad0eec21-84bf-41dc-84c8-a75b2f950cdf tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:12.521397 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ad0eec21-84bf-41dc-84c8-a75b2f950cdf tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:12.522366 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ad0eec21-84bf-41dc-84c8-a75b2f950cdf tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:12.522979 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ad0eec21-84bf-41dc-84c8-a75b2f950cdf tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:12.779624 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-ad0eec21-84bf-41dc-84c8-a75b2f950cdf tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '22e264440fff43cdb7be6702839af37f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da2248fa190342f0bfe9211cae1e403b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:34:12.790146 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-ad0eec21-84bf-41dc-84c8-a75b2f950cdf tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1" status: 200 len: 1504 microversion: 2.1 time: 0.281703 Jul 16 21:34:12.803371 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2439/4890] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:34:12 2022] GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1 => generated 1504 bytes in 296 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:34:12.816686 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-70247157-597b-4dc1-b479-ddb065ab8b72 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:34:12.817600 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-70247157-597b-4dc1-b479-ddb065ab8b72 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: 74c633d9-03da-4f42-a5b3-176d11af264f] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:34:12.833336 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-70247157-597b-4dc1-b479-ddb065ab8b72 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] HTTP exception thrown: Instance 74c633d9-03da-4f42-a5b3-176d11af264f could not be found. Jul 16 21:34:12.836341 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-70247157-597b-4dc1-b479-ddb065ab8b72 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Request method failure captured: Jul 16 21:34:12.836341 ncdv-master-1 devstack@n-api.service[138681]: request: DELETE /compute/v2.1/servers/74c633d9-03da-4f42-a5b3-176d11af264f HTTP/1.1 Jul 16 21:34:12.836341 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:34:12.836341 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:34:12.836341 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:34:12.836341 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 21:34:12.836341 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:34:12.836341 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:34:12.836341 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:34:12.836341 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:34:12.836341 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0y6QJDVdPNcLbUnwwgOJV-QkddLQ7yqnzBNVp2CrOe5UQqjQOER3spl3PswEW82ISSx8FhmuIrEwFSG69Ze1pTOmWBWZ8PURIgVHbN3yBPA3U2S2TKkQkFwyNuvvV03MCr9U0QGJIjXVOkiqFEAy8q87U_okZgCdI-BFjBFgrCdfAEo Jul 16 21:34:12.836341 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:34:12.836341 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:34:12.836341 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:34:12.836341 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:34:12.836341 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:34:12.836341 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:34:12.836341 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: da2248fa190342f0bfe9211cae1e403b Jul 16 21:34:12.836341 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-ServersNegativeTestJSON-1102403437 Jul 16 21:34:12.836341 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:34:12.836341 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:34:12.836341 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/da2248fa190342f0bfe9211cae1e403b"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/da2248fa190342f0bfe9211cae1e403b"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_da2248fa190342f0bfe9211cae1e403b"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/da2248fa190342f0bfe9211cae1e403b"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:34:12.836341 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-ServersNegativeTestJSON-1102403437 Jul 16 21:34:12.836341 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: da2248fa190342f0bfe9211cae1e403b Jul 16 21:34:12.836341 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-ServersNegativeTestJSON-1102403437 Jul 16 21:34:12.836341 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-ServersNegativeTestJSON-1102403437-project Jul 16 21:34:12.836341 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:34:12.836341 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:34:12.836341 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: 22e264440fff43cdb7be6702839af37f Jul 16 21:34:12.836341 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-ServersNegativeTestJSON-1102403437-project Jul 16 21:34:12.836341 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:34:12.836341 ncdv-master-1 devstack@n-api.service[138681]: exception: Instance 74c633d9-03da-4f42-a5b3-176d11af264f could not be found. Jul 16 21:34:12.836341 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': '74c633d9-03da-4f42-a5b3-176d11af264f'} Jul 16 21:34:12.836341 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:34:12.836341 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:34:12.836341 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:34:12.836341 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:34:12.836341 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:34:12.836341 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:34:12.836341 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2898, in _get_instance Jul 16 21:34:12.836341 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=instance_uuid) Jul 16 21:34:12.836341 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 74c633d9-03da-4f42-a5b3-176d11af264f could not be found. Jul 16 21:34:12.836341 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:34:12.836341 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:34:12.836341 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:34:12.836341 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:34:12.836341 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:34:12.836341 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:34:12.836341 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:34:12.836341 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:34:12.836341 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:34:12.836341 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:34:12.836341 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Jul 16 21:34:12.836341 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Jul 16 21:34:12.836341 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Jul 16 21:34:12.836341 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Jul 16 21:34:12.836341 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:34:12.836341 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:34:12.836341 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:34:12.836341 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:34:12.836341 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 74c633d9-03da-4f42-a5b3-176d11af264f could not be found. Jul 16 21:34:12.836341 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:34:12.836341 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:34:12.841159 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:34:12.841159 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:34:12.841159 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:34:12.841159 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:34:12.841159 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:34:12.841159 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:34:12.841159 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 74c633d9-03da-4f42-a5b3-176d11af264f could not be found. Jul 16 21:34:12.841159 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:34:12.842839 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-70247157-597b-4dc1-b479-ddb065ab8b72 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Returning 404 to user: Instance 74c633d9-03da-4f42-a5b3-176d11af264f could not be found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:34:12.843792 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-70247157-597b-4dc1-b479-ddb065ab8b72 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "DELETE /compute/v2.1/servers/74c633d9-03da-4f42-a5b3-176d11af264f" status: 404 len: 111 microversion: 2.1 time: 0.039912 Jul 16 21:34:12.848693 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2452/4891] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:34:12 2022] DELETE /compute/v2.1/servers/74c633d9-03da-4f42-a5b3-176d11af264f => generated 111 bytes in 46 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:34:12.857557 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-2e72727f-fc03-471c-b03f-2a7d7af385d0 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:34:12.861817 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-2e72727f-fc03-471c-b03f-2a7d7af385d0 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: e4c7a9cb-e55d-4819-9885-2e72deffddb1] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:34:12.865672 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2e72727f-fc03-471c-b03f-2a7d7af385d0 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:12.866475 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2e72727f-fc03-471c-b03f-2a7d7af385d0 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:12.867708 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2e72727f-fc03-471c-b03f-2a7d7af385d0 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:12.868563 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2e72727f-fc03-471c-b03f-2a7d7af385d0 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:13.132408 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-2e72727f-fc03-471c-b03f-2a7d7af385d0 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '22e264440fff43cdb7be6702839af37f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da2248fa190342f0bfe9211cae1e403b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:34:13.134699 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=100 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:34:13.148040 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-2e72727f-fc03-471c-b03f-2a7d7af385d0 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1" status: 200 len: 1504 microversion: 2.1 time: 0.294770 Jul 16 21:34:13.149175 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2440/4892] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:34:12 2022] GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1 => generated 1504 bytes in 296 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:34:13.176095 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-ea362962-5728-4b07-95e1-05da1d37261c tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:34:13.176095 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-ea362962-5728-4b07-95e1-05da1d37261c tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: e4c7a9cb-e55d-4819-9885-2e72deffddb1] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:34:13.181642 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ea362962-5728-4b07-95e1-05da1d37261c tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:13.182044 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ea362962-5728-4b07-95e1-05da1d37261c tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:13.183305 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ea362962-5728-4b07-95e1-05da1d37261c tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:13.184367 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ea362962-5728-4b07-95e1-05da1d37261c tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:13.446849 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-ea362962-5728-4b07-95e1-05da1d37261c tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '22e264440fff43cdb7be6702839af37f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da2248fa190342f0bfe9211cae1e403b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:34:13.457540 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-ea362962-5728-4b07-95e1-05da1d37261c tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1" status: 200 len: 1504 microversion: 2.1 time: 0.293607 Jul 16 21:34:13.458388 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2453/4893] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:34:13 2022] GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1 => generated 1504 bytes in 295 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:34:13.475665 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-b67036c5-01c6-45e2-b417-f36f255fb1b2 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:34:13.476165 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-b67036c5-01c6-45e2-b417-f36f255fb1b2 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Failed to fetch instance by id 9223372036854775808 {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2926}} Jul 16 21:34:13.477056 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-b67036c5-01c6-45e2-b417-f36f255fb1b2 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] HTTP exception thrown: Instance 9223372036854775808 could not be found. Jul 16 21:34:13.479104 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-b67036c5-01c6-45e2-b417-f36f255fb1b2 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Request method failure captured: Jul 16 21:34:13.479104 ncdv-master-1 devstack@n-api.service[138680]: request: DELETE /compute/v2.1/servers/9223372036854775808 HTTP/1.1 Jul 16 21:34:13.479104 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:34:13.479104 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:34:13.479104 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:34:13.479104 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 0 Jul 16 21:34:13.479104 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:34:13.479104 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:34:13.479104 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:34:13.479104 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:34:13.479104 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0y6QJDVdPNcLbUnwwgOJV-QkddLQ7yqnzBNVp2CrOe5UQqjQOER3spl3PswEW82ISSx8FhmuIrEwFSG69Ze1pTOmWBWZ8PURIgVHbN3yBPA3U2S2TKkQkFwyNuvvV03MCr9U0QGJIjXVOkiqFEAy8q87U_okZgCdI-BFjBFgrCdfAEo Jul 16 21:34:13.479104 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:34:13.479104 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:34:13.479104 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:34:13.479104 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:34:13.479104 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:34:13.479104 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:34:13.479104 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: da2248fa190342f0bfe9211cae1e403b Jul 16 21:34:13.479104 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-ServersNegativeTestJSON-1102403437 Jul 16 21:34:13.479104 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:34:13.479104 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:34:13.479104 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/da2248fa190342f0bfe9211cae1e403b"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/da2248fa190342f0bfe9211cae1e403b"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_da2248fa190342f0bfe9211cae1e403b"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/da2248fa190342f0bfe9211cae1e403b"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:34:13.479104 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-ServersNegativeTestJSON-1102403437 Jul 16 21:34:13.479104 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: da2248fa190342f0bfe9211cae1e403b Jul 16 21:34:13.479104 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-ServersNegativeTestJSON-1102403437 Jul 16 21:34:13.479104 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-ServersNegativeTestJSON-1102403437-project Jul 16 21:34:13.479104 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:34:13.479104 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:34:13.479104 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: 22e264440fff43cdb7be6702839af37f Jul 16 21:34:13.479104 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-ServersNegativeTestJSON-1102403437-project Jul 16 21:34:13.479104 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 21:34:13.479104 ncdv-master-1 devstack@n-api.service[138680]: exception: Instance 9223372036854775808 could not be found. Jul 16 21:34:13.479104 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': '9223372036854775808'} Jul 16 21:34:13.479104 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:34:13.479104 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:34:13.479104 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:34:13.479104 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:34:13.479104 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2927, in get Jul 16 21:34:13.479104 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=instance_id) Jul 16 21:34:13.479104 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 9223372036854775808 could not be found. Jul 16 21:34:13.479104 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:34:13.479104 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:34:13.479104 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:34:13.479104 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:34:13.479104 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:34:13.479104 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:34:13.479104 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:34:13.479104 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:34:13.479104 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:34:13.479104 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:34:13.479104 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Jul 16 21:34:13.479104 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Jul 16 21:34:13.479104 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Jul 16 21:34:13.479104 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Jul 16 21:34:13.479104 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:34:13.479104 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:34:13.479104 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:34:13.479104 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:34:13.479104 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 9223372036854775808 could not be found. Jul 16 21:34:13.479104 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:34:13.479104 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:34:13.479104 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:34:13.479104 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:34:13.479104 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:34:13.484646 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:34:13.484646 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:34:13.484646 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:34:13.484646 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 9223372036854775808 could not be found. Jul 16 21:34:13.484646 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:34:13.484646 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-b67036c5-01c6-45e2-b417-f36f255fb1b2 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Returning 404 to user: Instance 9223372036854775808 could not be found. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:34:13.484646 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-b67036c5-01c6-45e2-b417-f36f255fb1b2 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "DELETE /compute/v2.1/servers/9223372036854775808" status: 404 len: 94 microversion: 2.1 time: 0.011167 Jul 16 21:34:13.489108 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2441/4894] 10.222.0.16 () {58 vars in 1187 bytes} [Sat Jul 16 21:34:13 2022] DELETE /compute/v2.1/servers/9223372036854775808 => generated 94 bytes in 15 msecs (HTTP/1.1 404) 9 headers in 378 bytes (2 switches on core 0) Jul 16 21:34:13.494856 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-c4ee7b98-9d7d-4d63-81a7-0ebc339a1a08 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:34:13.495620 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-c4ee7b98-9d7d-4d63-81a7-0ebc339a1a08 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: e4c7a9cb-e55d-4819-9885-2e72deffddb1] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:34:13.502100 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c4ee7b98-9d7d-4d63-81a7-0ebc339a1a08 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:13.502552 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c4ee7b98-9d7d-4d63-81a7-0ebc339a1a08 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:13.503486 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c4ee7b98-9d7d-4d63-81a7-0ebc339a1a08 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:13.503924 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c4ee7b98-9d7d-4d63-81a7-0ebc339a1a08 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:13.653536 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-c4ee7b98-9d7d-4d63-81a7-0ebc339a1a08 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '22e264440fff43cdb7be6702839af37f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da2248fa190342f0bfe9211cae1e403b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:34:13.665818 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-c4ee7b98-9d7d-4d63-81a7-0ebc339a1a08 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1" status: 200 len: 1504 microversion: 2.1 time: 0.174705 Jul 16 21:34:13.666415 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2454/4895] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:34:13 2022] GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1 => generated 1504 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:34:13.685460 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-88736cd6-57b0-4867-914e-a6fe71cfd7f5 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:34:13.686007 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-88736cd6-57b0-4867-914e-a6fe71cfd7f5 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: e4c7a9cb-e55d-4819-9885-2e72deffddb1] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:34:13.700417 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-88736cd6-57b0-4867-914e-a6fe71cfd7f5 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:13.700417 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-88736cd6-57b0-4867-914e-a6fe71cfd7f5 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:13.700417 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-88736cd6-57b0-4867-914e-a6fe71cfd7f5 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:13.700417 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-88736cd6-57b0-4867-914e-a6fe71cfd7f5 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:13.865903 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-88736cd6-57b0-4867-914e-a6fe71cfd7f5 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '22e264440fff43cdb7be6702839af37f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da2248fa190342f0bfe9211cae1e403b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:34:13.875296 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-88736cd6-57b0-4867-914e-a6fe71cfd7f5 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1" status: 200 len: 1504 microversion: 2.1 time: 0.194703 Jul 16 21:34:13.875919 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2442/4896] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:34:13 2022] GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1 => generated 1504 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:34:13.892559 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-af923e9d-ced1-43c5-b9cb-e8b78eeeb7e0 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:34:13.893042 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-af923e9d-ced1-43c5-b9cb-e8b78eeeb7e0 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Failed to fetch instance by id -1 {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2926}} Jul 16 21:34:13.893583 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-af923e9d-ced1-43c5-b9cb-e8b78eeeb7e0 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] HTTP exception thrown: Instance -1 could not be found. Jul 16 21:34:13.895815 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-af923e9d-ced1-43c5-b9cb-e8b78eeeb7e0 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Request method failure captured: Jul 16 21:34:13.895815 ncdv-master-1 devstack@n-api.service[138681]: request: DELETE /compute/v2.1/servers/-1 HTTP/1.1 Jul 16 21:34:13.895815 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:34:13.895815 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:34:13.895815 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:34:13.895815 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 21:34:13.895815 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:34:13.895815 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:34:13.895815 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:34:13.895815 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:34:13.895815 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0y6QJDVdPNcLbUnwwgOJV-QkddLQ7yqnzBNVp2CrOe5UQqjQOER3spl3PswEW82ISSx8FhmuIrEwFSG69Ze1pTOmWBWZ8PURIgVHbN3yBPA3U2S2TKkQkFwyNuvvV03MCr9U0QGJIjXVOkiqFEAy8q87U_okZgCdI-BFjBFgrCdfAEo Jul 16 21:34:13.895815 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:34:13.895815 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:34:13.895815 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:34:13.895815 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:34:13.895815 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:34:13.895815 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:34:13.895815 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: da2248fa190342f0bfe9211cae1e403b Jul 16 21:34:13.895815 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-ServersNegativeTestJSON-1102403437 Jul 16 21:34:13.895815 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:34:13.895815 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:34:13.895815 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/da2248fa190342f0bfe9211cae1e403b"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/da2248fa190342f0bfe9211cae1e403b"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_da2248fa190342f0bfe9211cae1e403b"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/da2248fa190342f0bfe9211cae1e403b"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:34:13.895815 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-ServersNegativeTestJSON-1102403437 Jul 16 21:34:13.895815 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: da2248fa190342f0bfe9211cae1e403b Jul 16 21:34:13.895815 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-ServersNegativeTestJSON-1102403437 Jul 16 21:34:13.895815 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-ServersNegativeTestJSON-1102403437-project Jul 16 21:34:13.895815 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:34:13.895815 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:34:13.895815 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: 22e264440fff43cdb7be6702839af37f Jul 16 21:34:13.895815 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-ServersNegativeTestJSON-1102403437-project Jul 16 21:34:13.895815 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:34:13.895815 ncdv-master-1 devstack@n-api.service[138681]: exception: Instance -1 could not be found. Jul 16 21:34:13.895815 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': '-1'} Jul 16 21:34:13.895815 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:34:13.895815 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:34:13.895815 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:34:13.895815 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:34:13.895815 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2927, in get Jul 16 21:34:13.895815 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=instance_id) Jul 16 21:34:13.895815 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance -1 could not be found. Jul 16 21:34:13.895815 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:34:13.895815 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:34:13.895815 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:34:13.895815 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:34:13.895815 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:34:13.895815 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:34:13.895815 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:34:13.895815 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:34:13.895815 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:34:13.895815 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:34:13.895815 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Jul 16 21:34:13.895815 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Jul 16 21:34:13.895815 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Jul 16 21:34:13.895815 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Jul 16 21:34:13.895815 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:34:13.895815 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:34:13.895815 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:34:13.895815 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:34:13.895815 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance -1 could not be found. Jul 16 21:34:13.895815 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:34:13.895815 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:34:13.895815 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:34:13.895815 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:34:13.895815 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:34:13.895815 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:34:13.901388 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:34:13.901388 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:34:13.901388 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance -1 could not be found. Jul 16 21:34:13.901388 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:34:13.901388 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-af923e9d-ced1-43c5-b9cb-e8b78eeeb7e0 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Returning 404 to user: Instance -1 could not be found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:34:13.901388 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-af923e9d-ced1-43c5-b9cb-e8b78eeeb7e0 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "DELETE /compute/v2.1/servers/-1" status: 404 len: 77 microversion: 2.1 time: 0.008596 Jul 16 21:34:13.901388 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2455/4897] 10.222.0.16 () {58 vars in 1136 bytes} [Sat Jul 16 21:34:13 2022] DELETE /compute/v2.1/servers/-1 => generated 77 bytes in 13 msecs (HTTP/1.1 404) 9 headers in 378 bytes (1 switches on core 0) Jul 16 21:34:13.906634 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-3aa5579f-616e-45da-bf67-31a3845dc4d2 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:34:13.907560 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-3aa5579f-616e-45da-bf67-31a3845dc4d2 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: e4c7a9cb-e55d-4819-9885-2e72deffddb1] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:34:13.914540 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3aa5579f-616e-45da-bf67-31a3845dc4d2 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:13.915161 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3aa5579f-616e-45da-bf67-31a3845dc4d2 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:13.916011 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3aa5579f-616e-45da-bf67-31a3845dc4d2 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:13.916541 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3aa5579f-616e-45da-bf67-31a3845dc4d2 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:14.144179 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-3aa5579f-616e-45da-bf67-31a3845dc4d2 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '22e264440fff43cdb7be6702839af37f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da2248fa190342f0bfe9211cae1e403b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:34:14.152510 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-3aa5579f-616e-45da-bf67-31a3845dc4d2 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1" status: 200 len: 1504 microversion: 2.1 time: 0.250382 Jul 16 21:34:14.153711 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2443/4898] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:34:13 2022] GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1 => generated 1504 bytes in 252 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:34:14.174239 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-793b1d10-ee06-4d7f-9ed9-5d1200002f3f tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:34:14.176443 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-793b1d10-ee06-4d7f-9ed9-5d1200002f3f tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: e4c7a9cb-e55d-4819-9885-2e72deffddb1] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:34:14.186468 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-793b1d10-ee06-4d7f-9ed9-5d1200002f3f tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:14.186933 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-793b1d10-ee06-4d7f-9ed9-5d1200002f3f tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:14.188082 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-793b1d10-ee06-4d7f-9ed9-5d1200002f3f tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:14.188715 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-793b1d10-ee06-4d7f-9ed9-5d1200002f3f tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:14.457326 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-793b1d10-ee06-4d7f-9ed9-5d1200002f3f tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '22e264440fff43cdb7be6702839af37f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da2248fa190342f0bfe9211cae1e403b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:34:14.468640 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-793b1d10-ee06-4d7f-9ed9-5d1200002f3f tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1" status: 200 len: 1504 microversion: 2.1 time: 0.301097 Jul 16 21:34:14.470353 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2456/4899] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:34:14 2022] GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1 => generated 1504 bytes in 302 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:34:14.486481 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-3ddebbdd-4266-402e-b1a1-e40e5a12ff24 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Action: 'action', calling method: >, body: {"forceDelete": {}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:34:14.487078 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-3ddebbdd-4266-402e-b1a1-e40e5a12ff24 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: 45062f11-b66f-4e09-ba92-6900ae6ff7e6] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:34:14.494711 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-3ddebbdd-4266-402e-b1a1-e40e5a12ff24 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] HTTP exception thrown: Instance 45062f11-b66f-4e09-ba92-6900ae6ff7e6 could not be found. Jul 16 21:34:14.498921 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-3ddebbdd-4266-402e-b1a1-e40e5a12ff24 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Request method failure captured: Jul 16 21:34:14.498921 ncdv-master-1 devstack@n-api.service[138680]: request: POST /compute/v2.1/servers/45062f11-b66f-4e09-ba92-6900ae6ff7e6/action HTTP/1.1 Jul 16 21:34:14.498921 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:34:14.498921 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:34:14.498921 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:34:14.498921 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 19 Jul 16 21:34:14.498921 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:34:14.498921 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:34:14.498921 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:34:14.498921 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:34:14.498921 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0y6QJDVdPNcLbUnwwgOJV-QkddLQ7yqnzBNVp2CrOe5UQqjQOER3spl3PswEW82ISSx8FhmuIrEwFSG69Ze1pTOmWBWZ8PURIgVHbN3yBPA3U2S2TKkQkFwyNuvvV03MCr9U0QGJIjXVOkiqFEAy8q87U_okZgCdI-BFjBFgrCdfAEo Jul 16 21:34:14.498921 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:34:14.498921 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:34:14.498921 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:34:14.498921 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:34:14.498921 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:34:14.498921 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:34:14.498921 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: da2248fa190342f0bfe9211cae1e403b Jul 16 21:34:14.498921 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-ServersNegativeTestJSON-1102403437 Jul 16 21:34:14.498921 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:34:14.498921 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:34:14.498921 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/da2248fa190342f0bfe9211cae1e403b"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/da2248fa190342f0bfe9211cae1e403b"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_da2248fa190342f0bfe9211cae1e403b"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/da2248fa190342f0bfe9211cae1e403b"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:34:14.498921 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-ServersNegativeTestJSON-1102403437 Jul 16 21:34:14.498921 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: da2248fa190342f0bfe9211cae1e403b Jul 16 21:34:14.498921 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-ServersNegativeTestJSON-1102403437 Jul 16 21:34:14.498921 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-ServersNegativeTestJSON-1102403437-project Jul 16 21:34:14.498921 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:34:14.498921 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:34:14.498921 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: 22e264440fff43cdb7be6702839af37f Jul 16 21:34:14.498921 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-ServersNegativeTestJSON-1102403437-project Jul 16 21:34:14.498921 ncdv-master-1 devstack@n-api.service[138680]: Jul 16 21:34:14.498921 ncdv-master-1 devstack@n-api.service[138680]: {"forceDelete": {}} Jul 16 21:34:14.498921 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 21:34:14.498921 ncdv-master-1 devstack@n-api.service[138680]: exception: Instance 45062f11-b66f-4e09-ba92-6900ae6ff7e6 could not be found. Jul 16 21:34:14.498921 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': '45062f11-b66f-4e09-ba92-6900ae6ff7e6', 'body': {'forceDelete': {}}} Jul 16 21:34:14.498921 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:34:14.498921 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:34:14.498921 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:34:14.498921 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:34:14.498921 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:34:14.498921 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:34:14.498921 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2898, in _get_instance Jul 16 21:34:14.498921 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=instance_uuid) Jul 16 21:34:14.498921 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 45062f11-b66f-4e09-ba92-6900ae6ff7e6 could not be found. Jul 16 21:34:14.498921 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:34:14.498921 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:34:14.498921 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:34:14.498921 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:34:14.498921 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:34:14.498921 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:34:14.498921 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:34:14.498921 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:34:14.498921 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:34:14.498921 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:34:14.498921 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/deferred_delete.py", line 55, in _force_delete Jul 16 21:34:14.498921 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, id) Jul 16 21:34:14.498921 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:34:14.498921 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:34:14.498921 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 45062f11-b66f-4e09-ba92-6900ae6ff7e6 could not be found. Jul 16 21:34:14.498921 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:34:14.498921 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:34:14.498921 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:34:14.498921 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:34:14.498921 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:34:14.498921 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:34:14.504869 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:34:14.504869 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:34:14.504869 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 45062f11-b66f-4e09-ba92-6900ae6ff7e6 could not be found. Jul 16 21:34:14.504869 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:34:14.506713 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-3ddebbdd-4266-402e-b1a1-e40e5a12ff24 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Returning 404 to user: Instance 45062f11-b66f-4e09-ba92-6900ae6ff7e6 could not be found. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:34:14.507749 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-3ddebbdd-4266-402e-b1a1-e40e5a12ff24 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "POST /compute/v2.1/servers/45062f11-b66f-4e09-ba92-6900ae6ff7e6/action" status: 404 len: 111 microversion: 2.1 time: 0.025686 Jul 16 21:34:14.508424 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2444/4900] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 21:34:14 2022] POST /compute/v2.1/servers/45062f11-b66f-4e09-ba92-6900ae6ff7e6/action => generated 111 bytes in 27 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:34:14.517734 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-a0a869f0-ba41-4a1a-bae3-190e858b3573 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:34:14.519067 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-a0a869f0-ba41-4a1a-bae3-190e858b3573 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: e4c7a9cb-e55d-4819-9885-2e72deffddb1] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:34:14.526187 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a0a869f0-ba41-4a1a-bae3-190e858b3573 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:14.526735 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a0a869f0-ba41-4a1a-bae3-190e858b3573 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:14.527644 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a0a869f0-ba41-4a1a-bae3-190e858b3573 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:14.528085 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a0a869f0-ba41-4a1a-bae3-190e858b3573 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:14.772062 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-a0a869f0-ba41-4a1a-bae3-190e858b3573 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '22e264440fff43cdb7be6702839af37f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da2248fa190342f0bfe9211cae1e403b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:34:14.781128 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-a0a869f0-ba41-4a1a-bae3-190e858b3573 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1" status: 200 len: 1504 microversion: 2.1 time: 0.268076 Jul 16 21:34:14.781737 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2457/4901] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:34:14 2022] GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1 => generated 1504 bytes in 269 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:34:14.799998 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-d2be8139-9c6e-4706-af64-c464867049e0 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:34:14.801106 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-d2be8139-9c6e-4706-af64-c464867049e0 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: e4c7a9cb-e55d-4819-9885-2e72deffddb1] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:34:14.808478 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d2be8139-9c6e-4706-af64-c464867049e0 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:14.808937 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d2be8139-9c6e-4706-af64-c464867049e0 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:14.809714 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d2be8139-9c6e-4706-af64-c464867049e0 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:14.810342 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d2be8139-9c6e-4706-af64-c464867049e0 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:15.104221 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-d2be8139-9c6e-4706-af64-c464867049e0 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '22e264440fff43cdb7be6702839af37f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da2248fa190342f0bfe9211cae1e403b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:34:15.114255 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-d2be8139-9c6e-4706-af64-c464867049e0 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1" status: 200 len: 1504 microversion: 2.1 time: 0.316371 Jul 16 21:34:15.129907 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-aa71c23a-a993-4146-9a3f-571f495405c2 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Action: 'action', calling method: >, body: {"os-getConsoleOutput": {"length": 10}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:34:15.130912 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-aa71c23a-a993-4146-9a3f-571f495405c2 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: 432cf21a-c114-44c4-b181-5dac510b198e] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:34:15.135204 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2445/4902] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:34:14 2022] GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1 => generated 1504 bytes in 340 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:34:15.137232 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-aa71c23a-a993-4146-9a3f-571f495405c2 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] HTTP exception thrown: Instance 432cf21a-c114-44c4-b181-5dac510b198e could not be found. Jul 16 21:34:15.144817 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-aa71c23a-a993-4146-9a3f-571f495405c2 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Request method failure captured: Jul 16 21:34:15.144817 ncdv-master-1 devstack@n-api.service[138681]: request: POST /compute/v2.1/servers/432cf21a-c114-44c4-b181-5dac510b198e/action HTTP/1.1 Jul 16 21:34:15.144817 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:34:15.144817 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:34:15.144817 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:34:15.144817 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 39 Jul 16 21:34:15.144817 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:34:15.144817 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:34:15.144817 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:34:15.144817 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:34:15.144817 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0y6QJDVdPNcLbUnwwgOJV-QkddLQ7yqnzBNVp2CrOe5UQqjQOER3spl3PswEW82ISSx8FhmuIrEwFSG69Ze1pTOmWBWZ8PURIgVHbN3yBPA3U2S2TKkQkFwyNuvvV03MCr9U0QGJIjXVOkiqFEAy8q87U_okZgCdI-BFjBFgrCdfAEo Jul 16 21:34:15.144817 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:34:15.144817 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:34:15.144817 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:34:15.144817 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:34:15.144817 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:34:15.144817 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:34:15.144817 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: da2248fa190342f0bfe9211cae1e403b Jul 16 21:34:15.144817 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-ServersNegativeTestJSON-1102403437 Jul 16 21:34:15.144817 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:34:15.144817 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:34:15.144817 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/da2248fa190342f0bfe9211cae1e403b"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/da2248fa190342f0bfe9211cae1e403b"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_da2248fa190342f0bfe9211cae1e403b"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/da2248fa190342f0bfe9211cae1e403b"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:34:15.144817 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-ServersNegativeTestJSON-1102403437 Jul 16 21:34:15.144817 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: da2248fa190342f0bfe9211cae1e403b Jul 16 21:34:15.144817 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-ServersNegativeTestJSON-1102403437 Jul 16 21:34:15.144817 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-ServersNegativeTestJSON-1102403437-project Jul 16 21:34:15.144817 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:34:15.144817 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:34:15.144817 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: 22e264440fff43cdb7be6702839af37f Jul 16 21:34:15.144817 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-ServersNegativeTestJSON-1102403437-project Jul 16 21:34:15.144817 ncdv-master-1 devstack@n-api.service[138681]: Jul 16 21:34:15.144817 ncdv-master-1 devstack@n-api.service[138681]: {"os-getConsoleOutput": {"length": 10}} Jul 16 21:34:15.144817 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:34:15.144817 ncdv-master-1 devstack@n-api.service[138681]: exception: Instance 432cf21a-c114-44c4-b181-5dac510b198e could not be found. Jul 16 21:34:15.144817 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': '432cf21a-c114-44c4-b181-5dac510b198e', 'body': {'os-getConsoleOutput': {'length': 10}}} Jul 16 21:34:15.144817 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:34:15.144817 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:34:15.144817 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:34:15.144817 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:34:15.144817 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:34:15.144817 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:34:15.144817 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2898, in _get_instance Jul 16 21:34:15.144817 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=instance_uuid) Jul 16 21:34:15.144817 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 432cf21a-c114-44c4-b181-5dac510b198e could not be found. Jul 16 21:34:15.144817 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:34:15.144817 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:34:15.144817 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:34:15.144817 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:34:15.144817 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:34:15.144817 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:34:15.144817 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:34:15.144817 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:34:15.144817 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:34:15.144817 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:34:15.144817 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 16 21:34:15.144817 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 16 21:34:15.144817 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/console_output.py", line 41, in get_console_output Jul 16 21:34:15.144817 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, id) Jul 16 21:34:15.144817 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:34:15.144817 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:34:15.144817 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 432cf21a-c114-44c4-b181-5dac510b198e could not be found. Jul 16 21:34:15.144817 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:34:15.144817 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:34:15.144817 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:34:15.144817 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:34:15.150527 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:34:15.150527 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:34:15.150527 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:34:15.150527 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:34:15.150527 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 432cf21a-c114-44c4-b181-5dac510b198e could not be found. Jul 16 21:34:15.150527 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:34:15.150527 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-aa71c23a-a993-4146-9a3f-571f495405c2 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Returning 404 to user: Instance 432cf21a-c114-44c4-b181-5dac510b198e could not be found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:34:15.150527 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-aa71c23a-a993-4146-9a3f-571f495405c2 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "POST /compute/v2.1/servers/432cf21a-c114-44c4-b181-5dac510b198e/action" status: 404 len: 111 microversion: 2.1 time: 0.020834 Jul 16 21:34:15.155990 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-394183f8-55c6-4879-b198-5b4ef26ba137 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:34:15.156779 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-394183f8-55c6-4879-b198-5b4ef26ba137 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: e4c7a9cb-e55d-4819-9885-2e72deffddb1] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:34:15.160328 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2458/4903] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 21:34:15 2022] POST /compute/v2.1/servers/432cf21a-c114-44c4-b181-5dac510b198e/action => generated 111 bytes in 25 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:34:15.171270 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-394183f8-55c6-4879-b198-5b4ef26ba137 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:15.171737 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-394183f8-55c6-4879-b198-5b4ef26ba137 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:15.172711 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-394183f8-55c6-4879-b198-5b4ef26ba137 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:15.173408 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-394183f8-55c6-4879-b198-5b4ef26ba137 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:15.692598 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-394183f8-55c6-4879-b198-5b4ef26ba137 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '22e264440fff43cdb7be6702839af37f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da2248fa190342f0bfe9211cae1e403b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:34:15.727640 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-394183f8-55c6-4879-b198-5b4ef26ba137 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1" status: 200 len: 1504 microversion: 2.1 time: 0.575420 Jul 16 21:34:15.730665 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2446/4904] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:34:15 2022] GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1 => generated 1504 bytes in 578 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:34:15.756142 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-3aa873b0-2344-4762-a21b-ba0e0eafff2e tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:34:15.756750 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-3aa873b0-2344-4762-a21b-ba0e0eafff2e tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: e4c7a9cb-e55d-4819-9885-2e72deffddb1] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:34:15.766730 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3aa873b0-2344-4762-a21b-ba0e0eafff2e tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:15.770845 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3aa873b0-2344-4762-a21b-ba0e0eafff2e tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:15.770845 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3aa873b0-2344-4762-a21b-ba0e0eafff2e tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:15.770845 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3aa873b0-2344-4762-a21b-ba0e0eafff2e tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:16.300584 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-3aa873b0-2344-4762-a21b-ba0e0eafff2e tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '22e264440fff43cdb7be6702839af37f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da2248fa190342f0bfe9211cae1e403b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:34:16.311052 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-3aa873b0-2344-4762-a21b-ba0e0eafff2e tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1" status: 200 len: 1504 microversion: 2.1 time: 0.559995 Jul 16 21:34:16.329992 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2459/4905] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:34:15 2022] GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1 => generated 1504 bytes in 579 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:34:16.336225 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-cfcdb0f9-6fc4-49cc-9288-8f8911f74b04 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:34:16.336849 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-cfcdb0f9-6fc4-49cc-9288-8f8911f74b04 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: a9979583-36ca-4a84-a7b2-2d0e9cdadf62] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:34:16.345566 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-cfcdb0f9-6fc4-49cc-9288-8f8911f74b04 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] HTTP exception thrown: Instance a9979583-36ca-4a84-a7b2-2d0e9cdadf62 could not be found. Jul 16 21:34:16.349138 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-cfcdb0f9-6fc4-49cc-9288-8f8911f74b04 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Request method failure captured: Jul 16 21:34:16.349138 ncdv-master-1 devstack@n-api.service[138680]: request: GET /compute/v2.1/servers/a9979583-36ca-4a84-a7b2-2d0e9cdadf62 HTTP/1.1 Jul 16 21:34:16.349138 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:34:16.349138 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:34:16.349138 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:34:16.349138 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 0 Jul 16 21:34:16.349138 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:34:16.349138 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:34:16.349138 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:34:16.349138 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:34:16.349138 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0y6QJDVdPNcLbUnwwgOJV-QkddLQ7yqnzBNVp2CrOe5UQqjQOER3spl3PswEW82ISSx8FhmuIrEwFSG69Ze1pTOmWBWZ8PURIgVHbN3yBPA3U2S2TKkQkFwyNuvvV03MCr9U0QGJIjXVOkiqFEAy8q87U_okZgCdI-BFjBFgrCdfAEo Jul 16 21:34:16.349138 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:34:16.349138 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:34:16.349138 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:34:16.349138 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:34:16.349138 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:34:16.349138 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:34:16.349138 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: da2248fa190342f0bfe9211cae1e403b Jul 16 21:34:16.349138 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-ServersNegativeTestJSON-1102403437 Jul 16 21:34:16.349138 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:34:16.349138 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:34:16.349138 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/da2248fa190342f0bfe9211cae1e403b"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/da2248fa190342f0bfe9211cae1e403b"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_da2248fa190342f0bfe9211cae1e403b"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/da2248fa190342f0bfe9211cae1e403b"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:34:16.349138 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-ServersNegativeTestJSON-1102403437 Jul 16 21:34:16.349138 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: da2248fa190342f0bfe9211cae1e403b Jul 16 21:34:16.349138 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-ServersNegativeTestJSON-1102403437 Jul 16 21:34:16.349138 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-ServersNegativeTestJSON-1102403437-project Jul 16 21:34:16.349138 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:34:16.349138 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:34:16.349138 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: 22e264440fff43cdb7be6702839af37f Jul 16 21:34:16.349138 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-ServersNegativeTestJSON-1102403437-project Jul 16 21:34:16.349138 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 21:34:16.349138 ncdv-master-1 devstack@n-api.service[138680]: exception: Instance a9979583-36ca-4a84-a7b2-2d0e9cdadf62 could not be found. Jul 16 21:34:16.349138 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': 'a9979583-36ca-4a84-a7b2-2d0e9cdadf62'} Jul 16 21:34:16.349138 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:34:16.349138 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:34:16.349138 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:34:16.349138 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:34:16.349138 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:34:16.349138 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:34:16.349138 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2898, in _get_instance Jul 16 21:34:16.349138 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=instance_uuid) Jul 16 21:34:16.349138 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance a9979583-36ca-4a84-a7b2-2d0e9cdadf62 could not be found. Jul 16 21:34:16.349138 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:34:16.349138 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:34:16.349138 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:34:16.349138 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:34:16.349138 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:34:16.349138 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:34:16.349138 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:34:16.349138 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:34:16.349138 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:34:16.349138 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:34:16.349138 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:34:16.349138 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:34:16.349138 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:34:16.349138 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:34:16.349138 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:34:16.349138 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:34:16.349138 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance a9979583-36ca-4a84-a7b2-2d0e9cdadf62 could not be found. Jul 16 21:34:16.349138 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:34:16.349138 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:34:16.349138 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:34:16.349138 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:34:16.349138 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:34:16.354722 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:34:16.354722 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:34:16.354722 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:34:16.354722 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance a9979583-36ca-4a84-a7b2-2d0e9cdadf62 could not be found. Jul 16 21:34:16.354722 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:34:16.356376 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-cfcdb0f9-6fc4-49cc-9288-8f8911f74b04 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Returning 404 to user: Instance a9979583-36ca-4a84-a7b2-2d0e9cdadf62 could not be found. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:34:16.357444 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-cfcdb0f9-6fc4-49cc-9288-8f8911f74b04 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "GET /compute/v2.1/servers/a9979583-36ca-4a84-a7b2-2d0e9cdadf62" status: 404 len: 111 microversion: 2.1 time: 0.025648 Jul 16 21:34:16.368392 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-cbd4f621-0f9f-402a-afe2-5f08a488a26a tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:34:16.369400 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-cbd4f621-0f9f-402a-afe2-5f08a488a26a tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: e4c7a9cb-e55d-4819-9885-2e72deffddb1] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:34:16.372900 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2447/4906] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:34:16 2022] GET /compute/v2.1/servers/a9979583-36ca-4a84-a7b2-2d0e9cdadf62 => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:34:16.377056 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cbd4f621-0f9f-402a-afe2-5f08a488a26a tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:16.378595 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cbd4f621-0f9f-402a-afe2-5f08a488a26a tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:16.379857 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cbd4f621-0f9f-402a-afe2-5f08a488a26a tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:16.380491 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cbd4f621-0f9f-402a-afe2-5f08a488a26a tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:16.635822 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-cbd4f621-0f9f-402a-afe2-5f08a488a26a tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '22e264440fff43cdb7be6702839af37f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da2248fa190342f0bfe9211cae1e403b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:34:16.642680 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-cbd4f621-0f9f-402a-afe2-5f08a488a26a tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1" status: 200 len: 1504 microversion: 2.1 time: 0.278882 Jul 16 21:34:16.643426 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2460/4907] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:34:16 2022] GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1 => generated 1504 bytes in 281 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:34:16.660824 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-0f96111d-1bb2-4b79-9e46-0bba96848ab2 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:34:16.661657 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-0f96111d-1bb2-4b79-9e46-0bba96848ab2 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: e4c7a9cb-e55d-4819-9885-2e72deffddb1] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:34:16.668558 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0f96111d-1bb2-4b79-9e46-0bba96848ab2 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:16.669013 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0f96111d-1bb2-4b79-9e46-0bba96848ab2 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:16.669784 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0f96111d-1bb2-4b79-9e46-0bba96848ab2 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:16.670225 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0f96111d-1bb2-4b79-9e46-0bba96848ab2 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:16.860017 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-0f96111d-1bb2-4b79-9e46-0bba96848ab2 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '22e264440fff43cdb7be6702839af37f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da2248fa190342f0bfe9211cae1e403b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:34:16.883661 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-0f96111d-1bb2-4b79-9e46-0bba96848ab2 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1" status: 200 len: 1504 microversion: 2.1 time: 0.226846 Jul 16 21:34:16.884247 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2448/4908] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:34:16 2022] GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1 => generated 1504 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:34:16.994255 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-bd9a98ab-682f-457c-ac40-91838f6f534c tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServersNegativeTestJSON-server-1091086371", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452", "accessIPv4": "1.1.1.1.1.1", "networks": [{"uuid": "f14874ef-c1c1-4761-ab34-120b93719d08"}]}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:34:16.996885 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-bd9a98ab-682f-457c-ac40-91838f6f534c tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Request method failure captured: Jul 16 21:34:16.996885 ncdv-master-1 devstack@n-api.service[138681]: request: POST /compute/v2.1/servers HTTP/1.1 Jul 16 21:34:16.996885 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:34:16.996885 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:34:16.996885 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:34:16.996885 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 238 Jul 16 21:34:16.996885 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:34:16.996885 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:34:16.996885 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:34:16.996885 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:34:16.996885 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0y6QJDVdPNcLbUnwwgOJV-QkddLQ7yqnzBNVp2CrOe5UQqjQOER3spl3PswEW82ISSx8FhmuIrEwFSG69Ze1pTOmWBWZ8PURIgVHbN3yBPA3U2S2TKkQkFwyNuvvV03MCr9U0QGJIjXVOkiqFEAy8q87U_okZgCdI-BFjBFgrCdfAEo Jul 16 21:34:16.996885 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:34:16.996885 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:34:16.996885 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:34:16.996885 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:34:16.996885 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:34:16.996885 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:34:16.996885 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: da2248fa190342f0bfe9211cae1e403b Jul 16 21:34:16.996885 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-ServersNegativeTestJSON-1102403437 Jul 16 21:34:16.996885 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:34:16.996885 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:34:16.996885 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/da2248fa190342f0bfe9211cae1e403b"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/da2248fa190342f0bfe9211cae1e403b"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_da2248fa190342f0bfe9211cae1e403b"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/da2248fa190342f0bfe9211cae1e403b"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:34:16.996885 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-ServersNegativeTestJSON-1102403437 Jul 16 21:34:16.996885 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: da2248fa190342f0bfe9211cae1e403b Jul 16 21:34:16.996885 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-ServersNegativeTestJSON-1102403437 Jul 16 21:34:16.996885 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-ServersNegativeTestJSON-1102403437-project Jul 16 21:34:16.996885 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:34:16.996885 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:34:16.996885 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: 22e264440fff43cdb7be6702839af37f Jul 16 21:34:16.996885 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-ServersNegativeTestJSON-1102403437-project Jul 16 21:34:16.996885 ncdv-master-1 devstack@n-api.service[138681]: Jul 16 21:34:16.996885 ncdv-master-1 devstack@n-api.service[138681]: {"server": {"name": "tempest-ServersNegativeTestJSON-server-1091086371", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452", "accessIPv4": "1.1.1.1.1.1", "networks": [{"uuid": "f14874ef-c1c1-4761-ab34-120b93719d08"}]}} Jul 16 21:34:16.996885 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:34:16.996885 ncdv-master-1 devstack@n-api.service[138681]: exception: Invalid input for field/attribute accessIPv4. Value: 1.1.1.1.1.1. '1.1.1.1.1.1' is not a 'ipv4' Jul 16 21:34:16.996885 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'body': {'server': {'name': 'tempest-ServersNegativeTestJSON-server-1091086371', 'imageRef': '1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9', 'flavorRef': '452', 'accessIPv4': '1.1.1.1.1.1', 'networks': [{'uuid': 'f14874ef-c1c1-4761-ab34-120b93719d08'}]}}} Jul 16 21:34:16.996885 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:34:16.996885 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:34:16.996885 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 279, in validate Jul 16 21:34:16.996885 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Jul 16 21:34:16.996885 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/lib/python3/dist-packages/jsonschema/validators.py", line 353, in validate Jul 16 21:34:16.996885 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise error Jul 16 21:34:16.996885 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: '1.1.1.1.1.1' is not a 'ipv4' Jul 16 21:34:16.996885 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:34:16.996885 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Failed validating 'format' in schema['properties']['server']['properties']['accessIPv4']: Jul 16 21:34:16.996885 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi {'format': 'ipv4', 'type': 'string'} Jul 16 21:34:16.996885 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:34:16.996885 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi On instance['server']: Jul 16 21:34:16.996885 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi '1.1.1.1.1.1' Jul 16 21:34:16.996885 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:34:16.996885 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:34:16.996885 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:34:16.996885 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:34:16.996885 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:34:16.996885 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:34:16.996885 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:34:16.996885 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:34:16.996885 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:34:16.996885 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:34:16.996885 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 16 21:34:16.996885 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 16 21:34:16.996885 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Jul 16 21:34:16.996885 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Jul 16 21:34:16.996885 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Jul 16 21:34:16.996885 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Jul 16 21:34:17.002107 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 308, in validate Jul 16 21:34:17.002107 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Jul 16 21:34:17.002107 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute accessIPv4. Value: 1.1.1.1.1.1. '1.1.1.1.1.1' is not a 'ipv4' Jul 16 21:34:17.002107 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:34:17.002107 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:34:17.002107 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:34:17.002107 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:34:17.002107 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:34:17.002107 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:34:17.002107 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Jul 16 21:34:17.002107 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Jul 16 21:34:17.002107 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute accessIPv4. Value: 1.1.1.1.1.1. '1.1.1.1.1.1' is not a 'ipv4' Jul 16 21:34:17.002107 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:34:17.005006 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-bd9a98ab-682f-457c-ac40-91838f6f534c tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Returning 400 to user: Invalid input for field/attribute accessIPv4. Value: 1.1.1.1.1.1. '1.1.1.1.1.1' is not a 'ipv4' {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:34:17.005878 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-bd9a98ab-682f-457c-ac40-91838f6f534c tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 400 len: 139 microversion: 2.1 time: 0.015684 Jul 16 21:34:17.006725 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2461/4909] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 21:34:16 2022] POST /compute/v2.1/servers => generated 139 bytes in 17 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Jul 16 21:34:17.018798 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-ee0ec331-4650-4b3f-bf1f-f39c2cd30c66 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:34:17.019972 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-ee0ec331-4650-4b3f-bf1f-f39c2cd30c66 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: e4c7a9cb-e55d-4819-9885-2e72deffddb1] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:34:17.033084 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ee0ec331-4650-4b3f-bf1f-f39c2cd30c66 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:17.033826 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ee0ec331-4650-4b3f-bf1f-f39c2cd30c66 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:17.035481 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ee0ec331-4650-4b3f-bf1f-f39c2cd30c66 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:17.036723 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ee0ec331-4650-4b3f-bf1f-f39c2cd30c66 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:17.723826 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-ee0ec331-4650-4b3f-bf1f-f39c2cd30c66 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '22e264440fff43cdb7be6702839af37f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da2248fa190342f0bfe9211cae1e403b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:34:17.745201 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-ee0ec331-4650-4b3f-bf1f-f39c2cd30c66 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1" status: 200 len: 1504 microversion: 2.1 time: 0.731174 Jul 16 21:34:17.746430 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2449/4910] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:34:17 2022] GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1 => generated 1504 bytes in 734 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:34:17.765368 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-637c434a-16d1-4210-9054-6cf6669d64a4 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:34:17.766172 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-637c434a-16d1-4210-9054-6cf6669d64a4 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: e4c7a9cb-e55d-4819-9885-2e72deffddb1] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:34:17.784923 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-637c434a-16d1-4210-9054-6cf6669d64a4 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:17.785414 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-637c434a-16d1-4210-9054-6cf6669d64a4 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:17.786227 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-637c434a-16d1-4210-9054-6cf6669d64a4 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:17.786669 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-637c434a-16d1-4210-9054-6cf6669d64a4 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:18.117702 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-637c434a-16d1-4210-9054-6cf6669d64a4 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '22e264440fff43cdb7be6702839af37f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da2248fa190342f0bfe9211cae1e403b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:34:18.125503 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-637c434a-16d1-4210-9054-6cf6669d64a4 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1" status: 200 len: 1504 microversion: 2.1 time: 0.364853 Jul 16 21:34:18.126163 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2462/4911] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:34:17 2022] GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1 => generated 1504 bytes in 366 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:34:18.197344 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-ff2a1b3f-52f4-4cf3-906e-7a4c9247aa1f tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServersNegativeTestJSON-server-524617422", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452", "accessIPv6": "notvalid", "networks": [{"uuid": "f14874ef-c1c1-4761-ab34-120b93719d08"}]}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:34:18.211923 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-ff2a1b3f-52f4-4cf3-906e-7a4c9247aa1f tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Request method failure captured: Jul 16 21:34:18.211923 ncdv-master-1 devstack@n-api.service[138680]: request: POST /compute/v2.1/servers HTTP/1.1 Jul 16 21:34:18.211923 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:34:18.211923 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:34:18.211923 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:34:18.211923 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 234 Jul 16 21:34:18.211923 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:34:18.211923 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:34:18.211923 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:34:18.211923 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:34:18.211923 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0y6QJDVdPNcLbUnwwgOJV-QkddLQ7yqnzBNVp2CrOe5UQqjQOER3spl3PswEW82ISSx8FhmuIrEwFSG69Ze1pTOmWBWZ8PURIgVHbN3yBPA3U2S2TKkQkFwyNuvvV03MCr9U0QGJIjXVOkiqFEAy8q87U_okZgCdI-BFjBFgrCdfAEo Jul 16 21:34:18.211923 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:34:18.211923 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:34:18.211923 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:34:18.211923 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:34:18.211923 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:34:18.211923 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:34:18.211923 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: da2248fa190342f0bfe9211cae1e403b Jul 16 21:34:18.211923 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-ServersNegativeTestJSON-1102403437 Jul 16 21:34:18.211923 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:34:18.211923 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:34:18.211923 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/da2248fa190342f0bfe9211cae1e403b"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/da2248fa190342f0bfe9211cae1e403b"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_da2248fa190342f0bfe9211cae1e403b"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/da2248fa190342f0bfe9211cae1e403b"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:34:18.211923 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-ServersNegativeTestJSON-1102403437 Jul 16 21:34:18.211923 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: da2248fa190342f0bfe9211cae1e403b Jul 16 21:34:18.211923 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-ServersNegativeTestJSON-1102403437 Jul 16 21:34:18.211923 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-ServersNegativeTestJSON-1102403437-project Jul 16 21:34:18.211923 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:34:18.211923 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:34:18.211923 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: 22e264440fff43cdb7be6702839af37f Jul 16 21:34:18.211923 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-ServersNegativeTestJSON-1102403437-project Jul 16 21:34:18.211923 ncdv-master-1 devstack@n-api.service[138680]: Jul 16 21:34:18.211923 ncdv-master-1 devstack@n-api.service[138680]: {"server": {"name": "tempest-ServersNegativeTestJSON-server-524617422", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452", "accessIPv6": "notvalid", "networks": [{"uuid": "f14874ef-c1c1-4761-ab34-120b93719d08"}]}} Jul 16 21:34:18.211923 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 21:34:18.211923 ncdv-master-1 devstack@n-api.service[138680]: exception: Invalid input for field/attribute accessIPv6. Value: notvalid. 'notvalid' is not a 'ipv6' Jul 16 21:34:18.211923 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'body': {'server': {'name': 'tempest-ServersNegativeTestJSON-server-524617422', 'imageRef': '1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9', 'flavorRef': '452', 'accessIPv6': 'notvalid', 'networks': [{'uuid': 'f14874ef-c1c1-4761-ab34-120b93719d08'}]}}} Jul 16 21:34:18.211923 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:34:18.211923 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:34:18.211923 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 240, in check Jul 16 21:34:18.211923 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi result = func(instance) Jul 16 21:34:18.211923 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/lib/python3/dist-packages/jsonschema/_format.py", line 210, in is_ipv6 Jul 16 21:34:18.211923 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return socket.inet_pton(socket.AF_INET6, instance) Jul 16 21:34:18.211923 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi OSError: illegal IP address string passed to inet_pton Jul 16 21:34:18.211923 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:34:18.211923 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi The above exception was the direct cause of the following exception: Jul 16 21:34:18.211923 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:34:18.211923 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:34:18.211923 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 279, in validate Jul 16 21:34:18.211923 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Jul 16 21:34:18.211923 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/lib/python3/dist-packages/jsonschema/validators.py", line 353, in validate Jul 16 21:34:18.211923 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise error Jul 16 21:34:18.211923 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: 'notvalid' is not a 'ipv6' Jul 16 21:34:18.211923 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:34:18.211923 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Failed validating 'format' in schema['properties']['server']['properties']['accessIPv6']: Jul 16 21:34:18.211923 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi {'format': 'ipv6', 'type': 'string'} Jul 16 21:34:18.211923 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:34:18.211923 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi On instance['server']: Jul 16 21:34:18.211923 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi 'notvalid' Jul 16 21:34:18.211923 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:34:18.211923 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:34:18.211923 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:34:18.211923 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:34:18.211923 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:34:18.211923 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:34:18.211923 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:34:18.211923 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:34:18.217498 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:34:18.217498 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:34:18.217498 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 16 21:34:18.217498 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 16 21:34:18.217498 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Jul 16 21:34:18.217498 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Jul 16 21:34:18.217498 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Jul 16 21:34:18.217498 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Jul 16 21:34:18.217498 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 308, in validate Jul 16 21:34:18.217498 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Jul 16 21:34:18.217498 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute accessIPv6. Value: notvalid. 'notvalid' is not a 'ipv6' Jul 16 21:34:18.217498 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:34:18.217498 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:34:18.217498 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:34:18.217498 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:34:18.217498 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:34:18.217498 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:34:18.217498 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Jul 16 21:34:18.217498 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Jul 16 21:34:18.217498 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute accessIPv6. Value: notvalid. 'notvalid' is not a 'ipv6' Jul 16 21:34:18.217498 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:34:18.223770 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-ff2a1b3f-52f4-4cf3-906e-7a4c9247aa1f tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Returning 400 to user: Invalid input for field/attribute accessIPv6. Value: notvalid. 'notvalid' is not a 'ipv6' {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:34:18.223770 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-ff2a1b3f-52f4-4cf3-906e-7a4c9247aa1f tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 400 len: 133 microversion: 2.1 time: 0.035187 Jul 16 21:34:18.223770 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2450/4912] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 21:34:18 2022] POST /compute/v2.1/servers => generated 133 bytes in 37 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Jul 16 21:34:18.232659 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-82f462ed-d59e-4e17-a52e-ff374c7ba114 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:34:18.233283 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-82f462ed-d59e-4e17-a52e-ff374c7ba114 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: e4c7a9cb-e55d-4819-9885-2e72deffddb1] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:34:18.242135 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_api:SELECT=76 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:34:18.243728 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-82f462ed-d59e-4e17-a52e-ff374c7ba114 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:18.244377 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-82f462ed-d59e-4e17-a52e-ff374c7ba114 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:18.246473 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-82f462ed-d59e-4e17-a52e-ff374c7ba114 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:18.246473 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-82f462ed-d59e-4e17-a52e-ff374c7ba114 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:18.478947 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-82f462ed-d59e-4e17-a52e-ff374c7ba114 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '22e264440fff43cdb7be6702839af37f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da2248fa190342f0bfe9211cae1e403b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:34:18.489438 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-82f462ed-d59e-4e17-a52e-ff374c7ba114 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1" status: 200 len: 1504 microversion: 2.1 time: 0.261051 Jul 16 21:34:18.490250 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2463/4913] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:34:18 2022] GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1 => generated 1504 bytes in 264 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:34:18.509178 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-b3cd0497-5399-43f7-b45c-b4b5278ded4f tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:34:18.509998 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-b3cd0497-5399-43f7-b45c-b4b5278ded4f tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: e4c7a9cb-e55d-4819-9885-2e72deffddb1] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:34:18.519815 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b3cd0497-5399-43f7-b45c-b4b5278ded4f tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:18.520368 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b3cd0497-5399-43f7-b45c-b4b5278ded4f tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:18.521331 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b3cd0497-5399-43f7-b45c-b4b5278ded4f tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:18.522752 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b3cd0497-5399-43f7-b45c-b4b5278ded4f tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:18.754986 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-b3cd0497-5399-43f7-b45c-b4b5278ded4f tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '22e264440fff43cdb7be6702839af37f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da2248fa190342f0bfe9211cae1e403b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:34:18.764478 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-b3cd0497-5399-43f7-b45c-b4b5278ded4f tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1" status: 200 len: 1504 microversion: 2.1 time: 0.259206 Jul 16 21:34:18.765670 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2451/4914] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:34:18 2022] GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1 => generated 1504 bytes in 261 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:34:18.782094 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-d46561a1-8019-4a81-961f-c6c6cf005ec0 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Action: 'action', calling method: >, body: {"pause": {}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:34:18.782948 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-d46561a1-8019-4a81-961f-c6c6cf005ec0 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: 5f132729-fcb9-4bd2-a590-ba46d43344c3] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:34:18.794168 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-d46561a1-8019-4a81-961f-c6c6cf005ec0 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] HTTP exception thrown: Instance 5f132729-fcb9-4bd2-a590-ba46d43344c3 could not be found. Jul 16 21:34:18.798458 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-d46561a1-8019-4a81-961f-c6c6cf005ec0 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Request method failure captured: Jul 16 21:34:18.798458 ncdv-master-1 devstack@n-api.service[138681]: request: POST /compute/v2.1/servers/5f132729-fcb9-4bd2-a590-ba46d43344c3/action HTTP/1.1 Jul 16 21:34:18.798458 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:34:18.798458 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:34:18.798458 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:34:18.798458 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 13 Jul 16 21:34:18.798458 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:34:18.798458 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:34:18.798458 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:34:18.798458 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:34:18.798458 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0y6QJDVdPNcLbUnwwgOJV-QkddLQ7yqnzBNVp2CrOe5UQqjQOER3spl3PswEW82ISSx8FhmuIrEwFSG69Ze1pTOmWBWZ8PURIgVHbN3yBPA3U2S2TKkQkFwyNuvvV03MCr9U0QGJIjXVOkiqFEAy8q87U_okZgCdI-BFjBFgrCdfAEo Jul 16 21:34:18.798458 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:34:18.798458 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:34:18.798458 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:34:18.798458 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:34:18.798458 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:34:18.798458 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:34:18.798458 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: da2248fa190342f0bfe9211cae1e403b Jul 16 21:34:18.798458 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-ServersNegativeTestJSON-1102403437 Jul 16 21:34:18.798458 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:34:18.798458 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:34:18.798458 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/da2248fa190342f0bfe9211cae1e403b"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/da2248fa190342f0bfe9211cae1e403b"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_da2248fa190342f0bfe9211cae1e403b"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/da2248fa190342f0bfe9211cae1e403b"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:34:18.798458 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-ServersNegativeTestJSON-1102403437 Jul 16 21:34:18.798458 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: da2248fa190342f0bfe9211cae1e403b Jul 16 21:34:18.798458 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-ServersNegativeTestJSON-1102403437 Jul 16 21:34:18.798458 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-ServersNegativeTestJSON-1102403437-project Jul 16 21:34:18.798458 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:34:18.798458 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:34:18.798458 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: 22e264440fff43cdb7be6702839af37f Jul 16 21:34:18.798458 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-ServersNegativeTestJSON-1102403437-project Jul 16 21:34:18.798458 ncdv-master-1 devstack@n-api.service[138681]: Jul 16 21:34:18.798458 ncdv-master-1 devstack@n-api.service[138681]: {"pause": {}} Jul 16 21:34:18.798458 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:34:18.798458 ncdv-master-1 devstack@n-api.service[138681]: exception: Instance 5f132729-fcb9-4bd2-a590-ba46d43344c3 could not be found. Jul 16 21:34:18.798458 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': '5f132729-fcb9-4bd2-a590-ba46d43344c3', 'body': {'pause': {}}} Jul 16 21:34:18.798458 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:34:18.798458 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:34:18.798458 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:34:18.798458 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:34:18.798458 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:34:18.798458 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:34:18.798458 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2898, in _get_instance Jul 16 21:34:18.798458 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=instance_uuid) Jul 16 21:34:18.798458 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 5f132729-fcb9-4bd2-a590-ba46d43344c3 could not be found. Jul 16 21:34:18.798458 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:34:18.798458 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:34:18.798458 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:34:18.798458 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:34:18.798458 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:34:18.798458 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:34:18.798458 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:34:18.798458 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:34:18.798458 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:34:18.798458 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:34:18.798458 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/pause_server.py", line 36, in _pause Jul 16 21:34:18.798458 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi server = common.get_instance(self.compute_api, ctxt, id) Jul 16 21:34:18.798458 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:34:18.798458 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:34:18.798458 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 5f132729-fcb9-4bd2-a590-ba46d43344c3 could not be found. Jul 16 21:34:18.798458 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:34:18.798458 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:34:18.798458 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:34:18.798458 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:34:18.798458 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:34:18.798458 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:34:18.806923 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:34:18.806923 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:34:18.806923 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 5f132729-fcb9-4bd2-a590-ba46d43344c3 could not be found. Jul 16 21:34:18.806923 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:34:18.806923 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-d46561a1-8019-4a81-961f-c6c6cf005ec0 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Returning 404 to user: Instance 5f132729-fcb9-4bd2-a590-ba46d43344c3 could not be found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:34:18.806923 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-d46561a1-8019-4a81-961f-c6c6cf005ec0 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "POST /compute/v2.1/servers/5f132729-fcb9-4bd2-a590-ba46d43344c3/action" status: 404 len: 111 microversion: 2.1 time: 0.022408 Jul 16 21:34:18.806923 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2464/4915] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 21:34:18 2022] POST /compute/v2.1/servers/5f132729-fcb9-4bd2-a590-ba46d43344c3/action => generated 111 bytes in 24 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:34:18.809566 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-ce24397e-5364-4b59-bcf7-6dd6eca4bad8 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:34:18.810099 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-ce24397e-5364-4b59-bcf7-6dd6eca4bad8 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: e4c7a9cb-e55d-4819-9885-2e72deffddb1] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:34:18.817095 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ce24397e-5364-4b59-bcf7-6dd6eca4bad8 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:18.827326 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ce24397e-5364-4b59-bcf7-6dd6eca4bad8 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:18.830824 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ce24397e-5364-4b59-bcf7-6dd6eca4bad8 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:18.830824 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ce24397e-5364-4b59-bcf7-6dd6eca4bad8 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:19.227466 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-ce24397e-5364-4b59-bcf7-6dd6eca4bad8 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '22e264440fff43cdb7be6702839af37f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da2248fa190342f0bfe9211cae1e403b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:34:19.250802 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-ce24397e-5364-4b59-bcf7-6dd6eca4bad8 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1" status: 200 len: 1504 microversion: 2.1 time: 0.433171 Jul 16 21:34:19.250802 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2452/4916] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:34:18 2022] GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1 => generated 1504 bytes in 435 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:34:19.256573 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-a9288db9-2593-4367-80b3-37a926fb3197 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:34:19.257253 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-a9288db9-2593-4367-80b3-37a926fb3197 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: e4c7a9cb-e55d-4819-9885-2e72deffddb1] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:34:19.274468 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a9288db9-2593-4367-80b3-37a926fb3197 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:19.276415 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a9288db9-2593-4367-80b3-37a926fb3197 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:19.277365 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a9288db9-2593-4367-80b3-37a926fb3197 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:19.280026 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a9288db9-2593-4367-80b3-37a926fb3197 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:19.284928 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=100 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:34:19.655983 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-a9288db9-2593-4367-80b3-37a926fb3197 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '22e264440fff43cdb7be6702839af37f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da2248fa190342f0bfe9211cae1e403b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:34:19.667823 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-a9288db9-2593-4367-80b3-37a926fb3197 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1" status: 200 len: 1504 microversion: 2.1 time: 0.415639 Jul 16 21:34:19.668943 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2465/4917] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:34:19 2022] GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1 => generated 1504 bytes in 417 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:34:19.687501 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-e941cc76-4c5e-4f14-b55d-07e14b5e9c58 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Action: 'action', calling method: >, body: {"pause": {}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:34:19.688127 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-e941cc76-4c5e-4f14-b55d-07e14b5e9c58 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: e4c7a9cb-e55d-4819-9885-2e72deffddb1] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:34:19.696047 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e941cc76-4c5e-4f14-b55d-07e14b5e9c58 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:19.696700 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e941cc76-4c5e-4f14-b55d-07e14b5e9c58 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:19.697623 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e941cc76-4c5e-4f14-b55d-07e14b5e9c58 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:19.698227 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e941cc76-4c5e-4f14-b55d-07e14b5e9c58 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:20.035718 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-e941cc76-4c5e-4f14-b55d-07e14b5e9c58 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "POST /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1/action" status: 202 len: 0 microversion: 2.1 time: 0.353479 Jul 16 21:34:20.041564 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2453/4918] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 21:34:19 2022] POST /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1/action => generated 0 bytes in 360 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 21:34:20.050227 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-9aeb613c-986a-449a-b80a-6a3f62ca3214 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:34:20.050863 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-9aeb613c-986a-449a-b80a-6a3f62ca3214 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: e4c7a9cb-e55d-4819-9885-2e72deffddb1] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:34:20.084202 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9aeb613c-986a-449a-b80a-6a3f62ca3214 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:20.086469 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9aeb613c-986a-449a-b80a-6a3f62ca3214 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:20.087420 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9aeb613c-986a-449a-b80a-6a3f62ca3214 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:20.087878 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9aeb613c-986a-449a-b80a-6a3f62ca3214 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:20.418852 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-9aeb613c-986a-449a-b80a-6a3f62ca3214 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '22e264440fff43cdb7be6702839af37f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da2248fa190342f0bfe9211cae1e403b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:34:20.429863 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-9aeb613c-986a-449a-b80a-6a3f62ca3214 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1" status: 200 len: 1509 microversion: 2.1 time: 0.388787 Jul 16 21:34:20.431198 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2466/4919] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:34:20 2022] GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1 => generated 1509 bytes in 391 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:34:21.449553 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-d0156e8f-a71d-4f90-80bd-0dc0820c2855 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:34:21.450934 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-d0156e8f-a71d-4f90-80bd-0dc0820c2855 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: e4c7a9cb-e55d-4819-9885-2e72deffddb1] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:34:21.461213 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d0156e8f-a71d-4f90-80bd-0dc0820c2855 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:21.464822 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d0156e8f-a71d-4f90-80bd-0dc0820c2855 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:21.464822 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d0156e8f-a71d-4f90-80bd-0dc0820c2855 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:21.464822 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d0156e8f-a71d-4f90-80bd-0dc0820c2855 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:21.727519 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-d0156e8f-a71d-4f90-80bd-0dc0820c2855 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '22e264440fff43cdb7be6702839af37f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da2248fa190342f0bfe9211cae1e403b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:34:21.743322 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-d0156e8f-a71d-4f90-80bd-0dc0820c2855 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1" status: 200 len: 1489 microversion: 2.1 time: 0.295270 Jul 16 21:34:21.769038 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-73237423-eb48-4622-9a4d-1cb18e6b6a28 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Action: 'action', calling method: >, body: {"pause": {}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:34:21.771829 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-73237423-eb48-4622-9a4d-1cb18e6b6a28 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: e4c7a9cb-e55d-4819-9885-2e72deffddb1] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:34:21.781291 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2454/4920] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:34:21 2022] GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1 => generated 1489 bytes in 338 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:34:21.786467 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-73237423-eb48-4622-9a4d-1cb18e6b6a28 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:21.793539 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-73237423-eb48-4622-9a4d-1cb18e6b6a28 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:21.795893 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-73237423-eb48-4622-9a4d-1cb18e6b6a28 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:21.801232 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-73237423-eb48-4622-9a4d-1cb18e6b6a28 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:21.834245 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-73237423-eb48-4622-9a4d-1cb18e6b6a28 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] HTTP exception thrown: Cannot 'pause' instance e4c7a9cb-e55d-4819-9885-2e72deffddb1 while it is in vm_state paused Jul 16 21:34:21.835904 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-73237423-eb48-4622-9a4d-1cb18e6b6a28 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Request method failure captured: Jul 16 21:34:21.835904 ncdv-master-1 devstack@n-api.service[138681]: request: POST /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1/action HTTP/1.1 Jul 16 21:34:21.835904 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:34:21.835904 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:34:21.835904 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:34:21.835904 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 13 Jul 16 21:34:21.835904 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:34:21.835904 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:34:21.835904 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:34:21.835904 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:34:21.835904 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0y6QJDVdPNcLbUnwwgOJV-QkddLQ7yqnzBNVp2CrOe5UQqjQOER3spl3PswEW82ISSx8FhmuIrEwFSG69Ze1pTOmWBWZ8PURIgVHbN3yBPA3U2S2TKkQkFwyNuvvV03MCr9U0QGJIjXVOkiqFEAy8q87U_okZgCdI-BFjBFgrCdfAEo Jul 16 21:34:21.835904 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:34:21.835904 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:34:21.835904 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:34:21.835904 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:34:21.835904 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:34:21.835904 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:34:21.835904 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: da2248fa190342f0bfe9211cae1e403b Jul 16 21:34:21.835904 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-ServersNegativeTestJSON-1102403437 Jul 16 21:34:21.835904 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:34:21.835904 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:34:21.835904 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/da2248fa190342f0bfe9211cae1e403b"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/da2248fa190342f0bfe9211cae1e403b"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_da2248fa190342f0bfe9211cae1e403b"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/da2248fa190342f0bfe9211cae1e403b"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:34:21.835904 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-ServersNegativeTestJSON-1102403437 Jul 16 21:34:21.835904 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: da2248fa190342f0bfe9211cae1e403b Jul 16 21:34:21.835904 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-ServersNegativeTestJSON-1102403437 Jul 16 21:34:21.835904 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-ServersNegativeTestJSON-1102403437-project Jul 16 21:34:21.835904 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:34:21.835904 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:34:21.835904 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: 22e264440fff43cdb7be6702839af37f Jul 16 21:34:21.835904 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-ServersNegativeTestJSON-1102403437-project Jul 16 21:34:21.835904 ncdv-master-1 devstack@n-api.service[138681]: Jul 16 21:34:21.835904 ncdv-master-1 devstack@n-api.service[138681]: {"pause": {}} Jul 16 21:34:21.835904 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:34:21.835904 ncdv-master-1 devstack@n-api.service[138681]: exception: Cannot 'pause' instance e4c7a9cb-e55d-4819-9885-2e72deffddb1 while it is in vm_state paused Jul 16 21:34:21.835904 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': 'e4c7a9cb-e55d-4819-9885-2e72deffddb1', 'body': {'pause': {}}} Jul 16 21:34:21.835904 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:34:21.835904 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:34:21.835904 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/pause_server.py", line 41, in _pause Jul 16 21:34:21.835904 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi self.compute_api.pause(ctxt, server) Jul 16 21:34:21.835904 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner Jul 16 21:34:21.835904 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Jul 16 21:34:21.835904 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 145, in inner Jul 16 21:34:21.835904 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.InstanceInvalidState( Jul 16 21:34:21.835904 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.InstanceInvalidState: Instance e4c7a9cb-e55d-4819-9885-2e72deffddb1 in vm_state paused. Cannot pause while the instance is in this state. Jul 16 21:34:21.835904 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:34:21.835904 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:34:21.835904 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:34:21.835904 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:34:21.835904 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:34:21.835904 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:34:21.835904 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:34:21.835904 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:34:21.835904 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:34:21.835904 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:34:21.835904 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/pause_server.py", line 45, in _pause Jul 16 21:34:21.835904 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi common.raise_http_conflict_for_instance_invalid_state(state_error, Jul 16 21:34:21.835904 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 347, in raise_http_conflict_for_instance_invalid_state Jul 16 21:34:21.835904 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPConflict(explanation=msg) Jul 16 21:34:21.835904 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPConflict: Cannot 'pause' instance e4c7a9cb-e55d-4819-9885-2e72deffddb1 while it is in vm_state paused Jul 16 21:34:21.835904 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:34:21.835904 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:34:21.835904 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:34:21.835904 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:34:21.835904 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:34:21.841230 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:34:21.841230 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:34:21.841230 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:34:21.841230 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Cannot 'pause' instance e4c7a9cb-e55d-4819-9885-2e72deffddb1 while it is in vm_state paused Jul 16 21:34:21.841230 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:34:21.841230 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-73237423-eb48-4622-9a4d-1cb18e6b6a28 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Returning 409 to user: Cannot 'pause' instance e4c7a9cb-e55d-4819-9885-2e72deffddb1 while it is in vm_state paused {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:34:21.841230 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-73237423-eb48-4622-9a4d-1cb18e6b6a28 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "POST /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1/action" status: 409 len: 143 microversion: 2.1 time: 0.075687 Jul 16 21:34:21.844991 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2467/4921] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 21:34:21 2022] POST /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1/action => generated 143 bytes in 78 msecs (HTTP/1.1 409) 9 headers in 378 bytes (1 switches on core 0) Jul 16 21:34:21.851537 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-58844610-7a87-4452-9fdb-a1aab289b701 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Action: 'action', calling method: >, body: {"unpause": {}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:34:21.852079 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-58844610-7a87-4452-9fdb-a1aab289b701 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: e4c7a9cb-e55d-4819-9885-2e72deffddb1] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:34:21.860199 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-58844610-7a87-4452-9fdb-a1aab289b701 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:21.860771 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-58844610-7a87-4452-9fdb-a1aab289b701 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:21.861799 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-58844610-7a87-4452-9fdb-a1aab289b701 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:21.863137 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-58844610-7a87-4452-9fdb-a1aab289b701 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:22.195973 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-58844610-7a87-4452-9fdb-a1aab289b701 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "POST /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1/action" status: 202 len: 0 microversion: 2.1 time: 0.348718 Jul 16 21:34:22.196845 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2455/4922] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 21:34:21 2022] POST /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1/action => generated 0 bytes in 351 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 21:34:22.206703 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-edf97aa2-2b49-469f-9fdd-c001f85f3a14 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:34:22.207527 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-edf97aa2-2b49-469f-9fdd-c001f85f3a14 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: e4c7a9cb-e55d-4819-9885-2e72deffddb1] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:34:22.215831 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-edf97aa2-2b49-469f-9fdd-c001f85f3a14 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:22.216884 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-edf97aa2-2b49-469f-9fdd-c001f85f3a14 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:22.217749 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-edf97aa2-2b49-469f-9fdd-c001f85f3a14 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:22.218264 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-edf97aa2-2b49-469f-9fdd-c001f85f3a14 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:22.407544 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-edf97aa2-2b49-469f-9fdd-c001f85f3a14 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '22e264440fff43cdb7be6702839af37f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da2248fa190342f0bfe9211cae1e403b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:34:22.432074 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-edf97aa2-2b49-469f-9fdd-c001f85f3a14 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1" status: 200 len: 1496 microversion: 2.1 time: 0.230848 Jul 16 21:34:22.434449 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2468/4923] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:34:22 2022] GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1 => generated 1496 bytes in 234 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:34:23.453485 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-a65a4bb5-ff13-4457-933d-18308e93e2b4 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:34:23.454764 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-a65a4bb5-ff13-4457-933d-18308e93e2b4 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: e4c7a9cb-e55d-4819-9885-2e72deffddb1] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:34:23.464545 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a65a4bb5-ff13-4457-933d-18308e93e2b4 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:23.467694 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a65a4bb5-ff13-4457-933d-18308e93e2b4 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:23.467694 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a65a4bb5-ff13-4457-933d-18308e93e2b4 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:23.467694 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a65a4bb5-ff13-4457-933d-18308e93e2b4 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:23.766138 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-a65a4bb5-ff13-4457-933d-18308e93e2b4 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '22e264440fff43cdb7be6702839af37f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da2248fa190342f0bfe9211cae1e403b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:34:23.778751 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-a65a4bb5-ff13-4457-933d-18308e93e2b4 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1" status: 200 len: 1504 microversion: 2.1 time: 0.330004 Jul 16 21:34:23.779697 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2456/4924] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:34:23 2022] GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1 => generated 1504 bytes in 333 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:34:23.798519 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-eb8b512c-1952-42de-befa-5ca944eec72c tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:34:23.799181 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-eb8b512c-1952-42de-befa-5ca944eec72c tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: e4c7a9cb-e55d-4819-9885-2e72deffddb1] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:34:23.807209 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-eb8b512c-1952-42de-befa-5ca944eec72c tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:23.807668 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-eb8b512c-1952-42de-befa-5ca944eec72c tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:23.808456 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-eb8b512c-1952-42de-befa-5ca944eec72c tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:23.809101 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-eb8b512c-1952-42de-befa-5ca944eec72c tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:24.090038 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-eb8b512c-1952-42de-befa-5ca944eec72c tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '22e264440fff43cdb7be6702839af37f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da2248fa190342f0bfe9211cae1e403b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:34:24.100865 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-eb8b512c-1952-42de-befa-5ca944eec72c tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1" status: 200 len: 1504 microversion: 2.1 time: 0.306206 Jul 16 21:34:24.102101 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2469/4925] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:34:23 2022] GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1 => generated 1504 bytes in 308 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:34:24.121967 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-0b56d5d4-109d-4eb5-a22c-2a7012308896 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Action: 'action', calling method: >, body: {"reboot": {"type": "SOFT"}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:34:24.123679 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-0b56d5d4-109d-4eb5-a22c-2a7012308896 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: c77c9a95-ea71-40d7-b7a7-ded612d194da] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:34:24.132216 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0b56d5d4-109d-4eb5-a22c-2a7012308896 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:24.133054 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0b56d5d4-109d-4eb5-a22c-2a7012308896 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:24.141353 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0b56d5d4-109d-4eb5-a22c-2a7012308896 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:24.141851 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0b56d5d4-109d-4eb5-a22c-2a7012308896 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:24.165339 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-0b56d5d4-109d-4eb5-a22c-2a7012308896 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] HTTP exception thrown: Instance c77c9a95-ea71-40d7-b7a7-ded612d194da could not be found. Jul 16 21:34:24.176534 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-0b56d5d4-109d-4eb5-a22c-2a7012308896 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Request method failure captured: Jul 16 21:34:24.176534 ncdv-master-1 devstack@n-api.service[138680]: request: POST /compute/v2.1/servers/c77c9a95-ea71-40d7-b7a7-ded612d194da/action HTTP/1.1 Jul 16 21:34:24.176534 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:34:24.176534 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:34:24.176534 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:34:24.176534 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 28 Jul 16 21:34:24.176534 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:34:24.176534 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:34:24.176534 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:34:24.176534 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:34:24.176534 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0y6QJDVdPNcLbUnwwgOJV-QkddLQ7yqnzBNVp2CrOe5UQqjQOER3spl3PswEW82ISSx8FhmuIrEwFSG69Ze1pTOmWBWZ8PURIgVHbN3yBPA3U2S2TKkQkFwyNuvvV03MCr9U0QGJIjXVOkiqFEAy8q87U_okZgCdI-BFjBFgrCdfAEo Jul 16 21:34:24.176534 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:34:24.176534 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:34:24.176534 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:34:24.176534 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:34:24.176534 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:34:24.176534 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:34:24.176534 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: da2248fa190342f0bfe9211cae1e403b Jul 16 21:34:24.176534 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-ServersNegativeTestJSON-1102403437 Jul 16 21:34:24.176534 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:34:24.176534 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:34:24.176534 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/da2248fa190342f0bfe9211cae1e403b"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/da2248fa190342f0bfe9211cae1e403b"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_da2248fa190342f0bfe9211cae1e403b"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/da2248fa190342f0bfe9211cae1e403b"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:34:24.176534 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-ServersNegativeTestJSON-1102403437 Jul 16 21:34:24.176534 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: da2248fa190342f0bfe9211cae1e403b Jul 16 21:34:24.176534 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-ServersNegativeTestJSON-1102403437 Jul 16 21:34:24.176534 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-ServersNegativeTestJSON-1102403437-project Jul 16 21:34:24.176534 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:34:24.176534 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:34:24.176534 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: 22e264440fff43cdb7be6702839af37f Jul 16 21:34:24.176534 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-ServersNegativeTestJSON-1102403437-project Jul 16 21:34:24.176534 ncdv-master-1 devstack@n-api.service[138680]: Jul 16 21:34:24.176534 ncdv-master-1 devstack@n-api.service[138680]: {"reboot": {"type": "SOFT"}} Jul 16 21:34:24.176534 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 21:34:24.176534 ncdv-master-1 devstack@n-api.service[138680]: exception: Instance c77c9a95-ea71-40d7-b7a7-ded612d194da could not be found. Jul 16 21:34:24.176534 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': 'c77c9a95-ea71-40d7-b7a7-ded612d194da', 'body': {'reboot': {'type': 'SOFT'}}} Jul 16 21:34:24.176534 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:34:24.176534 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:34:24.176534 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:34:24.176534 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:34:24.176534 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:34:24.176534 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:34:24.176534 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:34:24.176534 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:34:24.176534 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:34:24.176534 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:34:24.176534 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance c77c9a95-ea71-40d7-b7a7-ded612d194da could not be found. Jul 16 21:34:24.176534 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:34:24.176534 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:34:24.176534 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:34:24.176534 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:34:24.176534 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:34:24.176534 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:34:24.176534 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:34:24.176534 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:34:24.176534 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:34:24.176534 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:34:24.176534 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 16 21:34:24.176534 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 16 21:34:24.176534 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1031, in _action_reboot Jul 16 21:34:24.176534 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, id) Jul 16 21:34:24.176534 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:34:24.176534 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:34:24.176534 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:34:24.176534 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:34:24.182135 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance c77c9a95-ea71-40d7-b7a7-ded612d194da could not be found. Jul 16 21:34:24.182135 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:34:24.182135 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:34:24.182135 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:34:24.182135 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:34:24.182135 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:34:24.182135 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:34:24.182135 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:34:24.182135 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:34:24.182135 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance c77c9a95-ea71-40d7-b7a7-ded612d194da could not be found. Jul 16 21:34:24.182135 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:34:24.182135 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-0b56d5d4-109d-4eb5-a22c-2a7012308896 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Returning 404 to user: Instance c77c9a95-ea71-40d7-b7a7-ded612d194da could not be found. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:34:24.182135 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-0b56d5d4-109d-4eb5-a22c-2a7012308896 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "POST /compute/v2.1/servers/c77c9a95-ea71-40d7-b7a7-ded612d194da/action" status: 404 len: 111 microversion: 2.1 time: 0.061359 Jul 16 21:34:24.182135 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2457/4926] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 21:34:24 2022] POST /compute/v2.1/servers/c77c9a95-ea71-40d7-b7a7-ded612d194da/action => generated 111 bytes in 64 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:34:24.191999 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-d7a0c812-0807-4884-8b1a-81124ff255e0 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:34:24.192853 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-d7a0c812-0807-4884-8b1a-81124ff255e0 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: e4c7a9cb-e55d-4819-9885-2e72deffddb1] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:34:24.202865 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d7a0c812-0807-4884-8b1a-81124ff255e0 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:24.207569 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d7a0c812-0807-4884-8b1a-81124ff255e0 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:24.208423 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d7a0c812-0807-4884-8b1a-81124ff255e0 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:24.208878 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d7a0c812-0807-4884-8b1a-81124ff255e0 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:24.449135 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-d7a0c812-0807-4884-8b1a-81124ff255e0 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '22e264440fff43cdb7be6702839af37f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da2248fa190342f0bfe9211cae1e403b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:34:24.458108 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-d7a0c812-0807-4884-8b1a-81124ff255e0 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1" status: 200 len: 1504 microversion: 2.1 time: 0.273348 Jul 16 21:34:24.458716 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2470/4927] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:34:24 2022] GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1 => generated 1504 bytes in 276 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:34:24.476176 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-be90e327-d859-4006-996c-5c713a8e96d4 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:34:24.476818 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-be90e327-d859-4006-996c-5c713a8e96d4 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: e4c7a9cb-e55d-4819-9885-2e72deffddb1] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:34:24.484129 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-be90e327-d859-4006-996c-5c713a8e96d4 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:24.484579 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-be90e327-d859-4006-996c-5c713a8e96d4 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:24.485388 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-be90e327-d859-4006-996c-5c713a8e96d4 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:24.485824 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-be90e327-d859-4006-996c-5c713a8e96d4 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:24.716220 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-be90e327-d859-4006-996c-5c713a8e96d4 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '22e264440fff43cdb7be6702839af37f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da2248fa190342f0bfe9211cae1e403b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:34:24.725133 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-be90e327-d859-4006-996c-5c713a8e96d4 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1" status: 200 len: 1504 microversion: 2.1 time: 0.252996 Jul 16 21:34:24.725604 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2458/4928] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:34:24 2022] GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1 => generated 1504 bytes in 254 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:34:24.743034 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-1d7779bf-c79a-4600-82b9-30fe550b58f9 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Action: 'action', calling method: >, body: {"reboot": {"type": "SOFT"}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:34:24.744678 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-1d7779bf-c79a-4600-82b9-30fe550b58f9 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: a6b7d412-693f-4664-874c-fd98c77777eb] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:34:24.751490 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-1d7779bf-c79a-4600-82b9-30fe550b58f9 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] HTTP exception thrown: Instance a6b7d412-693f-4664-874c-fd98c77777eb could not be found. Jul 16 21:34:24.753641 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-1d7779bf-c79a-4600-82b9-30fe550b58f9 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Request method failure captured: Jul 16 21:34:24.753641 ncdv-master-1 devstack@n-api.service[138681]: request: POST /compute/v2.1/servers/a6b7d412-693f-4664-874c-fd98c77777eb/action HTTP/1.1 Jul 16 21:34:24.753641 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:34:24.753641 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:34:24.753641 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:34:24.753641 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 28 Jul 16 21:34:24.753641 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:34:24.753641 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:34:24.753641 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:34:24.753641 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:34:24.753641 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0y6QJDVdPNcLbUnwwgOJV-QkddLQ7yqnzBNVp2CrOe5UQqjQOER3spl3PswEW82ISSx8FhmuIrEwFSG69Ze1pTOmWBWZ8PURIgVHbN3yBPA3U2S2TKkQkFwyNuvvV03MCr9U0QGJIjXVOkiqFEAy8q87U_okZgCdI-BFjBFgrCdfAEo Jul 16 21:34:24.753641 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:34:24.753641 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:34:24.753641 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:34:24.753641 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:34:24.753641 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:34:24.753641 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:34:24.753641 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: da2248fa190342f0bfe9211cae1e403b Jul 16 21:34:24.753641 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-ServersNegativeTestJSON-1102403437 Jul 16 21:34:24.753641 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:34:24.753641 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:34:24.753641 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/da2248fa190342f0bfe9211cae1e403b"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/da2248fa190342f0bfe9211cae1e403b"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_da2248fa190342f0bfe9211cae1e403b"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/da2248fa190342f0bfe9211cae1e403b"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:34:24.753641 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-ServersNegativeTestJSON-1102403437 Jul 16 21:34:24.753641 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: da2248fa190342f0bfe9211cae1e403b Jul 16 21:34:24.753641 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-ServersNegativeTestJSON-1102403437 Jul 16 21:34:24.753641 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-ServersNegativeTestJSON-1102403437-project Jul 16 21:34:24.753641 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:34:24.753641 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:34:24.753641 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: 22e264440fff43cdb7be6702839af37f Jul 16 21:34:24.753641 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-ServersNegativeTestJSON-1102403437-project Jul 16 21:34:24.753641 ncdv-master-1 devstack@n-api.service[138681]: Jul 16 21:34:24.753641 ncdv-master-1 devstack@n-api.service[138681]: {"reboot": {"type": "SOFT"}} Jul 16 21:34:24.753641 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:34:24.753641 ncdv-master-1 devstack@n-api.service[138681]: exception: Instance a6b7d412-693f-4664-874c-fd98c77777eb could not be found. Jul 16 21:34:24.753641 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': 'a6b7d412-693f-4664-874c-fd98c77777eb', 'body': {'reboot': {'type': 'SOFT'}}} Jul 16 21:34:24.753641 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:34:24.753641 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:34:24.753641 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:34:24.753641 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:34:24.753641 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:34:24.753641 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:34:24.753641 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2898, in _get_instance Jul 16 21:34:24.753641 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=instance_uuid) Jul 16 21:34:24.753641 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance a6b7d412-693f-4664-874c-fd98c77777eb could not be found. Jul 16 21:34:24.753641 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:34:24.753641 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:34:24.753641 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:34:24.753641 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:34:24.753641 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:34:24.753641 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:34:24.753641 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:34:24.753641 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:34:24.753641 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:34:24.753641 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:34:24.753641 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 16 21:34:24.753641 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 16 21:34:24.753641 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1031, in _action_reboot Jul 16 21:34:24.753641 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, id) Jul 16 21:34:24.753641 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:34:24.753641 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:34:24.753641 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:34:24.753641 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:34:24.753641 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance a6b7d412-693f-4664-874c-fd98c77777eb could not be found. Jul 16 21:34:24.753641 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:34:24.760411 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:34:24.760411 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:34:24.760411 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:34:24.760411 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:34:24.760411 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:34:24.760411 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:34:24.760411 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:34:24.760411 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance a6b7d412-693f-4664-874c-fd98c77777eb could not be found. Jul 16 21:34:24.760411 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:34:24.760411 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-1d7779bf-c79a-4600-82b9-30fe550b58f9 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Returning 404 to user: Instance a6b7d412-693f-4664-874c-fd98c77777eb could not be found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:34:24.760411 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-1d7779bf-c79a-4600-82b9-30fe550b58f9 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "POST /compute/v2.1/servers/a6b7d412-693f-4664-874c-fd98c77777eb/action" status: 404 len: 111 microversion: 2.1 time: 0.019729 Jul 16 21:34:24.760411 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2471/4929] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 21:34:24 2022] POST /compute/v2.1/servers/a6b7d412-693f-4664-874c-fd98c77777eb/action => generated 111 bytes in 22 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Jul 16 21:34:24.772498 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-c381ae86-8ec9-406a-a6cd-c69f388f0e74 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:34:24.773542 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-c381ae86-8ec9-406a-a6cd-c69f388f0e74 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: e4c7a9cb-e55d-4819-9885-2e72deffddb1] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:34:24.780302 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c381ae86-8ec9-406a-a6cd-c69f388f0e74 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:24.780756 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c381ae86-8ec9-406a-a6cd-c69f388f0e74 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:24.781868 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c381ae86-8ec9-406a-a6cd-c69f388f0e74 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:24.782320 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c381ae86-8ec9-406a-a6cd-c69f388f0e74 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:25.247502 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-c381ae86-8ec9-406a-a6cd-c69f388f0e74 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '22e264440fff43cdb7be6702839af37f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da2248fa190342f0bfe9211cae1e403b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:34:25.264532 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-c381ae86-8ec9-406a-a6cd-c69f388f0e74 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1" status: 200 len: 1504 microversion: 2.1 time: 0.497245 Jul 16 21:34:25.266433 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2459/4930] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:34:24 2022] GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1 => generated 1504 bytes in 500 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:34:25.286316 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-fed44203-e12b-4c16-8a99-e4931fc8d55c tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:34:25.287584 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-fed44203-e12b-4c16-8a99-e4931fc8d55c tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: e4c7a9cb-e55d-4819-9885-2e72deffddb1] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:34:25.296344 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fed44203-e12b-4c16-8a99-e4931fc8d55c tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:25.297162 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fed44203-e12b-4c16-8a99-e4931fc8d55c tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:25.298122 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fed44203-e12b-4c16-8a99-e4931fc8d55c tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:25.298733 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fed44203-e12b-4c16-8a99-e4931fc8d55c tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:25.453228 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-fed44203-e12b-4c16-8a99-e4931fc8d55c tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '22e264440fff43cdb7be6702839af37f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da2248fa190342f0bfe9211cae1e403b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:34:25.462003 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-fed44203-e12b-4c16-8a99-e4931fc8d55c tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1" status: 200 len: 1504 microversion: 2.1 time: 0.180445 Jul 16 21:34:25.462579 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2472/4931] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:34:25 2022] GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1 => generated 1504 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:34:25.480120 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-da3d0ea9-5c4e-4123-b843-2c1b0a816f3b tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Action: 'action', calling method: >, body: {"rebuild": {"imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9"}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:34:25.482923 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-da3d0ea9-5c4e-4123-b843-2c1b0a816f3b tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: c77c9a95-ea71-40d7-b7a7-ded612d194da] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:34:25.495407 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-da3d0ea9-5c4e-4123-b843-2c1b0a816f3b tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:25.495407 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-da3d0ea9-5c4e-4123-b843-2c1b0a816f3b tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:25.495407 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-da3d0ea9-5c4e-4123-b843-2c1b0a816f3b tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:25.495407 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-da3d0ea9-5c4e-4123-b843-2c1b0a816f3b tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:25.501149 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=96,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:34:25.511060 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-da3d0ea9-5c4e-4123-b843-2c1b0a816f3b tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] HTTP exception thrown: Instance c77c9a95-ea71-40d7-b7a7-ded612d194da could not be found. Jul 16 21:34:25.513697 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-da3d0ea9-5c4e-4123-b843-2c1b0a816f3b tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Request method failure captured: Jul 16 21:34:25.513697 ncdv-master-1 devstack@n-api.service[138680]: request: POST /compute/v2.1/servers/c77c9a95-ea71-40d7-b7a7-ded612d194da/action HTTP/1.1 Jul 16 21:34:25.513697 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:34:25.513697 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:34:25.513697 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:34:25.513697 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 65 Jul 16 21:34:25.513697 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:34:25.513697 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:34:25.513697 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:34:25.513697 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:34:25.513697 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0y6QJDVdPNcLbUnwwgOJV-QkddLQ7yqnzBNVp2CrOe5UQqjQOER3spl3PswEW82ISSx8FhmuIrEwFSG69Ze1pTOmWBWZ8PURIgVHbN3yBPA3U2S2TKkQkFwyNuvvV03MCr9U0QGJIjXVOkiqFEAy8q87U_okZgCdI-BFjBFgrCdfAEo Jul 16 21:34:25.513697 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:34:25.513697 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:34:25.513697 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:34:25.513697 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:34:25.513697 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:34:25.513697 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:34:25.513697 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: da2248fa190342f0bfe9211cae1e403b Jul 16 21:34:25.513697 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-ServersNegativeTestJSON-1102403437 Jul 16 21:34:25.513697 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:34:25.513697 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:34:25.513697 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/da2248fa190342f0bfe9211cae1e403b"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/da2248fa190342f0bfe9211cae1e403b"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_da2248fa190342f0bfe9211cae1e403b"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/da2248fa190342f0bfe9211cae1e403b"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:34:25.513697 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-ServersNegativeTestJSON-1102403437 Jul 16 21:34:25.513697 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: da2248fa190342f0bfe9211cae1e403b Jul 16 21:34:25.513697 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-ServersNegativeTestJSON-1102403437 Jul 16 21:34:25.513697 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-ServersNegativeTestJSON-1102403437-project Jul 16 21:34:25.513697 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:34:25.513697 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:34:25.513697 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: 22e264440fff43cdb7be6702839af37f Jul 16 21:34:25.513697 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-ServersNegativeTestJSON-1102403437-project Jul 16 21:34:25.513697 ncdv-master-1 devstack@n-api.service[138680]: Jul 16 21:34:25.513697 ncdv-master-1 devstack@n-api.service[138680]: {"rebuild": {"imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9"}} Jul 16 21:34:25.513697 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 21:34:25.513697 ncdv-master-1 devstack@n-api.service[138680]: exception: Instance c77c9a95-ea71-40d7-b7a7-ded612d194da could not be found. Jul 16 21:34:25.513697 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': 'c77c9a95-ea71-40d7-b7a7-ded612d194da', 'body': {'rebuild': {'imageRef': '1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9'}}} Jul 16 21:34:25.513697 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:34:25.513697 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:34:25.513697 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:34:25.513697 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:34:25.513697 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:34:25.513697 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:34:25.513697 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:34:25.513697 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:34:25.513697 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:34:25.513697 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:34:25.513697 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance c77c9a95-ea71-40d7-b7a7-ded612d194da could not be found. Jul 16 21:34:25.513697 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:34:25.513697 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:34:25.513697 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:34:25.513697 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:34:25.513697 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:34:25.513697 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:34:25.513697 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:34:25.513697 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:34:25.513697 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:34:25.513697 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:34:25.513697 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 16 21:34:25.513697 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 16 21:34:25.513697 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 16 21:34:25.513697 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 16 21:34:25.513697 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 16 21:34:25.513697 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 16 21:34:25.513697 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi [Previous line repeated 4 more times] Jul 16 21:34:25.513697 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1160, in _action_rebuild Jul 16 21:34:25.522605 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, id) Jul 16 21:34:25.522605 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:34:25.522605 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:34:25.522605 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:34:25.522605 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:34:25.522605 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance c77c9a95-ea71-40d7-b7a7-ded612d194da could not be found. Jul 16 21:34:25.522605 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:34:25.522605 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:34:25.522605 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:34:25.522605 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:34:25.522605 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:34:25.522605 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:34:25.522605 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:34:25.522605 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:34:25.522605 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance c77c9a95-ea71-40d7-b7a7-ded612d194da could not be found. Jul 16 21:34:25.522605 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:34:25.522605 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-da3d0ea9-5c4e-4123-b843-2c1b0a816f3b tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Returning 404 to user: Instance c77c9a95-ea71-40d7-b7a7-ded612d194da could not be found. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:34:25.522605 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-da3d0ea9-5c4e-4123-b843-2c1b0a816f3b tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "POST /compute/v2.1/servers/c77c9a95-ea71-40d7-b7a7-ded612d194da/action" status: 404 len: 111 microversion: 2.1 time: 0.042730 Jul 16 21:34:25.530458 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-8aa882d7-d999-4064-817d-f8d33dde1808 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:34:25.531138 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-8aa882d7-d999-4064-817d-f8d33dde1808 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: e4c7a9cb-e55d-4819-9885-2e72deffddb1] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:34:25.536702 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2460/4932] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 21:34:25 2022] POST /compute/v2.1/servers/c77c9a95-ea71-40d7-b7a7-ded612d194da/action => generated 111 bytes in 60 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:34:25.540419 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8aa882d7-d999-4064-817d-f8d33dde1808 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:25.541886 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8aa882d7-d999-4064-817d-f8d33dde1808 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:25.542397 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8aa882d7-d999-4064-817d-f8d33dde1808 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:25.543293 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8aa882d7-d999-4064-817d-f8d33dde1808 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:25.724646 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-8aa882d7-d999-4064-817d-f8d33dde1808 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '22e264440fff43cdb7be6702839af37f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da2248fa190342f0bfe9211cae1e403b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:34:25.733937 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-8aa882d7-d999-4064-817d-f8d33dde1808 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1" status: 200 len: 1504 microversion: 2.1 time: 0.207505 Jul 16 21:34:25.736017 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2473/4933] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:34:25 2022] GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1 => generated 1504 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:34:25.757329 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-ab6f4700-6761-4244-987b-4fec4378cb65 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:34:25.758069 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-ab6f4700-6761-4244-987b-4fec4378cb65 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: e4c7a9cb-e55d-4819-9885-2e72deffddb1] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:34:25.765616 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ab6f4700-6761-4244-987b-4fec4378cb65 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:25.766364 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ab6f4700-6761-4244-987b-4fec4378cb65 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:25.767246 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ab6f4700-6761-4244-987b-4fec4378cb65 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:25.767751 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ab6f4700-6761-4244-987b-4fec4378cb65 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:25.950364 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-ab6f4700-6761-4244-987b-4fec4378cb65 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '22e264440fff43cdb7be6702839af37f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da2248fa190342f0bfe9211cae1e403b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:34:25.959581 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-ab6f4700-6761-4244-987b-4fec4378cb65 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1" status: 200 len: 1504 microversion: 2.1 time: 0.209197 Jul 16 21:34:25.960856 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2461/4934] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:34:25 2022] GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1 => generated 1504 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:34:25.979892 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-96c8e026-1686-4a2f-b3b5-4aaa709a7f96 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Action: 'action', calling method: >, body: {"rebuild": {"imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9"}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:34:25.981616 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-96c8e026-1686-4a2f-b3b5-4aaa709a7f96 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: af8c1810-ea37-49dc-8edc-3df1b7257140] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:34:25.989807 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-96c8e026-1686-4a2f-b3b5-4aaa709a7f96 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] HTTP exception thrown: Instance af8c1810-ea37-49dc-8edc-3df1b7257140 could not be found. Jul 16 21:34:25.991904 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-96c8e026-1686-4a2f-b3b5-4aaa709a7f96 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Request method failure captured: Jul 16 21:34:25.991904 ncdv-master-1 devstack@n-api.service[138681]: request: POST /compute/v2.1/servers/af8c1810-ea37-49dc-8edc-3df1b7257140/action HTTP/1.1 Jul 16 21:34:25.991904 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:34:25.991904 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:34:25.991904 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:34:25.991904 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 65 Jul 16 21:34:25.991904 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:34:25.991904 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:34:25.991904 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:34:25.991904 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:34:25.991904 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0y6QJDVdPNcLbUnwwgOJV-QkddLQ7yqnzBNVp2CrOe5UQqjQOER3spl3PswEW82ISSx8FhmuIrEwFSG69Ze1pTOmWBWZ8PURIgVHbN3yBPA3U2S2TKkQkFwyNuvvV03MCr9U0QGJIjXVOkiqFEAy8q87U_okZgCdI-BFjBFgrCdfAEo Jul 16 21:34:25.991904 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:34:25.991904 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:34:25.991904 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:34:25.991904 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:34:25.991904 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:34:25.991904 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:34:25.991904 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: da2248fa190342f0bfe9211cae1e403b Jul 16 21:34:25.991904 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-ServersNegativeTestJSON-1102403437 Jul 16 21:34:25.991904 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:34:25.991904 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:34:25.991904 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/da2248fa190342f0bfe9211cae1e403b"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/da2248fa190342f0bfe9211cae1e403b"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_da2248fa190342f0bfe9211cae1e403b"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/da2248fa190342f0bfe9211cae1e403b"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:34:25.991904 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-ServersNegativeTestJSON-1102403437 Jul 16 21:34:25.991904 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: da2248fa190342f0bfe9211cae1e403b Jul 16 21:34:25.991904 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-ServersNegativeTestJSON-1102403437 Jul 16 21:34:25.991904 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-ServersNegativeTestJSON-1102403437-project Jul 16 21:34:25.991904 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:34:25.991904 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:34:25.991904 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: 22e264440fff43cdb7be6702839af37f Jul 16 21:34:25.991904 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-ServersNegativeTestJSON-1102403437-project Jul 16 21:34:25.991904 ncdv-master-1 devstack@n-api.service[138681]: Jul 16 21:34:25.991904 ncdv-master-1 devstack@n-api.service[138681]: {"rebuild": {"imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9"}} Jul 16 21:34:25.991904 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:34:25.991904 ncdv-master-1 devstack@n-api.service[138681]: exception: Instance af8c1810-ea37-49dc-8edc-3df1b7257140 could not be found. Jul 16 21:34:25.991904 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': 'af8c1810-ea37-49dc-8edc-3df1b7257140', 'body': {'rebuild': {'imageRef': '1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9'}}} Jul 16 21:34:25.991904 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:34:25.991904 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:34:25.991904 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:34:25.991904 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:34:25.991904 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:34:25.991904 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:34:25.991904 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2898, in _get_instance Jul 16 21:34:25.991904 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=instance_uuid) Jul 16 21:34:25.991904 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance af8c1810-ea37-49dc-8edc-3df1b7257140 could not be found. Jul 16 21:34:25.991904 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:34:25.991904 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:34:25.991904 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:34:25.991904 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:34:25.991904 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:34:25.991904 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:34:25.991904 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:34:25.991904 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:34:25.991904 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:34:25.991904 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:34:25.991904 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 16 21:34:25.991904 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 16 21:34:25.991904 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 16 21:34:25.991904 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 16 21:34:25.991904 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 16 21:34:25.991904 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 16 21:34:25.991904 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi [Previous line repeated 4 more times] Jul 16 21:34:25.991904 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1160, in _action_rebuild Jul 16 21:34:25.991904 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, id) Jul 16 21:34:25.991904 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:34:25.996553 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:34:25.996553 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:34:25.996553 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:34:25.996553 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance af8c1810-ea37-49dc-8edc-3df1b7257140 could not be found. Jul 16 21:34:25.996553 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:34:25.996553 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:34:25.996553 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:34:25.996553 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:34:25.996553 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:34:25.996553 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:34:25.996553 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:34:25.996553 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:34:25.996553 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance af8c1810-ea37-49dc-8edc-3df1b7257140 could not be found. Jul 16 21:34:25.996553 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:34:26.006311 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-96c8e026-1686-4a2f-b3b5-4aaa709a7f96 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Returning 404 to user: Instance af8c1810-ea37-49dc-8edc-3df1b7257140 could not be found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:34:26.006311 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-96c8e026-1686-4a2f-b3b5-4aaa709a7f96 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "POST /compute/v2.1/servers/af8c1810-ea37-49dc-8edc-3df1b7257140/action" status: 404 len: 111 microversion: 2.1 time: 0.026845 Jul 16 21:34:26.006311 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2474/4935] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 21:34:25 2022] POST /compute/v2.1/servers/af8c1810-ea37-49dc-8edc-3df1b7257140/action => generated 111 bytes in 29 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:34:26.011277 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-c174ce87-d74d-4ffc-80e0-b419fb40451e tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:34:26.011901 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-c174ce87-d74d-4ffc-80e0-b419fb40451e tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: e4c7a9cb-e55d-4819-9885-2e72deffddb1] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:34:26.018871 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c174ce87-d74d-4ffc-80e0-b419fb40451e tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:26.019689 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c174ce87-d74d-4ffc-80e0-b419fb40451e tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:26.021497 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c174ce87-d74d-4ffc-80e0-b419fb40451e tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:26.023108 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c174ce87-d74d-4ffc-80e0-b419fb40451e tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:26.215627 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-c174ce87-d74d-4ffc-80e0-b419fb40451e tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '22e264440fff43cdb7be6702839af37f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da2248fa190342f0bfe9211cae1e403b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:34:26.229624 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-c174ce87-d74d-4ffc-80e0-b419fb40451e tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1" status: 200 len: 1504 microversion: 2.1 time: 0.222159 Jul 16 21:34:26.239172 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2462/4936] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:34:26 2022] GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1 => generated 1504 bytes in 232 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:34:26.249199 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-9ecd4d93-fece-4077-9898-9c75d20fe433 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:34:26.249952 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-9ecd4d93-fece-4077-9898-9c75d20fe433 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: e4c7a9cb-e55d-4819-9885-2e72deffddb1] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:34:26.263952 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9ecd4d93-fece-4077-9898-9c75d20fe433 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:26.264502 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9ecd4d93-fece-4077-9898-9c75d20fe433 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:26.265622 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9ecd4d93-fece-4077-9898-9c75d20fe433 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:26.266275 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9ecd4d93-fece-4077-9898-9c75d20fe433 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:26.482332 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-9ecd4d93-fece-4077-9898-9c75d20fe433 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '22e264440fff43cdb7be6702839af37f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da2248fa190342f0bfe9211cae1e403b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:34:26.493034 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-9ecd4d93-fece-4077-9898-9c75d20fe433 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1" status: 200 len: 1504 microversion: 2.1 time: 0.247355 Jul 16 21:34:26.503578 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2475/4937] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:34:26 2022] GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1 => generated 1504 bytes in 259 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:34:26.512395 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-e7d489fb-3d4a-4579-ba85-1a82a004fbc0 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Action: 'action', calling method: >, body: {"restore": {}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:34:26.512952 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-e7d489fb-3d4a-4579-ba85-1a82a004fbc0 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: cd28e554-c820-4789-a8df-a1aad00b7c00] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:34:26.528595 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-e7d489fb-3d4a-4579-ba85-1a82a004fbc0 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] HTTP exception thrown: Instance cd28e554-c820-4789-a8df-a1aad00b7c00 could not be found. Jul 16 21:34:26.528595 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-e7d489fb-3d4a-4579-ba85-1a82a004fbc0 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Request method failure captured: Jul 16 21:34:26.528595 ncdv-master-1 devstack@n-api.service[138680]: request: POST /compute/v2.1/servers/cd28e554-c820-4789-a8df-a1aad00b7c00/action HTTP/1.1 Jul 16 21:34:26.528595 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:34:26.528595 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:34:26.528595 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:34:26.528595 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 15 Jul 16 21:34:26.528595 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:34:26.528595 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:34:26.528595 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:34:26.528595 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:34:26.528595 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0y6QJDVdPNcLbUnwwgOJV-QkddLQ7yqnzBNVp2CrOe5UQqjQOER3spl3PswEW82ISSx8FhmuIrEwFSG69Ze1pTOmWBWZ8PURIgVHbN3yBPA3U2S2TKkQkFwyNuvvV03MCr9U0QGJIjXVOkiqFEAy8q87U_okZgCdI-BFjBFgrCdfAEo Jul 16 21:34:26.528595 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:34:26.528595 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:34:26.528595 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:34:26.528595 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:34:26.528595 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:34:26.528595 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:34:26.528595 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: da2248fa190342f0bfe9211cae1e403b Jul 16 21:34:26.528595 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-ServersNegativeTestJSON-1102403437 Jul 16 21:34:26.528595 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:34:26.528595 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:34:26.528595 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/da2248fa190342f0bfe9211cae1e403b"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/da2248fa190342f0bfe9211cae1e403b"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_da2248fa190342f0bfe9211cae1e403b"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/da2248fa190342f0bfe9211cae1e403b"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:34:26.528595 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-ServersNegativeTestJSON-1102403437 Jul 16 21:34:26.528595 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: da2248fa190342f0bfe9211cae1e403b Jul 16 21:34:26.528595 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-ServersNegativeTestJSON-1102403437 Jul 16 21:34:26.528595 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-ServersNegativeTestJSON-1102403437-project Jul 16 21:34:26.528595 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:34:26.528595 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:34:26.528595 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: 22e264440fff43cdb7be6702839af37f Jul 16 21:34:26.528595 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-ServersNegativeTestJSON-1102403437-project Jul 16 21:34:26.528595 ncdv-master-1 devstack@n-api.service[138680]: Jul 16 21:34:26.528595 ncdv-master-1 devstack@n-api.service[138680]: {"restore": {}} Jul 16 21:34:26.528595 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 21:34:26.528595 ncdv-master-1 devstack@n-api.service[138680]: exception: Instance cd28e554-c820-4789-a8df-a1aad00b7c00 could not be found. Jul 16 21:34:26.528595 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': 'cd28e554-c820-4789-a8df-a1aad00b7c00', 'body': {'restore': {}}} Jul 16 21:34:26.528595 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:34:26.528595 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:34:26.528595 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:34:26.528595 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:34:26.528595 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:34:26.528595 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:34:26.528595 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2898, in _get_instance Jul 16 21:34:26.528595 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=instance_uuid) Jul 16 21:34:26.528595 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance cd28e554-c820-4789-a8df-a1aad00b7c00 could not be found. Jul 16 21:34:26.528595 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:34:26.528595 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:34:26.528595 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:34:26.528595 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:34:26.528595 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:34:26.528595 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:34:26.528595 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:34:26.528595 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:34:26.528595 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:34:26.528595 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:34:26.528595 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/deferred_delete.py", line 38, in _restore Jul 16 21:34:26.528595 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, id) Jul 16 21:34:26.528595 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:34:26.528595 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:34:26.528595 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance cd28e554-c820-4789-a8df-a1aad00b7c00 could not be found. Jul 16 21:34:26.528595 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:34:26.528595 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:34:26.528595 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:34:26.533209 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:34:26.533209 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:34:26.533209 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:34:26.533209 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:34:26.533209 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:34:26.533209 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance cd28e554-c820-4789-a8df-a1aad00b7c00 could not be found. Jul 16 21:34:26.533209 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:34:26.533209 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-e7d489fb-3d4a-4579-ba85-1a82a004fbc0 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Returning 404 to user: Instance cd28e554-c820-4789-a8df-a1aad00b7c00 could not be found. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:34:26.533209 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-e7d489fb-3d4a-4579-ba85-1a82a004fbc0 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "POST /compute/v2.1/servers/cd28e554-c820-4789-a8df-a1aad00b7c00/action" status: 404 len: 111 microversion: 2.1 time: 0.016017 Jul 16 21:34:26.533209 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2463/4938] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 21:34:26 2022] POST /compute/v2.1/servers/cd28e554-c820-4789-a8df-a1aad00b7c00/action => generated 111 bytes in 17 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Jul 16 21:34:26.539463 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-9aaa0ea6-d4f5-4a4d-8886-40e6e5714203 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:34:26.540294 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-9aaa0ea6-d4f5-4a4d-8886-40e6e5714203 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: e4c7a9cb-e55d-4819-9885-2e72deffddb1] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:34:26.546551 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9aaa0ea6-d4f5-4a4d-8886-40e6e5714203 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:26.547054 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9aaa0ea6-d4f5-4a4d-8886-40e6e5714203 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:26.547826 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9aaa0ea6-d4f5-4a4d-8886-40e6e5714203 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:26.548283 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9aaa0ea6-d4f5-4a4d-8886-40e6e5714203 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:26.706388 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-9aaa0ea6-d4f5-4a4d-8886-40e6e5714203 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '22e264440fff43cdb7be6702839af37f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da2248fa190342f0bfe9211cae1e403b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:34:26.717956 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-9aaa0ea6-d4f5-4a4d-8886-40e6e5714203 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1" status: 200 len: 1504 microversion: 2.1 time: 0.182147 Jul 16 21:34:26.718559 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2476/4939] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:34:26 2022] GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1 => generated 1504 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:34:26.736986 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-56ed00d1-9b62-4dbf-b705-c54a61fca205 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:34:26.737755 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-56ed00d1-9b62-4dbf-b705-c54a61fca205 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: e4c7a9cb-e55d-4819-9885-2e72deffddb1] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:34:26.745623 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-56ed00d1-9b62-4dbf-b705-c54a61fca205 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:26.746076 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-56ed00d1-9b62-4dbf-b705-c54a61fca205 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:26.746855 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-56ed00d1-9b62-4dbf-b705-c54a61fca205 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:26.747337 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-56ed00d1-9b62-4dbf-b705-c54a61fca205 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:27.049299 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-56ed00d1-9b62-4dbf-b705-c54a61fca205 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '22e264440fff43cdb7be6702839af37f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da2248fa190342f0bfe9211cae1e403b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:34:27.059853 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-56ed00d1-9b62-4dbf-b705-c54a61fca205 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1" status: 200 len: 1504 microversion: 2.1 time: 0.327549 Jul 16 21:34:27.060506 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2464/4940] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:34:26 2022] GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1 => generated 1504 bytes in 329 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:34:27.078590 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-72030393-943d-4019-be0d-121d76361ee3 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Action: 'action', calling method: >, body: {"restore": {}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:34:27.079279 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-72030393-943d-4019-be0d-121d76361ee3 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: e4c7a9cb-e55d-4819-9885-2e72deffddb1] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:34:27.095051 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-72030393-943d-4019-be0d-121d76361ee3 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:27.095575 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-72030393-943d-4019-be0d-121d76361ee3 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:27.096337 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-72030393-943d-4019-be0d-121d76361ee3 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:27.097538 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-72030393-943d-4019-be0d-121d76361ee3 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:27.153932 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-72030393-943d-4019-be0d-121d76361ee3 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] HTTP exception thrown: Cannot 'restore' instance e4c7a9cb-e55d-4819-9885-2e72deffddb1 while it is in vm_state active Jul 16 21:34:27.155683 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-72030393-943d-4019-be0d-121d76361ee3 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Request method failure captured: Jul 16 21:34:27.155683 ncdv-master-1 devstack@n-api.service[138681]: request: POST /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1/action HTTP/1.1 Jul 16 21:34:27.155683 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:34:27.155683 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:34:27.155683 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:34:27.155683 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 15 Jul 16 21:34:27.155683 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:34:27.155683 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:34:27.155683 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:34:27.155683 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:34:27.155683 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0y6QJDVdPNcLbUnwwgOJV-QkddLQ7yqnzBNVp2CrOe5UQqjQOER3spl3PswEW82ISSx8FhmuIrEwFSG69Ze1pTOmWBWZ8PURIgVHbN3yBPA3U2S2TKkQkFwyNuvvV03MCr9U0QGJIjXVOkiqFEAy8q87U_okZgCdI-BFjBFgrCdfAEo Jul 16 21:34:27.155683 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:34:27.155683 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:34:27.155683 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:34:27.155683 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:34:27.155683 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:34:27.155683 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:34:27.155683 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: da2248fa190342f0bfe9211cae1e403b Jul 16 21:34:27.155683 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-ServersNegativeTestJSON-1102403437 Jul 16 21:34:27.155683 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:34:27.155683 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:34:27.155683 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/da2248fa190342f0bfe9211cae1e403b"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/da2248fa190342f0bfe9211cae1e403b"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_da2248fa190342f0bfe9211cae1e403b"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/da2248fa190342f0bfe9211cae1e403b"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:34:27.155683 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-ServersNegativeTestJSON-1102403437 Jul 16 21:34:27.155683 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: da2248fa190342f0bfe9211cae1e403b Jul 16 21:34:27.155683 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-ServersNegativeTestJSON-1102403437 Jul 16 21:34:27.155683 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-ServersNegativeTestJSON-1102403437-project Jul 16 21:34:27.155683 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:34:27.155683 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:34:27.155683 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: 22e264440fff43cdb7be6702839af37f Jul 16 21:34:27.155683 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-ServersNegativeTestJSON-1102403437-project Jul 16 21:34:27.155683 ncdv-master-1 devstack@n-api.service[138681]: Jul 16 21:34:27.155683 ncdv-master-1 devstack@n-api.service[138681]: {"restore": {}} Jul 16 21:34:27.155683 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:34:27.155683 ncdv-master-1 devstack@n-api.service[138681]: exception: Cannot 'restore' instance e4c7a9cb-e55d-4819-9885-2e72deffddb1 while it is in vm_state active Jul 16 21:34:27.155683 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': 'e4c7a9cb-e55d-4819-9885-2e72deffddb1', 'body': {'restore': {}}} Jul 16 21:34:27.155683 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:34:27.155683 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:34:27.155683 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/deferred_delete.py", line 42, in _restore Jul 16 21:34:27.155683 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi self.compute_api.restore(context, instance) Jul 16 21:34:27.155683 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner Jul 16 21:34:27.155683 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Jul 16 21:34:27.155683 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 145, in inner Jul 16 21:34:27.155683 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.InstanceInvalidState( Jul 16 21:34:27.155683 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.InstanceInvalidState: Instance e4c7a9cb-e55d-4819-9885-2e72deffddb1 in vm_state active. Cannot restore while the instance is in this state. Jul 16 21:34:27.155683 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:34:27.155683 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:34:27.155683 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:34:27.155683 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:34:27.155683 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:34:27.155683 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:34:27.155683 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:34:27.155683 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:34:27.155683 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:34:27.155683 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:34:27.155683 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/deferred_delete.py", line 46, in _restore Jul 16 21:34:27.155683 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi common.raise_http_conflict_for_instance_invalid_state(state_error, Jul 16 21:34:27.155683 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 347, in raise_http_conflict_for_instance_invalid_state Jul 16 21:34:27.155683 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPConflict(explanation=msg) Jul 16 21:34:27.155683 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPConflict: Cannot 'restore' instance e4c7a9cb-e55d-4819-9885-2e72deffddb1 while it is in vm_state active Jul 16 21:34:27.155683 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:34:27.155683 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:34:27.155683 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:34:27.155683 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:34:27.155683 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:34:27.163079 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:34:27.163079 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:34:27.163079 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:34:27.163079 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Cannot 'restore' instance e4c7a9cb-e55d-4819-9885-2e72deffddb1 while it is in vm_state active Jul 16 21:34:27.163079 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:34:27.163079 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-72030393-943d-4019-be0d-121d76361ee3 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Returning 409 to user: Cannot 'restore' instance e4c7a9cb-e55d-4819-9885-2e72deffddb1 while it is in vm_state active {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:34:27.163079 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-72030393-943d-4019-be0d-121d76361ee3 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "POST /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1/action" status: 409 len: 145 microversion: 2.1 time: 0.084086 Jul 16 21:34:27.163079 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2477/4941] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 21:34:27 2022] POST /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1/action => generated 145 bytes in 86 msecs (HTTP/1.1 409) 9 headers in 378 bytes (1 switches on core 0) Jul 16 21:34:27.167933 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-c2a9e72b-c198-4b34-977f-5cc5a2d94ad8 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:34:27.168612 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-c2a9e72b-c198-4b34-977f-5cc5a2d94ad8 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: e4c7a9cb-e55d-4819-9885-2e72deffddb1] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:34:27.181671 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c2a9e72b-c198-4b34-977f-5cc5a2d94ad8 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:27.182607 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c2a9e72b-c198-4b34-977f-5cc5a2d94ad8 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:27.184040 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c2a9e72b-c198-4b34-977f-5cc5a2d94ad8 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:27.184741 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c2a9e72b-c198-4b34-977f-5cc5a2d94ad8 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:27.498421 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-c2a9e72b-c198-4b34-977f-5cc5a2d94ad8 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '22e264440fff43cdb7be6702839af37f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da2248fa190342f0bfe9211cae1e403b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:34:27.505632 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-c2a9e72b-c198-4b34-977f-5cc5a2d94ad8 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1" status: 200 len: 1504 microversion: 2.1 time: 0.342554 Jul 16 21:34:27.506664 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2465/4942] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:34:27 2022] GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1 => generated 1504 bytes in 344 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:34:27.525121 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-a4b737e3-ce28-429f-ba97-4882032c0992 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:34:27.525751 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-a4b737e3-ce28-429f-ba97-4882032c0992 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: e4c7a9cb-e55d-4819-9885-2e72deffddb1] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:34:27.544126 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a4b737e3-ce28-429f-ba97-4882032c0992 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:27.544126 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a4b737e3-ce28-429f-ba97-4882032c0992 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:27.544126 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a4b737e3-ce28-429f-ba97-4882032c0992 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:27.544126 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a4b737e3-ce28-429f-ba97-4882032c0992 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:27.803922 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-a4b737e3-ce28-429f-ba97-4882032c0992 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '22e264440fff43cdb7be6702839af37f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da2248fa190342f0bfe9211cae1e403b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:34:27.814242 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-a4b737e3-ce28-429f-ba97-4882032c0992 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1" status: 200 len: 1504 microversion: 2.1 time: 0.294022 Jul 16 21:34:27.827088 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2478/4943] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:34:27 2022] GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1 => generated 1504 bytes in 308 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:34:27.833902 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-936c0d69-6e32-4ff5-93ef-89f673374634 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Action: 'action', calling method: >, body: {"resume": {}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:34:27.834671 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-936c0d69-6e32-4ff5-93ef-89f673374634 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: bc409df3-5d70-47dd-ae43-7ff59282757f] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:34:27.847212 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-936c0d69-6e32-4ff5-93ef-89f673374634 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] HTTP exception thrown: Instance bc409df3-5d70-47dd-ae43-7ff59282757f could not be found. Jul 16 21:34:27.854850 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-936c0d69-6e32-4ff5-93ef-89f673374634 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Request method failure captured: Jul 16 21:34:27.854850 ncdv-master-1 devstack@n-api.service[138680]: request: POST /compute/v2.1/servers/bc409df3-5d70-47dd-ae43-7ff59282757f/action HTTP/1.1 Jul 16 21:34:27.854850 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:34:27.854850 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:34:27.854850 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:34:27.854850 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 14 Jul 16 21:34:27.854850 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:34:27.854850 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:34:27.854850 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:34:27.854850 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:34:27.854850 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0y6QJDVdPNcLbUnwwgOJV-QkddLQ7yqnzBNVp2CrOe5UQqjQOER3spl3PswEW82ISSx8FhmuIrEwFSG69Ze1pTOmWBWZ8PURIgVHbN3yBPA3U2S2TKkQkFwyNuvvV03MCr9U0QGJIjXVOkiqFEAy8q87U_okZgCdI-BFjBFgrCdfAEo Jul 16 21:34:27.854850 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:34:27.854850 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:34:27.854850 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:34:27.854850 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:34:27.854850 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:34:27.854850 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:34:27.854850 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: da2248fa190342f0bfe9211cae1e403b Jul 16 21:34:27.854850 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-ServersNegativeTestJSON-1102403437 Jul 16 21:34:27.854850 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:34:27.854850 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:34:27.854850 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/da2248fa190342f0bfe9211cae1e403b"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/da2248fa190342f0bfe9211cae1e403b"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_da2248fa190342f0bfe9211cae1e403b"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/da2248fa190342f0bfe9211cae1e403b"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:34:27.854850 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-ServersNegativeTestJSON-1102403437 Jul 16 21:34:27.854850 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: da2248fa190342f0bfe9211cae1e403b Jul 16 21:34:27.854850 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-ServersNegativeTestJSON-1102403437 Jul 16 21:34:27.854850 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-ServersNegativeTestJSON-1102403437-project Jul 16 21:34:27.854850 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:34:27.854850 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:34:27.854850 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: 22e264440fff43cdb7be6702839af37f Jul 16 21:34:27.854850 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-ServersNegativeTestJSON-1102403437-project Jul 16 21:34:27.854850 ncdv-master-1 devstack@n-api.service[138680]: Jul 16 21:34:27.854850 ncdv-master-1 devstack@n-api.service[138680]: {"resume": {}} Jul 16 21:34:27.854850 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 21:34:27.854850 ncdv-master-1 devstack@n-api.service[138680]: exception: Instance bc409df3-5d70-47dd-ae43-7ff59282757f could not be found. Jul 16 21:34:27.854850 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': 'bc409df3-5d70-47dd-ae43-7ff59282757f', 'body': {'resume': {}}} Jul 16 21:34:27.854850 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:34:27.854850 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:34:27.854850 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:34:27.854850 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:34:27.854850 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:34:27.854850 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:34:27.854850 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2898, in _get_instance Jul 16 21:34:27.854850 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=instance_uuid) Jul 16 21:34:27.854850 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance bc409df3-5d70-47dd-ae43-7ff59282757f could not be found. Jul 16 21:34:27.854850 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:34:27.854850 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:34:27.854850 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:34:27.854850 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:34:27.854850 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:34:27.854850 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:34:27.854850 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:34:27.854850 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:34:27.854850 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:34:27.854850 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:34:27.854850 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/suspend_server.py", line 55, in _resume Jul 16 21:34:27.854850 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi server = common.get_instance(self.compute_api, context, id) Jul 16 21:34:27.854850 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:34:27.854850 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:34:27.854850 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance bc409df3-5d70-47dd-ae43-7ff59282757f could not be found. Jul 16 21:34:27.854850 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:34:27.854850 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:34:27.854850 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:34:27.854850 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:34:27.854850 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:34:27.854850 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:34:27.861959 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:34:27.861959 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:34:27.861959 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance bc409df3-5d70-47dd-ae43-7ff59282757f could not be found. Jul 16 21:34:27.861959 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:34:27.861959 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-936c0d69-6e32-4ff5-93ef-89f673374634 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Returning 404 to user: Instance bc409df3-5d70-47dd-ae43-7ff59282757f could not be found. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:34:27.861959 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-936c0d69-6e32-4ff5-93ef-89f673374634 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "POST /compute/v2.1/servers/bc409df3-5d70-47dd-ae43-7ff59282757f/action" status: 404 len: 111 microversion: 2.1 time: 0.029860 Jul 16 21:34:27.861959 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2466/4944] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 21:34:27 2022] POST /compute/v2.1/servers/bc409df3-5d70-47dd-ae43-7ff59282757f/action => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:34:27.867241 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-f8434815-e9cd-4695-a2b1-aac3f405c6c9 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:34:27.868487 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-f8434815-e9cd-4695-a2b1-aac3f405c6c9 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: e4c7a9cb-e55d-4819-9885-2e72deffddb1] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:34:27.876239 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f8434815-e9cd-4695-a2b1-aac3f405c6c9 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:27.876980 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f8434815-e9cd-4695-a2b1-aac3f405c6c9 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:27.879177 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f8434815-e9cd-4695-a2b1-aac3f405c6c9 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:27.879808 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f8434815-e9cd-4695-a2b1-aac3f405c6c9 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:28.122614 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-f8434815-e9cd-4695-a2b1-aac3f405c6c9 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '22e264440fff43cdb7be6702839af37f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da2248fa190342f0bfe9211cae1e403b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:34:28.141824 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-f8434815-e9cd-4695-a2b1-aac3f405c6c9 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1" status: 200 len: 1504 microversion: 2.1 time: 0.278664 Jul 16 21:34:28.155151 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2479/4945] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:34:27 2022] GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1 => generated 1504 bytes in 285 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:34:28.159758 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-a8e19705-a66a-4944-8dcb-7637e41fe770 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:34:28.160404 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-a8e19705-a66a-4944-8dcb-7637e41fe770 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: e4c7a9cb-e55d-4819-9885-2e72deffddb1] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:34:28.176436 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a8e19705-a66a-4944-8dcb-7637e41fe770 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:28.176872 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a8e19705-a66a-4944-8dcb-7637e41fe770 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:28.178061 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a8e19705-a66a-4944-8dcb-7637e41fe770 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:28.178838 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a8e19705-a66a-4944-8dcb-7637e41fe770 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:28.360314 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-a8e19705-a66a-4944-8dcb-7637e41fe770 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '22e264440fff43cdb7be6702839af37f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da2248fa190342f0bfe9211cae1e403b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:34:28.368902 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-a8e19705-a66a-4944-8dcb-7637e41fe770 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1" status: 200 len: 1504 microversion: 2.1 time: 0.213262 Jul 16 21:34:28.369801 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2467/4946] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:34:28 2022] GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1 => generated 1504 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:34:28.386792 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-e0b74f08-bae3-4aa3-bc13-790b81724b22 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Action: 'action', calling method: >, body: {"resume": {}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:34:28.387363 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-e0b74f08-bae3-4aa3-bc13-790b81724b22 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: e4c7a9cb-e55d-4819-9885-2e72deffddb1] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:34:28.393548 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e0b74f08-bae3-4aa3-bc13-790b81724b22 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:28.394008 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e0b74f08-bae3-4aa3-bc13-790b81724b22 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:28.394976 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e0b74f08-bae3-4aa3-bc13-790b81724b22 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:28.395635 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e0b74f08-bae3-4aa3-bc13-790b81724b22 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:28.420372 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-e0b74f08-bae3-4aa3-bc13-790b81724b22 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] HTTP exception thrown: Cannot 'resume' instance e4c7a9cb-e55d-4819-9885-2e72deffddb1 while it is in vm_state active Jul 16 21:34:28.422674 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-e0b74f08-bae3-4aa3-bc13-790b81724b22 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Request method failure captured: Jul 16 21:34:28.422674 ncdv-master-1 devstack@n-api.service[138681]: request: POST /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1/action HTTP/1.1 Jul 16 21:34:28.422674 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:34:28.422674 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:34:28.422674 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:34:28.422674 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 14 Jul 16 21:34:28.422674 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:34:28.422674 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:34:28.422674 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:34:28.422674 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:34:28.422674 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0y6QJDVdPNcLbUnwwgOJV-QkddLQ7yqnzBNVp2CrOe5UQqjQOER3spl3PswEW82ISSx8FhmuIrEwFSG69Ze1pTOmWBWZ8PURIgVHbN3yBPA3U2S2TKkQkFwyNuvvV03MCr9U0QGJIjXVOkiqFEAy8q87U_okZgCdI-BFjBFgrCdfAEo Jul 16 21:34:28.422674 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:34:28.422674 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:34:28.422674 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:34:28.422674 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:34:28.422674 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:34:28.422674 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:34:28.422674 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: da2248fa190342f0bfe9211cae1e403b Jul 16 21:34:28.422674 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-ServersNegativeTestJSON-1102403437 Jul 16 21:34:28.422674 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:34:28.422674 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:34:28.422674 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/da2248fa190342f0bfe9211cae1e403b"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/da2248fa190342f0bfe9211cae1e403b"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_da2248fa190342f0bfe9211cae1e403b"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/da2248fa190342f0bfe9211cae1e403b"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:34:28.422674 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-ServersNegativeTestJSON-1102403437 Jul 16 21:34:28.422674 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: da2248fa190342f0bfe9211cae1e403b Jul 16 21:34:28.422674 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-ServersNegativeTestJSON-1102403437 Jul 16 21:34:28.422674 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-ServersNegativeTestJSON-1102403437-project Jul 16 21:34:28.422674 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:34:28.422674 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:34:28.422674 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: 22e264440fff43cdb7be6702839af37f Jul 16 21:34:28.422674 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-ServersNegativeTestJSON-1102403437-project Jul 16 21:34:28.422674 ncdv-master-1 devstack@n-api.service[138681]: Jul 16 21:34:28.422674 ncdv-master-1 devstack@n-api.service[138681]: {"resume": {}} Jul 16 21:34:28.422674 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:34:28.422674 ncdv-master-1 devstack@n-api.service[138681]: exception: Cannot 'resume' instance e4c7a9cb-e55d-4819-9885-2e72deffddb1 while it is in vm_state active Jul 16 21:34:28.422674 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': 'e4c7a9cb-e55d-4819-9885-2e72deffddb1', 'body': {'resume': {}}} Jul 16 21:34:28.422674 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:34:28.422674 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:34:28.422674 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/suspend_server.py", line 59, in _resume Jul 16 21:34:28.422674 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi self.compute_api.resume(context, server) Jul 16 21:34:28.422674 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner Jul 16 21:34:28.422674 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Jul 16 21:34:28.422674 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 145, in inner Jul 16 21:34:28.422674 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.InstanceInvalidState( Jul 16 21:34:28.422674 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.InstanceInvalidState: Instance e4c7a9cb-e55d-4819-9885-2e72deffddb1 in vm_state active. Cannot resume while the instance is in this state. Jul 16 21:34:28.422674 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:34:28.422674 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:34:28.422674 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:34:28.422674 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:34:28.422674 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:34:28.422674 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:34:28.422674 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:34:28.422674 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:34:28.422674 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:34:28.422674 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:34:28.422674 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/suspend_server.py", line 63, in _resume Jul 16 21:34:28.422674 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi common.raise_http_conflict_for_instance_invalid_state(state_error, Jul 16 21:34:28.422674 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 347, in raise_http_conflict_for_instance_invalid_state Jul 16 21:34:28.422674 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPConflict(explanation=msg) Jul 16 21:34:28.422674 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPConflict: Cannot 'resume' instance e4c7a9cb-e55d-4819-9885-2e72deffddb1 while it is in vm_state active Jul 16 21:34:28.422674 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:34:28.422674 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:34:28.422674 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:34:28.422674 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:34:28.422674 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:34:28.434481 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:34:28.434481 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:34:28.434481 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:34:28.434481 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Cannot 'resume' instance e4c7a9cb-e55d-4819-9885-2e72deffddb1 while it is in vm_state active Jul 16 21:34:28.434481 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:34:28.434481 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-e0b74f08-bae3-4aa3-bc13-790b81724b22 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Returning 409 to user: Cannot 'resume' instance e4c7a9cb-e55d-4819-9885-2e72deffddb1 while it is in vm_state active {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:34:28.434481 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-e0b74f08-bae3-4aa3-bc13-790b81724b22 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "POST /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1/action" status: 409 len: 144 microversion: 2.1 time: 0.041591 Jul 16 21:34:28.434481 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2480/4947] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 21:34:28 2022] POST /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1/action => generated 144 bytes in 43 msecs (HTTP/1.1 409) 9 headers in 378 bytes (1 switches on core 0) Jul 16 21:34:28.437467 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-467d6839-a0e9-46a8-95a8-c7f478656965 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:34:28.438340 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-467d6839-a0e9-46a8-95a8-c7f478656965 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: e4c7a9cb-e55d-4819-9885-2e72deffddb1] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:34:28.452259 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-467d6839-a0e9-46a8-95a8-c7f478656965 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:28.452806 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-467d6839-a0e9-46a8-95a8-c7f478656965 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:28.454673 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-467d6839-a0e9-46a8-95a8-c7f478656965 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:28.455752 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-467d6839-a0e9-46a8-95a8-c7f478656965 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:28.740469 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-467d6839-a0e9-46a8-95a8-c7f478656965 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '22e264440fff43cdb7be6702839af37f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da2248fa190342f0bfe9211cae1e403b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:34:28.748445 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-467d6839-a0e9-46a8-95a8-c7f478656965 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1" status: 200 len: 1504 microversion: 2.1 time: 0.314783 Jul 16 21:34:28.749266 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2468/4948] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:34:28 2022] GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1 => generated 1504 bytes in 316 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:34:28.770231 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-3924c24b-5008-42a5-839f-b3740991159a tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:34:28.771511 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-3924c24b-5008-42a5-839f-b3740991159a tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: e4c7a9cb-e55d-4819-9885-2e72deffddb1] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:34:28.780102 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3924c24b-5008-42a5-839f-b3740991159a tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:28.780837 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3924c24b-5008-42a5-839f-b3740991159a tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:28.781826 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3924c24b-5008-42a5-839f-b3740991159a tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:28.782446 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3924c24b-5008-42a5-839f-b3740991159a tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:29.442849 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-3924c24b-5008-42a5-839f-b3740991159a tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '22e264440fff43cdb7be6702839af37f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da2248fa190342f0bfe9211cae1e403b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:34:29.457970 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-3924c24b-5008-42a5-839f-b3740991159a tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1" status: 200 len: 1504 microversion: 2.1 time: 0.692616 Jul 16 21:34:29.460208 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2481/4949] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:34:28 2022] GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1 => generated 1504 bytes in 695 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:34:29.573740 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-9fb2e131-a790-49d5-b583-d3aad2f94eb1 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Action: 'create', calling method: >, body: {"server": {"name": "", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452", "networks": [{"uuid": "f14874ef-c1c1-4761-ab34-120b93719d08"}]}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:34:29.577004 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-9fb2e131-a790-49d5-b583-d3aad2f94eb1 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Request method failure captured: Jul 16 21:34:29.577004 ncdv-master-1 devstack@n-api.service[138680]: request: POST /compute/v2.1/servers HTTP/1.1 Jul 16 21:34:29.577004 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:34:29.577004 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:34:29.577004 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:34:29.577004 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 160 Jul 16 21:34:29.577004 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:34:29.577004 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:34:29.577004 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:34:29.577004 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:34:29.577004 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0y6QJDVdPNcLbUnwwgOJV-QkddLQ7yqnzBNVp2CrOe5UQqjQOER3spl3PswEW82ISSx8FhmuIrEwFSG69Ze1pTOmWBWZ8PURIgVHbN3yBPA3U2S2TKkQkFwyNuvvV03MCr9U0QGJIjXVOkiqFEAy8q87U_okZgCdI-BFjBFgrCdfAEo Jul 16 21:34:29.577004 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:34:29.577004 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:34:29.577004 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:34:29.577004 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:34:29.577004 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:34:29.577004 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:34:29.577004 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: da2248fa190342f0bfe9211cae1e403b Jul 16 21:34:29.577004 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-ServersNegativeTestJSON-1102403437 Jul 16 21:34:29.577004 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:34:29.577004 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:34:29.577004 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/da2248fa190342f0bfe9211cae1e403b"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/da2248fa190342f0bfe9211cae1e403b"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_da2248fa190342f0bfe9211cae1e403b"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/da2248fa190342f0bfe9211cae1e403b"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:34:29.577004 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-ServersNegativeTestJSON-1102403437 Jul 16 21:34:29.577004 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: da2248fa190342f0bfe9211cae1e403b Jul 16 21:34:29.577004 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-ServersNegativeTestJSON-1102403437 Jul 16 21:34:29.577004 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-ServersNegativeTestJSON-1102403437-project Jul 16 21:34:29.577004 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:34:29.577004 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:34:29.577004 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: 22e264440fff43cdb7be6702839af37f Jul 16 21:34:29.577004 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-ServersNegativeTestJSON-1102403437-project Jul 16 21:34:29.577004 ncdv-master-1 devstack@n-api.service[138680]: Jul 16 21:34:29.577004 ncdv-master-1 devstack@n-api.service[138680]: {"server": {"name": "", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452", "networks": [{"uuid": "f14874ef-c1c1-4761-ab34-120b93719d08"}]}} Jul 16 21:34:29.577004 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 21:34:29.577004 ncdv-master-1 devstack@n-api.service[138680]: exception: Invalid input for field/attribute name. Value: . '' is too short Jul 16 21:34:29.577004 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'body': {'server': {'name': '', 'imageRef': '1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9', 'flavorRef': '452', 'networks': [{'uuid': 'f14874ef-c1c1-4761-ab34-120b93719d08'}]}}} Jul 16 21:34:29.577004 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:34:29.577004 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:34:29.577004 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 279, in validate Jul 16 21:34:29.577004 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Jul 16 21:34:29.577004 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/lib/python3/dist-packages/jsonschema/validators.py", line 353, in validate Jul 16 21:34:29.577004 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise error Jul 16 21:34:29.577004 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: '' is too short Jul 16 21:34:29.577004 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:34:29.577004 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Failed validating 'minLength' in schema['properties']['server']['properties']['name']: Jul 16 21:34:29.577004 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi {'format': 'name', 'maxLength': 255, 'minLength': 1, 'type': 'string'} Jul 16 21:34:29.577004 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:34:29.577004 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi On instance['server']: Jul 16 21:34:29.577004 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi '' Jul 16 21:34:29.577004 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:34:29.577004 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:34:29.577004 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:34:29.577004 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:34:29.577004 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:34:29.577004 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:34:29.577004 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:34:29.577004 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:34:29.577004 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:34:29.577004 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:34:29.577004 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 16 21:34:29.577004 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 16 21:34:29.577004 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Jul 16 21:34:29.577004 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Jul 16 21:34:29.577004 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Jul 16 21:34:29.577004 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Jul 16 21:34:29.577004 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 308, in validate Jul 16 21:34:29.585647 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Jul 16 21:34:29.585647 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute name. Value: . '' is too short Jul 16 21:34:29.585647 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:34:29.585647 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:34:29.585647 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:34:29.585647 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:34:29.585647 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:34:29.585647 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:34:29.585647 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Jul 16 21:34:29.585647 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Jul 16 21:34:29.585647 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute name. Value: . '' is too short Jul 16 21:34:29.585647 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:34:29.585647 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-9fb2e131-a790-49d5-b583-d3aad2f94eb1 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Returning 400 to user: Invalid input for field/attribute name. Value: . '' is too short {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:34:29.585647 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-9fb2e131-a790-49d5-b583-d3aad2f94eb1 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 400 len: 108 microversion: 2.1 time: 0.010889 Jul 16 21:34:29.585647 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2469/4950] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 21:34:29 2022] POST /compute/v2.1/servers => generated 108 bytes in 14 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Jul 16 21:34:29.592673 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-6d36c48c-5b29-4da6-84e3-3c061ac76481 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:34:29.593410 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-6d36c48c-5b29-4da6-84e3-3c061ac76481 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: e4c7a9cb-e55d-4819-9885-2e72deffddb1] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:34:29.601170 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6d36c48c-5b29-4da6-84e3-3c061ac76481 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:29.601974 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6d36c48c-5b29-4da6-84e3-3c061ac76481 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:29.603053 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6d36c48c-5b29-4da6-84e3-3c061ac76481 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:29.603731 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6d36c48c-5b29-4da6-84e3-3c061ac76481 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:29.926548 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-6d36c48c-5b29-4da6-84e3-3c061ac76481 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '22e264440fff43cdb7be6702839af37f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da2248fa190342f0bfe9211cae1e403b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:34:29.938544 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-6d36c48c-5b29-4da6-84e3-3c061ac76481 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1" status: 200 len: 1504 microversion: 2.1 time: 0.349777 Jul 16 21:34:29.953551 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2482/4951] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:34:29 2022] GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1 => generated 1504 bytes in 368 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:34:29.959652 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-2fc8daa2-8135-49cc-b12b-210aecb47dca tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:34:29.960625 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-2fc8daa2-8135-49cc-b12b-210aecb47dca tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: e4c7a9cb-e55d-4819-9885-2e72deffddb1] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:34:29.977003 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2fc8daa2-8135-49cc-b12b-210aecb47dca tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:29.977467 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2fc8daa2-8135-49cc-b12b-210aecb47dca tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:29.979287 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2fc8daa2-8135-49cc-b12b-210aecb47dca tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:29.979788 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2fc8daa2-8135-49cc-b12b-210aecb47dca tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:30.212139 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-2fc8daa2-8135-49cc-b12b-210aecb47dca tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '22e264440fff43cdb7be6702839af37f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da2248fa190342f0bfe9211cae1e403b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:34:30.222068 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-2fc8daa2-8135-49cc-b12b-210aecb47dca tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1" status: 200 len: 1504 microversion: 2.1 time: 0.264938 Jul 16 21:34:30.223820 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2470/4952] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:34:29 2022] GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1 => generated 1504 bytes in 269 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:34:30.245146 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-4f35ac87-2d02-4a70-89f9-42ad425aee84 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Action: 'action', calling method: >, body: {"shelve": {}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:34:30.245296 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-4f35ac87-2d02-4a70-89f9-42ad425aee84 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: 55197b5a-909a-4553-9160-15b10e9ddec5] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:34:30.250709 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-4f35ac87-2d02-4a70-89f9-42ad425aee84 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] HTTP exception thrown: Instance 55197b5a-909a-4553-9160-15b10e9ddec5 could not be found. Jul 16 21:34:30.260493 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-4f35ac87-2d02-4a70-89f9-42ad425aee84 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Request method failure captured: Jul 16 21:34:30.260493 ncdv-master-1 devstack@n-api.service[138681]: request: POST /compute/v2.1/servers/55197b5a-909a-4553-9160-15b10e9ddec5/action HTTP/1.1 Jul 16 21:34:30.260493 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:34:30.260493 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:34:30.260493 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:34:30.260493 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 14 Jul 16 21:34:30.260493 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:34:30.260493 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:34:30.260493 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:34:30.260493 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:34:30.260493 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0y6QJDVdPNcLbUnwwgOJV-QkddLQ7yqnzBNVp2CrOe5UQqjQOER3spl3PswEW82ISSx8FhmuIrEwFSG69Ze1pTOmWBWZ8PURIgVHbN3yBPA3U2S2TKkQkFwyNuvvV03MCr9U0QGJIjXVOkiqFEAy8q87U_okZgCdI-BFjBFgrCdfAEo Jul 16 21:34:30.260493 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:34:30.260493 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:34:30.260493 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:34:30.260493 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:34:30.260493 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:34:30.260493 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:34:30.260493 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: da2248fa190342f0bfe9211cae1e403b Jul 16 21:34:30.260493 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-ServersNegativeTestJSON-1102403437 Jul 16 21:34:30.260493 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:34:30.260493 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:34:30.260493 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/da2248fa190342f0bfe9211cae1e403b"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/da2248fa190342f0bfe9211cae1e403b"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_da2248fa190342f0bfe9211cae1e403b"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/da2248fa190342f0bfe9211cae1e403b"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:34:30.260493 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-ServersNegativeTestJSON-1102403437 Jul 16 21:34:30.260493 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: da2248fa190342f0bfe9211cae1e403b Jul 16 21:34:30.260493 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-ServersNegativeTestJSON-1102403437 Jul 16 21:34:30.260493 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-ServersNegativeTestJSON-1102403437-project Jul 16 21:34:30.260493 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:34:30.260493 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:34:30.260493 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: 22e264440fff43cdb7be6702839af37f Jul 16 21:34:30.260493 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-ServersNegativeTestJSON-1102403437-project Jul 16 21:34:30.260493 ncdv-master-1 devstack@n-api.service[138681]: Jul 16 21:34:30.260493 ncdv-master-1 devstack@n-api.service[138681]: {"shelve": {}} Jul 16 21:34:30.260493 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:34:30.260493 ncdv-master-1 devstack@n-api.service[138681]: exception: Instance 55197b5a-909a-4553-9160-15b10e9ddec5 could not be found. Jul 16 21:34:30.260493 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': '55197b5a-909a-4553-9160-15b10e9ddec5', 'body': {'shelve': {}}} Jul 16 21:34:30.260493 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:34:30.260493 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:34:30.260493 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:34:30.260493 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:34:30.260493 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:34:30.260493 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:34:30.260493 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2898, in _get_instance Jul 16 21:34:30.260493 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=instance_uuid) Jul 16 21:34:30.260493 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 55197b5a-909a-4553-9160-15b10e9ddec5 could not be found. Jul 16 21:34:30.260493 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:34:30.260493 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:34:30.260493 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:34:30.260493 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:34:30.260493 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:34:30.260493 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:34:30.260493 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:34:30.260493 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:34:30.260493 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:34:30.260493 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:34:30.260493 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/shelve.py", line 44, in _shelve Jul 16 21:34:30.260493 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, id) Jul 16 21:34:30.260493 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:34:30.260493 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:34:30.260493 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 55197b5a-909a-4553-9160-15b10e9ddec5 could not be found. Jul 16 21:34:30.260493 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:34:30.260493 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:34:30.260493 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:34:30.260493 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:34:30.260493 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:34:30.260493 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:34:30.269561 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:34:30.269561 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:34:30.269561 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 55197b5a-909a-4553-9160-15b10e9ddec5 could not be found. Jul 16 21:34:30.269561 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:34:30.269561 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-4f35ac87-2d02-4a70-89f9-42ad425aee84 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Returning 404 to user: Instance 55197b5a-909a-4553-9160-15b10e9ddec5 could not be found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:34:30.269561 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-4f35ac87-2d02-4a70-89f9-42ad425aee84 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "POST /compute/v2.1/servers/55197b5a-909a-4553-9160-15b10e9ddec5/action" status: 404 len: 111 microversion: 2.1 time: 0.025884 Jul 16 21:34:30.269561 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2483/4953] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 21:34:30 2022] POST /compute/v2.1/servers/55197b5a-909a-4553-9160-15b10e9ddec5/action => generated 111 bytes in 28 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:34:30.272624 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-f20fd3ba-4de3-4d87-86e5-a4864660b870 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:34:30.273444 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-f20fd3ba-4de3-4d87-86e5-a4864660b870 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: e4c7a9cb-e55d-4819-9885-2e72deffddb1] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:34:30.279740 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f20fd3ba-4de3-4d87-86e5-a4864660b870 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:30.280256 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f20fd3ba-4de3-4d87-86e5-a4864660b870 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:30.286323 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f20fd3ba-4de3-4d87-86e5-a4864660b870 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:30.286323 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f20fd3ba-4de3-4d87-86e5-a4864660b870 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:30.448395 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-f20fd3ba-4de3-4d87-86e5-a4864660b870 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '22e264440fff43cdb7be6702839af37f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da2248fa190342f0bfe9211cae1e403b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:34:30.462543 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-f20fd3ba-4de3-4d87-86e5-a4864660b870 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1" status: 200 len: 1504 microversion: 2.1 time: 0.193786 Jul 16 21:34:30.464055 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2471/4954] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:34:30 2022] GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1 => generated 1504 bytes in 197 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:34:30.482377 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-701d32b8-b49d-44a2-b000-7203fa7aca68 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:34:30.483046 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-701d32b8-b49d-44a2-b000-7203fa7aca68 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: e4c7a9cb-e55d-4819-9885-2e72deffddb1] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:34:30.495589 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-701d32b8-b49d-44a2-b000-7203fa7aca68 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:30.496058 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-701d32b8-b49d-44a2-b000-7203fa7aca68 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:30.500804 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-701d32b8-b49d-44a2-b000-7203fa7aca68 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:30.501279 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-701d32b8-b49d-44a2-b000-7203fa7aca68 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:30.740132 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-701d32b8-b49d-44a2-b000-7203fa7aca68 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '22e264440fff43cdb7be6702839af37f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da2248fa190342f0bfe9211cae1e403b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:34:30.751988 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-701d32b8-b49d-44a2-b000-7203fa7aca68 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1" status: 200 len: 1504 microversion: 2.1 time: 0.273689 Jul 16 21:34:30.752995 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2484/4955] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:34:30 2022] GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1 => generated 1504 bytes in 275 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:34:30.770449 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-a7a177d9-110f-4ebe-80c0-4bc2cf0cccdd tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Action: 'action', calling method: >, body: {"shelve": {}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:34:30.772595 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-a7a177d9-110f-4ebe-80c0-4bc2cf0cccdd tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: e4c7a9cb-e55d-4819-9885-2e72deffddb1] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:34:30.787707 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a7a177d9-110f-4ebe-80c0-4bc2cf0cccdd tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:30.788180 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a7a177d9-110f-4ebe-80c0-4bc2cf0cccdd tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:30.789009 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a7a177d9-110f-4ebe-80c0-4bc2cf0cccdd tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:30.789671 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a7a177d9-110f-4ebe-80c0-4bc2cf0cccdd tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:30.818321 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.objects.instance [None req-a7a177d9-110f-4ebe-80c0-4bc2cf0cccdd tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lazy-loading 'flavor' on Instance uuid e4c7a9cb-e55d-4819-9885-2e72deffddb1 {{(pid=138680) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:34:31.703893 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-a7a177d9-110f-4ebe-80c0-4bc2cf0cccdd tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "POST /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1/action" status: 202 len: 0 microversion: 2.1 time: 0.937458 Jul 16 21:34:31.707755 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2472/4956] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 21:34:30 2022] POST /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1/action => generated 0 bytes in 942 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 21:34:31.716972 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-4aaf2108-235c-4a9f-8ac9-a2f8467cf697 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:34:31.718238 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-4aaf2108-235c-4a9f-8ac9-a2f8467cf697 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: e4c7a9cb-e55d-4819-9885-2e72deffddb1] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:34:31.726544 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4aaf2108-235c-4a9f-8ac9-a2f8467cf697 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:31.727041 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4aaf2108-235c-4a9f-8ac9-a2f8467cf697 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:31.728094 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4aaf2108-235c-4a9f-8ac9-a2f8467cf697 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:31.728552 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4aaf2108-235c-4a9f-8ac9-a2f8467cf697 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:31.977343 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-4aaf2108-235c-4a9f-8ac9-a2f8467cf697 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '22e264440fff43cdb7be6702839af37f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da2248fa190342f0bfe9211cae1e403b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:34:31.979928 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=100 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:34:31.986367 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-4aaf2108-235c-4a9f-8ac9-a2f8467cf697 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1" status: 200 len: 1510 microversion: 2.1 time: 0.274016 Jul 16 21:34:31.988710 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2485/4957] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:34:31 2022] GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1 => generated 1510 bytes in 277 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:34:33.012279 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-fbef876f-0895-46dd-92cc-e2445db49c66 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:34:33.014460 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-fbef876f-0895-46dd-92cc-e2445db49c66 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: e4c7a9cb-e55d-4819-9885-2e72deffddb1] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:34:33.043076 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-fbef876f-0895-46dd-92cc-e2445db49c66 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:33.043545 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-fbef876f-0895-46dd-92cc-e2445db49c66 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:33.044576 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-fbef876f-0895-46dd-92cc-e2445db49c66 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:33.045029 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-fbef876f-0895-46dd-92cc-e2445db49c66 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:33.305024 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-fbef876f-0895-46dd-92cc-e2445db49c66 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '22e264440fff43cdb7be6702839af37f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da2248fa190342f0bfe9211cae1e403b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:34:33.315837 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-fbef876f-0895-46dd-92cc-e2445db49c66 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1" status: 200 len: 1510 microversion: 2.1 time: 0.312462 Jul 16 21:34:33.315837 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2473/4958] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:34:32 2022] GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1 => generated 1510 bytes in 315 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:34:34.333107 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-6a0475bc-96c2-4a16-aac3-0ec6e876bb2c tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:34:34.336017 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-6a0475bc-96c2-4a16-aac3-0ec6e876bb2c tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: e4c7a9cb-e55d-4819-9885-2e72deffddb1] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:34:34.369445 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6a0475bc-96c2-4a16-aac3-0ec6e876bb2c tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:34.369984 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6a0475bc-96c2-4a16-aac3-0ec6e876bb2c tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:34.370789 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6a0475bc-96c2-4a16-aac3-0ec6e876bb2c tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:34.376706 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6a0475bc-96c2-4a16-aac3-0ec6e876bb2c tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:34.650101 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-6a0475bc-96c2-4a16-aac3-0ec6e876bb2c tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '22e264440fff43cdb7be6702839af37f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da2248fa190342f0bfe9211cae1e403b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:34:34.660992 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-6a0475bc-96c2-4a16-aac3-0ec6e876bb2c tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1" status: 200 len: 1510 microversion: 2.1 time: 0.332402 Jul 16 21:34:34.661708 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2486/4959] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:34:34 2022] GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1 => generated 1510 bytes in 334 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:34:35.679599 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-bfb1682f-dcdf-485c-90d2-73618c7aa6fb tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:34:35.680895 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-bfb1682f-dcdf-485c-90d2-73618c7aa6fb tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: e4c7a9cb-e55d-4819-9885-2e72deffddb1] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:34:35.875949 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-bfb1682f-dcdf-485c-90d2-73618c7aa6fb tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:35.876441 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-bfb1682f-dcdf-485c-90d2-73618c7aa6fb tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:35.877289 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-bfb1682f-dcdf-485c-90d2-73618c7aa6fb tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:35.877753 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-bfb1682f-dcdf-485c-90d2-73618c7aa6fb tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:36.091829 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-bfb1682f-dcdf-485c-90d2-73618c7aa6fb tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '22e264440fff43cdb7be6702839af37f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da2248fa190342f0bfe9211cae1e403b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:34:36.100823 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-bfb1682f-dcdf-485c-90d2-73618c7aa6fb tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1" status: 200 len: 1510 microversion: 2.1 time: 0.426504 Jul 16 21:34:36.102559 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2474/4960] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:34:35 2022] GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1 => generated 1510 bytes in 429 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:34:37.120743 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-6fc5fe56-f5c2-4616-bd68-fa18cd71771a tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:34:37.126599 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-6fc5fe56-f5c2-4616-bd68-fa18cd71771a tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: e4c7a9cb-e55d-4819-9885-2e72deffddb1] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:34:37.132973 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6fc5fe56-f5c2-4616-bd68-fa18cd71771a tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:37.133433 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6fc5fe56-f5c2-4616-bd68-fa18cd71771a tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:37.134621 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6fc5fe56-f5c2-4616-bd68-fa18cd71771a tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:37.135089 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6fc5fe56-f5c2-4616-bd68-fa18cd71771a tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:37.388849 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-6fc5fe56-f5c2-4616-bd68-fa18cd71771a tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '22e264440fff43cdb7be6702839af37f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da2248fa190342f0bfe9211cae1e403b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:34:37.399752 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-6fc5fe56-f5c2-4616-bd68-fa18cd71771a tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1" status: 200 len: 1510 microversion: 2.1 time: 0.283485 Jul 16 21:34:37.413351 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2487/4961] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:34:37 2022] GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1 => generated 1510 bytes in 298 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:34:38.425357 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-8bcb6449-ab21-4316-84a2-9484a2b897ff tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:34:38.426186 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-8bcb6449-ab21-4316-84a2-9484a2b897ff tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: e4c7a9cb-e55d-4819-9885-2e72deffddb1] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:34:38.436671 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8bcb6449-ab21-4316-84a2-9484a2b897ff tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:38.437389 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8bcb6449-ab21-4316-84a2-9484a2b897ff tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:38.438381 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8bcb6449-ab21-4316-84a2-9484a2b897ff tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:38.439140 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8bcb6449-ab21-4316-84a2-9484a2b897ff tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:38.621952 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-8bcb6449-ab21-4316-84a2-9484a2b897ff tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '22e264440fff43cdb7be6702839af37f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da2248fa190342f0bfe9211cae1e403b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:34:38.632660 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-8bcb6449-ab21-4316-84a2-9484a2b897ff tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1" status: 200 len: 1510 microversion: 2.1 time: 0.216057 Jul 16 21:34:38.633984 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2475/4962] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:34:38 2022] GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1 => generated 1510 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:34:39.653747 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-14369db3-7e04-4591-99f0-d82ab86396e0 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:34:39.657092 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-14369db3-7e04-4591-99f0-d82ab86396e0 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: e4c7a9cb-e55d-4819-9885-2e72deffddb1] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:34:39.668206 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-14369db3-7e04-4591-99f0-d82ab86396e0 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:39.671204 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-14369db3-7e04-4591-99f0-d82ab86396e0 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:39.674209 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-14369db3-7e04-4591-99f0-d82ab86396e0 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:39.676642 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-14369db3-7e04-4591-99f0-d82ab86396e0 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:39.896997 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-14369db3-7e04-4591-99f0-d82ab86396e0 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '22e264440fff43cdb7be6702839af37f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da2248fa190342f0bfe9211cae1e403b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:34:39.907650 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-14369db3-7e04-4591-99f0-d82ab86396e0 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1" status: 200 len: 1510 microversion: 2.1 time: 0.258322 Jul 16 21:34:39.908620 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2488/4963] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:34:39 2022] GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1 => generated 1510 bytes in 260 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:34:40.927949 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-bc93c07e-6a77-44a5-be28-be23946eb18c tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:34:40.928575 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-bc93c07e-6a77-44a5-be28-be23946eb18c tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: e4c7a9cb-e55d-4819-9885-2e72deffddb1] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:34:40.948597 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_api:SELECT=80 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:34:40.950277 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-bc93c07e-6a77-44a5-be28-be23946eb18c tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:40.950749 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-bc93c07e-6a77-44a5-be28-be23946eb18c tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:40.952131 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-bc93c07e-6a77-44a5-be28-be23946eb18c tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:40.952774 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-bc93c07e-6a77-44a5-be28-be23946eb18c tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:41.443268 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-bc93c07e-6a77-44a5-be28-be23946eb18c tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '22e264440fff43cdb7be6702839af37f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da2248fa190342f0bfe9211cae1e403b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:34:41.460222 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-bc93c07e-6a77-44a5-be28-be23946eb18c tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1" status: 200 len: 1510 microversion: 2.1 time: 0.537170 Jul 16 21:34:41.461375 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2476/4964] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:34:40 2022] GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1 => generated 1510 bytes in 539 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:34:42.481462 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-486588f7-c8c3-487a-bae3-6e147ddf9b2e tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:34:42.486928 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-486588f7-c8c3-487a-bae3-6e147ddf9b2e tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: e4c7a9cb-e55d-4819-9885-2e72deffddb1] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:34:42.518194 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-486588f7-c8c3-487a-bae3-6e147ddf9b2e tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:42.518858 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-486588f7-c8c3-487a-bae3-6e147ddf9b2e tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.014s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:42.519900 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-486588f7-c8c3-487a-bae3-6e147ddf9b2e tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:42.520348 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-486588f7-c8c3-487a-bae3-6e147ddf9b2e tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:42.780967 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-486588f7-c8c3-487a-bae3-6e147ddf9b2e tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '22e264440fff43cdb7be6702839af37f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da2248fa190342f0bfe9211cae1e403b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:34:42.794882 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-486588f7-c8c3-487a-bae3-6e147ddf9b2e tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1" status: 200 len: 1531 microversion: 2.1 time: 0.321084 Jul 16 21:34:42.796036 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2489/4965] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:34:42 2022] GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1 => generated 1531 bytes in 323 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:34:43.835943 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-c3231584-0291-4dc6-9e3d-bca11957208e tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:34:43.838250 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-c3231584-0291-4dc6-9e3d-bca11957208e tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: e4c7a9cb-e55d-4819-9885-2e72deffddb1] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:34:43.855499 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c3231584-0291-4dc6-9e3d-bca11957208e tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:43.857729 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c3231584-0291-4dc6-9e3d-bca11957208e tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:43.858778 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c3231584-0291-4dc6-9e3d-bca11957208e tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:43.869073 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c3231584-0291-4dc6-9e3d-bca11957208e tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:44.107223 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-c3231584-0291-4dc6-9e3d-bca11957208e tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '22e264440fff43cdb7be6702839af37f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da2248fa190342f0bfe9211cae1e403b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:34:44.120623 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-c3231584-0291-4dc6-9e3d-bca11957208e tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1" status: 200 len: 1531 microversion: 2.1 time: 0.305877 Jul 16 21:34:44.138270 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2477/4966] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:34:43 2022] GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1 => generated 1531 bytes in 323 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:34:45.144995 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-884db4a1-7dfb-4fe9-8706-97b12a4fce06 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:34:45.146672 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-884db4a1-7dfb-4fe9-8706-97b12a4fce06 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: e4c7a9cb-e55d-4819-9885-2e72deffddb1] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:34:45.168471 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-884db4a1-7dfb-4fe9-8706-97b12a4fce06 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:45.172292 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-884db4a1-7dfb-4fe9-8706-97b12a4fce06 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:45.175832 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-884db4a1-7dfb-4fe9-8706-97b12a4fce06 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:45.183582 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-884db4a1-7dfb-4fe9-8706-97b12a4fce06 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:45.555387 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-884db4a1-7dfb-4fe9-8706-97b12a4fce06 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '22e264440fff43cdb7be6702839af37f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da2248fa190342f0bfe9211cae1e403b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:34:45.571037 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-884db4a1-7dfb-4fe9-8706-97b12a4fce06 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1" status: 200 len: 1526 microversion: 2.1 time: 0.428838 Jul 16 21:34:45.585341 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2490/4967] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:34:45 2022] GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1 => generated 1526 bytes in 449 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:34:45.597348 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-98f5e8ac-c7be-4fbe-954f-c202a9ce8f0b tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:34:45.599252 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-98f5e8ac-c7be-4fbe-954f-c202a9ce8f0b tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] [instance: 8b77de54-4c8b-4ed5-84f5-68bf69f9809f] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:34:45.608769 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-98f5e8ac-c7be-4fbe-954f-c202a9ce8f0b tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:45.610583 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-98f5e8ac-c7be-4fbe-954f-c202a9ce8f0b tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:45.611680 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-98f5e8ac-c7be-4fbe-954f-c202a9ce8f0b tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:45.612816 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-98f5e8ac-c7be-4fbe-954f-c202a9ce8f0b tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:45.634202 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:34:45.665641 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-98f5e8ac-c7be-4fbe-954f-c202a9ce8f0b tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] 10.222.0.16 "GET /compute/v2.1/servers/8b77de54-4c8b-4ed5-84f5-68bf69f9809f/os-volume_attachments" status: 200 len: 25 microversion: 2.20 time: 0.068129 Jul 16 21:34:45.665641 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2478/4968] 10.222.0.16 () {60 vars in 1342 bytes} [Sat Jul 16 21:34:45 2022] GET /compute/v2.1/servers/8b77de54-4c8b-4ed5-84f5-68bf69f9809f/os-volume_attachments => generated 25 bytes in 70 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:34:45.802522 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-09210abe-0b93-4d6e-9fd0-f0ffa692db8d tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:34:45.803697 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-09210abe-0b93-4d6e-9fd0-f0ffa692db8d tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] [instance: 8b77de54-4c8b-4ed5-84f5-68bf69f9809f] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:34:45.827923 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-09210abe-0b93-4d6e-9fd0-f0ffa692db8d tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:45.828400 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-09210abe-0b93-4d6e-9fd0-f0ffa692db8d tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:45.833463 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-09210abe-0b93-4d6e-9fd0-f0ffa692db8d tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:45.834156 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-09210abe-0b93-4d6e-9fd0-f0ffa692db8d tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:45.911678 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-09210abe-0b93-4d6e-9fd0-f0ffa692db8d tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] [instance: 8b77de54-4c8b-4ed5-84f5-68bf69f9809f] Going to try to terminate instance {{(pid=138681) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 16 21:34:46.341069 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-09210abe-0b93-4d6e-9fd0-f0ffa692db8d tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] 10.222.0.16 "DELETE /compute/v2.1/servers/8b77de54-4c8b-4ed5-84f5-68bf69f9809f" status: 204 len: 0 microversion: 2.20 time: 0.546214 Jul 16 21:34:46.344069 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2491/4969] 10.222.0.16 () {60 vars in 1279 bytes} [Sat Jul 16 21:34:45 2022] DELETE /compute/v2.1/servers/8b77de54-4c8b-4ed5-84f5-68bf69f9809f => generated 0 bytes in 549 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Jul 16 21:34:46.355548 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-63002452-6598-48a4-a117-167c3fd1475e tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:34:46.356340 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-63002452-6598-48a4-a117-167c3fd1475e tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] [instance: 8b77de54-4c8b-4ed5-84f5-68bf69f9809f] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:34:46.380471 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-63002452-6598-48a4-a117-167c3fd1475e tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:46.381203 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-63002452-6598-48a4-a117-167c3fd1475e tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:46.386765 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-63002452-6598-48a4-a117-167c3fd1475e tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:46.387492 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-63002452-6598-48a4-a117-167c3fd1475e tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:46.599836 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-6537c74b-a13c-47a5-af77-957b30fc8149 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:34:46.612664 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-6537c74b-a13c-47a5-af77-957b30fc8149 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: e4c7a9cb-e55d-4819-9885-2e72deffddb1] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:34:46.696637 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6537c74b-a13c-47a5-af77-957b30fc8149 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:46.704762 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6537c74b-a13c-47a5-af77-957b30fc8149 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.010s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:46.719938 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6537c74b-a13c-47a5-af77-957b30fc8149 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:46.724134 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6537c74b-a13c-47a5-af77-957b30fc8149 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.017s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:46.778012 ncdv-master-1 devstack@n-api.service[138681]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jul 16 21:34:46.778358 ncdv-master-1 devstack@n-api.service[138681]: Sat Jul 16 21:34:46 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1 (ip 10.222.0.16) !!! Jul 16 21:34:47.330295 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-63002452-6598-48a4-a117-167c3fd1475e tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:34:47.353074 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-63002452-6598-48a4-a117-167c3fd1475e tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:34:47.362001 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-63002452-6598-48a4-a117-167c3fd1475e tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:34:47.371505 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-63002452-6598-48a4-a117-167c3fd1475e tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] 10.222.0.16 "GET /compute/v2.1/servers/8b77de54-4c8b-4ed5-84f5-68bf69f9809f" status: 200 len: 1723 microversion: 2.20 time: 1.020714 Jul 16 21:34:47.375480 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2479/4970] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:34:46 2022] GET /compute/v2.1/servers/8b77de54-4c8b-4ed5-84f5-68bf69f9809f => generated 1723 bytes in 1029 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:34:47.555653 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-6537c74b-a13c-47a5-af77-957b30fc8149 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '22e264440fff43cdb7be6702839af37f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da2248fa190342f0bfe9211cae1e403b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:34:47.564897 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-6537c74b-a13c-47a5-af77-957b30fc8149 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1" status: 200 len: 1526 microversion: 2.1 time: 0.977297 Jul 16 21:34:47.573057 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2492/4971] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:34:46 2022] GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1 => generated 1526 bytes in 986 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:34:48.393930 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-84075aaa-6c8d-44ec-9280-a270c77fd0ea tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:34:48.396906 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-84075aaa-6c8d-44ec-9280-a270c77fd0ea tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] [instance: 8b77de54-4c8b-4ed5-84f5-68bf69f9809f] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:34:48.412989 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-84075aaa-6c8d-44ec-9280-a270c77fd0ea tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:48.417952 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-84075aaa-6c8d-44ec-9280-a270c77fd0ea tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:48.422188 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-84075aaa-6c8d-44ec-9280-a270c77fd0ea tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:48.423265 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-84075aaa-6c8d-44ec-9280-a270c77fd0ea tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:48.589763 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-87139072-a7c6-460f-af0d-37bfa26a6b0c tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:34:48.590392 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-87139072-a7c6-460f-af0d-37bfa26a6b0c tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: e4c7a9cb-e55d-4819-9885-2e72deffddb1] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:34:48.639936 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-87139072-a7c6-460f-af0d-37bfa26a6b0c tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:48.647512 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-87139072-a7c6-460f-af0d-37bfa26a6b0c tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:48.660478 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-87139072-a7c6-460f-af0d-37bfa26a6b0c tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:48.660946 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-87139072-a7c6-460f-af0d-37bfa26a6b0c tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:49.000310 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-84075aaa-6c8d-44ec-9280-a270c77fd0ea tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:34:49.014795 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-84075aaa-6c8d-44ec-9280-a270c77fd0ea tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:34:49.024261 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-84075aaa-6c8d-44ec-9280-a270c77fd0ea tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:34:49.025524 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-84075aaa-6c8d-44ec-9280-a270c77fd0ea tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] 10.222.0.16 "GET /compute/v2.1/servers/8b77de54-4c8b-4ed5-84f5-68bf69f9809f" status: 200 len: 1723 microversion: 2.20 time: 0.635991 Jul 16 21:34:49.026100 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2480/4972] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:34:48 2022] GET /compute/v2.1/servers/8b77de54-4c8b-4ed5-84f5-68bf69f9809f => generated 1723 bytes in 637 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:34:49.281123 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-87139072-a7c6-460f-af0d-37bfa26a6b0c tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '22e264440fff43cdb7be6702839af37f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da2248fa190342f0bfe9211cae1e403b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:34:49.293328 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-87139072-a7c6-460f-af0d-37bfa26a6b0c tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1" status: 200 len: 1526 microversion: 2.1 time: 0.707877 Jul 16 21:34:49.294842 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2493/4973] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:34:48 2022] GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1 => generated 1526 bytes in 710 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:34:50.055755 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-9e218726-e52c-4bb8-bb56-4797b7c3e12f tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:34:50.061337 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-9e218726-e52c-4bb8-bb56-4797b7c3e12f tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] [instance: 8b77de54-4c8b-4ed5-84f5-68bf69f9809f] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:34:50.085208 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9e218726-e52c-4bb8-bb56-4797b7c3e12f tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:50.087889 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9e218726-e52c-4bb8-bb56-4797b7c3e12f tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:50.090030 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9e218726-e52c-4bb8-bb56-4797b7c3e12f tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:50.090698 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9e218726-e52c-4bb8-bb56-4797b7c3e12f tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:50.311499 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-7dce4f3e-5e61-42e0-91a7-398750141d19 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:34:50.313761 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-7dce4f3e-5e61-42e0-91a7-398750141d19 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: e4c7a9cb-e55d-4819-9885-2e72deffddb1] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:34:50.325766 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_api:SELECT=67,nova_api:UPDATE=1 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:34:50.351424 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7dce4f3e-5e61-42e0-91a7-398750141d19 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:50.358024 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7dce4f3e-5e61-42e0-91a7-398750141d19 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:50.358834 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7dce4f3e-5e61-42e0-91a7-398750141d19 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:50.359475 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7dce4f3e-5e61-42e0-91a7-398750141d19 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:50.396704 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-9e218726-e52c-4bb8-bb56-4797b7c3e12f tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:34:50.413071 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-9e218726-e52c-4bb8-bb56-4797b7c3e12f tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:34:50.425707 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-9e218726-e52c-4bb8-bb56-4797b7c3e12f tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:34:50.427099 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-9e218726-e52c-4bb8-bb56-4797b7c3e12f tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] 10.222.0.16 "GET /compute/v2.1/servers/8b77de54-4c8b-4ed5-84f5-68bf69f9809f" status: 200 len: 1657 microversion: 2.20 time: 0.381643 Jul 16 21:34:50.430612 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2481/4974] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:34:50 2022] GET /compute/v2.1/servers/8b77de54-4c8b-4ed5-84f5-68bf69f9809f => generated 1657 bytes in 387 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:34:50.908290 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-05950099-242c-4b60-9f2a-fb1e2c2ed865 req-240e0db3-7d5c-4831-9330-31ec93e20848 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "e4c7a9cb-e55d-4819-9885-2e72deffddb1", "name": "network-vif-plugged", "status": "completed", "tag": "0b459b05-0d03-4317-8a9b-bae73b0eaddd"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:34:50.930591 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-05950099-242c-4b60-9f2a-fb1e2c2ed865 req-240e0db3-7d5c-4831-9330-31ec93e20848 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:50.932000 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-05950099-242c-4b60-9f2a-fb1e2c2ed865 req-240e0db3-7d5c-4831-9330-31ec93e20848 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:50.962975 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-05950099-242c-4b60-9f2a-fb1e2c2ed865 req-240e0db3-7d5c-4831-9330-31ec93e20848 service nova] Creating event network-vif-plugged:0b459b05-0d03-4317-8a9b-bae73b0eaddd for instance e4c7a9cb-e55d-4819-9885-2e72deffddb1 on nch2-master-1 Jul 16 21:34:50.968685 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-05950099-242c-4b60-9f2a-fb1e2c2ed865 req-240e0db3-7d5c-4831-9330-31ec93e20848 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.491944 Jul 16 21:34:50.975641 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2482/4975] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:34:50 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 498 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:34:51.455368 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-a3cfad9e-62f1-4552-8f13-4fa87f7433c9 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:34:51.457714 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-a3cfad9e-62f1-4552-8f13-4fa87f7433c9 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] [instance: 8b77de54-4c8b-4ed5-84f5-68bf69f9809f] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:34:51.468934 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a3cfad9e-62f1-4552-8f13-4fa87f7433c9 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:51.480528 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a3cfad9e-62f1-4552-8f13-4fa87f7433c9 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.012s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:51.484190 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a3cfad9e-62f1-4552-8f13-4fa87f7433c9 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:51.486871 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a3cfad9e-62f1-4552-8f13-4fa87f7433c9 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:51.560777 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-7dce4f3e-5e61-42e0-91a7-398750141d19 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '22e264440fff43cdb7be6702839af37f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da2248fa190342f0bfe9211cae1e403b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:34:51.582208 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-7dce4f3e-5e61-42e0-91a7-398750141d19 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1" status: 200 len: 1526 microversion: 2.1 time: 1.272159 Jul 16 21:34:51.593526 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2494/4976] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:34:50 2022] GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1 => generated 1526 bytes in 1287 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:34:51.683406 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-a3cfad9e-62f1-4552-8f13-4fa87f7433c9 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:34:51.696883 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-a3cfad9e-62f1-4552-8f13-4fa87f7433c9 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:34:51.703886 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-a3cfad9e-62f1-4552-8f13-4fa87f7433c9 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'ca2c9e7e6d61414cbf2c7e17753bc0df', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e451ee7bfd2d4ab58d30c33fef334b55', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:34:51.705515 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-a3cfad9e-62f1-4552-8f13-4fa87f7433c9 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] 10.222.0.16 "GET /compute/v2.1/servers/8b77de54-4c8b-4ed5-84f5-68bf69f9809f" status: 200 len: 1657 microversion: 2.20 time: 0.254439 Jul 16 21:34:51.709609 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2483/4977] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:34:51 2022] GET /compute/v2.1/servers/8b77de54-4c8b-4ed5-84f5-68bf69f9809f => generated 1657 bytes in 259 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:34:51.961112 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-08c8e8b4-004e-414b-961a-66b593b42461 req-1bf708a3-52a1-4881-a436-d82feea1b18f service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "8b77de54-4c8b-4ed5-84f5-68bf69f9809f", "name": "network-vif-deleted", "tag": "5310cc77-6413-4115-9f89-f9fab39070d1"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:34:51.988568 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-08c8e8b4-004e-414b-961a-66b593b42461 req-1bf708a3-52a1-4881-a436-d82feea1b18f service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:51.989113 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-08c8e8b4-004e-414b-961a-66b593b42461 req-1bf708a3-52a1-4881-a436-d82feea1b18f service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:52.030508 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-08c8e8b4-004e-414b-961a-66b593b42461 req-1bf708a3-52a1-4881-a436-d82feea1b18f service nova] Creating event network-vif-deleted:5310cc77-6413-4115-9f89-f9fab39070d1 for instance 8b77de54-4c8b-4ed5-84f5-68bf69f9809f on nch2-master-1 Jul 16 21:34:52.052865 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-08c8e8b4-004e-414b-961a-66b593b42461 req-1bf708a3-52a1-4881-a436-d82feea1b18f service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.349551 Jul 16 21:34:52.057718 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2495/4978] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:34:51 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 355 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 16 21:34:52.601056 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-66d04b81-b684-45fd-acd2-1c8aad96c2c8 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:34:52.602425 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-66d04b81-b684-45fd-acd2-1c8aad96c2c8 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: e4c7a9cb-e55d-4819-9885-2e72deffddb1] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:34:52.623265 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-66d04b81-b684-45fd-acd2-1c8aad96c2c8 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:52.625065 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-66d04b81-b684-45fd-acd2-1c8aad96c2c8 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:52.626051 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-66d04b81-b684-45fd-acd2-1c8aad96c2c8 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:52.627647 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-66d04b81-b684-45fd-acd2-1c8aad96c2c8 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:52.730836 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-3cdec996-d0ff-433b-b64c-f76016b45419 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:34:52.732266 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-3cdec996-d0ff-433b-b64c-f76016b45419 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] [instance: 8b77de54-4c8b-4ed5-84f5-68bf69f9809f] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:34:52.745733 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3cdec996-d0ff-433b-b64c-f76016b45419 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:52.751616 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3cdec996-d0ff-433b-b64c-f76016b45419 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:52.753722 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3cdec996-d0ff-433b-b64c-f76016b45419 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:52.755549 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3cdec996-d0ff-433b-b64c-f76016b45419 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:52.790660 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-3cdec996-d0ff-433b-b64c-f76016b45419 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] HTTP exception thrown: Instance 8b77de54-4c8b-4ed5-84f5-68bf69f9809f could not be found. Jul 16 21:34:52.792654 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-3cdec996-d0ff-433b-b64c-f76016b45419 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Request method failure captured: Jul 16 21:34:52.792654 ncdv-master-1 devstack@n-api.service[138681]: request: GET /compute/v2.1/servers/8b77de54-4c8b-4ed5-84f5-68bf69f9809f HTTP/1.1 Jul 16 21:34:52.792654 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:34:52.792654 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:34:52.792654 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:34:52.792654 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 21:34:52.792654 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:34:52.792654 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:34:52.792654 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:34:52.792654 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:34:52.792654 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0y1ucaZ1yKJJ3ZclTlid5u9GxaiMbQlzksI7M27rJ0B6rB8jj8I68KlCkH-lQUpdPSiKnnTfAiP8PzCbWLft6KLltxUTtp9U114w8i4pdCqez2cdl7YB-ynlGpqTgM-nH_ZWHIpWAB7cPaNN1ImROhBb6Fvg2kKIxL9C97eioGmF_M4 Jul 16 21:34:52.792654 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:34:52.792654 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:34:52.792654 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:34:52.792654 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:34:52.792654 ncdv-master-1 devstack@n-api.service[138681]: X-Openstack-Nova-Api-Version: 2.20 Jul 16 21:34:52.792654 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:34:52.792654 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:34:52.792654 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: e451ee7bfd2d4ab58d30c33fef334b55 Jul 16 21:34:52.792654 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-2044012735 Jul 16 21:34:52.792654 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:34:52.792654 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:34:52.792654 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/e451ee7bfd2d4ab58d30c33fef334b55"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/e451ee7bfd2d4ab58d30c33fef334b55"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_e451ee7bfd2d4ab58d30c33fef334b55"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/e451ee7bfd2d4ab58d30c33fef334b55"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:34:52.792654 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-AttachVolumeShelveTestJSON-2044012735 Jul 16 21:34:52.792654 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: e451ee7bfd2d4ab58d30c33fef334b55 Jul 16 21:34:52.792654 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-2044012735 Jul 16 21:34:52.792654 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-AttachVolumeShelveTestJSON-2044012735-project Jul 16 21:34:52.792654 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:34:52.792654 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:34:52.792654 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: ca2c9e7e6d61414cbf2c7e17753bc0df Jul 16 21:34:52.792654 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-AttachVolumeShelveTestJSON-2044012735-project Jul 16 21:34:52.792654 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:34:52.792654 ncdv-master-1 devstack@n-api.service[138681]: exception: Instance 8b77de54-4c8b-4ed5-84f5-68bf69f9809f could not be found. Jul 16 21:34:52.792654 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': '8b77de54-4c8b-4ed5-84f5-68bf69f9809f'} Jul 16 21:34:52.792654 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:34:52.792654 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:34:52.792654 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:34:52.792654 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:34:52.792654 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:34:52.792654 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:34:52.792654 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:34:52.792654 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:34:52.792654 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:34:52.792654 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:34:52.792654 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 8b77de54-4c8b-4ed5-84f5-68bf69f9809f could not be found. Jul 16 21:34:52.792654 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:34:52.792654 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:34:52.792654 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:34:52.792654 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:34:52.792654 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:34:52.792654 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:34:52.792654 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:34:52.792654 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:34:52.792654 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:34:52.792654 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:34:52.792654 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:34:52.792654 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:34:52.792654 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:34:52.792654 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:34:52.792654 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:34:52.792654 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:34:52.792654 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 8b77de54-4c8b-4ed5-84f5-68bf69f9809f could not be found. Jul 16 21:34:52.792654 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:34:52.792654 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:34:52.798208 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:34:52.798208 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:34:52.798208 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:34:52.798208 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:34:52.798208 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:34:52.798208 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:34:52.798208 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 8b77de54-4c8b-4ed5-84f5-68bf69f9809f could not be found. Jul 16 21:34:52.798208 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:34:52.801314 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-3cdec996-d0ff-433b-b64c-f76016b45419 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Returning 404 to user: Instance 8b77de54-4c8b-4ed5-84f5-68bf69f9809f could not be found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:34:52.805593 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-3cdec996-d0ff-433b-b64c-f76016b45419 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] 10.222.0.16 "GET /compute/v2.1/servers/8b77de54-4c8b-4ed5-84f5-68bf69f9809f" status: 404 len: 111 microversion: 2.20 time: 0.080559 Jul 16 21:34:52.806543 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2496/4979] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:34:52 2022] GET /compute/v2.1/servers/8b77de54-4c8b-4ed5-84f5-68bf69f9809f => generated 111 bytes in 83 msecs (HTTP/1.1 404) 9 headers in 381 bytes (2 switches on core 0) Jul 16 21:34:52.831965 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-2604ab07-b20e-4a41-864f-69a8bad36e57 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Calling method '.version_select at 0x7fdd9d257ca0>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:34:52.955298 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-2604ab07-b20e-4a41-864f-69a8bad36e57 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] 10.222.0.16 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1464370033" status: 204 len: 0 microversion: 2.20 time: 0.136019 Jul 16 21:34:52.955298 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2497/4980] 10.222.0.16 () {60 vars in 1261 bytes} [Sat Jul 16 21:34:52 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1464370033 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Jul 16 21:34:52.964977 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-66d04b81-b684-45fd-acd2-1c8aad96c2c8 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '22e264440fff43cdb7be6702839af37f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da2248fa190342f0bfe9211cae1e403b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:34:52.975542 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-66d04b81-b684-45fd-acd2-1c8aad96c2c8 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1" status: 200 len: 1526 microversion: 2.1 time: 0.379203 Jul 16 21:34:52.986863 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2484/4981] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:34:52 2022] GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1 => generated 1526 bytes in 391 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:34:54.040687 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-96176d69-229e-4eee-885c-96bc22a87c1d tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:34:54.040687 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-96176d69-229e-4eee-885c-96bc22a87c1d tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: e4c7a9cb-e55d-4819-9885-2e72deffddb1] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:34:54.040687 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-96176d69-229e-4eee-885c-96bc22a87c1d tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:54.040687 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-96176d69-229e-4eee-885c-96bc22a87c1d tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:54.040687 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-96176d69-229e-4eee-885c-96bc22a87c1d tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:54.040687 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-96176d69-229e-4eee-885c-96bc22a87c1d tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:54.151406 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-96176d69-229e-4eee-885c-96bc22a87c1d tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '22e264440fff43cdb7be6702839af37f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da2248fa190342f0bfe9211cae1e403b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:34:54.159529 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-96176d69-229e-4eee-885c-96bc22a87c1d tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1" status: 200 len: 1526 microversion: 2.1 time: 0.169326 Jul 16 21:34:54.171232 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2498/4982] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:34:53 2022] GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1 => generated 1526 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:34:55.181367 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-b0dabf47-54ca-4acc-82c6-2980758a5f07 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:34:55.184627 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-b0dabf47-54ca-4acc-82c6-2980758a5f07 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: e4c7a9cb-e55d-4819-9885-2e72deffddb1] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:34:55.199213 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b0dabf47-54ca-4acc-82c6-2980758a5f07 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:55.199675 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b0dabf47-54ca-4acc-82c6-2980758a5f07 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:55.201820 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b0dabf47-54ca-4acc-82c6-2980758a5f07 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:55.201820 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b0dabf47-54ca-4acc-82c6-2980758a5f07 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:55.561776 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-b0dabf47-54ca-4acc-82c6-2980758a5f07 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '22e264440fff43cdb7be6702839af37f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da2248fa190342f0bfe9211cae1e403b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:34:55.580621 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-b0dabf47-54ca-4acc-82c6-2980758a5f07 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1" status: 200 len: 1526 microversion: 2.1 time: 0.396079 Jul 16 21:34:55.580621 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2485/4983] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:34:55 2022] GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1 => generated 1526 bytes in 403 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:34:56.588385 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-5f886e40-47c5-4c88-ae58-bbcbeeda6ee0 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:34:56.590230 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-5f886e40-47c5-4c88-ae58-bbcbeeda6ee0 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: e4c7a9cb-e55d-4819-9885-2e72deffddb1] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:34:56.600691 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5f886e40-47c5-4c88-ae58-bbcbeeda6ee0 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:56.601337 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5f886e40-47c5-4c88-ae58-bbcbeeda6ee0 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:56.605729 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5f886e40-47c5-4c88-ae58-bbcbeeda6ee0 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:56.605729 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5f886e40-47c5-4c88-ae58-bbcbeeda6ee0 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:56.904225 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-5f886e40-47c5-4c88-ae58-bbcbeeda6ee0 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '22e264440fff43cdb7be6702839af37f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da2248fa190342f0bfe9211cae1e403b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:34:56.919792 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-5f886e40-47c5-4c88-ae58-bbcbeeda6ee0 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1" status: 200 len: 1526 microversion: 2.1 time: 0.335433 Jul 16 21:34:56.926906 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2499/4984] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:34:56 2022] GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1 => generated 1526 bytes in 343 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:34:57.946552 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-87ed38d0-a2b1-427f-936b-25b68aad8ec9 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:34:57.949679 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-87ed38d0-a2b1-427f-936b-25b68aad8ec9 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: e4c7a9cb-e55d-4819-9885-2e72deffddb1] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:34:57.959378 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-87ed38d0-a2b1-427f-936b-25b68aad8ec9 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:57.963764 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-87ed38d0-a2b1-427f-936b-25b68aad8ec9 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:57.964654 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-87ed38d0-a2b1-427f-936b-25b68aad8ec9 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:57.968091 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-87ed38d0-a2b1-427f-936b-25b68aad8ec9 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:58.267825 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-87ed38d0-a2b1-427f-936b-25b68aad8ec9 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '22e264440fff43cdb7be6702839af37f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da2248fa190342f0bfe9211cae1e403b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:34:58.283421 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-87ed38d0-a2b1-427f-936b-25b68aad8ec9 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1" status: 200 len: 1526 microversion: 2.1 time: 0.342148 Jul 16 21:34:58.284376 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2486/4985] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:34:57 2022] GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1 => generated 1526 bytes in 349 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:34:59.303873 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-0c1de452-d3fd-4f03-a69b-50f64b38e60a tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:34:59.305793 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-0c1de452-d3fd-4f03-a69b-50f64b38e60a tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: e4c7a9cb-e55d-4819-9885-2e72deffddb1] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:34:59.326505 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0c1de452-d3fd-4f03-a69b-50f64b38e60a tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:59.327296 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0c1de452-d3fd-4f03-a69b-50f64b38e60a tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:34:59.328343 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0c1de452-d3fd-4f03-a69b-50f64b38e60a tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:34:59.329335 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0c1de452-d3fd-4f03-a69b-50f64b38e60a tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:00.133759 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-0c1de452-d3fd-4f03-a69b-50f64b38e60a tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '22e264440fff43cdb7be6702839af37f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da2248fa190342f0bfe9211cae1e403b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:35:00.147055 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-0c1de452-d3fd-4f03-a69b-50f64b38e60a tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1" status: 200 len: 1526 microversion: 2.1 time: 0.845331 Jul 16 21:35:00.147055 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2500/4986] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:34:59 2022] GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1 => generated 1526 bytes in 850 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:35:01.172879 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-e2877239-f9f1-4ceb-8f6e-300a6b4c58f9 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:35:01.178723 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-e2877239-f9f1-4ceb-8f6e-300a6b4c58f9 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: e4c7a9cb-e55d-4819-9885-2e72deffddb1] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:35:01.226787 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e2877239-f9f1-4ceb-8f6e-300a6b4c58f9 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:01.227988 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e2877239-f9f1-4ceb-8f6e-300a6b4c58f9 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:01.230021 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e2877239-f9f1-4ceb-8f6e-300a6b4c58f9 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:01.233621 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e2877239-f9f1-4ceb-8f6e-300a6b4c58f9 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:01.249936 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-366e50bc-a9d1-442c-b228-824cca8dbfe5 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:35:01.251195 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-366e50bc-a9d1-442c-b228-824cca8dbfe5 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] [instance: 8b77de54-4c8b-4ed5-84f5-68bf69f9809f] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:35:01.262256 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-366e50bc-a9d1-442c-b228-824cca8dbfe5 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:01.265906 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-366e50bc-a9d1-442c-b228-824cca8dbfe5 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:01.270340 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-366e50bc-a9d1-442c-b228-824cca8dbfe5 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:01.270340 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-366e50bc-a9d1-442c-b228-824cca8dbfe5 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:01.319873 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-366e50bc-a9d1-442c-b228-824cca8dbfe5 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] HTTP exception thrown: Instance 8b77de54-4c8b-4ed5-84f5-68bf69f9809f could not be found. Jul 16 21:35:01.322004 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-366e50bc-a9d1-442c-b228-824cca8dbfe5 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Request method failure captured: Jul 16 21:35:01.322004 ncdv-master-1 devstack@n-api.service[138681]: request: DELETE /compute/v2.1/servers/8b77de54-4c8b-4ed5-84f5-68bf69f9809f HTTP/1.1 Jul 16 21:35:01.322004 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:35:01.322004 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:35:01.322004 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:35:01.322004 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 21:35:01.322004 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:35:01.322004 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:35:01.322004 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:35:01.322004 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:35:01.322004 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0y1ucaZ1yKJJ3ZclTlid5u9GxaiMbQlzksI7M27rJ0B6rB8jj8I68KlCkH-lQUpdPSiKnnTfAiP8PzCbWLft6KLltxUTtp9U114w8i4pdCqez2cdl7YB-ynlGpqTgM-nH_ZWHIpWAB7cPaNN1ImROhBb6Fvg2kKIxL9C97eioGmF_M4 Jul 16 21:35:01.322004 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:35:01.322004 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:35:01.322004 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:35:01.322004 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:35:01.322004 ncdv-master-1 devstack@n-api.service[138681]: X-Openstack-Nova-Api-Version: 2.20 Jul 16 21:35:01.322004 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:35:01.322004 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:35:01.322004 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: e451ee7bfd2d4ab58d30c33fef334b55 Jul 16 21:35:01.322004 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-2044012735 Jul 16 21:35:01.322004 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:35:01.322004 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:35:01.322004 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/e451ee7bfd2d4ab58d30c33fef334b55"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/e451ee7bfd2d4ab58d30c33fef334b55"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_e451ee7bfd2d4ab58d30c33fef334b55"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/e451ee7bfd2d4ab58d30c33fef334b55"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:35:01.322004 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-AttachVolumeShelveTestJSON-2044012735 Jul 16 21:35:01.322004 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: e451ee7bfd2d4ab58d30c33fef334b55 Jul 16 21:35:01.322004 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-2044012735 Jul 16 21:35:01.322004 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-AttachVolumeShelveTestJSON-2044012735-project Jul 16 21:35:01.322004 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:35:01.322004 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:35:01.322004 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: ca2c9e7e6d61414cbf2c7e17753bc0df Jul 16 21:35:01.322004 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-AttachVolumeShelveTestJSON-2044012735-project Jul 16 21:35:01.322004 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:35:01.322004 ncdv-master-1 devstack@n-api.service[138681]: exception: Instance 8b77de54-4c8b-4ed5-84f5-68bf69f9809f could not be found. Jul 16 21:35:01.322004 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': '8b77de54-4c8b-4ed5-84f5-68bf69f9809f'} Jul 16 21:35:01.322004 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:35:01.322004 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:35:01.322004 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:35:01.322004 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:35:01.322004 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:35:01.322004 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:35:01.322004 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:35:01.322004 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:35:01.322004 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:35:01.322004 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:35:01.322004 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 8b77de54-4c8b-4ed5-84f5-68bf69f9809f could not be found. Jul 16 21:35:01.322004 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:35:01.322004 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:35:01.322004 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:35:01.322004 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:35:01.322004 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:35:01.322004 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:35:01.322004 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:35:01.322004 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:35:01.322004 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:35:01.322004 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:35:01.322004 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Jul 16 21:35:01.322004 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Jul 16 21:35:01.322004 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Jul 16 21:35:01.322004 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Jul 16 21:35:01.322004 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:35:01.322004 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:35:01.322004 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:35:01.322004 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:35:01.326540 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 8b77de54-4c8b-4ed5-84f5-68bf69f9809f could not be found. Jul 16 21:35:01.326540 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:35:01.326540 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:35:01.326540 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:35:01.326540 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:35:01.326540 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:35:01.326540 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:35:01.326540 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:35:01.326540 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:35:01.326540 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 8b77de54-4c8b-4ed5-84f5-68bf69f9809f could not be found. Jul 16 21:35:01.326540 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:35:01.333941 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-366e50bc-a9d1-442c-b228-824cca8dbfe5 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Returning 404 to user: Instance 8b77de54-4c8b-4ed5-84f5-68bf69f9809f could not be found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:35:01.335159 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-366e50bc-a9d1-442c-b228-824cca8dbfe5 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] 10.222.0.16 "DELETE /compute/v2.1/servers/8b77de54-4c8b-4ed5-84f5-68bf69f9809f" status: 404 len: 111 microversion: 2.20 time: 0.091629 Jul 16 21:35:01.337018 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2501/4987] 10.222.0.16 () {60 vars in 1279 bytes} [Sat Jul 16 21:35:01 2022] DELETE /compute/v2.1/servers/8b77de54-4c8b-4ed5-84f5-68bf69f9809f => generated 111 bytes in 94 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Jul 16 21:35:01.350898 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-b14f6d01-a9ec-4c83-ac76-04cb971f32ea tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:35:01.356316 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-b14f6d01-a9ec-4c83-ac76-04cb971f32ea tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] [instance: 8b77de54-4c8b-4ed5-84f5-68bf69f9809f] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:35:01.374336 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b14f6d01-a9ec-4c83-ac76-04cb971f32ea tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:01.375129 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b14f6d01-a9ec-4c83-ac76-04cb971f32ea tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:01.376256 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b14f6d01-a9ec-4c83-ac76-04cb971f32ea tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:01.377058 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b14f6d01-a9ec-4c83-ac76-04cb971f32ea tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:01.418136 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-b14f6d01-a9ec-4c83-ac76-04cb971f32ea tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] HTTP exception thrown: Instance 8b77de54-4c8b-4ed5-84f5-68bf69f9809f could not be found. Jul 16 21:35:01.422205 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-b14f6d01-a9ec-4c83-ac76-04cb971f32ea tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Request method failure captured: Jul 16 21:35:01.422205 ncdv-master-1 devstack@n-api.service[138681]: request: GET /compute/v2.1/servers/8b77de54-4c8b-4ed5-84f5-68bf69f9809f HTTP/1.1 Jul 16 21:35:01.422205 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:35:01.422205 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:35:01.422205 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:35:01.422205 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 21:35:01.422205 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:35:01.422205 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:35:01.422205 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:35:01.422205 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:35:01.422205 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0y1ucaZ1yKJJ3ZclTlid5u9GxaiMbQlzksI7M27rJ0B6rB8jj8I68KlCkH-lQUpdPSiKnnTfAiP8PzCbWLft6KLltxUTtp9U114w8i4pdCqez2cdl7YB-ynlGpqTgM-nH_ZWHIpWAB7cPaNN1ImROhBb6Fvg2kKIxL9C97eioGmF_M4 Jul 16 21:35:01.422205 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:35:01.422205 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:35:01.422205 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:35:01.422205 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:35:01.422205 ncdv-master-1 devstack@n-api.service[138681]: X-Openstack-Nova-Api-Version: 2.20 Jul 16 21:35:01.422205 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:35:01.422205 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:35:01.422205 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: e451ee7bfd2d4ab58d30c33fef334b55 Jul 16 21:35:01.422205 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-2044012735 Jul 16 21:35:01.422205 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:35:01.422205 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:35:01.422205 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/e451ee7bfd2d4ab58d30c33fef334b55"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/e451ee7bfd2d4ab58d30c33fef334b55"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_e451ee7bfd2d4ab58d30c33fef334b55"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/e451ee7bfd2d4ab58d30c33fef334b55"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:35:01.422205 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-AttachVolumeShelveTestJSON-2044012735 Jul 16 21:35:01.422205 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: e451ee7bfd2d4ab58d30c33fef334b55 Jul 16 21:35:01.422205 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-2044012735 Jul 16 21:35:01.422205 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-AttachVolumeShelveTestJSON-2044012735-project Jul 16 21:35:01.422205 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:35:01.422205 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:35:01.422205 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: ca2c9e7e6d61414cbf2c7e17753bc0df Jul 16 21:35:01.422205 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-AttachVolumeShelveTestJSON-2044012735-project Jul 16 21:35:01.422205 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:35:01.422205 ncdv-master-1 devstack@n-api.service[138681]: exception: Instance 8b77de54-4c8b-4ed5-84f5-68bf69f9809f could not be found. Jul 16 21:35:01.422205 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': '8b77de54-4c8b-4ed5-84f5-68bf69f9809f'} Jul 16 21:35:01.422205 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:35:01.422205 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:35:01.422205 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:35:01.422205 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:35:01.422205 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:35:01.422205 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:35:01.422205 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:35:01.422205 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:35:01.422205 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:35:01.422205 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:35:01.422205 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 8b77de54-4c8b-4ed5-84f5-68bf69f9809f could not be found. Jul 16 21:35:01.422205 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:35:01.422205 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:35:01.422205 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:35:01.422205 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:35:01.422205 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:35:01.422205 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:35:01.422205 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:35:01.422205 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:35:01.422205 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:35:01.422205 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:35:01.422205 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:35:01.422205 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:35:01.422205 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:35:01.422205 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:35:01.422205 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:35:01.422205 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:35:01.422205 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 8b77de54-4c8b-4ed5-84f5-68bf69f9809f could not be found. Jul 16 21:35:01.422205 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:35:01.422205 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:35:01.433344 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:35:01.433344 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:35:01.433344 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:35:01.433344 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:35:01.433344 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:35:01.433344 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:35:01.433344 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 8b77de54-4c8b-4ed5-84f5-68bf69f9809f could not be found. Jul 16 21:35:01.433344 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:35:01.443663 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-b14f6d01-a9ec-4c83-ac76-04cb971f32ea tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Returning 404 to user: Instance 8b77de54-4c8b-4ed5-84f5-68bf69f9809f could not be found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:35:01.445531 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-b14f6d01-a9ec-4c83-ac76-04cb971f32ea tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] 10.222.0.16 "GET /compute/v2.1/servers/8b77de54-4c8b-4ed5-84f5-68bf69f9809f" status: 404 len: 111 microversion: 2.20 time: 0.101519 Jul 16 21:35:01.446643 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2502/4988] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:35:01 2022] GET /compute/v2.1/servers/8b77de54-4c8b-4ed5-84f5-68bf69f9809f => generated 111 bytes in 104 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Jul 16 21:35:01.658303 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-e2877239-f9f1-4ceb-8f6e-300a6b4c58f9 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '22e264440fff43cdb7be6702839af37f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da2248fa190342f0bfe9211cae1e403b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:35:01.666842 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-e2877239-f9f1-4ceb-8f6e-300a6b4c58f9 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1" status: 200 len: 1526 microversion: 2.1 time: 0.505977 Jul 16 21:35:01.667474 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2487/4989] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:35:01 2022] GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1 => generated 1526 bytes in 508 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:35:02.686306 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-1ffd7be4-8f20-4fca-b92c-2f6bff24a426 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:35:02.687585 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-1ffd7be4-8f20-4fca-b92c-2f6bff24a426 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: e4c7a9cb-e55d-4819-9885-2e72deffddb1] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:35:02.708747 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1ffd7be4-8f20-4fca-b92c-2f6bff24a426 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:02.728627 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1ffd7be4-8f20-4fca-b92c-2f6bff24a426 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:02.728627 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1ffd7be4-8f20-4fca-b92c-2f6bff24a426 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:02.730647 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1ffd7be4-8f20-4fca-b92c-2f6bff24a426 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:03.232275 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-1ffd7be4-8f20-4fca-b92c-2f6bff24a426 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '22e264440fff43cdb7be6702839af37f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da2248fa190342f0bfe9211cae1e403b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:35:03.245018 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-1ffd7be4-8f20-4fca-b92c-2f6bff24a426 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1" status: 200 len: 1526 microversion: 2.1 time: 0.563082 Jul 16 21:35:03.258547 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2503/4990] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:35:02 2022] GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1 => generated 1526 bytes in 578 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:35:04.276845 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-372ffe7e-5914-4d2f-94b6-62b7c27f1b57 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:35:04.277478 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-372ffe7e-5914-4d2f-94b6-62b7c27f1b57 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: e4c7a9cb-e55d-4819-9885-2e72deffddb1] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:35:04.318588 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-372ffe7e-5914-4d2f-94b6-62b7c27f1b57 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:04.325054 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-372ffe7e-5914-4d2f-94b6-62b7c27f1b57 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:04.326133 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-372ffe7e-5914-4d2f-94b6-62b7c27f1b57 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:04.334521 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-372ffe7e-5914-4d2f-94b6-62b7c27f1b57 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:04.829958 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-b5774d21-476f-4db7-8566-6b1bb6d6b1af tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Action: 'create', calling method: .version_select at 0x7fdd9cf903a0>, body: {"keypair": {"name": "tempest-keypair-1683927826", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBC5pQdB9n1REkFMvN8h/yYxGZRYnthD1r0uY8+1dQDUBEPiOHQNqF1TuoE2zQXOtqvpGsP64KEdS65kso6JIcxbOEuTX/Cjn3rvxsu39abZ9Cuod/rkneYPSWehgiA6Oyw=="}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:35:04.881326 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-b5774d21-476f-4db7-8566-6b1bb6d6b1af tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Getting quotas for project 0fd0ce287cb44cd59045868ce688909a. Resources: {'key_pairs'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:35:04.897711 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-b5774d21-476f-4db7-8566-6b1bb6d6b1af tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Getting quotas for user 726b46c54e9541fbb7825607bac3abe5 and project 0fd0ce287cb44cd59045868ce688909a. Resources: {'key_pairs'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:35:04.915389 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-372ffe7e-5914-4d2f-94b6-62b7c27f1b57 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '22e264440fff43cdb7be6702839af37f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da2248fa190342f0bfe9211cae1e403b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:35:04.928451 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-372ffe7e-5914-4d2f-94b6-62b7c27f1b57 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1" status: 200 len: 1526 microversion: 2.1 time: 0.658003 Jul 16 21:35:04.932195 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2488/4991] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:35:04 2022] GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1 => generated 1526 bytes in 662 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:35:05.059993 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-b5774d21-476f-4db7-8566-6b1bb6d6b1af tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] 10.222.0.16 "POST /compute/v2.1/os-keypairs" status: 201 len: 401 microversion: 2.42 time: 0.233982 Jul 16 21:35:05.062060 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2504/4992] 10.222.0.16 () {62 vars in 1199 bytes} [Sat Jul 16 21:35:04 2022] POST /compute/v2.1/os-keypairs => generated 401 bytes in 242 msecs (HTTP/1.1 201) 9 headers in 364 bytes (1 switches on core 0) Jul 16 21:35:05.947398 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-4131e867-b394-4347-8ae0-0fc44049cee3 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:35:05.948092 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-4131e867-b394-4347-8ae0-0fc44049cee3 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: e4c7a9cb-e55d-4819-9885-2e72deffddb1] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:35:05.969519 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4131e867-b394-4347-8ae0-0fc44049cee3 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:05.970061 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4131e867-b394-4347-8ae0-0fc44049cee3 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:05.970903 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4131e867-b394-4347-8ae0-0fc44049cee3 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:05.977452 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4131e867-b394-4347-8ae0-0fc44049cee3 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:06.309863 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-ae035bbc-f3ac-47dc-8e66-dcf8bbc623bb tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:35:06.311454 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-ae035bbc-f3ac-47dc-8e66-dcf8bbc623bb tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] [instance: 08a33911-6e47-45fd-b499-155f199aacf8] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:35:06.336642 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ae035bbc-f3ac-47dc-8e66-dcf8bbc623bb tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:06.337101 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ae035bbc-f3ac-47dc-8e66-dcf8bbc623bb tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:06.337903 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ae035bbc-f3ac-47dc-8e66-dcf8bbc623bb tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:06.347859 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ae035bbc-f3ac-47dc-8e66-dcf8bbc623bb tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:06.366454 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=96,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:35:06.407462 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-4131e867-b394-4347-8ae0-0fc44049cee3 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '22e264440fff43cdb7be6702839af37f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da2248fa190342f0bfe9211cae1e403b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:35:06.420245 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-ae035bbc-f3ac-47dc-8e66-dcf8bbc623bb tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] HTTP exception thrown: Instance 08a33911-6e47-45fd-b499-155f199aacf8 could not be found. Jul 16 21:35:06.420490 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-4131e867-b394-4347-8ae0-0fc44049cee3 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1" status: 200 len: 1526 microversion: 2.1 time: 0.477043 Jul 16 21:35:06.421357 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2489/4993] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:35:05 2022] GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1 => generated 1526 bytes in 479 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:35:06.421939 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-ae035bbc-f3ac-47dc-8e66-dcf8bbc623bb tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Request method failure captured: Jul 16 21:35:06.421939 ncdv-master-1 devstack@n-api.service[138681]: request: DELETE /compute/v2.1/servers/08a33911-6e47-45fd-b499-155f199aacf8 HTTP/1.1 Jul 16 21:35:06.421939 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:35:06.421939 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:35:06.421939 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:35:06.421939 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 21:35:06.421939 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:35:06.421939 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:35:06.421939 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:35:06.421939 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:35:06.421939 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0y1ucaZ1yKJJ3ZclTlid5u9GxaiMbQlzksI7M27rJ0B6rB8jj8I68KlCkH-lQUpdPSiKnnTfAiP8PzCbWLft6KLltxUTtp9U114w8i4pdCqez2cdl7YB-ynlGpqTgM-nH_ZWHIpWAB7cPaNN1ImROhBb6Fvg2kKIxL9C97eioGmF_M4 Jul 16 21:35:06.421939 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:35:06.421939 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:35:06.421939 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:35:06.421939 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:35:06.421939 ncdv-master-1 devstack@n-api.service[138681]: X-Openstack-Nova-Api-Version: 2.20 Jul 16 21:35:06.421939 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:35:06.421939 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:35:06.421939 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: e451ee7bfd2d4ab58d30c33fef334b55 Jul 16 21:35:06.421939 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-2044012735 Jul 16 21:35:06.421939 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:35:06.421939 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:35:06.421939 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/e451ee7bfd2d4ab58d30c33fef334b55"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/e451ee7bfd2d4ab58d30c33fef334b55"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_e451ee7bfd2d4ab58d30c33fef334b55"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/e451ee7bfd2d4ab58d30c33fef334b55"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:35:06.421939 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-AttachVolumeShelveTestJSON-2044012735 Jul 16 21:35:06.421939 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: e451ee7bfd2d4ab58d30c33fef334b55 Jul 16 21:35:06.421939 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-2044012735 Jul 16 21:35:06.421939 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-AttachVolumeShelveTestJSON-2044012735-project Jul 16 21:35:06.421939 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:35:06.421939 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:35:06.421939 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: ca2c9e7e6d61414cbf2c7e17753bc0df Jul 16 21:35:06.421939 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-AttachVolumeShelveTestJSON-2044012735-project Jul 16 21:35:06.421939 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:35:06.421939 ncdv-master-1 devstack@n-api.service[138681]: exception: Instance 08a33911-6e47-45fd-b499-155f199aacf8 could not be found. Jul 16 21:35:06.421939 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': '08a33911-6e47-45fd-b499-155f199aacf8'} Jul 16 21:35:06.421939 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:35:06.421939 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:35:06.421939 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:35:06.421939 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:35:06.421939 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:35:06.421939 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:35:06.421939 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:35:06.421939 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:35:06.421939 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:35:06.421939 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:35:06.421939 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 08a33911-6e47-45fd-b499-155f199aacf8 could not be found. Jul 16 21:35:06.421939 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:35:06.421939 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:35:06.421939 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:35:06.421939 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:35:06.421939 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:35:06.421939 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:35:06.421939 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:35:06.421939 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:35:06.421939 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:35:06.421939 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:35:06.421939 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Jul 16 21:35:06.421939 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Jul 16 21:35:06.421939 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Jul 16 21:35:06.421939 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Jul 16 21:35:06.421939 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:35:06.421939 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:35:06.421939 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:35:06.421939 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:35:06.427217 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 08a33911-6e47-45fd-b499-155f199aacf8 could not be found. Jul 16 21:35:06.427217 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:35:06.427217 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:35:06.427217 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:35:06.427217 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:35:06.427217 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:35:06.427217 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:35:06.427217 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:35:06.427217 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:35:06.427217 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 08a33911-6e47-45fd-b499-155f199aacf8 could not be found. Jul 16 21:35:06.427217 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:35:06.427217 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-ae035bbc-f3ac-47dc-8e66-dcf8bbc623bb tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Returning 404 to user: Instance 08a33911-6e47-45fd-b499-155f199aacf8 could not be found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:35:06.427217 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-ae035bbc-f3ac-47dc-8e66-dcf8bbc623bb tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] 10.222.0.16 "DELETE /compute/v2.1/servers/08a33911-6e47-45fd-b499-155f199aacf8" status: 404 len: 111 microversion: 2.20 time: 0.121264 Jul 16 21:35:06.427217 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2505/4994] 10.222.0.16 () {60 vars in 1279 bytes} [Sat Jul 16 21:35:06 2022] DELETE /compute/v2.1/servers/08a33911-6e47-45fd-b499-155f199aacf8 => generated 111 bytes in 123 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Jul 16 21:35:06.441693 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-1b96ece9-950b-43bc-bb40-f076ba41e8b0 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:35:06.442323 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-1b96ece9-950b-43bc-bb40-f076ba41e8b0 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] [instance: 08a33911-6e47-45fd-b499-155f199aacf8] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:35:06.471111 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1b96ece9-950b-43bc-bb40-f076ba41e8b0 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.011s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:06.471655 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1b96ece9-950b-43bc-bb40-f076ba41e8b0 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:06.473165 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1b96ece9-950b-43bc-bb40-f076ba41e8b0 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:06.482502 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1b96ece9-950b-43bc-bb40-f076ba41e8b0 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:06.492576 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-1b96ece9-950b-43bc-bb40-f076ba41e8b0 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] HTTP exception thrown: Instance 08a33911-6e47-45fd-b499-155f199aacf8 could not be found. Jul 16 21:35:06.494971 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-1b96ece9-950b-43bc-bb40-f076ba41e8b0 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Request method failure captured: Jul 16 21:35:06.494971 ncdv-master-1 devstack@n-api.service[138680]: request: GET /compute/v2.1/servers/08a33911-6e47-45fd-b499-155f199aacf8 HTTP/1.1 Jul 16 21:35:06.494971 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:35:06.494971 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:35:06.494971 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:35:06.494971 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 0 Jul 16 21:35:06.494971 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:35:06.494971 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:35:06.494971 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:35:06.494971 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:35:06.494971 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0y1ucaZ1yKJJ3ZclTlid5u9GxaiMbQlzksI7M27rJ0B6rB8jj8I68KlCkH-lQUpdPSiKnnTfAiP8PzCbWLft6KLltxUTtp9U114w8i4pdCqez2cdl7YB-ynlGpqTgM-nH_ZWHIpWAB7cPaNN1ImROhBb6Fvg2kKIxL9C97eioGmF_M4 Jul 16 21:35:06.494971 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:35:06.494971 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:35:06.494971 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:35:06.494971 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:35:06.494971 ncdv-master-1 devstack@n-api.service[138680]: X-Openstack-Nova-Api-Version: 2.20 Jul 16 21:35:06.494971 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:35:06.494971 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:35:06.494971 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: e451ee7bfd2d4ab58d30c33fef334b55 Jul 16 21:35:06.494971 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-2044012735 Jul 16 21:35:06.494971 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:35:06.494971 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:35:06.494971 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/e451ee7bfd2d4ab58d30c33fef334b55"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/e451ee7bfd2d4ab58d30c33fef334b55"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_e451ee7bfd2d4ab58d30c33fef334b55"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/e451ee7bfd2d4ab58d30c33fef334b55"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:35:06.494971 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-AttachVolumeShelveTestJSON-2044012735 Jul 16 21:35:06.494971 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: e451ee7bfd2d4ab58d30c33fef334b55 Jul 16 21:35:06.494971 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-2044012735 Jul 16 21:35:06.494971 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-AttachVolumeShelveTestJSON-2044012735-project Jul 16 21:35:06.494971 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:35:06.494971 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:35:06.494971 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: ca2c9e7e6d61414cbf2c7e17753bc0df Jul 16 21:35:06.494971 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-AttachVolumeShelveTestJSON-2044012735-project Jul 16 21:35:06.494971 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 21:35:06.494971 ncdv-master-1 devstack@n-api.service[138680]: exception: Instance 08a33911-6e47-45fd-b499-155f199aacf8 could not be found. Jul 16 21:35:06.494971 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': '08a33911-6e47-45fd-b499-155f199aacf8'} Jul 16 21:35:06.494971 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:35:06.494971 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:35:06.494971 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:35:06.494971 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:35:06.494971 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:35:06.494971 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:35:06.494971 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:35:06.494971 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:35:06.494971 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:35:06.494971 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:35:06.494971 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 08a33911-6e47-45fd-b499-155f199aacf8 could not be found. Jul 16 21:35:06.494971 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:35:06.494971 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:35:06.494971 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:35:06.494971 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:35:06.494971 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:35:06.494971 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:35:06.494971 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:35:06.494971 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:35:06.494971 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:35:06.494971 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:35:06.494971 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:35:06.494971 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:35:06.494971 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:35:06.494971 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:35:06.494971 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:35:06.494971 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:35:06.494971 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 08a33911-6e47-45fd-b499-155f199aacf8 could not be found. Jul 16 21:35:06.494971 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:35:06.494971 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:35:06.500631 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:35:06.500631 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:35:06.500631 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:35:06.500631 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:35:06.500631 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:35:06.500631 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:35:06.500631 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 08a33911-6e47-45fd-b499-155f199aacf8 could not be found. Jul 16 21:35:06.500631 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:35:06.502447 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-1b96ece9-950b-43bc-bb40-f076ba41e8b0 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] Returning 404 to user: Instance 08a33911-6e47-45fd-b499-155f199aacf8 could not be found. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:35:06.504121 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-1b96ece9-950b-43bc-bb40-f076ba41e8b0 tempest-AttachVolumeShelveTestJSON-2044012735 tempest-AttachVolumeShelveTestJSON-2044012735-project] 10.222.0.16 "GET /compute/v2.1/servers/08a33911-6e47-45fd-b499-155f199aacf8" status: 404 len: 111 microversion: 2.20 time: 0.073510 Jul 16 21:35:06.508907 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2490/4995] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:35:06 2022] GET /compute/v2.1/servers/08a33911-6e47-45fd-b499-155f199aacf8 => generated 111 bytes in 78 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Jul 16 21:35:07.441454 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-5c3d3f91-df76-426e-9d49-5345ada19085 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:35:07.444901 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-5c3d3f91-df76-426e-9d49-5345ada19085 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: e4c7a9cb-e55d-4819-9885-2e72deffddb1] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:35:07.475680 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5c3d3f91-df76-426e-9d49-5345ada19085 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:07.476607 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5c3d3f91-df76-426e-9d49-5345ada19085 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:07.478375 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5c3d3f91-df76-426e-9d49-5345ada19085 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:07.479054 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5c3d3f91-df76-426e-9d49-5345ada19085 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:08.339719 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-5c3d3f91-df76-426e-9d49-5345ada19085 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '22e264440fff43cdb7be6702839af37f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da2248fa190342f0bfe9211cae1e403b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:35:08.339719 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-5c3d3f91-df76-426e-9d49-5345ada19085 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1" status: 200 len: 1526 microversion: 2.1 time: 0.852636 Jul 16 21:35:08.339719 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2506/4996] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:35:07 2022] GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1 => generated 1526 bytes in 868 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:35:09.312934 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-260fc329-811d-4c40-93b0-67ed55033034 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:35:09.312934 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-260fc329-811d-4c40-93b0-67ed55033034 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: e4c7a9cb-e55d-4819-9885-2e72deffddb1] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:35:09.319693 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-260fc329-811d-4c40-93b0-67ed55033034 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:09.320511 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-260fc329-811d-4c40-93b0-67ed55033034 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:09.321617 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-260fc329-811d-4c40-93b0-67ed55033034 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:09.322304 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-260fc329-811d-4c40-93b0-67ed55033034 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:10.224836 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-260fc329-811d-4c40-93b0-67ed55033034 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '22e264440fff43cdb7be6702839af37f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da2248fa190342f0bfe9211cae1e403b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:35:10.236374 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-260fc329-811d-4c40-93b0-67ed55033034 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1" status: 200 len: 1526 microversion: 2.1 time: 0.930660 Jul 16 21:35:10.240820 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2491/4997] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:35:09 2022] GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1 => generated 1526 bytes in 936 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:35:11.079625 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-3f8186c1-4e47-40a9-b381-bcf8677fdba9 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-device-tagging-server-765676754", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452", "config_drive": true, "networks": [{"uuid": "410da396-c124-4520-b157-45823a699682"}, {"port": "3e3ac495-fa5d-4999-b113-9a54b000c446", "tag": "port-1"}, {"port": "78c19415-67c3-482e-9663-0cdac41890d3", "tag": "port-2"}, {"uuid": "16f0a9a5-1f5e-47de-8eea-b7a298c4f0a8", "tag": "net-1"}, {"uuid": "16f0a9a5-1f5e-47de-8eea-b7a298c4f0a8"}, {"uuid": "b9a628ba-df74-44fd-be0a-bf092ec660dd", "fixed_ip": "10.2.2.100", "tag": "net-2-100"}, {"uuid": "b9a628ba-df74-44fd-be0a-bf092ec660dd", "fixed_ip": "10.2.2.200", "tag": "net-2-200"}], "block_device_mapping_v2": [{"uuid": "048fbb68-c884-46b4-aa78-58477a185196", "source_type": "volume", "destination_type": "volume", "boot_index": 0, "tag": "boot"}, {"uuid": "c4869fe9-11ee-40cf-8401-386b1a3bec57", "source_type": "volume", "destination_type": "volume", "boot_index": 1, "tag": "other"}, {"uuid": "35056502-3c4b-4440-9da4-2bbf9347f08f", "source_type": "volume", "destination_type": "volume", "boot_index": 2}], "security_groups": [{"name": "tempest-securitygroup--942956659"}], "key_name": "tempest-keypair-1683927826", "user_data": "IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIHVidW50dSB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH51YnVudHUvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo="}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:35:11.262109 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-9bd99c0b-cf77-460c-9711-0cad243f351f tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:35:11.263436 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-9bd99c0b-cf77-460c-9711-0cad243f351f tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: e4c7a9cb-e55d-4819-9885-2e72deffddb1] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:35:11.273794 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9bd99c0b-cf77-460c-9711-0cad243f351f tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:11.278623 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9bd99c0b-cf77-460c-9711-0cad243f351f tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:11.281629 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9bd99c0b-cf77-460c-9711-0cad243f351f tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:11.282279 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9bd99c0b-cf77-460c-9711-0cad243f351f tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:11.386825 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.network.neutron [None req-3f8186c1-4e47-40a9-b381-bcf8677fdba9 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] validate_networks() for [('410da396-c124-4520-b157-45823a699682', None, None, None, None, None), (None, None, '3e3ac495-fa5d-4999-b113-9a54b000c446', None, None, None), (None, None, '78c19415-67c3-482e-9663-0cdac41890d3', None, None, None), ('16f0a9a5-1f5e-47de-8eea-b7a298c4f0a8', None, None, None, None, None), ('16f0a9a5-1f5e-47de-8eea-b7a298c4f0a8', None, None, None, None, None), ('b9a628ba-df74-44fd-be0a-bf092ec660dd', '10.2.2.100', None, None, None, None), ('b9a628ba-df74-44fd-be0a-bf092ec660dd', '10.2.2.200', None, None, None, None)] {{(pid=138681) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 16 21:35:11.878618 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-9bd99c0b-cf77-460c-9711-0cad243f351f tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '22e264440fff43cdb7be6702839af37f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da2248fa190342f0bfe9211cae1e403b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:35:11.890788 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-9bd99c0b-cf77-460c-9711-0cad243f351f tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1" status: 200 len: 1508 microversion: 2.1 time: 0.632769 Jul 16 21:35:11.892009 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2492/4998] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:35:11 2022] GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1 => generated 1508 bytes in 635 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:35:12.908130 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-db839c57-6e50-46ae-8c09-9f8448cee0bd tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:35:12.908717 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-db839c57-6e50-46ae-8c09-9f8448cee0bd tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: e4c7a9cb-e55d-4819-9885-2e72deffddb1] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:35:12.911639 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_api:SELECT=34 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:35:12.923898 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-db839c57-6e50-46ae-8c09-9f8448cee0bd tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:12.924484 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-db839c57-6e50-46ae-8c09-9f8448cee0bd tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:12.925737 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-db839c57-6e50-46ae-8c09-9f8448cee0bd tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:12.926992 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-db839c57-6e50-46ae-8c09-9f8448cee0bd tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:13.674800 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-db839c57-6e50-46ae-8c09-9f8448cee0bd tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '22e264440fff43cdb7be6702839af37f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da2248fa190342f0bfe9211cae1e403b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:35:13.683705 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-db839c57-6e50-46ae-8c09-9f8448cee0bd tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1" status: 200 len: 1508 microversion: 2.1 time: 0.779653 Jul 16 21:35:13.684539 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2493/4999] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:35:12 2022] GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1 => generated 1508 bytes in 781 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:35:14.711389 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-b0a43cb6-eca3-43b9-a9d7-b8072f953509 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:35:14.714267 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-b0a43cb6-eca3-43b9-a9d7-b8072f953509 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: e4c7a9cb-e55d-4819-9885-2e72deffddb1] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:35:14.722509 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b0a43cb6-eca3-43b9-a9d7-b8072f953509 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:14.724015 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b0a43cb6-eca3-43b9-a9d7-b8072f953509 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:14.724998 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b0a43cb6-eca3-43b9-a9d7-b8072f953509 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:14.725589 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b0a43cb6-eca3-43b9-a9d7-b8072f953509 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:14.930579 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell0:SELECT=4 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:35:15.008218 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-b0a43cb6-eca3-43b9-a9d7-b8072f953509 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '22e264440fff43cdb7be6702839af37f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da2248fa190342f0bfe9211cae1e403b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:35:15.019848 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-b0a43cb6-eca3-43b9-a9d7-b8072f953509 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1" status: 200 len: 1508 microversion: 2.1 time: 0.319901 Jul 16 21:35:15.024075 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2494/5000] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:35:14 2022] GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1 => generated 1508 bytes in 327 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:35:15.248152 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-3f8186c1-4e47-40a9-b381-bcf8677fdba9 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Flavor limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 16 21:35:15.248660 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-3f8186c1-4e47-40a9-b381-bcf8677fdba9 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Image limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 16 21:35:15.249225 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-3f8186c1-4e47-40a9-b381-bcf8677fdba9 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Flavor pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 16 21:35:15.249718 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-3f8186c1-4e47-40a9-b381-bcf8677fdba9 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Image pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 16 21:35:15.250303 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-3f8186c1-4e47-40a9-b381-bcf8677fdba9 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 16 21:35:15.260240 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3f8186c1-4e47-40a9-b381-bcf8677fdba9 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:15.260707 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3f8186c1-4e47-40a9-b381-bcf8677fdba9 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:15.261530 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3f8186c1-4e47-40a9-b381-bcf8677fdba9 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:15.261982 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3f8186c1-4e47-40a9-b381-bcf8677fdba9 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:15.279584 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-3f8186c1-4e47-40a9-b381-bcf8677fdba9 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Getting quotas for project 0fd0ce287cb44cd59045868ce688909a. Resources: {'ram', 'instances', 'cores'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:35:15.285581 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-3f8186c1-4e47-40a9-b381-bcf8677fdba9 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Getting quotas for user 726b46c54e9541fbb7825607bac3abe5 and project 0fd0ce287cb44cd59045868ce688909a. Resources: {'ram', 'instances', 'cores'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:35:15.447172 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-3f8186c1-4e47-40a9-b381-bcf8677fdba9 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Going to run 1 instances... {{(pid=138681) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 16 21:35:15.450251 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-3f8186c1-4e47-40a9-b381-bcf8677fdba9 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] [instance: 19df5c33-44d4-4de9-947b-4707b78af23a] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=False,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag='boot',updated_at=,uuid=,volume_id='048fbb68-c884-46b4-aa78-58477a185196',volume_size=None,volume_type=None), BlockDeviceMapping(attachment_id=,boot_index=1,connection_info=None,created_at=,delete_on_termination=False,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag='other',updated_at=,uuid=,volume_id='c4869fe9-11ee-40cf-8401-386b1a3bec57',volume_size=None,volume_type=None), BlockDeviceMapping(attachment_id=,boot_index=2,connection_info=None,created_at=,delete_on_termination=False,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag=None,updated_at=,uuid=,volume_id='35056502-3c4b-4440-9da4-2bbf9347f08f',volume_size=None,volume_type=None)] {{(pid=138681) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 16 21:35:15.988424 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-f3150336-f770-4df9-8205-3363bdfd5f20 req-691e4d9d-c01f-43d8-be8c-7f0b9462da03 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "e4c7a9cb-e55d-4819-9885-2e72deffddb1", "name": "network-vif-unplugged", "status": "completed", "tag": "0b459b05-0d03-4317-8a9b-bae73b0eaddd"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:35:15.999925 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-f3150336-f770-4df9-8205-3363bdfd5f20 req-691e4d9d-c01f-43d8-be8c-7f0b9462da03 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:16.000386 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-f3150336-f770-4df9-8205-3363bdfd5f20 req-691e4d9d-c01f-43d8-be8c-7f0b9462da03 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:16.014903 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-f3150336-f770-4df9-8205-3363bdfd5f20 req-691e4d9d-c01f-43d8-be8c-7f0b9462da03 service nova] Creating event network-vif-unplugged:0b459b05-0d03-4317-8a9b-bae73b0eaddd for instance e4c7a9cb-e55d-4819-9885-2e72deffddb1 on nch2-master-1 Jul 16 21:35:16.020700 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-f3150336-f770-4df9-8205-3363bdfd5f20 req-691e4d9d-c01f-43d8-be8c-7f0b9462da03 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.036136 Jul 16 21:35:16.021207 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2495/5001] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:35:15 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 16 21:35:16.040161 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-93b65208-d739-4ede-8d58-0534c8490937 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:35:16.041620 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-93b65208-d739-4ede-8d58-0534c8490937 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: e4c7a9cb-e55d-4819-9885-2e72deffddb1] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:35:16.049304 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-93b65208-d739-4ede-8d58-0534c8490937 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:16.050333 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-93b65208-d739-4ede-8d58-0534c8490937 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:16.051422 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-93b65208-d739-4ede-8d58-0534c8490937 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:16.051912 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-93b65208-d739-4ede-8d58-0534c8490937 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:16.274220 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-93b65208-d739-4ede-8d58-0534c8490937 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '22e264440fff43cdb7be6702839af37f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da2248fa190342f0bfe9211cae1e403b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:35:16.287319 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-93b65208-d739-4ede-8d58-0534c8490937 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1" status: 200 len: 1508 microversion: 2.1 time: 0.251235 Jul 16 21:35:16.298132 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2496/5002] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:35:16 2022] GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1 => generated 1508 bytes in 264 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:35:17.019467 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-3f8186c1-4e47-40a9-b381-bcf8677fdba9 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 405 microversion: 2.42 time: 5.943426 Jul 16 21:35:17.028187 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2507/5003] 10.222.0.16 () {62 vars in 1188 bytes} [Sat Jul 16 21:35:11 2022] POST /compute/v2.1/servers => generated 405 bytes in 5948 msecs (HTTP/1.1 202) 10 headers in 453 bytes (1 switches on core 0) Jul 16 21:35:17.034881 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-d931f2a5-7f46-4f60-85a7-daec9a751180 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:35:17.036090 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-d931f2a5-7f46-4f60-85a7-daec9a751180 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] [instance: 19df5c33-44d4-4de9-947b-4707b78af23a] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:35:17.166276 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-d931f2a5-7f46-4f60-85a7-daec9a751180 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '726b46c54e9541fbb7825607bac3abe5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fd0ce287cb44cd59045868ce688909a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:35:17.177631 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-d931f2a5-7f46-4f60-85a7-daec9a751180 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '726b46c54e9541fbb7825607bac3abe5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fd0ce287cb44cd59045868ce688909a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:35:17.181127 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-d931f2a5-7f46-4f60-85a7-daec9a751180 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '726b46c54e9541fbb7825607bac3abe5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fd0ce287cb44cd59045868ce688909a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:35:17.188134 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-d931f2a5-7f46-4f60-85a7-daec9a751180 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] 10.222.0.16 "GET /compute/v2.1/servers/19df5c33-44d4-4de9-947b-4707b78af23a" status: 200 len: 1285 microversion: 2.42 time: 0.158412 Jul 16 21:35:17.196002 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2497/5004] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:35:17 2022] GET /compute/v2.1/servers/19df5c33-44d4-4de9-947b-4707b78af23a => generated 1285 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:35:17.308718 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-05d05800-e68c-4b4a-ad19-d702d40c0b86 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:35:17.309745 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-05d05800-e68c-4b4a-ad19-d702d40c0b86 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: e4c7a9cb-e55d-4819-9885-2e72deffddb1] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:35:17.323196 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-05d05800-e68c-4b4a-ad19-d702d40c0b86 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:17.324047 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-05d05800-e68c-4b4a-ad19-d702d40c0b86 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:17.325336 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-05d05800-e68c-4b4a-ad19-d702d40c0b86 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:17.326022 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-05d05800-e68c-4b4a-ad19-d702d40c0b86 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:17.564801 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-05d05800-e68c-4b4a-ad19-d702d40c0b86 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '22e264440fff43cdb7be6702839af37f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da2248fa190342f0bfe9211cae1e403b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:35:17.574393 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-05d05800-e68c-4b4a-ad19-d702d40c0b86 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1" status: 200 len: 1451 microversion: 2.1 time: 0.271114 Jul 16 21:35:17.575317 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2508/5005] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:35:17 2022] GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1 => generated 1451 bytes in 274 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:35:17.595758 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-93332168-b640-4b4c-90b8-656c4c6f242c tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:35:17.596382 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-93332168-b640-4b4c-90b8-656c4c6f242c tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: e4c7a9cb-e55d-4819-9885-2e72deffddb1] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:35:17.603739 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-93332168-b640-4b4c-90b8-656c4c6f242c tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:17.604207 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-93332168-b640-4b4c-90b8-656c4c6f242c tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:17.605252 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-93332168-b640-4b4c-90b8-656c4c6f242c tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:17.605704 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-93332168-b640-4b4c-90b8-656c4c6f242c tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:17.610308 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=100 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:35:17.891468 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-93332168-b640-4b4c-90b8-656c4c6f242c tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '22e264440fff43cdb7be6702839af37f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da2248fa190342f0bfe9211cae1e403b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:35:17.902847 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-93332168-b640-4b4c-90b8-656c4c6f242c tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1" status: 200 len: 1451 microversion: 2.1 time: 0.310791 Jul 16 21:35:17.904094 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2498/5006] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:35:17 2022] GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1 => generated 1451 bytes in 313 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:35:17.959888 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-3cae4b50-3ee6-4627-947d-d41f00f3cc5d tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Action: 'action', calling method: >, body: {"shelve": {}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:35:17.961553 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-3cae4b50-3ee6-4627-947d-d41f00f3cc5d tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: e4c7a9cb-e55d-4819-9885-2e72deffddb1] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:35:17.980117 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3cae4b50-3ee6-4627-947d-d41f00f3cc5d tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:17.980792 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3cae4b50-3ee6-4627-947d-d41f00f3cc5d tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:17.981790 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3cae4b50-3ee6-4627-947d-d41f00f3cc5d tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:17.982341 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3cae4b50-3ee6-4627-947d-d41f00f3cc5d tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:18.011647 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.objects.instance [None req-3cae4b50-3ee6-4627-947d-d41f00f3cc5d tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lazy-loading 'flavor' on Instance uuid e4c7a9cb-e55d-4819-9885-2e72deffddb1 {{(pid=138681) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:35:18.027426 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-3cae4b50-3ee6-4627-947d-d41f00f3cc5d tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] HTTP exception thrown: Cannot 'shelve' instance e4c7a9cb-e55d-4819-9885-2e72deffddb1 while it is in vm_state shelved_offloaded Jul 16 21:35:18.055342 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-3cae4b50-3ee6-4627-947d-d41f00f3cc5d tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Request method failure captured: Jul 16 21:35:18.055342 ncdv-master-1 devstack@n-api.service[138681]: request: POST /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1/action HTTP/1.1 Jul 16 21:35:18.055342 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:35:18.055342 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:35:18.055342 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:35:18.055342 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 14 Jul 16 21:35:18.055342 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:35:18.055342 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:35:18.055342 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:35:18.055342 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:35:18.055342 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0y6QJDVdPNcLbUnwwgOJV-QkddLQ7yqnzBNVp2CrOe5UQqjQOER3spl3PswEW82ISSx8FhmuIrEwFSG69Ze1pTOmWBWZ8PURIgVHbN3yBPA3U2S2TKkQkFwyNuvvV03MCr9U0QGJIjXVOkiqFEAy8q87U_okZgCdI-BFjBFgrCdfAEo Jul 16 21:35:18.055342 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:35:18.055342 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:35:18.055342 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:35:18.055342 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:35:18.055342 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:35:18.055342 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:35:18.055342 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: da2248fa190342f0bfe9211cae1e403b Jul 16 21:35:18.055342 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-ServersNegativeTestJSON-1102403437 Jul 16 21:35:18.055342 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:35:18.055342 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:35:18.055342 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/da2248fa190342f0bfe9211cae1e403b"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/da2248fa190342f0bfe9211cae1e403b"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_da2248fa190342f0bfe9211cae1e403b"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/da2248fa190342f0bfe9211cae1e403b"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:35:18.055342 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-ServersNegativeTestJSON-1102403437 Jul 16 21:35:18.055342 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: da2248fa190342f0bfe9211cae1e403b Jul 16 21:35:18.055342 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-ServersNegativeTestJSON-1102403437 Jul 16 21:35:18.055342 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-ServersNegativeTestJSON-1102403437-project Jul 16 21:35:18.055342 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:35:18.055342 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:35:18.055342 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: 22e264440fff43cdb7be6702839af37f Jul 16 21:35:18.055342 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-ServersNegativeTestJSON-1102403437-project Jul 16 21:35:18.055342 ncdv-master-1 devstack@n-api.service[138681]: Jul 16 21:35:18.055342 ncdv-master-1 devstack@n-api.service[138681]: {"shelve": {}} Jul 16 21:35:18.055342 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:35:18.055342 ncdv-master-1 devstack@n-api.service[138681]: exception: Cannot 'shelve' instance e4c7a9cb-e55d-4819-9885-2e72deffddb1 while it is in vm_state shelved_offloaded Jul 16 21:35:18.055342 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': 'e4c7a9cb-e55d-4819-9885-2e72deffddb1', 'body': {'shelve': {}}} Jul 16 21:35:18.055342 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:35:18.055342 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:35:18.055342 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/shelve.py", line 49, in _shelve Jul 16 21:35:18.055342 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi self.compute_api.shelve(context, instance) Jul 16 21:35:18.055342 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 360, in wrapper Jul 16 21:35:18.055342 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return func(self, context, instance, *args, **kwargs) Jul 16 21:35:18.055342 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 296, in inner Jul 16 21:35:18.055342 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Jul 16 21:35:18.055342 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 275, in inner Jul 16 21:35:18.055342 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Jul 16 21:35:18.055342 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 343, in wrapper Jul 16 21:35:18.055342 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return func(self, context, instance, *args, **kwargs) Jul 16 21:35:18.055342 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner Jul 16 21:35:18.055342 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Jul 16 21:35:18.055342 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 145, in inner Jul 16 21:35:18.055342 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.InstanceInvalidState( Jul 16 21:35:18.055342 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.InstanceInvalidState: Instance e4c7a9cb-e55d-4819-9885-2e72deffddb1 in vm_state shelved_offloaded. Cannot shelve while the instance is in this state. Jul 16 21:35:18.055342 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:35:18.055342 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:35:18.055342 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:35:18.055342 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:35:18.055342 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:35:18.055342 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:35:18.055342 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:35:18.055342 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:35:18.055342 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:35:18.055342 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:35:18.055342 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/shelve.py", line 56, in _shelve Jul 16 21:35:18.055342 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi common.raise_http_conflict_for_instance_invalid_state(state_error, Jul 16 21:35:18.058340 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 347, in raise_http_conflict_for_instance_invalid_state Jul 16 21:35:18.058340 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPConflict(explanation=msg) Jul 16 21:35:18.058340 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPConflict: Cannot 'shelve' instance e4c7a9cb-e55d-4819-9885-2e72deffddb1 while it is in vm_state shelved_offloaded Jul 16 21:35:18.058340 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:35:18.058340 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:35:18.058340 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:35:18.058340 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:35:18.058340 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:35:18.058340 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:35:18.058340 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:35:18.058340 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:35:18.058340 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Cannot 'shelve' instance e4c7a9cb-e55d-4819-9885-2e72deffddb1 while it is in vm_state shelved_offloaded Jul 16 21:35:18.058340 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:35:18.058340 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-3cae4b50-3ee6-4627-947d-d41f00f3cc5d tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Returning 409 to user: Cannot 'shelve' instance e4c7a9cb-e55d-4819-9885-2e72deffddb1 while it is in vm_state shelved_offloaded {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:35:18.058340 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-3cae4b50-3ee6-4627-947d-d41f00f3cc5d tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "POST /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1/action" status: 409 len: 155 microversion: 2.1 time: 0.092781 Jul 16 21:35:18.058340 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2509/5007] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 21:35:17 2022] POST /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1/action => generated 155 bytes in 95 msecs (HTTP/1.1 409) 9 headers in 378 bytes (1 switches on core 0) Jul 16 21:35:18.069005 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-bdfa2779-b508-4d6f-80c1-32eefbf56aa3 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Action: 'action', calling method: >, body: {"unshelve": null} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:35:18.070915 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-bdfa2779-b508-4d6f-80c1-32eefbf56aa3 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: e4c7a9cb-e55d-4819-9885-2e72deffddb1] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:35:18.079600 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-bdfa2779-b508-4d6f-80c1-32eefbf56aa3 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:18.080398 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-bdfa2779-b508-4d6f-80c1-32eefbf56aa3 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:18.081614 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-bdfa2779-b508-4d6f-80c1-32eefbf56aa3 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:18.082371 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-bdfa2779-b508-4d6f-80c1-32eefbf56aa3 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:18.209975 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-db2f2c7b-12f5-4044-bf6a-8fe94e6223a1 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:35:18.210910 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-db2f2c7b-12f5-4044-bf6a-8fe94e6223a1 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] [instance: 19df5c33-44d4-4de9-947b-4707b78af23a] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:35:18.306363 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-db2f2c7b-12f5-4044-bf6a-8fe94e6223a1 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '726b46c54e9541fbb7825607bac3abe5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fd0ce287cb44cd59045868ce688909a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:35:18.316126 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-db2f2c7b-12f5-4044-bf6a-8fe94e6223a1 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '726b46c54e9541fbb7825607bac3abe5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fd0ce287cb44cd59045868ce688909a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:35:18.318238 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-db2f2c7b-12f5-4044-bf6a-8fe94e6223a1 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '726b46c54e9541fbb7825607bac3abe5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fd0ce287cb44cd59045868ce688909a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:35:18.319451 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-db2f2c7b-12f5-4044-bf6a-8fe94e6223a1 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] 10.222.0.16 "GET /compute/v2.1/servers/19df5c33-44d4-4de9-947b-4707b78af23a" status: 200 len: 1285 microversion: 2.42 time: 0.114548 Jul 16 21:35:18.320020 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2510/5008] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:35:18 2022] GET /compute/v2.1/servers/19df5c33-44d4-4de9-947b-4707b78af23a => generated 1285 bytes in 116 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 16 21:35:18.460685 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-bdfa2779-b508-4d6f-80c1-32eefbf56aa3 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "POST /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1/action" status: 202 len: 0 microversion: 2.1 time: 0.406577 Jul 16 21:35:18.461892 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2499/5009] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 21:35:18 2022] POST /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1/action => generated 0 bytes in 408 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 21:35:18.471145 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-9d8f1ea8-8912-4ea8-8f6a-852f8d7bed0c tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:35:18.472113 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-9d8f1ea8-8912-4ea8-8f6a-852f8d7bed0c tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: e4c7a9cb-e55d-4819-9885-2e72deffddb1] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:35:18.481280 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9d8f1ea8-8912-4ea8-8f6a-852f8d7bed0c tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:18.481924 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9d8f1ea8-8912-4ea8-8f6a-852f8d7bed0c tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:18.482893 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9d8f1ea8-8912-4ea8-8f6a-852f8d7bed0c tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:18.483398 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9d8f1ea8-8912-4ea8-8f6a-852f8d7bed0c tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:18.652901 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-9d8f1ea8-8912-4ea8-8f6a-852f8d7bed0c tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '22e264440fff43cdb7be6702839af37f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da2248fa190342f0bfe9211cae1e403b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:35:18.666156 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-9d8f1ea8-8912-4ea8-8f6a-852f8d7bed0c tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1" status: 200 len: 1459 microversion: 2.1 time: 0.200092 Jul 16 21:35:18.666746 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2511/5010] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:35:18 2022] GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1 => generated 1459 bytes in 201 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:35:19.345696 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-62736753-17c7-4f1c-9245-375c866bd1df tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:35:19.346347 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-62736753-17c7-4f1c-9245-375c866bd1df tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] [instance: 19df5c33-44d4-4de9-947b-4707b78af23a] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:35:19.449251 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-62736753-17c7-4f1c-9245-375c866bd1df tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '726b46c54e9541fbb7825607bac3abe5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fd0ce287cb44cd59045868ce688909a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:35:19.459067 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-62736753-17c7-4f1c-9245-375c866bd1df tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '726b46c54e9541fbb7825607bac3abe5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fd0ce287cb44cd59045868ce688909a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:35:19.460735 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-62736753-17c7-4f1c-9245-375c866bd1df tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '726b46c54e9541fbb7825607bac3abe5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fd0ce287cb44cd59045868ce688909a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:35:19.466294 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-62736753-17c7-4f1c-9245-375c866bd1df tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] 10.222.0.16 "GET /compute/v2.1/servers/19df5c33-44d4-4de9-947b-4707b78af23a" status: 200 len: 1285 microversion: 2.42 time: 0.128383 Jul 16 21:35:19.466294 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2500/5011] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:35:19 2022] GET /compute/v2.1/servers/19df5c33-44d4-4de9-947b-4707b78af23a => generated 1285 bytes in 130 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:35:19.685852 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-8c8ed72f-d38a-456d-9b3a-8e0da34003f6 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:35:19.686752 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-8c8ed72f-d38a-456d-9b3a-8e0da34003f6 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: e4c7a9cb-e55d-4819-9885-2e72deffddb1] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:35:19.709008 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8c8ed72f-d38a-456d-9b3a-8e0da34003f6 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:19.709475 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8c8ed72f-d38a-456d-9b3a-8e0da34003f6 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:19.710299 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8c8ed72f-d38a-456d-9b3a-8e0da34003f6 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:19.715647 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8c8ed72f-d38a-456d-9b3a-8e0da34003f6 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:19.929830 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-8c8ed72f-d38a-456d-9b3a-8e0da34003f6 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '22e264440fff43cdb7be6702839af37f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da2248fa190342f0bfe9211cae1e403b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:35:19.941040 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-8c8ed72f-d38a-456d-9b3a-8e0da34003f6 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1" status: 200 len: 1459 microversion: 2.1 time: 0.260246 Jul 16 21:35:19.947398 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2512/5012] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:35:19 2022] GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1 => generated 1459 bytes in 268 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:35:20.483800 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-6a921ae9-3077-4112-ace8-1f4e28cd8a0a tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:35:20.484990 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-6a921ae9-3077-4112-ace8-1f4e28cd8a0a tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] [instance: 19df5c33-44d4-4de9-947b-4707b78af23a] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:35:20.496586 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6a921ae9-3077-4112-ace8-1f4e28cd8a0a tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:20.497327 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6a921ae9-3077-4112-ace8-1f4e28cd8a0a tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:20.498513 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6a921ae9-3077-4112-ace8-1f4e28cd8a0a tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:20.499440 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6a921ae9-3077-4112-ace8-1f4e28cd8a0a tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:20.591934 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-6a921ae9-3077-4112-ace8-1f4e28cd8a0a tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '726b46c54e9541fbb7825607bac3abe5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fd0ce287cb44cd59045868ce688909a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:35:20.604986 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-6a921ae9-3077-4112-ace8-1f4e28cd8a0a tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '726b46c54e9541fbb7825607bac3abe5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fd0ce287cb44cd59045868ce688909a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:35:20.606731 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-6a921ae9-3077-4112-ace8-1f4e28cd8a0a tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '726b46c54e9541fbb7825607bac3abe5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fd0ce287cb44cd59045868ce688909a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:35:20.608008 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-6a921ae9-3077-4112-ace8-1f4e28cd8a0a tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] 10.222.0.16 "GET /compute/v2.1/servers/19df5c33-44d4-4de9-947b-4707b78af23a" status: 200 len: 1575 microversion: 2.42 time: 0.130106 Jul 16 21:35:20.608726 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2501/5013] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:35:20 2022] GET /compute/v2.1/servers/19df5c33-44d4-4de9-947b-4707b78af23a => generated 1575 bytes in 131 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:35:20.961162 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-8bdf74be-99a8-42ae-8d7e-9f71b8f434bc tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:35:20.961812 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-8bdf74be-99a8-42ae-8d7e-9f71b8f434bc tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: e4c7a9cb-e55d-4819-9885-2e72deffddb1] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:35:20.973682 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8bdf74be-99a8-42ae-8d7e-9f71b8f434bc tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:20.974826 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8bdf74be-99a8-42ae-8d7e-9f71b8f434bc tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:20.975758 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8bdf74be-99a8-42ae-8d7e-9f71b8f434bc tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:20.976230 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8bdf74be-99a8-42ae-8d7e-9f71b8f434bc tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:21.169138 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-8bdf74be-99a8-42ae-8d7e-9f71b8f434bc tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '22e264440fff43cdb7be6702839af37f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da2248fa190342f0bfe9211cae1e403b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:35:21.192977 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-8bdf74be-99a8-42ae-8d7e-9f71b8f434bc tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1" status: 200 len: 1517 microversion: 2.1 time: 0.238198 Jul 16 21:35:21.194057 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2513/5014] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:35:20 2022] GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1 => generated 1517 bytes in 240 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:35:21.632815 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-9a7ad2e0-6d85-46dc-89b0-379db9958e34 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:35:21.633999 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-9a7ad2e0-6d85-46dc-89b0-379db9958e34 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] [instance: 19df5c33-44d4-4de9-947b-4707b78af23a] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:35:21.642331 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9a7ad2e0-6d85-46dc-89b0-379db9958e34 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:21.642942 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9a7ad2e0-6d85-46dc-89b0-379db9958e34 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:21.644854 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9a7ad2e0-6d85-46dc-89b0-379db9958e34 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:21.645541 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9a7ad2e0-6d85-46dc-89b0-379db9958e34 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:22.215397 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-1cc971e3-126c-46c5-b99d-97441cc2d7a1 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:35:22.216019 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-1cc971e3-126c-46c5-b99d-97441cc2d7a1 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: e4c7a9cb-e55d-4819-9885-2e72deffddb1] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:35:22.219792 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_api:SELECT=65,nova_api:DELETE=1,nova_api:INSERT=4 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:35:22.235037 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1cc971e3-126c-46c5-b99d-97441cc2d7a1 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:22.235708 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1cc971e3-126c-46c5-b99d-97441cc2d7a1 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:22.237528 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1cc971e3-126c-46c5-b99d-97441cc2d7a1 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:22.239764 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-9a7ad2e0-6d85-46dc-89b0-379db9958e34 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '726b46c54e9541fbb7825607bac3abe5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fd0ce287cb44cd59045868ce688909a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:35:22.240332 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1cc971e3-126c-46c5-b99d-97441cc2d7a1 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:22.263065 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-9a7ad2e0-6d85-46dc-89b0-379db9958e34 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '726b46c54e9541fbb7825607bac3abe5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fd0ce287cb44cd59045868ce688909a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:35:22.267433 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-9a7ad2e0-6d85-46dc-89b0-379db9958e34 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '726b46c54e9541fbb7825607bac3abe5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fd0ce287cb44cd59045868ce688909a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:35:22.270799 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-9a7ad2e0-6d85-46dc-89b0-379db9958e34 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] 10.222.0.16 "GET /compute/v2.1/servers/19df5c33-44d4-4de9-947b-4707b78af23a" status: 200 len: 1583 microversion: 2.42 time: 0.645356 Jul 16 21:35:22.279269 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2502/5015] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:35:21 2022] GET /compute/v2.1/servers/19df5c33-44d4-4de9-947b-4707b78af23a => generated 1583 bytes in 649 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:35:22.727958 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-1cc971e3-126c-46c5-b99d-97441cc2d7a1 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '22e264440fff43cdb7be6702839af37f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da2248fa190342f0bfe9211cae1e403b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:35:22.738918 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-1cc971e3-126c-46c5-b99d-97441cc2d7a1 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1" status: 200 len: 1517 microversion: 2.1 time: 0.527128 Jul 16 21:35:22.740196 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2514/5016] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:35:22 2022] GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1 => generated 1517 bytes in 529 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:35:23.291725 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-38bc73fe-e7df-4564-a490-7a0b14074f47 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:35:23.295483 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-38bc73fe-e7df-4564-a490-7a0b14074f47 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] [instance: 19df5c33-44d4-4de9-947b-4707b78af23a] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:35:23.303763 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-38bc73fe-e7df-4564-a490-7a0b14074f47 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:23.304476 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-38bc73fe-e7df-4564-a490-7a0b14074f47 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:23.305327 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-38bc73fe-e7df-4564-a490-7a0b14074f47 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:23.305972 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-38bc73fe-e7df-4564-a490-7a0b14074f47 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:23.430950 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-38bc73fe-e7df-4564-a490-7a0b14074f47 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '726b46c54e9541fbb7825607bac3abe5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fd0ce287cb44cd59045868ce688909a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:35:23.448599 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-38bc73fe-e7df-4564-a490-7a0b14074f47 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '726b46c54e9541fbb7825607bac3abe5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fd0ce287cb44cd59045868ce688909a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:35:23.448599 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-38bc73fe-e7df-4564-a490-7a0b14074f47 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '726b46c54e9541fbb7825607bac3abe5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fd0ce287cb44cd59045868ce688909a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:35:23.448599 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-38bc73fe-e7df-4564-a490-7a0b14074f47 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] 10.222.0.16 "GET /compute/v2.1/servers/19df5c33-44d4-4de9-947b-4707b78af23a" status: 200 len: 1593 microversion: 2.42 time: 0.158831 Jul 16 21:35:23.466700 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2503/5017] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:35:23 2022] GET /compute/v2.1/servers/19df5c33-44d4-4de9-947b-4707b78af23a => generated 1593 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 16 21:35:23.760955 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-9e2e7ac2-c2a4-4327-88f1-a727d148319d tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:35:23.763580 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-9e2e7ac2-c2a4-4327-88f1-a727d148319d tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: e4c7a9cb-e55d-4819-9885-2e72deffddb1] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:35:23.772584 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9e2e7ac2-c2a4-4327-88f1-a727d148319d tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:23.773397 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9e2e7ac2-c2a4-4327-88f1-a727d148319d tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:23.779402 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9e2e7ac2-c2a4-4327-88f1-a727d148319d tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:23.779402 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9e2e7ac2-c2a4-4327-88f1-a727d148319d tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:24.330233 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-9e2e7ac2-c2a4-4327-88f1-a727d148319d tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '22e264440fff43cdb7be6702839af37f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da2248fa190342f0bfe9211cae1e403b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:35:24.355562 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-9e2e7ac2-c2a4-4327-88f1-a727d148319d tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1" status: 200 len: 1517 microversion: 2.1 time: 0.599151 Jul 16 21:35:24.370855 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2515/5018] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:35:23 2022] GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1 => generated 1517 bytes in 604 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:35:24.469825 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-ba436961-490f-421d-99aa-def89865b732 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:35:24.470541 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-ba436961-490f-421d-99aa-def89865b732 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] [instance: 19df5c33-44d4-4de9-947b-4707b78af23a] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:35:24.506371 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ba436961-490f-421d-99aa-def89865b732 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:24.507191 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ba436961-490f-421d-99aa-def89865b732 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.011s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:24.509534 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ba436961-490f-421d-99aa-def89865b732 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:24.513406 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ba436961-490f-421d-99aa-def89865b732 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:25.214198 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-ba436961-490f-421d-99aa-def89865b732 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '726b46c54e9541fbb7825607bac3abe5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fd0ce287cb44cd59045868ce688909a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:35:25.227361 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-ba436961-490f-421d-99aa-def89865b732 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '726b46c54e9541fbb7825607bac3abe5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fd0ce287cb44cd59045868ce688909a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:35:25.229387 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-ba436961-490f-421d-99aa-def89865b732 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '726b46c54e9541fbb7825607bac3abe5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fd0ce287cb44cd59045868ce688909a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:35:25.235899 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-ba436961-490f-421d-99aa-def89865b732 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] 10.222.0.16 "GET /compute/v2.1/servers/19df5c33-44d4-4de9-947b-4707b78af23a" status: 200 len: 1660 microversion: 2.42 time: 0.767380 Jul 16 21:35:25.243599 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2504/5019] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:35:24 2022] GET /compute/v2.1/servers/19df5c33-44d4-4de9-947b-4707b78af23a => generated 1660 bytes in 779 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:35:25.392646 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-4ebf339f-3d25-4dd4-8a4b-0d097ba6cae4 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:35:25.394710 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-4ebf339f-3d25-4dd4-8a4b-0d097ba6cae4 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: e4c7a9cb-e55d-4819-9885-2e72deffddb1] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:35:25.404278 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell0:SELECT=2 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:35:25.406069 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4ebf339f-3d25-4dd4-8a4b-0d097ba6cae4 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:25.408137 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4ebf339f-3d25-4dd4-8a4b-0d097ba6cae4 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:25.411319 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4ebf339f-3d25-4dd4-8a4b-0d097ba6cae4 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:25.412028 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4ebf339f-3d25-4dd4-8a4b-0d097ba6cae4 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:25.967328 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-4ebf339f-3d25-4dd4-8a4b-0d097ba6cae4 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '22e264440fff43cdb7be6702839af37f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da2248fa190342f0bfe9211cae1e403b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:35:25.978968 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-4ebf339f-3d25-4dd4-8a4b-0d097ba6cae4 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1" status: 200 len: 1517 microversion: 2.1 time: 0.609483 Jul 16 21:35:25.980186 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2516/5020] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:35:25 2022] GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1 => generated 1517 bytes in 611 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:35:26.254582 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-2792d294-bffe-430d-832b-749a239ab6d1 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:35:26.255236 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-2792d294-bffe-430d-832b-749a239ab6d1 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] [instance: 19df5c33-44d4-4de9-947b-4707b78af23a] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:35:26.274710 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2792d294-bffe-430d-832b-749a239ab6d1 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:26.275203 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2792d294-bffe-430d-832b-749a239ab6d1 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:26.275996 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2792d294-bffe-430d-832b-749a239ab6d1 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:26.276437 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2792d294-bffe-430d-832b-749a239ab6d1 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:26.999950 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-1dec0b2d-7da0-4b54-bf43-e9e452d3566e tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:35:27.008768 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-1dec0b2d-7da0-4b54-bf43-e9e452d3566e tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: e4c7a9cb-e55d-4819-9885-2e72deffddb1] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:35:27.039292 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1dec0b2d-7da0-4b54-bf43-e9e452d3566e tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:27.039765 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1dec0b2d-7da0-4b54-bf43-e9e452d3566e tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:27.040568 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1dec0b2d-7da0-4b54-bf43-e9e452d3566e tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:27.041018 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1dec0b2d-7da0-4b54-bf43-e9e452d3566e tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:27.716840 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-2792d294-bffe-430d-832b-749a239ab6d1 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '726b46c54e9541fbb7825607bac3abe5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fd0ce287cb44cd59045868ce688909a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:35:27.739117 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-2792d294-bffe-430d-832b-749a239ab6d1 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '726b46c54e9541fbb7825607bac3abe5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fd0ce287cb44cd59045868ce688909a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:35:27.741903 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-2792d294-bffe-430d-832b-749a239ab6d1 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '726b46c54e9541fbb7825607bac3abe5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fd0ce287cb44cd59045868ce688909a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:35:27.746125 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-2792d294-bffe-430d-832b-749a239ab6d1 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] 10.222.0.16 "GET /compute/v2.1/servers/19df5c33-44d4-4de9-947b-4707b78af23a" status: 200 len: 1660 microversion: 2.42 time: 1.495430 Jul 16 21:35:27.746751 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2505/5021] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:35:26 2022] GET /compute/v2.1/servers/19df5c33-44d4-4de9-947b-4707b78af23a => generated 1660 bytes in 1497 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:35:27.944547 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-1dec0b2d-7da0-4b54-bf43-e9e452d3566e tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '22e264440fff43cdb7be6702839af37f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da2248fa190342f0bfe9211cae1e403b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:35:27.987782 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-1dec0b2d-7da0-4b54-bf43-e9e452d3566e tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1" status: 200 len: 1517 microversion: 2.1 time: 0.993338 Jul 16 21:35:27.991203 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2517/5022] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:35:26 2022] GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1 => generated 1517 bytes in 998 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:35:28.771341 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-2cf97051-3d8e-4c52-a286-71f82c25ea5a tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:35:28.775608 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-2cf97051-3d8e-4c52-a286-71f82c25ea5a tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] [instance: 19df5c33-44d4-4de9-947b-4707b78af23a] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:35:28.813083 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2cf97051-3d8e-4c52-a286-71f82c25ea5a tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:28.813561 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2cf97051-3d8e-4c52-a286-71f82c25ea5a tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:28.814369 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2cf97051-3d8e-4c52-a286-71f82c25ea5a tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:28.814823 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2cf97051-3d8e-4c52-a286-71f82c25ea5a tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:29.015469 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-14f26656-c201-41ab-8625-1e942a277b41 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:35:29.018242 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-14f26656-c201-41ab-8625-1e942a277b41 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: e4c7a9cb-e55d-4819-9885-2e72deffddb1] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:35:29.041471 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell0:SELECT=8 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:35:29.043307 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-14f26656-c201-41ab-8625-1e942a277b41 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:29.044065 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-14f26656-c201-41ab-8625-1e942a277b41 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:29.051846 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-14f26656-c201-41ab-8625-1e942a277b41 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:29.057864 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-14f26656-c201-41ab-8625-1e942a277b41 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:29.318183 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-2cf97051-3d8e-4c52-a286-71f82c25ea5a tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '726b46c54e9541fbb7825607bac3abe5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fd0ce287cb44cd59045868ce688909a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:35:29.356847 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-2cf97051-3d8e-4c52-a286-71f82c25ea5a tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '726b46c54e9541fbb7825607bac3abe5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fd0ce287cb44cd59045868ce688909a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:35:29.363174 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-2cf97051-3d8e-4c52-a286-71f82c25ea5a tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '726b46c54e9541fbb7825607bac3abe5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fd0ce287cb44cd59045868ce688909a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:35:29.366043 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-2cf97051-3d8e-4c52-a286-71f82c25ea5a tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] 10.222.0.16 "GET /compute/v2.1/servers/19df5c33-44d4-4de9-947b-4707b78af23a" status: 200 len: 1706 microversion: 2.42 time: 0.602870 Jul 16 21:35:29.368074 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2506/5023] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:35:28 2022] GET /compute/v2.1/servers/19df5c33-44d4-4de9-947b-4707b78af23a => generated 1706 bytes in 607 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:35:29.454261 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell0:SELECT=4 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:35:29.585104 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-14f26656-c201-41ab-8625-1e942a277b41 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '22e264440fff43cdb7be6702839af37f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da2248fa190342f0bfe9211cae1e403b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:35:29.590601 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-4d84ca2d-04e4-487d-a7f6-cf0d4b8724cb tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Action: 'create', calling method: .version_select at 0x7fdd8fcd4280>, body: {"keypair": {"name": "tempest-keypair-1708417433", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBPtiJ2GSHL/FQ1xymHQrwDk64VRIc5NQxN4wJJwyNVNALzkLp3SBg+eeQIM4VXObsX2ScdYlgkSf2xKDILIevfk0N/zP6ADaYoiuKhqxRMKN1td9BRwP++PwNtDP5ycMHQ=="}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:35:29.614393 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-14f26656-c201-41ab-8625-1e942a277b41 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1" status: 200 len: 1517 microversion: 2.1 time: 0.598932 Jul 16 21:35:29.619110 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2518/5024] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:35:28 2022] GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1 => generated 1517 bytes in 619 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:35:29.629800 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-4d84ca2d-04e4-487d-a7f6-cf0d4b8724cb tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Getting quotas for project e60c0863dc6f4313bdc873fda27ae2fc. Resources: {'key_pairs'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:35:29.650120 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-4d84ca2d-04e4-487d-a7f6-cf0d4b8724cb tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Getting quotas for user 0ab8b3f06e994bbca5d6dd14109bfe50 and project e60c0863dc6f4313bdc873fda27ae2fc. Resources: {'key_pairs'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:35:29.783166 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-4d84ca2d-04e4-487d-a7f6-cf0d4b8724cb tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] 10.222.0.16 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.413640 Jul 16 21:35:29.785425 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2507/5025] 10.222.0.16 () {60 vars in 1158 bytes} [Sat Jul 16 21:35:29 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 417 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:35:30.385539 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-8f8c61e1-6ac1-4918-a19e-2e16aba59a5e tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:35:30.386771 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-8f8c61e1-6ac1-4918-a19e-2e16aba59a5e tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] [instance: 19df5c33-44d4-4de9-947b-4707b78af23a] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:35:30.398388 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8f8c61e1-6ac1-4918-a19e-2e16aba59a5e tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:30.404053 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8f8c61e1-6ac1-4918-a19e-2e16aba59a5e tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:30.405522 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8f8c61e1-6ac1-4918-a19e-2e16aba59a5e tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:30.406003 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8f8c61e1-6ac1-4918-a19e-2e16aba59a5e tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:30.635024 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-89b44c8e-acf4-4bdb-aae1-ed05bda195a8 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:35:30.636002 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-89b44c8e-acf4-4bdb-aae1-ed05bda195a8 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: e4c7a9cb-e55d-4819-9885-2e72deffddb1] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:35:30.656341 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-89b44c8e-acf4-4bdb-aae1-ed05bda195a8 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:30.656842 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-89b44c8e-acf4-4bdb-aae1-ed05bda195a8 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:30.657910 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-89b44c8e-acf4-4bdb-aae1-ed05bda195a8 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:30.669150 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-89b44c8e-acf4-4bdb-aae1-ed05bda195a8 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.011s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:31.277368 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-8f8c61e1-6ac1-4918-a19e-2e16aba59a5e tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '726b46c54e9541fbb7825607bac3abe5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fd0ce287cb44cd59045868ce688909a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:35:31.294599 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-8f8c61e1-6ac1-4918-a19e-2e16aba59a5e tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '726b46c54e9541fbb7825607bac3abe5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fd0ce287cb44cd59045868ce688909a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:35:31.301767 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-8f8c61e1-6ac1-4918-a19e-2e16aba59a5e tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '726b46c54e9541fbb7825607bac3abe5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fd0ce287cb44cd59045868ce688909a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:35:31.301767 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-8f8c61e1-6ac1-4918-a19e-2e16aba59a5e tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] 10.222.0.16 "GET /compute/v2.1/servers/19df5c33-44d4-4de9-947b-4707b78af23a" status: 200 len: 1752 microversion: 2.42 time: 0.917310 Jul 16 21:35:31.305119 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2519/5026] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:35:30 2022] GET /compute/v2.1/servers/19df5c33-44d4-4de9-947b-4707b78af23a => generated 1752 bytes in 924 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:35:31.332868 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-89b44c8e-acf4-4bdb-aae1-ed05bda195a8 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '22e264440fff43cdb7be6702839af37f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da2248fa190342f0bfe9211cae1e403b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:35:31.341852 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-89b44c8e-acf4-4bdb-aae1-ed05bda195a8 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1" status: 200 len: 1517 microversion: 2.1 time: 0.711770 Jul 16 21:35:31.342774 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2508/5027] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:35:30 2022] GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1 => generated 1517 bytes in 713 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:35:32.326510 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-a2ea3735-060f-4f03-acd8-cb303b452bf3 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:35:32.327163 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-a2ea3735-060f-4f03-acd8-cb303b452bf3 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] [instance: 19df5c33-44d4-4de9-947b-4707b78af23a] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:35:32.338515 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a2ea3735-060f-4f03-acd8-cb303b452bf3 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:32.341130 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a2ea3735-060f-4f03-acd8-cb303b452bf3 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:32.341130 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a2ea3735-060f-4f03-acd8-cb303b452bf3 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:32.354179 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a2ea3735-060f-4f03-acd8-cb303b452bf3 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:32.385396 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-cbd9a77d-9ea7-4729-a63d-f4d2876518b8 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:35:32.386856 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-cbd9a77d-9ea7-4729-a63d-f4d2876518b8 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: e4c7a9cb-e55d-4819-9885-2e72deffddb1] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:35:32.411860 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cbd9a77d-9ea7-4729-a63d-f4d2876518b8 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:32.417479 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cbd9a77d-9ea7-4729-a63d-f4d2876518b8 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:32.422220 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cbd9a77d-9ea7-4729-a63d-f4d2876518b8 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:32.423237 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cbd9a77d-9ea7-4729-a63d-f4d2876518b8 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:33.188745 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-a2ea3735-060f-4f03-acd8-cb303b452bf3 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '726b46c54e9541fbb7825607bac3abe5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fd0ce287cb44cd59045868ce688909a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:35:33.202793 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-a2ea3735-060f-4f03-acd8-cb303b452bf3 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '726b46c54e9541fbb7825607bac3abe5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fd0ce287cb44cd59045868ce688909a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:35:33.207658 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-a2ea3735-060f-4f03-acd8-cb303b452bf3 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '726b46c54e9541fbb7825607bac3abe5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fd0ce287cb44cd59045868ce688909a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:35:33.223663 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-a2ea3735-060f-4f03-acd8-cb303b452bf3 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] 10.222.0.16 "GET /compute/v2.1/servers/19df5c33-44d4-4de9-947b-4707b78af23a" status: 200 len: 1752 microversion: 2.42 time: 0.888711 Jul 16 21:35:33.223663 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2520/5028] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:35:32 2022] GET /compute/v2.1/servers/19df5c33-44d4-4de9-947b-4707b78af23a => generated 1752 bytes in 893 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:35:33.272841 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-cbd9a77d-9ea7-4729-a63d-f4d2876518b8 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '22e264440fff43cdb7be6702839af37f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da2248fa190342f0bfe9211cae1e403b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:35:33.287927 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-cbd9a77d-9ea7-4729-a63d-f4d2876518b8 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1" status: 200 len: 1517 microversion: 2.1 time: 0.906406 Jul 16 21:35:33.305018 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2509/5029] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:35:32 2022] GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1 => generated 1517 bytes in 927 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:35:34.240688 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-20181aa6-b67b-4926-9345-d4e2419eb0c8 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:35:34.252105 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-20181aa6-b67b-4926-9345-d4e2419eb0c8 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] [instance: 19df5c33-44d4-4de9-947b-4707b78af23a] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:35:34.283857 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-20181aa6-b67b-4926-9345-d4e2419eb0c8 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:34.285240 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-20181aa6-b67b-4926-9345-d4e2419eb0c8 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:34.290516 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-20181aa6-b67b-4926-9345-d4e2419eb0c8 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:34.291239 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-20181aa6-b67b-4926-9345-d4e2419eb0c8 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:34.314381 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-6ef03a34-f215-44ce-a3e6-2278f390a544 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:35:34.321344 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-6ef03a34-f215-44ce-a3e6-2278f390a544 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: e4c7a9cb-e55d-4819-9885-2e72deffddb1] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:35:34.351361 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6ef03a34-f215-44ce-a3e6-2278f390a544 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:34.351361 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6ef03a34-f215-44ce-a3e6-2278f390a544 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:34.353318 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6ef03a34-f215-44ce-a3e6-2278f390a544 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:34.357227 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6ef03a34-f215-44ce-a3e6-2278f390a544 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:35.153263 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-20181aa6-b67b-4926-9345-d4e2419eb0c8 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '726b46c54e9541fbb7825607bac3abe5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fd0ce287cb44cd59045868ce688909a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:35:35.186269 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-20181aa6-b67b-4926-9345-d4e2419eb0c8 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '726b46c54e9541fbb7825607bac3abe5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fd0ce287cb44cd59045868ce688909a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:35:35.192412 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-20181aa6-b67b-4926-9345-d4e2419eb0c8 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '726b46c54e9541fbb7825607bac3abe5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fd0ce287cb44cd59045868ce688909a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:35:35.192412 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-20181aa6-b67b-4926-9345-d4e2419eb0c8 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] 10.222.0.16 "GET /compute/v2.1/servers/19df5c33-44d4-4de9-947b-4707b78af23a" status: 200 len: 1786 microversion: 2.42 time: 0.957047 Jul 16 21:35:35.196082 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2521/5030] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:35:34 2022] GET /compute/v2.1/servers/19df5c33-44d4-4de9-947b-4707b78af23a => generated 1786 bytes in 966 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:35:35.240123 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-6ef03a34-f215-44ce-a3e6-2278f390a544 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '22e264440fff43cdb7be6702839af37f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da2248fa190342f0bfe9211cae1e403b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:35:35.255821 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-6ef03a34-f215-44ce-a3e6-2278f390a544 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1" status: 200 len: 1517 microversion: 2.1 time: 0.943639 Jul 16 21:35:35.255821 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2510/5031] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:35:34 2022] GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1 => generated 1517 bytes in 951 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:35:36.211459 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-9744a175-97d5-4cc5-9f05-b428c5eb7d0d tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:35:36.214477 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-9744a175-97d5-4cc5-9f05-b428c5eb7d0d tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] [instance: 19df5c33-44d4-4de9-947b-4707b78af23a] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:35:36.259942 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9744a175-97d5-4cc5-9f05-b428c5eb7d0d tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:36.263845 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9744a175-97d5-4cc5-9f05-b428c5eb7d0d tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:36.266916 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9744a175-97d5-4cc5-9f05-b428c5eb7d0d tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:36.266916 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9744a175-97d5-4cc5-9f05-b428c5eb7d0d tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:36.281271 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-42478bb6-6d33-467c-8dc7-6526b9713849 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:35:36.282087 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-42478bb6-6d33-467c-8dc7-6526b9713849 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: e4c7a9cb-e55d-4819-9885-2e72deffddb1] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:35:36.304557 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-42478bb6-6d33-467c-8dc7-6526b9713849 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:36.308120 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-42478bb6-6d33-467c-8dc7-6526b9713849 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:36.311296 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-42478bb6-6d33-467c-8dc7-6526b9713849 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:36.313902 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-42478bb6-6d33-467c-8dc7-6526b9713849 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:36.604014 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-9744a175-97d5-4cc5-9f05-b428c5eb7d0d tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '726b46c54e9541fbb7825607bac3abe5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fd0ce287cb44cd59045868ce688909a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:35:36.621184 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-9744a175-97d5-4cc5-9f05-b428c5eb7d0d tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '726b46c54e9541fbb7825607bac3abe5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fd0ce287cb44cd59045868ce688909a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:35:36.623515 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-9744a175-97d5-4cc5-9f05-b428c5eb7d0d tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '726b46c54e9541fbb7825607bac3abe5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fd0ce287cb44cd59045868ce688909a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:35:36.634763 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-9744a175-97d5-4cc5-9f05-b428c5eb7d0d tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] 10.222.0.16 "GET /compute/v2.1/servers/19df5c33-44d4-4de9-947b-4707b78af23a" status: 200 len: 1786 microversion: 2.42 time: 0.428704 Jul 16 21:35:36.635807 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2522/5032] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:35:36 2022] GET /compute/v2.1/servers/19df5c33-44d4-4de9-947b-4707b78af23a => generated 1786 bytes in 431 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:35:36.692961 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-42478bb6-6d33-467c-8dc7-6526b9713849 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '22e264440fff43cdb7be6702839af37f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da2248fa190342f0bfe9211cae1e403b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:35:36.705562 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-42478bb6-6d33-467c-8dc7-6526b9713849 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1" status: 200 len: 1517 microversion: 2.1 time: 0.437016 Jul 16 21:35:36.707643 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2511/5033] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:35:36 2022] GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1 => generated 1517 bytes in 440 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:35:37.043305 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-8cac2dbc-b990-4143-aa96-be4acb2d069c tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeNegativeTest-server-236782866", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "4775de54-1ba2-4d76-a6e0-a5f8a9ef879c"}], "security_groups": [{"name": "tempest-securitygroup--1430374493"}], "key_name": "tempest-keypair-1708417433", "user_data": "}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:35:37.301805 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.network.neutron [None req-8cac2dbc-b990-4143-aa96-be4acb2d069c tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] validate_networks() for [('4775de54-1ba2-4d76-a6e0-a5f8a9ef879c', None, None, None, None, None)] {{(pid=138681) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 16 21:35:37.653834 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-70823a24-8b4c-42e7-b114-b59734a6ee0a tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:35:37.655921 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-70823a24-8b4c-42e7-b114-b59734a6ee0a tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] [instance: 19df5c33-44d4-4de9-947b-4707b78af23a] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:35:37.664184 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-70823a24-8b4c-42e7-b114-b59734a6ee0a tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:37.664979 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-70823a24-8b4c-42e7-b114-b59734a6ee0a tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:37.679629 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-70823a24-8b4c-42e7-b114-b59734a6ee0a tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:37.684042 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-70823a24-8b4c-42e7-b114-b59734a6ee0a tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:38.014947 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-70823a24-8b4c-42e7-b114-b59734a6ee0a tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '726b46c54e9541fbb7825607bac3abe5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fd0ce287cb44cd59045868ce688909a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:35:38.026045 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-70823a24-8b4c-42e7-b114-b59734a6ee0a tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '726b46c54e9541fbb7825607bac3abe5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fd0ce287cb44cd59045868ce688909a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:35:38.028067 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-70823a24-8b4c-42e7-b114-b59734a6ee0a tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '726b46c54e9541fbb7825607bac3abe5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fd0ce287cb44cd59045868ce688909a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:35:38.029564 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-70823a24-8b4c-42e7-b114-b59734a6ee0a tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] 10.222.0.16 "GET /compute/v2.1/servers/19df5c33-44d4-4de9-947b-4707b78af23a" status: 200 len: 1832 microversion: 2.42 time: 0.380011 Jul 16 21:35:38.030299 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2512/5034] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:35:37 2022] GET /compute/v2.1/servers/19df5c33-44d4-4de9-947b-4707b78af23a => generated 1832 bytes in 381 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 16 21:35:38.040096 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-8198e7ab-42ee-4a7d-b65e-5a13e8da406f tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:35:38.041194 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-8198e7ab-42ee-4a7d-b65e-5a13e8da406f tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: e4c7a9cb-e55d-4819-9885-2e72deffddb1] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:35:38.049153 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8198e7ab-42ee-4a7d-b65e-5a13e8da406f tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:38.050131 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8198e7ab-42ee-4a7d-b65e-5a13e8da406f tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:38.051022 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8198e7ab-42ee-4a7d-b65e-5a13e8da406f tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:38.051473 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8198e7ab-42ee-4a7d-b65e-5a13e8da406f tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:38.386262 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-8198e7ab-42ee-4a7d-b65e-5a13e8da406f tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '22e264440fff43cdb7be6702839af37f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da2248fa190342f0bfe9211cae1e403b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:35:38.392698 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-8cac2dbc-b990-4143-aa96-be4acb2d069c tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Flavor limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 16 21:35:38.393188 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-8cac2dbc-b990-4143-aa96-be4acb2d069c tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Image limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 16 21:35:38.394193 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-8cac2dbc-b990-4143-aa96-be4acb2d069c tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Flavor pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 16 21:35:38.394619 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-8cac2dbc-b990-4143-aa96-be4acb2d069c tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Image pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 16 21:35:38.395049 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-8cac2dbc-b990-4143-aa96-be4acb2d069c tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 16 21:35:38.402520 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-8198e7ab-42ee-4a7d-b65e-5a13e8da406f tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1" status: 200 len: 1517 microversion: 2.1 time: 0.371006 Jul 16 21:35:38.403173 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2513/5035] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:35:38 2022] GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1 => generated 1517 bytes in 372 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:35:38.412837 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8cac2dbc-b990-4143-aa96-be4acb2d069c tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:38.413300 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8cac2dbc-b990-4143-aa96-be4acb2d069c tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:38.414274 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8cac2dbc-b990-4143-aa96-be4acb2d069c tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:38.414803 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8cac2dbc-b990-4143-aa96-be4acb2d069c tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:38.452382 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-8cac2dbc-b990-4143-aa96-be4acb2d069c tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Getting quotas for project e60c0863dc6f4313bdc873fda27ae2fc. Resources: {'ram', 'instances', 'cores'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:35:38.459683 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-8cac2dbc-b990-4143-aa96-be4acb2d069c tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Getting quotas for user 0ab8b3f06e994bbca5d6dd14109bfe50 and project e60c0863dc6f4313bdc873fda27ae2fc. Resources: {'ram', 'instances', 'cores'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:35:38.485205 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-8cac2dbc-b990-4143-aa96-be4acb2d069c tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Going to run 1 instances... {{(pid=138681) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 16 21:35:38.487826 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-8cac2dbc-b990-4143-aa96-be4acb2d069c tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] [instance: 4df09776-dc3c-46ff-9e64-e20b5d4679fa] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138681) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 16 21:35:38.657870 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-8cac2dbc-b990-4143-aa96-be4acb2d069c tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 402 microversion: 2.1 time: 1.618712 Jul 16 21:35:38.669404 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-7193b387-f71f-41f0-b8a3-073b0eb64ecf tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:35:38.670686 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-7193b387-f71f-41f0-b8a3-073b0eb64ecf tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] [instance: 4df09776-dc3c-46ff-9e64-e20b5d4679fa] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:35:38.671113 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2523/5036] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 21:35:37 2022] POST /compute/v2.1/servers => generated 402 bytes in 1626 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 16 21:35:38.757743 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-7193b387-f71f-41f0-b8a3-073b0eb64ecf tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0ab8b3f06e994bbca5d6dd14109bfe50', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e60c0863dc6f4313bdc873fda27ae2fc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:35:38.766259 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-7193b387-f71f-41f0-b8a3-073b0eb64ecf tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] 10.222.0.16 "GET /compute/v2.1/servers/4df09776-dc3c-46ff-9e64-e20b5d4679fa" status: 200 len: 1241 microversion: 2.1 time: 0.100841 Jul 16 21:35:38.767578 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2514/5037] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:35:38 2022] GET /compute/v2.1/servers/4df09776-dc3c-46ff-9e64-e20b5d4679fa => generated 1241 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:35:38.847652 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-eeaa6c41-1640-449e-8faa-0e9ae405a2c2 req-ce37227c-0ab2-4e70-8cdc-b55d7c6fcb51 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "19df5c33-44d4-4de9-947b-4707b78af23a", "tag": "7d150f5e-be69-4d03-aff9-6fc347935783"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:35:38.858789 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-eeaa6c41-1640-449e-8faa-0e9ae405a2c2 req-ce37227c-0ab2-4e70-8cdc-b55d7c6fcb51 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:38.859788 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-eeaa6c41-1640-449e-8faa-0e9ae405a2c2 req-ce37227c-0ab2-4e70-8cdc-b55d7c6fcb51 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:38.890091 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-eeaa6c41-1640-449e-8faa-0e9ae405a2c2 req-ce37227c-0ab2-4e70-8cdc-b55d7c6fcb51 service nova] Creating event network-changed:7d150f5e-be69-4d03-aff9-6fc347935783 for instance 19df5c33-44d4-4de9-947b-4707b78af23a on nch2-master-1 Jul 16 21:35:38.913595 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-eeaa6c41-1640-449e-8faa-0e9ae405a2c2 req-ce37227c-0ab2-4e70-8cdc-b55d7c6fcb51 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.069425 Jul 16 21:35:38.914301 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2524/5038] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:35:38 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 71 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:35:39.053806 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-f3cf2a18-d238-48dd-aa2b-abe394eadbef tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:35:39.053806 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-f3cf2a18-d238-48dd-aa2b-abe394eadbef tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] [instance: 19df5c33-44d4-4de9-947b-4707b78af23a] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:35:39.059125 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f3cf2a18-d238-48dd-aa2b-abe394eadbef tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:39.059436 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f3cf2a18-d238-48dd-aa2b-abe394eadbef tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:39.060226 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f3cf2a18-d238-48dd-aa2b-abe394eadbef tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:39.060674 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f3cf2a18-d238-48dd-aa2b-abe394eadbef tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:39.282591 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-f3cf2a18-d238-48dd-aa2b-abe394eadbef tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '726b46c54e9541fbb7825607bac3abe5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fd0ce287cb44cd59045868ce688909a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:35:39.292603 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-f3cf2a18-d238-48dd-aa2b-abe394eadbef tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '726b46c54e9541fbb7825607bac3abe5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fd0ce287cb44cd59045868ce688909a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:35:39.294345 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-f3cf2a18-d238-48dd-aa2b-abe394eadbef tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '726b46c54e9541fbb7825607bac3abe5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fd0ce287cb44cd59045868ce688909a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:35:39.295552 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-f3cf2a18-d238-48dd-aa2b-abe394eadbef tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] 10.222.0.16 "GET /compute/v2.1/servers/19df5c33-44d4-4de9-947b-4707b78af23a" status: 200 len: 1832 microversion: 2.42 time: 0.249723 Jul 16 21:35:39.307064 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2515/5039] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:35:39 2022] GET /compute/v2.1/servers/19df5c33-44d4-4de9-947b-4707b78af23a => generated 1832 bytes in 262 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:35:39.422774 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-dca11b3b-cc91-47a3-a2eb-da74fef07b10 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:35:39.423635 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-dca11b3b-cc91-47a3-a2eb-da74fef07b10 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: e4c7a9cb-e55d-4819-9885-2e72deffddb1] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:35:39.431363 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-dca11b3b-cc91-47a3-a2eb-da74fef07b10 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:39.431934 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-dca11b3b-cc91-47a3-a2eb-da74fef07b10 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:39.432845 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-dca11b3b-cc91-47a3-a2eb-da74fef07b10 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:39.433359 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-dca11b3b-cc91-47a3-a2eb-da74fef07b10 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:39.463102 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=100 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:35:39.785038 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-aaab93d1-9635-4060-8138-c3595e5ae452 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:35:39.785997 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-aaab93d1-9635-4060-8138-c3595e5ae452 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] [instance: 4df09776-dc3c-46ff-9e64-e20b5d4679fa] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:35:39.907491 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-aaab93d1-9635-4060-8138-c3595e5ae452 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0ab8b3f06e994bbca5d6dd14109bfe50', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e60c0863dc6f4313bdc873fda27ae2fc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:35:39.921847 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-aaab93d1-9635-4060-8138-c3595e5ae452 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] 10.222.0.16 "GET /compute/v2.1/servers/4df09776-dc3c-46ff-9e64-e20b5d4679fa" status: 200 len: 1241 microversion: 2.1 time: 0.140765 Jul 16 21:35:39.923110 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2516/5040] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:35:39 2022] GET /compute/v2.1/servers/4df09776-dc3c-46ff-9e64-e20b5d4679fa => generated 1241 bytes in 143 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:35:40.092716 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-dca11b3b-cc91-47a3-a2eb-da74fef07b10 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '22e264440fff43cdb7be6702839af37f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da2248fa190342f0bfe9211cae1e403b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:35:40.099680 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-dca11b3b-cc91-47a3-a2eb-da74fef07b10 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1" status: 200 len: 1517 microversion: 2.1 time: 0.683071 Jul 16 21:35:40.108862 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2525/5041] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:35:39 2022] GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1 => generated 1517 bytes in 693 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:35:40.317099 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-8ae83ca3-cf37-4ea2-bf67-33c283b35a6c tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:35:40.318148 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-8ae83ca3-cf37-4ea2-bf67-33c283b35a6c tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] [instance: 19df5c33-44d4-4de9-947b-4707b78af23a] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:35:40.342095 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8ae83ca3-cf37-4ea2-bf67-33c283b35a6c tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:40.342931 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8ae83ca3-cf37-4ea2-bf67-33c283b35a6c tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:40.343951 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8ae83ca3-cf37-4ea2-bf67-33c283b35a6c tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:40.347625 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8ae83ca3-cf37-4ea2-bf67-33c283b35a6c tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:40.607302 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-8ae83ca3-cf37-4ea2-bf67-33c283b35a6c tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '726b46c54e9541fbb7825607bac3abe5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fd0ce287cb44cd59045868ce688909a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:35:40.610149 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:35:40.624992 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-8ae83ca3-cf37-4ea2-bf67-33c283b35a6c tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '726b46c54e9541fbb7825607bac3abe5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fd0ce287cb44cd59045868ce688909a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:35:40.630930 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-8ae83ca3-cf37-4ea2-bf67-33c283b35a6c tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '726b46c54e9541fbb7825607bac3abe5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fd0ce287cb44cd59045868ce688909a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:35:40.632279 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-8ae83ca3-cf37-4ea2-bf67-33c283b35a6c tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] 10.222.0.16 "GET /compute/v2.1/servers/19df5c33-44d4-4de9-947b-4707b78af23a" status: 200 len: 1832 microversion: 2.42 time: 0.320696 Jul 16 21:35:40.662917 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2517/5042] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:35:40 2022] GET /compute/v2.1/servers/19df5c33-44d4-4de9-947b-4707b78af23a => generated 1832 bytes in 348 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:35:40.940699 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-3ea749d8-b7de-45e8-81ff-0e016c0e4a16 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:35:40.941920 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-3ea749d8-b7de-45e8-81ff-0e016c0e4a16 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] [instance: 4df09776-dc3c-46ff-9e64-e20b5d4679fa] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:35:40.949094 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3ea749d8-b7de-45e8-81ff-0e016c0e4a16 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:40.949760 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3ea749d8-b7de-45e8-81ff-0e016c0e4a16 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:40.950544 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3ea749d8-b7de-45e8-81ff-0e016c0e4a16 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:40.950988 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3ea749d8-b7de-45e8-81ff-0e016c0e4a16 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:41.054121 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-3ea749d8-b7de-45e8-81ff-0e016c0e4a16 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0ab8b3f06e994bbca5d6dd14109bfe50', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e60c0863dc6f4313bdc873fda27ae2fc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:35:41.072646 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-e0570b3f-211b-418f-a21f-8e9405bc1795 req-c84ba3b0-1a36-4e0f-9e79-ce1f2d8015db service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "19df5c33-44d4-4de9-947b-4707b78af23a", "tag": "3e3ac495-fa5d-4999-b113-9a54b000c446"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:35:41.074049 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-3ea749d8-b7de-45e8-81ff-0e016c0e4a16 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] 10.222.0.16 "GET /compute/v2.1/servers/4df09776-dc3c-46ff-9e64-e20b5d4679fa" status: 200 len: 1293 microversion: 2.1 time: 0.137417 Jul 16 21:35:41.075469 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2526/5043] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:35:40 2022] GET /compute/v2.1/servers/4df09776-dc3c-46ff-9e64-e20b5d4679fa => generated 1293 bytes in 140 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:35:41.089100 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-e0570b3f-211b-418f-a21f-8e9405bc1795 req-c84ba3b0-1a36-4e0f-9e79-ce1f2d8015db service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:41.091093 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-e0570b3f-211b-418f-a21f-8e9405bc1795 req-c84ba3b0-1a36-4e0f-9e79-ce1f2d8015db service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:41.112712 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-e0570b3f-211b-418f-a21f-8e9405bc1795 req-c84ba3b0-1a36-4e0f-9e79-ce1f2d8015db service nova] Creating event network-changed:3e3ac495-fa5d-4999-b113-9a54b000c446 for instance 19df5c33-44d4-4de9-947b-4707b78af23a on nch2-master-1 Jul 16 21:35:41.120484 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-e0570b3f-211b-418f-a21f-8e9405bc1795 req-c84ba3b0-1a36-4e0f-9e79-ce1f2d8015db service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.051448 Jul 16 21:35:41.122011 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2518/5044] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:35:41 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 16 21:35:41.127177 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-c365ab98-7f44-44c9-8256-06382ed1f71b tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:35:41.139369 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-c365ab98-7f44-44c9-8256-06382ed1f71b tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: e4c7a9cb-e55d-4819-9885-2e72deffddb1] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:35:41.153801 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c365ab98-7f44-44c9-8256-06382ed1f71b tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:41.164364 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c365ab98-7f44-44c9-8256-06382ed1f71b tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:41.165254 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c365ab98-7f44-44c9-8256-06382ed1f71b tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:41.173135 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c365ab98-7f44-44c9-8256-06382ed1f71b tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:41.651795 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-ecc1e302-bfc5-4ae1-a12c-3101349899f8 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:35:41.652799 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-ecc1e302-bfc5-4ae1-a12c-3101349899f8 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] [instance: 19df5c33-44d4-4de9-947b-4707b78af23a] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:35:41.668818 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ecc1e302-bfc5-4ae1-a12c-3101349899f8 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:41.670306 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ecc1e302-bfc5-4ae1-a12c-3101349899f8 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:41.678077 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ecc1e302-bfc5-4ae1-a12c-3101349899f8 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:41.678673 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ecc1e302-bfc5-4ae1-a12c-3101349899f8 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:41.685322 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-c365ab98-7f44-44c9-8256-06382ed1f71b tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '22e264440fff43cdb7be6702839af37f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da2248fa190342f0bfe9211cae1e403b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:35:41.701973 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-c365ab98-7f44-44c9-8256-06382ed1f71b tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1" status: 200 len: 1517 microversion: 2.1 time: 0.586161 Jul 16 21:35:41.704493 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2527/5045] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:35:41 2022] GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1 => generated 1517 bytes in 589 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:35:42.038881 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-ecc1e302-bfc5-4ae1-a12c-3101349899f8 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '726b46c54e9541fbb7825607bac3abe5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fd0ce287cb44cd59045868ce688909a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:35:42.056733 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-ecc1e302-bfc5-4ae1-a12c-3101349899f8 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '726b46c54e9541fbb7825607bac3abe5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fd0ce287cb44cd59045868ce688909a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:35:42.056733 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-ecc1e302-bfc5-4ae1-a12c-3101349899f8 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '726b46c54e9541fbb7825607bac3abe5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fd0ce287cb44cd59045868ce688909a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:35:42.062120 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-ecc1e302-bfc5-4ae1-a12c-3101349899f8 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] 10.222.0.16 "GET /compute/v2.1/servers/19df5c33-44d4-4de9-947b-4707b78af23a" status: 200 len: 1853 microversion: 2.42 time: 0.415805 Jul 16 21:35:42.062916 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2519/5046] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:35:41 2022] GET /compute/v2.1/servers/19df5c33-44d4-4de9-947b-4707b78af23a => generated 1853 bytes in 417 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:35:42.097251 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-5fde8cf5-3506-4a7b-a39c-12b8d8dec043 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:35:42.097881 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-5fde8cf5-3506-4a7b-a39c-12b8d8dec043 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] [instance: 4df09776-dc3c-46ff-9e64-e20b5d4679fa] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:35:42.130472 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5fde8cf5-3506-4a7b-a39c-12b8d8dec043 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:42.130940 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5fde8cf5-3506-4a7b-a39c-12b8d8dec043 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:42.131783 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5fde8cf5-3506-4a7b-a39c-12b8d8dec043 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:42.132232 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5fde8cf5-3506-4a7b-a39c-12b8d8dec043 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:42.276333 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-5fde8cf5-3506-4a7b-a39c-12b8d8dec043 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0ab8b3f06e994bbca5d6dd14109bfe50', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e60c0863dc6f4313bdc873fda27ae2fc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:35:42.286304 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-5fde8cf5-3506-4a7b-a39c-12b8d8dec043 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] 10.222.0.16 "GET /compute/v2.1/servers/4df09776-dc3c-46ff-9e64-e20b5d4679fa" status: 200 len: 1299 microversion: 2.1 time: 0.193630 Jul 16 21:35:42.289607 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2528/5047] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:35:42 2022] GET /compute/v2.1/servers/4df09776-dc3c-46ff-9e64-e20b5d4679fa => generated 1299 bytes in 198 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:35:42.723101 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-865e2e86-366c-4f6c-babf-868ab099b323 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:35:42.723807 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-865e2e86-366c-4f6c-babf-868ab099b323 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: e4c7a9cb-e55d-4819-9885-2e72deffddb1] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:35:42.734470 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-865e2e86-366c-4f6c-babf-868ab099b323 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:42.735257 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-865e2e86-366c-4f6c-babf-868ab099b323 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:42.736291 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-865e2e86-366c-4f6c-babf-868ab099b323 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:42.737061 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-865e2e86-366c-4f6c-babf-868ab099b323 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:43.081619 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-1cc897e3-cdcc-4cd4-9282-0b9880aa07e2 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:35:43.082498 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-1cc897e3-cdcc-4cd4-9282-0b9880aa07e2 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] [instance: 19df5c33-44d4-4de9-947b-4707b78af23a] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:35:43.092969 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1cc897e3-cdcc-4cd4-9282-0b9880aa07e2 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:43.093604 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1cc897e3-cdcc-4cd4-9282-0b9880aa07e2 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:43.095253 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1cc897e3-cdcc-4cd4-9282-0b9880aa07e2 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:43.095852 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1cc897e3-cdcc-4cd4-9282-0b9880aa07e2 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:43.176842 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-865e2e86-366c-4f6c-babf-868ab099b323 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '22e264440fff43cdb7be6702839af37f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da2248fa190342f0bfe9211cae1e403b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:35:43.187517 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-865e2e86-366c-4f6c-babf-868ab099b323 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1" status: 200 len: 1517 microversion: 2.1 time: 0.469759 Jul 16 21:35:43.188839 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2520/5048] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:35:42 2022] GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1 => generated 1517 bytes in 472 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:35:43.194638 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-63bfa140-ce2d-4b45-b3f6-161013416b0d req-394d5ead-ecd0-46d3-b017-2e06aede2d8c service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "19df5c33-44d4-4de9-947b-4707b78af23a", "tag": "78c19415-67c3-482e-9663-0cdac41890d3"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:35:43.204785 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-63bfa140-ce2d-4b45-b3f6-161013416b0d req-394d5ead-ecd0-46d3-b017-2e06aede2d8c service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:43.207471 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-63bfa140-ce2d-4b45-b3f6-161013416b0d req-394d5ead-ecd0-46d3-b017-2e06aede2d8c service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:43.221111 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-63bfa140-ce2d-4b45-b3f6-161013416b0d req-394d5ead-ecd0-46d3-b017-2e06aede2d8c service nova] Creating event network-changed:78c19415-67c3-482e-9663-0cdac41890d3 for instance 19df5c33-44d4-4de9-947b-4707b78af23a on nch2-master-1 Jul 16 21:35:43.230568 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-63bfa140-ce2d-4b45-b3f6-161013416b0d req-394d5ead-ecd0-46d3-b017-2e06aede2d8c service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.038579 Jul 16 21:35:43.231170 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2521/5049] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:35:43 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:35:43.314991 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-757f604a-b828-42b8-b41c-9d9089cbe73b tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:35:43.316220 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-757f604a-b828-42b8-b41c-9d9089cbe73b tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] [instance: 4df09776-dc3c-46ff-9e64-e20b5d4679fa] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:35:43.325886 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_api:SELECT=77,nova_api:INSERT=1 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:35:43.328434 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-757f604a-b828-42b8-b41c-9d9089cbe73b tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:43.329103 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-757f604a-b828-42b8-b41c-9d9089cbe73b tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:43.330297 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-757f604a-b828-42b8-b41c-9d9089cbe73b tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:43.331744 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-757f604a-b828-42b8-b41c-9d9089cbe73b tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:43.702400 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-1cc897e3-cdcc-4cd4-9282-0b9880aa07e2 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '726b46c54e9541fbb7825607bac3abe5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fd0ce287cb44cd59045868ce688909a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:35:43.713945 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-1cc897e3-cdcc-4cd4-9282-0b9880aa07e2 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '726b46c54e9541fbb7825607bac3abe5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fd0ce287cb44cd59045868ce688909a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:35:43.716452 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-1cc897e3-cdcc-4cd4-9282-0b9880aa07e2 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '726b46c54e9541fbb7825607bac3abe5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fd0ce287cb44cd59045868ce688909a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:35:43.717656 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-1cc897e3-cdcc-4cd4-9282-0b9880aa07e2 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] 10.222.0.16 "GET /compute/v2.1/servers/19df5c33-44d4-4de9-947b-4707b78af23a" status: 200 len: 1874 microversion: 2.42 time: 0.640579 Jul 16 21:35:43.724177 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2529/5050] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:35:43 2022] GET /compute/v2.1/servers/19df5c33-44d4-4de9-947b-4707b78af23a => generated 1874 bytes in 647 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 16 21:35:43.764358 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-757f604a-b828-42b8-b41c-9d9089cbe73b tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0ab8b3f06e994bbca5d6dd14109bfe50', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e60c0863dc6f4313bdc873fda27ae2fc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:35:43.780358 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-757f604a-b828-42b8-b41c-9d9089cbe73b tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] 10.222.0.16 "GET /compute/v2.1/servers/4df09776-dc3c-46ff-9e64-e20b5d4679fa" status: 200 len: 1367 microversion: 2.1 time: 0.474025 Jul 16 21:35:43.781453 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2522/5051] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:35:43 2022] GET /compute/v2.1/servers/4df09776-dc3c-46ff-9e64-e20b5d4679fa => generated 1367 bytes in 476 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:35:44.121322 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-24b89f8b-d65a-42e1-a52c-8bc46f33079e req-8dc0cb3c-6277-4e07-8ae2-b9695cfe433f service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "e4c7a9cb-e55d-4819-9885-2e72deffddb1", "name": "network-vif-plugged", "status": "completed", "tag": "0b459b05-0d03-4317-8a9b-bae73b0eaddd"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:35:44.147698 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-24b89f8b-d65a-42e1-a52c-8bc46f33079e req-8dc0cb3c-6277-4e07-8ae2-b9695cfe433f service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:44.148479 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-24b89f8b-d65a-42e1-a52c-8bc46f33079e req-8dc0cb3c-6277-4e07-8ae2-b9695cfe433f service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:44.166252 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-24b89f8b-d65a-42e1-a52c-8bc46f33079e req-8dc0cb3c-6277-4e07-8ae2-b9695cfe433f service nova] Creating event network-vif-plugged:0b459b05-0d03-4317-8a9b-bae73b0eaddd for instance e4c7a9cb-e55d-4819-9885-2e72deffddb1 on nch2-master-1 Jul 16 21:35:44.176626 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-24b89f8b-d65a-42e1-a52c-8bc46f33079e req-8dc0cb3c-6277-4e07-8ae2-b9695cfe433f service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.065262 Jul 16 21:35:44.178411 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2530/5052] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:35:44 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 68 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:35:44.205691 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-d6d0bcef-e6fc-4cf2-8988-64be021e7223 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:35:44.207048 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-d6d0bcef-e6fc-4cf2-8988-64be021e7223 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: e4c7a9cb-e55d-4819-9885-2e72deffddb1] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:35:44.234338 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d6d0bcef-e6fc-4cf2-8988-64be021e7223 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:44.234821 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d6d0bcef-e6fc-4cf2-8988-64be021e7223 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:44.235763 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d6d0bcef-e6fc-4cf2-8988-64be021e7223 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:44.236150 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d6d0bcef-e6fc-4cf2-8988-64be021e7223 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:44.690118 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-d6d0bcef-e6fc-4cf2-8988-64be021e7223 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '22e264440fff43cdb7be6702839af37f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da2248fa190342f0bfe9211cae1e403b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:35:44.712003 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-d6d0bcef-e6fc-4cf2-8988-64be021e7223 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1" status: 200 len: 1517 microversion: 2.1 time: 0.501834 Jul 16 21:35:44.712003 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2523/5053] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:35:44 2022] GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1 => generated 1517 bytes in 503 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:35:44.722476 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-96d58370-78db-4c3c-8aea-1850481141f4 req-1d184507-efb2-4ce3-940d-5a9c647a2ff0 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "4df09776-dc3c-46ff-9e64-e20b5d4679fa", "tag": "cc6f4c0a-93e4-4c67-8490-1bb34324c862"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:35:44.736130 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-96d58370-78db-4c3c-8aea-1850481141f4 req-1d184507-efb2-4ce3-940d-5a9c647a2ff0 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:44.737050 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-96d58370-78db-4c3c-8aea-1850481141f4 req-1d184507-efb2-4ce3-940d-5a9c647a2ff0 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:44.749126 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-c1a583b6-36cd-499e-b132-01c0dc91da76 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:35:44.749844 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-c1a583b6-36cd-499e-b132-01c0dc91da76 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] [instance: 19df5c33-44d4-4de9-947b-4707b78af23a] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:35:44.760336 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c1a583b6-36cd-499e-b132-01c0dc91da76 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:44.762589 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c1a583b6-36cd-499e-b132-01c0dc91da76 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:44.763845 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c1a583b6-36cd-499e-b132-01c0dc91da76 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:44.764299 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c1a583b6-36cd-499e-b132-01c0dc91da76 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:44.764515 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-96d58370-78db-4c3c-8aea-1850481141f4 req-1d184507-efb2-4ce3-940d-5a9c647a2ff0 service nova] Creating event network-changed:cc6f4c0a-93e4-4c67-8490-1bb34324c862 for instance 4df09776-dc3c-46ff-9e64-e20b5d4679fa on nch1-master-1 Jul 16 21:35:44.771857 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-96d58370-78db-4c3c-8aea-1850481141f4 req-1d184507-efb2-4ce3-940d-5a9c647a2ff0 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.059486 Jul 16 21:35:44.781466 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2531/5054] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:35:44 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 65 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:35:44.801581 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-3c6ed6f6-fcde-4c8d-8ce3-dac1567e92c2 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:35:44.802516 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-3c6ed6f6-fcde-4c8d-8ce3-dac1567e92c2 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] [instance: 4df09776-dc3c-46ff-9e64-e20b5d4679fa] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:35:44.809930 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3c6ed6f6-fcde-4c8d-8ce3-dac1567e92c2 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:44.810713 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3c6ed6f6-fcde-4c8d-8ce3-dac1567e92c2 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:44.816918 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3c6ed6f6-fcde-4c8d-8ce3-dac1567e92c2 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:44.817681 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3c6ed6f6-fcde-4c8d-8ce3-dac1567e92c2 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:45.276527 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-c1a583b6-36cd-499e-b132-01c0dc91da76 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '726b46c54e9541fbb7825607bac3abe5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fd0ce287cb44cd59045868ce688909a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:35:45.288265 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-3c6ed6f6-fcde-4c8d-8ce3-dac1567e92c2 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0ab8b3f06e994bbca5d6dd14109bfe50', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e60c0863dc6f4313bdc873fda27ae2fc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:35:45.291329 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-c1a583b6-36cd-499e-b132-01c0dc91da76 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '726b46c54e9541fbb7825607bac3abe5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fd0ce287cb44cd59045868ce688909a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:35:45.305199 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-c1a583b6-36cd-499e-b132-01c0dc91da76 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '726b46c54e9541fbb7825607bac3abe5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fd0ce287cb44cd59045868ce688909a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:35:45.305199 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-c1a583b6-36cd-499e-b132-01c0dc91da76 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] 10.222.0.16 "GET /compute/v2.1/servers/19df5c33-44d4-4de9-947b-4707b78af23a" status: 200 len: 1874 microversion: 2.42 time: 0.550641 Jul 16 21:35:45.305199 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2524/5055] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:35:44 2022] GET /compute/v2.1/servers/19df5c33-44d4-4de9-947b-4707b78af23a => generated 1874 bytes in 552 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:35:45.308116 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-e2a1b805-d3b8-41ab-afce-40441176fbd3 req-fefacfce-f918-4941-9136-f7e614d710a4 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "19df5c33-44d4-4de9-947b-4707b78af23a", "tag": "535c3524-f5da-4ba5-a149-40d9c379754e"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:35:45.320769 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-3c6ed6f6-fcde-4c8d-8ce3-dac1567e92c2 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] 10.222.0.16 "GET /compute/v2.1/servers/4df09776-dc3c-46ff-9e64-e20b5d4679fa" status: 200 len: 1367 microversion: 2.1 time: 0.524592 Jul 16 21:35:45.321859 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2532/5056] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:35:44 2022] GET /compute/v2.1/servers/4df09776-dc3c-46ff-9e64-e20b5d4679fa => generated 1367 bytes in 527 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:35:45.332325 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-e2a1b805-d3b8-41ab-afce-40441176fbd3 req-fefacfce-f918-4941-9136-f7e614d710a4 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:45.333094 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-e2a1b805-d3b8-41ab-afce-40441176fbd3 req-fefacfce-f918-4941-9136-f7e614d710a4 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:45.348823 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-e2a1b805-d3b8-41ab-afce-40441176fbd3 req-fefacfce-f918-4941-9136-f7e614d710a4 service nova] Creating event network-changed:535c3524-f5da-4ba5-a149-40d9c379754e for instance 19df5c33-44d4-4de9-947b-4707b78af23a on nch2-master-1 Jul 16 21:35:45.355459 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-e2a1b805-d3b8-41ab-afce-40441176fbd3 req-fefacfce-f918-4941-9136-f7e614d710a4 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.050654 Jul 16 21:35:45.356971 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2525/5057] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:35:45 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 16 21:35:45.721224 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-1fdacbd8-d822-43b8-87d3-a8439ac1bbad tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:35:45.722401 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-1fdacbd8-d822-43b8-87d3-a8439ac1bbad tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: e4c7a9cb-e55d-4819-9885-2e72deffddb1] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:35:45.729629 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1fdacbd8-d822-43b8-87d3-a8439ac1bbad tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:45.731155 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1fdacbd8-d822-43b8-87d3-a8439ac1bbad tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:45.732056 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1fdacbd8-d822-43b8-87d3-a8439ac1bbad tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:45.732562 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1fdacbd8-d822-43b8-87d3-a8439ac1bbad tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:46.314274 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-bfdea13d-c9db-43a9-bab1-f9eb0efb94a9 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:35:46.315255 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-bfdea13d-c9db-43a9-bab1-f9eb0efb94a9 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] [instance: 19df5c33-44d4-4de9-947b-4707b78af23a] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:35:46.330928 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-bfdea13d-c9db-43a9-bab1-f9eb0efb94a9 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:46.331579 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-bfdea13d-c9db-43a9-bab1-f9eb0efb94a9 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:46.332578 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-bfdea13d-c9db-43a9-bab1-f9eb0efb94a9 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:46.333037 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-bfdea13d-c9db-43a9-bab1-f9eb0efb94a9 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:46.338241 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-1fdacbd8-d822-43b8-87d3-a8439ac1bbad tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '22e264440fff43cdb7be6702839af37f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da2248fa190342f0bfe9211cae1e403b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:35:46.351324 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-1fdacbd8-d822-43b8-87d3-a8439ac1bbad tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1" status: 200 len: 1517 microversion: 2.1 time: 0.633956 Jul 16 21:35:46.353164 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2533/5058] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:35:45 2022] GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1 => generated 1517 bytes in 637 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:35:46.363576 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-3a411fb1-eb3b-4f3b-9b9f-cdd9c2e19830 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:35:46.364209 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-3a411fb1-eb3b-4f3b-9b9f-cdd9c2e19830 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] [instance: 4df09776-dc3c-46ff-9e64-e20b5d4679fa] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:35:46.380602 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3a411fb1-eb3b-4f3b-9b9f-cdd9c2e19830 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:46.381068 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3a411fb1-eb3b-4f3b-9b9f-cdd9c2e19830 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:46.381865 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3a411fb1-eb3b-4f3b-9b9f-cdd9c2e19830 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:46.382316 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3a411fb1-eb3b-4f3b-9b9f-cdd9c2e19830 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:46.834812 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-3a411fb1-eb3b-4f3b-9b9f-cdd9c2e19830 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0ab8b3f06e994bbca5d6dd14109bfe50', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e60c0863dc6f4313bdc873fda27ae2fc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:35:46.848031 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-3a411fb1-eb3b-4f3b-9b9f-cdd9c2e19830 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] 10.222.0.16 "GET /compute/v2.1/servers/4df09776-dc3c-46ff-9e64-e20b5d4679fa" status: 200 len: 1367 microversion: 2.1 time: 0.493070 Jul 16 21:35:46.859739 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2534/5059] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:35:46 2022] GET /compute/v2.1/servers/4df09776-dc3c-46ff-9e64-e20b5d4679fa => generated 1367 bytes in 505 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:35:46.861317 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-bfdea13d-c9db-43a9-bab1-f9eb0efb94a9 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '726b46c54e9541fbb7825607bac3abe5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fd0ce287cb44cd59045868ce688909a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:35:46.880436 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-bfdea13d-c9db-43a9-bab1-f9eb0efb94a9 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '726b46c54e9541fbb7825607bac3abe5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fd0ce287cb44cd59045868ce688909a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:35:46.887149 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-bfdea13d-c9db-43a9-bab1-f9eb0efb94a9 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '726b46c54e9541fbb7825607bac3abe5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fd0ce287cb44cd59045868ce688909a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:35:46.887149 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-bfdea13d-c9db-43a9-bab1-f9eb0efb94a9 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] 10.222.0.16 "GET /compute/v2.1/servers/19df5c33-44d4-4de9-947b-4707b78af23a" status: 200 len: 1874 microversion: 2.42 time: 0.575897 Jul 16 21:35:46.899262 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2526/5060] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:35:46 2022] GET /compute/v2.1/servers/19df5c33-44d4-4de9-947b-4707b78af23a => generated 1874 bytes in 591 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 16 21:35:47.373007 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-434af019-c2e3-40b6-88a3-b719e942a811 req-70b2f444-6c8e-4cce-aa8a-8ff9dbc6c8f7 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "19df5c33-44d4-4de9-947b-4707b78af23a", "tag": "4c5dcf45-0de2-481d-9d69-0ca39f95898b"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:35:47.376300 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-f7f57474-c38c-45ad-953f-2a39d815b80f tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:35:47.383987 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-f7f57474-c38c-45ad-953f-2a39d815b80f tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: e4c7a9cb-e55d-4819-9885-2e72deffddb1] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:35:47.396003 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-434af019-c2e3-40b6-88a3-b719e942a811 req-70b2f444-6c8e-4cce-aa8a-8ff9dbc6c8f7 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:47.396235 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f7f57474-c38c-45ad-953f-2a39d815b80f tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:47.396330 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-434af019-c2e3-40b6-88a3-b719e942a811 req-70b2f444-6c8e-4cce-aa8a-8ff9dbc6c8f7 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:47.396424 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f7f57474-c38c-45ad-953f-2a39d815b80f tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:47.396424 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f7f57474-c38c-45ad-953f-2a39d815b80f tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:47.396424 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f7f57474-c38c-45ad-953f-2a39d815b80f tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:47.417981 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-434af019-c2e3-40b6-88a3-b719e942a811 req-70b2f444-6c8e-4cce-aa8a-8ff9dbc6c8f7 service nova] Creating event network-changed:4c5dcf45-0de2-481d-9d69-0ca39f95898b for instance 19df5c33-44d4-4de9-947b-4707b78af23a on nch2-master-1 Jul 16 21:35:47.430352 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-434af019-c2e3-40b6-88a3-b719e942a811 req-70b2f444-6c8e-4cce-aa8a-8ff9dbc6c8f7 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.061059 Jul 16 21:35:47.431580 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2527/5061] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:35:47 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 63 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:35:47.870203 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-c4a392b3-c95b-442b-8dce-74d0c8ca745b tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:35:47.871194 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-c4a392b3-c95b-442b-8dce-74d0c8ca745b tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] [instance: 4df09776-dc3c-46ff-9e64-e20b5d4679fa] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:35:47.880046 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c4a392b3-c95b-442b-8dce-74d0c8ca745b tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:47.880046 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c4a392b3-c95b-442b-8dce-74d0c8ca745b tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:47.880306 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c4a392b3-c95b-442b-8dce-74d0c8ca745b tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:47.880911 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c4a392b3-c95b-442b-8dce-74d0c8ca745b tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:48.024148 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-f7f57474-c38c-45ad-953f-2a39d815b80f tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '22e264440fff43cdb7be6702839af37f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da2248fa190342f0bfe9211cae1e403b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:35:48.033426 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-f7f57474-c38c-45ad-953f-2a39d815b80f tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1" status: 200 len: 1517 microversion: 2.1 time: 0.667365 Jul 16 21:35:48.046159 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2535/5062] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:35:47 2022] GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1 => generated 1517 bytes in 677 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:35:48.054957 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-82199981-e0ac-4fed-9156-f476bcda3163 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:35:48.055953 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-82199981-e0ac-4fed-9156-f476bcda3163 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] [instance: 19df5c33-44d4-4de9-947b-4707b78af23a] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:35:48.065386 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-82199981-e0ac-4fed-9156-f476bcda3163 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:48.065861 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-82199981-e0ac-4fed-9156-f476bcda3163 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:48.067115 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-82199981-e0ac-4fed-9156-f476bcda3163 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:48.068324 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-82199981-e0ac-4fed-9156-f476bcda3163 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:48.423051 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell0:SELECT=2 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:35:48.476214 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell0:SELECT=6 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:35:48.518837 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-c4a392b3-c95b-442b-8dce-74d0c8ca745b tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0ab8b3f06e994bbca5d6dd14109bfe50', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e60c0863dc6f4313bdc873fda27ae2fc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:35:48.527149 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-c4a392b3-c95b-442b-8dce-74d0c8ca745b tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] 10.222.0.16 "GET /compute/v2.1/servers/4df09776-dc3c-46ff-9e64-e20b5d4679fa" status: 200 len: 1367 microversion: 2.1 time: 0.665594 Jul 16 21:35:48.527838 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2528/5063] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:35:47 2022] GET /compute/v2.1/servers/4df09776-dc3c-46ff-9e64-e20b5d4679fa => generated 1367 bytes in 667 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:35:48.629993 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-82199981-e0ac-4fed-9156-f476bcda3163 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '726b46c54e9541fbb7825607bac3abe5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fd0ce287cb44cd59045868ce688909a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:35:48.640813 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-82199981-e0ac-4fed-9156-f476bcda3163 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '726b46c54e9541fbb7825607bac3abe5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fd0ce287cb44cd59045868ce688909a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:35:48.642487 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-82199981-e0ac-4fed-9156-f476bcda3163 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '726b46c54e9541fbb7825607bac3abe5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fd0ce287cb44cd59045868ce688909a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:35:48.644326 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-82199981-e0ac-4fed-9156-f476bcda3163 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] 10.222.0.16 "GET /compute/v2.1/servers/19df5c33-44d4-4de9-947b-4707b78af23a" status: 200 len: 1874 microversion: 2.42 time: 0.596747 Jul 16 21:35:48.645662 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2536/5064] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:35:48 2022] GET /compute/v2.1/servers/19df5c33-44d4-4de9-947b-4707b78af23a => generated 1874 bytes in 599 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:35:49.055121 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-8514bdcb-0fad-432f-a3e0-e2205c7a7ff9 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:35:49.055749 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-8514bdcb-0fad-432f-a3e0-e2205c7a7ff9 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: e4c7a9cb-e55d-4819-9885-2e72deffddb1] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:35:49.066389 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8514bdcb-0fad-432f-a3e0-e2205c7a7ff9 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:49.070057 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8514bdcb-0fad-432f-a3e0-e2205c7a7ff9 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:49.072560 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8514bdcb-0fad-432f-a3e0-e2205c7a7ff9 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:49.075267 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8514bdcb-0fad-432f-a3e0-e2205c7a7ff9 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:49.402457 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-8514bdcb-0fad-432f-a3e0-e2205c7a7ff9 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '22e264440fff43cdb7be6702839af37f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da2248fa190342f0bfe9211cae1e403b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:35:49.412828 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-8514bdcb-0fad-432f-a3e0-e2205c7a7ff9 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1" status: 200 len: 1504 microversion: 2.1 time: 0.364340 Jul 16 21:35:49.414190 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2529/5065] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:35:49 2022] GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1 => generated 1504 bytes in 367 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:35:49.435960 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-19be0692-c8d3-4608-970d-6c8281d59843 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:35:49.437163 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-19be0692-c8d3-4608-970d-6c8281d59843 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: e4c7a9cb-e55d-4819-9885-2e72deffddb1] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:35:49.448744 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-19be0692-c8d3-4608-970d-6c8281d59843 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:49.449429 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-19be0692-c8d3-4608-970d-6c8281d59843 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:49.450841 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-19be0692-c8d3-4608-970d-6c8281d59843 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:49.451526 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-19be0692-c8d3-4608-970d-6c8281d59843 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:49.494275 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-fd99a822-9514-4079-9a41-cc427970cbae req-5624a791-6d8c-4745-83d4-3bc425488d73 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "19df5c33-44d4-4de9-947b-4707b78af23a", "tag": "25ef7728-f1b7-4467-9f81-8af48bbcef2d"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:35:49.514372 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-fd99a822-9514-4079-9a41-cc427970cbae req-5624a791-6d8c-4745-83d4-3bc425488d73 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:49.514372 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-fd99a822-9514-4079-9a41-cc427970cbae req-5624a791-6d8c-4745-83d4-3bc425488d73 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:49.564868 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-fd99a822-9514-4079-9a41-cc427970cbae req-5624a791-6d8c-4745-83d4-3bc425488d73 service nova] Creating event network-changed:25ef7728-f1b7-4467-9f81-8af48bbcef2d for instance 19df5c33-44d4-4de9-947b-4707b78af23a on nch2-master-1 Jul 16 21:35:49.573960 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-fd99a822-9514-4079-9a41-cc427970cbae req-5624a791-6d8c-4745-83d4-3bc425488d73 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.083347 Jul 16 21:35:49.574843 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2530/5066] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:35:49 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 16 21:35:49.580858 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-abdc577e-5824-48fc-b7f0-d372fdcd75ac tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:35:49.581929 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-abdc577e-5824-48fc-b7f0-d372fdcd75ac tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] [instance: 4df09776-dc3c-46ff-9e64-e20b5d4679fa] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:35:49.589030 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-abdc577e-5824-48fc-b7f0-d372fdcd75ac tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:49.589643 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-abdc577e-5824-48fc-b7f0-d372fdcd75ac tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:49.591179 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-abdc577e-5824-48fc-b7f0-d372fdcd75ac tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:49.591780 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-abdc577e-5824-48fc-b7f0-d372fdcd75ac tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:49.912878 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell0:SELECT=8 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:35:49.952492 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-19be0692-c8d3-4608-970d-6c8281d59843 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '22e264440fff43cdb7be6702839af37f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da2248fa190342f0bfe9211cae1e403b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:35:49.966148 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-19be0692-c8d3-4608-970d-6c8281d59843 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1" status: 200 len: 1504 microversion: 2.1 time: 0.534210 Jul 16 21:35:49.966762 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2537/5067] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:35:49 2022] GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1 => generated 1504 bytes in 536 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:35:49.984224 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-0389e2a8-2ba4-4da7-b27e-047ac0157c4d tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:35:49.984224 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-0389e2a8-2ba4-4da7-b27e-047ac0157c4d tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] [instance: 19df5c33-44d4-4de9-947b-4707b78af23a] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:35:49.990264 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0389e2a8-2ba4-4da7-b27e-047ac0157c4d tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:49.990723 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0389e2a8-2ba4-4da7-b27e-047ac0157c4d tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:49.991540 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0389e2a8-2ba4-4da7-b27e-047ac0157c4d tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:49.991983 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0389e2a8-2ba4-4da7-b27e-047ac0157c4d tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:49.999587 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-abdc577e-5824-48fc-b7f0-d372fdcd75ac tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0ab8b3f06e994bbca5d6dd14109bfe50', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e60c0863dc6f4313bdc873fda27ae2fc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:35:50.011540 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-abdc577e-5824-48fc-b7f0-d372fdcd75ac tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] 10.222.0.16 "GET /compute/v2.1/servers/4df09776-dc3c-46ff-9e64-e20b5d4679fa" status: 200 len: 1367 microversion: 2.1 time: 0.435231 Jul 16 21:35:50.012339 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2531/5068] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:35:49 2022] GET /compute/v2.1/servers/4df09776-dc3c-46ff-9e64-e20b5d4679fa => generated 1367 bytes in 437 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:35:50.026757 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-33b746d7-f865-4adc-94dd-e1d77a191216 req-d22879e8-4f7f-453d-80de-16d11c54e873 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "4df09776-dc3c-46ff-9e64-e20b5d4679fa", "name": "network-vif-plugged", "status": "completed", "tag": "cc6f4c0a-93e4-4c67-8490-1bb34324c862"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:35:50.036191 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-33b746d7-f865-4adc-94dd-e1d77a191216 req-d22879e8-4f7f-453d-80de-16d11c54e873 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:50.036928 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-33b746d7-f865-4adc-94dd-e1d77a191216 req-d22879e8-4f7f-453d-80de-16d11c54e873 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:50.052234 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-33b746d7-f865-4adc-94dd-e1d77a191216 req-d22879e8-4f7f-453d-80de-16d11c54e873 service nova] Creating event network-vif-plugged:cc6f4c0a-93e4-4c67-8490-1bb34324c862 for instance 4df09776-dc3c-46ff-9e64-e20b5d4679fa on nch1-master-1 Jul 16 21:35:50.058169 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-33b746d7-f865-4adc-94dd-e1d77a191216 req-d22879e8-4f7f-453d-80de-16d11c54e873 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.035223 Jul 16 21:35:50.058986 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2532/5069] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:35:50 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:35:50.066744 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-9d4280a2-12c3-4178-a8ac-71e2377c8c8a tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:35:50.067695 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-9d4280a2-12c3-4178-a8ac-71e2377c8c8a tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: e4c7a9cb-e55d-4819-9885-2e72deffddb1] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:35:50.074639 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9d4280a2-12c3-4178-a8ac-71e2377c8c8a tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:50.075206 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9d4280a2-12c3-4178-a8ac-71e2377c8c8a tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:50.075998 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9d4280a2-12c3-4178-a8ac-71e2377c8c8a tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:50.076450 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9d4280a2-12c3-4178-a8ac-71e2377c8c8a tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:50.551291 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-9d4280a2-12c3-4178-a8ac-71e2377c8c8a tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '22e264440fff43cdb7be6702839af37f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da2248fa190342f0bfe9211cae1e403b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:35:50.562100 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-0389e2a8-2ba4-4da7-b27e-047ac0157c4d tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '726b46c54e9541fbb7825607bac3abe5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fd0ce287cb44cd59045868ce688909a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:35:50.582171 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-9d4280a2-12c3-4178-a8ac-71e2377c8c8a tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1" status: 200 len: 1504 microversion: 2.1 time: 0.521763 Jul 16 21:35:50.590064 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2533/5070] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:35:50 2022] GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1 => generated 1504 bytes in 530 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:35:50.590409 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-0389e2a8-2ba4-4da7-b27e-047ac0157c4d tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '726b46c54e9541fbb7825607bac3abe5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fd0ce287cb44cd59045868ce688909a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:35:50.592603 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-0389e2a8-2ba4-4da7-b27e-047ac0157c4d tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '726b46c54e9541fbb7825607bac3abe5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fd0ce287cb44cd59045868ce688909a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:35:50.594042 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-0389e2a8-2ba4-4da7-b27e-047ac0157c4d tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] 10.222.0.16 "GET /compute/v2.1/servers/19df5c33-44d4-4de9-947b-4707b78af23a" status: 200 len: 1874 microversion: 2.42 time: 0.625845 Jul 16 21:35:50.596078 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2538/5071] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:35:49 2022] GET /compute/v2.1/servers/19df5c33-44d4-4de9-947b-4707b78af23a => generated 1874 bytes in 628 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:35:50.606876 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-d7221be1-6968-4770-a5b9-39074f961fb3 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Action: 'action', calling method: >, body: {"os-stop": {}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:35:50.618885 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-d7221be1-6968-4770-a5b9-39074f961fb3 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] HTTP exception thrown: Instance dfb0a6d4-f92f-4012-8245-113c8d4a604c has no mapping to a cell. Jul 16 21:35:50.629170 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-d7221be1-6968-4770-a5b9-39074f961fb3 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Request method failure captured: Jul 16 21:35:50.629170 ncdv-master-1 devstack@n-api.service[138680]: request: POST /compute/v2.1/servers/dfb0a6d4-f92f-4012-8245-113c8d4a604c/action HTTP/1.1 Jul 16 21:35:50.629170 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:35:50.629170 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:35:50.629170 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:35:50.629170 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 15 Jul 16 21:35:50.629170 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:35:50.629170 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:35:50.629170 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:35:50.629170 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:35:50.629170 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0y6QJDVdPNcLbUnwwgOJV-QkddLQ7yqnzBNVp2CrOe5UQqjQOER3spl3PswEW82ISSx8FhmuIrEwFSG69Ze1pTOmWBWZ8PURIgVHbN3yBPA3U2S2TKkQkFwyNuvvV03MCr9U0QGJIjXVOkiqFEAy8q87U_okZgCdI-BFjBFgrCdfAEo Jul 16 21:35:50.629170 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:35:50.629170 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:35:50.629170 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:35:50.629170 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:35:50.629170 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:35:50.629170 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:35:50.629170 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: da2248fa190342f0bfe9211cae1e403b Jul 16 21:35:50.629170 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-ServersNegativeTestJSON-1102403437 Jul 16 21:35:50.629170 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:35:50.629170 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:35:50.629170 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/da2248fa190342f0bfe9211cae1e403b"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/da2248fa190342f0bfe9211cae1e403b"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_da2248fa190342f0bfe9211cae1e403b"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/da2248fa190342f0bfe9211cae1e403b"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:35:50.629170 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-ServersNegativeTestJSON-1102403437 Jul 16 21:35:50.629170 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: da2248fa190342f0bfe9211cae1e403b Jul 16 21:35:50.629170 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-ServersNegativeTestJSON-1102403437 Jul 16 21:35:50.629170 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-ServersNegativeTestJSON-1102403437-project Jul 16 21:35:50.629170 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:35:50.629170 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:35:50.629170 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: 22e264440fff43cdb7be6702839af37f Jul 16 21:35:50.629170 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-ServersNegativeTestJSON-1102403437-project Jul 16 21:35:50.629170 ncdv-master-1 devstack@n-api.service[138680]: Jul 16 21:35:50.629170 ncdv-master-1 devstack@n-api.service[138680]: {"os-stop": {}} Jul 16 21:35:50.629170 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 21:35:50.629170 ncdv-master-1 devstack@n-api.service[138680]: exception: Instance dfb0a6d4-f92f-4012-8245-113c8d4a604c has no mapping to a cell. Jul 16 21:35:50.629170 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': 'dfb0a6d4-f92f-4012-8245-113c8d4a604c', 'body': {'os-stop': {}}} Jul 16 21:35:50.629170 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:35:50.629170 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:35:50.629170 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1408, in _get_instance Jul 16 21:35:50.629170 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi mapping = objects.InstanceMapping.get_by_instance_uuid( Jul 16 21:35:50.629170 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Jul 16 21:35:50.629170 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Jul 16 21:35:50.629170 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/instance_mapping.py", line 112, in get_by_instance_uuid Jul 16 21:35:50.629170 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi db_mapping = cls._get_by_instance_uuid_from_db(context, instance_uuid) Jul 16 21:35:50.629170 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1010, in wrapper Jul 16 21:35:50.629170 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return fn(*args, **kwargs) Jul 16 21:35:50.629170 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/instance_mapping.py", line 106, in _get_by_instance_uuid_from_db Jul 16 21:35:50.629170 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.InstanceMappingNotFound(uuid=instance_uuid) Jul 16 21:35:50.629170 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.InstanceMappingNotFound: Instance dfb0a6d4-f92f-4012-8245-113c8d4a604c has no mapping to a cell. Jul 16 21:35:50.629170 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:35:50.629170 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:35:50.629170 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:35:50.629170 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:35:50.629170 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:35:50.629170 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:35:50.629170 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:35:50.629170 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:35:50.629170 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:35:50.629170 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:35:50.629170 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1441, in _stop_server Jul 16 21:35:50.629170 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, id) Jul 16 21:35:50.629170 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1415, in _get_instance Jul 16 21:35:50.629170 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:35:50.629170 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance dfb0a6d4-f92f-4012-8245-113c8d4a604c has no mapping to a cell. Jul 16 21:35:50.633305 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:35:50.633305 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:35:50.633305 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:35:50.633305 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:35:50.633305 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:35:50.633305 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:35:50.633305 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:35:50.633305 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:35:50.633305 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance dfb0a6d4-f92f-4012-8245-113c8d4a604c has no mapping to a cell. Jul 16 21:35:50.633305 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:35:50.636442 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-d7221be1-6968-4770-a5b9-39074f961fb3 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Returning 404 to user: Instance dfb0a6d4-f92f-4012-8245-113c8d4a604c has no mapping to a cell. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:35:50.638442 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-d7221be1-6968-4770-a5b9-39074f961fb3 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "POST /compute/v2.1/servers/dfb0a6d4-f92f-4012-8245-113c8d4a604c/action" status: 404 len: 117 microversion: 2.1 time: 0.035365 Jul 16 21:35:50.640951 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2534/5072] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 21:35:50 2022] POST /compute/v2.1/servers/dfb0a6d4-f92f-4012-8245-113c8d4a604c/action => generated 117 bytes in 40 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:35:50.650290 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-42651fb4-1904-4ba5-acec-e444bedaeaaa tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:35:50.650803 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-42651fb4-1904-4ba5-acec-e444bedaeaaa tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: e4c7a9cb-e55d-4819-9885-2e72deffddb1] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:35:50.661551 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-42651fb4-1904-4ba5-acec-e444bedaeaaa tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:50.662458 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-42651fb4-1904-4ba5-acec-e444bedaeaaa tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:50.665841 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-42651fb4-1904-4ba5-acec-e444bedaeaaa tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:50.666580 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-42651fb4-1904-4ba5-acec-e444bedaeaaa tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:50.879757 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-42651fb4-1904-4ba5-acec-e444bedaeaaa tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '22e264440fff43cdb7be6702839af37f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da2248fa190342f0bfe9211cae1e403b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:35:50.889813 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-42651fb4-1904-4ba5-acec-e444bedaeaaa tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1" status: 200 len: 1504 microversion: 2.1 time: 0.244189 Jul 16 21:35:50.891257 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2539/5073] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:35:50 2022] GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1 => generated 1504 bytes in 247 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:35:50.908705 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-7d6cf9b3-33a3-4878-a18f-a2529b1ca03a tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:35:50.909280 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-7d6cf9b3-33a3-4878-a18f-a2529b1ca03a tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: e4c7a9cb-e55d-4819-9885-2e72deffddb1] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:35:50.922120 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7d6cf9b3-33a3-4878-a18f-a2529b1ca03a tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:50.923085 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7d6cf9b3-33a3-4878-a18f-a2529b1ca03a tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:50.925182 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7d6cf9b3-33a3-4878-a18f-a2529b1ca03a tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:50.925857 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7d6cf9b3-33a3-4878-a18f-a2529b1ca03a tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:51.030011 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-cb21cc9a-a752-4bfe-8d89-0efd96e24df0 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:35:51.031473 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-cb21cc9a-a752-4bfe-8d89-0efd96e24df0 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] [instance: 4df09776-dc3c-46ff-9e64-e20b5d4679fa] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:35:51.039945 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cb21cc9a-a752-4bfe-8d89-0efd96e24df0 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:51.041082 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cb21cc9a-a752-4bfe-8d89-0efd96e24df0 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:51.042099 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cb21cc9a-a752-4bfe-8d89-0efd96e24df0 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:51.042765 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cb21cc9a-a752-4bfe-8d89-0efd96e24df0 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:51.312196 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-7d6cf9b3-33a3-4878-a18f-a2529b1ca03a tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '22e264440fff43cdb7be6702839af37f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da2248fa190342f0bfe9211cae1e403b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:35:51.328260 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-7d6cf9b3-33a3-4878-a18f-a2529b1ca03a tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1" status: 200 len: 1504 microversion: 2.1 time: 0.423861 Jul 16 21:35:51.329803 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2535/5074] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:35:50 2022] GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1 => generated 1504 bytes in 426 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:35:51.350792 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-fe9415da-db34-4340-94cd-edde7e0f2f04 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Action: 'action', calling method: >, body: {"suspend": {}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:35:51.351370 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-fe9415da-db34-4340-94cd-edde7e0f2f04 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: 3bb02b0a-3dbd-464a-b6b7-bd635c9f845b] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:35:51.360295 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-fe9415da-db34-4340-94cd-edde7e0f2f04 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] HTTP exception thrown: Instance 3bb02b0a-3dbd-464a-b6b7-bd635c9f845b could not be found. Jul 16 21:35:51.361813 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-fe9415da-db34-4340-94cd-edde7e0f2f04 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Request method failure captured: Jul 16 21:35:51.361813 ncdv-master-1 devstack@n-api.service[138680]: request: POST /compute/v2.1/servers/3bb02b0a-3dbd-464a-b6b7-bd635c9f845b/action HTTP/1.1 Jul 16 21:35:51.361813 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:35:51.361813 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:35:51.361813 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:35:51.361813 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 15 Jul 16 21:35:51.361813 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:35:51.361813 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:35:51.361813 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:35:51.361813 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:35:51.361813 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0y6QJDVdPNcLbUnwwgOJV-QkddLQ7yqnzBNVp2CrOe5UQqjQOER3spl3PswEW82ISSx8FhmuIrEwFSG69Ze1pTOmWBWZ8PURIgVHbN3yBPA3U2S2TKkQkFwyNuvvV03MCr9U0QGJIjXVOkiqFEAy8q87U_okZgCdI-BFjBFgrCdfAEo Jul 16 21:35:51.361813 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:35:51.361813 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:35:51.361813 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:35:51.361813 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:35:51.361813 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:35:51.361813 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:35:51.361813 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: da2248fa190342f0bfe9211cae1e403b Jul 16 21:35:51.361813 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-ServersNegativeTestJSON-1102403437 Jul 16 21:35:51.361813 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:35:51.361813 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:35:51.361813 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/da2248fa190342f0bfe9211cae1e403b"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/da2248fa190342f0bfe9211cae1e403b"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_da2248fa190342f0bfe9211cae1e403b"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/da2248fa190342f0bfe9211cae1e403b"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:35:51.361813 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-ServersNegativeTestJSON-1102403437 Jul 16 21:35:51.361813 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: da2248fa190342f0bfe9211cae1e403b Jul 16 21:35:51.361813 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-ServersNegativeTestJSON-1102403437 Jul 16 21:35:51.361813 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-ServersNegativeTestJSON-1102403437-project Jul 16 21:35:51.361813 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:35:51.361813 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:35:51.361813 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: 22e264440fff43cdb7be6702839af37f Jul 16 21:35:51.361813 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-ServersNegativeTestJSON-1102403437-project Jul 16 21:35:51.361813 ncdv-master-1 devstack@n-api.service[138680]: Jul 16 21:35:51.361813 ncdv-master-1 devstack@n-api.service[138680]: {"suspend": {}} Jul 16 21:35:51.361813 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 21:35:51.361813 ncdv-master-1 devstack@n-api.service[138680]: exception: Instance 3bb02b0a-3dbd-464a-b6b7-bd635c9f845b could not be found. Jul 16 21:35:51.361813 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': '3bb02b0a-3dbd-464a-b6b7-bd635c9f845b', 'body': {'suspend': {}}} Jul 16 21:35:51.361813 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:35:51.361813 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:35:51.361813 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:35:51.361813 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:35:51.361813 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:35:51.361813 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:35:51.361813 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2898, in _get_instance Jul 16 21:35:51.361813 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=instance_uuid) Jul 16 21:35:51.361813 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 3bb02b0a-3dbd-464a-b6b7-bd635c9f845b could not be found. Jul 16 21:35:51.361813 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:35:51.361813 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:35:51.361813 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:35:51.361813 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:35:51.361813 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:35:51.361813 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:35:51.361813 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:35:51.361813 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:35:51.361813 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:35:51.361813 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:35:51.361813 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/suspend_server.py", line 35, in _suspend Jul 16 21:35:51.361813 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi server = common.get_instance(self.compute_api, context, id) Jul 16 21:35:51.361813 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:35:51.361813 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:35:51.361813 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 3bb02b0a-3dbd-464a-b6b7-bd635c9f845b could not be found. Jul 16 21:35:51.361813 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:35:51.361813 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:35:51.361813 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:35:51.361813 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:35:51.361813 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:35:51.361813 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:35:51.367086 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:35:51.367086 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:35:51.367086 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 3bb02b0a-3dbd-464a-b6b7-bd635c9f845b could not be found. Jul 16 21:35:51.367086 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:35:51.367086 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-fe9415da-db34-4340-94cd-edde7e0f2f04 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Returning 404 to user: Instance 3bb02b0a-3dbd-464a-b6b7-bd635c9f845b could not be found. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:35:51.367086 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-fe9415da-db34-4340-94cd-edde7e0f2f04 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "POST /compute/v2.1/servers/3bb02b0a-3dbd-464a-b6b7-bd635c9f845b/action" status: 404 len: 111 microversion: 2.1 time: 0.019310 Jul 16 21:35:51.367086 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2536/5075] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 21:35:51 2022] POST /compute/v2.1/servers/3bb02b0a-3dbd-464a-b6b7-bd635c9f845b/action => generated 111 bytes in 22 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Jul 16 21:35:51.375288 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-cee56e6b-b17d-4c86-a9ac-108da2869da5 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:35:51.376200 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-cee56e6b-b17d-4c86-a9ac-108da2869da5 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: e4c7a9cb-e55d-4819-9885-2e72deffddb1] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:35:51.385889 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cee56e6b-b17d-4c86-a9ac-108da2869da5 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:51.386341 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cee56e6b-b17d-4c86-a9ac-108da2869da5 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:51.387156 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cee56e6b-b17d-4c86-a9ac-108da2869da5 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:51.387600 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cee56e6b-b17d-4c86-a9ac-108da2869da5 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:51.388657 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-cb21cc9a-a752-4bfe-8d89-0efd96e24df0 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0ab8b3f06e994bbca5d6dd14109bfe50', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e60c0863dc6f4313bdc873fda27ae2fc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:35:51.400966 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-cb21cc9a-a752-4bfe-8d89-0efd96e24df0 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] 10.222.0.16 "GET /compute/v2.1/servers/4df09776-dc3c-46ff-9e64-e20b5d4679fa" status: 200 len: 1367 microversion: 2.1 time: 0.375134 Jul 16 21:35:51.402322 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2540/5076] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:35:51 2022] GET /compute/v2.1/servers/4df09776-dc3c-46ff-9e64-e20b5d4679fa => generated 1367 bytes in 377 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:35:51.630513 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-f1fc4b06-5f51-4476-b0d9-a8d61a095f95 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:35:51.634261 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-f1fc4b06-5f51-4476-b0d9-a8d61a095f95 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] [instance: 19df5c33-44d4-4de9-947b-4707b78af23a] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:35:51.652317 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f1fc4b06-5f51-4476-b0d9-a8d61a095f95 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:51.654106 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f1fc4b06-5f51-4476-b0d9-a8d61a095f95 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:51.655442 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f1fc4b06-5f51-4476-b0d9-a8d61a095f95 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:51.656364 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f1fc4b06-5f51-4476-b0d9-a8d61a095f95 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:52.181756 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-cee56e6b-b17d-4c86-a9ac-108da2869da5 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '22e264440fff43cdb7be6702839af37f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da2248fa190342f0bfe9211cae1e403b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:35:52.195466 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-cee56e6b-b17d-4c86-a9ac-108da2869da5 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1" status: 200 len: 1504 microversion: 2.1 time: 0.825253 Jul 16 21:35:52.203133 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2537/5077] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:35:51 2022] GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1 => generated 1504 bytes in 831 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:35:52.211483 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-22902aae-2c79-4b35-8dcb-52d30dcb0e0e req-858bb12c-a81f-4b5a-ac9c-ba1e0a0b978e service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "19df5c33-44d4-4de9-947b-4707b78af23a", "tag": "e518c0f5-c1c1-47b3-b1e8-743d59bf079a"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:35:52.223732 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-f1fc4b06-5f51-4476-b0d9-a8d61a095f95 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '726b46c54e9541fbb7825607bac3abe5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fd0ce287cb44cd59045868ce688909a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:35:52.226357 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-22902aae-2c79-4b35-8dcb-52d30dcb0e0e req-858bb12c-a81f-4b5a-ac9c-ba1e0a0b978e service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:52.227063 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-22902aae-2c79-4b35-8dcb-52d30dcb0e0e req-858bb12c-a81f-4b5a-ac9c-ba1e0a0b978e service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:52.235359 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-f1fc4b06-5f51-4476-b0d9-a8d61a095f95 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '726b46c54e9541fbb7825607bac3abe5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fd0ce287cb44cd59045868ce688909a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:35:52.237857 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-f1fc4b06-5f51-4476-b0d9-a8d61a095f95 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '726b46c54e9541fbb7825607bac3abe5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fd0ce287cb44cd59045868ce688909a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:35:52.245803 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-f1fc4b06-5f51-4476-b0d9-a8d61a095f95 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] 10.222.0.16 "GET /compute/v2.1/servers/19df5c33-44d4-4de9-947b-4707b78af23a" status: 200 len: 1874 microversion: 2.42 time: 0.631064 Jul 16 21:35:52.246602 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2541/5078] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:35:51 2022] GET /compute/v2.1/servers/19df5c33-44d4-4de9-947b-4707b78af23a => generated 1874 bytes in 633 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:35:52.252560 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-e3480bd3-14b2-4085-9f52-9c3f1c8620c4 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:35:52.255342 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-22902aae-2c79-4b35-8dcb-52d30dcb0e0e req-858bb12c-a81f-4b5a-ac9c-ba1e0a0b978e service nova] Creating event network-changed:e518c0f5-c1c1-47b3-b1e8-743d59bf079a for instance 19df5c33-44d4-4de9-947b-4707b78af23a on nch2-master-1 Jul 16 21:35:52.259755 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-e3480bd3-14b2-4085-9f52-9c3f1c8620c4 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: e4c7a9cb-e55d-4819-9885-2e72deffddb1] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:35:52.263881 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-22902aae-2c79-4b35-8dcb-52d30dcb0e0e req-858bb12c-a81f-4b5a-ac9c-ba1e0a0b978e service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.058842 Jul 16 21:35:52.264724 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2538/5079] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:35:52 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 61 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:35:52.273919 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e3480bd3-14b2-4085-9f52-9c3f1c8620c4 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:52.277698 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e3480bd3-14b2-4085-9f52-9c3f1c8620c4 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:52.278588 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e3480bd3-14b2-4085-9f52-9c3f1c8620c4 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:52.279071 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e3480bd3-14b2-4085-9f52-9c3f1c8620c4 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:52.426940 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-903aa44b-447f-45ce-884c-ddd059955c0d tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:35:52.430649 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-903aa44b-447f-45ce-884c-ddd059955c0d tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] [instance: 4df09776-dc3c-46ff-9e64-e20b5d4679fa] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:35:52.441222 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-903aa44b-447f-45ce-884c-ddd059955c0d tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:52.441940 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-903aa44b-447f-45ce-884c-ddd059955c0d tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:52.443263 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-903aa44b-447f-45ce-884c-ddd059955c0d tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:52.444744 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-903aa44b-447f-45ce-884c-ddd059955c0d tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:52.571633 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-e3480bd3-14b2-4085-9f52-9c3f1c8620c4 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '22e264440fff43cdb7be6702839af37f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da2248fa190342f0bfe9211cae1e403b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:35:52.582754 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-e3480bd3-14b2-4085-9f52-9c3f1c8620c4 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1" status: 200 len: 1504 microversion: 2.1 time: 0.332186 Jul 16 21:35:52.594746 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2542/5080] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:35:52 2022] GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1 => generated 1504 bytes in 346 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:35:52.602332 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-98d34ddd-751b-498c-a794-e4f234c23168 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Action: 'action', calling method: >, body: {"suspend": {}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:35:52.603200 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-98d34ddd-751b-498c-a794-e4f234c23168 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: e4c7a9cb-e55d-4819-9885-2e72deffddb1] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:35:52.610295 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-98d34ddd-751b-498c-a794-e4f234c23168 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:52.610955 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-98d34ddd-751b-498c-a794-e4f234c23168 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:52.611926 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-98d34ddd-751b-498c-a794-e4f234c23168 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:52.613953 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-98d34ddd-751b-498c-a794-e4f234c23168 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:52.647331 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=100 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:35:52.653207 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.objects.instance [None req-98d34ddd-751b-498c-a794-e4f234c23168 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lazy-loading 'flavor' on Instance uuid e4c7a9cb-e55d-4819-9885-2e72deffddb1 {{(pid=138681) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:35:52.669445 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-903aa44b-447f-45ce-884c-ddd059955c0d tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0ab8b3f06e994bbca5d6dd14109bfe50', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e60c0863dc6f4313bdc873fda27ae2fc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:35:52.687563 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-903aa44b-447f-45ce-884c-ddd059955c0d tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] 10.222.0.16 "GET /compute/v2.1/servers/4df09776-dc3c-46ff-9e64-e20b5d4679fa" status: 200 len: 1558 microversion: 2.1 time: 0.267367 Jul 16 21:35:52.688164 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2539/5081] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:35:52 2022] GET /compute/v2.1/servers/4df09776-dc3c-46ff-9e64-e20b5d4679fa => generated 1558 bytes in 274 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:35:52.709633 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-c5eb5781-d149-4efa-855a-3f1b061a3882 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:35:52.710636 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-c5eb5781-d149-4efa-855a-3f1b061a3882 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] [instance: 4df09776-dc3c-46ff-9e64-e20b5d4679fa] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:35:52.717316 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c5eb5781-d149-4efa-855a-3f1b061a3882 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:52.717903 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c5eb5781-d149-4efa-855a-3f1b061a3882 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:52.718829 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c5eb5781-d149-4efa-855a-3f1b061a3882 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:52.719521 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c5eb5781-d149-4efa-855a-3f1b061a3882 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:52.833850 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-c5eb5781-d149-4efa-855a-3f1b061a3882 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] 10.222.0.16 "GET /compute/v2.1/servers/4df09776-dc3c-46ff-9e64-e20b5d4679fa/os-interface" status: 200 len: 285 microversion: 2.1 time: 0.131410 Jul 16 21:35:52.834581 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2540/5082] 10.222.0.16 () {58 vars in 1274 bytes} [Sat Jul 16 21:35:52 2022] GET /compute/v2.1/servers/4df09776-dc3c-46ff-9e64-e20b5d4679fa/os-interface => generated 285 bytes in 133 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 16 21:35:53.025825 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-98d34ddd-751b-498c-a794-e4f234c23168 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "POST /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1/action" status: 202 len: 0 microversion: 2.1 time: 0.428946 Jul 16 21:35:53.026709 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2543/5083] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 21:35:52 2022] POST /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1/action => generated 0 bytes in 431 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 21:35:53.035771 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-f94dc80b-5ec3-49df-97f1-fc056700ce15 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:35:53.036606 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-f94dc80b-5ec3-49df-97f1-fc056700ce15 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: e4c7a9cb-e55d-4819-9885-2e72deffddb1] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:35:53.047507 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f94dc80b-5ec3-49df-97f1-fc056700ce15 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:53.049610 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f94dc80b-5ec3-49df-97f1-fc056700ce15 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:53.050866 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f94dc80b-5ec3-49df-97f1-fc056700ce15 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:53.052795 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f94dc80b-5ec3-49df-97f1-fc056700ce15 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:53.072819 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=100 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:35:53.226301 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-f94dc80b-5ec3-49df-97f1-fc056700ce15 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '22e264440fff43cdb7be6702839af37f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da2248fa190342f0bfe9211cae1e403b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:35:53.236527 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-f94dc80b-5ec3-49df-97f1-fc056700ce15 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1" status: 200 len: 1512 microversion: 2.1 time: 0.205311 Jul 16 21:35:53.237678 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2541/5084] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:35:53 2022] GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1 => generated 1512 bytes in 207 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:35:53.278393 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-3d52ccd9-03d1-4408-9dfa-d2568d5b544b tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:35:53.279404 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-3d52ccd9-03d1-4408-9dfa-d2568d5b544b tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] [instance: 19df5c33-44d4-4de9-947b-4707b78af23a] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:35:53.301146 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3d52ccd9-03d1-4408-9dfa-d2568d5b544b tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:53.305620 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3d52ccd9-03d1-4408-9dfa-d2568d5b544b tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:53.305620 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3d52ccd9-03d1-4408-9dfa-d2568d5b544b tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:53.305620 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3d52ccd9-03d1-4408-9dfa-d2568d5b544b tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:53.678785 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-3d52ccd9-03d1-4408-9dfa-d2568d5b544b tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '726b46c54e9541fbb7825607bac3abe5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fd0ce287cb44cd59045868ce688909a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:35:53.700077 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-3d52ccd9-03d1-4408-9dfa-d2568d5b544b tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '726b46c54e9541fbb7825607bac3abe5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fd0ce287cb44cd59045868ce688909a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:35:53.703448 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-3d52ccd9-03d1-4408-9dfa-d2568d5b544b tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '726b46c54e9541fbb7825607bac3abe5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fd0ce287cb44cd59045868ce688909a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:35:53.705030 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-3d52ccd9-03d1-4408-9dfa-d2568d5b544b tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] 10.222.0.16 "GET /compute/v2.1/servers/19df5c33-44d4-4de9-947b-4707b78af23a" status: 200 len: 1874 microversion: 2.42 time: 0.442786 Jul 16 21:35:53.715513 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2544/5085] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:35:53 2022] GET /compute/v2.1/servers/19df5c33-44d4-4de9-947b-4707b78af23a => generated 1874 bytes in 454 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:35:54.131071 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-6ff382fe-82a1-4777-802c-ee3900381def req-ab384098-c84d-49c1-baee-1c2fb2023e9a service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "4df09776-dc3c-46ff-9e64-e20b5d4679fa", "tag": "cc6f4c0a-93e4-4c67-8490-1bb34324c862"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:35:54.150770 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-6ff382fe-82a1-4777-802c-ee3900381def req-ab384098-c84d-49c1-baee-1c2fb2023e9a service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:54.151678 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-6ff382fe-82a1-4777-802c-ee3900381def req-ab384098-c84d-49c1-baee-1c2fb2023e9a service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:54.171786 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-6ff382fe-82a1-4777-802c-ee3900381def req-ab384098-c84d-49c1-baee-1c2fb2023e9a service nova] Creating event network-changed:cc6f4c0a-93e4-4c67-8490-1bb34324c862 for instance 4df09776-dc3c-46ff-9e64-e20b5d4679fa on nch1-master-1 Jul 16 21:35:54.178712 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-6ff382fe-82a1-4777-802c-ee3900381def req-ab384098-c84d-49c1-baee-1c2fb2023e9a service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.058805 Jul 16 21:35:54.179651 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2542/5086] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:35:54 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 61 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:35:54.256147 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-471d8fe8-b405-4cb6-a85a-ca8f4f5fba0d tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:35:54.256873 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-471d8fe8-b405-4cb6-a85a-ca8f4f5fba0d tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: e4c7a9cb-e55d-4819-9885-2e72deffddb1] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:35:54.277565 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-471d8fe8-b405-4cb6-a85a-ca8f4f5fba0d tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:54.278311 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-471d8fe8-b405-4cb6-a85a-ca8f4f5fba0d tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:54.280212 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-471d8fe8-b405-4cb6-a85a-ca8f4f5fba0d tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:54.282672 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-471d8fe8-b405-4cb6-a85a-ca8f4f5fba0d tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:54.640581 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-471d8fe8-b405-4cb6-a85a-ca8f4f5fba0d tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '22e264440fff43cdb7be6702839af37f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da2248fa190342f0bfe9211cae1e403b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:35:54.650829 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-471d8fe8-b405-4cb6-a85a-ca8f4f5fba0d tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1" status: 200 len: 1495 microversion: 2.1 time: 0.400415 Jul 16 21:35:54.652188 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2545/5087] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:35:54 2022] GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1 => generated 1495 bytes in 402 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:35:54.674219 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-ae7650d7-881d-4f7e-8da1-459ceb516930 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Action: 'action', calling method: >, body: {"suspend": {}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:35:54.675506 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-ae7650d7-881d-4f7e-8da1-459ceb516930 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: e4c7a9cb-e55d-4819-9885-2e72deffddb1] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:35:54.690072 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ae7650d7-881d-4f7e-8da1-459ceb516930 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:54.691370 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ae7650d7-881d-4f7e-8da1-459ceb516930 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:54.692191 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ae7650d7-881d-4f7e-8da1-459ceb516930 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:54.695467 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ae7650d7-881d-4f7e-8da1-459ceb516930 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:54.729051 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-0801b27e-4644-4aca-9a03-60ca15a62e4f tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:35:54.729674 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-0801b27e-4644-4aca-9a03-60ca15a62e4f tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] [instance: 19df5c33-44d4-4de9-947b-4707b78af23a] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:35:54.742159 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0801b27e-4644-4aca-9a03-60ca15a62e4f tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:54.742618 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0801b27e-4644-4aca-9a03-60ca15a62e4f tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:54.747629 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0801b27e-4644-4aca-9a03-60ca15a62e4f tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:54.749342 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0801b27e-4644-4aca-9a03-60ca15a62e4f tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:54.758439 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.objects.instance [None req-ae7650d7-881d-4f7e-8da1-459ceb516930 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lazy-loading 'flavor' on Instance uuid e4c7a9cb-e55d-4819-9885-2e72deffddb1 {{(pid=138680) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:35:54.797856 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-ae7650d7-881d-4f7e-8da1-459ceb516930 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] HTTP exception thrown: Cannot 'suspend' instance e4c7a9cb-e55d-4819-9885-2e72deffddb1 while it is in vm_state suspended Jul 16 21:35:54.804933 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-ae7650d7-881d-4f7e-8da1-459ceb516930 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Request method failure captured: Jul 16 21:35:54.804933 ncdv-master-1 devstack@n-api.service[138680]: request: POST /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1/action HTTP/1.1 Jul 16 21:35:54.804933 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:35:54.804933 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:35:54.804933 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:35:54.804933 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 15 Jul 16 21:35:54.804933 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:35:54.804933 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:35:54.804933 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:35:54.804933 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:35:54.804933 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0y6QJDVdPNcLbUnwwgOJV-QkddLQ7yqnzBNVp2CrOe5UQqjQOER3spl3PswEW82ISSx8FhmuIrEwFSG69Ze1pTOmWBWZ8PURIgVHbN3yBPA3U2S2TKkQkFwyNuvvV03MCr9U0QGJIjXVOkiqFEAy8q87U_okZgCdI-BFjBFgrCdfAEo Jul 16 21:35:54.804933 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:35:54.804933 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:35:54.804933 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:35:54.804933 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:35:54.804933 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:35:54.804933 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:35:54.804933 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: da2248fa190342f0bfe9211cae1e403b Jul 16 21:35:54.804933 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-ServersNegativeTestJSON-1102403437 Jul 16 21:35:54.804933 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:35:54.804933 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:35:54.804933 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/da2248fa190342f0bfe9211cae1e403b"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/da2248fa190342f0bfe9211cae1e403b"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_da2248fa190342f0bfe9211cae1e403b"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/da2248fa190342f0bfe9211cae1e403b"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:35:54.804933 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-ServersNegativeTestJSON-1102403437 Jul 16 21:35:54.804933 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: da2248fa190342f0bfe9211cae1e403b Jul 16 21:35:54.804933 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-ServersNegativeTestJSON-1102403437 Jul 16 21:35:54.804933 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-ServersNegativeTestJSON-1102403437-project Jul 16 21:35:54.804933 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:35:54.804933 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:35:54.804933 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: 22e264440fff43cdb7be6702839af37f Jul 16 21:35:54.804933 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-ServersNegativeTestJSON-1102403437-project Jul 16 21:35:54.804933 ncdv-master-1 devstack@n-api.service[138680]: Jul 16 21:35:54.804933 ncdv-master-1 devstack@n-api.service[138680]: {"suspend": {}} Jul 16 21:35:54.804933 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 21:35:54.804933 ncdv-master-1 devstack@n-api.service[138680]: exception: Cannot 'suspend' instance e4c7a9cb-e55d-4819-9885-2e72deffddb1 while it is in vm_state suspended Jul 16 21:35:54.804933 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': 'e4c7a9cb-e55d-4819-9885-2e72deffddb1', 'body': {'suspend': {}}} Jul 16 21:35:54.804933 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:35:54.804933 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:35:54.804933 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/suspend_server.py", line 40, in _suspend Jul 16 21:35:54.804933 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi self.compute_api.suspend(context, server) Jul 16 21:35:54.804933 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 360, in wrapper Jul 16 21:35:54.804933 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return func(self, context, instance, *args, **kwargs) Jul 16 21:35:54.804933 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 296, in inner Jul 16 21:35:54.804933 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Jul 16 21:35:54.804933 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 343, in wrapper Jul 16 21:35:54.804933 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return func(self, context, instance, *args, **kwargs) Jul 16 21:35:54.804933 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 256, in inner Jul 16 21:35:54.804933 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Jul 16 21:35:54.804933 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner Jul 16 21:35:54.804933 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Jul 16 21:35:54.804933 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 145, in inner Jul 16 21:35:54.804933 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.InstanceInvalidState( Jul 16 21:35:54.804933 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.InstanceInvalidState: Instance e4c7a9cb-e55d-4819-9885-2e72deffddb1 in vm_state suspended. Cannot suspend while the instance is in this state. Jul 16 21:35:54.804933 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:35:54.804933 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:35:54.804933 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:35:54.804933 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:35:54.804933 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:35:54.804933 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:35:54.804933 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:35:54.804933 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:35:54.804933 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:35:54.804933 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:35:54.804933 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/suspend_server.py", line 44, in _suspend Jul 16 21:35:54.804933 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi common.raise_http_conflict_for_instance_invalid_state(state_error, Jul 16 21:35:54.809223 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 347, in raise_http_conflict_for_instance_invalid_state Jul 16 21:35:54.809223 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPConflict(explanation=msg) Jul 16 21:35:54.809223 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPConflict: Cannot 'suspend' instance e4c7a9cb-e55d-4819-9885-2e72deffddb1 while it is in vm_state suspended Jul 16 21:35:54.809223 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:35:54.809223 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:35:54.809223 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:35:54.809223 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:35:54.809223 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:35:54.809223 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:35:54.809223 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:35:54.809223 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:35:54.809223 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Cannot 'suspend' instance e4c7a9cb-e55d-4819-9885-2e72deffddb1 while it is in vm_state suspended Jul 16 21:35:54.809223 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:35:54.810850 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-ae7650d7-881d-4f7e-8da1-459ceb516930 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Returning 409 to user: Cannot 'suspend' instance e4c7a9cb-e55d-4819-9885-2e72deffddb1 while it is in vm_state suspended {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:35:54.811923 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-ae7650d7-881d-4f7e-8da1-459ceb516930 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "POST /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1/action" status: 409 len: 148 microversion: 2.1 time: 0.142845 Jul 16 21:35:54.815257 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2543/5088] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 21:35:54 2022] POST /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1/action => generated 148 bytes in 147 msecs (HTTP/1.1 409) 9 headers in 378 bytes (1 switches on core 0) Jul 16 21:35:54.835234 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-a68a4b46-f6c0-4f16-b27e-d79b49017ce9 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Action: 'action', calling method: >, body: {"resume": {}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:35:54.836394 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-a68a4b46-f6c0-4f16-b27e-d79b49017ce9 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: e4c7a9cb-e55d-4819-9885-2e72deffddb1] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:35:54.849307 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a68a4b46-f6c0-4f16-b27e-d79b49017ce9 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:54.851316 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a68a4b46-f6c0-4f16-b27e-d79b49017ce9 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:54.852281 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a68a4b46-f6c0-4f16-b27e-d79b49017ce9 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:54.853036 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a68a4b46-f6c0-4f16-b27e-d79b49017ce9 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:55.125896 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-0801b27e-4644-4aca-9a03-60ca15a62e4f tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '726b46c54e9541fbb7825607bac3abe5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fd0ce287cb44cd59045868ce688909a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:35:55.143261 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-0801b27e-4644-4aca-9a03-60ca15a62e4f tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '726b46c54e9541fbb7825607bac3abe5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fd0ce287cb44cd59045868ce688909a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:35:55.148014 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-0801b27e-4644-4aca-9a03-60ca15a62e4f tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '726b46c54e9541fbb7825607bac3abe5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fd0ce287cb44cd59045868ce688909a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:35:55.149205 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-0801b27e-4644-4aca-9a03-60ca15a62e4f tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] 10.222.0.16 "GET /compute/v2.1/servers/19df5c33-44d4-4de9-947b-4707b78af23a" status: 200 len: 1874 microversion: 2.42 time: 0.427896 Jul 16 21:35:55.150918 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2546/5089] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:35:54 2022] GET /compute/v2.1/servers/19df5c33-44d4-4de9-947b-4707b78af23a => generated 1874 bytes in 430 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:35:55.228173 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-a68a4b46-f6c0-4f16-b27e-d79b49017ce9 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "POST /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1/action" status: 202 len: 0 microversion: 2.1 time: 0.406883 Jul 16 21:35:55.231395 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2544/5090] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 21:35:54 2022] POST /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1/action => generated 0 bytes in 412 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 21:35:55.244169 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-1b937e9f-0ffa-4db8-8f9d-d120bb9aad93 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:35:55.245305 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-1b937e9f-0ffa-4db8-8f9d-d120bb9aad93 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: e4c7a9cb-e55d-4819-9885-2e72deffddb1] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:35:55.259532 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1b937e9f-0ffa-4db8-8f9d-d120bb9aad93 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:55.260340 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1b937e9f-0ffa-4db8-8f9d-d120bb9aad93 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:55.261370 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1b937e9f-0ffa-4db8-8f9d-d120bb9aad93 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:55.262021 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1b937e9f-0ffa-4db8-8f9d-d120bb9aad93 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:55.604117 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-1b937e9f-0ffa-4db8-8f9d-d120bb9aad93 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '22e264440fff43cdb7be6702839af37f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da2248fa190342f0bfe9211cae1e403b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:35:55.613730 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-1b937e9f-0ffa-4db8-8f9d-d120bb9aad93 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1" status: 200 len: 1501 microversion: 2.1 time: 0.378355 Jul 16 21:35:55.617408 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2547/5091] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:35:55 2022] GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1 => generated 1501 bytes in 383 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:35:56.170196 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-d2d7f958-c82e-46c3-b66d-888f61ad94a1 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:35:56.170910 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-d2d7f958-c82e-46c3-b66d-888f61ad94a1 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] [instance: 19df5c33-44d4-4de9-947b-4707b78af23a] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:35:56.178167 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d2d7f958-c82e-46c3-b66d-888f61ad94a1 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:56.178626 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d2d7f958-c82e-46c3-b66d-888f61ad94a1 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:56.183459 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d2d7f958-c82e-46c3-b66d-888f61ad94a1 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:56.183459 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d2d7f958-c82e-46c3-b66d-888f61ad94a1 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:56.374853 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-d2d7f958-c82e-46c3-b66d-888f61ad94a1 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '726b46c54e9541fbb7825607bac3abe5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fd0ce287cb44cd59045868ce688909a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:35:56.384732 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-d2d7f958-c82e-46c3-b66d-888f61ad94a1 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '726b46c54e9541fbb7825607bac3abe5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fd0ce287cb44cd59045868ce688909a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:35:56.386223 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-d2d7f958-c82e-46c3-b66d-888f61ad94a1 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '726b46c54e9541fbb7825607bac3abe5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fd0ce287cb44cd59045868ce688909a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:35:56.387446 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-d2d7f958-c82e-46c3-b66d-888f61ad94a1 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] 10.222.0.16 "GET /compute/v2.1/servers/19df5c33-44d4-4de9-947b-4707b78af23a" status: 200 len: 1874 microversion: 2.42 time: 0.222445 Jul 16 21:35:56.388010 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2545/5092] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:35:56 2022] GET /compute/v2.1/servers/19df5c33-44d4-4de9-947b-4707b78af23a => generated 1874 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:35:56.637894 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-0a6db819-6a3c-4573-8a98-2049470cb1f9 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:35:56.638815 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-0a6db819-6a3c-4573-8a98-2049470cb1f9 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: e4c7a9cb-e55d-4819-9885-2e72deffddb1] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:35:56.641737 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_api:SELECT=81,nova_api:INSERT=3 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:35:56.647723 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0a6db819-6a3c-4573-8a98-2049470cb1f9 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:56.648180 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0a6db819-6a3c-4573-8a98-2049470cb1f9 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:56.649394 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0a6db819-6a3c-4573-8a98-2049470cb1f9 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:56.649840 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0a6db819-6a3c-4573-8a98-2049470cb1f9 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:56.896860 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-0a6db819-6a3c-4573-8a98-2049470cb1f9 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '22e264440fff43cdb7be6702839af37f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da2248fa190342f0bfe9211cae1e403b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:35:56.905580 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-0a6db819-6a3c-4573-8a98-2049470cb1f9 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1" status: 200 len: 1501 microversion: 2.1 time: 0.271089 Jul 16 21:35:56.906948 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2548/5093] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:35:56 2022] GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1 => generated 1501 bytes in 273 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:35:57.407286 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-37da3e48-e93b-4274-9486-5442312b4d73 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:35:57.408013 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-37da3e48-e93b-4274-9486-5442312b4d73 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] [instance: 19df5c33-44d4-4de9-947b-4707b78af23a] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:35:57.416217 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-37da3e48-e93b-4274-9486-5442312b4d73 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:57.416700 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-37da3e48-e93b-4274-9486-5442312b4d73 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:57.417473 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-37da3e48-e93b-4274-9486-5442312b4d73 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:57.417914 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-37da3e48-e93b-4274-9486-5442312b4d73 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:57.629263 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-37da3e48-e93b-4274-9486-5442312b4d73 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '726b46c54e9541fbb7825607bac3abe5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fd0ce287cb44cd59045868ce688909a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:35:57.638746 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-37da3e48-e93b-4274-9486-5442312b4d73 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '726b46c54e9541fbb7825607bac3abe5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fd0ce287cb44cd59045868ce688909a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:35:57.640325 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-37da3e48-e93b-4274-9486-5442312b4d73 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '726b46c54e9541fbb7825607bac3abe5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fd0ce287cb44cd59045868ce688909a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:35:57.641485 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-37da3e48-e93b-4274-9486-5442312b4d73 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] 10.222.0.16 "GET /compute/v2.1/servers/19df5c33-44d4-4de9-947b-4707b78af23a" status: 200 len: 1874 microversion: 2.42 time: 0.238468 Jul 16 21:35:57.642402 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2546/5094] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:35:57 2022] GET /compute/v2.1/servers/19df5c33-44d4-4de9-947b-4707b78af23a => generated 1874 bytes in 240 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:35:57.923065 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-be686021-172a-4f6b-b3ab-5062271c2aa1 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:35:57.923616 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-be686021-172a-4f6b-b3ab-5062271c2aa1 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: e4c7a9cb-e55d-4819-9885-2e72deffddb1] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:35:57.932198 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-be686021-172a-4f6b-b3ab-5062271c2aa1 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:57.932652 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-be686021-172a-4f6b-b3ab-5062271c2aa1 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:57.933743 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-be686021-172a-4f6b-b3ab-5062271c2aa1 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:57.934178 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-be686021-172a-4f6b-b3ab-5062271c2aa1 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:58.105824 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-be686021-172a-4f6b-b3ab-5062271c2aa1 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '22e264440fff43cdb7be6702839af37f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da2248fa190342f0bfe9211cae1e403b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:35:58.114607 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-be686021-172a-4f6b-b3ab-5062271c2aa1 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1" status: 200 len: 1504 microversion: 2.1 time: 0.195637 Jul 16 21:35:58.115214 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2549/5095] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:35:57 2022] GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1 => generated 1504 bytes in 197 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:35:58.133654 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-aef40659-465a-49ea-9eeb-55b756c890cf tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:35:58.134268 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-aef40659-465a-49ea-9eeb-55b756c890cf tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: e4c7a9cb-e55d-4819-9885-2e72deffddb1] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:35:58.140800 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-aef40659-465a-49ea-9eeb-55b756c890cf tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:58.141260 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-aef40659-465a-49ea-9eeb-55b756c890cf tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:58.144646 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-aef40659-465a-49ea-9eeb-55b756c890cf tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:58.144646 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-aef40659-465a-49ea-9eeb-55b756c890cf tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:58.295481 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-aef40659-465a-49ea-9eeb-55b756c890cf tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '22e264440fff43cdb7be6702839af37f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da2248fa190342f0bfe9211cae1e403b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:35:58.305431 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-aef40659-465a-49ea-9eeb-55b756c890cf tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1" status: 200 len: 1504 microversion: 2.1 time: 0.175594 Jul 16 21:35:58.306410 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2547/5096] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:35:58 2022] GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1 => generated 1504 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:35:58.331736 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-e91a62b2-7b06-463a-a66e-ab4604898ad4 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Action: 'action', calling method: >, body: {"unpause": {}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:35:58.331736 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-e91a62b2-7b06-463a-a66e-ab4604898ad4 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: 32065378-f193-4efc-b0c1-2c1e521ef083] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:35:58.331736 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-e91a62b2-7b06-463a-a66e-ab4604898ad4 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] HTTP exception thrown: Instance 32065378-f193-4efc-b0c1-2c1e521ef083 could not be found. Jul 16 21:35:58.331736 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-e91a62b2-7b06-463a-a66e-ab4604898ad4 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Request method failure captured: Jul 16 21:35:58.331736 ncdv-master-1 devstack@n-api.service[138681]: request: POST /compute/v2.1/servers/32065378-f193-4efc-b0c1-2c1e521ef083/action HTTP/1.1 Jul 16 21:35:58.331736 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:35:58.331736 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:35:58.331736 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:35:58.331736 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 15 Jul 16 21:35:58.331736 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:35:58.331736 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:35:58.331736 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:35:58.331736 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:35:58.331736 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0y6QJDVdPNcLbUnwwgOJV-QkddLQ7yqnzBNVp2CrOe5UQqjQOER3spl3PswEW82ISSx8FhmuIrEwFSG69Ze1pTOmWBWZ8PURIgVHbN3yBPA3U2S2TKkQkFwyNuvvV03MCr9U0QGJIjXVOkiqFEAy8q87U_okZgCdI-BFjBFgrCdfAEo Jul 16 21:35:58.331736 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:35:58.331736 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:35:58.331736 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:35:58.331736 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:35:58.331736 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:35:58.331736 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:35:58.331736 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: da2248fa190342f0bfe9211cae1e403b Jul 16 21:35:58.331736 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-ServersNegativeTestJSON-1102403437 Jul 16 21:35:58.331736 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:35:58.331736 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:35:58.331736 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/da2248fa190342f0bfe9211cae1e403b"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/da2248fa190342f0bfe9211cae1e403b"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_da2248fa190342f0bfe9211cae1e403b"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/da2248fa190342f0bfe9211cae1e403b"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:35:58.331736 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-ServersNegativeTestJSON-1102403437 Jul 16 21:35:58.331736 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: da2248fa190342f0bfe9211cae1e403b Jul 16 21:35:58.331736 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-ServersNegativeTestJSON-1102403437 Jul 16 21:35:58.331736 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-ServersNegativeTestJSON-1102403437-project Jul 16 21:35:58.331736 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:35:58.331736 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:35:58.331736 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: 22e264440fff43cdb7be6702839af37f Jul 16 21:35:58.331736 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-ServersNegativeTestJSON-1102403437-project Jul 16 21:35:58.331736 ncdv-master-1 devstack@n-api.service[138681]: Jul 16 21:35:58.331736 ncdv-master-1 devstack@n-api.service[138681]: {"unpause": {}} Jul 16 21:35:58.331736 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:35:58.331736 ncdv-master-1 devstack@n-api.service[138681]: exception: Instance 32065378-f193-4efc-b0c1-2c1e521ef083 could not be found. Jul 16 21:35:58.331736 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': '32065378-f193-4efc-b0c1-2c1e521ef083', 'body': {'unpause': {}}} Jul 16 21:35:58.331736 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:35:58.331736 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:35:58.331736 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:35:58.331736 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:35:58.331736 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:35:58.331736 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:35:58.331736 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2898, in _get_instance Jul 16 21:35:58.331736 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=instance_uuid) Jul 16 21:35:58.331736 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 32065378-f193-4efc-b0c1-2c1e521ef083 could not be found. Jul 16 21:35:58.331736 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:35:58.331736 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:35:58.331736 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:35:58.331736 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:35:58.331736 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:35:58.331736 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:35:58.331736 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:35:58.331736 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:35:58.331736 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:35:58.335354 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:35:58.335354 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/pause_server.py", line 58, in _unpause Jul 16 21:35:58.335354 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi server = common.get_instance(self.compute_api, ctxt, id) Jul 16 21:35:58.335354 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:35:58.335354 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:35:58.335354 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 32065378-f193-4efc-b0c1-2c1e521ef083 could not be found. Jul 16 21:35:58.335354 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:35:58.335354 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:35:58.335354 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:35:58.335354 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:35:58.335354 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:35:58.335354 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:35:58.335354 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:35:58.335354 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:35:58.335354 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 32065378-f193-4efc-b0c1-2c1e521ef083 could not be found. Jul 16 21:35:58.335354 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:35:58.335354 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-e91a62b2-7b06-463a-a66e-ab4604898ad4 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Returning 404 to user: Instance 32065378-f193-4efc-b0c1-2c1e521ef083 could not be found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:35:58.335354 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-e91a62b2-7b06-463a-a66e-ab4604898ad4 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "POST /compute/v2.1/servers/32065378-f193-4efc-b0c1-2c1e521ef083/action" status: 404 len: 111 microversion: 2.1 time: 0.014269 Jul 16 21:35:58.335354 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2550/5097] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 21:35:58 2022] POST /compute/v2.1/servers/32065378-f193-4efc-b0c1-2c1e521ef083/action => generated 111 bytes in 16 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:35:58.341969 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-0c4ec26c-a93c-4833-a9e8-283009260fce tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:35:58.342626 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-0c4ec26c-a93c-4833-a9e8-283009260fce tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: e4c7a9cb-e55d-4819-9885-2e72deffddb1] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:35:58.356707 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0c4ec26c-a93c-4833-a9e8-283009260fce tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:58.356945 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0c4ec26c-a93c-4833-a9e8-283009260fce tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:58.357919 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0c4ec26c-a93c-4833-a9e8-283009260fce tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:58.358487 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0c4ec26c-a93c-4833-a9e8-283009260fce tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:58.511092 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-0c4ec26c-a93c-4833-a9e8-283009260fce tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '22e264440fff43cdb7be6702839af37f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da2248fa190342f0bfe9211cae1e403b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:35:58.519579 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-0c4ec26c-a93c-4833-a9e8-283009260fce tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1" status: 200 len: 1504 microversion: 2.1 time: 0.181541 Jul 16 21:35:58.520144 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2548/5098] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:35:58 2022] GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1 => generated 1504 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:35:58.539243 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-da3a65e6-016f-489e-8d0f-f72f6652c3fd tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:35:58.540013 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-da3a65e6-016f-489e-8d0f-f72f6652c3fd tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: e4c7a9cb-e55d-4819-9885-2e72deffddb1] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:35:58.546911 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-da3a65e6-016f-489e-8d0f-f72f6652c3fd tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:58.547403 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-da3a65e6-016f-489e-8d0f-f72f6652c3fd tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:58.548178 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-da3a65e6-016f-489e-8d0f-f72f6652c3fd tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:58.548627 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-da3a65e6-016f-489e-8d0f-f72f6652c3fd tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:58.662288 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-511b4f72-b214-4da7-92af-b7b3ed2b8d5d tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:35:58.662908 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-511b4f72-b214-4da7-92af-b7b3ed2b8d5d tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] [instance: 19df5c33-44d4-4de9-947b-4707b78af23a] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:35:58.669472 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-511b4f72-b214-4da7-92af-b7b3ed2b8d5d tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:58.670016 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-511b4f72-b214-4da7-92af-b7b3ed2b8d5d tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:58.670873 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-511b4f72-b214-4da7-92af-b7b3ed2b8d5d tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:58.671689 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-511b4f72-b214-4da7-92af-b7b3ed2b8d5d tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:58.690394 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-da3a65e6-016f-489e-8d0f-f72f6652c3fd tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '22e264440fff43cdb7be6702839af37f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da2248fa190342f0bfe9211cae1e403b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:35:58.698690 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-da3a65e6-016f-489e-8d0f-f72f6652c3fd tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1" status: 200 len: 1504 microversion: 2.1 time: 0.163197 Jul 16 21:35:58.700088 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2551/5099] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:35:58 2022] GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1 => generated 1504 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:35:58.722048 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-05a4074b-54d2-4df4-af57-e7b12c6a5182 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Action: 'action', calling method: >, body: {"unpause": {}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:35:58.728013 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-05a4074b-54d2-4df4-af57-e7b12c6a5182 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: e4c7a9cb-e55d-4819-9885-2e72deffddb1] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:35:58.729113 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-05a4074b-54d2-4df4-af57-e7b12c6a5182 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:58.729572 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-05a4074b-54d2-4df4-af57-e7b12c6a5182 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:58.730611 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-05a4074b-54d2-4df4-af57-e7b12c6a5182 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:58.731083 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-05a4074b-54d2-4df4-af57-e7b12c6a5182 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:58.760103 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-05a4074b-54d2-4df4-af57-e7b12c6a5182 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] HTTP exception thrown: Cannot 'unpause' instance e4c7a9cb-e55d-4819-9885-2e72deffddb1 while it is in vm_state active Jul 16 21:35:58.761959 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-05a4074b-54d2-4df4-af57-e7b12c6a5182 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Request method failure captured: Jul 16 21:35:58.761959 ncdv-master-1 devstack@n-api.service[138681]: request: POST /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1/action HTTP/1.1 Jul 16 21:35:58.761959 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:35:58.761959 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:35:58.761959 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:35:58.761959 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 15 Jul 16 21:35:58.761959 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:35:58.761959 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:35:58.761959 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:35:58.761959 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:35:58.761959 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0y6QJDVdPNcLbUnwwgOJV-QkddLQ7yqnzBNVp2CrOe5UQqjQOER3spl3PswEW82ISSx8FhmuIrEwFSG69Ze1pTOmWBWZ8PURIgVHbN3yBPA3U2S2TKkQkFwyNuvvV03MCr9U0QGJIjXVOkiqFEAy8q87U_okZgCdI-BFjBFgrCdfAEo Jul 16 21:35:58.761959 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:35:58.761959 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:35:58.761959 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:35:58.761959 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:35:58.761959 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:35:58.761959 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:35:58.761959 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: da2248fa190342f0bfe9211cae1e403b Jul 16 21:35:58.761959 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-ServersNegativeTestJSON-1102403437 Jul 16 21:35:58.761959 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:35:58.761959 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:35:58.761959 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/da2248fa190342f0bfe9211cae1e403b"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/da2248fa190342f0bfe9211cae1e403b"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_da2248fa190342f0bfe9211cae1e403b"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/da2248fa190342f0bfe9211cae1e403b"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:35:58.761959 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-ServersNegativeTestJSON-1102403437 Jul 16 21:35:58.761959 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: da2248fa190342f0bfe9211cae1e403b Jul 16 21:35:58.761959 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-ServersNegativeTestJSON-1102403437 Jul 16 21:35:58.761959 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-ServersNegativeTestJSON-1102403437-project Jul 16 21:35:58.761959 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:35:58.761959 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:35:58.761959 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: 22e264440fff43cdb7be6702839af37f Jul 16 21:35:58.761959 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-ServersNegativeTestJSON-1102403437-project Jul 16 21:35:58.761959 ncdv-master-1 devstack@n-api.service[138681]: Jul 16 21:35:58.761959 ncdv-master-1 devstack@n-api.service[138681]: {"unpause": {}} Jul 16 21:35:58.761959 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:35:58.761959 ncdv-master-1 devstack@n-api.service[138681]: exception: Cannot 'unpause' instance e4c7a9cb-e55d-4819-9885-2e72deffddb1 while it is in vm_state active Jul 16 21:35:58.761959 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': 'e4c7a9cb-e55d-4819-9885-2e72deffddb1', 'body': {'unpause': {}}} Jul 16 21:35:58.761959 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:35:58.761959 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:35:58.761959 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/pause_server.py", line 62, in _unpause Jul 16 21:35:58.761959 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi self.compute_api.unpause(ctxt, server) Jul 16 21:35:58.761959 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner Jul 16 21:35:58.761959 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Jul 16 21:35:58.761959 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 145, in inner Jul 16 21:35:58.761959 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.InstanceInvalidState( Jul 16 21:35:58.761959 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.InstanceInvalidState: Instance e4c7a9cb-e55d-4819-9885-2e72deffddb1 in vm_state active. Cannot unpause while the instance is in this state. Jul 16 21:35:58.761959 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:35:58.761959 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:35:58.761959 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:35:58.761959 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:35:58.761959 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:35:58.761959 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:35:58.761959 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:35:58.761959 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:35:58.761959 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:35:58.761959 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:35:58.761959 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/pause_server.py", line 66, in _unpause Jul 16 21:35:58.761959 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi common.raise_http_conflict_for_instance_invalid_state(state_error, Jul 16 21:35:58.761959 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 347, in raise_http_conflict_for_instance_invalid_state Jul 16 21:35:58.761959 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPConflict(explanation=msg) Jul 16 21:35:58.761959 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPConflict: Cannot 'unpause' instance e4c7a9cb-e55d-4819-9885-2e72deffddb1 while it is in vm_state active Jul 16 21:35:58.761959 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:35:58.761959 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:35:58.761959 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:35:58.761959 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:35:58.761959 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:35:58.765537 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:35:58.765537 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:35:58.765537 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:35:58.765537 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Cannot 'unpause' instance e4c7a9cb-e55d-4819-9885-2e72deffddb1 while it is in vm_state active Jul 16 21:35:58.765537 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:35:58.765537 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-05a4074b-54d2-4df4-af57-e7b12c6a5182 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Returning 409 to user: Cannot 'unpause' instance e4c7a9cb-e55d-4819-9885-2e72deffddb1 while it is in vm_state active {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:35:58.765537 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-05a4074b-54d2-4df4-af57-e7b12c6a5182 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "POST /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1/action" status: 409 len: 145 microversion: 2.1 time: 0.046157 Jul 16 21:35:58.765537 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2552/5100] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 21:35:58 2022] POST /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1/action => generated 145 bytes in 48 msecs (HTTP/1.1 409) 9 headers in 378 bytes (2 switches on core 0) Jul 16 21:35:58.772704 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-a818b0fc-8a88-462b-82b6-7d62b86f376c tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:35:58.773322 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-a818b0fc-8a88-462b-82b6-7d62b86f376c tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: e4c7a9cb-e55d-4819-9885-2e72deffddb1] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:35:58.779555 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a818b0fc-8a88-462b-82b6-7d62b86f376c tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:58.780105 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a818b0fc-8a88-462b-82b6-7d62b86f376c tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:58.780806 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a818b0fc-8a88-462b-82b6-7d62b86f376c tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:58.781195 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a818b0fc-8a88-462b-82b6-7d62b86f376c tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:58.938992 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-511b4f72-b214-4da7-92af-b7b3ed2b8d5d tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '726b46c54e9541fbb7825607bac3abe5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fd0ce287cb44cd59045868ce688909a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:35:58.961946 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-511b4f72-b214-4da7-92af-b7b3ed2b8d5d tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '726b46c54e9541fbb7825607bac3abe5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fd0ce287cb44cd59045868ce688909a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:35:58.963478 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-511b4f72-b214-4da7-92af-b7b3ed2b8d5d tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '726b46c54e9541fbb7825607bac3abe5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fd0ce287cb44cd59045868ce688909a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:35:58.964653 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-511b4f72-b214-4da7-92af-b7b3ed2b8d5d tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] 10.222.0.16 "GET /compute/v2.1/servers/19df5c33-44d4-4de9-947b-4707b78af23a" status: 200 len: 1874 microversion: 2.42 time: 0.307238 Jul 16 21:35:58.976647 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2549/5101] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:35:58 2022] GET /compute/v2.1/servers/19df5c33-44d4-4de9-947b-4707b78af23a => generated 1874 bytes in 308 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 16 21:35:58.976929 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-a818b0fc-8a88-462b-82b6-7d62b86f376c tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '22e264440fff43cdb7be6702839af37f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da2248fa190342f0bfe9211cae1e403b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:35:58.981942 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-a818b0fc-8a88-462b-82b6-7d62b86f376c tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1" status: 200 len: 1504 microversion: 2.1 time: 0.213273 Jul 16 21:35:58.982528 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2553/5102] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:35:58 2022] GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1 => generated 1504 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:35:59.001664 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-cb3ffc84-5dbc-49c0-917f-0d6165da74b1 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:35:59.001664 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-cb3ffc84-5dbc-49c0-917f-0d6165da74b1 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: e4c7a9cb-e55d-4819-9885-2e72deffddb1] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:35:59.006720 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cb3ffc84-5dbc-49c0-917f-0d6165da74b1 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:59.007127 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cb3ffc84-5dbc-49c0-917f-0d6165da74b1 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:59.008135 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cb3ffc84-5dbc-49c0-917f-0d6165da74b1 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:59.008851 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cb3ffc84-5dbc-49c0-917f-0d6165da74b1 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:59.165039 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-cb3ffc84-5dbc-49c0-917f-0d6165da74b1 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '22e264440fff43cdb7be6702839af37f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da2248fa190342f0bfe9211cae1e403b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:35:59.174055 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-cb3ffc84-5dbc-49c0-917f-0d6165da74b1 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1" status: 200 len: 1504 microversion: 2.1 time: 0.178926 Jul 16 21:35:59.175144 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2550/5103] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:35:58 2022] GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1 => generated 1504 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:35:59.193913 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-bef67986-baf0-4f53-94d9-14cdaedddf01 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Action: 'action', calling method: >, body: {"unshelve": null} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:35:59.194496 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-bef67986-baf0-4f53-94d9-14cdaedddf01 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: dba32b20-61fd-425f-8159-aaf2de9ddd23] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:35:59.202000 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-bef67986-baf0-4f53-94d9-14cdaedddf01 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] HTTP exception thrown: Instance dba32b20-61fd-425f-8159-aaf2de9ddd23 could not be found. Jul 16 21:35:59.203892 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-bef67986-baf0-4f53-94d9-14cdaedddf01 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Request method failure captured: Jul 16 21:35:59.203892 ncdv-master-1 devstack@n-api.service[138681]: request: POST /compute/v2.1/servers/dba32b20-61fd-425f-8159-aaf2de9ddd23/action HTTP/1.1 Jul 16 21:35:59.203892 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:35:59.203892 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:35:59.203892 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:35:59.203892 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 18 Jul 16 21:35:59.203892 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:35:59.203892 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:35:59.203892 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:35:59.203892 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:35:59.203892 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0y6QJDVdPNcLbUnwwgOJV-QkddLQ7yqnzBNVp2CrOe5UQqjQOER3spl3PswEW82ISSx8FhmuIrEwFSG69Ze1pTOmWBWZ8PURIgVHbN3yBPA3U2S2TKkQkFwyNuvvV03MCr9U0QGJIjXVOkiqFEAy8q87U_okZgCdI-BFjBFgrCdfAEo Jul 16 21:35:59.203892 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:35:59.203892 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:35:59.203892 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:35:59.203892 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:35:59.203892 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:35:59.203892 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:35:59.203892 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: da2248fa190342f0bfe9211cae1e403b Jul 16 21:35:59.203892 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-ServersNegativeTestJSON-1102403437 Jul 16 21:35:59.203892 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:35:59.203892 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:35:59.203892 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/da2248fa190342f0bfe9211cae1e403b"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/da2248fa190342f0bfe9211cae1e403b"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_da2248fa190342f0bfe9211cae1e403b"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/da2248fa190342f0bfe9211cae1e403b"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:35:59.203892 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-ServersNegativeTestJSON-1102403437 Jul 16 21:35:59.203892 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: da2248fa190342f0bfe9211cae1e403b Jul 16 21:35:59.203892 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-ServersNegativeTestJSON-1102403437 Jul 16 21:35:59.203892 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-ServersNegativeTestJSON-1102403437-project Jul 16 21:35:59.203892 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:35:59.203892 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:35:59.203892 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: 22e264440fff43cdb7be6702839af37f Jul 16 21:35:59.203892 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-ServersNegativeTestJSON-1102403437-project Jul 16 21:35:59.203892 ncdv-master-1 devstack@n-api.service[138681]: Jul 16 21:35:59.203892 ncdv-master-1 devstack@n-api.service[138681]: {"unshelve": null} Jul 16 21:35:59.203892 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:35:59.203892 ncdv-master-1 devstack@n-api.service[138681]: exception: Instance dba32b20-61fd-425f-8159-aaf2de9ddd23 could not be found. Jul 16 21:35:59.203892 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': 'dba32b20-61fd-425f-8159-aaf2de9ddd23', 'body': {'unshelve': None}} Jul 16 21:35:59.203892 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:35:59.203892 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:35:59.203892 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:35:59.203892 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:35:59.203892 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:35:59.203892 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:35:59.203892 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2898, in _get_instance Jul 16 21:35:59.203892 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=instance_uuid) Jul 16 21:35:59.203892 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance dba32b20-61fd-425f-8159-aaf2de9ddd23 could not be found. Jul 16 21:35:59.203892 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:35:59.203892 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:35:59.203892 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:35:59.203892 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:35:59.203892 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:35:59.203892 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:35:59.203892 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:35:59.203892 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:35:59.203892 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:35:59.203892 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:35:59.203892 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 16 21:35:59.203892 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 16 21:35:59.203892 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/shelve.py", line 94, in _unshelve Jul 16 21:35:59.203892 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, id) Jul 16 21:35:59.203892 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:35:59.203892 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:35:59.203892 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance dba32b20-61fd-425f-8159-aaf2de9ddd23 could not be found. Jul 16 21:35:59.203892 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:35:59.203892 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:35:59.203892 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:35:59.203892 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:35:59.209466 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:35:59.209466 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:35:59.209466 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:35:59.209466 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:35:59.209466 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance dba32b20-61fd-425f-8159-aaf2de9ddd23 could not be found. Jul 16 21:35:59.209466 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:35:59.209466 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-bef67986-baf0-4f53-94d9-14cdaedddf01 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Returning 404 to user: Instance dba32b20-61fd-425f-8159-aaf2de9ddd23 could not be found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:35:59.210603 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-bef67986-baf0-4f53-94d9-14cdaedddf01 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "POST /compute/v2.1/servers/dba32b20-61fd-425f-8159-aaf2de9ddd23/action" status: 404 len: 111 microversion: 2.1 time: 0.021010 Jul 16 21:35:59.211262 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2554/5104] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 21:35:59 2022] POST /compute/v2.1/servers/dba32b20-61fd-425f-8159-aaf2de9ddd23/action => generated 111 bytes in 23 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:35:59.219603 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-b888668c-4b5b-448a-81fa-a01e22039a9d tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:35:59.220233 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-b888668c-4b5b-448a-81fa-a01e22039a9d tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: e4c7a9cb-e55d-4819-9885-2e72deffddb1] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:35:59.226693 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b888668c-4b5b-448a-81fa-a01e22039a9d tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:59.227203 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b888668c-4b5b-448a-81fa-a01e22039a9d tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:59.227981 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b888668c-4b5b-448a-81fa-a01e22039a9d tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:59.228425 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b888668c-4b5b-448a-81fa-a01e22039a9d tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:59.371450 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-b888668c-4b5b-448a-81fa-a01e22039a9d tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '22e264440fff43cdb7be6702839af37f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da2248fa190342f0bfe9211cae1e403b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:35:59.381997 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-b888668c-4b5b-448a-81fa-a01e22039a9d tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1" status: 200 len: 1504 microversion: 2.1 time: 0.166327 Jul 16 21:35:59.382468 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2551/5105] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:35:59 2022] GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1 => generated 1504 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:35:59.399558 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-b1ff85fc-39c7-4d50-a325-ddf934ab1099 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:35:59.400181 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-b1ff85fc-39c7-4d50-a325-ddf934ab1099 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: e4c7a9cb-e55d-4819-9885-2e72deffddb1] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:35:59.407434 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b1ff85fc-39c7-4d50-a325-ddf934ab1099 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:59.407896 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b1ff85fc-39c7-4d50-a325-ddf934ab1099 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:59.408674 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b1ff85fc-39c7-4d50-a325-ddf934ab1099 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:59.409343 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b1ff85fc-39c7-4d50-a325-ddf934ab1099 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:59.557826 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-b1ff85fc-39c7-4d50-a325-ddf934ab1099 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '22e264440fff43cdb7be6702839af37f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da2248fa190342f0bfe9211cae1e403b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:35:59.566457 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-b1ff85fc-39c7-4d50-a325-ddf934ab1099 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1" status: 200 len: 1504 microversion: 2.1 time: 0.170522 Jul 16 21:35:59.567319 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2555/5106] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:35:59 2022] GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1 => generated 1504 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:35:59.584010 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-55476964-bf47-4239-8fd6-a5e592a17ed1 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Action: 'action', calling method: >, body: {"unshelve": null} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:35:59.584592 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-55476964-bf47-4239-8fd6-a5e592a17ed1 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: e4c7a9cb-e55d-4819-9885-2e72deffddb1] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:35:59.591662 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-55476964-bf47-4239-8fd6-a5e592a17ed1 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:59.592122 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-55476964-bf47-4239-8fd6-a5e592a17ed1 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:59.592897 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-55476964-bf47-4239-8fd6-a5e592a17ed1 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:59.593346 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-55476964-bf47-4239-8fd6-a5e592a17ed1 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:59.619158 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-55476964-bf47-4239-8fd6-a5e592a17ed1 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] HTTP exception thrown: Cannot 'unshelve' instance e4c7a9cb-e55d-4819-9885-2e72deffddb1 while it is in vm_state active Jul 16 21:35:59.621044 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-55476964-bf47-4239-8fd6-a5e592a17ed1 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Request method failure captured: Jul 16 21:35:59.621044 ncdv-master-1 devstack@n-api.service[138680]: request: POST /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1/action HTTP/1.1 Jul 16 21:35:59.621044 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:35:59.621044 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:35:59.621044 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:35:59.621044 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 18 Jul 16 21:35:59.621044 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:35:59.621044 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:35:59.621044 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:35:59.621044 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:35:59.621044 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0y6QJDVdPNcLbUnwwgOJV-QkddLQ7yqnzBNVp2CrOe5UQqjQOER3spl3PswEW82ISSx8FhmuIrEwFSG69Ze1pTOmWBWZ8PURIgVHbN3yBPA3U2S2TKkQkFwyNuvvV03MCr9U0QGJIjXVOkiqFEAy8q87U_okZgCdI-BFjBFgrCdfAEo Jul 16 21:35:59.621044 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:35:59.621044 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:35:59.621044 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:35:59.621044 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:35:59.621044 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:35:59.621044 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:35:59.621044 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: da2248fa190342f0bfe9211cae1e403b Jul 16 21:35:59.621044 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-ServersNegativeTestJSON-1102403437 Jul 16 21:35:59.621044 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:35:59.621044 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:35:59.621044 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/da2248fa190342f0bfe9211cae1e403b"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/da2248fa190342f0bfe9211cae1e403b"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_da2248fa190342f0bfe9211cae1e403b"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/da2248fa190342f0bfe9211cae1e403b"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:35:59.621044 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-ServersNegativeTestJSON-1102403437 Jul 16 21:35:59.621044 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: da2248fa190342f0bfe9211cae1e403b Jul 16 21:35:59.621044 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-ServersNegativeTestJSON-1102403437 Jul 16 21:35:59.621044 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-ServersNegativeTestJSON-1102403437-project Jul 16 21:35:59.621044 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:35:59.621044 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:35:59.621044 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: 22e264440fff43cdb7be6702839af37f Jul 16 21:35:59.621044 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-ServersNegativeTestJSON-1102403437-project Jul 16 21:35:59.621044 ncdv-master-1 devstack@n-api.service[138680]: Jul 16 21:35:59.621044 ncdv-master-1 devstack@n-api.service[138680]: {"unshelve": null} Jul 16 21:35:59.621044 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 21:35:59.621044 ncdv-master-1 devstack@n-api.service[138680]: exception: Cannot 'unshelve' instance e4c7a9cb-e55d-4819-9885-2e72deffddb1 while it is in vm_state active Jul 16 21:35:59.621044 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': 'e4c7a9cb-e55d-4819-9885-2e72deffddb1', 'body': {'unshelve': None}} Jul 16 21:35:59.621044 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:35:59.621044 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:35:59.621044 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/shelve.py", line 105, in _unshelve Jul 16 21:35:59.621044 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi self.compute_api.unshelve(context, instance, new_az=new_az) Jul 16 21:35:59.621044 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 375, in inner Jul 16 21:35:59.621044 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Jul 16 21:35:59.621044 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner Jul 16 21:35:59.621044 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Jul 16 21:35:59.621044 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 145, in inner Jul 16 21:35:59.621044 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.InstanceInvalidState( Jul 16 21:35:59.621044 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.InstanceInvalidState: Instance e4c7a9cb-e55d-4819-9885-2e72deffddb1 in vm_state active. Cannot unshelve while the instance is in this state. Jul 16 21:35:59.621044 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:35:59.621044 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:35:59.621044 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:35:59.621044 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:35:59.621044 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:35:59.621044 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:35:59.621044 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:35:59.621044 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:35:59.621044 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:35:59.621044 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:35:59.621044 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 16 21:35:59.621044 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 16 21:35:59.621044 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/shelve.py", line 111, in _unshelve Jul 16 21:35:59.621044 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi common.raise_http_conflict_for_instance_invalid_state(state_error, Jul 16 21:35:59.621044 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 347, in raise_http_conflict_for_instance_invalid_state Jul 16 21:35:59.621044 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPConflict(explanation=msg) Jul 16 21:35:59.621044 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPConflict: Cannot 'unshelve' instance e4c7a9cb-e55d-4819-9885-2e72deffddb1 while it is in vm_state active Jul 16 21:35:59.625725 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:35:59.625725 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:35:59.625725 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:35:59.625725 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:35:59.625725 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:35:59.625725 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:35:59.625725 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:35:59.625725 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:35:59.625725 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Cannot 'unshelve' instance e4c7a9cb-e55d-4819-9885-2e72deffddb1 while it is in vm_state active Jul 16 21:35:59.625725 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:35:59.625725 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-55476964-bf47-4239-8fd6-a5e592a17ed1 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Returning 409 to user: Cannot 'unshelve' instance e4c7a9cb-e55d-4819-9885-2e72deffddb1 while it is in vm_state active {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:35:59.625725 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-55476964-bf47-4239-8fd6-a5e592a17ed1 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "POST /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1/action" status: 409 len: 146 microversion: 2.1 time: 0.043385 Jul 16 21:35:59.625725 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2552/5107] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 21:35:59 2022] POST /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1/action => generated 146 bytes in 45 msecs (HTTP/1.1 409) 9 headers in 378 bytes (1 switches on core 0) Jul 16 21:35:59.633883 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-8d220444-0a41-4e75-b521-1abbac8be6f9 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:35:59.637749 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-8d220444-0a41-4e75-b521-1abbac8be6f9 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: e4c7a9cb-e55d-4819-9885-2e72deffddb1] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:35:59.641120 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8d220444-0a41-4e75-b521-1abbac8be6f9 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:59.641577 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8d220444-0a41-4e75-b521-1abbac8be6f9 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:59.642348 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8d220444-0a41-4e75-b521-1abbac8be6f9 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:59.642792 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8d220444-0a41-4e75-b521-1abbac8be6f9 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:59.802681 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-8d220444-0a41-4e75-b521-1abbac8be6f9 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '22e264440fff43cdb7be6702839af37f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da2248fa190342f0bfe9211cae1e403b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:35:59.811618 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-8d220444-0a41-4e75-b521-1abbac8be6f9 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1" status: 200 len: 1504 microversion: 2.1 time: 0.181537 Jul 16 21:35:59.824168 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2556/5108] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:35:59 2022] GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1 => generated 1504 bytes in 191 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:35:59.833097 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-49802ede-193c-4d50-83cf-b25e6e6b13a2 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:35:59.833922 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-49802ede-193c-4d50-83cf-b25e6e6b13a2 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: e4c7a9cb-e55d-4819-9885-2e72deffddb1] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:35:59.842377 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-49802ede-193c-4d50-83cf-b25e6e6b13a2 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:59.842833 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-49802ede-193c-4d50-83cf-b25e6e6b13a2 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:59.843834 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-49802ede-193c-4d50-83cf-b25e6e6b13a2 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:59.844652 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-49802ede-193c-4d50-83cf-b25e6e6b13a2 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:59.984740 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-04639184-aa73-49ec-bb6d-89c83815a07b tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:35:59.985770 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-04639184-aa73-49ec-bb6d-89c83815a07b tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] [instance: 19df5c33-44d4-4de9-947b-4707b78af23a] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:35:59.993185 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-04639184-aa73-49ec-bb6d-89c83815a07b tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:59.993648 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-04639184-aa73-49ec-bb6d-89c83815a07b tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:35:59.994431 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-04639184-aa73-49ec-bb6d-89c83815a07b tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:35:59.994821 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-49802ede-193c-4d50-83cf-b25e6e6b13a2 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '22e264440fff43cdb7be6702839af37f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da2248fa190342f0bfe9211cae1e403b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:35:59.994929 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-04639184-aa73-49ec-bb6d-89c83815a07b tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:36:00.004378 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-49802ede-193c-4d50-83cf-b25e6e6b13a2 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1" status: 200 len: 1504 microversion: 2.1 time: 0.175694 Jul 16 21:36:00.004966 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2553/5109] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:35:59 2022] GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1 => generated 1504 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:36:00.032075 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-9da37770-0b0d-42df-a36f-f45965a5dc49 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Action: 'update', calling method: >, body: {"server": {"name": "tempest-ServersNegativeTestJSON-server-657933810_updated"}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:36:00.033347 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-9da37770-0b0d-42df-a36f-f45965a5dc49 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: 4f82f9e6-5941-4188-91e3-6aea6e2a3ec8] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:36:00.039823 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-9da37770-0b0d-42df-a36f-f45965a5dc49 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] HTTP exception thrown: Instance 4f82f9e6-5941-4188-91e3-6aea6e2a3ec8 could not be found. Jul 16 21:36:00.041525 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-9da37770-0b0d-42df-a36f-f45965a5dc49 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Request method failure captured: Jul 16 21:36:00.041525 ncdv-master-1 devstack@n-api.service[138680]: request: PUT /compute/v2.1/servers/4f82f9e6-5941-4188-91e3-6aea6e2a3ec8 HTTP/1.1 Jul 16 21:36:00.041525 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:36:00.041525 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:36:00.041525 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:36:00.041525 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 80 Jul 16 21:36:00.041525 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:36:00.041525 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:36:00.041525 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:36:00.041525 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:36:00.041525 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0y6QJDVdPNcLbUnwwgOJV-QkddLQ7yqnzBNVp2CrOe5UQqjQOER3spl3PswEW82ISSx8FhmuIrEwFSG69Ze1pTOmWBWZ8PURIgVHbN3yBPA3U2S2TKkQkFwyNuvvV03MCr9U0QGJIjXVOkiqFEAy8q87U_okZgCdI-BFjBFgrCdfAEo Jul 16 21:36:00.041525 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:36:00.041525 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:36:00.041525 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:36:00.041525 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:36:00.041525 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:36:00.041525 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:36:00.041525 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: da2248fa190342f0bfe9211cae1e403b Jul 16 21:36:00.041525 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-ServersNegativeTestJSON-1102403437 Jul 16 21:36:00.041525 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:36:00.041525 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:36:00.041525 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/da2248fa190342f0bfe9211cae1e403b"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/da2248fa190342f0bfe9211cae1e403b"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_da2248fa190342f0bfe9211cae1e403b"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/da2248fa190342f0bfe9211cae1e403b"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:36:00.041525 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-ServersNegativeTestJSON-1102403437 Jul 16 21:36:00.041525 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: da2248fa190342f0bfe9211cae1e403b Jul 16 21:36:00.041525 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-ServersNegativeTestJSON-1102403437 Jul 16 21:36:00.041525 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-ServersNegativeTestJSON-1102403437-project Jul 16 21:36:00.041525 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:36:00.041525 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:36:00.041525 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: 22e264440fff43cdb7be6702839af37f Jul 16 21:36:00.041525 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-ServersNegativeTestJSON-1102403437-project Jul 16 21:36:00.041525 ncdv-master-1 devstack@n-api.service[138680]: Jul 16 21:36:00.041525 ncdv-master-1 devstack@n-api.service[138680]: {"server": {"name": "tempest-ServersNegativeTestJSON-server-657933810_updated"}} Jul 16 21:36:00.041525 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 21:36:00.041525 ncdv-master-1 devstack@n-api.service[138680]: exception: Instance 4f82f9e6-5941-4188-91e3-6aea6e2a3ec8 could not be found. Jul 16 21:36:00.041525 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': '4f82f9e6-5941-4188-91e3-6aea6e2a3ec8', 'body': {'server': {'name': 'tempest-ServersNegativeTestJSON-server-657933810_updated'}}} Jul 16 21:36:00.041525 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:36:00.041525 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:36:00.041525 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:36:00.041525 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:36:00.041525 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:36:00.041525 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:36:00.041525 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2898, in _get_instance Jul 16 21:36:00.041525 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=instance_uuid) Jul 16 21:36:00.041525 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 4f82f9e6-5941-4188-91e3-6aea6e2a3ec8 could not be found. Jul 16 21:36:00.041525 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:36:00.041525 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:36:00.041525 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:36:00.041525 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:36:00.041525 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:36:00.041525 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:36:00.041525 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:36:00.041525 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:36:00.041525 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:36:00.041525 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:36:00.041525 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 16 21:36:00.041525 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 16 21:36:00.041525 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 16 21:36:00.041525 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 16 21:36:00.041525 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 16 21:36:00.041525 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 16 21:36:00.041525 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi [Previous line repeated 1 more time] Jul 16 21:36:00.041525 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 915, in update Jul 16 21:36:00.041525 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_server(ctxt, req, id, is_detail=True) Jul 16 21:36:00.041525 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:36:00.047902 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:36:00.047902 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:36:00.047902 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:36:00.047902 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 4f82f9e6-5941-4188-91e3-6aea6e2a3ec8 could not be found. Jul 16 21:36:00.047902 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:36:00.047902 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:36:00.047902 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:36:00.047902 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:36:00.047902 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:36:00.047902 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:36:00.047902 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:36:00.047902 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:36:00.047902 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 4f82f9e6-5941-4188-91e3-6aea6e2a3ec8 could not be found. Jul 16 21:36:00.047902 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:36:00.047902 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-9da37770-0b0d-42df-a36f-f45965a5dc49 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Returning 404 to user: Instance 4f82f9e6-5941-4188-91e3-6aea6e2a3ec8 could not be found. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:36:00.047902 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-9da37770-0b0d-42df-a36f-f45965a5dc49 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "PUT /compute/v2.1/servers/4f82f9e6-5941-4188-91e3-6aea6e2a3ec8" status: 404 len: 111 microversion: 2.1 time: 0.015620 Jul 16 21:36:00.047902 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2554/5110] 10.222.0.16 () {60 vars in 1255 bytes} [Sat Jul 16 21:36:00 2022] PUT /compute/v2.1/servers/4f82f9e6-5941-4188-91e3-6aea6e2a3ec8 => generated 111 bytes in 17 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:36:00.055966 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-da383348-9de3-425b-b770-d31005b849d6 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:36:00.056607 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-da383348-9de3-425b-b770-d31005b849d6 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: e4c7a9cb-e55d-4819-9885-2e72deffddb1] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:36:00.064256 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-da383348-9de3-425b-b770-d31005b849d6 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:36:00.064723 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-da383348-9de3-425b-b770-d31005b849d6 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:36:00.066099 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-da383348-9de3-425b-b770-d31005b849d6 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:36:00.066553 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-da383348-9de3-425b-b770-d31005b849d6 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:36:00.254806 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-04639184-aa73-49ec-bb6d-89c83815a07b tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '726b46c54e9541fbb7825607bac3abe5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fd0ce287cb44cd59045868ce688909a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:36:00.263964 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-04639184-aa73-49ec-bb6d-89c83815a07b tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '726b46c54e9541fbb7825607bac3abe5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fd0ce287cb44cd59045868ce688909a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:36:00.265976 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-04639184-aa73-49ec-bb6d-89c83815a07b tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '726b46c54e9541fbb7825607bac3abe5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fd0ce287cb44cd59045868ce688909a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:36:00.267228 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-04639184-aa73-49ec-bb6d-89c83815a07b tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] 10.222.0.16 "GET /compute/v2.1/servers/19df5c33-44d4-4de9-947b-4707b78af23a" status: 200 len: 1874 microversion: 2.42 time: 0.286945 Jul 16 21:36:00.267792 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2557/5111] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:35:59 2022] GET /compute/v2.1/servers/19df5c33-44d4-4de9-947b-4707b78af23a => generated 1874 bytes in 288 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 16 21:36:00.282794 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-da383348-9de3-425b-b770-d31005b849d6 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '22e264440fff43cdb7be6702839af37f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da2248fa190342f0bfe9211cae1e403b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:36:00.291252 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-da383348-9de3-425b-b770-d31005b849d6 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1" status: 200 len: 1504 microversion: 2.1 time: 0.239001 Jul 16 21:36:00.292182 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2555/5112] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:36:00 2022] GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1 => generated 1504 bytes in 244 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:36:00.308950 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-768c2fcd-cbce-49bf-a1be-809348b927e3 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:36:00.309563 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-768c2fcd-cbce-49bf-a1be-809348b927e3 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: e4c7a9cb-e55d-4819-9885-2e72deffddb1] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:36:00.316814 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-768c2fcd-cbce-49bf-a1be-809348b927e3 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:36:00.317391 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-768c2fcd-cbce-49bf-a1be-809348b927e3 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:36:00.318194 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-768c2fcd-cbce-49bf-a1be-809348b927e3 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:36:00.318787 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-768c2fcd-cbce-49bf-a1be-809348b927e3 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:36:00.458619 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-768c2fcd-cbce-49bf-a1be-809348b927e3 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '22e264440fff43cdb7be6702839af37f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da2248fa190342f0bfe9211cae1e403b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:36:00.466591 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-768c2fcd-cbce-49bf-a1be-809348b927e3 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1" status: 200 len: 1504 microversion: 2.1 time: 0.161527 Jul 16 21:36:00.467187 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2558/5113] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:36:00 2022] GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1 => generated 1504 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:36:00.484952 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-bf191026-8803-46e8-ab9c-1c280d6588cf tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Action: 'update', calling method: >, body: {"server": {"name": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:36:00.497797 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-bf191026-8803-46e8-ab9c-1c280d6588cf tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Request method failure captured: Jul 16 21:36:00.497797 ncdv-master-1 devstack@n-api.service[138680]: request: PUT /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1 HTTP/1.1 Jul 16 21:36:00.497797 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:36:00.497797 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:36:00.497797 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:36:00.497797 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 280 Jul 16 21:36:00.497797 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:36:00.497797 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:36:00.497797 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:36:00.497797 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:36:00.497797 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0y6QJDVdPNcLbUnwwgOJV-QkddLQ7yqnzBNVp2CrOe5UQqjQOER3spl3PswEW82ISSx8FhmuIrEwFSG69Ze1pTOmWBWZ8PURIgVHbN3yBPA3U2S2TKkQkFwyNuvvV03MCr9U0QGJIjXVOkiqFEAy8q87U_okZgCdI-BFjBFgrCdfAEo Jul 16 21:36:00.497797 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:36:00.497797 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:36:00.497797 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:36:00.497797 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:36:00.497797 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:36:00.497797 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:36:00.497797 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: da2248fa190342f0bfe9211cae1e403b Jul 16 21:36:00.497797 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-ServersNegativeTestJSON-1102403437 Jul 16 21:36:00.497797 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:36:00.497797 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:36:00.497797 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/da2248fa190342f0bfe9211cae1e403b"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/da2248fa190342f0bfe9211cae1e403b"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_da2248fa190342f0bfe9211cae1e403b"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/da2248fa190342f0bfe9211cae1e403b"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:36:00.497797 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-ServersNegativeTestJSON-1102403437 Jul 16 21:36:00.497797 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: da2248fa190342f0bfe9211cae1e403b Jul 16 21:36:00.497797 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-ServersNegativeTestJSON-1102403437 Jul 16 21:36:00.497797 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-ServersNegativeTestJSON-1102403437-project Jul 16 21:36:00.497797 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:36:00.497797 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:36:00.497797 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: 22e264440fff43cdb7be6702839af37f Jul 16 21:36:00.497797 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-ServersNegativeTestJSON-1102403437-project Jul 16 21:36:00.497797 ncdv-master-1 devstack@n-api.service[138680]: Jul 16 21:36:00.497797 ncdv-master-1 devstack@n-api.service[138680]: {"server": {"name": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"}} Jul 16 21:36:00.497797 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 21:36:00.497797 ncdv-master-1 devstack@n-api.service[138680]: exception: Invalid input for field/attribute name. Value: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa. 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' is too long Jul 16 21:36:00.497797 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': 'e4c7a9cb-e55d-4819-9885-2e72deffddb1', 'body': {'server': {'name': 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa'}}} Jul 16 21:36:00.497797 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:36:00.497797 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:36:00.497797 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 279, in validate Jul 16 21:36:00.497797 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Jul 16 21:36:00.497797 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/lib/python3/dist-packages/jsonschema/validators.py", line 353, in validate Jul 16 21:36:00.497797 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise error Jul 16 21:36:00.497797 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' is too long Jul 16 21:36:00.497797 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:36:00.497797 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Failed validating 'maxLength' in schema['properties']['server']['properties']['name']: Jul 16 21:36:00.497797 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi {'format': 'name', 'maxLength': 255, 'minLength': 1, 'type': 'string'} Jul 16 21:36:00.497797 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:36:00.497797 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi On instance['server']: Jul 16 21:36:00.497797 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' Jul 16 21:36:00.497797 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:36:00.497797 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:36:00.497797 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:36:00.497797 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:36:00.501583 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:36:00.501583 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:36:00.501583 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:36:00.501583 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:36:00.501583 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:36:00.501583 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:36:00.501583 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 16 21:36:00.501583 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 16 21:36:00.501583 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Jul 16 21:36:00.501583 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Jul 16 21:36:00.501583 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Jul 16 21:36:00.501583 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Jul 16 21:36:00.501583 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 308, in validate Jul 16 21:36:00.501583 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Jul 16 21:36:00.501583 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute name. Value: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa. 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' is too long Jul 16 21:36:00.501583 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:36:00.501583 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:36:00.501583 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:36:00.501583 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:36:00.501583 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:36:00.501583 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:36:00.501583 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Jul 16 21:36:00.501583 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Jul 16 21:36:00.501583 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute name. Value: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa. 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' is too long Jul 16 21:36:00.501583 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:36:00.501583 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-bf191026-8803-46e8-ab9c-1c280d6588cf tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Returning 400 to user: Invalid input for field/attribute name. Value: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa. 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' is too long {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:36:00.501583 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-bf191026-8803-46e8-ab9c-1c280d6588cf tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "PUT /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1" status: 400 len: 619 microversion: 2.1 time: 0.019220 Jul 16 21:36:00.501583 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2556/5114] 10.222.0.16 () {60 vars in 1256 bytes} [Sat Jul 16 21:36:00 2022] PUT /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1 => generated 619 bytes in 21 msecs (HTTP/1.1 400) 9 headers in 381 bytes (2 switches on core 0) Jul 16 21:36:00.510599 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-71c87ced-7365-4575-ade5-2fcbdb55fcf6 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:36:00.511375 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-71c87ced-7365-4575-ade5-2fcbdb55fcf6 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: e4c7a9cb-e55d-4819-9885-2e72deffddb1] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:36:00.518185 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-71c87ced-7365-4575-ade5-2fcbdb55fcf6 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:36:00.519259 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-71c87ced-7365-4575-ade5-2fcbdb55fcf6 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:36:00.520288 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-71c87ced-7365-4575-ade5-2fcbdb55fcf6 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:36:00.520735 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-71c87ced-7365-4575-ade5-2fcbdb55fcf6 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:36:00.671474 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-71c87ced-7365-4575-ade5-2fcbdb55fcf6 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '22e264440fff43cdb7be6702839af37f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da2248fa190342f0bfe9211cae1e403b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:36:00.679311 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-71c87ced-7365-4575-ade5-2fcbdb55fcf6 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1" status: 200 len: 1504 microversion: 2.1 time: 0.172937 Jul 16 21:36:00.679980 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2559/5115] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:36:00 2022] GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1 => generated 1504 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:36:00.698435 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-901abcc9-4c1b-43df-bd0a-4619aaec3ddf tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:36:00.699086 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-901abcc9-4c1b-43df-bd0a-4619aaec3ddf tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: e4c7a9cb-e55d-4819-9885-2e72deffddb1] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:36:00.707036 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-901abcc9-4c1b-43df-bd0a-4619aaec3ddf tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:36:00.707507 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-901abcc9-4c1b-43df-bd0a-4619aaec3ddf tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:36:00.708283 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-901abcc9-4c1b-43df-bd0a-4619aaec3ddf tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:36:00.708735 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-901abcc9-4c1b-43df-bd0a-4619aaec3ddf tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:36:00.849549 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-901abcc9-4c1b-43df-bd0a-4619aaec3ddf tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '22e264440fff43cdb7be6702839af37f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da2248fa190342f0bfe9211cae1e403b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:36:00.866900 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-901abcc9-4c1b-43df-bd0a-4619aaec3ddf tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1" status: 200 len: 1504 microversion: 2.1 time: 0.173828 Jul 16 21:36:00.867859 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2557/5116] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:36:00 2022] GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1 => generated 1504 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:36:00.884220 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-a743f142-9cc5-4919-84f0-6b87f73fe9db tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Action: 'update', calling method: >, body: {"server": {"name": ""}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:36:00.886673 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-a743f142-9cc5-4919-84f0-6b87f73fe9db tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Request method failure captured: Jul 16 21:36:00.886673 ncdv-master-1 devstack@n-api.service[138681]: request: PUT /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1 HTTP/1.1 Jul 16 21:36:00.886673 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:36:00.886673 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:36:00.886673 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:36:00.886673 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 24 Jul 16 21:36:00.886673 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:36:00.886673 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:36:00.886673 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:36:00.886673 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:36:00.886673 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0y6QJDVdPNcLbUnwwgOJV-QkddLQ7yqnzBNVp2CrOe5UQqjQOER3spl3PswEW82ISSx8FhmuIrEwFSG69Ze1pTOmWBWZ8PURIgVHbN3yBPA3U2S2TKkQkFwyNuvvV03MCr9U0QGJIjXVOkiqFEAy8q87U_okZgCdI-BFjBFgrCdfAEo Jul 16 21:36:00.886673 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:36:00.886673 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:36:00.886673 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:36:00.886673 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:36:00.886673 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:36:00.886673 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:36:00.886673 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: da2248fa190342f0bfe9211cae1e403b Jul 16 21:36:00.886673 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-ServersNegativeTestJSON-1102403437 Jul 16 21:36:00.886673 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:36:00.886673 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:36:00.886673 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/da2248fa190342f0bfe9211cae1e403b"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/da2248fa190342f0bfe9211cae1e403b"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_da2248fa190342f0bfe9211cae1e403b"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/da2248fa190342f0bfe9211cae1e403b"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:36:00.886673 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-ServersNegativeTestJSON-1102403437 Jul 16 21:36:00.886673 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: da2248fa190342f0bfe9211cae1e403b Jul 16 21:36:00.886673 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-ServersNegativeTestJSON-1102403437 Jul 16 21:36:00.886673 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-ServersNegativeTestJSON-1102403437-project Jul 16 21:36:00.886673 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:36:00.886673 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:36:00.886673 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: 22e264440fff43cdb7be6702839af37f Jul 16 21:36:00.886673 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-ServersNegativeTestJSON-1102403437-project Jul 16 21:36:00.886673 ncdv-master-1 devstack@n-api.service[138681]: Jul 16 21:36:00.886673 ncdv-master-1 devstack@n-api.service[138681]: {"server": {"name": ""}} Jul 16 21:36:00.886673 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:36:00.886673 ncdv-master-1 devstack@n-api.service[138681]: exception: Invalid input for field/attribute name. Value: . '' is too short Jul 16 21:36:00.886673 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': 'e4c7a9cb-e55d-4819-9885-2e72deffddb1', 'body': {'server': {'name': ''}}} Jul 16 21:36:00.886673 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:36:00.886673 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:36:00.886673 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 279, in validate Jul 16 21:36:00.886673 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Jul 16 21:36:00.886673 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/lib/python3/dist-packages/jsonschema/validators.py", line 353, in validate Jul 16 21:36:00.886673 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise error Jul 16 21:36:00.886673 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: '' is too short Jul 16 21:36:00.886673 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:36:00.886673 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Failed validating 'minLength' in schema['properties']['server']['properties']['name']: Jul 16 21:36:00.886673 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi {'format': 'name', 'maxLength': 255, 'minLength': 1, 'type': 'string'} Jul 16 21:36:00.886673 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:36:00.886673 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi On instance['server']: Jul 16 21:36:00.886673 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi '' Jul 16 21:36:00.886673 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:36:00.886673 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:36:00.886673 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:36:00.886673 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:36:00.886673 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:36:00.886673 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:36:00.886673 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:36:00.886673 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:36:00.886673 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:36:00.886673 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:36:00.886673 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 16 21:36:00.886673 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 16 21:36:00.886673 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Jul 16 21:36:00.886673 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Jul 16 21:36:00.886673 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Jul 16 21:36:00.886673 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Jul 16 21:36:00.886673 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 308, in validate Jul 16 21:36:00.886673 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Jul 16 21:36:00.886673 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute name. Value: . '' is too short Jul 16 21:36:00.894927 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:36:00.894927 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:36:00.894927 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:36:00.894927 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:36:00.894927 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:36:00.894927 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:36:00.894927 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Jul 16 21:36:00.894927 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Jul 16 21:36:00.894927 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute name. Value: . '' is too short Jul 16 21:36:00.894927 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:36:00.894927 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-a743f142-9cc5-4919-84f0-6b87f73fe9db tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Returning 400 to user: Invalid input for field/attribute name. Value: . '' is too short {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:36:00.894927 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-a743f142-9cc5-4919-84f0-6b87f73fe9db tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "PUT /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1" status: 400 len: 108 microversion: 2.1 time: 0.008005 Jul 16 21:36:00.894927 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2560/5117] 10.222.0.16 () {60 vars in 1255 bytes} [Sat Jul 16 21:36:00 2022] PUT /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1 => generated 108 bytes in 9 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Jul 16 21:36:00.900201 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-fed28c35-a6f8-4413-93af-f8bf5a05cada tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:36:00.900832 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-fed28c35-a6f8-4413-93af-f8bf5a05cada tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: e4c7a9cb-e55d-4819-9885-2e72deffddb1] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:36:00.907136 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-fed28c35-a6f8-4413-93af-f8bf5a05cada tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:36:00.907599 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-fed28c35-a6f8-4413-93af-f8bf5a05cada tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:36:00.913962 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-fed28c35-a6f8-4413-93af-f8bf5a05cada tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:36:00.914542 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-fed28c35-a6f8-4413-93af-f8bf5a05cada tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:36:01.049498 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-fed28c35-a6f8-4413-93af-f8bf5a05cada tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '22e264440fff43cdb7be6702839af37f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da2248fa190342f0bfe9211cae1e403b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:36:01.057681 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-fed28c35-a6f8-4413-93af-f8bf5a05cada tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1" status: 200 len: 1504 microversion: 2.1 time: 0.161321 Jul 16 21:36:01.058316 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2558/5118] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:36:00 2022] GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1 => generated 1504 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:36:01.290527 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-ed8193c9-de64-4763-b454-19bc819608d6 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:36:01.291180 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-ed8193c9-de64-4763-b454-19bc819608d6 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] [instance: 19df5c33-44d4-4de9-947b-4707b78af23a] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:36:01.299157 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ed8193c9-de64-4763-b454-19bc819608d6 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:36:01.299157 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ed8193c9-de64-4763-b454-19bc819608d6 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:36:01.300343 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ed8193c9-de64-4763-b454-19bc819608d6 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:36:01.300791 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ed8193c9-de64-4763-b454-19bc819608d6 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:36:01.517438 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-ed8193c9-de64-4763-b454-19bc819608d6 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '726b46c54e9541fbb7825607bac3abe5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fd0ce287cb44cd59045868ce688909a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:36:01.520510 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:36:01.529080 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-ed8193c9-de64-4763-b454-19bc819608d6 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '726b46c54e9541fbb7825607bac3abe5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fd0ce287cb44cd59045868ce688909a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:36:01.530344 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-ed8193c9-de64-4763-b454-19bc819608d6 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '726b46c54e9541fbb7825607bac3abe5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fd0ce287cb44cd59045868ce688909a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:36:01.531557 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-ed8193c9-de64-4763-b454-19bc819608d6 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] 10.222.0.16 "GET /compute/v2.1/servers/19df5c33-44d4-4de9-947b-4707b78af23a" status: 200 len: 1874 microversion: 2.42 time: 0.245915 Jul 16 21:36:01.532114 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2561/5119] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:36:01 2022] GET /compute/v2.1/servers/19df5c33-44d4-4de9-947b-4707b78af23a => generated 1874 bytes in 247 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:36:02.550187 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-cc7eaf06-75ed-4be3-b5db-e1e2d96cdb1e tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:36:02.550829 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-cc7eaf06-75ed-4be3-b5db-e1e2d96cdb1e tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] [instance: 19df5c33-44d4-4de9-947b-4707b78af23a] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:36:02.557983 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cc7eaf06-75ed-4be3-b5db-e1e2d96cdb1e tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:36:02.558429 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cc7eaf06-75ed-4be3-b5db-e1e2d96cdb1e tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:36:02.559218 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cc7eaf06-75ed-4be3-b5db-e1e2d96cdb1e tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:36:02.559748 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cc7eaf06-75ed-4be3-b5db-e1e2d96cdb1e tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:36:02.750682 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-cc7eaf06-75ed-4be3-b5db-e1e2d96cdb1e tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '726b46c54e9541fbb7825607bac3abe5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fd0ce287cb44cd59045868ce688909a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:36:02.760704 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-cc7eaf06-75ed-4be3-b5db-e1e2d96cdb1e tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '726b46c54e9541fbb7825607bac3abe5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fd0ce287cb44cd59045868ce688909a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:36:02.762257 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-cc7eaf06-75ed-4be3-b5db-e1e2d96cdb1e tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '726b46c54e9541fbb7825607bac3abe5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fd0ce287cb44cd59045868ce688909a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:36:02.763478 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-cc7eaf06-75ed-4be3-b5db-e1e2d96cdb1e tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] 10.222.0.16 "GET /compute/v2.1/servers/19df5c33-44d4-4de9-947b-4707b78af23a" status: 200 len: 1874 microversion: 2.42 time: 0.217345 Jul 16 21:36:02.764041 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2559/5120] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:36:02 2022] GET /compute/v2.1/servers/19df5c33-44d4-4de9-947b-4707b78af23a => generated 1874 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:36:03.782978 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-0924f214-4fe1-4873-93b4-d2f0f10b99ee tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:36:03.783640 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-0924f214-4fe1-4873-93b4-d2f0f10b99ee tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] [instance: 19df5c33-44d4-4de9-947b-4707b78af23a] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:36:03.791561 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0924f214-4fe1-4873-93b4-d2f0f10b99ee tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:36:03.792018 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0924f214-4fe1-4873-93b4-d2f0f10b99ee tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:36:03.792796 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0924f214-4fe1-4873-93b4-d2f0f10b99ee tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:36:03.793241 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0924f214-4fe1-4873-93b4-d2f0f10b99ee tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:36:03.987224 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-0924f214-4fe1-4873-93b4-d2f0f10b99ee tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '726b46c54e9541fbb7825607bac3abe5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fd0ce287cb44cd59045868ce688909a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:36:03.996558 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-0924f214-4fe1-4873-93b4-d2f0f10b99ee tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '726b46c54e9541fbb7825607bac3abe5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fd0ce287cb44cd59045868ce688909a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:36:03.998237 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-0924f214-4fe1-4873-93b4-d2f0f10b99ee tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '726b46c54e9541fbb7825607bac3abe5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fd0ce287cb44cd59045868ce688909a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:36:03.999503 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-0924f214-4fe1-4873-93b4-d2f0f10b99ee tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] 10.222.0.16 "GET /compute/v2.1/servers/19df5c33-44d4-4de9-947b-4707b78af23a" status: 200 len: 1874 microversion: 2.42 time: 0.220577 Jul 16 21:36:04.000298 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2562/5121] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:36:03 2022] GET /compute/v2.1/servers/19df5c33-44d4-4de9-947b-4707b78af23a => generated 1874 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:36:05.061570 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-5942002c-5464-435b-a10b-f0ffd8a4f884 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:36:05.061570 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-5942002c-5464-435b-a10b-f0ffd8a4f884 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] [instance: 19df5c33-44d4-4de9-947b-4707b78af23a] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:36:05.061570 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5942002c-5464-435b-a10b-f0ffd8a4f884 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:36:05.061570 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5942002c-5464-435b-a10b-f0ffd8a4f884 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:36:05.061570 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5942002c-5464-435b-a10b-f0ffd8a4f884 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:36:05.061570 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5942002c-5464-435b-a10b-f0ffd8a4f884 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:36:05.061570 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:36:05.219800 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-5942002c-5464-435b-a10b-f0ffd8a4f884 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '726b46c54e9541fbb7825607bac3abe5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fd0ce287cb44cd59045868ce688909a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:36:05.229817 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-5942002c-5464-435b-a10b-f0ffd8a4f884 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '726b46c54e9541fbb7825607bac3abe5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fd0ce287cb44cd59045868ce688909a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:36:05.231870 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-5942002c-5464-435b-a10b-f0ffd8a4f884 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '726b46c54e9541fbb7825607bac3abe5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fd0ce287cb44cd59045868ce688909a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:36:05.233047 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-5942002c-5464-435b-a10b-f0ffd8a4f884 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] 10.222.0.16 "GET /compute/v2.1/servers/19df5c33-44d4-4de9-947b-4707b78af23a" status: 200 len: 1874 microversion: 2.42 time: 0.218489 Jul 16 21:36:05.235107 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2560/5122] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:36:05 2022] GET /compute/v2.1/servers/19df5c33-44d4-4de9-947b-4707b78af23a => generated 1874 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:36:06.254640 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-8d2d4b18-1c86-4335-920d-24bd18bee73f tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:36:06.255611 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-8d2d4b18-1c86-4335-920d-24bd18bee73f tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] [instance: 19df5c33-44d4-4de9-947b-4707b78af23a] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:36:06.263831 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8d2d4b18-1c86-4335-920d-24bd18bee73f tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:36:06.264301 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8d2d4b18-1c86-4335-920d-24bd18bee73f tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:36:06.265155 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8d2d4b18-1c86-4335-920d-24bd18bee73f tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:36:06.265608 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8d2d4b18-1c86-4335-920d-24bd18bee73f tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:36:06.501183 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-8d2d4b18-1c86-4335-920d-24bd18bee73f tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '726b46c54e9541fbb7825607bac3abe5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fd0ce287cb44cd59045868ce688909a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:36:06.512784 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-8d2d4b18-1c86-4335-920d-24bd18bee73f tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '726b46c54e9541fbb7825607bac3abe5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fd0ce287cb44cd59045868ce688909a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:36:06.514420 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-8d2d4b18-1c86-4335-920d-24bd18bee73f tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '726b46c54e9541fbb7825607bac3abe5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fd0ce287cb44cd59045868ce688909a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:36:06.515715 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-8d2d4b18-1c86-4335-920d-24bd18bee73f tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] 10.222.0.16 "GET /compute/v2.1/servers/19df5c33-44d4-4de9-947b-4707b78af23a" status: 200 len: 1874 microversion: 2.42 time: 0.266762 Jul 16 21:36:06.516920 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2563/5123] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:36:06 2022] GET /compute/v2.1/servers/19df5c33-44d4-4de9-947b-4707b78af23a => generated 1874 bytes in 269 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:36:07.537939 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-d9181b0d-1e12-43cd-aa84-fc9cdb701434 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:36:07.538404 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-d9181b0d-1e12-43cd-aa84-fc9cdb701434 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] [instance: 19df5c33-44d4-4de9-947b-4707b78af23a] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:36:07.546607 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d9181b0d-1e12-43cd-aa84-fc9cdb701434 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:36:07.547086 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d9181b0d-1e12-43cd-aa84-fc9cdb701434 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:36:07.547858 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d9181b0d-1e12-43cd-aa84-fc9cdb701434 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:36:07.548305 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d9181b0d-1e12-43cd-aa84-fc9cdb701434 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:36:07.738490 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-d9181b0d-1e12-43cd-aa84-fc9cdb701434 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '726b46c54e9541fbb7825607bac3abe5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fd0ce287cb44cd59045868ce688909a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:36:07.747638 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-d9181b0d-1e12-43cd-aa84-fc9cdb701434 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '726b46c54e9541fbb7825607bac3abe5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fd0ce287cb44cd59045868ce688909a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:36:07.749197 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-d9181b0d-1e12-43cd-aa84-fc9cdb701434 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '726b46c54e9541fbb7825607bac3abe5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fd0ce287cb44cd59045868ce688909a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:36:07.750399 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-d9181b0d-1e12-43cd-aa84-fc9cdb701434 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] 10.222.0.16 "GET /compute/v2.1/servers/19df5c33-44d4-4de9-947b-4707b78af23a" status: 200 len: 1874 microversion: 2.42 time: 0.217444 Jul 16 21:36:07.750969 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2561/5124] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:36:07 2022] GET /compute/v2.1/servers/19df5c33-44d4-4de9-947b-4707b78af23a => generated 1874 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:36:08.770384 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-dfbd18c9-4afb-4167-b3c6-478e564a4998 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:36:08.770915 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-dfbd18c9-4afb-4167-b3c6-478e564a4998 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] [instance: 19df5c33-44d4-4de9-947b-4707b78af23a] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:36:08.778843 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-dfbd18c9-4afb-4167-b3c6-478e564a4998 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:36:08.779329 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-dfbd18c9-4afb-4167-b3c6-478e564a4998 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:36:08.786143 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-dfbd18c9-4afb-4167-b3c6-478e564a4998 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:36:08.786143 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-dfbd18c9-4afb-4167-b3c6-478e564a4998 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:36:08.999730 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-dfbd18c9-4afb-4167-b3c6-478e564a4998 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '726b46c54e9541fbb7825607bac3abe5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fd0ce287cb44cd59045868ce688909a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:36:09.011281 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-dfbd18c9-4afb-4167-b3c6-478e564a4998 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '726b46c54e9541fbb7825607bac3abe5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fd0ce287cb44cd59045868ce688909a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:36:09.013096 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-dfbd18c9-4afb-4167-b3c6-478e564a4998 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '726b46c54e9541fbb7825607bac3abe5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fd0ce287cb44cd59045868ce688909a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:36:09.014307 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-dfbd18c9-4afb-4167-b3c6-478e564a4998 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] 10.222.0.16 "GET /compute/v2.1/servers/19df5c33-44d4-4de9-947b-4707b78af23a" status: 200 len: 1874 microversion: 2.42 time: 0.248430 Jul 16 21:36:09.014866 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2564/5125] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:36:08 2022] GET /compute/v2.1/servers/19df5c33-44d4-4de9-947b-4707b78af23a => generated 1874 bytes in 250 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 16 21:36:10.033285 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-7316133a-e8a1-4a15-afeb-f7d5ada968ae tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:36:10.034018 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-7316133a-e8a1-4a15-afeb-f7d5ada968ae tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] [instance: 19df5c33-44d4-4de9-947b-4707b78af23a] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:36:10.042214 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7316133a-e8a1-4a15-afeb-f7d5ada968ae tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:36:10.042813 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7316133a-e8a1-4a15-afeb-f7d5ada968ae tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:36:10.046430 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7316133a-e8a1-4a15-afeb-f7d5ada968ae tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:36:10.046430 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7316133a-e8a1-4a15-afeb-f7d5ada968ae tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:36:10.255901 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-7316133a-e8a1-4a15-afeb-f7d5ada968ae tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '726b46c54e9541fbb7825607bac3abe5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fd0ce287cb44cd59045868ce688909a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:36:10.265041 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-7316133a-e8a1-4a15-afeb-f7d5ada968ae tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '726b46c54e9541fbb7825607bac3abe5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fd0ce287cb44cd59045868ce688909a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:36:10.266546 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-7316133a-e8a1-4a15-afeb-f7d5ada968ae tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '726b46c54e9541fbb7825607bac3abe5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fd0ce287cb44cd59045868ce688909a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:36:10.267731 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-7316133a-e8a1-4a15-afeb-f7d5ada968ae tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] 10.222.0.16 "GET /compute/v2.1/servers/19df5c33-44d4-4de9-947b-4707b78af23a" status: 200 len: 1874 microversion: 2.42 time: 0.238722 Jul 16 21:36:10.268295 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2562/5126] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:36:10 2022] GET /compute/v2.1/servers/19df5c33-44d4-4de9-947b-4707b78af23a => generated 1874 bytes in 240 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:36:11.287137 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-f73fcd1e-e82f-460b-94ca-1ab4cf318fcc tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:36:11.287762 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-f73fcd1e-e82f-460b-94ca-1ab4cf318fcc tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] [instance: 19df5c33-44d4-4de9-947b-4707b78af23a] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:36:11.295927 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f73fcd1e-e82f-460b-94ca-1ab4cf318fcc tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:36:11.296550 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f73fcd1e-e82f-460b-94ca-1ab4cf318fcc tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:36:11.297339 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f73fcd1e-e82f-460b-94ca-1ab4cf318fcc tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:36:11.297813 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f73fcd1e-e82f-460b-94ca-1ab4cf318fcc tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:36:11.586526 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-f73fcd1e-e82f-460b-94ca-1ab4cf318fcc tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '726b46c54e9541fbb7825607bac3abe5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fd0ce287cb44cd59045868ce688909a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:36:11.603767 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-f73fcd1e-e82f-460b-94ca-1ab4cf318fcc tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '726b46c54e9541fbb7825607bac3abe5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fd0ce287cb44cd59045868ce688909a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:36:11.603767 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-f73fcd1e-e82f-460b-94ca-1ab4cf318fcc tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '726b46c54e9541fbb7825607bac3abe5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fd0ce287cb44cd59045868ce688909a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:36:11.603767 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-f73fcd1e-e82f-460b-94ca-1ab4cf318fcc tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] 10.222.0.16 "GET /compute/v2.1/servers/19df5c33-44d4-4de9-947b-4707b78af23a" status: 200 len: 1874 microversion: 2.42 time: 0.317251 Jul 16 21:36:11.604605 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2565/5127] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:36:11 2022] GET /compute/v2.1/servers/19df5c33-44d4-4de9-947b-4707b78af23a => generated 1874 bytes in 323 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:36:12.623723 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-46ac01f7-a447-4c34-9776-b716a3b3528b tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:36:12.624984 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-46ac01f7-a447-4c34-9776-b716a3b3528b tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] [instance: 19df5c33-44d4-4de9-947b-4707b78af23a] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:36:12.633940 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-6da3d949-87e8-4e3a-aafc-bc3f5b615ca3 req-86560a04-cb86-474a-8a6b-6a7b8ff192cf service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "19df5c33-44d4-4de9-947b-4707b78af23a", "name": "network-vif-plugged", "status": "completed", "tag": "3e3ac495-fa5d-4999-b113-9a54b000c446"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:36:12.641830 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-46ac01f7-a447-4c34-9776-b716a3b3528b tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:36:12.642834 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-46ac01f7-a447-4c34-9776-b716a3b3528b tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:36:12.647888 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-46ac01f7-a447-4c34-9776-b716a3b3528b tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:36:12.647888 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-46ac01f7-a447-4c34-9776-b716a3b3528b tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:36:12.658044 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-6da3d949-87e8-4e3a-aafc-bc3f5b615ca3 req-86560a04-cb86-474a-8a6b-6a7b8ff192cf service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:36:12.659261 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-6da3d949-87e8-4e3a-aafc-bc3f5b615ca3 req-86560a04-cb86-474a-8a6b-6a7b8ff192cf service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:36:12.684576 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-6da3d949-87e8-4e3a-aafc-bc3f5b615ca3 req-86560a04-cb86-474a-8a6b-6a7b8ff192cf service nova] Creating event network-vif-plugged:3e3ac495-fa5d-4999-b113-9a54b000c446 for instance 19df5c33-44d4-4de9-947b-4707b78af23a on nch2-master-1 Jul 16 21:36:12.699701 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-6da3d949-87e8-4e3a-aafc-bc3f5b615ca3 req-86560a04-cb86-474a-8a6b-6a7b8ff192cf service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.074583 Jul 16 21:36:12.700780 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2566/5128] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:36:12 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 77 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:36:12.872253 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-46ac01f7-a447-4c34-9776-b716a3b3528b tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '726b46c54e9541fbb7825607bac3abe5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fd0ce287cb44cd59045868ce688909a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:36:12.882356 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-46ac01f7-a447-4c34-9776-b716a3b3528b tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '726b46c54e9541fbb7825607bac3abe5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fd0ce287cb44cd59045868ce688909a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:36:12.883931 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-46ac01f7-a447-4c34-9776-b716a3b3528b tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '726b46c54e9541fbb7825607bac3abe5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fd0ce287cb44cd59045868ce688909a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:36:12.885129 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-46ac01f7-a447-4c34-9776-b716a3b3528b tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] 10.222.0.16 "GET /compute/v2.1/servers/19df5c33-44d4-4de9-947b-4707b78af23a" status: 200 len: 1874 microversion: 2.42 time: 0.270920 Jul 16 21:36:12.885696 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2563/5129] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:36:12 2022] GET /compute/v2.1/servers/19df5c33-44d4-4de9-947b-4707b78af23a => generated 1874 bytes in 272 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:36:13.908470 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-2ce4a21f-95a1-4621-8f98-4601936a43c8 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:36:13.908470 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-2ce4a21f-95a1-4621-8f98-4601936a43c8 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] [instance: 19df5c33-44d4-4de9-947b-4707b78af23a] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:36:13.914077 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2ce4a21f-95a1-4621-8f98-4601936a43c8 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:36:13.914607 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2ce4a21f-95a1-4621-8f98-4601936a43c8 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:36:13.915490 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2ce4a21f-95a1-4621-8f98-4601936a43c8 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:36:13.916000 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2ce4a21f-95a1-4621-8f98-4601936a43c8 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:36:14.131961 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-2ce4a21f-95a1-4621-8f98-4601936a43c8 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '726b46c54e9541fbb7825607bac3abe5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fd0ce287cb44cd59045868ce688909a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:36:14.144475 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-2ce4a21f-95a1-4621-8f98-4601936a43c8 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '726b46c54e9541fbb7825607bac3abe5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fd0ce287cb44cd59045868ce688909a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:36:14.146843 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-2ce4a21f-95a1-4621-8f98-4601936a43c8 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '726b46c54e9541fbb7825607bac3abe5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fd0ce287cb44cd59045868ce688909a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:36:14.148229 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-2ce4a21f-95a1-4621-8f98-4601936a43c8 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] 10.222.0.16 "GET /compute/v2.1/servers/19df5c33-44d4-4de9-947b-4707b78af23a" status: 200 len: 1874 microversion: 2.42 time: 0.247554 Jul 16 21:36:14.148804 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2567/5130] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:36:13 2022] GET /compute/v2.1/servers/19df5c33-44d4-4de9-947b-4707b78af23a => generated 1874 bytes in 250 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:36:14.733753 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-b62bc889-6192-4093-b238-22c3b8d7f858 req-c8b17df5-b632-4286-980c-ce55b0e8038c service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "19df5c33-44d4-4de9-947b-4707b78af23a", "name": "network-vif-plugged", "status": "completed", "tag": "78c19415-67c3-482e-9663-0cdac41890d3"}, {"server_uuid": "19df5c33-44d4-4de9-947b-4707b78af23a", "name": "network-vif-plugged", "status": "completed", "tag": "4c5dcf45-0de2-481d-9d69-0ca39f95898b"}, {"server_uuid": "19df5c33-44d4-4de9-947b-4707b78af23a", "name": "network-vif-plugged", "status": "completed", "tag": "7d150f5e-be69-4d03-aff9-6fc347935783"}, {"server_uuid": "19df5c33-44d4-4de9-947b-4707b78af23a", "name": "network-vif-plugged", "status": "completed", "tag": "535c3524-f5da-4ba5-a149-40d9c379754e"}, {"server_uuid": "19df5c33-44d4-4de9-947b-4707b78af23a", "name": "network-vif-plugged", "status": "completed", "tag": "25ef7728-f1b7-4467-9f81-8af48bbcef2d"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:36:14.752469 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-b62bc889-6192-4093-b238-22c3b8d7f858 req-c8b17df5-b632-4286-980c-ce55b0e8038c service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:36:14.753260 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-b62bc889-6192-4093-b238-22c3b8d7f858 req-c8b17df5-b632-4286-980c-ce55b0e8038c service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:36:14.777618 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-b62bc889-6192-4093-b238-22c3b8d7f858 req-c8b17df5-b632-4286-980c-ce55b0e8038c service nova] Creating event network-vif-plugged:78c19415-67c3-482e-9663-0cdac41890d3 for instance 19df5c33-44d4-4de9-947b-4707b78af23a on nch2-master-1 Jul 16 21:36:14.778165 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-b62bc889-6192-4093-b238-22c3b8d7f858 req-c8b17df5-b632-4286-980c-ce55b0e8038c service nova] Creating event network-vif-plugged:4c5dcf45-0de2-481d-9d69-0ca39f95898b for instance 19df5c33-44d4-4de9-947b-4707b78af23a on nch2-master-1 Jul 16 21:36:14.778681 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-b62bc889-6192-4093-b238-22c3b8d7f858 req-c8b17df5-b632-4286-980c-ce55b0e8038c service nova] Creating event network-vif-plugged:7d150f5e-be69-4d03-aff9-6fc347935783 for instance 19df5c33-44d4-4de9-947b-4707b78af23a on nch2-master-1 Jul 16 21:36:14.779210 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-b62bc889-6192-4093-b238-22c3b8d7f858 req-c8b17df5-b632-4286-980c-ce55b0e8038c service nova] Creating event network-vif-plugged:535c3524-f5da-4ba5-a149-40d9c379754e for instance 19df5c33-44d4-4de9-947b-4707b78af23a on nch2-master-1 Jul 16 21:36:14.779706 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-b62bc889-6192-4093-b238-22c3b8d7f858 req-c8b17df5-b632-4286-980c-ce55b0e8038c service nova] Creating event network-vif-plugged:25ef7728-f1b7-4467-9f81-8af48bbcef2d for instance 19df5c33-44d4-4de9-947b-4707b78af23a on nch2-master-1 Jul 16 21:36:14.786899 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-b62bc889-6192-4093-b238-22c3b8d7f858 req-c8b17df5-b632-4286-980c-ce55b0e8038c service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 867 microversion: 2.1 time: 0.057042 Jul 16 21:36:14.787678 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2564/5131] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:36:14 2022] POST /compute/v2.1/os-server-external-events => generated 867 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:36:15.166567 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-1e2bbee9-af60-4677-a7bc-b56300ce0d4b tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:36:15.167293 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-1e2bbee9-af60-4677-a7bc-b56300ce0d4b tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] [instance: 19df5c33-44d4-4de9-947b-4707b78af23a] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:36:15.174321 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1e2bbee9-af60-4677-a7bc-b56300ce0d4b tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:36:15.175154 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1e2bbee9-af60-4677-a7bc-b56300ce0d4b tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:36:15.175647 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1e2bbee9-af60-4677-a7bc-b56300ce0d4b tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:36:15.176088 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1e2bbee9-af60-4677-a7bc-b56300ce0d4b tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:36:15.368194 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-1e2bbee9-af60-4677-a7bc-b56300ce0d4b tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '726b46c54e9541fbb7825607bac3abe5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fd0ce287cb44cd59045868ce688909a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:36:15.377931 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-1e2bbee9-af60-4677-a7bc-b56300ce0d4b tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '726b46c54e9541fbb7825607bac3abe5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fd0ce287cb44cd59045868ce688909a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:36:15.379604 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-1e2bbee9-af60-4677-a7bc-b56300ce0d4b tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '726b46c54e9541fbb7825607bac3abe5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fd0ce287cb44cd59045868ce688909a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:36:15.381945 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-1e2bbee9-af60-4677-a7bc-b56300ce0d4b tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] 10.222.0.16 "GET /compute/v2.1/servers/19df5c33-44d4-4de9-947b-4707b78af23a" status: 200 len: 1874 microversion: 2.42 time: 0.220010 Jul 16 21:36:15.382513 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2568/5132] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:36:15 2022] GET /compute/v2.1/servers/19df5c33-44d4-4de9-947b-4707b78af23a => generated 1874 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:36:16.404427 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-a1674ef3-6416-4a70-aa74-adb5c44899f4 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:36:16.404427 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-a1674ef3-6416-4a70-aa74-adb5c44899f4 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] [instance: 19df5c33-44d4-4de9-947b-4707b78af23a] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:36:16.405440 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_api:SELECT=84 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:36:16.410453 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a1674ef3-6416-4a70-aa74-adb5c44899f4 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:36:16.410914 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a1674ef3-6416-4a70-aa74-adb5c44899f4 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:36:16.411895 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a1674ef3-6416-4a70-aa74-adb5c44899f4 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:36:16.412445 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a1674ef3-6416-4a70-aa74-adb5c44899f4 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:36:16.613895 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-a1674ef3-6416-4a70-aa74-adb5c44899f4 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '726b46c54e9541fbb7825607bac3abe5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fd0ce287cb44cd59045868ce688909a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:36:16.623292 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-a1674ef3-6416-4a70-aa74-adb5c44899f4 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '726b46c54e9541fbb7825607bac3abe5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fd0ce287cb44cd59045868ce688909a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:36:16.624797 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-a1674ef3-6416-4a70-aa74-adb5c44899f4 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '726b46c54e9541fbb7825607bac3abe5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fd0ce287cb44cd59045868ce688909a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:36:16.625957 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-a1674ef3-6416-4a70-aa74-adb5c44899f4 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] 10.222.0.16 "GET /compute/v2.1/servers/19df5c33-44d4-4de9-947b-4707b78af23a" status: 200 len: 1874 microversion: 2.42 time: 0.228333 Jul 16 21:36:16.626539 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2565/5133] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:36:16 2022] GET /compute/v2.1/servers/19df5c33-44d4-4de9-947b-4707b78af23a => generated 1874 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:36:16.770268 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-00940b42-9a8d-48a1-8df7-0d95b7df7a6a tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:36:16.771390 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-00940b42-9a8d-48a1-8df7-0d95b7df7a6a tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: c77c9a95-ea71-40d7-b7a7-ded612d194da] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:36:16.778471 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-00940b42-9a8d-48a1-8df7-0d95b7df7a6a tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:36:16.778928 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-00940b42-9a8d-48a1-8df7-0d95b7df7a6a tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:36:16.779749 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-00940b42-9a8d-48a1-8df7-0d95b7df7a6a tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:36:16.780198 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-00940b42-9a8d-48a1-8df7-0d95b7df7a6a tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:36:16.795869 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-00940b42-9a8d-48a1-8df7-0d95b7df7a6a tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] HTTP exception thrown: Instance c77c9a95-ea71-40d7-b7a7-ded612d194da could not be found. Jul 16 21:36:16.797561 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-00940b42-9a8d-48a1-8df7-0d95b7df7a6a tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Request method failure captured: Jul 16 21:36:16.797561 ncdv-master-1 devstack@n-api.service[138681]: request: DELETE /compute/v2.1/servers/c77c9a95-ea71-40d7-b7a7-ded612d194da HTTP/1.1 Jul 16 21:36:16.797561 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:36:16.797561 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:36:16.797561 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:36:16.797561 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 21:36:16.797561 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:36:16.797561 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:36:16.797561 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:36:16.797561 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:36:16.797561 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0y6QJDVdPNcLbUnwwgOJV-QkddLQ7yqnzBNVp2CrOe5UQqjQOER3spl3PswEW82ISSx8FhmuIrEwFSG69Ze1pTOmWBWZ8PURIgVHbN3yBPA3U2S2TKkQkFwyNuvvV03MCr9U0QGJIjXVOkiqFEAy8q87U_okZgCdI-BFjBFgrCdfAEo Jul 16 21:36:16.797561 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:36:16.797561 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:36:16.797561 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:36:16.797561 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:36:16.797561 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:36:16.797561 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:36:16.797561 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: da2248fa190342f0bfe9211cae1e403b Jul 16 21:36:16.797561 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-ServersNegativeTestJSON-1102403437 Jul 16 21:36:16.797561 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:36:16.797561 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:36:16.797561 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/da2248fa190342f0bfe9211cae1e403b"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/da2248fa190342f0bfe9211cae1e403b"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_da2248fa190342f0bfe9211cae1e403b"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/da2248fa190342f0bfe9211cae1e403b"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:36:16.797561 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-ServersNegativeTestJSON-1102403437 Jul 16 21:36:16.797561 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: da2248fa190342f0bfe9211cae1e403b Jul 16 21:36:16.797561 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-ServersNegativeTestJSON-1102403437 Jul 16 21:36:16.797561 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-ServersNegativeTestJSON-1102403437-project Jul 16 21:36:16.797561 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:36:16.797561 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:36:16.797561 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: 22e264440fff43cdb7be6702839af37f Jul 16 21:36:16.797561 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-ServersNegativeTestJSON-1102403437-project Jul 16 21:36:16.797561 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:36:16.797561 ncdv-master-1 devstack@n-api.service[138681]: exception: Instance c77c9a95-ea71-40d7-b7a7-ded612d194da could not be found. Jul 16 21:36:16.797561 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': 'c77c9a95-ea71-40d7-b7a7-ded612d194da'} Jul 16 21:36:16.797561 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:36:16.797561 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:36:16.797561 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:36:16.797561 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:36:16.797561 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:36:16.797561 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:36:16.797561 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:36:16.797561 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:36:16.797561 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:36:16.797561 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:36:16.797561 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance c77c9a95-ea71-40d7-b7a7-ded612d194da could not be found. Jul 16 21:36:16.797561 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:36:16.797561 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:36:16.797561 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:36:16.797561 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:36:16.797561 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:36:16.797561 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:36:16.797561 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:36:16.797561 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:36:16.797561 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:36:16.797561 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:36:16.797561 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Jul 16 21:36:16.797561 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Jul 16 21:36:16.797561 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Jul 16 21:36:16.797561 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Jul 16 21:36:16.797561 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:36:16.797561 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:36:16.797561 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:36:16.797561 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:36:16.805084 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance c77c9a95-ea71-40d7-b7a7-ded612d194da could not be found. Jul 16 21:36:16.805084 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:36:16.805084 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:36:16.805084 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:36:16.805084 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:36:16.805084 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:36:16.805084 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:36:16.805084 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:36:16.805084 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:36:16.805084 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance c77c9a95-ea71-40d7-b7a7-ded612d194da could not be found. Jul 16 21:36:16.805084 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:36:16.805084 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-00940b42-9a8d-48a1-8df7-0d95b7df7a6a tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Returning 404 to user: Instance c77c9a95-ea71-40d7-b7a7-ded612d194da could not be found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:36:16.805084 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-00940b42-9a8d-48a1-8df7-0d95b7df7a6a tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "DELETE /compute/v2.1/servers/c77c9a95-ea71-40d7-b7a7-ded612d194da" status: 404 len: 111 microversion: 2.1 time: 0.033031 Jul 16 21:36:16.805084 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2569/5134] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:36:16 2022] DELETE /compute/v2.1/servers/c77c9a95-ea71-40d7-b7a7-ded612d194da => generated 111 bytes in 34 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Jul 16 21:36:16.806801 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-c0a43aab-a3a6-49ec-ae87-d961109a0c07 req-0dbf3780-b5a3-4b1b-b4b8-2c41241d5c32 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "19df5c33-44d4-4de9-947b-4707b78af23a", "name": "network-vif-plugged", "status": "completed", "tag": "e518c0f5-c1c1-47b3-b1e8-743d59bf079a"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:36:16.807896 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-3859d22b-2860-4748-896b-fdd18d22452c tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:36:16.808545 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-3859d22b-2860-4748-896b-fdd18d22452c tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: c77c9a95-ea71-40d7-b7a7-ded612d194da] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:36:16.813123 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-c0a43aab-a3a6-49ec-ae87-d961109a0c07 req-0dbf3780-b5a3-4b1b-b4b8-2c41241d5c32 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:36:16.813538 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-c0a43aab-a3a6-49ec-ae87-d961109a0c07 req-0dbf3780-b5a3-4b1b-b4b8-2c41241d5c32 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:36:16.818372 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3859d22b-2860-4748-896b-fdd18d22452c tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:36:16.818372 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3859d22b-2860-4748-896b-fdd18d22452c tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:36:16.818372 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3859d22b-2860-4748-896b-fdd18d22452c tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:36:16.818372 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3859d22b-2860-4748-896b-fdd18d22452c tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:36:16.825795 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-c0a43aab-a3a6-49ec-ae87-d961109a0c07 req-0dbf3780-b5a3-4b1b-b4b8-2c41241d5c32 service nova] Creating event network-vif-plugged:e518c0f5-c1c1-47b3-b1e8-743d59bf079a for instance 19df5c33-44d4-4de9-947b-4707b78af23a on nch2-master-1 Jul 16 21:36:16.831219 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-c0a43aab-a3a6-49ec-ae87-d961109a0c07 req-0dbf3780-b5a3-4b1b-b4b8-2c41241d5c32 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.030779 Jul 16 21:36:16.831959 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2566/5135] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:36:16 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 16 21:36:16.832890 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-3859d22b-2860-4748-896b-fdd18d22452c tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] HTTP exception thrown: Instance c77c9a95-ea71-40d7-b7a7-ded612d194da could not be found. Jul 16 21:36:16.835238 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-3859d22b-2860-4748-896b-fdd18d22452c tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Request method failure captured: Jul 16 21:36:16.835238 ncdv-master-1 devstack@n-api.service[138681]: request: GET /compute/v2.1/servers/c77c9a95-ea71-40d7-b7a7-ded612d194da HTTP/1.1 Jul 16 21:36:16.835238 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:36:16.835238 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:36:16.835238 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:36:16.835238 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 21:36:16.835238 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:36:16.835238 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:36:16.835238 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:36:16.835238 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:36:16.835238 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0y6QJDVdPNcLbUnwwgOJV-QkddLQ7yqnzBNVp2CrOe5UQqjQOER3spl3PswEW82ISSx8FhmuIrEwFSG69Ze1pTOmWBWZ8PURIgVHbN3yBPA3U2S2TKkQkFwyNuvvV03MCr9U0QGJIjXVOkiqFEAy8q87U_okZgCdI-BFjBFgrCdfAEo Jul 16 21:36:16.835238 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:36:16.835238 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:36:16.835238 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:36:16.835238 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:36:16.835238 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:36:16.835238 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:36:16.835238 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: da2248fa190342f0bfe9211cae1e403b Jul 16 21:36:16.835238 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-ServersNegativeTestJSON-1102403437 Jul 16 21:36:16.835238 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:36:16.835238 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:36:16.835238 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/da2248fa190342f0bfe9211cae1e403b"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/da2248fa190342f0bfe9211cae1e403b"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_da2248fa190342f0bfe9211cae1e403b"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/da2248fa190342f0bfe9211cae1e403b"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:36:16.835238 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-ServersNegativeTestJSON-1102403437 Jul 16 21:36:16.835238 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: da2248fa190342f0bfe9211cae1e403b Jul 16 21:36:16.835238 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-ServersNegativeTestJSON-1102403437 Jul 16 21:36:16.835238 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-ServersNegativeTestJSON-1102403437-project Jul 16 21:36:16.835238 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:36:16.835238 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:36:16.835238 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: 22e264440fff43cdb7be6702839af37f Jul 16 21:36:16.835238 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-ServersNegativeTestJSON-1102403437-project Jul 16 21:36:16.835238 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:36:16.835238 ncdv-master-1 devstack@n-api.service[138681]: exception: Instance c77c9a95-ea71-40d7-b7a7-ded612d194da could not be found. Jul 16 21:36:16.835238 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': 'c77c9a95-ea71-40d7-b7a7-ded612d194da'} Jul 16 21:36:16.835238 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:36:16.835238 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:36:16.835238 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:36:16.835238 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:36:16.835238 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:36:16.835238 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:36:16.835238 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:36:16.835238 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:36:16.835238 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:36:16.835238 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:36:16.835238 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance c77c9a95-ea71-40d7-b7a7-ded612d194da could not be found. Jul 16 21:36:16.835238 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:36:16.835238 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:36:16.835238 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:36:16.835238 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:36:16.835238 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:36:16.835238 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:36:16.835238 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:36:16.835238 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:36:16.835238 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:36:16.835238 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:36:16.835238 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:36:16.835238 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:36:16.835238 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:36:16.835238 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:36:16.835238 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:36:16.835238 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:36:16.835238 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance c77c9a95-ea71-40d7-b7a7-ded612d194da could not be found. Jul 16 21:36:16.835238 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:36:16.835238 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:36:16.835238 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:36:16.838947 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:36:16.838947 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:36:16.838947 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:36:16.838947 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:36:16.838947 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:36:16.838947 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance c77c9a95-ea71-40d7-b7a7-ded612d194da could not be found. Jul 16 21:36:16.838947 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:36:16.838947 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-3859d22b-2860-4748-896b-fdd18d22452c tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Returning 404 to user: Instance c77c9a95-ea71-40d7-b7a7-ded612d194da could not be found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:36:16.838947 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-3859d22b-2860-4748-896b-fdd18d22452c tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "GET /compute/v2.1/servers/c77c9a95-ea71-40d7-b7a7-ded612d194da" status: 404 len: 111 microversion: 2.1 time: 0.032664 Jul 16 21:36:16.838947 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2570/5136] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:36:16 2022] GET /compute/v2.1/servers/c77c9a95-ea71-40d7-b7a7-ded612d194da => generated 111 bytes in 34 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:36:16.846474 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-c5c4311b-ff60-45ff-9a22-82f47cf26bbe tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:36:16.847289 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-c5c4311b-ff60-45ff-9a22-82f47cf26bbe tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: e4c7a9cb-e55d-4819-9885-2e72deffddb1] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:36:16.860257 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c5c4311b-ff60-45ff-9a22-82f47cf26bbe tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:36:16.861180 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c5c4311b-ff60-45ff-9a22-82f47cf26bbe tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:36:16.862089 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c5c4311b-ff60-45ff-9a22-82f47cf26bbe tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:36:16.862750 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c5c4311b-ff60-45ff-9a22-82f47cf26bbe tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:36:16.892782 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-c5c4311b-ff60-45ff-9a22-82f47cf26bbe tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: e4c7a9cb-e55d-4819-9885-2e72deffddb1] Going to try to terminate instance {{(pid=138680) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 16 21:36:17.203175 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-c5c4311b-ff60-45ff-9a22-82f47cf26bbe tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "DELETE /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1" status: 204 len: 0 microversion: 2.1 time: 0.360784 Jul 16 21:36:17.205165 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2567/5137] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:36:16 2022] DELETE /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 16 21:36:17.212933 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-83f4348a-1bb7-4be7-826a-1b9352404b6d tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:36:17.213550 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-83f4348a-1bb7-4be7-826a-1b9352404b6d tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: e4c7a9cb-e55d-4819-9885-2e72deffddb1] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:36:17.220308 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-83f4348a-1bb7-4be7-826a-1b9352404b6d tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:36:17.220759 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-83f4348a-1bb7-4be7-826a-1b9352404b6d tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:36:17.221526 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-83f4348a-1bb7-4be7-826a-1b9352404b6d tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:36:17.221960 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-83f4348a-1bb7-4be7-826a-1b9352404b6d tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:36:17.363362 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-83f4348a-1bb7-4be7-826a-1b9352404b6d tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '22e264440fff43cdb7be6702839af37f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da2248fa190342f0bfe9211cae1e403b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:36:17.372121 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-83f4348a-1bb7-4be7-826a-1b9352404b6d tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1" status: 200 len: 1510 microversion: 2.1 time: 0.164105 Jul 16 21:36:17.372682 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2571/5138] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:36:17 2022] GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1 => generated 1510 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:36:17.645579 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-38ec3282-8ddb-4171-9063-dbbbe0a6fb59 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:36:17.647104 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-38ec3282-8ddb-4171-9063-dbbbe0a6fb59 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] [instance: 19df5c33-44d4-4de9-947b-4707b78af23a] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:36:17.653969 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-38ec3282-8ddb-4171-9063-dbbbe0a6fb59 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:36:17.654431 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-38ec3282-8ddb-4171-9063-dbbbe0a6fb59 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:36:17.655243 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-38ec3282-8ddb-4171-9063-dbbbe0a6fb59 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:36:17.655691 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-38ec3282-8ddb-4171-9063-dbbbe0a6fb59 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:36:17.843396 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-38ec3282-8ddb-4171-9063-dbbbe0a6fb59 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '726b46c54e9541fbb7825607bac3abe5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fd0ce287cb44cd59045868ce688909a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:36:17.852705 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-38ec3282-8ddb-4171-9063-dbbbe0a6fb59 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '726b46c54e9541fbb7825607bac3abe5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fd0ce287cb44cd59045868ce688909a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:36:17.854257 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-38ec3282-8ddb-4171-9063-dbbbe0a6fb59 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '726b46c54e9541fbb7825607bac3abe5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fd0ce287cb44cd59045868ce688909a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:36:17.855699 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-38ec3282-8ddb-4171-9063-dbbbe0a6fb59 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] 10.222.0.16 "GET /compute/v2.1/servers/19df5c33-44d4-4de9-947b-4707b78af23a" status: 200 len: 1874 microversion: 2.42 time: 0.214475 Jul 16 21:36:17.856471 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2568/5139] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:36:17 2022] GET /compute/v2.1/servers/19df5c33-44d4-4de9-947b-4707b78af23a => generated 1874 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:36:18.392289 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-cb0fe9da-fd00-4b71-bdcc-2a794246e424 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:36:18.392289 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-cb0fe9da-fd00-4b71-bdcc-2a794246e424 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: e4c7a9cb-e55d-4819-9885-2e72deffddb1] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:36:18.399493 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cb0fe9da-fd00-4b71-bdcc-2a794246e424 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:36:18.399953 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cb0fe9da-fd00-4b71-bdcc-2a794246e424 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:36:18.400728 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cb0fe9da-fd00-4b71-bdcc-2a794246e424 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:36:18.401167 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cb0fe9da-fd00-4b71-bdcc-2a794246e424 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:36:18.556535 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-cb0fe9da-fd00-4b71-bdcc-2a794246e424 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '22e264440fff43cdb7be6702839af37f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da2248fa190342f0bfe9211cae1e403b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:36:18.564891 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-cb0fe9da-fd00-4b71-bdcc-2a794246e424 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1" status: 200 len: 1510 microversion: 2.1 time: 0.178893 Jul 16 21:36:18.565341 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2572/5140] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:36:18 2022] GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1 => generated 1510 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:36:18.875863 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-d616ea7a-25ba-40aa-8f31-dc46c438f078 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:36:18.876604 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-d616ea7a-25ba-40aa-8f31-dc46c438f078 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] [instance: 19df5c33-44d4-4de9-947b-4707b78af23a] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:36:18.884740 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d616ea7a-25ba-40aa-8f31-dc46c438f078 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:36:18.885204 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d616ea7a-25ba-40aa-8f31-dc46c438f078 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:36:18.885992 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d616ea7a-25ba-40aa-8f31-dc46c438f078 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:36:18.886451 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d616ea7a-25ba-40aa-8f31-dc46c438f078 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:36:19.113723 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-d616ea7a-25ba-40aa-8f31-dc46c438f078 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '726b46c54e9541fbb7825607bac3abe5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fd0ce287cb44cd59045868ce688909a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:36:19.126184 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-d616ea7a-25ba-40aa-8f31-dc46c438f078 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '726b46c54e9541fbb7825607bac3abe5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fd0ce287cb44cd59045868ce688909a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:36:19.127736 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-d616ea7a-25ba-40aa-8f31-dc46c438f078 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '726b46c54e9541fbb7825607bac3abe5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fd0ce287cb44cd59045868ce688909a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:36:19.128917 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-d616ea7a-25ba-40aa-8f31-dc46c438f078 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] 10.222.0.16 "GET /compute/v2.1/servers/19df5c33-44d4-4de9-947b-4707b78af23a" status: 200 len: 1874 microversion: 2.42 time: 0.257727 Jul 16 21:36:19.130117 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2569/5141] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:36:18 2022] GET /compute/v2.1/servers/19df5c33-44d4-4de9-947b-4707b78af23a => generated 1874 bytes in 259 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:36:19.581929 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-b5ce74ae-40b5-49a9-978a-e1164afec9c5 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:36:19.584789 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-b5ce74ae-40b5-49a9-978a-e1164afec9c5 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: e4c7a9cb-e55d-4819-9885-2e72deffddb1] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:36:19.591783 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b5ce74ae-40b5-49a9-978a-e1164afec9c5 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:36:19.592312 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b5ce74ae-40b5-49a9-978a-e1164afec9c5 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:36:19.593538 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b5ce74ae-40b5-49a9-978a-e1164afec9c5 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:36:19.594859 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b5ce74ae-40b5-49a9-978a-e1164afec9c5 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:36:19.819420 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-b5ce74ae-40b5-49a9-978a-e1164afec9c5 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '22e264440fff43cdb7be6702839af37f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da2248fa190342f0bfe9211cae1e403b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:36:19.830557 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-b5ce74ae-40b5-49a9-978a-e1164afec9c5 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1" status: 200 len: 1510 microversion: 2.1 time: 0.252580 Jul 16 21:36:19.831273 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2573/5142] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:36:19 2022] GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1 => generated 1510 bytes in 254 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:36:20.106619 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-aa384e27-11a7-4773-8f11-d07058a8a413 req-2874fc15-5170-439a-94ec-cb1545c2e77f service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "e4c7a9cb-e55d-4819-9885-2e72deffddb1", "name": "network-vif-deleted", "tag": "0b459b05-0d03-4317-8a9b-bae73b0eaddd"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:36:20.116651 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-aa384e27-11a7-4773-8f11-d07058a8a413 req-2874fc15-5170-439a-94ec-cb1545c2e77f service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:36:20.117110 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-aa384e27-11a7-4773-8f11-d07058a8a413 req-2874fc15-5170-439a-94ec-cb1545c2e77f service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:36:20.129100 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-aa384e27-11a7-4773-8f11-d07058a8a413 req-2874fc15-5170-439a-94ec-cb1545c2e77f service nova] Creating event network-vif-deleted:0b459b05-0d03-4317-8a9b-bae73b0eaddd for instance e4c7a9cb-e55d-4819-9885-2e72deffddb1 on nch2-master-1 Jul 16 21:36:20.134241 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-aa384e27-11a7-4773-8f11-d07058a8a413 req-2874fc15-5170-439a-94ec-cb1545c2e77f service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.031764 Jul 16 21:36:20.134846 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2570/5143] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:36:20 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:36:20.148443 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-dd2fc7f8-9bcb-47bc-b7f3-1637f894896c tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:36:20.149628 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-dd2fc7f8-9bcb-47bc-b7f3-1637f894896c tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] [instance: 19df5c33-44d4-4de9-947b-4707b78af23a] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:36:20.156383 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-dd2fc7f8-9bcb-47bc-b7f3-1637f894896c tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:36:20.156845 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-dd2fc7f8-9bcb-47bc-b7f3-1637f894896c tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:36:20.157693 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-dd2fc7f8-9bcb-47bc-b7f3-1637f894896c tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:36:20.158177 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-dd2fc7f8-9bcb-47bc-b7f3-1637f894896c tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:36:20.351621 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-dd2fc7f8-9bcb-47bc-b7f3-1637f894896c tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '726b46c54e9541fbb7825607bac3abe5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fd0ce287cb44cd59045868ce688909a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:36:20.361267 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-dd2fc7f8-9bcb-47bc-b7f3-1637f894896c tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '726b46c54e9541fbb7825607bac3abe5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fd0ce287cb44cd59045868ce688909a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:36:20.363598 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-dd2fc7f8-9bcb-47bc-b7f3-1637f894896c tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '726b46c54e9541fbb7825607bac3abe5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fd0ce287cb44cd59045868ce688909a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:36:20.364999 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-dd2fc7f8-9bcb-47bc-b7f3-1637f894896c tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] 10.222.0.16 "GET /compute/v2.1/servers/19df5c33-44d4-4de9-947b-4707b78af23a" status: 200 len: 1874 microversion: 2.42 time: 0.220706 Jul 16 21:36:20.381223 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2574/5144] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:36:20 2022] GET /compute/v2.1/servers/19df5c33-44d4-4de9-947b-4707b78af23a => generated 1874 bytes in 233 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:36:20.849590 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-c083e358-23bd-497d-bf0f-7cc07cf5c115 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:36:20.852739 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-c083e358-23bd-497d-bf0f-7cc07cf5c115 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: e4c7a9cb-e55d-4819-9885-2e72deffddb1] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:36:20.857201 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c083e358-23bd-497d-bf0f-7cc07cf5c115 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:36:20.857656 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c083e358-23bd-497d-bf0f-7cc07cf5c115 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:36:20.858675 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c083e358-23bd-497d-bf0f-7cc07cf5c115 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:36:20.859338 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c083e358-23bd-497d-bf0f-7cc07cf5c115 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:36:20.898804 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c083e358-23bd-497d-bf0f-7cc07cf5c115 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:36:20.898804 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c083e358-23bd-497d-bf0f-7cc07cf5c115 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:36:20.898804 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.objects.instance [None req-c083e358-23bd-497d-bf0f-7cc07cf5c115 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lazy-loading 'fault' on Instance uuid e4c7a9cb-e55d-4819-9885-2e72deffddb1 {{(pid=138680) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:36:20.982380 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-c083e358-23bd-497d-bf0f-7cc07cf5c115 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '22e264440fff43cdb7be6702839af37f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da2248fa190342f0bfe9211cae1e403b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:36:20.991697 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-c083e358-23bd-497d-bf0f-7cc07cf5c115 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1" status: 200 len: 1306 microversion: 2.1 time: 0.146884 Jul 16 21:36:20.992427 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2571/5145] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:36:20 2022] GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1 => generated 1306 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:36:21.384966 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-10d633c4-fde4-4e01-9f9c-d0a0d99ae6ab tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:36:21.385769 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-10d633c4-fde4-4e01-9f9c-d0a0d99ae6ab tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] [instance: 19df5c33-44d4-4de9-947b-4707b78af23a] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:36:21.393869 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-10d633c4-fde4-4e01-9f9c-d0a0d99ae6ab tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:36:21.394328 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-10d633c4-fde4-4e01-9f9c-d0a0d99ae6ab tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:36:21.395424 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-10d633c4-fde4-4e01-9f9c-d0a0d99ae6ab tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:36:21.396659 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-10d633c4-fde4-4e01-9f9c-d0a0d99ae6ab tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:36:21.601484 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-10d633c4-fde4-4e01-9f9c-d0a0d99ae6ab tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '726b46c54e9541fbb7825607bac3abe5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fd0ce287cb44cd59045868ce688909a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:36:21.621513 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-10d633c4-fde4-4e01-9f9c-d0a0d99ae6ab tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '726b46c54e9541fbb7825607bac3abe5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fd0ce287cb44cd59045868ce688909a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:36:21.623052 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-10d633c4-fde4-4e01-9f9c-d0a0d99ae6ab tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '726b46c54e9541fbb7825607bac3abe5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fd0ce287cb44cd59045868ce688909a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:36:21.624340 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-10d633c4-fde4-4e01-9f9c-d0a0d99ae6ab tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] 10.222.0.16 "GET /compute/v2.1/servers/19df5c33-44d4-4de9-947b-4707b78af23a" status: 200 len: 2833 microversion: 2.42 time: 0.244263 Jul 16 21:36:21.624914 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2575/5146] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:36:21 2022] GET /compute/v2.1/servers/19df5c33-44d4-4de9-947b-4707b78af23a => generated 2833 bytes in 245 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:36:21.645013 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-c6a81de4-b072-4afe-add2-2c1326c4a6c3 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:36:21.645669 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-c6a81de4-b072-4afe-add2-2c1326c4a6c3 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] [instance: 19df5c33-44d4-4de9-947b-4707b78af23a] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:36:21.653429 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c6a81de4-b072-4afe-add2-2c1326c4a6c3 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:36:21.653885 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c6a81de4-b072-4afe-add2-2c1326c4a6c3 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:36:21.654670 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c6a81de4-b072-4afe-add2-2c1326c4a6c3 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:36:21.655162 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c6a81de4-b072-4afe-add2-2c1326c4a6c3 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:36:21.780456 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-c6a81de4-b072-4afe-add2-2c1326c4a6c3 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] 10.222.0.16 "GET /compute/v2.1/servers/19df5c33-44d4-4de9-947b-4707b78af23a/os-interface" status: 200 len: 1832 microversion: 2.42 time: 0.139451 Jul 16 21:36:21.781189 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2572/5147] 10.222.0.16 () {60 vars in 1315 bytes} [Sat Jul 16 21:36:21 2022] GET /compute/v2.1/servers/19df5c33-44d4-4de9-947b-4707b78af23a/os-interface => generated 1832 bytes in 141 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:36:22.015030 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-967bcd07-13e8-4097-8015-100973dbc0f2 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:36:22.015884 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-967bcd07-13e8-4097-8015-100973dbc0f2 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] [instance: e4c7a9cb-e55d-4819-9885-2e72deffddb1] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:36:22.022519 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-967bcd07-13e8-4097-8015-100973dbc0f2 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:36:22.023323 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-967bcd07-13e8-4097-8015-100973dbc0f2 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:36:22.024256 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-967bcd07-13e8-4097-8015-100973dbc0f2 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:36:22.024838 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-967bcd07-13e8-4097-8015-100973dbc0f2 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:36:22.040628 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-967bcd07-13e8-4097-8015-100973dbc0f2 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] HTTP exception thrown: Instance e4c7a9cb-e55d-4819-9885-2e72deffddb1 could not be found. Jul 16 21:36:22.049893 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-967bcd07-13e8-4097-8015-100973dbc0f2 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Request method failure captured: Jul 16 21:36:22.049893 ncdv-master-1 devstack@n-api.service[138681]: request: GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1 HTTP/1.1 Jul 16 21:36:22.049893 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:36:22.049893 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:36:22.049893 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:36:22.049893 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 21:36:22.049893 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:36:22.049893 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:36:22.049893 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:36:22.049893 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:36:22.049893 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0y6QJDVdPNcLbUnwwgOJV-QkddLQ7yqnzBNVp2CrOe5UQqjQOER3spl3PswEW82ISSx8FhmuIrEwFSG69Ze1pTOmWBWZ8PURIgVHbN3yBPA3U2S2TKkQkFwyNuvvV03MCr9U0QGJIjXVOkiqFEAy8q87U_okZgCdI-BFjBFgrCdfAEo Jul 16 21:36:22.049893 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:36:22.049893 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:36:22.049893 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:36:22.049893 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:36:22.049893 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:36:22.049893 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:36:22.049893 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: da2248fa190342f0bfe9211cae1e403b Jul 16 21:36:22.049893 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-ServersNegativeTestJSON-1102403437 Jul 16 21:36:22.049893 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:36:22.049893 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:36:22.049893 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/da2248fa190342f0bfe9211cae1e403b"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/da2248fa190342f0bfe9211cae1e403b"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_da2248fa190342f0bfe9211cae1e403b"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/da2248fa190342f0bfe9211cae1e403b"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:36:22.049893 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-ServersNegativeTestJSON-1102403437 Jul 16 21:36:22.049893 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: da2248fa190342f0bfe9211cae1e403b Jul 16 21:36:22.049893 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-ServersNegativeTestJSON-1102403437 Jul 16 21:36:22.049893 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-ServersNegativeTestJSON-1102403437-project Jul 16 21:36:22.049893 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:36:22.049893 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:36:22.049893 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: 22e264440fff43cdb7be6702839af37f Jul 16 21:36:22.049893 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-ServersNegativeTestJSON-1102403437-project Jul 16 21:36:22.049893 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:36:22.049893 ncdv-master-1 devstack@n-api.service[138681]: exception: Instance e4c7a9cb-e55d-4819-9885-2e72deffddb1 could not be found. Jul 16 21:36:22.049893 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': 'e4c7a9cb-e55d-4819-9885-2e72deffddb1'} Jul 16 21:36:22.049893 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:36:22.049893 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:36:22.049893 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:36:22.049893 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:36:22.049893 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:36:22.049893 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:36:22.049893 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:36:22.049893 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:36:22.049893 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:36:22.049893 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:36:22.049893 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance e4c7a9cb-e55d-4819-9885-2e72deffddb1 could not be found. Jul 16 21:36:22.049893 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:36:22.049893 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:36:22.049893 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:36:22.049893 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:36:22.049893 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:36:22.049893 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:36:22.049893 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:36:22.049893 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:36:22.049893 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:36:22.049893 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:36:22.049893 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:36:22.049893 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:36:22.049893 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:36:22.049893 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:36:22.049893 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:36:22.049893 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:36:22.049893 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance e4c7a9cb-e55d-4819-9885-2e72deffddb1 could not be found. Jul 16 21:36:22.049893 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:36:22.049893 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:36:22.049893 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:36:22.054504 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:36:22.054504 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:36:22.054504 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:36:22.054504 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:36:22.054504 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:36:22.054504 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance e4c7a9cb-e55d-4819-9885-2e72deffddb1 could not be found. Jul 16 21:36:22.054504 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:36:22.056488 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-967bcd07-13e8-4097-8015-100973dbc0f2 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] Returning 404 to user: Instance e4c7a9cb-e55d-4819-9885-2e72deffddb1 could not be found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:36:22.057492 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-967bcd07-13e8-4097-8015-100973dbc0f2 tempest-ServersNegativeTestJSON-1102403437 tempest-ServersNegativeTestJSON-1102403437-project] 10.222.0.16 "GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1" status: 404 len: 111 microversion: 2.1 time: 0.047287 Jul 16 21:36:22.058408 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2576/5148] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:36:22 2022] GET /compute/v2.1/servers/e4c7a9cb-e55d-4819-9885-2e72deffddb1 => generated 111 bytes in 49 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:36:22.963392 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-806d97b6-5f2d-43e0-ba6d-983a4712f8d7 req-be5a2322-16d2-489b-86a6-53769f99c666 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "19df5c33-44d4-4de9-947b-4707b78af23a", "tag": "7d150f5e-be69-4d03-aff9-6fc347935783"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:36:22.982913 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-806d97b6-5f2d-43e0-ba6d-983a4712f8d7 req-be5a2322-16d2-489b-86a6-53769f99c666 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:36:22.983394 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-806d97b6-5f2d-43e0-ba6d-983a4712f8d7 req-be5a2322-16d2-489b-86a6-53769f99c666 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:36:23.009200 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-806d97b6-5f2d-43e0-ba6d-983a4712f8d7 req-be5a2322-16d2-489b-86a6-53769f99c666 service nova] Creating event network-changed:7d150f5e-be69-4d03-aff9-6fc347935783 for instance 19df5c33-44d4-4de9-947b-4707b78af23a on nch2-master-1 Jul 16 21:36:23.018581 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-806d97b6-5f2d-43e0-ba6d-983a4712f8d7 req-be5a2322-16d2-489b-86a6-53769f99c666 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.059602 Jul 16 21:36:23.024503 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2573/5149] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:36:22 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 66 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 16 21:36:29.014212 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-f44c99c0-4a54-4bef-beb4-d504c4d5d108 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:36:29.015933 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-f44c99c0-4a54-4bef-beb4-d504c4d5d108 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] [instance: 4df09776-dc3c-46ff-9e64-e20b5d4679fa] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:36:29.025867 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_api:SELECT=56 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:36:29.027796 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f44c99c0-4a54-4bef-beb4-d504c4d5d108 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:36:29.036017 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f44c99c0-4a54-4bef-beb4-d504c4d5d108 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:36:29.036017 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f44c99c0-4a54-4bef-beb4-d504c4d5d108 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:36:29.036017 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f44c99c0-4a54-4bef-beb4-d504c4d5d108 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:36:29.063081 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-f44c99c0-4a54-4bef-beb4-d504c4d5d108 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] 10.222.0.16 "GET /compute/v2.1/servers/4df09776-dc3c-46ff-9e64-e20b5d4679fa/ips" status: 200 len: 151 microversion: 2.1 time: 0.052515 Jul 16 21:36:29.064082 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2577/5150] 10.222.0.16 () {58 vars in 1247 bytes} [Sat Jul 16 21:36:29 2022] GET /compute/v2.1/servers/4df09776-dc3c-46ff-9e64-e20b5d4679fa/ips => generated 151 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:36:31.999123 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-4c63b404-f927-4cde-a423-a2b69ca91ad2 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "64cfebc6-4a34-42c3-8e38-188693d74ff7"}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:36:32.000392 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-4c63b404-f927-4cde-a423-a2b69ca91ad2 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] [instance: 4df09776-dc3c-46ff-9e64-e20b5d4679fa] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:36:32.008443 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4c63b404-f927-4cde-a423-a2b69ca91ad2 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:36:32.009013 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4c63b404-f927-4cde-a423-a2b69ca91ad2 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:36:32.009802 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4c63b404-f927-4cde-a423-a2b69ca91ad2 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:36:32.010248 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4c63b404-f927-4cde-a423-a2b69ca91ad2 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:36:32.970074 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-4c63b404-f927-4cde-a423-a2b69ca91ad2 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] 10.222.0.16 "POST /compute/v2.1/servers/4df09776-dc3c-46ff-9e64-e20b5d4679fa/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.976290 Jul 16 21:36:32.970773 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2574/5151] 10.222.0.16 () {60 vars in 1322 bytes} [Sat Jul 16 21:36:31 2022] POST /compute/v2.1/servers/4df09776-dc3c-46ff-9e64-e20b5d4679fa/os-volume_attachments => generated 194 bytes in 978 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:36:37.369730 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-60a8c3fb-11ce-4dc6-bec2-ac7b05586e69 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Action: 'create', calling method: .version_select at 0x7fdd9c4d58b0>, body: {"keypair": {"name": "tempest-keypair-476890770", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBICRUC59ZjKYn2ld1Wffd0Lgr+ogXzIzlSgVfVCRJGMIyW2oKxw+R8qK2w9mitkT0nqNDQROQnyM4OMacwM6aQRjdeFpwKJsR1bdn68VOFGnmbHphlq6moTw2t7G6CZ9cw=="}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:36:37.386097 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-60a8c3fb-11ce-4dc6-bec2-ac7b05586e69 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Getting quotas for project e60c0863dc6f4313bdc873fda27ae2fc. Resources: {'key_pairs'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:36:37.392093 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-60a8c3fb-11ce-4dc6-bec2-ac7b05586e69 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Getting quotas for user 0ab8b3f06e994bbca5d6dd14109bfe50 and project e60c0863dc6f4313bdc873fda27ae2fc. Resources: {'key_pairs'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:36:37.534455 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-60a8c3fb-11ce-4dc6-bec2-ac7b05586e69 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] 10.222.0.16 "POST /compute/v2.1/os-keypairs" status: 200 len: 385 microversion: 2.1 time: 0.171417 Jul 16 21:36:37.535323 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2578/5152] 10.222.0.16 () {60 vars in 1158 bytes} [Sat Jul 16 21:36:37 2022] POST /compute/v2.1/os-keypairs => generated 385 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:36:40.645467 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-094d8730-f876-490f-8ed1-785245053419 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeNegativeTest-server-527705356", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "4775de54-1ba2-4d76-a6e0-a5f8a9ef879c"}], "security_groups": [{"name": "tempest-securitygroup--937037585"}], "key_name": "tempest-keypair-476890770", "user_data": "}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:36:40.772270 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.network.neutron [None req-094d8730-f876-490f-8ed1-785245053419 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] validate_networks() for [('4775de54-1ba2-4d76-a6e0-a5f8a9ef879c', None, None, None, None, None)] {{(pid=138680) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 16 21:36:41.246407 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-094d8730-f876-490f-8ed1-785245053419 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Flavor limits 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 16 21:36:41.246842 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-094d8730-f876-490f-8ed1-785245053419 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Image limits 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 16 21:36:41.247732 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-094d8730-f876-490f-8ed1-785245053419 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Flavor pref 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 16 21:36:41.252675 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-094d8730-f876-490f-8ed1-785245053419 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Image pref 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 16 21:36:41.252675 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-094d8730-f876-490f-8ed1-785245053419 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 16 21:36:41.259035 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-094d8730-f876-490f-8ed1-785245053419 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:36:41.259516 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-094d8730-f876-490f-8ed1-785245053419 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:36:41.260319 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-094d8730-f876-490f-8ed1-785245053419 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:36:41.260757 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-094d8730-f876-490f-8ed1-785245053419 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:36:41.287455 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-094d8730-f876-490f-8ed1-785245053419 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Getting quotas for project e60c0863dc6f4313bdc873fda27ae2fc. Resources: {'ram', 'instances', 'cores'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:36:41.293989 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-094d8730-f876-490f-8ed1-785245053419 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Getting quotas for user 0ab8b3f06e994bbca5d6dd14109bfe50 and project e60c0863dc6f4313bdc873fda27ae2fc. Resources: {'ram', 'instances', 'cores'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:36:41.311086 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-094d8730-f876-490f-8ed1-785245053419 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Going to run 1 instances... {{(pid=138680) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 16 21:36:41.316703 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-094d8730-f876-490f-8ed1-785245053419 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] [instance: a8f70d9d-86c1-469f-bb32-a37c680cc80c] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138680) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 16 21:36:41.537626 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-094d8730-f876-490f-8ed1-785245053419 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 401 microversion: 2.1 time: 0.897318 Jul 16 21:36:41.539525 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2575/5153] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 21:36:40 2022] POST /compute/v2.1/servers => generated 401 bytes in 901 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 16 21:36:41.549340 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=79 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:36:41.556759 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-284112ed-acda-43a6-a590-9c9fa558f7b1 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:36:41.561020 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-284112ed-acda-43a6-a590-9c9fa558f7b1 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] [instance: a8f70d9d-86c1-469f-bb32-a37c680cc80c] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:36:41.644966 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-284112ed-acda-43a6-a590-9c9fa558f7b1 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0ab8b3f06e994bbca5d6dd14109bfe50', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e60c0863dc6f4313bdc873fda27ae2fc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:36:41.652522 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-284112ed-acda-43a6-a590-9c9fa558f7b1 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] 10.222.0.16 "GET /compute/v2.1/servers/a8f70d9d-86c1-469f-bb32-a37c680cc80c" status: 200 len: 1240 microversion: 2.1 time: 0.106412 Jul 16 21:36:41.653580 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2579/5154] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:36:41 2022] GET /compute/v2.1/servers/a8f70d9d-86c1-469f-bb32-a37c680cc80c => generated 1240 bytes in 108 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:36:42.677643 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-3a1a6b00-06d2-4737-8920-e6bdeef6b1b5 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:36:42.678275 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-3a1a6b00-06d2-4737-8920-e6bdeef6b1b5 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] [instance: a8f70d9d-86c1-469f-bb32-a37c680cc80c] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:36:42.711706 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3a1a6b00-06d2-4737-8920-e6bdeef6b1b5 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:36:42.712416 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3a1a6b00-06d2-4737-8920-e6bdeef6b1b5 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:36:42.713478 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3a1a6b00-06d2-4737-8920-e6bdeef6b1b5 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:36:42.713936 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3a1a6b00-06d2-4737-8920-e6bdeef6b1b5 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:36:42.718515 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=89,nova_cell1:UPDATE=3,nova_cell1:INSERT=2 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:36:42.779534 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-3a1a6b00-06d2-4737-8920-e6bdeef6b1b5 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0ab8b3f06e994bbca5d6dd14109bfe50', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e60c0863dc6f4313bdc873fda27ae2fc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:36:42.789040 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-3a1a6b00-06d2-4737-8920-e6bdeef6b1b5 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] 10.222.0.16 "GET /compute/v2.1/servers/a8f70d9d-86c1-469f-bb32-a37c680cc80c" status: 200 len: 1244 microversion: 2.1 time: 0.122179 Jul 16 21:36:42.789895 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2576/5155] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:36:42 2022] GET /compute/v2.1/servers/a8f70d9d-86c1-469f-bb32-a37c680cc80c => generated 1244 bytes in 124 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:36:43.808624 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-2a5170c9-ad5f-4d58-85d6-65e97a98393b tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:36:43.809434 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-2a5170c9-ad5f-4d58-85d6-65e97a98393b tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] [instance: a8f70d9d-86c1-469f-bb32-a37c680cc80c] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:36:43.818801 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2a5170c9-ad5f-4d58-85d6-65e97a98393b tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:36:43.820403 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2a5170c9-ad5f-4d58-85d6-65e97a98393b tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:36:43.821258 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2a5170c9-ad5f-4d58-85d6-65e97a98393b tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:36:43.821700 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2a5170c9-ad5f-4d58-85d6-65e97a98393b tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:36:43.894299 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-2a5170c9-ad5f-4d58-85d6-65e97a98393b tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0ab8b3f06e994bbca5d6dd14109bfe50', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e60c0863dc6f4313bdc873fda27ae2fc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:36:43.901908 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-2a5170c9-ad5f-4d58-85d6-65e97a98393b tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] 10.222.0.16 "GET /compute/v2.1/servers/a8f70d9d-86c1-469f-bb32-a37c680cc80c" status: 200 len: 1292 microversion: 2.1 time: 0.098407 Jul 16 21:36:43.902476 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2580/5156] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:36:43 2022] GET /compute/v2.1/servers/a8f70d9d-86c1-469f-bb32-a37c680cc80c => generated 1292 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:36:44.919142 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-a4f199f5-1351-4caf-9271-5c8d58ad42f1 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:36:44.920285 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-a4f199f5-1351-4caf-9271-5c8d58ad42f1 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] [instance: a8f70d9d-86c1-469f-bb32-a37c680cc80c] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:36:44.929445 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a4f199f5-1351-4caf-9271-5c8d58ad42f1 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:36:44.930100 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a4f199f5-1351-4caf-9271-5c8d58ad42f1 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:36:44.931059 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a4f199f5-1351-4caf-9271-5c8d58ad42f1 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:36:44.931738 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a4f199f5-1351-4caf-9271-5c8d58ad42f1 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:36:45.155243 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-a4f199f5-1351-4caf-9271-5c8d58ad42f1 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0ab8b3f06e994bbca5d6dd14109bfe50', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e60c0863dc6f4313bdc873fda27ae2fc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:36:45.174346 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-a4f199f5-1351-4caf-9271-5c8d58ad42f1 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] 10.222.0.16 "GET /compute/v2.1/servers/a8f70d9d-86c1-469f-bb32-a37c680cc80c" status: 200 len: 1365 microversion: 2.1 time: 0.252246 Jul 16 21:36:45.175231 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2577/5157] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:36:44 2022] GET /compute/v2.1/servers/a8f70d9d-86c1-469f-bb32-a37c680cc80c => generated 1365 bytes in 261 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:36:46.194990 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-6313bb72-09a6-4c30-bb0a-b5ad65f36437 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:36:46.196469 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-6313bb72-09a6-4c30-bb0a-b5ad65f36437 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] [instance: a8f70d9d-86c1-469f-bb32-a37c680cc80c] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:36:46.205369 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6313bb72-09a6-4c30-bb0a-b5ad65f36437 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:36:46.206023 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6313bb72-09a6-4c30-bb0a-b5ad65f36437 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:36:46.207525 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6313bb72-09a6-4c30-bb0a-b5ad65f36437 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:36:46.208216 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6313bb72-09a6-4c30-bb0a-b5ad65f36437 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:36:46.308544 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-e83bf169-4efc-47d3-9cdc-9f53356cc1f6 req-bd787aa2-fd00-44c6-8904-864840b121ac service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "a8f70d9d-86c1-469f-bb32-a37c680cc80c", "tag": "b54ca2a7-6d58-476e-b078-7c95d8ac7fed"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:36:46.328909 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-e83bf169-4efc-47d3-9cdc-9f53356cc1f6 req-bd787aa2-fd00-44c6-8904-864840b121ac service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:36:46.330948 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-e83bf169-4efc-47d3-9cdc-9f53356cc1f6 req-bd787aa2-fd00-44c6-8904-864840b121ac service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:36:46.353676 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-e83bf169-4efc-47d3-9cdc-9f53356cc1f6 req-bd787aa2-fd00-44c6-8904-864840b121ac service nova] Creating event network-changed:b54ca2a7-6d58-476e-b078-7c95d8ac7fed for instance a8f70d9d-86c1-469f-bb32-a37c680cc80c on nch1-master-1 Jul 16 21:36:46.364413 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-e83bf169-4efc-47d3-9cdc-9f53356cc1f6 req-bd787aa2-fd00-44c6-8904-864840b121ac service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.328107 Jul 16 21:36:46.365725 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2581/5158] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:36:46 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 330 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:36:46.460313 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-6313bb72-09a6-4c30-bb0a-b5ad65f36437 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0ab8b3f06e994bbca5d6dd14109bfe50', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e60c0863dc6f4313bdc873fda27ae2fc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:36:46.481651 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-6313bb72-09a6-4c30-bb0a-b5ad65f36437 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] 10.222.0.16 "GET /compute/v2.1/servers/a8f70d9d-86c1-469f-bb32-a37c680cc80c" status: 200 len: 1365 microversion: 2.1 time: 0.292554 Jul 16 21:36:46.492584 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2578/5159] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:36:46 2022] GET /compute/v2.1/servers/a8f70d9d-86c1-469f-bb32-a37c680cc80c => generated 1365 bytes in 304 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:36:47.501033 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-792410ec-ae64-4c2b-bbd8-305bdd8fd319 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:36:47.501741 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-792410ec-ae64-4c2b-bbd8-305bdd8fd319 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] [instance: a8f70d9d-86c1-469f-bb32-a37c680cc80c] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:36:47.508908 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-792410ec-ae64-4c2b-bbd8-305bdd8fd319 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:36:47.509363 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-792410ec-ae64-4c2b-bbd8-305bdd8fd319 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:36:47.510136 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-792410ec-ae64-4c2b-bbd8-305bdd8fd319 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:36:47.510582 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-792410ec-ae64-4c2b-bbd8-305bdd8fd319 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:36:47.665176 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-792410ec-ae64-4c2b-bbd8-305bdd8fd319 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0ab8b3f06e994bbca5d6dd14109bfe50', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e60c0863dc6f4313bdc873fda27ae2fc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:36:47.672805 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-792410ec-ae64-4c2b-bbd8-305bdd8fd319 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] 10.222.0.16 "GET /compute/v2.1/servers/a8f70d9d-86c1-469f-bb32-a37c680cc80c" status: 200 len: 1365 microversion: 2.1 time: 0.177074 Jul 16 21:36:47.673374 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2582/5160] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:36:47 2022] GET /compute/v2.1/servers/a8f70d9d-86c1-469f-bb32-a37c680cc80c => generated 1365 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:36:48.690630 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-8368e83a-97b7-4251-ac2f-9dae8bfce330 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:36:48.691271 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-8368e83a-97b7-4251-ac2f-9dae8bfce330 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] [instance: a8f70d9d-86c1-469f-bb32-a37c680cc80c] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:36:48.695777 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_api:SELECT=48,nova_api:UPDATE=1,nova_api:INSERT=3 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:36:48.701481 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8368e83a-97b7-4251-ac2f-9dae8bfce330 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:36:48.701941 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8368e83a-97b7-4251-ac2f-9dae8bfce330 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:36:48.702719 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8368e83a-97b7-4251-ac2f-9dae8bfce330 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:36:48.703226 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8368e83a-97b7-4251-ac2f-9dae8bfce330 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:36:48.851856 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-8368e83a-97b7-4251-ac2f-9dae8bfce330 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0ab8b3f06e994bbca5d6dd14109bfe50', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e60c0863dc6f4313bdc873fda27ae2fc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:36:48.860792 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-8368e83a-97b7-4251-ac2f-9dae8bfce330 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] 10.222.0.16 "GET /compute/v2.1/servers/a8f70d9d-86c1-469f-bb32-a37c680cc80c" status: 200 len: 1365 microversion: 2.1 time: 0.174398 Jul 16 21:36:48.861387 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2579/5161] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:36:48 2022] GET /compute/v2.1/servers/a8f70d9d-86c1-469f-bb32-a37c680cc80c => generated 1365 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:36:49.879268 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-98270716-9f32-4f1d-9650-ef5e4eb30b83 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:36:49.881003 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-98270716-9f32-4f1d-9650-ef5e4eb30b83 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] [instance: a8f70d9d-86c1-469f-bb32-a37c680cc80c] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:36:49.888953 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-98270716-9f32-4f1d-9650-ef5e4eb30b83 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:36:49.889462 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-98270716-9f32-4f1d-9650-ef5e4eb30b83 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:36:49.890288 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-98270716-9f32-4f1d-9650-ef5e4eb30b83 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:36:49.890751 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-98270716-9f32-4f1d-9650-ef5e4eb30b83 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:36:49.924359 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-a84ea796-9e49-49d0-96fd-6ede1c1e2139 req-a6b1c178-4bd1-4f8a-852e-e232f655cb65 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "a8f70d9d-86c1-469f-bb32-a37c680cc80c", "name": "network-vif-plugged", "status": "completed", "tag": "b54ca2a7-6d58-476e-b078-7c95d8ac7fed"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:36:49.935090 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-a84ea796-9e49-49d0-96fd-6ede1c1e2139 req-a6b1c178-4bd1-4f8a-852e-e232f655cb65 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:36:49.935803 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-a84ea796-9e49-49d0-96fd-6ede1c1e2139 req-a6b1c178-4bd1-4f8a-852e-e232f655cb65 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:36:49.949026 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-a84ea796-9e49-49d0-96fd-6ede1c1e2139 req-a6b1c178-4bd1-4f8a-852e-e232f655cb65 service nova] Creating event network-vif-plugged:b54ca2a7-6d58-476e-b078-7c95d8ac7fed for instance a8f70d9d-86c1-469f-bb32-a37c680cc80c on nch1-master-1 Jul 16 21:36:49.954192 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-a84ea796-9e49-49d0-96fd-6ede1c1e2139 req-a6b1c178-4bd1-4f8a-852e-e232f655cb65 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.034439 Jul 16 21:36:49.954880 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2580/5162] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:36:49 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 16 21:36:50.045720 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-98270716-9f32-4f1d-9650-ef5e4eb30b83 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0ab8b3f06e994bbca5d6dd14109bfe50', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e60c0863dc6f4313bdc873fda27ae2fc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:36:50.056456 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-98270716-9f32-4f1d-9650-ef5e4eb30b83 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] 10.222.0.16 "GET /compute/v2.1/servers/a8f70d9d-86c1-469f-bb32-a37c680cc80c" status: 200 len: 1365 microversion: 2.1 time: 0.182591 Jul 16 21:36:50.057059 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2583/5163] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:36:49 2022] GET /compute/v2.1/servers/a8f70d9d-86c1-469f-bb32-a37c680cc80c => generated 1365 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:36:51.074914 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-c2ed8be9-6d70-447b-915e-d7d5420e5e9c tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:36:51.075693 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-c2ed8be9-6d70-447b-915e-d7d5420e5e9c tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] [instance: a8f70d9d-86c1-469f-bb32-a37c680cc80c] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:36:51.084042 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c2ed8be9-6d70-447b-915e-d7d5420e5e9c tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:36:51.084496 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c2ed8be9-6d70-447b-915e-d7d5420e5e9c tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:36:51.085323 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c2ed8be9-6d70-447b-915e-d7d5420e5e9c tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:36:51.085802 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c2ed8be9-6d70-447b-915e-d7d5420e5e9c tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:36:51.228761 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-c2ed8be9-6d70-447b-915e-d7d5420e5e9c tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0ab8b3f06e994bbca5d6dd14109bfe50', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e60c0863dc6f4313bdc873fda27ae2fc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:36:51.237545 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-c2ed8be9-6d70-447b-915e-d7d5420e5e9c tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] 10.222.0.16 "GET /compute/v2.1/servers/a8f70d9d-86c1-469f-bb32-a37c680cc80c" status: 200 len: 1365 microversion: 2.1 time: 0.167178 Jul 16 21:36:51.241256 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2581/5164] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:36:51 2022] GET /compute/v2.1/servers/a8f70d9d-86c1-469f-bb32-a37c680cc80c => generated 1365 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:36:52.254402 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-ba383e35-6083-442a-b057-11970f43f7d4 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:36:52.255029 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-ba383e35-6083-442a-b057-11970f43f7d4 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] [instance: a8f70d9d-86c1-469f-bb32-a37c680cc80c] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:36:52.258706 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell0:SELECT=6 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:36:52.263698 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ba383e35-6083-442a-b057-11970f43f7d4 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:36:52.264151 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ba383e35-6083-442a-b057-11970f43f7d4 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:36:52.264958 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ba383e35-6083-442a-b057-11970f43f7d4 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:36:52.265398 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ba383e35-6083-442a-b057-11970f43f7d4 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:36:52.409588 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-ba383e35-6083-442a-b057-11970f43f7d4 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0ab8b3f06e994bbca5d6dd14109bfe50', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e60c0863dc6f4313bdc873fda27ae2fc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:36:52.418690 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-ba383e35-6083-442a-b057-11970f43f7d4 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] 10.222.0.16 "GET /compute/v2.1/servers/a8f70d9d-86c1-469f-bb32-a37c680cc80c" status: 200 len: 1556 microversion: 2.1 time: 0.168252 Jul 16 21:36:52.419316 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2584/5165] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:36:52 2022] GET /compute/v2.1/servers/a8f70d9d-86c1-469f-bb32-a37c680cc80c => generated 1556 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:36:52.436789 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-c35db86e-d30c-4728-a6b0-3066ac0ab52d tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:36:52.437394 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-c35db86e-d30c-4728-a6b0-3066ac0ab52d tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] [instance: a8f70d9d-86c1-469f-bb32-a37c680cc80c] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:36:52.439277 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell0:SELECT=2 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:36:52.442634 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell0:SELECT=6 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:36:52.446621 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c35db86e-d30c-4728-a6b0-3066ac0ab52d tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:36:52.447157 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c35db86e-d30c-4728-a6b0-3066ac0ab52d tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:36:52.447952 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c35db86e-d30c-4728-a6b0-3066ac0ab52d tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:36:52.448390 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c35db86e-d30c-4728-a6b0-3066ac0ab52d tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:36:52.544647 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-c35db86e-d30c-4728-a6b0-3066ac0ab52d tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] 10.222.0.16 "GET /compute/v2.1/servers/a8f70d9d-86c1-469f-bb32-a37c680cc80c/os-interface" status: 200 len: 285 microversion: 2.1 time: 0.112110 Jul 16 21:36:52.545255 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2582/5166] 10.222.0.16 () {58 vars in 1274 bytes} [Sat Jul 16 21:36:52 2022] GET /compute/v2.1/servers/a8f70d9d-86c1-469f-bb32-a37c680cc80c/os-interface => generated 285 bytes in 113 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 16 21:36:53.685682 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-38abeaa8-4da8-4367-80cd-61662c134a0f req-7207ad36-4fcf-46a7-9175-b3db5d675409 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "a8f70d9d-86c1-469f-bb32-a37c680cc80c", "tag": "b54ca2a7-6d58-476e-b078-7c95d8ac7fed"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:36:53.699291 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-38abeaa8-4da8-4367-80cd-61662c134a0f req-7207ad36-4fcf-46a7-9175-b3db5d675409 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:36:53.699752 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-38abeaa8-4da8-4367-80cd-61662c134a0f req-7207ad36-4fcf-46a7-9175-b3db5d675409 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:36:53.712291 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-38abeaa8-4da8-4367-80cd-61662c134a0f req-7207ad36-4fcf-46a7-9175-b3db5d675409 service nova] Creating event network-changed:b54ca2a7-6d58-476e-b078-7c95d8ac7fed for instance a8f70d9d-86c1-469f-bb32-a37c680cc80c on nch1-master-1 Jul 16 21:36:53.720670 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-38abeaa8-4da8-4367-80cd-61662c134a0f req-7207ad36-4fcf-46a7-9175-b3db5d675409 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.039918 Jul 16 21:36:53.721666 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2585/5167] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:36:53 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:37:23.450476 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_api:SELECT=7 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:37:23.451669 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=35 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:37:23.456421 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-a39e18be-2b51-4936-8205-722242f2aa13 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:37:23.456932 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-a39e18be-2b51-4936-8205-722242f2aa13 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] [instance: a8f70d9d-86c1-469f-bb32-a37c680cc80c] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:37:23.463460 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a39e18be-2b51-4936-8205-722242f2aa13 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:37:23.463931 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a39e18be-2b51-4936-8205-722242f2aa13 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:37:23.464709 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a39e18be-2b51-4936-8205-722242f2aa13 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:37:23.465162 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a39e18be-2b51-4936-8205-722242f2aa13 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:37:23.490864 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-a39e18be-2b51-4936-8205-722242f2aa13 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] 10.222.0.16 "GET /compute/v2.1/servers/a8f70d9d-86c1-469f-bb32-a37c680cc80c/ips" status: 200 len: 151 microversion: 2.1 time: 0.042082 Jul 16 21:37:23.491574 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2583/5168] 10.222.0.16 () {58 vars in 1247 bytes} [Sat Jul 16 21:37:23 2022] GET /compute/v2.1/servers/a8f70d9d-86c1-469f-bb32-a37c680cc80c/ips => generated 151 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:37:23.506017 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_api:SELECT=28,nova_api:INSERT=1 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:37:23.506017 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=28 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:37:23.510142 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-89a79b12-82dd-4241-b7db-b89f8e04fc55 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "64cfebc6-4a34-42c3-8e38-188693d74ff7"}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:37:23.511436 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-89a79b12-82dd-4241-b7db-b89f8e04fc55 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] [instance: a8f70d9d-86c1-469f-bb32-a37c680cc80c] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:37:23.517652 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-89a79b12-82dd-4241-b7db-b89f8e04fc55 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:37:23.518102 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-89a79b12-82dd-4241-b7db-b89f8e04fc55 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:37:23.518869 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-89a79b12-82dd-4241-b7db-b89f8e04fc55 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:37:23.519354 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-89a79b12-82dd-4241-b7db-b89f8e04fc55 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:37:23.641792 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-89a79b12-82dd-4241-b7db-b89f8e04fc55 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] HTTP exception thrown: Invalid volume: volume 64cfebc6-4a34-42c3-8e38-188693d74ff7 is already attached to instances: 4df09776-dc3c-46ff-9e64-e20b5d4679fa Jul 16 21:37:23.644211 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-89a79b12-82dd-4241-b7db-b89f8e04fc55 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Request method failure captured: Jul 16 21:37:23.644211 ncdv-master-1 devstack@n-api.service[138681]: request: POST /compute/v2.1/servers/a8f70d9d-86c1-469f-bb32-a37c680cc80c/os-volume_attachments HTTP/1.1 Jul 16 21:37:23.644211 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:37:23.644211 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:37:23.644211 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:37:23.644211 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 74 Jul 16 21:37:23.644211 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:37:23.644211 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:37:23.644211 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:37:23.644211 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:37:23.644211 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0y8gcGxfa3qumeDpAV_UzHtBqxWw5fgWdXEsN_P0XTe84Os21rKjc9QIlpjQBg7EBAnrc9gi4HTfF6VkzKBMP6W-EwHOC7W6W2mOvP55miTafGS3QI8-v38KjXcvgCGFOUcX_9nbDaYEoaZ_SfO91myDjzRdtv09P0hU8CwskGtYm8E Jul 16 21:37:23.644211 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:37:23.644211 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:37:23.644211 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:37:23.644211 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:37:23.644211 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:37:23.644211 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:37:23.644211 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: e60c0863dc6f4313bdc873fda27ae2fc Jul 16 21:37:23.644211 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-AttachVolumeNegativeTest-17004004 Jul 16 21:37:23.644211 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:37:23.644211 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:37:23.644211 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/e60c0863dc6f4313bdc873fda27ae2fc"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/e60c0863dc6f4313bdc873fda27ae2fc"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_e60c0863dc6f4313bdc873fda27ae2fc"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/e60c0863dc6f4313bdc873fda27ae2fc"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:37:23.644211 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-AttachVolumeNegativeTest-17004004 Jul 16 21:37:23.644211 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: e60c0863dc6f4313bdc873fda27ae2fc Jul 16 21:37:23.644211 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-17004004 Jul 16 21:37:23.644211 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-AttachVolumeNegativeTest-17004004-project Jul 16 21:37:23.644211 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:37:23.644211 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:37:23.644211 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: 0ab8b3f06e994bbca5d6dd14109bfe50 Jul 16 21:37:23.644211 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-AttachVolumeNegativeTest-17004004-project Jul 16 21:37:23.644211 ncdv-master-1 devstack@n-api.service[138681]: Jul 16 21:37:23.644211 ncdv-master-1 devstack@n-api.service[138681]: {"volumeAttachment": {"volumeId": "64cfebc6-4a34-42c3-8e38-188693d74ff7"}} Jul 16 21:37:23.644211 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:37:23.644211 ncdv-master-1 devstack@n-api.service[138681]: exception: Invalid volume: volume 64cfebc6-4a34-42c3-8e38-188693d74ff7 is already attached to instances: 4df09776-dc3c-46ff-9e64-e20b5d4679fa Jul 16 21:37:23.644211 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'server_id': 'a8f70d9d-86c1-469f-bb32-a37c680cc80c', 'body': {'volumeAttachment': {'volumeId': '64cfebc6-4a34-42c3-8e38-188693d74ff7'}}} Jul 16 21:37:23.644211 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:37:23.644211 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:37:23.644211 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 388, in create Jul 16 21:37:23.644211 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi device = self.compute_api.attach_volume( Jul 16 21:37:23.644211 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner Jul 16 21:37:23.644211 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Jul 16 21:37:23.644211 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 164, in inner Jul 16 21:37:23.644211 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Jul 16 21:37:23.644211 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 4985, in attach_volume Jul 16 21:37:23.644211 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi self._check_volume_already_attached(context, instance, volume) Jul 16 21:37:23.644211 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 4825, in _check_volume_already_attached Jul 16 21:37:23.644211 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=msg) Jul 16 21:37:23.644211 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: volume 64cfebc6-4a34-42c3-8e38-188693d74ff7 is already attached to instances: 4df09776-dc3c-46ff-9e64-e20b5d4679fa Jul 16 21:37:23.644211 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:37:23.644211 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:37:23.644211 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:37:23.644211 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:37:23.644211 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:37:23.644211 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:37:23.644211 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:37:23.644211 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:37:23.644211 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:37:23.644211 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:37:23.644211 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 16 21:37:23.644211 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 16 21:37:23.644211 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 16 21:37:23.644211 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 16 21:37:23.648134 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 16 21:37:23.648134 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 16 21:37:23.648134 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 406, in create Jul 16 21:37:23.648134 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Jul 16 21:37:23.648134 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: volume 64cfebc6-4a34-42c3-8e38-188693d74ff7 is already attached to instances: 4df09776-dc3c-46ff-9e64-e20b5d4679fa Jul 16 21:37:23.648134 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:37:23.648134 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:37:23.648134 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:37:23.648134 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:37:23.648134 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:37:23.648134 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:37:23.648134 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:37:23.648134 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:37:23.648134 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: volume 64cfebc6-4a34-42c3-8e38-188693d74ff7 is already attached to instances: 4df09776-dc3c-46ff-9e64-e20b5d4679fa Jul 16 21:37:23.648134 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:37:23.648134 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-89a79b12-82dd-4241-b7db-b89f8e04fc55 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Returning 400 to user: Invalid volume: volume 64cfebc6-4a34-42c3-8e38-188693d74ff7 is already attached to instances: 4df09776-dc3c-46ff-9e64-e20b5d4679fa {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:37:23.648134 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-89a79b12-82dd-4241-b7db-b89f8e04fc55 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] 10.222.0.16 "POST /compute/v2.1/servers/a8f70d9d-86c1-469f-bb32-a37c680cc80c/os-volume_attachments" status: 400 len: 174 microversion: 2.1 time: 0.145331 Jul 16 21:37:23.648134 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2586/5169] 10.222.0.16 () {60 vars in 1322 bytes} [Sat Jul 16 21:37:23 2022] POST /compute/v2.1/servers/a8f70d9d-86c1-469f-bb32-a37c680cc80c/os-volume_attachments => generated 174 bytes in 147 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Jul 16 21:37:23.656184 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-fa767ddd-9387-4b4b-83d4-6d2fe3157fdd tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:37:23.656840 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-fa767ddd-9387-4b4b-83d4-6d2fe3157fdd tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] [instance: a8f70d9d-86c1-469f-bb32-a37c680cc80c] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:37:23.665354 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-fa767ddd-9387-4b4b-83d4-6d2fe3157fdd tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:37:23.665354 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-fa767ddd-9387-4b4b-83d4-6d2fe3157fdd tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:37:23.665723 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-fa767ddd-9387-4b4b-83d4-6d2fe3157fdd tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:37:23.666171 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-fa767ddd-9387-4b4b-83d4-6d2fe3157fdd tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:37:23.692457 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-fa767ddd-9387-4b4b-83d4-6d2fe3157fdd tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] [instance: a8f70d9d-86c1-469f-bb32-a37c680cc80c] Going to try to terminate instance {{(pid=138680) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 16 21:37:24.022533 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-fa767ddd-9387-4b4b-83d4-6d2fe3157fdd tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] 10.222.0.16 "DELETE /compute/v2.1/servers/a8f70d9d-86c1-469f-bb32-a37c680cc80c" status: 204 len: 0 microversion: 2.1 time: 0.370041 Jul 16 21:37:24.024053 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2584/5170] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:37:23 2022] DELETE /compute/v2.1/servers/a8f70d9d-86c1-469f-bb32-a37c680cc80c => generated 0 bytes in 373 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 16 21:37:24.033183 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-ae93407e-f16c-4109-9548-1f9f7bb18266 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:37:24.034105 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-ae93407e-f16c-4109-9548-1f9f7bb18266 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] [instance: a8f70d9d-86c1-469f-bb32-a37c680cc80c] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:37:24.043476 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ae93407e-f16c-4109-9548-1f9f7bb18266 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:37:24.043927 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ae93407e-f16c-4109-9548-1f9f7bb18266 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:37:24.044709 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ae93407e-f16c-4109-9548-1f9f7bb18266 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:37:24.045148 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ae93407e-f16c-4109-9548-1f9f7bb18266 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:37:24.200837 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-ae93407e-f16c-4109-9548-1f9f7bb18266 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0ab8b3f06e994bbca5d6dd14109bfe50', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e60c0863dc6f4313bdc873fda27ae2fc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:37:24.206390 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-ae93407e-f16c-4109-9548-1f9f7bb18266 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] 10.222.0.16 "GET /compute/v2.1/servers/a8f70d9d-86c1-469f-bb32-a37c680cc80c" status: 200 len: 1681 microversion: 2.1 time: 0.177443 Jul 16 21:37:24.217088 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2587/5171] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:37:24 2022] GET /compute/v2.1/servers/a8f70d9d-86c1-469f-bb32-a37c680cc80c => generated 1681 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:37:25.225123 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-8ad5fa1a-0b80-4962-888d-eff5a36b29b8 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:37:25.225900 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-8ad5fa1a-0b80-4962-888d-eff5a36b29b8 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] [instance: a8f70d9d-86c1-469f-bb32-a37c680cc80c] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:37:25.235217 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8ad5fa1a-0b80-4962-888d-eff5a36b29b8 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:37:25.235743 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8ad5fa1a-0b80-4962-888d-eff5a36b29b8 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:37:25.236762 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8ad5fa1a-0b80-4962-888d-eff5a36b29b8 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:37:25.237358 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8ad5fa1a-0b80-4962-888d-eff5a36b29b8 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:37:25.392294 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-8ad5fa1a-0b80-4962-888d-eff5a36b29b8 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0ab8b3f06e994bbca5d6dd14109bfe50', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e60c0863dc6f4313bdc873fda27ae2fc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:37:25.400381 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-8ad5fa1a-0b80-4962-888d-eff5a36b29b8 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] 10.222.0.16 "GET /compute/v2.1/servers/a8f70d9d-86c1-469f-bb32-a37c680cc80c" status: 200 len: 1681 microversion: 2.1 time: 0.179545 Jul 16 21:37:25.400956 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2585/5172] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:37:25 2022] GET /compute/v2.1/servers/a8f70d9d-86c1-469f-bb32-a37c680cc80c => generated 1681 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:37:26.432424 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-dd4a93a1-4006-44b4-94b5-37d4f3d89f2f tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:37:26.432424 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-dd4a93a1-4006-44b4-94b5-37d4f3d89f2f tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] [instance: a8f70d9d-86c1-469f-bb32-a37c680cc80c] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:37:26.432424 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-dd4a93a1-4006-44b4-94b5-37d4f3d89f2f tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:37:26.432424 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-dd4a93a1-4006-44b4-94b5-37d4f3d89f2f tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:37:26.432424 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-dd4a93a1-4006-44b4-94b5-37d4f3d89f2f tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:37:26.432424 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-dd4a93a1-4006-44b4-94b5-37d4f3d89f2f tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:37:26.503577 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-dd4a93a1-4006-44b4-94b5-37d4f3d89f2f tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0ab8b3f06e994bbca5d6dd14109bfe50', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e60c0863dc6f4313bdc873fda27ae2fc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:37:26.514326 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-dd4a93a1-4006-44b4-94b5-37d4f3d89f2f tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] 10.222.0.16 "GET /compute/v2.1/servers/a8f70d9d-86c1-469f-bb32-a37c680cc80c" status: 200 len: 1614 microversion: 2.1 time: 0.100360 Jul 16 21:37:26.514909 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2588/5173] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:37:26 2022] GET /compute/v2.1/servers/a8f70d9d-86c1-469f-bb32-a37c680cc80c => generated 1614 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:37:26.840405 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-6f42fc6d-7d23-48da-a676-d67013c615d0 req-fff7e0a0-e835-4af1-8da2-e953060dc9f7 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "a8f70d9d-86c1-469f-bb32-a37c680cc80c", "name": "network-vif-deleted", "tag": "b54ca2a7-6d58-476e-b078-7c95d8ac7fed"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:37:26.850618 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-6f42fc6d-7d23-48da-a676-d67013c615d0 req-fff7e0a0-e835-4af1-8da2-e953060dc9f7 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:37:26.851528 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-6f42fc6d-7d23-48da-a676-d67013c615d0 req-fff7e0a0-e835-4af1-8da2-e953060dc9f7 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:37:26.875055 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-6f42fc6d-7d23-48da-a676-d67013c615d0 req-fff7e0a0-e835-4af1-8da2-e953060dc9f7 service nova] Creating event network-vif-deleted:b54ca2a7-6d58-476e-b078-7c95d8ac7fed for instance a8f70d9d-86c1-469f-bb32-a37c680cc80c on nch1-master-1 Jul 16 21:37:26.881348 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-6f42fc6d-7d23-48da-a676-d67013c615d0 req-fff7e0a0-e835-4af1-8da2-e953060dc9f7 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.045904 Jul 16 21:37:26.884596 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2586/5174] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:37:26 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:37:27.534404 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-e3c315bb-0e6b-41ca-b4af-75c9200d60ae tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:37:27.534404 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-e3c315bb-0e6b-41ca-b4af-75c9200d60ae tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] [instance: a8f70d9d-86c1-469f-bb32-a37c680cc80c] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:37:27.547355 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e3c315bb-0e6b-41ca-b4af-75c9200d60ae tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:37:27.547355 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e3c315bb-0e6b-41ca-b4af-75c9200d60ae tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:37:27.547635 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e3c315bb-0e6b-41ca-b4af-75c9200d60ae tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:37:27.547899 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e3c315bb-0e6b-41ca-b4af-75c9200d60ae tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:37:27.622634 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e3c315bb-0e6b-41ca-b4af-75c9200d60ae tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:37:27.623901 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e3c315bb-0e6b-41ca-b4af-75c9200d60ae tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:37:27.624554 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.objects.instance [None req-e3c315bb-0e6b-41ca-b4af-75c9200d60ae tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lazy-loading 'fault' on Instance uuid a8f70d9d-86c1-469f-bb32-a37c680cc80c {{(pid=138681) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:37:27.687263 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-e3c315bb-0e6b-41ca-b4af-75c9200d60ae tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0ab8b3f06e994bbca5d6dd14109bfe50', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e60c0863dc6f4313bdc873fda27ae2fc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:37:27.698415 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-e3c315bb-0e6b-41ca-b4af-75c9200d60ae tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] 10.222.0.16 "GET /compute/v2.1/servers/a8f70d9d-86c1-469f-bb32-a37c680cc80c" status: 200 len: 1330 microversion: 2.1 time: 0.169564 Jul 16 21:37:27.699412 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2589/5175] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:37:27 2022] GET /compute/v2.1/servers/a8f70d9d-86c1-469f-bb32-a37c680cc80c => generated 1330 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:37:28.718520 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-b3681474-d93f-4e7b-9e36-446a512a6443 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:37:28.719411 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-b3681474-d93f-4e7b-9e36-446a512a6443 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] [instance: a8f70d9d-86c1-469f-bb32-a37c680cc80c] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:37:28.725777 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b3681474-d93f-4e7b-9e36-446a512a6443 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:37:28.726240 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b3681474-d93f-4e7b-9e36-446a512a6443 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:37:28.727555 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b3681474-d93f-4e7b-9e36-446a512a6443 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:37:28.728003 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b3681474-d93f-4e7b-9e36-446a512a6443 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:37:28.744699 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-b3681474-d93f-4e7b-9e36-446a512a6443 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] HTTP exception thrown: Instance a8f70d9d-86c1-469f-bb32-a37c680cc80c could not be found. Jul 16 21:37:28.746975 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-b3681474-d93f-4e7b-9e36-446a512a6443 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Request method failure captured: Jul 16 21:37:28.746975 ncdv-master-1 devstack@n-api.service[138680]: request: GET /compute/v2.1/servers/a8f70d9d-86c1-469f-bb32-a37c680cc80c HTTP/1.1 Jul 16 21:37:28.746975 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:37:28.746975 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:37:28.746975 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:37:28.746975 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 0 Jul 16 21:37:28.746975 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:37:28.746975 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:37:28.746975 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:37:28.746975 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:37:28.746975 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0y8gcGxfa3qumeDpAV_UzHtBqxWw5fgWdXEsN_P0XTe84Os21rKjc9QIlpjQBg7EBAnrc9gi4HTfF6VkzKBMP6W-EwHOC7W6W2mOvP55miTafGS3QI8-v38KjXcvgCGFOUcX_9nbDaYEoaZ_SfO91myDjzRdtv09P0hU8CwskGtYm8E Jul 16 21:37:28.746975 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:37:28.746975 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:37:28.746975 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:37:28.746975 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:37:28.746975 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:37:28.746975 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:37:28.746975 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: e60c0863dc6f4313bdc873fda27ae2fc Jul 16 21:37:28.746975 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-AttachVolumeNegativeTest-17004004 Jul 16 21:37:28.746975 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:37:28.746975 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:37:28.746975 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/e60c0863dc6f4313bdc873fda27ae2fc"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/e60c0863dc6f4313bdc873fda27ae2fc"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_e60c0863dc6f4313bdc873fda27ae2fc"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/e60c0863dc6f4313bdc873fda27ae2fc"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:37:28.746975 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-AttachVolumeNegativeTest-17004004 Jul 16 21:37:28.746975 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: e60c0863dc6f4313bdc873fda27ae2fc Jul 16 21:37:28.746975 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-17004004 Jul 16 21:37:28.746975 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-AttachVolumeNegativeTest-17004004-project Jul 16 21:37:28.746975 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:37:28.746975 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:37:28.746975 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: 0ab8b3f06e994bbca5d6dd14109bfe50 Jul 16 21:37:28.746975 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-AttachVolumeNegativeTest-17004004-project Jul 16 21:37:28.746975 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 21:37:28.746975 ncdv-master-1 devstack@n-api.service[138680]: exception: Instance a8f70d9d-86c1-469f-bb32-a37c680cc80c could not be found. Jul 16 21:37:28.746975 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': 'a8f70d9d-86c1-469f-bb32-a37c680cc80c'} Jul 16 21:37:28.746975 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:37:28.746975 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:37:28.746975 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:37:28.746975 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:37:28.746975 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:37:28.746975 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:37:28.746975 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:37:28.746975 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:37:28.746975 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:37:28.746975 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:37:28.746975 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance a8f70d9d-86c1-469f-bb32-a37c680cc80c could not be found. Jul 16 21:37:28.746975 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:37:28.746975 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:37:28.746975 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:37:28.746975 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:37:28.746975 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:37:28.746975 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:37:28.746975 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:37:28.746975 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:37:28.746975 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:37:28.746975 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:37:28.746975 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:37:28.746975 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:37:28.746975 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:37:28.746975 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:37:28.746975 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:37:28.746975 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:37:28.746975 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance a8f70d9d-86c1-469f-bb32-a37c680cc80c could not be found. Jul 16 21:37:28.746975 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:37:28.746975 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:37:28.746975 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:37:28.751234 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:37:28.751234 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:37:28.751234 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:37:28.751234 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:37:28.751234 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:37:28.751234 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance a8f70d9d-86c1-469f-bb32-a37c680cc80c could not be found. Jul 16 21:37:28.751234 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:37:28.751234 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-b3681474-d93f-4e7b-9e36-446a512a6443 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Returning 404 to user: Instance a8f70d9d-86c1-469f-bb32-a37c680cc80c could not be found. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:37:28.751234 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-b3681474-d93f-4e7b-9e36-446a512a6443 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] 10.222.0.16 "GET /compute/v2.1/servers/a8f70d9d-86c1-469f-bb32-a37c680cc80c" status: 404 len: 111 microversion: 2.1 time: 0.036180 Jul 16 21:37:28.751234 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2587/5176] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:37:28 2022] GET /compute/v2.1/servers/a8f70d9d-86c1-469f-bb32-a37c680cc80c => generated 111 bytes in 38 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:37:28.757679 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-5551a874-4032-426d-b995-9a00a31e4cb2 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Calling method '.version_select at 0x7fdd9d519160>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:37:28.842941 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-5551a874-4032-426d-b995-9a00a31e4cb2 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] 10.222.0.16 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-476890770" status: 202 len: 0 microversion: 2.1 time: 0.089132 Jul 16 21:37:28.847968 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2590/5177] 10.222.0.16 () {58 vars in 1217 bytes} [Sat Jul 16 21:37:28 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-476890770 => generated 0 bytes in 91 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 21:37:30.044085 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-1ee1a6d6-3107-4a20-8a78-34bcbc88d865 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:37:30.044586 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-1ee1a6d6-3107-4a20-8a78-34bcbc88d865 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] [instance: 4df09776-dc3c-46ff-9e64-e20b5d4679fa] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:37:30.051709 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1ee1a6d6-3107-4a20-8a78-34bcbc88d865 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:37:30.052652 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1ee1a6d6-3107-4a20-8a78-34bcbc88d865 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:37:30.054595 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1ee1a6d6-3107-4a20-8a78-34bcbc88d865 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:37:30.054595 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1ee1a6d6-3107-4a20-8a78-34bcbc88d865 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:37:30.141863 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.objects.instance [None req-1ee1a6d6-3107-4a20-8a78-34bcbc88d865 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lazy-loading 'services' on Instance uuid 4df09776-dc3c-46ff-9e64-e20b5d4679fa {{(pid=138680) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:37:30.506581 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-1ee1a6d6-3107-4a20-8a78-34bcbc88d865 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] 10.222.0.16 "DELETE /compute/v2.1/servers/4df09776-dc3c-46ff-9e64-e20b5d4679fa/os-volume_attachments/64cfebc6-4a34-42c3-8e38-188693d74ff7" status: 202 len: 0 microversion: 2.1 time: 0.466651 Jul 16 21:37:30.507218 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2588/5178] 10.222.0.16 () {58 vars in 1415 bytes} [Sat Jul 16 21:37:30 2022] DELETE /compute/v2.1/servers/4df09776-dc3c-46ff-9e64-e20b5d4679fa/os-volume_attachments/64cfebc6-4a34-42c3-8e38-188693d74ff7 => generated 0 bytes in 469 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 21:37:30.517697 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-5452fe18-7492-42df-a62a-0f07b8bbae20 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:37:30.519573 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-5452fe18-7492-42df-a62a-0f07b8bbae20 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] [instance: 4df09776-dc3c-46ff-9e64-e20b5d4679fa] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:37:30.528472 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5452fe18-7492-42df-a62a-0f07b8bbae20 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:37:30.529163 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5452fe18-7492-42df-a62a-0f07b8bbae20 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:37:30.534796 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5452fe18-7492-42df-a62a-0f07b8bbae20 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:37:30.534796 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5452fe18-7492-42df-a62a-0f07b8bbae20 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:37:30.562588 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-5452fe18-7492-42df-a62a-0f07b8bbae20 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] 10.222.0.16 "GET /compute/v2.1/servers/4df09776-dc3c-46ff-9e64-e20b5d4679fa/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050118 Jul 16 21:37:30.574845 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2591/5179] 10.222.0.16 () {58 vars in 1301 bytes} [Sat Jul 16 21:37:30 2022] GET /compute/v2.1/servers/4df09776-dc3c-46ff-9e64-e20b5d4679fa/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:37:31.583274 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-cce9e958-fc10-4a39-8e5a-7f04cce8156e tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:37:31.586009 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-cce9e958-fc10-4a39-8e5a-7f04cce8156e tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] [instance: 4df09776-dc3c-46ff-9e64-e20b5d4679fa] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:37:31.598270 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cce9e958-fc10-4a39-8e5a-7f04cce8156e tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:37:31.598965 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cce9e958-fc10-4a39-8e5a-7f04cce8156e tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:37:31.599857 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cce9e958-fc10-4a39-8e5a-7f04cce8156e tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:37:31.600372 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cce9e958-fc10-4a39-8e5a-7f04cce8156e tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:37:31.639655 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-cce9e958-fc10-4a39-8e5a-7f04cce8156e tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] 10.222.0.16 "GET /compute/v2.1/servers/4df09776-dc3c-46ff-9e64-e20b5d4679fa/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.067564 Jul 16 21:37:31.641060 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2592/5180] 10.222.0.16 () {58 vars in 1301 bytes} [Sat Jul 16 21:37:31 2022] GET /compute/v2.1/servers/4df09776-dc3c-46ff-9e64-e20b5d4679fa/os-volume_attachments => generated 25 bytes in 70 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Jul 16 21:37:31.674884 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-3b70144c-2c13-4abf-b615-ac3e27aef19f tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:37:31.675947 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-3b70144c-2c13-4abf-b615-ac3e27aef19f tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] [instance: 3c0f0855-7a9c-4d8a-95ac-3bf41d32d4e6] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:37:31.695652 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3b70144c-2c13-4abf-b615-ac3e27aef19f tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:37:31.696159 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3b70144c-2c13-4abf-b615-ac3e27aef19f tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:37:31.697945 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3b70144c-2c13-4abf-b615-ac3e27aef19f tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:37:31.698549 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3b70144c-2c13-4abf-b615-ac3e27aef19f tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:37:31.727249 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-3f6dfc24-7968-4350-a33e-a7b4d90d3869 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:37:31.730148 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-3f6dfc24-7968-4350-a33e-a7b4d90d3869 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] [instance: 4df09776-dc3c-46ff-9e64-e20b5d4679fa] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:37:31.734697 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-3b70144c-2c13-4abf-b615-ac3e27aef19f tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] [instance: 3c0f0855-7a9c-4d8a-95ac-3bf41d32d4e6] Going to try to terminate instance {{(pid=138680) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 16 21:37:31.743834 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3f6dfc24-7968-4350-a33e-a7b4d90d3869 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:37:31.745017 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3f6dfc24-7968-4350-a33e-a7b4d90d3869 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:37:31.747332 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3f6dfc24-7968-4350-a33e-a7b4d90d3869 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:37:31.748053 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3f6dfc24-7968-4350-a33e-a7b4d90d3869 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:37:31.792485 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-3f6dfc24-7968-4350-a33e-a7b4d90d3869 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] [instance: 4df09776-dc3c-46ff-9e64-e20b5d4679fa] Going to try to terminate instance {{(pid=138681) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 16 21:37:32.118843 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-3b70144c-2c13-4abf-b615-ac3e27aef19f tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] 10.222.0.16 "DELETE /compute/v2.1/servers/3c0f0855-7a9c-4d8a-95ac-3bf41d32d4e6" status: 204 len: 0 microversion: 2.1 time: 0.775462 Jul 16 21:37:32.119092 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-3f6dfc24-7968-4350-a33e-a7b4d90d3869 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] 10.222.0.16 "DELETE /compute/v2.1/servers/4df09776-dc3c-46ff-9e64-e20b5d4679fa" status: 204 len: 0 microversion: 2.1 time: 0.398038 Jul 16 21:37:32.122881 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2593/5182] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:37:31 2022] DELETE /compute/v2.1/servers/4df09776-dc3c-46ff-9e64-e20b5d4679fa => generated 0 bytes in 401 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Jul 16 21:37:32.123212 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2589/5182] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:37:31 2022] DELETE /compute/v2.1/servers/3c0f0855-7a9c-4d8a-95ac-3bf41d32d4e6 => generated 0 bytes in 778 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 16 21:37:32.129091 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-e5bc854c-3eef-4dc0-906b-f39a9afb5223 tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:37:32.129555 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-99a72484-631e-4c4c-9131-1865c2c1adb7 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:37:32.129808 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-e5bc854c-3eef-4dc0-906b-f39a9afb5223 tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] [instance: 3c0f0855-7a9c-4d8a-95ac-3bf41d32d4e6] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:37:32.130408 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-99a72484-631e-4c4c-9131-1865c2c1adb7 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] [instance: 4df09776-dc3c-46ff-9e64-e20b5d4679fa] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:37:32.137210 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e5bc854c-3eef-4dc0-906b-f39a9afb5223 tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:37:32.137831 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e5bc854c-3eef-4dc0-906b-f39a9afb5223 tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:37:32.138532 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e5bc854c-3eef-4dc0-906b-f39a9afb5223 tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:37:32.138845 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-99a72484-631e-4c4c-9131-1865c2c1adb7 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:37:32.139147 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e5bc854c-3eef-4dc0-906b-f39a9afb5223 tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:37:32.139461 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-99a72484-631e-4c4c-9131-1865c2c1adb7 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:37:32.140491 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-99a72484-631e-4c4c-9131-1865c2c1adb7 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:37:32.140993 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-99a72484-631e-4c4c-9131-1865c2c1adb7 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:37:32.405703 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-99a72484-631e-4c4c-9131-1865c2c1adb7 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0ab8b3f06e994bbca5d6dd14109bfe50', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e60c0863dc6f4313bdc873fda27ae2fc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:37:32.416423 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-99a72484-631e-4c4c-9131-1865c2c1adb7 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] 10.222.0.16 "GET /compute/v2.1/servers/4df09776-dc3c-46ff-9e64-e20b5d4679fa" status: 200 len: 1683 microversion: 2.1 time: 0.291679 Jul 16 21:37:32.422051 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2590/5183] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:37:32 2022] GET /compute/v2.1/servers/4df09776-dc3c-46ff-9e64-e20b5d4679fa => generated 1683 bytes in 298 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:37:32.424840 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-e5bc854c-3eef-4dc0-906b-f39a9afb5223 tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4fa446573f7d42de816280280c3dec19', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ce30584a7dfa4ab19de73aa8b18b113b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:37:32.436723 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-e5bc854c-3eef-4dc0-906b-f39a9afb5223 tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] 10.222.0.16 "GET /compute/v2.1/servers/3c0f0855-7a9c-4d8a-95ac-3bf41d32d4e6" status: 200 len: 1731 microversion: 2.1 time: 0.311840 Jul 16 21:37:32.437869 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2594/5184] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:37:32 2022] GET /compute/v2.1/servers/3c0f0855-7a9c-4d8a-95ac-3bf41d32d4e6 => generated 1731 bytes in 314 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:37:33.439430 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-d8cb2178-455f-49b5-a3a1-a55c5d3e9185 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:37:33.440175 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-d8cb2178-455f-49b5-a3a1-a55c5d3e9185 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] [instance: 4df09776-dc3c-46ff-9e64-e20b5d4679fa] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:37:33.449331 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d8cb2178-455f-49b5-a3a1-a55c5d3e9185 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:37:33.456304 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-e8fdb0e6-1362-48fe-8a77-b3b412c6c2d4 tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:37:33.457757 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-e8fdb0e6-1362-48fe-8a77-b3b412c6c2d4 tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] [instance: 3c0f0855-7a9c-4d8a-95ac-3bf41d32d4e6] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:37:33.464295 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d8cb2178-455f-49b5-a3a1-a55c5d3e9185 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.013s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:37:33.465338 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d8cb2178-455f-49b5-a3a1-a55c5d3e9185 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:37:33.467735 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d8cb2178-455f-49b5-a3a1-a55c5d3e9185 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:37:33.475209 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e8fdb0e6-1362-48fe-8a77-b3b412c6c2d4 tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:37:33.475493 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e8fdb0e6-1362-48fe-8a77-b3b412c6c2d4 tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:37:33.476414 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e8fdb0e6-1362-48fe-8a77-b3b412c6c2d4 tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:37:33.476868 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e8fdb0e6-1362-48fe-8a77-b3b412c6c2d4 tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:37:33.748131 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-e8fdb0e6-1362-48fe-8a77-b3b412c6c2d4 tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4fa446573f7d42de816280280c3dec19', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ce30584a7dfa4ab19de73aa8b18b113b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:37:33.754433 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-d8cb2178-455f-49b5-a3a1-a55c5d3e9185 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0ab8b3f06e994bbca5d6dd14109bfe50', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e60c0863dc6f4313bdc873fda27ae2fc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:37:33.756947 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-e8fdb0e6-1362-48fe-8a77-b3b412c6c2d4 tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] 10.222.0.16 "GET /compute/v2.1/servers/3c0f0855-7a9c-4d8a-95ac-3bf41d32d4e6" status: 200 len: 1731 microversion: 2.1 time: 0.304670 Jul 16 21:37:33.766642 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-d8cb2178-455f-49b5-a3a1-a55c5d3e9185 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] 10.222.0.16 "GET /compute/v2.1/servers/4df09776-dc3c-46ff-9e64-e20b5d4679fa" status: 200 len: 1683 microversion: 2.1 time: 0.331753 Jul 16 21:37:33.767469 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2595/5185] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:37:33 2022] GET /compute/v2.1/servers/3c0f0855-7a9c-4d8a-95ac-3bf41d32d4e6 => generated 1731 bytes in 316 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:37:33.768665 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2591/5186] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:37:33 2022] GET /compute/v2.1/servers/4df09776-dc3c-46ff-9e64-e20b5d4679fa => generated 1683 bytes in 334 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:37:34.775830 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-37d62426-0f64-425c-8573-1d68f1337e6d tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:37:34.778770 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-37d62426-0f64-425c-8573-1d68f1337e6d tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] [instance: 3c0f0855-7a9c-4d8a-95ac-3bf41d32d4e6] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:37:34.786498 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-4d8cface-31f3-40cb-8457-b580df9616e0 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:37:34.788634 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-4d8cface-31f3-40cb-8457-b580df9616e0 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] [instance: 4df09776-dc3c-46ff-9e64-e20b5d4679fa] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:37:34.788808 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-37d62426-0f64-425c-8573-1d68f1337e6d tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:37:34.789277 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-37d62426-0f64-425c-8573-1d68f1337e6d tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:37:34.791268 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-37d62426-0f64-425c-8573-1d68f1337e6d tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:37:34.791951 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-37d62426-0f64-425c-8573-1d68f1337e6d tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:37:34.800065 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4d8cface-31f3-40cb-8457-b580df9616e0 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:37:34.801278 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4d8cface-31f3-40cb-8457-b580df9616e0 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:37:34.802985 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4d8cface-31f3-40cb-8457-b580df9616e0 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:37:34.803683 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4d8cface-31f3-40cb-8457-b580df9616e0 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:37:35.160662 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-4d8cface-31f3-40cb-8457-b580df9616e0 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0ab8b3f06e994bbca5d6dd14109bfe50', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e60c0863dc6f4313bdc873fda27ae2fc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:37:35.166107 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-37d62426-0f64-425c-8573-1d68f1337e6d tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4fa446573f7d42de816280280c3dec19', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ce30584a7dfa4ab19de73aa8b18b113b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:37:35.176631 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-4d8cface-31f3-40cb-8457-b580df9616e0 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] 10.222.0.16 "GET /compute/v2.1/servers/4df09776-dc3c-46ff-9e64-e20b5d4679fa" status: 200 len: 1683 microversion: 2.1 time: 0.394844 Jul 16 21:37:35.181858 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-37d62426-0f64-425c-8573-1d68f1337e6d tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] 10.222.0.16 "GET /compute/v2.1/servers/3c0f0855-7a9c-4d8a-95ac-3bf41d32d4e6" status: 200 len: 1731 microversion: 2.1 time: 0.411227 Jul 16 21:37:35.184333 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2592/5187] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:37:34 2022] GET /compute/v2.1/servers/4df09776-dc3c-46ff-9e64-e20b5d4679fa => generated 1683 bytes in 403 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:37:35.197720 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2596/5188] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:37:34 2022] GET /compute/v2.1/servers/3c0f0855-7a9c-4d8a-95ac-3bf41d32d4e6 => generated 1731 bytes in 425 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:37:36.126775 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-1a8e1c66-0a45-4fa6-93bf-e98decc5f80e req-16081cde-170f-4ea6-a391-648262862f8f service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "3c0f0855-7a9c-4d8a-95ac-3bf41d32d4e6", "name": "network-vif-deleted", "tag": "a9915b3b-bd00-4ade-884c-60255466029d"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:37:36.142899 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-1a8e1c66-0a45-4fa6-93bf-e98decc5f80e req-16081cde-170f-4ea6-a391-648262862f8f service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:37:36.144852 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-1a8e1c66-0a45-4fa6-93bf-e98decc5f80e req-16081cde-170f-4ea6-a391-648262862f8f service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:37:36.186957 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-1a8e1c66-0a45-4fa6-93bf-e98decc5f80e req-16081cde-170f-4ea6-a391-648262862f8f service nova] Creating event network-vif-deleted:a9915b3b-bd00-4ade-884c-60255466029d for instance 3c0f0855-7a9c-4d8a-95ac-3bf41d32d4e6 on nch1-master-1 Jul 16 21:37:36.196949 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-1a8e1c66-0a45-4fa6-93bf-e98decc5f80e req-16081cde-170f-4ea6-a391-648262862f8f service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.074429 Jul 16 21:37:36.197887 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2597/5189] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:37:36 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 78 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:37:36.207453 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-19e147ab-e604-4c7e-8478-94a048ee7385 tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:37:36.208512 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-19e147ab-e604-4c7e-8478-94a048ee7385 tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] [instance: 3c0f0855-7a9c-4d8a-95ac-3bf41d32d4e6] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:37:36.211027 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-4b735d15-e4fb-4850-9ab4-fef599ccd788 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:37:36.211752 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-4b735d15-e4fb-4850-9ab4-fef599ccd788 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] [instance: 4df09776-dc3c-46ff-9e64-e20b5d4679fa] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:37:36.232656 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4b735d15-e4fb-4850-9ab4-fef599ccd788 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:37:36.233157 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4b735d15-e4fb-4850-9ab4-fef599ccd788 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:37:36.233982 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4b735d15-e4fb-4850-9ab4-fef599ccd788 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:37:36.234430 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4b735d15-e4fb-4850-9ab4-fef599ccd788 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:37:36.246956 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-19e147ab-e604-4c7e-8478-94a048ee7385 tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:37:36.247667 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-19e147ab-e604-4c7e-8478-94a048ee7385 tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:37:36.249923 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-19e147ab-e604-4c7e-8478-94a048ee7385 tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:37:36.251059 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-19e147ab-e604-4c7e-8478-94a048ee7385 tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:37:36.459731 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-4b735d15-e4fb-4850-9ab4-fef599ccd788 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0ab8b3f06e994bbca5d6dd14109bfe50', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e60c0863dc6f4313bdc873fda27ae2fc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:37:36.461756 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-19e147ab-e604-4c7e-8478-94a048ee7385 tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4fa446573f7d42de816280280c3dec19', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ce30584a7dfa4ab19de73aa8b18b113b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:37:36.475371 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-19e147ab-e604-4c7e-8478-94a048ee7385 tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] 10.222.0.16 "GET /compute/v2.1/servers/3c0f0855-7a9c-4d8a-95ac-3bf41d32d4e6" status: 200 len: 1664 microversion: 2.1 time: 0.273889 Jul 16 21:37:36.478414 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2598/5190] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:37:36 2022] GET /compute/v2.1/servers/3c0f0855-7a9c-4d8a-95ac-3bf41d32d4e6 => generated 1664 bytes in 278 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:37:36.484828 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-4b735d15-e4fb-4850-9ab4-fef599ccd788 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] 10.222.0.16 "GET /compute/v2.1/servers/4df09776-dc3c-46ff-9e64-e20b5d4679fa" status: 200 len: 1615 microversion: 2.1 time: 0.277572 Jul 16 21:37:36.485604 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2593/5191] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:37:36 2022] GET /compute/v2.1/servers/4df09776-dc3c-46ff-9e64-e20b5d4679fa => generated 1615 bytes in 288 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:37:36.595387 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-ba6ab407-7f3b-44f6-8a92-8dd0cbfc23a5 req-5b4e25cd-31c8-4774-b4fc-a41fd9e121af service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "4df09776-dc3c-46ff-9e64-e20b5d4679fa", "name": "network-vif-deleted", "tag": "cc6f4c0a-93e4-4c67-8490-1bb34324c862"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:37:36.607521 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-ba6ab407-7f3b-44f6-8a92-8dd0cbfc23a5 req-5b4e25cd-31c8-4774-b4fc-a41fd9e121af service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:37:36.612081 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-ba6ab407-7f3b-44f6-8a92-8dd0cbfc23a5 req-5b4e25cd-31c8-4774-b4fc-a41fd9e121af service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:37:36.635366 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-ba6ab407-7f3b-44f6-8a92-8dd0cbfc23a5 req-5b4e25cd-31c8-4774-b4fc-a41fd9e121af service nova] Creating event network-vif-deleted:cc6f4c0a-93e4-4c67-8490-1bb34324c862 for instance 4df09776-dc3c-46ff-9e64-e20b5d4679fa on nch1-master-1 Jul 16 21:37:36.656866 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-ba6ab407-7f3b-44f6-8a92-8dd0cbfc23a5 req-5b4e25cd-31c8-4774-b4fc-a41fd9e121af service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.066381 Jul 16 21:37:36.661125 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2599/5192] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:37:36 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 69 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 16 21:37:37.505321 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-d6953298-1ad4-4267-b920-62d98c5859bd tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:37:37.511857 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-a4ba3994-c0f1-475b-bbca-72f05937fcff tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:37:37.511857 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-a4ba3994-c0f1-475b-bbca-72f05937fcff tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] [instance: 4df09776-dc3c-46ff-9e64-e20b5d4679fa] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:37:37.513942 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-d6953298-1ad4-4267-b920-62d98c5859bd tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] [instance: 3c0f0855-7a9c-4d8a-95ac-3bf41d32d4e6] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:37:37.532952 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d6953298-1ad4-4267-b920-62d98c5859bd tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:37:37.537248 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a4ba3994-c0f1-475b-bbca-72f05937fcff tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:37:37.537714 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a4ba3994-c0f1-475b-bbca-72f05937fcff tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:37:37.538498 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a4ba3994-c0f1-475b-bbca-72f05937fcff tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:37:37.538761 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d6953298-1ad4-4267-b920-62d98c5859bd tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:37:37.540553 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d6953298-1ad4-4267-b920-62d98c5859bd tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:37:37.541166 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a4ba3994-c0f1-475b-bbca-72f05937fcff tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:37:37.541788 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d6953298-1ad4-4267-b920-62d98c5859bd tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:37:37.633712 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a4ba3994-c0f1-475b-bbca-72f05937fcff tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:37:37.634761 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a4ba3994-c0f1-475b-bbca-72f05937fcff tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:37:37.635311 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d6953298-1ad4-4267-b920-62d98c5859bd tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:37:37.635737 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.objects.instance [None req-a4ba3994-c0f1-475b-bbca-72f05937fcff tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lazy-loading 'fault' on Instance uuid 4df09776-dc3c-46ff-9e64-e20b5d4679fa {{(pid=138681) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:37:37.635962 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d6953298-1ad4-4267-b920-62d98c5859bd tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.009s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:37:37.636830 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.objects.instance [None req-d6953298-1ad4-4267-b920-62d98c5859bd tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] Lazy-loading 'fault' on Instance uuid 3c0f0855-7a9c-4d8a-95ac-3bf41d32d4e6 {{(pid=138680) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:37:37.714013 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-d6953298-1ad4-4267-b920-62d98c5859bd tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4fa446573f7d42de816280280c3dec19', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ce30584a7dfa4ab19de73aa8b18b113b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:37:37.729234 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-d6953298-1ad4-4267-b920-62d98c5859bd tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] 10.222.0.16 "GET /compute/v2.1/servers/3c0f0855-7a9c-4d8a-95ac-3bf41d32d4e6" status: 200 len: 1388 microversion: 2.1 time: 0.235760 Jul 16 21:37:37.729835 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2594/5193] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:37:37 2022] GET /compute/v2.1/servers/3c0f0855-7a9c-4d8a-95ac-3bf41d32d4e6 => generated 1388 bytes in 240 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:37:37.753901 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-a4ba3994-c0f1-475b-bbca-72f05937fcff tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0ab8b3f06e994bbca5d6dd14109bfe50', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e60c0863dc6f4313bdc873fda27ae2fc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:37:37.763611 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-a4ba3994-c0f1-475b-bbca-72f05937fcff tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] 10.222.0.16 "GET /compute/v2.1/servers/4df09776-dc3c-46ff-9e64-e20b5d4679fa" status: 200 len: 1331 microversion: 2.1 time: 0.263221 Jul 16 21:37:37.763889 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2600/5194] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:37:37 2022] GET /compute/v2.1/servers/4df09776-dc3c-46ff-9e64-e20b5d4679fa => generated 1331 bytes in 265 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:37:38.749375 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-c39079c9-582e-4b18-9d43-67be546bf50c tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:37:38.750000 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-c39079c9-582e-4b18-9d43-67be546bf50c tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] [instance: 3c0f0855-7a9c-4d8a-95ac-3bf41d32d4e6] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:37:38.767208 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c39079c9-582e-4b18-9d43-67be546bf50c tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:37:38.768095 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c39079c9-582e-4b18-9d43-67be546bf50c tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:37:38.769451 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c39079c9-582e-4b18-9d43-67be546bf50c tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:37:38.770058 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c39079c9-582e-4b18-9d43-67be546bf50c tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:37:38.784070 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-d8115709-3c87-440d-94c4-1fd4e3274258 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:37:38.794302 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-c39079c9-582e-4b18-9d43-67be546bf50c tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] HTTP exception thrown: Instance 3c0f0855-7a9c-4d8a-95ac-3bf41d32d4e6 could not be found. Jul 16 21:37:38.795842 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-d8115709-3c87-440d-94c4-1fd4e3274258 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] [instance: 4df09776-dc3c-46ff-9e64-e20b5d4679fa] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:37:38.797839 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-c39079c9-582e-4b18-9d43-67be546bf50c tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] Request method failure captured: Jul 16 21:37:38.797839 ncdv-master-1 devstack@n-api.service[138680]: request: GET /compute/v2.1/servers/3c0f0855-7a9c-4d8a-95ac-3bf41d32d4e6 HTTP/1.1 Jul 16 21:37:38.797839 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:37:38.797839 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:37:38.797839 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:37:38.797839 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 0 Jul 16 21:37:38.797839 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:37:38.797839 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:37:38.797839 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:37:38.797839 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:37:38.797839 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0y5ZrPZ1DEmevFrmgwHafSnIPEjm0uhxavKzAvizBLU0g-OA_7sbuLApCfxlXmMGp6k1lScbkeiyJZR66Sxo77RCWm9AKpniGHH9C0aBh3D44ijHC3yktjzaeYJSYnukgPz-2B91Exg036zBj1M2ZxOY1LQOCiV9EUjhqThmPPgSSr8 Jul 16 21:37:38.797839 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:37:38.797839 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:37:38.797839 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:37:38.797839 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:37:38.797839 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:37:38.797839 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:37:38.797839 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: ce30584a7dfa4ab19de73aa8b18b113b Jul 16 21:37:38.797839 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-TestServerBasicOps-921841537 Jul 16 21:37:38.797839 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:37:38.797839 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:37:38.797839 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/ce30584a7dfa4ab19de73aa8b18b113b"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/ce30584a7dfa4ab19de73aa8b18b113b"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_ce30584a7dfa4ab19de73aa8b18b113b"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/ce30584a7dfa4ab19de73aa8b18b113b"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:37:38.797839 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-TestServerBasicOps-921841537 Jul 16 21:37:38.797839 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: ce30584a7dfa4ab19de73aa8b18b113b Jul 16 21:37:38.797839 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-TestServerBasicOps-921841537 Jul 16 21:37:38.797839 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-TestServerBasicOps-921841537-project Jul 16 21:37:38.797839 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:37:38.797839 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:37:38.797839 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: 4fa446573f7d42de816280280c3dec19 Jul 16 21:37:38.797839 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-TestServerBasicOps-921841537-project Jul 16 21:37:38.797839 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 21:37:38.797839 ncdv-master-1 devstack@n-api.service[138680]: exception: Instance 3c0f0855-7a9c-4d8a-95ac-3bf41d32d4e6 could not be found. Jul 16 21:37:38.797839 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': '3c0f0855-7a9c-4d8a-95ac-3bf41d32d4e6'} Jul 16 21:37:38.797839 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:37:38.797839 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:37:38.797839 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:37:38.797839 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:37:38.797839 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:37:38.797839 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:37:38.797839 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:37:38.797839 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:37:38.797839 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:37:38.797839 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:37:38.797839 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 3c0f0855-7a9c-4d8a-95ac-3bf41d32d4e6 could not be found. Jul 16 21:37:38.797839 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:37:38.797839 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:37:38.797839 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:37:38.797839 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:37:38.797839 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:37:38.797839 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:37:38.797839 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:37:38.797839 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:37:38.797839 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:37:38.797839 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:37:38.797839 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:37:38.797839 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:37:38.797839 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:37:38.797839 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:37:38.797839 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:37:38.797839 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:37:38.797839 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 3c0f0855-7a9c-4d8a-95ac-3bf41d32d4e6 could not be found. Jul 16 21:37:38.797839 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:37:38.797839 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:37:38.797839 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:37:38.801506 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:37:38.801506 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:37:38.801506 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:37:38.801506 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:37:38.801506 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:37:38.801506 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 3c0f0855-7a9c-4d8a-95ac-3bf41d32d4e6 could not be found. Jul 16 21:37:38.801506 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:37:38.803190 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-c39079c9-582e-4b18-9d43-67be546bf50c tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] Returning 404 to user: Instance 3c0f0855-7a9c-4d8a-95ac-3bf41d32d4e6 could not be found. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:37:38.804368 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-c39079c9-582e-4b18-9d43-67be546bf50c tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] 10.222.0.16 "GET /compute/v2.1/servers/3c0f0855-7a9c-4d8a-95ac-3bf41d32d4e6" status: 404 len: 111 microversion: 2.1 time: 0.060042 Jul 16 21:37:38.805226 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2595/5195] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:37:38 2022] GET /compute/v2.1/servers/3c0f0855-7a9c-4d8a-95ac-3bf41d32d4e6 => generated 111 bytes in 63 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:37:38.809754 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d8115709-3c87-440d-94c4-1fd4e3274258 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:37:38.810667 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d8115709-3c87-440d-94c4-1fd4e3274258 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:37:38.812423 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d8115709-3c87-440d-94c4-1fd4e3274258 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:37:38.815454 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d8115709-3c87-440d-94c4-1fd4e3274258 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:37:38.840681 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-d8115709-3c87-440d-94c4-1fd4e3274258 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] HTTP exception thrown: Instance 4df09776-dc3c-46ff-9e64-e20b5d4679fa could not be found. Jul 16 21:37:38.842581 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-d8115709-3c87-440d-94c4-1fd4e3274258 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Request method failure captured: Jul 16 21:37:38.842581 ncdv-master-1 devstack@n-api.service[138681]: request: GET /compute/v2.1/servers/4df09776-dc3c-46ff-9e64-e20b5d4679fa HTTP/1.1 Jul 16 21:37:38.842581 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:37:38.842581 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:37:38.842581 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:37:38.842581 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 21:37:38.842581 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:37:38.842581 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:37:38.842581 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:37:38.842581 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:37:38.842581 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0y8gcGxfa3qumeDpAV_UzHtBqxWw5fgWdXEsN_P0XTe84Os21rKjc9QIlpjQBg7EBAnrc9gi4HTfF6VkzKBMP6W-EwHOC7W6W2mOvP55miTafGS3QI8-v38KjXcvgCGFOUcX_9nbDaYEoaZ_SfO91myDjzRdtv09P0hU8CwskGtYm8E Jul 16 21:37:38.842581 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:37:38.842581 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:37:38.842581 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:37:38.842581 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:37:38.842581 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:37:38.842581 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:37:38.842581 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: e60c0863dc6f4313bdc873fda27ae2fc Jul 16 21:37:38.842581 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-AttachVolumeNegativeTest-17004004 Jul 16 21:37:38.842581 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:37:38.842581 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:37:38.842581 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/e60c0863dc6f4313bdc873fda27ae2fc"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/e60c0863dc6f4313bdc873fda27ae2fc"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_e60c0863dc6f4313bdc873fda27ae2fc"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/e60c0863dc6f4313bdc873fda27ae2fc"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:37:38.842581 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-AttachVolumeNegativeTest-17004004 Jul 16 21:37:38.842581 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: e60c0863dc6f4313bdc873fda27ae2fc Jul 16 21:37:38.842581 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-17004004 Jul 16 21:37:38.842581 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-AttachVolumeNegativeTest-17004004-project Jul 16 21:37:38.842581 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:37:38.842581 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:37:38.842581 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: 0ab8b3f06e994bbca5d6dd14109bfe50 Jul 16 21:37:38.842581 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-AttachVolumeNegativeTest-17004004-project Jul 16 21:37:38.842581 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:37:38.842581 ncdv-master-1 devstack@n-api.service[138681]: exception: Instance 4df09776-dc3c-46ff-9e64-e20b5d4679fa could not be found. Jul 16 21:37:38.842581 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': '4df09776-dc3c-46ff-9e64-e20b5d4679fa'} Jul 16 21:37:38.842581 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:37:38.842581 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:37:38.842581 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:37:38.842581 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:37:38.842581 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:37:38.842581 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:37:38.842581 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:37:38.842581 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:37:38.842581 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:37:38.842581 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:37:38.842581 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 4df09776-dc3c-46ff-9e64-e20b5d4679fa could not be found. Jul 16 21:37:38.842581 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:37:38.842581 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:37:38.842581 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:37:38.842581 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:37:38.842581 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:37:38.842581 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:37:38.842581 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:37:38.842581 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:37:38.842581 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:37:38.842581 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:37:38.842581 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:37:38.842581 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:37:38.842581 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:37:38.842581 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:37:38.842581 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:37:38.842581 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:37:38.842581 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 4df09776-dc3c-46ff-9e64-e20b5d4679fa could not be found. Jul 16 21:37:38.842581 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:37:38.842581 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:37:38.842581 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:37:38.846766 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:37:38.846766 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:37:38.846766 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:37:38.846766 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:37:38.846766 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:37:38.846766 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 4df09776-dc3c-46ff-9e64-e20b5d4679fa could not be found. Jul 16 21:37:38.846766 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:37:38.846766 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-d8115709-3c87-440d-94c4-1fd4e3274258 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Returning 404 to user: Instance 4df09776-dc3c-46ff-9e64-e20b5d4679fa could not be found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:37:38.846766 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-d8115709-3c87-440d-94c4-1fd4e3274258 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] 10.222.0.16 "GET /compute/v2.1/servers/4df09776-dc3c-46ff-9e64-e20b5d4679fa" status: 404 len: 111 microversion: 2.1 time: 0.065222 Jul 16 21:37:38.857270 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-c5683705-bff1-4228-981c-f6cbe5e18150 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Calling method '.version_select at 0x7fdd8fcf9820>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:37:38.872553 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2601/5196] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:37:38 2022] GET /compute/v2.1/servers/4df09776-dc3c-46ff-9e64-e20b5d4679fa => generated 111 bytes in 70 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:37:39.021207 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-c5683705-bff1-4228-981c-f6cbe5e18150 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] 10.222.0.16 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1708417433" status: 202 len: 0 microversion: 2.1 time: 0.170180 Jul 16 21:37:39.026057 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2596/5197] 10.222.0.16 () {58 vars in 1220 bytes} [Sat Jul 16 21:37:38 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1708417433 => generated 0 bytes in 176 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 21:37:39.782987 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-a8dd0cfd-8b6d-4b3e-826f-33336824c69d tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:37:39.784061 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-a8dd0cfd-8b6d-4b3e-826f-33336824c69d tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] [instance: 3c0f0855-7a9c-4d8a-95ac-3bf41d32d4e6] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:37:39.793833 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a8dd0cfd-8b6d-4b3e-826f-33336824c69d tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:37:39.794492 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a8dd0cfd-8b6d-4b3e-826f-33336824c69d tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:37:39.795950 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a8dd0cfd-8b6d-4b3e-826f-33336824c69d tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:37:39.796281 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a8dd0cfd-8b6d-4b3e-826f-33336824c69d tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:37:39.815297 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-a8dd0cfd-8b6d-4b3e-826f-33336824c69d tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] HTTP exception thrown: Instance 3c0f0855-7a9c-4d8a-95ac-3bf41d32d4e6 could not be found. Jul 16 21:37:39.817016 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-a8dd0cfd-8b6d-4b3e-826f-33336824c69d tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] Request method failure captured: Jul 16 21:37:39.817016 ncdv-master-1 devstack@n-api.service[138681]: request: DELETE /compute/v2.1/servers/3c0f0855-7a9c-4d8a-95ac-3bf41d32d4e6 HTTP/1.1 Jul 16 21:37:39.817016 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:37:39.817016 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:37:39.817016 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:37:39.817016 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 21:37:39.817016 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:37:39.817016 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:37:39.817016 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:37:39.817016 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:37:39.817016 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0y5ZrPZ1DEmevFrmgwHafSnIPEjm0uhxavKzAvizBLU0g-OA_7sbuLApCfxlXmMGp6k1lScbkeiyJZR66Sxo77RCWm9AKpniGHH9C0aBh3D44ijHC3yktjzaeYJSYnukgPz-2B91Exg036zBj1M2ZxOY1LQOCiV9EUjhqThmPPgSSr8 Jul 16 21:37:39.817016 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:37:39.817016 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:37:39.817016 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:37:39.817016 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:37:39.817016 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:37:39.817016 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:37:39.817016 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: ce30584a7dfa4ab19de73aa8b18b113b Jul 16 21:37:39.817016 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-TestServerBasicOps-921841537 Jul 16 21:37:39.817016 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:37:39.817016 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:37:39.817016 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/ce30584a7dfa4ab19de73aa8b18b113b"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/ce30584a7dfa4ab19de73aa8b18b113b"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_ce30584a7dfa4ab19de73aa8b18b113b"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/ce30584a7dfa4ab19de73aa8b18b113b"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:37:39.817016 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-TestServerBasicOps-921841537 Jul 16 21:37:39.817016 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: ce30584a7dfa4ab19de73aa8b18b113b Jul 16 21:37:39.817016 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-TestServerBasicOps-921841537 Jul 16 21:37:39.817016 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-TestServerBasicOps-921841537-project Jul 16 21:37:39.817016 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:37:39.817016 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:37:39.817016 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: 4fa446573f7d42de816280280c3dec19 Jul 16 21:37:39.817016 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-TestServerBasicOps-921841537-project Jul 16 21:37:39.817016 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:37:39.817016 ncdv-master-1 devstack@n-api.service[138681]: exception: Instance 3c0f0855-7a9c-4d8a-95ac-3bf41d32d4e6 could not be found. Jul 16 21:37:39.817016 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': '3c0f0855-7a9c-4d8a-95ac-3bf41d32d4e6'} Jul 16 21:37:39.817016 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:37:39.817016 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:37:39.817016 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:37:39.817016 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:37:39.817016 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:37:39.817016 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:37:39.817016 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:37:39.817016 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:37:39.817016 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:37:39.817016 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:37:39.817016 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 3c0f0855-7a9c-4d8a-95ac-3bf41d32d4e6 could not be found. Jul 16 21:37:39.817016 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:37:39.817016 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:37:39.817016 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:37:39.817016 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:37:39.817016 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:37:39.817016 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:37:39.817016 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:37:39.817016 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:37:39.817016 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:37:39.817016 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:37:39.817016 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Jul 16 21:37:39.817016 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Jul 16 21:37:39.817016 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Jul 16 21:37:39.817016 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Jul 16 21:37:39.817016 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:37:39.817016 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:37:39.817016 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:37:39.817016 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:37:39.817016 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 3c0f0855-7a9c-4d8a-95ac-3bf41d32d4e6 could not be found. Jul 16 21:37:39.821342 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:37:39.821342 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:37:39.821342 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:37:39.821342 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:37:39.821342 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:37:39.821342 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:37:39.821342 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:37:39.821342 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:37:39.821342 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 3c0f0855-7a9c-4d8a-95ac-3bf41d32d4e6 could not be found. Jul 16 21:37:39.821342 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:37:39.821342 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-a8dd0cfd-8b6d-4b3e-826f-33336824c69d tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] Returning 404 to user: Instance 3c0f0855-7a9c-4d8a-95ac-3bf41d32d4e6 could not be found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:37:39.822739 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-a8dd0cfd-8b6d-4b3e-826f-33336824c69d tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] 10.222.0.16 "DELETE /compute/v2.1/servers/3c0f0855-7a9c-4d8a-95ac-3bf41d32d4e6" status: 404 len: 111 microversion: 2.1 time: 0.043828 Jul 16 21:37:39.832183 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-6aba3ad9-1587-416d-a64f-76941d034b31 tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:37:39.833540 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-6aba3ad9-1587-416d-a64f-76941d034b31 tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] [instance: 3c0f0855-7a9c-4d8a-95ac-3bf41d32d4e6] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:37:39.835191 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2602/5198] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:37:39 2022] DELETE /compute/v2.1/servers/3c0f0855-7a9c-4d8a-95ac-3bf41d32d4e6 => generated 111 bytes in 57 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:37:39.842538 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6aba3ad9-1587-416d-a64f-76941d034b31 tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:37:39.843023 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6aba3ad9-1587-416d-a64f-76941d034b31 tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:37:39.843976 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6aba3ad9-1587-416d-a64f-76941d034b31 tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:37:39.844449 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6aba3ad9-1587-416d-a64f-76941d034b31 tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:37:39.859879 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-6aba3ad9-1587-416d-a64f-76941d034b31 tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] HTTP exception thrown: Instance 3c0f0855-7a9c-4d8a-95ac-3bf41d32d4e6 could not be found. Jul 16 21:37:39.861450 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-6aba3ad9-1587-416d-a64f-76941d034b31 tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] Request method failure captured: Jul 16 21:37:39.861450 ncdv-master-1 devstack@n-api.service[138680]: request: GET /compute/v2.1/servers/3c0f0855-7a9c-4d8a-95ac-3bf41d32d4e6 HTTP/1.1 Jul 16 21:37:39.861450 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:37:39.861450 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:37:39.861450 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:37:39.861450 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 0 Jul 16 21:37:39.861450 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:37:39.861450 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:37:39.861450 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:37:39.861450 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:37:39.861450 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0y5ZrPZ1DEmevFrmgwHafSnIPEjm0uhxavKzAvizBLU0g-OA_7sbuLApCfxlXmMGp6k1lScbkeiyJZR66Sxo77RCWm9AKpniGHH9C0aBh3D44ijHC3yktjzaeYJSYnukgPz-2B91Exg036zBj1M2ZxOY1LQOCiV9EUjhqThmPPgSSr8 Jul 16 21:37:39.861450 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:37:39.861450 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:37:39.861450 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:37:39.861450 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:37:39.861450 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:37:39.861450 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:37:39.861450 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: ce30584a7dfa4ab19de73aa8b18b113b Jul 16 21:37:39.861450 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-TestServerBasicOps-921841537 Jul 16 21:37:39.861450 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:37:39.861450 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:37:39.861450 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/ce30584a7dfa4ab19de73aa8b18b113b"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/ce30584a7dfa4ab19de73aa8b18b113b"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_ce30584a7dfa4ab19de73aa8b18b113b"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/ce30584a7dfa4ab19de73aa8b18b113b"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:37:39.861450 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-TestServerBasicOps-921841537 Jul 16 21:37:39.861450 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: ce30584a7dfa4ab19de73aa8b18b113b Jul 16 21:37:39.861450 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-TestServerBasicOps-921841537 Jul 16 21:37:39.861450 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-TestServerBasicOps-921841537-project Jul 16 21:37:39.861450 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:37:39.861450 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:37:39.861450 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: 4fa446573f7d42de816280280c3dec19 Jul 16 21:37:39.861450 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-TestServerBasicOps-921841537-project Jul 16 21:37:39.861450 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 21:37:39.861450 ncdv-master-1 devstack@n-api.service[138680]: exception: Instance 3c0f0855-7a9c-4d8a-95ac-3bf41d32d4e6 could not be found. Jul 16 21:37:39.861450 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': '3c0f0855-7a9c-4d8a-95ac-3bf41d32d4e6'} Jul 16 21:37:39.861450 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:37:39.861450 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:37:39.861450 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:37:39.861450 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:37:39.861450 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:37:39.861450 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:37:39.861450 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:37:39.861450 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:37:39.861450 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:37:39.861450 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:37:39.861450 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 3c0f0855-7a9c-4d8a-95ac-3bf41d32d4e6 could not be found. Jul 16 21:37:39.861450 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:37:39.861450 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:37:39.861450 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:37:39.861450 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:37:39.861450 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:37:39.861450 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:37:39.861450 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:37:39.861450 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:37:39.861450 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:37:39.861450 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:37:39.861450 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:37:39.861450 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:37:39.861450 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:37:39.861450 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:37:39.861450 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:37:39.861450 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:37:39.861450 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 3c0f0855-7a9c-4d8a-95ac-3bf41d32d4e6 could not be found. Jul 16 21:37:39.861450 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:37:39.861450 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:37:39.861450 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:37:39.865892 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:37:39.865892 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:37:39.865892 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:37:39.865892 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:37:39.865892 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:37:39.865892 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 3c0f0855-7a9c-4d8a-95ac-3bf41d32d4e6 could not be found. Jul 16 21:37:39.865892 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:37:39.867080 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-6aba3ad9-1587-416d-a64f-76941d034b31 tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] Returning 404 to user: Instance 3c0f0855-7a9c-4d8a-95ac-3bf41d32d4e6 could not be found. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:37:39.868492 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-6aba3ad9-1587-416d-a64f-76941d034b31 tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] 10.222.0.16 "GET /compute/v2.1/servers/3c0f0855-7a9c-4d8a-95ac-3bf41d32d4e6" status: 404 len: 111 microversion: 2.1 time: 0.040271 Jul 16 21:37:39.869304 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2597/5199] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:37:39 2022] GET /compute/v2.1/servers/3c0f0855-7a9c-4d8a-95ac-3bf41d32d4e6 => generated 111 bytes in 42 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:37:40.249743 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-75361961-54bc-40e1-8e17-1296eea08de8 tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] Calling method '.version_select at 0x7fdda2b89310>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:37:40.396382 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-75361961-54bc-40e1-8e17-1296eea08de8 tempest-TestServerBasicOps-921841537 tempest-TestServerBasicOps-921841537-project] 10.222.0.16 "DELETE /compute/v2.1/os-keypairs/tempest-TestServerBasicOps-956155148" status: 202 len: 0 microversion: 2.1 time: 0.150230 Jul 16 21:37:40.397119 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2603/5200] 10.222.0.16 () {58 vars in 1250 bytes} [Sat Jul 16 21:37:40 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestServerBasicOps-956155148 => generated 0 bytes in 152 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 21:37:40.449670 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-ba0dfc38-94de-4d76-8f08-7affca913aa2 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Action: 'create', calling method: .version_select at 0x7fdda2983820>, body: {"keypair": {"name": "tempest-keypair-1042899521", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBGvFYthj99dGJ6ek3dTiGUAzVUhrPCSsf63jSBvtURa6GkNex69cUrBOJBZhGYkLwX16MnmIjoWYUeAye6fRCMi4Bm3QxiEuOE7rHRbno+AghUkxPCua+9JdGxor7wTf7Q=="}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:37:40.480872 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-ba0dfc38-94de-4d76-8f08-7affca913aa2 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Getting quotas for project e60c0863dc6f4313bdc873fda27ae2fc. Resources: {'key_pairs'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:37:40.488232 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-ba0dfc38-94de-4d76-8f08-7affca913aa2 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Getting quotas for user 0ab8b3f06e994bbca5d6dd14109bfe50 and project e60c0863dc6f4313bdc873fda27ae2fc. Resources: {'key_pairs'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:37:40.698051 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-ba0dfc38-94de-4d76-8f08-7affca913aa2 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] 10.222.0.16 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.250559 Jul 16 21:37:40.701357 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2598/5201] 10.222.0.16 () {60 vars in 1158 bytes} [Sat Jul 16 21:37:40 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 257 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:37:45.668591 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-36d09c74-9c09-4260-ad9c-ae729e8fd88f tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeNegativeTest-server-1759811530", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "4775de54-1ba2-4d76-a6e0-a5f8a9ef879c"}], "security_groups": [{"name": "tempest-securitygroup--3980376"}], "key_name": "tempest-keypair-1042899521", "user_data": "}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:37:46.047110 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.network.neutron [None req-36d09c74-9c09-4260-ad9c-ae729e8fd88f tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] validate_networks() for [('4775de54-1ba2-4d76-a6e0-a5f8a9ef879c', None, None, None, None, None)] {{(pid=138681) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 16 21:37:47.342954 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-36d09c74-9c09-4260-ad9c-ae729e8fd88f tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Flavor limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 16 21:37:47.344530 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-36d09c74-9c09-4260-ad9c-ae729e8fd88f tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Image limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 16 21:37:47.345909 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-36d09c74-9c09-4260-ad9c-ae729e8fd88f tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Flavor pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 16 21:37:47.346854 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-36d09c74-9c09-4260-ad9c-ae729e8fd88f tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Image pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 16 21:37:47.349147 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-36d09c74-9c09-4260-ad9c-ae729e8fd88f tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 16 21:37:47.397626 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-36d09c74-9c09-4260-ad9c-ae729e8fd88f tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:37:47.398697 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-36d09c74-9c09-4260-ad9c-ae729e8fd88f tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:37:47.403117 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-36d09c74-9c09-4260-ad9c-ae729e8fd88f tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:37:47.403886 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-36d09c74-9c09-4260-ad9c-ae729e8fd88f tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:37:47.433259 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-36d09c74-9c09-4260-ad9c-ae729e8fd88f tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Getting quotas for project e60c0863dc6f4313bdc873fda27ae2fc. Resources: {'ram', 'instances', 'cores'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:37:47.440237 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-36d09c74-9c09-4260-ad9c-ae729e8fd88f tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Getting quotas for user 0ab8b3f06e994bbca5d6dd14109bfe50 and project e60c0863dc6f4313bdc873fda27ae2fc. Resources: {'ram', 'instances', 'cores'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:37:47.465824 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-36d09c74-9c09-4260-ad9c-ae729e8fd88f tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Going to run 1 instances... {{(pid=138681) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 16 21:37:47.468866 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-36d09c74-9c09-4260-ad9c-ae729e8fd88f tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] [instance: a7f9caa9-fff2-44b2-871d-7e4191ff39e5] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138681) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 16 21:37:47.591370 ncdv-master-1 devstack@n-api.service[138681]: ERROR oslo.messaging._drivers.impl_rabbit [None req-36d09c74-9c09-4260-ad9c-ae729e8fd88f tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] [0c5fcc66-0bfc-47a3-8ed2-9fa620832313] AMQP server on 10.222.0.16:5672 is unreachable: [Errno 104] Connection reset by peer. Trying again in 1 seconds.: ConnectionResetError: [Errno 104] Connection reset by peer Jul 16 21:37:48.601506 ncdv-master-1 devstack@n-api.service[138681]: INFO oslo.messaging._drivers.impl_rabbit [None req-36d09c74-9c09-4260-ad9c-ae729e8fd88f tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] [0c5fcc66-0bfc-47a3-8ed2-9fa620832313] Reconnected to AMQP server on 10.222.0.16:5672 via [amqp] client with port 39744. Jul 16 21:37:48.608011 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-36d09c74-9c09-4260-ad9c-ae729e8fd88f tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 399 microversion: 2.1 time: 2.949718 Jul 16 21:37:48.608778 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2604/5202] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 21:37:45 2022] POST /compute/v2.1/servers => generated 399 bytes in 2951 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 16 21:37:48.621062 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-a9c13ff7-b123-4419-9455-c00ad40daf74 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:37:48.621787 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-a9c13ff7-b123-4419-9455-c00ad40daf74 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] [instance: a7f9caa9-fff2-44b2-871d-7e4191ff39e5] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:37:48.730241 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-a9c13ff7-b123-4419-9455-c00ad40daf74 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0ab8b3f06e994bbca5d6dd14109bfe50', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e60c0863dc6f4313bdc873fda27ae2fc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:37:48.739633 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-a9c13ff7-b123-4419-9455-c00ad40daf74 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] 10.222.0.16 "GET /compute/v2.1/servers/a7f9caa9-fff2-44b2-871d-7e4191ff39e5" status: 200 len: 1242 microversion: 2.1 time: 0.124506 Jul 16 21:37:48.744360 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2599/5203] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:37:48 2022] GET /compute/v2.1/servers/a7f9caa9-fff2-44b2-871d-7e4191ff39e5 => generated 1242 bytes in 130 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:37:49.767319 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-2ddbbb03-7335-47b8-9a86-426ecd19a6ac tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:37:49.772634 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-2ddbbb03-7335-47b8-9a86-426ecd19a6ac tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] [instance: a7f9caa9-fff2-44b2-871d-7e4191ff39e5] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:37:49.854221 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-2ddbbb03-7335-47b8-9a86-426ecd19a6ac tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0ab8b3f06e994bbca5d6dd14109bfe50', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e60c0863dc6f4313bdc873fda27ae2fc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:37:49.863358 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-2ddbbb03-7335-47b8-9a86-426ecd19a6ac tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] 10.222.0.16 "GET /compute/v2.1/servers/a7f9caa9-fff2-44b2-871d-7e4191ff39e5" status: 200 len: 1242 microversion: 2.1 time: 0.103816 Jul 16 21:37:49.864326 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2605/5204] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:37:49 2022] GET /compute/v2.1/servers/a7f9caa9-fff2-44b2-871d-7e4191ff39e5 => generated 1242 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:37:50.888822 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-20f935e8-f153-42f4-9860-25553e5f1f74 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:37:50.888822 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-20f935e8-f153-42f4-9860-25553e5f1f74 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] [instance: a7f9caa9-fff2-44b2-871d-7e4191ff39e5] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:37:50.894759 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=87,nova_cell1:UPDATE=2,nova_cell1:INSERT=3 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:37:50.999105 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-20f935e8-f153-42f4-9860-25553e5f1f74 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0ab8b3f06e994bbca5d6dd14109bfe50', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e60c0863dc6f4313bdc873fda27ae2fc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:37:51.012286 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-20f935e8-f153-42f4-9860-25553e5f1f74 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] 10.222.0.16 "GET /compute/v2.1/servers/a7f9caa9-fff2-44b2-871d-7e4191ff39e5" status: 200 len: 1242 microversion: 2.1 time: 0.134343 Jul 16 21:37:51.023254 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2600/5205] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:37:50 2022] GET /compute/v2.1/servers/a7f9caa9-fff2-44b2-871d-7e4191ff39e5 => generated 1242 bytes in 146 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:37:52.031767 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-1063ef00-eb90-4fb1-837b-ccaa5e354913 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:37:52.032329 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-1063ef00-eb90-4fb1-837b-ccaa5e354913 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] [instance: a7f9caa9-fff2-44b2-871d-7e4191ff39e5] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:37:52.046183 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1063ef00-eb90-4fb1-837b-ccaa5e354913 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:37:52.046644 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1063ef00-eb90-4fb1-837b-ccaa5e354913 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:37:52.047458 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1063ef00-eb90-4fb1-837b-ccaa5e354913 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:37:52.047902 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1063ef00-eb90-4fb1-837b-ccaa5e354913 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:37:52.088727 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:37:52.293519 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-1063ef00-eb90-4fb1-837b-ccaa5e354913 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0ab8b3f06e994bbca5d6dd14109bfe50', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e60c0863dc6f4313bdc873fda27ae2fc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:37:52.314810 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-1063ef00-eb90-4fb1-837b-ccaa5e354913 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] 10.222.0.16 "GET /compute/v2.1/servers/a7f9caa9-fff2-44b2-871d-7e4191ff39e5" status: 200 len: 1246 microversion: 2.1 time: 0.288241 Jul 16 21:37:52.325218 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2606/5206] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:37:52 2022] GET /compute/v2.1/servers/a7f9caa9-fff2-44b2-871d-7e4191ff39e5 => generated 1246 bytes in 290 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:37:53.339342 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-1573accf-462d-408e-ae67-16bb28038aae tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:37:53.340035 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-1573accf-462d-408e-ae67-16bb28038aae tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] [instance: a7f9caa9-fff2-44b2-871d-7e4191ff39e5] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:37:53.352221 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1573accf-462d-408e-ae67-16bb28038aae tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:37:53.355540 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1573accf-462d-408e-ae67-16bb28038aae tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:37:53.355540 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1573accf-462d-408e-ae67-16bb28038aae tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:37:53.355540 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1573accf-462d-408e-ae67-16bb28038aae tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:37:53.502971 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-1573accf-462d-408e-ae67-16bb28038aae tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0ab8b3f06e994bbca5d6dd14109bfe50', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e60c0863dc6f4313bdc873fda27ae2fc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:37:53.520941 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-1573accf-462d-408e-ae67-16bb28038aae tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] 10.222.0.16 "GET /compute/v2.1/servers/a7f9caa9-fff2-44b2-871d-7e4191ff39e5" status: 200 len: 1294 microversion: 2.1 time: 0.189238 Jul 16 21:37:53.538793 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2601/5207] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:37:53 2022] GET /compute/v2.1/servers/a7f9caa9-fff2-44b2-871d-7e4191ff39e5 => generated 1294 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:37:54.549945 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-9e9cb7b3-3564-42be-8739-c51fbdea976d tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:37:54.557968 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-9e9cb7b3-3564-42be-8739-c51fbdea976d tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] [instance: a7f9caa9-fff2-44b2-871d-7e4191ff39e5] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:37:54.569320 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_api:SELECT=69,nova_api:DELETE=2,nova_api:UPDATE=1,nova_api:INSERT=3 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:37:54.572268 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9e9cb7b3-3564-42be-8739-c51fbdea976d tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:37:54.577350 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9e9cb7b3-3564-42be-8739-c51fbdea976d tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:37:54.577350 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9e9cb7b3-3564-42be-8739-c51fbdea976d tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:37:54.577350 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9e9cb7b3-3564-42be-8739-c51fbdea976d tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:37:54.709832 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-9e9cb7b3-3564-42be-8739-c51fbdea976d tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0ab8b3f06e994bbca5d6dd14109bfe50', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e60c0863dc6f4313bdc873fda27ae2fc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:37:54.732509 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-9e9cb7b3-3564-42be-8739-c51fbdea976d tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] 10.222.0.16 "GET /compute/v2.1/servers/a7f9caa9-fff2-44b2-871d-7e4191ff39e5" status: 200 len: 1300 microversion: 2.1 time: 0.190588 Jul 16 21:37:54.733491 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2607/5208] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:37:54 2022] GET /compute/v2.1/servers/a7f9caa9-fff2-44b2-871d-7e4191ff39e5 => generated 1300 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:37:55.540470 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-2b2367f2-e10a-4e61-bced-8a44939f5f3a tempest-TestAggregatesBasicOps-362841127 tempest-TestAggregatesBasicOps-362841127-project-admin] Action: 'create', calling method: >, body: {"aggregate": {"name": "tempest-aggregate-scenario-1563795592", "availability_zone": "foo_zone"}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:37:55.584189 ncdv-master-1 devstack@n-api.service[138680]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-aggregates:create": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:37:55.584189 ncdv-master-1 devstack@n-api.service[138680]: warnings.warn(msg) Jul 16 21:37:55.585681 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.objects.aggregate [None req-2b2367f2-e10a-4e61-bced-8a44939f5f3a tempest-TestAggregatesBasicOps-362841127 tempest-TestAggregatesBasicOps-362841127-project-admin] Generated uuid 6953e445-6ef5-4862-8257-57c9a251a8fd for aggregate {{(pid=138680) create /opt/stack/nova/nova/objects/aggregate.py:298}} Jul 16 21:37:55.620710 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_api:SELECT=65,nova_api:UPDATE=2,nova_api:DELETE=1,nova_api:INSERT=1 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:37:55.736048 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-2b2367f2-e10a-4e61-bced-8a44939f5f3a tempest-TestAggregatesBasicOps-362841127 tempest-TestAggregatesBasicOps-362841127-project-admin] 10.222.0.16 "POST /compute/v2.1/os-aggregates" status: 200 len: 209 microversion: 2.1 time: 0.395509 Jul 16 21:37:55.737418 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2602/5209] 10.222.0.16 () {60 vars in 1163 bytes} [Sat Jul 16 21:37:55 2022] POST /compute/v2.1/os-aggregates => generated 209 bytes in 398 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:37:55.749587 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-5f91d7d5-25c7-4dcb-89aa-59971e010365 tempest-TestAggregatesBasicOps-362841127 tempest-TestAggregatesBasicOps-362841127-project-admin] Action: 'action', calling method: >, body: {"set_metadata": {"metadata": {"meta_key": "meta_value"}}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:37:55.750547 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-4f768176-c4ea-4e25-b8e7-7428e2bf1c4b tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:37:55.751055 ncdv-master-1 devstack@n-api.service[138681]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-aggregates:set_metadata": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:37:55.751055 ncdv-master-1 devstack@n-api.service[138681]: warnings.warn(msg) Jul 16 21:37:55.751382 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-4f768176-c4ea-4e25-b8e7-7428e2bf1c4b tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] [instance: a7f9caa9-fff2-44b2-871d-7e4191ff39e5] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:37:55.758797 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4f768176-c4ea-4e25-b8e7-7428e2bf1c4b tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:37:55.759564 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4f768176-c4ea-4e25-b8e7-7428e2bf1c4b tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:37:55.760380 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4f768176-c4ea-4e25-b8e7-7428e2bf1c4b tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:37:55.761597 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4f768176-c4ea-4e25-b8e7-7428e2bf1c4b tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:37:55.890655 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-5f91d7d5-25c7-4dcb-89aa-59971e010365 tempest-TestAggregatesBasicOps-362841127 tempest-TestAggregatesBasicOps-362841127-project-admin] 10.222.0.16 "POST /compute/v2.1/os-aggregates/19/action" status: 200 len: 319 microversion: 2.1 time: 0.144733 Jul 16 21:37:55.892149 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2608/5210] 10.222.0.16 () {60 vars in 1193 bytes} [Sat Jul 16 21:37:55 2022] POST /compute/v2.1/os-aggregates/19/action => generated 319 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:37:55.902678 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-1142199b-74ea-4236-a992-13c459855954 tempest-TestAggregatesBasicOps-362841127 tempest-TestAggregatesBasicOps-362841127-project-admin] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:37:55.904874 ncdv-master-1 devstack@n-api.service[138681]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-services:list": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:37:55.904874 ncdv-master-1 devstack@n-api.service[138681]: warnings.warn(msg) Jul 16 21:37:55.906769 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1142199b-74ea-4236-a992-13c459855954 tempest-TestAggregatesBasicOps-362841127 tempest-TestAggregatesBasicOps-362841127-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:37:55.907288 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1142199b-74ea-4236-a992-13c459855954 tempest-TestAggregatesBasicOps-362841127 tempest-TestAggregatesBasicOps-362841127-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:37:55.909794 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1142199b-74ea-4236-a992-13c459855954 tempest-TestAggregatesBasicOps-362841127 tempest-TestAggregatesBasicOps-362841127-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:37:55.910253 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1142199b-74ea-4236-a992-13c459855954 tempest-TestAggregatesBasicOps-362841127 tempest-TestAggregatesBasicOps-362841127-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:37:55.963653 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-4f768176-c4ea-4e25-b8e7-7428e2bf1c4b tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0ab8b3f06e994bbca5d6dd14109bfe50', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e60c0863dc6f4313bdc873fda27ae2fc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:37:55.964689 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-1142199b-74ea-4236-a992-13c459855954 tempest-TestAggregatesBasicOps-362841127 tempest-TestAggregatesBasicOps-362841127-project-admin] 10.222.0.16 "GET /compute/v2.1/os-services?binary=nova-compute" status: 200 len: 380 microversion: 2.1 time: 0.066517 Jul 16 21:37:55.965342 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2609/5211] 10.222.0.16 () {58 vars in 1175 bytes} [Sat Jul 16 21:37:55 2022] GET /compute/v2.1/os-services?binary=nova-compute => generated 380 bytes in 68 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 16 21:37:55.976623 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-b7bd7d58-132e-4526-981d-c8c5cee97750 tempest-TestAggregatesBasicOps-362841127 tempest-TestAggregatesBasicOps-362841127-project-admin] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:37:55.977433 ncdv-master-1 devstack@n-api.service[138681]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-aggregates:index": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:37:55.977433 ncdv-master-1 devstack@n-api.service[138681]: warnings.warn(msg) Jul 16 21:37:55.984655 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-b7bd7d58-132e-4526-981d-c8c5cee97750 tempest-TestAggregatesBasicOps-362841127 tempest-TestAggregatesBasicOps-362841127-project-admin] 10.222.0.16 "GET /compute/v2.1/os-aggregates" status: 200 len: 298 microversion: 2.1 time: 0.011823 Jul 16 21:37:55.985545 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2610/5212] 10.222.0.16 () {58 vars in 1142 bytes} [Sat Jul 16 21:37:55 2022] GET /compute/v2.1/os-aggregates => generated 298 bytes in 13 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:37:55.986038 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-4f768176-c4ea-4e25-b8e7-7428e2bf1c4b tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] 10.222.0.16 "GET /compute/v2.1/servers/a7f9caa9-fff2-44b2-871d-7e4191ff39e5" status: 200 len: 1365 microversion: 2.1 time: 0.240054 Jul 16 21:37:55.995264 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2603/5213] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:37:55 2022] GET /compute/v2.1/servers/a7f9caa9-fff2-44b2-871d-7e4191ff39e5 => generated 1365 bytes in 242 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:37:55.996506 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-7d1e8055-0fe3-4ada-8bb6-2cfc411d1123 tempest-TestAggregatesBasicOps-362841127 tempest-TestAggregatesBasicOps-362841127-project-admin] Action: 'action', calling method: >, body: {"add_host": {"host": "nch1-master-1"}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:37:55.998075 ncdv-master-1 devstack@n-api.service[138681]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-aggregates:add_host": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:37:55.998075 ncdv-master-1 devstack@n-api.service[138681]: warnings.warn(msg) Jul 16 21:37:56.012920 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7d1e8055-0fe3-4ada-8bb6-2cfc411d1123 tempest-TestAggregatesBasicOps-362841127 tempest-TestAggregatesBasicOps-362841127-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:37:56.013985 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7d1e8055-0fe3-4ada-8bb6-2cfc411d1123 tempest-TestAggregatesBasicOps-362841127 tempest-TestAggregatesBasicOps-362841127-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:37:56.466395 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-7d1e8055-0fe3-4ada-8bb6-2cfc411d1123 tempest-TestAggregatesBasicOps-362841127 tempest-TestAggregatesBasicOps-362841127-project-admin] 10.222.0.16 "POST /compute/v2.1/os-aggregates/19/action" status: 200 len: 310 microversion: 2.1 time: 0.473547 Jul 16 21:37:56.470005 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2611/5214] 10.222.0.16 () {60 vars in 1193 bytes} [Sat Jul 16 21:37:55 2022] POST /compute/v2.1/os-aggregates/19/action => generated 310 bytes in 477 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:37:56.479379 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-d7d67913-3b32-4dde-9584-425c3dae8091 tempest-TestAggregatesBasicOps-362841127 tempest-TestAggregatesBasicOps-362841127-project-admin] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:37:56.480159 ncdv-master-1 devstack@n-api.service[138680]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-aggregates:show": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:37:56.480159 ncdv-master-1 devstack@n-api.service[138680]: warnings.warn(msg) Jul 16 21:37:56.502346 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-d7d67913-3b32-4dde-9584-425c3dae8091 tempest-TestAggregatesBasicOps-362841127 tempest-TestAggregatesBasicOps-362841127-project-admin] 10.222.0.16 "GET /compute/v2.1/os-aggregates/19" status: 200 len: 310 microversion: 2.1 time: 0.024010 Jul 16 21:37:56.511222 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2604/5215] 10.222.0.16 () {58 vars in 1151 bytes} [Sat Jul 16 21:37:56 2022] GET /compute/v2.1/os-aggregates/19 => generated 310 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:37:56.513958 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-aabc84e6-ed40-417e-9156-4a657ff32399 tempest-TestAggregatesBasicOps-362841127 tempest-TestAggregatesBasicOps-362841127-project-admin] Action: 'update', calling method: >, body: {"aggregate": {"name": "tempest-renamed-aggregate-scenario-1594291070", "availability_zone": "foo_zone"}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:37:56.515679 ncdv-master-1 devstack@n-api.service[138681]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-aggregates:update": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:37:56.515679 ncdv-master-1 devstack@n-api.service[138681]: warnings.warn(msg) Jul 16 21:37:56.550605 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-010f53c6-04e5-42e9-9020-48e237529d5e req-0a4e4c09-8a9f-4f9a-9c94-a565fb402438 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "a7f9caa9-fff2-44b2-871d-7e4191ff39e5", "tag": "8e33e390-df49-4cb3-a043-fa2763fb499b"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:37:56.579226 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-010f53c6-04e5-42e9-9020-48e237529d5e req-0a4e4c09-8a9f-4f9a-9c94-a565fb402438 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:37:56.581152 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-010f53c6-04e5-42e9-9020-48e237529d5e req-0a4e4c09-8a9f-4f9a-9c94-a565fb402438 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:37:56.668561 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-010f53c6-04e5-42e9-9020-48e237529d5e req-0a4e4c09-8a9f-4f9a-9c94-a565fb402438 service nova] Creating event network-changed:8e33e390-df49-4cb3-a043-fa2763fb499b for instance a7f9caa9-fff2-44b2-871d-7e4191ff39e5 on nch1-master-1 Jul 16 21:37:56.688124 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-010f53c6-04e5-42e9-9020-48e237529d5e req-0a4e4c09-8a9f-4f9a-9c94-a565fb402438 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.142856 Jul 16 21:37:56.689560 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2605/5216] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:37:56 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 16 21:37:56.754182 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-aabc84e6-ed40-417e-9156-4a657ff32399 tempest-TestAggregatesBasicOps-362841127 tempest-TestAggregatesBasicOps-362841127-project-admin] 10.222.0.16 "PUT /compute/v2.1/os-aggregates/19" status: 200 len: 342 microversion: 2.1 time: 0.243989 Jul 16 21:37:56.756581 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2612/5217] 10.222.0.16 () {60 vars in 1172 bytes} [Sat Jul 16 21:37:56 2022] PUT /compute/v2.1/os-aggregates/19 => generated 342 bytes in 248 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:37:56.768010 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-8e37e3ff-598a-4ef2-bf2f-485025e7c34c tempest-TestAggregatesBasicOps-362841127 tempest-TestAggregatesBasicOps-362841127-project-admin] Action: 'action', calling method: >, body: {"set_metadata": {"metadata": {"foo": "bar"}}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:37:56.769390 ncdv-master-1 devstack@n-api.service[138680]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-aggregates:set_metadata": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:37:56.769390 ncdv-master-1 devstack@n-api.service[138680]: warnings.warn(msg) Jul 16 21:37:56.888504 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-8e37e3ff-598a-4ef2-bf2f-485025e7c34c tempest-TestAggregatesBasicOps-362841127 tempest-TestAggregatesBasicOps-362841127-project-admin] 10.222.0.16 "POST /compute/v2.1/os-aggregates/19/action" status: 200 len: 356 microversion: 2.1 time: 0.124601 Jul 16 21:37:56.891979 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2606/5218] 10.222.0.16 () {60 vars in 1193 bytes} [Sat Jul 16 21:37:56 2022] POST /compute/v2.1/os-aggregates/19/action => generated 356 bytes in 129 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:37:56.909003 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-3db37afc-4113-4282-92a1-001683b14708 tempest-TestAggregatesBasicOps-362841127 tempest-TestAggregatesBasicOps-362841127-project-admin] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:37:56.910641 ncdv-master-1 devstack@n-api.service[138681]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-aggregates:show": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:37:56.910641 ncdv-master-1 devstack@n-api.service[138681]: warnings.warn(msg) Jul 16 21:37:56.929549 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-3db37afc-4113-4282-92a1-001683b14708 tempest-TestAggregatesBasicOps-362841127 tempest-TestAggregatesBasicOps-362841127-project-admin] 10.222.0.16 "GET /compute/v2.1/os-aggregates/19" status: 200 len: 356 microversion: 2.1 time: 0.028517 Jul 16 21:37:56.931976 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2613/5219] 10.222.0.16 () {58 vars in 1151 bytes} [Sat Jul 16 21:37:56 2022] GET /compute/v2.1/os-aggregates/19 => generated 356 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 16 21:37:56.943047 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-e1688d0d-93bf-4a1f-93bd-519ef1b60331 tempest-TestAggregatesBasicOps-362841127 tempest-TestAggregatesBasicOps-362841127-project-admin] Action: 'action', calling method: >, body: {"remove_host": {"host": "nch1-master-1"}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:37:56.945151 ncdv-master-1 devstack@n-api.service[138680]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-aggregates:remove_host": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:37:56.945151 ncdv-master-1 devstack@n-api.service[138680]: warnings.warn(msg) Jul 16 21:37:56.958406 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e1688d0d-93bf-4a1f-93bd-519ef1b60331 tempest-TestAggregatesBasicOps-362841127 tempest-TestAggregatesBasicOps-362841127-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:37:56.959275 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e1688d0d-93bf-4a1f-93bd-519ef1b60331 tempest-TestAggregatesBasicOps-362841127 tempest-TestAggregatesBasicOps-362841127-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:37:57.003753 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-0d910548-46a7-4f62-8c4c-d9a8b44ba2d0 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:37:57.005145 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-0d910548-46a7-4f62-8c4c-d9a8b44ba2d0 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] [instance: a7f9caa9-fff2-44b2-871d-7e4191ff39e5] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:37:57.012408 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0d910548-46a7-4f62-8c4c-d9a8b44ba2d0 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:37:57.013201 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0d910548-46a7-4f62-8c4c-d9a8b44ba2d0 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:37:57.014125 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0d910548-46a7-4f62-8c4c-d9a8b44ba2d0 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:37:57.019805 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0d910548-46a7-4f62-8c4c-d9a8b44ba2d0 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:37:57.521458 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-0d910548-46a7-4f62-8c4c-d9a8b44ba2d0 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0ab8b3f06e994bbca5d6dd14109bfe50', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e60c0863dc6f4313bdc873fda27ae2fc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:37:57.533971 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-0d910548-46a7-4f62-8c4c-d9a8b44ba2d0 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] 10.222.0.16 "GET /compute/v2.1/servers/a7f9caa9-fff2-44b2-871d-7e4191ff39e5" status: 200 len: 1369 microversion: 2.1 time: 0.534158 Jul 16 21:37:57.534675 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2614/5220] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:37:56 2022] GET /compute/v2.1/servers/a7f9caa9-fff2-44b2-871d-7e4191ff39e5 => generated 1369 bytes in 536 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:37:57.616515 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-e1688d0d-93bf-4a1f-93bd-519ef1b60331 tempest-TestAggregatesBasicOps-362841127 tempest-TestAggregatesBasicOps-362841127-project-admin] 10.222.0.16 "POST /compute/v2.1/os-aggregates/19/action" status: 200 len: 341 microversion: 2.1 time: 0.678147 Jul 16 21:37:57.625635 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2607/5221] 10.222.0.16 () {60 vars in 1193 bytes} [Sat Jul 16 21:37:56 2022] POST /compute/v2.1/os-aggregates/19/action => generated 341 bytes in 688 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:37:57.628925 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-4414ca96-c74c-46c8-915b-45723f9f56c8 tempest-TestAggregatesBasicOps-362841127 tempest-TestAggregatesBasicOps-362841127-project-admin] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:37:57.629668 ncdv-master-1 devstack@n-api.service[138681]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-aggregates:delete": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:37:57.629668 ncdv-master-1 devstack@n-api.service[138681]: warnings.warn(msg) Jul 16 21:37:57.756870 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-4414ca96-c74c-46c8-915b-45723f9f56c8 tempest-TestAggregatesBasicOps-362841127 tempest-TestAggregatesBasicOps-362841127-project-admin] 10.222.0.16 "DELETE /compute/v2.1/os-aggregates/19" status: 200 len: 0 microversion: 2.1 time: 0.131772 Jul 16 21:37:57.759737 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2615/5222] 10.222.0.16 () {58 vars in 1154 bytes} [Sat Jul 16 21:37:57 2022] DELETE /compute/v2.1/os-aggregates/19 => generated 0 bytes in 135 msecs (HTTP/1.1 200) 9 headers in 355 bytes (1 switches on core 0) Jul 16 21:37:58.553618 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-ae47046e-4ed7-4f19-99c6-38836ec324fc tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:37:58.554607 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-ae47046e-4ed7-4f19-99c6-38836ec324fc tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] [instance: a7f9caa9-fff2-44b2-871d-7e4191ff39e5] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:37:58.573175 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ae47046e-4ed7-4f19-99c6-38836ec324fc tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:37:58.577273 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ae47046e-4ed7-4f19-99c6-38836ec324fc tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:37:58.578443 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ae47046e-4ed7-4f19-99c6-38836ec324fc tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:37:58.579111 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ae47046e-4ed7-4f19-99c6-38836ec324fc tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:37:59.046395 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-ae47046e-4ed7-4f19-99c6-38836ec324fc tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0ab8b3f06e994bbca5d6dd14109bfe50', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e60c0863dc6f4313bdc873fda27ae2fc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:37:59.059302 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-ae47046e-4ed7-4f19-99c6-38836ec324fc tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] 10.222.0.16 "GET /compute/v2.1/servers/a7f9caa9-fff2-44b2-871d-7e4191ff39e5" status: 200 len: 1365 microversion: 2.1 time: 0.509843 Jul 16 21:37:59.068936 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2608/5223] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:37:58 2022] GET /compute/v2.1/servers/a7f9caa9-fff2-44b2-871d-7e4191ff39e5 => generated 1365 bytes in 514 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:38:00.076004 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell0:SELECT=8 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:38:00.080563 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-1116febc-aeb0-4f20-a74b-d12ec7d35dd5 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:38:00.081191 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-1116febc-aeb0-4f20-a74b-d12ec7d35dd5 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] [instance: a7f9caa9-fff2-44b2-871d-7e4191ff39e5] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:38:00.101266 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1116febc-aeb0-4f20-a74b-d12ec7d35dd5 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:38:00.101266 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1116febc-aeb0-4f20-a74b-d12ec7d35dd5 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:38:00.101266 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1116febc-aeb0-4f20-a74b-d12ec7d35dd5 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:38:00.112791 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1116febc-aeb0-4f20-a74b-d12ec7d35dd5 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:38:00.507166 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-1116febc-aeb0-4f20-a74b-d12ec7d35dd5 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0ab8b3f06e994bbca5d6dd14109bfe50', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e60c0863dc6f4313bdc873fda27ae2fc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:38:00.523699 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-1116febc-aeb0-4f20-a74b-d12ec7d35dd5 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] 10.222.0.16 "GET /compute/v2.1/servers/a7f9caa9-fff2-44b2-871d-7e4191ff39e5" status: 200 len: 1365 microversion: 2.1 time: 0.449889 Jul 16 21:38:00.525178 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2616/5224] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:38:00 2022] GET /compute/v2.1/servers/a7f9caa9-fff2-44b2-871d-7e4191ff39e5 => generated 1365 bytes in 452 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:38:01.542605 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-46c1cc1d-90ce-423a-b44f-9a6c9d209dfe tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:38:01.543394 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-46c1cc1d-90ce-423a-b44f-9a6c9d209dfe tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] [instance: a7f9caa9-fff2-44b2-871d-7e4191ff39e5] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:38:01.546529 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell0:SELECT=8 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:38:01.551892 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-46c1cc1d-90ce-423a-b44f-9a6c9d209dfe tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:38:01.552414 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-46c1cc1d-90ce-423a-b44f-9a6c9d209dfe tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:38:01.553258 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-46c1cc1d-90ce-423a-b44f-9a6c9d209dfe tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:38:01.553774 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-46c1cc1d-90ce-423a-b44f-9a6c9d209dfe tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:38:01.923529 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-46c1cc1d-90ce-423a-b44f-9a6c9d209dfe tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0ab8b3f06e994bbca5d6dd14109bfe50', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e60c0863dc6f4313bdc873fda27ae2fc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:38:01.931229 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-46c1cc1d-90ce-423a-b44f-9a6c9d209dfe tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] 10.222.0.16 "GET /compute/v2.1/servers/a7f9caa9-fff2-44b2-871d-7e4191ff39e5" status: 200 len: 1365 microversion: 2.1 time: 0.394037 Jul 16 21:38:01.932029 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2609/5225] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:38:01 2022] GET /compute/v2.1/servers/a7f9caa9-fff2-44b2-871d-7e4191ff39e5 => generated 1365 bytes in 396 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:38:02.949767 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-ade0cdae-4270-4284-845c-66876e19d29b tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:38:02.950507 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-ade0cdae-4270-4284-845c-66876e19d29b tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] [instance: a7f9caa9-fff2-44b2-871d-7e4191ff39e5] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:38:02.958033 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ade0cdae-4270-4284-845c-66876e19d29b tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:38:02.958481 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ade0cdae-4270-4284-845c-66876e19d29b tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:38:02.959381 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ade0cdae-4270-4284-845c-66876e19d29b tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:38:02.959817 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ade0cdae-4270-4284-845c-66876e19d29b tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:38:03.214265 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-ade0cdae-4270-4284-845c-66876e19d29b tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0ab8b3f06e994bbca5d6dd14109bfe50', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e60c0863dc6f4313bdc873fda27ae2fc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:38:03.224231 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-ade0cdae-4270-4284-845c-66876e19d29b tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] 10.222.0.16 "GET /compute/v2.1/servers/a7f9caa9-fff2-44b2-871d-7e4191ff39e5" status: 200 len: 1365 microversion: 2.1 time: 0.279154 Jul 16 21:38:03.227566 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2617/5226] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:38:02 2022] GET /compute/v2.1/servers/a7f9caa9-fff2-44b2-871d-7e4191ff39e5 => generated 1365 bytes in 281 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:38:04.243275 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-8fce91f3-b3f9-4793-bcca-9514f2cf62fd tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:38:04.244729 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-8fce91f3-b3f9-4793-bcca-9514f2cf62fd tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] [instance: a7f9caa9-fff2-44b2-871d-7e4191ff39e5] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:38:04.267150 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8fce91f3-b3f9-4793-bcca-9514f2cf62fd tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:38:04.269701 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8fce91f3-b3f9-4793-bcca-9514f2cf62fd tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:38:04.269701 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8fce91f3-b3f9-4793-bcca-9514f2cf62fd tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:38:04.269701 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8fce91f3-b3f9-4793-bcca-9514f2cf62fd tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:38:04.488661 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-0a8bab13-58c8-4534-9892-4e1135e0d6aa req-aabeef38-2016-43d4-bfcf-6ab4aa3eb1f7 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "a7f9caa9-fff2-44b2-871d-7e4191ff39e5", "name": "network-vif-plugged", "status": "completed", "tag": "8e33e390-df49-4cb3-a043-fa2763fb499b"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:38:04.501823 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-0a8bab13-58c8-4534-9892-4e1135e0d6aa req-aabeef38-2016-43d4-bfcf-6ab4aa3eb1f7 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:38:04.502487 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-0a8bab13-58c8-4534-9892-4e1135e0d6aa req-aabeef38-2016-43d4-bfcf-6ab4aa3eb1f7 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:38:04.522286 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-0a8bab13-58c8-4534-9892-4e1135e0d6aa req-aabeef38-2016-43d4-bfcf-6ab4aa3eb1f7 service nova] Creating event network-vif-plugged:8e33e390-df49-4cb3-a043-fa2763fb499b for instance a7f9caa9-fff2-44b2-871d-7e4191ff39e5 on nch1-master-1 Jul 16 21:38:04.531371 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-0a8bab13-58c8-4534-9892-4e1135e0d6aa req-aabeef38-2016-43d4-bfcf-6ab4aa3eb1f7 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.048081 Jul 16 21:38:04.533737 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2618/5227] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:38:04 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:38:04.765551 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-8fce91f3-b3f9-4793-bcca-9514f2cf62fd tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0ab8b3f06e994bbca5d6dd14109bfe50', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e60c0863dc6f4313bdc873fda27ae2fc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:38:04.776633 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-8fce91f3-b3f9-4793-bcca-9514f2cf62fd tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] 10.222.0.16 "GET /compute/v2.1/servers/a7f9caa9-fff2-44b2-871d-7e4191ff39e5" status: 200 len: 1365 microversion: 2.1 time: 0.538489 Jul 16 21:38:04.787646 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2610/5228] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:38:04 2022] GET /compute/v2.1/servers/a7f9caa9-fff2-44b2-871d-7e4191ff39e5 => generated 1365 bytes in 549 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:38:05.797538 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-a5d532dd-0575-4466-97a3-6455d36eebab tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:38:05.799347 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-a5d532dd-0575-4466-97a3-6455d36eebab tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] [instance: a7f9caa9-fff2-44b2-871d-7e4191ff39e5] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:38:05.831958 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a5d532dd-0575-4466-97a3-6455d36eebab tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:38:05.833101 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a5d532dd-0575-4466-97a3-6455d36eebab tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.023s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:38:05.834610 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a5d532dd-0575-4466-97a3-6455d36eebab tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:38:05.835808 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a5d532dd-0575-4466-97a3-6455d36eebab tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:38:05.930856 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell0:SELECT=4 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:38:06.365148 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-a5d532dd-0575-4466-97a3-6455d36eebab tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0ab8b3f06e994bbca5d6dd14109bfe50', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e60c0863dc6f4313bdc873fda27ae2fc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:38:06.366688 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-a5d532dd-0575-4466-97a3-6455d36eebab tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] 10.222.0.16 "GET /compute/v2.1/servers/a7f9caa9-fff2-44b2-871d-7e4191ff39e5" status: 200 len: 1365 microversion: 2.1 time: 0.574672 Jul 16 21:38:06.372150 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2619/5229] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:38:05 2022] GET /compute/v2.1/servers/a7f9caa9-fff2-44b2-871d-7e4191ff39e5 => generated 1365 bytes in 581 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:38:07.384348 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-f7fdc632-7ba8-416b-8553-70f86b2d69f5 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:38:07.385359 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-f7fdc632-7ba8-416b-8553-70f86b2d69f5 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] [instance: a7f9caa9-fff2-44b2-871d-7e4191ff39e5] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:38:07.392645 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f7fdc632-7ba8-416b-8553-70f86b2d69f5 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:38:07.393349 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f7fdc632-7ba8-416b-8553-70f86b2d69f5 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:38:07.394129 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f7fdc632-7ba8-416b-8553-70f86b2d69f5 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:38:07.394760 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f7fdc632-7ba8-416b-8553-70f86b2d69f5 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:38:07.892904 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-f7fdc632-7ba8-416b-8553-70f86b2d69f5 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0ab8b3f06e994bbca5d6dd14109bfe50', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e60c0863dc6f4313bdc873fda27ae2fc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:38:07.904342 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-f7fdc632-7ba8-416b-8553-70f86b2d69f5 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] 10.222.0.16 "GET /compute/v2.1/servers/a7f9caa9-fff2-44b2-871d-7e4191ff39e5" status: 200 len: 1555 microversion: 2.1 time: 0.524539 Jul 16 21:38:07.921676 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2611/5230] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:38:07 2022] GET /compute/v2.1/servers/a7f9caa9-fff2-44b2-871d-7e4191ff39e5 => generated 1555 bytes in 542 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:38:07.924551 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-017a2c90-fd40-4897-94f8-b917c7365253 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:38:07.925064 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-017a2c90-fd40-4897-94f8-b917c7365253 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] [instance: a7f9caa9-fff2-44b2-871d-7e4191ff39e5] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:38:07.944195 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-017a2c90-fd40-4897-94f8-b917c7365253 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:38:07.944691 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-017a2c90-fd40-4897-94f8-b917c7365253 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:38:07.945505 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-017a2c90-fd40-4897-94f8-b917c7365253 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:38:07.953652 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-017a2c90-fd40-4897-94f8-b917c7365253 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:38:08.162093 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-017a2c90-fd40-4897-94f8-b917c7365253 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] 10.222.0.16 "GET /compute/v2.1/servers/a7f9caa9-fff2-44b2-871d-7e4191ff39e5/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.243679 Jul 16 21:38:08.162698 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2620/5231] 10.222.0.16 () {58 vars in 1274 bytes} [Sat Jul 16 21:38:07 2022] GET /compute/v2.1/servers/a7f9caa9-fff2-44b2-871d-7e4191ff39e5/os-interface => generated 284 bytes in 245 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 16 21:38:10.741148 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-d36bbbb6-9955-4303-b40b-3721af9101cc req-ca6615e4-b939-4d38-9e00-2a70fc61a1c8 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "a7f9caa9-fff2-44b2-871d-7e4191ff39e5", "tag": "8e33e390-df49-4cb3-a043-fa2763fb499b"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:38:10.759965 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-d36bbbb6-9955-4303-b40b-3721af9101cc req-ca6615e4-b939-4d38-9e00-2a70fc61a1c8 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:38:10.760424 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-d36bbbb6-9955-4303-b40b-3721af9101cc req-ca6615e4-b939-4d38-9e00-2a70fc61a1c8 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:38:10.790826 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-d36bbbb6-9955-4303-b40b-3721af9101cc req-ca6615e4-b939-4d38-9e00-2a70fc61a1c8 service nova] Creating event network-changed:8e33e390-df49-4cb3-a043-fa2763fb499b for instance a7f9caa9-fff2-44b2-871d-7e4191ff39e5 on nch1-master-1 Jul 16 21:38:10.807317 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-d36bbbb6-9955-4303-b40b-3721af9101cc req-ca6615e4-b939-4d38-9e00-2a70fc61a1c8 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.067891 Jul 16 21:38:10.807993 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2612/5232] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:38:10 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 72 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:38:14.162140 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-e5de2c65-6892-467d-8fb7-2697999f4264 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesSnapshotTestJSON-instance-299925536", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452", "networks": [{"uuid": "fde2210a-0286-4643-88e0-b5abc1bd975b"}]}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:38:14.296555 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.network.neutron [None req-e5de2c65-6892-467d-8fb7-2697999f4264 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] validate_networks() for [('fde2210a-0286-4643-88e0-b5abc1bd975b', None, None, None, None, None)] {{(pid=138681) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 16 21:38:15.991686 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-e5de2c65-6892-467d-8fb7-2697999f4264 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Flavor limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 16 21:38:15.993086 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-e5de2c65-6892-467d-8fb7-2697999f4264 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Image limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 16 21:38:15.993783 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-e5de2c65-6892-467d-8fb7-2697999f4264 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Flavor pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 16 21:38:15.994246 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-e5de2c65-6892-467d-8fb7-2697999f4264 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Image pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 16 21:38:15.994667 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-e5de2c65-6892-467d-8fb7-2697999f4264 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 16 21:38:16.021692 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e5de2c65-6892-467d-8fb7-2697999f4264 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:38:16.022169 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e5de2c65-6892-467d-8fb7-2697999f4264 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:38:16.023040 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e5de2c65-6892-467d-8fb7-2697999f4264 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:38:16.027764 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e5de2c65-6892-467d-8fb7-2697999f4264 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:38:16.068376 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-e5de2c65-6892-467d-8fb7-2697999f4264 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Getting quotas for project 56b8099d9406431284b4dd58b5c16359. Resources: {'ram', 'instances', 'cores'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:38:16.076419 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-e5de2c65-6892-467d-8fb7-2697999f4264 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Getting quotas for user 95cd6b54d8164de1a524b407a1852ebd and project 56b8099d9406431284b4dd58b5c16359. Resources: {'ram', 'instances', 'cores'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:38:16.091292 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-e5de2c65-6892-467d-8fb7-2697999f4264 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Going to run 1 instances... {{(pid=138681) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 16 21:38:16.099395 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-e5de2c65-6892-467d-8fb7-2697999f4264 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] [instance: 21fc83e5-8f03-4930-93b1-0196f7327b3e] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138681) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 16 21:38:16.302188 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-e5de2c65-6892-467d-8fb7-2697999f4264 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 2.146671 Jul 16 21:38:16.316568 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2621/5233] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 21:38:14 2022] POST /compute/v2.1/servers => generated 380 bytes in 2160 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 16 21:38:16.350014 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-0cc47de1-1cd5-4ef6-bb4a-f894c1370e8b tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:38:16.355301 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-0cc47de1-1cd5-4ef6-bb4a-f894c1370e8b tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] [instance: 21fc83e5-8f03-4930-93b1-0196f7327b3e] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:38:16.970561 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-0cc47de1-1cd5-4ef6-bb4a-f894c1370e8b tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '95cd6b54d8164de1a524b407a1852ebd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '56b8099d9406431284b4dd58b5c16359', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:38:16.978865 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-0cc47de1-1cd5-4ef6-bb4a-f894c1370e8b tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] 10.222.0.16 "GET /compute/v2.1/servers/21fc83e5-8f03-4930-93b1-0196f7327b3e" status: 200 len: 1218 microversion: 2.1 time: 0.652423 Jul 16 21:38:16.988545 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2613/5234] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:38:16 2022] GET /compute/v2.1/servers/21fc83e5-8f03-4930-93b1-0196f7327b3e => generated 1218 bytes in 660 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:38:18.001191 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-9e6c28c9-1e89-4aa9-a90d-d006fccb01e3 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:38:18.008752 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-9e6c28c9-1e89-4aa9-a90d-d006fccb01e3 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] [instance: 21fc83e5-8f03-4930-93b1-0196f7327b3e] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:38:18.050252 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9e6c28c9-1e89-4aa9-a90d-d006fccb01e3 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:38:18.050719 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9e6c28c9-1e89-4aa9-a90d-d006fccb01e3 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:38:18.051553 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9e6c28c9-1e89-4aa9-a90d-d006fccb01e3 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:38:18.058036 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9e6c28c9-1e89-4aa9-a90d-d006fccb01e3 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:38:18.566779 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-9e6c28c9-1e89-4aa9-a90d-d006fccb01e3 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '95cd6b54d8164de1a524b407a1852ebd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '56b8099d9406431284b4dd58b5c16359', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:38:18.577339 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-9e6c28c9-1e89-4aa9-a90d-d006fccb01e3 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] 10.222.0.16 "GET /compute/v2.1/servers/21fc83e5-8f03-4930-93b1-0196f7327b3e" status: 200 len: 1222 microversion: 2.1 time: 0.585563 Jul 16 21:38:18.578348 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2622/5235] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:38:17 2022] GET /compute/v2.1/servers/21fc83e5-8f03-4930-93b1-0196f7327b3e => generated 1222 bytes in 587 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:38:18.843672 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-5204efeb-e564-44fb-bd54-80828e4ce6fd tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Action: 'create', calling method: .version_select at 0x7fdd8fcf9f70>, body: {"keypair": {"name": "tempest-TestNetworkAdvancedServerOps-1632011611", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBPt6uMCM5H5BOARXqbVg/ZNNBIgoNVnak4bfUzEJznmiNtBUwHY2CmKzONFKl/4klF9XLZRJFml8ibVwZKoWmS2PRty1fCPhh/s+o2xPylEy/wOOGYLCBe1sytypA+LK4g=="}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:38:18.858890 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-5204efeb-e564-44fb-bd54-80828e4ce6fd tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Getting quotas for project 77c9b7e351e24dc88207e9498f75f578. Resources: {'key_pairs'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:38:18.864780 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-5204efeb-e564-44fb-bd54-80828e4ce6fd tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Getting quotas for user 833136cbdeab4b9e8b6dd2d24c7c8753 and project 77c9b7e351e24dc88207e9498f75f578. Resources: {'key_pairs'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:38:18.986287 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-5204efeb-e564-44fb-bd54-80828e4ce6fd tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] 10.222.0.16 "POST /compute/v2.1/os-keypairs" status: 200 len: 407 microversion: 2.1 time: 0.337245 Jul 16 21:38:18.996197 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2614/5236] 10.222.0.16 () {60 vars in 1158 bytes} [Sat Jul 16 21:38:18 2022] POST /compute/v2.1/os-keypairs => generated 407 bytes in 342 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:38:19.598090 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-a35c342f-52b9-4aad-9692-2e580a6da6e7 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:38:19.598860 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-a35c342f-52b9-4aad-9692-2e580a6da6e7 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] [instance: 21fc83e5-8f03-4930-93b1-0196f7327b3e] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:38:19.609852 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a35c342f-52b9-4aad-9692-2e580a6da6e7 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:38:19.610373 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a35c342f-52b9-4aad-9692-2e580a6da6e7 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:38:19.611312 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a35c342f-52b9-4aad-9692-2e580a6da6e7 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:38:19.611973 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a35c342f-52b9-4aad-9692-2e580a6da6e7 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:38:19.709237 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-a35c342f-52b9-4aad-9692-2e580a6da6e7 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '95cd6b54d8164de1a524b407a1852ebd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '56b8099d9406431284b4dd58b5c16359', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:38:19.720503 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-a35c342f-52b9-4aad-9692-2e580a6da6e7 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] 10.222.0.16 "GET /compute/v2.1/servers/21fc83e5-8f03-4930-93b1-0196f7327b3e" status: 200 len: 1276 microversion: 2.1 time: 0.127866 Jul 16 21:38:19.721674 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2623/5237] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:38:19 2022] GET /compute/v2.1/servers/21fc83e5-8f03-4930-93b1-0196f7327b3e => generated 1276 bytes in 130 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:38:20.738342 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-e40dfca3-c02c-462f-a5ef-ddd8be0d9135 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:38:20.739063 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-e40dfca3-c02c-462f-a5ef-ddd8be0d9135 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] [instance: 21fc83e5-8f03-4930-93b1-0196f7327b3e] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:38:20.747308 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e40dfca3-c02c-462f-a5ef-ddd8be0d9135 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:38:20.751060 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e40dfca3-c02c-462f-a5ef-ddd8be0d9135 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:38:20.751060 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e40dfca3-c02c-462f-a5ef-ddd8be0d9135 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:38:20.751060 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e40dfca3-c02c-462f-a5ef-ddd8be0d9135 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:38:21.063379 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-e40dfca3-c02c-462f-a5ef-ddd8be0d9135 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '95cd6b54d8164de1a524b407a1852ebd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '56b8099d9406431284b4dd58b5c16359', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:38:21.081195 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-e40dfca3-c02c-462f-a5ef-ddd8be0d9135 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] 10.222.0.16 "GET /compute/v2.1/servers/21fc83e5-8f03-4930-93b1-0196f7327b3e" status: 200 len: 1318 microversion: 2.1 time: 0.346865 Jul 16 21:38:21.081798 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2615/5238] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:38:20 2022] GET /compute/v2.1/servers/21fc83e5-8f03-4930-93b1-0196f7327b3e => generated 1318 bytes in 348 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:38:21.597361 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-b63e37be-53a1-447c-93f0-c97819394827 req-71cecdd3-945c-4a92-aa8f-9d1fd9223c23 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "21fc83e5-8f03-4930-93b1-0196f7327b3e", "tag": "617271fa-0e6f-44d7-82b2-3da8cab88742"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:38:21.614079 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-b63e37be-53a1-447c-93f0-c97819394827 req-71cecdd3-945c-4a92-aa8f-9d1fd9223c23 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:38:21.614723 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-b63e37be-53a1-447c-93f0-c97819394827 req-71cecdd3-945c-4a92-aa8f-9d1fd9223c23 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:38:21.626841 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-b63e37be-53a1-447c-93f0-c97819394827 req-71cecdd3-945c-4a92-aa8f-9d1fd9223c23 service nova] Creating event network-changed:617271fa-0e6f-44d7-82b2-3da8cab88742 for instance 21fc83e5-8f03-4930-93b1-0196f7327b3e on nch1-master-1 Jul 16 21:38:21.632111 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-b63e37be-53a1-447c-93f0-c97819394827 req-71cecdd3-945c-4a92-aa8f-9d1fd9223c23 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.043329 Jul 16 21:38:21.633316 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2624/5239] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:38:21 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:38:22.102315 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-4b1e2d06-cc81-4536-b622-93198e017d26 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:38:22.103225 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-4b1e2d06-cc81-4536-b622-93198e017d26 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] [instance: 21fc83e5-8f03-4930-93b1-0196f7327b3e] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:38:22.111082 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4b1e2d06-cc81-4536-b622-93198e017d26 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:38:22.111542 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4b1e2d06-cc81-4536-b622-93198e017d26 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:38:22.112323 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4b1e2d06-cc81-4536-b622-93198e017d26 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:38:22.112764 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4b1e2d06-cc81-4536-b622-93198e017d26 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:38:22.446271 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-4b1e2d06-cc81-4536-b622-93198e017d26 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '95cd6b54d8164de1a524b407a1852ebd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '56b8099d9406431284b4dd58b5c16359', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:38:22.454717 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-4b1e2d06-cc81-4536-b622-93198e017d26 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] 10.222.0.16 "GET /compute/v2.1/servers/21fc83e5-8f03-4930-93b1-0196f7327b3e" status: 200 len: 1318 microversion: 2.1 time: 0.356465 Jul 16 21:38:22.455544 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2616/5240] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:38:22 2022] GET /compute/v2.1/servers/21fc83e5-8f03-4930-93b1-0196f7327b3e => generated 1318 bytes in 358 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:38:23.473717 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-336c1565-4c39-4e8b-866d-06a680dc82b6 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:38:23.474458 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-336c1565-4c39-4e8b-866d-06a680dc82b6 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] [instance: 21fc83e5-8f03-4930-93b1-0196f7327b3e] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:38:23.482167 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-336c1565-4c39-4e8b-866d-06a680dc82b6 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:38:23.482882 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-336c1565-4c39-4e8b-866d-06a680dc82b6 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:38:23.483861 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-336c1565-4c39-4e8b-866d-06a680dc82b6 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:38:23.484469 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-336c1565-4c39-4e8b-866d-06a680dc82b6 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:38:23.660854 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-336c1565-4c39-4e8b-866d-06a680dc82b6 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '95cd6b54d8164de1a524b407a1852ebd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '56b8099d9406431284b4dd58b5c16359', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:38:23.673001 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-336c1565-4c39-4e8b-866d-06a680dc82b6 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] 10.222.0.16 "GET /compute/v2.1/servers/21fc83e5-8f03-4930-93b1-0196f7327b3e" status: 200 len: 1318 microversion: 2.1 time: 0.204202 Jul 16 21:38:23.674293 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2625/5241] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:38:23 2022] GET /compute/v2.1/servers/21fc83e5-8f03-4930-93b1-0196f7327b3e => generated 1318 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:38:24.690468 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-b9c3c06e-9a6a-448d-8911-ad4264353515 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:38:24.691146 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-b9c3c06e-9a6a-448d-8911-ad4264353515 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] [instance: 21fc83e5-8f03-4930-93b1-0196f7327b3e] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:38:24.709860 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b9c3c06e-9a6a-448d-8911-ad4264353515 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:38:24.710375 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b9c3c06e-9a6a-448d-8911-ad4264353515 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:38:24.711274 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b9c3c06e-9a6a-448d-8911-ad4264353515 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:38:24.712170 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b9c3c06e-9a6a-448d-8911-ad4264353515 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:38:25.055195 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-b9c3c06e-9a6a-448d-8911-ad4264353515 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '95cd6b54d8164de1a524b407a1852ebd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '56b8099d9406431284b4dd58b5c16359', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:38:25.069912 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-b9c3c06e-9a6a-448d-8911-ad4264353515 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] 10.222.0.16 "GET /compute/v2.1/servers/21fc83e5-8f03-4930-93b1-0196f7327b3e" status: 200 len: 1318 microversion: 2.1 time: 0.382718 Jul 16 21:38:25.072257 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2617/5242] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:38:24 2022] GET /compute/v2.1/servers/21fc83e5-8f03-4930-93b1-0196f7327b3e => generated 1318 bytes in 386 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:38:26.090081 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-3f06e8b1-496a-4f87-9689-7fa206cb8cdf tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:38:26.090081 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-3f06e8b1-496a-4f87-9689-7fa206cb8cdf tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] [instance: 21fc83e5-8f03-4930-93b1-0196f7327b3e] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:38:26.102747 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell0:SELECT=2 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:38:26.103602 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell0:SELECT=6 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:38:26.105244 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3f06e8b1-496a-4f87-9689-7fa206cb8cdf tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:38:26.105668 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3f06e8b1-496a-4f87-9689-7fa206cb8cdf tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:38:26.106711 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3f06e8b1-496a-4f87-9689-7fa206cb8cdf tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:38:26.112145 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3f06e8b1-496a-4f87-9689-7fa206cb8cdf tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:38:26.502976 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-3f06e8b1-496a-4f87-9689-7fa206cb8cdf tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '95cd6b54d8164de1a524b407a1852ebd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '56b8099d9406431284b4dd58b5c16359', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:38:26.511264 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=71 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:38:26.515063 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-3f06e8b1-496a-4f87-9689-7fa206cb8cdf tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] 10.222.0.16 "GET /compute/v2.1/servers/21fc83e5-8f03-4930-93b1-0196f7327b3e" status: 200 len: 1318 microversion: 2.1 time: 0.430036 Jul 16 21:38:26.515063 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2626/5243] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:38:26 2022] GET /compute/v2.1/servers/21fc83e5-8f03-4930-93b1-0196f7327b3e => generated 1318 bytes in 431 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:38:27.532754 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-bc0d748b-48e8-43c2-bcc9-1e757fe5872d tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:38:27.535431 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-bc0d748b-48e8-43c2-bcc9-1e757fe5872d tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] [instance: 21fc83e5-8f03-4930-93b1-0196f7327b3e] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:38:27.546506 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-bc0d748b-48e8-43c2-bcc9-1e757fe5872d tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:38:27.547249 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-bc0d748b-48e8-43c2-bcc9-1e757fe5872d tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:38:27.548294 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-bc0d748b-48e8-43c2-bcc9-1e757fe5872d tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:38:27.548854 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-bc0d748b-48e8-43c2-bcc9-1e757fe5872d tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:38:27.572541 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=63 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:38:27.847779 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-bc0d748b-48e8-43c2-bcc9-1e757fe5872d tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '95cd6b54d8164de1a524b407a1852ebd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '56b8099d9406431284b4dd58b5c16359', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:38:27.859039 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-bc0d748b-48e8-43c2-bcc9-1e757fe5872d tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] 10.222.0.16 "GET /compute/v2.1/servers/21fc83e5-8f03-4930-93b1-0196f7327b3e" status: 200 len: 1318 microversion: 2.1 time: 0.328783 Jul 16 21:38:27.870794 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2618/5244] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:38:27 2022] GET /compute/v2.1/servers/21fc83e5-8f03-4930-93b1-0196f7327b3e => generated 1318 bytes in 343 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:38:28.650606 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-6cefbd88-a317-4bf1-8292-8d97e28d8cfd req-8b8a075b-e5b2-42e1-9471-80e6bcd974b0 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "21fc83e5-8f03-4930-93b1-0196f7327b3e", "name": "network-vif-plugged", "status": "completed", "tag": "617271fa-0e6f-44d7-82b2-3da8cab88742"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:38:28.676065 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_api:SELECT=73,nova_api:INSERT=5,nova_api:UPDATE=1,nova_api:DELETE=2 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:38:28.678046 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-6cefbd88-a317-4bf1-8292-8d97e28d8cfd req-8b8a075b-e5b2-42e1-9471-80e6bcd974b0 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:38:28.678967 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-6cefbd88-a317-4bf1-8292-8d97e28d8cfd req-8b8a075b-e5b2-42e1-9471-80e6bcd974b0 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:38:28.707753 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-6cefbd88-a317-4bf1-8292-8d97e28d8cfd req-8b8a075b-e5b2-42e1-9471-80e6bcd974b0 service nova] Creating event network-vif-plugged:617271fa-0e6f-44d7-82b2-3da8cab88742 for instance 21fc83e5-8f03-4930-93b1-0196f7327b3e on nch1-master-1 Jul 16 21:38:28.715514 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-6cefbd88-a317-4bf1-8292-8d97e28d8cfd req-8b8a075b-e5b2-42e1-9471-80e6bcd974b0 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.070445 Jul 16 21:38:28.720663 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2627/5245] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:38:28 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 76 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:38:28.887824 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-eb9c48e3-103a-4fa4-9f6c-789f122cf122 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:38:28.889292 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-eb9c48e3-103a-4fa4-9f6c-789f122cf122 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] [instance: 21fc83e5-8f03-4930-93b1-0196f7327b3e] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:38:28.892493 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell0:SELECT=6 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:38:28.894343 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_api:SELECT=57,nova_api:INSERT=4,nova_api:DELETE=1 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:38:28.911615 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-eb9c48e3-103a-4fa4-9f6c-789f122cf122 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:38:28.916330 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-eb9c48e3-103a-4fa4-9f6c-789f122cf122 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:38:28.919751 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-eb9c48e3-103a-4fa4-9f6c-789f122cf122 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:38:28.921010 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-eb9c48e3-103a-4fa4-9f6c-789f122cf122 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:38:29.218889 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-eb9c48e3-103a-4fa4-9f6c-789f122cf122 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '95cd6b54d8164de1a524b407a1852ebd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '56b8099d9406431284b4dd58b5c16359', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:38:29.233916 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-eb9c48e3-103a-4fa4-9f6c-789f122cf122 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] 10.222.0.16 "GET /compute/v2.1/servers/21fc83e5-8f03-4930-93b1-0196f7327b3e" status: 200 len: 1318 microversion: 2.1 time: 0.351943 Jul 16 21:38:29.234522 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2619/5246] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:38:28 2022] GET /compute/v2.1/servers/21fc83e5-8f03-4930-93b1-0196f7327b3e => generated 1318 bytes in 353 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:38:30.256073 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-cd8fc909-d749-44dd-8c42-36a7f26824c1 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:38:30.256929 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-cd8fc909-d749-44dd-8c42-36a7f26824c1 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] [instance: 21fc83e5-8f03-4930-93b1-0196f7327b3e] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:38:30.286995 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cd8fc909-d749-44dd-8c42-36a7f26824c1 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:38:30.289163 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cd8fc909-d749-44dd-8c42-36a7f26824c1 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.014s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:38:30.292966 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cd8fc909-d749-44dd-8c42-36a7f26824c1 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:38:30.293614 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cd8fc909-d749-44dd-8c42-36a7f26824c1 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:38:30.708890 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-cd8fc909-d749-44dd-8c42-36a7f26824c1 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '95cd6b54d8164de1a524b407a1852ebd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '56b8099d9406431284b4dd58b5c16359', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:38:30.721347 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-cd8fc909-d749-44dd-8c42-36a7f26824c1 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] 10.222.0.16 "GET /compute/v2.1/servers/21fc83e5-8f03-4930-93b1-0196f7327b3e" status: 200 len: 1506 microversion: 2.1 time: 0.473136 Jul 16 21:38:30.724115 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2628/5247] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:38:30 2022] GET /compute/v2.1/servers/21fc83e5-8f03-4930-93b1-0196f7327b3e => generated 1506 bytes in 477 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:38:34.129775 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-6eab712d-f9c4-4f26-ab6c-2432edafb362 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestNetworkAdvancedServerOps-server-760060315", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452", "networks": [{"uuid": "daad08e7-7b34-4483-93b0-960e066f3ae0"}], "key_name": "tempest-TestNetworkAdvancedServerOps-1632011611", "security_groups": [{"name": "tempest-secgroup-smoke-808229863"}]}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:38:34.367883 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.network.neutron [None req-6eab712d-f9c4-4f26-ab6c-2432edafb362 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] validate_networks() for [('daad08e7-7b34-4483-93b0-960e066f3ae0', None, None, None, None, None)] {{(pid=138680) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 16 21:38:35.278980 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-6eab712d-f9c4-4f26-ab6c-2432edafb362 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Flavor limits 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 16 21:38:35.279777 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-6eab712d-f9c4-4f26-ab6c-2432edafb362 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Image limits 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 16 21:38:35.280085 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-6eab712d-f9c4-4f26-ab6c-2432edafb362 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Flavor pref 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 16 21:38:35.281255 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-6eab712d-f9c4-4f26-ab6c-2432edafb362 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Image pref 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 16 21:38:35.281737 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-6eab712d-f9c4-4f26-ab6c-2432edafb362 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 16 21:38:35.303026 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6eab712d-f9c4-4f26-ab6c-2432edafb362 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:38:35.307639 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6eab712d-f9c4-4f26-ab6c-2432edafb362 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:38:35.311846 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6eab712d-f9c4-4f26-ab6c-2432edafb362 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:38:35.313905 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6eab712d-f9c4-4f26-ab6c-2432edafb362 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:38:35.374048 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-6eab712d-f9c4-4f26-ab6c-2432edafb362 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Getting quotas for project 77c9b7e351e24dc88207e9498f75f578. Resources: {'ram', 'instances', 'cores'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:38:35.387626 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-6eab712d-f9c4-4f26-ab6c-2432edafb362 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Getting quotas for user 833136cbdeab4b9e8b6dd2d24c7c8753 and project 77c9b7e351e24dc88207e9498f75f578. Resources: {'ram', 'instances', 'cores'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:38:35.421590 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-6eab712d-f9c4-4f26-ab6c-2432edafb362 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Going to run 1 instances... {{(pid=138680) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 16 21:38:35.424136 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-6eab712d-f9c4-4f26-ab6c-2432edafb362 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] [instance: 2ce56b72-9fe8-41ca-bd6e-31591ef3b494] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138680) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 16 21:38:35.668796 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-6eab712d-f9c4-4f26-ab6c-2432edafb362 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 405 microversion: 2.1 time: 1.542912 Jul 16 21:38:35.669734 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2620/5248] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 21:38:34 2022] POST /compute/v2.1/servers => generated 405 bytes in 1545 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 16 21:38:35.681569 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-233d20d0-6111-41b3-9171-58a2460a521f tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:38:35.682342 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-233d20d0-6111-41b3-9171-58a2460a521f tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] [instance: 2ce56b72-9fe8-41ca-bd6e-31591ef3b494] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:38:35.871207 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-233d20d0-6111-41b3-9171-58a2460a521f tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '833136cbdeab4b9e8b6dd2d24c7c8753', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '77c9b7e351e24dc88207e9498f75f578', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:38:35.883901 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-233d20d0-6111-41b3-9171-58a2460a521f tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] 10.222.0.16 "GET /compute/v2.1/servers/2ce56b72-9fe8-41ca-bd6e-31591ef3b494" status: 200 len: 1266 microversion: 2.1 time: 0.202239 Jul 16 21:38:35.884715 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2629/5249] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:38:35 2022] GET /compute/v2.1/servers/2ce56b72-9fe8-41ca-bd6e-31591ef3b494 => generated 1266 bytes in 209 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:38:36.905398 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-1fffcd79-1b6f-4b97-a338-781ef3b522ca tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:38:36.906163 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-1fffcd79-1b6f-4b97-a338-781ef3b522ca tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] [instance: 2ce56b72-9fe8-41ca-bd6e-31591ef3b494] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:38:37.077810 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-1fffcd79-1b6f-4b97-a338-781ef3b522ca tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '833136cbdeab4b9e8b6dd2d24c7c8753', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '77c9b7e351e24dc88207e9498f75f578', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:38:37.087757 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-1fffcd79-1b6f-4b97-a338-781ef3b522ca tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] 10.222.0.16 "GET /compute/v2.1/servers/2ce56b72-9fe8-41ca-bd6e-31591ef3b494" status: 200 len: 1266 microversion: 2.1 time: 0.188932 Jul 16 21:38:37.089638 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2621/5250] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:38:36 2022] GET /compute/v2.1/servers/2ce56b72-9fe8-41ca-bd6e-31591ef3b494 => generated 1266 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:38:38.108902 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-e3e099c7-60a8-43c7-bd4f-e6e57d1aab51 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:38:38.109531 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-e3e099c7-60a8-43c7-bd4f-e6e57d1aab51 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] [instance: 2ce56b72-9fe8-41ca-bd6e-31591ef3b494] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:38:38.136014 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e3e099c7-60a8-43c7-bd4f-e6e57d1aab51 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:38:38.136629 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e3e099c7-60a8-43c7-bd4f-e6e57d1aab51 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:38:38.138192 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e3e099c7-60a8-43c7-bd4f-e6e57d1aab51 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:38:38.138192 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e3e099c7-60a8-43c7-bd4f-e6e57d1aab51 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:38:38.265003 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-e3e099c7-60a8-43c7-bd4f-e6e57d1aab51 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '833136cbdeab4b9e8b6dd2d24c7c8753', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '77c9b7e351e24dc88207e9498f75f578', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:38:38.285938 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-e3e099c7-60a8-43c7-bd4f-e6e57d1aab51 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] 10.222.0.16 "GET /compute/v2.1/servers/2ce56b72-9fe8-41ca-bd6e-31591ef3b494" status: 200 len: 1262 microversion: 2.1 time: 0.184106 Jul 16 21:38:38.286553 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2630/5251] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:38:38 2022] GET /compute/v2.1/servers/2ce56b72-9fe8-41ca-bd6e-31591ef3b494 => generated 1262 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:38:39.306303 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-1c3b4203-74f8-4c34-8092-5635b535d6ca tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:38:39.307702 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-1c3b4203-74f8-4c34-8092-5635b535d6ca tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] [instance: 2ce56b72-9fe8-41ca-bd6e-31591ef3b494] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:38:39.322063 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1c3b4203-74f8-4c34-8092-5635b535d6ca tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:38:39.322524 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1c3b4203-74f8-4c34-8092-5635b535d6ca tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:38:39.323436 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1c3b4203-74f8-4c34-8092-5635b535d6ca tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:38:39.333887 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1c3b4203-74f8-4c34-8092-5635b535d6ca tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.010s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:38:39.478959 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-08dd3459-00a7-4c63-9f39-cf855bebfd33 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "cf391899-1c79-49a5-8aea-2e9463b0c6f8", "device": "/dev/sdb"}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:38:39.480231 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-08dd3459-00a7-4c63-9f39-cf855bebfd33 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] [instance: 21fc83e5-8f03-4930-93b1-0196f7327b3e] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:38:39.500799 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-08dd3459-00a7-4c63-9f39-cf855bebfd33 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:38:39.504596 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-08dd3459-00a7-4c63-9f39-cf855bebfd33 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:38:39.507584 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-08dd3459-00a7-4c63-9f39-cf855bebfd33 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:38:39.508834 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-08dd3459-00a7-4c63-9f39-cf855bebfd33 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:38:39.508995 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-1c3b4203-74f8-4c34-8092-5635b535d6ca tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '833136cbdeab4b9e8b6dd2d24c7c8753', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '77c9b7e351e24dc88207e9498f75f578', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:38:39.520172 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-1c3b4203-74f8-4c34-8092-5635b535d6ca tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] 10.222.0.16 "GET /compute/v2.1/servers/2ce56b72-9fe8-41ca-bd6e-31591ef3b494" status: 200 len: 1326 microversion: 2.1 time: 0.218663 Jul 16 21:38:39.521330 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2622/5252] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:38:39 2022] GET /compute/v2.1/servers/2ce56b72-9fe8-41ca-bd6e-31591ef3b494 => generated 1326 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:38:40.502440 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-08dd3459-00a7-4c63-9f39-cf855bebfd33 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] 10.222.0.16 "POST /compute/v2.1/servers/21fc83e5-8f03-4930-93b1-0196f7327b3e/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 1.028963 Jul 16 21:38:40.503594 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2631/5253] 10.222.0.16 () {60 vars in 1322 bytes} [Sat Jul 16 21:38:39 2022] POST /compute/v2.1/servers/21fc83e5-8f03-4930-93b1-0196f7327b3e/os-volume_attachments => generated 194 bytes in 1031 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 16 21:38:40.538876 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-084918d7-23da-4406-8c56-27418d4fff0b tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:38:40.540264 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-084918d7-23da-4406-8c56-27418d4fff0b tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] [instance: 2ce56b72-9fe8-41ca-bd6e-31591ef3b494] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:38:40.552572 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-084918d7-23da-4406-8c56-27418d4fff0b tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:38:40.553509 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-084918d7-23da-4406-8c56-27418d4fff0b tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:38:40.556270 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-084918d7-23da-4406-8c56-27418d4fff0b tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:38:40.563406 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-084918d7-23da-4406-8c56-27418d4fff0b tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:38:40.824761 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-084918d7-23da-4406-8c56-27418d4fff0b tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '833136cbdeab4b9e8b6dd2d24c7c8753', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '77c9b7e351e24dc88207e9498f75f578', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:38:40.834694 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-084918d7-23da-4406-8c56-27418d4fff0b tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] 10.222.0.16 "GET /compute/v2.1/servers/2ce56b72-9fe8-41ca-bd6e-31591ef3b494" status: 200 len: 1324 microversion: 2.1 time: 0.300433 Jul 16 21:38:40.835626 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2623/5254] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:38:40 2022] GET /compute/v2.1/servers/2ce56b72-9fe8-41ca-bd6e-31591ef3b494 => generated 1324 bytes in 302 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:38:41.859954 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-5bbbb785-0828-4297-8dd1-f0b3b1345a58 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:38:41.861270 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-5bbbb785-0828-4297-8dd1-f0b3b1345a58 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] [instance: 2ce56b72-9fe8-41ca-bd6e-31591ef3b494] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:38:41.889707 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5bbbb785-0828-4297-8dd1-f0b3b1345a58 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:38:41.899397 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5bbbb785-0828-4297-8dd1-f0b3b1345a58 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:38:41.899397 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5bbbb785-0828-4297-8dd1-f0b3b1345a58 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:38:41.899397 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5bbbb785-0828-4297-8dd1-f0b3b1345a58 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:38:42.363114 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-5bbbb785-0828-4297-8dd1-f0b3b1345a58 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '833136cbdeab4b9e8b6dd2d24c7c8753', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '77c9b7e351e24dc88207e9498f75f578', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:38:42.372684 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-5bbbb785-0828-4297-8dd1-f0b3b1345a58 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] 10.222.0.16 "GET /compute/v2.1/servers/2ce56b72-9fe8-41ca-bd6e-31591ef3b494" status: 200 len: 1391 microversion: 2.1 time: 0.523929 Jul 16 21:38:42.373422 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2632/5255] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:38:41 2022] GET /compute/v2.1/servers/2ce56b72-9fe8-41ca-bd6e-31591ef3b494 => generated 1391 bytes in 526 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:38:43.392110 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-c629f612-8044-4d74-b812-4e7ae2591ade tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:38:43.392744 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-c629f612-8044-4d74-b812-4e7ae2591ade tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] [instance: 2ce56b72-9fe8-41ca-bd6e-31591ef3b494] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:38:43.393120 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-a8b3358a-fc0a-4fa3-9220-6c1c0849b05a tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:38:43.393665 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-a8b3358a-fc0a-4fa3-9220-6c1c0849b05a tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] [instance: a7f9caa9-fff2-44b2-871d-7e4191ff39e5] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:38:43.403040 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c629f612-8044-4d74-b812-4e7ae2591ade tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:38:43.405488 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a8b3358a-fc0a-4fa3-9220-6c1c0849b05a tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:38:43.405959 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a8b3358a-fc0a-4fa3-9220-6c1c0849b05a tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:38:43.406589 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c629f612-8044-4d74-b812-4e7ae2591ade tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:38:43.406815 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a8b3358a-fc0a-4fa3-9220-6c1c0849b05a tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:38:43.409065 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c629f612-8044-4d74-b812-4e7ae2591ade tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:38:43.409524 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c629f612-8044-4d74-b812-4e7ae2591ade tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:38:43.416497 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a8b3358a-fc0a-4fa3-9220-6c1c0849b05a tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:38:43.478601 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-a8b3358a-fc0a-4fa3-9220-6c1c0849b05a tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] 10.222.0.16 "GET /compute/v2.1/servers/a7f9caa9-fff2-44b2-871d-7e4191ff39e5/ips" status: 200 len: 149 microversion: 2.1 time: 0.086006 Jul 16 21:38:43.482677 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2633/5256] 10.222.0.16 () {58 vars in 1247 bytes} [Sat Jul 16 21:38:43 2022] GET /compute/v2.1/servers/a7f9caa9-fff2-44b2-871d-7e4191ff39e5/ips => generated 149 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:38:43.618908 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-b735d426-4f13-4241-84f4-4dca2f92f55a req-01150ec1-4583-42d3-86cb-532a2ebf27e2 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "2ce56b72-9fe8-41ca-bd6e-31591ef3b494", "tag": "a75ca41e-859e-4a3f-a17c-93721015537e"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:38:43.633682 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-b735d426-4f13-4241-84f4-4dca2f92f55a req-01150ec1-4583-42d3-86cb-532a2ebf27e2 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:38:43.634745 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-b735d426-4f13-4241-84f4-4dca2f92f55a req-01150ec1-4583-42d3-86cb-532a2ebf27e2 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:38:43.658308 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-b735d426-4f13-4241-84f4-4dca2f92f55a req-01150ec1-4583-42d3-86cb-532a2ebf27e2 service nova] Creating event network-changed:a75ca41e-859e-4a3f-a17c-93721015537e for instance 2ce56b72-9fe8-41ca-bd6e-31591ef3b494 on nch2-master-1 Jul 16 21:38:43.666374 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-b735d426-4f13-4241-84f4-4dca2f92f55a req-01150ec1-4583-42d3-86cb-532a2ebf27e2 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.054444 Jul 16 21:38:43.667795 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2634/5257] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:38:43 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:38:43.800309 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-c629f612-8044-4d74-b812-4e7ae2591ade tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '833136cbdeab4b9e8b6dd2d24c7c8753', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '77c9b7e351e24dc88207e9498f75f578', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:38:43.811286 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-c629f612-8044-4d74-b812-4e7ae2591ade tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] 10.222.0.16 "GET /compute/v2.1/servers/2ce56b72-9fe8-41ca-bd6e-31591ef3b494" status: 200 len: 1391 microversion: 2.1 time: 0.424415 Jul 16 21:38:43.811917 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2624/5258] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:38:43 2022] GET /compute/v2.1/servers/2ce56b72-9fe8-41ca-bd6e-31591ef3b494 => generated 1391 bytes in 426 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:38:44.829527 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-8f918b2a-38bc-4f8c-9024-05d866523861 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:38:44.831576 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-8f918b2a-38bc-4f8c-9024-05d866523861 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] [instance: 2ce56b72-9fe8-41ca-bd6e-31591ef3b494] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:38:44.840879 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8f918b2a-38bc-4f8c-9024-05d866523861 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:38:44.841345 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8f918b2a-38bc-4f8c-9024-05d866523861 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:38:44.842136 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8f918b2a-38bc-4f8c-9024-05d866523861 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:38:44.842576 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8f918b2a-38bc-4f8c-9024-05d866523861 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:38:45.051990 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-8f918b2a-38bc-4f8c-9024-05d866523861 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '833136cbdeab4b9e8b6dd2d24c7c8753', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '77c9b7e351e24dc88207e9498f75f578', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:38:45.066440 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-8f918b2a-38bc-4f8c-9024-05d866523861 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] 10.222.0.16 "GET /compute/v2.1/servers/2ce56b72-9fe8-41ca-bd6e-31591ef3b494" status: 200 len: 1391 microversion: 2.1 time: 0.241569 Jul 16 21:38:45.067034 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2635/5259] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:38:44 2022] GET /compute/v2.1/servers/2ce56b72-9fe8-41ca-bd6e-31591ef3b494 => generated 1391 bytes in 244 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:38:46.083887 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-ee60ccd2-fb82-4b78-a684-4efafef6aae3 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:38:46.084593 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-ee60ccd2-fb82-4b78-a684-4efafef6aae3 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] [instance: 2ce56b72-9fe8-41ca-bd6e-31591ef3b494] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:38:46.089655 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell0:SELECT=2 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:38:46.097760 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ee60ccd2-fb82-4b78-a684-4efafef6aae3 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:38:46.098230 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ee60ccd2-fb82-4b78-a684-4efafef6aae3 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:38:46.099055 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ee60ccd2-fb82-4b78-a684-4efafef6aae3 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:38:46.099871 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ee60ccd2-fb82-4b78-a684-4efafef6aae3 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:38:46.275823 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-ee60ccd2-fb82-4b78-a684-4efafef6aae3 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '833136cbdeab4b9e8b6dd2d24c7c8753', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '77c9b7e351e24dc88207e9498f75f578', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:38:46.285337 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-ee60ccd2-fb82-4b78-a684-4efafef6aae3 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] 10.222.0.16 "GET /compute/v2.1/servers/2ce56b72-9fe8-41ca-bd6e-31591ef3b494" status: 200 len: 1391 microversion: 2.1 time: 0.205466 Jul 16 21:38:46.288283 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2625/5260] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:38:46 2022] GET /compute/v2.1/servers/2ce56b72-9fe8-41ca-bd6e-31591ef3b494 => generated 1391 bytes in 207 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:38:46.680077 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-c0460c8e-c39f-454a-a1d5-6fa9ca0e0870 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "0e8d79ef-dae3-4fef-bd30-da0b85b52de1"}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:38:46.681267 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-c0460c8e-c39f-454a-a1d5-6fa9ca0e0870 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] [instance: a7f9caa9-fff2-44b2-871d-7e4191ff39e5] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:38:46.685341 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell0:SELECT=2 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:38:46.689811 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c0460c8e-c39f-454a-a1d5-6fa9ca0e0870 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:38:46.690251 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c0460c8e-c39f-454a-a1d5-6fa9ca0e0870 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:38:46.691045 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c0460c8e-c39f-454a-a1d5-6fa9ca0e0870 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:38:46.691544 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c0460c8e-c39f-454a-a1d5-6fa9ca0e0870 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:38:47.303302 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-8865c2ba-0bbb-4096-9913-e102a803eaaa tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:38:47.303896 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-8865c2ba-0bbb-4096-9913-e102a803eaaa tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] [instance: 2ce56b72-9fe8-41ca-bd6e-31591ef3b494] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:38:47.309898 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell0:SELECT=8 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:38:47.315841 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8865c2ba-0bbb-4096-9913-e102a803eaaa tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:38:47.316739 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8865c2ba-0bbb-4096-9913-e102a803eaaa tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:38:47.317525 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8865c2ba-0bbb-4096-9913-e102a803eaaa tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:38:47.318179 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8865c2ba-0bbb-4096-9913-e102a803eaaa tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:38:47.468403 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-8865c2ba-0bbb-4096-9913-e102a803eaaa tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '833136cbdeab4b9e8b6dd2d24c7c8753', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '77c9b7e351e24dc88207e9498f75f578', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:38:47.476216 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-8865c2ba-0bbb-4096-9913-e102a803eaaa tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] 10.222.0.16 "GET /compute/v2.1/servers/2ce56b72-9fe8-41ca-bd6e-31591ef3b494" status: 200 len: 1391 microversion: 2.1 time: 0.177140 Jul 16 21:38:47.477424 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2626/5261] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:38:47 2022] GET /compute/v2.1/servers/2ce56b72-9fe8-41ca-bd6e-31591ef3b494 => generated 1391 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:38:47.673214 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-c0460c8e-c39f-454a-a1d5-6fa9ca0e0870 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] 10.222.0.16 "POST /compute/v2.1/servers/a7f9caa9-fff2-44b2-871d-7e4191ff39e5/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.996964 Jul 16 21:38:47.679512 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2636/5262] 10.222.0.16 () {60 vars in 1322 bytes} [Sat Jul 16 21:38:46 2022] POST /compute/v2.1/servers/a7f9caa9-fff2-44b2-871d-7e4191ff39e5/os-volume_attachments => generated 194 bytes in 1004 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:38:48.495428 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-789fd23d-4619-4586-be73-b712e58321b2 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:38:48.496062 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-789fd23d-4619-4586-be73-b712e58321b2 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] [instance: 2ce56b72-9fe8-41ca-bd6e-31591ef3b494] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:38:48.503564 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-789fd23d-4619-4586-be73-b712e58321b2 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:38:48.504035 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-789fd23d-4619-4586-be73-b712e58321b2 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:38:48.505043 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-789fd23d-4619-4586-be73-b712e58321b2 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:38:48.505496 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-789fd23d-4619-4586-be73-b712e58321b2 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:38:48.672829 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-789fd23d-4619-4586-be73-b712e58321b2 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '833136cbdeab4b9e8b6dd2d24c7c8753', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '77c9b7e351e24dc88207e9498f75f578', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:38:48.676640 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-789fd23d-4619-4586-be73-b712e58321b2 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] 10.222.0.16 "GET /compute/v2.1/servers/2ce56b72-9fe8-41ca-bd6e-31591ef3b494" status: 200 len: 1391 microversion: 2.1 time: 0.186326 Jul 16 21:38:48.677225 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2627/5263] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:38:48 2022] GET /compute/v2.1/servers/2ce56b72-9fe8-41ca-bd6e-31591ef3b494 => generated 1391 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:38:49.695814 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-d2fb7bc0-7758-46e7-a02f-71ce30acd4e0 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:38:49.697131 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-d2fb7bc0-7758-46e7-a02f-71ce30acd4e0 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] [instance: 2ce56b72-9fe8-41ca-bd6e-31591ef3b494] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:38:49.704195 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d2fb7bc0-7758-46e7-a02f-71ce30acd4e0 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:38:49.704654 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d2fb7bc0-7758-46e7-a02f-71ce30acd4e0 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:38:49.705432 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d2fb7bc0-7758-46e7-a02f-71ce30acd4e0 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:38:49.705911 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d2fb7bc0-7758-46e7-a02f-71ce30acd4e0 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:38:49.845647 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-d2fb7bc0-7758-46e7-a02f-71ce30acd4e0 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '833136cbdeab4b9e8b6dd2d24c7c8753', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '77c9b7e351e24dc88207e9498f75f578', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:38:49.854196 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-d2fb7bc0-7758-46e7-a02f-71ce30acd4e0 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] 10.222.0.16 "GET /compute/v2.1/servers/2ce56b72-9fe8-41ca-bd6e-31591ef3b494" status: 200 len: 1391 microversion: 2.1 time: 0.164214 Jul 16 21:38:49.855073 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2637/5264] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:38:49 2022] GET /compute/v2.1/servers/2ce56b72-9fe8-41ca-bd6e-31591ef3b494 => generated 1391 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:38:50.700868 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-150247a3-1d96-4ee7-b801-84db12e66813 req-435a64df-312e-4f47-9b8a-52753408280c service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "2ce56b72-9fe8-41ca-bd6e-31591ef3b494", "name": "network-vif-plugged", "status": "completed", "tag": "a75ca41e-859e-4a3f-a17c-93721015537e"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:38:50.710877 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-150247a3-1d96-4ee7-b801-84db12e66813 req-435a64df-312e-4f47-9b8a-52753408280c service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:38:50.711474 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-150247a3-1d96-4ee7-b801-84db12e66813 req-435a64df-312e-4f47-9b8a-52753408280c service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:38:50.724687 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-150247a3-1d96-4ee7-b801-84db12e66813 req-435a64df-312e-4f47-9b8a-52753408280c service nova] Creating event network-vif-plugged:a75ca41e-859e-4a3f-a17c-93721015537e for instance 2ce56b72-9fe8-41ca-bd6e-31591ef3b494 on nch2-master-1 Jul 16 21:38:50.734197 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-150247a3-1d96-4ee7-b801-84db12e66813 req-435a64df-312e-4f47-9b8a-52753408280c service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.037887 Jul 16 21:38:50.743936 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2628/5265] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:38:50 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 16 21:38:50.872633 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-3655e603-a406-46c9-b373-fdd4a44c5a40 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:38:50.873451 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-3655e603-a406-46c9-b373-fdd4a44c5a40 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] [instance: 2ce56b72-9fe8-41ca-bd6e-31591ef3b494] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:38:50.881524 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3655e603-a406-46c9-b373-fdd4a44c5a40 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:38:50.882136 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3655e603-a406-46c9-b373-fdd4a44c5a40 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:38:50.882979 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3655e603-a406-46c9-b373-fdd4a44c5a40 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:38:50.884122 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3655e603-a406-46c9-b373-fdd4a44c5a40 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:38:51.025592 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-3655e603-a406-46c9-b373-fdd4a44c5a40 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '833136cbdeab4b9e8b6dd2d24c7c8753', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '77c9b7e351e24dc88207e9498f75f578', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:38:51.033523 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-3655e603-a406-46c9-b373-fdd4a44c5a40 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] 10.222.0.16 "GET /compute/v2.1/servers/2ce56b72-9fe8-41ca-bd6e-31591ef3b494" status: 200 len: 1391 microversion: 2.1 time: 0.165499 Jul 16 21:38:51.034097 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2638/5266] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:38:50 2022] GET /compute/v2.1/servers/2ce56b72-9fe8-41ca-bd6e-31591ef3b494 => generated 1391 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:38:52.052045 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-fe2eec86-5ec8-4f53-a4b1-04de9a308956 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:38:52.052805 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-fe2eec86-5ec8-4f53-a4b1-04de9a308956 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] [instance: 2ce56b72-9fe8-41ca-bd6e-31591ef3b494] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:38:52.062139 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-fe2eec86-5ec8-4f53-a4b1-04de9a308956 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:38:52.067158 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-fe2eec86-5ec8-4f53-a4b1-04de9a308956 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:38:52.067158 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-fe2eec86-5ec8-4f53-a4b1-04de9a308956 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:38:52.067158 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-fe2eec86-5ec8-4f53-a4b1-04de9a308956 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:38:52.221626 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-fe2eec86-5ec8-4f53-a4b1-04de9a308956 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '833136cbdeab4b9e8b6dd2d24c7c8753', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '77c9b7e351e24dc88207e9498f75f578', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:38:52.237129 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-fe2eec86-5ec8-4f53-a4b1-04de9a308956 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] 10.222.0.16 "GET /compute/v2.1/servers/2ce56b72-9fe8-41ca-bd6e-31591ef3b494" status: 200 len: 1391 microversion: 2.1 time: 0.183272 Jul 16 21:38:52.237129 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2629/5267] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:38:52 2022] GET /compute/v2.1/servers/2ce56b72-9fe8-41ca-bd6e-31591ef3b494 => generated 1391 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:38:53.252271 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-4900ecec-0a8a-49de-a0bd-59006b8c1a43 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:38:53.252897 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-4900ecec-0a8a-49de-a0bd-59006b8c1a43 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] [instance: 2ce56b72-9fe8-41ca-bd6e-31591ef3b494] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:38:53.262635 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4900ecec-0a8a-49de-a0bd-59006b8c1a43 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:38:53.266538 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4900ecec-0a8a-49de-a0bd-59006b8c1a43 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:38:53.266538 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4900ecec-0a8a-49de-a0bd-59006b8c1a43 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:38:53.266538 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4900ecec-0a8a-49de-a0bd-59006b8c1a43 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:38:53.427432 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-4900ecec-0a8a-49de-a0bd-59006b8c1a43 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '833136cbdeab4b9e8b6dd2d24c7c8753', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '77c9b7e351e24dc88207e9498f75f578', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:38:53.435045 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-4900ecec-0a8a-49de-a0bd-59006b8c1a43 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] 10.222.0.16 "GET /compute/v2.1/servers/2ce56b72-9fe8-41ca-bd6e-31591ef3b494" status: 200 len: 1563 microversion: 2.1 time: 0.190153 Jul 16 21:38:53.435813 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2639/5268] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:38:53 2022] GET /compute/v2.1/servers/2ce56b72-9fe8-41ca-bd6e-31591ef3b494 => generated 1563 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:38:53.453943 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-292d818c-3577-440a-82bb-b9476e1eadcc tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:38:53.454525 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-292d818c-3577-440a-82bb-b9476e1eadcc tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] [instance: 2ce56b72-9fe8-41ca-bd6e-31591ef3b494] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:38:53.462342 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-292d818c-3577-440a-82bb-b9476e1eadcc tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:38:53.463094 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-292d818c-3577-440a-82bb-b9476e1eadcc tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:38:53.463707 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-292d818c-3577-440a-82bb-b9476e1eadcc tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:38:53.464155 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-292d818c-3577-440a-82bb-b9476e1eadcc tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:38:53.617920 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-292d818c-3577-440a-82bb-b9476e1eadcc tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '833136cbdeab4b9e8b6dd2d24c7c8753', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '77c9b7e351e24dc88207e9498f75f578', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:38:53.627236 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-292d818c-3577-440a-82bb-b9476e1eadcc tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] 10.222.0.16 "GET /compute/v2.1/servers/2ce56b72-9fe8-41ca-bd6e-31591ef3b494" status: 200 len: 1563 microversion: 2.1 time: 0.178988 Jul 16 21:38:53.628026 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2630/5269] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:38:53 2022] GET /compute/v2.1/servers/2ce56b72-9fe8-41ca-bd6e-31591ef3b494 => generated 1563 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:38:55.889772 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-20251cc0-7f75-4265-9c8f-0de679bfce0f tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:38:55.890482 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-20251cc0-7f75-4265-9c8f-0de679bfce0f tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] [instance: 2ce56b72-9fe8-41ca-bd6e-31591ef3b494] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:38:55.894028 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-effd142a-b2d4-418c-b0ba-d12216a99d64 req-cdc313e2-bc28-45a6-8eeb-b8c29627c3f6 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "2ce56b72-9fe8-41ca-bd6e-31591ef3b494", "tag": "a75ca41e-859e-4a3f-a17c-93721015537e"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:38:55.897674 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-20251cc0-7f75-4265-9c8f-0de679bfce0f tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:38:55.898251 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-20251cc0-7f75-4265-9c8f-0de679bfce0f tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:38:55.899050 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-20251cc0-7f75-4265-9c8f-0de679bfce0f tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:38:55.899583 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-20251cc0-7f75-4265-9c8f-0de679bfce0f tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:38:55.900831 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-effd142a-b2d4-418c-b0ba-d12216a99d64 req-cdc313e2-bc28-45a6-8eeb-b8c29627c3f6 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:38:55.901290 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-effd142a-b2d4-418c-b0ba-d12216a99d64 req-cdc313e2-bc28-45a6-8eeb-b8c29627c3f6 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:38:55.913847 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-effd142a-b2d4-418c-b0ba-d12216a99d64 req-cdc313e2-bc28-45a6-8eeb-b8c29627c3f6 service nova] Creating event network-changed:a75ca41e-859e-4a3f-a17c-93721015537e for instance 2ce56b72-9fe8-41ca-bd6e-31591ef3b494 on nch2-master-1 Jul 16 21:38:55.919620 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-effd142a-b2d4-418c-b0ba-d12216a99d64 req-cdc313e2-bc28-45a6-8eeb-b8c29627c3f6 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.031369 Jul 16 21:38:55.920599 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2631/5270] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:38:55 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 16 21:38:56.164799 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-20251cc0-7f75-4265-9c8f-0de679bfce0f tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '833136cbdeab4b9e8b6dd2d24c7c8753', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '77c9b7e351e24dc88207e9498f75f578', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:38:56.176811 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-20251cc0-7f75-4265-9c8f-0de679bfce0f tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] 10.222.0.16 "GET /compute/v2.1/servers/2ce56b72-9fe8-41ca-bd6e-31591ef3b494" status: 200 len: 1563 microversion: 2.1 time: 0.291596 Jul 16 21:38:56.178940 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2640/5271] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:38:55 2022] GET /compute/v2.1/servers/2ce56b72-9fe8-41ca-bd6e-31591ef3b494 => generated 1563 bytes in 294 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:38:57.623659 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-329a9f89-7bed-40f7-95f5-87457a7e7941 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestVolumeBootPattern-server-1782805578", "imageRef": "", "flavorRef": "452", "block_device_mapping_v2": [{"uuid": "1271e76a-1165-49b4-b725-7079124c1d75", "source_type": "snapshot", "destination_type": "volume", "boot_index": 0, "delete_on_termination": true}], "networks": [{"uuid": "94c922c8-150a-417e-8274-7473d6567b26"}]}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:38:57.719853 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.network.neutron [None req-329a9f89-7bed-40f7-95f5-87457a7e7941 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] validate_networks() for [('94c922c8-150a-417e-8274-7473d6567b26', None, None, None, None, None)] {{(pid=138680) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 16 21:38:58.128757 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-329a9f89-7bed-40f7-95f5-87457a7e7941 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Flavor limits 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 16 21:38:58.129138 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-329a9f89-7bed-40f7-95f5-87457a7e7941 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Image limits 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 16 21:38:58.129610 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-329a9f89-7bed-40f7-95f5-87457a7e7941 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Flavor pref 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 16 21:38:58.130017 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-329a9f89-7bed-40f7-95f5-87457a7e7941 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Image pref 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 16 21:38:58.130498 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-329a9f89-7bed-40f7-95f5-87457a7e7941 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 16 21:38:58.141349 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-329a9f89-7bed-40f7-95f5-87457a7e7941 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:38:58.146458 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-329a9f89-7bed-40f7-95f5-87457a7e7941 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:38:58.146458 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-329a9f89-7bed-40f7-95f5-87457a7e7941 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:38:58.146458 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-329a9f89-7bed-40f7-95f5-87457a7e7941 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:38:58.162473 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-329a9f89-7bed-40f7-95f5-87457a7e7941 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Getting quotas for project b533f85aeddf4d8db05f2c53a61befc3. Resources: {'ram', 'instances', 'cores'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:38:58.168064 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-329a9f89-7bed-40f7-95f5-87457a7e7941 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Getting quotas for user 62209b9802ab46c3b3d094d4d1ad86fe and project b533f85aeddf4d8db05f2c53a61befc3. Resources: {'ram', 'instances', 'cores'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:38:58.184460 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-329a9f89-7bed-40f7-95f5-87457a7e7941 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Going to run 1 instances... {{(pid=138680) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 16 21:38:58.188390 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-329a9f89-7bed-40f7-95f5-87457a7e7941 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] [instance: 8f28a606-c8d6-44cf-89d8-d856f3e8cfea] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id='1271e76a-1165-49b4-b725-7079124c1d75',source_type='snapshot',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138680) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 16 21:38:58.389606 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-329a9f89-7bed-40f7-95f5-87457a7e7941 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.771513 Jul 16 21:38:58.390179 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2632/5272] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 21:38:57 2022] POST /compute/v2.1/servers => generated 380 bytes in 773 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 16 21:38:58.400596 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-db52b81a-c57b-4f3b-b975-6c88b26aa734 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:38:58.401378 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-db52b81a-c57b-4f3b-b975-6c88b26aa734 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] [instance: 8f28a606-c8d6-44cf-89d8-d856f3e8cfea] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:38:58.460057 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-db52b81a-c57b-4f3b-b975-6c88b26aa734 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '62209b9802ab46c3b3d094d4d1ad86fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b533f85aeddf4d8db05f2c53a61befc3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:38:58.467666 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-db52b81a-c57b-4f3b-b975-6c88b26aa734 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] 10.222.0.16 "GET /compute/v2.1/servers/8f28a606-c8d6-44cf-89d8-d856f3e8cfea" status: 200 len: 1057 microversion: 2.1 time: 0.071528 Jul 16 21:38:58.476891 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2641/5273] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:38:58 2022] GET /compute/v2.1/servers/8f28a606-c8d6-44cf-89d8-d856f3e8cfea => generated 1057 bytes in 73 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:38:59.486066 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-b32aa62d-78b7-4ac9-9321-c57bc62ffa5c tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:38:59.486694 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-b32aa62d-78b7-4ac9-9321-c57bc62ffa5c tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] [instance: 8f28a606-c8d6-44cf-89d8-d856f3e8cfea] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:38:59.544167 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-b32aa62d-78b7-4ac9-9321-c57bc62ffa5c tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '62209b9802ab46c3b3d094d4d1ad86fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b533f85aeddf4d8db05f2c53a61befc3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:38:59.551151 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-b32aa62d-78b7-4ac9-9321-c57bc62ffa5c tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] 10.222.0.16 "GET /compute/v2.1/servers/8f28a606-c8d6-44cf-89d8-d856f3e8cfea" status: 200 len: 1057 microversion: 2.1 time: 0.070344 Jul 16 21:38:59.551692 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2633/5274] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:38:59 2022] GET /compute/v2.1/servers/8f28a606-c8d6-44cf-89d8-d856f3e8cfea => generated 1057 bytes in 72 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:39:00.567302 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-f6da8c7d-edc8-4a86-a9f7-5576c3242f67 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:39:00.567902 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-f6da8c7d-edc8-4a86-a9f7-5576c3242f67 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] [instance: 8f28a606-c8d6-44cf-89d8-d856f3e8cfea] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:39:00.576856 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f6da8c7d-edc8-4a86-a9f7-5576c3242f67 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:39:00.577243 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f6da8c7d-edc8-4a86-a9f7-5576c3242f67 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:39:00.578015 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f6da8c7d-edc8-4a86-a9f7-5576c3242f67 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:39:00.578458 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f6da8c7d-edc8-4a86-a9f7-5576c3242f67 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:39:00.582599 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=80,nova_cell1:UPDATE=4,nova_cell1:INSERT=2 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:39:00.641937 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-f6da8c7d-edc8-4a86-a9f7-5576c3242f67 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '62209b9802ab46c3b3d094d4d1ad86fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b533f85aeddf4d8db05f2c53a61befc3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:39:00.649285 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-f6da8c7d-edc8-4a86-a9f7-5576c3242f67 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] 10.222.0.16 "GET /compute/v2.1/servers/8f28a606-c8d6-44cf-89d8-d856f3e8cfea" status: 200 len: 1109 microversion: 2.1 time: 0.085930 Jul 16 21:39:00.650578 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2642/5275] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:39:00 2022] GET /compute/v2.1/servers/8f28a606-c8d6-44cf-89d8-d856f3e8cfea => generated 1109 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:39:01.668365 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-6f0fdd6b-344f-430a-899c-2098a6237c98 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:39:01.669129 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-6f0fdd6b-344f-430a-899c-2098a6237c98 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] [instance: 8f28a606-c8d6-44cf-89d8-d856f3e8cfea] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:39:01.676535 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6f0fdd6b-344f-430a-899c-2098a6237c98 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:39:01.676999 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6f0fdd6b-344f-430a-899c-2098a6237c98 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:39:01.677775 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6f0fdd6b-344f-430a-899c-2098a6237c98 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:39:01.678220 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6f0fdd6b-344f-430a-899c-2098a6237c98 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:39:01.683263 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=68 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:39:01.929025 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-6f0fdd6b-344f-430a-899c-2098a6237c98 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '62209b9802ab46c3b3d094d4d1ad86fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b533f85aeddf4d8db05f2c53a61befc3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:39:01.940625 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-6f0fdd6b-344f-430a-899c-2098a6237c98 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] 10.222.0.16 "GET /compute/v2.1/servers/8f28a606-c8d6-44cf-89d8-d856f3e8cfea" status: 200 len: 1169 microversion: 2.1 time: 0.276708 Jul 16 21:39:01.941451 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2634/5276] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:39:01 2022] GET /compute/v2.1/servers/8f28a606-c8d6-44cf-89d8-d856f3e8cfea => generated 1169 bytes in 279 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:39:02.963715 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-9602e4f1-cc40-44c4-8247-efdcbad65afe tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:39:02.965996 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-9602e4f1-cc40-44c4-8247-efdcbad65afe tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] [instance: 8f28a606-c8d6-44cf-89d8-d856f3e8cfea] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:39:02.985782 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9602e4f1-cc40-44c4-8247-efdcbad65afe tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:39:02.985782 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9602e4f1-cc40-44c4-8247-efdcbad65afe tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:39:02.985782 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9602e4f1-cc40-44c4-8247-efdcbad65afe tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:39:02.985782 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9602e4f1-cc40-44c4-8247-efdcbad65afe tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:39:03.221778 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-9602e4f1-cc40-44c4-8247-efdcbad65afe tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '62209b9802ab46c3b3d094d4d1ad86fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b533f85aeddf4d8db05f2c53a61befc3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:39:03.230764 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-9602e4f1-cc40-44c4-8247-efdcbad65afe tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] 10.222.0.16 "GET /compute/v2.1/servers/8f28a606-c8d6-44cf-89d8-d856f3e8cfea" status: 200 len: 1169 microversion: 2.1 time: 0.276695 Jul 16 21:39:03.231757 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2643/5277] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:39:02 2022] GET /compute/v2.1/servers/8f28a606-c8d6-44cf-89d8-d856f3e8cfea => generated 1169 bytes in 278 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:39:03.418076 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-63bd73c5-bc7b-4e3e-b72e-e40704be080b req-8e6cd285-dc70-445d-9dcf-b41e7f507325 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "8f28a606-c8d6-44cf-89d8-d856f3e8cfea", "tag": "5825fe15-c9bd-4ee3-89d8-bb8729347983"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:39:03.430590 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-63bd73c5-bc7b-4e3e-b72e-e40704be080b req-8e6cd285-dc70-445d-9dcf-b41e7f507325 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:39:03.431357 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-63bd73c5-bc7b-4e3e-b72e-e40704be080b req-8e6cd285-dc70-445d-9dcf-b41e7f507325 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:39:03.449926 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-63bd73c5-bc7b-4e3e-b72e-e40704be080b req-8e6cd285-dc70-445d-9dcf-b41e7f507325 service nova] Creating event network-changed:5825fe15-c9bd-4ee3-89d8-bb8729347983 for instance 8f28a606-c8d6-44cf-89d8-d856f3e8cfea on nch1-master-1 Jul 16 21:39:03.455061 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-63bd73c5-bc7b-4e3e-b72e-e40704be080b req-8e6cd285-dc70-445d-9dcf-b41e7f507325 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.043118 Jul 16 21:39:03.456006 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2635/5278] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:39:03 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:39:04.248173 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-ec3f656d-a918-485f-9099-bce51f6814f1 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:39:04.248859 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-ec3f656d-a918-485f-9099-bce51f6814f1 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] [instance: 8f28a606-c8d6-44cf-89d8-d856f3e8cfea] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:39:04.256091 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ec3f656d-a918-485f-9099-bce51f6814f1 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:39:04.256542 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ec3f656d-a918-485f-9099-bce51f6814f1 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:39:04.257301 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ec3f656d-a918-485f-9099-bce51f6814f1 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:39:04.257737 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ec3f656d-a918-485f-9099-bce51f6814f1 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:39:04.391939 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-ec3f656d-a918-485f-9099-bce51f6814f1 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '62209b9802ab46c3b3d094d4d1ad86fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b533f85aeddf4d8db05f2c53a61befc3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:39:04.400339 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-ec3f656d-a918-485f-9099-bce51f6814f1 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] 10.222.0.16 "GET /compute/v2.1/servers/8f28a606-c8d6-44cf-89d8-d856f3e8cfea" status: 200 len: 1169 microversion: 2.1 time: 0.156604 Jul 16 21:39:04.401107 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2644/5279] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:39:04 2022] GET /compute/v2.1/servers/8f28a606-c8d6-44cf-89d8-d856f3e8cfea => generated 1169 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:39:05.419496 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-d7b3fb84-1d6c-40a0-8e91-ebc682132644 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:39:05.420206 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-d7b3fb84-1d6c-40a0-8e91-ebc682132644 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] [instance: 8f28a606-c8d6-44cf-89d8-d856f3e8cfea] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:39:05.426619 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d7b3fb84-1d6c-40a0-8e91-ebc682132644 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:39:05.427108 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d7b3fb84-1d6c-40a0-8e91-ebc682132644 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:39:05.427881 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d7b3fb84-1d6c-40a0-8e91-ebc682132644 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:39:05.428329 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d7b3fb84-1d6c-40a0-8e91-ebc682132644 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:39:05.604692 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-d7b3fb84-1d6c-40a0-8e91-ebc682132644 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '62209b9802ab46c3b3d094d4d1ad86fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b533f85aeddf4d8db05f2c53a61befc3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:39:05.612969 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-d7b3fb84-1d6c-40a0-8e91-ebc682132644 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] 10.222.0.16 "GET /compute/v2.1/servers/8f28a606-c8d6-44cf-89d8-d856f3e8cfea" status: 200 len: 1169 microversion: 2.1 time: 0.199525 Jul 16 21:39:05.619066 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2636/5280] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:39:05 2022] GET /compute/v2.1/servers/8f28a606-c8d6-44cf-89d8-d856f3e8cfea => generated 1169 bytes in 201 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:39:06.631846 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-d6cdf331-89bc-4017-ac0b-f9c8431d599e tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:39:06.632409 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-d6cdf331-89bc-4017-ac0b-f9c8431d599e tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] [instance: 8f28a606-c8d6-44cf-89d8-d856f3e8cfea] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:39:06.640718 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d6cdf331-89bc-4017-ac0b-f9c8431d599e tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:39:06.641158 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d6cdf331-89bc-4017-ac0b-f9c8431d599e tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:39:06.641923 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d6cdf331-89bc-4017-ac0b-f9c8431d599e tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:39:06.642357 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d6cdf331-89bc-4017-ac0b-f9c8431d599e tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:39:06.816342 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-d6cdf331-89bc-4017-ac0b-f9c8431d599e tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '62209b9802ab46c3b3d094d4d1ad86fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b533f85aeddf4d8db05f2c53a61befc3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:39:06.825403 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-d6cdf331-89bc-4017-ac0b-f9c8431d599e tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] 10.222.0.16 "GET /compute/v2.1/servers/8f28a606-c8d6-44cf-89d8-d856f3e8cfea" status: 200 len: 1215 microversion: 2.1 time: 0.197963 Jul 16 21:39:06.826626 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2645/5281] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:39:06 2022] GET /compute/v2.1/servers/8f28a606-c8d6-44cf-89d8-d856f3e8cfea => generated 1215 bytes in 200 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:39:07.842347 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-c1982aa5-9acd-4046-9425-1878abbf4c85 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:39:07.842982 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-c1982aa5-9acd-4046-9425-1878abbf4c85 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] [instance: 8f28a606-c8d6-44cf-89d8-d856f3e8cfea] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:39:07.847060 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_api:SELECT=64,nova_api:INSERT=6 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:39:07.866693 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c1982aa5-9acd-4046-9425-1878abbf4c85 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:39:07.866693 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c1982aa5-9acd-4046-9425-1878abbf4c85 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:39:07.866693 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c1982aa5-9acd-4046-9425-1878abbf4c85 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:39:07.866693 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c1982aa5-9acd-4046-9425-1878abbf4c85 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:39:08.002884 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-c1982aa5-9acd-4046-9425-1878abbf4c85 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '62209b9802ab46c3b3d094d4d1ad86fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b533f85aeddf4d8db05f2c53a61befc3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:39:08.011121 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-c1982aa5-9acd-4046-9425-1878abbf4c85 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] 10.222.0.16 "GET /compute/v2.1/servers/8f28a606-c8d6-44cf-89d8-d856f3e8cfea" status: 200 len: 1203 microversion: 2.1 time: 0.172748 Jul 16 21:39:08.011701 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2637/5282] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:39:07 2022] GET /compute/v2.1/servers/8f28a606-c8d6-44cf-89d8-d856f3e8cfea => generated 1203 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:39:09.028429 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-49ae4715-8c23-49d3-bdef-041da2346ca2 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:39:09.029054 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-49ae4715-8c23-49d3-bdef-041da2346ca2 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] [instance: 8f28a606-c8d6-44cf-89d8-d856f3e8cfea] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:39:09.032070 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell0:SELECT=2 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:39:09.037455 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-49ae4715-8c23-49d3-bdef-041da2346ca2 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:39:09.040527 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-49ae4715-8c23-49d3-bdef-041da2346ca2 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:39:09.040527 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-49ae4715-8c23-49d3-bdef-041da2346ca2 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:39:09.040527 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-49ae4715-8c23-49d3-bdef-041da2346ca2 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:39:09.183365 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-49ae4715-8c23-49d3-bdef-041da2346ca2 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '62209b9802ab46c3b3d094d4d1ad86fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b533f85aeddf4d8db05f2c53a61befc3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:39:09.191493 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-49ae4715-8c23-49d3-bdef-041da2346ca2 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] 10.222.0.16 "GET /compute/v2.1/servers/8f28a606-c8d6-44cf-89d8-d856f3e8cfea" status: 200 len: 1203 microversion: 2.1 time: 0.166999 Jul 16 21:39:09.192071 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2646/5283] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:39:09 2022] GET /compute/v2.1/servers/8f28a606-c8d6-44cf-89d8-d856f3e8cfea => generated 1203 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:39:10.208272 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-b6c86c1a-b733-4f48-8fcc-a475e0332d9f tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:39:10.208891 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-b6c86c1a-b733-4f48-8fcc-a475e0332d9f tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] [instance: 8f28a606-c8d6-44cf-89d8-d856f3e8cfea] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:39:10.210740 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell0:SELECT=2 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:39:10.211882 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell0:SELECT=8 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:39:10.220047 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b6c86c1a-b733-4f48-8fcc-a475e0332d9f tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:39:10.220503 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b6c86c1a-b733-4f48-8fcc-a475e0332d9f tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:39:10.221266 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b6c86c1a-b733-4f48-8fcc-a475e0332d9f tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:39:10.221710 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b6c86c1a-b733-4f48-8fcc-a475e0332d9f tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:39:10.364497 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-b6c86c1a-b733-4f48-8fcc-a475e0332d9f tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '62209b9802ab46c3b3d094d4d1ad86fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b533f85aeddf4d8db05f2c53a61befc3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:39:10.372617 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-b6c86c1a-b733-4f48-8fcc-a475e0332d9f tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] 10.222.0.16 "GET /compute/v2.1/servers/8f28a606-c8d6-44cf-89d8-d856f3e8cfea" status: 200 len: 1203 microversion: 2.1 time: 0.168705 Jul 16 21:39:10.373196 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2638/5284] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:39:10 2022] GET /compute/v2.1/servers/8f28a606-c8d6-44cf-89d8-d856f3e8cfea => generated 1203 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:39:11.390263 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-166e5c3b-bcf2-40fa-86c0-c5845a6c1b76 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:39:11.390884 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-166e5c3b-bcf2-40fa-86c0-c5845a6c1b76 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] [instance: 8f28a606-c8d6-44cf-89d8-d856f3e8cfea] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:39:11.394065 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_api:SELECT=44 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:39:11.398929 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-166e5c3b-bcf2-40fa-86c0-c5845a6c1b76 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:39:11.399416 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-166e5c3b-bcf2-40fa-86c0-c5845a6c1b76 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:39:11.400191 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-166e5c3b-bcf2-40fa-86c0-c5845a6c1b76 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:39:11.400631 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-166e5c3b-bcf2-40fa-86c0-c5845a6c1b76 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:39:11.533200 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-166e5c3b-bcf2-40fa-86c0-c5845a6c1b76 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '62209b9802ab46c3b3d094d4d1ad86fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b533f85aeddf4d8db05f2c53a61befc3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:39:11.540900 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-166e5c3b-bcf2-40fa-86c0-c5845a6c1b76 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] 10.222.0.16 "GET /compute/v2.1/servers/8f28a606-c8d6-44cf-89d8-d856f3e8cfea" status: 200 len: 1203 microversion: 2.1 time: 0.155434 Jul 16 21:39:11.541483 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2647/5285] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:39:11 2022] GET /compute/v2.1/servers/8f28a606-c8d6-44cf-89d8-d856f3e8cfea => generated 1203 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:39:12.557770 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-cb0ba6e7-b9c9-4e2f-93ea-1ca02d5e6fd8 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:39:12.558386 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-cb0ba6e7-b9c9-4e2f-93ea-1ca02d5e6fd8 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] [instance: 8f28a606-c8d6-44cf-89d8-d856f3e8cfea] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:39:12.565757 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cb0ba6e7-b9c9-4e2f-93ea-1ca02d5e6fd8 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:39:12.566217 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cb0ba6e7-b9c9-4e2f-93ea-1ca02d5e6fd8 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:39:12.566999 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cb0ba6e7-b9c9-4e2f-93ea-1ca02d5e6fd8 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:39:12.567465 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cb0ba6e7-b9c9-4e2f-93ea-1ca02d5e6fd8 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:39:12.810927 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-cb0ba6e7-b9c9-4e2f-93ea-1ca02d5e6fd8 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '62209b9802ab46c3b3d094d4d1ad86fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b533f85aeddf4d8db05f2c53a61befc3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:39:12.818973 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-cb0ba6e7-b9c9-4e2f-93ea-1ca02d5e6fd8 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] 10.222.0.16 "GET /compute/v2.1/servers/8f28a606-c8d6-44cf-89d8-d856f3e8cfea" status: 200 len: 1203 microversion: 2.1 time: 0.264973 Jul 16 21:39:12.819897 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2639/5286] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:39:12 2022] GET /compute/v2.1/servers/8f28a606-c8d6-44cf-89d8-d856f3e8cfea => generated 1203 bytes in 267 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:39:13.836602 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-a68cf0a6-e731-4751-8854-b6516f4cc86f tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:39:13.837282 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-a68cf0a6-e731-4751-8854-b6516f4cc86f tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] [instance: 8f28a606-c8d6-44cf-89d8-d856f3e8cfea] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:39:13.847093 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a68cf0a6-e731-4751-8854-b6516f4cc86f tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:39:13.847878 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a68cf0a6-e731-4751-8854-b6516f4cc86f tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:39:13.848772 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a68cf0a6-e731-4751-8854-b6516f4cc86f tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:39:13.849771 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a68cf0a6-e731-4751-8854-b6516f4cc86f tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:39:14.070637 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-a68cf0a6-e731-4751-8854-b6516f4cc86f tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '62209b9802ab46c3b3d094d4d1ad86fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b533f85aeddf4d8db05f2c53a61befc3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:39:14.078502 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-a68cf0a6-e731-4751-8854-b6516f4cc86f tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] 10.222.0.16 "GET /compute/v2.1/servers/8f28a606-c8d6-44cf-89d8-d856f3e8cfea" status: 200 len: 1203 microversion: 2.1 time: 0.245886 Jul 16 21:39:14.079104 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2648/5287] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:39:13 2022] GET /compute/v2.1/servers/8f28a606-c8d6-44cf-89d8-d856f3e8cfea => generated 1203 bytes in 247 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:39:14.511503 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-02c678eb-d924-4e5a-b68a-168a1db453ea req-2dfa1042-580d-4be3-a40f-1a1d9703b209 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "8f28a606-c8d6-44cf-89d8-d856f3e8cfea", "name": "network-vif-plugged", "status": "completed", "tag": "5825fe15-c9bd-4ee3-89d8-bb8729347983"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:39:14.524781 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-02c678eb-d924-4e5a-b68a-168a1db453ea req-2dfa1042-580d-4be3-a40f-1a1d9703b209 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:39:14.525346 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-02c678eb-d924-4e5a-b68a-168a1db453ea req-2dfa1042-580d-4be3-a40f-1a1d9703b209 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:39:14.538747 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-02c678eb-d924-4e5a-b68a-168a1db453ea req-2dfa1042-580d-4be3-a40f-1a1d9703b209 service nova] Creating event network-vif-plugged:5825fe15-c9bd-4ee3-89d8-bb8729347983 for instance 8f28a606-c8d6-44cf-89d8-d856f3e8cfea on nch1-master-1 Jul 16 21:39:14.545845 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-02c678eb-d924-4e5a-b68a-168a1db453ea req-2dfa1042-580d-4be3-a40f-1a1d9703b209 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.038845 Jul 16 21:39:14.546525 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2640/5288] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:39:14 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:39:15.097529 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-2ab7b529-0995-4d58-be1a-02470cb66914 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:39:15.098102 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-2ab7b529-0995-4d58-be1a-02470cb66914 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] [instance: 8f28a606-c8d6-44cf-89d8-d856f3e8cfea] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:39:15.105318 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2ab7b529-0995-4d58-be1a-02470cb66914 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:39:15.107444 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2ab7b529-0995-4d58-be1a-02470cb66914 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:39:15.108297 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2ab7b529-0995-4d58-be1a-02470cb66914 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:39:15.108805 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2ab7b529-0995-4d58-be1a-02470cb66914 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:39:15.290401 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-2ab7b529-0995-4d58-be1a-02470cb66914 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '62209b9802ab46c3b3d094d4d1ad86fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b533f85aeddf4d8db05f2c53a61befc3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:39:15.298370 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-2ab7b529-0995-4d58-be1a-02470cb66914 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] 10.222.0.16 "GET /compute/v2.1/servers/8f28a606-c8d6-44cf-89d8-d856f3e8cfea" status: 200 len: 1203 microversion: 2.1 time: 0.205196 Jul 16 21:39:15.299029 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2649/5289] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:39:15 2022] GET /compute/v2.1/servers/8f28a606-c8d6-44cf-89d8-d856f3e8cfea => generated 1203 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:39:16.314964 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-87c8631e-85b3-4f9f-a88d-f6f026afcbef tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:39:16.315715 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-87c8631e-85b3-4f9f-a88d-f6f026afcbef tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] [instance: 8f28a606-c8d6-44cf-89d8-d856f3e8cfea] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:39:16.323142 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-87c8631e-85b3-4f9f-a88d-f6f026afcbef tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:39:16.323635 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-87c8631e-85b3-4f9f-a88d-f6f026afcbef tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:39:16.324417 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-87c8631e-85b3-4f9f-a88d-f6f026afcbef tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:39:16.324874 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-87c8631e-85b3-4f9f-a88d-f6f026afcbef tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:39:16.464819 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-87c8631e-85b3-4f9f-a88d-f6f026afcbef tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '62209b9802ab46c3b3d094d4d1ad86fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b533f85aeddf4d8db05f2c53a61befc3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:39:16.483227 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-87c8631e-85b3-4f9f-a88d-f6f026afcbef tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] 10.222.0.16 "GET /compute/v2.1/servers/8f28a606-c8d6-44cf-89d8-d856f3e8cfea" status: 200 len: 1391 microversion: 2.1 time: 0.172282 Jul 16 21:39:16.483855 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2641/5290] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:39:16 2022] GET /compute/v2.1/servers/8f28a606-c8d6-44cf-89d8-d856f3e8cfea => generated 1391 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:39:16.499883 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-094778a1-a85e-47ee-aa19-8ad30244d3f8 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:39:16.501016 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-094778a1-a85e-47ee-aa19-8ad30244d3f8 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] [instance: 8f28a606-c8d6-44cf-89d8-d856f3e8cfea] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:39:16.508264 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-094778a1-a85e-47ee-aa19-8ad30244d3f8 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:39:16.508758 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-094778a1-a85e-47ee-aa19-8ad30244d3f8 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:39:16.509851 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-094778a1-a85e-47ee-aa19-8ad30244d3f8 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:39:16.510446 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-094778a1-a85e-47ee-aa19-8ad30244d3f8 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:39:16.659296 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-094778a1-a85e-47ee-aa19-8ad30244d3f8 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '62209b9802ab46c3b3d094d4d1ad86fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b533f85aeddf4d8db05f2c53a61befc3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:39:16.667224 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-094778a1-a85e-47ee-aa19-8ad30244d3f8 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] 10.222.0.16 "GET /compute/v2.1/servers/8f28a606-c8d6-44cf-89d8-d856f3e8cfea" status: 200 len: 1391 microversion: 2.1 time: 0.171343 Jul 16 21:39:16.667800 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2650/5291] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:39:16 2022] GET /compute/v2.1/servers/8f28a606-c8d6-44cf-89d8-d856f3e8cfea => generated 1391 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:39:16.683570 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-f3f4a7ac-5cc8-435f-be6a-6e33e64d2d2f tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:39:16.684204 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-f3f4a7ac-5cc8-435f-be6a-6e33e64d2d2f tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] [instance: 8f28a606-c8d6-44cf-89d8-d856f3e8cfea] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:39:16.690464 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f3f4a7ac-5cc8-435f-be6a-6e33e64d2d2f tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:39:16.698575 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f3f4a7ac-5cc8-435f-be6a-6e33e64d2d2f tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:39:16.698575 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f3f4a7ac-5cc8-435f-be6a-6e33e64d2d2f tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:39:16.698575 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f3f4a7ac-5cc8-435f-be6a-6e33e64d2d2f tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:39:16.836621 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-f3f4a7ac-5cc8-435f-be6a-6e33e64d2d2f tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '62209b9802ab46c3b3d094d4d1ad86fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b533f85aeddf4d8db05f2c53a61befc3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:39:16.846299 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-f3f4a7ac-5cc8-435f-be6a-6e33e64d2d2f tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] 10.222.0.16 "GET /compute/v2.1/servers/8f28a606-c8d6-44cf-89d8-d856f3e8cfea" status: 200 len: 1391 microversion: 2.1 time: 0.165488 Jul 16 21:39:16.846299 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2642/5292] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:39:16 2022] GET /compute/v2.1/servers/8f28a606-c8d6-44cf-89d8-d856f3e8cfea => generated 1391 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:39:16.928660 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-a1fd944b-fabe-4725-8327-85662f0357bf tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:39:16.929163 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-a1fd944b-fabe-4725-8327-85662f0357bf tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] [instance: 8f28a606-c8d6-44cf-89d8-d856f3e8cfea] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:39:16.936282 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a1fd944b-fabe-4725-8327-85662f0357bf tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:39:16.936743 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a1fd944b-fabe-4725-8327-85662f0357bf tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:39:16.937570 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a1fd944b-fabe-4725-8327-85662f0357bf tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:39:16.937994 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a1fd944b-fabe-4725-8327-85662f0357bf tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:39:16.963856 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-a1fd944b-fabe-4725-8327-85662f0357bf tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] [instance: 8f28a606-c8d6-44cf-89d8-d856f3e8cfea] Going to try to terminate instance {{(pid=138681) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 16 21:39:17.296293 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-a1fd944b-fabe-4725-8327-85662f0357bf tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] 10.222.0.16 "DELETE /compute/v2.1/servers/8f28a606-c8d6-44cf-89d8-d856f3e8cfea" status: 204 len: 0 microversion: 2.1 time: 0.371118 Jul 16 21:39:17.296872 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2651/5293] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:39:16 2022] DELETE /compute/v2.1/servers/8f28a606-c8d6-44cf-89d8-d856f3e8cfea => generated 0 bytes in 373 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 16 21:39:17.305646 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-bdc19dc7-5c3d-494f-bcbe-ed4513d301dc tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:39:17.306524 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-bdc19dc7-5c3d-494f-bcbe-ed4513d301dc tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] [instance: 8f28a606-c8d6-44cf-89d8-d856f3e8cfea] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:39:17.314060 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-bdc19dc7-5c3d-494f-bcbe-ed4513d301dc tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:39:17.314516 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-bdc19dc7-5c3d-494f-bcbe-ed4513d301dc tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:39:17.315321 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-bdc19dc7-5c3d-494f-bcbe-ed4513d301dc tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:39:17.315895 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-bdc19dc7-5c3d-494f-bcbe-ed4513d301dc tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:39:17.457472 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-bdc19dc7-5c3d-494f-bcbe-ed4513d301dc tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '62209b9802ab46c3b3d094d4d1ad86fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b533f85aeddf4d8db05f2c53a61befc3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:39:17.466576 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-bdc19dc7-5c3d-494f-bcbe-ed4513d301dc tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] 10.222.0.16 "GET /compute/v2.1/servers/8f28a606-c8d6-44cf-89d8-d856f3e8cfea" status: 200 len: 1397 microversion: 2.1 time: 0.165239 Jul 16 21:39:17.467411 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2643/5294] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:39:17 2022] GET /compute/v2.1/servers/8f28a606-c8d6-44cf-89d8-d856f3e8cfea => generated 1397 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:39:18.484801 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-d0f9379f-c55e-40b9-a0d4-aab630e39440 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:39:18.485516 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-d0f9379f-c55e-40b9-a0d4-aab630e39440 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] [instance: 8f28a606-c8d6-44cf-89d8-d856f3e8cfea] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:39:18.492846 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d0f9379f-c55e-40b9-a0d4-aab630e39440 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:39:18.493303 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d0f9379f-c55e-40b9-a0d4-aab630e39440 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:39:18.494085 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d0f9379f-c55e-40b9-a0d4-aab630e39440 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:39:18.494532 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d0f9379f-c55e-40b9-a0d4-aab630e39440 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:39:18.634870 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-d0f9379f-c55e-40b9-a0d4-aab630e39440 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '62209b9802ab46c3b3d094d4d1ad86fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b533f85aeddf4d8db05f2c53a61befc3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:39:18.643540 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-d0f9379f-c55e-40b9-a0d4-aab630e39440 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] 10.222.0.16 "GET /compute/v2.1/servers/8f28a606-c8d6-44cf-89d8-d856f3e8cfea" status: 200 len: 1397 microversion: 2.1 time: 0.163420 Jul 16 21:39:18.644110 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2652/5295] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:39:18 2022] GET /compute/v2.1/servers/8f28a606-c8d6-44cf-89d8-d856f3e8cfea => generated 1397 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:39:19.326066 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-906753da-a8fe-4b65-b608-3bfd7a67f5c4 req-e8a3a474-0599-4224-8aee-70289d35ba83 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "8f28a606-c8d6-44cf-89d8-d856f3e8cfea", "name": "network-vif-deleted", "tag": "5825fe15-c9bd-4ee3-89d8-bb8729347983"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:39:19.335731 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-906753da-a8fe-4b65-b608-3bfd7a67f5c4 req-e8a3a474-0599-4224-8aee-70289d35ba83 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:39:19.336290 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-906753da-a8fe-4b65-b608-3bfd7a67f5c4 req-e8a3a474-0599-4224-8aee-70289d35ba83 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:39:19.348321 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-906753da-a8fe-4b65-b608-3bfd7a67f5c4 req-e8a3a474-0599-4224-8aee-70289d35ba83 service nova] Creating event network-vif-deleted:5825fe15-c9bd-4ee3-89d8-bb8729347983 for instance 8f28a606-c8d6-44cf-89d8-d856f3e8cfea on nch1-master-1 Jul 16 21:39:19.354079 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-906753da-a8fe-4b65-b608-3bfd7a67f5c4 req-e8a3a474-0599-4224-8aee-70289d35ba83 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.036278 Jul 16 21:39:19.354707 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2644/5296] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:39:19 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 16 21:39:19.464648 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-b41376fc-a13c-47ab-88b9-c3b55cbee2b8 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Action: 'action', calling method: >, body: {"os-getConsoleOutput": {}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:39:19.464648 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-b41376fc-a13c-47ab-88b9-c3b55cbee2b8 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] [instance: 2ce56b72-9fe8-41ca-bd6e-31591ef3b494] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:39:19.469624 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b41376fc-a13c-47ab-88b9-c3b55cbee2b8 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:39:19.470078 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b41376fc-a13c-47ab-88b9-c3b55cbee2b8 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:39:19.470840 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b41376fc-a13c-47ab-88b9-c3b55cbee2b8 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:39:19.471369 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b41376fc-a13c-47ab-88b9-c3b55cbee2b8 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:39:19.531514 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-b41376fc-a13c-47ab-88b9-c3b55cbee2b8 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] 10.222.0.16 "POST /compute/v2.1/servers/2ce56b72-9fe8-41ca-bd6e-31591ef3b494/action" status: 200 len: 24558 microversion: 2.1 time: 0.074898 Jul 16 21:39:19.532173 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2653/5297] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 21:39:19 2022] POST /compute/v2.1/servers/2ce56b72-9fe8-41ca-bd6e-31591ef3b494/action => generated 24558 bytes in 77 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Jul 16 21:39:19.664806 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-4d3fdc93-7dbe-4fa1-99d6-8cb55f1c0dcb tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:39:19.664806 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-4d3fdc93-7dbe-4fa1-99d6-8cb55f1c0dcb tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] [instance: 8f28a606-c8d6-44cf-89d8-d856f3e8cfea] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:39:19.668467 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4d3fdc93-7dbe-4fa1-99d6-8cb55f1c0dcb tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:39:19.669050 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4d3fdc93-7dbe-4fa1-99d6-8cb55f1c0dcb tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:39:19.669908 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4d3fdc93-7dbe-4fa1-99d6-8cb55f1c0dcb tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:39:19.670349 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4d3fdc93-7dbe-4fa1-99d6-8cb55f1c0dcb tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:39:19.734790 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-4d3fdc93-7dbe-4fa1-99d6-8cb55f1c0dcb tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '62209b9802ab46c3b3d094d4d1ad86fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b533f85aeddf4d8db05f2c53a61befc3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:39:19.742476 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-4d3fdc93-7dbe-4fa1-99d6-8cb55f1c0dcb tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] 10.222.0.16 "GET /compute/v2.1/servers/8f28a606-c8d6-44cf-89d8-d856f3e8cfea" status: 200 len: 1188 microversion: 2.1 time: 0.085705 Jul 16 21:39:19.743062 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2645/5298] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:39:19 2022] GET /compute/v2.1/servers/8f28a606-c8d6-44cf-89d8-d856f3e8cfea => generated 1188 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:39:20.760295 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-f32de51c-96c6-445c-b211-517d7c05179a tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:39:20.760927 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-f32de51c-96c6-445c-b211-517d7c05179a tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] [instance: 8f28a606-c8d6-44cf-89d8-d856f3e8cfea] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:39:20.769387 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f32de51c-96c6-445c-b211-517d7c05179a tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:39:20.769848 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f32de51c-96c6-445c-b211-517d7c05179a tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:39:20.770629 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f32de51c-96c6-445c-b211-517d7c05179a tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:39:20.771102 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f32de51c-96c6-445c-b211-517d7c05179a tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:39:20.786130 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-f32de51c-96c6-445c-b211-517d7c05179a tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] HTTP exception thrown: Instance 8f28a606-c8d6-44cf-89d8-d856f3e8cfea could not be found. Jul 16 21:39:20.787809 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-f32de51c-96c6-445c-b211-517d7c05179a tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Request method failure captured: Jul 16 21:39:20.787809 ncdv-master-1 devstack@n-api.service[138681]: request: GET /compute/v2.1/servers/8f28a606-c8d6-44cf-89d8-d856f3e8cfea HTTP/1.1 Jul 16 21:39:20.787809 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:39:20.787809 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:39:20.787809 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:39:20.787809 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 21:39:20.787809 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:39:20.787809 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:39:20.787809 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:39:20.787809 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:39:20.787809 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0y_WdH7eI0DbXzeyrxNR2zaPLU2Z0T2hgcgJaNL5zKj0IOkua_mwRXGhv8UgaywTvIB56iq06rxK294Knh0aCrbToiJY0kPn8yPuGVzddBFopXnN-hL_u7FbpY8zvcJRiL0zpWIKlPhQExzLv95jQK7KF6D9Y-KawL2vbM6jbQCfNnU Jul 16 21:39:20.787809 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:39:20.787809 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:39:20.787809 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:39:20.787809 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:39:20.787809 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:39:20.787809 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:39:20.787809 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: b533f85aeddf4d8db05f2c53a61befc3 Jul 16 21:39:20.787809 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-TestVolumeBootPattern-1625426921 Jul 16 21:39:20.787809 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:39:20.787809 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:39:20.787809 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/b533f85aeddf4d8db05f2c53a61befc3"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/b533f85aeddf4d8db05f2c53a61befc3"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_b533f85aeddf4d8db05f2c53a61befc3"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/b533f85aeddf4d8db05f2c53a61befc3"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:39:20.787809 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-TestVolumeBootPattern-1625426921 Jul 16 21:39:20.787809 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: b533f85aeddf4d8db05f2c53a61befc3 Jul 16 21:39:20.787809 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-TestVolumeBootPattern-1625426921 Jul 16 21:39:20.787809 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-TestVolumeBootPattern-1625426921-project Jul 16 21:39:20.787809 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:39:20.787809 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:39:20.787809 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: 62209b9802ab46c3b3d094d4d1ad86fe Jul 16 21:39:20.787809 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-TestVolumeBootPattern-1625426921-project Jul 16 21:39:20.787809 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:39:20.787809 ncdv-master-1 devstack@n-api.service[138681]: exception: Instance 8f28a606-c8d6-44cf-89d8-d856f3e8cfea could not be found. Jul 16 21:39:20.787809 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': '8f28a606-c8d6-44cf-89d8-d856f3e8cfea'} Jul 16 21:39:20.787809 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:39:20.787809 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:39:20.787809 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:39:20.787809 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:39:20.787809 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:39:20.787809 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:39:20.787809 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:39:20.787809 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:39:20.787809 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:39:20.787809 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:39:20.787809 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 8f28a606-c8d6-44cf-89d8-d856f3e8cfea could not be found. Jul 16 21:39:20.787809 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:39:20.787809 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:39:20.787809 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:39:20.787809 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:39:20.787809 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:39:20.787809 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:39:20.787809 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:39:20.787809 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:39:20.787809 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:39:20.787809 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:39:20.787809 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:39:20.787809 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:39:20.787809 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:39:20.787809 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:39:20.787809 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:39:20.787809 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:39:20.787809 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 8f28a606-c8d6-44cf-89d8-d856f3e8cfea could not be found. Jul 16 21:39:20.787809 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:39:20.787809 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:39:20.787809 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:39:20.792730 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:39:20.792730 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:39:20.792730 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:39:20.792730 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:39:20.792730 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:39:20.792730 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 8f28a606-c8d6-44cf-89d8-d856f3e8cfea could not be found. Jul 16 21:39:20.792730 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:39:20.792730 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-f32de51c-96c6-445c-b211-517d7c05179a tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Returning 404 to user: Instance 8f28a606-c8d6-44cf-89d8-d856f3e8cfea could not be found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:39:20.792730 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-f32de51c-96c6-445c-b211-517d7c05179a tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] 10.222.0.16 "GET /compute/v2.1/servers/8f28a606-c8d6-44cf-89d8-d856f3e8cfea" status: 404 len: 111 microversion: 2.1 time: 0.033399 Jul 16 21:39:20.792730 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2654/5299] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:39:20 2022] GET /compute/v2.1/servers/8f28a606-c8d6-44cf-89d8-d856f3e8cfea => generated 111 bytes in 35 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:39:26.087377 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-b82e34ee-31a2-4697-be4f-97b73fbac342 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:39:26.087917 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-b82e34ee-31a2-4697-be4f-97b73fbac342 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] [instance: 8f28a606-c8d6-44cf-89d8-d856f3e8cfea] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:39:26.095210 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b82e34ee-31a2-4697-be4f-97b73fbac342 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:39:26.095669 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b82e34ee-31a2-4697-be4f-97b73fbac342 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:39:26.100612 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b82e34ee-31a2-4697-be4f-97b73fbac342 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:39:26.100612 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b82e34ee-31a2-4697-be4f-97b73fbac342 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:39:26.110484 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-b82e34ee-31a2-4697-be4f-97b73fbac342 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] HTTP exception thrown: Instance 8f28a606-c8d6-44cf-89d8-d856f3e8cfea could not be found. Jul 16 21:39:26.112220 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-b82e34ee-31a2-4697-be4f-97b73fbac342 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Request method failure captured: Jul 16 21:39:26.112220 ncdv-master-1 devstack@n-api.service[138680]: request: DELETE /compute/v2.1/servers/8f28a606-c8d6-44cf-89d8-d856f3e8cfea HTTP/1.1 Jul 16 21:39:26.112220 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:39:26.112220 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:39:26.112220 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:39:26.112220 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 0 Jul 16 21:39:26.112220 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:39:26.112220 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:39:26.112220 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:39:26.112220 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:39:26.112220 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0y_WdH7eI0DbXzeyrxNR2zaPLU2Z0T2hgcgJaNL5zKj0IOkua_mwRXGhv8UgaywTvIB56iq06rxK294Knh0aCrbToiJY0kPn8yPuGVzddBFopXnN-hL_u7FbpY8zvcJRiL0zpWIKlPhQExzLv95jQK7KF6D9Y-KawL2vbM6jbQCfNnU Jul 16 21:39:26.112220 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:39:26.112220 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:39:26.112220 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:39:26.112220 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:39:26.112220 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:39:26.112220 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:39:26.112220 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: b533f85aeddf4d8db05f2c53a61befc3 Jul 16 21:39:26.112220 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-TestVolumeBootPattern-1625426921 Jul 16 21:39:26.112220 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:39:26.112220 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:39:26.112220 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/b533f85aeddf4d8db05f2c53a61befc3"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/b533f85aeddf4d8db05f2c53a61befc3"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_b533f85aeddf4d8db05f2c53a61befc3"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/b533f85aeddf4d8db05f2c53a61befc3"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:39:26.112220 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-TestVolumeBootPattern-1625426921 Jul 16 21:39:26.112220 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: b533f85aeddf4d8db05f2c53a61befc3 Jul 16 21:39:26.112220 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-TestVolumeBootPattern-1625426921 Jul 16 21:39:26.112220 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-TestVolumeBootPattern-1625426921-project Jul 16 21:39:26.112220 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:39:26.112220 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:39:26.112220 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: 62209b9802ab46c3b3d094d4d1ad86fe Jul 16 21:39:26.112220 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-TestVolumeBootPattern-1625426921-project Jul 16 21:39:26.112220 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 21:39:26.112220 ncdv-master-1 devstack@n-api.service[138680]: exception: Instance 8f28a606-c8d6-44cf-89d8-d856f3e8cfea could not be found. Jul 16 21:39:26.112220 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': '8f28a606-c8d6-44cf-89d8-d856f3e8cfea'} Jul 16 21:39:26.112220 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:39:26.112220 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:39:26.112220 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:39:26.112220 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:39:26.112220 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:39:26.112220 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:39:26.112220 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:39:26.112220 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:39:26.112220 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:39:26.112220 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:39:26.112220 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 8f28a606-c8d6-44cf-89d8-d856f3e8cfea could not be found. Jul 16 21:39:26.112220 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:39:26.112220 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:39:26.112220 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:39:26.112220 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:39:26.112220 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:39:26.112220 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:39:26.112220 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:39:26.112220 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:39:26.112220 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:39:26.112220 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:39:26.112220 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Jul 16 21:39:26.112220 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Jul 16 21:39:26.112220 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Jul 16 21:39:26.112220 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Jul 16 21:39:26.112220 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:39:26.112220 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:39:26.112220 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:39:26.112220 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:39:26.116586 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 8f28a606-c8d6-44cf-89d8-d856f3e8cfea could not be found. Jul 16 21:39:26.116586 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:39:26.116586 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:39:26.116586 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:39:26.116586 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:39:26.116586 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:39:26.116586 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:39:26.116586 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:39:26.116586 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:39:26.116586 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 8f28a606-c8d6-44cf-89d8-d856f3e8cfea could not be found. Jul 16 21:39:26.116586 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:39:26.116586 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-b82e34ee-31a2-4697-be4f-97b73fbac342 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Returning 404 to user: Instance 8f28a606-c8d6-44cf-89d8-d856f3e8cfea could not be found. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:39:26.116586 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-b82e34ee-31a2-4697-be4f-97b73fbac342 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] 10.222.0.16 "DELETE /compute/v2.1/servers/8f28a606-c8d6-44cf-89d8-d856f3e8cfea" status: 404 len: 111 microversion: 2.1 time: 0.030180 Jul 16 21:39:26.116586 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2646/5300] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:39:26 2022] DELETE /compute/v2.1/servers/8f28a606-c8d6-44cf-89d8-d856f3e8cfea => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:39:26.122580 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-f60c8877-2f6c-4aa2-b2fc-0df90462a34c tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:39:26.123632 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-f60c8877-2f6c-4aa2-b2fc-0df90462a34c tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] [instance: 8f28a606-c8d6-44cf-89d8-d856f3e8cfea] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:39:26.131711 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f60c8877-2f6c-4aa2-b2fc-0df90462a34c tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:39:26.132164 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f60c8877-2f6c-4aa2-b2fc-0df90462a34c tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:39:26.132928 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f60c8877-2f6c-4aa2-b2fc-0df90462a34c tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:39:26.133358 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f60c8877-2f6c-4aa2-b2fc-0df90462a34c tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:39:26.147386 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-f60c8877-2f6c-4aa2-b2fc-0df90462a34c tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] HTTP exception thrown: Instance 8f28a606-c8d6-44cf-89d8-d856f3e8cfea could not be found. Jul 16 21:39:26.148954 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-f60c8877-2f6c-4aa2-b2fc-0df90462a34c tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Request method failure captured: Jul 16 21:39:26.148954 ncdv-master-1 devstack@n-api.service[138681]: request: GET /compute/v2.1/servers/8f28a606-c8d6-44cf-89d8-d856f3e8cfea HTTP/1.1 Jul 16 21:39:26.148954 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:39:26.148954 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:39:26.148954 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:39:26.148954 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 21:39:26.148954 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:39:26.148954 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:39:26.148954 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:39:26.148954 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:39:26.148954 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0y_WdH7eI0DbXzeyrxNR2zaPLU2Z0T2hgcgJaNL5zKj0IOkua_mwRXGhv8UgaywTvIB56iq06rxK294Knh0aCrbToiJY0kPn8yPuGVzddBFopXnN-hL_u7FbpY8zvcJRiL0zpWIKlPhQExzLv95jQK7KF6D9Y-KawL2vbM6jbQCfNnU Jul 16 21:39:26.148954 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:39:26.148954 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:39:26.148954 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:39:26.148954 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:39:26.148954 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:39:26.148954 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:39:26.148954 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: b533f85aeddf4d8db05f2c53a61befc3 Jul 16 21:39:26.148954 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-TestVolumeBootPattern-1625426921 Jul 16 21:39:26.148954 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:39:26.148954 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:39:26.148954 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/b533f85aeddf4d8db05f2c53a61befc3"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/b533f85aeddf4d8db05f2c53a61befc3"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_b533f85aeddf4d8db05f2c53a61befc3"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/b533f85aeddf4d8db05f2c53a61befc3"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:39:26.148954 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-TestVolumeBootPattern-1625426921 Jul 16 21:39:26.148954 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: b533f85aeddf4d8db05f2c53a61befc3 Jul 16 21:39:26.148954 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-TestVolumeBootPattern-1625426921 Jul 16 21:39:26.148954 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-TestVolumeBootPattern-1625426921-project Jul 16 21:39:26.148954 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:39:26.148954 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:39:26.148954 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: 62209b9802ab46c3b3d094d4d1ad86fe Jul 16 21:39:26.148954 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-TestVolumeBootPattern-1625426921-project Jul 16 21:39:26.148954 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:39:26.148954 ncdv-master-1 devstack@n-api.service[138681]: exception: Instance 8f28a606-c8d6-44cf-89d8-d856f3e8cfea could not be found. Jul 16 21:39:26.148954 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': '8f28a606-c8d6-44cf-89d8-d856f3e8cfea'} Jul 16 21:39:26.148954 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:39:26.148954 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:39:26.148954 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:39:26.148954 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:39:26.148954 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:39:26.148954 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:39:26.148954 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:39:26.148954 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:39:26.148954 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:39:26.148954 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:39:26.148954 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 8f28a606-c8d6-44cf-89d8-d856f3e8cfea could not be found. Jul 16 21:39:26.148954 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:39:26.148954 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:39:26.148954 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:39:26.148954 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:39:26.148954 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:39:26.148954 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:39:26.148954 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:39:26.148954 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:39:26.148954 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:39:26.148954 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:39:26.148954 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:39:26.148954 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:39:26.148954 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:39:26.148954 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:39:26.148954 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:39:26.148954 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:39:26.148954 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 8f28a606-c8d6-44cf-89d8-d856f3e8cfea could not be found. Jul 16 21:39:26.148954 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:39:26.148954 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:39:26.148954 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:39:26.154899 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:39:26.154899 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:39:26.154899 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:39:26.154899 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:39:26.154899 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:39:26.154899 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 8f28a606-c8d6-44cf-89d8-d856f3e8cfea could not be found. Jul 16 21:39:26.154899 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:39:26.154899 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-f60c8877-2f6c-4aa2-b2fc-0df90462a34c tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Returning 404 to user: Instance 8f28a606-c8d6-44cf-89d8-d856f3e8cfea could not be found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:39:26.154899 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-f60c8877-2f6c-4aa2-b2fc-0df90462a34c tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] 10.222.0.16 "GET /compute/v2.1/servers/8f28a606-c8d6-44cf-89d8-d856f3e8cfea" status: 404 len: 111 microversion: 2.1 time: 0.031597 Jul 16 21:39:26.154899 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2655/5301] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:39:26 2022] GET /compute/v2.1/servers/8f28a606-c8d6-44cf-89d8-d856f3e8cfea => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Jul 16 21:39:30.545421 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-be1ffed2-0f74-4e56-9a82-7a5eb6c9fb0d tempest-TestNetworkAdvancedServerOps-756251536 tempest-TestNetworkAdvancedServerOps-756251536-project-admin] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:39:30.546035 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-be1ffed2-0f74-4e56-9a82-7a5eb6c9fb0d tempest-TestNetworkAdvancedServerOps-756251536 tempest-TestNetworkAdvancedServerOps-756251536-project-admin] [instance: 2ce56b72-9fe8-41ca-bd6e-31591ef3b494] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:39:30.554198 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-be1ffed2-0f74-4e56-9a82-7a5eb6c9fb0d tempest-TestNetworkAdvancedServerOps-756251536 tempest-TestNetworkAdvancedServerOps-756251536-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:39:30.554648 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-be1ffed2-0f74-4e56-9a82-7a5eb6c9fb0d tempest-TestNetworkAdvancedServerOps-756251536 tempest-TestNetworkAdvancedServerOps-756251536-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:39:30.555435 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-be1ffed2-0f74-4e56-9a82-7a5eb6c9fb0d tempest-TestNetworkAdvancedServerOps-756251536 tempest-TestNetworkAdvancedServerOps-756251536-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:39:30.555878 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-be1ffed2-0f74-4e56-9a82-7a5eb6c9fb0d tempest-TestNetworkAdvancedServerOps-756251536 tempest-TestNetworkAdvancedServerOps-756251536-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:39:30.852749 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-be1ffed2-0f74-4e56-9a82-7a5eb6c9fb0d tempest-TestNetworkAdvancedServerOps-756251536 tempest-TestNetworkAdvancedServerOps-756251536-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/2ce56b72-9fe8-41ca-bd6e-31591ef3b494" status: 200 len: 1833 microversion: 2.1 time: 0.310325 Jul 16 21:39:30.852955 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2647/5302] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:39:30 2022] GET /compute/v2.1/servers/2ce56b72-9fe8-41ca-bd6e-31591ef3b494 => generated 1833 bytes in 312 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:39:30.869791 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-270f90f9-b78a-4216-bbdc-0867dda67c51 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:39:30.870415 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-270f90f9-b78a-4216-bbdc-0867dda67c51 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] [instance: 2ce56b72-9fe8-41ca-bd6e-31591ef3b494] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:39:30.877804 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-270f90f9-b78a-4216-bbdc-0867dda67c51 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:39:30.878413 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-270f90f9-b78a-4216-bbdc-0867dda67c51 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:39:30.879646 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-270f90f9-b78a-4216-bbdc-0867dda67c51 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:39:30.880102 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-270f90f9-b78a-4216-bbdc-0867dda67c51 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:39:31.023017 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-270f90f9-b78a-4216-bbdc-0867dda67c51 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '833136cbdeab4b9e8b6dd2d24c7c8753', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '77c9b7e351e24dc88207e9498f75f578', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:39:31.031190 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-270f90f9-b78a-4216-bbdc-0867dda67c51 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] 10.222.0.16 "GET /compute/v2.1/servers/2ce56b72-9fe8-41ca-bd6e-31591ef3b494" status: 200 len: 1682 microversion: 2.1 time: 0.165199 Jul 16 21:39:31.031908 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2656/5303] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:39:30 2022] GET /compute/v2.1/servers/2ce56b72-9fe8-41ca-bd6e-31591ef3b494 => generated 1682 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:39:31.136166 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-df1a1bec-19ff-4a1c-b8f9-191ef1a82b5d tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Action: 'action', calling method: >, body: {"os-getConsoleOutput": {}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:39:31.137102 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-df1a1bec-19ff-4a1c-b8f9-191ef1a82b5d tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] [instance: 2ce56b72-9fe8-41ca-bd6e-31591ef3b494] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:39:31.143355 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-df1a1bec-19ff-4a1c-b8f9-191ef1a82b5d tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:39:31.144623 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-df1a1bec-19ff-4a1c-b8f9-191ef1a82b5d tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:39:31.145417 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-df1a1bec-19ff-4a1c-b8f9-191ef1a82b5d tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:39:31.145894 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-df1a1bec-19ff-4a1c-b8f9-191ef1a82b5d tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:39:31.198807 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-df1a1bec-19ff-4a1c-b8f9-191ef1a82b5d tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] 10.222.0.16 "POST /compute/v2.1/servers/2ce56b72-9fe8-41ca-bd6e-31591ef3b494/action" status: 200 len: 32636 microversion: 2.1 time: 0.066553 Jul 16 21:39:31.199492 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2648/5304] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 21:39:31 2022] POST /compute/v2.1/servers/2ce56b72-9fe8-41ca-bd6e-31591ef3b494/action => generated 32636 bytes in 68 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 16 21:39:31.438341 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-9599e2d3-f18f-4d8a-8cf7-b893d1489290 tempest-TestNetworkAdvancedServerOps-756251536 tempest-TestNetworkAdvancedServerOps-756251536-project-admin] Action: 'action', calling method: >, body: {"migrate": {}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:39:31.438964 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-9599e2d3-f18f-4d8a-8cf7-b893d1489290 tempest-TestNetworkAdvancedServerOps-756251536 tempest-TestNetworkAdvancedServerOps-756251536-project-admin] [instance: 2ce56b72-9fe8-41ca-bd6e-31591ef3b494] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:39:31.446292 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9599e2d3-f18f-4d8a-8cf7-b893d1489290 tempest-TestNetworkAdvancedServerOps-756251536 tempest-TestNetworkAdvancedServerOps-756251536-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:39:31.446739 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9599e2d3-f18f-4d8a-8cf7-b893d1489290 tempest-TestNetworkAdvancedServerOps-756251536 tempest-TestNetworkAdvancedServerOps-756251536-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:39:31.447528 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9599e2d3-f18f-4d8a-8cf7-b893d1489290 tempest-TestNetworkAdvancedServerOps-756251536 tempest-TestNetworkAdvancedServerOps-756251536-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:39:31.447963 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9599e2d3-f18f-4d8a-8cf7-b893d1489290 tempest-TestNetworkAdvancedServerOps-756251536 tempest-TestNetworkAdvancedServerOps-756251536-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:39:31.536677 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-9599e2d3-f18f-4d8a-8cf7-b893d1489290 tempest-TestNetworkAdvancedServerOps-756251536 tempest-TestNetworkAdvancedServerOps-756251536-project-admin] Policy check for compute:servers:resize:cross_cell failed with credentials {'is_admin': True, 'user_id': '12b39cdbf0db4052b7197313efcc4d21', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8736916be07c4e6886ed5280d4cce420', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:39:31.537357 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-9599e2d3-f18f-4d8a-8cf7-b893d1489290 tempest-TestNetworkAdvancedServerOps-756251536 tempest-TestNetworkAdvancedServerOps-756251536-project-admin] [instance: 2ce56b72-9fe8-41ca-bd6e-31591ef3b494] flavor_id is None. Assuming migration. {{(pid=138681) resize /opt/stack/nova/nova/compute/api.py:4142}} Jul 16 21:39:31.537802 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-9599e2d3-f18f-4d8a-8cf7-b893d1489290 tempest-TestNetworkAdvancedServerOps-756251536 tempest-TestNetworkAdvancedServerOps-756251536-project-admin] [instance: 2ce56b72-9fe8-41ca-bd6e-31591ef3b494] Old instance type m1.heat2, new instance type m1.heat2 {{(pid=138681) resize /opt/stack/nova/nova/compute/api.py:4166}} Jul 16 21:39:31.877691 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:39:32.072644 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-9599e2d3-f18f-4d8a-8cf7-b893d1489290 tempest-TestNetworkAdvancedServerOps-756251536 tempest-TestNetworkAdvancedServerOps-756251536-project-admin] 10.222.0.16 "POST /compute/v2.1/servers/2ce56b72-9fe8-41ca-bd6e-31591ef3b494/action" status: 202 len: 0 microversion: 2.1 time: 0.638132 Jul 16 21:39:32.073559 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2657/5305] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 21:39:31 2022] POST /compute/v2.1/servers/2ce56b72-9fe8-41ca-bd6e-31591ef3b494/action => generated 0 bytes in 640 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 21:39:32.083398 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-1dd677ff-5a58-4ca4-bc42-b6b548e687b1 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:39:32.088665 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-1dd677ff-5a58-4ca4-bc42-b6b548e687b1 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] [instance: 2ce56b72-9fe8-41ca-bd6e-31591ef3b494] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:39:32.098066 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1dd677ff-5a58-4ca4-bc42-b6b548e687b1 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:39:32.098539 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1dd677ff-5a58-4ca4-bc42-b6b548e687b1 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:39:32.099397 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1dd677ff-5a58-4ca4-bc42-b6b548e687b1 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:39:32.100318 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1dd677ff-5a58-4ca4-bc42-b6b548e687b1 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:39:32.106194 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=72 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:39:32.244907 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-1dd677ff-5a58-4ca4-bc42-b6b548e687b1 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '833136cbdeab4b9e8b6dd2d24c7c8753', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '77c9b7e351e24dc88207e9498f75f578', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:39:32.252668 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-1dd677ff-5a58-4ca4-bc42-b6b548e687b1 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] 10.222.0.16 "GET /compute/v2.1/servers/2ce56b72-9fe8-41ca-bd6e-31591ef3b494" status: 200 len: 1691 microversion: 2.1 time: 0.174509 Jul 16 21:39:32.253265 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2649/5306] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:39:32 2022] GET /compute/v2.1/servers/2ce56b72-9fe8-41ca-bd6e-31591ef3b494 => generated 1691 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:39:33.273895 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-89fd6b49-8d1b-4a6b-b95a-d159ee4a33d8 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:39:33.275205 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-89fd6b49-8d1b-4a6b-b95a-d159ee4a33d8 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] [instance: 2ce56b72-9fe8-41ca-bd6e-31591ef3b494] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:39:33.282413 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-89fd6b49-8d1b-4a6b-b95a-d159ee4a33d8 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:39:33.283186 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-89fd6b49-8d1b-4a6b-b95a-d159ee4a33d8 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:39:33.285512 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-89fd6b49-8d1b-4a6b-b95a-d159ee4a33d8 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:39:33.286162 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-89fd6b49-8d1b-4a6b-b95a-d159ee4a33d8 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:39:33.443900 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-89fd6b49-8d1b-4a6b-b95a-d159ee4a33d8 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '833136cbdeab4b9e8b6dd2d24c7c8753', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '77c9b7e351e24dc88207e9498f75f578', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:39:33.452335 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-89fd6b49-8d1b-4a6b-b95a-d159ee4a33d8 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] 10.222.0.16 "GET /compute/v2.1/servers/2ce56b72-9fe8-41ca-bd6e-31591ef3b494" status: 200 len: 1691 microversion: 2.1 time: 0.185167 Jul 16 21:39:33.457738 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2658/5307] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:39:33 2022] GET /compute/v2.1/servers/2ce56b72-9fe8-41ca-bd6e-31591ef3b494 => generated 1691 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:39:34.470497 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-27e2d200-a3af-40a9-bb71-bf6d4bfd9345 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:39:34.471247 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-27e2d200-a3af-40a9-bb71-bf6d4bfd9345 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] [instance: 2ce56b72-9fe8-41ca-bd6e-31591ef3b494] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:39:34.478317 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-27e2d200-a3af-40a9-bb71-bf6d4bfd9345 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:39:34.478782 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-27e2d200-a3af-40a9-bb71-bf6d4bfd9345 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:39:34.479585 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-27e2d200-a3af-40a9-bb71-bf6d4bfd9345 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:39:34.480036 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-27e2d200-a3af-40a9-bb71-bf6d4bfd9345 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:39:34.661718 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-27e2d200-a3af-40a9-bb71-bf6d4bfd9345 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '833136cbdeab4b9e8b6dd2d24c7c8753', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '77c9b7e351e24dc88207e9498f75f578', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:39:34.670391 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-27e2d200-a3af-40a9-bb71-bf6d4bfd9345 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] 10.222.0.16 "GET /compute/v2.1/servers/2ce56b72-9fe8-41ca-bd6e-31591ef3b494" status: 200 len: 1691 microversion: 2.1 time: 0.203613 Jul 16 21:39:34.672022 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2650/5308] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:39:34 2022] GET /compute/v2.1/servers/2ce56b72-9fe8-41ca-bd6e-31591ef3b494 => generated 1691 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:39:35.689637 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-aa4984ed-4682-4f84-b67f-04d203bb6a70 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:39:35.690989 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-aa4984ed-4682-4f84-b67f-04d203bb6a70 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] [instance: 2ce56b72-9fe8-41ca-bd6e-31591ef3b494] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:39:35.698226 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-aa4984ed-4682-4f84-b67f-04d203bb6a70 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:39:35.698683 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-aa4984ed-4682-4f84-b67f-04d203bb6a70 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:39:35.699567 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-aa4984ed-4682-4f84-b67f-04d203bb6a70 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:39:35.700228 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-aa4984ed-4682-4f84-b67f-04d203bb6a70 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:39:35.857757 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-aa4984ed-4682-4f84-b67f-04d203bb6a70 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '833136cbdeab4b9e8b6dd2d24c7c8753', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '77c9b7e351e24dc88207e9498f75f578', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:39:35.867407 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-aa4984ed-4682-4f84-b67f-04d203bb6a70 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] 10.222.0.16 "GET /compute/v2.1/servers/2ce56b72-9fe8-41ca-bd6e-31591ef3b494" status: 200 len: 1696 microversion: 2.1 time: 0.182206 Jul 16 21:39:35.867991 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2659/5309] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:39:35 2022] GET /compute/v2.1/servers/2ce56b72-9fe8-41ca-bd6e-31591ef3b494 => generated 1696 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:39:36.885126 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-b31615d6-df3f-43ed-b5dc-8133159161ad tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:39:36.885749 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-b31615d6-df3f-43ed-b5dc-8133159161ad tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] [instance: 2ce56b72-9fe8-41ca-bd6e-31591ef3b494] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:39:36.893079 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b31615d6-df3f-43ed-b5dc-8133159161ad tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:39:36.893592 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b31615d6-df3f-43ed-b5dc-8133159161ad tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:39:36.894665 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b31615d6-df3f-43ed-b5dc-8133159161ad tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:39:36.895144 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b31615d6-df3f-43ed-b5dc-8133159161ad tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:39:37.072177 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-b31615d6-df3f-43ed-b5dc-8133159161ad tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '833136cbdeab4b9e8b6dd2d24c7c8753', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '77c9b7e351e24dc88207e9498f75f578', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:39:37.079923 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-b31615d6-df3f-43ed-b5dc-8133159161ad tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] 10.222.0.16 "GET /compute/v2.1/servers/2ce56b72-9fe8-41ca-bd6e-31591ef3b494" status: 200 len: 1696 microversion: 2.1 time: 0.198746 Jul 16 21:39:37.080509 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2651/5310] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:39:36 2022] GET /compute/v2.1/servers/2ce56b72-9fe8-41ca-bd6e-31591ef3b494 => generated 1696 bytes in 200 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:39:38.098750 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-735a0f2e-3921-421d-bd43-83175c3db314 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:39:38.098750 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-735a0f2e-3921-421d-bd43-83175c3db314 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] [instance: 2ce56b72-9fe8-41ca-bd6e-31591ef3b494] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:39:38.105243 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-735a0f2e-3921-421d-bd43-83175c3db314 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:39:38.105699 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-735a0f2e-3921-421d-bd43-83175c3db314 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:39:38.106726 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-735a0f2e-3921-421d-bd43-83175c3db314 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:39:38.107212 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-735a0f2e-3921-421d-bd43-83175c3db314 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:39:38.249444 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-735a0f2e-3921-421d-bd43-83175c3db314 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '833136cbdeab4b9e8b6dd2d24c7c8753', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '77c9b7e351e24dc88207e9498f75f578', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:39:38.257061 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-735a0f2e-3921-421d-bd43-83175c3db314 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] 10.222.0.16 "GET /compute/v2.1/servers/2ce56b72-9fe8-41ca-bd6e-31591ef3b494" status: 200 len: 1696 microversion: 2.1 time: 0.163435 Jul 16 21:39:38.257740 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2660/5311] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:39:38 2022] GET /compute/v2.1/servers/2ce56b72-9fe8-41ca-bd6e-31591ef3b494 => generated 1696 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:39:39.274769 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-2bdedf66-6064-44ed-9243-c59ae1144fcd tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:39:39.278801 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-2bdedf66-6064-44ed-9243-c59ae1144fcd tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] [instance: 2ce56b72-9fe8-41ca-bd6e-31591ef3b494] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:39:39.279259 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_api:SELECT=30 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:39:39.284159 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2bdedf66-6064-44ed-9243-c59ae1144fcd tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:39:39.284778 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2bdedf66-6064-44ed-9243-c59ae1144fcd tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:39:39.286197 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2bdedf66-6064-44ed-9243-c59ae1144fcd tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:39:39.287081 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2bdedf66-6064-44ed-9243-c59ae1144fcd tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:39:39.430777 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-2bdedf66-6064-44ed-9243-c59ae1144fcd tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '833136cbdeab4b9e8b6dd2d24c7c8753', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '77c9b7e351e24dc88207e9498f75f578', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:39:39.438445 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-2bdedf66-6064-44ed-9243-c59ae1144fcd tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] 10.222.0.16 "GET /compute/v2.1/servers/2ce56b72-9fe8-41ca-bd6e-31591ef3b494" status: 200 len: 1696 microversion: 2.1 time: 0.168333 Jul 16 21:39:39.439086 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2652/5312] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:39:39 2022] GET /compute/v2.1/servers/2ce56b72-9fe8-41ca-bd6e-31591ef3b494 => generated 1696 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:39:40.456949 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-338e282c-3706-4b03-9227-c8fa0f4aaf42 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:39:40.458987 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-338e282c-3706-4b03-9227-c8fa0f4aaf42 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] [instance: 2ce56b72-9fe8-41ca-bd6e-31591ef3b494] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:39:40.466408 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-338e282c-3706-4b03-9227-c8fa0f4aaf42 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:39:40.466901 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-338e282c-3706-4b03-9227-c8fa0f4aaf42 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:39:40.467801 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-338e282c-3706-4b03-9227-c8fa0f4aaf42 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:39:40.468292 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-338e282c-3706-4b03-9227-c8fa0f4aaf42 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:39:40.613907 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-338e282c-3706-4b03-9227-c8fa0f4aaf42 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '833136cbdeab4b9e8b6dd2d24c7c8753', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '77c9b7e351e24dc88207e9498f75f578', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:39:40.621639 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-338e282c-3706-4b03-9227-c8fa0f4aaf42 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] 10.222.0.16 "GET /compute/v2.1/servers/2ce56b72-9fe8-41ca-bd6e-31591ef3b494" status: 200 len: 1696 microversion: 2.1 time: 0.169345 Jul 16 21:39:40.622204 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2661/5313] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:39:40 2022] GET /compute/v2.1/servers/2ce56b72-9fe8-41ca-bd6e-31591ef3b494 => generated 1696 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:39:41.640191 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-f06786d4-b08e-46be-b324-220982ea5e61 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:39:41.640755 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-f06786d4-b08e-46be-b324-220982ea5e61 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] [instance: 2ce56b72-9fe8-41ca-bd6e-31591ef3b494] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:39:41.648679 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f06786d4-b08e-46be-b324-220982ea5e61 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:39:41.649138 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f06786d4-b08e-46be-b324-220982ea5e61 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:39:41.649909 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f06786d4-b08e-46be-b324-220982ea5e61 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:39:41.650355 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f06786d4-b08e-46be-b324-220982ea5e61 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:39:41.784392 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-f06786d4-b08e-46be-b324-220982ea5e61 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '833136cbdeab4b9e8b6dd2d24c7c8753', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '77c9b7e351e24dc88207e9498f75f578', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:39:41.793053 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-f06786d4-b08e-46be-b324-220982ea5e61 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] 10.222.0.16 "GET /compute/v2.1/servers/2ce56b72-9fe8-41ca-bd6e-31591ef3b494" status: 200 len: 1696 microversion: 2.1 time: 0.156872 Jul 16 21:39:41.793635 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2653/5314] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:39:41 2022] GET /compute/v2.1/servers/2ce56b72-9fe8-41ca-bd6e-31591ef3b494 => generated 1696 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:39:42.811339 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-eaee8183-381c-4174-838e-e4f92389c0a9 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:39:42.812206 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-eaee8183-381c-4174-838e-e4f92389c0a9 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] [instance: 2ce56b72-9fe8-41ca-bd6e-31591ef3b494] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:39:42.815175 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_api:SELECT=37,nova_api:UPDATE=1 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:39:42.820294 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-eaee8183-381c-4174-838e-e4f92389c0a9 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:39:42.820770 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-eaee8183-381c-4174-838e-e4f92389c0a9 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:39:42.821532 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-eaee8183-381c-4174-838e-e4f92389c0a9 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:39:42.821974 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-eaee8183-381c-4174-838e-e4f92389c0a9 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:39:42.963400 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-eaee8183-381c-4174-838e-e4f92389c0a9 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '833136cbdeab4b9e8b6dd2d24c7c8753', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '77c9b7e351e24dc88207e9498f75f578', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:39:42.973500 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-eaee8183-381c-4174-838e-e4f92389c0a9 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] 10.222.0.16 "GET /compute/v2.1/servers/2ce56b72-9fe8-41ca-bd6e-31591ef3b494" status: 200 len: 1696 microversion: 2.1 time: 0.165710 Jul 16 21:39:42.974331 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2662/5315] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:39:42 2022] GET /compute/v2.1/servers/2ce56b72-9fe8-41ca-bd6e-31591ef3b494 => generated 1696 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:39:43.992084 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-61d7f550-9546-4fca-bcfc-911dccfdd1dd tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:39:43.992629 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-61d7f550-9546-4fca-bcfc-911dccfdd1dd tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] [instance: 2ce56b72-9fe8-41ca-bd6e-31591ef3b494] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:39:43.999669 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-61d7f550-9546-4fca-bcfc-911dccfdd1dd tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:39:44.000122 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-61d7f550-9546-4fca-bcfc-911dccfdd1dd tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:39:44.000884 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-61d7f550-9546-4fca-bcfc-911dccfdd1dd tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:39:44.001324 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-61d7f550-9546-4fca-bcfc-911dccfdd1dd tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:39:44.140430 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-61d7f550-9546-4fca-bcfc-911dccfdd1dd tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '833136cbdeab4b9e8b6dd2d24c7c8753', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '77c9b7e351e24dc88207e9498f75f578', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:39:44.148714 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-61d7f550-9546-4fca-bcfc-911dccfdd1dd tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] 10.222.0.16 "GET /compute/v2.1/servers/2ce56b72-9fe8-41ca-bd6e-31591ef3b494" status: 200 len: 1696 microversion: 2.1 time: 0.161154 Jul 16 21:39:44.149281 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2654/5316] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:39:43 2022] GET /compute/v2.1/servers/2ce56b72-9fe8-41ca-bd6e-31591ef3b494 => generated 1696 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:39:45.165865 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-b314aba3-3c90-4cd8-9586-107abf42ce90 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:39:45.166742 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-b314aba3-3c90-4cd8-9586-107abf42ce90 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] [instance: 2ce56b72-9fe8-41ca-bd6e-31591ef3b494] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:39:45.173799 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b314aba3-3c90-4cd8-9586-107abf42ce90 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:39:45.177846 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b314aba3-3c90-4cd8-9586-107abf42ce90 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:39:45.179137 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b314aba3-3c90-4cd8-9586-107abf42ce90 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:39:45.179823 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b314aba3-3c90-4cd8-9586-107abf42ce90 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:39:45.324510 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-b314aba3-3c90-4cd8-9586-107abf42ce90 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '833136cbdeab4b9e8b6dd2d24c7c8753', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '77c9b7e351e24dc88207e9498f75f578', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:39:45.332734 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-b314aba3-3c90-4cd8-9586-107abf42ce90 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] 10.222.0.16 "GET /compute/v2.1/servers/2ce56b72-9fe8-41ca-bd6e-31591ef3b494" status: 200 len: 1696 microversion: 2.1 time: 0.170767 Jul 16 21:39:45.333467 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2663/5317] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:39:45 2022] GET /compute/v2.1/servers/2ce56b72-9fe8-41ca-bd6e-31591ef3b494 => generated 1696 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:39:46.305263 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-5deb236a-d39c-46d5-9195-c61e16f4d607 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestVolumeBootPattern-volume-backed-server-1494937890", "imageRef": "", "flavorRef": "452", "block_device_mapping_v2": [{"uuid": "cb83e900-2518-45c5-9a34-d536cf6c7a3f", "source_type": "volume", "destination_type": "volume", "boot_index": 0, "delete_on_termination": true}], "networks": [{"uuid": "94c922c8-150a-417e-8274-7473d6567b26"}]}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:39:46.350417 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-29be37ef-512d-4bdc-abab-f15cf4fa8974 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:39:46.351066 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-29be37ef-512d-4bdc-abab-f15cf4fa8974 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] [instance: 2ce56b72-9fe8-41ca-bd6e-31591ef3b494] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:39:46.365413 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-29be37ef-512d-4bdc-abab-f15cf4fa8974 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:39:46.365413 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-29be37ef-512d-4bdc-abab-f15cf4fa8974 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:39:46.365413 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-29be37ef-512d-4bdc-abab-f15cf4fa8974 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:39:46.365413 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-29be37ef-512d-4bdc-abab-f15cf4fa8974 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:39:46.371341 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.network.neutron [None req-5deb236a-d39c-46d5-9195-c61e16f4d607 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] validate_networks() for [('94c922c8-150a-417e-8274-7473d6567b26', None, None, None, None, None)] {{(pid=138680) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 16 21:39:46.578830 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-29be37ef-512d-4bdc-abab-f15cf4fa8974 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '833136cbdeab4b9e8b6dd2d24c7c8753', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '77c9b7e351e24dc88207e9498f75f578', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:39:46.588578 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-29be37ef-512d-4bdc-abab-f15cf4fa8974 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] 10.222.0.16 "GET /compute/v2.1/servers/2ce56b72-9fe8-41ca-bd6e-31591ef3b494" status: 200 len: 1696 microversion: 2.1 time: 0.242793 Jul 16 21:39:46.589165 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2664/5318] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:39:46 2022] GET /compute/v2.1/servers/2ce56b72-9fe8-41ca-bd6e-31591ef3b494 => generated 1696 bytes in 244 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:39:47.144239 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-5deb236a-d39c-46d5-9195-c61e16f4d607 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Flavor limits 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 16 21:39:47.144700 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-5deb236a-d39c-46d5-9195-c61e16f4d607 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Image limits 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 16 21:39:47.145193 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-5deb236a-d39c-46d5-9195-c61e16f4d607 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Flavor pref 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 16 21:39:47.145660 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-5deb236a-d39c-46d5-9195-c61e16f4d607 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Image pref 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 16 21:39:47.146125 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-5deb236a-d39c-46d5-9195-c61e16f4d607 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 16 21:39:47.155185 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5deb236a-d39c-46d5-9195-c61e16f4d607 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:39:47.162466 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5deb236a-d39c-46d5-9195-c61e16f4d607 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:39:47.162466 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5deb236a-d39c-46d5-9195-c61e16f4d607 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:39:47.162466 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5deb236a-d39c-46d5-9195-c61e16f4d607 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:39:47.174281 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-5deb236a-d39c-46d5-9195-c61e16f4d607 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Getting quotas for project b533f85aeddf4d8db05f2c53a61befc3. Resources: {'ram', 'instances', 'cores'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:39:47.180283 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-5deb236a-d39c-46d5-9195-c61e16f4d607 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Getting quotas for user 62209b9802ab46c3b3d094d4d1ad86fe and project b533f85aeddf4d8db05f2c53a61befc3. Resources: {'ram', 'instances', 'cores'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:39:47.239432 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-5deb236a-d39c-46d5-9195-c61e16f4d607 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Going to run 1 instances... {{(pid=138680) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 16 21:39:47.241579 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-5deb236a-d39c-46d5-9195-c61e16f4d607 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] [instance: 997f89c5-8086-4a7e-8292-1c8e6a12aaf6] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag=None,updated_at=,uuid=,volume_id='cb83e900-2518-45c5-9a34-d536cf6c7a3f',volume_size=None,volume_type=None)] {{(pid=138680) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 16 21:39:47.605429 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-6441e3a3-bad6-487f-a583-c0dbf415bd97 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:39:47.606043 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-6441e3a3-bad6-487f-a583-c0dbf415bd97 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] [instance: 2ce56b72-9fe8-41ca-bd6e-31591ef3b494] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:39:47.613230 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6441e3a3-bad6-487f-a583-c0dbf415bd97 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:39:47.614821 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6441e3a3-bad6-487f-a583-c0dbf415bd97 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:39:47.618685 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6441e3a3-bad6-487f-a583-c0dbf415bd97 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:39:47.618685 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6441e3a3-bad6-487f-a583-c0dbf415bd97 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:39:47.628861 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-5deb236a-d39c-46d5-9195-c61e16f4d607 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 1.327678 Jul 16 21:39:47.630047 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2655/5319] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 21:39:46 2022] POST /compute/v2.1/servers => generated 380 bytes in 1329 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 16 21:39:47.640437 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-793cc389-7cf2-447f-b9ee-d187c0d181aa tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:39:47.641057 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-793cc389-7cf2-447f-b9ee-d187c0d181aa tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] [instance: 997f89c5-8086-4a7e-8292-1c8e6a12aaf6] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:39:47.720984 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-793cc389-7cf2-447f-b9ee-d187c0d181aa tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '62209b9802ab46c3b3d094d4d1ad86fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b533f85aeddf4d8db05f2c53a61befc3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:39:47.729718 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-793cc389-7cf2-447f-b9ee-d187c0d181aa tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] 10.222.0.16 "GET /compute/v2.1/servers/997f89c5-8086-4a7e-8292-1c8e6a12aaf6" status: 200 len: 1071 microversion: 2.1 time: 0.093807 Jul 16 21:39:47.730308 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2656/5320] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:39:47 2022] GET /compute/v2.1/servers/997f89c5-8086-4a7e-8292-1c8e6a12aaf6 => generated 1071 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:39:47.790178 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-6441e3a3-bad6-487f-a583-c0dbf415bd97 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '833136cbdeab4b9e8b6dd2d24c7c8753', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '77c9b7e351e24dc88207e9498f75f578', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:39:47.798141 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-6441e3a3-bad6-487f-a583-c0dbf415bd97 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] 10.222.0.16 "GET /compute/v2.1/servers/2ce56b72-9fe8-41ca-bd6e-31591ef3b494" status: 200 len: 1696 microversion: 2.1 time: 0.196553 Jul 16 21:39:47.798772 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2665/5321] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:39:47 2022] GET /compute/v2.1/servers/2ce56b72-9fe8-41ca-bd6e-31591ef3b494 => generated 1696 bytes in 198 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:39:48.747395 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-edcb8388-adac-4db4-9215-f39b54e41c4f tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:39:48.747941 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-edcb8388-adac-4db4-9215-f39b54e41c4f tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] [instance: 997f89c5-8086-4a7e-8292-1c8e6a12aaf6] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:39:48.759079 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-edcb8388-adac-4db4-9215-f39b54e41c4f tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:39:48.759541 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-edcb8388-adac-4db4-9215-f39b54e41c4f tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:39:48.760441 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-edcb8388-adac-4db4-9215-f39b54e41c4f tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:39:48.760885 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-edcb8388-adac-4db4-9215-f39b54e41c4f tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:39:48.816527 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-87d5b86c-8d9d-4bae-bbc4-dc977cd0088a tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:39:48.818110 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-87d5b86c-8d9d-4bae-bbc4-dc977cd0088a tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] [instance: 2ce56b72-9fe8-41ca-bd6e-31591ef3b494] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:39:48.826011 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-87d5b86c-8d9d-4bae-bbc4-dc977cd0088a tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:39:48.826704 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-87d5b86c-8d9d-4bae-bbc4-dc977cd0088a tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:39:48.828005 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-87d5b86c-8d9d-4bae-bbc4-dc977cd0088a tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:39:48.828804 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-87d5b86c-8d9d-4bae-bbc4-dc977cd0088a tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:39:48.830253 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-edcb8388-adac-4db4-9215-f39b54e41c4f tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '62209b9802ab46c3b3d094d4d1ad86fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b533f85aeddf4d8db05f2c53a61befc3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:39:48.840716 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-edcb8388-adac-4db4-9215-f39b54e41c4f tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] 10.222.0.16 "GET /compute/v2.1/servers/997f89c5-8086-4a7e-8292-1c8e6a12aaf6" status: 200 len: 1121 microversion: 2.1 time: 0.095893 Jul 16 21:39:48.840716 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2657/5322] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:39:48 2022] GET /compute/v2.1/servers/997f89c5-8086-4a7e-8292-1c8e6a12aaf6 => generated 1121 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:39:48.969427 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-87d5b86c-8d9d-4bae-bbc4-dc977cd0088a tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '833136cbdeab4b9e8b6dd2d24c7c8753', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '77c9b7e351e24dc88207e9498f75f578', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:39:48.977260 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-87d5b86c-8d9d-4bae-bbc4-dc977cd0088a tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] 10.222.0.16 "GET /compute/v2.1/servers/2ce56b72-9fe8-41ca-bd6e-31591ef3b494" status: 200 len: 1696 microversion: 2.1 time: 0.165064 Jul 16 21:39:48.977841 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2666/5323] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:39:48 2022] GET /compute/v2.1/servers/2ce56b72-9fe8-41ca-bd6e-31591ef3b494 => generated 1696 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:39:49.857116 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-f90ed3dd-9275-4f2e-94c9-1c8ecbb8b175 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:39:49.857839 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-f90ed3dd-9275-4f2e-94c9-1c8ecbb8b175 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] [instance: 997f89c5-8086-4a7e-8292-1c8e6a12aaf6] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:39:49.867453 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f90ed3dd-9275-4f2e-94c9-1c8ecbb8b175 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:39:49.868068 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f90ed3dd-9275-4f2e-94c9-1c8ecbb8b175 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:39:49.869323 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f90ed3dd-9275-4f2e-94c9-1c8ecbb8b175 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:39:49.869424 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f90ed3dd-9275-4f2e-94c9-1c8ecbb8b175 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:39:49.940867 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-f90ed3dd-9275-4f2e-94c9-1c8ecbb8b175 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '62209b9802ab46c3b3d094d4d1ad86fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b533f85aeddf4d8db05f2c53a61befc3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:39:49.952045 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-f90ed3dd-9275-4f2e-94c9-1c8ecbb8b175 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] 10.222.0.16 "GET /compute/v2.1/servers/997f89c5-8086-4a7e-8292-1c8e6a12aaf6" status: 200 len: 1187 microversion: 2.1 time: 0.099334 Jul 16 21:39:49.956265 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2658/5324] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:39:49 2022] GET /compute/v2.1/servers/997f89c5-8086-4a7e-8292-1c8e6a12aaf6 => generated 1187 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:39:49.994198 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-afaa94f9-9774-4185-ab2f-ec3b0e78132c tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:39:49.994961 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-afaa94f9-9774-4185-ab2f-ec3b0e78132c tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] [instance: 2ce56b72-9fe8-41ca-bd6e-31591ef3b494] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:39:50.002088 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-afaa94f9-9774-4185-ab2f-ec3b0e78132c tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:39:50.002613 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-afaa94f9-9774-4185-ab2f-ec3b0e78132c tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:39:50.003645 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-afaa94f9-9774-4185-ab2f-ec3b0e78132c tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:39:50.004860 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-afaa94f9-9774-4185-ab2f-ec3b0e78132c tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:39:50.229144 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-afaa94f9-9774-4185-ab2f-ec3b0e78132c tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '833136cbdeab4b9e8b6dd2d24c7c8753', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '77c9b7e351e24dc88207e9498f75f578', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:39:50.244417 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-afaa94f9-9774-4185-ab2f-ec3b0e78132c tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] 10.222.0.16 "GET /compute/v2.1/servers/2ce56b72-9fe8-41ca-bd6e-31591ef3b494" status: 200 len: 1696 microversion: 2.1 time: 0.253942 Jul 16 21:39:50.244941 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2667/5325] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:39:49 2022] GET /compute/v2.1/servers/2ce56b72-9fe8-41ca-bd6e-31591ef3b494 => generated 1696 bytes in 255 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:39:50.973329 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-913f3a54-e9ed-410f-ba2f-e4038a75b940 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:39:50.973987 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-913f3a54-e9ed-410f-ba2f-e4038a75b940 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] [instance: 997f89c5-8086-4a7e-8292-1c8e6a12aaf6] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:39:50.981100 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-913f3a54-e9ed-410f-ba2f-e4038a75b940 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:39:50.981814 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-913f3a54-e9ed-410f-ba2f-e4038a75b940 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:39:50.982583 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-913f3a54-e9ed-410f-ba2f-e4038a75b940 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:39:50.983045 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-913f3a54-e9ed-410f-ba2f-e4038a75b940 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:39:51.118023 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-913f3a54-e9ed-410f-ba2f-e4038a75b940 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '62209b9802ab46c3b3d094d4d1ad86fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b533f85aeddf4d8db05f2c53a61befc3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:39:51.125748 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-913f3a54-e9ed-410f-ba2f-e4038a75b940 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] 10.222.0.16 "GET /compute/v2.1/servers/997f89c5-8086-4a7e-8292-1c8e6a12aaf6" status: 200 len: 1229 microversion: 2.1 time: 0.156983 Jul 16 21:39:51.126320 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2659/5326] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:39:50 2022] GET /compute/v2.1/servers/997f89c5-8086-4a7e-8292-1c8e6a12aaf6 => generated 1229 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:39:51.262190 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-01bada5a-1a63-4ecc-8aea-be6b0d9b3c8d tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:39:51.262805 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-01bada5a-1a63-4ecc-8aea-be6b0d9b3c8d tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] [instance: 2ce56b72-9fe8-41ca-bd6e-31591ef3b494] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:39:51.272596 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-01bada5a-1a63-4ecc-8aea-be6b0d9b3c8d tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:39:51.273165 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-01bada5a-1a63-4ecc-8aea-be6b0d9b3c8d tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:39:51.274069 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-01bada5a-1a63-4ecc-8aea-be6b0d9b3c8d tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:39:51.274617 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-01bada5a-1a63-4ecc-8aea-be6b0d9b3c8d tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:39:51.465669 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-01bada5a-1a63-4ecc-8aea-be6b0d9b3c8d tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '833136cbdeab4b9e8b6dd2d24c7c8753', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '77c9b7e351e24dc88207e9498f75f578', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:39:51.473755 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-01bada5a-1a63-4ecc-8aea-be6b0d9b3c8d tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] 10.222.0.16 "GET /compute/v2.1/servers/2ce56b72-9fe8-41ca-bd6e-31591ef3b494" status: 200 len: 1696 microversion: 2.1 time: 0.215573 Jul 16 21:39:51.474434 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2668/5327] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:39:51 2022] GET /compute/v2.1/servers/2ce56b72-9fe8-41ca-bd6e-31591ef3b494 => generated 1696 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:39:51.599114 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-7b7336a3-6197-41e2-adec-55f0244c5b90 req-054d01f8-7502-4817-8f25-3a630c5d49de service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "997f89c5-8086-4a7e-8292-1c8e6a12aaf6", "tag": "60aea633-216f-4cec-a388-963ef6e6e2a0"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:39:51.612689 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-7b7336a3-6197-41e2-adec-55f0244c5b90 req-054d01f8-7502-4817-8f25-3a630c5d49de service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:39:51.616489 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-7b7336a3-6197-41e2-adec-55f0244c5b90 req-054d01f8-7502-4817-8f25-3a630c5d49de service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:39:51.630546 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-7b7336a3-6197-41e2-adec-55f0244c5b90 req-054d01f8-7502-4817-8f25-3a630c5d49de service nova] Creating event network-changed:60aea633-216f-4cec-a388-963ef6e6e2a0 for instance 997f89c5-8086-4a7e-8292-1c8e6a12aaf6 on nch2-master-1 Jul 16 21:39:51.636236 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-7b7336a3-6197-41e2-adec-55f0244c5b90 req-054d01f8-7502-4817-8f25-3a630c5d49de service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.040934 Jul 16 21:39:51.637010 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2660/5328] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:39:51 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:39:52.143437 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-c00001cb-1edc-4867-9156-57f4616b8c3d tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:39:52.144293 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-c00001cb-1edc-4867-9156-57f4616b8c3d tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] [instance: 997f89c5-8086-4a7e-8292-1c8e6a12aaf6] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:39:52.151300 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c00001cb-1edc-4867-9156-57f4616b8c3d tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:39:52.151856 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c00001cb-1edc-4867-9156-57f4616b8c3d tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:39:52.152627 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c00001cb-1edc-4867-9156-57f4616b8c3d tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:39:52.153073 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c00001cb-1edc-4867-9156-57f4616b8c3d tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:39:52.288607 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-c00001cb-1edc-4867-9156-57f4616b8c3d tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '62209b9802ab46c3b3d094d4d1ad86fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b533f85aeddf4d8db05f2c53a61befc3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:39:52.296596 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-c00001cb-1edc-4867-9156-57f4616b8c3d tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] 10.222.0.16 "GET /compute/v2.1/servers/997f89c5-8086-4a7e-8292-1c8e6a12aaf6" status: 200 len: 1229 microversion: 2.1 time: 0.157359 Jul 16 21:39:52.297341 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2669/5329] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:39:52 2022] GET /compute/v2.1/servers/997f89c5-8086-4a7e-8292-1c8e6a12aaf6 => generated 1229 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:39:52.492907 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-fb9df55d-e756-4edb-9280-c407acdf4581 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:39:52.494266 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-fb9df55d-e756-4edb-9280-c407acdf4581 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] [instance: 2ce56b72-9fe8-41ca-bd6e-31591ef3b494] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:39:52.502158 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-fb9df55d-e756-4edb-9280-c407acdf4581 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:39:52.502617 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-fb9df55d-e756-4edb-9280-c407acdf4581 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:39:52.504360 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-fb9df55d-e756-4edb-9280-c407acdf4581 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:39:52.504360 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-fb9df55d-e756-4edb-9280-c407acdf4581 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:39:52.690027 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-fb9df55d-e756-4edb-9280-c407acdf4581 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '833136cbdeab4b9e8b6dd2d24c7c8753', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '77c9b7e351e24dc88207e9498f75f578', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:39:52.700143 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-fb9df55d-e756-4edb-9280-c407acdf4581 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] 10.222.0.16 "GET /compute/v2.1/servers/2ce56b72-9fe8-41ca-bd6e-31591ef3b494" status: 200 len: 1696 microversion: 2.1 time: 0.212160 Jul 16 21:39:52.700735 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2661/5330] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:39:52 2022] GET /compute/v2.1/servers/2ce56b72-9fe8-41ca-bd6e-31591ef3b494 => generated 1696 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:39:53.319534 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-92faaab9-0fe8-4829-a1df-ac321b45dce3 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:39:53.319534 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-92faaab9-0fe8-4829-a1df-ac321b45dce3 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] [instance: 997f89c5-8086-4a7e-8292-1c8e6a12aaf6] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:39:53.326229 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-92faaab9-0fe8-4829-a1df-ac321b45dce3 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:39:53.326890 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-92faaab9-0fe8-4829-a1df-ac321b45dce3 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:39:53.328063 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-92faaab9-0fe8-4829-a1df-ac321b45dce3 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:39:53.328671 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-92faaab9-0fe8-4829-a1df-ac321b45dce3 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:39:53.589122 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-92faaab9-0fe8-4829-a1df-ac321b45dce3 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '62209b9802ab46c3b3d094d4d1ad86fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b533f85aeddf4d8db05f2c53a61befc3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:39:53.593502 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-92faaab9-0fe8-4829-a1df-ac321b45dce3 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] 10.222.0.16 "GET /compute/v2.1/servers/997f89c5-8086-4a7e-8292-1c8e6a12aaf6" status: 200 len: 1217 microversion: 2.1 time: 0.283915 Jul 16 21:39:53.594082 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2670/5331] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:39:53 2022] GET /compute/v2.1/servers/997f89c5-8086-4a7e-8292-1c8e6a12aaf6 => generated 1217 bytes in 285 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:39:53.717475 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-8834833e-3cba-4bd8-9c5e-20691eaf57a9 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:39:53.718090 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-8834833e-3cba-4bd8-9c5e-20691eaf57a9 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] [instance: 2ce56b72-9fe8-41ca-bd6e-31591ef3b494] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:39:53.726559 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8834833e-3cba-4bd8-9c5e-20691eaf57a9 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:39:53.727744 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8834833e-3cba-4bd8-9c5e-20691eaf57a9 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:39:53.728774 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8834833e-3cba-4bd8-9c5e-20691eaf57a9 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:39:53.729388 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8834833e-3cba-4bd8-9c5e-20691eaf57a9 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:39:53.946162 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-8834833e-3cba-4bd8-9c5e-20691eaf57a9 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '833136cbdeab4b9e8b6dd2d24c7c8753', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '77c9b7e351e24dc88207e9498f75f578', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:39:53.954616 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-8834833e-3cba-4bd8-9c5e-20691eaf57a9 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] 10.222.0.16 "GET /compute/v2.1/servers/2ce56b72-9fe8-41ca-bd6e-31591ef3b494" status: 200 len: 1695 microversion: 2.1 time: 0.241073 Jul 16 21:39:53.955324 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2662/5332] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:39:53 2022] GET /compute/v2.1/servers/2ce56b72-9fe8-41ca-bd6e-31591ef3b494 => generated 1695 bytes in 242 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:39:54.459678 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-0ef7d0fd-267a-4799-a24c-79af6560f19e req-2c55a8e1-b344-4de4-926f-ca70a2986478 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "2ce56b72-9fe8-41ca-bd6e-31591ef3b494", "name": "network-vif-unplugged", "status": "completed", "tag": "a75ca41e-859e-4a3f-a17c-93721015537e"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:39:54.468294 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-0ef7d0fd-267a-4799-a24c-79af6560f19e req-2c55a8e1-b344-4de4-926f-ca70a2986478 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:39:54.469185 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-0ef7d0fd-267a-4799-a24c-79af6560f19e req-2c55a8e1-b344-4de4-926f-ca70a2986478 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:39:54.482164 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-0ef7d0fd-267a-4799-a24c-79af6560f19e req-2c55a8e1-b344-4de4-926f-ca70a2986478 service nova] Creating event network-vif-unplugged:a75ca41e-859e-4a3f-a17c-93721015537e for instance 2ce56b72-9fe8-41ca-bd6e-31591ef3b494 on nch1-master-1 Jul 16 21:39:54.489593 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [req-0ef7d0fd-267a-4799-a24c-79af6560f19e req-2c55a8e1-b344-4de4-926f-ca70a2986478 service nova] Instance 2ce56b72-9fe8-41ca-bd6e-31591ef3b494 is migrating within the same cell, copying events to all relevant hosts: {'nch2-master-1', 'nch1-master-1'} {{(pid=138681) _get_relevant_hosts /opt/stack/nova/nova/compute/api.py:5807}} Jul 16 21:39:54.502886 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-0ef7d0fd-267a-4799-a24c-79af6560f19e req-2c55a8e1-b344-4de4-926f-ca70a2986478 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.256053 Jul 16 21:39:54.504125 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2671/5333] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:39:54 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 258 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 16 21:39:54.614445 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-6f642524-f34a-4bb2-816b-c4a9da321db6 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:39:54.621165 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-6f642524-f34a-4bb2-816b-c4a9da321db6 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] [instance: 997f89c5-8086-4a7e-8292-1c8e6a12aaf6] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:39:54.621474 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6f642524-f34a-4bb2-816b-c4a9da321db6 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:39:54.622052 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6f642524-f34a-4bb2-816b-c4a9da321db6 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:39:54.622842 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6f642524-f34a-4bb2-816b-c4a9da321db6 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:39:54.623309 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6f642524-f34a-4bb2-816b-c4a9da321db6 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:39:54.872728 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-6f642524-f34a-4bb2-816b-c4a9da321db6 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '62209b9802ab46c3b3d094d4d1ad86fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b533f85aeddf4d8db05f2c53a61befc3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:39:54.892422 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-6f642524-f34a-4bb2-816b-c4a9da321db6 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] 10.222.0.16 "GET /compute/v2.1/servers/997f89c5-8086-4a7e-8292-1c8e6a12aaf6" status: 200 len: 1217 microversion: 2.1 time: 0.282302 Jul 16 21:39:54.893110 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2663/5334] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:39:54 2022] GET /compute/v2.1/servers/997f89c5-8086-4a7e-8292-1c8e6a12aaf6 => generated 1217 bytes in 284 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:39:54.973036 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-e676c031-e756-47cd-814e-f5e9b7c7a6f6 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:39:54.973758 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-e676c031-e756-47cd-814e-f5e9b7c7a6f6 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] [instance: 2ce56b72-9fe8-41ca-bd6e-31591ef3b494] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:39:54.980761 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e676c031-e756-47cd-814e-f5e9b7c7a6f6 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:39:54.981250 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e676c031-e756-47cd-814e-f5e9b7c7a6f6 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:39:54.982315 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e676c031-e756-47cd-814e-f5e9b7c7a6f6 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:39:54.982778 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e676c031-e756-47cd-814e-f5e9b7c7a6f6 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:39:55.248564 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-e676c031-e756-47cd-814e-f5e9b7c7a6f6 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '833136cbdeab4b9e8b6dd2d24c7c8753', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '77c9b7e351e24dc88207e9498f75f578', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:39:55.258898 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-e676c031-e756-47cd-814e-f5e9b7c7a6f6 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] 10.222.0.16 "GET /compute/v2.1/servers/2ce56b72-9fe8-41ca-bd6e-31591ef3b494" status: 200 len: 1695 microversion: 2.1 time: 0.290129 Jul 16 21:39:55.259956 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2672/5335] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:39:54 2022] GET /compute/v2.1/servers/2ce56b72-9fe8-41ca-bd6e-31591ef3b494 => generated 1695 bytes in 292 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:39:55.909678 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-ebaa0151-feda-4c9b-b43d-4bc0c94b10ef tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:39:55.912923 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-ebaa0151-feda-4c9b-b43d-4bc0c94b10ef tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] [instance: 997f89c5-8086-4a7e-8292-1c8e6a12aaf6] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:39:55.917998 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ebaa0151-feda-4c9b-b43d-4bc0c94b10ef tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:39:55.918454 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ebaa0151-feda-4c9b-b43d-4bc0c94b10ef tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:39:55.919272 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ebaa0151-feda-4c9b-b43d-4bc0c94b10ef tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:39:55.919716 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ebaa0151-feda-4c9b-b43d-4bc0c94b10ef tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:39:56.058885 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-ebaa0151-feda-4c9b-b43d-4bc0c94b10ef tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '62209b9802ab46c3b3d094d4d1ad86fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b533f85aeddf4d8db05f2c53a61befc3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:39:56.066915 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-ebaa0151-feda-4c9b-b43d-4bc0c94b10ef tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] 10.222.0.16 "GET /compute/v2.1/servers/997f89c5-8086-4a7e-8292-1c8e6a12aaf6" status: 200 len: 1217 microversion: 2.1 time: 0.161061 Jul 16 21:39:56.067520 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2664/5336] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:39:55 2022] GET /compute/v2.1/servers/997f89c5-8086-4a7e-8292-1c8e6a12aaf6 => generated 1217 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:39:56.276954 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-8c701480-42e5-48ef-9098-609c89451ce8 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:39:56.277578 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-8c701480-42e5-48ef-9098-609c89451ce8 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] [instance: 2ce56b72-9fe8-41ca-bd6e-31591ef3b494] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:39:56.284828 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8c701480-42e5-48ef-9098-609c89451ce8 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:39:56.285336 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8c701480-42e5-48ef-9098-609c89451ce8 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:39:56.286236 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8c701480-42e5-48ef-9098-609c89451ce8 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:39:56.286688 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8c701480-42e5-48ef-9098-609c89451ce8 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:39:56.426908 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-8c701480-42e5-48ef-9098-609c89451ce8 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '833136cbdeab4b9e8b6dd2d24c7c8753', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '77c9b7e351e24dc88207e9498f75f578', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:39:56.434478 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-8c701480-42e5-48ef-9098-609c89451ce8 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] 10.222.0.16 "GET /compute/v2.1/servers/2ce56b72-9fe8-41ca-bd6e-31591ef3b494" status: 200 len: 1693 microversion: 2.1 time: 0.161450 Jul 16 21:39:56.435064 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2673/5337] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:39:56 2022] GET /compute/v2.1/servers/2ce56b72-9fe8-41ca-bd6e-31591ef3b494 => generated 1693 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:39:56.528972 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-8359abfe-559f-436e-9490-c7e62114503d req-3691656b-b642-4165-b969-1405e18728f4 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "2ce56b72-9fe8-41ca-bd6e-31591ef3b494", "tag": "a75ca41e-859e-4a3f-a17c-93721015537e"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:39:56.537347 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-8359abfe-559f-436e-9490-c7e62114503d req-3691656b-b642-4165-b969-1405e18728f4 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:39:56.537982 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-8359abfe-559f-436e-9490-c7e62114503d req-3691656b-b642-4165-b969-1405e18728f4 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:39:56.550051 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-8359abfe-559f-436e-9490-c7e62114503d req-3691656b-b642-4165-b969-1405e18728f4 service nova] Creating event network-changed:a75ca41e-859e-4a3f-a17c-93721015537e for instance 2ce56b72-9fe8-41ca-bd6e-31591ef3b494 on nch1-master-1 Jul 16 21:39:56.556075 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [req-8359abfe-559f-436e-9490-c7e62114503d req-3691656b-b642-4165-b969-1405e18728f4 service nova] Instance 2ce56b72-9fe8-41ca-bd6e-31591ef3b494 is migrating within the same cell, copying events to all relevant hosts: {'nch2-master-1', 'nch1-master-1'} {{(pid=138680) _get_relevant_hosts /opt/stack/nova/nova/compute/api.py:5807}} Jul 16 21:39:56.568004 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-8359abfe-559f-436e-9490-c7e62114503d req-3691656b-b642-4165-b969-1405e18728f4 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.042739 Jul 16 21:39:56.568672 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2665/5338] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:39:56 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:39:57.006158 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-99dd906c-407c-4ddc-a1ca-fc85eedbd374 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Action: 'action', calling method: >, body: {"os-getConsoleOutput": {}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:39:57.009130 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-99dd906c-407c-4ddc-a1ca-fc85eedbd374 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] [instance: 19df5c33-44d4-4de9-947b-4707b78af23a] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:39:57.016892 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-99dd906c-407c-4ddc-a1ca-fc85eedbd374 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:39:57.021193 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-99dd906c-407c-4ddc-a1ca-fc85eedbd374 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:39:57.021193 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-99dd906c-407c-4ddc-a1ca-fc85eedbd374 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:39:57.021193 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-99dd906c-407c-4ddc-a1ca-fc85eedbd374 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:39:57.025380 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:39:57.082803 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-99dd906c-407c-4ddc-a1ca-fc85eedbd374 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] 10.222.0.16 "POST /compute/v2.1/servers/19df5c33-44d4-4de9-947b-4707b78af23a/action" status: 200 len: 24590 microversion: 2.42 time: 0.293614 Jul 16 21:39:57.084249 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2674/5339] 10.222.0.16 () {62 vars in 1318 bytes} [Sat Jul 16 21:39:56 2022] POST /compute/v2.1/servers/19df5c33-44d4-4de9-947b-4707b78af23a/action => generated 24590 bytes in 296 msecs (HTTP/1.1 200) 9 headers in 361 bytes (1 switches on core 0) Jul 16 21:39:57.086893 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-94242f23-cd59-458f-b672-7a4ce608fcb7 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:39:57.088119 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-94242f23-cd59-458f-b672-7a4ce608fcb7 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] [instance: 997f89c5-8086-4a7e-8292-1c8e6a12aaf6] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:39:57.094786 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-94242f23-cd59-458f-b672-7a4ce608fcb7 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:39:57.100411 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-94242f23-cd59-458f-b672-7a4ce608fcb7 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:39:57.100411 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-94242f23-cd59-458f-b672-7a4ce608fcb7 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:39:57.100411 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-94242f23-cd59-458f-b672-7a4ce608fcb7 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:39:57.164749 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell0:SELECT=2 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:39:57.459160 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-cb8857a1-30be-4567-a34b-92be41963576 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:39:57.459813 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-cb8857a1-30be-4567-a34b-92be41963576 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] [instance: 2ce56b72-9fe8-41ca-bd6e-31591ef3b494] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:39:57.468312 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cb8857a1-30be-4567-a34b-92be41963576 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:39:57.468764 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cb8857a1-30be-4567-a34b-92be41963576 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:39:57.469539 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cb8857a1-30be-4567-a34b-92be41963576 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:39:57.469978 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cb8857a1-30be-4567-a34b-92be41963576 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:39:57.729285 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell0:SELECT=8 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:39:57.767741 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-94242f23-cd59-458f-b672-7a4ce608fcb7 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '62209b9802ab46c3b3d094d4d1ad86fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b533f85aeddf4d8db05f2c53a61befc3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:39:57.776687 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-94242f23-cd59-458f-b672-7a4ce608fcb7 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] 10.222.0.16 "GET /compute/v2.1/servers/997f89c5-8086-4a7e-8292-1c8e6a12aaf6" status: 200 len: 1217 microversion: 2.1 time: 0.696393 Jul 16 21:39:57.777483 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2666/5340] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:39:57 2022] GET /compute/v2.1/servers/997f89c5-8086-4a7e-8292-1c8e6a12aaf6 => generated 1217 bytes in 698 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:39:57.864583 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-cb8857a1-30be-4567-a34b-92be41963576 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '833136cbdeab4b9e8b6dd2d24c7c8753', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '77c9b7e351e24dc88207e9498f75f578', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:39:57.875834 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-cb8857a1-30be-4567-a34b-92be41963576 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] 10.222.0.16 "GET /compute/v2.1/servers/2ce56b72-9fe8-41ca-bd6e-31591ef3b494" status: 200 len: 1693 microversion: 2.1 time: 0.420878 Jul 16 21:39:57.887474 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2675/5341] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:39:57 2022] GET /compute/v2.1/servers/2ce56b72-9fe8-41ca-bd6e-31591ef3b494 => generated 1693 bytes in 434 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:39:58.793882 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-f899b596-b914-4e6a-ba5e-82c080a1e32b tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:39:58.794508 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-f899b596-b914-4e6a-ba5e-82c080a1e32b tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] [instance: 997f89c5-8086-4a7e-8292-1c8e6a12aaf6] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:39:58.802780 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f899b596-b914-4e6a-ba5e-82c080a1e32b tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:39:58.803555 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f899b596-b914-4e6a-ba5e-82c080a1e32b tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:39:58.804592 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f899b596-b914-4e6a-ba5e-82c080a1e32b tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:39:58.805161 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f899b596-b914-4e6a-ba5e-82c080a1e32b tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:39:58.898878 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-ae017db8-e5ed-4ccd-9b8f-cd42a302b739 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:39:58.900674 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-ae017db8-e5ed-4ccd-9b8f-cd42a302b739 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] [instance: 2ce56b72-9fe8-41ca-bd6e-31591ef3b494] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:39:58.909521 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ae017db8-e5ed-4ccd-9b8f-cd42a302b739 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:39:58.911198 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ae017db8-e5ed-4ccd-9b8f-cd42a302b739 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:39:58.912229 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ae017db8-e5ed-4ccd-9b8f-cd42a302b739 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:39:58.912772 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ae017db8-e5ed-4ccd-9b8f-cd42a302b739 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:39:59.027879 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-f899b596-b914-4e6a-ba5e-82c080a1e32b tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '62209b9802ab46c3b3d094d4d1ad86fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b533f85aeddf4d8db05f2c53a61befc3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:39:59.040671 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-f899b596-b914-4e6a-ba5e-82c080a1e32b tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] 10.222.0.16 "GET /compute/v2.1/servers/997f89c5-8086-4a7e-8292-1c8e6a12aaf6" status: 200 len: 1217 microversion: 2.1 time: 0.250482 Jul 16 21:39:59.047200 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2667/5342] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:39:58 2022] GET /compute/v2.1/servers/997f89c5-8086-4a7e-8292-1c8e6a12aaf6 => generated 1217 bytes in 256 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:39:59.108715 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-ae017db8-e5ed-4ccd-9b8f-cd42a302b739 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '833136cbdeab4b9e8b6dd2d24c7c8753', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '77c9b7e351e24dc88207e9498f75f578', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:39:59.117499 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-ae017db8-e5ed-4ccd-9b8f-cd42a302b739 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] 10.222.0.16 "GET /compute/v2.1/servers/2ce56b72-9fe8-41ca-bd6e-31591ef3b494" status: 200 len: 1690 microversion: 2.1 time: 0.226569 Jul 16 21:39:59.118467 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2676/5343] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:39:58 2022] GET /compute/v2.1/servers/2ce56b72-9fe8-41ca-bd6e-31591ef3b494 => generated 1690 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:39:59.142516 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-513bdd33-bab3-4939-b3f9-9f94acbcad18 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Action: 'action', calling method: >, body: {"confirmResize": {}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:39:59.142516 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-513bdd33-bab3-4939-b3f9-9f94acbcad18 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] [instance: 2ce56b72-9fe8-41ca-bd6e-31591ef3b494] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:39:59.146143 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-513bdd33-bab3-4939-b3f9-9f94acbcad18 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:39:59.146603 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-513bdd33-bab3-4939-b3f9-9f94acbcad18 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:39:59.147462 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-513bdd33-bab3-4939-b3f9-9f94acbcad18 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:39:59.148126 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-513bdd33-bab3-4939-b3f9-9f94acbcad18 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:39:59.165689 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=100 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:39:59.382839 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-513bdd33-bab3-4939-b3f9-9f94acbcad18 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] 10.222.0.16 "POST /compute/v2.1/servers/2ce56b72-9fe8-41ca-bd6e-31591ef3b494/action" status: 204 len: 0 microversion: 2.1 time: 0.249138 Jul 16 21:39:59.383552 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2668/5344] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 21:39:59 2022] POST /compute/v2.1/servers/2ce56b72-9fe8-41ca-bd6e-31591ef3b494/action => generated 0 bytes in 251 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 16 21:39:59.391679 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-d1c2bf17-ec13-4b77-acea-f52c25e96f03 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:39:59.392300 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-d1c2bf17-ec13-4b77-acea-f52c25e96f03 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] [instance: 2ce56b72-9fe8-41ca-bd6e-31591ef3b494] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:39:59.398623 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d1c2bf17-ec13-4b77-acea-f52c25e96f03 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:39:59.399109 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d1c2bf17-ec13-4b77-acea-f52c25e96f03 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:39:59.399889 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d1c2bf17-ec13-4b77-acea-f52c25e96f03 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:39:59.400330 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d1c2bf17-ec13-4b77-acea-f52c25e96f03 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:39:59.546688 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-d1c2bf17-ec13-4b77-acea-f52c25e96f03 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '833136cbdeab4b9e8b6dd2d24c7c8753', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '77c9b7e351e24dc88207e9498f75f578', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:39:59.554380 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-d1c2bf17-ec13-4b77-acea-f52c25e96f03 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] 10.222.0.16 "GET /compute/v2.1/servers/2ce56b72-9fe8-41ca-bd6e-31591ef3b494" status: 200 len: 1690 microversion: 2.1 time: 0.166516 Jul 16 21:39:59.555295 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2677/5345] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:39:59 2022] GET /compute/v2.1/servers/2ce56b72-9fe8-41ca-bd6e-31591ef3b494 => generated 1690 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:40:00.062941 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-909c51b8-97ff-4b20-8797-692b45b4eef4 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:40:00.063743 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-909c51b8-97ff-4b20-8797-692b45b4eef4 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] [instance: 997f89c5-8086-4a7e-8292-1c8e6a12aaf6] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:40:00.071228 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-909c51b8-97ff-4b20-8797-692b45b4eef4 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:40:00.078600 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-909c51b8-97ff-4b20-8797-692b45b4eef4 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:40:00.078600 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-909c51b8-97ff-4b20-8797-692b45b4eef4 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:40:00.078600 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-909c51b8-97ff-4b20-8797-692b45b4eef4 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:40:00.293529 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-909c51b8-97ff-4b20-8797-692b45b4eef4 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '62209b9802ab46c3b3d094d4d1ad86fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b533f85aeddf4d8db05f2c53a61befc3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:40:00.304533 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-909c51b8-97ff-4b20-8797-692b45b4eef4 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] 10.222.0.16 "GET /compute/v2.1/servers/997f89c5-8086-4a7e-8292-1c8e6a12aaf6" status: 200 len: 1217 microversion: 2.1 time: 0.247551 Jul 16 21:40:00.315045 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2669/5346] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:40:00 2022] GET /compute/v2.1/servers/997f89c5-8086-4a7e-8292-1c8e6a12aaf6 => generated 1217 bytes in 249 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:40:00.572901 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-f68be9f4-ee6b-4a34-b798-c3758e707977 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:40:00.573452 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-f68be9f4-ee6b-4a34-b798-c3758e707977 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] [instance: 2ce56b72-9fe8-41ca-bd6e-31591ef3b494] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:40:00.585345 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f68be9f4-ee6b-4a34-b798-c3758e707977 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:40:00.585345 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f68be9f4-ee6b-4a34-b798-c3758e707977 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:40:00.585345 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f68be9f4-ee6b-4a34-b798-c3758e707977 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:40:00.585345 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f68be9f4-ee6b-4a34-b798-c3758e707977 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:40:00.734044 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-f68be9f4-ee6b-4a34-b798-c3758e707977 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '833136cbdeab4b9e8b6dd2d24c7c8753', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '77c9b7e351e24dc88207e9498f75f578', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:40:00.743658 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-f68be9f4-ee6b-4a34-b798-c3758e707977 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] 10.222.0.16 "GET /compute/v2.1/servers/2ce56b72-9fe8-41ca-bd6e-31591ef3b494" status: 200 len: 1690 microversion: 2.1 time: 0.174666 Jul 16 21:40:00.744941 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2678/5347] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:40:00 2022] GET /compute/v2.1/servers/2ce56b72-9fe8-41ca-bd6e-31591ef3b494 => generated 1690 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:40:01.071344 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-b5e1fcfe-8882-4ce7-b1c8-5b75a573e868 req-253195d9-9f4a-4d13-8237-211e7e843c97 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "2ce56b72-9fe8-41ca-bd6e-31591ef3b494", "name": "network-vif-plugged", "status": "completed", "tag": "a75ca41e-859e-4a3f-a17c-93721015537e"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:40:01.080500 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-b5e1fcfe-8882-4ce7-b1c8-5b75a573e868 req-253195d9-9f4a-4d13-8237-211e7e843c97 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:40:01.081233 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-b5e1fcfe-8882-4ce7-b1c8-5b75a573e868 req-253195d9-9f4a-4d13-8237-211e7e843c97 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:40:01.105253 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-b5e1fcfe-8882-4ce7-b1c8-5b75a573e868 req-253195d9-9f4a-4d13-8237-211e7e843c97 service nova] Creating event network-vif-plugged:a75ca41e-859e-4a3f-a17c-93721015537e for instance 2ce56b72-9fe8-41ca-bd6e-31591ef3b494 on nch1-master-1 Jul 16 21:40:01.112277 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [req-b5e1fcfe-8882-4ce7-b1c8-5b75a573e868 req-253195d9-9f4a-4d13-8237-211e7e843c97 service nova] Instance 2ce56b72-9fe8-41ca-bd6e-31591ef3b494 is migrating within the same cell, copying events to all relevant hosts: {'nch2-master-1', 'nch1-master-1'} {{(pid=138680) _get_relevant_hosts /opt/stack/nova/nova/compute/api.py:5807}} Jul 16 21:40:01.124114 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-b5e1fcfe-8882-4ce7-b1c8-5b75a573e868 req-253195d9-9f4a-4d13-8237-211e7e843c97 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.279341 Jul 16 21:40:01.124706 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2670/5348] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:40:00 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 281 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:40:01.323316 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-65ddd441-fd75-4098-8964-61d9aba15328 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:40:01.324003 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-65ddd441-fd75-4098-8964-61d9aba15328 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] [instance: 997f89c5-8086-4a7e-8292-1c8e6a12aaf6] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:40:01.332062 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-65ddd441-fd75-4098-8964-61d9aba15328 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:40:01.332814 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-65ddd441-fd75-4098-8964-61d9aba15328 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:40:01.334055 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-65ddd441-fd75-4098-8964-61d9aba15328 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:40:01.334640 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-65ddd441-fd75-4098-8964-61d9aba15328 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:40:01.478918 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-65ddd441-fd75-4098-8964-61d9aba15328 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '62209b9802ab46c3b3d094d4d1ad86fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b533f85aeddf4d8db05f2c53a61befc3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:40:01.488744 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-65ddd441-fd75-4098-8964-61d9aba15328 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] 10.222.0.16 "GET /compute/v2.1/servers/997f89c5-8086-4a7e-8292-1c8e6a12aaf6" status: 200 len: 1217 microversion: 2.1 time: 0.170082 Jul 16 21:40:01.489955 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2679/5349] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:40:01 2022] GET /compute/v2.1/servers/997f89c5-8086-4a7e-8292-1c8e6a12aaf6 => generated 1217 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:40:01.767733 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-f3ae0d71-7563-4f92-b731-44b83e889e9a tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:40:01.768528 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-f3ae0d71-7563-4f92-b731-44b83e889e9a tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] [instance: 2ce56b72-9fe8-41ca-bd6e-31591ef3b494] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:40:01.776047 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f3ae0d71-7563-4f92-b731-44b83e889e9a tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:40:01.776612 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f3ae0d71-7563-4f92-b731-44b83e889e9a tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:40:01.777440 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f3ae0d71-7563-4f92-b731-44b83e889e9a tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:40:01.777957 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f3ae0d71-7563-4f92-b731-44b83e889e9a tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:40:01.925149 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-f3ae0d71-7563-4f92-b731-44b83e889e9a tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '833136cbdeab4b9e8b6dd2d24c7c8753', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '77c9b7e351e24dc88207e9498f75f578', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:40:01.933343 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-f3ae0d71-7563-4f92-b731-44b83e889e9a tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] 10.222.0.16 "GET /compute/v2.1/servers/2ce56b72-9fe8-41ca-bd6e-31591ef3b494" status: 200 len: 1682 microversion: 2.1 time: 0.170282 Jul 16 21:40:01.933920 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2671/5350] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:40:01 2022] GET /compute/v2.1/servers/2ce56b72-9fe8-41ca-bd6e-31591ef3b494 => generated 1682 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:40:02.513442 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-62249089-36b3-4d70-84ab-0cc43a3bf855 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:40:02.514095 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-62249089-36b3-4d70-84ab-0cc43a3bf855 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] [instance: 997f89c5-8086-4a7e-8292-1c8e6a12aaf6] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:40:02.529534 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-62249089-36b3-4d70-84ab-0cc43a3bf855 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:40:02.529992 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-62249089-36b3-4d70-84ab-0cc43a3bf855 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:40:02.530775 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-62249089-36b3-4d70-84ab-0cc43a3bf855 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:40:02.531306 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-62249089-36b3-4d70-84ab-0cc43a3bf855 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:40:02.665119 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-62249089-36b3-4d70-84ab-0cc43a3bf855 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '62209b9802ab46c3b3d094d4d1ad86fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b533f85aeddf4d8db05f2c53a61befc3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:40:02.672976 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-62249089-36b3-4d70-84ab-0cc43a3bf855 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] 10.222.0.16 "GET /compute/v2.1/servers/997f89c5-8086-4a7e-8292-1c8e6a12aaf6" status: 200 len: 1217 microversion: 2.1 time: 0.163477 Jul 16 21:40:02.673541 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2680/5351] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:40:02 2022] GET /compute/v2.1/servers/997f89c5-8086-4a7e-8292-1c8e6a12aaf6 => generated 1217 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:40:03.140589 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-871d3578-2a28-4c31-b458-c90c4f961009 req-993081e7-7c3b-4522-ba5d-52bef718e35b service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "997f89c5-8086-4a7e-8292-1c8e6a12aaf6", "name": "network-vif-plugged", "status": "completed", "tag": "60aea633-216f-4cec-a388-963ef6e6e2a0"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:40:03.150662 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-871d3578-2a28-4c31-b458-c90c4f961009 req-993081e7-7c3b-4522-ba5d-52bef718e35b service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:40:03.152219 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-871d3578-2a28-4c31-b458-c90c4f961009 req-993081e7-7c3b-4522-ba5d-52bef718e35b service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:40:03.162571 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-871d3578-2a28-4c31-b458-c90c4f961009 req-993081e7-7c3b-4522-ba5d-52bef718e35b service nova] Creating event network-vif-plugged:60aea633-216f-4cec-a388-963ef6e6e2a0 for instance 997f89c5-8086-4a7e-8292-1c8e6a12aaf6 on nch2-master-1 Jul 16 21:40:03.168194 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-871d3578-2a28-4c31-b458-c90c4f961009 req-993081e7-7c3b-4522-ba5d-52bef718e35b service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.030809 Jul 16 21:40:03.168316 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2672/5352] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:40:03 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:40:03.690495 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-0cd460ce-b93a-4e2e-81a1-087b805be88f tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:40:03.691163 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-0cd460ce-b93a-4e2e-81a1-087b805be88f tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] [instance: 997f89c5-8086-4a7e-8292-1c8e6a12aaf6] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:40:03.698238 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0cd460ce-b93a-4e2e-81a1-087b805be88f tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:40:03.698683 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0cd460ce-b93a-4e2e-81a1-087b805be88f tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:40:03.699541 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0cd460ce-b93a-4e2e-81a1-087b805be88f tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:40:03.699982 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0cd460ce-b93a-4e2e-81a1-087b805be88f tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:40:03.836331 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-0cd460ce-b93a-4e2e-81a1-087b805be88f tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '62209b9802ab46c3b3d094d4d1ad86fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b533f85aeddf4d8db05f2c53a61befc3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:40:03.843928 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-0cd460ce-b93a-4e2e-81a1-087b805be88f tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] 10.222.0.16 "GET /compute/v2.1/servers/997f89c5-8086-4a7e-8292-1c8e6a12aaf6" status: 200 len: 1217 microversion: 2.1 time: 0.157495 Jul 16 21:40:03.844503 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2681/5353] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:40:03 2022] GET /compute/v2.1/servers/997f89c5-8086-4a7e-8292-1c8e6a12aaf6 => generated 1217 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:40:04.863535 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-3ed4da5a-5f76-474e-aea2-a2f443c13417 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:40:04.864958 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-3ed4da5a-5f76-474e-aea2-a2f443c13417 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] [instance: 997f89c5-8086-4a7e-8292-1c8e6a12aaf6] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:40:04.872199 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3ed4da5a-5f76-474e-aea2-a2f443c13417 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:40:04.872655 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3ed4da5a-5f76-474e-aea2-a2f443c13417 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:40:04.873433 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3ed4da5a-5f76-474e-aea2-a2f443c13417 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:40:04.873895 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3ed4da5a-5f76-474e-aea2-a2f443c13417 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:40:05.008982 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-3ed4da5a-5f76-474e-aea2-a2f443c13417 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '62209b9802ab46c3b3d094d4d1ad86fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b533f85aeddf4d8db05f2c53a61befc3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:40:05.017132 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-3ed4da5a-5f76-474e-aea2-a2f443c13417 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] 10.222.0.16 "GET /compute/v2.1/servers/997f89c5-8086-4a7e-8292-1c8e6a12aaf6" status: 200 len: 1217 microversion: 2.1 time: 0.158816 Jul 16 21:40:05.017730 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2673/5354] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:40:04 2022] GET /compute/v2.1/servers/997f89c5-8086-4a7e-8292-1c8e6a12aaf6 => generated 1217 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:40:06.034936 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-cd90539e-6c6b-4de4-afba-a26deb11dcb7 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:40:06.041495 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-cd90539e-6c6b-4de4-afba-a26deb11dcb7 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] [instance: 997f89c5-8086-4a7e-8292-1c8e6a12aaf6] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:40:06.043400 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cd90539e-6c6b-4de4-afba-a26deb11dcb7 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:40:06.043898 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cd90539e-6c6b-4de4-afba-a26deb11dcb7 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:40:06.044685 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cd90539e-6c6b-4de4-afba-a26deb11dcb7 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:40:06.045126 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cd90539e-6c6b-4de4-afba-a26deb11dcb7 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:40:06.182438 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-cd90539e-6c6b-4de4-afba-a26deb11dcb7 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '62209b9802ab46c3b3d094d4d1ad86fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b533f85aeddf4d8db05f2c53a61befc3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:40:06.190225 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-cd90539e-6c6b-4de4-afba-a26deb11dcb7 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] 10.222.0.16 "GET /compute/v2.1/servers/997f89c5-8086-4a7e-8292-1c8e6a12aaf6" status: 200 len: 1404 microversion: 2.1 time: 0.159547 Jul 16 21:40:06.190799 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2682/5355] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:40:06 2022] GET /compute/v2.1/servers/997f89c5-8086-4a7e-8292-1c8e6a12aaf6 => generated 1404 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:40:06.207013 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-46ef72e4-3494-447c-9861-e070d719d147 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:40:06.207654 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-46ef72e4-3494-447c-9861-e070d719d147 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] [instance: 997f89c5-8086-4a7e-8292-1c8e6a12aaf6] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:40:06.215473 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-46ef72e4-3494-447c-9861-e070d719d147 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:40:06.215938 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-46ef72e4-3494-447c-9861-e070d719d147 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:40:06.216709 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-46ef72e4-3494-447c-9861-e070d719d147 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:40:06.217159 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-46ef72e4-3494-447c-9861-e070d719d147 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:40:06.360658 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-46ef72e4-3494-447c-9861-e070d719d147 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '62209b9802ab46c3b3d094d4d1ad86fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b533f85aeddf4d8db05f2c53a61befc3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:40:06.369362 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-46ef72e4-3494-447c-9861-e070d719d147 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] 10.222.0.16 "GET /compute/v2.1/servers/997f89c5-8086-4a7e-8292-1c8e6a12aaf6" status: 200 len: 1404 microversion: 2.1 time: 0.166162 Jul 16 21:40:06.373330 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2674/5356] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:40:06 2022] GET /compute/v2.1/servers/997f89c5-8086-4a7e-8292-1c8e6a12aaf6 => generated 1404 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:40:06.388793 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-e32f47cc-212a-46f3-858c-3b7ee9c1425c tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Action: 'action', calling method: >, body: {"createImage": {"name": "tempest-TestVolumeBootPatternsnapshot-1437460845"}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:40:06.389938 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-e32f47cc-212a-46f3-858c-3b7ee9c1425c tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] [instance: 997f89c5-8086-4a7e-8292-1c8e6a12aaf6] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:40:06.396387 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e32f47cc-212a-46f3-858c-3b7ee9c1425c tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:40:06.402261 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e32f47cc-212a-46f3-858c-3b7ee9c1425c tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:40:06.402261 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e32f47cc-212a-46f3-858c-3b7ee9c1425c tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:40:06.402261 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e32f47cc-212a-46f3-858c-3b7ee9c1425c tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:40:06.492830 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.compute.api [None req-e32f47cc-212a-46f3-858c-3b7ee9c1425c tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] [instance: 997f89c5-8086-4a7e-8292-1c8e6a12aaf6] Attempting to quiesce instance before volume snapshot. Jul 16 21:40:06.560843 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.compute.api [None req-e32f47cc-212a-46f3-858c-3b7ee9c1425c tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] [instance: 997f89c5-8086-4a7e-8292-1c8e6a12aaf6] Skipping quiescing instance because the operation is not supported by the underlying compute driver. Jul 16 21:40:06.760917 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-e32f47cc-212a-46f3-858c-3b7ee9c1425c tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] [instance: 997f89c5-8086-4a7e-8292-1c8e6a12aaf6] Creating snapshot from volume cb83e900-2518-45c5-9a34-d536cf6c7a3f. {{(pid=138681) snapshot_instance /opt/stack/nova/nova/compute/api.py:3454}} Jul 16 21:40:08.290728 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-e32f47cc-212a-46f3-858c-3b7ee9c1425c tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] 10.222.0.16 "POST /compute/v2.1/servers/997f89c5-8086-4a7e-8292-1c8e6a12aaf6/action" status: 202 len: 0 microversion: 2.1 time: 1.905247 Jul 16 21:40:08.291407 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2683/5357] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 21:40:06 2022] POST /compute/v2.1/servers/997f89c5-8086-4a7e-8292-1c8e6a12aaf6/action => generated 0 bytes in 1908 msecs (HTTP/1.1 202) 10 headers in 450 bytes (2 switches on core 0) Jul 16 21:40:09.497587 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-36c8884e-5ba4-40e4-b094-46e28b587ff4 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestVolumeBootPattern-image-snapshot-server-1206539668", "imageRef": "f0740261-4d4d-4fe6-9242-2bb8eea70d14", "flavorRef": "452", "networks": [{"uuid": "94c922c8-150a-417e-8274-7473d6567b26"}]}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:40:09.505308 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_api:SELECT=59,nova_api:INSERT=3 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:40:09.553829 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.network.neutron [None req-36c8884e-5ba4-40e4-b094-46e28b587ff4 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] validate_networks() for [('94c922c8-150a-417e-8274-7473d6567b26', None, None, None, None, None)] {{(pid=138680) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 16 21:40:09.934059 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-36c8884e-5ba4-40e4-b094-46e28b587ff4 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Flavor limits 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 16 21:40:09.934520 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-36c8884e-5ba4-40e4-b094-46e28b587ff4 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Image limits 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 16 21:40:09.935060 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-36c8884e-5ba4-40e4-b094-46e28b587ff4 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Flavor pref 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 16 21:40:09.935478 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-36c8884e-5ba4-40e4-b094-46e28b587ff4 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Image pref 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 16 21:40:09.935907 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-36c8884e-5ba4-40e4-b094-46e28b587ff4 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 16 21:40:09.945069 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-36c8884e-5ba4-40e4-b094-46e28b587ff4 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:40:09.945526 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-36c8884e-5ba4-40e4-b094-46e28b587ff4 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:40:09.946337 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-36c8884e-5ba4-40e4-b094-46e28b587ff4 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:40:09.946786 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-36c8884e-5ba4-40e4-b094-46e28b587ff4 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:40:09.963910 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-36c8884e-5ba4-40e4-b094-46e28b587ff4 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Getting quotas for project b533f85aeddf4d8db05f2c53a61befc3. Resources: {'ram', 'instances', 'cores'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:40:09.970058 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-36c8884e-5ba4-40e4-b094-46e28b587ff4 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Getting quotas for user 62209b9802ab46c3b3d094d4d1ad86fe and project b533f85aeddf4d8db05f2c53a61befc3. Resources: {'ram', 'instances', 'cores'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:40:09.983717 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-36c8884e-5ba4-40e4-b094-46e28b587ff4 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Going to run 1 instances... {{(pid=138680) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 16 21:40:09.986288 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-36c8884e-5ba4-40e4-b094-46e28b587ff4 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] [instance: 1ff78357-ba91-45ba-937e-cf12461b0610] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='volume',device_name='/dev/sda',device_type=None,disk_bus=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id='2dc18b93-7077-47bd-a336-808d90e32062',source_type='snapshot',tag=None,updated_at=,uuid=,volume_id=None,volume_size=1,volume_type=None)] {{(pid=138680) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 16 21:40:10.138716 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-36c8884e-5ba4-40e4-b094-46e28b587ff4 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.646088 Jul 16 21:40:10.139332 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2675/5358] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 21:40:09 2022] POST /compute/v2.1/servers => generated 380 bytes in 647 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 16 21:40:10.152045 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-494817d2-d3f9-432e-b65b-8dc7525f1ce8 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:40:10.152758 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-494817d2-d3f9-432e-b65b-8dc7525f1ce8 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] [instance: 1ff78357-ba91-45ba-937e-cf12461b0610] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:40:10.209926 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-494817d2-d3f9-432e-b65b-8dc7525f1ce8 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '62209b9802ab46c3b3d094d4d1ad86fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b533f85aeddf4d8db05f2c53a61befc3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:40:10.218245 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-494817d2-d3f9-432e-b65b-8dc7525f1ce8 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] 10.222.0.16 "GET /compute/v2.1/servers/1ff78357-ba91-45ba-937e-cf12461b0610" status: 200 len: 1230 microversion: 2.1 time: 0.072538 Jul 16 21:40:10.218835 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2684/5359] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:40:10 2022] GET /compute/v2.1/servers/1ff78357-ba91-45ba-937e-cf12461b0610 => generated 1230 bytes in 74 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:40:11.237671 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-c3c0fb73-6bfd-4282-8bcb-ae1285d97fa9 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:40:11.238419 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-c3c0fb73-6bfd-4282-8bcb-ae1285d97fa9 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] [instance: 1ff78357-ba91-45ba-937e-cf12461b0610] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:40:11.245781 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c3c0fb73-6bfd-4282-8bcb-ae1285d97fa9 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:40:11.246240 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c3c0fb73-6bfd-4282-8bcb-ae1285d97fa9 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:40:11.247035 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c3c0fb73-6bfd-4282-8bcb-ae1285d97fa9 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:40:11.247606 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c3c0fb73-6bfd-4282-8bcb-ae1285d97fa9 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:40:11.315577 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-c3c0fb73-6bfd-4282-8bcb-ae1285d97fa9 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '62209b9802ab46c3b3d094d4d1ad86fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b533f85aeddf4d8db05f2c53a61befc3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:40:11.323348 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-c3c0fb73-6bfd-4282-8bcb-ae1285d97fa9 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] 10.222.0.16 "GET /compute/v2.1/servers/1ff78357-ba91-45ba-937e-cf12461b0610" status: 200 len: 1234 microversion: 2.1 time: 0.091653 Jul 16 21:40:11.323915 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2676/5360] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:40:11 2022] GET /compute/v2.1/servers/1ff78357-ba91-45ba-937e-cf12461b0610 => generated 1234 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:40:12.340384 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-871a2edd-8586-445e-8c90-b2190d259ed3 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:40:12.341017 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-871a2edd-8586-445e-8c90-b2190d259ed3 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] [instance: 1ff78357-ba91-45ba-937e-cf12461b0610] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:40:12.349009 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-871a2edd-8586-445e-8c90-b2190d259ed3 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:40:12.349481 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-871a2edd-8586-445e-8c90-b2190d259ed3 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:40:12.350276 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-871a2edd-8586-445e-8c90-b2190d259ed3 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:40:12.350730 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-871a2edd-8586-445e-8c90-b2190d259ed3 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:40:12.450357 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-871a2edd-8586-445e-8c90-b2190d259ed3 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '62209b9802ab46c3b3d094d4d1ad86fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b533f85aeddf4d8db05f2c53a61befc3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:40:12.458913 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-871a2edd-8586-445e-8c90-b2190d259ed3 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] 10.222.0.16 "GET /compute/v2.1/servers/1ff78357-ba91-45ba-937e-cf12461b0610" status: 200 len: 1300 microversion: 2.1 time: 0.122429 Jul 16 21:40:12.460132 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2685/5361] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:40:12 2022] GET /compute/v2.1/servers/1ff78357-ba91-45ba-937e-cf12461b0610 => generated 1300 bytes in 124 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:40:13.476565 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-64541e16-22c4-4ac4-9145-00a0b8405ea9 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:40:13.477026 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-64541e16-22c4-4ac4-9145-00a0b8405ea9 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] [instance: 1ff78357-ba91-45ba-937e-cf12461b0610] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:40:13.485611 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-64541e16-22c4-4ac4-9145-00a0b8405ea9 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:40:13.486077 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-64541e16-22c4-4ac4-9145-00a0b8405ea9 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:40:13.486862 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-64541e16-22c4-4ac4-9145-00a0b8405ea9 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:40:13.487333 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-64541e16-22c4-4ac4-9145-00a0b8405ea9 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:40:13.747479 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-64541e16-22c4-4ac4-9145-00a0b8405ea9 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '62209b9802ab46c3b3d094d4d1ad86fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b533f85aeddf4d8db05f2c53a61befc3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:40:13.755715 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-64541e16-22c4-4ac4-9145-00a0b8405ea9 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] 10.222.0.16 "GET /compute/v2.1/servers/1ff78357-ba91-45ba-937e-cf12461b0610" status: 200 len: 1342 microversion: 2.1 time: 0.283551 Jul 16 21:40:13.756572 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2677/5362] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:40:13 2022] GET /compute/v2.1/servers/1ff78357-ba91-45ba-937e-cf12461b0610 => generated 1342 bytes in 285 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:40:13.985444 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-6da698d3-bd84-4ae7-8934-7800d6ca7259 req-b4bfcb8b-cd25-42d1-a2a3-7aaff12005a4 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "1ff78357-ba91-45ba-937e-cf12461b0610", "tag": "54a43910-9e17-4f09-9316-144444fbd323"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:40:13.990085 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_api:SELECT=50 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:40:13.997008 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-6da698d3-bd84-4ae7-8934-7800d6ca7259 req-b4bfcb8b-cd25-42d1-a2a3-7aaff12005a4 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:40:13.997752 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-6da698d3-bd84-4ae7-8934-7800d6ca7259 req-b4bfcb8b-cd25-42d1-a2a3-7aaff12005a4 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:40:14.010555 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-6da698d3-bd84-4ae7-8934-7800d6ca7259 req-b4bfcb8b-cd25-42d1-a2a3-7aaff12005a4 service nova] Creating event network-changed:54a43910-9e17-4f09-9316-144444fbd323 for instance 1ff78357-ba91-45ba-937e-cf12461b0610 on nch2-master-1 Jul 16 21:40:14.016970 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-6da698d3-bd84-4ae7-8934-7800d6ca7259 req-b4bfcb8b-cd25-42d1-a2a3-7aaff12005a4 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.036911 Jul 16 21:40:14.017593 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2686/5363] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:40:13 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:40:14.773951 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-89894837-caaa-4f23-929b-f06f6e90b97b tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:40:14.774564 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-89894837-caaa-4f23-929b-f06f6e90b97b tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] [instance: 1ff78357-ba91-45ba-937e-cf12461b0610] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:40:14.783381 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-89894837-caaa-4f23-929b-f06f6e90b97b tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:40:14.787060 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-89894837-caaa-4f23-929b-f06f6e90b97b tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:40:14.787060 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-89894837-caaa-4f23-929b-f06f6e90b97b tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:40:14.787060 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-89894837-caaa-4f23-929b-f06f6e90b97b tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:40:14.918853 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-89894837-caaa-4f23-929b-f06f6e90b97b tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '62209b9802ab46c3b3d094d4d1ad86fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b533f85aeddf4d8db05f2c53a61befc3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:40:14.926984 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-89894837-caaa-4f23-929b-f06f6e90b97b tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] 10.222.0.16 "GET /compute/v2.1/servers/1ff78357-ba91-45ba-937e-cf12461b0610" status: 200 len: 1342 microversion: 2.1 time: 0.157284 Jul 16 21:40:14.927577 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2678/5364] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:40:14 2022] GET /compute/v2.1/servers/1ff78357-ba91-45ba-937e-cf12461b0610 => generated 1342 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:40:15.945914 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-a92bce36-dde2-4328-943d-e781abfeb913 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:40:15.946661 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-a92bce36-dde2-4328-943d-e781abfeb913 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] [instance: 1ff78357-ba91-45ba-937e-cf12461b0610] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:40:15.954631 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a92bce36-dde2-4328-943d-e781abfeb913 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:40:15.955184 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a92bce36-dde2-4328-943d-e781abfeb913 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:40:15.955960 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a92bce36-dde2-4328-943d-e781abfeb913 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:40:15.956406 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a92bce36-dde2-4328-943d-e781abfeb913 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:40:16.092098 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-a92bce36-dde2-4328-943d-e781abfeb913 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '62209b9802ab46c3b3d094d4d1ad86fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b533f85aeddf4d8db05f2c53a61befc3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:40:16.100361 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-a92bce36-dde2-4328-943d-e781abfeb913 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] 10.222.0.16 "GET /compute/v2.1/servers/1ff78357-ba91-45ba-937e-cf12461b0610" status: 200 len: 1342 microversion: 2.1 time: 0.159369 Jul 16 21:40:16.100940 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2687/5365] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:40:15 2022] GET /compute/v2.1/servers/1ff78357-ba91-45ba-937e-cf12461b0610 => generated 1342 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:40:17.117627 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-1195a826-b676-4aa7-8caf-dbdec5d961ba tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:40:17.118322 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-1195a826-b676-4aa7-8caf-dbdec5d961ba tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] [instance: 1ff78357-ba91-45ba-937e-cf12461b0610] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:40:17.126543 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1195a826-b676-4aa7-8caf-dbdec5d961ba tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:40:17.126936 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1195a826-b676-4aa7-8caf-dbdec5d961ba tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:40:17.127749 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1195a826-b676-4aa7-8caf-dbdec5d961ba tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:40:17.128320 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1195a826-b676-4aa7-8caf-dbdec5d961ba tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:40:17.344416 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-1195a826-b676-4aa7-8caf-dbdec5d961ba tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '62209b9802ab46c3b3d094d4d1ad86fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b533f85aeddf4d8db05f2c53a61befc3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:40:17.351201 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-1195a826-b676-4aa7-8caf-dbdec5d961ba tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] 10.222.0.16 "GET /compute/v2.1/servers/1ff78357-ba91-45ba-937e-cf12461b0610" status: 200 len: 1342 microversion: 2.1 time: 0.237657 Jul 16 21:40:17.352402 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2679/5366] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:40:17 2022] GET /compute/v2.1/servers/1ff78357-ba91-45ba-937e-cf12461b0610 => generated 1342 bytes in 240 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:40:18.370417 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-03f5dc63-56c7-4de9-bb31-5143d5d697ba tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:40:18.371556 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-03f5dc63-56c7-4de9-bb31-5143d5d697ba tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] [instance: 1ff78357-ba91-45ba-937e-cf12461b0610] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:40:18.379277 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-03f5dc63-56c7-4de9-bb31-5143d5d697ba tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:40:18.379832 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-03f5dc63-56c7-4de9-bb31-5143d5d697ba tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:40:18.380655 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-03f5dc63-56c7-4de9-bb31-5143d5d697ba tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:40:18.381125 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-03f5dc63-56c7-4de9-bb31-5143d5d697ba tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:40:18.542479 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-03f5dc63-56c7-4de9-bb31-5143d5d697ba tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '62209b9802ab46c3b3d094d4d1ad86fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b533f85aeddf4d8db05f2c53a61befc3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:40:18.551517 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-03f5dc63-56c7-4de9-bb31-5143d5d697ba tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] 10.222.0.16 "GET /compute/v2.1/servers/1ff78357-ba91-45ba-937e-cf12461b0610" status: 200 len: 1342 microversion: 2.1 time: 0.185838 Jul 16 21:40:18.561638 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2688/5367] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:40:18 2022] GET /compute/v2.1/servers/1ff78357-ba91-45ba-937e-cf12461b0610 => generated 1342 bytes in 197 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:40:19.568711 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-8348843a-4d88-429d-8360-edf7662a91c6 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:40:19.569337 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-8348843a-4d88-429d-8360-edf7662a91c6 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] [instance: 1ff78357-ba91-45ba-937e-cf12461b0610] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:40:19.576503 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8348843a-4d88-429d-8360-edf7662a91c6 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:40:19.576963 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8348843a-4d88-429d-8360-edf7662a91c6 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:40:19.577742 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8348843a-4d88-429d-8360-edf7662a91c6 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:40:19.578187 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8348843a-4d88-429d-8360-edf7662a91c6 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:40:19.728941 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-8348843a-4d88-429d-8360-edf7662a91c6 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '62209b9802ab46c3b3d094d4d1ad86fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b533f85aeddf4d8db05f2c53a61befc3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:40:19.736045 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-8348843a-4d88-429d-8360-edf7662a91c6 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] 10.222.0.16 "GET /compute/v2.1/servers/1ff78357-ba91-45ba-937e-cf12461b0610" status: 200 len: 1342 microversion: 2.1 time: 0.171395 Jul 16 21:40:19.736639 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2680/5368] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:40:19 2022] GET /compute/v2.1/servers/1ff78357-ba91-45ba-937e-cf12461b0610 => generated 1342 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:40:20.754875 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-453e9743-74d7-465f-bb35-ff6bb7550b4b tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:40:20.755543 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-453e9743-74d7-465f-bb35-ff6bb7550b4b tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] [instance: 1ff78357-ba91-45ba-937e-cf12461b0610] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:40:20.758433 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell0:SELECT=2 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:40:20.763438 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-453e9743-74d7-465f-bb35-ff6bb7550b4b tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:40:20.763900 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-453e9743-74d7-465f-bb35-ff6bb7550b4b tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:40:20.764666 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-453e9743-74d7-465f-bb35-ff6bb7550b4b tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:40:20.765112 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-453e9743-74d7-465f-bb35-ff6bb7550b4b tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:40:20.918356 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-453e9743-74d7-465f-bb35-ff6bb7550b4b tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '62209b9802ab46c3b3d094d4d1ad86fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b533f85aeddf4d8db05f2c53a61befc3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:40:20.927211 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-453e9743-74d7-465f-bb35-ff6bb7550b4b tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] 10.222.0.16 "GET /compute/v2.1/servers/1ff78357-ba91-45ba-937e-cf12461b0610" status: 200 len: 1388 microversion: 2.1 time: 0.178148 Jul 16 21:40:20.928213 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2689/5369] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:40:20 2022] GET /compute/v2.1/servers/1ff78357-ba91-45ba-937e-cf12461b0610 => generated 1388 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:40:21.944869 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-8494043c-a004-42f7-a89b-0748fa616c64 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:40:21.945452 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-8494043c-a004-42f7-a89b-0748fa616c64 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] [instance: 1ff78357-ba91-45ba-937e-cf12461b0610] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:40:21.947380 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell0:SELECT=2 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:40:21.948099 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell0:SELECT=6 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:40:21.954431 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8494043c-a004-42f7-a89b-0748fa616c64 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:40:21.954887 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8494043c-a004-42f7-a89b-0748fa616c64 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:40:21.955686 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8494043c-a004-42f7-a89b-0748fa616c64 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:40:21.956151 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8494043c-a004-42f7-a89b-0748fa616c64 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:40:22.094356 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-8494043c-a004-42f7-a89b-0748fa616c64 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '62209b9802ab46c3b3d094d4d1ad86fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b533f85aeddf4d8db05f2c53a61befc3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:40:22.102508 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-8494043c-a004-42f7-a89b-0748fa616c64 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] 10.222.0.16 "GET /compute/v2.1/servers/1ff78357-ba91-45ba-937e-cf12461b0610" status: 200 len: 1376 microversion: 2.1 time: 0.161561 Jul 16 21:40:22.106358 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2681/5370] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:40:21 2022] GET /compute/v2.1/servers/1ff78357-ba91-45ba-937e-cf12461b0610 => generated 1376 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:40:23.121650 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-ec343a3a-8b65-4802-b8db-a98cadb715aa tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:40:23.122285 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-ec343a3a-8b65-4802-b8db-a98cadb715aa tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] [instance: 1ff78357-ba91-45ba-937e-cf12461b0610] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:40:23.129450 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ec343a3a-8b65-4802-b8db-a98cadb715aa tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:40:23.129932 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ec343a3a-8b65-4802-b8db-a98cadb715aa tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:40:23.130717 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ec343a3a-8b65-4802-b8db-a98cadb715aa tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:40:23.131209 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ec343a3a-8b65-4802-b8db-a98cadb715aa tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:40:23.149623 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=95,nova_cell1:INSERT=2,nova_cell1:UPDATE=3 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:40:23.277402 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-ec343a3a-8b65-4802-b8db-a98cadb715aa tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '62209b9802ab46c3b3d094d4d1ad86fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b533f85aeddf4d8db05f2c53a61befc3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:40:23.286223 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-ec343a3a-8b65-4802-b8db-a98cadb715aa tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] 10.222.0.16 "GET /compute/v2.1/servers/1ff78357-ba91-45ba-937e-cf12461b0610" status: 200 len: 1376 microversion: 2.1 time: 0.168392 Jul 16 21:40:23.286814 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2690/5371] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:40:23 2022] GET /compute/v2.1/servers/1ff78357-ba91-45ba-937e-cf12461b0610 => generated 1376 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:40:24.130115 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-724c0a57-a06d-4f70-9400-e4119c6f2d76 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Action: 'action', calling method: >, body: {"os-getConsoleOutput": {}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:40:24.131197 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-724c0a57-a06d-4f70-9400-e4119c6f2d76 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] [instance: 2ce56b72-9fe8-41ca-bd6e-31591ef3b494] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:40:24.138446 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-724c0a57-a06d-4f70-9400-e4119c6f2d76 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:40:24.138696 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-724c0a57-a06d-4f70-9400-e4119c6f2d76 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:40:24.139648 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-724c0a57-a06d-4f70-9400-e4119c6f2d76 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:40:24.140112 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-724c0a57-a06d-4f70-9400-e4119c6f2d76 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:40:24.197223 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-724c0a57-a06d-4f70-9400-e4119c6f2d76 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] 10.222.0.16 "POST /compute/v2.1/servers/2ce56b72-9fe8-41ca-bd6e-31591ef3b494/action" status: 200 len: 24545 microversion: 2.1 time: 0.071230 Jul 16 21:40:24.197893 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2682/5372] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 21:40:24 2022] POST /compute/v2.1/servers/2ce56b72-9fe8-41ca-bd6e-31591ef3b494/action => generated 24545 bytes in 73 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 16 21:40:24.303723 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-de9b94c9-f451-49f8-a742-ce3b05ab1db2 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:40:24.304523 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-de9b94c9-f451-49f8-a742-ce3b05ab1db2 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] [instance: 1ff78357-ba91-45ba-937e-cf12461b0610] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:40:24.312358 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-de9b94c9-f451-49f8-a742-ce3b05ab1db2 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:40:24.312902 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-de9b94c9-f451-49f8-a742-ce3b05ab1db2 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:40:24.313652 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-de9b94c9-f451-49f8-a742-ce3b05ab1db2 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:40:24.314102 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-de9b94c9-f451-49f8-a742-ce3b05ab1db2 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:40:24.463711 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-de9b94c9-f451-49f8-a742-ce3b05ab1db2 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '62209b9802ab46c3b3d094d4d1ad86fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b533f85aeddf4d8db05f2c53a61befc3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:40:24.473566 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-de9b94c9-f451-49f8-a742-ce3b05ab1db2 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] 10.222.0.16 "GET /compute/v2.1/servers/1ff78357-ba91-45ba-937e-cf12461b0610" status: 200 len: 1376 microversion: 2.1 time: 0.172661 Jul 16 21:40:24.473566 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2691/5373] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:40:24 2022] GET /compute/v2.1/servers/1ff78357-ba91-45ba-937e-cf12461b0610 => generated 1376 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:40:25.492316 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-c58216e2-da2a-4fc9-a1a1-015c2d6236fe tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:40:25.493749 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-c58216e2-da2a-4fc9-a1a1-015c2d6236fe tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] [instance: 1ff78357-ba91-45ba-937e-cf12461b0610] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:40:25.502423 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c58216e2-da2a-4fc9-a1a1-015c2d6236fe tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:40:25.502889 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c58216e2-da2a-4fc9-a1a1-015c2d6236fe tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:40:25.503706 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c58216e2-da2a-4fc9-a1a1-015c2d6236fe tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:40:25.504172 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c58216e2-da2a-4fc9-a1a1-015c2d6236fe tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:40:25.656820 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-c58216e2-da2a-4fc9-a1a1-015c2d6236fe tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '62209b9802ab46c3b3d094d4d1ad86fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b533f85aeddf4d8db05f2c53a61befc3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:40:25.664898 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-c58216e2-da2a-4fc9-a1a1-015c2d6236fe tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] 10.222.0.16 "GET /compute/v2.1/servers/1ff78357-ba91-45ba-937e-cf12461b0610" status: 200 len: 1376 microversion: 2.1 time: 0.177423 Jul 16 21:40:25.665475 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2683/5374] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:40:25 2022] GET /compute/v2.1/servers/1ff78357-ba91-45ba-937e-cf12461b0610 => generated 1376 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:40:26.693218 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-bddba7ee-a572-45f9-b2cd-5386d6d1c71c tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:40:26.694065 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-bddba7ee-a572-45f9-b2cd-5386d6d1c71c tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] [instance: 1ff78357-ba91-45ba-937e-cf12461b0610] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:40:26.701289 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-bddba7ee-a572-45f9-b2cd-5386d6d1c71c tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:40:26.701776 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-bddba7ee-a572-45f9-b2cd-5386d6d1c71c tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:40:26.702534 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-bddba7ee-a572-45f9-b2cd-5386d6d1c71c tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:40:26.702991 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-bddba7ee-a572-45f9-b2cd-5386d6d1c71c tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:40:26.848121 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-bddba7ee-a572-45f9-b2cd-5386d6d1c71c tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '62209b9802ab46c3b3d094d4d1ad86fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b533f85aeddf4d8db05f2c53a61befc3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:40:26.855664 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-bddba7ee-a572-45f9-b2cd-5386d6d1c71c tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] 10.222.0.16 "GET /compute/v2.1/servers/1ff78357-ba91-45ba-937e-cf12461b0610" status: 200 len: 1376 microversion: 2.1 time: 0.166400 Jul 16 21:40:26.856242 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2692/5375] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:40:26 2022] GET /compute/v2.1/servers/1ff78357-ba91-45ba-937e-cf12461b0610 => generated 1376 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:40:27.874801 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-9e02985a-3c14-4372-8ae1-e81879ab646a tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:40:27.875588 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-9e02985a-3c14-4372-8ae1-e81879ab646a tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] [instance: 1ff78357-ba91-45ba-937e-cf12461b0610] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:40:27.882577 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9e02985a-3c14-4372-8ae1-e81879ab646a tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:40:27.883063 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9e02985a-3c14-4372-8ae1-e81879ab646a tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:40:27.883889 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9e02985a-3c14-4372-8ae1-e81879ab646a tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:40:27.884394 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9e02985a-3c14-4372-8ae1-e81879ab646a tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:40:28.033821 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-9e02985a-3c14-4372-8ae1-e81879ab646a tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '62209b9802ab46c3b3d094d4d1ad86fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b533f85aeddf4d8db05f2c53a61befc3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:40:28.041632 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-9e02985a-3c14-4372-8ae1-e81879ab646a tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] 10.222.0.16 "GET /compute/v2.1/servers/1ff78357-ba91-45ba-937e-cf12461b0610" status: 200 len: 1376 microversion: 2.1 time: 0.170762 Jul 16 21:40:28.042288 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2684/5376] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:40:27 2022] GET /compute/v2.1/servers/1ff78357-ba91-45ba-937e-cf12461b0610 => generated 1376 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:40:29.059689 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-31b5abc6-2b27-4477-b66e-d9033efdaee0 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:40:29.060320 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-31b5abc6-2b27-4477-b66e-d9033efdaee0 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] [instance: 1ff78357-ba91-45ba-937e-cf12461b0610] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:40:29.072721 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-31b5abc6-2b27-4477-b66e-d9033efdaee0 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:40:29.072721 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-31b5abc6-2b27-4477-b66e-d9033efdaee0 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:40:29.072721 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-31b5abc6-2b27-4477-b66e-d9033efdaee0 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:40:29.072721 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-31b5abc6-2b27-4477-b66e-d9033efdaee0 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:40:29.207523 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-31b5abc6-2b27-4477-b66e-d9033efdaee0 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '62209b9802ab46c3b3d094d4d1ad86fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b533f85aeddf4d8db05f2c53a61befc3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:40:29.215104 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-31b5abc6-2b27-4477-b66e-d9033efdaee0 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] 10.222.0.16 "GET /compute/v2.1/servers/1ff78357-ba91-45ba-937e-cf12461b0610" status: 200 len: 1376 microversion: 2.1 time: 0.159327 Jul 16 21:40:29.215628 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2693/5377] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:40:29 2022] GET /compute/v2.1/servers/1ff78357-ba91-45ba-937e-cf12461b0610 => generated 1376 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:40:30.231945 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-fae318a6-df44-4fed-8718-df496927e8fa tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:40:30.232569 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-fae318a6-df44-4fed-8718-df496927e8fa tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] [instance: 1ff78357-ba91-45ba-937e-cf12461b0610] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:40:30.240750 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-fae318a6-df44-4fed-8718-df496927e8fa tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:40:30.241491 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-fae318a6-df44-4fed-8718-df496927e8fa tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:40:30.242321 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-fae318a6-df44-4fed-8718-df496927e8fa tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:40:30.242778 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-fae318a6-df44-4fed-8718-df496927e8fa tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:40:30.246157 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=93,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:40:30.380934 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-fae318a6-df44-4fed-8718-df496927e8fa tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '62209b9802ab46c3b3d094d4d1ad86fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b533f85aeddf4d8db05f2c53a61befc3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:40:30.388847 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-fae318a6-df44-4fed-8718-df496927e8fa tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] 10.222.0.16 "GET /compute/v2.1/servers/1ff78357-ba91-45ba-937e-cf12461b0610" status: 200 len: 1376 microversion: 2.1 time: 0.161028 Jul 16 21:40:30.389439 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2685/5378] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:40:30 2022] GET /compute/v2.1/servers/1ff78357-ba91-45ba-937e-cf12461b0610 => generated 1376 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:40:30.415223 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-2b6c7876-ca98-48b3-a17e-0eabfd3c16c1 tempest-TestNetworkAdvancedServerOps-756251536 tempest-TestNetworkAdvancedServerOps-756251536-project-admin] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:40:30.415853 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-2b6c7876-ca98-48b3-a17e-0eabfd3c16c1 tempest-TestNetworkAdvancedServerOps-756251536 tempest-TestNetworkAdvancedServerOps-756251536-project-admin] [instance: 2ce56b72-9fe8-41ca-bd6e-31591ef3b494] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:40:30.423594 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2b6c7876-ca98-48b3-a17e-0eabfd3c16c1 tempest-TestNetworkAdvancedServerOps-756251536 tempest-TestNetworkAdvancedServerOps-756251536-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:40:30.424060 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2b6c7876-ca98-48b3-a17e-0eabfd3c16c1 tempest-TestNetworkAdvancedServerOps-756251536 tempest-TestNetworkAdvancedServerOps-756251536-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:40:30.424845 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2b6c7876-ca98-48b3-a17e-0eabfd3c16c1 tempest-TestNetworkAdvancedServerOps-756251536 tempest-TestNetworkAdvancedServerOps-756251536-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:40:30.425624 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2b6c7876-ca98-48b3-a17e-0eabfd3c16c1 tempest-TestNetworkAdvancedServerOps-756251536 tempest-TestNetworkAdvancedServerOps-756251536-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:40:30.557667 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-2b6c7876-ca98-48b3-a17e-0eabfd3c16c1 tempest-TestNetworkAdvancedServerOps-756251536 tempest-TestNetworkAdvancedServerOps-756251536-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/2ce56b72-9fe8-41ca-bd6e-31591ef3b494" status: 200 len: 1833 microversion: 2.1 time: 0.146134 Jul 16 21:40:30.558237 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2694/5379] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:40:30 2022] GET /compute/v2.1/servers/2ce56b72-9fe8-41ca-bd6e-31591ef3b494 => generated 1833 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:40:31.407929 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-a2995134-2fd6-4942-94ec-67f6367d1656 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:40:31.408666 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-a2995134-2fd6-4942-94ec-67f6367d1656 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] [instance: 1ff78357-ba91-45ba-937e-cf12461b0610] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:40:31.416338 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a2995134-2fd6-4942-94ec-67f6367d1656 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:40:31.418482 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a2995134-2fd6-4942-94ec-67f6367d1656 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:40:31.418482 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a2995134-2fd6-4942-94ec-67f6367d1656 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:40:31.418482 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a2995134-2fd6-4942-94ec-67f6367d1656 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:40:31.654258 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-a2995134-2fd6-4942-94ec-67f6367d1656 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '62209b9802ab46c3b3d094d4d1ad86fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b533f85aeddf4d8db05f2c53a61befc3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:40:31.664224 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-a2995134-2fd6-4942-94ec-67f6367d1656 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] 10.222.0.16 "GET /compute/v2.1/servers/1ff78357-ba91-45ba-937e-cf12461b0610" status: 200 len: 1376 microversion: 2.1 time: 0.262150 Jul 16 21:40:31.665199 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2686/5380] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:40:31 2022] GET /compute/v2.1/servers/1ff78357-ba91-45ba-937e-cf12461b0610 => generated 1376 bytes in 264 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:40:31.943096 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-cda5d1fa-77d2-4b11-bbf7-fa88721d513d tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:40:31.943606 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-cda5d1fa-77d2-4b11-bbf7-fa88721d513d tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] [instance: 2ce56b72-9fe8-41ca-bd6e-31591ef3b494] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:40:31.946208 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-a67bf7b4-7277-4b45-bf51-ddcd15e4748f req-23931dc4-859f-4902-b28f-8e1c4ca9f161 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "2ce56b72-9fe8-41ca-bd6e-31591ef3b494", "tag": "a75ca41e-859e-4a3f-a17c-93721015537e"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:40:31.954113 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cda5d1fa-77d2-4b11-bbf7-fa88721d513d tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:40:31.954741 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cda5d1fa-77d2-4b11-bbf7-fa88721d513d tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:40:31.954888 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-a67bf7b4-7277-4b45-bf51-ddcd15e4748f req-23931dc4-859f-4902-b28f-8e1c4ca9f161 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:40:31.955215 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-a67bf7b4-7277-4b45-bf51-ddcd15e4748f req-23931dc4-859f-4902-b28f-8e1c4ca9f161 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:40:31.955593 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cda5d1fa-77d2-4b11-bbf7-fa88721d513d tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:40:31.956051 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cda5d1fa-77d2-4b11-bbf7-fa88721d513d tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:40:31.970191 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-a67bf7b4-7277-4b45-bf51-ddcd15e4748f req-23931dc4-859f-4902-b28f-8e1c4ca9f161 service nova] Creating event network-changed:a75ca41e-859e-4a3f-a17c-93721015537e for instance 2ce56b72-9fe8-41ca-bd6e-31591ef3b494 on nch1-master-1 Jul 16 21:40:31.976200 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-a67bf7b4-7277-4b45-bf51-ddcd15e4748f req-23931dc4-859f-4902-b28f-8e1c4ca9f161 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.033731 Jul 16 21:40:31.976822 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2687/5381] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:40:31 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:40:31.990006 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-cda5d1fa-77d2-4b11-bbf7-fa88721d513d tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] [instance: 2ce56b72-9fe8-41ca-bd6e-31591ef3b494] Going to try to terminate instance {{(pid=138681) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 16 21:40:32.262075 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-cda5d1fa-77d2-4b11-bbf7-fa88721d513d tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] 10.222.0.16 "DELETE /compute/v2.1/servers/2ce56b72-9fe8-41ca-bd6e-31591ef3b494" status: 204 len: 0 microversion: 2.1 time: 0.322598 Jul 16 21:40:32.262763 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2695/5382] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:40:31 2022] DELETE /compute/v2.1/servers/2ce56b72-9fe8-41ca-bd6e-31591ef3b494 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 16 21:40:32.272560 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-5f31eb6f-01c2-4d81-8264-22bc550d00fd tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:40:32.273432 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-5f31eb6f-01c2-4d81-8264-22bc550d00fd tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] [instance: 2ce56b72-9fe8-41ca-bd6e-31591ef3b494] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:40:32.280273 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5f31eb6f-01c2-4d81-8264-22bc550d00fd tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:40:32.280732 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5f31eb6f-01c2-4d81-8264-22bc550d00fd tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:40:32.282370 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5f31eb6f-01c2-4d81-8264-22bc550d00fd tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:40:32.283162 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5f31eb6f-01c2-4d81-8264-22bc550d00fd tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:40:32.428422 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-5f31eb6f-01c2-4d81-8264-22bc550d00fd tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '833136cbdeab4b9e8b6dd2d24c7c8753', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '77c9b7e351e24dc88207e9498f75f578', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:40:32.442329 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-5f31eb6f-01c2-4d81-8264-22bc550d00fd tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] 10.222.0.16 "GET /compute/v2.1/servers/2ce56b72-9fe8-41ca-bd6e-31591ef3b494" status: 200 len: 1688 microversion: 2.1 time: 0.173887 Jul 16 21:40:32.443433 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2688/5383] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:40:32 2022] GET /compute/v2.1/servers/2ce56b72-9fe8-41ca-bd6e-31591ef3b494 => generated 1688 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:40:32.682314 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-21227fba-8420-4c8b-ba5b-a32c83d97070 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:40:32.683045 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-21227fba-8420-4c8b-ba5b-a32c83d97070 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] [instance: 1ff78357-ba91-45ba-937e-cf12461b0610] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:40:32.690003 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-21227fba-8420-4c8b-ba5b-a32c83d97070 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:40:32.690464 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-21227fba-8420-4c8b-ba5b-a32c83d97070 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:40:32.691274 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-21227fba-8420-4c8b-ba5b-a32c83d97070 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:40:32.691727 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-21227fba-8420-4c8b-ba5b-a32c83d97070 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:40:32.832128 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-21227fba-8420-4c8b-ba5b-a32c83d97070 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '62209b9802ab46c3b3d094d4d1ad86fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b533f85aeddf4d8db05f2c53a61befc3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:40:32.842157 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-21227fba-8420-4c8b-ba5b-a32c83d97070 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] 10.222.0.16 "GET /compute/v2.1/servers/1ff78357-ba91-45ba-937e-cf12461b0610" status: 200 len: 1376 microversion: 2.1 time: 0.163909 Jul 16 21:40:32.843736 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2696/5384] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:40:32 2022] GET /compute/v2.1/servers/1ff78357-ba91-45ba-937e-cf12461b0610 => generated 1376 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:40:33.148164 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-00c52f5e-cafb-4212-b911-3cbbbb0db5be req-6265e261-0d34-4ba2-b81d-6c817fbffe03 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "1ff78357-ba91-45ba-937e-cf12461b0610", "name": "network-vif-plugged", "status": "completed", "tag": "54a43910-9e17-4f09-9316-144444fbd323"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:40:33.157292 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-00c52f5e-cafb-4212-b911-3cbbbb0db5be req-6265e261-0d34-4ba2-b81d-6c817fbffe03 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:40:33.157977 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-00c52f5e-cafb-4212-b911-3cbbbb0db5be req-6265e261-0d34-4ba2-b81d-6c817fbffe03 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:40:33.169362 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-00c52f5e-cafb-4212-b911-3cbbbb0db5be req-6265e261-0d34-4ba2-b81d-6c817fbffe03 service nova] Creating event network-vif-plugged:54a43910-9e17-4f09-9316-144444fbd323 for instance 1ff78357-ba91-45ba-937e-cf12461b0610 on nch2-master-1 Jul 16 21:40:33.175258 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-00c52f5e-cafb-4212-b911-3cbbbb0db5be req-6265e261-0d34-4ba2-b81d-6c817fbffe03 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.031162 Jul 16 21:40:33.178208 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2689/5385] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:40:33 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 16 21:40:33.461299 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-32d04bfe-8ff3-4002-98ec-f2861bfd9c26 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:40:33.461947 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-32d04bfe-8ff3-4002-98ec-f2861bfd9c26 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] [instance: 2ce56b72-9fe8-41ca-bd6e-31591ef3b494] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:40:33.471051 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-32d04bfe-8ff3-4002-98ec-f2861bfd9c26 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:40:33.471051 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-32d04bfe-8ff3-4002-98ec-f2861bfd9c26 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:40:33.471759 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-32d04bfe-8ff3-4002-98ec-f2861bfd9c26 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:40:33.472213 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-32d04bfe-8ff3-4002-98ec-f2861bfd9c26 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:40:33.685216 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-32d04bfe-8ff3-4002-98ec-f2861bfd9c26 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '833136cbdeab4b9e8b6dd2d24c7c8753', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '77c9b7e351e24dc88207e9498f75f578', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:40:33.693677 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-32d04bfe-8ff3-4002-98ec-f2861bfd9c26 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] 10.222.0.16 "GET /compute/v2.1/servers/2ce56b72-9fe8-41ca-bd6e-31591ef3b494" status: 200 len: 1688 microversion: 2.1 time: 0.236850 Jul 16 21:40:33.695101 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2697/5386] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:40:33 2022] GET /compute/v2.1/servers/2ce56b72-9fe8-41ca-bd6e-31591ef3b494 => generated 1688 bytes in 238 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:40:33.861353 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-8338e385-835b-47f1-9809-ba4b02f8618d tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:40:33.862065 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-8338e385-835b-47f1-9809-ba4b02f8618d tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] [instance: 1ff78357-ba91-45ba-937e-cf12461b0610] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:40:33.872109 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8338e385-835b-47f1-9809-ba4b02f8618d tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:40:33.872109 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8338e385-835b-47f1-9809-ba4b02f8618d tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:40:33.872109 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8338e385-835b-47f1-9809-ba4b02f8618d tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:40:33.872109 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8338e385-835b-47f1-9809-ba4b02f8618d tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:40:34.016433 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-8338e385-835b-47f1-9809-ba4b02f8618d tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '62209b9802ab46c3b3d094d4d1ad86fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b533f85aeddf4d8db05f2c53a61befc3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:40:34.029060 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-8338e385-835b-47f1-9809-ba4b02f8618d tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] 10.222.0.16 "GET /compute/v2.1/servers/1ff78357-ba91-45ba-937e-cf12461b0610" status: 200 len: 1376 microversion: 2.1 time: 0.173648 Jul 16 21:40:34.030455 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2690/5387] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:40:33 2022] GET /compute/v2.1/servers/1ff78357-ba91-45ba-937e-cf12461b0610 => generated 1376 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:40:34.057567 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-2f959c7c-9537-4e29-9692-03c913138bd3 req-320be246-a27e-45cf-9b66-eb5845574758 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "2ce56b72-9fe8-41ca-bd6e-31591ef3b494", "name": "network-vif-deleted", "tag": "a75ca41e-859e-4a3f-a17c-93721015537e"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:40:34.066735 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-2f959c7c-9537-4e29-9692-03c913138bd3 req-320be246-a27e-45cf-9b66-eb5845574758 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:40:34.067228 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-2f959c7c-9537-4e29-9692-03c913138bd3 req-320be246-a27e-45cf-9b66-eb5845574758 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:40:34.077997 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-2f959c7c-9537-4e29-9692-03c913138bd3 req-320be246-a27e-45cf-9b66-eb5845574758 service nova] Creating event network-vif-deleted:a75ca41e-859e-4a3f-a17c-93721015537e for instance 2ce56b72-9fe8-41ca-bd6e-31591ef3b494 on nch1-master-1 Jul 16 21:40:34.084148 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-2f959c7c-9537-4e29-9692-03c913138bd3 req-320be246-a27e-45cf-9b66-eb5845574758 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.030278 Jul 16 21:40:34.084740 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2698/5388] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:40:34 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 16 21:40:34.711705 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-aaec704b-1650-4e37-8f54-09b358e27588 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:40:34.712408 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-aaec704b-1650-4e37-8f54-09b358e27588 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] [instance: 2ce56b72-9fe8-41ca-bd6e-31591ef3b494] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:40:34.718696 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-aaec704b-1650-4e37-8f54-09b358e27588 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:40:34.719961 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-aaec704b-1650-4e37-8f54-09b358e27588 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:40:34.720646 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-aaec704b-1650-4e37-8f54-09b358e27588 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:40:34.721094 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-aaec704b-1650-4e37-8f54-09b358e27588 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:40:34.752622 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-aaec704b-1650-4e37-8f54-09b358e27588 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:40:34.753065 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-aaec704b-1650-4e37-8f54-09b358e27588 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:40:34.753523 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.objects.instance [None req-aaec704b-1650-4e37-8f54-09b358e27588 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lazy-loading 'fault' on Instance uuid 2ce56b72-9fe8-41ca-bd6e-31591ef3b494 {{(pid=138680) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:40:34.796191 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-aaec704b-1650-4e37-8f54-09b358e27588 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '833136cbdeab4b9e8b6dd2d24c7c8753', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '77c9b7e351e24dc88207e9498f75f578', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:40:34.805623 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-aaec704b-1650-4e37-8f54-09b358e27588 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] 10.222.0.16 "GET /compute/v2.1/servers/2ce56b72-9fe8-41ca-bd6e-31591ef3b494" status: 200 len: 1356 microversion: 2.1 time: 0.097842 Jul 16 21:40:34.806314 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2691/5389] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:40:34 2022] GET /compute/v2.1/servers/2ce56b72-9fe8-41ca-bd6e-31591ef3b494 => generated 1356 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:40:35.047911 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-2f8a00c0-13fa-4be2-a534-6aba19e97ab4 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:40:35.048627 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-2f8a00c0-13fa-4be2-a534-6aba19e97ab4 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] [instance: 1ff78357-ba91-45ba-937e-cf12461b0610] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:40:35.055450 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2f8a00c0-13fa-4be2-a534-6aba19e97ab4 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:40:35.073112 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2f8a00c0-13fa-4be2-a534-6aba19e97ab4 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:40:35.073112 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2f8a00c0-13fa-4be2-a534-6aba19e97ab4 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:40:35.073112 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2f8a00c0-13fa-4be2-a534-6aba19e97ab4 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:40:35.208115 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-2f8a00c0-13fa-4be2-a534-6aba19e97ab4 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '62209b9802ab46c3b3d094d4d1ad86fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b533f85aeddf4d8db05f2c53a61befc3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:40:35.215986 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-2f8a00c0-13fa-4be2-a534-6aba19e97ab4 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] 10.222.0.16 "GET /compute/v2.1/servers/1ff78357-ba91-45ba-937e-cf12461b0610" status: 200 len: 1563 microversion: 2.1 time: 0.173826 Jul 16 21:40:35.216558 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2699/5390] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:40:35 2022] GET /compute/v2.1/servers/1ff78357-ba91-45ba-937e-cf12461b0610 => generated 1563 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:40:35.232968 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-df936e19-8d0f-4068-9281-a7bee050ce23 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:40:35.233586 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-df936e19-8d0f-4068-9281-a7bee050ce23 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] [instance: 1ff78357-ba91-45ba-937e-cf12461b0610] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:40:35.241139 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-df936e19-8d0f-4068-9281-a7bee050ce23 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:40:35.241685 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-df936e19-8d0f-4068-9281-a7bee050ce23 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:40:35.245228 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-df936e19-8d0f-4068-9281-a7bee050ce23 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:40:35.245228 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-df936e19-8d0f-4068-9281-a7bee050ce23 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:40:35.377045 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-df936e19-8d0f-4068-9281-a7bee050ce23 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '62209b9802ab46c3b3d094d4d1ad86fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b533f85aeddf4d8db05f2c53a61befc3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:40:35.384810 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-df936e19-8d0f-4068-9281-a7bee050ce23 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] 10.222.0.16 "GET /compute/v2.1/servers/1ff78357-ba91-45ba-937e-cf12461b0610" status: 200 len: 1563 microversion: 2.1 time: 0.155964 Jul 16 21:40:35.385380 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2692/5391] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:40:35 2022] GET /compute/v2.1/servers/1ff78357-ba91-45ba-937e-cf12461b0610 => generated 1563 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:40:35.454185 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-ef3a3d17-f556-435d-95f7-567ef19e85b6 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:40:35.456081 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-ef3a3d17-f556-435d-95f7-567ef19e85b6 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] [instance: 1ff78357-ba91-45ba-937e-cf12461b0610] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:40:35.460949 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ef3a3d17-f556-435d-95f7-567ef19e85b6 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:40:35.461417 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ef3a3d17-f556-435d-95f7-567ef19e85b6 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:40:35.462280 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ef3a3d17-f556-435d-95f7-567ef19e85b6 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:40:35.462722 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ef3a3d17-f556-435d-95f7-567ef19e85b6 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:40:35.489330 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-ef3a3d17-f556-435d-95f7-567ef19e85b6 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] [instance: 1ff78357-ba91-45ba-937e-cf12461b0610] Going to try to terminate instance {{(pid=138681) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 16 21:40:35.804667 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-ef3a3d17-f556-435d-95f7-567ef19e85b6 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] 10.222.0.16 "DELETE /compute/v2.1/servers/1ff78357-ba91-45ba-937e-cf12461b0610" status: 204 len: 0 microversion: 2.1 time: 0.354640 Jul 16 21:40:35.805266 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2700/5392] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:40:35 2022] DELETE /compute/v2.1/servers/1ff78357-ba91-45ba-937e-cf12461b0610 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 16 21:40:35.814111 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-5a6eefb5-993d-4de1-a4c9-b3cc7e1a8e88 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:40:35.814723 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-5a6eefb5-993d-4de1-a4c9-b3cc7e1a8e88 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] [instance: 1ff78357-ba91-45ba-937e-cf12461b0610] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:40:35.820944 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5a6eefb5-993d-4de1-a4c9-b3cc7e1a8e88 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:40:35.821399 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5a6eefb5-993d-4de1-a4c9-b3cc7e1a8e88 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:40:35.822157 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5a6eefb5-993d-4de1-a4c9-b3cc7e1a8e88 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:40:35.822595 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5a6eefb5-993d-4de1-a4c9-b3cc7e1a8e88 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:40:35.832537 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-b7f86c70-878c-48e5-8028-5326da148090 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:40:35.833293 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-b7f86c70-878c-48e5-8028-5326da148090 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] [instance: 2ce56b72-9fe8-41ca-bd6e-31591ef3b494] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:40:35.839667 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b7f86c70-878c-48e5-8028-5326da148090 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:40:35.840133 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b7f86c70-878c-48e5-8028-5326da148090 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:40:35.840915 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b7f86c70-878c-48e5-8028-5326da148090 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:40:35.841416 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b7f86c70-878c-48e5-8028-5326da148090 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:40:35.858491 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-b7f86c70-878c-48e5-8028-5326da148090 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] HTTP exception thrown: Instance 2ce56b72-9fe8-41ca-bd6e-31591ef3b494 could not be found. Jul 16 21:40:35.863603 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-b7f86c70-878c-48e5-8028-5326da148090 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Request method failure captured: Jul 16 21:40:35.863603 ncdv-master-1 devstack@n-api.service[138681]: request: GET /compute/v2.1/servers/2ce56b72-9fe8-41ca-bd6e-31591ef3b494 HTTP/1.1 Jul 16 21:40:35.863603 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:40:35.863603 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:40:35.863603 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:40:35.863603 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 21:40:35.863603 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:40:35.863603 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:40:35.863603 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:40:35.863603 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:40:35.863603 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0y_J0q_hylsRBv1tuKDuvEX4h0-KJOaBkYlsP5Ic98XDN3p0UKvwASS-7n2zGgAIPr7rp74I-PGAzLIn5xeDmpiRzwuaSoKYKWvZ4y05aUl8opleXxkVzk0FUY6GEflOf-1AEqT9qASXsqi7YhmGfOHelDZ7TomdNPlQ8IF_8wXqbKY Jul 16 21:40:35.863603 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:40:35.863603 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:40:35.863603 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:40:35.863603 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:40:35.863603 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:40:35.863603 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:40:35.863603 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: 77c9b7e351e24dc88207e9498f75f578 Jul 16 21:40:35.863603 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-TestNetworkAdvancedServerOps-375332741 Jul 16 21:40:35.863603 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:40:35.863603 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:40:35.863603 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/77c9b7e351e24dc88207e9498f75f578"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/77c9b7e351e24dc88207e9498f75f578"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_77c9b7e351e24dc88207e9498f75f578"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/77c9b7e351e24dc88207e9498f75f578"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:40:35.863603 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-TestNetworkAdvancedServerOps-375332741 Jul 16 21:40:35.863603 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: 77c9b7e351e24dc88207e9498f75f578 Jul 16 21:40:35.863603 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-TestNetworkAdvancedServerOps-375332741 Jul 16 21:40:35.863603 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-TestNetworkAdvancedServerOps-375332741-project Jul 16 21:40:35.863603 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:40:35.863603 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:40:35.863603 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: 833136cbdeab4b9e8b6dd2d24c7c8753 Jul 16 21:40:35.863603 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-TestNetworkAdvancedServerOps-375332741-project Jul 16 21:40:35.863603 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:40:35.863603 ncdv-master-1 devstack@n-api.service[138681]: exception: Instance 2ce56b72-9fe8-41ca-bd6e-31591ef3b494 could not be found. Jul 16 21:40:35.863603 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': '2ce56b72-9fe8-41ca-bd6e-31591ef3b494'} Jul 16 21:40:35.863603 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:40:35.863603 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:40:35.863603 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:40:35.863603 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:40:35.863603 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:40:35.863603 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:40:35.863603 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:40:35.863603 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:40:35.863603 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:40:35.863603 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:40:35.863603 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 2ce56b72-9fe8-41ca-bd6e-31591ef3b494 could not be found. Jul 16 21:40:35.863603 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:40:35.863603 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:40:35.863603 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:40:35.863603 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:40:35.863603 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:40:35.863603 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:40:35.863603 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:40:35.863603 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:40:35.863603 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:40:35.863603 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:40:35.863603 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:40:35.863603 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:40:35.863603 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:40:35.863603 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:40:35.863603 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:40:35.863603 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:40:35.863603 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 2ce56b72-9fe8-41ca-bd6e-31591ef3b494 could not be found. Jul 16 21:40:35.863603 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:40:35.863603 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:40:35.863603 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:40:35.868863 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:40:35.868863 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:40:35.868863 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:40:35.868863 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:40:35.868863 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:40:35.868863 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 2ce56b72-9fe8-41ca-bd6e-31591ef3b494 could not be found. Jul 16 21:40:35.868863 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:40:35.868863 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-b7f86c70-878c-48e5-8028-5326da148090 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Returning 404 to user: Instance 2ce56b72-9fe8-41ca-bd6e-31591ef3b494 could not be found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:40:35.868863 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-b7f86c70-878c-48e5-8028-5326da148090 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] 10.222.0.16 "GET /compute/v2.1/servers/2ce56b72-9fe8-41ca-bd6e-31591ef3b494" status: 404 len: 111 microversion: 2.1 time: 0.036202 Jul 16 21:40:35.868863 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2701/5393] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:40:35 2022] GET /compute/v2.1/servers/2ce56b72-9fe8-41ca-bd6e-31591ef3b494 => generated 111 bytes in 38 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Jul 16 21:40:35.973459 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-5a6eefb5-993d-4de1-a4c9-b3cc7e1a8e88 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '62209b9802ab46c3b3d094d4d1ad86fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b533f85aeddf4d8db05f2c53a61befc3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:40:35.983579 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-5a6eefb5-993d-4de1-a4c9-b3cc7e1a8e88 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] 10.222.0.16 "GET /compute/v2.1/servers/1ff78357-ba91-45ba-937e-cf12461b0610" status: 200 len: 1569 microversion: 2.1 time: 0.173844 Jul 16 21:40:35.984997 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2693/5394] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:40:35 2022] GET /compute/v2.1/servers/1ff78357-ba91-45ba-937e-cf12461b0610 => generated 1569 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:40:37.004940 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-1703d99e-9840-46d6-9bf4-4df41c4879e0 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:40:37.007905 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-1703d99e-9840-46d6-9bf4-4df41c4879e0 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] [instance: 1ff78357-ba91-45ba-937e-cf12461b0610] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:40:37.017453 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1703d99e-9840-46d6-9bf4-4df41c4879e0 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:40:37.019395 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1703d99e-9840-46d6-9bf4-4df41c4879e0 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:40:37.019395 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1703d99e-9840-46d6-9bf4-4df41c4879e0 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:40:37.019395 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1703d99e-9840-46d6-9bf4-4df41c4879e0 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:40:37.156884 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-1703d99e-9840-46d6-9bf4-4df41c4879e0 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '62209b9802ab46c3b3d094d4d1ad86fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b533f85aeddf4d8db05f2c53a61befc3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:40:37.164668 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-1703d99e-9840-46d6-9bf4-4df41c4879e0 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] 10.222.0.16 "GET /compute/v2.1/servers/1ff78357-ba91-45ba-937e-cf12461b0610" status: 200 len: 1569 microversion: 2.1 time: 0.165298 Jul 16 21:40:37.165236 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2702/5395] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:40:36 2022] GET /compute/v2.1/servers/1ff78357-ba91-45ba-937e-cf12461b0610 => generated 1569 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:40:37.244024 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-017cca1f-5b10-4bce-b4ef-6be9ff258823 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:40:37.245132 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-017cca1f-5b10-4bce-b4ef-6be9ff258823 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] [instance: 21fc83e5-8f03-4930-93b1-0196f7327b3e] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:40:37.253812 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-017cca1f-5b10-4bce-b4ef-6be9ff258823 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:40:37.254363 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-017cca1f-5b10-4bce-b4ef-6be9ff258823 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:40:37.255184 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-017cca1f-5b10-4bce-b4ef-6be9ff258823 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:40:37.255636 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-017cca1f-5b10-4bce-b4ef-6be9ff258823 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:40:37.358790 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.objects.instance [None req-017cca1f-5b10-4bce-b4ef-6be9ff258823 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Lazy-loading 'services' on Instance uuid 21fc83e5-8f03-4930-93b1-0196f7327b3e {{(pid=138680) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:40:37.605228 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-017cca1f-5b10-4bce-b4ef-6be9ff258823 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] 10.222.0.16 "DELETE /compute/v2.1/servers/21fc83e5-8f03-4930-93b1-0196f7327b3e/os-volume_attachments/cf391899-1c79-49a5-8aea-2e9463b0c6f8" status: 202 len: 0 microversion: 2.1 time: 0.365290 Jul 16 21:40:37.607324 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2694/5396] 10.222.0.16 () {58 vars in 1415 bytes} [Sat Jul 16 21:40:37 2022] DELETE /compute/v2.1/servers/21fc83e5-8f03-4930-93b1-0196f7327b3e/os-volume_attachments/cf391899-1c79-49a5-8aea-2e9463b0c6f8 => generated 0 bytes in 368 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 21:40:38.115596 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-fbd6a9c0-8ed4-481c-ac17-bf6004f1b3fb req-7cb154c2-9d59-4fc7-8928-c190def8cfd5 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "1ff78357-ba91-45ba-937e-cf12461b0610", "name": "network-vif-deleted", "tag": "54a43910-9e17-4f09-9316-144444fbd323"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:40:38.126366 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-fbd6a9c0-8ed4-481c-ac17-bf6004f1b3fb req-7cb154c2-9d59-4fc7-8928-c190def8cfd5 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:40:38.126741 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-fbd6a9c0-8ed4-481c-ac17-bf6004f1b3fb req-7cb154c2-9d59-4fc7-8928-c190def8cfd5 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:40:38.142105 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-fbd6a9c0-8ed4-481c-ac17-bf6004f1b3fb req-7cb154c2-9d59-4fc7-8928-c190def8cfd5 service nova] Creating event network-vif-deleted:54a43910-9e17-4f09-9316-144444fbd323 for instance 1ff78357-ba91-45ba-937e-cf12461b0610 on nch2-master-1 Jul 16 21:40:38.147545 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-fbd6a9c0-8ed4-481c-ac17-bf6004f1b3fb req-7cb154c2-9d59-4fc7-8928-c190def8cfd5 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.036012 Jul 16 21:40:38.152094 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2703/5397] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:40:38 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:40:38.200567 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-8d38ec22-2e61-4cc5-b272-78c297529438 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:40:38.201532 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-8d38ec22-2e61-4cc5-b272-78c297529438 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] [instance: 1ff78357-ba91-45ba-937e-cf12461b0610] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:40:38.223347 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8d38ec22-2e61-4cc5-b272-78c297529438 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:40:38.223810 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8d38ec22-2e61-4cc5-b272-78c297529438 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:40:38.228768 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8d38ec22-2e61-4cc5-b272-78c297529438 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:40:38.229236 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8d38ec22-2e61-4cc5-b272-78c297529438 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:40:38.365606 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-8d38ec22-2e61-4cc5-b272-78c297529438 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '62209b9802ab46c3b3d094d4d1ad86fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b533f85aeddf4d8db05f2c53a61befc3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:40:38.373881 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-8d38ec22-2e61-4cc5-b272-78c297529438 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] 10.222.0.16 "GET /compute/v2.1/servers/1ff78357-ba91-45ba-937e-cf12461b0610" status: 200 len: 1361 microversion: 2.1 time: 0.194797 Jul 16 21:40:38.376012 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2695/5398] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:40:38 2022] GET /compute/v2.1/servers/1ff78357-ba91-45ba-937e-cf12461b0610 => generated 1361 bytes in 197 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:40:38.749564 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-f2bcf22b-2958-4a07-89ab-e28dc80050c7 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:40:38.751131 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-f2bcf22b-2958-4a07-89ab-e28dc80050c7 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] [instance: 21fc83e5-8f03-4930-93b1-0196f7327b3e] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:40:38.757957 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f2bcf22b-2958-4a07-89ab-e28dc80050c7 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:40:38.759022 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f2bcf22b-2958-4a07-89ab-e28dc80050c7 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:40:38.760001 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f2bcf22b-2958-4a07-89ab-e28dc80050c7 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:40:38.760939 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f2bcf22b-2958-4a07-89ab-e28dc80050c7 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:40:38.792829 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-f2bcf22b-2958-4a07-89ab-e28dc80050c7 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] [instance: 21fc83e5-8f03-4930-93b1-0196f7327b3e] Going to try to terminate instance {{(pid=138681) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 16 21:40:38.849208 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=95,nova_cell1:UPDATE=3,nova_cell1:INSERT=2 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:40:39.177899 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-f2bcf22b-2958-4a07-89ab-e28dc80050c7 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] 10.222.0.16 "DELETE /compute/v2.1/servers/21fc83e5-8f03-4930-93b1-0196f7327b3e" status: 204 len: 0 microversion: 2.1 time: 0.420966 Jul 16 21:40:39.182710 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2704/5399] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:40:38 2022] DELETE /compute/v2.1/servers/21fc83e5-8f03-4930-93b1-0196f7327b3e => generated 0 bytes in 436 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 16 21:40:39.189129 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-04eccaa4-7424-4833-999f-0acd1327e432 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:40:39.189749 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-04eccaa4-7424-4833-999f-0acd1327e432 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] [instance: 21fc83e5-8f03-4930-93b1-0196f7327b3e] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:40:39.198659 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-04eccaa4-7424-4833-999f-0acd1327e432 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:40:39.199135 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-04eccaa4-7424-4833-999f-0acd1327e432 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:40:39.200208 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-04eccaa4-7424-4833-999f-0acd1327e432 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:40:39.200650 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-04eccaa4-7424-4833-999f-0acd1327e432 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:40:39.393958 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-c6245593-c092-473c-96c9-d21870a4df80 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:40:39.394694 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-c6245593-c092-473c-96c9-d21870a4df80 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] [instance: 1ff78357-ba91-45ba-937e-cf12461b0610] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:40:39.408518 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c6245593-c092-473c-96c9-d21870a4df80 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:40:39.408997 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c6245593-c092-473c-96c9-d21870a4df80 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:40:39.409806 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c6245593-c092-473c-96c9-d21870a4df80 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:40:39.411070 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c6245593-c092-473c-96c9-d21870a4df80 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:40:39.449933 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c6245593-c092-473c-96c9-d21870a4df80 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:40:39.450591 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c6245593-c092-473c-96c9-d21870a4df80 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:40:39.451105 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.objects.instance [None req-c6245593-c092-473c-96c9-d21870a4df80 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lazy-loading 'fault' on Instance uuid 1ff78357-ba91-45ba-937e-cf12461b0610 {{(pid=138681) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:40:39.521562 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-04eccaa4-7424-4833-999f-0acd1327e432 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '95cd6b54d8164de1a524b407a1852ebd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '56b8099d9406431284b4dd58b5c16359', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:40:39.538609 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-04eccaa4-7424-4833-999f-0acd1327e432 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] 10.222.0.16 "GET /compute/v2.1/servers/21fc83e5-8f03-4930-93b1-0196f7327b3e" status: 200 len: 1512 microversion: 2.1 time: 0.354390 Jul 16 21:40:39.540371 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-c6245593-c092-473c-96c9-d21870a4df80 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '62209b9802ab46c3b3d094d4d1ad86fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b533f85aeddf4d8db05f2c53a61befc3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:40:39.552811 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2696/5400] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:40:39 2022] GET /compute/v2.1/servers/21fc83e5-8f03-4930-93b1-0196f7327b3e => generated 1512 bytes in 367 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:40:39.557019 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-c6245593-c092-473c-96c9-d21870a4df80 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] 10.222.0.16 "GET /compute/v2.1/servers/1ff78357-ba91-45ba-937e-cf12461b0610" status: 200 len: 1366 microversion: 2.1 time: 0.169631 Jul 16 21:40:39.559148 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2705/5401] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:40:39 2022] GET /compute/v2.1/servers/1ff78357-ba91-45ba-937e-cf12461b0610 => generated 1366 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:40:40.467723 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-64c34536-0a76-4e54-8e05-a558135da8f1 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Calling method '.version_select at 0x7fdd9db920d0>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:40:40.475179 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_api:SELECT=57,nova_api:INSERT=3 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:40:40.550696 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-64c34536-0a76-4e54-8e05-a558135da8f1 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] 10.222.0.16 "DELETE /compute/v2.1/os-keypairs/tempest-TestNetworkAdvancedServerOps-1632011611" status: 202 len: 0 microversion: 2.1 time: 0.086613 Jul 16 21:40:40.551529 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2697/5402] 10.222.0.16 () {58 vars in 1283 bytes} [Sat Jul 16 21:40:40 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestNetworkAdvancedServerOps-1632011611 => generated 0 bytes in 88 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 21:40:40.557185 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-be8557d3-f6fa-4711-abc1-68f926d354aa tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:40:40.557811 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-be8557d3-f6fa-4711-abc1-68f926d354aa tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] [instance: 21fc83e5-8f03-4930-93b1-0196f7327b3e] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:40:40.565739 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-be8557d3-f6fa-4711-abc1-68f926d354aa tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:40:40.566217 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-be8557d3-f6fa-4711-abc1-68f926d354aa tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:40:40.567112 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-be8557d3-f6fa-4711-abc1-68f926d354aa tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:40:40.567600 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-be8557d3-f6fa-4711-abc1-68f926d354aa tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:40:40.589868 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-cb4dee69-7d52-4bef-a95d-0c7f641cd1d2 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:40:40.598225 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-cb4dee69-7d52-4bef-a95d-0c7f641cd1d2 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] [instance: 1ff78357-ba91-45ba-937e-cf12461b0610] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:40:40.598225 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cb4dee69-7d52-4bef-a95d-0c7f641cd1d2 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:40:40.598225 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cb4dee69-7d52-4bef-a95d-0c7f641cd1d2 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:40:40.598694 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cb4dee69-7d52-4bef-a95d-0c7f641cd1d2 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:40:40.599350 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cb4dee69-7d52-4bef-a95d-0c7f641cd1d2 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:40:40.614389 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-cb4dee69-7d52-4bef-a95d-0c7f641cd1d2 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] HTTP exception thrown: Instance 1ff78357-ba91-45ba-937e-cf12461b0610 could not be found. Jul 16 21:40:40.616239 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-cb4dee69-7d52-4bef-a95d-0c7f641cd1d2 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Request method failure captured: Jul 16 21:40:40.616239 ncdv-master-1 devstack@n-api.service[138680]: request: GET /compute/v2.1/servers/1ff78357-ba91-45ba-937e-cf12461b0610 HTTP/1.1 Jul 16 21:40:40.616239 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:40:40.616239 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:40:40.616239 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:40:40.616239 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 0 Jul 16 21:40:40.616239 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:40:40.616239 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:40:40.616239 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:40:40.616239 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:40:40.616239 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0y_WdH7eI0DbXzeyrxNR2zaPLU2Z0T2hgcgJaNL5zKj0IOkua_mwRXGhv8UgaywTvIB56iq06rxK294Knh0aCrbToiJY0kPn8yPuGVzddBFopXnN-hL_u7FbpY8zvcJRiL0zpWIKlPhQExzLv95jQK7KF6D9Y-KawL2vbM6jbQCfNnU Jul 16 21:40:40.616239 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:40:40.616239 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:40:40.616239 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:40:40.616239 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:40:40.616239 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:40:40.616239 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:40:40.616239 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: b533f85aeddf4d8db05f2c53a61befc3 Jul 16 21:40:40.616239 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-TestVolumeBootPattern-1625426921 Jul 16 21:40:40.616239 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:40:40.616239 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:40:40.616239 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/b533f85aeddf4d8db05f2c53a61befc3"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/b533f85aeddf4d8db05f2c53a61befc3"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_b533f85aeddf4d8db05f2c53a61befc3"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/b533f85aeddf4d8db05f2c53a61befc3"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:40:40.616239 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-TestVolumeBootPattern-1625426921 Jul 16 21:40:40.616239 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: b533f85aeddf4d8db05f2c53a61befc3 Jul 16 21:40:40.616239 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-TestVolumeBootPattern-1625426921 Jul 16 21:40:40.616239 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-TestVolumeBootPattern-1625426921-project Jul 16 21:40:40.616239 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:40:40.616239 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:40:40.616239 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: 62209b9802ab46c3b3d094d4d1ad86fe Jul 16 21:40:40.616239 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-TestVolumeBootPattern-1625426921-project Jul 16 21:40:40.616239 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 21:40:40.616239 ncdv-master-1 devstack@n-api.service[138680]: exception: Instance 1ff78357-ba91-45ba-937e-cf12461b0610 could not be found. Jul 16 21:40:40.616239 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': '1ff78357-ba91-45ba-937e-cf12461b0610'} Jul 16 21:40:40.616239 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:40:40.616239 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:40:40.616239 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:40:40.616239 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:40:40.616239 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:40:40.616239 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:40:40.616239 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:40:40.616239 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:40:40.616239 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:40:40.616239 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:40:40.616239 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 1ff78357-ba91-45ba-937e-cf12461b0610 could not be found. Jul 16 21:40:40.616239 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:40:40.616239 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:40:40.616239 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:40:40.616239 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:40:40.616239 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:40:40.616239 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:40:40.616239 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:40:40.616239 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:40:40.616239 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:40:40.616239 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:40:40.616239 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:40:40.616239 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:40:40.616239 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:40:40.616239 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:40:40.616239 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:40:40.616239 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:40:40.616239 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 1ff78357-ba91-45ba-937e-cf12461b0610 could not be found. Jul 16 21:40:40.616239 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:40:40.616239 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:40:40.616239 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:40:40.621524 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:40:40.621524 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:40:40.621524 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:40:40.621524 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:40:40.621524 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:40:40.621524 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 1ff78357-ba91-45ba-937e-cf12461b0610 could not be found. Jul 16 21:40:40.621524 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:40:40.621524 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-cb4dee69-7d52-4bef-a95d-0c7f641cd1d2 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Returning 404 to user: Instance 1ff78357-ba91-45ba-937e-cf12461b0610 could not be found. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:40:40.622767 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-cb4dee69-7d52-4bef-a95d-0c7f641cd1d2 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] 10.222.0.16 "GET /compute/v2.1/servers/1ff78357-ba91-45ba-937e-cf12461b0610" status: 404 len: 111 microversion: 2.1 time: 0.044095 Jul 16 21:40:40.623587 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2698/5403] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:40:40 2022] GET /compute/v2.1/servers/1ff78357-ba91-45ba-937e-cf12461b0610 => generated 111 bytes in 45 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:40:40.629083 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-b69e445b-de07-4386-83ed-6616f9607590 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Action: 'create', calling method: .version_select at 0x7fdda2a98f70>, body: {"keypair": {"name": "tempest-TestNetworkAdvancedServerOps-1439709083", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBIawD6w1EpCjloeYhCCbA0+cQgXya2lopqVvqE+DlKvkTNr5MMm77h3pqHX8PukQ+SEFWO5EslVV7JeLkK8nRkqyyyYebF8jtF1pEJLtKrscWlWSHJanuriam9p6uSPaTw=="}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:40:40.645928 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-b69e445b-de07-4386-83ed-6616f9607590 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Getting quotas for project 77c9b7e351e24dc88207e9498f75f578. Resources: {'key_pairs'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:40:40.651327 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-b69e445b-de07-4386-83ed-6616f9607590 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Getting quotas for user 833136cbdeab4b9e8b6dd2d24c7c8753 and project 77c9b7e351e24dc88207e9498f75f578. Resources: {'key_pairs'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:40:40.718549 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-be8557d3-f6fa-4711-abc1-68f926d354aa tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '95cd6b54d8164de1a524b407a1852ebd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '56b8099d9406431284b4dd58b5c16359', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:40:40.726648 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-be8557d3-f6fa-4711-abc1-68f926d354aa tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] 10.222.0.16 "GET /compute/v2.1/servers/21fc83e5-8f03-4930-93b1-0196f7327b3e" status: 200 len: 1512 microversion: 2.1 time: 0.173228 Jul 16 21:40:40.727667 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2706/5404] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:40:40 2022] GET /compute/v2.1/servers/21fc83e5-8f03-4930-93b1-0196f7327b3e => generated 1512 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:40:40.760853 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-b69e445b-de07-4386-83ed-6616f9607590 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] 10.222.0.16 "POST /compute/v2.1/os-keypairs" status: 200 len: 407 microversion: 2.1 time: 0.136006 Jul 16 21:40:40.761467 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2699/5405] 10.222.0.16 () {60 vars in 1158 bytes} [Sat Jul 16 21:40:40 2022] POST /compute/v2.1/os-keypairs => generated 407 bytes in 137 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:40:41.227349 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-fe4c4f6c-1cdc-4848-b9ea-a8efe83c1125 req-c2f4ce99-53b3-4c7d-8e75-a9c3b16d9c8d service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "21fc83e5-8f03-4930-93b1-0196f7327b3e", "name": "network-vif-deleted", "tag": "617271fa-0e6f-44d7-82b2-3da8cab88742"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:40:41.236326 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-fe4c4f6c-1cdc-4848-b9ea-a8efe83c1125 req-c2f4ce99-53b3-4c7d-8e75-a9c3b16d9c8d service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:40:41.236821 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-fe4c4f6c-1cdc-4848-b9ea-a8efe83c1125 req-c2f4ce99-53b3-4c7d-8e75-a9c3b16d9c8d service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:40:41.248574 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-fe4c4f6c-1cdc-4848-b9ea-a8efe83c1125 req-c2f4ce99-53b3-4c7d-8e75-a9c3b16d9c8d service nova] Creating event network-vif-deleted:617271fa-0e6f-44d7-82b2-3da8cab88742 for instance 21fc83e5-8f03-4930-93b1-0196f7327b3e on nch1-master-1 Jul 16 21:40:41.254270 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-fe4c4f6c-1cdc-4848-b9ea-a8efe83c1125 req-c2f4ce99-53b3-4c7d-8e75-a9c3b16d9c8d service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.031856 Jul 16 21:40:41.255095 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2707/5406] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:40:41 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:40:41.744886 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-a6958de9-dc59-4ff3-b862-a7ba2e97cc33 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:40:41.746217 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-a6958de9-dc59-4ff3-b862-a7ba2e97cc33 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] [instance: 21fc83e5-8f03-4930-93b1-0196f7327b3e] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:40:41.754536 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a6958de9-dc59-4ff3-b862-a7ba2e97cc33 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:40:41.754999 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a6958de9-dc59-4ff3-b862-a7ba2e97cc33 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:40:41.755817 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a6958de9-dc59-4ff3-b862-a7ba2e97cc33 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:40:41.756387 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a6958de9-dc59-4ff3-b862-a7ba2e97cc33 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:40:41.792158 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a6958de9-dc59-4ff3-b862-a7ba2e97cc33 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:40:41.792620 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a6958de9-dc59-4ff3-b862-a7ba2e97cc33 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:40:41.793095 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.objects.instance [None req-a6958de9-dc59-4ff3-b862-a7ba2e97cc33 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Lazy-loading 'fault' on Instance uuid 21fc83e5-8f03-4930-93b1-0196f7327b3e {{(pid=138680) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:40:41.852781 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-a6958de9-dc59-4ff3-b862-a7ba2e97cc33 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '95cd6b54d8164de1a524b407a1852ebd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '56b8099d9406431284b4dd58b5c16359', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:40:41.862110 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-a6958de9-dc59-4ff3-b862-a7ba2e97cc33 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] 10.222.0.16 "GET /compute/v2.1/servers/21fc83e5-8f03-4930-93b1-0196f7327b3e" status: 200 len: 1308 microversion: 2.1 time: 0.120824 Jul 16 21:40:41.862780 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2700/5407] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:40:41 2022] GET /compute/v2.1/servers/21fc83e5-8f03-4930-93b1-0196f7327b3e => generated 1308 bytes in 122 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:40:42.890258 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-b7413312-a501-4910-8385-e97d9b653f27 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:40:42.891583 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-b7413312-a501-4910-8385-e97d9b653f27 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] [instance: 21fc83e5-8f03-4930-93b1-0196f7327b3e] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:40:42.899102 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b7413312-a501-4910-8385-e97d9b653f27 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:40:42.899606 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b7413312-a501-4910-8385-e97d9b653f27 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:40:42.900375 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b7413312-a501-4910-8385-e97d9b653f27 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:40:42.900824 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b7413312-a501-4910-8385-e97d9b653f27 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:40:42.917353 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-b7413312-a501-4910-8385-e97d9b653f27 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] HTTP exception thrown: Instance 21fc83e5-8f03-4930-93b1-0196f7327b3e could not be found. Jul 16 21:40:42.918980 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-b7413312-a501-4910-8385-e97d9b653f27 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Request method failure captured: Jul 16 21:40:42.918980 ncdv-master-1 devstack@n-api.service[138681]: request: GET /compute/v2.1/servers/21fc83e5-8f03-4930-93b1-0196f7327b3e HTTP/1.1 Jul 16 21:40:42.918980 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:40:42.918980 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:40:42.918980 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:40:42.918980 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 21:40:42.918980 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:40:42.918980 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:40:42.918980 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:40:42.918980 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:40:42.918980 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0y9Z2lw1KYjDsPS13NZHwxrz3jgJvrjI-IVHcVBEZED44KX5zFVfR6bb6-Pq4i-X0lnxFMPshF32DM5zlyWDG4Y42M1U2GEvd5SuvwIhXVwNJi1rFO6f_DEcjyEt99qd7XA56BqBMZam1ygGWlE86E4Kqh4PCcdasDMuFL2uSU2qKTg Jul 16 21:40:42.918980 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:40:42.918980 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:40:42.918980 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:40:42.918980 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:40:42.918980 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:40:42.918980 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:40:42.918980 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: 56b8099d9406431284b4dd58b5c16359 Jul 16 21:40:42.918980 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-VolumesSnapshotTestJSON-1954720325 Jul 16 21:40:42.918980 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:40:42.918980 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:40:42.918980 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/56b8099d9406431284b4dd58b5c16359"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/56b8099d9406431284b4dd58b5c16359"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_56b8099d9406431284b4dd58b5c16359"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/56b8099d9406431284b4dd58b5c16359"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:40:42.918980 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-VolumesSnapshotTestJSON-1954720325 Jul 16 21:40:42.918980 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: 56b8099d9406431284b4dd58b5c16359 Jul 16 21:40:42.918980 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-VolumesSnapshotTestJSON-1954720325 Jul 16 21:40:42.918980 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-VolumesSnapshotTestJSON-1954720325-project Jul 16 21:40:42.918980 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:40:42.918980 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:40:42.918980 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: 95cd6b54d8164de1a524b407a1852ebd Jul 16 21:40:42.918980 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-VolumesSnapshotTestJSON-1954720325-project Jul 16 21:40:42.918980 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:40:42.918980 ncdv-master-1 devstack@n-api.service[138681]: exception: Instance 21fc83e5-8f03-4930-93b1-0196f7327b3e could not be found. Jul 16 21:40:42.918980 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': '21fc83e5-8f03-4930-93b1-0196f7327b3e'} Jul 16 21:40:42.918980 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:40:42.918980 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:40:42.918980 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:40:42.918980 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:40:42.918980 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:40:42.918980 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:40:42.918980 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:40:42.918980 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:40:42.918980 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:40:42.918980 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:40:42.918980 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 21fc83e5-8f03-4930-93b1-0196f7327b3e could not be found. Jul 16 21:40:42.918980 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:40:42.918980 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:40:42.918980 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:40:42.918980 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:40:42.918980 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:40:42.918980 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:40:42.918980 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:40:42.918980 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:40:42.918980 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:40:42.918980 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:40:42.918980 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:40:42.918980 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:40:42.918980 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:40:42.918980 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:40:42.918980 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:40:42.918980 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:40:42.918980 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 21fc83e5-8f03-4930-93b1-0196f7327b3e could not be found. Jul 16 21:40:42.918980 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:40:42.918980 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:40:42.918980 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:40:42.923788 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:40:42.923788 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:40:42.923788 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:40:42.923788 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:40:42.923788 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:40:42.923788 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 21fc83e5-8f03-4930-93b1-0196f7327b3e could not be found. Jul 16 21:40:42.923788 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:40:42.923788 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-b7413312-a501-4910-8385-e97d9b653f27 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Returning 404 to user: Instance 21fc83e5-8f03-4930-93b1-0196f7327b3e could not be found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:40:42.923788 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-b7413312-a501-4910-8385-e97d9b653f27 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] 10.222.0.16 "GET /compute/v2.1/servers/21fc83e5-8f03-4930-93b1-0196f7327b3e" status: 404 len: 111 microversion: 2.1 time: 0.034524 Jul 16 21:40:42.923788 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2708/5408] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:40:42 2022] GET /compute/v2.1/servers/21fc83e5-8f03-4930-93b1-0196f7327b3e => generated 111 bytes in 36 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:40:50.976406 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-5cc460d9-425d-44ee-aff4-3daa1759ce32 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestNetworkAdvancedServerOps-server-1392236123", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452", "networks": [{"uuid": "399d5dd0-5d2c-439a-9b97-5af39e4bb8fc"}], "key_name": "tempest-TestNetworkAdvancedServerOps-1439709083", "security_groups": [{"name": "tempest-secgroup-smoke-1776498735"}]}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:40:50.984198 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell0:SELECT=4 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:40:51.122363 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.network.neutron [None req-5cc460d9-425d-44ee-aff4-3daa1759ce32 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] validate_networks() for [('399d5dd0-5d2c-439a-9b97-5af39e4bb8fc', None, None, None, None, None)] {{(pid=138680) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 16 21:40:51.153417 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-39dada0a-cc58-404a-af47-08ff8806e017 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesActionsTest-instance-1156856506", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452", "networks": [{"uuid": "b5201600-e01f-45ea-ba7f-ceb596ae69cf"}]}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:40:51.187761 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_api:SELECT=63,nova_api:UPDATE=3 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:40:51.263901 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.network.neutron [None req-39dada0a-cc58-404a-af47-08ff8806e017 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] validate_networks() for [('b5201600-e01f-45ea-ba7f-ceb596ae69cf', None, None, None, None, None)] {{(pid=138681) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 16 21:40:51.857618 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=78,nova_cell1:INSERT=1 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:40:51.887650 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-5cc460d9-425d-44ee-aff4-3daa1759ce32 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Flavor limits 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 16 21:40:51.888213 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-5cc460d9-425d-44ee-aff4-3daa1759ce32 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Image limits 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 16 21:40:51.888923 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-5cc460d9-425d-44ee-aff4-3daa1759ce32 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Flavor pref 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 16 21:40:51.889551 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-5cc460d9-425d-44ee-aff4-3daa1759ce32 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Image pref 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 16 21:40:51.890181 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-5cc460d9-425d-44ee-aff4-3daa1759ce32 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 16 21:40:51.906956 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5cc460d9-425d-44ee-aff4-3daa1759ce32 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:40:51.906956 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5cc460d9-425d-44ee-aff4-3daa1759ce32 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:40:51.906956 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5cc460d9-425d-44ee-aff4-3daa1759ce32 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:40:51.906956 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5cc460d9-425d-44ee-aff4-3daa1759ce32 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:40:51.928777 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-5cc460d9-425d-44ee-aff4-3daa1759ce32 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Getting quotas for project 77c9b7e351e24dc88207e9498f75f578. Resources: {'ram', 'instances', 'cores'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:40:51.934182 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-39dada0a-cc58-404a-af47-08ff8806e017 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Flavor limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 16 21:40:51.934621 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-39dada0a-cc58-404a-af47-08ff8806e017 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Image limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 16 21:40:51.935172 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-39dada0a-cc58-404a-af47-08ff8806e017 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Flavor pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 16 21:40:51.935602 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-39dada0a-cc58-404a-af47-08ff8806e017 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Image pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 16 21:40:51.935834 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-5cc460d9-425d-44ee-aff4-3daa1759ce32 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Getting quotas for user 833136cbdeab4b9e8b6dd2d24c7c8753 and project 77c9b7e351e24dc88207e9498f75f578. Resources: {'ram', 'instances', 'cores'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:40:51.936031 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-39dada0a-cc58-404a-af47-08ff8806e017 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 16 21:40:51.949501 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-39dada0a-cc58-404a-af47-08ff8806e017 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:40:51.950203 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-39dada0a-cc58-404a-af47-08ff8806e017 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:40:51.951227 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-39dada0a-cc58-404a-af47-08ff8806e017 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:40:51.951828 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-39dada0a-cc58-404a-af47-08ff8806e017 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:40:51.955760 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-5cc460d9-425d-44ee-aff4-3daa1759ce32 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Going to run 1 instances... {{(pid=138680) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 16 21:40:51.957970 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-5cc460d9-425d-44ee-aff4-3daa1759ce32 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] [instance: 35a74168-3c95-4bfe-b82a-e93b10ec5820] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138680) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 16 21:40:51.971081 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-39dada0a-cc58-404a-af47-08ff8806e017 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Getting quotas for project bd61da83cb56412a8757633c22561495. Resources: {'ram', 'instances', 'cores'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:40:51.976752 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-39dada0a-cc58-404a-af47-08ff8806e017 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Getting quotas for user f7948b58ee714cb1bfd4d8364dae19ee and project bd61da83cb56412a8757633c22561495. Resources: {'ram', 'instances', 'cores'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:40:51.990328 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-39dada0a-cc58-404a-af47-08ff8806e017 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Going to run 1 instances... {{(pid=138681) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 16 21:40:51.992671 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-39dada0a-cc58-404a-af47-08ff8806e017 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] [instance: cc9ed855-3638-40c6-92ae-7a4029ffae5d] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138681) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 16 21:40:52.093250 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-5cc460d9-425d-44ee-aff4-3daa1759ce32 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 406 microversion: 2.1 time: 1.121199 Jul 16 21:40:52.098416 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2701/5409] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 21:40:50 2022] POST /compute/v2.1/servers => generated 406 bytes in 1123 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 16 21:40:52.104933 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-6a76157d-ab5d-4a57-9ad9-22b3a8064fde tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:40:52.105784 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-6a76157d-ab5d-4a57-9ad9-22b3a8064fde tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] [instance: 35a74168-3c95-4bfe-b82a-e93b10ec5820] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:40:52.195084 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-6a76157d-ab5d-4a57-9ad9-22b3a8064fde tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '833136cbdeab4b9e8b6dd2d24c7c8753', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '77c9b7e351e24dc88207e9498f75f578', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:40:52.205944 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-6a76157d-ab5d-4a57-9ad9-22b3a8064fde tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] 10.222.0.16 "GET /compute/v2.1/servers/35a74168-3c95-4bfe-b82a-e93b10ec5820" status: 200 len: 1267 microversion: 2.1 time: 0.104951 Jul 16 21:40:52.209941 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2702/5410] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:40:52 2022] GET /compute/v2.1/servers/35a74168-3c95-4bfe-b82a-e93b10ec5820 => generated 1267 bytes in 109 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:40:52.471862 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-39dada0a-cc58-404a-af47-08ff8806e017 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 1.321983 Jul 16 21:40:52.473927 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2709/5411] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 21:40:51 2022] POST /compute/v2.1/servers => generated 380 bytes in 1325 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 16 21:40:52.488861 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-55ae5f57-6817-4b6e-9243-91c178f51791 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:40:52.488861 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-55ae5f57-6817-4b6e-9243-91c178f51791 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] [instance: cc9ed855-3638-40c6-92ae-7a4029ffae5d] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:40:52.586365 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-55ae5f57-6817-4b6e-9243-91c178f51791 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f7948b58ee714cb1bfd4d8364dae19ee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd61da83cb56412a8757633c22561495', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:40:52.593885 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-55ae5f57-6817-4b6e-9243-91c178f51791 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] 10.222.0.16 "GET /compute/v2.1/servers/cc9ed855-3638-40c6-92ae-7a4029ffae5d" status: 200 len: 1214 microversion: 2.1 time: 0.114395 Jul 16 21:40:52.604492 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2703/5412] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:40:52 2022] GET /compute/v2.1/servers/cc9ed855-3638-40c6-92ae-7a4029ffae5d => generated 1214 bytes in 125 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:40:53.228051 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-535624ab-676b-43e1-b311-cdc6042666c5 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:40:53.228689 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-535624ab-676b-43e1-b311-cdc6042666c5 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] [instance: 35a74168-3c95-4bfe-b82a-e93b10ec5820] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:40:53.286784 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-535624ab-676b-43e1-b311-cdc6042666c5 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '833136cbdeab4b9e8b6dd2d24c7c8753', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '77c9b7e351e24dc88207e9498f75f578', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:40:53.294590 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-535624ab-676b-43e1-b311-cdc6042666c5 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] 10.222.0.16 "GET /compute/v2.1/servers/35a74168-3c95-4bfe-b82a-e93b10ec5820" status: 200 len: 1267 microversion: 2.1 time: 0.071448 Jul 16 21:40:53.295207 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2710/5413] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:40:53 2022] GET /compute/v2.1/servers/35a74168-3c95-4bfe-b82a-e93b10ec5820 => generated 1267 bytes in 73 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:40:53.613375 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-081119ae-ca87-4080-bd04-a415c7bdd311 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:40:53.614104 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-081119ae-ca87-4080-bd04-a415c7bdd311 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] [instance: cc9ed855-3638-40c6-92ae-7a4029ffae5d] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:40:53.672168 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-081119ae-ca87-4080-bd04-a415c7bdd311 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f7948b58ee714cb1bfd4d8364dae19ee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd61da83cb56412a8757633c22561495', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:40:53.680250 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-081119ae-ca87-4080-bd04-a415c7bdd311 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] 10.222.0.16 "GET /compute/v2.1/servers/cc9ed855-3638-40c6-92ae-7a4029ffae5d" status: 200 len: 1214 microversion: 2.1 time: 0.072271 Jul 16 21:40:53.681598 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2704/5414] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:40:53 2022] GET /compute/v2.1/servers/cc9ed855-3638-40c6-92ae-7a4029ffae5d => generated 1214 bytes in 74 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:40:54.312553 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-810279e8-7f1c-400e-b2e1-91e2f73be2e5 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:40:54.313203 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-810279e8-7f1c-400e-b2e1-91e2f73be2e5 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] [instance: 35a74168-3c95-4bfe-b82a-e93b10ec5820] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:40:54.321445 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-810279e8-7f1c-400e-b2e1-91e2f73be2e5 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:40:54.321910 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-810279e8-7f1c-400e-b2e1-91e2f73be2e5 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:40:54.322681 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-810279e8-7f1c-400e-b2e1-91e2f73be2e5 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:40:54.323158 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-810279e8-7f1c-400e-b2e1-91e2f73be2e5 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:40:54.386122 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-810279e8-7f1c-400e-b2e1-91e2f73be2e5 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '833136cbdeab4b9e8b6dd2d24c7c8753', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '77c9b7e351e24dc88207e9498f75f578', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:40:54.393920 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-810279e8-7f1c-400e-b2e1-91e2f73be2e5 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] 10.222.0.16 "GET /compute/v2.1/servers/35a74168-3c95-4bfe-b82a-e93b10ec5820" status: 200 len: 1319 microversion: 2.1 time: 0.085460 Jul 16 21:40:54.394490 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2711/5415] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:40:54 2022] GET /compute/v2.1/servers/35a74168-3c95-4bfe-b82a-e93b10ec5820 => generated 1319 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:40:54.698189 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-75045c0a-fe9c-4cea-b619-57d577e0f0e6 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:40:54.698875 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-75045c0a-fe9c-4cea-b619-57d577e0f0e6 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] [instance: cc9ed855-3638-40c6-92ae-7a4029ffae5d] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:40:54.706780 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-75045c0a-fe9c-4cea-b619-57d577e0f0e6 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:40:54.707264 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-75045c0a-fe9c-4cea-b619-57d577e0f0e6 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:40:54.708052 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-75045c0a-fe9c-4cea-b619-57d577e0f0e6 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:40:54.708501 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-75045c0a-fe9c-4cea-b619-57d577e0f0e6 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:40:54.783456 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-75045c0a-fe9c-4cea-b619-57d577e0f0e6 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f7948b58ee714cb1bfd4d8364dae19ee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd61da83cb56412a8757633c22561495', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:40:54.793495 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-75045c0a-fe9c-4cea-b619-57d577e0f0e6 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] 10.222.0.16 "GET /compute/v2.1/servers/cc9ed855-3638-40c6-92ae-7a4029ffae5d" status: 200 len: 1210 microversion: 2.1 time: 0.099427 Jul 16 21:40:54.794141 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2705/5416] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:40:54 2022] GET /compute/v2.1/servers/cc9ed855-3638-40c6-92ae-7a4029ffae5d => generated 1210 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:40:55.411343 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-5b7904c9-43af-483b-af85-c6de340f80c8 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:40:55.412108 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-5b7904c9-43af-483b-af85-c6de340f80c8 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] [instance: 35a74168-3c95-4bfe-b82a-e93b10ec5820] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:40:55.419145 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5b7904c9-43af-483b-af85-c6de340f80c8 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:40:55.419717 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5b7904c9-43af-483b-af85-c6de340f80c8 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:40:55.420428 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5b7904c9-43af-483b-af85-c6de340f80c8 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:40:55.420874 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5b7904c9-43af-483b-af85-c6de340f80c8 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:40:55.535699 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-5b7904c9-43af-483b-af85-c6de340f80c8 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '833136cbdeab4b9e8b6dd2d24c7c8753', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '77c9b7e351e24dc88207e9498f75f578', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:40:55.548842 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-5b7904c9-43af-483b-af85-c6de340f80c8 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] 10.222.0.16 "GET /compute/v2.1/servers/35a74168-3c95-4bfe-b82a-e93b10ec5820" status: 200 len: 1325 microversion: 2.1 time: 0.141552 Jul 16 21:40:55.560178 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2712/5417] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:40:55 2022] GET /compute/v2.1/servers/35a74168-3c95-4bfe-b82a-e93b10ec5820 => generated 1325 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:40:55.811093 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-fb52fea7-374b-4f94-a25b-ce3652e6c11e tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:40:55.812877 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-fb52fea7-374b-4f94-a25b-ce3652e6c11e tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] [instance: cc9ed855-3638-40c6-92ae-7a4029ffae5d] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:40:55.829289 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-fb52fea7-374b-4f94-a25b-ce3652e6c11e tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:40:55.830184 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-fb52fea7-374b-4f94-a25b-ce3652e6c11e tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:40:55.831192 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-fb52fea7-374b-4f94-a25b-ce3652e6c11e tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:40:55.831803 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-fb52fea7-374b-4f94-a25b-ce3652e6c11e tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:40:55.926777 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-fb52fea7-374b-4f94-a25b-ce3652e6c11e tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f7948b58ee714cb1bfd4d8364dae19ee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd61da83cb56412a8757633c22561495', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:40:55.940857 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-fb52fea7-374b-4f94-a25b-ce3652e6c11e tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] 10.222.0.16 "GET /compute/v2.1/servers/cc9ed855-3638-40c6-92ae-7a4029ffae5d" status: 200 len: 1272 microversion: 2.1 time: 0.133644 Jul 16 21:40:55.941900 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2706/5418] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:40:55 2022] GET /compute/v2.1/servers/cc9ed855-3638-40c6-92ae-7a4029ffae5d => generated 1272 bytes in 135 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:40:56.573021 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-538c0a90-907e-43d6-b2ed-5d7550718333 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:40:56.575595 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-538c0a90-907e-43d6-b2ed-5d7550718333 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] [instance: 35a74168-3c95-4bfe-b82a-e93b10ec5820] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:40:56.592552 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-538c0a90-907e-43d6-b2ed-5d7550718333 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:40:56.593111 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-538c0a90-907e-43d6-b2ed-5d7550718333 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:40:56.594237 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-538c0a90-907e-43d6-b2ed-5d7550718333 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:40:56.597085 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-538c0a90-907e-43d6-b2ed-5d7550718333 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:40:56.736586 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-bd9957be-b4f6-4f6c-bcf7-a8e709e9be08 req-b5a7329c-3a2d-4d2e-8a0e-362a69f30934 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "35a74168-3c95-4bfe-b82a-e93b10ec5820", "tag": "94628a67-e3ab-4037-b7ce-cf51f08c17a6"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:40:56.751431 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-bd9957be-b4f6-4f6c-bcf7-a8e709e9be08 req-b5a7329c-3a2d-4d2e-8a0e-362a69f30934 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:40:56.753786 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-bd9957be-b4f6-4f6c-bcf7-a8e709e9be08 req-b5a7329c-3a2d-4d2e-8a0e-362a69f30934 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:40:56.776351 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-bd9957be-b4f6-4f6c-bcf7-a8e709e9be08 req-b5a7329c-3a2d-4d2e-8a0e-362a69f30934 service nova] Creating event network-changed:94628a67-e3ab-4037-b7ce-cf51f08c17a6 for instance 35a74168-3c95-4bfe-b82a-e93b10ec5820 on nch1-master-1 Jul 16 21:40:56.782279 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-bd9957be-b4f6-4f6c-bcf7-a8e709e9be08 req-b5a7329c-3a2d-4d2e-8a0e-362a69f30934 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.050306 Jul 16 21:40:56.783917 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2707/5419] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:40:56 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:40:56.856121 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-538c0a90-907e-43d6-b2ed-5d7550718333 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '833136cbdeab4b9e8b6dd2d24c7c8753', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '77c9b7e351e24dc88207e9498f75f578', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:40:56.865257 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-538c0a90-907e-43d6-b2ed-5d7550718333 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] 10.222.0.16 "GET /compute/v2.1/servers/35a74168-3c95-4bfe-b82a-e93b10ec5820" status: 200 len: 1393 microversion: 2.1 time: 0.303307 Jul 16 21:40:56.878373 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2713/5420] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:40:56 2022] GET /compute/v2.1/servers/35a74168-3c95-4bfe-b82a-e93b10ec5820 => generated 1393 bytes in 315 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:40:56.959314 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-a1e2c31a-e4c3-4dad-8c72-59048b46a506 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:40:56.962422 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-a1e2c31a-e4c3-4dad-8c72-59048b46a506 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] [instance: cc9ed855-3638-40c6-92ae-7a4029ffae5d] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:40:56.969380 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a1e2c31a-e4c3-4dad-8c72-59048b46a506 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:40:56.970138 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a1e2c31a-e4c3-4dad-8c72-59048b46a506 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:40:56.971153 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a1e2c31a-e4c3-4dad-8c72-59048b46a506 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:40:56.972087 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a1e2c31a-e4c3-4dad-8c72-59048b46a506 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:40:57.050412 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-c256f437-0ad0-4a3d-891b-0a898d8b3740 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesSnapshotTestJSON-instance-1973090003", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452", "networks": [{"uuid": "fde2210a-0286-4643-88e0-b5abc1bd975b"}]}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:40:57.128419 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.network.neutron [None req-c256f437-0ad0-4a3d-891b-0a898d8b3740 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] validate_networks() for [('fde2210a-0286-4643-88e0-b5abc1bd975b', None, None, None, None, None)] {{(pid=138681) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 16 21:40:57.297456 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-a1e2c31a-e4c3-4dad-8c72-59048b46a506 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f7948b58ee714cb1bfd4d8364dae19ee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd61da83cb56412a8757633c22561495', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:40:57.306666 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-a1e2c31a-e4c3-4dad-8c72-59048b46a506 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] 10.222.0.16 "GET /compute/v2.1/servers/cc9ed855-3638-40c6-92ae-7a4029ffae5d" status: 200 len: 1314 microversion: 2.1 time: 0.352576 Jul 16 21:40:57.307313 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2708/5421] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:40:56 2022] GET /compute/v2.1/servers/cc9ed855-3638-40c6-92ae-7a4029ffae5d => generated 1314 bytes in 354 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:40:57.612272 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-6cbac39a-1b73-4285-9e82-db2796c5e078 req-33654eae-7a52-4903-b4c9-7552f03756f8 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "cc9ed855-3638-40c6-92ae-7a4029ffae5d", "tag": "d5e68b0c-09a6-4b04-a437-f217475278e6"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:40:57.621645 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-6cbac39a-1b73-4285-9e82-db2796c5e078 req-33654eae-7a52-4903-b4c9-7552f03756f8 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:40:57.622361 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-6cbac39a-1b73-4285-9e82-db2796c5e078 req-33654eae-7a52-4903-b4c9-7552f03756f8 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:40:57.637528 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-6cbac39a-1b73-4285-9e82-db2796c5e078 req-33654eae-7a52-4903-b4c9-7552f03756f8 service nova] Creating event network-changed:d5e68b0c-09a6-4b04-a437-f217475278e6 for instance cc9ed855-3638-40c6-92ae-7a4029ffae5d on nch1-master-1 Jul 16 21:40:57.644432 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-6cbac39a-1b73-4285-9e82-db2796c5e078 req-33654eae-7a52-4903-b4c9-7552f03756f8 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.037464 Jul 16 21:40:57.648158 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2709/5422] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:40:57 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 16 21:40:57.757089 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-c256f437-0ad0-4a3d-891b-0a898d8b3740 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Flavor limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 16 21:40:57.763229 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-c256f437-0ad0-4a3d-891b-0a898d8b3740 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Image limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 16 21:40:57.763229 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-c256f437-0ad0-4a3d-891b-0a898d8b3740 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Flavor pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 16 21:40:57.763229 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-c256f437-0ad0-4a3d-891b-0a898d8b3740 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Image pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 16 21:40:57.763229 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-c256f437-0ad0-4a3d-891b-0a898d8b3740 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 16 21:40:57.768306 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c256f437-0ad0-4a3d-891b-0a898d8b3740 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:40:57.768764 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c256f437-0ad0-4a3d-891b-0a898d8b3740 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:40:57.769575 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c256f437-0ad0-4a3d-891b-0a898d8b3740 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:40:57.770024 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c256f437-0ad0-4a3d-891b-0a898d8b3740 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:40:57.787615 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-c256f437-0ad0-4a3d-891b-0a898d8b3740 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Getting quotas for project 56b8099d9406431284b4dd58b5c16359. Resources: {'ram', 'instances', 'cores'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:40:57.792688 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-c256f437-0ad0-4a3d-891b-0a898d8b3740 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Getting quotas for user 95cd6b54d8164de1a524b407a1852ebd and project 56b8099d9406431284b4dd58b5c16359. Resources: {'ram', 'instances', 'cores'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:40:57.805400 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-c256f437-0ad0-4a3d-891b-0a898d8b3740 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Going to run 1 instances... {{(pid=138681) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 16 21:40:57.807595 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-c256f437-0ad0-4a3d-891b-0a898d8b3740 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] [instance: 97d5b8fb-ce56-41ba-b786-5f8474c9b2f3] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138681) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 16 21:40:57.888643 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-9ecff65d-b86b-499e-aa36-9077c35ce5e4 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:40:57.888643 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-9ecff65d-b86b-499e-aa36-9077c35ce5e4 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] [instance: 35a74168-3c95-4bfe-b82a-e93b10ec5820] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:40:57.890339 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9ecff65d-b86b-499e-aa36-9077c35ce5e4 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:40:57.890793 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9ecff65d-b86b-499e-aa36-9077c35ce5e4 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:40:57.891595 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9ecff65d-b86b-499e-aa36-9077c35ce5e4 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:40:57.893293 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9ecff65d-b86b-499e-aa36-9077c35ce5e4 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:40:57.921943 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-c256f437-0ad0-4a3d-891b-0a898d8b3740 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.876113 Jul 16 21:40:57.922802 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2714/5423] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 21:40:57 2022] POST /compute/v2.1/servers => generated 380 bytes in 878 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 16 21:40:57.935100 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-7ba01bc6-64d6-4299-b00a-db837057a79d tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:40:57.935749 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-7ba01bc6-64d6-4299-b00a-db837057a79d tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] [instance: 97d5b8fb-ce56-41ba-b786-5f8474c9b2f3] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:40:57.997986 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-7ba01bc6-64d6-4299-b00a-db837057a79d tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '95cd6b54d8164de1a524b407a1852ebd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '56b8099d9406431284b4dd58b5c16359', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:40:58.004541 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-7ba01bc6-64d6-4299-b00a-db837057a79d tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] 10.222.0.16 "GET /compute/v2.1/servers/97d5b8fb-ce56-41ba-b786-5f8474c9b2f3" status: 200 len: 1219 microversion: 2.1 time: 0.073346 Jul 16 21:40:58.005133 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2715/5424] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:40:57 2022] GET /compute/v2.1/servers/97d5b8fb-ce56-41ba-b786-5f8474c9b2f3 => generated 1219 bytes in 75 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:40:58.062723 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-9ecff65d-b86b-499e-aa36-9077c35ce5e4 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '833136cbdeab4b9e8b6dd2d24c7c8753', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '77c9b7e351e24dc88207e9498f75f578', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:40:58.070916 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-9ecff65d-b86b-499e-aa36-9077c35ce5e4 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] 10.222.0.16 "GET /compute/v2.1/servers/35a74168-3c95-4bfe-b82a-e93b10ec5820" status: 200 len: 1393 microversion: 2.1 time: 0.192397 Jul 16 21:40:58.071619 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2710/5425] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:40:57 2022] GET /compute/v2.1/servers/35a74168-3c95-4bfe-b82a-e93b10ec5820 => generated 1393 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:40:58.324149 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-b895f0ad-dd9a-4f19-992a-1e7dfeb168e8 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:40:58.324775 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-b895f0ad-dd9a-4f19-992a-1e7dfeb168e8 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] [instance: cc9ed855-3638-40c6-92ae-7a4029ffae5d] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:40:58.331726 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b895f0ad-dd9a-4f19-992a-1e7dfeb168e8 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:40:58.332193 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b895f0ad-dd9a-4f19-992a-1e7dfeb168e8 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:40:58.332978 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b895f0ad-dd9a-4f19-992a-1e7dfeb168e8 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:40:58.333431 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b895f0ad-dd9a-4f19-992a-1e7dfeb168e8 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:40:58.538998 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-b895f0ad-dd9a-4f19-992a-1e7dfeb168e8 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f7948b58ee714cb1bfd4d8364dae19ee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd61da83cb56412a8757633c22561495', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:40:58.547231 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-b895f0ad-dd9a-4f19-992a-1e7dfeb168e8 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] 10.222.0.16 "GET /compute/v2.1/servers/cc9ed855-3638-40c6-92ae-7a4029ffae5d" status: 200 len: 1314 microversion: 2.1 time: 0.227118 Jul 16 21:40:58.548543 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2716/5426] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:40:58 2022] GET /compute/v2.1/servers/cc9ed855-3638-40c6-92ae-7a4029ffae5d => generated 1314 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:40:59.022223 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-4c47e248-7e24-43b8-a110-4341a47b7ffc tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:40:59.022881 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-4c47e248-7e24-43b8-a110-4341a47b7ffc tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] [instance: 97d5b8fb-ce56-41ba-b786-5f8474c9b2f3] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:40:59.030159 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4c47e248-7e24-43b8-a110-4341a47b7ffc tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:40:59.030958 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4c47e248-7e24-43b8-a110-4341a47b7ffc tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:40:59.031577 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4c47e248-7e24-43b8-a110-4341a47b7ffc tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:40:59.032020 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4c47e248-7e24-43b8-a110-4341a47b7ffc tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:40:59.102964 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-f1dda8fb-f459-4c46-9360-32c308d611a1 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:40:59.103643 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-f1dda8fb-f459-4c46-9360-32c308d611a1 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] [instance: 35a74168-3c95-4bfe-b82a-e93b10ec5820] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:40:59.111173 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f1dda8fb-f459-4c46-9360-32c308d611a1 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:40:59.111664 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f1dda8fb-f459-4c46-9360-32c308d611a1 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:40:59.111834 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-4c47e248-7e24-43b8-a110-4341a47b7ffc tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '95cd6b54d8164de1a524b407a1852ebd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '56b8099d9406431284b4dd58b5c16359', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:40:59.112459 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f1dda8fb-f459-4c46-9360-32c308d611a1 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:40:59.113100 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f1dda8fb-f459-4c46-9360-32c308d611a1 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:40:59.121033 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-4c47e248-7e24-43b8-a110-4341a47b7ffc tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] 10.222.0.16 "GET /compute/v2.1/servers/97d5b8fb-ce56-41ba-b786-5f8474c9b2f3" status: 200 len: 1223 microversion: 2.1 time: 0.102741 Jul 16 21:40:59.121878 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2711/5427] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:40:59 2022] GET /compute/v2.1/servers/97d5b8fb-ce56-41ba-b786-5f8474c9b2f3 => generated 1223 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:40:59.569379 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-198f1dbe-6990-4b5d-b70a-b9350f2cb04d tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:40:59.569379 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-198f1dbe-6990-4b5d-b70a-b9350f2cb04d tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] [instance: cc9ed855-3638-40c6-92ae-7a4029ffae5d] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:40:59.572766 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-198f1dbe-6990-4b5d-b70a-b9350f2cb04d tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:40:59.573230 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-198f1dbe-6990-4b5d-b70a-b9350f2cb04d tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:40:59.574132 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-198f1dbe-6990-4b5d-b70a-b9350f2cb04d tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:40:59.574586 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-198f1dbe-6990-4b5d-b70a-b9350f2cb04d tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:40:59.611000 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-f1dda8fb-f459-4c46-9360-32c308d611a1 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '833136cbdeab4b9e8b6dd2d24c7c8753', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '77c9b7e351e24dc88207e9498f75f578', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:40:59.618847 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-f1dda8fb-f459-4c46-9360-32c308d611a1 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] 10.222.0.16 "GET /compute/v2.1/servers/35a74168-3c95-4bfe-b82a-e93b10ec5820" status: 200 len: 1393 microversion: 2.1 time: 0.531442 Jul 16 21:40:59.628649 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2717/5428] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:40:59 2022] GET /compute/v2.1/servers/35a74168-3c95-4bfe-b82a-e93b10ec5820 => generated 1393 bytes in 542 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:40:59.714085 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-198f1dbe-6990-4b5d-b70a-b9350f2cb04d tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f7948b58ee714cb1bfd4d8364dae19ee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd61da83cb56412a8757633c22561495', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:40:59.721622 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-198f1dbe-6990-4b5d-b70a-b9350f2cb04d tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] 10.222.0.16 "GET /compute/v2.1/servers/cc9ed855-3638-40c6-92ae-7a4029ffae5d" status: 200 len: 1314 microversion: 2.1 time: 0.160343 Jul 16 21:40:59.727362 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2712/5429] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:40:59 2022] GET /compute/v2.1/servers/cc9ed855-3638-40c6-92ae-7a4029ffae5d => generated 1314 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:41:00.140110 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-dd89266a-ba33-44f9-a6bc-e2b0b4448d8d tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:41:00.141439 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-dd89266a-ba33-44f9-a6bc-e2b0b4448d8d tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] [instance: 97d5b8fb-ce56-41ba-b786-5f8474c9b2f3] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:41:00.148882 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-dd89266a-ba33-44f9-a6bc-e2b0b4448d8d tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:00.153134 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-dd89266a-ba33-44f9-a6bc-e2b0b4448d8d tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:00.153134 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-dd89266a-ba33-44f9-a6bc-e2b0b4448d8d tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:00.153134 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-dd89266a-ba33-44f9-a6bc-e2b0b4448d8d tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:00.253072 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-dd89266a-ba33-44f9-a6bc-e2b0b4448d8d tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '95cd6b54d8164de1a524b407a1852ebd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '56b8099d9406431284b4dd58b5c16359', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:41:00.260930 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-dd89266a-ba33-44f9-a6bc-e2b0b4448d8d tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] 10.222.0.16 "GET /compute/v2.1/servers/97d5b8fb-ce56-41ba-b786-5f8474c9b2f3" status: 200 len: 1279 microversion: 2.1 time: 0.125047 Jul 16 21:41:00.264019 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2718/5430] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:41:00 2022] GET /compute/v2.1/servers/97d5b8fb-ce56-41ba-b786-5f8474c9b2f3 => generated 1279 bytes in 126 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:41:00.636614 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-62f07542-7924-4d58-95cc-658cef4b221a tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:41:00.637262 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-62f07542-7924-4d58-95cc-658cef4b221a tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] [instance: 35a74168-3c95-4bfe-b82a-e93b10ec5820] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:41:00.644388 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-62f07542-7924-4d58-95cc-658cef4b221a tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:00.644768 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-62f07542-7924-4d58-95cc-658cef4b221a tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:00.645551 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-62f07542-7924-4d58-95cc-658cef4b221a tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:00.645999 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-62f07542-7924-4d58-95cc-658cef4b221a tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:00.739058 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-471bb47a-a4d0-4785-9bbc-e25d8ac2b73b tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:41:00.739725 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-471bb47a-a4d0-4785-9bbc-e25d8ac2b73b tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] [instance: cc9ed855-3638-40c6-92ae-7a4029ffae5d] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:41:00.746692 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-471bb47a-a4d0-4785-9bbc-e25d8ac2b73b tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:00.750375 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-471bb47a-a4d0-4785-9bbc-e25d8ac2b73b tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:00.750375 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-471bb47a-a4d0-4785-9bbc-e25d8ac2b73b tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:00.750375 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-471bb47a-a4d0-4785-9bbc-e25d8ac2b73b tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:00.856607 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-62f07542-7924-4d58-95cc-658cef4b221a tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '833136cbdeab4b9e8b6dd2d24c7c8753', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '77c9b7e351e24dc88207e9498f75f578', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:41:00.865446 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-62f07542-7924-4d58-95cc-658cef4b221a tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] 10.222.0.16 "GET /compute/v2.1/servers/35a74168-3c95-4bfe-b82a-e93b10ec5820" status: 200 len: 1393 microversion: 2.1 time: 0.232721 Jul 16 21:41:00.866137 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2713/5431] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:41:00 2022] GET /compute/v2.1/servers/35a74168-3c95-4bfe-b82a-e93b10ec5820 => generated 1393 bytes in 234 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:41:00.993831 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-471bb47a-a4d0-4785-9bbc-e25d8ac2b73b tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f7948b58ee714cb1bfd4d8364dae19ee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd61da83cb56412a8757633c22561495', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:41:01.001516 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-471bb47a-a4d0-4785-9bbc-e25d8ac2b73b tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] 10.222.0.16 "GET /compute/v2.1/servers/cc9ed855-3638-40c6-92ae-7a4029ffae5d" status: 200 len: 1314 microversion: 2.1 time: 0.267354 Jul 16 21:41:01.002204 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2719/5432] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:41:00 2022] GET /compute/v2.1/servers/cc9ed855-3638-40c6-92ae-7a4029ffae5d => generated 1314 bytes in 269 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:41:01.279383 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-24df4941-6f13-49bb-9005-829028fcff36 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:41:01.280201 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-24df4941-6f13-49bb-9005-829028fcff36 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] [instance: 97d5b8fb-ce56-41ba-b786-5f8474c9b2f3] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:41:01.288553 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-24df4941-6f13-49bb-9005-829028fcff36 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:01.289400 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-24df4941-6f13-49bb-9005-829028fcff36 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:01.290281 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-24df4941-6f13-49bb-9005-829028fcff36 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:01.290877 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-24df4941-6f13-49bb-9005-829028fcff36 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:01.504034 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-24df4941-6f13-49bb-9005-829028fcff36 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '95cd6b54d8164de1a524b407a1852ebd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '56b8099d9406431284b4dd58b5c16359', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:41:01.515627 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-24df4941-6f13-49bb-9005-829028fcff36 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] 10.222.0.16 "GET /compute/v2.1/servers/97d5b8fb-ce56-41ba-b786-5f8474c9b2f3" status: 200 len: 1319 microversion: 2.1 time: 0.240254 Jul 16 21:41:01.516223 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2714/5433] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:41:01 2022] GET /compute/v2.1/servers/97d5b8fb-ce56-41ba-b786-5f8474c9b2f3 => generated 1319 bytes in 242 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:41:01.882746 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-0bfd5955-7e08-4668-b3f6-fa694272ed04 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:41:01.883426 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-0bfd5955-7e08-4668-b3f6-fa694272ed04 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] [instance: 35a74168-3c95-4bfe-b82a-e93b10ec5820] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:41:01.890483 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0bfd5955-7e08-4668-b3f6-fa694272ed04 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:01.890955 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0bfd5955-7e08-4668-b3f6-fa694272ed04 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:01.891853 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0bfd5955-7e08-4668-b3f6-fa694272ed04 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:01.892333 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0bfd5955-7e08-4668-b3f6-fa694272ed04 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:01.953544 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-68873983-9018-4e32-84cb-0bbeb2e5dfb9 req-6bb0e944-a1c8-4860-8594-3128f6c1d858 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "97d5b8fb-ce56-41ba-b786-5f8474c9b2f3", "tag": "38773ab4-9a2c-4c8d-9703-dd1b20b11604"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:41:01.958377 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell0:SELECT=2 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:41:01.964947 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-68873983-9018-4e32-84cb-0bbeb2e5dfb9 req-6bb0e944-a1c8-4860-8594-3128f6c1d858 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:01.965405 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-68873983-9018-4e32-84cb-0bbeb2e5dfb9 req-6bb0e944-a1c8-4860-8594-3128f6c1d858 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:01.977484 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-68873983-9018-4e32-84cb-0bbeb2e5dfb9 req-6bb0e944-a1c8-4860-8594-3128f6c1d858 service nova] Creating event network-changed:38773ab4-9a2c-4c8d-9703-dd1b20b11604 for instance 97d5b8fb-ce56-41ba-b786-5f8474c9b2f3 on nch2-master-1 Jul 16 21:41:01.982986 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-68873983-9018-4e32-84cb-0bbeb2e5dfb9 req-6bb0e944-a1c8-4860-8594-3128f6c1d858 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.033660 Jul 16 21:41:01.983635 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2715/5434] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:41:01 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 16 21:41:02.019397 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-de088a9e-6a0d-4ec4-a86d-d3dfc3c2a46c tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:41:02.020338 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-de088a9e-6a0d-4ec4-a86d-d3dfc3c2a46c tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] [instance: cc9ed855-3638-40c6-92ae-7a4029ffae5d] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:41:02.026948 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-de088a9e-6a0d-4ec4-a86d-d3dfc3c2a46c tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:02.027542 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-de088a9e-6a0d-4ec4-a86d-d3dfc3c2a46c tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:02.028400 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-de088a9e-6a0d-4ec4-a86d-d3dfc3c2a46c tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:02.028919 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-de088a9e-6a0d-4ec4-a86d-d3dfc3c2a46c tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:02.135220 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-0bfd5955-7e08-4668-b3f6-fa694272ed04 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '833136cbdeab4b9e8b6dd2d24c7c8753', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '77c9b7e351e24dc88207e9498f75f578', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:41:02.143930 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-0bfd5955-7e08-4668-b3f6-fa694272ed04 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] 10.222.0.16 "GET /compute/v2.1/servers/35a74168-3c95-4bfe-b82a-e93b10ec5820" status: 200 len: 1393 microversion: 2.1 time: 0.265269 Jul 16 21:41:02.144524 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2720/5435] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:41:01 2022] GET /compute/v2.1/servers/35a74168-3c95-4bfe-b82a-e93b10ec5820 => generated 1393 bytes in 267 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:41:02.278777 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-de088a9e-6a0d-4ec4-a86d-d3dfc3c2a46c tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f7948b58ee714cb1bfd4d8364dae19ee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd61da83cb56412a8757633c22561495', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:41:02.287142 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-de088a9e-6a0d-4ec4-a86d-d3dfc3c2a46c tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] 10.222.0.16 "GET /compute/v2.1/servers/cc9ed855-3638-40c6-92ae-7a4029ffae5d" status: 200 len: 1314 microversion: 2.1 time: 0.272725 Jul 16 21:41:02.287721 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2716/5436] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:41:02 2022] GET /compute/v2.1/servers/cc9ed855-3638-40c6-92ae-7a4029ffae5d => generated 1314 bytes in 274 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:41:02.534265 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-86e411d1-8413-4fac-8345-faea69acc160 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:41:02.535072 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-86e411d1-8413-4fac-8345-faea69acc160 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] [instance: 97d5b8fb-ce56-41ba-b786-5f8474c9b2f3] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:41:02.541324 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-86e411d1-8413-4fac-8345-faea69acc160 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:02.546391 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-86e411d1-8413-4fac-8345-faea69acc160 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:02.546391 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-86e411d1-8413-4fac-8345-faea69acc160 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:02.546391 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-86e411d1-8413-4fac-8345-faea69acc160 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:02.766998 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-86e411d1-8413-4fac-8345-faea69acc160 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '95cd6b54d8164de1a524b407a1852ebd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '56b8099d9406431284b4dd58b5c16359', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:41:02.776732 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-86e411d1-8413-4fac-8345-faea69acc160 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] 10.222.0.16 "GET /compute/v2.1/servers/97d5b8fb-ce56-41ba-b786-5f8474c9b2f3" status: 200 len: 1319 microversion: 2.1 time: 0.245457 Jul 16 21:41:02.776732 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2721/5437] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:41:02 2022] GET /compute/v2.1/servers/97d5b8fb-ce56-41ba-b786-5f8474c9b2f3 => generated 1319 bytes in 247 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:41:03.161665 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-6f71fd28-ce22-4803-98ad-38c42b444186 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:41:03.171510 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-6f71fd28-ce22-4803-98ad-38c42b444186 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] [instance: 35a74168-3c95-4bfe-b82a-e93b10ec5820] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:41:03.171510 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6f71fd28-ce22-4803-98ad-38c42b444186 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:03.171510 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6f71fd28-ce22-4803-98ad-38c42b444186 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:03.171884 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6f71fd28-ce22-4803-98ad-38c42b444186 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:03.172263 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6f71fd28-ce22-4803-98ad-38c42b444186 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:03.304225 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-a05ebecf-e0fa-411a-9a14-896ceab3f862 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:41:03.305562 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-a05ebecf-e0fa-411a-9a14-896ceab3f862 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] [instance: cc9ed855-3638-40c6-92ae-7a4029ffae5d] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:41:03.313193 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a05ebecf-e0fa-411a-9a14-896ceab3f862 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:03.313772 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a05ebecf-e0fa-411a-9a14-896ceab3f862 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:03.314706 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a05ebecf-e0fa-411a-9a14-896ceab3f862 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:03.315200 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a05ebecf-e0fa-411a-9a14-896ceab3f862 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:03.320742 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-6f71fd28-ce22-4803-98ad-38c42b444186 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '833136cbdeab4b9e8b6dd2d24c7c8753', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '77c9b7e351e24dc88207e9498f75f578', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:41:03.331170 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-6f71fd28-ce22-4803-98ad-38c42b444186 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] 10.222.0.16 "GET /compute/v2.1/servers/35a74168-3c95-4bfe-b82a-e93b10ec5820" status: 200 len: 1393 microversion: 2.1 time: 0.174008 Jul 16 21:41:03.331764 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2717/5438] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:41:03 2022] GET /compute/v2.1/servers/35a74168-3c95-4bfe-b82a-e93b10ec5820 => generated 1393 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:41:03.464529 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-a05ebecf-e0fa-411a-9a14-896ceab3f862 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f7948b58ee714cb1bfd4d8364dae19ee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd61da83cb56412a8757633c22561495', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:41:03.472991 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-59eadc82-4849-40d5-a241-126d00880099 req-875a82b7-0fd3-428d-9dc6-2b74e134f9ae service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "35a74168-3c95-4bfe-b82a-e93b10ec5820", "name": "network-vif-plugged", "status": "completed", "tag": "94628a67-e3ab-4037-b7ce-cf51f08c17a6"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:41:03.476623 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-a05ebecf-e0fa-411a-9a14-896ceab3f862 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] 10.222.0.16 "GET /compute/v2.1/servers/cc9ed855-3638-40c6-92ae-7a4029ffae5d" status: 200 len: 1314 microversion: 2.1 time: 0.172843 Jul 16 21:41:03.476623 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2722/5439] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:41:03 2022] GET /compute/v2.1/servers/cc9ed855-3638-40c6-92ae-7a4029ffae5d => generated 1314 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:41:03.486463 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-59eadc82-4849-40d5-a241-126d00880099 req-875a82b7-0fd3-428d-9dc6-2b74e134f9ae service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:03.486942 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-59eadc82-4849-40d5-a241-126d00880099 req-875a82b7-0fd3-428d-9dc6-2b74e134f9ae service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:03.498519 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-59eadc82-4849-40d5-a241-126d00880099 req-875a82b7-0fd3-428d-9dc6-2b74e134f9ae service nova] Creating event network-vif-plugged:94628a67-e3ab-4037-b7ce-cf51f08c17a6 for instance 35a74168-3c95-4bfe-b82a-e93b10ec5820 on nch1-master-1 Jul 16 21:41:03.504352 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-59eadc82-4849-40d5-a241-126d00880099 req-875a82b7-0fd3-428d-9dc6-2b74e134f9ae service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.035336 Jul 16 21:41:03.504984 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2718/5440] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:41:03 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 16 21:41:03.792851 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-318b0fe2-005a-4b80-9f6a-f087dcdd37ba tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:41:03.793479 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-318b0fe2-005a-4b80-9f6a-f087dcdd37ba tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] [instance: 97d5b8fb-ce56-41ba-b786-5f8474c9b2f3] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:41:03.802339 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-318b0fe2-005a-4b80-9f6a-f087dcdd37ba tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:03.803196 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-318b0fe2-005a-4b80-9f6a-f087dcdd37ba tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:03.804270 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-318b0fe2-005a-4b80-9f6a-f087dcdd37ba tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:03.804850 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-318b0fe2-005a-4b80-9f6a-f087dcdd37ba tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:03.944207 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-318b0fe2-005a-4b80-9f6a-f087dcdd37ba tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '95cd6b54d8164de1a524b407a1852ebd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '56b8099d9406431284b4dd58b5c16359', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:41:03.952207 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-318b0fe2-005a-4b80-9f6a-f087dcdd37ba tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] 10.222.0.16 "GET /compute/v2.1/servers/97d5b8fb-ce56-41ba-b786-5f8474c9b2f3" status: 200 len: 1319 microversion: 2.1 time: 0.163716 Jul 16 21:41:03.953024 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2723/5441] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:41:03 2022] GET /compute/v2.1/servers/97d5b8fb-ce56-41ba-b786-5f8474c9b2f3 => generated 1319 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:41:04.349777 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-33f4b080-b40f-4a34-bcda-f05d2a8d5dfd tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:41:04.350507 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-33f4b080-b40f-4a34-bcda-f05d2a8d5dfd tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] [instance: 35a74168-3c95-4bfe-b82a-e93b10ec5820] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:41:04.352538 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell0:SELECT=12 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:41:04.358807 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-33f4b080-b40f-4a34-bcda-f05d2a8d5dfd tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:04.359319 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-33f4b080-b40f-4a34-bcda-f05d2a8d5dfd tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:04.360085 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-33f4b080-b40f-4a34-bcda-f05d2a8d5dfd tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:04.361225 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-33f4b080-b40f-4a34-bcda-f05d2a8d5dfd tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:04.490421 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-b8b09824-0a92-4a64-8a55-c21b992f40b0 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:41:04.490982 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-b8b09824-0a92-4a64-8a55-c21b992f40b0 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] [instance: cc9ed855-3638-40c6-92ae-7a4029ffae5d] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:41:04.498145 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b8b09824-0a92-4a64-8a55-c21b992f40b0 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:04.498611 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b8b09824-0a92-4a64-8a55-c21b992f40b0 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:04.499523 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b8b09824-0a92-4a64-8a55-c21b992f40b0 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:04.499999 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b8b09824-0a92-4a64-8a55-c21b992f40b0 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:04.503493 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-33f4b080-b40f-4a34-bcda-f05d2a8d5dfd tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '833136cbdeab4b9e8b6dd2d24c7c8753', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '77c9b7e351e24dc88207e9498f75f578', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:41:04.511214 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-33f4b080-b40f-4a34-bcda-f05d2a8d5dfd tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] 10.222.0.16 "GET /compute/v2.1/servers/35a74168-3c95-4bfe-b82a-e93b10ec5820" status: 200 len: 1393 microversion: 2.1 time: 0.166248 Jul 16 21:41:04.512664 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2719/5442] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:41:04 2022] GET /compute/v2.1/servers/35a74168-3c95-4bfe-b82a-e93b10ec5820 => generated 1393 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:41:04.635521 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-b8b09824-0a92-4a64-8a55-c21b992f40b0 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f7948b58ee714cb1bfd4d8364dae19ee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd61da83cb56412a8757633c22561495', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:41:04.643740 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-b8b09824-0a92-4a64-8a55-c21b992f40b0 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] 10.222.0.16 "GET /compute/v2.1/servers/cc9ed855-3638-40c6-92ae-7a4029ffae5d" status: 200 len: 1314 microversion: 2.1 time: 0.157963 Jul 16 21:41:04.644319 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2724/5443] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:41:04 2022] GET /compute/v2.1/servers/cc9ed855-3638-40c6-92ae-7a4029ffae5d => generated 1314 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:41:04.969600 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-728d0820-12bf-4ec6-9c36-b46a2ec20f89 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:41:04.970287 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-728d0820-12bf-4ec6-9c36-b46a2ec20f89 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] [instance: 97d5b8fb-ce56-41ba-b786-5f8474c9b2f3] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:41:04.976633 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-728d0820-12bf-4ec6-9c36-b46a2ec20f89 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:04.977092 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-728d0820-12bf-4ec6-9c36-b46a2ec20f89 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:04.977861 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-728d0820-12bf-4ec6-9c36-b46a2ec20f89 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:04.978305 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-728d0820-12bf-4ec6-9c36-b46a2ec20f89 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:05.119319 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-728d0820-12bf-4ec6-9c36-b46a2ec20f89 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '95cd6b54d8164de1a524b407a1852ebd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '56b8099d9406431284b4dd58b5c16359', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:41:05.128681 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-728d0820-12bf-4ec6-9c36-b46a2ec20f89 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] 10.222.0.16 "GET /compute/v2.1/servers/97d5b8fb-ce56-41ba-b786-5f8474c9b2f3" status: 200 len: 1319 microversion: 2.1 time: 0.162911 Jul 16 21:41:05.129274 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2720/5444] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:41:04 2022] GET /compute/v2.1/servers/97d5b8fb-ce56-41ba-b786-5f8474c9b2f3 => generated 1319 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:41:05.526035 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-567e12c2-11dc-42be-8954-c016f123d33a req-5c79d442-e556-48e2-bc78-ffccfb438d50 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "cc9ed855-3638-40c6-92ae-7a4029ffae5d", "name": "network-vif-plugged", "status": "completed", "tag": "d5e68b0c-09a6-4b04-a437-f217475278e6"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:41:05.529306 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-37ee93be-857e-42d8-82d1-6d850e02bf67 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:41:05.530450 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-37ee93be-857e-42d8-82d1-6d850e02bf67 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] [instance: 35a74168-3c95-4bfe-b82a-e93b10ec5820] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:41:05.535247 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-567e12c2-11dc-42be-8954-c016f123d33a req-5c79d442-e556-48e2-bc78-ffccfb438d50 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:05.535717 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-567e12c2-11dc-42be-8954-c016f123d33a req-5c79d442-e556-48e2-bc78-ffccfb438d50 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:05.537871 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-37ee93be-857e-42d8-82d1-6d850e02bf67 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:05.538336 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-37ee93be-857e-42d8-82d1-6d850e02bf67 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:05.539144 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-37ee93be-857e-42d8-82d1-6d850e02bf67 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:05.539592 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-37ee93be-857e-42d8-82d1-6d850e02bf67 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:05.541000 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=99,nova_cell1:INSERT=1 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:41:05.548984 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-567e12c2-11dc-42be-8954-c016f123d33a req-5c79d442-e556-48e2-bc78-ffccfb438d50 service nova] Creating event network-vif-plugged:d5e68b0c-09a6-4b04-a437-f217475278e6 for instance cc9ed855-3638-40c6-92ae-7a4029ffae5d on nch1-master-1 Jul 16 21:41:05.554066 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-567e12c2-11dc-42be-8954-c016f123d33a req-5c79d442-e556-48e2-bc78-ffccfb438d50 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.033389 Jul 16 21:41:05.554691 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2725/5445] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:41:05 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 16 21:41:05.662601 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-014487b1-5e53-4997-8a78-2c00777bc0bd tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:41:05.663884 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-014487b1-5e53-4997-8a78-2c00777bc0bd tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] [instance: cc9ed855-3638-40c6-92ae-7a4029ffae5d] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:41:05.672801 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-014487b1-5e53-4997-8a78-2c00777bc0bd tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:05.674313 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-014487b1-5e53-4997-8a78-2c00777bc0bd tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:05.675588 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-014487b1-5e53-4997-8a78-2c00777bc0bd tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:05.676142 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-014487b1-5e53-4997-8a78-2c00777bc0bd tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:05.769330 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-37ee93be-857e-42d8-82d1-6d850e02bf67 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '833136cbdeab4b9e8b6dd2d24c7c8753', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '77c9b7e351e24dc88207e9498f75f578', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:41:05.777558 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-37ee93be-857e-42d8-82d1-6d850e02bf67 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] 10.222.0.16 "GET /compute/v2.1/servers/35a74168-3c95-4bfe-b82a-e93b10ec5820" status: 200 len: 1565 microversion: 2.1 time: 0.252227 Jul 16 21:41:05.789063 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2721/5446] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:41:05 2022] GET /compute/v2.1/servers/35a74168-3c95-4bfe-b82a-e93b10ec5820 => generated 1565 bytes in 254 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:41:05.794787 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-31fe5dcf-e082-4506-b799-74e38cc5b69b tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:41:05.799061 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-31fe5dcf-e082-4506-b799-74e38cc5b69b tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] [instance: 35a74168-3c95-4bfe-b82a-e93b10ec5820] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:41:05.802559 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-31fe5dcf-e082-4506-b799-74e38cc5b69b tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:05.803043 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-31fe5dcf-e082-4506-b799-74e38cc5b69b tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:05.803839 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-31fe5dcf-e082-4506-b799-74e38cc5b69b tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:05.804288 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-31fe5dcf-e082-4506-b799-74e38cc5b69b tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:05.891906 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-014487b1-5e53-4997-8a78-2c00777bc0bd tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f7948b58ee714cb1bfd4d8364dae19ee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd61da83cb56412a8757633c22561495', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:41:05.900148 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-014487b1-5e53-4997-8a78-2c00777bc0bd tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] 10.222.0.16 "GET /compute/v2.1/servers/cc9ed855-3638-40c6-92ae-7a4029ffae5d" status: 200 len: 1314 microversion: 2.1 time: 0.242936 Jul 16 21:41:05.901086 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2726/5447] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:41:05 2022] GET /compute/v2.1/servers/cc9ed855-3638-40c6-92ae-7a4029ffae5d => generated 1314 bytes in 245 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:41:05.969450 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-31fe5dcf-e082-4506-b799-74e38cc5b69b tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '833136cbdeab4b9e8b6dd2d24c7c8753', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '77c9b7e351e24dc88207e9498f75f578', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:41:05.977771 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-31fe5dcf-e082-4506-b799-74e38cc5b69b tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] 10.222.0.16 "GET /compute/v2.1/servers/35a74168-3c95-4bfe-b82a-e93b10ec5820" status: 200 len: 1565 microversion: 2.1 time: 0.186818 Jul 16 21:41:05.978357 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2722/5448] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:41:05 2022] GET /compute/v2.1/servers/35a74168-3c95-4bfe-b82a-e93b10ec5820 => generated 1565 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:41:06.145380 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-3417bd72-4eca-42bc-94a3-0a3336be7034 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:41:06.145942 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-3417bd72-4eca-42bc-94a3-0a3336be7034 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] [instance: 97d5b8fb-ce56-41ba-b786-5f8474c9b2f3] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:41:06.152378 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3417bd72-4eca-42bc-94a3-0a3336be7034 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:06.154768 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3417bd72-4eca-42bc-94a3-0a3336be7034 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:06.154768 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3417bd72-4eca-42bc-94a3-0a3336be7034 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:06.154768 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3417bd72-4eca-42bc-94a3-0a3336be7034 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:06.292246 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-3417bd72-4eca-42bc-94a3-0a3336be7034 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '95cd6b54d8164de1a524b407a1852ebd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '56b8099d9406431284b4dd58b5c16359', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:41:06.300762 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-3417bd72-4eca-42bc-94a3-0a3336be7034 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] 10.222.0.16 "GET /compute/v2.1/servers/97d5b8fb-ce56-41ba-b786-5f8474c9b2f3" status: 200 len: 1319 microversion: 2.1 time: 0.159587 Jul 16 21:41:06.301355 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2727/5449] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:41:06 2022] GET /compute/v2.1/servers/97d5b8fb-ce56-41ba-b786-5f8474c9b2f3 => generated 1319 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:41:06.917924 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-a256f512-0365-4dee-8d85-8f9e80201993 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:41:06.918673 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-a256f512-0365-4dee-8d85-8f9e80201993 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] [instance: cc9ed855-3638-40c6-92ae-7a4029ffae5d] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:41:06.926367 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a256f512-0365-4dee-8d85-8f9e80201993 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:06.926881 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a256f512-0365-4dee-8d85-8f9e80201993 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:06.927871 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a256f512-0365-4dee-8d85-8f9e80201993 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:06.928315 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a256f512-0365-4dee-8d85-8f9e80201993 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:07.068706 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-a256f512-0365-4dee-8d85-8f9e80201993 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f7948b58ee714cb1bfd4d8364dae19ee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd61da83cb56412a8757633c22561495', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:41:07.072207 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=100 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:41:07.076670 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-a256f512-0365-4dee-8d85-8f9e80201993 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] 10.222.0.16 "GET /compute/v2.1/servers/cc9ed855-3638-40c6-92ae-7a4029ffae5d" status: 200 len: 1314 microversion: 2.1 time: 0.163064 Jul 16 21:41:07.077262 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2723/5450] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:41:06 2022] GET /compute/v2.1/servers/cc9ed855-3638-40c6-92ae-7a4029ffae5d => generated 1314 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:41:07.320672 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-8291cb31-4cf0-4dc7-bafd-69084bfc1404 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:41:07.321425 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-8291cb31-4cf0-4dc7-bafd-69084bfc1404 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] [instance: 97d5b8fb-ce56-41ba-b786-5f8474c9b2f3] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:41:07.327990 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8291cb31-4cf0-4dc7-bafd-69084bfc1404 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:07.328444 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8291cb31-4cf0-4dc7-bafd-69084bfc1404 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:07.329326 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8291cb31-4cf0-4dc7-bafd-69084bfc1404 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:07.329954 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8291cb31-4cf0-4dc7-bafd-69084bfc1404 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:07.463168 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-8291cb31-4cf0-4dc7-bafd-69084bfc1404 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '95cd6b54d8164de1a524b407a1852ebd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '56b8099d9406431284b4dd58b5c16359', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:41:07.472079 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-8291cb31-4cf0-4dc7-bafd-69084bfc1404 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] 10.222.0.16 "GET /compute/v2.1/servers/97d5b8fb-ce56-41ba-b786-5f8474c9b2f3" status: 200 len: 1319 microversion: 2.1 time: 0.158267 Jul 16 21:41:07.472685 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2728/5451] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:41:07 2022] GET /compute/v2.1/servers/97d5b8fb-ce56-41ba-b786-5f8474c9b2f3 => generated 1319 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:41:07.568858 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-fe855cf9-5356-4992-8e8a-ab4868a298ae req-0c078499-75f8-49ea-897d-b10f9480a2f8 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "97d5b8fb-ce56-41ba-b786-5f8474c9b2f3", "name": "network-vif-plugged", "status": "completed", "tag": "38773ab4-9a2c-4c8d-9703-dd1b20b11604"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:41:07.577989 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-fe855cf9-5356-4992-8e8a-ab4868a298ae req-0c078499-75f8-49ea-897d-b10f9480a2f8 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:07.578477 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-fe855cf9-5356-4992-8e8a-ab4868a298ae req-0c078499-75f8-49ea-897d-b10f9480a2f8 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:07.591086 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-fe855cf9-5356-4992-8e8a-ab4868a298ae req-0c078499-75f8-49ea-897d-b10f9480a2f8 service nova] Creating event network-vif-plugged:38773ab4-9a2c-4c8d-9703-dd1b20b11604 for instance 97d5b8fb-ce56-41ba-b786-5f8474c9b2f3 on nch2-master-1 Jul 16 21:41:07.598552 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-fe855cf9-5356-4992-8e8a-ab4868a298ae req-0c078499-75f8-49ea-897d-b10f9480a2f8 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.032398 Jul 16 21:41:07.599600 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2724/5452] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:41:07 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:41:08.094506 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-a738880b-f195-44d2-9369-d4e508b86563 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:41:08.096189 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-a738880b-f195-44d2-9369-d4e508b86563 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] [instance: cc9ed855-3638-40c6-92ae-7a4029ffae5d] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:41:08.101628 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell0:SELECT=4 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:41:08.102635 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell0:SELECT=16 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:41:08.104461 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a738880b-f195-44d2-9369-d4e508b86563 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:08.105106 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a738880b-f195-44d2-9369-d4e508b86563 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:08.110166 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-7ca68924-13b4-4fb6-a4cd-7213b6bcfc57 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:41:08.111631 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a738880b-f195-44d2-9369-d4e508b86563 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:08.111631 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a738880b-f195-44d2-9369-d4e508b86563 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:08.111810 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-7ca68924-13b4-4fb6-a4cd-7213b6bcfc57 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] [instance: 35a74168-3c95-4bfe-b82a-e93b10ec5820] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:41:08.115385 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7ca68924-13b4-4fb6-a4cd-7213b6bcfc57 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:08.115916 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7ca68924-13b4-4fb6-a4cd-7213b6bcfc57 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:08.116786 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7ca68924-13b4-4fb6-a4cd-7213b6bcfc57 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:08.117306 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7ca68924-13b4-4fb6-a4cd-7213b6bcfc57 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:08.322407 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-a738880b-f195-44d2-9369-d4e508b86563 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f7948b58ee714cb1bfd4d8364dae19ee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd61da83cb56412a8757633c22561495', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:41:08.325978 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-7ca68924-13b4-4fb6-a4cd-7213b6bcfc57 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '833136cbdeab4b9e8b6dd2d24c7c8753', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '77c9b7e351e24dc88207e9498f75f578', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:41:08.337392 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-7ca68924-13b4-4fb6-a4cd-7213b6bcfc57 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] 10.222.0.16 "GET /compute/v2.1/servers/35a74168-3c95-4bfe-b82a-e93b10ec5820" status: 200 len: 1565 microversion: 2.1 time: 0.237299 Jul 16 21:41:08.337941 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2725/5453] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:41:08 2022] GET /compute/v2.1/servers/35a74168-3c95-4bfe-b82a-e93b10ec5820 => generated 1565 bytes in 239 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:41:08.338208 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-a738880b-f195-44d2-9369-d4e508b86563 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] 10.222.0.16 "GET /compute/v2.1/servers/cc9ed855-3638-40c6-92ae-7a4029ffae5d" status: 200 len: 1497 microversion: 2.1 time: 0.247576 Jul 16 21:41:08.339326 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2729/5454] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:41:08 2022] GET /compute/v2.1/servers/cc9ed855-3638-40c6-92ae-7a4029ffae5d => generated 1497 bytes in 250 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:41:08.352370 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-fec86854-04b0-4f2a-b43d-651a08e59379 req-41a46f4f-d5ee-41e4-b6f5-63b6620219df service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "35a74168-3c95-4bfe-b82a-e93b10ec5820", "tag": "94628a67-e3ab-4037-b7ce-cf51f08c17a6"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:41:08.362700 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-fec86854-04b0-4f2a-b43d-651a08e59379 req-41a46f4f-d5ee-41e4-b6f5-63b6620219df service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:08.363213 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-fec86854-04b0-4f2a-b43d-651a08e59379 req-41a46f4f-d5ee-41e4-b6f5-63b6620219df service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:08.378578 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-fec86854-04b0-4f2a-b43d-651a08e59379 req-41a46f4f-d5ee-41e4-b6f5-63b6620219df service nova] Creating event network-changed:94628a67-e3ab-4037-b7ce-cf51f08c17a6 for instance 35a74168-3c95-4bfe-b82a-e93b10ec5820 on nch1-master-1 Jul 16 21:41:08.386189 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-fec86854-04b0-4f2a-b43d-651a08e59379 req-41a46f4f-d5ee-41e4-b6f5-63b6620219df service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.045004 Jul 16 21:41:08.387284 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2726/5455] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:41:08 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:41:08.492091 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-68e8eca9-c6f0-4f75-8443-ff3bc6b98380 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:41:08.493144 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-68e8eca9-c6f0-4f75-8443-ff3bc6b98380 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] [instance: 97d5b8fb-ce56-41ba-b786-5f8474c9b2f3] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:41:08.501574 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-68e8eca9-c6f0-4f75-8443-ff3bc6b98380 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:08.502343 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-68e8eca9-c6f0-4f75-8443-ff3bc6b98380 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:08.503557 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-68e8eca9-c6f0-4f75-8443-ff3bc6b98380 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:08.504225 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-68e8eca9-c6f0-4f75-8443-ff3bc6b98380 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:08.742860 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-68e8eca9-c6f0-4f75-8443-ff3bc6b98380 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '95cd6b54d8164de1a524b407a1852ebd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '56b8099d9406431284b4dd58b5c16359', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:41:08.752696 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-68e8eca9-c6f0-4f75-8443-ff3bc6b98380 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] 10.222.0.16 "GET /compute/v2.1/servers/97d5b8fb-ce56-41ba-b786-5f8474c9b2f3" status: 200 len: 1319 microversion: 2.1 time: 0.266773 Jul 16 21:41:08.757743 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2730/5456] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:41:08 2022] GET /compute/v2.1/servers/97d5b8fb-ce56-41ba-b786-5f8474c9b2f3 => generated 1319 bytes in 269 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:41:09.215524 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-c83eb447-6a3d-47a9-b21f-f7b5f03cf230 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:41:09.216134 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-c83eb447-6a3d-47a9-b21f-f7b5f03cf230 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] [instance: cc9ed855-3638-40c6-92ae-7a4029ffae5d] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:41:09.222709 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c83eb447-6a3d-47a9-b21f-f7b5f03cf230 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:09.223203 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c83eb447-6a3d-47a9-b21f-f7b5f03cf230 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:09.224013 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c83eb447-6a3d-47a9-b21f-f7b5f03cf230 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:09.224506 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c83eb447-6a3d-47a9-b21f-f7b5f03cf230 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:09.251302 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-c83eb447-6a3d-47a9-b21f-f7b5f03cf230 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] [instance: cc9ed855-3638-40c6-92ae-7a4029ffae5d] Going to try to terminate instance {{(pid=138680) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 16 21:41:09.502130 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_api:SELECT=95,nova_api:DELETE=1,nova_api:INSERT=4 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:41:09.616736 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-c83eb447-6a3d-47a9-b21f-f7b5f03cf230 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] 10.222.0.16 "DELETE /compute/v2.1/servers/cc9ed855-3638-40c6-92ae-7a4029ffae5d" status: 204 len: 0 microversion: 2.1 time: 0.406069 Jul 16 21:41:09.617574 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2727/5457] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:41:09 2022] DELETE /compute/v2.1/servers/cc9ed855-3638-40c6-92ae-7a4029ffae5d => generated 0 bytes in 408 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 16 21:41:09.626242 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-b766f679-4621-44f4-8fcc-21c22dccd285 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:41:09.628005 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-b766f679-4621-44f4-8fcc-21c22dccd285 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] [instance: cc9ed855-3638-40c6-92ae-7a4029ffae5d] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:41:09.634820 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b766f679-4621-44f4-8fcc-21c22dccd285 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:09.637652 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b766f679-4621-44f4-8fcc-21c22dccd285 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:09.637652 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b766f679-4621-44f4-8fcc-21c22dccd285 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:09.637652 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b766f679-4621-44f4-8fcc-21c22dccd285 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:09.771458 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-db3a3f6d-8d95-434d-84d0-c04db73762ff tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:41:09.772175 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-db3a3f6d-8d95-434d-84d0-c04db73762ff tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] [instance: 97d5b8fb-ce56-41ba-b786-5f8474c9b2f3] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:41:09.775402 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-b766f679-4621-44f4-8fcc-21c22dccd285 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f7948b58ee714cb1bfd4d8364dae19ee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd61da83cb56412a8757633c22561495', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:41:09.783906 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-b766f679-4621-44f4-8fcc-21c22dccd285 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] 10.222.0.16 "GET /compute/v2.1/servers/cc9ed855-3638-40c6-92ae-7a4029ffae5d" status: 200 len: 1503 microversion: 2.1 time: 0.161261 Jul 16 21:41:09.785271 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2731/5458] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:41:09 2022] GET /compute/v2.1/servers/cc9ed855-3638-40c6-92ae-7a4029ffae5d => generated 1503 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:41:09.785606 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-db3a3f6d-8d95-434d-84d0-c04db73762ff tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:09.786117 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-db3a3f6d-8d95-434d-84d0-c04db73762ff tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:09.786849 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-db3a3f6d-8d95-434d-84d0-c04db73762ff tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:09.787327 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-db3a3f6d-8d95-434d-84d0-c04db73762ff tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:09.923097 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-db3a3f6d-8d95-434d-84d0-c04db73762ff tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '95cd6b54d8164de1a524b407a1852ebd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '56b8099d9406431284b4dd58b5c16359', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:41:09.931344 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-db3a3f6d-8d95-434d-84d0-c04db73762ff tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] 10.222.0.16 "GET /compute/v2.1/servers/97d5b8fb-ce56-41ba-b786-5f8474c9b2f3" status: 200 len: 1508 microversion: 2.1 time: 0.164917 Jul 16 21:41:09.931921 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2728/5459] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:41:09 2022] GET /compute/v2.1/servers/97d5b8fb-ce56-41ba-b786-5f8474c9b2f3 => generated 1508 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:41:09.948417 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-1a378e7c-8613-4714-bc0a-815020de77ef tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "f4e6fb31-921e-4c81-bf9b-833752a58dfc", "device": "/dev/sdb"}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:41:09.949562 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-1a378e7c-8613-4714-bc0a-815020de77ef tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] [instance: 97d5b8fb-ce56-41ba-b786-5f8474c9b2f3] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:41:09.955858 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1a378e7c-8613-4714-bc0a-815020de77ef tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:09.956322 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1a378e7c-8613-4714-bc0a-815020de77ef tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:09.957093 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1a378e7c-8613-4714-bc0a-815020de77ef tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:09.957536 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1a378e7c-8613-4714-bc0a-815020de77ef tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:10.711365 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-1a378e7c-8613-4714-bc0a-815020de77ef tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] 10.222.0.16 "POST /compute/v2.1/servers/97d5b8fb-ce56-41ba-b786-5f8474c9b2f3/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.766769 Jul 16 21:41:10.712059 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2732/5460] 10.222.0.16 () {60 vars in 1322 bytes} [Sat Jul 16 21:41:09 2022] POST /compute/v2.1/servers/97d5b8fb-ce56-41ba-b786-5f8474c9b2f3/os-volume_attachments => generated 194 bytes in 768 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 16 21:41:10.801477 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-1166dbc9-b6cf-4634-a655-0bc7946c6f00 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:41:10.802100 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-1166dbc9-b6cf-4634-a655-0bc7946c6f00 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] [instance: cc9ed855-3638-40c6-92ae-7a4029ffae5d] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:41:10.809286 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1166dbc9-b6cf-4634-a655-0bc7946c6f00 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:10.809746 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1166dbc9-b6cf-4634-a655-0bc7946c6f00 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:10.810615 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1166dbc9-b6cf-4634-a655-0bc7946c6f00 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:10.811088 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1166dbc9-b6cf-4634-a655-0bc7946c6f00 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:10.991642 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-1166dbc9-b6cf-4634-a655-0bc7946c6f00 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f7948b58ee714cb1bfd4d8364dae19ee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd61da83cb56412a8757633c22561495', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:41:11.000273 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-1166dbc9-b6cf-4634-a655-0bc7946c6f00 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] 10.222.0.16 "GET /compute/v2.1/servers/cc9ed855-3638-40c6-92ae-7a4029ffae5d" status: 200 len: 1503 microversion: 2.1 time: 0.203194 Jul 16 21:41:11.002063 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2729/5461] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:41:10 2022] GET /compute/v2.1/servers/cc9ed855-3638-40c6-92ae-7a4029ffae5d => generated 1503 bytes in 205 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:41:11.538917 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-511db3c5-c6ff-4f0b-9170-75d7a80bbe1a req-c3c7fd91-1ce2-400c-b413-1e00c6919119 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "cc9ed855-3638-40c6-92ae-7a4029ffae5d", "name": "network-vif-deleted", "tag": "d5e68b0c-09a6-4b04-a437-f217475278e6"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:41:11.550215 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-511db3c5-c6ff-4f0b-9170-75d7a80bbe1a req-c3c7fd91-1ce2-400c-b413-1e00c6919119 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:11.550682 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-511db3c5-c6ff-4f0b-9170-75d7a80bbe1a req-c3c7fd91-1ce2-400c-b413-1e00c6919119 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:11.562008 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-511db3c5-c6ff-4f0b-9170-75d7a80bbe1a req-c3c7fd91-1ce2-400c-b413-1e00c6919119 service nova] Creating event network-vif-deleted:d5e68b0c-09a6-4b04-a437-f217475278e6 for instance cc9ed855-3638-40c6-92ae-7a4029ffae5d on nch1-master-1 Jul 16 21:41:11.567584 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-511db3c5-c6ff-4f0b-9170-75d7a80bbe1a req-c3c7fd91-1ce2-400c-b413-1e00c6919119 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.039971 Jul 16 21:41:11.568573 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2733/5462] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:41:11 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:41:12.018327 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-f3749760-88f7-4319-889c-548631ff4d79 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:41:12.022433 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-f3749760-88f7-4319-889c-548631ff4d79 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] [instance: cc9ed855-3638-40c6-92ae-7a4029ffae5d] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:41:12.026326 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f3749760-88f7-4319-889c-548631ff4d79 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:12.026822 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f3749760-88f7-4319-889c-548631ff4d79 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:12.027631 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f3749760-88f7-4319-889c-548631ff4d79 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:12.028108 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f3749760-88f7-4319-889c-548631ff4d79 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:12.060587 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f3749760-88f7-4319-889c-548631ff4d79 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:12.061052 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f3749760-88f7-4319-889c-548631ff4d79 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:12.061524 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.objects.instance [None req-f3749760-88f7-4319-889c-548631ff4d79 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Lazy-loading 'fault' on Instance uuid cc9ed855-3638-40c6-92ae-7a4029ffae5d {{(pid=138680) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:41:12.104480 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-f3749760-88f7-4319-889c-548631ff4d79 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f7948b58ee714cb1bfd4d8364dae19ee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd61da83cb56412a8757633c22561495', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:41:12.112550 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-f3749760-88f7-4319-889c-548631ff4d79 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] 10.222.0.16 "GET /compute/v2.1/servers/cc9ed855-3638-40c6-92ae-7a4029ffae5d" status: 200 len: 1304 microversion: 2.1 time: 0.098823 Jul 16 21:41:12.113130 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2730/5463] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:41:12 2022] GET /compute/v2.1/servers/cc9ed855-3638-40c6-92ae-7a4029ffae5d => generated 1304 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:41:13.131691 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-fff1516d-8f99-41d8-a5c4-c320a55153fc tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:41:13.132405 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-fff1516d-8f99-41d8-a5c4-c320a55153fc tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] [instance: cc9ed855-3638-40c6-92ae-7a4029ffae5d] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:41:13.139734 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fff1516d-8f99-41d8-a5c4-c320a55153fc tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:13.140194 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fff1516d-8f99-41d8-a5c4-c320a55153fc tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:13.140967 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fff1516d-8f99-41d8-a5c4-c320a55153fc tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:13.141694 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fff1516d-8f99-41d8-a5c4-c320a55153fc tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:13.154945 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-fff1516d-8f99-41d8-a5c4-c320a55153fc tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] HTTP exception thrown: Instance cc9ed855-3638-40c6-92ae-7a4029ffae5d could not be found. Jul 16 21:41:13.156764 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-fff1516d-8f99-41d8-a5c4-c320a55153fc tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Request method failure captured: Jul 16 21:41:13.156764 ncdv-master-1 devstack@n-api.service[138681]: request: GET /compute/v2.1/servers/cc9ed855-3638-40c6-92ae-7a4029ffae5d HTTP/1.1 Jul 16 21:41:13.156764 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:41:13.156764 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:41:13.156764 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:41:13.156764 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 21:41:13.156764 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:41:13.156764 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:41:13.156764 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:41:13.156764 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:41:13.156764 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0zBfzfsTz8nvjZ__aZKaS0OcxWaPhZotqFsyKstg35JuhL91Shc9WtTQxsRGXZcOZ3NevAgJv8vz5SU1B8bJsS81uq76QIGUkypmqIdZ-qCEEwqVe6H4UMReHiO-YHkHqEv4WBykzbJblB3Vyvt9xKyUUsi9Fc7qtZBaxcUfd5Hu5qI Jul 16 21:41:13.156764 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:41:13.156764 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:41:13.156764 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:41:13.156764 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:41:13.156764 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:41:13.156764 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:41:13.156764 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: bd61da83cb56412a8757633c22561495 Jul 16 21:41:13.156764 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-VolumesActionsTest-484965280 Jul 16 21:41:13.156764 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:41:13.156764 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:41:13.156764 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/bd61da83cb56412a8757633c22561495"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/bd61da83cb56412a8757633c22561495"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_bd61da83cb56412a8757633c22561495"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/bd61da83cb56412a8757633c22561495"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:41:13.156764 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-VolumesActionsTest-484965280 Jul 16 21:41:13.156764 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: bd61da83cb56412a8757633c22561495 Jul 16 21:41:13.156764 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-VolumesActionsTest-484965280 Jul 16 21:41:13.156764 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-VolumesActionsTest-484965280-project Jul 16 21:41:13.156764 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:41:13.156764 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:41:13.156764 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: f7948b58ee714cb1bfd4d8364dae19ee Jul 16 21:41:13.156764 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-VolumesActionsTest-484965280-project Jul 16 21:41:13.156764 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:41:13.156764 ncdv-master-1 devstack@n-api.service[138681]: exception: Instance cc9ed855-3638-40c6-92ae-7a4029ffae5d could not be found. Jul 16 21:41:13.156764 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': 'cc9ed855-3638-40c6-92ae-7a4029ffae5d'} Jul 16 21:41:13.156764 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:41:13.156764 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:41:13.156764 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:41:13.156764 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:41:13.156764 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:41:13.156764 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:41:13.156764 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:41:13.156764 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:41:13.156764 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:41:13.156764 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:41:13.156764 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance cc9ed855-3638-40c6-92ae-7a4029ffae5d could not be found. Jul 16 21:41:13.156764 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:41:13.156764 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:41:13.156764 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:41:13.156764 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:41:13.156764 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:41:13.156764 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:41:13.156764 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:41:13.156764 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:41:13.156764 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:41:13.156764 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:41:13.156764 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:41:13.156764 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:41:13.156764 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:41:13.156764 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:41:13.156764 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:41:13.156764 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:41:13.156764 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance cc9ed855-3638-40c6-92ae-7a4029ffae5d could not be found. Jul 16 21:41:13.156764 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:41:13.156764 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:41:13.156764 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:41:13.162852 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:41:13.162852 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:41:13.162852 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:41:13.162852 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:41:13.162852 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:41:13.162852 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance cc9ed855-3638-40c6-92ae-7a4029ffae5d could not be found. Jul 16 21:41:13.162852 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:41:13.162852 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-fff1516d-8f99-41d8-a5c4-c320a55153fc tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Returning 404 to user: Instance cc9ed855-3638-40c6-92ae-7a4029ffae5d could not be found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:41:13.162852 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-fff1516d-8f99-41d8-a5c4-c320a55153fc tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] 10.222.0.16 "GET /compute/v2.1/servers/cc9ed855-3638-40c6-92ae-7a4029ffae5d" status: 404 len: 111 microversion: 2.1 time: 0.031039 Jul 16 21:41:13.162852 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2734/5464] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:41:13 2022] GET /compute/v2.1/servers/cc9ed855-3638-40c6-92ae-7a4029ffae5d => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:41:13.209643 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-4cba47c8-0941-4e04-adc8-638c6daa8e8c tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesActionsTest-instance-112825963", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452", "networks": [{"uuid": "b5201600-e01f-45ea-ba7f-ceb596ae69cf"}]}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:41:13.262011 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.network.neutron [None req-4cba47c8-0941-4e04-adc8-638c6daa8e8c tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] validate_networks() for [('b5201600-e01f-45ea-ba7f-ceb596ae69cf', None, None, None, None, None)] {{(pid=138680) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 16 21:41:13.614049 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-4cba47c8-0941-4e04-adc8-638c6daa8e8c tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Flavor limits 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 16 21:41:13.614460 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-4cba47c8-0941-4e04-adc8-638c6daa8e8c tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Image limits 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 16 21:41:13.614937 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-4cba47c8-0941-4e04-adc8-638c6daa8e8c tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Flavor pref 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 16 21:41:13.615422 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-4cba47c8-0941-4e04-adc8-638c6daa8e8c tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Image pref 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 16 21:41:13.616051 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-4cba47c8-0941-4e04-adc8-638c6daa8e8c tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 16 21:41:13.633000 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4cba47c8-0941-4e04-adc8-638c6daa8e8c tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:13.633000 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4cba47c8-0941-4e04-adc8-638c6daa8e8c tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:13.633000 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4cba47c8-0941-4e04-adc8-638c6daa8e8c tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:13.633000 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4cba47c8-0941-4e04-adc8-638c6daa8e8c tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:13.644869 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-4cba47c8-0941-4e04-adc8-638c6daa8e8c tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Getting quotas for project bd61da83cb56412a8757633c22561495. Resources: {'ram', 'instances', 'cores'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:41:13.650187 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-4cba47c8-0941-4e04-adc8-638c6daa8e8c tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Getting quotas for user f7948b58ee714cb1bfd4d8364dae19ee and project bd61da83cb56412a8757633c22561495. Resources: {'ram', 'instances', 'cores'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:41:13.663535 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-4cba47c8-0941-4e04-adc8-638c6daa8e8c tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Going to run 1 instances... {{(pid=138680) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 16 21:41:13.665839 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-4cba47c8-0941-4e04-adc8-638c6daa8e8c tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] [instance: 9df14d8c-4fff-439d-af43-10a008a7b9d2] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138680) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 16 21:41:13.789895 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-4cba47c8-0941-4e04-adc8-638c6daa8e8c tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.584272 Jul 16 21:41:13.791258 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2731/5465] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 21:41:13 2022] POST /compute/v2.1/servers => generated 380 bytes in 586 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 16 21:41:13.801004 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-a9166c65-e583-4e29-971c-1fa6d3ee68f9 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:41:13.801642 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-a9166c65-e583-4e29-971c-1fa6d3ee68f9 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] [instance: 9df14d8c-4fff-439d-af43-10a008a7b9d2] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:41:13.860358 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-a9166c65-e583-4e29-971c-1fa6d3ee68f9 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f7948b58ee714cb1bfd4d8364dae19ee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd61da83cb56412a8757633c22561495', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:41:13.870435 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-a9166c65-e583-4e29-971c-1fa6d3ee68f9 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] 10.222.0.16 "GET /compute/v2.1/servers/9df14d8c-4fff-439d-af43-10a008a7b9d2" status: 200 len: 1213 microversion: 2.1 time: 0.073550 Jul 16 21:41:13.871347 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2735/5466] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:41:13 2022] GET /compute/v2.1/servers/9df14d8c-4fff-439d-af43-10a008a7b9d2 => generated 1213 bytes in 75 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:41:14.889746 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-608bb0c6-30aa-4354-8628-e5d7a63a7fee tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:41:14.890494 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-608bb0c6-30aa-4354-8628-e5d7a63a7fee tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] [instance: 9df14d8c-4fff-439d-af43-10a008a7b9d2] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:41:14.897809 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-608bb0c6-30aa-4354-8628-e5d7a63a7fee tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:14.898274 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-608bb0c6-30aa-4354-8628-e5d7a63a7fee tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:14.899082 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-608bb0c6-30aa-4354-8628-e5d7a63a7fee tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:14.899536 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-608bb0c6-30aa-4354-8628-e5d7a63a7fee tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:14.976528 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-608bb0c6-30aa-4354-8628-e5d7a63a7fee tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f7948b58ee714cb1bfd4d8364dae19ee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd61da83cb56412a8757633c22561495', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:41:14.985556 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-608bb0c6-30aa-4354-8628-e5d7a63a7fee tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] 10.222.0.16 "GET /compute/v2.1/servers/9df14d8c-4fff-439d-af43-10a008a7b9d2" status: 200 len: 1217 microversion: 2.1 time: 0.100658 Jul 16 21:41:14.986151 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2732/5467] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:41:14 2022] GET /compute/v2.1/servers/9df14d8c-4fff-439d-af43-10a008a7b9d2 => generated 1217 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:41:16.001959 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-acb452c6-5eaf-4fc2-8b23-ea7771d30278 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:41:16.002625 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-acb452c6-5eaf-4fc2-8b23-ea7771d30278 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] [instance: 9df14d8c-4fff-439d-af43-10a008a7b9d2] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:41:16.012702 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-acb452c6-5eaf-4fc2-8b23-ea7771d30278 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:16.013296 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-acb452c6-5eaf-4fc2-8b23-ea7771d30278 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:16.014797 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-acb452c6-5eaf-4fc2-8b23-ea7771d30278 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:16.015287 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-acb452c6-5eaf-4fc2-8b23-ea7771d30278 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:16.125426 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-acb452c6-5eaf-4fc2-8b23-ea7771d30278 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f7948b58ee714cb1bfd4d8364dae19ee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd61da83cb56412a8757633c22561495', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:41:16.135296 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-acb452c6-5eaf-4fc2-8b23-ea7771d30278 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] 10.222.0.16 "GET /compute/v2.1/servers/9df14d8c-4fff-439d-af43-10a008a7b9d2" status: 200 len: 1273 microversion: 2.1 time: 0.137323 Jul 16 21:41:16.136088 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2736/5468] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:41:15 2022] GET /compute/v2.1/servers/9df14d8c-4fff-439d-af43-10a008a7b9d2 => generated 1273 bytes in 139 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:41:17.152998 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-c6cd087a-c591-4013-a31d-7bac173c6149 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:41:17.153726 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-c6cd087a-c591-4013-a31d-7bac173c6149 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] [instance: 9df14d8c-4fff-439d-af43-10a008a7b9d2] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:41:17.161774 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c6cd087a-c591-4013-a31d-7bac173c6149 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:17.162245 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c6cd087a-c591-4013-a31d-7bac173c6149 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:17.163040 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c6cd087a-c591-4013-a31d-7bac173c6149 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:17.163481 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c6cd087a-c591-4013-a31d-7bac173c6149 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:17.455969 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-c6cd087a-c591-4013-a31d-7bac173c6149 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f7948b58ee714cb1bfd4d8364dae19ee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd61da83cb56412a8757633c22561495', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:41:17.464311 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-c6cd087a-c591-4013-a31d-7bac173c6149 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] 10.222.0.16 "GET /compute/v2.1/servers/9df14d8c-4fff-439d-af43-10a008a7b9d2" status: 200 len: 1313 microversion: 2.1 time: 0.315147 Jul 16 21:41:17.464879 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2733/5469] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:41:17 2022] GET /compute/v2.1/servers/9df14d8c-4fff-439d-af43-10a008a7b9d2 => generated 1313 bytes in 316 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:41:18.482229 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-74282da7-9a53-4242-80c7-f8e3f26ccd60 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:41:18.482920 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-74282da7-9a53-4242-80c7-f8e3f26ccd60 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] [instance: 9df14d8c-4fff-439d-af43-10a008a7b9d2] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:41:18.490632 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-74282da7-9a53-4242-80c7-f8e3f26ccd60 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:18.491255 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-74282da7-9a53-4242-80c7-f8e3f26ccd60 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:18.492049 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-74282da7-9a53-4242-80c7-f8e3f26ccd60 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:18.492509 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-74282da7-9a53-4242-80c7-f8e3f26ccd60 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:18.516259 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-60dc8ca3-ecd4-4b4a-bc4c-5dbb3bbd1d36 req-3a5f3867-a90c-40de-9508-e72d06be7ea6 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "9df14d8c-4fff-439d-af43-10a008a7b9d2", "tag": "6f102b18-5b0b-4710-90a7-2c2c4d79d1fd"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:41:18.527279 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-60dc8ca3-ecd4-4b4a-bc4c-5dbb3bbd1d36 req-3a5f3867-a90c-40de-9508-e72d06be7ea6 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:18.527729 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-60dc8ca3-ecd4-4b4a-bc4c-5dbb3bbd1d36 req-3a5f3867-a90c-40de-9508-e72d06be7ea6 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:18.539615 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-60dc8ca3-ecd4-4b4a-bc4c-5dbb3bbd1d36 req-3a5f3867-a90c-40de-9508-e72d06be7ea6 service nova] Creating event network-changed:6f102b18-5b0b-4710-90a7-2c2c4d79d1fd for instance 9df14d8c-4fff-439d-af43-10a008a7b9d2 on nch2-master-1 Jul 16 21:41:18.544524 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-60dc8ca3-ecd4-4b4a-bc4c-5dbb3bbd1d36 req-3a5f3867-a90c-40de-9508-e72d06be7ea6 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.032886 Jul 16 21:41:18.545105 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2734/5470] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:41:18 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 16 21:41:18.743896 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-74282da7-9a53-4242-80c7-f8e3f26ccd60 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f7948b58ee714cb1bfd4d8364dae19ee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd61da83cb56412a8757633c22561495', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:41:18.751460 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-74282da7-9a53-4242-80c7-f8e3f26ccd60 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] 10.222.0.16 "GET /compute/v2.1/servers/9df14d8c-4fff-439d-af43-10a008a7b9d2" status: 200 len: 1313 microversion: 2.1 time: 0.273508 Jul 16 21:41:18.752037 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2737/5471] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:41:18 2022] GET /compute/v2.1/servers/9df14d8c-4fff-439d-af43-10a008a7b9d2 => generated 1313 bytes in 275 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:41:19.770814 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-7734c476-e3ff-4ea1-95bb-b5082fadacd6 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:41:19.771569 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-7734c476-e3ff-4ea1-95bb-b5082fadacd6 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] [instance: 9df14d8c-4fff-439d-af43-10a008a7b9d2] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:41:19.777980 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7734c476-e3ff-4ea1-95bb-b5082fadacd6 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:19.778435 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7734c476-e3ff-4ea1-95bb-b5082fadacd6 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:19.779257 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7734c476-e3ff-4ea1-95bb-b5082fadacd6 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:19.779704 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7734c476-e3ff-4ea1-95bb-b5082fadacd6 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:19.918155 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-7734c476-e3ff-4ea1-95bb-b5082fadacd6 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f7948b58ee714cb1bfd4d8364dae19ee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd61da83cb56412a8757633c22561495', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:41:19.927154 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-7734c476-e3ff-4ea1-95bb-b5082fadacd6 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] 10.222.0.16 "GET /compute/v2.1/servers/9df14d8c-4fff-439d-af43-10a008a7b9d2" status: 200 len: 1313 microversion: 2.1 time: 0.161487 Jul 16 21:41:19.927782 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2735/5472] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:41:19 2022] GET /compute/v2.1/servers/9df14d8c-4fff-439d-af43-10a008a7b9d2 => generated 1313 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:41:20.390955 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-fc53307f-a8c7-44a7-9bfa-36ae4a80ac95 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:41:20.390955 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-fc53307f-a8c7-44a7-9bfa-36ae4a80ac95 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] [instance: 997f89c5-8086-4a7e-8292-1c8e6a12aaf6] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:41:20.393397 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fc53307f-a8c7-44a7-9bfa-36ae4a80ac95 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:20.394220 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fc53307f-a8c7-44a7-9bfa-36ae4a80ac95 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:20.395339 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fc53307f-a8c7-44a7-9bfa-36ae4a80ac95 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:20.395803 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fc53307f-a8c7-44a7-9bfa-36ae4a80ac95 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:20.426938 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-fc53307f-a8c7-44a7-9bfa-36ae4a80ac95 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] [instance: 997f89c5-8086-4a7e-8292-1c8e6a12aaf6] Going to try to terminate instance {{(pid=138681) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 16 21:41:20.718969 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-fc53307f-a8c7-44a7-9bfa-36ae4a80ac95 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] 10.222.0.16 "DELETE /compute/v2.1/servers/997f89c5-8086-4a7e-8292-1c8e6a12aaf6" status: 204 len: 0 microversion: 2.1 time: 0.341056 Jul 16 21:41:20.719553 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2738/5473] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:41:20 2022] DELETE /compute/v2.1/servers/997f89c5-8086-4a7e-8292-1c8e6a12aaf6 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 16 21:41:20.728435 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-0b25df3a-6f2e-409c-b484-a634dd1a7c0f tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:41:20.734172 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-0b25df3a-6f2e-409c-b484-a634dd1a7c0f tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] [instance: 997f89c5-8086-4a7e-8292-1c8e6a12aaf6] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:41:20.735531 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0b25df3a-6f2e-409c-b484-a634dd1a7c0f tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:20.735997 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0b25df3a-6f2e-409c-b484-a634dd1a7c0f tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:20.736779 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0b25df3a-6f2e-409c-b484-a634dd1a7c0f tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:20.737352 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0b25df3a-6f2e-409c-b484-a634dd1a7c0f tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:20.879576 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-0b25df3a-6f2e-409c-b484-a634dd1a7c0f tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '62209b9802ab46c3b3d094d4d1ad86fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b533f85aeddf4d8db05f2c53a61befc3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:41:20.887394 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-0b25df3a-6f2e-409c-b484-a634dd1a7c0f tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] 10.222.0.16 "GET /compute/v2.1/servers/997f89c5-8086-4a7e-8292-1c8e6a12aaf6" status: 200 len: 1410 microversion: 2.1 time: 0.162883 Jul 16 21:41:20.888004 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2736/5474] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:41:20 2022] GET /compute/v2.1/servers/997f89c5-8086-4a7e-8292-1c8e6a12aaf6 => generated 1410 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:41:20.945094 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-17a24bc1-c9c3-451b-9e92-363bc5868505 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:41:20.945811 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-17a24bc1-c9c3-451b-9e92-363bc5868505 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] [instance: 9df14d8c-4fff-439d-af43-10a008a7b9d2] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:41:20.947839 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_api:SELECT=93,nova_api:INSERT=6,nova_api:UPDATE=1 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:41:20.952651 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-17a24bc1-c9c3-451b-9e92-363bc5868505 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:20.953113 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-17a24bc1-c9c3-451b-9e92-363bc5868505 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:20.953894 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-17a24bc1-c9c3-451b-9e92-363bc5868505 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:20.954346 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-17a24bc1-c9c3-451b-9e92-363bc5868505 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:21.088843 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-17a24bc1-c9c3-451b-9e92-363bc5868505 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f7948b58ee714cb1bfd4d8364dae19ee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd61da83cb56412a8757633c22561495', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:41:21.096574 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-17a24bc1-c9c3-451b-9e92-363bc5868505 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] 10.222.0.16 "GET /compute/v2.1/servers/9df14d8c-4fff-439d-af43-10a008a7b9d2" status: 200 len: 1313 microversion: 2.1 time: 0.156258 Jul 16 21:41:21.097857 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2739/5475] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:41:20 2022] GET /compute/v2.1/servers/9df14d8c-4fff-439d-af43-10a008a7b9d2 => generated 1313 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:41:21.904368 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-4935cf24-c9b3-464f-be02-54ceb2568c8b tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:41:21.904983 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-4935cf24-c9b3-464f-be02-54ceb2568c8b tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] [instance: 997f89c5-8086-4a7e-8292-1c8e6a12aaf6] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:41:21.912893 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4935cf24-c9b3-464f-be02-54ceb2568c8b tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:21.913345 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4935cf24-c9b3-464f-be02-54ceb2568c8b tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:21.914109 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4935cf24-c9b3-464f-be02-54ceb2568c8b tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:21.914547 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4935cf24-c9b3-464f-be02-54ceb2568c8b tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:22.050235 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-4935cf24-c9b3-464f-be02-54ceb2568c8b tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '62209b9802ab46c3b3d094d4d1ad86fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b533f85aeddf4d8db05f2c53a61befc3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:41:22.058221 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-4935cf24-c9b3-464f-be02-54ceb2568c8b tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] 10.222.0.16 "GET /compute/v2.1/servers/997f89c5-8086-4a7e-8292-1c8e6a12aaf6" status: 200 len: 1410 microversion: 2.1 time: 0.157864 Jul 16 21:41:22.058804 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2737/5476] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:41:21 2022] GET /compute/v2.1/servers/997f89c5-8086-4a7e-8292-1c8e6a12aaf6 => generated 1410 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:41:22.113650 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-c7877f8c-7699-46fb-9ba3-7ba5553dfaf0 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:41:22.114221 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-c7877f8c-7699-46fb-9ba3-7ba5553dfaf0 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] [instance: 9df14d8c-4fff-439d-af43-10a008a7b9d2] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:41:22.123372 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c7877f8c-7699-46fb-9ba3-7ba5553dfaf0 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:22.123372 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c7877f8c-7699-46fb-9ba3-7ba5553dfaf0 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:22.131954 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c7877f8c-7699-46fb-9ba3-7ba5553dfaf0 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:22.131954 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c7877f8c-7699-46fb-9ba3-7ba5553dfaf0 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:22.275000 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-c7877f8c-7699-46fb-9ba3-7ba5553dfaf0 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f7948b58ee714cb1bfd4d8364dae19ee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd61da83cb56412a8757633c22561495', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:41:22.277583 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-c7877f8c-7699-46fb-9ba3-7ba5553dfaf0 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] 10.222.0.16 "GET /compute/v2.1/servers/9df14d8c-4fff-439d-af43-10a008a7b9d2" status: 200 len: 1313 microversion: 2.1 time: 0.167171 Jul 16 21:41:22.286125 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2740/5477] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:41:22 2022] GET /compute/v2.1/servers/9df14d8c-4fff-439d-af43-10a008a7b9d2 => generated 1313 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:41:23.080874 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-b9000bc1-2847-4194-8330-ebeee5f8e5f9 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:41:23.080874 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-b9000bc1-2847-4194-8330-ebeee5f8e5f9 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] [instance: 997f89c5-8086-4a7e-8292-1c8e6a12aaf6] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:41:23.085459 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b9000bc1-2847-4194-8330-ebeee5f8e5f9 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:23.089239 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b9000bc1-2847-4194-8330-ebeee5f8e5f9 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:23.089239 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b9000bc1-2847-4194-8330-ebeee5f8e5f9 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:23.089239 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b9000bc1-2847-4194-8330-ebeee5f8e5f9 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:23.295509 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-76bba492-93f4-46dd-a8e4-170900e10194 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:41:23.296772 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-76bba492-93f4-46dd-a8e4-170900e10194 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] [instance: 9df14d8c-4fff-439d-af43-10a008a7b9d2] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:41:23.306150 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-76bba492-93f4-46dd-a8e4-170900e10194 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:23.307534 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-76bba492-93f4-46dd-a8e4-170900e10194 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:23.309226 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-76bba492-93f4-46dd-a8e4-170900e10194 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:23.310141 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-76bba492-93f4-46dd-a8e4-170900e10194 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:23.327354 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-b9000bc1-2847-4194-8330-ebeee5f8e5f9 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '62209b9802ab46c3b3d094d4d1ad86fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b533f85aeddf4d8db05f2c53a61befc3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:41:23.337943 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-b9000bc1-2847-4194-8330-ebeee5f8e5f9 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] 10.222.0.16 "GET /compute/v2.1/servers/997f89c5-8086-4a7e-8292-1c8e6a12aaf6" status: 200 len: 1410 microversion: 2.1 time: 0.265256 Jul 16 21:41:23.349039 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2738/5478] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:41:23 2022] GET /compute/v2.1/servers/997f89c5-8086-4a7e-8292-1c8e6a12aaf6 => generated 1410 bytes in 277 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:41:23.508585 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-76bba492-93f4-46dd-a8e4-170900e10194 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f7948b58ee714cb1bfd4d8364dae19ee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd61da83cb56412a8757633c22561495', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:41:23.512265 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=95,nova_cell1:UPDATE=3,nova_cell1:INSERT=2 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:41:23.516990 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-76bba492-93f4-46dd-a8e4-170900e10194 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] 10.222.0.16 "GET /compute/v2.1/servers/9df14d8c-4fff-439d-af43-10a008a7b9d2" status: 200 len: 1313 microversion: 2.1 time: 0.226155 Jul 16 21:41:23.517693 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2741/5479] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:41:23 2022] GET /compute/v2.1/servers/9df14d8c-4fff-439d-af43-10a008a7b9d2 => generated 1313 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:41:23.752474 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-aeab91fa-6cb1-41c0-9bc9-e3a7ea270a76 req-569bf622-2a91-44ea-9222-0e5716ff8307 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "997f89c5-8086-4a7e-8292-1c8e6a12aaf6", "name": "network-vif-deleted", "tag": "60aea633-216f-4cec-a388-963ef6e6e2a0"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:41:23.758290 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell0:SELECT=2 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:41:23.758996 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell0:SELECT=6 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:41:23.766477 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-aeab91fa-6cb1-41c0-9bc9-e3a7ea270a76 req-569bf622-2a91-44ea-9222-0e5716ff8307 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:23.766477 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-aeab91fa-6cb1-41c0-9bc9-e3a7ea270a76 req-569bf622-2a91-44ea-9222-0e5716ff8307 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:23.777254 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-aeab91fa-6cb1-41c0-9bc9-e3a7ea270a76 req-569bf622-2a91-44ea-9222-0e5716ff8307 service nova] Creating event network-vif-deleted:60aea633-216f-4cec-a388-963ef6e6e2a0 for instance 997f89c5-8086-4a7e-8292-1c8e6a12aaf6 on nch2-master-1 Jul 16 21:41:23.782598 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-aeab91fa-6cb1-41c0-9bc9-e3a7ea270a76 req-569bf622-2a91-44ea-9222-0e5716ff8307 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.034494 Jul 16 21:41:23.791591 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2739/5480] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:41:23 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:41:24.355993 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-b027af40-eb9e-42b4-9c9f-dbb7b5e572be tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:41:24.356914 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-b027af40-eb9e-42b4-9c9f-dbb7b5e572be tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] [instance: 997f89c5-8086-4a7e-8292-1c8e6a12aaf6] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:41:24.358814 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell0:SELECT=2 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:41:24.365775 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b027af40-eb9e-42b4-9c9f-dbb7b5e572be tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:24.366234 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b027af40-eb9e-42b4-9c9f-dbb7b5e572be tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:24.367029 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b027af40-eb9e-42b4-9c9f-dbb7b5e572be tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:24.367621 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b027af40-eb9e-42b4-9c9f-dbb7b5e572be tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:24.444062 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-b027af40-eb9e-42b4-9c9f-dbb7b5e572be tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '62209b9802ab46c3b3d094d4d1ad86fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b533f85aeddf4d8db05f2c53a61befc3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:41:24.451753 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-b027af40-eb9e-42b4-9c9f-dbb7b5e572be tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] 10.222.0.16 "GET /compute/v2.1/servers/997f89c5-8086-4a7e-8292-1c8e6a12aaf6" status: 200 len: 1202 microversion: 2.1 time: 0.100085 Jul 16 21:41:24.452593 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2742/5481] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:41:24 2022] GET /compute/v2.1/servers/997f89c5-8086-4a7e-8292-1c8e6a12aaf6 => generated 1202 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:41:24.540401 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-c46501a5-b083-46ef-bf29-b0227f0c6f8a tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:41:24.544326 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-c46501a5-b083-46ef-bf29-b0227f0c6f8a tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] [instance: 9df14d8c-4fff-439d-af43-10a008a7b9d2] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:41:24.549118 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c46501a5-b083-46ef-bf29-b0227f0c6f8a tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:24.549584 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c46501a5-b083-46ef-bf29-b0227f0c6f8a tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:24.551569 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c46501a5-b083-46ef-bf29-b0227f0c6f8a tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:24.551995 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c46501a5-b083-46ef-bf29-b0227f0c6f8a tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:24.700456 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-c46501a5-b083-46ef-bf29-b0227f0c6f8a tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f7948b58ee714cb1bfd4d8364dae19ee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd61da83cb56412a8757633c22561495', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:41:24.709427 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-c46501a5-b083-46ef-bf29-b0227f0c6f8a tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] 10.222.0.16 "GET /compute/v2.1/servers/9df14d8c-4fff-439d-af43-10a008a7b9d2" status: 200 len: 1313 microversion: 2.1 time: 0.177285 Jul 16 21:41:24.710006 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2740/5482] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:41:24 2022] GET /compute/v2.1/servers/9df14d8c-4fff-439d-af43-10a008a7b9d2 => generated 1313 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:41:25.448474 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-f8b076e9-a666-499c-869f-3dfc0aec6950 req-4bc06fec-ec0d-473b-bddb-b6b2ff7858e4 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "9df14d8c-4fff-439d-af43-10a008a7b9d2", "name": "network-vif-plugged", "status": "completed", "tag": "6f102b18-5b0b-4710-90a7-2c2c4d79d1fd"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:41:25.458812 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-f8b076e9-a666-499c-869f-3dfc0aec6950 req-4bc06fec-ec0d-473b-bddb-b6b2ff7858e4 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:25.459452 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-f8b076e9-a666-499c-869f-3dfc0aec6950 req-4bc06fec-ec0d-473b-bddb-b6b2ff7858e4 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:25.469926 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-c2640e59-4973-4ed6-af51-a4c992ace8f0 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:41:25.470649 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-f8b076e9-a666-499c-869f-3dfc0aec6950 req-4bc06fec-ec0d-473b-bddb-b6b2ff7858e4 service nova] Creating event network-vif-plugged:6f102b18-5b0b-4710-90a7-2c2c4d79d1fd for instance 9df14d8c-4fff-439d-af43-10a008a7b9d2 on nch2-master-1 Jul 16 21:41:25.470802 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-c2640e59-4973-4ed6-af51-a4c992ace8f0 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] [instance: 997f89c5-8086-4a7e-8292-1c8e6a12aaf6] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:41:25.476649 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-f8b076e9-a666-499c-869f-3dfc0aec6950 req-4bc06fec-ec0d-473b-bddb-b6b2ff7858e4 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.032005 Jul 16 21:41:25.477255 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2743/5483] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:41:25 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 16 21:41:25.478739 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c2640e59-4973-4ed6-af51-a4c992ace8f0 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:25.479328 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c2640e59-4973-4ed6-af51-a4c992ace8f0 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:25.480086 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c2640e59-4973-4ed6-af51-a4c992ace8f0 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:25.480532 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c2640e59-4973-4ed6-af51-a4c992ace8f0 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:25.496914 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-c2640e59-4973-4ed6-af51-a4c992ace8f0 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] HTTP exception thrown: Instance 997f89c5-8086-4a7e-8292-1c8e6a12aaf6 could not be found. Jul 16 21:41:25.498647 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-c2640e59-4973-4ed6-af51-a4c992ace8f0 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Request method failure captured: Jul 16 21:41:25.498647 ncdv-master-1 devstack@n-api.service[138680]: request: GET /compute/v2.1/servers/997f89c5-8086-4a7e-8292-1c8e6a12aaf6 HTTP/1.1 Jul 16 21:41:25.498647 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:41:25.498647 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:41:25.498647 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:41:25.498647 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 0 Jul 16 21:41:25.498647 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:41:25.498647 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:41:25.498647 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:41:25.498647 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:41:25.498647 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0y_WdH7eI0DbXzeyrxNR2zaPLU2Z0T2hgcgJaNL5zKj0IOkua_mwRXGhv8UgaywTvIB56iq06rxK294Knh0aCrbToiJY0kPn8yPuGVzddBFopXnN-hL_u7FbpY8zvcJRiL0zpWIKlPhQExzLv95jQK7KF6D9Y-KawL2vbM6jbQCfNnU Jul 16 21:41:25.498647 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:41:25.498647 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:41:25.498647 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:41:25.498647 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:41:25.498647 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:41:25.498647 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:41:25.498647 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: b533f85aeddf4d8db05f2c53a61befc3 Jul 16 21:41:25.498647 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-TestVolumeBootPattern-1625426921 Jul 16 21:41:25.498647 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:41:25.498647 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:41:25.498647 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/b533f85aeddf4d8db05f2c53a61befc3"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/b533f85aeddf4d8db05f2c53a61befc3"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_b533f85aeddf4d8db05f2c53a61befc3"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/b533f85aeddf4d8db05f2c53a61befc3"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:41:25.498647 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-TestVolumeBootPattern-1625426921 Jul 16 21:41:25.498647 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: b533f85aeddf4d8db05f2c53a61befc3 Jul 16 21:41:25.498647 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-TestVolumeBootPattern-1625426921 Jul 16 21:41:25.498647 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-TestVolumeBootPattern-1625426921-project Jul 16 21:41:25.498647 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:41:25.498647 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:41:25.498647 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: 62209b9802ab46c3b3d094d4d1ad86fe Jul 16 21:41:25.498647 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-TestVolumeBootPattern-1625426921-project Jul 16 21:41:25.498647 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 21:41:25.498647 ncdv-master-1 devstack@n-api.service[138680]: exception: Instance 997f89c5-8086-4a7e-8292-1c8e6a12aaf6 could not be found. Jul 16 21:41:25.498647 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': '997f89c5-8086-4a7e-8292-1c8e6a12aaf6'} Jul 16 21:41:25.498647 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:41:25.498647 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:41:25.498647 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:41:25.498647 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:41:25.498647 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:41:25.498647 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:41:25.498647 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:41:25.498647 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:41:25.498647 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:41:25.498647 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:41:25.498647 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 997f89c5-8086-4a7e-8292-1c8e6a12aaf6 could not be found. Jul 16 21:41:25.498647 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:41:25.498647 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:41:25.498647 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:41:25.498647 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:41:25.498647 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:41:25.498647 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:41:25.498647 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:41:25.498647 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:41:25.498647 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:41:25.498647 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:41:25.498647 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:41:25.498647 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:41:25.498647 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:41:25.498647 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:41:25.498647 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:41:25.498647 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:41:25.498647 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 997f89c5-8086-4a7e-8292-1c8e6a12aaf6 could not be found. Jul 16 21:41:25.498647 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:41:25.498647 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:41:25.498647 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:41:25.504622 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:41:25.504622 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:41:25.504622 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:41:25.504622 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:41:25.504622 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:41:25.504622 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 997f89c5-8086-4a7e-8292-1c8e6a12aaf6 could not be found. Jul 16 21:41:25.504622 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:41:25.504622 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-c2640e59-4973-4ed6-af51-a4c992ace8f0 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Returning 404 to user: Instance 997f89c5-8086-4a7e-8292-1c8e6a12aaf6 could not be found. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:41:25.504622 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-c2640e59-4973-4ed6-af51-a4c992ace8f0 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] 10.222.0.16 "GET /compute/v2.1/servers/997f89c5-8086-4a7e-8292-1c8e6a12aaf6" status: 404 len: 111 microversion: 2.1 time: 0.034282 Jul 16 21:41:25.504622 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2741/5484] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:41:25 2022] GET /compute/v2.1/servers/997f89c5-8086-4a7e-8292-1c8e6a12aaf6 => generated 111 bytes in 36 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Jul 16 21:41:25.731456 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-da561cf1-cbe8-410e-aa4e-b3e50d5f5ac2 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:41:25.731456 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-da561cf1-cbe8-410e-aa4e-b3e50d5f5ac2 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] [instance: 9df14d8c-4fff-439d-af43-10a008a7b9d2] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:41:25.736464 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-da561cf1-cbe8-410e-aa4e-b3e50d5f5ac2 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:25.743430 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-da561cf1-cbe8-410e-aa4e-b3e50d5f5ac2 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:25.743430 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-da561cf1-cbe8-410e-aa4e-b3e50d5f5ac2 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:25.743430 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-da561cf1-cbe8-410e-aa4e-b3e50d5f5ac2 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:25.878924 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-da561cf1-cbe8-410e-aa4e-b3e50d5f5ac2 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f7948b58ee714cb1bfd4d8364dae19ee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd61da83cb56412a8757633c22561495', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:41:25.886527 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-da561cf1-cbe8-410e-aa4e-b3e50d5f5ac2 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] 10.222.0.16 "GET /compute/v2.1/servers/9df14d8c-4fff-439d-af43-10a008a7b9d2" status: 200 len: 1313 microversion: 2.1 time: 0.162679 Jul 16 21:41:25.887129 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2744/5485] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:41:25 2022] GET /compute/v2.1/servers/9df14d8c-4fff-439d-af43-10a008a7b9d2 => generated 1313 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:41:26.903722 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-71f1c429-5b74-4bb3-a8e1-3ea75aa948c6 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:41:26.904342 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-71f1c429-5b74-4bb3-a8e1-3ea75aa948c6 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] [instance: 9df14d8c-4fff-439d-af43-10a008a7b9d2] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:41:26.911970 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-71f1c429-5b74-4bb3-a8e1-3ea75aa948c6 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:26.912430 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-71f1c429-5b74-4bb3-a8e1-3ea75aa948c6 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:26.913194 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-71f1c429-5b74-4bb3-a8e1-3ea75aa948c6 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:26.913640 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-71f1c429-5b74-4bb3-a8e1-3ea75aa948c6 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:26.933260 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:41:27.055580 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-71f1c429-5b74-4bb3-a8e1-3ea75aa948c6 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f7948b58ee714cb1bfd4d8364dae19ee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd61da83cb56412a8757633c22561495', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:41:27.063933 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-71f1c429-5b74-4bb3-a8e1-3ea75aa948c6 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] 10.222.0.16 "GET /compute/v2.1/servers/9df14d8c-4fff-439d-af43-10a008a7b9d2" status: 200 len: 1313 microversion: 2.1 time: 0.164025 Jul 16 21:41:27.064504 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2742/5486] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:41:26 2022] GET /compute/v2.1/servers/9df14d8c-4fff-439d-af43-10a008a7b9d2 => generated 1313 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:41:28.081665 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-624134b1-caa3-4102-a2c6-1533ef0099d3 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:41:28.083369 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-624134b1-caa3-4102-a2c6-1533ef0099d3 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] [instance: 9df14d8c-4fff-439d-af43-10a008a7b9d2] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:41:28.090147 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-624134b1-caa3-4102-a2c6-1533ef0099d3 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:28.090641 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-624134b1-caa3-4102-a2c6-1533ef0099d3 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:28.091507 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-624134b1-caa3-4102-a2c6-1533ef0099d3 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:28.091949 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-624134b1-caa3-4102-a2c6-1533ef0099d3 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:28.229527 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-624134b1-caa3-4102-a2c6-1533ef0099d3 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f7948b58ee714cb1bfd4d8364dae19ee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd61da83cb56412a8757633c22561495', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:41:28.237385 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-624134b1-caa3-4102-a2c6-1533ef0099d3 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] 10.222.0.16 "GET /compute/v2.1/servers/9df14d8c-4fff-439d-af43-10a008a7b9d2" status: 200 len: 1497 microversion: 2.1 time: 0.159830 Jul 16 21:41:28.237943 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2745/5487] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:41:28 2022] GET /compute/v2.1/servers/9df14d8c-4fff-439d-af43-10a008a7b9d2 => generated 1497 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:41:28.926413 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-557f65ca-315d-4a4c-94ff-26da6c7bc4b5 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:41:28.927672 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-557f65ca-315d-4a4c-94ff-26da6c7bc4b5 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] [instance: 9df14d8c-4fff-439d-af43-10a008a7b9d2] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:41:28.934809 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-557f65ca-315d-4a4c-94ff-26da6c7bc4b5 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:28.935310 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-557f65ca-315d-4a4c-94ff-26da6c7bc4b5 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:28.936076 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-557f65ca-315d-4a4c-94ff-26da6c7bc4b5 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:28.936517 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-557f65ca-315d-4a4c-94ff-26da6c7bc4b5 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:28.965757 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-557f65ca-315d-4a4c-94ff-26da6c7bc4b5 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] [instance: 9df14d8c-4fff-439d-af43-10a008a7b9d2] Going to try to terminate instance {{(pid=138680) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 16 21:41:29.290730 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-557f65ca-315d-4a4c-94ff-26da6c7bc4b5 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] 10.222.0.16 "DELETE /compute/v2.1/servers/9df14d8c-4fff-439d-af43-10a008a7b9d2" status: 204 len: 0 microversion: 2.1 time: 0.367882 Jul 16 21:41:29.291409 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2743/5488] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:41:28 2022] DELETE /compute/v2.1/servers/9df14d8c-4fff-439d-af43-10a008a7b9d2 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 16 21:41:29.299740 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-5a3862c4-cb73-4520-9af5-08acb309c8f1 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:41:29.302402 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-5a3862c4-cb73-4520-9af5-08acb309c8f1 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] [instance: 9df14d8c-4fff-439d-af43-10a008a7b9d2] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:41:29.308408 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5a3862c4-cb73-4520-9af5-08acb309c8f1 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:29.308878 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5a3862c4-cb73-4520-9af5-08acb309c8f1 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:29.309681 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5a3862c4-cb73-4520-9af5-08acb309c8f1 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:29.310127 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5a3862c4-cb73-4520-9af5-08acb309c8f1 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:29.445267 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-5a3862c4-cb73-4520-9af5-08acb309c8f1 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f7948b58ee714cb1bfd4d8364dae19ee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd61da83cb56412a8757633c22561495', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:41:29.453009 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-5a3862c4-cb73-4520-9af5-08acb309c8f1 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] 10.222.0.16 "GET /compute/v2.1/servers/9df14d8c-4fff-439d-af43-10a008a7b9d2" status: 200 len: 1503 microversion: 2.1 time: 0.157315 Jul 16 21:41:29.453558 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2746/5489] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:41:29 2022] GET /compute/v2.1/servers/9df14d8c-4fff-439d-af43-10a008a7b9d2 => generated 1503 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:41:30.470357 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-c1761ac8-5450-48a1-b366-8e73d9061501 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:41:30.471111 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-c1761ac8-5450-48a1-b366-8e73d9061501 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] [instance: 9df14d8c-4fff-439d-af43-10a008a7b9d2] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:41:30.478805 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c1761ac8-5450-48a1-b366-8e73d9061501 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:30.479302 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c1761ac8-5450-48a1-b366-8e73d9061501 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:30.480103 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c1761ac8-5450-48a1-b366-8e73d9061501 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:30.480551 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c1761ac8-5450-48a1-b366-8e73d9061501 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:30.624798 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-c1761ac8-5450-48a1-b366-8e73d9061501 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f7948b58ee714cb1bfd4d8364dae19ee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd61da83cb56412a8757633c22561495', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:41:30.633008 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-c1761ac8-5450-48a1-b366-8e73d9061501 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] 10.222.0.16 "GET /compute/v2.1/servers/9df14d8c-4fff-439d-af43-10a008a7b9d2" status: 200 len: 1503 microversion: 2.1 time: 0.166893 Jul 16 21:41:30.633576 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2744/5490] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:41:30 2022] GET /compute/v2.1/servers/9df14d8c-4fff-439d-af43-10a008a7b9d2 => generated 1503 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:41:31.263440 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-1251de6b-40f5-441b-a93f-11e4edcb8828 req-af39e74b-c611-48c6-9112-021d3b4d1f98 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "9df14d8c-4fff-439d-af43-10a008a7b9d2", "name": "network-vif-deleted", "tag": "6f102b18-5b0b-4710-90a7-2c2c4d79d1fd"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:41:31.273118 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-1251de6b-40f5-441b-a93f-11e4edcb8828 req-af39e74b-c611-48c6-9112-021d3b4d1f98 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:31.273579 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-1251de6b-40f5-441b-a93f-11e4edcb8828 req-af39e74b-c611-48c6-9112-021d3b4d1f98 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:31.284454 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-1251de6b-40f5-441b-a93f-11e4edcb8828 req-af39e74b-c611-48c6-9112-021d3b4d1f98 service nova] Creating event network-vif-deleted:6f102b18-5b0b-4710-90a7-2c2c4d79d1fd for instance 9df14d8c-4fff-439d-af43-10a008a7b9d2 on nch2-master-1 Jul 16 21:41:31.290093 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-1251de6b-40f5-441b-a93f-11e4edcb8828 req-af39e74b-c611-48c6-9112-021d3b4d1f98 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.030293 Jul 16 21:41:31.293114 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2747/5491] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:41:31 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:41:31.624229 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-5179b36c-92d2-4a8a-9ce7-d49b2b5c1b95 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Action: 'action', calling method: >, body: {"os-getConsoleOutput": {}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:41:31.625212 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-5179b36c-92d2-4a8a-9ce7-d49b2b5c1b95 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] [instance: 35a74168-3c95-4bfe-b82a-e93b10ec5820] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:41:31.633139 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5179b36c-92d2-4a8a-9ce7-d49b2b5c1b95 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:31.633605 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5179b36c-92d2-4a8a-9ce7-d49b2b5c1b95 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:31.634385 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5179b36c-92d2-4a8a-9ce7-d49b2b5c1b95 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:31.634839 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5179b36c-92d2-4a8a-9ce7-d49b2b5c1b95 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:31.651128 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-ed6f627c-c660-410a-a7c6-028a6cd6308b tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:41:31.651876 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-ed6f627c-c660-410a-a7c6-028a6cd6308b tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] [instance: 9df14d8c-4fff-439d-af43-10a008a7b9d2] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:41:31.660106 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ed6f627c-c660-410a-a7c6-028a6cd6308b tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:31.660593 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ed6f627c-c660-410a-a7c6-028a6cd6308b tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:31.661387 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ed6f627c-c660-410a-a7c6-028a6cd6308b tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:31.662754 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ed6f627c-c660-410a-a7c6-028a6cd6308b tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:31.691673 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-5179b36c-92d2-4a8a-9ce7-d49b2b5c1b95 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] 10.222.0.16 "POST /compute/v2.1/servers/35a74168-3c95-4bfe-b82a-e93b10ec5820/action" status: 200 len: 24572 microversion: 2.1 time: 0.071790 Jul 16 21:41:31.692769 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2745/5492] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 21:41:31 2022] POST /compute/v2.1/servers/35a74168-3c95-4bfe-b82a-e93b10ec5820/action => generated 24572 bytes in 74 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 16 21:41:31.696995 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ed6f627c-c660-410a-a7c6-028a6cd6308b tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:31.697458 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ed6f627c-c660-410a-a7c6-028a6cd6308b tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:31.697918 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.objects.instance [None req-ed6f627c-c660-410a-a7c6-028a6cd6308b tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Lazy-loading 'fault' on Instance uuid 9df14d8c-4fff-439d-af43-10a008a7b9d2 {{(pid=138681) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:41:31.741241 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-ed6f627c-c660-410a-a7c6-028a6cd6308b tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f7948b58ee714cb1bfd4d8364dae19ee', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bd61da83cb56412a8757633c22561495', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:41:31.748900 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-ed6f627c-c660-410a-a7c6-028a6cd6308b tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] 10.222.0.16 "GET /compute/v2.1/servers/9df14d8c-4fff-439d-af43-10a008a7b9d2" status: 200 len: 1303 microversion: 2.1 time: 0.102959 Jul 16 21:41:31.749616 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2748/5493] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:41:31 2022] GET /compute/v2.1/servers/9df14d8c-4fff-439d-af43-10a008a7b9d2 => generated 1303 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:41:32.766304 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-884718e8-73c8-4ff4-908d-9de0543b8c87 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:41:32.767043 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-884718e8-73c8-4ff4-908d-9de0543b8c87 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] [instance: 9df14d8c-4fff-439d-af43-10a008a7b9d2] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:41:32.775320 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-884718e8-73c8-4ff4-908d-9de0543b8c87 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:32.775792 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-884718e8-73c8-4ff4-908d-9de0543b8c87 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:32.776565 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-884718e8-73c8-4ff4-908d-9de0543b8c87 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:32.777017 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-884718e8-73c8-4ff4-908d-9de0543b8c87 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:32.792503 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-884718e8-73c8-4ff4-908d-9de0543b8c87 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] HTTP exception thrown: Instance 9df14d8c-4fff-439d-af43-10a008a7b9d2 could not be found. Jul 16 21:41:32.794187 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-884718e8-73c8-4ff4-908d-9de0543b8c87 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Request method failure captured: Jul 16 21:41:32.794187 ncdv-master-1 devstack@n-api.service[138680]: request: GET /compute/v2.1/servers/9df14d8c-4fff-439d-af43-10a008a7b9d2 HTTP/1.1 Jul 16 21:41:32.794187 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:41:32.794187 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:41:32.794187 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:41:32.794187 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 0 Jul 16 21:41:32.794187 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:41:32.794187 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:41:32.794187 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:41:32.794187 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:41:32.794187 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0zBfzfsTz8nvjZ__aZKaS0OcxWaPhZotqFsyKstg35JuhL91Shc9WtTQxsRGXZcOZ3NevAgJv8vz5SU1B8bJsS81uq76QIGUkypmqIdZ-qCEEwqVe6H4UMReHiO-YHkHqEv4WBykzbJblB3Vyvt9xKyUUsi9Fc7qtZBaxcUfd5Hu5qI Jul 16 21:41:32.794187 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:41:32.794187 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:41:32.794187 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:41:32.794187 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:41:32.794187 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:41:32.794187 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:41:32.794187 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: bd61da83cb56412a8757633c22561495 Jul 16 21:41:32.794187 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-VolumesActionsTest-484965280 Jul 16 21:41:32.794187 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:41:32.794187 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:41:32.794187 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/bd61da83cb56412a8757633c22561495"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/bd61da83cb56412a8757633c22561495"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_bd61da83cb56412a8757633c22561495"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/bd61da83cb56412a8757633c22561495"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:41:32.794187 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-VolumesActionsTest-484965280 Jul 16 21:41:32.794187 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: bd61da83cb56412a8757633c22561495 Jul 16 21:41:32.794187 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-VolumesActionsTest-484965280 Jul 16 21:41:32.794187 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-VolumesActionsTest-484965280-project Jul 16 21:41:32.794187 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:41:32.794187 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:41:32.794187 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: f7948b58ee714cb1bfd4d8364dae19ee Jul 16 21:41:32.794187 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-VolumesActionsTest-484965280-project Jul 16 21:41:32.794187 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 21:41:32.794187 ncdv-master-1 devstack@n-api.service[138680]: exception: Instance 9df14d8c-4fff-439d-af43-10a008a7b9d2 could not be found. Jul 16 21:41:32.794187 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': '9df14d8c-4fff-439d-af43-10a008a7b9d2'} Jul 16 21:41:32.794187 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:41:32.794187 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:41:32.794187 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:41:32.794187 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:41:32.794187 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:41:32.794187 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:41:32.794187 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:41:32.794187 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:41:32.794187 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:41:32.794187 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:41:32.794187 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 9df14d8c-4fff-439d-af43-10a008a7b9d2 could not be found. Jul 16 21:41:32.794187 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:41:32.794187 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:41:32.794187 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:41:32.794187 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:41:32.794187 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:41:32.794187 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:41:32.794187 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:41:32.794187 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:41:32.794187 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:41:32.794187 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:41:32.794187 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:41:32.794187 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:41:32.794187 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:41:32.794187 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:41:32.794187 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:41:32.794187 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:41:32.794187 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 9df14d8c-4fff-439d-af43-10a008a7b9d2 could not be found. Jul 16 21:41:32.794187 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:41:32.794187 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:41:32.794187 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:41:32.802808 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:41:32.802808 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:41:32.802808 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:41:32.802808 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:41:32.802808 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:41:32.802808 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 9df14d8c-4fff-439d-af43-10a008a7b9d2 could not be found. Jul 16 21:41:32.802808 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:41:32.802808 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-884718e8-73c8-4ff4-908d-9de0543b8c87 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] Returning 404 to user: Instance 9df14d8c-4fff-439d-af43-10a008a7b9d2 could not be found. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:41:32.802808 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-884718e8-73c8-4ff4-908d-9de0543b8c87 tempest-VolumesActionsTest-484965280 tempest-VolumesActionsTest-484965280-project] 10.222.0.16 "GET /compute/v2.1/servers/9df14d8c-4fff-439d-af43-10a008a7b9d2" status: 404 len: 111 microversion: 2.1 time: 0.033693 Jul 16 21:41:32.802808 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2746/5494] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:41:32 2022] GET /compute/v2.1/servers/9df14d8c-4fff-439d-af43-10a008a7b9d2 => generated 111 bytes in 35 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:41:33.944541 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-314efecb-9dc7-482d-be29-d1905de66124 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:41:33.945078 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-314efecb-9dc7-482d-be29-d1905de66124 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] [instance: 1ff78357-ba91-45ba-937e-cf12461b0610] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:41:33.953515 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-314efecb-9dc7-482d-be29-d1905de66124 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:33.954058 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-314efecb-9dc7-482d-be29-d1905de66124 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:33.954860 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-314efecb-9dc7-482d-be29-d1905de66124 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:33.955363 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-314efecb-9dc7-482d-be29-d1905de66124 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:33.971154 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-314efecb-9dc7-482d-be29-d1905de66124 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] HTTP exception thrown: Instance 1ff78357-ba91-45ba-937e-cf12461b0610 could not be found. Jul 16 21:41:33.972882 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-314efecb-9dc7-482d-be29-d1905de66124 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Request method failure captured: Jul 16 21:41:33.972882 ncdv-master-1 devstack@n-api.service[138681]: request: DELETE /compute/v2.1/servers/1ff78357-ba91-45ba-937e-cf12461b0610 HTTP/1.1 Jul 16 21:41:33.972882 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:41:33.972882 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:41:33.972882 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:41:33.972882 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 21:41:33.972882 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:41:33.972882 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:41:33.972882 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:41:33.972882 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:41:33.972882 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0y_WdH7eI0DbXzeyrxNR2zaPLU2Z0T2hgcgJaNL5zKj0IOkua_mwRXGhv8UgaywTvIB56iq06rxK294Knh0aCrbToiJY0kPn8yPuGVzddBFopXnN-hL_u7FbpY8zvcJRiL0zpWIKlPhQExzLv95jQK7KF6D9Y-KawL2vbM6jbQCfNnU Jul 16 21:41:33.972882 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:41:33.972882 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:41:33.972882 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:41:33.972882 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:41:33.972882 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:41:33.972882 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:41:33.972882 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: b533f85aeddf4d8db05f2c53a61befc3 Jul 16 21:41:33.972882 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-TestVolumeBootPattern-1625426921 Jul 16 21:41:33.972882 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:41:33.972882 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:41:33.972882 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/b533f85aeddf4d8db05f2c53a61befc3"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/b533f85aeddf4d8db05f2c53a61befc3"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_b533f85aeddf4d8db05f2c53a61befc3"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/b533f85aeddf4d8db05f2c53a61befc3"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:41:33.972882 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-TestVolumeBootPattern-1625426921 Jul 16 21:41:33.972882 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: b533f85aeddf4d8db05f2c53a61befc3 Jul 16 21:41:33.972882 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-TestVolumeBootPattern-1625426921 Jul 16 21:41:33.972882 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-TestVolumeBootPattern-1625426921-project Jul 16 21:41:33.972882 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:41:33.972882 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:41:33.972882 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: 62209b9802ab46c3b3d094d4d1ad86fe Jul 16 21:41:33.972882 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-TestVolumeBootPattern-1625426921-project Jul 16 21:41:33.972882 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:41:33.972882 ncdv-master-1 devstack@n-api.service[138681]: exception: Instance 1ff78357-ba91-45ba-937e-cf12461b0610 could not be found. Jul 16 21:41:33.972882 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': '1ff78357-ba91-45ba-937e-cf12461b0610'} Jul 16 21:41:33.972882 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:41:33.972882 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:41:33.972882 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:41:33.972882 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:41:33.972882 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:41:33.972882 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:41:33.972882 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:41:33.972882 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:41:33.972882 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:41:33.972882 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:41:33.972882 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 1ff78357-ba91-45ba-937e-cf12461b0610 could not be found. Jul 16 21:41:33.972882 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:41:33.972882 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:41:33.972882 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:41:33.972882 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:41:33.972882 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:41:33.972882 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:41:33.972882 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:41:33.972882 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:41:33.972882 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:41:33.972882 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:41:33.972882 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Jul 16 21:41:33.972882 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Jul 16 21:41:33.972882 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Jul 16 21:41:33.972882 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Jul 16 21:41:33.972882 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:41:33.972882 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:41:33.972882 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:41:33.972882 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:41:33.982694 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 1ff78357-ba91-45ba-937e-cf12461b0610 could not be found. Jul 16 21:41:33.982694 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:41:33.982694 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:41:33.982694 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:41:33.982694 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:41:33.982694 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:41:33.982694 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:41:33.982694 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:41:33.982694 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:41:33.982694 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 1ff78357-ba91-45ba-937e-cf12461b0610 could not be found. Jul 16 21:41:33.982694 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:41:33.982694 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-314efecb-9dc7-482d-be29-d1905de66124 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Returning 404 to user: Instance 1ff78357-ba91-45ba-937e-cf12461b0610 could not be found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:41:33.982694 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-314efecb-9dc7-482d-be29-d1905de66124 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] 10.222.0.16 "DELETE /compute/v2.1/servers/1ff78357-ba91-45ba-937e-cf12461b0610" status: 404 len: 111 microversion: 2.1 time: 0.034638 Jul 16 21:41:33.982694 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2749/5495] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:41:33 2022] DELETE /compute/v2.1/servers/1ff78357-ba91-45ba-937e-cf12461b0610 => generated 111 bytes in 36 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:41:33.984533 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-e10e772a-f16c-4d56-b8bd-edbe43b66f2a tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:41:33.985158 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-e10e772a-f16c-4d56-b8bd-edbe43b66f2a tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] [instance: 1ff78357-ba91-45ba-937e-cf12461b0610] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:41:33.993747 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e10e772a-f16c-4d56-b8bd-edbe43b66f2a tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:33.994278 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e10e772a-f16c-4d56-b8bd-edbe43b66f2a tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:33.997252 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e10e772a-f16c-4d56-b8bd-edbe43b66f2a tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:33.997252 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e10e772a-f16c-4d56-b8bd-edbe43b66f2a tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:34.011816 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-e10e772a-f16c-4d56-b8bd-edbe43b66f2a tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] HTTP exception thrown: Instance 1ff78357-ba91-45ba-937e-cf12461b0610 could not be found. Jul 16 21:41:34.013471 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-e10e772a-f16c-4d56-b8bd-edbe43b66f2a tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Request method failure captured: Jul 16 21:41:34.013471 ncdv-master-1 devstack@n-api.service[138680]: request: GET /compute/v2.1/servers/1ff78357-ba91-45ba-937e-cf12461b0610 HTTP/1.1 Jul 16 21:41:34.013471 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:41:34.013471 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:41:34.013471 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:41:34.013471 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 0 Jul 16 21:41:34.013471 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:41:34.013471 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:41:34.013471 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:41:34.013471 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:41:34.013471 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0y_WdH7eI0DbXzeyrxNR2zaPLU2Z0T2hgcgJaNL5zKj0IOkua_mwRXGhv8UgaywTvIB56iq06rxK294Knh0aCrbToiJY0kPn8yPuGVzddBFopXnN-hL_u7FbpY8zvcJRiL0zpWIKlPhQExzLv95jQK7KF6D9Y-KawL2vbM6jbQCfNnU Jul 16 21:41:34.013471 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:41:34.013471 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:41:34.013471 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:41:34.013471 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:41:34.013471 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:41:34.013471 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:41:34.013471 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: b533f85aeddf4d8db05f2c53a61befc3 Jul 16 21:41:34.013471 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-TestVolumeBootPattern-1625426921 Jul 16 21:41:34.013471 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:41:34.013471 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:41:34.013471 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/b533f85aeddf4d8db05f2c53a61befc3"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/b533f85aeddf4d8db05f2c53a61befc3"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_b533f85aeddf4d8db05f2c53a61befc3"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/b533f85aeddf4d8db05f2c53a61befc3"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:41:34.013471 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-TestVolumeBootPattern-1625426921 Jul 16 21:41:34.013471 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: b533f85aeddf4d8db05f2c53a61befc3 Jul 16 21:41:34.013471 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-TestVolumeBootPattern-1625426921 Jul 16 21:41:34.013471 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-TestVolumeBootPattern-1625426921-project Jul 16 21:41:34.013471 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:41:34.013471 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:41:34.013471 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: 62209b9802ab46c3b3d094d4d1ad86fe Jul 16 21:41:34.013471 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-TestVolumeBootPattern-1625426921-project Jul 16 21:41:34.013471 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 21:41:34.013471 ncdv-master-1 devstack@n-api.service[138680]: exception: Instance 1ff78357-ba91-45ba-937e-cf12461b0610 could not be found. Jul 16 21:41:34.013471 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': '1ff78357-ba91-45ba-937e-cf12461b0610'} Jul 16 21:41:34.013471 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:41:34.013471 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:41:34.013471 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:41:34.013471 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:41:34.013471 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:41:34.013471 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:41:34.013471 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:41:34.013471 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:41:34.013471 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:41:34.013471 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:41:34.013471 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 1ff78357-ba91-45ba-937e-cf12461b0610 could not be found. Jul 16 21:41:34.013471 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:41:34.013471 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:41:34.013471 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:41:34.013471 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:41:34.013471 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:41:34.013471 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:41:34.013471 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:41:34.013471 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:41:34.013471 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:41:34.013471 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:41:34.013471 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:41:34.013471 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:41:34.013471 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:41:34.013471 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:41:34.013471 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:41:34.013471 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:41:34.013471 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 1ff78357-ba91-45ba-937e-cf12461b0610 could not be found. Jul 16 21:41:34.013471 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:41:34.013471 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:41:34.013471 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:41:34.022899 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:41:34.022899 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:41:34.022899 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:41:34.022899 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:41:34.022899 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:41:34.022899 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 1ff78357-ba91-45ba-937e-cf12461b0610 could not be found. Jul 16 21:41:34.022899 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:41:34.022899 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-e10e772a-f16c-4d56-b8bd-edbe43b66f2a tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Returning 404 to user: Instance 1ff78357-ba91-45ba-937e-cf12461b0610 could not be found. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:41:34.022899 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-e10e772a-f16c-4d56-b8bd-edbe43b66f2a tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] 10.222.0.16 "GET /compute/v2.1/servers/1ff78357-ba91-45ba-937e-cf12461b0610" status: 404 len: 111 microversion: 2.1 time: 0.036695 Jul 16 21:41:34.022899 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2747/5496] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:41:33 2022] GET /compute/v2.1/servers/1ff78357-ba91-45ba-937e-cf12461b0610 => generated 111 bytes in 38 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:41:35.094733 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-03462150-8d04-4e55-86ea-668a98842740 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:41:35.094733 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-03462150-8d04-4e55-86ea-668a98842740 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] [instance: 997f89c5-8086-4a7e-8292-1c8e6a12aaf6] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:41:35.101219 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-03462150-8d04-4e55-86ea-668a98842740 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:35.101690 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-03462150-8d04-4e55-86ea-668a98842740 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:35.102528 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-03462150-8d04-4e55-86ea-668a98842740 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:35.102987 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-03462150-8d04-4e55-86ea-668a98842740 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:35.123237 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-03462150-8d04-4e55-86ea-668a98842740 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] HTTP exception thrown: Instance 997f89c5-8086-4a7e-8292-1c8e6a12aaf6 could not be found. Jul 16 21:41:35.125022 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-03462150-8d04-4e55-86ea-668a98842740 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Request method failure captured: Jul 16 21:41:35.125022 ncdv-master-1 devstack@n-api.service[138681]: request: DELETE /compute/v2.1/servers/997f89c5-8086-4a7e-8292-1c8e6a12aaf6 HTTP/1.1 Jul 16 21:41:35.125022 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:41:35.125022 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:41:35.125022 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:41:35.125022 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 21:41:35.125022 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:41:35.125022 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:41:35.125022 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:41:35.125022 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:41:35.125022 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0y_WdH7eI0DbXzeyrxNR2zaPLU2Z0T2hgcgJaNL5zKj0IOkua_mwRXGhv8UgaywTvIB56iq06rxK294Knh0aCrbToiJY0kPn8yPuGVzddBFopXnN-hL_u7FbpY8zvcJRiL0zpWIKlPhQExzLv95jQK7KF6D9Y-KawL2vbM6jbQCfNnU Jul 16 21:41:35.125022 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:41:35.125022 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:41:35.125022 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:41:35.125022 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:41:35.125022 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:41:35.125022 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:41:35.125022 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: b533f85aeddf4d8db05f2c53a61befc3 Jul 16 21:41:35.125022 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-TestVolumeBootPattern-1625426921 Jul 16 21:41:35.125022 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:41:35.125022 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:41:35.125022 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/b533f85aeddf4d8db05f2c53a61befc3"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/b533f85aeddf4d8db05f2c53a61befc3"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_b533f85aeddf4d8db05f2c53a61befc3"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/b533f85aeddf4d8db05f2c53a61befc3"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:41:35.125022 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-TestVolumeBootPattern-1625426921 Jul 16 21:41:35.125022 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: b533f85aeddf4d8db05f2c53a61befc3 Jul 16 21:41:35.125022 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-TestVolumeBootPattern-1625426921 Jul 16 21:41:35.125022 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-TestVolumeBootPattern-1625426921-project Jul 16 21:41:35.125022 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:41:35.125022 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:41:35.125022 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: 62209b9802ab46c3b3d094d4d1ad86fe Jul 16 21:41:35.125022 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-TestVolumeBootPattern-1625426921-project Jul 16 21:41:35.125022 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:41:35.125022 ncdv-master-1 devstack@n-api.service[138681]: exception: Instance 997f89c5-8086-4a7e-8292-1c8e6a12aaf6 could not be found. Jul 16 21:41:35.125022 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': '997f89c5-8086-4a7e-8292-1c8e6a12aaf6'} Jul 16 21:41:35.125022 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:41:35.125022 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:41:35.125022 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:41:35.125022 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:41:35.125022 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:41:35.125022 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:41:35.125022 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:41:35.125022 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:41:35.125022 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:41:35.125022 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:41:35.125022 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 997f89c5-8086-4a7e-8292-1c8e6a12aaf6 could not be found. Jul 16 21:41:35.125022 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:41:35.125022 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:41:35.125022 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:41:35.125022 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:41:35.125022 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:41:35.125022 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:41:35.125022 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:41:35.125022 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:41:35.125022 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:41:35.125022 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:41:35.125022 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Jul 16 21:41:35.125022 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Jul 16 21:41:35.125022 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Jul 16 21:41:35.125022 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Jul 16 21:41:35.125022 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:41:35.125022 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:41:35.125022 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:41:35.125022 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:41:35.129715 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 997f89c5-8086-4a7e-8292-1c8e6a12aaf6 could not be found. Jul 16 21:41:35.129715 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:41:35.129715 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:41:35.129715 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:41:35.129715 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:41:35.129715 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:41:35.129715 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:41:35.129715 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:41:35.129715 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:41:35.129715 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 997f89c5-8086-4a7e-8292-1c8e6a12aaf6 could not be found. Jul 16 21:41:35.129715 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:41:35.129715 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-03462150-8d04-4e55-86ea-668a98842740 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Returning 404 to user: Instance 997f89c5-8086-4a7e-8292-1c8e6a12aaf6 could not be found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:41:35.129715 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-03462150-8d04-4e55-86ea-668a98842740 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] 10.222.0.16 "DELETE /compute/v2.1/servers/997f89c5-8086-4a7e-8292-1c8e6a12aaf6" status: 404 len: 111 microversion: 2.1 time: 0.039651 Jul 16 21:41:35.129715 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2750/5497] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:41:35 2022] DELETE /compute/v2.1/servers/997f89c5-8086-4a7e-8292-1c8e6a12aaf6 => generated 111 bytes in 41 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:41:35.137469 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-5ef54a09-3b01-4d96-8edc-c00876813fa6 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:41:35.138170 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-5ef54a09-3b01-4d96-8edc-c00876813fa6 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] [instance: 997f89c5-8086-4a7e-8292-1c8e6a12aaf6] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:41:35.149183 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5ef54a09-3b01-4d96-8edc-c00876813fa6 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:35.150105 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5ef54a09-3b01-4d96-8edc-c00876813fa6 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:35.151399 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5ef54a09-3b01-4d96-8edc-c00876813fa6 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:35.152327 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5ef54a09-3b01-4d96-8edc-c00876813fa6 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:35.173662 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-5ef54a09-3b01-4d96-8edc-c00876813fa6 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] HTTP exception thrown: Instance 997f89c5-8086-4a7e-8292-1c8e6a12aaf6 could not be found. Jul 16 21:41:35.175365 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-5ef54a09-3b01-4d96-8edc-c00876813fa6 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Request method failure captured: Jul 16 21:41:35.175365 ncdv-master-1 devstack@n-api.service[138680]: request: GET /compute/v2.1/servers/997f89c5-8086-4a7e-8292-1c8e6a12aaf6 HTTP/1.1 Jul 16 21:41:35.175365 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:41:35.175365 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:41:35.175365 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:41:35.175365 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 0 Jul 16 21:41:35.175365 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:41:35.175365 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:41:35.175365 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:41:35.175365 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:41:35.175365 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0y_WdH7eI0DbXzeyrxNR2zaPLU2Z0T2hgcgJaNL5zKj0IOkua_mwRXGhv8UgaywTvIB56iq06rxK294Knh0aCrbToiJY0kPn8yPuGVzddBFopXnN-hL_u7FbpY8zvcJRiL0zpWIKlPhQExzLv95jQK7KF6D9Y-KawL2vbM6jbQCfNnU Jul 16 21:41:35.175365 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:41:35.175365 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:41:35.175365 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:41:35.175365 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:41:35.175365 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:41:35.175365 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:41:35.175365 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: b533f85aeddf4d8db05f2c53a61befc3 Jul 16 21:41:35.175365 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-TestVolumeBootPattern-1625426921 Jul 16 21:41:35.175365 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:41:35.175365 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:41:35.175365 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/b533f85aeddf4d8db05f2c53a61befc3"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/b533f85aeddf4d8db05f2c53a61befc3"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_b533f85aeddf4d8db05f2c53a61befc3"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/b533f85aeddf4d8db05f2c53a61befc3"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:41:35.175365 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-TestVolumeBootPattern-1625426921 Jul 16 21:41:35.175365 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: b533f85aeddf4d8db05f2c53a61befc3 Jul 16 21:41:35.175365 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-TestVolumeBootPattern-1625426921 Jul 16 21:41:35.175365 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-TestVolumeBootPattern-1625426921-project Jul 16 21:41:35.175365 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:41:35.175365 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:41:35.175365 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: 62209b9802ab46c3b3d094d4d1ad86fe Jul 16 21:41:35.175365 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-TestVolumeBootPattern-1625426921-project Jul 16 21:41:35.175365 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 21:41:35.175365 ncdv-master-1 devstack@n-api.service[138680]: exception: Instance 997f89c5-8086-4a7e-8292-1c8e6a12aaf6 could not be found. Jul 16 21:41:35.175365 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': '997f89c5-8086-4a7e-8292-1c8e6a12aaf6'} Jul 16 21:41:35.175365 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:41:35.175365 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:41:35.175365 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:41:35.175365 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:41:35.175365 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:41:35.175365 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:41:35.175365 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:41:35.175365 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:41:35.175365 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:41:35.175365 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:41:35.175365 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 997f89c5-8086-4a7e-8292-1c8e6a12aaf6 could not be found. Jul 16 21:41:35.175365 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:41:35.175365 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:41:35.175365 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:41:35.175365 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:41:35.175365 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:41:35.175365 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:41:35.175365 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:41:35.175365 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:41:35.175365 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:41:35.175365 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:41:35.175365 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:41:35.175365 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:41:35.175365 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:41:35.175365 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:41:35.175365 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:41:35.175365 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:41:35.175365 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 997f89c5-8086-4a7e-8292-1c8e6a12aaf6 could not be found. Jul 16 21:41:35.175365 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:41:35.175365 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:41:35.175365 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:41:35.180279 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:41:35.180279 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:41:35.180279 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:41:35.180279 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:41:35.180279 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:41:35.180279 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 997f89c5-8086-4a7e-8292-1c8e6a12aaf6 could not be found. Jul 16 21:41:35.180279 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:41:35.182324 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-5ef54a09-3b01-4d96-8edc-c00876813fa6 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Returning 404 to user: Instance 997f89c5-8086-4a7e-8292-1c8e6a12aaf6 could not be found. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:41:35.183424 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-5ef54a09-3b01-4d96-8edc-c00876813fa6 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] 10.222.0.16 "GET /compute/v2.1/servers/997f89c5-8086-4a7e-8292-1c8e6a12aaf6" status: 404 len: 111 microversion: 2.1 time: 0.050906 Jul 16 21:41:35.184627 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2748/5498] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:41:35 2022] GET /compute/v2.1/servers/997f89c5-8086-4a7e-8292-1c8e6a12aaf6 => generated 111 bytes in 53 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:41:35.284827 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-968760be-ebb4-41c7-a88c-18dacbd8d0be tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Action: 'create', calling method: .version_select at 0x7fdda2b9caf0>, body: {"keypair": {"name": "tempest-TestVolumeBootPattern-1632947712", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBD4R+/86N4zyN92wPmheodgbKIM4VKeK7s4YXlruUSBZ7oPOkszFcBMpPIebLHAPCQF/90RL0mQqI/oE6l/ty95ZJnanBrJb0pMN6ji2Hnq1yMNdBPgle4s20Xyp9HFajQ=="}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:41:35.308223 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-968760be-ebb4-41c7-a88c-18dacbd8d0be tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Getting quotas for project b533f85aeddf4d8db05f2c53a61befc3. Resources: {'key_pairs'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:41:35.315226 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-968760be-ebb4-41c7-a88c-18dacbd8d0be tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Getting quotas for user 62209b9802ab46c3b3d094d4d1ad86fe and project b533f85aeddf4d8db05f2c53a61befc3. Resources: {'key_pairs'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:41:35.428718 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-968760be-ebb4-41c7-a88c-18dacbd8d0be tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] 10.222.0.16 "POST /compute/v2.1/os-keypairs" status: 200 len: 400 microversion: 2.1 time: 0.148226 Jul 16 21:41:35.430400 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2751/5499] 10.222.0.16 () {60 vars in 1158 bytes} [Sat Jul 16 21:41:35 2022] POST /compute/v2.1/os-keypairs => generated 400 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:41:38.142947 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-e21861c6-32aa-4d9f-9d04-241cb88a7c32 tempest-TestNetworkAdvancedServerOps-756251536 tempest-TestNetworkAdvancedServerOps-756251536-project-admin] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:41:38.143590 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-e21861c6-32aa-4d9f-9d04-241cb88a7c32 tempest-TestNetworkAdvancedServerOps-756251536 tempest-TestNetworkAdvancedServerOps-756251536-project-admin] [instance: 35a74168-3c95-4bfe-b82a-e93b10ec5820] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:41:38.151511 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e21861c6-32aa-4d9f-9d04-241cb88a7c32 tempest-TestNetworkAdvancedServerOps-756251536 tempest-TestNetworkAdvancedServerOps-756251536-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:38.155964 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e21861c6-32aa-4d9f-9d04-241cb88a7c32 tempest-TestNetworkAdvancedServerOps-756251536 tempest-TestNetworkAdvancedServerOps-756251536-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:38.157113 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e21861c6-32aa-4d9f-9d04-241cb88a7c32 tempest-TestNetworkAdvancedServerOps-756251536 tempest-TestNetworkAdvancedServerOps-756251536-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:38.157957 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e21861c6-32aa-4d9f-9d04-241cb88a7c32 tempest-TestNetworkAdvancedServerOps-756251536 tempest-TestNetworkAdvancedServerOps-756251536-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:38.314122 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-e21861c6-32aa-4d9f-9d04-241cb88a7c32 tempest-TestNetworkAdvancedServerOps-756251536 tempest-TestNetworkAdvancedServerOps-756251536-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/35a74168-3c95-4bfe-b82a-e93b10ec5820" status: 200 len: 1835 microversion: 2.1 time: 0.174606 Jul 16 21:41:38.324238 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2749/5500] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:41:38 2022] GET /compute/v2.1/servers/35a74168-3c95-4bfe-b82a-e93b10ec5820 => generated 1835 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:41:38.331995 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-b7d51053-e7a9-4617-85cb-16f15eb91c3c tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:41:38.332619 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-b7d51053-e7a9-4617-85cb-16f15eb91c3c tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] [instance: 35a74168-3c95-4bfe-b82a-e93b10ec5820] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:41:38.356632 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b7d51053-e7a9-4617-85cb-16f15eb91c3c tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:38.356632 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b7d51053-e7a9-4617-85cb-16f15eb91c3c tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:38.356632 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b7d51053-e7a9-4617-85cb-16f15eb91c3c tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:38.356632 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b7d51053-e7a9-4617-85cb-16f15eb91c3c tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:38.526123 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-b7d51053-e7a9-4617-85cb-16f15eb91c3c tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '833136cbdeab4b9e8b6dd2d24c7c8753', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '77c9b7e351e24dc88207e9498f75f578', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:41:38.543346 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-b7d51053-e7a9-4617-85cb-16f15eb91c3c tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] 10.222.0.16 "GET /compute/v2.1/servers/35a74168-3c95-4bfe-b82a-e93b10ec5820" status: 200 len: 1684 microversion: 2.1 time: 0.213178 Jul 16 21:41:38.551744 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2752/5501] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:41:38 2022] GET /compute/v2.1/servers/35a74168-3c95-4bfe-b82a-e93b10ec5820 => generated 1684 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:41:38.661286 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-33557291-4109-4193-82aa-045d49b3e15e tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Action: 'action', calling method: >, body: {"os-getConsoleOutput": {}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:41:38.661286 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-33557291-4109-4193-82aa-045d49b3e15e tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] [instance: 35a74168-3c95-4bfe-b82a-e93b10ec5820] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:41:38.675694 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-33557291-4109-4193-82aa-045d49b3e15e tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:38.681258 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-33557291-4109-4193-82aa-045d49b3e15e tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:38.681258 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-33557291-4109-4193-82aa-045d49b3e15e tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:38.681258 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-33557291-4109-4193-82aa-045d49b3e15e tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:38.763925 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-33557291-4109-4193-82aa-045d49b3e15e tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] 10.222.0.16 "POST /compute/v2.1/servers/35a74168-3c95-4bfe-b82a-e93b10ec5820/action" status: 200 len: 32639 microversion: 2.1 time: 0.108883 Jul 16 21:41:38.766604 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2750/5502] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 21:41:38 2022] POST /compute/v2.1/servers/35a74168-3c95-4bfe-b82a-e93b10ec5820/action => generated 32639 bytes in 112 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 16 21:41:39.008146 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-c4fbcc5a-0d53-4509-810c-64ea09936ebb tempest-TestNetworkAdvancedServerOps-756251536 tempest-TestNetworkAdvancedServerOps-756251536-project-admin] Action: 'action', calling method: >, body: {"migrate": {}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:41:39.099925 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-c4fbcc5a-0d53-4509-810c-64ea09936ebb tempest-TestNetworkAdvancedServerOps-756251536 tempest-TestNetworkAdvancedServerOps-756251536-project-admin] [instance: 35a74168-3c95-4bfe-b82a-e93b10ec5820] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:41:39.099925 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c4fbcc5a-0d53-4509-810c-64ea09936ebb tempest-TestNetworkAdvancedServerOps-756251536 tempest-TestNetworkAdvancedServerOps-756251536-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:39.099925 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c4fbcc5a-0d53-4509-810c-64ea09936ebb tempest-TestNetworkAdvancedServerOps-756251536 tempest-TestNetworkAdvancedServerOps-756251536-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:39.099925 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c4fbcc5a-0d53-4509-810c-64ea09936ebb tempest-TestNetworkAdvancedServerOps-756251536 tempest-TestNetworkAdvancedServerOps-756251536-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:39.099925 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c4fbcc5a-0d53-4509-810c-64ea09936ebb tempest-TestNetworkAdvancedServerOps-756251536 tempest-TestNetworkAdvancedServerOps-756251536-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:39.099925 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-c4fbcc5a-0d53-4509-810c-64ea09936ebb tempest-TestNetworkAdvancedServerOps-756251536 tempest-TestNetworkAdvancedServerOps-756251536-project-admin] Policy check for compute:servers:resize:cross_cell failed with credentials {'is_admin': True, 'user_id': '12b39cdbf0db4052b7197313efcc4d21', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8736916be07c4e6886ed5280d4cce420', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:41:39.099925 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-c4fbcc5a-0d53-4509-810c-64ea09936ebb tempest-TestNetworkAdvancedServerOps-756251536 tempest-TestNetworkAdvancedServerOps-756251536-project-admin] [instance: 35a74168-3c95-4bfe-b82a-e93b10ec5820] flavor_id is None. Assuming migration. {{(pid=138681) resize /opt/stack/nova/nova/compute/api.py:4142}} Jul 16 21:41:39.099925 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-c4fbcc5a-0d53-4509-810c-64ea09936ebb tempest-TestNetworkAdvancedServerOps-756251536 tempest-TestNetworkAdvancedServerOps-756251536-project-admin] [instance: 35a74168-3c95-4bfe-b82a-e93b10ec5820] Old instance type m1.heat2, new instance type m1.heat2 {{(pid=138681) resize /opt/stack/nova/nova/compute/api.py:4166}} Jul 16 21:41:39.311767 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-c4fbcc5a-0d53-4509-810c-64ea09936ebb tempest-TestNetworkAdvancedServerOps-756251536 tempest-TestNetworkAdvancedServerOps-756251536-project-admin] 10.222.0.16 "POST /compute/v2.1/servers/35a74168-3c95-4bfe-b82a-e93b10ec5820/action" status: 202 len: 0 microversion: 2.1 time: 0.308123 Jul 16 21:41:39.314854 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2753/5503] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 21:41:39 2022] POST /compute/v2.1/servers/35a74168-3c95-4bfe-b82a-e93b10ec5820/action => generated 0 bytes in 312 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 21:41:39.324112 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-f03e44db-338d-4c10-b32a-300be4707ecb tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:41:39.328038 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-f03e44db-338d-4c10-b32a-300be4707ecb tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] [instance: 35a74168-3c95-4bfe-b82a-e93b10ec5820] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:41:39.340660 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f03e44db-338d-4c10-b32a-300be4707ecb tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:39.341118 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f03e44db-338d-4c10-b32a-300be4707ecb tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:39.341915 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f03e44db-338d-4c10-b32a-300be4707ecb tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:39.342363 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f03e44db-338d-4c10-b32a-300be4707ecb tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:39.638373 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-f03e44db-338d-4c10-b32a-300be4707ecb tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '833136cbdeab4b9e8b6dd2d24c7c8753', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '77c9b7e351e24dc88207e9498f75f578', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:41:39.653305 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-f03e44db-338d-4c10-b32a-300be4707ecb tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] 10.222.0.16 "GET /compute/v2.1/servers/35a74168-3c95-4bfe-b82a-e93b10ec5820" status: 200 len: 1693 microversion: 2.1 time: 0.332256 Jul 16 21:41:39.656218 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2751/5504] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:41:39 2022] GET /compute/v2.1/servers/35a74168-3c95-4bfe-b82a-e93b10ec5820 => generated 1693 bytes in 338 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:41:40.680425 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-fe7b24f0-1472-4b98-a19a-e1575d62f999 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:41:40.680425 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-fe7b24f0-1472-4b98-a19a-e1575d62f999 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] [instance: 35a74168-3c95-4bfe-b82a-e93b10ec5820] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:41:40.682503 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fe7b24f0-1472-4b98-a19a-e1575d62f999 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:40.683026 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fe7b24f0-1472-4b98-a19a-e1575d62f999 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:40.683819 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fe7b24f0-1472-4b98-a19a-e1575d62f999 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:40.684271 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fe7b24f0-1472-4b98-a19a-e1575d62f999 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:40.822478 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-fe7b24f0-1472-4b98-a19a-e1575d62f999 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '833136cbdeab4b9e8b6dd2d24c7c8753', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '77c9b7e351e24dc88207e9498f75f578', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:41:40.830435 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-fe7b24f0-1472-4b98-a19a-e1575d62f999 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] 10.222.0.16 "GET /compute/v2.1/servers/35a74168-3c95-4bfe-b82a-e93b10ec5820" status: 200 len: 1693 microversion: 2.1 time: 0.160509 Jul 16 21:41:40.831043 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2754/5505] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:41:40 2022] GET /compute/v2.1/servers/35a74168-3c95-4bfe-b82a-e93b10ec5820 => generated 1693 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:41:41.848239 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-29ac37a0-2c48-4749-bda3-ca490c644d46 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:41:41.848880 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-29ac37a0-2c48-4749-bda3-ca490c644d46 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] [instance: 35a74168-3c95-4bfe-b82a-e93b10ec5820] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:41:41.852976 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_api:SELECT=69,nova_api:UPDATE=2,nova_api:INSERT=3 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:41:41.858248 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-29ac37a0-2c48-4749-bda3-ca490c644d46 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:41.858707 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-29ac37a0-2c48-4749-bda3-ca490c644d46 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:41.859513 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-29ac37a0-2c48-4749-bda3-ca490c644d46 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:41.859964 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-29ac37a0-2c48-4749-bda3-ca490c644d46 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:42.006760 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-29ac37a0-2c48-4749-bda3-ca490c644d46 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '833136cbdeab4b9e8b6dd2d24c7c8753', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '77c9b7e351e24dc88207e9498f75f578', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:41:42.014504 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-29ac37a0-2c48-4749-bda3-ca490c644d46 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] 10.222.0.16 "GET /compute/v2.1/servers/35a74168-3c95-4bfe-b82a-e93b10ec5820" status: 200 len: 1693 microversion: 2.1 time: 0.170209 Jul 16 21:41:42.015089 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2752/5506] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:41:41 2022] GET /compute/v2.1/servers/35a74168-3c95-4bfe-b82a-e93b10ec5820 => generated 1693 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:41:43.032991 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-516b316b-2e2c-4b7b-9530-e67a5f2261cd tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:41:43.034884 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-516b316b-2e2c-4b7b-9530-e67a5f2261cd tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] [instance: 35a74168-3c95-4bfe-b82a-e93b10ec5820] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:41:43.044231 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-516b316b-2e2c-4b7b-9530-e67a5f2261cd tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:43.049210 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-516b316b-2e2c-4b7b-9530-e67a5f2261cd tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:43.049210 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-516b316b-2e2c-4b7b-9530-e67a5f2261cd tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:43.049210 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-516b316b-2e2c-4b7b-9530-e67a5f2261cd tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:43.233850 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-516b316b-2e2c-4b7b-9530-e67a5f2261cd tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '833136cbdeab4b9e8b6dd2d24c7c8753', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '77c9b7e351e24dc88207e9498f75f578', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:41:43.243976 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-516b316b-2e2c-4b7b-9530-e67a5f2261cd tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] 10.222.0.16 "GET /compute/v2.1/servers/35a74168-3c95-4bfe-b82a-e93b10ec5820" status: 200 len: 1698 microversion: 2.1 time: 0.215738 Jul 16 21:41:43.244831 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2755/5507] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:41:43 2022] GET /compute/v2.1/servers/35a74168-3c95-4bfe-b82a-e93b10ec5820 => generated 1698 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:41:44.262847 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-d11ee972-d0ac-45f4-a34d-6cbb8309f764 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:41:44.263457 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-d11ee972-d0ac-45f4-a34d-6cbb8309f764 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] [instance: 35a74168-3c95-4bfe-b82a-e93b10ec5820] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:41:44.270659 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d11ee972-d0ac-45f4-a34d-6cbb8309f764 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:44.274469 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d11ee972-d0ac-45f4-a34d-6cbb8309f764 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:44.274469 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d11ee972-d0ac-45f4-a34d-6cbb8309f764 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:44.274469 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d11ee972-d0ac-45f4-a34d-6cbb8309f764 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:44.465838 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-d11ee972-d0ac-45f4-a34d-6cbb8309f764 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '833136cbdeab4b9e8b6dd2d24c7c8753', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '77c9b7e351e24dc88207e9498f75f578', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:41:44.475320 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-d11ee972-d0ac-45f4-a34d-6cbb8309f764 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] 10.222.0.16 "GET /compute/v2.1/servers/35a74168-3c95-4bfe-b82a-e93b10ec5820" status: 200 len: 1698 microversion: 2.1 time: 0.216455 Jul 16 21:41:44.481571 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2753/5508] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:41:44 2022] GET /compute/v2.1/servers/35a74168-3c95-4bfe-b82a-e93b10ec5820 => generated 1698 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:41:45.497324 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-3830825e-2aa3-4d17-b4cd-823c9f2e1a19 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:41:45.499107 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-3830825e-2aa3-4d17-b4cd-823c9f2e1a19 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] [instance: 35a74168-3c95-4bfe-b82a-e93b10ec5820] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:41:45.505110 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell0:SELECT=4 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:41:45.510081 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3830825e-2aa3-4d17-b4cd-823c9f2e1a19 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:45.510733 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3830825e-2aa3-4d17-b4cd-823c9f2e1a19 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:45.511601 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3830825e-2aa3-4d17-b4cd-823c9f2e1a19 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:45.512445 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3830825e-2aa3-4d17-b4cd-823c9f2e1a19 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:45.703548 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-3830825e-2aa3-4d17-b4cd-823c9f2e1a19 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '833136cbdeab4b9e8b6dd2d24c7c8753', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '77c9b7e351e24dc88207e9498f75f578', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:41:45.714701 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-3830825e-2aa3-4d17-b4cd-823c9f2e1a19 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] 10.222.0.16 "GET /compute/v2.1/servers/35a74168-3c95-4bfe-b82a-e93b10ec5820" status: 200 len: 1698 microversion: 2.1 time: 0.225366 Jul 16 21:41:45.728780 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2756/5509] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:41:45 2022] GET /compute/v2.1/servers/35a74168-3c95-4bfe-b82a-e93b10ec5820 => generated 1698 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:41:46.009250 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-891df0cf-812f-4f81-80d6-383250e37051 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestVolumeBootPattern-server-1895526112", "imageRef": "", "flavorRef": "452", "key_name": "tempest-TestVolumeBootPattern-1632947712", "security_groups": [{"name": "tempest-secgroup-smoke-459948046"}], "block_device_mapping_v2": [{"uuid": "183425a4-3857-401b-8f23-e0e83f107150", "source_type": "volume", "destination_type": "volume", "boot_index": 0, "delete_on_termination": false}], "networks": [{"uuid": "94c922c8-150a-417e-8274-7473d6567b26"}]}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:41:46.216167 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.network.neutron [None req-891df0cf-812f-4f81-80d6-383250e37051 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] validate_networks() for [('94c922c8-150a-417e-8274-7473d6567b26', None, None, None, None, None)] {{(pid=138680) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 16 21:41:46.727255 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-891df0cf-812f-4f81-80d6-383250e37051 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Flavor limits 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 16 21:41:46.727695 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-891df0cf-812f-4f81-80d6-383250e37051 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Image limits 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 16 21:41:46.728185 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-891df0cf-812f-4f81-80d6-383250e37051 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Flavor pref 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 16 21:41:46.728606 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-891df0cf-812f-4f81-80d6-383250e37051 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Image pref 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 16 21:41:46.729029 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-891df0cf-812f-4f81-80d6-383250e37051 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 16 21:41:46.737198 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-877ded31-8e34-4b1b-8a6a-692357124f5c tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:41:46.737775 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-877ded31-8e34-4b1b-8a6a-692357124f5c tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] [instance: 35a74168-3c95-4bfe-b82a-e93b10ec5820] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:41:46.746275 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-877ded31-8e34-4b1b-8a6a-692357124f5c tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:46.746972 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-877ded31-8e34-4b1b-8a6a-692357124f5c tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:46.748057 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-877ded31-8e34-4b1b-8a6a-692357124f5c tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:46.748753 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-877ded31-8e34-4b1b-8a6a-692357124f5c tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:46.753248 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-891df0cf-812f-4f81-80d6-383250e37051 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:46.758802 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-891df0cf-812f-4f81-80d6-383250e37051 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:46.758802 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-891df0cf-812f-4f81-80d6-383250e37051 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:46.758802 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-891df0cf-812f-4f81-80d6-383250e37051 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:46.789368 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-891df0cf-812f-4f81-80d6-383250e37051 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Getting quotas for project b533f85aeddf4d8db05f2c53a61befc3. Resources: {'ram', 'instances', 'cores'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:41:46.801716 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-891df0cf-812f-4f81-80d6-383250e37051 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Getting quotas for user 62209b9802ab46c3b3d094d4d1ad86fe and project b533f85aeddf4d8db05f2c53a61befc3. Resources: {'ram', 'instances', 'cores'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:41:46.881485 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-891df0cf-812f-4f81-80d6-383250e37051 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Going to run 1 instances... {{(pid=138680) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 16 21:41:46.883910 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-891df0cf-812f-4f81-80d6-383250e37051 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] [instance: 5865f75f-6142-4885-9fa6-0687a2c169cc] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=False,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag=None,updated_at=,uuid=,volume_id='183425a4-3857-401b-8f23-e0e83f107150',volume_size=None,volume_type=None)] {{(pid=138680) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 16 21:41:46.949623 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-877ded31-8e34-4b1b-8a6a-692357124f5c tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '833136cbdeab4b9e8b6dd2d24c7c8753', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '77c9b7e351e24dc88207e9498f75f578', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:41:46.960599 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-877ded31-8e34-4b1b-8a6a-692357124f5c tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] 10.222.0.16 "GET /compute/v2.1/servers/35a74168-3c95-4bfe-b82a-e93b10ec5820" status: 200 len: 1698 microversion: 2.1 time: 0.224858 Jul 16 21:41:46.961409 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2757/5510] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:41:46 2022] GET /compute/v2.1/servers/35a74168-3c95-4bfe-b82a-e93b10ec5820 => generated 1698 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:41:47.392376 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-891df0cf-812f-4f81-80d6-383250e37051 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 405 microversion: 2.1 time: 1.387436 Jul 16 21:41:47.406613 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-08f7c0c8-3862-406c-bf2b-d65a5d93f606 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:41:47.407608 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-08f7c0c8-3862-406c-bf2b-d65a5d93f606 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] [instance: 5865f75f-6142-4885-9fa6-0687a2c169cc] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:41:47.412808 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2754/5511] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 21:41:46 2022] POST /compute/v2.1/servers => generated 405 bytes in 1408 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 16 21:41:47.484160 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-08f7c0c8-3862-406c-bf2b-d65a5d93f606 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '62209b9802ab46c3b3d094d4d1ad86fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b533f85aeddf4d8db05f2c53a61befc3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:41:47.494399 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-08f7c0c8-3862-406c-bf2b-d65a5d93f606 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] 10.222.0.16 "GET /compute/v2.1/servers/5865f75f-6142-4885-9fa6-0687a2c169cc" status: 200 len: 1095 microversion: 2.1 time: 0.092689 Jul 16 21:41:47.504142 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2758/5512] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:41:47 2022] GET /compute/v2.1/servers/5865f75f-6142-4885-9fa6-0687a2c169cc => generated 1095 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:41:47.977677 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-7545f5a8-3204-4a5f-b459-088c88d6382a tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:41:47.978294 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-7545f5a8-3204-4a5f-b459-088c88d6382a tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] [instance: 35a74168-3c95-4bfe-b82a-e93b10ec5820] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:41:47.985046 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7545f5a8-3204-4a5f-b459-088c88d6382a tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:47.985500 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7545f5a8-3204-4a5f-b459-088c88d6382a tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:47.986330 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7545f5a8-3204-4a5f-b459-088c88d6382a tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:47.986743 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7545f5a8-3204-4a5f-b459-088c88d6382a tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:48.178405 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-7545f5a8-3204-4a5f-b459-088c88d6382a tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '833136cbdeab4b9e8b6dd2d24c7c8753', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '77c9b7e351e24dc88207e9498f75f578', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:41:48.187564 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-7545f5a8-3204-4a5f-b459-088c88d6382a tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] 10.222.0.16 "GET /compute/v2.1/servers/35a74168-3c95-4bfe-b82a-e93b10ec5820" status: 200 len: 1698 microversion: 2.1 time: 0.213694 Jul 16 21:41:48.188799 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2755/5513] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:41:47 2022] GET /compute/v2.1/servers/35a74168-3c95-4bfe-b82a-e93b10ec5820 => generated 1698 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:41:48.510815 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-ce095a26-9fae-48af-8028-8bfe76c3ba7b tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:41:48.512127 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-ce095a26-9fae-48af-8028-8bfe76c3ba7b tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] [instance: 5865f75f-6142-4885-9fa6-0687a2c169cc] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:41:48.526732 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ce095a26-9fae-48af-8028-8bfe76c3ba7b tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:48.529929 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ce095a26-9fae-48af-8028-8bfe76c3ba7b tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:48.530822 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ce095a26-9fae-48af-8028-8bfe76c3ba7b tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:48.531326 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ce095a26-9fae-48af-8028-8bfe76c3ba7b tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:48.611887 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-ce095a26-9fae-48af-8028-8bfe76c3ba7b tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '62209b9802ab46c3b3d094d4d1ad86fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b533f85aeddf4d8db05f2c53a61befc3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:41:48.627564 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-ce095a26-9fae-48af-8028-8bfe76c3ba7b tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] 10.222.0.16 "GET /compute/v2.1/servers/5865f75f-6142-4885-9fa6-0687a2c169cc" status: 200 len: 1145 microversion: 2.1 time: 0.120433 Jul 16 21:41:48.628516 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2759/5514] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:41:48 2022] GET /compute/v2.1/servers/5865f75f-6142-4885-9fa6-0687a2c169cc => generated 1145 bytes in 123 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:41:49.209515 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-79f56302-1996-431b-9014-5268a2c340bc tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:41:49.211752 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-79f56302-1996-431b-9014-5268a2c340bc tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] [instance: 35a74168-3c95-4bfe-b82a-e93b10ec5820] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:41:49.222550 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-79f56302-1996-431b-9014-5268a2c340bc tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:49.223615 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-79f56302-1996-431b-9014-5268a2c340bc tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:49.226342 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-79f56302-1996-431b-9014-5268a2c340bc tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:49.227200 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-79f56302-1996-431b-9014-5268a2c340bc tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:49.415997 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-79f56302-1996-431b-9014-5268a2c340bc tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '833136cbdeab4b9e8b6dd2d24c7c8753', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '77c9b7e351e24dc88207e9498f75f578', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:41:49.464977 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-79f56302-1996-431b-9014-5268a2c340bc tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] 10.222.0.16 "GET /compute/v2.1/servers/35a74168-3c95-4bfe-b82a-e93b10ec5820" status: 200 len: 1698 microversion: 2.1 time: 0.225824 Jul 16 21:41:49.464977 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2756/5515] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:41:49 2022] GET /compute/v2.1/servers/35a74168-3c95-4bfe-b82a-e93b10ec5820 => generated 1698 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:41:49.645982 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-be146195-b3f3-4e3f-9f2f-fc08a3553642 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:41:49.647644 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-be146195-b3f3-4e3f-9f2f-fc08a3553642 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] [instance: 5865f75f-6142-4885-9fa6-0687a2c169cc] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:41:49.656828 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-be146195-b3f3-4e3f-9f2f-fc08a3553642 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:49.657990 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-be146195-b3f3-4e3f-9f2f-fc08a3553642 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:49.658970 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-be146195-b3f3-4e3f-9f2f-fc08a3553642 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:49.660155 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-be146195-b3f3-4e3f-9f2f-fc08a3553642 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:49.786453 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-be146195-b3f3-4e3f-9f2f-fc08a3553642 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '62209b9802ab46c3b3d094d4d1ad86fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b533f85aeddf4d8db05f2c53a61befc3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:41:49.794619 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-be146195-b3f3-4e3f-9f2f-fc08a3553642 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] 10.222.0.16 "GET /compute/v2.1/servers/5865f75f-6142-4885-9fa6-0687a2c169cc" status: 200 len: 1193 microversion: 2.1 time: 0.153209 Jul 16 21:41:49.799831 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2760/5516] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:41:49 2022] GET /compute/v2.1/servers/5865f75f-6142-4885-9fa6-0687a2c169cc => generated 1193 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:41:50.462306 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-b01facf0-fb7d-4b67-baff-023a59a1da45 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:41:50.462918 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-b01facf0-fb7d-4b67-baff-023a59a1da45 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] [instance: 35a74168-3c95-4bfe-b82a-e93b10ec5820] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:41:50.473573 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b01facf0-fb7d-4b67-baff-023a59a1da45 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:50.478951 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b01facf0-fb7d-4b67-baff-023a59a1da45 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:50.483909 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b01facf0-fb7d-4b67-baff-023a59a1da45 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:50.485302 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b01facf0-fb7d-4b67-baff-023a59a1da45 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:50.764424 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-b01facf0-fb7d-4b67-baff-023a59a1da45 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '833136cbdeab4b9e8b6dd2d24c7c8753', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '77c9b7e351e24dc88207e9498f75f578', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:41:50.779583 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-b01facf0-fb7d-4b67-baff-023a59a1da45 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] 10.222.0.16 "GET /compute/v2.1/servers/35a74168-3c95-4bfe-b82a-e93b10ec5820" status: 200 len: 1698 microversion: 2.1 time: 0.322123 Jul 16 21:41:50.785481 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2757/5517] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:41:50 2022] GET /compute/v2.1/servers/35a74168-3c95-4bfe-b82a-e93b10ec5820 => generated 1698 bytes in 324 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:41:50.818918 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-6d36ccae-2f21-49a9-b694-623a2899cce1 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:41:50.824662 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-6d36ccae-2f21-49a9-b694-623a2899cce1 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] [instance: 5865f75f-6142-4885-9fa6-0687a2c169cc] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:41:50.835930 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6d36ccae-2f21-49a9-b694-623a2899cce1 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:50.839303 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6d36ccae-2f21-49a9-b694-623a2899cce1 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:50.841869 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6d36ccae-2f21-49a9-b694-623a2899cce1 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:50.844935 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6d36ccae-2f21-49a9-b694-623a2899cce1 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:50.856607 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:41:51.230501 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-6d36ccae-2f21-49a9-b694-623a2899cce1 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '62209b9802ab46c3b3d094d4d1ad86fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b533f85aeddf4d8db05f2c53a61befc3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:41:51.248156 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-6d36ccae-2f21-49a9-b694-623a2899cce1 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] 10.222.0.16 "GET /compute/v2.1/servers/5865f75f-6142-4885-9fa6-0687a2c169cc" status: 200 len: 1278 microversion: 2.1 time: 0.434543 Jul 16 21:41:51.248755 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2761/5518] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:41:50 2022] GET /compute/v2.1/servers/5865f75f-6142-4885-9fa6-0687a2c169cc => generated 1278 bytes in 436 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:41:51.796951 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-9f9da177-66d8-4a3c-a7b0-fc15cd8c7c2f tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:41:51.797699 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-9f9da177-66d8-4a3c-a7b0-fc15cd8c7c2f tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] [instance: 35a74168-3c95-4bfe-b82a-e93b10ec5820] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:41:51.804771 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9f9da177-66d8-4a3c-a7b0-fc15cd8c7c2f tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:51.805222 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9f9da177-66d8-4a3c-a7b0-fc15cd8c7c2f tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:51.805981 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9f9da177-66d8-4a3c-a7b0-fc15cd8c7c2f tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:51.806421 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9f9da177-66d8-4a3c-a7b0-fc15cd8c7c2f tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:52.144921 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-9f9da177-66d8-4a3c-a7b0-fc15cd8c7c2f tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '833136cbdeab4b9e8b6dd2d24c7c8753', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '77c9b7e351e24dc88207e9498f75f578', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:41:52.168379 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-9f9da177-66d8-4a3c-a7b0-fc15cd8c7c2f tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] 10.222.0.16 "GET /compute/v2.1/servers/35a74168-3c95-4bfe-b82a-e93b10ec5820" status: 200 len: 1698 microversion: 2.1 time: 0.375156 Jul 16 21:41:52.169654 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2758/5519] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:41:51 2022] GET /compute/v2.1/servers/35a74168-3c95-4bfe-b82a-e93b10ec5820 => generated 1698 bytes in 377 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:41:52.265699 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-63d0f82d-daf9-4ef1-9fc9-e775e48ddedd tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:41:52.267175 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-63d0f82d-daf9-4ef1-9fc9-e775e48ddedd tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] [instance: 5865f75f-6142-4885-9fa6-0687a2c169cc] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:41:52.272335 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_api:SELECT=61,nova_api:INSERT=1 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:41:52.276643 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-63d0f82d-daf9-4ef1-9fc9-e775e48ddedd tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:52.277290 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-63d0f82d-daf9-4ef1-9fc9-e775e48ddedd tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:52.278343 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-63d0f82d-daf9-4ef1-9fc9-e775e48ddedd tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:52.278975 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-63d0f82d-daf9-4ef1-9fc9-e775e48ddedd tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:52.491537 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-885fbb58-5a74-414a-8b04-3e937894403e req-2da18bfb-9b7b-4bcd-b9f9-3b5fc3e89a28 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "5865f75f-6142-4885-9fa6-0687a2c169cc", "tag": "c297d034-2431-40d9-9e2d-8119f3b5331b"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:41:52.503604 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-885fbb58-5a74-414a-8b04-3e937894403e req-2da18bfb-9b7b-4bcd-b9f9-3b5fc3e89a28 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:52.503604 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-885fbb58-5a74-414a-8b04-3e937894403e req-2da18bfb-9b7b-4bcd-b9f9-3b5fc3e89a28 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:52.541046 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-885fbb58-5a74-414a-8b04-3e937894403e req-2da18bfb-9b7b-4bcd-b9f9-3b5fc3e89a28 service nova] Creating event network-changed:c297d034-2431-40d9-9e2d-8119f3b5331b for instance 5865f75f-6142-4885-9fa6-0687a2c169cc on nch1-master-1 Jul 16 21:41:52.552578 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-885fbb58-5a74-414a-8b04-3e937894403e req-2da18bfb-9b7b-4bcd-b9f9-3b5fc3e89a28 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.063874 Jul 16 21:41:52.554163 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2759/5520] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:41:52 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 68 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 16 21:41:52.599319 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-63d0f82d-daf9-4ef1-9fc9-e775e48ddedd tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '62209b9802ab46c3b3d094d4d1ad86fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b533f85aeddf4d8db05f2c53a61befc3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:41:52.608938 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-63d0f82d-daf9-4ef1-9fc9-e775e48ddedd tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] 10.222.0.16 "GET /compute/v2.1/servers/5865f75f-6142-4885-9fa6-0687a2c169cc" status: 200 len: 1278 microversion: 2.1 time: 0.346676 Jul 16 21:41:52.610492 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2762/5521] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:41:52 2022] GET /compute/v2.1/servers/5865f75f-6142-4885-9fa6-0687a2c169cc => generated 1278 bytes in 349 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:41:53.186899 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-75765894-9130-46ff-a15e-a8e2bcc80a89 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:41:53.187667 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-75765894-9130-46ff-a15e-a8e2bcc80a89 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] [instance: 35a74168-3c95-4bfe-b82a-e93b10ec5820] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:41:53.202998 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-75765894-9130-46ff-a15e-a8e2bcc80a89 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:53.203813 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-75765894-9130-46ff-a15e-a8e2bcc80a89 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:53.205246 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-75765894-9130-46ff-a15e-a8e2bcc80a89 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:53.206093 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-75765894-9130-46ff-a15e-a8e2bcc80a89 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:53.507638 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-75765894-9130-46ff-a15e-a8e2bcc80a89 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '833136cbdeab4b9e8b6dd2d24c7c8753', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '77c9b7e351e24dc88207e9498f75f578', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:41:53.522240 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-75765894-9130-46ff-a15e-a8e2bcc80a89 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] 10.222.0.16 "GET /compute/v2.1/servers/35a74168-3c95-4bfe-b82a-e93b10ec5820" status: 200 len: 1698 microversion: 2.1 time: 0.339830 Jul 16 21:41:53.523323 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2760/5522] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:41:53 2022] GET /compute/v2.1/servers/35a74168-3c95-4bfe-b82a-e93b10ec5820 => generated 1698 bytes in 342 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:41:53.626181 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-6be6fbd8-09f8-4d48-a3ad-f84ec72d17e9 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:41:53.627597 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-6be6fbd8-09f8-4d48-a3ad-f84ec72d17e9 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] [instance: 5865f75f-6142-4885-9fa6-0687a2c169cc] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:41:53.640321 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6be6fbd8-09f8-4d48-a3ad-f84ec72d17e9 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:53.641123 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6be6fbd8-09f8-4d48-a3ad-f84ec72d17e9 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:53.642836 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6be6fbd8-09f8-4d48-a3ad-f84ec72d17e9 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:53.643522 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6be6fbd8-09f8-4d48-a3ad-f84ec72d17e9 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:53.804586 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-6be6fbd8-09f8-4d48-a3ad-f84ec72d17e9 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '62209b9802ab46c3b3d094d4d1ad86fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b533f85aeddf4d8db05f2c53a61befc3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:41:53.812366 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-6be6fbd8-09f8-4d48-a3ad-f84ec72d17e9 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] 10.222.0.16 "GET /compute/v2.1/servers/5865f75f-6142-4885-9fa6-0687a2c169cc" status: 200 len: 1266 microversion: 2.1 time: 0.190125 Jul 16 21:41:53.813022 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2763/5523] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:41:53 2022] GET /compute/v2.1/servers/5865f75f-6142-4885-9fa6-0687a2c169cc => generated 1266 bytes in 191 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:41:54.544538 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-21a1ed3d-9d3b-43f0-98e3-7cd2d14859ff tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:41:54.545868 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-21a1ed3d-9d3b-43f0-98e3-7cd2d14859ff tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] [instance: 35a74168-3c95-4bfe-b82a-e93b10ec5820] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:41:54.556850 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-21a1ed3d-9d3b-43f0-98e3-7cd2d14859ff tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:54.557543 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-21a1ed3d-9d3b-43f0-98e3-7cd2d14859ff tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:54.559149 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-21a1ed3d-9d3b-43f0-98e3-7cd2d14859ff tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:54.559788 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-21a1ed3d-9d3b-43f0-98e3-7cd2d14859ff tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:54.589831 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:41:54.778039 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-21a1ed3d-9d3b-43f0-98e3-7cd2d14859ff tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '833136cbdeab4b9e8b6dd2d24c7c8753', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '77c9b7e351e24dc88207e9498f75f578', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:41:54.787054 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-21a1ed3d-9d3b-43f0-98e3-7cd2d14859ff tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] 10.222.0.16 "GET /compute/v2.1/servers/35a74168-3c95-4bfe-b82a-e93b10ec5820" status: 200 len: 1698 microversion: 2.1 time: 0.246840 Jul 16 21:41:54.788446 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2761/5524] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:41:54 2022] GET /compute/v2.1/servers/35a74168-3c95-4bfe-b82a-e93b10ec5820 => generated 1698 bytes in 249 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:41:54.841112 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-7f441d80-e80d-4d10-bf5a-ac48ee538b84 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:41:54.842263 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-7f441d80-e80d-4d10-bf5a-ac48ee538b84 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] [instance: 5865f75f-6142-4885-9fa6-0687a2c169cc] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:41:54.860554 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7f441d80-e80d-4d10-bf5a-ac48ee538b84 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:54.861040 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7f441d80-e80d-4d10-bf5a-ac48ee538b84 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:54.861983 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7f441d80-e80d-4d10-bf5a-ac48ee538b84 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:54.862626 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7f441d80-e80d-4d10-bf5a-ac48ee538b84 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:55.061608 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-7f441d80-e80d-4d10-bf5a-ac48ee538b84 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '62209b9802ab46c3b3d094d4d1ad86fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b533f85aeddf4d8db05f2c53a61befc3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:41:55.081823 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-7f441d80-e80d-4d10-bf5a-ac48ee538b84 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] 10.222.0.16 "GET /compute/v2.1/servers/5865f75f-6142-4885-9fa6-0687a2c169cc" status: 200 len: 1266 microversion: 2.1 time: 0.253948 Jul 16 21:41:55.083542 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2764/5525] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:41:54 2022] GET /compute/v2.1/servers/5865f75f-6142-4885-9fa6-0687a2c169cc => generated 1266 bytes in 257 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:41:55.805884 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-a6e78214-b0f6-4ca3-abf7-3fdb2a9cad57 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:41:55.807246 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-a6e78214-b0f6-4ca3-abf7-3fdb2a9cad57 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] [instance: 35a74168-3c95-4bfe-b82a-e93b10ec5820] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:41:55.815316 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a6e78214-b0f6-4ca3-abf7-3fdb2a9cad57 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:55.816080 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a6e78214-b0f6-4ca3-abf7-3fdb2a9cad57 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:55.817124 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a6e78214-b0f6-4ca3-abf7-3fdb2a9cad57 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:55.817646 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a6e78214-b0f6-4ca3-abf7-3fdb2a9cad57 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:55.988885 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-a6e78214-b0f6-4ca3-abf7-3fdb2a9cad57 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '833136cbdeab4b9e8b6dd2d24c7c8753', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '77c9b7e351e24dc88207e9498f75f578', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:41:56.004312 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-a6e78214-b0f6-4ca3-abf7-3fdb2a9cad57 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] 10.222.0.16 "GET /compute/v2.1/servers/35a74168-3c95-4bfe-b82a-e93b10ec5820" status: 200 len: 1698 microversion: 2.1 time: 0.202492 Jul 16 21:41:56.006183 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2762/5526] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:41:55 2022] GET /compute/v2.1/servers/35a74168-3c95-4bfe-b82a-e93b10ec5820 => generated 1698 bytes in 204 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:41:56.101130 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-121b9933-f94b-4cd9-9198-1c899d9f37d5 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:41:56.101961 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-121b9933-f94b-4cd9-9198-1c899d9f37d5 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] [instance: 5865f75f-6142-4885-9fa6-0687a2c169cc] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:41:56.111144 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-121b9933-f94b-4cd9-9198-1c899d9f37d5 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:56.111696 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-121b9933-f94b-4cd9-9198-1c899d9f37d5 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:56.112823 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-121b9933-f94b-4cd9-9198-1c899d9f37d5 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:56.113280 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-121b9933-f94b-4cd9-9198-1c899d9f37d5 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:56.276954 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-121b9933-f94b-4cd9-9198-1c899d9f37d5 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '62209b9802ab46c3b3d094d4d1ad86fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b533f85aeddf4d8db05f2c53a61befc3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:41:56.290234 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-121b9933-f94b-4cd9-9198-1c899d9f37d5 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] 10.222.0.16 "GET /compute/v2.1/servers/5865f75f-6142-4885-9fa6-0687a2c169cc" status: 200 len: 1266 microversion: 2.1 time: 0.194309 Jul 16 21:41:56.293120 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2765/5527] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:41:56 2022] GET /compute/v2.1/servers/5865f75f-6142-4885-9fa6-0687a2c169cc => generated 1266 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:41:57.021835 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-99742618-f64d-46a8-8ccf-56838e0a5108 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:41:57.022601 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-99742618-f64d-46a8-8ccf-56838e0a5108 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] [instance: 35a74168-3c95-4bfe-b82a-e93b10ec5820] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:41:57.024515 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell0:SELECT=2 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:41:57.026110 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell0:SELECT=6 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:41:57.030997 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-99742618-f64d-46a8-8ccf-56838e0a5108 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:57.031487 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-99742618-f64d-46a8-8ccf-56838e0a5108 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:57.032249 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-99742618-f64d-46a8-8ccf-56838e0a5108 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:57.032690 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-99742618-f64d-46a8-8ccf-56838e0a5108 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:57.170701 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-99742618-f64d-46a8-8ccf-56838e0a5108 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '833136cbdeab4b9e8b6dd2d24c7c8753', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '77c9b7e351e24dc88207e9498f75f578', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:41:57.179236 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-99742618-f64d-46a8-8ccf-56838e0a5108 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] 10.222.0.16 "GET /compute/v2.1/servers/35a74168-3c95-4bfe-b82a-e93b10ec5820" status: 200 len: 1698 microversion: 2.1 time: 0.161334 Jul 16 21:41:57.179825 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2763/5528] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:41:57 2022] GET /compute/v2.1/servers/35a74168-3c95-4bfe-b82a-e93b10ec5820 => generated 1698 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:41:57.307566 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-ff7aa691-a923-4f5a-934f-f9df14f2443c tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:41:57.308330 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-ff7aa691-a923-4f5a-934f-f9df14f2443c tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] [instance: 5865f75f-6142-4885-9fa6-0687a2c169cc] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:41:57.315529 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ff7aa691-a923-4f5a-934f-f9df14f2443c tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:57.316000 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ff7aa691-a923-4f5a-934f-f9df14f2443c tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:57.316989 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ff7aa691-a923-4f5a-934f-f9df14f2443c tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:57.317443 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ff7aa691-a923-4f5a-934f-f9df14f2443c tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:57.456319 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-ff7aa691-a923-4f5a-934f-f9df14f2443c tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '62209b9802ab46c3b3d094d4d1ad86fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b533f85aeddf4d8db05f2c53a61befc3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:41:57.465507 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-ff7aa691-a923-4f5a-934f-f9df14f2443c tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] 10.222.0.16 "GET /compute/v2.1/servers/5865f75f-6142-4885-9fa6-0687a2c169cc" status: 200 len: 1266 microversion: 2.1 time: 0.162176 Jul 16 21:41:57.468166 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2766/5529] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:41:57 2022] GET /compute/v2.1/servers/5865f75f-6142-4885-9fa6-0687a2c169cc => generated 1266 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:41:58.201510 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-d9a3df2d-c1fb-4cdc-94d5-e56356912dd6 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:41:58.212053 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-d9a3df2d-c1fb-4cdc-94d5-e56356912dd6 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] [instance: 35a74168-3c95-4bfe-b82a-e93b10ec5820] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:41:58.212053 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d9a3df2d-c1fb-4cdc-94d5-e56356912dd6 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:58.212053 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d9a3df2d-c1fb-4cdc-94d5-e56356912dd6 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:58.212053 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d9a3df2d-c1fb-4cdc-94d5-e56356912dd6 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:58.212053 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d9a3df2d-c1fb-4cdc-94d5-e56356912dd6 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:58.347614 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-d9a3df2d-c1fb-4cdc-94d5-e56356912dd6 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '833136cbdeab4b9e8b6dd2d24c7c8753', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '77c9b7e351e24dc88207e9498f75f578', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:41:58.356483 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-d9a3df2d-c1fb-4cdc-94d5-e56356912dd6 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] 10.222.0.16 "GET /compute/v2.1/servers/35a74168-3c95-4bfe-b82a-e93b10ec5820" status: 200 len: 1698 microversion: 2.1 time: 0.158666 Jul 16 21:41:58.357053 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2764/5530] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:41:58 2022] GET /compute/v2.1/servers/35a74168-3c95-4bfe-b82a-e93b10ec5820 => generated 1698 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:41:58.483851 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-370cf830-8a07-4671-be67-228f154a2e4a tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:41:58.484395 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-370cf830-8a07-4671-be67-228f154a2e4a tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] [instance: 5865f75f-6142-4885-9fa6-0687a2c169cc] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:41:58.486228 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell0:SELECT=2 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:41:58.492875 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-370cf830-8a07-4671-be67-228f154a2e4a tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:58.493423 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-370cf830-8a07-4671-be67-228f154a2e4a tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:58.494211 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-370cf830-8a07-4671-be67-228f154a2e4a tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:58.494670 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-370cf830-8a07-4671-be67-228f154a2e4a tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:58.632019 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-370cf830-8a07-4671-be67-228f154a2e4a tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '62209b9802ab46c3b3d094d4d1ad86fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b533f85aeddf4d8db05f2c53a61befc3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:41:58.639627 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-370cf830-8a07-4671-be67-228f154a2e4a tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] 10.222.0.16 "GET /compute/v2.1/servers/5865f75f-6142-4885-9fa6-0687a2c169cc" status: 200 len: 1266 microversion: 2.1 time: 0.160171 Jul 16 21:41:58.640205 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2767/5531] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:41:58 2022] GET /compute/v2.1/servers/5865f75f-6142-4885-9fa6-0687a2c169cc => generated 1266 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:41:59.375515 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-009e7042-b620-4627-955c-75ff1f87af49 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:41:59.376419 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-009e7042-b620-4627-955c-75ff1f87af49 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] [instance: 35a74168-3c95-4bfe-b82a-e93b10ec5820] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:41:59.385379 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-009e7042-b620-4627-955c-75ff1f87af49 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:59.385991 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-009e7042-b620-4627-955c-75ff1f87af49 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:59.386887 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-009e7042-b620-4627-955c-75ff1f87af49 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:59.387449 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-009e7042-b620-4627-955c-75ff1f87af49 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:59.529162 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-009e7042-b620-4627-955c-75ff1f87af49 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '833136cbdeab4b9e8b6dd2d24c7c8753', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '77c9b7e351e24dc88207e9498f75f578', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:41:59.537051 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-009e7042-b620-4627-955c-75ff1f87af49 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] 10.222.0.16 "GET /compute/v2.1/servers/35a74168-3c95-4bfe-b82a-e93b10ec5820" status: 200 len: 1698 microversion: 2.1 time: 0.165787 Jul 16 21:41:59.537711 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2765/5532] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:41:59 2022] GET /compute/v2.1/servers/35a74168-3c95-4bfe-b82a-e93b10ec5820 => generated 1698 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:41:59.657028 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-a211d097-53f0-4c96-b502-9ae9688467a2 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:41:59.657664 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-a211d097-53f0-4c96-b502-9ae9688467a2 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] [instance: 5865f75f-6142-4885-9fa6-0687a2c169cc] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:41:59.665707 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-96c7c5fe-64aa-49c8-8484-d97297f15fac req-5e3154a9-95db-44b2-a971-ca37adda2749 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "5865f75f-6142-4885-9fa6-0687a2c169cc", "name": "network-vif-plugged", "status": "completed", "tag": "c297d034-2431-40d9-9e2d-8119f3b5331b"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:41:59.675586 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-96c7c5fe-64aa-49c8-8484-d97297f15fac req-5e3154a9-95db-44b2-a971-ca37adda2749 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:59.675786 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a211d097-53f0-4c96-b502-9ae9688467a2 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:59.676215 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a211d097-53f0-4c96-b502-9ae9688467a2 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:59.676349 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-96c7c5fe-64aa-49c8-8484-d97297f15fac req-5e3154a9-95db-44b2-a971-ca37adda2749 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:59.677009 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a211d097-53f0-4c96-b502-9ae9688467a2 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:41:59.677465 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a211d097-53f0-4c96-b502-9ae9688467a2 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:41:59.688478 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-96c7c5fe-64aa-49c8-8484-d97297f15fac req-5e3154a9-95db-44b2-a971-ca37adda2749 service nova] Creating event network-vif-plugged:c297d034-2431-40d9-9e2d-8119f3b5331b for instance 5865f75f-6142-4885-9fa6-0687a2c169cc on nch1-master-1 Jul 16 21:41:59.693551 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-96c7c5fe-64aa-49c8-8484-d97297f15fac req-5e3154a9-95db-44b2-a971-ca37adda2749 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.032130 Jul 16 21:41:59.694537 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2766/5533] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:41:59 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:41:59.890563 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-a211d097-53f0-4c96-b502-9ae9688467a2 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '62209b9802ab46c3b3d094d4d1ad86fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b533f85aeddf4d8db05f2c53a61befc3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:41:59.898193 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-a211d097-53f0-4c96-b502-9ae9688467a2 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] 10.222.0.16 "GET /compute/v2.1/servers/5865f75f-6142-4885-9fa6-0687a2c169cc" status: 200 len: 1266 microversion: 2.1 time: 0.245102 Jul 16 21:41:59.898929 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2768/5534] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:41:59 2022] GET /compute/v2.1/servers/5865f75f-6142-4885-9fa6-0687a2c169cc => generated 1266 bytes in 246 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:42:00.558555 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-0f9c6f86-1a3e-4301-9a9a-6fbb1ce834dd tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:42:00.559471 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-0f9c6f86-1a3e-4301-9a9a-6fbb1ce834dd tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] [instance: 35a74168-3c95-4bfe-b82a-e93b10ec5820] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:42:00.566170 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0f9c6f86-1a3e-4301-9a9a-6fbb1ce834dd tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:42:00.566631 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0f9c6f86-1a3e-4301-9a9a-6fbb1ce834dd tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:42:00.567475 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0f9c6f86-1a3e-4301-9a9a-6fbb1ce834dd tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:42:00.568018 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0f9c6f86-1a3e-4301-9a9a-6fbb1ce834dd tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:42:00.917483 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-aca3db0d-73e7-469e-a7f6-a16dd0665901 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:42:00.918114 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-aca3db0d-73e7-469e-a7f6-a16dd0665901 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] [instance: 5865f75f-6142-4885-9fa6-0687a2c169cc] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:42:00.925283 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-aca3db0d-73e7-469e-a7f6-a16dd0665901 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:42:00.925757 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-aca3db0d-73e7-469e-a7f6-a16dd0665901 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:42:00.926581 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-aca3db0d-73e7-469e-a7f6-a16dd0665901 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:42:00.927109 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-aca3db0d-73e7-469e-a7f6-a16dd0665901 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:42:01.126218 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-0f9c6f86-1a3e-4301-9a9a-6fbb1ce834dd tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '833136cbdeab4b9e8b6dd2d24c7c8753', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '77c9b7e351e24dc88207e9498f75f578', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:42:01.134335 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-0f9c6f86-1a3e-4301-9a9a-6fbb1ce834dd tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] 10.222.0.16 "GET /compute/v2.1/servers/35a74168-3c95-4bfe-b82a-e93b10ec5820" status: 200 len: 1697 microversion: 2.1 time: 0.582103 Jul 16 21:42:01.134930 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2767/5535] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:42:00 2022] GET /compute/v2.1/servers/35a74168-3c95-4bfe-b82a-e93b10ec5820 => generated 1697 bytes in 584 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:42:01.201905 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-aca3db0d-73e7-469e-a7f6-a16dd0665901 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '62209b9802ab46c3b3d094d4d1ad86fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b533f85aeddf4d8db05f2c53a61befc3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:42:01.212250 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-aca3db0d-73e7-469e-a7f6-a16dd0665901 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] 10.222.0.16 "GET /compute/v2.1/servers/5865f75f-6142-4885-9fa6-0687a2c169cc" status: 200 len: 1266 microversion: 2.1 time: 0.298745 Jul 16 21:42:01.212723 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2769/5536] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:42:00 2022] GET /compute/v2.1/servers/5865f75f-6142-4885-9fa6-0687a2c169cc => generated 1266 bytes in 301 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:42:01.233373 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-79dc858a-c71b-4535-affb-30c86f966aa7 req-be74a815-6a2f-46f9-9ba0-2a19797ec32c service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "35a74168-3c95-4bfe-b82a-e93b10ec5820", "name": "network-vif-unplugged", "status": "completed", "tag": "94628a67-e3ab-4037-b7ce-cf51f08c17a6"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:42:01.242702 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-79dc858a-c71b-4535-affb-30c86f966aa7 req-be74a815-6a2f-46f9-9ba0-2a19797ec32c service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:42:01.248854 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-79dc858a-c71b-4535-affb-30c86f966aa7 req-be74a815-6a2f-46f9-9ba0-2a19797ec32c service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:42:01.255115 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-79dc858a-c71b-4535-affb-30c86f966aa7 req-be74a815-6a2f-46f9-9ba0-2a19797ec32c service nova] Creating event network-vif-unplugged:94628a67-e3ab-4037-b7ce-cf51f08c17a6 for instance 35a74168-3c95-4bfe-b82a-e93b10ec5820 on nch2-master-1 Jul 16 21:42:01.261922 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [req-79dc858a-c71b-4535-affb-30c86f966aa7 req-be74a815-6a2f-46f9-9ba0-2a19797ec32c service nova] Instance 35a74168-3c95-4bfe-b82a-e93b10ec5820 is migrating within the same cell, copying events to all relevant hosts: {'nch2-master-1', 'nch1-master-1'} {{(pid=138680) _get_relevant_hosts /opt/stack/nova/nova/compute/api.py:5807}} Jul 16 21:42:01.276448 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-79dc858a-c71b-4535-affb-30c86f966aa7 req-be74a815-6a2f-46f9-9ba0-2a19797ec32c service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.046798 Jul 16 21:42:01.277100 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2768/5537] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:42:01 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:42:02.152413 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-1eb7357c-5671-484f-be65-6166eb342c90 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:42:02.153043 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-1eb7357c-5671-484f-be65-6166eb342c90 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] [instance: 35a74168-3c95-4bfe-b82a-e93b10ec5820] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:42:02.160479 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1eb7357c-5671-484f-be65-6166eb342c90 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:42:02.161043 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1eb7357c-5671-484f-be65-6166eb342c90 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:42:02.161837 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1eb7357c-5671-484f-be65-6166eb342c90 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:42:02.162387 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1eb7357c-5671-484f-be65-6166eb342c90 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:42:02.229986 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-ab3ea577-1bc0-4707-b37a-92705bf0a404 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:42:02.230695 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-ab3ea577-1bc0-4707-b37a-92705bf0a404 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] [instance: 5865f75f-6142-4885-9fa6-0687a2c169cc] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:42:02.238599 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ab3ea577-1bc0-4707-b37a-92705bf0a404 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:42:02.239096 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ab3ea577-1bc0-4707-b37a-92705bf0a404 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:42:02.239871 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ab3ea577-1bc0-4707-b37a-92705bf0a404 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:42:02.240319 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ab3ea577-1bc0-4707-b37a-92705bf0a404 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:42:02.453027 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-1eb7357c-5671-484f-be65-6166eb342c90 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '833136cbdeab4b9e8b6dd2d24c7c8753', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '77c9b7e351e24dc88207e9498f75f578', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:42:02.460609 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-1eb7357c-5671-484f-be65-6166eb342c90 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] 10.222.0.16 "GET /compute/v2.1/servers/35a74168-3c95-4bfe-b82a-e93b10ec5820" status: 200 len: 1697 microversion: 2.1 time: 0.312479 Jul 16 21:42:02.461192 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2770/5538] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:42:02 2022] GET /compute/v2.1/servers/35a74168-3c95-4bfe-b82a-e93b10ec5820 => generated 1697 bytes in 314 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:42:02.490531 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-ab3ea577-1bc0-4707-b37a-92705bf0a404 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '62209b9802ab46c3b3d094d4d1ad86fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b533f85aeddf4d8db05f2c53a61befc3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:42:02.498622 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-ab3ea577-1bc0-4707-b37a-92705bf0a404 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] 10.222.0.16 "GET /compute/v2.1/servers/5865f75f-6142-4885-9fa6-0687a2c169cc" status: 200 len: 1453 microversion: 2.1 time: 0.273108 Jul 16 21:42:02.499304 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2769/5539] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:42:02 2022] GET /compute/v2.1/servers/5865f75f-6142-4885-9fa6-0687a2c169cc => generated 1453 bytes in 275 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:42:02.515192 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-862821bb-dce5-4224-b86d-09b5f9c2621f tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:42:02.516081 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-862821bb-dce5-4224-b86d-09b5f9c2621f tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] [instance: 5865f75f-6142-4885-9fa6-0687a2c169cc] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:42:02.522220 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-862821bb-dce5-4224-b86d-09b5f9c2621f tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:42:02.522847 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-862821bb-dce5-4224-b86d-09b5f9c2621f tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:42:02.523705 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-862821bb-dce5-4224-b86d-09b5f9c2621f tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:42:02.524157 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-862821bb-dce5-4224-b86d-09b5f9c2621f tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:42:02.656453 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-862821bb-dce5-4224-b86d-09b5f9c2621f tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '62209b9802ab46c3b3d094d4d1ad86fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b533f85aeddf4d8db05f2c53a61befc3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:42:02.664401 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-862821bb-dce5-4224-b86d-09b5f9c2621f tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] 10.222.0.16 "GET /compute/v2.1/servers/5865f75f-6142-4885-9fa6-0687a2c169cc" status: 200 len: 1453 microversion: 2.1 time: 0.153539 Jul 16 21:42:02.665681 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2771/5540] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:42:02 2022] GET /compute/v2.1/servers/5865f75f-6142-4885-9fa6-0687a2c169cc => generated 1453 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:42:03.290792 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-2a080e2f-a593-45d7-a71e-2cf65f540836 req-4a0496d9-4199-4784-914a-f391d0b7f999 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "35a74168-3c95-4bfe-b82a-e93b10ec5820", "tag": "94628a67-e3ab-4037-b7ce-cf51f08c17a6"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:42:03.300393 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-2a080e2f-a593-45d7-a71e-2cf65f540836 req-4a0496d9-4199-4784-914a-f391d0b7f999 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:42:03.300944 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-2a080e2f-a593-45d7-a71e-2cf65f540836 req-4a0496d9-4199-4784-914a-f391d0b7f999 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:42:03.315881 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-2a080e2f-a593-45d7-a71e-2cf65f540836 req-4a0496d9-4199-4784-914a-f391d0b7f999 service nova] Creating event network-changed:94628a67-e3ab-4037-b7ce-cf51f08c17a6 for instance 35a74168-3c95-4bfe-b82a-e93b10ec5820 on nch2-master-1 Jul 16 21:42:03.321641 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [req-2a080e2f-a593-45d7-a71e-2cf65f540836 req-4a0496d9-4199-4784-914a-f391d0b7f999 service nova] Instance 35a74168-3c95-4bfe-b82a-e93b10ec5820 is migrating within the same cell, copying events to all relevant hosts: {'nch2-master-1', 'nch1-master-1'} {{(pid=138680) _get_relevant_hosts /opt/stack/nova/nova/compute/api.py:5807}} Jul 16 21:42:03.333811 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-2a080e2f-a593-45d7-a71e-2cf65f540836 req-4a0496d9-4199-4784-914a-f391d0b7f999 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.046808 Jul 16 21:42:03.334839 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2770/5541] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:42:03 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:42:03.480023 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-3e99962a-cf39-463f-8460-b31640b970c0 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:42:03.481594 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-3e99962a-cf39-463f-8460-b31640b970c0 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] [instance: 35a74168-3c95-4bfe-b82a-e93b10ec5820] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:42:03.489049 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3e99962a-cf39-463f-8460-b31640b970c0 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:42:03.489495 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3e99962a-cf39-463f-8460-b31640b970c0 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:42:03.490284 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3e99962a-cf39-463f-8460-b31640b970c0 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:42:03.491109 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3e99962a-cf39-463f-8460-b31640b970c0 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:42:03.721316 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-3e99962a-cf39-463f-8460-b31640b970c0 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '833136cbdeab4b9e8b6dd2d24c7c8753', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '77c9b7e351e24dc88207e9498f75f578', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:42:03.729759 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-3e99962a-cf39-463f-8460-b31640b970c0 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] 10.222.0.16 "GET /compute/v2.1/servers/35a74168-3c95-4bfe-b82a-e93b10ec5820" status: 200 len: 1695 microversion: 2.1 time: 0.253791 Jul 16 21:42:03.730453 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2772/5542] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:42:03 2022] GET /compute/v2.1/servers/35a74168-3c95-4bfe-b82a-e93b10ec5820 => generated 1695 bytes in 255 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:42:04.750352 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-f1655c03-d198-482d-b5e9-56782ab4afee tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:42:04.751373 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-f1655c03-d198-482d-b5e9-56782ab4afee tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] [instance: 35a74168-3c95-4bfe-b82a-e93b10ec5820] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:42:04.758915 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f1655c03-d198-482d-b5e9-56782ab4afee tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:42:04.759494 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f1655c03-d198-482d-b5e9-56782ab4afee tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:42:04.768638 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f1655c03-d198-482d-b5e9-56782ab4afee tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:42:04.768638 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f1655c03-d198-482d-b5e9-56782ab4afee tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:42:05.031574 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-f1655c03-d198-482d-b5e9-56782ab4afee tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '833136cbdeab4b9e8b6dd2d24c7c8753', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '77c9b7e351e24dc88207e9498f75f578', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:42:05.038358 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-f1655c03-d198-482d-b5e9-56782ab4afee tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] 10.222.0.16 "GET /compute/v2.1/servers/35a74168-3c95-4bfe-b82a-e93b10ec5820" status: 200 len: 1695 microversion: 2.1 time: 0.293264 Jul 16 21:42:05.039369 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2771/5543] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:42:04 2022] GET /compute/v2.1/servers/35a74168-3c95-4bfe-b82a-e93b10ec5820 => generated 1695 bytes in 295 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:42:05.308139 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-9b9a0dd2-ca52-4298-a14d-fe3caebdbb3d req-13eecbd4-8af6-44e1-9364-7d0ddf73a7d6 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "5865f75f-6142-4885-9fa6-0687a2c169cc", "tag": "c297d034-2431-40d9-9e2d-8119f3b5331b"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:42:05.318982 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-9b9a0dd2-ca52-4298-a14d-fe3caebdbb3d req-13eecbd4-8af6-44e1-9364-7d0ddf73a7d6 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:42:05.319760 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-9b9a0dd2-ca52-4298-a14d-fe3caebdbb3d req-13eecbd4-8af6-44e1-9364-7d0ddf73a7d6 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:42:05.334035 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-9b9a0dd2-ca52-4298-a14d-fe3caebdbb3d req-13eecbd4-8af6-44e1-9364-7d0ddf73a7d6 service nova] Creating event network-changed:c297d034-2431-40d9-9e2d-8119f3b5331b for instance 5865f75f-6142-4885-9fa6-0687a2c169cc on nch1-master-1 Jul 16 21:42:05.339317 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-9b9a0dd2-ca52-4298-a14d-fe3caebdbb3d req-13eecbd4-8af6-44e1-9364-7d0ddf73a7d6 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.104999 Jul 16 21:42:05.340034 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2773/5544] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:42:05 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:42:06.058145 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-4224928e-8c9e-49ba-81f4-c0fcd1be10a6 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:42:06.060636 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-4224928e-8c9e-49ba-81f4-c0fcd1be10a6 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] [instance: 35a74168-3c95-4bfe-b82a-e93b10ec5820] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:42:06.066023 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4224928e-8c9e-49ba-81f4-c0fcd1be10a6 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:42:06.066485 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4224928e-8c9e-49ba-81f4-c0fcd1be10a6 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:42:06.067291 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4224928e-8c9e-49ba-81f4-c0fcd1be10a6 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:42:06.067739 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4224928e-8c9e-49ba-81f4-c0fcd1be10a6 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:42:06.284645 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-4224928e-8c9e-49ba-81f4-c0fcd1be10a6 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '833136cbdeab4b9e8b6dd2d24c7c8753', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '77c9b7e351e24dc88207e9498f75f578', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:42:06.292485 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-4224928e-8c9e-49ba-81f4-c0fcd1be10a6 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] 10.222.0.16 "GET /compute/v2.1/servers/35a74168-3c95-4bfe-b82a-e93b10ec5820" status: 200 len: 1692 microversion: 2.1 time: 0.239516 Jul 16 21:42:06.293059 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2772/5545] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:42:06 2022] GET /compute/v2.1/servers/35a74168-3c95-4bfe-b82a-e93b10ec5820 => generated 1692 bytes in 241 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:42:06.311344 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-f4178a9b-4db9-48e8-9654-2b0852e51421 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Action: 'action', calling method: >, body: {"revertResize": {}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:42:06.311995 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-f4178a9b-4db9-48e8-9654-2b0852e51421 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] [instance: 35a74168-3c95-4bfe-b82a-e93b10ec5820] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:42:06.318374 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f4178a9b-4db9-48e8-9654-2b0852e51421 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:42:06.318850 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f4178a9b-4db9-48e8-9654-2b0852e51421 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:42:06.319734 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f4178a9b-4db9-48e8-9654-2b0852e51421 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:42:06.320187 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f4178a9b-4db9-48e8-9654-2b0852e51421 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:42:06.689172 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-f4178a9b-4db9-48e8-9654-2b0852e51421 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] 10.222.0.16 "POST /compute/v2.1/servers/35a74168-3c95-4bfe-b82a-e93b10ec5820/action" status: 202 len: 0 microversion: 2.1 time: 0.383141 Jul 16 21:42:06.689781 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2774/5546] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 21:42:06 2022] POST /compute/v2.1/servers/35a74168-3c95-4bfe-b82a-e93b10ec5820/action => generated 0 bytes in 385 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 21:42:06.698154 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-3b0babdb-d77c-4be5-8e65-422ba87ca7af tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:42:06.698778 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-3b0babdb-d77c-4be5-8e65-422ba87ca7af tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] [instance: 35a74168-3c95-4bfe-b82a-e93b10ec5820] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:42:06.707223 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3b0babdb-d77c-4be5-8e65-422ba87ca7af tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:42:06.707935 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3b0babdb-d77c-4be5-8e65-422ba87ca7af tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:42:06.713336 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3b0babdb-d77c-4be5-8e65-422ba87ca7af tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:42:06.713336 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3b0babdb-d77c-4be5-8e65-422ba87ca7af tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:42:06.919824 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-3b0babdb-d77c-4be5-8e65-422ba87ca7af tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '833136cbdeab4b9e8b6dd2d24c7c8753', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '77c9b7e351e24dc88207e9498f75f578', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:42:06.927749 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-3b0babdb-d77c-4be5-8e65-422ba87ca7af tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] 10.222.0.16 "GET /compute/v2.1/servers/35a74168-3c95-4bfe-b82a-e93b10ec5820" status: 200 len: 1691 microversion: 2.1 time: 0.233588 Jul 16 21:42:06.928528 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2773/5547] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:42:06 2022] GET /compute/v2.1/servers/35a74168-3c95-4bfe-b82a-e93b10ec5820 => generated 1691 bytes in 235 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:42:07.679417 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-bdbd51ba-23f7-4b9b-b27a-e2cd07432fdc req-c792a319-c109-4ce1-bac3-95e51e671251 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "35a74168-3c95-4bfe-b82a-e93b10ec5820", "name": "network-vif-plugged", "status": "completed", "tag": "94628a67-e3ab-4037-b7ce-cf51f08c17a6"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:42:07.689390 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-bdbd51ba-23f7-4b9b-b27a-e2cd07432fdc req-c792a319-c109-4ce1-bac3-95e51e671251 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:42:07.691837 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-bdbd51ba-23f7-4b9b-b27a-e2cd07432fdc req-c792a319-c109-4ce1-bac3-95e51e671251 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:42:07.701438 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-bdbd51ba-23f7-4b9b-b27a-e2cd07432fdc req-c792a319-c109-4ce1-bac3-95e51e671251 service nova] Creating event network-vif-plugged:94628a67-e3ab-4037-b7ce-cf51f08c17a6 for instance 35a74168-3c95-4bfe-b82a-e93b10ec5820 on nch2-master-1 Jul 16 21:42:07.706807 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [req-bdbd51ba-23f7-4b9b-b27a-e2cd07432fdc req-c792a319-c109-4ce1-bac3-95e51e671251 service nova] Instance 35a74168-3c95-4bfe-b82a-e93b10ec5820 is migrating within the same cell, copying events to all relevant hosts: {'nch2-master-1', 'nch1-master-1'} {{(pid=138681) _get_relevant_hosts /opt/stack/nova/nova/compute/api.py:5807}} Jul 16 21:42:07.717396 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-bdbd51ba-23f7-4b9b-b27a-e2cd07432fdc req-c792a319-c109-4ce1-bac3-95e51e671251 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.043133 Jul 16 21:42:07.718235 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2775/5548] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:42:07 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:42:07.945224 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-494b09f6-ba74-48fc-b3b2-29c8ed921620 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:42:07.946126 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-494b09f6-ba74-48fc-b3b2-29c8ed921620 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] [instance: 35a74168-3c95-4bfe-b82a-e93b10ec5820] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:42:07.954069 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-494b09f6-ba74-48fc-b3b2-29c8ed921620 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:42:07.954533 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-494b09f6-ba74-48fc-b3b2-29c8ed921620 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:42:07.955370 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-494b09f6-ba74-48fc-b3b2-29c8ed921620 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:42:07.955815 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-494b09f6-ba74-48fc-b3b2-29c8ed921620 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:42:08.098315 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-494b09f6-ba74-48fc-b3b2-29c8ed921620 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '833136cbdeab4b9e8b6dd2d24c7c8753', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '77c9b7e351e24dc88207e9498f75f578', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:42:08.106399 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-494b09f6-ba74-48fc-b3b2-29c8ed921620 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] 10.222.0.16 "GET /compute/v2.1/servers/35a74168-3c95-4bfe-b82a-e93b10ec5820" status: 200 len: 1691 microversion: 2.1 time: 0.164760 Jul 16 21:42:08.107107 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2774/5549] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:42:07 2022] GET /compute/v2.1/servers/35a74168-3c95-4bfe-b82a-e93b10ec5820 => generated 1691 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:42:09.129217 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-9e444ed9-6bee-4633-8f97-38dabf79771e tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:42:09.130898 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-9e444ed9-6bee-4633-8f97-38dabf79771e tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] [instance: 35a74168-3c95-4bfe-b82a-e93b10ec5820] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:42:09.138427 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9e444ed9-6bee-4633-8f97-38dabf79771e tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:42:09.144371 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9e444ed9-6bee-4633-8f97-38dabf79771e tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:42:09.144371 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9e444ed9-6bee-4633-8f97-38dabf79771e tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:42:09.144371 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9e444ed9-6bee-4633-8f97-38dabf79771e tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:42:09.284110 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-9e444ed9-6bee-4633-8f97-38dabf79771e tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '833136cbdeab4b9e8b6dd2d24c7c8753', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '77c9b7e351e24dc88207e9498f75f578', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:42:09.291977 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-9e444ed9-6bee-4633-8f97-38dabf79771e tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] 10.222.0.16 "GET /compute/v2.1/servers/35a74168-3c95-4bfe-b82a-e93b10ec5820" status: 200 len: 1691 microversion: 2.1 time: 0.168061 Jul 16 21:42:09.292555 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2776/5550] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:42:09 2022] GET /compute/v2.1/servers/35a74168-3c95-4bfe-b82a-e93b10ec5820 => generated 1691 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:42:10.154930 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-eefab35c-17fd-4856-8506-5d43e867446e tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Action: 'create', calling method: .version_select at 0x7fdd9cea6d30>, body: {"keypair": {"name": "tempest-TestSecurityGroupsBasicOps-1925752067", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBMO4ufabmNKN5aioyoCCt3f7B5Sr4yHI67JKAtvvnlsDb6Yw4x59xb5zSlQdGYdv7wlC2spVlDPFrCEQwbP0Y2YCImDg5vmdx003O1tup+C7KoLLYcJ8+REnxaiVU/F5RA=="}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:42:10.172363 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-eefab35c-17fd-4856-8506-5d43e867446e tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Getting quotas for project 251f01f982aa43ad853d53fa06f99190. Resources: {'key_pairs'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:42:10.177832 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-eefab35c-17fd-4856-8506-5d43e867446e tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Getting quotas for user 82819d3efcda4fe78e2b50bb7fab0a96 and project 251f01f982aa43ad853d53fa06f99190. Resources: {'key_pairs'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:42:10.280764 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-eefab35c-17fd-4856-8506-5d43e867446e tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] 10.222.0.16 "POST /compute/v2.1/os-keypairs" status: 200 len: 405 microversion: 2.1 time: 0.302263 Jul 16 21:42:10.285148 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2775/5551] 10.222.0.16 () {60 vars in 1158 bytes} [Sat Jul 16 21:42:09 2022] POST /compute/v2.1/os-keypairs => generated 405 bytes in 304 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:42:10.310935 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-3914f901-57bc-4ac1-8362-64c63ac03464 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:42:10.311602 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-3914f901-57bc-4ac1-8362-64c63ac03464 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] [instance: 35a74168-3c95-4bfe-b82a-e93b10ec5820] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:42:10.319927 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3914f901-57bc-4ac1-8362-64c63ac03464 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:42:10.320392 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3914f901-57bc-4ac1-8362-64c63ac03464 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:42:10.322180 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3914f901-57bc-4ac1-8362-64c63ac03464 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:42:10.322983 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3914f901-57bc-4ac1-8362-64c63ac03464 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:42:10.327483 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:42:10.603703 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-3914f901-57bc-4ac1-8362-64c63ac03464 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '833136cbdeab4b9e8b6dd2d24c7c8753', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '77c9b7e351e24dc88207e9498f75f578', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:42:10.612097 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-3914f901-57bc-4ac1-8362-64c63ac03464 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] 10.222.0.16 "GET /compute/v2.1/servers/35a74168-3c95-4bfe-b82a-e93b10ec5820" status: 200 len: 1691 microversion: 2.1 time: 0.306073 Jul 16 21:42:10.612692 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2777/5552] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:42:10 2022] GET /compute/v2.1/servers/35a74168-3c95-4bfe-b82a-e93b10ec5820 => generated 1691 bytes in 307 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:42:10.785566 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-93974f57-9ae1-4da3-ad72-fb0b6a4749e4 req-f89dddce-c415-4187-b218-a3bb76a4bd48 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "35a74168-3c95-4bfe-b82a-e93b10ec5820", "name": "network-vif-unplugged", "status": "completed", "tag": "94628a67-e3ab-4037-b7ce-cf51f08c17a6"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:42:10.794860 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-93974f57-9ae1-4da3-ad72-fb0b6a4749e4 req-f89dddce-c415-4187-b218-a3bb76a4bd48 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:42:10.795372 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-93974f57-9ae1-4da3-ad72-fb0b6a4749e4 req-f89dddce-c415-4187-b218-a3bb76a4bd48 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:42:10.806767 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-93974f57-9ae1-4da3-ad72-fb0b6a4749e4 req-f89dddce-c415-4187-b218-a3bb76a4bd48 service nova] Creating event network-vif-unplugged:94628a67-e3ab-4037-b7ce-cf51f08c17a6 for instance 35a74168-3c95-4bfe-b82a-e93b10ec5820 on nch1-master-1 Jul 16 21:42:10.813000 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [req-93974f57-9ae1-4da3-ad72-fb0b6a4749e4 req-f89dddce-c415-4187-b218-a3bb76a4bd48 service nova] Instance 35a74168-3c95-4bfe-b82a-e93b10ec5820 is migrating within the same cell, copying events to all relevant hosts: {'nch2-master-1', 'nch1-master-1'} {{(pid=138680) _get_relevant_hosts /opt/stack/nova/nova/compute/api.py:5807}} Jul 16 21:42:10.823630 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-93974f57-9ae1-4da3-ad72-fb0b6a4749e4 req-f89dddce-c415-4187-b218-a3bb76a4bd48 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.042140 Jul 16 21:42:10.824300 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2776/5553] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:42:10 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:42:11.629716 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-1644a59e-0566-4ed3-b38e-67003d84cd24 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:42:11.631785 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-1644a59e-0566-4ed3-b38e-67003d84cd24 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] [instance: 35a74168-3c95-4bfe-b82a-e93b10ec5820] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:42:11.643149 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1644a59e-0566-4ed3-b38e-67003d84cd24 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:42:11.643149 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1644a59e-0566-4ed3-b38e-67003d84cd24 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:42:11.643684 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1644a59e-0566-4ed3-b38e-67003d84cd24 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:42:11.644139 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1644a59e-0566-4ed3-b38e-67003d84cd24 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:42:11.932098 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-1644a59e-0566-4ed3-b38e-67003d84cd24 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '833136cbdeab4b9e8b6dd2d24c7c8753', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '77c9b7e351e24dc88207e9498f75f578', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:42:11.945702 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-1644a59e-0566-4ed3-b38e-67003d84cd24 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] 10.222.0.16 "GET /compute/v2.1/servers/35a74168-3c95-4bfe-b82a-e93b10ec5820" status: 200 len: 1691 microversion: 2.1 time: 0.319482 Jul 16 21:42:11.946567 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2778/5554] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:42:11 2022] GET /compute/v2.1/servers/35a74168-3c95-4bfe-b82a-e93b10ec5820 => generated 1691 bytes in 322 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:42:12.841267 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-ecdc9a2e-860e-4aff-99bf-d38242a5ea29 req-de9037c1-9126-4008-9425-56ac38c0bb07 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "35a74168-3c95-4bfe-b82a-e93b10ec5820", "tag": "94628a67-e3ab-4037-b7ce-cf51f08c17a6"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:42:12.845284 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_api:SELECT=72,nova_api:INSERT=4 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:42:12.851241 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-ecdc9a2e-860e-4aff-99bf-d38242a5ea29 req-de9037c1-9126-4008-9425-56ac38c0bb07 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:42:12.851691 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-ecdc9a2e-860e-4aff-99bf-d38242a5ea29 req-de9037c1-9126-4008-9425-56ac38c0bb07 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:42:12.866547 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-ecdc9a2e-860e-4aff-99bf-d38242a5ea29 req-de9037c1-9126-4008-9425-56ac38c0bb07 service nova] Creating event network-changed:94628a67-e3ab-4037-b7ce-cf51f08c17a6 for instance 35a74168-3c95-4bfe-b82a-e93b10ec5820 on nch1-master-1 Jul 16 21:42:12.873520 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [req-ecdc9a2e-860e-4aff-99bf-d38242a5ea29 req-de9037c1-9126-4008-9425-56ac38c0bb07 service nova] Instance 35a74168-3c95-4bfe-b82a-e93b10ec5820 is migrating within the same cell, copying events to all relevant hosts: {'nch2-master-1', 'nch1-master-1'} {{(pid=138680) _get_relevant_hosts /opt/stack/nova/nova/compute/api.py:5807}} Jul 16 21:42:12.884647 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-ecdc9a2e-860e-4aff-99bf-d38242a5ea29 req-de9037c1-9126-4008-9425-56ac38c0bb07 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.049122 Jul 16 21:42:12.885272 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2777/5555] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:42:12 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:42:12.975484 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-c6e67c8d-1f29-4704-9bc3-98f065fc8b17 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:42:12.977525 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-c6e67c8d-1f29-4704-9bc3-98f065fc8b17 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] [instance: 35a74168-3c95-4bfe-b82a-e93b10ec5820] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:42:12.987061 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c6e67c8d-1f29-4704-9bc3-98f065fc8b17 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:42:12.987480 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c6e67c8d-1f29-4704-9bc3-98f065fc8b17 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:42:12.988799 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c6e67c8d-1f29-4704-9bc3-98f065fc8b17 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:42:12.989783 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c6e67c8d-1f29-4704-9bc3-98f065fc8b17 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:42:13.224274 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-c6e67c8d-1f29-4704-9bc3-98f065fc8b17 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '833136cbdeab4b9e8b6dd2d24c7c8753', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '77c9b7e351e24dc88207e9498f75f578', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:42:13.254865 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-c6e67c8d-1f29-4704-9bc3-98f065fc8b17 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] 10.222.0.16 "GET /compute/v2.1/servers/35a74168-3c95-4bfe-b82a-e93b10ec5820" status: 200 len: 1691 microversion: 2.1 time: 0.294610 Jul 16 21:42:13.267619 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2779/5556] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:42:12 2022] GET /compute/v2.1/servers/35a74168-3c95-4bfe-b82a-e93b10ec5820 => generated 1691 bytes in 308 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:42:14.282271 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-d1e38a9a-d181-4690-ac14-8c78fbca3727 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:42:14.283391 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-d1e38a9a-d181-4690-ac14-8c78fbca3727 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] [instance: 35a74168-3c95-4bfe-b82a-e93b10ec5820] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:42:14.311710 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d1e38a9a-d181-4690-ac14-8c78fbca3727 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:42:14.312247 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d1e38a9a-d181-4690-ac14-8c78fbca3727 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:42:14.313177 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d1e38a9a-d181-4690-ac14-8c78fbca3727 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:42:14.313699 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d1e38a9a-d181-4690-ac14-8c78fbca3727 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:42:14.582406 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-d1e38a9a-d181-4690-ac14-8c78fbca3727 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '833136cbdeab4b9e8b6dd2d24c7c8753', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '77c9b7e351e24dc88207e9498f75f578', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:42:14.591053 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-d1e38a9a-d181-4690-ac14-8c78fbca3727 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] 10.222.0.16 "GET /compute/v2.1/servers/35a74168-3c95-4bfe-b82a-e93b10ec5820" status: 200 len: 1691 microversion: 2.1 time: 0.318652 Jul 16 21:42:14.591882 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2778/5557] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:42:14 2022] GET /compute/v2.1/servers/35a74168-3c95-4bfe-b82a-e93b10ec5820 => generated 1691 bytes in 321 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:42:15.611486 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-5e8615a4-4d77-4da3-9963-5000c8e25b69 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:42:15.614038 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-5e8615a4-4d77-4da3-9963-5000c8e25b69 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] [instance: 35a74168-3c95-4bfe-b82a-e93b10ec5820] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:42:15.622787 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5e8615a4-4d77-4da3-9963-5000c8e25b69 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:42:15.623652 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5e8615a4-4d77-4da3-9963-5000c8e25b69 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:42:15.629817 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5e8615a4-4d77-4da3-9963-5000c8e25b69 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:42:15.640619 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5e8615a4-4d77-4da3-9963-5000c8e25b69 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.015s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:42:15.855307 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-5e8615a4-4d77-4da3-9963-5000c8e25b69 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '833136cbdeab4b9e8b6dd2d24c7c8753', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '77c9b7e351e24dc88207e9498f75f578', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:42:15.869252 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-5e8615a4-4d77-4da3-9963-5000c8e25b69 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] 10.222.0.16 "GET /compute/v2.1/servers/35a74168-3c95-4bfe-b82a-e93b10ec5820" status: 200 len: 1684 microversion: 2.1 time: 0.263529 Jul 16 21:42:15.869836 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2780/5558] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:42:15 2022] GET /compute/v2.1/servers/35a74168-3c95-4bfe-b82a-e93b10ec5820 => generated 1684 bytes in 265 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:42:18.271877 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-e2d0e32e-7e0b-4d5d-b0b2-8b6becd5f77d req-574b0eb0-55ec-47d7-aac4-89c03cb0c8b6 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "35a74168-3c95-4bfe-b82a-e93b10ec5820", "name": "network-vif-plugged", "status": "completed", "tag": "94628a67-e3ab-4037-b7ce-cf51f08c17a6"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:42:18.281942 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-e2d0e32e-7e0b-4d5d-b0b2-8b6becd5f77d req-574b0eb0-55ec-47d7-aac4-89c03cb0c8b6 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:42:18.282483 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-e2d0e32e-7e0b-4d5d-b0b2-8b6becd5f77d req-574b0eb0-55ec-47d7-aac4-89c03cb0c8b6 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:42:18.297713 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-e2d0e32e-7e0b-4d5d-b0b2-8b6becd5f77d req-574b0eb0-55ec-47d7-aac4-89c03cb0c8b6 service nova] Creating event network-vif-plugged:94628a67-e3ab-4037-b7ce-cf51f08c17a6 for instance 35a74168-3c95-4bfe-b82a-e93b10ec5820 on nch1-master-1 Jul 16 21:42:18.303069 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-e2d0e32e-7e0b-4d5d-b0b2-8b6becd5f77d req-574b0eb0-55ec-47d7-aac4-89c03cb0c8b6 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.036052 Jul 16 21:42:18.303770 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2779/5559] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:42:18 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:42:18.625985 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-f1e03984-8a55-4866-ad04-eac429ff661e tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-server-tempest-TestSecurityGroupsBasicOps-1431728571-access_point-2027899880", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452", "networks": [{"uuid": "0cecca0d-3129-4c8c-89be-f105d25b0557"}], "key_name": "tempest-TestSecurityGroupsBasicOps-1925752067", "security_groups": [{"name": "tempest-secgroup_access--205505560"}, {"name": "tempest-secgroup_general--519926780"}]}, "os:scheduler_hints": {"different_host": []}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:42:18.779198 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.network.neutron [None req-f1e03984-8a55-4866-ad04-eac429ff661e tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] validate_networks() for [('0cecca0d-3129-4c8c-89be-f105d25b0557', None, None, None, None, None)] {{(pid=138681) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 16 21:42:19.290033 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-f1e03984-8a55-4866-ad04-eac429ff661e tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Flavor limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 16 21:42:19.291312 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-f1e03984-8a55-4866-ad04-eac429ff661e tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Image limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 16 21:42:19.292365 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-f1e03984-8a55-4866-ad04-eac429ff661e tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Flavor pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 16 21:42:19.293036 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-f1e03984-8a55-4866-ad04-eac429ff661e tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Image pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 16 21:42:19.294214 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-f1e03984-8a55-4866-ad04-eac429ff661e tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 16 21:42:19.306348 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f1e03984-8a55-4866-ad04-eac429ff661e tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:42:19.307147 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f1e03984-8a55-4866-ad04-eac429ff661e tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:42:19.308902 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f1e03984-8a55-4866-ad04-eac429ff661e tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:42:19.309606 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f1e03984-8a55-4866-ad04-eac429ff661e tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:42:19.343249 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-f1e03984-8a55-4866-ad04-eac429ff661e tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Getting quotas for project 251f01f982aa43ad853d53fa06f99190. Resources: {'ram', 'instances', 'cores'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:42:19.351925 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-f1e03984-8a55-4866-ad04-eac429ff661e tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Getting quotas for user 82819d3efcda4fe78e2b50bb7fab0a96 and project 251f01f982aa43ad853d53fa06f99190. Resources: {'ram', 'instances', 'cores'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:42:19.368228 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-f1e03984-8a55-4866-ad04-eac429ff661e tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Going to run 1 instances... {{(pid=138681) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 16 21:42:19.370354 ncdv-master-1 devstack@n-api.service[138681]: WARNING nova.utils [None req-f1e03984-8a55-4866-ad04-eac429ff661e tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Hostname tempest-server-tempest-TestSecurityGroupsBasicOps-1431728571-access_point-2027899880 is longer than 63, truncate it to tempest-server-tempest-TestSecurityGroupsBasicOps-1431728571-ac Jul 16 21:42:19.371182 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-f1e03984-8a55-4866-ad04-eac429ff661e tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] [instance: 0ec2632e-3da8-44ac-b9ba-7300f5bc1f99] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138681) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 16 21:42:19.529044 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-f1e03984-8a55-4866-ad04-eac429ff661e tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 456 microversion: 2.1 time: 0.906731 Jul 16 21:42:19.543365 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2781/5560] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 21:42:18 2022] POST /compute/v2.1/servers => generated 456 bytes in 922 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 16 21:42:19.546335 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-c8f2f954-18f9-41ce-8363-f3fa6d038948 tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:42:19.547692 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-c8f2f954-18f9-41ce-8363-f3fa6d038948 tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] [instance: 0ec2632e-3da8-44ac-b9ba-7300f5bc1f99] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:42:19.669219 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-c8f2f954-18f9-41ce-8363-f3fa6d038948 tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '82819d3efcda4fe78e2b50bb7fab0a96', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '251f01f982aa43ad853d53fa06f99190', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:42:19.675852 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-c8f2f954-18f9-41ce-8363-f3fa6d038948 tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] 10.222.0.16 "GET /compute/v2.1/servers/0ec2632e-3da8-44ac-b9ba-7300f5bc1f99" status: 200 len: 1295 microversion: 2.1 time: 0.135113 Jul 16 21:42:19.676802 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2780/5561] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:42:19 2022] GET /compute/v2.1/servers/0ec2632e-3da8-44ac-b9ba-7300f5bc1f99 => generated 1295 bytes in 137 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:42:20.695431 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-80aedd94-dbd5-4e95-95a6-b2c0900cd1c0 tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:42:20.696202 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-80aedd94-dbd5-4e95-95a6-b2c0900cd1c0 tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] [instance: 0ec2632e-3da8-44ac-b9ba-7300f5bc1f99] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:42:20.757009 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-80aedd94-dbd5-4e95-95a6-b2c0900cd1c0 tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '82819d3efcda4fe78e2b50bb7fab0a96', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '251f01f982aa43ad853d53fa06f99190', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:42:20.765084 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-80aedd94-dbd5-4e95-95a6-b2c0900cd1c0 tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] 10.222.0.16 "GET /compute/v2.1/servers/0ec2632e-3da8-44ac-b9ba-7300f5bc1f99" status: 200 len: 1295 microversion: 2.1 time: 0.074989 Jul 16 21:42:20.765691 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2782/5562] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:42:20 2022] GET /compute/v2.1/servers/0ec2632e-3da8-44ac-b9ba-7300f5bc1f99 => generated 1295 bytes in 76 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:42:21.782480 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-93d95bb0-bde2-442d-9eee-d4157731a0f4 tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:42:21.783130 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-93d95bb0-bde2-442d-9eee-d4157731a0f4 tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] [instance: 0ec2632e-3da8-44ac-b9ba-7300f5bc1f99] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:42:21.791105 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-93d95bb0-bde2-442d-9eee-d4157731a0f4 tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:42:21.791522 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-93d95bb0-bde2-442d-9eee-d4157731a0f4 tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:42:21.792294 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-93d95bb0-bde2-442d-9eee-d4157731a0f4 tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:42:21.792741 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-93d95bb0-bde2-442d-9eee-d4157731a0f4 tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:42:21.859622 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-93d95bb0-bde2-442d-9eee-d4157731a0f4 tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '82819d3efcda4fe78e2b50bb7fab0a96', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '251f01f982aa43ad853d53fa06f99190', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:42:21.867742 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-93d95bb0-bde2-442d-9eee-d4157731a0f4 tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] 10.222.0.16 "GET /compute/v2.1/servers/0ec2632e-3da8-44ac-b9ba-7300f5bc1f99" status: 200 len: 1347 microversion: 2.1 time: 0.089116 Jul 16 21:42:21.868136 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2781/5563] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:42:21 2022] GET /compute/v2.1/servers/0ec2632e-3da8-44ac-b9ba-7300f5bc1f99 => generated 1347 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:42:22.885837 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-207ef1b9-7b94-4ccd-b5a2-118b14528c2a tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:42:22.886508 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-207ef1b9-7b94-4ccd-b5a2-118b14528c2a tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] [instance: 0ec2632e-3da8-44ac-b9ba-7300f5bc1f99] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:42:22.894919 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-207ef1b9-7b94-4ccd-b5a2-118b14528c2a tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:42:22.895498 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-207ef1b9-7b94-4ccd-b5a2-118b14528c2a tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:42:22.896658 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-207ef1b9-7b94-4ccd-b5a2-118b14528c2a tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:42:22.897203 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-207ef1b9-7b94-4ccd-b5a2-118b14528c2a tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:42:23.097039 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-207ef1b9-7b94-4ccd-b5a2-118b14528c2a tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '82819d3efcda4fe78e2b50bb7fab0a96', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '251f01f982aa43ad853d53fa06f99190', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:42:23.108184 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-207ef1b9-7b94-4ccd-b5a2-118b14528c2a tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] 10.222.0.16 "GET /compute/v2.1/servers/0ec2632e-3da8-44ac-b9ba-7300f5bc1f99" status: 200 len: 1471 microversion: 2.1 time: 0.224272 Jul 16 21:42:23.108995 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2783/5564] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:42:22 2022] GET /compute/v2.1/servers/0ec2632e-3da8-44ac-b9ba-7300f5bc1f99 => generated 1471 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:42:23.934747 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-6b6c694c-d2c9-47e3-babb-7d563cb7b2b7 req-b12a69db-49b1-45a8-a4f1-7b9c66f30244 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "0ec2632e-3da8-44ac-b9ba-7300f5bc1f99", "tag": "96d14176-a9cf-4e6c-a04e-16fc22908cf3"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:42:23.944215 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-6b6c694c-d2c9-47e3-babb-7d563cb7b2b7 req-b12a69db-49b1-45a8-a4f1-7b9c66f30244 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:42:23.950410 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-6b6c694c-d2c9-47e3-babb-7d563cb7b2b7 req-b12a69db-49b1-45a8-a4f1-7b9c66f30244 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:42:23.963533 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-6b6c694c-d2c9-47e3-babb-7d563cb7b2b7 req-b12a69db-49b1-45a8-a4f1-7b9c66f30244 service nova] Creating event network-changed:96d14176-a9cf-4e6c-a04e-16fc22908cf3 for instance 0ec2632e-3da8-44ac-b9ba-7300f5bc1f99 on nch2-master-1 Jul 16 21:42:23.969178 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-6b6c694c-d2c9-47e3-babb-7d563cb7b2b7 req-b12a69db-49b1-45a8-a4f1-7b9c66f30244 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.038329 Jul 16 21:42:23.969912 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2782/5565] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:42:23 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:42:24.126164 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-8402398a-1ada-4c69-b76c-5ffbff433edf tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:42:24.130480 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-8402398a-1ada-4c69-b76c-5ffbff433edf tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] [instance: 0ec2632e-3da8-44ac-b9ba-7300f5bc1f99] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:42:24.133434 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_api:SELECT=63,nova_api:INSERT=3 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:42:24.139112 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8402398a-1ada-4c69-b76c-5ffbff433edf tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:42:24.144460 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8402398a-1ada-4c69-b76c-5ffbff433edf tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:42:24.144460 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8402398a-1ada-4c69-b76c-5ffbff433edf tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:42:24.144460 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8402398a-1ada-4c69-b76c-5ffbff433edf tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:42:24.368938 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-8402398a-1ada-4c69-b76c-5ffbff433edf tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '82819d3efcda4fe78e2b50bb7fab0a96', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '251f01f982aa43ad853d53fa06f99190', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:42:24.378325 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-8402398a-1ada-4c69-b76c-5ffbff433edf tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] 10.222.0.16 "GET /compute/v2.1/servers/0ec2632e-3da8-44ac-b9ba-7300f5bc1f99" status: 200 len: 1471 microversion: 2.1 time: 0.256264 Jul 16 21:42:24.379408 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2784/5566] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:42:24 2022] GET /compute/v2.1/servers/0ec2632e-3da8-44ac-b9ba-7300f5bc1f99 => generated 1471 bytes in 258 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:42:25.399106 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-25eb98fc-0dd5-4426-a605-9ea0c9b2182e tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:42:25.400232 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-25eb98fc-0dd5-4426-a605-9ea0c9b2182e tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] [instance: 0ec2632e-3da8-44ac-b9ba-7300f5bc1f99] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:42:25.409438 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-25eb98fc-0dd5-4426-a605-9ea0c9b2182e tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:42:25.417062 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-25eb98fc-0dd5-4426-a605-9ea0c9b2182e tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:42:25.417062 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-25eb98fc-0dd5-4426-a605-9ea0c9b2182e tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:42:25.417062 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-25eb98fc-0dd5-4426-a605-9ea0c9b2182e tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:42:25.417062 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=97 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:42:25.951936 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-25eb98fc-0dd5-4426-a605-9ea0c9b2182e tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '82819d3efcda4fe78e2b50bb7fab0a96', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '251f01f982aa43ad853d53fa06f99190', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:42:25.961648 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-25eb98fc-0dd5-4426-a605-9ea0c9b2182e tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] 10.222.0.16 "GET /compute/v2.1/servers/0ec2632e-3da8-44ac-b9ba-7300f5bc1f99" status: 200 len: 1471 microversion: 2.1 time: 0.568765 Jul 16 21:42:25.962518 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2783/5567] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:42:25 2022] GET /compute/v2.1/servers/0ec2632e-3da8-44ac-b9ba-7300f5bc1f99 => generated 1471 bytes in 571 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:42:26.982282 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-2ab1e928-eb6b-442b-9c84-c3f28ff87113 tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:42:26.982823 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-2ab1e928-eb6b-442b-9c84-c3f28ff87113 tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] [instance: 0ec2632e-3da8-44ac-b9ba-7300f5bc1f99] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:42:26.990098 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2ab1e928-eb6b-442b-9c84-c3f28ff87113 tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:42:26.990541 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2ab1e928-eb6b-442b-9c84-c3f28ff87113 tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:42:26.991437 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2ab1e928-eb6b-442b-9c84-c3f28ff87113 tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:42:26.991883 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2ab1e928-eb6b-442b-9c84-c3f28ff87113 tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:42:27.143208 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-2ab1e928-eb6b-442b-9c84-c3f28ff87113 tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '82819d3efcda4fe78e2b50bb7fab0a96', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '251f01f982aa43ad853d53fa06f99190', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:42:27.151165 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-2ab1e928-eb6b-442b-9c84-c3f28ff87113 tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] 10.222.0.16 "GET /compute/v2.1/servers/0ec2632e-3da8-44ac-b9ba-7300f5bc1f99" status: 200 len: 1471 microversion: 2.1 time: 0.174070 Jul 16 21:42:27.151748 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2785/5568] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:42:26 2022] GET /compute/v2.1/servers/0ec2632e-3da8-44ac-b9ba-7300f5bc1f99 => generated 1471 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:42:28.169688 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-0d19822c-8cc5-4f37-95ad-f366ab7e301f tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:42:28.170435 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-0d19822c-8cc5-4f37-95ad-f366ab7e301f tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] [instance: 0ec2632e-3da8-44ac-b9ba-7300f5bc1f99] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:42:28.177804 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0d19822c-8cc5-4f37-95ad-f366ab7e301f tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:42:28.180404 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0d19822c-8cc5-4f37-95ad-f366ab7e301f tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:42:28.180404 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0d19822c-8cc5-4f37-95ad-f366ab7e301f tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:42:28.180404 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0d19822c-8cc5-4f37-95ad-f366ab7e301f tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:42:28.324514 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-0d19822c-8cc5-4f37-95ad-f366ab7e301f tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '82819d3efcda4fe78e2b50bb7fab0a96', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '251f01f982aa43ad853d53fa06f99190', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:42:28.332143 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-0d19822c-8cc5-4f37-95ad-f366ab7e301f tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] 10.222.0.16 "GET /compute/v2.1/servers/0ec2632e-3da8-44ac-b9ba-7300f5bc1f99" status: 200 len: 1471 microversion: 2.1 time: 0.167262 Jul 16 21:42:28.332724 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2784/5569] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:42:28 2022] GET /compute/v2.1/servers/0ec2632e-3da8-44ac-b9ba-7300f5bc1f99 => generated 1471 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:42:29.348804 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-8e115e8d-4a18-41d0-9a4e-e32eb6e0c96b tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:42:29.349465 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-8e115e8d-4a18-41d0-9a4e-e32eb6e0c96b tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] [instance: 0ec2632e-3da8-44ac-b9ba-7300f5bc1f99] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:42:29.352234 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell0:SELECT=2 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:42:29.358794 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8e115e8d-4a18-41d0-9a4e-e32eb6e0c96b tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:42:29.359308 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8e115e8d-4a18-41d0-9a4e-e32eb6e0c96b tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:42:29.360104 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8e115e8d-4a18-41d0-9a4e-e32eb6e0c96b tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:42:29.360840 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8e115e8d-4a18-41d0-9a4e-e32eb6e0c96b tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:42:29.503441 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-8e115e8d-4a18-41d0-9a4e-e32eb6e0c96b tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '82819d3efcda4fe78e2b50bb7fab0a96', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '251f01f982aa43ad853d53fa06f99190', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:42:29.511169 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-8e115e8d-4a18-41d0-9a4e-e32eb6e0c96b tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] 10.222.0.16 "GET /compute/v2.1/servers/0ec2632e-3da8-44ac-b9ba-7300f5bc1f99" status: 200 len: 1471 microversion: 2.1 time: 0.166118 Jul 16 21:42:29.511748 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2786/5570] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:42:29 2022] GET /compute/v2.1/servers/0ec2632e-3da8-44ac-b9ba-7300f5bc1f99 => generated 1471 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:42:29.766627 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-5e223735-f30b-4847-94fe-ec220208e66d req-57f6e2c8-777f-4524-bdf9-1df9f1b73d22 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "0ec2632e-3da8-44ac-b9ba-7300f5bc1f99", "name": "network-vif-plugged", "status": "completed", "tag": "96d14176-a9cf-4e6c-a04e-16fc22908cf3"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:42:29.772999 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell0:SELECT=6 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:42:29.776387 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-5e223735-f30b-4847-94fe-ec220208e66d req-57f6e2c8-777f-4524-bdf9-1df9f1b73d22 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:42:29.776835 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-5e223735-f30b-4847-94fe-ec220208e66d req-57f6e2c8-777f-4524-bdf9-1df9f1b73d22 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:42:29.787828 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-5e223735-f30b-4847-94fe-ec220208e66d req-57f6e2c8-777f-4524-bdf9-1df9f1b73d22 service nova] Creating event network-vif-plugged:96d14176-a9cf-4e6c-a04e-16fc22908cf3 for instance 0ec2632e-3da8-44ac-b9ba-7300f5bc1f99 on nch2-master-1 Jul 16 21:42:29.793252 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-5e223735-f30b-4847-94fe-ec220208e66d req-57f6e2c8-777f-4524-bdf9-1df9f1b73d22 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.031098 Jul 16 21:42:29.796153 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2785/5571] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:42:29 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:42:30.527952 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-5f78db2b-6746-44c3-976a-8f30b2a334e0 tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:42:30.528916 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-5f78db2b-6746-44c3-976a-8f30b2a334e0 tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] [instance: 0ec2632e-3da8-44ac-b9ba-7300f5bc1f99] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:42:30.535882 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5f78db2b-6746-44c3-976a-8f30b2a334e0 tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:42:30.536348 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5f78db2b-6746-44c3-976a-8f30b2a334e0 tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:42:30.537123 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5f78db2b-6746-44c3-976a-8f30b2a334e0 tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:42:30.537570 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5f78db2b-6746-44c3-976a-8f30b2a334e0 tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:42:30.678900 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-5f78db2b-6746-44c3-976a-8f30b2a334e0 tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '82819d3efcda4fe78e2b50bb7fab0a96', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '251f01f982aa43ad853d53fa06f99190', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:42:30.687064 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-5f78db2b-6746-44c3-976a-8f30b2a334e0 tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] 10.222.0.16 "GET /compute/v2.1/servers/0ec2632e-3da8-44ac-b9ba-7300f5bc1f99" status: 200 len: 1471 microversion: 2.1 time: 0.162698 Jul 16 21:42:30.690059 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2787/5572] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:42:30 2022] GET /compute/v2.1/servers/0ec2632e-3da8-44ac-b9ba-7300f5bc1f99 => generated 1471 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:42:31.706063 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-7f97d6a9-ca30-45a4-a4b5-77a8969fe846 tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:42:31.706814 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-7f97d6a9-ca30-45a4-a4b5-77a8969fe846 tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] [instance: 0ec2632e-3da8-44ac-b9ba-7300f5bc1f99] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:42:31.713704 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7f97d6a9-ca30-45a4-a4b5-77a8969fe846 tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:42:31.715113 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7f97d6a9-ca30-45a4-a4b5-77a8969fe846 tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:42:31.715113 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7f97d6a9-ca30-45a4-a4b5-77a8969fe846 tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:42:31.715394 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7f97d6a9-ca30-45a4-a4b5-77a8969fe846 tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:42:31.858691 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-7f97d6a9-ca30-45a4-a4b5-77a8969fe846 tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '82819d3efcda4fe78e2b50bb7fab0a96', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '251f01f982aa43ad853d53fa06f99190', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:42:31.866533 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-7f97d6a9-ca30-45a4-a4b5-77a8969fe846 tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] 10.222.0.16 "GET /compute/v2.1/servers/0ec2632e-3da8-44ac-b9ba-7300f5bc1f99" status: 200 len: 1642 microversion: 2.1 time: 0.166179 Jul 16 21:42:31.868927 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2786/5573] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:42:31 2022] GET /compute/v2.1/servers/0ec2632e-3da8-44ac-b9ba-7300f5bc1f99 => generated 1642 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:42:31.882553 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-e5a55048-f5e6-40fc-8f02-c7835bdbf264 tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:42:31.883217 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-e5a55048-f5e6-40fc-8f02-c7835bdbf264 tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] [instance: 0ec2632e-3da8-44ac-b9ba-7300f5bc1f99] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:42:31.885161 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell0:SELECT=8 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:42:31.891576 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e5a55048-f5e6-40fc-8f02-c7835bdbf264 tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:42:31.892053 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e5a55048-f5e6-40fc-8f02-c7835bdbf264 tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:42:31.892873 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e5a55048-f5e6-40fc-8f02-c7835bdbf264 tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:42:31.893329 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e5a55048-f5e6-40fc-8f02-c7835bdbf264 tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:42:32.038824 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-e5a55048-f5e6-40fc-8f02-c7835bdbf264 tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '82819d3efcda4fe78e2b50bb7fab0a96', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '251f01f982aa43ad853d53fa06f99190', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:42:32.046473 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-e5a55048-f5e6-40fc-8f02-c7835bdbf264 tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] 10.222.0.16 "GET /compute/v2.1/servers/0ec2632e-3da8-44ac-b9ba-7300f5bc1f99" status: 200 len: 1642 microversion: 2.1 time: 0.167758 Jul 16 21:42:32.047113 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2788/5574] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:42:31 2022] GET /compute/v2.1/servers/0ec2632e-3da8-44ac-b9ba-7300f5bc1f99 => generated 1642 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:42:32.063228 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-223c4a37-053b-481f-978e-c906cbe404ed tempest-TestSecurityGroupsBasicOps-1387468095 tempest-TestSecurityGroupsBasicOps-1387468095-project-admin] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:42:32.063850 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-223c4a37-053b-481f-978e-c906cbe404ed tempest-TestSecurityGroupsBasicOps-1387468095 tempest-TestSecurityGroupsBasicOps-1387468095-project-admin] [instance: 0ec2632e-3da8-44ac-b9ba-7300f5bc1f99] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:42:32.070308 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-223c4a37-053b-481f-978e-c906cbe404ed tempest-TestSecurityGroupsBasicOps-1387468095 tempest-TestSecurityGroupsBasicOps-1387468095-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:42:32.070776 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-223c4a37-053b-481f-978e-c906cbe404ed tempest-TestSecurityGroupsBasicOps-1387468095 tempest-TestSecurityGroupsBasicOps-1387468095-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:42:32.071585 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-223c4a37-053b-481f-978e-c906cbe404ed tempest-TestSecurityGroupsBasicOps-1387468095 tempest-TestSecurityGroupsBasicOps-1387468095-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:42:32.072041 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-223c4a37-053b-481f-978e-c906cbe404ed tempest-TestSecurityGroupsBasicOps-1387468095 tempest-TestSecurityGroupsBasicOps-1387468095-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:42:32.378978 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-223c4a37-053b-481f-978e-c906cbe404ed tempest-TestSecurityGroupsBasicOps-1387468095 tempest-TestSecurityGroupsBasicOps-1387468095-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/0ec2632e-3da8-44ac-b9ba-7300f5bc1f99" status: 200 len: 1793 microversion: 2.1 time: 0.320463 Jul 16 21:42:32.383091 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2787/5575] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:42:32 2022] GET /compute/v2.1/servers/0ec2632e-3da8-44ac-b9ba-7300f5bc1f99 => generated 1793 bytes in 322 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:42:34.440623 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-420b497c-30fb-4287-a060-7b725ad406ff req-82edadf3-5328-4b03-ae45-27a2f027ae47 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "0ec2632e-3da8-44ac-b9ba-7300f5bc1f99", "tag": "96d14176-a9cf-4e6c-a04e-16fc22908cf3"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:42:34.451781 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-420b497c-30fb-4287-a060-7b725ad406ff req-82edadf3-5328-4b03-ae45-27a2f027ae47 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:42:34.452625 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-420b497c-30fb-4287-a060-7b725ad406ff req-82edadf3-5328-4b03-ae45-27a2f027ae47 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:42:34.465653 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-420b497c-30fb-4287-a060-7b725ad406ff req-82edadf3-5328-4b03-ae45-27a2f027ae47 service nova] Creating event network-changed:96d14176-a9cf-4e6c-a04e-16fc22908cf3 for instance 0ec2632e-3da8-44ac-b9ba-7300f5bc1f99 on nch2-master-1 Jul 16 21:42:34.472234 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-420b497c-30fb-4287-a060-7b725ad406ff req-82edadf3-5328-4b03-ae45-27a2f027ae47 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.036301 Jul 16 21:42:34.473265 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2789/5576] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:42:34 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:42:40.135831 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-719cffa1-e591-4ce5-a127-5f5d45077949 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Action: 'action', calling method: >, body: {"os-getConsoleOutput": {}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:42:40.136681 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-719cffa1-e591-4ce5-a127-5f5d45077949 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] [instance: 35a74168-3c95-4bfe-b82a-e93b10ec5820] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:42:40.143809 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-719cffa1-e591-4ce5-a127-5f5d45077949 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:42:40.145090 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-719cffa1-e591-4ce5-a127-5f5d45077949 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:42:40.145090 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-719cffa1-e591-4ce5-a127-5f5d45077949 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:42:40.145307 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-719cffa1-e591-4ce5-a127-5f5d45077949 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:42:40.207884 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-719cffa1-e591-4ce5-a127-5f5d45077949 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] 10.222.0.16 "POST /compute/v2.1/servers/35a74168-3c95-4bfe-b82a-e93b10ec5820/action" status: 200 len: 63096 microversion: 2.1 time: 0.075921 Jul 16 21:42:40.208561 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2788/5577] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 21:42:40 2022] POST /compute/v2.1/servers/35a74168-3c95-4bfe-b82a-e93b10ec5820/action => generated 63096 bytes in 78 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 16 21:42:46.419904 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_api:SELECT=11 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:42:46.420585 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=63 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:42:46.424497 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-85aec54f-6437-4841-81bd-34bec70e40b1 tempest-TestNetworkAdvancedServerOps-756251536 tempest-TestNetworkAdvancedServerOps-756251536-project-admin] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:42:46.425258 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-85aec54f-6437-4841-81bd-34bec70e40b1 tempest-TestNetworkAdvancedServerOps-756251536 tempest-TestNetworkAdvancedServerOps-756251536-project-admin] [instance: 35a74168-3c95-4bfe-b82a-e93b10ec5820] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:42:46.431550 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-85aec54f-6437-4841-81bd-34bec70e40b1 tempest-TestNetworkAdvancedServerOps-756251536 tempest-TestNetworkAdvancedServerOps-756251536-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:42:46.432010 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-85aec54f-6437-4841-81bd-34bec70e40b1 tempest-TestNetworkAdvancedServerOps-756251536 tempest-TestNetworkAdvancedServerOps-756251536-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:42:46.432790 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-85aec54f-6437-4841-81bd-34bec70e40b1 tempest-TestNetworkAdvancedServerOps-756251536 tempest-TestNetworkAdvancedServerOps-756251536-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:42:46.433237 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-85aec54f-6437-4841-81bd-34bec70e40b1 tempest-TestNetworkAdvancedServerOps-756251536 tempest-TestNetworkAdvancedServerOps-756251536-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:42:46.565572 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-85aec54f-6437-4841-81bd-34bec70e40b1 tempest-TestNetworkAdvancedServerOps-756251536 tempest-TestNetworkAdvancedServerOps-756251536-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/35a74168-3c95-4bfe-b82a-e93b10ec5820" status: 200 len: 1835 microversion: 2.1 time: 0.147794 Jul 16 21:42:46.566150 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2790/5578] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:42:46 2022] GET /compute/v2.1/servers/35a74168-3c95-4bfe-b82a-e93b10ec5820 => generated 1835 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:42:47.949059 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-30131df5-b9b8-4e5e-9849-a92527e5fcf5 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:42:47.949810 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-30131df5-b9b8-4e5e-9849-a92527e5fcf5 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] [instance: 35a74168-3c95-4bfe-b82a-e93b10ec5820] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:42:47.949999 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-c7a34c5d-a36e-4faf-8b73-106f404a2c20 req-17b3283f-2575-4054-98bd-1ad4560bdff2 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "35a74168-3c95-4bfe-b82a-e93b10ec5820", "tag": "94628a67-e3ab-4037-b7ce-cf51f08c17a6"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:42:47.957979 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-30131df5-b9b8-4e5e-9849-a92527e5fcf5 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:42:47.961512 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-30131df5-b9b8-4e5e-9849-a92527e5fcf5 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:42:47.961637 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-c7a34c5d-a36e-4faf-8b73-106f404a2c20 req-17b3283f-2575-4054-98bd-1ad4560bdff2 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:42:47.961637 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-c7a34c5d-a36e-4faf-8b73-106f404a2c20 req-17b3283f-2575-4054-98bd-1ad4560bdff2 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:42:47.961831 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-30131df5-b9b8-4e5e-9849-a92527e5fcf5 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:42:47.961831 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-30131df5-b9b8-4e5e-9849-a92527e5fcf5 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:42:47.973163 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-c7a34c5d-a36e-4faf-8b73-106f404a2c20 req-17b3283f-2575-4054-98bd-1ad4560bdff2 service nova] Creating event network-changed:94628a67-e3ab-4037-b7ce-cf51f08c17a6 for instance 35a74168-3c95-4bfe-b82a-e93b10ec5820 on nch1-master-1 Jul 16 21:42:47.978463 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-c7a34c5d-a36e-4faf-8b73-106f404a2c20 req-17b3283f-2575-4054-98bd-1ad4560bdff2 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.032484 Jul 16 21:42:47.979759 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2791/5579] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:42:47 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 16 21:42:47.987599 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-30131df5-b9b8-4e5e-9849-a92527e5fcf5 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] [instance: 35a74168-3c95-4bfe-b82a-e93b10ec5820] Going to try to terminate instance {{(pid=138680) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 16 21:42:48.257745 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-30131df5-b9b8-4e5e-9849-a92527e5fcf5 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] 10.222.0.16 "DELETE /compute/v2.1/servers/35a74168-3c95-4bfe-b82a-e93b10ec5820" status: 204 len: 0 microversion: 2.1 time: 0.314498 Jul 16 21:42:48.261581 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2789/5580] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:42:47 2022] DELETE /compute/v2.1/servers/35a74168-3c95-4bfe-b82a-e93b10ec5820 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 16 21:42:48.267430 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-aed0c000-4a1c-488f-8a5d-8e9071149cdc tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:42:48.268171 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-aed0c000-4a1c-488f-8a5d-8e9071149cdc tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] [instance: 35a74168-3c95-4bfe-b82a-e93b10ec5820] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:42:48.276226 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-aed0c000-4a1c-488f-8a5d-8e9071149cdc tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:42:48.276944 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-aed0c000-4a1c-488f-8a5d-8e9071149cdc tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:42:48.277739 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-aed0c000-4a1c-488f-8a5d-8e9071149cdc tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:42:48.278222 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-aed0c000-4a1c-488f-8a5d-8e9071149cdc tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:42:48.423660 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-aed0c000-4a1c-488f-8a5d-8e9071149cdc tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '833136cbdeab4b9e8b6dd2d24c7c8753', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '77c9b7e351e24dc88207e9498f75f578', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:42:48.432317 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-aed0c000-4a1c-488f-8a5d-8e9071149cdc tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] 10.222.0.16 "GET /compute/v2.1/servers/35a74168-3c95-4bfe-b82a-e93b10ec5820" status: 200 len: 1690 microversion: 2.1 time: 0.169119 Jul 16 21:42:48.433006 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2792/5581] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:42:48 2022] GET /compute/v2.1/servers/35a74168-3c95-4bfe-b82a-e93b10ec5820 => generated 1690 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:42:49.455273 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-0872325d-1a88-4cd9-b02b-31f7c1991539 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:42:49.455986 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-0872325d-1a88-4cd9-b02b-31f7c1991539 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] [instance: 35a74168-3c95-4bfe-b82a-e93b10ec5820] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:42:49.459884 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_api:SELECT=33,nova_api:UPDATE=1 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:42:49.465809 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0872325d-1a88-4cd9-b02b-31f7c1991539 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:42:49.466268 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0872325d-1a88-4cd9-b02b-31f7c1991539 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:42:49.467089 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0872325d-1a88-4cd9-b02b-31f7c1991539 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:42:49.467540 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0872325d-1a88-4cd9-b02b-31f7c1991539 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:42:49.707045 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-0872325d-1a88-4cd9-b02b-31f7c1991539 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '833136cbdeab4b9e8b6dd2d24c7c8753', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '77c9b7e351e24dc88207e9498f75f578', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:42:49.715182 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-0872325d-1a88-4cd9-b02b-31f7c1991539 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] 10.222.0.16 "GET /compute/v2.1/servers/35a74168-3c95-4bfe-b82a-e93b10ec5820" status: 200 len: 1690 microversion: 2.1 time: 0.268001 Jul 16 21:42:49.722657 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2790/5582] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:42:49 2022] GET /compute/v2.1/servers/35a74168-3c95-4bfe-b82a-e93b10ec5820 => generated 1690 bytes in 270 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:42:50.155727 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-181e341c-e745-47a6-a34c-c332e62f0b2f req-e63fdeb4-893d-4f39-b346-88953a134dd8 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "35a74168-3c95-4bfe-b82a-e93b10ec5820", "name": "network-vif-deleted", "tag": "94628a67-e3ab-4037-b7ce-cf51f08c17a6"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:42:50.165582 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-181e341c-e745-47a6-a34c-c332e62f0b2f req-e63fdeb4-893d-4f39-b346-88953a134dd8 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:42:50.166039 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-181e341c-e745-47a6-a34c-c332e62f0b2f req-e63fdeb4-893d-4f39-b346-88953a134dd8 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:42:50.177452 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-181e341c-e745-47a6-a34c-c332e62f0b2f req-e63fdeb4-893d-4f39-b346-88953a134dd8 service nova] Creating event network-vif-deleted:94628a67-e3ab-4037-b7ce-cf51f08c17a6 for instance 35a74168-3c95-4bfe-b82a-e93b10ec5820 on nch1-master-1 Jul 16 21:42:50.183288 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-181e341c-e745-47a6-a34c-c332e62f0b2f req-e63fdeb4-893d-4f39-b346-88953a134dd8 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.032058 Jul 16 21:42:50.184038 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2793/5583] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:42:50 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 16 21:42:50.733846 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-274c193e-e273-4fbd-adb0-5717ed6d7f07 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:42:50.734464 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-274c193e-e273-4fbd-adb0-5717ed6d7f07 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] [instance: 35a74168-3c95-4bfe-b82a-e93b10ec5820] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:42:50.742167 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-274c193e-e273-4fbd-adb0-5717ed6d7f07 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:42:50.742651 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-274c193e-e273-4fbd-adb0-5717ed6d7f07 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:42:50.743455 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-274c193e-e273-4fbd-adb0-5717ed6d7f07 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:42:50.743900 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-274c193e-e273-4fbd-adb0-5717ed6d7f07 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:42:50.776083 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-274c193e-e273-4fbd-adb0-5717ed6d7f07 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:42:50.776541 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-274c193e-e273-4fbd-adb0-5717ed6d7f07 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:42:50.776997 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.objects.instance [None req-274c193e-e273-4fbd-adb0-5717ed6d7f07 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lazy-loading 'fault' on Instance uuid 35a74168-3c95-4bfe-b82a-e93b10ec5820 {{(pid=138680) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:42:50.818138 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-274c193e-e273-4fbd-adb0-5717ed6d7f07 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '833136cbdeab4b9e8b6dd2d24c7c8753', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '77c9b7e351e24dc88207e9498f75f578', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:42:50.826546 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-274c193e-e273-4fbd-adb0-5717ed6d7f07 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] 10.222.0.16 "GET /compute/v2.1/servers/35a74168-3c95-4bfe-b82a-e93b10ec5820" status: 200 len: 1357 microversion: 2.1 time: 0.096788 Jul 16 21:42:50.827154 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2791/5584] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:42:50 2022] GET /compute/v2.1/servers/35a74168-3c95-4bfe-b82a-e93b10ec5820 => generated 1357 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:42:51.845952 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-b9f8d0f0-f17c-4ccd-8b1e-54396f70ea96 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:42:51.846701 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-b9f8d0f0-f17c-4ccd-8b1e-54396f70ea96 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] [instance: 35a74168-3c95-4bfe-b82a-e93b10ec5820] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:42:51.853154 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b9f8d0f0-f17c-4ccd-8b1e-54396f70ea96 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:42:51.854883 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b9f8d0f0-f17c-4ccd-8b1e-54396f70ea96 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:42:51.854883 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b9f8d0f0-f17c-4ccd-8b1e-54396f70ea96 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:42:51.854883 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b9f8d0f0-f17c-4ccd-8b1e-54396f70ea96 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:42:51.869307 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-b9f8d0f0-f17c-4ccd-8b1e-54396f70ea96 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] HTTP exception thrown: Instance 35a74168-3c95-4bfe-b82a-e93b10ec5820 could not be found. Jul 16 21:42:51.870975 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-b9f8d0f0-f17c-4ccd-8b1e-54396f70ea96 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Request method failure captured: Jul 16 21:42:51.870975 ncdv-master-1 devstack@n-api.service[138681]: request: GET /compute/v2.1/servers/35a74168-3c95-4bfe-b82a-e93b10ec5820 HTTP/1.1 Jul 16 21:42:51.870975 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:42:51.870975 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:42:51.870975 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:42:51.870975 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 21:42:51.870975 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:42:51.870975 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:42:51.870975 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:42:51.870975 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:42:51.870975 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0y_J0q_hylsRBv1tuKDuvEX4h0-KJOaBkYlsP5Ic98XDN3p0UKvwASS-7n2zGgAIPr7rp74I-PGAzLIn5xeDmpiRzwuaSoKYKWvZ4y05aUl8opleXxkVzk0FUY6GEflOf-1AEqT9qASXsqi7YhmGfOHelDZ7TomdNPlQ8IF_8wXqbKY Jul 16 21:42:51.870975 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:42:51.870975 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:42:51.870975 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:42:51.870975 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:42:51.870975 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:42:51.870975 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:42:51.870975 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: 77c9b7e351e24dc88207e9498f75f578 Jul 16 21:42:51.870975 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-TestNetworkAdvancedServerOps-375332741 Jul 16 21:42:51.870975 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:42:51.870975 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:42:51.870975 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/77c9b7e351e24dc88207e9498f75f578"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/77c9b7e351e24dc88207e9498f75f578"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_77c9b7e351e24dc88207e9498f75f578"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/77c9b7e351e24dc88207e9498f75f578"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:42:51.870975 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-TestNetworkAdvancedServerOps-375332741 Jul 16 21:42:51.870975 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: 77c9b7e351e24dc88207e9498f75f578 Jul 16 21:42:51.870975 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-TestNetworkAdvancedServerOps-375332741 Jul 16 21:42:51.870975 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-TestNetworkAdvancedServerOps-375332741-project Jul 16 21:42:51.870975 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:42:51.870975 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:42:51.870975 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: 833136cbdeab4b9e8b6dd2d24c7c8753 Jul 16 21:42:51.870975 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-TestNetworkAdvancedServerOps-375332741-project Jul 16 21:42:51.870975 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:42:51.870975 ncdv-master-1 devstack@n-api.service[138681]: exception: Instance 35a74168-3c95-4bfe-b82a-e93b10ec5820 could not be found. Jul 16 21:42:51.870975 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': '35a74168-3c95-4bfe-b82a-e93b10ec5820'} Jul 16 21:42:51.870975 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:42:51.870975 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:42:51.870975 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:42:51.870975 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:42:51.870975 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:42:51.870975 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:42:51.870975 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:42:51.870975 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:42:51.870975 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:42:51.870975 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:42:51.870975 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 35a74168-3c95-4bfe-b82a-e93b10ec5820 could not be found. Jul 16 21:42:51.870975 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:42:51.870975 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:42:51.870975 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:42:51.870975 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:42:51.870975 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:42:51.870975 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:42:51.870975 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:42:51.870975 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:42:51.870975 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:42:51.870975 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:42:51.870975 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:42:51.870975 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:42:51.870975 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:42:51.870975 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:42:51.870975 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:42:51.870975 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:42:51.870975 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 35a74168-3c95-4bfe-b82a-e93b10ec5820 could not be found. Jul 16 21:42:51.870975 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:42:51.870975 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:42:51.870975 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:42:51.875686 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:42:51.875686 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:42:51.875686 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:42:51.875686 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:42:51.875686 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:42:51.875686 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 35a74168-3c95-4bfe-b82a-e93b10ec5820 could not be found. Jul 16 21:42:51.875686 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:42:51.875686 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-b9f8d0f0-f17c-4ccd-8b1e-54396f70ea96 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Returning 404 to user: Instance 35a74168-3c95-4bfe-b82a-e93b10ec5820 could not be found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:42:51.875686 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-b9f8d0f0-f17c-4ccd-8b1e-54396f70ea96 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] 10.222.0.16 "GET /compute/v2.1/servers/35a74168-3c95-4bfe-b82a-e93b10ec5820" status: 404 len: 111 microversion: 2.1 time: 0.031595 Jul 16 21:42:51.875686 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2794/5585] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:42:51 2022] GET /compute/v2.1/servers/35a74168-3c95-4bfe-b82a-e93b10ec5820 => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:42:56.015775 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-f1d63a34-ccfb-4245-9ea2-862871e0a207 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Calling method '.version_select at 0x7fdd8fdbda60>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:42:56.097039 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-f1d63a34-ccfb-4245-9ea2-862871e0a207 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] 10.222.0.16 "DELETE /compute/v2.1/os-keypairs/tempest-TestNetworkAdvancedServerOps-1439709083" status: 202 len: 0 microversion: 2.1 time: 0.085130 Jul 16 21:42:56.097853 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2792/5586] 10.222.0.16 () {58 vars in 1283 bytes} [Sat Jul 16 21:42:56 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestNetworkAdvancedServerOps-1439709083 => generated 0 bytes in 87 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 21:42:56.136512 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-d8ba3488-354f-4a2e-be6c-1110c70d9399 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Action: 'create', calling method: .version_select at 0x7fdd8c51fd30>, body: {"keypair": {"name": "tempest-TestNetworkAdvancedServerOps-343919104", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBBgWe7fKqWFU+FkpDPkZFTHjwXZy+QrehBYbJ0fEMcaOWMiUhVyvP36C8kxaY/c6ZaLqhgitgsqb1F3sp7Vlli6/CV5TsIZvj8B0OFBLjFGDPmrI5aXugRNQxxMVsWir6g=="}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:42:56.152249 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-d8ba3488-354f-4a2e-be6c-1110c70d9399 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Getting quotas for project 77c9b7e351e24dc88207e9498f75f578. Resources: {'key_pairs'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:42:56.158192 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-d8ba3488-354f-4a2e-be6c-1110c70d9399 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Getting quotas for user 833136cbdeab4b9e8b6dd2d24c7c8753 and project 77c9b7e351e24dc88207e9498f75f578. Resources: {'key_pairs'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:42:56.269215 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-d8ba3488-354f-4a2e-be6c-1110c70d9399 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] 10.222.0.16 "POST /compute/v2.1/os-keypairs" status: 200 len: 406 microversion: 2.1 time: 0.136431 Jul 16 21:42:56.269745 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2795/5587] 10.222.0.16 () {60 vars in 1158 bytes} [Sat Jul 16 21:42:56 2022] POST /compute/v2.1/os-keypairs => generated 406 bytes in 138 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:42:57.819912 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-16ba8be2-64e9-4bdf-98ba-860ab48fd4d1 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:42:57.820423 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-16ba8be2-64e9-4bdf-98ba-860ab48fd4d1 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] [instance: 97d5b8fb-ce56-41ba-b786-5f8474c9b2f3] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:42:57.828038 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-16ba8be2-64e9-4bdf-98ba-860ab48fd4d1 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:42:57.828493 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-16ba8be2-64e9-4bdf-98ba-860ab48fd4d1 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:42:57.829269 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-16ba8be2-64e9-4bdf-98ba-860ab48fd4d1 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:42:57.829710 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-16ba8be2-64e9-4bdf-98ba-860ab48fd4d1 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:42:57.832333 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=58,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:42:57.916276 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.objects.instance [None req-16ba8be2-64e9-4bdf-98ba-860ab48fd4d1 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Lazy-loading 'services' on Instance uuid 97d5b8fb-ce56-41ba-b786-5f8474c9b2f3 {{(pid=138680) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:42:58.176424 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-16ba8be2-64e9-4bdf-98ba-860ab48fd4d1 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] 10.222.0.16 "DELETE /compute/v2.1/servers/97d5b8fb-ce56-41ba-b786-5f8474c9b2f3/os-volume_attachments/f4e6fb31-921e-4c81-bf9b-833752a58dfc" status: 202 len: 0 microversion: 2.1 time: 0.361077 Jul 16 21:42:58.177687 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2793/5588] 10.222.0.16 () {58 vars in 1415 bytes} [Sat Jul 16 21:42:57 2022] DELETE /compute/v2.1/servers/97d5b8fb-ce56-41ba-b786-5f8474c9b2f3/os-volume_attachments/f4e6fb31-921e-4c81-bf9b-833752a58dfc => generated 0 bytes in 363 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Jul 16 21:42:59.298197 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-92d8ede7-dfe6-4dbd-b790-93c0f6ab9355 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:42:59.298746 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-92d8ede7-dfe6-4dbd-b790-93c0f6ab9355 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] [instance: 97d5b8fb-ce56-41ba-b786-5f8474c9b2f3] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:42:59.307655 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-92d8ede7-dfe6-4dbd-b790-93c0f6ab9355 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:42:59.308122 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-92d8ede7-dfe6-4dbd-b790-93c0f6ab9355 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:42:59.308907 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-92d8ede7-dfe6-4dbd-b790-93c0f6ab9355 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:42:59.309380 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-92d8ede7-dfe6-4dbd-b790-93c0f6ab9355 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:42:59.336845 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-92d8ede7-dfe6-4dbd-b790-93c0f6ab9355 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] [instance: 97d5b8fb-ce56-41ba-b786-5f8474c9b2f3] Going to try to terminate instance {{(pid=138681) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 16 21:42:59.669371 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-92d8ede7-dfe6-4dbd-b790-93c0f6ab9355 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] 10.222.0.16 "DELETE /compute/v2.1/servers/97d5b8fb-ce56-41ba-b786-5f8474c9b2f3" status: 204 len: 0 microversion: 2.1 time: 0.375897 Jul 16 21:42:59.670084 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2796/5589] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:42:59 2022] DELETE /compute/v2.1/servers/97d5b8fb-ce56-41ba-b786-5f8474c9b2f3 => generated 0 bytes in 378 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 16 21:42:59.687770 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-37433079-8c2f-4c27-91c1-6d3a979a61ef tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:42:59.688619 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-37433079-8c2f-4c27-91c1-6d3a979a61ef tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] [instance: 97d5b8fb-ce56-41ba-b786-5f8474c9b2f3] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:42:59.696820 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-37433079-8c2f-4c27-91c1-6d3a979a61ef tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:42:59.699422 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-37433079-8c2f-4c27-91c1-6d3a979a61ef tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:42:59.699422 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-37433079-8c2f-4c27-91c1-6d3a979a61ef tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:42:59.699422 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-37433079-8c2f-4c27-91c1-6d3a979a61ef tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:42:59.840312 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-37433079-8c2f-4c27-91c1-6d3a979a61ef tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '95cd6b54d8164de1a524b407a1852ebd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '56b8099d9406431284b4dd58b5c16359', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:42:59.848404 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-37433079-8c2f-4c27-91c1-6d3a979a61ef tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] 10.222.0.16 "GET /compute/v2.1/servers/97d5b8fb-ce56-41ba-b786-5f8474c9b2f3" status: 200 len: 1514 microversion: 2.1 time: 0.164945 Jul 16 21:42:59.849786 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2794/5590] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:42:59 2022] GET /compute/v2.1/servers/97d5b8fb-ce56-41ba-b786-5f8474c9b2f3 => generated 1514 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:43:00.868119 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-11b27607-6783-4ed4-ba96-9ff578f00de6 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:43:00.868719 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-11b27607-6783-4ed4-ba96-9ff578f00de6 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] [instance: 97d5b8fb-ce56-41ba-b786-5f8474c9b2f3] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:43:00.875663 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-11b27607-6783-4ed4-ba96-9ff578f00de6 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:43:00.876126 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-11b27607-6783-4ed4-ba96-9ff578f00de6 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:43:00.876986 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-11b27607-6783-4ed4-ba96-9ff578f00de6 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:43:00.877696 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-11b27607-6783-4ed4-ba96-9ff578f00de6 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:43:01.029052 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-11b27607-6783-4ed4-ba96-9ff578f00de6 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '95cd6b54d8164de1a524b407a1852ebd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '56b8099d9406431284b4dd58b5c16359', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:43:01.038224 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-11b27607-6783-4ed4-ba96-9ff578f00de6 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] 10.222.0.16 "GET /compute/v2.1/servers/97d5b8fb-ce56-41ba-b786-5f8474c9b2f3" status: 200 len: 1514 microversion: 2.1 time: 0.175047 Jul 16 21:43:01.039462 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2797/5591] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:43:00 2022] GET /compute/v2.1/servers/97d5b8fb-ce56-41ba-b786-5f8474c9b2f3 => generated 1514 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:43:01.949024 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-b8e45c62-b000-447d-a7f7-6a30f3deb3a6 req-652d3254-75ad-467c-a522-52bf86eb5849 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "97d5b8fb-ce56-41ba-b786-5f8474c9b2f3", "name": "network-vif-deleted", "tag": "38773ab4-9a2c-4c8d-9703-dd1b20b11604"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:43:01.958251 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-b8e45c62-b000-447d-a7f7-6a30f3deb3a6 req-652d3254-75ad-467c-a522-52bf86eb5849 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:43:01.958711 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-b8e45c62-b000-447d-a7f7-6a30f3deb3a6 req-652d3254-75ad-467c-a522-52bf86eb5849 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:43:01.970766 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-b8e45c62-b000-447d-a7f7-6a30f3deb3a6 req-652d3254-75ad-467c-a522-52bf86eb5849 service nova] Creating event network-vif-deleted:38773ab4-9a2c-4c8d-9703-dd1b20b11604 for instance 97d5b8fb-ce56-41ba-b786-5f8474c9b2f3 on nch2-master-1 Jul 16 21:43:01.975785 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-b8e45c62-b000-447d-a7f7-6a30f3deb3a6 req-652d3254-75ad-467c-a522-52bf86eb5849 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.030829 Jul 16 21:43:01.976731 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2795/5592] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:43:01 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:43:02.058581 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-27259434-b0b6-45e3-ac28-5ab18026aba8 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:43:02.059555 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-27259434-b0b6-45e3-ac28-5ab18026aba8 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] [instance: 97d5b8fb-ce56-41ba-b786-5f8474c9b2f3] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:43:02.066604 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-27259434-b0b6-45e3-ac28-5ab18026aba8 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:43:02.067090 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-27259434-b0b6-45e3-ac28-5ab18026aba8 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:43:02.067875 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-27259434-b0b6-45e3-ac28-5ab18026aba8 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:43:02.068321 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-27259434-b0b6-45e3-ac28-5ab18026aba8 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:43:02.145824 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-27259434-b0b6-45e3-ac28-5ab18026aba8 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '95cd6b54d8164de1a524b407a1852ebd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '56b8099d9406431284b4dd58b5c16359', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:43:02.152600 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-27259434-b0b6-45e3-ac28-5ab18026aba8 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] 10.222.0.16 "GET /compute/v2.1/servers/97d5b8fb-ce56-41ba-b786-5f8474c9b2f3" status: 200 len: 1472 microversion: 2.1 time: 0.098578 Jul 16 21:43:02.153377 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2798/5593] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:43:02 2022] GET /compute/v2.1/servers/97d5b8fb-ce56-41ba-b786-5f8474c9b2f3 => generated 1472 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:43:03.170596 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-6742515b-b1ad-40e1-80a8-b752b7d73bb1 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:43:03.171349 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-6742515b-b1ad-40e1-80a8-b752b7d73bb1 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] [instance: 97d5b8fb-ce56-41ba-b786-5f8474c9b2f3] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:43:03.183520 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6742515b-b1ad-40e1-80a8-b752b7d73bb1 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:43:03.187384 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6742515b-b1ad-40e1-80a8-b752b7d73bb1 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:43:03.187384 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6742515b-b1ad-40e1-80a8-b752b7d73bb1 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:43:03.187384 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6742515b-b1ad-40e1-80a8-b752b7d73bb1 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:43:03.226443 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-6742515b-b1ad-40e1-80a8-b752b7d73bb1 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] HTTP exception thrown: Instance 97d5b8fb-ce56-41ba-b786-5f8474c9b2f3 could not be found. Jul 16 21:43:03.229100 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-6742515b-b1ad-40e1-80a8-b752b7d73bb1 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Request method failure captured: Jul 16 21:43:03.229100 ncdv-master-1 devstack@n-api.service[138680]: request: GET /compute/v2.1/servers/97d5b8fb-ce56-41ba-b786-5f8474c9b2f3 HTTP/1.1 Jul 16 21:43:03.229100 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:43:03.229100 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:43:03.229100 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:43:03.229100 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 0 Jul 16 21:43:03.229100 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:43:03.229100 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:43:03.229100 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:43:03.229100 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:43:03.229100 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0y9Z2lw1KYjDsPS13NZHwxrz3jgJvrjI-IVHcVBEZED44KX5zFVfR6bb6-Pq4i-X0lnxFMPshF32DM5zlyWDG4Y42M1U2GEvd5SuvwIhXVwNJi1rFO6f_DEcjyEt99qd7XA56BqBMZam1ygGWlE86E4Kqh4PCcdasDMuFL2uSU2qKTg Jul 16 21:43:03.229100 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:43:03.229100 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:43:03.229100 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:43:03.229100 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:43:03.229100 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:43:03.229100 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:43:03.229100 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: 56b8099d9406431284b4dd58b5c16359 Jul 16 21:43:03.229100 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-VolumesSnapshotTestJSON-1954720325 Jul 16 21:43:03.229100 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:43:03.229100 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:43:03.229100 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/56b8099d9406431284b4dd58b5c16359"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/56b8099d9406431284b4dd58b5c16359"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_56b8099d9406431284b4dd58b5c16359"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/56b8099d9406431284b4dd58b5c16359"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:43:03.229100 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-VolumesSnapshotTestJSON-1954720325 Jul 16 21:43:03.229100 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: 56b8099d9406431284b4dd58b5c16359 Jul 16 21:43:03.229100 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-VolumesSnapshotTestJSON-1954720325 Jul 16 21:43:03.229100 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-VolumesSnapshotTestJSON-1954720325-project Jul 16 21:43:03.229100 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:43:03.229100 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:43:03.229100 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: 95cd6b54d8164de1a524b407a1852ebd Jul 16 21:43:03.229100 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-VolumesSnapshotTestJSON-1954720325-project Jul 16 21:43:03.229100 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 21:43:03.229100 ncdv-master-1 devstack@n-api.service[138680]: exception: Instance 97d5b8fb-ce56-41ba-b786-5f8474c9b2f3 could not be found. Jul 16 21:43:03.229100 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': '97d5b8fb-ce56-41ba-b786-5f8474c9b2f3'} Jul 16 21:43:03.229100 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:43:03.229100 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:43:03.229100 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:43:03.229100 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:43:03.229100 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:43:03.229100 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:43:03.229100 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:43:03.229100 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:43:03.229100 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:43:03.229100 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:43:03.229100 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 97d5b8fb-ce56-41ba-b786-5f8474c9b2f3 could not be found. Jul 16 21:43:03.229100 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:43:03.229100 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:43:03.229100 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:43:03.229100 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:43:03.229100 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:43:03.229100 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:43:03.229100 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:43:03.229100 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:43:03.229100 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:43:03.229100 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:43:03.229100 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:43:03.229100 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:43:03.229100 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:43:03.229100 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:43:03.229100 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:43:03.229100 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:43:03.229100 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 97d5b8fb-ce56-41ba-b786-5f8474c9b2f3 could not be found. Jul 16 21:43:03.229100 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:43:03.229100 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:43:03.229100 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:43:03.250756 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:43:03.250756 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:43:03.250756 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:43:03.250756 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:43:03.250756 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:43:03.250756 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 97d5b8fb-ce56-41ba-b786-5f8474c9b2f3 could not be found. Jul 16 21:43:03.250756 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:43:03.250756 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-6742515b-b1ad-40e1-80a8-b752b7d73bb1 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] Returning 404 to user: Instance 97d5b8fb-ce56-41ba-b786-5f8474c9b2f3 could not be found. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:43:03.250756 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-6742515b-b1ad-40e1-80a8-b752b7d73bb1 tempest-VolumesSnapshotTestJSON-1954720325 tempest-VolumesSnapshotTestJSON-1954720325-project] 10.222.0.16 "GET /compute/v2.1/servers/97d5b8fb-ce56-41ba-b786-5f8474c9b2f3" status: 404 len: 111 microversion: 2.1 time: 0.064074 Jul 16 21:43:03.250756 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2796/5594] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:43:03 2022] GET /compute/v2.1/servers/97d5b8fb-ce56-41ba-b786-5f8474c9b2f3 => generated 111 bytes in 67 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:43:04.991678 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-b9fc7271-38b7-4056-9c69-40a1997b3891 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestNetworkAdvancedServerOps-server-1666306140", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452", "networks": [{"uuid": "4dbf80ce-1d7b-4c63-8508-048ec4cb57a3"}], "key_name": "tempest-TestNetworkAdvancedServerOps-343919104", "security_groups": [{"name": "tempest-secgroup-smoke-233904251"}]}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:43:05.129642 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.network.neutron [None req-b9fc7271-38b7-4056-9c69-40a1997b3891 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] validate_networks() for [('4dbf80ce-1d7b-4c63-8508-048ec4cb57a3', None, None, None, None, None)] {{(pid=138681) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 16 21:43:05.657402 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-b9fc7271-38b7-4056-9c69-40a1997b3891 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Flavor limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 16 21:43:05.657845 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-b9fc7271-38b7-4056-9c69-40a1997b3891 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Image limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 16 21:43:05.658339 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-b9fc7271-38b7-4056-9c69-40a1997b3891 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Flavor pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 16 21:43:05.658769 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-b9fc7271-38b7-4056-9c69-40a1997b3891 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Image pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 16 21:43:05.659233 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-b9fc7271-38b7-4056-9c69-40a1997b3891 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 16 21:43:05.670998 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b9fc7271-38b7-4056-9c69-40a1997b3891 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:43:05.673228 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b9fc7271-38b7-4056-9c69-40a1997b3891 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:43:05.674374 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b9fc7271-38b7-4056-9c69-40a1997b3891 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:43:05.675078 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b9fc7271-38b7-4056-9c69-40a1997b3891 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:43:05.696941 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-b9fc7271-38b7-4056-9c69-40a1997b3891 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Getting quotas for project 77c9b7e351e24dc88207e9498f75f578. Resources: {'ram', 'instances', 'cores'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:43:05.708136 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-b9fc7271-38b7-4056-9c69-40a1997b3891 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Getting quotas for user 833136cbdeab4b9e8b6dd2d24c7c8753 and project 77c9b7e351e24dc88207e9498f75f578. Resources: {'ram', 'instances', 'cores'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:43:05.725870 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-b9fc7271-38b7-4056-9c69-40a1997b3891 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Going to run 1 instances... {{(pid=138681) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 16 21:43:05.729381 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-b9fc7271-38b7-4056-9c69-40a1997b3891 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] [instance: 64b5a7a2-7d82-4c60-802e-8e92c14e0790] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138681) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 16 21:43:05.908459 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-b9fc7271-38b7-4056-9c69-40a1997b3891 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 405 microversion: 2.1 time: 0.923247 Jul 16 21:43:05.909113 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2799/5595] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 21:43:04 2022] POST /compute/v2.1/servers => generated 405 bytes in 925 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 16 21:43:05.928728 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-26251d7a-a85a-4499-b6db-baa71da17d96 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:43:05.929363 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-26251d7a-a85a-4499-b6db-baa71da17d96 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] [instance: 64b5a7a2-7d82-4c60-802e-8e92c14e0790] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:43:06.015431 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-26251d7a-a85a-4499-b6db-baa71da17d96 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '833136cbdeab4b9e8b6dd2d24c7c8753', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '77c9b7e351e24dc88207e9498f75f578', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:43:06.026231 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-26251d7a-a85a-4499-b6db-baa71da17d96 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] 10.222.0.16 "GET /compute/v2.1/servers/64b5a7a2-7d82-4c60-802e-8e92c14e0790" status: 200 len: 1266 microversion: 2.1 time: 0.105730 Jul 16 21:43:06.027184 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2797/5596] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:43:05 2022] GET /compute/v2.1/servers/64b5a7a2-7d82-4c60-802e-8e92c14e0790 => generated 1266 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:43:07.045186 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-8b0afa74-b411-4775-b14d-3f2e600f7ed3 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:43:07.046444 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-8b0afa74-b411-4775-b14d-3f2e600f7ed3 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] [instance: 64b5a7a2-7d82-4c60-802e-8e92c14e0790] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:43:07.061047 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8b0afa74-b411-4775-b14d-3f2e600f7ed3 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:43:07.063366 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8b0afa74-b411-4775-b14d-3f2e600f7ed3 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:43:07.063366 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8b0afa74-b411-4775-b14d-3f2e600f7ed3 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:43:07.063366 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8b0afa74-b411-4775-b14d-3f2e600f7ed3 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:43:07.166007 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-8b0afa74-b411-4775-b14d-3f2e600f7ed3 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '833136cbdeab4b9e8b6dd2d24c7c8753', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '77c9b7e351e24dc88207e9498f75f578', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:43:07.172378 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-8b0afa74-b411-4775-b14d-3f2e600f7ed3 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] 10.222.0.16 "GET /compute/v2.1/servers/64b5a7a2-7d82-4c60-802e-8e92c14e0790" status: 200 len: 1270 microversion: 2.1 time: 0.132997 Jul 16 21:43:07.173539 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2800/5597] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:43:07 2022] GET /compute/v2.1/servers/64b5a7a2-7d82-4c60-802e-8e92c14e0790 => generated 1270 bytes in 135 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:43:08.190292 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-03636d25-0e28-4222-96b9-8ec42193197d tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:43:08.191261 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-03636d25-0e28-4222-96b9-8ec42193197d tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] [instance: 64b5a7a2-7d82-4c60-802e-8e92c14e0790] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:43:08.199725 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-03636d25-0e28-4222-96b9-8ec42193197d tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:43:08.200319 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-03636d25-0e28-4222-96b9-8ec42193197d tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:43:08.201493 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-03636d25-0e28-4222-96b9-8ec42193197d tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:43:08.202117 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-03636d25-0e28-4222-96b9-8ec42193197d tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:43:08.306658 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-03636d25-0e28-4222-96b9-8ec42193197d tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '833136cbdeab4b9e8b6dd2d24c7c8753', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '77c9b7e351e24dc88207e9498f75f578', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:43:08.319377 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-03636d25-0e28-4222-96b9-8ec42193197d tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] 10.222.0.16 "GET /compute/v2.1/servers/64b5a7a2-7d82-4c60-802e-8e92c14e0790" status: 200 len: 1326 microversion: 2.1 time: 0.133060 Jul 16 21:43:08.320068 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2798/5598] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:43:08 2022] GET /compute/v2.1/servers/64b5a7a2-7d82-4c60-802e-8e92c14e0790 => generated 1326 bytes in 135 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:43:09.344152 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-54e5ee14-775f-45a6-8a87-2c85cb7c2761 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:43:09.344908 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-54e5ee14-775f-45a6-8a87-2c85cb7c2761 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] [instance: 64b5a7a2-7d82-4c60-802e-8e92c14e0790] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:43:09.352505 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-54e5ee14-775f-45a6-8a87-2c85cb7c2761 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:43:09.356552 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-54e5ee14-775f-45a6-8a87-2c85cb7c2761 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:43:09.356552 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-54e5ee14-775f-45a6-8a87-2c85cb7c2761 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:43:09.356552 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-54e5ee14-775f-45a6-8a87-2c85cb7c2761 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:43:09.588345 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-54e5ee14-775f-45a6-8a87-2c85cb7c2761 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '833136cbdeab4b9e8b6dd2d24c7c8753', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '77c9b7e351e24dc88207e9498f75f578', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:43:09.596314 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-54e5ee14-775f-45a6-8a87-2c85cb7c2761 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] 10.222.0.16 "GET /compute/v2.1/servers/64b5a7a2-7d82-4c60-802e-8e92c14e0790" status: 200 len: 1391 microversion: 2.1 time: 0.256079 Jul 16 21:43:09.596989 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2801/5599] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:43:09 2022] GET /compute/v2.1/servers/64b5a7a2-7d82-4c60-802e-8e92c14e0790 => generated 1391 bytes in 258 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:43:10.090505 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-3cd2e487-97e4-487b-9a79-7792af68beb2 req-4d33fe18-75f9-4091-b366-3179e0257a78 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "64b5a7a2-7d82-4c60-802e-8e92c14e0790", "tag": "7a7ec2ad-db68-4392-93f0-85d2811e10e1"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:43:10.100423 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-3cd2e487-97e4-487b-9a79-7792af68beb2 req-4d33fe18-75f9-4091-b366-3179e0257a78 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:43:10.100753 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-3cd2e487-97e4-487b-9a79-7792af68beb2 req-4d33fe18-75f9-4091-b366-3179e0257a78 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:43:10.113504 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-3cd2e487-97e4-487b-9a79-7792af68beb2 req-4d33fe18-75f9-4091-b366-3179e0257a78 service nova] Creating event network-changed:7a7ec2ad-db68-4392-93f0-85d2811e10e1 for instance 64b5a7a2-7d82-4c60-802e-8e92c14e0790 on nch1-master-1 Jul 16 21:43:10.120017 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-3cd2e487-97e4-487b-9a79-7792af68beb2 req-4d33fe18-75f9-4091-b366-3179e0257a78 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.033194 Jul 16 21:43:10.121010 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2799/5600] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:43:10 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:43:10.615904 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-5c9677d3-2ad3-4f2d-ba7d-a661b45db774 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:43:10.617005 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-5c9677d3-2ad3-4f2d-ba7d-a661b45db774 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] [instance: 64b5a7a2-7d82-4c60-802e-8e92c14e0790] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:43:10.624766 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5c9677d3-2ad3-4f2d-ba7d-a661b45db774 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:43:10.625568 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5c9677d3-2ad3-4f2d-ba7d-a661b45db774 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:43:10.626011 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5c9677d3-2ad3-4f2d-ba7d-a661b45db774 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:43:10.626498 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5c9677d3-2ad3-4f2d-ba7d-a661b45db774 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:43:10.851284 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-5c9677d3-2ad3-4f2d-ba7d-a661b45db774 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '833136cbdeab4b9e8b6dd2d24c7c8753', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '77c9b7e351e24dc88207e9498f75f578', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:43:10.859823 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-5c9677d3-2ad3-4f2d-ba7d-a661b45db774 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] 10.222.0.16 "GET /compute/v2.1/servers/64b5a7a2-7d82-4c60-802e-8e92c14e0790" status: 200 len: 1391 microversion: 2.1 time: 0.247978 Jul 16 21:43:10.860995 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2802/5601] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:43:10 2022] GET /compute/v2.1/servers/64b5a7a2-7d82-4c60-802e-8e92c14e0790 => generated 1391 bytes in 250 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:43:11.878112 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-ea508d45-1cb1-4812-94e0-3fe2e16eb1e3 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:43:11.878820 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-ea508d45-1cb1-4812-94e0-3fe2e16eb1e3 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] [instance: 64b5a7a2-7d82-4c60-802e-8e92c14e0790] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:43:11.886096 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ea508d45-1cb1-4812-94e0-3fe2e16eb1e3 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:43:11.886555 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ea508d45-1cb1-4812-94e0-3fe2e16eb1e3 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:43:11.887475 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ea508d45-1cb1-4812-94e0-3fe2e16eb1e3 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:43:11.887931 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ea508d45-1cb1-4812-94e0-3fe2e16eb1e3 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:43:12.067397 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-ea508d45-1cb1-4812-94e0-3fe2e16eb1e3 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '833136cbdeab4b9e8b6dd2d24c7c8753', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '77c9b7e351e24dc88207e9498f75f578', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:43:12.076481 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-ea508d45-1cb1-4812-94e0-3fe2e16eb1e3 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] 10.222.0.16 "GET /compute/v2.1/servers/64b5a7a2-7d82-4c60-802e-8e92c14e0790" status: 200 len: 1391 microversion: 2.1 time: 0.203362 Jul 16 21:43:12.079077 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2800/5602] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:43:11 2022] GET /compute/v2.1/servers/64b5a7a2-7d82-4c60-802e-8e92c14e0790 => generated 1391 bytes in 205 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:43:13.093699 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-f889dc2a-68de-4d2d-b658-834b0f2cc280 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:43:13.094426 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-f889dc2a-68de-4d2d-b658-834b0f2cc280 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] [instance: 64b5a7a2-7d82-4c60-802e-8e92c14e0790] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:43:13.102522 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f889dc2a-68de-4d2d-b658-834b0f2cc280 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:43:13.103776 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f889dc2a-68de-4d2d-b658-834b0f2cc280 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:43:13.104716 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f889dc2a-68de-4d2d-b658-834b0f2cc280 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:43:13.106399 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f889dc2a-68de-4d2d-b658-834b0f2cc280 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:43:13.555194 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-f889dc2a-68de-4d2d-b658-834b0f2cc280 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '833136cbdeab4b9e8b6dd2d24c7c8753', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '77c9b7e351e24dc88207e9498f75f578', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:43:13.564985 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-f889dc2a-68de-4d2d-b658-834b0f2cc280 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] 10.222.0.16 "GET /compute/v2.1/servers/64b5a7a2-7d82-4c60-802e-8e92c14e0790" status: 200 len: 1391 microversion: 2.1 time: 0.475169 Jul 16 21:43:13.566350 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2803/5603] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:43:13 2022] GET /compute/v2.1/servers/64b5a7a2-7d82-4c60-802e-8e92c14e0790 => generated 1391 bytes in 478 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:43:14.583201 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-d6dcb029-3ad3-46e9-b6e7-2ef9e23e682e tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:43:14.583820 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-d6dcb029-3ad3-46e9-b6e7-2ef9e23e682e tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] [instance: 64b5a7a2-7d82-4c60-802e-8e92c14e0790] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:43:14.591055 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d6dcb029-3ad3-46e9-b6e7-2ef9e23e682e tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:43:14.591738 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d6dcb029-3ad3-46e9-b6e7-2ef9e23e682e tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:43:14.592519 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d6dcb029-3ad3-46e9-b6e7-2ef9e23e682e tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:43:14.592961 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d6dcb029-3ad3-46e9-b6e7-2ef9e23e682e tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:43:14.751553 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-d6dcb029-3ad3-46e9-b6e7-2ef9e23e682e tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '833136cbdeab4b9e8b6dd2d24c7c8753', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '77c9b7e351e24dc88207e9498f75f578', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:43:14.759265 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-d6dcb029-3ad3-46e9-b6e7-2ef9e23e682e tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] 10.222.0.16 "GET /compute/v2.1/servers/64b5a7a2-7d82-4c60-802e-8e92c14e0790" status: 200 len: 1391 microversion: 2.1 time: 0.180012 Jul 16 21:43:14.759844 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2801/5604] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:43:14 2022] GET /compute/v2.1/servers/64b5a7a2-7d82-4c60-802e-8e92c14e0790 => generated 1391 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:43:15.776474 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-49a1c673-c287-4d91-a2af-4b547a5ec26b tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:43:15.777794 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-49a1c673-c287-4d91-a2af-4b547a5ec26b tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] [instance: 64b5a7a2-7d82-4c60-802e-8e92c14e0790] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:43:15.786946 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell0:SELECT=2 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:43:15.788385 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell0:SELECT=10 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:43:15.793353 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-49a1c673-c287-4d91-a2af-4b547a5ec26b tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:43:15.794562 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-49a1c673-c287-4d91-a2af-4b547a5ec26b tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:43:15.795477 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-49a1c673-c287-4d91-a2af-4b547a5ec26b tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:43:15.796200 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-49a1c673-c287-4d91-a2af-4b547a5ec26b tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:43:16.017387 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-49a1c673-c287-4d91-a2af-4b547a5ec26b tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '833136cbdeab4b9e8b6dd2d24c7c8753', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '77c9b7e351e24dc88207e9498f75f578', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:43:16.026794 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-49a1c673-c287-4d91-a2af-4b547a5ec26b tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] 10.222.0.16 "GET /compute/v2.1/servers/64b5a7a2-7d82-4c60-802e-8e92c14e0790" status: 200 len: 1391 microversion: 2.1 time: 0.254844 Jul 16 21:43:16.036775 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2804/5605] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:43:15 2022] GET /compute/v2.1/servers/64b5a7a2-7d82-4c60-802e-8e92c14e0790 => generated 1391 bytes in 266 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:43:16.147746 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-9b7b099e-52b7-4e44-b08e-d7ff2d7d65d2 req-eb197c46-9489-4de6-9ef7-102b89bf07c2 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "64b5a7a2-7d82-4c60-802e-8e92c14e0790", "name": "network-vif-plugged", "status": "completed", "tag": "7a7ec2ad-db68-4392-93f0-85d2811e10e1"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:43:16.153474 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell0:SELECT=2 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:43:16.157813 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-9b7b099e-52b7-4e44-b08e-d7ff2d7d65d2 req-eb197c46-9489-4de6-9ef7-102b89bf07c2 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:43:16.158274 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-9b7b099e-52b7-4e44-b08e-d7ff2d7d65d2 req-eb197c46-9489-4de6-9ef7-102b89bf07c2 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:43:16.173523 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-9b7b099e-52b7-4e44-b08e-d7ff2d7d65d2 req-eb197c46-9489-4de6-9ef7-102b89bf07c2 service nova] Creating event network-vif-plugged:7a7ec2ad-db68-4392-93f0-85d2811e10e1 for instance 64b5a7a2-7d82-4c60-802e-8e92c14e0790 on nch1-master-1 Jul 16 21:43:16.180340 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-9b7b099e-52b7-4e44-b08e-d7ff2d7d65d2 req-eb197c46-9489-4de6-9ef7-102b89bf07c2 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.035180 Jul 16 21:43:16.180340 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2802/5606] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:43:16 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:43:17.051850 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-bfc8e609-5887-44e8-aa7d-ccf074f778cf tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:43:17.052651 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-bfc8e609-5887-44e8-aa7d-ccf074f778cf tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] [instance: 64b5a7a2-7d82-4c60-802e-8e92c14e0790] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:43:17.056071 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_api:SELECT=58,nova_api:INSERT=4,nova_api:UPDATE=1 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:43:17.062126 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-bfc8e609-5887-44e8-aa7d-ccf074f778cf tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:43:17.063070 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-bfc8e609-5887-44e8-aa7d-ccf074f778cf tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:43:17.064068 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-bfc8e609-5887-44e8-aa7d-ccf074f778cf tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:43:17.064761 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-bfc8e609-5887-44e8-aa7d-ccf074f778cf tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:43:17.069788 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=77,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:43:17.438116 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-bfc8e609-5887-44e8-aa7d-ccf074f778cf tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '833136cbdeab4b9e8b6dd2d24c7c8753', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '77c9b7e351e24dc88207e9498f75f578', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:43:17.447210 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-bfc8e609-5887-44e8-aa7d-ccf074f778cf tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] 10.222.0.16 "GET /compute/v2.1/servers/64b5a7a2-7d82-4c60-802e-8e92c14e0790" status: 200 len: 1391 microversion: 2.1 time: 0.401804 Jul 16 21:43:17.449537 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2805/5607] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:43:17 2022] GET /compute/v2.1/servers/64b5a7a2-7d82-4c60-802e-8e92c14e0790 => generated 1391 bytes in 405 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:43:18.358206 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-7096eb0b-300e-4143-a5d4-48ca60d899a1 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Action: 'action', calling method: >, body: {"os-getConsoleOutput": {}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:43:18.359299 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-7096eb0b-300e-4143-a5d4-48ca60d899a1 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] [instance: 19df5c33-44d4-4de9-947b-4707b78af23a] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:43:18.369961 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7096eb0b-300e-4143-a5d4-48ca60d899a1 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:43:18.369961 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7096eb0b-300e-4143-a5d4-48ca60d899a1 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:43:18.370744 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7096eb0b-300e-4143-a5d4-48ca60d899a1 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:43:18.370744 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7096eb0b-300e-4143-a5d4-48ca60d899a1 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:43:18.472205 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-7096eb0b-300e-4143-a5d4-48ca60d899a1 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] 10.222.0.16 "POST /compute/v2.1/servers/19df5c33-44d4-4de9-947b-4707b78af23a/action" status: 200 len: 32524 microversion: 2.42 time: 0.119945 Jul 16 21:43:18.479308 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2803/5608] 10.222.0.16 () {62 vars in 1318 bytes} [Sat Jul 16 21:43:18 2022] POST /compute/v2.1/servers/19df5c33-44d4-4de9-947b-4707b78af23a/action => generated 32524 bytes in 129 msecs (HTTP/1.1 200) 9 headers in 361 bytes (1 switches on core 0) Jul 16 21:43:18.486004 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-6602c9fd-9040-4bb0-a340-d1140f46dd0a tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:43:18.487173 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-6602c9fd-9040-4bb0-a340-d1140f46dd0a tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] [instance: 19df5c33-44d4-4de9-947b-4707b78af23a] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:43:18.499523 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6602c9fd-9040-4bb0-a340-d1140f46dd0a tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:43:18.500959 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6602c9fd-9040-4bb0-a340-d1140f46dd0a tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:43:18.503320 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6602c9fd-9040-4bb0-a340-d1140f46dd0a tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:43:18.508533 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6602c9fd-9040-4bb0-a340-d1140f46dd0a tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:43:18.554041 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-6602c9fd-9040-4bb0-a340-d1140f46dd0a tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] [instance: 19df5c33-44d4-4de9-947b-4707b78af23a] Going to try to terminate instance {{(pid=138680) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 16 21:43:18.559450 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-eccd2bfc-a8a8-4287-9a54-a7bd4ba52a95 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:43:18.560157 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-eccd2bfc-a8a8-4287-9a54-a7bd4ba52a95 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] [instance: 64b5a7a2-7d82-4c60-802e-8e92c14e0790] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:43:18.566855 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-eccd2bfc-a8a8-4287-9a54-a7bd4ba52a95 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:43:18.567487 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-eccd2bfc-a8a8-4287-9a54-a7bd4ba52a95 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:43:18.568288 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-eccd2bfc-a8a8-4287-9a54-a7bd4ba52a95 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:43:18.568748 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-eccd2bfc-a8a8-4287-9a54-a7bd4ba52a95 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:43:19.016729 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-6602c9fd-9040-4bb0-a340-d1140f46dd0a tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] 10.222.0.16 "DELETE /compute/v2.1/servers/19df5c33-44d4-4de9-947b-4707b78af23a" status: 204 len: 0 microversion: 2.42 time: 0.534927 Jul 16 21:43:19.018520 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2804/5609] 10.222.0.16 () {60 vars in 1279 bytes} [Sat Jul 16 21:43:18 2022] DELETE /compute/v2.1/servers/19df5c33-44d4-4de9-947b-4707b78af23a => generated 0 bytes in 538 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Jul 16 21:43:19.026914 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-87a95e61-4d20-4c18-b91a-9e60b8e44d67 tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-server-smoke-1346327982", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452", "networks": [{"uuid": "afc296b8-c34c-4072-a4f3-a019297ff8dc"}], "key_name": "tempest-TestSecurityGroupsBasicOps-1925752067", "security_groups": []}, "os:scheduler_hints": {"different_host": ["0ec2632e-3da8-44ac-b9ba-7300f5bc1f99"]}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:43:19.190663 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-eccd2bfc-a8a8-4287-9a54-a7bd4ba52a95 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '833136cbdeab4b9e8b6dd2d24c7c8753', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '77c9b7e351e24dc88207e9498f75f578', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:43:19.212315 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-eccd2bfc-a8a8-4287-9a54-a7bd4ba52a95 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] 10.222.0.16 "GET /compute/v2.1/servers/64b5a7a2-7d82-4c60-802e-8e92c14e0790" status: 200 len: 1563 microversion: 2.1 time: 0.744890 Jul 16 21:43:19.212315 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2806/5610] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:43:18 2022] GET /compute/v2.1/servers/64b5a7a2-7d82-4c60-802e-8e92c14e0790 => generated 1563 bytes in 746 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:43:19.213612 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-225bb53b-8f56-4451-9157-9b50076ad83b tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:43:19.215132 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-225bb53b-8f56-4451-9157-9b50076ad83b tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] [instance: 19df5c33-44d4-4de9-947b-4707b78af23a] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:43:19.223628 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-225bb53b-8f56-4451-9157-9b50076ad83b tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:43:19.226244 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-225bb53b-8f56-4451-9157-9b50076ad83b tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:43:19.227460 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-225bb53b-8f56-4451-9157-9b50076ad83b tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:43:19.228469 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-225bb53b-8f56-4451-9157-9b50076ad83b tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:43:19.392697 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.network.neutron [None req-87a95e61-4d20-4c18-b91a-9e60b8e44d67 tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] validate_networks() for [('afc296b8-c34c-4072-a4f3-a019297ff8dc', None, None, None, None, None)] {{(pid=138680) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 16 21:43:19.684641 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-225bb53b-8f56-4451-9157-9b50076ad83b tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '726b46c54e9541fbb7825607bac3abe5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fd0ce287cb44cd59045868ce688909a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:43:19.716752 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-225bb53b-8f56-4451-9157-9b50076ad83b tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '726b46c54e9541fbb7825607bac3abe5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fd0ce287cb44cd59045868ce688909a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:43:19.720883 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-225bb53b-8f56-4451-9157-9b50076ad83b tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '726b46c54e9541fbb7825607bac3abe5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fd0ce287cb44cd59045868ce688909a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:43:19.721401 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-225bb53b-8f56-4451-9157-9b50076ad83b tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] 10.222.0.16 "GET /compute/v2.1/servers/19df5c33-44d4-4de9-947b-4707b78af23a" status: 200 len: 2958 microversion: 2.42 time: 0.511907 Jul 16 21:43:19.722339 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2807/5611] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:43:19 2022] GET /compute/v2.1/servers/19df5c33-44d4-4de9-947b-4707b78af23a => generated 2958 bytes in 514 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:43:19.732022 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-5c92c240-145a-4b43-a034-45958b081c55 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:43:19.732987 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-5c92c240-145a-4b43-a034-45958b081c55 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] [instance: 64b5a7a2-7d82-4c60-802e-8e92c14e0790] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:43:19.768482 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5c92c240-145a-4b43-a034-45958b081c55 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:43:19.769498 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5c92c240-145a-4b43-a034-45958b081c55 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:43:19.772568 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5c92c240-145a-4b43-a034-45958b081c55 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:43:19.773473 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5c92c240-145a-4b43-a034-45958b081c55 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:43:19.926838 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_api:SELECT=42,nova_api:DELETE=1,nova_api:UPDATE=1 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:43:20.155563 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-5c92c240-145a-4b43-a034-45958b081c55 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '833136cbdeab4b9e8b6dd2d24c7c8753', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '77c9b7e351e24dc88207e9498f75f578', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:43:20.178493 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-5c92c240-145a-4b43-a034-45958b081c55 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] 10.222.0.16 "GET /compute/v2.1/servers/64b5a7a2-7d82-4c60-802e-8e92c14e0790" status: 200 len: 1563 microversion: 2.1 time: 0.454705 Jul 16 21:43:20.183740 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2808/5612] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:43:19 2022] GET /compute/v2.1/servers/64b5a7a2-7d82-4c60-802e-8e92c14e0790 => generated 1563 bytes in 460 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:43:20.743130 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-e21aaa74-7b76-4bdc-b299-fc492ec331a9 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:43:20.744920 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-e21aaa74-7b76-4bdc-b299-fc492ec331a9 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] [instance: 19df5c33-44d4-4de9-947b-4707b78af23a] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:43:20.773506 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e21aaa74-7b76-4bdc-b299-fc492ec331a9 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:43:20.774297 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-87a95e61-4d20-4c18-b91a-9e60b8e44d67 tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Flavor limits 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 16 21:43:20.775125 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e21aaa74-7b76-4bdc-b299-fc492ec331a9 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.020s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:43:20.775743 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-87a95e61-4d20-4c18-b91a-9e60b8e44d67 tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Image limits 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 16 21:43:20.776343 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-87a95e61-4d20-4c18-b91a-9e60b8e44d67 tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Flavor pref 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 16 21:43:20.776859 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-87a95e61-4d20-4c18-b91a-9e60b8e44d67 tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Image pref 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 16 21:43:20.777379 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-87a95e61-4d20-4c18-b91a-9e60b8e44d67 tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 16 21:43:20.777812 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e21aaa74-7b76-4bdc-b299-fc492ec331a9 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:43:20.780407 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e21aaa74-7b76-4bdc-b299-fc492ec331a9 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:43:20.803550 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-87a95e61-4d20-4c18-b91a-9e60b8e44d67 tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:43:20.810208 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-87a95e61-4d20-4c18-b91a-9e60b8e44d67 tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:43:20.811254 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-87a95e61-4d20-4c18-b91a-9e60b8e44d67 tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:43:20.811841 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-87a95e61-4d20-4c18-b91a-9e60b8e44d67 tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:43:20.885313 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-87a95e61-4d20-4c18-b91a-9e60b8e44d67 tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Getting quotas for project 251f01f982aa43ad853d53fa06f99190. Resources: {'ram', 'instances', 'cores'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:43:20.901944 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-87a95e61-4d20-4c18-b91a-9e60b8e44d67 tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Getting quotas for user 82819d3efcda4fe78e2b50bb7fab0a96 and project 251f01f982aa43ad853d53fa06f99190. Resources: {'ram', 'instances', 'cores'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:43:20.951567 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-87a95e61-4d20-4c18-b91a-9e60b8e44d67 tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Going to run 1 instances... {{(pid=138680) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 16 21:43:20.954474 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-87a95e61-4d20-4c18-b91a-9e60b8e44d67 tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] [instance: a553b3eb-ae00-4d00-97d1-1d26fe2bdfff] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138680) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 16 21:43:21.140824 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-87a95e61-4d20-4c18-b91a-9e60b8e44d67 tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 361 microversion: 2.1 time: 2.120370 Jul 16 21:43:21.141918 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2805/5613] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 21:43:19 2022] POST /compute/v2.1/servers => generated 361 bytes in 2122 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 16 21:43:21.166124 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-bdb29e97-e02d-4fb4-84da-755345b87d21 tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:43:21.171933 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-bdb29e97-e02d-4fb4-84da-755345b87d21 tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] [instance: a553b3eb-ae00-4d00-97d1-1d26fe2bdfff] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:43:21.417980 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-e21aaa74-7b76-4bdc-b299-fc492ec331a9 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '726b46c54e9541fbb7825607bac3abe5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fd0ce287cb44cd59045868ce688909a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:43:21.436379 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-e21aaa74-7b76-4bdc-b299-fc492ec331a9 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '726b46c54e9541fbb7825607bac3abe5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fd0ce287cb44cd59045868ce688909a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:43:21.437906 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-e21aaa74-7b76-4bdc-b299-fc492ec331a9 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '726b46c54e9541fbb7825607bac3abe5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fd0ce287cb44cd59045868ce688909a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:43:21.439388 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-e21aaa74-7b76-4bdc-b299-fc492ec331a9 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] 10.222.0.16 "GET /compute/v2.1/servers/19df5c33-44d4-4de9-947b-4707b78af23a" status: 200 len: 2958 microversion: 2.42 time: 0.701283 Jul 16 21:43:21.439970 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2809/5614] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:43:20 2022] GET /compute/v2.1/servers/19df5c33-44d4-4de9-947b-4707b78af23a => generated 2958 bytes in 703 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 16 21:43:21.448693 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-bdb29e97-e02d-4fb4-84da-755345b87d21 tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '82819d3efcda4fe78e2b50bb7fab0a96', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '251f01f982aa43ad853d53fa06f99190', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:43:21.458712 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-bdb29e97-e02d-4fb4-84da-755345b87d21 tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] 10.222.0.16 "GET /compute/v2.1/servers/a553b3eb-ae00-4d00-97d1-1d26fe2bdfff" status: 200 len: 1242 microversion: 2.1 time: 0.303428 Jul 16 21:43:21.465056 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2806/5615] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:43:21 2022] GET /compute/v2.1/servers/a553b3eb-ae00-4d00-97d1-1d26fe2bdfff => generated 1242 bytes in 315 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:43:22.463919 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-98c9b360-163c-4451-b348-269831bccac7 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:43:22.464554 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-98c9b360-163c-4451-b348-269831bccac7 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] [instance: 19df5c33-44d4-4de9-947b-4707b78af23a] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:43:22.473272 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-98c9b360-163c-4451-b348-269831bccac7 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:43:22.476612 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-98c9b360-163c-4451-b348-269831bccac7 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:43:22.480976 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-98c9b360-163c-4451-b348-269831bccac7 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:43:22.481809 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-98c9b360-163c-4451-b348-269831bccac7 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:43:22.489281 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-b4b57edc-39de-4977-bc45-12d3ad2bd549 tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:43:22.489281 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-b4b57edc-39de-4977-bc45-12d3ad2bd549 tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] [instance: a553b3eb-ae00-4d00-97d1-1d26fe2bdfff] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:43:22.503372 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b4b57edc-39de-4977-bc45-12d3ad2bd549 tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:43:22.504633 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b4b57edc-39de-4977-bc45-12d3ad2bd549 tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:43:22.506155 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b4b57edc-39de-4977-bc45-12d3ad2bd549 tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:43:22.506824 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b4b57edc-39de-4977-bc45-12d3ad2bd549 tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:43:23.029088 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-b4b57edc-39de-4977-bc45-12d3ad2bd549 tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '82819d3efcda4fe78e2b50bb7fab0a96', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '251f01f982aa43ad853d53fa06f99190', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:43:23.037478 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-b4b57edc-39de-4977-bc45-12d3ad2bd549 tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] 10.222.0.16 "GET /compute/v2.1/servers/a553b3eb-ae00-4d00-97d1-1d26fe2bdfff" status: 200 len: 1246 microversion: 2.1 time: 0.556777 Jul 16 21:43:23.038059 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2807/5616] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:43:22 2022] GET /compute/v2.1/servers/a553b3eb-ae00-4d00-97d1-1d26fe2bdfff => generated 1246 bytes in 558 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:43:23.410126 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-98c9b360-163c-4451-b348-269831bccac7 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '726b46c54e9541fbb7825607bac3abe5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fd0ce287cb44cd59045868ce688909a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:43:23.424250 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-98c9b360-163c-4451-b348-269831bccac7 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '726b46c54e9541fbb7825607bac3abe5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fd0ce287cb44cd59045868ce688909a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:43:23.427293 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-98c9b360-163c-4451-b348-269831bccac7 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '726b46c54e9541fbb7825607bac3abe5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fd0ce287cb44cd59045868ce688909a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:43:23.431627 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-98c9b360-163c-4451-b348-269831bccac7 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] 10.222.0.16 "GET /compute/v2.1/servers/19df5c33-44d4-4de9-947b-4707b78af23a" status: 200 len: 2937 microversion: 2.42 time: 0.969890 Jul 16 21:43:23.431627 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2810/5617] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:43:22 2022] GET /compute/v2.1/servers/19df5c33-44d4-4de9-947b-4707b78af23a => generated 2937 bytes in 971 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:43:23.914151 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-7999ef4a-2318-495b-bf7f-2c7a13d72c0b tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:43:23.914768 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-7999ef4a-2318-495b-bf7f-2c7a13d72c0b tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] [instance: 64b5a7a2-7d82-4c60-802e-8e92c14e0790] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:43:23.940707 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7999ef4a-2318-495b-bf7f-2c7a13d72c0b tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:43:23.941159 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7999ef4a-2318-495b-bf7f-2c7a13d72c0b tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:43:23.942472 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7999ef4a-2318-495b-bf7f-2c7a13d72c0b tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:43:23.946454 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7999ef4a-2318-495b-bf7f-2c7a13d72c0b tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:43:24.048862 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-eb00ebb7-1156-466b-99cc-8d8a2bc07fe4 req-fabefb75-4afe-419a-872b-b12335c02d30 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "64b5a7a2-7d82-4c60-802e-8e92c14e0790", "tag": "7a7ec2ad-db68-4392-93f0-85d2811e10e1"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:43:24.062253 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-eb00ebb7-1156-466b-99cc-8d8a2bc07fe4 req-fabefb75-4afe-419a-872b-b12335c02d30 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:43:24.063106 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-eb00ebb7-1156-466b-99cc-8d8a2bc07fe4 req-fabefb75-4afe-419a-872b-b12335c02d30 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:43:24.083591 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-eb00ebb7-1156-466b-99cc-8d8a2bc07fe4 req-fabefb75-4afe-419a-872b-b12335c02d30 service nova] Creating event network-changed:7a7ec2ad-db68-4392-93f0-85d2811e10e1 for instance 64b5a7a2-7d82-4c60-802e-8e92c14e0790 on nch1-master-1 Jul 16 21:43:24.090646 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-eb00ebb7-1156-466b-99cc-8d8a2bc07fe4 req-fabefb75-4afe-419a-872b-b12335c02d30 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.045627 Jul 16 21:43:24.092382 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2811/5618] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:43:24 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:43:24.099498 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-ae39cf08-ce60-439f-a81b-a381554bc4a7 tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:43:24.100641 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-ae39cf08-ce60-439f-a81b-a381554bc4a7 tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] [instance: a553b3eb-ae00-4d00-97d1-1d26fe2bdfff] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:43:24.109931 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ae39cf08-ce60-439f-a81b-a381554bc4a7 tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:43:24.110886 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ae39cf08-ce60-439f-a81b-a381554bc4a7 tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:43:24.112528 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ae39cf08-ce60-439f-a81b-a381554bc4a7 tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:43:24.113350 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ae39cf08-ce60-439f-a81b-a381554bc4a7 tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:43:24.313255 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-ae39cf08-ce60-439f-a81b-a381554bc4a7 tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '82819d3efcda4fe78e2b50bb7fab0a96', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '251f01f982aa43ad853d53fa06f99190', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:43:24.327314 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-ae39cf08-ce60-439f-a81b-a381554bc4a7 tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] 10.222.0.16 "GET /compute/v2.1/servers/a553b3eb-ae00-4d00-97d1-1d26fe2bdfff" status: 200 len: 1312 microversion: 2.1 time: 0.232914 Jul 16 21:43:24.328848 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2812/5619] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:43:24 2022] GET /compute/v2.1/servers/a553b3eb-ae00-4d00-97d1-1d26fe2bdfff => generated 1312 bytes in 236 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:43:24.451142 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-195c143d-9e6d-4ee7-9c47-d788182c1016 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:43:24.451790 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-195c143d-9e6d-4ee7-9c47-d788182c1016 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] [instance: 19df5c33-44d4-4de9-947b-4707b78af23a] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:43:24.456783 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-7999ef4a-2318-495b-bf7f-2c7a13d72c0b tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '833136cbdeab4b9e8b6dd2d24c7c8753', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '77c9b7e351e24dc88207e9498f75f578', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:43:24.463173 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-195c143d-9e6d-4ee7-9c47-d788182c1016 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:43:24.463643 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-195c143d-9e6d-4ee7-9c47-d788182c1016 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:43:24.464449 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-195c143d-9e6d-4ee7-9c47-d788182c1016 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:43:24.469003 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-7999ef4a-2318-495b-bf7f-2c7a13d72c0b tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] 10.222.0.16 "GET /compute/v2.1/servers/64b5a7a2-7d82-4c60-802e-8e92c14e0790" status: 200 len: 1563 microversion: 2.1 time: 0.560625 Jul 16 21:43:24.469862 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-195c143d-9e6d-4ee7-9c47-d788182c1016 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:43:24.485202 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2808/5620] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:43:23 2022] GET /compute/v2.1/servers/64b5a7a2-7d82-4c60-802e-8e92c14e0790 => generated 1563 bytes in 579 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:43:25.015129 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-195c143d-9e6d-4ee7-9c47-d788182c1016 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '726b46c54e9541fbb7825607bac3abe5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fd0ce287cb44cd59045868ce688909a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:43:25.029094 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-195c143d-9e6d-4ee7-9c47-d788182c1016 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '726b46c54e9541fbb7825607bac3abe5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fd0ce287cb44cd59045868ce688909a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:43:25.030436 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-195c143d-9e6d-4ee7-9c47-d788182c1016 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '726b46c54e9541fbb7825607bac3abe5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fd0ce287cb44cd59045868ce688909a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:43:25.039718 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-195c143d-9e6d-4ee7-9c47-d788182c1016 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] 10.222.0.16 "GET /compute/v2.1/servers/19df5c33-44d4-4de9-947b-4707b78af23a" status: 200 len: 2916 microversion: 2.42 time: 0.590661 Jul 16 21:43:25.046015 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2813/5621] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:43:24 2022] GET /compute/v2.1/servers/19df5c33-44d4-4de9-947b-4707b78af23a => generated 2916 bytes in 601 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 16 21:43:25.262522 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-942982e0-2ac2-420d-ac20-00d21616fef5 req-d083ac08-4f03-4610-be3e-6cd7c7780ed6 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "19df5c33-44d4-4de9-947b-4707b78af23a", "name": "network-vif-unplugged", "status": "completed", "tag": "4c5dcf45-0de2-481d-9d69-0ca39f95898b"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:43:25.278594 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-942982e0-2ac2-420d-ac20-00d21616fef5 req-d083ac08-4f03-4610-be3e-6cd7c7780ed6 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:43:25.279565 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-942982e0-2ac2-420d-ac20-00d21616fef5 req-d083ac08-4f03-4610-be3e-6cd7c7780ed6 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:43:25.300169 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-942982e0-2ac2-420d-ac20-00d21616fef5 req-d083ac08-4f03-4610-be3e-6cd7c7780ed6 service nova] Creating event network-vif-unplugged:4c5dcf45-0de2-481d-9d69-0ca39f95898b for instance 19df5c33-44d4-4de9-947b-4707b78af23a on nch2-master-1 Jul 16 21:43:25.305749 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-942982e0-2ac2-420d-ac20-00d21616fef5 req-d083ac08-4f03-4610-be3e-6cd7c7780ed6 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.048989 Jul 16 21:43:25.307158 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2809/5622] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:43:25 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:43:25.346151 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-977698b8-c249-4ce2-9402-69242339d539 tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:43:25.346790 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-977698b8-c249-4ce2-9402-69242339d539 tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] [instance: a553b3eb-ae00-4d00-97d1-1d26fe2bdfff] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:43:25.357159 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-977698b8-c249-4ce2-9402-69242339d539 tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:43:25.369088 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-977698b8-c249-4ce2-9402-69242339d539 tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:43:25.369088 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-977698b8-c249-4ce2-9402-69242339d539 tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:43:25.369088 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-977698b8-c249-4ce2-9402-69242339d539 tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:43:25.632702 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-977698b8-c249-4ce2-9402-69242339d539 tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '82819d3efcda4fe78e2b50bb7fab0a96', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '251f01f982aa43ad853d53fa06f99190', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:43:25.646317 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-977698b8-c249-4ce2-9402-69242339d539 tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] 10.222.0.16 "GET /compute/v2.1/servers/a553b3eb-ae00-4d00-97d1-1d26fe2bdfff" status: 200 len: 1300 microversion: 2.1 time: 0.304214 Jul 16 21:43:25.658963 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2814/5623] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:43:25 2022] GET /compute/v2.1/servers/a553b3eb-ae00-4d00-97d1-1d26fe2bdfff => generated 1300 bytes in 318 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:43:26.007871 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-ba504380-db44-46c0-a60b-155ec4b121c2 req-13c9a9f0-529d-46be-9778-c87cf4e5a5b6 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "19df5c33-44d4-4de9-947b-4707b78af23a", "name": "network-vif-deleted", "tag": "4c5dcf45-0de2-481d-9d69-0ca39f95898b"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:43:26.039465 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-ba504380-db44-46c0-a60b-155ec4b121c2 req-13c9a9f0-529d-46be-9778-c87cf4e5a5b6 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:43:26.041285 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-ba504380-db44-46c0-a60b-155ec4b121c2 req-13c9a9f0-529d-46be-9778-c87cf4e5a5b6 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:43:26.067607 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-7d339b38-c43a-4e81-a164-8c72603fcd5d tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:43:26.070046 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-7d339b38-c43a-4e81-a164-8c72603fcd5d tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] [instance: 19df5c33-44d4-4de9-947b-4707b78af23a] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:43:26.085252 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7d339b38-c43a-4e81-a164-8c72603fcd5d tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:43:26.090195 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7d339b38-c43a-4e81-a164-8c72603fcd5d tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:43:26.091397 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7d339b38-c43a-4e81-a164-8c72603fcd5d tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:43:26.094409 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7d339b38-c43a-4e81-a164-8c72603fcd5d tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:43:26.102271 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-ba504380-db44-46c0-a60b-155ec4b121c2 req-13c9a9f0-529d-46be-9778-c87cf4e5a5b6 service nova] Creating event network-vif-deleted:4c5dcf45-0de2-481d-9d69-0ca39f95898b for instance 19df5c33-44d4-4de9-947b-4707b78af23a on nch2-master-1 Jul 16 21:43:26.121060 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-ba504380-db44-46c0-a60b-155ec4b121c2 req-13c9a9f0-529d-46be-9778-c87cf4e5a5b6 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.120942 Jul 16 21:43:26.122713 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2810/5624] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:43:25 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 124 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 16 21:43:26.498813 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-7d339b38-c43a-4e81-a164-8c72603fcd5d tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '726b46c54e9541fbb7825607bac3abe5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fd0ce287cb44cd59045868ce688909a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:43:26.524021 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-7d339b38-c43a-4e81-a164-8c72603fcd5d tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '726b46c54e9541fbb7825607bac3abe5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fd0ce287cb44cd59045868ce688909a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:43:26.528237 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-7d339b38-c43a-4e81-a164-8c72603fcd5d tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '726b46c54e9541fbb7825607bac3abe5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fd0ce287cb44cd59045868ce688909a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:43:26.530050 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-7d339b38-c43a-4e81-a164-8c72603fcd5d tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] 10.222.0.16 "GET /compute/v2.1/servers/19df5c33-44d4-4de9-947b-4707b78af23a" status: 200 len: 2870 microversion: 2.42 time: 0.466719 Jul 16 21:43:26.543946 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2815/5625] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:43:26 2022] GET /compute/v2.1/servers/19df5c33-44d4-4de9-947b-4707b78af23a => generated 2870 bytes in 482 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 16 21:43:26.684888 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-4a266ce4-faaf-4d1c-bf76-99e34708bd24 tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:43:26.686319 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-4a266ce4-faaf-4d1c-bf76-99e34708bd24 tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] [instance: a553b3eb-ae00-4d00-97d1-1d26fe2bdfff] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:43:26.700623 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4a266ce4-faaf-4d1c-bf76-99e34708bd24 tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:43:26.701379 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4a266ce4-faaf-4d1c-bf76-99e34708bd24 tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:43:26.702563 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4a266ce4-faaf-4d1c-bf76-99e34708bd24 tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:43:26.703504 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4a266ce4-faaf-4d1c-bf76-99e34708bd24 tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:43:26.979607 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-aebc8745-09eb-40ff-8b8e-887feb98a4c9 req-2f71bf35-24f1-4498-a3da-cbf1a1315b4e service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "a553b3eb-ae00-4d00-97d1-1d26fe2bdfff", "tag": "db2d4974-12e5-4ee1-82ef-bc072dec3b32"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:43:26.996543 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-aebc8745-09eb-40ff-8b8e-887feb98a4c9 req-2f71bf35-24f1-4498-a3da-cbf1a1315b4e service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:43:27.005565 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-aebc8745-09eb-40ff-8b8e-887feb98a4c9 req-2f71bf35-24f1-4498-a3da-cbf1a1315b4e service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.009s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:43:27.040899 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-aebc8745-09eb-40ff-8b8e-887feb98a4c9 req-2f71bf35-24f1-4498-a3da-cbf1a1315b4e service nova] Creating event network-changed:db2d4974-12e5-4ee1-82ef-bc072dec3b32 for instance a553b3eb-ae00-4d00-97d1-1d26fe2bdfff on nch1-master-1 Jul 16 21:43:27.050974 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-aebc8745-09eb-40ff-8b8e-887feb98a4c9 req-2f71bf35-24f1-4498-a3da-cbf1a1315b4e service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.075677 Jul 16 21:43:27.059144 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2816/5626] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:43:26 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:43:27.181384 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-4a266ce4-faaf-4d1c-bf76-99e34708bd24 tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '82819d3efcda4fe78e2b50bb7fab0a96', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '251f01f982aa43ad853d53fa06f99190', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:43:27.191866 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-4a266ce4-faaf-4d1c-bf76-99e34708bd24 tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] 10.222.0.16 "GET /compute/v2.1/servers/a553b3eb-ae00-4d00-97d1-1d26fe2bdfff" status: 200 len: 1300 microversion: 2.1 time: 0.527905 Jul 16 21:43:27.202400 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2811/5627] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:43:26 2022] GET /compute/v2.1/servers/a553b3eb-ae00-4d00-97d1-1d26fe2bdfff => generated 1300 bytes in 538 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:43:27.332156 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-b6ffed41-fbe0-4bfc-bf61-ffe0d457cbe3 req-7b8d4d17-de6e-4422-8a2b-de77beed2aef service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "19df5c33-44d4-4de9-947b-4707b78af23a", "name": "network-vif-unplugged", "status": "completed", "tag": "7d150f5e-be69-4d03-aff9-6fc347935783"}, {"server_uuid": "19df5c33-44d4-4de9-947b-4707b78af23a", "name": "network-vif-unplugged", "status": "completed", "tag": "535c3524-f5da-4ba5-a149-40d9c379754e"}, {"server_uuid": "19df5c33-44d4-4de9-947b-4707b78af23a", "name": "network-vif-unplugged", "status": "completed", "tag": "25ef7728-f1b7-4467-9f81-8af48bbcef2d"}, {"server_uuid": "19df5c33-44d4-4de9-947b-4707b78af23a", "name": "network-vif-unplugged", "status": "completed", "tag": "e518c0f5-c1c1-47b3-b1e8-743d59bf079a"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:43:27.350531 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-b6ffed41-fbe0-4bfc-bf61-ffe0d457cbe3 req-7b8d4d17-de6e-4422-8a2b-de77beed2aef service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:43:27.350531 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-b6ffed41-fbe0-4bfc-bf61-ffe0d457cbe3 req-7b8d4d17-de6e-4422-8a2b-de77beed2aef service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:43:27.359906 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-b6ffed41-fbe0-4bfc-bf61-ffe0d457cbe3 req-7b8d4d17-de6e-4422-8a2b-de77beed2aef service nova] Creating event network-vif-unplugged:7d150f5e-be69-4d03-aff9-6fc347935783 for instance 19df5c33-44d4-4de9-947b-4707b78af23a on nch2-master-1 Jul 16 21:43:27.360590 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-b6ffed41-fbe0-4bfc-bf61-ffe0d457cbe3 req-7b8d4d17-de6e-4422-8a2b-de77beed2aef service nova] Creating event network-vif-unplugged:535c3524-f5da-4ba5-a149-40d9c379754e for instance 19df5c33-44d4-4de9-947b-4707b78af23a on nch2-master-1 Jul 16 21:43:27.361179 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-b6ffed41-fbe0-4bfc-bf61-ffe0d457cbe3 req-7b8d4d17-de6e-4422-8a2b-de77beed2aef service nova] Creating event network-vif-unplugged:25ef7728-f1b7-4467-9f81-8af48bbcef2d for instance 19df5c33-44d4-4de9-947b-4707b78af23a on nch2-master-1 Jul 16 21:43:27.361733 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-b6ffed41-fbe0-4bfc-bf61-ffe0d457cbe3 req-7b8d4d17-de6e-4422-8a2b-de77beed2aef service nova] Creating event network-vif-unplugged:e518c0f5-c1c1-47b3-b1e8-743d59bf079a for instance 19df5c33-44d4-4de9-947b-4707b78af23a on nch2-master-1 Jul 16 21:43:27.369207 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-b6ffed41-fbe0-4bfc-bf61-ffe0d457cbe3 req-7b8d4d17-de6e-4422-8a2b-de77beed2aef service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 704 microversion: 2.1 time: 0.044146 Jul 16 21:43:27.369845 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2817/5628] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:43:27 2022] POST /compute/v2.1/os-server-external-events => generated 704 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:43:27.554089 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-13653127-5354-4422-8581-4bfd842358bf tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:43:27.554708 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-13653127-5354-4422-8581-4bfd842358bf tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] [instance: 19df5c33-44d4-4de9-947b-4707b78af23a] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:43:27.566098 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-13653127-5354-4422-8581-4bfd842358bf tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:43:27.566605 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-13653127-5354-4422-8581-4bfd842358bf tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:43:27.568960 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-13653127-5354-4422-8581-4bfd842358bf tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:43:27.569412 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-13653127-5354-4422-8581-4bfd842358bf tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:43:27.812720 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-13653127-5354-4422-8581-4bfd842358bf tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '726b46c54e9541fbb7825607bac3abe5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fd0ce287cb44cd59045868ce688909a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:43:27.829641 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-13653127-5354-4422-8581-4bfd842358bf tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '726b46c54e9541fbb7825607bac3abe5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fd0ce287cb44cd59045868ce688909a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:43:27.831912 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-13653127-5354-4422-8581-4bfd842358bf tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '726b46c54e9541fbb7825607bac3abe5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fd0ce287cb44cd59045868ce688909a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:43:27.833188 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-13653127-5354-4422-8581-4bfd842358bf tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] 10.222.0.16 "GET /compute/v2.1/servers/19df5c33-44d4-4de9-947b-4707b78af23a" status: 200 len: 2824 microversion: 2.42 time: 0.283085 Jul 16 21:43:27.833927 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2812/5629] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:43:27 2022] GET /compute/v2.1/servers/19df5c33-44d4-4de9-947b-4707b78af23a => generated 2824 bytes in 284 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:43:28.172121 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-3c404700-f67d-48ee-968d-5332d4f1cabe req-d0c32596-c28e-4d38-9d16-350875de5f29 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "19df5c33-44d4-4de9-947b-4707b78af23a", "name": "network-vif-deleted", "tag": "7d150f5e-be69-4d03-aff9-6fc347935783"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:43:28.180569 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-3c404700-f67d-48ee-968d-5332d4f1cabe req-d0c32596-c28e-4d38-9d16-350875de5f29 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:43:28.181829 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-3c404700-f67d-48ee-968d-5332d4f1cabe req-d0c32596-c28e-4d38-9d16-350875de5f29 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:43:28.196285 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-3c404700-f67d-48ee-968d-5332d4f1cabe req-d0c32596-c28e-4d38-9d16-350875de5f29 service nova] Creating event network-vif-deleted:7d150f5e-be69-4d03-aff9-6fc347935783 for instance 19df5c33-44d4-4de9-947b-4707b78af23a on nch2-master-1 Jul 16 21:43:28.202857 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-3c404700-f67d-48ee-968d-5332d4f1cabe req-d0c32596-c28e-4d38-9d16-350875de5f29 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.035949 Jul 16 21:43:28.203593 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2818/5630] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:43:28 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:43:28.210155 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-457a8f51-d552-491b-9627-83ba6d67c39d tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:43:28.211645 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-457a8f51-d552-491b-9627-83ba6d67c39d tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] [instance: a553b3eb-ae00-4d00-97d1-1d26fe2bdfff] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:43:28.220408 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-457a8f51-d552-491b-9627-83ba6d67c39d tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:43:28.221289 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-457a8f51-d552-491b-9627-83ba6d67c39d tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:43:28.222130 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-457a8f51-d552-491b-9627-83ba6d67c39d tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:43:28.223215 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-457a8f51-d552-491b-9627-83ba6d67c39d tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:43:28.227903 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=89,nova_cell1:INSERT=2,nova_cell1:UPDATE=1 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:43:28.341456 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-457a8f51-d552-491b-9627-83ba6d67c39d tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '82819d3efcda4fe78e2b50bb7fab0a96', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '251f01f982aa43ad853d53fa06f99190', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:43:28.349051 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-457a8f51-d552-491b-9627-83ba6d67c39d tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] 10.222.0.16 "GET /compute/v2.1/servers/a553b3eb-ae00-4d00-97d1-1d26fe2bdfff" status: 200 len: 1300 microversion: 2.1 time: 0.143139 Jul 16 21:43:28.352110 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2813/5631] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:43:28 2022] GET /compute/v2.1/servers/a553b3eb-ae00-4d00-97d1-1d26fe2bdfff => generated 1300 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:43:28.857529 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-25bdbf26-1cd0-41b9-bde7-975cd832d759 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:43:28.858630 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-25bdbf26-1cd0-41b9-bde7-975cd832d759 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] [instance: 19df5c33-44d4-4de9-947b-4707b78af23a] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:43:28.871872 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-25bdbf26-1cd0-41b9-bde7-975cd832d759 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:43:28.877212 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-25bdbf26-1cd0-41b9-bde7-975cd832d759 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:43:28.877212 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-25bdbf26-1cd0-41b9-bde7-975cd832d759 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:43:28.877212 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-25bdbf26-1cd0-41b9-bde7-975cd832d759 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:43:29.139514 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-25bdbf26-1cd0-41b9-bde7-975cd832d759 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '726b46c54e9541fbb7825607bac3abe5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fd0ce287cb44cd59045868ce688909a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:43:29.148782 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-25bdbf26-1cd0-41b9-bde7-975cd832d759 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '726b46c54e9541fbb7825607bac3abe5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fd0ce287cb44cd59045868ce688909a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:43:29.151263 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-25bdbf26-1cd0-41b9-bde7-975cd832d759 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '726b46c54e9541fbb7825607bac3abe5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fd0ce287cb44cd59045868ce688909a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:43:29.152509 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-25bdbf26-1cd0-41b9-bde7-975cd832d759 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] 10.222.0.16 "GET /compute/v2.1/servers/19df5c33-44d4-4de9-947b-4707b78af23a" status: 200 len: 2485 microversion: 2.42 time: 0.302431 Jul 16 21:43:29.153396 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2819/5632] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:43:28 2022] GET /compute/v2.1/servers/19df5c33-44d4-4de9-947b-4707b78af23a => generated 2485 bytes in 304 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:43:29.369975 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-9e21ea3e-8360-406b-a15a-6cd5fb4e051d tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:43:29.371206 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-9e21ea3e-8360-406b-a15a-6cd5fb4e051d tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] [instance: a553b3eb-ae00-4d00-97d1-1d26fe2bdfff] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:43:29.378957 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9e21ea3e-8360-406b-a15a-6cd5fb4e051d tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:43:29.380120 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9e21ea3e-8360-406b-a15a-6cd5fb4e051d tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:43:29.380977 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9e21ea3e-8360-406b-a15a-6cd5fb4e051d tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:43:29.381586 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9e21ea3e-8360-406b-a15a-6cd5fb4e051d tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:43:29.508109 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-9e21ea3e-8360-406b-a15a-6cd5fb4e051d tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '82819d3efcda4fe78e2b50bb7fab0a96', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '251f01f982aa43ad853d53fa06f99190', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:43:29.516791 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-9e21ea3e-8360-406b-a15a-6cd5fb4e051d tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] 10.222.0.16 "GET /compute/v2.1/servers/a553b3eb-ae00-4d00-97d1-1d26fe2bdfff" status: 200 len: 1300 microversion: 2.1 time: 0.151418 Jul 16 21:43:29.526524 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2814/5633] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:43:29 2022] GET /compute/v2.1/servers/a553b3eb-ae00-4d00-97d1-1d26fe2bdfff => generated 1300 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:43:30.172749 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-95067a00-c7a1-4f5a-aaf2-b14965eeb3a8 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:43:30.173377 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-95067a00-c7a1-4f5a-aaf2-b14965eeb3a8 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] [instance: 19df5c33-44d4-4de9-947b-4707b78af23a] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:43:30.180335 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-95067a00-c7a1-4f5a-aaf2-b14965eeb3a8 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:43:30.181308 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-95067a00-c7a1-4f5a-aaf2-b14965eeb3a8 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:43:30.182176 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-95067a00-c7a1-4f5a-aaf2-b14965eeb3a8 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:43:30.182623 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-95067a00-c7a1-4f5a-aaf2-b14965eeb3a8 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:43:30.227259 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-7a199dce-36d9-4573-98bb-c645036db808 req-9c4faf9a-17fd-4066-b35e-cac0ee70fb40 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "19df5c33-44d4-4de9-947b-4707b78af23a", "name": "network-vif-deleted", "tag": "535c3524-f5da-4ba5-a149-40d9c379754e"}, {"server_uuid": "19df5c33-44d4-4de9-947b-4707b78af23a", "name": "network-vif-deleted", "tag": "25ef7728-f1b7-4467-9f81-8af48bbcef2d"}, {"server_uuid": "19df5c33-44d4-4de9-947b-4707b78af23a", "name": "network-vif-deleted", "tag": "e518c0f5-c1c1-47b3-b1e8-743d59bf079a"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:43:30.236769 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-7a199dce-36d9-4573-98bb-c645036db808 req-9c4faf9a-17fd-4066-b35e-cac0ee70fb40 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:43:30.237369 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-7a199dce-36d9-4573-98bb-c645036db808 req-9c4faf9a-17fd-4066-b35e-cac0ee70fb40 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:43:30.249586 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-7a199dce-36d9-4573-98bb-c645036db808 req-9c4faf9a-17fd-4066-b35e-cac0ee70fb40 service nova] Creating event network-vif-deleted:535c3524-f5da-4ba5-a149-40d9c379754e for instance 19df5c33-44d4-4de9-947b-4707b78af23a on nch2-master-1 Jul 16 21:43:30.250122 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-7a199dce-36d9-4573-98bb-c645036db808 req-9c4faf9a-17fd-4066-b35e-cac0ee70fb40 service nova] Creating event network-vif-deleted:25ef7728-f1b7-4467-9f81-8af48bbcef2d for instance 19df5c33-44d4-4de9-947b-4707b78af23a on nch2-master-1 Jul 16 21:43:30.250637 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-7a199dce-36d9-4573-98bb-c645036db808 req-9c4faf9a-17fd-4066-b35e-cac0ee70fb40 service nova] Creating event network-vif-deleted:e518c0f5-c1c1-47b3-b1e8-743d59bf079a for instance 19df5c33-44d4-4de9-947b-4707b78af23a on nch2-master-1 Jul 16 21:43:30.259581 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-7a199dce-36d9-4573-98bb-c645036db808 req-9c4faf9a-17fd-4066-b35e-cac0ee70fb40 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 525 microversion: 2.1 time: 0.032850 Jul 16 21:43:30.259581 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2815/5634] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:43:30 2022] POST /compute/v2.1/os-server-external-events => generated 525 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 16 21:43:30.260369 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-95067a00-c7a1-4f5a-aaf2-b14965eeb3a8 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '726b46c54e9541fbb7825607bac3abe5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fd0ce287cb44cd59045868ce688909a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:43:30.270034 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-95067a00-c7a1-4f5a-aaf2-b14965eeb3a8 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '726b46c54e9541fbb7825607bac3abe5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fd0ce287cb44cd59045868ce688909a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:43:30.271577 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-95067a00-c7a1-4f5a-aaf2-b14965eeb3a8 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '726b46c54e9541fbb7825607bac3abe5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0fd0ce287cb44cd59045868ce688909a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:43:30.272761 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-95067a00-c7a1-4f5a-aaf2-b14965eeb3a8 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] 10.222.0.16 "GET /compute/v2.1/servers/19df5c33-44d4-4de9-947b-4707b78af23a" status: 200 len: 1604 microversion: 2.42 time: 0.104031 Jul 16 21:43:30.273327 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2820/5635] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:43:30 2022] GET /compute/v2.1/servers/19df5c33-44d4-4de9-947b-4707b78af23a => generated 1604 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 16 21:43:30.535263 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-887dab0b-bdf9-47d7-b9e1-076330aa3a7e tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:43:30.535966 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-887dab0b-bdf9-47d7-b9e1-076330aa3a7e tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] [instance: a553b3eb-ae00-4d00-97d1-1d26fe2bdfff] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:43:30.543340 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-887dab0b-bdf9-47d7-b9e1-076330aa3a7e tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:43:30.543805 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-887dab0b-bdf9-47d7-b9e1-076330aa3a7e tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:43:30.549203 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-887dab0b-bdf9-47d7-b9e1-076330aa3a7e tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:43:30.549203 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-887dab0b-bdf9-47d7-b9e1-076330aa3a7e tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:43:30.633766 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-887dab0b-bdf9-47d7-b9e1-076330aa3a7e tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '82819d3efcda4fe78e2b50bb7fab0a96', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '251f01f982aa43ad853d53fa06f99190', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:43:30.641884 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-887dab0b-bdf9-47d7-b9e1-076330aa3a7e tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] 10.222.0.16 "GET /compute/v2.1/servers/a553b3eb-ae00-4d00-97d1-1d26fe2bdfff" status: 200 len: 1300 microversion: 2.1 time: 0.111820 Jul 16 21:43:30.642457 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2816/5636] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:43:30 2022] GET /compute/v2.1/servers/a553b3eb-ae00-4d00-97d1-1d26fe2bdfff => generated 1300 bytes in 113 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:43:31.294210 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-be7d1d6a-6540-431a-923d-238c6180d259 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:43:31.294843 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-be7d1d6a-6540-431a-923d-238c6180d259 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] [instance: 19df5c33-44d4-4de9-947b-4707b78af23a] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:43:31.301916 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-be7d1d6a-6540-431a-923d-238c6180d259 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:43:31.302375 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-be7d1d6a-6540-431a-923d-238c6180d259 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:43:31.303191 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-be7d1d6a-6540-431a-923d-238c6180d259 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:43:31.303641 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-be7d1d6a-6540-431a-923d-238c6180d259 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:43:31.318470 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-be7d1d6a-6540-431a-923d-238c6180d259 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] HTTP exception thrown: Instance 19df5c33-44d4-4de9-947b-4707b78af23a could not be found. Jul 16 21:43:31.320175 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-be7d1d6a-6540-431a-923d-238c6180d259 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Request method failure captured: Jul 16 21:43:31.320175 ncdv-master-1 devstack@n-api.service[138681]: request: GET /compute/v2.1/servers/19df5c33-44d4-4de9-947b-4707b78af23a HTTP/1.1 Jul 16 21:43:31.320175 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:43:31.320175 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:43:31.320175 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:43:31.320175 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 21:43:31.320175 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:43:31.320175 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:43:31.320175 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:43:31.320175 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:43:31.320175 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0y7oD7aE1hTE8r7OjyFbjYd0FdMCbPYyBooSPqOkp_2b9POAeXgACjU8xEBWz7iMqVilEKMKPj2ty40cW4kfsNkUDdUoat-bnerVh7GrjaOzXaEErnsHqdP2Ut1wSSzYvDmsSon-NGtw0hNe9Dsx6JLUZSz4vYRjTLBIzBNhavhQilg Jul 16 21:43:31.320175 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:43:31.320175 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:43:31.320175 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:43:31.320175 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:43:31.320175 ncdv-master-1 devstack@n-api.service[138681]: X-Openstack-Nova-Api-Version: 2.42 Jul 16 21:43:31.320175 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:43:31.320175 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:43:31.320175 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: 0fd0ce287cb44cd59045868ce688909a Jul 16 21:43:31.320175 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-TaggedBootDevicesTest_v242-896656767 Jul 16 21:43:31.320175 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:43:31.320175 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:43:31.320175 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/0fd0ce287cb44cd59045868ce688909a"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/0fd0ce287cb44cd59045868ce688909a"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_0fd0ce287cb44cd59045868ce688909a"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/0fd0ce287cb44cd59045868ce688909a"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:43:31.320175 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-TaggedBootDevicesTest_v242-896656767 Jul 16 21:43:31.320175 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: 0fd0ce287cb44cd59045868ce688909a Jul 16 21:43:31.320175 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-TaggedBootDevicesTest_v242-896656767 Jul 16 21:43:31.320175 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-TaggedBootDevicesTest_v242-896656767-project Jul 16 21:43:31.320175 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:43:31.320175 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:43:31.320175 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: 726b46c54e9541fbb7825607bac3abe5 Jul 16 21:43:31.320175 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-TaggedBootDevicesTest_v242-896656767-project Jul 16 21:43:31.320175 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:43:31.320175 ncdv-master-1 devstack@n-api.service[138681]: exception: Instance 19df5c33-44d4-4de9-947b-4707b78af23a could not be found. Jul 16 21:43:31.320175 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': '19df5c33-44d4-4de9-947b-4707b78af23a'} Jul 16 21:43:31.320175 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:43:31.320175 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:43:31.320175 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:43:31.320175 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:43:31.320175 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:43:31.320175 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:43:31.320175 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:43:31.320175 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:43:31.320175 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:43:31.320175 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:43:31.320175 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 19df5c33-44d4-4de9-947b-4707b78af23a could not be found. Jul 16 21:43:31.320175 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:43:31.320175 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:43:31.320175 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:43:31.320175 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:43:31.320175 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:43:31.320175 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:43:31.320175 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:43:31.320175 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:43:31.320175 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:43:31.320175 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:43:31.320175 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:43:31.320175 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:43:31.320175 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:43:31.320175 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:43:31.320175 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:43:31.320175 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:43:31.320175 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 19df5c33-44d4-4de9-947b-4707b78af23a could not be found. Jul 16 21:43:31.320175 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:43:31.320175 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:43:31.326522 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:43:31.326522 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:43:31.326522 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:43:31.326522 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:43:31.326522 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:43:31.326522 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:43:31.326522 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 19df5c33-44d4-4de9-947b-4707b78af23a could not be found. Jul 16 21:43:31.326522 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:43:31.326522 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-be7d1d6a-6540-431a-923d-238c6180d259 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Returning 404 to user: Instance 19df5c33-44d4-4de9-947b-4707b78af23a could not be found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:43:31.326522 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-be7d1d6a-6540-431a-923d-238c6180d259 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] 10.222.0.16 "GET /compute/v2.1/servers/19df5c33-44d4-4de9-947b-4707b78af23a" status: 404 len: 111 microversion: 2.42 time: 0.032359 Jul 16 21:43:31.326522 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2821/5637] 10.222.0.16 () {60 vars in 1276 bytes} [Sat Jul 16 21:43:31 2022] GET /compute/v2.1/servers/19df5c33-44d4-4de9-947b-4707b78af23a => generated 111 bytes in 34 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Jul 16 21:43:31.331248 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-64004357-8f36-4310-803d-9590cd7fd224 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] Calling method '.version_select at 0x7fdd8fcde280>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:43:31.334549 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell0:SELECT=2 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:43:31.454014 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-64004357-8f36-4310-803d-9590cd7fd224 tempest-TaggedBootDevicesTest_v242-896656767 tempest-TaggedBootDevicesTest_v242-896656767-project] 10.222.0.16 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1683927826" status: 204 len: 0 microversion: 2.42 time: 0.124443 Jul 16 21:43:31.454014 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2817/5638] 10.222.0.16 () {60 vars in 1261 bytes} [Sat Jul 16 21:43:31 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1683927826 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Jul 16 21:43:31.660690 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-1008cd9c-a4c9-484d-a78d-9988caef1ee9 tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:43:31.661326 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-1008cd9c-a4c9-484d-a78d-9988caef1ee9 tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] [instance: a553b3eb-ae00-4d00-97d1-1d26fe2bdfff] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:43:31.669072 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1008cd9c-a4c9-484d-a78d-9988caef1ee9 tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:43:31.669873 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1008cd9c-a4c9-484d-a78d-9988caef1ee9 tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:43:31.670497 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1008cd9c-a4c9-484d-a78d-9988caef1ee9 tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:43:31.670963 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1008cd9c-a4c9-484d-a78d-9988caef1ee9 tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:43:31.789843 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-1008cd9c-a4c9-484d-a78d-9988caef1ee9 tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '82819d3efcda4fe78e2b50bb7fab0a96', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '251f01f982aa43ad853d53fa06f99190', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:43:31.798519 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-1008cd9c-a4c9-484d-a78d-9988caef1ee9 tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] 10.222.0.16 "GET /compute/v2.1/servers/a553b3eb-ae00-4d00-97d1-1d26fe2bdfff" status: 200 len: 1300 microversion: 2.1 time: 0.142967 Jul 16 21:43:31.799724 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2822/5639] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:43:31 2022] GET /compute/v2.1/servers/a553b3eb-ae00-4d00-97d1-1d26fe2bdfff => generated 1300 bytes in 145 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:43:32.813338 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell0:SELECT=8 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:43:32.819992 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-19aba1a7-35bf-4998-8501-3ae862af80fc tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:43:32.820687 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-19aba1a7-35bf-4998-8501-3ae862af80fc tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] [instance: a553b3eb-ae00-4d00-97d1-1d26fe2bdfff] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:43:32.835497 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-19aba1a7-35bf-4998-8501-3ae862af80fc tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:43:32.835497 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-19aba1a7-35bf-4998-8501-3ae862af80fc tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:43:32.835497 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-19aba1a7-35bf-4998-8501-3ae862af80fc tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:43:32.835497 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-19aba1a7-35bf-4998-8501-3ae862af80fc tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:43:32.923031 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-19aba1a7-35bf-4998-8501-3ae862af80fc tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '82819d3efcda4fe78e2b50bb7fab0a96', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '251f01f982aa43ad853d53fa06f99190', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:43:32.930879 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-19aba1a7-35bf-4998-8501-3ae862af80fc tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] 10.222.0.16 "GET /compute/v2.1/servers/a553b3eb-ae00-4d00-97d1-1d26fe2bdfff" status: 200 len: 1300 microversion: 2.1 time: 0.118475 Jul 16 21:43:32.931792 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2818/5640] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:43:32 2022] GET /compute/v2.1/servers/a553b3eb-ae00-4d00-97d1-1d26fe2bdfff => generated 1300 bytes in 120 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:43:33.894613 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-eff320a9-58bd-40d3-a326-bb6bfd3d7ce2 req-b866cc89-112a-4ac1-9c7a-481e03b7298a service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "a553b3eb-ae00-4d00-97d1-1d26fe2bdfff", "name": "network-vif-plugged", "status": "completed", "tag": "db2d4974-12e5-4ee1-82ef-bc072dec3b32"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:43:33.905927 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-eff320a9-58bd-40d3-a326-bb6bfd3d7ce2 req-b866cc89-112a-4ac1-9c7a-481e03b7298a service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:43:33.906579 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-eff320a9-58bd-40d3-a326-bb6bfd3d7ce2 req-b866cc89-112a-4ac1-9c7a-481e03b7298a service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:43:33.922167 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-eff320a9-58bd-40d3-a326-bb6bfd3d7ce2 req-b866cc89-112a-4ac1-9c7a-481e03b7298a service nova] Creating event network-vif-plugged:db2d4974-12e5-4ee1-82ef-bc072dec3b32 for instance a553b3eb-ae00-4d00-97d1-1d26fe2bdfff on nch1-master-1 Jul 16 21:43:33.923999 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-eff320a9-58bd-40d3-a326-bb6bfd3d7ce2 req-b866cc89-112a-4ac1-9c7a-481e03b7298a service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.033155 Jul 16 21:43:33.924801 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2823/5641] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:43:33 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:43:33.948631 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-9de5896d-79cb-4bef-a7c4-cb1b2174cb28 tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:43:33.949908 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-9de5896d-79cb-4bef-a7c4-cb1b2174cb28 tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] [instance: a553b3eb-ae00-4d00-97d1-1d26fe2bdfff] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:43:33.958728 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9de5896d-79cb-4bef-a7c4-cb1b2174cb28 tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:43:33.959390 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9de5896d-79cb-4bef-a7c4-cb1b2174cb28 tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:43:33.960404 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9de5896d-79cb-4bef-a7c4-cb1b2174cb28 tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:43:33.961050 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9de5896d-79cb-4bef-a7c4-cb1b2174cb28 tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:43:34.058155 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-9de5896d-79cb-4bef-a7c4-cb1b2174cb28 tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '82819d3efcda4fe78e2b50bb7fab0a96', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '251f01f982aa43ad853d53fa06f99190', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:43:34.070185 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-9de5896d-79cb-4bef-a7c4-cb1b2174cb28 tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] 10.222.0.16 "GET /compute/v2.1/servers/a553b3eb-ae00-4d00-97d1-1d26fe2bdfff" status: 200 len: 1300 microversion: 2.1 time: 0.125998 Jul 16 21:43:34.080939 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2819/5642] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:43:33 2022] GET /compute/v2.1/servers/a553b3eb-ae00-4d00-97d1-1d26fe2bdfff => generated 1300 bytes in 128 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:43:35.089540 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-f182264f-b26c-4a41-8b5f-727996026432 tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:43:35.090267 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-f182264f-b26c-4a41-8b5f-727996026432 tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] [instance: a553b3eb-ae00-4d00-97d1-1d26fe2bdfff] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:43:35.097387 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f182264f-b26c-4a41-8b5f-727996026432 tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:43:35.097860 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f182264f-b26c-4a41-8b5f-727996026432 tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:43:35.099032 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f182264f-b26c-4a41-8b5f-727996026432 tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:43:35.099588 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f182264f-b26c-4a41-8b5f-727996026432 tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:43:35.229023 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-f182264f-b26c-4a41-8b5f-727996026432 tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '82819d3efcda4fe78e2b50bb7fab0a96', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '251f01f982aa43ad853d53fa06f99190', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:43:35.239433 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-f182264f-b26c-4a41-8b5f-727996026432 tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] 10.222.0.16 "GET /compute/v2.1/servers/a553b3eb-ae00-4d00-97d1-1d26fe2bdfff" status: 200 len: 1300 microversion: 2.1 time: 0.154370 Jul 16 21:43:35.240543 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2824/5643] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:43:35 2022] GET /compute/v2.1/servers/a553b3eb-ae00-4d00-97d1-1d26fe2bdfff => generated 1300 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:43:36.260253 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-cfc3f109-8103-4f48-8f0e-58f47d282f2f tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:43:36.260872 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-cfc3f109-8103-4f48-8f0e-58f47d282f2f tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] [instance: a553b3eb-ae00-4d00-97d1-1d26fe2bdfff] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:43:36.267919 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cfc3f109-8103-4f48-8f0e-58f47d282f2f tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:43:36.271272 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cfc3f109-8103-4f48-8f0e-58f47d282f2f tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:43:36.271272 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cfc3f109-8103-4f48-8f0e-58f47d282f2f tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:43:36.271272 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cfc3f109-8103-4f48-8f0e-58f47d282f2f tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:43:36.354247 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-cfc3f109-8103-4f48-8f0e-58f47d282f2f tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '82819d3efcda4fe78e2b50bb7fab0a96', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '251f01f982aa43ad853d53fa06f99190', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:43:36.362548 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-cfc3f109-8103-4f48-8f0e-58f47d282f2f tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] 10.222.0.16 "GET /compute/v2.1/servers/a553b3eb-ae00-4d00-97d1-1d26fe2bdfff" status: 200 len: 1472 microversion: 2.1 time: 0.106863 Jul 16 21:43:36.363304 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2820/5644] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:43:36 2022] GET /compute/v2.1/servers/a553b3eb-ae00-4d00-97d1-1d26fe2bdfff => generated 1472 bytes in 108 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:43:36.379968 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-13812212-aff1-472a-8049-87d61ba3489a tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:43:36.384395 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-13812212-aff1-472a-8049-87d61ba3489a tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] [instance: a553b3eb-ae00-4d00-97d1-1d26fe2bdfff] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:43:36.388243 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-13812212-aff1-472a-8049-87d61ba3489a tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:43:36.388707 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-13812212-aff1-472a-8049-87d61ba3489a tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:43:36.389473 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-13812212-aff1-472a-8049-87d61ba3489a tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:43:36.389923 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-13812212-aff1-472a-8049-87d61ba3489a tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:43:36.476390 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-13812212-aff1-472a-8049-87d61ba3489a tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '82819d3efcda4fe78e2b50bb7fab0a96', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '251f01f982aa43ad853d53fa06f99190', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:43:36.479649 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=100 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:43:36.485949 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-13812212-aff1-472a-8049-87d61ba3489a tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] 10.222.0.16 "GET /compute/v2.1/servers/a553b3eb-ae00-4d00-97d1-1d26fe2bdfff" status: 200 len: 1472 microversion: 2.1 time: 0.109816 Jul 16 21:43:36.486516 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2825/5645] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:43:36 2022] GET /compute/v2.1/servers/a553b3eb-ae00-4d00-97d1-1d26fe2bdfff => generated 1472 bytes in 111 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:43:36.502015 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-f342abf1-7c74-49bd-86c5-dc964c272839 tempest-TestSecurityGroupsBasicOps-1387468095 tempest-TestSecurityGroupsBasicOps-1387468095-project-admin] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:43:36.504500 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-f342abf1-7c74-49bd-86c5-dc964c272839 tempest-TestSecurityGroupsBasicOps-1387468095 tempest-TestSecurityGroupsBasicOps-1387468095-project-admin] [instance: a553b3eb-ae00-4d00-97d1-1d26fe2bdfff] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:43:36.508996 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f342abf1-7c74-49bd-86c5-dc964c272839 tempest-TestSecurityGroupsBasicOps-1387468095 tempest-TestSecurityGroupsBasicOps-1387468095-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:43:36.509448 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f342abf1-7c74-49bd-86c5-dc964c272839 tempest-TestSecurityGroupsBasicOps-1387468095 tempest-TestSecurityGroupsBasicOps-1387468095-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:43:36.510245 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f342abf1-7c74-49bd-86c5-dc964c272839 tempest-TestSecurityGroupsBasicOps-1387468095 tempest-TestSecurityGroupsBasicOps-1387468095-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:43:36.510681 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f342abf1-7c74-49bd-86c5-dc964c272839 tempest-TestSecurityGroupsBasicOps-1387468095 tempest-TestSecurityGroupsBasicOps-1387468095-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:43:36.592325 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-f342abf1-7c74-49bd-86c5-dc964c272839 tempest-TestSecurityGroupsBasicOps-1387468095 tempest-TestSecurityGroupsBasicOps-1387468095-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/a553b3eb-ae00-4d00-97d1-1d26fe2bdfff" status: 200 len: 1623 microversion: 2.1 time: 0.094077 Jul 16 21:43:36.592896 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2821/5646] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:43:36 2022] GET /compute/v2.1/servers/a553b3eb-ae00-4d00-97d1-1d26fe2bdfff => generated 1623 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:43:36.608121 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-ee224b64-9998-4723-b60d-8c587f29fec5 tempest-TestSecurityGroupsBasicOps-1387468095 tempest-TestSecurityGroupsBasicOps-1387468095-project-admin] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:43:36.608962 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-ee224b64-9998-4723-b60d-8c587f29fec5 tempest-TestSecurityGroupsBasicOps-1387468095 tempest-TestSecurityGroupsBasicOps-1387468095-project-admin] [instance: 0ec2632e-3da8-44ac-b9ba-7300f5bc1f99] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:43:36.615532 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ee224b64-9998-4723-b60d-8c587f29fec5 tempest-TestSecurityGroupsBasicOps-1387468095 tempest-TestSecurityGroupsBasicOps-1387468095-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:43:36.615994 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ee224b64-9998-4723-b60d-8c587f29fec5 tempest-TestSecurityGroupsBasicOps-1387468095 tempest-TestSecurityGroupsBasicOps-1387468095-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:43:36.625038 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ee224b64-9998-4723-b60d-8c587f29fec5 tempest-TestSecurityGroupsBasicOps-1387468095 tempest-TestSecurityGroupsBasicOps-1387468095-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:43:36.625038 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ee224b64-9998-4723-b60d-8c587f29fec5 tempest-TestSecurityGroupsBasicOps-1387468095 tempest-TestSecurityGroupsBasicOps-1387468095-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:43:36.752776 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-ee224b64-9998-4723-b60d-8c587f29fec5 tempest-TestSecurityGroupsBasicOps-1387468095 tempest-TestSecurityGroupsBasicOps-1387468095-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/0ec2632e-3da8-44ac-b9ba-7300f5bc1f99" status: 200 len: 1911 microversion: 2.1 time: 0.148538 Jul 16 21:43:36.753354 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2826/5647] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:43:36 2022] GET /compute/v2.1/servers/0ec2632e-3da8-44ac-b9ba-7300f5bc1f99 => generated 1911 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:43:36.816045 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-586f7548-5f3a-4d4a-9e16-3d0833d15fb3 tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:43:36.816546 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-586f7548-5f3a-4d4a-9e16-3d0833d15fb3 tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] [instance: a553b3eb-ae00-4d00-97d1-1d26fe2bdfff] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:43:36.823268 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-586f7548-5f3a-4d4a-9e16-3d0833d15fb3 tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:43:36.823724 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-586f7548-5f3a-4d4a-9e16-3d0833d15fb3 tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:43:36.824488 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-586f7548-5f3a-4d4a-9e16-3d0833d15fb3 tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:43:36.824926 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-586f7548-5f3a-4d4a-9e16-3d0833d15fb3 tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:43:36.850765 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-586f7548-5f3a-4d4a-9e16-3d0833d15fb3 tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] [instance: a553b3eb-ae00-4d00-97d1-1d26fe2bdfff] Going to try to terminate instance {{(pid=138680) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 16 21:43:37.113210 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-586f7548-5f3a-4d4a-9e16-3d0833d15fb3 tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] 10.222.0.16 "DELETE /compute/v2.1/servers/a553b3eb-ae00-4d00-97d1-1d26fe2bdfff" status: 204 len: 0 microversion: 2.1 time: 0.300711 Jul 16 21:43:37.113815 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2822/5648] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:43:36 2022] DELETE /compute/v2.1/servers/a553b3eb-ae00-4d00-97d1-1d26fe2bdfff => generated 0 bytes in 302 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 16 21:43:37.122378 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-09116637-4830-458c-8544-7369c00b1f6a tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:43:37.122934 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-09116637-4830-458c-8544-7369c00b1f6a tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] [instance: a553b3eb-ae00-4d00-97d1-1d26fe2bdfff] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:43:37.129244 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-09116637-4830-458c-8544-7369c00b1f6a tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:43:37.129699 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-09116637-4830-458c-8544-7369c00b1f6a tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:43:37.130464 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-09116637-4830-458c-8544-7369c00b1f6a tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:43:37.130911 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-09116637-4830-458c-8544-7369c00b1f6a tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:43:37.217850 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-09116637-4830-458c-8544-7369c00b1f6a tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '82819d3efcda4fe78e2b50bb7fab0a96', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '251f01f982aa43ad853d53fa06f99190', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:43:37.225421 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-09116637-4830-458c-8544-7369c00b1f6a tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] 10.222.0.16 "GET /compute/v2.1/servers/a553b3eb-ae00-4d00-97d1-1d26fe2bdfff" status: 200 len: 1478 microversion: 2.1 time: 0.107379 Jul 16 21:43:37.225988 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2827/5649] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:43:37 2022] GET /compute/v2.1/servers/a553b3eb-ae00-4d00-97d1-1d26fe2bdfff => generated 1478 bytes in 108 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:43:38.245287 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-e9f4270d-5d5d-40e1-9be8-7377d87d4c94 tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:43:38.246088 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-e9f4270d-5d5d-40e1-9be8-7377d87d4c94 tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] [instance: a553b3eb-ae00-4d00-97d1-1d26fe2bdfff] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:43:38.253116 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e9f4270d-5d5d-40e1-9be8-7377d87d4c94 tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:43:38.254135 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e9f4270d-5d5d-40e1-9be8-7377d87d4c94 tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:43:38.254935 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e9f4270d-5d5d-40e1-9be8-7377d87d4c94 tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:43:38.255410 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e9f4270d-5d5d-40e1-9be8-7377d87d4c94 tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:43:38.339755 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-e9f4270d-5d5d-40e1-9be8-7377d87d4c94 tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '82819d3efcda4fe78e2b50bb7fab0a96', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '251f01f982aa43ad853d53fa06f99190', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:43:38.347734 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-e9f4270d-5d5d-40e1-9be8-7377d87d4c94 tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] 10.222.0.16 "GET /compute/v2.1/servers/a553b3eb-ae00-4d00-97d1-1d26fe2bdfff" status: 200 len: 1478 microversion: 2.1 time: 0.107444 Jul 16 21:43:38.348320 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2823/5650] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:43:38 2022] GET /compute/v2.1/servers/a553b3eb-ae00-4d00-97d1-1d26fe2bdfff => generated 1478 bytes in 109 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:43:38.990731 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-79871ae5-99e6-49eb-a144-c44c18eb4e8c req-31bfa755-480a-44c3-96c4-dd0e5618e80c service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "a553b3eb-ae00-4d00-97d1-1d26fe2bdfff", "name": "network-vif-deleted", "tag": "db2d4974-12e5-4ee1-82ef-bc072dec3b32"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:43:38.997765 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-79871ae5-99e6-49eb-a144-c44c18eb4e8c req-31bfa755-480a-44c3-96c4-dd0e5618e80c service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:43:38.998224 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-79871ae5-99e6-49eb-a144-c44c18eb4e8c req-31bfa755-480a-44c3-96c4-dd0e5618e80c service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:43:39.010889 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-79871ae5-99e6-49eb-a144-c44c18eb4e8c req-31bfa755-480a-44c3-96c4-dd0e5618e80c service nova] Creating event network-vif-deleted:db2d4974-12e5-4ee1-82ef-bc072dec3b32 for instance a553b3eb-ae00-4d00-97d1-1d26fe2bdfff on nch1-master-1 Jul 16 21:43:39.016035 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-79871ae5-99e6-49eb-a144-c44c18eb4e8c req-31bfa755-480a-44c3-96c4-dd0e5618e80c service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.032084 Jul 16 21:43:39.016638 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2828/5651] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:43:38 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:43:39.365440 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-973a9424-8fa6-433f-9304-5705da59f33c tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:43:39.366054 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-973a9424-8fa6-433f-9304-5705da59f33c tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] [instance: a553b3eb-ae00-4d00-97d1-1d26fe2bdfff] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:43:39.373823 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-973a9424-8fa6-433f-9304-5705da59f33c tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:43:39.374262 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-973a9424-8fa6-433f-9304-5705da59f33c tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:43:39.375056 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-973a9424-8fa6-433f-9304-5705da59f33c tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:43:39.375787 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-973a9424-8fa6-433f-9304-5705da59f33c tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:43:39.410601 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-973a9424-8fa6-433f-9304-5705da59f33c tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:43:39.411079 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-973a9424-8fa6-433f-9304-5705da59f33c tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:43:39.411533 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.objects.instance [None req-973a9424-8fa6-433f-9304-5705da59f33c tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Lazy-loading 'fault' on Instance uuid a553b3eb-ae00-4d00-97d1-1d26fe2bdfff {{(pid=138680) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:43:39.454963 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-973a9424-8fa6-433f-9304-5705da59f33c tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '82819d3efcda4fe78e2b50bb7fab0a96', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '251f01f982aa43ad853d53fa06f99190', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:43:39.462771 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-973a9424-8fa6-433f-9304-5705da59f33c tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] 10.222.0.16 "GET /compute/v2.1/servers/a553b3eb-ae00-4d00-97d1-1d26fe2bdfff" status: 200 len: 1332 microversion: 2.1 time: 0.101378 Jul 16 21:43:39.463775 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2824/5652] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:43:39 2022] GET /compute/v2.1/servers/a553b3eb-ae00-4d00-97d1-1d26fe2bdfff => generated 1332 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:43:40.483618 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-d6cc8a72-3a05-4f12-9a31-d0e1d821bebb tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:43:40.484348 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-d6cc8a72-3a05-4f12-9a31-d0e1d821bebb tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] [instance: a553b3eb-ae00-4d00-97d1-1d26fe2bdfff] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:43:40.490986 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d6cc8a72-3a05-4f12-9a31-d0e1d821bebb tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:43:40.491521 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d6cc8a72-3a05-4f12-9a31-d0e1d821bebb tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:43:40.492305 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d6cc8a72-3a05-4f12-9a31-d0e1d821bebb tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:43:40.492760 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d6cc8a72-3a05-4f12-9a31-d0e1d821bebb tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:43:40.508045 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-d6cc8a72-3a05-4f12-9a31-d0e1d821bebb tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] HTTP exception thrown: Instance a553b3eb-ae00-4d00-97d1-1d26fe2bdfff could not be found. Jul 16 21:43:40.509706 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-d6cc8a72-3a05-4f12-9a31-d0e1d821bebb tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Request method failure captured: Jul 16 21:43:40.509706 ncdv-master-1 devstack@n-api.service[138681]: request: GET /compute/v2.1/servers/a553b3eb-ae00-4d00-97d1-1d26fe2bdfff HTTP/1.1 Jul 16 21:43:40.509706 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:43:40.509706 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:43:40.509706 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:43:40.509706 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 21:43:40.509706 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:43:40.509706 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:43:40.509706 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:43:40.509706 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:43:40.509706 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0zCwoAonnoPAKEYFndPTwqqGm7gupBmiT-O-bERG5P5ex5l8lH8H2g296Ok4Ss9vGPzED_U1aWKRyUZGifNPeN-JvwsB7_KIc7ui4iH6T0BC1z9jWAcIwUABt3dFx6Vq8JXnDZudv0-sz8BbYOIleGoIYzWFYj7dhb2xhyjDXIUgzbM Jul 16 21:43:40.509706 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:43:40.509706 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:43:40.509706 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:43:40.509706 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:43:40.509706 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:43:40.509706 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:43:40.509706 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: 251f01f982aa43ad853d53fa06f99190 Jul 16 21:43:40.509706 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-TestSecurityGroupsBasicOps-1431728571 Jul 16 21:43:40.509706 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:43:40.509706 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:43:40.509706 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/251f01f982aa43ad853d53fa06f99190"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/251f01f982aa43ad853d53fa06f99190"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_251f01f982aa43ad853d53fa06f99190"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/251f01f982aa43ad853d53fa06f99190"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:43:40.509706 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-TestSecurityGroupsBasicOps-1431728571 Jul 16 21:43:40.509706 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: 251f01f982aa43ad853d53fa06f99190 Jul 16 21:43:40.509706 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-TestSecurityGroupsBasicOps-1431728571 Jul 16 21:43:40.509706 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-TestSecurityGroupsBasicOps-1431728571-project Jul 16 21:43:40.509706 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:43:40.509706 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:43:40.509706 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: 82819d3efcda4fe78e2b50bb7fab0a96 Jul 16 21:43:40.509706 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-TestSecurityGroupsBasicOps-1431728571-project Jul 16 21:43:40.509706 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:43:40.509706 ncdv-master-1 devstack@n-api.service[138681]: exception: Instance a553b3eb-ae00-4d00-97d1-1d26fe2bdfff could not be found. Jul 16 21:43:40.509706 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': 'a553b3eb-ae00-4d00-97d1-1d26fe2bdfff'} Jul 16 21:43:40.509706 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:43:40.509706 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:43:40.509706 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:43:40.509706 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:43:40.509706 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:43:40.509706 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:43:40.509706 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:43:40.509706 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:43:40.509706 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:43:40.509706 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:43:40.509706 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance a553b3eb-ae00-4d00-97d1-1d26fe2bdfff could not be found. Jul 16 21:43:40.509706 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:43:40.509706 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:43:40.509706 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:43:40.509706 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:43:40.509706 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:43:40.509706 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:43:40.509706 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:43:40.509706 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:43:40.509706 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:43:40.509706 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:43:40.509706 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:43:40.509706 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:43:40.509706 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:43:40.509706 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:43:40.509706 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:43:40.509706 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:43:40.509706 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance a553b3eb-ae00-4d00-97d1-1d26fe2bdfff could not be found. Jul 16 21:43:40.509706 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:43:40.509706 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:43:40.509706 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:43:40.515198 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:43:40.515198 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:43:40.515198 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:43:40.515198 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:43:40.515198 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:43:40.515198 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance a553b3eb-ae00-4d00-97d1-1d26fe2bdfff could not be found. Jul 16 21:43:40.515198 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:43:40.515198 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-d6cc8a72-3a05-4f12-9a31-d0e1d821bebb tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Returning 404 to user: Instance a553b3eb-ae00-4d00-97d1-1d26fe2bdfff could not be found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:43:40.515198 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-d6cc8a72-3a05-4f12-9a31-d0e1d821bebb tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] 10.222.0.16 "GET /compute/v2.1/servers/a553b3eb-ae00-4d00-97d1-1d26fe2bdfff" status: 404 len: 111 microversion: 2.1 time: 0.033878 Jul 16 21:43:40.515198 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2829/5653] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:43:40 2022] GET /compute/v2.1/servers/a553b3eb-ae00-4d00-97d1-1d26fe2bdfff => generated 111 bytes in 35 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:43:43.097007 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-4235279a-3148-42e3-9e95-f19c72268e95 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Action: 'action', calling method: >, body: {"os-getConsoleOutput": {}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:43:43.098121 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-4235279a-3148-42e3-9e95-f19c72268e95 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] [instance: 64b5a7a2-7d82-4c60-802e-8e92c14e0790] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:43:43.105959 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4235279a-3148-42e3-9e95-f19c72268e95 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:43:43.106425 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4235279a-3148-42e3-9e95-f19c72268e95 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:43:43.107252 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4235279a-3148-42e3-9e95-f19c72268e95 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:43:43.107736 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4235279a-3148-42e3-9e95-f19c72268e95 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:43:43.177305 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-4235279a-3148-42e3-9e95-f19c72268e95 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] 10.222.0.16 "POST /compute/v2.1/servers/64b5a7a2-7d82-4c60-802e-8e92c14e0790/action" status: 200 len: 24569 microversion: 2.1 time: 0.083089 Jul 16 21:43:43.179932 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2825/5654] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 21:43:43 2022] POST /compute/v2.1/servers/64b5a7a2-7d82-4c60-802e-8e92c14e0790/action => generated 24569 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 16 21:43:45.173721 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-a1976938-04c1-4db5-870a-be26b937e986 tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:43:45.174794 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-a1976938-04c1-4db5-870a-be26b937e986 tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] [instance: 0ec2632e-3da8-44ac-b9ba-7300f5bc1f99] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:43:45.176668 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-917ceb28-6a55-4415-863d-f59d71b090f4 req-44cb06c1-1e45-4d55-97d8-82679f073ef6 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "0ec2632e-3da8-44ac-b9ba-7300f5bc1f99", "tag": "96d14176-a9cf-4e6c-a04e-16fc22908cf3"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:43:45.186134 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a1976938-04c1-4db5-870a-be26b937e986 tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:43:45.193014 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a1976938-04c1-4db5-870a-be26b937e986 tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:43:45.193607 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-917ceb28-6a55-4415-863d-f59d71b090f4 req-44cb06c1-1e45-4d55-97d8-82679f073ef6 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:43:45.193731 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a1976938-04c1-4db5-870a-be26b937e986 tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:43:45.193826 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-917ceb28-6a55-4415-863d-f59d71b090f4 req-44cb06c1-1e45-4d55-97d8-82679f073ef6 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:43:45.193916 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a1976938-04c1-4db5-870a-be26b937e986 tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:43:45.203182 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-917ceb28-6a55-4415-863d-f59d71b090f4 req-44cb06c1-1e45-4d55-97d8-82679f073ef6 service nova] Creating event network-changed:96d14176-a9cf-4e6c-a04e-16fc22908cf3 for instance 0ec2632e-3da8-44ac-b9ba-7300f5bc1f99 on nch2-master-1 Jul 16 21:43:45.208525 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-917ceb28-6a55-4415-863d-f59d71b090f4 req-44cb06c1-1e45-4d55-97d8-82679f073ef6 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.036183 Jul 16 21:43:45.209826 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2826/5655] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:43:45 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:43:45.240618 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-a1976938-04c1-4db5-870a-be26b937e986 tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] [instance: 0ec2632e-3da8-44ac-b9ba-7300f5bc1f99] Going to try to terminate instance {{(pid=138681) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 16 21:43:45.577477 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-a1976938-04c1-4db5-870a-be26b937e986 tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] 10.222.0.16 "DELETE /compute/v2.1/servers/0ec2632e-3da8-44ac-b9ba-7300f5bc1f99" status: 204 len: 0 microversion: 2.1 time: 0.407472 Jul 16 21:43:45.578293 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2830/5656] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:43:45 2022] DELETE /compute/v2.1/servers/0ec2632e-3da8-44ac-b9ba-7300f5bc1f99 => generated 0 bytes in 411 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 16 21:43:45.590733 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-58136402-9a9c-42ff-86d3-2b1bf8d9969c tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:43:45.591734 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-58136402-9a9c-42ff-86d3-2b1bf8d9969c tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] [instance: 0ec2632e-3da8-44ac-b9ba-7300f5bc1f99] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:43:45.602640 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-58136402-9a9c-42ff-86d3-2b1bf8d9969c tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:43:45.603371 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-58136402-9a9c-42ff-86d3-2b1bf8d9969c tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:43:45.605906 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-58136402-9a9c-42ff-86d3-2b1bf8d9969c tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:43:45.606564 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-58136402-9a9c-42ff-86d3-2b1bf8d9969c tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:43:45.764743 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-58136402-9a9c-42ff-86d3-2b1bf8d9969c tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '82819d3efcda4fe78e2b50bb7fab0a96', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '251f01f982aa43ad853d53fa06f99190', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:43:45.773408 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-58136402-9a9c-42ff-86d3-2b1bf8d9969c tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] 10.222.0.16 "GET /compute/v2.1/servers/0ec2632e-3da8-44ac-b9ba-7300f5bc1f99" status: 200 len: 1766 microversion: 2.1 time: 0.189280 Jul 16 21:43:45.774055 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2827/5657] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:43:45 2022] GET /compute/v2.1/servers/0ec2632e-3da8-44ac-b9ba-7300f5bc1f99 => generated 1766 bytes in 191 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:43:46.793471 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-28002658-bdf8-4a5f-a137-d707fc9f5fe6 tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:43:46.794290 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-28002658-bdf8-4a5f-a137-d707fc9f5fe6 tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] [instance: 0ec2632e-3da8-44ac-b9ba-7300f5bc1f99] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:43:46.801300 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-28002658-bdf8-4a5f-a137-d707fc9f5fe6 tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:43:46.801881 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-28002658-bdf8-4a5f-a137-d707fc9f5fe6 tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:43:46.802588 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-28002658-bdf8-4a5f-a137-d707fc9f5fe6 tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:43:46.803059 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-28002658-bdf8-4a5f-a137-d707fc9f5fe6 tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:43:47.026473 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-28002658-bdf8-4a5f-a137-d707fc9f5fe6 tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '82819d3efcda4fe78e2b50bb7fab0a96', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '251f01f982aa43ad853d53fa06f99190', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:43:47.034801 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-28002658-bdf8-4a5f-a137-d707fc9f5fe6 tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] 10.222.0.16 "GET /compute/v2.1/servers/0ec2632e-3da8-44ac-b9ba-7300f5bc1f99" status: 200 len: 1766 microversion: 2.1 time: 0.246407 Jul 16 21:43:47.035665 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2831/5658] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:43:46 2022] GET /compute/v2.1/servers/0ec2632e-3da8-44ac-b9ba-7300f5bc1f99 => generated 1766 bytes in 248 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:43:47.639409 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-ce34d20d-aead-4b17-bdb4-8e772c7d7a71 req-08ca4731-cbd9-43fb-8616-1d981676764e service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "0ec2632e-3da8-44ac-b9ba-7300f5bc1f99", "name": "network-vif-deleted", "tag": "96d14176-a9cf-4e6c-a04e-16fc22908cf3"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:43:47.648882 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-ce34d20d-aead-4b17-bdb4-8e772c7d7a71 req-08ca4731-cbd9-43fb-8616-1d981676764e service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:43:47.649340 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-ce34d20d-aead-4b17-bdb4-8e772c7d7a71 req-08ca4731-cbd9-43fb-8616-1d981676764e service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:43:47.660991 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-ce34d20d-aead-4b17-bdb4-8e772c7d7a71 req-08ca4731-cbd9-43fb-8616-1d981676764e service nova] Creating event network-vif-deleted:96d14176-a9cf-4e6c-a04e-16fc22908cf3 for instance 0ec2632e-3da8-44ac-b9ba-7300f5bc1f99 on nch2-master-1 Jul 16 21:43:47.670415 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-ce34d20d-aead-4b17-bdb4-8e772c7d7a71 req-08ca4731-cbd9-43fb-8616-1d981676764e service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.031387 Jul 16 21:43:47.670415 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2828/5659] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:43:47 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:43:48.054094 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-d69a5eec-0899-4d2a-8554-decbb346104f tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:43:48.055723 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-d69a5eec-0899-4d2a-8554-decbb346104f tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] [instance: 0ec2632e-3da8-44ac-b9ba-7300f5bc1f99] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:43:48.064587 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d69a5eec-0899-4d2a-8554-decbb346104f tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:43:48.065184 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d69a5eec-0899-4d2a-8554-decbb346104f tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:43:48.065968 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d69a5eec-0899-4d2a-8554-decbb346104f tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:43:48.066416 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d69a5eec-0899-4d2a-8554-decbb346104f tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:43:48.102517 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d69a5eec-0899-4d2a-8554-decbb346104f tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:43:48.102992 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d69a5eec-0899-4d2a-8554-decbb346104f tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:43:48.103486 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.objects.instance [None req-d69a5eec-0899-4d2a-8554-decbb346104f tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Lazy-loading 'fault' on Instance uuid 0ec2632e-3da8-44ac-b9ba-7300f5bc1f99 {{(pid=138681) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:43:48.147053 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-d69a5eec-0899-4d2a-8554-decbb346104f tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '82819d3efcda4fe78e2b50bb7fab0a96', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '251f01f982aa43ad853d53fa06f99190', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:43:48.158191 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-d69a5eec-0899-4d2a-8554-decbb346104f tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] 10.222.0.16 "GET /compute/v2.1/servers/0ec2632e-3da8-44ac-b9ba-7300f5bc1f99" status: 200 len: 1385 microversion: 2.1 time: 0.107486 Jul 16 21:43:48.158191 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2832/5660] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:43:48 2022] GET /compute/v2.1/servers/0ec2632e-3da8-44ac-b9ba-7300f5bc1f99 => generated 1385 bytes in 109 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:43:49.174657 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-b9efa0c9-67c1-4aee-9df9-b257cb9e958f tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:43:49.175406 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-b9efa0c9-67c1-4aee-9df9-b257cb9e958f tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] [instance: 0ec2632e-3da8-44ac-b9ba-7300f5bc1f99] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:43:49.182566 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b9efa0c9-67c1-4aee-9df9-b257cb9e958f tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:43:49.183696 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b9efa0c9-67c1-4aee-9df9-b257cb9e958f tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:43:49.184476 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b9efa0c9-67c1-4aee-9df9-b257cb9e958f tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:43:49.184919 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b9efa0c9-67c1-4aee-9df9-b257cb9e958f tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:43:49.198151 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-b9efa0c9-67c1-4aee-9df9-b257cb9e958f tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] HTTP exception thrown: Instance 0ec2632e-3da8-44ac-b9ba-7300f5bc1f99 could not be found. Jul 16 21:43:49.200339 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-b9efa0c9-67c1-4aee-9df9-b257cb9e958f tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Request method failure captured: Jul 16 21:43:49.200339 ncdv-master-1 devstack@n-api.service[138680]: request: GET /compute/v2.1/servers/0ec2632e-3da8-44ac-b9ba-7300f5bc1f99 HTTP/1.1 Jul 16 21:43:49.200339 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:43:49.200339 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:43:49.200339 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:43:49.200339 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 0 Jul 16 21:43:49.200339 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:43:49.200339 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:43:49.200339 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:43:49.200339 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:43:49.200339 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0zCwoAonnoPAKEYFndPTwqqGm7gupBmiT-O-bERG5P5ex5l8lH8H2g296Ok4Ss9vGPzED_U1aWKRyUZGifNPeN-JvwsB7_KIc7ui4iH6T0BC1z9jWAcIwUABt3dFx6Vq8JXnDZudv0-sz8BbYOIleGoIYzWFYj7dhb2xhyjDXIUgzbM Jul 16 21:43:49.200339 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:43:49.200339 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:43:49.200339 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:43:49.200339 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:43:49.200339 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:43:49.200339 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:43:49.200339 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: 251f01f982aa43ad853d53fa06f99190 Jul 16 21:43:49.200339 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-TestSecurityGroupsBasicOps-1431728571 Jul 16 21:43:49.200339 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:43:49.200339 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:43:49.200339 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/251f01f982aa43ad853d53fa06f99190"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/251f01f982aa43ad853d53fa06f99190"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_251f01f982aa43ad853d53fa06f99190"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/251f01f982aa43ad853d53fa06f99190"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:43:49.200339 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-TestSecurityGroupsBasicOps-1431728571 Jul 16 21:43:49.200339 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: 251f01f982aa43ad853d53fa06f99190 Jul 16 21:43:49.200339 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-TestSecurityGroupsBasicOps-1431728571 Jul 16 21:43:49.200339 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-TestSecurityGroupsBasicOps-1431728571-project Jul 16 21:43:49.200339 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:43:49.200339 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:43:49.200339 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: 82819d3efcda4fe78e2b50bb7fab0a96 Jul 16 21:43:49.200339 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-TestSecurityGroupsBasicOps-1431728571-project Jul 16 21:43:49.200339 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 21:43:49.200339 ncdv-master-1 devstack@n-api.service[138680]: exception: Instance 0ec2632e-3da8-44ac-b9ba-7300f5bc1f99 could not be found. Jul 16 21:43:49.200339 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': '0ec2632e-3da8-44ac-b9ba-7300f5bc1f99'} Jul 16 21:43:49.200339 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:43:49.200339 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:43:49.200339 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:43:49.200339 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:43:49.200339 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:43:49.200339 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:43:49.200339 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:43:49.200339 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:43:49.200339 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:43:49.200339 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:43:49.200339 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 0ec2632e-3da8-44ac-b9ba-7300f5bc1f99 could not be found. Jul 16 21:43:49.200339 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:43:49.200339 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:43:49.200339 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:43:49.200339 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:43:49.200339 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:43:49.200339 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:43:49.200339 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:43:49.200339 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:43:49.200339 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:43:49.200339 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:43:49.200339 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:43:49.200339 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:43:49.200339 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:43:49.200339 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:43:49.200339 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:43:49.200339 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:43:49.200339 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 0ec2632e-3da8-44ac-b9ba-7300f5bc1f99 could not be found. Jul 16 21:43:49.200339 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:43:49.200339 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:43:49.200339 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:43:49.207592 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:43:49.207592 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:43:49.207592 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:43:49.207592 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:43:49.207592 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:43:49.207592 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 0ec2632e-3da8-44ac-b9ba-7300f5bc1f99 could not be found. Jul 16 21:43:49.207592 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:43:49.207592 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-b9efa0c9-67c1-4aee-9df9-b257cb9e958f tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Returning 404 to user: Instance 0ec2632e-3da8-44ac-b9ba-7300f5bc1f99 could not be found. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:43:49.207592 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-b9efa0c9-67c1-4aee-9df9-b257cb9e958f tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] 10.222.0.16 "GET /compute/v2.1/servers/0ec2632e-3da8-44ac-b9ba-7300f5bc1f99" status: 404 len: 111 microversion: 2.1 time: 0.031585 Jul 16 21:43:49.207592 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2829/5661] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:43:49 2022] GET /compute/v2.1/servers/0ec2632e-3da8-44ac-b9ba-7300f5bc1f99 => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:43:53.519153 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-69c9cdef-b42d-4978-af87-5ebdc1000677 tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] Calling method '.version_select at 0x7fdd8c5ec790>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:43:53.534695 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_api:SELECT=63,nova_api:UPDATE=1 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:43:53.651626 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-69c9cdef-b42d-4978-af87-5ebdc1000677 tempest-TestSecurityGroupsBasicOps-1431728571 tempest-TestSecurityGroupsBasicOps-1431728571-project] 10.222.0.16 "DELETE /compute/v2.1/os-keypairs/tempest-TestSecurityGroupsBasicOps-1925752067" status: 202 len: 0 microversion: 2.1 time: 0.136717 Jul 16 21:43:53.652161 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2833/5662] 10.222.0.16 () {58 vars in 1277 bytes} [Sat Jul 16 21:43:53 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestSecurityGroupsBasicOps-1925752067 => generated 0 bytes in 138 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 21:43:54.200614 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-25468142-0e8d-4b38-9f71-d47f67e7ed80 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:43:54.201270 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-25468142-0e8d-4b38-9f71-d47f67e7ed80 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] [instance: 64b5a7a2-7d82-4c60-802e-8e92c14e0790] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:43:54.204237 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_api:SELECT=71,nova_api:INSERT=3,nova_api:DELETE=1,nova_api:UPDATE=1 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:43:54.209908 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-25468142-0e8d-4b38-9f71-d47f67e7ed80 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:43:54.210369 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-25468142-0e8d-4b38-9f71-d47f67e7ed80 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:43:54.211203 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-25468142-0e8d-4b38-9f71-d47f67e7ed80 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:43:54.211680 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-25468142-0e8d-4b38-9f71-d47f67e7ed80 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:43:54.394376 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-25468142-0e8d-4b38-9f71-d47f67e7ed80 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '833136cbdeab4b9e8b6dd2d24c7c8753', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '77c9b7e351e24dc88207e9498f75f578', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:43:54.402264 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-25468142-0e8d-4b38-9f71-d47f67e7ed80 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] 10.222.0.16 "GET /compute/v2.1/servers/64b5a7a2-7d82-4c60-802e-8e92c14e0790" status: 200 len: 1682 microversion: 2.1 time: 0.205542 Jul 16 21:43:54.402854 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2830/5663] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:43:54 2022] GET /compute/v2.1/servers/64b5a7a2-7d82-4c60-802e-8e92c14e0790 => generated 1682 bytes in 207 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:43:54.514219 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-8a27d080-2876-4c08-999b-5e2534b2c07f tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Action: 'action', calling method: >, body: {"os-getConsoleOutput": {}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:43:54.515829 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-8a27d080-2876-4c08-999b-5e2534b2c07f tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] [instance: 64b5a7a2-7d82-4c60-802e-8e92c14e0790] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:43:54.523131 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8a27d080-2876-4c08-999b-5e2534b2c07f tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:43:54.523753 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8a27d080-2876-4c08-999b-5e2534b2c07f tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:43:54.524377 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8a27d080-2876-4c08-999b-5e2534b2c07f tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:43:54.524845 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8a27d080-2876-4c08-999b-5e2534b2c07f tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:43:54.586341 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-8a27d080-2876-4c08-999b-5e2534b2c07f tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] 10.222.0.16 "POST /compute/v2.1/servers/64b5a7a2-7d82-4c60-802e-8e92c14e0790/action" status: 200 len: 32641 microversion: 2.1 time: 0.076094 Jul 16 21:43:54.587153 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2834/5664] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 21:43:54 2022] POST /compute/v2.1/servers/64b5a7a2-7d82-4c60-802e-8e92c14e0790/action => generated 32641 bytes in 78 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 16 21:43:54.977909 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-8bf72f53-4a75-4063-95d0-a5a3b196430f tempest-TestNetworkAdvancedServerOps-756251536 tempest-TestNetworkAdvancedServerOps-756251536-project-admin] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:43:54.978523 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-8bf72f53-4a75-4063-95d0-a5a3b196430f tempest-TestNetworkAdvancedServerOps-756251536 tempest-TestNetworkAdvancedServerOps-756251536-project-admin] [instance: 64b5a7a2-7d82-4c60-802e-8e92c14e0790] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:43:54.985158 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8bf72f53-4a75-4063-95d0-a5a3b196430f tempest-TestNetworkAdvancedServerOps-756251536 tempest-TestNetworkAdvancedServerOps-756251536-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:43:54.985609 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8bf72f53-4a75-4063-95d0-a5a3b196430f tempest-TestNetworkAdvancedServerOps-756251536 tempest-TestNetworkAdvancedServerOps-756251536-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:43:54.990968 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8bf72f53-4a75-4063-95d0-a5a3b196430f tempest-TestNetworkAdvancedServerOps-756251536 tempest-TestNetworkAdvancedServerOps-756251536-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:43:54.990968 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8bf72f53-4a75-4063-95d0-a5a3b196430f tempest-TestNetworkAdvancedServerOps-756251536 tempest-TestNetworkAdvancedServerOps-756251536-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:43:55.117840 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:43:55.125127 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-8bf72f53-4a75-4063-95d0-a5a3b196430f tempest-TestNetworkAdvancedServerOps-756251536 tempest-TestNetworkAdvancedServerOps-756251536-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/64b5a7a2-7d82-4c60-802e-8e92c14e0790" status: 200 len: 1833 microversion: 2.1 time: 0.388417 Jul 16 21:43:55.125726 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2831/5665] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:43:54 2022] GET /compute/v2.1/servers/64b5a7a2-7d82-4c60-802e-8e92c14e0790 => generated 1833 bytes in 391 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:43:55.159193 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-75c961ef-cf67-41af-8bc6-b4814806aefa tempest-TestNetworkAdvancedServerOps-756251536 tempest-TestNetworkAdvancedServerOps-756251536-project-admin] Action: 'action', calling method: >, body: {"os-migrateLive": {"host": null, "block_migration": true, "disk_over_commit": false}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:43:55.159193 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-75c961ef-cf67-41af-8bc6-b4814806aefa tempest-TestNetworkAdvancedServerOps-756251536 tempest-TestNetworkAdvancedServerOps-756251536-project-admin] [instance: 64b5a7a2-7d82-4c60-802e-8e92c14e0790] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:43:55.159942 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-75c961ef-cf67-41af-8bc6-b4814806aefa tempest-TestNetworkAdvancedServerOps-756251536 tempest-TestNetworkAdvancedServerOps-756251536-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:43:55.160645 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-75c961ef-cf67-41af-8bc6-b4814806aefa tempest-TestNetworkAdvancedServerOps-756251536 tempest-TestNetworkAdvancedServerOps-756251536-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:43:55.161515 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-75c961ef-cf67-41af-8bc6-b4814806aefa tempest-TestNetworkAdvancedServerOps-756251536 tempest-TestNetworkAdvancedServerOps-756251536-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:43:55.161894 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-75c961ef-cf67-41af-8bc6-b4814806aefa tempest-TestNetworkAdvancedServerOps-756251536 tempest-TestNetworkAdvancedServerOps-756251536-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:43:55.206580 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.objects.instance [None req-75c961ef-cf67-41af-8bc6-b4814806aefa tempest-TestNetworkAdvancedServerOps-756251536 tempest-TestNetworkAdvancedServerOps-756251536-project-admin] Lazy-loading 'flavor' on Instance uuid 64b5a7a2-7d82-4c60-802e-8e92c14e0790 {{(pid=138681) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:43:55.221407 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-75c961ef-cf67-41af-8bc6-b4814806aefa tempest-TestNetworkAdvancedServerOps-756251536 tempest-TestNetworkAdvancedServerOps-756251536-project-admin] [instance: 64b5a7a2-7d82-4c60-802e-8e92c14e0790] Going to try to live migrate instance to another host {{(pid=138681) live_migrate /opt/stack/nova/nova/compute/api.py:5349}} Jul 16 21:43:58.505001 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-75c961ef-cf67-41af-8bc6-b4814806aefa tempest-TestNetworkAdvancedServerOps-756251536 tempest-TestNetworkAdvancedServerOps-756251536-project-admin] 10.222.0.16 "POST /compute/v2.1/servers/64b5a7a2-7d82-4c60-802e-8e92c14e0790/action" status: 202 len: 0 microversion: 2.1 time: 3.358876 Jul 16 21:43:58.505836 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2835/5666] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 21:43:55 2022] POST /compute/v2.1/servers/64b5a7a2-7d82-4c60-802e-8e92c14e0790/action => generated 0 bytes in 3361 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 21:43:58.514398 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-d5f2905b-6cd9-4fb8-b2c3-604ed27cd20e tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:43:58.515001 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-d5f2905b-6cd9-4fb8-b2c3-604ed27cd20e tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] [instance: 64b5a7a2-7d82-4c60-802e-8e92c14e0790] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:43:58.523496 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d5f2905b-6cd9-4fb8-b2c3-604ed27cd20e tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:43:58.523954 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d5f2905b-6cd9-4fb8-b2c3-604ed27cd20e tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:43:58.524731 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d5f2905b-6cd9-4fb8-b2c3-604ed27cd20e tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:43:58.525177 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d5f2905b-6cd9-4fb8-b2c3-604ed27cd20e tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:43:58.668144 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-d5f2905b-6cd9-4fb8-b2c3-604ed27cd20e tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '833136cbdeab4b9e8b6dd2d24c7c8753', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '77c9b7e351e24dc88207e9498f75f578', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:43:58.676570 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-d5f2905b-6cd9-4fb8-b2c3-604ed27cd20e tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] 10.222.0.16 "GET /compute/v2.1/servers/64b5a7a2-7d82-4c60-802e-8e92c14e0790" status: 200 len: 1692 microversion: 2.1 time: 0.165769 Jul 16 21:43:58.676854 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2832/5667] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:43:58 2022] GET /compute/v2.1/servers/64b5a7a2-7d82-4c60-802e-8e92c14e0790 => generated 1692 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:43:59.695527 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-061d862f-1a2a-4af7-bbc9-3bac49f35382 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:43:59.696335 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-061d862f-1a2a-4af7-bbc9-3bac49f35382 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] [instance: 64b5a7a2-7d82-4c60-802e-8e92c14e0790] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:43:59.716257 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-061d862f-1a2a-4af7-bbc9-3bac49f35382 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:43:59.717162 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-061d862f-1a2a-4af7-bbc9-3bac49f35382 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:43:59.718243 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-061d862f-1a2a-4af7-bbc9-3bac49f35382 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:43:59.719132 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-061d862f-1a2a-4af7-bbc9-3bac49f35382 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:43:59.981928 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-061d862f-1a2a-4af7-bbc9-3bac49f35382 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '833136cbdeab4b9e8b6dd2d24c7c8753', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '77c9b7e351e24dc88207e9498f75f578', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:43:59.994889 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-061d862f-1a2a-4af7-bbc9-3bac49f35382 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] 10.222.0.16 "GET /compute/v2.1/servers/64b5a7a2-7d82-4c60-802e-8e92c14e0790" status: 200 len: 1692 microversion: 2.1 time: 0.304117 Jul 16 21:43:59.996167 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2836/5668] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:43:59 2022] GET /compute/v2.1/servers/64b5a7a2-7d82-4c60-802e-8e92c14e0790 => generated 1692 bytes in 307 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:44:00.069927 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-682c1775-4a35-431c-a246-bcb181e55edf req-5f1efffa-c1d9-4ec0-b280-3895b24c7d0e service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "64b5a7a2-7d82-4c60-802e-8e92c14e0790", "name": "network-vif-unplugged", "status": "completed", "tag": "7a7ec2ad-db68-4392-93f0-85d2811e10e1"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:44:00.081251 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-682c1775-4a35-431c-a246-bcb181e55edf req-5f1efffa-c1d9-4ec0-b280-3895b24c7d0e service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:44:00.082045 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-682c1775-4a35-431c-a246-bcb181e55edf req-5f1efffa-c1d9-4ec0-b280-3895b24c7d0e service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:44:00.095889 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-682c1775-4a35-431c-a246-bcb181e55edf req-5f1efffa-c1d9-4ec0-b280-3895b24c7d0e service nova] Creating event network-vif-unplugged:7a7ec2ad-db68-4392-93f0-85d2811e10e1 for instance 64b5a7a2-7d82-4c60-802e-8e92c14e0790 on nch1-master-1 Jul 16 21:44:00.101649 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-682c1775-4a35-431c-a246-bcb181e55edf req-5f1efffa-c1d9-4ec0-b280-3895b24c7d0e service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.035354 Jul 16 21:44:00.102814 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2833/5669] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:44:00 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:44:01.021704 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-f5125113-ba7d-410b-87b4-33dbd43405d1 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:44:01.023166 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-f5125113-ba7d-410b-87b4-33dbd43405d1 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] [instance: 64b5a7a2-7d82-4c60-802e-8e92c14e0790] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:44:01.031625 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f5125113-ba7d-410b-87b4-33dbd43405d1 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:44:01.032500 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f5125113-ba7d-410b-87b4-33dbd43405d1 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:44:01.034090 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f5125113-ba7d-410b-87b4-33dbd43405d1 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:44:01.034785 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f5125113-ba7d-410b-87b4-33dbd43405d1 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:44:01.227617 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-f5125113-ba7d-410b-87b4-33dbd43405d1 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '833136cbdeab4b9e8b6dd2d24c7c8753', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '77c9b7e351e24dc88207e9498f75f578', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:44:01.242987 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-f5125113-ba7d-410b-87b4-33dbd43405d1 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] 10.222.0.16 "GET /compute/v2.1/servers/64b5a7a2-7d82-4c60-802e-8e92c14e0790" status: 200 len: 1692 microversion: 2.1 time: 0.233691 Jul 16 21:44:01.244882 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2837/5670] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:44:01 2022] GET /compute/v2.1/servers/64b5a7a2-7d82-4c60-802e-8e92c14e0790 => generated 1692 bytes in 236 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:44:02.120240 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-0a0766ea-2d5d-4a95-91a9-d6ade1499170 req-098f452d-acb9-4f5a-9b92-1b67b5c3a839 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "64b5a7a2-7d82-4c60-802e-8e92c14e0790", "tag": "7a7ec2ad-db68-4392-93f0-85d2811e10e1"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:44:02.132645 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-0a0766ea-2d5d-4a95-91a9-d6ade1499170 req-098f452d-acb9-4f5a-9b92-1b67b5c3a839 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:44:02.133253 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-0a0766ea-2d5d-4a95-91a9-d6ade1499170 req-098f452d-acb9-4f5a-9b92-1b67b5c3a839 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:44:02.149946 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-0a0766ea-2d5d-4a95-91a9-d6ade1499170 req-098f452d-acb9-4f5a-9b92-1b67b5c3a839 service nova] Creating event network-changed:7a7ec2ad-db68-4392-93f0-85d2811e10e1 for instance 64b5a7a2-7d82-4c60-802e-8e92c14e0790 on nch1-master-1 Jul 16 21:44:02.151870 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-0a0766ea-2d5d-4a95-91a9-d6ade1499170 req-098f452d-acb9-4f5a-9b92-1b67b5c3a839 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.035547 Jul 16 21:44:02.152594 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2834/5671] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:44:02 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:44:02.263957 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-9453281c-bd8f-4ec4-86c5-fdb902da413f tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:44:02.265139 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-9453281c-bd8f-4ec4-86c5-fdb902da413f tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] [instance: 64b5a7a2-7d82-4c60-802e-8e92c14e0790] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:44:02.276708 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9453281c-bd8f-4ec4-86c5-fdb902da413f tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:44:02.277190 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9453281c-bd8f-4ec4-86c5-fdb902da413f tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:44:02.278198 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9453281c-bd8f-4ec4-86c5-fdb902da413f tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:44:02.279444 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9453281c-bd8f-4ec4-86c5-fdb902da413f tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:44:02.439546 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-9453281c-bd8f-4ec4-86c5-fdb902da413f tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '833136cbdeab4b9e8b6dd2d24c7c8753', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '77c9b7e351e24dc88207e9498f75f578', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:44:02.448899 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-9453281c-bd8f-4ec4-86c5-fdb902da413f tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] 10.222.0.16 "GET /compute/v2.1/servers/64b5a7a2-7d82-4c60-802e-8e92c14e0790" status: 200 len: 1692 microversion: 2.1 time: 0.190314 Jul 16 21:44:02.451745 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2838/5672] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:44:02 2022] GET /compute/v2.1/servers/64b5a7a2-7d82-4c60-802e-8e92c14e0790 => generated 1692 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:44:03.470342 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-4232f3a7-1c14-4ae3-aba6-d791fcfb9d64 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:44:03.478804 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-4232f3a7-1c14-4ae3-aba6-d791fcfb9d64 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] [instance: 64b5a7a2-7d82-4c60-802e-8e92c14e0790] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:44:03.488698 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4232f3a7-1c14-4ae3-aba6-d791fcfb9d64 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:44:03.494391 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4232f3a7-1c14-4ae3-aba6-d791fcfb9d64 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:44:03.494391 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4232f3a7-1c14-4ae3-aba6-d791fcfb9d64 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:44:03.494391 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4232f3a7-1c14-4ae3-aba6-d791fcfb9d64 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:44:03.993337 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-4232f3a7-1c14-4ae3-aba6-d791fcfb9d64 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '833136cbdeab4b9e8b6dd2d24c7c8753', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '77c9b7e351e24dc88207e9498f75f578', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:44:04.003361 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-4232f3a7-1c14-4ae3-aba6-d791fcfb9d64 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] 10.222.0.16 "GET /compute/v2.1/servers/64b5a7a2-7d82-4c60-802e-8e92c14e0790" status: 200 len: 1692 microversion: 2.1 time: 0.538504 Jul 16 21:44:04.008587 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2835/5673] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:44:03 2022] GET /compute/v2.1/servers/64b5a7a2-7d82-4c60-802e-8e92c14e0790 => generated 1692 bytes in 545 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:44:05.026470 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-c9c20a41-54bd-49dd-9351-2dabbbb2ba75 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:44:05.027882 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-c9c20a41-54bd-49dd-9351-2dabbbb2ba75 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] [instance: 64b5a7a2-7d82-4c60-802e-8e92c14e0790] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:44:05.043669 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c9c20a41-54bd-49dd-9351-2dabbbb2ba75 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:44:05.043669 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c9c20a41-54bd-49dd-9351-2dabbbb2ba75 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:44:05.043669 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c9c20a41-54bd-49dd-9351-2dabbbb2ba75 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:44:05.052304 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c9c20a41-54bd-49dd-9351-2dabbbb2ba75 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:44:05.415676 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-c9c20a41-54bd-49dd-9351-2dabbbb2ba75 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '833136cbdeab4b9e8b6dd2d24c7c8753', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '77c9b7e351e24dc88207e9498f75f578', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:44:05.431481 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-c9c20a41-54bd-49dd-9351-2dabbbb2ba75 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] 10.222.0.16 "GET /compute/v2.1/servers/64b5a7a2-7d82-4c60-802e-8e92c14e0790" status: 200 len: 1692 microversion: 2.1 time: 0.409836 Jul 16 21:44:05.432898 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2839/5674] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:44:05 2022] GET /compute/v2.1/servers/64b5a7a2-7d82-4c60-802e-8e92c14e0790 => generated 1692 bytes in 412 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:44:06.450686 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-05405853-064a-42ab-a324-cf9113caf674 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:44:06.452745 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-05405853-064a-42ab-a324-cf9113caf674 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] [instance: 64b5a7a2-7d82-4c60-802e-8e92c14e0790] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:44:06.466196 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-05405853-064a-42ab-a324-cf9113caf674 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:44:06.466196 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-05405853-064a-42ab-a324-cf9113caf674 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:44:06.466196 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-05405853-064a-42ab-a324-cf9113caf674 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:44:06.466196 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-05405853-064a-42ab-a324-cf9113caf674 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:44:06.671089 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-05405853-064a-42ab-a324-cf9113caf674 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '833136cbdeab4b9e8b6dd2d24c7c8753', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '77c9b7e351e24dc88207e9498f75f578', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:44:06.680426 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-05405853-064a-42ab-a324-cf9113caf674 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] 10.222.0.16 "GET /compute/v2.1/servers/64b5a7a2-7d82-4c60-802e-8e92c14e0790" status: 200 len: 1692 microversion: 2.1 time: 0.234558 Jul 16 21:44:06.681639 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2836/5675] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:44:06 2022] GET /compute/v2.1/servers/64b5a7a2-7d82-4c60-802e-8e92c14e0790 => generated 1692 bytes in 236 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:44:06.975344 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-08134137-a39b-44ec-b6ba-69907980044a req-67f81417-2c1b-4dcc-b1db-3f39cdaa77db service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "64b5a7a2-7d82-4c60-802e-8e92c14e0790", "name": "network-vif-unplugged", "status": "completed", "tag": "7a7ec2ad-db68-4392-93f0-85d2811e10e1"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:44:06.984744 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-08134137-a39b-44ec-b6ba-69907980044a req-67f81417-2c1b-4dcc-b1db-3f39cdaa77db service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:44:06.985201 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-08134137-a39b-44ec-b6ba-69907980044a req-67f81417-2c1b-4dcc-b1db-3f39cdaa77db service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:44:06.990285 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=87,nova_cell1:UPDATE=3,nova_cell1:INSERT=2 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:44:07.000775 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-08134137-a39b-44ec-b6ba-69907980044a req-67f81417-2c1b-4dcc-b1db-3f39cdaa77db service nova] Creating event network-vif-unplugged:7a7ec2ad-db68-4392-93f0-85d2811e10e1 for instance 64b5a7a2-7d82-4c60-802e-8e92c14e0790 on nch1-master-1 Jul 16 21:44:07.012034 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-08134137-a39b-44ec-b6ba-69907980044a req-67f81417-2c1b-4dcc-b1db-3f39cdaa77db service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.041252 Jul 16 21:44:07.012943 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2840/5676] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:44:06 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:44:07.699035 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-08ee8b74-6e40-4b3b-8f07-97c2b5b30170 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:44:07.701228 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-08ee8b74-6e40-4b3b-8f07-97c2b5b30170 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] [instance: 64b5a7a2-7d82-4c60-802e-8e92c14e0790] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:44:07.707310 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-08ee8b74-6e40-4b3b-8f07-97c2b5b30170 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:44:07.708714 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-08ee8b74-6e40-4b3b-8f07-97c2b5b30170 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:44:07.709717 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-08ee8b74-6e40-4b3b-8f07-97c2b5b30170 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:44:07.710333 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-08ee8b74-6e40-4b3b-8f07-97c2b5b30170 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:44:07.906945 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-08ee8b74-6e40-4b3b-8f07-97c2b5b30170 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '833136cbdeab4b9e8b6dd2d24c7c8753', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '77c9b7e351e24dc88207e9498f75f578', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:44:07.916149 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-08ee8b74-6e40-4b3b-8f07-97c2b5b30170 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] 10.222.0.16 "GET /compute/v2.1/servers/64b5a7a2-7d82-4c60-802e-8e92c14e0790" status: 200 len: 1692 microversion: 2.1 time: 0.220577 Jul 16 21:44:07.928637 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2837/5677] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:44:07 2022] GET /compute/v2.1/servers/64b5a7a2-7d82-4c60-802e-8e92c14e0790 => generated 1692 bytes in 234 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:44:08.940574 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-19db7270-cb1d-4178-853d-b5cebdfb97ff tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:44:08.948197 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-19db7270-cb1d-4178-853d-b5cebdfb97ff tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] [instance: 64b5a7a2-7d82-4c60-802e-8e92c14e0790] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:44:08.977722 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-19db7270-cb1d-4178-853d-b5cebdfb97ff tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:44:08.982649 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-19db7270-cb1d-4178-853d-b5cebdfb97ff tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:44:08.982649 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-19db7270-cb1d-4178-853d-b5cebdfb97ff tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:44:08.982649 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-19db7270-cb1d-4178-853d-b5cebdfb97ff tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:44:09.824059 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-19db7270-cb1d-4178-853d-b5cebdfb97ff tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '833136cbdeab4b9e8b6dd2d24c7c8753', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '77c9b7e351e24dc88207e9498f75f578', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:44:09.833734 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-19db7270-cb1d-4178-853d-b5cebdfb97ff tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] 10.222.0.16 "GET /compute/v2.1/servers/64b5a7a2-7d82-4c60-802e-8e92c14e0790" status: 200 len: 1692 microversion: 2.1 time: 0.903409 Jul 16 21:44:09.843839 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2841/5678] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:44:08 2022] GET /compute/v2.1/servers/64b5a7a2-7d82-4c60-802e-8e92c14e0790 => generated 1692 bytes in 914 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:44:10.854549 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-70791a14-2ef0-4183-8194-f10569bdbd56 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:44:10.856809 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-70791a14-2ef0-4183-8194-f10569bdbd56 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] [instance: 64b5a7a2-7d82-4c60-802e-8e92c14e0790] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:44:10.868983 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-70791a14-2ef0-4183-8194-f10569bdbd56 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:44:10.870197 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-70791a14-2ef0-4183-8194-f10569bdbd56 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:44:10.873542 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-70791a14-2ef0-4183-8194-f10569bdbd56 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:44:10.875958 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-70791a14-2ef0-4183-8194-f10569bdbd56 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:44:11.238464 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-70791a14-2ef0-4183-8194-f10569bdbd56 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '833136cbdeab4b9e8b6dd2d24c7c8753', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '77c9b7e351e24dc88207e9498f75f578', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:44:11.256781 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-70791a14-2ef0-4183-8194-f10569bdbd56 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] 10.222.0.16 "GET /compute/v2.1/servers/64b5a7a2-7d82-4c60-802e-8e92c14e0790" status: 200 len: 1682 microversion: 2.1 time: 0.406300 Jul 16 21:44:11.258978 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2838/5679] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:44:10 2022] GET /compute/v2.1/servers/64b5a7a2-7d82-4c60-802e-8e92c14e0790 => generated 1682 bytes in 409 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:44:11.279903 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-f8758c17-b045-49e0-b7f3-b3a13d4ac1ec tempest-TestNetworkAdvancedServerOps-756251536 tempest-TestNetworkAdvancedServerOps-756251536-project-admin] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:44:11.283339 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-f8758c17-b045-49e0-b7f3-b3a13d4ac1ec tempest-TestNetworkAdvancedServerOps-756251536 tempest-TestNetworkAdvancedServerOps-756251536-project-admin] [instance: 64b5a7a2-7d82-4c60-802e-8e92c14e0790] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:44:11.295926 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f8758c17-b045-49e0-b7f3-b3a13d4ac1ec tempest-TestNetworkAdvancedServerOps-756251536 tempest-TestNetworkAdvancedServerOps-756251536-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:44:11.305713 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f8758c17-b045-49e0-b7f3-b3a13d4ac1ec tempest-TestNetworkAdvancedServerOps-756251536 tempest-TestNetworkAdvancedServerOps-756251536-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.009s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:44:11.313382 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f8758c17-b045-49e0-b7f3-b3a13d4ac1ec tempest-TestNetworkAdvancedServerOps-756251536 tempest-TestNetworkAdvancedServerOps-756251536-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:44:11.313382 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f8758c17-b045-49e0-b7f3-b3a13d4ac1ec tempest-TestNetworkAdvancedServerOps-756251536 tempest-TestNetworkAdvancedServerOps-756251536-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:44:11.743215 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-f8758c17-b045-49e0-b7f3-b3a13d4ac1ec tempest-TestNetworkAdvancedServerOps-756251536 tempest-TestNetworkAdvancedServerOps-756251536-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/64b5a7a2-7d82-4c60-802e-8e92c14e0790" status: 200 len: 1833 microversion: 2.1 time: 0.469378 Jul 16 21:44:11.747563 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2842/5680] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:44:11 2022] GET /compute/v2.1/servers/64b5a7a2-7d82-4c60-802e-8e92c14e0790 => generated 1833 bytes in 476 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:44:11.771241 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-c144bfca-529d-4a28-b3de-e30c7c1d4595 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:44:11.772100 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-c144bfca-529d-4a28-b3de-e30c7c1d4595 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] [instance: 64b5a7a2-7d82-4c60-802e-8e92c14e0790] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:44:11.779325 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c144bfca-529d-4a28-b3de-e30c7c1d4595 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:44:11.780018 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c144bfca-529d-4a28-b3de-e30c7c1d4595 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:44:11.782020 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c144bfca-529d-4a28-b3de-e30c7c1d4595 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:44:11.782628 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c144bfca-529d-4a28-b3de-e30c7c1d4595 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:44:12.169574 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-c144bfca-529d-4a28-b3de-e30c7c1d4595 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '833136cbdeab4b9e8b6dd2d24c7c8753', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '77c9b7e351e24dc88207e9498f75f578', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:44:12.178872 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-c144bfca-529d-4a28-b3de-e30c7c1d4595 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] 10.222.0.16 "GET /compute/v2.1/servers/64b5a7a2-7d82-4c60-802e-8e92c14e0790" status: 200 len: 1682 microversion: 2.1 time: 0.413827 Jul 16 21:44:12.195986 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2839/5681] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:44:11 2022] GET /compute/v2.1/servers/64b5a7a2-7d82-4c60-802e-8e92c14e0790 => generated 1682 bytes in 432 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:44:12.337504 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-4521b743-a5e7-488c-94d4-4103ab74d0f4 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Action: 'action', calling method: >, body: {"os-getConsoleOutput": {}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:44:12.338637 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-4521b743-a5e7-488c-94d4-4103ab74d0f4 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] [instance: 64b5a7a2-7d82-4c60-802e-8e92c14e0790] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:44:12.349430 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4521b743-a5e7-488c-94d4-4103ab74d0f4 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:44:12.352226 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4521b743-a5e7-488c-94d4-4103ab74d0f4 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:44:12.353793 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4521b743-a5e7-488c-94d4-4103ab74d0f4 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:44:12.354499 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4521b743-a5e7-488c-94d4-4103ab74d0f4 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:44:12.442157 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-4521b743-a5e7-488c-94d4-4103ab74d0f4 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] 10.222.0.16 "POST /compute/v2.1/servers/64b5a7a2-7d82-4c60-802e-8e92c14e0790/action" status: 200 len: 38621 microversion: 2.1 time: 0.102838 Jul 16 21:44:12.442157 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2843/5682] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 21:44:12 2022] POST /compute/v2.1/servers/64b5a7a2-7d82-4c60-802e-8e92c14e0790/action => generated 38621 bytes in 106 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 16 21:44:14.217554 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-aaacaab2-618f-4e69-91dd-eb267ba9d3f2 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:44:14.218757 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-aaacaab2-618f-4e69-91dd-eb267ba9d3f2 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] [instance: 64b5a7a2-7d82-4c60-802e-8e92c14e0790] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:44:14.221900 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-80f03aa2-9dbc-4dfe-bad9-a28ccc6865d5 req-28b25d7f-c8ee-484a-9634-95302a4b19a7 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "64b5a7a2-7d82-4c60-802e-8e92c14e0790", "tag": "7a7ec2ad-db68-4392-93f0-85d2811e10e1"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:44:14.233295 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-80f03aa2-9dbc-4dfe-bad9-a28ccc6865d5 req-28b25d7f-c8ee-484a-9634-95302a4b19a7 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:44:14.233580 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-aaacaab2-618f-4e69-91dd-eb267ba9d3f2 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:44:14.234070 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-aaacaab2-618f-4e69-91dd-eb267ba9d3f2 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:44:14.234588 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-80f03aa2-9dbc-4dfe-bad9-a28ccc6865d5 req-28b25d7f-c8ee-484a-9634-95302a4b19a7 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:44:14.234842 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-aaacaab2-618f-4e69-91dd-eb267ba9d3f2 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:44:14.235311 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-aaacaab2-618f-4e69-91dd-eb267ba9d3f2 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:44:14.248384 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-80f03aa2-9dbc-4dfe-bad9-a28ccc6865d5 req-28b25d7f-c8ee-484a-9634-95302a4b19a7 service nova] Creating event network-changed:7a7ec2ad-db68-4392-93f0-85d2811e10e1 for instance 64b5a7a2-7d82-4c60-802e-8e92c14e0790 on nch2-master-1 Jul 16 21:44:14.261079 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-80f03aa2-9dbc-4dfe-bad9-a28ccc6865d5 req-28b25d7f-c8ee-484a-9634-95302a4b19a7 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.044731 Jul 16 21:44:14.263506 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2844/5683] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:44:14 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:44:14.283245 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-aaacaab2-618f-4e69-91dd-eb267ba9d3f2 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] [instance: 64b5a7a2-7d82-4c60-802e-8e92c14e0790] Going to try to terminate instance {{(pid=138680) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 16 21:44:14.594572 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-aaacaab2-618f-4e69-91dd-eb267ba9d3f2 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] 10.222.0.16 "DELETE /compute/v2.1/servers/64b5a7a2-7d82-4c60-802e-8e92c14e0790" status: 204 len: 0 microversion: 2.1 time: 0.381833 Jul 16 21:44:14.596740 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2840/5684] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:44:14 2022] DELETE /compute/v2.1/servers/64b5a7a2-7d82-4c60-802e-8e92c14e0790 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 16 21:44:14.606284 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-858548b5-4efa-4081-b8ab-79bb5b5821bb tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:44:14.607039 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-858548b5-4efa-4081-b8ab-79bb5b5821bb tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] [instance: 64b5a7a2-7d82-4c60-802e-8e92c14e0790] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:44:14.615617 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-858548b5-4efa-4081-b8ab-79bb5b5821bb tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:44:14.625533 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-858548b5-4efa-4081-b8ab-79bb5b5821bb tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.010s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:44:14.626603 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-858548b5-4efa-4081-b8ab-79bb5b5821bb tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:44:14.627965 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-858548b5-4efa-4081-b8ab-79bb5b5821bb tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:44:14.889256 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-858548b5-4efa-4081-b8ab-79bb5b5821bb tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '833136cbdeab4b9e8b6dd2d24c7c8753', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '77c9b7e351e24dc88207e9498f75f578', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:44:14.900884 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-858548b5-4efa-4081-b8ab-79bb5b5821bb tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] 10.222.0.16 "GET /compute/v2.1/servers/64b5a7a2-7d82-4c60-802e-8e92c14e0790" status: 200 len: 1688 microversion: 2.1 time: 0.299574 Jul 16 21:44:14.912947 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2845/5685] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:44:14 2022] GET /compute/v2.1/servers/64b5a7a2-7d82-4c60-802e-8e92c14e0790 => generated 1688 bytes in 312 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:44:15.923085 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-5d61613f-861d-4cab-bcac-f554ab3d2294 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:44:15.924416 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-5d61613f-861d-4cab-bcac-f554ab3d2294 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] [instance: 64b5a7a2-7d82-4c60-802e-8e92c14e0790] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:44:15.948891 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5d61613f-861d-4cab-bcac-f554ab3d2294 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:44:15.949397 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5d61613f-861d-4cab-bcac-f554ab3d2294 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:44:15.950408 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5d61613f-861d-4cab-bcac-f554ab3d2294 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:44:15.950861 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5d61613f-861d-4cab-bcac-f554ab3d2294 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:44:16.419721 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-5d61613f-861d-4cab-bcac-f554ab3d2294 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '833136cbdeab4b9e8b6dd2d24c7c8753', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '77c9b7e351e24dc88207e9498f75f578', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:44:16.428958 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-5d61613f-861d-4cab-bcac-f554ab3d2294 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] 10.222.0.16 "GET /compute/v2.1/servers/64b5a7a2-7d82-4c60-802e-8e92c14e0790" status: 200 len: 1688 microversion: 2.1 time: 0.512361 Jul 16 21:44:16.429568 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2841/5686] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:44:15 2022] GET /compute/v2.1/servers/64b5a7a2-7d82-4c60-802e-8e92c14e0790 => generated 1688 bytes in 514 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:44:16.804546 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-ac685c59-1c27-4732-8775-f237285b7318 req-968835d0-4a7c-4316-8935-ccd1dcd633bf service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "64b5a7a2-7d82-4c60-802e-8e92c14e0790", "name": "network-vif-deleted", "tag": "7a7ec2ad-db68-4392-93f0-85d2811e10e1"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:44:16.837213 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-ac685c59-1c27-4732-8775-f237285b7318 req-968835d0-4a7c-4316-8935-ccd1dcd633bf service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:44:16.843215 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-ac685c59-1c27-4732-8775-f237285b7318 req-968835d0-4a7c-4316-8935-ccd1dcd633bf service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:44:16.867028 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-ac685c59-1c27-4732-8775-f237285b7318 req-968835d0-4a7c-4316-8935-ccd1dcd633bf service nova] Creating event network-vif-deleted:7a7ec2ad-db68-4392-93f0-85d2811e10e1 for instance 64b5a7a2-7d82-4c60-802e-8e92c14e0790 on nch2-master-1 Jul 16 21:44:16.871433 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-ac685c59-1c27-4732-8775-f237285b7318 req-968835d0-4a7c-4316-8935-ccd1dcd633bf service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.072258 Jul 16 21:44:16.872490 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2846/5687] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:44:16 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 74 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:44:17.448039 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-2fabe972-1097-41e8-acfd-7f97f608eb46 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:44:17.450598 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-2fabe972-1097-41e8-acfd-7f97f608eb46 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] [instance: 64b5a7a2-7d82-4c60-802e-8e92c14e0790] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:44:17.478302 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2fabe972-1097-41e8-acfd-7f97f608eb46 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:44:17.478887 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2fabe972-1097-41e8-acfd-7f97f608eb46 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:44:17.480183 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2fabe972-1097-41e8-acfd-7f97f608eb46 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:44:17.485796 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2fabe972-1097-41e8-acfd-7f97f608eb46 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:44:17.546878 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2fabe972-1097-41e8-acfd-7f97f608eb46 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:44:17.547798 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2fabe972-1097-41e8-acfd-7f97f608eb46 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:44:17.548506 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.objects.instance [None req-2fabe972-1097-41e8-acfd-7f97f608eb46 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lazy-loading 'fault' on Instance uuid 64b5a7a2-7d82-4c60-802e-8e92c14e0790 {{(pid=138680) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:44:17.602934 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-2fabe972-1097-41e8-acfd-7f97f608eb46 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '833136cbdeab4b9e8b6dd2d24c7c8753', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '77c9b7e351e24dc88207e9498f75f578', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:44:17.613215 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-2fabe972-1097-41e8-acfd-7f97f608eb46 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] 10.222.0.16 "GET /compute/v2.1/servers/64b5a7a2-7d82-4c60-802e-8e92c14e0790" status: 200 len: 1356 microversion: 2.1 time: 0.170421 Jul 16 21:44:17.624501 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2842/5688] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:44:17 2022] GET /compute/v2.1/servers/64b5a7a2-7d82-4c60-802e-8e92c14e0790 => generated 1356 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:44:18.634497 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-ed7b0ea5-0752-4b7b-896f-4f6268993018 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:44:18.636393 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-ed7b0ea5-0752-4b7b-896f-4f6268993018 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] [instance: 64b5a7a2-7d82-4c60-802e-8e92c14e0790] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:44:18.645567 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ed7b0ea5-0752-4b7b-896f-4f6268993018 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:44:18.646878 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ed7b0ea5-0752-4b7b-896f-4f6268993018 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:44:18.648476 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ed7b0ea5-0752-4b7b-896f-4f6268993018 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:44:18.649281 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ed7b0ea5-0752-4b7b-896f-4f6268993018 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:44:18.669004 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-ed7b0ea5-0752-4b7b-896f-4f6268993018 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] HTTP exception thrown: Instance 64b5a7a2-7d82-4c60-802e-8e92c14e0790 could not be found. Jul 16 21:44:18.670729 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-ed7b0ea5-0752-4b7b-896f-4f6268993018 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Request method failure captured: Jul 16 21:44:18.670729 ncdv-master-1 devstack@n-api.service[138681]: request: GET /compute/v2.1/servers/64b5a7a2-7d82-4c60-802e-8e92c14e0790 HTTP/1.1 Jul 16 21:44:18.670729 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:44:18.670729 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:44:18.670729 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:44:18.670729 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 21:44:18.670729 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:44:18.670729 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:44:18.670729 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:44:18.670729 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:44:18.670729 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0y_J0q_hylsRBv1tuKDuvEX4h0-KJOaBkYlsP5Ic98XDN3p0UKvwASS-7n2zGgAIPr7rp74I-PGAzLIn5xeDmpiRzwuaSoKYKWvZ4y05aUl8opleXxkVzk0FUY6GEflOf-1AEqT9qASXsqi7YhmGfOHelDZ7TomdNPlQ8IF_8wXqbKY Jul 16 21:44:18.670729 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:44:18.670729 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:44:18.670729 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:44:18.670729 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:44:18.670729 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:44:18.670729 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:44:18.670729 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: 77c9b7e351e24dc88207e9498f75f578 Jul 16 21:44:18.670729 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-TestNetworkAdvancedServerOps-375332741 Jul 16 21:44:18.670729 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:44:18.670729 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:44:18.670729 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/77c9b7e351e24dc88207e9498f75f578"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/77c9b7e351e24dc88207e9498f75f578"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_77c9b7e351e24dc88207e9498f75f578"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/77c9b7e351e24dc88207e9498f75f578"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:44:18.670729 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-TestNetworkAdvancedServerOps-375332741 Jul 16 21:44:18.670729 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: 77c9b7e351e24dc88207e9498f75f578 Jul 16 21:44:18.670729 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-TestNetworkAdvancedServerOps-375332741 Jul 16 21:44:18.670729 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-TestNetworkAdvancedServerOps-375332741-project Jul 16 21:44:18.670729 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:44:18.670729 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:44:18.670729 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: 833136cbdeab4b9e8b6dd2d24c7c8753 Jul 16 21:44:18.670729 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-TestNetworkAdvancedServerOps-375332741-project Jul 16 21:44:18.670729 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:44:18.670729 ncdv-master-1 devstack@n-api.service[138681]: exception: Instance 64b5a7a2-7d82-4c60-802e-8e92c14e0790 could not be found. Jul 16 21:44:18.670729 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': '64b5a7a2-7d82-4c60-802e-8e92c14e0790'} Jul 16 21:44:18.670729 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:44:18.670729 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:44:18.670729 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:44:18.670729 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:44:18.670729 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:44:18.670729 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:44:18.670729 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:44:18.670729 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:44:18.670729 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:44:18.670729 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:44:18.670729 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 64b5a7a2-7d82-4c60-802e-8e92c14e0790 could not be found. Jul 16 21:44:18.670729 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:44:18.670729 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:44:18.670729 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:44:18.670729 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:44:18.670729 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:44:18.670729 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:44:18.670729 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:44:18.670729 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:44:18.670729 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:44:18.670729 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:44:18.670729 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:44:18.670729 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:44:18.670729 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:44:18.670729 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:44:18.670729 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:44:18.670729 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:44:18.670729 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 64b5a7a2-7d82-4c60-802e-8e92c14e0790 could not be found. Jul 16 21:44:18.670729 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:44:18.670729 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:44:18.670729 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:44:18.677241 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:44:18.677241 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:44:18.677241 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:44:18.677241 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:44:18.677241 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:44:18.677241 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 64b5a7a2-7d82-4c60-802e-8e92c14e0790 could not be found. Jul 16 21:44:18.677241 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:44:18.677241 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-ed7b0ea5-0752-4b7b-896f-4f6268993018 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Returning 404 to user: Instance 64b5a7a2-7d82-4c60-802e-8e92c14e0790 could not be found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:44:18.677241 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-ed7b0ea5-0752-4b7b-896f-4f6268993018 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] 10.222.0.16 "GET /compute/v2.1/servers/64b5a7a2-7d82-4c60-802e-8e92c14e0790" status: 404 len: 111 microversion: 2.1 time: 0.044100 Jul 16 21:44:18.677989 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2847/5689] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:44:18 2022] GET /compute/v2.1/servers/64b5a7a2-7d82-4c60-802e-8e92c14e0790 => generated 111 bytes in 51 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:44:25.650170 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-0c50ffeb-5248-4ae4-ae9e-110703bd53a6 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Calling method '.version_select at 0x7fdda2a5d790>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:44:25.684378 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_api:SELECT=33,nova_api:UPDATE=1 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:44:25.765463 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-0c50ffeb-5248-4ae4-ae9e-110703bd53a6 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] 10.222.0.16 "DELETE /compute/v2.1/os-keypairs/tempest-TestNetworkAdvancedServerOps-343919104" status: 202 len: 0 microversion: 2.1 time: 0.123911 Jul 16 21:44:25.770888 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2843/5690] 10.222.0.16 () {58 vars in 1280 bytes} [Sat Jul 16 21:44:25 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestNetworkAdvancedServerOps-343919104 => generated 0 bytes in 126 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 21:44:25.800821 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-f0d41c72-54ba-4915-b5f5-456a6076360d tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Action: 'create', calling method: .version_select at 0x7fdd8c5579d0>, body: {"keypair": {"name": "tempest-TestNetworkAdvancedServerOps-800712517", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBCznC2WuQAgSbbnIsxc/gVizVn2Dgi5gAuSOmII64RRlw9eve6YgTnktB/iR0nvsA0XDmojCnLoyw3HXyYmwgMW2tscNbbFYcyHNc6K/6Q7fK94UUCej0XS/jdEGXPJvhg=="}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:44:25.806272 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_api:SELECT=33,nova_api:DELETE=1 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:44:25.840799 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-f0d41c72-54ba-4915-b5f5-456a6076360d tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Getting quotas for project 77c9b7e351e24dc88207e9498f75f578. Resources: {'key_pairs'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:44:25.848682 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-f0d41c72-54ba-4915-b5f5-456a6076360d tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Getting quotas for user 833136cbdeab4b9e8b6dd2d24c7c8753 and project 77c9b7e351e24dc88207e9498f75f578. Resources: {'key_pairs'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:44:25.972366 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-f0d41c72-54ba-4915-b5f5-456a6076360d tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] 10.222.0.16 "POST /compute/v2.1/os-keypairs" status: 200 len: 406 microversion: 2.1 time: 0.176522 Jul 16 21:44:25.973022 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2848/5691] 10.222.0.16 () {60 vars in 1158 bytes} [Sat Jul 16 21:44:25 2022] POST /compute/v2.1/os-keypairs => generated 406 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:44:26.311575 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-84f8a87e-cb05-4efe-be6b-cf88d78ffb67 tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-InstanceActionsTestJSON-server-959431053", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452", "networks": [{"uuid": "79433876-0e3a-4679-9d93-5915aa6cd875"}]}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:44:26.367904 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.network.neutron [None req-84f8a87e-cb05-4efe-be6b-cf88d78ffb67 tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] validate_networks() for [('79433876-0e3a-4679-9d93-5915aa6cd875', None, None, None, None, None)] {{(pid=138680) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 16 21:44:26.869003 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-84f8a87e-cb05-4efe-be6b-cf88d78ffb67 tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] Flavor limits 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 16 21:44:26.869557 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-84f8a87e-cb05-4efe-be6b-cf88d78ffb67 tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] Image limits 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 16 21:44:26.869915 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-84f8a87e-cb05-4efe-be6b-cf88d78ffb67 tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] Flavor pref 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 16 21:44:26.870527 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-84f8a87e-cb05-4efe-be6b-cf88d78ffb67 tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] Image pref 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 16 21:44:26.870956 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-84f8a87e-cb05-4efe-be6b-cf88d78ffb67 tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 16 21:44:26.882520 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-84f8a87e-cb05-4efe-be6b-cf88d78ffb67 tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:44:26.882963 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-84f8a87e-cb05-4efe-be6b-cf88d78ffb67 tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:44:26.883855 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-84f8a87e-cb05-4efe-be6b-cf88d78ffb67 tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:44:26.884309 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-84f8a87e-cb05-4efe-be6b-cf88d78ffb67 tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:44:26.904424 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-84f8a87e-cb05-4efe-be6b-cf88d78ffb67 tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] Getting quotas for project 17459e07487944958c9c307e59e1c8dc. Resources: {'ram', 'instances', 'cores'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:44:26.911267 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-84f8a87e-cb05-4efe-be6b-cf88d78ffb67 tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] Getting quotas for user beae3bc561e141fc9229b40466840078 and project 17459e07487944958c9c307e59e1c8dc. Resources: {'ram', 'instances', 'cores'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:44:26.925636 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-84f8a87e-cb05-4efe-be6b-cf88d78ffb67 tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] Going to run 1 instances... {{(pid=138680) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 16 21:44:26.928026 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-84f8a87e-cb05-4efe-be6b-cf88d78ffb67 tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] [instance: 183fd84f-828f-49a3-9ec9-9831dbfcc555] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138680) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 16 21:44:27.126172 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-84f8a87e-cb05-4efe-be6b-cf88d78ffb67 tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 1.025333 Jul 16 21:44:27.129270 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2844/5692] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 21:44:26 2022] POST /compute/v2.1/servers => generated 380 bytes in 1027 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 16 21:44:27.138791 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-8b8ee80b-1e6e-4d33-aecd-ca4d47930c87 tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:44:27.139556 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-8b8ee80b-1e6e-4d33-aecd-ca4d47930c87 tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] [instance: 183fd84f-828f-49a3-9ec9-9831dbfcc555] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:44:27.212567 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-8b8ee80b-1e6e-4d33-aecd-ca4d47930c87 tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'beae3bc561e141fc9229b40466840078', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '17459e07487944958c9c307e59e1c8dc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:44:27.221645 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-8b8ee80b-1e6e-4d33-aecd-ca4d47930c87 tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] 10.222.0.16 "GET /compute/v2.1/servers/183fd84f-828f-49a3-9ec9-9831dbfcc555" status: 200 len: 1216 microversion: 2.1 time: 0.087688 Jul 16 21:44:27.222204 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2849/5693] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:44:27 2022] GET /compute/v2.1/servers/183fd84f-828f-49a3-9ec9-9831dbfcc555 => generated 1216 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:44:28.240210 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-8b8c182e-74dd-494b-a67c-7720b348c6b3 tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:44:28.240721 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-8b8c182e-74dd-494b-a67c-7720b348c6b3 tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] [instance: 183fd84f-828f-49a3-9ec9-9831dbfcc555] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:44:28.311684 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-8b8c182e-74dd-494b-a67c-7720b348c6b3 tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'beae3bc561e141fc9229b40466840078', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '17459e07487944958c9c307e59e1c8dc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:44:28.311684 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-8b8c182e-74dd-494b-a67c-7720b348c6b3 tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] 10.222.0.16 "GET /compute/v2.1/servers/183fd84f-828f-49a3-9ec9-9831dbfcc555" status: 200 len: 1216 microversion: 2.1 time: 0.073497 Jul 16 21:44:28.311684 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2845/5694] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:44:28 2022] GET /compute/v2.1/servers/183fd84f-828f-49a3-9ec9-9831dbfcc555 => generated 1216 bytes in 75 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:44:29.331896 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-d144b3ff-571a-4c05-8e68-3be336ad8d55 tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:44:29.333701 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-d144b3ff-571a-4c05-8e68-3be336ad8d55 tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] [instance: 183fd84f-828f-49a3-9ec9-9831dbfcc555] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:44:29.339822 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=29 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:44:29.346044 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d144b3ff-571a-4c05-8e68-3be336ad8d55 tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:44:29.346846 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d144b3ff-571a-4c05-8e68-3be336ad8d55 tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:44:29.347978 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d144b3ff-571a-4c05-8e68-3be336ad8d55 tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:44:29.348678 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d144b3ff-571a-4c05-8e68-3be336ad8d55 tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:44:29.465225 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-d144b3ff-571a-4c05-8e68-3be336ad8d55 tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'beae3bc561e141fc9229b40466840078', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '17459e07487944958c9c307e59e1c8dc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:44:29.491246 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-d144b3ff-571a-4c05-8e68-3be336ad8d55 tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] 10.222.0.16 "GET /compute/v2.1/servers/183fd84f-828f-49a3-9ec9-9831dbfcc555" status: 200 len: 1212 microversion: 2.1 time: 0.168787 Jul 16 21:44:29.492995 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2850/5695] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:44:29 2022] GET /compute/v2.1/servers/183fd84f-828f-49a3-9ec9-9831dbfcc555 => generated 1212 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:44:30.518664 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-9fa53f02-2cb2-4964-959f-37c35d3e1599 tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:44:30.528650 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-9fa53f02-2cb2-4964-959f-37c35d3e1599 tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] [instance: 183fd84f-828f-49a3-9ec9-9831dbfcc555] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:44:30.566740 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9fa53f02-2cb2-4964-959f-37c35d3e1599 tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:44:30.568462 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9fa53f02-2cb2-4964-959f-37c35d3e1599 tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:44:30.572234 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9fa53f02-2cb2-4964-959f-37c35d3e1599 tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:44:30.574972 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9fa53f02-2cb2-4964-959f-37c35d3e1599 tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:44:30.598614 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=72,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:44:30.730821 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-9fa53f02-2cb2-4964-959f-37c35d3e1599 tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'beae3bc561e141fc9229b40466840078', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '17459e07487944958c9c307e59e1c8dc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:44:30.737512 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-9fa53f02-2cb2-4964-959f-37c35d3e1599 tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] 10.222.0.16 "GET /compute/v2.1/servers/183fd84f-828f-49a3-9ec9-9831dbfcc555" status: 200 len: 1286 microversion: 2.1 time: 0.230304 Jul 16 21:44:30.738930 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2846/5696] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:44:30 2022] GET /compute/v2.1/servers/183fd84f-828f-49a3-9ec9-9831dbfcc555 => generated 1286 bytes in 233 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:44:31.760552 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-913984f5-2b42-439b-970e-e3cb6dd8af34 tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:44:31.762582 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-913984f5-2b42-439b-970e-e3cb6dd8af34 tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] [instance: 183fd84f-828f-49a3-9ec9-9831dbfcc555] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:44:31.777247 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-913984f5-2b42-439b-970e-e3cb6dd8af34 tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:44:31.778296 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-913984f5-2b42-439b-970e-e3cb6dd8af34 tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:44:31.779809 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-913984f5-2b42-439b-970e-e3cb6dd8af34 tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:44:31.780493 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-913984f5-2b42-439b-970e-e3cb6dd8af34 tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:44:32.077261 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-913984f5-2b42-439b-970e-e3cb6dd8af34 tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'beae3bc561e141fc9229b40466840078', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '17459e07487944958c9c307e59e1c8dc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:44:32.086953 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-913984f5-2b42-439b-970e-e3cb6dd8af34 tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] 10.222.0.16 "GET /compute/v2.1/servers/183fd84f-828f-49a3-9ec9-9831dbfcc555" status: 200 len: 1316 microversion: 2.1 time: 0.334402 Jul 16 21:44:32.096590 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2851/5697] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:44:31 2022] GET /compute/v2.1/servers/183fd84f-828f-49a3-9ec9-9831dbfcc555 => generated 1316 bytes in 337 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:44:32.719316 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-97f3edf0-87ac-40c5-ba22-f31d121fb5a2 req-71f0d135-5edc-4fd8-b9bc-525294fdde6d service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "183fd84f-828f-49a3-9ec9-9831dbfcc555", "tag": "fb93beeb-497d-431a-835c-95c0e5fb3fc3"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:44:32.744934 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-97f3edf0-87ac-40c5-ba22-f31d121fb5a2 req-71f0d135-5edc-4fd8-b9bc-525294fdde6d service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:44:32.745395 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-97f3edf0-87ac-40c5-ba22-f31d121fb5a2 req-71f0d135-5edc-4fd8-b9bc-525294fdde6d service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:44:32.768601 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-97f3edf0-87ac-40c5-ba22-f31d121fb5a2 req-71f0d135-5edc-4fd8-b9bc-525294fdde6d service nova] Creating event network-changed:fb93beeb-497d-431a-835c-95c0e5fb3fc3 for instance 183fd84f-828f-49a3-9ec9-9831dbfcc555 on nch2-master-1 Jul 16 21:44:32.775743 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-97f3edf0-87ac-40c5-ba22-f31d121fb5a2 req-71f0d135-5edc-4fd8-b9bc-525294fdde6d service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.061752 Jul 16 21:44:32.779734 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2847/5698] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:44:32 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 67 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 16 21:44:33.104327 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-40aa0c3d-726a-4e38-8139-8f5d94a360f3 tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:44:33.105448 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-40aa0c3d-726a-4e38-8139-8f5d94a360f3 tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] [instance: 183fd84f-828f-49a3-9ec9-9831dbfcc555] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:44:33.112639 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-40aa0c3d-726a-4e38-8139-8f5d94a360f3 tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:44:33.113112 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-40aa0c3d-726a-4e38-8139-8f5d94a360f3 tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:44:33.113896 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-40aa0c3d-726a-4e38-8139-8f5d94a360f3 tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:44:33.114787 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-40aa0c3d-726a-4e38-8139-8f5d94a360f3 tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:44:33.377739 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-40aa0c3d-726a-4e38-8139-8f5d94a360f3 tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'beae3bc561e141fc9229b40466840078', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '17459e07487944958c9c307e59e1c8dc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:44:33.385619 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-40aa0c3d-726a-4e38-8139-8f5d94a360f3 tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] 10.222.0.16 "GET /compute/v2.1/servers/183fd84f-828f-49a3-9ec9-9831dbfcc555" status: 200 len: 1316 microversion: 2.1 time: 0.285114 Jul 16 21:44:33.386384 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2852/5699] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:44:33 2022] GET /compute/v2.1/servers/183fd84f-828f-49a3-9ec9-9831dbfcc555 => generated 1316 bytes in 287 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:44:34.411181 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-b4268dbe-cdc9-4b24-a922-9dfb5085e52a tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:44:34.411898 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-b4268dbe-cdc9-4b24-a922-9dfb5085e52a tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] [instance: 183fd84f-828f-49a3-9ec9-9831dbfcc555] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:44:34.431191 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b4268dbe-cdc9-4b24-a922-9dfb5085e52a tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:44:34.431673 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b4268dbe-cdc9-4b24-a922-9dfb5085e52a tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:44:34.432498 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b4268dbe-cdc9-4b24-a922-9dfb5085e52a tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:44:34.432949 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b4268dbe-cdc9-4b24-a922-9dfb5085e52a tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:44:34.618421 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-b4268dbe-cdc9-4b24-a922-9dfb5085e52a tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'beae3bc561e141fc9229b40466840078', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '17459e07487944958c9c307e59e1c8dc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:44:34.628611 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-b4268dbe-cdc9-4b24-a922-9dfb5085e52a tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] 10.222.0.16 "GET /compute/v2.1/servers/183fd84f-828f-49a3-9ec9-9831dbfcc555" status: 200 len: 1316 microversion: 2.1 time: 0.226960 Jul 16 21:44:34.629663 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2848/5700] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:44:34 2022] GET /compute/v2.1/servers/183fd84f-828f-49a3-9ec9-9831dbfcc555 => generated 1316 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:44:35.646320 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-502e829c-ce0b-4314-bbd4-53ac36f6d69a tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:44:35.647876 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-502e829c-ce0b-4314-bbd4-53ac36f6d69a tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] [instance: 183fd84f-828f-49a3-9ec9-9831dbfcc555] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:44:35.659553 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-502e829c-ce0b-4314-bbd4-53ac36f6d69a tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:44:35.660272 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-502e829c-ce0b-4314-bbd4-53ac36f6d69a tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:44:35.661276 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-502e829c-ce0b-4314-bbd4-53ac36f6d69a tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:44:35.662930 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-502e829c-ce0b-4314-bbd4-53ac36f6d69a tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:44:35.851917 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-502e829c-ce0b-4314-bbd4-53ac36f6d69a tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'beae3bc561e141fc9229b40466840078', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '17459e07487944958c9c307e59e1c8dc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:44:35.869174 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-502e829c-ce0b-4314-bbd4-53ac36f6d69a tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] 10.222.0.16 "GET /compute/v2.1/servers/183fd84f-828f-49a3-9ec9-9831dbfcc555" status: 200 len: 1316 microversion: 2.1 time: 0.227471 Jul 16 21:44:35.870236 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2853/5701] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:44:35 2022] GET /compute/v2.1/servers/183fd84f-828f-49a3-9ec9-9831dbfcc555 => generated 1316 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:44:36.887688 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-f2d4ed74-9d76-421d-9fb6-f7f115dc42f0 tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:44:36.888322 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-f2d4ed74-9d76-421d-9fb6-f7f115dc42f0 tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] [instance: 183fd84f-828f-49a3-9ec9-9831dbfcc555] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:44:36.895686 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell0:SELECT=2 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:44:36.898115 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f2d4ed74-9d76-421d-9fb6-f7f115dc42f0 tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:44:36.898291 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f2d4ed74-9d76-421d-9fb6-f7f115dc42f0 tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:44:36.899666 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f2d4ed74-9d76-421d-9fb6-f7f115dc42f0 tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:44:36.900310 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f2d4ed74-9d76-421d-9fb6-f7f115dc42f0 tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:44:37.138961 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-f2d4ed74-9d76-421d-9fb6-f7f115dc42f0 tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'beae3bc561e141fc9229b40466840078', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '17459e07487944958c9c307e59e1c8dc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:44:37.151172 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-f2d4ed74-9d76-421d-9fb6-f7f115dc42f0 tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] 10.222.0.16 "GET /compute/v2.1/servers/183fd84f-828f-49a3-9ec9-9831dbfcc555" status: 200 len: 1316 microversion: 2.1 time: 0.267653 Jul 16 21:44:37.153404 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2849/5702] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:44:36 2022] GET /compute/v2.1/servers/183fd84f-828f-49a3-9ec9-9831dbfcc555 => generated 1316 bytes in 271 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:44:37.580314 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-69fb5d44-66b9-4071-8644-c347699f6767 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestNetworkAdvancedServerOps-server-207306916", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452", "networks": [{"uuid": "409a308f-18ed-414a-8af7-cadbf928bc01"}], "key_name": "tempest-TestNetworkAdvancedServerOps-800712517", "security_groups": [{"name": "tempest-secgroup-smoke-458531176"}]}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:44:37.590354 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell0:SELECT=6 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:44:37.729801 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.network.neutron [None req-69fb5d44-66b9-4071-8644-c347699f6767 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] validate_networks() for [('409a308f-18ed-414a-8af7-cadbf928bc01', None, None, None, None, None)] {{(pid=138681) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 16 21:44:38.170108 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-982ec72c-6c41-4dfb-9a38-750297445969 tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:44:38.171355 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-982ec72c-6c41-4dfb-9a38-750297445969 tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] [instance: 183fd84f-828f-49a3-9ec9-9831dbfcc555] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:44:38.187697 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-982ec72c-6c41-4dfb-9a38-750297445969 tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:44:38.192644 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-982ec72c-6c41-4dfb-9a38-750297445969 tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:44:38.192644 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-982ec72c-6c41-4dfb-9a38-750297445969 tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:44:38.192644 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-982ec72c-6c41-4dfb-9a38-750297445969 tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:44:38.305561 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell0:SELECT=8 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:44:38.742148 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-69fb5d44-66b9-4071-8644-c347699f6767 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Flavor limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 16 21:44:38.742648 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-69fb5d44-66b9-4071-8644-c347699f6767 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Image limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 16 21:44:38.743088 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-69fb5d44-66b9-4071-8644-c347699f6767 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Flavor pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 16 21:44:38.743518 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-69fb5d44-66b9-4071-8644-c347699f6767 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Image pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 16 21:44:38.743949 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-69fb5d44-66b9-4071-8644-c347699f6767 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 16 21:44:38.759928 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-69fb5d44-66b9-4071-8644-c347699f6767 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:44:38.760405 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-69fb5d44-66b9-4071-8644-c347699f6767 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:44:38.761229 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-69fb5d44-66b9-4071-8644-c347699f6767 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:44:38.761684 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-69fb5d44-66b9-4071-8644-c347699f6767 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:44:38.805624 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-982ec72c-6c41-4dfb-9a38-750297445969 tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'beae3bc561e141fc9229b40466840078', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '17459e07487944958c9c307e59e1c8dc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:44:38.814572 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-69fb5d44-66b9-4071-8644-c347699f6767 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Getting quotas for project 77c9b7e351e24dc88207e9498f75f578. Resources: {'ram', 'instances', 'cores'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:44:38.819348 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-69fb5d44-66b9-4071-8644-c347699f6767 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Getting quotas for user 833136cbdeab4b9e8b6dd2d24c7c8753 and project 77c9b7e351e24dc88207e9498f75f578. Resources: {'ram', 'instances', 'cores'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:44:38.823541 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-982ec72c-6c41-4dfb-9a38-750297445969 tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] 10.222.0.16 "GET /compute/v2.1/servers/183fd84f-828f-49a3-9ec9-9831dbfcc555" status: 200 len: 1316 microversion: 2.1 time: 0.657403 Jul 16 21:44:38.835066 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2850/5703] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:44:38 2022] GET /compute/v2.1/servers/183fd84f-828f-49a3-9ec9-9831dbfcc555 => generated 1316 bytes in 669 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:44:38.836276 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-69fb5d44-66b9-4071-8644-c347699f6767 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Going to run 1 instances... {{(pid=138681) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 16 21:44:38.838356 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-69fb5d44-66b9-4071-8644-c347699f6767 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] [instance: 0b62eebf-c7ed-4fa1-a4d1-17947e5d8226] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138681) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 16 21:44:39.006448 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-69fb5d44-66b9-4071-8644-c347699f6767 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 405 microversion: 2.1 time: 1.432045 Jul 16 21:44:39.007248 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2854/5704] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 21:44:37 2022] POST /compute/v2.1/servers => generated 405 bytes in 1434 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 16 21:44:39.021930 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-1ed17129-7247-4895-a48e-f7b1f59579f2 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:44:39.022615 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-1ed17129-7247-4895-a48e-f7b1f59579f2 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] [instance: 0b62eebf-c7ed-4fa1-a4d1-17947e5d8226] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:44:39.141736 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-1ed17129-7247-4895-a48e-f7b1f59579f2 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '833136cbdeab4b9e8b6dd2d24c7c8753', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '77c9b7e351e24dc88207e9498f75f578', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:44:39.150948 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-1ed17129-7247-4895-a48e-f7b1f59579f2 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] 10.222.0.16 "GET /compute/v2.1/servers/0b62eebf-c7ed-4fa1-a4d1-17947e5d8226" status: 200 len: 1265 microversion: 2.1 time: 0.133090 Jul 16 21:44:39.153855 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2851/5705] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:44:39 2022] GET /compute/v2.1/servers/0b62eebf-c7ed-4fa1-a4d1-17947e5d8226 => generated 1265 bytes in 135 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:44:39.842771 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-a70ad9d7-ccb1-476e-90fc-8064f9491a99 tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:44:39.843464 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-a70ad9d7-ccb1-476e-90fc-8064f9491a99 tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] [instance: 183fd84f-828f-49a3-9ec9-9831dbfcc555] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:44:39.850559 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a70ad9d7-ccb1-476e-90fc-8064f9491a99 tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:44:39.850704 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a70ad9d7-ccb1-476e-90fc-8064f9491a99 tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:44:39.851436 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a70ad9d7-ccb1-476e-90fc-8064f9491a99 tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:44:39.851890 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a70ad9d7-ccb1-476e-90fc-8064f9491a99 tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:44:40.034793 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-a70ad9d7-ccb1-476e-90fc-8064f9491a99 tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'beae3bc561e141fc9229b40466840078', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '17459e07487944958c9c307e59e1c8dc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:44:40.046654 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-a70ad9d7-ccb1-476e-90fc-8064f9491a99 tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] 10.222.0.16 "GET /compute/v2.1/servers/183fd84f-828f-49a3-9ec9-9831dbfcc555" status: 200 len: 1316 microversion: 2.1 time: 0.208768 Jul 16 21:44:40.060314 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2855/5706] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:44:39 2022] GET /compute/v2.1/servers/183fd84f-828f-49a3-9ec9-9831dbfcc555 => generated 1316 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:44:40.176242 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-a8ebd98d-30bf-4fd3-bd7a-4e3f3ada5b67 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:44:40.177500 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-a8ebd98d-30bf-4fd3-bd7a-4e3f3ada5b67 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] [instance: 0b62eebf-c7ed-4fa1-a4d1-17947e5d8226] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:44:40.185556 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a8ebd98d-30bf-4fd3-bd7a-4e3f3ada5b67 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:44:40.186412 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a8ebd98d-30bf-4fd3-bd7a-4e3f3ada5b67 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:44:40.187406 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a8ebd98d-30bf-4fd3-bd7a-4e3f3ada5b67 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:44:40.188271 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a8ebd98d-30bf-4fd3-bd7a-4e3f3ada5b67 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:44:40.286722 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-a8ebd98d-30bf-4fd3-bd7a-4e3f3ada5b67 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '833136cbdeab4b9e8b6dd2d24c7c8753', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '77c9b7e351e24dc88207e9498f75f578', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:44:40.305508 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-a8ebd98d-30bf-4fd3-bd7a-4e3f3ada5b67 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] 10.222.0.16 "GET /compute/v2.1/servers/0b62eebf-c7ed-4fa1-a4d1-17947e5d8226" status: 200 len: 1269 microversion: 2.1 time: 0.139930 Jul 16 21:44:40.310212 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2852/5707] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:44:40 2022] GET /compute/v2.1/servers/0b62eebf-c7ed-4fa1-a4d1-17947e5d8226 => generated 1269 bytes in 142 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:44:40.736574 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-c6d165cf-4099-4e46-ba9c-4b4d12200548 req-aa3c4298-c90a-427c-8210-927675eb5352 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "183fd84f-828f-49a3-9ec9-9831dbfcc555", "name": "network-vif-plugged", "status": "completed", "tag": "fb93beeb-497d-431a-835c-95c0e5fb3fc3"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:44:40.753395 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-c6d165cf-4099-4e46-ba9c-4b4d12200548 req-aa3c4298-c90a-427c-8210-927675eb5352 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:44:40.753876 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-c6d165cf-4099-4e46-ba9c-4b4d12200548 req-aa3c4298-c90a-427c-8210-927675eb5352 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:44:41.065601 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-1caf05e1-3956-434d-bf7c-e6e01a50daba tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:44:41.066231 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-1caf05e1-3956-434d-bf7c-e6e01a50daba tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] [instance: 183fd84f-828f-49a3-9ec9-9831dbfcc555] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:44:41.076028 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1caf05e1-3956-434d-bf7c-e6e01a50daba tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:44:41.076489 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1caf05e1-3956-434d-bf7c-e6e01a50daba tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:44:41.087312 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1caf05e1-3956-434d-bf7c-e6e01a50daba tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:44:41.087757 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1caf05e1-3956-434d-bf7c-e6e01a50daba tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:44:41.088303 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-c6d165cf-4099-4e46-ba9c-4b4d12200548 req-aa3c4298-c90a-427c-8210-927675eb5352 service nova] Creating event network-vif-plugged:fb93beeb-497d-431a-835c-95c0e5fb3fc3 for instance 183fd84f-828f-49a3-9ec9-9831dbfcc555 on nch2-master-1 Jul 16 21:44:41.100229 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-c6d165cf-4099-4e46-ba9c-4b4d12200548 req-aa3c4298-c90a-427c-8210-927675eb5352 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.367711 Jul 16 21:44:41.101136 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2856/5708] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:44:40 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 370 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:44:41.324013 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-f8850c2e-6097-4ad8-a5d6-9eb9f72ed0fb tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:44:41.324917 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-f8850c2e-6097-4ad8-a5d6-9eb9f72ed0fb tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] [instance: 0b62eebf-c7ed-4fa1-a4d1-17947e5d8226] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:44:41.339644 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f8850c2e-6097-4ad8-a5d6-9eb9f72ed0fb tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:44:41.341183 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f8850c2e-6097-4ad8-a5d6-9eb9f72ed0fb tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:44:41.342300 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f8850c2e-6097-4ad8-a5d6-9eb9f72ed0fb tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:44:41.342977 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f8850c2e-6097-4ad8-a5d6-9eb9f72ed0fb tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:44:41.386737 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-1caf05e1-3956-434d-bf7c-e6e01a50daba tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'beae3bc561e141fc9229b40466840078', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '17459e07487944958c9c307e59e1c8dc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:44:41.400359 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-1caf05e1-3956-434d-bf7c-e6e01a50daba tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] 10.222.0.16 "GET /compute/v2.1/servers/183fd84f-828f-49a3-9ec9-9831dbfcc555" status: 200 len: 1316 microversion: 2.1 time: 0.338090 Jul 16 21:44:41.401441 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2853/5709] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:44:41 2022] GET /compute/v2.1/servers/183fd84f-828f-49a3-9ec9-9831dbfcc555 => generated 1316 bytes in 341 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:44:41.463306 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-f8850c2e-6097-4ad8-a5d6-9eb9f72ed0fb tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '833136cbdeab4b9e8b6dd2d24c7c8753', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '77c9b7e351e24dc88207e9498f75f578', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:44:41.475130 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-f8850c2e-6097-4ad8-a5d6-9eb9f72ed0fb tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] 10.222.0.16 "GET /compute/v2.1/servers/0b62eebf-c7ed-4fa1-a4d1-17947e5d8226" status: 200 len: 1317 microversion: 2.1 time: 0.155381 Jul 16 21:44:41.487762 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2857/5710] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:44:41 2022] GET /compute/v2.1/servers/0b62eebf-c7ed-4fa1-a4d1-17947e5d8226 => generated 1317 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:44:42.420920 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-f95f4c8c-7477-4c09-af9b-c5bf49f1ba4f tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:44:42.421943 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-f95f4c8c-7477-4c09-af9b-c5bf49f1ba4f tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] [instance: 183fd84f-828f-49a3-9ec9-9831dbfcc555] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:44:42.432553 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f95f4c8c-7477-4c09-af9b-c5bf49f1ba4f tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:44:42.433304 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f95f4c8c-7477-4c09-af9b-c5bf49f1ba4f tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:44:42.434321 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f95f4c8c-7477-4c09-af9b-c5bf49f1ba4f tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:44:42.435299 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f95f4c8c-7477-4c09-af9b-c5bf49f1ba4f tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:44:42.497706 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-51dd04d5-d2fb-4aab-8151-759811ac81e3 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:44:42.498330 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-51dd04d5-d2fb-4aab-8151-759811ac81e3 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] [instance: 0b62eebf-c7ed-4fa1-a4d1-17947e5d8226] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:44:42.512032 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-51dd04d5-d2fb-4aab-8151-759811ac81e3 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:44:42.512499 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-51dd04d5-d2fb-4aab-8151-759811ac81e3 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:44:42.513294 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-51dd04d5-d2fb-4aab-8151-759811ac81e3 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:44:42.513744 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-51dd04d5-d2fb-4aab-8151-759811ac81e3 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:44:42.862179 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-f95f4c8c-7477-4c09-af9b-c5bf49f1ba4f tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'beae3bc561e141fc9229b40466840078', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '17459e07487944958c9c307e59e1c8dc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:44:42.873893 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-f95f4c8c-7477-4c09-af9b-c5bf49f1ba4f tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] 10.222.0.16 "GET /compute/v2.1/servers/183fd84f-828f-49a3-9ec9-9831dbfcc555" status: 200 len: 1316 microversion: 2.1 time: 0.459697 Jul 16 21:44:42.879520 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2854/5711] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:44:42 2022] GET /compute/v2.1/servers/183fd84f-828f-49a3-9ec9-9831dbfcc555 => generated 1316 bytes in 466 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:44:42.964198 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-51dd04d5-d2fb-4aab-8151-759811ac81e3 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '833136cbdeab4b9e8b6dd2d24c7c8753', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '77c9b7e351e24dc88207e9498f75f578', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:44:42.975917 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-51dd04d5-d2fb-4aab-8151-759811ac81e3 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] 10.222.0.16 "GET /compute/v2.1/servers/0b62eebf-c7ed-4fa1-a4d1-17947e5d8226" status: 200 len: 1390 microversion: 2.1 time: 0.481942 Jul 16 21:44:42.976498 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2858/5712] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:44:42 2022] GET /compute/v2.1/servers/0b62eebf-c7ed-4fa1-a4d1-17947e5d8226 => generated 1390 bytes in 483 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:44:43.897118 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-2d9b1296-a45b-4d3b-8755-0a526ca57ceb tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:44:43.898344 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-2d9b1296-a45b-4d3b-8755-0a526ca57ceb tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] [instance: 183fd84f-828f-49a3-9ec9-9831dbfcc555] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:44:43.916090 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2d9b1296-a45b-4d3b-8755-0a526ca57ceb tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:44:43.921585 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2d9b1296-a45b-4d3b-8755-0a526ca57ceb tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:44:43.922446 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2d9b1296-a45b-4d3b-8755-0a526ca57ceb tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:44:43.923448 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2d9b1296-a45b-4d3b-8755-0a526ca57ceb tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:44:43.994109 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-3b39cfe4-87ac-4d80-9440-62b1f7579659 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:44:43.995799 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-3b39cfe4-87ac-4d80-9440-62b1f7579659 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] [instance: 0b62eebf-c7ed-4fa1-a4d1-17947e5d8226] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:44:44.013600 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3b39cfe4-87ac-4d80-9440-62b1f7579659 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:44:44.016303 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3b39cfe4-87ac-4d80-9440-62b1f7579659 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:44:44.017456 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3b39cfe4-87ac-4d80-9440-62b1f7579659 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:44:44.018097 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3b39cfe4-87ac-4d80-9440-62b1f7579659 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:44:44.374676 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-2d9b1296-a45b-4d3b-8755-0a526ca57ceb tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'beae3bc561e141fc9229b40466840078', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '17459e07487944958c9c307e59e1c8dc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:44:44.377956 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-3b39cfe4-87ac-4d80-9440-62b1f7579659 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '833136cbdeab4b9e8b6dd2d24c7c8753', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '77c9b7e351e24dc88207e9498f75f578', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:44:44.387044 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-2d9b1296-a45b-4d3b-8755-0a526ca57ceb tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] 10.222.0.16 "GET /compute/v2.1/servers/183fd84f-828f-49a3-9ec9-9831dbfcc555" status: 200 len: 1506 microversion: 2.1 time: 0.494328 Jul 16 21:44:44.387667 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2855/5713] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:44:43 2022] GET /compute/v2.1/servers/183fd84f-828f-49a3-9ec9-9831dbfcc555 => generated 1506 bytes in 496 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:44:44.394938 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-341e7f13-49f2-4afb-b877-9a68fcef2573 req-8e238876-d51a-43c6-8f6d-41dc8237f7cb service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "0b62eebf-c7ed-4fa1-a4d1-17947e5d8226", "tag": "f2720e89-583c-4f0a-bdaf-e22e35af2d14"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:44:44.405413 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-3b39cfe4-87ac-4d80-9440-62b1f7579659 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] 10.222.0.16 "GET /compute/v2.1/servers/0b62eebf-c7ed-4fa1-a4d1-17947e5d8226" status: 200 len: 1390 microversion: 2.1 time: 0.407151 Jul 16 21:44:44.405413 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2859/5714] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:44:43 2022] GET /compute/v2.1/servers/0b62eebf-c7ed-4fa1-a4d1-17947e5d8226 => generated 1390 bytes in 413 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:44:44.406120 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-341e7f13-49f2-4afb-b877-9a68fcef2573 req-8e238876-d51a-43c6-8f6d-41dc8237f7cb service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:44:44.406761 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-341e7f13-49f2-4afb-b877-9a68fcef2573 req-8e238876-d51a-43c6-8f6d-41dc8237f7cb service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:44:44.420785 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-4eefd8d7-f796-4c79-8022-8c39e173c30a tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:44:44.421484 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-4eefd8d7-f796-4c79-8022-8c39e173c30a tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] [instance: 183fd84f-828f-49a3-9ec9-9831dbfcc555] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:44:44.427115 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-341e7f13-49f2-4afb-b877-9a68fcef2573 req-8e238876-d51a-43c6-8f6d-41dc8237f7cb service nova] Creating event network-changed:f2720e89-583c-4f0a-bdaf-e22e35af2d14 for instance 0b62eebf-c7ed-4fa1-a4d1-17947e5d8226 on nch1-master-1 Jul 16 21:44:44.434018 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-341e7f13-49f2-4afb-b877-9a68fcef2573 req-8e238876-d51a-43c6-8f6d-41dc8237f7cb service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.044464 Jul 16 21:44:44.435704 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2856/5715] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:44:44 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:44:44.436785 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4eefd8d7-f796-4c79-8022-8c39e173c30a tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:44:44.437318 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4eefd8d7-f796-4c79-8022-8c39e173c30a tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:44:44.438177 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4eefd8d7-f796-4c79-8022-8c39e173c30a tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:44:44.439230 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4eefd8d7-f796-4c79-8022-8c39e173c30a tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:44:44.476422 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-4eefd8d7-f796-4c79-8022-8c39e173c30a tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] Policy check for os_compute_api:os-instance-actions:events failed with credentials {'is_admin': False, 'user_id': 'beae3bc561e141fc9229b40466840078', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '17459e07487944958c9c307e59e1c8dc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:44:44.478160 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-4eefd8d7-f796-4c79-8022-8c39e173c30a tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] 10.222.0.16 "GET /compute/v2.1/servers/183fd84f-828f-49a3-9ec9-9831dbfcc555/os-instance-actions/req-84f8a87e-cb05-4efe-be6b-cf88d78ffb67" status: 200 len: 313 microversion: 2.1 time: 0.063532 Jul 16 21:44:44.479256 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2860/5716] 10.222.0.16 () {58 vars in 1418 bytes} [Sat Jul 16 21:44:44 2022] GET /compute/v2.1/servers/183fd84f-828f-49a3-9ec9-9831dbfcc555/os-instance-actions/req-84f8a87e-cb05-4efe-be6b-cf88d78ffb67 => generated 313 bytes in 65 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:44:44.492240 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-ceee1fcf-b04d-4213-8114-63be3a8f7444 tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] Action: 'action', calling method: >, body: {"reboot": {"type": "HARD"}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:44:44.493495 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-ceee1fcf-b04d-4213-8114-63be3a8f7444 tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] [instance: 183fd84f-828f-49a3-9ec9-9831dbfcc555] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:44:44.504277 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ceee1fcf-b04d-4213-8114-63be3a8f7444 tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:44:44.505038 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ceee1fcf-b04d-4213-8114-63be3a8f7444 tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:44:44.507106 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ceee1fcf-b04d-4213-8114-63be3a8f7444 tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:44:44.510039 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ceee1fcf-b04d-4213-8114-63be3a8f7444 tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:44:44.841178 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-ceee1fcf-b04d-4213-8114-63be3a8f7444 tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] 10.222.0.16 "POST /compute/v2.1/servers/183fd84f-828f-49a3-9ec9-9831dbfcc555/action" status: 202 len: 0 microversion: 2.1 time: 0.353020 Jul 16 21:44:44.852454 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2857/5717] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 21:44:44 2022] POST /compute/v2.1/servers/183fd84f-828f-49a3-9ec9-9831dbfcc555/action => generated 0 bytes in 356 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 21:44:44.858885 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-9e061403-8b74-4549-8ad1-6bd24a196f4e tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:44:44.859902 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-9e061403-8b74-4549-8ad1-6bd24a196f4e tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] [instance: 183fd84f-828f-49a3-9ec9-9831dbfcc555] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:44:44.873684 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9e061403-8b74-4549-8ad1-6bd24a196f4e tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:44:44.875706 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9e061403-8b74-4549-8ad1-6bd24a196f4e tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:44:44.876693 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9e061403-8b74-4549-8ad1-6bd24a196f4e tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:44:44.877337 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9e061403-8b74-4549-8ad1-6bd24a196f4e tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:44:45.379056 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-9e061403-8b74-4549-8ad1-6bd24a196f4e tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'beae3bc561e141fc9229b40466840078', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '17459e07487944958c9c307e59e1c8dc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:44:45.393789 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-9e061403-8b74-4549-8ad1-6bd24a196f4e tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] 10.222.0.16 "GET /compute/v2.1/servers/183fd84f-828f-49a3-9ec9-9831dbfcc555" status: 200 len: 1508 microversion: 2.1 time: 0.545830 Jul 16 21:44:45.395751 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2861/5718] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:44:44 2022] GET /compute/v2.1/servers/183fd84f-828f-49a3-9ec9-9831dbfcc555 => generated 1508 bytes in 548 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:44:45.425266 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-ec9ec0ce-5886-4d70-a522-ccce4d2a1eed tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:44:45.431175 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-ec9ec0ce-5886-4d70-a522-ccce4d2a1eed tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] [instance: 0b62eebf-c7ed-4fa1-a4d1-17947e5d8226] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:44:45.433996 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ec9ec0ce-5886-4d70-a522-ccce4d2a1eed tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:44:45.434738 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ec9ec0ce-5886-4d70-a522-ccce4d2a1eed tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:44:45.435865 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ec9ec0ce-5886-4d70-a522-ccce4d2a1eed tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:44:45.436545 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ec9ec0ce-5886-4d70-a522-ccce4d2a1eed tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:44:46.138979 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-ec9ec0ce-5886-4d70-a522-ccce4d2a1eed tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '833136cbdeab4b9e8b6dd2d24c7c8753', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '77c9b7e351e24dc88207e9498f75f578', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:44:46.152126 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-ec9ec0ce-5886-4d70-a522-ccce4d2a1eed tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] 10.222.0.16 "GET /compute/v2.1/servers/0b62eebf-c7ed-4fa1-a4d1-17947e5d8226" status: 200 len: 1390 microversion: 2.1 time: 0.731762 Jul 16 21:44:46.162418 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2858/5719] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:44:45 2022] GET /compute/v2.1/servers/0b62eebf-c7ed-4fa1-a4d1-17947e5d8226 => generated 1390 bytes in 734 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:44:46.413238 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-89fd35d3-bfd4-4aea-b2ea-010b2aef959e tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:44:46.413848 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-89fd35d3-bfd4-4aea-b2ea-010b2aef959e tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] [instance: 183fd84f-828f-49a3-9ec9-9831dbfcc555] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:44:46.421401 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-89fd35d3-bfd4-4aea-b2ea-010b2aef959e tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:44:46.424884 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-89fd35d3-bfd4-4aea-b2ea-010b2aef959e tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:44:46.426218 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-89fd35d3-bfd4-4aea-b2ea-010b2aef959e tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:44:46.427710 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-89fd35d3-bfd4-4aea-b2ea-010b2aef959e tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:44:46.680146 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-89fd35d3-bfd4-4aea-b2ea-010b2aef959e tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'beae3bc561e141fc9229b40466840078', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '17459e07487944958c9c307e59e1c8dc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:44:46.689853 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-89fd35d3-bfd4-4aea-b2ea-010b2aef959e tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] 10.222.0.16 "GET /compute/v2.1/servers/183fd84f-828f-49a3-9ec9-9831dbfcc555" status: 200 len: 1508 microversion: 2.1 time: 0.280854 Jul 16 21:44:46.692980 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2862/5720] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:44:46 2022] GET /compute/v2.1/servers/183fd84f-828f-49a3-9ec9-9831dbfcc555 => generated 1508 bytes in 284 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:44:47.159655 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-f6028617-0682-4346-b559-ae17fb37ea7e tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:44:47.161378 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-f6028617-0682-4346-b559-ae17fb37ea7e tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] [instance: a7f9caa9-fff2-44b2-871d-7e4191ff39e5] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:44:47.172503 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-2883c63a-7ea6-4f6c-a6e2-e51e1d27ba20 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:44:47.174109 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-2883c63a-7ea6-4f6c-a6e2-e51e1d27ba20 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] [instance: 0b62eebf-c7ed-4fa1-a4d1-17947e5d8226] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:44:47.178401 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f6028617-0682-4346-b559-ae17fb37ea7e tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:44:47.179553 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f6028617-0682-4346-b559-ae17fb37ea7e tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:44:47.182803 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2883c63a-7ea6-4f6c-a6e2-e51e1d27ba20 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:44:47.183002 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f6028617-0682-4346-b559-ae17fb37ea7e tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:44:47.183601 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f6028617-0682-4346-b559-ae17fb37ea7e tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:44:47.183974 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2883c63a-7ea6-4f6c-a6e2-e51e1d27ba20 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:44:47.185087 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2883c63a-7ea6-4f6c-a6e2-e51e1d27ba20 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:44:47.185756 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2883c63a-7ea6-4f6c-a6e2-e51e1d27ba20 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:44:47.217881 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-f6028617-0682-4346-b559-ae17fb37ea7e tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] [instance: a7f9caa9-fff2-44b2-871d-7e4191ff39e5] Going to try to terminate instance {{(pid=138680) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 16 21:44:47.503851 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-2883c63a-7ea6-4f6c-a6e2-e51e1d27ba20 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '833136cbdeab4b9e8b6dd2d24c7c8753', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '77c9b7e351e24dc88207e9498f75f578', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:44:47.525050 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-2883c63a-7ea6-4f6c-a6e2-e51e1d27ba20 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] 10.222.0.16 "GET /compute/v2.1/servers/0b62eebf-c7ed-4fa1-a4d1-17947e5d8226" status: 200 len: 1390 microversion: 2.1 time: 0.359069 Jul 16 21:44:47.526193 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2863/5721] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:44:47 2022] GET /compute/v2.1/servers/0b62eebf-c7ed-4fa1-a4d1-17947e5d8226 => generated 1390 bytes in 361 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:44:47.633198 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-f6028617-0682-4346-b559-ae17fb37ea7e tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] 10.222.0.16 "DELETE /compute/v2.1/servers/a7f9caa9-fff2-44b2-871d-7e4191ff39e5" status: 204 len: 0 microversion: 2.1 time: 0.477784 Jul 16 21:44:47.634027 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2859/5722] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:44:47 2022] DELETE /compute/v2.1/servers/a7f9caa9-fff2-44b2-871d-7e4191ff39e5 => generated 0 bytes in 480 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 16 21:44:47.644269 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-669225aa-5e6c-42f7-9b77-6ba3112a1e84 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:44:47.645179 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-669225aa-5e6c-42f7-9b77-6ba3112a1e84 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] [instance: a7f9caa9-fff2-44b2-871d-7e4191ff39e5] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:44:47.651586 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-669225aa-5e6c-42f7-9b77-6ba3112a1e84 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:44:47.652054 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-669225aa-5e6c-42f7-9b77-6ba3112a1e84 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:44:47.653073 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-669225aa-5e6c-42f7-9b77-6ba3112a1e84 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:44:47.668508 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-669225aa-5e6c-42f7-9b77-6ba3112a1e84 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:44:47.708365 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-8d6c0a7a-677a-4c0c-aef4-124ce433ced6 tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:44:47.712385 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-8d6c0a7a-677a-4c0c-aef4-124ce433ced6 tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] [instance: 183fd84f-828f-49a3-9ec9-9831dbfcc555] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:44:47.722147 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8d6c0a7a-677a-4c0c-aef4-124ce433ced6 tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:44:47.722858 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8d6c0a7a-677a-4c0c-aef4-124ce433ced6 tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:44:47.724098 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8d6c0a7a-677a-4c0c-aef4-124ce433ced6 tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:44:47.724685 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8d6c0a7a-677a-4c0c-aef4-124ce433ced6 tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:44:48.049406 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-669225aa-5e6c-42f7-9b77-6ba3112a1e84 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0ab8b3f06e994bbca5d6dd14109bfe50', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e60c0863dc6f4313bdc873fda27ae2fc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:44:48.059263 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-669225aa-5e6c-42f7-9b77-6ba3112a1e84 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] 10.222.0.16 "GET /compute/v2.1/servers/a7f9caa9-fff2-44b2-871d-7e4191ff39e5" status: 200 len: 1679 microversion: 2.1 time: 0.420411 Jul 16 21:44:48.059990 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2864/5723] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:44:47 2022] GET /compute/v2.1/servers/a7f9caa9-fff2-44b2-871d-7e4191ff39e5 => generated 1679 bytes in 422 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:44:48.074212 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-8d6c0a7a-677a-4c0c-aef4-124ce433ced6 tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'beae3bc561e141fc9229b40466840078', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '17459e07487944958c9c307e59e1c8dc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:44:48.084537 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-8d6c0a7a-677a-4c0c-aef4-124ce433ced6 tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] 10.222.0.16 "GET /compute/v2.1/servers/183fd84f-828f-49a3-9ec9-9831dbfcc555" status: 200 len: 1506 microversion: 2.1 time: 0.381137 Jul 16 21:44:48.085442 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2860/5724] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:44:47 2022] GET /compute/v2.1/servers/183fd84f-828f-49a3-9ec9-9831dbfcc555 => generated 1506 bytes in 383 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:44:48.105125 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-54544da0-51e9-4847-9909-43ddc84d14f9 tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] Calling method '.version_select at 0x7fdd9c238f70>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:44:48.106306 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-54544da0-51e9-4847-9909-43ddc84d14f9 tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] [instance: 183fd84f-828f-49a3-9ec9-9831dbfcc555] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:44:48.114150 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-54544da0-51e9-4847-9909-43ddc84d14f9 tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:44:48.114981 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-54544da0-51e9-4847-9909-43ddc84d14f9 tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:44:48.116016 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-54544da0-51e9-4847-9909-43ddc84d14f9 tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:44:48.116640 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-54544da0-51e9-4847-9909-43ddc84d14f9 tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:44:48.171138 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-54544da0-51e9-4847-9909-43ddc84d14f9 tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] 10.222.0.16 "GET /compute/v2.1/servers/183fd84f-828f-49a3-9ec9-9831dbfcc555/os-instance-actions" status: 200 len: 611 microversion: 2.1 time: 0.073949 Jul 16 21:44:48.172483 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2865/5725] 10.222.0.16 () {58 vars in 1295 bytes} [Sat Jul 16 21:44:48 2022] GET /compute/v2.1/servers/183fd84f-828f-49a3-9ec9-9831dbfcc555/os-instance-actions => generated 611 bytes in 76 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:44:48.185895 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-55c8a35e-509b-4360-851d-7d938e462ec6 tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:44:48.187147 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-55c8a35e-509b-4360-851d-7d938e462ec6 tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] [instance: 183fd84f-828f-49a3-9ec9-9831dbfcc555] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:44:48.200830 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-55c8a35e-509b-4360-851d-7d938e462ec6 tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:44:48.201522 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-55c8a35e-509b-4360-851d-7d938e462ec6 tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:44:48.204703 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-55c8a35e-509b-4360-851d-7d938e462ec6 tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:44:48.210178 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-55c8a35e-509b-4360-851d-7d938e462ec6 tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:44:48.262843 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-55c8a35e-509b-4360-851d-7d938e462ec6 tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] [instance: 183fd84f-828f-49a3-9ec9-9831dbfcc555] Going to try to terminate instance {{(pid=138680) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 16 21:44:48.307352 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=96,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:44:48.543729 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-e03ecf50-1856-421a-a62a-f049f50093aa tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:44:48.545134 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-e03ecf50-1856-421a-a62a-f049f50093aa tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] [instance: 0b62eebf-c7ed-4fa1-a4d1-17947e5d8226] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:44:48.555103 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e03ecf50-1856-421a-a62a-f049f50093aa tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:44:48.559155 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e03ecf50-1856-421a-a62a-f049f50093aa tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:44:48.560385 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e03ecf50-1856-421a-a62a-f049f50093aa tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:44:48.561136 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e03ecf50-1856-421a-a62a-f049f50093aa tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:44:48.661624 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-55c8a35e-509b-4360-851d-7d938e462ec6 tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] 10.222.0.16 "DELETE /compute/v2.1/servers/183fd84f-828f-49a3-9ec9-9831dbfcc555" status: 204 len: 0 microversion: 2.1 time: 0.479313 Jul 16 21:44:48.664479 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2861/5726] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:44:48 2022] DELETE /compute/v2.1/servers/183fd84f-828f-49a3-9ec9-9831dbfcc555 => generated 0 bytes in 483 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 16 21:44:48.674696 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-b2400d6d-e627-4126-aed8-b2b762b7e398 tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:44:48.675459 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-b2400d6d-e627-4126-aed8-b2b762b7e398 tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] [instance: 183fd84f-828f-49a3-9ec9-9831dbfcc555] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:44:48.681783 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b2400d6d-e627-4126-aed8-b2b762b7e398 tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:44:48.682244 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b2400d6d-e627-4126-aed8-b2b762b7e398 tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:44:48.683846 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b2400d6d-e627-4126-aed8-b2b762b7e398 tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:44:48.684313 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b2400d6d-e627-4126-aed8-b2b762b7e398 tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:44:48.742868 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-e03ecf50-1856-421a-a62a-f049f50093aa tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '833136cbdeab4b9e8b6dd2d24c7c8753', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '77c9b7e351e24dc88207e9498f75f578', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:44:48.759445 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-e03ecf50-1856-421a-a62a-f049f50093aa tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] 10.222.0.16 "GET /compute/v2.1/servers/0b62eebf-c7ed-4fa1-a4d1-17947e5d8226" status: 200 len: 1390 microversion: 2.1 time: 0.219631 Jul 16 21:44:48.763383 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2866/5727] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:44:48 2022] GET /compute/v2.1/servers/0b62eebf-c7ed-4fa1-a4d1-17947e5d8226 => generated 1390 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:44:48.892226 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-b2400d6d-e627-4126-aed8-b2b762b7e398 tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'beae3bc561e141fc9229b40466840078', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '17459e07487944958c9c307e59e1c8dc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:44:48.901695 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-b2400d6d-e627-4126-aed8-b2b762b7e398 tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] 10.222.0.16 "GET /compute/v2.1/servers/183fd84f-828f-49a3-9ec9-9831dbfcc555" status: 200 len: 1512 microversion: 2.1 time: 0.232016 Jul 16 21:44:48.903049 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2862/5728] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:44:48 2022] GET /compute/v2.1/servers/183fd84f-828f-49a3-9ec9-9831dbfcc555 => generated 1512 bytes in 234 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:44:49.080761 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-27b0bfe0-77ae-4d88-a082-185b955470fe tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:44:49.081803 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-27b0bfe0-77ae-4d88-a082-185b955470fe tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] [instance: a7f9caa9-fff2-44b2-871d-7e4191ff39e5] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:44:49.090320 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell0:SELECT=2 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:44:49.091448 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell0:SELECT=6 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:44:49.093581 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-27b0bfe0-77ae-4d88-a082-185b955470fe tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:44:49.094306 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-27b0bfe0-77ae-4d88-a082-185b955470fe tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:44:49.095688 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-27b0bfe0-77ae-4d88-a082-185b955470fe tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:44:49.096249 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-27b0bfe0-77ae-4d88-a082-185b955470fe tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:44:49.171466 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-27b0bfe0-77ae-4d88-a082-185b955470fe tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0ab8b3f06e994bbca5d6dd14109bfe50', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e60c0863dc6f4313bdc873fda27ae2fc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:44:49.176326 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=100 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:44:49.182306 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-27b0bfe0-77ae-4d88-a082-185b955470fe tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] 10.222.0.16 "GET /compute/v2.1/servers/a7f9caa9-fff2-44b2-871d-7e4191ff39e5" status: 200 len: 1614 microversion: 2.1 time: 0.106909 Jul 16 21:44:49.182878 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2867/5729] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:44:49 2022] GET /compute/v2.1/servers/a7f9caa9-fff2-44b2-871d-7e4191ff39e5 => generated 1614 bytes in 108 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:44:49.362865 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-f0b5bef2-193e-4b52-8990-45ad676825b0 req-ac0402a3-03a5-46ed-87ca-71ea03fdd536 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "a7f9caa9-fff2-44b2-871d-7e4191ff39e5", "name": "network-vif-deleted", "tag": "8e33e390-df49-4cb3-a043-fa2763fb499b"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:44:49.368593 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell0:SELECT=2 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:44:49.375622 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-f0b5bef2-193e-4b52-8990-45ad676825b0 req-ac0402a3-03a5-46ed-87ca-71ea03fdd536 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:44:49.377173 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-f0b5bef2-193e-4b52-8990-45ad676825b0 req-ac0402a3-03a5-46ed-87ca-71ea03fdd536 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:44:49.398743 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-f0b5bef2-193e-4b52-8990-45ad676825b0 req-ac0402a3-03a5-46ed-87ca-71ea03fdd536 service nova] Creating event network-vif-deleted:8e33e390-df49-4cb3-a043-fa2763fb499b for instance a7f9caa9-fff2-44b2-871d-7e4191ff39e5 on nch1-master-1 Jul 16 21:44:49.400932 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-f0b5bef2-193e-4b52-8990-45ad676825b0 req-ac0402a3-03a5-46ed-87ca-71ea03fdd536 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.042788 Jul 16 21:44:49.401838 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2863/5730] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:44:49 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 16 21:44:49.787756 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-381de8d7-23a3-428c-a4c7-fe76f9be60e9 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:44:49.788404 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-381de8d7-23a3-428c-a4c7-fe76f9be60e9 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] [instance: 0b62eebf-c7ed-4fa1-a4d1-17947e5d8226] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:44:49.796965 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-381de8d7-23a3-428c-a4c7-fe76f9be60e9 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:44:49.800782 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-381de8d7-23a3-428c-a4c7-fe76f9be60e9 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:44:49.800782 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-381de8d7-23a3-428c-a4c7-fe76f9be60e9 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:44:49.800782 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-381de8d7-23a3-428c-a4c7-fe76f9be60e9 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:44:49.922416 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-6733c999-b7cb-493f-a6c8-bab5649d7df7 tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:44:49.923964 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-6733c999-b7cb-493f-a6c8-bab5649d7df7 tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] [instance: 183fd84f-828f-49a3-9ec9-9831dbfcc555] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:44:49.933689 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6733c999-b7cb-493f-a6c8-bab5649d7df7 tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:44:49.934473 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6733c999-b7cb-493f-a6c8-bab5649d7df7 tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:44:49.935819 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6733c999-b7cb-493f-a6c8-bab5649d7df7 tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:44:49.944526 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6733c999-b7cb-493f-a6c8-bab5649d7df7 tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:44:50.067174 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-381de8d7-23a3-428c-a4c7-fe76f9be60e9 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '833136cbdeab4b9e8b6dd2d24c7c8753', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '77c9b7e351e24dc88207e9498f75f578', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:44:50.079178 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-381de8d7-23a3-428c-a4c7-fe76f9be60e9 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] 10.222.0.16 "GET /compute/v2.1/servers/0b62eebf-c7ed-4fa1-a4d1-17947e5d8226" status: 200 len: 1390 microversion: 2.1 time: 0.294015 Jul 16 21:44:50.079178 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2868/5731] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:44:49 2022] GET /compute/v2.1/servers/0b62eebf-c7ed-4fa1-a4d1-17947e5d8226 => generated 1390 bytes in 297 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:44:50.201816 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-f103b7c7-9102-4803-bd06-bff09ef276e2 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:44:50.202470 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-f103b7c7-9102-4803-bd06-bff09ef276e2 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] [instance: a7f9caa9-fff2-44b2-871d-7e4191ff39e5] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:44:50.216399 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f103b7c7-9102-4803-bd06-bff09ef276e2 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:44:50.217033 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f103b7c7-9102-4803-bd06-bff09ef276e2 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:44:50.224983 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f103b7c7-9102-4803-bd06-bff09ef276e2 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:44:50.228403 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f103b7c7-9102-4803-bd06-bff09ef276e2 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:44:50.266660 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-6733c999-b7cb-493f-a6c8-bab5649d7df7 tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'beae3bc561e141fc9229b40466840078', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '17459e07487944958c9c307e59e1c8dc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:44:50.277511 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-6733c999-b7cb-493f-a6c8-bab5649d7df7 tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] 10.222.0.16 "GET /compute/v2.1/servers/183fd84f-828f-49a3-9ec9-9831dbfcc555" status: 200 len: 1512 microversion: 2.1 time: 0.358399 Jul 16 21:44:50.278156 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2864/5732] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:44:49 2022] GET /compute/v2.1/servers/183fd84f-828f-49a3-9ec9-9831dbfcc555 => generated 1512 bytes in 363 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:44:50.319134 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f103b7c7-9102-4803-bd06-bff09ef276e2 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:44:50.319914 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f103b7c7-9102-4803-bd06-bff09ef276e2 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:44:50.320840 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.objects.instance [None req-f103b7c7-9102-4803-bd06-bff09ef276e2 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lazy-loading 'fault' on Instance uuid a7f9caa9-fff2-44b2-871d-7e4191ff39e5 {{(pid=138681) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:44:50.419204 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-f103b7c7-9102-4803-bd06-bff09ef276e2 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0ab8b3f06e994bbca5d6dd14109bfe50', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e60c0863dc6f4313bdc873fda27ae2fc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:44:50.432135 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-f103b7c7-9102-4803-bd06-bff09ef276e2 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] 10.222.0.16 "GET /compute/v2.1/servers/a7f9caa9-fff2-44b2-871d-7e4191ff39e5" status: 200 len: 1420 microversion: 2.1 time: 0.233101 Jul 16 21:44:50.434354 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2869/5733] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:44:50 2022] GET /compute/v2.1/servers/a7f9caa9-fff2-44b2-871d-7e4191ff39e5 => generated 1420 bytes in 238 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:44:50.733606 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-e47dc88c-1a1e-49e1-91ca-818d23a1021a req-83afc513-35c5-4860-924f-db0075cacae0 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "0b62eebf-c7ed-4fa1-a4d1-17947e5d8226", "name": "network-vif-plugged", "status": "completed", "tag": "f2720e89-583c-4f0a-bdaf-e22e35af2d14"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:44:50.744323 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-e47dc88c-1a1e-49e1-91ca-818d23a1021a req-83afc513-35c5-4860-924f-db0075cacae0 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:44:50.744766 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-e47dc88c-1a1e-49e1-91ca-818d23a1021a req-83afc513-35c5-4860-924f-db0075cacae0 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:44:50.762575 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-e47dc88c-1a1e-49e1-91ca-818d23a1021a req-83afc513-35c5-4860-924f-db0075cacae0 service nova] Creating event network-vif-plugged:f2720e89-583c-4f0a-bdaf-e22e35af2d14 for instance 0b62eebf-c7ed-4fa1-a4d1-17947e5d8226 on nch1-master-1 Jul 16 21:44:50.777473 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-e47dc88c-1a1e-49e1-91ca-818d23a1021a req-83afc513-35c5-4860-924f-db0075cacae0 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.047985 Jul 16 21:44:50.778322 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2865/5734] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:44:50 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 16 21:44:51.094001 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-7fbe3900-6597-4d49-bfc4-36429c1d4eec tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:44:51.096138 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-7fbe3900-6597-4d49-bfc4-36429c1d4eec tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] [instance: 0b62eebf-c7ed-4fa1-a4d1-17947e5d8226] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:44:51.106220 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-1d78bc7c-6800-4a75-9649-35934517c782 req-f95f277c-0c58-419c-b2e7-db32f93d585a service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "183fd84f-828f-49a3-9ec9-9831dbfcc555", "name": "network-vif-deleted", "tag": "fb93beeb-497d-431a-835c-95c0e5fb3fc3"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:44:51.109282 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7fbe3900-6597-4d49-bfc4-36429c1d4eec tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:44:51.109773 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7fbe3900-6597-4d49-bfc4-36429c1d4eec tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:44:51.117706 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7fbe3900-6597-4d49-bfc4-36429c1d4eec tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:44:51.118612 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7fbe3900-6597-4d49-bfc4-36429c1d4eec tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:44:51.119645 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-1d78bc7c-6800-4a75-9649-35934517c782 req-f95f277c-0c58-419c-b2e7-db32f93d585a service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:44:51.120120 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-1d78bc7c-6800-4a75-9649-35934517c782 req-f95f277c-0c58-419c-b2e7-db32f93d585a service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:44:51.139457 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-1d78bc7c-6800-4a75-9649-35934517c782 req-f95f277c-0c58-419c-b2e7-db32f93d585a service nova] Creating event network-vif-deleted:fb93beeb-497d-431a-835c-95c0e5fb3fc3 for instance 183fd84f-828f-49a3-9ec9-9831dbfcc555 on nch2-master-1 Jul 16 21:44:51.146803 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-1d78bc7c-6800-4a75-9649-35934517c782 req-f95f277c-0c58-419c-b2e7-db32f93d585a service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.045520 Jul 16 21:44:51.151113 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2866/5735] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:44:51 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 16 21:44:51.298534 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-6256c97f-25cb-40ff-a3b3-72d08bb4e513 tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:44:51.299640 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-6256c97f-25cb-40ff-a3b3-72d08bb4e513 tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] [instance: 183fd84f-828f-49a3-9ec9-9831dbfcc555] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:44:51.308350 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6256c97f-25cb-40ff-a3b3-72d08bb4e513 tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:44:51.309076 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6256c97f-25cb-40ff-a3b3-72d08bb4e513 tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:44:51.310239 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6256c97f-25cb-40ff-a3b3-72d08bb4e513 tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:44:51.310937 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6256c97f-25cb-40ff-a3b3-72d08bb4e513 tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:44:51.352930 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-7fbe3900-6597-4d49-bfc4-36429c1d4eec tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '833136cbdeab4b9e8b6dd2d24c7c8753', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '77c9b7e351e24dc88207e9498f75f578', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:44:51.380535 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-7fbe3900-6597-4d49-bfc4-36429c1d4eec tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] 10.222.0.16 "GET /compute/v2.1/servers/0b62eebf-c7ed-4fa1-a4d1-17947e5d8226" status: 200 len: 1390 microversion: 2.1 time: 0.288624 Jul 16 21:44:51.382987 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2870/5736] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:44:51 2022] GET /compute/v2.1/servers/0b62eebf-c7ed-4fa1-a4d1-17947e5d8226 => generated 1390 bytes in 294 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:44:51.444480 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-6256c97f-25cb-40ff-a3b3-72d08bb4e513 tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'beae3bc561e141fc9229b40466840078', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '17459e07487944958c9c307e59e1c8dc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:44:51.453555 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-6256c97f-25cb-40ff-a3b3-72d08bb4e513 tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] 10.222.0.16 "GET /compute/v2.1/servers/183fd84f-828f-49a3-9ec9-9831dbfcc555" status: 200 len: 1301 microversion: 2.1 time: 0.159840 Jul 16 21:44:51.454266 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2867/5737] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:44:51 2022] GET /compute/v2.1/servers/183fd84f-828f-49a3-9ec9-9831dbfcc555 => generated 1301 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:44:51.468248 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-04216edc-d066-4814-8d0e-3974a462248e tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:44:51.469030 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-04216edc-d066-4814-8d0e-3974a462248e tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] [instance: a7f9caa9-fff2-44b2-871d-7e4191ff39e5] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:44:51.475501 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-04216edc-d066-4814-8d0e-3974a462248e tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:44:51.476046 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-04216edc-d066-4814-8d0e-3974a462248e tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:44:51.486553 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-04216edc-d066-4814-8d0e-3974a462248e tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:44:51.487644 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-04216edc-d066-4814-8d0e-3974a462248e tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:44:51.506681 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-04216edc-d066-4814-8d0e-3974a462248e tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] HTTP exception thrown: Instance a7f9caa9-fff2-44b2-871d-7e4191ff39e5 could not be found. Jul 16 21:44:51.508692 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-04216edc-d066-4814-8d0e-3974a462248e tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Request method failure captured: Jul 16 21:44:51.508692 ncdv-master-1 devstack@n-api.service[138681]: request: GET /compute/v2.1/servers/a7f9caa9-fff2-44b2-871d-7e4191ff39e5 HTTP/1.1 Jul 16 21:44:51.508692 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:44:51.508692 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:44:51.508692 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:44:51.508692 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 21:44:51.508692 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:44:51.508692 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:44:51.508692 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:44:51.508692 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:44:51.508692 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0y8gcGxfa3qumeDpAV_UzHtBqxWw5fgWdXEsN_P0XTe84Os21rKjc9QIlpjQBg7EBAnrc9gi4HTfF6VkzKBMP6W-EwHOC7W6W2mOvP55miTafGS3QI8-v38KjXcvgCGFOUcX_9nbDaYEoaZ_SfO91myDjzRdtv09P0hU8CwskGtYm8E Jul 16 21:44:51.508692 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:44:51.508692 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:44:51.508692 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:44:51.508692 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:44:51.508692 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:44:51.508692 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:44:51.508692 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: e60c0863dc6f4313bdc873fda27ae2fc Jul 16 21:44:51.508692 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-AttachVolumeNegativeTest-17004004 Jul 16 21:44:51.508692 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:44:51.508692 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:44:51.508692 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/e60c0863dc6f4313bdc873fda27ae2fc"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/e60c0863dc6f4313bdc873fda27ae2fc"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_e60c0863dc6f4313bdc873fda27ae2fc"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/e60c0863dc6f4313bdc873fda27ae2fc"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:44:51.508692 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-AttachVolumeNegativeTest-17004004 Jul 16 21:44:51.508692 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: e60c0863dc6f4313bdc873fda27ae2fc Jul 16 21:44:51.508692 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-17004004 Jul 16 21:44:51.508692 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-AttachVolumeNegativeTest-17004004-project Jul 16 21:44:51.508692 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:44:51.508692 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:44:51.508692 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: 0ab8b3f06e994bbca5d6dd14109bfe50 Jul 16 21:44:51.508692 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-AttachVolumeNegativeTest-17004004-project Jul 16 21:44:51.508692 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:44:51.508692 ncdv-master-1 devstack@n-api.service[138681]: exception: Instance a7f9caa9-fff2-44b2-871d-7e4191ff39e5 could not be found. Jul 16 21:44:51.508692 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': 'a7f9caa9-fff2-44b2-871d-7e4191ff39e5'} Jul 16 21:44:51.508692 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:44:51.508692 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:44:51.508692 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:44:51.508692 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:44:51.508692 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:44:51.508692 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:44:51.508692 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:44:51.508692 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:44:51.508692 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:44:51.508692 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:44:51.508692 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance a7f9caa9-fff2-44b2-871d-7e4191ff39e5 could not be found. Jul 16 21:44:51.508692 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:44:51.508692 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:44:51.508692 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:44:51.508692 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:44:51.508692 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:44:51.508692 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:44:51.508692 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:44:51.508692 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:44:51.508692 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:44:51.508692 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:44:51.508692 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:44:51.508692 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:44:51.508692 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:44:51.508692 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:44:51.508692 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:44:51.508692 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:44:51.508692 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance a7f9caa9-fff2-44b2-871d-7e4191ff39e5 could not be found. Jul 16 21:44:51.508692 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:44:51.508692 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:44:51.508692 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:44:51.514867 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:44:51.514867 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:44:51.514867 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:44:51.514867 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:44:51.514867 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:44:51.514867 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance a7f9caa9-fff2-44b2-871d-7e4191ff39e5 could not be found. Jul 16 21:44:51.514867 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:44:51.514867 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-04216edc-d066-4814-8d0e-3974a462248e tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Returning 404 to user: Instance a7f9caa9-fff2-44b2-871d-7e4191ff39e5 could not be found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:44:51.514867 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-04216edc-d066-4814-8d0e-3974a462248e tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] 10.222.0.16 "GET /compute/v2.1/servers/a7f9caa9-fff2-44b2-871d-7e4191ff39e5" status: 404 len: 111 microversion: 2.1 time: 0.046329 Jul 16 21:44:51.516224 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2871/5738] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:44:51 2022] GET /compute/v2.1/servers/a7f9caa9-fff2-44b2-871d-7e4191ff39e5 => generated 111 bytes in 53 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:44:51.521663 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-6bacb9f2-be33-4b24-85ad-2687b2425acf tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Calling method '.version_select at 0x7fdd8fc37430>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:44:51.651251 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-6bacb9f2-be33-4b24-85ad-2687b2425acf tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] 10.222.0.16 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1042899521" status: 202 len: 0 microversion: 2.1 time: 0.133187 Jul 16 21:44:51.653306 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2868/5739] 10.222.0.16 () {58 vars in 1220 bytes} [Sat Jul 16 21:44:51 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1042899521 => generated 0 bytes in 135 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 21:44:52.398855 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-53e01828-2a68-448a-a00d-1f8bf648cb47 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:44:52.399960 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-53e01828-2a68-448a-a00d-1f8bf648cb47 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] [instance: 0b62eebf-c7ed-4fa1-a4d1-17947e5d8226] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:44:52.417108 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-53e01828-2a68-448a-a00d-1f8bf648cb47 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:44:52.417634 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-53e01828-2a68-448a-a00d-1f8bf648cb47 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:44:52.418453 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-53e01828-2a68-448a-a00d-1f8bf648cb47 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:44:52.419153 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-53e01828-2a68-448a-a00d-1f8bf648cb47 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:44:52.472522 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-57a404da-3870-4855-b3ee-44d307092a77 tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:44:52.473873 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-57a404da-3870-4855-b3ee-44d307092a77 tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] [instance: 183fd84f-828f-49a3-9ec9-9831dbfcc555] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:44:52.484206 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-57a404da-3870-4855-b3ee-44d307092a77 tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:44:52.484774 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-57a404da-3870-4855-b3ee-44d307092a77 tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:44:52.486378 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-57a404da-3870-4855-b3ee-44d307092a77 tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:44:52.486830 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-57a404da-3870-4855-b3ee-44d307092a77 tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:44:52.542067 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-57a404da-3870-4855-b3ee-44d307092a77 tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:44:52.543158 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-57a404da-3870-4855-b3ee-44d307092a77 tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:44:52.543784 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.objects.instance [None req-57a404da-3870-4855-b3ee-44d307092a77 tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] Lazy-loading 'fault' on Instance uuid 183fd84f-828f-49a3-9ec9-9831dbfcc555 {{(pid=138680) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:44:52.649207 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-57a404da-3870-4855-b3ee-44d307092a77 tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'beae3bc561e141fc9229b40466840078', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '17459e07487944958c9c307e59e1c8dc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:44:52.661857 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-57a404da-3870-4855-b3ee-44d307092a77 tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] 10.222.0.16 "GET /compute/v2.1/servers/183fd84f-828f-49a3-9ec9-9831dbfcc555" status: 200 len: 1306 microversion: 2.1 time: 0.194340 Jul 16 21:44:52.663286 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2869/5740] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:44:52 2022] GET /compute/v2.1/servers/183fd84f-828f-49a3-9ec9-9831dbfcc555 => generated 1306 bytes in 198 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:44:52.707254 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-53e01828-2a68-448a-a00d-1f8bf648cb47 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '833136cbdeab4b9e8b6dd2d24c7c8753', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '77c9b7e351e24dc88207e9498f75f578', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:44:52.734147 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-53e01828-2a68-448a-a00d-1f8bf648cb47 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] 10.222.0.16 "GET /compute/v2.1/servers/0b62eebf-c7ed-4fa1-a4d1-17947e5d8226" status: 200 len: 1562 microversion: 2.1 time: 0.339150 Jul 16 21:44:52.736503 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2872/5741] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:44:52 2022] GET /compute/v2.1/servers/0b62eebf-c7ed-4fa1-a4d1-17947e5d8226 => generated 1562 bytes in 342 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:44:52.760401 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-80ec460f-b052-4950-ae86-314c8a742c16 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:44:52.761771 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-80ec460f-b052-4950-ae86-314c8a742c16 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] [instance: 0b62eebf-c7ed-4fa1-a4d1-17947e5d8226] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:44:52.779222 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-80ec460f-b052-4950-ae86-314c8a742c16 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:44:52.779222 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-80ec460f-b052-4950-ae86-314c8a742c16 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:44:52.781052 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-80ec460f-b052-4950-ae86-314c8a742c16 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:44:52.781751 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-80ec460f-b052-4950-ae86-314c8a742c16 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:44:52.983389 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-80ec460f-b052-4950-ae86-314c8a742c16 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '833136cbdeab4b9e8b6dd2d24c7c8753', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '77c9b7e351e24dc88207e9498f75f578', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:44:52.991892 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-80ec460f-b052-4950-ae86-314c8a742c16 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] 10.222.0.16 "GET /compute/v2.1/servers/0b62eebf-c7ed-4fa1-a4d1-17947e5d8226" status: 200 len: 1562 microversion: 2.1 time: 0.237685 Jul 16 21:44:52.992471 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2870/5742] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:44:52 2022] GET /compute/v2.1/servers/0b62eebf-c7ed-4fa1-a4d1-17947e5d8226 => generated 1562 bytes in 239 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:44:53.280138 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-a65b90f0-2d44-451c-aae7-54f7e365bcb4 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Action: 'create', calling method: .version_select at 0x7fdd8c5ecdc0>, body: {"keypair": {"name": "tempest-keypair-2120625668", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBM9uo45XigEoR9CGf1CaUVaLGiNAlC2yy4C04rHW821l+iNxEBkhmnaTCXM1XebDpoTggEZ5vXac2yNzz/RcRyiKftSYS84qCWzlmv8GgwoaAWT4m7aY/3NY0skolq5uRg=="}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:44:53.306107 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-a65b90f0-2d44-451c-aae7-54f7e365bcb4 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Getting quotas for project e60c0863dc6f4313bdc873fda27ae2fc. Resources: {'key_pairs'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:44:53.315050 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-a65b90f0-2d44-451c-aae7-54f7e365bcb4 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Getting quotas for user 0ab8b3f06e994bbca5d6dd14109bfe50 and project e60c0863dc6f4313bdc873fda27ae2fc. Resources: {'key_pairs'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:44:53.476149 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-a65b90f0-2d44-451c-aae7-54f7e365bcb4 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] 10.222.0.16 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.205139 Jul 16 21:44:53.476149 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2873/5743] 10.222.0.16 () {60 vars in 1158 bytes} [Sat Jul 16 21:44:53 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 209 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:44:53.685116 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-250f4850-b682-458b-b545-a8cf72432f73 tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:44:53.685739 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-250f4850-b682-458b-b545-a8cf72432f73 tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] [instance: 183fd84f-828f-49a3-9ec9-9831dbfcc555] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:44:53.694904 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-250f4850-b682-458b-b545-a8cf72432f73 tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:44:53.695393 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-250f4850-b682-458b-b545-a8cf72432f73 tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:44:53.700264 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-250f4850-b682-458b-b545-a8cf72432f73 tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:44:53.707477 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-250f4850-b682-458b-b545-a8cf72432f73 tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:44:53.737070 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-250f4850-b682-458b-b545-a8cf72432f73 tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] HTTP exception thrown: Instance 183fd84f-828f-49a3-9ec9-9831dbfcc555 could not be found. Jul 16 21:44:53.740326 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-250f4850-b682-458b-b545-a8cf72432f73 tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] Request method failure captured: Jul 16 21:44:53.740326 ncdv-master-1 devstack@n-api.service[138680]: request: GET /compute/v2.1/servers/183fd84f-828f-49a3-9ec9-9831dbfcc555 HTTP/1.1 Jul 16 21:44:53.740326 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:44:53.740326 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:44:53.740326 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:44:53.740326 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 0 Jul 16 21:44:53.740326 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:44:53.740326 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:44:53.740326 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:44:53.740326 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:44:53.740326 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0zE5bw1AAejNulLfM1DJs-FGlpzHCxd88xCvRPY-QCbk21GigSSUZ0NuxbrD3m6LH1j--6BeHKszW8wdRSQHkaYyJ1nd83O-E7QeiEV6xvSCGm-qjH-HNzQQ6FQej0TExqzByUNV_gsEfXB7N2HQrP-1uGg8OuoQa6CLC0b2aMnV8Lo Jul 16 21:44:53.740326 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:44:53.740326 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:44:53.740326 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:44:53.740326 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:44:53.740326 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:44:53.740326 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:44:53.740326 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: 17459e07487944958c9c307e59e1c8dc Jul 16 21:44:53.740326 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-InstanceActionsTestJSON-1060560037 Jul 16 21:44:53.740326 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:44:53.740326 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:44:53.740326 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/17459e07487944958c9c307e59e1c8dc"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/17459e07487944958c9c307e59e1c8dc"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_17459e07487944958c9c307e59e1c8dc"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/17459e07487944958c9c307e59e1c8dc"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:44:53.740326 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-InstanceActionsTestJSON-1060560037 Jul 16 21:44:53.740326 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: 17459e07487944958c9c307e59e1c8dc Jul 16 21:44:53.740326 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-InstanceActionsTestJSON-1060560037 Jul 16 21:44:53.740326 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-InstanceActionsTestJSON-1060560037-project Jul 16 21:44:53.740326 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:44:53.740326 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:44:53.740326 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: beae3bc561e141fc9229b40466840078 Jul 16 21:44:53.740326 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-InstanceActionsTestJSON-1060560037-project Jul 16 21:44:53.740326 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 21:44:53.740326 ncdv-master-1 devstack@n-api.service[138680]: exception: Instance 183fd84f-828f-49a3-9ec9-9831dbfcc555 could not be found. Jul 16 21:44:53.740326 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': '183fd84f-828f-49a3-9ec9-9831dbfcc555'} Jul 16 21:44:53.740326 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:44:53.740326 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:44:53.740326 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:44:53.740326 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:44:53.740326 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:44:53.740326 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:44:53.740326 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:44:53.740326 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:44:53.740326 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:44:53.740326 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:44:53.740326 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 183fd84f-828f-49a3-9ec9-9831dbfcc555 could not be found. Jul 16 21:44:53.740326 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:44:53.740326 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:44:53.740326 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:44:53.740326 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:44:53.740326 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:44:53.740326 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:44:53.740326 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:44:53.740326 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:44:53.740326 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:44:53.740326 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:44:53.740326 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:44:53.740326 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:44:53.740326 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:44:53.740326 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:44:53.740326 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:44:53.740326 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:44:53.740326 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 183fd84f-828f-49a3-9ec9-9831dbfcc555 could not be found. Jul 16 21:44:53.740326 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:44:53.740326 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:44:53.740326 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:44:53.760208 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:44:53.760208 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:44:53.760208 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:44:53.760208 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:44:53.760208 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:44:53.760208 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 183fd84f-828f-49a3-9ec9-9831dbfcc555 could not be found. Jul 16 21:44:53.760208 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:44:53.760208 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-250f4850-b682-458b-b545-a8cf72432f73 tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] Returning 404 to user: Instance 183fd84f-828f-49a3-9ec9-9831dbfcc555 could not be found. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:44:53.760208 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-250f4850-b682-458b-b545-a8cf72432f73 tempest-InstanceActionsTestJSON-1060560037 tempest-InstanceActionsTestJSON-1060560037-project] 10.222.0.16 "GET /compute/v2.1/servers/183fd84f-828f-49a3-9ec9-9831dbfcc555" status: 404 len: 111 microversion: 2.1 time: 0.061191 Jul 16 21:44:53.760208 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2871/5744] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:44:53 2022] GET /compute/v2.1/servers/183fd84f-828f-49a3-9ec9-9831dbfcc555 => generated 111 bytes in 63 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:44:56.561133 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-e8b2c207-9069-42ad-9ef0-6cc8c116930a tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeNegativeTest-server-328583061", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "4775de54-1ba2-4d76-a6e0-a5f8a9ef879c"}], "security_groups": [{"name": "tempest-securitygroup--535601306"}], "key_name": "tempest-keypair-2120625668", "user_data": "}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:44:56.700226 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.network.neutron [None req-e8b2c207-9069-42ad-9ef0-6cc8c116930a tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] validate_networks() for [('4775de54-1ba2-4d76-a6e0-a5f8a9ef879c', None, None, None, None, None)] {{(pid=138681) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 16 21:44:57.255981 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-e8b2c207-9069-42ad-9ef0-6cc8c116930a tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Flavor limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 16 21:44:57.256413 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-e8b2c207-9069-42ad-9ef0-6cc8c116930a tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Image limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 16 21:44:57.262441 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-e8b2c207-9069-42ad-9ef0-6cc8c116930a tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Flavor pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 16 21:44:57.263285 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-e8b2c207-9069-42ad-9ef0-6cc8c116930a tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Image pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 16 21:44:57.263725 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-e8b2c207-9069-42ad-9ef0-6cc8c116930a tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 16 21:44:57.281408 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e8b2c207-9069-42ad-9ef0-6cc8c116930a tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:44:57.284024 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e8b2c207-9069-42ad-9ef0-6cc8c116930a tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:44:57.285071 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e8b2c207-9069-42ad-9ef0-6cc8c116930a tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:44:57.285717 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e8b2c207-9069-42ad-9ef0-6cc8c116930a tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:44:57.315215 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-e8b2c207-9069-42ad-9ef0-6cc8c116930a tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Getting quotas for project e60c0863dc6f4313bdc873fda27ae2fc. Resources: {'ram', 'instances', 'cores'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:44:57.320428 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_api:SELECT=95,nova_api:INSERT=5 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:44:57.322157 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-e8b2c207-9069-42ad-9ef0-6cc8c116930a tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Getting quotas for user 0ab8b3f06e994bbca5d6dd14109bfe50 and project e60c0863dc6f4313bdc873fda27ae2fc. Resources: {'ram', 'instances', 'cores'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:44:57.341996 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-e8b2c207-9069-42ad-9ef0-6cc8c116930a tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Going to run 1 instances... {{(pid=138681) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 16 21:44:57.346447 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-e8b2c207-9069-42ad-9ef0-6cc8c116930a tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] [instance: 770fd99a-ef16-4922-847d-bcadc19b90f6] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138681) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 16 21:44:57.522605 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-e8b2c207-9069-42ad-9ef0-6cc8c116930a tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 401 microversion: 2.1 time: 0.966170 Jul 16 21:44:57.523610 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2874/5745] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 21:44:56 2022] POST /compute/v2.1/servers => generated 401 bytes in 968 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Jul 16 21:44:57.535671 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-721c4982-6713-48cb-9694-74900fbfda0b tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:44:57.537469 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-721c4982-6713-48cb-9694-74900fbfda0b tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] [instance: 770fd99a-ef16-4922-847d-bcadc19b90f6] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:44:57.546118 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_api:SELECT=87,nova_api:DELETE=2,nova_api:INSERT=3,nova_api:UPDATE=2 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:44:57.617432 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-721c4982-6713-48cb-9694-74900fbfda0b tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0ab8b3f06e994bbca5d6dd14109bfe50', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e60c0863dc6f4313bdc873fda27ae2fc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:44:57.628290 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-721c4982-6713-48cb-9694-74900fbfda0b tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] 10.222.0.16 "GET /compute/v2.1/servers/770fd99a-ef16-4922-847d-bcadc19b90f6" status: 200 len: 1241 microversion: 2.1 time: 0.098363 Jul 16 21:44:57.629484 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2872/5746] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:44:57 2022] GET /compute/v2.1/servers/770fd99a-ef16-4922-847d-bcadc19b90f6 => generated 1241 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:44:58.648132 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-0c77040c-f4c8-4e04-8442-036f88a3b24a tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:44:58.649515 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-0c77040c-f4c8-4e04-8442-036f88a3b24a tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] [instance: 770fd99a-ef16-4922-847d-bcadc19b90f6] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:44:58.681117 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0c77040c-f4c8-4e04-8442-036f88a3b24a tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:44:58.682881 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0c77040c-f4c8-4e04-8442-036f88a3b24a tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:44:58.684101 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0c77040c-f4c8-4e04-8442-036f88a3b24a tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:44:58.685980 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0c77040c-f4c8-4e04-8442-036f88a3b24a tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:44:58.785106 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-b11a292b-e9d8-4aa5-8dc1-d1273c059f04 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:44:58.786479 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-b11a292b-e9d8-4aa5-8dc1-d1273c059f04 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] [instance: 0b62eebf-c7ed-4fa1-a4d1-17947e5d8226] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:44:58.817944 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b11a292b-e9d8-4aa5-8dc1-d1273c059f04 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:44:58.822587 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b11a292b-e9d8-4aa5-8dc1-d1273c059f04 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:44:58.827119 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b11a292b-e9d8-4aa5-8dc1-d1273c059f04 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:44:58.830602 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b11a292b-e9d8-4aa5-8dc1-d1273c059f04 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:44:59.034668 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-0c77040c-f4c8-4e04-8442-036f88a3b24a tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0ab8b3f06e994bbca5d6dd14109bfe50', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e60c0863dc6f4313bdc873fda27ae2fc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:44:59.045422 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-0c77040c-f4c8-4e04-8442-036f88a3b24a tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] 10.222.0.16 "GET /compute/v2.1/servers/770fd99a-ef16-4922-847d-bcadc19b90f6" status: 200 len: 1245 microversion: 2.1 time: 0.401667 Jul 16 21:44:59.057870 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2875/5747] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:44:58 2022] GET /compute/v2.1/servers/770fd99a-ef16-4922-847d-bcadc19b90f6 => generated 1245 bytes in 408 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:44:59.061814 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-886afaa4-992f-40fb-ae8d-504716a45c7e req-ce9c76f3-b7ec-4524-afee-389f6942d271 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "0b62eebf-c7ed-4fa1-a4d1-17947e5d8226", "tag": "f2720e89-583c-4f0a-bdaf-e22e35af2d14"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:44:59.080743 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-886afaa4-992f-40fb-ae8d-504716a45c7e req-ce9c76f3-b7ec-4524-afee-389f6942d271 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:44:59.081628 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-886afaa4-992f-40fb-ae8d-504716a45c7e req-ce9c76f3-b7ec-4524-afee-389f6942d271 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:44:59.095302 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-886afaa4-992f-40fb-ae8d-504716a45c7e req-ce9c76f3-b7ec-4524-afee-389f6942d271 service nova] Creating event network-changed:f2720e89-583c-4f0a-bdaf-e22e35af2d14 for instance 0b62eebf-c7ed-4fa1-a4d1-17947e5d8226 on nch1-master-1 Jul 16 21:44:59.103457 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-886afaa4-992f-40fb-ae8d-504716a45c7e req-ce9c76f3-b7ec-4524-afee-389f6942d271 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.051738 Jul 16 21:44:59.114226 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2876/5748] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:44:59 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 64 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:44:59.416283 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-b11a292b-e9d8-4aa5-8dc1-d1273c059f04 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '833136cbdeab4b9e8b6dd2d24c7c8753', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '77c9b7e351e24dc88207e9498f75f578', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:44:59.450758 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-b11a292b-e9d8-4aa5-8dc1-d1273c059f04 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] 10.222.0.16 "GET /compute/v2.1/servers/0b62eebf-c7ed-4fa1-a4d1-17947e5d8226" status: 200 len: 1562 microversion: 2.1 time: 0.676557 Jul 16 21:44:59.468509 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2873/5749] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:44:58 2022] GET /compute/v2.1/servers/0b62eebf-c7ed-4fa1-a4d1-17947e5d8226 => generated 1562 bytes in 695 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:45:00.067476 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-0e83ede4-d9c6-48f2-b558-d648a038b351 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:45:00.069449 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-0e83ede4-d9c6-48f2-b558-d648a038b351 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] [instance: 770fd99a-ef16-4922-847d-bcadc19b90f6] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:45:00.078259 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0e83ede4-d9c6-48f2-b558-d648a038b351 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:00.079075 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0e83ede4-d9c6-48f2-b558-d648a038b351 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:00.081388 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0e83ede4-d9c6-48f2-b558-d648a038b351 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:00.081632 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0e83ede4-d9c6-48f2-b558-d648a038b351 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:00.211048 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-0e83ede4-d9c6-48f2-b558-d648a038b351 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0ab8b3f06e994bbca5d6dd14109bfe50', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e60c0863dc6f4313bdc873fda27ae2fc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:45:00.242722 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-0e83ede4-d9c6-48f2-b558-d648a038b351 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] 10.222.0.16 "GET /compute/v2.1/servers/770fd99a-ef16-4922-847d-bcadc19b90f6" status: 200 len: 1293 microversion: 2.1 time: 0.180701 Jul 16 21:45:00.264969 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2877/5750] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:45:00 2022] GET /compute/v2.1/servers/770fd99a-ef16-4922-847d-bcadc19b90f6 => generated 1293 bytes in 204 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:45:01.060489 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-6f9b42cb-6d47-492a-b13b-a8749fb09659 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:45:01.061357 ncdv-master-1 devstack@n-api.service[138680]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-availability-zone:detail": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:45:01.061357 ncdv-master-1 devstack@n-api.service[138680]: warnings.warn(msg) Jul 16 21:45:01.062502 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6f9b42cb-6d47-492a-b13b-a8749fb09659 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:01.063523 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6f9b42cb-6d47-492a-b13b-a8749fb09659 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:01.064379 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6f9b42cb-6d47-492a-b13b-a8749fb09659 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:01.064885 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6f9b42cb-6d47-492a-b13b-a8749fb09659 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:01.091262 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-6f9b42cb-6d47-492a-b13b-a8749fb09659 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] 10.222.0.16 "GET /compute/v2.1/os-availability-zone/detail" status: 200 len: 613 microversion: 2.1 time: 0.197500 Jul 16 21:45:01.091868 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2874/5751] 10.222.0.16 () {58 vars in 1184 bytes} [Sat Jul 16 21:45:00 2022] GET /compute/v2.1/os-availability-zone/detail => generated 613 bytes in 199 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:45:01.261559 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-4d6690e7-24bf-4f9d-897f-a4397a79b387 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:45:01.262882 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-4d6690e7-24bf-4f9d-897f-a4397a79b387 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] [instance: 770fd99a-ef16-4922-847d-bcadc19b90f6] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:45:01.287785 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-e37bac5f-9fc0-4fcf-8b2e-e587ce7455f4 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestServerMultinode-server-834811864", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452", "availability_zone": "nova:nch2-master-1", "networks": [{"uuid": "a8fefdbc-7112-4617-926a-b233743507a1"}]}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:45:01.291738 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4d6690e7-24bf-4f9d-897f-a4397a79b387 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:01.292209 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4d6690e7-24bf-4f9d-897f-a4397a79b387 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:01.296375 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4d6690e7-24bf-4f9d-897f-a4397a79b387 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:01.311578 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4d6690e7-24bf-4f9d-897f-a4397a79b387 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:01.393556 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.network.neutron [None req-e37bac5f-9fc0-4fcf-8b2e-e587ce7455f4 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] validate_networks() for [('a8fefdbc-7112-4617-926a-b233743507a1', None, None, None, None, None)] {{(pid=138680) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 16 21:45:01.537619 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-4d6690e7-24bf-4f9d-897f-a4397a79b387 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0ab8b3f06e994bbca5d6dd14109bfe50', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e60c0863dc6f4313bdc873fda27ae2fc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:45:01.570072 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-4d6690e7-24bf-4f9d-897f-a4397a79b387 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] 10.222.0.16 "GET /compute/v2.1/servers/770fd99a-ef16-4922-847d-bcadc19b90f6" status: 200 len: 1299 microversion: 2.1 time: 0.300863 Jul 16 21:45:01.570072 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2878/5752] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:45:01 2022] GET /compute/v2.1/servers/770fd99a-ef16-4922-847d-bcadc19b90f6 => generated 1299 bytes in 310 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:45:02.265009 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-e37bac5f-9fc0-4fcf-8b2e-e587ce7455f4 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Flavor limits 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 16 21:45:02.265530 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-e37bac5f-9fc0-4fcf-8b2e-e587ce7455f4 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Image limits 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 16 21:45:02.265918 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-e37bac5f-9fc0-4fcf-8b2e-e587ce7455f4 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Flavor pref 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 16 21:45:02.266511 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-e37bac5f-9fc0-4fcf-8b2e-e587ce7455f4 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Image pref 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 16 21:45:02.267204 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-e37bac5f-9fc0-4fcf-8b2e-e587ce7455f4 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 16 21:45:02.277493 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e37bac5f-9fc0-4fcf-8b2e-e587ce7455f4 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:02.278271 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e37bac5f-9fc0-4fcf-8b2e-e587ce7455f4 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:02.279374 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e37bac5f-9fc0-4fcf-8b2e-e587ce7455f4 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:02.280058 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e37bac5f-9fc0-4fcf-8b2e-e587ce7455f4 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:02.305509 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-e37bac5f-9fc0-4fcf-8b2e-e587ce7455f4 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Getting quotas for project b351f1a2d6434cd0b3b03628a0befe14. Resources: {'ram', 'instances', 'cores'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:45:02.312014 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-e37bac5f-9fc0-4fcf-8b2e-e587ce7455f4 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Getting quotas for user e32ea3fe570b40b59110b70fbb0c3caf and project b351f1a2d6434cd0b3b03628a0befe14. Resources: {'ram', 'instances', 'cores'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:45:02.326759 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-e37bac5f-9fc0-4fcf-8b2e-e587ce7455f4 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Going to run 1 instances... {{(pid=138680) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 16 21:45:02.330116 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-e37bac5f-9fc0-4fcf-8b2e-e587ce7455f4 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] [instance: 0793bca6-2775-4777-a4e3-1fa6a2e45eac] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138680) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 16 21:45:02.577475 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-32abf820-1d9f-4eec-be5a-935f89a5da4f tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:45:02.578279 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-32abf820-1d9f-4eec-be5a-935f89a5da4f tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] [instance: 770fd99a-ef16-4922-847d-bcadc19b90f6] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:45:02.598340 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-32abf820-1d9f-4eec-be5a-935f89a5da4f tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:02.598878 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-32abf820-1d9f-4eec-be5a-935f89a5da4f tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:02.602211 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-32abf820-1d9f-4eec-be5a-935f89a5da4f tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:02.602733 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-32abf820-1d9f-4eec-be5a-935f89a5da4f tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:02.632305 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-e37bac5f-9fc0-4fcf-8b2e-e587ce7455f4 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 1.349470 Jul 16 21:45:02.637732 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2875/5753] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 21:45:01 2022] POST /compute/v2.1/servers => generated 380 bytes in 1351 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 16 21:45:02.656055 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-75235122-327a-4979-8b61-58ac9c9a8e0e tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:45:02.656848 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-75235122-327a-4979-8b61-58ac9c9a8e0e tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] [instance: 0793bca6-2775-4777-a4e3-1fa6a2e45eac] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:45:02.954426 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-75235122-327a-4979-8b61-58ac9c9a8e0e tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/0793bca6-2775-4777-a4e3-1fa6a2e45eac" status: 200 len: 1328 microversion: 2.1 time: 0.304778 Jul 16 21:45:02.956412 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2876/5754] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:45:02 2022] GET /compute/v2.1/servers/0793bca6-2775-4777-a4e3-1fa6a2e45eac => generated 1328 bytes in 313 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:45:03.078912 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-32abf820-1d9f-4eec-be5a-935f89a5da4f tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0ab8b3f06e994bbca5d6dd14109bfe50', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e60c0863dc6f4313bdc873fda27ae2fc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:45:03.101911 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-32abf820-1d9f-4eec-be5a-935f89a5da4f tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] 10.222.0.16 "GET /compute/v2.1/servers/770fd99a-ef16-4922-847d-bcadc19b90f6" status: 200 len: 1366 microversion: 2.1 time: 0.516583 Jul 16 21:45:03.101911 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2879/5755] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:45:02 2022] GET /compute/v2.1/servers/770fd99a-ef16-4922-847d-bcadc19b90f6 => generated 1366 bytes in 521 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:45:03.974028 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-99289f67-6cf5-4293-ba73-76b623132631 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:45:03.974901 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-99289f67-6cf5-4293-ba73-76b623132631 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] [instance: 0793bca6-2775-4777-a4e3-1fa6a2e45eac] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:45:04.048646 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-99289f67-6cf5-4293-ba73-76b623132631 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/0793bca6-2775-4777-a4e3-1fa6a2e45eac" status: 200 len: 1328 microversion: 2.1 time: 0.079504 Jul 16 21:45:04.053061 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2877/5756] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:45:03 2022] GET /compute/v2.1/servers/0793bca6-2775-4777-a4e3-1fa6a2e45eac => generated 1328 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:45:04.119815 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-2fda7fe9-b717-4ebc-97f2-c471ac523893 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:45:04.120479 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-2fda7fe9-b717-4ebc-97f2-c471ac523893 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] [instance: 770fd99a-ef16-4922-847d-bcadc19b90f6] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:45:04.128908 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2fda7fe9-b717-4ebc-97f2-c471ac523893 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:04.129385 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2fda7fe9-b717-4ebc-97f2-c471ac523893 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:04.130790 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2fda7fe9-b717-4ebc-97f2-c471ac523893 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:04.130790 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2fda7fe9-b717-4ebc-97f2-c471ac523893 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:04.401598 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-2fda7fe9-b717-4ebc-97f2-c471ac523893 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0ab8b3f06e994bbca5d6dd14109bfe50', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e60c0863dc6f4313bdc873fda27ae2fc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:45:04.444916 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-2fda7fe9-b717-4ebc-97f2-c471ac523893 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] 10.222.0.16 "GET /compute/v2.1/servers/770fd99a-ef16-4922-847d-bcadc19b90f6" status: 200 len: 1366 microversion: 2.1 time: 0.329288 Jul 16 21:45:04.446073 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2880/5757] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:45:04 2022] GET /compute/v2.1/servers/770fd99a-ef16-4922-847d-bcadc19b90f6 => generated 1366 bytes in 333 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:45:04.610137 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-088819de-7040-4ddc-ae67-35708ba39871 req-7d4b2103-102d-4c6f-b648-e469d05b3f8e service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "770fd99a-ef16-4922-847d-bcadc19b90f6", "tag": "152e5772-bc79-4772-820e-b77b68f320f3"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:45:04.619368 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-088819de-7040-4ddc-ae67-35708ba39871 req-7d4b2103-102d-4c6f-b648-e469d05b3f8e service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:04.619849 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-088819de-7040-4ddc-ae67-35708ba39871 req-7d4b2103-102d-4c6f-b648-e469d05b3f8e service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:04.636934 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-088819de-7040-4ddc-ae67-35708ba39871 req-7d4b2103-102d-4c6f-b648-e469d05b3f8e service nova] Creating event network-changed:152e5772-bc79-4772-820e-b77b68f320f3 for instance 770fd99a-ef16-4922-847d-bcadc19b90f6 on nch2-master-1 Jul 16 21:45:04.647028 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-088819de-7040-4ddc-ae67-35708ba39871 req-7d4b2103-102d-4c6f-b648-e469d05b3f8e service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.385058 Jul 16 21:45:04.647888 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2878/5758] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:45:04 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 388 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 16 21:45:05.065437 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-5110e1ca-b4ea-4263-860e-366044fb1657 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:45:05.066047 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-5110e1ca-b4ea-4263-860e-366044fb1657 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] [instance: 0793bca6-2775-4777-a4e3-1fa6a2e45eac] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:45:05.072556 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5110e1ca-b4ea-4263-860e-366044fb1657 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:05.073022 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5110e1ca-b4ea-4263-860e-366044fb1657 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:05.073809 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5110e1ca-b4ea-4263-860e-366044fb1657 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:05.074270 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5110e1ca-b4ea-4263-860e-366044fb1657 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:05.152532 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-5110e1ca-b4ea-4263-860e-366044fb1657 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/0793bca6-2775-4777-a4e3-1fa6a2e45eac" status: 200 len: 1337 microversion: 2.1 time: 0.090242 Jul 16 21:45:05.152532 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2881/5759] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:45:05 2022] GET /compute/v2.1/servers/0793bca6-2775-4777-a4e3-1fa6a2e45eac => generated 1337 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:45:05.463437 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-39e7a368-bbe7-48ee-beaa-917fdfc534c8 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:45:05.464149 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-39e7a368-bbe7-48ee-beaa-917fdfc534c8 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] [instance: 770fd99a-ef16-4922-847d-bcadc19b90f6] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:45:05.471861 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-39e7a368-bbe7-48ee-beaa-917fdfc534c8 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:05.472667 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-39e7a368-bbe7-48ee-beaa-917fdfc534c8 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:05.473306 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-39e7a368-bbe7-48ee-beaa-917fdfc534c8 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:05.473748 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-39e7a368-bbe7-48ee-beaa-917fdfc534c8 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:05.626793 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-39e7a368-bbe7-48ee-beaa-917fdfc534c8 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0ab8b3f06e994bbca5d6dd14109bfe50', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e60c0863dc6f4313bdc873fda27ae2fc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:45:05.635183 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-39e7a368-bbe7-48ee-beaa-917fdfc534c8 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] 10.222.0.16 "GET /compute/v2.1/servers/770fd99a-ef16-4922-847d-bcadc19b90f6" status: 200 len: 1366 microversion: 2.1 time: 0.177065 Jul 16 21:45:05.636129 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2879/5760] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:45:05 2022] GET /compute/v2.1/servers/770fd99a-ef16-4922-847d-bcadc19b90f6 => generated 1366 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:45:05.640136 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-90ed2705-633f-4cbb-bc28-cb90e845f26f tempest-MultipleCreateNegativeTestJSON-456382249 tempest-MultipleCreateNegativeTestJSON-456382249-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-MultipleCreateNegativeTestJSON-server-168757948", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452", "min_count": 3, "max_count": 2}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:45:05.642680 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-90ed2705-633f-4cbb-bc28-cb90e845f26f tempest-MultipleCreateNegativeTestJSON-456382249 tempest-MultipleCreateNegativeTestJSON-456382249-project] HTTP exception thrown: min_count must be <= max_count Jul 16 21:45:05.644266 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-90ed2705-633f-4cbb-bc28-cb90e845f26f tempest-MultipleCreateNegativeTestJSON-456382249 tempest-MultipleCreateNegativeTestJSON-456382249-project] Request method failure captured: Jul 16 21:45:05.644266 ncdv-master-1 devstack@n-api.service[138681]: request: POST /compute/v2.1/servers HTTP/1.1 Jul 16 21:45:05.644266 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:45:05.644266 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:45:05.644266 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:45:05.644266 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 183 Jul 16 21:45:05.644266 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:45:05.644266 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:45:05.644266 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:45:05.644266 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:45:05.644266 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0zFhsGwcXRW3LE25gSnlTSa-5RFYaq8jUEqNhXq-uZVnT-ZDjrxxf_uUcsjFBQU0ed7IC8XSX0G9EnN4MNxjimFSlNjmAUCKvM-DxLhOdGSvFXS8Srh0Ldt2zcsi6l1vTIaJBNv5v1SLIgjA3RVK5SRPMbo3vnN9RCOU19bz7-SnKoA Jul 16 21:45:05.644266 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:45:05.644266 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:45:05.644266 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:45:05.644266 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:45:05.644266 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:45:05.644266 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:45:05.644266 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: fd2229fe58ec4362967b0bdda9101cef Jul 16 21:45:05.644266 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-MultipleCreateNegativeTestJSON-456382249 Jul 16 21:45:05.644266 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:45:05.644266 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:45:05.644266 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/fd2229fe58ec4362967b0bdda9101cef"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/fd2229fe58ec4362967b0bdda9101cef"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_fd2229fe58ec4362967b0bdda9101cef"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/fd2229fe58ec4362967b0bdda9101cef"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:45:05.644266 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-MultipleCreateNegativeTestJSON-456382249 Jul 16 21:45:05.644266 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: fd2229fe58ec4362967b0bdda9101cef Jul 16 21:45:05.644266 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-MultipleCreateNegativeTestJSON-456382249 Jul 16 21:45:05.644266 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-MultipleCreateNegativeTestJSON-456382249-project Jul 16 21:45:05.644266 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:45:05.644266 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:45:05.644266 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: 62b825dcfc864969b2cae9308c975e6d Jul 16 21:45:05.644266 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-MultipleCreateNegativeTestJSON-456382249-project Jul 16 21:45:05.644266 ncdv-master-1 devstack@n-api.service[138681]: Jul 16 21:45:05.644266 ncdv-master-1 devstack@n-api.service[138681]: {"server": {"name": "tempest-MultipleCreateNegativeTestJSON-server-168757948", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452", "min_count": 3, "max_count": 2}} Jul 16 21:45:05.644266 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:45:05.644266 ncdv-master-1 devstack@n-api.service[138681]: exception: min_count must be <= max_count Jul 16 21:45:05.644266 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'body': {'server': {'name': 'tempest-MultipleCreateNegativeTestJSON-server-168757948', 'imageRef': '1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9', 'flavorRef': '452', 'min_count': 3, 'max_count': 2}}} Jul 16 21:45:05.644266 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:45:05.644266 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:45:05.644266 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:45:05.644266 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:45:05.644266 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:45:05.644266 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:45:05.644266 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:45:05.644266 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:45:05.644266 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 16 21:45:05.644266 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 16 21:45:05.644266 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 16 21:45:05.644266 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 16 21:45:05.644266 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 16 21:45:05.644266 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 16 21:45:05.644266 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi [Previous line repeated 10 more times] Jul 16 21:45:05.644266 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 726, in create Jul 16 21:45:05.644266 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=msg) Jul 16 21:45:05.644266 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: min_count must be <= max_count Jul 16 21:45:05.644266 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:45:05.644266 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:45:05.644266 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:45:05.644266 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:45:05.644266 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:45:05.644266 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:45:05.644266 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:45:05.644266 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:45:05.644266 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: min_count must be <= max_count Jul 16 21:45:05.644266 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:45:05.656393 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-90ed2705-633f-4cbb-bc28-cb90e845f26f tempest-MultipleCreateNegativeTestJSON-456382249 tempest-MultipleCreateNegativeTestJSON-456382249-project] Returning 400 to user: min_count must be <= max_count {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:45:05.656393 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-90ed2705-633f-4cbb-bc28-cb90e845f26f tempest-MultipleCreateNegativeTestJSON-456382249 tempest-MultipleCreateNegativeTestJSON-456382249-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 400 len: 74 microversion: 2.1 time: 0.134046 Jul 16 21:45:05.656393 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2882/5761] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 21:45:05 2022] POST /compute/v2.1/servers => generated 74 bytes in 135 msecs (HTTP/1.1 400) 9 headers in 380 bytes (2 switches on core 0) Jul 16 21:45:05.701067 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-2cfbd9e1-ad8a-4432-9721-66617d5f47dd tempest-MultipleCreateNegativeTestJSON-456382249 tempest-MultipleCreateNegativeTestJSON-456382249-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-MultipleCreateNegativeTestJSON-server-337988007", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452", "max_count": 0}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:45:05.704582 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-2cfbd9e1-ad8a-4432-9721-66617d5f47dd tempest-MultipleCreateNegativeTestJSON-456382249 tempest-MultipleCreateNegativeTestJSON-456382249-project] Request method failure captured: Jul 16 21:45:05.704582 ncdv-master-1 devstack@n-api.service[138680]: request: POST /compute/v2.1/servers HTTP/1.1 Jul 16 21:45:05.704582 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:45:05.704582 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:45:05.704582 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:45:05.704582 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 167 Jul 16 21:45:05.704582 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:45:05.704582 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:45:05.704582 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:45:05.704582 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:45:05.704582 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0zFhsGwcXRW3LE25gSnlTSa-5RFYaq8jUEqNhXq-uZVnT-ZDjrxxf_uUcsjFBQU0ed7IC8XSX0G9EnN4MNxjimFSlNjmAUCKvM-DxLhOdGSvFXS8Srh0Ldt2zcsi6l1vTIaJBNv5v1SLIgjA3RVK5SRPMbo3vnN9RCOU19bz7-SnKoA Jul 16 21:45:05.704582 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:45:05.704582 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:45:05.704582 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:45:05.704582 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:45:05.704582 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:45:05.704582 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:45:05.704582 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: fd2229fe58ec4362967b0bdda9101cef Jul 16 21:45:05.704582 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-MultipleCreateNegativeTestJSON-456382249 Jul 16 21:45:05.704582 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:45:05.704582 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:45:05.704582 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/fd2229fe58ec4362967b0bdda9101cef"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/fd2229fe58ec4362967b0bdda9101cef"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_fd2229fe58ec4362967b0bdda9101cef"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/fd2229fe58ec4362967b0bdda9101cef"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:45:05.704582 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-MultipleCreateNegativeTestJSON-456382249 Jul 16 21:45:05.704582 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: fd2229fe58ec4362967b0bdda9101cef Jul 16 21:45:05.704582 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-MultipleCreateNegativeTestJSON-456382249 Jul 16 21:45:05.704582 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-MultipleCreateNegativeTestJSON-456382249-project Jul 16 21:45:05.704582 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:45:05.704582 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:45:05.704582 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: 62b825dcfc864969b2cae9308c975e6d Jul 16 21:45:05.704582 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-MultipleCreateNegativeTestJSON-456382249-project Jul 16 21:45:05.704582 ncdv-master-1 devstack@n-api.service[138680]: Jul 16 21:45:05.704582 ncdv-master-1 devstack@n-api.service[138680]: {"server": {"name": "tempest-MultipleCreateNegativeTestJSON-server-337988007", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452", "max_count": 0}} Jul 16 21:45:05.704582 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 21:45:05.704582 ncdv-master-1 devstack@n-api.service[138680]: exception: Invalid input for field/attribute max_count. Value: 0. 0 is less than the minimum of 1 Jul 16 21:45:05.704582 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'body': {'server': {'name': 'tempest-MultipleCreateNegativeTestJSON-server-337988007', 'imageRef': '1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9', 'flavorRef': '452', 'max_count': 0}}} Jul 16 21:45:05.704582 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:45:05.704582 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:45:05.704582 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 279, in validate Jul 16 21:45:05.704582 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Jul 16 21:45:05.704582 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/lib/python3/dist-packages/jsonschema/validators.py", line 353, in validate Jul 16 21:45:05.704582 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise error Jul 16 21:45:05.704582 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: 0 is less than the minimum of 1 Jul 16 21:45:05.704582 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:45:05.704582 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Failed validating 'minimum' in schema['properties']['server']['properties']['max_count']: Jul 16 21:45:05.704582 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi {'minLength': 1, Jul 16 21:45:05.704582 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi 'minimum': 1, Jul 16 21:45:05.704582 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi 'pattern': '^[0-9]*$', Jul 16 21:45:05.704582 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi 'type': ['integer', 'string']} Jul 16 21:45:05.704582 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:45:05.704582 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi On instance['server']: Jul 16 21:45:05.704582 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi 0 Jul 16 21:45:05.704582 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:45:05.704582 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:45:05.704582 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:45:05.704582 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:45:05.704582 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:45:05.704582 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:45:05.704582 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:45:05.704582 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:45:05.704582 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:45:05.704582 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:45:05.704582 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 16 21:45:05.704582 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 16 21:45:05.704582 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Jul 16 21:45:05.704582 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Jul 16 21:45:05.704582 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Jul 16 21:45:05.718116 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Jul 16 21:45:05.718116 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 308, in validate Jul 16 21:45:05.718116 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Jul 16 21:45:05.718116 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute max_count. Value: 0. 0 is less than the minimum of 1 Jul 16 21:45:05.718116 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:45:05.718116 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:45:05.718116 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:45:05.718116 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:45:05.718116 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:45:05.718116 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:45:05.718116 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Jul 16 21:45:05.718116 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Jul 16 21:45:05.718116 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute max_count. Value: 0. 0 is less than the minimum of 1 Jul 16 21:45:05.718116 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:45:05.718116 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-2cfbd9e1-ad8a-4432-9721-66617d5f47dd tempest-MultipleCreateNegativeTestJSON-456382249 tempest-MultipleCreateNegativeTestJSON-456382249-project] Returning 400 to user: Invalid input for field/attribute max_count. Value: 0. 0 is less than the minimum of 1 {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:45:05.718116 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-2cfbd9e1-ad8a-4432-9721-66617d5f47dd tempest-MultipleCreateNegativeTestJSON-456382249 tempest-MultipleCreateNegativeTestJSON-456382249-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 400 len: 130 microversion: 2.1 time: 0.009195 Jul 16 21:45:05.718116 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2880/5762] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 21:45:05 2022] POST /compute/v2.1/servers => generated 130 bytes in 10 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Jul 16 21:45:05.758423 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-1f43fe2d-0de2-457d-80db-290c60201b3a tempest-MultipleCreateNegativeTestJSON-456382249 tempest-MultipleCreateNegativeTestJSON-456382249-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-MultipleCreateNegativeTestJSON-server-34443219", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452", "max_count": 2.5}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:45:05.761233 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-1f43fe2d-0de2-457d-80db-290c60201b3a tempest-MultipleCreateNegativeTestJSON-456382249 tempest-MultipleCreateNegativeTestJSON-456382249-project] Request method failure captured: Jul 16 21:45:05.761233 ncdv-master-1 devstack@n-api.service[138681]: request: POST /compute/v2.1/servers HTTP/1.1 Jul 16 21:45:05.761233 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:45:05.761233 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:45:05.761233 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:45:05.761233 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 168 Jul 16 21:45:05.761233 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:45:05.761233 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:45:05.761233 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:45:05.761233 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:45:05.761233 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0zFhsGwcXRW3LE25gSnlTSa-5RFYaq8jUEqNhXq-uZVnT-ZDjrxxf_uUcsjFBQU0ed7IC8XSX0G9EnN4MNxjimFSlNjmAUCKvM-DxLhOdGSvFXS8Srh0Ldt2zcsi6l1vTIaJBNv5v1SLIgjA3RVK5SRPMbo3vnN9RCOU19bz7-SnKoA Jul 16 21:45:05.761233 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:45:05.761233 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:45:05.761233 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:45:05.761233 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:45:05.761233 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:45:05.761233 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:45:05.761233 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: fd2229fe58ec4362967b0bdda9101cef Jul 16 21:45:05.761233 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-MultipleCreateNegativeTestJSON-456382249 Jul 16 21:45:05.761233 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:45:05.761233 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:45:05.761233 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/fd2229fe58ec4362967b0bdda9101cef"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/fd2229fe58ec4362967b0bdda9101cef"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_fd2229fe58ec4362967b0bdda9101cef"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/fd2229fe58ec4362967b0bdda9101cef"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:45:05.761233 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-MultipleCreateNegativeTestJSON-456382249 Jul 16 21:45:05.761233 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: fd2229fe58ec4362967b0bdda9101cef Jul 16 21:45:05.761233 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-MultipleCreateNegativeTestJSON-456382249 Jul 16 21:45:05.761233 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-MultipleCreateNegativeTestJSON-456382249-project Jul 16 21:45:05.761233 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:45:05.761233 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:45:05.761233 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: 62b825dcfc864969b2cae9308c975e6d Jul 16 21:45:05.761233 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-MultipleCreateNegativeTestJSON-456382249-project Jul 16 21:45:05.761233 ncdv-master-1 devstack@n-api.service[138681]: Jul 16 21:45:05.761233 ncdv-master-1 devstack@n-api.service[138681]: {"server": {"name": "tempest-MultipleCreateNegativeTestJSON-server-34443219", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452", "max_count": 2.5}} Jul 16 21:45:05.761233 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:45:05.761233 ncdv-master-1 devstack@n-api.service[138681]: exception: Invalid input for field/attribute max_count. Value: 2.5. 2.5 is not of type 'integer', 'string' Jul 16 21:45:05.761233 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'body': {'server': {'name': 'tempest-MultipleCreateNegativeTestJSON-server-34443219', 'imageRef': '1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9', 'flavorRef': '452', 'max_count': 2.5}}} Jul 16 21:45:05.761233 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:45:05.761233 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:45:05.761233 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 279, in validate Jul 16 21:45:05.761233 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Jul 16 21:45:05.761233 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/lib/python3/dist-packages/jsonschema/validators.py", line 353, in validate Jul 16 21:45:05.761233 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise error Jul 16 21:45:05.761233 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: 2.5 is not of type 'integer', 'string' Jul 16 21:45:05.761233 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:45:05.761233 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Failed validating 'type' in schema['properties']['server']['properties']['max_count']: Jul 16 21:45:05.761233 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi {'minLength': 1, Jul 16 21:45:05.761233 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi 'minimum': 1, Jul 16 21:45:05.761233 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi 'pattern': '^[0-9]*$', Jul 16 21:45:05.761233 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi 'type': ['integer', 'string']} Jul 16 21:45:05.761233 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:45:05.761233 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi On instance['server']: Jul 16 21:45:05.761233 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi 2.5 Jul 16 21:45:05.761233 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:45:05.761233 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:45:05.761233 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:45:05.761233 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:45:05.761233 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:45:05.761233 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:45:05.761233 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:45:05.761233 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:45:05.761233 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:45:05.761233 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:45:05.761233 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 16 21:45:05.761233 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 16 21:45:05.761233 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Jul 16 21:45:05.761233 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Jul 16 21:45:05.767996 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Jul 16 21:45:05.767996 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Jul 16 21:45:05.767996 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 308, in validate Jul 16 21:45:05.767996 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Jul 16 21:45:05.767996 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute max_count. Value: 2.5. 2.5 is not of type 'integer', 'string' Jul 16 21:45:05.767996 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:45:05.767996 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:45:05.767996 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:45:05.767996 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:45:05.767996 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:45:05.767996 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:45:05.767996 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Jul 16 21:45:05.767996 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Jul 16 21:45:05.767996 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute max_count. Value: 2.5. 2.5 is not of type 'integer', 'string' Jul 16 21:45:05.767996 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:45:05.767996 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-1f43fe2d-0de2-457d-80db-290c60201b3a tempest-MultipleCreateNegativeTestJSON-456382249 tempest-MultipleCreateNegativeTestJSON-456382249-project] Returning 400 to user: Invalid input for field/attribute max_count. Value: 2.5. 2.5 is not of type 'integer', 'string' {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:45:05.767996 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-1f43fe2d-0de2-457d-80db-290c60201b3a tempest-MultipleCreateNegativeTestJSON-456382249 tempest-MultipleCreateNegativeTestJSON-456382249-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 400 len: 139 microversion: 2.1 time: 0.008107 Jul 16 21:45:05.767996 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2883/5763] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 21:45:05 2022] POST /compute/v2.1/servers => generated 139 bytes in 9 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Jul 16 21:45:05.815746 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-11a7206d-e90d-4ea5-a0c1-bea377380905 tempest-MultipleCreateNegativeTestJSON-456382249 tempest-MultipleCreateNegativeTestJSON-456382249-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-MultipleCreateNegativeTestJSON-server-1169214132", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452", "min_count": 0}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:45:05.818972 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-11a7206d-e90d-4ea5-a0c1-bea377380905 tempest-MultipleCreateNegativeTestJSON-456382249 tempest-MultipleCreateNegativeTestJSON-456382249-project] Request method failure captured: Jul 16 21:45:05.818972 ncdv-master-1 devstack@n-api.service[138680]: request: POST /compute/v2.1/servers HTTP/1.1 Jul 16 21:45:05.818972 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:45:05.818972 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:45:05.818972 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:45:05.818972 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 168 Jul 16 21:45:05.818972 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:45:05.818972 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:45:05.818972 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:45:05.818972 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:45:05.818972 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0zFhsGwcXRW3LE25gSnlTSa-5RFYaq8jUEqNhXq-uZVnT-ZDjrxxf_uUcsjFBQU0ed7IC8XSX0G9EnN4MNxjimFSlNjmAUCKvM-DxLhOdGSvFXS8Srh0Ldt2zcsi6l1vTIaJBNv5v1SLIgjA3RVK5SRPMbo3vnN9RCOU19bz7-SnKoA Jul 16 21:45:05.818972 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:45:05.818972 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:45:05.818972 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:45:05.818972 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:45:05.818972 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:45:05.818972 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:45:05.818972 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: fd2229fe58ec4362967b0bdda9101cef Jul 16 21:45:05.818972 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-MultipleCreateNegativeTestJSON-456382249 Jul 16 21:45:05.818972 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:45:05.818972 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:45:05.818972 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/fd2229fe58ec4362967b0bdda9101cef"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/fd2229fe58ec4362967b0bdda9101cef"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_fd2229fe58ec4362967b0bdda9101cef"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/fd2229fe58ec4362967b0bdda9101cef"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:45:05.818972 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-MultipleCreateNegativeTestJSON-456382249 Jul 16 21:45:05.818972 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: fd2229fe58ec4362967b0bdda9101cef Jul 16 21:45:05.818972 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-MultipleCreateNegativeTestJSON-456382249 Jul 16 21:45:05.818972 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-MultipleCreateNegativeTestJSON-456382249-project Jul 16 21:45:05.818972 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:45:05.818972 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:45:05.818972 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: 62b825dcfc864969b2cae9308c975e6d Jul 16 21:45:05.818972 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-MultipleCreateNegativeTestJSON-456382249-project Jul 16 21:45:05.818972 ncdv-master-1 devstack@n-api.service[138680]: Jul 16 21:45:05.818972 ncdv-master-1 devstack@n-api.service[138680]: {"server": {"name": "tempest-MultipleCreateNegativeTestJSON-server-1169214132", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452", "min_count": 0}} Jul 16 21:45:05.818972 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 21:45:05.818972 ncdv-master-1 devstack@n-api.service[138680]: exception: Invalid input for field/attribute min_count. Value: 0. 0 is less than the minimum of 1 Jul 16 21:45:05.818972 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'body': {'server': {'name': 'tempest-MultipleCreateNegativeTestJSON-server-1169214132', 'imageRef': '1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9', 'flavorRef': '452', 'min_count': 0}}} Jul 16 21:45:05.818972 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:45:05.818972 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:45:05.818972 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 279, in validate Jul 16 21:45:05.818972 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Jul 16 21:45:05.818972 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/lib/python3/dist-packages/jsonschema/validators.py", line 353, in validate Jul 16 21:45:05.818972 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise error Jul 16 21:45:05.818972 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: 0 is less than the minimum of 1 Jul 16 21:45:05.818972 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:45:05.818972 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Failed validating 'minimum' in schema['properties']['server']['properties']['min_count']: Jul 16 21:45:05.818972 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi {'minLength': 1, Jul 16 21:45:05.818972 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi 'minimum': 1, Jul 16 21:45:05.818972 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi 'pattern': '^[0-9]*$', Jul 16 21:45:05.818972 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi 'type': ['integer', 'string']} Jul 16 21:45:05.818972 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:45:05.818972 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi On instance['server']: Jul 16 21:45:05.818972 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi 0 Jul 16 21:45:05.818972 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:45:05.818972 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:45:05.818972 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:45:05.818972 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:45:05.818972 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:45:05.818972 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:45:05.818972 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:45:05.818972 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:45:05.818972 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:45:05.818972 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:45:05.818972 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 16 21:45:05.818972 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 16 21:45:05.818972 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Jul 16 21:45:05.818972 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Jul 16 21:45:05.818972 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Jul 16 21:45:05.824308 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Jul 16 21:45:05.824308 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 308, in validate Jul 16 21:45:05.824308 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Jul 16 21:45:05.824308 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute min_count. Value: 0. 0 is less than the minimum of 1 Jul 16 21:45:05.824308 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:45:05.824308 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:45:05.824308 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:45:05.824308 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:45:05.824308 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:45:05.824308 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:45:05.824308 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Jul 16 21:45:05.824308 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Jul 16 21:45:05.824308 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute min_count. Value: 0. 0 is less than the minimum of 1 Jul 16 21:45:05.824308 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:45:05.826171 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-11a7206d-e90d-4ea5-a0c1-bea377380905 tempest-MultipleCreateNegativeTestJSON-456382249 tempest-MultipleCreateNegativeTestJSON-456382249-project] Returning 400 to user: Invalid input for field/attribute min_count. Value: 0. 0 is less than the minimum of 1 {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:45:05.827175 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-11a7206d-e90d-4ea5-a0c1-bea377380905 tempest-MultipleCreateNegativeTestJSON-456382249 tempest-MultipleCreateNegativeTestJSON-456382249-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 400 len: 130 microversion: 2.1 time: 0.015694 Jul 16 21:45:05.827806 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2881/5764] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 21:45:05 2022] POST /compute/v2.1/servers => generated 130 bytes in 17 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Jul 16 21:45:05.880592 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-1764edb0-7290-41fe-8809-3ee6df56805c tempest-MultipleCreateNegativeTestJSON-456382249 tempest-MultipleCreateNegativeTestJSON-456382249-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-MultipleCreateNegativeTestJSON-server-2097396901", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452", "min_count": 2.5}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:45:05.883517 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-1764edb0-7290-41fe-8809-3ee6df56805c tempest-MultipleCreateNegativeTestJSON-456382249 tempest-MultipleCreateNegativeTestJSON-456382249-project] Request method failure captured: Jul 16 21:45:05.883517 ncdv-master-1 devstack@n-api.service[138681]: request: POST /compute/v2.1/servers HTTP/1.1 Jul 16 21:45:05.883517 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:45:05.883517 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:45:05.883517 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:45:05.883517 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 170 Jul 16 21:45:05.883517 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:45:05.883517 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:45:05.883517 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:45:05.883517 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:45:05.883517 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0zFhsGwcXRW3LE25gSnlTSa-5RFYaq8jUEqNhXq-uZVnT-ZDjrxxf_uUcsjFBQU0ed7IC8XSX0G9EnN4MNxjimFSlNjmAUCKvM-DxLhOdGSvFXS8Srh0Ldt2zcsi6l1vTIaJBNv5v1SLIgjA3RVK5SRPMbo3vnN9RCOU19bz7-SnKoA Jul 16 21:45:05.883517 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:45:05.883517 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:45:05.883517 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:45:05.883517 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:45:05.883517 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:45:05.883517 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:45:05.883517 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: fd2229fe58ec4362967b0bdda9101cef Jul 16 21:45:05.883517 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-MultipleCreateNegativeTestJSON-456382249 Jul 16 21:45:05.883517 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:45:05.883517 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:45:05.883517 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/fd2229fe58ec4362967b0bdda9101cef"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/fd2229fe58ec4362967b0bdda9101cef"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_fd2229fe58ec4362967b0bdda9101cef"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/fd2229fe58ec4362967b0bdda9101cef"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:45:05.883517 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-MultipleCreateNegativeTestJSON-456382249 Jul 16 21:45:05.883517 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: fd2229fe58ec4362967b0bdda9101cef Jul 16 21:45:05.883517 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-MultipleCreateNegativeTestJSON-456382249 Jul 16 21:45:05.883517 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-MultipleCreateNegativeTestJSON-456382249-project Jul 16 21:45:05.883517 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:45:05.883517 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:45:05.883517 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: 62b825dcfc864969b2cae9308c975e6d Jul 16 21:45:05.883517 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-MultipleCreateNegativeTestJSON-456382249-project Jul 16 21:45:05.883517 ncdv-master-1 devstack@n-api.service[138681]: Jul 16 21:45:05.883517 ncdv-master-1 devstack@n-api.service[138681]: {"server": {"name": "tempest-MultipleCreateNegativeTestJSON-server-2097396901", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452", "min_count": 2.5}} Jul 16 21:45:05.883517 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:45:05.883517 ncdv-master-1 devstack@n-api.service[138681]: exception: Invalid input for field/attribute min_count. Value: 2.5. 2.5 is not of type 'integer', 'string' Jul 16 21:45:05.883517 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'body': {'server': {'name': 'tempest-MultipleCreateNegativeTestJSON-server-2097396901', 'imageRef': '1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9', 'flavorRef': '452', 'min_count': 2.5}}} Jul 16 21:45:05.883517 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:45:05.883517 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:45:05.883517 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 279, in validate Jul 16 21:45:05.883517 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Jul 16 21:45:05.883517 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/lib/python3/dist-packages/jsonschema/validators.py", line 353, in validate Jul 16 21:45:05.883517 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise error Jul 16 21:45:05.883517 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: 2.5 is not of type 'integer', 'string' Jul 16 21:45:05.883517 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:45:05.883517 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Failed validating 'type' in schema['properties']['server']['properties']['min_count']: Jul 16 21:45:05.883517 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi {'minLength': 1, Jul 16 21:45:05.883517 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi 'minimum': 1, Jul 16 21:45:05.883517 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi 'pattern': '^[0-9]*$', Jul 16 21:45:05.883517 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi 'type': ['integer', 'string']} Jul 16 21:45:05.883517 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:45:05.883517 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi On instance['server']: Jul 16 21:45:05.883517 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi 2.5 Jul 16 21:45:05.883517 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:45:05.883517 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:45:05.883517 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:45:05.883517 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:45:05.883517 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:45:05.883517 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:45:05.883517 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:45:05.883517 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:45:05.883517 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:45:05.883517 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:45:05.883517 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 16 21:45:05.883517 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 16 21:45:05.883517 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Jul 16 21:45:05.883517 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Jul 16 21:45:05.889075 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Jul 16 21:45:05.889075 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Jul 16 21:45:05.889075 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 308, in validate Jul 16 21:45:05.889075 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Jul 16 21:45:05.889075 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute min_count. Value: 2.5. 2.5 is not of type 'integer', 'string' Jul 16 21:45:05.889075 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:45:05.889075 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:45:05.889075 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:45:05.889075 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:45:05.889075 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:45:05.889075 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:45:05.889075 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Jul 16 21:45:05.889075 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Jul 16 21:45:05.889075 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute min_count. Value: 2.5. 2.5 is not of type 'integer', 'string' Jul 16 21:45:05.889075 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:45:05.889075 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-1764edb0-7290-41fe-8809-3ee6df56805c tempest-MultipleCreateNegativeTestJSON-456382249 tempest-MultipleCreateNegativeTestJSON-456382249-project] Returning 400 to user: Invalid input for field/attribute min_count. Value: 2.5. 2.5 is not of type 'integer', 'string' {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:45:05.889075 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-1764edb0-7290-41fe-8809-3ee6df56805c tempest-MultipleCreateNegativeTestJSON-456382249 tempest-MultipleCreateNegativeTestJSON-456382249-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 400 len: 139 microversion: 2.1 time: 0.010674 Jul 16 21:45:05.889075 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2884/5765] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 21:45:05 2022] POST /compute/v2.1/servers => generated 139 bytes in 14 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Jul 16 21:45:06.171233 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-133c12c7-4abb-470f-9066-ddb9cccdb2a0 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:45:06.171971 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-133c12c7-4abb-470f-9066-ddb9cccdb2a0 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] [instance: 0793bca6-2775-4777-a4e3-1fa6a2e45eac] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:45:06.186954 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-133c12c7-4abb-470f-9066-ddb9cccdb2a0 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:06.187733 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-133c12c7-4abb-470f-9066-ddb9cccdb2a0 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:06.188990 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-133c12c7-4abb-470f-9066-ddb9cccdb2a0 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:06.189607 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-133c12c7-4abb-470f-9066-ddb9cccdb2a0 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:06.265439 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-133c12c7-4abb-470f-9066-ddb9cccdb2a0 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/0793bca6-2775-4777-a4e3-1fa6a2e45eac" status: 200 len: 1415 microversion: 2.1 time: 0.100071 Jul 16 21:45:06.266081 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2882/5766] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:45:06 2022] GET /compute/v2.1/servers/0793bca6-2775-4777-a4e3-1fa6a2e45eac => generated 1415 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:45:06.654102 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-f32c0cb3-4376-4851-b82d-31201743a75f tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:45:06.654744 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-f32c0cb3-4376-4851-b82d-31201743a75f tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] [instance: 770fd99a-ef16-4922-847d-bcadc19b90f6] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:45:06.662209 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f32c0cb3-4376-4851-b82d-31201743a75f tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:06.662671 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f32c0cb3-4376-4851-b82d-31201743a75f tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:06.663509 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f32c0cb3-4376-4851-b82d-31201743a75f tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:06.664028 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f32c0cb3-4376-4851-b82d-31201743a75f tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:07.283987 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-8e19b601-a6c4-48a4-82f1-089330efac2e tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:45:07.284610 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-8e19b601-a6c4-48a4-82f1-089330efac2e tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] [instance: 0793bca6-2775-4777-a4e3-1fa6a2e45eac] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:45:07.290122 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-f32c0cb3-4376-4851-b82d-31201743a75f tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0ab8b3f06e994bbca5d6dd14109bfe50', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e60c0863dc6f4313bdc873fda27ae2fc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:45:07.291508 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8e19b601-a6c4-48a4-82f1-089330efac2e tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:07.291990 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8e19b601-a6c4-48a4-82f1-089330efac2e tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:07.292841 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8e19b601-a6c4-48a4-82f1-089330efac2e tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:07.293609 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8e19b601-a6c4-48a4-82f1-089330efac2e tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:07.301953 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell0:SELECT=2 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:45:07.301953 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-f32c0cb3-4376-4851-b82d-31201743a75f tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] 10.222.0.16 "GET /compute/v2.1/servers/770fd99a-ef16-4922-847d-bcadc19b90f6" status: 200 len: 1366 microversion: 2.1 time: 0.651449 Jul 16 21:45:07.302232 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2885/5767] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:45:06 2022] GET /compute/v2.1/servers/770fd99a-ef16-4922-847d-bcadc19b90f6 => generated 1366 bytes in 653 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:45:07.365946 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-8e19b601-a6c4-48a4-82f1-089330efac2e tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/0793bca6-2775-4777-a4e3-1fa6a2e45eac" status: 200 len: 1421 microversion: 2.1 time: 0.087498 Jul 16 21:45:07.366607 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2883/5768] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:45:07 2022] GET /compute/v2.1/servers/0793bca6-2775-4777-a4e3-1fa6a2e45eac => generated 1421 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:45:08.320627 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-dc7623d3-d0dd-4981-9a84-33bb48a2910f tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:45:08.321263 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-dc7623d3-d0dd-4981-9a84-33bb48a2910f tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] [instance: 770fd99a-ef16-4922-847d-bcadc19b90f6] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:45:08.323161 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell0:SELECT=10 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:45:08.330675 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-dc7623d3-d0dd-4981-9a84-33bb48a2910f tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:08.331448 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-dc7623d3-d0dd-4981-9a84-33bb48a2910f tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:08.332427 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-dc7623d3-d0dd-4981-9a84-33bb48a2910f tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:08.334055 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-dc7623d3-d0dd-4981-9a84-33bb48a2910f tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:08.385273 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-ba87fa33-b8e5-491e-8b05-e3b15774bf78 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:45:08.385895 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-ba87fa33-b8e5-491e-8b05-e3b15774bf78 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] [instance: 0793bca6-2775-4777-a4e3-1fa6a2e45eac] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:45:08.394616 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ba87fa33-b8e5-491e-8b05-e3b15774bf78 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:08.395287 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ba87fa33-b8e5-491e-8b05-e3b15774bf78 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:08.396110 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ba87fa33-b8e5-491e-8b05-e3b15774bf78 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:08.396736 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ba87fa33-b8e5-491e-8b05-e3b15774bf78 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:08.613985 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-dc7623d3-d0dd-4981-9a84-33bb48a2910f tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0ab8b3f06e994bbca5d6dd14109bfe50', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e60c0863dc6f4313bdc873fda27ae2fc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:45:08.622093 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-dc7623d3-d0dd-4981-9a84-33bb48a2910f tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] 10.222.0.16 "GET /compute/v2.1/servers/770fd99a-ef16-4922-847d-bcadc19b90f6" status: 200 len: 1366 microversion: 2.1 time: 0.305796 Jul 16 21:45:08.622718 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2886/5769] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:45:08 2022] GET /compute/v2.1/servers/770fd99a-ef16-4922-847d-bcadc19b90f6 => generated 1366 bytes in 307 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:45:08.664024 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-ba87fa33-b8e5-491e-8b05-e3b15774bf78 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/0793bca6-2775-4777-a4e3-1fa6a2e45eac" status: 200 len: 1463 microversion: 2.1 time: 0.282687 Jul 16 21:45:08.665812 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2884/5770] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:45:08 2022] GET /compute/v2.1/servers/0793bca6-2775-4777-a4e3-1fa6a2e45eac => generated 1463 bytes in 285 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:45:09.090892 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-4d8f5792-997a-45d8-ab79-7fc4b2062b44 req-33836a72-5987-4072-9a09-a480c99fd077 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "0793bca6-2775-4777-a4e3-1fa6a2e45eac", "tag": "de32579a-0010-470f-b775-3c60e514ec44"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:45:09.102564 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-4d8f5792-997a-45d8-ab79-7fc4b2062b44 req-33836a72-5987-4072-9a09-a480c99fd077 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:09.103553 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-4d8f5792-997a-45d8-ab79-7fc4b2062b44 req-33836a72-5987-4072-9a09-a480c99fd077 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:09.129687 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-4d8f5792-997a-45d8-ab79-7fc4b2062b44 req-33836a72-5987-4072-9a09-a480c99fd077 service nova] Creating event network-changed:de32579a-0010-470f-b775-3c60e514ec44 for instance 0793bca6-2775-4777-a4e3-1fa6a2e45eac on nch2-master-1 Jul 16 21:45:09.144532 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-4d8f5792-997a-45d8-ab79-7fc4b2062b44 req-33836a72-5987-4072-9a09-a480c99fd077 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.059042 Jul 16 21:45:09.145636 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2887/5771] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:45:09 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 63 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:45:09.640322 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-c570b1b4-ba5a-4978-b809-7c3fd214c087 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:45:09.641536 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-c570b1b4-ba5a-4978-b809-7c3fd214c087 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] [instance: 770fd99a-ef16-4922-847d-bcadc19b90f6] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:45:09.650476 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c570b1b4-ba5a-4978-b809-7c3fd214c087 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:09.650930 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c570b1b4-ba5a-4978-b809-7c3fd214c087 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:09.651746 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c570b1b4-ba5a-4978-b809-7c3fd214c087 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:09.652189 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c570b1b4-ba5a-4978-b809-7c3fd214c087 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:09.684226 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-892e80cf-3683-447c-a710-fbf5848d2427 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:45:09.684936 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-892e80cf-3683-447c-a710-fbf5848d2427 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] [instance: 0793bca6-2775-4777-a4e3-1fa6a2e45eac] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:45:09.694750 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-892e80cf-3683-447c-a710-fbf5848d2427 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:09.694750 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-892e80cf-3683-447c-a710-fbf5848d2427 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:09.695347 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-892e80cf-3683-447c-a710-fbf5848d2427 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:09.695498 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-892e80cf-3683-447c-a710-fbf5848d2427 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:09.915379 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-c570b1b4-ba5a-4978-b809-7c3fd214c087 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0ab8b3f06e994bbca5d6dd14109bfe50', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e60c0863dc6f4313bdc873fda27ae2fc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:45:09.925036 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-c570b1b4-ba5a-4978-b809-7c3fd214c087 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] 10.222.0.16 "GET /compute/v2.1/servers/770fd99a-ef16-4922-847d-bcadc19b90f6" status: 200 len: 1366 microversion: 2.1 time: 0.288779 Jul 16 21:45:09.925640 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2885/5772] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:45:09 2022] GET /compute/v2.1/servers/770fd99a-ef16-4922-847d-bcadc19b90f6 => generated 1366 bytes in 290 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:45:09.983885 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-892e80cf-3683-447c-a710-fbf5848d2427 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/0793bca6-2775-4777-a4e3-1fa6a2e45eac" status: 200 len: 1463 microversion: 2.1 time: 0.304487 Jul 16 21:45:09.984848 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2888/5773] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:45:09 2022] GET /compute/v2.1/servers/0793bca6-2775-4777-a4e3-1fa6a2e45eac => generated 1463 bytes in 307 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:45:10.879450 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-d62c0f2e-01d9-47eb-a4f9-c6ab4ba46750 req-f42cd407-2632-4041-87a1-5b565fb21ec7 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "770fd99a-ef16-4922-847d-bcadc19b90f6", "name": "network-vif-plugged", "status": "completed", "tag": "152e5772-bc79-4772-820e-b77b68f320f3"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:45:10.887912 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-d62c0f2e-01d9-47eb-a4f9-c6ab4ba46750 req-f42cd407-2632-4041-87a1-5b565fb21ec7 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:10.888381 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-d62c0f2e-01d9-47eb-a4f9-c6ab4ba46750 req-f42cd407-2632-4041-87a1-5b565fb21ec7 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:10.905819 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-d62c0f2e-01d9-47eb-a4f9-c6ab4ba46750 req-f42cd407-2632-4041-87a1-5b565fb21ec7 service nova] Creating event network-vif-plugged:152e5772-bc79-4772-820e-b77b68f320f3 for instance 770fd99a-ef16-4922-847d-bcadc19b90f6 on nch2-master-1 Jul 16 21:45:10.912824 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-d62c0f2e-01d9-47eb-a4f9-c6ab4ba46750 req-f42cd407-2632-4041-87a1-5b565fb21ec7 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.200747 Jul 16 21:45:10.913951 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2886/5774] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:45:10 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 203 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:45:10.949209 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-b9660384-5563-4ad0-995a-fbb313baf848 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:45:10.949869 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-b9660384-5563-4ad0-995a-fbb313baf848 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] [instance: 770fd99a-ef16-4922-847d-bcadc19b90f6] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:45:10.959427 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b9660384-5563-4ad0-995a-fbb313baf848 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:10.960615 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b9660384-5563-4ad0-995a-fbb313baf848 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:10.961485 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b9660384-5563-4ad0-995a-fbb313baf848 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:10.961991 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b9660384-5563-4ad0-995a-fbb313baf848 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:11.004106 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-403259dc-a3d5-410e-a9bb-d82df16cfdec tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:45:11.004830 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-403259dc-a3d5-410e-a9bb-d82df16cfdec tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] [instance: 0793bca6-2775-4777-a4e3-1fa6a2e45eac] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:45:11.012053 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-403259dc-a3d5-410e-a9bb-d82df16cfdec tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:11.012771 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-403259dc-a3d5-410e-a9bb-d82df16cfdec tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:11.013639 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-403259dc-a3d5-410e-a9bb-d82df16cfdec tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:11.014230 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-403259dc-a3d5-410e-a9bb-d82df16cfdec tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:11.162452 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-b9660384-5563-4ad0-995a-fbb313baf848 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0ab8b3f06e994bbca5d6dd14109bfe50', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e60c0863dc6f4313bdc873fda27ae2fc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:45:11.177330 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-b9660384-5563-4ad0-995a-fbb313baf848 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] 10.222.0.16 "GET /compute/v2.1/servers/770fd99a-ef16-4922-847d-bcadc19b90f6" status: 200 len: 1366 microversion: 2.1 time: 0.239328 Jul 16 21:45:11.178024 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2889/5775] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:45:10 2022] GET /compute/v2.1/servers/770fd99a-ef16-4922-847d-bcadc19b90f6 => generated 1366 bytes in 241 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:45:11.232212 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-403259dc-a3d5-410e-a9bb-d82df16cfdec tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/0793bca6-2775-4777-a4e3-1fa6a2e45eac" status: 200 len: 1463 microversion: 2.1 time: 0.232876 Jul 16 21:45:11.232947 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2887/5776] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:45:10 2022] GET /compute/v2.1/servers/0793bca6-2775-4777-a4e3-1fa6a2e45eac => generated 1463 bytes in 234 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:45:12.195222 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-5ade2c6b-c1a6-4fe5-a0d5-a108b8428040 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:45:12.196546 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-5ade2c6b-c1a6-4fe5-a0d5-a108b8428040 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] [instance: 770fd99a-ef16-4922-847d-bcadc19b90f6] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:45:12.205676 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5ade2c6b-c1a6-4fe5-a0d5-a108b8428040 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:12.206414 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5ade2c6b-c1a6-4fe5-a0d5-a108b8428040 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:12.207666 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5ade2c6b-c1a6-4fe5-a0d5-a108b8428040 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:12.212509 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5ade2c6b-c1a6-4fe5-a0d5-a108b8428040 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:12.251133 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-58bb3cc1-4578-4126-9516-095e1ba7e5a4 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:45:12.252503 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-58bb3cc1-4578-4126-9516-095e1ba7e5a4 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] [instance: 0793bca6-2775-4777-a4e3-1fa6a2e45eac] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:45:12.261216 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-58bb3cc1-4578-4126-9516-095e1ba7e5a4 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:12.262027 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-58bb3cc1-4578-4126-9516-095e1ba7e5a4 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:12.263138 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-58bb3cc1-4578-4126-9516-095e1ba7e5a4 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:12.263804 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-58bb3cc1-4578-4126-9516-095e1ba7e5a4 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:12.287104 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:45:12.293390 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell0:SELECT=4 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:45:12.472679 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-5ade2c6b-c1a6-4fe5-a0d5-a108b8428040 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0ab8b3f06e994bbca5d6dd14109bfe50', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e60c0863dc6f4313bdc873fda27ae2fc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:45:12.481915 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=100 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:45:12.486378 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-5ade2c6b-c1a6-4fe5-a0d5-a108b8428040 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] 10.222.0.16 "GET /compute/v2.1/servers/770fd99a-ef16-4922-847d-bcadc19b90f6" status: 200 len: 1366 microversion: 2.1 time: 0.294115 Jul 16 21:45:12.497283 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2890/5777] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:45:12 2022] GET /compute/v2.1/servers/770fd99a-ef16-4922-847d-bcadc19b90f6 => generated 1366 bytes in 307 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:45:12.530768 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-58bb3cc1-4578-4126-9516-095e1ba7e5a4 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/0793bca6-2775-4777-a4e3-1fa6a2e45eac" status: 200 len: 1463 microversion: 2.1 time: 0.281468 Jul 16 21:45:12.535820 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2888/5778] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:45:12 2022] GET /compute/v2.1/servers/0793bca6-2775-4777-a4e3-1fa6a2e45eac => generated 1463 bytes in 291 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:45:13.504194 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-e604e4a8-0b6e-45ea-b841-ab925cfe6b6c tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:45:13.504826 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-e604e4a8-0b6e-45ea-b841-ab925cfe6b6c tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] [instance: 770fd99a-ef16-4922-847d-bcadc19b90f6] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:45:13.513026 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e604e4a8-0b6e-45ea-b841-ab925cfe6b6c tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:13.513769 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e604e4a8-0b6e-45ea-b841-ab925cfe6b6c tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:13.515002 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e604e4a8-0b6e-45ea-b841-ab925cfe6b6c tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:13.515718 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e604e4a8-0b6e-45ea-b841-ab925cfe6b6c tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:13.552946 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-88f8e6d1-263e-43fb-9fa3-ccfe124ae812 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:45:13.554269 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-88f8e6d1-263e-43fb-9fa3-ccfe124ae812 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] [instance: 0793bca6-2775-4777-a4e3-1fa6a2e45eac] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:45:13.564422 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-88f8e6d1-263e-43fb-9fa3-ccfe124ae812 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:13.565416 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-88f8e6d1-263e-43fb-9fa3-ccfe124ae812 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:13.566454 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-88f8e6d1-263e-43fb-9fa3-ccfe124ae812 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:13.567144 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-88f8e6d1-263e-43fb-9fa3-ccfe124ae812 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:13.765325 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-e604e4a8-0b6e-45ea-b841-ab925cfe6b6c tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0ab8b3f06e994bbca5d6dd14109bfe50', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e60c0863dc6f4313bdc873fda27ae2fc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:45:13.789967 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-e604e4a8-0b6e-45ea-b841-ab925cfe6b6c tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] 10.222.0.16 "GET /compute/v2.1/servers/770fd99a-ef16-4922-847d-bcadc19b90f6" status: 200 len: 1556 microversion: 2.1 time: 0.290287 Jul 16 21:45:13.790573 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2891/5779] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:45:13 2022] GET /compute/v2.1/servers/770fd99a-ef16-4922-847d-bcadc19b90f6 => generated 1556 bytes in 292 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:45:13.809712 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-dcdedb93-bbf9-407c-834b-84d4b640803e tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:45:13.810274 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-dcdedb93-bbf9-407c-834b-84d4b640803e tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] [instance: 770fd99a-ef16-4922-847d-bcadc19b90f6] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:45:13.818247 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-dcdedb93-bbf9-407c-834b-84d4b640803e tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:13.819290 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-dcdedb93-bbf9-407c-834b-84d4b640803e tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:13.820110 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-dcdedb93-bbf9-407c-834b-84d4b640803e tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:13.820568 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-dcdedb93-bbf9-407c-834b-84d4b640803e tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:13.836495 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-88f8e6d1-263e-43fb-9fa3-ccfe124ae812 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/0793bca6-2775-4777-a4e3-1fa6a2e45eac" status: 200 len: 1463 microversion: 2.1 time: 0.287378 Jul 16 21:45:13.837597 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2889/5780] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:45:13 2022] GET /compute/v2.1/servers/0793bca6-2775-4777-a4e3-1fa6a2e45eac => generated 1463 bytes in 289 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:45:13.930843 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-dcdedb93-bbf9-407c-834b-84d4b640803e tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] 10.222.0.16 "GET /compute/v2.1/servers/770fd99a-ef16-4922-847d-bcadc19b90f6/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.126082 Jul 16 21:45:13.931891 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2892/5781] 10.222.0.16 () {58 vars in 1274 bytes} [Sat Jul 16 21:45:13 2022] GET /compute/v2.1/servers/770fd99a-ef16-4922-847d-bcadc19b90f6/os-interface => generated 284 bytes in 128 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 16 21:45:14.855665 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-16ce9eaa-6148-451e-bf97-f6b5b1a97286 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:45:14.858097 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-16ce9eaa-6148-451e-bf97-f6b5b1a97286 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] [instance: 0793bca6-2775-4777-a4e3-1fa6a2e45eac] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:45:14.890565 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell0:SELECT=12 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:45:14.890565 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-16ce9eaa-6148-451e-bf97-f6b5b1a97286 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:14.890565 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-16ce9eaa-6148-451e-bf97-f6b5b1a97286 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:14.890565 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-16ce9eaa-6148-451e-bf97-f6b5b1a97286 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:14.890565 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-16ce9eaa-6148-451e-bf97-f6b5b1a97286 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:15.142074 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-a982ced5-1b59-44b4-ae79-678cd553d64e tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Action: 'create', calling method: .version_select at 0x7fdda2a24310>, body: {"keypair": {"name": "tempest-keypair-225926832", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBJ9+gOzhZ2rjFaRxe4xTCB/pOy2Xc5P0JYECsNV8tyqyHJK8ZGZbMOX0tx/IGGqSVPAXcMrW8q7/eMuEkYz+iPIg2+OBHXdxR3kpRNja5Lv2DLQy+hMfrprYi2K6MSIl9g=="}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:45:15.158943 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-a982ced5-1b59-44b4-ae79-678cd553d64e tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Getting quotas for project 1e7cd69b8ddd4aceb45a4aaacdfccea7. Resources: {'key_pairs'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:45:15.159958 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-16ce9eaa-6148-451e-bf97-f6b5b1a97286 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/0793bca6-2775-4777-a4e3-1fa6a2e45eac" status: 200 len: 1463 microversion: 2.1 time: 0.307255 Jul 16 21:45:15.160782 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2890/5782] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:45:14 2022] GET /compute/v2.1/servers/0793bca6-2775-4777-a4e3-1fa6a2e45eac => generated 1463 bytes in 311 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:45:15.166215 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-a982ced5-1b59-44b4-ae79-678cd553d64e tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Getting quotas for user cfe293b9ed3645229bed5ef8ce17cfec and project 1e7cd69b8ddd4aceb45a4aaacdfccea7. Resources: {'key_pairs'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:45:15.364565 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-fdd3354a-4e48-466a-810e-6fa7412ca455 req-fa284777-cd21-4849-988a-f539250fb834 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "770fd99a-ef16-4922-847d-bcadc19b90f6", "tag": "152e5772-bc79-4772-820e-b77b68f320f3"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:45:15.368783 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-a982ced5-1b59-44b4-ae79-678cd553d64e tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "POST /compute/v2.1/os-keypairs" status: 200 len: 385 microversion: 2.1 time: 0.359369 Jul 16 21:45:15.369275 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2893/5783] 10.222.0.16 () {60 vars in 1158 bytes} [Sat Jul 16 21:45:15 2022] POST /compute/v2.1/os-keypairs => generated 385 bytes in 363 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:45:15.374790 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-fdd3354a-4e48-466a-810e-6fa7412ca455 req-fa284777-cd21-4849-988a-f539250fb834 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:15.374790 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-fdd3354a-4e48-466a-810e-6fa7412ca455 req-fa284777-cd21-4849-988a-f539250fb834 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:15.400198 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-fdd3354a-4e48-466a-810e-6fa7412ca455 req-fa284777-cd21-4849-988a-f539250fb834 service nova] Creating event network-changed:152e5772-bc79-4772-820e-b77b68f320f3 for instance 770fd99a-ef16-4922-847d-bcadc19b90f6 on nch2-master-1 Jul 16 21:45:15.406079 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-fdd3354a-4e48-466a-810e-6fa7412ca455 req-fa284777-cd21-4849-988a-f539250fb834 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.047420 Jul 16 21:45:15.407273 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2891/5784] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:45:15 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 16 21:45:16.182667 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-12efcda4-dd2a-4c4c-a37a-c8c58cd4f02b tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:45:16.182667 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-12efcda4-dd2a-4c4c-a37a-c8c58cd4f02b tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] [instance: 0793bca6-2775-4777-a4e3-1fa6a2e45eac] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:45:16.189477 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-12efcda4-dd2a-4c4c-a37a-c8c58cd4f02b tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:16.190486 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-12efcda4-dd2a-4c4c-a37a-c8c58cd4f02b tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:16.191556 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-12efcda4-dd2a-4c4c-a37a-c8c58cd4f02b tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:16.192021 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-12efcda4-dd2a-4c4c-a37a-c8c58cd4f02b tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:16.355651 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-12efcda4-dd2a-4c4c-a37a-c8c58cd4f02b tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/0793bca6-2775-4777-a4e3-1fa6a2e45eac" status: 200 len: 1463 microversion: 2.1 time: 0.182017 Jul 16 21:45:16.356599 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2894/5785] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:45:16 2022] GET /compute/v2.1/servers/0793bca6-2775-4777-a4e3-1fa6a2e45eac => generated 1463 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:45:16.735410 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-f2f64656-a86d-4895-9a01-c8cb271a6b0a req-7c7bf8ae-313d-4cef-b4a1-7a17a92a7bf6 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "0793bca6-2775-4777-a4e3-1fa6a2e45eac", "name": "network-vif-plugged", "status": "completed", "tag": "de32579a-0010-470f-b775-3c60e514ec44"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:45:16.744326 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-f2f64656-a86d-4895-9a01-c8cb271a6b0a req-7c7bf8ae-313d-4cef-b4a1-7a17a92a7bf6 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:16.744887 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-f2f64656-a86d-4895-9a01-c8cb271a6b0a req-7c7bf8ae-313d-4cef-b4a1-7a17a92a7bf6 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:16.756215 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-f2f64656-a86d-4895-9a01-c8cb271a6b0a req-7c7bf8ae-313d-4cef-b4a1-7a17a92a7bf6 service nova] Creating event network-vif-plugged:de32579a-0010-470f-b775-3c60e514ec44 for instance 0793bca6-2775-4777-a4e3-1fa6a2e45eac on nch2-master-1 Jul 16 21:45:16.761043 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-f2f64656-a86d-4895-9a01-c8cb271a6b0a req-7c7bf8ae-313d-4cef-b4a1-7a17a92a7bf6 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.031888 Jul 16 21:45:16.761648 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2892/5786] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:45:16 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 16 21:45:17.374792 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-46e9a582-bb54-469e-934f-ed34b8142595 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:45:17.376219 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-46e9a582-bb54-469e-934f-ed34b8142595 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] [instance: 0793bca6-2775-4777-a4e3-1fa6a2e45eac] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:45:17.387285 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-46e9a582-bb54-469e-934f-ed34b8142595 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:17.387956 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-46e9a582-bb54-469e-934f-ed34b8142595 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:17.388755 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-46e9a582-bb54-469e-934f-ed34b8142595 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:17.389207 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-46e9a582-bb54-469e-934f-ed34b8142595 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:17.646945 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-46e9a582-bb54-469e-934f-ed34b8142595 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/0793bca6-2775-4777-a4e3-1fa6a2e45eac" status: 200 len: 1463 microversion: 2.1 time: 0.274155 Jul 16 21:45:17.648014 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2895/5787] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:45:17 2022] GET /compute/v2.1/servers/0793bca6-2775-4777-a4e3-1fa6a2e45eac => generated 1463 bytes in 279 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:45:18.024438 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-056d3ec5-1db3-416c-8239-57b1e2919f5d tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Action: 'action', calling method: >, body: {"os-getConsoleOutput": {}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:45:18.025484 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-056d3ec5-1db3-416c-8239-57b1e2919f5d tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] [instance: 0b62eebf-c7ed-4fa1-a4d1-17947e5d8226] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:45:18.053236 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-056d3ec5-1db3-416c-8239-57b1e2919f5d tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:18.055320 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-056d3ec5-1db3-416c-8239-57b1e2919f5d tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:18.059429 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-056d3ec5-1db3-416c-8239-57b1e2919f5d tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:18.063362 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-056d3ec5-1db3-416c-8239-57b1e2919f5d tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:18.131448 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-056d3ec5-1db3-416c-8239-57b1e2919f5d tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] 10.222.0.16 "POST /compute/v2.1/servers/0b62eebf-c7ed-4fa1-a4d1-17947e5d8226/action" status: 200 len: 24557 microversion: 2.1 time: 0.116233 Jul 16 21:45:18.132916 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2893/5788] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 21:45:18 2022] POST /compute/v2.1/servers/0b62eebf-c7ed-4fa1-a4d1-17947e5d8226/action => generated 24557 bytes in 118 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 16 21:45:18.336019 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-05ea5df9-2e4a-453a-bb0b-9c24ce6c7831 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerActionsTestJSON-server-732935322", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "af28c06e-7ce2-4ca3-b6a4-f1ae8d70db1a"}], "security_groups": [{"name": "tempest-securitygroup--161971440"}], "key_name": "tempest-keypair-225926832", "user_data": "}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:45:18.461030 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.network.neutron [None req-05ea5df9-2e4a-453a-bb0b-9c24ce6c7831 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] validate_networks() for [('af28c06e-7ce2-4ca3-b6a4-f1ae8d70db1a', None, None, None, None, None)] {{(pid=138681) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 16 21:45:18.666274 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-392e1361-22b6-464c-8c10-b0c22545b9f1 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:45:18.667055 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-392e1361-22b6-464c-8c10-b0c22545b9f1 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] [instance: 0793bca6-2775-4777-a4e3-1fa6a2e45eac] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:45:18.683942 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-392e1361-22b6-464c-8c10-b0c22545b9f1 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:18.684416 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-392e1361-22b6-464c-8c10-b0c22545b9f1 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:18.685434 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-392e1361-22b6-464c-8c10-b0c22545b9f1 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:18.689199 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-392e1361-22b6-464c-8c10-b0c22545b9f1 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:19.388717 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-392e1361-22b6-464c-8c10-b0c22545b9f1 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/0793bca6-2775-4777-a4e3-1fa6a2e45eac" status: 200 len: 1649 microversion: 2.1 time: 0.727437 Jul 16 21:45:19.400896 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2894/5789] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:45:18 2022] GET /compute/v2.1/servers/0793bca6-2775-4777-a4e3-1fa6a2e45eac => generated 1649 bytes in 740 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:45:19.409442 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-dc252c4a-bb2b-4640-87f0-a7d4a26c2ca9 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:45:19.410383 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-dc252c4a-bb2b-4640-87f0-a7d4a26c2ca9 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] [instance: 0793bca6-2775-4777-a4e3-1fa6a2e45eac] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:45:19.419001 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-dc252c4a-bb2b-4640-87f0-a7d4a26c2ca9 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:19.419778 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-dc252c4a-bb2b-4640-87f0-a7d4a26c2ca9 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:19.420742 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-dc252c4a-bb2b-4640-87f0-a7d4a26c2ca9 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:19.421356 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-dc252c4a-bb2b-4640-87f0-a7d4a26c2ca9 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:19.663365 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-05ea5df9-2e4a-453a-bb0b-9c24ce6c7831 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Flavor limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 16 21:45:19.664063 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-05ea5df9-2e4a-453a-bb0b-9c24ce6c7831 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Image limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 16 21:45:19.664565 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-05ea5df9-2e4a-453a-bb0b-9c24ce6c7831 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Flavor pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 16 21:45:19.664992 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-05ea5df9-2e4a-453a-bb0b-9c24ce6c7831 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Image pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 16 21:45:19.665418 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-05ea5df9-2e4a-453a-bb0b-9c24ce6c7831 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 16 21:45:19.677850 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-05ea5df9-2e4a-453a-bb0b-9c24ce6c7831 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:19.678708 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-05ea5df9-2e4a-453a-bb0b-9c24ce6c7831 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:19.679712 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-05ea5df9-2e4a-453a-bb0b-9c24ce6c7831 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:19.680103 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-05ea5df9-2e4a-453a-bb0b-9c24ce6c7831 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:19.700751 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-05ea5df9-2e4a-453a-bb0b-9c24ce6c7831 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Getting quotas for project 1e7cd69b8ddd4aceb45a4aaacdfccea7. Resources: {'ram', 'instances', 'cores'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:45:19.707786 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-05ea5df9-2e4a-453a-bb0b-9c24ce6c7831 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Getting quotas for user cfe293b9ed3645229bed5ef8ce17cfec and project 1e7cd69b8ddd4aceb45a4aaacdfccea7. Resources: {'ram', 'instances', 'cores'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:45:19.723140 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-05ea5df9-2e4a-453a-bb0b-9c24ce6c7831 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Going to run 1 instances... {{(pid=138681) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 16 21:45:19.726967 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-05ea5df9-2e4a-453a-bb0b-9c24ce6c7831 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: 3618ceb8-9d10-4219-8885-d2d5004c71ad] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138681) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 16 21:45:19.781181 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-dc252c4a-bb2b-4640-87f0-a7d4a26c2ca9 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/0793bca6-2775-4777-a4e3-1fa6a2e45eac" status: 200 len: 1649 microversion: 2.1 time: 0.377381 Jul 16 21:45:19.781880 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2895/5790] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:45:19 2022] GET /compute/v2.1/servers/0793bca6-2775-4777-a4e3-1fa6a2e45eac => generated 1649 bytes in 380 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:45:19.800770 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-8277ee87-4552-4d3b-8743-4fe6a93fdc17 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:45:19.801459 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-8277ee87-4552-4d3b-8743-4fe6a93fdc17 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] [instance: 0793bca6-2775-4777-a4e3-1fa6a2e45eac] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:45:19.808635 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8277ee87-4552-4d3b-8743-4fe6a93fdc17 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:19.809111 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8277ee87-4552-4d3b-8743-4fe6a93fdc17 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:19.809938 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8277ee87-4552-4d3b-8743-4fe6a93fdc17 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:19.810569 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8277ee87-4552-4d3b-8743-4fe6a93fdc17 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:19.865347 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-05ea5df9-2e4a-453a-bb0b-9c24ce6c7831 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 408 microversion: 2.1 time: 1.534374 Jul 16 21:45:19.871795 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2896/5791] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 21:45:18 2022] POST /compute/v2.1/servers => generated 408 bytes in 1543 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 16 21:45:19.883314 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-0957a6b1-0d0e-4ddf-a813-3bea27c7c259 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:45:19.896393 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-0957a6b1-0d0e-4ddf-a813-3bea27c7c259 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: 3618ceb8-9d10-4219-8885-d2d5004c71ad] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:45:19.976627 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-0957a6b1-0d0e-4ddf-a813-3bea27c7c259 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:45:19.987889 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-0957a6b1-0d0e-4ddf-a813-3bea27c7c259 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad" status: 200 len: 1237 microversion: 2.1 time: 0.109266 Jul 16 21:45:19.999331 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2897/5792] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:45:19 2022] GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad => generated 1237 bytes in 121 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:45:20.091389 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-8277ee87-4552-4d3b-8743-4fe6a93fdc17 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/0793bca6-2775-4777-a4e3-1fa6a2e45eac" status: 200 len: 1649 microversion: 2.1 time: 0.294540 Jul 16 21:45:20.091972 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2896/5793] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:45:19 2022] GET /compute/v2.1/servers/0793bca6-2775-4777-a4e3-1fa6a2e45eac => generated 1649 bytes in 296 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:45:20.213093 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-680398e9-baa9-4abb-a788-be0d834187c9 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestServerMultinode-server-1653194363", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452", "availability_zone": "nova:nch1-master-1", "networks": [{"uuid": "a8fefdbc-7112-4617-926a-b233743507a1"}]}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:45:20.275390 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.network.neutron [None req-680398e9-baa9-4abb-a788-be0d834187c9 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] validate_networks() for [('a8fefdbc-7112-4617-926a-b233743507a1', None, None, None, None, None)] {{(pid=138681) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 16 21:45:20.660179 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-680398e9-baa9-4abb-a788-be0d834187c9 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Flavor limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 16 21:45:20.660619 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-680398e9-baa9-4abb-a788-be0d834187c9 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Image limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 16 21:45:20.661120 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-680398e9-baa9-4abb-a788-be0d834187c9 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Flavor pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 16 21:45:20.661546 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-680398e9-baa9-4abb-a788-be0d834187c9 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Image pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 16 21:45:20.661976 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-680398e9-baa9-4abb-a788-be0d834187c9 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 16 21:45:20.671292 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-680398e9-baa9-4abb-a788-be0d834187c9 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:20.672040 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-680398e9-baa9-4abb-a788-be0d834187c9 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:20.672739 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-680398e9-baa9-4abb-a788-be0d834187c9 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:20.673191 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-680398e9-baa9-4abb-a788-be0d834187c9 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:20.690439 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-680398e9-baa9-4abb-a788-be0d834187c9 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Getting quotas for project b351f1a2d6434cd0b3b03628a0befe14. Resources: {'ram', 'instances', 'cores'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:45:20.695535 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-680398e9-baa9-4abb-a788-be0d834187c9 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Getting quotas for user e32ea3fe570b40b59110b70fbb0c3caf and project b351f1a2d6434cd0b3b03628a0befe14. Resources: {'ram', 'instances', 'cores'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:45:20.708410 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-680398e9-baa9-4abb-a788-be0d834187c9 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Going to run 1 instances... {{(pid=138681) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 16 21:45:20.710588 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-680398e9-baa9-4abb-a788-be0d834187c9 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] [instance: f0c3fd0a-44d4-4209-9c2f-817382c0afa7] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138681) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 16 21:45:20.877878 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-680398e9-baa9-4abb-a788-be0d834187c9 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.668470 Jul 16 21:45:20.878525 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2898/5794] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 21:45:20 2022] POST /compute/v2.1/servers => generated 380 bytes in 670 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 16 21:45:20.890625 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-f9afb1ea-ab6f-4967-ab2e-fbedb144e54c tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:45:20.892621 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-f9afb1ea-ab6f-4967-ab2e-fbedb144e54c tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] [instance: f0c3fd0a-44d4-4209-9c2f-817382c0afa7] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:45:20.960590 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-f9afb1ea-ab6f-4967-ab2e-fbedb144e54c tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/f0c3fd0a-44d4-4209-9c2f-817382c0afa7" status: 200 len: 1329 microversion: 2.1 time: 0.073703 Jul 16 21:45:20.961175 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2897/5795] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:45:20 2022] GET /compute/v2.1/servers/f0c3fd0a-44d4-4209-9c2f-817382c0afa7 => generated 1329 bytes in 75 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:45:21.007820 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-e7b58331-dd8c-4d97-8bd8-351061afccd7 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:45:21.008461 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-e7b58331-dd8c-4d97-8bd8-351061afccd7 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: 3618ceb8-9d10-4219-8885-d2d5004c71ad] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:45:21.070539 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-e7b58331-dd8c-4d97-8bd8-351061afccd7 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:45:21.078729 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-e7b58331-dd8c-4d97-8bd8-351061afccd7 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad" status: 200 len: 1237 microversion: 2.1 time: 0.076428 Jul 16 21:45:21.084874 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2899/5796] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:45:21 2022] GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad => generated 1237 bytes in 79 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:45:21.978002 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-923278f2-a73a-4ea9-b838-8477c465b224 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:45:21.980304 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-923278f2-a73a-4ea9-b838-8477c465b224 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] [instance: f0c3fd0a-44d4-4209-9c2f-817382c0afa7] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:45:21.990597 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-923278f2-a73a-4ea9-b838-8477c465b224 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:21.991083 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-923278f2-a73a-4ea9-b838-8477c465b224 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:21.991863 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-923278f2-a73a-4ea9-b838-8477c465b224 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:21.992328 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-923278f2-a73a-4ea9-b838-8477c465b224 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:22.068289 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-923278f2-a73a-4ea9-b838-8477c465b224 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/f0c3fd0a-44d4-4209-9c2f-817382c0afa7" status: 200 len: 1346 microversion: 2.1 time: 0.094223 Jul 16 21:45:22.068862 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2898/5797] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:45:21 2022] GET /compute/v2.1/servers/f0c3fd0a-44d4-4209-9c2f-817382c0afa7 => generated 1346 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:45:22.097166 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-81024d8f-42be-4d5b-9cdc-0dd7fc8ab8f0 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:45:22.098595 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-81024d8f-42be-4d5b-9cdc-0dd7fc8ab8f0 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: 3618ceb8-9d10-4219-8885-d2d5004c71ad] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:45:22.102095 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_api:SELECT=90,nova_api:INSERT=10 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:45:22.107304 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-81024d8f-42be-4d5b-9cdc-0dd7fc8ab8f0 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:22.107770 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-81024d8f-42be-4d5b-9cdc-0dd7fc8ab8f0 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:22.108605 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-81024d8f-42be-4d5b-9cdc-0dd7fc8ab8f0 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:22.109062 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-81024d8f-42be-4d5b-9cdc-0dd7fc8ab8f0 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:22.175737 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-81024d8f-42be-4d5b-9cdc-0dd7fc8ab8f0 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:45:22.183262 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-81024d8f-42be-4d5b-9cdc-0dd7fc8ab8f0 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad" status: 200 len: 1289 microversion: 2.1 time: 0.090941 Jul 16 21:45:22.183826 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2900/5798] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:45:22 2022] GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad => generated 1289 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:45:22.886746 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-1b6e085c-9cc7-47a6-b2f3-dc0ff71ff77f tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Action: 'action', calling method: >, body: {"os-getConsoleOutput": {}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:45:22.887796 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-1b6e085c-9cc7-47a6-b2f3-dc0ff71ff77f tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] [instance: 5865f75f-6142-4885-9fa6-0687a2c169cc] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:45:22.894327 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1b6e085c-9cc7-47a6-b2f3-dc0ff71ff77f tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:22.894781 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1b6e085c-9cc7-47a6-b2f3-dc0ff71ff77f tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:22.896348 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1b6e085c-9cc7-47a6-b2f3-dc0ff71ff77f tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:22.896348 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1b6e085c-9cc7-47a6-b2f3-dc0ff71ff77f tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:22.949798 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-1b6e085c-9cc7-47a6-b2f3-dc0ff71ff77f tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] 10.222.0.16 "POST /compute/v2.1/servers/5865f75f-6142-4885-9fa6-0687a2c169cc/action" status: 200 len: 24570 microversion: 2.1 time: 0.180585 Jul 16 21:45:22.950444 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2899/5799] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 21:45:22 2022] POST /compute/v2.1/servers/5865f75f-6142-4885-9fa6-0687a2c169cc/action => generated 24570 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Jul 16 21:45:23.085568 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-d29d5f3a-5ba9-42f9-b577-fdb6344967ee tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:45:23.090291 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-d29d5f3a-5ba9-42f9-b577-fdb6344967ee tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] [instance: f0c3fd0a-44d4-4209-9c2f-817382c0afa7] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:45:23.093362 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d29d5f3a-5ba9-42f9-b577-fdb6344967ee tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:23.093958 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d29d5f3a-5ba9-42f9-b577-fdb6344967ee tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:23.094958 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d29d5f3a-5ba9-42f9-b577-fdb6344967ee tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:23.095868 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d29d5f3a-5ba9-42f9-b577-fdb6344967ee tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:23.203800 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-d29d5f3a-5ba9-42f9-b577-fdb6344967ee tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/f0c3fd0a-44d4-4209-9c2f-817382c0afa7" status: 200 len: 1416 microversion: 2.1 time: 0.122210 Jul 16 21:45:23.204648 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-becdea0b-67f5-4de4-982f-8218f655873c tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:45:23.205296 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-becdea0b-67f5-4de4-982f-8218f655873c tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: 3618ceb8-9d10-4219-8885-d2d5004c71ad] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:45:23.215629 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-becdea0b-67f5-4de4-982f-8218f655873c tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:23.216619 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-becdea0b-67f5-4de4-982f-8218f655873c tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:23.217685 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-becdea0b-67f5-4de4-982f-8218f655873c tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:23.218868 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2901/5800] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:45:23 2022] GET /compute/v2.1/servers/f0c3fd0a-44d4-4209-9c2f-817382c0afa7 => generated 1416 bytes in 138 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:45:23.219485 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-becdea0b-67f5-4de4-982f-8218f655873c tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:23.309077 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-becdea0b-67f5-4de4-982f-8218f655873c tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:45:23.318481 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-becdea0b-67f5-4de4-982f-8218f655873c tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad" status: 200 len: 1295 microversion: 2.1 time: 0.120532 Jul 16 21:45:23.319553 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2900/5801] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:45:23 2022] GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad => generated 1295 bytes in 123 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:45:24.114236 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-eaf555ee-4a42-49ce-a3ae-b102c8bf3b60 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:45:24.128070 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-1040be8f-4f65-459d-9a81-d5e584d776db req-e217310c-f8ae-4038-b975-8937e3128090 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "5865f75f-6142-4885-9fa6-0687a2c169cc", "tag": "c297d034-2431-40d9-9e2d-8119f3b5331b"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:45:24.128225 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-eaf555ee-4a42-49ce-a3ae-b102c8bf3b60 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] [instance: 5865f75f-6142-4885-9fa6-0687a2c169cc] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:45:24.137998 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-1040be8f-4f65-459d-9a81-d5e584d776db req-e217310c-f8ae-4038-b975-8937e3128090 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:24.138758 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-1040be8f-4f65-459d-9a81-d5e584d776db req-e217310c-f8ae-4038-b975-8937e3128090 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:24.146984 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-eaf555ee-4a42-49ce-a3ae-b102c8bf3b60 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:24.149551 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-eaf555ee-4a42-49ce-a3ae-b102c8bf3b60 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:24.160633 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-1040be8f-4f65-459d-9a81-d5e584d776db req-e217310c-f8ae-4038-b975-8937e3128090 service nova] Creating event network-changed:c297d034-2431-40d9-9e2d-8119f3b5331b for instance 5865f75f-6142-4885-9fa6-0687a2c169cc on nch1-master-1 Jul 16 21:45:24.160965 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-eaf555ee-4a42-49ce-a3ae-b102c8bf3b60 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:24.160965 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-eaf555ee-4a42-49ce-a3ae-b102c8bf3b60 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:24.161129 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-1040be8f-4f65-459d-9a81-d5e584d776db req-e217310c-f8ae-4038-b975-8937e3128090 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.042809 Jul 16 21:45:24.161129 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2901/5802] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:45:24 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:45:24.207888 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-eaf555ee-4a42-49ce-a3ae-b102c8bf3b60 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] [instance: 5865f75f-6142-4885-9fa6-0687a2c169cc] Going to try to terminate instance {{(pid=138681) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 16 21:45:24.228525 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-9619f21c-9bda-4f60-90ed-da06318e728d tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:45:24.229609 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-9619f21c-9bda-4f60-90ed-da06318e728d tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] [instance: f0c3fd0a-44d4-4209-9c2f-817382c0afa7] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:45:24.246480 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9619f21c-9bda-4f60-90ed-da06318e728d tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:24.247099 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9619f21c-9bda-4f60-90ed-da06318e728d tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:24.248112 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9619f21c-9bda-4f60-90ed-da06318e728d tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:24.258813 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9619f21c-9bda-4f60-90ed-da06318e728d tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:24.598418 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-9619f21c-9bda-4f60-90ed-da06318e728d tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/f0c3fd0a-44d4-4209-9c2f-817382c0afa7" status: 200 len: 1464 microversion: 2.1 time: 0.378846 Jul 16 21:45:24.600067 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2902/5803] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:45:24 2022] GET /compute/v2.1/servers/f0c3fd0a-44d4-4209-9c2f-817382c0afa7 => generated 1464 bytes in 381 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:45:24.611401 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-cbbae771-2d81-4e2e-895d-1f6af47c4419 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:45:24.612994 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-cbbae771-2d81-4e2e-895d-1f6af47c4419 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: 3618ceb8-9d10-4219-8885-d2d5004c71ad] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:45:24.621239 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cbbae771-2d81-4e2e-895d-1f6af47c4419 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:24.621798 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cbbae771-2d81-4e2e-895d-1f6af47c4419 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:24.623059 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cbbae771-2d81-4e2e-895d-1f6af47c4419 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:24.623775 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cbbae771-2d81-4e2e-895d-1f6af47c4419 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:24.638102 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-eaf555ee-4a42-49ce-a3ae-b102c8bf3b60 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] 10.222.0.16 "DELETE /compute/v2.1/servers/5865f75f-6142-4885-9fa6-0687a2c169cc" status: 204 len: 0 microversion: 2.1 time: 0.530331 Jul 16 21:45:24.638730 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2902/5804] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:45:24 2022] DELETE /compute/v2.1/servers/5865f75f-6142-4885-9fa6-0687a2c169cc => generated 0 bytes in 532 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 16 21:45:24.645675 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-34ef0e71-dc55-4dcd-9550-540701866d0d tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Action: 'action', calling method: >, body: {"rebuild": {"imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9"}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:45:24.647356 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-34ef0e71-dc55-4dcd-9550-540701866d0d tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] [instance: 0b62eebf-c7ed-4fa1-a4d1-17947e5d8226] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:45:24.662728 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-34ef0e71-dc55-4dcd-9550-540701866d0d tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:24.667534 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-34ef0e71-dc55-4dcd-9550-540701866d0d tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:24.668469 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-34ef0e71-dc55-4dcd-9550-540701866d0d tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:24.668952 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-34ef0e71-dc55-4dcd-9550-540701866d0d tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:24.787803 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-34ef0e71-dc55-4dcd-9550-540701866d0d tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Flavor limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 16 21:45:24.787803 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-34ef0e71-dc55-4dcd-9550-540701866d0d tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Image limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 16 21:45:24.787803 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-34ef0e71-dc55-4dcd-9550-540701866d0d tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Flavor pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 16 21:45:24.787803 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-34ef0e71-dc55-4dcd-9550-540701866d0d tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Image pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 16 21:45:24.787803 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-34ef0e71-dc55-4dcd-9550-540701866d0d tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 16 21:45:24.863869 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-cbbae771-2d81-4e2e-895d-1f6af47c4419 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:45:24.877284 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-cbbae771-2d81-4e2e-895d-1f6af47c4419 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad" status: 200 len: 1362 microversion: 2.1 time: 0.275593 Jul 16 21:45:24.877876 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2903/5805] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:45:24 2022] GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad => generated 1362 bytes in 277 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:45:24.882571 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-7d469df8-3304-444e-a339-77b7c0d68bf3 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:45:24.883206 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-7d469df8-3304-444e-a339-77b7c0d68bf3 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] [instance: 5865f75f-6142-4885-9fa6-0687a2c169cc] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:45:24.890170 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7d469df8-3304-444e-a339-77b7c0d68bf3 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:24.890629 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7d469df8-3304-444e-a339-77b7c0d68bf3 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:24.891438 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7d469df8-3304-444e-a339-77b7c0d68bf3 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:24.891944 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7d469df8-3304-444e-a339-77b7c0d68bf3 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:25.052886 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-34ef0e71-dc55-4dcd-9550-540701866d0d tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] [instance: 0b62eebf-c7ed-4fa1-a4d1-17947e5d8226] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:45:25.065440 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-34ef0e71-dc55-4dcd-9550-540701866d0d tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:25.066038 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-34ef0e71-dc55-4dcd-9550-540701866d0d tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:25.067843 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-34ef0e71-dc55-4dcd-9550-540701866d0d tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:25.069152 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-34ef0e71-dc55-4dcd-9550-540701866d0d tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:25.081699 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-7d469df8-3304-444e-a339-77b7c0d68bf3 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '62209b9802ab46c3b3d094d4d1ad86fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b533f85aeddf4d8db05f2c53a61befc3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:45:25.096118 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-7d469df8-3304-444e-a339-77b7c0d68bf3 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] 10.222.0.16 "GET /compute/v2.1/servers/5865f75f-6142-4885-9fa6-0687a2c169cc" status: 200 len: 1577 microversion: 2.1 time: 0.216177 Jul 16 21:45:25.096848 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2904/5806] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:45:24 2022] GET /compute/v2.1/servers/5865f75f-6142-4885-9fa6-0687a2c169cc => generated 1577 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:45:25.125143 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-34ef0e71-dc55-4dcd-9550-540701866d0d tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] 10.222.0.16 "POST /compute/v2.1/servers/0b62eebf-c7ed-4fa1-a4d1-17947e5d8226/action" status: 202 len: 1137 microversion: 2.1 time: 0.484622 Jul 16 21:45:25.126515 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2903/5807] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 21:45:24 2022] POST /compute/v2.1/servers/0b62eebf-c7ed-4fa1-a4d1-17947e5d8226/action => generated 1137 bytes in 487 msecs (HTTP/1.1 202) 10 headers in 452 bytes (1 switches on core 0) Jul 16 21:45:25.144308 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-1760aacc-ef01-4e3c-8b0f-537d2b1427c9 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:45:25.145358 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-1760aacc-ef01-4e3c-8b0f-537d2b1427c9 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] [instance: 0b62eebf-c7ed-4fa1-a4d1-17947e5d8226] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:45:25.152798 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1760aacc-ef01-4e3c-8b0f-537d2b1427c9 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:25.153678 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1760aacc-ef01-4e3c-8b0f-537d2b1427c9 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:25.154693 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1760aacc-ef01-4e3c-8b0f-537d2b1427c9 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:25.155450 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1760aacc-ef01-4e3c-8b0f-537d2b1427c9 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:25.368401 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-1760aacc-ef01-4e3c-8b0f-537d2b1427c9 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '833136cbdeab4b9e8b6dd2d24c7c8753', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '77c9b7e351e24dc88207e9498f75f578', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:45:25.377040 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-1760aacc-ef01-4e3c-8b0f-537d2b1427c9 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] 10.222.0.16 "GET /compute/v2.1/servers/0b62eebf-c7ed-4fa1-a4d1-17947e5d8226" status: 200 len: 1690 microversion: 2.1 time: 0.237608 Jul 16 21:45:25.377804 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2905/5808] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:45:25 2022] GET /compute/v2.1/servers/0b62eebf-c7ed-4fa1-a4d1-17947e5d8226 => generated 1690 bytes in 239 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:45:25.457416 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-4dde5c4b-82ea-45e7-a16c-d2b2959b18e6 req-63a7d236-6a93-417e-9309-abb769ac2af3 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "3618ceb8-9d10-4219-8885-d2d5004c71ad", "tag": "ef163361-e10a-4408-b519-5d053d2a55d6"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:45:25.473705 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-4dde5c4b-82ea-45e7-a16c-d2b2959b18e6 req-63a7d236-6a93-417e-9309-abb769ac2af3 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:25.474084 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-4dde5c4b-82ea-45e7-a16c-d2b2959b18e6 req-63a7d236-6a93-417e-9309-abb769ac2af3 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:25.487206 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-4dde5c4b-82ea-45e7-a16c-d2b2959b18e6 req-63a7d236-6a93-417e-9309-abb769ac2af3 service nova] Creating event network-changed:ef163361-e10a-4408-b519-5d053d2a55d6 for instance 3618ceb8-9d10-4219-8885-d2d5004c71ad on nch1-master-1 Jul 16 21:45:25.498594 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-4dde5c4b-82ea-45e7-a16c-d2b2959b18e6 req-63a7d236-6a93-417e-9309-abb769ac2af3 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.046066 Jul 16 21:45:25.499756 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2904/5809] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:45:25 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:45:25.617353 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-e4810ffa-9516-412b-a172-7ac85ce84f10 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:45:25.617985 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-e4810ffa-9516-412b-a172-7ac85ce84f10 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] [instance: f0c3fd0a-44d4-4209-9c2f-817382c0afa7] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:45:25.625120 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e4810ffa-9516-412b-a172-7ac85ce84f10 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:25.625593 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e4810ffa-9516-412b-a172-7ac85ce84f10 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:25.627050 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e4810ffa-9516-412b-a172-7ac85ce84f10 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:25.627696 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e4810ffa-9516-412b-a172-7ac85ce84f10 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:25.897832 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-80d004fb-7cce-4e9f-9ac1-0b1459140489 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:45:25.899151 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-80d004fb-7cce-4e9f-9ac1-0b1459140489 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: 3618ceb8-9d10-4219-8885-d2d5004c71ad] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:45:25.906119 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-e4810ffa-9516-412b-a172-7ac85ce84f10 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/f0c3fd0a-44d4-4209-9c2f-817382c0afa7" status: 200 len: 1464 microversion: 2.1 time: 0.292637 Jul 16 21:45:25.920008 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2906/5810] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:45:25 2022] GET /compute/v2.1/servers/f0c3fd0a-44d4-4209-9c2f-817382c0afa7 => generated 1464 bytes in 307 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:45:25.929053 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-80d004fb-7cce-4e9f-9ac1-0b1459140489 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:25.929529 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-80d004fb-7cce-4e9f-9ac1-0b1459140489 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:25.930342 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-80d004fb-7cce-4e9f-9ac1-0b1459140489 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:25.930834 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-80d004fb-7cce-4e9f-9ac1-0b1459140489 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:26.114849 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-b64f7dc2-5460-49cc-8fca-a668fe000dc3 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:45:26.115555 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-b64f7dc2-5460-49cc-8fca-a668fe000dc3 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] [instance: 5865f75f-6142-4885-9fa6-0687a2c169cc] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:45:26.131047 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b64f7dc2-5460-49cc-8fca-a668fe000dc3 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:26.133725 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b64f7dc2-5460-49cc-8fca-a668fe000dc3 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:26.133725 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b64f7dc2-5460-49cc-8fca-a668fe000dc3 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:26.133725 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b64f7dc2-5460-49cc-8fca-a668fe000dc3 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:26.167859 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-80d004fb-7cce-4e9f-9ac1-0b1459140489 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:45:26.178125 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-80d004fb-7cce-4e9f-9ac1-0b1459140489 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad" status: 200 len: 1362 microversion: 2.1 time: 0.286915 Jul 16 21:45:26.183428 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2905/5811] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:45:25 2022] GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad => generated 1362 bytes in 294 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:45:26.329691 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-b64f7dc2-5460-49cc-8fca-a668fe000dc3 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '62209b9802ab46c3b3d094d4d1ad86fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b533f85aeddf4d8db05f2c53a61befc3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:45:26.337677 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-b64f7dc2-5460-49cc-8fca-a668fe000dc3 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] 10.222.0.16 "GET /compute/v2.1/servers/5865f75f-6142-4885-9fa6-0687a2c169cc" status: 200 len: 1577 microversion: 2.1 time: 0.227678 Jul 16 21:45:26.338277 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2907/5812] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:45:26 2022] GET /compute/v2.1/servers/5865f75f-6142-4885-9fa6-0687a2c169cc => generated 1577 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:45:26.394537 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-7e09379e-c545-44ff-9045-239efba2981d tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:45:26.395275 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-7e09379e-c545-44ff-9045-239efba2981d tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] [instance: 0b62eebf-c7ed-4fa1-a4d1-17947e5d8226] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:45:26.401540 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7e09379e-c545-44ff-9045-239efba2981d tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:26.402080 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7e09379e-c545-44ff-9045-239efba2981d tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:26.402962 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7e09379e-c545-44ff-9045-239efba2981d tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:26.403515 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7e09379e-c545-44ff-9045-239efba2981d tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:26.650892 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-7e09379e-c545-44ff-9045-239efba2981d tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '833136cbdeab4b9e8b6dd2d24c7c8753', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '77c9b7e351e24dc88207e9498f75f578', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:45:26.664000 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-7e09379e-c545-44ff-9045-239efba2981d tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] 10.222.0.16 "GET /compute/v2.1/servers/0b62eebf-c7ed-4fa1-a4d1-17947e5d8226" status: 200 len: 1690 microversion: 2.1 time: 0.274083 Jul 16 21:45:26.676270 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2906/5813] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:45:26 2022] GET /compute/v2.1/servers/0b62eebf-c7ed-4fa1-a4d1-17947e5d8226 => generated 1690 bytes in 287 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:45:26.928324 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-8ae42d2c-656b-4c7a-8a84-09f41a14bd8a tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:45:26.932612 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-8ae42d2c-656b-4c7a-8a84-09f41a14bd8a tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] [instance: f0c3fd0a-44d4-4209-9c2f-817382c0afa7] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:45:26.951306 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8ae42d2c-656b-4c7a-8a84-09f41a14bd8a tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:26.951978 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8ae42d2c-656b-4c7a-8a84-09f41a14bd8a tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:26.953643 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8ae42d2c-656b-4c7a-8a84-09f41a14bd8a tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:26.954354 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8ae42d2c-656b-4c7a-8a84-09f41a14bd8a tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:27.089692 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=100 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:45:27.097667 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-8ae42d2c-656b-4c7a-8a84-09f41a14bd8a tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/f0c3fd0a-44d4-4209-9c2f-817382c0afa7" status: 200 len: 1464 microversion: 2.1 time: 0.176302 Jul 16 21:45:27.098483 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2908/5814] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:45:26 2022] GET /compute/v2.1/servers/f0c3fd0a-44d4-4209-9c2f-817382c0afa7 => generated 1464 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:45:27.200476 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-a47d5d2d-04ac-4b25-ab2b-9b0c265864cf tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:45:27.201121 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-a47d5d2d-04ac-4b25-ab2b-9b0c265864cf tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: 3618ceb8-9d10-4219-8885-d2d5004c71ad] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:45:27.208760 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a47d5d2d-04ac-4b25-ab2b-9b0c265864cf tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:27.209236 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a47d5d2d-04ac-4b25-ab2b-9b0c265864cf tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:27.210049 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a47d5d2d-04ac-4b25-ab2b-9b0c265864cf tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:27.210710 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a47d5d2d-04ac-4b25-ab2b-9b0c265864cf tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:27.292746 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-0cec372d-440a-45d7-a8e2-6eb32b6ee189 req-e57661ee-7499-498c-b308-0ddaf3aeebe1 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "5865f75f-6142-4885-9fa6-0687a2c169cc", "name": "network-vif-deleted", "tag": "c297d034-2431-40d9-9e2d-8119f3b5331b"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:45:27.303105 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-0cec372d-440a-45d7-a8e2-6eb32b6ee189 req-e57661ee-7499-498c-b308-0ddaf3aeebe1 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:27.303568 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-0cec372d-440a-45d7-a8e2-6eb32b6ee189 req-e57661ee-7499-498c-b308-0ddaf3aeebe1 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:27.315652 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-0cec372d-440a-45d7-a8e2-6eb32b6ee189 req-e57661ee-7499-498c-b308-0ddaf3aeebe1 service nova] Creating event network-vif-deleted:c297d034-2431-40d9-9e2d-8119f3b5331b for instance 5865f75f-6142-4885-9fa6-0687a2c169cc on nch1-master-1 Jul 16 21:45:27.322725 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-0cec372d-440a-45d7-a8e2-6eb32b6ee189 req-e57661ee-7499-498c-b308-0ddaf3aeebe1 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.040239 Jul 16 21:45:27.323391 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2909/5815] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:45:27 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 16 21:45:27.356576 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-56dcc6e0-b7a7-4c3b-81ca-03a40f3c857b tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:45:27.357420 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-56dcc6e0-b7a7-4c3b-81ca-03a40f3c857b tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] [instance: 5865f75f-6142-4885-9fa6-0687a2c169cc] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:45:27.365367 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-56dcc6e0-b7a7-4c3b-81ca-03a40f3c857b tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:27.365923 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-56dcc6e0-b7a7-4c3b-81ca-03a40f3c857b tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:27.367310 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-56dcc6e0-b7a7-4c3b-81ca-03a40f3c857b tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:27.367993 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-56dcc6e0-b7a7-4c3b-81ca-03a40f3c857b tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:27.404598 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-a47d5d2d-04ac-4b25-ab2b-9b0c265864cf tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:45:27.422495 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-a47d5d2d-04ac-4b25-ab2b-9b0c265864cf tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad" status: 200 len: 1362 microversion: 2.1 time: 0.216761 Jul 16 21:45:27.422495 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2907/5816] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:45:27 2022] GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad => generated 1362 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:45:27.437704 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-56dcc6e0-b7a7-4c3b-81ca-03a40f3c857b tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '62209b9802ab46c3b3d094d4d1ad86fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b533f85aeddf4d8db05f2c53a61befc3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:45:27.447036 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-56dcc6e0-b7a7-4c3b-81ca-03a40f3c857b tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] 10.222.0.16 "GET /compute/v2.1/servers/5865f75f-6142-4885-9fa6-0687a2c169cc" status: 200 len: 1392 microversion: 2.1 time: 0.095153 Jul 16 21:45:27.447913 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2910/5817] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:45:27 2022] GET /compute/v2.1/servers/5865f75f-6142-4885-9fa6-0687a2c169cc => generated 1392 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:45:27.519849 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-8fe51bd3-3015-4a45-8c15-376ab7eada1f req-8d23afcc-0b0e-4cb1-a717-9761bcd6f250 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "f0c3fd0a-44d4-4209-9c2f-817382c0afa7", "tag": "d082e13f-d028-4bc6-9b63-7bea91b4befb"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:45:27.528842 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-8fe51bd3-3015-4a45-8c15-376ab7eada1f req-8d23afcc-0b0e-4cb1-a717-9761bcd6f250 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:27.529306 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-8fe51bd3-3015-4a45-8c15-376ab7eada1f req-8d23afcc-0b0e-4cb1-a717-9761bcd6f250 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:27.541097 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-8fe51bd3-3015-4a45-8c15-376ab7eada1f req-8d23afcc-0b0e-4cb1-a717-9761bcd6f250 service nova] Creating event network-changed:d082e13f-d028-4bc6-9b63-7bea91b4befb for instance f0c3fd0a-44d4-4209-9c2f-817382c0afa7 on nch1-master-1 Jul 16 21:45:27.545799 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-8fe51bd3-3015-4a45-8c15-376ab7eada1f req-8d23afcc-0b0e-4cb1-a717-9761bcd6f250 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.030628 Jul 16 21:45:27.546379 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2908/5818] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:45:27 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:45:27.681967 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-0fb5d04e-33cf-4ed3-9287-6e6490801080 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:45:27.682495 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-0fb5d04e-33cf-4ed3-9287-6e6490801080 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] [instance: 0b62eebf-c7ed-4fa1-a4d1-17947e5d8226] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:45:27.684536 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_api:SELECT=97,nova_api:INSERT=3 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:45:27.689951 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0fb5d04e-33cf-4ed3-9287-6e6490801080 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:27.690446 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0fb5d04e-33cf-4ed3-9287-6e6490801080 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:27.691224 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0fb5d04e-33cf-4ed3-9287-6e6490801080 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:27.691664 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0fb5d04e-33cf-4ed3-9287-6e6490801080 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:27.832443 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-0fb5d04e-33cf-4ed3-9287-6e6490801080 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '833136cbdeab4b9e8b6dd2d24c7c8753', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '77c9b7e351e24dc88207e9498f75f578', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:45:27.840163 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-0fb5d04e-33cf-4ed3-9287-6e6490801080 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] 10.222.0.16 "GET /compute/v2.1/servers/0b62eebf-c7ed-4fa1-a4d1-17947e5d8226" status: 200 len: 1690 microversion: 2.1 time: 0.162183 Jul 16 21:45:27.840745 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2911/5819] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:45:27 2022] GET /compute/v2.1/servers/0b62eebf-c7ed-4fa1-a4d1-17947e5d8226 => generated 1690 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:45:28.116405 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-fa3a349d-98a2-4236-812a-27352cd0ffe5 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:45:28.117279 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-fa3a349d-98a2-4236-812a-27352cd0ffe5 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] [instance: f0c3fd0a-44d4-4209-9c2f-817382c0afa7] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:45:28.123818 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fa3a349d-98a2-4236-812a-27352cd0ffe5 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:28.124273 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fa3a349d-98a2-4236-812a-27352cd0ffe5 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:28.125064 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fa3a349d-98a2-4236-812a-27352cd0ffe5 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:28.125506 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fa3a349d-98a2-4236-812a-27352cd0ffe5 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:28.352513 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=96,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:45:28.357402 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-fa3a349d-98a2-4236-812a-27352cd0ffe5 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/f0c3fd0a-44d4-4209-9c2f-817382c0afa7" status: 200 len: 1464 microversion: 2.1 time: 0.245316 Jul 16 21:45:28.357981 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2909/5820] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:45:28 2022] GET /compute/v2.1/servers/f0c3fd0a-44d4-4209-9c2f-817382c0afa7 => generated 1464 bytes in 247 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:45:28.430731 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-d9073211-a9d8-4a3f-9d07-8026ccb20c08 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:45:28.431460 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-d9073211-a9d8-4a3f-9d07-8026ccb20c08 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: 3618ceb8-9d10-4219-8885-d2d5004c71ad] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:45:28.438601 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d9073211-a9d8-4a3f-9d07-8026ccb20c08 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:28.439462 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d9073211-a9d8-4a3f-9d07-8026ccb20c08 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:28.440420 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d9073211-a9d8-4a3f-9d07-8026ccb20c08 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:28.440930 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d9073211-a9d8-4a3f-9d07-8026ccb20c08 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:28.466774 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-28355228-4e30-4351-8910-ea2b80068a89 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:45:28.467816 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-28355228-4e30-4351-8910-ea2b80068a89 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] [instance: 5865f75f-6142-4885-9fa6-0687a2c169cc] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:45:28.475482 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-28355228-4e30-4351-8910-ea2b80068a89 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:28.476078 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-28355228-4e30-4351-8910-ea2b80068a89 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:28.476896 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-28355228-4e30-4351-8910-ea2b80068a89 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:28.477376 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-28355228-4e30-4351-8910-ea2b80068a89 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:28.610145 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-28355228-4e30-4351-8910-ea2b80068a89 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '62209b9802ab46c3b3d094d4d1ad86fe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b533f85aeddf4d8db05f2c53a61befc3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:45:28.618681 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-28355228-4e30-4351-8910-ea2b80068a89 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] 10.222.0.16 "GET /compute/v2.1/servers/5865f75f-6142-4885-9fa6-0687a2c169cc" status: 200 len: 1226 microversion: 2.1 time: 0.157401 Jul 16 21:45:28.619290 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2910/5821] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:45:28 2022] GET /compute/v2.1/servers/5865f75f-6142-4885-9fa6-0687a2c169cc => generated 1226 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:45:28.712900 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-d9073211-a9d8-4a3f-9d07-8026ccb20c08 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:45:28.722454 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-d9073211-a9d8-4a3f-9d07-8026ccb20c08 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad" status: 200 len: 1362 microversion: 2.1 time: 0.294902 Jul 16 21:45:28.722454 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2912/5822] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:45:28 2022] GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad => generated 1362 bytes in 296 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:45:28.857986 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-b884cf8b-a657-4b30-8fab-1060c56fe669 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:45:28.858542 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-b884cf8b-a657-4b30-8fab-1060c56fe669 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] [instance: 0b62eebf-c7ed-4fa1-a4d1-17947e5d8226] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:45:28.866772 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b884cf8b-a657-4b30-8fab-1060c56fe669 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:28.867371 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b884cf8b-a657-4b30-8fab-1060c56fe669 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:28.875892 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b884cf8b-a657-4b30-8fab-1060c56fe669 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:28.875892 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b884cf8b-a657-4b30-8fab-1060c56fe669 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:29.137623 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-b884cf8b-a657-4b30-8fab-1060c56fe669 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '833136cbdeab4b9e8b6dd2d24c7c8753', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '77c9b7e351e24dc88207e9498f75f578', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:45:29.151830 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-b884cf8b-a657-4b30-8fab-1060c56fe669 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] 10.222.0.16 "GET /compute/v2.1/servers/0b62eebf-c7ed-4fa1-a4d1-17947e5d8226" status: 200 len: 1690 microversion: 2.1 time: 0.297969 Jul 16 21:45:29.161994 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2911/5823] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:45:28 2022] GET /compute/v2.1/servers/0b62eebf-c7ed-4fa1-a4d1-17947e5d8226 => generated 1690 bytes in 309 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:45:29.376249 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-4be62f88-e82a-4d5d-b6de-1f732e256300 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:45:29.377487 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-4be62f88-e82a-4d5d-b6de-1f732e256300 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] [instance: f0c3fd0a-44d4-4209-9c2f-817382c0afa7] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:45:29.384032 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4be62f88-e82a-4d5d-b6de-1f732e256300 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:29.384488 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4be62f88-e82a-4d5d-b6de-1f732e256300 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:29.385328 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4be62f88-e82a-4d5d-b6de-1f732e256300 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:29.385774 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4be62f88-e82a-4d5d-b6de-1f732e256300 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:29.517296 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-4be62f88-e82a-4d5d-b6de-1f732e256300 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/f0c3fd0a-44d4-4209-9c2f-817382c0afa7" status: 200 len: 1464 microversion: 2.1 time: 0.146083 Jul 16 21:45:29.518623 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2913/5824] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:45:29 2022] GET /compute/v2.1/servers/f0c3fd0a-44d4-4209-9c2f-817382c0afa7 => generated 1464 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:45:29.636577 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-171c5fba-21c2-4138-b947-05df01223970 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:45:29.637114 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-171c5fba-21c2-4138-b947-05df01223970 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] [instance: 5865f75f-6142-4885-9fa6-0687a2c169cc] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:45:29.644513 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-171c5fba-21c2-4138-b947-05df01223970 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:29.644972 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-171c5fba-21c2-4138-b947-05df01223970 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:29.645744 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-171c5fba-21c2-4138-b947-05df01223970 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:29.646193 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-171c5fba-21c2-4138-b947-05df01223970 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:29.660166 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-171c5fba-21c2-4138-b947-05df01223970 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] HTTP exception thrown: Instance 5865f75f-6142-4885-9fa6-0687a2c169cc could not be found. Jul 16 21:45:29.661824 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-171c5fba-21c2-4138-b947-05df01223970 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Request method failure captured: Jul 16 21:45:29.661824 ncdv-master-1 devstack@n-api.service[138681]: request: GET /compute/v2.1/servers/5865f75f-6142-4885-9fa6-0687a2c169cc HTTP/1.1 Jul 16 21:45:29.661824 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:45:29.661824 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:45:29.661824 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:45:29.661824 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 21:45:29.661824 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:45:29.661824 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:45:29.661824 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:45:29.661824 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:45:29.661824 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0y_WdH7eI0DbXzeyrxNR2zaPLU2Z0T2hgcgJaNL5zKj0IOkua_mwRXGhv8UgaywTvIB56iq06rxK294Knh0aCrbToiJY0kPn8yPuGVzddBFopXnN-hL_u7FbpY8zvcJRiL0zpWIKlPhQExzLv95jQK7KF6D9Y-KawL2vbM6jbQCfNnU Jul 16 21:45:29.661824 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:45:29.661824 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:45:29.661824 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:45:29.661824 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:45:29.661824 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:45:29.661824 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:45:29.661824 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: b533f85aeddf4d8db05f2c53a61befc3 Jul 16 21:45:29.661824 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-TestVolumeBootPattern-1625426921 Jul 16 21:45:29.661824 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:45:29.661824 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:45:29.661824 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/b533f85aeddf4d8db05f2c53a61befc3"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/b533f85aeddf4d8db05f2c53a61befc3"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_b533f85aeddf4d8db05f2c53a61befc3"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/b533f85aeddf4d8db05f2c53a61befc3"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:45:29.661824 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-TestVolumeBootPattern-1625426921 Jul 16 21:45:29.661824 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: b533f85aeddf4d8db05f2c53a61befc3 Jul 16 21:45:29.661824 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-TestVolumeBootPattern-1625426921 Jul 16 21:45:29.661824 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-TestVolumeBootPattern-1625426921-project Jul 16 21:45:29.661824 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:45:29.661824 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:45:29.661824 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: 62209b9802ab46c3b3d094d4d1ad86fe Jul 16 21:45:29.661824 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-TestVolumeBootPattern-1625426921-project Jul 16 21:45:29.661824 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:45:29.661824 ncdv-master-1 devstack@n-api.service[138681]: exception: Instance 5865f75f-6142-4885-9fa6-0687a2c169cc could not be found. Jul 16 21:45:29.661824 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': '5865f75f-6142-4885-9fa6-0687a2c169cc'} Jul 16 21:45:29.661824 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:45:29.661824 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:45:29.661824 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:45:29.661824 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:45:29.661824 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:45:29.661824 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:45:29.661824 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:45:29.661824 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:45:29.661824 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:45:29.661824 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:45:29.661824 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 5865f75f-6142-4885-9fa6-0687a2c169cc could not be found. Jul 16 21:45:29.661824 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:45:29.661824 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:45:29.661824 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:45:29.661824 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:45:29.661824 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:45:29.661824 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:45:29.661824 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:45:29.661824 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:45:29.661824 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:45:29.661824 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:45:29.661824 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:45:29.661824 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:45:29.661824 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:45:29.661824 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:45:29.661824 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:45:29.661824 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:45:29.661824 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 5865f75f-6142-4885-9fa6-0687a2c169cc could not be found. Jul 16 21:45:29.661824 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:45:29.661824 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:45:29.661824 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:45:29.667389 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:45:29.667389 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:45:29.667389 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:45:29.667389 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:45:29.667389 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:45:29.667389 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 5865f75f-6142-4885-9fa6-0687a2c169cc could not be found. Jul 16 21:45:29.667389 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:45:29.667389 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-171c5fba-21c2-4138-b947-05df01223970 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Returning 404 to user: Instance 5865f75f-6142-4885-9fa6-0687a2c169cc could not be found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:45:29.667389 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-171c5fba-21c2-4138-b947-05df01223970 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] 10.222.0.16 "GET /compute/v2.1/servers/5865f75f-6142-4885-9fa6-0687a2c169cc" status: 404 len: 111 microversion: 2.1 time: 0.030756 Jul 16 21:45:29.667389 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2912/5825] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:45:29 2022] GET /compute/v2.1/servers/5865f75f-6142-4885-9fa6-0687a2c169cc => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:45:29.740113 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-3ce77f44-8455-4bea-9c16-255e7fa364b1 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:45:29.740691 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-3ce77f44-8455-4bea-9c16-255e7fa364b1 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: 3618ceb8-9d10-4219-8885-d2d5004c71ad] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:45:29.747136 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3ce77f44-8455-4bea-9c16-255e7fa364b1 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:29.747587 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3ce77f44-8455-4bea-9c16-255e7fa364b1 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:29.748365 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3ce77f44-8455-4bea-9c16-255e7fa364b1 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:29.748804 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3ce77f44-8455-4bea-9c16-255e7fa364b1 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:29.883405 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-3ce77f44-8455-4bea-9c16-255e7fa364b1 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:45:29.894180 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-3ce77f44-8455-4bea-9c16-255e7fa364b1 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad" status: 200 len: 1362 microversion: 2.1 time: 0.158758 Jul 16 21:45:29.894995 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2914/5826] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:45:29 2022] GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad => generated 1362 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:45:30.169629 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-5e51170d-6575-4ab6-95f8-bbb6d0352a64 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:45:30.170186 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-5e51170d-6575-4ab6-95f8-bbb6d0352a64 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] [instance: 0b62eebf-c7ed-4fa1-a4d1-17947e5d8226] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:45:30.177767 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5e51170d-6575-4ab6-95f8-bbb6d0352a64 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:30.178352 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5e51170d-6575-4ab6-95f8-bbb6d0352a64 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:30.179129 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5e51170d-6575-4ab6-95f8-bbb6d0352a64 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:30.179584 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5e51170d-6575-4ab6-95f8-bbb6d0352a64 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:30.324100 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-5e51170d-6575-4ab6-95f8-bbb6d0352a64 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '833136cbdeab4b9e8b6dd2d24c7c8753', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '77c9b7e351e24dc88207e9498f75f578', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:45:30.334283 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-5e51170d-6575-4ab6-95f8-bbb6d0352a64 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] 10.222.0.16 "GET /compute/v2.1/servers/0b62eebf-c7ed-4fa1-a4d1-17947e5d8226" status: 200 len: 1690 microversion: 2.1 time: 0.168683 Jul 16 21:45:30.334977 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2913/5827] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:45:30 2022] GET /compute/v2.1/servers/0b62eebf-c7ed-4fa1-a4d1-17947e5d8226 => generated 1690 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:45:30.537328 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-3434c537-190a-418c-babf-df346baa4b25 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:45:30.538205 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-3434c537-190a-418c-babf-df346baa4b25 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] [instance: f0c3fd0a-44d4-4209-9c2f-817382c0afa7] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:45:30.545285 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3434c537-190a-418c-babf-df346baa4b25 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:30.545746 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3434c537-190a-418c-babf-df346baa4b25 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:30.546545 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3434c537-190a-418c-babf-df346baa4b25 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:30.546971 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3434c537-190a-418c-babf-df346baa4b25 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:30.682325 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-3434c537-190a-418c-babf-df346baa4b25 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/f0c3fd0a-44d4-4209-9c2f-817382c0afa7" status: 200 len: 1464 microversion: 2.1 time: 0.150573 Jul 16 21:45:30.682948 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2915/5828] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:45:30 2022] GET /compute/v2.1/servers/f0c3fd0a-44d4-4209-9c2f-817382c0afa7 => generated 1464 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:45:30.911804 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-a8361c6d-5156-43e0-9993-95a539b52be9 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:45:30.913755 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-a8361c6d-5156-43e0-9993-95a539b52be9 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: 3618ceb8-9d10-4219-8885-d2d5004c71ad] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:45:30.915429 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell0:SELECT=4 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:45:30.920402 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a8361c6d-5156-43e0-9993-95a539b52be9 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:30.920861 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a8361c6d-5156-43e0-9993-95a539b52be9 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:30.921686 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a8361c6d-5156-43e0-9993-95a539b52be9 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:30.922086 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a8361c6d-5156-43e0-9993-95a539b52be9 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:31.073190 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-a8361c6d-5156-43e0-9993-95a539b52be9 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:45:31.075614 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell0:SELECT=20 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:45:31.082207 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-a8361c6d-5156-43e0-9993-95a539b52be9 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad" status: 200 len: 1362 microversion: 2.1 time: 0.174598 Jul 16 21:45:31.083017 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2914/5829] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:45:30 2022] GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad => generated 1362 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:45:31.352976 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-4decbd99-d87b-45cc-8a1c-6ee55d3db66b tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:45:31.353579 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-4decbd99-d87b-45cc-8a1c-6ee55d3db66b tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] [instance: 0b62eebf-c7ed-4fa1-a4d1-17947e5d8226] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:45:31.356174 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell0:SELECT=2 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:45:31.361056 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4decbd99-d87b-45cc-8a1c-6ee55d3db66b tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:31.362537 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4decbd99-d87b-45cc-8a1c-6ee55d3db66b tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:31.363338 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4decbd99-d87b-45cc-8a1c-6ee55d3db66b tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:31.363779 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4decbd99-d87b-45cc-8a1c-6ee55d3db66b tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:31.516139 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-4decbd99-d87b-45cc-8a1c-6ee55d3db66b tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '833136cbdeab4b9e8b6dd2d24c7c8753', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '77c9b7e351e24dc88207e9498f75f578', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:45:31.525728 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-4decbd99-d87b-45cc-8a1c-6ee55d3db66b tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] 10.222.0.16 "GET /compute/v2.1/servers/0b62eebf-c7ed-4fa1-a4d1-17947e5d8226" status: 200 len: 1690 microversion: 2.1 time: 0.176293 Jul 16 21:45:31.526523 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2916/5830] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:45:31 2022] GET /compute/v2.1/servers/0b62eebf-c7ed-4fa1-a4d1-17947e5d8226 => generated 1690 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:45:31.699532 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-fd3489a2-01d1-4ab4-ab7d-c3e3b10f95cf tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:45:31.700164 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-fd3489a2-01d1-4ab4-ab7d-c3e3b10f95cf tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] [instance: f0c3fd0a-44d4-4209-9c2f-817382c0afa7] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:45:31.707531 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fd3489a2-01d1-4ab4-ab7d-c3e3b10f95cf tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:31.709272 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fd3489a2-01d1-4ab4-ab7d-c3e3b10f95cf tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:31.709272 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fd3489a2-01d1-4ab4-ab7d-c3e3b10f95cf tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:31.709272 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fd3489a2-01d1-4ab4-ab7d-c3e3b10f95cf tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:31.841802 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-fd3489a2-01d1-4ab4-ab7d-c3e3b10f95cf tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/f0c3fd0a-44d4-4209-9c2f-817382c0afa7" status: 200 len: 1464 microversion: 2.1 time: 0.145991 Jul 16 21:45:31.842382 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2915/5831] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:45:31 2022] GET /compute/v2.1/servers/f0c3fd0a-44d4-4209-9c2f-817382c0afa7 => generated 1464 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:45:32.098453 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-267713ce-19b9-4410-a412-e3cf0f93a9a6 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:45:32.099835 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-267713ce-19b9-4410-a412-e3cf0f93a9a6 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: 3618ceb8-9d10-4219-8885-d2d5004c71ad] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:45:32.107439 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-267713ce-19b9-4410-a412-e3cf0f93a9a6 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:32.108806 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-267713ce-19b9-4410-a412-e3cf0f93a9a6 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:32.108806 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-267713ce-19b9-4410-a412-e3cf0f93a9a6 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:32.109337 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-267713ce-19b9-4410-a412-e3cf0f93a9a6 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:32.249059 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-267713ce-19b9-4410-a412-e3cf0f93a9a6 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:45:32.256837 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-267713ce-19b9-4410-a412-e3cf0f93a9a6 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad" status: 200 len: 1362 microversion: 2.1 time: 0.162461 Jul 16 21:45:32.257418 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2917/5832] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:45:32 2022] GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad => generated 1362 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:45:32.543112 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-f032bbfb-be84-4a05-876e-e5744ccc7053 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:45:32.544480 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-f032bbfb-be84-4a05-876e-e5744ccc7053 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] [instance: 0b62eebf-c7ed-4fa1-a4d1-17947e5d8226] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:45:32.553317 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f032bbfb-be84-4a05-876e-e5744ccc7053 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:32.560894 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f032bbfb-be84-4a05-876e-e5744ccc7053 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:32.560894 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f032bbfb-be84-4a05-876e-e5744ccc7053 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:32.560894 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f032bbfb-be84-4a05-876e-e5744ccc7053 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:32.696973 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-f032bbfb-be84-4a05-876e-e5744ccc7053 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '833136cbdeab4b9e8b6dd2d24c7c8753', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '77c9b7e351e24dc88207e9498f75f578', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:45:32.705501 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-f032bbfb-be84-4a05-876e-e5744ccc7053 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] 10.222.0.16 "GET /compute/v2.1/servers/0b62eebf-c7ed-4fa1-a4d1-17947e5d8226" status: 200 len: 1690 microversion: 2.1 time: 0.165749 Jul 16 21:45:32.706151 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2916/5833] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:45:32 2022] GET /compute/v2.1/servers/0b62eebf-c7ed-4fa1-a4d1-17947e5d8226 => generated 1690 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:45:32.707225 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-a27445c2-a2d0-45ec-8abc-cfae02240887 req-85198474-d620-44a4-aa80-e6956379927c service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "3618ceb8-9d10-4219-8885-d2d5004c71ad", "name": "network-vif-plugged", "status": "completed", "tag": "ef163361-e10a-4408-b519-5d053d2a55d6"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:45:32.716878 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-a27445c2-a2d0-45ec-8abc-cfae02240887 req-85198474-d620-44a4-aa80-e6956379927c service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:32.717366 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-a27445c2-a2d0-45ec-8abc-cfae02240887 req-85198474-d620-44a4-aa80-e6956379927c service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:32.731947 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-a27445c2-a2d0-45ec-8abc-cfae02240887 req-85198474-d620-44a4-aa80-e6956379927c service nova] Creating event network-vif-plugged:ef163361-e10a-4408-b519-5d053d2a55d6 for instance 3618ceb8-9d10-4219-8885-d2d5004c71ad on nch1-master-1 Jul 16 21:45:32.737909 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-a27445c2-a2d0-45ec-8abc-cfae02240887 req-85198474-d620-44a4-aa80-e6956379927c service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.034981 Jul 16 21:45:32.745273 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2918/5834] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:45:32 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 16 21:45:32.860170 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-efa547a4-5322-425c-a267-b9517c5f5738 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:45:32.860820 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-efa547a4-5322-425c-a267-b9517c5f5738 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] [instance: f0c3fd0a-44d4-4209-9c2f-817382c0afa7] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:45:32.867702 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-efa547a4-5322-425c-a267-b9517c5f5738 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:32.868484 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-efa547a4-5322-425c-a267-b9517c5f5738 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:32.870265 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-efa547a4-5322-425c-a267-b9517c5f5738 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:32.870891 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-efa547a4-5322-425c-a267-b9517c5f5738 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:33.276296 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-917aaa5d-04d1-4ade-9e6b-e0ff5ef8c1d5 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:45:33.277003 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-917aaa5d-04d1-4ade-9e6b-e0ff5ef8c1d5 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: 3618ceb8-9d10-4219-8885-d2d5004c71ad] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:45:33.283902 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-917aaa5d-04d1-4ade-9e6b-e0ff5ef8c1d5 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:33.284365 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-917aaa5d-04d1-4ade-9e6b-e0ff5ef8c1d5 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:33.285190 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-917aaa5d-04d1-4ade-9e6b-e0ff5ef8c1d5 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:33.285636 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-917aaa5d-04d1-4ade-9e6b-e0ff5ef8c1d5 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:33.373883 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-efa547a4-5322-425c-a267-b9517c5f5738 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/f0c3fd0a-44d4-4209-9c2f-817382c0afa7" status: 200 len: 1464 microversion: 2.1 time: 0.518150 Jul 16 21:45:33.375596 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2917/5835] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:45:32 2022] GET /compute/v2.1/servers/f0c3fd0a-44d4-4209-9c2f-817382c0afa7 => generated 1464 bytes in 520 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:45:33.457208 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-917aaa5d-04d1-4ade-9e6b-e0ff5ef8c1d5 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:45:33.464916 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-917aaa5d-04d1-4ade-9e6b-e0ff5ef8c1d5 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad" status: 200 len: 1362 microversion: 2.1 time: 0.193779 Jul 16 21:45:33.465483 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2919/5836] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:45:33 2022] GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad => generated 1362 bytes in 195 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:45:33.723330 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-1f4775a9-2196-4618-9434-d234c724c626 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:45:33.723971 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-1f4775a9-2196-4618-9434-d234c724c626 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] [instance: 0b62eebf-c7ed-4fa1-a4d1-17947e5d8226] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:45:33.731167 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1f4775a9-2196-4618-9434-d234c724c626 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:33.731634 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1f4775a9-2196-4618-9434-d234c724c626 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:33.732420 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1f4775a9-2196-4618-9434-d234c724c626 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:33.732873 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1f4775a9-2196-4618-9434-d234c724c626 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:33.871887 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-1f4775a9-2196-4618-9434-d234c724c626 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '833136cbdeab4b9e8b6dd2d24c7c8753', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '77c9b7e351e24dc88207e9498f75f578', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:45:33.879862 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-1f4775a9-2196-4618-9434-d234c724c626 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] 10.222.0.16 "GET /compute/v2.1/servers/0b62eebf-c7ed-4fa1-a4d1-17947e5d8226" status: 200 len: 1690 microversion: 2.1 time: 0.160669 Jul 16 21:45:33.880477 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2918/5837] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:45:33 2022] GET /compute/v2.1/servers/0b62eebf-c7ed-4fa1-a4d1-17947e5d8226 => generated 1690 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:45:34.391640 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-8868270a-6193-4cc6-9e5b-e4abd220bb5b tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:45:34.392187 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-8868270a-6193-4cc6-9e5b-e4abd220bb5b tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] [instance: f0c3fd0a-44d4-4209-9c2f-817382c0afa7] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:45:34.399833 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8868270a-6193-4cc6-9e5b-e4abd220bb5b tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:34.400282 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8868270a-6193-4cc6-9e5b-e4abd220bb5b tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:34.401172 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8868270a-6193-4cc6-9e5b-e4abd220bb5b tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:34.401656 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8868270a-6193-4cc6-9e5b-e4abd220bb5b tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:34.482240 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-c0ce5be7-cd05-4068-9308-b832b3073596 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:45:34.482877 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-c0ce5be7-cd05-4068-9308-b832b3073596 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: 3618ceb8-9d10-4219-8885-d2d5004c71ad] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:45:34.498852 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c0ce5be7-cd05-4068-9308-b832b3073596 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:34.499357 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c0ce5be7-cd05-4068-9308-b832b3073596 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:34.500156 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c0ce5be7-cd05-4068-9308-b832b3073596 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:34.500616 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c0ce5be7-cd05-4068-9308-b832b3073596 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:34.544888 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-8868270a-6193-4cc6-9e5b-e4abd220bb5b tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/f0c3fd0a-44d4-4209-9c2f-817382c0afa7" status: 200 len: 1464 microversion: 2.1 time: 0.156980 Jul 16 21:45:34.545941 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2920/5838] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:45:34 2022] GET /compute/v2.1/servers/f0c3fd0a-44d4-4209-9c2f-817382c0afa7 => generated 1464 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:45:34.644546 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-c0ce5be7-cd05-4068-9308-b832b3073596 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:45:34.652396 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-c0ce5be7-cd05-4068-9308-b832b3073596 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad" status: 200 len: 1548 microversion: 2.1 time: 0.174540 Jul 16 21:45:34.652953 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2919/5839] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:45:34 2022] GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad => generated 1548 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:45:34.669892 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-707398e3-0bbe-4274-a5f1-424354d25bd0 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:45:34.670389 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-707398e3-0bbe-4274-a5f1-424354d25bd0 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: 3618ceb8-9d10-4219-8885-d2d5004c71ad] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:45:34.677569 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-707398e3-0bbe-4274-a5f1-424354d25bd0 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:34.678679 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-707398e3-0bbe-4274-a5f1-424354d25bd0 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:34.678844 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-707398e3-0bbe-4274-a5f1-424354d25bd0 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:34.679303 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-707398e3-0bbe-4274-a5f1-424354d25bd0 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:34.757411 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-fb0f8041-430e-470e-ad15-93d1f3668682 req-2c5186e3-6929-457d-94b2-e5c2d4cc9ad4 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "f0c3fd0a-44d4-4209-9c2f-817382c0afa7", "name": "network-vif-plugged", "status": "completed", "tag": "d082e13f-d028-4bc6-9b63-7bea91b4befb"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:45:34.765018 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-707398e3-0bbe-4274-a5f1-424354d25bd0 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.099041 Jul 16 21:45:34.765758 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2921/5840] 10.222.0.16 () {58 vars in 1274 bytes} [Sat Jul 16 21:45:34 2022] GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad/os-interface => generated 284 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:45:34.776855 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-fb0f8041-430e-470e-ad15-93d1f3668682 req-2c5186e3-6929-457d-94b2-e5c2d4cc9ad4 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:34.777311 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-fb0f8041-430e-470e-ad15-93d1f3668682 req-2c5186e3-6929-457d-94b2-e5c2d4cc9ad4 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:34.791160 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-fb0f8041-430e-470e-ad15-93d1f3668682 req-2c5186e3-6929-457d-94b2-e5c2d4cc9ad4 service nova] Creating event network-vif-plugged:d082e13f-d028-4bc6-9b63-7bea91b4befb for instance f0c3fd0a-44d4-4209-9c2f-817382c0afa7 on nch1-master-1 Jul 16 21:45:34.796734 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-fb0f8041-430e-470e-ad15-93d1f3668682 req-2c5186e3-6929-457d-94b2-e5c2d4cc9ad4 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.045004 Jul 16 21:45:34.797572 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2920/5841] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:45:34 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 16 21:45:34.905358 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-f8c62c08-6726-47b6-b405-cfd7cb94d1ca tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:45:34.905358 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-f8c62c08-6726-47b6-b405-cfd7cb94d1ca tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] [instance: 0b62eebf-c7ed-4fa1-a4d1-17947e5d8226] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:45:34.905933 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f8c62c08-6726-47b6-b405-cfd7cb94d1ca tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:34.906399 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f8c62c08-6726-47b6-b405-cfd7cb94d1ca tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:34.907238 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f8c62c08-6726-47b6-b405-cfd7cb94d1ca tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:34.907684 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f8c62c08-6726-47b6-b405-cfd7cb94d1ca tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:35.207433 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-f8c62c08-6726-47b6-b405-cfd7cb94d1ca tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '833136cbdeab4b9e8b6dd2d24c7c8753', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '77c9b7e351e24dc88207e9498f75f578', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:45:35.219140 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-f8c62c08-6726-47b6-b405-cfd7cb94d1ca tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] 10.222.0.16 "GET /compute/v2.1/servers/0b62eebf-c7ed-4fa1-a4d1-17947e5d8226" status: 200 len: 1690 microversion: 2.1 time: 0.325067 Jul 16 21:45:35.229737 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2922/5842] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:45:34 2022] GET /compute/v2.1/servers/0b62eebf-c7ed-4fa1-a4d1-17947e5d8226 => generated 1690 bytes in 328 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:45:35.563208 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-49b85085-6862-46ab-843d-da89952dea43 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:45:35.564047 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-49b85085-6862-46ab-843d-da89952dea43 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] [instance: f0c3fd0a-44d4-4209-9c2f-817382c0afa7] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:45:35.571919 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-49b85085-6862-46ab-843d-da89952dea43 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:35.572409 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-49b85085-6862-46ab-843d-da89952dea43 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:35.573454 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-49b85085-6862-46ab-843d-da89952dea43 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:35.573974 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-49b85085-6862-46ab-843d-da89952dea43 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:35.831424 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-49b85085-6862-46ab-843d-da89952dea43 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/f0c3fd0a-44d4-4209-9c2f-817382c0afa7" status: 200 len: 1464 microversion: 2.1 time: 0.272546 Jul 16 21:45:35.832224 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2921/5843] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:45:35 2022] GET /compute/v2.1/servers/f0c3fd0a-44d4-4209-9c2f-817382c0afa7 => generated 1464 bytes in 274 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:45:36.122062 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-9beb4c28-be1b-488e-91c1-59b162b3cffd req-a33856e0-f382-4e84-983f-ed85b03a3b3a service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "3618ceb8-9d10-4219-8885-d2d5004c71ad", "tag": "ef163361-e10a-4408-b519-5d053d2a55d6"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:45:36.131811 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-9beb4c28-be1b-488e-91c1-59b162b3cffd req-a33856e0-f382-4e84-983f-ed85b03a3b3a service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:36.132261 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-9beb4c28-be1b-488e-91c1-59b162b3cffd req-a33856e0-f382-4e84-983f-ed85b03a3b3a service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:36.144326 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-9beb4c28-be1b-488e-91c1-59b162b3cffd req-a33856e0-f382-4e84-983f-ed85b03a3b3a service nova] Creating event network-changed:ef163361-e10a-4408-b519-5d053d2a55d6 for instance 3618ceb8-9d10-4219-8885-d2d5004c71ad on nch1-master-1 Jul 16 21:45:36.149189 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-9beb4c28-be1b-488e-91c1-59b162b3cffd req-a33856e0-f382-4e84-983f-ed85b03a3b3a service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.030805 Jul 16 21:45:36.149804 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2923/5844] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:45:36 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:45:36.237860 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-c62dea4b-777a-4061-aa60-137a94b3d92d tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:45:36.239434 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-c62dea4b-777a-4061-aa60-137a94b3d92d tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] [instance: 0b62eebf-c7ed-4fa1-a4d1-17947e5d8226] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:45:36.245406 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c62dea4b-777a-4061-aa60-137a94b3d92d tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:36.245870 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c62dea4b-777a-4061-aa60-137a94b3d92d tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:36.246656 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c62dea4b-777a-4061-aa60-137a94b3d92d tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:36.247306 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c62dea4b-777a-4061-aa60-137a94b3d92d tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:36.465087 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-c62dea4b-777a-4061-aa60-137a94b3d92d tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '833136cbdeab4b9e8b6dd2d24c7c8753', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '77c9b7e351e24dc88207e9498f75f578', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:45:36.473346 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-c62dea4b-777a-4061-aa60-137a94b3d92d tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] 10.222.0.16 "GET /compute/v2.1/servers/0b62eebf-c7ed-4fa1-a4d1-17947e5d8226" status: 200 len: 1690 microversion: 2.1 time: 0.240114 Jul 16 21:45:36.473941 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2922/5845] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:45:36 2022] GET /compute/v2.1/servers/0b62eebf-c7ed-4fa1-a4d1-17947e5d8226 => generated 1690 bytes in 241 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:45:36.849591 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-b7a63319-a21f-4423-b90e-1c2d4cf8efc5 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:45:36.850323 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-b7a63319-a21f-4423-b90e-1c2d4cf8efc5 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] [instance: f0c3fd0a-44d4-4209-9c2f-817382c0afa7] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:45:36.856993 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b7a63319-a21f-4423-b90e-1c2d4cf8efc5 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:36.857459 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b7a63319-a21f-4423-b90e-1c2d4cf8efc5 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:36.858240 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b7a63319-a21f-4423-b90e-1c2d4cf8efc5 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:36.858686 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b7a63319-a21f-4423-b90e-1c2d4cf8efc5 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:36.992595 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-b7a63319-a21f-4423-b90e-1c2d4cf8efc5 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/f0c3fd0a-44d4-4209-9c2f-817382c0afa7" status: 200 len: 1649 microversion: 2.1 time: 0.147710 Jul 16 21:45:36.993327 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2924/5846] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:45:36 2022] GET /compute/v2.1/servers/f0c3fd0a-44d4-4209-9c2f-817382c0afa7 => generated 1649 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:45:37.009884 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-72e89f40-1856-40f8-b998-153fee3b972f tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:45:37.010541 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-72e89f40-1856-40f8-b998-153fee3b972f tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] [instance: f0c3fd0a-44d4-4209-9c2f-817382c0afa7] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:45:37.018139 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-72e89f40-1856-40f8-b998-153fee3b972f tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:37.018620 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-72e89f40-1856-40f8-b998-153fee3b972f tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:37.019443 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-72e89f40-1856-40f8-b998-153fee3b972f tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:37.019904 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-72e89f40-1856-40f8-b998-153fee3b972f tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:37.168852 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-72e89f40-1856-40f8-b998-153fee3b972f tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/f0c3fd0a-44d4-4209-9c2f-817382c0afa7" status: 200 len: 1649 microversion: 2.1 time: 0.163585 Jul 16 21:45:37.169476 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2923/5847] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:45:37 2022] GET /compute/v2.1/servers/f0c3fd0a-44d4-4209-9c2f-817382c0afa7 => generated 1649 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:45:37.185939 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-078dadef-e8cd-4770-9b6e-36ccdd4e2be0 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:45:37.186474 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-078dadef-e8cd-4770-9b6e-36ccdd4e2be0 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] [instance: f0c3fd0a-44d4-4209-9c2f-817382c0afa7] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:45:37.194266 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-078dadef-e8cd-4770-9b6e-36ccdd4e2be0 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:37.194592 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-078dadef-e8cd-4770-9b6e-36ccdd4e2be0 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:37.195417 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-078dadef-e8cd-4770-9b6e-36ccdd4e2be0 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:37.195862 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-078dadef-e8cd-4770-9b6e-36ccdd4e2be0 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:37.334466 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-078dadef-e8cd-4770-9b6e-36ccdd4e2be0 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/f0c3fd0a-44d4-4209-9c2f-817382c0afa7" status: 200 len: 1649 microversion: 2.1 time: 0.152315 Jul 16 21:45:37.335092 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2925/5848] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:45:37 2022] GET /compute/v2.1/servers/f0c3fd0a-44d4-4209-9c2f-817382c0afa7 => generated 1649 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:45:37.351750 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-9343c2ba-6dff-43e5-87e7-40af7038b02f tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:45:37.352341 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-9343c2ba-6dff-43e5-87e7-40af7038b02f tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] [instance: f0c3fd0a-44d4-4209-9c2f-817382c0afa7] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:45:37.358866 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9343c2ba-6dff-43e5-87e7-40af7038b02f tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:37.359434 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9343c2ba-6dff-43e5-87e7-40af7038b02f tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:37.360278 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9343c2ba-6dff-43e5-87e7-40af7038b02f tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:37.360861 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9343c2ba-6dff-43e5-87e7-40af7038b02f tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:37.387106 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-9343c2ba-6dff-43e5-87e7-40af7038b02f tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] [instance: f0c3fd0a-44d4-4209-9c2f-817382c0afa7] Going to try to terminate instance {{(pid=138681) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 16 21:45:37.492722 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-c88ccf68-54a7-4d51-b67f-5137a07fc650 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:45:37.493743 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-c88ccf68-54a7-4d51-b67f-5137a07fc650 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] [instance: 0b62eebf-c7ed-4fa1-a4d1-17947e5d8226] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:45:37.500483 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c88ccf68-54a7-4d51-b67f-5137a07fc650 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:37.500934 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c88ccf68-54a7-4d51-b67f-5137a07fc650 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:37.502454 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c88ccf68-54a7-4d51-b67f-5137a07fc650 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:37.503303 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c88ccf68-54a7-4d51-b67f-5137a07fc650 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:37.694584 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-9343c2ba-6dff-43e5-87e7-40af7038b02f tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] 10.222.0.16 "DELETE /compute/v2.1/servers/f0c3fd0a-44d4-4209-9c2f-817382c0afa7" status: 204 len: 0 microversion: 2.1 time: 0.346428 Jul 16 21:45:37.695453 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2924/5849] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:45:37 2022] DELETE /compute/v2.1/servers/f0c3fd0a-44d4-4209-9c2f-817382c0afa7 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 16 21:45:37.704072 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-b1db4809-8bc9-4cb6-9358-e5f27e985d0d tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:45:37.704950 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-b1db4809-8bc9-4cb6-9358-e5f27e985d0d tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] [instance: f0c3fd0a-44d4-4209-9c2f-817382c0afa7] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:45:37.711682 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b1db4809-8bc9-4cb6-9358-e5f27e985d0d tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:37.712157 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b1db4809-8bc9-4cb6-9358-e5f27e985d0d tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:37.712950 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b1db4809-8bc9-4cb6-9358-e5f27e985d0d tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:37.713439 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b1db4809-8bc9-4cb6-9358-e5f27e985d0d tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:37.731473 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-c88ccf68-54a7-4d51-b67f-5137a07fc650 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '833136cbdeab4b9e8b6dd2d24c7c8753', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '77c9b7e351e24dc88207e9498f75f578', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:45:37.742921 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-c88ccf68-54a7-4d51-b67f-5137a07fc650 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] 10.222.0.16 "GET /compute/v2.1/servers/0b62eebf-c7ed-4fa1-a4d1-17947e5d8226" status: 200 len: 1690 microversion: 2.1 time: 0.255135 Jul 16 21:45:37.743901 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2926/5850] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:45:37 2022] GET /compute/v2.1/servers/0b62eebf-c7ed-4fa1-a4d1-17947e5d8226 => generated 1690 bytes in 257 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:45:37.925151 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-b1db4809-8bc9-4cb6-9358-e5f27e985d0d tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/f0c3fd0a-44d4-4209-9c2f-817382c0afa7" status: 200 len: 1655 microversion: 2.1 time: 0.225062 Jul 16 21:45:37.929415 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2925/5851] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:45:37 2022] GET /compute/v2.1/servers/f0c3fd0a-44d4-4209-9c2f-817382c0afa7 => generated 1655 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:45:38.760999 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-d9b4f3e1-eb04-4840-86d0-934b94a4398d tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:45:38.761612 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-d9b4f3e1-eb04-4840-86d0-934b94a4398d tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] [instance: 0b62eebf-c7ed-4fa1-a4d1-17947e5d8226] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:45:38.769995 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d9b4f3e1-eb04-4840-86d0-934b94a4398d tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:38.770446 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d9b4f3e1-eb04-4840-86d0-934b94a4398d tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:38.771231 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d9b4f3e1-eb04-4840-86d0-934b94a4398d tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:38.771670 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d9b4f3e1-eb04-4840-86d0-934b94a4398d tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:38.929388 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-d9b4f3e1-eb04-4840-86d0-934b94a4398d tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '833136cbdeab4b9e8b6dd2d24c7c8753', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '77c9b7e351e24dc88207e9498f75f578', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:45:38.932492 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=100 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:45:38.938815 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-d9b4f3e1-eb04-4840-86d0-934b94a4398d tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] 10.222.0.16 "GET /compute/v2.1/servers/0b62eebf-c7ed-4fa1-a4d1-17947e5d8226" status: 200 len: 1708 microversion: 2.1 time: 0.181911 Jul 16 21:45:38.940014 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2927/5852] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:45:38 2022] GET /compute/v2.1/servers/0b62eebf-c7ed-4fa1-a4d1-17947e5d8226 => generated 1708 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:45:38.942759 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-34e3a4d5-beb3-473f-9af5-aa4cc17f0184 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:45:38.943421 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-34e3a4d5-beb3-473f-9af5-aa4cc17f0184 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] [instance: f0c3fd0a-44d4-4209-9c2f-817382c0afa7] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:45:38.950398 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-34e3a4d5-beb3-473f-9af5-aa4cc17f0184 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:38.950864 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-34e3a4d5-beb3-473f-9af5-aa4cc17f0184 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:38.951676 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-34e3a4d5-beb3-473f-9af5-aa4cc17f0184 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:38.952130 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-34e3a4d5-beb3-473f-9af5-aa4cc17f0184 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:38.957984 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:45:39.114019 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-34e3a4d5-beb3-473f-9af5-aa4cc17f0184 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/f0c3fd0a-44d4-4209-9c2f-817382c0afa7" status: 200 len: 1655 microversion: 2.1 time: 0.176048 Jul 16 21:45:39.114585 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2926/5853] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:45:38 2022] GET /compute/v2.1/servers/f0c3fd0a-44d4-4209-9c2f-817382c0afa7 => generated 1655 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:45:39.960175 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-cc68bd1f-88af-4731-9d38-2eecea68bce3 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:45:39.961906 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-cc68bd1f-88af-4731-9d38-2eecea68bce3 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] [instance: 0b62eebf-c7ed-4fa1-a4d1-17947e5d8226] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:45:39.971157 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cc68bd1f-88af-4731-9d38-2eecea68bce3 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:39.971819 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cc68bd1f-88af-4731-9d38-2eecea68bce3 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:39.972706 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cc68bd1f-88af-4731-9d38-2eecea68bce3 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:39.973435 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cc68bd1f-88af-4731-9d38-2eecea68bce3 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:40.131451 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-967ed8d7-c849-4ddc-a737-e473133d159b tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:45:40.132204 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-967ed8d7-c849-4ddc-a737-e473133d159b tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] [instance: f0c3fd0a-44d4-4209-9c2f-817382c0afa7] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:45:40.140657 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-967ed8d7-c849-4ddc-a737-e473133d159b tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:40.141126 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-967ed8d7-c849-4ddc-a737-e473133d159b tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:40.141914 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-967ed8d7-c849-4ddc-a737-e473133d159b tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:40.142984 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-967ed8d7-c849-4ddc-a737-e473133d159b tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:40.181903 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-cc68bd1f-88af-4731-9d38-2eecea68bce3 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '833136cbdeab4b9e8b6dd2d24c7c8753', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '77c9b7e351e24dc88207e9498f75f578', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:45:40.191824 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-cc68bd1f-88af-4731-9d38-2eecea68bce3 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] 10.222.0.16 "GET /compute/v2.1/servers/0b62eebf-c7ed-4fa1-a4d1-17947e5d8226" status: 200 len: 1696 microversion: 2.1 time: 0.238098 Jul 16 21:45:40.192538 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2928/5854] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:45:39 2022] GET /compute/v2.1/servers/0b62eebf-c7ed-4fa1-a4d1-17947e5d8226 => generated 1696 bytes in 240 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:45:40.308232 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-967ed8d7-c849-4ddc-a737-e473133d159b tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/f0c3fd0a-44d4-4209-9c2f-817382c0afa7" status: 200 len: 1655 microversion: 2.1 time: 0.180488 Jul 16 21:45:40.308810 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2927/5855] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:45:40 2022] GET /compute/v2.1/servers/f0c3fd0a-44d4-4209-9c2f-817382c0afa7 => generated 1655 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:45:40.450777 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-dcb7414a-089b-4ca8-9625-5758637ea0c2 req-94069aa9-0481-476e-8f1b-adc548a491e4 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "f0c3fd0a-44d4-4209-9c2f-817382c0afa7", "name": "network-vif-deleted", "tag": "d082e13f-d028-4bc6-9b63-7bea91b4befb"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:45:40.460409 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-dcb7414a-089b-4ca8-9625-5758637ea0c2 req-94069aa9-0481-476e-8f1b-adc548a491e4 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:40.461133 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-dcb7414a-089b-4ca8-9625-5758637ea0c2 req-94069aa9-0481-476e-8f1b-adc548a491e4 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:40.478325 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-dcb7414a-089b-4ca8-9625-5758637ea0c2 req-94069aa9-0481-476e-8f1b-adc548a491e4 service nova] Creating event network-vif-deleted:d082e13f-d028-4bc6-9b63-7bea91b4befb for instance f0c3fd0a-44d4-4209-9c2f-817382c0afa7 on nch1-master-1 Jul 16 21:45:40.488061 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-dcb7414a-089b-4ca8-9625-5758637ea0c2 req-94069aa9-0481-476e-8f1b-adc548a491e4 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.040420 Jul 16 21:45:40.490204 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2929/5856] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:45:40 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:45:41.211081 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-00604b7a-ccf6-437d-a09f-184842b6ca05 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:45:41.211673 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-00604b7a-ccf6-437d-a09f-184842b6ca05 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] [instance: 0b62eebf-c7ed-4fa1-a4d1-17947e5d8226] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:45:41.218722 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-00604b7a-ccf6-437d-a09f-184842b6ca05 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:41.219219 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-00604b7a-ccf6-437d-a09f-184842b6ca05 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:41.219986 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-00604b7a-ccf6-437d-a09f-184842b6ca05 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:41.220435 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-00604b7a-ccf6-437d-a09f-184842b6ca05 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:41.325745 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-aa43244e-251f-49b8-9e37-95769becc489 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:45:41.326458 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-aa43244e-251f-49b8-9e37-95769becc489 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] [instance: f0c3fd0a-44d4-4209-9c2f-817382c0afa7] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:45:41.334658 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-aa43244e-251f-49b8-9e37-95769becc489 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:41.341199 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-aa43244e-251f-49b8-9e37-95769becc489 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:41.341199 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-aa43244e-251f-49b8-9e37-95769becc489 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:41.341199 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-aa43244e-251f-49b8-9e37-95769becc489 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:41.375339 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-00604b7a-ccf6-437d-a09f-184842b6ca05 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '833136cbdeab4b9e8b6dd2d24c7c8753', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '77c9b7e351e24dc88207e9498f75f578', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:45:41.376189 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-aa43244e-251f-49b8-9e37-95769becc489 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:41.376649 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-aa43244e-251f-49b8-9e37-95769becc489 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:41.377192 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.objects.instance [None req-aa43244e-251f-49b8-9e37-95769becc489 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Lazy-loading 'fault' on Instance uuid f0c3fd0a-44d4-4209-9c2f-817382c0afa7 {{(pid=138680) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:45:41.392033 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-00604b7a-ccf6-437d-a09f-184842b6ca05 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] 10.222.0.16 "GET /compute/v2.1/servers/0b62eebf-c7ed-4fa1-a4d1-17947e5d8226" status: 200 len: 1681 microversion: 2.1 time: 0.184263 Jul 16 21:45:41.393394 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2928/5857] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:45:41 2022] GET /compute/v2.1/servers/0b62eebf-c7ed-4fa1-a4d1-17947e5d8226 => generated 1681 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:45:41.440598 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-aa43244e-251f-49b8-9e37-95769becc489 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/f0c3fd0a-44d4-4209-9c2f-817382c0afa7" status: 200 len: 1454 microversion: 2.1 time: 0.118857 Jul 16 21:45:41.441180 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2930/5858] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:45:41 2022] GET /compute/v2.1/servers/f0c3fd0a-44d4-4209-9c2f-817382c0afa7 => generated 1454 bytes in 120 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:45:42.460610 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-c165017f-8416-4f5b-8361-71d864a3f5f2 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:45:42.461384 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-c165017f-8416-4f5b-8361-71d864a3f5f2 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] [instance: f0c3fd0a-44d4-4209-9c2f-817382c0afa7] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:45:42.468493 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c165017f-8416-4f5b-8361-71d864a3f5f2 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:42.468970 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c165017f-8416-4f5b-8361-71d864a3f5f2 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:42.469836 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c165017f-8416-4f5b-8361-71d864a3f5f2 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:42.470424 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c165017f-8416-4f5b-8361-71d864a3f5f2 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:42.485372 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-c165017f-8416-4f5b-8361-71d864a3f5f2 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] HTTP exception thrown: Instance f0c3fd0a-44d4-4209-9c2f-817382c0afa7 could not be found. Jul 16 21:45:42.487105 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-c165017f-8416-4f5b-8361-71d864a3f5f2 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Request method failure captured: Jul 16 21:45:42.487105 ncdv-master-1 devstack@n-api.service[138681]: request: GET /compute/v2.1/servers/f0c3fd0a-44d4-4209-9c2f-817382c0afa7 HTTP/1.1 Jul 16 21:45:42.487105 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:45:42.487105 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:45:42.487105 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:45:42.487105 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 21:45:42.487105 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:45:42.487105 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:45:42.487105 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:45:42.487105 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:45:42.487105 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0zFcma7a4JqkpFoL7qcYAfaKJDj4IAgkqCV58VDDectkeWQE6HN67mvy7PWtHii8IVR2oMKIL-CFNfPnkcB61-S-HOnYS7hNyV2fh2ERwNTg0uisa3y3rHEc14zZ_4u9wBUfZabMVvu2qp9Kez39flyDV9fTQ4UYMd9EEJpy6W_6v1s Jul 16 21:45:42.487105 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:45:42.487105 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:45:42.487105 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:45:42.487105 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:45:42.487105 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:45:42.487105 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:45:42.487105 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: b351f1a2d6434cd0b3b03628a0befe14 Jul 16 21:45:42.487105 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-TestServerMultinode-1172294623 Jul 16 21:45:42.487105 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,admin,reader Jul 16 21:45:42.487105 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,admin,reader Jul 16 21:45:42.487105 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/b351f1a2d6434cd0b3b03628a0befe14"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/b351f1a2d6434cd0b3b03628a0befe14"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_b351f1a2d6434cd0b3b03628a0befe14"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/b351f1a2d6434cd0b3b03628a0befe14"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:45:42.487105 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-TestServerMultinode-1172294623 Jul 16 21:45:42.487105 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: b351f1a2d6434cd0b3b03628a0befe14 Jul 16 21:45:42.487105 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-TestServerMultinode-1172294623 Jul 16 21:45:42.487105 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-TestServerMultinode-1172294623-project-admin Jul 16 21:45:42.487105 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:45:42.487105 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:45:42.487105 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: e32ea3fe570b40b59110b70fbb0c3caf Jul 16 21:45:42.487105 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-TestServerMultinode-1172294623-project-admin Jul 16 21:45:42.487105 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:45:42.487105 ncdv-master-1 devstack@n-api.service[138681]: exception: Instance f0c3fd0a-44d4-4209-9c2f-817382c0afa7 could not be found. Jul 16 21:45:42.487105 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': 'f0c3fd0a-44d4-4209-9c2f-817382c0afa7'} Jul 16 21:45:42.487105 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:45:42.487105 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:45:42.487105 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:45:42.487105 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:45:42.487105 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:45:42.487105 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:45:42.487105 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:45:42.487105 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:45:42.487105 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:45:42.487105 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:45:42.487105 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance f0c3fd0a-44d4-4209-9c2f-817382c0afa7 could not be found. Jul 16 21:45:42.487105 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:45:42.487105 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:45:42.487105 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:45:42.487105 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:45:42.487105 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:45:42.487105 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:45:42.487105 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:45:42.487105 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:45:42.487105 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:45:42.487105 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:45:42.487105 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:45:42.487105 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:45:42.487105 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:45:42.487105 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:45:42.487105 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:45:42.487105 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:45:42.487105 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance f0c3fd0a-44d4-4209-9c2f-817382c0afa7 could not be found. Jul 16 21:45:42.487105 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:45:42.487105 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:45:42.487105 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:45:42.492467 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:45:42.492467 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:45:42.492467 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:45:42.492467 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:45:42.492467 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:45:42.492467 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance f0c3fd0a-44d4-4209-9c2f-817382c0afa7 could not be found. Jul 16 21:45:42.492467 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:45:42.492467 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-c165017f-8416-4f5b-8361-71d864a3f5f2 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Returning 404 to user: Instance f0c3fd0a-44d4-4209-9c2f-817382c0afa7 could not be found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:45:42.492467 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-c165017f-8416-4f5b-8361-71d864a3f5f2 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/f0c3fd0a-44d4-4209-9c2f-817382c0afa7" status: 404 len: 111 microversion: 2.1 time: 0.032331 Jul 16 21:45:42.492467 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2929/5859] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:45:42 2022] GET /compute/v2.1/servers/f0c3fd0a-44d4-4209-9c2f-817382c0afa7 => generated 111 bytes in 34 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:45:42.498577 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-f163d32c-47bf-4149-a5b3-a59c7c8d1610 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:45:42.502306 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-f163d32c-47bf-4149-a5b3-a59c7c8d1610 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] [instance: 0793bca6-2775-4777-a4e3-1fa6a2e45eac] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:45:42.511900 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f163d32c-47bf-4149-a5b3-a59c7c8d1610 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:42.513143 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f163d32c-47bf-4149-a5b3-a59c7c8d1610 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:42.517834 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f163d32c-47bf-4149-a5b3-a59c7c8d1610 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:42.518579 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f163d32c-47bf-4149-a5b3-a59c7c8d1610 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:42.546599 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-f163d32c-47bf-4149-a5b3-a59c7c8d1610 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] [instance: 0793bca6-2775-4777-a4e3-1fa6a2e45eac] Going to try to terminate instance {{(pid=138680) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 16 21:45:42.734704 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-7d42c905-c054-4483-aaf6-7f705403f758 req-feea77a4-aa35-475b-86d5-a583331fc242 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "0b62eebf-c7ed-4fa1-a4d1-17947e5d8226", "name": "network-vif-unplugged", "status": "completed", "tag": "f2720e89-583c-4f0a-bdaf-e22e35af2d14"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:45:42.737548 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-7d42c905-c054-4483-aaf6-7f705403f758 req-feea77a4-aa35-475b-86d5-a583331fc242 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:42.738112 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-7d42c905-c054-4483-aaf6-7f705403f758 req-feea77a4-aa35-475b-86d5-a583331fc242 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:42.751900 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-7d42c905-c054-4483-aaf6-7f705403f758 req-feea77a4-aa35-475b-86d5-a583331fc242 service nova] Creating event network-vif-unplugged:f2720e89-583c-4f0a-bdaf-e22e35af2d14 for instance 0b62eebf-c7ed-4fa1-a4d1-17947e5d8226 on nch1-master-1 Jul 16 21:45:42.759091 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-7d42c905-c054-4483-aaf6-7f705403f758 req-feea77a4-aa35-475b-86d5-a583331fc242 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.037616 Jul 16 21:45:42.762838 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2930/5860] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:45:42 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 16 21:45:42.941519 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-f163d32c-47bf-4149-a5b3-a59c7c8d1610 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] 10.222.0.16 "DELETE /compute/v2.1/servers/0793bca6-2775-4777-a4e3-1fa6a2e45eac" status: 204 len: 0 microversion: 2.1 time: 0.446795 Jul 16 21:45:42.943986 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2931/5861] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:45:42 2022] DELETE /compute/v2.1/servers/0793bca6-2775-4777-a4e3-1fa6a2e45eac => generated 0 bytes in 450 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 16 21:45:42.954244 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-37aa4a12-498b-493c-8a62-40f55a6bd5c8 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:45:42.954978 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-37aa4a12-498b-493c-8a62-40f55a6bd5c8 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] [instance: 0793bca6-2775-4777-a4e3-1fa6a2e45eac] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:45:42.962111 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-37aa4a12-498b-493c-8a62-40f55a6bd5c8 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:42.962577 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-37aa4a12-498b-493c-8a62-40f55a6bd5c8 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:42.963384 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-37aa4a12-498b-493c-8a62-40f55a6bd5c8 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:42.963841 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-37aa4a12-498b-493c-8a62-40f55a6bd5c8 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:43.096073 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-37aa4a12-498b-493c-8a62-40f55a6bd5c8 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/0793bca6-2775-4777-a4e3-1fa6a2e45eac" status: 200 len: 1655 microversion: 2.1 time: 0.146015 Jul 16 21:45:43.096696 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2931/5862] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:45:42 2022] GET /compute/v2.1/servers/0793bca6-2775-4777-a4e3-1fa6a2e45eac => generated 1655 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:45:43.889148 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-1936dcb2-8ced-4749-b0f7-0b1b16ccbf07 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] Calling method '.version_select at 0x7fdd8fbf3d30>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:45:43.993722 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-1936dcb2-8ced-4749-b0f7-0b1b16ccbf07 tempest-TestVolumeBootPattern-1625426921 tempest-TestVolumeBootPattern-1625426921-project] 10.222.0.16 "DELETE /compute/v2.1/os-keypairs/tempest-TestVolumeBootPattern-1632947712" status: 202 len: 0 microversion: 2.1 time: 0.109643 Jul 16 21:45:43.994361 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2932/5863] 10.222.0.16 () {58 vars in 1262 bytes} [Sat Jul 16 21:45:43 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestVolumeBootPattern-1632947712 => generated 0 bytes in 111 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 21:45:44.121954 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-e3a0b626-18dd-426a-8dd5-224494f46410 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:45:44.122520 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-e3a0b626-18dd-426a-8dd5-224494f46410 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] [instance: 0793bca6-2775-4777-a4e3-1fa6a2e45eac] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:45:44.133414 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e3a0b626-18dd-426a-8dd5-224494f46410 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:44.133874 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e3a0b626-18dd-426a-8dd5-224494f46410 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:44.134664 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e3a0b626-18dd-426a-8dd5-224494f46410 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:44.135141 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e3a0b626-18dd-426a-8dd5-224494f46410 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:44.276363 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-e3a0b626-18dd-426a-8dd5-224494f46410 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/0793bca6-2775-4777-a4e3-1fa6a2e45eac" status: 200 len: 1655 microversion: 2.1 time: 0.158828 Jul 16 21:45:44.276949 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2932/5864] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:45:44 2022] GET /compute/v2.1/servers/0793bca6-2775-4777-a4e3-1fa6a2e45eac => generated 1655 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:45:44.779202 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-f787cd36-2bcb-4a2f-ac03-ce9753ff9766 req-252a162b-2a69-4ca3-acf6-d7a8e99f85b6 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "0b62eebf-c7ed-4fa1-a4d1-17947e5d8226", "name": "network-vif-plugged", "status": "completed", "tag": "f2720e89-583c-4f0a-bdaf-e22e35af2d14"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:45:44.792430 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-f787cd36-2bcb-4a2f-ac03-ce9753ff9766 req-252a162b-2a69-4ca3-acf6-d7a8e99f85b6 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:44.792974 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-f787cd36-2bcb-4a2f-ac03-ce9753ff9766 req-252a162b-2a69-4ca3-acf6-d7a8e99f85b6 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:44.805138 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-f787cd36-2bcb-4a2f-ac03-ce9753ff9766 req-252a162b-2a69-4ca3-acf6-d7a8e99f85b6 service nova] Creating event network-vif-plugged:f2720e89-583c-4f0a-bdaf-e22e35af2d14 for instance 0b62eebf-c7ed-4fa1-a4d1-17947e5d8226 on nch1-master-1 Jul 16 21:45:44.814096 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-f787cd36-2bcb-4a2f-ac03-ce9753ff9766 req-252a162b-2a69-4ca3-acf6-d7a8e99f85b6 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.038693 Jul 16 21:45:44.814941 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2933/5865] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:45:44 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:45:45.050278 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-03bb06a5-4c07-407d-9038-71554b028987 req-27bd8e87-9af0-4db3-a3f4-c265b6424ae5 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "0793bca6-2775-4777-a4e3-1fa6a2e45eac", "name": "network-vif-deleted", "tag": "de32579a-0010-470f-b775-3c60e514ec44"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:45:45.059963 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-03bb06a5-4c07-407d-9038-71554b028987 req-27bd8e87-9af0-4db3-a3f4-c265b6424ae5 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:45.060418 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-03bb06a5-4c07-407d-9038-71554b028987 req-27bd8e87-9af0-4db3-a3f4-c265b6424ae5 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:45.073728 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-03bb06a5-4c07-407d-9038-71554b028987 req-27bd8e87-9af0-4db3-a3f4-c265b6424ae5 service nova] Creating event network-vif-deleted:de32579a-0010-470f-b775-3c60e514ec44 for instance 0793bca6-2775-4777-a4e3-1fa6a2e45eac on nch2-master-1 Jul 16 21:45:45.079051 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-03bb06a5-4c07-407d-9038-71554b028987 req-27bd8e87-9af0-4db3-a3f4-c265b6424ae5 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.032402 Jul 16 21:45:45.079675 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2933/5866] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:45:45 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:45:45.302542 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-9172f46a-0c84-4117-8282-bbb9a5112d0c tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:45:45.303304 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-9172f46a-0c84-4117-8282-bbb9a5112d0c tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] [instance: 0793bca6-2775-4777-a4e3-1fa6a2e45eac] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:45:45.314937 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9172f46a-0c84-4117-8282-bbb9a5112d0c tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:45.315737 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9172f46a-0c84-4117-8282-bbb9a5112d0c tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:45.316369 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9172f46a-0c84-4117-8282-bbb9a5112d0c tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:45.317407 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9172f46a-0c84-4117-8282-bbb9a5112d0c tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:45.401085 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-9172f46a-0c84-4117-8282-bbb9a5112d0c tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/0793bca6-2775-4777-a4e3-1fa6a2e45eac" status: 200 len: 1448 microversion: 2.1 time: 0.110218 Jul 16 21:45:45.401687 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2934/5867] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:45:45 2022] GET /compute/v2.1/servers/0793bca6-2775-4777-a4e3-1fa6a2e45eac => generated 1448 bytes in 112 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:45:46.420149 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-059c6900-5240-4586-bd71-e8f2c8b31b40 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:45:46.421202 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-059c6900-5240-4586-bd71-e8f2c8b31b40 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] [instance: 0793bca6-2775-4777-a4e3-1fa6a2e45eac] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:45:46.428818 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-059c6900-5240-4586-bd71-e8f2c8b31b40 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:46.429461 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-059c6900-5240-4586-bd71-e8f2c8b31b40 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:46.430858 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-059c6900-5240-4586-bd71-e8f2c8b31b40 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:46.431758 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-059c6900-5240-4586-bd71-e8f2c8b31b40 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:46.448553 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-059c6900-5240-4586-bd71-e8f2c8b31b40 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] HTTP exception thrown: Instance 0793bca6-2775-4777-a4e3-1fa6a2e45eac could not be found. Jul 16 21:45:46.450435 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-059c6900-5240-4586-bd71-e8f2c8b31b40 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Request method failure captured: Jul 16 21:45:46.450435 ncdv-master-1 devstack@n-api.service[138681]: request: GET /compute/v2.1/servers/0793bca6-2775-4777-a4e3-1fa6a2e45eac HTTP/1.1 Jul 16 21:45:46.450435 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:45:46.450435 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:45:46.450435 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:45:46.450435 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 21:45:46.450435 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:45:46.450435 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:45:46.450435 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:45:46.450435 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:45:46.450435 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0zFcma7a4JqkpFoL7qcYAfaKJDj4IAgkqCV58VDDectkeWQE6HN67mvy7PWtHii8IVR2oMKIL-CFNfPnkcB61-S-HOnYS7hNyV2fh2ERwNTg0uisa3y3rHEc14zZ_4u9wBUfZabMVvu2qp9Kez39flyDV9fTQ4UYMd9EEJpy6W_6v1s Jul 16 21:45:46.450435 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:45:46.450435 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:45:46.450435 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:45:46.450435 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:45:46.450435 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:45:46.450435 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:45:46.450435 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: b351f1a2d6434cd0b3b03628a0befe14 Jul 16 21:45:46.450435 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-TestServerMultinode-1172294623 Jul 16 21:45:46.450435 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,admin,reader Jul 16 21:45:46.450435 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,admin,reader Jul 16 21:45:46.450435 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/b351f1a2d6434cd0b3b03628a0befe14"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/b351f1a2d6434cd0b3b03628a0befe14"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_b351f1a2d6434cd0b3b03628a0befe14"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/b351f1a2d6434cd0b3b03628a0befe14"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:45:46.450435 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-TestServerMultinode-1172294623 Jul 16 21:45:46.450435 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: b351f1a2d6434cd0b3b03628a0befe14 Jul 16 21:45:46.450435 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-TestServerMultinode-1172294623 Jul 16 21:45:46.450435 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-TestServerMultinode-1172294623-project-admin Jul 16 21:45:46.450435 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:45:46.450435 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:45:46.450435 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: e32ea3fe570b40b59110b70fbb0c3caf Jul 16 21:45:46.450435 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-TestServerMultinode-1172294623-project-admin Jul 16 21:45:46.450435 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:45:46.450435 ncdv-master-1 devstack@n-api.service[138681]: exception: Instance 0793bca6-2775-4777-a4e3-1fa6a2e45eac could not be found. Jul 16 21:45:46.450435 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': '0793bca6-2775-4777-a4e3-1fa6a2e45eac'} Jul 16 21:45:46.450435 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:45:46.450435 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:45:46.450435 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:45:46.450435 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:45:46.450435 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:45:46.450435 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:45:46.450435 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:45:46.450435 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:45:46.450435 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:45:46.450435 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:45:46.450435 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 0793bca6-2775-4777-a4e3-1fa6a2e45eac could not be found. Jul 16 21:45:46.450435 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:45:46.450435 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:45:46.450435 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:45:46.450435 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:45:46.450435 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:45:46.450435 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:45:46.450435 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:45:46.450435 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:45:46.450435 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:45:46.450435 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:45:46.450435 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:45:46.450435 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:45:46.450435 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:45:46.450435 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:45:46.450435 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:45:46.450435 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:45:46.450435 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 0793bca6-2775-4777-a4e3-1fa6a2e45eac could not be found. Jul 16 21:45:46.450435 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:45:46.450435 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:45:46.450435 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:45:46.459900 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:45:46.459900 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:45:46.459900 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:45:46.459900 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:45:46.459900 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:45:46.459900 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 0793bca6-2775-4777-a4e3-1fa6a2e45eac could not be found. Jul 16 21:45:46.459900 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:45:46.459900 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-059c6900-5240-4586-bd71-e8f2c8b31b40 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] Returning 404 to user: Instance 0793bca6-2775-4777-a4e3-1fa6a2e45eac could not be found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:45:46.459900 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-059c6900-5240-4586-bd71-e8f2c8b31b40 tempest-TestServerMultinode-1172294623 tempest-TestServerMultinode-1172294623-project-admin] 10.222.0.16 "GET /compute/v2.1/servers/0793bca6-2775-4777-a4e3-1fa6a2e45eac" status: 404 len: 111 microversion: 2.1 time: 0.039578 Jul 16 21:45:46.459900 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2934/5868] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:45:46 2022] GET /compute/v2.1/servers/0793bca6-2775-4777-a4e3-1fa6a2e45eac => generated 111 bytes in 42 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:45:51.255916 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-96b472b4-7f94-4892-9faa-4dbc04223a3c tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:45:51.255916 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-96b472b4-7f94-4892-9faa-4dbc04223a3c tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] [instance: 770fd99a-ef16-4922-847d-bcadc19b90f6] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:45:51.260283 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-96b472b4-7f94-4892-9faa-4dbc04223a3c tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:51.260817 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-96b472b4-7f94-4892-9faa-4dbc04223a3c tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:51.261676 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-96b472b4-7f94-4892-9faa-4dbc04223a3c tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:51.262269 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-96b472b4-7f94-4892-9faa-4dbc04223a3c tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:51.297275 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-96b472b4-7f94-4892-9faa-4dbc04223a3c tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] 10.222.0.16 "GET /compute/v2.1/servers/770fd99a-ef16-4922-847d-bcadc19b90f6/ips" status: 200 len: 149 microversion: 2.1 time: 0.204460 Jul 16 21:45:51.298735 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2935/5869] 10.222.0.16 () {58 vars in 1247 bytes} [Sat Jul 16 21:45:51 2022] GET /compute/v2.1/servers/770fd99a-ef16-4922-847d-bcadc19b90f6/ips => generated 149 bytes in 207 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:45:54.237145 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-a43f242c-80b5-4d43-9255-83dbc67b01ad tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "29d846c4-eb90-4ff5-a11b-24a261dfb1cc"}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:45:54.239852 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-a43f242c-80b5-4d43-9255-83dbc67b01ad tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] [instance: 770fd99a-ef16-4922-847d-bcadc19b90f6] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:45:54.247503 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_api:SELECT=84,nova_api:UPDATE=2 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:45:54.254343 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a43f242c-80b5-4d43-9255-83dbc67b01ad tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:54.255764 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a43f242c-80b5-4d43-9255-83dbc67b01ad tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:54.257491 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a43f242c-80b5-4d43-9255-83dbc67b01ad tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:54.258552 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a43f242c-80b5-4d43-9255-83dbc67b01ad tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:55.307482 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-a43f242c-80b5-4d43-9255-83dbc67b01ad tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] 10.222.0.16 "POST /compute/v2.1/servers/770fd99a-ef16-4922-847d-bcadc19b90f6/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 1.078235 Jul 16 21:45:55.308631 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2935/5870] 10.222.0.16 () {60 vars in 1322 bytes} [Sat Jul 16 21:45:54 2022] POST /compute/v2.1/servers/770fd99a-ef16-4922-847d-bcadc19b90f6/os-volume_attachments => generated 194 bytes in 1081 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 16 21:45:56.286730 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-3267dd46-668c-436b-ab63-2c7e1e09b766 tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] Action: 'create', calling method: .version_select at 0x7fdd8fbc8d30>, body: {"keypair": {"name": "tempest-TestVolumeBackupRestore-644300084", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBKDXyt8NJOhFzTGRC1BDvZXNMdqJ/tOGoAOWiTZalJ5A9IArR+M6VSOjBGxz8X+6NQa+PnfOSmWrUto+WXzJNou3sUWaFBGuy64lMWn6sVWDbrfAeDLaf+2CbHc+pRaAew=="}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:45:56.303175 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-3267dd46-668c-436b-ab63-2c7e1e09b766 tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] Getting quotas for project 8f296b14a61a4997bf411c9b0fdc39d5. Resources: {'key_pairs'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:45:56.308670 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-3267dd46-668c-436b-ab63-2c7e1e09b766 tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] Getting quotas for user 001ed11a062a4cb08a52d1294ef5880b and project 8f296b14a61a4997bf411c9b0fdc39d5. Resources: {'key_pairs'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:45:56.387323 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-3267dd46-668c-436b-ab63-2c7e1e09b766 tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] 10.222.0.16 "POST /compute/v2.1/os-keypairs" status: 200 len: 401 microversion: 2.1 time: 0.100522 Jul 16 21:45:56.387323 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2936/5871] 10.222.0.16 () {60 vars in 1158 bytes} [Sat Jul 16 21:45:56 2022] POST /compute/v2.1/os-keypairs => generated 401 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:45:59.177970 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-501d9f67-9f22-4c36-8150-38f04d69d8b8 tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestVolumeBackupRestore-server-1247148144", "imageRef": "", "flavorRef": "452", "block_device_mapping_v2": [{"uuid": "34b057b6-39af-4a53-9a45-d9404aea035f", "source_type": "volume", "destination_type": "volume", "boot_index": 0}], "key_name": "tempest-TestVolumeBackupRestore-644300084", "security_groups": [{"name": "tempest-secgroup-smoke-653050115"}], "networks": [{"uuid": "b57459df-1e3b-4df2-a8d0-37b6c96f88da"}]}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:45:59.293334 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.network.neutron [None req-501d9f67-9f22-4c36-8150-38f04d69d8b8 tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] validate_networks() for [('b57459df-1e3b-4df2-a8d0-37b6c96f88da', None, None, None, None, None)] {{(pid=138681) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 16 21:45:59.786972 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-830c3473-c553-4893-9281-3473f03f9703 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:45:59.787538 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-830c3473-c553-4893-9281-3473f03f9703 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] [instance: 770fd99a-ef16-4922-847d-bcadc19b90f6] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:45:59.790147 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-501d9f67-9f22-4c36-8150-38f04d69d8b8 tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] Flavor limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 16 21:45:59.790300 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_api:SELECT=69,nova_api:UPDATE=1,nova_api:DELETE=1,nova_api:INSERT=1 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:45:59.790726 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-501d9f67-9f22-4c36-8150-38f04d69d8b8 tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] Image limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 16 21:45:59.791359 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-501d9f67-9f22-4c36-8150-38f04d69d8b8 tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] Flavor pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 16 21:45:59.792025 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-501d9f67-9f22-4c36-8150-38f04d69d8b8 tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] Image pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 16 21:45:59.792480 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-501d9f67-9f22-4c36-8150-38f04d69d8b8 tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 16 21:45:59.796310 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-830c3473-c553-4893-9281-3473f03f9703 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:59.796803 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-830c3473-c553-4893-9281-3473f03f9703 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:59.797583 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-830c3473-c553-4893-9281-3473f03f9703 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:59.798032 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-830c3473-c553-4893-9281-3473f03f9703 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:59.802029 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-501d9f67-9f22-4c36-8150-38f04d69d8b8 tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:59.810915 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-501d9f67-9f22-4c36-8150-38f04d69d8b8 tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:59.810915 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-501d9f67-9f22-4c36-8150-38f04d69d8b8 tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:45:59.810915 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-501d9f67-9f22-4c36-8150-38f04d69d8b8 tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:45:59.821833 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-501d9f67-9f22-4c36-8150-38f04d69d8b8 tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] Getting quotas for project 8f296b14a61a4997bf411c9b0fdc39d5. Resources: {'ram', 'instances', 'cores'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:45:59.827485 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-501d9f67-9f22-4c36-8150-38f04d69d8b8 tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] Getting quotas for user 001ed11a062a4cb08a52d1294ef5880b and project 8f296b14a61a4997bf411c9b0fdc39d5. Resources: {'ram', 'instances', 'cores'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:45:59.885575 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.objects.instance [None req-830c3473-c553-4893-9281-3473f03f9703 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lazy-loading 'services' on Instance uuid 770fd99a-ef16-4922-847d-bcadc19b90f6 {{(pid=138680) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:45:59.891065 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-501d9f67-9f22-4c36-8150-38f04d69d8b8 tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] Going to run 1 instances... {{(pid=138681) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 16 21:45:59.893318 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-501d9f67-9f22-4c36-8150-38f04d69d8b8 tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] [instance: a83a3f66-378b-40a4-baf0-0d67c58f1524] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=False,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag=None,updated_at=,uuid=,volume_id='34b057b6-39af-4a53-9a45-d9404aea035f',volume_size=None,volume_type=None)] {{(pid=138681) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 16 21:46:00.122184 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-830c3473-c553-4893-9281-3473f03f9703 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] 10.222.0.16 "DELETE /compute/v2.1/servers/770fd99a-ef16-4922-847d-bcadc19b90f6/os-volume_attachments/29d846c4-eb90-4ff5-a11b-24a261dfb1cc" status: 202 len: 0 microversion: 2.1 time: 0.339093 Jul 16 21:46:00.122796 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2937/5872] 10.222.0.16 () {58 vars in 1415 bytes} [Sat Jul 16 21:45:59 2022] DELETE /compute/v2.1/servers/770fd99a-ef16-4922-847d-bcadc19b90f6/os-volume_attachments/29d846c4-eb90-4ff5-a11b-24a261dfb1cc => generated 0 bytes in 341 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Jul 16 21:46:00.132202 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-1a6fe623-5379-43ba-affd-b2b9018dbd54 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:46:00.133225 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-1a6fe623-5379-43ba-affd-b2b9018dbd54 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] [instance: 770fd99a-ef16-4922-847d-bcadc19b90f6] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:46:00.139793 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1a6fe623-5379-43ba-affd-b2b9018dbd54 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:46:00.140257 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1a6fe623-5379-43ba-affd-b2b9018dbd54 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:46:00.141075 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1a6fe623-5379-43ba-affd-b2b9018dbd54 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:46:00.141517 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1a6fe623-5379-43ba-affd-b2b9018dbd54 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:46:00.171071 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-1a6fe623-5379-43ba-affd-b2b9018dbd54 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] 10.222.0.16 "GET /compute/v2.1/servers/770fd99a-ef16-4922-847d-bcadc19b90f6/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043778 Jul 16 21:46:00.171863 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2938/5873] 10.222.0.16 () {58 vars in 1301 bytes} [Sat Jul 16 21:46:00 2022] GET /compute/v2.1/servers/770fd99a-ef16-4922-847d-bcadc19b90f6/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 16 21:46:00.346141 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-501d9f67-9f22-4c36-8150-38f04d69d8b8 tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 405 microversion: 2.1 time: 1.173522 Jul 16 21:46:00.346786 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2936/5874] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 21:45:59 2022] POST /compute/v2.1/servers => generated 405 bytes in 1175 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 16 21:46:00.357776 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-705ecb8d-b1cb-42ff-a140-44704436c0e2 tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:46:00.358396 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-705ecb8d-b1cb-42ff-a140-44704436c0e2 tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] [instance: a83a3f66-378b-40a4-baf0-0d67c58f1524] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:46:00.414950 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-705ecb8d-b1cb-42ff-a140-44704436c0e2 tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '001ed11a062a4cb08a52d1294ef5880b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8f296b14a61a4997bf411c9b0fdc39d5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:46:00.422109 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-705ecb8d-b1cb-42ff-a140-44704436c0e2 tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] 10.222.0.16 "GET /compute/v2.1/servers/a83a3f66-378b-40a4-baf0-0d67c58f1524" status: 200 len: 1098 microversion: 2.1 time: 0.068613 Jul 16 21:46:00.422693 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2939/5875] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:46:00 2022] GET /compute/v2.1/servers/a83a3f66-378b-40a4-baf0-0d67c58f1524 => generated 1098 bytes in 70 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:46:01.184966 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-95ec0ce1-43f2-4298-96ef-743357669697 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:46:01.185917 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-95ec0ce1-43f2-4298-96ef-743357669697 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] [instance: 770fd99a-ef16-4922-847d-bcadc19b90f6] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:46:01.192564 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-95ec0ce1-43f2-4298-96ef-743357669697 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:46:01.193034 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-95ec0ce1-43f2-4298-96ef-743357669697 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:46:01.193928 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-95ec0ce1-43f2-4298-96ef-743357669697 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:46:01.194385 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-95ec0ce1-43f2-4298-96ef-743357669697 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:46:01.230696 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-95ec0ce1-43f2-4298-96ef-743357669697 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] 10.222.0.16 "GET /compute/v2.1/servers/770fd99a-ef16-4922-847d-bcadc19b90f6/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.050722 Jul 16 21:46:01.237428 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2937/5876] 10.222.0.16 () {58 vars in 1301 bytes} [Sat Jul 16 21:46:01 2022] GET /compute/v2.1/servers/770fd99a-ef16-4922-847d-bcadc19b90f6/os-volume_attachments => generated 25 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Jul 16 21:46:01.296503 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-f91e6c60-d1ea-454e-8331-febfa17c0037 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:46:01.310862 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-f91e6c60-d1ea-454e-8331-febfa17c0037 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] [instance: 770fd99a-ef16-4922-847d-bcadc19b90f6] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:46:01.310862 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f91e6c60-d1ea-454e-8331-febfa17c0037 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:46:01.310862 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f91e6c60-d1ea-454e-8331-febfa17c0037 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:46:01.310862 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f91e6c60-d1ea-454e-8331-febfa17c0037 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:46:01.310862 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f91e6c60-d1ea-454e-8331-febfa17c0037 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:46:01.339667 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-f91e6c60-d1ea-454e-8331-febfa17c0037 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] [instance: 770fd99a-ef16-4922-847d-bcadc19b90f6] Going to try to terminate instance {{(pid=138680) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 16 21:46:01.439989 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-631949d6-70b8-4cb5-80c3-3a6de4fbd11c tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:46:01.440563 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-631949d6-70b8-4cb5-80c3-3a6de4fbd11c tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] [instance: a83a3f66-378b-40a4-baf0-0d67c58f1524] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:46:01.499857 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-631949d6-70b8-4cb5-80c3-3a6de4fbd11c tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '001ed11a062a4cb08a52d1294ef5880b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8f296b14a61a4997bf411c9b0fdc39d5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:46:01.507431 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-631949d6-70b8-4cb5-80c3-3a6de4fbd11c tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] 10.222.0.16 "GET /compute/v2.1/servers/a83a3f66-378b-40a4-baf0-0d67c58f1524" status: 200 len: 1098 microversion: 2.1 time: 0.071468 Jul 16 21:46:01.508842 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2938/5877] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:46:01 2022] GET /compute/v2.1/servers/a83a3f66-378b-40a4-baf0-0d67c58f1524 => generated 1098 bytes in 73 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:46:01.626524 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-f91e6c60-d1ea-454e-8331-febfa17c0037 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] 10.222.0.16 "DELETE /compute/v2.1/servers/770fd99a-ef16-4922-847d-bcadc19b90f6" status: 204 len: 0 microversion: 2.1 time: 0.334641 Jul 16 21:46:01.627507 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2940/5878] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:46:01 2022] DELETE /compute/v2.1/servers/770fd99a-ef16-4922-847d-bcadc19b90f6 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 16 21:46:01.636644 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-9c1c583f-d018-43b2-aa39-3457daf65cfa tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:46:01.637271 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-9c1c583f-d018-43b2-aa39-3457daf65cfa tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] [instance: 770fd99a-ef16-4922-847d-bcadc19b90f6] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:46:01.643502 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9c1c583f-d018-43b2-aa39-3457daf65cfa tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:46:01.643969 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9c1c583f-d018-43b2-aa39-3457daf65cfa tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:46:01.644743 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9c1c583f-d018-43b2-aa39-3457daf65cfa tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:46:01.645191 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9c1c583f-d018-43b2-aa39-3457daf65cfa tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:46:01.776994 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-9c1c583f-d018-43b2-aa39-3457daf65cfa tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0ab8b3f06e994bbca5d6dd14109bfe50', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e60c0863dc6f4313bdc873fda27ae2fc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:46:01.784628 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-9c1c583f-d018-43b2-aa39-3457daf65cfa tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] 10.222.0.16 "GET /compute/v2.1/servers/770fd99a-ef16-4922-847d-bcadc19b90f6" status: 200 len: 1680 microversion: 2.1 time: 0.151952 Jul 16 21:46:01.785191 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2939/5879] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:46:01 2022] GET /compute/v2.1/servers/770fd99a-ef16-4922-847d-bcadc19b90f6 => generated 1680 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:46:02.525812 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-66c17ad7-b7fa-4389-91f3-a7dcef99d069 tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:46:02.526568 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-66c17ad7-b7fa-4389-91f3-a7dcef99d069 tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] [instance: a83a3f66-378b-40a4-baf0-0d67c58f1524] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:46:02.533732 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-66c17ad7-b7fa-4389-91f3-a7dcef99d069 tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:46:02.534963 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-66c17ad7-b7fa-4389-91f3-a7dcef99d069 tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:46:02.534963 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-66c17ad7-b7fa-4389-91f3-a7dcef99d069 tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:46:02.535452 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-66c17ad7-b7fa-4389-91f3-a7dcef99d069 tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:46:02.599809 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-66c17ad7-b7fa-4389-91f3-a7dcef99d069 tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '001ed11a062a4cb08a52d1294ef5880b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8f296b14a61a4997bf411c9b0fdc39d5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:46:02.607841 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-66c17ad7-b7fa-4389-91f3-a7dcef99d069 tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] 10.222.0.16 "GET /compute/v2.1/servers/a83a3f66-378b-40a4-baf0-0d67c58f1524" status: 200 len: 1148 microversion: 2.1 time: 0.086954 Jul 16 21:46:02.608425 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2941/5880] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:46:02 2022] GET /compute/v2.1/servers/a83a3f66-378b-40a4-baf0-0d67c58f1524 => generated 1148 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:46:02.803856 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-746a512c-9f4f-4e23-90ff-7ae03856712d tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:46:02.804490 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-746a512c-9f4f-4e23-90ff-7ae03856712d tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] [instance: 770fd99a-ef16-4922-847d-bcadc19b90f6] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:46:02.811492 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-746a512c-9f4f-4e23-90ff-7ae03856712d tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:46:02.811952 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-746a512c-9f4f-4e23-90ff-7ae03856712d tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:46:02.812767 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-746a512c-9f4f-4e23-90ff-7ae03856712d tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:46:02.813215 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-746a512c-9f4f-4e23-90ff-7ae03856712d tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:46:02.950681 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-746a512c-9f4f-4e23-90ff-7ae03856712d tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0ab8b3f06e994bbca5d6dd14109bfe50', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e60c0863dc6f4313bdc873fda27ae2fc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:46:02.958379 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-746a512c-9f4f-4e23-90ff-7ae03856712d tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] 10.222.0.16 "GET /compute/v2.1/servers/770fd99a-ef16-4922-847d-bcadc19b90f6" status: 200 len: 1680 microversion: 2.1 time: 0.159225 Jul 16 21:46:02.958983 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2940/5881] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:46:02 2022] GET /compute/v2.1/servers/770fd99a-ef16-4922-847d-bcadc19b90f6 => generated 1680 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:46:03.625399 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-9e3a31fe-4663-497b-932c-42100d36831e tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:46:03.626054 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-9e3a31fe-4663-497b-932c-42100d36831e tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] [instance: a83a3f66-378b-40a4-baf0-0d67c58f1524] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:46:03.633290 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9e3a31fe-4663-497b-932c-42100d36831e tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:46:03.633720 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9e3a31fe-4663-497b-932c-42100d36831e tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:46:03.634495 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9e3a31fe-4663-497b-932c-42100d36831e tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:46:03.634946 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9e3a31fe-4663-497b-932c-42100d36831e tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:46:03.732940 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-9e3a31fe-4663-497b-932c-42100d36831e tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '001ed11a062a4cb08a52d1294ef5880b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8f296b14a61a4997bf411c9b0fdc39d5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:46:03.741135 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-9e3a31fe-4663-497b-932c-42100d36831e tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] 10.222.0.16 "GET /compute/v2.1/servers/a83a3f66-378b-40a4-baf0-0d67c58f1524" status: 200 len: 1140 microversion: 2.1 time: 0.119561 Jul 16 21:46:03.750657 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2942/5882] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:46:03 2022] GET /compute/v2.1/servers/a83a3f66-378b-40a4-baf0-0d67c58f1524 => generated 1140 bytes in 122 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:46:03.975886 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-e98e9ee6-9d0a-4328-93b5-cae79e43471e tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:46:03.976547 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-e98e9ee6-9d0a-4328-93b5-cae79e43471e tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] [instance: 770fd99a-ef16-4922-847d-bcadc19b90f6] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:46:03.984013 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e98e9ee6-9d0a-4328-93b5-cae79e43471e tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:46:03.984481 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e98e9ee6-9d0a-4328-93b5-cae79e43471e tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:46:03.985437 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e98e9ee6-9d0a-4328-93b5-cae79e43471e tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:46:03.985892 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e98e9ee6-9d0a-4328-93b5-cae79e43471e tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:46:04.087246 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-e98e9ee6-9d0a-4328-93b5-cae79e43471e tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0ab8b3f06e994bbca5d6dd14109bfe50', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e60c0863dc6f4313bdc873fda27ae2fc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:46:04.095071 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-e98e9ee6-9d0a-4328-93b5-cae79e43471e tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] 10.222.0.16 "GET /compute/v2.1/servers/770fd99a-ef16-4922-847d-bcadc19b90f6" status: 200 len: 1613 microversion: 2.1 time: 0.123211 Jul 16 21:46:04.095669 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2941/5883] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:46:03 2022] GET /compute/v2.1/servers/770fd99a-ef16-4922-847d-bcadc19b90f6 => generated 1613 bytes in 125 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:46:04.354572 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-461f526b-88e3-4739-8628-94d0127375df req-2cd899f5-b5f7-4176-a0fb-747641f9d66f service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "770fd99a-ef16-4922-847d-bcadc19b90f6", "name": "network-vif-deleted", "tag": "152e5772-bc79-4772-820e-b77b68f320f3"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:46:04.369456 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-461f526b-88e3-4739-8628-94d0127375df req-2cd899f5-b5f7-4176-a0fb-747641f9d66f service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:46:04.369906 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-461f526b-88e3-4739-8628-94d0127375df req-2cd899f5-b5f7-4176-a0fb-747641f9d66f service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:46:04.390799 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-461f526b-88e3-4739-8628-94d0127375df req-2cd899f5-b5f7-4176-a0fb-747641f9d66f service nova] Creating event network-vif-deleted:152e5772-bc79-4772-820e-b77b68f320f3 for instance 770fd99a-ef16-4922-847d-bcadc19b90f6 on nch2-master-1 Jul 16 21:46:04.402734 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-461f526b-88e3-4739-8628-94d0127375df req-2cd899f5-b5f7-4176-a0fb-747641f9d66f service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.052248 Jul 16 21:46:04.407597 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2943/5884] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:46:04 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:46:04.758733 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-77df6bf6-2403-48cd-a82a-c2426f009b50 tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:46:04.759569 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-77df6bf6-2403-48cd-a82a-c2426f009b50 tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] [instance: a83a3f66-378b-40a4-baf0-0d67c58f1524] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:46:04.766873 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-77df6bf6-2403-48cd-a82a-c2426f009b50 tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:46:04.767713 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-77df6bf6-2403-48cd-a82a-c2426f009b50 tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:46:04.769420 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-77df6bf6-2403-48cd-a82a-c2426f009b50 tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:46:04.770112 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-77df6bf6-2403-48cd-a82a-c2426f009b50 tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:46:04.855546 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-77df6bf6-2403-48cd-a82a-c2426f009b50 tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '001ed11a062a4cb08a52d1294ef5880b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8f296b14a61a4997bf411c9b0fdc39d5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:46:04.867421 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-77df6bf6-2403-48cd-a82a-c2426f009b50 tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] 10.222.0.16 "GET /compute/v2.1/servers/a83a3f66-378b-40a4-baf0-0d67c58f1524" status: 200 len: 1214 microversion: 2.1 time: 0.111326 Jul 16 21:46:04.868066 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2942/5885] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:46:04 2022] GET /compute/v2.1/servers/a83a3f66-378b-40a4-baf0-0d67c58f1524 => generated 1214 bytes in 115 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:46:05.112852 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-5ff01855-c26c-449d-ac1c-19f70b166513 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:46:05.115842 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-5ff01855-c26c-449d-ac1c-19f70b166513 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] [instance: 770fd99a-ef16-4922-847d-bcadc19b90f6] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:46:05.131629 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5ff01855-c26c-449d-ac1c-19f70b166513 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:46:05.132200 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5ff01855-c26c-449d-ac1c-19f70b166513 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:46:05.133102 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5ff01855-c26c-449d-ac1c-19f70b166513 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:46:05.133608 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5ff01855-c26c-449d-ac1c-19f70b166513 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:46:05.183330 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5ff01855-c26c-449d-ac1c-19f70b166513 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:46:05.183890 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5ff01855-c26c-449d-ac1c-19f70b166513 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:46:05.184351 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.objects.instance [None req-5ff01855-c26c-449d-ac1c-19f70b166513 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lazy-loading 'fault' on Instance uuid 770fd99a-ef16-4922-847d-bcadc19b90f6 {{(pid=138680) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:46:05.230870 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-5ff01855-c26c-449d-ac1c-19f70b166513 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0ab8b3f06e994bbca5d6dd14109bfe50', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e60c0863dc6f4313bdc873fda27ae2fc', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:46:05.239742 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-5ff01855-c26c-449d-ac1c-19f70b166513 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] 10.222.0.16 "GET /compute/v2.1/servers/770fd99a-ef16-4922-847d-bcadc19b90f6" status: 200 len: 1331 microversion: 2.1 time: 0.131244 Jul 16 21:46:05.242160 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2944/5886] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:46:05 2022] GET /compute/v2.1/servers/770fd99a-ef16-4922-847d-bcadc19b90f6 => generated 1331 bytes in 134 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:46:05.884864 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-72fb99fa-afa2-4a6d-8658-a673efa4b419 tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:46:05.889006 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-72fb99fa-afa2-4a6d-8658-a673efa4b419 tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] [instance: a83a3f66-378b-40a4-baf0-0d67c58f1524] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:46:05.892991 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-72fb99fa-afa2-4a6d-8658-a673efa4b419 tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:46:05.893886 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-72fb99fa-afa2-4a6d-8658-a673efa4b419 tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:46:05.894230 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-72fb99fa-afa2-4a6d-8658-a673efa4b419 tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:46:05.894679 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-72fb99fa-afa2-4a6d-8658-a673efa4b419 tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:46:06.128468 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-72fb99fa-afa2-4a6d-8658-a673efa4b419 tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '001ed11a062a4cb08a52d1294ef5880b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8f296b14a61a4997bf411c9b0fdc39d5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:46:06.138681 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-72fb99fa-afa2-4a6d-8658-a673efa4b419 tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] 10.222.0.16 "GET /compute/v2.1/servers/a83a3f66-378b-40a4-baf0-0d67c58f1524" status: 200 len: 1281 microversion: 2.1 time: 0.257767 Jul 16 21:46:06.139726 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2943/5887] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:46:05 2022] GET /compute/v2.1/servers/a83a3f66-378b-40a4-baf0-0d67c58f1524 => generated 1281 bytes in 259 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:46:06.261817 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-ad6f018b-8677-4cd9-8fa1-4a1a26c49a1d tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:46:06.265289 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-ad6f018b-8677-4cd9-8fa1-4a1a26c49a1d tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] [instance: 770fd99a-ef16-4922-847d-bcadc19b90f6] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:46:06.269374 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ad6f018b-8677-4cd9-8fa1-4a1a26c49a1d tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:46:06.269832 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ad6f018b-8677-4cd9-8fa1-4a1a26c49a1d tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:46:06.270796 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ad6f018b-8677-4cd9-8fa1-4a1a26c49a1d tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:46:06.271743 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ad6f018b-8677-4cd9-8fa1-4a1a26c49a1d tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:46:06.274722 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=95,nova_cell1:UPDATE=2,nova_cell1:INSERT=3 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:46:06.286258 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-ad6f018b-8677-4cd9-8fa1-4a1a26c49a1d tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] HTTP exception thrown: Instance 770fd99a-ef16-4922-847d-bcadc19b90f6 could not be found. Jul 16 21:46:06.288490 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-ad6f018b-8677-4cd9-8fa1-4a1a26c49a1d tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Request method failure captured: Jul 16 21:46:06.288490 ncdv-master-1 devstack@n-api.service[138680]: request: GET /compute/v2.1/servers/770fd99a-ef16-4922-847d-bcadc19b90f6 HTTP/1.1 Jul 16 21:46:06.288490 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:46:06.288490 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:46:06.288490 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:46:06.288490 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 0 Jul 16 21:46:06.288490 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:46:06.288490 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:46:06.288490 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:46:06.288490 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:46:06.288490 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0y8gcGxfa3qumeDpAV_UzHtBqxWw5fgWdXEsN_P0XTe84Os21rKjc9QIlpjQBg7EBAnrc9gi4HTfF6VkzKBMP6W-EwHOC7W6W2mOvP55miTafGS3QI8-v38KjXcvgCGFOUcX_9nbDaYEoaZ_SfO91myDjzRdtv09P0hU8CwskGtYm8E Jul 16 21:46:06.288490 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:46:06.288490 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:46:06.288490 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:46:06.288490 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:46:06.288490 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:46:06.288490 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:46:06.288490 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: e60c0863dc6f4313bdc873fda27ae2fc Jul 16 21:46:06.288490 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-AttachVolumeNegativeTest-17004004 Jul 16 21:46:06.288490 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:46:06.288490 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:46:06.288490 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/e60c0863dc6f4313bdc873fda27ae2fc"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/e60c0863dc6f4313bdc873fda27ae2fc"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_e60c0863dc6f4313bdc873fda27ae2fc"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/e60c0863dc6f4313bdc873fda27ae2fc"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:46:06.288490 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-AttachVolumeNegativeTest-17004004 Jul 16 21:46:06.288490 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: e60c0863dc6f4313bdc873fda27ae2fc Jul 16 21:46:06.288490 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-17004004 Jul 16 21:46:06.288490 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-AttachVolumeNegativeTest-17004004-project Jul 16 21:46:06.288490 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:46:06.288490 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:46:06.288490 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: 0ab8b3f06e994bbca5d6dd14109bfe50 Jul 16 21:46:06.288490 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-AttachVolumeNegativeTest-17004004-project Jul 16 21:46:06.288490 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 21:46:06.288490 ncdv-master-1 devstack@n-api.service[138680]: exception: Instance 770fd99a-ef16-4922-847d-bcadc19b90f6 could not be found. Jul 16 21:46:06.288490 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': '770fd99a-ef16-4922-847d-bcadc19b90f6'} Jul 16 21:46:06.288490 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:46:06.288490 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:46:06.288490 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:46:06.288490 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:46:06.288490 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:46:06.288490 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:46:06.288490 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:46:06.288490 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:46:06.288490 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:46:06.288490 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:46:06.288490 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 770fd99a-ef16-4922-847d-bcadc19b90f6 could not be found. Jul 16 21:46:06.288490 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:46:06.288490 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:46:06.288490 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:46:06.288490 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:46:06.288490 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:46:06.288490 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:46:06.288490 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:46:06.288490 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:46:06.288490 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:46:06.288490 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:46:06.288490 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:46:06.288490 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:46:06.288490 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:46:06.288490 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:46:06.288490 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:46:06.288490 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:46:06.288490 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 770fd99a-ef16-4922-847d-bcadc19b90f6 could not be found. Jul 16 21:46:06.288490 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:46:06.288490 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:46:06.288490 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:46:06.294071 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:46:06.294071 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:46:06.294071 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:46:06.294071 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:46:06.294071 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:46:06.294071 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 770fd99a-ef16-4922-847d-bcadc19b90f6 could not be found. Jul 16 21:46:06.294071 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:46:06.294071 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-ad6f018b-8677-4cd9-8fa1-4a1a26c49a1d tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Returning 404 to user: Instance 770fd99a-ef16-4922-847d-bcadc19b90f6 could not be found. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:46:06.294071 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-ad6f018b-8677-4cd9-8fa1-4a1a26c49a1d tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] 10.222.0.16 "GET /compute/v2.1/servers/770fd99a-ef16-4922-847d-bcadc19b90f6" status: 404 len: 111 microversion: 2.1 time: 0.031913 Jul 16 21:46:06.294071 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2945/5888] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:46:06 2022] GET /compute/v2.1/servers/770fd99a-ef16-4922-847d-bcadc19b90f6 => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:46:06.299232 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-df6ecf6f-0483-4d72-bdf9-b98d79623581 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Calling method '.version_select at 0x7fdd8c4c8c10>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:46:06.410639 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-df6ecf6f-0483-4d72-bdf9-b98d79623581 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] 10.222.0.16 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-2120625668" status: 202 len: 0 microversion: 2.1 time: 0.115356 Jul 16 21:46:06.411438 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2944/5889] 10.222.0.16 () {58 vars in 1220 bytes} [Sat Jul 16 21:46:06 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-2120625668 => generated 0 bytes in 117 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 21:46:06.425831 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-004ae223-1c9c-40a1-9176-3afb5d49d21c req-fa42c944-c966-4382-b822-8eb258250e5f service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "a83a3f66-378b-40a4-baf0-0d67c58f1524", "tag": "3f6f502f-e93a-46fe-b6c5-70ff0daa5af0"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:46:06.437330 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-004ae223-1c9c-40a1-9176-3afb5d49d21c req-fa42c944-c966-4382-b822-8eb258250e5f service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:46:06.437790 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-004ae223-1c9c-40a1-9176-3afb5d49d21c req-fa42c944-c966-4382-b822-8eb258250e5f service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:46:06.450500 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-004ae223-1c9c-40a1-9176-3afb5d49d21c req-fa42c944-c966-4382-b822-8eb258250e5f service nova] Creating event network-changed:3f6f502f-e93a-46fe-b6c5-70ff0daa5af0 for instance a83a3f66-378b-40a4-baf0-0d67c58f1524 on nch2-master-1 Jul 16 21:46:06.455875 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-004ae223-1c9c-40a1-9176-3afb5d49d21c req-fa42c944-c966-4382-b822-8eb258250e5f service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.033944 Jul 16 21:46:06.456488 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2946/5890] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:46:06 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:46:07.156543 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-96a33579-4dfc-4aba-9cfd-c07ef4b6eb2f tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:46:07.157352 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-96a33579-4dfc-4aba-9cfd-c07ef4b6eb2f tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] [instance: a83a3f66-378b-40a4-baf0-0d67c58f1524] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:46:07.164318 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-96a33579-4dfc-4aba-9cfd-c07ef4b6eb2f tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:46:07.164781 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-96a33579-4dfc-4aba-9cfd-c07ef4b6eb2f tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:46:07.165567 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-96a33579-4dfc-4aba-9cfd-c07ef4b6eb2f tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:46:07.166019 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-96a33579-4dfc-4aba-9cfd-c07ef4b6eb2f tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:46:07.443553 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-96a33579-4dfc-4aba-9cfd-c07ef4b6eb2f tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '001ed11a062a4cb08a52d1294ef5880b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8f296b14a61a4997bf411c9b0fdc39d5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:46:07.461747 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-96a33579-4dfc-4aba-9cfd-c07ef4b6eb2f tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] 10.222.0.16 "GET /compute/v2.1/servers/a83a3f66-378b-40a4-baf0-0d67c58f1524" status: 200 len: 1269 microversion: 2.1 time: 0.309852 Jul 16 21:46:07.462375 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2945/5891] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:46:07 2022] GET /compute/v2.1/servers/a83a3f66-378b-40a4-baf0-0d67c58f1524 => generated 1269 bytes in 311 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:46:07.609384 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-6a03b000-af66-40f7-a234-39bd7eb63459 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Action: 'action', calling method: >, body: {"os-getConsoleOutput": {}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:46:07.611581 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-6a03b000-af66-40f7-a234-39bd7eb63459 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] [instance: 0b62eebf-c7ed-4fa1-a4d1-17947e5d8226] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:46:07.619340 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6a03b000-af66-40f7-a234-39bd7eb63459 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:46:07.623089 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6a03b000-af66-40f7-a234-39bd7eb63459 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:46:07.623089 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6a03b000-af66-40f7-a234-39bd7eb63459 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:46:07.623089 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6a03b000-af66-40f7-a234-39bd7eb63459 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:46:07.680980 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-6a03b000-af66-40f7-a234-39bd7eb63459 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] 10.222.0.16 "POST /compute/v2.1/servers/0b62eebf-c7ed-4fa1-a4d1-17947e5d8226/action" status: 200 len: 24570 microversion: 2.1 time: 0.077520 Jul 16 21:46:07.681687 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2947/5892] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 21:46:07 2022] POST /compute/v2.1/servers/0b62eebf-c7ed-4fa1-a4d1-17947e5d8226/action => generated 24570 bytes in 80 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 16 21:46:08.479893 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-6b12862c-9b2a-4955-889c-07caa0c76159 tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:46:08.480537 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-6b12862c-9b2a-4955-889c-07caa0c76159 tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] [instance: a83a3f66-378b-40a4-baf0-0d67c58f1524] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:46:08.487619 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6b12862c-9b2a-4955-889c-07caa0c76159 tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:46:08.488709 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6b12862c-9b2a-4955-889c-07caa0c76159 tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:46:08.488889 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6b12862c-9b2a-4955-889c-07caa0c76159 tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:46:08.489336 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6b12862c-9b2a-4955-889c-07caa0c76159 tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:46:08.630239 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-6b12862c-9b2a-4955-889c-07caa0c76159 tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '001ed11a062a4cb08a52d1294ef5880b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8f296b14a61a4997bf411c9b0fdc39d5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:46:08.633903 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:46:08.644444 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-6b12862c-9b2a-4955-889c-07caa0c76159 tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] 10.222.0.16 "GET /compute/v2.1/servers/a83a3f66-378b-40a4-baf0-0d67c58f1524" status: 200 len: 1269 microversion: 2.1 time: 0.163863 Jul 16 21:46:08.644444 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2946/5893] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:46:08 2022] GET /compute/v2.1/servers/a83a3f66-378b-40a4-baf0-0d67c58f1524 => generated 1269 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:46:09.656764 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-b88e1cfa-30cc-42e9-9914-87a4d1b5a5db tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:46:09.657467 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-b88e1cfa-30cc-42e9-9914-87a4d1b5a5db tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] [instance: a83a3f66-378b-40a4-baf0-0d67c58f1524] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:46:09.666098 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b88e1cfa-30cc-42e9-9914-87a4d1b5a5db tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:46:09.666593 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b88e1cfa-30cc-42e9-9914-87a4d1b5a5db tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:46:09.667375 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b88e1cfa-30cc-42e9-9914-87a4d1b5a5db tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:46:09.667814 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b88e1cfa-30cc-42e9-9914-87a4d1b5a5db tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:46:09.802932 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-b88e1cfa-30cc-42e9-9914-87a4d1b5a5db tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '001ed11a062a4cb08a52d1294ef5880b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8f296b14a61a4997bf411c9b0fdc39d5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:46:09.811335 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-b88e1cfa-30cc-42e9-9914-87a4d1b5a5db tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] 10.222.0.16 "GET /compute/v2.1/servers/a83a3f66-378b-40a4-baf0-0d67c58f1524" status: 200 len: 1269 microversion: 2.1 time: 0.158784 Jul 16 21:46:09.811901 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2948/5894] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:46:09 2022] GET /compute/v2.1/servers/a83a3f66-378b-40a4-baf0-0d67c58f1524 => generated 1269 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:46:10.828381 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-891f33e6-6b70-4df4-bec4-79e0b32564e6 tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:46:10.829014 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-891f33e6-6b70-4df4-bec4-79e0b32564e6 tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] [instance: a83a3f66-378b-40a4-baf0-0d67c58f1524] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:46:10.832099 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell0:SELECT=2 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:46:10.838318 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-891f33e6-6b70-4df4-bec4-79e0b32564e6 tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:46:10.842499 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-891f33e6-6b70-4df4-bec4-79e0b32564e6 tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:46:10.842499 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-891f33e6-6b70-4df4-bec4-79e0b32564e6 tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:46:10.842499 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-891f33e6-6b70-4df4-bec4-79e0b32564e6 tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:46:10.964309 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-9e9acc7b-6512-4c84-aabc-a51df17d7ffd tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:46:10.964844 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-9e9acc7b-6512-4c84-aabc-a51df17d7ffd tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] [instance: 770fd99a-ef16-4922-847d-bcadc19b90f6] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:46:10.967603 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell0:SELECT=6 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:46:10.973674 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9e9acc7b-6512-4c84-aabc-a51df17d7ffd tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:46:10.974135 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9e9acc7b-6512-4c84-aabc-a51df17d7ffd tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:46:10.974910 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9e9acc7b-6512-4c84-aabc-a51df17d7ffd tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:46:10.975423 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9e9acc7b-6512-4c84-aabc-a51df17d7ffd tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:46:10.983449 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-891f33e6-6b70-4df4-bec4-79e0b32564e6 tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '001ed11a062a4cb08a52d1294ef5880b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8f296b14a61a4997bf411c9b0fdc39d5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:46:10.991158 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-891f33e6-6b70-4df4-bec4-79e0b32564e6 tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] 10.222.0.16 "GET /compute/v2.1/servers/a83a3f66-378b-40a4-baf0-0d67c58f1524" status: 200 len: 1269 microversion: 2.1 time: 0.166693 Jul 16 21:46:10.991951 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2947/5895] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:46:10 2022] GET /compute/v2.1/servers/a83a3f66-378b-40a4-baf0-0d67c58f1524 => generated 1269 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:46:10.992177 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-9e9acc7b-6512-4c84-aabc-a51df17d7ffd tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] HTTP exception thrown: Instance 770fd99a-ef16-4922-847d-bcadc19b90f6 could not be found. Jul 16 21:46:10.994548 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-9e9acc7b-6512-4c84-aabc-a51df17d7ffd tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Request method failure captured: Jul 16 21:46:10.994548 ncdv-master-1 devstack@n-api.service[138680]: request: DELETE /compute/v2.1/servers/770fd99a-ef16-4922-847d-bcadc19b90f6 HTTP/1.1 Jul 16 21:46:10.994548 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:46:10.994548 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:46:10.994548 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:46:10.994548 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 0 Jul 16 21:46:10.994548 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:46:10.994548 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:46:10.994548 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:46:10.994548 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:46:10.994548 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0y8gcGxfa3qumeDpAV_UzHtBqxWw5fgWdXEsN_P0XTe84Os21rKjc9QIlpjQBg7EBAnrc9gi4HTfF6VkzKBMP6W-EwHOC7W6W2mOvP55miTafGS3QI8-v38KjXcvgCGFOUcX_9nbDaYEoaZ_SfO91myDjzRdtv09P0hU8CwskGtYm8E Jul 16 21:46:10.994548 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:46:10.994548 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:46:10.994548 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:46:10.994548 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:46:10.994548 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:46:10.994548 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:46:10.994548 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: e60c0863dc6f4313bdc873fda27ae2fc Jul 16 21:46:10.994548 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-AttachVolumeNegativeTest-17004004 Jul 16 21:46:10.994548 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:46:10.994548 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:46:10.994548 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/e60c0863dc6f4313bdc873fda27ae2fc"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/e60c0863dc6f4313bdc873fda27ae2fc"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_e60c0863dc6f4313bdc873fda27ae2fc"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/e60c0863dc6f4313bdc873fda27ae2fc"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:46:10.994548 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-AttachVolumeNegativeTest-17004004 Jul 16 21:46:10.994548 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: e60c0863dc6f4313bdc873fda27ae2fc Jul 16 21:46:10.994548 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-17004004 Jul 16 21:46:10.994548 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-AttachVolumeNegativeTest-17004004-project Jul 16 21:46:10.994548 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:46:10.994548 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:46:10.994548 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: 0ab8b3f06e994bbca5d6dd14109bfe50 Jul 16 21:46:10.994548 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-AttachVolumeNegativeTest-17004004-project Jul 16 21:46:10.994548 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 21:46:10.994548 ncdv-master-1 devstack@n-api.service[138680]: exception: Instance 770fd99a-ef16-4922-847d-bcadc19b90f6 could not be found. Jul 16 21:46:10.994548 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': '770fd99a-ef16-4922-847d-bcadc19b90f6'} Jul 16 21:46:10.994548 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:46:10.994548 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:46:10.994548 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:46:10.994548 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:46:10.994548 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:46:10.994548 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:46:10.994548 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:46:10.994548 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:46:10.994548 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:46:10.994548 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:46:10.994548 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 770fd99a-ef16-4922-847d-bcadc19b90f6 could not be found. Jul 16 21:46:10.994548 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:46:10.994548 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:46:10.994548 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:46:10.994548 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:46:10.994548 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:46:10.994548 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:46:10.994548 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:46:10.994548 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:46:10.994548 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:46:10.994548 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:46:10.994548 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Jul 16 21:46:10.994548 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Jul 16 21:46:10.994548 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Jul 16 21:46:10.994548 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Jul 16 21:46:10.994548 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:46:10.994548 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:46:10.994548 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:46:10.994548 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:46:10.999253 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 770fd99a-ef16-4922-847d-bcadc19b90f6 could not be found. Jul 16 21:46:10.999253 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:46:10.999253 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:46:10.999253 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:46:10.999253 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:46:10.999253 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:46:10.999253 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:46:10.999253 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:46:10.999253 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:46:10.999253 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 770fd99a-ef16-4922-847d-bcadc19b90f6 could not be found. Jul 16 21:46:10.999253 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:46:10.999253 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-9e9acc7b-6512-4c84-aabc-a51df17d7ffd tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Returning 404 to user: Instance 770fd99a-ef16-4922-847d-bcadc19b90f6 could not be found. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:46:10.999253 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-9e9acc7b-6512-4c84-aabc-a51df17d7ffd tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] 10.222.0.16 "DELETE /compute/v2.1/servers/770fd99a-ef16-4922-847d-bcadc19b90f6" status: 404 len: 111 microversion: 2.1 time: 0.036135 Jul 16 21:46:10.999253 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2949/5896] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:46:10 2022] DELETE /compute/v2.1/servers/770fd99a-ef16-4922-847d-bcadc19b90f6 => generated 111 bytes in 37 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:46:11.004939 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-7282b35b-acf7-4136-bc9f-264e3fff6c6f tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:46:11.005564 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-7282b35b-acf7-4136-bc9f-264e3fff6c6f tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] [instance: 770fd99a-ef16-4922-847d-bcadc19b90f6] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:46:11.011861 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7282b35b-acf7-4136-bc9f-264e3fff6c6f tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:46:11.012326 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7282b35b-acf7-4136-bc9f-264e3fff6c6f tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:46:11.013098 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7282b35b-acf7-4136-bc9f-264e3fff6c6f tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:46:11.013551 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7282b35b-acf7-4136-bc9f-264e3fff6c6f tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:46:11.027240 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-7282b35b-acf7-4136-bc9f-264e3fff6c6f tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] HTTP exception thrown: Instance 770fd99a-ef16-4922-847d-bcadc19b90f6 could not be found. Jul 16 21:46:11.028891 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-7282b35b-acf7-4136-bc9f-264e3fff6c6f tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Request method failure captured: Jul 16 21:46:11.028891 ncdv-master-1 devstack@n-api.service[138681]: request: GET /compute/v2.1/servers/770fd99a-ef16-4922-847d-bcadc19b90f6 HTTP/1.1 Jul 16 21:46:11.028891 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:46:11.028891 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:46:11.028891 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:46:11.028891 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 21:46:11.028891 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:46:11.028891 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:46:11.028891 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:46:11.028891 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:46:11.028891 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0y8gcGxfa3qumeDpAV_UzHtBqxWw5fgWdXEsN_P0XTe84Os21rKjc9QIlpjQBg7EBAnrc9gi4HTfF6VkzKBMP6W-EwHOC7W6W2mOvP55miTafGS3QI8-v38KjXcvgCGFOUcX_9nbDaYEoaZ_SfO91myDjzRdtv09P0hU8CwskGtYm8E Jul 16 21:46:11.028891 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:46:11.028891 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:46:11.028891 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:46:11.028891 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:46:11.028891 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:46:11.028891 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:46:11.028891 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: e60c0863dc6f4313bdc873fda27ae2fc Jul 16 21:46:11.028891 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-AttachVolumeNegativeTest-17004004 Jul 16 21:46:11.028891 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:46:11.028891 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:46:11.028891 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/e60c0863dc6f4313bdc873fda27ae2fc"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/e60c0863dc6f4313bdc873fda27ae2fc"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_e60c0863dc6f4313bdc873fda27ae2fc"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/e60c0863dc6f4313bdc873fda27ae2fc"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:46:11.028891 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-AttachVolumeNegativeTest-17004004 Jul 16 21:46:11.028891 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: e60c0863dc6f4313bdc873fda27ae2fc Jul 16 21:46:11.028891 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-17004004 Jul 16 21:46:11.028891 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-AttachVolumeNegativeTest-17004004-project Jul 16 21:46:11.028891 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:46:11.028891 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:46:11.028891 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: 0ab8b3f06e994bbca5d6dd14109bfe50 Jul 16 21:46:11.028891 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-AttachVolumeNegativeTest-17004004-project Jul 16 21:46:11.028891 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:46:11.028891 ncdv-master-1 devstack@n-api.service[138681]: exception: Instance 770fd99a-ef16-4922-847d-bcadc19b90f6 could not be found. Jul 16 21:46:11.028891 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': '770fd99a-ef16-4922-847d-bcadc19b90f6'} Jul 16 21:46:11.028891 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:46:11.028891 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:46:11.028891 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:46:11.028891 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:46:11.028891 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:46:11.028891 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:46:11.028891 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:46:11.028891 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:46:11.028891 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:46:11.028891 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:46:11.028891 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 770fd99a-ef16-4922-847d-bcadc19b90f6 could not be found. Jul 16 21:46:11.028891 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:46:11.028891 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:46:11.028891 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:46:11.028891 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:46:11.028891 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:46:11.028891 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:46:11.028891 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:46:11.028891 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:46:11.028891 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:46:11.028891 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:46:11.028891 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:46:11.028891 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:46:11.028891 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:46:11.028891 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:46:11.028891 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:46:11.028891 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:46:11.028891 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 770fd99a-ef16-4922-847d-bcadc19b90f6 could not be found. Jul 16 21:46:11.028891 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:46:11.028891 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:46:11.028891 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:46:11.034192 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:46:11.034192 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:46:11.034192 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:46:11.034192 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:46:11.034192 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:46:11.034192 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 770fd99a-ef16-4922-847d-bcadc19b90f6 could not be found. Jul 16 21:46:11.034192 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:46:11.034192 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-7282b35b-acf7-4136-bc9f-264e3fff6c6f tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Returning 404 to user: Instance 770fd99a-ef16-4922-847d-bcadc19b90f6 could not be found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:46:11.034192 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-7282b35b-acf7-4136-bc9f-264e3fff6c6f tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] 10.222.0.16 "GET /compute/v2.1/servers/770fd99a-ef16-4922-847d-bcadc19b90f6" status: 404 len: 111 microversion: 2.1 time: 0.029293 Jul 16 21:46:11.034192 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2948/5897] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:46:10 2022] GET /compute/v2.1/servers/770fd99a-ef16-4922-847d-bcadc19b90f6 => generated 111 bytes in 30 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Jul 16 21:46:12.009024 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-66979abb-ec9b-4349-a43f-000d6229ed4f tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:46:12.009670 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-66979abb-ec9b-4349-a43f-000d6229ed4f tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] [instance: a83a3f66-378b-40a4-baf0-0d67c58f1524] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:46:12.017189 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-66979abb-ec9b-4349-a43f-000d6229ed4f tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:46:12.017646 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-66979abb-ec9b-4349-a43f-000d6229ed4f tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:46:12.018423 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-66979abb-ec9b-4349-a43f-000d6229ed4f tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:46:12.018869 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-66979abb-ec9b-4349-a43f-000d6229ed4f tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:46:12.156070 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-66979abb-ec9b-4349-a43f-000d6229ed4f tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '001ed11a062a4cb08a52d1294ef5880b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8f296b14a61a4997bf411c9b0fdc39d5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:46:12.163703 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-66979abb-ec9b-4349-a43f-000d6229ed4f tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] 10.222.0.16 "GET /compute/v2.1/servers/a83a3f66-378b-40a4-baf0-0d67c58f1524" status: 200 len: 1269 microversion: 2.1 time: 0.158563 Jul 16 21:46:12.171235 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2950/5898] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:46:12 2022] GET /compute/v2.1/servers/a83a3f66-378b-40a4-baf0-0d67c58f1524 => generated 1269 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:46:12.202169 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-cee334d2-c4e4-4d01-ba11-00e7de535328 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:46:12.202804 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-cee334d2-c4e4-4d01-ba11-00e7de535328 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: 3618ceb8-9d10-4219-8885-d2d5004c71ad] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:46:12.204776 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell0:SELECT=8 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:46:12.210557 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cee334d2-c4e4-4d01-ba11-00e7de535328 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:46:12.212463 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cee334d2-c4e4-4d01-ba11-00e7de535328 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:46:12.212725 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cee334d2-c4e4-4d01-ba11-00e7de535328 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:46:12.213204 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cee334d2-c4e4-4d01-ba11-00e7de535328 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:46:12.347294 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-cee334d2-c4e4-4d01-ba11-00e7de535328 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:46:12.356146 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-cee334d2-c4e4-4d01-ba11-00e7de535328 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad" status: 200 len: 1665 microversion: 2.1 time: 0.158201 Jul 16 21:46:12.356760 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2949/5899] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:46:12 2022] GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad => generated 1665 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:46:12.373433 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-defc740f-55f6-41ab-8222-6a8d0be85e5c tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:46:12.374084 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-defc740f-55f6-41ab-8222-6a8d0be85e5c tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: 3618ceb8-9d10-4219-8885-d2d5004c71ad] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:46:12.380217 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-defc740f-55f6-41ab-8222-6a8d0be85e5c tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:46:12.384806 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-defc740f-55f6-41ab-8222-6a8d0be85e5c tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:46:12.384806 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-defc740f-55f6-41ab-8222-6a8d0be85e5c tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:46:12.384806 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-defc740f-55f6-41ab-8222-6a8d0be85e5c tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:46:12.516520 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-defc740f-55f6-41ab-8222-6a8d0be85e5c tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:46:12.524988 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-defc740f-55f6-41ab-8222-6a8d0be85e5c tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad" status: 200 len: 1665 microversion: 2.1 time: 0.155488 Jul 16 21:46:12.525558 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2951/5900] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:46:12 2022] GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad => generated 1665 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:46:12.541676 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-cb0c8cb6-f8ad-4c77-aa02-58d4682b182d tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:46:12.542323 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-cb0c8cb6-f8ad-4c77-aa02-58d4682b182d tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: 3618ceb8-9d10-4219-8885-d2d5004c71ad] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:46:12.548591 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cb0c8cb6-f8ad-4c77-aa02-58d4682b182d tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:46:12.549057 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cb0c8cb6-f8ad-4c77-aa02-58d4682b182d tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:46:12.549852 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cb0c8cb6-f8ad-4c77-aa02-58d4682b182d tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:46:12.550289 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cb0c8cb6-f8ad-4c77-aa02-58d4682b182d tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:46:12.685200 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-cb0c8cb6-f8ad-4c77-aa02-58d4682b182d tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:46:12.693853 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-cb0c8cb6-f8ad-4c77-aa02-58d4682b182d tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad" status: 200 len: 1665 microversion: 2.1 time: 0.156717 Jul 16 21:46:12.694492 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2950/5901] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:46:12 2022] GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad => generated 1665 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:46:12.710570 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-786dba61-7b5c-4f95-8d7e-6890d063e856 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Action: 'action', calling method: >, body: {"createBackup": {"backup_type": "daily", "rotation": 2, "name": "tempest-backup-1-286709973"}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:46:12.711649 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-786dba61-7b5c-4f95-8d7e-6890d063e856 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: 3618ceb8-9d10-4219-8885-d2d5004c71ad] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:46:12.718342 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-786dba61-7b5c-4f95-8d7e-6890d063e856 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:46:12.718793 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-786dba61-7b5c-4f95-8d7e-6890d063e856 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:46:12.721767 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-786dba61-7b5c-4f95-8d7e-6890d063e856 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:46:12.721767 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-786dba61-7b5c-4f95-8d7e-6890d063e856 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:46:13.150350 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-786dba61-7b5c-4f95-8d7e-6890d063e856 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "POST /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad/action" status: 202 len: 0 microversion: 2.1 time: 0.443537 Jul 16 21:46:13.151084 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2952/5902] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 21:46:12 2022] POST /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad/action => generated 0 bytes in 445 msecs (HTTP/1.1 202) 10 headers in 456 bytes (1 switches on core 0) Jul 16 21:46:13.181701 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-5347fa0b-4565-428c-9490-2ac3a35ef252 tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:46:13.182327 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-5347fa0b-4565-428c-9490-2ac3a35ef252 tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] [instance: a83a3f66-378b-40a4-baf0-0d67c58f1524] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:46:13.188908 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5347fa0b-4565-428c-9490-2ac3a35ef252 tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:46:13.189429 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5347fa0b-4565-428c-9490-2ac3a35ef252 tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:46:13.190223 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5347fa0b-4565-428c-9490-2ac3a35ef252 tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:46:13.191032 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5347fa0b-4565-428c-9490-2ac3a35ef252 tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:46:13.347745 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-5347fa0b-4565-428c-9490-2ac3a35ef252 tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '001ed11a062a4cb08a52d1294ef5880b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8f296b14a61a4997bf411c9b0fdc39d5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:46:13.364616 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-5347fa0b-4565-428c-9490-2ac3a35ef252 tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] 10.222.0.16 "GET /compute/v2.1/servers/a83a3f66-378b-40a4-baf0-0d67c58f1524" status: 200 len: 1269 microversion: 2.1 time: 0.187439 Jul 16 21:46:13.365228 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2951/5903] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:46:13 2022] GET /compute/v2.1/servers/a83a3f66-378b-40a4-baf0-0d67c58f1524 => generated 1269 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:46:14.383150 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-c9887942-072c-449f-80f5-cf3506f53f1d tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:46:14.383946 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-c9887942-072c-449f-80f5-cf3506f53f1d tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] [instance: a83a3f66-378b-40a4-baf0-0d67c58f1524] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:46:14.390467 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c9887942-072c-449f-80f5-cf3506f53f1d tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:46:14.390927 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c9887942-072c-449f-80f5-cf3506f53f1d tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:46:14.391768 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c9887942-072c-449f-80f5-cf3506f53f1d tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:46:14.392219 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c9887942-072c-449f-80f5-cf3506f53f1d tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:46:14.424324 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-bfa0abe8-9975-493d-93c3-087274f1f591 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:46:14.424834 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-bfa0abe8-9975-493d-93c3-087274f1f591 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] [instance: a7f9caa9-fff2-44b2-871d-7e4191ff39e5] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:46:14.442694 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-bfa0abe8-9975-493d-93c3-087274f1f591 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:46:14.443543 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-bfa0abe8-9975-493d-93c3-087274f1f591 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:46:14.444369 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-bfa0abe8-9975-493d-93c3-087274f1f591 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:46:14.444827 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-bfa0abe8-9975-493d-93c3-087274f1f591 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:46:14.460995 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-bfa0abe8-9975-493d-93c3-087274f1f591 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] HTTP exception thrown: Instance a7f9caa9-fff2-44b2-871d-7e4191ff39e5 could not be found. Jul 16 21:46:14.462833 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-bfa0abe8-9975-493d-93c3-087274f1f591 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Request method failure captured: Jul 16 21:46:14.462833 ncdv-master-1 devstack@n-api.service[138681]: request: DELETE /compute/v2.1/servers/a7f9caa9-fff2-44b2-871d-7e4191ff39e5 HTTP/1.1 Jul 16 21:46:14.462833 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:46:14.462833 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:46:14.462833 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:46:14.462833 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 21:46:14.462833 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:46:14.462833 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:46:14.462833 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:46:14.462833 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:46:14.462833 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0y8gcGxfa3qumeDpAV_UzHtBqxWw5fgWdXEsN_P0XTe84Os21rKjc9QIlpjQBg7EBAnrc9gi4HTfF6VkzKBMP6W-EwHOC7W6W2mOvP55miTafGS3QI8-v38KjXcvgCGFOUcX_9nbDaYEoaZ_SfO91myDjzRdtv09P0hU8CwskGtYm8E Jul 16 21:46:14.462833 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:46:14.462833 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:46:14.462833 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:46:14.462833 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:46:14.462833 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:46:14.462833 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:46:14.462833 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: e60c0863dc6f4313bdc873fda27ae2fc Jul 16 21:46:14.462833 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-AttachVolumeNegativeTest-17004004 Jul 16 21:46:14.462833 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:46:14.462833 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:46:14.462833 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/e60c0863dc6f4313bdc873fda27ae2fc"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/e60c0863dc6f4313bdc873fda27ae2fc"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_e60c0863dc6f4313bdc873fda27ae2fc"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/e60c0863dc6f4313bdc873fda27ae2fc"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:46:14.462833 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-AttachVolumeNegativeTest-17004004 Jul 16 21:46:14.462833 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: e60c0863dc6f4313bdc873fda27ae2fc Jul 16 21:46:14.462833 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-17004004 Jul 16 21:46:14.462833 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-AttachVolumeNegativeTest-17004004-project Jul 16 21:46:14.462833 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:46:14.462833 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:46:14.462833 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: 0ab8b3f06e994bbca5d6dd14109bfe50 Jul 16 21:46:14.462833 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-AttachVolumeNegativeTest-17004004-project Jul 16 21:46:14.462833 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:46:14.462833 ncdv-master-1 devstack@n-api.service[138681]: exception: Instance a7f9caa9-fff2-44b2-871d-7e4191ff39e5 could not be found. Jul 16 21:46:14.462833 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': 'a7f9caa9-fff2-44b2-871d-7e4191ff39e5'} Jul 16 21:46:14.462833 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:46:14.462833 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:46:14.462833 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:46:14.462833 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:46:14.462833 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:46:14.462833 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:46:14.462833 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:46:14.462833 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:46:14.462833 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:46:14.462833 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:46:14.462833 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance a7f9caa9-fff2-44b2-871d-7e4191ff39e5 could not be found. Jul 16 21:46:14.462833 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:46:14.462833 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:46:14.462833 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:46:14.462833 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:46:14.462833 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:46:14.462833 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:46:14.462833 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:46:14.462833 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:46:14.462833 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:46:14.462833 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:46:14.462833 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Jul 16 21:46:14.462833 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Jul 16 21:46:14.462833 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Jul 16 21:46:14.462833 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Jul 16 21:46:14.462833 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:46:14.462833 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:46:14.462833 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:46:14.462833 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:46:14.468779 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance a7f9caa9-fff2-44b2-871d-7e4191ff39e5 could not be found. Jul 16 21:46:14.468779 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:46:14.468779 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:46:14.468779 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:46:14.468779 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:46:14.468779 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:46:14.468779 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:46:14.468779 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:46:14.468779 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:46:14.468779 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance a7f9caa9-fff2-44b2-871d-7e4191ff39e5 could not be found. Jul 16 21:46:14.468779 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:46:14.468779 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-bfa0abe8-9975-493d-93c3-087274f1f591 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Returning 404 to user: Instance a7f9caa9-fff2-44b2-871d-7e4191ff39e5 could not be found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:46:14.468779 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-bfa0abe8-9975-493d-93c3-087274f1f591 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] 10.222.0.16 "DELETE /compute/v2.1/servers/a7f9caa9-fff2-44b2-871d-7e4191ff39e5" status: 404 len: 111 microversion: 2.1 time: 0.044060 Jul 16 21:46:14.468779 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2952/5904] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:46:14 2022] DELETE /compute/v2.1/servers/a7f9caa9-fff2-44b2-871d-7e4191ff39e5 => generated 111 bytes in 45 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Jul 16 21:46:14.473002 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-173deb11-86a2-4ac8-a859-4c77a8541df9 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:46:14.473737 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-173deb11-86a2-4ac8-a859-4c77a8541df9 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] [instance: a7f9caa9-fff2-44b2-871d-7e4191ff39e5] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:46:14.480134 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-173deb11-86a2-4ac8-a859-4c77a8541df9 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:46:14.480595 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-173deb11-86a2-4ac8-a859-4c77a8541df9 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:46:14.481565 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-173deb11-86a2-4ac8-a859-4c77a8541df9 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:46:14.482020 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-173deb11-86a2-4ac8-a859-4c77a8541df9 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:46:14.496574 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-173deb11-86a2-4ac8-a859-4c77a8541df9 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] HTTP exception thrown: Instance a7f9caa9-fff2-44b2-871d-7e4191ff39e5 could not be found. Jul 16 21:46:14.498158 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-173deb11-86a2-4ac8-a859-4c77a8541df9 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Request method failure captured: Jul 16 21:46:14.498158 ncdv-master-1 devstack@n-api.service[138681]: request: GET /compute/v2.1/servers/a7f9caa9-fff2-44b2-871d-7e4191ff39e5 HTTP/1.1 Jul 16 21:46:14.498158 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:46:14.498158 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:46:14.498158 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:46:14.498158 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 21:46:14.498158 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:46:14.498158 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:46:14.498158 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:46:14.498158 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:46:14.498158 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0y8gcGxfa3qumeDpAV_UzHtBqxWw5fgWdXEsN_P0XTe84Os21rKjc9QIlpjQBg7EBAnrc9gi4HTfF6VkzKBMP6W-EwHOC7W6W2mOvP55miTafGS3QI8-v38KjXcvgCGFOUcX_9nbDaYEoaZ_SfO91myDjzRdtv09P0hU8CwskGtYm8E Jul 16 21:46:14.498158 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:46:14.498158 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:46:14.498158 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:46:14.498158 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:46:14.498158 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:46:14.498158 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:46:14.498158 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: e60c0863dc6f4313bdc873fda27ae2fc Jul 16 21:46:14.498158 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-AttachVolumeNegativeTest-17004004 Jul 16 21:46:14.498158 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:46:14.498158 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:46:14.498158 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/e60c0863dc6f4313bdc873fda27ae2fc"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/e60c0863dc6f4313bdc873fda27ae2fc"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_e60c0863dc6f4313bdc873fda27ae2fc"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/e60c0863dc6f4313bdc873fda27ae2fc"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:46:14.498158 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-AttachVolumeNegativeTest-17004004 Jul 16 21:46:14.498158 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: e60c0863dc6f4313bdc873fda27ae2fc Jul 16 21:46:14.498158 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-17004004 Jul 16 21:46:14.498158 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-AttachVolumeNegativeTest-17004004-project Jul 16 21:46:14.498158 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:46:14.498158 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:46:14.498158 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: 0ab8b3f06e994bbca5d6dd14109bfe50 Jul 16 21:46:14.498158 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-AttachVolumeNegativeTest-17004004-project Jul 16 21:46:14.498158 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:46:14.498158 ncdv-master-1 devstack@n-api.service[138681]: exception: Instance a7f9caa9-fff2-44b2-871d-7e4191ff39e5 could not be found. Jul 16 21:46:14.498158 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': 'a7f9caa9-fff2-44b2-871d-7e4191ff39e5'} Jul 16 21:46:14.498158 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:46:14.498158 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:46:14.498158 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:46:14.498158 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:46:14.498158 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:46:14.498158 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:46:14.498158 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:46:14.498158 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:46:14.498158 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:46:14.498158 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:46:14.498158 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance a7f9caa9-fff2-44b2-871d-7e4191ff39e5 could not be found. Jul 16 21:46:14.498158 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:46:14.498158 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:46:14.498158 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:46:14.498158 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:46:14.498158 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:46:14.498158 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:46:14.498158 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:46:14.498158 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:46:14.498158 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:46:14.498158 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:46:14.498158 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:46:14.498158 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:46:14.498158 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:46:14.498158 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:46:14.498158 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:46:14.498158 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:46:14.498158 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance a7f9caa9-fff2-44b2-871d-7e4191ff39e5 could not be found. Jul 16 21:46:14.498158 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:46:14.498158 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:46:14.498158 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:46:14.503435 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:46:14.503435 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:46:14.503435 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:46:14.503435 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:46:14.503435 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:46:14.503435 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance a7f9caa9-fff2-44b2-871d-7e4191ff39e5 could not be found. Jul 16 21:46:14.503435 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:46:14.503435 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-173deb11-86a2-4ac8-a859-4c77a8541df9 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Returning 404 to user: Instance a7f9caa9-fff2-44b2-871d-7e4191ff39e5 could not be found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:46:14.503435 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-173deb11-86a2-4ac8-a859-4c77a8541df9 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] 10.222.0.16 "GET /compute/v2.1/servers/a7f9caa9-fff2-44b2-871d-7e4191ff39e5" status: 404 len: 111 microversion: 2.1 time: 0.030499 Jul 16 21:46:14.503435 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2953/5905] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:46:14 2022] GET /compute/v2.1/servers/a7f9caa9-fff2-44b2-871d-7e4191ff39e5 => generated 111 bytes in 34 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Jul 16 21:46:14.507818 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-36c34464-c338-4767-85af-8558cb9020e4 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:46:14.509134 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-36c34464-c338-4767-85af-8558cb9020e4 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] [instance: a8f70d9d-86c1-469f-bb32-a37c680cc80c] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:46:14.515610 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-36c34464-c338-4767-85af-8558cb9020e4 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:46:14.516123 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-36c34464-c338-4767-85af-8558cb9020e4 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:46:14.516933 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-36c34464-c338-4767-85af-8558cb9020e4 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:46:14.517384 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-36c34464-c338-4767-85af-8558cb9020e4 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:46:14.531364 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-36c34464-c338-4767-85af-8558cb9020e4 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] HTTP exception thrown: Instance a8f70d9d-86c1-469f-bb32-a37c680cc80c could not be found. Jul 16 21:46:14.532943 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-36c34464-c338-4767-85af-8558cb9020e4 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Request method failure captured: Jul 16 21:46:14.532943 ncdv-master-1 devstack@n-api.service[138681]: request: DELETE /compute/v2.1/servers/a8f70d9d-86c1-469f-bb32-a37c680cc80c HTTP/1.1 Jul 16 21:46:14.532943 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:46:14.532943 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:46:14.532943 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:46:14.532943 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 21:46:14.532943 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:46:14.532943 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:46:14.532943 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:46:14.532943 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:46:14.532943 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0y8gcGxfa3qumeDpAV_UzHtBqxWw5fgWdXEsN_P0XTe84Os21rKjc9QIlpjQBg7EBAnrc9gi4HTfF6VkzKBMP6W-EwHOC7W6W2mOvP55miTafGS3QI8-v38KjXcvgCGFOUcX_9nbDaYEoaZ_SfO91myDjzRdtv09P0hU8CwskGtYm8E Jul 16 21:46:14.532943 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:46:14.532943 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:46:14.532943 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:46:14.532943 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:46:14.532943 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:46:14.532943 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:46:14.532943 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: e60c0863dc6f4313bdc873fda27ae2fc Jul 16 21:46:14.532943 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-AttachVolumeNegativeTest-17004004 Jul 16 21:46:14.532943 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:46:14.532943 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:46:14.532943 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/e60c0863dc6f4313bdc873fda27ae2fc"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/e60c0863dc6f4313bdc873fda27ae2fc"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_e60c0863dc6f4313bdc873fda27ae2fc"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/e60c0863dc6f4313bdc873fda27ae2fc"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:46:14.532943 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-AttachVolumeNegativeTest-17004004 Jul 16 21:46:14.532943 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: e60c0863dc6f4313bdc873fda27ae2fc Jul 16 21:46:14.532943 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-17004004 Jul 16 21:46:14.532943 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-AttachVolumeNegativeTest-17004004-project Jul 16 21:46:14.532943 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:46:14.532943 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:46:14.532943 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: 0ab8b3f06e994bbca5d6dd14109bfe50 Jul 16 21:46:14.532943 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-AttachVolumeNegativeTest-17004004-project Jul 16 21:46:14.532943 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:46:14.532943 ncdv-master-1 devstack@n-api.service[138681]: exception: Instance a8f70d9d-86c1-469f-bb32-a37c680cc80c could not be found. Jul 16 21:46:14.532943 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': 'a8f70d9d-86c1-469f-bb32-a37c680cc80c'} Jul 16 21:46:14.532943 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:46:14.532943 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:46:14.532943 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:46:14.532943 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:46:14.532943 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:46:14.532943 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:46:14.532943 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:46:14.532943 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:46:14.532943 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:46:14.532943 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:46:14.532943 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance a8f70d9d-86c1-469f-bb32-a37c680cc80c could not be found. Jul 16 21:46:14.532943 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:46:14.532943 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:46:14.532943 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:46:14.532943 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:46:14.532943 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:46:14.532943 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:46:14.532943 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:46:14.532943 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:46:14.532943 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:46:14.532943 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:46:14.532943 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Jul 16 21:46:14.532943 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Jul 16 21:46:14.532943 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Jul 16 21:46:14.532943 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Jul 16 21:46:14.532943 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:46:14.532943 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:46:14.532943 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:46:14.532943 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:46:14.538478 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance a8f70d9d-86c1-469f-bb32-a37c680cc80c could not be found. Jul 16 21:46:14.538478 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:46:14.538478 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:46:14.538478 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:46:14.538478 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:46:14.538478 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:46:14.538478 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:46:14.538478 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:46:14.538478 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:46:14.538478 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance a8f70d9d-86c1-469f-bb32-a37c680cc80c could not be found. Jul 16 21:46:14.538478 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:46:14.538478 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-36c34464-c338-4767-85af-8558cb9020e4 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Returning 404 to user: Instance a8f70d9d-86c1-469f-bb32-a37c680cc80c could not be found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:46:14.538478 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-36c34464-c338-4767-85af-8558cb9020e4 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] 10.222.0.16 "DELETE /compute/v2.1/servers/a8f70d9d-86c1-469f-bb32-a37c680cc80c" status: 404 len: 111 microversion: 2.1 time: 0.030183 Jul 16 21:46:14.538478 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2954/5906] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:46:14 2022] DELETE /compute/v2.1/servers/a8f70d9d-86c1-469f-bb32-a37c680cc80c => generated 111 bytes in 34 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:46:14.544946 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-da1bc69e-7056-482a-abd9-a1855134f32f tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:46:14.544946 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-da1bc69e-7056-482a-abd9-a1855134f32f tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] [instance: a8f70d9d-86c1-469f-bb32-a37c680cc80c] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:46:14.549083 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-c9887942-072c-449f-80f5-cf3506f53f1d tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '001ed11a062a4cb08a52d1294ef5880b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8f296b14a61a4997bf411c9b0fdc39d5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:46:14.550040 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-da1bc69e-7056-482a-abd9-a1855134f32f tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:46:14.550515 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-da1bc69e-7056-482a-abd9-a1855134f32f tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:46:14.551369 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-da1bc69e-7056-482a-abd9-a1855134f32f tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:46:14.551827 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-da1bc69e-7056-482a-abd9-a1855134f32f tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:46:14.557859 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-c9887942-072c-449f-80f5-cf3506f53f1d tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] 10.222.0.16 "GET /compute/v2.1/servers/a83a3f66-378b-40a4-baf0-0d67c58f1524" status: 200 len: 1269 microversion: 2.1 time: 0.179872 Jul 16 21:46:14.558436 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2953/5907] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:46:14 2022] GET /compute/v2.1/servers/a83a3f66-378b-40a4-baf0-0d67c58f1524 => generated 1269 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:46:14.566354 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-da1bc69e-7056-482a-abd9-a1855134f32f tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] HTTP exception thrown: Instance a8f70d9d-86c1-469f-bb32-a37c680cc80c could not be found. Jul 16 21:46:14.567975 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-da1bc69e-7056-482a-abd9-a1855134f32f tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Request method failure captured: Jul 16 21:46:14.567975 ncdv-master-1 devstack@n-api.service[138681]: request: GET /compute/v2.1/servers/a8f70d9d-86c1-469f-bb32-a37c680cc80c HTTP/1.1 Jul 16 21:46:14.567975 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:46:14.567975 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:46:14.567975 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:46:14.567975 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 21:46:14.567975 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:46:14.567975 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:46:14.567975 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:46:14.567975 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:46:14.567975 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0y8gcGxfa3qumeDpAV_UzHtBqxWw5fgWdXEsN_P0XTe84Os21rKjc9QIlpjQBg7EBAnrc9gi4HTfF6VkzKBMP6W-EwHOC7W6W2mOvP55miTafGS3QI8-v38KjXcvgCGFOUcX_9nbDaYEoaZ_SfO91myDjzRdtv09P0hU8CwskGtYm8E Jul 16 21:46:14.567975 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:46:14.567975 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:46:14.567975 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:46:14.567975 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:46:14.567975 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:46:14.567975 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:46:14.567975 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: e60c0863dc6f4313bdc873fda27ae2fc Jul 16 21:46:14.567975 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-AttachVolumeNegativeTest-17004004 Jul 16 21:46:14.567975 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:46:14.567975 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:46:14.567975 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/e60c0863dc6f4313bdc873fda27ae2fc"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/e60c0863dc6f4313bdc873fda27ae2fc"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_e60c0863dc6f4313bdc873fda27ae2fc"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/e60c0863dc6f4313bdc873fda27ae2fc"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:46:14.567975 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-AttachVolumeNegativeTest-17004004 Jul 16 21:46:14.567975 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: e60c0863dc6f4313bdc873fda27ae2fc Jul 16 21:46:14.567975 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-17004004 Jul 16 21:46:14.567975 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-AttachVolumeNegativeTest-17004004-project Jul 16 21:46:14.567975 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:46:14.567975 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:46:14.567975 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: 0ab8b3f06e994bbca5d6dd14109bfe50 Jul 16 21:46:14.567975 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-AttachVolumeNegativeTest-17004004-project Jul 16 21:46:14.567975 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:46:14.567975 ncdv-master-1 devstack@n-api.service[138681]: exception: Instance a8f70d9d-86c1-469f-bb32-a37c680cc80c could not be found. Jul 16 21:46:14.567975 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': 'a8f70d9d-86c1-469f-bb32-a37c680cc80c'} Jul 16 21:46:14.567975 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:46:14.567975 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:46:14.567975 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:46:14.567975 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:46:14.567975 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:46:14.567975 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:46:14.567975 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:46:14.567975 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:46:14.567975 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:46:14.567975 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:46:14.567975 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance a8f70d9d-86c1-469f-bb32-a37c680cc80c could not be found. Jul 16 21:46:14.567975 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:46:14.567975 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:46:14.567975 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:46:14.567975 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:46:14.567975 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:46:14.567975 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:46:14.567975 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:46:14.567975 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:46:14.567975 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:46:14.567975 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:46:14.567975 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:46:14.567975 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:46:14.567975 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:46:14.567975 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:46:14.567975 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:46:14.567975 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:46:14.567975 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance a8f70d9d-86c1-469f-bb32-a37c680cc80c could not be found. Jul 16 21:46:14.567975 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:46:14.567975 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:46:14.567975 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:46:14.573975 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:46:14.573975 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:46:14.573975 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:46:14.573975 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:46:14.573975 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:46:14.573975 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance a8f70d9d-86c1-469f-bb32-a37c680cc80c could not be found. Jul 16 21:46:14.573975 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:46:14.573975 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-da1bc69e-7056-482a-abd9-a1855134f32f tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Returning 404 to user: Instance a8f70d9d-86c1-469f-bb32-a37c680cc80c could not be found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:46:14.573975 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-da1bc69e-7056-482a-abd9-a1855134f32f tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] 10.222.0.16 "GET /compute/v2.1/servers/a8f70d9d-86c1-469f-bb32-a37c680cc80c" status: 404 len: 111 microversion: 2.1 time: 0.030520 Jul 16 21:46:14.573975 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2955/5908] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:46:14 2022] GET /compute/v2.1/servers/a8f70d9d-86c1-469f-bb32-a37c680cc80c => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:46:14.892477 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-e63a86af-0007-4aa6-8126-693d83b4cd05 req-deef4d84-d9e0-4e82-a009-bd470eb9ed43 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "a83a3f66-378b-40a4-baf0-0d67c58f1524", "name": "network-vif-plugged", "status": "completed", "tag": "3f6f502f-e93a-46fe-b6c5-70ff0daa5af0"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:46:14.901288 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-e63a86af-0007-4aa6-8126-693d83b4cd05 req-deef4d84-d9e0-4e82-a009-bd470eb9ed43 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:46:14.902976 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-e63a86af-0007-4aa6-8126-693d83b4cd05 req-deef4d84-d9e0-4e82-a009-bd470eb9ed43 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:46:14.913173 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-e63a86af-0007-4aa6-8126-693d83b4cd05 req-deef4d84-d9e0-4e82-a009-bd470eb9ed43 service nova] Creating event network-vif-plugged:3f6f502f-e93a-46fe-b6c5-70ff0daa5af0 for instance a83a3f66-378b-40a4-baf0-0d67c58f1524 on nch2-master-1 Jul 16 21:46:14.918950 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-e63a86af-0007-4aa6-8126-693d83b4cd05 req-deef4d84-d9e0-4e82-a009-bd470eb9ed43 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.031504 Jul 16 21:46:14.919893 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2954/5909] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:46:14 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 16 21:46:15.575320 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-4c8c7ba4-e49c-4564-940d-10d5075859bc tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:46:15.575982 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-4c8c7ba4-e49c-4564-940d-10d5075859bc tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] [instance: a83a3f66-378b-40a4-baf0-0d67c58f1524] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:46:15.582926 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4c8c7ba4-e49c-4564-940d-10d5075859bc tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:46:15.587494 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4c8c7ba4-e49c-4564-940d-10d5075859bc tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:46:15.587494 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4c8c7ba4-e49c-4564-940d-10d5075859bc tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:46:15.587494 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4c8c7ba4-e49c-4564-940d-10d5075859bc tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:46:16.046949 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-4c8c7ba4-e49c-4564-940d-10d5075859bc tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '001ed11a062a4cb08a52d1294ef5880b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8f296b14a61a4997bf411c9b0fdc39d5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:46:16.054910 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-4c8c7ba4-e49c-4564-940d-10d5075859bc tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] 10.222.0.16 "GET /compute/v2.1/servers/a83a3f66-378b-40a4-baf0-0d67c58f1524" status: 200 len: 1269 microversion: 2.1 time: 0.484127 Jul 16 21:46:16.056622 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2956/5910] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:46:15 2022] GET /compute/v2.1/servers/a83a3f66-378b-40a4-baf0-0d67c58f1524 => generated 1269 bytes in 486 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:46:17.075624 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-3a7aea59-53f7-4073-aa8f-a0c97fbd35b8 tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:46:17.076333 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-3a7aea59-53f7-4073-aa8f-a0c97fbd35b8 tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] [instance: a83a3f66-378b-40a4-baf0-0d67c58f1524] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:46:17.086566 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3a7aea59-53f7-4073-aa8f-a0c97fbd35b8 tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:46:17.086566 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3a7aea59-53f7-4073-aa8f-a0c97fbd35b8 tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:46:17.086566 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3a7aea59-53f7-4073-aa8f-a0c97fbd35b8 tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:46:17.086566 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3a7aea59-53f7-4073-aa8f-a0c97fbd35b8 tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:46:17.230187 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-3a7aea59-53f7-4073-aa8f-a0c97fbd35b8 tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '001ed11a062a4cb08a52d1294ef5880b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8f296b14a61a4997bf411c9b0fdc39d5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:46:17.238106 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-3a7aea59-53f7-4073-aa8f-a0c97fbd35b8 tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] 10.222.0.16 "GET /compute/v2.1/servers/a83a3f66-378b-40a4-baf0-0d67c58f1524" status: 200 len: 1458 microversion: 2.1 time: 0.168760 Jul 16 21:46:17.238696 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2955/5911] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:46:17 2022] GET /compute/v2.1/servers/a83a3f66-378b-40a4-baf0-0d67c58f1524 => generated 1458 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:46:17.254467 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-3470c49d-a622-4784-8423-fe053f293c03 tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:46:17.255123 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-3470c49d-a622-4784-8423-fe053f293c03 tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] [instance: a83a3f66-378b-40a4-baf0-0d67c58f1524] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:46:17.264127 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3470c49d-a622-4784-8423-fe053f293c03 tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:46:17.270348 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3470c49d-a622-4784-8423-fe053f293c03 tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:46:17.270348 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3470c49d-a622-4784-8423-fe053f293c03 tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:46:17.270348 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3470c49d-a622-4784-8423-fe053f293c03 tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:46:17.410586 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-3470c49d-a622-4784-8423-fe053f293c03 tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '001ed11a062a4cb08a52d1294ef5880b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8f296b14a61a4997bf411c9b0fdc39d5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:46:17.418728 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-3470c49d-a622-4784-8423-fe053f293c03 tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] 10.222.0.16 "GET /compute/v2.1/servers/a83a3f66-378b-40a4-baf0-0d67c58f1524" status: 200 len: 1458 microversion: 2.1 time: 0.168128 Jul 16 21:46:17.423978 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2957/5912] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:46:17 2022] GET /compute/v2.1/servers/a83a3f66-378b-40a4-baf0-0d67c58f1524 => generated 1458 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:46:18.747498 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-55a1e6f2-22ec-4204-9a81-629083f64f56 req-4453e307-52da-4edb-94c8-29dc58f252fd service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "3618ceb8-9d10-4219-8885-d2d5004c71ad", "name": "network-vif-plugged", "status": "completed", "tag": "ef163361-e10a-4408-b519-5d053d2a55d6"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:46:18.761433 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-55a1e6f2-22ec-4204-9a81-629083f64f56 req-4453e307-52da-4edb-94c8-29dc58f252fd service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:46:18.761891 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-55a1e6f2-22ec-4204-9a81-629083f64f56 req-4453e307-52da-4edb-94c8-29dc58f252fd service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:46:18.778682 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-55a1e6f2-22ec-4204-9a81-629083f64f56 req-4453e307-52da-4edb-94c8-29dc58f252fd service nova] Creating event network-vif-plugged:ef163361-e10a-4408-b519-5d053d2a55d6 for instance 3618ceb8-9d10-4219-8885-d2d5004c71ad on nch1-master-1 Jul 16 21:46:18.787612 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-55a1e6f2-22ec-4204-9a81-629083f64f56 req-4453e307-52da-4edb-94c8-29dc58f252fd service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.046575 Jul 16 21:46:18.788355 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2956/5913] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:46:18 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:46:19.034630 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-4608a422-2ca2-482f-a053-d9642ec5272b tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:46:19.035832 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-4608a422-2ca2-482f-a053-d9642ec5272b tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] [instance: 4df09776-dc3c-46ff-9e64-e20b5d4679fa] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:46:19.044803 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4608a422-2ca2-482f-a053-d9642ec5272b tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:46:19.045545 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4608a422-2ca2-482f-a053-d9642ec5272b tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:46:19.046718 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4608a422-2ca2-482f-a053-d9642ec5272b tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:46:19.048258 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4608a422-2ca2-482f-a053-d9642ec5272b tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:46:19.067729 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-4608a422-2ca2-482f-a053-d9642ec5272b tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] HTTP exception thrown: Instance 4df09776-dc3c-46ff-9e64-e20b5d4679fa could not be found. Jul 16 21:46:19.069633 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-4608a422-2ca2-482f-a053-d9642ec5272b tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Request method failure captured: Jul 16 21:46:19.069633 ncdv-master-1 devstack@n-api.service[138681]: request: DELETE /compute/v2.1/servers/4df09776-dc3c-46ff-9e64-e20b5d4679fa HTTP/1.1 Jul 16 21:46:19.069633 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:46:19.069633 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:46:19.069633 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:46:19.069633 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 21:46:19.069633 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:46:19.069633 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:46:19.069633 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:46:19.069633 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:46:19.069633 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0y8gcGxfa3qumeDpAV_UzHtBqxWw5fgWdXEsN_P0XTe84Os21rKjc9QIlpjQBg7EBAnrc9gi4HTfF6VkzKBMP6W-EwHOC7W6W2mOvP55miTafGS3QI8-v38KjXcvgCGFOUcX_9nbDaYEoaZ_SfO91myDjzRdtv09P0hU8CwskGtYm8E Jul 16 21:46:19.069633 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:46:19.069633 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:46:19.069633 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:46:19.069633 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:46:19.069633 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:46:19.069633 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:46:19.069633 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: e60c0863dc6f4313bdc873fda27ae2fc Jul 16 21:46:19.069633 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-AttachVolumeNegativeTest-17004004 Jul 16 21:46:19.069633 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:46:19.069633 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:46:19.069633 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/e60c0863dc6f4313bdc873fda27ae2fc"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/e60c0863dc6f4313bdc873fda27ae2fc"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_e60c0863dc6f4313bdc873fda27ae2fc"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/e60c0863dc6f4313bdc873fda27ae2fc"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:46:19.069633 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-AttachVolumeNegativeTest-17004004 Jul 16 21:46:19.069633 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: e60c0863dc6f4313bdc873fda27ae2fc Jul 16 21:46:19.069633 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-17004004 Jul 16 21:46:19.069633 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-AttachVolumeNegativeTest-17004004-project Jul 16 21:46:19.069633 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:46:19.069633 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:46:19.069633 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: 0ab8b3f06e994bbca5d6dd14109bfe50 Jul 16 21:46:19.069633 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-AttachVolumeNegativeTest-17004004-project Jul 16 21:46:19.069633 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:46:19.069633 ncdv-master-1 devstack@n-api.service[138681]: exception: Instance 4df09776-dc3c-46ff-9e64-e20b5d4679fa could not be found. Jul 16 21:46:19.069633 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': '4df09776-dc3c-46ff-9e64-e20b5d4679fa'} Jul 16 21:46:19.069633 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:46:19.069633 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:46:19.069633 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:46:19.069633 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:46:19.069633 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:46:19.069633 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:46:19.069633 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:46:19.069633 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:46:19.069633 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:46:19.069633 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:46:19.069633 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 4df09776-dc3c-46ff-9e64-e20b5d4679fa could not be found. Jul 16 21:46:19.069633 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:46:19.069633 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:46:19.069633 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:46:19.069633 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:46:19.069633 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:46:19.069633 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:46:19.069633 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:46:19.069633 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:46:19.069633 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:46:19.069633 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:46:19.069633 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Jul 16 21:46:19.069633 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Jul 16 21:46:19.069633 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Jul 16 21:46:19.069633 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Jul 16 21:46:19.069633 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:46:19.069633 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:46:19.069633 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:46:19.069633 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:46:19.074826 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 4df09776-dc3c-46ff-9e64-e20b5d4679fa could not be found. Jul 16 21:46:19.074826 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:46:19.074826 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:46:19.074826 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:46:19.074826 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:46:19.074826 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:46:19.074826 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:46:19.074826 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:46:19.074826 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:46:19.074826 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 4df09776-dc3c-46ff-9e64-e20b5d4679fa could not be found. Jul 16 21:46:19.074826 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:46:19.077165 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-4608a422-2ca2-482f-a053-d9642ec5272b tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Returning 404 to user: Instance 4df09776-dc3c-46ff-9e64-e20b5d4679fa could not be found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:46:19.078127 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-4608a422-2ca2-482f-a053-d9642ec5272b tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] 10.222.0.16 "DELETE /compute/v2.1/servers/4df09776-dc3c-46ff-9e64-e20b5d4679fa" status: 404 len: 111 microversion: 2.1 time: 0.050197 Jul 16 21:46:19.079067 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2958/5914] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:46:19 2022] DELETE /compute/v2.1/servers/4df09776-dc3c-46ff-9e64-e20b5d4679fa => generated 111 bytes in 52 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Jul 16 21:46:19.091114 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-eb8f2baf-b268-4469-9be6-f0d6f8a2eec1 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:46:19.092247 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-eb8f2baf-b268-4469-9be6-f0d6f8a2eec1 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] [instance: 4df09776-dc3c-46ff-9e64-e20b5d4679fa] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:46:19.099339 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-eb8f2baf-b268-4469-9be6-f0d6f8a2eec1 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:46:19.099592 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-eb8f2baf-b268-4469-9be6-f0d6f8a2eec1 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:46:19.100496 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-eb8f2baf-b268-4469-9be6-f0d6f8a2eec1 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:46:19.100980 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-eb8f2baf-b268-4469-9be6-f0d6f8a2eec1 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:46:19.120357 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-eb8f2baf-b268-4469-9be6-f0d6f8a2eec1 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] HTTP exception thrown: Instance 4df09776-dc3c-46ff-9e64-e20b5d4679fa could not be found. Jul 16 21:46:19.122471 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-eb8f2baf-b268-4469-9be6-f0d6f8a2eec1 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Request method failure captured: Jul 16 21:46:19.122471 ncdv-master-1 devstack@n-api.service[138680]: request: GET /compute/v2.1/servers/4df09776-dc3c-46ff-9e64-e20b5d4679fa HTTP/1.1 Jul 16 21:46:19.122471 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:46:19.122471 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:46:19.122471 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:46:19.122471 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 0 Jul 16 21:46:19.122471 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:46:19.122471 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:46:19.122471 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:46:19.122471 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:46:19.122471 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0y8gcGxfa3qumeDpAV_UzHtBqxWw5fgWdXEsN_P0XTe84Os21rKjc9QIlpjQBg7EBAnrc9gi4HTfF6VkzKBMP6W-EwHOC7W6W2mOvP55miTafGS3QI8-v38KjXcvgCGFOUcX_9nbDaYEoaZ_SfO91myDjzRdtv09P0hU8CwskGtYm8E Jul 16 21:46:19.122471 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:46:19.122471 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:46:19.122471 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:46:19.122471 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:46:19.122471 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:46:19.122471 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:46:19.122471 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: e60c0863dc6f4313bdc873fda27ae2fc Jul 16 21:46:19.122471 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-AttachVolumeNegativeTest-17004004 Jul 16 21:46:19.122471 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:46:19.122471 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:46:19.122471 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/e60c0863dc6f4313bdc873fda27ae2fc"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/e60c0863dc6f4313bdc873fda27ae2fc"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_e60c0863dc6f4313bdc873fda27ae2fc"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/e60c0863dc6f4313bdc873fda27ae2fc"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:46:19.122471 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-AttachVolumeNegativeTest-17004004 Jul 16 21:46:19.122471 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: e60c0863dc6f4313bdc873fda27ae2fc Jul 16 21:46:19.122471 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-17004004 Jul 16 21:46:19.122471 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-AttachVolumeNegativeTest-17004004-project Jul 16 21:46:19.122471 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:46:19.122471 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:46:19.122471 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: 0ab8b3f06e994bbca5d6dd14109bfe50 Jul 16 21:46:19.122471 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-AttachVolumeNegativeTest-17004004-project Jul 16 21:46:19.122471 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 21:46:19.122471 ncdv-master-1 devstack@n-api.service[138680]: exception: Instance 4df09776-dc3c-46ff-9e64-e20b5d4679fa could not be found. Jul 16 21:46:19.122471 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': '4df09776-dc3c-46ff-9e64-e20b5d4679fa'} Jul 16 21:46:19.122471 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:46:19.122471 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:46:19.122471 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:46:19.122471 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:46:19.122471 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:46:19.122471 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:46:19.122471 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:46:19.122471 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:46:19.122471 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:46:19.122471 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:46:19.122471 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 4df09776-dc3c-46ff-9e64-e20b5d4679fa could not be found. Jul 16 21:46:19.122471 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:46:19.122471 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:46:19.122471 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:46:19.122471 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:46:19.122471 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:46:19.122471 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:46:19.122471 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:46:19.122471 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:46:19.122471 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:46:19.122471 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:46:19.122471 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:46:19.122471 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:46:19.122471 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:46:19.122471 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:46:19.122471 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:46:19.122471 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:46:19.122471 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 4df09776-dc3c-46ff-9e64-e20b5d4679fa could not be found. Jul 16 21:46:19.122471 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:46:19.122471 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:46:19.122471 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:46:19.127790 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:46:19.127790 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:46:19.127790 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:46:19.127790 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:46:19.127790 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:46:19.127790 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 4df09776-dc3c-46ff-9e64-e20b5d4679fa could not be found. Jul 16 21:46:19.127790 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:46:19.144070 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-eb8f2baf-b268-4469-9be6-f0d6f8a2eec1 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] Returning 404 to user: Instance 4df09776-dc3c-46ff-9e64-e20b5d4679fa could not be found. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:46:19.145255 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-eb8f2baf-b268-4469-9be6-f0d6f8a2eec1 tempest-AttachVolumeNegativeTest-17004004 tempest-AttachVolumeNegativeTest-17004004-project] 10.222.0.16 "GET /compute/v2.1/servers/4df09776-dc3c-46ff-9e64-e20b5d4679fa" status: 404 len: 111 microversion: 2.1 time: 0.062131 Jul 16 21:46:19.146357 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2957/5915] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:46:19 2022] GET /compute/v2.1/servers/4df09776-dc3c-46ff-9e64-e20b5d4679fa => generated 111 bytes in 64 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Jul 16 21:46:20.011866 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-4236c920-c0fc-411f-b5d8-304385fa3158 req-f91ea303-d449-44a2-bd2e-9b195f1e2eef service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "a83a3f66-378b-40a4-baf0-0d67c58f1524", "tag": "3f6f502f-e93a-46fe-b6c5-70ff0daa5af0"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:46:20.031528 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-4236c920-c0fc-411f-b5d8-304385fa3158 req-f91ea303-d449-44a2-bd2e-9b195f1e2eef service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:46:20.032154 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-4236c920-c0fc-411f-b5d8-304385fa3158 req-f91ea303-d449-44a2-bd2e-9b195f1e2eef service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:46:20.059624 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-4236c920-c0fc-411f-b5d8-304385fa3158 req-f91ea303-d449-44a2-bd2e-9b195f1e2eef service nova] Creating event network-changed:3f6f502f-e93a-46fe-b6c5-70ff0daa5af0 for instance a83a3f66-378b-40a4-baf0-0d67c58f1524 on nch2-master-1 Jul 16 21:46:20.075416 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-4236c920-c0fc-411f-b5d8-304385fa3158 req-f91ea303-d449-44a2-bd2e-9b195f1e2eef service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.068196 Jul 16 21:46:20.078229 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2959/5916] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:46:20 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 72 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:46:20.699372 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-3df12639-824d-4041-9dea-d6379737e14b tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:46:20.700657 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-3df12639-824d-4041-9dea-d6379737e14b tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] [instance: 0b62eebf-c7ed-4fa1-a4d1-17947e5d8226] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:46:20.711626 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-24e1fe72-d57f-4630-be78-ed051a8cd0e8 req-6d6a51dc-6b32-4d05-8005-d80f6b7e4eff service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "0b62eebf-c7ed-4fa1-a4d1-17947e5d8226", "tag": "f2720e89-583c-4f0a-bdaf-e22e35af2d14"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:46:20.725444 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3df12639-824d-4041-9dea-d6379737e14b tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:46:20.728680 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3df12639-824d-4041-9dea-d6379737e14b tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:46:20.728680 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3df12639-824d-4041-9dea-d6379737e14b tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:46:20.728680 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3df12639-824d-4041-9dea-d6379737e14b tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:46:20.737405 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-24e1fe72-d57f-4630-be78-ed051a8cd0e8 req-6d6a51dc-6b32-4d05-8005-d80f6b7e4eff service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:46:20.739187 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-24e1fe72-d57f-4630-be78-ed051a8cd0e8 req-6d6a51dc-6b32-4d05-8005-d80f6b7e4eff service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:46:20.774657 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-24e1fe72-d57f-4630-be78-ed051a8cd0e8 req-6d6a51dc-6b32-4d05-8005-d80f6b7e4eff service nova] Creating event network-changed:f2720e89-583c-4f0a-bdaf-e22e35af2d14 for instance 0b62eebf-c7ed-4fa1-a4d1-17947e5d8226 on nch1-master-1 Jul 16 21:46:20.793473 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-24e1fe72-d57f-4630-be78-ed051a8cd0e8 req-6d6a51dc-6b32-4d05-8005-d80f6b7e4eff service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.085701 Jul 16 21:46:20.794690 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2960/5917] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:46:20 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 16 21:46:20.841964 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-3df12639-824d-4041-9dea-d6379737e14b tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] [instance: 0b62eebf-c7ed-4fa1-a4d1-17947e5d8226] Going to try to terminate instance {{(pid=138680) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 16 21:46:21.300876 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-3df12639-824d-4041-9dea-d6379737e14b tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] 10.222.0.16 "DELETE /compute/v2.1/servers/0b62eebf-c7ed-4fa1-a4d1-17947e5d8226" status: 204 len: 0 microversion: 2.1 time: 0.606470 Jul 16 21:46:21.304543 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2958/5918] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:46:20 2022] DELETE /compute/v2.1/servers/0b62eebf-c7ed-4fa1-a4d1-17947e5d8226 => generated 0 bytes in 608 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 16 21:46:21.310676 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-8a68688b-7ca7-4071-be2d-94d87d375eab tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:46:21.312147 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-8a68688b-7ca7-4071-be2d-94d87d375eab tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] [instance: 0b62eebf-c7ed-4fa1-a4d1-17947e5d8226] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:46:21.321885 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8a68688b-7ca7-4071-be2d-94d87d375eab tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:46:21.322878 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8a68688b-7ca7-4071-be2d-94d87d375eab tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:46:21.324026 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8a68688b-7ca7-4071-be2d-94d87d375eab tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:46:21.324711 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8a68688b-7ca7-4071-be2d-94d87d375eab tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:46:21.861807 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-8a68688b-7ca7-4071-be2d-94d87d375eab tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '833136cbdeab4b9e8b6dd2d24c7c8753', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '77c9b7e351e24dc88207e9498f75f578', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:46:21.873566 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-8a68688b-7ca7-4071-be2d-94d87d375eab tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] 10.222.0.16 "GET /compute/v2.1/servers/0b62eebf-c7ed-4fa1-a4d1-17947e5d8226" status: 200 len: 1687 microversion: 2.1 time: 0.566058 Jul 16 21:46:21.886910 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2961/5919] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:46:21 2022] GET /compute/v2.1/servers/0b62eebf-c7ed-4fa1-a4d1-17947e5d8226 => generated 1687 bytes in 581 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:46:22.894898 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-4584deaf-8449-4a6b-8c53-01ee275d491c tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:46:22.895645 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-4584deaf-8449-4a6b-8c53-01ee275d491c tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] [instance: 0b62eebf-c7ed-4fa1-a4d1-17947e5d8226] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:46:22.920907 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4584deaf-8449-4a6b-8c53-01ee275d491c tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:46:22.921380 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4584deaf-8449-4a6b-8c53-01ee275d491c tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:46:22.922172 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4584deaf-8449-4a6b-8c53-01ee275d491c tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:46:22.922626 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4584deaf-8449-4a6b-8c53-01ee275d491c tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:46:23.079183 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-84429a93-dd42-43d2-8293-698f1caf1c33 req-241ce8b3-57ba-45ac-ba7d-df873caf5674 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "a83a3f66-378b-40a4-baf0-0d67c58f1524", "tag": "3f6f502f-e93a-46fe-b6c5-70ff0daa5af0"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:46:23.107722 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-84429a93-dd42-43d2-8293-698f1caf1c33 req-241ce8b3-57ba-45ac-ba7d-df873caf5674 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:46:23.108518 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-84429a93-dd42-43d2-8293-698f1caf1c33 req-241ce8b3-57ba-45ac-ba7d-df873caf5674 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.019s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:46:23.123042 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-84429a93-dd42-43d2-8293-698f1caf1c33 req-241ce8b3-57ba-45ac-ba7d-df873caf5674 service nova] Creating event network-changed:3f6f502f-e93a-46fe-b6c5-70ff0daa5af0 for instance a83a3f66-378b-40a4-baf0-0d67c58f1524 on nch2-master-1 Jul 16 21:46:23.141958 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-84429a93-dd42-43d2-8293-698f1caf1c33 req-241ce8b3-57ba-45ac-ba7d-df873caf5674 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.066497 Jul 16 21:46:23.142869 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2962/5920] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:46:23 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 68 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 16 21:46:23.586659 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-4584deaf-8449-4a6b-8c53-01ee275d491c tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '833136cbdeab4b9e8b6dd2d24c7c8753', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '77c9b7e351e24dc88207e9498f75f578', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:46:23.594712 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-4584deaf-8449-4a6b-8c53-01ee275d491c tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] 10.222.0.16 "GET /compute/v2.1/servers/0b62eebf-c7ed-4fa1-a4d1-17947e5d8226" status: 200 len: 1687 microversion: 2.1 time: 0.704730 Jul 16 21:46:23.610745 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2959/5921] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:46:22 2022] GET /compute/v2.1/servers/0b62eebf-c7ed-4fa1-a4d1-17947e5d8226 => generated 1687 bytes in 721 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:46:24.207909 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-6a76b2eb-e812-4f62-8fee-40f40e83c62d req-cceb714c-c659-4aee-91ba-b58bd38a6926 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "0b62eebf-c7ed-4fa1-a4d1-17947e5d8226", "name": "network-vif-deleted", "tag": "f2720e89-583c-4f0a-bdaf-e22e35af2d14"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:46:24.222480 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-6a76b2eb-e812-4f62-8fee-40f40e83c62d req-cceb714c-c659-4aee-91ba-b58bd38a6926 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:46:24.222962 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-6a76b2eb-e812-4f62-8fee-40f40e83c62d req-cceb714c-c659-4aee-91ba-b58bd38a6926 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:46:24.267586 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-6a76b2eb-e812-4f62-8fee-40f40e83c62d req-cceb714c-c659-4aee-91ba-b58bd38a6926 service nova] Creating event network-vif-deleted:f2720e89-583c-4f0a-bdaf-e22e35af2d14 for instance 0b62eebf-c7ed-4fa1-a4d1-17947e5d8226 on nch1-master-1 Jul 16 21:46:24.293526 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-6a76b2eb-e812-4f62-8fee-40f40e83c62d req-cceb714c-c659-4aee-91ba-b58bd38a6926 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.090455 Jul 16 21:46:24.294536 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2963/5922] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:46:24 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:46:24.613986 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-f7dc67b8-35aa-4ea0-8357-4eb679e8a507 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:46:24.614971 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-f7dc67b8-35aa-4ea0-8357-4eb679e8a507 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] [instance: 0b62eebf-c7ed-4fa1-a4d1-17947e5d8226] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:46:24.639181 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f7dc67b8-35aa-4ea0-8357-4eb679e8a507 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:46:24.639715 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f7dc67b8-35aa-4ea0-8357-4eb679e8a507 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:46:24.642439 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f7dc67b8-35aa-4ea0-8357-4eb679e8a507 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:46:24.642977 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f7dc67b8-35aa-4ea0-8357-4eb679e8a507 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:46:24.720668 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f7dc67b8-35aa-4ea0-8357-4eb679e8a507 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:46:24.721468 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f7dc67b8-35aa-4ea0-8357-4eb679e8a507 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:46:24.722367 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.objects.instance [None req-f7dc67b8-35aa-4ea0-8357-4eb679e8a507 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lazy-loading 'fault' on Instance uuid 0b62eebf-c7ed-4fa1-a4d1-17947e5d8226 {{(pid=138680) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:46:24.793017 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-f7dc67b8-35aa-4ea0-8357-4eb679e8a507 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '833136cbdeab4b9e8b6dd2d24c7c8753', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '77c9b7e351e24dc88207e9498f75f578', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:46:24.805003 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-f7dc67b8-35aa-4ea0-8357-4eb679e8a507 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] 10.222.0.16 "GET /compute/v2.1/servers/0b62eebf-c7ed-4fa1-a4d1-17947e5d8226" status: 200 len: 1355 microversion: 2.1 time: 0.195836 Jul 16 21:46:24.806152 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2960/5923] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:46:24 2022] GET /compute/v2.1/servers/0b62eebf-c7ed-4fa1-a4d1-17947e5d8226 => generated 1355 bytes in 198 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:46:25.172278 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-3d7f11ba-b750-406a-be72-acbbd203f340 req-914faf7c-a4c0-44af-be16-1011d1c18520 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "a83a3f66-378b-40a4-baf0-0d67c58f1524", "tag": "3f6f502f-e93a-46fe-b6c5-70ff0daa5af0"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:46:25.179866 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_api:SELECT=74,nova_api:INSERT=3,nova_api:DELETE=1 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:46:25.183531 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-3d7f11ba-b750-406a-be72-acbbd203f340 req-914faf7c-a4c0-44af-be16-1011d1c18520 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:46:25.184415 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-3d7f11ba-b750-406a-be72-acbbd203f340 req-914faf7c-a4c0-44af-be16-1011d1c18520 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:46:25.196225 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-3d7f11ba-b750-406a-be72-acbbd203f340 req-914faf7c-a4c0-44af-be16-1011d1c18520 service nova] Creating event network-changed:3f6f502f-e93a-46fe-b6c5-70ff0daa5af0 for instance a83a3f66-378b-40a4-baf0-0d67c58f1524 on nch2-master-1 Jul 16 21:46:25.202708 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-3d7f11ba-b750-406a-be72-acbbd203f340 req-914faf7c-a4c0-44af-be16-1011d1c18520 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.034383 Jul 16 21:46:25.204021 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2964/5924] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:46:25 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:46:25.824232 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-7c70eee3-019e-4f23-8e16-e449a7ca0e1b tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:46:25.824892 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-7c70eee3-019e-4f23-8e16-e449a7ca0e1b tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] [instance: 0b62eebf-c7ed-4fa1-a4d1-17947e5d8226] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:46:25.852389 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7c70eee3-019e-4f23-8e16-e449a7ca0e1b tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:46:25.854282 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7c70eee3-019e-4f23-8e16-e449a7ca0e1b tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:46:25.855181 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7c70eee3-019e-4f23-8e16-e449a7ca0e1b tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:46:25.855638 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7c70eee3-019e-4f23-8e16-e449a7ca0e1b tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:46:25.906423 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-7c70eee3-019e-4f23-8e16-e449a7ca0e1b tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] HTTP exception thrown: Instance 0b62eebf-c7ed-4fa1-a4d1-17947e5d8226 could not be found. Jul 16 21:46:25.908714 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-7c70eee3-019e-4f23-8e16-e449a7ca0e1b tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Request method failure captured: Jul 16 21:46:25.908714 ncdv-master-1 devstack@n-api.service[138680]: request: GET /compute/v2.1/servers/0b62eebf-c7ed-4fa1-a4d1-17947e5d8226 HTTP/1.1 Jul 16 21:46:25.908714 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:46:25.908714 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:46:25.908714 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:46:25.908714 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 0 Jul 16 21:46:25.908714 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:46:25.908714 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:46:25.908714 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:46:25.908714 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:46:25.908714 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0y_J0q_hylsRBv1tuKDuvEX4h0-KJOaBkYlsP5Ic98XDN3p0UKvwASS-7n2zGgAIPr7rp74I-PGAzLIn5xeDmpiRzwuaSoKYKWvZ4y05aUl8opleXxkVzk0FUY6GEflOf-1AEqT9qASXsqi7YhmGfOHelDZ7TomdNPlQ8IF_8wXqbKY Jul 16 21:46:25.908714 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:46:25.908714 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:46:25.908714 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:46:25.908714 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:46:25.908714 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:46:25.908714 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:46:25.908714 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: 77c9b7e351e24dc88207e9498f75f578 Jul 16 21:46:25.908714 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-TestNetworkAdvancedServerOps-375332741 Jul 16 21:46:25.908714 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:46:25.908714 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:46:25.908714 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/77c9b7e351e24dc88207e9498f75f578"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/77c9b7e351e24dc88207e9498f75f578"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_77c9b7e351e24dc88207e9498f75f578"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/77c9b7e351e24dc88207e9498f75f578"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:46:25.908714 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-TestNetworkAdvancedServerOps-375332741 Jul 16 21:46:25.908714 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: 77c9b7e351e24dc88207e9498f75f578 Jul 16 21:46:25.908714 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-TestNetworkAdvancedServerOps-375332741 Jul 16 21:46:25.908714 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-TestNetworkAdvancedServerOps-375332741-project Jul 16 21:46:25.908714 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:46:25.908714 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:46:25.908714 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: 833136cbdeab4b9e8b6dd2d24c7c8753 Jul 16 21:46:25.908714 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-TestNetworkAdvancedServerOps-375332741-project Jul 16 21:46:25.908714 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 21:46:25.908714 ncdv-master-1 devstack@n-api.service[138680]: exception: Instance 0b62eebf-c7ed-4fa1-a4d1-17947e5d8226 could not be found. Jul 16 21:46:25.908714 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': '0b62eebf-c7ed-4fa1-a4d1-17947e5d8226'} Jul 16 21:46:25.908714 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:46:25.908714 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:46:25.908714 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:46:25.908714 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:46:25.908714 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:46:25.908714 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:46:25.908714 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:46:25.908714 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:46:25.908714 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:46:25.908714 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:46:25.908714 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 0b62eebf-c7ed-4fa1-a4d1-17947e5d8226 could not be found. Jul 16 21:46:25.908714 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:46:25.908714 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:46:25.908714 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:46:25.908714 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:46:25.908714 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:46:25.908714 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:46:25.908714 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:46:25.908714 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:46:25.908714 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:46:25.908714 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:46:25.908714 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:46:25.908714 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:46:25.908714 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:46:25.908714 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:46:25.908714 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:46:25.908714 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:46:25.908714 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 0b62eebf-c7ed-4fa1-a4d1-17947e5d8226 could not be found. Jul 16 21:46:25.908714 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:46:25.908714 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:46:25.908714 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:46:25.914530 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:46:25.914530 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:46:25.914530 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:46:25.914530 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:46:25.914530 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:46:25.914530 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 0b62eebf-c7ed-4fa1-a4d1-17947e5d8226 could not be found. Jul 16 21:46:25.914530 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:46:25.917502 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-7c70eee3-019e-4f23-8e16-e449a7ca0e1b tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Returning 404 to user: Instance 0b62eebf-c7ed-4fa1-a4d1-17947e5d8226 could not be found. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:46:25.918782 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-7c70eee3-019e-4f23-8e16-e449a7ca0e1b tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] 10.222.0.16 "GET /compute/v2.1/servers/0b62eebf-c7ed-4fa1-a4d1-17947e5d8226" status: 404 len: 111 microversion: 2.1 time: 0.099131 Jul 16 21:46:25.923278 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2961/5925] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:46:25 2022] GET /compute/v2.1/servers/0b62eebf-c7ed-4fa1-a4d1-17947e5d8226 => generated 111 bytes in 105 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:46:31.698692 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-a829b486-6401-42f4-8134-2e180d663c87 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] Calling method '.version_select at 0x7fdd9d4921f0>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:46:32.147312 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-a829b486-6401-42f4-8134-2e180d663c87 tempest-TestNetworkAdvancedServerOps-375332741 tempest-TestNetworkAdvancedServerOps-375332741-project] 10.222.0.16 "DELETE /compute/v2.1/os-keypairs/tempest-TestNetworkAdvancedServerOps-800712517" status: 202 len: 0 microversion: 2.1 time: 0.465389 Jul 16 21:46:32.147910 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2965/5926] 10.222.0.16 () {58 vars in 1280 bytes} [Sat Jul 16 21:46:31 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestNetworkAdvancedServerOps-800712517 => generated 0 bytes in 467 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 21:46:33.966195 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-e38ef54d-3b82-4463-9140-baee68ae9a49 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:46:33.966915 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-e38ef54d-3b82-4463-9140-baee68ae9a49 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: 3618ceb8-9d10-4219-8885-d2d5004c71ad] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:46:33.972258 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_api:SELECT=66,nova_api:UPDATE=2 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:46:33.976210 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e38ef54d-3b82-4463-9140-baee68ae9a49 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:46:33.976744 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e38ef54d-3b82-4463-9140-baee68ae9a49 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:46:33.977553 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e38ef54d-3b82-4463-9140-baee68ae9a49 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:46:33.978406 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e38ef54d-3b82-4463-9140-baee68ae9a49 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:46:34.136162 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-e38ef54d-3b82-4463-9140-baee68ae9a49 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:46:34.144889 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-e38ef54d-3b82-4463-9140-baee68ae9a49 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad" status: 200 len: 1665 microversion: 2.1 time: 0.183130 Jul 16 21:46:34.147149 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2962/5927] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:46:33 2022] GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad => generated 1665 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:46:34.168386 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-511eaddc-51e9-4dd4-a3a4-0efc7c5a5727 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Action: 'action', calling method: >, body: {"createBackup": {"backup_type": "daily", "rotation": 2, "name": "tempest-backup-2-841938831"}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:46:34.169628 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-511eaddc-51e9-4dd4-a3a4-0efc7c5a5727 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: 3618ceb8-9d10-4219-8885-d2d5004c71ad] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:46:34.179796 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-511eaddc-51e9-4dd4-a3a4-0efc7c5a5727 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:46:34.187183 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-511eaddc-51e9-4dd4-a3a4-0efc7c5a5727 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:46:34.187183 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-511eaddc-51e9-4dd4-a3a4-0efc7c5a5727 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:46:34.187183 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-511eaddc-51e9-4dd4-a3a4-0efc7c5a5727 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:46:34.762513 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-511eaddc-51e9-4dd4-a3a4-0efc7c5a5727 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "POST /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad/action" status: 202 len: 0 microversion: 2.1 time: 0.598991 Jul 16 21:46:34.768382 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2966/5928] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 21:46:34 2022] POST /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad/action => generated 0 bytes in 601 msecs (HTTP/1.1 202) 10 headers in 456 bytes (1 switches on core 0) Jul 16 21:46:36.767274 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-4c3c7a39-df87-4904-9653-d6779916a5fb req-8d6d4b13-137f-4f71-ab49-a0c45ce531f4 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "3618ceb8-9d10-4219-8885-d2d5004c71ad", "name": "network-vif-unplugged", "status": "completed", "tag": "ef163361-e10a-4408-b519-5d053d2a55d6"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:46:36.776316 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-4c3c7a39-df87-4904-9653-d6779916a5fb req-8d6d4b13-137f-4f71-ab49-a0c45ce531f4 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:46:36.776767 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-4c3c7a39-df87-4904-9653-d6779916a5fb req-8d6d4b13-137f-4f71-ab49-a0c45ce531f4 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:46:36.791483 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-4c3c7a39-df87-4904-9653-d6779916a5fb req-8d6d4b13-137f-4f71-ab49-a0c45ce531f4 service nova] Creating event network-vif-unplugged:ef163361-e10a-4408-b519-5d053d2a55d6 for instance 3618ceb8-9d10-4219-8885-d2d5004c71ad on nch1-master-1 Jul 16 21:46:36.797850 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-4c3c7a39-df87-4904-9653-d6779916a5fb req-8d6d4b13-137f-4f71-ab49-a0c45ce531f4 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.034546 Jul 16 21:46:36.798457 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2963/5929] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:46:36 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:46:40.873533 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-8cc04fb9-5bea-4170-8bf3-119e72c136f3 req-d66f8099-386d-496d-8752-02aa53c23997 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "3618ceb8-9d10-4219-8885-d2d5004c71ad", "name": "network-vif-plugged", "status": "completed", "tag": "ef163361-e10a-4408-b519-5d053d2a55d6"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:46:40.882699 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-8cc04fb9-5bea-4170-8bf3-119e72c136f3 req-d66f8099-386d-496d-8752-02aa53c23997 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:46:40.883304 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-8cc04fb9-5bea-4170-8bf3-119e72c136f3 req-d66f8099-386d-496d-8752-02aa53c23997 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:46:40.900957 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-8cc04fb9-5bea-4170-8bf3-119e72c136f3 req-d66f8099-386d-496d-8752-02aa53c23997 service nova] Creating event network-vif-plugged:ef163361-e10a-4408-b519-5d053d2a55d6 for instance 3618ceb8-9d10-4219-8885-d2d5004c71ad on nch1-master-1 Jul 16 21:46:40.906694 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-8cc04fb9-5bea-4170-8bf3-119e72c136f3 req-d66f8099-386d-496d-8752-02aa53c23997 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.039237 Jul 16 21:46:40.907850 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2967/5930] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:46:40 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 16 21:46:51.371054 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-1abebcdd-4122-4da4-b9ac-9bb735194d96 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:46:51.371836 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-1abebcdd-4122-4da4-b9ac-9bb735194d96 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: 3618ceb8-9d10-4219-8885-d2d5004c71ad] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:46:51.379174 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_api:SELECT=3 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:46:51.379858 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=95,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:46:51.381513 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1abebcdd-4122-4da4-b9ac-9bb735194d96 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:46:51.382002 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1abebcdd-4122-4da4-b9ac-9bb735194d96 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:46:51.382818 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1abebcdd-4122-4da4-b9ac-9bb735194d96 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:46:51.383285 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1abebcdd-4122-4da4-b9ac-9bb735194d96 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:46:51.524924 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-1abebcdd-4122-4da4-b9ac-9bb735194d96 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:46:51.534300 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-1abebcdd-4122-4da4-b9ac-9bb735194d96 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad" status: 200 len: 1678 microversion: 2.1 time: 0.168278 Jul 16 21:46:51.534890 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2964/5931] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:46:51 2022] GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad => generated 1678 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:46:52.550472 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_api:SELECT=8,nova_api:DELETE=1 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:46:52.551353 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=78,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:46:52.560341 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-922fe71c-eb17-4d3f-bd19-2680c4906031 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:46:52.560341 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-922fe71c-eb17-4d3f-bd19-2680c4906031 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: 3618ceb8-9d10-4219-8885-d2d5004c71ad] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:46:52.562387 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-922fe71c-eb17-4d3f-bd19-2680c4906031 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:46:52.562853 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-922fe71c-eb17-4d3f-bd19-2680c4906031 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:46:52.563712 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-922fe71c-eb17-4d3f-bd19-2680c4906031 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:46:52.564168 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-922fe71c-eb17-4d3f-bd19-2680c4906031 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:46:52.780523 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-922fe71c-eb17-4d3f-bd19-2680c4906031 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:46:52.788388 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-922fe71c-eb17-4d3f-bd19-2680c4906031 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad" status: 200 len: 1665 microversion: 2.1 time: 0.239748 Jul 16 21:46:52.788862 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2968/5932] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:46:52 2022] GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad => generated 1665 bytes in 241 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:46:52.806234 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-5195e209-6e83-4326-abd8-e55008c9b196 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Action: 'action', calling method: >, body: {"createBackup": {"backup_type": "daily", "rotation": 2, "name": "tempest-backup-3-780729306"}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:46:52.807357 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-5195e209-6e83-4326-abd8-e55008c9b196 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: 3618ceb8-9d10-4219-8885-d2d5004c71ad] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:46:52.814985 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5195e209-6e83-4326-abd8-e55008c9b196 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:46:52.817227 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5195e209-6e83-4326-abd8-e55008c9b196 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:46:52.817227 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5195e209-6e83-4326-abd8-e55008c9b196 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:46:52.817227 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5195e209-6e83-4326-abd8-e55008c9b196 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:46:53.355884 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-5195e209-6e83-4326-abd8-e55008c9b196 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "POST /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad/action" status: 202 len: 0 microversion: 2.1 time: 0.553950 Jul 16 21:46:53.365648 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2965/5933] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 21:46:52 2022] POST /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad/action => generated 0 bytes in 556 msecs (HTTP/1.1 202) 10 headers in 456 bytes (1 switches on core 0) Jul 16 21:46:53.368343 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-26655019-e829-437b-89ca-c8591094df19 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:46:53.368580 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-26655019-e829-437b-89ca-c8591094df19 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: 3618ceb8-9d10-4219-8885-d2d5004c71ad] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:46:53.377342 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-26655019-e829-437b-89ca-c8591094df19 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:46:53.377814 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-26655019-e829-437b-89ca-c8591094df19 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:46:53.378612 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-26655019-e829-437b-89ca-c8591094df19 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:46:53.379138 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-26655019-e829-437b-89ca-c8591094df19 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:46:53.594528 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-26655019-e829-437b-89ca-c8591094df19 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:46:53.602818 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-26655019-e829-437b-89ca-c8591094df19 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad" status: 200 len: 1675 microversion: 2.1 time: 0.241474 Jul 16 21:46:53.603441 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2969/5934] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:46:53 2022] GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad => generated 1675 bytes in 243 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:46:54.621962 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-4dc8a319-d8af-4ae1-a2eb-185c012c70c0 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:46:54.622807 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-4dc8a319-d8af-4ae1-a2eb-185c012c70c0 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: 3618ceb8-9d10-4219-8885-d2d5004c71ad] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:46:54.629867 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4dc8a319-d8af-4ae1-a2eb-185c012c70c0 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:46:54.630729 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4dc8a319-d8af-4ae1-a2eb-185c012c70c0 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:46:54.631538 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4dc8a319-d8af-4ae1-a2eb-185c012c70c0 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:46:54.631980 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4dc8a319-d8af-4ae1-a2eb-185c012c70c0 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:46:54.790190 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-4dc8a319-d8af-4ae1-a2eb-185c012c70c0 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:46:54.800463 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-4dc8a319-d8af-4ae1-a2eb-185c012c70c0 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad" status: 200 len: 1675 microversion: 2.1 time: 0.182607 Jul 16 21:46:54.810855 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2966/5935] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:46:54 2022] GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad => generated 1675 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:46:54.840319 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-802e226f-f402-44cf-a7ed-eab7b00863d7 req-ddd1a064-2f52-48de-82c9-8884aca852b5 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "3618ceb8-9d10-4219-8885-d2d5004c71ad", "name": "network-vif-unplugged", "status": "completed", "tag": "ef163361-e10a-4408-b519-5d053d2a55d6"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:46:54.850144 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-802e226f-f402-44cf-a7ed-eab7b00863d7 req-ddd1a064-2f52-48de-82c9-8884aca852b5 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:46:54.850605 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-802e226f-f402-44cf-a7ed-eab7b00863d7 req-ddd1a064-2f52-48de-82c9-8884aca852b5 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:46:54.862075 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-802e226f-f402-44cf-a7ed-eab7b00863d7 req-ddd1a064-2f52-48de-82c9-8884aca852b5 service nova] Creating event network-vif-unplugged:ef163361-e10a-4408-b519-5d053d2a55d6 for instance 3618ceb8-9d10-4219-8885-d2d5004c71ad on nch1-master-1 Jul 16 21:46:54.867374 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-802e226f-f402-44cf-a7ed-eab7b00863d7 req-ddd1a064-2f52-48de-82c9-8884aca852b5 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.030769 Jul 16 21:46:54.867974 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2970/5936] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:46:54 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:46:55.817741 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-16619a9f-333b-4355-8655-6d73f36b31c5 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:46:55.818363 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-16619a9f-333b-4355-8655-6d73f36b31c5 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: 3618ceb8-9d10-4219-8885-d2d5004c71ad] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:46:55.825984 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-16619a9f-333b-4355-8655-6d73f36b31c5 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:46:55.830866 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-16619a9f-333b-4355-8655-6d73f36b31c5 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:46:55.832167 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-16619a9f-333b-4355-8655-6d73f36b31c5 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:46:55.832167 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-16619a9f-333b-4355-8655-6d73f36b31c5 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:46:55.969417 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-16619a9f-333b-4355-8655-6d73f36b31c5 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:46:55.977082 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-16619a9f-333b-4355-8655-6d73f36b31c5 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad" status: 200 len: 1683 microversion: 2.1 time: 0.163267 Jul 16 21:46:55.977904 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2967/5937] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:46:55 2022] GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad => generated 1683 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:46:56.995844 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-73db6239-e944-426d-a293-8c9a68433512 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:46:56.996656 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-73db6239-e944-426d-a293-8c9a68433512 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: 3618ceb8-9d10-4219-8885-d2d5004c71ad] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:46:57.005225 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-73db6239-e944-426d-a293-8c9a68433512 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:46:57.005940 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-73db6239-e944-426d-a293-8c9a68433512 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:46:57.007202 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-73db6239-e944-426d-a293-8c9a68433512 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:46:57.007202 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-73db6239-e944-426d-a293-8c9a68433512 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:46:57.168096 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-73db6239-e944-426d-a293-8c9a68433512 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:46:57.178823 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-73db6239-e944-426d-a293-8c9a68433512 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad" status: 200 len: 1683 microversion: 2.1 time: 0.186588 Jul 16 21:46:57.178823 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2971/5938] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:46:56 2022] GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad => generated 1683 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:46:58.195431 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-517f3973-4535-4c67-bca2-8882ccd6261f tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:46:58.196100 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-517f3973-4535-4c67-bca2-8882ccd6261f tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: 3618ceb8-9d10-4219-8885-d2d5004c71ad] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:46:58.211794 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-517f3973-4535-4c67-bca2-8882ccd6261f tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:46:58.219710 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-517f3973-4535-4c67-bca2-8882ccd6261f tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:46:58.220733 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-517f3973-4535-4c67-bca2-8882ccd6261f tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:46:58.222136 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-517f3973-4535-4c67-bca2-8882ccd6261f tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:46:58.448438 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-517f3973-4535-4c67-bca2-8882ccd6261f tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:46:58.461048 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-517f3973-4535-4c67-bca2-8882ccd6261f tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad" status: 200 len: 1678 microversion: 2.1 time: 0.270019 Jul 16 21:46:58.461764 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2968/5939] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:46:58 2022] GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad => generated 1678 bytes in 272 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:46:59.025162 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-0c72828b-e71e-4741-b25f-0fccc3aeafb9 req-3f577afa-3732-4c0b-9c98-fd10d89ea219 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "3618ceb8-9d10-4219-8885-d2d5004c71ad", "name": "network-vif-plugged", "status": "completed", "tag": "ef163361-e10a-4408-b519-5d053d2a55d6"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:46:59.044545 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-0c72828b-e71e-4741-b25f-0fccc3aeafb9 req-3f577afa-3732-4c0b-9c98-fd10d89ea219 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:46:59.045294 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-0c72828b-e71e-4741-b25f-0fccc3aeafb9 req-3f577afa-3732-4c0b-9c98-fd10d89ea219 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:46:59.068519 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-0c72828b-e71e-4741-b25f-0fccc3aeafb9 req-3f577afa-3732-4c0b-9c98-fd10d89ea219 service nova] Creating event network-vif-plugged:ef163361-e10a-4408-b519-5d053d2a55d6 for instance 3618ceb8-9d10-4219-8885-d2d5004c71ad on nch1-master-1 Jul 16 21:46:59.075494 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-0c72828b-e71e-4741-b25f-0fccc3aeafb9 req-3f577afa-3732-4c0b-9c98-fd10d89ea219 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.056552 Jul 16 21:46:59.076661 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2972/5940] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:46:59 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 61 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 16 21:46:59.483648 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-635d4173-bc6f-42a5-ae76-5634766fe9ad tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:46:59.500343 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-635d4173-bc6f-42a5-ae76-5634766fe9ad tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: 3618ceb8-9d10-4219-8885-d2d5004c71ad] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:46:59.518422 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-635d4173-bc6f-42a5-ae76-5634766fe9ad tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:46:59.519902 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-635d4173-bc6f-42a5-ae76-5634766fe9ad tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:46:59.521360 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-635d4173-bc6f-42a5-ae76-5634766fe9ad tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:46:59.522095 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-635d4173-bc6f-42a5-ae76-5634766fe9ad tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:46:59.732786 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-635d4173-bc6f-42a5-ae76-5634766fe9ad tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:46:59.751462 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-635d4173-bc6f-42a5-ae76-5634766fe9ad tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad" status: 200 len: 1678 microversion: 2.1 time: 0.275290 Jul 16 21:46:59.753003 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2969/5941] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:46:59 2022] GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad => generated 1678 bytes in 278 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:47:00.784511 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-969e47b6-5d18-406b-99ab-ff96031c25e9 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:47:00.786158 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-969e47b6-5d18-406b-99ab-ff96031c25e9 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: 3618ceb8-9d10-4219-8885-d2d5004c71ad] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:47:00.837519 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-969e47b6-5d18-406b-99ab-ff96031c25e9 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:47:00.838617 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-969e47b6-5d18-406b-99ab-ff96031c25e9 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:47:00.840858 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-969e47b6-5d18-406b-99ab-ff96031c25e9 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:47:00.844372 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-969e47b6-5d18-406b-99ab-ff96031c25e9 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:47:01.151058 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-969e47b6-5d18-406b-99ab-ff96031c25e9 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:47:01.160550 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-969e47b6-5d18-406b-99ab-ff96031c25e9 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad" status: 200 len: 1678 microversion: 2.1 time: 0.394172 Jul 16 21:47:01.176523 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2973/5942] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:47:00 2022] GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad => generated 1678 bytes in 411 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:47:02.184578 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-2cfcc006-b96e-445a-9bca-8107247692fd tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:47:02.185208 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-2cfcc006-b96e-445a-9bca-8107247692fd tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: 3618ceb8-9d10-4219-8885-d2d5004c71ad] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:47:02.198249 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2cfcc006-b96e-445a-9bca-8107247692fd tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:47:02.198705 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2cfcc006-b96e-445a-9bca-8107247692fd tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:47:02.201251 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2cfcc006-b96e-445a-9bca-8107247692fd tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:47:02.201701 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2cfcc006-b96e-445a-9bca-8107247692fd tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:47:02.417653 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-2cfcc006-b96e-445a-9bca-8107247692fd tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:47:02.426659 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-2cfcc006-b96e-445a-9bca-8107247692fd tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad" status: 200 len: 1678 microversion: 2.1 time: 0.246360 Jul 16 21:47:02.439686 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2970/5943] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:47:02 2022] GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad => generated 1678 bytes in 260 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:47:03.445708 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-49aac3dd-4a0f-4579-abe9-de4d74c5be60 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:47:03.451723 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-49aac3dd-4a0f-4579-abe9-de4d74c5be60 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: 3618ceb8-9d10-4219-8885-d2d5004c71ad] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:47:03.460837 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-49aac3dd-4a0f-4579-abe9-de4d74c5be60 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:47:03.461738 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-49aac3dd-4a0f-4579-abe9-de4d74c5be60 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:47:03.462831 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-49aac3dd-4a0f-4579-abe9-de4d74c5be60 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:47:03.463776 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-49aac3dd-4a0f-4579-abe9-de4d74c5be60 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:47:03.615668 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-49aac3dd-4a0f-4579-abe9-de4d74c5be60 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:47:03.626127 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-49aac3dd-4a0f-4579-abe9-de4d74c5be60 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad" status: 200 len: 1678 microversion: 2.1 time: 0.183868 Jul 16 21:47:03.631454 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2974/5944] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:47:03 2022] GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad => generated 1678 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:47:04.647673 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-78ed9c31-4e62-4e44-bb53-619f087c0861 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:47:04.648493 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-78ed9c31-4e62-4e44-bb53-619f087c0861 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: 3618ceb8-9d10-4219-8885-d2d5004c71ad] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:47:04.657926 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-78ed9c31-4e62-4e44-bb53-619f087c0861 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:47:04.658523 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-78ed9c31-4e62-4e44-bb53-619f087c0861 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:47:04.659456 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-78ed9c31-4e62-4e44-bb53-619f087c0861 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:47:04.659993 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-78ed9c31-4e62-4e44-bb53-619f087c0861 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:47:04.856262 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-78ed9c31-4e62-4e44-bb53-619f087c0861 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:47:04.856262 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-78ed9c31-4e62-4e44-bb53-619f087c0861 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad" status: 200 len: 1678 microversion: 2.1 time: 0.208265 Jul 16 21:47:04.856262 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2971/5945] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:47:04 2022] GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad => generated 1678 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:47:05.959276 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-c42173dc-f3cc-41df-ac13-cb7539f3fa81 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:47:05.960496 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-c42173dc-f3cc-41df-ac13-cb7539f3fa81 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: 3618ceb8-9d10-4219-8885-d2d5004c71ad] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:47:05.971706 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c42173dc-f3cc-41df-ac13-cb7539f3fa81 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:47:05.985224 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c42173dc-f3cc-41df-ac13-cb7539f3fa81 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.009s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:47:05.989682 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c42173dc-f3cc-41df-ac13-cb7539f3fa81 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:47:05.990464 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c42173dc-f3cc-41df-ac13-cb7539f3fa81 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:47:06.230524 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-c42173dc-f3cc-41df-ac13-cb7539f3fa81 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:47:06.239072 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-c42173dc-f3cc-41df-ac13-cb7539f3fa81 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad" status: 200 len: 1678 microversion: 2.1 time: 0.287124 Jul 16 21:47:06.248777 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2975/5946] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:47:05 2022] GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad => generated 1678 bytes in 298 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:47:07.261276 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-c1b11ea0-8da3-4eca-9a96-fe5aeb754de2 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:47:07.261797 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-c1b11ea0-8da3-4eca-9a96-fe5aeb754de2 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: 3618ceb8-9d10-4219-8885-d2d5004c71ad] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:47:07.283510 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c1b11ea0-8da3-4eca-9a96-fe5aeb754de2 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:47:07.295550 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c1b11ea0-8da3-4eca-9a96-fe5aeb754de2 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:47:07.302073 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c1b11ea0-8da3-4eca-9a96-fe5aeb754de2 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:47:07.303061 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c1b11ea0-8da3-4eca-9a96-fe5aeb754de2 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:47:07.566386 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-c1b11ea0-8da3-4eca-9a96-fe5aeb754de2 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:47:07.574626 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-c1b11ea0-8da3-4eca-9a96-fe5aeb754de2 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad" status: 200 len: 1678 microversion: 2.1 time: 0.321621 Jul 16 21:47:07.575576 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2972/5947] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:47:07 2022] GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad => generated 1678 bytes in 323 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:47:08.593054 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-6f2b5cfa-40a2-460c-a361-628cbe2f6b7d tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:47:08.593777 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-6f2b5cfa-40a2-460c-a361-628cbe2f6b7d tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: 3618ceb8-9d10-4219-8885-d2d5004c71ad] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:47:08.601781 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6f2b5cfa-40a2-460c-a361-628cbe2f6b7d tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:47:08.614442 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6f2b5cfa-40a2-460c-a361-628cbe2f6b7d tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:47:08.614442 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6f2b5cfa-40a2-460c-a361-628cbe2f6b7d tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:47:08.614442 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6f2b5cfa-40a2-460c-a361-628cbe2f6b7d tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:47:08.826119 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-6f2b5cfa-40a2-460c-a361-628cbe2f6b7d tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:47:08.834320 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-6f2b5cfa-40a2-460c-a361-628cbe2f6b7d tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad" status: 200 len: 1678 microversion: 2.1 time: 0.245263 Jul 16 21:47:08.835644 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2976/5948] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:47:08 2022] GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad => generated 1678 bytes in 248 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:47:09.852554 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-c1a195e0-5e61-4a14-a3a0-9d8754a69134 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:47:09.853175 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-c1a195e0-5e61-4a14-a3a0-9d8754a69134 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: 3618ceb8-9d10-4219-8885-d2d5004c71ad] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:47:09.863412 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c1a195e0-5e61-4a14-a3a0-9d8754a69134 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:47:09.866324 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c1a195e0-5e61-4a14-a3a0-9d8754a69134 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:47:09.866324 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c1a195e0-5e61-4a14-a3a0-9d8754a69134 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:47:09.866324 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c1a195e0-5e61-4a14-a3a0-9d8754a69134 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:47:10.099153 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-c1a195e0-5e61-4a14-a3a0-9d8754a69134 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:47:10.107219 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-c1a195e0-5e61-4a14-a3a0-9d8754a69134 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad" status: 200 len: 1678 microversion: 2.1 time: 0.258799 Jul 16 21:47:10.107862 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2973/5949] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:47:09 2022] GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad => generated 1678 bytes in 260 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:47:11.125920 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-d496199b-121b-4049-aa5a-28f550cb23e2 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:47:11.126622 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-d496199b-121b-4049-aa5a-28f550cb23e2 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: 3618ceb8-9d10-4219-8885-d2d5004c71ad] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:47:11.133815 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d496199b-121b-4049-aa5a-28f550cb23e2 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:47:11.134280 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d496199b-121b-4049-aa5a-28f550cb23e2 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:47:11.135126 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d496199b-121b-4049-aa5a-28f550cb23e2 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:47:11.135601 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d496199b-121b-4049-aa5a-28f550cb23e2 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:47:11.399975 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-d496199b-121b-4049-aa5a-28f550cb23e2 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:47:11.401204 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-d496199b-121b-4049-aa5a-28f550cb23e2 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad" status: 200 len: 1665 microversion: 2.1 time: 0.279864 Jul 16 21:47:11.401772 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2977/5950] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:47:11 2022] GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad => generated 1665 bytes in 281 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:47:12.538479 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-fbe7e945-a074-4bc8-8f6f-aa8ec4bfc897 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:47:12.539121 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-fbe7e945-a074-4bc8-8f6f-aa8ec4bfc897 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: 3618ceb8-9d10-4219-8885-d2d5004c71ad] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:47:12.547311 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-fbe7e945-a074-4bc8-8f6f-aa8ec4bfc897 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:47:12.547873 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-fbe7e945-a074-4bc8-8f6f-aa8ec4bfc897 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:47:12.548677 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-fbe7e945-a074-4bc8-8f6f-aa8ec4bfc897 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:47:12.549150 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-fbe7e945-a074-4bc8-8f6f-aa8ec4bfc897 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:47:12.684341 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-fbe7e945-a074-4bc8-8f6f-aa8ec4bfc897 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:47:12.692366 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-fbe7e945-a074-4bc8-8f6f-aa8ec4bfc897 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad" status: 200 len: 1665 microversion: 2.1 time: 0.157870 Jul 16 21:47:12.692981 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2974/5951] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:47:12 2022] GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad => generated 1665 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:47:14.882073 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-89ffe01c-d20d-4289-b67f-7a489ab4ca67 req-7ad95cc5-ddaf-43d3-a8d4-f7a3eccbf71a service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "3618ceb8-9d10-4219-8885-d2d5004c71ad", "name": "network-vif-unplugged", "status": "completed", "tag": "ef163361-e10a-4408-b519-5d053d2a55d6"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:47:14.892317 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-89ffe01c-d20d-4289-b67f-7a489ab4ca67 req-7ad95cc5-ddaf-43d3-a8d4-f7a3eccbf71a service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:47:14.892771 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-89ffe01c-d20d-4289-b67f-7a489ab4ca67 req-7ad95cc5-ddaf-43d3-a8d4-f7a3eccbf71a service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:47:14.907027 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-89ffe01c-d20d-4289-b67f-7a489ab4ca67 req-7ad95cc5-ddaf-43d3-a8d4-f7a3eccbf71a service nova] Creating event network-vif-unplugged:ef163361-e10a-4408-b519-5d053d2a55d6 for instance 3618ceb8-9d10-4219-8885-d2d5004c71ad on nch1-master-1 Jul 16 21:47:14.909903 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-89ffe01c-d20d-4289-b67f-7a489ab4ca67 req-7ad95cc5-ddaf-43d3-a8d4-f7a3eccbf71a service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.031661 Jul 16 21:47:14.910540 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2978/5952] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:47:14 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:47:15.459991 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-40f2c59c-fdf0-41c7-b307-2651ce60d94e tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:47:15.462426 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-40f2c59c-fdf0-41c7-b307-2651ce60d94e tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: 3618ceb8-9d10-4219-8885-d2d5004c71ad] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:47:15.467663 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-40f2c59c-fdf0-41c7-b307-2651ce60d94e tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:47:15.468117 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-40f2c59c-fdf0-41c7-b307-2651ce60d94e tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:47:15.468891 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-40f2c59c-fdf0-41c7-b307-2651ce60d94e tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:47:15.469333 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-40f2c59c-fdf0-41c7-b307-2651ce60d94e tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:47:15.624303 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-40f2c59c-fdf0-41c7-b307-2651ce60d94e tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:47:15.633301 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-40f2c59c-fdf0-41c7-b307-2651ce60d94e tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad" status: 200 len: 1665 microversion: 2.1 time: 0.177210 Jul 16 21:47:15.633876 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2975/5953] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:47:15 2022] GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad => generated 1665 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:47:15.650474 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-8dd66981-3551-46ec-ba24-7445771639b0 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:47:15.651246 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-8dd66981-3551-46ec-ba24-7445771639b0 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: 3618ceb8-9d10-4219-8885-d2d5004c71ad] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:47:15.658512 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8dd66981-3551-46ec-ba24-7445771639b0 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:47:15.659052 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8dd66981-3551-46ec-ba24-7445771639b0 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:47:15.659842 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8dd66981-3551-46ec-ba24-7445771639b0 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:47:15.660295 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8dd66981-3551-46ec-ba24-7445771639b0 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:47:15.853810 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-8dd66981-3551-46ec-ba24-7445771639b0 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:47:15.861673 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-8dd66981-3551-46ec-ba24-7445771639b0 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad" status: 200 len: 1665 microversion: 2.1 time: 0.215529 Jul 16 21:47:15.862236 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2979/5954] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:47:15 2022] GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad => generated 1665 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:47:15.878289 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-b8fdb0a1-530c-49c0-803c-58b3f1b354fa tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:47:15.878894 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-b8fdb0a1-530c-49c0-803c-58b3f1b354fa tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: 3618ceb8-9d10-4219-8885-d2d5004c71ad] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:47:15.885094 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b8fdb0a1-530c-49c0-803c-58b3f1b354fa tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:47:15.885543 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b8fdb0a1-530c-49c0-803c-58b3f1b354fa tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:47:15.886302 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b8fdb0a1-530c-49c0-803c-58b3f1b354fa tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:47:15.886740 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b8fdb0a1-530c-49c0-803c-58b3f1b354fa tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:47:16.103539 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-b8fdb0a1-530c-49c0-803c-58b3f1b354fa tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:47:16.111711 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-b8fdb0a1-530c-49c0-803c-58b3f1b354fa tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad" status: 200 len: 1665 microversion: 2.1 time: 0.237982 Jul 16 21:47:16.112293 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2976/5955] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:47:15 2022] GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad => generated 1665 bytes in 239 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:47:16.128934 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-abb2e775-31a4-4ef7-9dde-2e8c6c933ce0 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Action: 'action', calling method: >, body: {"reboot": {"type": "HARD"}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:47:16.129920 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-abb2e775-31a4-4ef7-9dde-2e8c6c933ce0 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: 3618ceb8-9d10-4219-8885-d2d5004c71ad] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:47:16.136910 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-abb2e775-31a4-4ef7-9dde-2e8c6c933ce0 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:47:16.137375 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-abb2e775-31a4-4ef7-9dde-2e8c6c933ce0 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:47:16.138169 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-abb2e775-31a4-4ef7-9dde-2e8c6c933ce0 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:47:16.138622 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-abb2e775-31a4-4ef7-9dde-2e8c6c933ce0 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:47:16.375672 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-abb2e775-31a4-4ef7-9dde-2e8c6c933ce0 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "POST /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad/action" status: 202 len: 0 microversion: 2.1 time: 0.251281 Jul 16 21:47:16.380043 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2980/5956] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 21:47:16 2022] POST /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad/action => generated 0 bytes in 254 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 21:47:16.386092 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-3fd633bf-d817-4db2-b11e-1506e6abb6d4 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:47:16.387189 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-3fd633bf-d817-4db2-b11e-1506e6abb6d4 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: 3618ceb8-9d10-4219-8885-d2d5004c71ad] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:47:16.395729 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3fd633bf-d817-4db2-b11e-1506e6abb6d4 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:47:16.396471 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3fd633bf-d817-4db2-b11e-1506e6abb6d4 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:47:16.397599 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3fd633bf-d817-4db2-b11e-1506e6abb6d4 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:47:16.398213 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3fd633bf-d817-4db2-b11e-1506e6abb6d4 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:47:16.578913 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-3fd633bf-d817-4db2-b11e-1506e6abb6d4 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:47:16.584122 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-3fd633bf-d817-4db2-b11e-1506e6abb6d4 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad" status: 200 len: 1667 microversion: 2.1 time: 0.202649 Jul 16 21:47:16.585343 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2977/5957] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:47:16 2022] GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad => generated 1667 bytes in 205 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:47:17.606804 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-e8d5068f-c2be-435f-9152-3c6a31e53ac7 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:47:17.608718 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-e8d5068f-c2be-435f-9152-3c6a31e53ac7 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: 3618ceb8-9d10-4219-8885-d2d5004c71ad] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:47:17.615909 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e8d5068f-c2be-435f-9152-3c6a31e53ac7 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:47:17.622791 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e8d5068f-c2be-435f-9152-3c6a31e53ac7 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:47:17.622791 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e8d5068f-c2be-435f-9152-3c6a31e53ac7 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:47:17.622791 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e8d5068f-c2be-435f-9152-3c6a31e53ac7 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:47:17.867773 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-e8d5068f-c2be-435f-9152-3c6a31e53ac7 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:47:17.876561 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-e8d5068f-c2be-435f-9152-3c6a31e53ac7 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad" status: 200 len: 1667 microversion: 2.1 time: 0.275977 Jul 16 21:47:17.881874 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2981/5958] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:47:17 2022] GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad => generated 1667 bytes in 283 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:47:18.898195 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-6ba6c928-8e80-4967-908d-7c84f0e76f3f tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:47:18.898906 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-6ba6c928-8e80-4967-908d-7c84f0e76f3f tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: 3618ceb8-9d10-4219-8885-d2d5004c71ad] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:47:18.905494 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6ba6c928-8e80-4967-908d-7c84f0e76f3f tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:47:18.905949 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6ba6c928-8e80-4967-908d-7c84f0e76f3f tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:47:18.906714 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6ba6c928-8e80-4967-908d-7c84f0e76f3f tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:47:18.907183 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6ba6c928-8e80-4967-908d-7c84f0e76f3f tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:47:19.049145 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-6ba6c928-8e80-4967-908d-7c84f0e76f3f tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:47:19.056839 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-6ba6c928-8e80-4967-908d-7c84f0e76f3f tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad" status: 200 len: 1665 microversion: 2.1 time: 0.162975 Jul 16 21:47:19.057581 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2978/5959] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:47:18 2022] GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad => generated 1665 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:47:19.084281 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-a67a6834-738a-482b-b0c1-87dba57ac257 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Action: 'action', calling method: >, body: {"os-getConsoleOutput": {"length": 3}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:47:19.085272 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-a67a6834-738a-482b-b0c1-87dba57ac257 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: 3618ceb8-9d10-4219-8885-d2d5004c71ad] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:47:19.092284 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a67a6834-738a-482b-b0c1-87dba57ac257 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:47:19.092746 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a67a6834-738a-482b-b0c1-87dba57ac257 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:47:19.093528 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a67a6834-738a-482b-b0c1-87dba57ac257 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:47:19.093978 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a67a6834-738a-482b-b0c1-87dba57ac257 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:47:19.144707 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-a67a6834-738a-482b-b0c1-87dba57ac257 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "POST /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad/action" status: 200 len: 322 microversion: 2.1 time: 0.064613 Jul 16 21:47:19.149465 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2982/5960] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 21:47:19 2022] POST /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad/action => generated 322 bytes in 66 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:47:19.154734 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-5efec679-6ec4-4556-b0bc-0bbb45375413 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:47:19.155475 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-5efec679-6ec4-4556-b0bc-0bbb45375413 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: 3618ceb8-9d10-4219-8885-d2d5004c71ad] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:47:19.162524 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5efec679-6ec4-4556-b0bc-0bbb45375413 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:47:19.163090 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5efec679-6ec4-4556-b0bc-0bbb45375413 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:47:19.164035 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5efec679-6ec4-4556-b0bc-0bbb45375413 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:47:19.164530 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5efec679-6ec4-4556-b0bc-0bbb45375413 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:47:19.168776 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:47:19.302624 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-5efec679-6ec4-4556-b0bc-0bbb45375413 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:47:19.310494 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-5efec679-6ec4-4556-b0bc-0bbb45375413 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad" status: 200 len: 1665 microversion: 2.1 time: 0.159395 Jul 16 21:47:19.310862 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2979/5961] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:47:19 2022] GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad => generated 1665 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:47:19.328640 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-408c89f2-7053-4bda-a736-50be50b7384e tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:47:19.329363 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-408c89f2-7053-4bda-a736-50be50b7384e tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: 3618ceb8-9d10-4219-8885-d2d5004c71ad] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:47:19.337501 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-408c89f2-7053-4bda-a736-50be50b7384e tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:47:19.338041 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-408c89f2-7053-4bda-a736-50be50b7384e tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:47:19.338864 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-408c89f2-7053-4bda-a736-50be50b7384e tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:47:19.339351 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-408c89f2-7053-4bda-a736-50be50b7384e tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:47:19.478730 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-408c89f2-7053-4bda-a736-50be50b7384e tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:47:19.486737 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-408c89f2-7053-4bda-a736-50be50b7384e tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad" status: 200 len: 1665 microversion: 2.1 time: 0.162218 Jul 16 21:47:19.487287 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2983/5962] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:47:19 2022] GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad => generated 1665 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:47:19.504306 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-430e0323-2aff-466d-a636-bac9a27ba19b tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:47:19.504932 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-430e0323-2aff-466d-a636-bac9a27ba19b tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: 3618ceb8-9d10-4219-8885-d2d5004c71ad] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:47:19.511563 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-430e0323-2aff-466d-a636-bac9a27ba19b tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:47:19.512027 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-430e0323-2aff-466d-a636-bac9a27ba19b tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:47:19.512800 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-430e0323-2aff-466d-a636-bac9a27ba19b tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:47:19.513246 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-430e0323-2aff-466d-a636-bac9a27ba19b tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:47:19.650188 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-430e0323-2aff-466d-a636-bac9a27ba19b tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:47:19.657726 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-430e0323-2aff-466d-a636-bac9a27ba19b tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad" status: 200 len: 1665 microversion: 2.1 time: 0.158119 Jul 16 21:47:19.658379 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2980/5963] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:47:19 2022] GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad => generated 1665 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:47:19.673945 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-69c8ecce-6644-4023-b1d4-21eee53ac527 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:47:19.674580 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-69c8ecce-6644-4023-b1d4-21eee53ac527 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: 3618ceb8-9d10-4219-8885-d2d5004c71ad] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:47:19.681323 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-69c8ecce-6644-4023-b1d4-21eee53ac527 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:47:19.681795 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-69c8ecce-6644-4023-b1d4-21eee53ac527 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:47:19.684237 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-69c8ecce-6644-4023-b1d4-21eee53ac527 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:47:19.684237 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-69c8ecce-6644-4023-b1d4-21eee53ac527 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:47:19.820518 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-69c8ecce-6644-4023-b1d4-21eee53ac527 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:47:19.829417 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-69c8ecce-6644-4023-b1d4-21eee53ac527 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad" status: 200 len: 1665 microversion: 2.1 time: 0.159503 Jul 16 21:47:19.830062 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2984/5964] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:47:19 2022] GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad => generated 1665 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:47:19.943116 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-851dce0b-173b-4f15-ac10-9233d4840c98 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerActionsTestJSON-server-102799881", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452", "networks": [{"uuid": "af28c06e-7ce2-4ca3-b6a4-f1ae8d70db1a"}]}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:47:19.998470 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.network.neutron [None req-851dce0b-173b-4f15-ac10-9233d4840c98 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] validate_networks() for [('af28c06e-7ce2-4ca3-b6a4-f1ae8d70db1a', None, None, None, None, None)] {{(pid=138680) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 16 21:47:20.749503 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-851dce0b-173b-4f15-ac10-9233d4840c98 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Flavor limits 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 16 21:47:20.749809 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-851dce0b-173b-4f15-ac10-9233d4840c98 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Image limits 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 16 21:47:20.750382 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-851dce0b-173b-4f15-ac10-9233d4840c98 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Flavor pref 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 16 21:47:20.750799 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-851dce0b-173b-4f15-ac10-9233d4840c98 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Image pref 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 16 21:47:20.751257 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-851dce0b-173b-4f15-ac10-9233d4840c98 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 16 21:47:20.760725 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-851dce0b-173b-4f15-ac10-9233d4840c98 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:47:20.761179 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-851dce0b-173b-4f15-ac10-9233d4840c98 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:47:20.761985 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-851dce0b-173b-4f15-ac10-9233d4840c98 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:47:20.762449 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-851dce0b-173b-4f15-ac10-9233d4840c98 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:47:20.780870 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-851dce0b-173b-4f15-ac10-9233d4840c98 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Getting quotas for project 1e7cd69b8ddd4aceb45a4aaacdfccea7. Resources: {'ram', 'instances', 'cores'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:47:20.786761 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-851dce0b-173b-4f15-ac10-9233d4840c98 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Getting quotas for user cfe293b9ed3645229bed5ef8ce17cfec and project 1e7cd69b8ddd4aceb45a4aaacdfccea7. Resources: {'ram', 'instances', 'cores'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:47:20.799477 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-851dce0b-173b-4f15-ac10-9233d4840c98 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Going to run 1 instances... {{(pid=138680) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 16 21:47:20.801705 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-851dce0b-173b-4f15-ac10-9233d4840c98 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: eb9f9cfd-8fe0-4803-8f57-e477e1efb7ad] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138680) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 16 21:47:20.916362 ncdv-master-1 devstack@n-api.service[138680]: ERROR oslo.messaging._drivers.impl_rabbit [None req-851dce0b-173b-4f15-ac10-9233d4840c98 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [be7d23ec-c955-4e1f-9879-b3520b4af3f4] AMQP server on 10.222.0.16:5672 is unreachable: [Errno 104] Connection reset by peer. Trying again in 1 seconds.: ConnectionResetError: [Errno 104] Connection reset by peer Jul 16 21:47:21.923794 ncdv-master-1 devstack@n-api.service[138680]: INFO oslo.messaging._drivers.impl_rabbit [None req-851dce0b-173b-4f15-ac10-9233d4840c98 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [be7d23ec-c955-4e1f-9879-b3520b4af3f4] Reconnected to AMQP server on 10.222.0.16:5672 via [amqp] client with port 50424. Jul 16 21:47:21.929449 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-851dce0b-173b-4f15-ac10-9233d4840c98 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 1.990947 Jul 16 21:47:21.930266 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2981/5965] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 21:47:19 2022] POST /compute/v2.1/servers => generated 380 bytes in 1992 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Jul 16 21:47:21.940562 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-3d81d207-2e4e-48d5-8085-0ee2aa152958 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:47:21.942605 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-3d81d207-2e4e-48d5-8085-0ee2aa152958 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: eb9f9cfd-8fe0-4803-8f57-e477e1efb7ad] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:47:22.008438 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-3d81d207-2e4e-48d5-8085-0ee2aa152958 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:47:22.019857 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-3d81d207-2e4e-48d5-8085-0ee2aa152958 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/eb9f9cfd-8fe0-4803-8f57-e477e1efb7ad" status: 200 len: 1214 microversion: 2.1 time: 0.083326 Jul 16 21:47:22.020668 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2985/5966] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:47:21 2022] GET /compute/v2.1/servers/eb9f9cfd-8fe0-4803-8f57-e477e1efb7ad => generated 1214 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:47:23.038364 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-eed32935-68a9-43dd-b702-8918d2766df4 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:47:23.038994 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-eed32935-68a9-43dd-b702-8918d2766df4 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: eb9f9cfd-8fe0-4803-8f57-e477e1efb7ad] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:47:23.041276 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_api:SELECT=48,nova_api:INSERT=3 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:47:23.046549 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-eed32935-68a9-43dd-b702-8918d2766df4 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:47:23.047104 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-eed32935-68a9-43dd-b702-8918d2766df4 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:47:23.047918 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-eed32935-68a9-43dd-b702-8918d2766df4 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:47:23.048357 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-eed32935-68a9-43dd-b702-8918d2766df4 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:47:23.167234 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-eed32935-68a9-43dd-b702-8918d2766df4 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:47:23.175571 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-eed32935-68a9-43dd-b702-8918d2766df4 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/eb9f9cfd-8fe0-4803-8f57-e477e1efb7ad" status: 200 len: 1218 microversion: 2.1 time: 0.142290 Jul 16 21:47:23.176335 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2982/5967] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:47:23 2022] GET /compute/v2.1/servers/eb9f9cfd-8fe0-4803-8f57-e477e1efb7ad => generated 1218 bytes in 144 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:47:24.193834 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-f8bad871-46df-4330-80cd-3a5301984d51 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:47:24.194484 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-f8bad871-46df-4330-80cd-3a5301984d51 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: eb9f9cfd-8fe0-4803-8f57-e477e1efb7ad] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:47:24.211133 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_api:SELECT=39 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:47:24.213066 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f8bad871-46df-4330-80cd-3a5301984d51 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:47:24.213544 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f8bad871-46df-4330-80cd-3a5301984d51 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:47:24.214357 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f8bad871-46df-4330-80cd-3a5301984d51 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:47:24.214999 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f8bad871-46df-4330-80cd-3a5301984d51 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:47:24.224068 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=93,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:47:24.372739 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-f8bad871-46df-4330-80cd-3a5301984d51 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:47:24.390724 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-f8bad871-46df-4330-80cd-3a5301984d51 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/eb9f9cfd-8fe0-4803-8f57-e477e1efb7ad" status: 200 len: 1274 microversion: 2.1 time: 0.201244 Jul 16 21:47:24.391964 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2986/5968] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:47:24 2022] GET /compute/v2.1/servers/eb9f9cfd-8fe0-4803-8f57-e477e1efb7ad => generated 1274 bytes in 204 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:47:25.412216 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-6a72089e-aa8c-477a-b1bb-18539e59f4c6 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:47:25.412853 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-6a72089e-aa8c-477a-b1bb-18539e59f4c6 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: eb9f9cfd-8fe0-4803-8f57-e477e1efb7ad] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:47:25.420725 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6a72089e-aa8c-477a-b1bb-18539e59f4c6 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:47:25.421422 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6a72089e-aa8c-477a-b1bb-18539e59f4c6 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:47:25.422402 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6a72089e-aa8c-477a-b1bb-18539e59f4c6 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:47:25.423363 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6a72089e-aa8c-477a-b1bb-18539e59f4c6 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:47:25.631692 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-6a72089e-aa8c-477a-b1bb-18539e59f4c6 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:47:25.640986 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-6a72089e-aa8c-477a-b1bb-18539e59f4c6 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/eb9f9cfd-8fe0-4803-8f57-e477e1efb7ad" status: 200 len: 1314 microversion: 2.1 time: 0.233517 Jul 16 21:47:25.642505 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2983/5969] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:47:25 2022] GET /compute/v2.1/servers/eb9f9cfd-8fe0-4803-8f57-e477e1efb7ad => generated 1314 bytes in 235 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:47:26.659071 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-9762f4b1-6fdd-49bf-b142-b2115452e600 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:47:26.660301 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-9762f4b1-6fdd-49bf-b142-b2115452e600 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: eb9f9cfd-8fe0-4803-8f57-e477e1efb7ad] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:47:26.668454 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9762f4b1-6fdd-49bf-b142-b2115452e600 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:47:26.669194 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9762f4b1-6fdd-49bf-b142-b2115452e600 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:47:26.670150 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9762f4b1-6fdd-49bf-b142-b2115452e600 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:47:26.670763 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9762f4b1-6fdd-49bf-b142-b2115452e600 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:47:26.953386 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-9762f4b1-6fdd-49bf-b142-b2115452e600 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:47:26.971296 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-9762f4b1-6fdd-49bf-b142-b2115452e600 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/eb9f9cfd-8fe0-4803-8f57-e477e1efb7ad" status: 200 len: 1314 microversion: 2.1 time: 0.315840 Jul 16 21:47:26.975429 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2987/5970] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:47:26 2022] GET /compute/v2.1/servers/eb9f9cfd-8fe0-4803-8f57-e477e1efb7ad => generated 1314 bytes in 321 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:47:27.430930 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-c0dd560d-c084-4946-a45f-e2368e911eea req-fe9bf0fd-93e4-4516-abd7-6d5afeed191d service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "eb9f9cfd-8fe0-4803-8f57-e477e1efb7ad", "tag": "9d393a2c-3f5b-46a5-a145-dd932faa5e91"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:47:27.446835 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-c0dd560d-c084-4946-a45f-e2368e911eea req-fe9bf0fd-93e4-4516-abd7-6d5afeed191d service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:47:27.449135 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-c0dd560d-c084-4946-a45f-e2368e911eea req-fe9bf0fd-93e4-4516-abd7-6d5afeed191d service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:47:27.461015 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-c0dd560d-c084-4946-a45f-e2368e911eea req-fe9bf0fd-93e4-4516-abd7-6d5afeed191d service nova] Creating event network-changed:9d393a2c-3f5b-46a5-a145-dd932faa5e91 for instance eb9f9cfd-8fe0-4803-8f57-e477e1efb7ad on nch1-master-1 Jul 16 21:47:27.465952 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-c0dd560d-c084-4946-a45f-e2368e911eea req-fe9bf0fd-93e4-4516-abd7-6d5afeed191d service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.226414 Jul 16 21:47:27.466838 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2984/5971] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:47:27 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:47:27.988497 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-f3837db1-105d-450d-b378-32e283d73836 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:47:27.989973 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-f3837db1-105d-450d-b378-32e283d73836 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: eb9f9cfd-8fe0-4803-8f57-e477e1efb7ad] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:47:27.997733 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f3837db1-105d-450d-b378-32e283d73836 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:47:28.000160 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f3837db1-105d-450d-b378-32e283d73836 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:47:28.000160 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f3837db1-105d-450d-b378-32e283d73836 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:47:28.000160 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f3837db1-105d-450d-b378-32e283d73836 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:47:28.241982 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-f3837db1-105d-450d-b378-32e283d73836 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:47:28.250303 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-f3837db1-105d-450d-b378-32e283d73836 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/eb9f9cfd-8fe0-4803-8f57-e477e1efb7ad" status: 200 len: 1314 microversion: 2.1 time: 0.265789 Jul 16 21:47:28.251141 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2988/5972] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:47:27 2022] GET /compute/v2.1/servers/eb9f9cfd-8fe0-4803-8f57-e477e1efb7ad => generated 1314 bytes in 267 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:47:29.270451 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-831151e4-4960-42a5-83d3-841974ef9f14 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:47:29.271330 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-831151e4-4960-42a5-83d3-841974ef9f14 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: eb9f9cfd-8fe0-4803-8f57-e477e1efb7ad] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:47:29.280270 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-831151e4-4960-42a5-83d3-841974ef9f14 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:47:29.280787 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-831151e4-4960-42a5-83d3-841974ef9f14 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:47:29.281667 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-831151e4-4960-42a5-83d3-841974ef9f14 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:47:29.282362 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-831151e4-4960-42a5-83d3-841974ef9f14 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:47:29.422368 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-831151e4-4960-42a5-83d3-841974ef9f14 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:47:29.431470 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-831151e4-4960-42a5-83d3-841974ef9f14 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/eb9f9cfd-8fe0-4803-8f57-e477e1efb7ad" status: 200 len: 1314 microversion: 2.1 time: 0.166766 Jul 16 21:47:29.432268 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2985/5973] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:47:29 2022] GET /compute/v2.1/servers/eb9f9cfd-8fe0-4803-8f57-e477e1efb7ad => generated 1314 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:47:30.447977 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-60bb3ca6-b87f-4ae0-910f-3fe9d0f75ff0 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:47:30.448615 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-60bb3ca6-b87f-4ae0-910f-3fe9d0f75ff0 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: eb9f9cfd-8fe0-4803-8f57-e477e1efb7ad] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:47:30.455733 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-60bb3ca6-b87f-4ae0-910f-3fe9d0f75ff0 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:47:30.456213 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-60bb3ca6-b87f-4ae0-910f-3fe9d0f75ff0 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:47:30.457005 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-60bb3ca6-b87f-4ae0-910f-3fe9d0f75ff0 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:47:30.457460 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-60bb3ca6-b87f-4ae0-910f-3fe9d0f75ff0 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:47:30.606001 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-60bb3ca6-b87f-4ae0-910f-3fe9d0f75ff0 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:47:30.614225 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-60bb3ca6-b87f-4ae0-910f-3fe9d0f75ff0 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/eb9f9cfd-8fe0-4803-8f57-e477e1efb7ad" status: 200 len: 1314 microversion: 2.1 time: 0.170147 Jul 16 21:47:30.614795 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2989/5974] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:47:30 2022] GET /compute/v2.1/servers/eb9f9cfd-8fe0-4803-8f57-e477e1efb7ad => generated 1314 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:47:31.631353 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-3af7ec14-ddd6-45a8-9700-bb97f5fdf16f tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:47:31.632140 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-3af7ec14-ddd6-45a8-9700-bb97f5fdf16f tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: eb9f9cfd-8fe0-4803-8f57-e477e1efb7ad] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:47:31.635145 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell0:SELECT=2 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:47:31.638519 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell0:SELECT=6 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:47:31.645141 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3af7ec14-ddd6-45a8-9700-bb97f5fdf16f tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:47:31.645597 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3af7ec14-ddd6-45a8-9700-bb97f5fdf16f tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:47:31.646414 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3af7ec14-ddd6-45a8-9700-bb97f5fdf16f tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:47:31.646854 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3af7ec14-ddd6-45a8-9700-bb97f5fdf16f tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:47:31.789389 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-3af7ec14-ddd6-45a8-9700-bb97f5fdf16f tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:47:31.797270 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-3af7ec14-ddd6-45a8-9700-bb97f5fdf16f tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/eb9f9cfd-8fe0-4803-8f57-e477e1efb7ad" status: 200 len: 1314 microversion: 2.1 time: 0.170986 Jul 16 21:47:31.797854 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2986/5975] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:47:31 2022] GET /compute/v2.1/servers/eb9f9cfd-8fe0-4803-8f57-e477e1efb7ad => generated 1314 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:47:32.814052 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-2c30b562-c207-4b21-aa42-2227da2e5142 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:47:32.815824 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-2c30b562-c207-4b21-aa42-2227da2e5142 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: eb9f9cfd-8fe0-4803-8f57-e477e1efb7ad] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:47:32.822219 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell0:SELECT=2 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:47:32.824039 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2c30b562-c207-4b21-aa42-2227da2e5142 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:47:32.829230 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2c30b562-c207-4b21-aa42-2227da2e5142 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:47:32.829230 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2c30b562-c207-4b21-aa42-2227da2e5142 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:47:32.829230 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2c30b562-c207-4b21-aa42-2227da2e5142 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:47:32.976818 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-2c30b562-c207-4b21-aa42-2227da2e5142 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:47:32.986086 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-2c30b562-c207-4b21-aa42-2227da2e5142 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/eb9f9cfd-8fe0-4803-8f57-e477e1efb7ad" status: 200 len: 1314 microversion: 2.1 time: 0.176130 Jul 16 21:47:32.987552 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2990/5976] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:47:32 2022] GET /compute/v2.1/servers/eb9f9cfd-8fe0-4803-8f57-e477e1efb7ad => generated 1314 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:47:33.036808 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-6e966f40-d3ca-4770-addb-deabf84d88ec req-f056b49e-441f-4bce-bdaf-35a53030735b service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "eb9f9cfd-8fe0-4803-8f57-e477e1efb7ad", "name": "network-vif-plugged", "status": "completed", "tag": "9d393a2c-3f5b-46a5-a145-dd932faa5e91"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:47:33.046765 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-6e966f40-d3ca-4770-addb-deabf84d88ec req-f056b49e-441f-4bce-bdaf-35a53030735b service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:47:33.047280 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-6e966f40-d3ca-4770-addb-deabf84d88ec req-f056b49e-441f-4bce-bdaf-35a53030735b service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:47:33.060377 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-6e966f40-d3ca-4770-addb-deabf84d88ec req-f056b49e-441f-4bce-bdaf-35a53030735b service nova] Creating event network-vif-plugged:9d393a2c-3f5b-46a5-a145-dd932faa5e91 for instance eb9f9cfd-8fe0-4803-8f57-e477e1efb7ad on nch1-master-1 Jul 16 21:47:33.067173 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-6e966f40-d3ca-4770-addb-deabf84d88ec req-f056b49e-441f-4bce-bdaf-35a53030735b service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.035049 Jul 16 21:47:33.067998 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2987/5977] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:47:33 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:47:34.005059 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-4a214c29-4756-4bfa-8b64-0c8453965e13 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:47:34.005689 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-4a214c29-4756-4bfa-8b64-0c8453965e13 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: eb9f9cfd-8fe0-4803-8f57-e477e1efb7ad] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:47:34.012876 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4a214c29-4756-4bfa-8b64-0c8453965e13 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:47:34.013342 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4a214c29-4756-4bfa-8b64-0c8453965e13 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:47:34.014128 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4a214c29-4756-4bfa-8b64-0c8453965e13 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:47:34.014577 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4a214c29-4756-4bfa-8b64-0c8453965e13 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:47:34.151629 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-4a214c29-4756-4bfa-8b64-0c8453965e13 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:47:34.160003 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-4a214c29-4756-4bfa-8b64-0c8453965e13 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/eb9f9cfd-8fe0-4803-8f57-e477e1efb7ad" status: 200 len: 1314 microversion: 2.1 time: 0.159142 Jul 16 21:47:34.160577 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2991/5978] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:47:33 2022] GET /compute/v2.1/servers/eb9f9cfd-8fe0-4803-8f57-e477e1efb7ad => generated 1314 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:47:35.178077 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-8a38d2f8-83b8-4e9d-9b43-cc3ed5d34bf0 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:47:35.178760 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-8a38d2f8-83b8-4e9d-9b43-cc3ed5d34bf0 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: eb9f9cfd-8fe0-4803-8f57-e477e1efb7ad] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:47:35.185171 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8a38d2f8-83b8-4e9d-9b43-cc3ed5d34bf0 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:47:35.185624 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8a38d2f8-83b8-4e9d-9b43-cc3ed5d34bf0 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:47:35.186391 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8a38d2f8-83b8-4e9d-9b43-cc3ed5d34bf0 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:47:35.186829 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8a38d2f8-83b8-4e9d-9b43-cc3ed5d34bf0 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:47:35.329533 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-8a38d2f8-83b8-4e9d-9b43-cc3ed5d34bf0 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:47:35.339478 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-8a38d2f8-83b8-4e9d-9b43-cc3ed5d34bf0 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/eb9f9cfd-8fe0-4803-8f57-e477e1efb7ad" status: 200 len: 1500 microversion: 2.1 time: 0.166390 Jul 16 21:47:35.340097 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2988/5979] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:47:35 2022] GET /compute/v2.1/servers/eb9f9cfd-8fe0-4803-8f57-e477e1efb7ad => generated 1500 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:47:35.357282 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-64e621a0-4159-4a55-a984-0b1f757abc74 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Action: 'action', calling method: >, body: {"os-stop": {}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:47:35.365071 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-64e621a0-4159-4a55-a984-0b1f757abc74 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:47:35.365585 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-64e621a0-4159-4a55-a984-0b1f757abc74 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:47:35.389068 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-64e621a0-4159-4a55-a984-0b1f757abc74 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: eb9f9cfd-8fe0-4803-8f57-e477e1efb7ad] Going to try to stop instance {{(pid=138681) force_stop /opt/stack/nova/nova/compute/api.py:2707}} Jul 16 21:47:35.584079 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-64e621a0-4159-4a55-a984-0b1f757abc74 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "POST /compute/v2.1/servers/eb9f9cfd-8fe0-4803-8f57-e477e1efb7ad/action" status: 202 len: 0 microversion: 2.1 time: 0.230348 Jul 16 21:47:35.584881 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2992/5980] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 21:47:35 2022] POST /compute/v2.1/servers/eb9f9cfd-8fe0-4803-8f57-e477e1efb7ad/action => generated 0 bytes in 233 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 21:47:35.604105 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-2389a4fd-e562-4112-ae5b-3f73c930779f tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:47:35.604840 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-2389a4fd-e562-4112-ae5b-3f73c930779f tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: eb9f9cfd-8fe0-4803-8f57-e477e1efb7ad] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:47:35.611928 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2389a4fd-e562-4112-ae5b-3f73c930779f tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:47:35.612809 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2389a4fd-e562-4112-ae5b-3f73c930779f tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:47:35.613661 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2389a4fd-e562-4112-ae5b-3f73c930779f tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:47:35.614175 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2389a4fd-e562-4112-ae5b-3f73c930779f tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:47:35.781331 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-2389a4fd-e562-4112-ae5b-3f73c930779f tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:47:35.789899 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-2389a4fd-e562-4112-ae5b-3f73c930779f tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/eb9f9cfd-8fe0-4803-8f57-e477e1efb7ad" status: 200 len: 1510 microversion: 2.1 time: 0.193959 Jul 16 21:47:35.790545 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2989/5981] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:47:35 2022] GET /compute/v2.1/servers/eb9f9cfd-8fe0-4803-8f57-e477e1efb7ad => generated 1510 bytes in 202 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:47:36.809150 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-a93126d9-b831-4ed3-9952-a94a8f6cc9c7 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:47:36.810803 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-a93126d9-b831-4ed3-9952-a94a8f6cc9c7 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: eb9f9cfd-8fe0-4803-8f57-e477e1efb7ad] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:47:36.826483 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a93126d9-b831-4ed3-9952-a94a8f6cc9c7 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:47:36.831731 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a93126d9-b831-4ed3-9952-a94a8f6cc9c7 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:47:36.831731 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a93126d9-b831-4ed3-9952-a94a8f6cc9c7 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:47:36.831731 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a93126d9-b831-4ed3-9952-a94a8f6cc9c7 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:47:37.053968 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-a93126d9-b831-4ed3-9952-a94a8f6cc9c7 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:47:37.061819 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-a93126d9-b831-4ed3-9952-a94a8f6cc9c7 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/eb9f9cfd-8fe0-4803-8f57-e477e1efb7ad" status: 200 len: 1510 microversion: 2.1 time: 0.257060 Jul 16 21:47:37.062446 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2993/5982] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:47:36 2022] GET /compute/v2.1/servers/eb9f9cfd-8fe0-4803-8f57-e477e1efb7ad => generated 1510 bytes in 259 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:47:38.083949 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-458cd9b4-32db-4933-9415-310e5090249a tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:47:38.083949 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-458cd9b4-32db-4933-9415-310e5090249a tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: eb9f9cfd-8fe0-4803-8f57-e477e1efb7ad] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:47:38.090075 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-458cd9b4-32db-4933-9415-310e5090249a tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:47:38.090534 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-458cd9b4-32db-4933-9415-310e5090249a tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:47:38.091343 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-458cd9b4-32db-4933-9415-310e5090249a tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:47:38.091952 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-458cd9b4-32db-4933-9415-310e5090249a tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:47:38.347195 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-458cd9b4-32db-4933-9415-310e5090249a tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:47:38.356162 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-458cd9b4-32db-4933-9415-310e5090249a tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/eb9f9cfd-8fe0-4803-8f57-e477e1efb7ad" status: 200 len: 1510 microversion: 2.1 time: 0.278593 Jul 16 21:47:38.365895 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2990/5983] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:47:38 2022] GET /compute/v2.1/servers/eb9f9cfd-8fe0-4803-8f57-e477e1efb7ad => generated 1510 bytes in 289 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:47:39.373572 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-3e6d77ae-89df-4590-925b-50c7b579aad6 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:47:39.374204 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-3e6d77ae-89df-4590-925b-50c7b579aad6 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: eb9f9cfd-8fe0-4803-8f57-e477e1efb7ad] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:47:39.381645 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3e6d77ae-89df-4590-925b-50c7b579aad6 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:47:39.382111 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3e6d77ae-89df-4590-925b-50c7b579aad6 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:47:39.382901 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3e6d77ae-89df-4590-925b-50c7b579aad6 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:47:39.383393 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3e6d77ae-89df-4590-925b-50c7b579aad6 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:47:39.536310 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-3e6d77ae-89df-4590-925b-50c7b579aad6 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:47:39.545546 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-3e6d77ae-89df-4590-925b-50c7b579aad6 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/eb9f9cfd-8fe0-4803-8f57-e477e1efb7ad" status: 200 len: 1510 microversion: 2.1 time: 0.175940 Jul 16 21:47:39.546193 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2994/5984] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:47:39 2022] GET /compute/v2.1/servers/eb9f9cfd-8fe0-4803-8f57-e477e1efb7ad => generated 1510 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:47:40.562359 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-25cee71c-0fbf-47bc-b4b3-1b41f04b9654 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:47:40.563001 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-25cee71c-0fbf-47bc-b4b3-1b41f04b9654 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: eb9f9cfd-8fe0-4803-8f57-e477e1efb7ad] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:47:40.570581 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-25cee71c-0fbf-47bc-b4b3-1b41f04b9654 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:47:40.571074 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-25cee71c-0fbf-47bc-b4b3-1b41f04b9654 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:47:40.571892 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-25cee71c-0fbf-47bc-b4b3-1b41f04b9654 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:47:40.572592 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-25cee71c-0fbf-47bc-b4b3-1b41f04b9654 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:47:40.721751 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-25cee71c-0fbf-47bc-b4b3-1b41f04b9654 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:47:40.735151 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-25cee71c-0fbf-47bc-b4b3-1b41f04b9654 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/eb9f9cfd-8fe0-4803-8f57-e477e1efb7ad" status: 200 len: 1510 microversion: 2.1 time: 0.176964 Jul 16 21:47:40.736709 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2991/5985] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:47:40 2022] GET /compute/v2.1/servers/eb9f9cfd-8fe0-4803-8f57-e477e1efb7ad => generated 1510 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:47:41.752760 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-9cceb666-8067-4e55-9f1d-7a47cc96c97c tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:47:41.753406 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-9cceb666-8067-4e55-9f1d-7a47cc96c97c tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: eb9f9cfd-8fe0-4803-8f57-e477e1efb7ad] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:47:41.762397 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9cceb666-8067-4e55-9f1d-7a47cc96c97c tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:47:41.763110 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9cceb666-8067-4e55-9f1d-7a47cc96c97c tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:47:41.763929 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9cceb666-8067-4e55-9f1d-7a47cc96c97c tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:47:41.764397 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9cceb666-8067-4e55-9f1d-7a47cc96c97c tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:47:41.927675 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-9cceb666-8067-4e55-9f1d-7a47cc96c97c tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:47:41.938793 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-9cceb666-8067-4e55-9f1d-7a47cc96c97c tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/eb9f9cfd-8fe0-4803-8f57-e477e1efb7ad" status: 200 len: 1510 microversion: 2.1 time: 0.189927 Jul 16 21:47:41.941295 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2995/5986] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:47:41 2022] GET /compute/v2.1/servers/eb9f9cfd-8fe0-4803-8f57-e477e1efb7ad => generated 1510 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:47:42.958368 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-6bf34b4d-e6e1-4e46-b793-bd55ef885b43 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:47:42.958938 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-6bf34b4d-e6e1-4e46-b793-bd55ef885b43 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: eb9f9cfd-8fe0-4803-8f57-e477e1efb7ad] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:47:42.966047 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6bf34b4d-e6e1-4e46-b793-bd55ef885b43 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:47:42.966510 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6bf34b4d-e6e1-4e46-b793-bd55ef885b43 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:47:42.967309 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6bf34b4d-e6e1-4e46-b793-bd55ef885b43 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:47:42.967764 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6bf34b4d-e6e1-4e46-b793-bd55ef885b43 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:47:43.107637 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-6bf34b4d-e6e1-4e46-b793-bd55ef885b43 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:47:43.115986 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-6bf34b4d-e6e1-4e46-b793-bd55ef885b43 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/eb9f9cfd-8fe0-4803-8f57-e477e1efb7ad" status: 200 len: 1510 microversion: 2.1 time: 0.162194 Jul 16 21:47:43.116894 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2992/5987] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:47:42 2022] GET /compute/v2.1/servers/eb9f9cfd-8fe0-4803-8f57-e477e1efb7ad => generated 1510 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:47:44.134884 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-baef19ff-57d9-4502-a822-7da8d2d2e070 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:47:44.135461 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-baef19ff-57d9-4502-a822-7da8d2d2e070 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: eb9f9cfd-8fe0-4803-8f57-e477e1efb7ad] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:47:44.142518 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-baef19ff-57d9-4502-a822-7da8d2d2e070 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:47:44.143119 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-baef19ff-57d9-4502-a822-7da8d2d2e070 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:47:44.143958 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-baef19ff-57d9-4502-a822-7da8d2d2e070 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:47:44.144418 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-baef19ff-57d9-4502-a822-7da8d2d2e070 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:47:44.349921 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-baef19ff-57d9-4502-a822-7da8d2d2e070 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:47:44.358111 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-baef19ff-57d9-4502-a822-7da8d2d2e070 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/eb9f9cfd-8fe0-4803-8f57-e477e1efb7ad" status: 200 len: 1510 microversion: 2.1 time: 0.228924 Jul 16 21:47:44.358702 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2996/5988] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:47:44 2022] GET /compute/v2.1/servers/eb9f9cfd-8fe0-4803-8f57-e477e1efb7ad => generated 1510 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:47:45.376707 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-cb94374b-bfe6-4b29-bcf4-a16660f98517 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:47:45.377730 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-cb94374b-bfe6-4b29-bcf4-a16660f98517 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: eb9f9cfd-8fe0-4803-8f57-e477e1efb7ad] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:47:45.385632 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cb94374b-bfe6-4b29-bcf4-a16660f98517 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:47:45.386248 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cb94374b-bfe6-4b29-bcf4-a16660f98517 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:47:45.387131 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cb94374b-bfe6-4b29-bcf4-a16660f98517 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:47:45.387830 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cb94374b-bfe6-4b29-bcf4-a16660f98517 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:47:45.646699 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-cb94374b-bfe6-4b29-bcf4-a16660f98517 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:47:45.658472 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-cb94374b-bfe6-4b29-bcf4-a16660f98517 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/eb9f9cfd-8fe0-4803-8f57-e477e1efb7ad" status: 200 len: 1510 microversion: 2.1 time: 0.285512 Jul 16 21:47:45.659512 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2993/5989] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:47:45 2022] GET /compute/v2.1/servers/eb9f9cfd-8fe0-4803-8f57-e477e1efb7ad => generated 1510 bytes in 288 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:47:46.676817 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-e2d5a0cd-8899-4f3c-95ce-a4042109f120 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:47:46.677652 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-e2d5a0cd-8899-4f3c-95ce-a4042109f120 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: eb9f9cfd-8fe0-4803-8f57-e477e1efb7ad] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:47:46.685056 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e2d5a0cd-8899-4f3c-95ce-a4042109f120 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:47:46.685597 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e2d5a0cd-8899-4f3c-95ce-a4042109f120 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:47:46.686487 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e2d5a0cd-8899-4f3c-95ce-a4042109f120 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:47:46.686935 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e2d5a0cd-8899-4f3c-95ce-a4042109f120 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:47:46.884732 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-e2d5a0cd-8899-4f3c-95ce-a4042109f120 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:47:46.892490 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-e2d5a0cd-8899-4f3c-95ce-a4042109f120 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/eb9f9cfd-8fe0-4803-8f57-e477e1efb7ad" status: 200 len: 1510 microversion: 2.1 time: 0.220360 Jul 16 21:47:46.893070 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2997/5990] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:47:46 2022] GET /compute/v2.1/servers/eb9f9cfd-8fe0-4803-8f57-e477e1efb7ad => generated 1510 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:47:47.910425 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-bc15c915-4da8-4186-be99-5c3501770eb4 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:47:47.911127 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-bc15c915-4da8-4186-be99-5c3501770eb4 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: eb9f9cfd-8fe0-4803-8f57-e477e1efb7ad] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:47:47.918221 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-bc15c915-4da8-4186-be99-5c3501770eb4 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:47:47.924118 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-bc15c915-4da8-4186-be99-5c3501770eb4 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:47:47.924118 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-bc15c915-4da8-4186-be99-5c3501770eb4 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:47:47.924118 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-bc15c915-4da8-4186-be99-5c3501770eb4 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:47:48.063807 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-bc15c915-4da8-4186-be99-5c3501770eb4 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:47:48.071898 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-bc15c915-4da8-4186-be99-5c3501770eb4 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/eb9f9cfd-8fe0-4803-8f57-e477e1efb7ad" status: 200 len: 1510 microversion: 2.1 time: 0.166184 Jul 16 21:47:48.072494 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2994/5991] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:47:47 2022] GET /compute/v2.1/servers/eb9f9cfd-8fe0-4803-8f57-e477e1efb7ad => generated 1510 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:47:49.091081 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-bd9f1907-18b7-4742-bc3a-37a9319a0e63 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:47:49.097762 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-bd9f1907-18b7-4742-bc3a-37a9319a0e63 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: eb9f9cfd-8fe0-4803-8f57-e477e1efb7ad] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:47:49.104085 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-bd9f1907-18b7-4742-bc3a-37a9319a0e63 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:47:49.104762 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-bd9f1907-18b7-4742-bc3a-37a9319a0e63 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:47:49.105750 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-bd9f1907-18b7-4742-bc3a-37a9319a0e63 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:47:49.106250 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-bd9f1907-18b7-4742-bc3a-37a9319a0e63 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:47:49.251478 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-bd9f1907-18b7-4742-bc3a-37a9319a0e63 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:47:49.259467 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-bd9f1907-18b7-4742-bc3a-37a9319a0e63 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/eb9f9cfd-8fe0-4803-8f57-e477e1efb7ad" status: 200 len: 1510 microversion: 2.1 time: 0.173397 Jul 16 21:47:49.269782 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2998/5992] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:47:49 2022] GET /compute/v2.1/servers/eb9f9cfd-8fe0-4803-8f57-e477e1efb7ad => generated 1510 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:47:50.276356 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-e9bb4e6f-0b70-4526-abea-d5d0aea8de25 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:47:50.277111 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-e9bb4e6f-0b70-4526-abea-d5d0aea8de25 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: eb9f9cfd-8fe0-4803-8f57-e477e1efb7ad] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:47:50.285351 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e9bb4e6f-0b70-4526-abea-d5d0aea8de25 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:47:50.285865 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e9bb4e6f-0b70-4526-abea-d5d0aea8de25 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:47:50.286676 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e9bb4e6f-0b70-4526-abea-d5d0aea8de25 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:47:50.287177 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e9bb4e6f-0b70-4526-abea-d5d0aea8de25 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:47:50.291860 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=83 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:47:50.430045 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-e9bb4e6f-0b70-4526-abea-d5d0aea8de25 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:47:50.437574 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-e9bb4e6f-0b70-4526-abea-d5d0aea8de25 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/eb9f9cfd-8fe0-4803-8f57-e477e1efb7ad" status: 200 len: 1510 microversion: 2.1 time: 0.165321 Jul 16 21:47:50.438146 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2995/5993] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:47:50 2022] GET /compute/v2.1/servers/eb9f9cfd-8fe0-4803-8f57-e477e1efb7ad => generated 1510 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:47:51.453736 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-028944fe-52b0-4881-a18c-605d250cb64d tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:47:51.454509 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-028944fe-52b0-4881-a18c-605d250cb64d tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: eb9f9cfd-8fe0-4803-8f57-e477e1efb7ad] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:47:51.461919 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-028944fe-52b0-4881-a18c-605d250cb64d tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:47:51.462388 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-028944fe-52b0-4881-a18c-605d250cb64d tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:47:51.463240 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-028944fe-52b0-4881-a18c-605d250cb64d tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:47:51.463702 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-028944fe-52b0-4881-a18c-605d250cb64d tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:47:51.601321 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-028944fe-52b0-4881-a18c-605d250cb64d tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:47:51.609185 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-028944fe-52b0-4881-a18c-605d250cb64d tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/eb9f9cfd-8fe0-4803-8f57-e477e1efb7ad" status: 200 len: 1510 microversion: 2.1 time: 0.159419 Jul 16 21:47:51.609799 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 2999/5994] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:47:51 2022] GET /compute/v2.1/servers/eb9f9cfd-8fe0-4803-8f57-e477e1efb7ad => generated 1510 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:47:52.626993 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-763951de-df5d-4bae-a833-1bccb232bf3f tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:47:52.627914 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-763951de-df5d-4bae-a833-1bccb232bf3f tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: eb9f9cfd-8fe0-4803-8f57-e477e1efb7ad] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:47:52.635705 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-763951de-df5d-4bae-a833-1bccb232bf3f tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:47:52.636161 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-763951de-df5d-4bae-a833-1bccb232bf3f tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:47:52.637048 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-763951de-df5d-4bae-a833-1bccb232bf3f tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:47:52.637714 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-763951de-df5d-4bae-a833-1bccb232bf3f tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:47:52.816092 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-763951de-df5d-4bae-a833-1bccb232bf3f tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:47:52.825589 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-763951de-df5d-4bae-a833-1bccb232bf3f tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/eb9f9cfd-8fe0-4803-8f57-e477e1efb7ad" status: 200 len: 1510 microversion: 2.1 time: 0.202924 Jul 16 21:47:52.837374 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2996/5995] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:47:52 2022] GET /compute/v2.1/servers/eb9f9cfd-8fe0-4803-8f57-e477e1efb7ad => generated 1510 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:47:53.843597 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-dda894b0-a05a-411a-b28c-39ba42af4884 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:47:53.848120 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-dda894b0-a05a-411a-b28c-39ba42af4884 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: eb9f9cfd-8fe0-4803-8f57-e477e1efb7ad] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:47:53.853957 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-dda894b0-a05a-411a-b28c-39ba42af4884 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:47:53.854475 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-dda894b0-a05a-411a-b28c-39ba42af4884 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:47:53.855353 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-dda894b0-a05a-411a-b28c-39ba42af4884 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:47:53.855823 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-dda894b0-a05a-411a-b28c-39ba42af4884 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:47:54.014655 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-dda894b0-a05a-411a-b28c-39ba42af4884 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:47:54.026897 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-dda894b0-a05a-411a-b28c-39ba42af4884 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/eb9f9cfd-8fe0-4803-8f57-e477e1efb7ad" status: 200 len: 1510 microversion: 2.1 time: 0.187889 Jul 16 21:47:54.027524 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3000/5996] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:47:53 2022] GET /compute/v2.1/servers/eb9f9cfd-8fe0-4803-8f57-e477e1efb7ad => generated 1510 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:47:55.045242 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-7c2d8499-3434-4d81-bbe1-103799ef48eb tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:47:55.045934 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-7c2d8499-3434-4d81-bbe1-103799ef48eb tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: eb9f9cfd-8fe0-4803-8f57-e477e1efb7ad] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:47:55.054316 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7c2d8499-3434-4d81-bbe1-103799ef48eb tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:47:55.054768 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7c2d8499-3434-4d81-bbe1-103799ef48eb tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:47:55.058734 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7c2d8499-3434-4d81-bbe1-103799ef48eb tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:47:55.058734 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7c2d8499-3434-4d81-bbe1-103799ef48eb tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:47:55.220203 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-7c2d8499-3434-4d81-bbe1-103799ef48eb tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:47:55.232010 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-7c2d8499-3434-4d81-bbe1-103799ef48eb tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/eb9f9cfd-8fe0-4803-8f57-e477e1efb7ad" status: 200 len: 1510 microversion: 2.1 time: 0.187375 Jul 16 21:47:55.232010 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2997/5997] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:47:55 2022] GET /compute/v2.1/servers/eb9f9cfd-8fe0-4803-8f57-e477e1efb7ad => generated 1510 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:47:56.245997 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-b7f1483b-d84f-4cd5-b6bf-9402a299d899 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:47:56.246713 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-b7f1483b-d84f-4cd5-b6bf-9402a299d899 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: eb9f9cfd-8fe0-4803-8f57-e477e1efb7ad] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:47:56.254096 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b7f1483b-d84f-4cd5-b6bf-9402a299d899 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:47:56.254596 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b7f1483b-d84f-4cd5-b6bf-9402a299d899 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:47:56.256194 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b7f1483b-d84f-4cd5-b6bf-9402a299d899 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:47:56.256868 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b7f1483b-d84f-4cd5-b6bf-9402a299d899 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:47:56.492035 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-b7f1483b-d84f-4cd5-b6bf-9402a299d899 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:47:56.495265 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:47:56.503596 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-b7f1483b-d84f-4cd5-b6bf-9402a299d899 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/eb9f9cfd-8fe0-4803-8f57-e477e1efb7ad" status: 200 len: 1510 microversion: 2.1 time: 0.261641 Jul 16 21:47:56.508414 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3001/5998] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:47:56 2022] GET /compute/v2.1/servers/eb9f9cfd-8fe0-4803-8f57-e477e1efb7ad => generated 1510 bytes in 263 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:47:57.522272 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-830efb7e-3d24-432a-8a6f-f0d0197005ab tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:47:57.522897 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-830efb7e-3d24-432a-8a6f-f0d0197005ab tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: eb9f9cfd-8fe0-4803-8f57-e477e1efb7ad] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:47:57.525994 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_api:SELECT=32 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:47:57.531451 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-830efb7e-3d24-432a-8a6f-f0d0197005ab tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:47:57.532014 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-830efb7e-3d24-432a-8a6f-f0d0197005ab tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:47:57.532736 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-830efb7e-3d24-432a-8a6f-f0d0197005ab tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:47:57.533507 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-830efb7e-3d24-432a-8a6f-f0d0197005ab tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:47:57.748113 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-830efb7e-3d24-432a-8a6f-f0d0197005ab tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:47:57.756353 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-830efb7e-3d24-432a-8a6f-f0d0197005ab tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/eb9f9cfd-8fe0-4803-8f57-e477e1efb7ad" status: 200 len: 1510 microversion: 2.1 time: 0.237988 Jul 16 21:47:57.757149 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2998/5999] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:47:57 2022] GET /compute/v2.1/servers/eb9f9cfd-8fe0-4803-8f57-e477e1efb7ad => generated 1510 bytes in 240 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:47:58.785272 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-ab4f993d-3085-465a-949c-529d22b79e9b tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:47:58.785833 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-ab4f993d-3085-465a-949c-529d22b79e9b tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: eb9f9cfd-8fe0-4803-8f57-e477e1efb7ad] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:47:58.789184 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_api:SELECT=32 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:47:58.795354 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ab4f993d-3085-465a-949c-529d22b79e9b tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:47:58.795821 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ab4f993d-3085-465a-949c-529d22b79e9b tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:47:58.796638 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ab4f993d-3085-465a-949c-529d22b79e9b tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:47:58.797102 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ab4f993d-3085-465a-949c-529d22b79e9b tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:47:58.994602 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-ab4f993d-3085-465a-949c-529d22b79e9b tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:47:59.001539 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-ab4f993d-3085-465a-949c-529d22b79e9b tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/eb9f9cfd-8fe0-4803-8f57-e477e1efb7ad" status: 200 len: 1510 microversion: 2.1 time: 0.231208 Jul 16 21:47:59.002132 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3002/6000] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:47:58 2022] GET /compute/v2.1/servers/eb9f9cfd-8fe0-4803-8f57-e477e1efb7ad => generated 1510 bytes in 232 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:48:00.018908 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-88a34b42-c2a7-4326-85e6-f13d0bdb73ac tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:48:00.020419 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-88a34b42-c2a7-4326-85e6-f13d0bdb73ac tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: eb9f9cfd-8fe0-4803-8f57-e477e1efb7ad] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:48:00.027666 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-88a34b42-c2a7-4326-85e6-f13d0bdb73ac tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:48:00.028193 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-88a34b42-c2a7-4326-85e6-f13d0bdb73ac tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:48:00.028965 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-88a34b42-c2a7-4326-85e6-f13d0bdb73ac tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:48:00.029412 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-88a34b42-c2a7-4326-85e6-f13d0bdb73ac tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:48:00.165473 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-88a34b42-c2a7-4326-85e6-f13d0bdb73ac tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:48:00.171648 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-88a34b42-c2a7-4326-85e6-f13d0bdb73ac tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/eb9f9cfd-8fe0-4803-8f57-e477e1efb7ad" status: 200 len: 1510 microversion: 2.1 time: 0.156674 Jul 16 21:48:00.172228 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 2999/6001] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:48:00 2022] GET /compute/v2.1/servers/eb9f9cfd-8fe0-4803-8f57-e477e1efb7ad => generated 1510 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:48:01.196229 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-178a25c9-f603-487a-b062-eb49423e6f7d tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:48:01.198765 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-178a25c9-f603-487a-b062-eb49423e6f7d tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: eb9f9cfd-8fe0-4803-8f57-e477e1efb7ad] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:48:01.218017 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-178a25c9-f603-487a-b062-eb49423e6f7d tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:48:01.218468 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-178a25c9-f603-487a-b062-eb49423e6f7d tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:48:01.219293 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-178a25c9-f603-487a-b062-eb49423e6f7d tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:48:01.221512 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-178a25c9-f603-487a-b062-eb49423e6f7d tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:48:01.464855 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-178a25c9-f603-487a-b062-eb49423e6f7d tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:48:01.473321 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-178a25c9-f603-487a-b062-eb49423e6f7d tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/eb9f9cfd-8fe0-4803-8f57-e477e1efb7ad" status: 200 len: 1510 microversion: 2.1 time: 0.287889 Jul 16 21:48:01.473906 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3003/6002] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:48:01 2022] GET /compute/v2.1/servers/eb9f9cfd-8fe0-4803-8f57-e477e1efb7ad => generated 1510 bytes in 289 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:48:02.491522 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-e3d5e4d3-8ebd-4501-8bf0-0efc06589eca tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:48:02.492148 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-e3d5e4d3-8ebd-4501-8bf0-0efc06589eca tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: eb9f9cfd-8fe0-4803-8f57-e477e1efb7ad] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:48:02.499262 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e3d5e4d3-8ebd-4501-8bf0-0efc06589eca tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:48:02.499728 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e3d5e4d3-8ebd-4501-8bf0-0efc06589eca tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:48:02.500576 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e3d5e4d3-8ebd-4501-8bf0-0efc06589eca tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:48:02.500955 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e3d5e4d3-8ebd-4501-8bf0-0efc06589eca tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:48:02.725999 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-e3d5e4d3-8ebd-4501-8bf0-0efc06589eca tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:48:02.733908 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-e3d5e4d3-8ebd-4501-8bf0-0efc06589eca tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/eb9f9cfd-8fe0-4803-8f57-e477e1efb7ad" status: 200 len: 1487 microversion: 2.1 time: 0.246462 Jul 16 21:48:02.734488 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3000/6003] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:48:02 2022] GET /compute/v2.1/servers/eb9f9cfd-8fe0-4803-8f57-e477e1efb7ad => generated 1487 bytes in 248 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:48:02.750137 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-1191f91a-94e3-477c-9c85-877ff4d69292 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Action: 'action', calling method: >, body: {"os-getConsoleOutput": {"length": 3}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:48:02.751230 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-1191f91a-94e3-477c-9c85-877ff4d69292 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: 3618ceb8-9d10-4219-8885-d2d5004c71ad] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:48:02.759223 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1191f91a-94e3-477c-9c85-877ff4d69292 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:48:02.759746 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1191f91a-94e3-477c-9c85-877ff4d69292 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:48:02.760540 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1191f91a-94e3-477c-9c85-877ff4d69292 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:48:02.761004 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1191f91a-94e3-477c-9c85-877ff4d69292 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:48:02.812228 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-1191f91a-94e3-477c-9c85-877ff4d69292 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "POST /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad/action" status: 200 len: 178 microversion: 2.1 time: 0.065786 Jul 16 21:48:02.817463 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3004/6004] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 21:48:02 2022] POST /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad/action => generated 178 bytes in 67 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:48:02.823229 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-2a6bb95c-9e65-467f-a73f-5916eaf41704 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:48:02.827568 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-2a6bb95c-9e65-467f-a73f-5916eaf41704 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: 3618ceb8-9d10-4219-8885-d2d5004c71ad] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:48:02.830371 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2a6bb95c-9e65-467f-a73f-5916eaf41704 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:48:02.830831 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2a6bb95c-9e65-467f-a73f-5916eaf41704 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:48:02.832430 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2a6bb95c-9e65-467f-a73f-5916eaf41704 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:48:02.832887 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2a6bb95c-9e65-467f-a73f-5916eaf41704 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:48:02.976410 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-2a6bb95c-9e65-467f-a73f-5916eaf41704 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:48:02.984674 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-2a6bb95c-9e65-467f-a73f-5916eaf41704 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad" status: 200 len: 1665 microversion: 2.1 time: 0.165352 Jul 16 21:48:02.985262 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3001/6005] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:48:02 2022] GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad => generated 1665 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:48:03.015487 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-97a1a862-0ef5-4e8f-b412-e80e9470c50e tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:48:03.015487 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-97a1a862-0ef5-4e8f-b412-e80e9470c50e tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: 3618ceb8-9d10-4219-8885-d2d5004c71ad] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:48:03.017897 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-97a1a862-0ef5-4e8f-b412-e80e9470c50e tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:48:03.018370 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-97a1a862-0ef5-4e8f-b412-e80e9470c50e tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:48:03.019736 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-97a1a862-0ef5-4e8f-b412-e80e9470c50e tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:48:03.020193 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-97a1a862-0ef5-4e8f-b412-e80e9470c50e tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:48:03.157023 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-97a1a862-0ef5-4e8f-b412-e80e9470c50e tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:48:03.164815 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-97a1a862-0ef5-4e8f-b412-e80e9470c50e tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad" status: 200 len: 1665 microversion: 2.1 time: 0.159668 Jul 16 21:48:03.165404 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3005/6006] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:48:03 2022] GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad => generated 1665 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:48:03.181315 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-8994c04b-9a6e-4228-b9d4-43d54da33c34 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:48:03.181982 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-8994c04b-9a6e-4228-b9d4-43d54da33c34 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: 3618ceb8-9d10-4219-8885-d2d5004c71ad] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:48:03.188311 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8994c04b-9a6e-4228-b9d4-43d54da33c34 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:48:03.188804 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8994c04b-9a6e-4228-b9d4-43d54da33c34 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:48:03.189576 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8994c04b-9a6e-4228-b9d4-43d54da33c34 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:48:03.190018 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8994c04b-9a6e-4228-b9d4-43d54da33c34 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:48:03.334112 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-8994c04b-9a6e-4228-b9d4-43d54da33c34 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:48:03.343475 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-8994c04b-9a6e-4228-b9d4-43d54da33c34 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad" status: 200 len: 1665 microversion: 2.1 time: 0.165740 Jul 16 21:48:03.344075 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3002/6007] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:48:03 2022] GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad => generated 1665 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:48:03.360805 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-a042531b-49b7-4e44-bf58-e183c6eeb14c tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:48:03.361470 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-a042531b-49b7-4e44-bf58-e183c6eeb14c tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: 3618ceb8-9d10-4219-8885-d2d5004c71ad] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:48:03.369795 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a042531b-49b7-4e44-bf58-e183c6eeb14c tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:48:03.370268 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a042531b-49b7-4e44-bf58-e183c6eeb14c tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:48:03.371081 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a042531b-49b7-4e44-bf58-e183c6eeb14c tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:48:03.371535 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a042531b-49b7-4e44-bf58-e183c6eeb14c tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:48:03.549465 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-a042531b-49b7-4e44-bf58-e183c6eeb14c tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:48:03.558669 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-a042531b-49b7-4e44-bf58-e183c6eeb14c tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad" status: 200 len: 1665 microversion: 2.1 time: 0.202177 Jul 16 21:48:03.559305 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3006/6008] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:48:03 2022] GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad => generated 1665 bytes in 204 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:48:03.679999 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-c8cd9c77-2598-488c-b8b1-cd45f11255f9 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerActionsTestJSON-server-237955574", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452", "networks": [{"uuid": "af28c06e-7ce2-4ca3-b6a4-f1ae8d70db1a"}]}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:48:03.743590 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.network.neutron [None req-c8cd9c77-2598-488c-b8b1-cd45f11255f9 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] validate_networks() for [('af28c06e-7ce2-4ca3-b6a4-f1ae8d70db1a', None, None, None, None, None)] {{(pid=138680) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 16 21:48:04.333859 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-c8cd9c77-2598-488c-b8b1-cd45f11255f9 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Flavor limits 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 16 21:48:04.334580 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-c8cd9c77-2598-488c-b8b1-cd45f11255f9 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Image limits 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 16 21:48:04.335700 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-c8cd9c77-2598-488c-b8b1-cd45f11255f9 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Flavor pref 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 16 21:48:04.336308 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-c8cd9c77-2598-488c-b8b1-cd45f11255f9 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Image pref 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 16 21:48:04.336955 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-c8cd9c77-2598-488c-b8b1-cd45f11255f9 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 16 21:48:04.348261 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c8cd9c77-2598-488c-b8b1-cd45f11255f9 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:48:04.348975 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c8cd9c77-2598-488c-b8b1-cd45f11255f9 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:48:04.350016 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c8cd9c77-2598-488c-b8b1-cd45f11255f9 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:48:04.350652 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c8cd9c77-2598-488c-b8b1-cd45f11255f9 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:48:04.372966 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-c8cd9c77-2598-488c-b8b1-cd45f11255f9 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Getting quotas for project 1e7cd69b8ddd4aceb45a4aaacdfccea7. Resources: {'ram', 'instances', 'cores'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:48:04.380895 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-c8cd9c77-2598-488c-b8b1-cd45f11255f9 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Getting quotas for user cfe293b9ed3645229bed5ef8ce17cfec and project 1e7cd69b8ddd4aceb45a4aaacdfccea7. Resources: {'ram', 'instances', 'cores'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:48:04.396147 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-c8cd9c77-2598-488c-b8b1-cd45f11255f9 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Going to run 1 instances... {{(pid=138680) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 16 21:48:04.399939 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-c8cd9c77-2598-488c-b8b1-cd45f11255f9 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: bbfa7c88-b995-4c25-b981-1ada326424ed] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138680) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 16 21:48:04.564469 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-c8cd9c77-2598-488c-b8b1-cd45f11255f9 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.889138 Jul 16 21:48:04.565164 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3003/6009] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 21:48:03 2022] POST /compute/v2.1/servers => generated 380 bytes in 891 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 16 21:48:04.576055 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-0b0ce386-855c-4e1a-8a4a-65e96bafd2e6 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:48:04.576765 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-0b0ce386-855c-4e1a-8a4a-65e96bafd2e6 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: bbfa7c88-b995-4c25-b981-1ada326424ed] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:48:04.636081 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-0b0ce386-855c-4e1a-8a4a-65e96bafd2e6 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:48:04.645688 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-0b0ce386-855c-4e1a-8a4a-65e96bafd2e6 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/bbfa7c88-b995-4c25-b981-1ada326424ed" status: 200 len: 1214 microversion: 2.1 time: 0.071697 Jul 16 21:48:04.645688 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3007/6010] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:48:04 2022] GET /compute/v2.1/servers/bbfa7c88-b995-4c25-b981-1ada326424ed => generated 1214 bytes in 74 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:48:04.736509 ncdv-master-1 devstack@n-api.service[138680]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jul 16 21:48:04.736982 ncdv-master-1 devstack@n-api.service[138680]: Sat Jul 16 21:48:04 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Jul 16 21:48:05.663027 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-66d1d4a2-9932-4dfa-9694-466a8a41fe61 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:48:05.663889 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-66d1d4a2-9932-4dfa-9694-466a8a41fe61 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: bbfa7c88-b995-4c25-b981-1ada326424ed] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:48:05.673265 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-66d1d4a2-9932-4dfa-9694-466a8a41fe61 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:48:05.673715 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-66d1d4a2-9932-4dfa-9694-466a8a41fe61 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:48:05.674497 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-66d1d4a2-9932-4dfa-9694-466a8a41fe61 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:48:05.674967 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-66d1d4a2-9932-4dfa-9694-466a8a41fe61 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:48:05.756081 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-66d1d4a2-9932-4dfa-9694-466a8a41fe61 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:48:05.764072 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-66d1d4a2-9932-4dfa-9694-466a8a41fe61 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/bbfa7c88-b995-4c25-b981-1ada326424ed" status: 200 len: 1218 microversion: 2.1 time: 0.106393 Jul 16 21:48:05.764664 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3004/6011] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:48:05 2022] GET /compute/v2.1/servers/bbfa7c88-b995-4c25-b981-1ada326424ed => generated 1218 bytes in 108 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:48:06.782690 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-146358ba-90b9-4825-be07-aa216bf1f045 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:48:06.783653 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-146358ba-90b9-4825-be07-aa216bf1f045 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: bbfa7c88-b995-4c25-b981-1ada326424ed] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:48:06.792203 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-146358ba-90b9-4825-be07-aa216bf1f045 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:48:06.792988 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-146358ba-90b9-4825-be07-aa216bf1f045 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:48:06.794005 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-146358ba-90b9-4825-be07-aa216bf1f045 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:48:06.794653 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-146358ba-90b9-4825-be07-aa216bf1f045 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:48:06.885451 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-146358ba-90b9-4825-be07-aa216bf1f045 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:48:06.893570 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-146358ba-90b9-4825-be07-aa216bf1f045 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/bbfa7c88-b995-4c25-b981-1ada326424ed" status: 200 len: 1274 microversion: 2.1 time: 0.115049 Jul 16 21:48:06.895144 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3008/6012] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:48:06 2022] GET /compute/v2.1/servers/bbfa7c88-b995-4c25-b981-1ada326424ed => generated 1274 bytes in 117 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:48:07.918734 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-dfab8f24-64c1-46c1-b7dc-830f23cffcb2 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:48:07.924685 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-dfab8f24-64c1-46c1-b7dc-830f23cffcb2 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: bbfa7c88-b995-4c25-b981-1ada326424ed] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:48:07.932248 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-dfab8f24-64c1-46c1-b7dc-830f23cffcb2 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:48:07.933606 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-dfab8f24-64c1-46c1-b7dc-830f23cffcb2 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:48:07.934841 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-dfab8f24-64c1-46c1-b7dc-830f23cffcb2 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:48:07.935608 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-dfab8f24-64c1-46c1-b7dc-830f23cffcb2 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:48:08.119203 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-dfab8f24-64c1-46c1-b7dc-830f23cffcb2 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:48:08.130437 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-dfab8f24-64c1-46c1-b7dc-830f23cffcb2 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/bbfa7c88-b995-4c25-b981-1ada326424ed" status: 200 len: 1314 microversion: 2.1 time: 0.216850 Jul 16 21:48:08.131373 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3005/6013] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:48:07 2022] GET /compute/v2.1/servers/bbfa7c88-b995-4c25-b981-1ada326424ed => generated 1314 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:48:08.935084 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-0729f346-a2e9-4f34-b4a5-a51f5bf3a7f9 req-810992aa-56b2-48ce-a30e-7891fa8682b3 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "bbfa7c88-b995-4c25-b981-1ada326424ed", "tag": "1e6fddcb-af8d-4b4f-98ba-73539bd88138"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:48:08.947377 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-0729f346-a2e9-4f34-b4a5-a51f5bf3a7f9 req-810992aa-56b2-48ce-a30e-7891fa8682b3 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:48:08.948123 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-0729f346-a2e9-4f34-b4a5-a51f5bf3a7f9 req-810992aa-56b2-48ce-a30e-7891fa8682b3 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:48:08.982856 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-0729f346-a2e9-4f34-b4a5-a51f5bf3a7f9 req-810992aa-56b2-48ce-a30e-7891fa8682b3 service nova] Creating event network-changed:1e6fddcb-af8d-4b4f-98ba-73539bd88138 for instance bbfa7c88-b995-4c25-b981-1ada326424ed on nch2-master-1 Jul 16 21:48:08.989492 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-0729f346-a2e9-4f34-b4a5-a51f5bf3a7f9 req-810992aa-56b2-48ce-a30e-7891fa8682b3 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.059109 Jul 16 21:48:08.990248 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3009/6014] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:48:08 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 61 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:48:09.149613 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-6c1e1dce-bb5c-492f-9e6e-a1296eefabd6 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:48:09.151068 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-6c1e1dce-bb5c-492f-9e6e-a1296eefabd6 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: bbfa7c88-b995-4c25-b981-1ada326424ed] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:48:09.158794 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6c1e1dce-bb5c-492f-9e6e-a1296eefabd6 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:48:09.159554 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6c1e1dce-bb5c-492f-9e6e-a1296eefabd6 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:48:09.161015 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6c1e1dce-bb5c-492f-9e6e-a1296eefabd6 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:48:09.161557 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6c1e1dce-bb5c-492f-9e6e-a1296eefabd6 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:48:09.394762 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-6c1e1dce-bb5c-492f-9e6e-a1296eefabd6 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:48:09.394762 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-6c1e1dce-bb5c-492f-9e6e-a1296eefabd6 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/bbfa7c88-b995-4c25-b981-1ada326424ed" status: 200 len: 1314 microversion: 2.1 time: 0.250216 Jul 16 21:48:09.403786 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3006/6015] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:48:09 2022] GET /compute/v2.1/servers/bbfa7c88-b995-4c25-b981-1ada326424ed => generated 1314 bytes in 252 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:48:10.415169 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-be361fc1-4b6d-4b8b-aa55-a2f04e6acbf5 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:48:10.415874 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-be361fc1-4b6d-4b8b-aa55-a2f04e6acbf5 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: bbfa7c88-b995-4c25-b981-1ada326424ed] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:48:10.423175 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-be361fc1-4b6d-4b8b-aa55-a2f04e6acbf5 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:48:10.423637 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-be361fc1-4b6d-4b8b-aa55-a2f04e6acbf5 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:48:10.424405 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-be361fc1-4b6d-4b8b-aa55-a2f04e6acbf5 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:48:10.424854 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-be361fc1-4b6d-4b8b-aa55-a2f04e6acbf5 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:48:10.569269 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-be361fc1-4b6d-4b8b-aa55-a2f04e6acbf5 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:48:10.576813 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-be361fc1-4b6d-4b8b-aa55-a2f04e6acbf5 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/bbfa7c88-b995-4c25-b981-1ada326424ed" status: 200 len: 1314 microversion: 2.1 time: 0.166148 Jul 16 21:48:10.577384 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3010/6016] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:48:10 2022] GET /compute/v2.1/servers/bbfa7c88-b995-4c25-b981-1ada326424ed => generated 1314 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:48:11.593812 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-ef886651-46fa-41ca-bdfb-6a62f56c205d tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:48:11.594859 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-ef886651-46fa-41ca-bdfb-6a62f56c205d tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: bbfa7c88-b995-4c25-b981-1ada326424ed] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:48:11.604436 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ef886651-46fa-41ca-bdfb-6a62f56c205d tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:48:11.604893 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ef886651-46fa-41ca-bdfb-6a62f56c205d tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:48:11.605759 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ef886651-46fa-41ca-bdfb-6a62f56c205d tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:48:11.606163 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ef886651-46fa-41ca-bdfb-6a62f56c205d tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:48:11.741267 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-ef886651-46fa-41ca-bdfb-6a62f56c205d tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:48:11.750670 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-ef886651-46fa-41ca-bdfb-6a62f56c205d tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/bbfa7c88-b995-4c25-b981-1ada326424ed" status: 200 len: 1314 microversion: 2.1 time: 0.160746 Jul 16 21:48:11.751293 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3007/6017] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:48:11 2022] GET /compute/v2.1/servers/bbfa7c88-b995-4c25-b981-1ada326424ed => generated 1314 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:48:12.775274 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-5ba5edaf-bc38-4437-a25b-f917f15b8add tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:48:12.775979 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-5ba5edaf-bc38-4437-a25b-f917f15b8add tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: bbfa7c88-b995-4c25-b981-1ada326424ed] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:48:12.782788 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5ba5edaf-bc38-4437-a25b-f917f15b8add tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:48:12.783307 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5ba5edaf-bc38-4437-a25b-f917f15b8add tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:48:12.784089 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5ba5edaf-bc38-4437-a25b-f917f15b8add tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:48:12.785042 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5ba5edaf-bc38-4437-a25b-f917f15b8add tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:48:12.926772 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-5ba5edaf-bc38-4437-a25b-f917f15b8add tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:48:12.934631 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-5ba5edaf-bc38-4437-a25b-f917f15b8add tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/bbfa7c88-b995-4c25-b981-1ada326424ed" status: 200 len: 1314 microversion: 2.1 time: 0.163309 Jul 16 21:48:12.935238 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3011/6018] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:48:12 2022] GET /compute/v2.1/servers/bbfa7c88-b995-4c25-b981-1ada326424ed => generated 1314 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:48:13.952095 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-e4be638f-aefc-460e-9fe7-e886c62cf516 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:48:13.952706 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-e4be638f-aefc-460e-9fe7-e886c62cf516 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: bbfa7c88-b995-4c25-b981-1ada326424ed] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:48:13.960457 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e4be638f-aefc-460e-9fe7-e886c62cf516 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:48:13.961307 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e4be638f-aefc-460e-9fe7-e886c62cf516 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:48:13.961722 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e4be638f-aefc-460e-9fe7-e886c62cf516 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:48:13.962171 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e4be638f-aefc-460e-9fe7-e886c62cf516 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:48:14.119043 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-e4be638f-aefc-460e-9fe7-e886c62cf516 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:48:14.128514 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-e4be638f-aefc-460e-9fe7-e886c62cf516 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/bbfa7c88-b995-4c25-b981-1ada326424ed" status: 200 len: 1314 microversion: 2.1 time: 0.180258 Jul 16 21:48:14.138305 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3008/6019] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:48:13 2022] GET /compute/v2.1/servers/bbfa7c88-b995-4c25-b981-1ada326424ed => generated 1314 bytes in 191 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:48:15.147975 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-4c41f90f-e505-4bf3-93df-8e9cdfba9e2d tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:48:15.148617 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-4c41f90f-e505-4bf3-93df-8e9cdfba9e2d tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: bbfa7c88-b995-4c25-b981-1ada326424ed] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:48:15.152375 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell0:SELECT=2 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:48:15.158054 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4c41f90f-e505-4bf3-93df-8e9cdfba9e2d tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:48:15.158531 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4c41f90f-e505-4bf3-93df-8e9cdfba9e2d tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:48:15.159443 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4c41f90f-e505-4bf3-93df-8e9cdfba9e2d tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:48:15.160173 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4c41f90f-e505-4bf3-93df-8e9cdfba9e2d tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:48:15.326833 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-4c41f90f-e505-4bf3-93df-8e9cdfba9e2d tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:48:15.335140 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-4c41f90f-e505-4bf3-93df-8e9cdfba9e2d tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/bbfa7c88-b995-4c25-b981-1ada326424ed" status: 200 len: 1314 microversion: 2.1 time: 0.192641 Jul 16 21:48:15.335679 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3012/6020] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:48:15 2022] GET /compute/v2.1/servers/bbfa7c88-b995-4c25-b981-1ada326424ed => generated 1314 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:48:15.454222 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-b8f59c4d-462b-42e3-923a-14a047b62aac req-2fe3f86f-332d-4b92-8396-09a593829e26 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "bbfa7c88-b995-4c25-b981-1ada326424ed", "name": "network-vif-plugged", "status": "completed", "tag": "1e6fddcb-af8d-4b4f-98ba-73539bd88138"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:48:15.459689 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell0:SELECT=2 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:48:15.465009 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell0:SELECT=6 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:48:15.472927 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-b8f59c4d-462b-42e3-923a-14a047b62aac req-2fe3f86f-332d-4b92-8396-09a593829e26 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:48:15.473456 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-b8f59c4d-462b-42e3-923a-14a047b62aac req-2fe3f86f-332d-4b92-8396-09a593829e26 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:48:15.791779 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-b8f59c4d-462b-42e3-923a-14a047b62aac req-2fe3f86f-332d-4b92-8396-09a593829e26 service nova] Creating event network-vif-plugged:1e6fddcb-af8d-4b4f-98ba-73539bd88138 for instance bbfa7c88-b995-4c25-b981-1ada326424ed on nch2-master-1 Jul 16 21:48:15.803659 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-b8f59c4d-462b-42e3-923a-14a047b62aac req-2fe3f86f-332d-4b92-8396-09a593829e26 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.355884 Jul 16 21:48:15.805634 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3009/6021] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:48:15 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 359 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:48:16.353239 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-39d8eea7-2fca-4fac-a3f4-92be49c309b1 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:48:16.353934 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-39d8eea7-2fca-4fac-a3f4-92be49c309b1 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: bbfa7c88-b995-4c25-b981-1ada326424ed] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:48:16.363707 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-39d8eea7-2fca-4fac-a3f4-92be49c309b1 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:48:16.364876 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-39d8eea7-2fca-4fac-a3f4-92be49c309b1 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:48:16.365720 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-39d8eea7-2fca-4fac-a3f4-92be49c309b1 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:48:16.366170 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-39d8eea7-2fca-4fac-a3f4-92be49c309b1 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:48:16.593879 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-39d8eea7-2fca-4fac-a3f4-92be49c309b1 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:48:16.605284 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-39d8eea7-2fca-4fac-a3f4-92be49c309b1 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/bbfa7c88-b995-4c25-b981-1ada326424ed" status: 200 len: 1314 microversion: 2.1 time: 0.256768 Jul 16 21:48:16.606195 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3013/6022] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:48:16 2022] GET /compute/v2.1/servers/bbfa7c88-b995-4c25-b981-1ada326424ed => generated 1314 bytes in 259 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:48:17.624400 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-764429c3-bde8-4a0b-aa0c-a3ac2f6a0191 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:48:17.625228 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-764429c3-bde8-4a0b-aa0c-a3ac2f6a0191 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: bbfa7c88-b995-4c25-b981-1ada326424ed] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:48:17.636322 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-764429c3-bde8-4a0b-aa0c-a3ac2f6a0191 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:48:17.638079 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-764429c3-bde8-4a0b-aa0c-a3ac2f6a0191 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:48:17.638938 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-764429c3-bde8-4a0b-aa0c-a3ac2f6a0191 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:48:17.642938 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-764429c3-bde8-4a0b-aa0c-a3ac2f6a0191 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:48:17.790783 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-764429c3-bde8-4a0b-aa0c-a3ac2f6a0191 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:48:17.799800 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-764429c3-bde8-4a0b-aa0c-a3ac2f6a0191 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/bbfa7c88-b995-4c25-b981-1ada326424ed" status: 200 len: 1501 microversion: 2.1 time: 0.181458 Jul 16 21:48:17.801004 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3010/6023] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:48:17 2022] GET /compute/v2.1/servers/bbfa7c88-b995-4c25-b981-1ada326424ed => generated 1501 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:48:17.820605 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-47da745c-b888-4e61-8cad-7d37cd7d6af2 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Action: 'action', calling method: >, body: {"os-getConsoleOutput": {}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:48:17.822377 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-47da745c-b888-4e61-8cad-7d37cd7d6af2 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: bbfa7c88-b995-4c25-b981-1ada326424ed] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:48:17.835260 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-47da745c-b888-4e61-8cad-7d37cd7d6af2 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:48:17.836800 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-47da745c-b888-4e61-8cad-7d37cd7d6af2 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:48:17.837305 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-47da745c-b888-4e61-8cad-7d37cd7d6af2 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:48:17.838045 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-47da745c-b888-4e61-8cad-7d37cd7d6af2 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:48:17.897264 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-47da745c-b888-4e61-8cad-7d37cd7d6af2 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "POST /compute/v2.1/servers/bbfa7c88-b995-4c25-b981-1ada326424ed/action" status: 200 len: 14 microversion: 2.1 time: 0.081640 Jul 16 21:48:17.898567 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3014/6024] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 21:48:17 2022] POST /compute/v2.1/servers/bbfa7c88-b995-4c25-b981-1ada326424ed/action => generated 14 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 356 bytes (2 switches on core 0) Jul 16 21:48:18.909354 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-28374673-0193-46bc-93fc-76c36d1771ad tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Action: 'action', calling method: >, body: {"os-getConsoleOutput": {}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:48:18.910254 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-28374673-0193-46bc-93fc-76c36d1771ad tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: bbfa7c88-b995-4c25-b981-1ada326424ed] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:48:18.917461 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-28374673-0193-46bc-93fc-76c36d1771ad tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:48:18.917912 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-28374673-0193-46bc-93fc-76c36d1771ad tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:48:18.918699 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-28374673-0193-46bc-93fc-76c36d1771ad tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:48:18.919191 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-28374673-0193-46bc-93fc-76c36d1771ad tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:48:18.958947 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-28374673-0193-46bc-93fc-76c36d1771ad tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "POST /compute/v2.1/servers/bbfa7c88-b995-4c25-b981-1ada326424ed/action" status: 200 len: 14 microversion: 2.1 time: 0.053444 Jul 16 21:48:18.959601 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3011/6025] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 21:48:18 2022] POST /compute/v2.1/servers/bbfa7c88-b995-4c25-b981-1ada326424ed/action => generated 14 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Jul 16 21:48:19.970707 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-428f7f7a-1aa1-4bc7-a3bc-5f9ce007b792 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Action: 'action', calling method: >, body: {"os-getConsoleOutput": {}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:48:19.971718 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-428f7f7a-1aa1-4bc7-a3bc-5f9ce007b792 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: bbfa7c88-b995-4c25-b981-1ada326424ed] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:48:19.981799 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-428f7f7a-1aa1-4bc7-a3bc-5f9ce007b792 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:48:19.982264 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-428f7f7a-1aa1-4bc7-a3bc-5f9ce007b792 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:48:19.985280 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-428f7f7a-1aa1-4bc7-a3bc-5f9ce007b792 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:48:19.985971 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-428f7f7a-1aa1-4bc7-a3bc-5f9ce007b792 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:48:20.041312 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-428f7f7a-1aa1-4bc7-a3bc-5f9ce007b792 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "POST /compute/v2.1/servers/bbfa7c88-b995-4c25-b981-1ada326424ed/action" status: 200 len: 14 microversion: 2.1 time: 0.074880 Jul 16 21:48:20.042112 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3015/6026] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 21:48:19 2022] POST /compute/v2.1/servers/bbfa7c88-b995-4c25-b981-1ada326424ed/action => generated 14 bytes in 77 msecs (HTTP/1.1 200) 9 headers in 356 bytes (2 switches on core 0) Jul 16 21:48:21.055259 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-009379bb-af49-46d4-98cc-b8b2fffde133 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Action: 'action', calling method: >, body: {"os-getConsoleOutput": {}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:48:21.056403 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-009379bb-af49-46d4-98cc-b8b2fffde133 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: bbfa7c88-b995-4c25-b981-1ada326424ed] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:48:21.064252 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-009379bb-af49-46d4-98cc-b8b2fffde133 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:48:21.064746 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-009379bb-af49-46d4-98cc-b8b2fffde133 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:48:21.065615 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-009379bb-af49-46d4-98cc-b8b2fffde133 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:48:21.066221 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-009379bb-af49-46d4-98cc-b8b2fffde133 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:48:21.069917 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=92 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:48:21.107310 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-009379bb-af49-46d4-98cc-b8b2fffde133 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "POST /compute/v2.1/servers/bbfa7c88-b995-4c25-b981-1ada326424ed/action" status: 200 len: 14 microversion: 2.1 time: 0.058338 Jul 16 21:48:21.110789 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3012/6027] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 21:48:21 2022] POST /compute/v2.1/servers/bbfa7c88-b995-4c25-b981-1ada326424ed/action => generated 14 bytes in 60 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Jul 16 21:48:22.126691 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-c9a8993c-1c3d-42a3-8f59-57a2a3badde0 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Action: 'action', calling method: >, body: {"os-getConsoleOutput": {}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:48:22.126691 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-c9a8993c-1c3d-42a3-8f59-57a2a3badde0 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: bbfa7c88-b995-4c25-b981-1ada326424ed] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:48:22.132007 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c9a8993c-1c3d-42a3-8f59-57a2a3badde0 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:48:22.132476 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c9a8993c-1c3d-42a3-8f59-57a2a3badde0 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:48:22.133274 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c9a8993c-1c3d-42a3-8f59-57a2a3badde0 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:48:22.133726 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c9a8993c-1c3d-42a3-8f59-57a2a3badde0 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:48:22.175754 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-c9a8993c-1c3d-42a3-8f59-57a2a3badde0 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "POST /compute/v2.1/servers/bbfa7c88-b995-4c25-b981-1ada326424ed/action" status: 200 len: 14 microversion: 2.1 time: 0.061083 Jul 16 21:48:22.176711 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3016/6028] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 21:48:22 2022] POST /compute/v2.1/servers/bbfa7c88-b995-4c25-b981-1ada326424ed/action => generated 14 bytes in 63 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Jul 16 21:48:23.187590 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-3654be66-7e47-46e4-932d-d589cf378d34 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Action: 'action', calling method: >, body: {"os-getConsoleOutput": {}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:48:23.188692 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-3654be66-7e47-46e4-932d-d589cf378d34 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: bbfa7c88-b995-4c25-b981-1ada326424ed] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:48:23.196878 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3654be66-7e47-46e4-932d-d589cf378d34 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:48:23.197330 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3654be66-7e47-46e4-932d-d589cf378d34 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:48:23.198106 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3654be66-7e47-46e4-932d-d589cf378d34 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:48:23.198554 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3654be66-7e47-46e4-932d-d589cf378d34 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:48:23.239390 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-3654be66-7e47-46e4-932d-d589cf378d34 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "POST /compute/v2.1/servers/bbfa7c88-b995-4c25-b981-1ada326424ed/action" status: 200 len: 14 microversion: 2.1 time: 0.055227 Jul 16 21:48:23.243073 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3013/6029] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 21:48:23 2022] POST /compute/v2.1/servers/bbfa7c88-b995-4c25-b981-1ada326424ed/action => generated 14 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Jul 16 21:48:24.251682 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-23b023d1-a532-4e2a-a8b9-f9f702976191 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Action: 'action', calling method: >, body: {"os-getConsoleOutput": {}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:48:24.252728 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-23b023d1-a532-4e2a-a8b9-f9f702976191 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: bbfa7c88-b995-4c25-b981-1ada326424ed] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:48:24.260975 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-23b023d1-a532-4e2a-a8b9-f9f702976191 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:48:24.261391 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-23b023d1-a532-4e2a-a8b9-f9f702976191 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:48:24.262178 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-23b023d1-a532-4e2a-a8b9-f9f702976191 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:48:24.262690 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-23b023d1-a532-4e2a-a8b9-f9f702976191 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:48:24.300820 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-23b023d1-a532-4e2a-a8b9-f9f702976191 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "POST /compute/v2.1/servers/bbfa7c88-b995-4c25-b981-1ada326424ed/action" status: 200 len: 14 microversion: 2.1 time: 0.053445 Jul 16 21:48:24.302105 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3017/6030] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 21:48:24 2022] POST /compute/v2.1/servers/bbfa7c88-b995-4c25-b981-1ada326424ed/action => generated 14 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Jul 16 21:48:25.313187 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-567efa9c-8116-4621-8f8c-7c12b0e511d6 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Action: 'action', calling method: >, body: {"os-getConsoleOutput": {}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:48:25.314242 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-567efa9c-8116-4621-8f8c-7c12b0e511d6 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: bbfa7c88-b995-4c25-b981-1ada326424ed] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:48:25.322977 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-567efa9c-8116-4621-8f8c-7c12b0e511d6 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:48:25.323363 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-567efa9c-8116-4621-8f8c-7c12b0e511d6 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:48:25.324563 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-567efa9c-8116-4621-8f8c-7c12b0e511d6 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:48:25.325005 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-567efa9c-8116-4621-8f8c-7c12b0e511d6 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:48:25.368212 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-567efa9c-8116-4621-8f8c-7c12b0e511d6 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "POST /compute/v2.1/servers/bbfa7c88-b995-4c25-b981-1ada326424ed/action" status: 200 len: 14 microversion: 2.1 time: 0.058866 Jul 16 21:48:25.369628 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3014/6031] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 21:48:25 2022] POST /compute/v2.1/servers/bbfa7c88-b995-4c25-b981-1ada326424ed/action => generated 14 bytes in 62 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Jul 16 21:48:26.380762 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-dc38c5c2-2a56-4873-b4b6-640c7a802936 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Action: 'action', calling method: >, body: {"os-getConsoleOutput": {}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:48:26.381850 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-dc38c5c2-2a56-4873-b4b6-640c7a802936 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: bbfa7c88-b995-4c25-b981-1ada326424ed] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:48:26.389733 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-dc38c5c2-2a56-4873-b4b6-640c7a802936 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:48:26.390197 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-dc38c5c2-2a56-4873-b4b6-640c7a802936 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:48:26.392722 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-dc38c5c2-2a56-4873-b4b6-640c7a802936 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:48:26.392722 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-dc38c5c2-2a56-4873-b4b6-640c7a802936 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:48:26.436713 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-dc38c5c2-2a56-4873-b4b6-640c7a802936 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "POST /compute/v2.1/servers/bbfa7c88-b995-4c25-b981-1ada326424ed/action" status: 200 len: 14 microversion: 2.1 time: 0.060395 Jul 16 21:48:26.437791 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3018/6032] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 21:48:26 2022] POST /compute/v2.1/servers/bbfa7c88-b995-4c25-b981-1ada326424ed/action => generated 14 bytes in 63 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Jul 16 21:48:27.450341 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-adaffe2d-5482-49ee-9718-e213b290ba2f tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Action: 'action', calling method: >, body: {"os-getConsoleOutput": {}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:48:27.452015 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-adaffe2d-5482-49ee-9718-e213b290ba2f tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: bbfa7c88-b995-4c25-b981-1ada326424ed] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:48:27.460056 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-adaffe2d-5482-49ee-9718-e213b290ba2f tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:48:27.460524 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-adaffe2d-5482-49ee-9718-e213b290ba2f tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:48:27.461317 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-adaffe2d-5482-49ee-9718-e213b290ba2f tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:48:27.461766 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-adaffe2d-5482-49ee-9718-e213b290ba2f tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:48:27.506382 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-adaffe2d-5482-49ee-9718-e213b290ba2f tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "POST /compute/v2.1/servers/bbfa7c88-b995-4c25-b981-1ada326424ed/action" status: 200 len: 14 microversion: 2.1 time: 0.061081 Jul 16 21:48:27.507080 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3015/6033] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 21:48:27 2022] POST /compute/v2.1/servers/bbfa7c88-b995-4c25-b981-1ada326424ed/action => generated 14 bytes in 63 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Jul 16 21:48:28.517742 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-fc412f86-9d80-4d9d-b43c-67e0aab34f4d tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Action: 'action', calling method: >, body: {"os-getConsoleOutput": {}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:48:28.518819 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-fc412f86-9d80-4d9d-b43c-67e0aab34f4d tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: bbfa7c88-b995-4c25-b981-1ada326424ed] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:48:28.527486 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fc412f86-9d80-4d9d-b43c-67e0aab34f4d tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:48:28.527962 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fc412f86-9d80-4d9d-b43c-67e0aab34f4d tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:48:28.528757 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fc412f86-9d80-4d9d-b43c-67e0aab34f4d tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:48:28.529217 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fc412f86-9d80-4d9d-b43c-67e0aab34f4d tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:48:28.574217 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-fc412f86-9d80-4d9d-b43c-67e0aab34f4d tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "POST /compute/v2.1/servers/bbfa7c88-b995-4c25-b981-1ada326424ed/action" status: 200 len: 14 microversion: 2.1 time: 0.060630 Jul 16 21:48:28.574869 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3019/6034] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 21:48:28 2022] POST /compute/v2.1/servers/bbfa7c88-b995-4c25-b981-1ada326424ed/action => generated 14 bytes in 62 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Jul 16 21:48:29.592996 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-f29e4492-dc3f-4118-a780-81a650f0a4e8 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Action: 'action', calling method: >, body: {"os-getConsoleOutput": {}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:48:29.594040 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-f29e4492-dc3f-4118-a780-81a650f0a4e8 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: bbfa7c88-b995-4c25-b981-1ada326424ed] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:48:29.597368 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_api:SELECT=47,nova_api:INSERT=3 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:48:29.603809 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f29e4492-dc3f-4118-a780-81a650f0a4e8 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:48:29.604279 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f29e4492-dc3f-4118-a780-81a650f0a4e8 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:48:29.605241 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f29e4492-dc3f-4118-a780-81a650f0a4e8 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:48:29.605690 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f29e4492-dc3f-4118-a780-81a650f0a4e8 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:48:29.649634 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-f29e4492-dc3f-4118-a780-81a650f0a4e8 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "POST /compute/v2.1/servers/bbfa7c88-b995-4c25-b981-1ada326424ed/action" status: 200 len: 14 microversion: 2.1 time: 0.060822 Jul 16 21:48:29.653127 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3016/6035] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 21:48:29 2022] POST /compute/v2.1/servers/bbfa7c88-b995-4c25-b981-1ada326424ed/action => generated 14 bytes in 63 msecs (HTTP/1.1 200) 9 headers in 356 bytes (2 switches on core 0) Jul 16 21:48:30.662063 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-2fba539d-8016-4b00-afdc-911e317f62d9 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Action: 'action', calling method: >, body: {"os-getConsoleOutput": {}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:48:30.663224 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-2fba539d-8016-4b00-afdc-911e317f62d9 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: bbfa7c88-b995-4c25-b981-1ada326424ed] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:48:30.666298 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_api:SELECT=38 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:48:30.671942 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2fba539d-8016-4b00-afdc-911e317f62d9 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:48:30.672426 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2fba539d-8016-4b00-afdc-911e317f62d9 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:48:30.673208 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2fba539d-8016-4b00-afdc-911e317f62d9 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:48:30.673664 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2fba539d-8016-4b00-afdc-911e317f62d9 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:48:30.678190 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=86 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:48:30.728382 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-2fba539d-8016-4b00-afdc-911e317f62d9 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "POST /compute/v2.1/servers/bbfa7c88-b995-4c25-b981-1ada326424ed/action" status: 200 len: 8208 microversion: 2.1 time: 0.071306 Jul 16 21:48:30.729361 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3020/6036] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 21:48:30 2022] POST /compute/v2.1/servers/bbfa7c88-b995-4c25-b981-1ada326424ed/action => generated 8208 bytes in 73 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:48:30.739173 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-06fee75a-ee29-473e-8d09-504efc4604f8 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:48:30.740677 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-06fee75a-ee29-473e-8d09-504efc4604f8 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: 3618ceb8-9d10-4219-8885-d2d5004c71ad] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:48:30.749137 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-06fee75a-ee29-473e-8d09-504efc4604f8 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:48:30.749594 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-06fee75a-ee29-473e-8d09-504efc4604f8 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:48:30.750620 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-06fee75a-ee29-473e-8d09-504efc4604f8 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:48:30.750866 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-06fee75a-ee29-473e-8d09-504efc4604f8 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:48:31.065156 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-06fee75a-ee29-473e-8d09-504efc4604f8 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:48:31.074417 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-06fee75a-ee29-473e-8d09-504efc4604f8 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad" status: 200 len: 1665 microversion: 2.1 time: 0.339791 Jul 16 21:48:31.075187 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3017/6037] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:48:30 2022] GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad => generated 1665 bytes in 341 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:48:31.094176 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-d108f145-ce2e-4654-8279-6e14a87ecbe6 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:48:31.094897 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-d108f145-ce2e-4654-8279-6e14a87ecbe6 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: 3618ceb8-9d10-4219-8885-d2d5004c71ad] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:48:31.102752 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d108f145-ce2e-4654-8279-6e14a87ecbe6 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:48:31.103271 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d108f145-ce2e-4654-8279-6e14a87ecbe6 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:48:31.104050 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d108f145-ce2e-4654-8279-6e14a87ecbe6 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:48:31.104502 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d108f145-ce2e-4654-8279-6e14a87ecbe6 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:48:31.424421 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-d108f145-ce2e-4654-8279-6e14a87ecbe6 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:48:31.433309 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-d108f145-ce2e-4654-8279-6e14a87ecbe6 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad" status: 200 len: 1665 microversion: 2.1 time: 0.343974 Jul 16 21:48:31.433905 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3021/6038] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:48:31 2022] GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad => generated 1665 bytes in 345 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:48:31.449901 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-9693e00f-eca4-4d90-a81f-fc4051327095 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:48:31.450532 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-9693e00f-eca4-4d90-a81f-fc4051327095 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: 3618ceb8-9d10-4219-8885-d2d5004c71ad] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:48:31.457495 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9693e00f-eca4-4d90-a81f-fc4051327095 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:48:31.457895 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9693e00f-eca4-4d90-a81f-fc4051327095 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:48:31.458972 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9693e00f-eca4-4d90-a81f-fc4051327095 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:48:31.459445 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9693e00f-eca4-4d90-a81f-fc4051327095 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:48:31.688179 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-9693e00f-eca4-4d90-a81f-fc4051327095 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:48:31.696007 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-9693e00f-eca4-4d90-a81f-fc4051327095 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad" status: 200 len: 1665 microversion: 2.1 time: 0.250198 Jul 16 21:48:31.696578 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3018/6039] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:48:31 2022] GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad => generated 1665 bytes in 251 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:48:31.712527 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-68fcb629-47d5-4c85-896a-ad2ae9e015ca tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:48:31.713167 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-68fcb629-47d5-4c85-896a-ad2ae9e015ca tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: 3618ceb8-9d10-4219-8885-d2d5004c71ad] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:48:31.720746 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-68fcb629-47d5-4c85-896a-ad2ae9e015ca tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:48:31.726386 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-68fcb629-47d5-4c85-896a-ad2ae9e015ca tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:48:31.726386 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-68fcb629-47d5-4c85-896a-ad2ae9e015ca tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:48:31.726386 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-68fcb629-47d5-4c85-896a-ad2ae9e015ca tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:48:31.866218 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-68fcb629-47d5-4c85-896a-ad2ae9e015ca tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:48:31.874571 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-68fcb629-47d5-4c85-896a-ad2ae9e015ca tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad" status: 200 len: 1665 microversion: 2.1 time: 0.166451 Jul 16 21:48:31.875172 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3022/6040] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:48:31 2022] GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad => generated 1665 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:48:31.891088 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-00931681-8f63-4468-a868-b254a914e7d9 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Action: 'action', calling method: >, body: {"lock": {}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:48:31.891660 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-00931681-8f63-4468-a868-b254a914e7d9 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: 3618ceb8-9d10-4219-8885-d2d5004c71ad] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:48:31.898491 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-00931681-8f63-4468-a868-b254a914e7d9 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:48:31.899681 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-00931681-8f63-4468-a868-b254a914e7d9 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:48:31.900476 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-00931681-8f63-4468-a868-b254a914e7d9 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:48:31.900921 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-00931681-8f63-4468-a868-b254a914e7d9 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:48:32.137966 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-00931681-8f63-4468-a868-b254a914e7d9 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: 3618ceb8-9d10-4219-8885-d2d5004c71ad] Locking {{(pid=138680) lock /opt/stack/nova/nova/compute/api.py:4700}} Jul 16 21:48:32.301525 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-00931681-8f63-4468-a868-b254a914e7d9 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "POST /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad/action" status: 202 len: 0 microversion: 2.1 time: 0.414735 Jul 16 21:48:32.302377 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3019/6041] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 21:48:31 2022] POST /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad/action => generated 0 bytes in 416 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 21:48:32.310890 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-4720ef97-83d6-4364-8fdd-95e4bdca4b11 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:48:32.311620 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-4720ef97-83d6-4364-8fdd-95e4bdca4b11 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: 3618ceb8-9d10-4219-8885-d2d5004c71ad] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:48:32.318931 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4720ef97-83d6-4364-8fdd-95e4bdca4b11 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:48:32.319420 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4720ef97-83d6-4364-8fdd-95e4bdca4b11 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:48:32.320214 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4720ef97-83d6-4364-8fdd-95e4bdca4b11 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:48:32.320668 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4720ef97-83d6-4364-8fdd-95e4bdca4b11 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:48:32.462197 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-4720ef97-83d6-4364-8fdd-95e4bdca4b11 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:48:32.469814 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-4720ef97-83d6-4364-8fdd-95e4bdca4b11 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad" status: 200 len: 1665 microversion: 2.1 time: 0.162884 Jul 16 21:48:32.470395 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3023/6042] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:48:32 2022] GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad => generated 1665 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:48:32.486939 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-daf70a82-2255-4f5a-8692-8d7571a01e8d tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Action: 'action', calling method: >, body: {"os-stop": {}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:48:32.494196 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-daf70a82-2255-4f5a-8692-8d7571a01e8d tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:48:32.494663 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-daf70a82-2255-4f5a-8692-8d7571a01e8d tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:48:32.520834 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-daf70a82-2255-4f5a-8692-8d7571a01e8d tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] HTTP exception thrown: Instance 3618ceb8-9d10-4219-8885-d2d5004c71ad is locked Jul 16 21:48:32.522433 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-daf70a82-2255-4f5a-8692-8d7571a01e8d tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Request method failure captured: Jul 16 21:48:32.522433 ncdv-master-1 devstack@n-api.service[138680]: request: POST /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad/action HTTP/1.1 Jul 16 21:48:32.522433 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:48:32.522433 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:48:32.522433 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:48:32.522433 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 15 Jul 16 21:48:32.522433 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:48:32.522433 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:48:32.522433 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:48:32.522433 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:48:32.522433 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0zFqdHrIN0eOi2qg4drPydwu6SP4etFLyy6aDaqxzq0WQeMBWfEvVpgvWD4EE3q6tgQ3bkIiQt72uW6dWCfa-p4Yuvsc21rwij0v1H7vvz1_kRSKLl1Dn4VMOw8LrCFQcDku8xWPGzPME_8kBIMghBmgh0Bs2vThYjVU90trc_bK9dc Jul 16 21:48:32.522433 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:48:32.522433 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:48:32.522433 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:48:32.522433 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:48:32.522433 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:48:32.522433 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:48:32.522433 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: 1e7cd69b8ddd4aceb45a4aaacdfccea7 Jul 16 21:48:32.522433 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-ServerActionsTestJSON-1394927270 Jul 16 21:48:32.522433 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:48:32.522433 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:48:32.522433 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/1e7cd69b8ddd4aceb45a4aaacdfccea7"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/1e7cd69b8ddd4aceb45a4aaacdfccea7"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_1e7cd69b8ddd4aceb45a4aaacdfccea7"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/1e7cd69b8ddd4aceb45a4aaacdfccea7"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:48:32.522433 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-ServerActionsTestJSON-1394927270 Jul 16 21:48:32.522433 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: 1e7cd69b8ddd4aceb45a4aaacdfccea7 Jul 16 21:48:32.522433 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-ServerActionsTestJSON-1394927270 Jul 16 21:48:32.522433 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-ServerActionsTestJSON-1394927270-project Jul 16 21:48:32.522433 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:48:32.522433 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:48:32.522433 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: cfe293b9ed3645229bed5ef8ce17cfec Jul 16 21:48:32.522433 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-ServerActionsTestJSON-1394927270-project Jul 16 21:48:32.522433 ncdv-master-1 devstack@n-api.service[138680]: Jul 16 21:48:32.522433 ncdv-master-1 devstack@n-api.service[138680]: {"os-stop": {}} Jul 16 21:48:32.522433 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 21:48:32.522433 ncdv-master-1 devstack@n-api.service[138680]: exception: Instance 3618ceb8-9d10-4219-8885-d2d5004c71ad is locked Jul 16 21:48:32.522433 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': '3618ceb8-9d10-4219-8885-d2d5004c71ad', 'body': {'os-stop': {}}} Jul 16 21:48:32.522433 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:48:32.522433 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:48:32.522433 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1446, in _stop_server Jul 16 21:48:32.522433 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi self.compute_api.stop(context, instance) Jul 16 21:48:32.522433 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 237, in inner Jul 16 21:48:32.522433 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.InstanceIsLocked(instance_uuid=instance.uuid) Jul 16 21:48:32.522433 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.InstanceIsLocked: Instance 3618ceb8-9d10-4219-8885-d2d5004c71ad is locked Jul 16 21:48:32.522433 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:48:32.522433 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:48:32.522433 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:48:32.522433 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:48:32.522433 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:48:32.522433 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:48:32.522433 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:48:32.522433 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:48:32.522433 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:48:32.522433 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:48:32.522433 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1448, in _stop_server Jul 16 21:48:32.522433 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPConflict(explanation=e.format_message()) Jul 16 21:48:32.522433 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPConflict: Instance 3618ceb8-9d10-4219-8885-d2d5004c71ad is locked Jul 16 21:48:32.522433 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:48:32.522433 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:48:32.522433 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:48:32.522433 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:48:32.522433 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:48:32.522433 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:48:32.522433 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:48:32.522433 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:48:32.522433 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 3618ceb8-9d10-4219-8885-d2d5004c71ad is locked Jul 16 21:48:32.522433 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:48:32.532424 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-daf70a82-2255-4f5a-8692-8d7571a01e8d tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Returning 409 to user: Instance 3618ceb8-9d10-4219-8885-d2d5004c71ad is locked {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:48:32.532424 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-daf70a82-2255-4f5a-8692-8d7571a01e8d tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "POST /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad/action" status: 409 len: 107 microversion: 2.1 time: 0.041152 Jul 16 21:48:32.532424 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3020/6043] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 21:48:32 2022] POST /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad/action => generated 107 bytes in 42 msecs (HTTP/1.1 409) 9 headers in 378 bytes (1 switches on core 0) Jul 16 21:48:32.532879 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-0d97fa09-7202-4173-b556-85c47e0910b0 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Action: 'action', calling method: >, body: {"unlock": {}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:48:32.533457 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-0d97fa09-7202-4173-b556-85c47e0910b0 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: 3618ceb8-9d10-4219-8885-d2d5004c71ad] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:48:32.540083 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0d97fa09-7202-4173-b556-85c47e0910b0 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:48:32.540556 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0d97fa09-7202-4173-b556-85c47e0910b0 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:48:32.541360 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0d97fa09-7202-4173-b556-85c47e0910b0 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:48:32.541799 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0d97fa09-7202-4173-b556-85c47e0910b0 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:48:32.738521 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-0d97fa09-7202-4173-b556-85c47e0910b0 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: 3618ceb8-9d10-4219-8885-d2d5004c71ad] Unlocking {{(pid=138681) unlock /opt/stack/nova/nova/compute/api.py:4729}} Jul 16 21:48:32.978860 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-0d97fa09-7202-4173-b556-85c47e0910b0 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "POST /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad/action" status: 202 len: 0 microversion: 2.1 time: 0.444020 Jul 16 21:48:32.978860 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3024/6044] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 21:48:32 2022] POST /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad/action => generated 0 bytes in 445 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 21:48:32.983763 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-874c2d45-a644-48a2-ad52-20973301467e tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Action: 'action', calling method: >, body: {"os-stop": {}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:48:33.006070 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-874c2d45-a644-48a2-ad52-20973301467e tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:48:33.006853 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-874c2d45-a644-48a2-ad52-20973301467e tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:48:33.030414 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-874c2d45-a644-48a2-ad52-20973301467e tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: 3618ceb8-9d10-4219-8885-d2d5004c71ad] Going to try to stop instance {{(pid=138680) force_stop /opt/stack/nova/nova/compute/api.py:2707}} Jul 16 21:48:33.222940 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-874c2d45-a644-48a2-ad52-20973301467e tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "POST /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad/action" status: 202 len: 0 microversion: 2.1 time: 0.243362 Jul 16 21:48:33.224122 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3021/6045] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 21:48:32 2022] POST /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad/action => generated 0 bytes in 246 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 21:48:33.237940 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-2cfdcf50-f529-4cc0-9b84-e64beef1162d tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:48:33.239737 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-2cfdcf50-f529-4cc0-9b84-e64beef1162d tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: 3618ceb8-9d10-4219-8885-d2d5004c71ad] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:48:33.251284 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2cfdcf50-f529-4cc0-9b84-e64beef1162d tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:48:33.251772 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2cfdcf50-f529-4cc0-9b84-e64beef1162d tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:48:33.252682 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2cfdcf50-f529-4cc0-9b84-e64beef1162d tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:48:33.253193 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2cfdcf50-f529-4cc0-9b84-e64beef1162d tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:48:33.698159 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-2cfdcf50-f529-4cc0-9b84-e64beef1162d tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:48:33.706174 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-2cfdcf50-f529-4cc0-9b84-e64beef1162d tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad" status: 200 len: 1675 microversion: 2.1 time: 0.472439 Jul 16 21:48:33.706762 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3025/6046] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:48:33 2022] GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad => generated 1675 bytes in 474 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:48:34.723996 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-964c40ed-f001-447f-b9fe-df131ee0a8e1 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:48:34.724753 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-964c40ed-f001-447f-b9fe-df131ee0a8e1 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: 3618ceb8-9d10-4219-8885-d2d5004c71ad] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:48:34.733354 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-964c40ed-f001-447f-b9fe-df131ee0a8e1 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:48:34.733812 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-964c40ed-f001-447f-b9fe-df131ee0a8e1 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:48:34.736421 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-964c40ed-f001-447f-b9fe-df131ee0a8e1 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:48:34.736421 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-964c40ed-f001-447f-b9fe-df131ee0a8e1 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:48:34.884604 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-964c40ed-f001-447f-b9fe-df131ee0a8e1 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:48:34.893632 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-964c40ed-f001-447f-b9fe-df131ee0a8e1 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad" status: 200 len: 1675 microversion: 2.1 time: 0.173653 Jul 16 21:48:34.894323 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3022/6047] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:48:34 2022] GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad => generated 1675 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:48:35.912207 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-b2c5e356-835d-4cd3-a2f0-d64e0191f31d tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:48:35.912973 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-b2c5e356-835d-4cd3-a2f0-d64e0191f31d tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: 3618ceb8-9d10-4219-8885-d2d5004c71ad] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:48:35.920485 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b2c5e356-835d-4cd3-a2f0-d64e0191f31d tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:48:35.921018 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b2c5e356-835d-4cd3-a2f0-d64e0191f31d tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:48:35.922010 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b2c5e356-835d-4cd3-a2f0-d64e0191f31d tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:48:35.922470 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b2c5e356-835d-4cd3-a2f0-d64e0191f31d tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:48:36.092070 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-b2c5e356-835d-4cd3-a2f0-d64e0191f31d tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:48:36.101378 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-b2c5e356-835d-4cd3-a2f0-d64e0191f31d tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad" status: 200 len: 1675 microversion: 2.1 time: 0.193097 Jul 16 21:48:36.101960 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3026/6048] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:48:35 2022] GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad => generated 1675 bytes in 195 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:48:37.126294 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-df8f5402-b823-4a39-a93a-2f253e53ad77 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:48:37.127346 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-df8f5402-b823-4a39-a93a-2f253e53ad77 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: 3618ceb8-9d10-4219-8885-d2d5004c71ad] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:48:37.134928 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-df8f5402-b823-4a39-a93a-2f253e53ad77 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:48:37.135434 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-df8f5402-b823-4a39-a93a-2f253e53ad77 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:48:37.136220 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-df8f5402-b823-4a39-a93a-2f253e53ad77 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:48:37.136709 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-df8f5402-b823-4a39-a93a-2f253e53ad77 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:48:37.295903 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-df8f5402-b823-4a39-a93a-2f253e53ad77 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:48:37.303814 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-df8f5402-b823-4a39-a93a-2f253e53ad77 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad" status: 200 len: 1675 microversion: 2.1 time: 0.181983 Jul 16 21:48:37.304525 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3023/6049] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:48:37 2022] GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad => generated 1675 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:48:38.322312 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-fed1f833-338b-4e20-ace6-e1e021d13e60 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:48:38.323750 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-fed1f833-338b-4e20-ace6-e1e021d13e60 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: 3618ceb8-9d10-4219-8885-d2d5004c71ad] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:48:38.331604 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fed1f833-338b-4e20-ace6-e1e021d13e60 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:48:38.332512 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fed1f833-338b-4e20-ace6-e1e021d13e60 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:48:38.333309 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fed1f833-338b-4e20-ace6-e1e021d13e60 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:48:38.335763 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fed1f833-338b-4e20-ace6-e1e021d13e60 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:48:38.502816 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-fed1f833-338b-4e20-ace6-e1e021d13e60 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:48:38.511256 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-fed1f833-338b-4e20-ace6-e1e021d13e60 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad" status: 200 len: 1675 microversion: 2.1 time: 0.193630 Jul 16 21:48:38.516728 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3027/6050] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:48:38 2022] GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad => generated 1675 bytes in 195 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:48:39.528270 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-715faca7-6613-40bc-b68a-58bc5d7aa57d tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:48:39.530670 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-715faca7-6613-40bc-b68a-58bc5d7aa57d tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: 3618ceb8-9d10-4219-8885-d2d5004c71ad] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:48:39.538930 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-715faca7-6613-40bc-b68a-58bc5d7aa57d tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:48:39.539706 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-715faca7-6613-40bc-b68a-58bc5d7aa57d tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:48:39.540508 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-715faca7-6613-40bc-b68a-58bc5d7aa57d tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:48:39.541016 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-715faca7-6613-40bc-b68a-58bc5d7aa57d tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:48:39.701825 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-715faca7-6613-40bc-b68a-58bc5d7aa57d tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:48:39.710117 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-715faca7-6613-40bc-b68a-58bc5d7aa57d tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad" status: 200 len: 1675 microversion: 2.1 time: 0.185566 Jul 16 21:48:39.711070 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3024/6051] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:48:39 2022] GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad => generated 1675 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:48:40.728415 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-9e00e41f-266b-4714-9eea-e0f00839baf1 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:48:40.730289 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-9e00e41f-266b-4714-9eea-e0f00839baf1 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: 3618ceb8-9d10-4219-8885-d2d5004c71ad] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:48:40.741652 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9e00e41f-266b-4714-9eea-e0f00839baf1 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:48:40.742406 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9e00e41f-266b-4714-9eea-e0f00839baf1 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:48:40.743646 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9e00e41f-266b-4714-9eea-e0f00839baf1 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:48:40.744268 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9e00e41f-266b-4714-9eea-e0f00839baf1 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:48:40.912784 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-9e00e41f-266b-4714-9eea-e0f00839baf1 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:48:40.920814 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-9e00e41f-266b-4714-9eea-e0f00839baf1 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad" status: 200 len: 1675 microversion: 2.1 time: 0.196884 Jul 16 21:48:40.921396 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3028/6052] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:48:40 2022] GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad => generated 1675 bytes in 198 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:48:41.937803 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-66ec8fba-3b88-4123-b16b-c47f7a19c183 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:48:41.938657 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-66ec8fba-3b88-4123-b16b-c47f7a19c183 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: 3618ceb8-9d10-4219-8885-d2d5004c71ad] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:48:41.946608 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-66ec8fba-3b88-4123-b16b-c47f7a19c183 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:48:41.947096 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-66ec8fba-3b88-4123-b16b-c47f7a19c183 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:48:41.948114 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-66ec8fba-3b88-4123-b16b-c47f7a19c183 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:48:41.948804 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-66ec8fba-3b88-4123-b16b-c47f7a19c183 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:48:42.085348 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-66ec8fba-3b88-4123-b16b-c47f7a19c183 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:48:42.093020 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-66ec8fba-3b88-4123-b16b-c47f7a19c183 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad" status: 200 len: 1675 microversion: 2.1 time: 0.159226 Jul 16 21:48:42.093589 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3025/6053] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:48:41 2022] GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad => generated 1675 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:48:43.111082 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-7268ffd6-f0c8-4325-b4e4-8d78d54310a9 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:48:43.111712 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-7268ffd6-f0c8-4325-b4e4-8d78d54310a9 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: 3618ceb8-9d10-4219-8885-d2d5004c71ad] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:48:43.119646 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7268ffd6-f0c8-4325-b4e4-8d78d54310a9 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:48:43.120114 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7268ffd6-f0c8-4325-b4e4-8d78d54310a9 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:48:43.120908 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7268ffd6-f0c8-4325-b4e4-8d78d54310a9 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:48:43.121363 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7268ffd6-f0c8-4325-b4e4-8d78d54310a9 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:48:43.265588 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-7268ffd6-f0c8-4325-b4e4-8d78d54310a9 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:48:43.273373 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-7268ffd6-f0c8-4325-b4e4-8d78d54310a9 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad" status: 200 len: 1675 microversion: 2.1 time: 0.166285 Jul 16 21:48:43.273935 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3029/6054] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:48:43 2022] GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad => generated 1675 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:48:44.290728 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-1b59da0f-d165-408a-94f0-fc7cae57deb9 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:48:44.291693 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-1b59da0f-d165-408a-94f0-fc7cae57deb9 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: 3618ceb8-9d10-4219-8885-d2d5004c71ad] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:48:44.298402 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1b59da0f-d165-408a-94f0-fc7cae57deb9 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:48:44.298860 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1b59da0f-d165-408a-94f0-fc7cae57deb9 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:48:44.299669 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1b59da0f-d165-408a-94f0-fc7cae57deb9 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:48:44.300114 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1b59da0f-d165-408a-94f0-fc7cae57deb9 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:48:44.440679 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-1b59da0f-d165-408a-94f0-fc7cae57deb9 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:48:44.448537 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-1b59da0f-d165-408a-94f0-fc7cae57deb9 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad" status: 200 len: 1652 microversion: 2.1 time: 0.162626 Jul 16 21:48:44.449365 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3026/6055] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:48:44 2022] GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad => generated 1652 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:48:44.466314 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-ca1100f3-e326-4365-a30f-a196edf50495 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Action: 'action', calling method: >, body: {"os-start": {}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:48:44.473460 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ca1100f3-e326-4365-a30f-a196edf50495 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:48:44.473938 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ca1100f3-e326-4365-a30f-a196edf50495 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:48:44.497610 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-ca1100f3-e326-4365-a30f-a196edf50495 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: 3618ceb8-9d10-4219-8885-d2d5004c71ad] Going to try to start instance {{(pid=138681) start /opt/stack/nova/nova/compute/api.py:2730}} Jul 16 21:48:44.705252 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-ca1100f3-e326-4365-a30f-a196edf50495 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "POST /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad/action" status: 202 len: 0 microversion: 2.1 time: 0.243016 Jul 16 21:48:44.705865 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3030/6056] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 21:48:44 2022] POST /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad/action => generated 0 bytes in 245 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 21:48:44.714873 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-7d239cae-7430-47e3-8321-c199356f2d01 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:48:44.715550 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-7d239cae-7430-47e3-8321-c199356f2d01 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: 3618ceb8-9d10-4219-8885-d2d5004c71ad] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:48:44.722075 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7d239cae-7430-47e3-8321-c199356f2d01 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:48:44.728073 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7d239cae-7430-47e3-8321-c199356f2d01 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:48:44.728073 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7d239cae-7430-47e3-8321-c199356f2d01 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:48:44.728073 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7d239cae-7430-47e3-8321-c199356f2d01 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:48:44.728685 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=92,nova_cell1:INSERT=3,nova_cell1:UPDATE=5 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:48:44.864090 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-7d239cae-7430-47e3-8321-c199356f2d01 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:48:44.874298 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-7d239cae-7430-47e3-8321-c199356f2d01 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad" status: 200 len: 1661 microversion: 2.1 time: 0.163492 Jul 16 21:48:44.874990 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3027/6057] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:48:44 2022] GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad => generated 1661 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:48:45.894080 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-3dbe2bd4-4d66-4f74-8e46-ed2694d552d2 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:48:45.894786 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-3dbe2bd4-4d66-4f74-8e46-ed2694d552d2 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: 3618ceb8-9d10-4219-8885-d2d5004c71ad] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:48:45.901747 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3dbe2bd4-4d66-4f74-8e46-ed2694d552d2 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:48:45.902209 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3dbe2bd4-4d66-4f74-8e46-ed2694d552d2 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:48:45.902982 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3dbe2bd4-4d66-4f74-8e46-ed2694d552d2 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:48:45.903526 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3dbe2bd4-4d66-4f74-8e46-ed2694d552d2 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:48:46.047944 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-3dbe2bd4-4d66-4f74-8e46-ed2694d552d2 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:48:46.056543 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-3dbe2bd4-4d66-4f74-8e46-ed2694d552d2 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad" status: 200 len: 1661 microversion: 2.1 time: 0.167633 Jul 16 21:48:46.057118 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3031/6058] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:48:45 2022] GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad => generated 1661 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:48:47.074627 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-0a3fe751-436c-45b9-bfb2-8fcfec07a569 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:48:47.075288 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-0a3fe751-436c-45b9-bfb2-8fcfec07a569 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: 3618ceb8-9d10-4219-8885-d2d5004c71ad] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:48:47.082861 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0a3fe751-436c-45b9-bfb2-8fcfec07a569 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:48:47.083459 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0a3fe751-436c-45b9-bfb2-8fcfec07a569 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:48:47.084418 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0a3fe751-436c-45b9-bfb2-8fcfec07a569 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:48:47.084922 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0a3fe751-436c-45b9-bfb2-8fcfec07a569 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:48:47.222167 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-0a3fe751-436c-45b9-bfb2-8fcfec07a569 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:48:47.230658 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-0a3fe751-436c-45b9-bfb2-8fcfec07a569 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad" status: 200 len: 1665 microversion: 2.1 time: 0.160511 Jul 16 21:48:47.231460 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3028/6059] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:48:47 2022] GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad => generated 1665 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:48:47.247720 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-c779988e-22f1-4576-bd3c-a6351fafa0e9 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:48:47.248353 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-c779988e-22f1-4576-bd3c-a6351fafa0e9 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: 3618ceb8-9d10-4219-8885-d2d5004c71ad] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:48:47.256286 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c779988e-22f1-4576-bd3c-a6351fafa0e9 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:48:47.256762 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c779988e-22f1-4576-bd3c-a6351fafa0e9 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:48:47.257564 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c779988e-22f1-4576-bd3c-a6351fafa0e9 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:48:47.258133 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c779988e-22f1-4576-bd3c-a6351fafa0e9 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:48:47.397203 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-c779988e-22f1-4576-bd3c-a6351fafa0e9 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:48:47.404833 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-c779988e-22f1-4576-bd3c-a6351fafa0e9 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad" status: 200 len: 1665 microversion: 2.1 time: 0.161418 Jul 16 21:48:47.405413 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3032/6060] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:48:47 2022] GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad => generated 1665 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:48:47.421673 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-c93f0eb1-c920-4844-ab2d-99aea16f3c4a tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Action: 'action', calling method: >, body: {"unlock": {}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:48:47.422229 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-c93f0eb1-c920-4844-ab2d-99aea16f3c4a tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: 3618ceb8-9d10-4219-8885-d2d5004c71ad] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:48:47.428994 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c93f0eb1-c920-4844-ab2d-99aea16f3c4a tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:48:47.429483 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c93f0eb1-c920-4844-ab2d-99aea16f3c4a tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:48:47.430259 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c93f0eb1-c920-4844-ab2d-99aea16f3c4a tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:48:47.430708 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c93f0eb1-c920-4844-ab2d-99aea16f3c4a tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:48:47.692687 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-c93f0eb1-c920-4844-ab2d-99aea16f3c4a tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: 3618ceb8-9d10-4219-8885-d2d5004c71ad] Unlocking {{(pid=138680) unlock /opt/stack/nova/nova/compute/api.py:4729}} Jul 16 21:48:47.915890 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-c93f0eb1-c920-4844-ab2d-99aea16f3c4a tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "POST /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad/action" status: 202 len: 0 microversion: 2.1 time: 0.498013 Jul 16 21:48:47.916468 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3029/6061] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 21:48:47 2022] POST /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad/action => generated 0 bytes in 499 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 21:48:47.926866 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-32407ef6-27c0-4411-96a5-df250a93fa51 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:48:47.930055 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-32407ef6-27c0-4411-96a5-df250a93fa51 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: 3618ceb8-9d10-4219-8885-d2d5004c71ad] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:48:47.934642 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-32407ef6-27c0-4411-96a5-df250a93fa51 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:48:47.935345 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-32407ef6-27c0-4411-96a5-df250a93fa51 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:48:47.936155 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-32407ef6-27c0-4411-96a5-df250a93fa51 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:48:47.936667 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-32407ef6-27c0-4411-96a5-df250a93fa51 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:48:47.955739 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=92,nova_cell1:INSERT=3,nova_cell1:UPDATE=5 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:48:48.148779 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-32407ef6-27c0-4411-96a5-df250a93fa51 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:48:48.156552 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-32407ef6-27c0-4411-96a5-df250a93fa51 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad" status: 200 len: 1665 microversion: 2.1 time: 0.233740 Jul 16 21:48:48.157123 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3033/6062] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:48:47 2022] GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad => generated 1665 bytes in 235 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:48:48.173572 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-e436fbb2-c3c9-4d5c-ad72-82d5429aabe7 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:48:48.174179 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-e436fbb2-c3c9-4d5c-ad72-82d5429aabe7 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: 3618ceb8-9d10-4219-8885-d2d5004c71ad] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:48:48.181047 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e436fbb2-c3c9-4d5c-ad72-82d5429aabe7 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:48:48.184740 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e436fbb2-c3c9-4d5c-ad72-82d5429aabe7 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:48:48.184740 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e436fbb2-c3c9-4d5c-ad72-82d5429aabe7 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:48:48.184740 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e436fbb2-c3c9-4d5c-ad72-82d5429aabe7 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:48:48.329908 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-e436fbb2-c3c9-4d5c-ad72-82d5429aabe7 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:48:48.338849 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-e436fbb2-c3c9-4d5c-ad72-82d5429aabe7 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad" status: 200 len: 1665 microversion: 2.1 time: 0.169257 Jul 16 21:48:48.339466 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3030/6063] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:48:48 2022] GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad => generated 1665 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:48:48.355934 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-47f173a0-b996-46d9-a456-02a0c46975dd tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:48:48.356576 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-47f173a0-b996-46d9-a456-02a0c46975dd tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: 3618ceb8-9d10-4219-8885-d2d5004c71ad] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:48:48.364620 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-47f173a0-b996-46d9-a456-02a0c46975dd tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:48:48.365220 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-47f173a0-b996-46d9-a456-02a0c46975dd tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:48:48.366024 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-47f173a0-b996-46d9-a456-02a0c46975dd tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:48:48.366495 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-47f173a0-b996-46d9-a456-02a0c46975dd tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:48:48.502483 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-47f173a0-b996-46d9-a456-02a0c46975dd tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:48:48.510847 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-47f173a0-b996-46d9-a456-02a0c46975dd tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad" status: 200 len: 1665 microversion: 2.1 time: 0.158908 Jul 16 21:48:48.511459 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3034/6064] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:48:48 2022] GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad => generated 1665 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:48:48.529614 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-1ecef9c6-17f4-4bef-9a54-b7ec139cbf99 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Action: 'action', calling method: >, body: {"pause": {}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:48:48.530165 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-1ecef9c6-17f4-4bef-9a54-b7ec139cbf99 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: 3618ceb8-9d10-4219-8885-d2d5004c71ad] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:48:48.536351 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1ecef9c6-17f4-4bef-9a54-b7ec139cbf99 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:48:48.536925 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1ecef9c6-17f4-4bef-9a54-b7ec139cbf99 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:48:48.537720 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1ecef9c6-17f4-4bef-9a54-b7ec139cbf99 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:48:48.538171 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1ecef9c6-17f4-4bef-9a54-b7ec139cbf99 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:48:48.727673 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-1ecef9c6-17f4-4bef-9a54-b7ec139cbf99 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "POST /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad/action" status: 202 len: 0 microversion: 2.1 time: 0.202979 Jul 16 21:48:48.731293 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3031/6065] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 21:48:48 2022] POST /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad/action => generated 0 bytes in 205 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 21:48:48.744898 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-3fa0c69b-6c9b-49ea-81b6-5672a7e01b91 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:48:48.745530 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-3fa0c69b-6c9b-49ea-81b6-5672a7e01b91 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: 3618ceb8-9d10-4219-8885-d2d5004c71ad] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:48:48.751634 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3fa0c69b-6c9b-49ea-81b6-5672a7e01b91 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:48:48.752102 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3fa0c69b-6c9b-49ea-81b6-5672a7e01b91 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:48:48.752885 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3fa0c69b-6c9b-49ea-81b6-5672a7e01b91 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:48:48.753341 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3fa0c69b-6c9b-49ea-81b6-5672a7e01b91 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:48:48.901715 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-3fa0c69b-6c9b-49ea-81b6-5672a7e01b91 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:48:48.908922 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-3fa0c69b-6c9b-49ea-81b6-5672a7e01b91 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad" status: 200 len: 1670 microversion: 2.1 time: 0.168561 Jul 16 21:48:48.909510 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3035/6066] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:48:48 2022] GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad => generated 1670 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:48:49.928758 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-e2825d9c-7d7b-4c20-a79b-bcd166b98215 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:48:49.930114 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-e2825d9c-7d7b-4c20-a79b-bcd166b98215 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: 3618ceb8-9d10-4219-8885-d2d5004c71ad] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:48:49.936951 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e2825d9c-7d7b-4c20-a79b-bcd166b98215 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:48:49.937332 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e2825d9c-7d7b-4c20-a79b-bcd166b98215 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:48:49.938084 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e2825d9c-7d7b-4c20-a79b-bcd166b98215 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:48:49.938529 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e2825d9c-7d7b-4c20-a79b-bcd166b98215 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:48:50.086183 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-e2825d9c-7d7b-4c20-a79b-bcd166b98215 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:48:50.093901 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-e2825d9c-7d7b-4c20-a79b-bcd166b98215 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad" status: 200 len: 1650 microversion: 2.1 time: 0.170791 Jul 16 21:48:50.094469 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3032/6067] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:48:49 2022] GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad => generated 1650 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:48:50.110651 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-fbf9296d-1c4f-4e7c-a523-705b25be25a8 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Action: 'action', calling method: >, body: {"unpause": {}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:48:50.111297 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-fbf9296d-1c4f-4e7c-a523-705b25be25a8 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: 3618ceb8-9d10-4219-8885-d2d5004c71ad] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:48:50.119768 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fbf9296d-1c4f-4e7c-a523-705b25be25a8 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:48:50.126739 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fbf9296d-1c4f-4e7c-a523-705b25be25a8 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:48:50.126739 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fbf9296d-1c4f-4e7c-a523-705b25be25a8 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:48:50.126739 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fbf9296d-1c4f-4e7c-a523-705b25be25a8 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:48:50.339919 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-fbf9296d-1c4f-4e7c-a523-705b25be25a8 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "POST /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad/action" status: 202 len: 0 microversion: 2.1 time: 0.232992 Jul 16 21:48:50.343632 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3036/6068] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 21:48:50 2022] POST /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad/action => generated 0 bytes in 235 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 21:48:50.348886 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-bc9d6dae-c578-4cce-81e1-5420a71111d6 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:48:50.349843 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-bc9d6dae-c578-4cce-81e1-5420a71111d6 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: 3618ceb8-9d10-4219-8885-d2d5004c71ad] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:48:50.356793 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-bc9d6dae-c578-4cce-81e1-5420a71111d6 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:48:50.361065 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-bc9d6dae-c578-4cce-81e1-5420a71111d6 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:48:50.361065 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-bc9d6dae-c578-4cce-81e1-5420a71111d6 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:48:50.361065 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-bc9d6dae-c578-4cce-81e1-5420a71111d6 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:48:50.502489 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-bc9d6dae-c578-4cce-81e1-5420a71111d6 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:48:50.511051 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-bc9d6dae-c578-4cce-81e1-5420a71111d6 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad" status: 200 len: 1657 microversion: 2.1 time: 0.166031 Jul 16 21:48:50.512141 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3033/6069] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:48:50 2022] GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad => generated 1657 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:48:51.529790 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-c4389d48-1057-49c7-b30d-3a274bb7b4ff tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:48:51.530502 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-c4389d48-1057-49c7-b30d-3a274bb7b4ff tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: 3618ceb8-9d10-4219-8885-d2d5004c71ad] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:48:51.539034 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c4389d48-1057-49c7-b30d-3a274bb7b4ff tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:48:51.539865 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c4389d48-1057-49c7-b30d-3a274bb7b4ff tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:48:51.540668 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c4389d48-1057-49c7-b30d-3a274bb7b4ff tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:48:51.541126 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c4389d48-1057-49c7-b30d-3a274bb7b4ff tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:48:51.741122 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-c4389d48-1057-49c7-b30d-3a274bb7b4ff tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:48:51.749688 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-c4389d48-1057-49c7-b30d-3a274bb7b4ff tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad" status: 200 len: 1665 microversion: 2.1 time: 0.225077 Jul 16 21:48:51.750284 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3037/6070] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:48:51 2022] GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad => generated 1665 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:48:51.767143 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-89b3b0ed-e9ae-4dce-b40f-f1d73b5ec45c tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:48:51.767769 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-89b3b0ed-e9ae-4dce-b40f-f1d73b5ec45c tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: 3618ceb8-9d10-4219-8885-d2d5004c71ad] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:48:51.775898 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-89b3b0ed-e9ae-4dce-b40f-f1d73b5ec45c tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:48:51.776372 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-89b3b0ed-e9ae-4dce-b40f-f1d73b5ec45c tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:48:51.777137 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-89b3b0ed-e9ae-4dce-b40f-f1d73b5ec45c tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:48:51.777582 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-89b3b0ed-e9ae-4dce-b40f-f1d73b5ec45c tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:48:51.910896 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-89b3b0ed-e9ae-4dce-b40f-f1d73b5ec45c tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:48:51.921930 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-89b3b0ed-e9ae-4dce-b40f-f1d73b5ec45c tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad" status: 200 len: 1665 microversion: 2.1 time: 0.155932 Jul 16 21:48:51.921930 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3034/6071] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:48:51 2022] GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad => generated 1665 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:48:51.937855 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-b5785a2e-1c67-4168-9a98-39baa0d2dc42 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:48:51.938644 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-b5785a2e-1c67-4168-9a98-39baa0d2dc42 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: 3618ceb8-9d10-4219-8885-d2d5004c71ad] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:48:51.945845 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b5785a2e-1c67-4168-9a98-39baa0d2dc42 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:48:51.946397 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b5785a2e-1c67-4168-9a98-39baa0d2dc42 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:48:51.947465 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b5785a2e-1c67-4168-9a98-39baa0d2dc42 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:48:51.947922 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b5785a2e-1c67-4168-9a98-39baa0d2dc42 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:48:52.083515 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-b5785a2e-1c67-4168-9a98-39baa0d2dc42 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:48:52.091184 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-b5785a2e-1c67-4168-9a98-39baa0d2dc42 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad" status: 200 len: 1665 microversion: 2.1 time: 0.157432 Jul 16 21:48:52.091699 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3038/6072] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:48:51 2022] GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad => generated 1665 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:48:52.107236 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-021a630b-0f77-47b1-883a-4c430fc4afa6 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:48:52.107872 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-021a630b-0f77-47b1-883a-4c430fc4afa6 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: 3618ceb8-9d10-4219-8885-d2d5004c71ad] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:48:52.114160 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-021a630b-0f77-47b1-883a-4c430fc4afa6 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:48:52.119903 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-021a630b-0f77-47b1-883a-4c430fc4afa6 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:48:52.119903 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-021a630b-0f77-47b1-883a-4c430fc4afa6 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:48:52.119903 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-021a630b-0f77-47b1-883a-4c430fc4afa6 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:48:52.259837 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-021a630b-0f77-47b1-883a-4c430fc4afa6 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:48:52.267738 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-021a630b-0f77-47b1-883a-4c430fc4afa6 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad" status: 200 len: 1665 microversion: 2.1 time: 0.164566 Jul 16 21:48:52.268313 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3035/6073] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:48:52 2022] GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad => generated 1665 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:48:52.283736 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-ea167957-63e1-4fe2-a52d-d14021c9702a tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:48:52.284376 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-ea167957-63e1-4fe2-a52d-d14021c9702a tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: 3618ceb8-9d10-4219-8885-d2d5004c71ad] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:48:52.292856 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ea167957-63e1-4fe2-a52d-d14021c9702a tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:48:52.297786 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ea167957-63e1-4fe2-a52d-d14021c9702a tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:48:52.297786 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ea167957-63e1-4fe2-a52d-d14021c9702a tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:48:52.297786 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ea167957-63e1-4fe2-a52d-d14021c9702a tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:48:52.432775 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-ea167957-63e1-4fe2-a52d-d14021c9702a tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:48:52.443348 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-ea167957-63e1-4fe2-a52d-d14021c9702a tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad" status: 200 len: 1665 microversion: 2.1 time: 0.162353 Jul 16 21:48:52.444851 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3039/6074] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:48:52 2022] GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad => generated 1665 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:48:52.460166 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-63e00a11-2098-4142-99ed-377ff2808fc4 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:48:52.460869 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-63e00a11-2098-4142-99ed-377ff2808fc4 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: 3618ceb8-9d10-4219-8885-d2d5004c71ad] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:48:52.468410 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-63e00a11-2098-4142-99ed-377ff2808fc4 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:48:52.468410 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-63e00a11-2098-4142-99ed-377ff2808fc4 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:48:52.468995 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-63e00a11-2098-4142-99ed-377ff2808fc4 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:48:52.469446 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-63e00a11-2098-4142-99ed-377ff2808fc4 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:48:52.639195 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-63e00a11-2098-4142-99ed-377ff2808fc4 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:48:52.650223 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-63e00a11-2098-4142-99ed-377ff2808fc4 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad" status: 200 len: 1665 microversion: 2.1 time: 0.194235 Jul 16 21:48:52.662075 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3036/6075] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:48:52 2022] GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad => generated 1665 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:49:19.177577 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-7317ad59-896e-4759-928c-4152c7a20b70 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Action: 'action', calling method: >, body: {"reboot": {"type": "HARD"}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:49:19.178559 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-7317ad59-896e-4759-928c-4152c7a20b70 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: 3618ceb8-9d10-4219-8885-d2d5004c71ad] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:49:19.181222 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_api:SELECT=39 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:49:19.184144 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=42,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:49:19.186860 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7317ad59-896e-4759-928c-4152c7a20b70 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:49:19.187375 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7317ad59-896e-4759-928c-4152c7a20b70 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:49:19.188157 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7317ad59-896e-4759-928c-4152c7a20b70 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:49:19.188607 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7317ad59-896e-4759-928c-4152c7a20b70 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:49:19.468634 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-7317ad59-896e-4759-928c-4152c7a20b70 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "POST /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad/action" status: 202 len: 0 microversion: 2.1 time: 0.295521 Jul 16 21:49:19.472164 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3040/6076] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 21:49:19 2022] POST /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad/action => generated 0 bytes in 297 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Jul 16 21:49:19.478937 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-7ee599c2-0643-471d-b06f-6ad5430ef879 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:49:19.479709 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-7ee599c2-0643-471d-b06f-6ad5430ef879 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: 3618ceb8-9d10-4219-8885-d2d5004c71ad] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:49:19.482157 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_api:SELECT=41 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:49:19.482899 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=71,nova_cell1:INSERT=3,nova_cell1:UPDATE=5 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:49:19.487929 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7ee599c2-0643-471d-b06f-6ad5430ef879 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:49:19.488368 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7ee599c2-0643-471d-b06f-6ad5430ef879 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:49:19.489817 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7ee599c2-0643-471d-b06f-6ad5430ef879 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:49:19.490482 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7ee599c2-0643-471d-b06f-6ad5430ef879 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:49:19.636845 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-7ee599c2-0643-471d-b06f-6ad5430ef879 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:49:19.646629 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-7ee599c2-0643-471d-b06f-6ad5430ef879 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad" status: 200 len: 1667 microversion: 2.1 time: 0.172255 Jul 16 21:49:19.648165 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3037/6077] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:49:19 2022] GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad => generated 1667 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:49:20.667512 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-c5f4a8a4-b68e-4df0-bc53-f5f4e15f17be tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:49:20.672012 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-c5f4a8a4-b68e-4df0-bc53-f5f4e15f17be tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: 3618ceb8-9d10-4219-8885-d2d5004c71ad] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:49:20.677290 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c5f4a8a4-b68e-4df0-bc53-f5f4e15f17be tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:49:20.677770 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c5f4a8a4-b68e-4df0-bc53-f5f4e15f17be tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:49:20.678670 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c5f4a8a4-b68e-4df0-bc53-f5f4e15f17be tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:49:20.679185 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c5f4a8a4-b68e-4df0-bc53-f5f4e15f17be tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:49:20.823336 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-c5f4a8a4-b68e-4df0-bc53-f5f4e15f17be tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:49:20.831701 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-c5f4a8a4-b68e-4df0-bc53-f5f4e15f17be tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad" status: 200 len: 1667 microversion: 2.1 time: 0.169570 Jul 16 21:49:20.832343 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3041/6078] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:49:20 2022] GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad => generated 1667 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:49:21.850374 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-8a894015-007f-4d59-b5d1-43b00c9435e9 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:49:21.851190 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-8a894015-007f-4d59-b5d1-43b00c9435e9 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: 3618ceb8-9d10-4219-8885-d2d5004c71ad] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:49:21.859083 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8a894015-007f-4d59-b5d1-43b00c9435e9 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:49:21.859786 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8a894015-007f-4d59-b5d1-43b00c9435e9 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:49:21.860786 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8a894015-007f-4d59-b5d1-43b00c9435e9 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:49:21.861407 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8a894015-007f-4d59-b5d1-43b00c9435e9 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:49:22.487363 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-8a894015-007f-4d59-b5d1-43b00c9435e9 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:49:22.497332 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-8a894015-007f-4d59-b5d1-43b00c9435e9 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad" status: 200 len: 1665 microversion: 2.1 time: 0.651620 Jul 16 21:49:22.497995 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3038/6079] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:49:21 2022] GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad => generated 1665 bytes in 653 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:49:46.087522 ncdv-master-1 devstack@n-api.service[138681]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jul 16 21:49:46.088028 ncdv-master-1 devstack@n-api.service[138681]: Sat Jul 16 21:49:46 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Jul 16 21:49:54.612041 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-136dedc0-9757-40bb-b80e-34935c8a95c9 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:49:54.612678 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-136dedc0-9757-40bb-b80e-34935c8a95c9 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: 3618ceb8-9d10-4219-8885-d2d5004c71ad] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:49:54.628655 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_api:SELECT=4 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:49:54.630246 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=16,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:49:54.632282 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-136dedc0-9757-40bb-b80e-34935c8a95c9 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:49:54.632392 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-136dedc0-9757-40bb-b80e-34935c8a95c9 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:49:54.633165 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-136dedc0-9757-40bb-b80e-34935c8a95c9 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:49:54.633628 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-136dedc0-9757-40bb-b80e-34935c8a95c9 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:49:54.830511 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-136dedc0-9757-40bb-b80e-34935c8a95c9 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:49:54.838975 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-136dedc0-9757-40bb-b80e-34935c8a95c9 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad" status: 200 len: 1665 microversion: 2.1 time: 0.230771 Jul 16 21:49:54.839529 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3042/6080] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:49:54 2022] GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad => generated 1665 bytes in 232 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:49:54.859556 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-1ee49b03-4e42-402b-ab37-8c7a413f3fb0 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:49:54.860256 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-1ee49b03-4e42-402b-ab37-8c7a413f3fb0 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: 3618ceb8-9d10-4219-8885-d2d5004c71ad] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:49:54.862706 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_api:SELECT=4 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:49:54.866054 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=12 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:49:54.880018 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1ee49b03-4e42-402b-ab37-8c7a413f3fb0 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:49:54.880900 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1ee49b03-4e42-402b-ab37-8c7a413f3fb0 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:49:54.881648 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1ee49b03-4e42-402b-ab37-8c7a413f3fb0 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:49:54.882388 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1ee49b03-4e42-402b-ab37-8c7a413f3fb0 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:49:55.049916 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-1ee49b03-4e42-402b-ab37-8c7a413f3fb0 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:49:55.058977 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-1ee49b03-4e42-402b-ab37-8c7a413f3fb0 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad" status: 200 len: 1665 microversion: 2.1 time: 0.204070 Jul 16 21:49:55.059709 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3039/6081] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:49:54 2022] GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad => generated 1665 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:49:55.076380 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-0b040a9b-d6a5-4837-baa5-7011fa9dca48 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:49:55.077057 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-0b040a9b-d6a5-4837-baa5-7011fa9dca48 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: 3618ceb8-9d10-4219-8885-d2d5004c71ad] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:49:55.085086 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0b040a9b-d6a5-4837-baa5-7011fa9dca48 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:49:55.085554 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0b040a9b-d6a5-4837-baa5-7011fa9dca48 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:49:55.086348 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0b040a9b-d6a5-4837-baa5-7011fa9dca48 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:49:55.091717 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0b040a9b-d6a5-4837-baa5-7011fa9dca48 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:49:55.239252 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-0b040a9b-d6a5-4837-baa5-7011fa9dca48 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:49:55.247525 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-0b040a9b-d6a5-4837-baa5-7011fa9dca48 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad" status: 200 len: 1665 microversion: 2.1 time: 0.175344 Jul 16 21:49:55.248106 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3043/6082] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:49:55 2022] GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad => generated 1665 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:49:55.263717 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-61bae3f6-43e7-400d-9f2e-68d60bcc51bf tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:49:55.264657 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-61bae3f6-43e7-400d-9f2e-68d60bcc51bf tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: 3618ceb8-9d10-4219-8885-d2d5004c71ad] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:49:55.271865 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-61bae3f6-43e7-400d-9f2e-68d60bcc51bf tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:49:55.272326 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-61bae3f6-43e7-400d-9f2e-68d60bcc51bf tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:49:55.273116 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-61bae3f6-43e7-400d-9f2e-68d60bcc51bf tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:49:55.273559 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-61bae3f6-43e7-400d-9f2e-68d60bcc51bf tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:49:55.413115 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-61bae3f6-43e7-400d-9f2e-68d60bcc51bf tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:49:55.421002 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-61bae3f6-43e7-400d-9f2e-68d60bcc51bf tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad" status: 200 len: 1665 microversion: 2.1 time: 0.161231 Jul 16 21:49:55.421590 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3040/6083] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:49:55 2022] GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad => generated 1665 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:49:55.538747 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-4cd9455b-1a5f-49e7-9a4f-43d68f2c173c tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerActionsTestJSON-server-1393982734", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452", "networks": [{"uuid": "af28c06e-7ce2-4ca3-b6a4-f1ae8d70db1a"}]}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:49:55.595555 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.network.neutron [None req-4cd9455b-1a5f-49e7-9a4f-43d68f2c173c tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] validate_networks() for [('af28c06e-7ce2-4ca3-b6a4-f1ae8d70db1a', None, None, None, None, None)] {{(pid=138681) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 16 21:49:55.998108 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-4cd9455b-1a5f-49e7-9a4f-43d68f2c173c tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Flavor limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 16 21:49:55.998685 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-4cd9455b-1a5f-49e7-9a4f-43d68f2c173c tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Image limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 16 21:49:55.999076 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-4cd9455b-1a5f-49e7-9a4f-43d68f2c173c tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Flavor pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 16 21:49:55.999502 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-4cd9455b-1a5f-49e7-9a4f-43d68f2c173c tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Image pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 16 21:49:55.999954 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-4cd9455b-1a5f-49e7-9a4f-43d68f2c173c tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 16 21:49:56.010187 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4cd9455b-1a5f-49e7-9a4f-43d68f2c173c tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:49:56.010649 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4cd9455b-1a5f-49e7-9a4f-43d68f2c173c tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:49:56.011503 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4cd9455b-1a5f-49e7-9a4f-43d68f2c173c tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:49:56.011959 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4cd9455b-1a5f-49e7-9a4f-43d68f2c173c tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:49:56.029701 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-4cd9455b-1a5f-49e7-9a4f-43d68f2c173c tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Getting quotas for project 1e7cd69b8ddd4aceb45a4aaacdfccea7. Resources: {'ram', 'instances', 'cores'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:49:56.035639 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-4cd9455b-1a5f-49e7-9a4f-43d68f2c173c tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Getting quotas for user cfe293b9ed3645229bed5ef8ce17cfec and project 1e7cd69b8ddd4aceb45a4aaacdfccea7. Resources: {'ram', 'instances', 'cores'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:49:56.048258 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-4cd9455b-1a5f-49e7-9a4f-43d68f2c173c tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Going to run 1 instances... {{(pid=138681) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 16 21:49:56.050558 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-4cd9455b-1a5f-49e7-9a4f-43d68f2c173c tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: 5d7c5a9b-cfb1-49ce-be31-b19603177834] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138681) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 16 21:49:56.171967 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-4cd9455b-1a5f-49e7-9a4f-43d68f2c173c tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.638421 Jul 16 21:49:56.178186 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3044/6084] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 21:49:55 2022] POST /compute/v2.1/servers => generated 380 bytes in 640 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 16 21:49:56.183498 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-47af9da3-b70a-4368-b113-b066c0ec4fb8 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:49:56.184137 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-47af9da3-b70a-4368-b113-b066c0ec4fb8 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: 5d7c5a9b-cfb1-49ce-be31-b19603177834] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:49:56.242666 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-47af9da3-b70a-4368-b113-b066c0ec4fb8 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:49:56.250094 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-47af9da3-b70a-4368-b113-b066c0ec4fb8 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/5d7c5a9b-cfb1-49ce-be31-b19603177834" status: 200 len: 1215 microversion: 2.1 time: 0.070438 Jul 16 21:49:56.255109 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3041/6085] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:49:56 2022] GET /compute/v2.1/servers/5d7c5a9b-cfb1-49ce-be31-b19603177834 => generated 1215 bytes in 72 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:49:57.267422 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-2e7e0fa3-282e-47b4-8a3c-0c8e28954591 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:49:57.268052 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-2e7e0fa3-282e-47b4-8a3c-0c8e28954591 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: 5d7c5a9b-cfb1-49ce-be31-b19603177834] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:49:57.329175 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-2e7e0fa3-282e-47b4-8a3c-0c8e28954591 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:49:57.338147 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-2e7e0fa3-282e-47b4-8a3c-0c8e28954591 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/5d7c5a9b-cfb1-49ce-be31-b19603177834" status: 200 len: 1215 microversion: 2.1 time: 0.074583 Jul 16 21:49:57.338720 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3045/6086] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:49:57 2022] GET /compute/v2.1/servers/5d7c5a9b-cfb1-49ce-be31-b19603177834 => generated 1215 bytes in 76 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:49:58.355651 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-8b614bc7-304a-4688-9acb-0521394f1001 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:49:58.356264 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-8b614bc7-304a-4688-9acb-0521394f1001 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: 5d7c5a9b-cfb1-49ce-be31-b19603177834] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:49:58.417805 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-8b614bc7-304a-4688-9acb-0521394f1001 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:49:58.424748 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-8b614bc7-304a-4688-9acb-0521394f1001 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/5d7c5a9b-cfb1-49ce-be31-b19603177834" status: 200 len: 1215 microversion: 2.1 time: 0.073125 Jul 16 21:49:58.425312 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3042/6087] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:49:58 2022] GET /compute/v2.1/servers/5d7c5a9b-cfb1-49ce-be31-b19603177834 => generated 1215 bytes in 74 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:49:59.442160 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-028b069e-7131-4ad1-9389-a9573251fc7c tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:49:59.442864 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-028b069e-7131-4ad1-9389-a9573251fc7c tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: 5d7c5a9b-cfb1-49ce-be31-b19603177834] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:49:59.449759 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-028b069e-7131-4ad1-9389-a9573251fc7c tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:49:59.450224 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-028b069e-7131-4ad1-9389-a9573251fc7c tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:49:59.451026 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-028b069e-7131-4ad1-9389-a9573251fc7c tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:49:59.451484 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-028b069e-7131-4ad1-9389-a9573251fc7c tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:49:59.515800 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-028b069e-7131-4ad1-9389-a9573251fc7c tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:49:59.524420 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-028b069e-7131-4ad1-9389-a9573251fc7c tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/5d7c5a9b-cfb1-49ce-be31-b19603177834" status: 200 len: 1219 microversion: 2.1 time: 0.086522 Jul 16 21:49:59.525003 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3046/6088] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:49:59 2022] GET /compute/v2.1/servers/5d7c5a9b-cfb1-49ce-be31-b19603177834 => generated 1219 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:50:00.541642 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-df656b78-e293-4614-95c9-5d7c51596a9a tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:50:00.542320 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-df656b78-e293-4614-95c9-5d7c51596a9a tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: 5d7c5a9b-cfb1-49ce-be31-b19603177834] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:50:00.553116 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-df656b78-e293-4614-95c9-5d7c51596a9a tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:50:00.553638 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-df656b78-e293-4614-95c9-5d7c51596a9a tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:50:00.554430 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-df656b78-e293-4614-95c9-5d7c51596a9a tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:50:00.554870 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-df656b78-e293-4614-95c9-5d7c51596a9a tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:50:00.641214 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-df656b78-e293-4614-95c9-5d7c51596a9a tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:50:00.651569 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-df656b78-e293-4614-95c9-5d7c51596a9a tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/5d7c5a9b-cfb1-49ce-be31-b19603177834" status: 200 len: 1211 microversion: 2.1 time: 0.113930 Jul 16 21:50:00.652254 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3043/6089] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:50:00 2022] GET /compute/v2.1/servers/5d7c5a9b-cfb1-49ce-be31-b19603177834 => generated 1211 bytes in 115 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:50:01.669297 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-92e77d59-9599-4ba6-b613-d7153fe7542b tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:50:01.669934 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-92e77d59-9599-4ba6-b613-d7153fe7542b tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: 5d7c5a9b-cfb1-49ce-be31-b19603177834] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:50:01.691086 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-92e77d59-9599-4ba6-b613-d7153fe7542b tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:50:01.692134 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-92e77d59-9599-4ba6-b613-d7153fe7542b tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:50:01.693035 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-92e77d59-9599-4ba6-b613-d7153fe7542b tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:50:01.693524 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-92e77d59-9599-4ba6-b613-d7153fe7542b tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:50:01.855872 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-92e77d59-9599-4ba6-b613-d7153fe7542b tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:50:01.865610 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-92e77d59-9599-4ba6-b613-d7153fe7542b tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/5d7c5a9b-cfb1-49ce-be31-b19603177834" status: 200 len: 1273 microversion: 2.1 time: 0.200114 Jul 16 21:50:01.876771 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3047/6090] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:50:01 2022] GET /compute/v2.1/servers/5d7c5a9b-cfb1-49ce-be31-b19603177834 => generated 1273 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:50:02.884527 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-f7816885-a80a-4fe4-b3bf-52c1014b387b tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:50:02.885913 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-f7816885-a80a-4fe4-b3bf-52c1014b387b tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: 5d7c5a9b-cfb1-49ce-be31-b19603177834] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:50:02.906388 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f7816885-a80a-4fe4-b3bf-52c1014b387b tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:50:02.907244 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f7816885-a80a-4fe4-b3bf-52c1014b387b tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:50:02.908096 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f7816885-a80a-4fe4-b3bf-52c1014b387b tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:50:02.908543 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f7816885-a80a-4fe4-b3bf-52c1014b387b tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:50:03.377519 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-f7816885-a80a-4fe4-b3bf-52c1014b387b tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:50:03.412108 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-f7816885-a80a-4fe4-b3bf-52c1014b387b tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/5d7c5a9b-cfb1-49ce-be31-b19603177834" status: 200 len: 1315 microversion: 2.1 time: 0.532678 Jul 16 21:50:03.413318 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3044/6091] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:50:02 2022] GET /compute/v2.1/servers/5d7c5a9b-cfb1-49ce-be31-b19603177834 => generated 1315 bytes in 535 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:50:03.424284 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-2e2e7c38-4b6d-4316-b037-7605a052a9f9 req-75b93716-5bcd-4886-99a6-e8314116bfa7 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "5d7c5a9b-cfb1-49ce-be31-b19603177834", "tag": "e6e85788-edc2-468b-8bf0-8710fb76b373"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:50:03.442006 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-2e2e7c38-4b6d-4316-b037-7605a052a9f9 req-75b93716-5bcd-4886-99a6-e8314116bfa7 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:50:03.442006 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-2e2e7c38-4b6d-4316-b037-7605a052a9f9 req-75b93716-5bcd-4886-99a6-e8314116bfa7 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:50:03.462960 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-2e2e7c38-4b6d-4316-b037-7605a052a9f9 req-75b93716-5bcd-4886-99a6-e8314116bfa7 service nova] Creating event network-changed:e6e85788-edc2-468b-8bf0-8710fb76b373 for instance 5d7c5a9b-cfb1-49ce-be31-b19603177834 on nch1-master-1 Jul 16 21:50:03.472263 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-2e2e7c38-4b6d-4316-b037-7605a052a9f9 req-75b93716-5bcd-4886-99a6-e8314116bfa7 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.061732 Jul 16 21:50:03.473665 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3048/6092] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:50:03 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 64 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:50:04.430021 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-a36940e2-4f3b-4640-a612-cfc7601bd34a tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:50:04.430672 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-a36940e2-4f3b-4640-a612-cfc7601bd34a tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: 5d7c5a9b-cfb1-49ce-be31-b19603177834] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:50:04.446022 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a36940e2-4f3b-4640-a612-cfc7601bd34a tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:50:04.446571 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a36940e2-4f3b-4640-a612-cfc7601bd34a tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:50:04.447334 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a36940e2-4f3b-4640-a612-cfc7601bd34a tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:50:04.447935 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a36940e2-4f3b-4640-a612-cfc7601bd34a tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:50:04.799609 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-a36940e2-4f3b-4640-a612-cfc7601bd34a tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:50:04.808757 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-a36940e2-4f3b-4640-a612-cfc7601bd34a tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/5d7c5a9b-cfb1-49ce-be31-b19603177834" status: 200 len: 1315 microversion: 2.1 time: 0.383133 Jul 16 21:50:04.825428 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3045/6093] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:50:04 2022] GET /compute/v2.1/servers/5d7c5a9b-cfb1-49ce-be31-b19603177834 => generated 1315 bytes in 400 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:50:05.827573 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-bdbefa54-572f-46db-a9b6-449203af9756 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:50:05.828336 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-bdbefa54-572f-46db-a9b6-449203af9756 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: 5d7c5a9b-cfb1-49ce-be31-b19603177834] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:50:05.836901 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-bdbefa54-572f-46db-a9b6-449203af9756 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:50:05.837374 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-bdbefa54-572f-46db-a9b6-449203af9756 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:50:05.838181 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-bdbefa54-572f-46db-a9b6-449203af9756 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:50:05.838640 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-bdbefa54-572f-46db-a9b6-449203af9756 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:50:06.021540 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell0:SELECT=2 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:50:06.126886 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-bdbefa54-572f-46db-a9b6-449203af9756 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:50:06.135823 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-bdbefa54-572f-46db-a9b6-449203af9756 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/5d7c5a9b-cfb1-49ce-be31-b19603177834" status: 200 len: 1315 microversion: 2.1 time: 0.313632 Jul 16 21:50:06.136423 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3049/6094] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:50:05 2022] GET /compute/v2.1/servers/5d7c5a9b-cfb1-49ce-be31-b19603177834 => generated 1315 bytes in 315 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:50:07.155381 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-8c8ed5e1-b318-41e7-a85f-c79e19feb90f tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:50:07.156007 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-8c8ed5e1-b318-41e7-a85f-c79e19feb90f tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: 5d7c5a9b-cfb1-49ce-be31-b19603177834] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:50:07.164850 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8c8ed5e1-b318-41e7-a85f-c79e19feb90f tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:50:07.165557 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8c8ed5e1-b318-41e7-a85f-c79e19feb90f tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:50:07.166370 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8c8ed5e1-b318-41e7-a85f-c79e19feb90f tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:50:07.166889 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8c8ed5e1-b318-41e7-a85f-c79e19feb90f tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:50:07.337511 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-8c8ed5e1-b318-41e7-a85f-c79e19feb90f tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:50:07.346575 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-8c8ed5e1-b318-41e7-a85f-c79e19feb90f tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/5d7c5a9b-cfb1-49ce-be31-b19603177834" status: 200 len: 1315 microversion: 2.1 time: 0.196491 Jul 16 21:50:07.347202 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3046/6095] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:50:07 2022] GET /compute/v2.1/servers/5d7c5a9b-cfb1-49ce-be31-b19603177834 => generated 1315 bytes in 198 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:50:08.364245 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-f82a1f54-b542-49d5-9299-c86519cb2ba8 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:50:08.364891 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-f82a1f54-b542-49d5-9299-c86519cb2ba8 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: 5d7c5a9b-cfb1-49ce-be31-b19603177834] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:50:08.368097 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell0:SELECT=8 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:50:08.373246 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f82a1f54-b542-49d5-9299-c86519cb2ba8 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:50:08.373660 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f82a1f54-b542-49d5-9299-c86519cb2ba8 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:50:08.374457 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f82a1f54-b542-49d5-9299-c86519cb2ba8 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:50:08.374921 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f82a1f54-b542-49d5-9299-c86519cb2ba8 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:50:08.679824 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-f82a1f54-b542-49d5-9299-c86519cb2ba8 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:50:08.694635 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-f82a1f54-b542-49d5-9299-c86519cb2ba8 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/5d7c5a9b-cfb1-49ce-be31-b19603177834" status: 200 len: 1315 microversion: 2.1 time: 0.334610 Jul 16 21:50:08.704384 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3050/6096] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:50:08 2022] GET /compute/v2.1/servers/5d7c5a9b-cfb1-49ce-be31-b19603177834 => generated 1315 bytes in 344 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:50:09.714448 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-abff2c25-2de0-4110-b077-1a7d6619ab58 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:50:09.715196 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-abff2c25-2de0-4110-b077-1a7d6619ab58 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: 5d7c5a9b-cfb1-49ce-be31-b19603177834] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:50:09.719515 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell0:SELECT=4 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:50:09.727392 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-abff2c25-2de0-4110-b077-1a7d6619ab58 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:50:09.727853 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-abff2c25-2de0-4110-b077-1a7d6619ab58 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:50:09.728800 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-abff2c25-2de0-4110-b077-1a7d6619ab58 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:50:09.729398 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-abff2c25-2de0-4110-b077-1a7d6619ab58 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:50:09.911711 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-abff2c25-2de0-4110-b077-1a7d6619ab58 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:50:09.922941 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-abff2c25-2de0-4110-b077-1a7d6619ab58 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/5d7c5a9b-cfb1-49ce-be31-b19603177834" status: 200 len: 1315 microversion: 2.1 time: 0.214323 Jul 16 21:50:09.923754 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3047/6097] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:50:09 2022] GET /compute/v2.1/servers/5d7c5a9b-cfb1-49ce-be31-b19603177834 => generated 1315 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:50:10.664694 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-72e9e5ba-2b4f-400f-a9ca-1189c1e2b095 req-e1f43da5-b003-4c8d-aa27-aaf7b9cf0586 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "5d7c5a9b-cfb1-49ce-be31-b19603177834", "name": "network-vif-plugged", "status": "completed", "tag": "e6e85788-edc2-468b-8bf0-8710fb76b373"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:50:10.673619 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-72e9e5ba-2b4f-400f-a9ca-1189c1e2b095 req-e1f43da5-b003-4c8d-aa27-aaf7b9cf0586 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:50:10.674369 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-72e9e5ba-2b4f-400f-a9ca-1189c1e2b095 req-e1f43da5-b003-4c8d-aa27-aaf7b9cf0586 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:50:10.693166 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-72e9e5ba-2b4f-400f-a9ca-1189c1e2b095 req-e1f43da5-b003-4c8d-aa27-aaf7b9cf0586 service nova] Creating event network-vif-plugged:e6e85788-edc2-468b-8bf0-8710fb76b373 for instance 5d7c5a9b-cfb1-49ce-be31-b19603177834 on nch1-master-1 Jul 16 21:50:10.699063 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-72e9e5ba-2b4f-400f-a9ca-1189c1e2b095 req-e1f43da5-b003-4c8d-aa27-aaf7b9cf0586 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.261245 Jul 16 21:50:10.700443 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3051/6098] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:50:10 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 264 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:50:10.944096 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-e2f7dbb6-78a0-4b1d-95e5-a6e673d73c40 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:50:10.944096 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-e2f7dbb6-78a0-4b1d-95e5-a6e673d73c40 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: 5d7c5a9b-cfb1-49ce-be31-b19603177834] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:50:10.950335 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e2f7dbb6-78a0-4b1d-95e5-a6e673d73c40 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:50:10.950811 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e2f7dbb6-78a0-4b1d-95e5-a6e673d73c40 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:50:10.951643 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e2f7dbb6-78a0-4b1d-95e5-a6e673d73c40 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:50:10.952242 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e2f7dbb6-78a0-4b1d-95e5-a6e673d73c40 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:50:11.115845 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-e2f7dbb6-78a0-4b1d-95e5-a6e673d73c40 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:50:11.125010 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-e2f7dbb6-78a0-4b1d-95e5-a6e673d73c40 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/5d7c5a9b-cfb1-49ce-be31-b19603177834" status: 200 len: 1315 microversion: 2.1 time: 0.187463 Jul 16 21:50:11.125541 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3048/6099] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:50:10 2022] GET /compute/v2.1/servers/5d7c5a9b-cfb1-49ce-be31-b19603177834 => generated 1315 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:50:12.145032 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-675f87e2-050b-43ad-a0d4-a903357a5d37 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:50:12.147834 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-675f87e2-050b-43ad-a0d4-a903357a5d37 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: 5d7c5a9b-cfb1-49ce-be31-b19603177834] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:50:12.158145 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-675f87e2-050b-43ad-a0d4-a903357a5d37 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:50:12.160481 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-675f87e2-050b-43ad-a0d4-a903357a5d37 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:50:12.161533 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-675f87e2-050b-43ad-a0d4-a903357a5d37 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:50:12.162189 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-675f87e2-050b-43ad-a0d4-a903357a5d37 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:50:12.375813 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-675f87e2-050b-43ad-a0d4-a903357a5d37 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:50:12.386911 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-675f87e2-050b-43ad-a0d4-a903357a5d37 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/5d7c5a9b-cfb1-49ce-be31-b19603177834" status: 200 len: 1315 microversion: 2.1 time: 0.248384 Jul 16 21:50:12.389731 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3052/6100] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:50:12 2022] GET /compute/v2.1/servers/5d7c5a9b-cfb1-49ce-be31-b19603177834 => generated 1315 bytes in 252 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:50:13.410499 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-570b5384-b65c-4323-87fc-da45731f105a tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:50:13.411331 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-570b5384-b65c-4323-87fc-da45731f105a tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: 5d7c5a9b-cfb1-49ce-be31-b19603177834] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:50:13.421916 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-570b5384-b65c-4323-87fc-da45731f105a tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:50:13.422370 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-570b5384-b65c-4323-87fc-da45731f105a tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:50:13.423999 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-570b5384-b65c-4323-87fc-da45731f105a tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:50:13.424595 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-570b5384-b65c-4323-87fc-da45731f105a tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:50:13.605010 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-570b5384-b65c-4323-87fc-da45731f105a tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:50:13.616017 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-570b5384-b65c-4323-87fc-da45731f105a tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/5d7c5a9b-cfb1-49ce-be31-b19603177834" status: 200 len: 1502 microversion: 2.1 time: 0.209982 Jul 16 21:50:13.616839 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3049/6101] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:50:13 2022] GET /compute/v2.1/servers/5d7c5a9b-cfb1-49ce-be31-b19603177834 => generated 1502 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:50:13.643430 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-0fdfc7eb-071f-4d60-86d4-a9d66c0d034f tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Action: 'action', calling method: >, body: {"removeSecurityGroup": {"name": "default"}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:50:13.646683 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-0fdfc7eb-071f-4d60-86d4-a9d66c0d034f tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: 5d7c5a9b-cfb1-49ce-be31-b19603177834] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:50:13.660400 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0fdfc7eb-071f-4d60-86d4-a9d66c0d034f tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:50:13.661578 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0fdfc7eb-071f-4d60-86d4-a9d66c0d034f tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:50:13.662778 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0fdfc7eb-071f-4d60-86d4-a9d66c0d034f tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:50:13.663518 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0fdfc7eb-071f-4d60-86d4-a9d66c0d034f tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:50:13.881285 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.network.security_group_api [None req-0fdfc7eb-071f-4d60-86d4-a9d66c0d034f tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Removing security group cfaaa81e-9a63-4756-8586-f2da340fbf75 from port e6e85788-edc2-468b-8bf0-8710fb76b373 Jul 16 21:50:14.367992 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-0fdfc7eb-071f-4d60-86d4-a9d66c0d034f tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "POST /compute/v2.1/servers/5d7c5a9b-cfb1-49ce-be31-b19603177834/action" status: 202 len: 0 microversion: 2.1 time: 0.735554 Jul 16 21:50:14.368608 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3053/6102] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 21:50:13 2022] POST /compute/v2.1/servers/5d7c5a9b-cfb1-49ce-be31-b19603177834/action => generated 0 bytes in 737 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 21:50:14.383098 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-f466162e-5de9-499f-8eab-b1180230b909 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:50:14.383098 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-f466162e-5de9-499f-8eab-b1180230b909 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: 5d7c5a9b-cfb1-49ce-be31-b19603177834] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:50:14.385530 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f466162e-5de9-499f-8eab-b1180230b909 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:50:14.386097 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f466162e-5de9-499f-8eab-b1180230b909 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:50:14.388168 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f466162e-5de9-499f-8eab-b1180230b909 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:50:14.388724 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f466162e-5de9-499f-8eab-b1180230b909 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:50:14.526916 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-7b8ede0c-e506-498a-b766-5c16126e6c82 req-0927208d-534d-4f21-94ce-febd20a9c2d8 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "5d7c5a9b-cfb1-49ce-be31-b19603177834", "tag": "e6e85788-edc2-468b-8bf0-8710fb76b373"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:50:14.536558 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-7b8ede0c-e506-498a-b766-5c16126e6c82 req-0927208d-534d-4f21-94ce-febd20a9c2d8 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:50:14.537036 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-7b8ede0c-e506-498a-b766-5c16126e6c82 req-0927208d-534d-4f21-94ce-febd20a9c2d8 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:50:14.560924 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-7b8ede0c-e506-498a-b766-5c16126e6c82 req-0927208d-534d-4f21-94ce-febd20a9c2d8 service nova] Creating event network-changed:e6e85788-edc2-468b-8bf0-8710fb76b373 for instance 5d7c5a9b-cfb1-49ce-be31-b19603177834 on nch1-master-1 Jul 16 21:50:14.566851 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-7b8ede0c-e506-498a-b766-5c16126e6c82 req-0927208d-534d-4f21-94ce-febd20a9c2d8 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.183707 Jul 16 21:50:14.567940 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3054/6103] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:50:14 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 16 21:50:14.596290 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-f466162e-5de9-499f-8eab-b1180230b909 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:50:14.605990 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-f466162e-5de9-499f-8eab-b1180230b909 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/5d7c5a9b-cfb1-49ce-be31-b19603177834" status: 200 len: 1460 microversion: 2.1 time: 0.233019 Jul 16 21:50:14.618889 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3050/6104] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:50:14 2022] GET /compute/v2.1/servers/5d7c5a9b-cfb1-49ce-be31-b19603177834 => generated 1460 bytes in 246 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:50:14.624883 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-16804d08-9f76-4375-8594-c787894ddf1f tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:50:14.625620 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-16804d08-9f76-4375-8594-c787894ddf1f tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: 3618ceb8-9d10-4219-8885-d2d5004c71ad] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:50:14.633653 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-16804d08-9f76-4375-8594-c787894ddf1f tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:50:14.634485 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-16804d08-9f76-4375-8594-c787894ddf1f tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:50:14.638271 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-16804d08-9f76-4375-8594-c787894ddf1f tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:50:14.638994 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-16804d08-9f76-4375-8594-c787894ddf1f tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:50:14.821039 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-16804d08-9f76-4375-8594-c787894ddf1f tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:50:14.832883 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-16804d08-9f76-4375-8594-c787894ddf1f tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad" status: 200 len: 1665 microversion: 2.1 time: 0.212746 Jul 16 21:50:14.833490 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3055/6105] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:50:14 2022] GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad => generated 1665 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:50:14.854423 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-e0376dd5-43cf-4cfe-bd4e-4dca575e25af tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:50:14.855503 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-e0376dd5-43cf-4cfe-bd4e-4dca575e25af tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: 3618ceb8-9d10-4219-8885-d2d5004c71ad] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:50:14.864236 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e0376dd5-43cf-4cfe-bd4e-4dca575e25af tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:50:14.864883 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e0376dd5-43cf-4cfe-bd4e-4dca575e25af tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:50:14.869079 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e0376dd5-43cf-4cfe-bd4e-4dca575e25af tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:50:14.869573 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e0376dd5-43cf-4cfe-bd4e-4dca575e25af tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:50:15.044323 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-e0376dd5-43cf-4cfe-bd4e-4dca575e25af tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:50:15.053426 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-e0376dd5-43cf-4cfe-bd4e-4dca575e25af tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad" status: 200 len: 1665 microversion: 2.1 time: 0.202898 Jul 16 21:50:15.054030 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3051/6106] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:50:14 2022] GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad => generated 1665 bytes in 204 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:50:15.070143 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-d9e78576-840c-4500-815e-54ebff1a10af tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:50:15.070862 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-d9e78576-840c-4500-815e-54ebff1a10af tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: 3618ceb8-9d10-4219-8885-d2d5004c71ad] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:50:15.077753 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d9e78576-840c-4500-815e-54ebff1a10af tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:50:15.078229 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d9e78576-840c-4500-815e-54ebff1a10af tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:50:15.079057 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d9e78576-840c-4500-815e-54ebff1a10af tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:50:15.079534 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d9e78576-840c-4500-815e-54ebff1a10af tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:50:15.261834 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-d9e78576-840c-4500-815e-54ebff1a10af tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:50:15.271414 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-d9e78576-840c-4500-815e-54ebff1a10af tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad" status: 200 len: 1665 microversion: 2.1 time: 0.205653 Jul 16 21:50:15.273617 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3056/6107] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:50:15 2022] GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad => generated 1665 bytes in 209 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:50:15.289954 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-106e931a-322e-4cbe-85d5-ffe3e9635384 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:50:15.290388 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-106e931a-322e-4cbe-85d5-ffe3e9635384 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: 3618ceb8-9d10-4219-8885-d2d5004c71ad] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:50:15.298069 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-106e931a-322e-4cbe-85d5-ffe3e9635384 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:50:15.298534 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-106e931a-322e-4cbe-85d5-ffe3e9635384 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:50:15.299937 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-106e931a-322e-4cbe-85d5-ffe3e9635384 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:50:15.300904 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-106e931a-322e-4cbe-85d5-ffe3e9635384 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:50:15.456330 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-106e931a-322e-4cbe-85d5-ffe3e9635384 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:50:15.467735 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-106e931a-322e-4cbe-85d5-ffe3e9635384 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad" status: 200 len: 1665 microversion: 2.1 time: 0.181841 Jul 16 21:50:15.475828 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3052/6108] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:50:15 2022] GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad => generated 1665 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:50:15.540954 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-fb7f73b3-8da1-4b60-a9c0-24c38d2e3dcc tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerActionsTestJSON-server-834594648", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452", "networks": [{"uuid": "af28c06e-7ce2-4ca3-b6a4-f1ae8d70db1a"}]}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:50:15.623454 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.network.neutron [None req-fb7f73b3-8da1-4b60-a9c0-24c38d2e3dcc tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] validate_networks() for [('af28c06e-7ce2-4ca3-b6a4-f1ae8d70db1a', None, None, None, None, None)] {{(pid=138681) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 16 21:50:16.138578 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-fb7f73b3-8da1-4b60-a9c0-24c38d2e3dcc tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Flavor limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 16 21:50:16.139247 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-fb7f73b3-8da1-4b60-a9c0-24c38d2e3dcc tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Image limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 16 21:50:16.139826 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-fb7f73b3-8da1-4b60-a9c0-24c38d2e3dcc tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Flavor pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 16 21:50:16.140395 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-fb7f73b3-8da1-4b60-a9c0-24c38d2e3dcc tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Image pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 16 21:50:16.141006 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-fb7f73b3-8da1-4b60-a9c0-24c38d2e3dcc tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 16 21:50:16.152709 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fb7f73b3-8da1-4b60-a9c0-24c38d2e3dcc tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:50:16.153177 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fb7f73b3-8da1-4b60-a9c0-24c38d2e3dcc tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:50:16.153996 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fb7f73b3-8da1-4b60-a9c0-24c38d2e3dcc tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:50:16.154659 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fb7f73b3-8da1-4b60-a9c0-24c38d2e3dcc tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:50:16.176156 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-fb7f73b3-8da1-4b60-a9c0-24c38d2e3dcc tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Getting quotas for project 1e7cd69b8ddd4aceb45a4aaacdfccea7. Resources: {'ram', 'instances', 'cores'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:50:16.183229 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-fb7f73b3-8da1-4b60-a9c0-24c38d2e3dcc tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Getting quotas for user cfe293b9ed3645229bed5ef8ce17cfec and project 1e7cd69b8ddd4aceb45a4aaacdfccea7. Resources: {'ram', 'instances', 'cores'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:50:16.200965 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-fb7f73b3-8da1-4b60-a9c0-24c38d2e3dcc tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Going to run 1 instances... {{(pid=138681) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 16 21:50:16.203620 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-fb7f73b3-8da1-4b60-a9c0-24c38d2e3dcc tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: a2bd21dc-856d-41b0-8d5c-139452473742] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138681) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 16 21:50:16.380058 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-fb7f73b3-8da1-4b60-a9c0-24c38d2e3dcc tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.845485 Jul 16 21:50:16.383068 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3057/6109] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 21:50:15 2022] POST /compute/v2.1/servers => generated 380 bytes in 847 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 16 21:50:16.391979 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-5412ca00-d108-4f11-bcf2-6e6abeb00301 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:50:16.397334 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-5412ca00-d108-4f11-bcf2-6e6abeb00301 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: a2bd21dc-856d-41b0-8d5c-139452473742] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:50:16.491818 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-5412ca00-d108-4f11-bcf2-6e6abeb00301 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:50:16.502581 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-5412ca00-d108-4f11-bcf2-6e6abeb00301 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/a2bd21dc-856d-41b0-8d5c-139452473742" status: 200 len: 1214 microversion: 2.1 time: 0.115069 Jul 16 21:50:16.509832 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3053/6110] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:50:16 2022] GET /compute/v2.1/servers/a2bd21dc-856d-41b0-8d5c-139452473742 => generated 1214 bytes in 123 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:50:17.527741 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-e3178340-a469-4f7f-8f81-d6a1ce7bc608 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:50:17.528964 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-e3178340-a469-4f7f-8f81-d6a1ce7bc608 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: a2bd21dc-856d-41b0-8d5c-139452473742] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:50:17.537837 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e3178340-a469-4f7f-8f81-d6a1ce7bc608 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:50:17.538542 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e3178340-a469-4f7f-8f81-d6a1ce7bc608 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:50:17.540052 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e3178340-a469-4f7f-8f81-d6a1ce7bc608 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:50:17.540819 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e3178340-a469-4f7f-8f81-d6a1ce7bc608 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:50:17.630996 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-e3178340-a469-4f7f-8f81-d6a1ce7bc608 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:50:17.646703 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-e3178340-a469-4f7f-8f81-d6a1ce7bc608 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/a2bd21dc-856d-41b0-8d5c-139452473742" status: 200 len: 1218 microversion: 2.1 time: 0.124097 Jul 16 21:50:17.647265 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3058/6111] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:50:17 2022] GET /compute/v2.1/servers/a2bd21dc-856d-41b0-8d5c-139452473742 => generated 1218 bytes in 126 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:50:18.664602 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-4cfbba8f-c724-4550-8ca3-420a0bcca63a tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:50:18.665949 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-4cfbba8f-c724-4550-8ca3-420a0bcca63a tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: a2bd21dc-856d-41b0-8d5c-139452473742] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:50:18.673764 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4cfbba8f-c724-4550-8ca3-420a0bcca63a tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:50:18.674222 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4cfbba8f-c724-4550-8ca3-420a0bcca63a tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:50:18.675055 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4cfbba8f-c724-4550-8ca3-420a0bcca63a tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:50:18.677192 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4cfbba8f-c724-4550-8ca3-420a0bcca63a tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:50:18.759933 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-4cfbba8f-c724-4550-8ca3-420a0bcca63a tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:50:18.770011 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-4cfbba8f-c724-4550-8ca3-420a0bcca63a tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/a2bd21dc-856d-41b0-8d5c-139452473742" status: 200 len: 1266 microversion: 2.1 time: 0.108389 Jul 16 21:50:18.770621 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3054/6112] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:50:18 2022] GET /compute/v2.1/servers/a2bd21dc-856d-41b0-8d5c-139452473742 => generated 1266 bytes in 111 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:50:19.788364 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-de17e810-b0ed-45b4-a3cd-7b19877dedcf tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:50:19.789079 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-de17e810-b0ed-45b4-a3cd-7b19877dedcf tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: a2bd21dc-856d-41b0-8d5c-139452473742] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:50:19.801630 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-de17e810-b0ed-45b4-a3cd-7b19877dedcf tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:50:19.802111 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-de17e810-b0ed-45b4-a3cd-7b19877dedcf tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:50:19.803130 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-de17e810-b0ed-45b4-a3cd-7b19877dedcf tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:50:19.803775 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-de17e810-b0ed-45b4-a3cd-7b19877dedcf tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:50:19.934261 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-de17e810-b0ed-45b4-a3cd-7b19877dedcf tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:50:19.944325 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-de17e810-b0ed-45b4-a3cd-7b19877dedcf tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/a2bd21dc-856d-41b0-8d5c-139452473742" status: 200 len: 1272 microversion: 2.1 time: 0.160576 Jul 16 21:50:19.945501 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3059/6113] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:50:19 2022] GET /compute/v2.1/servers/a2bd21dc-856d-41b0-8d5c-139452473742 => generated 1272 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:50:20.962267 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-564ec3d3-a3b6-4c15-80d1-da44cdffb1e4 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:50:20.962941 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-564ec3d3-a3b6-4c15-80d1-da44cdffb1e4 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: a2bd21dc-856d-41b0-8d5c-139452473742] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:50:20.970032 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-564ec3d3-a3b6-4c15-80d1-da44cdffb1e4 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:50:20.970497 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-564ec3d3-a3b6-4c15-80d1-da44cdffb1e4 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:50:20.971312 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-564ec3d3-a3b6-4c15-80d1-da44cdffb1e4 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:50:20.972250 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-564ec3d3-a3b6-4c15-80d1-da44cdffb1e4 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:50:21.306292 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-564ec3d3-a3b6-4c15-80d1-da44cdffb1e4 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:50:21.318005 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-564ec3d3-a3b6-4c15-80d1-da44cdffb1e4 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/a2bd21dc-856d-41b0-8d5c-139452473742" status: 200 len: 1314 microversion: 2.1 time: 0.357511 Jul 16 21:50:21.326097 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3055/6114] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:50:20 2022] GET /compute/v2.1/servers/a2bd21dc-856d-41b0-8d5c-139452473742 => generated 1314 bytes in 369 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:50:22.010329 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-b73fc4d5-8adc-4c7b-bd91-7a6d118d86e3 req-9b160b18-bcea-4f64-b2b2-0ae474f0fd99 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "a2bd21dc-856d-41b0-8d5c-139452473742", "tag": "85e05f99-40c7-4abf-a10c-a4f578c0caf4"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:50:22.021798 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-b73fc4d5-8adc-4c7b-bd91-7a6d118d86e3 req-9b160b18-bcea-4f64-b2b2-0ae474f0fd99 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:50:22.027168 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-b73fc4d5-8adc-4c7b-bd91-7a6d118d86e3 req-9b160b18-bcea-4f64-b2b2-0ae474f0fd99 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:50:22.042482 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-b73fc4d5-8adc-4c7b-bd91-7a6d118d86e3 req-9b160b18-bcea-4f64-b2b2-0ae474f0fd99 service nova] Creating event network-changed:85e05f99-40c7-4abf-a10c-a4f578c0caf4 for instance a2bd21dc-856d-41b0-8d5c-139452473742 on nch2-master-1 Jul 16 21:50:22.048863 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-b73fc4d5-8adc-4c7b-bd91-7a6d118d86e3 req-9b160b18-bcea-4f64-b2b2-0ae474f0fd99 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.042208 Jul 16 21:50:22.050171 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3060/6115] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:50:22 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:50:22.333803 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-44ab2096-ab84-4953-8a72-f22fe5a54498 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:50:22.334531 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-44ab2096-ab84-4953-8a72-f22fe5a54498 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: a2bd21dc-856d-41b0-8d5c-139452473742] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:50:22.342138 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-44ab2096-ab84-4953-8a72-f22fe5a54498 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:50:22.342686 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-44ab2096-ab84-4953-8a72-f22fe5a54498 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:50:22.343620 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-44ab2096-ab84-4953-8a72-f22fe5a54498 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:50:22.344138 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-44ab2096-ab84-4953-8a72-f22fe5a54498 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:50:22.557187 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-44ab2096-ab84-4953-8a72-f22fe5a54498 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:50:22.565522 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-44ab2096-ab84-4953-8a72-f22fe5a54498 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/a2bd21dc-856d-41b0-8d5c-139452473742" status: 200 len: 1314 microversion: 2.1 time: 0.235828 Jul 16 21:50:22.566236 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3056/6116] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:50:22 2022] GET /compute/v2.1/servers/a2bd21dc-856d-41b0-8d5c-139452473742 => generated 1314 bytes in 237 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:50:23.584306 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-751e3fea-3f11-48f6-acce-79c611721162 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:50:23.585007 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-751e3fea-3f11-48f6-acce-79c611721162 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: a2bd21dc-856d-41b0-8d5c-139452473742] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:50:23.591845 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-751e3fea-3f11-48f6-acce-79c611721162 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:50:23.592527 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-751e3fea-3f11-48f6-acce-79c611721162 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:50:23.593498 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-751e3fea-3f11-48f6-acce-79c611721162 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:50:23.594100 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-751e3fea-3f11-48f6-acce-79c611721162 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:50:23.743418 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-751e3fea-3f11-48f6-acce-79c611721162 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:50:23.751001 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-751e3fea-3f11-48f6-acce-79c611721162 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/a2bd21dc-856d-41b0-8d5c-139452473742" status: 200 len: 1314 microversion: 2.1 time: 0.172734 Jul 16 21:50:23.751600 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3061/6117] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:50:23 2022] GET /compute/v2.1/servers/a2bd21dc-856d-41b0-8d5c-139452473742 => generated 1314 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:50:24.767623 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-a5d2911a-4c6c-4560-8ef1-16006d8a9487 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:50:24.768243 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-a5d2911a-4c6c-4560-8ef1-16006d8a9487 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: a2bd21dc-856d-41b0-8d5c-139452473742] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:50:24.776841 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a5d2911a-4c6c-4560-8ef1-16006d8a9487 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:50:24.777935 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a5d2911a-4c6c-4560-8ef1-16006d8a9487 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:50:24.778543 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a5d2911a-4c6c-4560-8ef1-16006d8a9487 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:50:24.778990 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a5d2911a-4c6c-4560-8ef1-16006d8a9487 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:50:24.939494 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-a5d2911a-4c6c-4560-8ef1-16006d8a9487 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:50:24.950607 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-a5d2911a-4c6c-4560-8ef1-16006d8a9487 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/a2bd21dc-856d-41b0-8d5c-139452473742" status: 200 len: 1314 microversion: 2.1 time: 0.186257 Jul 16 21:50:24.951863 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3057/6118] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:50:24 2022] GET /compute/v2.1/servers/a2bd21dc-856d-41b0-8d5c-139452473742 => generated 1314 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:50:25.967852 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-dfc6dcee-207f-428f-adcf-503d62e32bf8 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:50:25.968832 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-dfc6dcee-207f-428f-adcf-503d62e32bf8 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: a2bd21dc-856d-41b0-8d5c-139452473742] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:50:25.976312 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_api:SELECT=65,nova_api:INSERT=6 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:50:25.978506 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-dfc6dcee-207f-428f-adcf-503d62e32bf8 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:50:25.979260 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-dfc6dcee-207f-428f-adcf-503d62e32bf8 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:50:25.980307 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-dfc6dcee-207f-428f-adcf-503d62e32bf8 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:50:25.981236 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-dfc6dcee-207f-428f-adcf-503d62e32bf8 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:50:25.986141 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=89 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:50:26.163779 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell0:SELECT=2 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:50:26.168770 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-dfc6dcee-207f-428f-adcf-503d62e32bf8 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:50:26.177939 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-dfc6dcee-207f-428f-adcf-503d62e32bf8 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/a2bd21dc-856d-41b0-8d5c-139452473742" status: 200 len: 1314 microversion: 2.1 time: 0.214010 Jul 16 21:50:26.187822 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3062/6119] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:50:25 2022] GET /compute/v2.1/servers/a2bd21dc-856d-41b0-8d5c-139452473742 => generated 1314 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:50:27.195166 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-25c08e0b-a841-4c7d-8fb9-f3484808e8f8 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:50:27.195795 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-25c08e0b-a841-4c7d-8fb9-f3484808e8f8 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: a2bd21dc-856d-41b0-8d5c-139452473742] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:50:27.198539 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell0:SELECT=2 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:50:27.199261 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_api:SELECT=45 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:50:27.204458 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-25c08e0b-a841-4c7d-8fb9-f3484808e8f8 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:50:27.205012 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-25c08e0b-a841-4c7d-8fb9-f3484808e8f8 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:50:27.205875 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-25c08e0b-a841-4c7d-8fb9-f3484808e8f8 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:50:27.207305 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-25c08e0b-a841-4c7d-8fb9-f3484808e8f8 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:50:27.213595 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=97 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:50:27.512692 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-25c08e0b-a841-4c7d-8fb9-f3484808e8f8 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:50:27.529846 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-25c08e0b-a841-4c7d-8fb9-f3484808e8f8 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/a2bd21dc-856d-41b0-8d5c-139452473742" status: 200 len: 1314 microversion: 2.1 time: 0.338587 Jul 16 21:50:27.531605 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3058/6120] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:50:27 2022] GET /compute/v2.1/servers/a2bd21dc-856d-41b0-8d5c-139452473742 => generated 1314 bytes in 341 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:50:28.433313 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-5b9494d0-589b-4fc7-b6e8-6441eec54522 req-1d77b4ea-fc95-4b46-b863-d608971fe5a6 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "a2bd21dc-856d-41b0-8d5c-139452473742", "name": "network-vif-plugged", "status": "completed", "tag": "85e05f99-40c7-4abf-a10c-a4f578c0caf4"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:50:28.439719 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell0:SELECT=6 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:50:28.446218 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-5b9494d0-589b-4fc7-b6e8-6441eec54522 req-1d77b4ea-fc95-4b46-b863-d608971fe5a6 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:50:28.446706 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-5b9494d0-589b-4fc7-b6e8-6441eec54522 req-1d77b4ea-fc95-4b46-b863-d608971fe5a6 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:50:28.460515 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-5b9494d0-589b-4fc7-b6e8-6441eec54522 req-1d77b4ea-fc95-4b46-b863-d608971fe5a6 service nova] Creating event network-vif-plugged:85e05f99-40c7-4abf-a10c-a4f578c0caf4 for instance a2bd21dc-856d-41b0-8d5c-139452473742 on nch2-master-1 Jul 16 21:50:28.466316 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-5b9494d0-589b-4fc7-b6e8-6441eec54522 req-1d77b4ea-fc95-4b46-b863-d608971fe5a6 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.038530 Jul 16 21:50:28.466876 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3063/6121] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:50:28 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:50:28.552056 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-864a5181-6c3e-410e-add9-e2500fedc793 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:50:28.553561 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-864a5181-6c3e-410e-add9-e2500fedc793 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: a2bd21dc-856d-41b0-8d5c-139452473742] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:50:28.562934 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-864a5181-6c3e-410e-add9-e2500fedc793 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:50:28.563765 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-864a5181-6c3e-410e-add9-e2500fedc793 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:50:28.564791 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-864a5181-6c3e-410e-add9-e2500fedc793 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:50:28.566263 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-864a5181-6c3e-410e-add9-e2500fedc793 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:50:28.756417 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-864a5181-6c3e-410e-add9-e2500fedc793 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:50:28.764043 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-864a5181-6c3e-410e-add9-e2500fedc793 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/a2bd21dc-856d-41b0-8d5c-139452473742" status: 200 len: 1314 microversion: 2.1 time: 0.220376 Jul 16 21:50:28.764626 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3059/6122] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:50:28 2022] GET /compute/v2.1/servers/a2bd21dc-856d-41b0-8d5c-139452473742 => generated 1314 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:50:29.782440 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-67cedbf7-16f2-4cbe-be1c-f21166616ca7 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:50:29.783190 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-67cedbf7-16f2-4cbe-be1c-f21166616ca7 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: a2bd21dc-856d-41b0-8d5c-139452473742] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:50:29.791228 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-67cedbf7-16f2-4cbe-be1c-f21166616ca7 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:50:29.791690 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-67cedbf7-16f2-4cbe-be1c-f21166616ca7 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:50:29.792932 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-67cedbf7-16f2-4cbe-be1c-f21166616ca7 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:50:29.792932 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-67cedbf7-16f2-4cbe-be1c-f21166616ca7 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:50:30.009092 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-67cedbf7-16f2-4cbe-be1c-f21166616ca7 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:50:30.019353 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-67cedbf7-16f2-4cbe-be1c-f21166616ca7 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/a2bd21dc-856d-41b0-8d5c-139452473742" status: 200 len: 1314 microversion: 2.1 time: 0.241242 Jul 16 21:50:30.035388 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3064/6123] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:50:29 2022] GET /compute/v2.1/servers/a2bd21dc-856d-41b0-8d5c-139452473742 => generated 1314 bytes in 258 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:50:31.037994 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-f7383e62-632f-45de-b728-0fda9ef034af tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:50:31.039287 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-f7383e62-632f-45de-b728-0fda9ef034af tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: a2bd21dc-856d-41b0-8d5c-139452473742] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:50:31.052592 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f7383e62-632f-45de-b728-0fda9ef034af tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:50:31.052592 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f7383e62-632f-45de-b728-0fda9ef034af tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:50:31.052592 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f7383e62-632f-45de-b728-0fda9ef034af tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:50:31.052592 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f7383e62-632f-45de-b728-0fda9ef034af tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:50:31.189932 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-f7383e62-632f-45de-b728-0fda9ef034af tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:50:31.197770 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-f7383e62-632f-45de-b728-0fda9ef034af tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/a2bd21dc-856d-41b0-8d5c-139452473742" status: 200 len: 1501 microversion: 2.1 time: 0.163982 Jul 16 21:50:31.198347 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3060/6124] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:50:31 2022] GET /compute/v2.1/servers/a2bd21dc-856d-41b0-8d5c-139452473742 => generated 1501 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:50:31.214702 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-6d41bb9b-e93a-4b4e-86f3-577ada350f8b tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Action: 'action', calling method: >, body: {"pause": {}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:50:31.215332 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-6d41bb9b-e93a-4b4e-86f3-577ada350f8b tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: a2bd21dc-856d-41b0-8d5c-139452473742] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:50:31.222478 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6d41bb9b-e93a-4b4e-86f3-577ada350f8b tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:50:31.223500 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6d41bb9b-e93a-4b4e-86f3-577ada350f8b tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:50:31.224552 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6d41bb9b-e93a-4b4e-86f3-577ada350f8b tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:50:31.225003 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6d41bb9b-e93a-4b4e-86f3-577ada350f8b tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:50:31.452419 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-6d41bb9b-e93a-4b4e-86f3-577ada350f8b tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "POST /compute/v2.1/servers/a2bd21dc-856d-41b0-8d5c-139452473742/action" status: 202 len: 0 microversion: 2.1 time: 0.241447 Jul 16 21:50:31.453385 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3065/6125] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 21:50:31 2022] POST /compute/v2.1/servers/a2bd21dc-856d-41b0-8d5c-139452473742/action => generated 0 bytes in 243 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 21:50:31.461584 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-0f2b3566-705d-4dd7-97cb-bdc0d5c5d151 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:50:31.462311 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-0f2b3566-705d-4dd7-97cb-bdc0d5c5d151 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: a2bd21dc-856d-41b0-8d5c-139452473742] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:50:31.469043 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0f2b3566-705d-4dd7-97cb-bdc0d5c5d151 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:50:31.469595 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0f2b3566-705d-4dd7-97cb-bdc0d5c5d151 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:50:31.470611 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0f2b3566-705d-4dd7-97cb-bdc0d5c5d151 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:50:31.471161 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0f2b3566-705d-4dd7-97cb-bdc0d5c5d151 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:50:31.674660 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-0f2b3566-705d-4dd7-97cb-bdc0d5c5d151 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:50:31.684664 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-0f2b3566-705d-4dd7-97cb-bdc0d5c5d151 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/a2bd21dc-856d-41b0-8d5c-139452473742" status: 200 len: 1506 microversion: 2.1 time: 0.226032 Jul 16 21:50:31.686277 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3061/6126] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:50:31 2022] GET /compute/v2.1/servers/a2bd21dc-856d-41b0-8d5c-139452473742 => generated 1506 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:50:32.705953 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-87844269-5f60-4822-9565-61c0276d5578 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:50:32.706839 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-87844269-5f60-4822-9565-61c0276d5578 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: a2bd21dc-856d-41b0-8d5c-139452473742] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:50:32.713456 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-87844269-5f60-4822-9565-61c0276d5578 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:50:32.713925 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-87844269-5f60-4822-9565-61c0276d5578 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:50:32.714716 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-87844269-5f60-4822-9565-61c0276d5578 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:50:32.715220 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-87844269-5f60-4822-9565-61c0276d5578 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:50:32.851593 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-87844269-5f60-4822-9565-61c0276d5578 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:50:32.859171 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-87844269-5f60-4822-9565-61c0276d5578 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/a2bd21dc-856d-41b0-8d5c-139452473742" status: 200 len: 1486 microversion: 2.1 time: 0.158643 Jul 16 21:50:32.859745 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3066/6127] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:50:32 2022] GET /compute/v2.1/servers/a2bd21dc-856d-41b0-8d5c-139452473742 => generated 1486 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:50:32.877156 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-2c3f9258-869a-4426-ab90-a654c3e80a7b tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Action: 'action', calling method: >, body: {"shelve": {}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:50:32.877880 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-2c3f9258-869a-4426-ab90-a654c3e80a7b tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: a2bd21dc-856d-41b0-8d5c-139452473742] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:50:32.885316 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2c3f9258-869a-4426-ab90-a654c3e80a7b tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:50:32.885856 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2c3f9258-869a-4426-ab90-a654c3e80a7b tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:50:32.886708 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2c3f9258-869a-4426-ab90-a654c3e80a7b tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:50:32.887254 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2c3f9258-869a-4426-ab90-a654c3e80a7b tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:50:32.914932 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.objects.instance [None req-2c3f9258-869a-4426-ab90-a654c3e80a7b tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lazy-loading 'flavor' on Instance uuid a2bd21dc-856d-41b0-8d5c-139452473742 {{(pid=138680) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:50:33.467874 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-2c3f9258-869a-4426-ab90-a654c3e80a7b tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "POST /compute/v2.1/servers/a2bd21dc-856d-41b0-8d5c-139452473742/action" status: 202 len: 0 microversion: 2.1 time: 0.594689 Jul 16 21:50:33.469222 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3062/6128] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 21:50:32 2022] POST /compute/v2.1/servers/a2bd21dc-856d-41b0-8d5c-139452473742/action => generated 0 bytes in 597 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 21:50:33.480820 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-37852d29-f9cb-4b23-b5fd-f5a1ecf4461e tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:50:33.482606 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-37852d29-f9cb-4b23-b5fd-f5a1ecf4461e tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: a2bd21dc-856d-41b0-8d5c-139452473742] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:50:33.498385 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-37852d29-f9cb-4b23-b5fd-f5a1ecf4461e tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:50:33.499118 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-37852d29-f9cb-4b23-b5fd-f5a1ecf4461e tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:50:33.500412 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-37852d29-f9cb-4b23-b5fd-f5a1ecf4461e tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:50:33.501463 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-37852d29-f9cb-4b23-b5fd-f5a1ecf4461e tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:50:33.675586 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-37852d29-f9cb-4b23-b5fd-f5a1ecf4461e tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:50:33.684180 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-37852d29-f9cb-4b23-b5fd-f5a1ecf4461e tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/a2bd21dc-856d-41b0-8d5c-139452473742" status: 200 len: 1492 microversion: 2.1 time: 0.209288 Jul 16 21:50:33.684833 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3067/6129] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:50:33 2022] GET /compute/v2.1/servers/a2bd21dc-856d-41b0-8d5c-139452473742 => generated 1492 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:50:34.705906 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-49613b93-50d3-48cd-a709-0fb7b6b2d023 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:50:34.706608 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-49613b93-50d3-48cd-a709-0fb7b6b2d023 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: a2bd21dc-856d-41b0-8d5c-139452473742] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:50:34.714076 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-49613b93-50d3-48cd-a709-0fb7b6b2d023 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:50:34.714528 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-49613b93-50d3-48cd-a709-0fb7b6b2d023 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:50:34.715324 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-49613b93-50d3-48cd-a709-0fb7b6b2d023 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:50:34.715763 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-49613b93-50d3-48cd-a709-0fb7b6b2d023 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:50:34.856391 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-49613b93-50d3-48cd-a709-0fb7b6b2d023 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:50:34.865059 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-49613b93-50d3-48cd-a709-0fb7b6b2d023 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/a2bd21dc-856d-41b0-8d5c-139452473742" status: 200 len: 1513 microversion: 2.1 time: 0.164269 Jul 16 21:50:34.865653 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3063/6130] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:50:34 2022] GET /compute/v2.1/servers/a2bd21dc-856d-41b0-8d5c-139452473742 => generated 1513 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:50:35.882028 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-2d0519f5-833e-4547-85f0-886ef88a4967 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:50:35.882672 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-2d0519f5-833e-4547-85f0-886ef88a4967 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: a2bd21dc-856d-41b0-8d5c-139452473742] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:50:35.889758 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2d0519f5-833e-4547-85f0-886ef88a4967 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:50:35.890240 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2d0519f5-833e-4547-85f0-886ef88a4967 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:50:35.891082 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2d0519f5-833e-4547-85f0-886ef88a4967 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:50:35.891847 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2d0519f5-833e-4547-85f0-886ef88a4967 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:50:36.039221 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-2d0519f5-833e-4547-85f0-886ef88a4967 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:50:36.049103 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-2d0519f5-833e-4547-85f0-886ef88a4967 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/a2bd21dc-856d-41b0-8d5c-139452473742" status: 200 len: 1513 microversion: 2.1 time: 0.171283 Jul 16 21:50:36.049694 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3068/6131] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:50:35 2022] GET /compute/v2.1/servers/a2bd21dc-856d-41b0-8d5c-139452473742 => generated 1513 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:50:37.071677 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-08c35759-56b5-4224-bf4a-086c1d6e7ed0 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:50:37.072905 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-08c35759-56b5-4224-bf4a-086c1d6e7ed0 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: a2bd21dc-856d-41b0-8d5c-139452473742] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:50:37.081287 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-08c35759-56b5-4224-bf4a-086c1d6e7ed0 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:50:37.081758 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-08c35759-56b5-4224-bf4a-086c1d6e7ed0 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:50:37.082652 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-08c35759-56b5-4224-bf4a-086c1d6e7ed0 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:50:37.083273 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-08c35759-56b5-4224-bf4a-086c1d6e7ed0 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:50:37.266231 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-08c35759-56b5-4224-bf4a-086c1d6e7ed0 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:50:37.275523 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-08c35759-56b5-4224-bf4a-086c1d6e7ed0 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/a2bd21dc-856d-41b0-8d5c-139452473742" status: 200 len: 1508 microversion: 2.1 time: 0.213139 Jul 16 21:50:37.277027 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3064/6132] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:50:37 2022] GET /compute/v2.1/servers/a2bd21dc-856d-41b0-8d5c-139452473742 => generated 1508 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:50:38.068041 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-49bdb207-6564-494f-9aea-151965574c9d req-221ecaee-d53d-4ecc-a155-6e0756fc622f service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "a2bd21dc-856d-41b0-8d5c-139452473742", "name": "network-vif-plugged", "status": "completed", "tag": "85e05f99-40c7-4abf-a10c-a4f578c0caf4"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:50:38.078416 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-49bdb207-6564-494f-9aea-151965574c9d req-221ecaee-d53d-4ecc-a155-6e0756fc622f service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:50:38.078966 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-49bdb207-6564-494f-9aea-151965574c9d req-221ecaee-d53d-4ecc-a155-6e0756fc622f service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:50:38.095237 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-49bdb207-6564-494f-9aea-151965574c9d req-221ecaee-d53d-4ecc-a155-6e0756fc622f service nova] Creating event network-vif-plugged:85e05f99-40c7-4abf-a10c-a4f578c0caf4 for instance a2bd21dc-856d-41b0-8d5c-139452473742 on nch2-master-1 Jul 16 21:50:38.103512 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-49bdb207-6564-494f-9aea-151965574c9d req-221ecaee-d53d-4ecc-a155-6e0756fc622f service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.038982 Jul 16 21:50:38.104916 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3069/6133] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:50:38 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:50:38.296183 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-53e8ea7e-68a8-4a90-8f17-42cee342ee3f tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:50:38.296772 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-53e8ea7e-68a8-4a90-8f17-42cee342ee3f tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: a2bd21dc-856d-41b0-8d5c-139452473742] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:50:38.304371 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-53e8ea7e-68a8-4a90-8f17-42cee342ee3f tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:50:38.305089 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-53e8ea7e-68a8-4a90-8f17-42cee342ee3f tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:50:38.306138 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-53e8ea7e-68a8-4a90-8f17-42cee342ee3f tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:50:38.307276 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-53e8ea7e-68a8-4a90-8f17-42cee342ee3f tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:50:38.544849 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-53e8ea7e-68a8-4a90-8f17-42cee342ee3f tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:50:38.555212 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-53e8ea7e-68a8-4a90-8f17-42cee342ee3f tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/a2bd21dc-856d-41b0-8d5c-139452473742" status: 200 len: 1508 microversion: 2.1 time: 0.263055 Jul 16 21:50:38.557228 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3065/6134] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:50:38 2022] GET /compute/v2.1/servers/a2bd21dc-856d-41b0-8d5c-139452473742 => generated 1508 bytes in 266 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:50:39.578343 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-b9ffb689-d148-4915-883f-3b0911c368ca tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:50:39.579950 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-b9ffb689-d148-4915-883f-3b0911c368ca tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: a2bd21dc-856d-41b0-8d5c-139452473742] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:50:39.592105 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b9ffb689-d148-4915-883f-3b0911c368ca tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:50:39.592815 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b9ffb689-d148-4915-883f-3b0911c368ca tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:50:39.593766 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b9ffb689-d148-4915-883f-3b0911c368ca tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:50:39.594402 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b9ffb689-d148-4915-883f-3b0911c368ca tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:50:39.782092 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-b9ffb689-d148-4915-883f-3b0911c368ca tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:50:39.790783 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-b9ffb689-d148-4915-883f-3b0911c368ca tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/a2bd21dc-856d-41b0-8d5c-139452473742" status: 200 len: 1508 microversion: 2.1 time: 0.219772 Jul 16 21:50:39.792230 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3070/6135] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:50:39 2022] GET /compute/v2.1/servers/a2bd21dc-856d-41b0-8d5c-139452473742 => generated 1508 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:50:40.810134 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-f20f0a84-19ad-4692-960d-b1c373b35cec tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:50:40.810826 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-f20f0a84-19ad-4692-960d-b1c373b35cec tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: a2bd21dc-856d-41b0-8d5c-139452473742] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:50:40.826007 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f20f0a84-19ad-4692-960d-b1c373b35cec tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:50:40.826708 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f20f0a84-19ad-4692-960d-b1c373b35cec tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:50:40.827791 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f20f0a84-19ad-4692-960d-b1c373b35cec tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:50:40.828163 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f20f0a84-19ad-4692-960d-b1c373b35cec tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:50:41.019559 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-f20f0a84-19ad-4692-960d-b1c373b35cec tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:50:41.028266 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-f20f0a84-19ad-4692-960d-b1c373b35cec tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/a2bd21dc-856d-41b0-8d5c-139452473742" status: 200 len: 1508 microversion: 2.1 time: 0.222202 Jul 16 21:50:41.029522 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3066/6136] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:50:40 2022] GET /compute/v2.1/servers/a2bd21dc-856d-41b0-8d5c-139452473742 => generated 1508 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:50:42.047423 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-35fd185b-fca6-41eb-a983-a87cd1742cec tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:50:42.048689 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-35fd185b-fca6-41eb-a983-a87cd1742cec tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: a2bd21dc-856d-41b0-8d5c-139452473742] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:50:42.056986 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-35fd185b-fca6-41eb-a983-a87cd1742cec tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:50:42.057964 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-35fd185b-fca6-41eb-a983-a87cd1742cec tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:50:42.058857 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-35fd185b-fca6-41eb-a983-a87cd1742cec tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:50:42.059419 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-35fd185b-fca6-41eb-a983-a87cd1742cec tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:50:42.236341 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-35fd185b-fca6-41eb-a983-a87cd1742cec tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:50:42.247537 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-35fd185b-fca6-41eb-a983-a87cd1742cec tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/a2bd21dc-856d-41b0-8d5c-139452473742" status: 200 len: 1508 microversion: 2.1 time: 0.204426 Jul 16 21:50:42.251821 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3071/6137] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:50:42 2022] GET /compute/v2.1/servers/a2bd21dc-856d-41b0-8d5c-139452473742 => generated 1508 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:50:43.268403 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-a030383e-fe03-4be1-96b1-d506947eebac tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:50:43.268965 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-a030383e-fe03-4be1-96b1-d506947eebac tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: a2bd21dc-856d-41b0-8d5c-139452473742] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:50:43.276509 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a030383e-fe03-4be1-96b1-d506947eebac tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:50:43.276967 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a030383e-fe03-4be1-96b1-d506947eebac tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:50:43.277746 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a030383e-fe03-4be1-96b1-d506947eebac tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:50:43.278226 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a030383e-fe03-4be1-96b1-d506947eebac tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:50:43.426414 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-a030383e-fe03-4be1-96b1-d506947eebac tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:50:43.445564 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-a030383e-fe03-4be1-96b1-d506947eebac tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/a2bd21dc-856d-41b0-8d5c-139452473742" status: 200 len: 1508 microversion: 2.1 time: 0.170107 Jul 16 21:50:43.445564 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3067/6138] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:50:43 2022] GET /compute/v2.1/servers/a2bd21dc-856d-41b0-8d5c-139452473742 => generated 1508 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:50:44.457307 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-35be826c-1abd-4623-9352-8c2731642bbf tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:50:44.458510 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-35be826c-1abd-4623-9352-8c2731642bbf tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: a2bd21dc-856d-41b0-8d5c-139452473742] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:50:44.466351 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-35be826c-1abd-4623-9352-8c2731642bbf tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:50:44.466827 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-35be826c-1abd-4623-9352-8c2731642bbf tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:50:44.467663 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-35be826c-1abd-4623-9352-8c2731642bbf tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:50:44.468123 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-35be826c-1abd-4623-9352-8c2731642bbf tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:50:44.637645 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-35be826c-1abd-4623-9352-8c2731642bbf tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:50:44.647182 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-35be826c-1abd-4623-9352-8c2731642bbf tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/a2bd21dc-856d-41b0-8d5c-139452473742" status: 200 len: 1508 microversion: 2.1 time: 0.199245 Jul 16 21:50:44.648153 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3072/6139] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:50:44 2022] GET /compute/v2.1/servers/a2bd21dc-856d-41b0-8d5c-139452473742 => generated 1508 bytes in 201 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:50:45.667703 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-743e04e2-8748-430d-ab81-d24cca26ec8a tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:50:45.668445 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-743e04e2-8748-430d-ab81-d24cca26ec8a tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: a2bd21dc-856d-41b0-8d5c-139452473742] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:50:45.677016 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-743e04e2-8748-430d-ab81-d24cca26ec8a tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:50:45.677874 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-743e04e2-8748-430d-ab81-d24cca26ec8a tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:50:45.678764 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-743e04e2-8748-430d-ab81-d24cca26ec8a tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:50:45.679435 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-743e04e2-8748-430d-ab81-d24cca26ec8a tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:50:45.870098 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-743e04e2-8748-430d-ab81-d24cca26ec8a tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:50:45.880364 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-743e04e2-8748-430d-ab81-d24cca26ec8a tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/a2bd21dc-856d-41b0-8d5c-139452473742" status: 200 len: 1508 microversion: 2.1 time: 0.217929 Jul 16 21:50:45.881544 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3068/6140] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:50:45 2022] GET /compute/v2.1/servers/a2bd21dc-856d-41b0-8d5c-139452473742 => generated 1508 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:50:46.899206 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-8ebaf5f6-86dd-4a07-b439-d3adab88e60e tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:50:46.900475 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-8ebaf5f6-86dd-4a07-b439-d3adab88e60e tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: a2bd21dc-856d-41b0-8d5c-139452473742] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:50:46.930385 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8ebaf5f6-86dd-4a07-b439-d3adab88e60e tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:50:46.931922 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8ebaf5f6-86dd-4a07-b439-d3adab88e60e tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:50:46.933274 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8ebaf5f6-86dd-4a07-b439-d3adab88e60e tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:50:46.934049 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8ebaf5f6-86dd-4a07-b439-d3adab88e60e tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:50:47.125619 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-8ebaf5f6-86dd-4a07-b439-d3adab88e60e tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:50:47.133911 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-8ebaf5f6-86dd-4a07-b439-d3adab88e60e tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/a2bd21dc-856d-41b0-8d5c-139452473742" status: 200 len: 1508 microversion: 2.1 time: 0.239738 Jul 16 21:50:47.134331 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3073/6141] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:50:46 2022] GET /compute/v2.1/servers/a2bd21dc-856d-41b0-8d5c-139452473742 => generated 1508 bytes in 241 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:50:48.152313 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-f3a9464a-01ad-4af6-b555-bc26486b57af tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:50:48.153500 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-f3a9464a-01ad-4af6-b555-bc26486b57af tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: a2bd21dc-856d-41b0-8d5c-139452473742] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:50:48.161126 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f3a9464a-01ad-4af6-b555-bc26486b57af tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:50:48.161594 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f3a9464a-01ad-4af6-b555-bc26486b57af tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:50:48.162384 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f3a9464a-01ad-4af6-b555-bc26486b57af tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:50:48.162887 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f3a9464a-01ad-4af6-b555-bc26486b57af tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:50:48.350701 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-f3a9464a-01ad-4af6-b555-bc26486b57af tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:50:48.358840 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-f3a9464a-01ad-4af6-b555-bc26486b57af tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/a2bd21dc-856d-41b0-8d5c-139452473742" status: 200 len: 1505 microversion: 2.1 time: 0.211075 Jul 16 21:50:48.359548 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3069/6142] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:50:48 2022] GET /compute/v2.1/servers/a2bd21dc-856d-41b0-8d5c-139452473742 => generated 1505 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:50:49.387553 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-2d600f73-fd73-4eaa-88d2-36eb03bcf494 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:50:49.388185 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-2d600f73-fd73-4eaa-88d2-36eb03bcf494 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: a2bd21dc-856d-41b0-8d5c-139452473742] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:50:49.395090 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2d600f73-fd73-4eaa-88d2-36eb03bcf494 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:50:49.395556 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2d600f73-fd73-4eaa-88d2-36eb03bcf494 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:50:49.396338 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2d600f73-fd73-4eaa-88d2-36eb03bcf494 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:50:49.396795 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2d600f73-fd73-4eaa-88d2-36eb03bcf494 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:50:49.540199 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-2d600f73-fd73-4eaa-88d2-36eb03bcf494 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:50:49.548497 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-2d600f73-fd73-4eaa-88d2-36eb03bcf494 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/a2bd21dc-856d-41b0-8d5c-139452473742" status: 200 len: 1505 microversion: 2.1 time: 0.164954 Jul 16 21:50:49.549071 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3074/6143] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:50:49 2022] GET /compute/v2.1/servers/a2bd21dc-856d-41b0-8d5c-139452473742 => generated 1505 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:50:50.565636 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-3e3c1cdc-729c-4cf1-ab7b-7422ad7f6da7 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:50:50.569303 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-3e3c1cdc-729c-4cf1-ab7b-7422ad7f6da7 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: a2bd21dc-856d-41b0-8d5c-139452473742] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:50:50.573925 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3e3c1cdc-729c-4cf1-ab7b-7422ad7f6da7 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:50:50.574381 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3e3c1cdc-729c-4cf1-ab7b-7422ad7f6da7 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:50:50.575202 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3e3c1cdc-729c-4cf1-ab7b-7422ad7f6da7 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:50:50.575648 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3e3c1cdc-729c-4cf1-ab7b-7422ad7f6da7 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:50:50.725765 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-3e3c1cdc-729c-4cf1-ab7b-7422ad7f6da7 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:50:50.742020 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-3e3c1cdc-729c-4cf1-ab7b-7422ad7f6da7 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/a2bd21dc-856d-41b0-8d5c-139452473742" status: 200 len: 1505 microversion: 2.1 time: 0.180367 Jul 16 21:50:50.742620 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3070/6144] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:50:50 2022] GET /compute/v2.1/servers/a2bd21dc-856d-41b0-8d5c-139452473742 => generated 1505 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:50:51.760901 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-06301a8b-094b-4fe2-8857-a7946c957ed2 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:50:51.766124 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-06301a8b-094b-4fe2-8857-a7946c957ed2 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: a2bd21dc-856d-41b0-8d5c-139452473742] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:50:51.769056 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-06301a8b-094b-4fe2-8857-a7946c957ed2 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:50:51.769629 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-06301a8b-094b-4fe2-8857-a7946c957ed2 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:50:51.770787 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-06301a8b-094b-4fe2-8857-a7946c957ed2 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:50:51.771407 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-06301a8b-094b-4fe2-8857-a7946c957ed2 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:50:51.910878 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-06301a8b-094b-4fe2-8857-a7946c957ed2 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:50:51.919379 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-06301a8b-094b-4fe2-8857-a7946c957ed2 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/a2bd21dc-856d-41b0-8d5c-139452473742" status: 200 len: 1448 microversion: 2.1 time: 0.162673 Jul 16 21:50:51.919970 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3075/6145] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:50:51 2022] GET /compute/v2.1/servers/a2bd21dc-856d-41b0-8d5c-139452473742 => generated 1448 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:50:51.941579 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-2d7a7ef7-d3e0-4487-bfb0-da8edd3444dd tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:50:51.942211 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-2d7a7ef7-d3e0-4487-bfb0-da8edd3444dd tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: 3618ceb8-9d10-4219-8885-d2d5004c71ad] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:50:51.955194 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2d7a7ef7-d3e0-4487-bfb0-da8edd3444dd tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:50:51.957328 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2d7a7ef7-d3e0-4487-bfb0-da8edd3444dd tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:50:51.957328 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2d7a7ef7-d3e0-4487-bfb0-da8edd3444dd tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:50:51.957328 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2d7a7ef7-d3e0-4487-bfb0-da8edd3444dd tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:50:52.072998 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-6bd69103-8ee8-4c35-86a9-99737bad153f req-0eae6331-8531-4f69-9129-97c84e077089 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "a2bd21dc-856d-41b0-8d5c-139452473742", "name": "network-vif-unplugged", "status": "completed", "tag": "85e05f99-40c7-4abf-a10c-a4f578c0caf4"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:50:52.084391 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-6bd69103-8ee8-4c35-86a9-99737bad153f req-0eae6331-8531-4f69-9129-97c84e077089 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:50:52.084856 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-6bd69103-8ee8-4c35-86a9-99737bad153f req-0eae6331-8531-4f69-9129-97c84e077089 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:50:52.100969 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.compute.server_external_events [req-6bd69103-8ee8-4c35-86a9-99737bad153f req-0eae6331-8531-4f69-9129-97c84e077089 service nova] Unable to find a host for instance a2bd21dc-856d-41b0-8d5c-139452473742. Dropping event network-vif-unplugged {{(pid=138681) create /opt/stack/nova/nova/api/openstack/compute/server_external_events.py:143}} Jul 16 21:50:52.101911 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-6bd69103-8ee8-4c35-86a9-99737bad153f req-0eae6331-8531-4f69-9129-97c84e077089 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 207 len: 182 microversion: 2.1 time: 0.036529 Jul 16 21:50:52.102767 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3076/6146] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:50:52 2022] POST /compute/v2.1/os-server-external-events => generated 182 bytes in 38 msecs (HTTP/1.1 207) 9 headers in 367 bytes (2 switches on core 0) Jul 16 21:50:52.122199 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-2d7a7ef7-d3e0-4487-bfb0-da8edd3444dd tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:50:52.130200 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-2d7a7ef7-d3e0-4487-bfb0-da8edd3444dd tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad" status: 200 len: 1665 microversion: 2.1 time: 0.192941 Jul 16 21:50:52.130846 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3071/6147] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:50:51 2022] GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad => generated 1665 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:50:52.151573 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-d3a2b2e6-b59f-4ce0-84fe-a323a87eaefb tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:50:52.152204 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-d3a2b2e6-b59f-4ce0-84fe-a323a87eaefb tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: 3618ceb8-9d10-4219-8885-d2d5004c71ad] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:50:52.158876 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d3a2b2e6-b59f-4ce0-84fe-a323a87eaefb tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:50:52.159459 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d3a2b2e6-b59f-4ce0-84fe-a323a87eaefb tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:50:52.161450 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d3a2b2e6-b59f-4ce0-84fe-a323a87eaefb tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:50:52.161899 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d3a2b2e6-b59f-4ce0-84fe-a323a87eaefb tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:50:52.308270 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-d3a2b2e6-b59f-4ce0-84fe-a323a87eaefb tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:50:52.316108 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-d3a2b2e6-b59f-4ce0-84fe-a323a87eaefb tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad" status: 200 len: 1665 microversion: 2.1 time: 0.168655 Jul 16 21:50:52.316685 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3077/6148] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:50:52 2022] GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad => generated 1665 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:50:52.332063 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-0507c836-fa5a-4faa-a317-3cac805110de tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:50:52.332711 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-0507c836-fa5a-4faa-a317-3cac805110de tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: 3618ceb8-9d10-4219-8885-d2d5004c71ad] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:50:52.339288 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0507c836-fa5a-4faa-a317-3cac805110de tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:50:52.339747 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0507c836-fa5a-4faa-a317-3cac805110de tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:50:52.340524 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0507c836-fa5a-4faa-a317-3cac805110de tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:50:52.340969 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0507c836-fa5a-4faa-a317-3cac805110de tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:50:52.477176 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-0507c836-fa5a-4faa-a317-3cac805110de tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:50:52.479698 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:50:52.485701 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-0507c836-fa5a-4faa-a317-3cac805110de tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad" status: 200 len: 1665 microversion: 2.1 time: 0.157667 Jul 16 21:50:52.486281 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3072/6149] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:50:52 2022] GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad => generated 1665 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:50:52.502143 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-dd535fb0-ff34-4b08-bcf1-8bdc5c105426 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:50:52.502823 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-dd535fb0-ff34-4b08-bcf1-8bdc5c105426 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: 3618ceb8-9d10-4219-8885-d2d5004c71ad] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:50:52.509135 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-dd535fb0-ff34-4b08-bcf1-8bdc5c105426 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:50:52.509607 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-dd535fb0-ff34-4b08-bcf1-8bdc5c105426 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:50:52.510431 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-dd535fb0-ff34-4b08-bcf1-8bdc5c105426 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:50:52.510899 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-dd535fb0-ff34-4b08-bcf1-8bdc5c105426 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:50:52.652137 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-dd535fb0-ff34-4b08-bcf1-8bdc5c105426 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:50:52.660455 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-dd535fb0-ff34-4b08-bcf1-8bdc5c105426 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad" status: 200 len: 1665 microversion: 2.1 time: 0.162508 Jul 16 21:50:52.661698 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3078/6150] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:50:52 2022] GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad => generated 1665 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:50:52.678740 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-4683d338-16ad-4221-927f-41ebf4cb389d tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Action: 'action', calling method: >, body: {"os-stop": {}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:50:52.685686 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4683d338-16ad-4221-927f-41ebf4cb389d tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:50:52.686260 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4683d338-16ad-4221-927f-41ebf4cb389d tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:50:52.709612 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-4683d338-16ad-4221-927f-41ebf4cb389d tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: 3618ceb8-9d10-4219-8885-d2d5004c71ad] Going to try to stop instance {{(pid=138680) force_stop /opt/stack/nova/nova/compute/api.py:2707}} Jul 16 21:50:52.942952 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-4683d338-16ad-4221-927f-41ebf4cb389d tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "POST /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad/action" status: 202 len: 0 microversion: 2.1 time: 0.268254 Jul 16 21:50:52.943679 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3073/6151] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 21:50:52 2022] POST /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad/action => generated 0 bytes in 270 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 21:50:52.952045 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-8b7e971f-9874-4821-9bf4-e0f814fcadbe tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:50:52.952676 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-8b7e971f-9874-4821-9bf4-e0f814fcadbe tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: 3618ceb8-9d10-4219-8885-d2d5004c71ad] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:50:52.960351 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8b7e971f-9874-4821-9bf4-e0f814fcadbe tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:50:52.960812 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8b7e971f-9874-4821-9bf4-e0f814fcadbe tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:50:52.961588 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8b7e971f-9874-4821-9bf4-e0f814fcadbe tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:50:52.962040 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8b7e971f-9874-4821-9bf4-e0f814fcadbe tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:50:53.112675 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-8b7e971f-9874-4821-9bf4-e0f814fcadbe tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:50:53.115837 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:50:53.121876 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-8b7e971f-9874-4821-9bf4-e0f814fcadbe tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad" status: 200 len: 1675 microversion: 2.1 time: 0.173828 Jul 16 21:50:53.122458 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3079/6152] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:50:52 2022] GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad => generated 1675 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:50:54.140967 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-352e65c9-245d-46de-a76b-d11be422f2b4 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:50:54.141672 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-352e65c9-245d-46de-a76b-d11be422f2b4 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: 3618ceb8-9d10-4219-8885-d2d5004c71ad] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:50:54.149595 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-352e65c9-245d-46de-a76b-d11be422f2b4 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:50:54.150048 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-352e65c9-245d-46de-a76b-d11be422f2b4 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:50:54.150987 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-352e65c9-245d-46de-a76b-d11be422f2b4 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:50:54.151450 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-352e65c9-245d-46de-a76b-d11be422f2b4 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:50:54.391070 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-352e65c9-245d-46de-a76b-d11be422f2b4 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:50:54.399482 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-352e65c9-245d-46de-a76b-d11be422f2b4 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad" status: 200 len: 1675 microversion: 2.1 time: 0.263822 Jul 16 21:50:54.400073 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3074/6153] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:50:54 2022] GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad => generated 1675 bytes in 265 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:50:55.031651 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-1d14b56d-79f5-4695-be5e-c3fedb836303 tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:50:55.032170 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-1d14b56d-79f5-4695-be5e-c3fedb836303 tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] [instance: a83a3f66-378b-40a4-baf0-0d67c58f1524] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:50:55.037605 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-62c9b39c-8c5a-4fd5-a45d-dec58919ca14 req-eeb55ba8-81a7-4d46-9180-a3fd61f72f5e service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "a83a3f66-378b-40a4-baf0-0d67c58f1524", "tag": "3f6f502f-e93a-46fe-b6c5-70ff0daa5af0"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:50:55.050601 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-62c9b39c-8c5a-4fd5-a45d-dec58919ca14 req-eeb55ba8-81a7-4d46-9180-a3fd61f72f5e service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:50:55.052788 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1d14b56d-79f5-4695-be5e-c3fedb836303 tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:50:55.058241 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1d14b56d-79f5-4695-be5e-c3fedb836303 tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:50:55.058241 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1d14b56d-79f5-4695-be5e-c3fedb836303 tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:50:55.058241 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1d14b56d-79f5-4695-be5e-c3fedb836303 tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:50:55.058530 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-62c9b39c-8c5a-4fd5-a45d-dec58919ca14 req-eeb55ba8-81a7-4d46-9180-a3fd61f72f5e service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:50:55.070740 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-62c9b39c-8c5a-4fd5-a45d-dec58919ca14 req-eeb55ba8-81a7-4d46-9180-a3fd61f72f5e service nova] Creating event network-changed:3f6f502f-e93a-46fe-b6c5-70ff0daa5af0 for instance a83a3f66-378b-40a4-baf0-0d67c58f1524 on nch2-master-1 Jul 16 21:50:55.076358 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-62c9b39c-8c5a-4fd5-a45d-dec58919ca14 req-eeb55ba8-81a7-4d46-9180-a3fd61f72f5e service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.051268 Jul 16 21:50:55.076971 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3075/6154] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:50:55 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:50:55.094030 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-1d14b56d-79f5-4695-be5e-c3fedb836303 tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] [instance: a83a3f66-378b-40a4-baf0-0d67c58f1524] Going to try to terminate instance {{(pid=138681) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 16 21:50:55.418611 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-aa07386c-e673-4bda-8f11-303961a2b501 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:50:55.419423 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-aa07386c-e673-4bda-8f11-303961a2b501 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: 3618ceb8-9d10-4219-8885-d2d5004c71ad] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:50:55.428671 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-aa07386c-e673-4bda-8f11-303961a2b501 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:50:55.429414 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-aa07386c-e673-4bda-8f11-303961a2b501 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:50:55.430231 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-aa07386c-e673-4bda-8f11-303961a2b501 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:50:55.430675 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-aa07386c-e673-4bda-8f11-303961a2b501 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:50:55.469109 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-1d14b56d-79f5-4695-be5e-c3fedb836303 tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] 10.222.0.16 "DELETE /compute/v2.1/servers/a83a3f66-378b-40a4-baf0-0d67c58f1524" status: 204 len: 0 microversion: 2.1 time: 0.445480 Jul 16 21:50:55.470645 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3080/6155] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:50:55 2022] DELETE /compute/v2.1/servers/a83a3f66-378b-40a4-baf0-0d67c58f1524 => generated 0 bytes in 448 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 16 21:50:55.480660 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-5c3a0dbb-3445-42d5-82fb-091939cc4931 tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:50:55.481567 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-5c3a0dbb-3445-42d5-82fb-091939cc4931 tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] [instance: a83a3f66-378b-40a4-baf0-0d67c58f1524] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:50:55.489530 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5c3a0dbb-3445-42d5-82fb-091939cc4931 tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:50:55.490471 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5c3a0dbb-3445-42d5-82fb-091939cc4931 tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:50:55.491855 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5c3a0dbb-3445-42d5-82fb-091939cc4931 tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:50:55.492463 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5c3a0dbb-3445-42d5-82fb-091939cc4931 tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:50:55.683382 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-aa07386c-e673-4bda-8f11-303961a2b501 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:50:55.691939 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-aa07386c-e673-4bda-8f11-303961a2b501 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad" status: 200 len: 1675 microversion: 2.1 time: 0.279288 Jul 16 21:50:55.693518 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3076/6156] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:50:55 2022] GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad => generated 1675 bytes in 282 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:50:55.703288 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-5c3a0dbb-3445-42d5-82fb-091939cc4931 tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '001ed11a062a4cb08a52d1294ef5880b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8f296b14a61a4997bf411c9b0fdc39d5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:50:55.711784 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-5c3a0dbb-3445-42d5-82fb-091939cc4931 tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] 10.222.0.16 "GET /compute/v2.1/servers/a83a3f66-378b-40a4-baf0-0d67c58f1524" status: 200 len: 1582 microversion: 2.1 time: 0.235816 Jul 16 21:50:55.712471 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3081/6157] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:50:55 2022] GET /compute/v2.1/servers/a83a3f66-378b-40a4-baf0-0d67c58f1524 => generated 1582 bytes in 238 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:50:56.715650 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-41586f96-70fb-4fac-8c1a-ae05b0e5bb4b tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:50:56.716266 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-41586f96-70fb-4fac-8c1a-ae05b0e5bb4b tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: 3618ceb8-9d10-4219-8885-d2d5004c71ad] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:50:56.724701 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-41586f96-70fb-4fac-8c1a-ae05b0e5bb4b tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:50:56.729494 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-41586f96-70fb-4fac-8c1a-ae05b0e5bb4b tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:50:56.729494 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-41586f96-70fb-4fac-8c1a-ae05b0e5bb4b tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:50:56.729494 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-41586f96-70fb-4fac-8c1a-ae05b0e5bb4b tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:50:56.730326 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-158e9a88-93df-4e4a-b15a-e1db577735f4 tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:50:56.730958 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-158e9a88-93df-4e4a-b15a-e1db577735f4 tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] [instance: a83a3f66-378b-40a4-baf0-0d67c58f1524] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:50:56.738368 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-158e9a88-93df-4e4a-b15a-e1db577735f4 tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:50:56.738834 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-158e9a88-93df-4e4a-b15a-e1db577735f4 tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:50:56.740834 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-158e9a88-93df-4e4a-b15a-e1db577735f4 tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:50:56.741266 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-158e9a88-93df-4e4a-b15a-e1db577735f4 tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:50:56.944049 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-41586f96-70fb-4fac-8c1a-ae05b0e5bb4b tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:50:56.947291 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-158e9a88-93df-4e4a-b15a-e1db577735f4 tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '001ed11a062a4cb08a52d1294ef5880b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8f296b14a61a4997bf411c9b0fdc39d5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:50:56.954916 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-158e9a88-93df-4e4a-b15a-e1db577735f4 tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] 10.222.0.16 "GET /compute/v2.1/servers/a83a3f66-378b-40a4-baf0-0d67c58f1524" status: 200 len: 1582 microversion: 2.1 time: 0.228456 Jul 16 21:50:56.960704 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-41586f96-70fb-4fac-8c1a-ae05b0e5bb4b tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad" status: 200 len: 1675 microversion: 2.1 time: 0.243551 Jul 16 21:50:56.961330 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3082/6158] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:50:56 2022] GET /compute/v2.1/servers/a83a3f66-378b-40a4-baf0-0d67c58f1524 => generated 1582 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:50:56.961411 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3077/6159] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:50:56 2022] GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad => generated 1675 bytes in 245 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:50:57.957770 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-613a4fef-b617-4e8e-bf85-943b57fd8c4c req-12c117e3-3999-49f6-8e4d-2f7bf7cf6f10 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "a83a3f66-378b-40a4-baf0-0d67c58f1524", "name": "network-vif-deleted", "tag": "3f6f502f-e93a-46fe-b6c5-70ff0daa5af0"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:50:57.968677 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-613a4fef-b617-4e8e-bf85-943b57fd8c4c req-12c117e3-3999-49f6-8e4d-2f7bf7cf6f10 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:50:57.969178 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-613a4fef-b617-4e8e-bf85-943b57fd8c4c req-12c117e3-3999-49f6-8e4d-2f7bf7cf6f10 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:50:57.972742 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-92e3e705-b74c-4e9f-a547-62b419ee1fd8 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:50:57.974472 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-92e3e705-b74c-4e9f-a547-62b419ee1fd8 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: 3618ceb8-9d10-4219-8885-d2d5004c71ad] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:50:57.977412 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_api:SELECT=40 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:50:57.982616 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-92e3e705-b74c-4e9f-a547-62b419ee1fd8 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:50:57.983111 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-92e3e705-b74c-4e9f-a547-62b419ee1fd8 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:50:57.984009 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-92e3e705-b74c-4e9f-a547-62b419ee1fd8 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:50:57.984373 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-92e3e705-b74c-4e9f-a547-62b419ee1fd8 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:50:57.984923 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-613a4fef-b617-4e8e-bf85-943b57fd8c4c req-12c117e3-3999-49f6-8e4d-2f7bf7cf6f10 service nova] Creating event network-vif-deleted:3f6f502f-e93a-46fe-b6c5-70ff0daa5af0 for instance a83a3f66-378b-40a4-baf0-0d67c58f1524 on nch2-master-1 Jul 16 21:50:57.991625 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-613a4fef-b617-4e8e-bf85-943b57fd8c4c req-12c117e3-3999-49f6-8e4d-2f7bf7cf6f10 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.037781 Jul 16 21:50:57.992228 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3083/6160] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:50:57 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 16 21:50:57.998668 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-fd6d13af-a95d-4bb2-9a15-6e1d7d0f4465 tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:50:57.999678 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-fd6d13af-a95d-4bb2-9a15-6e1d7d0f4465 tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] [instance: a83a3f66-378b-40a4-baf0-0d67c58f1524] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:50:58.008179 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fd6d13af-a95d-4bb2-9a15-6e1d7d0f4465 tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:50:58.008863 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fd6d13af-a95d-4bb2-9a15-6e1d7d0f4465 tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:50:58.009837 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fd6d13af-a95d-4bb2-9a15-6e1d7d0f4465 tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:50:58.010854 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fd6d13af-a95d-4bb2-9a15-6e1d7d0f4465 tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:50:58.095840 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-fd6d13af-a95d-4bb2-9a15-6e1d7d0f4465 tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '001ed11a062a4cb08a52d1294ef5880b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8f296b14a61a4997bf411c9b0fdc39d5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:50:58.106256 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-fd6d13af-a95d-4bb2-9a15-6e1d7d0f4465 tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] 10.222.0.16 "GET /compute/v2.1/servers/a83a3f66-378b-40a4-baf0-0d67c58f1524" status: 200 len: 1397 microversion: 2.1 time: 0.112760 Jul 16 21:50:58.106751 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3084/6161] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:50:57 2022] GET /compute/v2.1/servers/a83a3f66-378b-40a4-baf0-0d67c58f1524 => generated 1397 bytes in 114 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:50:58.160399 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-92e3e705-b74c-4e9f-a547-62b419ee1fd8 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:50:58.168815 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-92e3e705-b74c-4e9f-a547-62b419ee1fd8 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad" status: 200 len: 1675 microversion: 2.1 time: 0.199958 Jul 16 21:50:58.169391 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3078/6162] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:50:57 2022] GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad => generated 1675 bytes in 201 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:50:59.123482 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-4ce4c334-9271-4745-a8aa-440943c199a9 tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:50:59.124142 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-4ce4c334-9271-4745-a8aa-440943c199a9 tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] [instance: a83a3f66-378b-40a4-baf0-0d67c58f1524] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:50:59.128166 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_api:SELECT=51,nova_api:UPDATE=1 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:50:59.132012 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4ce4c334-9271-4745-a8aa-440943c199a9 tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:50:59.132542 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4ce4c334-9271-4745-a8aa-440943c199a9 tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:50:59.133341 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4ce4c334-9271-4745-a8aa-440943c199a9 tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:50:59.133799 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4ce4c334-9271-4745-a8aa-440943c199a9 tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:50:59.169198 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4ce4c334-9271-4745-a8aa-440943c199a9 tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:50:59.169669 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4ce4c334-9271-4745-a8aa-440943c199a9 tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:50:59.170165 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.objects.instance [None req-4ce4c334-9271-4745-a8aa-440943c199a9 tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] Lazy-loading 'fault' on Instance uuid a83a3f66-378b-40a4-baf0-0d67c58f1524 {{(pid=138681) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:50:59.185810 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-6223f8cc-9f95-4ee6-a302-e23e7cf8da8c tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:50:59.186427 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-6223f8cc-9f95-4ee6-a302-e23e7cf8da8c tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: 3618ceb8-9d10-4219-8885-d2d5004c71ad] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:50:59.194787 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6223f8cc-9f95-4ee6-a302-e23e7cf8da8c tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:50:59.195298 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6223f8cc-9f95-4ee6-a302-e23e7cf8da8c tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:50:59.196095 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6223f8cc-9f95-4ee6-a302-e23e7cf8da8c tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:50:59.196546 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6223f8cc-9f95-4ee6-a302-e23e7cf8da8c tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:50:59.215888 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-4ce4c334-9271-4745-a8aa-440943c199a9 tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '001ed11a062a4cb08a52d1294ef5880b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8f296b14a61a4997bf411c9b0fdc39d5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:50:59.236650 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-4ce4c334-9271-4745-a8aa-440943c199a9 tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] 10.222.0.16 "GET /compute/v2.1/servers/a83a3f66-378b-40a4-baf0-0d67c58f1524" status: 200 len: 1234 microversion: 2.1 time: 0.116862 Jul 16 21:50:59.238270 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3085/6163] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:50:59 2022] GET /compute/v2.1/servers/a83a3f66-378b-40a4-baf0-0d67c58f1524 => generated 1234 bytes in 119 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:50:59.366704 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-6223f8cc-9f95-4ee6-a302-e23e7cf8da8c tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:50:59.374745 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-6223f8cc-9f95-4ee6-a302-e23e7cf8da8c tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad" status: 200 len: 1675 microversion: 2.1 time: 0.192942 Jul 16 21:50:59.375356 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3079/6164] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:50:59 2022] GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad => generated 1675 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:51:00.265298 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-8030cba7-773a-4c52-a5a2-58ef9ffd8fdb tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:51:00.265928 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-8030cba7-773a-4c52-a5a2-58ef9ffd8fdb tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] [instance: a83a3f66-378b-40a4-baf0-0d67c58f1524] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:51:00.274682 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8030cba7-773a-4c52-a5a2-58ef9ffd8fdb tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:51:00.276971 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8030cba7-773a-4c52-a5a2-58ef9ffd8fdb tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:51:00.276971 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8030cba7-773a-4c52-a5a2-58ef9ffd8fdb tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:51:00.276971 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8030cba7-773a-4c52-a5a2-58ef9ffd8fdb tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:51:00.290845 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-8030cba7-773a-4c52-a5a2-58ef9ffd8fdb tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] HTTP exception thrown: Instance a83a3f66-378b-40a4-baf0-0d67c58f1524 could not be found. Jul 16 21:51:00.292537 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-8030cba7-773a-4c52-a5a2-58ef9ffd8fdb tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] Request method failure captured: Jul 16 21:51:00.292537 ncdv-master-1 devstack@n-api.service[138681]: request: GET /compute/v2.1/servers/a83a3f66-378b-40a4-baf0-0d67c58f1524 HTTP/1.1 Jul 16 21:51:00.292537 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:51:00.292537 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:51:00.292537 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:51:00.292537 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 21:51:00.292537 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:51:00.292537 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:51:00.292537 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:51:00.292537 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:51:00.292537 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0zEnwsrc4eMwI06aRcdfbNi7XtuEyxmiSdd7vPlwPgstA4cxOnif9F5NDwuawimU6UOOuiYohSidLPjModel97LgL0BsHRLS9kXB85hD6qyLTAu_kc9Urp6ZPTmvNiSl6N7xWgFUmQuP6degp8Z9_n76uAbFs3YKwe3-7AUVoLBOTbI Jul 16 21:51:00.292537 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:51:00.292537 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:51:00.292537 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:51:00.292537 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:51:00.292537 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:51:00.292537 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:51:00.292537 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: 8f296b14a61a4997bf411c9b0fdc39d5 Jul 16 21:51:00.292537 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-TestVolumeBackupRestore-577066854 Jul 16 21:51:00.292537 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:51:00.292537 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:51:00.292537 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/8f296b14a61a4997bf411c9b0fdc39d5"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/8f296b14a61a4997bf411c9b0fdc39d5"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_8f296b14a61a4997bf411c9b0fdc39d5"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/8f296b14a61a4997bf411c9b0fdc39d5"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:51:00.292537 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-TestVolumeBackupRestore-577066854 Jul 16 21:51:00.292537 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: 8f296b14a61a4997bf411c9b0fdc39d5 Jul 16 21:51:00.292537 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-TestVolumeBackupRestore-577066854 Jul 16 21:51:00.292537 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-TestVolumeBackupRestore-577066854-project Jul 16 21:51:00.292537 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:51:00.292537 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:51:00.292537 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: 001ed11a062a4cb08a52d1294ef5880b Jul 16 21:51:00.292537 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-TestVolumeBackupRestore-577066854-project Jul 16 21:51:00.292537 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:51:00.292537 ncdv-master-1 devstack@n-api.service[138681]: exception: Instance a83a3f66-378b-40a4-baf0-0d67c58f1524 could not be found. Jul 16 21:51:00.292537 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': 'a83a3f66-378b-40a4-baf0-0d67c58f1524'} Jul 16 21:51:00.292537 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:51:00.292537 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:51:00.292537 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:51:00.292537 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:51:00.292537 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:51:00.292537 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:51:00.292537 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:51:00.292537 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:51:00.292537 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:51:00.292537 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:51:00.292537 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance a83a3f66-378b-40a4-baf0-0d67c58f1524 could not be found. Jul 16 21:51:00.292537 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:51:00.292537 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:51:00.292537 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:51:00.292537 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:51:00.292537 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:51:00.292537 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:51:00.292537 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:51:00.292537 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:51:00.292537 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:51:00.292537 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:51:00.292537 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:51:00.292537 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:51:00.292537 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:51:00.292537 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:51:00.292537 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:51:00.292537 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:51:00.292537 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance a83a3f66-378b-40a4-baf0-0d67c58f1524 could not be found. Jul 16 21:51:00.292537 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:51:00.292537 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:51:00.292537 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:51:00.296006 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:51:00.296006 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:51:00.296006 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:51:00.296006 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:51:00.296006 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:51:00.296006 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance a83a3f66-378b-40a4-baf0-0d67c58f1524 could not be found. Jul 16 21:51:00.296006 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:51:00.296006 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-8030cba7-773a-4c52-a5a2-58ef9ffd8fdb tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] Returning 404 to user: Instance a83a3f66-378b-40a4-baf0-0d67c58f1524 could not be found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:51:00.296006 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-8030cba7-773a-4c52-a5a2-58ef9ffd8fdb tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] 10.222.0.16 "GET /compute/v2.1/servers/a83a3f66-378b-40a4-baf0-0d67c58f1524" status: 404 len: 111 microversion: 2.1 time: 0.032867 Jul 16 21:51:00.296006 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3086/6165] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:51:00 2022] GET /compute/v2.1/servers/a83a3f66-378b-40a4-baf0-0d67c58f1524 => generated 111 bytes in 34 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:51:00.394146 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-ee53d358-0539-4b7f-adf7-7f7264d46eda tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:51:00.398496 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-ee53d358-0539-4b7f-adf7-7f7264d46eda tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: 3618ceb8-9d10-4219-8885-d2d5004c71ad] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:51:00.408661 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ee53d358-0539-4b7f-adf7-7f7264d46eda tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:51:00.409343 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ee53d358-0539-4b7f-adf7-7f7264d46eda tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:51:00.410140 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ee53d358-0539-4b7f-adf7-7f7264d46eda tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:51:00.410587 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ee53d358-0539-4b7f-adf7-7f7264d46eda tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:51:00.560117 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-ee53d358-0539-4b7f-adf7-7f7264d46eda tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:51:00.568248 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-ee53d358-0539-4b7f-adf7-7f7264d46eda tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad" status: 200 len: 1675 microversion: 2.1 time: 0.180278 Jul 16 21:51:00.568840 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3080/6166] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:51:00 2022] GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad => generated 1675 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:51:00.647983 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-cc60d575-0462-4d81-b589-80ed2fdff050 tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] Calling method '.version_select at 0x7fdd9d233af0>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:51:00.804205 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-cc60d575-0462-4d81-b589-80ed2fdff050 tempest-TestVolumeBackupRestore-577066854 tempest-TestVolumeBackupRestore-577066854-project] 10.222.0.16 "DELETE /compute/v2.1/os-keypairs/tempest-TestVolumeBackupRestore-644300084" status: 202 len: 0 microversion: 2.1 time: 0.161131 Jul 16 21:51:00.804906 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3087/6167] 10.222.0.16 () {58 vars in 1265 bytes} [Sat Jul 16 21:51:00 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestVolumeBackupRestore-644300084 => generated 0 bytes in 163 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 21:51:01.585768 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-89275f83-abbe-4fed-8aa1-06b47612d9d4 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:51:01.586528 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-89275f83-abbe-4fed-8aa1-06b47612d9d4 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: 3618ceb8-9d10-4219-8885-d2d5004c71ad] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:51:01.595034 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-89275f83-abbe-4fed-8aa1-06b47612d9d4 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:51:01.595515 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-89275f83-abbe-4fed-8aa1-06b47612d9d4 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:51:01.596282 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-89275f83-abbe-4fed-8aa1-06b47612d9d4 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:51:01.596818 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-89275f83-abbe-4fed-8aa1-06b47612d9d4 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:51:01.735547 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-89275f83-abbe-4fed-8aa1-06b47612d9d4 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:51:01.744562 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-89275f83-abbe-4fed-8aa1-06b47612d9d4 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad" status: 200 len: 1675 microversion: 2.1 time: 0.163652 Jul 16 21:51:01.745723 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3081/6168] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:51:01 2022] GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad => generated 1675 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:51:02.762973 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-b680be31-304f-45de-8ff8-3c2c50712ccd tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:51:02.763567 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-b680be31-304f-45de-8ff8-3c2c50712ccd tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: 3618ceb8-9d10-4219-8885-d2d5004c71ad] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:51:02.770685 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b680be31-304f-45de-8ff8-3c2c50712ccd tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:51:02.771189 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b680be31-304f-45de-8ff8-3c2c50712ccd tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:51:02.771971 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b680be31-304f-45de-8ff8-3c2c50712ccd tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:51:02.772427 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b680be31-304f-45de-8ff8-3c2c50712ccd tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:51:02.906657 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-b680be31-304f-45de-8ff8-3c2c50712ccd tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:51:02.915441 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-b680be31-304f-45de-8ff8-3c2c50712ccd tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad" status: 200 len: 1675 microversion: 2.1 time: 0.157458 Jul 16 21:51:02.916186 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3088/6169] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:51:02 2022] GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad => generated 1675 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:51:03.933322 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-268ae143-260e-45d0-8f99-67c37ffd65cf tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:51:03.933943 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-268ae143-260e-45d0-8f99-67c37ffd65cf tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: 3618ceb8-9d10-4219-8885-d2d5004c71ad] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:51:03.941087 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-268ae143-260e-45d0-8f99-67c37ffd65cf tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:51:03.941546 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-268ae143-260e-45d0-8f99-67c37ffd65cf tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:51:03.942946 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-268ae143-260e-45d0-8f99-67c37ffd65cf tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:51:03.943413 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-268ae143-260e-45d0-8f99-67c37ffd65cf tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:51:04.075887 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-268ae143-260e-45d0-8f99-67c37ffd65cf tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:51:04.083732 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-268ae143-260e-45d0-8f99-67c37ffd65cf tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad" status: 200 len: 1652 microversion: 2.1 time: 0.154320 Jul 16 21:51:04.084310 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3082/6170] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:51:03 2022] GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad => generated 1652 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:51:04.101416 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-7de20a5d-98a1-48a1-9497-2545828630ca tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Action: 'action', calling method: >, body: {"os-start": {}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:51:04.108987 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7de20a5d-98a1-48a1-9497-2545828630ca tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:51:04.109459 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7de20a5d-98a1-48a1-9497-2545828630ca tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:51:04.132992 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-7de20a5d-98a1-48a1-9497-2545828630ca tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: 3618ceb8-9d10-4219-8885-d2d5004c71ad] Going to try to start instance {{(pid=138681) start /opt/stack/nova/nova/compute/api.py:2730}} Jul 16 21:51:04.348671 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-7de20a5d-98a1-48a1-9497-2545828630ca tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "POST /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad/action" status: 202 len: 0 microversion: 2.1 time: 0.251312 Jul 16 21:51:04.349206 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3089/6171] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 21:51:04 2022] POST /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad/action => generated 0 bytes in 253 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Jul 16 21:51:04.357846 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-f5567620-f8fa-495c-b7ec-8eb5147493c6 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:51:04.359543 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-f5567620-f8fa-495c-b7ec-8eb5147493c6 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: 3618ceb8-9d10-4219-8885-d2d5004c71ad] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:51:04.364808 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f5567620-f8fa-495c-b7ec-8eb5147493c6 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:51:04.365291 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f5567620-f8fa-495c-b7ec-8eb5147493c6 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:51:04.366073 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f5567620-f8fa-495c-b7ec-8eb5147493c6 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:51:04.366525 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f5567620-f8fa-495c-b7ec-8eb5147493c6 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:51:04.506428 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-f5567620-f8fa-495c-b7ec-8eb5147493c6 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:51:04.514501 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-f5567620-f8fa-495c-b7ec-8eb5147493c6 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad" status: 200 len: 1661 microversion: 2.1 time: 0.160692 Jul 16 21:51:04.515397 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3083/6172] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:51:04 2022] GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad => generated 1661 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:51:05.533632 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-eed86d87-855b-4757-a1bb-a49ab2474a1f tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:51:05.534589 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-eed86d87-855b-4757-a1bb-a49ab2474a1f tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: 3618ceb8-9d10-4219-8885-d2d5004c71ad] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:51:05.541212 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-eed86d87-855b-4757-a1bb-a49ab2474a1f tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:51:05.541713 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-eed86d87-855b-4757-a1bb-a49ab2474a1f tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:51:05.542522 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-eed86d87-855b-4757-a1bb-a49ab2474a1f tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:51:05.542979 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-eed86d87-855b-4757-a1bb-a49ab2474a1f tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:51:05.680909 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-eed86d87-855b-4757-a1bb-a49ab2474a1f tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:51:05.691125 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-eed86d87-855b-4757-a1bb-a49ab2474a1f tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad" status: 200 len: 1661 microversion: 2.1 time: 0.162246 Jul 16 21:51:05.700561 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3090/6173] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:51:05 2022] GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad => generated 1661 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:51:06.708108 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-1d8d209b-f822-4890-a66a-3fcc3e70803b tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:51:06.708762 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-1d8d209b-f822-4890-a66a-3fcc3e70803b tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: 3618ceb8-9d10-4219-8885-d2d5004c71ad] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:51:06.715527 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1d8d209b-f822-4890-a66a-3fcc3e70803b tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:51:06.715980 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1d8d209b-f822-4890-a66a-3fcc3e70803b tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:51:06.716757 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1d8d209b-f822-4890-a66a-3fcc3e70803b tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:51:06.717973 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1d8d209b-f822-4890-a66a-3fcc3e70803b tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:51:07.147833 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-1d8d209b-f822-4890-a66a-3fcc3e70803b tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:51:07.155401 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-1d8d209b-f822-4890-a66a-3fcc3e70803b tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad" status: 200 len: 1665 microversion: 2.1 time: 0.451921 Jul 16 21:51:07.155979 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3084/6174] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:51:06 2022] GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad => generated 1665 bytes in 453 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:51:07.178027 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-e0739deb-96d0-4b85-81d6-d8998de4b806 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:51:07.178655 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-e0739deb-96d0-4b85-81d6-d8998de4b806 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: 3618ceb8-9d10-4219-8885-d2d5004c71ad] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:51:07.185835 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e0739deb-96d0-4b85-81d6-d8998de4b806 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:51:07.186300 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e0739deb-96d0-4b85-81d6-d8998de4b806 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:51:07.187097 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e0739deb-96d0-4b85-81d6-d8998de4b806 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:51:07.187548 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e0739deb-96d0-4b85-81d6-d8998de4b806 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:51:07.320053 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-e0739deb-96d0-4b85-81d6-d8998de4b806 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:51:07.327676 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-e0739deb-96d0-4b85-81d6-d8998de4b806 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad" status: 200 len: 1665 microversion: 2.1 time: 0.153811 Jul 16 21:51:07.328239 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3091/6175] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:51:07 2022] GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad => generated 1665 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:51:07.346267 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-b10e2a13-b444-476a-a817-ccdb4d36d37e tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:51:07.346885 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-b10e2a13-b444-476a-a817-ccdb4d36d37e tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: 3618ceb8-9d10-4219-8885-d2d5004c71ad] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:51:07.354431 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b10e2a13-b444-476a-a817-ccdb4d36d37e tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:51:07.354938 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b10e2a13-b444-476a-a817-ccdb4d36d37e tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:51:07.355763 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b10e2a13-b444-476a-a817-ccdb4d36d37e tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:51:07.356205 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b10e2a13-b444-476a-a817-ccdb4d36d37e tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:51:07.506130 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-b10e2a13-b444-476a-a817-ccdb4d36d37e tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:51:07.518912 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-b10e2a13-b444-476a-a817-ccdb4d36d37e tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad" status: 200 len: 1665 microversion: 2.1 time: 0.176453 Jul 16 21:51:07.519518 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3085/6176] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:51:07 2022] GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad => generated 1665 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:51:07.535398 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-a5d9fd5e-fd4e-4bb7-b1c0-a269fac455a1 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:51:07.536058 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-a5d9fd5e-fd4e-4bb7-b1c0-a269fac455a1 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: 3618ceb8-9d10-4219-8885-d2d5004c71ad] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:51:07.542038 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a5d9fd5e-fd4e-4bb7-b1c0-a269fac455a1 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:51:07.542504 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a5d9fd5e-fd4e-4bb7-b1c0-a269fac455a1 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:51:07.543338 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a5d9fd5e-fd4e-4bb7-b1c0-a269fac455a1 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:51:07.543787 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a5d9fd5e-fd4e-4bb7-b1c0-a269fac455a1 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:51:07.682852 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-a5d9fd5e-fd4e-4bb7-b1c0-a269fac455a1 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:51:07.690630 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-a5d9fd5e-fd4e-4bb7-b1c0-a269fac455a1 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad" status: 200 len: 1665 microversion: 2.1 time: 0.159424 Jul 16 21:51:07.691227 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3092/6177] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:51:07 2022] GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad => generated 1665 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:51:07.707558 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-2455cc30-e219-4cae-b85a-4155a58acd86 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:51:07.708381 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-2455cc30-e219-4cae-b85a-4155a58acd86 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: 3618ceb8-9d10-4219-8885-d2d5004c71ad] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:51:07.715444 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2455cc30-e219-4cae-b85a-4155a58acd86 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:51:07.715898 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2455cc30-e219-4cae-b85a-4155a58acd86 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:51:07.716663 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2455cc30-e219-4cae-b85a-4155a58acd86 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:51:07.717104 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2455cc30-e219-4cae-b85a-4155a58acd86 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:51:07.859074 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-2455cc30-e219-4cae-b85a-4155a58acd86 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:51:07.867657 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-2455cc30-e219-4cae-b85a-4155a58acd86 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad" status: 200 len: 1665 microversion: 2.1 time: 0.164099 Jul 16 21:51:07.868851 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3086/6178] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:51:07 2022] GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad => generated 1665 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:51:07.883865 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-b6103cb0-dc29-46f9-afad-7b072ccbab72 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Action: 'action', calling method: >, body: {"suspend": {}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:51:07.884411 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-b6103cb0-dc29-46f9-afad-7b072ccbab72 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: 3618ceb8-9d10-4219-8885-d2d5004c71ad] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:51:07.890767 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b6103cb0-dc29-46f9-afad-7b072ccbab72 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:51:07.891264 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b6103cb0-dc29-46f9-afad-7b072ccbab72 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:51:07.892030 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b6103cb0-dc29-46f9-afad-7b072ccbab72 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:51:07.892479 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b6103cb0-dc29-46f9-afad-7b072ccbab72 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:51:07.917005 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.objects.instance [None req-b6103cb0-dc29-46f9-afad-7b072ccbab72 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lazy-loading 'flavor' on Instance uuid 3618ceb8-9d10-4219-8885-d2d5004c71ad {{(pid=138681) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:51:08.205771 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-b6103cb0-dc29-46f9-afad-7b072ccbab72 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "POST /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad/action" status: 202 len: 0 microversion: 2.1 time: 0.325726 Jul 16 21:51:08.206779 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3093/6179] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 21:51:07 2022] POST /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad/action => generated 0 bytes in 327 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 21:51:08.224296 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-5b1ae493-c9b3-4c1b-8145-b2f6492c9238 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:51:08.224937 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-5b1ae493-c9b3-4c1b-8145-b2f6492c9238 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: 3618ceb8-9d10-4219-8885-d2d5004c71ad] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:51:08.231465 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5b1ae493-c9b3-4c1b-8145-b2f6492c9238 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:51:08.231928 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5b1ae493-c9b3-4c1b-8145-b2f6492c9238 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:51:08.232737 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5b1ae493-c9b3-4c1b-8145-b2f6492c9238 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:51:08.233187 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5b1ae493-c9b3-4c1b-8145-b2f6492c9238 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:51:08.375040 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-5b1ae493-c9b3-4c1b-8145-b2f6492c9238 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:51:08.383281 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-5b1ae493-c9b3-4c1b-8145-b2f6492c9238 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad" status: 200 len: 1673 microversion: 2.1 time: 0.171571 Jul 16 21:51:08.383868 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3087/6180] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:51:08 2022] GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad => generated 1673 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:51:09.402423 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-084591d5-c133-4848-a9ea-d082846c4bff tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:51:09.403172 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-084591d5-c133-4848-a9ea-d082846c4bff tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: 3618ceb8-9d10-4219-8885-d2d5004c71ad] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:51:09.410411 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-084591d5-c133-4848-a9ea-d082846c4bff tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:51:09.410867 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-084591d5-c133-4848-a9ea-d082846c4bff tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:51:09.411820 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-084591d5-c133-4848-a9ea-d082846c4bff tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:51:09.412283 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-084591d5-c133-4848-a9ea-d082846c4bff tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:51:09.417362 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=93,nova_cell1:UPDATE=4,nova_cell1:INSERT=3 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:51:09.547334 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-084591d5-c133-4848-a9ea-d082846c4bff tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:51:09.555226 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-084591d5-c133-4848-a9ea-d082846c4bff tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad" status: 200 len: 1656 microversion: 2.1 time: 0.157059 Jul 16 21:51:09.555804 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3094/6181] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:51:09 2022] GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad => generated 1656 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:51:09.575031 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-ea6f623f-637b-4027-bb87-f515e6d0499f tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Action: 'action', calling method: >, body: {"resume": {}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:51:09.581605 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-ea6f623f-637b-4027-bb87-f515e6d0499f tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: 3618ceb8-9d10-4219-8885-d2d5004c71ad] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:51:09.583139 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ea6f623f-637b-4027-bb87-f515e6d0499f tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:51:09.584279 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ea6f623f-637b-4027-bb87-f515e6d0499f tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:51:09.585119 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ea6f623f-637b-4027-bb87-f515e6d0499f tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:51:09.585569 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ea6f623f-637b-4027-bb87-f515e6d0499f tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:51:09.623185 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:51:09.811665 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-ea6f623f-637b-4027-bb87-f515e6d0499f tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "POST /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad/action" status: 202 len: 0 microversion: 2.1 time: 0.243668 Jul 16 21:51:09.816289 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3088/6182] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 21:51:09 2022] POST /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad/action => generated 0 bytes in 245 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 21:51:09.820521 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-9d2f397c-b2c6-42ba-b0e1-0628cbcae599 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:51:09.821655 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-9d2f397c-b2c6-42ba-b0e1-0628cbcae599 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: 3618ceb8-9d10-4219-8885-d2d5004c71ad] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:51:09.829077 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9d2f397c-b2c6-42ba-b0e1-0628cbcae599 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:51:09.829548 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9d2f397c-b2c6-42ba-b0e1-0628cbcae599 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:51:09.830341 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9d2f397c-b2c6-42ba-b0e1-0628cbcae599 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:51:09.830800 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9d2f397c-b2c6-42ba-b0e1-0628cbcae599 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:51:09.973070 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-9d2f397c-b2c6-42ba-b0e1-0628cbcae599 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:51:09.983112 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-9d2f397c-b2c6-42ba-b0e1-0628cbcae599 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad" status: 200 len: 1662 microversion: 2.1 time: 0.166511 Jul 16 21:51:09.983721 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3095/6183] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:51:09 2022] GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad => generated 1662 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:51:11.003144 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-3c744e9b-2a43-432f-b284-6e59a8bc785c tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:51:11.004048 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-3c744e9b-2a43-432f-b284-6e59a8bc785c tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: 3618ceb8-9d10-4219-8885-d2d5004c71ad] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:51:11.010995 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3c744e9b-2a43-432f-b284-6e59a8bc785c tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:51:11.011499 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3c744e9b-2a43-432f-b284-6e59a8bc785c tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:51:11.012291 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3c744e9b-2a43-432f-b284-6e59a8bc785c tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:51:11.012741 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3c744e9b-2a43-432f-b284-6e59a8bc785c tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:51:11.164484 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-3c744e9b-2a43-432f-b284-6e59a8bc785c tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:51:11.172465 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-3c744e9b-2a43-432f-b284-6e59a8bc785c tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad" status: 200 len: 1662 microversion: 2.1 time: 0.174978 Jul 16 21:51:11.173234 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3089/6184] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:51:10 2022] GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad => generated 1662 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:51:12.190735 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-dd642ac8-42f5-4955-8942-75f0f59ffba0 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:51:12.191399 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-dd642ac8-42f5-4955-8942-75f0f59ffba0 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: 3618ceb8-9d10-4219-8885-d2d5004c71ad] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:51:12.198627 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-dd642ac8-42f5-4955-8942-75f0f59ffba0 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:51:12.199140 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-dd642ac8-42f5-4955-8942-75f0f59ffba0 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:51:12.199933 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-dd642ac8-42f5-4955-8942-75f0f59ffba0 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:51:12.200392 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-dd642ac8-42f5-4955-8942-75f0f59ffba0 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:51:12.339424 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-dd642ac8-42f5-4955-8942-75f0f59ffba0 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:51:12.347110 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-dd642ac8-42f5-4955-8942-75f0f59ffba0 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad" status: 200 len: 1665 microversion: 2.1 time: 0.160912 Jul 16 21:51:12.347711 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3096/6185] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:51:12 2022] GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad => generated 1665 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:51:12.366563 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-cbf89188-eefc-4ad7-a949-b26e2e2fd3c9 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:51:12.367251 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-cbf89188-eefc-4ad7-a949-b26e2e2fd3c9 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: 3618ceb8-9d10-4219-8885-d2d5004c71ad] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:51:12.374254 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cbf89188-eefc-4ad7-a949-b26e2e2fd3c9 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:51:12.374711 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cbf89188-eefc-4ad7-a949-b26e2e2fd3c9 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:51:12.375503 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cbf89188-eefc-4ad7-a949-b26e2e2fd3c9 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:51:12.375950 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cbf89188-eefc-4ad7-a949-b26e2e2fd3c9 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:51:12.511319 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-cbf89188-eefc-4ad7-a949-b26e2e2fd3c9 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:51:12.519169 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-cbf89188-eefc-4ad7-a949-b26e2e2fd3c9 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad" status: 200 len: 1665 microversion: 2.1 time: 0.156795 Jul 16 21:51:12.519792 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3090/6186] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:51:12 2022] GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad => generated 1665 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:51:12.537141 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-6d783481-45de-4ef3-bde6-a418038d960b tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:51:12.537821 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-6d783481-45de-4ef3-bde6-a418038d960b tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: a2bd21dc-856d-41b0-8d5c-139452473742] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:51:12.543994 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6d783481-45de-4ef3-bde6-a418038d960b tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:51:12.551628 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6d783481-45de-4ef3-bde6-a418038d960b tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:51:12.551628 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6d783481-45de-4ef3-bde6-a418038d960b tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:51:12.551628 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6d783481-45de-4ef3-bde6-a418038d960b tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:51:12.572538 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-6d783481-45de-4ef3-bde6-a418038d960b tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: a2bd21dc-856d-41b0-8d5c-139452473742] Going to try to terminate instance {{(pid=138681) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 16 21:51:12.578685 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.compute.api [None req-6d783481-45de-4ef3-bde6-a418038d960b tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: a2bd21dc-856d-41b0-8d5c-139452473742] Working on deleting snapshot 7f537f04-0128-45bb-8de1-d8272d1472d4 from shelved instance... Jul 16 21:51:14.028985 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-6d783481-45de-4ef3-bde6-a418038d960b tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: a2bd21dc-856d-41b0-8d5c-139452473742] Doing local delete in cell 327e8df1-39cc-4b36-9ea6-6a9b88ed54fc(cell1) {{(pid=138681) _delete /opt/stack/nova/nova/compute/api.py:2489}} Jul 16 21:51:14.029859 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6d783481-45de-4ef3-bde6-a418038d960b tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:51:14.030322 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6d783481-45de-4ef3-bde6-a418038d960b tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:51:14.030763 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.compute.api [None req-6d783481-45de-4ef3-bde6-a418038d960b tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: a2bd21dc-856d-41b0-8d5c-139452473742] instance is in SHELVED_OFFLOADED state, cleanup the instance's info from database. Jul 16 21:51:14.034679 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.network.neutron [None req-6d783481-45de-4ef3-bde6-a418038d960b tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: a2bd21dc-856d-41b0-8d5c-139452473742] deallocate_for_instance() {{(pid=138681) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1783}} Jul 16 21:51:14.722541 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-1a0853a3-f96f-4871-8003-3f23b0021eff req-50daa2f0-dffa-4904-9988-6de965681149 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "a2bd21dc-856d-41b0-8d5c-139452473742", "name": "network-vif-deleted", "tag": "85e05f99-40c7-4abf-a10c-a4f578c0caf4"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:51:14.734643 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-1a0853a3-f96f-4871-8003-3f23b0021eff req-50daa2f0-dffa-4904-9988-6de965681149 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:51:14.735431 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-1a0853a3-f96f-4871-8003-3f23b0021eff req-50daa2f0-dffa-4904-9988-6de965681149 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:51:14.748994 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.compute.server_external_events [req-1a0853a3-f96f-4871-8003-3f23b0021eff req-50daa2f0-dffa-4904-9988-6de965681149 service nova] Unable to find a host for instance a2bd21dc-856d-41b0-8d5c-139452473742. Dropping event network-vif-deleted {{(pid=138680) create /opt/stack/nova/nova/api/openstack/compute/server_external_events.py:143}} Jul 16 21:51:14.750024 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-1a0853a3-f96f-4871-8003-3f23b0021eff req-50daa2f0-dffa-4904-9988-6de965681149 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 207 len: 180 microversion: 2.1 time: 0.031673 Jul 16 21:51:14.753539 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3091/6187] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:51:14 2022] POST /compute/v2.1/os-server-external-events => generated 180 bytes in 33 msecs (HTTP/1.1 207) 9 headers in 367 bytes (2 switches on core 0) Jul 16 21:51:14.772852 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.network.neutron [None req-6d783481-45de-4ef3-bde6-a418038d960b tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: a2bd21dc-856d-41b0-8d5c-139452473742] Updating instance_info_cache with network_info: [] {{(pid=138681) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:117}} Jul 16 21:51:15.445437 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-6d783481-45de-4ef3-bde6-a418038d960b tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "DELETE /compute/v2.1/servers/a2bd21dc-856d-41b0-8d5c-139452473742" status: 204 len: 0 microversion: 2.1 time: 2.912059 Jul 16 21:51:15.446092 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3097/6188] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:51:12 2022] DELETE /compute/v2.1/servers/a2bd21dc-856d-41b0-8d5c-139452473742 => generated 0 bytes in 2913 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 16 21:51:15.454133 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-d1330dc2-4856-4e52-b8ad-765db3a0de2a tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:51:15.454759 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-d1330dc2-4856-4e52-b8ad-765db3a0de2a tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: a2bd21dc-856d-41b0-8d5c-139452473742] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:51:15.460967 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d1330dc2-4856-4e52-b8ad-765db3a0de2a tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:51:15.461613 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d1330dc2-4856-4e52-b8ad-765db3a0de2a tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:51:15.462427 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d1330dc2-4856-4e52-b8ad-765db3a0de2a tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:51:15.462885 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d1330dc2-4856-4e52-b8ad-765db3a0de2a tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:51:15.477762 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-d1330dc2-4856-4e52-b8ad-765db3a0de2a tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] HTTP exception thrown: Instance a2bd21dc-856d-41b0-8d5c-139452473742 could not be found. Jul 16 21:51:15.483087 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-d1330dc2-4856-4e52-b8ad-765db3a0de2a tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Request method failure captured: Jul 16 21:51:15.483087 ncdv-master-1 devstack@n-api.service[138680]: request: GET /compute/v2.1/servers/a2bd21dc-856d-41b0-8d5c-139452473742 HTTP/1.1 Jul 16 21:51:15.483087 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:51:15.483087 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:51:15.483087 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:51:15.483087 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 0 Jul 16 21:51:15.483087 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:51:15.483087 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:51:15.483087 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:51:15.483087 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:51:15.483087 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0zFqdHrIN0eOi2qg4drPydwu6SP4etFLyy6aDaqxzq0WQeMBWfEvVpgvWD4EE3q6tgQ3bkIiQt72uW6dWCfa-p4Yuvsc21rwij0v1H7vvz1_kRSKLl1Dn4VMOw8LrCFQcDku8xWPGzPME_8kBIMghBmgh0Bs2vThYjVU90trc_bK9dc Jul 16 21:51:15.483087 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:51:15.483087 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:51:15.483087 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:51:15.483087 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:51:15.483087 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:51:15.483087 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:51:15.483087 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: 1e7cd69b8ddd4aceb45a4aaacdfccea7 Jul 16 21:51:15.483087 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-ServerActionsTestJSON-1394927270 Jul 16 21:51:15.483087 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:51:15.483087 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:51:15.483087 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/1e7cd69b8ddd4aceb45a4aaacdfccea7"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/1e7cd69b8ddd4aceb45a4aaacdfccea7"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_1e7cd69b8ddd4aceb45a4aaacdfccea7"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/1e7cd69b8ddd4aceb45a4aaacdfccea7"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:51:15.483087 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-ServerActionsTestJSON-1394927270 Jul 16 21:51:15.483087 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: 1e7cd69b8ddd4aceb45a4aaacdfccea7 Jul 16 21:51:15.483087 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-ServerActionsTestJSON-1394927270 Jul 16 21:51:15.483087 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-ServerActionsTestJSON-1394927270-project Jul 16 21:51:15.483087 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:51:15.483087 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:51:15.483087 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: cfe293b9ed3645229bed5ef8ce17cfec Jul 16 21:51:15.483087 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-ServerActionsTestJSON-1394927270-project Jul 16 21:51:15.483087 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 21:51:15.483087 ncdv-master-1 devstack@n-api.service[138680]: exception: Instance a2bd21dc-856d-41b0-8d5c-139452473742 could not be found. Jul 16 21:51:15.483087 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': 'a2bd21dc-856d-41b0-8d5c-139452473742'} Jul 16 21:51:15.483087 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:51:15.483087 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:51:15.483087 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:51:15.483087 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:51:15.483087 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:51:15.483087 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:51:15.483087 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:51:15.483087 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:51:15.483087 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:51:15.483087 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:51:15.483087 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance a2bd21dc-856d-41b0-8d5c-139452473742 could not be found. Jul 16 21:51:15.483087 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:51:15.483087 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:51:15.483087 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:51:15.483087 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:51:15.483087 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:51:15.483087 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:51:15.483087 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:51:15.483087 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:51:15.483087 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:51:15.483087 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:51:15.483087 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:51:15.483087 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:51:15.483087 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:51:15.483087 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:51:15.483087 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:51:15.483087 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:51:15.483087 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance a2bd21dc-856d-41b0-8d5c-139452473742 could not be found. Jul 16 21:51:15.483087 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:51:15.483087 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:51:15.483087 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:51:15.487284 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:51:15.487284 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:51:15.487284 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:51:15.487284 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:51:15.487284 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:51:15.487284 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance a2bd21dc-856d-41b0-8d5c-139452473742 could not be found. Jul 16 21:51:15.487284 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:51:15.487284 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-d1330dc2-4856-4e52-b8ad-765db3a0de2a tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Returning 404 to user: Instance a2bd21dc-856d-41b0-8d5c-139452473742 could not be found. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:51:15.487284 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-d1330dc2-4856-4e52-b8ad-765db3a0de2a tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/a2bd21dc-856d-41b0-8d5c-139452473742" status: 404 len: 111 microversion: 2.1 time: 0.030942 Jul 16 21:51:15.487284 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3092/6189] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:51:15 2022] GET /compute/v2.1/servers/a2bd21dc-856d-41b0-8d5c-139452473742 => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:51:15.490657 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-11aa1ca2-5b72-4893-b8a1-8dafbc2ac20d tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:51:15.491233 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-11aa1ca2-5b72-4893-b8a1-8dafbc2ac20d tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: 5d7c5a9b-cfb1-49ce-be31-b19603177834] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:51:15.498170 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-11aa1ca2-5b72-4893-b8a1-8dafbc2ac20d tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:51:15.498640 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-11aa1ca2-5b72-4893-b8a1-8dafbc2ac20d tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:51:15.499472 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-11aa1ca2-5b72-4893-b8a1-8dafbc2ac20d tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:51:15.499932 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-11aa1ca2-5b72-4893-b8a1-8dafbc2ac20d tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:51:15.525110 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-11aa1ca2-5b72-4893-b8a1-8dafbc2ac20d tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: 5d7c5a9b-cfb1-49ce-be31-b19603177834] Going to try to terminate instance {{(pid=138681) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 16 21:51:15.878033 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-11aa1ca2-5b72-4893-b8a1-8dafbc2ac20d tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "DELETE /compute/v2.1/servers/5d7c5a9b-cfb1-49ce-be31-b19603177834" status: 204 len: 0 microversion: 2.1 time: 0.391092 Jul 16 21:51:15.882751 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3098/6190] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:51:15 2022] DELETE /compute/v2.1/servers/5d7c5a9b-cfb1-49ce-be31-b19603177834 => generated 0 bytes in 393 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 16 21:51:15.888718 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-a527f62e-8fa6-4844-817b-dbb2518fde06 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:51:15.889494 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-a527f62e-8fa6-4844-817b-dbb2518fde06 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: 5d7c5a9b-cfb1-49ce-be31-b19603177834] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:51:15.897557 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a527f62e-8fa6-4844-817b-dbb2518fde06 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:51:15.898333 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a527f62e-8fa6-4844-817b-dbb2518fde06 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:51:15.899229 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a527f62e-8fa6-4844-817b-dbb2518fde06 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:51:15.899739 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a527f62e-8fa6-4844-817b-dbb2518fde06 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:51:16.032972 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-a527f62e-8fa6-4844-817b-dbb2518fde06 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:51:16.035293 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-a527f62e-8fa6-4844-817b-dbb2518fde06 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/5d7c5a9b-cfb1-49ce-be31-b19603177834" status: 200 len: 1466 microversion: 2.1 time: 0.150555 Jul 16 21:51:16.036181 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3093/6191] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:51:15 2022] GET /compute/v2.1/servers/5d7c5a9b-cfb1-49ce-be31-b19603177834 => generated 1466 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:51:17.054415 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-7dd7c4f9-2cef-43fd-b562-07139af97a25 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:51:17.055475 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-7dd7c4f9-2cef-43fd-b562-07139af97a25 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: 5d7c5a9b-cfb1-49ce-be31-b19603177834] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:51:17.063446 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7dd7c4f9-2cef-43fd-b562-07139af97a25 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:51:17.064063 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7dd7c4f9-2cef-43fd-b562-07139af97a25 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:51:17.064968 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7dd7c4f9-2cef-43fd-b562-07139af97a25 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:51:17.065529 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7dd7c4f9-2cef-43fd-b562-07139af97a25 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:51:17.165737 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-7dd7c4f9-2cef-43fd-b562-07139af97a25 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:51:17.173823 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-7dd7c4f9-2cef-43fd-b562-07139af97a25 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/5d7c5a9b-cfb1-49ce-be31-b19603177834" status: 200 len: 1466 microversion: 2.1 time: 0.124250 Jul 16 21:51:17.175121 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3099/6192] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:51:17 2022] GET /compute/v2.1/servers/5d7c5a9b-cfb1-49ce-be31-b19603177834 => generated 1466 bytes in 126 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:51:18.026075 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-78984cbe-45d6-4e8f-9315-2b06acec95f0 req-1caccb13-584f-4c06-acb9-46f6a269a168 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "5d7c5a9b-cfb1-49ce-be31-b19603177834", "name": "network-vif-deleted", "tag": "e6e85788-edc2-468b-8bf0-8710fb76b373"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:51:18.041554 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-78984cbe-45d6-4e8f-9315-2b06acec95f0 req-1caccb13-584f-4c06-acb9-46f6a269a168 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:51:18.042160 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-78984cbe-45d6-4e8f-9315-2b06acec95f0 req-1caccb13-584f-4c06-acb9-46f6a269a168 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:51:18.060596 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-78984cbe-45d6-4e8f-9315-2b06acec95f0 req-1caccb13-584f-4c06-acb9-46f6a269a168 service nova] Creating event network-vif-deleted:e6e85788-edc2-468b-8bf0-8710fb76b373 for instance 5d7c5a9b-cfb1-49ce-be31-b19603177834 on nch1-master-1 Jul 16 21:51:18.071373 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-78984cbe-45d6-4e8f-9315-2b06acec95f0 req-1caccb13-584f-4c06-acb9-46f6a269a168 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.050453 Jul 16 21:51:18.074904 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3094/6193] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:51:18 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:51:18.192182 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-3be7a666-b823-4620-afea-93c3078154e9 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:51:18.192831 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-3be7a666-b823-4620-afea-93c3078154e9 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: 5d7c5a9b-cfb1-49ce-be31-b19603177834] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:51:18.200882 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3be7a666-b823-4620-afea-93c3078154e9 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:51:18.201467 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3be7a666-b823-4620-afea-93c3078154e9 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:51:18.202371 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3be7a666-b823-4620-afea-93c3078154e9 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:51:18.202893 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3be7a666-b823-4620-afea-93c3078154e9 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:51:18.277591 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-3be7a666-b823-4620-afea-93c3078154e9 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:51:18.285370 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-3be7a666-b823-4620-afea-93c3078154e9 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/5d7c5a9b-cfb1-49ce-be31-b19603177834" status: 200 len: 1300 microversion: 2.1 time: 0.097169 Jul 16 21:51:18.285922 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3100/6194] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:51:18 2022] GET /compute/v2.1/servers/5d7c5a9b-cfb1-49ce-be31-b19603177834 => generated 1300 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:51:19.302444 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-80eaa88e-3bf6-4724-b088-65a781f1ed30 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:51:19.303188 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-80eaa88e-3bf6-4724-b088-65a781f1ed30 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: 5d7c5a9b-cfb1-49ce-be31-b19603177834] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:51:19.310823 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-80eaa88e-3bf6-4724-b088-65a781f1ed30 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:51:19.311391 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-80eaa88e-3bf6-4724-b088-65a781f1ed30 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:51:19.312225 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-80eaa88e-3bf6-4724-b088-65a781f1ed30 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:51:19.312673 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-80eaa88e-3bf6-4724-b088-65a781f1ed30 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:51:19.331819 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-80eaa88e-3bf6-4724-b088-65a781f1ed30 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] HTTP exception thrown: Instance 5d7c5a9b-cfb1-49ce-be31-b19603177834 could not be found. Jul 16 21:51:19.333514 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-80eaa88e-3bf6-4724-b088-65a781f1ed30 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Request method failure captured: Jul 16 21:51:19.333514 ncdv-master-1 devstack@n-api.service[138680]: request: GET /compute/v2.1/servers/5d7c5a9b-cfb1-49ce-be31-b19603177834 HTTP/1.1 Jul 16 21:51:19.333514 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:51:19.333514 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:51:19.333514 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:51:19.333514 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 0 Jul 16 21:51:19.333514 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:51:19.333514 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:51:19.333514 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:51:19.333514 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:51:19.333514 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0zFqdHrIN0eOi2qg4drPydwu6SP4etFLyy6aDaqxzq0WQeMBWfEvVpgvWD4EE3q6tgQ3bkIiQt72uW6dWCfa-p4Yuvsc21rwij0v1H7vvz1_kRSKLl1Dn4VMOw8LrCFQcDku8xWPGzPME_8kBIMghBmgh0Bs2vThYjVU90trc_bK9dc Jul 16 21:51:19.333514 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:51:19.333514 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:51:19.333514 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:51:19.333514 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:51:19.333514 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:51:19.333514 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:51:19.333514 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: 1e7cd69b8ddd4aceb45a4aaacdfccea7 Jul 16 21:51:19.333514 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-ServerActionsTestJSON-1394927270 Jul 16 21:51:19.333514 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:51:19.333514 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:51:19.333514 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/1e7cd69b8ddd4aceb45a4aaacdfccea7"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/1e7cd69b8ddd4aceb45a4aaacdfccea7"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_1e7cd69b8ddd4aceb45a4aaacdfccea7"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/1e7cd69b8ddd4aceb45a4aaacdfccea7"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:51:19.333514 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-ServerActionsTestJSON-1394927270 Jul 16 21:51:19.333514 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: 1e7cd69b8ddd4aceb45a4aaacdfccea7 Jul 16 21:51:19.333514 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-ServerActionsTestJSON-1394927270 Jul 16 21:51:19.333514 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-ServerActionsTestJSON-1394927270-project Jul 16 21:51:19.333514 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:51:19.333514 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:51:19.333514 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: cfe293b9ed3645229bed5ef8ce17cfec Jul 16 21:51:19.333514 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-ServerActionsTestJSON-1394927270-project Jul 16 21:51:19.333514 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 21:51:19.333514 ncdv-master-1 devstack@n-api.service[138680]: exception: Instance 5d7c5a9b-cfb1-49ce-be31-b19603177834 could not be found. Jul 16 21:51:19.333514 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': '5d7c5a9b-cfb1-49ce-be31-b19603177834'} Jul 16 21:51:19.333514 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:51:19.333514 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:51:19.333514 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:51:19.333514 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:51:19.333514 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:51:19.333514 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:51:19.333514 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:51:19.333514 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:51:19.333514 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:51:19.333514 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:51:19.333514 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 5d7c5a9b-cfb1-49ce-be31-b19603177834 could not be found. Jul 16 21:51:19.333514 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:51:19.333514 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:51:19.333514 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:51:19.333514 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:51:19.333514 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:51:19.333514 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:51:19.333514 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:51:19.333514 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:51:19.333514 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:51:19.333514 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:51:19.333514 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:51:19.333514 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:51:19.333514 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:51:19.333514 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:51:19.333514 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:51:19.333514 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:51:19.333514 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 5d7c5a9b-cfb1-49ce-be31-b19603177834 could not be found. Jul 16 21:51:19.333514 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:51:19.333514 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:51:19.333514 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:51:19.340858 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:51:19.340858 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:51:19.340858 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:51:19.340858 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:51:19.340858 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:51:19.340858 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 5d7c5a9b-cfb1-49ce-be31-b19603177834 could not be found. Jul 16 21:51:19.340858 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:51:19.340858 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-80eaa88e-3bf6-4724-b088-65a781f1ed30 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Returning 404 to user: Instance 5d7c5a9b-cfb1-49ce-be31-b19603177834 could not be found. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:51:19.340858 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-80eaa88e-3bf6-4724-b088-65a781f1ed30 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/5d7c5a9b-cfb1-49ce-be31-b19603177834" status: 404 len: 111 microversion: 2.1 time: 0.036828 Jul 16 21:51:19.340858 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3095/6195] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:51:19 2022] GET /compute/v2.1/servers/5d7c5a9b-cfb1-49ce-be31-b19603177834 => generated 111 bytes in 39 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:51:19.344486 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-5c38f839-7389-4751-b375-d0c74f1755d9 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:51:19.345008 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-5c38f839-7389-4751-b375-d0c74f1755d9 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: bbfa7c88-b995-4c25-b981-1ada326424ed] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:51:19.352112 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5c38f839-7389-4751-b375-d0c74f1755d9 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:51:19.352587 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5c38f839-7389-4751-b375-d0c74f1755d9 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:51:19.353378 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5c38f839-7389-4751-b375-d0c74f1755d9 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:51:19.353835 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5c38f839-7389-4751-b375-d0c74f1755d9 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:51:19.384667 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-5c38f839-7389-4751-b375-d0c74f1755d9 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: bbfa7c88-b995-4c25-b981-1ada326424ed] Going to try to terminate instance {{(pid=138681) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 16 21:51:19.421002 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=77,nova_cell1:UPDATE=18,nova_cell1:SAVEPOINT=1,nova_cell1:RELEASE=1,nova_cell1:DELETE=2,nova_cell1:INSERT=1 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:51:19.771719 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-5c38f839-7389-4751-b375-d0c74f1755d9 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "DELETE /compute/v2.1/servers/bbfa7c88-b995-4c25-b981-1ada326424ed" status: 204 len: 0 microversion: 2.1 time: 0.430797 Jul 16 21:51:19.774231 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3101/6196] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:51:19 2022] DELETE /compute/v2.1/servers/bbfa7c88-b995-4c25-b981-1ada326424ed => generated 0 bytes in 434 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 16 21:51:19.782439 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-14f958c8-aa28-4576-bae0-8ce01549b81c tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:51:19.782981 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-14f958c8-aa28-4576-bae0-8ce01549b81c tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: bbfa7c88-b995-4c25-b981-1ada326424ed] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:51:19.790354 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-14f958c8-aa28-4576-bae0-8ce01549b81c tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:51:19.790817 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-14f958c8-aa28-4576-bae0-8ce01549b81c tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:51:19.791662 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-14f958c8-aa28-4576-bae0-8ce01549b81c tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:51:19.792107 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-14f958c8-aa28-4576-bae0-8ce01549b81c tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:51:19.934639 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-14f958c8-aa28-4576-bae0-8ce01549b81c tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:51:19.942349 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-14f958c8-aa28-4576-bae0-8ce01549b81c tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/bbfa7c88-b995-4c25-b981-1ada326424ed" status: 200 len: 1507 microversion: 2.1 time: 0.163844 Jul 16 21:51:19.942945 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3096/6197] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:51:19 2022] GET /compute/v2.1/servers/bbfa7c88-b995-4c25-b981-1ada326424ed => generated 1507 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:51:20.961608 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-355daf90-2ed1-4313-a490-da571a302fa7 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:51:20.962230 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-355daf90-2ed1-4313-a490-da571a302fa7 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: bbfa7c88-b995-4c25-b981-1ada326424ed] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:51:20.970596 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-355daf90-2ed1-4313-a490-da571a302fa7 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:51:20.971902 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-355daf90-2ed1-4313-a490-da571a302fa7 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:51:20.972694 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-355daf90-2ed1-4313-a490-da571a302fa7 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:51:20.973547 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-355daf90-2ed1-4313-a490-da571a302fa7 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:51:21.575268 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-355daf90-2ed1-4313-a490-da571a302fa7 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:51:21.594596 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-355daf90-2ed1-4313-a490-da571a302fa7 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/bbfa7c88-b995-4c25-b981-1ada326424ed" status: 200 len: 1507 microversion: 2.1 time: 0.637817 Jul 16 21:51:21.595334 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3102/6198] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:51:20 2022] GET /compute/v2.1/servers/bbfa7c88-b995-4c25-b981-1ada326424ed => generated 1507 bytes in 640 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:51:22.313825 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-bd32229a-62f3-4d60-83f5-e82a722f6219 req-5f9d66cc-abe5-4f31-8119-b46fbd2f1f86 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "bbfa7c88-b995-4c25-b981-1ada326424ed", "name": "network-vif-deleted", "tag": "1e6fddcb-af8d-4b4f-98ba-73539bd88138"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:51:22.328930 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-bd32229a-62f3-4d60-83f5-e82a722f6219 req-5f9d66cc-abe5-4f31-8119-b46fbd2f1f86 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:51:22.329625 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-bd32229a-62f3-4d60-83f5-e82a722f6219 req-5f9d66cc-abe5-4f31-8119-b46fbd2f1f86 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:51:22.347040 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-bd32229a-62f3-4d60-83f5-e82a722f6219 req-5f9d66cc-abe5-4f31-8119-b46fbd2f1f86 service nova] Creating event network-vif-deleted:1e6fddcb-af8d-4b4f-98ba-73539bd88138 for instance bbfa7c88-b995-4c25-b981-1ada326424ed on nch2-master-1 Jul 16 21:51:22.369296 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-bd32229a-62f3-4d60-83f5-e82a722f6219 req-5f9d66cc-abe5-4f31-8119-b46fbd2f1f86 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.058382 Jul 16 21:51:22.370219 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3097/6199] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:51:22 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 62 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:51:22.612695 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-cc292f21-ca79-45b6-b92b-f5a9923592a4 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:51:22.613325 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-cc292f21-ca79-45b6-b92b-f5a9923592a4 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: bbfa7c88-b995-4c25-b981-1ada326424ed] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:51:22.621789 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cc292f21-ca79-45b6-b92b-f5a9923592a4 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:51:22.621789 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cc292f21-ca79-45b6-b92b-f5a9923592a4 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:51:22.622297 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cc292f21-ca79-45b6-b92b-f5a9923592a4 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:51:22.623045 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cc292f21-ca79-45b6-b92b-f5a9923592a4 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:51:22.708167 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-cc292f21-ca79-45b6-b92b-f5a9923592a4 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:51:22.718215 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-cc292f21-ca79-45b6-b92b-f5a9923592a4 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/bbfa7c88-b995-4c25-b981-1ada326424ed" status: 200 len: 1299 microversion: 2.1 time: 0.109586 Jul 16 21:51:22.731712 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3103/6200] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:51:22 2022] GET /compute/v2.1/servers/bbfa7c88-b995-4c25-b981-1ada326424ed => generated 1299 bytes in 121 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:51:23.738333 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-58649cdb-a2d5-4f28-bb99-6858780a52e5 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:51:23.739083 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-58649cdb-a2d5-4f28-bb99-6858780a52e5 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: bbfa7c88-b995-4c25-b981-1ada326424ed] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:51:23.746995 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-58649cdb-a2d5-4f28-bb99-6858780a52e5 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:51:23.747514 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-58649cdb-a2d5-4f28-bb99-6858780a52e5 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:51:23.748330 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-58649cdb-a2d5-4f28-bb99-6858780a52e5 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:51:23.748771 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-58649cdb-a2d5-4f28-bb99-6858780a52e5 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:51:23.769186 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-58649cdb-a2d5-4f28-bb99-6858780a52e5 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] HTTP exception thrown: Instance bbfa7c88-b995-4c25-b981-1ada326424ed could not be found. Jul 16 21:51:23.770939 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-58649cdb-a2d5-4f28-bb99-6858780a52e5 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Request method failure captured: Jul 16 21:51:23.770939 ncdv-master-1 devstack@n-api.service[138680]: request: GET /compute/v2.1/servers/bbfa7c88-b995-4c25-b981-1ada326424ed HTTP/1.1 Jul 16 21:51:23.770939 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:51:23.770939 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:51:23.770939 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:51:23.770939 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 0 Jul 16 21:51:23.770939 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:51:23.770939 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:51:23.770939 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:51:23.770939 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:51:23.770939 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0zFqdHrIN0eOi2qg4drPydwu6SP4etFLyy6aDaqxzq0WQeMBWfEvVpgvWD4EE3q6tgQ3bkIiQt72uW6dWCfa-p4Yuvsc21rwij0v1H7vvz1_kRSKLl1Dn4VMOw8LrCFQcDku8xWPGzPME_8kBIMghBmgh0Bs2vThYjVU90trc_bK9dc Jul 16 21:51:23.770939 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:51:23.770939 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:51:23.770939 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:51:23.770939 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:51:23.770939 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:51:23.770939 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:51:23.770939 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: 1e7cd69b8ddd4aceb45a4aaacdfccea7 Jul 16 21:51:23.770939 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-ServerActionsTestJSON-1394927270 Jul 16 21:51:23.770939 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:51:23.770939 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:51:23.770939 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/1e7cd69b8ddd4aceb45a4aaacdfccea7"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/1e7cd69b8ddd4aceb45a4aaacdfccea7"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_1e7cd69b8ddd4aceb45a4aaacdfccea7"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/1e7cd69b8ddd4aceb45a4aaacdfccea7"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:51:23.770939 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-ServerActionsTestJSON-1394927270 Jul 16 21:51:23.770939 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: 1e7cd69b8ddd4aceb45a4aaacdfccea7 Jul 16 21:51:23.770939 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-ServerActionsTestJSON-1394927270 Jul 16 21:51:23.770939 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-ServerActionsTestJSON-1394927270-project Jul 16 21:51:23.770939 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:51:23.770939 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:51:23.770939 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: cfe293b9ed3645229bed5ef8ce17cfec Jul 16 21:51:23.770939 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-ServerActionsTestJSON-1394927270-project Jul 16 21:51:23.770939 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 21:51:23.770939 ncdv-master-1 devstack@n-api.service[138680]: exception: Instance bbfa7c88-b995-4c25-b981-1ada326424ed could not be found. Jul 16 21:51:23.770939 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': 'bbfa7c88-b995-4c25-b981-1ada326424ed'} Jul 16 21:51:23.770939 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:51:23.770939 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:51:23.770939 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:51:23.770939 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:51:23.770939 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:51:23.770939 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:51:23.770939 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:51:23.770939 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:51:23.770939 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:51:23.770939 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:51:23.770939 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance bbfa7c88-b995-4c25-b981-1ada326424ed could not be found. Jul 16 21:51:23.770939 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:51:23.770939 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:51:23.770939 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:51:23.770939 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:51:23.770939 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:51:23.770939 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:51:23.770939 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:51:23.770939 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:51:23.770939 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:51:23.770939 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:51:23.770939 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:51:23.770939 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:51:23.770939 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:51:23.770939 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:51:23.770939 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:51:23.770939 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:51:23.770939 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance bbfa7c88-b995-4c25-b981-1ada326424ed could not be found. Jul 16 21:51:23.770939 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:51:23.770939 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:51:23.770939 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:51:23.775647 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:51:23.775647 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:51:23.775647 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:51:23.775647 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:51:23.775647 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:51:23.775647 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance bbfa7c88-b995-4c25-b981-1ada326424ed could not be found. Jul 16 21:51:23.775647 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:51:23.775647 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-58649cdb-a2d5-4f28-bb99-6858780a52e5 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Returning 404 to user: Instance bbfa7c88-b995-4c25-b981-1ada326424ed could not be found. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:51:23.775647 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-58649cdb-a2d5-4f28-bb99-6858780a52e5 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/bbfa7c88-b995-4c25-b981-1ada326424ed" status: 404 len: 111 microversion: 2.1 time: 0.039579 Jul 16 21:51:23.776836 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3098/6201] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:51:23 2022] GET /compute/v2.1/servers/bbfa7c88-b995-4c25-b981-1ada326424ed => generated 111 bytes in 45 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:51:23.783271 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-1b29a83c-c0f1-452c-a14b-1210bec1246c tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:51:23.783820 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-1b29a83c-c0f1-452c-a14b-1210bec1246c tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: eb9f9cfd-8fe0-4803-8f57-e477e1efb7ad] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:51:23.792941 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1b29a83c-c0f1-452c-a14b-1210bec1246c tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:51:23.793309 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1b29a83c-c0f1-452c-a14b-1210bec1246c tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:51:23.794130 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1b29a83c-c0f1-452c-a14b-1210bec1246c tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:51:23.794577 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1b29a83c-c0f1-452c-a14b-1210bec1246c tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:51:23.822773 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-1b29a83c-c0f1-452c-a14b-1210bec1246c tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: eb9f9cfd-8fe0-4803-8f57-e477e1efb7ad] Going to try to terminate instance {{(pid=138681) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 16 21:51:24.173622 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-1b29a83c-c0f1-452c-a14b-1210bec1246c tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "DELETE /compute/v2.1/servers/eb9f9cfd-8fe0-4803-8f57-e477e1efb7ad" status: 204 len: 0 microversion: 2.1 time: 0.394331 Jul 16 21:51:24.174318 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3104/6202] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:51:23 2022] DELETE /compute/v2.1/servers/eb9f9cfd-8fe0-4803-8f57-e477e1efb7ad => generated 0 bytes in 396 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 16 21:51:24.183437 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-cc0bf4ff-0be0-4723-9249-b1d610a07c13 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:51:24.184055 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-cc0bf4ff-0be0-4723-9249-b1d610a07c13 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: eb9f9cfd-8fe0-4803-8f57-e477e1efb7ad] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:51:24.190124 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cc0bf4ff-0be0-4723-9249-b1d610a07c13 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:51:24.190579 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cc0bf4ff-0be0-4723-9249-b1d610a07c13 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:51:24.191385 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cc0bf4ff-0be0-4723-9249-b1d610a07c13 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:51:24.191826 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cc0bf4ff-0be0-4723-9249-b1d610a07c13 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:51:24.333295 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-cc0bf4ff-0be0-4723-9249-b1d610a07c13 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:51:24.341693 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-cc0bf4ff-0be0-4723-9249-b1d610a07c13 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/eb9f9cfd-8fe0-4803-8f57-e477e1efb7ad" status: 200 len: 1493 microversion: 2.1 time: 0.163014 Jul 16 21:51:24.342461 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3099/6203] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:51:24 2022] GET /compute/v2.1/servers/eb9f9cfd-8fe0-4803-8f57-e477e1efb7ad => generated 1493 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:51:24.745795 ncdv-master-1 devstack@n-api.service[138680]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jul 16 21:51:24.746420 ncdv-master-1 devstack@n-api.service[138680]: Sat Jul 16 21:51:24 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Jul 16 21:51:25.359806 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-309b6051-0c6d-43c9-a073-972d142db238 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:51:25.360538 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-309b6051-0c6d-43c9-a073-972d142db238 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: eb9f9cfd-8fe0-4803-8f57-e477e1efb7ad] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:51:25.368060 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-309b6051-0c6d-43c9-a073-972d142db238 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:51:25.368528 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-309b6051-0c6d-43c9-a073-972d142db238 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:51:25.369308 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-309b6051-0c6d-43c9-a073-972d142db238 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:51:25.369775 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-309b6051-0c6d-43c9-a073-972d142db238 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:51:25.505834 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-309b6051-0c6d-43c9-a073-972d142db238 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:51:25.513946 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-309b6051-0c6d-43c9-a073-972d142db238 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/eb9f9cfd-8fe0-4803-8f57-e477e1efb7ad" status: 200 len: 1493 microversion: 2.1 time: 0.158584 Jul 16 21:51:25.514529 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3105/6204] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:51:25 2022] GET /compute/v2.1/servers/eb9f9cfd-8fe0-4803-8f57-e477e1efb7ad => generated 1493 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:51:25.979911 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-23667eaf-0ea9-463d-9abe-879b39e95b30 req-0450f744-1844-4994-91fc-8bb1ddb64183 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "eb9f9cfd-8fe0-4803-8f57-e477e1efb7ad", "name": "network-vif-deleted", "tag": "9d393a2c-3f5b-46a5-a145-dd932faa5e91"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:51:25.993803 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-23667eaf-0ea9-463d-9abe-879b39e95b30 req-0450f744-1844-4994-91fc-8bb1ddb64183 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:51:25.995109 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-23667eaf-0ea9-463d-9abe-879b39e95b30 req-0450f744-1844-4994-91fc-8bb1ddb64183 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:51:26.008643 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-23667eaf-0ea9-463d-9abe-879b39e95b30 req-0450f744-1844-4994-91fc-8bb1ddb64183 service nova] Creating event network-vif-deleted:9d393a2c-3f5b-46a5-a145-dd932faa5e91 for instance eb9f9cfd-8fe0-4803-8f57-e477e1efb7ad on nch1-master-1 Jul 16 21:51:26.014485 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-23667eaf-0ea9-463d-9abe-879b39e95b30 req-0450f744-1844-4994-91fc-8bb1ddb64183 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.039717 Jul 16 21:51:26.016637 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3100/6205] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:51:25 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:51:26.531940 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-538e338e-6129-448c-ba6f-b75d66d507d3 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:51:26.532573 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-538e338e-6129-448c-ba6f-b75d66d507d3 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: eb9f9cfd-8fe0-4803-8f57-e477e1efb7ad] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:51:26.541996 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-538e338e-6129-448c-ba6f-b75d66d507d3 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:51:26.542561 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-538e338e-6129-448c-ba6f-b75d66d507d3 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:51:26.543491 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-538e338e-6129-448c-ba6f-b75d66d507d3 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:51:26.544106 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-538e338e-6129-448c-ba6f-b75d66d507d3 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:51:26.582870 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-538e338e-6129-448c-ba6f-b75d66d507d3 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:51:26.583374 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-538e338e-6129-448c-ba6f-b75d66d507d3 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:51:26.583847 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.objects.instance [None req-538e338e-6129-448c-ba6f-b75d66d507d3 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lazy-loading 'fault' on Instance uuid eb9f9cfd-8fe0-4803-8f57-e477e1efb7ad {{(pid=138681) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:51:26.632994 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-538e338e-6129-448c-ba6f-b75d66d507d3 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:51:26.641073 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-538e338e-6129-448c-ba6f-b75d66d507d3 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/eb9f9cfd-8fe0-4803-8f57-e477e1efb7ad" status: 200 len: 1304 microversion: 2.1 time: 0.113045 Jul 16 21:51:26.641548 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3106/6206] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:51:26 2022] GET /compute/v2.1/servers/eb9f9cfd-8fe0-4803-8f57-e477e1efb7ad => generated 1304 bytes in 114 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:51:27.658363 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-255e45ef-8ab3-484d-a70d-79d472075391 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:51:27.659119 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-255e45ef-8ab3-484d-a70d-79d472075391 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: eb9f9cfd-8fe0-4803-8f57-e477e1efb7ad] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:51:27.666608 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-255e45ef-8ab3-484d-a70d-79d472075391 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:51:27.667119 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-255e45ef-8ab3-484d-a70d-79d472075391 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:51:27.667920 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-255e45ef-8ab3-484d-a70d-79d472075391 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:51:27.668407 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-255e45ef-8ab3-484d-a70d-79d472075391 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:51:27.682345 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-255e45ef-8ab3-484d-a70d-79d472075391 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] HTTP exception thrown: Instance eb9f9cfd-8fe0-4803-8f57-e477e1efb7ad could not be found. Jul 16 21:51:27.684060 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-255e45ef-8ab3-484d-a70d-79d472075391 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Request method failure captured: Jul 16 21:51:27.684060 ncdv-master-1 devstack@n-api.service[138680]: request: GET /compute/v2.1/servers/eb9f9cfd-8fe0-4803-8f57-e477e1efb7ad HTTP/1.1 Jul 16 21:51:27.684060 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:51:27.684060 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:51:27.684060 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:51:27.684060 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 0 Jul 16 21:51:27.684060 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:51:27.684060 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:51:27.684060 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:51:27.684060 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:51:27.684060 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0zFqdHrIN0eOi2qg4drPydwu6SP4etFLyy6aDaqxzq0WQeMBWfEvVpgvWD4EE3q6tgQ3bkIiQt72uW6dWCfa-p4Yuvsc21rwij0v1H7vvz1_kRSKLl1Dn4VMOw8LrCFQcDku8xWPGzPME_8kBIMghBmgh0Bs2vThYjVU90trc_bK9dc Jul 16 21:51:27.684060 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:51:27.684060 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:51:27.684060 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:51:27.684060 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:51:27.684060 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:51:27.684060 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:51:27.684060 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: 1e7cd69b8ddd4aceb45a4aaacdfccea7 Jul 16 21:51:27.684060 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-ServerActionsTestJSON-1394927270 Jul 16 21:51:27.684060 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:51:27.684060 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:51:27.684060 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/1e7cd69b8ddd4aceb45a4aaacdfccea7"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/1e7cd69b8ddd4aceb45a4aaacdfccea7"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_1e7cd69b8ddd4aceb45a4aaacdfccea7"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/1e7cd69b8ddd4aceb45a4aaacdfccea7"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:51:27.684060 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-ServerActionsTestJSON-1394927270 Jul 16 21:51:27.684060 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: 1e7cd69b8ddd4aceb45a4aaacdfccea7 Jul 16 21:51:27.684060 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-ServerActionsTestJSON-1394927270 Jul 16 21:51:27.684060 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-ServerActionsTestJSON-1394927270-project Jul 16 21:51:27.684060 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:51:27.684060 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:51:27.684060 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: cfe293b9ed3645229bed5ef8ce17cfec Jul 16 21:51:27.684060 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-ServerActionsTestJSON-1394927270-project Jul 16 21:51:27.684060 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 21:51:27.684060 ncdv-master-1 devstack@n-api.service[138680]: exception: Instance eb9f9cfd-8fe0-4803-8f57-e477e1efb7ad could not be found. Jul 16 21:51:27.684060 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': 'eb9f9cfd-8fe0-4803-8f57-e477e1efb7ad'} Jul 16 21:51:27.684060 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:51:27.684060 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:51:27.684060 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:51:27.684060 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:51:27.684060 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:51:27.684060 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:51:27.684060 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:51:27.684060 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:51:27.684060 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:51:27.684060 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:51:27.684060 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance eb9f9cfd-8fe0-4803-8f57-e477e1efb7ad could not be found. Jul 16 21:51:27.684060 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:51:27.684060 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:51:27.684060 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:51:27.684060 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:51:27.684060 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:51:27.684060 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:51:27.684060 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:51:27.684060 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:51:27.684060 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:51:27.684060 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:51:27.684060 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:51:27.684060 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:51:27.684060 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:51:27.684060 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:51:27.684060 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:51:27.684060 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:51:27.684060 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance eb9f9cfd-8fe0-4803-8f57-e477e1efb7ad could not be found. Jul 16 21:51:27.684060 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:51:27.684060 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:51:27.684060 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:51:27.688309 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:51:27.688309 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:51:27.688309 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:51:27.688309 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:51:27.688309 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:51:27.688309 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance eb9f9cfd-8fe0-4803-8f57-e477e1efb7ad could not be found. Jul 16 21:51:27.688309 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:51:27.688309 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-255e45ef-8ab3-484d-a70d-79d472075391 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Returning 404 to user: Instance eb9f9cfd-8fe0-4803-8f57-e477e1efb7ad could not be found. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:51:27.688309 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-255e45ef-8ab3-484d-a70d-79d472075391 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/eb9f9cfd-8fe0-4803-8f57-e477e1efb7ad" status: 404 len: 111 microversion: 2.1 time: 0.031507 Jul 16 21:51:27.688309 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3101/6207] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:51:27 2022] GET /compute/v2.1/servers/eb9f9cfd-8fe0-4803-8f57-e477e1efb7ad => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:51:27.694119 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-09e5097d-d3fe-4a65-94a9-e685e4a43e0f tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:51:27.695327 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-09e5097d-d3fe-4a65-94a9-e685e4a43e0f tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: 3618ceb8-9d10-4219-8885-d2d5004c71ad] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:51:27.702291 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-09e5097d-d3fe-4a65-94a9-e685e4a43e0f tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:51:27.702764 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-09e5097d-d3fe-4a65-94a9-e685e4a43e0f tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:51:27.703581 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-09e5097d-d3fe-4a65-94a9-e685e4a43e0f tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:51:27.704042 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-09e5097d-d3fe-4a65-94a9-e685e4a43e0f tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:51:27.729647 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-09e5097d-d3fe-4a65-94a9-e685e4a43e0f tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: 3618ceb8-9d10-4219-8885-d2d5004c71ad] Going to try to terminate instance {{(pid=138681) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 16 21:51:28.034538 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-09e5097d-d3fe-4a65-94a9-e685e4a43e0f tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "DELETE /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad" status: 204 len: 0 microversion: 2.1 time: 0.343999 Jul 16 21:51:28.035162 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3107/6208] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:51:27 2022] DELETE /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad => generated 0 bytes in 346 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Jul 16 21:51:28.044377 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-cd5e735c-e041-43e0-9cd3-fed0f3302b22 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:51:28.044989 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-cd5e735c-e041-43e0-9cd3-fed0f3302b22 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: 3618ceb8-9d10-4219-8885-d2d5004c71ad] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:51:28.051230 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cd5e735c-e041-43e0-9cd3-fed0f3302b22 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:51:28.051687 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cd5e735c-e041-43e0-9cd3-fed0f3302b22 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:51:28.052443 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cd5e735c-e041-43e0-9cd3-fed0f3302b22 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:51:28.052889 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cd5e735c-e041-43e0-9cd3-fed0f3302b22 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:51:28.194889 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-cd5e735c-e041-43e0-9cd3-fed0f3302b22 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:51:28.202664 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-cd5e735c-e041-43e0-9cd3-fed0f3302b22 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad" status: 200 len: 1671 microversion: 2.1 time: 0.162238 Jul 16 21:51:28.203276 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3102/6209] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:51:28 2022] GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad => generated 1671 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:51:29.220691 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-2ba9c1d2-48f8-4135-87fa-f3621d095915 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:51:29.221340 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-2ba9c1d2-48f8-4135-87fa-f3621d095915 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: 3618ceb8-9d10-4219-8885-d2d5004c71ad] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:51:29.229616 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2ba9c1d2-48f8-4135-87fa-f3621d095915 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:51:29.230081 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2ba9c1d2-48f8-4135-87fa-f3621d095915 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:51:29.230859 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2ba9c1d2-48f8-4135-87fa-f3621d095915 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:51:29.231351 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2ba9c1d2-48f8-4135-87fa-f3621d095915 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:51:29.373316 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-2ba9c1d2-48f8-4135-87fa-f3621d095915 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:51:29.380934 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-2ba9c1d2-48f8-4135-87fa-f3621d095915 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad" status: 200 len: 1671 microversion: 2.1 time: 0.164608 Jul 16 21:51:29.382575 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3108/6210] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:51:29 2022] GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad => generated 1671 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:51:30.400134 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-79c77cc6-3442-40ad-91b7-5685dcbb6d32 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:51:30.400767 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-79c77cc6-3442-40ad-91b7-5685dcbb6d32 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: 3618ceb8-9d10-4219-8885-d2d5004c71ad] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:51:30.403866 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_api:SELECT=50 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:51:30.408894 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-79c77cc6-3442-40ad-91b7-5685dcbb6d32 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:51:30.409339 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-79c77cc6-3442-40ad-91b7-5685dcbb6d32 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:51:30.410108 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-79c77cc6-3442-40ad-91b7-5685dcbb6d32 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:51:30.410549 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-79c77cc6-3442-40ad-91b7-5685dcbb6d32 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:51:30.553142 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-79c77cc6-3442-40ad-91b7-5685dcbb6d32 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:51:30.561158 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-79c77cc6-3442-40ad-91b7-5685dcbb6d32 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad" status: 200 len: 1671 microversion: 2.1 time: 0.165703 Jul 16 21:51:30.561735 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3103/6211] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:51:30 2022] GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad => generated 1671 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:51:31.568572 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-79977b8e-5f29-4fda-873e-f9344f609ead req-cfa5c745-b81d-4b6e-b952-e7e4feb01318 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "3618ceb8-9d10-4219-8885-d2d5004c71ad", "name": "network-vif-deleted", "tag": "ef163361-e10a-4408-b519-5d053d2a55d6"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:51:31.576790 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_api:SELECT=80,nova_api:DELETE=1,nova_api:UPDATE=5 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:51:31.579307 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-20e90c36-0ccc-4df8-a7f9-e4eae74534f1 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:51:31.580184 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-20e90c36-0ccc-4df8-a7f9-e4eae74534f1 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: 3618ceb8-9d10-4219-8885-d2d5004c71ad] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:51:31.582476 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-79977b8e-5f29-4fda-873e-f9344f609ead req-cfa5c745-b81d-4b6e-b952-e7e4feb01318 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:51:31.582937 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-79977b8e-5f29-4fda-873e-f9344f609ead req-cfa5c745-b81d-4b6e-b952-e7e4feb01318 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:51:31.593333 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-20e90c36-0ccc-4df8-a7f9-e4eae74534f1 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:51:31.595071 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-20e90c36-0ccc-4df8-a7f9-e4eae74534f1 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:51:31.595981 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-20e90c36-0ccc-4df8-a7f9-e4eae74534f1 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:51:31.596146 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-79977b8e-5f29-4fda-873e-f9344f609ead req-cfa5c745-b81d-4b6e-b952-e7e4feb01318 service nova] Creating event network-vif-deleted:ef163361-e10a-4408-b519-5d053d2a55d6 for instance 3618ceb8-9d10-4219-8885-d2d5004c71ad on nch1-master-1 Jul 16 21:51:31.600955 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-20e90c36-0ccc-4df8-a7f9-e4eae74534f1 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:51:31.602312 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-79977b8e-5f29-4fda-873e-f9344f609ead req-cfa5c745-b81d-4b6e-b952-e7e4feb01318 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.037597 Jul 16 21:51:31.603120 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3109/6212] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:51:31 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:51:31.690023 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-20e90c36-0ccc-4df8-a7f9-e4eae74534f1 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'cfe293b9ed3645229bed5ef8ce17cfec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7cd69b8ddd4aceb45a4aaacdfccea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:51:31.703381 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-20e90c36-0ccc-4df8-a7f9-e4eae74534f1 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad" status: 200 len: 1604 microversion: 2.1 time: 0.126430 Jul 16 21:51:31.712773 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3104/6213] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:51:31 2022] GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad => generated 1604 bytes in 138 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:51:32.721903 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-5fbeb2e4-2355-4dbb-ab14-fd587dd5b2b8 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:51:32.722970 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-5fbeb2e4-2355-4dbb-ab14-fd587dd5b2b8 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] [instance: 3618ceb8-9d10-4219-8885-d2d5004c71ad] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:51:32.730605 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5fbeb2e4-2355-4dbb-ab14-fd587dd5b2b8 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:51:32.731096 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5fbeb2e4-2355-4dbb-ab14-fd587dd5b2b8 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:51:32.731885 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5fbeb2e4-2355-4dbb-ab14-fd587dd5b2b8 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:51:32.732333 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5fbeb2e4-2355-4dbb-ab14-fd587dd5b2b8 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:51:32.746580 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-5fbeb2e4-2355-4dbb-ab14-fd587dd5b2b8 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] HTTP exception thrown: Instance 3618ceb8-9d10-4219-8885-d2d5004c71ad could not be found. Jul 16 21:51:32.748282 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-5fbeb2e4-2355-4dbb-ab14-fd587dd5b2b8 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Request method failure captured: Jul 16 21:51:32.748282 ncdv-master-1 devstack@n-api.service[138681]: request: GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad HTTP/1.1 Jul 16 21:51:32.748282 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:51:32.748282 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:51:32.748282 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:51:32.748282 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 21:51:32.748282 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:51:32.748282 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:51:32.748282 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:51:32.748282 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:51:32.748282 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0zFqdHrIN0eOi2qg4drPydwu6SP4etFLyy6aDaqxzq0WQeMBWfEvVpgvWD4EE3q6tgQ3bkIiQt72uW6dWCfa-p4Yuvsc21rwij0v1H7vvz1_kRSKLl1Dn4VMOw8LrCFQcDku8xWPGzPME_8kBIMghBmgh0Bs2vThYjVU90trc_bK9dc Jul 16 21:51:32.748282 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:51:32.748282 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:51:32.748282 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:51:32.748282 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:51:32.748282 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:51:32.748282 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:51:32.748282 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: 1e7cd69b8ddd4aceb45a4aaacdfccea7 Jul 16 21:51:32.748282 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-ServerActionsTestJSON-1394927270 Jul 16 21:51:32.748282 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:51:32.748282 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:51:32.748282 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/1e7cd69b8ddd4aceb45a4aaacdfccea7"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/1e7cd69b8ddd4aceb45a4aaacdfccea7"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_1e7cd69b8ddd4aceb45a4aaacdfccea7"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/1e7cd69b8ddd4aceb45a4aaacdfccea7"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:51:32.748282 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-ServerActionsTestJSON-1394927270 Jul 16 21:51:32.748282 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: 1e7cd69b8ddd4aceb45a4aaacdfccea7 Jul 16 21:51:32.748282 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-ServerActionsTestJSON-1394927270 Jul 16 21:51:32.748282 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-ServerActionsTestJSON-1394927270-project Jul 16 21:51:32.748282 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:51:32.748282 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:51:32.748282 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: cfe293b9ed3645229bed5ef8ce17cfec Jul 16 21:51:32.748282 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-ServerActionsTestJSON-1394927270-project Jul 16 21:51:32.748282 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:51:32.748282 ncdv-master-1 devstack@n-api.service[138681]: exception: Instance 3618ceb8-9d10-4219-8885-d2d5004c71ad could not be found. Jul 16 21:51:32.748282 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': '3618ceb8-9d10-4219-8885-d2d5004c71ad'} Jul 16 21:51:32.748282 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:51:32.748282 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:51:32.748282 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:51:32.748282 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:51:32.748282 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:51:32.748282 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:51:32.748282 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:51:32.748282 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:51:32.748282 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:51:32.748282 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:51:32.748282 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 3618ceb8-9d10-4219-8885-d2d5004c71ad could not be found. Jul 16 21:51:32.748282 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:51:32.748282 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:51:32.748282 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:51:32.748282 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:51:32.748282 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:51:32.748282 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:51:32.748282 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:51:32.748282 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:51:32.748282 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:51:32.748282 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:51:32.748282 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:51:32.748282 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:51:32.748282 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:51:32.748282 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:51:32.748282 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:51:32.748282 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:51:32.748282 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 3618ceb8-9d10-4219-8885-d2d5004c71ad could not be found. Jul 16 21:51:32.748282 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:51:32.748282 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:51:32.748282 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:51:32.752668 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:51:32.752668 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:51:32.752668 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:51:32.752668 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:51:32.752668 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:51:32.752668 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 3618ceb8-9d10-4219-8885-d2d5004c71ad could not be found. Jul 16 21:51:32.752668 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:51:32.752668 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-5fbeb2e4-2355-4dbb-ab14-fd587dd5b2b8 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Returning 404 to user: Instance 3618ceb8-9d10-4219-8885-d2d5004c71ad could not be found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:51:32.752668 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-5fbeb2e4-2355-4dbb-ab14-fd587dd5b2b8 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad" status: 404 len: 111 microversion: 2.1 time: 0.033746 Jul 16 21:51:32.752668 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3110/6214] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:51:32 2022] GET /compute/v2.1/servers/3618ceb8-9d10-4219-8885-d2d5004c71ad => generated 111 bytes in 35 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:51:32.758905 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-55e3e943-1f18-463f-b01e-a99ae26b40f4 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] Calling method '.version_select at 0x7fdd9c6b6ca0>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:51:32.875653 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-55e3e943-1f18-463f-b01e-a99ae26b40f4 tempest-ServerActionsTestJSON-1394927270 tempest-ServerActionsTestJSON-1394927270-project] 10.222.0.16 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-225926832" status: 202 len: 0 microversion: 2.1 time: 0.120932 Jul 16 21:51:32.876357 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3105/6215] 10.222.0.16 () {58 vars in 1217 bytes} [Sat Jul 16 21:51:32 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-225926832 => generated 0 bytes in 122 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 21:51:45.536065 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=68,nova_cell1:INSERT=3,nova_cell1:UPDATE=2 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:51:45.680139 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-deb1312a-4793-48c5-a062-42435b2089ce tempest-ServerMetadataNegativeTestJSON-1202925361 tempest-ServerMetadataNegativeTestJSON-1202925361-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerMetadataNegativeTestJSON-server-1232799627", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452", "metadata": {}, "networks": [{"uuid": "52901b70-e3ce-40b9-9a01-0d9a211da013"}]}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:51:45.742956 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.network.neutron [None req-deb1312a-4793-48c5-a062-42435b2089ce tempest-ServerMetadataNegativeTestJSON-1202925361 tempest-ServerMetadataNegativeTestJSON-1202925361-project] validate_networks() for [('52901b70-e3ce-40b9-9a01-0d9a211da013', None, None, None, None, None)] {{(pid=138681) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 16 21:51:46.219315 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-deb1312a-4793-48c5-a062-42435b2089ce tempest-ServerMetadataNegativeTestJSON-1202925361 tempest-ServerMetadataNegativeTestJSON-1202925361-project] Flavor limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 16 21:51:46.219765 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-deb1312a-4793-48c5-a062-42435b2089ce tempest-ServerMetadataNegativeTestJSON-1202925361 tempest-ServerMetadataNegativeTestJSON-1202925361-project] Image limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 16 21:51:46.220297 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-deb1312a-4793-48c5-a062-42435b2089ce tempest-ServerMetadataNegativeTestJSON-1202925361 tempest-ServerMetadataNegativeTestJSON-1202925361-project] Flavor pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 16 21:51:46.220759 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-deb1312a-4793-48c5-a062-42435b2089ce tempest-ServerMetadataNegativeTestJSON-1202925361 tempest-ServerMetadataNegativeTestJSON-1202925361-project] Image pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 16 21:51:46.221225 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-deb1312a-4793-48c5-a062-42435b2089ce tempest-ServerMetadataNegativeTestJSON-1202925361 tempest-ServerMetadataNegativeTestJSON-1202925361-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 16 21:51:46.231751 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-deb1312a-4793-48c5-a062-42435b2089ce tempest-ServerMetadataNegativeTestJSON-1202925361 tempest-ServerMetadataNegativeTestJSON-1202925361-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:51:46.232234 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-deb1312a-4793-48c5-a062-42435b2089ce tempest-ServerMetadataNegativeTestJSON-1202925361 tempest-ServerMetadataNegativeTestJSON-1202925361-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:51:46.233076 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-deb1312a-4793-48c5-a062-42435b2089ce tempest-ServerMetadataNegativeTestJSON-1202925361 tempest-ServerMetadataNegativeTestJSON-1202925361-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:51:46.238118 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-deb1312a-4793-48c5-a062-42435b2089ce tempest-ServerMetadataNegativeTestJSON-1202925361 tempest-ServerMetadataNegativeTestJSON-1202925361-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:51:46.257329 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-deb1312a-4793-48c5-a062-42435b2089ce tempest-ServerMetadataNegativeTestJSON-1202925361 tempest-ServerMetadataNegativeTestJSON-1202925361-project] Getting quotas for project ecb79581f83f46f6a02fba932e20f3fe. Resources: {'ram', 'instances', 'cores'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:51:46.262971 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-deb1312a-4793-48c5-a062-42435b2089ce tempest-ServerMetadataNegativeTestJSON-1202925361 tempest-ServerMetadataNegativeTestJSON-1202925361-project] Getting quotas for user 8d3cada15a55453faaf23a147c44afcf and project ecb79581f83f46f6a02fba932e20f3fe. Resources: {'ram', 'instances', 'cores'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:51:46.275654 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-deb1312a-4793-48c5-a062-42435b2089ce tempest-ServerMetadataNegativeTestJSON-1202925361 tempest-ServerMetadataNegativeTestJSON-1202925361-project] Going to run 1 instances... {{(pid=138681) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 16 21:51:46.277965 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-deb1312a-4793-48c5-a062-42435b2089ce tempest-ServerMetadataNegativeTestJSON-1202925361 tempest-ServerMetadataNegativeTestJSON-1202925361-project] [instance: c1c65e9e-289c-4707-b6f7-45fe851f4fae] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138681) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 16 21:51:46.435896 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-deb1312a-4793-48c5-a062-42435b2089ce tempest-ServerMetadataNegativeTestJSON-1202925361 tempest-ServerMetadataNegativeTestJSON-1202925361-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.905698 Jul 16 21:51:46.436557 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3111/6216] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 21:51:45 2022] POST /compute/v2.1/servers => generated 380 bytes in 907 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 16 21:51:46.445812 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=67,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:51:46.447232 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_api:SELECT=6,nova_api:DELETE=1 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:51:46.451234 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-de7e35d4-967d-432c-84cf-767c976696a9 tempest-ServerMetadataNegativeTestJSON-1202925361 tempest-ServerMetadataNegativeTestJSON-1202925361-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:51:46.451861 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-de7e35d4-967d-432c-84cf-767c976696a9 tempest-ServerMetadataNegativeTestJSON-1202925361 tempest-ServerMetadataNegativeTestJSON-1202925361-project] [instance: c1c65e9e-289c-4707-b6f7-45fe851f4fae] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:51:46.520564 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-de7e35d4-967d-432c-84cf-767c976696a9 tempest-ServerMetadataNegativeTestJSON-1202925361 tempest-ServerMetadataNegativeTestJSON-1202925361-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8d3cada15a55453faaf23a147c44afcf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ecb79581f83f46f6a02fba932e20f3fe', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:51:46.535503 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-de7e35d4-967d-432c-84cf-767c976696a9 tempest-ServerMetadataNegativeTestJSON-1202925361 tempest-ServerMetadataNegativeTestJSON-1202925361-project] 10.222.0.16 "GET /compute/v2.1/servers/c1c65e9e-289c-4707-b6f7-45fe851f4fae" status: 200 len: 1224 microversion: 2.1 time: 0.092710 Jul 16 21:51:46.536108 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3106/6217] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:51:46 2022] GET /compute/v2.1/servers/c1c65e9e-289c-4707-b6f7-45fe851f4fae => generated 1224 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:51:47.554264 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-dccf9ffd-b307-4789-a6e5-3f25b9121ed5 tempest-ServerMetadataNegativeTestJSON-1202925361 tempest-ServerMetadataNegativeTestJSON-1202925361-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:51:47.554931 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-dccf9ffd-b307-4789-a6e5-3f25b9121ed5 tempest-ServerMetadataNegativeTestJSON-1202925361 tempest-ServerMetadataNegativeTestJSON-1202925361-project] [instance: c1c65e9e-289c-4707-b6f7-45fe851f4fae] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:51:47.637878 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-dccf9ffd-b307-4789-a6e5-3f25b9121ed5 tempest-ServerMetadataNegativeTestJSON-1202925361 tempest-ServerMetadataNegativeTestJSON-1202925361-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8d3cada15a55453faaf23a147c44afcf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ecb79581f83f46f6a02fba932e20f3fe', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:51:47.647574 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-dccf9ffd-b307-4789-a6e5-3f25b9121ed5 tempest-ServerMetadataNegativeTestJSON-1202925361 tempest-ServerMetadataNegativeTestJSON-1202925361-project] 10.222.0.16 "GET /compute/v2.1/servers/c1c65e9e-289c-4707-b6f7-45fe851f4fae" status: 200 len: 1224 microversion: 2.1 time: 0.098001 Jul 16 21:51:47.648173 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3112/6218] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:51:47 2022] GET /compute/v2.1/servers/c1c65e9e-289c-4707-b6f7-45fe851f4fae => generated 1224 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:51:48.664657 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-54faeae7-8e7c-4c9d-8474-ff86cfc142ae tempest-ServerMetadataNegativeTestJSON-1202925361 tempest-ServerMetadataNegativeTestJSON-1202925361-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:51:48.665398 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-54faeae7-8e7c-4c9d-8474-ff86cfc142ae tempest-ServerMetadataNegativeTestJSON-1202925361 tempest-ServerMetadataNegativeTestJSON-1202925361-project] [instance: c1c65e9e-289c-4707-b6f7-45fe851f4fae] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:51:48.674062 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-54faeae7-8e7c-4c9d-8474-ff86cfc142ae tempest-ServerMetadataNegativeTestJSON-1202925361 tempest-ServerMetadataNegativeTestJSON-1202925361-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:51:48.674636 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-54faeae7-8e7c-4c9d-8474-ff86cfc142ae tempest-ServerMetadataNegativeTestJSON-1202925361 tempest-ServerMetadataNegativeTestJSON-1202925361-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:51:48.677849 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-54faeae7-8e7c-4c9d-8474-ff86cfc142ae tempest-ServerMetadataNegativeTestJSON-1202925361 tempest-ServerMetadataNegativeTestJSON-1202925361-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:51:48.677849 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-54faeae7-8e7c-4c9d-8474-ff86cfc142ae tempest-ServerMetadataNegativeTestJSON-1202925361 tempest-ServerMetadataNegativeTestJSON-1202925361-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:51:48.747297 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-54faeae7-8e7c-4c9d-8474-ff86cfc142ae tempest-ServerMetadataNegativeTestJSON-1202925361 tempest-ServerMetadataNegativeTestJSON-1202925361-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8d3cada15a55453faaf23a147c44afcf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ecb79581f83f46f6a02fba932e20f3fe', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:51:48.755678 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-54faeae7-8e7c-4c9d-8474-ff86cfc142ae tempest-ServerMetadataNegativeTestJSON-1202925361 tempest-ServerMetadataNegativeTestJSON-1202925361-project] 10.222.0.16 "GET /compute/v2.1/servers/c1c65e9e-289c-4707-b6f7-45fe851f4fae" status: 200 len: 1276 microversion: 2.1 time: 0.095598 Jul 16 21:51:48.756389 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3107/6219] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:51:48 2022] GET /compute/v2.1/servers/c1c65e9e-289c-4707-b6f7-45fe851f4fae => generated 1276 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:51:49.773598 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-d1625431-3977-4627-ade2-ad0fe5b9b490 tempest-ServerMetadataNegativeTestJSON-1202925361 tempest-ServerMetadataNegativeTestJSON-1202925361-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:51:49.782334 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-d1625431-3977-4627-ade2-ad0fe5b9b490 tempest-ServerMetadataNegativeTestJSON-1202925361 tempest-ServerMetadataNegativeTestJSON-1202925361-project] [instance: c1c65e9e-289c-4707-b6f7-45fe851f4fae] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:51:49.792065 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d1625431-3977-4627-ade2-ad0fe5b9b490 tempest-ServerMetadataNegativeTestJSON-1202925361 tempest-ServerMetadataNegativeTestJSON-1202925361-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:51:49.792846 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d1625431-3977-4627-ade2-ad0fe5b9b490 tempest-ServerMetadataNegativeTestJSON-1202925361 tempest-ServerMetadataNegativeTestJSON-1202925361-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:51:49.793399 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d1625431-3977-4627-ade2-ad0fe5b9b490 tempest-ServerMetadataNegativeTestJSON-1202925361 tempest-ServerMetadataNegativeTestJSON-1202925361-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:51:49.793857 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d1625431-3977-4627-ade2-ad0fe5b9b490 tempest-ServerMetadataNegativeTestJSON-1202925361 tempest-ServerMetadataNegativeTestJSON-1202925361-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:51:49.956943 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-d1625431-3977-4627-ade2-ad0fe5b9b490 tempest-ServerMetadataNegativeTestJSON-1202925361 tempest-ServerMetadataNegativeTestJSON-1202925361-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8d3cada15a55453faaf23a147c44afcf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ecb79581f83f46f6a02fba932e20f3fe', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:51:49.965254 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-d1625431-3977-4627-ade2-ad0fe5b9b490 tempest-ServerMetadataNegativeTestJSON-1202925361 tempest-ServerMetadataNegativeTestJSON-1202925361-project] 10.222.0.16 "GET /compute/v2.1/servers/c1c65e9e-289c-4707-b6f7-45fe851f4fae" status: 200 len: 1324 microversion: 2.1 time: 0.196721 Jul 16 21:51:49.966102 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3113/6220] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:51:49 2022] GET /compute/v2.1/servers/c1c65e9e-289c-4707-b6f7-45fe851f4fae => generated 1324 bytes in 199 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:51:50.653102 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-a80aa1cd-a2fb-45f5-ac26-e772b405f68a req-a57d9ec7-21cd-472b-885c-bf24ff01ba80 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "c1c65e9e-289c-4707-b6f7-45fe851f4fae", "tag": "8100c1c9-4e96-49f0-a067-63ae803ab7be"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:51:50.665560 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-a80aa1cd-a2fb-45f5-ac26-e772b405f68a req-a57d9ec7-21cd-472b-885c-bf24ff01ba80 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:51:50.666013 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-a80aa1cd-a2fb-45f5-ac26-e772b405f68a req-a57d9ec7-21cd-472b-885c-bf24ff01ba80 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:51:50.680490 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-a80aa1cd-a2fb-45f5-ac26-e772b405f68a req-a57d9ec7-21cd-472b-885c-bf24ff01ba80 service nova] Creating event network-changed:8100c1c9-4e96-49f0-a067-63ae803ab7be for instance c1c65e9e-289c-4707-b6f7-45fe851f4fae on nch1-master-1 Jul 16 21:51:50.684661 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-a80aa1cd-a2fb-45f5-ac26-e772b405f68a req-a57d9ec7-21cd-472b-885c-bf24ff01ba80 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.035221 Jul 16 21:51:50.685628 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3108/6221] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:51:50 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:51:50.985154 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-e8cae996-9d9a-490c-87df-19331f6de7af tempest-ServerMetadataNegativeTestJSON-1202925361 tempest-ServerMetadataNegativeTestJSON-1202925361-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:51:50.985797 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-e8cae996-9d9a-490c-87df-19331f6de7af tempest-ServerMetadataNegativeTestJSON-1202925361 tempest-ServerMetadataNegativeTestJSON-1202925361-project] [instance: c1c65e9e-289c-4707-b6f7-45fe851f4fae] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:51:50.994156 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e8cae996-9d9a-490c-87df-19331f6de7af tempest-ServerMetadataNegativeTestJSON-1202925361 tempest-ServerMetadataNegativeTestJSON-1202925361-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:51:50.994630 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e8cae996-9d9a-490c-87df-19331f6de7af tempest-ServerMetadataNegativeTestJSON-1202925361 tempest-ServerMetadataNegativeTestJSON-1202925361-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:51:50.995480 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e8cae996-9d9a-490c-87df-19331f6de7af tempest-ServerMetadataNegativeTestJSON-1202925361 tempest-ServerMetadataNegativeTestJSON-1202925361-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:51:50.995941 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e8cae996-9d9a-490c-87df-19331f6de7af tempest-ServerMetadataNegativeTestJSON-1202925361 tempest-ServerMetadataNegativeTestJSON-1202925361-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:51:51.138432 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-e8cae996-9d9a-490c-87df-19331f6de7af tempest-ServerMetadataNegativeTestJSON-1202925361 tempest-ServerMetadataNegativeTestJSON-1202925361-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8d3cada15a55453faaf23a147c44afcf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ecb79581f83f46f6a02fba932e20f3fe', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:51:51.146620 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-e8cae996-9d9a-490c-87df-19331f6de7af tempest-ServerMetadataNegativeTestJSON-1202925361 tempest-ServerMetadataNegativeTestJSON-1202925361-project] 10.222.0.16 "GET /compute/v2.1/servers/c1c65e9e-289c-4707-b6f7-45fe851f4fae" status: 200 len: 1324 microversion: 2.1 time: 0.165437 Jul 16 21:51:51.147234 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3114/6222] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:51:50 2022] GET /compute/v2.1/servers/c1c65e9e-289c-4707-b6f7-45fe851f4fae => generated 1324 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:51:52.167506 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-b69474f8-b39e-4e48-97c0-54921f16b12a tempest-ServerMetadataNegativeTestJSON-1202925361 tempest-ServerMetadataNegativeTestJSON-1202925361-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:51:52.168186 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-b69474f8-b39e-4e48-97c0-54921f16b12a tempest-ServerMetadataNegativeTestJSON-1202925361 tempest-ServerMetadataNegativeTestJSON-1202925361-project] [instance: c1c65e9e-289c-4707-b6f7-45fe851f4fae] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:51:52.177061 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b69474f8-b39e-4e48-97c0-54921f16b12a tempest-ServerMetadataNegativeTestJSON-1202925361 tempest-ServerMetadataNegativeTestJSON-1202925361-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:51:52.177682 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b69474f8-b39e-4e48-97c0-54921f16b12a tempest-ServerMetadataNegativeTestJSON-1202925361 tempest-ServerMetadataNegativeTestJSON-1202925361-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:51:52.178592 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b69474f8-b39e-4e48-97c0-54921f16b12a tempest-ServerMetadataNegativeTestJSON-1202925361 tempest-ServerMetadataNegativeTestJSON-1202925361-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:51:52.179250 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b69474f8-b39e-4e48-97c0-54921f16b12a tempest-ServerMetadataNegativeTestJSON-1202925361 tempest-ServerMetadataNegativeTestJSON-1202925361-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:51:52.317050 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-b69474f8-b39e-4e48-97c0-54921f16b12a tempest-ServerMetadataNegativeTestJSON-1202925361 tempest-ServerMetadataNegativeTestJSON-1202925361-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8d3cada15a55453faaf23a147c44afcf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ecb79581f83f46f6a02fba932e20f3fe', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:51:52.325338 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-b69474f8-b39e-4e48-97c0-54921f16b12a tempest-ServerMetadataNegativeTestJSON-1202925361 tempest-ServerMetadataNegativeTestJSON-1202925361-project] 10.222.0.16 "GET /compute/v2.1/servers/c1c65e9e-289c-4707-b6f7-45fe851f4fae" status: 200 len: 1324 microversion: 2.1 time: 0.163321 Jul 16 21:51:52.325919 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3109/6223] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:51:52 2022] GET /compute/v2.1/servers/c1c65e9e-289c-4707-b6f7-45fe851f4fae => generated 1324 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:51:53.342807 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-872ca819-c178-4d76-97f0-a58417fdb4e8 tempest-ServerMetadataNegativeTestJSON-1202925361 tempest-ServerMetadataNegativeTestJSON-1202925361-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:51:53.343459 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-872ca819-c178-4d76-97f0-a58417fdb4e8 tempest-ServerMetadataNegativeTestJSON-1202925361 tempest-ServerMetadataNegativeTestJSON-1202925361-project] [instance: c1c65e9e-289c-4707-b6f7-45fe851f4fae] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:51:53.351689 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-872ca819-c178-4d76-97f0-a58417fdb4e8 tempest-ServerMetadataNegativeTestJSON-1202925361 tempest-ServerMetadataNegativeTestJSON-1202925361-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:51:53.352154 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-872ca819-c178-4d76-97f0-a58417fdb4e8 tempest-ServerMetadataNegativeTestJSON-1202925361 tempest-ServerMetadataNegativeTestJSON-1202925361-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:51:53.352979 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-872ca819-c178-4d76-97f0-a58417fdb4e8 tempest-ServerMetadataNegativeTestJSON-1202925361 tempest-ServerMetadataNegativeTestJSON-1202925361-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:51:53.353437 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-872ca819-c178-4d76-97f0-a58417fdb4e8 tempest-ServerMetadataNegativeTestJSON-1202925361 tempest-ServerMetadataNegativeTestJSON-1202925361-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:51:53.487539 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-872ca819-c178-4d76-97f0-a58417fdb4e8 tempest-ServerMetadataNegativeTestJSON-1202925361 tempest-ServerMetadataNegativeTestJSON-1202925361-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8d3cada15a55453faaf23a147c44afcf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ecb79581f83f46f6a02fba932e20f3fe', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:51:53.495561 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-872ca819-c178-4d76-97f0-a58417fdb4e8 tempest-ServerMetadataNegativeTestJSON-1202925361 tempest-ServerMetadataNegativeTestJSON-1202925361-project] 10.222.0.16 "GET /compute/v2.1/servers/c1c65e9e-289c-4707-b6f7-45fe851f4fae" status: 200 len: 1324 microversion: 2.1 time: 0.156857 Jul 16 21:51:53.496134 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3115/6224] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:51:53 2022] GET /compute/v2.1/servers/c1c65e9e-289c-4707-b6f7-45fe851f4fae => generated 1324 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:51:54.512650 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-79c7000a-f8c8-4f01-84d7-2483f33176e6 tempest-ServerMetadataNegativeTestJSON-1202925361 tempest-ServerMetadataNegativeTestJSON-1202925361-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:51:54.513269 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-79c7000a-f8c8-4f01-84d7-2483f33176e6 tempest-ServerMetadataNegativeTestJSON-1202925361 tempest-ServerMetadataNegativeTestJSON-1202925361-project] [instance: c1c65e9e-289c-4707-b6f7-45fe851f4fae] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:51:54.520533 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-79c7000a-f8c8-4f01-84d7-2483f33176e6 tempest-ServerMetadataNegativeTestJSON-1202925361 tempest-ServerMetadataNegativeTestJSON-1202925361-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:51:54.520987 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-79c7000a-f8c8-4f01-84d7-2483f33176e6 tempest-ServerMetadataNegativeTestJSON-1202925361 tempest-ServerMetadataNegativeTestJSON-1202925361-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:51:54.521761 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-79c7000a-f8c8-4f01-84d7-2483f33176e6 tempest-ServerMetadataNegativeTestJSON-1202925361 tempest-ServerMetadataNegativeTestJSON-1202925361-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:51:54.522207 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-79c7000a-f8c8-4f01-84d7-2483f33176e6 tempest-ServerMetadataNegativeTestJSON-1202925361 tempest-ServerMetadataNegativeTestJSON-1202925361-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:51:54.670255 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-79c7000a-f8c8-4f01-84d7-2483f33176e6 tempest-ServerMetadataNegativeTestJSON-1202925361 tempest-ServerMetadataNegativeTestJSON-1202925361-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8d3cada15a55453faaf23a147c44afcf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ecb79581f83f46f6a02fba932e20f3fe', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:51:54.678011 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-79c7000a-f8c8-4f01-84d7-2483f33176e6 tempest-ServerMetadataNegativeTestJSON-1202925361 tempest-ServerMetadataNegativeTestJSON-1202925361-project] 10.222.0.16 "GET /compute/v2.1/servers/c1c65e9e-289c-4707-b6f7-45fe851f4fae" status: 200 len: 1324 microversion: 2.1 time: 0.169456 Jul 16 21:51:54.678580 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3110/6225] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:51:54 2022] GET /compute/v2.1/servers/c1c65e9e-289c-4707-b6f7-45fe851f4fae => generated 1324 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:51:55.696043 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-6f499958-9b5e-435c-af8b-7bae385fc9c4 tempest-ServerMetadataNegativeTestJSON-1202925361 tempest-ServerMetadataNegativeTestJSON-1202925361-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:51:55.696702 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-6f499958-9b5e-435c-af8b-7bae385fc9c4 tempest-ServerMetadataNegativeTestJSON-1202925361 tempest-ServerMetadataNegativeTestJSON-1202925361-project] [instance: c1c65e9e-289c-4707-b6f7-45fe851f4fae] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:51:55.704309 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6f499958-9b5e-435c-af8b-7bae385fc9c4 tempest-ServerMetadataNegativeTestJSON-1202925361 tempest-ServerMetadataNegativeTestJSON-1202925361-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:51:55.704775 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6f499958-9b5e-435c-af8b-7bae385fc9c4 tempest-ServerMetadataNegativeTestJSON-1202925361 tempest-ServerMetadataNegativeTestJSON-1202925361-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:51:55.705557 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6f499958-9b5e-435c-af8b-7bae385fc9c4 tempest-ServerMetadataNegativeTestJSON-1202925361 tempest-ServerMetadataNegativeTestJSON-1202925361-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:51:55.706006 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6f499958-9b5e-435c-af8b-7bae385fc9c4 tempest-ServerMetadataNegativeTestJSON-1202925361 tempest-ServerMetadataNegativeTestJSON-1202925361-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:51:55.839639 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-6f499958-9b5e-435c-af8b-7bae385fc9c4 tempest-ServerMetadataNegativeTestJSON-1202925361 tempest-ServerMetadataNegativeTestJSON-1202925361-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8d3cada15a55453faaf23a147c44afcf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ecb79581f83f46f6a02fba932e20f3fe', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:51:55.847955 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-6f499958-9b5e-435c-af8b-7bae385fc9c4 tempest-ServerMetadataNegativeTestJSON-1202925361 tempest-ServerMetadataNegativeTestJSON-1202925361-project] 10.222.0.16 "GET /compute/v2.1/servers/c1c65e9e-289c-4707-b6f7-45fe851f4fae" status: 200 len: 1324 microversion: 2.1 time: 0.157089 Jul 16 21:51:55.858450 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3116/6226] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:51:55 2022] GET /compute/v2.1/servers/c1c65e9e-289c-4707-b6f7-45fe851f4fae => generated 1324 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:51:56.292790 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-0a02d5b8-e0f7-4cbb-8e6a-6b5eadedc5dd req-e910ca82-050a-4653-bd74-b48cc96c915f service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "c1c65e9e-289c-4707-b6f7-45fe851f4fae", "name": "network-vif-plugged", "status": "completed", "tag": "8100c1c9-4e96-49f0-a067-63ae803ab7be"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:51:56.301850 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-0a02d5b8-e0f7-4cbb-8e6a-6b5eadedc5dd req-e910ca82-050a-4653-bd74-b48cc96c915f service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:51:56.302301 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-0a02d5b8-e0f7-4cbb-8e6a-6b5eadedc5dd req-e910ca82-050a-4653-bd74-b48cc96c915f service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:51:56.313547 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-0a02d5b8-e0f7-4cbb-8e6a-6b5eadedc5dd req-e910ca82-050a-4653-bd74-b48cc96c915f service nova] Creating event network-vif-plugged:8100c1c9-4e96-49f0-a067-63ae803ab7be for instance c1c65e9e-289c-4707-b6f7-45fe851f4fae on nch1-master-1 Jul 16 21:51:56.318525 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-0a02d5b8-e0f7-4cbb-8e6a-6b5eadedc5dd req-e910ca82-050a-4653-bd74-b48cc96c915f service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.030472 Jul 16 21:51:56.319157 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3111/6227] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:51:56 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:51:56.865987 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-970c5f20-08ae-4b22-a870-1714731eaba2 tempest-ServerMetadataNegativeTestJSON-1202925361 tempest-ServerMetadataNegativeTestJSON-1202925361-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:51:56.866694 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-970c5f20-08ae-4b22-a870-1714731eaba2 tempest-ServerMetadataNegativeTestJSON-1202925361 tempest-ServerMetadataNegativeTestJSON-1202925361-project] [instance: c1c65e9e-289c-4707-b6f7-45fe851f4fae] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:51:56.871212 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell0:SELECT=2 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:51:56.875772 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-970c5f20-08ae-4b22-a870-1714731eaba2 tempest-ServerMetadataNegativeTestJSON-1202925361 tempest-ServerMetadataNegativeTestJSON-1202925361-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:51:56.876240 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-970c5f20-08ae-4b22-a870-1714731eaba2 tempest-ServerMetadataNegativeTestJSON-1202925361 tempest-ServerMetadataNegativeTestJSON-1202925361-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:51:56.877028 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-970c5f20-08ae-4b22-a870-1714731eaba2 tempest-ServerMetadataNegativeTestJSON-1202925361 tempest-ServerMetadataNegativeTestJSON-1202925361-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:51:56.877637 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-970c5f20-08ae-4b22-a870-1714731eaba2 tempest-ServerMetadataNegativeTestJSON-1202925361 tempest-ServerMetadataNegativeTestJSON-1202925361-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:51:57.013070 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-970c5f20-08ae-4b22-a870-1714731eaba2 tempest-ServerMetadataNegativeTestJSON-1202925361 tempest-ServerMetadataNegativeTestJSON-1202925361-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8d3cada15a55453faaf23a147c44afcf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ecb79581f83f46f6a02fba932e20f3fe', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:51:57.020705 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-970c5f20-08ae-4b22-a870-1714731eaba2 tempest-ServerMetadataNegativeTestJSON-1202925361 tempest-ServerMetadataNegativeTestJSON-1202925361-project] 10.222.0.16 "GET /compute/v2.1/servers/c1c65e9e-289c-4707-b6f7-45fe851f4fae" status: 200 len: 1324 microversion: 2.1 time: 0.158659 Jul 16 21:51:57.021279 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3117/6228] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:51:56 2022] GET /compute/v2.1/servers/c1c65e9e-289c-4707-b6f7-45fe851f4fae => generated 1324 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:51:58.035800 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell0:SELECT=2 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:51:58.041093 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-969be7da-79c3-424f-8171-e3d71b1ca56c tempest-ServerMetadataNegativeTestJSON-1202925361 tempest-ServerMetadataNegativeTestJSON-1202925361-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:51:58.041809 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-969be7da-79c3-424f-8171-e3d71b1ca56c tempest-ServerMetadataNegativeTestJSON-1202925361 tempest-ServerMetadataNegativeTestJSON-1202925361-project] [instance: c1c65e9e-289c-4707-b6f7-45fe851f4fae] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:51:58.048591 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-969be7da-79c3-424f-8171-e3d71b1ca56c tempest-ServerMetadataNegativeTestJSON-1202925361 tempest-ServerMetadataNegativeTestJSON-1202925361-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:51:58.049053 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-969be7da-79c3-424f-8171-e3d71b1ca56c tempest-ServerMetadataNegativeTestJSON-1202925361 tempest-ServerMetadataNegativeTestJSON-1202925361-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:51:58.049828 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-969be7da-79c3-424f-8171-e3d71b1ca56c tempest-ServerMetadataNegativeTestJSON-1202925361 tempest-ServerMetadataNegativeTestJSON-1202925361-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:51:58.050278 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-969be7da-79c3-424f-8171-e3d71b1ca56c tempest-ServerMetadataNegativeTestJSON-1202925361 tempest-ServerMetadataNegativeTestJSON-1202925361-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:51:58.197193 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-969be7da-79c3-424f-8171-e3d71b1ca56c tempest-ServerMetadataNegativeTestJSON-1202925361 tempest-ServerMetadataNegativeTestJSON-1202925361-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8d3cada15a55453faaf23a147c44afcf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ecb79581f83f46f6a02fba932e20f3fe', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:51:58.205986 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-969be7da-79c3-424f-8171-e3d71b1ca56c tempest-ServerMetadataNegativeTestJSON-1202925361 tempest-ServerMetadataNegativeTestJSON-1202925361-project] 10.222.0.16 "GET /compute/v2.1/servers/c1c65e9e-289c-4707-b6f7-45fe851f4fae" status: 200 len: 1520 microversion: 2.1 time: 0.171184 Jul 16 21:51:58.206589 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3112/6229] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:51:58 2022] GET /compute/v2.1/servers/c1c65e9e-289c-4707-b6f7-45fe851f4fae => generated 1520 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:51:58.319322 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-88fde721-1180-49a2-bced-82ce3b6886d3 tempest-ServerMetadataNegativeTestJSON-1202925361 tempest-ServerMetadataNegativeTestJSON-1202925361-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerMetadataNegativeTestJSON-server-365389387", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452", "metadata": {"": "data1"}, "networks": [{"uuid": "52901b70-e3ce-40b9-9a01-0d9a211da013"}]}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:51:58.322161 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-88fde721-1180-49a2-bced-82ce3b6886d3 tempest-ServerMetadataNegativeTestJSON-1202925361 tempest-ServerMetadataNegativeTestJSON-1202925361-project] Request method failure captured: Jul 16 21:51:58.322161 ncdv-master-1 devstack@n-api.service[138681]: request: POST /compute/v2.1/servers HTTP/1.1 Jul 16 21:51:58.322161 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:51:58.322161 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:51:58.322161 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:51:58.322161 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 242 Jul 16 21:51:58.322161 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:51:58.322161 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:51:58.322161 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:51:58.322161 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:51:58.322161 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0zLxzOplVHrs_HBQTEpuXRWANZiLYDoW82TFVevTZvvxbXxSdIS7ylDq68-X582trWyj8eL2Huzc3wP9CALwiQ1gr21xZhghWgku74rXV4Biu5GwzLBHQmKz5LX7GzFeshYs4UtF22ibFRiX6lmCLX7mITppYfmjCt4QO1m1QGIJpZc Jul 16 21:51:58.322161 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:51:58.322161 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:51:58.322161 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:51:58.322161 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:51:58.322161 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:51:58.322161 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:51:58.322161 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: ecb79581f83f46f6a02fba932e20f3fe Jul 16 21:51:58.322161 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-ServerMetadataNegativeTestJSON-1202925361 Jul 16 21:51:58.322161 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:51:58.322161 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:51:58.322161 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/ecb79581f83f46f6a02fba932e20f3fe"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/ecb79581f83f46f6a02fba932e20f3fe"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_ecb79581f83f46f6a02fba932e20f3fe"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/ecb79581f83f46f6a02fba932e20f3fe"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:51:58.322161 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-ServerMetadataNegativeTestJSON-1202925361 Jul 16 21:51:58.322161 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: ecb79581f83f46f6a02fba932e20f3fe Jul 16 21:51:58.322161 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-ServerMetadataNegativeTestJSON-1202925361 Jul 16 21:51:58.322161 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-ServerMetadataNegativeTestJSON-1202925361-project Jul 16 21:51:58.322161 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:51:58.322161 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:51:58.322161 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: 8d3cada15a55453faaf23a147c44afcf Jul 16 21:51:58.322161 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-ServerMetadataNegativeTestJSON-1202925361-project Jul 16 21:51:58.322161 ncdv-master-1 devstack@n-api.service[138681]: Jul 16 21:51:58.322161 ncdv-master-1 devstack@n-api.service[138681]: {"server": {"name": "tempest-ServerMetadataNegativeTestJSON-server-365389387", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452", "metadata": {"": "data1"}, "networks": [{"uuid": "52901b70-e3ce-40b9-9a01-0d9a211da013"}]}} Jul 16 21:51:58.322161 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:51:58.322161 ncdv-master-1 devstack@n-api.service[138681]: exception: Invalid input for field/attribute metadata. Value: {'': 'data1'}. '' does not match any of the regexes: '^[a-zA-Z0-9-_:. ]{1,255}$' Jul 16 21:51:58.322161 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'body': {'server': {'name': 'tempest-ServerMetadataNegativeTestJSON-server-365389387', 'imageRef': '1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9', 'flavorRef': '452', 'metadata': {'': 'data1'}, 'networks': [{'uuid': '52901b70-e3ce-40b9-9a01-0d9a211da013'}]}}} Jul 16 21:51:58.322161 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:51:58.322161 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:51:58.322161 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 279, in validate Jul 16 21:51:58.322161 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Jul 16 21:51:58.322161 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/lib/python3/dist-packages/jsonschema/validators.py", line 353, in validate Jul 16 21:51:58.322161 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise error Jul 16 21:51:58.322161 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: '' does not match any of the regexes: '^[a-zA-Z0-9-_:. ]{1,255}$' Jul 16 21:51:58.322161 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:51:58.322161 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Failed validating 'additionalProperties' in schema['properties']['server']['properties']['metadata']: Jul 16 21:51:58.322161 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi {'additionalProperties': False, Jul 16 21:51:58.322161 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi 'patternProperties': {'^[a-zA-Z0-9-_:. ]{1,255}$': {'maxLength': 255, Jul 16 21:51:58.322161 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi 'type': 'string'}}, Jul 16 21:51:58.322161 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi 'type': 'object'} Jul 16 21:51:58.322161 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:51:58.322161 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi On instance['server']: Jul 16 21:51:58.322161 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi {'': 'data1'} Jul 16 21:51:58.322161 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:51:58.322161 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:51:58.322161 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:51:58.322161 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:51:58.322161 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:51:58.322161 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:51:58.322161 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:51:58.322161 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:51:58.322161 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:51:58.322161 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:51:58.322161 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 16 21:51:58.322161 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 16 21:51:58.327772 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Jul 16 21:51:58.327772 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Jul 16 21:51:58.327772 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Jul 16 21:51:58.327772 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Jul 16 21:51:58.327772 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 308, in validate Jul 16 21:51:58.327772 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Jul 16 21:51:58.327772 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute metadata. Value: {'': 'data1'}. '' does not match any of the regexes: '^[a-zA-Z0-9-_:. ]{1,255}$' Jul 16 21:51:58.327772 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:51:58.327772 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:51:58.327772 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:51:58.327772 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:51:58.327772 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:51:58.327772 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:51:58.327772 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Jul 16 21:51:58.327772 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Jul 16 21:51:58.327772 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute metadata. Value: {'': 'data1'}. '' does not match any of the regexes: '^[a-zA-Z0-9-_:. ]{1,255}$' Jul 16 21:51:58.327772 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:51:58.327772 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-88fde721-1180-49a2-bced-82ce3b6886d3 tempest-ServerMetadataNegativeTestJSON-1202925361 tempest-ServerMetadataNegativeTestJSON-1202925361-project] Returning 400 to user: Invalid input for field/attribute metadata. Value: {'': 'data1'}. '' does not match any of the regexes: '^[a-zA-Z0-9-_:. ]{1,255}$' {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:51:58.327772 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-88fde721-1180-49a2-bced-82ce3b6886d3 tempest-ServerMetadataNegativeTestJSON-1202925361 tempest-ServerMetadataNegativeTestJSON-1202925361-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 400 len: 175 microversion: 2.1 time: 0.008420 Jul 16 21:51:58.327772 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3118/6230] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 21:51:58 2022] POST /compute/v2.1/servers => generated 175 bytes in 10 msecs (HTTP/1.1 400) 9 headers in 381 bytes (2 switches on core 0) Jul 16 21:51:58.334994 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-006b33b0-c6d6-4236-a15b-b3ea6192cffa tempest-ServerMetadataNegativeTestJSON-1202925361 tempest-ServerMetadataNegativeTestJSON-1202925361-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:51:58.335532 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-006b33b0-c6d6-4236-a15b-b3ea6192cffa tempest-ServerMetadataNegativeTestJSON-1202925361 tempest-ServerMetadataNegativeTestJSON-1202925361-project] [instance: 14095b59-d389-4358-93d2-7a41dddcb402] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:51:58.341157 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-006b33b0-c6d6-4236-a15b-b3ea6192cffa tempest-ServerMetadataNegativeTestJSON-1202925361 tempest-ServerMetadataNegativeTestJSON-1202925361-project] HTTP exception thrown: Instance 14095b59-d389-4358-93d2-7a41dddcb402 could not be found. Jul 16 21:51:58.344265 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-006b33b0-c6d6-4236-a15b-b3ea6192cffa tempest-ServerMetadataNegativeTestJSON-1202925361 tempest-ServerMetadataNegativeTestJSON-1202925361-project] Request method failure captured: Jul 16 21:51:58.344265 ncdv-master-1 devstack@n-api.service[138680]: request: GET /compute/v2.1/servers/14095b59-d389-4358-93d2-7a41dddcb402/metadata HTTP/1.1 Jul 16 21:51:58.344265 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:51:58.344265 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:51:58.344265 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:51:58.344265 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 0 Jul 16 21:51:58.344265 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:51:58.344265 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:51:58.344265 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:51:58.344265 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:51:58.344265 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0zLxzOplVHrs_HBQTEpuXRWANZiLYDoW82TFVevTZvvxbXxSdIS7ylDq68-X582trWyj8eL2Huzc3wP9CALwiQ1gr21xZhghWgku74rXV4Biu5GwzLBHQmKz5LX7GzFeshYs4UtF22ibFRiX6lmCLX7mITppYfmjCt4QO1m1QGIJpZc Jul 16 21:51:58.344265 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:51:58.344265 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:51:58.344265 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:51:58.344265 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:51:58.344265 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:51:58.344265 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:51:58.344265 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: ecb79581f83f46f6a02fba932e20f3fe Jul 16 21:51:58.344265 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-ServerMetadataNegativeTestJSON-1202925361 Jul 16 21:51:58.344265 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:51:58.344265 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:51:58.344265 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/ecb79581f83f46f6a02fba932e20f3fe"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/ecb79581f83f46f6a02fba932e20f3fe"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_ecb79581f83f46f6a02fba932e20f3fe"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/ecb79581f83f46f6a02fba932e20f3fe"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:51:58.344265 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-ServerMetadataNegativeTestJSON-1202925361 Jul 16 21:51:58.344265 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: ecb79581f83f46f6a02fba932e20f3fe Jul 16 21:51:58.344265 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-ServerMetadataNegativeTestJSON-1202925361 Jul 16 21:51:58.344265 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-ServerMetadataNegativeTestJSON-1202925361-project Jul 16 21:51:58.344265 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:51:58.344265 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:51:58.344265 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: 8d3cada15a55453faaf23a147c44afcf Jul 16 21:51:58.344265 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-ServerMetadataNegativeTestJSON-1202925361-project Jul 16 21:51:58.344265 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 21:51:58.344265 ncdv-master-1 devstack@n-api.service[138680]: exception: Instance 14095b59-d389-4358-93d2-7a41dddcb402 could not be found. Jul 16 21:51:58.344265 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'server_id': '14095b59-d389-4358-93d2-7a41dddcb402'} Jul 16 21:51:58.344265 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:51:58.344265 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:51:58.344265 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:51:58.344265 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:51:58.344265 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:51:58.344265 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:51:58.344265 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2898, in _get_instance Jul 16 21:51:58.344265 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=instance_uuid) Jul 16 21:51:58.344265 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 14095b59-d389-4358-93d2-7a41dddcb402 could not be found. Jul 16 21:51:58.344265 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:51:58.344265 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:51:58.344265 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:51:58.344265 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:51:58.344265 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:51:58.344265 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:51:58.344265 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:51:58.344265 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:51:58.344265 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:51:58.344265 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:51:58.344265 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/server_metadata.py", line 54, in index Jul 16 21:51:58.344265 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi server = common.get_instance(self.compute_api, context, server_id) Jul 16 21:51:58.344265 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:51:58.344265 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:51:58.344265 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 14095b59-d389-4358-93d2-7a41dddcb402 could not be found. Jul 16 21:51:58.344265 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:51:58.344265 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:51:58.344265 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:51:58.344265 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:51:58.344265 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:51:58.344265 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:51:58.349318 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:51:58.349318 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:51:58.349318 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 14095b59-d389-4358-93d2-7a41dddcb402 could not be found. Jul 16 21:51:58.349318 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:51:58.349318 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-006b33b0-c6d6-4236-a15b-b3ea6192cffa tempest-ServerMetadataNegativeTestJSON-1202925361 tempest-ServerMetadataNegativeTestJSON-1202925361-project] Returning 404 to user: Instance 14095b59-d389-4358-93d2-7a41dddcb402 could not be found. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:51:58.349318 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-006b33b0-c6d6-4236-a15b-b3ea6192cffa tempest-ServerMetadataNegativeTestJSON-1202925361 tempest-ServerMetadataNegativeTestJSON-1202925361-project] 10.222.0.16 "GET /compute/v2.1/servers/14095b59-d389-4358-93d2-7a41dddcb402/metadata" status: 404 len: 111 microversion: 2.1 time: 0.014704 Jul 16 21:51:58.349318 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3113/6231] 10.222.0.16 () {58 vars in 1262 bytes} [Sat Jul 16 21:51:58 2022] GET /compute/v2.1/servers/14095b59-d389-4358-93d2-7a41dddcb402/metadata => generated 111 bytes in 16 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:51:58.391335 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell0:SELECT=8 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:51:58.396589 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-1382edab-0af5-4ff8-bae3-37caf1acd883 tempest-ServerMetadataNegativeTestJSON-1202925361 tempest-ServerMetadataNegativeTestJSON-1202925361-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerMetadataNegativeTestJSON-server-673067414", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452", "metadata": {"kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk": "data1"}, "networks": [{"uuid": "52901b70-e3ce-40b9-9a01-0d9a211da013"}]}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:51:58.400194 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-1382edab-0af5-4ff8-bae3-37caf1acd883 tempest-ServerMetadataNegativeTestJSON-1202925361 tempest-ServerMetadataNegativeTestJSON-1202925361-project] Request method failure captured: Jul 16 21:51:58.400194 ncdv-master-1 devstack@n-api.service[138681]: request: POST /compute/v2.1/servers HTTP/1.1 Jul 16 21:51:58.400194 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:51:58.400194 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:51:58.400194 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:51:58.400194 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 498 Jul 16 21:51:58.400194 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:51:58.400194 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:51:58.400194 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:51:58.400194 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:51:58.400194 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0zLxzOplVHrs_HBQTEpuXRWANZiLYDoW82TFVevTZvvxbXxSdIS7ylDq68-X582trWyj8eL2Huzc3wP9CALwiQ1gr21xZhghWgku74rXV4Biu5GwzLBHQmKz5LX7GzFeshYs4UtF22ibFRiX6lmCLX7mITppYfmjCt4QO1m1QGIJpZc Jul 16 21:51:58.400194 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:51:58.400194 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:51:58.400194 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:51:58.400194 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:51:58.400194 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:51:58.400194 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:51:58.400194 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: ecb79581f83f46f6a02fba932e20f3fe Jul 16 21:51:58.400194 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-ServerMetadataNegativeTestJSON-1202925361 Jul 16 21:51:58.400194 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:51:58.400194 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:51:58.400194 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/ecb79581f83f46f6a02fba932e20f3fe"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/ecb79581f83f46f6a02fba932e20f3fe"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_ecb79581f83f46f6a02fba932e20f3fe"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/ecb79581f83f46f6a02fba932e20f3fe"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:51:58.400194 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-ServerMetadataNegativeTestJSON-1202925361 Jul 16 21:51:58.400194 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: ecb79581f83f46f6a02fba932e20f3fe Jul 16 21:51:58.400194 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-ServerMetadataNegativeTestJSON-1202925361 Jul 16 21:51:58.400194 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-ServerMetadataNegativeTestJSON-1202925361-project Jul 16 21:51:58.400194 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:51:58.400194 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:51:58.400194 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: 8d3cada15a55453faaf23a147c44afcf Jul 16 21:51:58.400194 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-ServerMetadataNegativeTestJSON-1202925361-project Jul 16 21:51:58.400194 ncdv-master-1 devstack@n-api.service[138681]: Jul 16 21:51:58.400194 ncdv-master-1 devstack@n-api.service[138681]: {"server": {"name": "tempest-ServerMetadataNegativeTestJSON-server-673067414", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452", "metadata": {"kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk": "data1"}, "networks": [{"uuid": "52901b70-e3ce-40b9-9a01-0d9a211da013"}]}} Jul 16 21:51:58.400194 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:51:58.400194 ncdv-master-1 devstack@n-api.service[138681]: exception: Invalid input for field/attribute metadata. Value: {'kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk': 'data1'}. 'kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk' does not match any of the regexes: '^[a-zA-Z0-9-_:. ]{1,255}$' Jul 16 21:51:58.400194 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'body': {'server': {'name': 'tempest-ServerMetadataNegativeTestJSON-server-673067414', 'imageRef': '1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9', 'flavorRef': '452', 'metadata': {'kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk': 'data1'}, 'networks': [{'uuid': '52901b70-e3ce-40b9-9a01-0d9a211da013'}]}}} Jul 16 21:51:58.400194 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:51:58.400194 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:51:58.400194 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 279, in validate Jul 16 21:51:58.400194 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Jul 16 21:51:58.400194 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/lib/python3/dist-packages/jsonschema/validators.py", line 353, in validate Jul 16 21:51:58.400194 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise error Jul 16 21:51:58.400194 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: 'kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk' does not match any of the regexes: '^[a-zA-Z0-9-_:. ]{1,255}$' Jul 16 21:51:58.400194 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:51:58.400194 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Failed validating 'additionalProperties' in schema['properties']['server']['properties']['metadata']: Jul 16 21:51:58.400194 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi {'additionalProperties': False, Jul 16 21:51:58.400194 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi 'patternProperties': {'^[a-zA-Z0-9-_:. ]{1,255}$': {'maxLength': 255, Jul 16 21:51:58.400194 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi 'type': 'string'}}, Jul 16 21:51:58.400194 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi 'type': 'object'} Jul 16 21:51:58.403833 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:51:58.403833 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi On instance['server']: Jul 16 21:51:58.403833 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi {'kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk': 'data1'} Jul 16 21:51:58.403833 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:51:58.403833 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:51:58.403833 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:51:58.403833 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:51:58.403833 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:51:58.403833 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:51:58.403833 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:51:58.403833 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:51:58.403833 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:51:58.403833 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:51:58.403833 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 16 21:51:58.403833 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 16 21:51:58.403833 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Jul 16 21:51:58.403833 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Jul 16 21:51:58.403833 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Jul 16 21:51:58.403833 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Jul 16 21:51:58.403833 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 308, in validate Jul 16 21:51:58.403833 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Jul 16 21:51:58.403833 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute metadata. Value: {'kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk': 'data1'}. 'kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk' does not match any of the regexes: '^[a-zA-Z0-9-_:. ]{1,255}$' Jul 16 21:51:58.403833 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:51:58.403833 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:51:58.403833 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:51:58.403833 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:51:58.403833 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:51:58.403833 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:51:58.403833 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Jul 16 21:51:58.403833 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Jul 16 21:51:58.403833 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute metadata. Value: {'kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk': 'data1'}. 'kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk' does not match any of the regexes: '^[a-zA-Z0-9-_:. ]{1,255}$' Jul 16 21:51:58.403833 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:51:58.403833 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-1382edab-0af5-4ff8-bae3-37caf1acd883 tempest-ServerMetadataNegativeTestJSON-1202925361 tempest-ServerMetadataNegativeTestJSON-1202925361-project] Returning 400 to user: Invalid input for field/attribute metadata. Value: {'kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk': 'data1'}. 'kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk' does not match any of the regexes: '^[a-zA-Z0-9-_:. ]{1,255}$' {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:51:58.403833 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-1382edab-0af5-4ff8-bae3-37caf1acd883 tempest-ServerMetadataNegativeTestJSON-1202925361 tempest-ServerMetadataNegativeTestJSON-1202925361-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 400 len: 687 microversion: 2.1 time: 0.011244 Jul 16 21:51:58.403833 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3119/6232] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 21:51:58 2022] POST /compute/v2.1/servers => generated 687 bytes in 12 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Jul 16 21:51:58.454730 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-607b8aff-d6ed-4d28-8232-344debda0d9a tempest-ServerMetadataNegativeTestJSON-1202925361 tempest-ServerMetadataNegativeTestJSON-1202925361-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerMetadataNegativeTestJSON-server-1360964732", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452", "metadata": {"kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk": "data1"}, "networks": [{"uuid": "52901b70-e3ce-40b9-9a01-0d9a211da013"}]}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:51:58.457711 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-607b8aff-d6ed-4d28-8232-344debda0d9a tempest-ServerMetadataNegativeTestJSON-1202925361 tempest-ServerMetadataNegativeTestJSON-1202925361-project] Request method failure captured: Jul 16 21:51:58.457711 ncdv-master-1 devstack@n-api.service[138680]: request: POST /compute/v2.1/servers HTTP/1.1 Jul 16 21:51:58.457711 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:51:58.457711 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:51:58.457711 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:51:58.457711 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 500 Jul 16 21:51:58.457711 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:51:58.457711 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:51:58.457711 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:51:58.457711 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:51:58.457711 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0zLxzOplVHrs_HBQTEpuXRWANZiLYDoW82TFVevTZvvxbXxSdIS7ylDq68-X582trWyj8eL2Huzc3wP9CALwiQ1gr21xZhghWgku74rXV4Biu5GwzLBHQmKz5LX7GzFeshYs4UtF22ibFRiX6lmCLX7mITppYfmjCt4QO1m1QGIJpZc Jul 16 21:51:58.457711 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:51:58.457711 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:51:58.457711 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:51:58.457711 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:51:58.457711 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:51:58.457711 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:51:58.457711 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: ecb79581f83f46f6a02fba932e20f3fe Jul 16 21:51:58.457711 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-ServerMetadataNegativeTestJSON-1202925361 Jul 16 21:51:58.457711 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:51:58.457711 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:51:58.457711 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/ecb79581f83f46f6a02fba932e20f3fe"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/ecb79581f83f46f6a02fba932e20f3fe"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_ecb79581f83f46f6a02fba932e20f3fe"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/ecb79581f83f46f6a02fba932e20f3fe"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:51:58.457711 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-ServerMetadataNegativeTestJSON-1202925361 Jul 16 21:51:58.457711 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: ecb79581f83f46f6a02fba932e20f3fe Jul 16 21:51:58.457711 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-ServerMetadataNegativeTestJSON-1202925361 Jul 16 21:51:58.457711 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-ServerMetadataNegativeTestJSON-1202925361-project Jul 16 21:51:58.457711 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:51:58.457711 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:51:58.457711 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: 8d3cada15a55453faaf23a147c44afcf Jul 16 21:51:58.457711 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-ServerMetadataNegativeTestJSON-1202925361-project Jul 16 21:51:58.457711 ncdv-master-1 devstack@n-api.service[138680]: Jul 16 21:51:58.457711 ncdv-master-1 devstack@n-api.service[138680]: {"server": {"name": "tempest-ServerMetadataNegativeTestJSON-server-1360964732", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452", "metadata": {"kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk": "data1"}, "networks": [{"uuid": "52901b70-e3ce-40b9-9a01-0d9a211da013"}]}} Jul 16 21:51:58.457711 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 21:51:58.457711 ncdv-master-1 devstack@n-api.service[138680]: exception: Invalid input for field/attribute metadata. Value: {'kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk': 'data1'}. 'kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk' does not match any of the regexes: '^[a-zA-Z0-9-_:. ]{1,255}$' Jul 16 21:51:58.457711 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'body': {'server': {'name': 'tempest-ServerMetadataNegativeTestJSON-server-1360964732', 'imageRef': '1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9', 'flavorRef': '452', 'metadata': {'kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk': 'data1'}, 'networks': [{'uuid': '52901b70-e3ce-40b9-9a01-0d9a211da013'}]}}} Jul 16 21:51:58.457711 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:51:58.457711 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:51:58.457711 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 279, in validate Jul 16 21:51:58.457711 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Jul 16 21:51:58.457711 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/lib/python3/dist-packages/jsonschema/validators.py", line 353, in validate Jul 16 21:51:58.457711 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise error Jul 16 21:51:58.457711 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: 'kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk' does not match any of the regexes: '^[a-zA-Z0-9-_:. ]{1,255}$' Jul 16 21:51:58.457711 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:51:58.457711 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Failed validating 'additionalProperties' in schema['properties']['server']['properties']['metadata']: Jul 16 21:51:58.457711 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi {'additionalProperties': False, Jul 16 21:51:58.457711 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi 'patternProperties': {'^[a-zA-Z0-9-_:. ]{1,255}$': {'maxLength': 255, Jul 16 21:51:58.457711 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi 'type': 'string'}}, Jul 16 21:51:58.460503 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi 'type': 'object'} Jul 16 21:51:58.460503 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:51:58.460503 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi On instance['server']: Jul 16 21:51:58.460503 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi {'kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk': 'data1'} Jul 16 21:51:58.460503 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:51:58.460503 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:51:58.460503 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:51:58.460503 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:51:58.460503 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:51:58.460503 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:51:58.460503 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:51:58.460503 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:51:58.460503 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:51:58.460503 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:51:58.460503 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 16 21:51:58.460503 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 16 21:51:58.460503 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Jul 16 21:51:58.460503 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Jul 16 21:51:58.460503 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Jul 16 21:51:58.460503 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Jul 16 21:51:58.460503 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 308, in validate Jul 16 21:51:58.460503 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Jul 16 21:51:58.460503 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute metadata. Value: {'kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk': 'data1'}. 'kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk' does not match any of the regexes: '^[a-zA-Z0-9-_:. ]{1,255}$' Jul 16 21:51:58.460503 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:51:58.460503 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:51:58.460503 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:51:58.460503 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:51:58.460503 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:51:58.460503 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:51:58.460503 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Jul 16 21:51:58.460503 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Jul 16 21:51:58.460503 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute metadata. Value: {'kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk': 'data1'}. 'kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk' does not match any of the regexes: '^[a-zA-Z0-9-_:. ]{1,255}$' Jul 16 21:51:58.460503 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:51:58.460503 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-607b8aff-d6ed-4d28-8232-344debda0d9a tempest-ServerMetadataNegativeTestJSON-1202925361 tempest-ServerMetadataNegativeTestJSON-1202925361-project] Returning 400 to user: Invalid input for field/attribute metadata. Value: {'kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk': 'data1'}. 'kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk' does not match any of the regexes: '^[a-zA-Z0-9-_:. ]{1,255}$' {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:51:58.460503 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-607b8aff-d6ed-4d28-8232-344debda0d9a tempest-ServerMetadataNegativeTestJSON-1202925361 tempest-ServerMetadataNegativeTestJSON-1202925361-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 400 len: 689 microversion: 2.1 time: 0.008376 Jul 16 21:51:58.460503 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3114/6233] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 21:51:58 2022] POST /compute/v2.1/servers => generated 689 bytes in 10 msecs (HTTP/1.1 400) 9 headers in 381 bytes (2 switches on core 0) Jul 16 21:51:58.507453 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-2b9cd000-76b0-4f3e-b830-ccde9c5731ea tempest-ServerMetadataNegativeTestJSON-1202925361 tempest-ServerMetadataNegativeTestJSON-1202925361-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerMetadataNegativeTestJSON-server-1768035064", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452", "metadata": {"kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk": "data1"}, "networks": [{"uuid": "52901b70-e3ce-40b9-9a01-0d9a211da013"}]}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:51:58.510378 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-2b9cd000-76b0-4f3e-b830-ccde9c5731ea tempest-ServerMetadataNegativeTestJSON-1202925361 tempest-ServerMetadataNegativeTestJSON-1202925361-project] Request method failure captured: Jul 16 21:51:58.510378 ncdv-master-1 devstack@n-api.service[138681]: request: POST /compute/v2.1/servers HTTP/1.1 Jul 16 21:51:58.510378 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:51:58.510378 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:51:58.510378 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:51:58.510378 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 754 Jul 16 21:51:58.510378 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:51:58.510378 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:51:58.510378 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:51:58.510378 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:51:58.510378 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0zLxzOplVHrs_HBQTEpuXRWANZiLYDoW82TFVevTZvvxbXxSdIS7ylDq68-X582trWyj8eL2Huzc3wP9CALwiQ1gr21xZhghWgku74rXV4Biu5GwzLBHQmKz5LX7GzFeshYs4UtF22ibFRiX6lmCLX7mITppYfmjCt4QO1m1QGIJpZc Jul 16 21:51:58.510378 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:51:58.510378 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:51:58.510378 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:51:58.510378 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:51:58.510378 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:51:58.510378 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:51:58.510378 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: ecb79581f83f46f6a02fba932e20f3fe Jul 16 21:51:58.510378 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-ServerMetadataNegativeTestJSON-1202925361 Jul 16 21:51:58.510378 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:51:58.510378 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:51:58.510378 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/ecb79581f83f46f6a02fba932e20f3fe"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/ecb79581f83f46f6a02fba932e20f3fe"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_ecb79581f83f46f6a02fba932e20f3fe"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/ecb79581f83f46f6a02fba932e20f3fe"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:51:58.510378 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-ServerMetadataNegativeTestJSON-1202925361 Jul 16 21:51:58.510378 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: ecb79581f83f46f6a02fba932e20f3fe Jul 16 21:51:58.510378 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-ServerMetadataNegativeTestJSON-1202925361 Jul 16 21:51:58.510378 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-ServerMetadataNegativeTestJSON-1202925361-project Jul 16 21:51:58.510378 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:51:58.510378 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:51:58.510378 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: 8d3cada15a55453faaf23a147c44afcf Jul 16 21:51:58.510378 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-ServerMetadataNegativeTestJSON-1202925361-project Jul 16 21:51:58.510378 ncdv-master-1 devstack@n-api.service[138681]: Jul 16 21:51:58.510378 ncdv-master-1 devstack@n-api.service[138681]: {"server": {"name": "tempest-ServerMetadataNegativeTestJSON-server-1768035064", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452", "metadata": {"kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk": "data1"}, "networks": [{"uuid": "52901b70-e3ce-40b9-9a01-0d9a211da013"}]}} Jul 16 21:51:58.510378 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:51:58.510378 ncdv-master-1 devstack@n-api.service[138681]: exception: Invalid input for field/attribute metadata. Value: {'kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk': 'data1'}. 'kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk' does not match any of the regexes: '^[a-zA-Z0-9-_:. ]{1,255}$' Jul 16 21:51:58.510378 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'body': {'server': {'name': 'tempest-ServerMetadataNegativeTestJSON-server-1768035064', 'imageRef': '1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9', 'flavorRef': '452', 'metadata': {'kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk': 'data1'}, 'networks': [{'uuid': '52901b70-e3ce-40b9-9a01-0d9a211da013'}]}}} Jul 16 21:51:58.510378 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:51:58.510378 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:51:58.510378 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 279, in validate Jul 16 21:51:58.510378 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Jul 16 21:51:58.510378 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/lib/python3/dist-packages/jsonschema/validators.py", line 353, in validate Jul 16 21:51:58.513568 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise error Jul 16 21:51:58.513568 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: 'kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk' does not match any of the regexes: '^[a-zA-Z0-9-_:. ]{1,255}$' Jul 16 21:51:58.513568 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:51:58.513568 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Failed validating 'additionalProperties' in schema['properties']['server']['properties']['metadata']: Jul 16 21:51:58.513568 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi {'additionalProperties': False, Jul 16 21:51:58.513568 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi 'patternProperties': {'^[a-zA-Z0-9-_:. ]{1,255}$': {'maxLength': 255, Jul 16 21:51:58.513568 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi 'type': 'string'}}, Jul 16 21:51:58.513568 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi 'type': 'object'} Jul 16 21:51:58.513568 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:51:58.513568 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi On instance['server']: Jul 16 21:51:58.513568 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi {'kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk': 'data1'} Jul 16 21:51:58.513568 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:51:58.513568 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:51:58.513568 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:51:58.513568 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:51:58.513568 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:51:58.513568 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:51:58.513568 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:51:58.513568 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:51:58.513568 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:51:58.513568 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:51:58.513568 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 16 21:51:58.513568 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 16 21:51:58.513568 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Jul 16 21:51:58.513568 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Jul 16 21:51:58.513568 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Jul 16 21:51:58.513568 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Jul 16 21:51:58.513568 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 308, in validate Jul 16 21:51:58.513568 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Jul 16 21:51:58.513568 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute metadata. Value: {'kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk': 'data1'}. 'kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk' does not match any of the regexes: '^[a-zA-Z0-9-_:. ]{1,255}$' Jul 16 21:51:58.513568 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:51:58.513568 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:51:58.513568 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:51:58.513568 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:51:58.513568 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:51:58.513568 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:51:58.513568 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Jul 16 21:51:58.513568 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Jul 16 21:51:58.516070 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute metadata. Value: {'kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk': 'data1'}. 'kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk' does not match any of the regexes: '^[a-zA-Z0-9-_:. ]{1,255}$' Jul 16 21:51:58.516070 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:51:58.516070 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-2b9cd000-76b0-4f3e-b830-ccde9c5731ea tempest-ServerMetadataNegativeTestJSON-1202925361 tempest-ServerMetadataNegativeTestJSON-1202925361-project] Returning 400 to user: Invalid input for field/attribute metadata. Value: {'kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk': 'data1'}. 'kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk' does not match any of the regexes: '^[a-zA-Z0-9-_:. ]{1,255}$' {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:51:58.516070 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-2b9cd000-76b0-4f3e-b830-ccde9c5731ea tempest-ServerMetadataNegativeTestJSON-1202925361 tempest-ServerMetadataNegativeTestJSON-1202925361-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 400 len: 1197 microversion: 2.1 time: 0.009112 Jul 16 21:51:58.516070 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3120/6234] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 21:51:58 2022] POST /compute/v2.1/servers => generated 1197 bytes in 10 msecs (HTTP/1.1 400) 9 headers in 382 bytes (1 switches on core 0) Jul 16 21:51:58.559073 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-7874759a-7391-4b12-a512-c177650d2caf tempest-ServerMetadataNegativeTestJSON-1202925361 tempest-ServerMetadataNegativeTestJSON-1202925361-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerMetadataNegativeTestJSON-server-153278711", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452", "metadata": {"kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk": "data1"}, "networks": [{"uuid": "52901b70-e3ce-40b9-9a01-0d9a211da013"}]}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:51:58.561973 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-7874759a-7391-4b12-a512-c177650d2caf tempest-ServerMetadataNegativeTestJSON-1202925361 tempest-ServerMetadataNegativeTestJSON-1202925361-project] Request method failure captured: Jul 16 21:51:58.561973 ncdv-master-1 devstack@n-api.service[138680]: request: POST /compute/v2.1/servers HTTP/1.1 Jul 16 21:51:58.561973 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:51:58.561973 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:51:58.561973 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:51:58.561973 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 1265 Jul 16 21:51:58.561973 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:51:58.561973 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:51:58.561973 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:51:58.561973 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:51:58.561973 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0zLxzOplVHrs_HBQTEpuXRWANZiLYDoW82TFVevTZvvxbXxSdIS7ylDq68-X582trWyj8eL2Huzc3wP9CALwiQ1gr21xZhghWgku74rXV4Biu5GwzLBHQmKz5LX7GzFeshYs4UtF22ibFRiX6lmCLX7mITppYfmjCt4QO1m1QGIJpZc Jul 16 21:51:58.561973 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:51:58.561973 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:51:58.561973 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:51:58.561973 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:51:58.561973 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:51:58.561973 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:51:58.561973 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: ecb79581f83f46f6a02fba932e20f3fe Jul 16 21:51:58.561973 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-ServerMetadataNegativeTestJSON-1202925361 Jul 16 21:51:58.561973 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:51:58.561973 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:51:58.561973 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/ecb79581f83f46f6a02fba932e20f3fe"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/ecb79581f83f46f6a02fba932e20f3fe"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_ecb79581f83f46f6a02fba932e20f3fe"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/ecb79581f83f46f6a02fba932e20f3fe"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:51:58.561973 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-ServerMetadataNegativeTestJSON-1202925361 Jul 16 21:51:58.561973 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: ecb79581f83f46f6a02fba932e20f3fe Jul 16 21:51:58.561973 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-ServerMetadataNegativeTestJSON-1202925361 Jul 16 21:51:58.561973 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-ServerMetadataNegativeTestJSON-1202925361-project Jul 16 21:51:58.561973 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:51:58.561973 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:51:58.561973 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: 8d3cada15a55453faaf23a147c44afcf Jul 16 21:51:58.561973 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-ServerMetadataNegativeTestJSON-1202925361-project Jul 16 21:51:58.561973 ncdv-master-1 devstack@n-api.service[138680]: Jul 16 21:51:58.561973 ncdv-master-1 devstack@n-api.service[138680]: {"server": {"name": "tempest-ServerMetadataNegativeTestJSON-server-153278711", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452", "metadata": {"kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk": "data1"}, "networks": [{"uuid": "52901b70-e3ce-40b9-9a01-0d9a211da013"}]}} Jul 16 21:51:58.561973 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 21:51:58.564248 ncdv-master-1 devstack@n-api.service[138680]: exception: Invalid input for field/attribute metadata. Value: {'kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk': 'data1'}. 'kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk' does not match any of the regexes: '^[a-zA-Z0-9-_:. ]{1,255}$' Jul 16 21:51:58.564248 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'body': {'server': {'name': 'tempest-ServerMetadataNegativeTestJSON-server-153278711', 'imageRef': '1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9', 'flavorRef': '452', 'metadata': {'kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk': 'data1'}, 'networks': [{'uuid': '52901b70-e3ce-40b9-9a01-0d9a211da013'}]}}} Jul 16 21:51:58.564248 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:51:58.564248 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:51:58.564248 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 279, in validate Jul 16 21:51:58.564248 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Jul 16 21:51:58.564248 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/lib/python3/dist-packages/jsonschema/validators.py", line 353, in validate Jul 16 21:51:58.564248 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise error Jul 16 21:51:58.564248 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: 'kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk' does not match any of the regexes: '^[a-zA-Z0-9-_:. ]{1,255}$' Jul 16 21:51:58.564248 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:51:58.564248 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Failed validating 'additionalProperties' in schema['properties']['server']['properties']['metadata']: Jul 16 21:51:58.564248 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi {'additionalProperties': False, Jul 16 21:51:58.564248 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi 'patternProperties': {'^[a-zA-Z0-9-_:. ]{1,255}$': {'maxLength': 255, Jul 16 21:51:58.564248 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi 'type': 'string'}}, Jul 16 21:51:58.564248 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi 'type': 'object'} Jul 16 21:51:58.564248 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:51:58.564248 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi On instance['server']: Jul 16 21:51:58.565332 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi {'kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk': 'data1'} Jul 16 21:51:58.565332 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:51:58.565332 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:51:58.565332 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:51:58.565332 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:51:58.565332 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:51:58.565332 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:51:58.565332 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:51:58.565332 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:51:58.565332 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:51:58.565332 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:51:58.565332 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 16 21:51:58.565332 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 16 21:51:58.565332 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Jul 16 21:51:58.565332 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Jul 16 21:51:58.565332 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Jul 16 21:51:58.565332 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Jul 16 21:51:58.565332 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 308, in validate Jul 16 21:51:58.565332 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Jul 16 21:51:58.565332 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute metadata. Value: {'kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk': 'data1'}. 'kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk' does not match any of the regexes: '^[a-zA-Z0-9-_:. ]{1,255}$' Jul 16 21:51:58.565332 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:51:58.565332 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:51:58.565332 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:51:58.565332 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:51:58.565332 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:51:58.565332 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:51:58.565332 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Jul 16 21:51:58.565332 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Jul 16 21:51:58.567317 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute metadata. Value: {'kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk': 'data1'}. 'kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk' does not match any of the regexes: '^[a-zA-Z0-9-_:. ]{1,255}$' Jul 16 21:51:58.567317 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:51:58.567317 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-7874759a-7391-4b12-a512-c177650d2caf tempest-ServerMetadataNegativeTestJSON-1202925361 tempest-ServerMetadataNegativeTestJSON-1202925361-project] Returning 400 to user: Invalid input for field/attribute metadata. Value: {'kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk': 'data1'}. 'kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk' does not match any of the regexes: '^[a-zA-Z0-9-_:. ]{1,255}$' {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:51:58.567317 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-7874759a-7391-4b12-a512-c177650d2caf tempest-ServerMetadataNegativeTestJSON-1202925361 tempest-ServerMetadataNegativeTestJSON-1202925361-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 400 len: 2221 microversion: 2.1 time: 0.009954 Jul 16 21:51:58.567317 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3115/6235] 10.222.0.16 () {60 vars in 1147 bytes} [Sat Jul 16 21:51:58 2022] POST /compute/v2.1/servers => generated 2221 bytes in 11 msecs (HTTP/1.1 400) 9 headers in 382 bytes (1 switches on core 0) Jul 16 21:51:58.576244 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-74b50293-f85f-44eb-bc6c-fb0c55b0f9e3 tempest-ServerMetadataNegativeTestJSON-1202925361 tempest-ServerMetadataNegativeTestJSON-1202925361-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:51:58.576753 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-74b50293-f85f-44eb-bc6c-fb0c55b0f9e3 tempest-ServerMetadataNegativeTestJSON-1202925361 tempest-ServerMetadataNegativeTestJSON-1202925361-project] [instance: bd977bc6-5327-47e4-bbbd-18d25f068513] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:51:58.583469 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-74b50293-f85f-44eb-bc6c-fb0c55b0f9e3 tempest-ServerMetadataNegativeTestJSON-1202925361 tempest-ServerMetadataNegativeTestJSON-1202925361-project] HTTP exception thrown: Instance bd977bc6-5327-47e4-bbbd-18d25f068513 could not be found. Jul 16 21:51:58.585495 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-74b50293-f85f-44eb-bc6c-fb0c55b0f9e3 tempest-ServerMetadataNegativeTestJSON-1202925361 tempest-ServerMetadataNegativeTestJSON-1202925361-project] Request method failure captured: Jul 16 21:51:58.585495 ncdv-master-1 devstack@n-api.service[138681]: request: GET /compute/v2.1/servers/bd977bc6-5327-47e4-bbbd-18d25f068513/metadata/test2 HTTP/1.1 Jul 16 21:51:58.585495 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:51:58.585495 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:51:58.585495 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:51:58.585495 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 21:51:58.585495 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:51:58.585495 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:51:58.585495 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:51:58.585495 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:51:58.585495 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0zLxzOplVHrs_HBQTEpuXRWANZiLYDoW82TFVevTZvvxbXxSdIS7ylDq68-X582trWyj8eL2Huzc3wP9CALwiQ1gr21xZhghWgku74rXV4Biu5GwzLBHQmKz5LX7GzFeshYs4UtF22ibFRiX6lmCLX7mITppYfmjCt4QO1m1QGIJpZc Jul 16 21:51:58.585495 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:51:58.585495 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:51:58.585495 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:51:58.585495 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:51:58.585495 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:51:58.585495 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:51:58.585495 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: ecb79581f83f46f6a02fba932e20f3fe Jul 16 21:51:58.585495 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-ServerMetadataNegativeTestJSON-1202925361 Jul 16 21:51:58.585495 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:51:58.585495 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:51:58.585495 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/ecb79581f83f46f6a02fba932e20f3fe"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/ecb79581f83f46f6a02fba932e20f3fe"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_ecb79581f83f46f6a02fba932e20f3fe"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/ecb79581f83f46f6a02fba932e20f3fe"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:51:58.585495 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-ServerMetadataNegativeTestJSON-1202925361 Jul 16 21:51:58.585495 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: ecb79581f83f46f6a02fba932e20f3fe Jul 16 21:51:58.585495 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-ServerMetadataNegativeTestJSON-1202925361 Jul 16 21:51:58.585495 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-ServerMetadataNegativeTestJSON-1202925361-project Jul 16 21:51:58.585495 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:51:58.585495 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:51:58.585495 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: 8d3cada15a55453faaf23a147c44afcf Jul 16 21:51:58.585495 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-ServerMetadataNegativeTestJSON-1202925361-project Jul 16 21:51:58.585495 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:51:58.585495 ncdv-master-1 devstack@n-api.service[138681]: exception: Instance bd977bc6-5327-47e4-bbbd-18d25f068513 could not be found. Jul 16 21:51:58.585495 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'server_id': 'bd977bc6-5327-47e4-bbbd-18d25f068513', 'id': 'test2'} Jul 16 21:51:58.585495 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:51:58.585495 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:51:58.585495 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:51:58.585495 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:51:58.585495 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:51:58.585495 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:51:58.585495 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2898, in _get_instance Jul 16 21:51:58.585495 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=instance_uuid) Jul 16 21:51:58.585495 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance bd977bc6-5327-47e4-bbbd-18d25f068513 could not be found. Jul 16 21:51:58.585495 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:51:58.585495 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:51:58.585495 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:51:58.585495 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:51:58.585495 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:51:58.585495 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:51:58.585495 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:51:58.585495 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:51:58.585495 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:51:58.585495 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:51:58.585495 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/server_metadata.py", line 129, in show Jul 16 21:51:58.585495 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi server = common.get_instance(self.compute_api, context, server_id) Jul 16 21:51:58.585495 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:51:58.585495 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:51:58.585495 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance bd977bc6-5327-47e4-bbbd-18d25f068513 could not be found. Jul 16 21:51:58.585495 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:51:58.585495 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:51:58.585495 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:51:58.585495 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:51:58.585495 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:51:58.585495 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:51:58.589407 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:51:58.589407 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:51:58.589407 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance bd977bc6-5327-47e4-bbbd-18d25f068513 could not be found. Jul 16 21:51:58.589407 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:51:58.589407 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-74b50293-f85f-44eb-bc6c-fb0c55b0f9e3 tempest-ServerMetadataNegativeTestJSON-1202925361 tempest-ServerMetadataNegativeTestJSON-1202925361-project] Returning 404 to user: Instance bd977bc6-5327-47e4-bbbd-18d25f068513 could not be found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:51:58.589407 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-74b50293-f85f-44eb-bc6c-fb0c55b0f9e3 tempest-ServerMetadataNegativeTestJSON-1202925361 tempest-ServerMetadataNegativeTestJSON-1202925361-project] 10.222.0.16 "GET /compute/v2.1/servers/bd977bc6-5327-47e4-bbbd-18d25f068513/metadata/test2" status: 404 len: 111 microversion: 2.1 time: 0.016420 Jul 16 21:51:58.589407 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3121/6236] 10.222.0.16 () {58 vars in 1280 bytes} [Sat Jul 16 21:51:58 2022] GET /compute/v2.1/servers/bd977bc6-5327-47e4-bbbd-18d25f068513/metadata/test2 => generated 111 bytes in 18 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:51:58.602256 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-39e71372-ec37-40d8-99af-f90306683921 tempest-ServerMetadataNegativeTestJSON-1202925361 tempest-ServerMetadataNegativeTestJSON-1202925361-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:51:58.602756 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-39e71372-ec37-40d8-99af-f90306683921 tempest-ServerMetadataNegativeTestJSON-1202925361 tempest-ServerMetadataNegativeTestJSON-1202925361-project] [instance: c1c65e9e-289c-4707-b6f7-45fe851f4fae] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:51:58.609358 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-39e71372-ec37-40d8-99af-f90306683921 tempest-ServerMetadataNegativeTestJSON-1202925361 tempest-ServerMetadataNegativeTestJSON-1202925361-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:51:58.609809 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-39e71372-ec37-40d8-99af-f90306683921 tempest-ServerMetadataNegativeTestJSON-1202925361 tempest-ServerMetadataNegativeTestJSON-1202925361-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:51:58.610643 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-39e71372-ec37-40d8-99af-f90306683921 tempest-ServerMetadataNegativeTestJSON-1202925361 tempest-ServerMetadataNegativeTestJSON-1202925361-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:51:58.611118 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-39e71372-ec37-40d8-99af-f90306683921 tempest-ServerMetadataNegativeTestJSON-1202925361 tempest-ServerMetadataNegativeTestJSON-1202925361-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:51:58.637404 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-39e71372-ec37-40d8-99af-f90306683921 tempest-ServerMetadataNegativeTestJSON-1202925361 tempest-ServerMetadataNegativeTestJSON-1202925361-project] [instance: c1c65e9e-289c-4707-b6f7-45fe851f4fae] Going to try to terminate instance {{(pid=138680) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 16 21:51:58.987981 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-39e71372-ec37-40d8-99af-f90306683921 tempest-ServerMetadataNegativeTestJSON-1202925361 tempest-ServerMetadataNegativeTestJSON-1202925361-project] 10.222.0.16 "DELETE /compute/v2.1/servers/c1c65e9e-289c-4707-b6f7-45fe851f4fae" status: 204 len: 0 microversion: 2.1 time: 0.389196 Jul 16 21:51:58.988609 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3116/6237] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:51:58 2022] DELETE /compute/v2.1/servers/c1c65e9e-289c-4707-b6f7-45fe851f4fae => generated 0 bytes in 391 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 16 21:51:58.997910 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-4e7a18a7-8693-4ea4-8580-9fe3e4c824ae tempest-ServerMetadataNegativeTestJSON-1202925361 tempest-ServerMetadataNegativeTestJSON-1202925361-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:51:58.998633 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-4e7a18a7-8693-4ea4-8580-9fe3e4c824ae tempest-ServerMetadataNegativeTestJSON-1202925361 tempest-ServerMetadataNegativeTestJSON-1202925361-project] [instance: c1c65e9e-289c-4707-b6f7-45fe851f4fae] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:51:59.004872 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4e7a18a7-8693-4ea4-8580-9fe3e4c824ae tempest-ServerMetadataNegativeTestJSON-1202925361 tempest-ServerMetadataNegativeTestJSON-1202925361-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:51:59.005368 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4e7a18a7-8693-4ea4-8580-9fe3e4c824ae tempest-ServerMetadataNegativeTestJSON-1202925361 tempest-ServerMetadataNegativeTestJSON-1202925361-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:51:59.006151 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4e7a18a7-8693-4ea4-8580-9fe3e4c824ae tempest-ServerMetadataNegativeTestJSON-1202925361 tempest-ServerMetadataNegativeTestJSON-1202925361-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:51:59.006621 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4e7a18a7-8693-4ea4-8580-9fe3e4c824ae tempest-ServerMetadataNegativeTestJSON-1202925361 tempest-ServerMetadataNegativeTestJSON-1202925361-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:51:59.146987 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-4e7a18a7-8693-4ea4-8580-9fe3e4c824ae tempest-ServerMetadataNegativeTestJSON-1202925361 tempest-ServerMetadataNegativeTestJSON-1202925361-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8d3cada15a55453faaf23a147c44afcf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ecb79581f83f46f6a02fba932e20f3fe', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:51:59.156057 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-4e7a18a7-8693-4ea4-8580-9fe3e4c824ae tempest-ServerMetadataNegativeTestJSON-1202925361 tempest-ServerMetadataNegativeTestJSON-1202925361-project] 10.222.0.16 "GET /compute/v2.1/servers/c1c65e9e-289c-4707-b6f7-45fe851f4fae" status: 200 len: 1526 microversion: 2.1 time: 0.163313 Jul 16 21:51:59.156637 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3122/6238] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:51:58 2022] GET /compute/v2.1/servers/c1c65e9e-289c-4707-b6f7-45fe851f4fae => generated 1526 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:52:00.173926 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-026733a8-6dc6-4ec4-9c47-db99c39b281b tempest-ServerMetadataNegativeTestJSON-1202925361 tempest-ServerMetadataNegativeTestJSON-1202925361-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:52:00.174775 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-026733a8-6dc6-4ec4-9c47-db99c39b281b tempest-ServerMetadataNegativeTestJSON-1202925361 tempest-ServerMetadataNegativeTestJSON-1202925361-project] [instance: c1c65e9e-289c-4707-b6f7-45fe851f4fae] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:52:00.181813 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-026733a8-6dc6-4ec4-9c47-db99c39b281b tempest-ServerMetadataNegativeTestJSON-1202925361 tempest-ServerMetadataNegativeTestJSON-1202925361-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:52:00.182269 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-026733a8-6dc6-4ec4-9c47-db99c39b281b tempest-ServerMetadataNegativeTestJSON-1202925361 tempest-ServerMetadataNegativeTestJSON-1202925361-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:52:00.184639 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-026733a8-6dc6-4ec4-9c47-db99c39b281b tempest-ServerMetadataNegativeTestJSON-1202925361 tempest-ServerMetadataNegativeTestJSON-1202925361-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:52:00.184639 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-026733a8-6dc6-4ec4-9c47-db99c39b281b tempest-ServerMetadataNegativeTestJSON-1202925361 tempest-ServerMetadataNegativeTestJSON-1202925361-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:52:00.325092 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-026733a8-6dc6-4ec4-9c47-db99c39b281b tempest-ServerMetadataNegativeTestJSON-1202925361 tempest-ServerMetadataNegativeTestJSON-1202925361-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8d3cada15a55453faaf23a147c44afcf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ecb79581f83f46f6a02fba932e20f3fe', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:52:00.333128 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-026733a8-6dc6-4ec4-9c47-db99c39b281b tempest-ServerMetadataNegativeTestJSON-1202925361 tempest-ServerMetadataNegativeTestJSON-1202925361-project] 10.222.0.16 "GET /compute/v2.1/servers/c1c65e9e-289c-4707-b6f7-45fe851f4fae" status: 200 len: 1526 microversion: 2.1 time: 0.163478 Jul 16 21:52:00.333716 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3117/6239] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:52:00 2022] GET /compute/v2.1/servers/c1c65e9e-289c-4707-b6f7-45fe851f4fae => generated 1526 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:52:00.800962 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-3268b447-4c73-479f-b07c-a5671bcf4530 req-ac05a09f-d53e-4806-9154-6455c96a4270 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "c1c65e9e-289c-4707-b6f7-45fe851f4fae", "name": "network-vif-deleted", "tag": "8100c1c9-4e96-49f0-a067-63ae803ab7be"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:52:00.811294 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-3268b447-4c73-479f-b07c-a5671bcf4530 req-ac05a09f-d53e-4806-9154-6455c96a4270 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:52:00.811835 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-3268b447-4c73-479f-b07c-a5671bcf4530 req-ac05a09f-d53e-4806-9154-6455c96a4270 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:52:00.824182 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-3268b447-4c73-479f-b07c-a5671bcf4530 req-ac05a09f-d53e-4806-9154-6455c96a4270 service nova] Creating event network-vif-deleted:8100c1c9-4e96-49f0-a067-63ae803ab7be for instance c1c65e9e-289c-4707-b6f7-45fe851f4fae on nch1-master-1 Jul 16 21:52:00.829589 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-3268b447-4c73-479f-b07c-a5671bcf4530 req-ac05a09f-d53e-4806-9154-6455c96a4270 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.032420 Jul 16 21:52:00.830221 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3123/6240] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:52:00 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:52:01.350537 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-f6208eef-61eb-4846-8ed8-2e01d465c844 tempest-ServerMetadataNegativeTestJSON-1202925361 tempest-ServerMetadataNegativeTestJSON-1202925361-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:52:01.351225 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-f6208eef-61eb-4846-8ed8-2e01d465c844 tempest-ServerMetadataNegativeTestJSON-1202925361 tempest-ServerMetadataNegativeTestJSON-1202925361-project] [instance: c1c65e9e-289c-4707-b6f7-45fe851f4fae] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:52:01.359082 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f6208eef-61eb-4846-8ed8-2e01d465c844 tempest-ServerMetadataNegativeTestJSON-1202925361 tempest-ServerMetadataNegativeTestJSON-1202925361-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:52:01.364445 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f6208eef-61eb-4846-8ed8-2e01d465c844 tempest-ServerMetadataNegativeTestJSON-1202925361 tempest-ServerMetadataNegativeTestJSON-1202925361-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:52:01.364445 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f6208eef-61eb-4846-8ed8-2e01d465c844 tempest-ServerMetadataNegativeTestJSON-1202925361 tempest-ServerMetadataNegativeTestJSON-1202925361-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:52:01.364445 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f6208eef-61eb-4846-8ed8-2e01d465c844 tempest-ServerMetadataNegativeTestJSON-1202925361 tempest-ServerMetadataNegativeTestJSON-1202925361-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:52:01.393031 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f6208eef-61eb-4846-8ed8-2e01d465c844 tempest-ServerMetadataNegativeTestJSON-1202925361 tempest-ServerMetadataNegativeTestJSON-1202925361-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:52:01.393509 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f6208eef-61eb-4846-8ed8-2e01d465c844 tempest-ServerMetadataNegativeTestJSON-1202925361 tempest-ServerMetadataNegativeTestJSON-1202925361-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:52:01.394029 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.objects.instance [None req-f6208eef-61eb-4846-8ed8-2e01d465c844 tempest-ServerMetadataNegativeTestJSON-1202925361 tempest-ServerMetadataNegativeTestJSON-1202925361-project] Lazy-loading 'fault' on Instance uuid c1c65e9e-289c-4707-b6f7-45fe851f4fae {{(pid=138680) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:52:01.435736 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-f6208eef-61eb-4846-8ed8-2e01d465c844 tempest-ServerMetadataNegativeTestJSON-1202925361 tempest-ServerMetadataNegativeTestJSON-1202925361-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8d3cada15a55453faaf23a147c44afcf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ecb79581f83f46f6a02fba932e20f3fe', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:52:01.443222 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-f6208eef-61eb-4846-8ed8-2e01d465c844 tempest-ServerMetadataNegativeTestJSON-1202925361 tempest-ServerMetadataNegativeTestJSON-1202925361-project] 10.222.0.16 "GET /compute/v2.1/servers/c1c65e9e-289c-4707-b6f7-45fe851f4fae" status: 200 len: 1314 microversion: 2.1 time: 0.096833 Jul 16 21:52:01.443788 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3118/6241] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:52:01 2022] GET /compute/v2.1/servers/c1c65e9e-289c-4707-b6f7-45fe851f4fae => generated 1314 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:52:02.465536 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-9238ce40-0c94-4304-a783-b29d952913b5 tempest-ServerMetadataNegativeTestJSON-1202925361 tempest-ServerMetadataNegativeTestJSON-1202925361-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:52:02.466168 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-9238ce40-0c94-4304-a783-b29d952913b5 tempest-ServerMetadataNegativeTestJSON-1202925361 tempest-ServerMetadataNegativeTestJSON-1202925361-project] [instance: c1c65e9e-289c-4707-b6f7-45fe851f4fae] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:52:02.473127 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9238ce40-0c94-4304-a783-b29d952913b5 tempest-ServerMetadataNegativeTestJSON-1202925361 tempest-ServerMetadataNegativeTestJSON-1202925361-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:52:02.473583 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9238ce40-0c94-4304-a783-b29d952913b5 tempest-ServerMetadataNegativeTestJSON-1202925361 tempest-ServerMetadataNegativeTestJSON-1202925361-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:52:02.474359 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9238ce40-0c94-4304-a783-b29d952913b5 tempest-ServerMetadataNegativeTestJSON-1202925361 tempest-ServerMetadataNegativeTestJSON-1202925361-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:52:02.474810 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9238ce40-0c94-4304-a783-b29d952913b5 tempest-ServerMetadataNegativeTestJSON-1202925361 tempest-ServerMetadataNegativeTestJSON-1202925361-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:52:02.488846 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-9238ce40-0c94-4304-a783-b29d952913b5 tempest-ServerMetadataNegativeTestJSON-1202925361 tempest-ServerMetadataNegativeTestJSON-1202925361-project] HTTP exception thrown: Instance c1c65e9e-289c-4707-b6f7-45fe851f4fae could not be found. Jul 16 21:52:02.490582 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-9238ce40-0c94-4304-a783-b29d952913b5 tempest-ServerMetadataNegativeTestJSON-1202925361 tempest-ServerMetadataNegativeTestJSON-1202925361-project] Request method failure captured: Jul 16 21:52:02.490582 ncdv-master-1 devstack@n-api.service[138681]: request: GET /compute/v2.1/servers/c1c65e9e-289c-4707-b6f7-45fe851f4fae HTTP/1.1 Jul 16 21:52:02.490582 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:52:02.490582 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:52:02.490582 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:52:02.490582 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 21:52:02.490582 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:52:02.490582 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:52:02.490582 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:52:02.490582 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:52:02.490582 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0zLxzOplVHrs_HBQTEpuXRWANZiLYDoW82TFVevTZvvxbXxSdIS7ylDq68-X582trWyj8eL2Huzc3wP9CALwiQ1gr21xZhghWgku74rXV4Biu5GwzLBHQmKz5LX7GzFeshYs4UtF22ibFRiX6lmCLX7mITppYfmjCt4QO1m1QGIJpZc Jul 16 21:52:02.490582 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:52:02.490582 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:52:02.490582 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:52:02.490582 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:52:02.490582 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:52:02.490582 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:52:02.490582 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: ecb79581f83f46f6a02fba932e20f3fe Jul 16 21:52:02.490582 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-ServerMetadataNegativeTestJSON-1202925361 Jul 16 21:52:02.490582 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:52:02.490582 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:52:02.490582 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/ecb79581f83f46f6a02fba932e20f3fe"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/ecb79581f83f46f6a02fba932e20f3fe"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_ecb79581f83f46f6a02fba932e20f3fe"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/ecb79581f83f46f6a02fba932e20f3fe"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:52:02.490582 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-ServerMetadataNegativeTestJSON-1202925361 Jul 16 21:52:02.490582 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: ecb79581f83f46f6a02fba932e20f3fe Jul 16 21:52:02.490582 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-ServerMetadataNegativeTestJSON-1202925361 Jul 16 21:52:02.490582 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-ServerMetadataNegativeTestJSON-1202925361-project Jul 16 21:52:02.490582 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:52:02.490582 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:52:02.490582 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: 8d3cada15a55453faaf23a147c44afcf Jul 16 21:52:02.490582 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-ServerMetadataNegativeTestJSON-1202925361-project Jul 16 21:52:02.490582 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:52:02.490582 ncdv-master-1 devstack@n-api.service[138681]: exception: Instance c1c65e9e-289c-4707-b6f7-45fe851f4fae could not be found. Jul 16 21:52:02.490582 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': 'c1c65e9e-289c-4707-b6f7-45fe851f4fae'} Jul 16 21:52:02.490582 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:52:02.490582 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:52:02.490582 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:52:02.490582 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:52:02.490582 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:52:02.490582 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:52:02.490582 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:52:02.490582 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:52:02.490582 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:52:02.490582 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:52:02.490582 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance c1c65e9e-289c-4707-b6f7-45fe851f4fae could not be found. Jul 16 21:52:02.490582 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:52:02.490582 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:52:02.490582 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:52:02.490582 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:52:02.490582 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:52:02.490582 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:52:02.490582 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:52:02.490582 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:52:02.490582 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:52:02.490582 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:52:02.490582 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:52:02.490582 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:52:02.490582 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:52:02.490582 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:52:02.490582 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:52:02.490582 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:52:02.490582 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance c1c65e9e-289c-4707-b6f7-45fe851f4fae could not be found. Jul 16 21:52:02.490582 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:52:02.490582 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:52:02.494668 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:52:02.494668 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:52:02.494668 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:52:02.494668 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:52:02.494668 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:52:02.494668 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:52:02.494668 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance c1c65e9e-289c-4707-b6f7-45fe851f4fae could not be found. Jul 16 21:52:02.494668 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:52:02.494668 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-9238ce40-0c94-4304-a783-b29d952913b5 tempest-ServerMetadataNegativeTestJSON-1202925361 tempest-ServerMetadataNegativeTestJSON-1202925361-project] Returning 404 to user: Instance c1c65e9e-289c-4707-b6f7-45fe851f4fae could not be found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:52:02.494668 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-9238ce40-0c94-4304-a783-b29d952913b5 tempest-ServerMetadataNegativeTestJSON-1202925361 tempest-ServerMetadataNegativeTestJSON-1202925361-project] 10.222.0.16 "GET /compute/v2.1/servers/c1c65e9e-289c-4707-b6f7-45fe851f4fae" status: 404 len: 111 microversion: 2.1 time: 0.032957 Jul 16 21:52:02.494668 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3124/6242] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:52:02 2022] GET /compute/v2.1/servers/c1c65e9e-289c-4707-b6f7-45fe851f4fae => generated 111 bytes in 34 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:53:02.613218 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_api:SELECT=31,nova_api:UPDATE=1 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:53:02.614886 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=56,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:53:02.745896 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-782779e4-088a-40af-9b11-4fae5a072502 tempest-PortsAdminExtendedAttrsIpV6TestJSON-2134331308 tempest-PortsAdminExtendedAttrsIpV6TestJSON-2134331308-project-admin] Calling method '.version_select at 0x7fdd9d3320d0>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:53:02.746787 ncdv-master-1 devstack@n-api.service[138680]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-hypervisors:list": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:53:02.746787 ncdv-master-1 devstack@n-api.service[138680]: warnings.warn(msg) Jul 16 21:53:02.747996 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-782779e4-088a-40af-9b11-4fae5a072502 tempest-PortsAdminExtendedAttrsIpV6TestJSON-2134331308 tempest-PortsAdminExtendedAttrsIpV6TestJSON-2134331308-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:53:02.748450 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-782779e4-088a-40af-9b11-4fae5a072502 tempest-PortsAdminExtendedAttrsIpV6TestJSON-2134331308 tempest-PortsAdminExtendedAttrsIpV6TestJSON-2134331308-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:53:02.762532 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-782779e4-088a-40af-9b11-4fae5a072502 tempest-PortsAdminExtendedAttrsIpV6TestJSON-2134331308 tempest-PortsAdminExtendedAttrsIpV6TestJSON-2134331308-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:53:02.762989 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-782779e4-088a-40af-9b11-4fae5a072502 tempest-PortsAdminExtendedAttrsIpV6TestJSON-2134331308 tempest-PortsAdminExtendedAttrsIpV6TestJSON-2134331308-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:53:02.773334 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-782779e4-088a-40af-9b11-4fae5a072502 tempest-PortsAdminExtendedAttrsIpV6TestJSON-2134331308 tempest-PortsAdminExtendedAttrsIpV6TestJSON-2134331308-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:53:02.773791 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-782779e4-088a-40af-9b11-4fae5a072502 tempest-PortsAdminExtendedAttrsIpV6TestJSON-2134331308 tempest-PortsAdminExtendedAttrsIpV6TestJSON-2134331308-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:53:02.779663 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-782779e4-088a-40af-9b11-4fae5a072502 tempest-PortsAdminExtendedAttrsIpV6TestJSON-2134331308 tempest-PortsAdminExtendedAttrsIpV6TestJSON-2134331308-project-admin] 10.222.0.16 "GET /compute/v2.1/os-hypervisors" status: 200 len: 191 microversion: 2.1 time: 0.168711 Jul 16 21:53:02.780256 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3119/6243] 10.222.0.16 () {58 vars in 1145 bytes} [Sat Jul 16 21:53:02 2022] GET /compute/v2.1/os-hypervisors => generated 191 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:53:04.940703 ncdv-master-1 devstack@n-api.service[138680]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jul 16 21:53:04.941301 ncdv-master-1 devstack@n-api.service[138680]: Sat Jul 16 21:53:04 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Jul 16 21:54:44.754253 ncdv-master-1 devstack@n-api.service[138680]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jul 16 21:54:44.755156 ncdv-master-1 devstack@n-api.service[138680]: Sat Jul 16 21:54:44 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Jul 16 21:54:46.818002 ncdv-master-1 devstack@n-api.service[138681]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jul 16 21:54:46.826476 ncdv-master-1 devstack@n-api.service[138681]: Sat Jul 16 21:54:46 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Jul 16 21:55:08.952285 ncdv-master-1 devstack@n-api.service[138681]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jul 16 21:55:08.955341 ncdv-master-1 devstack@n-api.service[138681]: Sat Jul 16 21:55:08 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Jul 16 21:55:30.851955 ncdv-master-1 devstack@n-api.service[138681]: ERROR oslo.messaging._drivers.impl_rabbit [-] [4de8847c-05a2-4cd6-bd0e-256d7a754675] AMQP server on 10.222.0.16:5672 is unreachable: . Trying again in 1 seconds.: amqp.exceptions.RecoverableConnectionError: Jul 16 21:55:30.852937 ncdv-master-1 devstack@n-api.service[138681]: ERROR oslo.messaging._drivers.impl_rabbit [-] [e4397b8a-69d0-4b1f-ae48-d345afeb4816] AMQP server on 10.222.0.16:5672 is unreachable: . Trying again in 1 seconds.: amqp.exceptions.RecoverableConnectionError: Jul 16 21:55:30.853541 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_api:SELECT=38,nova_api:INSERT=3 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:55:30.854303 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=42 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:55:30.972393 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-259a97bf-5622-4dec-9b25-9103942b207d tempest-VolumesActionsTest-763750246 tempest-VolumesActionsTest-763750246-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesActionsTest-instance-770401519", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452", "networks": [{"uuid": "2aac0fa2-2625-4c0c-ae01-205928d98e37"}]}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:55:31.146096 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.network.neutron [None req-259a97bf-5622-4dec-9b25-9103942b207d tempest-VolumesActionsTest-763750246 tempest-VolumesActionsTest-763750246-project] validate_networks() for [('2aac0fa2-2625-4c0c-ae01-205928d98e37', None, None, None, None, None)] {{(pid=138681) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 16 21:55:31.500040 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-259a97bf-5622-4dec-9b25-9103942b207d tempest-VolumesActionsTest-763750246 tempest-VolumesActionsTest-763750246-project] Flavor limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 16 21:55:31.500483 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-259a97bf-5622-4dec-9b25-9103942b207d tempest-VolumesActionsTest-763750246 tempest-VolumesActionsTest-763750246-project] Image limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 16 21:55:31.500976 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-259a97bf-5622-4dec-9b25-9103942b207d tempest-VolumesActionsTest-763750246 tempest-VolumesActionsTest-763750246-project] Flavor pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 16 21:55:31.501401 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-259a97bf-5622-4dec-9b25-9103942b207d tempest-VolumesActionsTest-763750246 tempest-VolumesActionsTest-763750246-project] Image pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 16 21:55:31.501833 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-259a97bf-5622-4dec-9b25-9103942b207d tempest-VolumesActionsTest-763750246 tempest-VolumesActionsTest-763750246-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 16 21:55:31.511196 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-259a97bf-5622-4dec-9b25-9103942b207d tempest-VolumesActionsTest-763750246 tempest-VolumesActionsTest-763750246-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:55:31.511666 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-259a97bf-5622-4dec-9b25-9103942b207d tempest-VolumesActionsTest-763750246 tempest-VolumesActionsTest-763750246-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:55:31.512494 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-259a97bf-5622-4dec-9b25-9103942b207d tempest-VolumesActionsTest-763750246 tempest-VolumesActionsTest-763750246-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:55:31.513126 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-259a97bf-5622-4dec-9b25-9103942b207d tempest-VolumesActionsTest-763750246 tempest-VolumesActionsTest-763750246-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:55:31.530487 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-259a97bf-5622-4dec-9b25-9103942b207d tempest-VolumesActionsTest-763750246 tempest-VolumesActionsTest-763750246-project] Getting quotas for project 13ce121469a145a4b00edd3a082413a4. Resources: {'ram', 'instances', 'cores'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:55:31.536381 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-259a97bf-5622-4dec-9b25-9103942b207d tempest-VolumesActionsTest-763750246 tempest-VolumesActionsTest-763750246-project] Getting quotas for user 2331af9efc5e4af7a901621b7590e80a and project 13ce121469a145a4b00edd3a082413a4. Resources: {'ram', 'instances', 'cores'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:55:31.550384 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-259a97bf-5622-4dec-9b25-9103942b207d tempest-VolumesActionsTest-763750246 tempest-VolumesActionsTest-763750246-project] Going to run 1 instances... {{(pid=138681) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 16 21:55:31.552786 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-259a97bf-5622-4dec-9b25-9103942b207d tempest-VolumesActionsTest-763750246 tempest-VolumesActionsTest-763750246-project] [instance: 2ba83fff-f116-4f5e-8711-550d4a6e2989] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138681) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 16 21:55:31.679578 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-259a97bf-5622-4dec-9b25-9103942b207d tempest-VolumesActionsTest-763750246 tempest-VolumesActionsTest-763750246-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.829689 Jul 16 21:55:31.680240 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3125/6244] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 21:55:30 2022] POST /compute/v2.1/servers => generated 380 bytes in 831 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 16 21:55:31.688570 ncdv-master-1 devstack@n-api.service[138680]: ERROR oslo.messaging._drivers.impl_rabbit [-] [a302809d-3282-48f5-9662-a738982e3059] AMQP server on 10.222.0.16:5672 is unreachable: . Trying again in 1 seconds.: amqp.exceptions.RecoverableConnectionError: Jul 16 21:55:31.692385 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-4fb1ba7e-4b51-4cf0-9d5a-3c75b91744c2 tempest-VolumesActionsTest-763750246 tempest-VolumesActionsTest-763750246-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:55:31.693966 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-4fb1ba7e-4b51-4cf0-9d5a-3c75b91744c2 tempest-VolumesActionsTest-763750246 tempest-VolumesActionsTest-763750246-project] [instance: 2ba83fff-f116-4f5e-8711-550d4a6e2989] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:55:31.696977 ncdv-master-1 devstack@n-api.service[138680]: ERROR oslo.messaging._drivers.impl_rabbit [-] [7d526b5e-7cab-48f3-be00-1b242d865b86] AMQP server on 10.222.0.16:5672 is unreachable: . Trying again in 1 seconds.: amqp.exceptions.RecoverableConnectionError: Jul 16 21:55:31.707151 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_api:SELECT=4 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:55:31.874785 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-4fb1ba7e-4b51-4cf0-9d5a-3c75b91744c2 tempest-VolumesActionsTest-763750246 tempest-VolumesActionsTest-763750246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2331af9efc5e4af7a901621b7590e80a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13ce121469a145a4b00edd3a082413a4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:55:31.884451 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-4fb1ba7e-4b51-4cf0-9d5a-3c75b91744c2 tempest-VolumesActionsTest-763750246 tempest-VolumesActionsTest-763750246-project] 10.222.0.16 "GET /compute/v2.1/servers/2ba83fff-f116-4f5e-8711-550d4a6e2989" status: 200 len: 1213 microversion: 2.1 time: 0.195294 Jul 16 21:55:31.884451 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3120/6245] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:55:31 2022] GET /compute/v2.1/servers/2ba83fff-f116-4f5e-8711-550d4a6e2989 => generated 1213 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:55:32.901255 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-4873a884-7206-44b5-82ba-2280e4bb31fc tempest-VolumesActionsTest-763750246 tempest-VolumesActionsTest-763750246-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:55:32.901962 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-4873a884-7206-44b5-82ba-2280e4bb31fc tempest-VolumesActionsTest-763750246 tempest-VolumesActionsTest-763750246-project] [instance: 2ba83fff-f116-4f5e-8711-550d4a6e2989] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:55:32.912051 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4873a884-7206-44b5-82ba-2280e4bb31fc tempest-VolumesActionsTest-763750246 tempest-VolumesActionsTest-763750246-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:55:32.912516 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4873a884-7206-44b5-82ba-2280e4bb31fc tempest-VolumesActionsTest-763750246 tempest-VolumesActionsTest-763750246-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:55:32.913303 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4873a884-7206-44b5-82ba-2280e4bb31fc tempest-VolumesActionsTest-763750246 tempest-VolumesActionsTest-763750246-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:55:32.913929 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4873a884-7206-44b5-82ba-2280e4bb31fc tempest-VolumesActionsTest-763750246 tempest-VolumesActionsTest-763750246-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:55:32.918564 ncdv-master-1 devstack@n-api.service[138681]: INFO oslo.messaging._drivers.impl_rabbit [-] [4de8847c-05a2-4cd6-bd0e-256d7a754675] Reconnected to AMQP server on 10.222.0.16:5672 via [amqp] client with port 55828. Jul 16 21:55:32.921143 ncdv-master-1 devstack@n-api.service[138681]: INFO oslo.messaging._drivers.impl_rabbit [-] [e4397b8a-69d0-4b1f-ae48-d345afeb4816] Reconnected to AMQP server on 10.222.0.16:5672 via [amqp] client with port 55830. Jul 16 21:55:32.985148 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-4873a884-7206-44b5-82ba-2280e4bb31fc tempest-VolumesActionsTest-763750246 tempest-VolumesActionsTest-763750246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2331af9efc5e4af7a901621b7590e80a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13ce121469a145a4b00edd3a082413a4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:55:32.992950 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-4873a884-7206-44b5-82ba-2280e4bb31fc tempest-VolumesActionsTest-763750246 tempest-VolumesActionsTest-763750246-project] 10.222.0.16 "GET /compute/v2.1/servers/2ba83fff-f116-4f5e-8711-550d4a6e2989" status: 200 len: 1217 microversion: 2.1 time: 0.096956 Jul 16 21:55:32.999148 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3126/6246] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:55:32 2022] GET /compute/v2.1/servers/2ba83fff-f116-4f5e-8711-550d4a6e2989 => generated 1217 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:55:34.010670 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-323795fd-3fe3-4622-ab38-f53f97a46d79 tempest-VolumesActionsTest-763750246 tempest-VolumesActionsTest-763750246-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:55:34.012056 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-323795fd-3fe3-4622-ab38-f53f97a46d79 tempest-VolumesActionsTest-763750246 tempest-VolumesActionsTest-763750246-project] [instance: 2ba83fff-f116-4f5e-8711-550d4a6e2989] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:55:34.024925 ncdv-master-1 devstack@n-api.service[138680]: INFO oslo.messaging._drivers.impl_rabbit [-] [7d526b5e-7cab-48f3-be00-1b242d865b86] Reconnected to AMQP server on 10.222.0.16:5672 via [amqp] client with port 55836. Jul 16 21:55:34.033515 ncdv-master-1 devstack@n-api.service[138680]: INFO oslo.messaging._drivers.impl_rabbit [-] [a302809d-3282-48f5-9662-a738982e3059] Reconnected to AMQP server on 10.222.0.16:5672 via [amqp] client with port 55834. Jul 16 21:55:34.039656 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-323795fd-3fe3-4622-ab38-f53f97a46d79 tempest-VolumesActionsTest-763750246 tempest-VolumesActionsTest-763750246-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:55:34.040207 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-323795fd-3fe3-4622-ab38-f53f97a46d79 tempest-VolumesActionsTest-763750246 tempest-VolumesActionsTest-763750246-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:55:34.041089 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-323795fd-3fe3-4622-ab38-f53f97a46d79 tempest-VolumesActionsTest-763750246 tempest-VolumesActionsTest-763750246-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:55:34.041602 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-323795fd-3fe3-4622-ab38-f53f97a46d79 tempest-VolumesActionsTest-763750246 tempest-VolumesActionsTest-763750246-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:55:34.109142 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-323795fd-3fe3-4622-ab38-f53f97a46d79 tempest-VolumesActionsTest-763750246 tempest-VolumesActionsTest-763750246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2331af9efc5e4af7a901621b7590e80a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13ce121469a145a4b00edd3a082413a4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:55:34.118217 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-323795fd-3fe3-4622-ab38-f53f97a46d79 tempest-VolumesActionsTest-763750246 tempest-VolumesActionsTest-763750246-project] 10.222.0.16 "GET /compute/v2.1/servers/2ba83fff-f116-4f5e-8711-550d4a6e2989" status: 200 len: 1283 microversion: 2.1 time: 0.112031 Jul 16 21:55:34.118814 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3121/6247] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:55:34 2022] GET /compute/v2.1/servers/2ba83fff-f116-4f5e-8711-550d4a6e2989 => generated 1283 bytes in 114 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:55:35.136170 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-712a9f5f-96c1-4494-8e11-5b52b9f08cfc tempest-VolumesActionsTest-763750246 tempest-VolumesActionsTest-763750246-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:55:35.136802 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-712a9f5f-96c1-4494-8e11-5b52b9f08cfc tempest-VolumesActionsTest-763750246 tempest-VolumesActionsTest-763750246-project] [instance: 2ba83fff-f116-4f5e-8711-550d4a6e2989] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:55:35.145146 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-712a9f5f-96c1-4494-8e11-5b52b9f08cfc tempest-VolumesActionsTest-763750246 tempest-VolumesActionsTest-763750246-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:55:35.145622 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-712a9f5f-96c1-4494-8e11-5b52b9f08cfc tempest-VolumesActionsTest-763750246 tempest-VolumesActionsTest-763750246-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:55:35.146423 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-712a9f5f-96c1-4494-8e11-5b52b9f08cfc tempest-VolumesActionsTest-763750246 tempest-VolumesActionsTest-763750246-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:55:35.146879 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-712a9f5f-96c1-4494-8e11-5b52b9f08cfc tempest-VolumesActionsTest-763750246 tempest-VolumesActionsTest-763750246-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:55:35.283465 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-712a9f5f-96c1-4494-8e11-5b52b9f08cfc tempest-VolumesActionsTest-763750246 tempest-VolumesActionsTest-763750246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2331af9efc5e4af7a901621b7590e80a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13ce121469a145a4b00edd3a082413a4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:55:35.291135 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-712a9f5f-96c1-4494-8e11-5b52b9f08cfc tempest-VolumesActionsTest-763750246 tempest-VolumesActionsTest-763750246-project] 10.222.0.16 "GET /compute/v2.1/servers/2ba83fff-f116-4f5e-8711-550d4a6e2989" status: 200 len: 1313 microversion: 2.1 time: 0.159227 Jul 16 21:55:35.292081 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3127/6248] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:55:35 2022] GET /compute/v2.1/servers/2ba83fff-f116-4f5e-8711-550d4a6e2989 => generated 1313 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:55:35.516125 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-93933033-4c33-4910-9791-33f4271bfc1b req-cd0b943a-fe36-40bf-a8fc-f22cd20d1c6a service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "2ba83fff-f116-4f5e-8711-550d4a6e2989", "tag": "bdf82442-2fcd-4ffc-801d-5a1d5ea61c9f"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:55:35.524664 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-93933033-4c33-4910-9791-33f4271bfc1b req-cd0b943a-fe36-40bf-a8fc-f22cd20d1c6a service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:55:35.525236 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-93933033-4c33-4910-9791-33f4271bfc1b req-cd0b943a-fe36-40bf-a8fc-f22cd20d1c6a service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:55:35.539623 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-93933033-4c33-4910-9791-33f4271bfc1b req-cd0b943a-fe36-40bf-a8fc-f22cd20d1c6a service nova] Creating event network-changed:bdf82442-2fcd-4ffc-801d-5a1d5ea61c9f for instance 2ba83fff-f116-4f5e-8711-550d4a6e2989 on nch1-master-1 Jul 16 21:55:35.544580 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-93933033-4c33-4910-9791-33f4271bfc1b req-cd0b943a-fe36-40bf-a8fc-f22cd20d1c6a service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.102854 Jul 16 21:55:35.545191 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3122/6249] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:55:35 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:55:36.308418 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-2d51e469-063d-4dc8-8f22-a881e067fda4 tempest-VolumesActionsTest-763750246 tempest-VolumesActionsTest-763750246-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:55:36.309051 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-2d51e469-063d-4dc8-8f22-a881e067fda4 tempest-VolumesActionsTest-763750246 tempest-VolumesActionsTest-763750246-project] [instance: 2ba83fff-f116-4f5e-8711-550d4a6e2989] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:55:36.316140 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2d51e469-063d-4dc8-8f22-a881e067fda4 tempest-VolumesActionsTest-763750246 tempest-VolumesActionsTest-763750246-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:55:36.316628 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2d51e469-063d-4dc8-8f22-a881e067fda4 tempest-VolumesActionsTest-763750246 tempest-VolumesActionsTest-763750246-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:55:36.317385 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2d51e469-063d-4dc8-8f22-a881e067fda4 tempest-VolumesActionsTest-763750246 tempest-VolumesActionsTest-763750246-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:55:36.317839 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2d51e469-063d-4dc8-8f22-a881e067fda4 tempest-VolumesActionsTest-763750246 tempest-VolumesActionsTest-763750246-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:55:36.450022 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-2d51e469-063d-4dc8-8f22-a881e067fda4 tempest-VolumesActionsTest-763750246 tempest-VolumesActionsTest-763750246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2331af9efc5e4af7a901621b7590e80a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13ce121469a145a4b00edd3a082413a4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:55:36.458382 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-2d51e469-063d-4dc8-8f22-a881e067fda4 tempest-VolumesActionsTest-763750246 tempest-VolumesActionsTest-763750246-project] 10.222.0.16 "GET /compute/v2.1/servers/2ba83fff-f116-4f5e-8711-550d4a6e2989" status: 200 len: 1313 microversion: 2.1 time: 0.153904 Jul 16 21:55:36.458976 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3128/6250] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:55:36 2022] GET /compute/v2.1/servers/2ba83fff-f116-4f5e-8711-550d4a6e2989 => generated 1313 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:55:37.477999 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-f0f94d9d-bd56-4003-abd0-b3b58ce1011f tempest-VolumesActionsTest-763750246 tempest-VolumesActionsTest-763750246-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:55:37.478775 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-f0f94d9d-bd56-4003-abd0-b3b58ce1011f tempest-VolumesActionsTest-763750246 tempest-VolumesActionsTest-763750246-project] [instance: 2ba83fff-f116-4f5e-8711-550d4a6e2989] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:55:37.485822 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f0f94d9d-bd56-4003-abd0-b3b58ce1011f tempest-VolumesActionsTest-763750246 tempest-VolumesActionsTest-763750246-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:55:37.486280 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f0f94d9d-bd56-4003-abd0-b3b58ce1011f tempest-VolumesActionsTest-763750246 tempest-VolumesActionsTest-763750246-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:55:37.487149 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f0f94d9d-bd56-4003-abd0-b3b58ce1011f tempest-VolumesActionsTest-763750246 tempest-VolumesActionsTest-763750246-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:55:37.487599 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f0f94d9d-bd56-4003-abd0-b3b58ce1011f tempest-VolumesActionsTest-763750246 tempest-VolumesActionsTest-763750246-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:55:37.634667 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-f0f94d9d-bd56-4003-abd0-b3b58ce1011f tempest-VolumesActionsTest-763750246 tempest-VolumesActionsTest-763750246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2331af9efc5e4af7a901621b7590e80a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13ce121469a145a4b00edd3a082413a4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:55:37.640787 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-f0f94d9d-bd56-4003-abd0-b3b58ce1011f tempest-VolumesActionsTest-763750246 tempest-VolumesActionsTest-763750246-project] 10.222.0.16 "GET /compute/v2.1/servers/2ba83fff-f116-4f5e-8711-550d4a6e2989" status: 200 len: 1313 microversion: 2.1 time: 0.168194 Jul 16 21:55:37.641373 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3123/6251] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:55:37 2022] GET /compute/v2.1/servers/2ba83fff-f116-4f5e-8711-550d4a6e2989 => generated 1313 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:55:38.658626 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-935d8a2f-96cf-4d71-b67e-7aecdf0f56e5 tempest-VolumesActionsTest-763750246 tempest-VolumesActionsTest-763750246-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:55:38.659301 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-935d8a2f-96cf-4d71-b67e-7aecdf0f56e5 tempest-VolumesActionsTest-763750246 tempest-VolumesActionsTest-763750246-project] [instance: 2ba83fff-f116-4f5e-8711-550d4a6e2989] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:55:38.666619 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-935d8a2f-96cf-4d71-b67e-7aecdf0f56e5 tempest-VolumesActionsTest-763750246 tempest-VolumesActionsTest-763750246-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:55:38.667128 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-935d8a2f-96cf-4d71-b67e-7aecdf0f56e5 tempest-VolumesActionsTest-763750246 tempest-VolumesActionsTest-763750246-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:55:38.667925 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-935d8a2f-96cf-4d71-b67e-7aecdf0f56e5 tempest-VolumesActionsTest-763750246 tempest-VolumesActionsTest-763750246-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:55:38.668388 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-935d8a2f-96cf-4d71-b67e-7aecdf0f56e5 tempest-VolumesActionsTest-763750246 tempest-VolumesActionsTest-763750246-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:55:38.807172 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-935d8a2f-96cf-4d71-b67e-7aecdf0f56e5 tempest-VolumesActionsTest-763750246 tempest-VolumesActionsTest-763750246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2331af9efc5e4af7a901621b7590e80a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13ce121469a145a4b00edd3a082413a4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:55:38.815176 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-935d8a2f-96cf-4d71-b67e-7aecdf0f56e5 tempest-VolumesActionsTest-763750246 tempest-VolumesActionsTest-763750246-project] 10.222.0.16 "GET /compute/v2.1/servers/2ba83fff-f116-4f5e-8711-550d4a6e2989" status: 200 len: 1313 microversion: 2.1 time: 0.160572 Jul 16 21:55:38.823891 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3129/6252] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:55:38 2022] GET /compute/v2.1/servers/2ba83fff-f116-4f5e-8711-550d4a6e2989 => generated 1313 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:55:39.832640 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-ba96e015-46a4-4ccb-8523-23687ead3454 tempest-VolumesActionsTest-763750246 tempest-VolumesActionsTest-763750246-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:55:39.833250 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-ba96e015-46a4-4ccb-8523-23687ead3454 tempest-VolumesActionsTest-763750246 tempest-VolumesActionsTest-763750246-project] [instance: 2ba83fff-f116-4f5e-8711-550d4a6e2989] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:55:39.840201 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ba96e015-46a4-4ccb-8523-23687ead3454 tempest-VolumesActionsTest-763750246 tempest-VolumesActionsTest-763750246-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:55:39.840651 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ba96e015-46a4-4ccb-8523-23687ead3454 tempest-VolumesActionsTest-763750246 tempest-VolumesActionsTest-763750246-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:55:39.841406 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ba96e015-46a4-4ccb-8523-23687ead3454 tempest-VolumesActionsTest-763750246 tempest-VolumesActionsTest-763750246-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:55:39.841845 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ba96e015-46a4-4ccb-8523-23687ead3454 tempest-VolumesActionsTest-763750246 tempest-VolumesActionsTest-763750246-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:55:39.974847 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-ba96e015-46a4-4ccb-8523-23687ead3454 tempest-VolumesActionsTest-763750246 tempest-VolumesActionsTest-763750246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2331af9efc5e4af7a901621b7590e80a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13ce121469a145a4b00edd3a082413a4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:55:39.982341 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-ba96e015-46a4-4ccb-8523-23687ead3454 tempest-VolumesActionsTest-763750246 tempest-VolumesActionsTest-763750246-project] 10.222.0.16 "GET /compute/v2.1/servers/2ba83fff-f116-4f5e-8711-550d4a6e2989" status: 200 len: 1313 microversion: 2.1 time: 0.153606 Jul 16 21:55:39.982915 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3124/6253] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:55:39 2022] GET /compute/v2.1/servers/2ba83fff-f116-4f5e-8711-550d4a6e2989 => generated 1313 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:55:40.999457 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-8da4ed83-9d87-4759-a5da-526aad188964 tempest-VolumesActionsTest-763750246 tempest-VolumesActionsTest-763750246-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:55:41.000081 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-8da4ed83-9d87-4759-a5da-526aad188964 tempest-VolumesActionsTest-763750246 tempest-VolumesActionsTest-763750246-project] [instance: 2ba83fff-f116-4f5e-8711-550d4a6e2989] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:55:41.007591 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8da4ed83-9d87-4759-a5da-526aad188964 tempest-VolumesActionsTest-763750246 tempest-VolumesActionsTest-763750246-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:55:41.008086 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8da4ed83-9d87-4759-a5da-526aad188964 tempest-VolumesActionsTest-763750246 tempest-VolumesActionsTest-763750246-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:55:41.008874 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8da4ed83-9d87-4759-a5da-526aad188964 tempest-VolumesActionsTest-763750246 tempest-VolumesActionsTest-763750246-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:55:41.009324 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8da4ed83-9d87-4759-a5da-526aad188964 tempest-VolumesActionsTest-763750246 tempest-VolumesActionsTest-763750246-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:55:41.151208 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-8da4ed83-9d87-4759-a5da-526aad188964 tempest-VolumesActionsTest-763750246 tempest-VolumesActionsTest-763750246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2331af9efc5e4af7a901621b7590e80a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13ce121469a145a4b00edd3a082413a4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:55:41.160088 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-8da4ed83-9d87-4759-a5da-526aad188964 tempest-VolumesActionsTest-763750246 tempest-VolumesActionsTest-763750246-project] 10.222.0.16 "GET /compute/v2.1/servers/2ba83fff-f116-4f5e-8711-550d4a6e2989" status: 200 len: 1313 microversion: 2.1 time: 0.164663 Jul 16 21:55:41.160670 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3130/6254] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:55:40 2022] GET /compute/v2.1/servers/2ba83fff-f116-4f5e-8711-550d4a6e2989 => generated 1313 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:55:41.301207 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-7f50c0db-89d5-4db0-b463-dc1ee61700dd req-bb29f417-fa2b-445c-8aa5-2675ec6acdb0 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "2ba83fff-f116-4f5e-8711-550d4a6e2989", "name": "network-vif-plugged", "status": "completed", "tag": "bdf82442-2fcd-4ffc-801d-5a1d5ea61c9f"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:55:41.309167 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-7f50c0db-89d5-4db0-b463-dc1ee61700dd req-bb29f417-fa2b-445c-8aa5-2675ec6acdb0 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:55:41.309615 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-7f50c0db-89d5-4db0-b463-dc1ee61700dd req-bb29f417-fa2b-445c-8aa5-2675ec6acdb0 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:55:41.321333 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-7f50c0db-89d5-4db0-b463-dc1ee61700dd req-bb29f417-fa2b-445c-8aa5-2675ec6acdb0 service nova] Creating event network-vif-plugged:bdf82442-2fcd-4ffc-801d-5a1d5ea61c9f for instance 2ba83fff-f116-4f5e-8711-550d4a6e2989 on nch1-master-1 Jul 16 21:55:41.326408 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-7f50c0db-89d5-4db0-b463-dc1ee61700dd req-bb29f417-fa2b-445c-8aa5-2675ec6acdb0 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.093915 Jul 16 21:55:41.327589 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3125/6255] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:55:41 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:55:42.178020 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-803af34a-5ce1-489a-9f59-07897403071a tempest-VolumesActionsTest-763750246 tempest-VolumesActionsTest-763750246-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:55:42.178656 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-803af34a-5ce1-489a-9f59-07897403071a tempest-VolumesActionsTest-763750246 tempest-VolumesActionsTest-763750246-project] [instance: 2ba83fff-f116-4f5e-8711-550d4a6e2989] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:55:42.180577 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell0:SELECT=2 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:55:42.181296 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell0:SELECT=6 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:55:42.187316 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-803af34a-5ce1-489a-9f59-07897403071a tempest-VolumesActionsTest-763750246 tempest-VolumesActionsTest-763750246-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:55:42.187792 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-803af34a-5ce1-489a-9f59-07897403071a tempest-VolumesActionsTest-763750246 tempest-VolumesActionsTest-763750246-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:55:42.188580 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-803af34a-5ce1-489a-9f59-07897403071a tempest-VolumesActionsTest-763750246 tempest-VolumesActionsTest-763750246-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:55:42.189038 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-803af34a-5ce1-489a-9f59-07897403071a tempest-VolumesActionsTest-763750246 tempest-VolumesActionsTest-763750246-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:55:42.323585 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-803af34a-5ce1-489a-9f59-07897403071a tempest-VolumesActionsTest-763750246 tempest-VolumesActionsTest-763750246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2331af9efc5e4af7a901621b7590e80a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13ce121469a145a4b00edd3a082413a4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:55:42.331181 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-803af34a-5ce1-489a-9f59-07897403071a tempest-VolumesActionsTest-763750246 tempest-VolumesActionsTest-763750246-project] 10.222.0.16 "GET /compute/v2.1/servers/2ba83fff-f116-4f5e-8711-550d4a6e2989" status: 200 len: 1313 microversion: 2.1 time: 0.157084 Jul 16 21:55:42.331745 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3131/6256] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:55:42 2022] GET /compute/v2.1/servers/2ba83fff-f116-4f5e-8711-550d4a6e2989 => generated 1313 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:55:43.346185 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell0:SELECT=2 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:55:43.351232 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-59215ed6-3338-4b50-9d3b-e0b53447e7b8 tempest-VolumesActionsTest-763750246 tempest-VolumesActionsTest-763750246-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:55:43.365856 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-59215ed6-3338-4b50-9d3b-e0b53447e7b8 tempest-VolumesActionsTest-763750246 tempest-VolumesActionsTest-763750246-project] [instance: 2ba83fff-f116-4f5e-8711-550d4a6e2989] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:55:43.365856 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-59215ed6-3338-4b50-9d3b-e0b53447e7b8 tempest-VolumesActionsTest-763750246 tempest-VolumesActionsTest-763750246-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:55:43.365856 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-59215ed6-3338-4b50-9d3b-e0b53447e7b8 tempest-VolumesActionsTest-763750246 tempest-VolumesActionsTest-763750246-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:55:43.365856 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-59215ed6-3338-4b50-9d3b-e0b53447e7b8 tempest-VolumesActionsTest-763750246 tempest-VolumesActionsTest-763750246-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:55:43.365856 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-59215ed6-3338-4b50-9d3b-e0b53447e7b8 tempest-VolumesActionsTest-763750246 tempest-VolumesActionsTest-763750246-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:55:43.493103 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-59215ed6-3338-4b50-9d3b-e0b53447e7b8 tempest-VolumesActionsTest-763750246 tempest-VolumesActionsTest-763750246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2331af9efc5e4af7a901621b7590e80a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13ce121469a145a4b00edd3a082413a4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:55:43.501671 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-59215ed6-3338-4b50-9d3b-e0b53447e7b8 tempest-VolumesActionsTest-763750246 tempest-VolumesActionsTest-763750246-project] 10.222.0.16 "GET /compute/v2.1/servers/2ba83fff-f116-4f5e-8711-550d4a6e2989" status: 200 len: 1496 microversion: 2.1 time: 0.156538 Jul 16 21:55:43.502250 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3126/6257] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:55:43 2022] GET /compute/v2.1/servers/2ba83fff-f116-4f5e-8711-550d4a6e2989 => generated 1496 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:55:46.483697 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-8fdd180d-3026-437f-afdb-6ea267dd0ccd tempest-VolumesActionsTest-763750246 tempest-VolumesActionsTest-763750246-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:55:46.484234 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-8fdd180d-3026-437f-afdb-6ea267dd0ccd tempest-VolumesActionsTest-763750246 tempest-VolumesActionsTest-763750246-project] [instance: 2ba83fff-f116-4f5e-8711-550d4a6e2989] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:55:46.491325 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8fdd180d-3026-437f-afdb-6ea267dd0ccd tempest-VolumesActionsTest-763750246 tempest-VolumesActionsTest-763750246-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:55:46.491827 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8fdd180d-3026-437f-afdb-6ea267dd0ccd tempest-VolumesActionsTest-763750246 tempest-VolumesActionsTest-763750246-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:55:46.492613 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8fdd180d-3026-437f-afdb-6ea267dd0ccd tempest-VolumesActionsTest-763750246 tempest-VolumesActionsTest-763750246-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:55:46.493073 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8fdd180d-3026-437f-afdb-6ea267dd0ccd tempest-VolumesActionsTest-763750246 tempest-VolumesActionsTest-763750246-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:55:46.519421 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-8fdd180d-3026-437f-afdb-6ea267dd0ccd tempest-VolumesActionsTest-763750246 tempest-VolumesActionsTest-763750246-project] [instance: 2ba83fff-f116-4f5e-8711-550d4a6e2989] Going to try to terminate instance {{(pid=138681) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 16 21:55:46.738781 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-8fdd180d-3026-437f-afdb-6ea267dd0ccd tempest-VolumesActionsTest-763750246 tempest-VolumesActionsTest-763750246-project] 10.222.0.16 "DELETE /compute/v2.1/servers/2ba83fff-f116-4f5e-8711-550d4a6e2989" status: 204 len: 0 microversion: 2.1 time: 0.258718 Jul 16 21:55:46.739407 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3132/6258] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:55:46 2022] DELETE /compute/v2.1/servers/2ba83fff-f116-4f5e-8711-550d4a6e2989 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 16 21:55:46.748100 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-d512358b-d47f-48ca-b7bc-818927766ed8 tempest-VolumesActionsTest-763750246 tempest-VolumesActionsTest-763750246-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:55:46.750003 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-d512358b-d47f-48ca-b7bc-818927766ed8 tempest-VolumesActionsTest-763750246 tempest-VolumesActionsTest-763750246-project] [instance: 2ba83fff-f116-4f5e-8711-550d4a6e2989] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:55:46.757361 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d512358b-d47f-48ca-b7bc-818927766ed8 tempest-VolumesActionsTest-763750246 tempest-VolumesActionsTest-763750246-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:55:46.757822 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d512358b-d47f-48ca-b7bc-818927766ed8 tempest-VolumesActionsTest-763750246 tempest-VolumesActionsTest-763750246-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:55:46.758625 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d512358b-d47f-48ca-b7bc-818927766ed8 tempest-VolumesActionsTest-763750246 tempest-VolumesActionsTest-763750246-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:55:46.759124 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d512358b-d47f-48ca-b7bc-818927766ed8 tempest-VolumesActionsTest-763750246 tempest-VolumesActionsTest-763750246-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:55:46.895499 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-d512358b-d47f-48ca-b7bc-818927766ed8 tempest-VolumesActionsTest-763750246 tempest-VolumesActionsTest-763750246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2331af9efc5e4af7a901621b7590e80a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13ce121469a145a4b00edd3a082413a4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:55:46.903082 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-d512358b-d47f-48ca-b7bc-818927766ed8 tempest-VolumesActionsTest-763750246 tempest-VolumesActionsTest-763750246-project] 10.222.0.16 "GET /compute/v2.1/servers/2ba83fff-f116-4f5e-8711-550d4a6e2989" status: 200 len: 1502 microversion: 2.1 time: 0.159237 Jul 16 21:55:46.903645 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3127/6259] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:55:46 2022] GET /compute/v2.1/servers/2ba83fff-f116-4f5e-8711-550d4a6e2989 => generated 1502 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:55:47.921944 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-8fb80396-8466-45b5-8cd9-336e490414ed tempest-VolumesActionsTest-763750246 tempest-VolumesActionsTest-763750246-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:55:47.922671 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-8fb80396-8466-45b5-8cd9-336e490414ed tempest-VolumesActionsTest-763750246 tempest-VolumesActionsTest-763750246-project] [instance: 2ba83fff-f116-4f5e-8711-550d4a6e2989] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:55:47.929979 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8fb80396-8466-45b5-8cd9-336e490414ed tempest-VolumesActionsTest-763750246 tempest-VolumesActionsTest-763750246-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:55:47.930421 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8fb80396-8466-45b5-8cd9-336e490414ed tempest-VolumesActionsTest-763750246 tempest-VolumesActionsTest-763750246-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:55:47.931254 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8fb80396-8466-45b5-8cd9-336e490414ed tempest-VolumesActionsTest-763750246 tempest-VolumesActionsTest-763750246-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:55:47.931714 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8fb80396-8466-45b5-8cd9-336e490414ed tempest-VolumesActionsTest-763750246 tempest-VolumesActionsTest-763750246-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:55:48.063454 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-8fb80396-8466-45b5-8cd9-336e490414ed tempest-VolumesActionsTest-763750246 tempest-VolumesActionsTest-763750246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2331af9efc5e4af7a901621b7590e80a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13ce121469a145a4b00edd3a082413a4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:55:48.071157 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-8fb80396-8466-45b5-8cd9-336e490414ed tempest-VolumesActionsTest-763750246 tempest-VolumesActionsTest-763750246-project] 10.222.0.16 "GET /compute/v2.1/servers/2ba83fff-f116-4f5e-8711-550d4a6e2989" status: 200 len: 1502 microversion: 2.1 time: 0.154736 Jul 16 21:55:48.071939 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3133/6260] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:55:47 2022] GET /compute/v2.1/servers/2ba83fff-f116-4f5e-8711-550d4a6e2989 => generated 1502 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:55:48.830504 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-d86abd77-2b1d-46bb-afb4-040777413ecd req-2298ba7f-9b71-48e4-9c95-fcb406c74c8c service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "2ba83fff-f116-4f5e-8711-550d4a6e2989", "name": "network-vif-deleted", "tag": "bdf82442-2fcd-4ffc-801d-5a1d5ea61c9f"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:55:48.839184 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-d86abd77-2b1d-46bb-afb4-040777413ecd req-2298ba7f-9b71-48e4-9c95-fcb406c74c8c service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:55:48.839634 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-d86abd77-2b1d-46bb-afb4-040777413ecd req-2298ba7f-9b71-48e4-9c95-fcb406c74c8c service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:55:48.851256 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-d86abd77-2b1d-46bb-afb4-040777413ecd req-2298ba7f-9b71-48e4-9c95-fcb406c74c8c service nova] Creating event network-vif-deleted:bdf82442-2fcd-4ffc-801d-5a1d5ea61c9f for instance 2ba83fff-f116-4f5e-8711-550d4a6e2989 on nch1-master-1 Jul 16 21:55:48.856370 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-d86abd77-2b1d-46bb-afb4-040777413ecd req-2298ba7f-9b71-48e4-9c95-fcb406c74c8c service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.029710 Jul 16 21:55:48.856977 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3128/6261] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:55:48 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 31 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:55:49.090107 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-eeaa9f88-c044-42ea-9325-b915c4114816 tempest-VolumesActionsTest-763750246 tempest-VolumesActionsTest-763750246-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:55:49.090736 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-eeaa9f88-c044-42ea-9325-b915c4114816 tempest-VolumesActionsTest-763750246 tempest-VolumesActionsTest-763750246-project] [instance: 2ba83fff-f116-4f5e-8711-550d4a6e2989] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:55:49.098114 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-eeaa9f88-c044-42ea-9325-b915c4114816 tempest-VolumesActionsTest-763750246 tempest-VolumesActionsTest-763750246-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:55:49.098584 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-eeaa9f88-c044-42ea-9325-b915c4114816 tempest-VolumesActionsTest-763750246 tempest-VolumesActionsTest-763750246-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:55:49.099404 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-eeaa9f88-c044-42ea-9325-b915c4114816 tempest-VolumesActionsTest-763750246 tempest-VolumesActionsTest-763750246-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:55:49.099871 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-eeaa9f88-c044-42ea-9325-b915c4114816 tempest-VolumesActionsTest-763750246 tempest-VolumesActionsTest-763750246-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:55:49.132514 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-eeaa9f88-c044-42ea-9325-b915c4114816 tempest-VolumesActionsTest-763750246 tempest-VolumesActionsTest-763750246-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:55:49.132988 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-eeaa9f88-c044-42ea-9325-b915c4114816 tempest-VolumesActionsTest-763750246 tempest-VolumesActionsTest-763750246-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:55:49.133456 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.objects.instance [None req-eeaa9f88-c044-42ea-9325-b915c4114816 tempest-VolumesActionsTest-763750246 tempest-VolumesActionsTest-763750246-project] Lazy-loading 'fault' on Instance uuid 2ba83fff-f116-4f5e-8711-550d4a6e2989 {{(pid=138681) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:55:49.175332 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-eeaa9f88-c044-42ea-9325-b915c4114816 tempest-VolumesActionsTest-763750246 tempest-VolumesActionsTest-763750246-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2331af9efc5e4af7a901621b7590e80a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '13ce121469a145a4b00edd3a082413a4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:55:49.183213 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-eeaa9f88-c044-42ea-9325-b915c4114816 tempest-VolumesActionsTest-763750246 tempest-VolumesActionsTest-763750246-project] 10.222.0.16 "GET /compute/v2.1/servers/2ba83fff-f116-4f5e-8711-550d4a6e2989" status: 200 len: 1303 microversion: 2.1 time: 0.097500 Jul 16 21:55:49.183788 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3134/6262] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:55:49 2022] GET /compute/v2.1/servers/2ba83fff-f116-4f5e-8711-550d4a6e2989 => generated 1303 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:55:50.201711 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-84a994a5-573b-4bee-9b04-abbbeabb5a8b tempest-VolumesActionsTest-763750246 tempest-VolumesActionsTest-763750246-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:55:50.202420 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-84a994a5-573b-4bee-9b04-abbbeabb5a8b tempest-VolumesActionsTest-763750246 tempest-VolumesActionsTest-763750246-project] [instance: 2ba83fff-f116-4f5e-8711-550d4a6e2989] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:55:50.209193 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-84a994a5-573b-4bee-9b04-abbbeabb5a8b tempest-VolumesActionsTest-763750246 tempest-VolumesActionsTest-763750246-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:55:50.209651 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-84a994a5-573b-4bee-9b04-abbbeabb5a8b tempest-VolumesActionsTest-763750246 tempest-VolumesActionsTest-763750246-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:55:50.210422 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-84a994a5-573b-4bee-9b04-abbbeabb5a8b tempest-VolumesActionsTest-763750246 tempest-VolumesActionsTest-763750246-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:55:50.210865 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-84a994a5-573b-4bee-9b04-abbbeabb5a8b tempest-VolumesActionsTest-763750246 tempest-VolumesActionsTest-763750246-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:55:50.225718 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-84a994a5-573b-4bee-9b04-abbbeabb5a8b tempest-VolumesActionsTest-763750246 tempest-VolumesActionsTest-763750246-project] HTTP exception thrown: Instance 2ba83fff-f116-4f5e-8711-550d4a6e2989 could not be found. Jul 16 21:55:50.227445 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-84a994a5-573b-4bee-9b04-abbbeabb5a8b tempest-VolumesActionsTest-763750246 tempest-VolumesActionsTest-763750246-project] Request method failure captured: Jul 16 21:55:50.227445 ncdv-master-1 devstack@n-api.service[138680]: request: GET /compute/v2.1/servers/2ba83fff-f116-4f5e-8711-550d4a6e2989 HTTP/1.1 Jul 16 21:55:50.227445 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:55:50.227445 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:55:50.227445 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:55:50.227445 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 0 Jul 16 21:55:50.227445 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:55:50.227445 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:55:50.227445 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:55:50.227445 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:55:50.227445 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0zPQGGDhq7BEk7v5pwwvMN2BjlhAcdvadyDM6tZACy4oneh7eODhJeDvWHrRP8H6okYz_2lD233hTuZyA1DHwpG-4J6ns9zAUYrHQnMxjdTc-q-XPL6uwjmWMI0EpTRlqMocc6PWeP6MbkGylZCbMfaXEKuSx8kQcUQvVzfJ4GIzL1g Jul 16 21:55:50.227445 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:55:50.227445 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:55:50.227445 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:55:50.227445 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:55:50.227445 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:55:50.227445 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:55:50.227445 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: 13ce121469a145a4b00edd3a082413a4 Jul 16 21:55:50.227445 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-VolumesActionsTest-763750246 Jul 16 21:55:50.227445 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:55:50.227445 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:55:50.227445 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/13ce121469a145a4b00edd3a082413a4"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/13ce121469a145a4b00edd3a082413a4"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_13ce121469a145a4b00edd3a082413a4"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/13ce121469a145a4b00edd3a082413a4"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:55:50.227445 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-VolumesActionsTest-763750246 Jul 16 21:55:50.227445 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: 13ce121469a145a4b00edd3a082413a4 Jul 16 21:55:50.227445 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-VolumesActionsTest-763750246 Jul 16 21:55:50.227445 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-VolumesActionsTest-763750246-project Jul 16 21:55:50.227445 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:55:50.227445 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:55:50.227445 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: 2331af9efc5e4af7a901621b7590e80a Jul 16 21:55:50.227445 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-VolumesActionsTest-763750246-project Jul 16 21:55:50.227445 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 21:55:50.227445 ncdv-master-1 devstack@n-api.service[138680]: exception: Instance 2ba83fff-f116-4f5e-8711-550d4a6e2989 could not be found. Jul 16 21:55:50.227445 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': '2ba83fff-f116-4f5e-8711-550d4a6e2989'} Jul 16 21:55:50.227445 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:55:50.227445 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:55:50.227445 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:55:50.227445 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:55:50.227445 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:55:50.227445 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:55:50.227445 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:55:50.227445 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:55:50.227445 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:55:50.227445 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:55:50.227445 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 2ba83fff-f116-4f5e-8711-550d4a6e2989 could not be found. Jul 16 21:55:50.227445 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:55:50.227445 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:55:50.227445 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:55:50.227445 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:55:50.227445 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:55:50.227445 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:55:50.227445 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:55:50.227445 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:55:50.227445 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:55:50.227445 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:55:50.227445 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:55:50.227445 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:55:50.227445 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:55:50.227445 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:55:50.227445 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:55:50.227445 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:55:50.227445 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 2ba83fff-f116-4f5e-8711-550d4a6e2989 could not be found. Jul 16 21:55:50.227445 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:55:50.227445 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:55:50.227445 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:55:50.238290 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:55:50.238290 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:55:50.238290 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:55:50.238290 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:55:50.238290 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:55:50.238290 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 2ba83fff-f116-4f5e-8711-550d4a6e2989 could not be found. Jul 16 21:55:50.238290 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:55:50.238290 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-84a994a5-573b-4bee-9b04-abbbeabb5a8b tempest-VolumesActionsTest-763750246 tempest-VolumesActionsTest-763750246-project] Returning 404 to user: Instance 2ba83fff-f116-4f5e-8711-550d4a6e2989 could not be found. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:55:50.238290 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-84a994a5-573b-4bee-9b04-abbbeabb5a8b tempest-VolumesActionsTest-763750246 tempest-VolumesActionsTest-763750246-project] 10.222.0.16 "GET /compute/v2.1/servers/2ba83fff-f116-4f5e-8711-550d4a6e2989" status: 404 len: 111 microversion: 2.1 time: 0.032334 Jul 16 21:55:50.238290 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3129/6263] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:55:50 2022] GET /compute/v2.1/servers/2ba83fff-f116-4f5e-8711-550d4a6e2989 => generated 111 bytes in 34 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:56:24.949573 ncdv-master-1 devstack@n-api.service[138680]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jul 16 21:56:24.950266 ncdv-master-1 devstack@n-api.service[138680]: Sat Jul 16 21:56:24 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Jul 16 21:56:26.099673 ncdv-master-1 devstack@n-api.service[138681]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jul 16 21:56:26.100867 ncdv-master-1 devstack@n-api.service[138681]: Sat Jul 16 21:56:26 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Jul 16 21:57:36.633262 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_api:SELECT=38,nova_api:INSERT=3,nova_api:UPDATE=1 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:57:36.634103 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=66,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:57:36.770700 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-2bd95696-4fdc-4d46-baeb-6ad13e22574c tempest-ServersAdminNegativeTestJSON-211662210 tempest-ServersAdminNegativeTestJSON-211662210-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServersAdminNegativeTestJSON-server-705335730", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452"}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:57:36.820273 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.network.neutron [None req-2bd95696-4fdc-4d46-baeb-6ad13e22574c tempest-ServersAdminNegativeTestJSON-211662210 tempest-ServersAdminNegativeTestJSON-211662210-project] validate_networks() for None {{(pid=138681) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 16 21:57:37.093355 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-2bd95696-4fdc-4d46-baeb-6ad13e22574c tempest-ServersAdminNegativeTestJSON-211662210 tempest-ServersAdminNegativeTestJSON-211662210-project] Flavor limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 16 21:57:37.093799 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-2bd95696-4fdc-4d46-baeb-6ad13e22574c tempest-ServersAdminNegativeTestJSON-211662210 tempest-ServersAdminNegativeTestJSON-211662210-project] Image limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 16 21:57:37.094323 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-2bd95696-4fdc-4d46-baeb-6ad13e22574c tempest-ServersAdminNegativeTestJSON-211662210 tempest-ServersAdminNegativeTestJSON-211662210-project] Flavor pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 16 21:57:37.094751 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-2bd95696-4fdc-4d46-baeb-6ad13e22574c tempest-ServersAdminNegativeTestJSON-211662210 tempest-ServersAdminNegativeTestJSON-211662210-project] Image pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 16 21:57:37.095219 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-2bd95696-4fdc-4d46-baeb-6ad13e22574c tempest-ServersAdminNegativeTestJSON-211662210 tempest-ServersAdminNegativeTestJSON-211662210-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 16 21:57:37.105039 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2bd95696-4fdc-4d46-baeb-6ad13e22574c tempest-ServersAdminNegativeTestJSON-211662210 tempest-ServersAdminNegativeTestJSON-211662210-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:57:37.105503 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2bd95696-4fdc-4d46-baeb-6ad13e22574c tempest-ServersAdminNegativeTestJSON-211662210 tempest-ServersAdminNegativeTestJSON-211662210-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:57:37.106321 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2bd95696-4fdc-4d46-baeb-6ad13e22574c tempest-ServersAdminNegativeTestJSON-211662210 tempest-ServersAdminNegativeTestJSON-211662210-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:57:37.106779 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2bd95696-4fdc-4d46-baeb-6ad13e22574c tempest-ServersAdminNegativeTestJSON-211662210 tempest-ServersAdminNegativeTestJSON-211662210-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:57:37.123961 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-2bd95696-4fdc-4d46-baeb-6ad13e22574c tempest-ServersAdminNegativeTestJSON-211662210 tempest-ServersAdminNegativeTestJSON-211662210-project] Getting quotas for project 46b1acf286d840738d93acc4ee506029. Resources: {'ram', 'instances', 'cores'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:57:37.129449 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-2bd95696-4fdc-4d46-baeb-6ad13e22574c tempest-ServersAdminNegativeTestJSON-211662210 tempest-ServersAdminNegativeTestJSON-211662210-project] Getting quotas for user 92d1d7ddc009428bb340e9c6f20aa7f3 and project 46b1acf286d840738d93acc4ee506029. Resources: {'ram', 'instances', 'cores'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:57:37.141782 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-2bd95696-4fdc-4d46-baeb-6ad13e22574c tempest-ServersAdminNegativeTestJSON-211662210 tempest-ServersAdminNegativeTestJSON-211662210-project] Going to run 1 instances... {{(pid=138681) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 16 21:57:37.144069 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-2bd95696-4fdc-4d46-baeb-6ad13e22574c tempest-ServersAdminNegativeTestJSON-211662210 tempest-ServersAdminNegativeTestJSON-211662210-project] [instance: e317fe1f-3367-4d70-a4c7-e357b93e3a49] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138681) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 16 21:57:37.308802 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-2bd95696-4fdc-4d46-baeb-6ad13e22574c tempest-ServersAdminNegativeTestJSON-211662210 tempest-ServersAdminNegativeTestJSON-211662210-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.677505 Jul 16 21:57:37.309371 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3135/6264] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 21:57:36 2022] POST /compute/v2.1/servers => generated 380 bytes in 679 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 16 21:57:37.320854 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-f9405ea8-0b07-43d9-8f00-6e077bdb7d72 tempest-ServersAdminNegativeTestJSON-211662210 tempest-ServersAdminNegativeTestJSON-211662210-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:57:37.321480 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-f9405ea8-0b07-43d9-8f00-6e077bdb7d72 tempest-ServersAdminNegativeTestJSON-211662210 tempest-ServersAdminNegativeTestJSON-211662210-project] [instance: e317fe1f-3367-4d70-a4c7-e357b93e3a49] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:57:37.323857 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_api:SELECT=22 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:57:37.324553 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=44 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:57:37.387196 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-f9405ea8-0b07-43d9-8f00-6e077bdb7d72 tempest-ServersAdminNegativeTestJSON-211662210 tempest-ServersAdminNegativeTestJSON-211662210-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92d1d7ddc009428bb340e9c6f20aa7f3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '46b1acf286d840738d93acc4ee506029', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:57:37.397095 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-f9405ea8-0b07-43d9-8f00-6e077bdb7d72 tempest-ServersAdminNegativeTestJSON-211662210 tempest-ServersAdminNegativeTestJSON-211662210-project] 10.222.0.16 "GET /compute/v2.1/servers/e317fe1f-3367-4d70-a4c7-e357b93e3a49" status: 200 len: 1221 microversion: 2.1 time: 0.081059 Jul 16 21:57:37.398101 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3130/6265] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:57:37 2022] GET /compute/v2.1/servers/e317fe1f-3367-4d70-a4c7-e357b93e3a49 => generated 1221 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:57:38.416897 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-5cacb108-b7b6-41bc-8142-9563b8351600 tempest-ServersAdminNegativeTestJSON-211662210 tempest-ServersAdminNegativeTestJSON-211662210-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:57:38.417566 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-5cacb108-b7b6-41bc-8142-9563b8351600 tempest-ServersAdminNegativeTestJSON-211662210 tempest-ServersAdminNegativeTestJSON-211662210-project] [instance: e317fe1f-3367-4d70-a4c7-e357b93e3a49] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:57:38.477947 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-5cacb108-b7b6-41bc-8142-9563b8351600 tempest-ServersAdminNegativeTestJSON-211662210 tempest-ServersAdminNegativeTestJSON-211662210-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92d1d7ddc009428bb340e9c6f20aa7f3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '46b1acf286d840738d93acc4ee506029', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:57:38.485763 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-5cacb108-b7b6-41bc-8142-9563b8351600 tempest-ServersAdminNegativeTestJSON-211662210 tempest-ServersAdminNegativeTestJSON-211662210-project] 10.222.0.16 "GET /compute/v2.1/servers/e317fe1f-3367-4d70-a4c7-e357b93e3a49" status: 200 len: 1221 microversion: 2.1 time: 0.074192 Jul 16 21:57:38.486381 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3136/6266] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:57:38 2022] GET /compute/v2.1/servers/e317fe1f-3367-4d70-a4c7-e357b93e3a49 => generated 1221 bytes in 76 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:57:39.503437 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-405bced5-6e11-4a84-9e77-470799036133 tempest-ServersAdminNegativeTestJSON-211662210 tempest-ServersAdminNegativeTestJSON-211662210-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:57:39.504528 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-405bced5-6e11-4a84-9e77-470799036133 tempest-ServersAdminNegativeTestJSON-211662210 tempest-ServersAdminNegativeTestJSON-211662210-project] [instance: e317fe1f-3367-4d70-a4c7-e357b93e3a49] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:57:39.511820 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-405bced5-6e11-4a84-9e77-470799036133 tempest-ServersAdminNegativeTestJSON-211662210 tempest-ServersAdminNegativeTestJSON-211662210-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:57:39.512275 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-405bced5-6e11-4a84-9e77-470799036133 tempest-ServersAdminNegativeTestJSON-211662210 tempest-ServersAdminNegativeTestJSON-211662210-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:57:39.513050 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-405bced5-6e11-4a84-9e77-470799036133 tempest-ServersAdminNegativeTestJSON-211662210 tempest-ServersAdminNegativeTestJSON-211662210-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:57:39.513488 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-405bced5-6e11-4a84-9e77-470799036133 tempest-ServersAdminNegativeTestJSON-211662210 tempest-ServersAdminNegativeTestJSON-211662210-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:57:39.576229 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-405bced5-6e11-4a84-9e77-470799036133 tempest-ServersAdminNegativeTestJSON-211662210 tempest-ServersAdminNegativeTestJSON-211662210-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92d1d7ddc009428bb340e9c6f20aa7f3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '46b1acf286d840738d93acc4ee506029', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:57:39.584428 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-405bced5-6e11-4a84-9e77-470799036133 tempest-ServersAdminNegativeTestJSON-211662210 tempest-ServersAdminNegativeTestJSON-211662210-project] 10.222.0.16 "GET /compute/v2.1/servers/e317fe1f-3367-4d70-a4c7-e357b93e3a49" status: 200 len: 1225 microversion: 2.1 time: 0.085006 Jul 16 21:57:39.584993 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3131/6267] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:57:39 2022] GET /compute/v2.1/servers/e317fe1f-3367-4d70-a4c7-e357b93e3a49 => generated 1225 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:57:40.601419 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-1f86a15d-058f-4861-a7bc-7916e51661e0 tempest-ServersAdminNegativeTestJSON-211662210 tempest-ServersAdminNegativeTestJSON-211662210-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:57:40.602058 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-1f86a15d-058f-4861-a7bc-7916e51661e0 tempest-ServersAdminNegativeTestJSON-211662210 tempest-ServersAdminNegativeTestJSON-211662210-project] [instance: e317fe1f-3367-4d70-a4c7-e357b93e3a49] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:57:40.609262 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1f86a15d-058f-4861-a7bc-7916e51661e0 tempest-ServersAdminNegativeTestJSON-211662210 tempest-ServersAdminNegativeTestJSON-211662210-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:57:40.609732 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1f86a15d-058f-4861-a7bc-7916e51661e0 tempest-ServersAdminNegativeTestJSON-211662210 tempest-ServersAdminNegativeTestJSON-211662210-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:57:40.610628 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1f86a15d-058f-4861-a7bc-7916e51661e0 tempest-ServersAdminNegativeTestJSON-211662210 tempest-ServersAdminNegativeTestJSON-211662210-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:57:40.611121 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1f86a15d-058f-4861-a7bc-7916e51661e0 tempest-ServersAdminNegativeTestJSON-211662210 tempest-ServersAdminNegativeTestJSON-211662210-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:57:40.677054 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-1f86a15d-058f-4861-a7bc-7916e51661e0 tempest-ServersAdminNegativeTestJSON-211662210 tempest-ServersAdminNegativeTestJSON-211662210-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92d1d7ddc009428bb340e9c6f20aa7f3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '46b1acf286d840738d93acc4ee506029', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:57:40.685286 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-1f86a15d-058f-4861-a7bc-7916e51661e0 tempest-ServersAdminNegativeTestJSON-211662210 tempest-ServersAdminNegativeTestJSON-211662210-project] 10.222.0.16 "GET /compute/v2.1/servers/e317fe1f-3367-4d70-a4c7-e357b93e3a49" status: 200 len: 1291 microversion: 2.1 time: 0.088531 Jul 16 21:57:40.685856 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3137/6268] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:57:40 2022] GET /compute/v2.1/servers/e317fe1f-3367-4d70-a4c7-e357b93e3a49 => generated 1291 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:57:41.704149 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-a8adb0a2-57ce-4cbd-ab7a-1e674f8a3a10 tempest-ServersAdminNegativeTestJSON-211662210 tempest-ServersAdminNegativeTestJSON-211662210-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:57:41.704819 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-a8adb0a2-57ce-4cbd-ab7a-1e674f8a3a10 tempest-ServersAdminNegativeTestJSON-211662210 tempest-ServersAdminNegativeTestJSON-211662210-project] [instance: e317fe1f-3367-4d70-a4c7-e357b93e3a49] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:57:41.712439 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a8adb0a2-57ce-4cbd-ab7a-1e674f8a3a10 tempest-ServersAdminNegativeTestJSON-211662210 tempest-ServersAdminNegativeTestJSON-211662210-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:57:41.712906 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a8adb0a2-57ce-4cbd-ab7a-1e674f8a3a10 tempest-ServersAdminNegativeTestJSON-211662210 tempest-ServersAdminNegativeTestJSON-211662210-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:57:41.713684 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a8adb0a2-57ce-4cbd-ab7a-1e674f8a3a10 tempest-ServersAdminNegativeTestJSON-211662210 tempest-ServersAdminNegativeTestJSON-211662210-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:57:41.714138 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a8adb0a2-57ce-4cbd-ab7a-1e674f8a3a10 tempest-ServersAdminNegativeTestJSON-211662210 tempest-ServersAdminNegativeTestJSON-211662210-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:57:41.848851 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-a8adb0a2-57ce-4cbd-ab7a-1e674f8a3a10 tempest-ServersAdminNegativeTestJSON-211662210 tempest-ServersAdminNegativeTestJSON-211662210-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92d1d7ddc009428bb340e9c6f20aa7f3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '46b1acf286d840738d93acc4ee506029', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:57:41.856590 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-a8adb0a2-57ce-4cbd-ab7a-1e674f8a3a10 tempest-ServersAdminNegativeTestJSON-211662210 tempest-ServersAdminNegativeTestJSON-211662210-project] 10.222.0.16 "GET /compute/v2.1/servers/e317fe1f-3367-4d70-a4c7-e357b93e3a49" status: 200 len: 1321 microversion: 2.1 time: 0.156402 Jul 16 21:57:41.857177 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3132/6269] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:57:41 2022] GET /compute/v2.1/servers/e317fe1f-3367-4d70-a4c7-e357b93e3a49 => generated 1321 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:57:42.303071 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-ce226de8-0f57-400f-8e9d-fd0f0d273764 req-1d60fe69-1f3f-481c-8364-2e12a9fbca71 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "e317fe1f-3367-4d70-a4c7-e357b93e3a49", "tag": "31c8a385-ad6a-44cf-b2a9-b5e40a468f8d"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:57:42.314116 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-ce226de8-0f57-400f-8e9d-fd0f0d273764 req-1d60fe69-1f3f-481c-8364-2e12a9fbca71 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:57:42.314663 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-ce226de8-0f57-400f-8e9d-fd0f0d273764 req-1d60fe69-1f3f-481c-8364-2e12a9fbca71 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:57:42.327876 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-ce226de8-0f57-400f-8e9d-fd0f0d273764 req-1d60fe69-1f3f-481c-8364-2e12a9fbca71 service nova] Creating event network-changed:31c8a385-ad6a-44cf-b2a9-b5e40a468f8d for instance e317fe1f-3367-4d70-a4c7-e357b93e3a49 on nch1-master-1 Jul 16 21:57:42.333613 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-ce226de8-0f57-400f-8e9d-fd0f0d273764 req-1d60fe69-1f3f-481c-8364-2e12a9fbca71 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.034759 Jul 16 21:57:42.334417 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3138/6270] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:57:42 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 16 21:57:42.874027 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-9f037981-d35b-412a-b104-b079b254f7c1 tempest-ServersAdminNegativeTestJSON-211662210 tempest-ServersAdminNegativeTestJSON-211662210-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:57:42.874707 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-9f037981-d35b-412a-b104-b079b254f7c1 tempest-ServersAdminNegativeTestJSON-211662210 tempest-ServersAdminNegativeTestJSON-211662210-project] [instance: e317fe1f-3367-4d70-a4c7-e357b93e3a49] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:57:42.882066 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9f037981-d35b-412a-b104-b079b254f7c1 tempest-ServersAdminNegativeTestJSON-211662210 tempest-ServersAdminNegativeTestJSON-211662210-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:57:42.882520 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9f037981-d35b-412a-b104-b079b254f7c1 tempest-ServersAdminNegativeTestJSON-211662210 tempest-ServersAdminNegativeTestJSON-211662210-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:57:42.883318 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9f037981-d35b-412a-b104-b079b254f7c1 tempest-ServersAdminNegativeTestJSON-211662210 tempest-ServersAdminNegativeTestJSON-211662210-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:57:42.883761 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9f037981-d35b-412a-b104-b079b254f7c1 tempest-ServersAdminNegativeTestJSON-211662210 tempest-ServersAdminNegativeTestJSON-211662210-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:57:43.033027 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-9f037981-d35b-412a-b104-b079b254f7c1 tempest-ServersAdminNegativeTestJSON-211662210 tempest-ServersAdminNegativeTestJSON-211662210-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92d1d7ddc009428bb340e9c6f20aa7f3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '46b1acf286d840738d93acc4ee506029', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:57:43.041202 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-9f037981-d35b-412a-b104-b079b254f7c1 tempest-ServersAdminNegativeTestJSON-211662210 tempest-ServersAdminNegativeTestJSON-211662210-project] 10.222.0.16 "GET /compute/v2.1/servers/e317fe1f-3367-4d70-a4c7-e357b93e3a49" status: 200 len: 1321 microversion: 2.1 time: 0.171285 Jul 16 21:57:43.041775 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3133/6271] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:57:42 2022] GET /compute/v2.1/servers/e317fe1f-3367-4d70-a4c7-e357b93e3a49 => generated 1321 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:57:44.060153 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-291f7077-81a5-4fe0-9823-d472d6915917 tempest-ServersAdminNegativeTestJSON-211662210 tempest-ServersAdminNegativeTestJSON-211662210-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:57:44.060878 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-291f7077-81a5-4fe0-9823-d472d6915917 tempest-ServersAdminNegativeTestJSON-211662210 tempest-ServersAdminNegativeTestJSON-211662210-project] [instance: e317fe1f-3367-4d70-a4c7-e357b93e3a49] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:57:44.067873 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-291f7077-81a5-4fe0-9823-d472d6915917 tempest-ServersAdminNegativeTestJSON-211662210 tempest-ServersAdminNegativeTestJSON-211662210-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:57:44.068439 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-291f7077-81a5-4fe0-9823-d472d6915917 tempest-ServersAdminNegativeTestJSON-211662210 tempest-ServersAdminNegativeTestJSON-211662210-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:57:44.069221 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-291f7077-81a5-4fe0-9823-d472d6915917 tempest-ServersAdminNegativeTestJSON-211662210 tempest-ServersAdminNegativeTestJSON-211662210-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:57:44.069679 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-291f7077-81a5-4fe0-9823-d472d6915917 tempest-ServersAdminNegativeTestJSON-211662210 tempest-ServersAdminNegativeTestJSON-211662210-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:57:44.204921 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-291f7077-81a5-4fe0-9823-d472d6915917 tempest-ServersAdminNegativeTestJSON-211662210 tempest-ServersAdminNegativeTestJSON-211662210-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92d1d7ddc009428bb340e9c6f20aa7f3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '46b1acf286d840738d93acc4ee506029', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:57:44.212536 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-291f7077-81a5-4fe0-9823-d472d6915917 tempest-ServersAdminNegativeTestJSON-211662210 tempest-ServersAdminNegativeTestJSON-211662210-project] 10.222.0.16 "GET /compute/v2.1/servers/e317fe1f-3367-4d70-a4c7-e357b93e3a49" status: 200 len: 1321 microversion: 2.1 time: 0.157574 Jul 16 21:57:44.213110 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3139/6272] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:57:44 2022] GET /compute/v2.1/servers/e317fe1f-3367-4d70-a4c7-e357b93e3a49 => generated 1321 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:57:45.229841 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-e09596cf-0187-4a42-a2cd-3ae84c819faa tempest-ServersAdminNegativeTestJSON-211662210 tempest-ServersAdminNegativeTestJSON-211662210-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:57:45.230364 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-e09596cf-0187-4a42-a2cd-3ae84c819faa tempest-ServersAdminNegativeTestJSON-211662210 tempest-ServersAdminNegativeTestJSON-211662210-project] [instance: e317fe1f-3367-4d70-a4c7-e357b93e3a49] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:57:45.237378 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e09596cf-0187-4a42-a2cd-3ae84c819faa tempest-ServersAdminNegativeTestJSON-211662210 tempest-ServersAdminNegativeTestJSON-211662210-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:57:45.237833 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e09596cf-0187-4a42-a2cd-3ae84c819faa tempest-ServersAdminNegativeTestJSON-211662210 tempest-ServersAdminNegativeTestJSON-211662210-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:57:45.239104 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e09596cf-0187-4a42-a2cd-3ae84c819faa tempest-ServersAdminNegativeTestJSON-211662210 tempest-ServersAdminNegativeTestJSON-211662210-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:57:45.239555 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e09596cf-0187-4a42-a2cd-3ae84c819faa tempest-ServersAdminNegativeTestJSON-211662210 tempest-ServersAdminNegativeTestJSON-211662210-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:57:45.369113 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-e09596cf-0187-4a42-a2cd-3ae84c819faa tempest-ServersAdminNegativeTestJSON-211662210 tempest-ServersAdminNegativeTestJSON-211662210-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92d1d7ddc009428bb340e9c6f20aa7f3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '46b1acf286d840738d93acc4ee506029', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:57:45.376595 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-e09596cf-0187-4a42-a2cd-3ae84c819faa tempest-ServersAdminNegativeTestJSON-211662210 tempest-ServersAdminNegativeTestJSON-211662210-project] 10.222.0.16 "GET /compute/v2.1/servers/e317fe1f-3367-4d70-a4c7-e357b93e3a49" status: 200 len: 1321 microversion: 2.1 time: 0.150796 Jul 16 21:57:45.377197 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3134/6273] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:57:45 2022] GET /compute/v2.1/servers/e317fe1f-3367-4d70-a4c7-e357b93e3a49 => generated 1321 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:57:46.394209 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-b6790736-5ee2-496b-b270-76d980759542 tempest-ServersAdminNegativeTestJSON-211662210 tempest-ServersAdminNegativeTestJSON-211662210-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:57:46.394870 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-b6790736-5ee2-496b-b270-76d980759542 tempest-ServersAdminNegativeTestJSON-211662210 tempest-ServersAdminNegativeTestJSON-211662210-project] [instance: e317fe1f-3367-4d70-a4c7-e357b93e3a49] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:57:46.401862 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b6790736-5ee2-496b-b270-76d980759542 tempest-ServersAdminNegativeTestJSON-211662210 tempest-ServersAdminNegativeTestJSON-211662210-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:57:46.402320 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b6790736-5ee2-496b-b270-76d980759542 tempest-ServersAdminNegativeTestJSON-211662210 tempest-ServersAdminNegativeTestJSON-211662210-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:57:46.403120 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b6790736-5ee2-496b-b270-76d980759542 tempest-ServersAdminNegativeTestJSON-211662210 tempest-ServersAdminNegativeTestJSON-211662210-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:57:46.403570 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b6790736-5ee2-496b-b270-76d980759542 tempest-ServersAdminNegativeTestJSON-211662210 tempest-ServersAdminNegativeTestJSON-211662210-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:57:46.534205 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-b6790736-5ee2-496b-b270-76d980759542 tempest-ServersAdminNegativeTestJSON-211662210 tempest-ServersAdminNegativeTestJSON-211662210-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92d1d7ddc009428bb340e9c6f20aa7f3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '46b1acf286d840738d93acc4ee506029', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:57:46.542315 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-b6790736-5ee2-496b-b270-76d980759542 tempest-ServersAdminNegativeTestJSON-211662210 tempest-ServersAdminNegativeTestJSON-211662210-project] 10.222.0.16 "GET /compute/v2.1/servers/e317fe1f-3367-4d70-a4c7-e357b93e3a49" status: 200 len: 1321 microversion: 2.1 time: 0.152115 Jul 16 21:57:46.542878 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3140/6274] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:57:46 2022] GET /compute/v2.1/servers/e317fe1f-3367-4d70-a4c7-e357b93e3a49 => generated 1321 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:57:47.559759 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-520705fb-4797-4f94-bbd4-c4889c6b63af tempest-ServersAdminNegativeTestJSON-211662210 tempest-ServersAdminNegativeTestJSON-211662210-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:57:47.560371 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-520705fb-4797-4f94-bbd4-c4889c6b63af tempest-ServersAdminNegativeTestJSON-211662210 tempest-ServersAdminNegativeTestJSON-211662210-project] [instance: e317fe1f-3367-4d70-a4c7-e357b93e3a49] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:57:47.563107 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell0:SELECT=2 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:57:47.568170 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-520705fb-4797-4f94-bbd4-c4889c6b63af tempest-ServersAdminNegativeTestJSON-211662210 tempest-ServersAdminNegativeTestJSON-211662210-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:57:47.568622 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-520705fb-4797-4f94-bbd4-c4889c6b63af tempest-ServersAdminNegativeTestJSON-211662210 tempest-ServersAdminNegativeTestJSON-211662210-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:57:47.569386 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-520705fb-4797-4f94-bbd4-c4889c6b63af tempest-ServersAdminNegativeTestJSON-211662210 tempest-ServersAdminNegativeTestJSON-211662210-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:57:47.569823 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-520705fb-4797-4f94-bbd4-c4889c6b63af tempest-ServersAdminNegativeTestJSON-211662210 tempest-ServersAdminNegativeTestJSON-211662210-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:57:47.700472 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-520705fb-4797-4f94-bbd4-c4889c6b63af tempest-ServersAdminNegativeTestJSON-211662210 tempest-ServersAdminNegativeTestJSON-211662210-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92d1d7ddc009428bb340e9c6f20aa7f3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '46b1acf286d840738d93acc4ee506029', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:57:47.708732 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-520705fb-4797-4f94-bbd4-c4889c6b63af tempest-ServersAdminNegativeTestJSON-211662210 tempest-ServersAdminNegativeTestJSON-211662210-project] 10.222.0.16 "GET /compute/v2.1/servers/e317fe1f-3367-4d70-a4c7-e357b93e3a49" status: 200 len: 1321 microversion: 2.1 time: 0.152611 Jul 16 21:57:47.709311 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3135/6275] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:57:47 2022] GET /compute/v2.1/servers/e317fe1f-3367-4d70-a4c7-e357b93e3a49 => generated 1321 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:57:47.948362 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-7939fc05-1f0f-413f-90c0-83183a35c62e req-eff1e4c3-6ce4-4392-9d79-9db096533ba2 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "e317fe1f-3367-4d70-a4c7-e357b93e3a49", "name": "network-vif-plugged", "status": "completed", "tag": "31c8a385-ad6a-44cf-b2a9-b5e40a468f8d"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:57:47.953353 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell0:SELECT=2 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:57:47.959195 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-7939fc05-1f0f-413f-90c0-83183a35c62e req-eff1e4c3-6ce4-4392-9d79-9db096533ba2 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:57:47.959660 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-7939fc05-1f0f-413f-90c0-83183a35c62e req-eff1e4c3-6ce4-4392-9d79-9db096533ba2 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:57:47.970719 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-7939fc05-1f0f-413f-90c0-83183a35c62e req-eff1e4c3-6ce4-4392-9d79-9db096533ba2 service nova] Creating event network-vif-plugged:31c8a385-ad6a-44cf-b2a9-b5e40a468f8d for instance e317fe1f-3367-4d70-a4c7-e357b93e3a49 on nch1-master-1 Jul 16 21:57:47.976193 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-7939fc05-1f0f-413f-90c0-83183a35c62e req-eff1e4c3-6ce4-4392-9d79-9db096533ba2 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.032050 Jul 16 21:57:47.976783 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3141/6276] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:57:47 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:57:48.726126 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-7b82d450-2915-4cd2-b4ee-44999438595e tempest-ServersAdminNegativeTestJSON-211662210 tempest-ServersAdminNegativeTestJSON-211662210-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:57:48.726740 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-7b82d450-2915-4cd2-b4ee-44999438595e tempest-ServersAdminNegativeTestJSON-211662210 tempest-ServersAdminNegativeTestJSON-211662210-project] [instance: e317fe1f-3367-4d70-a4c7-e357b93e3a49] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:57:48.733911 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7b82d450-2915-4cd2-b4ee-44999438595e tempest-ServersAdminNegativeTestJSON-211662210 tempest-ServersAdminNegativeTestJSON-211662210-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:57:48.734392 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7b82d450-2915-4cd2-b4ee-44999438595e tempest-ServersAdminNegativeTestJSON-211662210 tempest-ServersAdminNegativeTestJSON-211662210-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:57:48.735194 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7b82d450-2915-4cd2-b4ee-44999438595e tempest-ServersAdminNegativeTestJSON-211662210 tempest-ServersAdminNegativeTestJSON-211662210-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:57:48.735649 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7b82d450-2915-4cd2-b4ee-44999438595e tempest-ServersAdminNegativeTestJSON-211662210 tempest-ServersAdminNegativeTestJSON-211662210-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:57:48.880406 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-7b82d450-2915-4cd2-b4ee-44999438595e tempest-ServersAdminNegativeTestJSON-211662210 tempest-ServersAdminNegativeTestJSON-211662210-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92d1d7ddc009428bb340e9c6f20aa7f3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '46b1acf286d840738d93acc4ee506029', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:57:48.888302 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-7b82d450-2915-4cd2-b4ee-44999438595e tempest-ServersAdminNegativeTestJSON-211662210 tempest-ServersAdminNegativeTestJSON-211662210-project] 10.222.0.16 "GET /compute/v2.1/servers/e317fe1f-3367-4d70-a4c7-e357b93e3a49" status: 200 len: 1321 microversion: 2.1 time: 0.166573 Jul 16 21:57:48.888882 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3136/6277] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:57:48 2022] GET /compute/v2.1/servers/e317fe1f-3367-4d70-a4c7-e357b93e3a49 => generated 1321 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:57:49.906282 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-4c5868ed-2544-470d-8bc6-077555428ed3 tempest-ServersAdminNegativeTestJSON-211662210 tempest-ServersAdminNegativeTestJSON-211662210-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:57:49.907002 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-4c5868ed-2544-470d-8bc6-077555428ed3 tempest-ServersAdminNegativeTestJSON-211662210 tempest-ServersAdminNegativeTestJSON-211662210-project] [instance: e317fe1f-3367-4d70-a4c7-e357b93e3a49] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:57:49.909375 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell0:SELECT=8 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:57:49.916346 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4c5868ed-2544-470d-8bc6-077555428ed3 tempest-ServersAdminNegativeTestJSON-211662210 tempest-ServersAdminNegativeTestJSON-211662210-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:57:49.917609 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4c5868ed-2544-470d-8bc6-077555428ed3 tempest-ServersAdminNegativeTestJSON-211662210 tempest-ServersAdminNegativeTestJSON-211662210-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:57:49.917609 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4c5868ed-2544-470d-8bc6-077555428ed3 tempest-ServersAdminNegativeTestJSON-211662210 tempest-ServersAdminNegativeTestJSON-211662210-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:57:49.918012 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4c5868ed-2544-470d-8bc6-077555428ed3 tempest-ServersAdminNegativeTestJSON-211662210 tempest-ServersAdminNegativeTestJSON-211662210-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:57:50.053204 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-4c5868ed-2544-470d-8bc6-077555428ed3 tempest-ServersAdminNegativeTestJSON-211662210 tempest-ServersAdminNegativeTestJSON-211662210-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92d1d7ddc009428bb340e9c6f20aa7f3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '46b1acf286d840738d93acc4ee506029', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:57:50.060857 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-4c5868ed-2544-470d-8bc6-077555428ed3 tempest-ServersAdminNegativeTestJSON-211662210 tempest-ServersAdminNegativeTestJSON-211662210-project] 10.222.0.16 "GET /compute/v2.1/servers/e317fe1f-3367-4d70-a4c7-e357b93e3a49" status: 200 len: 1470 microversion: 2.1 time: 0.158828 Jul 16 21:57:50.061442 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3142/6278] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:57:49 2022] GET /compute/v2.1/servers/e317fe1f-3367-4d70-a4c7-e357b93e3a49 => generated 1470 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:57:50.135468 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-c23ba9ff-228d-4317-a69d-cba0647675f4 tempest-ServersAdminNegativeTestJSON-1082789623 tempest-ServersAdminNegativeTestJSON-1082789623-project-admin] Calling method '.version_select at 0x7fdda2b1a3a0>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:57:50.200915 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-c23ba9ff-228d-4317-a69d-cba0647675f4 tempest-ServersAdminNegativeTestJSON-1082789623 tempest-ServersAdminNegativeTestJSON-1082789623-project-admin] 10.222.0.16 "GET /compute/v2.1/os-quota-sets/1ffacafb54b044fb8e92ffb08136d1d2" status: 200 len: 371 microversion: 2.1 time: 0.125299 Jul 16 21:57:50.206717 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3137/6279] 10.222.0.16 () {58 vars in 1241 bytes} [Sat Jul 16 21:57:50 2022] GET /compute/v2.1/os-quota-sets/1ffacafb54b044fb8e92ffb08136d1d2 => generated 371 bytes in 127 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:57:50.212450 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-9b5861a8-2863-4003-8780-028b4bebe18c tempest-ServersAdminNegativeTestJSON-1082789623 tempest-ServersAdminNegativeTestJSON-1082789623-project-admin] Action: 'create', calling method: >, body: {"flavor": {"ram": 51201, "vcpus": 1, "disk": 5, "name": "tempest-ServersAdminNegativeTestJSON-flavor-880938508", "id": 720895039, "os-flavor-access:is_public": "True"}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:57:50.214125 ncdv-master-1 devstack@n-api.service[138681]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-flavor-manage:create": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:57:50.214125 ncdv-master-1 devstack@n-api.service[138681]: warnings.warn(msg) Jul 16 21:57:50.314532 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-9b5861a8-2863-4003-8780-028b4bebe18c tempest-ServersAdminNegativeTestJSON-1082789623 tempest-ServersAdminNegativeTestJSON-1082789623-project-admin] 10.222.0.16 "POST /compute/v2.1/flavors" status: 200 len: 434 microversion: 2.1 time: 0.105731 Jul 16 21:57:50.315742 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3143/6280] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 21:57:50 2022] POST /compute/v2.1/flavors => generated 434 bytes in 108 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:57:50.326969 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-80eb11c1-bd9b-4584-baac-785864766a30 tempest-ServersAdminNegativeTestJSON-1082789623 tempest-ServersAdminNegativeTestJSON-1082789623-project-admin] Action: 'action', calling method: >, body: {"resize": {"flavorRef": "720895039"}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:57:50.328074 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-80eb11c1-bd9b-4584-baac-785864766a30 tempest-ServersAdminNegativeTestJSON-1082789623 tempest-ServersAdminNegativeTestJSON-1082789623-project-admin] [instance: e317fe1f-3367-4d70-a4c7-e357b93e3a49] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:57:50.336205 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-80eb11c1-bd9b-4584-baac-785864766a30 tempest-ServersAdminNegativeTestJSON-1082789623 tempest-ServersAdminNegativeTestJSON-1082789623-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:57:50.336663 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-80eb11c1-bd9b-4584-baac-785864766a30 tempest-ServersAdminNegativeTestJSON-1082789623 tempest-ServersAdminNegativeTestJSON-1082789623-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:57:50.337422 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-80eb11c1-bd9b-4584-baac-785864766a30 tempest-ServersAdminNegativeTestJSON-1082789623 tempest-ServersAdminNegativeTestJSON-1082789623-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:57:50.337863 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-80eb11c1-bd9b-4584-baac-785864766a30 tempest-ServersAdminNegativeTestJSON-1082789623 tempest-ServersAdminNegativeTestJSON-1082789623-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:57:50.524513 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-80eb11c1-bd9b-4584-baac-785864766a30 tempest-ServersAdminNegativeTestJSON-1082789623 tempest-ServersAdminNegativeTestJSON-1082789623-project-admin] Policy check for compute:servers:resize:cross_cell failed with credentials {'is_admin': True, 'user_id': '2b7eba2f41e342d39333caabc176932c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1ffacafb54b044fb8e92ffb08136d1d2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:57:50.531102 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-80eb11c1-bd9b-4584-baac-785864766a30 tempest-ServersAdminNegativeTestJSON-1082789623 tempest-ServersAdminNegativeTestJSON-1082789623-project-admin] [instance: e317fe1f-3367-4d70-a4c7-e357b93e3a49] Old instance type m1.heat2, new instance type tempest-ServersAdminNegativeTestJSON-flavor-880938508 {{(pid=138680) resize /opt/stack/nova/nova/compute/api.py:4166}} Jul 16 21:57:50.538467 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-80eb11c1-bd9b-4584-baac-785864766a30 tempest-ServersAdminNegativeTestJSON-1082789623 tempest-ServersAdminNegativeTestJSON-1082789623-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:57:50.538918 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-80eb11c1-bd9b-4584-baac-785864766a30 tempest-ServersAdminNegativeTestJSON-1082789623 tempest-ServersAdminNegativeTestJSON-1082789623-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:57:50.539751 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-80eb11c1-bd9b-4584-baac-785864766a30 tempest-ServersAdminNegativeTestJSON-1082789623 tempest-ServersAdminNegativeTestJSON-1082789623-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:57:50.540171 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-80eb11c1-bd9b-4584-baac-785864766a30 tempest-ServersAdminNegativeTestJSON-1082789623 tempest-ServersAdminNegativeTestJSON-1082789623-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:57:50.562539 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-80eb11c1-bd9b-4584-baac-785864766a30 tempest-ServersAdminNegativeTestJSON-1082789623 tempest-ServersAdminNegativeTestJSON-1082789623-project-admin] Getting quotas for project 46b1acf286d840738d93acc4ee506029. Resources: {'ram', 'cores'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:57:50.568335 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-80eb11c1-bd9b-4584-baac-785864766a30 tempest-ServersAdminNegativeTestJSON-1082789623 tempest-ServersAdminNegativeTestJSON-1082789623-project-admin] Getting quotas for user 92d1d7ddc009428bb340e9c6f20aa7f3 and project 46b1acf286d840738d93acc4ee506029. Resources: {'ram', 'cores'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:57:50.581001 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.compute.api [None req-80eb11c1-bd9b-4584-baac-785864766a30 tempest-ServersAdminNegativeTestJSON-1082789623 tempest-ServersAdminNegativeTestJSON-1082789623-project-admin] ram quota exceeded for 1ffacafb54b044fb8e92ffb08136d1d2, tried to resize instance. Jul 16 21:57:50.581540 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-80eb11c1-bd9b-4584-baac-785864766a30 tempest-ServersAdminNegativeTestJSON-1082789623 tempest-ServersAdminNegativeTestJSON-1082789623-project-admin] HTTP exception thrown: Quota exceeded for ram: Requested 50177, but already used 1024 of 51200 ram Jul 16 21:57:50.584062 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-80eb11c1-bd9b-4584-baac-785864766a30 tempest-ServersAdminNegativeTestJSON-1082789623 tempest-ServersAdminNegativeTestJSON-1082789623-project-admin] Request method failure captured: Jul 16 21:57:50.584062 ncdv-master-1 devstack@n-api.service[138680]: request: POST /compute/v2.1/servers/e317fe1f-3367-4d70-a4c7-e357b93e3a49/action HTTP/1.1 Jul 16 21:57:50.584062 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:57:50.584062 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:57:50.584062 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:57:50.584062 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 38 Jul 16 21:57:50.584062 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:57:50.584062 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:57:50.584062 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:57:50.584062 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:57:50.584062 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0zRQ2YehRqS-EiXF0RAe3semt3R3JiJbneKL4FGngNI_Vs9CZQdVsZa26zNZMf2Be97JOz81XAUTNnwR58SqinVWRlNjsnciPAwvYWPTlu5Qh6YikV3_55XqpxpmPrS82hXQh3bkC4uqyxjl39MAc5_s9GWc80ebXBNpiqLvITTU4iM Jul 16 21:57:50.584062 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:57:50.584062 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:57:50.584062 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:57:50.584062 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:57:50.584062 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:57:50.584062 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:57:50.584062 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: 1ffacafb54b044fb8e92ffb08136d1d2 Jul 16 21:57:50.584062 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-ServersAdminNegativeTestJSON-1082789623 Jul 16 21:57:50.584062 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,admin,reader Jul 16 21:57:50.584062 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,admin,reader Jul 16 21:57:50.584062 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/1ffacafb54b044fb8e92ffb08136d1d2"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/1ffacafb54b044fb8e92ffb08136d1d2"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_1ffacafb54b044fb8e92ffb08136d1d2"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/1ffacafb54b044fb8e92ffb08136d1d2"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:57:50.584062 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-ServersAdminNegativeTestJSON-1082789623 Jul 16 21:57:50.584062 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: 1ffacafb54b044fb8e92ffb08136d1d2 Jul 16 21:57:50.584062 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-ServersAdminNegativeTestJSON-1082789623 Jul 16 21:57:50.584062 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-ServersAdminNegativeTestJSON-1082789623-project-admin Jul 16 21:57:50.584062 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:57:50.584062 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:57:50.584062 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: 2b7eba2f41e342d39333caabc176932c Jul 16 21:57:50.584062 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-ServersAdminNegativeTestJSON-1082789623-project-admin Jul 16 21:57:50.584062 ncdv-master-1 devstack@n-api.service[138680]: Jul 16 21:57:50.584062 ncdv-master-1 devstack@n-api.service[138680]: {"resize": {"flavorRef": "720895039"}} Jul 16 21:57:50.584062 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 21:57:50.584062 ncdv-master-1 devstack@n-api.service[138680]: exception: Quota exceeded for ram: Requested 50177, but already used 1024 of 51200 ram Jul 16 21:57:50.584062 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': 'e317fe1f-3367-4d70-a4c7-e357b93e3a49', 'body': {'resize': {'flavorRef': '720895039'}}} Jul 16 21:57:50.584062 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:57:50.584062 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:57:50.584062 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 3821, in _check_quota_for_upsize Jul 16 21:57:50.584062 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi objects.Quotas.check_deltas(context, res_deltas, Jul 16 21:57:50.584062 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Jul 16 21:57:50.584062 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Jul 16 21:57:50.584062 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/quotas.py", line 370, in check_deltas Jul 16 21:57:50.584062 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc Jul 16 21:57:50.584062 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/quotas.py", line 365, in check_deltas Jul 16 21:57:50.584062 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi cls.limit_check_project_and_user(context, **check_kwargs) Jul 16 21:57:50.584062 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Jul 16 21:57:50.584062 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Jul 16 21:57:50.584062 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/quotas.py", line 298, in limit_check_project_and_user Jul 16 21:57:50.584062 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return quota.QUOTAS.limit_check_project_and_user(context, Jul 16 21:57:50.584062 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/quota.py", line 1150, in limit_check_project_and_user Jul 16 21:57:50.584062 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return self._driver.limit_check_project_and_user( Jul 16 21:57:50.584062 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/quota.py", line 613, in limit_check_project_and_user Jul 16 21:57:50.584062 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.OverQuota(overs=sorted(overs), Jul 16 21:57:50.584062 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.OverQuota: Quota exceeded for resources: ['ram'] Jul 16 21:57:50.584062 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:57:50.584062 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:57:50.584062 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:57:50.584062 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:57:50.584062 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1053, in _resize Jul 16 21:57:50.584062 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi self.compute_api.resize(context, instance, flavor_id, Jul 16 21:57:50.584062 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 375, in inner Jul 16 21:57:50.584062 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Jul 16 21:57:50.584062 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 360, in wrapper Jul 16 21:57:50.588253 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return func(self, context, instance, *args, **kwargs) Jul 16 21:57:50.588253 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 296, in inner Jul 16 21:57:50.588253 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Jul 16 21:57:50.588253 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 343, in wrapper Jul 16 21:57:50.588253 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return func(self, context, instance, *args, **kwargs) Jul 16 21:57:50.588253 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner Jul 16 21:57:50.588253 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Jul 16 21:57:50.588253 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 164, in inner Jul 16 21:57:50.588253 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Jul 16 21:57:50.588253 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 228, in wrapped Jul 16 21:57:50.588253 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Jul 16 21:57:50.588253 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 4190, in resize Jul 16 21:57:50.588253 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi self._check_quota_for_upsize(context, instance, Jul 16 21:57:50.588253 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 3839, in _check_quota_for_upsize Jul 16 21:57:50.588253 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.TooManyInstances(overs=overs, Jul 16 21:57:50.588253 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.TooManyInstances: Quota exceeded for ram: Requested 50177, but already used 1024 of 51200 ram Jul 16 21:57:50.588253 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:57:50.588253 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:57:50.588253 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:57:50.588253 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:57:50.588253 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:57:50.588253 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:57:50.588253 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:57:50.588253 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:57:50.588253 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:57:50.588253 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:57:50.588253 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 16 21:57:50.588253 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 16 21:57:50.588253 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1139, in _action_resize Jul 16 21:57:50.588253 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi self._resize(req, id, flavor_ref, **kwargs) Jul 16 21:57:50.588253 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1056, in _resize Jul 16 21:57:50.588253 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPForbidden( Jul 16 21:57:50.588253 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPForbidden: Quota exceeded for ram: Requested 50177, but already used 1024 of 51200 ram Jul 16 21:57:50.588253 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:57:50.588253 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:57:50.588253 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:57:50.588253 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:57:50.588253 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:57:50.588253 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:57:50.588253 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:57:50.588253 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:57:50.588253 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Quota exceeded for ram: Requested 50177, but already used 1024 of 51200 ram Jul 16 21:57:50.588253 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:57:50.588253 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-80eb11c1-bd9b-4584-baac-785864766a30 tempest-ServersAdminNegativeTestJSON-1082789623 tempest-ServersAdminNegativeTestJSON-1082789623-project-admin] Returning 403 to user: Quota exceeded for ram: Requested 50177, but already used 1024 of 51200 ram {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:57:50.588253 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-80eb11c1-bd9b-4584-baac-785864766a30 tempest-ServersAdminNegativeTestJSON-1082789623 tempest-ServersAdminNegativeTestJSON-1082789623-project-admin] 10.222.0.16 "POST /compute/v2.1/servers/e317fe1f-3367-4d70-a4c7-e357b93e3a49/action" status: 403 len: 118 microversion: 2.1 time: 0.262390 Jul 16 21:57:50.588253 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3138/6281] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 21:57:50 2022] POST /compute/v2.1/servers/e317fe1f-3367-4d70-a4c7-e357b93e3a49/action => generated 118 bytes in 264 msecs (HTTP/1.1 403) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:57:50.595105 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-4b622f53-de52-4dca-be3b-7ad6e1529216 tempest-ServersAdminNegativeTestJSON-1082789623 tempest-ServersAdminNegativeTestJSON-1082789623-project-admin] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:57:50.595869 ncdv-master-1 devstack@n-api.service[138681]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-flavor-manage:delete": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:57:50.595869 ncdv-master-1 devstack@n-api.service[138681]: warnings.warn(msg) Jul 16 21:57:50.679222 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-4b622f53-de52-4dca-be3b-7ad6e1529216 tempest-ServersAdminNegativeTestJSON-1082789623 tempest-ServersAdminNegativeTestJSON-1082789623-project-admin] 10.222.0.16 "DELETE /compute/v2.1/flavors/720895039" status: 202 len: 0 microversion: 2.1 time: 0.088021 Jul 16 21:57:50.679809 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3144/6282] 10.222.0.16 () {58 vars in 1157 bytes} [Sat Jul 16 21:57:50 2022] DELETE /compute/v2.1/flavors/720895039 => generated 0 bytes in 90 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 21:57:50.687769 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-e2c3a60c-776f-449b-a30f-909787a965a5 tempest-ServersAdminNegativeTestJSON-1082789623 tempest-ServersAdminNegativeTestJSON-1082789623-project-admin] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:57:50.707030 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-e2c3a60c-776f-449b-a30f-909787a965a5 tempest-ServersAdminNegativeTestJSON-1082789623 tempest-ServersAdminNegativeTestJSON-1082789623-project-admin] 10.222.0.16 "GET /compute/v2.1/flavors/detail" status: 200 len: 4991 microversion: 2.1 time: 0.022920 Jul 16 21:57:50.708564 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3139/6283] 10.222.0.16 () {58 vars in 1145 bytes} [Sat Jul 16 21:57:50 2022] GET /compute/v2.1/flavors/detail => generated 4991 bytes in 25 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:57:50.726097 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-01f0f895-f83d-47d6-906c-e227f78f4cb0 tempest-ServersAdminNegativeTestJSON-1082789623 tempest-ServersAdminNegativeTestJSON-1082789623-project-admin] Calling method '.version_select at 0x7fdd8c4cbaf0>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:57:50.794574 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-01f0f895-f83d-47d6-906c-e227f78f4cb0 tempest-ServersAdminNegativeTestJSON-1082789623 tempest-ServersAdminNegativeTestJSON-1082789623-project-admin] 10.222.0.16 "GET /compute/v2.1/os-quota-sets/1ffacafb54b044fb8e92ffb08136d1d2" status: 200 len: 371 microversion: 2.1 time: 0.072903 Jul 16 21:57:50.795187 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3145/6284] 10.222.0.16 () {58 vars in 1241 bytes} [Sat Jul 16 21:57:50 2022] GET /compute/v2.1/os-quota-sets/1ffacafb54b044fb8e92ffb08136d1d2 => generated 371 bytes in 74 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:57:50.805110 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-d0f12722-773c-4987-b56f-d321c0a8c46d tempest-ServersAdminNegativeTestJSON-1082789623 tempest-ServersAdminNegativeTestJSON-1082789623-project-admin] Action: 'create', calling method: >, body: {"flavor": {"ram": 512, "vcpus": 21, "disk": 5, "name": "tempest-ServersAdminNegativeTestJSON-flavor-1565157184", "id": 192349271, "os-flavor-access:is_public": "True"}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:57:50.806717 ncdv-master-1 devstack@n-api.service[138680]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-flavor-manage:create": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:57:50.806717 ncdv-master-1 devstack@n-api.service[138680]: warnings.warn(msg) Jul 16 21:57:50.874341 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-d0f12722-773c-4987-b56f-d321c0a8c46d tempest-ServersAdminNegativeTestJSON-1082789623 tempest-ServersAdminNegativeTestJSON-1082789623-project-admin] 10.222.0.16 "POST /compute/v2.1/flavors" status: 200 len: 434 microversion: 2.1 time: 0.072842 Jul 16 21:57:50.878069 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3140/6285] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 21:57:50 2022] POST /compute/v2.1/flavors => generated 434 bytes in 74 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:57:50.886175 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-822334b9-7f87-47d0-aef0-f10e1412ec72 tempest-ServersAdminNegativeTestJSON-1082789623 tempest-ServersAdminNegativeTestJSON-1082789623-project-admin] Action: 'action', calling method: >, body: {"resize": {"flavorRef": "192349271"}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:57:50.887201 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-822334b9-7f87-47d0-aef0-f10e1412ec72 tempest-ServersAdminNegativeTestJSON-1082789623 tempest-ServersAdminNegativeTestJSON-1082789623-project-admin] [instance: e317fe1f-3367-4d70-a4c7-e357b93e3a49] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:57:50.893377 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-822334b9-7f87-47d0-aef0-f10e1412ec72 tempest-ServersAdminNegativeTestJSON-1082789623 tempest-ServersAdminNegativeTestJSON-1082789623-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:57:50.895475 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-822334b9-7f87-47d0-aef0-f10e1412ec72 tempest-ServersAdminNegativeTestJSON-1082789623 tempest-ServersAdminNegativeTestJSON-1082789623-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:57:50.895475 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-822334b9-7f87-47d0-aef0-f10e1412ec72 tempest-ServersAdminNegativeTestJSON-1082789623 tempest-ServersAdminNegativeTestJSON-1082789623-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:57:50.895864 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-822334b9-7f87-47d0-aef0-f10e1412ec72 tempest-ServersAdminNegativeTestJSON-1082789623 tempest-ServersAdminNegativeTestJSON-1082789623-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:57:51.032121 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-822334b9-7f87-47d0-aef0-f10e1412ec72 tempest-ServersAdminNegativeTestJSON-1082789623 tempest-ServersAdminNegativeTestJSON-1082789623-project-admin] Policy check for compute:servers:resize:cross_cell failed with credentials {'is_admin': True, 'user_id': '2b7eba2f41e342d39333caabc176932c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1ffacafb54b044fb8e92ffb08136d1d2', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:57:51.038813 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-822334b9-7f87-47d0-aef0-f10e1412ec72 tempest-ServersAdminNegativeTestJSON-1082789623 tempest-ServersAdminNegativeTestJSON-1082789623-project-admin] [instance: e317fe1f-3367-4d70-a4c7-e357b93e3a49] Old instance type m1.heat2, new instance type tempest-ServersAdminNegativeTestJSON-flavor-1565157184 {{(pid=138681) resize /opt/stack/nova/nova/compute/api.py:4166}} Jul 16 21:57:51.045212 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-822334b9-7f87-47d0-aef0-f10e1412ec72 tempest-ServersAdminNegativeTestJSON-1082789623 tempest-ServersAdminNegativeTestJSON-1082789623-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:57:51.045671 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-822334b9-7f87-47d0-aef0-f10e1412ec72 tempest-ServersAdminNegativeTestJSON-1082789623 tempest-ServersAdminNegativeTestJSON-1082789623-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:57:51.046467 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-822334b9-7f87-47d0-aef0-f10e1412ec72 tempest-ServersAdminNegativeTestJSON-1082789623 tempest-ServersAdminNegativeTestJSON-1082789623-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:57:51.046914 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-822334b9-7f87-47d0-aef0-f10e1412ec72 tempest-ServersAdminNegativeTestJSON-1082789623 tempest-ServersAdminNegativeTestJSON-1082789623-project-admin] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:57:51.067654 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-822334b9-7f87-47d0-aef0-f10e1412ec72 tempest-ServersAdminNegativeTestJSON-1082789623 tempest-ServersAdminNegativeTestJSON-1082789623-project-admin] Getting quotas for project 46b1acf286d840738d93acc4ee506029. Resources: {'ram', 'cores'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:57:51.072505 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-822334b9-7f87-47d0-aef0-f10e1412ec72 tempest-ServersAdminNegativeTestJSON-1082789623 tempest-ServersAdminNegativeTestJSON-1082789623-project-admin] Getting quotas for user 92d1d7ddc009428bb340e9c6f20aa7f3 and project 46b1acf286d840738d93acc4ee506029. Resources: {'ram', 'cores'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:57:51.085092 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.compute.api [None req-822334b9-7f87-47d0-aef0-f10e1412ec72 tempest-ServersAdminNegativeTestJSON-1082789623 tempest-ServersAdminNegativeTestJSON-1082789623-project-admin] cores quota exceeded for 1ffacafb54b044fb8e92ffb08136d1d2, tried to resize instance. Jul 16 21:57:51.085970 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-822334b9-7f87-47d0-aef0-f10e1412ec72 tempest-ServersAdminNegativeTestJSON-1082789623 tempest-ServersAdminNegativeTestJSON-1082789623-project-admin] HTTP exception thrown: Quota exceeded for cores: Requested 20, but already used 1 of 20 cores Jul 16 21:57:51.102262 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-822334b9-7f87-47d0-aef0-f10e1412ec72 tempest-ServersAdminNegativeTestJSON-1082789623 tempest-ServersAdminNegativeTestJSON-1082789623-project-admin] Request method failure captured: Jul 16 21:57:51.102262 ncdv-master-1 devstack@n-api.service[138681]: request: POST /compute/v2.1/servers/e317fe1f-3367-4d70-a4c7-e357b93e3a49/action HTTP/1.1 Jul 16 21:57:51.102262 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:57:51.102262 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:57:51.102262 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:57:51.102262 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 38 Jul 16 21:57:51.102262 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:57:51.102262 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:57:51.102262 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:57:51.102262 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:57:51.102262 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0zRQ2YehRqS-EiXF0RAe3semt3R3JiJbneKL4FGngNI_Vs9CZQdVsZa26zNZMf2Be97JOz81XAUTNnwR58SqinVWRlNjsnciPAwvYWPTlu5Qh6YikV3_55XqpxpmPrS82hXQh3bkC4uqyxjl39MAc5_s9GWc80ebXBNpiqLvITTU4iM Jul 16 21:57:51.102262 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:57:51.102262 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:57:51.102262 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:57:51.102262 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:57:51.102262 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:57:51.102262 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:57:51.102262 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: 1ffacafb54b044fb8e92ffb08136d1d2 Jul 16 21:57:51.102262 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-ServersAdminNegativeTestJSON-1082789623 Jul 16 21:57:51.102262 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,admin,reader Jul 16 21:57:51.102262 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,admin,reader Jul 16 21:57:51.102262 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/1ffacafb54b044fb8e92ffb08136d1d2"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/1ffacafb54b044fb8e92ffb08136d1d2"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_1ffacafb54b044fb8e92ffb08136d1d2"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/1ffacafb54b044fb8e92ffb08136d1d2"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:57:51.102262 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-ServersAdminNegativeTestJSON-1082789623 Jul 16 21:57:51.102262 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: 1ffacafb54b044fb8e92ffb08136d1d2 Jul 16 21:57:51.102262 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-ServersAdminNegativeTestJSON-1082789623 Jul 16 21:57:51.102262 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-ServersAdminNegativeTestJSON-1082789623-project-admin Jul 16 21:57:51.102262 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:57:51.102262 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:57:51.102262 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: 2b7eba2f41e342d39333caabc176932c Jul 16 21:57:51.102262 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-ServersAdminNegativeTestJSON-1082789623-project-admin Jul 16 21:57:51.102262 ncdv-master-1 devstack@n-api.service[138681]: Jul 16 21:57:51.102262 ncdv-master-1 devstack@n-api.service[138681]: {"resize": {"flavorRef": "192349271"}} Jul 16 21:57:51.102262 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:57:51.102262 ncdv-master-1 devstack@n-api.service[138681]: exception: Quota exceeded for cores: Requested 20, but already used 1 of 20 cores Jul 16 21:57:51.102262 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': 'e317fe1f-3367-4d70-a4c7-e357b93e3a49', 'body': {'resize': {'flavorRef': '192349271'}}} Jul 16 21:57:51.102262 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:57:51.102262 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:57:51.102262 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 3821, in _check_quota_for_upsize Jul 16 21:57:51.102262 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi objects.Quotas.check_deltas(context, res_deltas, Jul 16 21:57:51.102262 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Jul 16 21:57:51.102262 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Jul 16 21:57:51.102262 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/quotas.py", line 370, in check_deltas Jul 16 21:57:51.102262 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc Jul 16 21:57:51.102262 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/quotas.py", line 365, in check_deltas Jul 16 21:57:51.102262 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi cls.limit_check_project_and_user(context, **check_kwargs) Jul 16 21:57:51.102262 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Jul 16 21:57:51.102262 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Jul 16 21:57:51.102262 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/quotas.py", line 298, in limit_check_project_and_user Jul 16 21:57:51.102262 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return quota.QUOTAS.limit_check_project_and_user(context, Jul 16 21:57:51.102262 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/quota.py", line 1150, in limit_check_project_and_user Jul 16 21:57:51.102262 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return self._driver.limit_check_project_and_user( Jul 16 21:57:51.102262 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/quota.py", line 613, in limit_check_project_and_user Jul 16 21:57:51.102262 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.OverQuota(overs=sorted(overs), Jul 16 21:57:51.102262 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.OverQuota: Quota exceeded for resources: ['cores'] Jul 16 21:57:51.102262 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:57:51.102262 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:57:51.102262 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:57:51.102262 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:57:51.102262 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1053, in _resize Jul 16 21:57:51.102262 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi self.compute_api.resize(context, instance, flavor_id, Jul 16 21:57:51.102262 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 375, in inner Jul 16 21:57:51.102262 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Jul 16 21:57:51.102262 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 360, in wrapper Jul 16 21:57:51.106663 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return func(self, context, instance, *args, **kwargs) Jul 16 21:57:51.106663 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 296, in inner Jul 16 21:57:51.106663 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Jul 16 21:57:51.106663 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 343, in wrapper Jul 16 21:57:51.106663 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return func(self, context, instance, *args, **kwargs) Jul 16 21:57:51.106663 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner Jul 16 21:57:51.106663 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Jul 16 21:57:51.106663 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 164, in inner Jul 16 21:57:51.106663 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Jul 16 21:57:51.106663 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 228, in wrapped Jul 16 21:57:51.106663 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Jul 16 21:57:51.106663 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 4190, in resize Jul 16 21:57:51.106663 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi self._check_quota_for_upsize(context, instance, Jul 16 21:57:51.106663 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 3839, in _check_quota_for_upsize Jul 16 21:57:51.106663 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.TooManyInstances(overs=overs, Jul 16 21:57:51.106663 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.TooManyInstances: Quota exceeded for cores: Requested 20, but already used 1 of 20 cores Jul 16 21:57:51.106663 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:57:51.106663 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:57:51.106663 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:57:51.106663 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:57:51.106663 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:57:51.106663 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:57:51.106663 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:57:51.106663 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:57:51.106663 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:57:51.106663 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:57:51.106663 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 16 21:57:51.106663 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 16 21:57:51.106663 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1139, in _action_resize Jul 16 21:57:51.106663 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi self._resize(req, id, flavor_ref, **kwargs) Jul 16 21:57:51.106663 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1056, in _resize Jul 16 21:57:51.106663 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPForbidden( Jul 16 21:57:51.106663 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPForbidden: Quota exceeded for cores: Requested 20, but already used 1 of 20 cores Jul 16 21:57:51.106663 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:57:51.106663 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:57:51.106663 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:57:51.106663 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:57:51.106663 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:57:51.106663 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:57:51.106663 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:57:51.106663 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:57:51.106663 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Quota exceeded for cores: Requested 20, but already used 1 of 20 cores Jul 16 21:57:51.106663 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:57:51.106663 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-822334b9-7f87-47d0-aef0-f10e1412ec72 tempest-ServersAdminNegativeTestJSON-1082789623 tempest-ServersAdminNegativeTestJSON-1082789623-project-admin] Returning 403 to user: Quota exceeded for cores: Requested 20, but already used 1 of 20 cores {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:57:51.106663 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-822334b9-7f87-47d0-aef0-f10e1412ec72 tempest-ServersAdminNegativeTestJSON-1082789623 tempest-ServersAdminNegativeTestJSON-1082789623-project-admin] 10.222.0.16 "POST /compute/v2.1/servers/e317fe1f-3367-4d70-a4c7-e357b93e3a49/action" status: 403 len: 113 microversion: 2.1 time: 0.221457 Jul 16 21:57:51.106663 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3146/6286] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 21:57:50 2022] POST /compute/v2.1/servers/e317fe1f-3367-4d70-a4c7-e357b93e3a49/action => generated 113 bytes in 223 msecs (HTTP/1.1 403) 9 headers in 379 bytes (2 switches on core 0) Jul 16 21:57:51.112658 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-080f3cc4-bb24-4896-b1ce-253628e77b68 tempest-ServersAdminNegativeTestJSON-1082789623 tempest-ServersAdminNegativeTestJSON-1082789623-project-admin] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:57:51.113415 ncdv-master-1 devstack@n-api.service[138680]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-flavor-manage:delete": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 16 21:57:51.113415 ncdv-master-1 devstack@n-api.service[138680]: warnings.warn(msg) Jul 16 21:57:51.170374 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-080f3cc4-bb24-4896-b1ce-253628e77b68 tempest-ServersAdminNegativeTestJSON-1082789623 tempest-ServersAdminNegativeTestJSON-1082789623-project-admin] 10.222.0.16 "DELETE /compute/v2.1/flavors/192349271" status: 202 len: 0 microversion: 2.1 time: 0.061429 Jul 16 21:57:51.171044 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3141/6287] 10.222.0.16 () {58 vars in 1157 bytes} [Sat Jul 16 21:57:51 2022] DELETE /compute/v2.1/flavors/192349271 => generated 0 bytes in 63 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Jul 16 21:57:51.178960 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-e28c4c70-6a88-4dd1-8de4-bcd390680675 tempest-ServersAdminNegativeTestJSON-1082789623 tempest-ServersAdminNegativeTestJSON-1082789623-project-admin] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:57:51.198728 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-e28c4c70-6a88-4dd1-8de4-bcd390680675 tempest-ServersAdminNegativeTestJSON-1082789623 tempest-ServersAdminNegativeTestJSON-1082789623-project-admin] 10.222.0.16 "GET /compute/v2.1/flavors/detail" status: 200 len: 4991 microversion: 2.1 time: 0.023451 Jul 16 21:57:51.199321 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3147/6288] 10.222.0.16 () {58 vars in 1145 bytes} [Sat Jul 16 21:57:51 2022] GET /compute/v2.1/flavors/detail => generated 4991 bytes in 25 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:57:51.216567 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-dcb9b6a8-06a3-4862-9ab1-b64d026fde12 tempest-ServersAdminNegativeTestJSON-211662210 tempest-ServersAdminNegativeTestJSON-211662210-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:57:51.217074 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-dcb9b6a8-06a3-4862-9ab1-b64d026fde12 tempest-ServersAdminNegativeTestJSON-211662210 tempest-ServersAdminNegativeTestJSON-211662210-project] [instance: e317fe1f-3367-4d70-a4c7-e357b93e3a49] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:57:51.224429 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-dcb9b6a8-06a3-4862-9ab1-b64d026fde12 tempest-ServersAdminNegativeTestJSON-211662210 tempest-ServersAdminNegativeTestJSON-211662210-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:57:51.224883 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-dcb9b6a8-06a3-4862-9ab1-b64d026fde12 tempest-ServersAdminNegativeTestJSON-211662210 tempest-ServersAdminNegativeTestJSON-211662210-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:57:51.225643 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-dcb9b6a8-06a3-4862-9ab1-b64d026fde12 tempest-ServersAdminNegativeTestJSON-211662210 tempest-ServersAdminNegativeTestJSON-211662210-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:57:51.226083 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-dcb9b6a8-06a3-4862-9ab1-b64d026fde12 tempest-ServersAdminNegativeTestJSON-211662210 tempest-ServersAdminNegativeTestJSON-211662210-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:57:51.250696 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-dcb9b6a8-06a3-4862-9ab1-b64d026fde12 tempest-ServersAdminNegativeTestJSON-211662210 tempest-ServersAdminNegativeTestJSON-211662210-project] [instance: e317fe1f-3367-4d70-a4c7-e357b93e3a49] Going to try to terminate instance {{(pid=138680) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 16 21:57:51.490435 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-dcb9b6a8-06a3-4862-9ab1-b64d026fde12 tempest-ServersAdminNegativeTestJSON-211662210 tempest-ServersAdminNegativeTestJSON-211662210-project] 10.222.0.16 "DELETE /compute/v2.1/servers/e317fe1f-3367-4d70-a4c7-e357b93e3a49" status: 204 len: 0 microversion: 2.1 time: 0.277584 Jul 16 21:57:51.490950 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3142/6289] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:57:51 2022] DELETE /compute/v2.1/servers/e317fe1f-3367-4d70-a4c7-e357b93e3a49 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 16 21:57:51.499099 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-570c23af-3181-46f6-a74c-8c5cf734f641 tempest-ServersAdminNegativeTestJSON-211662210 tempest-ServersAdminNegativeTestJSON-211662210-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:57:51.504144 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-570c23af-3181-46f6-a74c-8c5cf734f641 tempest-ServersAdminNegativeTestJSON-211662210 tempest-ServersAdminNegativeTestJSON-211662210-project] [instance: e317fe1f-3367-4d70-a4c7-e357b93e3a49] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:57:51.506094 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-570c23af-3181-46f6-a74c-8c5cf734f641 tempest-ServersAdminNegativeTestJSON-211662210 tempest-ServersAdminNegativeTestJSON-211662210-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:57:51.506566 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-570c23af-3181-46f6-a74c-8c5cf734f641 tempest-ServersAdminNegativeTestJSON-211662210 tempest-ServersAdminNegativeTestJSON-211662210-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:57:51.507390 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-570c23af-3181-46f6-a74c-8c5cf734f641 tempest-ServersAdminNegativeTestJSON-211662210 tempest-ServersAdminNegativeTestJSON-211662210-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:57:51.507845 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-570c23af-3181-46f6-a74c-8c5cf734f641 tempest-ServersAdminNegativeTestJSON-211662210 tempest-ServersAdminNegativeTestJSON-211662210-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:57:51.638726 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-570c23af-3181-46f6-a74c-8c5cf734f641 tempest-ServersAdminNegativeTestJSON-211662210 tempest-ServersAdminNegativeTestJSON-211662210-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92d1d7ddc009428bb340e9c6f20aa7f3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '46b1acf286d840738d93acc4ee506029', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:57:51.647666 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-570c23af-3181-46f6-a74c-8c5cf734f641 tempest-ServersAdminNegativeTestJSON-211662210 tempest-ServersAdminNegativeTestJSON-211662210-project] 10.222.0.16 "GET /compute/v2.1/servers/e317fe1f-3367-4d70-a4c7-e357b93e3a49" status: 200 len: 1476 microversion: 2.1 time: 0.152607 Jul 16 21:57:51.648349 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3148/6290] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:57:51 2022] GET /compute/v2.1/servers/e317fe1f-3367-4d70-a4c7-e357b93e3a49 => generated 1476 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:57:52.666176 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-67461920-c018-4b88-95bb-edf7e76e5ea9 tempest-ServersAdminNegativeTestJSON-211662210 tempest-ServersAdminNegativeTestJSON-211662210-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:57:52.666883 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-67461920-c018-4b88-95bb-edf7e76e5ea9 tempest-ServersAdminNegativeTestJSON-211662210 tempest-ServersAdminNegativeTestJSON-211662210-project] [instance: e317fe1f-3367-4d70-a4c7-e357b93e3a49] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:57:52.673857 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-67461920-c018-4b88-95bb-edf7e76e5ea9 tempest-ServersAdminNegativeTestJSON-211662210 tempest-ServersAdminNegativeTestJSON-211662210-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:57:52.674328 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-67461920-c018-4b88-95bb-edf7e76e5ea9 tempest-ServersAdminNegativeTestJSON-211662210 tempest-ServersAdminNegativeTestJSON-211662210-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:57:52.675093 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-67461920-c018-4b88-95bb-edf7e76e5ea9 tempest-ServersAdminNegativeTestJSON-211662210 tempest-ServersAdminNegativeTestJSON-211662210-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:57:52.675542 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-67461920-c018-4b88-95bb-edf7e76e5ea9 tempest-ServersAdminNegativeTestJSON-211662210 tempest-ServersAdminNegativeTestJSON-211662210-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:57:52.810358 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-67461920-c018-4b88-95bb-edf7e76e5ea9 tempest-ServersAdminNegativeTestJSON-211662210 tempest-ServersAdminNegativeTestJSON-211662210-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92d1d7ddc009428bb340e9c6f20aa7f3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '46b1acf286d840738d93acc4ee506029', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:57:52.817950 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-67461920-c018-4b88-95bb-edf7e76e5ea9 tempest-ServersAdminNegativeTestJSON-211662210 tempest-ServersAdminNegativeTestJSON-211662210-project] 10.222.0.16 "GET /compute/v2.1/servers/e317fe1f-3367-4d70-a4c7-e357b93e3a49" status: 200 len: 1476 microversion: 2.1 time: 0.156007 Jul 16 21:57:52.818740 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3143/6291] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:57:52 2022] GET /compute/v2.1/servers/e317fe1f-3367-4d70-a4c7-e357b93e3a49 => generated 1476 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:57:53.265922 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-30d52614-58f0-43f4-9632-0dddfdfc3e36 req-2e7678d7-8fd8-45c4-bb79-c138827e026f service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "e317fe1f-3367-4d70-a4c7-e357b93e3a49", "name": "network-vif-deleted", "tag": "31c8a385-ad6a-44cf-b2a9-b5e40a468f8d"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:57:53.276398 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-30d52614-58f0-43f4-9632-0dddfdfc3e36 req-2e7678d7-8fd8-45c4-bb79-c138827e026f service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:57:53.276866 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-30d52614-58f0-43f4-9632-0dddfdfc3e36 req-2e7678d7-8fd8-45c4-bb79-c138827e026f service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:57:53.289544 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-30d52614-58f0-43f4-9632-0dddfdfc3e36 req-2e7678d7-8fd8-45c4-bb79-c138827e026f service nova] Creating event network-vif-deleted:31c8a385-ad6a-44cf-b2a9-b5e40a468f8d for instance e317fe1f-3367-4d70-a4c7-e357b93e3a49 on nch1-master-1 Jul 16 21:57:53.294845 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-30d52614-58f0-43f4-9632-0dddfdfc3e36 req-2e7678d7-8fd8-45c4-bb79-c138827e026f service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.032818 Jul 16 21:57:53.295603 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3149/6292] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:57:53 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:57:53.835882 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-afe4f473-fab9-46e5-9a1a-a19ed9a1f182 tempest-ServersAdminNegativeTestJSON-211662210 tempest-ServersAdminNegativeTestJSON-211662210-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:57:53.836501 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-afe4f473-fab9-46e5-9a1a-a19ed9a1f182 tempest-ServersAdminNegativeTestJSON-211662210 tempest-ServersAdminNegativeTestJSON-211662210-project] [instance: e317fe1f-3367-4d70-a4c7-e357b93e3a49] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:57:53.843915 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-afe4f473-fab9-46e5-9a1a-a19ed9a1f182 tempest-ServersAdminNegativeTestJSON-211662210 tempest-ServersAdminNegativeTestJSON-211662210-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:57:53.844376 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-afe4f473-fab9-46e5-9a1a-a19ed9a1f182 tempest-ServersAdminNegativeTestJSON-211662210 tempest-ServersAdminNegativeTestJSON-211662210-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:57:53.845181 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-afe4f473-fab9-46e5-9a1a-a19ed9a1f182 tempest-ServersAdminNegativeTestJSON-211662210 tempest-ServersAdminNegativeTestJSON-211662210-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:57:53.845623 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-afe4f473-fab9-46e5-9a1a-a19ed9a1f182 tempest-ServersAdminNegativeTestJSON-211662210 tempest-ServersAdminNegativeTestJSON-211662210-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:57:53.882671 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-afe4f473-fab9-46e5-9a1a-a19ed9a1f182 tempest-ServersAdminNegativeTestJSON-211662210 tempest-ServersAdminNegativeTestJSON-211662210-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:57:53.883157 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-afe4f473-fab9-46e5-9a1a-a19ed9a1f182 tempest-ServersAdminNegativeTestJSON-211662210 tempest-ServersAdminNegativeTestJSON-211662210-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:57:53.883612 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.objects.instance [None req-afe4f473-fab9-46e5-9a1a-a19ed9a1f182 tempest-ServersAdminNegativeTestJSON-211662210 tempest-ServersAdminNegativeTestJSON-211662210-project] Lazy-loading 'fault' on Instance uuid e317fe1f-3367-4d70-a4c7-e357b93e3a49 {{(pid=138680) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:57:53.924526 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-afe4f473-fab9-46e5-9a1a-a19ed9a1f182 tempest-ServersAdminNegativeTestJSON-211662210 tempest-ServersAdminNegativeTestJSON-211662210-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92d1d7ddc009428bb340e9c6f20aa7f3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '46b1acf286d840738d93acc4ee506029', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:57:53.932699 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-afe4f473-fab9-46e5-9a1a-a19ed9a1f182 tempest-ServersAdminNegativeTestJSON-211662210 tempest-ServersAdminNegativeTestJSON-211662210-project] 10.222.0.16 "GET /compute/v2.1/servers/e317fe1f-3367-4d70-a4c7-e357b93e3a49" status: 200 len: 1311 microversion: 2.1 time: 0.101473 Jul 16 21:57:53.933275 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3144/6293] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:57:53 2022] GET /compute/v2.1/servers/e317fe1f-3367-4d70-a4c7-e357b93e3a49 => generated 1311 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:57:54.954567 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-611d4618-04b5-46aa-a197-6428f5a90536 tempest-ServersAdminNegativeTestJSON-211662210 tempest-ServersAdminNegativeTestJSON-211662210-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:57:54.955305 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-611d4618-04b5-46aa-a197-6428f5a90536 tempest-ServersAdminNegativeTestJSON-211662210 tempest-ServersAdminNegativeTestJSON-211662210-project] [instance: e317fe1f-3367-4d70-a4c7-e357b93e3a49] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:57:54.962396 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-611d4618-04b5-46aa-a197-6428f5a90536 tempest-ServersAdminNegativeTestJSON-211662210 tempest-ServersAdminNegativeTestJSON-211662210-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:57:54.962853 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-611d4618-04b5-46aa-a197-6428f5a90536 tempest-ServersAdminNegativeTestJSON-211662210 tempest-ServersAdminNegativeTestJSON-211662210-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:57:54.963668 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-611d4618-04b5-46aa-a197-6428f5a90536 tempest-ServersAdminNegativeTestJSON-211662210 tempest-ServersAdminNegativeTestJSON-211662210-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:57:54.964122 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-611d4618-04b5-46aa-a197-6428f5a90536 tempest-ServersAdminNegativeTestJSON-211662210 tempest-ServersAdminNegativeTestJSON-211662210-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:57:54.978262 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-611d4618-04b5-46aa-a197-6428f5a90536 tempest-ServersAdminNegativeTestJSON-211662210 tempest-ServersAdminNegativeTestJSON-211662210-project] HTTP exception thrown: Instance e317fe1f-3367-4d70-a4c7-e357b93e3a49 could not be found. Jul 16 21:57:54.979934 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-611d4618-04b5-46aa-a197-6428f5a90536 tempest-ServersAdminNegativeTestJSON-211662210 tempest-ServersAdminNegativeTestJSON-211662210-project] Request method failure captured: Jul 16 21:57:54.979934 ncdv-master-1 devstack@n-api.service[138681]: request: GET /compute/v2.1/servers/e317fe1f-3367-4d70-a4c7-e357b93e3a49 HTTP/1.1 Jul 16 21:57:54.979934 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:57:54.979934 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:57:54.979934 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:57:54.979934 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 21:57:54.979934 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:57:54.979934 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:57:54.979934 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:57:54.979934 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:57:54.979934 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0zRPsYlNqhcL4DA3wAG7r_ZcRmDV2peU04qGeHpQbAasBKkqJ6zsB66d3I7BjT2aB9Onhw7W3PithgIw4ouaifnMbz2I1cOXHNi1VjJI_wy6o2d8j5K9e3JeIq0C1seyG_sSoUbfKye5b9SkO429xssJDXu_ebcNFBqErd-bM915tMQ Jul 16 21:57:54.979934 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:57:54.979934 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:57:54.979934 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:57:54.979934 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:57:54.979934 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:57:54.979934 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:57:54.979934 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: 46b1acf286d840738d93acc4ee506029 Jul 16 21:57:54.979934 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-ServersAdminNegativeTestJSON-211662210 Jul 16 21:57:54.979934 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:57:54.979934 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:57:54.979934 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/46b1acf286d840738d93acc4ee506029"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/46b1acf286d840738d93acc4ee506029"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_46b1acf286d840738d93acc4ee506029"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/46b1acf286d840738d93acc4ee506029"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:57:54.979934 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-ServersAdminNegativeTestJSON-211662210 Jul 16 21:57:54.979934 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: 46b1acf286d840738d93acc4ee506029 Jul 16 21:57:54.979934 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-ServersAdminNegativeTestJSON-211662210 Jul 16 21:57:54.979934 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-ServersAdminNegativeTestJSON-211662210-project Jul 16 21:57:54.979934 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:57:54.979934 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:57:54.979934 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: 92d1d7ddc009428bb340e9c6f20aa7f3 Jul 16 21:57:54.979934 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-ServersAdminNegativeTestJSON-211662210-project Jul 16 21:57:54.979934 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:57:54.979934 ncdv-master-1 devstack@n-api.service[138681]: exception: Instance e317fe1f-3367-4d70-a4c7-e357b93e3a49 could not be found. Jul 16 21:57:54.979934 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': 'e317fe1f-3367-4d70-a4c7-e357b93e3a49'} Jul 16 21:57:54.979934 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:57:54.979934 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:57:54.979934 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:57:54.979934 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:57:54.979934 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:57:54.979934 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:57:54.979934 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:57:54.979934 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:57:54.979934 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:57:54.979934 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:57:54.979934 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance e317fe1f-3367-4d70-a4c7-e357b93e3a49 could not be found. Jul 16 21:57:54.979934 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:57:54.979934 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:57:54.979934 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:57:54.979934 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:57:54.979934 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:57:54.979934 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:57:54.979934 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:57:54.979934 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:57:54.979934 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:57:54.979934 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:57:54.979934 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:57:54.979934 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:57:54.979934 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:57:54.979934 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:57:54.979934 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:57:54.979934 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:57:54.979934 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance e317fe1f-3367-4d70-a4c7-e357b93e3a49 could not be found. Jul 16 21:57:54.979934 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:57:54.979934 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:57:54.979934 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:57:54.984141 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:57:54.984141 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:57:54.984141 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:57:54.984141 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:57:54.984141 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:57:54.984141 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance e317fe1f-3367-4d70-a4c7-e357b93e3a49 could not be found. Jul 16 21:57:54.984141 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:57:54.984141 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-611d4618-04b5-46aa-a197-6428f5a90536 tempest-ServersAdminNegativeTestJSON-211662210 tempest-ServersAdminNegativeTestJSON-211662210-project] Returning 404 to user: Instance e317fe1f-3367-4d70-a4c7-e357b93e3a49 could not be found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:57:54.984141 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-611d4618-04b5-46aa-a197-6428f5a90536 tempest-ServersAdminNegativeTestJSON-211662210 tempest-ServersAdminNegativeTestJSON-211662210-project] 10.222.0.16 "GET /compute/v2.1/servers/e317fe1f-3367-4d70-a4c7-e357b93e3a49" status: 404 len: 111 microversion: 2.1 time: 0.031984 Jul 16 21:57:54.984141 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3150/6294] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:57:54 2022] GET /compute/v2.1/servers/e317fe1f-3367-4d70-a4c7-e357b93e3a49 => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:58:03.028154 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell0:SELECT=2 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:58:03.028945 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell0:SELECT=3 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:58:03.152126 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-055153b7-f540-4a40-b5de-b075704feb9f tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-DeleteServersTestJSON-server-748444953", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452", "networks": [{"uuid": "3e41ed2c-d7a5-4ebc-b7cf-42df13233cba"}]}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:58:03.205464 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.network.neutron [None req-055153b7-f540-4a40-b5de-b075704feb9f tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] validate_networks() for [('3e41ed2c-d7a5-4ebc-b7cf-42df13233cba', None, None, None, None, None)] {{(pid=138680) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 16 21:58:03.628811 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-055153b7-f540-4a40-b5de-b075704feb9f tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Flavor limits 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 16 21:58:03.629333 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-055153b7-f540-4a40-b5de-b075704feb9f tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Image limits 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 16 21:58:03.629986 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-055153b7-f540-4a40-b5de-b075704feb9f tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Flavor pref 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 16 21:58:03.630443 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-055153b7-f540-4a40-b5de-b075704feb9f tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Image pref 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 16 21:58:03.631532 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-055153b7-f540-4a40-b5de-b075704feb9f tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 16 21:58:03.642799 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-055153b7-f540-4a40-b5de-b075704feb9f tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:58:03.642799 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-055153b7-f540-4a40-b5de-b075704feb9f tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:58:03.643131 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-055153b7-f540-4a40-b5de-b075704feb9f tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:58:03.643521 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-055153b7-f540-4a40-b5de-b075704feb9f tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:58:03.664275 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-055153b7-f540-4a40-b5de-b075704feb9f tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Getting quotas for project 8849a8b6fc0648ed817530d7b57fe09a. Resources: {'ram', 'instances', 'cores'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:58:03.670389 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-055153b7-f540-4a40-b5de-b075704feb9f tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Getting quotas for user 303ba697a1f94661b93254ac0f198670 and project 8849a8b6fc0648ed817530d7b57fe09a. Resources: {'ram', 'instances', 'cores'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:58:03.684524 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-055153b7-f540-4a40-b5de-b075704feb9f tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Going to run 1 instances... {{(pid=138680) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 16 21:58:03.686846 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-055153b7-f540-4a40-b5de-b075704feb9f tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] [instance: f6044bec-67bc-46bd-9681-db14474b2a5b] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138680) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 16 21:58:03.797650 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-055153b7-f540-4a40-b5de-b075704feb9f tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.775111 Jul 16 21:58:03.803382 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3145/6295] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 21:58:03 2022] POST /compute/v2.1/servers => generated 380 bytes in 777 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 16 21:58:03.809073 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-d7950e7e-b836-4bf8-ac7b-df07ea1e86fa tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:58:03.809955 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-d7950e7e-b836-4bf8-ac7b-df07ea1e86fa tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] [instance: f6044bec-67bc-46bd-9681-db14474b2a5b] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:58:03.812730 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell0:SELECT=2 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:58:03.814102 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell0:SELECT=3 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:58:03.906264 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-d7950e7e-b836-4bf8-ac7b-df07ea1e86fa tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '303ba697a1f94661b93254ac0f198670', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8849a8b6fc0648ed817530d7b57fe09a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:58:03.915906 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-d7950e7e-b836-4bf8-ac7b-df07ea1e86fa tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] 10.222.0.16 "GET /compute/v2.1/servers/f6044bec-67bc-46bd-9681-db14474b2a5b" status: 200 len: 1214 microversion: 2.1 time: 0.110795 Jul 16 21:58:03.916503 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3151/6296] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:58:03 2022] GET /compute/v2.1/servers/f6044bec-67bc-46bd-9681-db14474b2a5b => generated 1214 bytes in 112 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:58:04.765876 ncdv-master-1 devstack@n-api.service[138680]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jul 16 21:58:04.766400 ncdv-master-1 devstack@n-api.service[138680]: Sat Jul 16 21:58:04 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Jul 16 21:58:04.933620 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-700e8e1e-d90a-4da1-bc3f-a47c7f5baff0 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:58:04.934375 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-700e8e1e-d90a-4da1-bc3f-a47c7f5baff0 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] [instance: f6044bec-67bc-46bd-9681-db14474b2a5b] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:58:04.941528 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-700e8e1e-d90a-4da1-bc3f-a47c7f5baff0 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:58:04.942074 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-700e8e1e-d90a-4da1-bc3f-a47c7f5baff0 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:58:04.942941 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-700e8e1e-d90a-4da1-bc3f-a47c7f5baff0 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:58:04.944542 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-700e8e1e-d90a-4da1-bc3f-a47c7f5baff0 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:58:05.013805 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-700e8e1e-d90a-4da1-bc3f-a47c7f5baff0 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '303ba697a1f94661b93254ac0f198670', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8849a8b6fc0648ed817530d7b57fe09a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:58:05.021512 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-700e8e1e-d90a-4da1-bc3f-a47c7f5baff0 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] 10.222.0.16 "GET /compute/v2.1/servers/f6044bec-67bc-46bd-9681-db14474b2a5b" status: 200 len: 1218 microversion: 2.1 time: 0.093052 Jul 16 21:58:05.022122 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3146/6297] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:58:04 2022] GET /compute/v2.1/servers/f6044bec-67bc-46bd-9681-db14474b2a5b => generated 1218 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:58:06.039257 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-ad81991f-9aa0-48b5-82e6-d90d447ecc60 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:58:06.046952 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-ad81991f-9aa0-48b5-82e6-d90d447ecc60 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] [instance: f6044bec-67bc-46bd-9681-db14474b2a5b] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:58:06.060726 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=53 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:58:06.066007 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ad81991f-9aa0-48b5-82e6-d90d447ecc60 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:58:06.066481 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ad81991f-9aa0-48b5-82e6-d90d447ecc60 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:58:06.067902 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ad81991f-9aa0-48b5-82e6-d90d447ecc60 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:58:06.068543 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ad81991f-9aa0-48b5-82e6-d90d447ecc60 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:58:06.152912 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-ad81991f-9aa0-48b5-82e6-d90d447ecc60 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '303ba697a1f94661b93254ac0f198670', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8849a8b6fc0648ed817530d7b57fe09a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:58:06.161814 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-ad81991f-9aa0-48b5-82e6-d90d447ecc60 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] 10.222.0.16 "GET /compute/v2.1/servers/f6044bec-67bc-46bd-9681-db14474b2a5b" status: 200 len: 1274 microversion: 2.1 time: 0.126306 Jul 16 21:58:06.162604 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3152/6298] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:58:06 2022] GET /compute/v2.1/servers/f6044bec-67bc-46bd-9681-db14474b2a5b => generated 1274 bytes in 128 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:58:07.180040 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-5f42021d-8943-401b-9617-f0e51c6f15c7 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:58:07.180851 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-5f42021d-8943-401b-9617-f0e51c6f15c7 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] [instance: f6044bec-67bc-46bd-9681-db14474b2a5b] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:58:07.188191 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5f42021d-8943-401b-9617-f0e51c6f15c7 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:58:07.188644 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5f42021d-8943-401b-9617-f0e51c6f15c7 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:58:07.189422 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5f42021d-8943-401b-9617-f0e51c6f15c7 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:58:07.189868 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5f42021d-8943-401b-9617-f0e51c6f15c7 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:58:07.338872 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-5f42021d-8943-401b-9617-f0e51c6f15c7 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '303ba697a1f94661b93254ac0f198670', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8849a8b6fc0648ed817530d7b57fe09a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:58:07.347522 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-5f42021d-8943-401b-9617-f0e51c6f15c7 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] 10.222.0.16 "GET /compute/v2.1/servers/f6044bec-67bc-46bd-9681-db14474b2a5b" status: 200 len: 1314 microversion: 2.1 time: 0.171454 Jul 16 21:58:07.348114 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3147/6299] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:58:07 2022] GET /compute/v2.1/servers/f6044bec-67bc-46bd-9681-db14474b2a5b => generated 1314 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:58:07.896627 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-254103ce-1caa-41d6-ba81-176c919ded8a req-534d045d-8c19-469a-ae6f-ab5b698cd11a service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "f6044bec-67bc-46bd-9681-db14474b2a5b", "tag": "18d15a1a-e539-4ace-a5ee-61272d6388ff"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:58:07.903830 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_api:SELECT=67,nova_api:INSERT=4,nova_api:DELETE=3 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:58:07.914956 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-254103ce-1caa-41d6-ba81-176c919ded8a req-534d045d-8c19-469a-ae6f-ab5b698cd11a service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:58:07.915455 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-254103ce-1caa-41d6-ba81-176c919ded8a req-534d045d-8c19-469a-ae6f-ab5b698cd11a service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:58:07.930183 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-254103ce-1caa-41d6-ba81-176c919ded8a req-534d045d-8c19-469a-ae6f-ab5b698cd11a service nova] Creating event network-changed:18d15a1a-e539-4ace-a5ee-61272d6388ff for instance f6044bec-67bc-46bd-9681-db14474b2a5b on nch1-master-1 Jul 16 21:58:07.935733 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-254103ce-1caa-41d6-ba81-176c919ded8a req-534d045d-8c19-469a-ae6f-ab5b698cd11a service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.043582 Jul 16 21:58:07.936933 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3153/6300] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:58:07 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:58:08.364124 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-28223710-3da6-449d-a7b0-ecc278114922 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:58:08.364746 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-28223710-3da6-449d-a7b0-ecc278114922 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] [instance: f6044bec-67bc-46bd-9681-db14474b2a5b] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:58:08.368670 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_api:SELECT=72,nova_api:INSERT=4,nova_api:DELETE=3,nova_api:UPDATE=1 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:58:08.373647 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-28223710-3da6-449d-a7b0-ecc278114922 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:58:08.374095 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-28223710-3da6-449d-a7b0-ecc278114922 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:58:08.374874 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-28223710-3da6-449d-a7b0-ecc278114922 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:58:08.375335 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-28223710-3da6-449d-a7b0-ecc278114922 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:58:08.516028 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-28223710-3da6-449d-a7b0-ecc278114922 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '303ba697a1f94661b93254ac0f198670', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8849a8b6fc0648ed817530d7b57fe09a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:58:08.524317 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-28223710-3da6-449d-a7b0-ecc278114922 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] 10.222.0.16 "GET /compute/v2.1/servers/f6044bec-67bc-46bd-9681-db14474b2a5b" status: 200 len: 1314 microversion: 2.1 time: 0.163426 Jul 16 21:58:08.533968 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3148/6301] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:58:08 2022] GET /compute/v2.1/servers/f6044bec-67bc-46bd-9681-db14474b2a5b => generated 1314 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:58:09.543480 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-24ce60ff-aa5b-4b3a-827c-fe77fc3141bc tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:58:09.544202 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-24ce60ff-aa5b-4b3a-827c-fe77fc3141bc tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] [instance: f6044bec-67bc-46bd-9681-db14474b2a5b] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:58:09.550994 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-24ce60ff-aa5b-4b3a-827c-fe77fc3141bc tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:58:09.551496 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-24ce60ff-aa5b-4b3a-827c-fe77fc3141bc tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:58:09.552268 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-24ce60ff-aa5b-4b3a-827c-fe77fc3141bc tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:58:09.552717 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-24ce60ff-aa5b-4b3a-827c-fe77fc3141bc tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:58:09.688569 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-24ce60ff-aa5b-4b3a-827c-fe77fc3141bc tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '303ba697a1f94661b93254ac0f198670', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8849a8b6fc0648ed817530d7b57fe09a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:58:09.696433 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-24ce60ff-aa5b-4b3a-827c-fe77fc3141bc tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] 10.222.0.16 "GET /compute/v2.1/servers/f6044bec-67bc-46bd-9681-db14474b2a5b" status: 200 len: 1314 microversion: 2.1 time: 0.158765 Jul 16 21:58:09.698877 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3154/6302] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:58:09 2022] GET /compute/v2.1/servers/f6044bec-67bc-46bd-9681-db14474b2a5b => generated 1314 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:58:10.713285 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-8533d546-5694-4581-b913-297bf7d86f77 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:58:10.713918 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-8533d546-5694-4581-b913-297bf7d86f77 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] [instance: f6044bec-67bc-46bd-9681-db14474b2a5b] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:58:10.721231 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8533d546-5694-4581-b913-297bf7d86f77 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:58:10.721696 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8533d546-5694-4581-b913-297bf7d86f77 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:58:10.722474 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8533d546-5694-4581-b913-297bf7d86f77 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:58:10.722925 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8533d546-5694-4581-b913-297bf7d86f77 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:58:10.725533 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:58:10.861537 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-8533d546-5694-4581-b913-297bf7d86f77 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '303ba697a1f94661b93254ac0f198670', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8849a8b6fc0648ed817530d7b57fe09a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:58:10.870380 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-8533d546-5694-4581-b913-297bf7d86f77 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] 10.222.0.16 "GET /compute/v2.1/servers/f6044bec-67bc-46bd-9681-db14474b2a5b" status: 200 len: 1314 microversion: 2.1 time: 0.161290 Jul 16 21:58:10.871064 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3149/6303] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:58:10 2022] GET /compute/v2.1/servers/f6044bec-67bc-46bd-9681-db14474b2a5b => generated 1314 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:58:11.888460 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-aa523ac2-cbb5-410a-ae24-7865094802fa tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:58:11.889225 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-aa523ac2-cbb5-410a-ae24-7865094802fa tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] [instance: f6044bec-67bc-46bd-9681-db14474b2a5b] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:58:11.896152 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-aa523ac2-cbb5-410a-ae24-7865094802fa tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:58:11.896614 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-aa523ac2-cbb5-410a-ae24-7865094802fa tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:58:11.897392 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-aa523ac2-cbb5-410a-ae24-7865094802fa tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:58:11.897902 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-aa523ac2-cbb5-410a-ae24-7865094802fa tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:58:12.032503 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-aa523ac2-cbb5-410a-ae24-7865094802fa tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '303ba697a1f94661b93254ac0f198670', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8849a8b6fc0648ed817530d7b57fe09a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:58:12.040219 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-aa523ac2-cbb5-410a-ae24-7865094802fa tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] 10.222.0.16 "GET /compute/v2.1/servers/f6044bec-67bc-46bd-9681-db14474b2a5b" status: 200 len: 1314 microversion: 2.1 time: 0.155755 Jul 16 21:58:12.040793 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3155/6304] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:58:11 2022] GET /compute/v2.1/servers/f6044bec-67bc-46bd-9681-db14474b2a5b => generated 1314 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:58:13.057356 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-7785a518-1af7-4bdd-9b76-31b8b3f7e159 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:58:13.057982 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-7785a518-1af7-4bdd-9b76-31b8b3f7e159 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] [instance: f6044bec-67bc-46bd-9681-db14474b2a5b] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:58:13.065078 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7785a518-1af7-4bdd-9b76-31b8b3f7e159 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:58:13.065542 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7785a518-1af7-4bdd-9b76-31b8b3f7e159 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:58:13.066315 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7785a518-1af7-4bdd-9b76-31b8b3f7e159 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:58:13.066763 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7785a518-1af7-4bdd-9b76-31b8b3f7e159 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:58:13.203000 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-7785a518-1af7-4bdd-9b76-31b8b3f7e159 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '303ba697a1f94661b93254ac0f198670', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8849a8b6fc0648ed817530d7b57fe09a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:58:13.210644 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-7785a518-1af7-4bdd-9b76-31b8b3f7e159 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] 10.222.0.16 "GET /compute/v2.1/servers/f6044bec-67bc-46bd-9681-db14474b2a5b" status: 200 len: 1314 microversion: 2.1 time: 0.157237 Jul 16 21:58:13.211909 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3150/6305] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:58:13 2022] GET /compute/v2.1/servers/f6044bec-67bc-46bd-9681-db14474b2a5b => generated 1314 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:58:14.089556 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-56acd260-1285-480e-8904-5e8ff09f1563 req-a06c2842-3da7-4415-8f1c-063a0fda5eb2 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "f6044bec-67bc-46bd-9681-db14474b2a5b", "name": "network-vif-plugged", "status": "completed", "tag": "18d15a1a-e539-4ace-a5ee-61272d6388ff"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:58:14.094283 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell0:SELECT=2 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:58:14.099943 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-56acd260-1285-480e-8904-5e8ff09f1563 req-a06c2842-3da7-4415-8f1c-063a0fda5eb2 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:58:14.100409 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-56acd260-1285-480e-8904-5e8ff09f1563 req-a06c2842-3da7-4415-8f1c-063a0fda5eb2 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:58:14.112318 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-56acd260-1285-480e-8904-5e8ff09f1563 req-a06c2842-3da7-4415-8f1c-063a0fda5eb2 service nova] Creating event network-vif-plugged:18d15a1a-e539-4ace-a5ee-61272d6388ff for instance f6044bec-67bc-46bd-9681-db14474b2a5b on nch1-master-1 Jul 16 21:58:14.117491 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-56acd260-1285-480e-8904-5e8ff09f1563 req-a06c2842-3da7-4415-8f1c-063a0fda5eb2 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.032804 Jul 16 21:58:14.118083 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3156/6306] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:58:14 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:58:14.229835 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-671dedaa-c42a-4fcf-8ba0-a689f75dd6f3 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:58:14.230462 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-671dedaa-c42a-4fcf-8ba0-a689f75dd6f3 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] [instance: f6044bec-67bc-46bd-9681-db14474b2a5b] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:58:14.233392 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell0:SELECT=2 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:58:14.234069 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell0:SELECT=6 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:58:14.238632 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-671dedaa-c42a-4fcf-8ba0-a689f75dd6f3 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:58:14.239116 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-671dedaa-c42a-4fcf-8ba0-a689f75dd6f3 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:58:14.239894 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-671dedaa-c42a-4fcf-8ba0-a689f75dd6f3 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:58:14.240339 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-671dedaa-c42a-4fcf-8ba0-a689f75dd6f3 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:58:14.380120 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-671dedaa-c42a-4fcf-8ba0-a689f75dd6f3 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '303ba697a1f94661b93254ac0f198670', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8849a8b6fc0648ed817530d7b57fe09a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:58:14.388250 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-671dedaa-c42a-4fcf-8ba0-a689f75dd6f3 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] 10.222.0.16 "GET /compute/v2.1/servers/f6044bec-67bc-46bd-9681-db14474b2a5b" status: 200 len: 1314 microversion: 2.1 time: 0.163531 Jul 16 21:58:14.388824 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3151/6307] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:58:14 2022] GET /compute/v2.1/servers/f6044bec-67bc-46bd-9681-db14474b2a5b => generated 1314 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:58:15.406815 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-eba1f63d-47b4-4783-9b23-bc3e85522557 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:58:15.407511 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-eba1f63d-47b4-4783-9b23-bc3e85522557 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] [instance: f6044bec-67bc-46bd-9681-db14474b2a5b] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:58:15.414857 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-eba1f63d-47b4-4783-9b23-bc3e85522557 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:58:15.416953 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-eba1f63d-47b4-4783-9b23-bc3e85522557 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:58:15.416953 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-eba1f63d-47b4-4783-9b23-bc3e85522557 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:58:15.416953 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-eba1f63d-47b4-4783-9b23-bc3e85522557 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:58:15.551891 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-eba1f63d-47b4-4783-9b23-bc3e85522557 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '303ba697a1f94661b93254ac0f198670', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8849a8b6fc0648ed817530d7b57fe09a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:58:15.559702 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-eba1f63d-47b4-4783-9b23-bc3e85522557 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] 10.222.0.16 "GET /compute/v2.1/servers/f6044bec-67bc-46bd-9681-db14474b2a5b" status: 200 len: 1501 microversion: 2.1 time: 0.158151 Jul 16 21:58:15.560268 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3157/6308] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:58:15 2022] GET /compute/v2.1/servers/f6044bec-67bc-46bd-9681-db14474b2a5b => generated 1501 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:58:15.575992 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-f10ca1e9-4de8-4b37-b716-415ad7d97bd6 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Action: 'action', calling method: >, body: {"resize": {"flavorRef": "451"}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:58:15.577039 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-f10ca1e9-4de8-4b37-b716-415ad7d97bd6 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] [instance: f6044bec-67bc-46bd-9681-db14474b2a5b] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:58:15.583999 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f10ca1e9-4de8-4b37-b716-415ad7d97bd6 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:58:15.584748 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f10ca1e9-4de8-4b37-b716-415ad7d97bd6 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:58:15.585530 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f10ca1e9-4de8-4b37-b716-415ad7d97bd6 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:58:15.586510 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f10ca1e9-4de8-4b37-b716-415ad7d97bd6 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:58:15.615142 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-f10ca1e9-4de8-4b37-b716-415ad7d97bd6 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Policy check for compute:servers:resize:cross_cell failed with credentials {'is_admin': False, 'user_id': '303ba697a1f94661b93254ac0f198670', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8849a8b6fc0648ed817530d7b57fe09a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:58:15.621006 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-f10ca1e9-4de8-4b37-b716-415ad7d97bd6 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] [instance: f6044bec-67bc-46bd-9681-db14474b2a5b] Old instance type m1.heat2, new instance type m1.heat {{(pid=138680) resize /opt/stack/nova/nova/compute/api.py:4166}} Jul 16 21:58:15.627338 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-f10ca1e9-4de8-4b37-b716-415ad7d97bd6 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Flavor limits 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 16 21:58:15.627767 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-f10ca1e9-4de8-4b37-b716-415ad7d97bd6 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Image limits 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 16 21:58:15.628243 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-f10ca1e9-4de8-4b37-b716-415ad7d97bd6 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Flavor pref 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 16 21:58:15.628656 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-f10ca1e9-4de8-4b37-b716-415ad7d97bd6 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Image pref 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 16 21:58:15.629069 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-f10ca1e9-4de8-4b37-b716-415ad7d97bd6 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 16 21:58:15.854568 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-f10ca1e9-4de8-4b37-b716-415ad7d97bd6 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] 10.222.0.16 "POST /compute/v2.1/servers/f6044bec-67bc-46bd-9681-db14474b2a5b/action" status: 202 len: 0 microversion: 2.1 time: 0.282400 Jul 16 21:58:15.855223 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3152/6309] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 21:58:15 2022] POST /compute/v2.1/servers/f6044bec-67bc-46bd-9681-db14474b2a5b/action => generated 0 bytes in 284 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 21:58:15.864242 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-fd056c9b-5ddb-4c20-978c-d2540e738395 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:58:15.864689 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-fd056c9b-5ddb-4c20-978c-d2540e738395 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] [instance: f6044bec-67bc-46bd-9681-db14474b2a5b] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:58:15.871099 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fd056c9b-5ddb-4c20-978c-d2540e738395 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:58:15.871590 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fd056c9b-5ddb-4c20-978c-d2540e738395 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:58:15.872430 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fd056c9b-5ddb-4c20-978c-d2540e738395 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:58:15.872869 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fd056c9b-5ddb-4c20-978c-d2540e738395 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:58:16.007207 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-fd056c9b-5ddb-4c20-978c-d2540e738395 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '303ba697a1f94661b93254ac0f198670', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8849a8b6fc0648ed817530d7b57fe09a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:58:16.015451 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-fd056c9b-5ddb-4c20-978c-d2540e738395 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] 10.222.0.16 "GET /compute/v2.1/servers/f6044bec-67bc-46bd-9681-db14474b2a5b" status: 200 len: 1510 microversion: 2.1 time: 0.155753 Jul 16 21:58:16.018691 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3158/6310] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:58:15 2022] GET /compute/v2.1/servers/f6044bec-67bc-46bd-9681-db14474b2a5b => generated 1510 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:58:17.034716 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-388bb1cd-a0cd-4a28-9592-c20227d7f7ec tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:58:17.035358 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-388bb1cd-a0cd-4a28-9592-c20227d7f7ec tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] [instance: f6044bec-67bc-46bd-9681-db14474b2a5b] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:58:17.041952 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-388bb1cd-a0cd-4a28-9592-c20227d7f7ec tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:58:17.042413 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-388bb1cd-a0cd-4a28-9592-c20227d7f7ec tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:58:17.043219 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-388bb1cd-a0cd-4a28-9592-c20227d7f7ec tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:58:17.043668 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-388bb1cd-a0cd-4a28-9592-c20227d7f7ec tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:58:17.186474 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-388bb1cd-a0cd-4a28-9592-c20227d7f7ec tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '303ba697a1f94661b93254ac0f198670', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8849a8b6fc0648ed817530d7b57fe09a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:58:17.194124 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-388bb1cd-a0cd-4a28-9592-c20227d7f7ec tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] 10.222.0.16 "GET /compute/v2.1/servers/f6044bec-67bc-46bd-9681-db14474b2a5b" status: 200 len: 1510 microversion: 2.1 time: 0.164278 Jul 16 21:58:17.194714 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3153/6311] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:58:17 2022] GET /compute/v2.1/servers/f6044bec-67bc-46bd-9681-db14474b2a5b => generated 1510 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:58:18.212674 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-2197ff64-063b-4d7e-900a-2bcac0c82f83 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:58:18.213298 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-2197ff64-063b-4d7e-900a-2bcac0c82f83 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] [instance: f6044bec-67bc-46bd-9681-db14474b2a5b] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:58:18.220763 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2197ff64-063b-4d7e-900a-2bcac0c82f83 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:58:18.221229 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2197ff64-063b-4d7e-900a-2bcac0c82f83 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:58:18.222005 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2197ff64-063b-4d7e-900a-2bcac0c82f83 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:58:18.222455 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2197ff64-063b-4d7e-900a-2bcac0c82f83 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:58:18.357542 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-2197ff64-063b-4d7e-900a-2bcac0c82f83 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '303ba697a1f94661b93254ac0f198670', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8849a8b6fc0648ed817530d7b57fe09a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:58:18.365121 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-2197ff64-063b-4d7e-900a-2bcac0c82f83 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] 10.222.0.16 "GET /compute/v2.1/servers/f6044bec-67bc-46bd-9681-db14474b2a5b" status: 200 len: 1510 microversion: 2.1 time: 0.157105 Jul 16 21:58:18.365712 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3159/6312] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:58:18 2022] GET /compute/v2.1/servers/f6044bec-67bc-46bd-9681-db14474b2a5b => generated 1510 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:58:19.381463 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-7be42f0c-d7e4-4357-ad13-a7cd1182e7a4 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:58:19.382077 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-7be42f0c-d7e4-4357-ad13-a7cd1182e7a4 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] [instance: f6044bec-67bc-46bd-9681-db14474b2a5b] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:58:19.390638 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7be42f0c-d7e4-4357-ad13-a7cd1182e7a4 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:58:19.391146 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7be42f0c-d7e4-4357-ad13-a7cd1182e7a4 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:58:19.392429 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7be42f0c-d7e4-4357-ad13-a7cd1182e7a4 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:58:19.392885 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7be42f0c-d7e4-4357-ad13-a7cd1182e7a4 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:58:19.528436 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-7be42f0c-d7e4-4357-ad13-a7cd1182e7a4 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '303ba697a1f94661b93254ac0f198670', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8849a8b6fc0648ed817530d7b57fe09a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:58:19.536351 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-7be42f0c-d7e4-4357-ad13-a7cd1182e7a4 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] 10.222.0.16 "GET /compute/v2.1/servers/f6044bec-67bc-46bd-9681-db14474b2a5b" status: 200 len: 1515 microversion: 2.1 time: 0.158885 Jul 16 21:58:19.536942 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3154/6313] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:58:19 2022] GET /compute/v2.1/servers/f6044bec-67bc-46bd-9681-db14474b2a5b => generated 1515 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:58:20.556109 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-35329be8-1e77-47d1-97ff-62ee2531a596 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:58:20.557296 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-35329be8-1e77-47d1-97ff-62ee2531a596 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] [instance: f6044bec-67bc-46bd-9681-db14474b2a5b] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:58:20.563846 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-35329be8-1e77-47d1-97ff-62ee2531a596 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:58:20.564297 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-35329be8-1e77-47d1-97ff-62ee2531a596 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:58:20.565108 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-35329be8-1e77-47d1-97ff-62ee2531a596 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:58:20.565569 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-35329be8-1e77-47d1-97ff-62ee2531a596 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:58:20.703195 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-35329be8-1e77-47d1-97ff-62ee2531a596 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '303ba697a1f94661b93254ac0f198670', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8849a8b6fc0648ed817530d7b57fe09a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:58:20.710698 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-35329be8-1e77-47d1-97ff-62ee2531a596 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] 10.222.0.16 "GET /compute/v2.1/servers/f6044bec-67bc-46bd-9681-db14474b2a5b" status: 200 len: 1515 microversion: 2.1 time: 0.158478 Jul 16 21:58:20.711295 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3160/6314] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:58:20 2022] GET /compute/v2.1/servers/f6044bec-67bc-46bd-9681-db14474b2a5b => generated 1515 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:58:21.729336 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-1456bed9-37d6-469f-930d-b87096ac6aad tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:58:21.729957 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-1456bed9-37d6-469f-930d-b87096ac6aad tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] [instance: f6044bec-67bc-46bd-9681-db14474b2a5b] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:58:21.736967 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1456bed9-37d6-469f-930d-b87096ac6aad tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:58:21.737428 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1456bed9-37d6-469f-930d-b87096ac6aad tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:58:21.738205 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1456bed9-37d6-469f-930d-b87096ac6aad tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:58:21.738649 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1456bed9-37d6-469f-930d-b87096ac6aad tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:58:21.875856 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-1456bed9-37d6-469f-930d-b87096ac6aad tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '303ba697a1f94661b93254ac0f198670', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8849a8b6fc0648ed817530d7b57fe09a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:58:21.877051 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-1456bed9-37d6-469f-930d-b87096ac6aad tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] 10.222.0.16 "GET /compute/v2.1/servers/f6044bec-67bc-46bd-9681-db14474b2a5b" status: 200 len: 1515 microversion: 2.1 time: 0.152359 Jul 16 21:58:21.877620 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3155/6315] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:58:21 2022] GET /compute/v2.1/servers/f6044bec-67bc-46bd-9681-db14474b2a5b => generated 1515 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:58:22.893902 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-879f8190-1df7-4487-87cc-a20052646bbf tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:58:22.894532 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-879f8190-1df7-4487-87cc-a20052646bbf tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] [instance: f6044bec-67bc-46bd-9681-db14474b2a5b] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:58:22.902328 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-879f8190-1df7-4487-87cc-a20052646bbf tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:58:22.902796 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-879f8190-1df7-4487-87cc-a20052646bbf tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:58:22.903598 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-879f8190-1df7-4487-87cc-a20052646bbf tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:58:22.904061 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-879f8190-1df7-4487-87cc-a20052646bbf tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:58:23.037569 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-879f8190-1df7-4487-87cc-a20052646bbf tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '303ba697a1f94661b93254ac0f198670', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8849a8b6fc0648ed817530d7b57fe09a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:58:23.045579 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-879f8190-1df7-4487-87cc-a20052646bbf tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] 10.222.0.16 "GET /compute/v2.1/servers/f6044bec-67bc-46bd-9681-db14474b2a5b" status: 200 len: 1515 microversion: 2.1 time: 0.155771 Jul 16 21:58:23.046156 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3161/6316] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:58:22 2022] GET /compute/v2.1/servers/f6044bec-67bc-46bd-9681-db14474b2a5b => generated 1515 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:58:24.061993 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-2788431c-b685-4a2c-89a1-14457ccd541d tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:58:24.062606 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-2788431c-b685-4a2c-89a1-14457ccd541d tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] [instance: f6044bec-67bc-46bd-9681-db14474b2a5b] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:58:24.069525 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2788431c-b685-4a2c-89a1-14457ccd541d tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:58:24.069976 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2788431c-b685-4a2c-89a1-14457ccd541d tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:58:24.070730 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2788431c-b685-4a2c-89a1-14457ccd541d tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:58:24.071201 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2788431c-b685-4a2c-89a1-14457ccd541d tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:58:24.205153 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-2788431c-b685-4a2c-89a1-14457ccd541d tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '303ba697a1f94661b93254ac0f198670', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8849a8b6fc0648ed817530d7b57fe09a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:58:24.213443 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-2788431c-b685-4a2c-89a1-14457ccd541d tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] 10.222.0.16 "GET /compute/v2.1/servers/f6044bec-67bc-46bd-9681-db14474b2a5b" status: 200 len: 1515 microversion: 2.1 time: 0.154827 Jul 16 21:58:24.214284 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3156/6317] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:58:24 2022] GET /compute/v2.1/servers/f6044bec-67bc-46bd-9681-db14474b2a5b => generated 1515 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:58:25.230404 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-63b98e7a-a3e7-4942-bd35-f7945127ef59 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:58:25.230980 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-63b98e7a-a3e7-4942-bd35-f7945127ef59 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] [instance: f6044bec-67bc-46bd-9681-db14474b2a5b] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:58:25.238104 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-63b98e7a-a3e7-4942-bd35-f7945127ef59 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:58:25.238573 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-63b98e7a-a3e7-4942-bd35-f7945127ef59 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:58:25.239382 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-63b98e7a-a3e7-4942-bd35-f7945127ef59 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:58:25.239840 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-63b98e7a-a3e7-4942-bd35-f7945127ef59 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:58:25.372026 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-63b98e7a-a3e7-4942-bd35-f7945127ef59 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '303ba697a1f94661b93254ac0f198670', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8849a8b6fc0648ed817530d7b57fe09a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:58:25.379648 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-63b98e7a-a3e7-4942-bd35-f7945127ef59 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] 10.222.0.16 "GET /compute/v2.1/servers/f6044bec-67bc-46bd-9681-db14474b2a5b" status: 200 len: 1515 microversion: 2.1 time: 0.153429 Jul 16 21:58:25.380226 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3162/6318] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:58:25 2022] GET /compute/v2.1/servers/f6044bec-67bc-46bd-9681-db14474b2a5b => generated 1515 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:58:26.397450 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-e0a681c6-56a2-4f95-b6fe-69a148f7e354 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:58:26.398054 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-e0a681c6-56a2-4f95-b6fe-69a148f7e354 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] [instance: f6044bec-67bc-46bd-9681-db14474b2a5b] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:58:26.406430 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e0a681c6-56a2-4f95-b6fe-69a148f7e354 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:58:26.406934 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e0a681c6-56a2-4f95-b6fe-69a148f7e354 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:58:26.407781 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e0a681c6-56a2-4f95-b6fe-69a148f7e354 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:58:26.408229 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e0a681c6-56a2-4f95-b6fe-69a148f7e354 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:58:26.859475 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-e0a681c6-56a2-4f95-b6fe-69a148f7e354 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '303ba697a1f94661b93254ac0f198670', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8849a8b6fc0648ed817530d7b57fe09a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:58:26.867442 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-e0a681c6-56a2-4f95-b6fe-69a148f7e354 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] 10.222.0.16 "GET /compute/v2.1/servers/f6044bec-67bc-46bd-9681-db14474b2a5b" status: 200 len: 1515 microversion: 2.1 time: 0.473839 Jul 16 21:58:26.868115 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3157/6319] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:58:26 2022] GET /compute/v2.1/servers/f6044bec-67bc-46bd-9681-db14474b2a5b => generated 1515 bytes in 475 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:58:27.884682 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-8738a18a-95f0-4ecc-8b13-fc54ee11082f tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:58:27.885337 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-8738a18a-95f0-4ecc-8b13-fc54ee11082f tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] [instance: f6044bec-67bc-46bd-9681-db14474b2a5b] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:58:27.892444 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8738a18a-95f0-4ecc-8b13-fc54ee11082f tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:58:27.892910 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8738a18a-95f0-4ecc-8b13-fc54ee11082f tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:58:27.893713 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8738a18a-95f0-4ecc-8b13-fc54ee11082f tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:58:27.894163 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8738a18a-95f0-4ecc-8b13-fc54ee11082f tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:58:28.023770 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-8738a18a-95f0-4ecc-8b13-fc54ee11082f tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '303ba697a1f94661b93254ac0f198670', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8849a8b6fc0648ed817530d7b57fe09a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:58:28.031960 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-8738a18a-95f0-4ecc-8b13-fc54ee11082f tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] 10.222.0.16 "GET /compute/v2.1/servers/f6044bec-67bc-46bd-9681-db14474b2a5b" status: 200 len: 1515 microversion: 2.1 time: 0.151769 Jul 16 21:58:28.032538 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3163/6320] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:58:27 2022] GET /compute/v2.1/servers/f6044bec-67bc-46bd-9681-db14474b2a5b => generated 1515 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:58:29.049606 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-09a01a7d-7af4-4d46-9710-fda7426abb36 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:58:29.050849 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-09a01a7d-7af4-4d46-9710-fda7426abb36 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] [instance: f6044bec-67bc-46bd-9681-db14474b2a5b] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:58:29.058098 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-09a01a7d-7af4-4d46-9710-fda7426abb36 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:58:29.058559 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-09a01a7d-7af4-4d46-9710-fda7426abb36 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:58:29.059459 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-09a01a7d-7af4-4d46-9710-fda7426abb36 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:58:29.059920 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-09a01a7d-7af4-4d46-9710-fda7426abb36 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:58:29.197998 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-09a01a7d-7af4-4d46-9710-fda7426abb36 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '303ba697a1f94661b93254ac0f198670', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8849a8b6fc0648ed817530d7b57fe09a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:58:29.205899 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-09a01a7d-7af4-4d46-9710-fda7426abb36 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] 10.222.0.16 "GET /compute/v2.1/servers/f6044bec-67bc-46bd-9681-db14474b2a5b" status: 200 len: 1515 microversion: 2.1 time: 0.160188 Jul 16 21:58:29.208885 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3158/6321] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:58:29 2022] GET /compute/v2.1/servers/f6044bec-67bc-46bd-9681-db14474b2a5b => generated 1515 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:58:30.223900 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-ad7aaec0-904e-4077-9a10-ecee0d9ee070 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:58:30.224534 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-ad7aaec0-904e-4077-9a10-ecee0d9ee070 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] [instance: f6044bec-67bc-46bd-9681-db14474b2a5b] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:58:30.232294 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ad7aaec0-904e-4077-9a10-ecee0d9ee070 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:58:30.232710 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ad7aaec0-904e-4077-9a10-ecee0d9ee070 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:58:30.238968 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ad7aaec0-904e-4077-9a10-ecee0d9ee070 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:58:30.238968 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ad7aaec0-904e-4077-9a10-ecee0d9ee070 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:58:30.367336 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-ad7aaec0-904e-4077-9a10-ecee0d9ee070 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '303ba697a1f94661b93254ac0f198670', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8849a8b6fc0648ed817530d7b57fe09a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:58:30.375709 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-ad7aaec0-904e-4077-9a10-ecee0d9ee070 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] 10.222.0.16 "GET /compute/v2.1/servers/f6044bec-67bc-46bd-9681-db14474b2a5b" status: 200 len: 1515 microversion: 2.1 time: 0.155888 Jul 16 21:58:30.376284 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3164/6322] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:58:30 2022] GET /compute/v2.1/servers/f6044bec-67bc-46bd-9681-db14474b2a5b => generated 1515 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:58:31.393332 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-2e35863e-285e-4ef1-8e57-c03024a0ad39 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:58:31.393988 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-2e35863e-285e-4ef1-8e57-c03024a0ad39 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] [instance: f6044bec-67bc-46bd-9681-db14474b2a5b] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:58:31.401031 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2e35863e-285e-4ef1-8e57-c03024a0ad39 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:58:31.401489 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2e35863e-285e-4ef1-8e57-c03024a0ad39 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:58:31.402259 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2e35863e-285e-4ef1-8e57-c03024a0ad39 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:58:31.402709 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2e35863e-285e-4ef1-8e57-c03024a0ad39 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:58:31.535559 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-2e35863e-285e-4ef1-8e57-c03024a0ad39 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '303ba697a1f94661b93254ac0f198670', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8849a8b6fc0648ed817530d7b57fe09a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:58:31.543981 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-2e35863e-285e-4ef1-8e57-c03024a0ad39 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] 10.222.0.16 "GET /compute/v2.1/servers/f6044bec-67bc-46bd-9681-db14474b2a5b" status: 200 len: 1515 microversion: 2.1 time: 0.154531 Jul 16 21:58:31.544579 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3159/6323] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:58:31 2022] GET /compute/v2.1/servers/f6044bec-67bc-46bd-9681-db14474b2a5b => generated 1515 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:58:32.561968 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-72e73bc7-c53c-400c-876b-01970c9e424e tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:58:32.562516 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-72e73bc7-c53c-400c-876b-01970c9e424e tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] [instance: f6044bec-67bc-46bd-9681-db14474b2a5b] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:58:32.569909 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-72e73bc7-c53c-400c-876b-01970c9e424e tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:58:32.570379 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-72e73bc7-c53c-400c-876b-01970c9e424e tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:58:32.571187 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-72e73bc7-c53c-400c-876b-01970c9e424e tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:58:32.571790 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-72e73bc7-c53c-400c-876b-01970c9e424e tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:58:32.702153 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-72e73bc7-c53c-400c-876b-01970c9e424e tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '303ba697a1f94661b93254ac0f198670', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8849a8b6fc0648ed817530d7b57fe09a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:58:32.709759 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-72e73bc7-c53c-400c-876b-01970c9e424e tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] 10.222.0.16 "GET /compute/v2.1/servers/f6044bec-67bc-46bd-9681-db14474b2a5b" status: 200 len: 1515 microversion: 2.1 time: 0.151925 Jul 16 21:58:32.710350 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3165/6324] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:58:32 2022] GET /compute/v2.1/servers/f6044bec-67bc-46bd-9681-db14474b2a5b => generated 1515 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:58:33.728101 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-61854aaf-14a3-4378-9a63-2e5411eb7cd2 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:58:33.728840 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-61854aaf-14a3-4378-9a63-2e5411eb7cd2 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] [instance: f6044bec-67bc-46bd-9681-db14474b2a5b] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:58:33.735726 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-61854aaf-14a3-4378-9a63-2e5411eb7cd2 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:58:33.736180 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-61854aaf-14a3-4378-9a63-2e5411eb7cd2 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:58:33.736944 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-61854aaf-14a3-4378-9a63-2e5411eb7cd2 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:58:33.737387 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-61854aaf-14a3-4378-9a63-2e5411eb7cd2 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:58:33.875051 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-61854aaf-14a3-4378-9a63-2e5411eb7cd2 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '303ba697a1f94661b93254ac0f198670', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8849a8b6fc0648ed817530d7b57fe09a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:58:33.882826 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-61854aaf-14a3-4378-9a63-2e5411eb7cd2 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] 10.222.0.16 "GET /compute/v2.1/servers/f6044bec-67bc-46bd-9681-db14474b2a5b" status: 200 len: 1515 microversion: 2.1 time: 0.158690 Jul 16 21:58:33.883438 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3160/6325] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:58:33 2022] GET /compute/v2.1/servers/f6044bec-67bc-46bd-9681-db14474b2a5b => generated 1515 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:58:34.901316 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-60be1922-45d1-45d1-aabb-4d0647e977f3 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:58:34.901948 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-60be1922-45d1-45d1-aabb-4d0647e977f3 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] [instance: f6044bec-67bc-46bd-9681-db14474b2a5b] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:58:34.908253 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-60be1922-45d1-45d1-aabb-4d0647e977f3 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:58:34.908713 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-60be1922-45d1-45d1-aabb-4d0647e977f3 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:58:34.909573 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-60be1922-45d1-45d1-aabb-4d0647e977f3 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:58:34.910020 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-60be1922-45d1-45d1-aabb-4d0647e977f3 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:58:35.043582 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-60be1922-45d1-45d1-aabb-4d0647e977f3 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '303ba697a1f94661b93254ac0f198670', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8849a8b6fc0648ed817530d7b57fe09a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:58:35.050993 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-60be1922-45d1-45d1-aabb-4d0647e977f3 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] 10.222.0.16 "GET /compute/v2.1/servers/f6044bec-67bc-46bd-9681-db14474b2a5b" status: 200 len: 1515 microversion: 2.1 time: 0.154725 Jul 16 21:58:35.051591 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3166/6326] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:58:34 2022] GET /compute/v2.1/servers/f6044bec-67bc-46bd-9681-db14474b2a5b => generated 1515 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:58:36.069198 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-8bf8a40c-1da8-491a-b7ed-2835a28bfb99 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:58:36.069905 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-8bf8a40c-1da8-491a-b7ed-2835a28bfb99 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] [instance: f6044bec-67bc-46bd-9681-db14474b2a5b] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:58:36.077189 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8bf8a40c-1da8-491a-b7ed-2835a28bfb99 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:58:36.082195 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8bf8a40c-1da8-491a-b7ed-2835a28bfb99 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:58:36.082195 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8bf8a40c-1da8-491a-b7ed-2835a28bfb99 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:58:36.082195 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8bf8a40c-1da8-491a-b7ed-2835a28bfb99 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:58:36.210063 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-8bf8a40c-1da8-491a-b7ed-2835a28bfb99 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '303ba697a1f94661b93254ac0f198670', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8849a8b6fc0648ed817530d7b57fe09a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:58:36.218766 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-8bf8a40c-1da8-491a-b7ed-2835a28bfb99 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] 10.222.0.16 "GET /compute/v2.1/servers/f6044bec-67bc-46bd-9681-db14474b2a5b" status: 200 len: 1515 microversion: 2.1 time: 0.153487 Jul 16 21:58:36.219378 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3161/6327] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:58:36 2022] GET /compute/v2.1/servers/f6044bec-67bc-46bd-9681-db14474b2a5b => generated 1515 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:58:37.235917 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-5a0ffa97-fab2-4e0b-9277-b8429bdbf663 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:58:37.236546 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-5a0ffa97-fab2-4e0b-9277-b8429bdbf663 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] [instance: f6044bec-67bc-46bd-9681-db14474b2a5b] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:58:37.243649 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5a0ffa97-fab2-4e0b-9277-b8429bdbf663 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:58:37.244530 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5a0ffa97-fab2-4e0b-9277-b8429bdbf663 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:58:37.244878 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5a0ffa97-fab2-4e0b-9277-b8429bdbf663 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:58:37.245332 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5a0ffa97-fab2-4e0b-9277-b8429bdbf663 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:58:37.378733 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-5a0ffa97-fab2-4e0b-9277-b8429bdbf663 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '303ba697a1f94661b93254ac0f198670', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8849a8b6fc0648ed817530d7b57fe09a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:58:37.386576 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-5a0ffa97-fab2-4e0b-9277-b8429bdbf663 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] 10.222.0.16 "GET /compute/v2.1/servers/f6044bec-67bc-46bd-9681-db14474b2a5b" status: 200 len: 1515 microversion: 2.1 time: 0.154613 Jul 16 21:58:37.387162 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3167/6328] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:58:37 2022] GET /compute/v2.1/servers/f6044bec-67bc-46bd-9681-db14474b2a5b => generated 1515 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:58:38.403481 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-3e5dad09-6c80-427e-9f38-a8b36ae2cc04 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:58:38.404099 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-3e5dad09-6c80-427e-9f38-a8b36ae2cc04 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] [instance: f6044bec-67bc-46bd-9681-db14474b2a5b] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:58:38.412040 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3e5dad09-6c80-427e-9f38-a8b36ae2cc04 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:58:38.415879 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3e5dad09-6c80-427e-9f38-a8b36ae2cc04 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:58:38.415879 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3e5dad09-6c80-427e-9f38-a8b36ae2cc04 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:58:38.415879 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3e5dad09-6c80-427e-9f38-a8b36ae2cc04 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:58:38.564862 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-3e5dad09-6c80-427e-9f38-a8b36ae2cc04 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '303ba697a1f94661b93254ac0f198670', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8849a8b6fc0648ed817530d7b57fe09a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:58:38.572897 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-3e5dad09-6c80-427e-9f38-a8b36ae2cc04 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] 10.222.0.16 "GET /compute/v2.1/servers/f6044bec-67bc-46bd-9681-db14474b2a5b" status: 200 len: 1515 microversion: 2.1 time: 0.173746 Jul 16 21:58:38.573473 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3162/6329] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:58:38 2022] GET /compute/v2.1/servers/f6044bec-67bc-46bd-9681-db14474b2a5b => generated 1515 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:58:39.591367 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-f50bcfd6-a958-40a7-8329-8defea88e02f tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:58:39.592307 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-f50bcfd6-a958-40a7-8329-8defea88e02f tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] [instance: f6044bec-67bc-46bd-9681-db14474b2a5b] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:58:39.599210 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f50bcfd6-a958-40a7-8329-8defea88e02f tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:58:39.599682 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f50bcfd6-a958-40a7-8329-8defea88e02f tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:58:39.600464 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f50bcfd6-a958-40a7-8329-8defea88e02f tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:58:39.600917 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f50bcfd6-a958-40a7-8329-8defea88e02f tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:58:39.604334 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=89 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:58:39.752076 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-f50bcfd6-a958-40a7-8329-8defea88e02f tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '303ba697a1f94661b93254ac0f198670', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8849a8b6fc0648ed817530d7b57fe09a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:58:39.760820 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-f50bcfd6-a958-40a7-8329-8defea88e02f tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] 10.222.0.16 "GET /compute/v2.1/servers/f6044bec-67bc-46bd-9681-db14474b2a5b" status: 200 len: 1515 microversion: 2.1 time: 0.174449 Jul 16 21:58:39.761495 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3168/6330] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:58:39 2022] GET /compute/v2.1/servers/f6044bec-67bc-46bd-9681-db14474b2a5b => generated 1515 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:58:40.778331 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-49e44c1b-62e1-4c25-a127-75918cd3e702 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:58:40.778954 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-49e44c1b-62e1-4c25-a127-75918cd3e702 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] [instance: f6044bec-67bc-46bd-9681-db14474b2a5b] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:58:40.781769 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_api:SELECT=34 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:58:40.786604 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-49e44c1b-62e1-4c25-a127-75918cd3e702 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:58:40.787086 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-49e44c1b-62e1-4c25-a127-75918cd3e702 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:58:40.787885 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-49e44c1b-62e1-4c25-a127-75918cd3e702 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:58:40.788326 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-49e44c1b-62e1-4c25-a127-75918cd3e702 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:58:40.919448 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-49e44c1b-62e1-4c25-a127-75918cd3e702 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '303ba697a1f94661b93254ac0f198670', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8849a8b6fc0648ed817530d7b57fe09a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:58:40.921725 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=94,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:58:40.927747 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-49e44c1b-62e1-4c25-a127-75918cd3e702 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] 10.222.0.16 "GET /compute/v2.1/servers/f6044bec-67bc-46bd-9681-db14474b2a5b" status: 200 len: 1515 microversion: 2.1 time: 0.154014 Jul 16 21:58:40.934481 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3163/6331] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:58:40 2022] GET /compute/v2.1/servers/f6044bec-67bc-46bd-9681-db14474b2a5b => generated 1515 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:58:41.945699 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-cb97b7e1-199c-4ddb-8a32-234b5a99e9fd tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:58:41.946326 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-cb97b7e1-199c-4ddb-8a32-234b5a99e9fd tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] [instance: f6044bec-67bc-46bd-9681-db14474b2a5b] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:58:41.949605 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_api:SELECT=32 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:58:41.954390 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cb97b7e1-199c-4ddb-8a32-234b5a99e9fd tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:58:41.954857 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cb97b7e1-199c-4ddb-8a32-234b5a99e9fd tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:58:41.955670 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cb97b7e1-199c-4ddb-8a32-234b5a99e9fd tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:58:41.956118 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cb97b7e1-199c-4ddb-8a32-234b5a99e9fd tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:58:42.085152 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-cb97b7e1-199c-4ddb-8a32-234b5a99e9fd tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '303ba697a1f94661b93254ac0f198670', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8849a8b6fc0648ed817530d7b57fe09a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:58:42.093715 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-cb97b7e1-199c-4ddb-8a32-234b5a99e9fd tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] 10.222.0.16 "GET /compute/v2.1/servers/f6044bec-67bc-46bd-9681-db14474b2a5b" status: 200 len: 1515 microversion: 2.1 time: 0.151905 Jul 16 21:58:42.094294 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3169/6332] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:58:41 2022] GET /compute/v2.1/servers/f6044bec-67bc-46bd-9681-db14474b2a5b => generated 1515 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:58:43.110730 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-0ca8ef38-2781-499e-aef1-e43fd9b4bc81 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:58:43.111418 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-0ca8ef38-2781-499e-aef1-e43fd9b4bc81 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] [instance: f6044bec-67bc-46bd-9681-db14474b2a5b] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:58:43.118878 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0ca8ef38-2781-499e-aef1-e43fd9b4bc81 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:58:43.119354 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0ca8ef38-2781-499e-aef1-e43fd9b4bc81 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:58:43.120795 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0ca8ef38-2781-499e-aef1-e43fd9b4bc81 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:58:43.121550 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0ca8ef38-2781-499e-aef1-e43fd9b4bc81 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:58:43.258131 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-0ca8ef38-2781-499e-aef1-e43fd9b4bc81 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '303ba697a1f94661b93254ac0f198670', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8849a8b6fc0648ed817530d7b57fe09a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:58:43.265126 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-0ca8ef38-2781-499e-aef1-e43fd9b4bc81 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] 10.222.0.16 "GET /compute/v2.1/servers/f6044bec-67bc-46bd-9681-db14474b2a5b" status: 200 len: 1515 microversion: 2.1 time: 0.158645 Jul 16 21:58:43.265705 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3164/6333] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:58:43 2022] GET /compute/v2.1/servers/f6044bec-67bc-46bd-9681-db14474b2a5b => generated 1515 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:58:44.282236 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-fc48e87e-37fd-437c-b02d-372c2e42a4c6 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:58:44.282860 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-fc48e87e-37fd-437c-b02d-372c2e42a4c6 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] [instance: f6044bec-67bc-46bd-9681-db14474b2a5b] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:58:44.289846 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fc48e87e-37fd-437c-b02d-372c2e42a4c6 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:58:44.290309 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fc48e87e-37fd-437c-b02d-372c2e42a4c6 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:58:44.291105 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fc48e87e-37fd-437c-b02d-372c2e42a4c6 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:58:44.291558 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fc48e87e-37fd-437c-b02d-372c2e42a4c6 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:58:44.422753 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-fc48e87e-37fd-437c-b02d-372c2e42a4c6 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '303ba697a1f94661b93254ac0f198670', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8849a8b6fc0648ed817530d7b57fe09a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:58:44.430318 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-fc48e87e-37fd-437c-b02d-372c2e42a4c6 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] 10.222.0.16 "GET /compute/v2.1/servers/f6044bec-67bc-46bd-9681-db14474b2a5b" status: 200 len: 1515 microversion: 2.1 time: 0.152391 Jul 16 21:58:44.430883 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3170/6334] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:58:44 2022] GET /compute/v2.1/servers/f6044bec-67bc-46bd-9681-db14474b2a5b => generated 1515 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:58:45.447952 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-8302d07e-37f1-42e8-988e-6cf3afaef35e tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:58:45.448517 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-8302d07e-37f1-42e8-988e-6cf3afaef35e tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] [instance: f6044bec-67bc-46bd-9681-db14474b2a5b] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:58:45.456249 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8302d07e-37f1-42e8-988e-6cf3afaef35e tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:58:45.456699 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8302d07e-37f1-42e8-988e-6cf3afaef35e tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:58:45.457488 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8302d07e-37f1-42e8-988e-6cf3afaef35e tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:58:45.457926 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8302d07e-37f1-42e8-988e-6cf3afaef35e tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:58:45.591909 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-8302d07e-37f1-42e8-988e-6cf3afaef35e tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '303ba697a1f94661b93254ac0f198670', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8849a8b6fc0648ed817530d7b57fe09a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:58:45.599602 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-8302d07e-37f1-42e8-988e-6cf3afaef35e tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] 10.222.0.16 "GET /compute/v2.1/servers/f6044bec-67bc-46bd-9681-db14474b2a5b" status: 200 len: 1515 microversion: 2.1 time: 0.155653 Jul 16 21:58:45.600174 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3165/6335] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:58:45 2022] GET /compute/v2.1/servers/f6044bec-67bc-46bd-9681-db14474b2a5b => generated 1515 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:58:46.616221 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-cdc24bce-05dd-455d-83bf-07034237f11a tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:58:46.617184 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-cdc24bce-05dd-455d-83bf-07034237f11a tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] [instance: f6044bec-67bc-46bd-9681-db14474b2a5b] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:58:46.624282 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cdc24bce-05dd-455d-83bf-07034237f11a tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:58:46.624759 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cdc24bce-05dd-455d-83bf-07034237f11a tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:58:46.625660 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cdc24bce-05dd-455d-83bf-07034237f11a tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:58:46.626084 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cdc24bce-05dd-455d-83bf-07034237f11a tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:58:46.871637 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-cdc24bce-05dd-455d-83bf-07034237f11a tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '303ba697a1f94661b93254ac0f198670', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8849a8b6fc0648ed817530d7b57fe09a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:58:46.879064 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-cdc24bce-05dd-455d-83bf-07034237f11a tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] 10.222.0.16 "GET /compute/v2.1/servers/f6044bec-67bc-46bd-9681-db14474b2a5b" status: 200 len: 1514 microversion: 2.1 time: 0.266797 Jul 16 21:58:46.879648 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3171/6336] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:58:46 2022] GET /compute/v2.1/servers/f6044bec-67bc-46bd-9681-db14474b2a5b => generated 1514 bytes in 268 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:58:47.222737 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-6fe2efbf-3fce-4403-a4b5-989306843c33 req-fe0473f8-124e-4349-9b3a-43d81f7dc64b service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "f6044bec-67bc-46bd-9681-db14474b2a5b", "name": "network-vif-unplugged", "status": "completed", "tag": "18d15a1a-e539-4ace-a5ee-61272d6388ff"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:58:47.230986 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-6fe2efbf-3fce-4403-a4b5-989306843c33 req-fe0473f8-124e-4349-9b3a-43d81f7dc64b service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:58:47.231473 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-6fe2efbf-3fce-4403-a4b5-989306843c33 req-fe0473f8-124e-4349-9b3a-43d81f7dc64b service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:58:47.242840 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-6fe2efbf-3fce-4403-a4b5-989306843c33 req-fe0473f8-124e-4349-9b3a-43d81f7dc64b service nova] Creating event network-vif-unplugged:18d15a1a-e539-4ace-a5ee-61272d6388ff for instance f6044bec-67bc-46bd-9681-db14474b2a5b on nch2-master-1 Jul 16 21:58:47.249045 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [req-6fe2efbf-3fce-4403-a4b5-989306843c33 req-fe0473f8-124e-4349-9b3a-43d81f7dc64b service nova] Instance f6044bec-67bc-46bd-9681-db14474b2a5b is migrating within the same cell, copying events to all relevant hosts: {'nch2-master-1', 'nch1-master-1'} {{(pid=138680) _get_relevant_hosts /opt/stack/nova/nova/compute/api.py:5807}} Jul 16 21:58:47.260069 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-6fe2efbf-3fce-4403-a4b5-989306843c33 req-fe0473f8-124e-4349-9b3a-43d81f7dc64b service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.104574 Jul 16 21:58:47.260687 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3166/6337] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:58:47 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 106 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:58:47.898400 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-8b3e0d7b-56f2-4f1a-8877-f8b7711480c9 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:58:47.899081 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-8b3e0d7b-56f2-4f1a-8877-f8b7711480c9 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] [instance: f6044bec-67bc-46bd-9681-db14474b2a5b] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:58:47.907463 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8b3e0d7b-56f2-4f1a-8877-f8b7711480c9 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:58:47.907864 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8b3e0d7b-56f2-4f1a-8877-f8b7711480c9 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:58:47.908838 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8b3e0d7b-56f2-4f1a-8877-f8b7711480c9 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:58:47.909287 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8b3e0d7b-56f2-4f1a-8877-f8b7711480c9 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:58:48.126116 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-8b3e0d7b-56f2-4f1a-8877-f8b7711480c9 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '303ba697a1f94661b93254ac0f198670', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8849a8b6fc0648ed817530d7b57fe09a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:58:48.133796 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-8b3e0d7b-56f2-4f1a-8877-f8b7711480c9 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] 10.222.0.16 "GET /compute/v2.1/servers/f6044bec-67bc-46bd-9681-db14474b2a5b" status: 200 len: 1514 microversion: 2.1 time: 0.240493 Jul 16 21:58:48.134637 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3172/6338] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:58:47 2022] GET /compute/v2.1/servers/f6044bec-67bc-46bd-9681-db14474b2a5b => generated 1514 bytes in 242 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:58:49.153299 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-1fad8222-ded0-4fe7-916f-b5ce3b9cce4c tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:58:49.153930 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-1fad8222-ded0-4fe7-916f-b5ce3b9cce4c tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] [instance: f6044bec-67bc-46bd-9681-db14474b2a5b] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:58:49.161469 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1fad8222-ded0-4fe7-916f-b5ce3b9cce4c tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:58:49.161926 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1fad8222-ded0-4fe7-916f-b5ce3b9cce4c tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:58:49.162688 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1fad8222-ded0-4fe7-916f-b5ce3b9cce4c tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:58:49.163167 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1fad8222-ded0-4fe7-916f-b5ce3b9cce4c tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:58:49.276546 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-78d7d2a0-21ea-4f07-a4b7-3ba2396ac535 req-2e497f80-bfb5-4a82-9367-20542070515f service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "f6044bec-67bc-46bd-9681-db14474b2a5b", "tag": "18d15a1a-e539-4ace-a5ee-61272d6388ff"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:58:49.285622 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-78d7d2a0-21ea-4f07-a4b7-3ba2396ac535 req-2e497f80-bfb5-4a82-9367-20542070515f service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:58:49.286080 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-78d7d2a0-21ea-4f07-a4b7-3ba2396ac535 req-2e497f80-bfb5-4a82-9367-20542070515f service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:58:49.298509 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-78d7d2a0-21ea-4f07-a4b7-3ba2396ac535 req-2e497f80-bfb5-4a82-9367-20542070515f service nova] Creating event network-changed:18d15a1a-e539-4ace-a5ee-61272d6388ff for instance f6044bec-67bc-46bd-9681-db14474b2a5b on nch2-master-1 Jul 16 21:58:49.303885 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [req-78d7d2a0-21ea-4f07-a4b7-3ba2396ac535 req-2e497f80-bfb5-4a82-9367-20542070515f service nova] Instance f6044bec-67bc-46bd-9681-db14474b2a5b is migrating within the same cell, copying events to all relevant hosts: {'nch2-master-1', 'nch1-master-1'} {{(pid=138681) _get_relevant_hosts /opt/stack/nova/nova/compute/api.py:5807}} Jul 16 21:58:49.305706 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-1fad8222-ded0-4fe7-916f-b5ce3b9cce4c tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '303ba697a1f94661b93254ac0f198670', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8849a8b6fc0648ed817530d7b57fe09a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:58:49.315452 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-78d7d2a0-21ea-4f07-a4b7-3ba2396ac535 req-2e497f80-bfb5-4a82-9367-20542070515f service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.042474 Jul 16 21:58:49.315646 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-1fad8222-ded0-4fe7-916f-b5ce3b9cce4c tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] 10.222.0.16 "GET /compute/v2.1/servers/f6044bec-67bc-46bd-9681-db14474b2a5b" status: 200 len: 1512 microversion: 2.1 time: 0.167592 Jul 16 21:58:49.316158 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3173/6339] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:58:49 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 16 21:58:49.316297 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3167/6340] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:58:49 2022] GET /compute/v2.1/servers/f6044bec-67bc-46bd-9681-db14474b2a5b => generated 1512 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:58:50.344216 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-37c6799a-f849-4bc5-a755-4109ca85162e tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:58:50.344954 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-37c6799a-f849-4bc5-a755-4109ca85162e tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] [instance: f6044bec-67bc-46bd-9681-db14474b2a5b] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:58:50.351786 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-37c6799a-f849-4bc5-a755-4109ca85162e tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:58:50.352242 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-37c6799a-f849-4bc5-a755-4109ca85162e tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:58:50.353014 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-37c6799a-f849-4bc5-a755-4109ca85162e tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:58:50.353449 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-37c6799a-f849-4bc5-a755-4109ca85162e tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:58:50.515494 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-37c6799a-f849-4bc5-a755-4109ca85162e tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '303ba697a1f94661b93254ac0f198670', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8849a8b6fc0648ed817530d7b57fe09a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:58:50.524021 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-37c6799a-f849-4bc5-a755-4109ca85162e tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] 10.222.0.16 "GET /compute/v2.1/servers/f6044bec-67bc-46bd-9681-db14474b2a5b" status: 200 len: 1512 microversion: 2.1 time: 0.184623 Jul 16 21:58:50.524609 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3168/6341] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:58:50 2022] GET /compute/v2.1/servers/f6044bec-67bc-46bd-9681-db14474b2a5b => generated 1512 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:58:51.544080 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-5d5991e1-67da-4586-a3c1-6299ed4192b6 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:58:51.544835 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-5d5991e1-67da-4586-a3c1-6299ed4192b6 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] [instance: f6044bec-67bc-46bd-9681-db14474b2a5b] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:58:51.551713 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5d5991e1-67da-4586-a3c1-6299ed4192b6 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:58:51.552177 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5d5991e1-67da-4586-a3c1-6299ed4192b6 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:58:51.552980 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5d5991e1-67da-4586-a3c1-6299ed4192b6 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:58:51.553428 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5d5991e1-67da-4586-a3c1-6299ed4192b6 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:58:51.695457 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-5d5991e1-67da-4586-a3c1-6299ed4192b6 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '303ba697a1f94661b93254ac0f198670', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8849a8b6fc0648ed817530d7b57fe09a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:58:51.703283 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-5d5991e1-67da-4586-a3c1-6299ed4192b6 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] 10.222.0.16 "GET /compute/v2.1/servers/f6044bec-67bc-46bd-9681-db14474b2a5b" status: 200 len: 1509 microversion: 2.1 time: 0.165336 Jul 16 21:58:51.703859 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3174/6342] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:58:51 2022] GET /compute/v2.1/servers/f6044bec-67bc-46bd-9681-db14474b2a5b => generated 1509 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:58:51.720111 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-9b2fc88c-2a9d-4cbf-a599-a384baba48f8 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:58:51.720651 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-9b2fc88c-2a9d-4cbf-a599-a384baba48f8 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] [instance: f6044bec-67bc-46bd-9681-db14474b2a5b] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:58:51.727602 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9b2fc88c-2a9d-4cbf-a599-a384baba48f8 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:58:51.728059 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9b2fc88c-2a9d-4cbf-a599-a384baba48f8 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:58:51.728826 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9b2fc88c-2a9d-4cbf-a599-a384baba48f8 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:58:51.729271 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9b2fc88c-2a9d-4cbf-a599-a384baba48f8 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:58:51.755770 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-9b2fc88c-2a9d-4cbf-a599-a384baba48f8 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] [instance: f6044bec-67bc-46bd-9681-db14474b2a5b] Going to try to terminate instance {{(pid=138680) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 16 21:58:51.881712 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.compute.api [None req-9b2fc88c-2a9d-4cbf-a599-a384baba48f8 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] [instance: f6044bec-67bc-46bd-9681-db14474b2a5b] Found an unconfirmed migration during delete, id: 16, status: finished Jul 16 21:58:52.326813 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-a454977f-8084-482a-8731-6ad01fd2a8ae req-985c907a-e557-462d-8ab7-c54bf483f751 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "f6044bec-67bc-46bd-9681-db14474b2a5b", "name": "network-vif-plugged", "status": "completed", "tag": "18d15a1a-e539-4ace-a5ee-61272d6388ff"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:58:52.336998 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-a454977f-8084-482a-8731-6ad01fd2a8ae req-985c907a-e557-462d-8ab7-c54bf483f751 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:58:52.339869 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-a454977f-8084-482a-8731-6ad01fd2a8ae req-985c907a-e557-462d-8ab7-c54bf483f751 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:58:52.350351 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-a454977f-8084-482a-8731-6ad01fd2a8ae req-985c907a-e557-462d-8ab7-c54bf483f751 service nova] Creating event network-vif-plugged:18d15a1a-e539-4ace-a5ee-61272d6388ff for instance f6044bec-67bc-46bd-9681-db14474b2a5b on nch2-master-1 Jul 16 21:58:52.356571 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [req-a454977f-8084-482a-8731-6ad01fd2a8ae req-985c907a-e557-462d-8ab7-c54bf483f751 service nova] Instance f6044bec-67bc-46bd-9681-db14474b2a5b is migrating within the same cell, copying events to all relevant hosts: {'nch2-master-1', 'nch1-master-1'} {{(pid=138681) _get_relevant_hosts /opt/stack/nova/nova/compute/api.py:5807}} Jul 16 21:58:52.368356 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-a454977f-8084-482a-8731-6ad01fd2a8ae req-985c907a-e557-462d-8ab7-c54bf483f751 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.046389 Jul 16 21:58:52.369101 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3175/6343] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:58:52 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 16 21:58:53.878369 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-9b2fc88c-2a9d-4cbf-a599-a384baba48f8 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] 10.222.0.16 "DELETE /compute/v2.1/servers/f6044bec-67bc-46bd-9681-db14474b2a5b" status: 204 len: 0 microversion: 2.1 time: 2.161786 Jul 16 21:58:53.881426 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3169/6344] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:58:51 2022] DELETE /compute/v2.1/servers/f6044bec-67bc-46bd-9681-db14474b2a5b => generated 0 bytes in 2163 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 16 21:58:53.889467 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-b46bb2ac-0d8e-4384-a4c6-9b316e1b4a37 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:58:53.890187 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-b46bb2ac-0d8e-4384-a4c6-9b316e1b4a37 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] [instance: f6044bec-67bc-46bd-9681-db14474b2a5b] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:58:53.896485 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b46bb2ac-0d8e-4384-a4c6-9b316e1b4a37 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:58:53.898746 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b46bb2ac-0d8e-4384-a4c6-9b316e1b4a37 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:58:53.898746 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b46bb2ac-0d8e-4384-a4c6-9b316e1b4a37 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:58:53.898746 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b46bb2ac-0d8e-4384-a4c6-9b316e1b4a37 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:58:54.033735 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-b46bb2ac-0d8e-4384-a4c6-9b316e1b4a37 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '303ba697a1f94661b93254ac0f198670', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8849a8b6fc0648ed817530d7b57fe09a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:58:54.041331 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-b46bb2ac-0d8e-4384-a4c6-9b316e1b4a37 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] 10.222.0.16 "GET /compute/v2.1/servers/f6044bec-67bc-46bd-9681-db14474b2a5b" status: 200 len: 1501 microversion: 2.1 time: 0.157677 Jul 16 21:58:54.041904 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3176/6345] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:58:53 2022] GET /compute/v2.1/servers/f6044bec-67bc-46bd-9681-db14474b2a5b => generated 1501 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:58:55.059329 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-8e01b3ad-e139-455a-9043-0746044c49e2 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:58:55.060030 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-8e01b3ad-e139-455a-9043-0746044c49e2 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] [instance: f6044bec-67bc-46bd-9681-db14474b2a5b] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:58:55.067523 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8e01b3ad-e139-455a-9043-0746044c49e2 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:58:55.067978 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8e01b3ad-e139-455a-9043-0746044c49e2 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:58:55.068743 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8e01b3ad-e139-455a-9043-0746044c49e2 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:58:55.069183 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8e01b3ad-e139-455a-9043-0746044c49e2 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:58:55.202102 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-8e01b3ad-e139-455a-9043-0746044c49e2 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '303ba697a1f94661b93254ac0f198670', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8849a8b6fc0648ed817530d7b57fe09a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:58:55.209832 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-8e01b3ad-e139-455a-9043-0746044c49e2 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] 10.222.0.16 "GET /compute/v2.1/servers/f6044bec-67bc-46bd-9681-db14474b2a5b" status: 200 len: 1501 microversion: 2.1 time: 0.154836 Jul 16 21:58:55.210406 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3170/6346] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:58:55 2022] GET /compute/v2.1/servers/f6044bec-67bc-46bd-9681-db14474b2a5b => generated 1501 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:58:55.899311 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-18b18dc7-759a-465b-a39e-4e299a25fa65 req-55f500ed-00d6-431b-aedb-d7ecfc108f31 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "f6044bec-67bc-46bd-9681-db14474b2a5b", "name": "network-vif-deleted", "tag": "18d15a1a-e539-4ace-a5ee-61272d6388ff"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:58:55.909344 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-18b18dc7-759a-465b-a39e-4e299a25fa65 req-55f500ed-00d6-431b-aedb-d7ecfc108f31 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:58:55.909807 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-18b18dc7-759a-465b-a39e-4e299a25fa65 req-55f500ed-00d6-431b-aedb-d7ecfc108f31 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:58:55.921776 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-18b18dc7-759a-465b-a39e-4e299a25fa65 req-55f500ed-00d6-431b-aedb-d7ecfc108f31 service nova] Creating event network-vif-deleted:18d15a1a-e539-4ace-a5ee-61272d6388ff for instance f6044bec-67bc-46bd-9681-db14474b2a5b on nch2-master-1 Jul 16 21:58:55.927061 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-18b18dc7-759a-465b-a39e-4e299a25fa65 req-55f500ed-00d6-431b-aedb-d7ecfc108f31 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.031603 Jul 16 21:58:55.927693 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3177/6347] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:58:55 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:58:56.227902 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-437b7abc-8dd0-4ada-847e-bf76a23d46e8 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:58:56.233717 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-437b7abc-8dd0-4ada-847e-bf76a23d46e8 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] [instance: f6044bec-67bc-46bd-9681-db14474b2a5b] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:58:56.236767 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-437b7abc-8dd0-4ada-847e-bf76a23d46e8 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:58:56.237226 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-437b7abc-8dd0-4ada-847e-bf76a23d46e8 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:58:56.238006 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-437b7abc-8dd0-4ada-847e-bf76a23d46e8 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:58:56.238453 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-437b7abc-8dd0-4ada-847e-bf76a23d46e8 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:58:56.273953 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-437b7abc-8dd0-4ada-847e-bf76a23d46e8 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:58:56.274493 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-437b7abc-8dd0-4ada-847e-bf76a23d46e8 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:58:56.275099 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.objects.instance [None req-437b7abc-8dd0-4ada-847e-bf76a23d46e8 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lazy-loading 'fault' on Instance uuid f6044bec-67bc-46bd-9681-db14474b2a5b {{(pid=138680) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 21:58:56.320108 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-437b7abc-8dd0-4ada-847e-bf76a23d46e8 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '303ba697a1f94661b93254ac0f198670', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8849a8b6fc0648ed817530d7b57fe09a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:58:56.327303 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-437b7abc-8dd0-4ada-847e-bf76a23d46e8 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] 10.222.0.16 "GET /compute/v2.1/servers/f6044bec-67bc-46bd-9681-db14474b2a5b" status: 200 len: 1304 microversion: 2.1 time: 0.103661 Jul 16 21:58:56.328001 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3171/6348] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:58:56 2022] GET /compute/v2.1/servers/f6044bec-67bc-46bd-9681-db14474b2a5b => generated 1304 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:58:57.346499 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-fd65ae5a-ff4c-4f2a-9f9f-6a8e2f089e32 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:58:57.347245 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-fd65ae5a-ff4c-4f2a-9f9f-6a8e2f089e32 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] [instance: f6044bec-67bc-46bd-9681-db14474b2a5b] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:58:57.353755 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fd65ae5a-ff4c-4f2a-9f9f-6a8e2f089e32 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:58:57.354216 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fd65ae5a-ff4c-4f2a-9f9f-6a8e2f089e32 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:58:57.355751 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fd65ae5a-ff4c-4f2a-9f9f-6a8e2f089e32 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:58:57.356310 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fd65ae5a-ff4c-4f2a-9f9f-6a8e2f089e32 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:58:57.369610 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-fd65ae5a-ff4c-4f2a-9f9f-6a8e2f089e32 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] HTTP exception thrown: Instance f6044bec-67bc-46bd-9681-db14474b2a5b could not be found. Jul 16 21:58:57.372094 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-fd65ae5a-ff4c-4f2a-9f9f-6a8e2f089e32 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Request method failure captured: Jul 16 21:58:57.372094 ncdv-master-1 devstack@n-api.service[138681]: request: GET /compute/v2.1/servers/f6044bec-67bc-46bd-9681-db14474b2a5b HTTP/1.1 Jul 16 21:58:57.372094 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:58:57.372094 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:58:57.372094 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:58:57.372094 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 21:58:57.372094 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:58:57.372094 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:58:57.372094 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:58:57.372094 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:58:57.372094 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0zRqNHpWi2qbLNehxkHMzv6ps_8BiTf-fJHKiCIU70FS-pZVzEYPOfiKu5LUPTJ0uK3DbYBBJW_Z3zpLOZkyu2P9ZgKbmK2bYOpEzmACtBvfqV8ciQLwrez7vUmHra3OgerxwkO3OuA7mujM9K34d5skY083XhRKpf_jEW06xMaJKN4 Jul 16 21:58:57.372094 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:58:57.372094 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:58:57.372094 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:58:57.372094 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:58:57.372094 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:58:57.372094 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:58:57.372094 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: 8849a8b6fc0648ed817530d7b57fe09a Jul 16 21:58:57.372094 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-DeleteServersTestJSON-1493471914 Jul 16 21:58:57.372094 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:58:57.372094 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:58:57.372094 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/8849a8b6fc0648ed817530d7b57fe09a"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/8849a8b6fc0648ed817530d7b57fe09a"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_8849a8b6fc0648ed817530d7b57fe09a"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/8849a8b6fc0648ed817530d7b57fe09a"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:58:57.372094 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-DeleteServersTestJSON-1493471914 Jul 16 21:58:57.372094 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: 8849a8b6fc0648ed817530d7b57fe09a Jul 16 21:58:57.372094 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-DeleteServersTestJSON-1493471914 Jul 16 21:58:57.372094 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-DeleteServersTestJSON-1493471914-project Jul 16 21:58:57.372094 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:58:57.372094 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:58:57.372094 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: 303ba697a1f94661b93254ac0f198670 Jul 16 21:58:57.372094 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-DeleteServersTestJSON-1493471914-project Jul 16 21:58:57.372094 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:58:57.372094 ncdv-master-1 devstack@n-api.service[138681]: exception: Instance f6044bec-67bc-46bd-9681-db14474b2a5b could not be found. Jul 16 21:58:57.372094 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': 'f6044bec-67bc-46bd-9681-db14474b2a5b'} Jul 16 21:58:57.372094 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:58:57.372094 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:58:57.372094 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:58:57.372094 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:58:57.372094 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:58:57.372094 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:58:57.372094 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:58:57.372094 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:58:57.372094 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:58:57.372094 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:58:57.372094 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance f6044bec-67bc-46bd-9681-db14474b2a5b could not be found. Jul 16 21:58:57.372094 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:58:57.372094 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:58:57.372094 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:58:57.372094 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:58:57.372094 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:58:57.372094 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:58:57.372094 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:58:57.372094 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:58:57.372094 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:58:57.372094 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:58:57.372094 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:58:57.372094 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:58:57.372094 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:58:57.372094 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:58:57.372094 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:58:57.372094 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:58:57.372094 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance f6044bec-67bc-46bd-9681-db14474b2a5b could not be found. Jul 16 21:58:57.372094 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:58:57.372094 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:58:57.372094 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:58:57.377164 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:58:57.377164 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:58:57.377164 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:58:57.377164 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:58:57.377164 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:58:57.377164 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance f6044bec-67bc-46bd-9681-db14474b2a5b could not be found. Jul 16 21:58:57.377164 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:58:57.377164 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-fd65ae5a-ff4c-4f2a-9f9f-6a8e2f089e32 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Returning 404 to user: Instance f6044bec-67bc-46bd-9681-db14474b2a5b could not be found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:58:57.377164 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-fd65ae5a-ff4c-4f2a-9f9f-6a8e2f089e32 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] 10.222.0.16 "GET /compute/v2.1/servers/f6044bec-67bc-46bd-9681-db14474b2a5b" status: 404 len: 111 microversion: 2.1 time: 0.032156 Jul 16 21:58:57.377164 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3178/6349] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:58:57 2022] GET /compute/v2.1/servers/f6044bec-67bc-46bd-9681-db14474b2a5b => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:58:57.388998 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-985307ee-7fe8-40c8-b9c4-4a08a7202520 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:58:57.389508 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-985307ee-7fe8-40c8-b9c4-4a08a7202520 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] [instance: f6044bec-67bc-46bd-9681-db14474b2a5b] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:58:57.396443 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-985307ee-7fe8-40c8-b9c4-4a08a7202520 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:58:57.396897 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-985307ee-7fe8-40c8-b9c4-4a08a7202520 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:58:57.397651 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-985307ee-7fe8-40c8-b9c4-4a08a7202520 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:58:57.398090 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-985307ee-7fe8-40c8-b9c4-4a08a7202520 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:58:57.411799 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-985307ee-7fe8-40c8-b9c4-4a08a7202520 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] HTTP exception thrown: Instance f6044bec-67bc-46bd-9681-db14474b2a5b could not be found. Jul 16 21:58:57.413441 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-985307ee-7fe8-40c8-b9c4-4a08a7202520 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Request method failure captured: Jul 16 21:58:57.413441 ncdv-master-1 devstack@n-api.service[138680]: request: DELETE /compute/v2.1/servers/f6044bec-67bc-46bd-9681-db14474b2a5b HTTP/1.1 Jul 16 21:58:57.413441 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 21:58:57.413441 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 21:58:57.413441 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 21:58:57.413441 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 0 Jul 16 21:58:57.413441 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 21:58:57.413441 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 21:58:57.413441 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 21:58:57.413441 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 21:58:57.413441 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0zRqNHpWi2qbLNehxkHMzv6ps_8BiTf-fJHKiCIU70FS-pZVzEYPOfiKu5LUPTJ0uK3DbYBBJW_Z3zpLOZkyu2P9ZgKbmK2bYOpEzmACtBvfqV8ciQLwrez7vUmHra3OgerxwkO3OuA7mujM9K34d5skY083XhRKpf_jEW06xMaJKN4 Jul 16 21:58:57.413441 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 21:58:57.413441 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 21:58:57.413441 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 21:58:57.413441 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 21:58:57.413441 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 21:58:57.413441 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 21:58:57.413441 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: 8849a8b6fc0648ed817530d7b57fe09a Jul 16 21:58:57.413441 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-DeleteServersTestJSON-1493471914 Jul 16 21:58:57.413441 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 21:58:57.413441 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 21:58:57.413441 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/8849a8b6fc0648ed817530d7b57fe09a"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/8849a8b6fc0648ed817530d7b57fe09a"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_8849a8b6fc0648ed817530d7b57fe09a"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/8849a8b6fc0648ed817530d7b57fe09a"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:58:57.413441 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-DeleteServersTestJSON-1493471914 Jul 16 21:58:57.413441 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: 8849a8b6fc0648ed817530d7b57fe09a Jul 16 21:58:57.413441 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-DeleteServersTestJSON-1493471914 Jul 16 21:58:57.413441 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-DeleteServersTestJSON-1493471914-project Jul 16 21:58:57.413441 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 21:58:57.413441 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 21:58:57.413441 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: 303ba697a1f94661b93254ac0f198670 Jul 16 21:58:57.413441 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-DeleteServersTestJSON-1493471914-project Jul 16 21:58:57.413441 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 21:58:57.413441 ncdv-master-1 devstack@n-api.service[138680]: exception: Instance f6044bec-67bc-46bd-9681-db14474b2a5b could not be found. Jul 16 21:58:57.413441 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': 'f6044bec-67bc-46bd-9681-db14474b2a5b'} Jul 16 21:58:57.413441 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:58:57.413441 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:58:57.413441 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:58:57.413441 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:58:57.413441 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:58:57.413441 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:58:57.413441 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:58:57.413441 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:58:57.413441 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:58:57.413441 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:58:57.413441 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance f6044bec-67bc-46bd-9681-db14474b2a5b could not be found. Jul 16 21:58:57.413441 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:58:57.413441 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:58:57.413441 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:58:57.413441 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:58:57.413441 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:58:57.413441 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:58:57.413441 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:58:57.413441 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:58:57.413441 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:58:57.413441 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:58:57.413441 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Jul 16 21:58:57.413441 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Jul 16 21:58:57.413441 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Jul 16 21:58:57.413441 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Jul 16 21:58:57.413441 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:58:57.413441 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:58:57.413441 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:58:57.413441 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:58:57.418758 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance f6044bec-67bc-46bd-9681-db14474b2a5b could not be found. Jul 16 21:58:57.418758 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:58:57.418758 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:58:57.418758 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:58:57.418758 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:58:57.418758 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:58:57.418758 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:58:57.418758 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:58:57.418758 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:58:57.418758 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance f6044bec-67bc-46bd-9681-db14474b2a5b could not be found. Jul 16 21:58:57.418758 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 21:58:57.418758 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-985307ee-7fe8-40c8-b9c4-4a08a7202520 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Returning 404 to user: Instance f6044bec-67bc-46bd-9681-db14474b2a5b could not be found. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:58:57.418758 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-985307ee-7fe8-40c8-b9c4-4a08a7202520 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] 10.222.0.16 "DELETE /compute/v2.1/servers/f6044bec-67bc-46bd-9681-db14474b2a5b" status: 404 len: 111 microversion: 2.1 time: 0.029866 Jul 16 21:58:57.418758 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3172/6350] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 21:58:57 2022] DELETE /compute/v2.1/servers/f6044bec-67bc-46bd-9681-db14474b2a5b => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:58:57.423933 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-3f804236-ae59-4956-b749-a2feead30a43 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:58:57.424561 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-3f804236-ae59-4956-b749-a2feead30a43 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] [instance: f6044bec-67bc-46bd-9681-db14474b2a5b] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:58:57.430570 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3f804236-ae59-4956-b749-a2feead30a43 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:58:57.431051 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3f804236-ae59-4956-b749-a2feead30a43 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:58:57.431818 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3f804236-ae59-4956-b749-a2feead30a43 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:58:57.432261 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3f804236-ae59-4956-b749-a2feead30a43 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:58:57.445452 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-3f804236-ae59-4956-b749-a2feead30a43 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] HTTP exception thrown: Instance f6044bec-67bc-46bd-9681-db14474b2a5b could not be found. Jul 16 21:58:57.446934 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-3f804236-ae59-4956-b749-a2feead30a43 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Request method failure captured: Jul 16 21:58:57.446934 ncdv-master-1 devstack@n-api.service[138681]: request: GET /compute/v2.1/servers/f6044bec-67bc-46bd-9681-db14474b2a5b HTTP/1.1 Jul 16 21:58:57.446934 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 21:58:57.446934 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 21:58:57.446934 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 21:58:57.446934 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 21:58:57.446934 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 21:58:57.446934 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 21:58:57.446934 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 21:58:57.446934 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 21:58:57.446934 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0zRqNHpWi2qbLNehxkHMzv6ps_8BiTf-fJHKiCIU70FS-pZVzEYPOfiKu5LUPTJ0uK3DbYBBJW_Z3zpLOZkyu2P9ZgKbmK2bYOpEzmACtBvfqV8ciQLwrez7vUmHra3OgerxwkO3OuA7mujM9K34d5skY083XhRKpf_jEW06xMaJKN4 Jul 16 21:58:57.446934 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 21:58:57.446934 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 21:58:57.446934 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 21:58:57.446934 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 21:58:57.446934 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 21:58:57.446934 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 21:58:57.446934 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: 8849a8b6fc0648ed817530d7b57fe09a Jul 16 21:58:57.446934 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-DeleteServersTestJSON-1493471914 Jul 16 21:58:57.446934 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 21:58:57.446934 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 21:58:57.446934 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/8849a8b6fc0648ed817530d7b57fe09a"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/8849a8b6fc0648ed817530d7b57fe09a"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_8849a8b6fc0648ed817530d7b57fe09a"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/8849a8b6fc0648ed817530d7b57fe09a"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 21:58:57.446934 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-DeleteServersTestJSON-1493471914 Jul 16 21:58:57.446934 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: 8849a8b6fc0648ed817530d7b57fe09a Jul 16 21:58:57.446934 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-DeleteServersTestJSON-1493471914 Jul 16 21:58:57.446934 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-DeleteServersTestJSON-1493471914-project Jul 16 21:58:57.446934 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 21:58:57.446934 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 21:58:57.446934 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: 303ba697a1f94661b93254ac0f198670 Jul 16 21:58:57.446934 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-DeleteServersTestJSON-1493471914-project Jul 16 21:58:57.446934 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 21:58:57.446934 ncdv-master-1 devstack@n-api.service[138681]: exception: Instance f6044bec-67bc-46bd-9681-db14474b2a5b could not be found. Jul 16 21:58:57.446934 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': 'f6044bec-67bc-46bd-9681-db14474b2a5b'} Jul 16 21:58:57.446934 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 21:58:57.446934 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:58:57.446934 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 21:58:57.446934 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 21:58:57.446934 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 21:58:57.446934 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 21:58:57.446934 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 21:58:57.446934 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 21:58:57.446934 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 21:58:57.446934 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 21:58:57.446934 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance f6044bec-67bc-46bd-9681-db14474b2a5b could not be found. Jul 16 21:58:57.446934 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:58:57.446934 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:58:57.446934 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:58:57.446934 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:58:57.446934 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:58:57.446934 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:58:57.446934 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 21:58:57.446934 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 21:58:57.446934 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 21:58:57.446934 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 21:58:57.446934 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 21:58:57.446934 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 21:58:57.446934 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 21:58:57.446934 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 21:58:57.446934 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 21:58:57.446934 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 21:58:57.446934 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance f6044bec-67bc-46bd-9681-db14474b2a5b could not be found. Jul 16 21:58:57.446934 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:58:57.446934 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 21:58:57.446934 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:58:57.451665 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 21:58:57.451665 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 21:58:57.451665 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 21:58:57.451665 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 21:58:57.451665 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 21:58:57.451665 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance f6044bec-67bc-46bd-9681-db14474b2a5b could not be found. Jul 16 21:58:57.451665 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 21:58:57.451665 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-3f804236-ae59-4956-b749-a2feead30a43 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] Returning 404 to user: Instance f6044bec-67bc-46bd-9681-db14474b2a5b could not be found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 21:58:57.451665 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-3f804236-ae59-4956-b749-a2feead30a43 tempest-DeleteServersTestJSON-1493471914 tempest-DeleteServersTestJSON-1493471914-project] 10.222.0.16 "GET /compute/v2.1/servers/f6044bec-67bc-46bd-9681-db14474b2a5b" status: 404 len: 111 microversion: 2.1 time: 0.028198 Jul 16 21:58:57.451665 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3179/6351] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:58:57 2022] GET /compute/v2.1/servers/f6044bec-67bc-46bd-9681-db14474b2a5b => generated 111 bytes in 29 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 21:59:08.147516 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_api:SELECT=26,nova_api:UPDATE=1 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:59:08.148443 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=62,nova_cell1:UPDATE=1,nova_cell1:INSERT=2 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:59:08.279940 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-85908462-8228-4531-8e8d-951b3791281b tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerDiskConfigTestJSON-server-1545025904", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452", "networks": [{"uuid": "9b7d6baa-385f-4a11-9e81-1d39ce412e76"}]}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:59:08.344276 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.network.neutron [None req-85908462-8228-4531-8e8d-951b3791281b tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] validate_networks() for [('9b7d6baa-385f-4a11-9e81-1d39ce412e76', None, None, None, None, None)] {{(pid=138680) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 16 21:59:08.721833 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-85908462-8228-4531-8e8d-951b3791281b tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Flavor limits 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 16 21:59:08.722267 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-85908462-8228-4531-8e8d-951b3791281b tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Image limits 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 16 21:59:08.722786 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-85908462-8228-4531-8e8d-951b3791281b tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Flavor pref 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 16 21:59:08.723252 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-85908462-8228-4531-8e8d-951b3791281b tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Image pref 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 16 21:59:08.723672 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-85908462-8228-4531-8e8d-951b3791281b tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 16 21:59:08.732990 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-85908462-8228-4531-8e8d-951b3791281b tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:59:08.733474 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-85908462-8228-4531-8e8d-951b3791281b tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:59:08.734278 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-85908462-8228-4531-8e8d-951b3791281b tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:59:08.734724 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-85908462-8228-4531-8e8d-951b3791281b tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:59:08.752491 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-85908462-8228-4531-8e8d-951b3791281b tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Getting quotas for project 9934bd813edb4e55a2d7c61207a9c46d. Resources: {'ram', 'instances', 'cores'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 21:59:08.757798 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-85908462-8228-4531-8e8d-951b3791281b tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Getting quotas for user d980641608134eadb72afa6fddb6adeb and project 9934bd813edb4e55a2d7c61207a9c46d. Resources: {'ram', 'instances', 'cores'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 21:59:08.770840 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-85908462-8228-4531-8e8d-951b3791281b tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Going to run 1 instances... {{(pid=138680) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 16 21:59:08.773159 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-85908462-8228-4531-8e8d-951b3791281b tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] [instance: 853637be-2161-4b9e-903a-2cbce6ff6881] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138680) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 16 21:59:08.886140 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-85908462-8228-4531-8e8d-951b3791281b tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.744345 Jul 16 21:59:08.892332 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3173/6352] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 21:59:08 2022] POST /compute/v2.1/servers => generated 380 bytes in 746 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 16 21:59:08.897721 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-52f76a78-4fcb-4e55-8798-3f7c4dd694f1 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:59:08.898358 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-52f76a78-4fcb-4e55-8798-3f7c4dd694f1 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] [instance: 853637be-2161-4b9e-903a-2cbce6ff6881] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:59:08.902037 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_api:SELECT=21 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:59:08.903713 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=55 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:59:08.960759 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-52f76a78-4fcb-4e55-8798-3f7c4dd694f1 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd980641608134eadb72afa6fddb6adeb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9934bd813edb4e55a2d7c61207a9c46d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:59:08.970578 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-52f76a78-4fcb-4e55-8798-3f7c4dd694f1 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] 10.222.0.16 "GET /compute/v2.1/servers/853637be-2161-4b9e-903a-2cbce6ff6881" status: 200 len: 1218 microversion: 2.1 time: 0.076634 Jul 16 21:59:08.971660 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3180/6353] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:59:08 2022] GET /compute/v2.1/servers/853637be-2161-4b9e-903a-2cbce6ff6881 => generated 1218 bytes in 78 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:59:09.989794 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-d6673e07-afbf-4eaa-8cc4-98d232efb120 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:59:09.990373 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-d6673e07-afbf-4eaa-8cc4-98d232efb120 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] [instance: 853637be-2161-4b9e-903a-2cbce6ff6881] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:59:09.999052 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d6673e07-afbf-4eaa-8cc4-98d232efb120 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:59:09.999524 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d6673e07-afbf-4eaa-8cc4-98d232efb120 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:59:10.000319 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d6673e07-afbf-4eaa-8cc4-98d232efb120 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:59:10.000761 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d6673e07-afbf-4eaa-8cc4-98d232efb120 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:59:10.078872 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-d6673e07-afbf-4eaa-8cc4-98d232efb120 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd980641608134eadb72afa6fddb6adeb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9934bd813edb4e55a2d7c61207a9c46d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:59:10.087505 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-d6673e07-afbf-4eaa-8cc4-98d232efb120 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] 10.222.0.16 "GET /compute/v2.1/servers/853637be-2161-4b9e-903a-2cbce6ff6881" status: 200 len: 1222 microversion: 2.1 time: 0.103004 Jul 16 21:59:10.089017 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3174/6354] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:59:09 2022] GET /compute/v2.1/servers/853637be-2161-4b9e-903a-2cbce6ff6881 => generated 1222 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:59:11.110667 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-618cd36e-1f7c-4edd-bf7f-3fb5584646d9 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:59:11.111425 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-618cd36e-1f7c-4edd-bf7f-3fb5584646d9 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] [instance: 853637be-2161-4b9e-903a-2cbce6ff6881] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:59:11.140538 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-618cd36e-1f7c-4edd-bf7f-3fb5584646d9 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:59:11.141287 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-618cd36e-1f7c-4edd-bf7f-3fb5584646d9 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:59:11.142331 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-618cd36e-1f7c-4edd-bf7f-3fb5584646d9 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:59:11.142952 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-618cd36e-1f7c-4edd-bf7f-3fb5584646d9 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:59:11.226507 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-618cd36e-1f7c-4edd-bf7f-3fb5584646d9 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd980641608134eadb72afa6fddb6adeb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9934bd813edb4e55a2d7c61207a9c46d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:59:11.236160 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-618cd36e-1f7c-4edd-bf7f-3fb5584646d9 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] 10.222.0.16 "GET /compute/v2.1/servers/853637be-2161-4b9e-903a-2cbce6ff6881" status: 200 len: 1278 microversion: 2.1 time: 0.130016 Jul 16 21:59:11.236781 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3181/6355] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:59:11 2022] GET /compute/v2.1/servers/853637be-2161-4b9e-903a-2cbce6ff6881 => generated 1278 bytes in 132 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:59:12.254114 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-ae0b44c2-478e-49f8-8370-61fa3598d40a tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:59:12.254959 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-ae0b44c2-478e-49f8-8370-61fa3598d40a tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] [instance: 853637be-2161-4b9e-903a-2cbce6ff6881] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:59:12.262922 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ae0b44c2-478e-49f8-8370-61fa3598d40a tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:59:12.263423 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ae0b44c2-478e-49f8-8370-61fa3598d40a tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:59:12.264208 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ae0b44c2-478e-49f8-8370-61fa3598d40a tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:59:12.264655 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ae0b44c2-478e-49f8-8370-61fa3598d40a tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:59:12.482266 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-ae0b44c2-478e-49f8-8370-61fa3598d40a tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd980641608134eadb72afa6fddb6adeb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9934bd813edb4e55a2d7c61207a9c46d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:59:12.489864 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-ae0b44c2-478e-49f8-8370-61fa3598d40a tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] 10.222.0.16 "GET /compute/v2.1/servers/853637be-2161-4b9e-903a-2cbce6ff6881" status: 200 len: 1318 microversion: 2.1 time: 0.240426 Jul 16 21:59:12.490454 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3175/6356] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:59:12 2022] GET /compute/v2.1/servers/853637be-2161-4b9e-903a-2cbce6ff6881 => generated 1318 bytes in 242 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:59:12.936323 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-7d79867f-543d-4fd9-8c74-cedb1bcff982 req-2c9efa8d-8a6c-4c89-967f-2026f96d4084 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "853637be-2161-4b9e-903a-2cbce6ff6881", "tag": "341e460d-4246-420d-8082-44048820ac75"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:59:12.945822 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-7d79867f-543d-4fd9-8c74-cedb1bcff982 req-2c9efa8d-8a6c-4c89-967f-2026f96d4084 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:59:12.951557 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-7d79867f-543d-4fd9-8c74-cedb1bcff982 req-2c9efa8d-8a6c-4c89-967f-2026f96d4084 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:59:12.961231 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-7d79867f-543d-4fd9-8c74-cedb1bcff982 req-2c9efa8d-8a6c-4c89-967f-2026f96d4084 service nova] Creating event network-changed:341e460d-4246-420d-8082-44048820ac75 for instance 853637be-2161-4b9e-903a-2cbce6ff6881 on nch2-master-1 Jul 16 21:59:12.967319 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-7d79867f-543d-4fd9-8c74-cedb1bcff982 req-2c9efa8d-8a6c-4c89-967f-2026f96d4084 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.036470 Jul 16 21:59:12.968199 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3182/6357] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:59:12 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:59:13.507915 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-bd28c437-e34d-4b6b-98c3-cc9a2595dc84 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:59:13.508554 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-bd28c437-e34d-4b6b-98c3-cc9a2595dc84 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] [instance: 853637be-2161-4b9e-903a-2cbce6ff6881] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:59:13.516467 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-bd28c437-e34d-4b6b-98c3-cc9a2595dc84 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:59:13.516864 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-bd28c437-e34d-4b6b-98c3-cc9a2595dc84 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:59:13.517647 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-bd28c437-e34d-4b6b-98c3-cc9a2595dc84 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:59:13.518093 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-bd28c437-e34d-4b6b-98c3-cc9a2595dc84 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:59:13.748533 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-bd28c437-e34d-4b6b-98c3-cc9a2595dc84 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd980641608134eadb72afa6fddb6adeb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9934bd813edb4e55a2d7c61207a9c46d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:59:13.756130 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-bd28c437-e34d-4b6b-98c3-cc9a2595dc84 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] 10.222.0.16 "GET /compute/v2.1/servers/853637be-2161-4b9e-903a-2cbce6ff6881" status: 200 len: 1318 microversion: 2.1 time: 0.252200 Jul 16 21:59:13.756709 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3176/6358] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:59:13 2022] GET /compute/v2.1/servers/853637be-2161-4b9e-903a-2cbce6ff6881 => generated 1318 bytes in 253 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:59:14.774229 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-7590421d-e87a-4bdd-86c5-c3c312086c07 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:59:14.774952 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-7590421d-e87a-4bdd-86c5-c3c312086c07 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] [instance: 853637be-2161-4b9e-903a-2cbce6ff6881] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:59:14.782621 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7590421d-e87a-4bdd-86c5-c3c312086c07 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:59:14.783045 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7590421d-e87a-4bdd-86c5-c3c312086c07 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:59:14.783830 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7590421d-e87a-4bdd-86c5-c3c312086c07 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:59:14.784279 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7590421d-e87a-4bdd-86c5-c3c312086c07 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:59:14.915886 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-7590421d-e87a-4bdd-86c5-c3c312086c07 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd980641608134eadb72afa6fddb6adeb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9934bd813edb4e55a2d7c61207a9c46d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:59:14.924480 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-7590421d-e87a-4bdd-86c5-c3c312086c07 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] 10.222.0.16 "GET /compute/v2.1/servers/853637be-2161-4b9e-903a-2cbce6ff6881" status: 200 len: 1318 microversion: 2.1 time: 0.154526 Jul 16 21:59:14.925058 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3183/6359] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:59:14 2022] GET /compute/v2.1/servers/853637be-2161-4b9e-903a-2cbce6ff6881 => generated 1318 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:59:15.941700 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-94c508dd-85fa-44b8-8b87-ebd1b3e32f66 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:59:15.942325 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-94c508dd-85fa-44b8-8b87-ebd1b3e32f66 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] [instance: 853637be-2161-4b9e-903a-2cbce6ff6881] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:59:15.949473 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-94c508dd-85fa-44b8-8b87-ebd1b3e32f66 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:59:15.954750 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-94c508dd-85fa-44b8-8b87-ebd1b3e32f66 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:59:15.954750 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-94c508dd-85fa-44b8-8b87-ebd1b3e32f66 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:59:15.954750 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-94c508dd-85fa-44b8-8b87-ebd1b3e32f66 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:59:16.417007 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-94c508dd-85fa-44b8-8b87-ebd1b3e32f66 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd980641608134eadb72afa6fddb6adeb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9934bd813edb4e55a2d7c61207a9c46d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:59:16.426300 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-94c508dd-85fa-44b8-8b87-ebd1b3e32f66 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] 10.222.0.16 "GET /compute/v2.1/servers/853637be-2161-4b9e-903a-2cbce6ff6881" status: 200 len: 1318 microversion: 2.1 time: 0.488536 Jul 16 21:59:16.426872 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3177/6360] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:59:15 2022] GET /compute/v2.1/servers/853637be-2161-4b9e-903a-2cbce6ff6881 => generated 1318 bytes in 490 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:59:17.443345 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-9ca97ded-3b50-480b-9f98-cc2bcf186e95 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:59:17.443887 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-9ca97ded-3b50-480b-9f98-cc2bcf186e95 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] [instance: 853637be-2161-4b9e-903a-2cbce6ff6881] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:59:17.450903 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9ca97ded-3b50-480b-9f98-cc2bcf186e95 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:59:17.451389 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9ca97ded-3b50-480b-9f98-cc2bcf186e95 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:59:17.452155 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9ca97ded-3b50-480b-9f98-cc2bcf186e95 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:59:17.452603 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9ca97ded-3b50-480b-9f98-cc2bcf186e95 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:59:17.581908 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-9ca97ded-3b50-480b-9f98-cc2bcf186e95 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd980641608134eadb72afa6fddb6adeb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9934bd813edb4e55a2d7c61207a9c46d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:59:17.589732 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-9ca97ded-3b50-480b-9f98-cc2bcf186e95 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] 10.222.0.16 "GET /compute/v2.1/servers/853637be-2161-4b9e-903a-2cbce6ff6881" status: 200 len: 1318 microversion: 2.1 time: 0.150381 Jul 16 21:59:17.590300 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3184/6361] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:59:17 2022] GET /compute/v2.1/servers/853637be-2161-4b9e-903a-2cbce6ff6881 => generated 1318 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:59:18.427824 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-e037d3ce-d354-4545-8239-3bf75385af92 req-c205509e-49a8-4da7-9457-f2eee86d3d48 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "853637be-2161-4b9e-903a-2cbce6ff6881", "name": "network-vif-plugged", "status": "completed", "tag": "341e460d-4246-420d-8082-44048820ac75"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:59:18.437916 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-e037d3ce-d354-4545-8239-3bf75385af92 req-c205509e-49a8-4da7-9457-f2eee86d3d48 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:59:18.438375 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-e037d3ce-d354-4545-8239-3bf75385af92 req-c205509e-49a8-4da7-9457-f2eee86d3d48 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:59:18.449452 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-e037d3ce-d354-4545-8239-3bf75385af92 req-c205509e-49a8-4da7-9457-f2eee86d3d48 service nova] Creating event network-vif-plugged:341e460d-4246-420d-8082-44048820ac75 for instance 853637be-2161-4b9e-903a-2cbce6ff6881 on nch2-master-1 Jul 16 21:59:18.454684 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-e037d3ce-d354-4545-8239-3bf75385af92 req-c205509e-49a8-4da7-9457-f2eee86d3d48 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.030594 Jul 16 21:59:18.455336 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3178/6362] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:59:18 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:59:18.607266 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-aa02fd22-1814-40a2-88b3-05819bf2298a tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:59:18.607903 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-aa02fd22-1814-40a2-88b3-05819bf2298a tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] [instance: 853637be-2161-4b9e-903a-2cbce6ff6881] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:59:18.614981 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-aa02fd22-1814-40a2-88b3-05819bf2298a tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:59:18.620070 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-aa02fd22-1814-40a2-88b3-05819bf2298a tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:59:18.620070 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-aa02fd22-1814-40a2-88b3-05819bf2298a tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:59:18.620070 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-aa02fd22-1814-40a2-88b3-05819bf2298a tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:59:18.747696 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-aa02fd22-1814-40a2-88b3-05819bf2298a tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd980641608134eadb72afa6fddb6adeb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9934bd813edb4e55a2d7c61207a9c46d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:59:18.755246 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-aa02fd22-1814-40a2-88b3-05819bf2298a tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] 10.222.0.16 "GET /compute/v2.1/servers/853637be-2161-4b9e-903a-2cbce6ff6881" status: 200 len: 1318 microversion: 2.1 time: 0.152627 Jul 16 21:59:18.755818 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3185/6363] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:59:18 2022] GET /compute/v2.1/servers/853637be-2161-4b9e-903a-2cbce6ff6881 => generated 1318 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:59:19.769884 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell0:SELECT=2 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:59:19.773594 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell0:SELECT=6 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:59:19.776905 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-4c51ae2d-ac89-4f9c-80bc-85b5c9676cc9 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:59:19.778172 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-4c51ae2d-ac89-4f9c-80bc-85b5c9676cc9 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] [instance: 853637be-2161-4b9e-903a-2cbce6ff6881] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:59:19.785053 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4c51ae2d-ac89-4f9c-80bc-85b5c9676cc9 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:59:19.785513 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4c51ae2d-ac89-4f9c-80bc-85b5c9676cc9 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:59:19.786290 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4c51ae2d-ac89-4f9c-80bc-85b5c9676cc9 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:59:19.786728 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4c51ae2d-ac89-4f9c-80bc-85b5c9676cc9 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:59:19.925005 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-4c51ae2d-ac89-4f9c-80bc-85b5c9676cc9 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd980641608134eadb72afa6fddb6adeb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9934bd813edb4e55a2d7c61207a9c46d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:59:19.932720 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-4c51ae2d-ac89-4f9c-80bc-85b5c9676cc9 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] 10.222.0.16 "GET /compute/v2.1/servers/853637be-2161-4b9e-903a-2cbce6ff6881" status: 200 len: 1318 microversion: 2.1 time: 0.163781 Jul 16 21:59:19.936034 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3179/6364] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:59:19 2022] GET /compute/v2.1/servers/853637be-2161-4b9e-903a-2cbce6ff6881 => generated 1318 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:59:20.951420 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-bdad04bd-081c-40b5-9161-7a9142fd267e tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:59:20.951977 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-bdad04bd-081c-40b5-9161-7a9142fd267e tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] [instance: 853637be-2161-4b9e-903a-2cbce6ff6881] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:59:20.954008 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell0:SELECT=2 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:59:20.959800 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-bdad04bd-081c-40b5-9161-7a9142fd267e tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:59:20.960269 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-bdad04bd-081c-40b5-9161-7a9142fd267e tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:59:20.961208 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-bdad04bd-081c-40b5-9161-7a9142fd267e tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:59:20.961498 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-bdad04bd-081c-40b5-9161-7a9142fd267e tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:59:21.104595 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-bdad04bd-081c-40b5-9161-7a9142fd267e tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd980641608134eadb72afa6fddb6adeb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9934bd813edb4e55a2d7c61207a9c46d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:59:21.112666 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-bdad04bd-081c-40b5-9161-7a9142fd267e tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] 10.222.0.16 "GET /compute/v2.1/servers/853637be-2161-4b9e-903a-2cbce6ff6881" status: 200 len: 1508 microversion: 2.1 time: 0.165461 Jul 16 21:59:21.113237 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3186/6365] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:59:20 2022] GET /compute/v2.1/servers/853637be-2161-4b9e-903a-2cbce6ff6881 => generated 1508 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:59:21.129098 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-66df386c-ce04-47c1-a7c4-52b275878dce tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:59:21.129727 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-66df386c-ce04-47c1-a7c4-52b275878dce tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] [instance: 853637be-2161-4b9e-903a-2cbce6ff6881] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:59:21.136755 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-66df386c-ce04-47c1-a7c4-52b275878dce tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:59:21.137241 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-66df386c-ce04-47c1-a7c4-52b275878dce tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:59:21.138540 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-66df386c-ce04-47c1-a7c4-52b275878dce tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:59:21.138984 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-66df386c-ce04-47c1-a7c4-52b275878dce tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:59:21.268137 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-66df386c-ce04-47c1-a7c4-52b275878dce tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd980641608134eadb72afa6fddb6adeb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9934bd813edb4e55a2d7c61207a9c46d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:59:21.275690 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-66df386c-ce04-47c1-a7c4-52b275878dce tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] 10.222.0.16 "GET /compute/v2.1/servers/853637be-2161-4b9e-903a-2cbce6ff6881" status: 200 len: 1508 microversion: 2.1 time: 0.150518 Jul 16 21:59:21.276258 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3180/6366] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:59:21 2022] GET /compute/v2.1/servers/853637be-2161-4b9e-903a-2cbce6ff6881 => generated 1508 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:59:21.292518 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-7cced5c9-69d8-45a1-9c07-7439b03c425a tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Action: 'update', calling method: >, body: {"server": {"OS-DCF:diskConfig": "AUTO"}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:59:21.293655 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-7cced5c9-69d8-45a1-9c07-7439b03c425a tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] [instance: 853637be-2161-4b9e-903a-2cbce6ff6881] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:59:21.300118 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7cced5c9-69d8-45a1-9c07-7439b03c425a tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:59:21.300588 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7cced5c9-69d8-45a1-9c07-7439b03c425a tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:59:21.301364 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7cced5c9-69d8-45a1-9c07-7439b03c425a tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:59:21.301815 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7cced5c9-69d8-45a1-9c07-7439b03c425a tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:59:21.429986 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-7cced5c9-69d8-45a1-9c07-7439b03c425a tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] 10.222.0.16 "PUT /compute/v2.1/servers/853637be-2161-4b9e-903a-2cbce6ff6881" status: 200 len: 1080 microversion: 2.1 time: 0.141601 Jul 16 21:59:21.430680 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3187/6367] 10.222.0.16 () {60 vars in 1255 bytes} [Sat Jul 16 21:59:21 2022] PUT /compute/v2.1/servers/853637be-2161-4b9e-903a-2cbce6ff6881 => generated 1080 bytes in 143 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:59:21.445013 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-7d9b61be-6f1b-4a34-a1de-13c30eaa5b20 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:59:21.445634 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-7d9b61be-6f1b-4a34-a1de-13c30eaa5b20 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] [instance: 853637be-2161-4b9e-903a-2cbce6ff6881] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:59:21.452874 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7d9b61be-6f1b-4a34-a1de-13c30eaa5b20 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:59:21.453315 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7d9b61be-6f1b-4a34-a1de-13c30eaa5b20 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:59:21.454080 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7d9b61be-6f1b-4a34-a1de-13c30eaa5b20 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:59:21.454527 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7d9b61be-6f1b-4a34-a1de-13c30eaa5b20 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:59:21.591818 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-7d9b61be-6f1b-4a34-a1de-13c30eaa5b20 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd980641608134eadb72afa6fddb6adeb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9934bd813edb4e55a2d7c61207a9c46d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:59:21.599330 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-7d9b61be-6f1b-4a34-a1de-13c30eaa5b20 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] 10.222.0.16 "GET /compute/v2.1/servers/853637be-2161-4b9e-903a-2cbce6ff6881" status: 200 len: 1506 microversion: 2.1 time: 0.158141 Jul 16 21:59:21.599915 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3181/6368] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:59:21 2022] GET /compute/v2.1/servers/853637be-2161-4b9e-903a-2cbce6ff6881 => generated 1506 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:59:21.615648 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-d9f3fb2e-7279-43c1-b26c-4f29ad80eff2 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:59:21.616314 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-d9f3fb2e-7279-43c1-b26c-4f29ad80eff2 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] [instance: 853637be-2161-4b9e-903a-2cbce6ff6881] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:59:21.623266 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d9f3fb2e-7279-43c1-b26c-4f29ad80eff2 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:59:21.623741 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d9f3fb2e-7279-43c1-b26c-4f29ad80eff2 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:59:21.624518 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d9f3fb2e-7279-43c1-b26c-4f29ad80eff2 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:59:21.624971 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d9f3fb2e-7279-43c1-b26c-4f29ad80eff2 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:59:21.758879 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-d9f3fb2e-7279-43c1-b26c-4f29ad80eff2 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd980641608134eadb72afa6fddb6adeb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9934bd813edb4e55a2d7c61207a9c46d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:59:21.766936 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-d9f3fb2e-7279-43c1-b26c-4f29ad80eff2 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] 10.222.0.16 "GET /compute/v2.1/servers/853637be-2161-4b9e-903a-2cbce6ff6881" status: 200 len: 1506 microversion: 2.1 time: 0.155267 Jul 16 21:59:21.767534 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3188/6369] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:59:21 2022] GET /compute/v2.1/servers/853637be-2161-4b9e-903a-2cbce6ff6881 => generated 1506 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:59:21.783474 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-31707496-164f-4a99-b088-80c76b7e10e7 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Action: 'action', calling method: >, body: {"resize": {"flavorRef": "451", "OS-DCF:diskConfig": "MANUAL"}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:59:21.784007 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-31707496-164f-4a99-b088-80c76b7e10e7 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] [instance: 853637be-2161-4b9e-903a-2cbce6ff6881] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:59:21.790090 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-31707496-164f-4a99-b088-80c76b7e10e7 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:59:21.790549 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-31707496-164f-4a99-b088-80c76b7e10e7 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:59:21.791398 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-31707496-164f-4a99-b088-80c76b7e10e7 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:59:21.791842 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-31707496-164f-4a99-b088-80c76b7e10e7 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:59:21.820727 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-31707496-164f-4a99-b088-80c76b7e10e7 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Policy check for compute:servers:resize:cross_cell failed with credentials {'is_admin': False, 'user_id': 'd980641608134eadb72afa6fddb6adeb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9934bd813edb4e55a2d7c61207a9c46d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:59:21.827074 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-31707496-164f-4a99-b088-80c76b7e10e7 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] [instance: 853637be-2161-4b9e-903a-2cbce6ff6881] Old instance type m1.heat2, new instance type m1.heat {{(pid=138680) resize /opt/stack/nova/nova/compute/api.py:4166}} Jul 16 21:59:21.833470 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-31707496-164f-4a99-b088-80c76b7e10e7 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Flavor limits 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 16 21:59:21.833896 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-31707496-164f-4a99-b088-80c76b7e10e7 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Image limits 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 16 21:59:21.834382 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-31707496-164f-4a99-b088-80c76b7e10e7 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Flavor pref 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 16 21:59:21.834793 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-31707496-164f-4a99-b088-80c76b7e10e7 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Image pref 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 16 21:59:21.835244 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-31707496-164f-4a99-b088-80c76b7e10e7 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 16 21:59:22.069963 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-31707496-164f-4a99-b088-80c76b7e10e7 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] 10.222.0.16 "POST /compute/v2.1/servers/853637be-2161-4b9e-903a-2cbce6ff6881/action" status: 202 len: 0 microversion: 2.1 time: 0.290794 Jul 16 21:59:22.070585 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3182/6370] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 21:59:21 2022] POST /compute/v2.1/servers/853637be-2161-4b9e-903a-2cbce6ff6881/action => generated 0 bytes in 292 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 21:59:22.078788 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-58179105-da4f-4fe9-94b3-48093239a3c8 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:59:22.079476 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-58179105-da4f-4fe9-94b3-48093239a3c8 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] [instance: 853637be-2161-4b9e-903a-2cbce6ff6881] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:59:22.086251 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-58179105-da4f-4fe9-94b3-48093239a3c8 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:59:22.086718 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-58179105-da4f-4fe9-94b3-48093239a3c8 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:59:22.087528 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-58179105-da4f-4fe9-94b3-48093239a3c8 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:59:22.088301 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-58179105-da4f-4fe9-94b3-48093239a3c8 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:59:22.221614 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-58179105-da4f-4fe9-94b3-48093239a3c8 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd980641608134eadb72afa6fddb6adeb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9934bd813edb4e55a2d7c61207a9c46d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:59:22.229393 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-58179105-da4f-4fe9-94b3-48093239a3c8 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] 10.222.0.16 "GET /compute/v2.1/servers/853637be-2161-4b9e-903a-2cbce6ff6881" status: 200 len: 1517 microversion: 2.1 time: 0.154541 Jul 16 21:59:22.232883 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3189/6371] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:59:22 2022] GET /compute/v2.1/servers/853637be-2161-4b9e-903a-2cbce6ff6881 => generated 1517 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:59:23.249350 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-360b3899-42b9-42e3-bc45-873d651351bf tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:59:23.249929 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-360b3899-42b9-42e3-bc45-873d651351bf tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] [instance: 853637be-2161-4b9e-903a-2cbce6ff6881] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:59:23.256187 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-360b3899-42b9-42e3-bc45-873d651351bf tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:59:23.256635 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-360b3899-42b9-42e3-bc45-873d651351bf tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:59:23.257434 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-360b3899-42b9-42e3-bc45-873d651351bf tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:59:23.257992 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-360b3899-42b9-42e3-bc45-873d651351bf tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:59:23.400719 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-360b3899-42b9-42e3-bc45-873d651351bf tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd980641608134eadb72afa6fddb6adeb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9934bd813edb4e55a2d7c61207a9c46d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:59:23.408473 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-360b3899-42b9-42e3-bc45-873d651351bf tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] 10.222.0.16 "GET /compute/v2.1/servers/853637be-2161-4b9e-903a-2cbce6ff6881" status: 200 len: 1517 microversion: 2.1 time: 0.165158 Jul 16 21:59:23.409034 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3183/6372] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:59:23 2022] GET /compute/v2.1/servers/853637be-2161-4b9e-903a-2cbce6ff6881 => generated 1517 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:59:24.425790 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-86642d65-b64e-4d04-85a3-7ec5482e1a7c tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:59:24.427760 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-86642d65-b64e-4d04-85a3-7ec5482e1a7c tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] [instance: 853637be-2161-4b9e-903a-2cbce6ff6881] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:59:24.433229 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-86642d65-b64e-4d04-85a3-7ec5482e1a7c tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:59:24.433719 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-86642d65-b64e-4d04-85a3-7ec5482e1a7c tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:59:24.434564 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-86642d65-b64e-4d04-85a3-7ec5482e1a7c tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:59:24.435035 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-86642d65-b64e-4d04-85a3-7ec5482e1a7c tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:59:24.658153 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-86642d65-b64e-4d04-85a3-7ec5482e1a7c tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd980641608134eadb72afa6fddb6adeb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9934bd813edb4e55a2d7c61207a9c46d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:59:24.665686 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-86642d65-b64e-4d04-85a3-7ec5482e1a7c tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] 10.222.0.16 "GET /compute/v2.1/servers/853637be-2161-4b9e-903a-2cbce6ff6881" status: 200 len: 1517 microversion: 2.1 time: 0.243855 Jul 16 21:59:24.666257 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3190/6373] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:59:24 2022] GET /compute/v2.1/servers/853637be-2161-4b9e-903a-2cbce6ff6881 => generated 1517 bytes in 245 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:59:25.683307 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-4896c47a-b911-4584-922c-bfbfe59295bf tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:59:25.683936 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-4896c47a-b911-4584-922c-bfbfe59295bf tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] [instance: 853637be-2161-4b9e-903a-2cbce6ff6881] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:59:25.691065 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4896c47a-b911-4584-922c-bfbfe59295bf tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:59:25.692284 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4896c47a-b911-4584-922c-bfbfe59295bf tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:59:25.692416 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4896c47a-b911-4584-922c-bfbfe59295bf tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:59:25.692832 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4896c47a-b911-4584-922c-bfbfe59295bf tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:59:25.830920 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-4896c47a-b911-4584-922c-bfbfe59295bf tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd980641608134eadb72afa6fddb6adeb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9934bd813edb4e55a2d7c61207a9c46d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:59:25.838829 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-4896c47a-b911-4584-922c-bfbfe59295bf tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] 10.222.0.16 "GET /compute/v2.1/servers/853637be-2161-4b9e-903a-2cbce6ff6881" status: 200 len: 1522 microversion: 2.1 time: 0.159571 Jul 16 21:59:25.839444 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3184/6374] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:59:25 2022] GET /compute/v2.1/servers/853637be-2161-4b9e-903a-2cbce6ff6881 => generated 1522 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:59:26.856217 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-9cefc361-89f5-4e56-9db9-804ae645a0aa tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:59:26.856844 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-9cefc361-89f5-4e56-9db9-804ae645a0aa tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] [instance: 853637be-2161-4b9e-903a-2cbce6ff6881] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:59:26.864437 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9cefc361-89f5-4e56-9db9-804ae645a0aa tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:59:26.864898 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9cefc361-89f5-4e56-9db9-804ae645a0aa tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:59:26.865726 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9cefc361-89f5-4e56-9db9-804ae645a0aa tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:59:26.866177 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9cefc361-89f5-4e56-9db9-804ae645a0aa tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:59:27.000299 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-9cefc361-89f5-4e56-9db9-804ae645a0aa tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd980641608134eadb72afa6fddb6adeb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9934bd813edb4e55a2d7c61207a9c46d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:59:27.007944 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-9cefc361-89f5-4e56-9db9-804ae645a0aa tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] 10.222.0.16 "GET /compute/v2.1/servers/853637be-2161-4b9e-903a-2cbce6ff6881" status: 200 len: 1522 microversion: 2.1 time: 0.155621 Jul 16 21:59:27.008516 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3191/6375] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:59:26 2022] GET /compute/v2.1/servers/853637be-2161-4b9e-903a-2cbce6ff6881 => generated 1522 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:59:28.025182 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-0a565c03-c9f9-4d75-b161-b8998ef2bdce tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:59:28.025803 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-0a565c03-c9f9-4d75-b161-b8998ef2bdce tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] [instance: 853637be-2161-4b9e-903a-2cbce6ff6881] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:59:28.033505 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0a565c03-c9f9-4d75-b161-b8998ef2bdce tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:59:28.033960 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0a565c03-c9f9-4d75-b161-b8998ef2bdce tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:59:28.034724 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0a565c03-c9f9-4d75-b161-b8998ef2bdce tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:59:28.035289 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0a565c03-c9f9-4d75-b161-b8998ef2bdce tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:59:28.172434 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-0a565c03-c9f9-4d75-b161-b8998ef2bdce tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd980641608134eadb72afa6fddb6adeb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9934bd813edb4e55a2d7c61207a9c46d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:59:28.180108 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-0a565c03-c9f9-4d75-b161-b8998ef2bdce tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] 10.222.0.16 "GET /compute/v2.1/servers/853637be-2161-4b9e-903a-2cbce6ff6881" status: 200 len: 1522 microversion: 2.1 time: 0.158812 Jul 16 21:59:28.180683 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3185/6376] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:59:28 2022] GET /compute/v2.1/servers/853637be-2161-4b9e-903a-2cbce6ff6881 => generated 1522 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:59:29.197812 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-015a81c0-6ce0-42ea-b398-757758ea046c tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:59:29.198358 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-015a81c0-6ce0-42ea-b398-757758ea046c tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] [instance: 853637be-2161-4b9e-903a-2cbce6ff6881] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:59:29.205590 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-015a81c0-6ce0-42ea-b398-757758ea046c tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:59:29.206053 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-015a81c0-6ce0-42ea-b398-757758ea046c tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:59:29.206823 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-015a81c0-6ce0-42ea-b398-757758ea046c tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:59:29.207298 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-015a81c0-6ce0-42ea-b398-757758ea046c tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:59:29.336027 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-015a81c0-6ce0-42ea-b398-757758ea046c tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd980641608134eadb72afa6fddb6adeb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9934bd813edb4e55a2d7c61207a9c46d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:59:29.344814 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-015a81c0-6ce0-42ea-b398-757758ea046c tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] 10.222.0.16 "GET /compute/v2.1/servers/853637be-2161-4b9e-903a-2cbce6ff6881" status: 200 len: 1522 microversion: 2.1 time: 0.150996 Jul 16 21:59:29.345371 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3192/6377] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:59:29 2022] GET /compute/v2.1/servers/853637be-2161-4b9e-903a-2cbce6ff6881 => generated 1522 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:59:30.362244 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-0b91b2bd-d706-429f-8ec9-d3e66d1f18d8 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:59:30.362868 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-0b91b2bd-d706-429f-8ec9-d3e66d1f18d8 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] [instance: 853637be-2161-4b9e-903a-2cbce6ff6881] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:59:30.370055 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0b91b2bd-d706-429f-8ec9-d3e66d1f18d8 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:59:30.370513 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0b91b2bd-d706-429f-8ec9-d3e66d1f18d8 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:59:30.371305 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0b91b2bd-d706-429f-8ec9-d3e66d1f18d8 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:59:30.371752 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0b91b2bd-d706-429f-8ec9-d3e66d1f18d8 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:59:30.505701 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-0b91b2bd-d706-429f-8ec9-d3e66d1f18d8 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd980641608134eadb72afa6fddb6adeb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9934bd813edb4e55a2d7c61207a9c46d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:59:30.513237 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-0b91b2bd-d706-429f-8ec9-d3e66d1f18d8 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] 10.222.0.16 "GET /compute/v2.1/servers/853637be-2161-4b9e-903a-2cbce6ff6881" status: 200 len: 1522 microversion: 2.1 time: 0.155515 Jul 16 21:59:30.513804 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3186/6378] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:59:30 2022] GET /compute/v2.1/servers/853637be-2161-4b9e-903a-2cbce6ff6881 => generated 1522 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:59:31.530213 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-cf6e2f04-ae91-4600-a9b7-fc798b628a72 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:59:31.530902 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-cf6e2f04-ae91-4600-a9b7-fc798b628a72 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] [instance: 853637be-2161-4b9e-903a-2cbce6ff6881] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:59:31.538007 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cf6e2f04-ae91-4600-a9b7-fc798b628a72 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:59:31.538472 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cf6e2f04-ae91-4600-a9b7-fc798b628a72 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:59:31.539284 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cf6e2f04-ae91-4600-a9b7-fc798b628a72 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:59:31.539820 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cf6e2f04-ae91-4600-a9b7-fc798b628a72 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:59:31.672528 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-cf6e2f04-ae91-4600-a9b7-fc798b628a72 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd980641608134eadb72afa6fddb6adeb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9934bd813edb4e55a2d7c61207a9c46d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:59:31.680145 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-cf6e2f04-ae91-4600-a9b7-fc798b628a72 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] 10.222.0.16 "GET /compute/v2.1/servers/853637be-2161-4b9e-903a-2cbce6ff6881" status: 200 len: 1522 microversion: 2.1 time: 0.153836 Jul 16 21:59:31.680743 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3193/6379] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:59:31 2022] GET /compute/v2.1/servers/853637be-2161-4b9e-903a-2cbce6ff6881 => generated 1522 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:59:32.697721 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-c228f4a4-06ed-4a57-b9db-5537aff7bd94 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:59:32.698341 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-c228f4a4-06ed-4a57-b9db-5537aff7bd94 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] [instance: 853637be-2161-4b9e-903a-2cbce6ff6881] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:59:32.706048 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c228f4a4-06ed-4a57-b9db-5537aff7bd94 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:59:32.706505 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c228f4a4-06ed-4a57-b9db-5537aff7bd94 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:59:32.707300 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c228f4a4-06ed-4a57-b9db-5537aff7bd94 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:59:32.707742 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c228f4a4-06ed-4a57-b9db-5537aff7bd94 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:59:32.840222 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-c228f4a4-06ed-4a57-b9db-5537aff7bd94 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd980641608134eadb72afa6fddb6adeb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9934bd813edb4e55a2d7c61207a9c46d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:59:32.848220 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-c228f4a4-06ed-4a57-b9db-5537aff7bd94 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] 10.222.0.16 "GET /compute/v2.1/servers/853637be-2161-4b9e-903a-2cbce6ff6881" status: 200 len: 1522 microversion: 2.1 time: 0.154352 Jul 16 21:59:32.848793 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3187/6380] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:59:32 2022] GET /compute/v2.1/servers/853637be-2161-4b9e-903a-2cbce6ff6881 => generated 1522 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:59:33.865816 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-4f24397e-1867-46a5-965f-0b5ff08685bd tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:59:33.866472 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-4f24397e-1867-46a5-965f-0b5ff08685bd tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] [instance: 853637be-2161-4b9e-903a-2cbce6ff6881] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:59:33.872972 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4f24397e-1867-46a5-965f-0b5ff08685bd tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:59:33.873493 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4f24397e-1867-46a5-965f-0b5ff08685bd tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:59:33.880466 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4f24397e-1867-46a5-965f-0b5ff08685bd tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:59:33.880466 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4f24397e-1867-46a5-965f-0b5ff08685bd tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:59:34.017332 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-4f24397e-1867-46a5-965f-0b5ff08685bd tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd980641608134eadb72afa6fddb6adeb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9934bd813edb4e55a2d7c61207a9c46d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:59:34.025230 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-4f24397e-1867-46a5-965f-0b5ff08685bd tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] 10.222.0.16 "GET /compute/v2.1/servers/853637be-2161-4b9e-903a-2cbce6ff6881" status: 200 len: 1522 microversion: 2.1 time: 0.163823 Jul 16 21:59:34.025799 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3194/6381] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:59:33 2022] GET /compute/v2.1/servers/853637be-2161-4b9e-903a-2cbce6ff6881 => generated 1522 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:59:35.043263 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-f39883a6-2e70-4b3d-8084-00a25ba197d2 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:59:35.043884 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-f39883a6-2e70-4b3d-8084-00a25ba197d2 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] [instance: 853637be-2161-4b9e-903a-2cbce6ff6881] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:59:35.050910 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f39883a6-2e70-4b3d-8084-00a25ba197d2 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:59:35.051405 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f39883a6-2e70-4b3d-8084-00a25ba197d2 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:59:35.052182 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f39883a6-2e70-4b3d-8084-00a25ba197d2 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:59:35.052625 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f39883a6-2e70-4b3d-8084-00a25ba197d2 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:59:35.202131 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-f39883a6-2e70-4b3d-8084-00a25ba197d2 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd980641608134eadb72afa6fddb6adeb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9934bd813edb4e55a2d7c61207a9c46d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:59:35.210003 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-f39883a6-2e70-4b3d-8084-00a25ba197d2 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] 10.222.0.16 "GET /compute/v2.1/servers/853637be-2161-4b9e-903a-2cbce6ff6881" status: 200 len: 1522 microversion: 2.1 time: 0.170679 Jul 16 21:59:35.210872 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3188/6382] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:59:35 2022] GET /compute/v2.1/servers/853637be-2161-4b9e-903a-2cbce6ff6881 => generated 1522 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:59:36.227967 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-33464a0e-4cdf-4445-a2dd-472cc01000a6 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:59:36.228591 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-33464a0e-4cdf-4445-a2dd-472cc01000a6 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] [instance: 853637be-2161-4b9e-903a-2cbce6ff6881] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:59:36.236317 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-33464a0e-4cdf-4445-a2dd-472cc01000a6 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:59:36.236779 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-33464a0e-4cdf-4445-a2dd-472cc01000a6 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:59:36.237548 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-33464a0e-4cdf-4445-a2dd-472cc01000a6 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:59:36.237994 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-33464a0e-4cdf-4445-a2dd-472cc01000a6 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:59:36.371385 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-33464a0e-4cdf-4445-a2dd-472cc01000a6 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd980641608134eadb72afa6fddb6adeb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9934bd813edb4e55a2d7c61207a9c46d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:59:36.379624 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-33464a0e-4cdf-4445-a2dd-472cc01000a6 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] 10.222.0.16 "GET /compute/v2.1/servers/853637be-2161-4b9e-903a-2cbce6ff6881" status: 200 len: 1522 microversion: 2.1 time: 0.155557 Jul 16 21:59:36.380200 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3195/6383] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:59:36 2022] GET /compute/v2.1/servers/853637be-2161-4b9e-903a-2cbce6ff6881 => generated 1522 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:59:37.396414 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-e8b7c696-3db5-4bde-8d8c-b06820e617fa tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:59:37.397033 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-e8b7c696-3db5-4bde-8d8c-b06820e617fa tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] [instance: 853637be-2161-4b9e-903a-2cbce6ff6881] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:59:37.404090 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e8b7c696-3db5-4bde-8d8c-b06820e617fa tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:59:37.404549 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e8b7c696-3db5-4bde-8d8c-b06820e617fa tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:59:37.405965 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e8b7c696-3db5-4bde-8d8c-b06820e617fa tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:59:37.406413 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e8b7c696-3db5-4bde-8d8c-b06820e617fa tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:59:37.425399 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:59:37.538745 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-e8b7c696-3db5-4bde-8d8c-b06820e617fa tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd980641608134eadb72afa6fddb6adeb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9934bd813edb4e55a2d7c61207a9c46d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:59:37.546501 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-e8b7c696-3db5-4bde-8d8c-b06820e617fa tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] 10.222.0.16 "GET /compute/v2.1/servers/853637be-2161-4b9e-903a-2cbce6ff6881" status: 200 len: 1522 microversion: 2.1 time: 0.154141 Jul 16 21:59:37.547099 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3189/6384] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:59:37 2022] GET /compute/v2.1/servers/853637be-2161-4b9e-903a-2cbce6ff6881 => generated 1522 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:59:38.564211 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-e25dae66-79db-42d1-bcf2-75df46108ec4 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:59:38.565185 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-e25dae66-79db-42d1-bcf2-75df46108ec4 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] [instance: 853637be-2161-4b9e-903a-2cbce6ff6881] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:59:38.572161 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e25dae66-79db-42d1-bcf2-75df46108ec4 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:59:38.572628 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e25dae66-79db-42d1-bcf2-75df46108ec4 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:59:38.573404 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e25dae66-79db-42d1-bcf2-75df46108ec4 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:59:38.573853 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e25dae66-79db-42d1-bcf2-75df46108ec4 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:59:38.706050 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-e25dae66-79db-42d1-bcf2-75df46108ec4 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd980641608134eadb72afa6fddb6adeb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9934bd813edb4e55a2d7c61207a9c46d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:59:38.714011 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-e25dae66-79db-42d1-bcf2-75df46108ec4 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] 10.222.0.16 "GET /compute/v2.1/servers/853637be-2161-4b9e-903a-2cbce6ff6881" status: 200 len: 1522 microversion: 2.1 time: 0.154555 Jul 16 21:59:38.714581 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3196/6385] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:59:38 2022] GET /compute/v2.1/servers/853637be-2161-4b9e-903a-2cbce6ff6881 => generated 1522 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:59:39.731092 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-cdf62b1d-f5a8-4807-b0f5-99e384f0803c tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:59:39.731750 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-cdf62b1d-f5a8-4807-b0f5-99e384f0803c tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] [instance: 853637be-2161-4b9e-903a-2cbce6ff6881] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:59:39.734471 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_api:SELECT=50,nova_api:INSERT=3 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:59:39.739519 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cdf62b1d-f5a8-4807-b0f5-99e384f0803c tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:59:39.739973 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cdf62b1d-f5a8-4807-b0f5-99e384f0803c tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:59:39.740746 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cdf62b1d-f5a8-4807-b0f5-99e384f0803c tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:59:39.741187 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cdf62b1d-f5a8-4807-b0f5-99e384f0803c tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:59:39.878705 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-cdf62b1d-f5a8-4807-b0f5-99e384f0803c tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd980641608134eadb72afa6fddb6adeb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9934bd813edb4e55a2d7c61207a9c46d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:59:39.883824 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-cdf62b1d-f5a8-4807-b0f5-99e384f0803c tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] 10.222.0.16 "GET /compute/v2.1/servers/853637be-2161-4b9e-903a-2cbce6ff6881" status: 200 len: 1522 microversion: 2.1 time: 0.156722 Jul 16 21:59:39.884427 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3190/6386] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:59:39 2022] GET /compute/v2.1/servers/853637be-2161-4b9e-903a-2cbce6ff6881 => generated 1522 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:59:40.901696 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-1f58e3bb-9d76-4200-b8c0-8402ba6b55c9 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:59:40.902330 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-1f58e3bb-9d76-4200-b8c0-8402ba6b55c9 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] [instance: 853637be-2161-4b9e-903a-2cbce6ff6881] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:59:40.905702 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_api:SELECT=37 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:59:40.910525 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1f58e3bb-9d76-4200-b8c0-8402ba6b55c9 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:59:40.910996 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1f58e3bb-9d76-4200-b8c0-8402ba6b55c9 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:59:40.911828 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1f58e3bb-9d76-4200-b8c0-8402ba6b55c9 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:59:40.912343 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1f58e3bb-9d76-4200-b8c0-8402ba6b55c9 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:59:40.930875 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=99,nova_cell1:UPDATE=1 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 21:59:41.043567 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-1f58e3bb-9d76-4200-b8c0-8402ba6b55c9 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd980641608134eadb72afa6fddb6adeb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9934bd813edb4e55a2d7c61207a9c46d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:59:41.052074 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-1f58e3bb-9d76-4200-b8c0-8402ba6b55c9 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] 10.222.0.16 "GET /compute/v2.1/servers/853637be-2161-4b9e-903a-2cbce6ff6881" status: 200 len: 1522 microversion: 2.1 time: 0.154383 Jul 16 21:59:41.052665 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3197/6387] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:59:40 2022] GET /compute/v2.1/servers/853637be-2161-4b9e-903a-2cbce6ff6881 => generated 1522 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:59:42.069373 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-bf82e16a-a940-495e-9cc2-a50c8c75873e tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:59:42.069996 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-bf82e16a-a940-495e-9cc2-a50c8c75873e tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] [instance: 853637be-2161-4b9e-903a-2cbce6ff6881] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:59:42.077329 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-bf82e16a-a940-495e-9cc2-a50c8c75873e tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:59:42.077780 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-bf82e16a-a940-495e-9cc2-a50c8c75873e tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:59:42.078585 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-bf82e16a-a940-495e-9cc2-a50c8c75873e tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:59:42.079096 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-bf82e16a-a940-495e-9cc2-a50c8c75873e tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:59:42.212690 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-bf82e16a-a940-495e-9cc2-a50c8c75873e tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd980641608134eadb72afa6fddb6adeb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9934bd813edb4e55a2d7c61207a9c46d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:59:42.221010 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-bf82e16a-a940-495e-9cc2-a50c8c75873e tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] 10.222.0.16 "GET /compute/v2.1/servers/853637be-2161-4b9e-903a-2cbce6ff6881" status: 200 len: 1522 microversion: 2.1 time: 0.155618 Jul 16 21:59:42.221605 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3191/6388] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:59:42 2022] GET /compute/v2.1/servers/853637be-2161-4b9e-903a-2cbce6ff6881 => generated 1522 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:59:43.238849 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-cc288a24-f1b5-4153-a11f-ed7a6aa12602 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:59:43.245621 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-cc288a24-f1b5-4153-a11f-ed7a6aa12602 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] [instance: 853637be-2161-4b9e-903a-2cbce6ff6881] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:59:43.246573 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cc288a24-f1b5-4153-a11f-ed7a6aa12602 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:59:43.247131 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cc288a24-f1b5-4153-a11f-ed7a6aa12602 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:59:43.247906 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cc288a24-f1b5-4153-a11f-ed7a6aa12602 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:59:43.248352 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cc288a24-f1b5-4153-a11f-ed7a6aa12602 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:59:43.378541 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-cc288a24-f1b5-4153-a11f-ed7a6aa12602 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd980641608134eadb72afa6fddb6adeb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9934bd813edb4e55a2d7c61207a9c46d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:59:43.386105 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-cc288a24-f1b5-4153-a11f-ed7a6aa12602 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] 10.222.0.16 "GET /compute/v2.1/servers/853637be-2161-4b9e-903a-2cbce6ff6881" status: 200 len: 1522 microversion: 2.1 time: 0.152041 Jul 16 21:59:43.386685 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3198/6389] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:59:43 2022] GET /compute/v2.1/servers/853637be-2161-4b9e-903a-2cbce6ff6881 => generated 1522 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:59:44.403662 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-850fc347-961f-4168-8b7d-930d8db72b0b tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:59:44.404275 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-850fc347-961f-4168-8b7d-930d8db72b0b tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] [instance: 853637be-2161-4b9e-903a-2cbce6ff6881] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:59:44.411465 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-850fc347-961f-4168-8b7d-930d8db72b0b tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:59:44.411919 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-850fc347-961f-4168-8b7d-930d8db72b0b tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:59:44.412673 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-850fc347-961f-4168-8b7d-930d8db72b0b tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:59:44.413111 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-850fc347-961f-4168-8b7d-930d8db72b0b tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:59:44.541639 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-850fc347-961f-4168-8b7d-930d8db72b0b tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd980641608134eadb72afa6fddb6adeb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9934bd813edb4e55a2d7c61207a9c46d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:59:44.550043 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-850fc347-961f-4168-8b7d-930d8db72b0b tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] 10.222.0.16 "GET /compute/v2.1/servers/853637be-2161-4b9e-903a-2cbce6ff6881" status: 200 len: 1522 microversion: 2.1 time: 0.150393 Jul 16 21:59:44.550622 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3192/6390] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:59:44 2022] GET /compute/v2.1/servers/853637be-2161-4b9e-903a-2cbce6ff6881 => generated 1522 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:59:45.567450 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-27f8d4fb-449f-4863-9db0-9802e196e0fd tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:59:45.568110 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-27f8d4fb-449f-4863-9db0-9802e196e0fd tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] [instance: 853637be-2161-4b9e-903a-2cbce6ff6881] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:59:45.575783 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-27f8d4fb-449f-4863-9db0-9802e196e0fd tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:59:45.576257 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-27f8d4fb-449f-4863-9db0-9802e196e0fd tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:59:45.577037 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-27f8d4fb-449f-4863-9db0-9802e196e0fd tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:59:45.577546 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-27f8d4fb-449f-4863-9db0-9802e196e0fd tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:59:45.711595 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-27f8d4fb-449f-4863-9db0-9802e196e0fd tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd980641608134eadb72afa6fddb6adeb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9934bd813edb4e55a2d7c61207a9c46d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:59:45.719451 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-27f8d4fb-449f-4863-9db0-9802e196e0fd tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] 10.222.0.16 "GET /compute/v2.1/servers/853637be-2161-4b9e-903a-2cbce6ff6881" status: 200 len: 1522 microversion: 2.1 time: 0.155843 Jul 16 21:59:45.720891 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3199/6391] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:59:45 2022] GET /compute/v2.1/servers/853637be-2161-4b9e-903a-2cbce6ff6881 => generated 1522 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:59:46.115895 ncdv-master-1 devstack@n-api.service[138681]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jul 16 21:59:46.116166 ncdv-master-1 devstack@n-api.service[138681]: Sat Jul 16 21:59:46 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Jul 16 21:59:46.736775 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-5c71cf72-7532-442e-ad9b-b7ce79abafe5 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:59:46.737396 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-5c71cf72-7532-442e-ad9b-b7ce79abafe5 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] [instance: 853637be-2161-4b9e-903a-2cbce6ff6881] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:59:46.744504 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5c71cf72-7532-442e-ad9b-b7ce79abafe5 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:59:46.744954 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5c71cf72-7532-442e-ad9b-b7ce79abafe5 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:59:46.745710 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5c71cf72-7532-442e-ad9b-b7ce79abafe5 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:59:46.746147 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5c71cf72-7532-442e-ad9b-b7ce79abafe5 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:59:46.836901 ncdv-master-1 devstack@n-api.service[138681]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jul 16 21:59:46.837628 ncdv-master-1 devstack@n-api.service[138681]: Sat Jul 16 21:59:46 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Jul 16 21:59:46.879673 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-5c71cf72-7532-442e-ad9b-b7ce79abafe5 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd980641608134eadb72afa6fddb6adeb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9934bd813edb4e55a2d7c61207a9c46d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:59:46.887444 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-5c71cf72-7532-442e-ad9b-b7ce79abafe5 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] 10.222.0.16 "GET /compute/v2.1/servers/853637be-2161-4b9e-903a-2cbce6ff6881" status: 200 len: 1522 microversion: 2.1 time: 0.154910 Jul 16 21:59:46.888014 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3193/6392] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:59:46 2022] GET /compute/v2.1/servers/853637be-2161-4b9e-903a-2cbce6ff6881 => generated 1522 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:59:47.907743 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-36c12d64-305b-45ba-a693-05b48a7edbf5 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:59:47.908371 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-36c12d64-305b-45ba-a693-05b48a7edbf5 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] [instance: 853637be-2161-4b9e-903a-2cbce6ff6881] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:59:47.915995 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-36c12d64-305b-45ba-a693-05b48a7edbf5 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:59:47.916459 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-36c12d64-305b-45ba-a693-05b48a7edbf5 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:59:47.917230 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-36c12d64-305b-45ba-a693-05b48a7edbf5 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:59:47.917684 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-36c12d64-305b-45ba-a693-05b48a7edbf5 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:59:48.052757 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-36c12d64-305b-45ba-a693-05b48a7edbf5 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd980641608134eadb72afa6fddb6adeb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9934bd813edb4e55a2d7c61207a9c46d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:59:48.060509 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-36c12d64-305b-45ba-a693-05b48a7edbf5 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] 10.222.0.16 "GET /compute/v2.1/servers/853637be-2161-4b9e-903a-2cbce6ff6881" status: 200 len: 1522 microversion: 2.1 time: 0.158009 Jul 16 21:59:48.061069 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3200/6393] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:59:47 2022] GET /compute/v2.1/servers/853637be-2161-4b9e-903a-2cbce6ff6881 => generated 1522 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:59:49.076937 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-5b50a194-0646-43db-ab64-6afa2ad2efca tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:59:49.077686 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-5b50a194-0646-43db-ab64-6afa2ad2efca tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] [instance: 853637be-2161-4b9e-903a-2cbce6ff6881] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:59:49.084758 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5b50a194-0646-43db-ab64-6afa2ad2efca tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:59:49.085215 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5b50a194-0646-43db-ab64-6afa2ad2efca tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:59:49.085986 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5b50a194-0646-43db-ab64-6afa2ad2efca tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:59:49.086432 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5b50a194-0646-43db-ab64-6afa2ad2efca tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:59:49.218466 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-5b50a194-0646-43db-ab64-6afa2ad2efca tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd980641608134eadb72afa6fddb6adeb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9934bd813edb4e55a2d7c61207a9c46d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:59:49.226287 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-5b50a194-0646-43db-ab64-6afa2ad2efca tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] 10.222.0.16 "GET /compute/v2.1/servers/853637be-2161-4b9e-903a-2cbce6ff6881" status: 200 len: 1522 microversion: 2.1 time: 0.153277 Jul 16 21:59:49.226872 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3194/6394] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:59:49 2022] GET /compute/v2.1/servers/853637be-2161-4b9e-903a-2cbce6ff6881 => generated 1522 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:59:50.243823 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-f518d6f4-1bd5-4316-8e67-85ae329d5ade tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:59:50.244454 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-f518d6f4-1bd5-4316-8e67-85ae329d5ade tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] [instance: 853637be-2161-4b9e-903a-2cbce6ff6881] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:59:50.251428 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f518d6f4-1bd5-4316-8e67-85ae329d5ade tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:59:50.251913 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f518d6f4-1bd5-4316-8e67-85ae329d5ade tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:59:50.253901 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f518d6f4-1bd5-4316-8e67-85ae329d5ade tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:59:50.253901 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f518d6f4-1bd5-4316-8e67-85ae329d5ade tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:59:50.387774 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-f518d6f4-1bd5-4316-8e67-85ae329d5ade tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd980641608134eadb72afa6fddb6adeb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9934bd813edb4e55a2d7c61207a9c46d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:59:50.395830 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-f518d6f4-1bd5-4316-8e67-85ae329d5ade tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] 10.222.0.16 "GET /compute/v2.1/servers/853637be-2161-4b9e-903a-2cbce6ff6881" status: 200 len: 1522 microversion: 2.1 time: 0.155941 Jul 16 21:59:50.396393 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3201/6395] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:59:50 2022] GET /compute/v2.1/servers/853637be-2161-4b9e-903a-2cbce6ff6881 => generated 1522 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:59:51.412769 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-1334212c-0e8b-4421-aac9-1fe4d9c4ae4e tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:59:51.413396 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-1334212c-0e8b-4421-aac9-1fe4d9c4ae4e tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] [instance: 853637be-2161-4b9e-903a-2cbce6ff6881] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:59:51.420444 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1334212c-0e8b-4421-aac9-1fe4d9c4ae4e tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:59:51.420900 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1334212c-0e8b-4421-aac9-1fe4d9c4ae4e tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:59:51.422861 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1334212c-0e8b-4421-aac9-1fe4d9c4ae4e tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:59:51.423344 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1334212c-0e8b-4421-aac9-1fe4d9c4ae4e tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:59:51.553029 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-1334212c-0e8b-4421-aac9-1fe4d9c4ae4e tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd980641608134eadb72afa6fddb6adeb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9934bd813edb4e55a2d7c61207a9c46d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:59:51.560679 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-1334212c-0e8b-4421-aac9-1fe4d9c4ae4e tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] 10.222.0.16 "GET /compute/v2.1/servers/853637be-2161-4b9e-903a-2cbce6ff6881" status: 200 len: 1522 microversion: 2.1 time: 0.152009 Jul 16 21:59:51.561261 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3195/6396] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:59:51 2022] GET /compute/v2.1/servers/853637be-2161-4b9e-903a-2cbce6ff6881 => generated 1522 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:59:52.577885 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-dc2dee73-a76d-48c4-aed7-78d49a37f4e3 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:59:52.579137 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-dc2dee73-a76d-48c4-aed7-78d49a37f4e3 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] [instance: 853637be-2161-4b9e-903a-2cbce6ff6881] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:59:52.585670 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-dc2dee73-a76d-48c4-aed7-78d49a37f4e3 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:59:52.586141 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-dc2dee73-a76d-48c4-aed7-78d49a37f4e3 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:59:52.587191 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-dc2dee73-a76d-48c4-aed7-78d49a37f4e3 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:59:52.587671 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-dc2dee73-a76d-48c4-aed7-78d49a37f4e3 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:59:52.723145 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-dc2dee73-a76d-48c4-aed7-78d49a37f4e3 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd980641608134eadb72afa6fddb6adeb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9934bd813edb4e55a2d7c61207a9c46d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:59:52.730736 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-dc2dee73-a76d-48c4-aed7-78d49a37f4e3 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] 10.222.0.16 "GET /compute/v2.1/servers/853637be-2161-4b9e-903a-2cbce6ff6881" status: 200 len: 1522 microversion: 2.1 time: 0.156791 Jul 16 21:59:52.731343 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3202/6397] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:59:52 2022] GET /compute/v2.1/servers/853637be-2161-4b9e-903a-2cbce6ff6881 => generated 1522 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:59:53.664704 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-02b9a644-e07a-4372-a4f1-deaf88f58a36 req-043d59b7-cd82-4a0e-8339-6d89a46973e9 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "853637be-2161-4b9e-903a-2cbce6ff6881", "name": "network-vif-unplugged", "status": "completed", "tag": "341e460d-4246-420d-8082-44048820ac75"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:59:53.675438 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-02b9a644-e07a-4372-a4f1-deaf88f58a36 req-043d59b7-cd82-4a0e-8339-6d89a46973e9 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:59:53.678596 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-02b9a644-e07a-4372-a4f1-deaf88f58a36 req-043d59b7-cd82-4a0e-8339-6d89a46973e9 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:59:53.688609 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-02b9a644-e07a-4372-a4f1-deaf88f58a36 req-043d59b7-cd82-4a0e-8339-6d89a46973e9 service nova] Creating event network-vif-unplugged:341e460d-4246-420d-8082-44048820ac75 for instance 853637be-2161-4b9e-903a-2cbce6ff6881 on nch1-master-1 Jul 16 21:59:53.694002 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [req-02b9a644-e07a-4372-a4f1-deaf88f58a36 req-043d59b7-cd82-4a0e-8339-6d89a46973e9 service nova] Instance 853637be-2161-4b9e-903a-2cbce6ff6881 is migrating within the same cell, copying events to all relevant hosts: {'nch2-master-1', 'nch1-master-1'} {{(pid=138680) _get_relevant_hosts /opt/stack/nova/nova/compute/api.py:5807}} Jul 16 21:59:53.705252 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-02b9a644-e07a-4372-a4f1-deaf88f58a36 req-043d59b7-cd82-4a0e-8339-6d89a46973e9 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.044282 Jul 16 21:59:53.711535 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3196/6398] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:59:53 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 16 21:59:53.748523 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-c6126ac6-6061-44e7-91c5-2c648aeae695 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:59:53.749158 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-c6126ac6-6061-44e7-91c5-2c648aeae695 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] [instance: 853637be-2161-4b9e-903a-2cbce6ff6881] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:59:53.756918 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c6126ac6-6061-44e7-91c5-2c648aeae695 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:59:53.758402 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c6126ac6-6061-44e7-91c5-2c648aeae695 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:59:53.758402 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c6126ac6-6061-44e7-91c5-2c648aeae695 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:59:53.758603 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c6126ac6-6061-44e7-91c5-2c648aeae695 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:59:53.918380 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-c6126ac6-6061-44e7-91c5-2c648aeae695 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd980641608134eadb72afa6fddb6adeb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9934bd813edb4e55a2d7c61207a9c46d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:59:53.927173 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-c6126ac6-6061-44e7-91c5-2c648aeae695 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] 10.222.0.16 "GET /compute/v2.1/servers/853637be-2161-4b9e-903a-2cbce6ff6881" status: 200 len: 1521 microversion: 2.1 time: 0.182550 Jul 16 21:59:53.928151 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3203/6399] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:59:53 2022] GET /compute/v2.1/servers/853637be-2161-4b9e-903a-2cbce6ff6881 => generated 1521 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 21:59:54.947047 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-8caf5e98-d720-46b3-9b4e-6fba11f2f322 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:59:54.947778 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-8caf5e98-d720-46b3-9b4e-6fba11f2f322 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] [instance: 853637be-2161-4b9e-903a-2cbce6ff6881] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:59:54.955174 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8caf5e98-d720-46b3-9b4e-6fba11f2f322 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:59:54.955629 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8caf5e98-d720-46b3-9b4e-6fba11f2f322 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:59:54.956393 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8caf5e98-d720-46b3-9b4e-6fba11f2f322 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:59:54.956830 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8caf5e98-d720-46b3-9b4e-6fba11f2f322 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:59:55.092873 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-8caf5e98-d720-46b3-9b4e-6fba11f2f322 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd980641608134eadb72afa6fddb6adeb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9934bd813edb4e55a2d7c61207a9c46d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:59:55.101078 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-8caf5e98-d720-46b3-9b4e-6fba11f2f322 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] 10.222.0.16 "GET /compute/v2.1/servers/853637be-2161-4b9e-903a-2cbce6ff6881" status: 200 len: 1519 microversion: 2.1 time: 0.159554 Jul 16 21:59:55.101651 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3197/6400] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:59:54 2022] GET /compute/v2.1/servers/853637be-2161-4b9e-903a-2cbce6ff6881 => generated 1519 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:59:55.720280 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-80a75883-c6fd-406c-8fb6-9026d4a4171b req-ea49cd4f-0e53-463d-98f3-dd5cd299beed service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "853637be-2161-4b9e-903a-2cbce6ff6881", "tag": "341e460d-4246-420d-8082-44048820ac75"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:59:55.729146 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-80a75883-c6fd-406c-8fb6-9026d4a4171b req-ea49cd4f-0e53-463d-98f3-dd5cd299beed service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:59:55.729644 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-80a75883-c6fd-406c-8fb6-9026d4a4171b req-ea49cd4f-0e53-463d-98f3-dd5cd299beed service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:59:55.741596 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-80a75883-c6fd-406c-8fb6-9026d4a4171b req-ea49cd4f-0e53-463d-98f3-dd5cd299beed service nova] Creating event network-changed:341e460d-4246-420d-8082-44048820ac75 for instance 853637be-2161-4b9e-903a-2cbce6ff6881 on nch1-master-1 Jul 16 21:59:55.746871 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [req-80a75883-c6fd-406c-8fb6-9026d4a4171b req-ea49cd4f-0e53-463d-98f3-dd5cd299beed service nova] Instance 853637be-2161-4b9e-903a-2cbce6ff6881 is migrating within the same cell, copying events to all relevant hosts: {'nch2-master-1', 'nch1-master-1'} {{(pid=138681) _get_relevant_hosts /opt/stack/nova/nova/compute/api.py:5807}} Jul 16 21:59:55.758376 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-80a75883-c6fd-406c-8fb6-9026d4a4171b req-ea49cd4f-0e53-463d-98f3-dd5cd299beed service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.042181 Jul 16 21:59:55.764070 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3204/6401] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 21:59:55 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 21:59:56.118070 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-20ff3cb2-40d7-4d96-bfa2-2ded9c33dd09 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:59:56.119780 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-20ff3cb2-40d7-4d96-bfa2-2ded9c33dd09 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] [instance: 853637be-2161-4b9e-903a-2cbce6ff6881] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:59:56.127038 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-20ff3cb2-40d7-4d96-bfa2-2ded9c33dd09 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:59:56.127511 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-20ff3cb2-40d7-4d96-bfa2-2ded9c33dd09 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:59:56.128296 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-20ff3cb2-40d7-4d96-bfa2-2ded9c33dd09 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:59:56.128744 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-20ff3cb2-40d7-4d96-bfa2-2ded9c33dd09 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:59:56.342871 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-20ff3cb2-40d7-4d96-bfa2-2ded9c33dd09 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd980641608134eadb72afa6fddb6adeb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9934bd813edb4e55a2d7c61207a9c46d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:59:56.352730 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-20ff3cb2-40d7-4d96-bfa2-2ded9c33dd09 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] 10.222.0.16 "GET /compute/v2.1/servers/853637be-2161-4b9e-903a-2cbce6ff6881" status: 200 len: 1519 microversion: 2.1 time: 0.238671 Jul 16 21:59:56.353326 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3198/6402] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:59:56 2022] GET /compute/v2.1/servers/853637be-2161-4b9e-903a-2cbce6ff6881 => generated 1519 bytes in 240 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:59:57.372768 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-91ef1718-5f88-4fbf-823a-35fd6407acd0 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:59:57.373516 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-91ef1718-5f88-4fbf-823a-35fd6407acd0 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] [instance: 853637be-2161-4b9e-903a-2cbce6ff6881] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:59:57.380160 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-91ef1718-5f88-4fbf-823a-35fd6407acd0 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:59:57.380601 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-91ef1718-5f88-4fbf-823a-35fd6407acd0 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:59:57.381374 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-91ef1718-5f88-4fbf-823a-35fd6407acd0 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:59:57.381826 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-91ef1718-5f88-4fbf-823a-35fd6407acd0 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:59:57.512273 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-91ef1718-5f88-4fbf-823a-35fd6407acd0 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd980641608134eadb72afa6fddb6adeb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9934bd813edb4e55a2d7c61207a9c46d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:59:57.519938 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-91ef1718-5f88-4fbf-823a-35fd6407acd0 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] 10.222.0.16 "GET /compute/v2.1/servers/853637be-2161-4b9e-903a-2cbce6ff6881" status: 200 len: 1516 microversion: 2.1 time: 0.152318 Jul 16 21:59:57.520507 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3205/6403] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:59:57 2022] GET /compute/v2.1/servers/853637be-2161-4b9e-903a-2cbce6ff6881 => generated 1516 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:59:57.536547 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-618732b3-91e7-4812-8c80-5bc5ad4a239c tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Action: 'action', calling method: >, body: {"confirmResize": {}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 21:59:57.537111 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-618732b3-91e7-4812-8c80-5bc5ad4a239c tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] [instance: 853637be-2161-4b9e-903a-2cbce6ff6881] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:59:57.544034 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-618732b3-91e7-4812-8c80-5bc5ad4a239c tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:59:57.544484 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-618732b3-91e7-4812-8c80-5bc5ad4a239c tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:59:57.545248 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-618732b3-91e7-4812-8c80-5bc5ad4a239c tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:59:57.545680 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-618732b3-91e7-4812-8c80-5bc5ad4a239c tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:59:57.733655 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-618732b3-91e7-4812-8c80-5bc5ad4a239c tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] 10.222.0.16 "POST /compute/v2.1/servers/853637be-2161-4b9e-903a-2cbce6ff6881/action" status: 204 len: 0 microversion: 2.1 time: 0.200711 Jul 16 21:59:57.734441 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3199/6404] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 21:59:57 2022] POST /compute/v2.1/servers/853637be-2161-4b9e-903a-2cbce6ff6881/action => generated 0 bytes in 202 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Jul 16 21:59:57.742839 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-8967a007-6862-42f8-ac91-e2597249b346 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:59:57.743565 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-8967a007-6862-42f8-ac91-e2597249b346 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] [instance: 853637be-2161-4b9e-903a-2cbce6ff6881] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:59:57.750001 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8967a007-6862-42f8-ac91-e2597249b346 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:59:57.750479 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8967a007-6862-42f8-ac91-e2597249b346 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:59:57.751332 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8967a007-6862-42f8-ac91-e2597249b346 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:59:57.751789 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8967a007-6862-42f8-ac91-e2597249b346 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:59:57.894473 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-8967a007-6862-42f8-ac91-e2597249b346 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd980641608134eadb72afa6fddb6adeb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9934bd813edb4e55a2d7c61207a9c46d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:59:57.902375 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-8967a007-6862-42f8-ac91-e2597249b346 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] 10.222.0.16 "GET /compute/v2.1/servers/853637be-2161-4b9e-903a-2cbce6ff6881" status: 200 len: 1516 microversion: 2.1 time: 0.163512 Jul 16 21:59:57.902951 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3206/6405] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:59:57 2022] GET /compute/v2.1/servers/853637be-2161-4b9e-903a-2cbce6ff6881 => generated 1516 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 21:59:58.922479 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-05188625-6c8c-466c-90a8-aa7fdbb76025 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 21:59:58.923243 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-05188625-6c8c-466c-90a8-aa7fdbb76025 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] [instance: 853637be-2161-4b9e-903a-2cbce6ff6881] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 21:59:58.929689 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-05188625-6c8c-466c-90a8-aa7fdbb76025 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:59:58.930147 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-05188625-6c8c-466c-90a8-aa7fdbb76025 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:59:58.930905 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-05188625-6c8c-466c-90a8-aa7fdbb76025 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 21:59:58.931371 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-05188625-6c8c-466c-90a8-aa7fdbb76025 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 21:59:59.068518 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-05188625-6c8c-466c-90a8-aa7fdbb76025 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd980641608134eadb72afa6fddb6adeb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9934bd813edb4e55a2d7c61207a9c46d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 21:59:59.076359 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-05188625-6c8c-466c-90a8-aa7fdbb76025 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] 10.222.0.16 "GET /compute/v2.1/servers/853637be-2161-4b9e-903a-2cbce6ff6881" status: 200 len: 1516 microversion: 2.1 time: 0.159773 Jul 16 21:59:59.076984 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3200/6406] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 21:59:58 2022] GET /compute/v2.1/servers/853637be-2161-4b9e-903a-2cbce6ff6881 => generated 1516 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:00:00.094502 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-6412ed77-3653-4029-a2c0-542aceadbf05 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:00:00.095173 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-6412ed77-3653-4029-a2c0-542aceadbf05 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] [instance: 853637be-2161-4b9e-903a-2cbce6ff6881] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:00:00.102140 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6412ed77-3653-4029-a2c0-542aceadbf05 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:00:00.102600 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6412ed77-3653-4029-a2c0-542aceadbf05 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:00:00.103390 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6412ed77-3653-4029-a2c0-542aceadbf05 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:00:00.103838 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6412ed77-3653-4029-a2c0-542aceadbf05 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:00:00.236508 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-6412ed77-3653-4029-a2c0-542aceadbf05 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd980641608134eadb72afa6fddb6adeb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9934bd813edb4e55a2d7c61207a9c46d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:00:00.244071 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-6412ed77-3653-4029-a2c0-542aceadbf05 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] 10.222.0.16 "GET /compute/v2.1/servers/853637be-2161-4b9e-903a-2cbce6ff6881" status: 200 len: 1508 microversion: 2.1 time: 0.153551 Jul 16 22:00:00.244640 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3207/6407] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:00:00 2022] GET /compute/v2.1/servers/853637be-2161-4b9e-903a-2cbce6ff6881 => generated 1508 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 22:00:00.260868 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-21c70a72-0f0b-4e3f-81e8-8eb3253c9a3b tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:00:00.261556 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-21c70a72-0f0b-4e3f-81e8-8eb3253c9a3b tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] [instance: 853637be-2161-4b9e-903a-2cbce6ff6881] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:00:00.268546 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-21c70a72-0f0b-4e3f-81e8-8eb3253c9a3b tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:00:00.269010 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-21c70a72-0f0b-4e3f-81e8-8eb3253c9a3b tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:00:00.269774 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-21c70a72-0f0b-4e3f-81e8-8eb3253c9a3b tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:00:00.270258 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-21c70a72-0f0b-4e3f-81e8-8eb3253c9a3b tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:00:00.408579 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-21c70a72-0f0b-4e3f-81e8-8eb3253c9a3b tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd980641608134eadb72afa6fddb6adeb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9934bd813edb4e55a2d7c61207a9c46d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:00:00.416468 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-21c70a72-0f0b-4e3f-81e8-8eb3253c9a3b tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] 10.222.0.16 "GET /compute/v2.1/servers/853637be-2161-4b9e-903a-2cbce6ff6881" status: 200 len: 1508 microversion: 2.1 time: 0.159517 Jul 16 22:00:00.417043 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3201/6408] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:00:00 2022] GET /compute/v2.1/servers/853637be-2161-4b9e-903a-2cbce6ff6881 => generated 1508 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:00:00.436143 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-056de1d7-38b8-494f-8887-b32de6444e8d tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Action: 'create', calling method: .version_select at 0x7fdd8ff69e50>, body: {"keypair": {"name": "tempest-keypair-1008228667", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBLih1w1S7YAzRYo8te82q2n8//IDurKuF+LqAozLqveo25ZSLdRNc5fJwyvVnHnULVKybkwx4PnbKSiT/6kNqQXVJAqCXa56BXJiYMTW8jHglS/HRr6p/G37N30B+68stg=="}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:00:00.451437 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-056de1d7-38b8-494f-8887-b32de6444e8d tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Getting quotas for project 9934bd813edb4e55a2d7c61207a9c46d. Resources: {'key_pairs'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 22:00:00.457660 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-056de1d7-38b8-494f-8887-b32de6444e8d tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Getting quotas for user d980641608134eadb72afa6fddb6adeb and project 9934bd813edb4e55a2d7c61207a9c46d. Resources: {'key_pairs'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 22:00:00.553281 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-056de1d7-38b8-494f-8887-b32de6444e8d tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] 10.222.0.16 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.121063 Jul 16 22:00:00.559188 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3208/6409] 10.222.0.16 () {60 vars in 1158 bytes} [Sat Jul 16 22:00:00 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 122 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 16 22:00:00.560106 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-88f6de54-7a3d-40a6-beee-659ce8e32ed0 req-341d5e19-49ed-4346-bb5c-6b9610994766 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "853637be-2161-4b9e-903a-2cbce6ff6881", "name": "network-vif-plugged", "status": "completed", "tag": "341e460d-4246-420d-8082-44048820ac75"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:00:00.570260 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-88f6de54-7a3d-40a6-beee-659ce8e32ed0 req-341d5e19-49ed-4346-bb5c-6b9610994766 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:00:00.571050 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-88f6de54-7a3d-40a6-beee-659ce8e32ed0 req-341d5e19-49ed-4346-bb5c-6b9610994766 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:00:00.583763 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-88f6de54-7a3d-40a6-beee-659ce8e32ed0 req-341d5e19-49ed-4346-bb5c-6b9610994766 service nova] Creating event network-vif-plugged:341e460d-4246-420d-8082-44048820ac75 for instance 853637be-2161-4b9e-903a-2cbce6ff6881 on nch1-master-1 Jul 16 22:00:00.589324 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-88f6de54-7a3d-40a6-beee-659ce8e32ed0 req-341d5e19-49ed-4346-bb5c-6b9610994766 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.033538 Jul 16 22:00:00.590947 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3202/6410] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 22:00:00 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 16 22:00:02.807254 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-5e2c7b35-bb6e-4723-a193-bc6ce50f6d1f tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:00:02.809817 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-5e2c7b35-bb6e-4723-a193-bc6ce50f6d1f tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] [instance: 853637be-2161-4b9e-903a-2cbce6ff6881] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:00:02.814881 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5e2c7b35-bb6e-4723-a193-bc6ce50f6d1f tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:00:02.815377 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5e2c7b35-bb6e-4723-a193-bc6ce50f6d1f tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:00:02.816157 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5e2c7b35-bb6e-4723-a193-bc6ce50f6d1f tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:00:02.816616 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5e2c7b35-bb6e-4723-a193-bc6ce50f6d1f tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:00:02.949289 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-5e2c7b35-bb6e-4723-a193-bc6ce50f6d1f tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd980641608134eadb72afa6fddb6adeb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9934bd813edb4e55a2d7c61207a9c46d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:00:02.958041 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-5e2c7b35-bb6e-4723-a193-bc6ce50f6d1f tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] 10.222.0.16 "GET /compute/v2.1/servers/853637be-2161-4b9e-903a-2cbce6ff6881" status: 200 len: 1508 microversion: 2.1 time: 0.155697 Jul 16 22:00:02.958608 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3209/6411] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:00:02 2022] GET /compute/v2.1/servers/853637be-2161-4b9e-903a-2cbce6ff6881 => generated 1508 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:00:02.975409 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-cc24e90a-dd9b-4d3d-9cb8-0e35f9d663aa tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:00:02.975998 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-cc24e90a-dd9b-4d3d-9cb8-0e35f9d663aa tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] [instance: 853637be-2161-4b9e-903a-2cbce6ff6881] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:00:02.983340 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cc24e90a-dd9b-4d3d-9cb8-0e35f9d663aa tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:00:02.983960 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cc24e90a-dd9b-4d3d-9cb8-0e35f9d663aa tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:00:02.984807 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cc24e90a-dd9b-4d3d-9cb8-0e35f9d663aa tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:00:02.985254 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cc24e90a-dd9b-4d3d-9cb8-0e35f9d663aa tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:00:03.010016 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-cc24e90a-dd9b-4d3d-9cb8-0e35f9d663aa tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] [instance: 853637be-2161-4b9e-903a-2cbce6ff6881] Going to try to terminate instance {{(pid=138680) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 16 22:00:03.294233 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-cc24e90a-dd9b-4d3d-9cb8-0e35f9d663aa tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] 10.222.0.16 "DELETE /compute/v2.1/servers/853637be-2161-4b9e-903a-2cbce6ff6881" status: 204 len: 0 microversion: 2.1 time: 0.322653 Jul 16 22:00:03.294785 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3203/6412] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 22:00:02 2022] DELETE /compute/v2.1/servers/853637be-2161-4b9e-903a-2cbce6ff6881 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 16 22:00:03.353897 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-451db02e-0231-4a07-aa70-ae36b0532cb5 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerDiskConfigTestJSON-server-656889101", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452", "networks": [{"uuid": "9b7d6baa-385f-4a11-9e81-1d39ce412e76"}]}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:00:03.404301 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.network.neutron [None req-451db02e-0231-4a07-aa70-ae36b0532cb5 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] validate_networks() for [('9b7d6baa-385f-4a11-9e81-1d39ce412e76', None, None, None, None, None)] {{(pid=138681) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 16 22:00:03.770690 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-451db02e-0231-4a07-aa70-ae36b0532cb5 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Flavor limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 16 22:00:03.771174 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-451db02e-0231-4a07-aa70-ae36b0532cb5 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Image limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 16 22:00:03.771668 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-451db02e-0231-4a07-aa70-ae36b0532cb5 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Flavor pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 16 22:00:03.772091 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-451db02e-0231-4a07-aa70-ae36b0532cb5 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Image pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 16 22:00:03.772510 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-451db02e-0231-4a07-aa70-ae36b0532cb5 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 16 22:00:03.781357 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-451db02e-0231-4a07-aa70-ae36b0532cb5 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:00:03.781815 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-451db02e-0231-4a07-aa70-ae36b0532cb5 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:00:03.782610 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-451db02e-0231-4a07-aa70-ae36b0532cb5 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:00:03.783092 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-451db02e-0231-4a07-aa70-ae36b0532cb5 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:00:03.800818 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-451db02e-0231-4a07-aa70-ae36b0532cb5 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Getting quotas for project 9934bd813edb4e55a2d7c61207a9c46d. Resources: {'ram', 'instances', 'cores'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 22:00:03.806077 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-451db02e-0231-4a07-aa70-ae36b0532cb5 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Getting quotas for user d980641608134eadb72afa6fddb6adeb and project 9934bd813edb4e55a2d7c61207a9c46d. Resources: {'ram', 'instances', 'cores'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 22:00:03.818964 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-451db02e-0231-4a07-aa70-ae36b0532cb5 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Going to run 1 instances... {{(pid=138681) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 16 22:00:03.821277 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-451db02e-0231-4a07-aa70-ae36b0532cb5 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] [instance: 96ec3b45-063c-4bc9-84e5-e59532a23783] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138681) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 16 22:00:03.926763 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-451db02e-0231-4a07-aa70-ae36b0532cb5 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.576662 Jul 16 22:00:03.927530 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3210/6413] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 22:00:03 2022] POST /compute/v2.1/servers => generated 380 bytes in 578 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 16 22:00:03.939134 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-64bc8261-7695-41a2-bbf4-a4cf63f41737 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:00:03.939968 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-64bc8261-7695-41a2-bbf4-a4cf63f41737 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] [instance: 96ec3b45-063c-4bc9-84e5-e59532a23783] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:00:03.998575 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-64bc8261-7695-41a2-bbf4-a4cf63f41737 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd980641608134eadb72afa6fddb6adeb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9934bd813edb4e55a2d7c61207a9c46d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:00:04.007093 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-64bc8261-7695-41a2-bbf4-a4cf63f41737 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] 10.222.0.16 "GET /compute/v2.1/servers/96ec3b45-063c-4bc9-84e5-e59532a23783" status: 200 len: 1217 microversion: 2.1 time: 0.073162 Jul 16 22:00:04.007674 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3204/6414] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:00:03 2022] GET /compute/v2.1/servers/96ec3b45-063c-4bc9-84e5-e59532a23783 => generated 1217 bytes in 75 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:00:05.027183 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-ea277d10-5f1d-415e-8c0b-1635952ba3a7 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:00:05.028389 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-ea277d10-5f1d-415e-8c0b-1635952ba3a7 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] [instance: 96ec3b45-063c-4bc9-84e5-e59532a23783] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:00:05.035505 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ea277d10-5f1d-415e-8c0b-1635952ba3a7 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:00:05.036045 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ea277d10-5f1d-415e-8c0b-1635952ba3a7 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:00:05.036953 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ea277d10-5f1d-415e-8c0b-1635952ba3a7 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:00:05.037474 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ea277d10-5f1d-415e-8c0b-1635952ba3a7 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:00:05.111713 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-ea277d10-5f1d-415e-8c0b-1635952ba3a7 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd980641608134eadb72afa6fddb6adeb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9934bd813edb4e55a2d7c61207a9c46d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:00:05.113105 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-ad18ead1-f4c4-4e1c-8f3b-07196f48c125 req-0f477fb0-d926-4b21-91b9-19029ca4c6cb service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "853637be-2161-4b9e-903a-2cbce6ff6881", "name": "network-vif-deleted", "tag": "341e460d-4246-420d-8082-44048820ac75"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:00:05.120062 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-ea277d10-5f1d-415e-8c0b-1635952ba3a7 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] 10.222.0.16 "GET /compute/v2.1/servers/96ec3b45-063c-4bc9-84e5-e59532a23783" status: 200 len: 1221 microversion: 2.1 time: 0.099003 Jul 16 22:00:05.124942 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3211/6415] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:00:05 2022] GET /compute/v2.1/servers/96ec3b45-063c-4bc9-84e5-e59532a23783 => generated 1221 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:00:05.126695 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-ad18ead1-f4c4-4e1c-8f3b-07196f48c125 req-0f477fb0-d926-4b21-91b9-19029ca4c6cb service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:00:05.127188 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-ad18ead1-f4c4-4e1c-8f3b-07196f48c125 req-0f477fb0-d926-4b21-91b9-19029ca4c6cb service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:00:05.138954 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-ad18ead1-f4c4-4e1c-8f3b-07196f48c125 req-0f477fb0-d926-4b21-91b9-19029ca4c6cb service nova] Creating event network-vif-deleted:341e460d-4246-420d-8082-44048820ac75 for instance 853637be-2161-4b9e-903a-2cbce6ff6881 on nch1-master-1 Jul 16 22:00:05.143881 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-ad18ead1-f4c4-4e1c-8f3b-07196f48c125 req-0f477fb0-d926-4b21-91b9-19029ca4c6cb service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.035345 Jul 16 22:00:05.145029 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3205/6416] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 22:00:05 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 16 22:00:06.136637 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-a52b3f73-ced1-4956-b9e3-5909314937a9 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:00:06.137278 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-a52b3f73-ced1-4956-b9e3-5909314937a9 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] [instance: 96ec3b45-063c-4bc9-84e5-e59532a23783] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:00:06.145126 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a52b3f73-ced1-4956-b9e3-5909314937a9 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:00:06.145126 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a52b3f73-ced1-4956-b9e3-5909314937a9 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:00:06.145785 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a52b3f73-ced1-4956-b9e3-5909314937a9 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:00:06.147665 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a52b3f73-ced1-4956-b9e3-5909314937a9 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:00:06.212145 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-a52b3f73-ced1-4956-b9e3-5909314937a9 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd980641608134eadb72afa6fddb6adeb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9934bd813edb4e55a2d7c61207a9c46d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:00:06.219927 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-a52b3f73-ced1-4956-b9e3-5909314937a9 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] 10.222.0.16 "GET /compute/v2.1/servers/96ec3b45-063c-4bc9-84e5-e59532a23783" status: 200 len: 1287 microversion: 2.1 time: 0.087154 Jul 16 22:00:06.220502 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3212/6417] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:00:06 2022] GET /compute/v2.1/servers/96ec3b45-063c-4bc9-84e5-e59532a23783 => generated 1287 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:00:07.241063 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-fdf5c859-b9b9-454c-aef9-e9e8b08fd233 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:00:07.241866 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-fdf5c859-b9b9-454c-aef9-e9e8b08fd233 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] [instance: 96ec3b45-063c-4bc9-84e5-e59532a23783] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:00:07.249255 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-fdf5c859-b9b9-454c-aef9-e9e8b08fd233 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:00:07.250077 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-fdf5c859-b9b9-454c-aef9-e9e8b08fd233 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:00:07.251651 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-fdf5c859-b9b9-454c-aef9-e9e8b08fd233 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:00:07.252216 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-fdf5c859-b9b9-454c-aef9-e9e8b08fd233 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:00:07.274611 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=96,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:00:07.405402 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-fdf5c859-b9b9-454c-aef9-e9e8b08fd233 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd980641608134eadb72afa6fddb6adeb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9934bd813edb4e55a2d7c61207a9c46d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:00:07.414774 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-fdf5c859-b9b9-454c-aef9-e9e8b08fd233 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] 10.222.0.16 "GET /compute/v2.1/servers/96ec3b45-063c-4bc9-84e5-e59532a23783" status: 200 len: 1317 microversion: 2.1 time: 0.180235 Jul 16 22:00:07.415559 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3206/6418] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:00:07 2022] GET /compute/v2.1/servers/96ec3b45-063c-4bc9-84e5-e59532a23783 => generated 1317 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:00:07.538227 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-c38a21fe-b720-4073-a1d3-e9fec9599e00 req-ea46fb8e-8a2c-4dae-bf69-353da17ca5c6 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "96ec3b45-063c-4bc9-84e5-e59532a23783", "tag": "2bcfc300-6586-4eae-a738-286a969a6f27"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:00:07.547773 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-c38a21fe-b720-4073-a1d3-e9fec9599e00 req-ea46fb8e-8a2c-4dae-bf69-353da17ca5c6 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:00:07.548236 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-c38a21fe-b720-4073-a1d3-e9fec9599e00 req-ea46fb8e-8a2c-4dae-bf69-353da17ca5c6 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:00:07.561021 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-c38a21fe-b720-4073-a1d3-e9fec9599e00 req-ea46fb8e-8a2c-4dae-bf69-353da17ca5c6 service nova] Creating event network-changed:2bcfc300-6586-4eae-a738-286a969a6f27 for instance 96ec3b45-063c-4bc9-84e5-e59532a23783 on nch1-master-1 Jul 16 22:00:07.565938 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-c38a21fe-b720-4073-a1d3-e9fec9599e00 req-ea46fb8e-8a2c-4dae-bf69-353da17ca5c6 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.032456 Jul 16 22:00:07.567934 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3213/6419] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 22:00:07 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 22:00:08.432249 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-2f859c00-b42c-476c-992f-26957566748a tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:00:08.432899 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-2f859c00-b42c-476c-992f-26957566748a tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] [instance: 96ec3b45-063c-4bc9-84e5-e59532a23783] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:00:08.439876 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2f859c00-b42c-476c-992f-26957566748a tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:00:08.443226 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2f859c00-b42c-476c-992f-26957566748a tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:00:08.443226 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2f859c00-b42c-476c-992f-26957566748a tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:00:08.443226 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2f859c00-b42c-476c-992f-26957566748a tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:00:08.579410 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-2f859c00-b42c-476c-992f-26957566748a tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd980641608134eadb72afa6fddb6adeb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9934bd813edb4e55a2d7c61207a9c46d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:00:08.588174 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-2f859c00-b42c-476c-992f-26957566748a tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] 10.222.0.16 "GET /compute/v2.1/servers/96ec3b45-063c-4bc9-84e5-e59532a23783" status: 200 len: 1317 microversion: 2.1 time: 0.160112 Jul 16 22:00:08.588756 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3207/6420] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:00:08 2022] GET /compute/v2.1/servers/96ec3b45-063c-4bc9-84e5-e59532a23783 => generated 1317 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:00:09.606283 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-35e64ee9-7d2b-44a2-807a-96d9e34ecc43 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:00:09.606993 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-35e64ee9-7d2b-44a2-807a-96d9e34ecc43 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] [instance: 96ec3b45-063c-4bc9-84e5-e59532a23783] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:00:09.613568 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-35e64ee9-7d2b-44a2-807a-96d9e34ecc43 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:00:09.614028 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-35e64ee9-7d2b-44a2-807a-96d9e34ecc43 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:00:09.614793 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-35e64ee9-7d2b-44a2-807a-96d9e34ecc43 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:00:09.615285 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-35e64ee9-7d2b-44a2-807a-96d9e34ecc43 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:00:09.745777 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-35e64ee9-7d2b-44a2-807a-96d9e34ecc43 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd980641608134eadb72afa6fddb6adeb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9934bd813edb4e55a2d7c61207a9c46d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:00:09.753236 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-35e64ee9-7d2b-44a2-807a-96d9e34ecc43 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] 10.222.0.16 "GET /compute/v2.1/servers/96ec3b45-063c-4bc9-84e5-e59532a23783" status: 200 len: 1317 microversion: 2.1 time: 0.152075 Jul 16 22:00:09.753957 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3214/6421] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:00:09 2022] GET /compute/v2.1/servers/96ec3b45-063c-4bc9-84e5-e59532a23783 => generated 1317 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:00:10.770343 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-358ecdfe-aeff-478a-83b6-716ffbcddad6 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:00:10.770959 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-358ecdfe-aeff-478a-83b6-716ffbcddad6 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] [instance: 96ec3b45-063c-4bc9-84e5-e59532a23783] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:00:10.773766 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_api:SELECT=43,nova_api:UPDATE=1 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:00:10.778542 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-358ecdfe-aeff-478a-83b6-716ffbcddad6 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:00:10.778992 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-358ecdfe-aeff-478a-83b6-716ffbcddad6 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:00:10.779780 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-358ecdfe-aeff-478a-83b6-716ffbcddad6 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:00:10.780221 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-358ecdfe-aeff-478a-83b6-716ffbcddad6 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:00:11.236561 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-358ecdfe-aeff-478a-83b6-716ffbcddad6 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd980641608134eadb72afa6fddb6adeb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9934bd813edb4e55a2d7c61207a9c46d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:00:11.244093 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-358ecdfe-aeff-478a-83b6-716ffbcddad6 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] 10.222.0.16 "GET /compute/v2.1/servers/96ec3b45-063c-4bc9-84e5-e59532a23783" status: 200 len: 1317 microversion: 2.1 time: 0.477696 Jul 16 22:00:11.244667 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3208/6422] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:00:10 2022] GET /compute/v2.1/servers/96ec3b45-063c-4bc9-84e5-e59532a23783 => generated 1317 bytes in 479 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:00:12.260722 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-cf5aa80c-0408-47aa-a53f-f2250cf76df8 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:00:12.261428 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-cf5aa80c-0408-47aa-a53f-f2250cf76df8 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] [instance: 96ec3b45-063c-4bc9-84e5-e59532a23783] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:00:12.264416 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_api:SELECT=56,nova_api:INSERT=4 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:00:12.269388 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cf5aa80c-0408-47aa-a53f-f2250cf76df8 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:00:12.269912 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cf5aa80c-0408-47aa-a53f-f2250cf76df8 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:00:12.270742 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cf5aa80c-0408-47aa-a53f-f2250cf76df8 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:00:12.271302 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cf5aa80c-0408-47aa-a53f-f2250cf76df8 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:00:12.273976 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=89 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:00:12.407038 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-cf5aa80c-0408-47aa-a53f-f2250cf76df8 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd980641608134eadb72afa6fddb6adeb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9934bd813edb4e55a2d7c61207a9c46d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:00:12.416176 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-cf5aa80c-0408-47aa-a53f-f2250cf76df8 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] 10.222.0.16 "GET /compute/v2.1/servers/96ec3b45-063c-4bc9-84e5-e59532a23783" status: 200 len: 1317 microversion: 2.1 time: 0.159401 Jul 16 22:00:12.417360 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3215/6423] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:00:12 2022] GET /compute/v2.1/servers/96ec3b45-063c-4bc9-84e5-e59532a23783 => generated 1317 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:00:13.433329 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-2dd694b6-9e2b-4e37-9a3a-14eca3174c17 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:00:13.433955 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-2dd694b6-9e2b-4e37-9a3a-14eca3174c17 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] [instance: 96ec3b45-063c-4bc9-84e5-e59532a23783] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:00:13.441014 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2dd694b6-9e2b-4e37-9a3a-14eca3174c17 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:00:13.441472 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2dd694b6-9e2b-4e37-9a3a-14eca3174c17 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:00:13.442963 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2dd694b6-9e2b-4e37-9a3a-14eca3174c17 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:00:13.443441 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2dd694b6-9e2b-4e37-9a3a-14eca3174c17 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:00:13.573139 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-2dd694b6-9e2b-4e37-9a3a-14eca3174c17 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd980641608134eadb72afa6fddb6adeb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9934bd813edb4e55a2d7c61207a9c46d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:00:13.580763 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-2dd694b6-9e2b-4e37-9a3a-14eca3174c17 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] 10.222.0.16 "GET /compute/v2.1/servers/96ec3b45-063c-4bc9-84e5-e59532a23783" status: 200 len: 1317 microversion: 2.1 time: 0.151443 Jul 16 22:00:13.581390 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3209/6424] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:00:13 2022] GET /compute/v2.1/servers/96ec3b45-063c-4bc9-84e5-e59532a23783 => generated 1317 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:00:14.598501 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-04c74410-96e1-4347-92b5-e65613598c8b tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:00:14.599219 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-04c74410-96e1-4347-92b5-e65613598c8b tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] [instance: 96ec3b45-063c-4bc9-84e5-e59532a23783] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:00:14.601955 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell0:SELECT=2 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:00:14.602693 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell0:SELECT=10 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:00:14.607726 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-04c74410-96e1-4347-92b5-e65613598c8b tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:00:14.608195 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-04c74410-96e1-4347-92b5-e65613598c8b tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:00:14.608999 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-04c74410-96e1-4347-92b5-e65613598c8b tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:00:14.609454 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-04c74410-96e1-4347-92b5-e65613598c8b tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:00:14.623367 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-7bfd67ab-684f-43ff-b5fa-0b6a8d684875 req-578ebd05-7b89-48a1-808a-4596f72da513 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "96ec3b45-063c-4bc9-84e5-e59532a23783", "name": "network-vif-plugged", "status": "completed", "tag": "2bcfc300-6586-4eae-a738-286a969a6f27"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:00:14.628722 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell0:SELECT=2 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:00:14.633921 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-7bfd67ab-684f-43ff-b5fa-0b6a8d684875 req-578ebd05-7b89-48a1-808a-4596f72da513 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:00:14.634372 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-7bfd67ab-684f-43ff-b5fa-0b6a8d684875 req-578ebd05-7b89-48a1-808a-4596f72da513 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:00:14.649354 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-7bfd67ab-684f-43ff-b5fa-0b6a8d684875 req-578ebd05-7b89-48a1-808a-4596f72da513 service nova] Creating event network-vif-plugged:2bcfc300-6586-4eae-a738-286a969a6f27 for instance 96ec3b45-063c-4bc9-84e5-e59532a23783 on nch1-master-1 Jul 16 22:00:14.650768 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-7bfd67ab-684f-43ff-b5fa-0b6a8d684875 req-578ebd05-7b89-48a1-808a-4596f72da513 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.031163 Jul 16 22:00:14.651409 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3210/6425] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 22:00:14 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 16 22:00:14.750083 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-04c74410-96e1-4347-92b5-e65613598c8b tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd980641608134eadb72afa6fddb6adeb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9934bd813edb4e55a2d7c61207a9c46d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:00:14.760707 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-04c74410-96e1-4347-92b5-e65613598c8b tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] 10.222.0.16 "GET /compute/v2.1/servers/96ec3b45-063c-4bc9-84e5-e59532a23783" status: 200 len: 1317 microversion: 2.1 time: 0.166648 Jul 16 22:00:14.761332 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3216/6426] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:00:14 2022] GET /compute/v2.1/servers/96ec3b45-063c-4bc9-84e5-e59532a23783 => generated 1317 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:00:15.779568 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-6b2e66d9-c9b3-45d8-99bb-c5d6d883866e tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:00:15.781007 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-6b2e66d9-c9b3-45d8-99bb-c5d6d883866e tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] [instance: 96ec3b45-063c-4bc9-84e5-e59532a23783] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:00:15.787700 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6b2e66d9-c9b3-45d8-99bb-c5d6d883866e tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:00:15.788154 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6b2e66d9-c9b3-45d8-99bb-c5d6d883866e tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:00:15.788929 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6b2e66d9-c9b3-45d8-99bb-c5d6d883866e tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:00:15.789417 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6b2e66d9-c9b3-45d8-99bb-c5d6d883866e tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:00:15.927028 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-6b2e66d9-c9b3-45d8-99bb-c5d6d883866e tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd980641608134eadb72afa6fddb6adeb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9934bd813edb4e55a2d7c61207a9c46d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:00:15.934920 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-6b2e66d9-c9b3-45d8-99bb-c5d6d883866e tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] 10.222.0.16 "GET /compute/v2.1/servers/96ec3b45-063c-4bc9-84e5-e59532a23783" status: 200 len: 1317 microversion: 2.1 time: 0.160960 Jul 16 22:00:15.937287 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3211/6427] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:00:15 2022] GET /compute/v2.1/servers/96ec3b45-063c-4bc9-84e5-e59532a23783 => generated 1317 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:00:16.952529 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-1e1b883d-3f6e-40ea-b1e0-cb191893927a tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:00:16.953154 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-1e1b883d-3f6e-40ea-b1e0-cb191893927a tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] [instance: 96ec3b45-063c-4bc9-84e5-e59532a23783] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:00:16.961785 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1e1b883d-3f6e-40ea-b1e0-cb191893927a tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:00:16.962247 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1e1b883d-3f6e-40ea-b1e0-cb191893927a tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:00:16.963041 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1e1b883d-3f6e-40ea-b1e0-cb191893927a tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:00:16.963501 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1e1b883d-3f6e-40ea-b1e0-cb191893927a tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:00:17.096153 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-1e1b883d-3f6e-40ea-b1e0-cb191893927a tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd980641608134eadb72afa6fddb6adeb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9934bd813edb4e55a2d7c61207a9c46d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:00:17.103798 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-1e1b883d-3f6e-40ea-b1e0-cb191893927a tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] 10.222.0.16 "GET /compute/v2.1/servers/96ec3b45-063c-4bc9-84e5-e59532a23783" status: 200 len: 1507 microversion: 2.1 time: 0.155403 Jul 16 22:00:17.104374 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3217/6428] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:00:16 2022] GET /compute/v2.1/servers/96ec3b45-063c-4bc9-84e5-e59532a23783 => generated 1507 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:00:17.120543 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-83620386-91a7-4d3c-9a13-8b16f1fb816c tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:00:17.121093 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-83620386-91a7-4d3c-9a13-8b16f1fb816c tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] [instance: 96ec3b45-063c-4bc9-84e5-e59532a23783] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:00:17.127994 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-83620386-91a7-4d3c-9a13-8b16f1fb816c tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:00:17.128450 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-83620386-91a7-4d3c-9a13-8b16f1fb816c tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:00:17.129802 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-83620386-91a7-4d3c-9a13-8b16f1fb816c tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:00:17.130248 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-83620386-91a7-4d3c-9a13-8b16f1fb816c tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:00:17.261173 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-83620386-91a7-4d3c-9a13-8b16f1fb816c tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd980641608134eadb72afa6fddb6adeb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9934bd813edb4e55a2d7c61207a9c46d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:00:17.269134 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-83620386-91a7-4d3c-9a13-8b16f1fb816c tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] 10.222.0.16 "GET /compute/v2.1/servers/96ec3b45-063c-4bc9-84e5-e59532a23783" status: 200 len: 1507 microversion: 2.1 time: 0.152488 Jul 16 22:00:17.269771 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3212/6429] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:00:17 2022] GET /compute/v2.1/servers/96ec3b45-063c-4bc9-84e5-e59532a23783 => generated 1507 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 22:00:17.285647 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-1acfc018-f861-48b7-947c-c6fe2c49980d tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Action: 'action', calling method: >, body: {"resize": {"flavorRef": "451", "OS-DCF:diskConfig": "AUTO"}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:00:17.286744 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-1acfc018-f861-48b7-947c-c6fe2c49980d tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] [instance: 96ec3b45-063c-4bc9-84e5-e59532a23783] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:00:17.292866 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1acfc018-f861-48b7-947c-c6fe2c49980d tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:00:17.293328 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1acfc018-f861-48b7-947c-c6fe2c49980d tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:00:17.296316 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1acfc018-f861-48b7-947c-c6fe2c49980d tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:00:17.296316 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1acfc018-f861-48b7-947c-c6fe2c49980d tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:00:17.323331 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-1acfc018-f861-48b7-947c-c6fe2c49980d tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Policy check for compute:servers:resize:cross_cell failed with credentials {'is_admin': False, 'user_id': 'd980641608134eadb72afa6fddb6adeb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9934bd813edb4e55a2d7c61207a9c46d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:00:17.329182 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-1acfc018-f861-48b7-947c-c6fe2c49980d tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] [instance: 96ec3b45-063c-4bc9-84e5-e59532a23783] Old instance type m1.heat2, new instance type m1.heat {{(pid=138681) resize /opt/stack/nova/nova/compute/api.py:4166}} Jul 16 22:00:17.335341 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-1acfc018-f861-48b7-947c-c6fe2c49980d tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Flavor limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 16 22:00:17.335777 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-1acfc018-f861-48b7-947c-c6fe2c49980d tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Image limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 16 22:00:17.336256 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-1acfc018-f861-48b7-947c-c6fe2c49980d tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Flavor pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 16 22:00:17.336979 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-1acfc018-f861-48b7-947c-c6fe2c49980d tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Image pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 16 22:00:17.337396 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-1acfc018-f861-48b7-947c-c6fe2c49980d tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 16 22:00:17.553477 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-1acfc018-f861-48b7-947c-c6fe2c49980d tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] 10.222.0.16 "POST /compute/v2.1/servers/96ec3b45-063c-4bc9-84e5-e59532a23783/action" status: 202 len: 0 microversion: 2.1 time: 0.267018 Jul 16 22:00:17.553477 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3218/6430] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 22:00:17 2022] POST /compute/v2.1/servers/96ec3b45-063c-4bc9-84e5-e59532a23783/action => generated 0 bytes in 268 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Jul 16 22:00:17.558136 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-48ce7616-8929-4679-b9a8-16f4761a1690 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:00:17.558753 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-48ce7616-8929-4679-b9a8-16f4761a1690 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] [instance: 96ec3b45-063c-4bc9-84e5-e59532a23783] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:00:17.564895 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-48ce7616-8929-4679-b9a8-16f4761a1690 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:00:17.565434 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-48ce7616-8929-4679-b9a8-16f4761a1690 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:00:17.566199 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-48ce7616-8929-4679-b9a8-16f4761a1690 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:00:17.566638 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-48ce7616-8929-4679-b9a8-16f4761a1690 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:00:17.700119 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-48ce7616-8929-4679-b9a8-16f4761a1690 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd980641608134eadb72afa6fddb6adeb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9934bd813edb4e55a2d7c61207a9c46d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:00:17.707793 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-48ce7616-8929-4679-b9a8-16f4761a1690 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] 10.222.0.16 "GET /compute/v2.1/servers/96ec3b45-063c-4bc9-84e5-e59532a23783" status: 200 len: 1514 microversion: 2.1 time: 0.153512 Jul 16 22:00:17.708378 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3213/6431] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:00:17 2022] GET /compute/v2.1/servers/96ec3b45-063c-4bc9-84e5-e59532a23783 => generated 1514 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:00:18.726145 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-ca385b25-46a0-46b0-8719-c524f9201352 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:00:18.727404 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-ca385b25-46a0-46b0-8719-c524f9201352 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] [instance: 96ec3b45-063c-4bc9-84e5-e59532a23783] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:00:18.733608 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ca385b25-46a0-46b0-8719-c524f9201352 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:00:18.734077 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ca385b25-46a0-46b0-8719-c524f9201352 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:00:18.734860 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ca385b25-46a0-46b0-8719-c524f9201352 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:00:18.735353 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ca385b25-46a0-46b0-8719-c524f9201352 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:00:18.867574 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-ca385b25-46a0-46b0-8719-c524f9201352 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd980641608134eadb72afa6fddb6adeb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9934bd813edb4e55a2d7c61207a9c46d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:00:18.875702 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-ca385b25-46a0-46b0-8719-c524f9201352 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] 10.222.0.16 "GET /compute/v2.1/servers/96ec3b45-063c-4bc9-84e5-e59532a23783" status: 200 len: 1514 microversion: 2.1 time: 0.155131 Jul 16 22:00:18.876329 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3219/6432] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:00:18 2022] GET /compute/v2.1/servers/96ec3b45-063c-4bc9-84e5-e59532a23783 => generated 1514 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:00:19.893758 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-c50d5cfe-d4d1-465f-83e4-ee5a1b0fcd6b tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:00:19.894395 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-c50d5cfe-d4d1-465f-83e4-ee5a1b0fcd6b tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] [instance: 96ec3b45-063c-4bc9-84e5-e59532a23783] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:00:19.904053 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c50d5cfe-d4d1-465f-83e4-ee5a1b0fcd6b tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:00:19.904053 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c50d5cfe-d4d1-465f-83e4-ee5a1b0fcd6b tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:00:19.904053 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c50d5cfe-d4d1-465f-83e4-ee5a1b0fcd6b tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:00:19.904053 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c50d5cfe-d4d1-465f-83e4-ee5a1b0fcd6b tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:00:20.045239 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-c50d5cfe-d4d1-465f-83e4-ee5a1b0fcd6b tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd980641608134eadb72afa6fddb6adeb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9934bd813edb4e55a2d7c61207a9c46d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:00:20.054118 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-c50d5cfe-d4d1-465f-83e4-ee5a1b0fcd6b tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] 10.222.0.16 "GET /compute/v2.1/servers/96ec3b45-063c-4bc9-84e5-e59532a23783" status: 200 len: 1514 microversion: 2.1 time: 0.164247 Jul 16 22:00:20.054703 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3214/6433] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:00:19 2022] GET /compute/v2.1/servers/96ec3b45-063c-4bc9-84e5-e59532a23783 => generated 1514 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:00:21.071961 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-a5586143-614d-4e23-bcc8-93f7ab71b676 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:00:21.072588 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-a5586143-614d-4e23-bcc8-93f7ab71b676 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] [instance: 96ec3b45-063c-4bc9-84e5-e59532a23783] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:00:21.079514 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a5586143-614d-4e23-bcc8-93f7ab71b676 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:00:21.079974 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a5586143-614d-4e23-bcc8-93f7ab71b676 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:00:21.080741 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a5586143-614d-4e23-bcc8-93f7ab71b676 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:00:21.081188 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a5586143-614d-4e23-bcc8-93f7ab71b676 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:00:21.211898 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-a5586143-614d-4e23-bcc8-93f7ab71b676 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd980641608134eadb72afa6fddb6adeb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9934bd813edb4e55a2d7c61207a9c46d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:00:21.219312 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-a5586143-614d-4e23-bcc8-93f7ab71b676 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] 10.222.0.16 "GET /compute/v2.1/servers/96ec3b45-063c-4bc9-84e5-e59532a23783" status: 200 len: 1519 microversion: 2.1 time: 0.151214 Jul 16 22:00:21.219876 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3220/6434] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:00:21 2022] GET /compute/v2.1/servers/96ec3b45-063c-4bc9-84e5-e59532a23783 => generated 1519 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:00:22.237335 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-f42c8954-70b6-4150-9269-077c8e12d355 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:00:22.237859 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-f42c8954-70b6-4150-9269-077c8e12d355 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] [instance: 96ec3b45-063c-4bc9-84e5-e59532a23783] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:00:22.245043 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f42c8954-70b6-4150-9269-077c8e12d355 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:00:22.245508 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f42c8954-70b6-4150-9269-077c8e12d355 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:00:22.246273 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f42c8954-70b6-4150-9269-077c8e12d355 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:00:22.246724 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f42c8954-70b6-4150-9269-077c8e12d355 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:00:22.380955 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-f42c8954-70b6-4150-9269-077c8e12d355 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd980641608134eadb72afa6fddb6adeb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9934bd813edb4e55a2d7c61207a9c46d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:00:22.388782 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-f42c8954-70b6-4150-9269-077c8e12d355 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] 10.222.0.16 "GET /compute/v2.1/servers/96ec3b45-063c-4bc9-84e5-e59532a23783" status: 200 len: 1519 microversion: 2.1 time: 0.155433 Jul 16 22:00:22.389354 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3215/6435] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:00:22 2022] GET /compute/v2.1/servers/96ec3b45-063c-4bc9-84e5-e59532a23783 => generated 1519 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:00:23.405703 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-b45bfff1-13eb-4639-b3a8-08fe9f0eaab4 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:00:23.406335 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-b45bfff1-13eb-4639-b3a8-08fe9f0eaab4 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] [instance: 96ec3b45-063c-4bc9-84e5-e59532a23783] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:00:23.414207 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b45bfff1-13eb-4639-b3a8-08fe9f0eaab4 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:00:23.414673 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b45bfff1-13eb-4639-b3a8-08fe9f0eaab4 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:00:23.415485 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b45bfff1-13eb-4639-b3a8-08fe9f0eaab4 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:00:23.415939 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b45bfff1-13eb-4639-b3a8-08fe9f0eaab4 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:00:23.544785 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-b45bfff1-13eb-4639-b3a8-08fe9f0eaab4 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd980641608134eadb72afa6fddb6adeb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9934bd813edb4e55a2d7c61207a9c46d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:00:23.553189 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-b45bfff1-13eb-4639-b3a8-08fe9f0eaab4 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] 10.222.0.16 "GET /compute/v2.1/servers/96ec3b45-063c-4bc9-84e5-e59532a23783" status: 200 len: 1519 microversion: 2.1 time: 0.151195 Jul 16 22:00:23.553754 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3221/6436] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:00:23 2022] GET /compute/v2.1/servers/96ec3b45-063c-4bc9-84e5-e59532a23783 => generated 1519 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:00:24.570379 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-4d03db03-7d1c-4d76-9091-06b1406ad210 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:00:24.571027 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-4d03db03-7d1c-4d76-9091-06b1406ad210 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] [instance: 96ec3b45-063c-4bc9-84e5-e59532a23783] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:00:24.578039 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4d03db03-7d1c-4d76-9091-06b1406ad210 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:00:24.578493 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4d03db03-7d1c-4d76-9091-06b1406ad210 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:00:24.579280 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4d03db03-7d1c-4d76-9091-06b1406ad210 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:00:24.579717 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4d03db03-7d1c-4d76-9091-06b1406ad210 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:00:24.708880 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-4d03db03-7d1c-4d76-9091-06b1406ad210 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd980641608134eadb72afa6fddb6adeb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9934bd813edb4e55a2d7c61207a9c46d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:00:24.716632 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-4d03db03-7d1c-4d76-9091-06b1406ad210 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] 10.222.0.16 "GET /compute/v2.1/servers/96ec3b45-063c-4bc9-84e5-e59532a23783" status: 200 len: 1519 microversion: 2.1 time: 0.150275 Jul 16 22:00:24.717208 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3216/6437] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:00:24 2022] GET /compute/v2.1/servers/96ec3b45-063c-4bc9-84e5-e59532a23783 => generated 1519 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:00:25.734072 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-b6e1a0c3-aa4c-4773-93a8-9a6b79bf847b tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:00:25.734696 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-b6e1a0c3-aa4c-4773-93a8-9a6b79bf847b tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] [instance: 96ec3b45-063c-4bc9-84e5-e59532a23783] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:00:25.742256 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b6e1a0c3-aa4c-4773-93a8-9a6b79bf847b tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:00:25.742716 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b6e1a0c3-aa4c-4773-93a8-9a6b79bf847b tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:00:25.743557 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b6e1a0c3-aa4c-4773-93a8-9a6b79bf847b tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:00:25.744035 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b6e1a0c3-aa4c-4773-93a8-9a6b79bf847b tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:00:25.878367 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-b6e1a0c3-aa4c-4773-93a8-9a6b79bf847b tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd980641608134eadb72afa6fddb6adeb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9934bd813edb4e55a2d7c61207a9c46d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:00:25.886112 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-b6e1a0c3-aa4c-4773-93a8-9a6b79bf847b tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] 10.222.0.16 "GET /compute/v2.1/servers/96ec3b45-063c-4bc9-84e5-e59532a23783" status: 200 len: 1519 microversion: 2.1 time: 0.155904 Jul 16 22:00:25.886683 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3222/6438] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:00:25 2022] GET /compute/v2.1/servers/96ec3b45-063c-4bc9-84e5-e59532a23783 => generated 1519 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:00:26.902661 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-ce887605-8367-4d65-81ea-1a5b5136e904 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:00:26.903330 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-ce887605-8367-4d65-81ea-1a5b5136e904 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] [instance: 96ec3b45-063c-4bc9-84e5-e59532a23783] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:00:26.910398 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ce887605-8367-4d65-81ea-1a5b5136e904 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:00:26.910857 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ce887605-8367-4d65-81ea-1a5b5136e904 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:00:26.911658 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ce887605-8367-4d65-81ea-1a5b5136e904 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:00:26.912422 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ce887605-8367-4d65-81ea-1a5b5136e904 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:00:27.041316 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-ce887605-8367-4d65-81ea-1a5b5136e904 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd980641608134eadb72afa6fddb6adeb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9934bd813edb4e55a2d7c61207a9c46d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:00:27.048996 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-ce887605-8367-4d65-81ea-1a5b5136e904 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] 10.222.0.16 "GET /compute/v2.1/servers/96ec3b45-063c-4bc9-84e5-e59532a23783" status: 200 len: 1519 microversion: 2.1 time: 0.150305 Jul 16 22:00:27.049567 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3217/6439] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:00:26 2022] GET /compute/v2.1/servers/96ec3b45-063c-4bc9-84e5-e59532a23783 => generated 1519 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:00:28.067072 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-4c244677-e1cb-4a7b-a56f-67c6fb06385c tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:00:28.067756 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-4c244677-e1cb-4a7b-a56f-67c6fb06385c tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] [instance: 96ec3b45-063c-4bc9-84e5-e59532a23783] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:00:28.074845 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4c244677-e1cb-4a7b-a56f-67c6fb06385c tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:00:28.075337 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4c244677-e1cb-4a7b-a56f-67c6fb06385c tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:00:28.076108 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4c244677-e1cb-4a7b-a56f-67c6fb06385c tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:00:28.076559 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4c244677-e1cb-4a7b-a56f-67c6fb06385c tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:00:28.215901 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-4c244677-e1cb-4a7b-a56f-67c6fb06385c tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd980641608134eadb72afa6fddb6adeb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9934bd813edb4e55a2d7c61207a9c46d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:00:28.223427 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-4c244677-e1cb-4a7b-a56f-67c6fb06385c tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] 10.222.0.16 "GET /compute/v2.1/servers/96ec3b45-063c-4bc9-84e5-e59532a23783" status: 200 len: 1519 microversion: 2.1 time: 0.160260 Jul 16 22:00:28.223996 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3223/6440] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:00:28 2022] GET /compute/v2.1/servers/96ec3b45-063c-4bc9-84e5-e59532a23783 => generated 1519 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:00:29.240443 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-c829b342-4f82-433c-ad69-e28e1024d786 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:00:29.240973 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-c829b342-4f82-433c-ad69-e28e1024d786 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] [instance: 96ec3b45-063c-4bc9-84e5-e59532a23783] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:00:29.248213 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c829b342-4f82-433c-ad69-e28e1024d786 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:00:29.248666 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c829b342-4f82-433c-ad69-e28e1024d786 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:00:29.249431 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c829b342-4f82-433c-ad69-e28e1024d786 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:00:29.249869 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c829b342-4f82-433c-ad69-e28e1024d786 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:00:29.380793 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-c829b342-4f82-433c-ad69-e28e1024d786 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd980641608134eadb72afa6fddb6adeb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9934bd813edb4e55a2d7c61207a9c46d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:00:29.388981 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-c829b342-4f82-433c-ad69-e28e1024d786 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] 10.222.0.16 "GET /compute/v2.1/servers/96ec3b45-063c-4bc9-84e5-e59532a23783" status: 200 len: 1519 microversion: 2.1 time: 0.152498 Jul 16 22:00:29.389548 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3218/6441] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:00:29 2022] GET /compute/v2.1/servers/96ec3b45-063c-4bc9-84e5-e59532a23783 => generated 1519 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:00:30.406010 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-1d4cb9ae-0c0a-44df-b5dd-db8a41511a93 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:00:30.406643 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-1d4cb9ae-0c0a-44df-b5dd-db8a41511a93 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] [instance: 96ec3b45-063c-4bc9-84e5-e59532a23783] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:00:30.414403 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1d4cb9ae-0c0a-44df-b5dd-db8a41511a93 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:00:30.414873 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1d4cb9ae-0c0a-44df-b5dd-db8a41511a93 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:00:30.415779 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1d4cb9ae-0c0a-44df-b5dd-db8a41511a93 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:00:30.416233 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1d4cb9ae-0c0a-44df-b5dd-db8a41511a93 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:00:30.548850 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-1d4cb9ae-0c0a-44df-b5dd-db8a41511a93 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd980641608134eadb72afa6fddb6adeb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9934bd813edb4e55a2d7c61207a9c46d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:00:30.556531 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-1d4cb9ae-0c0a-44df-b5dd-db8a41511a93 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] 10.222.0.16 "GET /compute/v2.1/servers/96ec3b45-063c-4bc9-84e5-e59532a23783" status: 200 len: 1519 microversion: 2.1 time: 0.154391 Jul 16 22:00:30.557117 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3224/6442] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:00:30 2022] GET /compute/v2.1/servers/96ec3b45-063c-4bc9-84e5-e59532a23783 => generated 1519 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:00:31.573717 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-6676217c-1948-4d4b-9605-ecabdf1636bf tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:00:31.579387 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-6676217c-1948-4d4b-9605-ecabdf1636bf tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] [instance: 96ec3b45-063c-4bc9-84e5-e59532a23783] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:00:31.581310 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6676217c-1948-4d4b-9605-ecabdf1636bf tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:00:31.581774 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6676217c-1948-4d4b-9605-ecabdf1636bf tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:00:31.582529 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6676217c-1948-4d4b-9605-ecabdf1636bf tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:00:31.582967 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6676217c-1948-4d4b-9605-ecabdf1636bf tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:00:31.716441 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-6676217c-1948-4d4b-9605-ecabdf1636bf tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd980641608134eadb72afa6fddb6adeb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9934bd813edb4e55a2d7c61207a9c46d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:00:31.724597 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-6676217c-1948-4d4b-9605-ecabdf1636bf tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] 10.222.0.16 "GET /compute/v2.1/servers/96ec3b45-063c-4bc9-84e5-e59532a23783" status: 200 len: 1519 microversion: 2.1 time: 0.155440 Jul 16 22:00:31.725163 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3219/6443] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:00:31 2022] GET /compute/v2.1/servers/96ec3b45-063c-4bc9-84e5-e59532a23783 => generated 1519 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 22:00:32.741045 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-d3be2de5-7875-45ba-a758-6962ce0010d1 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:00:32.742096 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-d3be2de5-7875-45ba-a758-6962ce0010d1 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] [instance: 96ec3b45-063c-4bc9-84e5-e59532a23783] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:00:32.749015 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d3be2de5-7875-45ba-a758-6962ce0010d1 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:00:32.749477 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d3be2de5-7875-45ba-a758-6962ce0010d1 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:00:32.750247 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d3be2de5-7875-45ba-a758-6962ce0010d1 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:00:32.750694 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d3be2de5-7875-45ba-a758-6962ce0010d1 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:00:32.939790 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-d3be2de5-7875-45ba-a758-6962ce0010d1 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd980641608134eadb72afa6fddb6adeb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9934bd813edb4e55a2d7c61207a9c46d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:00:32.948273 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-d3be2de5-7875-45ba-a758-6962ce0010d1 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] 10.222.0.16 "GET /compute/v2.1/servers/96ec3b45-063c-4bc9-84e5-e59532a23783" status: 200 len: 1519 microversion: 2.1 time: 0.211168 Jul 16 22:00:32.948873 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3225/6444] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:00:32 2022] GET /compute/v2.1/servers/96ec3b45-063c-4bc9-84e5-e59532a23783 => generated 1519 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:00:33.965627 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-6c38614a-ec19-4202-9720-8d5dd9eb64d0 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:00:33.966300 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-6c38614a-ec19-4202-9720-8d5dd9eb64d0 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] [instance: 96ec3b45-063c-4bc9-84e5-e59532a23783] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:00:33.973307 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6c38614a-ec19-4202-9720-8d5dd9eb64d0 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:00:33.973759 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6c38614a-ec19-4202-9720-8d5dd9eb64d0 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:00:33.974512 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6c38614a-ec19-4202-9720-8d5dd9eb64d0 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:00:33.974952 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6c38614a-ec19-4202-9720-8d5dd9eb64d0 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:00:34.169989 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-6c38614a-ec19-4202-9720-8d5dd9eb64d0 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd980641608134eadb72afa6fddb6adeb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9934bd813edb4e55a2d7c61207a9c46d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:00:34.177711 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-6c38614a-ec19-4202-9720-8d5dd9eb64d0 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] 10.222.0.16 "GET /compute/v2.1/servers/96ec3b45-063c-4bc9-84e5-e59532a23783" status: 200 len: 1519 microversion: 2.1 time: 0.216002 Jul 16 22:00:34.178299 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3220/6445] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:00:33 2022] GET /compute/v2.1/servers/96ec3b45-063c-4bc9-84e5-e59532a23783 => generated 1519 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:00:35.195743 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-d5a89b76-3236-4e0e-a856-efda1b50cb57 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:00:35.196347 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-d5a89b76-3236-4e0e-a856-efda1b50cb57 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] [instance: 96ec3b45-063c-4bc9-84e5-e59532a23783] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:00:35.203282 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d5a89b76-3236-4e0e-a856-efda1b50cb57 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:00:35.203745 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d5a89b76-3236-4e0e-a856-efda1b50cb57 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:00:35.204539 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d5a89b76-3236-4e0e-a856-efda1b50cb57 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:00:35.204981 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d5a89b76-3236-4e0e-a856-efda1b50cb57 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:00:35.345944 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-d5a89b76-3236-4e0e-a856-efda1b50cb57 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd980641608134eadb72afa6fddb6adeb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9934bd813edb4e55a2d7c61207a9c46d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:00:35.354126 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-d5a89b76-3236-4e0e-a856-efda1b50cb57 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] 10.222.0.16 "GET /compute/v2.1/servers/96ec3b45-063c-4bc9-84e5-e59532a23783" status: 200 len: 1519 microversion: 2.1 time: 0.162582 Jul 16 22:00:35.354992 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3226/6446] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:00:35 2022] GET /compute/v2.1/servers/96ec3b45-063c-4bc9-84e5-e59532a23783 => generated 1519 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:00:36.372358 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-44ab12b2-3799-4954-9844-3323575b870f tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:00:36.372890 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-44ab12b2-3799-4954-9844-3323575b870f tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] [instance: 96ec3b45-063c-4bc9-84e5-e59532a23783] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:00:36.380156 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-44ab12b2-3799-4954-9844-3323575b870f tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:00:36.380629 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-44ab12b2-3799-4954-9844-3323575b870f tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:00:36.381402 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-44ab12b2-3799-4954-9844-3323575b870f tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:00:36.382512 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-44ab12b2-3799-4954-9844-3323575b870f tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:00:36.516823 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-44ab12b2-3799-4954-9844-3323575b870f tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd980641608134eadb72afa6fddb6adeb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9934bd813edb4e55a2d7c61207a9c46d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:00:36.524407 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-44ab12b2-3799-4954-9844-3323575b870f tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] 10.222.0.16 "GET /compute/v2.1/servers/96ec3b45-063c-4bc9-84e5-e59532a23783" status: 200 len: 1519 microversion: 2.1 time: 0.156072 Jul 16 22:00:36.524985 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3221/6447] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:00:36 2022] GET /compute/v2.1/servers/96ec3b45-063c-4bc9-84e5-e59532a23783 => generated 1519 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:00:37.542182 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-ac01a679-f421-49c6-a4aa-f48b00822137 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:00:37.542719 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-ac01a679-f421-49c6-a4aa-f48b00822137 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] [instance: 96ec3b45-063c-4bc9-84e5-e59532a23783] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:00:37.549663 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ac01a679-f421-49c6-a4aa-f48b00822137 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:00:37.550128 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ac01a679-f421-49c6-a4aa-f48b00822137 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:00:37.550891 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ac01a679-f421-49c6-a4aa-f48b00822137 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:00:37.551437 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ac01a679-f421-49c6-a4aa-f48b00822137 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:00:37.681990 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-ac01a679-f421-49c6-a4aa-f48b00822137 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd980641608134eadb72afa6fddb6adeb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9934bd813edb4e55a2d7c61207a9c46d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:00:37.689619 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-ac01a679-f421-49c6-a4aa-f48b00822137 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] 10.222.0.16 "GET /compute/v2.1/servers/96ec3b45-063c-4bc9-84e5-e59532a23783" status: 200 len: 1519 microversion: 2.1 time: 0.151716 Jul 16 22:00:37.690186 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3227/6448] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:00:37 2022] GET /compute/v2.1/servers/96ec3b45-063c-4bc9-84e5-e59532a23783 => generated 1519 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:00:38.707464 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-dada4925-ee1f-44e5-9cbd-f106989cddcd tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:00:38.708025 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-dada4925-ee1f-44e5-9cbd-f106989cddcd tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] [instance: 96ec3b45-063c-4bc9-84e5-e59532a23783] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:00:38.715895 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-dada4925-ee1f-44e5-9cbd-f106989cddcd tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:00:38.716357 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-dada4925-ee1f-44e5-9cbd-f106989cddcd tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:00:38.717125 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-dada4925-ee1f-44e5-9cbd-f106989cddcd tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:00:38.717570 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-dada4925-ee1f-44e5-9cbd-f106989cddcd tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:00:38.720262 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=92 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:00:38.850748 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-dada4925-ee1f-44e5-9cbd-f106989cddcd tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd980641608134eadb72afa6fddb6adeb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9934bd813edb4e55a2d7c61207a9c46d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:00:38.858404 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-dada4925-ee1f-44e5-9cbd-f106989cddcd tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] 10.222.0.16 "GET /compute/v2.1/servers/96ec3b45-063c-4bc9-84e5-e59532a23783" status: 200 len: 1519 microversion: 2.1 time: 0.154944 Jul 16 22:00:38.858980 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3222/6449] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:00:38 2022] GET /compute/v2.1/servers/96ec3b45-063c-4bc9-84e5-e59532a23783 => generated 1519 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:00:39.875886 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-c5801bbb-f09d-40e3-97a6-eafee4855a6e tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:00:39.876521 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-c5801bbb-f09d-40e3-97a6-eafee4855a6e tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] [instance: 96ec3b45-063c-4bc9-84e5-e59532a23783] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:00:39.883782 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c5801bbb-f09d-40e3-97a6-eafee4855a6e tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:00:39.884250 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c5801bbb-f09d-40e3-97a6-eafee4855a6e tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:00:39.885067 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c5801bbb-f09d-40e3-97a6-eafee4855a6e tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:00:39.885525 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c5801bbb-f09d-40e3-97a6-eafee4855a6e tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:00:40.021023 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-c5801bbb-f09d-40e3-97a6-eafee4855a6e tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd980641608134eadb72afa6fddb6adeb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9934bd813edb4e55a2d7c61207a9c46d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:00:40.029135 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-c5801bbb-f09d-40e3-97a6-eafee4855a6e tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] 10.222.0.16 "GET /compute/v2.1/servers/96ec3b45-063c-4bc9-84e5-e59532a23783" status: 200 len: 1519 microversion: 2.1 time: 0.157323 Jul 16 22:00:40.029708 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3228/6450] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:00:39 2022] GET /compute/v2.1/servers/96ec3b45-063c-4bc9-84e5-e59532a23783 => generated 1519 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:00:41.045668 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-7739f332-5c53-45d9-b72d-3a70746cb84e tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:00:41.046284 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-7739f332-5c53-45d9-b72d-3a70746cb84e tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] [instance: 96ec3b45-063c-4bc9-84e5-e59532a23783] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:00:41.048995 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_api:SELECT=30 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:00:41.054439 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7739f332-5c53-45d9-b72d-3a70746cb84e tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:00:41.055075 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7739f332-5c53-45d9-b72d-3a70746cb84e tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:00:41.059502 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7739f332-5c53-45d9-b72d-3a70746cb84e tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:00:41.059502 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7739f332-5c53-45d9-b72d-3a70746cb84e tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:00:41.187525 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-7739f332-5c53-45d9-b72d-3a70746cb84e tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd980641608134eadb72afa6fddb6adeb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9934bd813edb4e55a2d7c61207a9c46d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:00:41.195458 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-7739f332-5c53-45d9-b72d-3a70746cb84e tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] 10.222.0.16 "GET /compute/v2.1/servers/96ec3b45-063c-4bc9-84e5-e59532a23783" status: 200 len: 1519 microversion: 2.1 time: 0.153641 Jul 16 22:00:41.196045 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3223/6451] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:00:41 2022] GET /compute/v2.1/servers/96ec3b45-063c-4bc9-84e5-e59532a23783 => generated 1519 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:00:42.213043 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-1651c4eb-72f8-4043-9f33-aef1fa34ae73 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:00:42.213566 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-1651c4eb-72f8-4043-9f33-aef1fa34ae73 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] [instance: 96ec3b45-063c-4bc9-84e5-e59532a23783] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:00:42.220519 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1651c4eb-72f8-4043-9f33-aef1fa34ae73 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:00:42.220999 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1651c4eb-72f8-4043-9f33-aef1fa34ae73 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:00:42.221752 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1651c4eb-72f8-4043-9f33-aef1fa34ae73 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:00:42.222201 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1651c4eb-72f8-4043-9f33-aef1fa34ae73 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:00:42.358819 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-1651c4eb-72f8-4043-9f33-aef1fa34ae73 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd980641608134eadb72afa6fddb6adeb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9934bd813edb4e55a2d7c61207a9c46d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:00:42.361115 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=94,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:00:42.367829 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-1651c4eb-72f8-4043-9f33-aef1fa34ae73 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] 10.222.0.16 "GET /compute/v2.1/servers/96ec3b45-063c-4bc9-84e5-e59532a23783" status: 200 len: 1519 microversion: 2.1 time: 0.159076 Jul 16 22:00:42.368399 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3229/6452] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:00:42 2022] GET /compute/v2.1/servers/96ec3b45-063c-4bc9-84e5-e59532a23783 => generated 1519 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:00:43.386227 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-7f9757c3-7efe-4cc9-8b89-711dd953b063 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:00:43.386886 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-7f9757c3-7efe-4cc9-8b89-711dd953b063 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] [instance: 96ec3b45-063c-4bc9-84e5-e59532a23783] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:00:43.393939 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7f9757c3-7efe-4cc9-8b89-711dd953b063 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:00:43.394398 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7f9757c3-7efe-4cc9-8b89-711dd953b063 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:00:43.395243 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7f9757c3-7efe-4cc9-8b89-711dd953b063 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:00:43.395684 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7f9757c3-7efe-4cc9-8b89-711dd953b063 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:00:43.540937 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-7f9757c3-7efe-4cc9-8b89-711dd953b063 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd980641608134eadb72afa6fddb6adeb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9934bd813edb4e55a2d7c61207a9c46d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:00:43.550562 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-7f9757c3-7efe-4cc9-8b89-711dd953b063 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] 10.222.0.16 "GET /compute/v2.1/servers/96ec3b45-063c-4bc9-84e5-e59532a23783" status: 200 len: 1519 microversion: 2.1 time: 0.169014 Jul 16 22:00:43.551570 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3224/6453] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:00:43 2022] GET /compute/v2.1/servers/96ec3b45-063c-4bc9-84e5-e59532a23783 => generated 1519 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:00:44.570253 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-bf74fa24-46bf-49f7-be71-cfbe9d5721d2 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:00:44.570933 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-bf74fa24-46bf-49f7-be71-cfbe9d5721d2 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] [instance: 96ec3b45-063c-4bc9-84e5-e59532a23783] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:00:44.573607 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_api:SELECT=34 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:00:44.578694 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-bf74fa24-46bf-49f7-be71-cfbe9d5721d2 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:00:44.579185 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-bf74fa24-46bf-49f7-be71-cfbe9d5721d2 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:00:44.579954 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-bf74fa24-46bf-49f7-be71-cfbe9d5721d2 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:00:44.580401 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-bf74fa24-46bf-49f7-be71-cfbe9d5721d2 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:00:44.713924 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-bf74fa24-46bf-49f7-be71-cfbe9d5721d2 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd980641608134eadb72afa6fddb6adeb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9934bd813edb4e55a2d7c61207a9c46d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:00:44.721441 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-bf74fa24-46bf-49f7-be71-cfbe9d5721d2 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] 10.222.0.16 "GET /compute/v2.1/servers/96ec3b45-063c-4bc9-84e5-e59532a23783" status: 200 len: 1519 microversion: 2.1 time: 0.156229 Jul 16 22:00:44.722017 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3230/6454] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:00:44 2022] GET /compute/v2.1/servers/96ec3b45-063c-4bc9-84e5-e59532a23783 => generated 1519 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:00:45.739291 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-71ea2601-04e9-4302-94a9-58c36a6e18bd tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:00:45.739910 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-71ea2601-04e9-4302-94a9-58c36a6e18bd tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] [instance: 96ec3b45-063c-4bc9-84e5-e59532a23783] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:00:45.747087 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-71ea2601-04e9-4302-94a9-58c36a6e18bd tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:00:45.749596 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-71ea2601-04e9-4302-94a9-58c36a6e18bd tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:00:45.749596 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-71ea2601-04e9-4302-94a9-58c36a6e18bd tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:00:45.749596 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-71ea2601-04e9-4302-94a9-58c36a6e18bd tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:00:45.880333 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-71ea2601-04e9-4302-94a9-58c36a6e18bd tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd980641608134eadb72afa6fddb6adeb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9934bd813edb4e55a2d7c61207a9c46d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:00:45.888623 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-71ea2601-04e9-4302-94a9-58c36a6e18bd tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] 10.222.0.16 "GET /compute/v2.1/servers/96ec3b45-063c-4bc9-84e5-e59532a23783" status: 200 len: 1519 microversion: 2.1 time: 0.153277 Jul 16 22:00:45.889197 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3225/6455] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:00:45 2022] GET /compute/v2.1/servers/96ec3b45-063c-4bc9-84e5-e59532a23783 => generated 1519 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:00:46.905781 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-db64d0df-f231-4b44-8a3b-45aaa1376f22 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:00:46.906409 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-db64d0df-f231-4b44-8a3b-45aaa1376f22 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] [instance: 96ec3b45-063c-4bc9-84e5-e59532a23783] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:00:46.913765 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-db64d0df-f231-4b44-8a3b-45aaa1376f22 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:00:46.914227 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-db64d0df-f231-4b44-8a3b-45aaa1376f22 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:00:46.915017 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-db64d0df-f231-4b44-8a3b-45aaa1376f22 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:00:46.915481 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-db64d0df-f231-4b44-8a3b-45aaa1376f22 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:00:47.046990 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-db64d0df-f231-4b44-8a3b-45aaa1376f22 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd980641608134eadb72afa6fddb6adeb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9934bd813edb4e55a2d7c61207a9c46d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:00:47.055270 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-db64d0df-f231-4b44-8a3b-45aaa1376f22 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] 10.222.0.16 "GET /compute/v2.1/servers/96ec3b45-063c-4bc9-84e5-e59532a23783" status: 200 len: 1519 microversion: 2.1 time: 0.153390 Jul 16 22:00:47.055843 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3231/6456] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:00:46 2022] GET /compute/v2.1/servers/96ec3b45-063c-4bc9-84e5-e59532a23783 => generated 1519 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:00:48.072787 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-21fae2df-859c-4b86-813b-ffb11345e237 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:00:48.073408 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-21fae2df-859c-4b86-813b-ffb11345e237 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] [instance: 96ec3b45-063c-4bc9-84e5-e59532a23783] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:00:48.080581 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-21fae2df-859c-4b86-813b-ffb11345e237 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:00:48.080978 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-21fae2df-859c-4b86-813b-ffb11345e237 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:00:48.081748 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-21fae2df-859c-4b86-813b-ffb11345e237 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:00:48.082191 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-21fae2df-859c-4b86-813b-ffb11345e237 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:00:48.219304 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-21fae2df-859c-4b86-813b-ffb11345e237 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd980641608134eadb72afa6fddb6adeb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9934bd813edb4e55a2d7c61207a9c46d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:00:48.226909 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-21fae2df-859c-4b86-813b-ffb11345e237 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] 10.222.0.16 "GET /compute/v2.1/servers/96ec3b45-063c-4bc9-84e5-e59532a23783" status: 200 len: 1519 microversion: 2.1 time: 0.158593 Jul 16 22:00:48.227521 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3226/6457] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:00:48 2022] GET /compute/v2.1/servers/96ec3b45-063c-4bc9-84e5-e59532a23783 => generated 1519 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:00:49.244523 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-8a612c80-57a6-42db-92af-4b8c56121541 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:00:49.245157 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-8a612c80-57a6-42db-92af-4b8c56121541 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] [instance: 96ec3b45-063c-4bc9-84e5-e59532a23783] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:00:49.252297 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8a612c80-57a6-42db-92af-4b8c56121541 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:00:49.252767 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8a612c80-57a6-42db-92af-4b8c56121541 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:00:49.253540 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8a612c80-57a6-42db-92af-4b8c56121541 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:00:49.253993 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8a612c80-57a6-42db-92af-4b8c56121541 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:00:49.384341 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-8a612c80-57a6-42db-92af-4b8c56121541 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd980641608134eadb72afa6fddb6adeb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9934bd813edb4e55a2d7c61207a9c46d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:00:49.392094 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-8a612c80-57a6-42db-92af-4b8c56121541 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] 10.222.0.16 "GET /compute/v2.1/servers/96ec3b45-063c-4bc9-84e5-e59532a23783" status: 200 len: 1519 microversion: 2.1 time: 0.151983 Jul 16 22:00:49.392619 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3232/6458] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:00:49 2022] GET /compute/v2.1/servers/96ec3b45-063c-4bc9-84e5-e59532a23783 => generated 1519 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:00:50.409177 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-53869e93-c3d7-474d-bd84-cb914db7a205 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:00:50.409798 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-53869e93-c3d7-474d-bd84-cb914db7a205 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] [instance: 96ec3b45-063c-4bc9-84e5-e59532a23783] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:00:50.417201 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-53869e93-c3d7-474d-bd84-cb914db7a205 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:00:50.417657 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-53869e93-c3d7-474d-bd84-cb914db7a205 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:00:50.418420 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-53869e93-c3d7-474d-bd84-cb914db7a205 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:00:50.418856 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-53869e93-c3d7-474d-bd84-cb914db7a205 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:00:50.552824 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-53869e93-c3d7-474d-bd84-cb914db7a205 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd980641608134eadb72afa6fddb6adeb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9934bd813edb4e55a2d7c61207a9c46d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:00:50.561123 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-53869e93-c3d7-474d-bd84-cb914db7a205 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] 10.222.0.16 "GET /compute/v2.1/servers/96ec3b45-063c-4bc9-84e5-e59532a23783" status: 200 len: 1519 microversion: 2.1 time: 0.155873 Jul 16 22:00:50.561697 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3227/6459] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:00:50 2022] GET /compute/v2.1/servers/96ec3b45-063c-4bc9-84e5-e59532a23783 => generated 1519 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:00:51.578152 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-4fac6f5f-3332-4172-b7fc-764a1a0fdd66 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:00:51.578778 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-4fac6f5f-3332-4172-b7fc-764a1a0fdd66 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] [instance: 96ec3b45-063c-4bc9-84e5-e59532a23783] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:00:51.586773 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4fac6f5f-3332-4172-b7fc-764a1a0fdd66 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:00:51.587271 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4fac6f5f-3332-4172-b7fc-764a1a0fdd66 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:00:51.588050 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4fac6f5f-3332-4172-b7fc-764a1a0fdd66 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:00:51.588502 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4fac6f5f-3332-4172-b7fc-764a1a0fdd66 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:00:51.718348 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-4fac6f5f-3332-4172-b7fc-764a1a0fdd66 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd980641608134eadb72afa6fddb6adeb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9934bd813edb4e55a2d7c61207a9c46d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:00:51.726224 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-4fac6f5f-3332-4172-b7fc-764a1a0fdd66 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] 10.222.0.16 "GET /compute/v2.1/servers/96ec3b45-063c-4bc9-84e5-e59532a23783" status: 200 len: 1519 microversion: 2.1 time: 0.151980 Jul 16 22:00:51.727703 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3233/6460] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:00:51 2022] GET /compute/v2.1/servers/96ec3b45-063c-4bc9-84e5-e59532a23783 => generated 1519 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:00:52.743802 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-72bb6345-d26b-4299-8e72-2cc6b0e796ac tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:00:52.744473 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-72bb6345-d26b-4299-8e72-2cc6b0e796ac tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] [instance: 96ec3b45-063c-4bc9-84e5-e59532a23783] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:00:52.751824 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-72bb6345-d26b-4299-8e72-2cc6b0e796ac tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:00:52.752304 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-72bb6345-d26b-4299-8e72-2cc6b0e796ac tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:00:52.753084 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-72bb6345-d26b-4299-8e72-2cc6b0e796ac tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:00:52.753530 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-72bb6345-d26b-4299-8e72-2cc6b0e796ac tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:00:52.928792 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-72bb6345-d26b-4299-8e72-2cc6b0e796ac tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd980641608134eadb72afa6fddb6adeb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9934bd813edb4e55a2d7c61207a9c46d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:00:52.936922 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-72bb6345-d26b-4299-8e72-2cc6b0e796ac tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] 10.222.0.16 "GET /compute/v2.1/servers/96ec3b45-063c-4bc9-84e5-e59532a23783" status: 200 len: 1518 microversion: 2.1 time: 0.197604 Jul 16 22:00:52.937524 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3228/6461] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:00:52 2022] GET /compute/v2.1/servers/96ec3b45-063c-4bc9-84e5-e59532a23783 => generated 1518 bytes in 199 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:00:53.282722 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-dfdeb01a-fa87-4ccb-8280-970a7b7d24d8 req-a8519b36-67b5-4a89-a0c0-bb6d149e9a2f service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "96ec3b45-063c-4bc9-84e5-e59532a23783", "name": "network-vif-unplugged", "status": "completed", "tag": "2bcfc300-6586-4eae-a738-286a969a6f27"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:00:53.291652 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-dfdeb01a-fa87-4ccb-8280-970a7b7d24d8 req-a8519b36-67b5-4a89-a0c0-bb6d149e9a2f service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:00:53.292107 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-dfdeb01a-fa87-4ccb-8280-970a7b7d24d8 req-a8519b36-67b5-4a89-a0c0-bb6d149e9a2f service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:00:53.304068 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-dfdeb01a-fa87-4ccb-8280-970a7b7d24d8 req-a8519b36-67b5-4a89-a0c0-bb6d149e9a2f service nova] Creating event network-vif-unplugged:2bcfc300-6586-4eae-a738-286a969a6f27 for instance 96ec3b45-063c-4bc9-84e5-e59532a23783 on nch2-master-1 Jul 16 22:00:53.309911 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [req-dfdeb01a-fa87-4ccb-8280-970a7b7d24d8 req-a8519b36-67b5-4a89-a0c0-bb6d149e9a2f service nova] Instance 96ec3b45-063c-4bc9-84e5-e59532a23783 is migrating within the same cell, copying events to all relevant hosts: {'nch2-master-1', 'nch1-master-1'} {{(pid=138681) _get_relevant_hosts /opt/stack/nova/nova/compute/api.py:5807}} Jul 16 22:00:53.321236 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-dfdeb01a-fa87-4ccb-8280-970a7b7d24d8 req-a8519b36-67b5-4a89-a0c0-bb6d149e9a2f service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.042295 Jul 16 22:00:53.321844 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3234/6462] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 22:00:53 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 22:00:53.957404 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-fc5ae4b0-30d3-4f81-ba00-e48df46f2b9e tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:00:53.957404 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-fc5ae4b0-30d3-4f81-ba00-e48df46f2b9e tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] [instance: 96ec3b45-063c-4bc9-84e5-e59532a23783] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:00:53.963441 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-fc5ae4b0-30d3-4f81-ba00-e48df46f2b9e tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:00:53.963897 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-fc5ae4b0-30d3-4f81-ba00-e48df46f2b9e tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:00:53.964676 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-fc5ae4b0-30d3-4f81-ba00-e48df46f2b9e tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:00:53.965122 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-fc5ae4b0-30d3-4f81-ba00-e48df46f2b9e tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:00:54.172857 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-fc5ae4b0-30d3-4f81-ba00-e48df46f2b9e tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd980641608134eadb72afa6fddb6adeb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9934bd813edb4e55a2d7c61207a9c46d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:00:54.180978 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-fc5ae4b0-30d3-4f81-ba00-e48df46f2b9e tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] 10.222.0.16 "GET /compute/v2.1/servers/96ec3b45-063c-4bc9-84e5-e59532a23783" status: 200 len: 1518 microversion: 2.1 time: 0.230056 Jul 16 22:00:54.181568 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3229/6463] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:00:53 2022] GET /compute/v2.1/servers/96ec3b45-063c-4bc9-84e5-e59532a23783 => generated 1518 bytes in 231 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:00:55.200172 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-c05d725a-5cb9-4ec0-bd3e-1a8dcc5fc31d tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:00:55.200935 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-c05d725a-5cb9-4ec0-bd3e-1a8dcc5fc31d tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] [instance: 96ec3b45-063c-4bc9-84e5-e59532a23783] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:00:55.207445 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c05d725a-5cb9-4ec0-bd3e-1a8dcc5fc31d tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:00:55.207905 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c05d725a-5cb9-4ec0-bd3e-1a8dcc5fc31d tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:00:55.208675 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c05d725a-5cb9-4ec0-bd3e-1a8dcc5fc31d tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:00:55.209127 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c05d725a-5cb9-4ec0-bd3e-1a8dcc5fc31d tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:00:55.340443 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-45d9999c-7a69-4ed9-8d3f-ea32bd42dfda req-8a4a961f-0256-4f08-b92d-0bbdd441b23f service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "96ec3b45-063c-4bc9-84e5-e59532a23783", "tag": "2bcfc300-6586-4eae-a738-286a969a6f27"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:00:55.349323 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-45d9999c-7a69-4ed9-8d3f-ea32bd42dfda req-8a4a961f-0256-4f08-b92d-0bbdd441b23f service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:00:55.349768 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-45d9999c-7a69-4ed9-8d3f-ea32bd42dfda req-8a4a961f-0256-4f08-b92d-0bbdd441b23f service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:00:55.352782 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-c05d725a-5cb9-4ec0-bd3e-1a8dcc5fc31d tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd980641608134eadb72afa6fddb6adeb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9934bd813edb4e55a2d7c61207a9c46d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:00:55.360521 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-c05d725a-5cb9-4ec0-bd3e-1a8dcc5fc31d tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] 10.222.0.16 "GET /compute/v2.1/servers/96ec3b45-063c-4bc9-84e5-e59532a23783" status: 200 len: 1516 microversion: 2.1 time: 0.165791 Jul 16 22:00:55.361178 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3235/6464] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:00:55 2022] GET /compute/v2.1/servers/96ec3b45-063c-4bc9-84e5-e59532a23783 => generated 1516 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:00:55.361567 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-45d9999c-7a69-4ed9-8d3f-ea32bd42dfda req-8a4a961f-0256-4f08-b92d-0bbdd441b23f service nova] Creating event network-changed:2bcfc300-6586-4eae-a738-286a969a6f27 for instance 96ec3b45-063c-4bc9-84e5-e59532a23783 on nch2-master-1 Jul 16 22:00:55.367362 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [req-45d9999c-7a69-4ed9-8d3f-ea32bd42dfda req-8a4a961f-0256-4f08-b92d-0bbdd441b23f service nova] Instance 96ec3b45-063c-4bc9-84e5-e59532a23783 is migrating within the same cell, copying events to all relevant hosts: {'nch2-master-1', 'nch1-master-1'} {{(pid=138680) _get_relevant_hosts /opt/stack/nova/nova/compute/api.py:5807}} Jul 16 22:00:55.377880 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-45d9999c-7a69-4ed9-8d3f-ea32bd42dfda req-8a4a961f-0256-4f08-b92d-0bbdd441b23f service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.041268 Jul 16 22:00:55.378466 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3230/6465] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 22:00:55 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 22:00:56.377944 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-d330b6a1-bfaf-42b3-9558-43f47a640679 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:00:56.378657 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-d330b6a1-bfaf-42b3-9558-43f47a640679 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] [instance: 96ec3b45-063c-4bc9-84e5-e59532a23783] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:00:56.385958 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d330b6a1-bfaf-42b3-9558-43f47a640679 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:00:56.386496 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d330b6a1-bfaf-42b3-9558-43f47a640679 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:00:56.387396 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d330b6a1-bfaf-42b3-9558-43f47a640679 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:00:56.387980 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d330b6a1-bfaf-42b3-9558-43f47a640679 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:00:56.581925 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-d330b6a1-bfaf-42b3-9558-43f47a640679 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd980641608134eadb72afa6fddb6adeb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9934bd813edb4e55a2d7c61207a9c46d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:00:56.590899 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-d330b6a1-bfaf-42b3-9558-43f47a640679 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] 10.222.0.16 "GET /compute/v2.1/servers/96ec3b45-063c-4bc9-84e5-e59532a23783" status: 200 len: 1516 microversion: 2.1 time: 0.216967 Jul 16 22:00:56.591753 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3236/6466] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:00:56 2022] GET /compute/v2.1/servers/96ec3b45-063c-4bc9-84e5-e59532a23783 => generated 1516 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:00:57.610521 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-1fd0411a-3e0c-4496-8606-fda38fe0509b tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:00:57.611304 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-1fd0411a-3e0c-4496-8606-fda38fe0509b tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] [instance: 96ec3b45-063c-4bc9-84e5-e59532a23783] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:00:57.618234 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1fd0411a-3e0c-4496-8606-fda38fe0509b tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:00:57.618686 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1fd0411a-3e0c-4496-8606-fda38fe0509b tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:00:57.619486 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1fd0411a-3e0c-4496-8606-fda38fe0509b tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:00:57.619931 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1fd0411a-3e0c-4496-8606-fda38fe0509b tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:00:57.752820 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-1fd0411a-3e0c-4496-8606-fda38fe0509b tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd980641608134eadb72afa6fddb6adeb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9934bd813edb4e55a2d7c61207a9c46d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:00:57.760722 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-1fd0411a-3e0c-4496-8606-fda38fe0509b tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] 10.222.0.16 "GET /compute/v2.1/servers/96ec3b45-063c-4bc9-84e5-e59532a23783" status: 200 len: 1513 microversion: 2.1 time: 0.155684 Jul 16 22:00:57.761308 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3231/6467] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:00:57 2022] GET /compute/v2.1/servers/96ec3b45-063c-4bc9-84e5-e59532a23783 => generated 1513 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:00:57.777211 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-5550c44b-7801-45e2-b9c9-46c4206b0273 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Action: 'action', calling method: >, body: {"confirmResize": {}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:00:57.780292 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-5550c44b-7801-45e2-b9c9-46c4206b0273 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] [instance: 96ec3b45-063c-4bc9-84e5-e59532a23783] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:00:57.784710 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5550c44b-7801-45e2-b9c9-46c4206b0273 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:00:57.785179 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5550c44b-7801-45e2-b9c9-46c4206b0273 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:00:57.785995 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5550c44b-7801-45e2-b9c9-46c4206b0273 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:00:57.786456 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5550c44b-7801-45e2-b9c9-46c4206b0273 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:00:57.990391 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-5550c44b-7801-45e2-b9c9-46c4206b0273 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] 10.222.0.16 "POST /compute/v2.1/servers/96ec3b45-063c-4bc9-84e5-e59532a23783/action" status: 204 len: 0 microversion: 2.1 time: 0.216877 Jul 16 22:00:57.991076 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3237/6468] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 22:00:57 2022] POST /compute/v2.1/servers/96ec3b45-063c-4bc9-84e5-e59532a23783/action => generated 0 bytes in 218 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Jul 16 22:00:57.999662 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-6794539c-fb28-466c-9ec8-d8db71e46a11 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:00:58.000369 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-6794539c-fb28-466c-9ec8-d8db71e46a11 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] [instance: 96ec3b45-063c-4bc9-84e5-e59532a23783] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:00:58.006645 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6794539c-fb28-466c-9ec8-d8db71e46a11 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:00:58.007144 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6794539c-fb28-466c-9ec8-d8db71e46a11 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:00:58.007919 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6794539c-fb28-466c-9ec8-d8db71e46a11 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:00:58.008363 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6794539c-fb28-466c-9ec8-d8db71e46a11 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:00:58.150762 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-6794539c-fb28-466c-9ec8-d8db71e46a11 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd980641608134eadb72afa6fddb6adeb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9934bd813edb4e55a2d7c61207a9c46d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:00:58.158301 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-6794539c-fb28-466c-9ec8-d8db71e46a11 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] 10.222.0.16 "GET /compute/v2.1/servers/96ec3b45-063c-4bc9-84e5-e59532a23783" status: 200 len: 1513 microversion: 2.1 time: 0.162584 Jul 16 22:00:58.158851 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3232/6469] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:00:57 2022] GET /compute/v2.1/servers/96ec3b45-063c-4bc9-84e5-e59532a23783 => generated 1513 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:00:59.103741 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-5197e529-3501-47e1-a967-192277778205 req-29dc57df-1765-42c7-b0da-fa75c3646828 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "96ec3b45-063c-4bc9-84e5-e59532a23783", "name": "network-vif-plugged", "status": "completed", "tag": "2bcfc300-6586-4eae-a738-286a969a6f27"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:00:59.112049 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-5197e529-3501-47e1-a967-192277778205 req-29dc57df-1765-42c7-b0da-fa75c3646828 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:00:59.112516 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-5197e529-3501-47e1-a967-192277778205 req-29dc57df-1765-42c7-b0da-fa75c3646828 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:00:59.125579 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-5197e529-3501-47e1-a967-192277778205 req-29dc57df-1765-42c7-b0da-fa75c3646828 service nova] Creating event network-vif-plugged:2bcfc300-6586-4eae-a738-286a969a6f27 for instance 96ec3b45-063c-4bc9-84e5-e59532a23783 on nch2-master-1 Jul 16 22:00:59.131397 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [req-5197e529-3501-47e1-a967-192277778205 req-29dc57df-1765-42c7-b0da-fa75c3646828 service nova] Instance 96ec3b45-063c-4bc9-84e5-e59532a23783 is migrating within the same cell, copying events to all relevant hosts: {'nch2-master-1', 'nch1-master-1'} {{(pid=138681) _get_relevant_hosts /opt/stack/nova/nova/compute/api.py:5807}} Jul 16 22:00:59.142592 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-5197e529-3501-47e1-a967-192277778205 req-29dc57df-1765-42c7-b0da-fa75c3646828 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.111029 Jul 16 22:00:59.143484 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3238/6470] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 22:00:59 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 113 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 16 22:00:59.176300 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-3403bcc0-a6d7-4062-abc9-db21671cec6c tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:00:59.176926 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-3403bcc0-a6d7-4062-abc9-db21671cec6c tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] [instance: 96ec3b45-063c-4bc9-84e5-e59532a23783] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:00:59.184781 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3403bcc0-a6d7-4062-abc9-db21671cec6c tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:00:59.185278 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3403bcc0-a6d7-4062-abc9-db21671cec6c tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:00:59.186055 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3403bcc0-a6d7-4062-abc9-db21671cec6c tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:00:59.186497 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3403bcc0-a6d7-4062-abc9-db21671cec6c tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:00:59.320834 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-3403bcc0-a6d7-4062-abc9-db21671cec6c tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd980641608134eadb72afa6fddb6adeb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9934bd813edb4e55a2d7c61207a9c46d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:00:59.328560 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-3403bcc0-a6d7-4062-abc9-db21671cec6c tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] 10.222.0.16 "GET /compute/v2.1/servers/96ec3b45-063c-4bc9-84e5-e59532a23783" status: 200 len: 1513 microversion: 2.1 time: 0.156553 Jul 16 22:00:59.331328 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3233/6471] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:00:59 2022] GET /compute/v2.1/servers/96ec3b45-063c-4bc9-84e5-e59532a23783 => generated 1513 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:01:00.347994 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-097084c5-32b9-4410-87d1-2ef1ed6ccf6c tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:01:00.348708 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-097084c5-32b9-4410-87d1-2ef1ed6ccf6c tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] [instance: 96ec3b45-063c-4bc9-84e5-e59532a23783] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:01:00.355880 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-097084c5-32b9-4410-87d1-2ef1ed6ccf6c tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:01:00.356345 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-097084c5-32b9-4410-87d1-2ef1ed6ccf6c tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:01:00.357110 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-097084c5-32b9-4410-87d1-2ef1ed6ccf6c tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:01:00.357555 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-097084c5-32b9-4410-87d1-2ef1ed6ccf6c tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:01:00.494442 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-097084c5-32b9-4410-87d1-2ef1ed6ccf6c tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd980641608134eadb72afa6fddb6adeb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9934bd813edb4e55a2d7c61207a9c46d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:01:00.502046 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-097084c5-32b9-4410-87d1-2ef1ed6ccf6c tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] 10.222.0.16 "GET /compute/v2.1/servers/96ec3b45-063c-4bc9-84e5-e59532a23783" status: 200 len: 1505 microversion: 2.1 time: 0.159209 Jul 16 22:01:00.502633 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3239/6472] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:01:00 2022] GET /compute/v2.1/servers/96ec3b45-063c-4bc9-84e5-e59532a23783 => generated 1505 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:01:00.518565 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-0d1d947f-02d1-42d0-89c4-3fe419fbe580 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:01:00.519221 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-0d1d947f-02d1-42d0-89c4-3fe419fbe580 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] [instance: 96ec3b45-063c-4bc9-84e5-e59532a23783] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:01:00.527117 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0d1d947f-02d1-42d0-89c4-3fe419fbe580 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:01:00.527584 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0d1d947f-02d1-42d0-89c4-3fe419fbe580 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:01:00.528359 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0d1d947f-02d1-42d0-89c4-3fe419fbe580 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:01:00.528810 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0d1d947f-02d1-42d0-89c4-3fe419fbe580 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:01:00.659234 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-0d1d947f-02d1-42d0-89c4-3fe419fbe580 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd980641608134eadb72afa6fddb6adeb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9934bd813edb4e55a2d7c61207a9c46d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:01:00.667286 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-0d1d947f-02d1-42d0-89c4-3fe419fbe580 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] 10.222.0.16 "GET /compute/v2.1/servers/96ec3b45-063c-4bc9-84e5-e59532a23783" status: 200 len: 1505 microversion: 2.1 time: 0.152627 Jul 16 22:01:00.674702 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3234/6473] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:01:00 2022] GET /compute/v2.1/servers/96ec3b45-063c-4bc9-84e5-e59532a23783 => generated 1505 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:01:00.683168 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-da27e925-8642-42fe-a80a-21ccc3b00f26 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:01:00.683797 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-da27e925-8642-42fe-a80a-21ccc3b00f26 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] [instance: 96ec3b45-063c-4bc9-84e5-e59532a23783] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:01:00.689877 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-da27e925-8642-42fe-a80a-21ccc3b00f26 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:01:00.690379 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-da27e925-8642-42fe-a80a-21ccc3b00f26 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:01:00.691151 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-da27e925-8642-42fe-a80a-21ccc3b00f26 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:01:00.691603 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-da27e925-8642-42fe-a80a-21ccc3b00f26 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:01:00.820161 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-da27e925-8642-42fe-a80a-21ccc3b00f26 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd980641608134eadb72afa6fddb6adeb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9934bd813edb4e55a2d7c61207a9c46d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:01:00.827884 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-da27e925-8642-42fe-a80a-21ccc3b00f26 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] 10.222.0.16 "GET /compute/v2.1/servers/96ec3b45-063c-4bc9-84e5-e59532a23783" status: 200 len: 1505 microversion: 2.1 time: 0.148664 Jul 16 22:01:00.828458 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3240/6474] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:01:00 2022] GET /compute/v2.1/servers/96ec3b45-063c-4bc9-84e5-e59532a23783 => generated 1505 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:01:00.844809 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-c4ef1b0e-f2eb-42b6-8d47-8d137c690354 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:01:00.845349 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-c4ef1b0e-f2eb-42b6-8d47-8d137c690354 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] [instance: 96ec3b45-063c-4bc9-84e5-e59532a23783] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:01:00.851761 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c4ef1b0e-f2eb-42b6-8d47-8d137c690354 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:01:00.852225 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c4ef1b0e-f2eb-42b6-8d47-8d137c690354 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:01:00.853004 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c4ef1b0e-f2eb-42b6-8d47-8d137c690354 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:01:00.853456 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c4ef1b0e-f2eb-42b6-8d47-8d137c690354 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:01:00.879040 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-c4ef1b0e-f2eb-42b6-8d47-8d137c690354 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] [instance: 96ec3b45-063c-4bc9-84e5-e59532a23783] Going to try to terminate instance {{(pid=138680) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 16 22:01:01.087777 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-c4ef1b0e-f2eb-42b6-8d47-8d137c690354 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] 10.222.0.16 "DELETE /compute/v2.1/servers/96ec3b45-063c-4bc9-84e5-e59532a23783" status: 204 len: 0 microversion: 2.1 time: 0.246812 Jul 16 22:01:01.088386 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3235/6475] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 22:01:00 2022] DELETE /compute/v2.1/servers/96ec3b45-063c-4bc9-84e5-e59532a23783 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 16 22:01:01.107057 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-e582b828-226b-43a1-a86b-37cb5c63b358 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:01:01.107585 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-e582b828-226b-43a1-a86b-37cb5c63b358 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] [instance: 96ec3b45-063c-4bc9-84e5-e59532a23783] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:01:01.113878 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e582b828-226b-43a1-a86b-37cb5c63b358 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:01:01.114352 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e582b828-226b-43a1-a86b-37cb5c63b358 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:01:01.115212 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e582b828-226b-43a1-a86b-37cb5c63b358 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:01:01.116195 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e582b828-226b-43a1-a86b-37cb5c63b358 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:01:01.141385 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-e582b828-226b-43a1-a86b-37cb5c63b358 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] [instance: 96ec3b45-063c-4bc9-84e5-e59532a23783] Going to try to terminate instance {{(pid=138681) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 16 22:01:01.185173 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.compute.api [None req-e582b828-226b-43a1-a86b-37cb5c63b358 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] [instance: 96ec3b45-063c-4bc9-84e5-e59532a23783] Instance is already in deleting state, ignoring this request Jul 16 22:01:01.186007 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-e582b828-226b-43a1-a86b-37cb5c63b358 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] 10.222.0.16 "DELETE /compute/v2.1/servers/96ec3b45-063c-4bc9-84e5-e59532a23783" status: 204 len: 0 microversion: 2.1 time: 0.082954 Jul 16 22:01:01.186571 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3241/6476] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 22:01:01 2022] DELETE /compute/v2.1/servers/96ec3b45-063c-4bc9-84e5-e59532a23783 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 16 22:01:01.196008 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-7aae28f8-d268-4412-995c-caf5effae206 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:01:01.199349 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-7aae28f8-d268-4412-995c-caf5effae206 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] [instance: 96ec3b45-063c-4bc9-84e5-e59532a23783] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:01:01.202820 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7aae28f8-d268-4412-995c-caf5effae206 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:01:01.203339 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7aae28f8-d268-4412-995c-caf5effae206 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:01:01.204097 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7aae28f8-d268-4412-995c-caf5effae206 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:01:01.204539 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7aae28f8-d268-4412-995c-caf5effae206 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:01:01.337636 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-7aae28f8-d268-4412-995c-caf5effae206 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd980641608134eadb72afa6fddb6adeb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9934bd813edb4e55a2d7c61207a9c46d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:01:01.345194 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-7aae28f8-d268-4412-995c-caf5effae206 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] 10.222.0.16 "GET /compute/v2.1/servers/96ec3b45-063c-4bc9-84e5-e59532a23783" status: 200 len: 1511 microversion: 2.1 time: 0.154243 Jul 16 22:01:01.345776 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3236/6477] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:01:01 2022] GET /compute/v2.1/servers/96ec3b45-063c-4bc9-84e5-e59532a23783 => generated 1511 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:01:02.362757 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-7302976f-35b9-49ae-9d9c-4edf76f1d766 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:01:02.363466 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-7302976f-35b9-49ae-9d9c-4edf76f1d766 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] [instance: 96ec3b45-063c-4bc9-84e5-e59532a23783] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:01:02.371100 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7302976f-35b9-49ae-9d9c-4edf76f1d766 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:01:02.371779 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7302976f-35b9-49ae-9d9c-4edf76f1d766 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:01:02.372564 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7302976f-35b9-49ae-9d9c-4edf76f1d766 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:01:02.373018 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7302976f-35b9-49ae-9d9c-4edf76f1d766 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:01:02.507659 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-7302976f-35b9-49ae-9d9c-4edf76f1d766 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd980641608134eadb72afa6fddb6adeb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9934bd813edb4e55a2d7c61207a9c46d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:01:02.515439 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-7302976f-35b9-49ae-9d9c-4edf76f1d766 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] 10.222.0.16 "GET /compute/v2.1/servers/96ec3b45-063c-4bc9-84e5-e59532a23783" status: 200 len: 1511 microversion: 2.1 time: 0.156641 Jul 16 22:01:02.516020 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3242/6478] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:01:02 2022] GET /compute/v2.1/servers/96ec3b45-063c-4bc9-84e5-e59532a23783 => generated 1511 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:01:03.071130 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-27b6cada-1514-4d9f-9f17-03051ce81ea5 req-4a9678fe-30ba-4d39-a98f-d923d9e47d12 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "96ec3b45-063c-4bc9-84e5-e59532a23783", "name": "network-vif-deleted", "tag": "2bcfc300-6586-4eae-a738-286a969a6f27"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:01:03.080469 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-27b6cada-1514-4d9f-9f17-03051ce81ea5 req-4a9678fe-30ba-4d39-a98f-d923d9e47d12 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:01:03.081564 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-27b6cada-1514-4d9f-9f17-03051ce81ea5 req-4a9678fe-30ba-4d39-a98f-d923d9e47d12 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:01:03.096672 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-27b6cada-1514-4d9f-9f17-03051ce81ea5 req-4a9678fe-30ba-4d39-a98f-d923d9e47d12 service nova] Creating event network-vif-deleted:2bcfc300-6586-4eae-a738-286a969a6f27 for instance 96ec3b45-063c-4bc9-84e5-e59532a23783 on nch2-master-1 Jul 16 22:01:03.102929 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-27b6cada-1514-4d9f-9f17-03051ce81ea5 req-4a9678fe-30ba-4d39-a98f-d923d9e47d12 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.036576 Jul 16 22:01:03.107232 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3237/6479] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 22:01:03 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 16 22:01:03.533181 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-ffb98f42-2f49-4f1c-b5ff-45cfa63f94fb tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:01:03.533808 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-ffb98f42-2f49-4f1c-b5ff-45cfa63f94fb tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] [instance: 96ec3b45-063c-4bc9-84e5-e59532a23783] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:01:03.540041 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ffb98f42-2f49-4f1c-b5ff-45cfa63f94fb tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:01:03.540543 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ffb98f42-2f49-4f1c-b5ff-45cfa63f94fb tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:01:03.541311 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ffb98f42-2f49-4f1c-b5ff-45cfa63f94fb tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:01:03.541765 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ffb98f42-2f49-4f1c-b5ff-45cfa63f94fb tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:01:03.573856 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ffb98f42-2f49-4f1c-b5ff-45cfa63f94fb tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:01:03.574326 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ffb98f42-2f49-4f1c-b5ff-45cfa63f94fb tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:01:03.574889 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.objects.instance [None req-ffb98f42-2f49-4f1c-b5ff-45cfa63f94fb tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lazy-loading 'fault' on Instance uuid 96ec3b45-063c-4bc9-84e5-e59532a23783 {{(pid=138681) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 22:01:03.619924 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-ffb98f42-2f49-4f1c-b5ff-45cfa63f94fb tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd980641608134eadb72afa6fddb6adeb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9934bd813edb4e55a2d7c61207a9c46d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:01:03.627949 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-ffb98f42-2f49-4f1c-b5ff-45cfa63f94fb tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] 10.222.0.16 "GET /compute/v2.1/servers/96ec3b45-063c-4bc9-84e5-e59532a23783" status: 200 len: 1305 microversion: 2.1 time: 0.098990 Jul 16 22:01:03.628529 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3243/6480] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:01:03 2022] GET /compute/v2.1/servers/96ec3b45-063c-4bc9-84e5-e59532a23783 => generated 1305 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:01:04.645348 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-3dae4c85-5365-481f-8f49-7be6af80cee3 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:01:04.645993 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-3dae4c85-5365-481f-8f49-7be6af80cee3 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] [instance: 96ec3b45-063c-4bc9-84e5-e59532a23783] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:01:04.653212 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3dae4c85-5365-481f-8f49-7be6af80cee3 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:01:04.653660 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3dae4c85-5365-481f-8f49-7be6af80cee3 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:01:04.654416 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3dae4c85-5365-481f-8f49-7be6af80cee3 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:01:04.654855 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3dae4c85-5365-481f-8f49-7be6af80cee3 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:01:04.657505 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:01:04.669365 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-3dae4c85-5365-481f-8f49-7be6af80cee3 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] HTTP exception thrown: Instance 96ec3b45-063c-4bc9-84e5-e59532a23783 could not be found. Jul 16 22:01:04.670978 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-3dae4c85-5365-481f-8f49-7be6af80cee3 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Request method failure captured: Jul 16 22:01:04.670978 ncdv-master-1 devstack@n-api.service[138680]: request: GET /compute/v2.1/servers/96ec3b45-063c-4bc9-84e5-e59532a23783 HTTP/1.1 Jul 16 22:01:04.670978 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 22:01:04.670978 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 22:01:04.670978 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 22:01:04.670978 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 0 Jul 16 22:01:04.670978 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 22:01:04.670978 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 22:01:04.670978 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 22:01:04.670978 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 22:01:04.670978 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0zSsUzdSjZhOJ5p3MtpGuccUhz6ijQBptmTpiHwkbuueH63QrJFni4S3TCMczYSSsep48c_tRK5S5WYKIpZcmMu7U6ganJXIpii2kyCWaoJnA-DILUTN5meVn-RWIs-0FO80lJw5P-yjrA_2d1TFlYQVDiqNuMYea_ZM7QogjabwIu8 Jul 16 22:01:04.670978 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 22:01:04.670978 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 22:01:04.670978 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 22:01:04.670978 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 22:01:04.670978 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 22:01:04.670978 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 22:01:04.670978 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: 9934bd813edb4e55a2d7c61207a9c46d Jul 16 22:01:04.670978 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-ServerDiskConfigTestJSON-184551575 Jul 16 22:01:04.670978 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 22:01:04.670978 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 22:01:04.670978 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/9934bd813edb4e55a2d7c61207a9c46d"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/9934bd813edb4e55a2d7c61207a9c46d"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_9934bd813edb4e55a2d7c61207a9c46d"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/9934bd813edb4e55a2d7c61207a9c46d"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 22:01:04.670978 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-ServerDiskConfigTestJSON-184551575 Jul 16 22:01:04.670978 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: 9934bd813edb4e55a2d7c61207a9c46d Jul 16 22:01:04.670978 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-ServerDiskConfigTestJSON-184551575 Jul 16 22:01:04.670978 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-ServerDiskConfigTestJSON-184551575-project Jul 16 22:01:04.670978 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 22:01:04.670978 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 22:01:04.670978 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: d980641608134eadb72afa6fddb6adeb Jul 16 22:01:04.670978 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-ServerDiskConfigTestJSON-184551575-project Jul 16 22:01:04.670978 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 22:01:04.670978 ncdv-master-1 devstack@n-api.service[138680]: exception: Instance 96ec3b45-063c-4bc9-84e5-e59532a23783 could not be found. Jul 16 22:01:04.670978 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': '96ec3b45-063c-4bc9-84e5-e59532a23783'} Jul 16 22:01:04.670978 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 22:01:04.670978 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 22:01:04.670978 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 22:01:04.670978 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 22:01:04.670978 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 22:01:04.670978 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 22:01:04.670978 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 22:01:04.670978 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 22:01:04.670978 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 22:01:04.670978 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 22:01:04.670978 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 96ec3b45-063c-4bc9-84e5-e59532a23783 could not be found. Jul 16 22:01:04.670978 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 22:01:04.670978 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 22:01:04.670978 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 22:01:04.670978 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 22:01:04.670978 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 22:01:04.670978 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 22:01:04.670978 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 22:01:04.670978 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 22:01:04.670978 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 22:01:04.670978 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 22:01:04.670978 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 22:01:04.670978 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 22:01:04.670978 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 22:01:04.670978 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 22:01:04.670978 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 22:01:04.670978 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 22:01:04.670978 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 96ec3b45-063c-4bc9-84e5-e59532a23783 could not be found. Jul 16 22:01:04.670978 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 22:01:04.670978 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 22:01:04.670978 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 22:01:04.675541 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 22:01:04.675541 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 22:01:04.675541 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 22:01:04.675541 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 22:01:04.675541 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 22:01:04.675541 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 96ec3b45-063c-4bc9-84e5-e59532a23783 could not be found. Jul 16 22:01:04.675541 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 22:01:04.675541 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-3dae4c85-5365-481f-8f49-7be6af80cee3 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Returning 404 to user: Instance 96ec3b45-063c-4bc9-84e5-e59532a23783 could not be found. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 22:01:04.675541 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-3dae4c85-5365-481f-8f49-7be6af80cee3 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] 10.222.0.16 "GET /compute/v2.1/servers/96ec3b45-063c-4bc9-84e5-e59532a23783" status: 404 len: 111 microversion: 2.1 time: 0.031362 Jul 16 22:01:04.675541 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3238/6481] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:01:04 2022] GET /compute/v2.1/servers/96ec3b45-063c-4bc9-84e5-e59532a23783 => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 22:01:04.681588 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-92e8d8e1-b1fc-479b-baac-fc7622caf30b tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Calling method '.version_select at 0x7fdd8fe6c790>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:01:04.745262 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-92e8d8e1-b1fc-479b-baac-fc7622caf30b tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] 10.222.0.16 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1008228667" status: 202 len: 0 microversion: 2.1 time: 0.067912 Jul 16 22:01:04.745844 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3244/6482] 10.222.0.16 () {58 vars in 1220 bytes} [Sat Jul 16 22:01:04 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1008228667 => generated 0 bytes in 69 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 22:01:05.478682 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-e6a22a35-ffa4-47cd-9522-fc811f85aef2 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:01:05.483092 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-e6a22a35-ffa4-47cd-9522-fc811f85aef2 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] [instance: 853637be-2161-4b9e-903a-2cbce6ff6881] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:01:05.485658 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e6a22a35-ffa4-47cd-9522-fc811f85aef2 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:01:05.486114 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e6a22a35-ffa4-47cd-9522-fc811f85aef2 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:01:05.486883 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e6a22a35-ffa4-47cd-9522-fc811f85aef2 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:01:05.487363 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e6a22a35-ffa4-47cd-9522-fc811f85aef2 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:01:05.500622 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-e6a22a35-ffa4-47cd-9522-fc811f85aef2 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] HTTP exception thrown: Instance 853637be-2161-4b9e-903a-2cbce6ff6881 could not be found. Jul 16 22:01:05.502153 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-e6a22a35-ffa4-47cd-9522-fc811f85aef2 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Request method failure captured: Jul 16 22:01:05.502153 ncdv-master-1 devstack@n-api.service[138680]: request: DELETE /compute/v2.1/servers/853637be-2161-4b9e-903a-2cbce6ff6881 HTTP/1.1 Jul 16 22:01:05.502153 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 22:01:05.502153 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 22:01:05.502153 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 22:01:05.502153 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 0 Jul 16 22:01:05.502153 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 22:01:05.502153 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 22:01:05.502153 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 22:01:05.502153 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 22:01:05.502153 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0zSsUzdSjZhOJ5p3MtpGuccUhz6ijQBptmTpiHwkbuueH63QrJFni4S3TCMczYSSsep48c_tRK5S5WYKIpZcmMu7U6ganJXIpii2kyCWaoJnA-DILUTN5meVn-RWIs-0FO80lJw5P-yjrA_2d1TFlYQVDiqNuMYea_ZM7QogjabwIu8 Jul 16 22:01:05.502153 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 22:01:05.502153 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 22:01:05.502153 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 22:01:05.502153 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 22:01:05.502153 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 22:01:05.502153 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 22:01:05.502153 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: 9934bd813edb4e55a2d7c61207a9c46d Jul 16 22:01:05.502153 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-ServerDiskConfigTestJSON-184551575 Jul 16 22:01:05.502153 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 22:01:05.502153 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 22:01:05.502153 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/9934bd813edb4e55a2d7c61207a9c46d"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/9934bd813edb4e55a2d7c61207a9c46d"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_9934bd813edb4e55a2d7c61207a9c46d"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/9934bd813edb4e55a2d7c61207a9c46d"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 22:01:05.502153 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-ServerDiskConfigTestJSON-184551575 Jul 16 22:01:05.502153 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: 9934bd813edb4e55a2d7c61207a9c46d Jul 16 22:01:05.502153 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-ServerDiskConfigTestJSON-184551575 Jul 16 22:01:05.502153 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-ServerDiskConfigTestJSON-184551575-project Jul 16 22:01:05.502153 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 22:01:05.502153 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 22:01:05.502153 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: d980641608134eadb72afa6fddb6adeb Jul 16 22:01:05.502153 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-ServerDiskConfigTestJSON-184551575-project Jul 16 22:01:05.502153 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 22:01:05.502153 ncdv-master-1 devstack@n-api.service[138680]: exception: Instance 853637be-2161-4b9e-903a-2cbce6ff6881 could not be found. Jul 16 22:01:05.502153 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': '853637be-2161-4b9e-903a-2cbce6ff6881'} Jul 16 22:01:05.502153 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 22:01:05.502153 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 22:01:05.502153 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 22:01:05.502153 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 22:01:05.502153 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 22:01:05.502153 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 22:01:05.502153 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 22:01:05.502153 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 22:01:05.502153 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 22:01:05.502153 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 22:01:05.502153 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 853637be-2161-4b9e-903a-2cbce6ff6881 could not be found. Jul 16 22:01:05.502153 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 22:01:05.502153 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 22:01:05.502153 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 22:01:05.502153 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 22:01:05.502153 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 22:01:05.502153 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 22:01:05.502153 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 22:01:05.502153 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 22:01:05.502153 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 22:01:05.502153 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 22:01:05.502153 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Jul 16 22:01:05.502153 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Jul 16 22:01:05.502153 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Jul 16 22:01:05.502153 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Jul 16 22:01:05.502153 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 22:01:05.502153 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 22:01:05.502153 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 22:01:05.502153 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 22:01:05.511794 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 853637be-2161-4b9e-903a-2cbce6ff6881 could not be found. Jul 16 22:01:05.511794 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 22:01:05.511794 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 22:01:05.511794 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 22:01:05.511794 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 22:01:05.511794 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 22:01:05.511794 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 22:01:05.511794 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 22:01:05.511794 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 22:01:05.511794 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 853637be-2161-4b9e-903a-2cbce6ff6881 could not be found. Jul 16 22:01:05.511794 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 22:01:05.511794 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-e6a22a35-ffa4-47cd-9522-fc811f85aef2 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Returning 404 to user: Instance 853637be-2161-4b9e-903a-2cbce6ff6881 could not be found. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 22:01:05.511794 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-e6a22a35-ffa4-47cd-9522-fc811f85aef2 tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] 10.222.0.16 "DELETE /compute/v2.1/servers/853637be-2161-4b9e-903a-2cbce6ff6881" status: 404 len: 111 microversion: 2.1 time: 0.030332 Jul 16 22:01:05.511794 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3239/6483] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 22:01:05 2022] DELETE /compute/v2.1/servers/853637be-2161-4b9e-903a-2cbce6ff6881 => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 22:01:05.514107 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-3441ffb0-9670-4c27-bda4-e005934dad1a tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:01:05.514717 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-3441ffb0-9670-4c27-bda4-e005934dad1a tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] [instance: 853637be-2161-4b9e-903a-2cbce6ff6881] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:01:05.522387 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3441ffb0-9670-4c27-bda4-e005934dad1a tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:01:05.523144 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3441ffb0-9670-4c27-bda4-e005934dad1a tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:01:05.523930 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3441ffb0-9670-4c27-bda4-e005934dad1a tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:01:05.524382 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3441ffb0-9670-4c27-bda4-e005934dad1a tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:01:05.528572 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:01:05.538160 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-3441ffb0-9670-4c27-bda4-e005934dad1a tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] HTTP exception thrown: Instance 853637be-2161-4b9e-903a-2cbce6ff6881 could not be found. Jul 16 22:01:05.540514 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-3441ffb0-9670-4c27-bda4-e005934dad1a tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Request method failure captured: Jul 16 22:01:05.540514 ncdv-master-1 devstack@n-api.service[138681]: request: GET /compute/v2.1/servers/853637be-2161-4b9e-903a-2cbce6ff6881 HTTP/1.1 Jul 16 22:01:05.540514 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 22:01:05.540514 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 22:01:05.540514 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 22:01:05.540514 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 22:01:05.540514 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 22:01:05.540514 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 22:01:05.540514 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 22:01:05.540514 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 22:01:05.540514 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0zSsUzdSjZhOJ5p3MtpGuccUhz6ijQBptmTpiHwkbuueH63QrJFni4S3TCMczYSSsep48c_tRK5S5WYKIpZcmMu7U6ganJXIpii2kyCWaoJnA-DILUTN5meVn-RWIs-0FO80lJw5P-yjrA_2d1TFlYQVDiqNuMYea_ZM7QogjabwIu8 Jul 16 22:01:05.540514 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 22:01:05.540514 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 22:01:05.540514 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 22:01:05.540514 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 22:01:05.540514 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 22:01:05.540514 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 22:01:05.540514 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: 9934bd813edb4e55a2d7c61207a9c46d Jul 16 22:01:05.540514 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-ServerDiskConfigTestJSON-184551575 Jul 16 22:01:05.540514 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 22:01:05.540514 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 22:01:05.540514 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/9934bd813edb4e55a2d7c61207a9c46d"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/9934bd813edb4e55a2d7c61207a9c46d"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_9934bd813edb4e55a2d7c61207a9c46d"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/9934bd813edb4e55a2d7c61207a9c46d"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 22:01:05.540514 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-ServerDiskConfigTestJSON-184551575 Jul 16 22:01:05.540514 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: 9934bd813edb4e55a2d7c61207a9c46d Jul 16 22:01:05.540514 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-ServerDiskConfigTestJSON-184551575 Jul 16 22:01:05.540514 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-ServerDiskConfigTestJSON-184551575-project Jul 16 22:01:05.540514 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 22:01:05.540514 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 22:01:05.540514 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: d980641608134eadb72afa6fddb6adeb Jul 16 22:01:05.540514 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-ServerDiskConfigTestJSON-184551575-project Jul 16 22:01:05.540514 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 22:01:05.540514 ncdv-master-1 devstack@n-api.service[138681]: exception: Instance 853637be-2161-4b9e-903a-2cbce6ff6881 could not be found. Jul 16 22:01:05.540514 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': '853637be-2161-4b9e-903a-2cbce6ff6881'} Jul 16 22:01:05.540514 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 22:01:05.540514 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 22:01:05.540514 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 22:01:05.540514 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 22:01:05.540514 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 22:01:05.540514 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 22:01:05.540514 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 22:01:05.540514 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 22:01:05.540514 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 22:01:05.540514 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 22:01:05.540514 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 853637be-2161-4b9e-903a-2cbce6ff6881 could not be found. Jul 16 22:01:05.540514 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 22:01:05.540514 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 22:01:05.540514 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 22:01:05.540514 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 22:01:05.540514 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 22:01:05.540514 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 22:01:05.540514 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 22:01:05.540514 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 22:01:05.540514 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 22:01:05.540514 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 22:01:05.540514 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 22:01:05.540514 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 22:01:05.540514 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 22:01:05.540514 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 22:01:05.540514 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 22:01:05.540514 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 22:01:05.540514 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 853637be-2161-4b9e-903a-2cbce6ff6881 could not be found. Jul 16 22:01:05.540514 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 22:01:05.540514 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 22:01:05.540514 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 22:01:05.548636 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 22:01:05.548636 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 22:01:05.548636 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 22:01:05.548636 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 22:01:05.548636 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 22:01:05.548636 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 853637be-2161-4b9e-903a-2cbce6ff6881 could not be found. Jul 16 22:01:05.548636 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 22:01:05.548636 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-3441ffb0-9670-4c27-bda4-e005934dad1a tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] Returning 404 to user: Instance 853637be-2161-4b9e-903a-2cbce6ff6881 could not be found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 22:01:05.548636 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-3441ffb0-9670-4c27-bda4-e005934dad1a tempest-ServerDiskConfigTestJSON-184551575 tempest-ServerDiskConfigTestJSON-184551575-project] 10.222.0.16 "GET /compute/v2.1/servers/853637be-2161-4b9e-903a-2cbce6ff6881" status: 404 len: 111 microversion: 2.1 time: 0.031332 Jul 16 22:01:05.548636 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3245/6484] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:01:05 2022] GET /compute/v2.1/servers/853637be-2161-4b9e-903a-2cbce6ff6881 => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 22:01:16.215306 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_api:SELECT=38,nova_api:UPDATE=1 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:01:16.337960 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-4df54445-c74a-468d-ab81-81fb867d7407 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Action: 'create', calling method: .version_select at 0x7fdda2960ee0>, body: {"keypair": {"name": "tempest-keypair-26228548", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBKny/cbzYh5Z8/2Sq36c7gik1UD9TvpkeANtnUlRT40VizKas6fp1l+pIWiaPCXQSoM1YcH0RDCoKXjyatd+yV0+ND/OS3rxQqtNRInuIk+gofHG5fc9frvoMoeowDy6Aw=="}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:01:16.352513 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-4df54445-c74a-468d-ab81-81fb867d7407 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Getting quotas for project fd21231d9af34d9aa4f9e03229b49a30. Resources: {'key_pairs'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 22:01:16.358134 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-4df54445-c74a-468d-ab81-81fb867d7407 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Getting quotas for user 718f6de3b1a1453d9579d421a574691e and project fd21231d9af34d9aa4f9e03229b49a30. Resources: {'key_pairs'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 22:01:16.371835 ncdv-master-1 devstack@n-api.service[138680]: ERROR oslo.messaging._drivers.impl_rabbit [None req-4df54445-c74a-468d-ab81-81fb867d7407 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [9e1e01de-99c7-4dc5-ab92-351db8ba54ae] AMQP server on 10.222.0.16:5672 is unreachable: [Errno 104] Connection reset by peer. Trying again in 1 seconds.: ConnectionResetError: [Errno 104] Connection reset by peer Jul 16 22:01:17.382117 ncdv-master-1 devstack@n-api.service[138680]: INFO oslo.messaging._drivers.impl_rabbit [None req-4df54445-c74a-468d-ab81-81fb867d7407 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [9e1e01de-99c7-4dc5-ab92-351db8ba54ae] Reconnected to AMQP server on 10.222.0.16:5672 via [amqp] client with port 58058. Jul 16 22:01:17.458108 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-4df54445-c74a-468d-ab81-81fb867d7407 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "POST /compute/v2.1/os-keypairs" status: 200 len: 384 microversion: 2.1 time: 1.250170 Jul 16 22:01:17.458659 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3240/6485] 10.222.0.16 () {60 vars in 1158 bytes} [Sat Jul 16 22:01:16 2022] POST /compute/v2.1/os-keypairs => generated 384 bytes in 1251 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 22:01:19.764743 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-3d885b28-f388-4c97-81ce-1bf7b734d705 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerActionsTestJSON-server-987432969", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "b3593d62-fee4-4445-83ed-ea9589fa3fcd"}], "security_groups": [{"name": "tempest-securitygroup--389060960"}], "key_name": "tempest-keypair-26228548", "user_data": "}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:01:19.772751 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_api:SELECT=34,nova_api:DELETE=1 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:01:19.887176 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.network.neutron [None req-3d885b28-f388-4c97-81ce-1bf7b734d705 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] validate_networks() for [('b3593d62-fee4-4445-83ed-ea9589fa3fcd', None, None, None, None, None)] {{(pid=138681) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 16 22:01:20.286667 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-3d885b28-f388-4c97-81ce-1bf7b734d705 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Flavor limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 16 22:01:20.287149 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-3d885b28-f388-4c97-81ce-1bf7b734d705 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Image limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 16 22:01:20.287645 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-3d885b28-f388-4c97-81ce-1bf7b734d705 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Flavor pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 16 22:01:20.288072 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-3d885b28-f388-4c97-81ce-1bf7b734d705 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Image pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 16 22:01:20.288504 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-3d885b28-f388-4c97-81ce-1bf7b734d705 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 16 22:01:20.298279 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3d885b28-f388-4c97-81ce-1bf7b734d705 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:01:20.298750 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3d885b28-f388-4c97-81ce-1bf7b734d705 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:01:20.301391 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3d885b28-f388-4c97-81ce-1bf7b734d705 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:01:20.301391 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3d885b28-f388-4c97-81ce-1bf7b734d705 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:01:20.317026 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-3d885b28-f388-4c97-81ce-1bf7b734d705 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Getting quotas for project fd21231d9af34d9aa4f9e03229b49a30. Resources: {'ram', 'instances', 'cores'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 22:01:20.322537 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-3d885b28-f388-4c97-81ce-1bf7b734d705 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Getting quotas for user 718f6de3b1a1453d9579d421a574691e and project fd21231d9af34d9aa4f9e03229b49a30. Resources: {'ram', 'instances', 'cores'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 22:01:20.335385 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-3d885b28-f388-4c97-81ce-1bf7b734d705 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Going to run 1 instances... {{(pid=138681) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 16 22:01:20.338241 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-3d885b28-f388-4c97-81ce-1bf7b734d705 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: ccef2dfa-df64-4f44-96db-85598e217ba8] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138681) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 16 22:01:20.453872 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-3d885b28-f388-4c97-81ce-1bf7b734d705 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 408 microversion: 2.1 time: 0.694371 Jul 16 22:01:20.454441 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3246/6486] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 22:01:19 2022] POST /compute/v2.1/servers => generated 408 bytes in 696 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 16 22:01:20.466217 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-a866c57a-ac2d-45a4-823c-61daf98a145c tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:01:20.466840 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-a866c57a-ac2d-45a4-823c-61daf98a145c tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: ccef2dfa-df64-4f44-96db-85598e217ba8] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:01:20.527228 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-a866c57a-ac2d-45a4-823c-61daf98a145c tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '718f6de3b1a1453d9579d421a574691e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd21231d9af34d9aa4f9e03229b49a30', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:01:20.534120 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-a866c57a-ac2d-45a4-823c-61daf98a145c tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/ccef2dfa-df64-4f44-96db-85598e217ba8" status: 200 len: 1236 microversion: 2.1 time: 0.073227 Jul 16 22:01:20.534709 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3241/6487] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:01:20 2022] GET /compute/v2.1/servers/ccef2dfa-df64-4f44-96db-85598e217ba8 => generated 1236 bytes in 75 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:01:21.552752 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-7ad6f67c-56a0-4662-b070-f9aca92dffde tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:01:21.553431 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-7ad6f67c-56a0-4662-b070-f9aca92dffde tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: ccef2dfa-df64-4f44-96db-85598e217ba8] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:01:21.561183 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7ad6f67c-56a0-4662-b070-f9aca92dffde tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:01:21.561709 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7ad6f67c-56a0-4662-b070-f9aca92dffde tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:01:21.562423 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7ad6f67c-56a0-4662-b070-f9aca92dffde tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:01:21.562882 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7ad6f67c-56a0-4662-b070-f9aca92dffde tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:01:21.629808 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-7ad6f67c-56a0-4662-b070-f9aca92dffde tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '718f6de3b1a1453d9579d421a574691e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd21231d9af34d9aa4f9e03229b49a30', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:01:21.637642 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-7ad6f67c-56a0-4662-b070-f9aca92dffde tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/ccef2dfa-df64-4f44-96db-85598e217ba8" status: 200 len: 1232 microversion: 2.1 time: 0.089790 Jul 16 22:01:21.638215 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3247/6488] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:01:21 2022] GET /compute/v2.1/servers/ccef2dfa-df64-4f44-96db-85598e217ba8 => generated 1232 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:01:22.654226 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-b8f9d780-7c5d-45ee-8c9b-9ec367623619 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:01:22.654841 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-b8f9d780-7c5d-45ee-8c9b-9ec367623619 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: ccef2dfa-df64-4f44-96db-85598e217ba8] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:01:22.662590 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b8f9d780-7c5d-45ee-8c9b-9ec367623619 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:01:22.663394 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b8f9d780-7c5d-45ee-8c9b-9ec367623619 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:01:22.663891 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b8f9d780-7c5d-45ee-8c9b-9ec367623619 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:01:22.664332 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b8f9d780-7c5d-45ee-8c9b-9ec367623619 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:01:22.741003 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-b8f9d780-7c5d-45ee-8c9b-9ec367623619 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '718f6de3b1a1453d9579d421a574691e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd21231d9af34d9aa4f9e03229b49a30', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:01:22.749078 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-b8f9d780-7c5d-45ee-8c9b-9ec367623619 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/ccef2dfa-df64-4f44-96db-85598e217ba8" status: 200 len: 1306 microversion: 2.1 time: 0.098903 Jul 16 22:01:22.749655 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3242/6489] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:01:22 2022] GET /compute/v2.1/servers/ccef2dfa-df64-4f44-96db-85598e217ba8 => generated 1306 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:01:23.766621 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-ae6fdd6b-09d8-483a-ac15-949f3ea9a581 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:01:23.767204 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-ae6fdd6b-09d8-483a-ac15-949f3ea9a581 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: ccef2dfa-df64-4f44-96db-85598e217ba8] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:01:23.774917 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ae6fdd6b-09d8-483a-ac15-949f3ea9a581 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:01:23.775499 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ae6fdd6b-09d8-483a-ac15-949f3ea9a581 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:01:23.776905 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ae6fdd6b-09d8-483a-ac15-949f3ea9a581 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:01:23.777358 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ae6fdd6b-09d8-483a-ac15-949f3ea9a581 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:01:23.982779 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-ae6fdd6b-09d8-483a-ac15-949f3ea9a581 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '718f6de3b1a1453d9579d421a574691e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd21231d9af34d9aa4f9e03229b49a30', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:01:23.985837 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-ae6fdd6b-09d8-483a-ac15-949f3ea9a581 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/ccef2dfa-df64-4f44-96db-85598e217ba8" status: 200 len: 1361 microversion: 2.1 time: 0.223194 Jul 16 22:01:23.986409 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3248/6490] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:01:23 2022] GET /compute/v2.1/servers/ccef2dfa-df64-4f44-96db-85598e217ba8 => generated 1361 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:01:24.108904 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-6b554e34-6aea-46a8-9917-d8dd050d0ba8 req-9e4588d5-3357-4a6b-aee4-4068d00f0055 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "ccef2dfa-df64-4f44-96db-85598e217ba8", "tag": "cf61d729-6d57-4176-8d56-32b10b837b8f"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:01:24.121544 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-6b554e34-6aea-46a8-9917-d8dd050d0ba8 req-9e4588d5-3357-4a6b-aee4-4068d00f0055 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:01:24.121749 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-6b554e34-6aea-46a8-9917-d8dd050d0ba8 req-9e4588d5-3357-4a6b-aee4-4068d00f0055 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:01:24.133758 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-6b554e34-6aea-46a8-9917-d8dd050d0ba8 req-9e4588d5-3357-4a6b-aee4-4068d00f0055 service nova] Creating event network-changed:cf61d729-6d57-4176-8d56-32b10b837b8f for instance ccef2dfa-df64-4f44-96db-85598e217ba8 on nch2-master-1 Jul 16 22:01:24.140809 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-6b554e34-6aea-46a8-9917-d8dd050d0ba8 req-9e4588d5-3357-4a6b-aee4-4068d00f0055 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.037897 Jul 16 22:01:24.141430 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3243/6491] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 22:01:24 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 22:01:24.972143 ncdv-master-1 devstack@n-api.service[138680]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jul 16 22:01:24.972382 ncdv-master-1 devstack@n-api.service[138680]: Sat Jul 16 22:01:24 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Jul 16 22:01:25.003928 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-da68922d-da81-49a5-82e1-0b601f908fc1 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:01:25.004674 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-da68922d-da81-49a5-82e1-0b601f908fc1 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: ccef2dfa-df64-4f44-96db-85598e217ba8] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:01:25.013388 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-da68922d-da81-49a5-82e1-0b601f908fc1 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:01:25.014109 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-da68922d-da81-49a5-82e1-0b601f908fc1 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:01:25.014897 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-da68922d-da81-49a5-82e1-0b601f908fc1 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:01:25.015412 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-da68922d-da81-49a5-82e1-0b601f908fc1 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:01:25.203651 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-da68922d-da81-49a5-82e1-0b601f908fc1 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '718f6de3b1a1453d9579d421a574691e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd21231d9af34d9aa4f9e03229b49a30', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:01:25.211913 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-da68922d-da81-49a5-82e1-0b601f908fc1 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/ccef2dfa-df64-4f44-96db-85598e217ba8" status: 200 len: 1361 microversion: 2.1 time: 0.211989 Jul 16 22:01:25.217829 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3249/6492] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:01:24 2022] GET /compute/v2.1/servers/ccef2dfa-df64-4f44-96db-85598e217ba8 => generated 1361 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:01:26.231740 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-5336af94-13fc-4410-9995-94ed96362738 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:01:26.232453 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-5336af94-13fc-4410-9995-94ed96362738 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: ccef2dfa-df64-4f44-96db-85598e217ba8] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:01:26.238737 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5336af94-13fc-4410-9995-94ed96362738 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:01:26.239253 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5336af94-13fc-4410-9995-94ed96362738 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:01:26.240029 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5336af94-13fc-4410-9995-94ed96362738 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:01:26.240474 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5336af94-13fc-4410-9995-94ed96362738 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:01:26.371757 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-5336af94-13fc-4410-9995-94ed96362738 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '718f6de3b1a1453d9579d421a574691e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd21231d9af34d9aa4f9e03229b49a30', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:01:26.379307 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-5336af94-13fc-4410-9995-94ed96362738 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/ccef2dfa-df64-4f44-96db-85598e217ba8" status: 200 len: 1361 microversion: 2.1 time: 0.153494 Jul 16 22:01:26.379874 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3244/6493] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:01:26 2022] GET /compute/v2.1/servers/ccef2dfa-df64-4f44-96db-85598e217ba8 => generated 1361 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:01:27.395737 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-314807a4-817a-4a79-a9be-20181aa23644 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:01:27.396825 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-314807a4-817a-4a79-a9be-20181aa23644 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: ccef2dfa-df64-4f44-96db-85598e217ba8] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:01:27.404305 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-314807a4-817a-4a79-a9be-20181aa23644 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:01:27.404768 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-314807a4-817a-4a79-a9be-20181aa23644 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:01:27.405560 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-314807a4-817a-4a79-a9be-20181aa23644 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:01:27.406008 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-314807a4-817a-4a79-a9be-20181aa23644 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:01:27.534873 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-314807a4-817a-4a79-a9be-20181aa23644 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '718f6de3b1a1453d9579d421a574691e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd21231d9af34d9aa4f9e03229b49a30', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:01:27.542988 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-314807a4-817a-4a79-a9be-20181aa23644 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/ccef2dfa-df64-4f44-96db-85598e217ba8" status: 200 len: 1361 microversion: 2.1 time: 0.151162 Jul 16 22:01:27.543575 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3250/6494] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:01:27 2022] GET /compute/v2.1/servers/ccef2dfa-df64-4f44-96db-85598e217ba8 => generated 1361 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:01:28.560555 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-4ffc36b2-0c90-4d3b-8977-8ed07ccd7d81 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:01:28.561292 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-4ffc36b2-0c90-4d3b-8977-8ed07ccd7d81 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: ccef2dfa-df64-4f44-96db-85598e217ba8] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:01:28.568265 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4ffc36b2-0c90-4d3b-8977-8ed07ccd7d81 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:01:28.568725 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4ffc36b2-0c90-4d3b-8977-8ed07ccd7d81 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:01:28.569562 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4ffc36b2-0c90-4d3b-8977-8ed07ccd7d81 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:01:28.570009 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4ffc36b2-0c90-4d3b-8977-8ed07ccd7d81 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:01:28.706104 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-4ffc36b2-0c90-4d3b-8977-8ed07ccd7d81 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '718f6de3b1a1453d9579d421a574691e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd21231d9af34d9aa4f9e03229b49a30', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:01:28.714497 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-4ffc36b2-0c90-4d3b-8977-8ed07ccd7d81 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/ccef2dfa-df64-4f44-96db-85598e217ba8" status: 200 len: 1361 microversion: 2.1 time: 0.158470 Jul 16 22:01:28.715093 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3245/6495] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:01:28 2022] GET /compute/v2.1/servers/ccef2dfa-df64-4f44-96db-85598e217ba8 => generated 1361 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:01:29.732230 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-7ce50443-d955-4e78-b53b-67bc4da77074 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:01:29.732767 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-7ce50443-d955-4e78-b53b-67bc4da77074 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: ccef2dfa-df64-4f44-96db-85598e217ba8] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:01:29.739787 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7ce50443-d955-4e78-b53b-67bc4da77074 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:01:29.740358 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7ce50443-d955-4e78-b53b-67bc4da77074 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:01:29.741057 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7ce50443-d955-4e78-b53b-67bc4da77074 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:01:29.741503 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7ce50443-d955-4e78-b53b-67bc4da77074 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:01:29.875250 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-7ce50443-d955-4e78-b53b-67bc4da77074 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '718f6de3b1a1453d9579d421a574691e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd21231d9af34d9aa4f9e03229b49a30', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:01:29.882720 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-7ce50443-d955-4e78-b53b-67bc4da77074 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/ccef2dfa-df64-4f44-96db-85598e217ba8" status: 200 len: 1361 microversion: 2.1 time: 0.155194 Jul 16 22:01:29.883310 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3251/6496] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:01:29 2022] GET /compute/v2.1/servers/ccef2dfa-df64-4f44-96db-85598e217ba8 => generated 1361 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:01:30.899632 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-4d38bf28-2a7f-49a0-bcdd-8734df017173 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:01:30.900263 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-4d38bf28-2a7f-49a0-bcdd-8734df017173 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: ccef2dfa-df64-4f44-96db-85598e217ba8] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:01:30.903497 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell0:SELECT=6 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:01:30.918626 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4d38bf28-2a7f-49a0-bcdd-8734df017173 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:01:30.918626 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4d38bf28-2a7f-49a0-bcdd-8734df017173 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:01:30.918626 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4d38bf28-2a7f-49a0-bcdd-8734df017173 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:01:30.918626 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4d38bf28-2a7f-49a0-bcdd-8734df017173 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:01:31.048922 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-4d38bf28-2a7f-49a0-bcdd-8734df017173 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '718f6de3b1a1453d9579d421a574691e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd21231d9af34d9aa4f9e03229b49a30', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:01:31.056751 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-4d38bf28-2a7f-49a0-bcdd-8734df017173 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/ccef2dfa-df64-4f44-96db-85598e217ba8" status: 200 len: 1361 microversion: 2.1 time: 0.160898 Jul 16 22:01:31.057328 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3246/6497] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:01:30 2022] GET /compute/v2.1/servers/ccef2dfa-df64-4f44-96db-85598e217ba8 => generated 1361 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:01:31.139828 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-2346d466-a92a-4e42-8f99-ed0b55dd9a43 req-6d09aa1c-dc84-423b-90e9-dbec5e4ff2d8 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "ccef2dfa-df64-4f44-96db-85598e217ba8", "name": "network-vif-plugged", "status": "completed", "tag": "cf61d729-6d57-4176-8d56-32b10b837b8f"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:01:31.144742 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell0:SELECT=2 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:01:31.145460 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell0:SELECT=6 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:01:31.151968 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-2346d466-a92a-4e42-8f99-ed0b55dd9a43 req-6d09aa1c-dc84-423b-90e9-dbec5e4ff2d8 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:01:31.153187 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-2346d466-a92a-4e42-8f99-ed0b55dd9a43 req-6d09aa1c-dc84-423b-90e9-dbec5e4ff2d8 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:01:31.164094 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-2346d466-a92a-4e42-8f99-ed0b55dd9a43 req-6d09aa1c-dc84-423b-90e9-dbec5e4ff2d8 service nova] Creating event network-vif-plugged:cf61d729-6d57-4176-8d56-32b10b837b8f for instance ccef2dfa-df64-4f44-96db-85598e217ba8 on nch2-master-1 Jul 16 22:01:31.169815 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-2346d466-a92a-4e42-8f99-ed0b55dd9a43 req-6d09aa1c-dc84-423b-90e9-dbec5e4ff2d8 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.036236 Jul 16 22:01:31.170471 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3252/6498] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 22:01:31 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 22:01:32.074308 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-0df31591-cdcf-4719-b16e-16e57dc6d819 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:01:32.075485 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-0df31591-cdcf-4719-b16e-16e57dc6d819 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: ccef2dfa-df64-4f44-96db-85598e217ba8] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:01:32.082561 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0df31591-cdcf-4719-b16e-16e57dc6d819 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:01:32.083037 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0df31591-cdcf-4719-b16e-16e57dc6d819 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:01:32.083871 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0df31591-cdcf-4719-b16e-16e57dc6d819 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:01:32.084312 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0df31591-cdcf-4719-b16e-16e57dc6d819 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:01:32.212615 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-0df31591-cdcf-4719-b16e-16e57dc6d819 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '718f6de3b1a1453d9579d421a574691e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd21231d9af34d9aa4f9e03229b49a30', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:01:32.220187 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-0df31591-cdcf-4719-b16e-16e57dc6d819 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/ccef2dfa-df64-4f44-96db-85598e217ba8" status: 200 len: 1361 microversion: 2.1 time: 0.150297 Jul 16 22:01:32.220768 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3247/6499] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:01:32 2022] GET /compute/v2.1/servers/ccef2dfa-df64-4f44-96db-85598e217ba8 => generated 1361 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:01:33.238583 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-5d42e77a-832b-46bf-bdae-5b8139ea180d tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:01:33.239333 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-5d42e77a-832b-46bf-bdae-5b8139ea180d tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: ccef2dfa-df64-4f44-96db-85598e217ba8] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:01:33.246831 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5d42e77a-832b-46bf-bdae-5b8139ea180d tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:01:33.247322 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5d42e77a-832b-46bf-bdae-5b8139ea180d tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:01:33.248327 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5d42e77a-832b-46bf-bdae-5b8139ea180d tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:01:33.248813 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5d42e77a-832b-46bf-bdae-5b8139ea180d tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:01:33.391050 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-5d42e77a-832b-46bf-bdae-5b8139ea180d tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '718f6de3b1a1453d9579d421a574691e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd21231d9af34d9aa4f9e03229b49a30', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:01:33.398669 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-5d42e77a-832b-46bf-bdae-5b8139ea180d tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/ccef2dfa-df64-4f44-96db-85598e217ba8" status: 200 len: 1547 microversion: 2.1 time: 0.165414 Jul 16 22:01:33.399273 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3253/6500] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:01:33 2022] GET /compute/v2.1/servers/ccef2dfa-df64-4f44-96db-85598e217ba8 => generated 1547 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:01:33.415871 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-f7540b56-b2e0-4037-915e-5d9601c6eb69 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:01:33.418188 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-f7540b56-b2e0-4037-915e-5d9601c6eb69 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: ccef2dfa-df64-4f44-96db-85598e217ba8] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:01:33.423994 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f7540b56-b2e0-4037-915e-5d9601c6eb69 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:01:33.424457 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f7540b56-b2e0-4037-915e-5d9601c6eb69 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:01:33.425226 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f7540b56-b2e0-4037-915e-5d9601c6eb69 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:01:33.425731 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f7540b56-b2e0-4037-915e-5d9601c6eb69 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:01:33.508273 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-f7540b56-b2e0-4037-915e-5d9601c6eb69 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/ccef2dfa-df64-4f44-96db-85598e217ba8/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.096233 Jul 16 22:01:33.508903 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3248/6501] 10.222.0.16 () {58 vars in 1274 bytes} [Sat Jul 16 22:01:33 2022] GET /compute/v2.1/servers/ccef2dfa-df64-4f44-96db-85598e217ba8/os-interface => generated 284 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 22:01:34.625858 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-0655dbe8-acdc-4ec3-892b-68339b40fc15 req-e1105f85-50d7-4141-a55c-7504dda9dcb6 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "ccef2dfa-df64-4f44-96db-85598e217ba8", "tag": "cf61d729-6d57-4176-8d56-32b10b837b8f"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:01:34.635459 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-0655dbe8-acdc-4ec3-892b-68339b40fc15 req-e1105f85-50d7-4141-a55c-7504dda9dcb6 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:01:34.635914 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-0655dbe8-acdc-4ec3-892b-68339b40fc15 req-e1105f85-50d7-4141-a55c-7504dda9dcb6 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:01:34.650405 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-0655dbe8-acdc-4ec3-892b-68339b40fc15 req-e1105f85-50d7-4141-a55c-7504dda9dcb6 service nova] Creating event network-changed:cf61d729-6d57-4176-8d56-32b10b837b8f for instance ccef2dfa-df64-4f44-96db-85598e217ba8 on nch2-master-1 Jul 16 22:01:34.656555 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-0655dbe8-acdc-4ec3-892b-68339b40fc15 req-e1105f85-50d7-4141-a55c-7504dda9dcb6 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.034437 Jul 16 22:01:34.657578 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3254/6502] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 22:01:34 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 22:02:10.887787 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_api:SELECT=29,nova_api:INSERT=1 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:02:10.888591 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=39 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:02:10.892431 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-074812b3-68d1-47fb-9a80-8b195bb72ff2 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:02:10.893258 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-074812b3-68d1-47fb-9a80-8b195bb72ff2 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: ccef2dfa-df64-4f44-96db-85598e217ba8] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:02:10.900429 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-074812b3-68d1-47fb-9a80-8b195bb72ff2 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:02:10.900889 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-074812b3-68d1-47fb-9a80-8b195bb72ff2 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:02:10.901656 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-074812b3-68d1-47fb-9a80-8b195bb72ff2 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:02:10.902101 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-074812b3-68d1-47fb-9a80-8b195bb72ff2 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:02:11.035382 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-074812b3-68d1-47fb-9a80-8b195bb72ff2 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '718f6de3b1a1453d9579d421a574691e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd21231d9af34d9aa4f9e03229b49a30', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:02:11.042892 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-074812b3-68d1-47fb-9a80-8b195bb72ff2 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/ccef2dfa-df64-4f44-96db-85598e217ba8" status: 200 len: 1666 microversion: 2.1 time: 0.157309 Jul 16 22:02:11.043624 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3249/6503] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:02:10 2022] GET /compute/v2.1/servers/ccef2dfa-df64-4f44-96db-85598e217ba8 => generated 1666 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:02:11.058861 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_api:SELECT=31,nova_api:INSERT=3 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:02:11.059689 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=42 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:02:11.064119 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-fe0b4102-59ce-4ebf-a8b1-40d85eb7c0ec tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:02:11.064972 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-fe0b4102-59ce-4ebf-a8b1-40d85eb7c0ec tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: ccef2dfa-df64-4f44-96db-85598e217ba8] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:02:11.071423 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fe0b4102-59ce-4ebf-a8b1-40d85eb7c0ec tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:02:11.071892 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fe0b4102-59ce-4ebf-a8b1-40d85eb7c0ec tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:02:11.072703 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fe0b4102-59ce-4ebf-a8b1-40d85eb7c0ec tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:02:11.073183 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fe0b4102-59ce-4ebf-a8b1-40d85eb7c0ec tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:02:11.206304 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-fe0b4102-59ce-4ebf-a8b1-40d85eb7c0ec tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '718f6de3b1a1453d9579d421a574691e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd21231d9af34d9aa4f9e03229b49a30', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:02:11.214228 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-fe0b4102-59ce-4ebf-a8b1-40d85eb7c0ec tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/ccef2dfa-df64-4f44-96db-85598e217ba8" status: 200 len: 1666 microversion: 2.1 time: 0.157994 Jul 16 22:02:11.214793 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3255/6504] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:02:11 2022] GET /compute/v2.1/servers/ccef2dfa-df64-4f44-96db-85598e217ba8 => generated 1666 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:02:11.230829 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-bd2b1c39-a887-487c-9392-f32cce5742f4 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:02:11.231474 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-bd2b1c39-a887-487c-9392-f32cce5742f4 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: ccef2dfa-df64-4f44-96db-85598e217ba8] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:02:11.238239 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-bd2b1c39-a887-487c-9392-f32cce5742f4 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:02:11.238695 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-bd2b1c39-a887-487c-9392-f32cce5742f4 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:02:11.239497 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-bd2b1c39-a887-487c-9392-f32cce5742f4 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:02:11.239942 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-bd2b1c39-a887-487c-9392-f32cce5742f4 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:02:11.371767 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-bd2b1c39-a887-487c-9392-f32cce5742f4 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '718f6de3b1a1453d9579d421a574691e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd21231d9af34d9aa4f9e03229b49a30', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:02:11.379656 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-bd2b1c39-a887-487c-9392-f32cce5742f4 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/ccef2dfa-df64-4f44-96db-85598e217ba8" status: 200 len: 1666 microversion: 2.1 time: 0.152615 Jul 16 22:02:11.380237 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3250/6505] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:02:11 2022] GET /compute/v2.1/servers/ccef2dfa-df64-4f44-96db-85598e217ba8 => generated 1666 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:02:11.396623 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-df56db40-9b51-42f2-88a3-b80b26d61dab tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Action: 'action', calling method: >, body: {"resize": {"flavorRef": "451"}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:02:11.404161 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-df56db40-9b51-42f2-88a3-b80b26d61dab tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: ccef2dfa-df64-4f44-96db-85598e217ba8] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:02:11.404161 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-df56db40-9b51-42f2-88a3-b80b26d61dab tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:02:11.404513 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-df56db40-9b51-42f2-88a3-b80b26d61dab tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:02:11.404960 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-df56db40-9b51-42f2-88a3-b80b26d61dab tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:02:11.405435 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-df56db40-9b51-42f2-88a3-b80b26d61dab tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:02:11.433814 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-df56db40-9b51-42f2-88a3-b80b26d61dab tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Policy check for compute:servers:resize:cross_cell failed with credentials {'is_admin': False, 'user_id': '718f6de3b1a1453d9579d421a574691e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd21231d9af34d9aa4f9e03229b49a30', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:02:11.440757 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-df56db40-9b51-42f2-88a3-b80b26d61dab tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: ccef2dfa-df64-4f44-96db-85598e217ba8] Old instance type m1.heat2, new instance type m1.heat {{(pid=138681) resize /opt/stack/nova/nova/compute/api.py:4166}} Jul 16 22:02:11.446010 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-df56db40-9b51-42f2-88a3-b80b26d61dab tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Flavor limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 16 22:02:11.446450 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-df56db40-9b51-42f2-88a3-b80b26d61dab tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Image limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 16 22:02:11.446941 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-df56db40-9b51-42f2-88a3-b80b26d61dab tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Flavor pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 16 22:02:11.447397 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-df56db40-9b51-42f2-88a3-b80b26d61dab tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Image pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 16 22:02:11.447824 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-df56db40-9b51-42f2-88a3-b80b26d61dab tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 16 22:02:11.609014 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-df56db40-9b51-42f2-88a3-b80b26d61dab tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "POST /compute/v2.1/servers/ccef2dfa-df64-4f44-96db-85598e217ba8/action" status: 202 len: 0 microversion: 2.1 time: 0.217014 Jul 16 22:02:11.609780 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3256/6506] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 22:02:11 2022] POST /compute/v2.1/servers/ccef2dfa-df64-4f44-96db-85598e217ba8/action => generated 0 bytes in 218 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 22:02:11.618110 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-a8f18bc3-c830-454b-8eda-7533ad765b92 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:02:11.618808 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-a8f18bc3-c830-454b-8eda-7533ad765b92 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: ccef2dfa-df64-4f44-96db-85598e217ba8] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:02:11.625040 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a8f18bc3-c830-454b-8eda-7533ad765b92 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:02:11.625494 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a8f18bc3-c830-454b-8eda-7533ad765b92 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:02:11.626260 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a8f18bc3-c830-454b-8eda-7533ad765b92 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:02:11.626696 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a8f18bc3-c830-454b-8eda-7533ad765b92 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:02:11.760060 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-a8f18bc3-c830-454b-8eda-7533ad765b92 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '718f6de3b1a1453d9579d421a574691e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd21231d9af34d9aa4f9e03229b49a30', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:02:11.768483 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-a8f18bc3-c830-454b-8eda-7533ad765b92 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/ccef2dfa-df64-4f44-96db-85598e217ba8" status: 200 len: 1675 microversion: 2.1 time: 0.153769 Jul 16 22:02:11.769074 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3251/6507] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:02:11 2022] GET /compute/v2.1/servers/ccef2dfa-df64-4f44-96db-85598e217ba8 => generated 1675 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:02:12.787171 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-26d73da8-20f0-47d4-9ab9-8091a802671e tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:02:12.787804 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-26d73da8-20f0-47d4-9ab9-8091a802671e tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: ccef2dfa-df64-4f44-96db-85598e217ba8] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:02:12.794113 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-26d73da8-20f0-47d4-9ab9-8091a802671e tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:02:12.794580 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-26d73da8-20f0-47d4-9ab9-8091a802671e tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:02:12.795394 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-26d73da8-20f0-47d4-9ab9-8091a802671e tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:02:12.795849 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-26d73da8-20f0-47d4-9ab9-8091a802671e tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:02:12.935656 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-26d73da8-20f0-47d4-9ab9-8091a802671e tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '718f6de3b1a1453d9579d421a574691e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd21231d9af34d9aa4f9e03229b49a30', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:02:12.943606 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-26d73da8-20f0-47d4-9ab9-8091a802671e tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/ccef2dfa-df64-4f44-96db-85598e217ba8" status: 200 len: 1675 microversion: 2.1 time: 0.161809 Jul 16 22:02:12.944188 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3257/6508] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:02:12 2022] GET /compute/v2.1/servers/ccef2dfa-df64-4f44-96db-85598e217ba8 => generated 1675 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:02:13.961319 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-63b33f7a-c8ae-43ee-b1a7-cc987293f2d8 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:02:13.961928 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-63b33f7a-c8ae-43ee-b1a7-cc987293f2d8 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: ccef2dfa-df64-4f44-96db-85598e217ba8] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:02:13.971461 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-63b33f7a-c8ae-43ee-b1a7-cc987293f2d8 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:02:13.971461 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-63b33f7a-c8ae-43ee-b1a7-cc987293f2d8 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:02:13.971461 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-63b33f7a-c8ae-43ee-b1a7-cc987293f2d8 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:02:13.971461 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-63b33f7a-c8ae-43ee-b1a7-cc987293f2d8 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:02:14.105775 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-63b33f7a-c8ae-43ee-b1a7-cc987293f2d8 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '718f6de3b1a1453d9579d421a574691e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd21231d9af34d9aa4f9e03229b49a30', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:02:14.114148 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-63b33f7a-c8ae-43ee-b1a7-cc987293f2d8 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/ccef2dfa-df64-4f44-96db-85598e217ba8" status: 200 len: 1675 microversion: 2.1 time: 0.157001 Jul 16 22:02:14.114731 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3252/6509] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:02:13 2022] GET /compute/v2.1/servers/ccef2dfa-df64-4f44-96db-85598e217ba8 => generated 1675 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:02:15.131491 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-eb5b8751-a659-4b7d-babf-013496736fc5 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:02:15.132057 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-eb5b8751-a659-4b7d-babf-013496736fc5 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: ccef2dfa-df64-4f44-96db-85598e217ba8] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:02:15.139149 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-eb5b8751-a659-4b7d-babf-013496736fc5 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:02:15.139616 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-eb5b8751-a659-4b7d-babf-013496736fc5 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:02:15.140392 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-eb5b8751-a659-4b7d-babf-013496736fc5 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:02:15.140843 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-eb5b8751-a659-4b7d-babf-013496736fc5 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:02:15.274734 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-eb5b8751-a659-4b7d-babf-013496736fc5 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '718f6de3b1a1453d9579d421a574691e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd21231d9af34d9aa4f9e03229b49a30', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:02:15.282429 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-eb5b8751-a659-4b7d-babf-013496736fc5 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/ccef2dfa-df64-4f44-96db-85598e217ba8" status: 200 len: 1680 microversion: 2.1 time: 0.155635 Jul 16 22:02:15.282993 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3258/6510] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:02:15 2022] GET /compute/v2.1/servers/ccef2dfa-df64-4f44-96db-85598e217ba8 => generated 1680 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:02:16.299773 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-e803b3e9-dc49-406e-a2b7-bdd9ede65c26 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:02:16.300322 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-e803b3e9-dc49-406e-a2b7-bdd9ede65c26 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: ccef2dfa-df64-4f44-96db-85598e217ba8] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:02:16.307408 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e803b3e9-dc49-406e-a2b7-bdd9ede65c26 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:02:16.307852 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e803b3e9-dc49-406e-a2b7-bdd9ede65c26 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:02:16.308642 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e803b3e9-dc49-406e-a2b7-bdd9ede65c26 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:02:16.309087 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e803b3e9-dc49-406e-a2b7-bdd9ede65c26 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:02:16.438858 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-e803b3e9-dc49-406e-a2b7-bdd9ede65c26 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '718f6de3b1a1453d9579d421a574691e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd21231d9af34d9aa4f9e03229b49a30', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:02:16.447031 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-e803b3e9-dc49-406e-a2b7-bdd9ede65c26 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/ccef2dfa-df64-4f44-96db-85598e217ba8" status: 200 len: 1680 microversion: 2.1 time: 0.151452 Jul 16 22:02:16.447618 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3253/6511] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:02:16 2022] GET /compute/v2.1/servers/ccef2dfa-df64-4f44-96db-85598e217ba8 => generated 1680 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:02:17.464868 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-2af00b6c-cd45-450c-8a3f-c2bfc3ad1472 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:02:17.465420 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-2af00b6c-cd45-450c-8a3f-c2bfc3ad1472 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: ccef2dfa-df64-4f44-96db-85598e217ba8] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:02:17.473611 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2af00b6c-cd45-450c-8a3f-c2bfc3ad1472 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:02:17.474071 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2af00b6c-cd45-450c-8a3f-c2bfc3ad1472 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:02:17.474844 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2af00b6c-cd45-450c-8a3f-c2bfc3ad1472 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:02:17.475314 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2af00b6c-cd45-450c-8a3f-c2bfc3ad1472 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:02:17.608507 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-2af00b6c-cd45-450c-8a3f-c2bfc3ad1472 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '718f6de3b1a1453d9579d421a574691e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd21231d9af34d9aa4f9e03229b49a30', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:02:17.616431 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-2af00b6c-cd45-450c-8a3f-c2bfc3ad1472 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/ccef2dfa-df64-4f44-96db-85598e217ba8" status: 200 len: 1680 microversion: 2.1 time: 0.155546 Jul 16 22:02:17.617067 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3259/6512] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:02:17 2022] GET /compute/v2.1/servers/ccef2dfa-df64-4f44-96db-85598e217ba8 => generated 1680 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:02:18.633864 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-744239bd-64e8-4ffc-add3-c63455b5e600 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:02:18.634477 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-744239bd-64e8-4ffc-add3-c63455b5e600 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: ccef2dfa-df64-4f44-96db-85598e217ba8] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:02:18.641579 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-744239bd-64e8-4ffc-add3-c63455b5e600 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:02:18.642048 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-744239bd-64e8-4ffc-add3-c63455b5e600 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:02:18.642931 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-744239bd-64e8-4ffc-add3-c63455b5e600 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:02:18.643600 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-744239bd-64e8-4ffc-add3-c63455b5e600 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:02:18.774642 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-744239bd-64e8-4ffc-add3-c63455b5e600 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '718f6de3b1a1453d9579d421a574691e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd21231d9af34d9aa4f9e03229b49a30', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:02:18.782177 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-744239bd-64e8-4ffc-add3-c63455b5e600 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/ccef2dfa-df64-4f44-96db-85598e217ba8" status: 200 len: 1680 microversion: 2.1 time: 0.152194 Jul 16 22:02:18.782754 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3254/6513] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:02:18 2022] GET /compute/v2.1/servers/ccef2dfa-df64-4f44-96db-85598e217ba8 => generated 1680 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:02:19.800404 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-b8288c77-5f33-4fa5-a4e1-3fe5e5aba0f0 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:02:19.801029 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-b8288c77-5f33-4fa5-a4e1-3fe5e5aba0f0 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: ccef2dfa-df64-4f44-96db-85598e217ba8] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:02:19.808187 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b8288c77-5f33-4fa5-a4e1-3fe5e5aba0f0 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:02:19.808646 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b8288c77-5f33-4fa5-a4e1-3fe5e5aba0f0 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:02:19.809417 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b8288c77-5f33-4fa5-a4e1-3fe5e5aba0f0 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:02:19.809865 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b8288c77-5f33-4fa5-a4e1-3fe5e5aba0f0 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:02:19.945578 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-b8288c77-5f33-4fa5-a4e1-3fe5e5aba0f0 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '718f6de3b1a1453d9579d421a574691e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd21231d9af34d9aa4f9e03229b49a30', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:02:19.953363 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-b8288c77-5f33-4fa5-a4e1-3fe5e5aba0f0 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/ccef2dfa-df64-4f44-96db-85598e217ba8" status: 200 len: 1680 microversion: 2.1 time: 0.157364 Jul 16 22:02:19.953932 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3260/6514] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:02:19 2022] GET /compute/v2.1/servers/ccef2dfa-df64-4f44-96db-85598e217ba8 => generated 1680 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:02:20.971556 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-764083b0-378b-4fe8-b413-4035c756bf07 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:02:20.972179 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-764083b0-378b-4fe8-b413-4035c756bf07 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: ccef2dfa-df64-4f44-96db-85598e217ba8] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:02:20.980256 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-764083b0-378b-4fe8-b413-4035c756bf07 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:02:20.980710 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-764083b0-378b-4fe8-b413-4035c756bf07 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:02:20.981468 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-764083b0-378b-4fe8-b413-4035c756bf07 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:02:20.981906 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-764083b0-378b-4fe8-b413-4035c756bf07 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:02:21.118240 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-764083b0-378b-4fe8-b413-4035c756bf07 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '718f6de3b1a1453d9579d421a574691e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd21231d9af34d9aa4f9e03229b49a30', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:02:21.125835 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-764083b0-378b-4fe8-b413-4035c756bf07 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/ccef2dfa-df64-4f44-96db-85598e217ba8" status: 200 len: 1680 microversion: 2.1 time: 0.158799 Jul 16 22:02:21.126408 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3255/6515] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:02:20 2022] GET /compute/v2.1/servers/ccef2dfa-df64-4f44-96db-85598e217ba8 => generated 1680 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:02:22.143977 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-5e10242b-3c75-42c0-99f9-e2c3e382a76d tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:02:22.144570 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-5e10242b-3c75-42c0-99f9-e2c3e382a76d tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: ccef2dfa-df64-4f44-96db-85598e217ba8] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:02:22.151687 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5e10242b-3c75-42c0-99f9-e2c3e382a76d tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:02:22.152157 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5e10242b-3c75-42c0-99f9-e2c3e382a76d tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:02:22.152929 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5e10242b-3c75-42c0-99f9-e2c3e382a76d tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:02:22.153386 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5e10242b-3c75-42c0-99f9-e2c3e382a76d tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:02:22.283917 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-5e10242b-3c75-42c0-99f9-e2c3e382a76d tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '718f6de3b1a1453d9579d421a574691e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd21231d9af34d9aa4f9e03229b49a30', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:02:22.291944 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-5e10242b-3c75-42c0-99f9-e2c3e382a76d tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/ccef2dfa-df64-4f44-96db-85598e217ba8" status: 200 len: 1680 microversion: 2.1 time: 0.152063 Jul 16 22:02:22.292641 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3261/6516] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:02:22 2022] GET /compute/v2.1/servers/ccef2dfa-df64-4f44-96db-85598e217ba8 => generated 1680 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:02:23.309690 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-38475bbe-6579-49be-818b-5a5175ce5d7a tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:02:23.310305 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-38475bbe-6579-49be-818b-5a5175ce5d7a tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: ccef2dfa-df64-4f44-96db-85598e217ba8] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:02:23.317822 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-38475bbe-6579-49be-818b-5a5175ce5d7a tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:02:23.318277 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-38475bbe-6579-49be-818b-5a5175ce5d7a tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:02:23.319057 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-38475bbe-6579-49be-818b-5a5175ce5d7a tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:02:23.319525 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-38475bbe-6579-49be-818b-5a5175ce5d7a tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:02:23.451845 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-38475bbe-6579-49be-818b-5a5175ce5d7a tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '718f6de3b1a1453d9579d421a574691e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd21231d9af34d9aa4f9e03229b49a30', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:02:23.459645 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-38475bbe-6579-49be-818b-5a5175ce5d7a tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/ccef2dfa-df64-4f44-96db-85598e217ba8" status: 200 len: 1680 microversion: 2.1 time: 0.153877 Jul 16 22:02:23.460220 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3256/6517] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:02:23 2022] GET /compute/v2.1/servers/ccef2dfa-df64-4f44-96db-85598e217ba8 => generated 1680 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:02:24.477181 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-988d5ccf-1a1d-47c1-8d40-e102e3db5b5e tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:02:24.477787 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-988d5ccf-1a1d-47c1-8d40-e102e3db5b5e tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: ccef2dfa-df64-4f44-96db-85598e217ba8] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:02:24.484628 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-988d5ccf-1a1d-47c1-8d40-e102e3db5b5e tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:02:24.485088 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-988d5ccf-1a1d-47c1-8d40-e102e3db5b5e tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:02:24.485856 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-988d5ccf-1a1d-47c1-8d40-e102e3db5b5e tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:02:24.486398 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-988d5ccf-1a1d-47c1-8d40-e102e3db5b5e tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:02:24.616972 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-988d5ccf-1a1d-47c1-8d40-e102e3db5b5e tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '718f6de3b1a1453d9579d421a574691e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd21231d9af34d9aa4f9e03229b49a30', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:02:24.624705 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-988d5ccf-1a1d-47c1-8d40-e102e3db5b5e tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/ccef2dfa-df64-4f44-96db-85598e217ba8" status: 200 len: 1680 microversion: 2.1 time: 0.151388 Jul 16 22:02:24.625272 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3262/6518] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:02:24 2022] GET /compute/v2.1/servers/ccef2dfa-df64-4f44-96db-85598e217ba8 => generated 1680 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:02:25.647690 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-55f99da4-5451-40c8-bd55-c5c87863c290 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:02:25.648309 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-55f99da4-5451-40c8-bd55-c5c87863c290 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: ccef2dfa-df64-4f44-96db-85598e217ba8] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:02:25.655720 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-55f99da4-5451-40c8-bd55-c5c87863c290 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:02:25.656176 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-55f99da4-5451-40c8-bd55-c5c87863c290 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:02:25.656979 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-55f99da4-5451-40c8-bd55-c5c87863c290 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:02:25.657420 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-55f99da4-5451-40c8-bd55-c5c87863c290 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:02:25.789553 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-55f99da4-5451-40c8-bd55-c5c87863c290 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '718f6de3b1a1453d9579d421a574691e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd21231d9af34d9aa4f9e03229b49a30', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:02:25.798626 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-55f99da4-5451-40c8-bd55-c5c87863c290 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/ccef2dfa-df64-4f44-96db-85598e217ba8" status: 200 len: 1680 microversion: 2.1 time: 0.154940 Jul 16 22:02:25.799513 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3257/6519] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:02:25 2022] GET /compute/v2.1/servers/ccef2dfa-df64-4f44-96db-85598e217ba8 => generated 1680 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:02:26.848781 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-b37dcda0-f882-4b34-aa25-5997bf9957fe tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:02:26.849410 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-b37dcda0-f882-4b34-aa25-5997bf9957fe tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: ccef2dfa-df64-4f44-96db-85598e217ba8] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:02:26.856351 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b37dcda0-f882-4b34-aa25-5997bf9957fe tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:02:26.856821 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b37dcda0-f882-4b34-aa25-5997bf9957fe tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:02:26.857594 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b37dcda0-f882-4b34-aa25-5997bf9957fe tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:02:26.858051 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b37dcda0-f882-4b34-aa25-5997bf9957fe tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:02:26.990573 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-b37dcda0-f882-4b34-aa25-5997bf9957fe tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '718f6de3b1a1453d9579d421a574691e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd21231d9af34d9aa4f9e03229b49a30', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:02:26.998183 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-b37dcda0-f882-4b34-aa25-5997bf9957fe tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/ccef2dfa-df64-4f44-96db-85598e217ba8" status: 200 len: 1680 microversion: 2.1 time: 0.153940 Jul 16 22:02:26.998758 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3263/6520] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:02:26 2022] GET /compute/v2.1/servers/ccef2dfa-df64-4f44-96db-85598e217ba8 => generated 1680 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:02:28.015511 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-089797da-3eea-4665-8044-87bf3eec9e3b tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:02:28.016154 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-089797da-3eea-4665-8044-87bf3eec9e3b tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: ccef2dfa-df64-4f44-96db-85598e217ba8] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:02:28.023323 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-089797da-3eea-4665-8044-87bf3eec9e3b tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:02:28.023781 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-089797da-3eea-4665-8044-87bf3eec9e3b tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:02:28.024540 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-089797da-3eea-4665-8044-87bf3eec9e3b tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:02:28.024983 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-089797da-3eea-4665-8044-87bf3eec9e3b tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:02:28.154874 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-089797da-3eea-4665-8044-87bf3eec9e3b tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '718f6de3b1a1453d9579d421a574691e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd21231d9af34d9aa4f9e03229b49a30', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:02:28.163326 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-089797da-3eea-4665-8044-87bf3eec9e3b tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/ccef2dfa-df64-4f44-96db-85598e217ba8" status: 200 len: 1680 microversion: 2.1 time: 0.151695 Jul 16 22:02:28.163904 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3258/6521] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:02:28 2022] GET /compute/v2.1/servers/ccef2dfa-df64-4f44-96db-85598e217ba8 => generated 1680 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 22:02:29.180999 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-24c2975e-a469-44c4-a0e2-2abd6dd3dc3d tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:02:29.181575 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-24c2975e-a469-44c4-a0e2-2abd6dd3dc3d tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: ccef2dfa-df64-4f44-96db-85598e217ba8] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:02:29.188640 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-24c2975e-a469-44c4-a0e2-2abd6dd3dc3d tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:02:29.189105 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-24c2975e-a469-44c4-a0e2-2abd6dd3dc3d tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:02:29.189884 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-24c2975e-a469-44c4-a0e2-2abd6dd3dc3d tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:02:29.190337 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-24c2975e-a469-44c4-a0e2-2abd6dd3dc3d tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:02:29.322706 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-24c2975e-a469-44c4-a0e2-2abd6dd3dc3d tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '718f6de3b1a1453d9579d421a574691e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd21231d9af34d9aa4f9e03229b49a30', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:02:29.330504 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-24c2975e-a469-44c4-a0e2-2abd6dd3dc3d tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/ccef2dfa-df64-4f44-96db-85598e217ba8" status: 200 len: 1680 microversion: 2.1 time: 0.153446 Jul 16 22:02:29.331100 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3264/6522] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:02:29 2022] GET /compute/v2.1/servers/ccef2dfa-df64-4f44-96db-85598e217ba8 => generated 1680 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:02:30.348410 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-50dcf32d-a0ca-4cd9-8981-ae4097312ee1 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:02:30.349066 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-50dcf32d-a0ca-4cd9-8981-ae4097312ee1 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: ccef2dfa-df64-4f44-96db-85598e217ba8] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:02:30.356102 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-50dcf32d-a0ca-4cd9-8981-ae4097312ee1 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:02:30.357533 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-50dcf32d-a0ca-4cd9-8981-ae4097312ee1 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:02:30.357533 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-50dcf32d-a0ca-4cd9-8981-ae4097312ee1 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:02:30.357852 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-50dcf32d-a0ca-4cd9-8981-ae4097312ee1 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:02:30.489745 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-50dcf32d-a0ca-4cd9-8981-ae4097312ee1 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '718f6de3b1a1453d9579d421a574691e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd21231d9af34d9aa4f9e03229b49a30', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:02:30.497321 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-50dcf32d-a0ca-4cd9-8981-ae4097312ee1 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/ccef2dfa-df64-4f44-96db-85598e217ba8" status: 200 len: 1680 microversion: 2.1 time: 0.153318 Jul 16 22:02:30.497886 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3259/6523] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:02:30 2022] GET /compute/v2.1/servers/ccef2dfa-df64-4f44-96db-85598e217ba8 => generated 1680 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:02:31.514599 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-2472044f-3b06-4df9-9a77-82ac71d711f1 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:02:31.515256 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-2472044f-3b06-4df9-9a77-82ac71d711f1 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: ccef2dfa-df64-4f44-96db-85598e217ba8] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:02:31.522319 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2472044f-3b06-4df9-9a77-82ac71d711f1 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:02:31.522786 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2472044f-3b06-4df9-9a77-82ac71d711f1 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:02:31.523593 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2472044f-3b06-4df9-9a77-82ac71d711f1 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:02:31.524049 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2472044f-3b06-4df9-9a77-82ac71d711f1 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:02:31.656911 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-2472044f-3b06-4df9-9a77-82ac71d711f1 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '718f6de3b1a1453d9579d421a574691e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd21231d9af34d9aa4f9e03229b49a30', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:02:31.665295 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-2472044f-3b06-4df9-9a77-82ac71d711f1 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/ccef2dfa-df64-4f44-96db-85598e217ba8" status: 200 len: 1680 microversion: 2.1 time: 0.154654 Jul 16 22:02:31.665864 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3265/6524] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:02:31 2022] GET /compute/v2.1/servers/ccef2dfa-df64-4f44-96db-85598e217ba8 => generated 1680 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:02:32.683541 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-7b1bb417-94ca-45c6-8e73-d548faaeaa2d tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:02:32.684676 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-7b1bb417-94ca-45c6-8e73-d548faaeaa2d tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: ccef2dfa-df64-4f44-96db-85598e217ba8] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:02:32.698178 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7b1bb417-94ca-45c6-8e73-d548faaeaa2d tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:02:32.699073 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7b1bb417-94ca-45c6-8e73-d548faaeaa2d tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:02:32.700460 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7b1bb417-94ca-45c6-8e73-d548faaeaa2d tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:02:32.701255 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7b1bb417-94ca-45c6-8e73-d548faaeaa2d tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:02:32.847495 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-ee88b06e-7fbb-4e8a-a38c-5800854a4369 req-baf3a864-3293-4059-a303-ccf666732104 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "ccef2dfa-df64-4f44-96db-85598e217ba8", "name": "network-vif-unplugged", "status": "completed", "tag": "cf61d729-6d57-4176-8d56-32b10b837b8f"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:02:32.856611 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-ee88b06e-7fbb-4e8a-a38c-5800854a4369 req-baf3a864-3293-4059-a303-ccf666732104 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:02:32.857085 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-ee88b06e-7fbb-4e8a-a38c-5800854a4369 req-baf3a864-3293-4059-a303-ccf666732104 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:02:32.874242 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-ee88b06e-7fbb-4e8a-a38c-5800854a4369 req-baf3a864-3293-4059-a303-ccf666732104 service nova] Creating event network-vif-unplugged:cf61d729-6d57-4176-8d56-32b10b837b8f for instance ccef2dfa-df64-4f44-96db-85598e217ba8 on nch1-master-1 Jul 16 22:02:32.877150 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-7b1bb417-94ca-45c6-8e73-d548faaeaa2d tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '718f6de3b1a1453d9579d421a574691e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd21231d9af34d9aa4f9e03229b49a30', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:02:32.881541 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [req-ee88b06e-7fbb-4e8a-a38c-5800854a4369 req-baf3a864-3293-4059-a303-ccf666732104 service nova] Instance ccef2dfa-df64-4f44-96db-85598e217ba8 is migrating within the same cell, copying events to all relevant hosts: {'nch2-master-1', 'nch1-master-1'} {{(pid=138681) _get_relevant_hosts /opt/stack/nova/nova/compute/api.py:5807}} Jul 16 22:02:32.888021 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-7b1bb417-94ca-45c6-8e73-d548faaeaa2d tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/ccef2dfa-df64-4f44-96db-85598e217ba8" status: 200 len: 1679 microversion: 2.1 time: 0.210031 Jul 16 22:02:32.889354 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3260/6525] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:02:32 2022] GET /compute/v2.1/servers/ccef2dfa-df64-4f44-96db-85598e217ba8 => generated 1679 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:02:32.896924 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-ee88b06e-7fbb-4e8a-a38c-5800854a4369 req-baf3a864-3293-4059-a303-ccf666732104 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.125746 Jul 16 22:02:32.898278 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3266/6526] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 22:02:32 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 128 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 16 22:02:33.906147 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-91a75a7a-a6d0-4753-922d-21bebc2c2f0b tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:02:33.906869 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-91a75a7a-a6d0-4753-922d-21bebc2c2f0b tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: ccef2dfa-df64-4f44-96db-85598e217ba8] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:02:33.914150 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-91a75a7a-a6d0-4753-922d-21bebc2c2f0b tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:02:33.919258 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-91a75a7a-a6d0-4753-922d-21bebc2c2f0b tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:02:33.919258 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-91a75a7a-a6d0-4753-922d-21bebc2c2f0b tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:02:33.919258 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-91a75a7a-a6d0-4753-922d-21bebc2c2f0b tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:02:34.051412 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-91a75a7a-a6d0-4753-922d-21bebc2c2f0b tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '718f6de3b1a1453d9579d421a574691e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd21231d9af34d9aa4f9e03229b49a30', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:02:34.059220 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-91a75a7a-a6d0-4753-922d-21bebc2c2f0b tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/ccef2dfa-df64-4f44-96db-85598e217ba8" status: 200 len: 1679 microversion: 2.1 time: 0.157158 Jul 16 22:02:34.059810 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3261/6527] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:02:33 2022] GET /compute/v2.1/servers/ccef2dfa-df64-4f44-96db-85598e217ba8 => generated 1679 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:02:34.928565 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-a462c44a-885f-4bd4-8632-5f694d1b4e09 req-b9f9e878-8375-45fb-96d6-e26258cbb230 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "ccef2dfa-df64-4f44-96db-85598e217ba8", "tag": "cf61d729-6d57-4176-8d56-32b10b837b8f"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:02:34.936801 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-a462c44a-885f-4bd4-8632-5f694d1b4e09 req-b9f9e878-8375-45fb-96d6-e26258cbb230 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:02:34.937255 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-a462c44a-885f-4bd4-8632-5f694d1b4e09 req-b9f9e878-8375-45fb-96d6-e26258cbb230 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:02:34.948739 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-a462c44a-885f-4bd4-8632-5f694d1b4e09 req-b9f9e878-8375-45fb-96d6-e26258cbb230 service nova] Creating event network-changed:cf61d729-6d57-4176-8d56-32b10b837b8f for instance ccef2dfa-df64-4f44-96db-85598e217ba8 on nch1-master-1 Jul 16 22:02:34.954067 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [req-a462c44a-885f-4bd4-8632-5f694d1b4e09 req-b9f9e878-8375-45fb-96d6-e26258cbb230 service nova] Instance ccef2dfa-df64-4f44-96db-85598e217ba8 is migrating within the same cell, copying events to all relevant hosts: {'nch2-master-1', 'nch1-master-1'} {{(pid=138681) _get_relevant_hosts /opt/stack/nova/nova/compute/api.py:5807}} Jul 16 22:02:34.965464 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-a462c44a-885f-4bd4-8632-5f694d1b4e09 req-b9f9e878-8375-45fb-96d6-e26258cbb230 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.042045 Jul 16 22:02:34.966042 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3267/6528] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 22:02:34 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 22:02:35.077067 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-6aa8511f-3a14-4cd2-b460-d007460b9cff tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:02:35.077756 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-6aa8511f-3a14-4cd2-b460-d007460b9cff tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: ccef2dfa-df64-4f44-96db-85598e217ba8] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:02:35.085639 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6aa8511f-3a14-4cd2-b460-d007460b9cff tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:02:35.086037 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6aa8511f-3a14-4cd2-b460-d007460b9cff tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:02:35.086842 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6aa8511f-3a14-4cd2-b460-d007460b9cff tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:02:35.088301 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6aa8511f-3a14-4cd2-b460-d007460b9cff tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:02:35.226341 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-6aa8511f-3a14-4cd2-b460-d007460b9cff tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '718f6de3b1a1453d9579d421a574691e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd21231d9af34d9aa4f9e03229b49a30', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:02:35.234035 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-6aa8511f-3a14-4cd2-b460-d007460b9cff tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/ccef2dfa-df64-4f44-96db-85598e217ba8" status: 200 len: 1677 microversion: 2.1 time: 0.160890 Jul 16 22:02:35.234623 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3262/6529] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:02:35 2022] GET /compute/v2.1/servers/ccef2dfa-df64-4f44-96db-85598e217ba8 => generated 1677 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:02:36.253162 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-453ad321-a31c-4185-9eb1-e167fc30d492 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:02:36.253880 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-453ad321-a31c-4185-9eb1-e167fc30d492 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: ccef2dfa-df64-4f44-96db-85598e217ba8] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:02:36.260368 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-453ad321-a31c-4185-9eb1-e167fc30d492 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:02:36.260836 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-453ad321-a31c-4185-9eb1-e167fc30d492 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:02:36.261615 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-453ad321-a31c-4185-9eb1-e167fc30d492 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:02:36.262071 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-453ad321-a31c-4185-9eb1-e167fc30d492 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:02:36.396812 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-453ad321-a31c-4185-9eb1-e167fc30d492 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '718f6de3b1a1453d9579d421a574691e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd21231d9af34d9aa4f9e03229b49a30', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:02:36.404404 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-453ad321-a31c-4185-9eb1-e167fc30d492 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/ccef2dfa-df64-4f44-96db-85598e217ba8" status: 200 len: 1674 microversion: 2.1 time: 0.156364 Jul 16 22:02:36.404986 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3268/6530] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:02:36 2022] GET /compute/v2.1/servers/ccef2dfa-df64-4f44-96db-85598e217ba8 => generated 1674 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:02:36.421995 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-1bb17802-1d7c-4d59-b4c4-e50d26ad32bf tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Action: 'action', calling method: >, body: {"confirmResize": {}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:02:36.422575 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-1bb17802-1d7c-4d59-b4c4-e50d26ad32bf tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: ccef2dfa-df64-4f44-96db-85598e217ba8] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:02:36.429784 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1bb17802-1d7c-4d59-b4c4-e50d26ad32bf tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:02:36.430249 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1bb17802-1d7c-4d59-b4c4-e50d26ad32bf tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:02:36.431050 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1bb17802-1d7c-4d59-b4c4-e50d26ad32bf tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:02:36.431507 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1bb17802-1d7c-4d59-b4c4-e50d26ad32bf tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:02:36.622567 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-1bb17802-1d7c-4d59-b4c4-e50d26ad32bf tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "POST /compute/v2.1/servers/ccef2dfa-df64-4f44-96db-85598e217ba8/action" status: 204 len: 0 microversion: 2.1 time: 0.204293 Jul 16 22:02:36.631086 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3263/6531] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 22:02:36 2022] POST /compute/v2.1/servers/ccef2dfa-df64-4f44-96db-85598e217ba8/action => generated 0 bytes in 206 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 16 22:02:36.632143 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-c5398ed5-4e63-470b-b538-5c693064338e tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:02:36.632772 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-c5398ed5-4e63-470b-b538-5c693064338e tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: ccef2dfa-df64-4f44-96db-85598e217ba8] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:02:36.638985 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c5398ed5-4e63-470b-b538-5c693064338e tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:02:36.639739 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c5398ed5-4e63-470b-b538-5c693064338e tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:02:36.640534 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c5398ed5-4e63-470b-b538-5c693064338e tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:02:36.641157 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c5398ed5-4e63-470b-b538-5c693064338e tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:02:36.782447 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-c5398ed5-4e63-470b-b538-5c693064338e tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '718f6de3b1a1453d9579d421a574691e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd21231d9af34d9aa4f9e03229b49a30', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:02:36.790539 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-c5398ed5-4e63-470b-b538-5c693064338e tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/ccef2dfa-df64-4f44-96db-85598e217ba8" status: 200 len: 1674 microversion: 2.1 time: 0.162328 Jul 16 22:02:36.791136 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3269/6532] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:02:36 2022] GET /compute/v2.1/servers/ccef2dfa-df64-4f44-96db-85598e217ba8 => generated 1674 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:02:37.809790 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-13c3cfde-04e5-4597-9b1b-b8844d8b612a tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:02:37.810593 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-13c3cfde-04e5-4597-9b1b-b8844d8b612a tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: ccef2dfa-df64-4f44-96db-85598e217ba8] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:02:37.817837 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-13c3cfde-04e5-4597-9b1b-b8844d8b612a tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:02:37.820007 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-13c3cfde-04e5-4597-9b1b-b8844d8b612a tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:02:37.820007 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-13c3cfde-04e5-4597-9b1b-b8844d8b612a tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:02:37.820007 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-13c3cfde-04e5-4597-9b1b-b8844d8b612a tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:02:37.835329 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:02:37.954808 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-13c3cfde-04e5-4597-9b1b-b8844d8b612a tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '718f6de3b1a1453d9579d421a574691e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd21231d9af34d9aa4f9e03229b49a30', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:02:37.963521 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-13c3cfde-04e5-4597-9b1b-b8844d8b612a tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/ccef2dfa-df64-4f44-96db-85598e217ba8" status: 200 len: 1674 microversion: 2.1 time: 0.158783 Jul 16 22:02:37.964109 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3264/6533] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:02:37 2022] GET /compute/v2.1/servers/ccef2dfa-df64-4f44-96db-85598e217ba8 => generated 1674 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:02:38.981008 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-43feb55c-8272-4533-9a60-b3744d2c7ba1 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:02:38.981639 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-43feb55c-8272-4533-9a60-b3744d2c7ba1 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: ccef2dfa-df64-4f44-96db-85598e217ba8] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:02:38.989299 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-43feb55c-8272-4533-9a60-b3744d2c7ba1 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:02:38.989764 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-43feb55c-8272-4533-9a60-b3744d2c7ba1 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:02:38.990545 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-43feb55c-8272-4533-9a60-b3744d2c7ba1 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:02:38.990998 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-43feb55c-8272-4533-9a60-b3744d2c7ba1 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:02:39.121648 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-43feb55c-8272-4533-9a60-b3744d2c7ba1 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '718f6de3b1a1453d9579d421a574691e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd21231d9af34d9aa4f9e03229b49a30', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:02:39.125233 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:02:39.130510 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-43feb55c-8272-4533-9a60-b3744d2c7ba1 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/ccef2dfa-df64-4f44-96db-85598e217ba8" status: 200 len: 1666 microversion: 2.1 time: 0.153449 Jul 16 22:02:39.131094 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3270/6534] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:02:38 2022] GET /compute/v2.1/servers/ccef2dfa-df64-4f44-96db-85598e217ba8 => generated 1666 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:02:39.147310 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-ab818957-0688-4ce1-94ad-25c495a29c2f tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:02:39.148799 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-ab818957-0688-4ce1-94ad-25c495a29c2f tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: ccef2dfa-df64-4f44-96db-85598e217ba8] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:02:39.155082 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ab818957-0688-4ce1-94ad-25c495a29c2f tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:02:39.155508 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ab818957-0688-4ce1-94ad-25c495a29c2f tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:02:39.156284 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ab818957-0688-4ce1-94ad-25c495a29c2f tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:02:39.156730 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ab818957-0688-4ce1-94ad-25c495a29c2f tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:02:39.292115 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-ab818957-0688-4ce1-94ad-25c495a29c2f tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '718f6de3b1a1453d9579d421a574691e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd21231d9af34d9aa4f9e03229b49a30', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:02:39.299770 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-ab818957-0688-4ce1-94ad-25c495a29c2f tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/ccef2dfa-df64-4f44-96db-85598e217ba8" status: 200 len: 1666 microversion: 2.1 time: 0.156361 Jul 16 22:02:39.300347 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3265/6535] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:02:39 2022] GET /compute/v2.1/servers/ccef2dfa-df64-4f44-96db-85598e217ba8 => generated 1666 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:02:39.316415 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-0ff60f2d-073a-48ef-80e8-a70eb1e74a2e tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:02:39.317305 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-0ff60f2d-073a-48ef-80e8-a70eb1e74a2e tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: ccef2dfa-df64-4f44-96db-85598e217ba8] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:02:39.323889 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0ff60f2d-073a-48ef-80e8-a70eb1e74a2e tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:02:39.324357 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0ff60f2d-073a-48ef-80e8-a70eb1e74a2e tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:02:39.325150 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0ff60f2d-073a-48ef-80e8-a70eb1e74a2e tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:02:39.325605 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0ff60f2d-073a-48ef-80e8-a70eb1e74a2e tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:02:39.365435 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-9cd90c14-40fe-4eee-a9a6-c9c53761bae7 req-eba74f7c-4529-4a29-9a8b-922fd2f459c7 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "ccef2dfa-df64-4f44-96db-85598e217ba8", "name": "network-vif-plugged", "status": "completed", "tag": "cf61d729-6d57-4176-8d56-32b10b837b8f"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:02:39.375839 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-9cd90c14-40fe-4eee-a9a6-c9c53761bae7 req-eba74f7c-4529-4a29-9a8b-922fd2f459c7 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:02:39.378093 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-9cd90c14-40fe-4eee-a9a6-c9c53761bae7 req-eba74f7c-4529-4a29-9a8b-922fd2f459c7 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:02:39.387921 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-9cd90c14-40fe-4eee-a9a6-c9c53761bae7 req-eba74f7c-4529-4a29-9a8b-922fd2f459c7 service nova] Creating event network-vif-plugged:cf61d729-6d57-4176-8d56-32b10b837b8f for instance ccef2dfa-df64-4f44-96db-85598e217ba8 on nch1-master-1 Jul 16 22:02:39.393740 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-9cd90c14-40fe-4eee-a9a6-c9c53761bae7 req-eba74f7c-4529-4a29-9a8b-922fd2f459c7 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.032470 Jul 16 22:02:39.394343 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3266/6536] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 22:02:39 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 16 22:02:39.460846 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-0ff60f2d-073a-48ef-80e8-a70eb1e74a2e tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '718f6de3b1a1453d9579d421a574691e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd21231d9af34d9aa4f9e03229b49a30', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:02:39.468455 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-0ff60f2d-073a-48ef-80e8-a70eb1e74a2e tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/ccef2dfa-df64-4f44-96db-85598e217ba8" status: 200 len: 1666 microversion: 2.1 time: 0.156559 Jul 16 22:02:39.469035 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3271/6537] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:02:39 2022] GET /compute/v2.1/servers/ccef2dfa-df64-4f44-96db-85598e217ba8 => generated 1666 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:02:39.486037 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-680dd0e3-39ac-4fa8-91c2-7e3da6e328f5 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:02:39.486655 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-680dd0e3-39ac-4fa8-91c2-7e3da6e328f5 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: ccef2dfa-df64-4f44-96db-85598e217ba8] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:02:39.493090 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-680dd0e3-39ac-4fa8-91c2-7e3da6e328f5 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:02:39.493501 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-680dd0e3-39ac-4fa8-91c2-7e3da6e328f5 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:02:39.494316 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-680dd0e3-39ac-4fa8-91c2-7e3da6e328f5 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:02:39.495030 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-680dd0e3-39ac-4fa8-91c2-7e3da6e328f5 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:02:39.524341 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-680dd0e3-39ac-4fa8-91c2-7e3da6e328f5 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: ccef2dfa-df64-4f44-96db-85598e217ba8] Going to try to terminate instance {{(pid=138680) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 16 22:02:39.803266 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-680dd0e3-39ac-4fa8-91c2-7e3da6e328f5 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "DELETE /compute/v2.1/servers/ccef2dfa-df64-4f44-96db-85598e217ba8" status: 204 len: 0 microversion: 2.1 time: 0.321317 Jul 16 22:02:39.803881 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3267/6538] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 22:02:39 2022] DELETE /compute/v2.1/servers/ccef2dfa-df64-4f44-96db-85598e217ba8 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 16 22:02:39.812645 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-2bad2822-297e-43cc-b4cc-a14b1b0868a2 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:02:39.813356 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-2bad2822-297e-43cc-b4cc-a14b1b0868a2 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: ccef2dfa-df64-4f44-96db-85598e217ba8] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:02:39.821143 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2bad2822-297e-43cc-b4cc-a14b1b0868a2 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:02:39.821609 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2bad2822-297e-43cc-b4cc-a14b1b0868a2 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:02:39.822397 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2bad2822-297e-43cc-b4cc-a14b1b0868a2 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:02:39.822857 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2bad2822-297e-43cc-b4cc-a14b1b0868a2 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:02:39.958251 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-2bad2822-297e-43cc-b4cc-a14b1b0868a2 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '718f6de3b1a1453d9579d421a574691e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd21231d9af34d9aa4f9e03229b49a30', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:02:39.966612 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-2bad2822-297e-43cc-b4cc-a14b1b0868a2 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/ccef2dfa-df64-4f44-96db-85598e217ba8" status: 200 len: 1672 microversion: 2.1 time: 0.158328 Jul 16 22:02:39.967409 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3272/6539] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:02:39 2022] GET /compute/v2.1/servers/ccef2dfa-df64-4f44-96db-85598e217ba8 => generated 1672 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:02:40.985387 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-bbac2e09-3b93-4c0d-b4b1-c8d1f8fa9733 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:02:40.986912 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-bbac2e09-3b93-4c0d-b4b1-c8d1f8fa9733 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: ccef2dfa-df64-4f44-96db-85598e217ba8] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:02:40.989606 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_api:SELECT=44,nova_api:UPDATE=1 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:02:40.994411 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-bbac2e09-3b93-4c0d-b4b1-c8d1f8fa9733 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:02:40.994867 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-bbac2e09-3b93-4c0d-b4b1-c8d1f8fa9733 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:02:40.995667 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-bbac2e09-3b93-4c0d-b4b1-c8d1f8fa9733 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:02:40.996111 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-bbac2e09-3b93-4c0d-b4b1-c8d1f8fa9733 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:02:41.131737 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-bbac2e09-3b93-4c0d-b4b1-c8d1f8fa9733 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '718f6de3b1a1453d9579d421a574691e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd21231d9af34d9aa4f9e03229b49a30', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:02:41.139580 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-bbac2e09-3b93-4c0d-b4b1-c8d1f8fa9733 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/ccef2dfa-df64-4f44-96db-85598e217ba8" status: 200 len: 1672 microversion: 2.1 time: 0.158395 Jul 16 22:02:41.140207 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3268/6540] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:02:40 2022] GET /compute/v2.1/servers/ccef2dfa-df64-4f44-96db-85598e217ba8 => generated 1672 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:02:42.157190 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-2a65076f-05b4-4283-b63e-cf4474e6e471 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:02:42.158505 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-2a65076f-05b4-4283-b63e-cf4474e6e471 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: ccef2dfa-df64-4f44-96db-85598e217ba8] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:02:42.161560 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_api:SELECT=41 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:02:42.166699 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2a65076f-05b4-4283-b63e-cf4474e6e471 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:02:42.167201 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2a65076f-05b4-4283-b63e-cf4474e6e471 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:02:42.167998 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2a65076f-05b4-4283-b63e-cf4474e6e471 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:02:42.168452 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2a65076f-05b4-4283-b63e-cf4474e6e471 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:02:42.235192 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-2a65076f-05b4-4283-b63e-cf4474e6e471 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '718f6de3b1a1453d9579d421a574691e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd21231d9af34d9aa4f9e03229b49a30', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:02:42.244009 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-2a65076f-05b4-4283-b63e-cf4474e6e471 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/ccef2dfa-df64-4f44-96db-85598e217ba8" status: 200 len: 1605 microversion: 2.1 time: 0.090705 Jul 16 22:02:42.244600 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3273/6541] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:02:42 2022] GET /compute/v2.1/servers/ccef2dfa-df64-4f44-96db-85598e217ba8 => generated 1605 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:02:42.537767 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-efc441e7-9d61-44ee-8426-814ebd4acf16 req-4c170dac-d0b5-4296-be73-caf969c5720d service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "ccef2dfa-df64-4f44-96db-85598e217ba8", "name": "network-vif-deleted", "tag": "cf61d729-6d57-4176-8d56-32b10b837b8f"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:02:42.546847 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-efc441e7-9d61-44ee-8426-814ebd4acf16 req-4c170dac-d0b5-4296-be73-caf969c5720d service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:02:42.547439 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-efc441e7-9d61-44ee-8426-814ebd4acf16 req-4c170dac-d0b5-4296-be73-caf969c5720d service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:02:42.560050 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-efc441e7-9d61-44ee-8426-814ebd4acf16 req-4c170dac-d0b5-4296-be73-caf969c5720d service nova] Creating event network-vif-deleted:cf61d729-6d57-4176-8d56-32b10b837b8f for instance ccef2dfa-df64-4f44-96db-85598e217ba8 on nch1-master-1 Jul 16 22:02:42.565747 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-efc441e7-9d61-44ee-8426-814ebd4acf16 req-4c170dac-d0b5-4296-be73-caf969c5720d service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.032487 Jul 16 22:02:42.566483 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3269/6542] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 22:02:42 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 22:02:43.262000 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-576a9ba2-9348-4b1b-8ab1-8e9b2ddd9b60 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:02:43.262669 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-576a9ba2-9348-4b1b-8ab1-8e9b2ddd9b60 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: ccef2dfa-df64-4f44-96db-85598e217ba8] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:02:43.272496 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-576a9ba2-9348-4b1b-8ab1-8e9b2ddd9b60 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:02:43.275917 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-576a9ba2-9348-4b1b-8ab1-8e9b2ddd9b60 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:02:43.275917 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-576a9ba2-9348-4b1b-8ab1-8e9b2ddd9b60 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:02:43.275917 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-576a9ba2-9348-4b1b-8ab1-8e9b2ddd9b60 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:02:43.309969 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-576a9ba2-9348-4b1b-8ab1-8e9b2ddd9b60 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:02:43.316723 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-576a9ba2-9348-4b1b-8ab1-8e9b2ddd9b60 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:02:43.316723 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.objects.instance [None req-576a9ba2-9348-4b1b-8ab1-8e9b2ddd9b60 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lazy-loading 'fault' on Instance uuid ccef2dfa-df64-4f44-96db-85598e217ba8 {{(pid=138681) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 22:02:43.694264 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-576a9ba2-9348-4b1b-8ab1-8e9b2ddd9b60 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '718f6de3b1a1453d9579d421a574691e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd21231d9af34d9aa4f9e03229b49a30', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:02:43.701493 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-576a9ba2-9348-4b1b-8ab1-8e9b2ddd9b60 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/ccef2dfa-df64-4f44-96db-85598e217ba8" status: 200 len: 1326 microversion: 2.1 time: 0.443759 Jul 16 22:02:43.702071 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3274/6543] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:02:43 2022] GET /compute/v2.1/servers/ccef2dfa-df64-4f44-96db-85598e217ba8 => generated 1326 bytes in 445 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:02:44.721166 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-2e4df95b-f1b0-4896-a52a-15878dfcc33e tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:02:44.721875 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-2e4df95b-f1b0-4896-a52a-15878dfcc33e tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: ccef2dfa-df64-4f44-96db-85598e217ba8] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:02:44.728400 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2e4df95b-f1b0-4896-a52a-15878dfcc33e tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:02:44.728928 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2e4df95b-f1b0-4896-a52a-15878dfcc33e tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:02:44.729695 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2e4df95b-f1b0-4896-a52a-15878dfcc33e tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:02:44.730138 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2e4df95b-f1b0-4896-a52a-15878dfcc33e tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:02:44.744165 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-2e4df95b-f1b0-4896-a52a-15878dfcc33e tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] HTTP exception thrown: Instance ccef2dfa-df64-4f44-96db-85598e217ba8 could not be found. Jul 16 22:02:44.745816 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-2e4df95b-f1b0-4896-a52a-15878dfcc33e tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Request method failure captured: Jul 16 22:02:44.745816 ncdv-master-1 devstack@n-api.service[138680]: request: GET /compute/v2.1/servers/ccef2dfa-df64-4f44-96db-85598e217ba8 HTTP/1.1 Jul 16 22:02:44.745816 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 22:02:44.745816 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 22:02:44.745816 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 22:02:44.745816 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 0 Jul 16 22:02:44.745816 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 22:02:44.745816 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 22:02:44.745816 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 22:02:44.745816 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 22:02:44.745816 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0zUsISRuFerqPWLnIU6pWtJZ1G33kS5UQBpOE-Qn9W1zRIsDO0obaxdGmXyhe5FNe11pjr8YSFwmEAtO4tE-QrUwHNB6R_8ZgLrv3bb_zQTEpcOValb2w1FWdJ1eIHV_96nHchz9PmXAJpOM5Cp_lLo9a71M8MQcZrQgoRzDb41oZDA Jul 16 22:02:44.745816 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 22:02:44.745816 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 22:02:44.745816 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 22:02:44.745816 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 22:02:44.745816 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 22:02:44.745816 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 22:02:44.745816 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: fd21231d9af34d9aa4f9e03229b49a30 Jul 16 22:02:44.745816 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-ServerActionsTestJSON-164809533 Jul 16 22:02:44.745816 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 22:02:44.745816 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 22:02:44.745816 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/fd21231d9af34d9aa4f9e03229b49a30"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/fd21231d9af34d9aa4f9e03229b49a30"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_fd21231d9af34d9aa4f9e03229b49a30"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/fd21231d9af34d9aa4f9e03229b49a30"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 22:02:44.745816 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-ServerActionsTestJSON-164809533 Jul 16 22:02:44.745816 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: fd21231d9af34d9aa4f9e03229b49a30 Jul 16 22:02:44.745816 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-ServerActionsTestJSON-164809533 Jul 16 22:02:44.745816 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-ServerActionsTestJSON-164809533-project Jul 16 22:02:44.745816 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 22:02:44.745816 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 22:02:44.745816 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: 718f6de3b1a1453d9579d421a574691e Jul 16 22:02:44.745816 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-ServerActionsTestJSON-164809533-project Jul 16 22:02:44.745816 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 22:02:44.745816 ncdv-master-1 devstack@n-api.service[138680]: exception: Instance ccef2dfa-df64-4f44-96db-85598e217ba8 could not be found. Jul 16 22:02:44.745816 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': 'ccef2dfa-df64-4f44-96db-85598e217ba8'} Jul 16 22:02:44.745816 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 22:02:44.745816 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 22:02:44.745816 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 22:02:44.745816 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 22:02:44.745816 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 22:02:44.745816 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 22:02:44.745816 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 22:02:44.745816 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 22:02:44.745816 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 22:02:44.745816 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 22:02:44.745816 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance ccef2dfa-df64-4f44-96db-85598e217ba8 could not be found. Jul 16 22:02:44.745816 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 22:02:44.745816 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 22:02:44.745816 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 22:02:44.745816 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 22:02:44.745816 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 22:02:44.745816 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 22:02:44.745816 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 22:02:44.745816 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 22:02:44.745816 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 22:02:44.745816 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 22:02:44.745816 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 22:02:44.745816 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 22:02:44.745816 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 22:02:44.745816 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 22:02:44.745816 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 22:02:44.745816 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 22:02:44.745816 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance ccef2dfa-df64-4f44-96db-85598e217ba8 could not be found. Jul 16 22:02:44.745816 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 22:02:44.745816 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 22:02:44.745816 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 22:02:44.750756 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 22:02:44.750756 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 22:02:44.750756 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 22:02:44.750756 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 22:02:44.750756 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 22:02:44.750756 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance ccef2dfa-df64-4f44-96db-85598e217ba8 could not be found. Jul 16 22:02:44.750756 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 22:02:44.750756 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-2e4df95b-f1b0-4896-a52a-15878dfcc33e tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Returning 404 to user: Instance ccef2dfa-df64-4f44-96db-85598e217ba8 could not be found. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 22:02:44.750756 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-2e4df95b-f1b0-4896-a52a-15878dfcc33e tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/ccef2dfa-df64-4f44-96db-85598e217ba8" status: 404 len: 111 microversion: 2.1 time: 0.032149 Jul 16 22:02:44.750756 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3270/6544] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:02:44 2022] GET /compute/v2.1/servers/ccef2dfa-df64-4f44-96db-85598e217ba8 => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 22:02:44.761678 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-e0421d1c-12cb-4e4e-a7e9-f2ebf0c8b066 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:02:44.762302 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-e0421d1c-12cb-4e4e-a7e9-f2ebf0c8b066 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: ccef2dfa-df64-4f44-96db-85598e217ba8] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:02:44.769473 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e0421d1c-12cb-4e4e-a7e9-f2ebf0c8b066 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:02:44.769936 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e0421d1c-12cb-4e4e-a7e9-f2ebf0c8b066 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:02:44.770697 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e0421d1c-12cb-4e4e-a7e9-f2ebf0c8b066 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:02:44.771174 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e0421d1c-12cb-4e4e-a7e9-f2ebf0c8b066 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:02:44.784726 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-e0421d1c-12cb-4e4e-a7e9-f2ebf0c8b066 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] HTTP exception thrown: Instance ccef2dfa-df64-4f44-96db-85598e217ba8 could not be found. Jul 16 22:02:44.786344 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-e0421d1c-12cb-4e4e-a7e9-f2ebf0c8b066 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Request method failure captured: Jul 16 22:02:44.786344 ncdv-master-1 devstack@n-api.service[138681]: request: GET /compute/v2.1/servers/ccef2dfa-df64-4f44-96db-85598e217ba8 HTTP/1.1 Jul 16 22:02:44.786344 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 22:02:44.786344 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 22:02:44.786344 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 22:02:44.786344 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 22:02:44.786344 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 22:02:44.786344 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 22:02:44.786344 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 22:02:44.786344 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 22:02:44.786344 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0zUsISRuFerqPWLnIU6pWtJZ1G33kS5UQBpOE-Qn9W1zRIsDO0obaxdGmXyhe5FNe11pjr8YSFwmEAtO4tE-QrUwHNB6R_8ZgLrv3bb_zQTEpcOValb2w1FWdJ1eIHV_96nHchz9PmXAJpOM5Cp_lLo9a71M8MQcZrQgoRzDb41oZDA Jul 16 22:02:44.786344 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 22:02:44.786344 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 22:02:44.786344 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 22:02:44.786344 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 22:02:44.786344 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 22:02:44.786344 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 22:02:44.786344 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: fd21231d9af34d9aa4f9e03229b49a30 Jul 16 22:02:44.786344 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-ServerActionsTestJSON-164809533 Jul 16 22:02:44.786344 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 22:02:44.786344 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 22:02:44.786344 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/fd21231d9af34d9aa4f9e03229b49a30"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/fd21231d9af34d9aa4f9e03229b49a30"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_fd21231d9af34d9aa4f9e03229b49a30"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/fd21231d9af34d9aa4f9e03229b49a30"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 22:02:44.786344 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-ServerActionsTestJSON-164809533 Jul 16 22:02:44.786344 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: fd21231d9af34d9aa4f9e03229b49a30 Jul 16 22:02:44.786344 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-ServerActionsTestJSON-164809533 Jul 16 22:02:44.786344 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-ServerActionsTestJSON-164809533-project Jul 16 22:02:44.786344 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 22:02:44.786344 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 22:02:44.786344 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: 718f6de3b1a1453d9579d421a574691e Jul 16 22:02:44.786344 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-ServerActionsTestJSON-164809533-project Jul 16 22:02:44.786344 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 22:02:44.786344 ncdv-master-1 devstack@n-api.service[138681]: exception: Instance ccef2dfa-df64-4f44-96db-85598e217ba8 could not be found. Jul 16 22:02:44.786344 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': 'ccef2dfa-df64-4f44-96db-85598e217ba8'} Jul 16 22:02:44.786344 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 22:02:44.786344 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 22:02:44.786344 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 22:02:44.786344 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 22:02:44.786344 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 22:02:44.786344 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 22:02:44.786344 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 22:02:44.786344 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 22:02:44.786344 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 22:02:44.786344 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 22:02:44.786344 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance ccef2dfa-df64-4f44-96db-85598e217ba8 could not be found. Jul 16 22:02:44.786344 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 22:02:44.786344 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 22:02:44.786344 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 22:02:44.786344 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 22:02:44.786344 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 22:02:44.786344 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 22:02:44.786344 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 22:02:44.786344 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 22:02:44.786344 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 22:02:44.786344 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 22:02:44.786344 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 22:02:44.786344 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 22:02:44.786344 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 22:02:44.786344 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 22:02:44.786344 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 22:02:44.786344 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 22:02:44.786344 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance ccef2dfa-df64-4f44-96db-85598e217ba8 could not be found. Jul 16 22:02:44.786344 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 22:02:44.786344 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 22:02:44.786344 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 22:02:44.790772 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 22:02:44.790772 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 22:02:44.790772 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 22:02:44.790772 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 22:02:44.790772 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 22:02:44.790772 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance ccef2dfa-df64-4f44-96db-85598e217ba8 could not be found. Jul 16 22:02:44.790772 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 22:02:44.790772 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-e0421d1c-12cb-4e4e-a7e9-f2ebf0c8b066 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Returning 404 to user: Instance ccef2dfa-df64-4f44-96db-85598e217ba8 could not be found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 22:02:44.790772 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-e0421d1c-12cb-4e4e-a7e9-f2ebf0c8b066 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/ccef2dfa-df64-4f44-96db-85598e217ba8" status: 404 len: 111 microversion: 2.1 time: 0.030015 Jul 16 22:02:44.790772 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3275/6545] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:02:44 2022] GET /compute/v2.1/servers/ccef2dfa-df64-4f44-96db-85598e217ba8 => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 22:02:44.836575 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-3ed37ea9-86f3-4f7d-aa0f-af6171e94a35 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerActionsTestJSON-server-152091973", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452", "networks": [{"uuid": "b3593d62-fee4-4445-83ed-ea9589fa3fcd"}], "security_groups": [{"name": "tempest-securitygroup--389060960"}], "key_name": "tempest-keypair-26228548", "user_data": "IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIHVidW50dSB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH51YnVudHUvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo="}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:02:44.940737 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.network.neutron [None req-3ed37ea9-86f3-4f7d-aa0f-af6171e94a35 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] validate_networks() for [('b3593d62-fee4-4445-83ed-ea9589fa3fcd', None, None, None, None, None)] {{(pid=138680) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 16 22:02:45.313531 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-3ed37ea9-86f3-4f7d-aa0f-af6171e94a35 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Flavor limits 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 16 22:02:45.313988 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-3ed37ea9-86f3-4f7d-aa0f-af6171e94a35 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Image limits 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 16 22:02:45.314465 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-3ed37ea9-86f3-4f7d-aa0f-af6171e94a35 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Flavor pref 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 16 22:02:45.314874 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-3ed37ea9-86f3-4f7d-aa0f-af6171e94a35 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Image pref 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 16 22:02:45.315324 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-3ed37ea9-86f3-4f7d-aa0f-af6171e94a35 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 16 22:02:45.324711 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3ed37ea9-86f3-4f7d-aa0f-af6171e94a35 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:02:45.325176 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3ed37ea9-86f3-4f7d-aa0f-af6171e94a35 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:02:45.325985 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3ed37ea9-86f3-4f7d-aa0f-af6171e94a35 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:02:45.326466 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3ed37ea9-86f3-4f7d-aa0f-af6171e94a35 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:02:45.344495 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-3ed37ea9-86f3-4f7d-aa0f-af6171e94a35 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Getting quotas for project fd21231d9af34d9aa4f9e03229b49a30. Resources: {'ram', 'instances', 'cores'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 22:02:45.350434 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-3ed37ea9-86f3-4f7d-aa0f-af6171e94a35 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Getting quotas for user 718f6de3b1a1453d9579d421a574691e and project fd21231d9af34d9aa4f9e03229b49a30. Resources: {'ram', 'instances', 'cores'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 22:02:45.363310 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-3ed37ea9-86f3-4f7d-aa0f-af6171e94a35 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Going to run 1 instances... {{(pid=138680) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 16 22:02:45.365601 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-3ed37ea9-86f3-4f7d-aa0f-af6171e94a35 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: 3447d99f-2076-4204-b4d2-55339d60ca3a] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138680) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 16 22:02:45.470344 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-3ed37ea9-86f3-4f7d-aa0f-af6171e94a35 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 405 microversion: 2.1 time: 0.637012 Jul 16 22:02:45.471466 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3271/6546] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 22:02:44 2022] POST /compute/v2.1/servers => generated 405 bytes in 639 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 16 22:02:45.481350 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-5cd6670a-f932-48a4-88b4-578d8dc175d3 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:02:45.482072 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-5cd6670a-f932-48a4-88b4-578d8dc175d3 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: 3447d99f-2076-4204-b4d2-55339d60ca3a] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:02:45.536379 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-5cd6670a-f932-48a4-88b4-578d8dc175d3 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '718f6de3b1a1453d9579d421a574691e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd21231d9af34d9aa4f9e03229b49a30', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:02:45.543243 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-5cd6670a-f932-48a4-88b4-578d8dc175d3 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/3447d99f-2076-4204-b4d2-55339d60ca3a" status: 200 len: 1236 microversion: 2.1 time: 0.065803 Jul 16 22:02:45.543811 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3276/6547] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:02:45 2022] GET /compute/v2.1/servers/3447d99f-2076-4204-b4d2-55339d60ca3a => generated 1236 bytes in 67 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:02:46.560488 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-724dcd33-8da7-49d1-994f-b7949b280b15 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:02:46.561118 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-724dcd33-8da7-49d1-994f-b7949b280b15 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: 3447d99f-2076-4204-b4d2-55339d60ca3a] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:02:46.569269 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-724dcd33-8da7-49d1-994f-b7949b280b15 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:02:46.569741 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-724dcd33-8da7-49d1-994f-b7949b280b15 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:02:46.570526 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-724dcd33-8da7-49d1-994f-b7949b280b15 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:02:46.570973 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-724dcd33-8da7-49d1-994f-b7949b280b15 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:02:46.633372 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-724dcd33-8da7-49d1-994f-b7949b280b15 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '718f6de3b1a1453d9579d421a574691e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd21231d9af34d9aa4f9e03229b49a30', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:02:46.641010 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-724dcd33-8da7-49d1-994f-b7949b280b15 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/3447d99f-2076-4204-b4d2-55339d60ca3a" status: 200 len: 1240 microversion: 2.1 time: 0.084460 Jul 16 22:02:46.642095 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3272/6548] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:02:46 2022] GET /compute/v2.1/servers/3447d99f-2076-4204-b4d2-55339d60ca3a => generated 1240 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:02:47.662852 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-a25e9e4f-52d5-445a-a658-05a62001e94b tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:02:47.663841 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-a25e9e4f-52d5-445a-a658-05a62001e94b tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: 3447d99f-2076-4204-b4d2-55339d60ca3a] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:02:47.673239 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a25e9e4f-52d5-445a-a658-05a62001e94b tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:02:47.673973 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a25e9e4f-52d5-445a-a658-05a62001e94b tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:02:47.674971 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a25e9e4f-52d5-445a-a658-05a62001e94b tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:02:47.675458 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a25e9e4f-52d5-445a-a658-05a62001e94b tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:02:47.751036 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-a25e9e4f-52d5-445a-a658-05a62001e94b tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '718f6de3b1a1453d9579d421a574691e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd21231d9af34d9aa4f9e03229b49a30', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:02:47.759802 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-a25e9e4f-52d5-445a-a658-05a62001e94b tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/3447d99f-2076-4204-b4d2-55339d60ca3a" status: 200 len: 1306 microversion: 2.1 time: 0.102151 Jul 16 22:02:47.760396 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3277/6549] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:02:47 2022] GET /compute/v2.1/servers/3447d99f-2076-4204-b4d2-55339d60ca3a => generated 1306 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:02:48.777790 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-22dd68fb-1538-4e81-aaa7-2e062b5de8c8 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:02:48.779063 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-22dd68fb-1538-4e81-aaa7-2e062b5de8c8 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: 3447d99f-2076-4204-b4d2-55339d60ca3a] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:02:48.787344 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-22dd68fb-1538-4e81-aaa7-2e062b5de8c8 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:02:48.788076 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-22dd68fb-1538-4e81-aaa7-2e062b5de8c8 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:02:48.788850 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-22dd68fb-1538-4e81-aaa7-2e062b5de8c8 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:02:48.789450 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-22dd68fb-1538-4e81-aaa7-2e062b5de8c8 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:02:48.931464 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-22dd68fb-1538-4e81-aaa7-2e062b5de8c8 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '718f6de3b1a1453d9579d421a574691e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd21231d9af34d9aa4f9e03229b49a30', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:02:48.939749 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-22dd68fb-1538-4e81-aaa7-2e062b5de8c8 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/3447d99f-2076-4204-b4d2-55339d60ca3a" status: 200 len: 1361 microversion: 2.1 time: 0.165849 Jul 16 22:02:48.947036 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3273/6550] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:02:48 2022] GET /compute/v2.1/servers/3447d99f-2076-4204-b4d2-55339d60ca3a => generated 1361 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:02:49.017685 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-8622ea5f-756d-4038-9979-e2aa2e2ab090 req-effa1a3a-6dd3-4c64-beba-a7dbc12802a4 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "3447d99f-2076-4204-b4d2-55339d60ca3a", "tag": "35e08dc0-521e-4023-9b5b-53b6391890cf"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:02:49.028827 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-8622ea5f-756d-4038-9979-e2aa2e2ab090 req-effa1a3a-6dd3-4c64-beba-a7dbc12802a4 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:02:49.029295 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-8622ea5f-756d-4038-9979-e2aa2e2ab090 req-effa1a3a-6dd3-4c64-beba-a7dbc12802a4 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:02:49.042638 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-8622ea5f-756d-4038-9979-e2aa2e2ab090 req-effa1a3a-6dd3-4c64-beba-a7dbc12802a4 service nova] Creating event network-changed:35e08dc0-521e-4023-9b5b-53b6391890cf for instance 3447d99f-2076-4204-b4d2-55339d60ca3a on nch1-master-1 Jul 16 22:02:49.048291 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-8622ea5f-756d-4038-9979-e2aa2e2ab090 req-effa1a3a-6dd3-4c64-beba-a7dbc12802a4 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.034254 Jul 16 22:02:49.049324 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3278/6551] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 22:02:49 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 22:02:49.957303 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-d2a19831-855d-47c0-b24a-566b900f3186 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:02:49.958228 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-d2a19831-855d-47c0-b24a-566b900f3186 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: 3447d99f-2076-4204-b4d2-55339d60ca3a] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:02:49.965339 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d2a19831-855d-47c0-b24a-566b900f3186 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:02:49.965797 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d2a19831-855d-47c0-b24a-566b900f3186 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:02:49.966567 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d2a19831-855d-47c0-b24a-566b900f3186 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:02:49.967097 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d2a19831-855d-47c0-b24a-566b900f3186 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:02:50.097401 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-d2a19831-855d-47c0-b24a-566b900f3186 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '718f6de3b1a1453d9579d421a574691e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd21231d9af34d9aa4f9e03229b49a30', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:02:50.105647 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-d2a19831-855d-47c0-b24a-566b900f3186 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/3447d99f-2076-4204-b4d2-55339d60ca3a" status: 200 len: 1361 microversion: 2.1 time: 0.152631 Jul 16 22:02:50.106235 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3274/6552] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:02:49 2022] GET /compute/v2.1/servers/3447d99f-2076-4204-b4d2-55339d60ca3a => generated 1361 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:02:51.124553 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-9c4bdd81-084f-4fa4-8efd-6329d6ba56c1 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:02:51.125180 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-9c4bdd81-084f-4fa4-8efd-6329d6ba56c1 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: 3447d99f-2076-4204-b4d2-55339d60ca3a] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:02:51.132146 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9c4bdd81-084f-4fa4-8efd-6329d6ba56c1 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:02:51.132616 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9c4bdd81-084f-4fa4-8efd-6329d6ba56c1 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:02:51.133385 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9c4bdd81-084f-4fa4-8efd-6329d6ba56c1 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:02:51.133833 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9c4bdd81-084f-4fa4-8efd-6329d6ba56c1 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:02:51.268016 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-9c4bdd81-084f-4fa4-8efd-6329d6ba56c1 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '718f6de3b1a1453d9579d421a574691e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd21231d9af34d9aa4f9e03229b49a30', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:02:51.275698 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-9c4bdd81-084f-4fa4-8efd-6329d6ba56c1 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/3447d99f-2076-4204-b4d2-55339d60ca3a" status: 200 len: 1361 microversion: 2.1 time: 0.156463 Jul 16 22:02:51.276258 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3279/6553] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:02:51 2022] GET /compute/v2.1/servers/3447d99f-2076-4204-b4d2-55339d60ca3a => generated 1361 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:02:52.293351 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-159fdfbc-5cf3-42e0-8b68-0842d8fe1da0 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:02:52.293872 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-159fdfbc-5cf3-42e0-8b68-0842d8fe1da0 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: 3447d99f-2076-4204-b4d2-55339d60ca3a] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:02:52.301543 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-159fdfbc-5cf3-42e0-8b68-0842d8fe1da0 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:02:52.301996 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-159fdfbc-5cf3-42e0-8b68-0842d8fe1da0 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:02:52.302757 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-159fdfbc-5cf3-42e0-8b68-0842d8fe1da0 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:02:52.303229 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-159fdfbc-5cf3-42e0-8b68-0842d8fe1da0 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:02:52.432980 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-159fdfbc-5cf3-42e0-8b68-0842d8fe1da0 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '718f6de3b1a1453d9579d421a574691e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd21231d9af34d9aa4f9e03229b49a30', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:02:52.440665 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-159fdfbc-5cf3-42e0-8b68-0842d8fe1da0 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/3447d99f-2076-4204-b4d2-55339d60ca3a" status: 200 len: 1361 microversion: 2.1 time: 0.151266 Jul 16 22:02:52.441231 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3275/6554] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:02:52 2022] GET /compute/v2.1/servers/3447d99f-2076-4204-b4d2-55339d60ca3a => generated 1361 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:02:53.458771 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-45a0a412-3871-49b2-ad4c-4dd7df80a66b tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:02:53.459456 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-45a0a412-3871-49b2-ad4c-4dd7df80a66b tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: 3447d99f-2076-4204-b4d2-55339d60ca3a] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:02:53.466996 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-45a0a412-3871-49b2-ad4c-4dd7df80a66b tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:02:53.467495 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-45a0a412-3871-49b2-ad4c-4dd7df80a66b tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:02:53.468271 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-45a0a412-3871-49b2-ad4c-4dd7df80a66b tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:02:53.468716 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-45a0a412-3871-49b2-ad4c-4dd7df80a66b tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:02:53.609174 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-45a0a412-3871-49b2-ad4c-4dd7df80a66b tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '718f6de3b1a1453d9579d421a574691e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd21231d9af34d9aa4f9e03229b49a30', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:02:53.617381 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-45a0a412-3871-49b2-ad4c-4dd7df80a66b tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/3447d99f-2076-4204-b4d2-55339d60ca3a" status: 200 len: 1361 microversion: 2.1 time: 0.163480 Jul 16 22:02:53.617958 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3280/6555] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:02:53 2022] GET /compute/v2.1/servers/3447d99f-2076-4204-b4d2-55339d60ca3a => generated 1361 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:02:54.635876 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-beb66418-0a60-4c7c-8f7b-607e500b16e9 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:02:54.636491 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-beb66418-0a60-4c7c-8f7b-607e500b16e9 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: 3447d99f-2076-4204-b4d2-55339d60ca3a] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:02:54.644326 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-beb66418-0a60-4c7c-8f7b-607e500b16e9 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:02:54.644856 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-beb66418-0a60-4c7c-8f7b-607e500b16e9 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:02:54.645616 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-beb66418-0a60-4c7c-8f7b-607e500b16e9 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:02:54.646053 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-beb66418-0a60-4c7c-8f7b-607e500b16e9 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:02:54.774595 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-beb66418-0a60-4c7c-8f7b-607e500b16e9 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '718f6de3b1a1453d9579d421a574691e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd21231d9af34d9aa4f9e03229b49a30', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:02:54.782099 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-beb66418-0a60-4c7c-8f7b-607e500b16e9 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/3447d99f-2076-4204-b4d2-55339d60ca3a" status: 200 len: 1361 microversion: 2.1 time: 0.150121 Jul 16 22:02:54.782680 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3276/6556] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:02:54 2022] GET /compute/v2.1/servers/3447d99f-2076-4204-b4d2-55339d60ca3a => generated 1361 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:02:55.442643 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-ea178f3a-4b4a-48ae-8c56-c4e247d71927 req-f806ae38-c29d-4580-abf3-f7267e762549 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "3447d99f-2076-4204-b4d2-55339d60ca3a", "name": "network-vif-plugged", "status": "completed", "tag": "35e08dc0-521e-4023-9b5b-53b6391890cf"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:02:55.455618 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-ea178f3a-4b4a-48ae-8c56-c4e247d71927 req-f806ae38-c29d-4580-abf3-f7267e762549 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:02:55.455618 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-ea178f3a-4b4a-48ae-8c56-c4e247d71927 req-f806ae38-c29d-4580-abf3-f7267e762549 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:02:55.463370 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-ea178f3a-4b4a-48ae-8c56-c4e247d71927 req-f806ae38-c29d-4580-abf3-f7267e762549 service nova] Creating event network-vif-plugged:35e08dc0-521e-4023-9b5b-53b6391890cf for instance 3447d99f-2076-4204-b4d2-55339d60ca3a on nch1-master-1 Jul 16 22:02:55.468554 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-ea178f3a-4b4a-48ae-8c56-c4e247d71927 req-f806ae38-c29d-4580-abf3-f7267e762549 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.029670 Jul 16 22:02:55.469144 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3281/6557] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 22:02:55 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 31 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 22:02:55.799444 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-b94f02f0-1811-4bc5-a3d5-e105aa620b0d tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:02:55.800190 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-b94f02f0-1811-4bc5-a3d5-e105aa620b0d tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: 3447d99f-2076-4204-b4d2-55339d60ca3a] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:02:55.802218 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell0:SELECT=2 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:02:55.802904 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell0:SELECT=6 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:02:55.808727 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b94f02f0-1811-4bc5-a3d5-e105aa620b0d tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:02:55.810960 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b94f02f0-1811-4bc5-a3d5-e105aa620b0d tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:02:55.810960 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b94f02f0-1811-4bc5-a3d5-e105aa620b0d tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:02:55.810960 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b94f02f0-1811-4bc5-a3d5-e105aa620b0d tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:02:55.946712 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-b94f02f0-1811-4bc5-a3d5-e105aa620b0d tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '718f6de3b1a1453d9579d421a574691e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd21231d9af34d9aa4f9e03229b49a30', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:02:55.954911 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-b94f02f0-1811-4bc5-a3d5-e105aa620b0d tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/3447d99f-2076-4204-b4d2-55339d60ca3a" status: 200 len: 1361 microversion: 2.1 time: 0.159392 Jul 16 22:02:55.955520 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3277/6558] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:02:55 2022] GET /compute/v2.1/servers/3447d99f-2076-4204-b4d2-55339d60ca3a => generated 1361 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:02:56.974049 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-82d7b07a-1d3a-4074-ada1-0db5b2b23b22 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:02:56.974679 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-82d7b07a-1d3a-4074-ada1-0db5b2b23b22 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: 3447d99f-2076-4204-b4d2-55339d60ca3a] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:02:56.976690 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell0:SELECT=2 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:02:56.982602 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-82d7b07a-1d3a-4074-ada1-0db5b2b23b22 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:02:56.983097 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-82d7b07a-1d3a-4074-ada1-0db5b2b23b22 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:02:56.983887 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-82d7b07a-1d3a-4074-ada1-0db5b2b23b22 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:02:56.984317 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-82d7b07a-1d3a-4074-ada1-0db5b2b23b22 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:02:57.119837 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-82d7b07a-1d3a-4074-ada1-0db5b2b23b22 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '718f6de3b1a1453d9579d421a574691e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd21231d9af34d9aa4f9e03229b49a30', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:02:57.127428 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-82d7b07a-1d3a-4074-ada1-0db5b2b23b22 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/3447d99f-2076-4204-b4d2-55339d60ca3a" status: 200 len: 1548 microversion: 2.1 time: 0.158461 Jul 16 22:02:57.128012 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3282/6559] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:02:56 2022] GET /compute/v2.1/servers/3447d99f-2076-4204-b4d2-55339d60ca3a => generated 1548 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:02:57.144594 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-057b3769-f2ef-4fc5-af1d-12360b5cc930 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:02:57.145122 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-057b3769-f2ef-4fc5-af1d-12360b5cc930 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: 3447d99f-2076-4204-b4d2-55339d60ca3a] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:02:57.152040 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-057b3769-f2ef-4fc5-af1d-12360b5cc930 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:02:57.152493 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-057b3769-f2ef-4fc5-af1d-12360b5cc930 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:02:57.153250 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-057b3769-f2ef-4fc5-af1d-12360b5cc930 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:02:57.153689 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-057b3769-f2ef-4fc5-af1d-12360b5cc930 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:02:57.231753 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-057b3769-f2ef-4fc5-af1d-12360b5cc930 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/3447d99f-2076-4204-b4d2-55339d60ca3a/os-interface" status: 200 len: 285 microversion: 2.1 time: 0.091671 Jul 16 22:02:57.232365 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3278/6560] 10.222.0.16 () {58 vars in 1274 bytes} [Sat Jul 16 22:02:57 2022] GET /compute/v2.1/servers/3447d99f-2076-4204-b4d2-55339d60ca3a/os-interface => generated 285 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 22:02:58.300854 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-3ff080c9-2b80-4031-940a-4c26837d64dc req-760abc27-9afb-44eb-8167-ef937a99aa5c service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "3447d99f-2076-4204-b4d2-55339d60ca3a", "tag": "35e08dc0-521e-4023-9b5b-53b6391890cf"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:02:58.309745 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-3ff080c9-2b80-4031-940a-4c26837d64dc req-760abc27-9afb-44eb-8167-ef937a99aa5c service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:02:58.310210 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-3ff080c9-2b80-4031-940a-4c26837d64dc req-760abc27-9afb-44eb-8167-ef937a99aa5c service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:02:58.321717 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-3ff080c9-2b80-4031-940a-4c26837d64dc req-760abc27-9afb-44eb-8167-ef937a99aa5c service nova] Creating event network-changed:35e08dc0-521e-4023-9b5b-53b6391890cf for instance 3447d99f-2076-4204-b4d2-55339d60ca3a on nch1-master-1 Jul 16 22:02:58.326944 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-3ff080c9-2b80-4031-940a-4c26837d64dc req-760abc27-9afb-44eb-8167-ef937a99aa5c service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.030420 Jul 16 22:02:58.327903 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3283/6561] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 22:02:58 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 22:03:31.255110 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_api:SELECT=34,nova_api:INSERT=3 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:03:31.255830 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=77,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:03:31.259835 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-6e4d5f43-3efa-472d-9ce9-0d1d70c6ad12 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Action: 'action', calling method: >, body: {"os-stop": {}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:03:31.265945 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6e4d5f43-3efa-472d-9ce9-0d1d70c6ad12 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:03:31.271601 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6e4d5f43-3efa-472d-9ce9-0d1d70c6ad12 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:03:31.289559 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-6e4d5f43-3efa-472d-9ce9-0d1d70c6ad12 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: 3447d99f-2076-4204-b4d2-55339d60ca3a] Going to try to stop instance {{(pid=138680) force_stop /opt/stack/nova/nova/compute/api.py:2707}} Jul 16 22:03:31.470579 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-6e4d5f43-3efa-472d-9ce9-0d1d70c6ad12 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "POST /compute/v2.1/servers/3447d99f-2076-4204-b4d2-55339d60ca3a/action" status: 202 len: 0 microversion: 2.1 time: 0.217265 Jul 16 22:03:31.471346 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3279/6562] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 22:03:31 2022] POST /compute/v2.1/servers/3447d99f-2076-4204-b4d2-55339d60ca3a/action => generated 0 bytes in 219 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 22:03:31.480319 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-2944d1bf-ce09-4ede-89e6-542c8cf0694b tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:03:31.481042 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-2944d1bf-ce09-4ede-89e6-542c8cf0694b tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: 3447d99f-2076-4204-b4d2-55339d60ca3a] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:03:31.483532 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_api:SELECT=25 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:03:31.484215 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=58 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:03:31.489513 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2944d1bf-ce09-4ede-89e6-542c8cf0694b tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:03:31.489974 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2944d1bf-ce09-4ede-89e6-542c8cf0694b tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:03:31.490750 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2944d1bf-ce09-4ede-89e6-542c8cf0694b tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:03:31.491245 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2944d1bf-ce09-4ede-89e6-542c8cf0694b tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:03:31.630436 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-2944d1bf-ce09-4ede-89e6-542c8cf0694b tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '718f6de3b1a1453d9579d421a574691e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd21231d9af34d9aa4f9e03229b49a30', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:03:31.638164 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-2944d1bf-ce09-4ede-89e6-542c8cf0694b tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/3447d99f-2076-4204-b4d2-55339d60ca3a" status: 200 len: 1677 microversion: 2.1 time: 0.162171 Jul 16 22:03:31.638889 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3284/6563] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:03:31 2022] GET /compute/v2.1/servers/3447d99f-2076-4204-b4d2-55339d60ca3a => generated 1677 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:03:32.657403 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-530066a0-3924-438e-ae9e-f55ba8ac3b7c tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:03:32.658140 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-530066a0-3924-438e-ae9e-f55ba8ac3b7c tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: 3447d99f-2076-4204-b4d2-55339d60ca3a] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:03:32.666247 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-530066a0-3924-438e-ae9e-f55ba8ac3b7c tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:03:32.666704 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-530066a0-3924-438e-ae9e-f55ba8ac3b7c tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:03:32.667512 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-530066a0-3924-438e-ae9e-f55ba8ac3b7c tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:03:32.667962 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-530066a0-3924-438e-ae9e-f55ba8ac3b7c tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:03:32.804258 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-530066a0-3924-438e-ae9e-f55ba8ac3b7c tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '718f6de3b1a1453d9579d421a574691e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd21231d9af34d9aa4f9e03229b49a30', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:03:32.811851 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-530066a0-3924-438e-ae9e-f55ba8ac3b7c tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/3447d99f-2076-4204-b4d2-55339d60ca3a" status: 200 len: 1677 microversion: 2.1 time: 0.159707 Jul 16 22:03:32.812424 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3280/6564] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:03:32 2022] GET /compute/v2.1/servers/3447d99f-2076-4204-b4d2-55339d60ca3a => generated 1677 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:03:33.829846 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-769c7e35-9756-443a-b798-1cd5d7e6108e tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:03:33.830898 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-769c7e35-9756-443a-b798-1cd5d7e6108e tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: 3447d99f-2076-4204-b4d2-55339d60ca3a] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:03:33.837027 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-769c7e35-9756-443a-b798-1cd5d7e6108e tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:03:33.837492 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-769c7e35-9756-443a-b798-1cd5d7e6108e tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:03:33.838268 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-769c7e35-9756-443a-b798-1cd5d7e6108e tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:03:33.838719 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-769c7e35-9756-443a-b798-1cd5d7e6108e tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:03:33.968503 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-769c7e35-9756-443a-b798-1cd5d7e6108e tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '718f6de3b1a1453d9579d421a574691e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd21231d9af34d9aa4f9e03229b49a30', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:03:33.976929 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-769c7e35-9756-443a-b798-1cd5d7e6108e tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/3447d99f-2076-4204-b4d2-55339d60ca3a" status: 200 len: 1677 microversion: 2.1 time: 0.151109 Jul 16 22:03:33.977504 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3285/6565] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:03:33 2022] GET /compute/v2.1/servers/3447d99f-2076-4204-b4d2-55339d60ca3a => generated 1677 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:03:34.993834 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-f0246d18-71c3-4392-908a-5152fc0b1549 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:03:34.994479 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-f0246d18-71c3-4392-908a-5152fc0b1549 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: 3447d99f-2076-4204-b4d2-55339d60ca3a] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:03:35.000754 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f0246d18-71c3-4392-908a-5152fc0b1549 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:03:35.001209 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f0246d18-71c3-4392-908a-5152fc0b1549 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:03:35.002919 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f0246d18-71c3-4392-908a-5152fc0b1549 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:03:35.003390 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f0246d18-71c3-4392-908a-5152fc0b1549 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:03:35.144732 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-f0246d18-71c3-4392-908a-5152fc0b1549 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '718f6de3b1a1453d9579d421a574691e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd21231d9af34d9aa4f9e03229b49a30', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:03:35.152706 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-f0246d18-71c3-4392-908a-5152fc0b1549 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/3447d99f-2076-4204-b4d2-55339d60ca3a" status: 200 len: 1677 microversion: 2.1 time: 0.162939 Jul 16 22:03:35.153294 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3281/6566] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:03:34 2022] GET /compute/v2.1/servers/3447d99f-2076-4204-b4d2-55339d60ca3a => generated 1677 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:03:36.170142 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-b64c4c50-a762-4220-8958-6fb40b966658 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:03:36.170694 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-b64c4c50-a762-4220-8958-6fb40b966658 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: 3447d99f-2076-4204-b4d2-55339d60ca3a] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:03:36.179065 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b64c4c50-a762-4220-8958-6fb40b966658 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:03:36.179567 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b64c4c50-a762-4220-8958-6fb40b966658 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:03:36.180351 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b64c4c50-a762-4220-8958-6fb40b966658 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:03:36.180801 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b64c4c50-a762-4220-8958-6fb40b966658 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:03:36.318208 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-b64c4c50-a762-4220-8958-6fb40b966658 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '718f6de3b1a1453d9579d421a574691e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd21231d9af34d9aa4f9e03229b49a30', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:03:36.329563 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-b64c4c50-a762-4220-8958-6fb40b966658 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/3447d99f-2076-4204-b4d2-55339d60ca3a" status: 200 len: 1677 microversion: 2.1 time: 0.163190 Jul 16 22:03:36.330530 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3286/6567] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:03:36 2022] GET /compute/v2.1/servers/3447d99f-2076-4204-b4d2-55339d60ca3a => generated 1677 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:03:37.349294 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-96805051-fed5-4b38-a73f-622536563dde tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:03:37.349902 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-96805051-fed5-4b38-a73f-622536563dde tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: 3447d99f-2076-4204-b4d2-55339d60ca3a] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:03:37.356928 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-96805051-fed5-4b38-a73f-622536563dde tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:03:37.357388 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-96805051-fed5-4b38-a73f-622536563dde tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:03:37.358153 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-96805051-fed5-4b38-a73f-622536563dde tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:03:37.358597 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-96805051-fed5-4b38-a73f-622536563dde tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:03:37.490688 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-96805051-fed5-4b38-a73f-622536563dde tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '718f6de3b1a1453d9579d421a574691e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd21231d9af34d9aa4f9e03229b49a30', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:03:37.498486 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-96805051-fed5-4b38-a73f-622536563dde tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/3447d99f-2076-4204-b4d2-55339d60ca3a" status: 200 len: 1677 microversion: 2.1 time: 0.153705 Jul 16 22:03:37.499115 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3282/6568] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:03:37 2022] GET /compute/v2.1/servers/3447d99f-2076-4204-b4d2-55339d60ca3a => generated 1677 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 22:03:38.515976 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-b7de78a0-7b6d-4e1d-8472-d003e5a2b14b tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:03:38.516830 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-b7de78a0-7b6d-4e1d-8472-d003e5a2b14b tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: 3447d99f-2076-4204-b4d2-55339d60ca3a] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:03:38.524025 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b7de78a0-7b6d-4e1d-8472-d003e5a2b14b tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:03:38.524492 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b7de78a0-7b6d-4e1d-8472-d003e5a2b14b tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:03:38.525279 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b7de78a0-7b6d-4e1d-8472-d003e5a2b14b tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:03:38.525728 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b7de78a0-7b6d-4e1d-8472-d003e5a2b14b tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:03:38.672772 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-b7de78a0-7b6d-4e1d-8472-d003e5a2b14b tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '718f6de3b1a1453d9579d421a574691e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd21231d9af34d9aa4f9e03229b49a30', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:03:38.681131 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-b7de78a0-7b6d-4e1d-8472-d003e5a2b14b tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/3447d99f-2076-4204-b4d2-55339d60ca3a" status: 200 len: 1677 microversion: 2.1 time: 0.169055 Jul 16 22:03:38.681707 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3287/6569] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:03:38 2022] GET /compute/v2.1/servers/3447d99f-2076-4204-b4d2-55339d60ca3a => generated 1677 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:03:39.698693 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-7435bda8-109d-4b50-a351-996f810556a7 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:03:39.699302 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-7435bda8-109d-4b50-a351-996f810556a7 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: 3447d99f-2076-4204-b4d2-55339d60ca3a] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:03:39.707191 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7435bda8-109d-4b50-a351-996f810556a7 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:03:39.707643 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7435bda8-109d-4b50-a351-996f810556a7 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:03:39.708402 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7435bda8-109d-4b50-a351-996f810556a7 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:03:39.708837 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7435bda8-109d-4b50-a351-996f810556a7 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:03:39.839297 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-7435bda8-109d-4b50-a351-996f810556a7 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '718f6de3b1a1453d9579d421a574691e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd21231d9af34d9aa4f9e03229b49a30', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:03:39.847323 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-7435bda8-109d-4b50-a351-996f810556a7 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/3447d99f-2076-4204-b4d2-55339d60ca3a" status: 200 len: 1677 microversion: 2.1 time: 0.152664 Jul 16 22:03:39.848142 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3283/6570] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:03:39 2022] GET /compute/v2.1/servers/3447d99f-2076-4204-b4d2-55339d60ca3a => generated 1677 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:03:40.865485 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-3dc0acb2-eea8-4d64-a259-529670fff3ab tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:03:40.866040 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-3dc0acb2-eea8-4d64-a259-529670fff3ab tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: 3447d99f-2076-4204-b4d2-55339d60ca3a] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:03:40.873091 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3dc0acb2-eea8-4d64-a259-529670fff3ab tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:03:40.873550 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3dc0acb2-eea8-4d64-a259-529670fff3ab tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:03:40.874321 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3dc0acb2-eea8-4d64-a259-529670fff3ab tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:03:40.874767 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3dc0acb2-eea8-4d64-a259-529670fff3ab tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:03:41.009465 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-3dc0acb2-eea8-4d64-a259-529670fff3ab tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '718f6de3b1a1453d9579d421a574691e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd21231d9af34d9aa4f9e03229b49a30', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:03:41.016931 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-3dc0acb2-eea8-4d64-a259-529670fff3ab tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/3447d99f-2076-4204-b4d2-55339d60ca3a" status: 200 len: 1677 microversion: 2.1 time: 0.155983 Jul 16 22:03:41.018195 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3288/6571] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:03:40 2022] GET /compute/v2.1/servers/3447d99f-2076-4204-b4d2-55339d60ca3a => generated 1677 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:03:42.035101 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-720bb7c5-a7ac-4c41-bebf-a33afcb368a6 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:03:42.035760 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-720bb7c5-a7ac-4c41-bebf-a33afcb368a6 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: 3447d99f-2076-4204-b4d2-55339d60ca3a] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:03:42.042989 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-720bb7c5-a7ac-4c41-bebf-a33afcb368a6 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:03:42.044001 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-720bb7c5-a7ac-4c41-bebf-a33afcb368a6 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:03:42.044322 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-720bb7c5-a7ac-4c41-bebf-a33afcb368a6 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:03:42.044797 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-720bb7c5-a7ac-4c41-bebf-a33afcb368a6 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:03:42.177659 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-720bb7c5-a7ac-4c41-bebf-a33afcb368a6 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '718f6de3b1a1453d9579d421a574691e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd21231d9af34d9aa4f9e03229b49a30', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:03:42.185470 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-720bb7c5-a7ac-4c41-bebf-a33afcb368a6 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/3447d99f-2076-4204-b4d2-55339d60ca3a" status: 200 len: 1677 microversion: 2.1 time: 0.154515 Jul 16 22:03:42.186066 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3284/6572] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:03:42 2022] GET /compute/v2.1/servers/3447d99f-2076-4204-b4d2-55339d60ca3a => generated 1677 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:03:43.202633 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-fb7af6e0-56b3-46ee-aef6-48c61506ce25 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:03:43.203323 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-fb7af6e0-56b3-46ee-aef6-48c61506ce25 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: 3447d99f-2076-4204-b4d2-55339d60ca3a] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:03:43.211553 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fb7af6e0-56b3-46ee-aef6-48c61506ce25 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:03:43.212018 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fb7af6e0-56b3-46ee-aef6-48c61506ce25 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:03:43.212796 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fb7af6e0-56b3-46ee-aef6-48c61506ce25 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:03:43.213248 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fb7af6e0-56b3-46ee-aef6-48c61506ce25 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:03:43.350536 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-fb7af6e0-56b3-46ee-aef6-48c61506ce25 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '718f6de3b1a1453d9579d421a574691e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd21231d9af34d9aa4f9e03229b49a30', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:03:43.358075 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-fb7af6e0-56b3-46ee-aef6-48c61506ce25 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/3447d99f-2076-4204-b4d2-55339d60ca3a" status: 200 len: 1654 microversion: 2.1 time: 0.159383 Jul 16 22:03:43.358638 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3289/6573] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:03:43 2022] GET /compute/v2.1/servers/3447d99f-2076-4204-b4d2-55339d60ca3a => generated 1654 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:03:43.374490 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-f3bb024c-542a-428b-9097-276b7dcf6a0b tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Action: 'action', calling method: >, body: {"resize": {"flavorRef": "451"}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:03:43.375584 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-f3bb024c-542a-428b-9097-276b7dcf6a0b tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: 3447d99f-2076-4204-b4d2-55339d60ca3a] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:03:43.383036 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f3bb024c-542a-428b-9097-276b7dcf6a0b tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:03:43.383497 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f3bb024c-542a-428b-9097-276b7dcf6a0b tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:03:43.384263 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f3bb024c-542a-428b-9097-276b7dcf6a0b tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:03:43.384703 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f3bb024c-542a-428b-9097-276b7dcf6a0b tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:03:43.413313 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-f3bb024c-542a-428b-9097-276b7dcf6a0b tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Policy check for compute:servers:resize:cross_cell failed with credentials {'is_admin': False, 'user_id': '718f6de3b1a1453d9579d421a574691e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd21231d9af34d9aa4f9e03229b49a30', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:03:43.419136 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-f3bb024c-542a-428b-9097-276b7dcf6a0b tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: 3447d99f-2076-4204-b4d2-55339d60ca3a] Old instance type m1.heat2, new instance type m1.heat {{(pid=138680) resize /opt/stack/nova/nova/compute/api.py:4166}} Jul 16 22:03:43.425411 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-f3bb024c-542a-428b-9097-276b7dcf6a0b tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Flavor limits 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 16 22:03:43.428242 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-f3bb024c-542a-428b-9097-276b7dcf6a0b tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Image limits 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 16 22:03:43.428242 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-f3bb024c-542a-428b-9097-276b7dcf6a0b tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Flavor pref 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 16 22:03:43.428242 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-f3bb024c-542a-428b-9097-276b7dcf6a0b tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Image pref 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 16 22:03:43.428242 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-f3bb024c-542a-428b-9097-276b7dcf6a0b tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 16 22:03:43.648909 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-f3bb024c-542a-428b-9097-276b7dcf6a0b tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "POST /compute/v2.1/servers/3447d99f-2076-4204-b4d2-55339d60ca3a/action" status: 202 len: 0 microversion: 2.1 time: 0.278202 Jul 16 22:03:43.649534 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3285/6574] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 22:03:43 2022] POST /compute/v2.1/servers/3447d99f-2076-4204-b4d2-55339d60ca3a/action => generated 0 bytes in 280 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 22:03:43.658148 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-1a5a5529-04bb-4042-923d-c0b1339413a7 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:03:43.658799 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-1a5a5529-04bb-4042-923d-c0b1339413a7 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: 3447d99f-2076-4204-b4d2-55339d60ca3a] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:03:43.665050 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1a5a5529-04bb-4042-923d-c0b1339413a7 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:03:43.665516 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1a5a5529-04bb-4042-923d-c0b1339413a7 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:03:43.666303 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1a5a5529-04bb-4042-923d-c0b1339413a7 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:03:43.666753 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1a5a5529-04bb-4042-923d-c0b1339413a7 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:03:43.798053 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-1a5a5529-04bb-4042-923d-c0b1339413a7 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '718f6de3b1a1453d9579d421a574691e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd21231d9af34d9aa4f9e03229b49a30', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:03:43.806871 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-1a5a5529-04bb-4042-923d-c0b1339413a7 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/3447d99f-2076-4204-b4d2-55339d60ca3a" status: 200 len: 1677 microversion: 2.1 time: 0.152736 Jul 16 22:03:43.807472 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3290/6575] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:03:43 2022] GET /compute/v2.1/servers/3447d99f-2076-4204-b4d2-55339d60ca3a => generated 1677 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:03:44.826748 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-67055a49-6a75-4316-9925-74ccc9c74fee tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:03:44.827402 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-67055a49-6a75-4316-9925-74ccc9c74fee tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: 3447d99f-2076-4204-b4d2-55339d60ca3a] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:03:44.835176 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-67055a49-6a75-4316-9925-74ccc9c74fee tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:03:44.835642 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-67055a49-6a75-4316-9925-74ccc9c74fee tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:03:44.836442 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-67055a49-6a75-4316-9925-74ccc9c74fee tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:03:44.836886 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-67055a49-6a75-4316-9925-74ccc9c74fee tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:03:44.970368 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-67055a49-6a75-4316-9925-74ccc9c74fee tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '718f6de3b1a1453d9579d421a574691e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd21231d9af34d9aa4f9e03229b49a30', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:03:44.978772 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-67055a49-6a75-4316-9925-74ccc9c74fee tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/3447d99f-2076-4204-b4d2-55339d60ca3a" status: 200 len: 1677 microversion: 2.1 time: 0.158075 Jul 16 22:03:44.979375 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3286/6576] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:03:44 2022] GET /compute/v2.1/servers/3447d99f-2076-4204-b4d2-55339d60ca3a => generated 1677 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:03:45.996635 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-81572b90-77b3-4dc5-a4c8-53d1f1945052 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:03:45.997262 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-81572b90-77b3-4dc5-a4c8-53d1f1945052 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: 3447d99f-2076-4204-b4d2-55339d60ca3a] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:03:46.005017 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-81572b90-77b3-4dc5-a4c8-53d1f1945052 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:03:46.005484 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-81572b90-77b3-4dc5-a4c8-53d1f1945052 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:03:46.006261 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-81572b90-77b3-4dc5-a4c8-53d1f1945052 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:03:46.006713 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-81572b90-77b3-4dc5-a4c8-53d1f1945052 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:03:46.140794 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-81572b90-77b3-4dc5-a4c8-53d1f1945052 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '718f6de3b1a1453d9579d421a574691e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd21231d9af34d9aa4f9e03229b49a30', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:03:46.148758 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-81572b90-77b3-4dc5-a4c8-53d1f1945052 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/3447d99f-2076-4204-b4d2-55339d60ca3a" status: 200 len: 1677 microversion: 2.1 time: 0.156078 Jul 16 22:03:46.149335 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3291/6577] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:03:45 2022] GET /compute/v2.1/servers/3447d99f-2076-4204-b4d2-55339d60ca3a => generated 1677 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:03:47.167239 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-b2982fb1-d85a-4998-b962-b318ce0ad62f tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:03:47.167852 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-b2982fb1-d85a-4998-b962-b318ce0ad62f tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: 3447d99f-2076-4204-b4d2-55339d60ca3a] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:03:47.180035 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b2982fb1-d85a-4998-b962-b318ce0ad62f tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:03:47.180496 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b2982fb1-d85a-4998-b962-b318ce0ad62f tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:03:47.181264 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b2982fb1-d85a-4998-b962-b318ce0ad62f tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:03:47.181705 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b2982fb1-d85a-4998-b962-b318ce0ad62f tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:03:47.318053 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-b2982fb1-d85a-4998-b962-b318ce0ad62f tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '718f6de3b1a1453d9579d421a574691e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd21231d9af34d9aa4f9e03229b49a30', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:03:47.325579 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-b2982fb1-d85a-4998-b962-b318ce0ad62f tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/3447d99f-2076-4204-b4d2-55339d60ca3a" status: 200 len: 1682 microversion: 2.1 time: 0.162308 Jul 16 22:03:47.326153 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3287/6578] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:03:47 2022] GET /compute/v2.1/servers/3447d99f-2076-4204-b4d2-55339d60ca3a => generated 1682 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:03:48.343246 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-f9f7b99d-8b2a-4b06-b0d8-25023350aba9 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:03:48.343860 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-f9f7b99d-8b2a-4b06-b0d8-25023350aba9 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: 3447d99f-2076-4204-b4d2-55339d60ca3a] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:03:48.351811 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f9f7b99d-8b2a-4b06-b0d8-25023350aba9 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:03:48.352277 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f9f7b99d-8b2a-4b06-b0d8-25023350aba9 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:03:48.353058 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f9f7b99d-8b2a-4b06-b0d8-25023350aba9 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:03:48.353510 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f9f7b99d-8b2a-4b06-b0d8-25023350aba9 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:03:48.489727 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-f9f7b99d-8b2a-4b06-b0d8-25023350aba9 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '718f6de3b1a1453d9579d421a574691e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd21231d9af34d9aa4f9e03229b49a30', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:03:48.497385 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-f9f7b99d-8b2a-4b06-b0d8-25023350aba9 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/3447d99f-2076-4204-b4d2-55339d60ca3a" status: 200 len: 1682 microversion: 2.1 time: 0.158058 Jul 16 22:03:48.497955 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3292/6579] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:03:48 2022] GET /compute/v2.1/servers/3447d99f-2076-4204-b4d2-55339d60ca3a => generated 1682 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:03:49.515311 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-77d1f9f0-cd46-46f3-ac79-0e53a31d0ad3 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:03:49.515871 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-77d1f9f0-cd46-46f3-ac79-0e53a31d0ad3 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: 3447d99f-2076-4204-b4d2-55339d60ca3a] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:03:49.525398 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-77d1f9f0-cd46-46f3-ac79-0e53a31d0ad3 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:03:49.525864 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-77d1f9f0-cd46-46f3-ac79-0e53a31d0ad3 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:03:49.526659 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-77d1f9f0-cd46-46f3-ac79-0e53a31d0ad3 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:03:49.527141 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-77d1f9f0-cd46-46f3-ac79-0e53a31d0ad3 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:03:49.626434 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-e56b5bc6-2991-4fa1-b721-be350f59e18d req-6d62a4d3-7ba3-4bab-9b31-108cb0484fd2 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "3447d99f-2076-4204-b4d2-55339d60ca3a", "name": "network-vif-unplugged", "status": "completed", "tag": "35e08dc0-521e-4023-9b5b-53b6391890cf"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:03:49.636065 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-e56b5bc6-2991-4fa1-b721-be350f59e18d req-6d62a4d3-7ba3-4bab-9b31-108cb0484fd2 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:03:49.636810 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-e56b5bc6-2991-4fa1-b721-be350f59e18d req-6d62a4d3-7ba3-4bab-9b31-108cb0484fd2 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:03:49.650244 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-e56b5bc6-2991-4fa1-b721-be350f59e18d req-6d62a4d3-7ba3-4bab-9b31-108cb0484fd2 service nova] Creating event network-vif-unplugged:35e08dc0-521e-4023-9b5b-53b6391890cf for instance 3447d99f-2076-4204-b4d2-55339d60ca3a on nch2-master-1 Jul 16 22:03:49.656850 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [req-e56b5bc6-2991-4fa1-b721-be350f59e18d req-6d62a4d3-7ba3-4bab-9b31-108cb0484fd2 service nova] Instance 3447d99f-2076-4204-b4d2-55339d60ca3a is migrating within the same cell, copying events to all relevant hosts: {'nch2-master-1', 'nch1-master-1'} {{(pid=138681) _get_relevant_hosts /opt/stack/nova/nova/compute/api.py:5807}} Jul 16 22:03:49.672046 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-e56b5bc6-2991-4fa1-b721-be350f59e18d req-6d62a4d3-7ba3-4bab-9b31-108cb0484fd2 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.049166 Jul 16 22:03:49.677151 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3293/6580] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 22:03:49 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 16 22:03:49.682765 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-77d1f9f0-cd46-46f3-ac79-0e53a31d0ad3 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '718f6de3b1a1453d9579d421a574691e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd21231d9af34d9aa4f9e03229b49a30', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:03:49.691077 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-77d1f9f0-cd46-46f3-ac79-0e53a31d0ad3 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/3447d99f-2076-4204-b4d2-55339d60ca3a" status: 200 len: 1681 microversion: 2.1 time: 0.179795 Jul 16 22:03:49.691989 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3288/6581] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:03:49 2022] GET /compute/v2.1/servers/3447d99f-2076-4204-b4d2-55339d60ca3a => generated 1681 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:03:50.709782 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-de5a4b67-3284-4cd0-a127-becaf62331e7 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:03:50.710505 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-de5a4b67-3284-4cd0-a127-becaf62331e7 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: 3447d99f-2076-4204-b4d2-55339d60ca3a] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:03:50.716836 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-de5a4b67-3284-4cd0-a127-becaf62331e7 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:03:50.721028 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-de5a4b67-3284-4cd0-a127-becaf62331e7 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:03:50.721028 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-de5a4b67-3284-4cd0-a127-becaf62331e7 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:03:50.721028 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-de5a4b67-3284-4cd0-a127-becaf62331e7 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:03:50.856238 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-de5a4b67-3284-4cd0-a127-becaf62331e7 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '718f6de3b1a1453d9579d421a574691e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd21231d9af34d9aa4f9e03229b49a30', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:03:50.864501 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-de5a4b67-3284-4cd0-a127-becaf62331e7 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/3447d99f-2076-4204-b4d2-55339d60ca3a" status: 200 len: 1681 microversion: 2.1 time: 0.160200 Jul 16 22:03:50.865500 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3294/6582] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:03:50 2022] GET /compute/v2.1/servers/3447d99f-2076-4204-b4d2-55339d60ca3a => generated 1681 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:03:51.686861 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-96f83187-1703-4cf4-8c88-803954b3f153 req-7e8a0644-2ef4-4fa5-b64a-a99093c6ef69 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "3447d99f-2076-4204-b4d2-55339d60ca3a", "tag": "35e08dc0-521e-4023-9b5b-53b6391890cf"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:03:51.696389 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-96f83187-1703-4cf4-8c88-803954b3f153 req-7e8a0644-2ef4-4fa5-b64a-a99093c6ef69 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:03:51.696843 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-96f83187-1703-4cf4-8c88-803954b3f153 req-7e8a0644-2ef4-4fa5-b64a-a99093c6ef69 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:03:51.710475 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-96f83187-1703-4cf4-8c88-803954b3f153 req-7e8a0644-2ef4-4fa5-b64a-a99093c6ef69 service nova] Creating event network-changed:35e08dc0-521e-4023-9b5b-53b6391890cf for instance 3447d99f-2076-4204-b4d2-55339d60ca3a on nch2-master-1 Jul 16 22:03:51.715911 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [req-96f83187-1703-4cf4-8c88-803954b3f153 req-7e8a0644-2ef4-4fa5-b64a-a99093c6ef69 service nova] Instance 3447d99f-2076-4204-b4d2-55339d60ca3a is migrating within the same cell, copying events to all relevant hosts: {'nch2-master-1', 'nch1-master-1'} {{(pid=138680) _get_relevant_hosts /opt/stack/nova/nova/compute/api.py:5807}} Jul 16 22:03:51.727360 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-96f83187-1703-4cf4-8c88-803954b3f153 req-7e8a0644-2ef4-4fa5-b64a-a99093c6ef69 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.044106 Jul 16 22:03:51.728189 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3289/6583] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 22:03:51 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 22:03:51.882634 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-12492a17-20f3-4cd6-9969-3398cb0466ef tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:03:51.883389 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-12492a17-20f3-4cd6-9969-3398cb0466ef tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: 3447d99f-2076-4204-b4d2-55339d60ca3a] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:03:51.890449 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-12492a17-20f3-4cd6-9969-3398cb0466ef tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:03:51.890915 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-12492a17-20f3-4cd6-9969-3398cb0466ef tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:03:51.891715 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-12492a17-20f3-4cd6-9969-3398cb0466ef tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:03:51.892485 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-12492a17-20f3-4cd6-9969-3398cb0466ef tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:03:52.025307 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-12492a17-20f3-4cd6-9969-3398cb0466ef tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '718f6de3b1a1453d9579d421a574691e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd21231d9af34d9aa4f9e03229b49a30', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:03:52.032870 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-12492a17-20f3-4cd6-9969-3398cb0466ef tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/3447d99f-2076-4204-b4d2-55339d60ca3a" status: 200 len: 1679 microversion: 2.1 time: 0.154216 Jul 16 22:03:52.033447 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3295/6584] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:03:51 2022] GET /compute/v2.1/servers/3447d99f-2076-4204-b4d2-55339d60ca3a => generated 1679 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:03:53.052596 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-f7d7d533-9111-4a74-9b40-e18e03c3ec66 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:03:53.053298 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-f7d7d533-9111-4a74-9b40-e18e03c3ec66 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: 3447d99f-2076-4204-b4d2-55339d60ca3a] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:03:53.059731 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f7d7d533-9111-4a74-9b40-e18e03c3ec66 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:03:53.060184 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f7d7d533-9111-4a74-9b40-e18e03c3ec66 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:03:53.060953 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f7d7d533-9111-4a74-9b40-e18e03c3ec66 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:03:53.061394 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f7d7d533-9111-4a74-9b40-e18e03c3ec66 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:03:53.198568 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-f7d7d533-9111-4a74-9b40-e18e03c3ec66 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '718f6de3b1a1453d9579d421a574691e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd21231d9af34d9aa4f9e03229b49a30', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:03:53.207091 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-f7d7d533-9111-4a74-9b40-e18e03c3ec66 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/3447d99f-2076-4204-b4d2-55339d60ca3a" status: 200 len: 1675 microversion: 2.1 time: 0.160520 Jul 16 22:03:53.207673 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3290/6585] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:03:53 2022] GET /compute/v2.1/servers/3447d99f-2076-4204-b4d2-55339d60ca3a => generated 1675 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:03:53.223523 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-d56add3c-6db8-4251-8977-41caf5810d78 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Action: 'action', calling method: >, body: {"confirmResize": {}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:03:53.224085 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-d56add3c-6db8-4251-8977-41caf5810d78 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: 3447d99f-2076-4204-b4d2-55339d60ca3a] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:03:53.230939 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d56add3c-6db8-4251-8977-41caf5810d78 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:03:53.231436 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d56add3c-6db8-4251-8977-41caf5810d78 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:03:53.232211 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d56add3c-6db8-4251-8977-41caf5810d78 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:03:53.232656 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d56add3c-6db8-4251-8977-41caf5810d78 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:03:53.391898 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-d56add3c-6db8-4251-8977-41caf5810d78 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "POST /compute/v2.1/servers/3447d99f-2076-4204-b4d2-55339d60ca3a/action" status: 204 len: 0 microversion: 2.1 time: 0.172217 Jul 16 22:03:53.392581 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3296/6586] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 22:03:53 2022] POST /compute/v2.1/servers/3447d99f-2076-4204-b4d2-55339d60ca3a/action => generated 0 bytes in 174 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 16 22:03:53.400934 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-4166547d-291e-4916-b996-6f4bcdf9edc3 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:03:53.401552 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-4166547d-291e-4916-b996-6f4bcdf9edc3 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: 3447d99f-2076-4204-b4d2-55339d60ca3a] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:03:53.407901 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4166547d-291e-4916-b996-6f4bcdf9edc3 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:03:53.408359 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4166547d-291e-4916-b996-6f4bcdf9edc3 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:03:53.409325 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4166547d-291e-4916-b996-6f4bcdf9edc3 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:03:53.409766 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4166547d-291e-4916-b996-6f4bcdf9edc3 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:03:53.554557 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-4166547d-291e-4916-b996-6f4bcdf9edc3 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '718f6de3b1a1453d9579d421a574691e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd21231d9af34d9aa4f9e03229b49a30', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:03:53.566205 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-4166547d-291e-4916-b996-6f4bcdf9edc3 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/3447d99f-2076-4204-b4d2-55339d60ca3a" status: 200 len: 1675 microversion: 2.1 time: 0.167307 Jul 16 22:03:53.566959 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3291/6587] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:03:53 2022] GET /compute/v2.1/servers/3447d99f-2076-4204-b4d2-55339d60ca3a => generated 1675 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 22:03:54.586322 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-1eeeeba8-bbfc-4214-953c-28eb6ff98859 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:03:54.587090 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-1eeeeba8-bbfc-4214-953c-28eb6ff98859 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: 3447d99f-2076-4204-b4d2-55339d60ca3a] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:03:54.593476 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1eeeeba8-bbfc-4214-953c-28eb6ff98859 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:03:54.593943 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1eeeeba8-bbfc-4214-953c-28eb6ff98859 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:03:54.594722 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1eeeeba8-bbfc-4214-953c-28eb6ff98859 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:03:54.595234 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1eeeeba8-bbfc-4214-953c-28eb6ff98859 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:03:54.730653 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-1eeeeba8-bbfc-4214-953c-28eb6ff98859 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '718f6de3b1a1453d9579d421a574691e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd21231d9af34d9aa4f9e03229b49a30', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:03:54.738653 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-1eeeeba8-bbfc-4214-953c-28eb6ff98859 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/3447d99f-2076-4204-b4d2-55339d60ca3a" status: 200 len: 1675 microversion: 2.1 time: 0.157711 Jul 16 22:03:54.742421 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3297/6588] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:03:54 2022] GET /compute/v2.1/servers/3447d99f-2076-4204-b4d2-55339d60ca3a => generated 1675 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 22:03:55.690843 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-9fa12d23-50be-41d4-841e-6ec1f823092f req-5ae62fff-ee48-4736-b7e4-e02dc18d23ea service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "3447d99f-2076-4204-b4d2-55339d60ca3a", "name": "network-vif-plugged", "status": "completed", "tag": "35e08dc0-521e-4023-9b5b-53b6391890cf"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:03:55.699799 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-9fa12d23-50be-41d4-841e-6ec1f823092f req-5ae62fff-ee48-4736-b7e4-e02dc18d23ea service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:03:55.700255 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-9fa12d23-50be-41d4-841e-6ec1f823092f req-5ae62fff-ee48-4736-b7e4-e02dc18d23ea service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:03:55.712310 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-9fa12d23-50be-41d4-841e-6ec1f823092f req-5ae62fff-ee48-4736-b7e4-e02dc18d23ea service nova] Creating event network-vif-plugged:35e08dc0-521e-4023-9b5b-53b6391890cf for instance 3447d99f-2076-4204-b4d2-55339d60ca3a on nch2-master-1 Jul 16 22:03:55.717239 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-9fa12d23-50be-41d4-841e-6ec1f823092f req-5ae62fff-ee48-4736-b7e4-e02dc18d23ea service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.031075 Jul 16 22:03:55.719345 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3292/6589] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 22:03:55 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 22:03:55.756727 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-ddb6b044-ba92-4afb-b752-ba5a92d8a81a tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:03:55.757345 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-ddb6b044-ba92-4afb-b752-ba5a92d8a81a tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: 3447d99f-2076-4204-b4d2-55339d60ca3a] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:03:55.764500 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ddb6b044-ba92-4afb-b752-ba5a92d8a81a tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:03:55.764972 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ddb6b044-ba92-4afb-b752-ba5a92d8a81a tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:03:55.765767 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ddb6b044-ba92-4afb-b752-ba5a92d8a81a tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:03:55.766222 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ddb6b044-ba92-4afb-b752-ba5a92d8a81a tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:03:55.901475 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-ddb6b044-ba92-4afb-b752-ba5a92d8a81a tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '718f6de3b1a1453d9579d421a574691e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd21231d9af34d9aa4f9e03229b49a30', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:03:55.909705 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-ddb6b044-ba92-4afb-b752-ba5a92d8a81a tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/3447d99f-2076-4204-b4d2-55339d60ca3a" status: 200 len: 1654 microversion: 2.1 time: 0.156863 Jul 16 22:03:55.910270 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3298/6590] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:03:55 2022] GET /compute/v2.1/servers/3447d99f-2076-4204-b4d2-55339d60ca3a => generated 1654 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 22:03:55.927138 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-ce529717-b39a-498d-9c09-0d0d49a07d6e tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:03:55.927922 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-ce529717-b39a-498d-9c09-0d0d49a07d6e tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: 3447d99f-2076-4204-b4d2-55339d60ca3a] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:03:55.934639 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ce529717-b39a-498d-9c09-0d0d49a07d6e tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:03:55.935129 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ce529717-b39a-498d-9c09-0d0d49a07d6e tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:03:55.935901 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ce529717-b39a-498d-9c09-0d0d49a07d6e tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:03:55.936347 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ce529717-b39a-498d-9c09-0d0d49a07d6e tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:03:56.066922 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-ce529717-b39a-498d-9c09-0d0d49a07d6e tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '718f6de3b1a1453d9579d421a574691e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd21231d9af34d9aa4f9e03229b49a30', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:03:56.074789 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-ce529717-b39a-498d-9c09-0d0d49a07d6e tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/3447d99f-2076-4204-b4d2-55339d60ca3a" status: 200 len: 1654 microversion: 2.1 time: 0.152628 Jul 16 22:03:56.075390 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3293/6591] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:03:55 2022] GET /compute/v2.1/servers/3447d99f-2076-4204-b4d2-55339d60ca3a => generated 1654 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:03:56.091554 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-2dcfc69c-663c-4d84-a2b6-2526a6cf9fb6 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Action: 'action', calling method: >, body: {"os-start": {}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:03:56.098520 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2dcfc69c-663c-4d84-a2b6-2526a6cf9fb6 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:03:56.100401 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2dcfc69c-663c-4d84-a2b6-2526a6cf9fb6 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:03:56.121586 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-2dcfc69c-663c-4d84-a2b6-2526a6cf9fb6 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: 3447d99f-2076-4204-b4d2-55339d60ca3a] Going to try to start instance {{(pid=138681) start /opt/stack/nova/nova/compute/api.py:2730}} Jul 16 22:03:56.126100 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:03:56.301991 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-2dcfc69c-663c-4d84-a2b6-2526a6cf9fb6 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "POST /compute/v2.1/servers/3447d99f-2076-4204-b4d2-55339d60ca3a/action" status: 202 len: 0 microversion: 2.1 time: 0.214097 Jul 16 22:03:56.302528 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3299/6592] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 22:03:56 2022] POST /compute/v2.1/servers/3447d99f-2076-4204-b4d2-55339d60ca3a/action => generated 0 bytes in 216 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 22:03:56.310931 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-820bc490-5313-42f6-b09a-f825c335b6d6 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:03:56.311583 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-820bc490-5313-42f6-b09a-f825c335b6d6 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: 3447d99f-2076-4204-b4d2-55339d60ca3a] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:03:56.318117 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-820bc490-5313-42f6-b09a-f825c335b6d6 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:03:56.318569 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-820bc490-5313-42f6-b09a-f825c335b6d6 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:03:56.319381 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-820bc490-5313-42f6-b09a-f825c335b6d6 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:03:56.319840 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-820bc490-5313-42f6-b09a-f825c335b6d6 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:03:56.324019 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=96,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:03:56.458708 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-820bc490-5313-42f6-b09a-f825c335b6d6 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '718f6de3b1a1453d9579d421a574691e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd21231d9af34d9aa4f9e03229b49a30', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:03:56.466645 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-820bc490-5313-42f6-b09a-f825c335b6d6 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/3447d99f-2076-4204-b4d2-55339d60ca3a" status: 200 len: 1663 microversion: 2.1 time: 0.159754 Jul 16 22:03:56.467289 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3294/6593] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:03:56 2022] GET /compute/v2.1/servers/3447d99f-2076-4204-b4d2-55339d60ca3a => generated 1663 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:03:57.486372 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-ce579914-8e96-4a83-86a8-4d9c0fe038e5 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:03:57.487104 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-ce579914-8e96-4a83-86a8-4d9c0fe038e5 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: 3447d99f-2076-4204-b4d2-55339d60ca3a] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:03:57.495183 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ce579914-8e96-4a83-86a8-4d9c0fe038e5 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:03:57.495651 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ce579914-8e96-4a83-86a8-4d9c0fe038e5 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:03:57.496418 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ce579914-8e96-4a83-86a8-4d9c0fe038e5 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:03:57.496866 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ce579914-8e96-4a83-86a8-4d9c0fe038e5 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:03:57.631599 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-ce579914-8e96-4a83-86a8-4d9c0fe038e5 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '718f6de3b1a1453d9579d421a574691e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd21231d9af34d9aa4f9e03229b49a30', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:03:57.639256 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-ce579914-8e96-4a83-86a8-4d9c0fe038e5 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/3447d99f-2076-4204-b4d2-55339d60ca3a" status: 200 len: 1663 microversion: 2.1 time: 0.157707 Jul 16 22:03:57.639832 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3300/6594] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:03:57 2022] GET /compute/v2.1/servers/3447d99f-2076-4204-b4d2-55339d60ca3a => generated 1663 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:03:58.657546 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-ae2b6c41-d7f8-4b76-ac22-c2a6a8917524 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:03:58.658242 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-ae2b6c41-d7f8-4b76-ac22-c2a6a8917524 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: 3447d99f-2076-4204-b4d2-55339d60ca3a] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:03:58.665236 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ae2b6c41-d7f8-4b76-ac22-c2a6a8917524 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:03:58.665695 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ae2b6c41-d7f8-4b76-ac22-c2a6a8917524 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:03:58.666460 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ae2b6c41-d7f8-4b76-ac22-c2a6a8917524 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:03:58.666904 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ae2b6c41-d7f8-4b76-ac22-c2a6a8917524 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:03:58.797250 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-ae2b6c41-d7f8-4b76-ac22-c2a6a8917524 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '718f6de3b1a1453d9579d421a574691e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd21231d9af34d9aa4f9e03229b49a30', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:03:58.805553 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-ae2b6c41-d7f8-4b76-ac22-c2a6a8917524 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/3447d99f-2076-4204-b4d2-55339d60ca3a" status: 200 len: 1667 microversion: 2.1 time: 0.152279 Jul 16 22:03:58.806123 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3295/6595] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:03:58 2022] GET /compute/v2.1/servers/3447d99f-2076-4204-b4d2-55339d60ca3a => generated 1667 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:03:58.822563 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-6cc54e58-4992-4a70-9377-82b719cffa1b tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:03:58.823204 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-6cc54e58-4992-4a70-9377-82b719cffa1b tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: 3447d99f-2076-4204-b4d2-55339d60ca3a] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:03:58.830180 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6cc54e58-4992-4a70-9377-82b719cffa1b tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:03:58.830643 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6cc54e58-4992-4a70-9377-82b719cffa1b tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:03:58.831773 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6cc54e58-4992-4a70-9377-82b719cffa1b tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:03:58.832225 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6cc54e58-4992-4a70-9377-82b719cffa1b tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:03:58.859779 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-6cc54e58-4992-4a70-9377-82b719cffa1b tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: 3447d99f-2076-4204-b4d2-55339d60ca3a] Going to try to terminate instance {{(pid=138681) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 16 22:03:59.116581 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-6cc54e58-4992-4a70-9377-82b719cffa1b tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "DELETE /compute/v2.1/servers/3447d99f-2076-4204-b4d2-55339d60ca3a" status: 204 len: 0 microversion: 2.1 time: 0.297689 Jul 16 22:03:59.117209 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3301/6596] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 22:03:58 2022] DELETE /compute/v2.1/servers/3447d99f-2076-4204-b4d2-55339d60ca3a => generated 0 bytes in 300 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 16 22:03:59.125546 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-66b2c6df-d38b-4615-aad0-dd6e1ceb81f9 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:03:59.126167 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-66b2c6df-d38b-4615-aad0-dd6e1ceb81f9 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: 3447d99f-2076-4204-b4d2-55339d60ca3a] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:03:59.132369 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-66b2c6df-d38b-4615-aad0-dd6e1ceb81f9 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:03:59.132828 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-66b2c6df-d38b-4615-aad0-dd6e1ceb81f9 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:03:59.133595 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-66b2c6df-d38b-4615-aad0-dd6e1ceb81f9 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:03:59.134042 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-66b2c6df-d38b-4615-aad0-dd6e1ceb81f9 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:03:59.267652 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-66b2c6df-d38b-4615-aad0-dd6e1ceb81f9 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '718f6de3b1a1453d9579d421a574691e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd21231d9af34d9aa4f9e03229b49a30', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:03:59.275850 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-66b2c6df-d38b-4615-aad0-dd6e1ceb81f9 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/3447d99f-2076-4204-b4d2-55339d60ca3a" status: 200 len: 1673 microversion: 2.1 time: 0.154116 Jul 16 22:03:59.276420 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3296/6597] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:03:59 2022] GET /compute/v2.1/servers/3447d99f-2076-4204-b4d2-55339d60ca3a => generated 1673 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:04:00.294269 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-57bad4ec-7ba1-4e4f-b74b-da66b3b6fe01 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:04:00.294904 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-57bad4ec-7ba1-4e4f-b74b-da66b3b6fe01 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: 3447d99f-2076-4204-b4d2-55339d60ca3a] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:04:00.301843 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-57bad4ec-7ba1-4e4f-b74b-da66b3b6fe01 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:04:00.302512 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-57bad4ec-7ba1-4e4f-b74b-da66b3b6fe01 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:04:00.303328 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-57bad4ec-7ba1-4e4f-b74b-da66b3b6fe01 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:04:00.303780 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-57bad4ec-7ba1-4e4f-b74b-da66b3b6fe01 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:04:00.437708 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-57bad4ec-7ba1-4e4f-b74b-da66b3b6fe01 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '718f6de3b1a1453d9579d421a574691e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd21231d9af34d9aa4f9e03229b49a30', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:04:00.446524 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-57bad4ec-7ba1-4e4f-b74b-da66b3b6fe01 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/3447d99f-2076-4204-b4d2-55339d60ca3a" status: 200 len: 1673 microversion: 2.1 time: 0.156460 Jul 16 22:04:00.447112 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3302/6598] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:04:00 2022] GET /compute/v2.1/servers/3447d99f-2076-4204-b4d2-55339d60ca3a => generated 1673 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:04:01.464354 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-a8593558-aad0-4fe2-954f-020feae5c6b8 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:04:01.464977 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-a8593558-aad0-4fe2-954f-020feae5c6b8 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: 3447d99f-2076-4204-b4d2-55339d60ca3a] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:04:01.468533 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_api:SELECT=41 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:04:01.472683 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-f6604f11-63f3-4302-8d32-499f3c69db56 req-942d5770-5999-4df0-859e-7b139ab93a0c service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "3447d99f-2076-4204-b4d2-55339d60ca3a", "name": "network-vif-deleted", "tag": "35e08dc0-521e-4023-9b5b-53b6391890cf"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:04:01.477904 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a8593558-aad0-4fe2-954f-020feae5c6b8 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:04:01.478375 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a8593558-aad0-4fe2-954f-020feae5c6b8 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:04:01.479217 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a8593558-aad0-4fe2-954f-020feae5c6b8 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:04:01.479799 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a8593558-aad0-4fe2-954f-020feae5c6b8 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:04:01.482864 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-f6604f11-63f3-4302-8d32-499f3c69db56 req-942d5770-5999-4df0-859e-7b139ab93a0c service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:04:01.483354 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-f6604f11-63f3-4302-8d32-499f3c69db56 req-942d5770-5999-4df0-859e-7b139ab93a0c service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:04:01.497241 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-f6604f11-63f3-4302-8d32-499f3c69db56 req-942d5770-5999-4df0-859e-7b139ab93a0c service nova] Creating event network-vif-deleted:35e08dc0-521e-4023-9b5b-53b6391890cf for instance 3447d99f-2076-4204-b4d2-55339d60ca3a on nch2-master-1 Jul 16 22:04:01.502204 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-f6604f11-63f3-4302-8d32-499f3c69db56 req-942d5770-5999-4df0-859e-7b139ab93a0c service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.033083 Jul 16 22:04:01.502811 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3303/6599] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 22:04:01 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 16 22:04:01.554087 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-a8593558-aad0-4fe2-954f-020feae5c6b8 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '718f6de3b1a1453d9579d421a574691e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd21231d9af34d9aa4f9e03229b49a30', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:04:01.566253 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-a8593558-aad0-4fe2-954f-020feae5c6b8 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/3447d99f-2076-4204-b4d2-55339d60ca3a" status: 200 len: 1606 microversion: 2.1 time: 0.105828 Jul 16 22:04:01.567093 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3297/6600] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:04:01 2022] GET /compute/v2.1/servers/3447d99f-2076-4204-b4d2-55339d60ca3a => generated 1606 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:04:02.586312 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-73ce9daa-3fcb-4dac-93b9-b481c261dae1 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:04:02.586986 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-73ce9daa-3fcb-4dac-93b9-b481c261dae1 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: 3447d99f-2076-4204-b4d2-55339d60ca3a] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:04:02.590933 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_api:SELECT=46,nova_api:UPDATE=1 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:04:02.594669 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-73ce9daa-3fcb-4dac-93b9-b481c261dae1 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:04:02.595173 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-73ce9daa-3fcb-4dac-93b9-b481c261dae1 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:04:02.595956 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-73ce9daa-3fcb-4dac-93b9-b481c261dae1 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:04:02.596415 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-73ce9daa-3fcb-4dac-93b9-b481c261dae1 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:04:02.610384 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-73ce9daa-3fcb-4dac-93b9-b481c261dae1 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] HTTP exception thrown: Instance 3447d99f-2076-4204-b4d2-55339d60ca3a could not be found. Jul 16 22:04:02.612078 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-73ce9daa-3fcb-4dac-93b9-b481c261dae1 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Request method failure captured: Jul 16 22:04:02.612078 ncdv-master-1 devstack@n-api.service[138681]: request: GET /compute/v2.1/servers/3447d99f-2076-4204-b4d2-55339d60ca3a HTTP/1.1 Jul 16 22:04:02.612078 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 22:04:02.612078 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 22:04:02.612078 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 22:04:02.612078 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 22:04:02.612078 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 22:04:02.612078 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 22:04:02.612078 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 22:04:02.612078 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 22:04:02.612078 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0zUsISRuFerqPWLnIU6pWtJZ1G33kS5UQBpOE-Qn9W1zRIsDO0obaxdGmXyhe5FNe11pjr8YSFwmEAtO4tE-QrUwHNB6R_8ZgLrv3bb_zQTEpcOValb2w1FWdJ1eIHV_96nHchz9PmXAJpOM5Cp_lLo9a71M8MQcZrQgoRzDb41oZDA Jul 16 22:04:02.612078 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 22:04:02.612078 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 22:04:02.612078 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 22:04:02.612078 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 22:04:02.612078 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 22:04:02.612078 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 22:04:02.612078 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: fd21231d9af34d9aa4f9e03229b49a30 Jul 16 22:04:02.612078 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-ServerActionsTestJSON-164809533 Jul 16 22:04:02.612078 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 22:04:02.612078 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 22:04:02.612078 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/fd21231d9af34d9aa4f9e03229b49a30"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/fd21231d9af34d9aa4f9e03229b49a30"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_fd21231d9af34d9aa4f9e03229b49a30"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/fd21231d9af34d9aa4f9e03229b49a30"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 22:04:02.612078 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-ServerActionsTestJSON-164809533 Jul 16 22:04:02.612078 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: fd21231d9af34d9aa4f9e03229b49a30 Jul 16 22:04:02.612078 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-ServerActionsTestJSON-164809533 Jul 16 22:04:02.612078 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-ServerActionsTestJSON-164809533-project Jul 16 22:04:02.612078 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 22:04:02.612078 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 22:04:02.612078 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: 718f6de3b1a1453d9579d421a574691e Jul 16 22:04:02.612078 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-ServerActionsTestJSON-164809533-project Jul 16 22:04:02.612078 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 22:04:02.612078 ncdv-master-1 devstack@n-api.service[138681]: exception: Instance 3447d99f-2076-4204-b4d2-55339d60ca3a could not be found. Jul 16 22:04:02.612078 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': '3447d99f-2076-4204-b4d2-55339d60ca3a'} Jul 16 22:04:02.612078 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 22:04:02.612078 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 22:04:02.612078 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 22:04:02.612078 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 22:04:02.612078 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 22:04:02.612078 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 22:04:02.612078 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 22:04:02.612078 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 22:04:02.612078 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 22:04:02.612078 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 22:04:02.612078 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 3447d99f-2076-4204-b4d2-55339d60ca3a could not be found. Jul 16 22:04:02.612078 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 22:04:02.612078 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 22:04:02.612078 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 22:04:02.612078 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 22:04:02.612078 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 22:04:02.612078 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 22:04:02.612078 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 22:04:02.612078 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 22:04:02.612078 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 22:04:02.612078 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 22:04:02.612078 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 22:04:02.612078 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 22:04:02.612078 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 22:04:02.612078 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 22:04:02.612078 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 22:04:02.612078 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 22:04:02.612078 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 3447d99f-2076-4204-b4d2-55339d60ca3a could not be found. Jul 16 22:04:02.612078 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 22:04:02.612078 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 22:04:02.612078 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 22:04:02.616988 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 22:04:02.616988 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 22:04:02.616988 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 22:04:02.616988 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 22:04:02.616988 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 22:04:02.616988 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 3447d99f-2076-4204-b4d2-55339d60ca3a could not be found. Jul 16 22:04:02.616988 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 22:04:02.616988 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-73ce9daa-3fcb-4dac-93b9-b481c261dae1 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Returning 404 to user: Instance 3447d99f-2076-4204-b4d2-55339d60ca3a could not be found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 22:04:02.616988 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-73ce9daa-3fcb-4dac-93b9-b481c261dae1 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/3447d99f-2076-4204-b4d2-55339d60ca3a" status: 404 len: 111 microversion: 2.1 time: 0.032830 Jul 16 22:04:02.616988 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3304/6601] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:04:02 2022] GET /compute/v2.1/servers/3447d99f-2076-4204-b4d2-55339d60ca3a => generated 111 bytes in 34 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 22:04:02.632171 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-e69d24d5-5312-49bc-afbf-0b819f790130 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:04:02.632790 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-e69d24d5-5312-49bc-afbf-0b819f790130 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: 3447d99f-2076-4204-b4d2-55339d60ca3a] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:04:02.639985 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e69d24d5-5312-49bc-afbf-0b819f790130 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:04:02.640439 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e69d24d5-5312-49bc-afbf-0b819f790130 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:04:02.641202 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e69d24d5-5312-49bc-afbf-0b819f790130 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:04:02.641644 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e69d24d5-5312-49bc-afbf-0b819f790130 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:04:02.656905 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-e69d24d5-5312-49bc-afbf-0b819f790130 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] HTTP exception thrown: Instance 3447d99f-2076-4204-b4d2-55339d60ca3a could not be found. Jul 16 22:04:02.658592 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-e69d24d5-5312-49bc-afbf-0b819f790130 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Request method failure captured: Jul 16 22:04:02.658592 ncdv-master-1 devstack@n-api.service[138680]: request: GET /compute/v2.1/servers/3447d99f-2076-4204-b4d2-55339d60ca3a HTTP/1.1 Jul 16 22:04:02.658592 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 22:04:02.658592 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 22:04:02.658592 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 22:04:02.658592 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 0 Jul 16 22:04:02.658592 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 22:04:02.658592 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 22:04:02.658592 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 22:04:02.658592 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 22:04:02.658592 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0zUsISRuFerqPWLnIU6pWtJZ1G33kS5UQBpOE-Qn9W1zRIsDO0obaxdGmXyhe5FNe11pjr8YSFwmEAtO4tE-QrUwHNB6R_8ZgLrv3bb_zQTEpcOValb2w1FWdJ1eIHV_96nHchz9PmXAJpOM5Cp_lLo9a71M8MQcZrQgoRzDb41oZDA Jul 16 22:04:02.658592 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 22:04:02.658592 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 22:04:02.658592 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 22:04:02.658592 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 22:04:02.658592 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 22:04:02.658592 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 22:04:02.658592 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: fd21231d9af34d9aa4f9e03229b49a30 Jul 16 22:04:02.658592 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-ServerActionsTestJSON-164809533 Jul 16 22:04:02.658592 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 22:04:02.658592 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 22:04:02.658592 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/fd21231d9af34d9aa4f9e03229b49a30"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/fd21231d9af34d9aa4f9e03229b49a30"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_fd21231d9af34d9aa4f9e03229b49a30"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/fd21231d9af34d9aa4f9e03229b49a30"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 22:04:02.658592 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-ServerActionsTestJSON-164809533 Jul 16 22:04:02.658592 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: fd21231d9af34d9aa4f9e03229b49a30 Jul 16 22:04:02.658592 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-ServerActionsTestJSON-164809533 Jul 16 22:04:02.658592 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-ServerActionsTestJSON-164809533-project Jul 16 22:04:02.658592 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 22:04:02.658592 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 22:04:02.658592 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: 718f6de3b1a1453d9579d421a574691e Jul 16 22:04:02.658592 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-ServerActionsTestJSON-164809533-project Jul 16 22:04:02.658592 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 22:04:02.658592 ncdv-master-1 devstack@n-api.service[138680]: exception: Instance 3447d99f-2076-4204-b4d2-55339d60ca3a could not be found. Jul 16 22:04:02.658592 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': '3447d99f-2076-4204-b4d2-55339d60ca3a'} Jul 16 22:04:02.658592 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 22:04:02.658592 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 22:04:02.658592 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 22:04:02.658592 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 22:04:02.658592 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 22:04:02.658592 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 22:04:02.658592 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 22:04:02.658592 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 22:04:02.658592 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 22:04:02.658592 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 22:04:02.658592 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 3447d99f-2076-4204-b4d2-55339d60ca3a could not be found. Jul 16 22:04:02.658592 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 22:04:02.658592 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 22:04:02.658592 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 22:04:02.658592 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 22:04:02.658592 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 22:04:02.658592 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 22:04:02.658592 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 22:04:02.658592 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 22:04:02.658592 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 22:04:02.658592 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 22:04:02.658592 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 22:04:02.658592 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 22:04:02.658592 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 22:04:02.658592 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 22:04:02.658592 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 22:04:02.658592 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 22:04:02.658592 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 3447d99f-2076-4204-b4d2-55339d60ca3a could not be found. Jul 16 22:04:02.658592 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 22:04:02.658592 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 22:04:02.658592 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 22:04:02.664232 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 22:04:02.664232 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 22:04:02.664232 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 22:04:02.664232 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 22:04:02.664232 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 22:04:02.664232 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 3447d99f-2076-4204-b4d2-55339d60ca3a could not be found. Jul 16 22:04:02.664232 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 22:04:02.664232 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-e69d24d5-5312-49bc-afbf-0b819f790130 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Returning 404 to user: Instance 3447d99f-2076-4204-b4d2-55339d60ca3a could not be found. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 22:04:02.664232 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-e69d24d5-5312-49bc-afbf-0b819f790130 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/3447d99f-2076-4204-b4d2-55339d60ca3a" status: 404 len: 111 microversion: 2.1 time: 0.033281 Jul 16 22:04:02.664232 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3298/6602] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:04:02 2022] GET /compute/v2.1/servers/3447d99f-2076-4204-b4d2-55339d60ca3a => generated 111 bytes in 35 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 22:04:02.711105 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-232d2cbd-bba9-46b1-8ae0-7e24d6dfed73 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerActionsTestJSON-server-1588838485", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452", "networks": [{"uuid": "b3593d62-fee4-4445-83ed-ea9589fa3fcd"}], "security_groups": [{"name": "tempest-securitygroup--389060960"}], "key_name": "tempest-keypair-26228548", "user_data": "IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIHVidW50dSB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH51YnVudHUvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo="}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:04:02.814261 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.network.neutron [None req-232d2cbd-bba9-46b1-8ae0-7e24d6dfed73 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] validate_networks() for [('b3593d62-fee4-4445-83ed-ea9589fa3fcd', None, None, None, None, None)] {{(pid=138681) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 16 22:04:03.242572 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-232d2cbd-bba9-46b1-8ae0-7e24d6dfed73 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Flavor limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 16 22:04:03.243039 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-232d2cbd-bba9-46b1-8ae0-7e24d6dfed73 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Image limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 16 22:04:03.243547 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-232d2cbd-bba9-46b1-8ae0-7e24d6dfed73 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Flavor pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 16 22:04:03.243976 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-232d2cbd-bba9-46b1-8ae0-7e24d6dfed73 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Image pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 16 22:04:03.244412 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-232d2cbd-bba9-46b1-8ae0-7e24d6dfed73 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 16 22:04:03.253350 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-232d2cbd-bba9-46b1-8ae0-7e24d6dfed73 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:04:03.253826 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-232d2cbd-bba9-46b1-8ae0-7e24d6dfed73 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:04:03.254673 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-232d2cbd-bba9-46b1-8ae0-7e24d6dfed73 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:04:03.255271 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-232d2cbd-bba9-46b1-8ae0-7e24d6dfed73 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:04:03.272941 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-232d2cbd-bba9-46b1-8ae0-7e24d6dfed73 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Getting quotas for project fd21231d9af34d9aa4f9e03229b49a30. Resources: {'ram', 'instances', 'cores'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 22:04:03.278142 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-232d2cbd-bba9-46b1-8ae0-7e24d6dfed73 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Getting quotas for user 718f6de3b1a1453d9579d421a574691e and project fd21231d9af34d9aa4f9e03229b49a30. Resources: {'ram', 'instances', 'cores'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 22:04:03.291275 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-232d2cbd-bba9-46b1-8ae0-7e24d6dfed73 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Going to run 1 instances... {{(pid=138681) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 16 22:04:03.293545 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-232d2cbd-bba9-46b1-8ae0-7e24d6dfed73 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: fa9945a3-51cc-4366-bc7f-31ddb4f0fa01] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138681) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 16 22:04:03.418402 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-232d2cbd-bba9-46b1-8ae0-7e24d6dfed73 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 405 microversion: 2.1 time: 0.712500 Jul 16 22:04:03.418975 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3305/6603] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 22:04:02 2022] POST /compute/v2.1/servers => generated 405 bytes in 714 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Jul 16 22:04:03.430426 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-61562b8c-5884-4451-af39-69aab1a43acb tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:04:03.431149 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-61562b8c-5884-4451-af39-69aab1a43acb tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: fa9945a3-51cc-4366-bc7f-31ddb4f0fa01] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:04:03.489735 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-61562b8c-5884-4451-af39-69aab1a43acb tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '718f6de3b1a1453d9579d421a574691e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd21231d9af34d9aa4f9e03229b49a30', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:04:03.497515 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-61562b8c-5884-4451-af39-69aab1a43acb tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/fa9945a3-51cc-4366-bc7f-31ddb4f0fa01" status: 200 len: 1237 microversion: 2.1 time: 0.072092 Jul 16 22:04:03.498090 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3299/6604] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:04:03 2022] GET /compute/v2.1/servers/fa9945a3-51cc-4366-bc7f-31ddb4f0fa01 => generated 1237 bytes in 73 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:04:04.516313 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-b6f99068-0d87-4d54-8bf1-a2c0ae5666d2 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:04:04.516954 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-b6f99068-0d87-4d54-8bf1-a2c0ae5666d2 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: fa9945a3-51cc-4366-bc7f-31ddb4f0fa01] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:04:04.524641 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b6f99068-0d87-4d54-8bf1-a2c0ae5666d2 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:04:04.525199 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b6f99068-0d87-4d54-8bf1-a2c0ae5666d2 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:04:04.525908 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b6f99068-0d87-4d54-8bf1-a2c0ae5666d2 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:04:04.526365 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b6f99068-0d87-4d54-8bf1-a2c0ae5666d2 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:04:04.588801 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-b6f99068-0d87-4d54-8bf1-a2c0ae5666d2 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '718f6de3b1a1453d9579d421a574691e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd21231d9af34d9aa4f9e03229b49a30', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:04:04.596509 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-b6f99068-0d87-4d54-8bf1-a2c0ae5666d2 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/fa9945a3-51cc-4366-bc7f-31ddb4f0fa01" status: 200 len: 1241 microversion: 2.1 time: 0.085018 Jul 16 22:04:04.597089 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3306/6605] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:04:04 2022] GET /compute/v2.1/servers/fa9945a3-51cc-4366-bc7f-31ddb4f0fa01 => generated 1241 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 22:04:05.616550 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-aa09e86b-e9cc-408f-9ce9-00fd70bf5fee tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:04:05.617164 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-aa09e86b-e9cc-408f-9ce9-00fd70bf5fee tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: fa9945a3-51cc-4366-bc7f-31ddb4f0fa01] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:04:05.624377 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-aa09e86b-e9cc-408f-9ce9-00fd70bf5fee tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:04:05.624832 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-aa09e86b-e9cc-408f-9ce9-00fd70bf5fee tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:04:05.625595 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-aa09e86b-e9cc-408f-9ce9-00fd70bf5fee tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:04:05.626032 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-aa09e86b-e9cc-408f-9ce9-00fd70bf5fee tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:04:05.688208 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-aa09e86b-e9cc-408f-9ce9-00fd70bf5fee tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '718f6de3b1a1453d9579d421a574691e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd21231d9af34d9aa4f9e03229b49a30', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:04:05.696692 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-aa09e86b-e9cc-408f-9ce9-00fd70bf5fee tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/fa9945a3-51cc-4366-bc7f-31ddb4f0fa01" status: 200 len: 1307 microversion: 2.1 time: 0.084964 Jul 16 22:04:05.697840 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3300/6606] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:04:05 2022] GET /compute/v2.1/servers/fa9945a3-51cc-4366-bc7f-31ddb4f0fa01 => generated 1307 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 22:04:06.721864 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-8a4db789-948a-4429-ae07-e18b4a8719af tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:04:06.722502 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-8a4db789-948a-4429-ae07-e18b4a8719af tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: fa9945a3-51cc-4366-bc7f-31ddb4f0fa01] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:04:06.730698 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8a4db789-948a-4429-ae07-e18b4a8719af tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:04:06.731206 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8a4db789-948a-4429-ae07-e18b4a8719af tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:04:06.732000 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8a4db789-948a-4429-ae07-e18b4a8719af tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:04:06.732444 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8a4db789-948a-4429-ae07-e18b4a8719af tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:04:06.881810 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-8a4db789-948a-4429-ae07-e18b4a8719af tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '718f6de3b1a1453d9579d421a574691e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd21231d9af34d9aa4f9e03229b49a30', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:04:06.890204 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-8a4db789-948a-4429-ae07-e18b4a8719af tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/fa9945a3-51cc-4366-bc7f-31ddb4f0fa01" status: 200 len: 1362 microversion: 2.1 time: 0.172223 Jul 16 22:04:06.891356 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3307/6607] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:04:06 2022] GET /compute/v2.1/servers/fa9945a3-51cc-4366-bc7f-31ddb4f0fa01 => generated 1362 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:04:07.059136 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-730a7d99-9885-46a0-b29c-24dcab32f78b req-13908ef1-ef51-4b72-9ddb-abf5d362f01b service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "fa9945a3-51cc-4366-bc7f-31ddb4f0fa01", "tag": "1ca4d162-bc94-4c13-a163-3f3656a349e8"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:04:07.074266 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-730a7d99-9885-46a0-b29c-24dcab32f78b req-13908ef1-ef51-4b72-9ddb-abf5d362f01b service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:04:07.074266 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-730a7d99-9885-46a0-b29c-24dcab32f78b req-13908ef1-ef51-4b72-9ddb-abf5d362f01b service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:04:07.084130 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-730a7d99-9885-46a0-b29c-24dcab32f78b req-13908ef1-ef51-4b72-9ddb-abf5d362f01b service nova] Creating event network-changed:1ca4d162-bc94-4c13-a163-3f3656a349e8 for instance fa9945a3-51cc-4366-bc7f-31ddb4f0fa01 on nch1-master-1 Jul 16 22:04:07.089222 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-730a7d99-9885-46a0-b29c-24dcab32f78b req-13908ef1-ef51-4b72-9ddb-abf5d362f01b service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.033803 Jul 16 22:04:07.090217 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3301/6608] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 22:04:07 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 22:04:07.908671 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-f4267b5c-5382-4e1d-a948-5e6f8c5f8eaf tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:04:07.909339 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-f4267b5c-5382-4e1d-a948-5e6f8c5f8eaf tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: fa9945a3-51cc-4366-bc7f-31ddb4f0fa01] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:04:07.916984 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f4267b5c-5382-4e1d-a948-5e6f8c5f8eaf tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:04:07.917445 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f4267b5c-5382-4e1d-a948-5e6f8c5f8eaf tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:04:07.918215 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f4267b5c-5382-4e1d-a948-5e6f8c5f8eaf tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:04:07.920638 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f4267b5c-5382-4e1d-a948-5e6f8c5f8eaf tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:04:08.048720 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-f4267b5c-5382-4e1d-a948-5e6f8c5f8eaf tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '718f6de3b1a1453d9579d421a574691e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd21231d9af34d9aa4f9e03229b49a30', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:04:08.056837 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-f4267b5c-5382-4e1d-a948-5e6f8c5f8eaf tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/fa9945a3-51cc-4366-bc7f-31ddb4f0fa01" status: 200 len: 1362 microversion: 2.1 time: 0.152326 Jul 16 22:04:08.057408 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3308/6609] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:04:07 2022] GET /compute/v2.1/servers/fa9945a3-51cc-4366-bc7f-31ddb4f0fa01 => generated 1362 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:04:09.075725 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-065e5979-a0bb-4b05-9825-1167e8922e5e tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:04:09.076433 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-065e5979-a0bb-4b05-9825-1167e8922e5e tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: fa9945a3-51cc-4366-bc7f-31ddb4f0fa01] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:04:09.083324 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-065e5979-a0bb-4b05-9825-1167e8922e5e tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:04:09.083767 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-065e5979-a0bb-4b05-9825-1167e8922e5e tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:04:09.084537 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-065e5979-a0bb-4b05-9825-1167e8922e5e tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:04:09.084981 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-065e5979-a0bb-4b05-9825-1167e8922e5e tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:04:09.217954 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-065e5979-a0bb-4b05-9825-1167e8922e5e tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '718f6de3b1a1453d9579d421a574691e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd21231d9af34d9aa4f9e03229b49a30', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:04:09.225536 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-065e5979-a0bb-4b05-9825-1167e8922e5e tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/fa9945a3-51cc-4366-bc7f-31ddb4f0fa01" status: 200 len: 1362 microversion: 2.1 time: 0.155051 Jul 16 22:04:09.226116 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3302/6610] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:04:09 2022] GET /compute/v2.1/servers/fa9945a3-51cc-4366-bc7f-31ddb4f0fa01 => generated 1362 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:04:10.243670 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-8a971c8c-7bdd-41b6-ab7f-a07acbbfede4 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:04:10.244232 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-8a971c8c-7bdd-41b6-ab7f-a07acbbfede4 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: fa9945a3-51cc-4366-bc7f-31ddb4f0fa01] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:04:10.251167 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8a971c8c-7bdd-41b6-ab7f-a07acbbfede4 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:04:10.251636 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8a971c8c-7bdd-41b6-ab7f-a07acbbfede4 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:04:10.252416 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8a971c8c-7bdd-41b6-ab7f-a07acbbfede4 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:04:10.252871 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8a971c8c-7bdd-41b6-ab7f-a07acbbfede4 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:04:10.390320 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-8a971c8c-7bdd-41b6-ab7f-a07acbbfede4 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '718f6de3b1a1453d9579d421a574691e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd21231d9af34d9aa4f9e03229b49a30', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:04:10.398275 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-8a971c8c-7bdd-41b6-ab7f-a07acbbfede4 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/fa9945a3-51cc-4366-bc7f-31ddb4f0fa01" status: 200 len: 1362 microversion: 2.1 time: 0.159294 Jul 16 22:04:10.398843 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3309/6611] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:04:10 2022] GET /compute/v2.1/servers/fa9945a3-51cc-4366-bc7f-31ddb4f0fa01 => generated 1362 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:04:11.415878 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-061cf6cb-b96e-4032-b601-1f3138c0e801 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:04:11.416490 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-061cf6cb-b96e-4032-b601-1f3138c0e801 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: fa9945a3-51cc-4366-bc7f-31ddb4f0fa01] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:04:11.423656 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-061cf6cb-b96e-4032-b601-1f3138c0e801 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:04:11.424109 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-061cf6cb-b96e-4032-b601-1f3138c0e801 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:04:11.424864 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-061cf6cb-b96e-4032-b601-1f3138c0e801 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:04:11.425301 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-061cf6cb-b96e-4032-b601-1f3138c0e801 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:04:11.559712 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-061cf6cb-b96e-4032-b601-1f3138c0e801 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '718f6de3b1a1453d9579d421a574691e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd21231d9af34d9aa4f9e03229b49a30', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:04:11.567087 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-061cf6cb-b96e-4032-b601-1f3138c0e801 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/fa9945a3-51cc-4366-bc7f-31ddb4f0fa01" status: 200 len: 1362 microversion: 2.1 time: 0.155141 Jul 16 22:04:11.567779 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3303/6612] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:04:11 2022] GET /compute/v2.1/servers/fa9945a3-51cc-4366-bc7f-31ddb4f0fa01 => generated 1362 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:04:12.584991 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-78bf636b-3dfd-4afc-8537-9ec1f417633a tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:04:12.585601 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-78bf636b-3dfd-4afc-8537-9ec1f417633a tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: fa9945a3-51cc-4366-bc7f-31ddb4f0fa01] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:04:12.592562 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-78bf636b-3dfd-4afc-8537-9ec1f417633a tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:04:12.593028 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-78bf636b-3dfd-4afc-8537-9ec1f417633a tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:04:12.593806 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-78bf636b-3dfd-4afc-8537-9ec1f417633a tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:04:12.594261 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-78bf636b-3dfd-4afc-8537-9ec1f417633a tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:04:12.728943 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-78bf636b-3dfd-4afc-8537-9ec1f417633a tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '718f6de3b1a1453d9579d421a574691e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd21231d9af34d9aa4f9e03229b49a30', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:04:12.736454 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-78bf636b-3dfd-4afc-8537-9ec1f417633a tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/fa9945a3-51cc-4366-bc7f-31ddb4f0fa01" status: 200 len: 1362 microversion: 2.1 time: 0.155732 Jul 16 22:04:12.737018 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3310/6613] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:04:12 2022] GET /compute/v2.1/servers/fa9945a3-51cc-4366-bc7f-31ddb4f0fa01 => generated 1362 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:04:13.755727 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-bd05b45d-b344-4407-8d80-27d0f1ebcc98 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:04:13.756492 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-bd05b45d-b344-4407-8d80-27d0f1ebcc98 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: fa9945a3-51cc-4366-bc7f-31ddb4f0fa01] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:04:13.759565 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell0:SELECT=2 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:04:13.765110 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-bd05b45d-b344-4407-8d80-27d0f1ebcc98 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:04:13.765577 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-bd05b45d-b344-4407-8d80-27d0f1ebcc98 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:04:13.766535 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-bd05b45d-b344-4407-8d80-27d0f1ebcc98 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:04:13.766989 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-bd05b45d-b344-4407-8d80-27d0f1ebcc98 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:04:13.845183 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-1c3025fc-976b-4e4f-a3f4-419934156591 req-5046cc92-2b1b-4177-8a9a-0dfcc91f988f service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "fa9945a3-51cc-4366-bc7f-31ddb4f0fa01", "name": "network-vif-plugged", "status": "completed", "tag": "1ca4d162-bc94-4c13-a163-3f3656a349e8"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:04:13.850390 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell0:SELECT=2 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:04:13.851356 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell0:SELECT=6 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:04:13.857740 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-1c3025fc-976b-4e4f-a3f4-419934156591 req-5046cc92-2b1b-4177-8a9a-0dfcc91f988f service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:04:13.861129 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-1c3025fc-976b-4e4f-a3f4-419934156591 req-5046cc92-2b1b-4177-8a9a-0dfcc91f988f service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:04:13.870726 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-1c3025fc-976b-4e4f-a3f4-419934156591 req-5046cc92-2b1b-4177-8a9a-0dfcc91f988f service nova] Creating event network-vif-plugged:1ca4d162-bc94-4c13-a163-3f3656a349e8 for instance fa9945a3-51cc-4366-bc7f-31ddb4f0fa01 on nch1-master-1 Jul 16 22:04:13.876074 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-1c3025fc-976b-4e4f-a3f4-419934156591 req-5046cc92-2b1b-4177-8a9a-0dfcc91f988f service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.035047 Jul 16 22:04:13.876848 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3311/6614] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 22:04:13 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 16 22:04:13.910328 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-bd05b45d-b344-4407-8d80-27d0f1ebcc98 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '718f6de3b1a1453d9579d421a574691e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd21231d9af34d9aa4f9e03229b49a30', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:04:13.918791 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-bd05b45d-b344-4407-8d80-27d0f1ebcc98 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/fa9945a3-51cc-4366-bc7f-31ddb4f0fa01" status: 200 len: 1362 microversion: 2.1 time: 0.168112 Jul 16 22:04:13.919412 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3304/6615] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:04:13 2022] GET /compute/v2.1/servers/fa9945a3-51cc-4366-bc7f-31ddb4f0fa01 => generated 1362 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:04:14.938220 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-28205f40-24fc-4842-8947-5a2f04ab781c tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:04:14.938870 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-28205f40-24fc-4842-8947-5a2f04ab781c tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: fa9945a3-51cc-4366-bc7f-31ddb4f0fa01] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:04:14.947907 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-28205f40-24fc-4842-8947-5a2f04ab781c tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:04:14.948376 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-28205f40-24fc-4842-8947-5a2f04ab781c tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:04:14.949162 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-28205f40-24fc-4842-8947-5a2f04ab781c tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:04:14.949619 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-28205f40-24fc-4842-8947-5a2f04ab781c tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:04:15.088770 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-28205f40-24fc-4842-8947-5a2f04ab781c tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '718f6de3b1a1453d9579d421a574691e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd21231d9af34d9aa4f9e03229b49a30', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:04:15.096419 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-28205f40-24fc-4842-8947-5a2f04ab781c tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/fa9945a3-51cc-4366-bc7f-31ddb4f0fa01" status: 200 len: 1362 microversion: 2.1 time: 0.162542 Jul 16 22:04:15.096986 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3312/6616] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:04:14 2022] GET /compute/v2.1/servers/fa9945a3-51cc-4366-bc7f-31ddb4f0fa01 => generated 1362 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:04:16.113749 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-4c821838-daa8-48cd-b13c-685d3b8f73a3 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:04:16.114411 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-4c821838-daa8-48cd-b13c-685d3b8f73a3 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: fa9945a3-51cc-4366-bc7f-31ddb4f0fa01] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:04:16.121839 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4c821838-daa8-48cd-b13c-685d3b8f73a3 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:04:16.122237 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4c821838-daa8-48cd-b13c-685d3b8f73a3 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:04:16.123032 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4c821838-daa8-48cd-b13c-685d3b8f73a3 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:04:16.123479 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4c821838-daa8-48cd-b13c-685d3b8f73a3 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:04:16.252762 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-4c821838-daa8-48cd-b13c-685d3b8f73a3 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '718f6de3b1a1453d9579d421a574691e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd21231d9af34d9aa4f9e03229b49a30', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:04:16.261069 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-4c821838-daa8-48cd-b13c-685d3b8f73a3 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/fa9945a3-51cc-4366-bc7f-31ddb4f0fa01" status: 200 len: 1548 microversion: 2.1 time: 0.151207 Jul 16 22:04:16.261649 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3305/6617] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:04:16 2022] GET /compute/v2.1/servers/fa9945a3-51cc-4366-bc7f-31ddb4f0fa01 => generated 1548 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:04:16.278432 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-4c2e4a6f-f6f4-4109-8d45-54a483cb6a00 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:04:16.282855 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-4c2e4a6f-f6f4-4109-8d45-54a483cb6a00 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: fa9945a3-51cc-4366-bc7f-31ddb4f0fa01] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:04:16.286074 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4c2e4a6f-f6f4-4109-8d45-54a483cb6a00 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:04:16.286591 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4c2e4a6f-f6f4-4109-8d45-54a483cb6a00 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:04:16.287408 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4c2e4a6f-f6f4-4109-8d45-54a483cb6a00 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:04:16.287871 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4c2e4a6f-f6f4-4109-8d45-54a483cb6a00 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:04:16.364829 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-4c2e4a6f-f6f4-4109-8d45-54a483cb6a00 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/fa9945a3-51cc-4366-bc7f-31ddb4f0fa01/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.090068 Jul 16 22:04:16.365604 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3313/6618] 10.222.0.16 () {58 vars in 1274 bytes} [Sat Jul 16 22:04:16 2022] GET /compute/v2.1/servers/fa9945a3-51cc-4366-bc7f-31ddb4f0fa01/os-interface => generated 284 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 22:04:17.735832 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-eecbdc4b-2e7f-4ad9-80cf-09bd06b4939d req-6ab01334-bc8c-411c-9ebe-ffe538d00865 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "fa9945a3-51cc-4366-bc7f-31ddb4f0fa01", "tag": "1ca4d162-bc94-4c13-a163-3f3656a349e8"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:04:17.744813 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-eecbdc4b-2e7f-4ad9-80cf-09bd06b4939d req-6ab01334-bc8c-411c-9ebe-ffe538d00865 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:04:17.745266 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-eecbdc4b-2e7f-4ad9-80cf-09bd06b4939d req-6ab01334-bc8c-411c-9ebe-ffe538d00865 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:04:17.756994 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-eecbdc4b-2e7f-4ad9-80cf-09bd06b4939d req-6ab01334-bc8c-411c-9ebe-ffe538d00865 service nova] Creating event network-changed:1ca4d162-bc94-4c13-a163-3f3656a349e8 for instance fa9945a3-51cc-4366-bc7f-31ddb4f0fa01 on nch1-master-1 Jul 16 22:04:17.762017 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-eecbdc4b-2e7f-4ad9-80cf-09bd06b4939d req-6ab01334-bc8c-411c-9ebe-ffe538d00865 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.095480 Jul 16 22:04:17.762744 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3306/6619] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 22:04:17 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 22:04:50.599505 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_api:SELECT=32,nova_api:INSERT=3 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:04:50.600318 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=79,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:04:50.604268 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-3f71f2a3-71c2-447d-8698-9d43dc730786 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Action: 'action', calling method: >, body: {"resize": {"flavorRef": "451"}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:04:50.605250 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-3f71f2a3-71c2-447d-8698-9d43dc730786 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: fa9945a3-51cc-4366-bc7f-31ddb4f0fa01] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:04:50.611312 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3f71f2a3-71c2-447d-8698-9d43dc730786 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:04:50.611804 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3f71f2a3-71c2-447d-8698-9d43dc730786 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:04:50.612563 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3f71f2a3-71c2-447d-8698-9d43dc730786 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:04:50.613017 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3f71f2a3-71c2-447d-8698-9d43dc730786 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:04:50.642919 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-3f71f2a3-71c2-447d-8698-9d43dc730786 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Policy check for compute:servers:resize:cross_cell failed with credentials {'is_admin': False, 'user_id': '718f6de3b1a1453d9579d421a574691e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd21231d9af34d9aa4f9e03229b49a30', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:04:50.648924 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-3f71f2a3-71c2-447d-8698-9d43dc730786 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: fa9945a3-51cc-4366-bc7f-31ddb4f0fa01] Old instance type m1.heat2, new instance type m1.heat {{(pid=138681) resize /opt/stack/nova/nova/compute/api.py:4166}} Jul 16 22:04:50.655229 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-3f71f2a3-71c2-447d-8698-9d43dc730786 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Flavor limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 16 22:04:50.655670 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-3f71f2a3-71c2-447d-8698-9d43dc730786 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Image limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 16 22:04:50.656164 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-3f71f2a3-71c2-447d-8698-9d43dc730786 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Flavor pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 16 22:04:50.656592 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-3f71f2a3-71c2-447d-8698-9d43dc730786 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Image pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 16 22:04:50.657019 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-3f71f2a3-71c2-447d-8698-9d43dc730786 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 16 22:04:50.849973 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-3f71f2a3-71c2-447d-8698-9d43dc730786 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "POST /compute/v2.1/servers/fa9945a3-51cc-4366-bc7f-31ddb4f0fa01/action" status: 202 len: 0 microversion: 2.1 time: 0.252443 Jul 16 22:04:50.850747 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3314/6620] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 22:04:50 2022] POST /compute/v2.1/servers/fa9945a3-51cc-4366-bc7f-31ddb4f0fa01/action => generated 0 bytes in 254 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 22:04:50.859838 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-604bde23-533f-41c3-a649-6bea2257bb7a tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:04:50.860631 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-604bde23-533f-41c3-a649-6bea2257bb7a tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: fa9945a3-51cc-4366-bc7f-31ddb4f0fa01] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:04:50.864638 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_api:SELECT=21 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:04:50.865582 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=58 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:04:50.870955 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-604bde23-533f-41c3-a649-6bea2257bb7a tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:04:50.871432 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-604bde23-533f-41c3-a649-6bea2257bb7a tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:04:50.872210 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-604bde23-533f-41c3-a649-6bea2257bb7a tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:04:50.872650 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-604bde23-533f-41c3-a649-6bea2257bb7a tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:04:51.010586 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-604bde23-533f-41c3-a649-6bea2257bb7a tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '718f6de3b1a1453d9579d421a574691e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd21231d9af34d9aa4f9e03229b49a30', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:04:51.018427 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-604bde23-533f-41c3-a649-6bea2257bb7a tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/fa9945a3-51cc-4366-bc7f-31ddb4f0fa01" status: 200 len: 1676 microversion: 2.1 time: 0.163145 Jul 16 22:04:51.019096 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3307/6621] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:04:50 2022] GET /compute/v2.1/servers/fa9945a3-51cc-4366-bc7f-31ddb4f0fa01 => generated 1676 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:04:52.037635 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-858e6797-8042-4683-bb02-a87a93446f8c tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:04:52.038266 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-858e6797-8042-4683-bb02-a87a93446f8c tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: fa9945a3-51cc-4366-bc7f-31ddb4f0fa01] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:04:52.047188 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-858e6797-8042-4683-bb02-a87a93446f8c tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:04:52.048105 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-858e6797-8042-4683-bb02-a87a93446f8c tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:04:52.048468 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-858e6797-8042-4683-bb02-a87a93446f8c tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:04:52.048919 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-858e6797-8042-4683-bb02-a87a93446f8c tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:04:52.200356 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-858e6797-8042-4683-bb02-a87a93446f8c tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '718f6de3b1a1453d9579d421a574691e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd21231d9af34d9aa4f9e03229b49a30', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:04:52.208396 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-858e6797-8042-4683-bb02-a87a93446f8c tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/fa9945a3-51cc-4366-bc7f-31ddb4f0fa01" status: 200 len: 1676 microversion: 2.1 time: 0.175148 Jul 16 22:04:52.214241 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3315/6622] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:04:52 2022] GET /compute/v2.1/servers/fa9945a3-51cc-4366-bc7f-31ddb4f0fa01 => generated 1676 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:04:53.226872 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-b52e3176-ad6c-4580-acdb-63002406bddc tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:04:53.227632 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-b52e3176-ad6c-4580-acdb-63002406bddc tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: fa9945a3-51cc-4366-bc7f-31ddb4f0fa01] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:04:53.235123 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b52e3176-ad6c-4580-acdb-63002406bddc tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:04:53.235580 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b52e3176-ad6c-4580-acdb-63002406bddc tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:04:53.236341 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b52e3176-ad6c-4580-acdb-63002406bddc tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:04:53.236782 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b52e3176-ad6c-4580-acdb-63002406bddc tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:04:53.370117 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-b52e3176-ad6c-4580-acdb-63002406bddc tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '718f6de3b1a1453d9579d421a574691e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd21231d9af34d9aa4f9e03229b49a30', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:04:53.377721 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-b52e3176-ad6c-4580-acdb-63002406bddc tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/fa9945a3-51cc-4366-bc7f-31ddb4f0fa01" status: 200 len: 1676 microversion: 2.1 time: 0.155881 Jul 16 22:04:53.378295 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3308/6623] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:04:53 2022] GET /compute/v2.1/servers/fa9945a3-51cc-4366-bc7f-31ddb4f0fa01 => generated 1676 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:04:54.395545 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-9f885558-95a2-48e8-8fb3-dd2ad0372851 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:04:54.396175 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-9f885558-95a2-48e8-8fb3-dd2ad0372851 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: fa9945a3-51cc-4366-bc7f-31ddb4f0fa01] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:04:54.403900 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9f885558-95a2-48e8-8fb3-dd2ad0372851 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:04:54.404423 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9f885558-95a2-48e8-8fb3-dd2ad0372851 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:04:54.405200 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9f885558-95a2-48e8-8fb3-dd2ad0372851 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:04:54.405650 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9f885558-95a2-48e8-8fb3-dd2ad0372851 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:04:54.540231 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-9f885558-95a2-48e8-8fb3-dd2ad0372851 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '718f6de3b1a1453d9579d421a574691e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd21231d9af34d9aa4f9e03229b49a30', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:04:54.547847 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-9f885558-95a2-48e8-8fb3-dd2ad0372851 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/fa9945a3-51cc-4366-bc7f-31ddb4f0fa01" status: 200 len: 1681 microversion: 2.1 time: 0.156169 Jul 16 22:04:54.548431 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3316/6624] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:04:54 2022] GET /compute/v2.1/servers/fa9945a3-51cc-4366-bc7f-31ddb4f0fa01 => generated 1681 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:04:55.565424 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-0d521739-8a69-46ad-8225-a0e91ec1b850 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:04:55.566527 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-0d521739-8a69-46ad-8225-a0e91ec1b850 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: fa9945a3-51cc-4366-bc7f-31ddb4f0fa01] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:04:55.573681 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0d521739-8a69-46ad-8225-a0e91ec1b850 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:04:55.574151 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0d521739-8a69-46ad-8225-a0e91ec1b850 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:04:55.574907 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0d521739-8a69-46ad-8225-a0e91ec1b850 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:04:55.575371 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0d521739-8a69-46ad-8225-a0e91ec1b850 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:04:55.712155 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-0d521739-8a69-46ad-8225-a0e91ec1b850 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '718f6de3b1a1453d9579d421a574691e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd21231d9af34d9aa4f9e03229b49a30', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:04:55.719628 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-0d521739-8a69-46ad-8225-a0e91ec1b850 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/fa9945a3-51cc-4366-bc7f-31ddb4f0fa01" status: 200 len: 1681 microversion: 2.1 time: 0.158135 Jul 16 22:04:55.720197 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3309/6625] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:04:55 2022] GET /compute/v2.1/servers/fa9945a3-51cc-4366-bc7f-31ddb4f0fa01 => generated 1681 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:04:56.737350 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-1cb1fda3-43f6-43a6-8903-dda2c3262c0a tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:04:56.738013 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-1cb1fda3-43f6-43a6-8903-dda2c3262c0a tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: fa9945a3-51cc-4366-bc7f-31ddb4f0fa01] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:04:56.745582 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1cb1fda3-43f6-43a6-8903-dda2c3262c0a tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:04:56.746142 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1cb1fda3-43f6-43a6-8903-dda2c3262c0a tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:04:56.746923 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1cb1fda3-43f6-43a6-8903-dda2c3262c0a tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:04:56.747465 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1cb1fda3-43f6-43a6-8903-dda2c3262c0a tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:04:56.878000 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-1cb1fda3-43f6-43a6-8903-dda2c3262c0a tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '718f6de3b1a1453d9579d421a574691e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd21231d9af34d9aa4f9e03229b49a30', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:04:56.886455 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-1cb1fda3-43f6-43a6-8903-dda2c3262c0a tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/fa9945a3-51cc-4366-bc7f-31ddb4f0fa01" status: 200 len: 1681 microversion: 2.1 time: 0.153126 Jul 16 22:04:56.887044 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3317/6626] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:04:56 2022] GET /compute/v2.1/servers/fa9945a3-51cc-4366-bc7f-31ddb4f0fa01 => generated 1681 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:04:57.904049 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-835694f3-4011-40eb-8264-fdbbd775d993 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:04:57.904669 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-835694f3-4011-40eb-8264-fdbbd775d993 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: fa9945a3-51cc-4366-bc7f-31ddb4f0fa01] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:04:57.911736 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-835694f3-4011-40eb-8264-fdbbd775d993 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:04:57.912287 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-835694f3-4011-40eb-8264-fdbbd775d993 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:04:57.913091 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-835694f3-4011-40eb-8264-fdbbd775d993 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:04:57.913538 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-835694f3-4011-40eb-8264-fdbbd775d993 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:04:58.050489 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-835694f3-4011-40eb-8264-fdbbd775d993 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '718f6de3b1a1453d9579d421a574691e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd21231d9af34d9aa4f9e03229b49a30', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:04:58.058199 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-835694f3-4011-40eb-8264-fdbbd775d993 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/fa9945a3-51cc-4366-bc7f-31ddb4f0fa01" status: 200 len: 1681 microversion: 2.1 time: 0.158344 Jul 16 22:04:58.058765 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3310/6627] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:04:57 2022] GET /compute/v2.1/servers/fa9945a3-51cc-4366-bc7f-31ddb4f0fa01 => generated 1681 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:04:59.075214 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-70b9acb3-c4d6-4540-8fef-5c37a7defeab tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:04:59.075871 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-70b9acb3-c4d6-4540-8fef-5c37a7defeab tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: fa9945a3-51cc-4366-bc7f-31ddb4f0fa01] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:04:59.083087 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-70b9acb3-c4d6-4540-8fef-5c37a7defeab tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:04:59.083581 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-70b9acb3-c4d6-4540-8fef-5c37a7defeab tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:04:59.084362 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-70b9acb3-c4d6-4540-8fef-5c37a7defeab tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:04:59.084814 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-70b9acb3-c4d6-4540-8fef-5c37a7defeab tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:04:59.216702 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-70b9acb3-c4d6-4540-8fef-5c37a7defeab tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '718f6de3b1a1453d9579d421a574691e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd21231d9af34d9aa4f9e03229b49a30', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:04:59.224261 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-70b9acb3-c4d6-4540-8fef-5c37a7defeab tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/fa9945a3-51cc-4366-bc7f-31ddb4f0fa01" status: 200 len: 1681 microversion: 2.1 time: 0.153058 Jul 16 22:04:59.224835 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3318/6628] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:04:59 2022] GET /compute/v2.1/servers/fa9945a3-51cc-4366-bc7f-31ddb4f0fa01 => generated 1681 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:05:00.242294 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-fb7f4594-02f3-43d1-ae8e-8fff0808ebfa tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:05:00.242880 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-fb7f4594-02f3-43d1-ae8e-8fff0808ebfa tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: fa9945a3-51cc-4366-bc7f-31ddb4f0fa01] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:05:00.250493 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-fb7f4594-02f3-43d1-ae8e-8fff0808ebfa tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:05:00.250950 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-fb7f4594-02f3-43d1-ae8e-8fff0808ebfa tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:05:00.251814 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-fb7f4594-02f3-43d1-ae8e-8fff0808ebfa tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:05:00.252257 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-fb7f4594-02f3-43d1-ae8e-8fff0808ebfa tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:05:00.385556 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-fb7f4594-02f3-43d1-ae8e-8fff0808ebfa tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '718f6de3b1a1453d9579d421a574691e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd21231d9af34d9aa4f9e03229b49a30', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:05:00.393921 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-fb7f4594-02f3-43d1-ae8e-8fff0808ebfa tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/fa9945a3-51cc-4366-bc7f-31ddb4f0fa01" status: 200 len: 1681 microversion: 2.1 time: 0.155658 Jul 16 22:05:00.394498 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3311/6629] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:05:00 2022] GET /compute/v2.1/servers/fa9945a3-51cc-4366-bc7f-31ddb4f0fa01 => generated 1681 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:05:01.410884 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-ad760f4a-66d3-4a8a-8e81-bdeae39b8a8e tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:05:01.411541 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-ad760f4a-66d3-4a8a-8e81-bdeae39b8a8e tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: fa9945a3-51cc-4366-bc7f-31ddb4f0fa01] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:05:01.419130 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ad760f4a-66d3-4a8a-8e81-bdeae39b8a8e tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:05:01.419685 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ad760f4a-66d3-4a8a-8e81-bdeae39b8a8e tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:05:01.420470 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ad760f4a-66d3-4a8a-8e81-bdeae39b8a8e tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:05:01.420924 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ad760f4a-66d3-4a8a-8e81-bdeae39b8a8e tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:05:01.552658 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-ad760f4a-66d3-4a8a-8e81-bdeae39b8a8e tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '718f6de3b1a1453d9579d421a574691e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd21231d9af34d9aa4f9e03229b49a30', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:05:01.560766 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-ad760f4a-66d3-4a8a-8e81-bdeae39b8a8e tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/fa9945a3-51cc-4366-bc7f-31ddb4f0fa01" status: 200 len: 1681 microversion: 2.1 time: 0.153820 Jul 16 22:05:01.561334 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3319/6630] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:05:01 2022] GET /compute/v2.1/servers/fa9945a3-51cc-4366-bc7f-31ddb4f0fa01 => generated 1681 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:05:02.578485 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-d23ed3d3-1ece-4905-ac91-5f3a1f70f876 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:05:02.579147 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-d23ed3d3-1ece-4905-ac91-5f3a1f70f876 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: fa9945a3-51cc-4366-bc7f-31ddb4f0fa01] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:05:02.586453 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d23ed3d3-1ece-4905-ac91-5f3a1f70f876 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:05:02.586910 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d23ed3d3-1ece-4905-ac91-5f3a1f70f876 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:05:02.587717 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d23ed3d3-1ece-4905-ac91-5f3a1f70f876 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:05:02.588170 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d23ed3d3-1ece-4905-ac91-5f3a1f70f876 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:05:02.720144 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-d23ed3d3-1ece-4905-ac91-5f3a1f70f876 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '718f6de3b1a1453d9579d421a574691e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd21231d9af34d9aa4f9e03229b49a30', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:05:02.727741 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-d23ed3d3-1ece-4905-ac91-5f3a1f70f876 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/fa9945a3-51cc-4366-bc7f-31ddb4f0fa01" status: 200 len: 1681 microversion: 2.1 time: 0.153821 Jul 16 22:05:02.728311 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3312/6631] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:05:02 2022] GET /compute/v2.1/servers/fa9945a3-51cc-4366-bc7f-31ddb4f0fa01 => generated 1681 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:05:03.745659 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-0ce3fade-5293-4e79-bb13-961370991fae tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:05:03.746189 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-0ce3fade-5293-4e79-bb13-961370991fae tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: fa9945a3-51cc-4366-bc7f-31ddb4f0fa01] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:05:03.752771 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0ce3fade-5293-4e79-bb13-961370991fae tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:05:03.753237 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0ce3fade-5293-4e79-bb13-961370991fae tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:05:03.754019 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0ce3fade-5293-4e79-bb13-961370991fae tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:05:03.754469 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0ce3fade-5293-4e79-bb13-961370991fae tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:05:03.892516 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-0ce3fade-5293-4e79-bb13-961370991fae tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '718f6de3b1a1453d9579d421a574691e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd21231d9af34d9aa4f9e03229b49a30', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:05:03.900245 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-0ce3fade-5293-4e79-bb13-961370991fae tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/fa9945a3-51cc-4366-bc7f-31ddb4f0fa01" status: 200 len: 1681 microversion: 2.1 time: 0.158858 Jul 16 22:05:03.902368 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3320/6632] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:05:03 2022] GET /compute/v2.1/servers/fa9945a3-51cc-4366-bc7f-31ddb4f0fa01 => generated 1681 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:05:04.917992 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-8e266392-fd67-4e0f-a8a6-2814e4bb46e7 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:05:04.918613 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-8e266392-fd67-4e0f-a8a6-2814e4bb46e7 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: fa9945a3-51cc-4366-bc7f-31ddb4f0fa01] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:05:04.925420 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8e266392-fd67-4e0f-a8a6-2814e4bb46e7 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:05:04.925907 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8e266392-fd67-4e0f-a8a6-2814e4bb46e7 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:05:04.926669 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8e266392-fd67-4e0f-a8a6-2814e4bb46e7 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:05:04.927146 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8e266392-fd67-4e0f-a8a6-2814e4bb46e7 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:05:05.058751 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-8e266392-fd67-4e0f-a8a6-2814e4bb46e7 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '718f6de3b1a1453d9579d421a574691e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd21231d9af34d9aa4f9e03229b49a30', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:05:05.066379 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-8e266392-fd67-4e0f-a8a6-2814e4bb46e7 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/fa9945a3-51cc-4366-bc7f-31ddb4f0fa01" status: 200 len: 1681 microversion: 2.1 time: 0.152591 Jul 16 22:05:05.066946 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3313/6633] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:05:04 2022] GET /compute/v2.1/servers/fa9945a3-51cc-4366-bc7f-31ddb4f0fa01 => generated 1681 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:05:06.084490 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-5feb919f-04d9-49c8-bf36-8c3a5cd58fda tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:05:06.085143 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-5feb919f-04d9-49c8-bf36-8c3a5cd58fda tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: fa9945a3-51cc-4366-bc7f-31ddb4f0fa01] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:05:06.092879 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5feb919f-04d9-49c8-bf36-8c3a5cd58fda tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:05:06.093540 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5feb919f-04d9-49c8-bf36-8c3a5cd58fda tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:05:06.094222 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5feb919f-04d9-49c8-bf36-8c3a5cd58fda tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:05:06.094667 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5feb919f-04d9-49c8-bf36-8c3a5cd58fda tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:05:06.230551 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-5feb919f-04d9-49c8-bf36-8c3a5cd58fda tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '718f6de3b1a1453d9579d421a574691e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd21231d9af34d9aa4f9e03229b49a30', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:05:06.238765 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-5feb919f-04d9-49c8-bf36-8c3a5cd58fda tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/fa9945a3-51cc-4366-bc7f-31ddb4f0fa01" status: 200 len: 1681 microversion: 2.1 time: 0.158539 Jul 16 22:05:06.239366 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3321/6634] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:05:06 2022] GET /compute/v2.1/servers/fa9945a3-51cc-4366-bc7f-31ddb4f0fa01 => generated 1681 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:05:07.256827 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-0ff031d6-b0f2-40d6-9069-12919035b7ab tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:05:07.257454 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-0ff031d6-b0f2-40d6-9069-12919035b7ab tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: fa9945a3-51cc-4366-bc7f-31ddb4f0fa01] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:05:07.265029 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0ff031d6-b0f2-40d6-9069-12919035b7ab tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:05:07.265498 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0ff031d6-b0f2-40d6-9069-12919035b7ab tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:05:07.266273 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0ff031d6-b0f2-40d6-9069-12919035b7ab tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:05:07.266723 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0ff031d6-b0f2-40d6-9069-12919035b7ab tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:05:07.398665 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-0ff031d6-b0f2-40d6-9069-12919035b7ab tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '718f6de3b1a1453d9579d421a574691e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd21231d9af34d9aa4f9e03229b49a30', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:05:07.406516 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-0ff031d6-b0f2-40d6-9069-12919035b7ab tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/fa9945a3-51cc-4366-bc7f-31ddb4f0fa01" status: 200 len: 1681 microversion: 2.1 time: 0.153578 Jul 16 22:05:07.407110 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3314/6635] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:05:07 2022] GET /compute/v2.1/servers/fa9945a3-51cc-4366-bc7f-31ddb4f0fa01 => generated 1681 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:05:08.424085 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-62d0d6f0-a37d-44ff-a71d-aadd3c596f84 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:05:08.424657 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-62d0d6f0-a37d-44ff-a71d-aadd3c596f84 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: fa9945a3-51cc-4366-bc7f-31ddb4f0fa01] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:05:08.431679 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-62d0d6f0-a37d-44ff-a71d-aadd3c596f84 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:05:08.432139 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-62d0d6f0-a37d-44ff-a71d-aadd3c596f84 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:05:08.432942 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-62d0d6f0-a37d-44ff-a71d-aadd3c596f84 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:05:08.433392 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-62d0d6f0-a37d-44ff-a71d-aadd3c596f84 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:05:08.568735 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-62d0d6f0-a37d-44ff-a71d-aadd3c596f84 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '718f6de3b1a1453d9579d421a574691e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd21231d9af34d9aa4f9e03229b49a30', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:05:08.576243 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-62d0d6f0-a37d-44ff-a71d-aadd3c596f84 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/fa9945a3-51cc-4366-bc7f-31ddb4f0fa01" status: 200 len: 1681 microversion: 2.1 time: 0.156455 Jul 16 22:05:08.576820 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3322/6636] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:05:08 2022] GET /compute/v2.1/servers/fa9945a3-51cc-4366-bc7f-31ddb4f0fa01 => generated 1681 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 22:05:09.593974 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-549c04a4-8c2d-4de2-9c71-76bdc74495ba tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:05:09.594603 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-549c04a4-8c2d-4de2-9c71-76bdc74495ba tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: fa9945a3-51cc-4366-bc7f-31ddb4f0fa01] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:05:09.601641 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-549c04a4-8c2d-4de2-9c71-76bdc74495ba tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:05:09.602107 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-549c04a4-8c2d-4de2-9c71-76bdc74495ba tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:05:09.602878 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-549c04a4-8c2d-4de2-9c71-76bdc74495ba tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:05:09.603353 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-549c04a4-8c2d-4de2-9c71-76bdc74495ba tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:05:09.736415 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-549c04a4-8c2d-4de2-9c71-76bdc74495ba tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '718f6de3b1a1453d9579d421a574691e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd21231d9af34d9aa4f9e03229b49a30', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:05:09.744007 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-549c04a4-8c2d-4de2-9c71-76bdc74495ba tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/fa9945a3-51cc-4366-bc7f-31ddb4f0fa01" status: 200 len: 1681 microversion: 2.1 time: 0.153994 Jul 16 22:05:09.744658 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3315/6637] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:05:09 2022] GET /compute/v2.1/servers/fa9945a3-51cc-4366-bc7f-31ddb4f0fa01 => generated 1681 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:05:10.761784 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-080492c8-9a3f-4d0f-82ea-17fed5e8b761 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:05:10.762416 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-080492c8-9a3f-4d0f-82ea-17fed5e8b761 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: fa9945a3-51cc-4366-bc7f-31ddb4f0fa01] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:05:10.769788 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-080492c8-9a3f-4d0f-82ea-17fed5e8b761 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:05:10.772998 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-080492c8-9a3f-4d0f-82ea-17fed5e8b761 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:05:10.772998 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-080492c8-9a3f-4d0f-82ea-17fed5e8b761 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:05:10.772998 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-080492c8-9a3f-4d0f-82ea-17fed5e8b761 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:05:10.903519 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-080492c8-9a3f-4d0f-82ea-17fed5e8b761 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '718f6de3b1a1453d9579d421a574691e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd21231d9af34d9aa4f9e03229b49a30', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:05:10.911414 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-080492c8-9a3f-4d0f-82ea-17fed5e8b761 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/fa9945a3-51cc-4366-bc7f-31ddb4f0fa01" status: 200 len: 1681 microversion: 2.1 time: 0.153527 Jul 16 22:05:10.912198 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3323/6638] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:05:10 2022] GET /compute/v2.1/servers/fa9945a3-51cc-4366-bc7f-31ddb4f0fa01 => generated 1681 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:05:11.701555 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-17b0d355-5dda-4941-901f-fceb51867cbe req-f45d475f-92a3-49a6-a495-0f45922ddf11 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "fa9945a3-51cc-4366-bc7f-31ddb4f0fa01", "name": "network-vif-unplugged", "status": "completed", "tag": "1ca4d162-bc94-4c13-a163-3f3656a349e8"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:05:11.712047 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-17b0d355-5dda-4941-901f-fceb51867cbe req-f45d475f-92a3-49a6-a495-0f45922ddf11 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:05:11.712503 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-17b0d355-5dda-4941-901f-fceb51867cbe req-f45d475f-92a3-49a6-a495-0f45922ddf11 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:05:11.727372 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-17b0d355-5dda-4941-901f-fceb51867cbe req-f45d475f-92a3-49a6-a495-0f45922ddf11 service nova] Creating event network-vif-unplugged:1ca4d162-bc94-4c13-a163-3f3656a349e8 for instance fa9945a3-51cc-4366-bc7f-31ddb4f0fa01 on nch2-master-1 Jul 16 22:05:11.732763 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [req-17b0d355-5dda-4941-901f-fceb51867cbe req-f45d475f-92a3-49a6-a495-0f45922ddf11 service nova] Instance fa9945a3-51cc-4366-bc7f-31ddb4f0fa01 is migrating within the same cell, copying events to all relevant hosts: {'nch2-master-1', 'nch1-master-1'} {{(pid=138680) _get_relevant_hosts /opt/stack/nova/nova/compute/api.py:5807}} Jul 16 22:05:11.743638 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-17b0d355-5dda-4941-901f-fceb51867cbe req-f45d475f-92a3-49a6-a495-0f45922ddf11 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.045821 Jul 16 22:05:11.744314 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3316/6639] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 22:05:11 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 22:05:11.929089 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-1e403ba3-04d5-4b91-b83c-5eeaf0963fd9 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:05:11.929748 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-1e403ba3-04d5-4b91-b83c-5eeaf0963fd9 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: fa9945a3-51cc-4366-bc7f-31ddb4f0fa01] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:05:11.939424 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1e403ba3-04d5-4b91-b83c-5eeaf0963fd9 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:05:11.939895 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1e403ba3-04d5-4b91-b83c-5eeaf0963fd9 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:05:11.940742 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1e403ba3-04d5-4b91-b83c-5eeaf0963fd9 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:05:11.941411 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1e403ba3-04d5-4b91-b83c-5eeaf0963fd9 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:05:12.094181 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-1e403ba3-04d5-4b91-b83c-5eeaf0963fd9 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '718f6de3b1a1453d9579d421a574691e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd21231d9af34d9aa4f9e03229b49a30', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:05:12.102040 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-1e403ba3-04d5-4b91-b83c-5eeaf0963fd9 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/fa9945a3-51cc-4366-bc7f-31ddb4f0fa01" status: 200 len: 1680 microversion: 2.1 time: 0.177053 Jul 16 22:05:12.102824 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3324/6640] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:05:11 2022] GET /compute/v2.1/servers/fa9945a3-51cc-4366-bc7f-31ddb4f0fa01 => generated 1680 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:05:13.122006 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-581b9c83-6609-49df-b8ce-bfffe19cf232 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:05:13.122721 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-581b9c83-6609-49df-b8ce-bfffe19cf232 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: fa9945a3-51cc-4366-bc7f-31ddb4f0fa01] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:05:13.129938 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-581b9c83-6609-49df-b8ce-bfffe19cf232 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:05:13.130424 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-581b9c83-6609-49df-b8ce-bfffe19cf232 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:05:13.131225 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-581b9c83-6609-49df-b8ce-bfffe19cf232 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:05:13.131676 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-581b9c83-6609-49df-b8ce-bfffe19cf232 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:05:13.268102 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-581b9c83-6609-49df-b8ce-bfffe19cf232 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '718f6de3b1a1453d9579d421a574691e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd21231d9af34d9aa4f9e03229b49a30', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:05:13.275898 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-581b9c83-6609-49df-b8ce-bfffe19cf232 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/fa9945a3-51cc-4366-bc7f-31ddb4f0fa01" status: 200 len: 1680 microversion: 2.1 time: 0.159266 Jul 16 22:05:13.276482 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3317/6641] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:05:13 2022] GET /compute/v2.1/servers/fa9945a3-51cc-4366-bc7f-31ddb4f0fa01 => generated 1680 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:05:13.758420 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-ae264463-5027-4289-9b50-999775116f1a req-26eb04ba-39ff-45c4-a31b-f48eb610ee11 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "fa9945a3-51cc-4366-bc7f-31ddb4f0fa01", "tag": "1ca4d162-bc94-4c13-a163-3f3656a349e8"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:05:13.768045 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-ae264463-5027-4289-9b50-999775116f1a req-26eb04ba-39ff-45c4-a31b-f48eb610ee11 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:05:13.768511 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-ae264463-5027-4289-9b50-999775116f1a req-26eb04ba-39ff-45c4-a31b-f48eb610ee11 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:05:13.780034 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-ae264463-5027-4289-9b50-999775116f1a req-26eb04ba-39ff-45c4-a31b-f48eb610ee11 service nova] Creating event network-changed:1ca4d162-bc94-4c13-a163-3f3656a349e8 for instance fa9945a3-51cc-4366-bc7f-31ddb4f0fa01 on nch2-master-1 Jul 16 22:05:13.786136 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [req-ae264463-5027-4289-9b50-999775116f1a req-26eb04ba-39ff-45c4-a31b-f48eb610ee11 service nova] Instance fa9945a3-51cc-4366-bc7f-31ddb4f0fa01 is migrating within the same cell, copying events to all relevant hosts: {'nch2-master-1', 'nch1-master-1'} {{(pid=138681) _get_relevant_hosts /opt/stack/nova/nova/compute/api.py:5807}} Jul 16 22:05:13.797573 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-ae264463-5027-4289-9b50-999775116f1a req-26eb04ba-39ff-45c4-a31b-f48eb610ee11 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.042820 Jul 16 22:05:13.799402 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3325/6642] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 22:05:13 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 22:05:14.293794 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-756ed5fb-3775-47e0-8d8e-1816b1842bd8 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:05:14.294447 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-756ed5fb-3775-47e0-8d8e-1816b1842bd8 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: fa9945a3-51cc-4366-bc7f-31ddb4f0fa01] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:05:14.301672 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-756ed5fb-3775-47e0-8d8e-1816b1842bd8 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:05:14.302137 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-756ed5fb-3775-47e0-8d8e-1816b1842bd8 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:05:14.302919 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-756ed5fb-3775-47e0-8d8e-1816b1842bd8 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:05:14.303401 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-756ed5fb-3775-47e0-8d8e-1816b1842bd8 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:05:14.435679 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-756ed5fb-3775-47e0-8d8e-1816b1842bd8 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '718f6de3b1a1453d9579d421a574691e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd21231d9af34d9aa4f9e03229b49a30', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:05:14.443968 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-756ed5fb-3775-47e0-8d8e-1816b1842bd8 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/fa9945a3-51cc-4366-bc7f-31ddb4f0fa01" status: 200 len: 1678 microversion: 2.1 time: 0.154217 Jul 16 22:05:14.444545 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3318/6643] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:05:14 2022] GET /compute/v2.1/servers/fa9945a3-51cc-4366-bc7f-31ddb4f0fa01 => generated 1678 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:05:15.462533 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-6c7f6acd-7d5d-49a9-98f5-8073f1096d38 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:05:15.463329 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-6c7f6acd-7d5d-49a9-98f5-8073f1096d38 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: fa9945a3-51cc-4366-bc7f-31ddb4f0fa01] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:05:15.470357 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6c7f6acd-7d5d-49a9-98f5-8073f1096d38 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:05:15.470823 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6c7f6acd-7d5d-49a9-98f5-8073f1096d38 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:05:15.471754 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6c7f6acd-7d5d-49a9-98f5-8073f1096d38 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:05:15.472212 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6c7f6acd-7d5d-49a9-98f5-8073f1096d38 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:05:15.610067 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-6c7f6acd-7d5d-49a9-98f5-8073f1096d38 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '718f6de3b1a1453d9579d421a574691e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd21231d9af34d9aa4f9e03229b49a30', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:05:15.617789 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-6c7f6acd-7d5d-49a9-98f5-8073f1096d38 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/fa9945a3-51cc-4366-bc7f-31ddb4f0fa01" status: 200 len: 1675 microversion: 2.1 time: 0.160802 Jul 16 22:05:15.618354 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3326/6644] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:05:15 2022] GET /compute/v2.1/servers/fa9945a3-51cc-4366-bc7f-31ddb4f0fa01 => generated 1675 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:05:15.634957 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-f8697f3b-1146-4d1c-94c1-eb5e9b996ae8 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Action: 'action', calling method: >, body: {"revertResize": {}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:05:15.635532 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-f8697f3b-1146-4d1c-94c1-eb5e9b996ae8 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: fa9945a3-51cc-4366-bc7f-31ddb4f0fa01] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:05:15.643200 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f8697f3b-1146-4d1c-94c1-eb5e9b996ae8 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:05:15.643661 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f8697f3b-1146-4d1c-94c1-eb5e9b996ae8 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:05:15.644428 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f8697f3b-1146-4d1c-94c1-eb5e9b996ae8 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:05:15.644871 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f8697f3b-1146-4d1c-94c1-eb5e9b996ae8 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:05:15.680849 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f8697f3b-1146-4d1c-94c1-eb5e9b996ae8 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:05:15.681242 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f8697f3b-1146-4d1c-94c1-eb5e9b996ae8 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:05:15.682035 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f8697f3b-1146-4d1c-94c1-eb5e9b996ae8 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:05:15.682480 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f8697f3b-1146-4d1c-94c1-eb5e9b996ae8 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:05:15.704571 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-f8697f3b-1146-4d1c-94c1-eb5e9b996ae8 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Getting quotas for project fd21231d9af34d9aa4f9e03229b49a30. Resources: {'ram', 'cores'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 22:05:15.709849 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-f8697f3b-1146-4d1c-94c1-eb5e9b996ae8 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Getting quotas for user 718f6de3b1a1453d9579d421a574691e and project fd21231d9af34d9aa4f9e03229b49a30. Resources: {'ram', 'cores'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 22:05:16.073181 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-f8697f3b-1146-4d1c-94c1-eb5e9b996ae8 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "POST /compute/v2.1/servers/fa9945a3-51cc-4366-bc7f-31ddb4f0fa01/action" status: 202 len: 0 microversion: 2.1 time: 0.441797 Jul 16 22:05:16.076897 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3319/6645] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 22:05:15 2022] POST /compute/v2.1/servers/fa9945a3-51cc-4366-bc7f-31ddb4f0fa01/action => generated 0 bytes in 443 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 22:05:16.083988 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-2f330be9-f2a8-4f44-bc59-d587715b4ece tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:05:16.084979 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-2f330be9-f2a8-4f44-bc59-d587715b4ece tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: fa9945a3-51cc-4366-bc7f-31ddb4f0fa01] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:05:16.091610 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2f330be9-f2a8-4f44-bc59-d587715b4ece tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:05:16.095199 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2f330be9-f2a8-4f44-bc59-d587715b4ece tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:05:16.095199 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2f330be9-f2a8-4f44-bc59-d587715b4ece tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:05:16.095199 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2f330be9-f2a8-4f44-bc59-d587715b4ece tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:05:16.229788 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-2f330be9-f2a8-4f44-bc59-d587715b4ece tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '718f6de3b1a1453d9579d421a574691e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd21231d9af34d9aa4f9e03229b49a30', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:05:16.238338 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-2f330be9-f2a8-4f44-bc59-d587715b4ece tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/fa9945a3-51cc-4366-bc7f-31ddb4f0fa01" status: 200 len: 1674 microversion: 2.1 time: 0.160035 Jul 16 22:05:16.239256 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3327/6646] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:05:16 2022] GET /compute/v2.1/servers/fa9945a3-51cc-4366-bc7f-31ddb4f0fa01 => generated 1674 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:05:17.257196 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-04835b2d-f59f-4efe-9ec2-69275ca49b87 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:05:17.257901 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-04835b2d-f59f-4efe-9ec2-69275ca49b87 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: fa9945a3-51cc-4366-bc7f-31ddb4f0fa01] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:05:17.265945 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-04835b2d-f59f-4efe-9ec2-69275ca49b87 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:05:17.266406 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-04835b2d-f59f-4efe-9ec2-69275ca49b87 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:05:17.267218 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-04835b2d-f59f-4efe-9ec2-69275ca49b87 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:05:17.267665 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-04835b2d-f59f-4efe-9ec2-69275ca49b87 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:05:17.404035 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-04835b2d-f59f-4efe-9ec2-69275ca49b87 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '718f6de3b1a1453d9579d421a574691e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd21231d9af34d9aa4f9e03229b49a30', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:05:17.407797 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:05:17.412394 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-04835b2d-f59f-4efe-9ec2-69275ca49b87 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/fa9945a3-51cc-4366-bc7f-31ddb4f0fa01" status: 200 len: 1674 microversion: 2.1 time: 0.160381 Jul 16 22:05:17.417443 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3320/6647] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:05:17 2022] GET /compute/v2.1/servers/fa9945a3-51cc-4366-bc7f-31ddb4f0fa01 => generated 1674 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:05:18.181424 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-fe977f66-7df1-409e-9bb1-7e79d42ada73 req-5f6afd59-99b0-4eed-b7bd-4a85a9bd8e99 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "fa9945a3-51cc-4366-bc7f-31ddb4f0fa01", "name": "network-vif-plugged", "status": "completed", "tag": "1ca4d162-bc94-4c13-a163-3f3656a349e8"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:05:18.191235 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-fe977f66-7df1-409e-9bb1-7e79d42ada73 req-5f6afd59-99b0-4eed-b7bd-4a85a9bd8e99 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:05:18.191770 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-fe977f66-7df1-409e-9bb1-7e79d42ada73 req-5f6afd59-99b0-4eed-b7bd-4a85a9bd8e99 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:05:18.203735 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-fe977f66-7df1-409e-9bb1-7e79d42ada73 req-5f6afd59-99b0-4eed-b7bd-4a85a9bd8e99 service nova] Creating event network-vif-plugged:1ca4d162-bc94-4c13-a163-3f3656a349e8 for instance fa9945a3-51cc-4366-bc7f-31ddb4f0fa01 on nch2-master-1 Jul 16 22:05:18.209992 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [req-fe977f66-7df1-409e-9bb1-7e79d42ada73 req-5f6afd59-99b0-4eed-b7bd-4a85a9bd8e99 service nova] Instance fa9945a3-51cc-4366-bc7f-31ddb4f0fa01 is migrating within the same cell, copying events to all relevant hosts: {'nch2-master-1', 'nch1-master-1'} {{(pid=138681) _get_relevant_hosts /opt/stack/nova/nova/compute/api.py:5807}} Jul 16 22:05:18.221936 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-fe977f66-7df1-409e-9bb1-7e79d42ada73 req-5f6afd59-99b0-4eed-b7bd-4a85a9bd8e99 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.043480 Jul 16 22:05:18.222556 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3328/6648] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 22:05:18 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 22:05:18.430787 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-77094515-e510-4f14-9f6e-d6354ee51c44 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:05:18.431364 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-77094515-e510-4f14-9f6e-d6354ee51c44 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: fa9945a3-51cc-4366-bc7f-31ddb4f0fa01] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:05:18.439092 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-77094515-e510-4f14-9f6e-d6354ee51c44 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:05:18.440252 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-77094515-e510-4f14-9f6e-d6354ee51c44 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:05:18.441034 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-77094515-e510-4f14-9f6e-d6354ee51c44 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:05:18.441481 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-77094515-e510-4f14-9f6e-d6354ee51c44 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:05:18.576953 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-77094515-e510-4f14-9f6e-d6354ee51c44 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '718f6de3b1a1453d9579d421a574691e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd21231d9af34d9aa4f9e03229b49a30', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:05:18.585920 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-77094515-e510-4f14-9f6e-d6354ee51c44 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/fa9945a3-51cc-4366-bc7f-31ddb4f0fa01" status: 200 len: 1674 microversion: 2.1 time: 0.159165 Jul 16 22:05:18.586497 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3321/6649] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:05:18 2022] GET /compute/v2.1/servers/fa9945a3-51cc-4366-bc7f-31ddb4f0fa01 => generated 1674 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:05:19.603737 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-a87dbc5c-b5a2-453b-823b-4373e4ad7fe3 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:05:19.604481 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-a87dbc5c-b5a2-453b-823b-4373e4ad7fe3 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: fa9945a3-51cc-4366-bc7f-31ddb4f0fa01] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:05:19.612185 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a87dbc5c-b5a2-453b-823b-4373e4ad7fe3 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:05:19.612659 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a87dbc5c-b5a2-453b-823b-4373e4ad7fe3 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:05:19.613442 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a87dbc5c-b5a2-453b-823b-4373e4ad7fe3 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:05:19.613886 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a87dbc5c-b5a2-453b-823b-4373e4ad7fe3 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:05:19.756961 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-a87dbc5c-b5a2-453b-823b-4373e4ad7fe3 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '718f6de3b1a1453d9579d421a574691e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd21231d9af34d9aa4f9e03229b49a30', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:05:19.758191 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:05:19.762672 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-a87dbc5c-b5a2-453b-823b-4373e4ad7fe3 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/fa9945a3-51cc-4366-bc7f-31ddb4f0fa01" status: 200 len: 1674 microversion: 2.1 time: 0.163060 Jul 16 22:05:19.763287 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3329/6650] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:05:19 2022] GET /compute/v2.1/servers/fa9945a3-51cc-4366-bc7f-31ddb4f0fa01 => generated 1674 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:05:20.108548 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-1823f988-0ec6-45d8-b4fd-b65f1225e9e1 req-22781cdc-58fd-46c8-8cf5-3ae69488c978 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "fa9945a3-51cc-4366-bc7f-31ddb4f0fa01", "name": "network-vif-unplugged", "status": "completed", "tag": "1ca4d162-bc94-4c13-a163-3f3656a349e8"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:05:20.118405 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-1823f988-0ec6-45d8-b4fd-b65f1225e9e1 req-22781cdc-58fd-46c8-8cf5-3ae69488c978 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:05:20.118862 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-1823f988-0ec6-45d8-b4fd-b65f1225e9e1 req-22781cdc-58fd-46c8-8cf5-3ae69488c978 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:05:20.134945 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-1823f988-0ec6-45d8-b4fd-b65f1225e9e1 req-22781cdc-58fd-46c8-8cf5-3ae69488c978 service nova] Creating event network-vif-unplugged:1ca4d162-bc94-4c13-a163-3f3656a349e8 for instance fa9945a3-51cc-4366-bc7f-31ddb4f0fa01 on nch1-master-1 Jul 16 22:05:20.141586 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [req-1823f988-0ec6-45d8-b4fd-b65f1225e9e1 req-22781cdc-58fd-46c8-8cf5-3ae69488c978 service nova] Instance fa9945a3-51cc-4366-bc7f-31ddb4f0fa01 is migrating within the same cell, copying events to all relevant hosts: {'nch2-master-1', 'nch1-master-1'} {{(pid=138680) _get_relevant_hosts /opt/stack/nova/nova/compute/api.py:5807}} Jul 16 22:05:20.156777 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-1823f988-0ec6-45d8-b4fd-b65f1225e9e1 req-22781cdc-58fd-46c8-8cf5-3ae69488c978 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.053521 Jul 16 22:05:20.157536 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3322/6651] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 22:05:20 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 16 22:05:20.781200 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-75a36389-6af7-46b9-b0ae-4a49c5055561 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:05:20.782071 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-75a36389-6af7-46b9-b0ae-4a49c5055561 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: fa9945a3-51cc-4366-bc7f-31ddb4f0fa01] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:05:20.786168 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_api:SELECT=37 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:05:20.791703 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-75a36389-6af7-46b9-b0ae-4a49c5055561 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:05:20.792324 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-75a36389-6af7-46b9-b0ae-4a49c5055561 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:05:20.793315 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-75a36389-6af7-46b9-b0ae-4a49c5055561 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:05:20.793988 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-75a36389-6af7-46b9-b0ae-4a49c5055561 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:05:20.945101 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-75a36389-6af7-46b9-b0ae-4a49c5055561 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '718f6de3b1a1453d9579d421a574691e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd21231d9af34d9aa4f9e03229b49a30', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:05:20.952893 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-75a36389-6af7-46b9-b0ae-4a49c5055561 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/fa9945a3-51cc-4366-bc7f-31ddb4f0fa01" status: 200 len: 1674 microversion: 2.1 time: 0.175551 Jul 16 22:05:20.953414 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3330/6652] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:05:20 2022] GET /compute/v2.1/servers/fa9945a3-51cc-4366-bc7f-31ddb4f0fa01 => generated 1674 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:05:21.972108 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-92f41b71-9c92-4dce-81ab-368309815345 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:05:21.975148 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-92f41b71-9c92-4dce-81ab-368309815345 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: fa9945a3-51cc-4366-bc7f-31ddb4f0fa01] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:05:21.980783 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-92f41b71-9c92-4dce-81ab-368309815345 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:05:21.981218 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-92f41b71-9c92-4dce-81ab-368309815345 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:05:21.982000 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-92f41b71-9c92-4dce-81ab-368309815345 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:05:21.982443 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-92f41b71-9c92-4dce-81ab-368309815345 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:05:22.130046 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-92f41b71-9c92-4dce-81ab-368309815345 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '718f6de3b1a1453d9579d421a574691e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd21231d9af34d9aa4f9e03229b49a30', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:05:22.138144 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-92f41b71-9c92-4dce-81ab-368309815345 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/fa9945a3-51cc-4366-bc7f-31ddb4f0fa01" status: 200 len: 1674 microversion: 2.1 time: 0.171768 Jul 16 22:05:22.138727 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3323/6653] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:05:21 2022] GET /compute/v2.1/servers/fa9945a3-51cc-4366-bc7f-31ddb4f0fa01 => generated 1674 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:05:22.175931 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-67ae4a4b-4751-4e38-ba79-d1847d3758d4 req-5a120ee8-4d8f-402a-b5e5-690a8fef84ad service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "fa9945a3-51cc-4366-bc7f-31ddb4f0fa01", "tag": "1ca4d162-bc94-4c13-a163-3f3656a349e8"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:05:22.184953 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-67ae4a4b-4751-4e38-ba79-d1847d3758d4 req-5a120ee8-4d8f-402a-b5e5-690a8fef84ad service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:05:22.185420 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-67ae4a4b-4751-4e38-ba79-d1847d3758d4 req-5a120ee8-4d8f-402a-b5e5-690a8fef84ad service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:05:22.197645 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-67ae4a4b-4751-4e38-ba79-d1847d3758d4 req-5a120ee8-4d8f-402a-b5e5-690a8fef84ad service nova] Creating event network-changed:1ca4d162-bc94-4c13-a163-3f3656a349e8 for instance fa9945a3-51cc-4366-bc7f-31ddb4f0fa01 on nch1-master-1 Jul 16 22:05:22.202991 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [req-67ae4a4b-4751-4e38-ba79-d1847d3758d4 req-5a120ee8-4d8f-402a-b5e5-690a8fef84ad service nova] Instance fa9945a3-51cc-4366-bc7f-31ddb4f0fa01 is migrating within the same cell, copying events to all relevant hosts: {'nch2-master-1', 'nch1-master-1'} {{(pid=138681) _get_relevant_hosts /opt/stack/nova/nova/compute/api.py:5807}} Jul 16 22:05:22.214520 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-67ae4a4b-4751-4e38-ba79-d1847d3758d4 req-5a120ee8-4d8f-402a-b5e5-690a8fef84ad service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.042675 Jul 16 22:05:22.215150 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3331/6654] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 22:05:22 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 16 22:05:23.156303 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-7291bd2f-e1c1-4b6d-8a02-03c3a941e261 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:05:23.156981 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-7291bd2f-e1c1-4b6d-8a02-03c3a941e261 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: fa9945a3-51cc-4366-bc7f-31ddb4f0fa01] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:05:23.164878 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7291bd2f-e1c1-4b6d-8a02-03c3a941e261 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:05:23.165424 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7291bd2f-e1c1-4b6d-8a02-03c3a941e261 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:05:23.166201 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7291bd2f-e1c1-4b6d-8a02-03c3a941e261 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:05:23.166658 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7291bd2f-e1c1-4b6d-8a02-03c3a941e261 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:05:23.303716 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-7291bd2f-e1c1-4b6d-8a02-03c3a941e261 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '718f6de3b1a1453d9579d421a574691e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd21231d9af34d9aa4f9e03229b49a30', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:05:23.311487 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-7291bd2f-e1c1-4b6d-8a02-03c3a941e261 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/fa9945a3-51cc-4366-bc7f-31ddb4f0fa01" status: 200 len: 1667 microversion: 2.1 time: 0.159202 Jul 16 22:05:23.312065 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3324/6655] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:05:23 2022] GET /compute/v2.1/servers/fa9945a3-51cc-4366-bc7f-31ddb4f0fa01 => generated 1667 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:05:23.331076 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-73f0ac9e-471c-40e0-bc66-06fe88f849e4 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:05:23.331980 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-73f0ac9e-471c-40e0-bc66-06fe88f849e4 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: fa9945a3-51cc-4366-bc7f-31ddb4f0fa01] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:05:23.338544 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-73f0ac9e-471c-40e0-bc66-06fe88f849e4 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:05:23.339027 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-73f0ac9e-471c-40e0-bc66-06fe88f849e4 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:05:23.339796 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-73f0ac9e-471c-40e0-bc66-06fe88f849e4 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:05:23.340243 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-73f0ac9e-471c-40e0-bc66-06fe88f849e4 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:05:23.471539 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-73f0ac9e-471c-40e0-bc66-06fe88f849e4 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '718f6de3b1a1453d9579d421a574691e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd21231d9af34d9aa4f9e03229b49a30', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:05:23.479167 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-73f0ac9e-471c-40e0-bc66-06fe88f849e4 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/fa9945a3-51cc-4366-bc7f-31ddb4f0fa01" status: 200 len: 1667 microversion: 2.1 time: 0.154613 Jul 16 22:05:23.479734 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3332/6656] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:05:23 2022] GET /compute/v2.1/servers/fa9945a3-51cc-4366-bc7f-31ddb4f0fa01 => generated 1667 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:05:23.495095 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-dc186d59-f348-483a-9e52-cd642145974b tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:05:23.495720 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-dc186d59-f348-483a-9e52-cd642145974b tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: fa9945a3-51cc-4366-bc7f-31ddb4f0fa01] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:05:23.497701 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_api:SELECT=51,nova_api:UPDATE=1 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:05:23.502484 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-dc186d59-f348-483a-9e52-cd642145974b tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:05:23.502940 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-dc186d59-f348-483a-9e52-cd642145974b tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:05:23.503741 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-dc186d59-f348-483a-9e52-cd642145974b tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:05:23.504182 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-dc186d59-f348-483a-9e52-cd642145974b tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:05:23.637483 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-dc186d59-f348-483a-9e52-cd642145974b tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '718f6de3b1a1453d9579d421a574691e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd21231d9af34d9aa4f9e03229b49a30', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:05:23.645496 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-dc186d59-f348-483a-9e52-cd642145974b tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/fa9945a3-51cc-4366-bc7f-31ddb4f0fa01" status: 200 len: 1667 microversion: 2.1 time: 0.154244 Jul 16 22:05:23.646071 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3325/6657] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:05:23 2022] GET /compute/v2.1/servers/fa9945a3-51cc-4366-bc7f-31ddb4f0fa01 => generated 1667 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:05:23.662298 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-90b7ccc9-8fb3-4716-a97b-6796da3c7818 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:05:23.662824 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-90b7ccc9-8fb3-4716-a97b-6796da3c7818 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: fa9945a3-51cc-4366-bc7f-31ddb4f0fa01] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:05:23.669012 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-90b7ccc9-8fb3-4716-a97b-6796da3c7818 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:05:23.669480 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-90b7ccc9-8fb3-4716-a97b-6796da3c7818 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:05:23.670278 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-90b7ccc9-8fb3-4716-a97b-6796da3c7818 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:05:23.670722 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-90b7ccc9-8fb3-4716-a97b-6796da3c7818 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:05:23.695629 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-90b7ccc9-8fb3-4716-a97b-6796da3c7818 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: fa9945a3-51cc-4366-bc7f-31ddb4f0fa01] Going to try to terminate instance {{(pid=138681) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 16 22:05:23.999279 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-90b7ccc9-8fb3-4716-a97b-6796da3c7818 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "DELETE /compute/v2.1/servers/fa9945a3-51cc-4366-bc7f-31ddb4f0fa01" status: 204 len: 0 microversion: 2.1 time: 0.341366 Jul 16 22:05:23.999874 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3333/6658] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 22:05:23 2022] DELETE /compute/v2.1/servers/fa9945a3-51cc-4366-bc7f-31ddb4f0fa01 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 16 22:05:24.008714 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-0480715f-13b7-4fc7-a389-1f2fd18d9999 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:05:24.009335 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-0480715f-13b7-4fc7-a389-1f2fd18d9999 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: fa9945a3-51cc-4366-bc7f-31ddb4f0fa01] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:05:24.015618 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0480715f-13b7-4fc7-a389-1f2fd18d9999 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:05:24.016076 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0480715f-13b7-4fc7-a389-1f2fd18d9999 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:05:24.016877 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0480715f-13b7-4fc7-a389-1f2fd18d9999 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:05:24.018030 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0480715f-13b7-4fc7-a389-1f2fd18d9999 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:05:24.151566 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-0480715f-13b7-4fc7-a389-1f2fd18d9999 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '718f6de3b1a1453d9579d421a574691e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd21231d9af34d9aa4f9e03229b49a30', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:05:24.159326 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-0480715f-13b7-4fc7-a389-1f2fd18d9999 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/fa9945a3-51cc-4366-bc7f-31ddb4f0fa01" status: 200 len: 1673 microversion: 2.1 time: 0.154423 Jul 16 22:05:24.159949 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3326/6659] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:05:24 2022] GET /compute/v2.1/servers/fa9945a3-51cc-4366-bc7f-31ddb4f0fa01 => generated 1673 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:05:25.176892 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-c56975b5-3be8-49d8-839e-0c1975f46c3e tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:05:25.177553 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-c56975b5-3be8-49d8-839e-0c1975f46c3e tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: fa9945a3-51cc-4366-bc7f-31ddb4f0fa01] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:05:25.184506 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c56975b5-3be8-49d8-839e-0c1975f46c3e tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:05:25.184964 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c56975b5-3be8-49d8-839e-0c1975f46c3e tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:05:25.185736 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c56975b5-3be8-49d8-839e-0c1975f46c3e tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:05:25.186185 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c56975b5-3be8-49d8-839e-0c1975f46c3e tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:05:25.321443 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-c56975b5-3be8-49d8-839e-0c1975f46c3e tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '718f6de3b1a1453d9579d421a574691e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd21231d9af34d9aa4f9e03229b49a30', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:05:25.329406 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-c56975b5-3be8-49d8-839e-0c1975f46c3e tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/fa9945a3-51cc-4366-bc7f-31ddb4f0fa01" status: 200 len: 1673 microversion: 2.1 time: 0.156662 Jul 16 22:05:25.330913 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3334/6660] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:05:25 2022] GET /compute/v2.1/servers/fa9945a3-51cc-4366-bc7f-31ddb4f0fa01 => generated 1673 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:05:26.347845 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-4d008113-2597-4ea4-a783-a065b096c367 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:05:26.348499 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-4d008113-2597-4ea4-a783-a065b096c367 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: fa9945a3-51cc-4366-bc7f-31ddb4f0fa01] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:05:26.351343 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell0:SELECT=3 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:05:26.356388 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4d008113-2597-4ea4-a783-a065b096c367 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:05:26.356851 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4d008113-2597-4ea4-a783-a065b096c367 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:05:26.357632 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4d008113-2597-4ea4-a783-a065b096c367 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:05:26.358195 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4d008113-2597-4ea4-a783-a065b096c367 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:05:26.426544 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-4d008113-2597-4ea4-a783-a065b096c367 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '718f6de3b1a1453d9579d421a574691e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd21231d9af34d9aa4f9e03229b49a30', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:05:26.434994 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-4d008113-2597-4ea4-a783-a065b096c367 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/fa9945a3-51cc-4366-bc7f-31ddb4f0fa01" status: 200 len: 1606 microversion: 2.1 time: 0.091050 Jul 16 22:05:26.445944 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3327/6661] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:05:26 2022] GET /compute/v2.1/servers/fa9945a3-51cc-4366-bc7f-31ddb4f0fa01 => generated 1606 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:05:26.502754 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-521eb39c-e7dc-42ed-be0e-79e67f2aef10 req-5b3da5dc-1cce-4c3c-b12b-0d5ce938ab57 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "fa9945a3-51cc-4366-bc7f-31ddb4f0fa01", "name": "network-vif-deleted", "tag": "1ca4d162-bc94-4c13-a163-3f3656a349e8"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:05:26.513115 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-521eb39c-e7dc-42ed-be0e-79e67f2aef10 req-5b3da5dc-1cce-4c3c-b12b-0d5ce938ab57 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:05:26.513650 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-521eb39c-e7dc-42ed-be0e-79e67f2aef10 req-5b3da5dc-1cce-4c3c-b12b-0d5ce938ab57 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:05:26.525002 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-521eb39c-e7dc-42ed-be0e-79e67f2aef10 req-5b3da5dc-1cce-4c3c-b12b-0d5ce938ab57 service nova] Creating event network-vif-deleted:1ca4d162-bc94-4c13-a163-3f3656a349e8 for instance fa9945a3-51cc-4366-bc7f-31ddb4f0fa01 on nch1-master-1 Jul 16 22:05:26.530494 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-521eb39c-e7dc-42ed-be0e-79e67f2aef10 req-5b3da5dc-1cce-4c3c-b12b-0d5ce938ab57 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.031864 Jul 16 22:05:26.531140 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3335/6662] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 22:05:26 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 22:05:27.452465 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-b9c83ba9-9767-468a-9342-61f6b0d32b9a tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:05:27.453090 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-b9c83ba9-9767-468a-9342-61f6b0d32b9a tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: fa9945a3-51cc-4366-bc7f-31ddb4f0fa01] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:05:27.460614 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b9c83ba9-9767-468a-9342-61f6b0d32b9a tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:05:27.461075 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b9c83ba9-9767-468a-9342-61f6b0d32b9a tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:05:27.461872 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b9c83ba9-9767-468a-9342-61f6b0d32b9a tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:05:27.462307 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b9c83ba9-9767-468a-9342-61f6b0d32b9a tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:05:27.476688 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-b9c83ba9-9767-468a-9342-61f6b0d32b9a tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] HTTP exception thrown: Instance fa9945a3-51cc-4366-bc7f-31ddb4f0fa01 could not be found. Jul 16 22:05:27.478348 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-b9c83ba9-9767-468a-9342-61f6b0d32b9a tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Request method failure captured: Jul 16 22:05:27.478348 ncdv-master-1 devstack@n-api.service[138680]: request: GET /compute/v2.1/servers/fa9945a3-51cc-4366-bc7f-31ddb4f0fa01 HTTP/1.1 Jul 16 22:05:27.478348 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 22:05:27.478348 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 22:05:27.478348 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 22:05:27.478348 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 0 Jul 16 22:05:27.478348 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 22:05:27.478348 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 22:05:27.478348 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 22:05:27.478348 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 22:05:27.478348 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0zUsISRuFerqPWLnIU6pWtJZ1G33kS5UQBpOE-Qn9W1zRIsDO0obaxdGmXyhe5FNe11pjr8YSFwmEAtO4tE-QrUwHNB6R_8ZgLrv3bb_zQTEpcOValb2w1FWdJ1eIHV_96nHchz9PmXAJpOM5Cp_lLo9a71M8MQcZrQgoRzDb41oZDA Jul 16 22:05:27.478348 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 22:05:27.478348 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 22:05:27.478348 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 22:05:27.478348 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 22:05:27.478348 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 22:05:27.478348 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 22:05:27.478348 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: fd21231d9af34d9aa4f9e03229b49a30 Jul 16 22:05:27.478348 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-ServerActionsTestJSON-164809533 Jul 16 22:05:27.478348 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 22:05:27.478348 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 22:05:27.478348 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/fd21231d9af34d9aa4f9e03229b49a30"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/fd21231d9af34d9aa4f9e03229b49a30"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_fd21231d9af34d9aa4f9e03229b49a30"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/fd21231d9af34d9aa4f9e03229b49a30"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 22:05:27.478348 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-ServerActionsTestJSON-164809533 Jul 16 22:05:27.478348 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: fd21231d9af34d9aa4f9e03229b49a30 Jul 16 22:05:27.478348 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-ServerActionsTestJSON-164809533 Jul 16 22:05:27.478348 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-ServerActionsTestJSON-164809533-project Jul 16 22:05:27.478348 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 22:05:27.478348 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 22:05:27.478348 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: 718f6de3b1a1453d9579d421a574691e Jul 16 22:05:27.478348 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-ServerActionsTestJSON-164809533-project Jul 16 22:05:27.478348 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 22:05:27.478348 ncdv-master-1 devstack@n-api.service[138680]: exception: Instance fa9945a3-51cc-4366-bc7f-31ddb4f0fa01 could not be found. Jul 16 22:05:27.478348 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': 'fa9945a3-51cc-4366-bc7f-31ddb4f0fa01'} Jul 16 22:05:27.478348 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 22:05:27.478348 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 22:05:27.478348 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 22:05:27.478348 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 22:05:27.478348 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 22:05:27.478348 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 22:05:27.478348 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 22:05:27.478348 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 22:05:27.478348 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 22:05:27.478348 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 22:05:27.478348 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance fa9945a3-51cc-4366-bc7f-31ddb4f0fa01 could not be found. Jul 16 22:05:27.478348 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 22:05:27.478348 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 22:05:27.478348 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 22:05:27.478348 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 22:05:27.478348 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 22:05:27.478348 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 22:05:27.478348 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 22:05:27.478348 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 22:05:27.478348 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 22:05:27.478348 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 22:05:27.478348 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 22:05:27.478348 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 22:05:27.478348 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 22:05:27.478348 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 22:05:27.478348 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 22:05:27.478348 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 22:05:27.478348 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance fa9945a3-51cc-4366-bc7f-31ddb4f0fa01 could not be found. Jul 16 22:05:27.478348 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 22:05:27.478348 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 22:05:27.478348 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 22:05:27.482644 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 22:05:27.482644 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 22:05:27.482644 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 22:05:27.482644 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 22:05:27.482644 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 22:05:27.482644 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance fa9945a3-51cc-4366-bc7f-31ddb4f0fa01 could not be found. Jul 16 22:05:27.482644 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 22:05:27.482644 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-b9c83ba9-9767-468a-9342-61f6b0d32b9a tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Returning 404 to user: Instance fa9945a3-51cc-4366-bc7f-31ddb4f0fa01 could not be found. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 22:05:27.482644 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-b9c83ba9-9767-468a-9342-61f6b0d32b9a tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/fa9945a3-51cc-4366-bc7f-31ddb4f0fa01" status: 404 len: 111 microversion: 2.1 time: 0.031231 Jul 16 22:05:27.482644 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3328/6663] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:05:27 2022] GET /compute/v2.1/servers/fa9945a3-51cc-4366-bc7f-31ddb4f0fa01 => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 22:05:27.499789 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-ec9b5882-605e-46b5-8523-d6950eb6e21e tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:05:27.504846 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-ec9b5882-605e-46b5-8523-d6950eb6e21e tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: fa9945a3-51cc-4366-bc7f-31ddb4f0fa01] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:05:27.507028 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ec9b5882-605e-46b5-8523-d6950eb6e21e tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:05:27.507513 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ec9b5882-605e-46b5-8523-d6950eb6e21e tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:05:27.508288 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ec9b5882-605e-46b5-8523-d6950eb6e21e tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:05:27.508740 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ec9b5882-605e-46b5-8523-d6950eb6e21e tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:05:27.522917 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-ec9b5882-605e-46b5-8523-d6950eb6e21e tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] HTTP exception thrown: Instance fa9945a3-51cc-4366-bc7f-31ddb4f0fa01 could not be found. Jul 16 22:05:27.524595 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-ec9b5882-605e-46b5-8523-d6950eb6e21e tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Request method failure captured: Jul 16 22:05:27.524595 ncdv-master-1 devstack@n-api.service[138681]: request: GET /compute/v2.1/servers/fa9945a3-51cc-4366-bc7f-31ddb4f0fa01 HTTP/1.1 Jul 16 22:05:27.524595 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 22:05:27.524595 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 22:05:27.524595 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 22:05:27.524595 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 22:05:27.524595 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 22:05:27.524595 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 22:05:27.524595 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 22:05:27.524595 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 22:05:27.524595 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0zUsISRuFerqPWLnIU6pWtJZ1G33kS5UQBpOE-Qn9W1zRIsDO0obaxdGmXyhe5FNe11pjr8YSFwmEAtO4tE-QrUwHNB6R_8ZgLrv3bb_zQTEpcOValb2w1FWdJ1eIHV_96nHchz9PmXAJpOM5Cp_lLo9a71M8MQcZrQgoRzDb41oZDA Jul 16 22:05:27.524595 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 22:05:27.524595 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 22:05:27.524595 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 22:05:27.524595 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 22:05:27.524595 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 22:05:27.524595 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 22:05:27.524595 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: fd21231d9af34d9aa4f9e03229b49a30 Jul 16 22:05:27.524595 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-ServerActionsTestJSON-164809533 Jul 16 22:05:27.524595 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 22:05:27.524595 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 22:05:27.524595 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/fd21231d9af34d9aa4f9e03229b49a30"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/fd21231d9af34d9aa4f9e03229b49a30"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_fd21231d9af34d9aa4f9e03229b49a30"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/fd21231d9af34d9aa4f9e03229b49a30"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 22:05:27.524595 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-ServerActionsTestJSON-164809533 Jul 16 22:05:27.524595 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: fd21231d9af34d9aa4f9e03229b49a30 Jul 16 22:05:27.524595 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-ServerActionsTestJSON-164809533 Jul 16 22:05:27.524595 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-ServerActionsTestJSON-164809533-project Jul 16 22:05:27.524595 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 22:05:27.524595 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 22:05:27.524595 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: 718f6de3b1a1453d9579d421a574691e Jul 16 22:05:27.524595 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-ServerActionsTestJSON-164809533-project Jul 16 22:05:27.524595 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 22:05:27.524595 ncdv-master-1 devstack@n-api.service[138681]: exception: Instance fa9945a3-51cc-4366-bc7f-31ddb4f0fa01 could not be found. Jul 16 22:05:27.524595 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': 'fa9945a3-51cc-4366-bc7f-31ddb4f0fa01'} Jul 16 22:05:27.524595 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 22:05:27.524595 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 22:05:27.524595 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 22:05:27.524595 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 22:05:27.524595 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 22:05:27.524595 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 22:05:27.524595 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 22:05:27.524595 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 22:05:27.524595 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 22:05:27.524595 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 22:05:27.524595 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance fa9945a3-51cc-4366-bc7f-31ddb4f0fa01 could not be found. Jul 16 22:05:27.524595 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 22:05:27.524595 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 22:05:27.524595 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 22:05:27.524595 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 22:05:27.524595 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 22:05:27.524595 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 22:05:27.524595 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 22:05:27.524595 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 22:05:27.524595 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 22:05:27.524595 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 22:05:27.524595 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 22:05:27.524595 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 22:05:27.524595 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 22:05:27.524595 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 22:05:27.524595 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 22:05:27.524595 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 22:05:27.524595 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance fa9945a3-51cc-4366-bc7f-31ddb4f0fa01 could not be found. Jul 16 22:05:27.524595 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 22:05:27.524595 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 22:05:27.524595 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 22:05:27.530201 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 22:05:27.530201 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 22:05:27.530201 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 22:05:27.530201 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 22:05:27.530201 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 22:05:27.530201 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance fa9945a3-51cc-4366-bc7f-31ddb4f0fa01 could not be found. Jul 16 22:05:27.530201 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 22:05:27.530201 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-ec9b5882-605e-46b5-8523-d6950eb6e21e tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Returning 404 to user: Instance fa9945a3-51cc-4366-bc7f-31ddb4f0fa01 could not be found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 22:05:27.530201 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-ec9b5882-605e-46b5-8523-d6950eb6e21e tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/fa9945a3-51cc-4366-bc7f-31ddb4f0fa01" status: 404 len: 111 microversion: 2.1 time: 0.030578 Jul 16 22:05:27.530201 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3336/6664] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:05:27 2022] GET /compute/v2.1/servers/fa9945a3-51cc-4366-bc7f-31ddb4f0fa01 => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 22:05:27.576850 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-7d3c6d08-8afb-4cf6-92d6-df8a865721eb tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerActionsTestJSON-server-977702268", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452", "networks": [{"uuid": "b3593d62-fee4-4445-83ed-ea9589fa3fcd"}], "security_groups": [{"name": "tempest-securitygroup--389060960"}], "key_name": "tempest-keypair-26228548", "user_data": "IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIHVidW50dSB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH51YnVudHUvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo="}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:05:27.681764 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.network.neutron [None req-7d3c6d08-8afb-4cf6-92d6-df8a865721eb tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] validate_networks() for [('b3593d62-fee4-4445-83ed-ea9589fa3fcd', None, None, None, None, None)] {{(pid=138680) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 16 22:05:28.104118 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-7d3c6d08-8afb-4cf6-92d6-df8a865721eb tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Flavor limits 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 16 22:05:28.104554 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-7d3c6d08-8afb-4cf6-92d6-df8a865721eb tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Image limits 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 16 22:05:28.105069 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-7d3c6d08-8afb-4cf6-92d6-df8a865721eb tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Flavor pref 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 16 22:05:28.105520 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-7d3c6d08-8afb-4cf6-92d6-df8a865721eb tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Image pref 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 16 22:05:28.105944 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-7d3c6d08-8afb-4cf6-92d6-df8a865721eb tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 16 22:05:28.115319 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7d3c6d08-8afb-4cf6-92d6-df8a865721eb tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:05:28.115782 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7d3c6d08-8afb-4cf6-92d6-df8a865721eb tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:05:28.116884 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7d3c6d08-8afb-4cf6-92d6-df8a865721eb tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:05:28.117337 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7d3c6d08-8afb-4cf6-92d6-df8a865721eb tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:05:28.134296 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-7d3c6d08-8afb-4cf6-92d6-df8a865721eb tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Getting quotas for project fd21231d9af34d9aa4f9e03229b49a30. Resources: {'ram', 'instances', 'cores'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 22:05:28.139684 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-7d3c6d08-8afb-4cf6-92d6-df8a865721eb tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Getting quotas for user 718f6de3b1a1453d9579d421a574691e and project fd21231d9af34d9aa4f9e03229b49a30. Resources: {'ram', 'instances', 'cores'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 22:05:28.152626 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-7d3c6d08-8afb-4cf6-92d6-df8a865721eb tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Going to run 1 instances... {{(pid=138680) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 16 22:05:28.154924 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-7d3c6d08-8afb-4cf6-92d6-df8a865721eb tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: 7efe6f96-4156-45dc-b337-fb451e51fdd4] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138680) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 16 22:05:28.264574 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-7d3c6d08-8afb-4cf6-92d6-df8a865721eb tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 405 microversion: 2.1 time: 0.691645 Jul 16 22:05:28.265277 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3329/6665] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 22:05:27 2022] POST /compute/v2.1/servers => generated 405 bytes in 693 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 16 22:05:28.276079 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-519ba178-d4f9-4bb9-a12a-7a29b5a594f1 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:05:28.277711 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-519ba178-d4f9-4bb9-a12a-7a29b5a594f1 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: 7efe6f96-4156-45dc-b337-fb451e51fdd4] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:05:28.336663 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-519ba178-d4f9-4bb9-a12a-7a29b5a594f1 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '718f6de3b1a1453d9579d421a574691e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd21231d9af34d9aa4f9e03229b49a30', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:05:28.342113 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-519ba178-d4f9-4bb9-a12a-7a29b5a594f1 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/7efe6f96-4156-45dc-b337-fb451e51fdd4" status: 200 len: 1236 microversion: 2.1 time: 0.069869 Jul 16 22:05:28.342799 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3337/6666] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:05:28 2022] GET /compute/v2.1/servers/7efe6f96-4156-45dc-b337-fb451e51fdd4 => generated 1236 bytes in 71 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:05:29.360823 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-f54e9493-5e50-486c-b900-e6312bd676e2 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:05:29.361453 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-f54e9493-5e50-486c-b900-e6312bd676e2 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: 7efe6f96-4156-45dc-b337-fb451e51fdd4] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:05:29.368119 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f54e9493-5e50-486c-b900-e6312bd676e2 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:05:29.368966 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f54e9493-5e50-486c-b900-e6312bd676e2 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:05:29.369750 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f54e9493-5e50-486c-b900-e6312bd676e2 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:05:29.370206 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f54e9493-5e50-486c-b900-e6312bd676e2 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:05:29.436307 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-f54e9493-5e50-486c-b900-e6312bd676e2 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '718f6de3b1a1453d9579d421a574691e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd21231d9af34d9aa4f9e03229b49a30', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:05:29.444736 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-f54e9493-5e50-486c-b900-e6312bd676e2 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/7efe6f96-4156-45dc-b337-fb451e51fdd4" status: 200 len: 1232 microversion: 2.1 time: 0.089147 Jul 16 22:05:29.447156 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3330/6667] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:05:29 2022] GET /compute/v2.1/servers/7efe6f96-4156-45dc-b337-fb451e51fdd4 => generated 1232 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:05:30.464266 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-9dbd8fb6-856f-4b83-80c5-5aee5f282d77 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:05:30.465083 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-9dbd8fb6-856f-4b83-80c5-5aee5f282d77 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: 7efe6f96-4156-45dc-b337-fb451e51fdd4] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:05:30.477436 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9dbd8fb6-856f-4b83-80c5-5aee5f282d77 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:05:30.477982 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9dbd8fb6-856f-4b83-80c5-5aee5f282d77 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:05:30.479035 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9dbd8fb6-856f-4b83-80c5-5aee5f282d77 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:05:30.479579 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9dbd8fb6-856f-4b83-80c5-5aee5f282d77 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:05:30.550669 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-9dbd8fb6-856f-4b83-80c5-5aee5f282d77 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '718f6de3b1a1453d9579d421a574691e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd21231d9af34d9aa4f9e03229b49a30', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:05:30.558235 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-9dbd8fb6-856f-4b83-80c5-5aee5f282d77 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/7efe6f96-4156-45dc-b337-fb451e51fdd4" status: 200 len: 1306 microversion: 2.1 time: 0.098627 Jul 16 22:05:30.558810 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3338/6668] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:05:30 2022] GET /compute/v2.1/servers/7efe6f96-4156-45dc-b337-fb451e51fdd4 => generated 1306 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:05:31.580960 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-4d30d62d-3690-4f46-90d0-09f012f22751 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:05:31.582388 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-4d30d62d-3690-4f46-90d0-09f012f22751 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: 7efe6f96-4156-45dc-b337-fb451e51fdd4] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:05:31.596653 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4d30d62d-3690-4f46-90d0-09f012f22751 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:05:31.597269 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4d30d62d-3690-4f46-90d0-09f012f22751 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:05:31.598150 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4d30d62d-3690-4f46-90d0-09f012f22751 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:05:31.598711 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4d30d62d-3690-4f46-90d0-09f012f22751 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:05:31.753436 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-4d30d62d-3690-4f46-90d0-09f012f22751 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '718f6de3b1a1453d9579d421a574691e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd21231d9af34d9aa4f9e03229b49a30', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:05:31.761593 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-4d30d62d-3690-4f46-90d0-09f012f22751 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/7efe6f96-4156-45dc-b337-fb451e51fdd4" status: 200 len: 1361 microversion: 2.1 time: 0.188746 Jul 16 22:05:31.762182 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3331/6669] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:05:31 2022] GET /compute/v2.1/servers/7efe6f96-4156-45dc-b337-fb451e51fdd4 => generated 1361 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:05:31.886193 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-ed0d799c-17f0-4717-ba94-fb121259fdd7 req-709b62f5-1a20-46ef-98f6-cc426ab26fab service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "7efe6f96-4156-45dc-b337-fb451e51fdd4", "tag": "8a32905f-f42d-49e2-a600-9834e8649a5e"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:05:31.898040 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-ed0d799c-17f0-4717-ba94-fb121259fdd7 req-709b62f5-1a20-46ef-98f6-cc426ab26fab service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:05:31.898563 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-ed0d799c-17f0-4717-ba94-fb121259fdd7 req-709b62f5-1a20-46ef-98f6-cc426ab26fab service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:05:31.911927 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-ed0d799c-17f0-4717-ba94-fb121259fdd7 req-709b62f5-1a20-46ef-98f6-cc426ab26fab service nova] Creating event network-changed:8a32905f-f42d-49e2-a600-9834e8649a5e for instance 7efe6f96-4156-45dc-b337-fb451e51fdd4 on nch1-master-1 Jul 16 22:05:31.918234 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-ed0d799c-17f0-4717-ba94-fb121259fdd7 req-709b62f5-1a20-46ef-98f6-cc426ab26fab service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.034581 Jul 16 22:05:31.918234 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3339/6670] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 22:05:31 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 16 22:05:32.779982 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-5b6b486f-10e3-411e-83fe-7f7240594b15 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:05:32.780583 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-5b6b486f-10e3-411e-83fe-7f7240594b15 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: 7efe6f96-4156-45dc-b337-fb451e51fdd4] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:05:32.788149 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5b6b486f-10e3-411e-83fe-7f7240594b15 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:05:32.788607 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5b6b486f-10e3-411e-83fe-7f7240594b15 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:05:32.789495 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5b6b486f-10e3-411e-83fe-7f7240594b15 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:05:32.789943 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5b6b486f-10e3-411e-83fe-7f7240594b15 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:05:32.921183 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-5b6b486f-10e3-411e-83fe-7f7240594b15 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '718f6de3b1a1453d9579d421a574691e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd21231d9af34d9aa4f9e03229b49a30', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:05:32.929347 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-5b6b486f-10e3-411e-83fe-7f7240594b15 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/7efe6f96-4156-45dc-b337-fb451e51fdd4" status: 200 len: 1361 microversion: 2.1 time: 0.153567 Jul 16 22:05:32.929954 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3332/6671] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:05:32 2022] GET /compute/v2.1/servers/7efe6f96-4156-45dc-b337-fb451e51fdd4 => generated 1361 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:05:33.948592 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-9b8092bf-c9fb-4420-8c1c-6c55b62d98aa tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:05:33.949308 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-9b8092bf-c9fb-4420-8c1c-6c55b62d98aa tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: 7efe6f96-4156-45dc-b337-fb451e51fdd4] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:05:33.956185 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9b8092bf-c9fb-4420-8c1c-6c55b62d98aa tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:05:33.956819 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9b8092bf-c9fb-4420-8c1c-6c55b62d98aa tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:05:33.957551 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9b8092bf-c9fb-4420-8c1c-6c55b62d98aa tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:05:33.958007 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9b8092bf-c9fb-4420-8c1c-6c55b62d98aa tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:05:34.148924 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-9b8092bf-c9fb-4420-8c1c-6c55b62d98aa tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '718f6de3b1a1453d9579d421a574691e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd21231d9af34d9aa4f9e03229b49a30', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:05:34.156483 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-9b8092bf-c9fb-4420-8c1c-6c55b62d98aa tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/7efe6f96-4156-45dc-b337-fb451e51fdd4" status: 200 len: 1361 microversion: 2.1 time: 0.213340 Jul 16 22:05:34.157053 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3340/6672] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:05:33 2022] GET /compute/v2.1/servers/7efe6f96-4156-45dc-b337-fb451e51fdd4 => generated 1361 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:05:35.175223 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-80439d00-c1c1-45f5-b1a2-9ab530cd1b23 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:05:35.175852 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-80439d00-c1c1-45f5-b1a2-9ab530cd1b23 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: 7efe6f96-4156-45dc-b337-fb451e51fdd4] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:05:35.182144 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-80439d00-c1c1-45f5-b1a2-9ab530cd1b23 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:05:35.182607 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-80439d00-c1c1-45f5-b1a2-9ab530cd1b23 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:05:35.183405 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-80439d00-c1c1-45f5-b1a2-9ab530cd1b23 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:05:35.183853 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-80439d00-c1c1-45f5-b1a2-9ab530cd1b23 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:05:35.375895 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-80439d00-c1c1-45f5-b1a2-9ab530cd1b23 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '718f6de3b1a1453d9579d421a574691e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd21231d9af34d9aa4f9e03229b49a30', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:05:35.383812 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-80439d00-c1c1-45f5-b1a2-9ab530cd1b23 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/7efe6f96-4156-45dc-b337-fb451e51fdd4" status: 200 len: 1361 microversion: 2.1 time: 0.213615 Jul 16 22:05:35.384373 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3333/6673] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:05:35 2022] GET /compute/v2.1/servers/7efe6f96-4156-45dc-b337-fb451e51fdd4 => generated 1361 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:05:36.402203 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-7a199083-787f-4876-aa27-062db91edbde tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:05:36.402871 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-7a199083-787f-4876-aa27-062db91edbde tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: 7efe6f96-4156-45dc-b337-fb451e51fdd4] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:05:36.410883 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7a199083-787f-4876-aa27-062db91edbde tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:05:36.411396 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7a199083-787f-4876-aa27-062db91edbde tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:05:36.412223 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7a199083-787f-4876-aa27-062db91edbde tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:05:36.412673 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7a199083-787f-4876-aa27-062db91edbde tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:05:36.549558 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-7a199083-787f-4876-aa27-062db91edbde tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '718f6de3b1a1453d9579d421a574691e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd21231d9af34d9aa4f9e03229b49a30', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:05:36.557251 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-7a199083-787f-4876-aa27-062db91edbde tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/7efe6f96-4156-45dc-b337-fb451e51fdd4" status: 200 len: 1361 microversion: 2.1 time: 0.159233 Jul 16 22:05:36.557829 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3341/6674] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:05:36 2022] GET /compute/v2.1/servers/7efe6f96-4156-45dc-b337-fb451e51fdd4 => generated 1361 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:05:37.574115 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-47874d1c-0c2c-45e4-b8be-4a4128f06231 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:05:37.574756 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-47874d1c-0c2c-45e4-b8be-4a4128f06231 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: 7efe6f96-4156-45dc-b337-fb451e51fdd4] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:05:37.582687 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-47874d1c-0c2c-45e4-b8be-4a4128f06231 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:05:37.583176 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-47874d1c-0c2c-45e4-b8be-4a4128f06231 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:05:37.583947 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-47874d1c-0c2c-45e4-b8be-4a4128f06231 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:05:37.584390 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-47874d1c-0c2c-45e4-b8be-4a4128f06231 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:05:37.715380 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-47874d1c-0c2c-45e4-b8be-4a4128f06231 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '718f6de3b1a1453d9579d421a574691e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd21231d9af34d9aa4f9e03229b49a30', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:05:37.723764 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-47874d1c-0c2c-45e4-b8be-4a4128f06231 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/7efe6f96-4156-45dc-b337-fb451e51fdd4" status: 200 len: 1361 microversion: 2.1 time: 0.153571 Jul 16 22:05:37.724336 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3334/6675] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:05:37 2022] GET /compute/v2.1/servers/7efe6f96-4156-45dc-b337-fb451e51fdd4 => generated 1361 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:05:38.353116 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-2e164be5-515d-4d5c-882b-cf841859b77f req-8a047466-b626-4f2f-95fd-e8fb6c6d1f5a service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "7efe6f96-4156-45dc-b337-fb451e51fdd4", "name": "network-vif-plugged", "status": "completed", "tag": "8a32905f-f42d-49e2-a600-9834e8649a5e"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:05:38.357625 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell0:SELECT=2 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:05:38.363762 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-2e164be5-515d-4d5c-882b-cf841859b77f req-8a047466-b626-4f2f-95fd-e8fb6c6d1f5a service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:05:38.364227 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-2e164be5-515d-4d5c-882b-cf841859b77f req-8a047466-b626-4f2f-95fd-e8fb6c6d1f5a service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:05:38.375415 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-2e164be5-515d-4d5c-882b-cf841859b77f req-8a047466-b626-4f2f-95fd-e8fb6c6d1f5a service nova] Creating event network-vif-plugged:8a32905f-f42d-49e2-a600-9834e8649a5e for instance 7efe6f96-4156-45dc-b337-fb451e51fdd4 on nch1-master-1 Jul 16 22:05:38.380907 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-2e164be5-515d-4d5c-882b-cf841859b77f req-8a047466-b626-4f2f-95fd-e8fb6c6d1f5a service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.032004 Jul 16 22:05:38.381525 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3342/6676] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 22:05:38 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 22:05:38.740759 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-8228bf64-38fe-4568-aa67-ea9040128b2d tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:05:38.744586 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-8228bf64-38fe-4568-aa67-ea9040128b2d tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: 7efe6f96-4156-45dc-b337-fb451e51fdd4] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:05:38.744586 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell0:SELECT=2 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:05:38.744586 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell0:SELECT=6 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:05:38.749668 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8228bf64-38fe-4568-aa67-ea9040128b2d tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:05:38.750126 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8228bf64-38fe-4568-aa67-ea9040128b2d tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:05:38.750896 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8228bf64-38fe-4568-aa67-ea9040128b2d tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:05:38.752240 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8228bf64-38fe-4568-aa67-ea9040128b2d tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:05:38.889154 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-8228bf64-38fe-4568-aa67-ea9040128b2d tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '718f6de3b1a1453d9579d421a574691e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd21231d9af34d9aa4f9e03229b49a30', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:05:38.898956 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-8228bf64-38fe-4568-aa67-ea9040128b2d tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/7efe6f96-4156-45dc-b337-fb451e51fdd4" status: 200 len: 1361 microversion: 2.1 time: 0.161651 Jul 16 22:05:38.899570 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3335/6677] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:05:38 2022] GET /compute/v2.1/servers/7efe6f96-4156-45dc-b337-fb451e51fdd4 => generated 1361 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:05:39.917981 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-cdf7ff0c-61c5-42af-b5c2-f75b80ef796b tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:05:39.918697 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-cdf7ff0c-61c5-42af-b5c2-f75b80ef796b tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: 7efe6f96-4156-45dc-b337-fb451e51fdd4] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:05:39.925606 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cdf7ff0c-61c5-42af-b5c2-f75b80ef796b tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:05:39.926416 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cdf7ff0c-61c5-42af-b5c2-f75b80ef796b tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:05:39.926893 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cdf7ff0c-61c5-42af-b5c2-f75b80ef796b tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:05:39.927368 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cdf7ff0c-61c5-42af-b5c2-f75b80ef796b tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:05:40.058978 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-cdf7ff0c-61c5-42af-b5c2-f75b80ef796b tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '718f6de3b1a1453d9579d421a574691e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd21231d9af34d9aa4f9e03229b49a30', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:05:40.066761 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-cdf7ff0c-61c5-42af-b5c2-f75b80ef796b tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/7efe6f96-4156-45dc-b337-fb451e51fdd4" status: 200 len: 1548 microversion: 2.1 time: 0.154069 Jul 16 22:05:40.067395 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3343/6678] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:05:39 2022] GET /compute/v2.1/servers/7efe6f96-4156-45dc-b337-fb451e51fdd4 => generated 1548 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:05:40.086161 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-c5fa271e-681d-43d1-b1b5-7a0f8b83a8a5 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:05:40.086692 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-c5fa271e-681d-43d1-b1b5-7a0f8b83a8a5 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: 7efe6f96-4156-45dc-b337-fb451e51fdd4] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:05:40.093869 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c5fa271e-681d-43d1-b1b5-7a0f8b83a8a5 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:05:40.094267 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c5fa271e-681d-43d1-b1b5-7a0f8b83a8a5 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:05:40.095638 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c5fa271e-681d-43d1-b1b5-7a0f8b83a8a5 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:05:40.096175 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c5fa271e-681d-43d1-b1b5-7a0f8b83a8a5 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:05:40.178484 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-c5fa271e-681d-43d1-b1b5-7a0f8b83a8a5 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/7efe6f96-4156-45dc-b337-fb451e51fdd4/os-interface" status: 200 len: 285 microversion: 2.1 time: 0.096350 Jul 16 22:05:40.179156 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3336/6679] 10.222.0.16 () {58 vars in 1274 bytes} [Sat Jul 16 22:05:40 2022] GET /compute/v2.1/servers/7efe6f96-4156-45dc-b337-fb451e51fdd4/os-interface => generated 285 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 16 22:05:41.219798 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-69d9272e-9984-4543-82a4-210ecfa5971c req-29a6f266-0e4c-445f-802f-08d3cf08857d service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "7efe6f96-4156-45dc-b337-fb451e51fdd4", "tag": "8a32905f-f42d-49e2-a600-9834e8649a5e"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:05:41.229070 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-69d9272e-9984-4543-82a4-210ecfa5971c req-29a6f266-0e4c-445f-802f-08d3cf08857d service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:05:41.229629 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-69d9272e-9984-4543-82a4-210ecfa5971c req-29a6f266-0e4c-445f-802f-08d3cf08857d service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:05:41.240937 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-69d9272e-9984-4543-82a4-210ecfa5971c req-29a6f266-0e4c-445f-802f-08d3cf08857d service nova] Creating event network-changed:8a32905f-f42d-49e2-a600-9834e8649a5e for instance 7efe6f96-4156-45dc-b337-fb451e51fdd4 on nch1-master-1 Jul 16 22:05:41.245895 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-69d9272e-9984-4543-82a4-210ecfa5971c req-29a6f266-0e4c-445f-802f-08d3cf08857d service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.029861 Jul 16 22:05:41.246504 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3344/6680] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 22:05:41 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 31 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 22:06:15.810422 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_api:SELECT=36,nova_api:INSERT=3 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:06:15.811349 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=84 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:06:15.815531 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-53ce68d1-d83a-4e69-a509-21880890d262 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:06:15.816137 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-53ce68d1-d83a-4e69-a509-21880890d262 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: 7efe6f96-4156-45dc-b337-fb451e51fdd4] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:06:15.822348 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-53ce68d1-d83a-4e69-a509-21880890d262 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:06:15.826183 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-53ce68d1-d83a-4e69-a509-21880890d262 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:06:15.826183 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-53ce68d1-d83a-4e69-a509-21880890d262 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:06:15.826183 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-53ce68d1-d83a-4e69-a509-21880890d262 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:06:16.113996 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-53ce68d1-d83a-4e69-a509-21880890d262 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '718f6de3b1a1453d9579d421a574691e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd21231d9af34d9aa4f9e03229b49a30', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:06:16.121707 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-53ce68d1-d83a-4e69-a509-21880890d262 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/7efe6f96-4156-45dc-b337-fb451e51fdd4" status: 200 len: 1667 microversion: 2.1 time: 0.313893 Jul 16 22:06:16.122284 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3337/6681] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:06:15 2022] GET /compute/v2.1/servers/7efe6f96-4156-45dc-b337-fb451e51fdd4 => generated 1667 bytes in 315 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:06:16.137382 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_api:SELECT=36,nova_api:UPDATE=1 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:06:16.138155 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=70,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:06:16.142626 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-d391ba20-9f01-4608-aa24-5e31e4bd728c tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "676c3d70-8f91-42ab-ab5c-273031e5f37b"}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:06:16.144455 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-d391ba20-9f01-4608-aa24-5e31e4bd728c tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: 7efe6f96-4156-45dc-b337-fb451e51fdd4] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:06:16.150887 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d391ba20-9f01-4608-aa24-5e31e4bd728c tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:06:16.153864 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d391ba20-9f01-4608-aa24-5e31e4bd728c tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:06:16.153864 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d391ba20-9f01-4608-aa24-5e31e4bd728c tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:06:16.153864 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d391ba20-9f01-4608-aa24-5e31e4bd728c tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:06:17.344584 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-d391ba20-9f01-4608-aa24-5e31e4bd728c tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "POST /compute/v2.1/servers/7efe6f96-4156-45dc-b337-fb451e51fdd4/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 1.209321 Jul 16 22:06:17.345278 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3345/6682] 10.222.0.16 () {60 vars in 1322 bytes} [Sat Jul 16 22:06:16 2022] POST /compute/v2.1/servers/7efe6f96-4156-45dc-b337-fb451e51fdd4/os-volume_attachments => generated 194 bytes in 1211 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 16 22:06:21.680794 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-4fdd1ec6-cda4-45bc-9d40-bceef53db968 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Action: 'action', calling method: >, body: {"resize": {"flavorRef": "451"}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:06:21.683992 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-4fdd1ec6-cda4-45bc-9d40-bceef53db968 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: 7efe6f96-4156-45dc-b337-fb451e51fdd4] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:06:21.689034 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4fdd1ec6-cda4-45bc-9d40-bceef53db968 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:06:21.689496 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4fdd1ec6-cda4-45bc-9d40-bceef53db968 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:06:21.690976 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4fdd1ec6-cda4-45bc-9d40-bceef53db968 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:06:21.691413 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4fdd1ec6-cda4-45bc-9d40-bceef53db968 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:06:21.720306 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-4fdd1ec6-cda4-45bc-9d40-bceef53db968 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Policy check for compute:servers:resize:cross_cell failed with credentials {'is_admin': False, 'user_id': '718f6de3b1a1453d9579d421a574691e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd21231d9af34d9aa4f9e03229b49a30', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:06:21.726552 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-4fdd1ec6-cda4-45bc-9d40-bceef53db968 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: 7efe6f96-4156-45dc-b337-fb451e51fdd4] Old instance type m1.heat2, new instance type m1.heat {{(pid=138680) resize /opt/stack/nova/nova/compute/api.py:4166}} Jul 16 22:06:21.733359 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-4fdd1ec6-cda4-45bc-9d40-bceef53db968 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Flavor limits 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 16 22:06:21.733791 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-4fdd1ec6-cda4-45bc-9d40-bceef53db968 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Image limits 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 16 22:06:21.734276 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-4fdd1ec6-cda4-45bc-9d40-bceef53db968 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Flavor pref 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 16 22:06:21.734698 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-4fdd1ec6-cda4-45bc-9d40-bceef53db968 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Image pref 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 16 22:06:21.735146 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-4fdd1ec6-cda4-45bc-9d40-bceef53db968 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 16 22:06:21.930912 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-4fdd1ec6-cda4-45bc-9d40-bceef53db968 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "POST /compute/v2.1/servers/7efe6f96-4156-45dc-b337-fb451e51fdd4/action" status: 202 len: 0 microversion: 2.1 time: 0.254161 Jul 16 22:06:21.931574 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3338/6683] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 22:06:21 2022] POST /compute/v2.1/servers/7efe6f96-4156-45dc-b337-fb451e51fdd4/action => generated 0 bytes in 256 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 22:06:21.941210 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-e2a13a1d-97d7-4738-a78f-3c21f7fc95da tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:06:21.941990 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-e2a13a1d-97d7-4738-a78f-3c21f7fc95da tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: 7efe6f96-4156-45dc-b337-fb451e51fdd4] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:06:21.949520 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e2a13a1d-97d7-4738-a78f-3c21f7fc95da tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:06:21.949983 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e2a13a1d-97d7-4738-a78f-3c21f7fc95da tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:06:21.950755 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e2a13a1d-97d7-4738-a78f-3c21f7fc95da tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:06:21.951234 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e2a13a1d-97d7-4738-a78f-3c21f7fc95da tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:06:22.088964 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-e2a13a1d-97d7-4738-a78f-3c21f7fc95da tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '718f6de3b1a1453d9579d421a574691e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd21231d9af34d9aa4f9e03229b49a30', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:06:22.098103 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-e2a13a1d-97d7-4738-a78f-3c21f7fc95da tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/7efe6f96-4156-45dc-b337-fb451e51fdd4" status: 200 len: 1722 microversion: 2.1 time: 0.161854 Jul 16 22:06:22.098684 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3346/6684] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:06:21 2022] GET /compute/v2.1/servers/7efe6f96-4156-45dc-b337-fb451e51fdd4 => generated 1722 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:06:23.116123 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-2f5d4bb6-fc82-4f0c-a0be-fad95ba7d9e0 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:06:23.116750 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-2f5d4bb6-fc82-4f0c-a0be-fad95ba7d9e0 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: 7efe6f96-4156-45dc-b337-fb451e51fdd4] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:06:23.122909 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2f5d4bb6-fc82-4f0c-a0be-fad95ba7d9e0 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:06:23.123853 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2f5d4bb6-fc82-4f0c-a0be-fad95ba7d9e0 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:06:23.124174 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2f5d4bb6-fc82-4f0c-a0be-fad95ba7d9e0 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:06:23.124693 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2f5d4bb6-fc82-4f0c-a0be-fad95ba7d9e0 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:06:23.258297 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-2f5d4bb6-fc82-4f0c-a0be-fad95ba7d9e0 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '718f6de3b1a1453d9579d421a574691e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd21231d9af34d9aa4f9e03229b49a30', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:06:23.266487 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-2f5d4bb6-fc82-4f0c-a0be-fad95ba7d9e0 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/7efe6f96-4156-45dc-b337-fb451e51fdd4" status: 200 len: 1722 microversion: 2.1 time: 0.155160 Jul 16 22:06:23.267081 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3339/6685] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:06:23 2022] GET /compute/v2.1/servers/7efe6f96-4156-45dc-b337-fb451e51fdd4 => generated 1722 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 22:06:24.284889 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-a5e21ef8-6c63-4145-ae6b-0a469d637c12 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:06:24.285516 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-a5e21ef8-6c63-4145-ae6b-0a469d637c12 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: 7efe6f96-4156-45dc-b337-fb451e51fdd4] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:06:24.292567 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a5e21ef8-6c63-4145-ae6b-0a469d637c12 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:06:24.293037 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a5e21ef8-6c63-4145-ae6b-0a469d637c12 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:06:24.293823 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a5e21ef8-6c63-4145-ae6b-0a469d637c12 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:06:24.294281 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a5e21ef8-6c63-4145-ae6b-0a469d637c12 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:06:24.430420 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-a5e21ef8-6c63-4145-ae6b-0a469d637c12 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '718f6de3b1a1453d9579d421a574691e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd21231d9af34d9aa4f9e03229b49a30', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:06:24.438701 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-a5e21ef8-6c63-4145-ae6b-0a469d637c12 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/7efe6f96-4156-45dc-b337-fb451e51fdd4" status: 200 len: 1722 microversion: 2.1 time: 0.158396 Jul 16 22:06:24.439314 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3347/6686] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:06:24 2022] GET /compute/v2.1/servers/7efe6f96-4156-45dc-b337-fb451e51fdd4 => generated 1722 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:06:24.779879 ncdv-master-1 devstack@n-api.service[138680]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jul 16 22:06:24.780419 ncdv-master-1 devstack@n-api.service[138680]: Sat Jul 16 22:06:24 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Jul 16 22:06:25.457013 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-9ae40a79-2cd6-4778-87ae-8d0ef41d800f tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:06:25.457713 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-9ae40a79-2cd6-4778-87ae-8d0ef41d800f tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: 7efe6f96-4156-45dc-b337-fb451e51fdd4] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:06:25.464701 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9ae40a79-2cd6-4778-87ae-8d0ef41d800f tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:06:25.465158 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9ae40a79-2cd6-4778-87ae-8d0ef41d800f tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:06:25.465930 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9ae40a79-2cd6-4778-87ae-8d0ef41d800f tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:06:25.466371 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9ae40a79-2cd6-4778-87ae-8d0ef41d800f tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:06:25.595506 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-9ae40a79-2cd6-4778-87ae-8d0ef41d800f tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '718f6de3b1a1453d9579d421a574691e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd21231d9af34d9aa4f9e03229b49a30', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:06:25.603494 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-9ae40a79-2cd6-4778-87ae-8d0ef41d800f tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/7efe6f96-4156-45dc-b337-fb451e51fdd4" status: 200 len: 1727 microversion: 2.1 time: 0.151378 Jul 16 22:06:25.604084 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3340/6687] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:06:25 2022] GET /compute/v2.1/servers/7efe6f96-4156-45dc-b337-fb451e51fdd4 => generated 1727 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:06:26.127743 ncdv-master-1 devstack@n-api.service[138681]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jul 16 22:06:26.128231 ncdv-master-1 devstack@n-api.service[138681]: Sat Jul 16 22:06:26 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Jul 16 22:06:26.621973 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-80129874-0c1d-4168-ad51-e7f88c3b2d4b tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:06:26.622623 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-80129874-0c1d-4168-ad51-e7f88c3b2d4b tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: 7efe6f96-4156-45dc-b337-fb451e51fdd4] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:06:26.630553 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-80129874-0c1d-4168-ad51-e7f88c3b2d4b tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:06:26.631113 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-80129874-0c1d-4168-ad51-e7f88c3b2d4b tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:06:26.631889 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-80129874-0c1d-4168-ad51-e7f88c3b2d4b tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:06:26.632345 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-80129874-0c1d-4168-ad51-e7f88c3b2d4b tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:06:26.763895 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-80129874-0c1d-4168-ad51-e7f88c3b2d4b tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '718f6de3b1a1453d9579d421a574691e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd21231d9af34d9aa4f9e03229b49a30', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:06:26.773125 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-80129874-0c1d-4168-ad51-e7f88c3b2d4b tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/7efe6f96-4156-45dc-b337-fb451e51fdd4" status: 200 len: 1727 microversion: 2.1 time: 0.155244 Jul 16 22:06:26.773685 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3348/6688] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:06:26 2022] GET /compute/v2.1/servers/7efe6f96-4156-45dc-b337-fb451e51fdd4 => generated 1727 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:06:27.791387 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-53817d69-1597-4227-9bd9-f7bd78a677ef tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:06:27.792061 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-53817d69-1597-4227-9bd9-f7bd78a677ef tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: 7efe6f96-4156-45dc-b337-fb451e51fdd4] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:06:27.799392 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-53817d69-1597-4227-9bd9-f7bd78a677ef tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:06:27.799872 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-53817d69-1597-4227-9bd9-f7bd78a677ef tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:06:27.800632 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-53817d69-1597-4227-9bd9-f7bd78a677ef tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:06:27.801117 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-53817d69-1597-4227-9bd9-f7bd78a677ef tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:06:27.936492 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-53817d69-1597-4227-9bd9-f7bd78a677ef tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '718f6de3b1a1453d9579d421a574691e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd21231d9af34d9aa4f9e03229b49a30', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:06:27.944859 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-53817d69-1597-4227-9bd9-f7bd78a677ef tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/7efe6f96-4156-45dc-b337-fb451e51fdd4" status: 200 len: 1727 microversion: 2.1 time: 0.157620 Jul 16 22:06:27.945444 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3341/6689] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:06:27 2022] GET /compute/v2.1/servers/7efe6f96-4156-45dc-b337-fb451e51fdd4 => generated 1727 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:06:28.962155 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-2c6d5577-c41f-46d4-9081-6d1ebf1acc24 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:06:28.962791 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-2c6d5577-c41f-46d4-9081-6d1ebf1acc24 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: 7efe6f96-4156-45dc-b337-fb451e51fdd4] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:06:28.971795 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2c6d5577-c41f-46d4-9081-6d1ebf1acc24 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:06:28.972332 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2c6d5577-c41f-46d4-9081-6d1ebf1acc24 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:06:28.973142 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2c6d5577-c41f-46d4-9081-6d1ebf1acc24 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:06:28.973604 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2c6d5577-c41f-46d4-9081-6d1ebf1acc24 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:06:29.113048 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-2c6d5577-c41f-46d4-9081-6d1ebf1acc24 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '718f6de3b1a1453d9579d421a574691e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd21231d9af34d9aa4f9e03229b49a30', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:06:29.121544 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-2c6d5577-c41f-46d4-9081-6d1ebf1acc24 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/7efe6f96-4156-45dc-b337-fb451e51fdd4" status: 200 len: 1727 microversion: 2.1 time: 0.163278 Jul 16 22:06:29.122326 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3349/6690] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:06:28 2022] GET /compute/v2.1/servers/7efe6f96-4156-45dc-b337-fb451e51fdd4 => generated 1727 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:06:30.138962 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-37ba3df6-c951-42d3-9f3f-d1a730c0a910 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:06:30.139532 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-37ba3df6-c951-42d3-9f3f-d1a730c0a910 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: 7efe6f96-4156-45dc-b337-fb451e51fdd4] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:06:30.146501 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-37ba3df6-c951-42d3-9f3f-d1a730c0a910 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:06:30.146957 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-37ba3df6-c951-42d3-9f3f-d1a730c0a910 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:06:30.147749 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-37ba3df6-c951-42d3-9f3f-d1a730c0a910 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:06:30.148193 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-37ba3df6-c951-42d3-9f3f-d1a730c0a910 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:06:30.284350 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-37ba3df6-c951-42d3-9f3f-d1a730c0a910 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '718f6de3b1a1453d9579d421a574691e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd21231d9af34d9aa4f9e03229b49a30', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:06:30.292510 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-37ba3df6-c951-42d3-9f3f-d1a730c0a910 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/7efe6f96-4156-45dc-b337-fb451e51fdd4" status: 200 len: 1727 microversion: 2.1 time: 0.157525 Jul 16 22:06:30.293099 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3342/6691] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:06:30 2022] GET /compute/v2.1/servers/7efe6f96-4156-45dc-b337-fb451e51fdd4 => generated 1727 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:06:31.310483 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-b044bbd5-69d1-46b5-84df-ae8c0f69ede1 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:06:31.311152 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-b044bbd5-69d1-46b5-84df-ae8c0f69ede1 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: 7efe6f96-4156-45dc-b337-fb451e51fdd4] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:06:31.318701 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b044bbd5-69d1-46b5-84df-ae8c0f69ede1 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:06:31.319206 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b044bbd5-69d1-46b5-84df-ae8c0f69ede1 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:06:31.319992 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b044bbd5-69d1-46b5-84df-ae8c0f69ede1 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:06:31.320446 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b044bbd5-69d1-46b5-84df-ae8c0f69ede1 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:06:31.449968 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-b044bbd5-69d1-46b5-84df-ae8c0f69ede1 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '718f6de3b1a1453d9579d421a574691e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd21231d9af34d9aa4f9e03229b49a30', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:06:31.458594 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-b044bbd5-69d1-46b5-84df-ae8c0f69ede1 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/7efe6f96-4156-45dc-b337-fb451e51fdd4" status: 200 len: 1727 microversion: 2.1 time: 0.152053 Jul 16 22:06:31.459184 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3350/6692] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:06:31 2022] GET /compute/v2.1/servers/7efe6f96-4156-45dc-b337-fb451e51fdd4 => generated 1727 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:06:32.476471 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-e47da441-99a5-4760-9b66-b012ff290a3e tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:06:32.477081 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-e47da441-99a5-4760-9b66-b012ff290a3e tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: 7efe6f96-4156-45dc-b337-fb451e51fdd4] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:06:32.484077 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e47da441-99a5-4760-9b66-b012ff290a3e tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:06:32.484541 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e47da441-99a5-4760-9b66-b012ff290a3e tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:06:32.485304 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e47da441-99a5-4760-9b66-b012ff290a3e tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:06:32.485747 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e47da441-99a5-4760-9b66-b012ff290a3e tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:06:32.618593 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-e47da441-99a5-4760-9b66-b012ff290a3e tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '718f6de3b1a1453d9579d421a574691e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd21231d9af34d9aa4f9e03229b49a30', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:06:32.626809 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-e47da441-99a5-4760-9b66-b012ff290a3e tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/7efe6f96-4156-45dc-b337-fb451e51fdd4" status: 200 len: 1727 microversion: 2.1 time: 0.155014 Jul 16 22:06:32.627439 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3343/6693] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:06:32 2022] GET /compute/v2.1/servers/7efe6f96-4156-45dc-b337-fb451e51fdd4 => generated 1727 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:06:33.644546 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-0982742e-78a4-4660-8dd7-03492e035cad tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:06:33.645174 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-0982742e-78a4-4660-8dd7-03492e035cad tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: 7efe6f96-4156-45dc-b337-fb451e51fdd4] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:06:33.653161 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0982742e-78a4-4660-8dd7-03492e035cad tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:06:33.653629 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0982742e-78a4-4660-8dd7-03492e035cad tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:06:33.654404 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0982742e-78a4-4660-8dd7-03492e035cad tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:06:33.654856 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0982742e-78a4-4660-8dd7-03492e035cad tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:06:33.788542 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-0982742e-78a4-4660-8dd7-03492e035cad tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '718f6de3b1a1453d9579d421a574691e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd21231d9af34d9aa4f9e03229b49a30', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:06:33.796547 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-0982742e-78a4-4660-8dd7-03492e035cad tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/7efe6f96-4156-45dc-b337-fb451e51fdd4" status: 200 len: 1727 microversion: 2.1 time: 0.155934 Jul 16 22:06:33.797128 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3351/6694] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:06:33 2022] GET /compute/v2.1/servers/7efe6f96-4156-45dc-b337-fb451e51fdd4 => generated 1727 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 22:06:34.813831 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-c3ab78b4-3800-4466-b145-4973ba6b515c tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:06:34.825428 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-c3ab78b4-3800-4466-b145-4973ba6b515c tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: 7efe6f96-4156-45dc-b337-fb451e51fdd4] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:06:34.825428 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c3ab78b4-3800-4466-b145-4973ba6b515c tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:06:34.825428 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c3ab78b4-3800-4466-b145-4973ba6b515c tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:06:34.825428 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c3ab78b4-3800-4466-b145-4973ba6b515c tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:06:34.825428 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c3ab78b4-3800-4466-b145-4973ba6b515c tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:06:34.951716 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-c3ab78b4-3800-4466-b145-4973ba6b515c tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '718f6de3b1a1453d9579d421a574691e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd21231d9af34d9aa4f9e03229b49a30', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:06:34.959944 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-c3ab78b4-3800-4466-b145-4973ba6b515c tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/7efe6f96-4156-45dc-b337-fb451e51fdd4" status: 200 len: 1727 microversion: 2.1 time: 0.150173 Jul 16 22:06:34.960513 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3344/6695] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:06:34 2022] GET /compute/v2.1/servers/7efe6f96-4156-45dc-b337-fb451e51fdd4 => generated 1727 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:06:35.977071 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-9884a16a-31d3-41bd-8ea0-b2c68f2cbab9 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:06:35.977756 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-9884a16a-31d3-41bd-8ea0-b2c68f2cbab9 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: 7efe6f96-4156-45dc-b337-fb451e51fdd4] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:06:35.985032 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9884a16a-31d3-41bd-8ea0-b2c68f2cbab9 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:06:35.985499 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9884a16a-31d3-41bd-8ea0-b2c68f2cbab9 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:06:35.986286 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9884a16a-31d3-41bd-8ea0-b2c68f2cbab9 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:06:35.986734 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9884a16a-31d3-41bd-8ea0-b2c68f2cbab9 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:06:36.123621 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-9884a16a-31d3-41bd-8ea0-b2c68f2cbab9 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '718f6de3b1a1453d9579d421a574691e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd21231d9af34d9aa4f9e03229b49a30', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:06:36.132497 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-9884a16a-31d3-41bd-8ea0-b2c68f2cbab9 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/7efe6f96-4156-45dc-b337-fb451e51fdd4" status: 200 len: 1727 microversion: 2.1 time: 0.159362 Jul 16 22:06:36.133276 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3352/6696] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:06:35 2022] GET /compute/v2.1/servers/7efe6f96-4156-45dc-b337-fb451e51fdd4 => generated 1727 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:06:37.149984 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-ae7ecb4c-12b4-4b7e-a41d-bf0b8686eeec tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:06:37.150533 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-ae7ecb4c-12b4-4b7e-a41d-bf0b8686eeec tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: 7efe6f96-4156-45dc-b337-fb451e51fdd4] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:06:37.157691 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ae7ecb4c-12b4-4b7e-a41d-bf0b8686eeec tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:06:37.158157 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ae7ecb4c-12b4-4b7e-a41d-bf0b8686eeec tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:06:37.158928 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ae7ecb4c-12b4-4b7e-a41d-bf0b8686eeec tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:06:37.159403 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ae7ecb4c-12b4-4b7e-a41d-bf0b8686eeec tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:06:37.370619 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-ae7ecb4c-12b4-4b7e-a41d-bf0b8686eeec tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '718f6de3b1a1453d9579d421a574691e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd21231d9af34d9aa4f9e03229b49a30', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:06:37.378752 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-ae7ecb4c-12b4-4b7e-a41d-bf0b8686eeec tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/7efe6f96-4156-45dc-b337-fb451e51fdd4" status: 200 len: 1727 microversion: 2.1 time: 0.233361 Jul 16 22:06:37.379359 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3345/6697] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:06:37 2022] GET /compute/v2.1/servers/7efe6f96-4156-45dc-b337-fb451e51fdd4 => generated 1727 bytes in 235 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:06:38.399553 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-76e7af2f-fb30-4452-82dc-43a8c2a0ad84 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:06:38.400696 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-76e7af2f-fb30-4452-82dc-43a8c2a0ad84 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: 7efe6f96-4156-45dc-b337-fb451e51fdd4] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:06:38.409053 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-76e7af2f-fb30-4452-82dc-43a8c2a0ad84 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:06:38.409597 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-76e7af2f-fb30-4452-82dc-43a8c2a0ad84 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:06:38.410456 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-76e7af2f-fb30-4452-82dc-43a8c2a0ad84 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:06:38.410919 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-76e7af2f-fb30-4452-82dc-43a8c2a0ad84 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:06:38.560938 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-76e7af2f-fb30-4452-82dc-43a8c2a0ad84 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '718f6de3b1a1453d9579d421a574691e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd21231d9af34d9aa4f9e03229b49a30', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:06:38.570599 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-76e7af2f-fb30-4452-82dc-43a8c2a0ad84 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/7efe6f96-4156-45dc-b337-fb451e51fdd4" status: 200 len: 1726 microversion: 2.1 time: 0.177102 Jul 16 22:06:38.571212 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3353/6698] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:06:38 2022] GET /compute/v2.1/servers/7efe6f96-4156-45dc-b337-fb451e51fdd4 => generated 1726 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:06:38.809230 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-55685766-5d6c-4981-baa2-6a1bfbfaa3ae req-f41dc8d2-3939-4e50-8aa6-a4f5ddc6c3f9 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "7efe6f96-4156-45dc-b337-fb451e51fdd4", "name": "network-vif-unplugged", "status": "completed", "tag": "8a32905f-f42d-49e2-a600-9834e8649a5e"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:06:38.818682 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-55685766-5d6c-4981-baa2-6a1bfbfaa3ae req-f41dc8d2-3939-4e50-8aa6-a4f5ddc6c3f9 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:06:38.819183 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-55685766-5d6c-4981-baa2-6a1bfbfaa3ae req-f41dc8d2-3939-4e50-8aa6-a4f5ddc6c3f9 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:06:38.831764 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-55685766-5d6c-4981-baa2-6a1bfbfaa3ae req-f41dc8d2-3939-4e50-8aa6-a4f5ddc6c3f9 service nova] Creating event network-vif-unplugged:8a32905f-f42d-49e2-a600-9834e8649a5e for instance 7efe6f96-4156-45dc-b337-fb451e51fdd4 on nch2-master-1 Jul 16 22:06:38.837824 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [req-55685766-5d6c-4981-baa2-6a1bfbfaa3ae req-f41dc8d2-3939-4e50-8aa6-a4f5ddc6c3f9 service nova] Instance 7efe6f96-4156-45dc-b337-fb451e51fdd4 is migrating within the same cell, copying events to all relevant hosts: {'nch2-master-1', 'nch1-master-1'} {{(pid=138680) _get_relevant_hosts /opt/stack/nova/nova/compute/api.py:5807}} Jul 16 22:06:38.848608 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-55685766-5d6c-4981-baa2-6a1bfbfaa3ae req-f41dc8d2-3939-4e50-8aa6-a4f5ddc6c3f9 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.043071 Jul 16 22:06:38.849219 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3346/6699] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 22:06:38 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 22:06:39.588436 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-31bb0315-de18-4160-aaee-970f22dc59ff tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:06:39.589074 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-31bb0315-de18-4160-aaee-970f22dc59ff tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: 7efe6f96-4156-45dc-b337-fb451e51fdd4] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:06:39.596233 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-31bb0315-de18-4160-aaee-970f22dc59ff tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:06:39.596752 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-31bb0315-de18-4160-aaee-970f22dc59ff tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:06:39.597538 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-31bb0315-de18-4160-aaee-970f22dc59ff tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:06:39.597997 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-31bb0315-de18-4160-aaee-970f22dc59ff tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:06:40.164337 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-31bb0315-de18-4160-aaee-970f22dc59ff tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '718f6de3b1a1453d9579d421a574691e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd21231d9af34d9aa4f9e03229b49a30', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:06:40.172765 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-31bb0315-de18-4160-aaee-970f22dc59ff tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/7efe6f96-4156-45dc-b337-fb451e51fdd4" status: 200 len: 1726 microversion: 2.1 time: 0.589090 Jul 16 22:06:40.173544 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3354/6700] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:06:39 2022] GET /compute/v2.1/servers/7efe6f96-4156-45dc-b337-fb451e51fdd4 => generated 1726 bytes in 591 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:06:40.867136 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-b044cd2b-6caa-41ba-bc66-69c8f81e7586 req-5f053757-051b-427e-8292-0d26d678c129 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "7efe6f96-4156-45dc-b337-fb451e51fdd4", "tag": "8a32905f-f42d-49e2-a600-9834e8649a5e"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:06:40.875900 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-b044cd2b-6caa-41ba-bc66-69c8f81e7586 req-5f053757-051b-427e-8292-0d26d678c129 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:06:40.876352 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-b044cd2b-6caa-41ba-bc66-69c8f81e7586 req-5f053757-051b-427e-8292-0d26d678c129 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:06:40.914316 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-b044cd2b-6caa-41ba-bc66-69c8f81e7586 req-5f053757-051b-427e-8292-0d26d678c129 service nova] Creating event network-changed:8a32905f-f42d-49e2-a600-9834e8649a5e for instance 7efe6f96-4156-45dc-b337-fb451e51fdd4 on nch2-master-1 Jul 16 22:06:40.914316 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [req-b044cd2b-6caa-41ba-bc66-69c8f81e7586 req-5f053757-051b-427e-8292-0d26d678c129 service nova] Instance 7efe6f96-4156-45dc-b337-fb451e51fdd4 is migrating within the same cell, copying events to all relevant hosts: {'nch2-master-1', 'nch1-master-1'} {{(pid=138680) _get_relevant_hosts /opt/stack/nova/nova/compute/api.py:5807}} Jul 16 22:06:40.914316 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-b044cd2b-6caa-41ba-bc66-69c8f81e7586 req-5f053757-051b-427e-8292-0d26d678c129 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.044530 Jul 16 22:06:40.914316 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3347/6701] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 22:06:40 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 22:06:41.199779 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-3f2f2aa3-d435-462e-9e24-b476e4d2f58d tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:06:41.201240 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-3f2f2aa3-d435-462e-9e24-b476e4d2f58d tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: 7efe6f96-4156-45dc-b337-fb451e51fdd4] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:06:41.208937 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3f2f2aa3-d435-462e-9e24-b476e4d2f58d tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:06:41.209834 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3f2f2aa3-d435-462e-9e24-b476e4d2f58d tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:06:41.212227 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3f2f2aa3-d435-462e-9e24-b476e4d2f58d tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:06:41.212764 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3f2f2aa3-d435-462e-9e24-b476e4d2f58d tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:06:41.472782 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-3f2f2aa3-d435-462e-9e24-b476e4d2f58d tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '718f6de3b1a1453d9579d421a574691e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd21231d9af34d9aa4f9e03229b49a30', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:06:41.475238 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-3f2f2aa3-d435-462e-9e24-b476e4d2f58d tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/7efe6f96-4156-45dc-b337-fb451e51fdd4" status: 200 len: 1724 microversion: 2.1 time: 0.288930 Jul 16 22:06:41.476440 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3355/6702] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:06:41 2022] GET /compute/v2.1/servers/7efe6f96-4156-45dc-b337-fb451e51fdd4 => generated 1724 bytes in 291 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:06:42.495295 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-30ed4e90-9b2c-4a80-93bc-97397e06c8f5 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:06:42.496062 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-30ed4e90-9b2c-4a80-93bc-97397e06c8f5 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: 7efe6f96-4156-45dc-b337-fb451e51fdd4] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:06:42.502315 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-30ed4e90-9b2c-4a80-93bc-97397e06c8f5 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:06:42.502780 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-30ed4e90-9b2c-4a80-93bc-97397e06c8f5 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:06:42.503583 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-30ed4e90-9b2c-4a80-93bc-97397e06c8f5 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:06:42.504037 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-30ed4e90-9b2c-4a80-93bc-97397e06c8f5 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:06:42.639171 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-30ed4e90-9b2c-4a80-93bc-97397e06c8f5 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '718f6de3b1a1453d9579d421a574691e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd21231d9af34d9aa4f9e03229b49a30', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:06:42.647209 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-30ed4e90-9b2c-4a80-93bc-97397e06c8f5 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/7efe6f96-4156-45dc-b337-fb451e51fdd4" status: 200 len: 1724 microversion: 2.1 time: 0.157725 Jul 16 22:06:42.648264 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3348/6703] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:06:42 2022] GET /compute/v2.1/servers/7efe6f96-4156-45dc-b337-fb451e51fdd4 => generated 1724 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:06:43.665382 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-d6dd24e3-24d4-4b6c-9219-c0f26355beb8 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:06:43.666088 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-d6dd24e3-24d4-4b6c-9219-c0f26355beb8 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: 7efe6f96-4156-45dc-b337-fb451e51fdd4] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:06:43.673479 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d6dd24e3-24d4-4b6c-9219-c0f26355beb8 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:06:43.673949 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d6dd24e3-24d4-4b6c-9219-c0f26355beb8 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:06:43.674732 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d6dd24e3-24d4-4b6c-9219-c0f26355beb8 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:06:43.675223 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d6dd24e3-24d4-4b6c-9219-c0f26355beb8 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:06:43.817437 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-d6dd24e3-24d4-4b6c-9219-c0f26355beb8 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '718f6de3b1a1453d9579d421a574691e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd21231d9af34d9aa4f9e03229b49a30', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:06:43.822231 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-d6dd24e3-24d4-4b6c-9219-c0f26355beb8 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/7efe6f96-4156-45dc-b337-fb451e51fdd4" status: 200 len: 1724 microversion: 2.1 time: 0.161286 Jul 16 22:06:43.822822 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3356/6704] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:06:43 2022] GET /compute/v2.1/servers/7efe6f96-4156-45dc-b337-fb451e51fdd4 => generated 1724 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:06:44.841042 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-6b1bb7b4-198f-4f34-b600-ce9bbf31caae tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:06:44.841652 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-6b1bb7b4-198f-4f34-b600-ce9bbf31caae tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: 7efe6f96-4156-45dc-b337-fb451e51fdd4] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:06:44.848689 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6b1bb7b4-198f-4f34-b600-ce9bbf31caae tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:06:44.849141 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6b1bb7b4-198f-4f34-b600-ce9bbf31caae tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:06:44.849899 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6b1bb7b4-198f-4f34-b600-ce9bbf31caae tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:06:44.850338 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6b1bb7b4-198f-4f34-b600-ce9bbf31caae tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:06:44.988219 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-6b1bb7b4-198f-4f34-b600-ce9bbf31caae tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '718f6de3b1a1453d9579d421a574691e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd21231d9af34d9aa4f9e03229b49a30', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:06:44.996980 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-6b1bb7b4-198f-4f34-b600-ce9bbf31caae tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/7efe6f96-4156-45dc-b337-fb451e51fdd4" status: 200 len: 1724 microversion: 2.1 time: 0.159820 Jul 16 22:06:44.997563 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3349/6705] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:06:44 2022] GET /compute/v2.1/servers/7efe6f96-4156-45dc-b337-fb451e51fdd4 => generated 1724 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 22:06:46.013850 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-74aab971-2ed5-49f6-8b62-7e2e5415ca53 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:06:46.014472 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-74aab971-2ed5-49f6-8b62-7e2e5415ca53 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: 7efe6f96-4156-45dc-b337-fb451e51fdd4] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:06:46.021590 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-74aab971-2ed5-49f6-8b62-7e2e5415ca53 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:06:46.022051 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-74aab971-2ed5-49f6-8b62-7e2e5415ca53 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:06:46.022830 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-74aab971-2ed5-49f6-8b62-7e2e5415ca53 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:06:46.023305 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-74aab971-2ed5-49f6-8b62-7e2e5415ca53 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:06:46.162593 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-74aab971-2ed5-49f6-8b62-7e2e5415ca53 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '718f6de3b1a1453d9579d421a574691e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd21231d9af34d9aa4f9e03229b49a30', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:06:46.171538 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-74aab971-2ed5-49f6-8b62-7e2e5415ca53 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/7efe6f96-4156-45dc-b337-fb451e51fdd4" status: 200 len: 1724 microversion: 2.1 time: 0.161629 Jul 16 22:06:46.172133 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3357/6706] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:06:46 2022] GET /compute/v2.1/servers/7efe6f96-4156-45dc-b337-fb451e51fdd4 => generated 1724 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:06:47.189583 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-115539d7-d853-4e47-9b26-9f9b8bedea36 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:06:47.190207 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-115539d7-d853-4e47-9b26-9f9b8bedea36 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: 7efe6f96-4156-45dc-b337-fb451e51fdd4] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:06:47.193222 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_api:SELECT=31 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:06:47.198928 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-115539d7-d853-4e47-9b26-9f9b8bedea36 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:06:47.199417 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-115539d7-d853-4e47-9b26-9f9b8bedea36 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:06:47.200198 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-115539d7-d853-4e47-9b26-9f9b8bedea36 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:06:47.200642 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-115539d7-d853-4e47-9b26-9f9b8bedea36 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:06:47.203646 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=81,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:06:47.337231 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-115539d7-d853-4e47-9b26-9f9b8bedea36 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '718f6de3b1a1453d9579d421a574691e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd21231d9af34d9aa4f9e03229b49a30', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:06:47.345143 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-115539d7-d853-4e47-9b26-9f9b8bedea36 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/7efe6f96-4156-45dc-b337-fb451e51fdd4" status: 200 len: 1721 microversion: 2.1 time: 0.159581 Jul 16 22:06:47.345717 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3350/6707] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:06:47 2022] GET /compute/v2.1/servers/7efe6f96-4156-45dc-b337-fb451e51fdd4 => generated 1721 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:06:47.361643 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-f7a0ed9b-2b50-4ced-9b4a-0d42f265d692 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Action: 'action', calling method: >, body: {"revertResize": {}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:06:47.362202 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-f7a0ed9b-2b50-4ced-9b4a-0d42f265d692 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: 7efe6f96-4156-45dc-b337-fb451e51fdd4] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:06:47.364909 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_api:SELECT=27 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:06:47.370432 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f7a0ed9b-2b50-4ced-9b4a-0d42f265d692 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:06:47.373298 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f7a0ed9b-2b50-4ced-9b4a-0d42f265d692 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:06:47.373298 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f7a0ed9b-2b50-4ced-9b4a-0d42f265d692 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:06:47.373298 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f7a0ed9b-2b50-4ced-9b4a-0d42f265d692 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:06:47.376208 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=84,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:06:47.409730 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f7a0ed9b-2b50-4ced-9b4a-0d42f265d692 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:06:47.410218 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f7a0ed9b-2b50-4ced-9b4a-0d42f265d692 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:06:47.411082 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f7a0ed9b-2b50-4ced-9b4a-0d42f265d692 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:06:47.411540 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f7a0ed9b-2b50-4ced-9b4a-0d42f265d692 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:06:47.433443 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-f7a0ed9b-2b50-4ced-9b4a-0d42f265d692 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Getting quotas for project fd21231d9af34d9aa4f9e03229b49a30. Resources: {'ram', 'cores'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 22:06:47.438663 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-f7a0ed9b-2b50-4ced-9b4a-0d42f265d692 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Getting quotas for user 718f6de3b1a1453d9579d421a574691e and project fd21231d9af34d9aa4f9e03229b49a30. Resources: {'ram', 'cores'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 22:06:47.811224 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-f7a0ed9b-2b50-4ced-9b4a-0d42f265d692 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "POST /compute/v2.1/servers/7efe6f96-4156-45dc-b337-fb451e51fdd4/action" status: 202 len: 0 microversion: 2.1 time: 0.453344 Jul 16 22:06:47.811842 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3358/6708] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 22:06:47 2022] POST /compute/v2.1/servers/7efe6f96-4156-45dc-b337-fb451e51fdd4/action => generated 0 bytes in 455 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 22:06:47.820270 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-002ad15e-5095-49a0-a2ad-c2b2f25a9fda tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:06:47.820967 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-002ad15e-5095-49a0-a2ad-c2b2f25a9fda tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: 7efe6f96-4156-45dc-b337-fb451e51fdd4] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:06:47.828500 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-002ad15e-5095-49a0-a2ad-c2b2f25a9fda tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:06:47.828962 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-002ad15e-5095-49a0-a2ad-c2b2f25a9fda tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:06:47.829752 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-002ad15e-5095-49a0-a2ad-c2b2f25a9fda tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:06:47.830206 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-002ad15e-5095-49a0-a2ad-c2b2f25a9fda tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:06:48.030038 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-002ad15e-5095-49a0-a2ad-c2b2f25a9fda tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '718f6de3b1a1453d9579d421a574691e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd21231d9af34d9aa4f9e03229b49a30', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:06:48.038148 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-002ad15e-5095-49a0-a2ad-c2b2f25a9fda tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/7efe6f96-4156-45dc-b337-fb451e51fdd4" status: 200 len: 1720 microversion: 2.1 time: 0.222055 Jul 16 22:06:48.038915 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3351/6709] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:06:47 2022] GET /compute/v2.1/servers/7efe6f96-4156-45dc-b337-fb451e51fdd4 => generated 1720 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:06:48.653300 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-e68ddcde-58c9-41ed-8bf7-47f54effc1fc req-4ddaf211-1b6b-4fc7-9d3c-58de89d335ee service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "7efe6f96-4156-45dc-b337-fb451e51fdd4", "name": "network-vif-plugged", "status": "completed", "tag": "8a32905f-f42d-49e2-a600-9834e8649a5e"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:06:48.662945 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-e68ddcde-58c9-41ed-8bf7-47f54effc1fc req-4ddaf211-1b6b-4fc7-9d3c-58de89d335ee service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:06:48.664773 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-e68ddcde-58c9-41ed-8bf7-47f54effc1fc req-4ddaf211-1b6b-4fc7-9d3c-58de89d335ee service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:06:48.676254 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-e68ddcde-58c9-41ed-8bf7-47f54effc1fc req-4ddaf211-1b6b-4fc7-9d3c-58de89d335ee service nova] Creating event network-vif-plugged:8a32905f-f42d-49e2-a600-9834e8649a5e for instance 7efe6f96-4156-45dc-b337-fb451e51fdd4 on nch2-master-1 Jul 16 22:06:48.683715 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [req-e68ddcde-58c9-41ed-8bf7-47f54effc1fc req-4ddaf211-1b6b-4fc7-9d3c-58de89d335ee service nova] Instance 7efe6f96-4156-45dc-b337-fb451e51fdd4 is migrating within the same cell, copying events to all relevant hosts: {'nch2-master-1', 'nch1-master-1'} {{(pid=138681) _get_relevant_hosts /opt/stack/nova/nova/compute/api.py:5807}} Jul 16 22:06:48.696039 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-e68ddcde-58c9-41ed-8bf7-47f54effc1fc req-4ddaf211-1b6b-4fc7-9d3c-58de89d335ee service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.103671 Jul 16 22:06:48.699213 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3359/6710] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 22:06:48 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 22:06:49.056082 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-1e0f65a9-d430-4a47-9bda-b6f58c561c81 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:06:49.056700 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-1e0f65a9-d430-4a47-9bda-b6f58c561c81 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: 7efe6f96-4156-45dc-b337-fb451e51fdd4] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:06:49.063700 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1e0f65a9-d430-4a47-9bda-b6f58c561c81 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:06:49.064154 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1e0f65a9-d430-4a47-9bda-b6f58c561c81 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:06:49.064915 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1e0f65a9-d430-4a47-9bda-b6f58c561c81 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:06:49.065358 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1e0f65a9-d430-4a47-9bda-b6f58c561c81 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:06:49.213025 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-1e0f65a9-d430-4a47-9bda-b6f58c561c81 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '718f6de3b1a1453d9579d421a574691e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd21231d9af34d9aa4f9e03229b49a30', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:06:49.221062 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-1e0f65a9-d430-4a47-9bda-b6f58c561c81 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/7efe6f96-4156-45dc-b337-fb451e51fdd4" status: 200 len: 1720 microversion: 2.1 time: 0.169849 Jul 16 22:06:49.221638 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3352/6711] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:06:49 2022] GET /compute/v2.1/servers/7efe6f96-4156-45dc-b337-fb451e51fdd4 => generated 1720 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:06:50.240538 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-26bbef12-408f-4469-b1c8-0b02cf1d5e65 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:06:50.241171 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-26bbef12-408f-4469-b1c8-0b02cf1d5e65 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: 7efe6f96-4156-45dc-b337-fb451e51fdd4] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:06:50.247769 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-26bbef12-408f-4469-b1c8-0b02cf1d5e65 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:06:50.248235 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-26bbef12-408f-4469-b1c8-0b02cf1d5e65 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:06:50.248999 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-26bbef12-408f-4469-b1c8-0b02cf1d5e65 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:06:50.249448 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-26bbef12-408f-4469-b1c8-0b02cf1d5e65 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:06:50.384393 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-26bbef12-408f-4469-b1c8-0b02cf1d5e65 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '718f6de3b1a1453d9579d421a574691e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd21231d9af34d9aa4f9e03229b49a30', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:06:50.392921 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-26bbef12-408f-4469-b1c8-0b02cf1d5e65 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/7efe6f96-4156-45dc-b337-fb451e51fdd4" status: 200 len: 1720 microversion: 2.1 time: 0.157376 Jul 16 22:06:50.393496 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3360/6712] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:06:50 2022] GET /compute/v2.1/servers/7efe6f96-4156-45dc-b337-fb451e51fdd4 => generated 1720 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:06:51.410659 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-ac10a438-616d-446f-8801-b7bc52ee123c tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:06:51.411323 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-ac10a438-616d-446f-8801-b7bc52ee123c tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: 7efe6f96-4156-45dc-b337-fb451e51fdd4] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:06:51.419001 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ac10a438-616d-446f-8801-b7bc52ee123c tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:06:51.419497 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ac10a438-616d-446f-8801-b7bc52ee123c tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:06:51.420300 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ac10a438-616d-446f-8801-b7bc52ee123c tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:06:51.420750 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ac10a438-616d-446f-8801-b7bc52ee123c tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:06:51.557258 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-ac10a438-616d-446f-8801-b7bc52ee123c tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '718f6de3b1a1453d9579d421a574691e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd21231d9af34d9aa4f9e03229b49a30', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:06:51.565655 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-ac10a438-616d-446f-8801-b7bc52ee123c tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/7efe6f96-4156-45dc-b337-fb451e51fdd4" status: 200 len: 1720 microversion: 2.1 time: 0.158831 Jul 16 22:06:51.572733 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3353/6713] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:06:51 2022] GET /compute/v2.1/servers/7efe6f96-4156-45dc-b337-fb451e51fdd4 => generated 1720 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:06:52.573356 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-ec604851-b02c-4308-b3e3-9285ae2f4864 req-60a29733-b097-4bfa-a87b-fec5359f821f service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "7efe6f96-4156-45dc-b337-fb451e51fdd4", "name": "network-vif-unplugged", "status": "completed", "tag": "8a32905f-f42d-49e2-a600-9834e8649a5e"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:06:52.582532 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-b5ff3d22-429a-4ca9-b048-4615030d965a tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:06:52.583238 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-ec604851-b02c-4308-b3e3-9285ae2f4864 req-60a29733-b097-4bfa-a87b-fec5359f821f service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:06:52.583379 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-b5ff3d22-429a-4ca9-b048-4615030d965a tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: 7efe6f96-4156-45dc-b337-fb451e51fdd4] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:06:52.583706 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-ec604851-b02c-4308-b3e3-9285ae2f4864 req-60a29733-b097-4bfa-a87b-fec5359f821f service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:06:52.590778 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b5ff3d22-429a-4ca9-b048-4615030d965a tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:06:52.591253 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b5ff3d22-429a-4ca9-b048-4615030d965a tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:06:52.592026 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b5ff3d22-429a-4ca9-b048-4615030d965a tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:06:52.600395 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b5ff3d22-429a-4ca9-b048-4615030d965a tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:06:52.600517 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-ec604851-b02c-4308-b3e3-9285ae2f4864 req-60a29733-b097-4bfa-a87b-fec5359f821f service nova] Creating event network-vif-unplugged:8a32905f-f42d-49e2-a600-9834e8649a5e for instance 7efe6f96-4156-45dc-b337-fb451e51fdd4 on nch1-master-1 Jul 16 22:06:52.603548 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [req-ec604851-b02c-4308-b3e3-9285ae2f4864 req-60a29733-b097-4bfa-a87b-fec5359f821f service nova] Instance 7efe6f96-4156-45dc-b337-fb451e51fdd4 is migrating within the same cell, copying events to all relevant hosts: {'nch2-master-1', 'nch1-master-1'} {{(pid=138681) _get_relevant_hosts /opt/stack/nova/nova/compute/api.py:5807}} Jul 16 22:06:52.614445 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-ec604851-b02c-4308-b3e3-9285ae2f4864 req-60a29733-b097-4bfa-a87b-fec5359f821f service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.045235 Jul 16 22:06:52.615372 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3361/6714] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 22:06:52 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 22:06:52.734194 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-b5ff3d22-429a-4ca9-b048-4615030d965a tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '718f6de3b1a1453d9579d421a574691e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd21231d9af34d9aa4f9e03229b49a30', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:06:52.742522 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-b5ff3d22-429a-4ca9-b048-4615030d965a tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/7efe6f96-4156-45dc-b337-fb451e51fdd4" status: 200 len: 1720 microversion: 2.1 time: 0.163627 Jul 16 22:06:52.743618 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3354/6715] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:06:52 2022] GET /compute/v2.1/servers/7efe6f96-4156-45dc-b337-fb451e51fdd4 => generated 1720 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 22:06:53.760759 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-99143079-1945-463f-b7f8-879ed1aba2e0 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:06:53.761481 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-99143079-1945-463f-b7f8-879ed1aba2e0 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: 7efe6f96-4156-45dc-b337-fb451e51fdd4] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:06:53.768887 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-99143079-1945-463f-b7f8-879ed1aba2e0 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:06:53.778415 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-99143079-1945-463f-b7f8-879ed1aba2e0 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:06:53.778415 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-99143079-1945-463f-b7f8-879ed1aba2e0 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:06:53.778415 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-99143079-1945-463f-b7f8-879ed1aba2e0 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:06:53.932893 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-99143079-1945-463f-b7f8-879ed1aba2e0 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '718f6de3b1a1453d9579d421a574691e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd21231d9af34d9aa4f9e03229b49a30', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:06:53.941691 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-99143079-1945-463f-b7f8-879ed1aba2e0 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/7efe6f96-4156-45dc-b337-fb451e51fdd4" status: 200 len: 1720 microversion: 2.1 time: 0.185328 Jul 16 22:06:53.942281 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3362/6716] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:06:53 2022] GET /compute/v2.1/servers/7efe6f96-4156-45dc-b337-fb451e51fdd4 => generated 1720 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:06:54.629963 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-6a67ff63-1b55-4476-9f5b-2c76d662d5cf req-0b523d69-1704-4709-834b-94dc8ca7d10b service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "7efe6f96-4156-45dc-b337-fb451e51fdd4", "tag": "8a32905f-f42d-49e2-a600-9834e8649a5e"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:06:54.639744 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-6a67ff63-1b55-4476-9f5b-2c76d662d5cf req-0b523d69-1704-4709-834b-94dc8ca7d10b service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:06:54.640192 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-6a67ff63-1b55-4476-9f5b-2c76d662d5cf req-0b523d69-1704-4709-834b-94dc8ca7d10b service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:06:54.652364 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-6a67ff63-1b55-4476-9f5b-2c76d662d5cf req-0b523d69-1704-4709-834b-94dc8ca7d10b service nova] Creating event network-changed:8a32905f-f42d-49e2-a600-9834e8649a5e for instance 7efe6f96-4156-45dc-b337-fb451e51fdd4 on nch1-master-1 Jul 16 22:06:54.657668 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [req-6a67ff63-1b55-4476-9f5b-2c76d662d5cf req-0b523d69-1704-4709-834b-94dc8ca7d10b service nova] Instance 7efe6f96-4156-45dc-b337-fb451e51fdd4 is migrating within the same cell, copying events to all relevant hosts: {'nch2-master-1', 'nch1-master-1'} {{(pid=138680) _get_relevant_hosts /opt/stack/nova/nova/compute/api.py:5807}} Jul 16 22:06:54.669118 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-6a67ff63-1b55-4476-9f5b-2c76d662d5cf req-0b523d69-1704-4709-834b-94dc8ca7d10b service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.042852 Jul 16 22:06:54.669770 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3355/6717] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 22:06:54 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 22:06:54.960470 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-7195979a-1186-451a-a178-9baa9e03b52f tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:06:54.961165 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-7195979a-1186-451a-a178-9baa9e03b52f tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: 7efe6f96-4156-45dc-b337-fb451e51fdd4] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:06:54.969325 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7195979a-1186-451a-a178-9baa9e03b52f tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:06:54.970110 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7195979a-1186-451a-a178-9baa9e03b52f tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:06:54.971144 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7195979a-1186-451a-a178-9baa9e03b52f tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:06:54.971795 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7195979a-1186-451a-a178-9baa9e03b52f tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:06:55.168155 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-7195979a-1186-451a-a178-9baa9e03b52f tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '718f6de3b1a1453d9579d421a574691e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd21231d9af34d9aa4f9e03229b49a30', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:06:55.185447 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-7195979a-1186-451a-a178-9baa9e03b52f tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/7efe6f96-4156-45dc-b337-fb451e51fdd4" status: 200 len: 1720 microversion: 2.1 time: 0.228924 Jul 16 22:06:55.185447 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3363/6718] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:06:54 2022] GET /compute/v2.1/servers/7efe6f96-4156-45dc-b337-fb451e51fdd4 => generated 1720 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:06:56.204085 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-0fc99526-0bfb-487f-af6c-e0e0c4ede102 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:06:56.204762 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-0fc99526-0bfb-487f-af6c-e0e0c4ede102 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: 7efe6f96-4156-45dc-b337-fb451e51fdd4] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:06:56.212846 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0fc99526-0bfb-487f-af6c-e0e0c4ede102 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:06:56.213301 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0fc99526-0bfb-487f-af6c-e0e0c4ede102 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:06:56.214073 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0fc99526-0bfb-487f-af6c-e0e0c4ede102 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:06:56.214554 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0fc99526-0bfb-487f-af6c-e0e0c4ede102 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:06:56.350131 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-0fc99526-0bfb-487f-af6c-e0e0c4ede102 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '718f6de3b1a1453d9579d421a574691e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd21231d9af34d9aa4f9e03229b49a30', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:06:56.360223 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-0fc99526-0bfb-487f-af6c-e0e0c4ede102 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/7efe6f96-4156-45dc-b337-fb451e51fdd4" status: 200 len: 1720 microversion: 2.1 time: 0.161968 Jul 16 22:06:56.363074 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3356/6719] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:06:56 2022] GET /compute/v2.1/servers/7efe6f96-4156-45dc-b337-fb451e51fdd4 => generated 1720 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:06:57.378186 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-ed704ca6-d2a4-4c9e-bdcd-37ea6bd1b2e7 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:06:57.378819 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-ed704ca6-d2a4-4c9e-bdcd-37ea6bd1b2e7 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: 7efe6f96-4156-45dc-b337-fb451e51fdd4] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:06:57.385946 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ed704ca6-d2a4-4c9e-bdcd-37ea6bd1b2e7 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:06:57.386415 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ed704ca6-d2a4-4c9e-bdcd-37ea6bd1b2e7 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:06:57.387223 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ed704ca6-d2a4-4c9e-bdcd-37ea6bd1b2e7 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:06:57.387682 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ed704ca6-d2a4-4c9e-bdcd-37ea6bd1b2e7 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:06:57.424322 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell0:SELECT=3 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:06:57.527477 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-ed704ca6-d2a4-4c9e-bdcd-37ea6bd1b2e7 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '718f6de3b1a1453d9579d421a574691e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd21231d9af34d9aa4f9e03229b49a30', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:06:57.535920 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-ed704ca6-d2a4-4c9e-bdcd-37ea6bd1b2e7 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/7efe6f96-4156-45dc-b337-fb451e51fdd4" status: 200 len: 1720 microversion: 2.1 time: 0.161661 Jul 16 22:06:57.536529 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3364/6720] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:06:57 2022] GET /compute/v2.1/servers/7efe6f96-4156-45dc-b337-fb451e51fdd4 => generated 1720 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:06:58.553870 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-83952830-ae34-4051-a915-28dc97ea4ce2 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:06:58.554487 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-83952830-ae34-4051-a915-28dc97ea4ce2 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: 7efe6f96-4156-45dc-b337-fb451e51fdd4] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:06:58.562733 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-83952830-ae34-4051-a915-28dc97ea4ce2 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:06:58.563232 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-83952830-ae34-4051-a915-28dc97ea4ce2 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:06:58.565263 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-83952830-ae34-4051-a915-28dc97ea4ce2 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:06:58.565712 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-83952830-ae34-4051-a915-28dc97ea4ce2 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:06:58.707898 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-83952830-ae34-4051-a915-28dc97ea4ce2 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '718f6de3b1a1453d9579d421a574691e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd21231d9af34d9aa4f9e03229b49a30', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:06:58.718495 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-83952830-ae34-4051-a915-28dc97ea4ce2 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/7efe6f96-4156-45dc-b337-fb451e51fdd4" status: 200 len: 1713 microversion: 2.1 time: 0.167985 Jul 16 22:06:58.720048 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3357/6721] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:06:58 2022] GET /compute/v2.1/servers/7efe6f96-4156-45dc-b337-fb451e51fdd4 => generated 1713 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:06:58.737938 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-062a5253-0c70-4c68-9df2-c09f3436bb84 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:06:58.740035 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-062a5253-0c70-4c68-9df2-c09f3436bb84 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: 7efe6f96-4156-45dc-b337-fb451e51fdd4] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:06:58.747031 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-062a5253-0c70-4c68-9df2-c09f3436bb84 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:06:58.747503 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-062a5253-0c70-4c68-9df2-c09f3436bb84 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:06:58.748281 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-062a5253-0c70-4c68-9df2-c09f3436bb84 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:06:58.748781 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-062a5253-0c70-4c68-9df2-c09f3436bb84 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:06:58.886110 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-062a5253-0c70-4c68-9df2-c09f3436bb84 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '718f6de3b1a1453d9579d421a574691e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd21231d9af34d9aa4f9e03229b49a30', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:06:58.894286 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-062a5253-0c70-4c68-9df2-c09f3436bb84 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/7efe6f96-4156-45dc-b337-fb451e51fdd4" status: 200 len: 1713 microversion: 2.1 time: 0.160817 Jul 16 22:06:58.894901 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3365/6722] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:06:58 2022] GET /compute/v2.1/servers/7efe6f96-4156-45dc-b337-fb451e51fdd4 => generated 1713 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:06:58.910583 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-aac37e90-139d-4870-a4da-9baa064cf87e tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:06:58.911240 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-aac37e90-139d-4870-a4da-9baa064cf87e tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: 7efe6f96-4156-45dc-b337-fb451e51fdd4] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:06:58.917557 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-aac37e90-139d-4870-a4da-9baa064cf87e tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:06:58.918027 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-aac37e90-139d-4870-a4da-9baa064cf87e tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:06:58.918799 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-aac37e90-139d-4870-a4da-9baa064cf87e tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:06:58.919274 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-aac37e90-139d-4870-a4da-9baa064cf87e tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:06:59.060910 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-aac37e90-139d-4870-a4da-9baa064cf87e tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '718f6de3b1a1453d9579d421a574691e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd21231d9af34d9aa4f9e03229b49a30', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:06:59.069385 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-aac37e90-139d-4870-a4da-9baa064cf87e tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/7efe6f96-4156-45dc-b337-fb451e51fdd4" status: 200 len: 1713 microversion: 2.1 time: 0.162555 Jul 16 22:06:59.069983 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3358/6723] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:06:58 2022] GET /compute/v2.1/servers/7efe6f96-4156-45dc-b337-fb451e51fdd4 => generated 1713 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:06:59.087899 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-828b5fda-5944-4ed4-8589-a30f9576f19c tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:06:59.088627 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-828b5fda-5944-4ed4-8589-a30f9576f19c tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: 7efe6f96-4156-45dc-b337-fb451e51fdd4] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:06:59.095460 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-828b5fda-5944-4ed4-8589-a30f9576f19c tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:06:59.095927 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-828b5fda-5944-4ed4-8589-a30f9576f19c tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:06:59.096702 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-828b5fda-5944-4ed4-8589-a30f9576f19c tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:06:59.097152 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-828b5fda-5944-4ed4-8589-a30f9576f19c tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:06:59.123527 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-828b5fda-5944-4ed4-8589-a30f9576f19c tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: 7efe6f96-4156-45dc-b337-fb451e51fdd4] Going to try to terminate instance {{(pid=138681) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 16 22:06:59.379983 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-828b5fda-5944-4ed4-8589-a30f9576f19c tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "DELETE /compute/v2.1/servers/7efe6f96-4156-45dc-b337-fb451e51fdd4" status: 204 len: 0 microversion: 2.1 time: 0.296589 Jul 16 22:06:59.380593 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3366/6724] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 22:06:59 2022] DELETE /compute/v2.1/servers/7efe6f96-4156-45dc-b337-fb451e51fdd4 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 16 22:06:59.390052 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-b3297aa3-4d30-4c80-a94e-1b43373f6cec tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:06:59.390791 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-b3297aa3-4d30-4c80-a94e-1b43373f6cec tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: 7efe6f96-4156-45dc-b337-fb451e51fdd4] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:06:59.397347 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b3297aa3-4d30-4c80-a94e-1b43373f6cec tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:06:59.397812 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b3297aa3-4d30-4c80-a94e-1b43373f6cec tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:06:59.399084 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b3297aa3-4d30-4c80-a94e-1b43373f6cec tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:06:59.399629 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b3297aa3-4d30-4c80-a94e-1b43373f6cec tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:06:59.529653 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-b3297aa3-4d30-4c80-a94e-1b43373f6cec tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '718f6de3b1a1453d9579d421a574691e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd21231d9af34d9aa4f9e03229b49a30', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:06:59.537941 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-b3297aa3-4d30-4c80-a94e-1b43373f6cec tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/7efe6f96-4156-45dc-b337-fb451e51fdd4" status: 200 len: 1719 microversion: 2.1 time: 0.152192 Jul 16 22:06:59.538526 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3359/6725] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:06:59 2022] GET /compute/v2.1/servers/7efe6f96-4156-45dc-b337-fb451e51fdd4 => generated 1719 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:07:00.557098 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-5f9c623b-02bd-42e4-afb3-0e14a30d1f0f tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:07:00.559097 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-5f9c623b-02bd-42e4-afb3-0e14a30d1f0f tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: 7efe6f96-4156-45dc-b337-fb451e51fdd4] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:07:00.565377 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5f9c623b-02bd-42e4-afb3-0e14a30d1f0f tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:07:00.565850 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5f9c623b-02bd-42e4-afb3-0e14a30d1f0f tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:07:00.566647 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5f9c623b-02bd-42e4-afb3-0e14a30d1f0f tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:07:00.567186 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5f9c623b-02bd-42e4-afb3-0e14a30d1f0f tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:07:00.705704 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-5f9c623b-02bd-42e4-afb3-0e14a30d1f0f tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '718f6de3b1a1453d9579d421a574691e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd21231d9af34d9aa4f9e03229b49a30', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:07:00.715310 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-5f9c623b-02bd-42e4-afb3-0e14a30d1f0f tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/7efe6f96-4156-45dc-b337-fb451e51fdd4" status: 200 len: 1719 microversion: 2.1 time: 0.163152 Jul 16 22:07:00.715902 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3367/6726] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:07:00 2022] GET /compute/v2.1/servers/7efe6f96-4156-45dc-b337-fb451e51fdd4 => generated 1719 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:07:00.747586 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-864eeec8-e27b-4b0f-b263-41473642b7d0 req-0f8b01b5-140e-46e2-82f8-95a3a134d173 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "7efe6f96-4156-45dc-b337-fb451e51fdd4", "name": "network-vif-plugged", "status": "completed", "tag": "8a32905f-f42d-49e2-a600-9834e8649a5e"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:07:00.756731 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-864eeec8-e27b-4b0f-b263-41473642b7d0 req-0f8b01b5-140e-46e2-82f8-95a3a134d173 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:07:00.757180 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-864eeec8-e27b-4b0f-b263-41473642b7d0 req-0f8b01b5-140e-46e2-82f8-95a3a134d173 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:07:00.768950 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-864eeec8-e27b-4b0f-b263-41473642b7d0 req-0f8b01b5-140e-46e2-82f8-95a3a134d173 service nova] Creating event network-vif-plugged:8a32905f-f42d-49e2-a600-9834e8649a5e for instance 7efe6f96-4156-45dc-b337-fb451e51fdd4 on nch1-master-1 Jul 16 22:07:00.774994 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-864eeec8-e27b-4b0f-b263-41473642b7d0 req-0f8b01b5-140e-46e2-82f8-95a3a134d173 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.030804 Jul 16 22:07:00.776895 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3360/6727] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 22:07:00 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 16 22:07:01.733090 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-91e958ba-5c22-4f22-96a4-a6808d9ba838 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:07:01.734438 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-91e958ba-5c22-4f22-96a4-a6808d9ba838 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: 7efe6f96-4156-45dc-b337-fb451e51fdd4] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:07:01.741852 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-91e958ba-5c22-4f22-96a4-a6808d9ba838 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:07:01.742705 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-91e958ba-5c22-4f22-96a4-a6808d9ba838 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:07:01.744136 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-91e958ba-5c22-4f22-96a4-a6808d9ba838 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:07:01.744136 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-91e958ba-5c22-4f22-96a4-a6808d9ba838 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:07:01.811328 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-91e958ba-5c22-4f22-96a4-a6808d9ba838 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '718f6de3b1a1453d9579d421a574691e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd21231d9af34d9aa4f9e03229b49a30', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:07:01.819670 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-91e958ba-5c22-4f22-96a4-a6808d9ba838 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/7efe6f96-4156-45dc-b337-fb451e51fdd4" status: 200 len: 1652 microversion: 2.1 time: 0.090532 Jul 16 22:07:01.830054 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3368/6728] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:07:01 2022] GET /compute/v2.1/servers/7efe6f96-4156-45dc-b337-fb451e51fdd4 => generated 1652 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:07:01.878622 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-1b69bc6c-b323-492b-b88f-aec8f23c2ae8 req-f26b9d7f-1774-4b35-b855-cc4acf705c39 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "7efe6f96-4156-45dc-b337-fb451e51fdd4", "name": "network-vif-deleted", "tag": "8a32905f-f42d-49e2-a600-9834e8649a5e"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:07:01.886088 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-1b69bc6c-b323-492b-b88f-aec8f23c2ae8 req-f26b9d7f-1774-4b35-b855-cc4acf705c39 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:07:01.886541 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-1b69bc6c-b323-492b-b88f-aec8f23c2ae8 req-f26b9d7f-1774-4b35-b855-cc4acf705c39 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:07:01.897484 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-1b69bc6c-b323-492b-b88f-aec8f23c2ae8 req-f26b9d7f-1774-4b35-b855-cc4acf705c39 service nova] Creating event network-vif-deleted:8a32905f-f42d-49e2-a600-9834e8649a5e for instance 7efe6f96-4156-45dc-b337-fb451e51fdd4 on nch1-master-1 Jul 16 22:07:01.905260 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-1b69bc6c-b323-492b-b88f-aec8f23c2ae8 req-f26b9d7f-1774-4b35-b855-cc4acf705c39 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.033747 Jul 16 22:07:01.905881 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3361/6729] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 22:07:01 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 22:07:02.838782 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-4a11a021-41de-4f80-a5a6-6e3e91863d90 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:07:02.839464 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-4a11a021-41de-4f80-a5a6-6e3e91863d90 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: 7efe6f96-4156-45dc-b337-fb451e51fdd4] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:07:02.846721 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4a11a021-41de-4f80-a5a6-6e3e91863d90 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:07:02.847240 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4a11a021-41de-4f80-a5a6-6e3e91863d90 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:07:02.848034 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4a11a021-41de-4f80-a5a6-6e3e91863d90 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:07:02.848490 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4a11a021-41de-4f80-a5a6-6e3e91863d90 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:07:02.862992 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-4a11a021-41de-4f80-a5a6-6e3e91863d90 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] HTTP exception thrown: Instance 7efe6f96-4156-45dc-b337-fb451e51fdd4 could not be found. Jul 16 22:07:02.864712 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-4a11a021-41de-4f80-a5a6-6e3e91863d90 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Request method failure captured: Jul 16 22:07:02.864712 ncdv-master-1 devstack@n-api.service[138681]: request: GET /compute/v2.1/servers/7efe6f96-4156-45dc-b337-fb451e51fdd4 HTTP/1.1 Jul 16 22:07:02.864712 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 22:07:02.864712 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 22:07:02.864712 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 22:07:02.864712 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 22:07:02.864712 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 22:07:02.864712 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 22:07:02.864712 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 22:07:02.864712 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 22:07:02.864712 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0zUsISRuFerqPWLnIU6pWtJZ1G33kS5UQBpOE-Qn9W1zRIsDO0obaxdGmXyhe5FNe11pjr8YSFwmEAtO4tE-QrUwHNB6R_8ZgLrv3bb_zQTEpcOValb2w1FWdJ1eIHV_96nHchz9PmXAJpOM5Cp_lLo9a71M8MQcZrQgoRzDb41oZDA Jul 16 22:07:02.864712 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 22:07:02.864712 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 22:07:02.864712 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 22:07:02.864712 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 22:07:02.864712 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 22:07:02.864712 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 22:07:02.864712 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: fd21231d9af34d9aa4f9e03229b49a30 Jul 16 22:07:02.864712 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-ServerActionsTestJSON-164809533 Jul 16 22:07:02.864712 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 22:07:02.864712 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 22:07:02.864712 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/fd21231d9af34d9aa4f9e03229b49a30"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/fd21231d9af34d9aa4f9e03229b49a30"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_fd21231d9af34d9aa4f9e03229b49a30"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/fd21231d9af34d9aa4f9e03229b49a30"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 22:07:02.864712 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-ServerActionsTestJSON-164809533 Jul 16 22:07:02.864712 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: fd21231d9af34d9aa4f9e03229b49a30 Jul 16 22:07:02.864712 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-ServerActionsTestJSON-164809533 Jul 16 22:07:02.864712 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-ServerActionsTestJSON-164809533-project Jul 16 22:07:02.864712 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 22:07:02.864712 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 22:07:02.864712 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: 718f6de3b1a1453d9579d421a574691e Jul 16 22:07:02.864712 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-ServerActionsTestJSON-164809533-project Jul 16 22:07:02.864712 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 22:07:02.864712 ncdv-master-1 devstack@n-api.service[138681]: exception: Instance 7efe6f96-4156-45dc-b337-fb451e51fdd4 could not be found. Jul 16 22:07:02.864712 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': '7efe6f96-4156-45dc-b337-fb451e51fdd4'} Jul 16 22:07:02.864712 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 22:07:02.864712 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 22:07:02.864712 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 22:07:02.864712 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 22:07:02.864712 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 22:07:02.864712 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 22:07:02.864712 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 22:07:02.864712 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 22:07:02.864712 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 22:07:02.864712 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 22:07:02.864712 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 7efe6f96-4156-45dc-b337-fb451e51fdd4 could not be found. Jul 16 22:07:02.864712 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 22:07:02.864712 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 22:07:02.864712 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 22:07:02.864712 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 22:07:02.864712 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 22:07:02.864712 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 22:07:02.864712 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 22:07:02.864712 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 22:07:02.864712 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 22:07:02.864712 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 22:07:02.864712 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 22:07:02.864712 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 22:07:02.864712 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 22:07:02.864712 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 22:07:02.864712 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 22:07:02.864712 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 22:07:02.864712 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 7efe6f96-4156-45dc-b337-fb451e51fdd4 could not be found. Jul 16 22:07:02.864712 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 22:07:02.864712 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 22:07:02.864712 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 22:07:02.872014 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 22:07:02.872014 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 22:07:02.872014 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 22:07:02.872014 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 22:07:02.872014 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 22:07:02.872014 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 7efe6f96-4156-45dc-b337-fb451e51fdd4 could not be found. Jul 16 22:07:02.872014 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 22:07:02.872014 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-4a11a021-41de-4f80-a5a6-6e3e91863d90 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Returning 404 to user: Instance 7efe6f96-4156-45dc-b337-fb451e51fdd4 could not be found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 22:07:02.872014 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-4a11a021-41de-4f80-a5a6-6e3e91863d90 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/7efe6f96-4156-45dc-b337-fb451e51fdd4" status: 404 len: 111 microversion: 2.1 time: 0.032251 Jul 16 22:07:02.872014 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3369/6730] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:07:02 2022] GET /compute/v2.1/servers/7efe6f96-4156-45dc-b337-fb451e51fdd4 => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 22:07:02.926587 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-eab2e315-0e6c-4c96-8963-5f9f670a5c8c tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:07:02.928332 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-eab2e315-0e6c-4c96-8963-5f9f670a5c8c tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: 7efe6f96-4156-45dc-b337-fb451e51fdd4] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:07:02.935343 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-eab2e315-0e6c-4c96-8963-5f9f670a5c8c tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:07:02.935827 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-eab2e315-0e6c-4c96-8963-5f9f670a5c8c tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:07:02.936922 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-eab2e315-0e6c-4c96-8963-5f9f670a5c8c tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:07:02.937369 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-eab2e315-0e6c-4c96-8963-5f9f670a5c8c tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:07:02.952024 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-eab2e315-0e6c-4c96-8963-5f9f670a5c8c tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] HTTP exception thrown: Instance 7efe6f96-4156-45dc-b337-fb451e51fdd4 could not be found. Jul 16 22:07:02.955861 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-eab2e315-0e6c-4c96-8963-5f9f670a5c8c tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Request method failure captured: Jul 16 22:07:02.955861 ncdv-master-1 devstack@n-api.service[138680]: request: GET /compute/v2.1/servers/7efe6f96-4156-45dc-b337-fb451e51fdd4/os-volume_attachments HTTP/1.1 Jul 16 22:07:02.955861 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 22:07:02.955861 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 22:07:02.955861 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 22:07:02.955861 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 0 Jul 16 22:07:02.955861 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 22:07:02.955861 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 22:07:02.955861 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 22:07:02.955861 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 22:07:02.955861 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0zUsISRuFerqPWLnIU6pWtJZ1G33kS5UQBpOE-Qn9W1zRIsDO0obaxdGmXyhe5FNe11pjr8YSFwmEAtO4tE-QrUwHNB6R_8ZgLrv3bb_zQTEpcOValb2w1FWdJ1eIHV_96nHchz9PmXAJpOM5Cp_lLo9a71M8MQcZrQgoRzDb41oZDA Jul 16 22:07:02.955861 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 22:07:02.955861 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 22:07:02.955861 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 22:07:02.955861 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 22:07:02.955861 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 22:07:02.955861 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 22:07:02.955861 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: fd21231d9af34d9aa4f9e03229b49a30 Jul 16 22:07:02.955861 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-ServerActionsTestJSON-164809533 Jul 16 22:07:02.955861 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 22:07:02.955861 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 22:07:02.955861 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/fd21231d9af34d9aa4f9e03229b49a30"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/fd21231d9af34d9aa4f9e03229b49a30"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_fd21231d9af34d9aa4f9e03229b49a30"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/fd21231d9af34d9aa4f9e03229b49a30"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 22:07:02.955861 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-ServerActionsTestJSON-164809533 Jul 16 22:07:02.955861 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: fd21231d9af34d9aa4f9e03229b49a30 Jul 16 22:07:02.955861 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-ServerActionsTestJSON-164809533 Jul 16 22:07:02.955861 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-ServerActionsTestJSON-164809533-project Jul 16 22:07:02.955861 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 22:07:02.955861 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 22:07:02.955861 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: 718f6de3b1a1453d9579d421a574691e Jul 16 22:07:02.955861 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-ServerActionsTestJSON-164809533-project Jul 16 22:07:02.955861 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 22:07:02.955861 ncdv-master-1 devstack@n-api.service[138680]: exception: Instance 7efe6f96-4156-45dc-b337-fb451e51fdd4 could not be found. Jul 16 22:07:02.955861 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'server_id': '7efe6f96-4156-45dc-b337-fb451e51fdd4'} Jul 16 22:07:02.955861 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 22:07:02.955861 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 22:07:02.955861 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 22:07:02.955861 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 22:07:02.955861 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 22:07:02.955861 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 22:07:02.955861 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 22:07:02.955861 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 22:07:02.955861 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 22:07:02.955861 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 22:07:02.955861 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 7efe6f96-4156-45dc-b337-fb451e51fdd4 could not be found. Jul 16 22:07:02.955861 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 22:07:02.955861 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 22:07:02.955861 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 22:07:02.955861 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 22:07:02.955861 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 22:07:02.955861 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 22:07:02.955861 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 22:07:02.955861 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 22:07:02.955861 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 22:07:02.955861 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 22:07:02.955861 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper Jul 16 22:07:02.955861 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 16 22:07:02.955861 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper Jul 16 22:07:02.955861 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 16 22:07:02.955861 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 303, in index Jul 16 22:07:02.955861 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, server_id) Jul 16 22:07:02.955861 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 22:07:02.955861 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 22:07:02.955861 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 7efe6f96-4156-45dc-b337-fb451e51fdd4 could not be found. Jul 16 22:07:02.960671 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 22:07:02.960671 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 22:07:02.960671 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 22:07:02.960671 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 22:07:02.960671 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 22:07:02.960671 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 22:07:02.960671 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 22:07:02.960671 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 22:07:02.960671 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 7efe6f96-4156-45dc-b337-fb451e51fdd4 could not be found. Jul 16 22:07:02.960671 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 22:07:02.960671 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-eab2e315-0e6c-4c96-8963-5f9f670a5c8c tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Returning 404 to user: Instance 7efe6f96-4156-45dc-b337-fb451e51fdd4 could not be found. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 22:07:02.960671 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-eab2e315-0e6c-4c96-8963-5f9f670a5c8c tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/7efe6f96-4156-45dc-b337-fb451e51fdd4/os-volume_attachments" status: 404 len: 111 microversion: 2.1 time: 0.034133 Jul 16 22:07:02.960671 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3362/6731] 10.222.0.16 () {58 vars in 1301 bytes} [Sat Jul 16 22:07:02 2022] GET /compute/v2.1/servers/7efe6f96-4156-45dc-b337-fb451e51fdd4/os-volume_attachments => generated 111 bytes in 35 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 22:07:06.363349 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-99764b60-47cd-4a70-a07c-f623f4b9ee8e tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:07:06.363875 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-99764b60-47cd-4a70-a07c-f623f4b9ee8e tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: 7efe6f96-4156-45dc-b337-fb451e51fdd4] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:07:06.371081 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-99764b60-47cd-4a70-a07c-f623f4b9ee8e tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:07:06.371561 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-99764b60-47cd-4a70-a07c-f623f4b9ee8e tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:07:06.372342 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-99764b60-47cd-4a70-a07c-f623f4b9ee8e tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:07:06.372800 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-99764b60-47cd-4a70-a07c-f623f4b9ee8e tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:07:06.386292 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-99764b60-47cd-4a70-a07c-f623f4b9ee8e tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] HTTP exception thrown: Instance 7efe6f96-4156-45dc-b337-fb451e51fdd4 could not be found. Jul 16 22:07:06.387874 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-99764b60-47cd-4a70-a07c-f623f4b9ee8e tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Request method failure captured: Jul 16 22:07:06.387874 ncdv-master-1 devstack@n-api.service[138681]: request: DELETE /compute/v2.1/servers/7efe6f96-4156-45dc-b337-fb451e51fdd4 HTTP/1.1 Jul 16 22:07:06.387874 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 22:07:06.387874 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 22:07:06.387874 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 22:07:06.387874 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 22:07:06.387874 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 22:07:06.387874 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 22:07:06.387874 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 22:07:06.387874 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 22:07:06.387874 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0zUsISRuFerqPWLnIU6pWtJZ1G33kS5UQBpOE-Qn9W1zRIsDO0obaxdGmXyhe5FNe11pjr8YSFwmEAtO4tE-QrUwHNB6R_8ZgLrv3bb_zQTEpcOValb2w1FWdJ1eIHV_96nHchz9PmXAJpOM5Cp_lLo9a71M8MQcZrQgoRzDb41oZDA Jul 16 22:07:06.387874 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 22:07:06.387874 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 22:07:06.387874 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 22:07:06.387874 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 22:07:06.387874 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 22:07:06.387874 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 22:07:06.387874 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: fd21231d9af34d9aa4f9e03229b49a30 Jul 16 22:07:06.387874 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-ServerActionsTestJSON-164809533 Jul 16 22:07:06.387874 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 22:07:06.387874 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 22:07:06.387874 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/fd21231d9af34d9aa4f9e03229b49a30"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/fd21231d9af34d9aa4f9e03229b49a30"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_fd21231d9af34d9aa4f9e03229b49a30"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/fd21231d9af34d9aa4f9e03229b49a30"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 22:07:06.387874 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-ServerActionsTestJSON-164809533 Jul 16 22:07:06.387874 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: fd21231d9af34d9aa4f9e03229b49a30 Jul 16 22:07:06.387874 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-ServerActionsTestJSON-164809533 Jul 16 22:07:06.387874 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-ServerActionsTestJSON-164809533-project Jul 16 22:07:06.387874 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 22:07:06.387874 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 22:07:06.387874 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: 718f6de3b1a1453d9579d421a574691e Jul 16 22:07:06.387874 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-ServerActionsTestJSON-164809533-project Jul 16 22:07:06.387874 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 22:07:06.387874 ncdv-master-1 devstack@n-api.service[138681]: exception: Instance 7efe6f96-4156-45dc-b337-fb451e51fdd4 could not be found. Jul 16 22:07:06.387874 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': '7efe6f96-4156-45dc-b337-fb451e51fdd4'} Jul 16 22:07:06.387874 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 22:07:06.387874 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 22:07:06.387874 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 22:07:06.387874 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 22:07:06.387874 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 22:07:06.387874 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 22:07:06.387874 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 22:07:06.387874 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 22:07:06.387874 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 22:07:06.387874 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 22:07:06.387874 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 7efe6f96-4156-45dc-b337-fb451e51fdd4 could not be found. Jul 16 22:07:06.387874 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 22:07:06.387874 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 22:07:06.387874 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 22:07:06.387874 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 22:07:06.387874 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 22:07:06.387874 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 22:07:06.387874 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 22:07:06.387874 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 22:07:06.387874 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 22:07:06.387874 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 22:07:06.387874 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Jul 16 22:07:06.387874 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Jul 16 22:07:06.387874 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Jul 16 22:07:06.387874 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Jul 16 22:07:06.387874 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 22:07:06.387874 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 22:07:06.387874 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 22:07:06.387874 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 22:07:06.392203 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 7efe6f96-4156-45dc-b337-fb451e51fdd4 could not be found. Jul 16 22:07:06.392203 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 22:07:06.392203 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 22:07:06.392203 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 22:07:06.392203 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 22:07:06.392203 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 22:07:06.392203 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 22:07:06.392203 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 22:07:06.392203 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 22:07:06.392203 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 7efe6f96-4156-45dc-b337-fb451e51fdd4 could not be found. Jul 16 22:07:06.392203 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 22:07:06.392203 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-99764b60-47cd-4a70-a07c-f623f4b9ee8e tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Returning 404 to user: Instance 7efe6f96-4156-45dc-b337-fb451e51fdd4 could not be found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 22:07:06.392203 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-99764b60-47cd-4a70-a07c-f623f4b9ee8e tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "DELETE /compute/v2.1/servers/7efe6f96-4156-45dc-b337-fb451e51fdd4" status: 404 len: 111 microversion: 2.1 time: 0.030102 Jul 16 22:07:06.392203 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3370/6732] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 22:07:06 2022] DELETE /compute/v2.1/servers/7efe6f96-4156-45dc-b337-fb451e51fdd4 => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 22:07:06.397747 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-31318e12-ec62-4167-a2e8-bf4e177532a3 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:07:06.398366 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-31318e12-ec62-4167-a2e8-bf4e177532a3 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: 7efe6f96-4156-45dc-b337-fb451e51fdd4] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:07:06.404649 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-31318e12-ec62-4167-a2e8-bf4e177532a3 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:07:06.405104 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-31318e12-ec62-4167-a2e8-bf4e177532a3 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:07:06.405869 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-31318e12-ec62-4167-a2e8-bf4e177532a3 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:07:06.406313 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-31318e12-ec62-4167-a2e8-bf4e177532a3 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:07:06.420348 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-31318e12-ec62-4167-a2e8-bf4e177532a3 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] HTTP exception thrown: Instance 7efe6f96-4156-45dc-b337-fb451e51fdd4 could not be found. Jul 16 22:07:06.421852 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-31318e12-ec62-4167-a2e8-bf4e177532a3 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Request method failure captured: Jul 16 22:07:06.421852 ncdv-master-1 devstack@n-api.service[138680]: request: GET /compute/v2.1/servers/7efe6f96-4156-45dc-b337-fb451e51fdd4 HTTP/1.1 Jul 16 22:07:06.421852 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 22:07:06.421852 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 22:07:06.421852 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 22:07:06.421852 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 0 Jul 16 22:07:06.421852 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 22:07:06.421852 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 22:07:06.421852 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 22:07:06.421852 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 22:07:06.421852 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0zUsISRuFerqPWLnIU6pWtJZ1G33kS5UQBpOE-Qn9W1zRIsDO0obaxdGmXyhe5FNe11pjr8YSFwmEAtO4tE-QrUwHNB6R_8ZgLrv3bb_zQTEpcOValb2w1FWdJ1eIHV_96nHchz9PmXAJpOM5Cp_lLo9a71M8MQcZrQgoRzDb41oZDA Jul 16 22:07:06.421852 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 22:07:06.421852 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 22:07:06.421852 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 22:07:06.421852 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 22:07:06.421852 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 22:07:06.421852 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 22:07:06.421852 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: fd21231d9af34d9aa4f9e03229b49a30 Jul 16 22:07:06.421852 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-ServerActionsTestJSON-164809533 Jul 16 22:07:06.421852 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 22:07:06.421852 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 22:07:06.421852 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/fd21231d9af34d9aa4f9e03229b49a30"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/fd21231d9af34d9aa4f9e03229b49a30"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_fd21231d9af34d9aa4f9e03229b49a30"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/fd21231d9af34d9aa4f9e03229b49a30"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 22:07:06.421852 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-ServerActionsTestJSON-164809533 Jul 16 22:07:06.421852 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: fd21231d9af34d9aa4f9e03229b49a30 Jul 16 22:07:06.421852 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-ServerActionsTestJSON-164809533 Jul 16 22:07:06.421852 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-ServerActionsTestJSON-164809533-project Jul 16 22:07:06.421852 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 22:07:06.421852 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 22:07:06.421852 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: 718f6de3b1a1453d9579d421a574691e Jul 16 22:07:06.421852 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-ServerActionsTestJSON-164809533-project Jul 16 22:07:06.421852 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 22:07:06.421852 ncdv-master-1 devstack@n-api.service[138680]: exception: Instance 7efe6f96-4156-45dc-b337-fb451e51fdd4 could not be found. Jul 16 22:07:06.421852 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': '7efe6f96-4156-45dc-b337-fb451e51fdd4'} Jul 16 22:07:06.421852 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 22:07:06.421852 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 22:07:06.421852 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 22:07:06.421852 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 22:07:06.421852 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 22:07:06.421852 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 22:07:06.421852 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 22:07:06.421852 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 22:07:06.421852 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 22:07:06.421852 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 22:07:06.421852 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 7efe6f96-4156-45dc-b337-fb451e51fdd4 could not be found. Jul 16 22:07:06.421852 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 22:07:06.421852 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 22:07:06.421852 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 22:07:06.421852 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 22:07:06.421852 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 22:07:06.421852 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 22:07:06.421852 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 22:07:06.421852 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 22:07:06.421852 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 22:07:06.421852 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 22:07:06.421852 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 22:07:06.421852 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 22:07:06.421852 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 22:07:06.421852 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 22:07:06.421852 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 22:07:06.421852 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 22:07:06.421852 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 7efe6f96-4156-45dc-b337-fb451e51fdd4 could not be found. Jul 16 22:07:06.421852 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 22:07:06.421852 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 22:07:06.421852 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 22:07:06.426379 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 22:07:06.426379 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 22:07:06.426379 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 22:07:06.426379 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 22:07:06.426379 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 22:07:06.426379 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 7efe6f96-4156-45dc-b337-fb451e51fdd4 could not be found. Jul 16 22:07:06.426379 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 22:07:06.426379 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-31318e12-ec62-4167-a2e8-bf4e177532a3 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Returning 404 to user: Instance 7efe6f96-4156-45dc-b337-fb451e51fdd4 could not be found. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 22:07:06.426379 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-31318e12-ec62-4167-a2e8-bf4e177532a3 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/7efe6f96-4156-45dc-b337-fb451e51fdd4" status: 404 len: 111 microversion: 2.1 time: 0.029463 Jul 16 22:07:06.426379 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3363/6733] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:07:06 2022] GET /compute/v2.1/servers/7efe6f96-4156-45dc-b337-fb451e51fdd4 => generated 111 bytes in 30 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Jul 16 22:07:06.432499 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-4165e596-dee4-4629-86dd-1fa15738a718 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:07:06.433002 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-4165e596-dee4-4629-86dd-1fa15738a718 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: fa9945a3-51cc-4366-bc7f-31ddb4f0fa01] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:07:06.439237 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4165e596-dee4-4629-86dd-1fa15738a718 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:07:06.439740 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4165e596-dee4-4629-86dd-1fa15738a718 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:07:06.440520 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4165e596-dee4-4629-86dd-1fa15738a718 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:07:06.440972 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4165e596-dee4-4629-86dd-1fa15738a718 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:07:06.457720 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-4165e596-dee4-4629-86dd-1fa15738a718 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] HTTP exception thrown: Instance fa9945a3-51cc-4366-bc7f-31ddb4f0fa01 could not be found. Jul 16 22:07:06.459310 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-4165e596-dee4-4629-86dd-1fa15738a718 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Request method failure captured: Jul 16 22:07:06.459310 ncdv-master-1 devstack@n-api.service[138681]: request: DELETE /compute/v2.1/servers/fa9945a3-51cc-4366-bc7f-31ddb4f0fa01 HTTP/1.1 Jul 16 22:07:06.459310 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 22:07:06.459310 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 22:07:06.459310 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 22:07:06.459310 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 22:07:06.459310 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 22:07:06.459310 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 22:07:06.459310 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 22:07:06.459310 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 22:07:06.459310 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0zUsISRuFerqPWLnIU6pWtJZ1G33kS5UQBpOE-Qn9W1zRIsDO0obaxdGmXyhe5FNe11pjr8YSFwmEAtO4tE-QrUwHNB6R_8ZgLrv3bb_zQTEpcOValb2w1FWdJ1eIHV_96nHchz9PmXAJpOM5Cp_lLo9a71M8MQcZrQgoRzDb41oZDA Jul 16 22:07:06.459310 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 22:07:06.459310 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 22:07:06.459310 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 22:07:06.459310 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 22:07:06.459310 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 22:07:06.459310 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 22:07:06.459310 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: fd21231d9af34d9aa4f9e03229b49a30 Jul 16 22:07:06.459310 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-ServerActionsTestJSON-164809533 Jul 16 22:07:06.459310 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 22:07:06.459310 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 22:07:06.459310 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/fd21231d9af34d9aa4f9e03229b49a30"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/fd21231d9af34d9aa4f9e03229b49a30"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_fd21231d9af34d9aa4f9e03229b49a30"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/fd21231d9af34d9aa4f9e03229b49a30"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 22:07:06.459310 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-ServerActionsTestJSON-164809533 Jul 16 22:07:06.459310 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: fd21231d9af34d9aa4f9e03229b49a30 Jul 16 22:07:06.459310 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-ServerActionsTestJSON-164809533 Jul 16 22:07:06.459310 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-ServerActionsTestJSON-164809533-project Jul 16 22:07:06.459310 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 22:07:06.459310 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 22:07:06.459310 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: 718f6de3b1a1453d9579d421a574691e Jul 16 22:07:06.459310 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-ServerActionsTestJSON-164809533-project Jul 16 22:07:06.459310 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 22:07:06.459310 ncdv-master-1 devstack@n-api.service[138681]: exception: Instance fa9945a3-51cc-4366-bc7f-31ddb4f0fa01 could not be found. Jul 16 22:07:06.459310 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': 'fa9945a3-51cc-4366-bc7f-31ddb4f0fa01'} Jul 16 22:07:06.459310 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 22:07:06.459310 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 22:07:06.459310 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 22:07:06.459310 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 22:07:06.459310 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 22:07:06.459310 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 22:07:06.459310 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 22:07:06.459310 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 22:07:06.459310 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 22:07:06.459310 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 22:07:06.459310 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance fa9945a3-51cc-4366-bc7f-31ddb4f0fa01 could not be found. Jul 16 22:07:06.459310 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 22:07:06.459310 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 22:07:06.459310 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 22:07:06.459310 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 22:07:06.459310 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 22:07:06.459310 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 22:07:06.459310 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 22:07:06.459310 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 22:07:06.459310 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 22:07:06.459310 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 22:07:06.459310 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Jul 16 22:07:06.459310 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Jul 16 22:07:06.459310 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Jul 16 22:07:06.459310 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Jul 16 22:07:06.459310 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 22:07:06.459310 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 22:07:06.459310 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 22:07:06.459310 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 22:07:06.464398 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance fa9945a3-51cc-4366-bc7f-31ddb4f0fa01 could not be found. Jul 16 22:07:06.464398 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 22:07:06.464398 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 22:07:06.464398 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 22:07:06.464398 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 22:07:06.464398 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 22:07:06.464398 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 22:07:06.464398 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 22:07:06.464398 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 22:07:06.464398 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance fa9945a3-51cc-4366-bc7f-31ddb4f0fa01 could not be found. Jul 16 22:07:06.464398 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 22:07:06.464398 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-4165e596-dee4-4629-86dd-1fa15738a718 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Returning 404 to user: Instance fa9945a3-51cc-4366-bc7f-31ddb4f0fa01 could not be found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 22:07:06.464398 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-4165e596-dee4-4629-86dd-1fa15738a718 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "DELETE /compute/v2.1/servers/fa9945a3-51cc-4366-bc7f-31ddb4f0fa01" status: 404 len: 111 microversion: 2.1 time: 0.032855 Jul 16 22:07:06.464398 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3371/6734] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 22:07:06 2022] DELETE /compute/v2.1/servers/fa9945a3-51cc-4366-bc7f-31ddb4f0fa01 => generated 111 bytes in 34 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Jul 16 22:07:06.469651 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-e749a93e-537c-40f6-8af9-8c1fe1c8ad7f tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:07:06.470323 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-e749a93e-537c-40f6-8af9-8c1fe1c8ad7f tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: fa9945a3-51cc-4366-bc7f-31ddb4f0fa01] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:07:06.476524 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e749a93e-537c-40f6-8af9-8c1fe1c8ad7f tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:07:06.477503 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e749a93e-537c-40f6-8af9-8c1fe1c8ad7f tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:07:06.478277 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e749a93e-537c-40f6-8af9-8c1fe1c8ad7f tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:07:06.478720 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e749a93e-537c-40f6-8af9-8c1fe1c8ad7f tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:07:06.491970 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-e749a93e-537c-40f6-8af9-8c1fe1c8ad7f tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] HTTP exception thrown: Instance fa9945a3-51cc-4366-bc7f-31ddb4f0fa01 could not be found. Jul 16 22:07:06.494146 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-e749a93e-537c-40f6-8af9-8c1fe1c8ad7f tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Request method failure captured: Jul 16 22:07:06.494146 ncdv-master-1 devstack@n-api.service[138680]: request: GET /compute/v2.1/servers/fa9945a3-51cc-4366-bc7f-31ddb4f0fa01 HTTP/1.1 Jul 16 22:07:06.494146 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 22:07:06.494146 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 22:07:06.494146 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 22:07:06.494146 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 0 Jul 16 22:07:06.494146 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 22:07:06.494146 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 22:07:06.494146 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 22:07:06.494146 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 22:07:06.494146 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0zUsISRuFerqPWLnIU6pWtJZ1G33kS5UQBpOE-Qn9W1zRIsDO0obaxdGmXyhe5FNe11pjr8YSFwmEAtO4tE-QrUwHNB6R_8ZgLrv3bb_zQTEpcOValb2w1FWdJ1eIHV_96nHchz9PmXAJpOM5Cp_lLo9a71M8MQcZrQgoRzDb41oZDA Jul 16 22:07:06.494146 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 22:07:06.494146 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 22:07:06.494146 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 22:07:06.494146 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 22:07:06.494146 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 22:07:06.494146 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 22:07:06.494146 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: fd21231d9af34d9aa4f9e03229b49a30 Jul 16 22:07:06.494146 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-ServerActionsTestJSON-164809533 Jul 16 22:07:06.494146 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 22:07:06.494146 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 22:07:06.494146 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/fd21231d9af34d9aa4f9e03229b49a30"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/fd21231d9af34d9aa4f9e03229b49a30"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_fd21231d9af34d9aa4f9e03229b49a30"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/fd21231d9af34d9aa4f9e03229b49a30"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 22:07:06.494146 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-ServerActionsTestJSON-164809533 Jul 16 22:07:06.494146 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: fd21231d9af34d9aa4f9e03229b49a30 Jul 16 22:07:06.494146 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-ServerActionsTestJSON-164809533 Jul 16 22:07:06.494146 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-ServerActionsTestJSON-164809533-project Jul 16 22:07:06.494146 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 22:07:06.494146 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 22:07:06.494146 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: 718f6de3b1a1453d9579d421a574691e Jul 16 22:07:06.494146 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-ServerActionsTestJSON-164809533-project Jul 16 22:07:06.494146 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 22:07:06.494146 ncdv-master-1 devstack@n-api.service[138680]: exception: Instance fa9945a3-51cc-4366-bc7f-31ddb4f0fa01 could not be found. Jul 16 22:07:06.494146 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': 'fa9945a3-51cc-4366-bc7f-31ddb4f0fa01'} Jul 16 22:07:06.494146 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 22:07:06.494146 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 22:07:06.494146 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 22:07:06.494146 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 22:07:06.494146 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 22:07:06.494146 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 22:07:06.494146 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 22:07:06.494146 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 22:07:06.494146 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 22:07:06.494146 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 22:07:06.494146 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance fa9945a3-51cc-4366-bc7f-31ddb4f0fa01 could not be found. Jul 16 22:07:06.494146 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 22:07:06.494146 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 22:07:06.494146 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 22:07:06.494146 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 22:07:06.494146 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 22:07:06.494146 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 22:07:06.494146 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 22:07:06.494146 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 22:07:06.494146 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 22:07:06.494146 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 22:07:06.494146 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 22:07:06.494146 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 22:07:06.494146 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 22:07:06.494146 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 22:07:06.494146 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 22:07:06.494146 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 22:07:06.494146 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance fa9945a3-51cc-4366-bc7f-31ddb4f0fa01 could not be found. Jul 16 22:07:06.494146 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 22:07:06.494146 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 22:07:06.494146 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 22:07:06.498964 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 22:07:06.498964 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 22:07:06.498964 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 22:07:06.498964 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 22:07:06.498964 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 22:07:06.498964 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance fa9945a3-51cc-4366-bc7f-31ddb4f0fa01 could not be found. Jul 16 22:07:06.498964 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 22:07:06.498964 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-e749a93e-537c-40f6-8af9-8c1fe1c8ad7f tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Returning 404 to user: Instance fa9945a3-51cc-4366-bc7f-31ddb4f0fa01 could not be found. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 22:07:06.498964 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-e749a93e-537c-40f6-8af9-8c1fe1c8ad7f tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/fa9945a3-51cc-4366-bc7f-31ddb4f0fa01" status: 404 len: 111 microversion: 2.1 time: 0.029850 Jul 16 22:07:06.498964 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3364/6735] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:07:06 2022] GET /compute/v2.1/servers/fa9945a3-51cc-4366-bc7f-31ddb4f0fa01 => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Jul 16 22:07:06.504137 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-4d896419-7187-4ef7-9ceb-420a306b960f tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:07:06.504656 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-4d896419-7187-4ef7-9ceb-420a306b960f tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: 3447d99f-2076-4204-b4d2-55339d60ca3a] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:07:06.510968 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4d896419-7187-4ef7-9ceb-420a306b960f tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:07:06.511472 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4d896419-7187-4ef7-9ceb-420a306b960f tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:07:06.512250 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4d896419-7187-4ef7-9ceb-420a306b960f tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:07:06.512701 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4d896419-7187-4ef7-9ceb-420a306b960f tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:07:06.526754 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-4d896419-7187-4ef7-9ceb-420a306b960f tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] HTTP exception thrown: Instance 3447d99f-2076-4204-b4d2-55339d60ca3a could not be found. Jul 16 22:07:06.528351 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-4d896419-7187-4ef7-9ceb-420a306b960f tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Request method failure captured: Jul 16 22:07:06.528351 ncdv-master-1 devstack@n-api.service[138681]: request: DELETE /compute/v2.1/servers/3447d99f-2076-4204-b4d2-55339d60ca3a HTTP/1.1 Jul 16 22:07:06.528351 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 22:07:06.528351 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 22:07:06.528351 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 22:07:06.528351 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 22:07:06.528351 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 22:07:06.528351 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 22:07:06.528351 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 22:07:06.528351 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 22:07:06.528351 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0zUsISRuFerqPWLnIU6pWtJZ1G33kS5UQBpOE-Qn9W1zRIsDO0obaxdGmXyhe5FNe11pjr8YSFwmEAtO4tE-QrUwHNB6R_8ZgLrv3bb_zQTEpcOValb2w1FWdJ1eIHV_96nHchz9PmXAJpOM5Cp_lLo9a71M8MQcZrQgoRzDb41oZDA Jul 16 22:07:06.528351 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 22:07:06.528351 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 22:07:06.528351 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 22:07:06.528351 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 22:07:06.528351 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 22:07:06.528351 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 22:07:06.528351 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: fd21231d9af34d9aa4f9e03229b49a30 Jul 16 22:07:06.528351 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-ServerActionsTestJSON-164809533 Jul 16 22:07:06.528351 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 22:07:06.528351 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 22:07:06.528351 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/fd21231d9af34d9aa4f9e03229b49a30"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/fd21231d9af34d9aa4f9e03229b49a30"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_fd21231d9af34d9aa4f9e03229b49a30"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/fd21231d9af34d9aa4f9e03229b49a30"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 22:07:06.528351 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-ServerActionsTestJSON-164809533 Jul 16 22:07:06.528351 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: fd21231d9af34d9aa4f9e03229b49a30 Jul 16 22:07:06.528351 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-ServerActionsTestJSON-164809533 Jul 16 22:07:06.528351 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-ServerActionsTestJSON-164809533-project Jul 16 22:07:06.528351 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 22:07:06.528351 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 22:07:06.528351 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: 718f6de3b1a1453d9579d421a574691e Jul 16 22:07:06.528351 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-ServerActionsTestJSON-164809533-project Jul 16 22:07:06.528351 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 22:07:06.528351 ncdv-master-1 devstack@n-api.service[138681]: exception: Instance 3447d99f-2076-4204-b4d2-55339d60ca3a could not be found. Jul 16 22:07:06.528351 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': '3447d99f-2076-4204-b4d2-55339d60ca3a'} Jul 16 22:07:06.528351 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 22:07:06.528351 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 22:07:06.528351 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 22:07:06.528351 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 22:07:06.528351 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 22:07:06.528351 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 22:07:06.528351 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 22:07:06.528351 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 22:07:06.528351 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 22:07:06.528351 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 22:07:06.528351 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 3447d99f-2076-4204-b4d2-55339d60ca3a could not be found. Jul 16 22:07:06.528351 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 22:07:06.528351 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 22:07:06.528351 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 22:07:06.528351 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 22:07:06.528351 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 22:07:06.528351 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 22:07:06.528351 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 22:07:06.528351 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 22:07:06.528351 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 22:07:06.528351 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 22:07:06.528351 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Jul 16 22:07:06.528351 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Jul 16 22:07:06.528351 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Jul 16 22:07:06.528351 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Jul 16 22:07:06.528351 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 22:07:06.528351 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 22:07:06.528351 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 22:07:06.528351 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 22:07:06.532816 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 3447d99f-2076-4204-b4d2-55339d60ca3a could not be found. Jul 16 22:07:06.532816 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 22:07:06.532816 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 22:07:06.532816 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 22:07:06.532816 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 22:07:06.532816 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 22:07:06.532816 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 22:07:06.532816 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 22:07:06.532816 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 22:07:06.532816 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 3447d99f-2076-4204-b4d2-55339d60ca3a could not be found. Jul 16 22:07:06.532816 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 22:07:06.532816 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-4d896419-7187-4ef7-9ceb-420a306b960f tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Returning 404 to user: Instance 3447d99f-2076-4204-b4d2-55339d60ca3a could not be found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 22:07:06.532816 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-4d896419-7187-4ef7-9ceb-420a306b960f tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "DELETE /compute/v2.1/servers/3447d99f-2076-4204-b4d2-55339d60ca3a" status: 404 len: 111 microversion: 2.1 time: 0.029531 Jul 16 22:07:06.532816 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3372/6736] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 22:07:06 2022] DELETE /compute/v2.1/servers/3447d99f-2076-4204-b4d2-55339d60ca3a => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 22:07:06.538164 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-e78cdfb0-5286-43a2-b79c-7e5a5f01b67d tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:07:06.538796 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-e78cdfb0-5286-43a2-b79c-7e5a5f01b67d tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: 3447d99f-2076-4204-b4d2-55339d60ca3a] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:07:06.545787 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e78cdfb0-5286-43a2-b79c-7e5a5f01b67d tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:07:06.546244 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e78cdfb0-5286-43a2-b79c-7e5a5f01b67d tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:07:06.547030 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e78cdfb0-5286-43a2-b79c-7e5a5f01b67d tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:07:06.547471 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e78cdfb0-5286-43a2-b79c-7e5a5f01b67d tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:07:06.561541 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-e78cdfb0-5286-43a2-b79c-7e5a5f01b67d tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] HTTP exception thrown: Instance 3447d99f-2076-4204-b4d2-55339d60ca3a could not be found. Jul 16 22:07:06.563075 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-e78cdfb0-5286-43a2-b79c-7e5a5f01b67d tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Request method failure captured: Jul 16 22:07:06.563075 ncdv-master-1 devstack@n-api.service[138680]: request: GET /compute/v2.1/servers/3447d99f-2076-4204-b4d2-55339d60ca3a HTTP/1.1 Jul 16 22:07:06.563075 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 22:07:06.563075 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 22:07:06.563075 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 22:07:06.563075 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 0 Jul 16 22:07:06.563075 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 22:07:06.563075 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 22:07:06.563075 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 22:07:06.563075 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 22:07:06.563075 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0zUsISRuFerqPWLnIU6pWtJZ1G33kS5UQBpOE-Qn9W1zRIsDO0obaxdGmXyhe5FNe11pjr8YSFwmEAtO4tE-QrUwHNB6R_8ZgLrv3bb_zQTEpcOValb2w1FWdJ1eIHV_96nHchz9PmXAJpOM5Cp_lLo9a71M8MQcZrQgoRzDb41oZDA Jul 16 22:07:06.563075 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 22:07:06.563075 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 22:07:06.563075 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 22:07:06.563075 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 22:07:06.563075 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 22:07:06.563075 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 22:07:06.563075 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: fd21231d9af34d9aa4f9e03229b49a30 Jul 16 22:07:06.563075 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-ServerActionsTestJSON-164809533 Jul 16 22:07:06.563075 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 22:07:06.563075 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 22:07:06.563075 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/fd21231d9af34d9aa4f9e03229b49a30"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/fd21231d9af34d9aa4f9e03229b49a30"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_fd21231d9af34d9aa4f9e03229b49a30"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/fd21231d9af34d9aa4f9e03229b49a30"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 22:07:06.563075 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-ServerActionsTestJSON-164809533 Jul 16 22:07:06.563075 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: fd21231d9af34d9aa4f9e03229b49a30 Jul 16 22:07:06.563075 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-ServerActionsTestJSON-164809533 Jul 16 22:07:06.563075 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-ServerActionsTestJSON-164809533-project Jul 16 22:07:06.563075 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 22:07:06.563075 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 22:07:06.563075 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: 718f6de3b1a1453d9579d421a574691e Jul 16 22:07:06.563075 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-ServerActionsTestJSON-164809533-project Jul 16 22:07:06.563075 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 22:07:06.563075 ncdv-master-1 devstack@n-api.service[138680]: exception: Instance 3447d99f-2076-4204-b4d2-55339d60ca3a could not be found. Jul 16 22:07:06.563075 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': '3447d99f-2076-4204-b4d2-55339d60ca3a'} Jul 16 22:07:06.563075 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 22:07:06.563075 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 22:07:06.563075 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 22:07:06.563075 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 22:07:06.563075 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 22:07:06.563075 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 22:07:06.563075 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 22:07:06.563075 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 22:07:06.563075 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 22:07:06.563075 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 22:07:06.563075 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 3447d99f-2076-4204-b4d2-55339d60ca3a could not be found. Jul 16 22:07:06.563075 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 22:07:06.563075 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 22:07:06.563075 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 22:07:06.563075 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 22:07:06.563075 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 22:07:06.563075 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 22:07:06.563075 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 22:07:06.563075 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 22:07:06.563075 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 22:07:06.563075 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 22:07:06.563075 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 22:07:06.563075 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 22:07:06.563075 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 22:07:06.563075 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 22:07:06.563075 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 22:07:06.563075 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 22:07:06.563075 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 3447d99f-2076-4204-b4d2-55339d60ca3a could not be found. Jul 16 22:07:06.563075 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 22:07:06.563075 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 22:07:06.563075 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 22:07:06.567437 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 22:07:06.567437 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 22:07:06.567437 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 22:07:06.567437 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 22:07:06.567437 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 22:07:06.567437 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 3447d99f-2076-4204-b4d2-55339d60ca3a could not be found. Jul 16 22:07:06.567437 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 22:07:06.567437 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-e78cdfb0-5286-43a2-b79c-7e5a5f01b67d tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Returning 404 to user: Instance 3447d99f-2076-4204-b4d2-55339d60ca3a could not be found. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 22:07:06.567437 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-e78cdfb0-5286-43a2-b79c-7e5a5f01b67d tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/3447d99f-2076-4204-b4d2-55339d60ca3a" status: 404 len: 111 microversion: 2.1 time: 0.030159 Jul 16 22:07:06.567437 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3365/6737] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:07:06 2022] GET /compute/v2.1/servers/3447d99f-2076-4204-b4d2-55339d60ca3a => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 22:07:06.572964 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-7cda7754-b5c8-49fd-b797-79741bfabe26 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:07:06.573486 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-7cda7754-b5c8-49fd-b797-79741bfabe26 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: ccef2dfa-df64-4f44-96db-85598e217ba8] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:07:06.580103 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7cda7754-b5c8-49fd-b797-79741bfabe26 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:07:06.580579 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7cda7754-b5c8-49fd-b797-79741bfabe26 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:07:06.581362 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7cda7754-b5c8-49fd-b797-79741bfabe26 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:07:06.581822 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7cda7754-b5c8-49fd-b797-79741bfabe26 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:07:06.595987 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-7cda7754-b5c8-49fd-b797-79741bfabe26 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] HTTP exception thrown: Instance ccef2dfa-df64-4f44-96db-85598e217ba8 could not be found. Jul 16 22:07:06.597520 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-7cda7754-b5c8-49fd-b797-79741bfabe26 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Request method failure captured: Jul 16 22:07:06.597520 ncdv-master-1 devstack@n-api.service[138681]: request: DELETE /compute/v2.1/servers/ccef2dfa-df64-4f44-96db-85598e217ba8 HTTP/1.1 Jul 16 22:07:06.597520 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 22:07:06.597520 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 22:07:06.597520 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 22:07:06.597520 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 22:07:06.597520 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 22:07:06.597520 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 22:07:06.597520 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 22:07:06.597520 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 22:07:06.597520 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0zUsISRuFerqPWLnIU6pWtJZ1G33kS5UQBpOE-Qn9W1zRIsDO0obaxdGmXyhe5FNe11pjr8YSFwmEAtO4tE-QrUwHNB6R_8ZgLrv3bb_zQTEpcOValb2w1FWdJ1eIHV_96nHchz9PmXAJpOM5Cp_lLo9a71M8MQcZrQgoRzDb41oZDA Jul 16 22:07:06.597520 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 22:07:06.597520 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 22:07:06.597520 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 22:07:06.597520 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 22:07:06.597520 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 22:07:06.597520 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 22:07:06.597520 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: fd21231d9af34d9aa4f9e03229b49a30 Jul 16 22:07:06.597520 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-ServerActionsTestJSON-164809533 Jul 16 22:07:06.597520 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 22:07:06.597520 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 22:07:06.597520 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/fd21231d9af34d9aa4f9e03229b49a30"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/fd21231d9af34d9aa4f9e03229b49a30"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_fd21231d9af34d9aa4f9e03229b49a30"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/fd21231d9af34d9aa4f9e03229b49a30"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 22:07:06.597520 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-ServerActionsTestJSON-164809533 Jul 16 22:07:06.597520 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: fd21231d9af34d9aa4f9e03229b49a30 Jul 16 22:07:06.597520 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-ServerActionsTestJSON-164809533 Jul 16 22:07:06.597520 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-ServerActionsTestJSON-164809533-project Jul 16 22:07:06.597520 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 22:07:06.597520 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 22:07:06.597520 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: 718f6de3b1a1453d9579d421a574691e Jul 16 22:07:06.597520 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-ServerActionsTestJSON-164809533-project Jul 16 22:07:06.597520 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 22:07:06.597520 ncdv-master-1 devstack@n-api.service[138681]: exception: Instance ccef2dfa-df64-4f44-96db-85598e217ba8 could not be found. Jul 16 22:07:06.597520 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': 'ccef2dfa-df64-4f44-96db-85598e217ba8'} Jul 16 22:07:06.597520 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 22:07:06.597520 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 22:07:06.597520 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 22:07:06.597520 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 22:07:06.597520 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 22:07:06.597520 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 22:07:06.597520 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 22:07:06.597520 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 22:07:06.597520 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 22:07:06.597520 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 22:07:06.597520 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance ccef2dfa-df64-4f44-96db-85598e217ba8 could not be found. Jul 16 22:07:06.597520 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 22:07:06.597520 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 22:07:06.597520 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 22:07:06.597520 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 22:07:06.597520 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 22:07:06.597520 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 22:07:06.597520 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 22:07:06.597520 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 22:07:06.597520 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 22:07:06.597520 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 22:07:06.597520 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Jul 16 22:07:06.597520 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Jul 16 22:07:06.597520 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Jul 16 22:07:06.597520 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Jul 16 22:07:06.597520 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 22:07:06.597520 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 22:07:06.597520 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 22:07:06.597520 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 22:07:06.601534 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance ccef2dfa-df64-4f44-96db-85598e217ba8 could not be found. Jul 16 22:07:06.601534 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 22:07:06.601534 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 22:07:06.601534 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 22:07:06.601534 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 22:07:06.601534 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 22:07:06.601534 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 22:07:06.601534 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 22:07:06.601534 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 22:07:06.601534 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance ccef2dfa-df64-4f44-96db-85598e217ba8 could not be found. Jul 16 22:07:06.601534 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 22:07:06.601534 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-7cda7754-b5c8-49fd-b797-79741bfabe26 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Returning 404 to user: Instance ccef2dfa-df64-4f44-96db-85598e217ba8 could not be found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 22:07:06.601534 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-7cda7754-b5c8-49fd-b797-79741bfabe26 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "DELETE /compute/v2.1/servers/ccef2dfa-df64-4f44-96db-85598e217ba8" status: 404 len: 111 microversion: 2.1 time: 0.029913 Jul 16 22:07:06.601534 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3373/6738] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 22:07:06 2022] DELETE /compute/v2.1/servers/ccef2dfa-df64-4f44-96db-85598e217ba8 => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Jul 16 22:07:06.607817 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-8c0ed32c-2274-484e-abcf-bf716e35e8c6 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:07:06.608437 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-8c0ed32c-2274-484e-abcf-bf716e35e8c6 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] [instance: ccef2dfa-df64-4f44-96db-85598e217ba8] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:07:06.614583 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8c0ed32c-2274-484e-abcf-bf716e35e8c6 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:07:06.615078 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8c0ed32c-2274-484e-abcf-bf716e35e8c6 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:07:06.615851 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8c0ed32c-2274-484e-abcf-bf716e35e8c6 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:07:06.616330 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8c0ed32c-2274-484e-abcf-bf716e35e8c6 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:07:06.629765 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-8c0ed32c-2274-484e-abcf-bf716e35e8c6 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] HTTP exception thrown: Instance ccef2dfa-df64-4f44-96db-85598e217ba8 could not be found. Jul 16 22:07:06.631314 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-8c0ed32c-2274-484e-abcf-bf716e35e8c6 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Request method failure captured: Jul 16 22:07:06.631314 ncdv-master-1 devstack@n-api.service[138680]: request: GET /compute/v2.1/servers/ccef2dfa-df64-4f44-96db-85598e217ba8 HTTP/1.1 Jul 16 22:07:06.631314 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 22:07:06.631314 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 22:07:06.631314 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 22:07:06.631314 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 0 Jul 16 22:07:06.631314 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 22:07:06.631314 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 22:07:06.631314 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 22:07:06.631314 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 22:07:06.631314 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0zUsISRuFerqPWLnIU6pWtJZ1G33kS5UQBpOE-Qn9W1zRIsDO0obaxdGmXyhe5FNe11pjr8YSFwmEAtO4tE-QrUwHNB6R_8ZgLrv3bb_zQTEpcOValb2w1FWdJ1eIHV_96nHchz9PmXAJpOM5Cp_lLo9a71M8MQcZrQgoRzDb41oZDA Jul 16 22:07:06.631314 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 22:07:06.631314 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 22:07:06.631314 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 22:07:06.631314 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 22:07:06.631314 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 22:07:06.631314 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 22:07:06.631314 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: fd21231d9af34d9aa4f9e03229b49a30 Jul 16 22:07:06.631314 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-ServerActionsTestJSON-164809533 Jul 16 22:07:06.631314 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 22:07:06.631314 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 22:07:06.631314 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/fd21231d9af34d9aa4f9e03229b49a30"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/fd21231d9af34d9aa4f9e03229b49a30"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_fd21231d9af34d9aa4f9e03229b49a30"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/fd21231d9af34d9aa4f9e03229b49a30"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 22:07:06.631314 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-ServerActionsTestJSON-164809533 Jul 16 22:07:06.631314 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: fd21231d9af34d9aa4f9e03229b49a30 Jul 16 22:07:06.631314 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-ServerActionsTestJSON-164809533 Jul 16 22:07:06.631314 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-ServerActionsTestJSON-164809533-project Jul 16 22:07:06.631314 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 22:07:06.631314 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 22:07:06.631314 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: 718f6de3b1a1453d9579d421a574691e Jul 16 22:07:06.631314 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-ServerActionsTestJSON-164809533-project Jul 16 22:07:06.631314 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 22:07:06.631314 ncdv-master-1 devstack@n-api.service[138680]: exception: Instance ccef2dfa-df64-4f44-96db-85598e217ba8 could not be found. Jul 16 22:07:06.631314 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': 'ccef2dfa-df64-4f44-96db-85598e217ba8'} Jul 16 22:07:06.631314 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 22:07:06.631314 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 22:07:06.631314 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 22:07:06.631314 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 22:07:06.631314 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 22:07:06.631314 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 22:07:06.631314 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 22:07:06.631314 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 22:07:06.631314 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 22:07:06.631314 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 22:07:06.631314 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance ccef2dfa-df64-4f44-96db-85598e217ba8 could not be found. Jul 16 22:07:06.631314 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 22:07:06.631314 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 22:07:06.631314 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 22:07:06.631314 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 22:07:06.631314 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 22:07:06.631314 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 22:07:06.631314 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 22:07:06.631314 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 22:07:06.631314 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 22:07:06.631314 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 22:07:06.631314 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 22:07:06.631314 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 22:07:06.631314 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 22:07:06.631314 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 22:07:06.631314 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 22:07:06.631314 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 22:07:06.631314 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance ccef2dfa-df64-4f44-96db-85598e217ba8 could not be found. Jul 16 22:07:06.631314 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 22:07:06.631314 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 22:07:06.631314 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 22:07:06.636804 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 22:07:06.636804 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 22:07:06.636804 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 22:07:06.636804 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 22:07:06.636804 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 22:07:06.636804 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance ccef2dfa-df64-4f44-96db-85598e217ba8 could not be found. Jul 16 22:07:06.636804 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 22:07:06.636804 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-8c0ed32c-2274-484e-abcf-bf716e35e8c6 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Returning 404 to user: Instance ccef2dfa-df64-4f44-96db-85598e217ba8 could not be found. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 22:07:06.636804 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-8c0ed32c-2274-484e-abcf-bf716e35e8c6 tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "GET /compute/v2.1/servers/ccef2dfa-df64-4f44-96db-85598e217ba8" status: 404 len: 111 microversion: 2.1 time: 0.028676 Jul 16 22:07:06.636804 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3366/6739] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:07:06 2022] GET /compute/v2.1/servers/ccef2dfa-df64-4f44-96db-85598e217ba8 => generated 111 bytes in 30 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Jul 16 22:07:06.641328 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-ae7c71d4-65e1-4484-92c7-7e319ca15fbd tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] Calling method '.version_select at 0x7fdd8c429550>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:07:06.721046 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-ae7c71d4-65e1-4484-92c7-7e319ca15fbd tempest-ServerActionsTestJSON-164809533 tempest-ServerActionsTestJSON-164809533-project] 10.222.0.16 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-26228548" status: 202 len: 0 microversion: 2.1 time: 0.083558 Jul 16 22:07:06.721617 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3374/6740] 10.222.0.16 () {58 vars in 1214 bytes} [Sat Jul 16 22:07:06 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-26228548 => generated 0 bytes in 85 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 22:07:18.203271 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_api:SELECT=33 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:07:18.204026 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=71 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:07:18.327580 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-e708d748-f672-4806-a445-e61dd7f048cf tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServersNegativeTestJSON-server-111598597", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452", "networks": [{"uuid": "51ae6879-cb74-494d-a439-27c219365dd7"}]}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:07:18.392119 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.network.neutron [None req-e708d748-f672-4806-a445-e61dd7f048cf tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] validate_networks() for [('51ae6879-cb74-494d-a439-27c219365dd7', None, None, None, None, None)] {{(pid=138680) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 16 22:07:18.860491 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-e708d748-f672-4806-a445-e61dd7f048cf tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Flavor limits 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 16 22:07:18.860926 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-e708d748-f672-4806-a445-e61dd7f048cf tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Image limits 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 16 22:07:18.861417 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-e708d748-f672-4806-a445-e61dd7f048cf tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Flavor pref 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 16 22:07:18.861831 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-e708d748-f672-4806-a445-e61dd7f048cf tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Image pref 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 16 22:07:18.862248 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-e708d748-f672-4806-a445-e61dd7f048cf tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 16 22:07:18.875873 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e708d748-f672-4806-a445-e61dd7f048cf tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:07:18.876336 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e708d748-f672-4806-a445-e61dd7f048cf tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:07:18.877144 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e708d748-f672-4806-a445-e61dd7f048cf tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:07:18.877585 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e708d748-f672-4806-a445-e61dd7f048cf tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:07:18.900363 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-e708d748-f672-4806-a445-e61dd7f048cf tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Getting quotas for project 1617cbc6e7cf4eed8d2f8968da996b9c. Resources: {'ram', 'instances', 'cores'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 22:07:18.905876 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-e708d748-f672-4806-a445-e61dd7f048cf tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Getting quotas for user b91f7cf2a74f45858448608b096b0c31 and project 1617cbc6e7cf4eed8d2f8968da996b9c. Resources: {'ram', 'instances', 'cores'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 22:07:18.920023 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-e708d748-f672-4806-a445-e61dd7f048cf tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Going to run 1 instances... {{(pid=138680) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 16 22:07:18.922350 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-e708d748-f672-4806-a445-e61dd7f048cf tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] [instance: dc0abe67-6867-4794-aaa6-5235d45edf2a] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138680) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 16 22:07:19.051748 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-e708d748-f672-4806-a445-e61dd7f048cf tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.852808 Jul 16 22:07:19.051748 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3367/6741] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 22:07:18 2022] POST /compute/v2.1/servers => generated 380 bytes in 855 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 16 22:07:19.061307 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=94,nova_cell1:UPDATE=3,nova_cell1:INSERT=2 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:07:19.062198 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_api:SELECT=53,nova_api:UPDATE=2,nova_api:DELETE=1 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:07:19.068142 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-aa5f1ce7-7230-444f-b114-cd0491801dfa tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:07:19.068639 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-aa5f1ce7-7230-444f-b114-cd0491801dfa tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] [instance: dc0abe67-6867-4794-aaa6-5235d45edf2a] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:07:19.146703 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-aa5f1ce7-7230-444f-b114-cd0491801dfa tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b91f7cf2a74f45858448608b096b0c31', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1617cbc6e7cf4eed8d2f8968da996b9c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:07:19.160127 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-aa5f1ce7-7230-444f-b114-cd0491801dfa tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] 10.222.0.16 "GET /compute/v2.1/servers/dc0abe67-6867-4794-aaa6-5235d45edf2a" status: 200 len: 1216 microversion: 2.1 time: 0.100862 Jul 16 22:07:19.160724 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3375/6742] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:07:19 2022] GET /compute/v2.1/servers/dc0abe67-6867-4794-aaa6-5235d45edf2a => generated 1216 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 22:07:20.180910 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-c465484e-2eb0-4fa6-8624-2a8f8f36d9f7 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:07:20.181539 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-c465484e-2eb0-4fa6-8624-2a8f8f36d9f7 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] [instance: dc0abe67-6867-4794-aaa6-5235d45edf2a] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:07:20.191221 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c465484e-2eb0-4fa6-8624-2a8f8f36d9f7 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:07:20.191705 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c465484e-2eb0-4fa6-8624-2a8f8f36d9f7 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:07:20.192702 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c465484e-2eb0-4fa6-8624-2a8f8f36d9f7 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:07:20.193494 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c465484e-2eb0-4fa6-8624-2a8f8f36d9f7 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:07:20.266716 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-c465484e-2eb0-4fa6-8624-2a8f8f36d9f7 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b91f7cf2a74f45858448608b096b0c31', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1617cbc6e7cf4eed8d2f8968da996b9c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:07:20.277503 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-c465484e-2eb0-4fa6-8624-2a8f8f36d9f7 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] 10.222.0.16 "GET /compute/v2.1/servers/dc0abe67-6867-4794-aaa6-5235d45edf2a" status: 200 len: 1220 microversion: 2.1 time: 0.103104 Jul 16 22:07:20.278298 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3368/6743] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:07:20 2022] GET /compute/v2.1/servers/dc0abe67-6867-4794-aaa6-5235d45edf2a => generated 1220 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 22:07:21.295646 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-0df411be-5f9f-4383-815d-fb84069ccf9c tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:07:21.296379 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-0df411be-5f9f-4383-815d-fb84069ccf9c tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] [instance: dc0abe67-6867-4794-aaa6-5235d45edf2a] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:07:21.304691 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0df411be-5f9f-4383-815d-fb84069ccf9c tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:07:21.305392 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0df411be-5f9f-4383-815d-fb84069ccf9c tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:07:21.306197 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0df411be-5f9f-4383-815d-fb84069ccf9c tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:07:21.306653 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0df411be-5f9f-4383-815d-fb84069ccf9c tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:07:21.398024 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-0df411be-5f9f-4383-815d-fb84069ccf9c tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b91f7cf2a74f45858448608b096b0c31', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1617cbc6e7cf4eed8d2f8968da996b9c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:07:21.407189 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-0df411be-5f9f-4383-815d-fb84069ccf9c tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] 10.222.0.16 "GET /compute/v2.1/servers/dc0abe67-6867-4794-aaa6-5235d45edf2a" status: 200 len: 1276 microversion: 2.1 time: 0.116139 Jul 16 22:07:21.411054 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3376/6744] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:07:21 2022] GET /compute/v2.1/servers/dc0abe67-6867-4794-aaa6-5235d45edf2a => generated 1276 bytes in 118 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:07:22.425323 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-b0008eb4-9b36-4e42-9133-e91dc762b354 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:07:22.426318 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-b0008eb4-9b36-4e42-9133-e91dc762b354 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] [instance: dc0abe67-6867-4794-aaa6-5235d45edf2a] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:07:22.435103 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b0008eb4-9b36-4e42-9133-e91dc762b354 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:07:22.435857 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b0008eb4-9b36-4e42-9133-e91dc762b354 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:07:22.436849 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b0008eb4-9b36-4e42-9133-e91dc762b354 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:07:22.437770 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b0008eb4-9b36-4e42-9133-e91dc762b354 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:07:22.644817 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-b0008eb4-9b36-4e42-9133-e91dc762b354 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b91f7cf2a74f45858448608b096b0c31', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1617cbc6e7cf4eed8d2f8968da996b9c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:07:22.653077 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-b0008eb4-9b36-4e42-9133-e91dc762b354 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] 10.222.0.16 "GET /compute/v2.1/servers/dc0abe67-6867-4794-aaa6-5235d45edf2a" status: 200 len: 1316 microversion: 2.1 time: 0.231719 Jul 16 22:07:22.653738 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3369/6745] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:07:22 2022] GET /compute/v2.1/servers/dc0abe67-6867-4794-aaa6-5235d45edf2a => generated 1316 bytes in 233 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:07:23.292018 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-63ff4121-f9e9-443a-bb25-d7c9bbc47490 req-6fe55ecb-5af7-4662-92a2-de279a65c010 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "dc0abe67-6867-4794-aaa6-5235d45edf2a", "tag": "0087cbc7-985b-4473-a370-59e9c23bfda0"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:07:23.301353 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-63ff4121-f9e9-443a-bb25-d7c9bbc47490 req-6fe55ecb-5af7-4662-92a2-de279a65c010 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:07:23.301822 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-63ff4121-f9e9-443a-bb25-d7c9bbc47490 req-6fe55ecb-5af7-4662-92a2-de279a65c010 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:07:23.313905 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-63ff4121-f9e9-443a-bb25-d7c9bbc47490 req-6fe55ecb-5af7-4662-92a2-de279a65c010 service nova] Creating event network-changed:0087cbc7-985b-4473-a370-59e9c23bfda0 for instance dc0abe67-6867-4794-aaa6-5235d45edf2a on nch2-master-1 Jul 16 22:07:23.319035 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-63ff4121-f9e9-443a-bb25-d7c9bbc47490 req-6fe55ecb-5af7-4662-92a2-de279a65c010 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.110188 Jul 16 22:07:23.319664 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3377/6746] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 22:07:23 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 112 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 22:07:23.670120 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-ecac5505-f28f-4da8-864d-6c8adb7008ee tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:07:23.670738 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-ecac5505-f28f-4da8-864d-6c8adb7008ee tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] [instance: dc0abe67-6867-4794-aaa6-5235d45edf2a] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:07:23.677601 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ecac5505-f28f-4da8-864d-6c8adb7008ee tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:07:23.678054 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ecac5505-f28f-4da8-864d-6c8adb7008ee tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:07:23.678822 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ecac5505-f28f-4da8-864d-6c8adb7008ee tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:07:23.679322 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ecac5505-f28f-4da8-864d-6c8adb7008ee tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:07:23.814275 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-ecac5505-f28f-4da8-864d-6c8adb7008ee tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b91f7cf2a74f45858448608b096b0c31', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1617cbc6e7cf4eed8d2f8968da996b9c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:07:23.822878 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-ecac5505-f28f-4da8-864d-6c8adb7008ee tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] 10.222.0.16 "GET /compute/v2.1/servers/dc0abe67-6867-4794-aaa6-5235d45edf2a" status: 200 len: 1316 microversion: 2.1 time: 0.156645 Jul 16 22:07:23.823560 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3370/6747] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:07:23 2022] GET /compute/v2.1/servers/dc0abe67-6867-4794-aaa6-5235d45edf2a => generated 1316 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:07:24.842376 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-251e846f-0c33-4f25-b170-2e447c452dbf tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:07:24.843382 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-251e846f-0c33-4f25-b170-2e447c452dbf tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] [instance: dc0abe67-6867-4794-aaa6-5235d45edf2a] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:07:24.849777 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-251e846f-0c33-4f25-b170-2e447c452dbf tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:07:24.850241 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-251e846f-0c33-4f25-b170-2e447c452dbf tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:07:24.851040 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-251e846f-0c33-4f25-b170-2e447c452dbf tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:07:24.852312 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-251e846f-0c33-4f25-b170-2e447c452dbf tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:07:25.407573 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-251e846f-0c33-4f25-b170-2e447c452dbf tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b91f7cf2a74f45858448608b096b0c31', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1617cbc6e7cf4eed8d2f8968da996b9c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:07:25.415576 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-251e846f-0c33-4f25-b170-2e447c452dbf tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] 10.222.0.16 "GET /compute/v2.1/servers/dc0abe67-6867-4794-aaa6-5235d45edf2a" status: 200 len: 1316 microversion: 2.1 time: 0.578533 Jul 16 22:07:25.416153 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3378/6748] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:07:24 2022] GET /compute/v2.1/servers/dc0abe67-6867-4794-aaa6-5235d45edf2a => generated 1316 bytes in 581 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:07:26.433496 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-5046d59f-3316-4c57-953d-3403d2c0f898 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:07:26.434048 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-5046d59f-3316-4c57-953d-3403d2c0f898 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] [instance: dc0abe67-6867-4794-aaa6-5235d45edf2a] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:07:26.440891 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5046d59f-3316-4c57-953d-3403d2c0f898 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:07:26.441345 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5046d59f-3316-4c57-953d-3403d2c0f898 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:07:26.442113 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5046d59f-3316-4c57-953d-3403d2c0f898 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:07:26.442548 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5046d59f-3316-4c57-953d-3403d2c0f898 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:07:26.580159 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-5046d59f-3316-4c57-953d-3403d2c0f898 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b91f7cf2a74f45858448608b096b0c31', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1617cbc6e7cf4eed8d2f8968da996b9c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:07:26.588562 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-5046d59f-3316-4c57-953d-3403d2c0f898 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] 10.222.0.16 "GET /compute/v2.1/servers/dc0abe67-6867-4794-aaa6-5235d45edf2a" status: 200 len: 1316 microversion: 2.1 time: 0.159806 Jul 16 22:07:26.589143 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3371/6749] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:07:26 2022] GET /compute/v2.1/servers/dc0abe67-6867-4794-aaa6-5235d45edf2a => generated 1316 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:07:27.612248 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-59cd759a-b737-4160-ab2b-fdb03690915c tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:07:27.612801 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-59cd759a-b737-4160-ab2b-fdb03690915c tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] [instance: dc0abe67-6867-4794-aaa6-5235d45edf2a] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:07:27.620480 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-59cd759a-b737-4160-ab2b-fdb03690915c tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:07:27.620953 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-59cd759a-b737-4160-ab2b-fdb03690915c tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:07:27.621740 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-59cd759a-b737-4160-ab2b-fdb03690915c tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:07:27.622199 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-59cd759a-b737-4160-ab2b-fdb03690915c tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:07:27.751698 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-59cd759a-b737-4160-ab2b-fdb03690915c tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b91f7cf2a74f45858448608b096b0c31', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1617cbc6e7cf4eed8d2f8968da996b9c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:07:27.759342 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-59cd759a-b737-4160-ab2b-fdb03690915c tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] 10.222.0.16 "GET /compute/v2.1/servers/dc0abe67-6867-4794-aaa6-5235d45edf2a" status: 200 len: 1316 microversion: 2.1 time: 0.151102 Jul 16 22:07:27.759916 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3379/6750] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:07:27 2022] GET /compute/v2.1/servers/dc0abe67-6867-4794-aaa6-5235d45edf2a => generated 1316 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:07:28.651641 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-6eee03e1-4a35-4053-ad5f-3da74a8dccdd req-c708c6dc-3c14-461e-9094-dfa4606369e2 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "dc0abe67-6867-4794-aaa6-5235d45edf2a", "name": "network-vif-plugged", "status": "completed", "tag": "0087cbc7-985b-4473-a370-59e9c23bfda0"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:07:28.660736 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-6eee03e1-4a35-4053-ad5f-3da74a8dccdd req-c708c6dc-3c14-461e-9094-dfa4606369e2 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:07:28.661269 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-6eee03e1-4a35-4053-ad5f-3da74a8dccdd req-c708c6dc-3c14-461e-9094-dfa4606369e2 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:07:28.672526 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-6eee03e1-4a35-4053-ad5f-3da74a8dccdd req-c708c6dc-3c14-461e-9094-dfa4606369e2 service nova] Creating event network-vif-plugged:0087cbc7-985b-4473-a370-59e9c23bfda0 for instance dc0abe67-6867-4794-aaa6-5235d45edf2a on nch2-master-1 Jul 16 22:07:28.677619 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-6eee03e1-4a35-4053-ad5f-3da74a8dccdd req-c708c6dc-3c14-461e-9094-dfa4606369e2 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.029691 Jul 16 22:07:28.678208 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3372/6751] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 22:07:28 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 31 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 22:07:28.776648 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-d0017134-89ef-4c3e-81dd-849904bbffb2 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:07:28.778535 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-d0017134-89ef-4c3e-81dd-849904bbffb2 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] [instance: dc0abe67-6867-4794-aaa6-5235d45edf2a] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:07:28.785852 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d0017134-89ef-4c3e-81dd-849904bbffb2 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:07:28.786315 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d0017134-89ef-4c3e-81dd-849904bbffb2 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:07:28.787363 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d0017134-89ef-4c3e-81dd-849904bbffb2 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:07:28.787815 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d0017134-89ef-4c3e-81dd-849904bbffb2 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:07:28.941232 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-d0017134-89ef-4c3e-81dd-849904bbffb2 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b91f7cf2a74f45858448608b096b0c31', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1617cbc6e7cf4eed8d2f8968da996b9c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:07:28.948871 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-d0017134-89ef-4c3e-81dd-849904bbffb2 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] 10.222.0.16 "GET /compute/v2.1/servers/dc0abe67-6867-4794-aaa6-5235d45edf2a" status: 200 len: 1316 microversion: 2.1 time: 0.176677 Jul 16 22:07:28.949477 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3380/6752] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:07:28 2022] GET /compute/v2.1/servers/dc0abe67-6867-4794-aaa6-5235d45edf2a => generated 1316 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:07:29.962950 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell0:SELECT=2 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:07:29.963766 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell0:SELECT=6 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:07:29.969118 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-4d505fa5-71e5-4e3f-8aaf-e65325f6d133 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:07:29.969826 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-4d505fa5-71e5-4e3f-8aaf-e65325f6d133 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] [instance: dc0abe67-6867-4794-aaa6-5235d45edf2a] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:07:29.976051 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4d505fa5-71e5-4e3f-8aaf-e65325f6d133 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:07:29.976509 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4d505fa5-71e5-4e3f-8aaf-e65325f6d133 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:07:29.981480 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4d505fa5-71e5-4e3f-8aaf-e65325f6d133 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:07:29.981480 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4d505fa5-71e5-4e3f-8aaf-e65325f6d133 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:07:30.112569 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-4d505fa5-71e5-4e3f-8aaf-e65325f6d133 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b91f7cf2a74f45858448608b096b0c31', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1617cbc6e7cf4eed8d2f8968da996b9c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:07:30.120967 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-4d505fa5-71e5-4e3f-8aaf-e65325f6d133 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] 10.222.0.16 "GET /compute/v2.1/servers/dc0abe67-6867-4794-aaa6-5235d45edf2a" status: 200 len: 1316 microversion: 2.1 time: 0.158988 Jul 16 22:07:30.121536 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3373/6753] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:07:29 2022] GET /compute/v2.1/servers/dc0abe67-6867-4794-aaa6-5235d45edf2a => generated 1316 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:07:31.138330 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-5d9d02cc-c637-4ad2-9b7b-f0e4bfe95f9b tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:07:31.140743 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-5d9d02cc-c637-4ad2-9b7b-f0e4bfe95f9b tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] [instance: dc0abe67-6867-4794-aaa6-5235d45edf2a] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:07:31.141186 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell0:SELECT=2 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:07:31.146902 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5d9d02cc-c637-4ad2-9b7b-f0e4bfe95f9b tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:07:31.147406 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5d9d02cc-c637-4ad2-9b7b-f0e4bfe95f9b tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:07:31.148229 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5d9d02cc-c637-4ad2-9b7b-f0e4bfe95f9b tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:07:31.148688 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5d9d02cc-c637-4ad2-9b7b-f0e4bfe95f9b tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:07:31.284454 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-5d9d02cc-c637-4ad2-9b7b-f0e4bfe95f9b tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b91f7cf2a74f45858448608b096b0c31', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1617cbc6e7cf4eed8d2f8968da996b9c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:07:31.291989 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-5d9d02cc-c637-4ad2-9b7b-f0e4bfe95f9b tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] 10.222.0.16 "GET /compute/v2.1/servers/dc0abe67-6867-4794-aaa6-5235d45edf2a" status: 200 len: 1505 microversion: 2.1 time: 0.158178 Jul 16 22:07:31.292559 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3381/6754] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:07:31 2022] GET /compute/v2.1/servers/dc0abe67-6867-4794-aaa6-5235d45edf2a => generated 1505 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:07:31.345951 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-f9bc1179-05a6-4812-9e71-c2adc106ba05 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServersNegativeTestJSON-server-1157294576", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452", "networks": [{"uuid": "51ae6879-cb74-494d-a439-27c219365dd7"}]}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:07:31.400241 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.network.neutron [None req-f9bc1179-05a6-4812-9e71-c2adc106ba05 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] validate_networks() for [('51ae6879-cb74-494d-a439-27c219365dd7', None, None, None, None, None)] {{(pid=138680) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 16 22:07:31.776118 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-f9bc1179-05a6-4812-9e71-c2adc106ba05 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Flavor limits 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 16 22:07:31.779736 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-f9bc1179-05a6-4812-9e71-c2adc106ba05 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Image limits 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 16 22:07:31.779736 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-f9bc1179-05a6-4812-9e71-c2adc106ba05 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Flavor pref 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 16 22:07:31.779736 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-f9bc1179-05a6-4812-9e71-c2adc106ba05 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Image pref 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 16 22:07:31.779736 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-f9bc1179-05a6-4812-9e71-c2adc106ba05 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 16 22:07:31.786739 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f9bc1179-05a6-4812-9e71-c2adc106ba05 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:07:31.787243 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f9bc1179-05a6-4812-9e71-c2adc106ba05 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:07:31.788044 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f9bc1179-05a6-4812-9e71-c2adc106ba05 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:07:31.788484 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f9bc1179-05a6-4812-9e71-c2adc106ba05 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:07:31.805555 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-f9bc1179-05a6-4812-9e71-c2adc106ba05 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Getting quotas for project 1617cbc6e7cf4eed8d2f8968da996b9c. Resources: {'ram', 'instances', 'cores'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 22:07:31.811595 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-f9bc1179-05a6-4812-9e71-c2adc106ba05 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Getting quotas for user b91f7cf2a74f45858448608b096b0c31 and project 1617cbc6e7cf4eed8d2f8968da996b9c. Resources: {'ram', 'instances', 'cores'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 22:07:31.824170 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-f9bc1179-05a6-4812-9e71-c2adc106ba05 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Going to run 1 instances... {{(pid=138680) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 16 22:07:31.826833 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-f9bc1179-05a6-4812-9e71-c2adc106ba05 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] [instance: c918af94-53fe-4a9b-9b3f-b9829038314f] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138680) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 16 22:07:31.950443 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-f9bc1179-05a6-4812-9e71-c2adc106ba05 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.608273 Jul 16 22:07:31.951413 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3374/6755] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 22:07:31 2022] POST /compute/v2.1/servers => generated 380 bytes in 610 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 16 22:07:31.961273 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-647fef3f-3957-4f1f-9a98-d917b0544780 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:07:31.961900 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-647fef3f-3957-4f1f-9a98-d917b0544780 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] [instance: c918af94-53fe-4a9b-9b3f-b9829038314f] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:07:32.017099 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-647fef3f-3957-4f1f-9a98-d917b0544780 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b91f7cf2a74f45858448608b096b0c31', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1617cbc6e7cf4eed8d2f8968da996b9c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:07:32.024225 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-647fef3f-3957-4f1f-9a98-d917b0544780 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] 10.222.0.16 "GET /compute/v2.1/servers/c918af94-53fe-4a9b-9b3f-b9829038314f" status: 200 len: 1217 microversion: 2.1 time: 0.066682 Jul 16 22:07:32.029641 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3382/6756] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:07:31 2022] GET /compute/v2.1/servers/c918af94-53fe-4a9b-9b3f-b9829038314f => generated 1217 bytes in 68 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:07:33.044199 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-447149d6-729a-458c-b245-d3535578a33e tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:07:33.045622 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-447149d6-729a-458c-b245-d3535578a33e tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] [instance: c918af94-53fe-4a9b-9b3f-b9829038314f] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:07:33.051949 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-447149d6-729a-458c-b245-d3535578a33e tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:07:33.052406 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-447149d6-729a-458c-b245-d3535578a33e tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:07:33.053168 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-447149d6-729a-458c-b245-d3535578a33e tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:07:33.053609 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-447149d6-729a-458c-b245-d3535578a33e tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:07:33.133416 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-447149d6-729a-458c-b245-d3535578a33e tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b91f7cf2a74f45858448608b096b0c31', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1617cbc6e7cf4eed8d2f8968da996b9c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:07:33.135615 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-447149d6-729a-458c-b245-d3535578a33e tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] 10.222.0.16 "GET /compute/v2.1/servers/c918af94-53fe-4a9b-9b3f-b9829038314f" status: 200 len: 1221 microversion: 2.1 time: 0.097524 Jul 16 22:07:33.136182 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3375/6757] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:07:33 2022] GET /compute/v2.1/servers/c918af94-53fe-4a9b-9b3f-b9829038314f => generated 1221 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:07:34.154212 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-798aea4f-ac62-4176-9c51-78e052c62de3 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:07:34.155027 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-798aea4f-ac62-4176-9c51-78e052c62de3 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] [instance: c918af94-53fe-4a9b-9b3f-b9829038314f] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:07:34.161987 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-798aea4f-ac62-4176-9c51-78e052c62de3 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:07:34.162506 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-798aea4f-ac62-4176-9c51-78e052c62de3 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:07:34.163409 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-798aea4f-ac62-4176-9c51-78e052c62de3 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:07:34.163776 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-798aea4f-ac62-4176-9c51-78e052c62de3 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:07:34.232308 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-798aea4f-ac62-4176-9c51-78e052c62de3 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b91f7cf2a74f45858448608b096b0c31', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1617cbc6e7cf4eed8d2f8968da996b9c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:07:34.240079 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-798aea4f-ac62-4176-9c51-78e052c62de3 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] 10.222.0.16 "GET /compute/v2.1/servers/c918af94-53fe-4a9b-9b3f-b9829038314f" status: 200 len: 1277 microversion: 2.1 time: 0.090828 Jul 16 22:07:34.240691 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3383/6758] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:07:34 2022] GET /compute/v2.1/servers/c918af94-53fe-4a9b-9b3f-b9829038314f => generated 1277 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:07:35.257521 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-0110f066-d7dd-4619-ac81-9c8894d08fc0 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:07:35.258131 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-0110f066-d7dd-4619-ac81-9c8894d08fc0 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] [instance: c918af94-53fe-4a9b-9b3f-b9829038314f] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:07:35.265237 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0110f066-d7dd-4619-ac81-9c8894d08fc0 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:07:35.265627 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0110f066-d7dd-4619-ac81-9c8894d08fc0 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:07:35.266397 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0110f066-d7dd-4619-ac81-9c8894d08fc0 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:07:35.266836 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0110f066-d7dd-4619-ac81-9c8894d08fc0 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:07:35.400505 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-0110f066-d7dd-4619-ac81-9c8894d08fc0 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b91f7cf2a74f45858448608b096b0c31', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1617cbc6e7cf4eed8d2f8968da996b9c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:07:35.409285 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-0110f066-d7dd-4619-ac81-9c8894d08fc0 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] 10.222.0.16 "GET /compute/v2.1/servers/c918af94-53fe-4a9b-9b3f-b9829038314f" status: 200 len: 1317 microversion: 2.1 time: 0.155921 Jul 16 22:07:35.410041 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3376/6759] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:07:35 2022] GET /compute/v2.1/servers/c918af94-53fe-4a9b-9b3f-b9829038314f => generated 1317 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:07:35.810056 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-622b5d52-580b-4911-a590-49425fed90ff req-01791250-b80a-4e69-a456-e8c963da2f5f service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "c918af94-53fe-4a9b-9b3f-b9829038314f", "tag": "ad2a167a-899d-4599-bb7d-fbcdca4cf0ce"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:07:35.821020 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-622b5d52-580b-4911-a590-49425fed90ff req-01791250-b80a-4e69-a456-e8c963da2f5f service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:07:35.821897 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-622b5d52-580b-4911-a590-49425fed90ff req-01791250-b80a-4e69-a456-e8c963da2f5f service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:07:35.835868 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-622b5d52-580b-4911-a590-49425fed90ff req-01791250-b80a-4e69-a456-e8c963da2f5f service nova] Creating event network-changed:ad2a167a-899d-4599-bb7d-fbcdca4cf0ce for instance c918af94-53fe-4a9b-9b3f-b9829038314f on nch1-master-1 Jul 16 22:07:35.841230 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-622b5d52-580b-4911-a590-49425fed90ff req-01791250-b80a-4e69-a456-e8c963da2f5f service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.093657 Jul 16 22:07:35.842155 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3384/6760] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 22:07:35 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 22:07:36.427654 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-49b5cf62-11bf-4171-b928-28f8b7849886 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:07:36.428293 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-49b5cf62-11bf-4171-b928-28f8b7849886 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] [instance: c918af94-53fe-4a9b-9b3f-b9829038314f] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:07:36.435919 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-49b5cf62-11bf-4171-b928-28f8b7849886 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:07:36.436375 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-49b5cf62-11bf-4171-b928-28f8b7849886 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:07:36.437148 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-49b5cf62-11bf-4171-b928-28f8b7849886 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:07:36.437594 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-49b5cf62-11bf-4171-b928-28f8b7849886 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:07:36.566702 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-49b5cf62-11bf-4171-b928-28f8b7849886 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b91f7cf2a74f45858448608b096b0c31', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1617cbc6e7cf4eed8d2f8968da996b9c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:07:36.574849 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-49b5cf62-11bf-4171-b928-28f8b7849886 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] 10.222.0.16 "GET /compute/v2.1/servers/c918af94-53fe-4a9b-9b3f-b9829038314f" status: 200 len: 1317 microversion: 2.1 time: 0.150605 Jul 16 22:07:36.576046 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3377/6761] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:07:36 2022] GET /compute/v2.1/servers/c918af94-53fe-4a9b-9b3f-b9829038314f => generated 1317 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:07:37.592978 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-bba61ba2-1758-493e-a658-23e959bf5a97 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:07:37.593740 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-bba61ba2-1758-493e-a658-23e959bf5a97 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] [instance: c918af94-53fe-4a9b-9b3f-b9829038314f] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:07:37.600118 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-bba61ba2-1758-493e-a658-23e959bf5a97 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:07:37.600590 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-bba61ba2-1758-493e-a658-23e959bf5a97 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:07:37.601371 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-bba61ba2-1758-493e-a658-23e959bf5a97 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:07:37.601830 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-bba61ba2-1758-493e-a658-23e959bf5a97 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:07:37.737219 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-bba61ba2-1758-493e-a658-23e959bf5a97 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b91f7cf2a74f45858448608b096b0c31', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1617cbc6e7cf4eed8d2f8968da996b9c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:07:37.744751 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-bba61ba2-1758-493e-a658-23e959bf5a97 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] 10.222.0.16 "GET /compute/v2.1/servers/c918af94-53fe-4a9b-9b3f-b9829038314f" status: 200 len: 1317 microversion: 2.1 time: 0.156925 Jul 16 22:07:37.745336 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3385/6762] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:07:37 2022] GET /compute/v2.1/servers/c918af94-53fe-4a9b-9b3f-b9829038314f => generated 1317 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:07:38.762504 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-3d3550d4-e016-4237-8b3d-dcb9aeca0ec6 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:07:38.763151 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-3d3550d4-e016-4237-8b3d-dcb9aeca0ec6 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] [instance: c918af94-53fe-4a9b-9b3f-b9829038314f] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:07:38.770197 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3d3550d4-e016-4237-8b3d-dcb9aeca0ec6 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:07:38.770659 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3d3550d4-e016-4237-8b3d-dcb9aeca0ec6 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:07:38.771471 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3d3550d4-e016-4237-8b3d-dcb9aeca0ec6 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:07:38.771924 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3d3550d4-e016-4237-8b3d-dcb9aeca0ec6 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:07:38.905754 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-3d3550d4-e016-4237-8b3d-dcb9aeca0ec6 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b91f7cf2a74f45858448608b096b0c31', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1617cbc6e7cf4eed8d2f8968da996b9c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:07:38.913550 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-3d3550d4-e016-4237-8b3d-dcb9aeca0ec6 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] 10.222.0.16 "GET /compute/v2.1/servers/c918af94-53fe-4a9b-9b3f-b9829038314f" status: 200 len: 1317 microversion: 2.1 time: 0.155601 Jul 16 22:07:38.914121 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3378/6763] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:07:38 2022] GET /compute/v2.1/servers/c918af94-53fe-4a9b-9b3f-b9829038314f => generated 1317 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:07:39.930664 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-46701a59-660e-4ae2-8c23-1fa5ae7f20ae tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:07:39.931332 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-46701a59-660e-4ae2-8c23-1fa5ae7f20ae tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] [instance: c918af94-53fe-4a9b-9b3f-b9829038314f] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:07:39.938472 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-46701a59-660e-4ae2-8c23-1fa5ae7f20ae tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:07:39.938945 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-46701a59-660e-4ae2-8c23-1fa5ae7f20ae tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:07:39.939799 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-46701a59-660e-4ae2-8c23-1fa5ae7f20ae tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:07:39.940291 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-46701a59-660e-4ae2-8c23-1fa5ae7f20ae tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:07:40.073642 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-46701a59-660e-4ae2-8c23-1fa5ae7f20ae tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b91f7cf2a74f45858448608b096b0c31', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1617cbc6e7cf4eed8d2f8968da996b9c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:07:40.082169 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-46701a59-660e-4ae2-8c23-1fa5ae7f20ae tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] 10.222.0.16 "GET /compute/v2.1/servers/c918af94-53fe-4a9b-9b3f-b9829038314f" status: 200 len: 1317 microversion: 2.1 time: 0.155735 Jul 16 22:07:40.082749 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3386/6764] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:07:39 2022] GET /compute/v2.1/servers/c918af94-53fe-4a9b-9b3f-b9829038314f => generated 1317 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:07:40.842838 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-e1a65621-e5c5-40fe-9da4-bd0b275aac03 req-f2cdc699-e72d-4d6b-b853-2657cfb783ba service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "c918af94-53fe-4a9b-9b3f-b9829038314f", "name": "network-vif-plugged", "status": "completed", "tag": "ad2a167a-899d-4599-bb7d-fbcdca4cf0ce"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:07:40.851897 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-e1a65621-e5c5-40fe-9da4-bd0b275aac03 req-f2cdc699-e72d-4d6b-b853-2657cfb783ba service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:07:40.854073 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-e1a65621-e5c5-40fe-9da4-bd0b275aac03 req-f2cdc699-e72d-4d6b-b853-2657cfb783ba service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:07:40.863811 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-e1a65621-e5c5-40fe-9da4-bd0b275aac03 req-f2cdc699-e72d-4d6b-b853-2657cfb783ba service nova] Creating event network-vif-plugged:ad2a167a-899d-4599-bb7d-fbcdca4cf0ce for instance c918af94-53fe-4a9b-9b3f-b9829038314f on nch1-master-1 Jul 16 22:07:40.868789 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-e1a65621-e5c5-40fe-9da4-bd0b275aac03 req-f2cdc699-e72d-4d6b-b853-2657cfb783ba service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.029928 Jul 16 22:07:40.869389 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3379/6765] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 22:07:40 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 31 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 22:07:41.100056 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-ec0709f8-ce1e-4052-92a6-654aff1f74a0 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:07:41.100766 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-ec0709f8-ce1e-4052-92a6-654aff1f74a0 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] [instance: c918af94-53fe-4a9b-9b3f-b9829038314f] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:07:41.108957 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ec0709f8-ce1e-4052-92a6-654aff1f74a0 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:07:41.109434 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ec0709f8-ce1e-4052-92a6-654aff1f74a0 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:07:41.110233 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ec0709f8-ce1e-4052-92a6-654aff1f74a0 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:07:41.110795 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ec0709f8-ce1e-4052-92a6-654aff1f74a0 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:07:41.336145 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-ec0709f8-ce1e-4052-92a6-654aff1f74a0 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b91f7cf2a74f45858448608b096b0c31', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1617cbc6e7cf4eed8d2f8968da996b9c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:07:41.344545 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-ec0709f8-ce1e-4052-92a6-654aff1f74a0 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] 10.222.0.16 "GET /compute/v2.1/servers/c918af94-53fe-4a9b-9b3f-b9829038314f" status: 200 len: 1317 microversion: 2.1 time: 0.248514 Jul 16 22:07:41.354232 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3387/6766] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:07:41 2022] GET /compute/v2.1/servers/c918af94-53fe-4a9b-9b3f-b9829038314f => generated 1317 bytes in 259 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:07:42.359493 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell0:SELECT=2 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:07:42.360300 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell0:SELECT=6 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:07:42.365298 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-4832f202-b564-4c1a-aca0-dddd90ecbd44 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:07:42.366344 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-4832f202-b564-4c1a-aca0-dddd90ecbd44 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] [instance: c918af94-53fe-4a9b-9b3f-b9829038314f] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:07:42.372355 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4832f202-b564-4c1a-aca0-dddd90ecbd44 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:07:42.373498 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4832f202-b564-4c1a-aca0-dddd90ecbd44 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:07:42.374273 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4832f202-b564-4c1a-aca0-dddd90ecbd44 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:07:42.374720 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4832f202-b564-4c1a-aca0-dddd90ecbd44 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:07:42.510163 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-4832f202-b564-4c1a-aca0-dddd90ecbd44 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b91f7cf2a74f45858448608b096b0c31', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1617cbc6e7cf4eed8d2f8968da996b9c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:07:42.518008 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-4832f202-b564-4c1a-aca0-dddd90ecbd44 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] 10.222.0.16 "GET /compute/v2.1/servers/c918af94-53fe-4a9b-9b3f-b9829038314f" status: 200 len: 1506 microversion: 2.1 time: 0.159427 Jul 16 22:07:42.518519 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3380/6767] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:07:42 2022] GET /compute/v2.1/servers/c918af94-53fe-4a9b-9b3f-b9829038314f => generated 1506 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 22:07:42.534912 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-153d2874-af21-429c-a0b2-8372fc9c28f2 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:07:42.535459 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-153d2874-af21-429c-a0b2-8372fc9c28f2 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] [instance: c918af94-53fe-4a9b-9b3f-b9829038314f] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:07:42.537511 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell0:SELECT=2 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:07:42.543247 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-153d2874-af21-429c-a0b2-8372fc9c28f2 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:07:42.543722 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-153d2874-af21-429c-a0b2-8372fc9c28f2 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:07:42.544496 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-153d2874-af21-429c-a0b2-8372fc9c28f2 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:07:42.545577 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-153d2874-af21-429c-a0b2-8372fc9c28f2 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:07:42.573140 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-153d2874-af21-429c-a0b2-8372fc9c28f2 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] [instance: c918af94-53fe-4a9b-9b3f-b9829038314f] Going to try to terminate instance {{(pid=138681) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 16 22:07:42.928943 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-153d2874-af21-429c-a0b2-8372fc9c28f2 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] 10.222.0.16 "DELETE /compute/v2.1/servers/c918af94-53fe-4a9b-9b3f-b9829038314f" status: 204 len: 0 microversion: 2.1 time: 0.397677 Jul 16 22:07:42.929577 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3388/6768] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 22:07:42 2022] DELETE /compute/v2.1/servers/c918af94-53fe-4a9b-9b3f-b9829038314f => generated 0 bytes in 399 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Jul 16 22:07:42.937922 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-accd85f2-34a2-4099-8c68-0e2e48567995 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:07:42.938545 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-accd85f2-34a2-4099-8c68-0e2e48567995 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] [instance: c918af94-53fe-4a9b-9b3f-b9829038314f] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:07:42.944707 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-accd85f2-34a2-4099-8c68-0e2e48567995 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:07:42.945159 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-accd85f2-34a2-4099-8c68-0e2e48567995 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:07:42.945927 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-accd85f2-34a2-4099-8c68-0e2e48567995 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:07:42.946370 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-accd85f2-34a2-4099-8c68-0e2e48567995 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:07:43.079974 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-accd85f2-34a2-4099-8c68-0e2e48567995 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b91f7cf2a74f45858448608b096b0c31', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1617cbc6e7cf4eed8d2f8968da996b9c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:07:43.087655 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-accd85f2-34a2-4099-8c68-0e2e48567995 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] 10.222.0.16 "GET /compute/v2.1/servers/c918af94-53fe-4a9b-9b3f-b9829038314f" status: 200 len: 1512 microversion: 2.1 time: 0.153847 Jul 16 22:07:43.088246 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3381/6769] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:07:42 2022] GET /compute/v2.1/servers/c918af94-53fe-4a9b-9b3f-b9829038314f => generated 1512 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:07:44.108227 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-91d56788-cca5-4f7a-afac-70597172aed8 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:07:44.108968 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-91d56788-cca5-4f7a-afac-70597172aed8 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] [instance: c918af94-53fe-4a9b-9b3f-b9829038314f] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:07:44.116820 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-91d56788-cca5-4f7a-afac-70597172aed8 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:07:44.116820 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-91d56788-cca5-4f7a-afac-70597172aed8 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:07:44.117129 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-91d56788-cca5-4f7a-afac-70597172aed8 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:07:44.117583 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-91d56788-cca5-4f7a-afac-70597172aed8 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:07:44.259911 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-91d56788-cca5-4f7a-afac-70597172aed8 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b91f7cf2a74f45858448608b096b0c31', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1617cbc6e7cf4eed8d2f8968da996b9c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:07:44.268385 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-91d56788-cca5-4f7a-afac-70597172aed8 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] 10.222.0.16 "GET /compute/v2.1/servers/c918af94-53fe-4a9b-9b3f-b9829038314f" status: 200 len: 1512 microversion: 2.1 time: 0.165247 Jul 16 22:07:44.269000 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3389/6770] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:07:44 2022] GET /compute/v2.1/servers/c918af94-53fe-4a9b-9b3f-b9829038314f => generated 1512 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:07:45.019730 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-34cd09b9-b3eb-457e-8357-4df0519eb69a req-cd281711-1d07-4388-a795-1c26c653f804 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "c918af94-53fe-4a9b-9b3f-b9829038314f", "name": "network-vif-deleted", "tag": "ad2a167a-899d-4599-bb7d-fbcdca4cf0ce"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:07:45.031616 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-34cd09b9-b3eb-457e-8357-4df0519eb69a req-cd281711-1d07-4388-a795-1c26c653f804 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:07:45.032435 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-34cd09b9-b3eb-457e-8357-4df0519eb69a req-cd281711-1d07-4388-a795-1c26c653f804 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:07:45.049278 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-34cd09b9-b3eb-457e-8357-4df0519eb69a req-cd281711-1d07-4388-a795-1c26c653f804 service nova] Creating event network-vif-deleted:ad2a167a-899d-4599-bb7d-fbcdca4cf0ce for instance c918af94-53fe-4a9b-9b3f-b9829038314f on nch1-master-1 Jul 16 22:07:45.054890 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-34cd09b9-b3eb-457e-8357-4df0519eb69a req-cd281711-1d07-4388-a795-1c26c653f804 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.038536 Jul 16 22:07:45.055664 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3382/6771] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 22:07:45 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 16 22:07:45.285653 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-20e842f2-f737-444c-a839-e36da664951f tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:07:45.286205 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-20e842f2-f737-444c-a839-e36da664951f tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] [instance: c918af94-53fe-4a9b-9b3f-b9829038314f] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:07:45.293574 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-20e842f2-f737-444c-a839-e36da664951f tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:07:45.294081 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-20e842f2-f737-444c-a839-e36da664951f tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:07:45.294915 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-20e842f2-f737-444c-a839-e36da664951f tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:07:45.295531 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-20e842f2-f737-444c-a839-e36da664951f tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:07:45.361035 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-20e842f2-f737-444c-a839-e36da664951f tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b91f7cf2a74f45858448608b096b0c31', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1617cbc6e7cf4eed8d2f8968da996b9c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:07:45.368759 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-20e842f2-f737-444c-a839-e36da664951f tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] 10.222.0.16 "GET /compute/v2.1/servers/c918af94-53fe-4a9b-9b3f-b9829038314f" status: 200 len: 1302 microversion: 2.1 time: 0.087263 Jul 16 22:07:45.369345 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3390/6772] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:07:45 2022] GET /compute/v2.1/servers/c918af94-53fe-4a9b-9b3f-b9829038314f => generated 1302 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:07:46.386762 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-26791119-415c-4c6a-8aab-53ba20b7a6ce tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:07:46.387547 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-26791119-415c-4c6a-8aab-53ba20b7a6ce tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] [instance: c918af94-53fe-4a9b-9b3f-b9829038314f] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:07:46.395476 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-26791119-415c-4c6a-8aab-53ba20b7a6ce tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:07:46.395936 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-26791119-415c-4c6a-8aab-53ba20b7a6ce tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:07:46.396702 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-26791119-415c-4c6a-8aab-53ba20b7a6ce tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:07:46.397147 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-26791119-415c-4c6a-8aab-53ba20b7a6ce tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:07:46.410652 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-26791119-415c-4c6a-8aab-53ba20b7a6ce tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] HTTP exception thrown: Instance c918af94-53fe-4a9b-9b3f-b9829038314f could not be found. Jul 16 22:07:46.412431 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-26791119-415c-4c6a-8aab-53ba20b7a6ce tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Request method failure captured: Jul 16 22:07:46.412431 ncdv-master-1 devstack@n-api.service[138680]: request: GET /compute/v2.1/servers/c918af94-53fe-4a9b-9b3f-b9829038314f HTTP/1.1 Jul 16 22:07:46.412431 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 22:07:46.412431 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 22:07:46.412431 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 22:07:46.412431 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 0 Jul 16 22:07:46.412431 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 22:07:46.412431 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 22:07:46.412431 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 22:07:46.412431 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 22:07:46.412431 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0zaW0UoY6UJ6FJHYDQ4r97OlyPnvxU6e2w9qGk5WCAayEbLeZeNRgzNo9liblYswSYqcMHs4F3E_P-9GEAa6A2r4CdzhoZvP1848KJhQMPdK0xmvHKxA1-TYGf2CGi8ZFNp8FEAXnHHm0r0NZNndypHrkZ6k6Q41f-GxWs_uQ1YL9wo Jul 16 22:07:46.412431 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 22:07:46.412431 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 22:07:46.412431 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 22:07:46.412431 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 22:07:46.412431 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 22:07:46.412431 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 22:07:46.412431 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: 1617cbc6e7cf4eed8d2f8968da996b9c Jul 16 22:07:46.412431 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-ServersNegativeTestJSON-95771019 Jul 16 22:07:46.412431 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 22:07:46.412431 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 22:07:46.412431 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/1617cbc6e7cf4eed8d2f8968da996b9c"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/1617cbc6e7cf4eed8d2f8968da996b9c"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_1617cbc6e7cf4eed8d2f8968da996b9c"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/1617cbc6e7cf4eed8d2f8968da996b9c"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 22:07:46.412431 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-ServersNegativeTestJSON-95771019 Jul 16 22:07:46.412431 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: 1617cbc6e7cf4eed8d2f8968da996b9c Jul 16 22:07:46.412431 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-ServersNegativeTestJSON-95771019 Jul 16 22:07:46.412431 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-ServersNegativeTestJSON-95771019-project Jul 16 22:07:46.412431 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 22:07:46.412431 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 22:07:46.412431 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: b91f7cf2a74f45858448608b096b0c31 Jul 16 22:07:46.412431 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-ServersNegativeTestJSON-95771019-project Jul 16 22:07:46.412431 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 22:07:46.412431 ncdv-master-1 devstack@n-api.service[138680]: exception: Instance c918af94-53fe-4a9b-9b3f-b9829038314f could not be found. Jul 16 22:07:46.412431 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': 'c918af94-53fe-4a9b-9b3f-b9829038314f'} Jul 16 22:07:46.412431 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 22:07:46.412431 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 22:07:46.412431 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 22:07:46.412431 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 22:07:46.412431 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 22:07:46.412431 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 22:07:46.412431 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 22:07:46.412431 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 22:07:46.412431 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 22:07:46.412431 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 22:07:46.412431 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance c918af94-53fe-4a9b-9b3f-b9829038314f could not be found. Jul 16 22:07:46.412431 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 22:07:46.412431 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 22:07:46.412431 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 22:07:46.412431 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 22:07:46.412431 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 22:07:46.412431 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 22:07:46.412431 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 22:07:46.412431 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 22:07:46.412431 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 22:07:46.412431 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 22:07:46.412431 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 22:07:46.412431 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 22:07:46.412431 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 22:07:46.412431 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 22:07:46.412431 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 22:07:46.412431 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 22:07:46.412431 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance c918af94-53fe-4a9b-9b3f-b9829038314f could not be found. Jul 16 22:07:46.412431 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 22:07:46.412431 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 22:07:46.412431 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 22:07:46.419886 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 22:07:46.419886 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 22:07:46.419886 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 22:07:46.419886 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 22:07:46.419886 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 22:07:46.419886 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance c918af94-53fe-4a9b-9b3f-b9829038314f could not be found. Jul 16 22:07:46.419886 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 22:07:46.419886 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-26791119-415c-4c6a-8aab-53ba20b7a6ce tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Returning 404 to user: Instance c918af94-53fe-4a9b-9b3f-b9829038314f could not be found. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 22:07:46.419886 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-26791119-415c-4c6a-8aab-53ba20b7a6ce tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] 10.222.0.16 "GET /compute/v2.1/servers/c918af94-53fe-4a9b-9b3f-b9829038314f" status: 404 len: 111 microversion: 2.1 time: 0.031443 Jul 16 22:07:46.419886 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3383/6773] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:07:46 2022] GET /compute/v2.1/servers/c918af94-53fe-4a9b-9b3f-b9829038314f => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 22:07:46.423252 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-40a560a2-0d03-45a7-b708-4d76174af369 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:07:46.423879 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-40a560a2-0d03-45a7-b708-4d76174af369 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] [instance: dc0abe67-6867-4794-aaa6-5235d45edf2a] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:07:46.431691 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-40a560a2-0d03-45a7-b708-4d76174af369 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:07:46.432279 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-40a560a2-0d03-45a7-b708-4d76174af369 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:07:46.433077 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-40a560a2-0d03-45a7-b708-4d76174af369 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:07:46.433542 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-40a560a2-0d03-45a7-b708-4d76174af369 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:07:46.571130 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-40a560a2-0d03-45a7-b708-4d76174af369 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b91f7cf2a74f45858448608b096b0c31', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1617cbc6e7cf4eed8d2f8968da996b9c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:07:46.579127 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-40a560a2-0d03-45a7-b708-4d76174af369 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] 10.222.0.16 "GET /compute/v2.1/servers/dc0abe67-6867-4794-aaa6-5235d45edf2a" status: 200 len: 1505 microversion: 2.1 time: 0.159918 Jul 16 22:07:46.579712 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3391/6774] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:07:46 2022] GET /compute/v2.1/servers/dc0abe67-6867-4794-aaa6-5235d45edf2a => generated 1505 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:07:46.595884 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-7e15b107-1e03-40b4-a621-557c089d33fc tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Action: 'action', calling method: >, body: {"resize": {"flavorRef": "452"}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:07:46.596857 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-7e15b107-1e03-40b4-a621-557c089d33fc tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] [instance: c7f025ec-1d4c-422d-938a-de0c38b24343] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:07:46.602377 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-7e15b107-1e03-40b4-a621-557c089d33fc tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] HTTP exception thrown: Instance c7f025ec-1d4c-422d-938a-de0c38b24343 could not be found. Jul 16 22:07:46.603946 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-7e15b107-1e03-40b4-a621-557c089d33fc tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Request method failure captured: Jul 16 22:07:46.603946 ncdv-master-1 devstack@n-api.service[138680]: request: POST /compute/v2.1/servers/c7f025ec-1d4c-422d-938a-de0c38b24343/action HTTP/1.1 Jul 16 22:07:46.603946 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 22:07:46.603946 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 22:07:46.603946 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 22:07:46.603946 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 32 Jul 16 22:07:46.603946 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 22:07:46.603946 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 22:07:46.603946 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 22:07:46.603946 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 22:07:46.603946 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0zaW0UoY6UJ6FJHYDQ4r97OlyPnvxU6e2w9qGk5WCAayEbLeZeNRgzNo9liblYswSYqcMHs4F3E_P-9GEAa6A2r4CdzhoZvP1848KJhQMPdK0xmvHKxA1-TYGf2CGi8ZFNp8FEAXnHHm0r0NZNndypHrkZ6k6Q41f-GxWs_uQ1YL9wo Jul 16 22:07:46.603946 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 22:07:46.603946 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 22:07:46.603946 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 22:07:46.603946 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 22:07:46.603946 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 22:07:46.603946 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 22:07:46.603946 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: 1617cbc6e7cf4eed8d2f8968da996b9c Jul 16 22:07:46.603946 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-ServersNegativeTestJSON-95771019 Jul 16 22:07:46.603946 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 22:07:46.603946 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 22:07:46.603946 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/1617cbc6e7cf4eed8d2f8968da996b9c"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/1617cbc6e7cf4eed8d2f8968da996b9c"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_1617cbc6e7cf4eed8d2f8968da996b9c"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/1617cbc6e7cf4eed8d2f8968da996b9c"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 22:07:46.603946 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-ServersNegativeTestJSON-95771019 Jul 16 22:07:46.603946 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: 1617cbc6e7cf4eed8d2f8968da996b9c Jul 16 22:07:46.603946 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-ServersNegativeTestJSON-95771019 Jul 16 22:07:46.603946 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-ServersNegativeTestJSON-95771019-project Jul 16 22:07:46.603946 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 22:07:46.603946 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 22:07:46.603946 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: b91f7cf2a74f45858448608b096b0c31 Jul 16 22:07:46.603946 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-ServersNegativeTestJSON-95771019-project Jul 16 22:07:46.603946 ncdv-master-1 devstack@n-api.service[138680]: Jul 16 22:07:46.603946 ncdv-master-1 devstack@n-api.service[138680]: {"resize": {"flavorRef": "452"}} Jul 16 22:07:46.603946 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 22:07:46.603946 ncdv-master-1 devstack@n-api.service[138680]: exception: Instance c7f025ec-1d4c-422d-938a-de0c38b24343 could not be found. Jul 16 22:07:46.603946 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': 'c7f025ec-1d4c-422d-938a-de0c38b24343', 'body': {'resize': {'flavorRef': '452'}}} Jul 16 22:07:46.603946 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 22:07:46.603946 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 22:07:46.603946 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 22:07:46.603946 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 22:07:46.603946 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 22:07:46.603946 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 22:07:46.603946 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2898, in _get_instance Jul 16 22:07:46.603946 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=instance_uuid) Jul 16 22:07:46.603946 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance c7f025ec-1d4c-422d-938a-de0c38b24343 could not be found. Jul 16 22:07:46.603946 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 22:07:46.603946 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 22:07:46.603946 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 22:07:46.603946 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 22:07:46.603946 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 22:07:46.603946 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 22:07:46.603946 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 22:07:46.603946 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 22:07:46.603946 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 22:07:46.603946 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 22:07:46.603946 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 16 22:07:46.603946 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 16 22:07:46.603946 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1139, in _action_resize Jul 16 22:07:46.603946 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi self._resize(req, id, flavor_ref, **kwargs) Jul 16 22:07:46.603946 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1046, in _resize Jul 16 22:07:46.603946 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_id, Jul 16 22:07:46.603946 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 22:07:46.603946 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 22:07:46.603946 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 22:07:46.603946 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 22:07:46.609595 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance c7f025ec-1d4c-422d-938a-de0c38b24343 could not be found. Jul 16 22:07:46.609595 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 22:07:46.609595 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 22:07:46.609595 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 22:07:46.609595 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 22:07:46.609595 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 22:07:46.609595 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 22:07:46.609595 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 22:07:46.609595 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 22:07:46.609595 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance c7f025ec-1d4c-422d-938a-de0c38b24343 could not be found. Jul 16 22:07:46.609595 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 22:07:46.609595 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-7e15b107-1e03-40b4-a621-557c089d33fc tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Returning 404 to user: Instance c7f025ec-1d4c-422d-938a-de0c38b24343 could not be found. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 22:07:46.609595 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-7e15b107-1e03-40b4-a621-557c089d33fc tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] 10.222.0.16 "POST /compute/v2.1/servers/c7f025ec-1d4c-422d-938a-de0c38b24343/action" status: 404 len: 111 microversion: 2.1 time: 0.013641 Jul 16 22:07:46.609595 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3384/6775] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 22:07:46 2022] POST /compute/v2.1/servers/c7f025ec-1d4c-422d-938a-de0c38b24343/action => generated 111 bytes in 15 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 22:07:46.614916 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-bebebb35-1cd8-48dd-8f09-9313e9530342 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:07:46.615589 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-bebebb35-1cd8-48dd-8f09-9313e9530342 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] [instance: dc0abe67-6867-4794-aaa6-5235d45edf2a] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:07:46.621779 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-bebebb35-1cd8-48dd-8f09-9313e9530342 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:07:46.622252 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-bebebb35-1cd8-48dd-8f09-9313e9530342 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:07:46.623682 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-bebebb35-1cd8-48dd-8f09-9313e9530342 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:07:46.624145 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-bebebb35-1cd8-48dd-8f09-9313e9530342 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:07:46.759349 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-bebebb35-1cd8-48dd-8f09-9313e9530342 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b91f7cf2a74f45858448608b096b0c31', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1617cbc6e7cf4eed8d2f8968da996b9c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:07:46.767106 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-bebebb35-1cd8-48dd-8f09-9313e9530342 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] 10.222.0.16 "GET /compute/v2.1/servers/dc0abe67-6867-4794-aaa6-5235d45edf2a" status: 200 len: 1505 microversion: 2.1 time: 0.156024 Jul 16 22:07:46.767679 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3392/6776] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:07:46 2022] GET /compute/v2.1/servers/dc0abe67-6867-4794-aaa6-5235d45edf2a => generated 1505 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:07:46.784275 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-f729bf40-cc41-459a-8734-669e21d02203 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:07:46.784918 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-f729bf40-cc41-459a-8734-669e21d02203 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] [instance: dc0abe67-6867-4794-aaa6-5235d45edf2a] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:07:46.792032 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f729bf40-cc41-459a-8734-669e21d02203 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:07:46.792491 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f729bf40-cc41-459a-8734-669e21d02203 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:07:46.793259 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f729bf40-cc41-459a-8734-669e21d02203 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:07:46.793704 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f729bf40-cc41-459a-8734-669e21d02203 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:07:46.929387 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-f729bf40-cc41-459a-8734-669e21d02203 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b91f7cf2a74f45858448608b096b0c31', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1617cbc6e7cf4eed8d2f8968da996b9c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:07:46.937207 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-f729bf40-cc41-459a-8734-669e21d02203 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] 10.222.0.16 "GET /compute/v2.1/servers/dc0abe67-6867-4794-aaa6-5235d45edf2a" status: 200 len: 1505 microversion: 2.1 time: 0.156688 Jul 16 22:07:46.937779 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3385/6777] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:07:46 2022] GET /compute/v2.1/servers/dc0abe67-6867-4794-aaa6-5235d45edf2a => generated 1505 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:07:46.954252 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-c91a9402-0fbe-4e77-8ace-b63b3086ae88 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Action: 'action', calling method: >, body: {"resize": {"flavorRef": "d65e7c41-822c-4348-9278-f12126e9e4e7"}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:07:46.955345 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-c91a9402-0fbe-4e77-8ace-b63b3086ae88 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] [instance: dc0abe67-6867-4794-aaa6-5235d45edf2a] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:07:46.961766 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c91a9402-0fbe-4e77-8ace-b63b3086ae88 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:07:46.962242 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c91a9402-0fbe-4e77-8ace-b63b3086ae88 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:07:46.963052 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c91a9402-0fbe-4e77-8ace-b63b3086ae88 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:07:46.963518 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c91a9402-0fbe-4e77-8ace-b63b3086ae88 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:07:46.968488 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:07:46.996307 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-c91a9402-0fbe-4e77-8ace-b63b3086ae88 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Policy check for compute:servers:resize:cross_cell failed with credentials {'is_admin': False, 'user_id': 'b91f7cf2a74f45858448608b096b0c31', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1617cbc6e7cf4eed8d2f8968da996b9c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:07:47.001711 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-c91a9402-0fbe-4e77-8ace-b63b3086ae88 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] HTTP exception thrown: Flavor d65e7c41-822c-4348-9278-f12126e9e4e7 could not be found. Jul 16 22:07:47.003745 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-c91a9402-0fbe-4e77-8ace-b63b3086ae88 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Request method failure captured: Jul 16 22:07:47.003745 ncdv-master-1 devstack@n-api.service[138681]: request: POST /compute/v2.1/servers/dc0abe67-6867-4794-aaa6-5235d45edf2a/action HTTP/1.1 Jul 16 22:07:47.003745 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 22:07:47.003745 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 22:07:47.003745 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 22:07:47.003745 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 65 Jul 16 22:07:47.003745 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 22:07:47.003745 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 22:07:47.003745 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 22:07:47.003745 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 22:07:47.003745 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0zaW0UoY6UJ6FJHYDQ4r97OlyPnvxU6e2w9qGk5WCAayEbLeZeNRgzNo9liblYswSYqcMHs4F3E_P-9GEAa6A2r4CdzhoZvP1848KJhQMPdK0xmvHKxA1-TYGf2CGi8ZFNp8FEAXnHHm0r0NZNndypHrkZ6k6Q41f-GxWs_uQ1YL9wo Jul 16 22:07:47.003745 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 22:07:47.003745 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 22:07:47.003745 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 22:07:47.003745 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 22:07:47.003745 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 22:07:47.003745 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 22:07:47.003745 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: 1617cbc6e7cf4eed8d2f8968da996b9c Jul 16 22:07:47.003745 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-ServersNegativeTestJSON-95771019 Jul 16 22:07:47.003745 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 22:07:47.003745 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 22:07:47.003745 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/1617cbc6e7cf4eed8d2f8968da996b9c"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/1617cbc6e7cf4eed8d2f8968da996b9c"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_1617cbc6e7cf4eed8d2f8968da996b9c"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/1617cbc6e7cf4eed8d2f8968da996b9c"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 22:07:47.003745 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-ServersNegativeTestJSON-95771019 Jul 16 22:07:47.003745 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: 1617cbc6e7cf4eed8d2f8968da996b9c Jul 16 22:07:47.003745 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-ServersNegativeTestJSON-95771019 Jul 16 22:07:47.003745 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-ServersNegativeTestJSON-95771019-project Jul 16 22:07:47.003745 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 22:07:47.003745 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 22:07:47.003745 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: b91f7cf2a74f45858448608b096b0c31 Jul 16 22:07:47.003745 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-ServersNegativeTestJSON-95771019-project Jul 16 22:07:47.003745 ncdv-master-1 devstack@n-api.service[138681]: Jul 16 22:07:47.003745 ncdv-master-1 devstack@n-api.service[138681]: {"resize": {"flavorRef": "d65e7c41-822c-4348-9278-f12126e9e4e7"}} Jul 16 22:07:47.003745 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 22:07:47.003745 ncdv-master-1 devstack@n-api.service[138681]: exception: Flavor d65e7c41-822c-4348-9278-f12126e9e4e7 could not be found. Jul 16 22:07:47.003745 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': 'dc0abe67-6867-4794-aaa6-5235d45edf2a', 'body': {'resize': {'flavorRef': 'd65e7c41-822c-4348-9278-f12126e9e4e7'}}} Jul 16 22:07:47.003745 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 22:07:47.003745 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 22:07:47.003745 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1053, in _resize Jul 16 22:07:47.003745 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi self.compute_api.resize(context, instance, flavor_id, Jul 16 22:07:47.003745 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 375, in inner Jul 16 22:07:47.003745 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Jul 16 22:07:47.003745 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 360, in wrapper Jul 16 22:07:47.003745 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return func(self, context, instance, *args, **kwargs) Jul 16 22:07:47.003745 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 296, in inner Jul 16 22:07:47.003745 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Jul 16 22:07:47.003745 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 343, in wrapper Jul 16 22:07:47.003745 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return func(self, context, instance, *args, **kwargs) Jul 16 22:07:47.003745 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner Jul 16 22:07:47.003745 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Jul 16 22:07:47.003745 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 164, in inner Jul 16 22:07:47.003745 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Jul 16 22:07:47.003745 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 228, in wrapped Jul 16 22:07:47.003745 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Jul 16 22:07:47.003745 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 4146, in resize Jul 16 22:07:47.003745 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi new_flavor = flavors.get_flavor_by_flavor_id( Jul 16 22:07:47.003745 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Jul 16 22:07:47.003745 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Jul 16 22:07:47.003745 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Jul 16 22:07:47.003745 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Jul 16 22:07:47.003745 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Jul 16 22:07:47.003745 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Jul 16 22:07:47.003745 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Jul 16 22:07:47.003745 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 22:07:47.008035 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Jul 16 22:07:47.008035 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Jul 16 22:07:47.008035 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor d65e7c41-822c-4348-9278-f12126e9e4e7 could not be found. Jul 16 22:07:47.008035 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 22:07:47.008035 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 22:07:47.008035 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 22:07:47.008035 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 22:07:47.008035 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 22:07:47.008035 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 22:07:47.008035 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 22:07:47.008035 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 22:07:47.008035 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 22:07:47.008035 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 22:07:47.008035 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 16 22:07:47.008035 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 16 22:07:47.008035 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1139, in _action_resize Jul 16 22:07:47.008035 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi self._resize(req, id, flavor_ref, **kwargs) Jul 16 22:07:47.008035 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1083, in _resize Jul 16 22:07:47.008035 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Jul 16 22:07:47.008035 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Flavor d65e7c41-822c-4348-9278-f12126e9e4e7 could not be found. Jul 16 22:07:47.008035 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 22:07:47.008035 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 22:07:47.008035 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 22:07:47.008035 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 22:07:47.008035 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 22:07:47.008035 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 22:07:47.008035 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 22:07:47.008035 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 22:07:47.008035 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor d65e7c41-822c-4348-9278-f12126e9e4e7 could not be found. Jul 16 22:07:47.008035 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 22:07:47.008035 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-c91a9402-0fbe-4e77-8ace-b63b3086ae88 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Returning 400 to user: Flavor d65e7c41-822c-4348-9278-f12126e9e4e7 could not be found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 22:07:47.008035 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-c91a9402-0fbe-4e77-8ace-b63b3086ae88 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] 10.222.0.16 "POST /compute/v2.1/servers/dc0abe67-6867-4794-aaa6-5235d45edf2a/action" status: 400 len: 107 microversion: 2.1 time: 0.055737 Jul 16 22:07:47.008035 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3393/6778] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 22:07:46 2022] POST /compute/v2.1/servers/dc0abe67-6867-4794-aaa6-5235d45edf2a/action => generated 107 bytes in 57 msecs (HTTP/1.1 400) 9 headers in 381 bytes (2 switches on core 0) Jul 16 22:07:47.014334 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-74c30aa4-6e0b-4c30-ac7a-41dce7a7d811 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:07:47.014958 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-74c30aa4-6e0b-4c30-ac7a-41dce7a7d811 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] [instance: dc0abe67-6867-4794-aaa6-5235d45edf2a] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:07:47.021099 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-74c30aa4-6e0b-4c30-ac7a-41dce7a7d811 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:07:47.021606 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-74c30aa4-6e0b-4c30-ac7a-41dce7a7d811 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:07:47.022433 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-74c30aa4-6e0b-4c30-ac7a-41dce7a7d811 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:07:47.022882 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-74c30aa4-6e0b-4c30-ac7a-41dce7a7d811 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:07:47.155046 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-74c30aa4-6e0b-4c30-ac7a-41dce7a7d811 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b91f7cf2a74f45858448608b096b0c31', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1617cbc6e7cf4eed8d2f8968da996b9c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:07:47.160317 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-74c30aa4-6e0b-4c30-ac7a-41dce7a7d811 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] 10.222.0.16 "GET /compute/v2.1/servers/dc0abe67-6867-4794-aaa6-5235d45edf2a" status: 200 len: 1505 microversion: 2.1 time: 0.150076 Jul 16 22:07:47.161186 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3386/6779] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:07:47 2022] GET /compute/v2.1/servers/dc0abe67-6867-4794-aaa6-5235d45edf2a => generated 1505 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:07:47.178561 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-7d59655f-9dfe-47dc-b20f-c7700fe022b1 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:07:47.179216 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-7d59655f-9dfe-47dc-b20f-c7700fe022b1 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] [instance: dc0abe67-6867-4794-aaa6-5235d45edf2a] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:07:47.186094 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7d59655f-9dfe-47dc-b20f-c7700fe022b1 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:07:47.189856 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7d59655f-9dfe-47dc-b20f-c7700fe022b1 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:07:47.189856 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7d59655f-9dfe-47dc-b20f-c7700fe022b1 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:07:47.189856 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7d59655f-9dfe-47dc-b20f-c7700fe022b1 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:07:47.323652 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-7d59655f-9dfe-47dc-b20f-c7700fe022b1 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b91f7cf2a74f45858448608b096b0c31', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1617cbc6e7cf4eed8d2f8968da996b9c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:07:47.332101 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-7d59655f-9dfe-47dc-b20f-c7700fe022b1 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] 10.222.0.16 "GET /compute/v2.1/servers/dc0abe67-6867-4794-aaa6-5235d45edf2a" status: 200 len: 1505 microversion: 2.1 time: 0.157769 Jul 16 22:07:47.332672 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3394/6780] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:07:47 2022] GET /compute/v2.1/servers/dc0abe67-6867-4794-aaa6-5235d45edf2a => generated 1505 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:07:47.349654 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-126ce60c-f534-40e4-9109-3df9df0b42e7 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Action: 'action', calling method: >, body: {"resize": {"flavorRef": ""}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:07:47.352268 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-126ce60c-f534-40e4-9109-3df9df0b42e7 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Request method failure captured: Jul 16 22:07:47.352268 ncdv-master-1 devstack@n-api.service[138680]: request: POST /compute/v2.1/servers/dc0abe67-6867-4794-aaa6-5235d45edf2a/action HTTP/1.1 Jul 16 22:07:47.352268 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 22:07:47.352268 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 22:07:47.352268 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 22:07:47.352268 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 29 Jul 16 22:07:47.352268 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 22:07:47.352268 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 22:07:47.352268 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 22:07:47.352268 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 22:07:47.352268 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0zaW0UoY6UJ6FJHYDQ4r97OlyPnvxU6e2w9qGk5WCAayEbLeZeNRgzNo9liblYswSYqcMHs4F3E_P-9GEAa6A2r4CdzhoZvP1848KJhQMPdK0xmvHKxA1-TYGf2CGi8ZFNp8FEAXnHHm0r0NZNndypHrkZ6k6Q41f-GxWs_uQ1YL9wo Jul 16 22:07:47.352268 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 22:07:47.352268 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 22:07:47.352268 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 22:07:47.352268 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 22:07:47.352268 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 22:07:47.352268 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 22:07:47.352268 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: 1617cbc6e7cf4eed8d2f8968da996b9c Jul 16 22:07:47.352268 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-ServersNegativeTestJSON-95771019 Jul 16 22:07:47.352268 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 22:07:47.352268 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 22:07:47.352268 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/1617cbc6e7cf4eed8d2f8968da996b9c"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/1617cbc6e7cf4eed8d2f8968da996b9c"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_1617cbc6e7cf4eed8d2f8968da996b9c"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/1617cbc6e7cf4eed8d2f8968da996b9c"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 22:07:47.352268 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-ServersNegativeTestJSON-95771019 Jul 16 22:07:47.352268 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: 1617cbc6e7cf4eed8d2f8968da996b9c Jul 16 22:07:47.352268 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-ServersNegativeTestJSON-95771019 Jul 16 22:07:47.352268 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-ServersNegativeTestJSON-95771019-project Jul 16 22:07:47.352268 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 22:07:47.352268 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 22:07:47.352268 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: b91f7cf2a74f45858448608b096b0c31 Jul 16 22:07:47.352268 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-ServersNegativeTestJSON-95771019-project Jul 16 22:07:47.352268 ncdv-master-1 devstack@n-api.service[138680]: Jul 16 22:07:47.352268 ncdv-master-1 devstack@n-api.service[138680]: {"resize": {"flavorRef": ""}} Jul 16 22:07:47.352268 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 22:07:47.352268 ncdv-master-1 devstack@n-api.service[138680]: exception: Invalid input for field/attribute flavorRef. Value: . '' is too short Jul 16 22:07:47.352268 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': 'dc0abe67-6867-4794-aaa6-5235d45edf2a', 'body': {'resize': {'flavorRef': ''}}} Jul 16 22:07:47.352268 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 22:07:47.352268 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 22:07:47.352268 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 279, in validate Jul 16 22:07:47.352268 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Jul 16 22:07:47.352268 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/lib/python3/dist-packages/jsonschema/validators.py", line 353, in validate Jul 16 22:07:47.352268 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise error Jul 16 22:07:47.352268 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: '' is too short Jul 16 22:07:47.352268 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 22:07:47.352268 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Failed validating 'minLength' in schema['properties']['resize']['properties']['flavorRef']: Jul 16 22:07:47.352268 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi {'minLength': 1, 'type': ['string', 'integer']} Jul 16 22:07:47.352268 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 22:07:47.352268 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi On instance['resize']: Jul 16 22:07:47.352268 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi '' Jul 16 22:07:47.352268 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 22:07:47.352268 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 22:07:47.352268 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 22:07:47.352268 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 22:07:47.352268 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 22:07:47.352268 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 22:07:47.352268 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 22:07:47.352268 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 22:07:47.352268 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 22:07:47.352268 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 22:07:47.352268 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Jul 16 22:07:47.352268 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Jul 16 22:07:47.352268 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Jul 16 22:07:47.352268 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Jul 16 22:07:47.352268 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 308, in validate Jul 16 22:07:47.352268 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Jul 16 22:07:47.352268 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute flavorRef. Value: . '' is too short Jul 16 22:07:47.352268 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 22:07:47.352268 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 22:07:47.352268 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 22:07:47.361854 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 22:07:47.361854 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 22:07:47.361854 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 22:07:47.361854 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Jul 16 22:07:47.361854 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Jul 16 22:07:47.361854 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute flavorRef. Value: . '' is too short Jul 16 22:07:47.361854 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 22:07:47.361854 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-126ce60c-f534-40e4-9109-3df9df0b42e7 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Returning 400 to user: Invalid input for field/attribute flavorRef. Value: . '' is too short {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 22:07:47.361854 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-126ce60c-f534-40e4-9109-3df9df0b42e7 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] 10.222.0.16 "POST /compute/v2.1/servers/dc0abe67-6867-4794-aaa6-5235d45edf2a/action" status: 400 len: 113 microversion: 2.1 time: 0.009130 Jul 16 22:07:47.361854 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3387/6781] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 22:07:47 2022] POST /compute/v2.1/servers/dc0abe67-6867-4794-aaa6-5235d45edf2a/action => generated 113 bytes in 11 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Jul 16 22:07:47.364070 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-0d9369e4-de66-4da2-bc8b-6b44a408c0db tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:07:47.364702 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-0d9369e4-de66-4da2-bc8b-6b44a408c0db tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] [instance: dc0abe67-6867-4794-aaa6-5235d45edf2a] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:07:47.371180 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0d9369e4-de66-4da2-bc8b-6b44a408c0db tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:07:47.371681 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0d9369e4-de66-4da2-bc8b-6b44a408c0db tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:07:47.372456 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0d9369e4-de66-4da2-bc8b-6b44a408c0db tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:07:47.372907 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0d9369e4-de66-4da2-bc8b-6b44a408c0db tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:07:47.505048 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-0d9369e4-de66-4da2-bc8b-6b44a408c0db tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b91f7cf2a74f45858448608b096b0c31', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1617cbc6e7cf4eed8d2f8968da996b9c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:07:47.512656 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-0d9369e4-de66-4da2-bc8b-6b44a408c0db tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] 10.222.0.16 "GET /compute/v2.1/servers/dc0abe67-6867-4794-aaa6-5235d45edf2a" status: 200 len: 1505 microversion: 2.1 time: 0.153038 Jul 16 22:07:47.513231 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3395/6782] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:07:47 2022] GET /compute/v2.1/servers/dc0abe67-6867-4794-aaa6-5235d45edf2a => generated 1505 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 22:07:47.535269 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-db197504-499a-48d6-b557-f7a4052505fe tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:07:47.535779 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-db197504-499a-48d6-b557-f7a4052505fe tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] [instance: c918af94-53fe-4a9b-9b3f-b9829038314f] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:07:47.542915 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-db197504-499a-48d6-b557-f7a4052505fe tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:07:47.543416 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-db197504-499a-48d6-b557-f7a4052505fe tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:07:47.544186 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-db197504-499a-48d6-b557-f7a4052505fe tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:07:47.544633 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-db197504-499a-48d6-b557-f7a4052505fe tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:07:47.557718 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-db197504-499a-48d6-b557-f7a4052505fe tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] HTTP exception thrown: Instance c918af94-53fe-4a9b-9b3f-b9829038314f could not be found. Jul 16 22:07:47.559339 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-db197504-499a-48d6-b557-f7a4052505fe tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Request method failure captured: Jul 16 22:07:47.559339 ncdv-master-1 devstack@n-api.service[138680]: request: DELETE /compute/v2.1/servers/c918af94-53fe-4a9b-9b3f-b9829038314f HTTP/1.1 Jul 16 22:07:47.559339 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 22:07:47.559339 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 22:07:47.559339 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 22:07:47.559339 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 0 Jul 16 22:07:47.559339 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 22:07:47.559339 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 22:07:47.559339 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 22:07:47.559339 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 22:07:47.559339 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0zaW0UoY6UJ6FJHYDQ4r97OlyPnvxU6e2w9qGk5WCAayEbLeZeNRgzNo9liblYswSYqcMHs4F3E_P-9GEAa6A2r4CdzhoZvP1848KJhQMPdK0xmvHKxA1-TYGf2CGi8ZFNp8FEAXnHHm0r0NZNndypHrkZ6k6Q41f-GxWs_uQ1YL9wo Jul 16 22:07:47.559339 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 22:07:47.559339 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 22:07:47.559339 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 22:07:47.559339 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 22:07:47.559339 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 22:07:47.559339 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 22:07:47.559339 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: 1617cbc6e7cf4eed8d2f8968da996b9c Jul 16 22:07:47.559339 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-ServersNegativeTestJSON-95771019 Jul 16 22:07:47.559339 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 22:07:47.559339 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 22:07:47.559339 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/1617cbc6e7cf4eed8d2f8968da996b9c"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/1617cbc6e7cf4eed8d2f8968da996b9c"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_1617cbc6e7cf4eed8d2f8968da996b9c"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/1617cbc6e7cf4eed8d2f8968da996b9c"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 22:07:47.559339 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-ServersNegativeTestJSON-95771019 Jul 16 22:07:47.559339 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: 1617cbc6e7cf4eed8d2f8968da996b9c Jul 16 22:07:47.559339 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-ServersNegativeTestJSON-95771019 Jul 16 22:07:47.559339 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-ServersNegativeTestJSON-95771019-project Jul 16 22:07:47.559339 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 22:07:47.559339 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 22:07:47.559339 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: b91f7cf2a74f45858448608b096b0c31 Jul 16 22:07:47.559339 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-ServersNegativeTestJSON-95771019-project Jul 16 22:07:47.559339 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 22:07:47.559339 ncdv-master-1 devstack@n-api.service[138680]: exception: Instance c918af94-53fe-4a9b-9b3f-b9829038314f could not be found. Jul 16 22:07:47.559339 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': 'c918af94-53fe-4a9b-9b3f-b9829038314f'} Jul 16 22:07:47.559339 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 22:07:47.559339 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 22:07:47.559339 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 22:07:47.559339 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 22:07:47.559339 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 22:07:47.559339 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 22:07:47.559339 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 22:07:47.559339 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 22:07:47.559339 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 22:07:47.559339 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 22:07:47.559339 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance c918af94-53fe-4a9b-9b3f-b9829038314f could not be found. Jul 16 22:07:47.559339 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 22:07:47.559339 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 22:07:47.559339 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 22:07:47.559339 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 22:07:47.559339 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 22:07:47.559339 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 22:07:47.559339 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 22:07:47.559339 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 22:07:47.559339 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 22:07:47.559339 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 22:07:47.559339 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Jul 16 22:07:47.559339 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Jul 16 22:07:47.559339 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Jul 16 22:07:47.559339 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Jul 16 22:07:47.559339 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 22:07:47.559339 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 22:07:47.559339 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 22:07:47.559339 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 22:07:47.564582 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance c918af94-53fe-4a9b-9b3f-b9829038314f could not be found. Jul 16 22:07:47.564582 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 22:07:47.564582 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 22:07:47.564582 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 22:07:47.564582 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 22:07:47.564582 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 22:07:47.564582 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 22:07:47.564582 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 22:07:47.564582 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 22:07:47.564582 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance c918af94-53fe-4a9b-9b3f-b9829038314f could not be found. Jul 16 22:07:47.564582 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 22:07:47.564582 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-db197504-499a-48d6-b557-f7a4052505fe tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Returning 404 to user: Instance c918af94-53fe-4a9b-9b3f-b9829038314f could not be found. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 22:07:47.564582 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-db197504-499a-48d6-b557-f7a4052505fe tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] 10.222.0.16 "DELETE /compute/v2.1/servers/c918af94-53fe-4a9b-9b3f-b9829038314f" status: 404 len: 111 microversion: 2.1 time: 0.030986 Jul 16 22:07:47.564582 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3388/6783] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 22:07:47 2022] DELETE /compute/v2.1/servers/c918af94-53fe-4a9b-9b3f-b9829038314f => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 22:07:47.569641 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-9343a919-c736-44d2-a3b4-a7ab48888e4c tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:07:47.570268 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-9343a919-c736-44d2-a3b4-a7ab48888e4c tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] [instance: c918af94-53fe-4a9b-9b3f-b9829038314f] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:07:47.578909 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9343a919-c736-44d2-a3b4-a7ab48888e4c tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:07:47.579412 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9343a919-c736-44d2-a3b4-a7ab48888e4c tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:07:47.580180 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9343a919-c736-44d2-a3b4-a7ab48888e4c tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:07:47.580628 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9343a919-c736-44d2-a3b4-a7ab48888e4c tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:07:47.594857 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-9343a919-c736-44d2-a3b4-a7ab48888e4c tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] HTTP exception thrown: Instance c918af94-53fe-4a9b-9b3f-b9829038314f could not be found. Jul 16 22:07:47.596416 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-9343a919-c736-44d2-a3b4-a7ab48888e4c tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Request method failure captured: Jul 16 22:07:47.596416 ncdv-master-1 devstack@n-api.service[138681]: request: GET /compute/v2.1/servers/c918af94-53fe-4a9b-9b3f-b9829038314f HTTP/1.1 Jul 16 22:07:47.596416 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 22:07:47.596416 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 22:07:47.596416 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 22:07:47.596416 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 22:07:47.596416 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 22:07:47.596416 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 22:07:47.596416 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 22:07:47.596416 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 22:07:47.596416 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0zaW0UoY6UJ6FJHYDQ4r97OlyPnvxU6e2w9qGk5WCAayEbLeZeNRgzNo9liblYswSYqcMHs4F3E_P-9GEAa6A2r4CdzhoZvP1848KJhQMPdK0xmvHKxA1-TYGf2CGi8ZFNp8FEAXnHHm0r0NZNndypHrkZ6k6Q41f-GxWs_uQ1YL9wo Jul 16 22:07:47.596416 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 22:07:47.596416 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 22:07:47.596416 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 22:07:47.596416 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 22:07:47.596416 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 22:07:47.596416 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 22:07:47.596416 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: 1617cbc6e7cf4eed8d2f8968da996b9c Jul 16 22:07:47.596416 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-ServersNegativeTestJSON-95771019 Jul 16 22:07:47.596416 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 22:07:47.596416 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 22:07:47.596416 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/1617cbc6e7cf4eed8d2f8968da996b9c"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/1617cbc6e7cf4eed8d2f8968da996b9c"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_1617cbc6e7cf4eed8d2f8968da996b9c"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/1617cbc6e7cf4eed8d2f8968da996b9c"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 22:07:47.596416 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-ServersNegativeTestJSON-95771019 Jul 16 22:07:47.596416 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: 1617cbc6e7cf4eed8d2f8968da996b9c Jul 16 22:07:47.596416 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-ServersNegativeTestJSON-95771019 Jul 16 22:07:47.596416 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-ServersNegativeTestJSON-95771019-project Jul 16 22:07:47.596416 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 22:07:47.596416 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 22:07:47.596416 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: b91f7cf2a74f45858448608b096b0c31 Jul 16 22:07:47.596416 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-ServersNegativeTestJSON-95771019-project Jul 16 22:07:47.596416 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 22:07:47.596416 ncdv-master-1 devstack@n-api.service[138681]: exception: Instance c918af94-53fe-4a9b-9b3f-b9829038314f could not be found. Jul 16 22:07:47.596416 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': 'c918af94-53fe-4a9b-9b3f-b9829038314f'} Jul 16 22:07:47.596416 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 22:07:47.596416 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 22:07:47.596416 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 22:07:47.596416 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 22:07:47.596416 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 22:07:47.596416 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 22:07:47.596416 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 22:07:47.596416 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 22:07:47.596416 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 22:07:47.596416 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 22:07:47.596416 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance c918af94-53fe-4a9b-9b3f-b9829038314f could not be found. Jul 16 22:07:47.596416 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 22:07:47.596416 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 22:07:47.596416 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 22:07:47.596416 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 22:07:47.596416 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 22:07:47.596416 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 22:07:47.596416 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 22:07:47.596416 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 22:07:47.596416 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 22:07:47.596416 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 22:07:47.596416 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 22:07:47.596416 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 22:07:47.596416 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 22:07:47.596416 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 22:07:47.596416 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 22:07:47.596416 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 22:07:47.596416 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance c918af94-53fe-4a9b-9b3f-b9829038314f could not be found. Jul 16 22:07:47.596416 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 22:07:47.596416 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 22:07:47.596416 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 22:07:47.604927 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 22:07:47.604927 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 22:07:47.604927 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 22:07:47.604927 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 22:07:47.604927 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 22:07:47.604927 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance c918af94-53fe-4a9b-9b3f-b9829038314f could not be found. Jul 16 22:07:47.604927 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 22:07:47.604927 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-9343a919-c736-44d2-a3b4-a7ab48888e4c tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Returning 404 to user: Instance c918af94-53fe-4a9b-9b3f-b9829038314f could not be found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 22:07:47.604927 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-9343a919-c736-44d2-a3b4-a7ab48888e4c tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] 10.222.0.16 "GET /compute/v2.1/servers/c918af94-53fe-4a9b-9b3f-b9829038314f" status: 404 len: 111 microversion: 2.1 time: 0.032283 Jul 16 22:07:47.604927 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3396/6784] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:07:47 2022] GET /compute/v2.1/servers/c918af94-53fe-4a9b-9b3f-b9829038314f => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 22:07:47.606291 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-329669be-4089-44b6-b1c6-391d89058998 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:07:47.606830 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-329669be-4089-44b6-b1c6-391d89058998 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] [instance: dc0abe67-6867-4794-aaa6-5235d45edf2a] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:07:47.613338 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-329669be-4089-44b6-b1c6-391d89058998 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:07:47.613806 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-329669be-4089-44b6-b1c6-391d89058998 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:07:47.614572 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-329669be-4089-44b6-b1c6-391d89058998 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:07:47.615039 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-329669be-4089-44b6-b1c6-391d89058998 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:07:47.641519 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-329669be-4089-44b6-b1c6-391d89058998 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] [instance: dc0abe67-6867-4794-aaa6-5235d45edf2a] Going to try to terminate instance {{(pid=138680) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 16 22:07:47.655268 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=100 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:07:47.892987 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-329669be-4089-44b6-b1c6-391d89058998 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] 10.222.0.16 "DELETE /compute/v2.1/servers/dc0abe67-6867-4794-aaa6-5235d45edf2a" status: 204 len: 0 microversion: 2.1 time: 0.290228 Jul 16 22:07:47.893585 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3389/6785] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 22:07:47 2022] DELETE /compute/v2.1/servers/dc0abe67-6867-4794-aaa6-5235d45edf2a => generated 0 bytes in 292 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 16 22:07:47.901833 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-cea30a45-a198-4816-803e-dfa83b8ef50c tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:07:47.902528 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-cea30a45-a198-4816-803e-dfa83b8ef50c tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] [instance: dc0abe67-6867-4794-aaa6-5235d45edf2a] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:07:47.908935 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cea30a45-a198-4816-803e-dfa83b8ef50c tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:07:47.909401 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cea30a45-a198-4816-803e-dfa83b8ef50c tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:07:47.910179 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cea30a45-a198-4816-803e-dfa83b8ef50c tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:07:47.910635 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cea30a45-a198-4816-803e-dfa83b8ef50c tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:07:48.047618 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-cea30a45-a198-4816-803e-dfa83b8ef50c tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b91f7cf2a74f45858448608b096b0c31', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1617cbc6e7cf4eed8d2f8968da996b9c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:07:48.055553 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-cea30a45-a198-4816-803e-dfa83b8ef50c tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] 10.222.0.16 "GET /compute/v2.1/servers/dc0abe67-6867-4794-aaa6-5235d45edf2a" status: 200 len: 1511 microversion: 2.1 time: 0.157763 Jul 16 22:07:48.056122 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3397/6786] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:07:47 2022] GET /compute/v2.1/servers/dc0abe67-6867-4794-aaa6-5235d45edf2a => generated 1511 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:07:49.073751 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-3e5ebf0b-e779-4481-aee8-7969ff034bd8 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:07:49.074532 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-3e5ebf0b-e779-4481-aee8-7969ff034bd8 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] [instance: dc0abe67-6867-4794-aaa6-5235d45edf2a] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:07:49.077649 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_api:SELECT=72,nova_api:INSERT=6,nova_api:UPDATE=1 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:07:49.082455 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3e5ebf0b-e779-4481-aee8-7969ff034bd8 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:07:49.082917 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3e5ebf0b-e779-4481-aee8-7969ff034bd8 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:07:49.083793 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3e5ebf0b-e779-4481-aee8-7969ff034bd8 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:07:49.084243 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3e5ebf0b-e779-4481-aee8-7969ff034bd8 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:07:49.223110 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-3e5ebf0b-e779-4481-aee8-7969ff034bd8 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b91f7cf2a74f45858448608b096b0c31', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1617cbc6e7cf4eed8d2f8968da996b9c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:07:49.230680 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-3e5ebf0b-e779-4481-aee8-7969ff034bd8 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] 10.222.0.16 "GET /compute/v2.1/servers/dc0abe67-6867-4794-aaa6-5235d45edf2a" status: 200 len: 1511 microversion: 2.1 time: 0.161167 Jul 16 22:07:49.231287 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3390/6787] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:07:49 2022] GET /compute/v2.1/servers/dc0abe67-6867-4794-aaa6-5235d45edf2a => generated 1511 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:07:49.965211 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-ac18919a-5500-4581-bff8-95ed06c98bc0 req-8c5e861a-bb7e-4442-955e-751c8f1b1a85 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "dc0abe67-6867-4794-aaa6-5235d45edf2a", "name": "network-vif-deleted", "tag": "0087cbc7-985b-4473-a370-59e9c23bfda0"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:07:49.974522 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-ac18919a-5500-4581-bff8-95ed06c98bc0 req-8c5e861a-bb7e-4442-955e-751c8f1b1a85 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:07:49.974989 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-ac18919a-5500-4581-bff8-95ed06c98bc0 req-8c5e861a-bb7e-4442-955e-751c8f1b1a85 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:07:49.987625 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-ac18919a-5500-4581-bff8-95ed06c98bc0 req-8c5e861a-bb7e-4442-955e-751c8f1b1a85 service nova] Creating event network-vif-deleted:0087cbc7-985b-4473-a370-59e9c23bfda0 for instance dc0abe67-6867-4794-aaa6-5235d45edf2a on nch2-master-1 Jul 16 22:07:49.993097 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-ac18919a-5500-4581-bff8-95ed06c98bc0 req-8c5e861a-bb7e-4442-955e-751c8f1b1a85 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.031592 Jul 16 22:07:49.993716 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3398/6788] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 22:07:49 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 22:07:50.248865 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-cfb0a581-a35e-431a-b254-1f0812aa0be3 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:07:50.249407 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-cfb0a581-a35e-431a-b254-1f0812aa0be3 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] [instance: dc0abe67-6867-4794-aaa6-5235d45edf2a] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:07:50.256480 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cfb0a581-a35e-431a-b254-1f0812aa0be3 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:07:50.256948 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cfb0a581-a35e-431a-b254-1f0812aa0be3 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:07:50.257718 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cfb0a581-a35e-431a-b254-1f0812aa0be3 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:07:50.258168 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cfb0a581-a35e-431a-b254-1f0812aa0be3 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:07:50.322353 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-cfb0a581-a35e-431a-b254-1f0812aa0be3 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b91f7cf2a74f45858448608b096b0c31', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1617cbc6e7cf4eed8d2f8968da996b9c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:07:50.331121 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-cfb0a581-a35e-431a-b254-1f0812aa0be3 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] 10.222.0.16 "GET /compute/v2.1/servers/dc0abe67-6867-4794-aaa6-5235d45edf2a" status: 200 len: 1301 microversion: 2.1 time: 0.086410 Jul 16 22:07:50.331708 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3391/6789] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:07:50 2022] GET /compute/v2.1/servers/dc0abe67-6867-4794-aaa6-5235d45edf2a => generated 1301 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 22:07:51.349078 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-9f60dc8c-8179-4ba3-b759-7388d48a3747 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:07:51.349795 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-9f60dc8c-8179-4ba3-b759-7388d48a3747 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] [instance: dc0abe67-6867-4794-aaa6-5235d45edf2a] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:07:51.357015 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9f60dc8c-8179-4ba3-b759-7388d48a3747 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:07:51.357490 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9f60dc8c-8179-4ba3-b759-7388d48a3747 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:07:51.358260 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9f60dc8c-8179-4ba3-b759-7388d48a3747 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:07:51.358708 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9f60dc8c-8179-4ba3-b759-7388d48a3747 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:07:51.372725 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-9f60dc8c-8179-4ba3-b759-7388d48a3747 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] HTTP exception thrown: Instance dc0abe67-6867-4794-aaa6-5235d45edf2a could not be found. Jul 16 22:07:51.374337 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-9f60dc8c-8179-4ba3-b759-7388d48a3747 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Request method failure captured: Jul 16 22:07:51.374337 ncdv-master-1 devstack@n-api.service[138681]: request: GET /compute/v2.1/servers/dc0abe67-6867-4794-aaa6-5235d45edf2a HTTP/1.1 Jul 16 22:07:51.374337 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 22:07:51.374337 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 22:07:51.374337 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 22:07:51.374337 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 22:07:51.374337 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 22:07:51.374337 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 22:07:51.374337 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 22:07:51.374337 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 22:07:51.374337 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0zaW0UoY6UJ6FJHYDQ4r97OlyPnvxU6e2w9qGk5WCAayEbLeZeNRgzNo9liblYswSYqcMHs4F3E_P-9GEAa6A2r4CdzhoZvP1848KJhQMPdK0xmvHKxA1-TYGf2CGi8ZFNp8FEAXnHHm0r0NZNndypHrkZ6k6Q41f-GxWs_uQ1YL9wo Jul 16 22:07:51.374337 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 22:07:51.374337 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 22:07:51.374337 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 22:07:51.374337 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 22:07:51.374337 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 22:07:51.374337 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 22:07:51.374337 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: 1617cbc6e7cf4eed8d2f8968da996b9c Jul 16 22:07:51.374337 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-ServersNegativeTestJSON-95771019 Jul 16 22:07:51.374337 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 22:07:51.374337 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 22:07:51.374337 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/1617cbc6e7cf4eed8d2f8968da996b9c"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/1617cbc6e7cf4eed8d2f8968da996b9c"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_1617cbc6e7cf4eed8d2f8968da996b9c"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/1617cbc6e7cf4eed8d2f8968da996b9c"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 22:07:51.374337 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-ServersNegativeTestJSON-95771019 Jul 16 22:07:51.374337 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: 1617cbc6e7cf4eed8d2f8968da996b9c Jul 16 22:07:51.374337 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-ServersNegativeTestJSON-95771019 Jul 16 22:07:51.374337 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-ServersNegativeTestJSON-95771019-project Jul 16 22:07:51.374337 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 22:07:51.374337 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 22:07:51.374337 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: b91f7cf2a74f45858448608b096b0c31 Jul 16 22:07:51.374337 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-ServersNegativeTestJSON-95771019-project Jul 16 22:07:51.374337 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 22:07:51.374337 ncdv-master-1 devstack@n-api.service[138681]: exception: Instance dc0abe67-6867-4794-aaa6-5235d45edf2a could not be found. Jul 16 22:07:51.374337 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': 'dc0abe67-6867-4794-aaa6-5235d45edf2a'} Jul 16 22:07:51.374337 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 22:07:51.374337 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 22:07:51.374337 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 22:07:51.374337 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 22:07:51.374337 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 22:07:51.374337 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 22:07:51.374337 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 22:07:51.374337 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 22:07:51.374337 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 22:07:51.374337 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 22:07:51.374337 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance dc0abe67-6867-4794-aaa6-5235d45edf2a could not be found. Jul 16 22:07:51.374337 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 22:07:51.374337 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 22:07:51.374337 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 22:07:51.374337 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 22:07:51.374337 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 22:07:51.374337 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 22:07:51.374337 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 22:07:51.374337 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 22:07:51.374337 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 22:07:51.374337 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 22:07:51.374337 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 22:07:51.374337 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 22:07:51.374337 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 22:07:51.374337 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 22:07:51.374337 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 22:07:51.374337 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 22:07:51.374337 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance dc0abe67-6867-4794-aaa6-5235d45edf2a could not be found. Jul 16 22:07:51.374337 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 22:07:51.374337 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 22:07:51.374337 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 22:07:51.379358 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 22:07:51.379358 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 22:07:51.379358 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 22:07:51.379358 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 22:07:51.379358 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 22:07:51.379358 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance dc0abe67-6867-4794-aaa6-5235d45edf2a could not be found. Jul 16 22:07:51.379358 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 22:07:51.379358 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-9f60dc8c-8179-4ba3-b759-7388d48a3747 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] Returning 404 to user: Instance dc0abe67-6867-4794-aaa6-5235d45edf2a could not be found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 22:07:51.379358 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-9f60dc8c-8179-4ba3-b759-7388d48a3747 tempest-ServersNegativeTestJSON-95771019 tempest-ServersNegativeTestJSON-95771019-project] 10.222.0.16 "GET /compute/v2.1/servers/dc0abe67-6867-4794-aaa6-5235d45edf2a" status: 404 len: 111 microversion: 2.1 time: 0.031088 Jul 16 22:07:51.379358 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3399/6790] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:07:51 2022] GET /compute/v2.1/servers/dc0abe67-6867-4794-aaa6-5235d45edf2a => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 22:08:06.852144 ncdv-master-1 devstack@n-api.service[138681]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jul 16 22:08:06.852801 ncdv-master-1 devstack@n-api.service[138681]: Sat Jul 16 22:08:06 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Jul 16 22:08:47.483156 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-c5ad69ea-a186-4c15-b25b-7927429dd77e tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Action: 'create', calling method: .version_select at 0x7fdd8fce59d0>, body: {"keypair": {"name": "tempest-TestMinimumBasicScenario-1668340565", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBE8nq3zLtjkRxUCpNO7nyQw7Si3t++es+catdRDPGtctpv1WPdXLiA9I9kivr53M5UNiJy1BcBrj4+tEzyx26ZF8Lg0G2k5N0OW1ZS8YxZop+G58xY4HSTFeP7qWRJSbuA=="}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:08:47.487313 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_api:SELECT=3 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:08:47.490485 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=18,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:08:47.501170 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-c5ad69ea-a186-4c15-b25b-7927429dd77e tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Getting quotas for project deb1e913bedd4fb5b286e21c60e27a41. Resources: {'key_pairs'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 22:08:47.506256 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-c5ad69ea-a186-4c15-b25b-7927429dd77e tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Getting quotas for user d749cd77cf5c489e8c4e57e02ce768fd and project deb1e913bedd4fb5b286e21c60e27a41. Resources: {'key_pairs'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 22:08:47.618174 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-c5ad69ea-a186-4c15-b25b-7927429dd77e tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] 10.222.0.16 "POST /compute/v2.1/os-keypairs" status: 200 len: 403 microversion: 2.1 time: 0.139303 Jul 16 22:08:47.618780 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3392/6791] 10.222.0.16 () {60 vars in 1158 bytes} [Sat Jul 16 22:08:47 2022] POST /compute/v2.1/os-keypairs => generated 403 bytes in 141 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 22:08:47.667249 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-51456fd1-3317-4a6f-8ffe-f310ce85db53 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestMinimumBasicScenario-server-1321825583", "imageRef": "d714c43d-c90e-4585-9c04-d0800a9d9919", "flavorRef": "452", "key_name": "tempest-TestMinimumBasicScenario-1668340565", "networks": [{"uuid": "bf8f9392-1062-4eb2-b42d-afeb45f5873f"}]}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:08:47.674793 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_api:SELECT=61,nova_api:UPDATE=1 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:08:47.675507 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=26 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:08:47.722149 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.network.neutron [None req-51456fd1-3317-4a6f-8ffe-f310ce85db53 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] validate_networks() for [('bf8f9392-1062-4eb2-b42d-afeb45f5873f', None, None, None, None, None)] {{(pid=138681) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 16 22:08:48.223843 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-51456fd1-3317-4a6f-8ffe-f310ce85db53 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Flavor limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 16 22:08:48.224280 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-51456fd1-3317-4a6f-8ffe-f310ce85db53 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Image limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 16 22:08:48.224767 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-51456fd1-3317-4a6f-8ffe-f310ce85db53 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Flavor pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 16 22:08:48.225190 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-51456fd1-3317-4a6f-8ffe-f310ce85db53 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Image pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 16 22:08:48.225615 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-51456fd1-3317-4a6f-8ffe-f310ce85db53 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 16 22:08:48.235170 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-51456fd1-3317-4a6f-8ffe-f310ce85db53 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:08:48.235635 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-51456fd1-3317-4a6f-8ffe-f310ce85db53 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:08:48.236442 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-51456fd1-3317-4a6f-8ffe-f310ce85db53 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:08:48.236889 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-51456fd1-3317-4a6f-8ffe-f310ce85db53 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:08:48.256441 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-51456fd1-3317-4a6f-8ffe-f310ce85db53 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Getting quotas for project deb1e913bedd4fb5b286e21c60e27a41. Resources: {'ram', 'instances', 'cores'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 22:08:48.261795 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-51456fd1-3317-4a6f-8ffe-f310ce85db53 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Getting quotas for user d749cd77cf5c489e8c4e57e02ce768fd and project deb1e913bedd4fb5b286e21c60e27a41. Resources: {'ram', 'instances', 'cores'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 22:08:48.274864 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-51456fd1-3317-4a6f-8ffe-f310ce85db53 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Going to run 1 instances... {{(pid=138681) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 16 22:08:48.277311 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-51456fd1-3317-4a6f-8ffe-f310ce85db53 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] [instance: d9281e80-5a8e-4a1e-9e1f-4de3b0dc6131] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='d714c43d-c90e-4585-9c04-d0800a9d9919',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138681) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 16 22:08:48.420528 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-51456fd1-3317-4a6f-8ffe-f310ce85db53 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.757044 Jul 16 22:08:48.421361 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3400/6792] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 22:08:47 2022] POST /compute/v2.1/servers => generated 380 bytes in 759 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 16 22:08:48.432022 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-b5dc7130-6343-4162-9ec9-fa64ecce8362 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:08:48.437408 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-b5dc7130-6343-4162-9ec9-fa64ecce8362 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] [instance: d9281e80-5a8e-4a1e-9e1f-4de3b0dc6131] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:08:48.491893 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-b5dc7130-6343-4162-9ec9-fa64ecce8362 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd749cd77cf5c489e8c4e57e02ce768fd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'deb1e913bedd4fb5b286e21c60e27a41', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:08:48.498925 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-b5dc7130-6343-4162-9ec9-fa64ecce8362 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] 10.222.0.16 "GET /compute/v2.1/servers/d9281e80-5a8e-4a1e-9e1f-4de3b0dc6131" status: 200 len: 1259 microversion: 2.1 time: 0.071349 Jul 16 22:08:48.500166 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3393/6793] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:08:48 2022] GET /compute/v2.1/servers/d9281e80-5a8e-4a1e-9e1f-4de3b0dc6131 => generated 1259 bytes in 73 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:08:49.518257 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-63d498d6-21e0-4c59-a29b-39eaf8e0ca05 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:08:49.518848 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-63d498d6-21e0-4c59-a29b-39eaf8e0ca05 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] [instance: d9281e80-5a8e-4a1e-9e1f-4de3b0dc6131] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:08:49.525793 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-63d498d6-21e0-4c59-a29b-39eaf8e0ca05 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:08:49.528165 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-63d498d6-21e0-4c59-a29b-39eaf8e0ca05 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:08:49.528165 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-63d498d6-21e0-4c59-a29b-39eaf8e0ca05 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:08:49.528165 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-63d498d6-21e0-4c59-a29b-39eaf8e0ca05 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:08:49.589730 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-63d498d6-21e0-4c59-a29b-39eaf8e0ca05 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd749cd77cf5c489e8c4e57e02ce768fd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'deb1e913bedd4fb5b286e21c60e27a41', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:08:49.598492 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-63d498d6-21e0-4c59-a29b-39eaf8e0ca05 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] 10.222.0.16 "GET /compute/v2.1/servers/d9281e80-5a8e-4a1e-9e1f-4de3b0dc6131" status: 200 len: 1263 microversion: 2.1 time: 0.086228 Jul 16 22:08:49.599356 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3401/6794] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:08:49 2022] GET /compute/v2.1/servers/d9281e80-5a8e-4a1e-9e1f-4de3b0dc6131 => generated 1263 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:08:50.617792 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-665ce88f-f438-434d-8e77-187e38dfd012 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:08:50.618437 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-665ce88f-f438-434d-8e77-187e38dfd012 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] [instance: d9281e80-5a8e-4a1e-9e1f-4de3b0dc6131] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:08:50.626864 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-665ce88f-f438-434d-8e77-187e38dfd012 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:08:50.627373 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-665ce88f-f438-434d-8e77-187e38dfd012 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:08:50.628156 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-665ce88f-f438-434d-8e77-187e38dfd012 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:08:50.628600 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-665ce88f-f438-434d-8e77-187e38dfd012 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:08:50.691536 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-665ce88f-f438-434d-8e77-187e38dfd012 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd749cd77cf5c489e8c4e57e02ce768fd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'deb1e913bedd4fb5b286e21c60e27a41', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:08:50.699433 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-665ce88f-f438-434d-8e77-187e38dfd012 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] 10.222.0.16 "GET /compute/v2.1/servers/d9281e80-5a8e-4a1e-9e1f-4de3b0dc6131" status: 200 len: 1329 microversion: 2.1 time: 0.086692 Jul 16 22:08:50.700206 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3394/6795] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:08:50 2022] GET /compute/v2.1/servers/d9281e80-5a8e-4a1e-9e1f-4de3b0dc6131 => generated 1329 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:08:51.719880 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-5e2b7b81-b1ec-44ce-8c40-b914bfa65231 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:08:51.721572 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-5e2b7b81-b1ec-44ce-8c40-b914bfa65231 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] [instance: d9281e80-5a8e-4a1e-9e1f-4de3b0dc6131] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:08:51.736562 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5e2b7b81-b1ec-44ce-8c40-b914bfa65231 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:08:51.738849 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5e2b7b81-b1ec-44ce-8c40-b914bfa65231 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:08:51.740309 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5e2b7b81-b1ec-44ce-8c40-b914bfa65231 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:08:51.740929 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5e2b7b81-b1ec-44ce-8c40-b914bfa65231 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:08:51.890269 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-5e2b7b81-b1ec-44ce-8c40-b914bfa65231 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd749cd77cf5c489e8c4e57e02ce768fd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'deb1e913bedd4fb5b286e21c60e27a41', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:08:51.898596 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-5e2b7b81-b1ec-44ce-8c40-b914bfa65231 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] 10.222.0.16 "GET /compute/v2.1/servers/d9281e80-5a8e-4a1e-9e1f-4de3b0dc6131" status: 200 len: 1359 microversion: 2.1 time: 0.185164 Jul 16 22:08:51.899215 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3402/6796] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:08:51 2022] GET /compute/v2.1/servers/d9281e80-5a8e-4a1e-9e1f-4de3b0dc6131 => generated 1359 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:08:52.917083 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-c6baf25d-e4e5-4afe-b21f-c39e8bac4c2b tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:08:52.917713 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-c6baf25d-e4e5-4afe-b21f-c39e8bac4c2b tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] [instance: d9281e80-5a8e-4a1e-9e1f-4de3b0dc6131] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:08:52.925301 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c6baf25d-e4e5-4afe-b21f-c39e8bac4c2b tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:08:52.925759 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c6baf25d-e4e5-4afe-b21f-c39e8bac4c2b tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:08:52.926536 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c6baf25d-e4e5-4afe-b21f-c39e8bac4c2b tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:08:52.926984 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c6baf25d-e4e5-4afe-b21f-c39e8bac4c2b tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:08:53.073617 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-c6baf25d-e4e5-4afe-b21f-c39e8bac4c2b tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd749cd77cf5c489e8c4e57e02ce768fd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'deb1e913bedd4fb5b286e21c60e27a41', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:08:53.083581 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-c6baf25d-e4e5-4afe-b21f-c39e8bac4c2b tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] 10.222.0.16 "GET /compute/v2.1/servers/d9281e80-5a8e-4a1e-9e1f-4de3b0dc6131" status: 200 len: 1359 microversion: 2.1 time: 0.170664 Jul 16 22:08:53.084254 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3395/6797] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:08:52 2022] GET /compute/v2.1/servers/d9281e80-5a8e-4a1e-9e1f-4de3b0dc6131 => generated 1359 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:08:54.101608 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-b4aa6f39-a70f-4e88-b48a-7fef21c27dcf tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:08:54.102247 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-b4aa6f39-a70f-4e88-b48a-7fef21c27dcf tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] [instance: d9281e80-5a8e-4a1e-9e1f-4de3b0dc6131] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:08:54.110356 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b4aa6f39-a70f-4e88-b48a-7fef21c27dcf tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:08:54.110905 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b4aa6f39-a70f-4e88-b48a-7fef21c27dcf tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:08:54.111829 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b4aa6f39-a70f-4e88-b48a-7fef21c27dcf tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:08:54.112347 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b4aa6f39-a70f-4e88-b48a-7fef21c27dcf tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:08:54.264053 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-b4aa6f39-a70f-4e88-b48a-7fef21c27dcf tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd749cd77cf5c489e8c4e57e02ce768fd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'deb1e913bedd4fb5b286e21c60e27a41', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:08:54.272723 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-b4aa6f39-a70f-4e88-b48a-7fef21c27dcf tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] 10.222.0.16 "GET /compute/v2.1/servers/d9281e80-5a8e-4a1e-9e1f-4de3b0dc6131" status: 200 len: 1359 microversion: 2.1 time: 0.175305 Jul 16 22:08:54.273482 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3403/6798] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:08:54 2022] GET /compute/v2.1/servers/d9281e80-5a8e-4a1e-9e1f-4de3b0dc6131 => generated 1359 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:08:55.290710 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-a3145aff-46e9-432f-b19f-a466f65c002e tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:08:55.291348 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-a3145aff-46e9-432f-b19f-a466f65c002e tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] [instance: d9281e80-5a8e-4a1e-9e1f-4de3b0dc6131] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:08:55.299042 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a3145aff-46e9-432f-b19f-a466f65c002e tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:08:55.299508 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a3145aff-46e9-432f-b19f-a466f65c002e tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:08:55.300288 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a3145aff-46e9-432f-b19f-a466f65c002e tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:08:55.300737 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a3145aff-46e9-432f-b19f-a466f65c002e tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:08:55.457710 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-a3145aff-46e9-432f-b19f-a466f65c002e tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd749cd77cf5c489e8c4e57e02ce768fd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'deb1e913bedd4fb5b286e21c60e27a41', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:08:55.465626 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-a3145aff-46e9-432f-b19f-a466f65c002e tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] 10.222.0.16 "GET /compute/v2.1/servers/d9281e80-5a8e-4a1e-9e1f-4de3b0dc6131" status: 200 len: 1359 microversion: 2.1 time: 0.178996 Jul 16 22:08:55.466452 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3396/6799] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:08:55 2022] GET /compute/v2.1/servers/d9281e80-5a8e-4a1e-9e1f-4de3b0dc6131 => generated 1359 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:08:56.486550 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-f6ff6287-7b3e-4f0c-9128-ae7d31b5fd24 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:08:56.487969 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-f6ff6287-7b3e-4f0c-9128-ae7d31b5fd24 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] [instance: d9281e80-5a8e-4a1e-9e1f-4de3b0dc6131] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:08:56.503688 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f6ff6287-7b3e-4f0c-9128-ae7d31b5fd24 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:08:56.504184 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f6ff6287-7b3e-4f0c-9128-ae7d31b5fd24 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:08:56.505006 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f6ff6287-7b3e-4f0c-9128-ae7d31b5fd24 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:08:56.505483 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f6ff6287-7b3e-4f0c-9128-ae7d31b5fd24 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:08:56.663044 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-f6ff6287-7b3e-4f0c-9128-ae7d31b5fd24 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd749cd77cf5c489e8c4e57e02ce768fd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'deb1e913bedd4fb5b286e21c60e27a41', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:08:56.672086 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-f6ff6287-7b3e-4f0c-9128-ae7d31b5fd24 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] 10.222.0.16 "GET /compute/v2.1/servers/d9281e80-5a8e-4a1e-9e1f-4de3b0dc6131" status: 200 len: 1359 microversion: 2.1 time: 0.190509 Jul 16 22:08:56.673429 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3404/6800] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:08:56 2022] GET /compute/v2.1/servers/d9281e80-5a8e-4a1e-9e1f-4de3b0dc6131 => generated 1359 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:08:57.691148 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-af504c41-91fb-42ce-83f3-5489c7934c65 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:08:57.692041 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-af504c41-91fb-42ce-83f3-5489c7934c65 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] [instance: d9281e80-5a8e-4a1e-9e1f-4de3b0dc6131] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:08:57.700462 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-af504c41-91fb-42ce-83f3-5489c7934c65 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:08:57.701119 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-af504c41-91fb-42ce-83f3-5489c7934c65 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:08:57.702076 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-af504c41-91fb-42ce-83f3-5489c7934c65 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:08:57.702529 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-af504c41-91fb-42ce-83f3-5489c7934c65 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:08:57.860846 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-af504c41-91fb-42ce-83f3-5489c7934c65 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd749cd77cf5c489e8c4e57e02ce768fd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'deb1e913bedd4fb5b286e21c60e27a41', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:08:57.869429 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-af504c41-91fb-42ce-83f3-5489c7934c65 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] 10.222.0.16 "GET /compute/v2.1/servers/d9281e80-5a8e-4a1e-9e1f-4de3b0dc6131" status: 200 len: 1359 microversion: 2.1 time: 0.182944 Jul 16 22:08:57.870264 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3397/6801] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:08:57 2022] GET /compute/v2.1/servers/d9281e80-5a8e-4a1e-9e1f-4de3b0dc6131 => generated 1359 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:08:58.887722 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-f7a55175-00c4-4c55-8865-cc5e7ee2b969 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:08:58.888359 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-f7a55175-00c4-4c55-8865-cc5e7ee2b969 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] [instance: d9281e80-5a8e-4a1e-9e1f-4de3b0dc6131] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:08:58.891856 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell0:SELECT=2 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:08:58.892547 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell0:SELECT=6 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:08:58.897473 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f7a55175-00c4-4c55-8865-cc5e7ee2b969 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:08:58.897940 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f7a55175-00c4-4c55-8865-cc5e7ee2b969 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:08:58.898816 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f7a55175-00c4-4c55-8865-cc5e7ee2b969 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:08:58.899425 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f7a55175-00c4-4c55-8865-cc5e7ee2b969 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:08:59.042649 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-f7a55175-00c4-4c55-8865-cc5e7ee2b969 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd749cd77cf5c489e8c4e57e02ce768fd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'deb1e913bedd4fb5b286e21c60e27a41', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:08:59.051288 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-f7a55175-00c4-4c55-8865-cc5e7ee2b969 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] 10.222.0.16 "GET /compute/v2.1/servers/d9281e80-5a8e-4a1e-9e1f-4de3b0dc6131" status: 200 len: 1359 microversion: 2.1 time: 0.167574 Jul 16 22:08:59.051870 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3405/6802] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:08:58 2022] GET /compute/v2.1/servers/d9281e80-5a8e-4a1e-9e1f-4de3b0dc6131 => generated 1359 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:08:59.806506 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-460f3b16-93a2-492e-9af9-6189f4f40427 req-a0589d7c-b5f6-4227-9c8b-b3a4cfad8692 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "d9281e80-5a8e-4a1e-9e1f-4de3b0dc6131", "tag": "7d4f58e2-1e5d-4e39-8582-ff5c8c553cc1"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:08:59.811177 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell0:SELECT=6 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:08:59.823188 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-460f3b16-93a2-492e-9af9-6189f4f40427 req-a0589d7c-b5f6-4227-9c8b-b3a4cfad8692 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:08:59.823642 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-460f3b16-93a2-492e-9af9-6189f4f40427 req-a0589d7c-b5f6-4227-9c8b-b3a4cfad8692 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:08:59.836634 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-460f3b16-93a2-492e-9af9-6189f4f40427 req-a0589d7c-b5f6-4227-9c8b-b3a4cfad8692 service nova] Creating event network-changed:7d4f58e2-1e5d-4e39-8582-ff5c8c553cc1 for instance d9281e80-5a8e-4a1e-9e1f-4de3b0dc6131 on nch1-master-1 Jul 16 22:08:59.841647 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-460f3b16-93a2-492e-9af9-6189f4f40427 req-a0589d7c-b5f6-4227-9c8b-b3a4cfad8692 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.039472 Jul 16 22:08:59.842437 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3398/6803] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 22:08:59 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 22:09:00.069488 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-518d6fd1-4eaf-4875-8e1a-f66eaa446c1d tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:09:00.070294 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-518d6fd1-4eaf-4875-8e1a-f66eaa446c1d tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] [instance: d9281e80-5a8e-4a1e-9e1f-4de3b0dc6131] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:09:00.079319 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-518d6fd1-4eaf-4875-8e1a-f66eaa446c1d tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:09:00.083109 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-518d6fd1-4eaf-4875-8e1a-f66eaa446c1d tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:09:00.083109 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-518d6fd1-4eaf-4875-8e1a-f66eaa446c1d tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:09:00.083109 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-518d6fd1-4eaf-4875-8e1a-f66eaa446c1d tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:09:00.224864 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-518d6fd1-4eaf-4875-8e1a-f66eaa446c1d tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd749cd77cf5c489e8c4e57e02ce768fd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'deb1e913bedd4fb5b286e21c60e27a41', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:09:00.232698 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-518d6fd1-4eaf-4875-8e1a-f66eaa446c1d tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] 10.222.0.16 "GET /compute/v2.1/servers/d9281e80-5a8e-4a1e-9e1f-4de3b0dc6131" status: 200 len: 1359 microversion: 2.1 time: 0.167952 Jul 16 22:09:00.235160 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3406/6804] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:09:00 2022] GET /compute/v2.1/servers/d9281e80-5a8e-4a1e-9e1f-4de3b0dc6131 => generated 1359 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:09:01.252149 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-cf966128-1085-4a70-b578-6be2595b3082 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:09:01.252846 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-cf966128-1085-4a70-b578-6be2595b3082 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] [instance: d9281e80-5a8e-4a1e-9e1f-4de3b0dc6131] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:09:01.259854 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cf966128-1085-4a70-b578-6be2595b3082 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:09:01.260301 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cf966128-1085-4a70-b578-6be2595b3082 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:09:01.261085 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cf966128-1085-4a70-b578-6be2595b3082 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:09:01.261536 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cf966128-1085-4a70-b578-6be2595b3082 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:09:01.395820 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-cf966128-1085-4a70-b578-6be2595b3082 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd749cd77cf5c489e8c4e57e02ce768fd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'deb1e913bedd4fb5b286e21c60e27a41', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:09:01.403284 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-cf966128-1085-4a70-b578-6be2595b3082 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] 10.222.0.16 "GET /compute/v2.1/servers/d9281e80-5a8e-4a1e-9e1f-4de3b0dc6131" status: 200 len: 1359 microversion: 2.1 time: 0.157115 Jul 16 22:09:01.403865 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3399/6805] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:09:01 2022] GET /compute/v2.1/servers/d9281e80-5a8e-4a1e-9e1f-4de3b0dc6131 => generated 1359 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:09:02.420394 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-53b15d9c-1b81-4b76-9697-082883830737 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:09:02.420976 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-53b15d9c-1b81-4b76-9697-082883830737 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] [instance: d9281e80-5a8e-4a1e-9e1f-4de3b0dc6131] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:09:02.427959 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-53b15d9c-1b81-4b76-9697-082883830737 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:09:02.428422 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-53b15d9c-1b81-4b76-9697-082883830737 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:09:02.429193 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-53b15d9c-1b81-4b76-9697-082883830737 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:09:02.429644 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-53b15d9c-1b81-4b76-9697-082883830737 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:09:02.564863 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-53b15d9c-1b81-4b76-9697-082883830737 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd749cd77cf5c489e8c4e57e02ce768fd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'deb1e913bedd4fb5b286e21c60e27a41', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:09:02.572514 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-53b15d9c-1b81-4b76-9697-082883830737 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] 10.222.0.16 "GET /compute/v2.1/servers/d9281e80-5a8e-4a1e-9e1f-4de3b0dc6131" status: 200 len: 1359 microversion: 2.1 time: 0.156133 Jul 16 22:09:02.573115 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3407/6806] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:09:02 2022] GET /compute/v2.1/servers/d9281e80-5a8e-4a1e-9e1f-4de3b0dc6131 => generated 1359 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:09:03.589864 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-4f455a6d-af1c-410a-8fd7-326e824054e8 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:09:03.590507 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-4f455a6d-af1c-410a-8fd7-326e824054e8 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] [instance: d9281e80-5a8e-4a1e-9e1f-4de3b0dc6131] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:09:03.597710 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4f455a6d-af1c-410a-8fd7-326e824054e8 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:09:03.598166 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4f455a6d-af1c-410a-8fd7-326e824054e8 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:09:03.599000 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4f455a6d-af1c-410a-8fd7-326e824054e8 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:09:03.599496 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4f455a6d-af1c-410a-8fd7-326e824054e8 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:09:03.730293 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-4f455a6d-af1c-410a-8fd7-326e824054e8 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd749cd77cf5c489e8c4e57e02ce768fd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'deb1e913bedd4fb5b286e21c60e27a41', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:09:03.738418 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-4f455a6d-af1c-410a-8fd7-326e824054e8 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] 10.222.0.16 "GET /compute/v2.1/servers/d9281e80-5a8e-4a1e-9e1f-4de3b0dc6131" status: 200 len: 1359 microversion: 2.1 time: 0.152532 Jul 16 22:09:03.739002 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3400/6807] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:09:03 2022] GET /compute/v2.1/servers/d9281e80-5a8e-4a1e-9e1f-4de3b0dc6131 => generated 1359 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:09:04.755357 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-356bc661-4443-4669-9983-97fa252def61 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:09:04.755981 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-356bc661-4443-4669-9983-97fa252def61 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] [instance: d9281e80-5a8e-4a1e-9e1f-4de3b0dc6131] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:09:04.762125 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-356bc661-4443-4669-9983-97fa252def61 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:09:04.762587 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-356bc661-4443-4669-9983-97fa252def61 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:09:04.763388 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-356bc661-4443-4669-9983-97fa252def61 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:09:04.763840 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-356bc661-4443-4669-9983-97fa252def61 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:09:04.900094 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-356bc661-4443-4669-9983-97fa252def61 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd749cd77cf5c489e8c4e57e02ce768fd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'deb1e913bedd4fb5b286e21c60e27a41', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:09:04.907632 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-356bc661-4443-4669-9983-97fa252def61 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] 10.222.0.16 "GET /compute/v2.1/servers/d9281e80-5a8e-4a1e-9e1f-4de3b0dc6131" status: 200 len: 1359 microversion: 2.1 time: 0.157039 Jul 16 22:09:04.908715 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3408/6808] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:09:04 2022] GET /compute/v2.1/servers/d9281e80-5a8e-4a1e-9e1f-4de3b0dc6131 => generated 1359 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:09:05.442921 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-0bd450af-49a4-43df-84a0-83833b335d00 req-840a8802-c929-4d1c-957b-1272fc2fc78b service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "d9281e80-5a8e-4a1e-9e1f-4de3b0dc6131", "name": "network-vif-plugged", "status": "completed", "tag": "7d4f58e2-1e5d-4e39-8582-ff5c8c553cc1"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:09:05.451801 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-0bd450af-49a4-43df-84a0-83833b335d00 req-840a8802-c929-4d1c-957b-1272fc2fc78b service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:09:05.452245 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-0bd450af-49a4-43df-84a0-83833b335d00 req-840a8802-c929-4d1c-957b-1272fc2fc78b service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:09:05.463925 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-0bd450af-49a4-43df-84a0-83833b335d00 req-840a8802-c929-4d1c-957b-1272fc2fc78b service nova] Creating event network-vif-plugged:7d4f58e2-1e5d-4e39-8582-ff5c8c553cc1 for instance d9281e80-5a8e-4a1e-9e1f-4de3b0dc6131 on nch1-master-1 Jul 16 22:09:05.468866 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-0bd450af-49a4-43df-84a0-83833b335d00 req-840a8802-c929-4d1c-957b-1272fc2fc78b service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.030251 Jul 16 22:09:05.469457 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3401/6809] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 22:09:05 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 22:09:05.926006 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-bb5dd483-b744-47df-ae40-036dc33db190 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:09:05.926646 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-bb5dd483-b744-47df-ae40-036dc33db190 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] [instance: d9281e80-5a8e-4a1e-9e1f-4de3b0dc6131] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:09:05.933742 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-bb5dd483-b744-47df-ae40-036dc33db190 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:09:05.934216 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-bb5dd483-b744-47df-ae40-036dc33db190 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:09:05.935001 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-bb5dd483-b744-47df-ae40-036dc33db190 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:09:05.935550 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-bb5dd483-b744-47df-ae40-036dc33db190 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:09:06.072434 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-bb5dd483-b744-47df-ae40-036dc33db190 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd749cd77cf5c489e8c4e57e02ce768fd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'deb1e913bedd4fb5b286e21c60e27a41', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:09:06.080837 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-bb5dd483-b744-47df-ae40-036dc33db190 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] 10.222.0.16 "GET /compute/v2.1/servers/d9281e80-5a8e-4a1e-9e1f-4de3b0dc6131" status: 200 len: 1359 microversion: 2.1 time: 0.159035 Jul 16 22:09:06.082793 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3409/6810] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:09:05 2022] GET /compute/v2.1/servers/d9281e80-5a8e-4a1e-9e1f-4de3b0dc6131 => generated 1359 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:09:07.099806 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-516f1756-4189-4de5-9bfe-34a9bbc716ac tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:09:07.100586 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-516f1756-4189-4de5-9bfe-34a9bbc716ac tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] [instance: d9281e80-5a8e-4a1e-9e1f-4de3b0dc6131] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:09:07.106897 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-516f1756-4189-4de5-9bfe-34a9bbc716ac tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:09:07.107392 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-516f1756-4189-4de5-9bfe-34a9bbc716ac tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:09:07.108167 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-516f1756-4189-4de5-9bfe-34a9bbc716ac tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:09:07.108612 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-516f1756-4189-4de5-9bfe-34a9bbc716ac tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:09:07.239432 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-516f1756-4189-4de5-9bfe-34a9bbc716ac tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd749cd77cf5c489e8c4e57e02ce768fd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'deb1e913bedd4fb5b286e21c60e27a41', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:09:07.247051 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-516f1756-4189-4de5-9bfe-34a9bbc716ac tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] 10.222.0.16 "GET /compute/v2.1/servers/d9281e80-5a8e-4a1e-9e1f-4de3b0dc6131" status: 200 len: 1550 microversion: 2.1 time: 0.152921 Jul 16 22:09:07.247632 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3402/6811] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:09:07 2022] GET /compute/v2.1/servers/d9281e80-5a8e-4a1e-9e1f-4de3b0dc6131 => generated 1550 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:09:07.263654 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-1197e242-3de4-4c31-afbd-aa0351fba70b tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:09:07.264312 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-1197e242-3de4-4c31-afbd-aa0351fba70b tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] [instance: d9281e80-5a8e-4a1e-9e1f-4de3b0dc6131] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:09:07.271378 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1197e242-3de4-4c31-afbd-aa0351fba70b tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:09:07.272576 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1197e242-3de4-4c31-afbd-aa0351fba70b tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:09:07.272706 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1197e242-3de4-4c31-afbd-aa0351fba70b tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:09:07.273058 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1197e242-3de4-4c31-afbd-aa0351fba70b tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:09:07.406313 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-1197e242-3de4-4c31-afbd-aa0351fba70b tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd749cd77cf5c489e8c4e57e02ce768fd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'deb1e913bedd4fb5b286e21c60e27a41', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:09:07.414765 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-1197e242-3de4-4c31-afbd-aa0351fba70b tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] 10.222.0.16 "GET /compute/v2.1/servers/d9281e80-5a8e-4a1e-9e1f-4de3b0dc6131" status: 200 len: 1550 microversion: 2.1 time: 0.155049 Jul 16 22:09:07.415379 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3410/6812] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:09:07 2022] GET /compute/v2.1/servers/d9281e80-5a8e-4a1e-9e1f-4de3b0dc6131 => generated 1550 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:09:07.430546 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-a825fd0b-b3c6-47c8-8746-006745dcf957 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:09:07.433811 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-a825fd0b-b3c6-47c8-8746-006745dcf957 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Policy check for os_compute_api:servers:allow_all_filters failed with credentials {'is_admin': False, 'user_id': 'd749cd77cf5c489e8c4e57e02ce768fd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'deb1e913bedd4fb5b286e21c60e27a41', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:09:07.434574 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-a825fd0b-b3c6-47c8-8746-006745dcf957 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Searching by: {'deleted': False, 'project_id': 'deb1e913bedd4fb5b286e21c60e27a41'} {{(pid=138680) get_all /opt/stack/nova/nova/compute/api.py:2962}} Jul 16 22:09:07.440160 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a825fd0b-b3c6-47c8-8746-006745dcf957 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:09:07.440539 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a825fd0b-b3c6-47c8-8746-006745dcf957 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:09:07.441324 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a825fd0b-b3c6-47c8-8746-006745dcf957 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:09:07.441759 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a825fd0b-b3c6-47c8-8746-006745dcf957 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:09:07.465577 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.multi_cell_list [None req-a825fd0b-b3c6-47c8-8746-006745dcf957 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Listed batch of 1 results from cell out of 1000 limit. Returned 1 total so far. {{(pid=138680) do_query /opt/stack/nova/nova/compute/multi_cell_list.py:373}} Jul 16 22:09:07.491966 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-a825fd0b-b3c6-47c8-8746-006745dcf957 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] 10.222.0.16 "GET /compute/v2.1/servers" status: 200 len: 343 microversion: 2.1 time: 0.065221 Jul 16 22:09:07.492542 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3403/6813] 10.222.0.16 () {58 vars in 1124 bytes} [Sat Jul 16 22:09:07 2022] GET /compute/v2.1/servers => generated 343 bytes in 66 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 22:09:07.503256 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-2b0932c4-681b-4ac7-975c-100f9c1fe741 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:09:07.504036 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-2b0932c4-681b-4ac7-975c-100f9c1fe741 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] [instance: d9281e80-5a8e-4a1e-9e1f-4de3b0dc6131] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:09:07.510174 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2b0932c4-681b-4ac7-975c-100f9c1fe741 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:09:07.510644 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2b0932c4-681b-4ac7-975c-100f9c1fe741 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:09:07.511468 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2b0932c4-681b-4ac7-975c-100f9c1fe741 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:09:07.511928 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2b0932c4-681b-4ac7-975c-100f9c1fe741 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:09:07.640540 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-2b0932c4-681b-4ac7-975c-100f9c1fe741 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd749cd77cf5c489e8c4e57e02ce768fd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'deb1e913bedd4fb5b286e21c60e27a41', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:09:07.648898 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-2b0932c4-681b-4ac7-975c-100f9c1fe741 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] 10.222.0.16 "GET /compute/v2.1/servers/d9281e80-5a8e-4a1e-9e1f-4de3b0dc6131" status: 200 len: 1550 microversion: 2.1 time: 0.149993 Jul 16 22:09:07.649477 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3411/6814] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:09:07 2022] GET /compute/v2.1/servers/d9281e80-5a8e-4a1e-9e1f-4de3b0dc6131 => generated 1550 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:09:09.503551 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-733fd47a-49d9-49ee-9289-febfd300f9db tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "3a5baf74-a675-4ae1-962b-69743c9186ec"}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:09:09.504826 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-733fd47a-49d9-49ee-9289-febfd300f9db tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] [instance: d9281e80-5a8e-4a1e-9e1f-4de3b0dc6131] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:09:09.512050 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-733fd47a-49d9-49ee-9289-febfd300f9db tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:09:09.512515 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-733fd47a-49d9-49ee-9289-febfd300f9db tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:09:09.513286 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-733fd47a-49d9-49ee-9289-febfd300f9db tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:09:09.513736 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-733fd47a-49d9-49ee-9289-febfd300f9db tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:09:10.243481 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-733fd47a-49d9-49ee-9289-febfd300f9db tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] 10.222.0.16 "POST /compute/v2.1/servers/d9281e80-5a8e-4a1e-9e1f-4de3b0dc6131/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.744274 Jul 16 22:09:10.244081 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3404/6815] 10.222.0.16 () {60 vars in 1322 bytes} [Sat Jul 16 22:09:09 2022] POST /compute/v2.1/servers/d9281e80-5a8e-4a1e-9e1f-4de3b0dc6131/os-volume_attachments => generated 194 bytes in 746 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 22:09:16.096046 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-1292ec6c-fe01-4a58-a954-cace6e9ccda4 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:09:16.096683 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-1292ec6c-fe01-4a58-a954-cace6e9ccda4 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] [instance: d9281e80-5a8e-4a1e-9e1f-4de3b0dc6131] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:09:16.104389 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1292ec6c-fe01-4a58-a954-cace6e9ccda4 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:09:16.104867 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1292ec6c-fe01-4a58-a954-cace6e9ccda4 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:09:16.105653 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1292ec6c-fe01-4a58-a954-cace6e9ccda4 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:09:16.106107 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1292ec6c-fe01-4a58-a954-cace6e9ccda4 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:09:16.236771 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-1292ec6c-fe01-4a58-a954-cace6e9ccda4 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd749cd77cf5c489e8c4e57e02ce768fd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'deb1e913bedd4fb5b286e21c60e27a41', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:09:16.245569 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-1292ec6c-fe01-4a58-a954-cace6e9ccda4 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] 10.222.0.16 "GET /compute/v2.1/servers/d9281e80-5a8e-4a1e-9e1f-4de3b0dc6131" status: 200 len: 1596 microversion: 2.1 time: 0.153505 Jul 16 22:09:16.246154 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3412/6816] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:09:16 2022] GET /compute/v2.1/servers/d9281e80-5a8e-4a1e-9e1f-4de3b0dc6131 => generated 1596 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:09:18.197076 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-761b2252-43a8-481e-9745-189ea8016238 req-e8c08e9c-c316-4517-a13f-bad5e5f8f785 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "d9281e80-5a8e-4a1e-9e1f-4de3b0dc6131", "tag": "7d4f58e2-1e5d-4e39-8582-ff5c8c553cc1"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:09:18.201858 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell0:SELECT=2 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:09:18.202677 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_api:SELECT=38,nova_api:INSERT=1 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:09:18.208206 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-761b2252-43a8-481e-9745-189ea8016238 req-e8c08e9c-c316-4517-a13f-bad5e5f8f785 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:09:18.209733 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-761b2252-43a8-481e-9745-189ea8016238 req-e8c08e9c-c316-4517-a13f-bad5e5f8f785 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:09:18.223715 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-761b2252-43a8-481e-9745-189ea8016238 req-e8c08e9c-c316-4517-a13f-bad5e5f8f785 service nova] Creating event network-changed:7d4f58e2-1e5d-4e39-8582-ff5c8c553cc1 for instance d9281e80-5a8e-4a1e-9e1f-4de3b0dc6131 on nch1-master-1 Jul 16 22:09:18.228957 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-761b2252-43a8-481e-9745-189ea8016238 req-e8c08e9c-c316-4517-a13f-bad5e5f8f785 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.036577 Jul 16 22:09:18.229675 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3405/6817] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 22:09:18 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 22:09:20.655333 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-f6ef89d1-c8fa-4283-860c-00d3798a54c4 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Action: 'action', calling method: >, body: {"addSecurityGroup": {"name": "tempest-secgroup-smoke-1090120400"}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:09:20.655932 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-f6ef89d1-c8fa-4283-860c-00d3798a54c4 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] [instance: d9281e80-5a8e-4a1e-9e1f-4de3b0dc6131] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:09:20.658720 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_api:SELECT=40,nova_api:INSERT=3 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:09:20.663612 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f6ef89d1-c8fa-4283-860c-00d3798a54c4 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:09:20.664089 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f6ef89d1-c8fa-4283-860c-00d3798a54c4 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:09:20.664990 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f6ef89d1-c8fa-4283-860c-00d3798a54c4 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:09:20.665405 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f6ef89d1-c8fa-4283-860c-00d3798a54c4 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:09:20.669131 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=75 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:09:20.796074 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.network.security_group_api [None req-f6ef89d1-c8fa-4283-860c-00d3798a54c4 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Adding security group da962d39-df8d-46b0-845c-3092cc3e2f54 to port 7d4f58e2-1e5d-4e39-8582-ff5c8c553cc1 Jul 16 22:09:21.199937 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-f6ef89d1-c8fa-4283-860c-00d3798a54c4 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] 10.222.0.16 "POST /compute/v2.1/servers/d9281e80-5a8e-4a1e-9e1f-4de3b0dc6131/action" status: 202 len: 0 microversion: 2.1 time: 0.548619 Jul 16 22:09:21.203660 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3413/6818] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 22:09:20 2022] POST /compute/v2.1/servers/d9281e80-5a8e-4a1e-9e1f-4de3b0dc6131/action => generated 0 bytes in 550 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 22:09:21.212080 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-b75ea75d-7d29-49fd-a65d-4d6205a2c64c tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:09:21.212080 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-b75ea75d-7d29-49fd-a65d-4d6205a2c64c tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] [instance: d9281e80-5a8e-4a1e-9e1f-4de3b0dc6131] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:09:21.217693 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-17239140-4a7e-4169-9a4c-8e115ffa7b38 req-0fd72853-04bf-4bdc-bffd-a4378c79edc0 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "d9281e80-5a8e-4a1e-9e1f-4de3b0dc6131", "tag": "7d4f58e2-1e5d-4e39-8582-ff5c8c553cc1"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:09:21.234793 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-17239140-4a7e-4169-9a4c-8e115ffa7b38 req-0fd72853-04bf-4bdc-bffd-a4378c79edc0 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:09:21.236892 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b75ea75d-7d29-49fd-a65d-4d6205a2c64c tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:09:21.237583 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b75ea75d-7d29-49fd-a65d-4d6205a2c64c tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:09:21.237727 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-17239140-4a7e-4169-9a4c-8e115ffa7b38 req-0fd72853-04bf-4bdc-bffd-a4378c79edc0 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:09:21.238417 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b75ea75d-7d29-49fd-a65d-4d6205a2c64c tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:09:21.238871 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b75ea75d-7d29-49fd-a65d-4d6205a2c64c tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:09:21.262149 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-17239140-4a7e-4169-9a4c-8e115ffa7b38 req-0fd72853-04bf-4bdc-bffd-a4378c79edc0 service nova] Creating event network-changed:7d4f58e2-1e5d-4e39-8582-ff5c8c553cc1 for instance d9281e80-5a8e-4a1e-9e1f-4de3b0dc6131 on nch1-master-1 Jul 16 22:09:21.268852 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-17239140-4a7e-4169-9a4c-8e115ffa7b38 req-0fd72853-04bf-4bdc-bffd-a4378c79edc0 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.054983 Jul 16 22:09:21.269482 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3414/6819] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 22:09:21 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 22:09:21.433125 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-b75ea75d-7d29-49fd-a65d-4d6205a2c64c tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd749cd77cf5c489e8c4e57e02ce768fd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'deb1e913bedd4fb5b286e21c60e27a41', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:09:21.443968 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-b75ea75d-7d29-49fd-a65d-4d6205a2c64c tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] 10.222.0.16 "GET /compute/v2.1/servers/d9281e80-5a8e-4a1e-9e1f-4de3b0dc6131" status: 200 len: 1761 microversion: 2.1 time: 0.238744 Jul 16 22:09:21.444972 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3406/6820] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:09:21 2022] GET /compute/v2.1/servers/d9281e80-5a8e-4a1e-9e1f-4de3b0dc6131 => generated 1761 bytes in 240 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:09:46.136203 ncdv-master-1 devstack@n-api.service[138681]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jul 16 22:09:46.136756 ncdv-master-1 devstack@n-api.service[138681]: Sat Jul 16 22:09:46 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Jul 16 22:11:24.790254 ncdv-master-1 devstack@n-api.service[138680]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jul 16 22:11:24.790877 ncdv-master-1 devstack@n-api.service[138680]: Sat Jul 16 22:11:24 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Jul 16 22:11:26.860399 ncdv-master-1 devstack@n-api.service[138681]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jul 16 22:11:26.860970 ncdv-master-1 devstack@n-api.service[138681]: Sat Jul 16 22:11:26 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Jul 16 22:11:32.362811 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_api:SELECT=3 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:11:32.363604 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=5 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:11:32.367883 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-2fbaeca5-ce89-4de1-910a-b8338c1c9b6f tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Action: 'action', calling method: >, body: {"reboot": {"type": "HARD"}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:11:32.368942 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-2fbaeca5-ce89-4de1-910a-b8338c1c9b6f tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] [instance: d9281e80-5a8e-4a1e-9e1f-4de3b0dc6131] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:11:32.375931 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2fbaeca5-ce89-4de1-910a-b8338c1c9b6f tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:11:32.376460 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2fbaeca5-ce89-4de1-910a-b8338c1c9b6f tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:11:32.377236 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2fbaeca5-ce89-4de1-910a-b8338c1c9b6f tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:11:32.377687 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2fbaeca5-ce89-4de1-910a-b8338c1c9b6f tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:11:32.587989 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-2fbaeca5-ce89-4de1-910a-b8338c1c9b6f tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] 10.222.0.16 "POST /compute/v2.1/servers/d9281e80-5a8e-4a1e-9e1f-4de3b0dc6131/action" status: 202 len: 0 microversion: 2.1 time: 0.227022 Jul 16 22:11:32.588562 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3415/6821] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 22:11:32 2022] POST /compute/v2.1/servers/d9281e80-5a8e-4a1e-9e1f-4de3b0dc6131/action => generated 0 bytes in 229 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 22:11:32.595712 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_api:SELECT=3 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:11:32.596489 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=73,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:11:32.600425 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-bb9e6632-0831-400e-aa45-27608873f804 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:11:32.601042 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-bb9e6632-0831-400e-aa45-27608873f804 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] [instance: d9281e80-5a8e-4a1e-9e1f-4de3b0dc6131] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:11:32.607046 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-bb9e6632-0831-400e-aa45-27608873f804 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:11:32.607524 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-bb9e6632-0831-400e-aa45-27608873f804 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:11:32.608306 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-bb9e6632-0831-400e-aa45-27608873f804 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:11:32.608739 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-bb9e6632-0831-400e-aa45-27608873f804 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:11:32.899590 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-bb9e6632-0831-400e-aa45-27608873f804 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd749cd77cf5c489e8c4e57e02ce768fd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'deb1e913bedd4fb5b286e21c60e27a41', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:11:32.909354 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-bb9e6632-0831-400e-aa45-27608873f804 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] 10.222.0.16 "GET /compute/v2.1/servers/d9281e80-5a8e-4a1e-9e1f-4de3b0dc6131" status: 200 len: 1763 microversion: 2.1 time: 0.316283 Jul 16 22:11:32.910085 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3407/6822] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:11:32 2022] GET /compute/v2.1/servers/d9281e80-5a8e-4a1e-9e1f-4de3b0dc6131 => generated 1763 bytes in 318 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 22:11:33.928600 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-00a2132c-bd0e-47bf-9973-3482fc96602f tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:11:33.929418 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-00a2132c-bd0e-47bf-9973-3482fc96602f tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] [instance: d9281e80-5a8e-4a1e-9e1f-4de3b0dc6131] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:11:33.932767 ncdv-master-1 devstack@n-api.service[138681]: ERROR oslo.messaging._drivers.impl_rabbit [-] [e4397b8a-69d0-4b1f-ae48-d345afeb4816] AMQP server on 10.222.0.16:5672 is unreachable: . Trying again in 1 seconds.: amqp.exceptions.RecoverableConnectionError: Jul 16 22:11:33.933697 ncdv-master-1 devstack@n-api.service[138681]: ERROR oslo.messaging._drivers.impl_rabbit [-] [4de8847c-05a2-4cd6-bd0e-256d7a754675] AMQP server on 10.222.0.16:5672 is unreachable: . Trying again in 1 seconds.: amqp.exceptions.RecoverableConnectionError: Jul 16 22:11:33.939351 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-00a2132c-bd0e-47bf-9973-3482fc96602f tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:11:33.939821 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-00a2132c-bd0e-47bf-9973-3482fc96602f tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:11:33.940607 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-00a2132c-bd0e-47bf-9973-3482fc96602f tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:11:33.941058 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-00a2132c-bd0e-47bf-9973-3482fc96602f tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:11:34.142886 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-00a2132c-bd0e-47bf-9973-3482fc96602f tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd749cd77cf5c489e8c4e57e02ce768fd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'deb1e913bedd4fb5b286e21c60e27a41', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:11:34.151030 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-00a2132c-bd0e-47bf-9973-3482fc96602f tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] 10.222.0.16 "GET /compute/v2.1/servers/d9281e80-5a8e-4a1e-9e1f-4de3b0dc6131" status: 200 len: 1761 microversion: 2.1 time: 0.227335 Jul 16 22:11:34.151605 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3416/6823] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:11:33 2022] GET /compute/v2.1/servers/d9281e80-5a8e-4a1e-9e1f-4de3b0dc6131 => generated 1761 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:11:59.854756 ncdv-master-1 devstack@n-api.service[138680]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jul 16 22:11:59.855479 ncdv-master-1 devstack@n-api.service[138680]: Sat Jul 16 22:11:59 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Jul 16 22:12:47.295488 ncdv-master-1 devstack@n-api.service[138680]: ERROR oslo.messaging._drivers.impl_rabbit [-] [a302809d-3282-48f5-9662-a738982e3059] AMQP server on 10.222.0.16:5672 is unreachable: . Trying again in 1 seconds.: amqp.exceptions.RecoverableConnectionError: Jul 16 22:12:47.296270 ncdv-master-1 devstack@n-api.service[138680]: ERROR oslo.messaging._drivers.impl_rabbit [-] [7d526b5e-7cab-48f3-be00-1b242d865b86] AMQP server on 10.222.0.16:5672 is unreachable: . Trying again in 1 seconds.: amqp.exceptions.RecoverableConnectionError: Jul 16 22:12:47.296981 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_api:SELECT=2 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:12:47.297723 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=6 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:12:47.301800 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-40a25c35-841d-47a2-a8a5-36eb80c746a3 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Action: 'action', calling method: >, body: {"removeSecurityGroup": {"name": "tempest-secgroup-smoke-1090120400"}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:12:47.302338 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-40a25c35-841d-47a2-a8a5-36eb80c746a3 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] [instance: d9281e80-5a8e-4a1e-9e1f-4de3b0dc6131] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:12:47.309644 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-40a25c35-841d-47a2-a8a5-36eb80c746a3 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:12:47.310189 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-40a25c35-841d-47a2-a8a5-36eb80c746a3 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:12:47.310965 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-40a25c35-841d-47a2-a8a5-36eb80c746a3 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:12:47.311432 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-40a25c35-841d-47a2-a8a5-36eb80c746a3 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:12:47.448757 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.network.security_group_api [None req-40a25c35-841d-47a2-a8a5-36eb80c746a3 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Removing security group da962d39-df8d-46b0-845c-3092cc3e2f54 from port 7d4f58e2-1e5d-4e39-8582-ff5c8c553cc1 Jul 16 22:12:47.959323 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-40a25c35-841d-47a2-a8a5-36eb80c746a3 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] 10.222.0.16 "POST /compute/v2.1/servers/d9281e80-5a8e-4a1e-9e1f-4de3b0dc6131/action" status: 202 len: 0 microversion: 2.1 time: 0.665496 Jul 16 22:12:47.960240 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3408/6824] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 22:12:47 2022] POST /compute/v2.1/servers/d9281e80-5a8e-4a1e-9e1f-4de3b0dc6131/action => generated 0 bytes in 667 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 22:12:47.979294 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_api:SELECT=4 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:12:47.980683 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=16,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:12:47.995381 ncdv-master-1 devstack@n-api.service[138681]: INFO oslo.messaging._drivers.impl_rabbit [-] [4de8847c-05a2-4cd6-bd0e-256d7a754675] Reconnected to AMQP server on 10.222.0.16:5672 via [amqp] client with port 60182. Jul 16 22:12:47.996385 ncdv-master-1 devstack@n-api.service[138681]: INFO oslo.messaging._drivers.impl_rabbit [-] [e4397b8a-69d0-4b1f-ae48-d345afeb4816] Reconnected to AMQP server on 10.222.0.16:5672 via [amqp] client with port 60180. Jul 16 22:12:48.094487 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-a42284fc-587a-4dec-b4bd-fcf947fd75f8 req-a801eebd-7962-46c5-8654-7213e3af694b service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "d9281e80-5a8e-4a1e-9e1f-4de3b0dc6131", "tag": "7d4f58e2-1e5d-4e39-8582-ff5c8c553cc1"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:12:48.109155 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-a42284fc-587a-4dec-b4bd-fcf947fd75f8 req-a801eebd-7962-46c5-8654-7213e3af694b service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:12:48.109723 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-a42284fc-587a-4dec-b4bd-fcf947fd75f8 req-a801eebd-7962-46c5-8654-7213e3af694b service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:12:48.140287 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-a42284fc-587a-4dec-b4bd-fcf947fd75f8 req-a801eebd-7962-46c5-8654-7213e3af694b service nova] Creating event network-changed:7d4f58e2-1e5d-4e39-8582-ff5c8c553cc1 for instance d9281e80-5a8e-4a1e-9e1f-4de3b0dc6131 on nch1-master-1 Jul 16 22:12:48.148600 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-a42284fc-587a-4dec-b4bd-fcf947fd75f8 req-a801eebd-7962-46c5-8654-7213e3af694b service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.172742 Jul 16 22:12:48.152933 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3417/6825] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 22:12:47 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 22:12:49.606446 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-4000fe36-9fea-45d7-8b2a-6b1d8ef739c1 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:12:49.607062 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-4000fe36-9fea-45d7-8b2a-6b1d8ef739c1 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] [instance: d9281e80-5a8e-4a1e-9e1f-4de3b0dc6131] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:12:49.616086 ncdv-master-1 devstack@n-api.service[138680]: INFO oslo.messaging._drivers.impl_rabbit [-] [7d526b5e-7cab-48f3-be00-1b242d865b86] Reconnected to AMQP server on 10.222.0.16:5672 via [amqp] client with port 60202. Jul 16 22:12:49.622845 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4000fe36-9fea-45d7-8b2a-6b1d8ef739c1 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:12:49.623347 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4000fe36-9fea-45d7-8b2a-6b1d8ef739c1 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:12:49.624128 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4000fe36-9fea-45d7-8b2a-6b1d8ef739c1 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:12:49.624583 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4000fe36-9fea-45d7-8b2a-6b1d8ef739c1 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:12:49.630815 ncdv-master-1 devstack@n-api.service[138680]: INFO oslo.messaging._drivers.impl_rabbit [-] [a302809d-3282-48f5-9662-a738982e3059] Reconnected to AMQP server on 10.222.0.16:5672 via [amqp] client with port 60200. Jul 16 22:12:49.710655 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.objects.instance [None req-4000fe36-9fea-45d7-8b2a-6b1d8ef739c1 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lazy-loading 'services' on Instance uuid d9281e80-5a8e-4a1e-9e1f-4de3b0dc6131 {{(pid=138680) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 22:12:49.903059 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-4000fe36-9fea-45d7-8b2a-6b1d8ef739c1 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] 10.222.0.16 "DELETE /compute/v2.1/servers/d9281e80-5a8e-4a1e-9e1f-4de3b0dc6131/os-volume_attachments/3a5baf74-a675-4ae1-962b-69743c9186ec" status: 202 len: 0 microversion: 2.1 time: 0.304938 Jul 16 22:12:49.903668 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3409/6826] 10.222.0.16 () {58 vars in 1415 bytes} [Sat Jul 16 22:12:49 2022] DELETE /compute/v2.1/servers/d9281e80-5a8e-4a1e-9e1f-4de3b0dc6131/os-volume_attachments/3a5baf74-a675-4ae1-962b-69743c9186ec => generated 0 bytes in 307 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 22:12:49.915135 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-96f6906b-52e7-4e1a-b66d-57afa7b1455b tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:12:49.916537 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-96f6906b-52e7-4e1a-b66d-57afa7b1455b tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] [instance: d9281e80-5a8e-4a1e-9e1f-4de3b0dc6131] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:12:49.923925 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-96f6906b-52e7-4e1a-b66d-57afa7b1455b tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:12:49.924402 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-96f6906b-52e7-4e1a-b66d-57afa7b1455b tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:12:49.925205 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-96f6906b-52e7-4e1a-b66d-57afa7b1455b tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:12:49.925683 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-96f6906b-52e7-4e1a-b66d-57afa7b1455b tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:12:49.957603 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-96f6906b-52e7-4e1a-b66d-57afa7b1455b tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] 10.222.0.16 "GET /compute/v2.1/servers/d9281e80-5a8e-4a1e-9e1f-4de3b0dc6131/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049224 Jul 16 22:12:49.958376 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3418/6827] 10.222.0.16 () {58 vars in 1301 bytes} [Sat Jul 16 22:12:49 2022] GET /compute/v2.1/servers/d9281e80-5a8e-4a1e-9e1f-4de3b0dc6131/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 16 22:12:50.165097 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-68f840a3-f668-4303-ac77-724a9229c772 req-48ac6e85-a683-4157-b9c6-39c7816a471f service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "d9281e80-5a8e-4a1e-9e1f-4de3b0dc6131", "tag": "7d4f58e2-1e5d-4e39-8582-ff5c8c553cc1"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:12:50.173519 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-68f840a3-f668-4303-ac77-724a9229c772 req-48ac6e85-a683-4157-b9c6-39c7816a471f service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:12:50.173960 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-68f840a3-f668-4303-ac77-724a9229c772 req-48ac6e85-a683-4157-b9c6-39c7816a471f service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:12:50.185225 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-68f840a3-f668-4303-ac77-724a9229c772 req-48ac6e85-a683-4157-b9c6-39c7816a471f service nova] Creating event network-changed:7d4f58e2-1e5d-4e39-8582-ff5c8c553cc1 for instance d9281e80-5a8e-4a1e-9e1f-4de3b0dc6131 on nch1-master-1 Jul 16 22:12:50.189979 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-68f840a3-f668-4303-ac77-724a9229c772 req-48ac6e85-a683-4157-b9c6-39c7816a471f service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.029022 Jul 16 22:12:50.190577 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3410/6828] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 22:12:50 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 31 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 22:12:50.970375 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-7c613104-1f2b-4748-9eca-9034e7f5d046 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:12:50.971363 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-7c613104-1f2b-4748-9eca-9034e7f5d046 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] [instance: d9281e80-5a8e-4a1e-9e1f-4de3b0dc6131] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:12:50.978948 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7c613104-1f2b-4748-9eca-9034e7f5d046 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:12:50.979447 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7c613104-1f2b-4748-9eca-9034e7f5d046 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:12:50.980216 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7c613104-1f2b-4748-9eca-9034e7f5d046 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:12:50.980664 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7c613104-1f2b-4748-9eca-9034e7f5d046 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:12:51.008961 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-7c613104-1f2b-4748-9eca-9034e7f5d046 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] 10.222.0.16 "GET /compute/v2.1/servers/d9281e80-5a8e-4a1e-9e1f-4de3b0dc6131/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.043038 Jul 16 22:12:51.009521 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3419/6829] 10.222.0.16 () {58 vars in 1301 bytes} [Sat Jul 16 22:12:50 2022] GET /compute/v2.1/servers/d9281e80-5a8e-4a1e-9e1f-4de3b0dc6131/os-volume_attachments => generated 25 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Jul 16 22:12:51.020932 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-59dc7405-187e-4e25-811d-684114c13463 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:12:51.021558 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-59dc7405-187e-4e25-811d-684114c13463 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] [instance: d9281e80-5a8e-4a1e-9e1f-4de3b0dc6131] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:12:51.028580 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-59dc7405-187e-4e25-811d-684114c13463 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:12:51.029042 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-59dc7405-187e-4e25-811d-684114c13463 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:12:51.029816 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-59dc7405-187e-4e25-811d-684114c13463 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:12:51.030263 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-59dc7405-187e-4e25-811d-684114c13463 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:12:51.105373 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-59dc7405-187e-4e25-811d-684114c13463 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] HTTP exception thrown: Instance d9281e80-5a8e-4a1e-9e1f-4de3b0dc6131 is not attached to volume 3a5baf74-a675-4ae1-962b-69743c9186ec Jul 16 22:12:51.110852 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-59dc7405-187e-4e25-811d-684114c13463 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Request method failure captured: Jul 16 22:12:51.110852 ncdv-master-1 devstack@n-api.service[138680]: request: DELETE /compute/v2.1/servers/d9281e80-5a8e-4a1e-9e1f-4de3b0dc6131/os-volume_attachments/3a5baf74-a675-4ae1-962b-69743c9186ec HTTP/1.1 Jul 16 22:12:51.110852 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 22:12:51.110852 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 22:12:51.110852 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 22:12:51.110852 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 0 Jul 16 22:12:51.110852 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 22:12:51.110852 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 22:12:51.110852 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 22:12:51.110852 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 22:12:51.110852 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0zbe3rCsn-3QaicbUyd_l2yeeYdFw5j3pI0YNVDnyDW1adHPbvFyygFfhKMkLZKzSd5wCIHBh53_Ahvryl0lX_fYnOhaAfRD_aC5xNe-jnygMGiPOaIiG3WVsRGIF7WYX8xbkLiWHdzOeuosVkQ9hfVKA2k7v4UqqZHZ5ysM0snBRFk Jul 16 22:12:51.110852 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 22:12:51.110852 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 22:12:51.110852 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 22:12:51.110852 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 22:12:51.110852 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 22:12:51.110852 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 22:12:51.110852 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: deb1e913bedd4fb5b286e21c60e27a41 Jul 16 22:12:51.110852 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-TestMinimumBasicScenario-1144486311 Jul 16 22:12:51.110852 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 22:12:51.110852 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 22:12:51.110852 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/deb1e913bedd4fb5b286e21c60e27a41"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/deb1e913bedd4fb5b286e21c60e27a41"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_deb1e913bedd4fb5b286e21c60e27a41"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/deb1e913bedd4fb5b286e21c60e27a41"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 22:12:51.110852 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-TestMinimumBasicScenario-1144486311 Jul 16 22:12:51.110852 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: deb1e913bedd4fb5b286e21c60e27a41 Jul 16 22:12:51.110852 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-TestMinimumBasicScenario-1144486311 Jul 16 22:12:51.110852 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-TestMinimumBasicScenario-1144486311-project Jul 16 22:12:51.110852 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 22:12:51.110852 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 22:12:51.110852 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: d749cd77cf5c489e8c4e57e02ce768fd Jul 16 22:12:51.110852 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-TestMinimumBasicScenario-1144486311-project Jul 16 22:12:51.110852 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 22:12:51.110852 ncdv-master-1 devstack@n-api.service[138680]: exception: Instance d9281e80-5a8e-4a1e-9e1f-4de3b0dc6131 is not attached to volume 3a5baf74-a675-4ae1-962b-69743c9186ec Jul 16 22:12:51.110852 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'server_id': 'd9281e80-5a8e-4a1e-9e1f-4de3b0dc6131', 'id': '3a5baf74-a675-4ae1-962b-69743c9186ec'} Jul 16 22:12:51.110852 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 22:12:51.110852 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 22:12:51.110852 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 552, in delete Jul 16 22:12:51.110852 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi bdm = objects.BlockDeviceMapping.get_by_volume_and_instance( Jul 16 22:12:51.110852 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Jul 16 22:12:51.110852 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Jul 16 22:12:51.110852 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/block_device.py", line 281, in get_by_volume_and_instance Jul 16 22:12:51.110852 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.VolumeBDMNotFound(volume_id=volume_id) Jul 16 22:12:51.110852 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.VolumeBDMNotFound: No volume Block Device Mapping with id 3a5baf74-a675-4ae1-962b-69743c9186ec. Jul 16 22:12:51.110852 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 22:12:51.110852 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 22:12:51.110852 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 22:12:51.110852 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 22:12:51.110852 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 22:12:51.110852 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 22:12:51.110852 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 22:12:51.110852 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 22:12:51.110852 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 22:12:51.110852 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 22:12:51.110852 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 558, in delete Jul 16 22:12:51.110852 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=msg) Jul 16 22:12:51.110852 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance d9281e80-5a8e-4a1e-9e1f-4de3b0dc6131 is not attached to volume 3a5baf74-a675-4ae1-962b-69743c9186ec Jul 16 22:12:51.110852 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 22:12:51.110852 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 22:12:51.110852 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 22:12:51.110852 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 22:12:51.110852 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 22:12:51.110852 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 22:12:51.115697 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 22:12:51.115697 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 22:12:51.115697 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance d9281e80-5a8e-4a1e-9e1f-4de3b0dc6131 is not attached to volume 3a5baf74-a675-4ae1-962b-69743c9186ec Jul 16 22:12:51.115697 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 22:12:51.115697 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-59dc7405-187e-4e25-811d-684114c13463 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Returning 404 to user: Instance d9281e80-5a8e-4a1e-9e1f-4de3b0dc6131 is not attached to volume 3a5baf74-a675-4ae1-962b-69743c9186ec {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 22:12:51.115697 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-59dc7405-187e-4e25-811d-684114c13463 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] 10.222.0.16 "DELETE /compute/v2.1/servers/d9281e80-5a8e-4a1e-9e1f-4de3b0dc6131/os-volume_attachments/3a5baf74-a675-4ae1-962b-69743c9186ec" status: 404 len: 154 microversion: 2.1 time: 0.096727 Jul 16 22:12:51.115697 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3411/6830] 10.222.0.16 () {58 vars in 1415 bytes} [Sat Jul 16 22:12:51 2022] DELETE /compute/v2.1/servers/d9281e80-5a8e-4a1e-9e1f-4de3b0dc6131/os-volume_attachments/3a5baf74-a675-4ae1-962b-69743c9186ec => generated 154 bytes in 98 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 22:12:54.452215 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-9c36dfc6-45e9-45b6-9ce1-4d140fefc7ee tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:12:54.452731 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-9c36dfc6-45e9-45b6-9ce1-4d140fefc7ee tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] [instance: d9281e80-5a8e-4a1e-9e1f-4de3b0dc6131] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:12:54.459809 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9c36dfc6-45e9-45b6-9ce1-4d140fefc7ee tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:12:54.460230 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9c36dfc6-45e9-45b6-9ce1-4d140fefc7ee tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:12:54.461012 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9c36dfc6-45e9-45b6-9ce1-4d140fefc7ee tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:12:54.461464 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9c36dfc6-45e9-45b6-9ce1-4d140fefc7ee tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:12:54.486331 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-9c36dfc6-45e9-45b6-9ce1-4d140fefc7ee tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] [instance: d9281e80-5a8e-4a1e-9e1f-4de3b0dc6131] Going to try to terminate instance {{(pid=138681) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 16 22:12:54.724960 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-9c36dfc6-45e9-45b6-9ce1-4d140fefc7ee tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] 10.222.0.16 "DELETE /compute/v2.1/servers/d9281e80-5a8e-4a1e-9e1f-4de3b0dc6131" status: 204 len: 0 microversion: 2.1 time: 0.276299 Jul 16 22:12:54.726041 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3420/6831] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 22:12:54 2022] DELETE /compute/v2.1/servers/d9281e80-5a8e-4a1e-9e1f-4de3b0dc6131 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 16 22:12:54.734475 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-91dc12ec-44af-4ca7-832b-4490d7931d65 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:12:54.735123 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-91dc12ec-44af-4ca7-832b-4490d7931d65 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] [instance: d9281e80-5a8e-4a1e-9e1f-4de3b0dc6131] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:12:54.746246 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-91dc12ec-44af-4ca7-832b-4490d7931d65 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:12:54.746707 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-91dc12ec-44af-4ca7-832b-4490d7931d65 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:12:54.747522 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-91dc12ec-44af-4ca7-832b-4490d7931d65 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:12:54.748017 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-91dc12ec-44af-4ca7-832b-4490d7931d65 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:12:54.884553 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-91dc12ec-44af-4ca7-832b-4490d7931d65 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd749cd77cf5c489e8c4e57e02ce768fd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'deb1e913bedd4fb5b286e21c60e27a41', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:12:54.892197 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-91dc12ec-44af-4ca7-832b-4490d7931d65 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] 10.222.0.16 "GET /compute/v2.1/servers/d9281e80-5a8e-4a1e-9e1f-4de3b0dc6131" status: 200 len: 1556 microversion: 2.1 time: 0.161728 Jul 16 22:12:54.892761 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3412/6832] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:12:54 2022] GET /compute/v2.1/servers/d9281e80-5a8e-4a1e-9e1f-4de3b0dc6131 => generated 1556 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:12:55.910409 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-064e4d89-5741-4692-bc8f-4337c61baad6 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:12:55.911168 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-064e4d89-5741-4692-bc8f-4337c61baad6 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] [instance: d9281e80-5a8e-4a1e-9e1f-4de3b0dc6131] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:12:55.918970 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-064e4d89-5741-4692-bc8f-4337c61baad6 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:12:55.919749 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-064e4d89-5741-4692-bc8f-4337c61baad6 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:12:55.920266 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-064e4d89-5741-4692-bc8f-4337c61baad6 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:12:55.920725 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-064e4d89-5741-4692-bc8f-4337c61baad6 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:12:56.053071 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-064e4d89-5741-4692-bc8f-4337c61baad6 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd749cd77cf5c489e8c4e57e02ce768fd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'deb1e913bedd4fb5b286e21c60e27a41', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:12:56.061269 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-064e4d89-5741-4692-bc8f-4337c61baad6 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] 10.222.0.16 "GET /compute/v2.1/servers/d9281e80-5a8e-4a1e-9e1f-4de3b0dc6131" status: 200 len: 1556 microversion: 2.1 time: 0.155246 Jul 16 22:12:56.061834 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3421/6833] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:12:55 2022] GET /compute/v2.1/servers/d9281e80-5a8e-4a1e-9e1f-4de3b0dc6131 => generated 1556 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:12:56.818411 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-adb3a5c7-3894-45c2-b05e-6a9871739282 req-141f6fd5-108f-40fc-8aaf-a08723f41746 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "d9281e80-5a8e-4a1e-9e1f-4de3b0dc6131", "name": "network-vif-deleted", "tag": "7d4f58e2-1e5d-4e39-8582-ff5c8c553cc1"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:12:56.827256 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-adb3a5c7-3894-45c2-b05e-6a9871739282 req-141f6fd5-108f-40fc-8aaf-a08723f41746 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:12:56.827794 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-adb3a5c7-3894-45c2-b05e-6a9871739282 req-141f6fd5-108f-40fc-8aaf-a08723f41746 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:12:56.839854 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-adb3a5c7-3894-45c2-b05e-6a9871739282 req-141f6fd5-108f-40fc-8aaf-a08723f41746 service nova] Creating event network-vif-deleted:7d4f58e2-1e5d-4e39-8582-ff5c8c553cc1 for instance d9281e80-5a8e-4a1e-9e1f-4de3b0dc6131 on nch1-master-1 Jul 16 22:12:56.844784 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-adb3a5c7-3894-45c2-b05e-6a9871739282 req-141f6fd5-108f-40fc-8aaf-a08723f41746 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.089340 Jul 16 22:12:56.845466 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3413/6834] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 22:12:56 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 22:12:57.080755 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-9334d0be-7881-45a2-93c9-9932dd20bf26 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:12:57.081821 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-9334d0be-7881-45a2-93c9-9932dd20bf26 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] [instance: d9281e80-5a8e-4a1e-9e1f-4de3b0dc6131] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:12:57.089901 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9334d0be-7881-45a2-93c9-9932dd20bf26 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:12:57.090363 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9334d0be-7881-45a2-93c9-9932dd20bf26 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:12:57.091199 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9334d0be-7881-45a2-93c9-9932dd20bf26 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:12:57.091655 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9334d0be-7881-45a2-93c9-9932dd20bf26 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:12:57.125419 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9334d0be-7881-45a2-93c9-9932dd20bf26 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:12:57.125894 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9334d0be-7881-45a2-93c9-9932dd20bf26 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:12:57.126376 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.objects.instance [None req-9334d0be-7881-45a2-93c9-9932dd20bf26 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lazy-loading 'fault' on Instance uuid d9281e80-5a8e-4a1e-9e1f-4de3b0dc6131 {{(pid=138681) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 22:12:57.170224 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-9334d0be-7881-45a2-93c9-9932dd20bf26 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd749cd77cf5c489e8c4e57e02ce768fd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'deb1e913bedd4fb5b286e21c60e27a41', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:12:57.178342 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-9334d0be-7881-45a2-93c9-9932dd20bf26 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] 10.222.0.16 "GET /compute/v2.1/servers/d9281e80-5a8e-4a1e-9e1f-4de3b0dc6131" status: 200 len: 1349 microversion: 2.1 time: 0.102233 Jul 16 22:12:57.179043 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3422/6835] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:12:57 2022] GET /compute/v2.1/servers/d9281e80-5a8e-4a1e-9e1f-4de3b0dc6131 => generated 1349 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:12:58.197611 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-ca3af9e7-c481-4c8b-8195-cef8b7008ec8 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:12:58.198288 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-ca3af9e7-c481-4c8b-8195-cef8b7008ec8 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] [instance: d9281e80-5a8e-4a1e-9e1f-4de3b0dc6131] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:12:58.204649 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ca3af9e7-c481-4c8b-8195-cef8b7008ec8 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:12:58.205109 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ca3af9e7-c481-4c8b-8195-cef8b7008ec8 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:12:58.205881 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ca3af9e7-c481-4c8b-8195-cef8b7008ec8 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:12:58.206329 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ca3af9e7-c481-4c8b-8195-cef8b7008ec8 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:12:58.220303 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-ca3af9e7-c481-4c8b-8195-cef8b7008ec8 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] HTTP exception thrown: Instance d9281e80-5a8e-4a1e-9e1f-4de3b0dc6131 could not be found. Jul 16 22:12:58.221924 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-ca3af9e7-c481-4c8b-8195-cef8b7008ec8 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Request method failure captured: Jul 16 22:12:58.221924 ncdv-master-1 devstack@n-api.service[138680]: request: GET /compute/v2.1/servers/d9281e80-5a8e-4a1e-9e1f-4de3b0dc6131 HTTP/1.1 Jul 16 22:12:58.221924 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 22:12:58.221924 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 22:12:58.221924 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 22:12:58.221924 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 0 Jul 16 22:12:58.221924 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 22:12:58.221924 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 22:12:58.221924 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 22:12:58.221924 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 22:12:58.221924 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0zbe3rCsn-3QaicbUyd_l2yeeYdFw5j3pI0YNVDnyDW1adHPbvFyygFfhKMkLZKzSd5wCIHBh53_Ahvryl0lX_fYnOhaAfRD_aC5xNe-jnygMGiPOaIiG3WVsRGIF7WYX8xbkLiWHdzOeuosVkQ9hfVKA2k7v4UqqZHZ5ysM0snBRFk Jul 16 22:12:58.221924 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 22:12:58.221924 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 22:12:58.221924 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 22:12:58.221924 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 22:12:58.221924 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 22:12:58.221924 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 22:12:58.221924 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: deb1e913bedd4fb5b286e21c60e27a41 Jul 16 22:12:58.221924 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-TestMinimumBasicScenario-1144486311 Jul 16 22:12:58.221924 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 22:12:58.221924 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 22:12:58.221924 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/deb1e913bedd4fb5b286e21c60e27a41"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/deb1e913bedd4fb5b286e21c60e27a41"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_deb1e913bedd4fb5b286e21c60e27a41"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/deb1e913bedd4fb5b286e21c60e27a41"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 22:12:58.221924 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-TestMinimumBasicScenario-1144486311 Jul 16 22:12:58.221924 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: deb1e913bedd4fb5b286e21c60e27a41 Jul 16 22:12:58.221924 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-TestMinimumBasicScenario-1144486311 Jul 16 22:12:58.221924 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-TestMinimumBasicScenario-1144486311-project Jul 16 22:12:58.221924 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 22:12:58.221924 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 22:12:58.221924 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: d749cd77cf5c489e8c4e57e02ce768fd Jul 16 22:12:58.221924 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-TestMinimumBasicScenario-1144486311-project Jul 16 22:12:58.221924 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 22:12:58.221924 ncdv-master-1 devstack@n-api.service[138680]: exception: Instance d9281e80-5a8e-4a1e-9e1f-4de3b0dc6131 could not be found. Jul 16 22:12:58.221924 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': 'd9281e80-5a8e-4a1e-9e1f-4de3b0dc6131'} Jul 16 22:12:58.221924 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 22:12:58.221924 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 22:12:58.221924 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 22:12:58.221924 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 22:12:58.221924 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 22:12:58.221924 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 22:12:58.221924 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 22:12:58.221924 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 22:12:58.221924 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 22:12:58.221924 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 22:12:58.221924 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance d9281e80-5a8e-4a1e-9e1f-4de3b0dc6131 could not be found. Jul 16 22:12:58.221924 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 22:12:58.221924 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 22:12:58.221924 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 22:12:58.221924 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 22:12:58.221924 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 22:12:58.221924 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 22:12:58.221924 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 22:12:58.221924 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 22:12:58.221924 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 22:12:58.221924 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 22:12:58.221924 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 22:12:58.221924 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 22:12:58.221924 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 22:12:58.221924 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 22:12:58.221924 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 22:12:58.221924 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 22:12:58.221924 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance d9281e80-5a8e-4a1e-9e1f-4de3b0dc6131 could not be found. Jul 16 22:12:58.221924 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 22:12:58.221924 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 22:12:58.221924 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 22:12:58.227947 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 22:12:58.227947 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 22:12:58.227947 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 22:12:58.227947 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 22:12:58.227947 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 22:12:58.227947 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance d9281e80-5a8e-4a1e-9e1f-4de3b0dc6131 could not be found. Jul 16 22:12:58.227947 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 22:12:58.227947 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-ca3af9e7-c481-4c8b-8195-cef8b7008ec8 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Returning 404 to user: Instance d9281e80-5a8e-4a1e-9e1f-4de3b0dc6131 could not be found. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 22:12:58.227947 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-ca3af9e7-c481-4c8b-8195-cef8b7008ec8 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] 10.222.0.16 "GET /compute/v2.1/servers/d9281e80-5a8e-4a1e-9e1f-4de3b0dc6131" status: 404 len: 111 microversion: 2.1 time: 0.031446 Jul 16 22:12:58.227947 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3414/6836] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:12:58 2022] GET /compute/v2.1/servers/d9281e80-5a8e-4a1e-9e1f-4de3b0dc6131 => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 22:12:58.233066 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-0bd7e635-50f2-47af-b129-f98994828a1f tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Calling method '.version_select at 0x7fdd9d069040>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:12:58.236281 ncdv-master-1 devstack@n-api.service[138681]: ERROR oslo.messaging._drivers.impl_rabbit [None req-0bd7e635-50f2-47af-b129-f98994828a1f tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] [0cc6ec90-aeaa-4138-9056-1296518c79df] AMQP server on 10.222.0.16:5672 is unreachable: [Errno 104] Connection reset by peer. Trying again in 1 seconds.: ConnectionResetError: [Errno 104] Connection reset by peer Jul 16 22:12:59.249442 ncdv-master-1 devstack@n-api.service[138681]: INFO oslo.messaging._drivers.impl_rabbit [None req-0bd7e635-50f2-47af-b129-f98994828a1f tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] [0cc6ec90-aeaa-4138-9056-1296518c79df] Reconnected to AMQP server on 10.222.0.16:5672 via [amqp] client with port 60270. Jul 16 22:12:59.320535 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-0bd7e635-50f2-47af-b129-f98994828a1f tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] 10.222.0.16 "DELETE /compute/v2.1/os-keypairs/tempest-TestMinimumBasicScenario-1668340565" status: 202 len: 0 microversion: 2.1 time: 1.091460 Jul 16 22:12:59.321148 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3423/6837] 10.222.0.16 () {58 vars in 1271 bytes} [Sat Jul 16 22:12:58 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestMinimumBasicScenario-1668340565 => generated 0 bytes in 1093 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 22:13:04.989970 ncdv-master-1 devstack@n-api.service[138680]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jul 16 22:13:04.991223 ncdv-master-1 devstack@n-api.service[138680]: Sat Jul 16 22:13:04 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Jul 16 22:13:10.504993 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-1b350ba1-e08a-488b-8d8a-5672d607b890 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Action: 'create', calling method: .version_select at 0x7fdd8fbcf8b0>, body: {"keypair": {"name": "tempest-TestMinimumBasicScenario-772695359", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBLoEVRrcMPnx+0R78ZJVwjmLrbR8mCW4jSjQBw0g7QxG01B/qBwXfslBMl/7j/1oDrn7gHChPPPw6mBIAerw95fDCwpoTuqH/HpArFxFagxWl1ep3PTN6sUV5NQuvxTeHQ=="}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:13:10.510682 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_api:SELECT=14 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:13:10.511996 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=31,nova_cell1:INSERT=1 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:13:10.523288 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-1b350ba1-e08a-488b-8d8a-5672d607b890 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Getting quotas for project deb1e913bedd4fb5b286e21c60e27a41. Resources: {'key_pairs'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 22:13:10.528627 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-1b350ba1-e08a-488b-8d8a-5672d607b890 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Getting quotas for user d749cd77cf5c489e8c4e57e02ce768fd and project deb1e913bedd4fb5b286e21c60e27a41. Resources: {'key_pairs'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 22:13:10.628877 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-1b350ba1-e08a-488b-8d8a-5672d607b890 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] 10.222.0.16 "POST /compute/v2.1/os-keypairs" status: 200 len: 402 microversion: 2.1 time: 0.127854 Jul 16 22:13:10.629488 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3415/6838] 10.222.0.16 () {60 vars in 1158 bytes} [Sat Jul 16 22:13:10 2022] POST /compute/v2.1/os-keypairs => generated 402 bytes in 129 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 22:13:10.675857 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=42,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:13:10.676837 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_api:SELECT=22,nova_api:UPDATE=1,nova_api:DELETE=1 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:13:10.681676 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-d4327bef-6370-4481-9344-5d586d9f9ac4 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestMinimumBasicScenario-server-4780287", "imageRef": "a6ff19cf-4441-40c7-b562-e3d97c668a50", "flavorRef": "452", "key_name": "tempest-TestMinimumBasicScenario-772695359", "networks": [{"uuid": "bf8f9392-1062-4eb2-b42d-afeb45f5873f"}]}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:13:10.733127 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.network.neutron [None req-d4327bef-6370-4481-9344-5d586d9f9ac4 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] validate_networks() for [('bf8f9392-1062-4eb2-b42d-afeb45f5873f', None, None, None, None, None)] {{(pid=138681) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 16 22:13:11.109098 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-d4327bef-6370-4481-9344-5d586d9f9ac4 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Flavor limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 16 22:13:11.109551 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-d4327bef-6370-4481-9344-5d586d9f9ac4 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Image limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 16 22:13:11.110048 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-d4327bef-6370-4481-9344-5d586d9f9ac4 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Flavor pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 16 22:13:11.110513 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-d4327bef-6370-4481-9344-5d586d9f9ac4 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Image pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 16 22:13:11.110946 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-d4327bef-6370-4481-9344-5d586d9f9ac4 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 16 22:13:11.120871 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d4327bef-6370-4481-9344-5d586d9f9ac4 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:13:11.121348 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d4327bef-6370-4481-9344-5d586d9f9ac4 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:13:11.122170 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d4327bef-6370-4481-9344-5d586d9f9ac4 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:13:11.122626 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d4327bef-6370-4481-9344-5d586d9f9ac4 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:13:11.140828 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-d4327bef-6370-4481-9344-5d586d9f9ac4 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Getting quotas for project deb1e913bedd4fb5b286e21c60e27a41. Resources: {'ram', 'instances', 'cores'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 22:13:11.146333 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-d4327bef-6370-4481-9344-5d586d9f9ac4 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Getting quotas for user d749cd77cf5c489e8c4e57e02ce768fd and project deb1e913bedd4fb5b286e21c60e27a41. Resources: {'ram', 'instances', 'cores'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 22:13:11.159238 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-d4327bef-6370-4481-9344-5d586d9f9ac4 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Going to run 1 instances... {{(pid=138681) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 16 22:13:11.161462 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-d4327bef-6370-4481-9344-5d586d9f9ac4 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] [instance: 41010ce2-2848-458a-b6da-5d2ac455bcc0] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='a6ff19cf-4441-40c7-b562-e3d97c668a50',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138681) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 16 22:13:11.264932 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-d4327bef-6370-4481-9344-5d586d9f9ac4 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.590779 Jul 16 22:13:11.265638 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3424/6839] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 22:13:10 2022] POST /compute/v2.1/servers => generated 380 bytes in 592 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 16 22:13:11.276358 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-857ccc9e-bc06-4105-966d-0b7201ad4389 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:13:11.277641 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-857ccc9e-bc06-4105-966d-0b7201ad4389 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] [instance: 41010ce2-2848-458a-b6da-5d2ac455bcc0] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:13:11.336450 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-857ccc9e-bc06-4105-966d-0b7201ad4389 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd749cd77cf5c489e8c4e57e02ce768fd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'deb1e913bedd4fb5b286e21c60e27a41', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:13:11.343371 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-857ccc9e-bc06-4105-966d-0b7201ad4389 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] 10.222.0.16 "GET /compute/v2.1/servers/41010ce2-2848-458a-b6da-5d2ac455bcc0" status: 200 len: 1255 microversion: 2.1 time: 0.071527 Jul 16 22:13:11.343942 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3416/6840] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:13:11 2022] GET /compute/v2.1/servers/41010ce2-2848-458a-b6da-5d2ac455bcc0 => generated 1255 bytes in 73 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 22:13:12.360743 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-d6037234-2d6b-4d30-a2d2-cb71719e8a55 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:13:12.361383 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-d6037234-2d6b-4d30-a2d2-cb71719e8a55 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] [instance: 41010ce2-2848-458a-b6da-5d2ac455bcc0] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:13:12.369043 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d6037234-2d6b-4d30-a2d2-cb71719e8a55 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:13:12.369522 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d6037234-2d6b-4d30-a2d2-cb71719e8a55 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:13:12.370332 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d6037234-2d6b-4d30-a2d2-cb71719e8a55 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:13:12.370795 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d6037234-2d6b-4d30-a2d2-cb71719e8a55 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:13:12.438245 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-d6037234-2d6b-4d30-a2d2-cb71719e8a55 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd749cd77cf5c489e8c4e57e02ce768fd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'deb1e913bedd4fb5b286e21c60e27a41', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:13:12.447160 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-d6037234-2d6b-4d30-a2d2-cb71719e8a55 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] 10.222.0.16 "GET /compute/v2.1/servers/41010ce2-2848-458a-b6da-5d2ac455bcc0" status: 200 len: 1259 microversion: 2.1 time: 0.090328 Jul 16 22:13:12.447796 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3425/6841] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:13:12 2022] GET /compute/v2.1/servers/41010ce2-2848-458a-b6da-5d2ac455bcc0 => generated 1259 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:13:13.469546 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-8e74e53f-3cb5-4c7a-879e-dcc9f3058495 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:13:13.469546 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-8e74e53f-3cb5-4c7a-879e-dcc9f3058495 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] [instance: 41010ce2-2848-458a-b6da-5d2ac455bcc0] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:13:13.473224 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8e74e53f-3cb5-4c7a-879e-dcc9f3058495 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:13:13.473688 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8e74e53f-3cb5-4c7a-879e-dcc9f3058495 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:13:13.474472 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8e74e53f-3cb5-4c7a-879e-dcc9f3058495 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:13:13.474918 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8e74e53f-3cb5-4c7a-879e-dcc9f3058495 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:13:13.544568 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-8e74e53f-3cb5-4c7a-879e-dcc9f3058495 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd749cd77cf5c489e8c4e57e02ce768fd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'deb1e913bedd4fb5b286e21c60e27a41', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:13:13.552748 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-8e74e53f-3cb5-4c7a-879e-dcc9f3058495 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] 10.222.0.16 "GET /compute/v2.1/servers/41010ce2-2848-458a-b6da-5d2ac455bcc0" status: 200 len: 1313 microversion: 2.1 time: 0.092397 Jul 16 22:13:13.553325 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3417/6842] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:13:13 2022] GET /compute/v2.1/servers/41010ce2-2848-458a-b6da-5d2ac455bcc0 => generated 1313 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:13:14.571657 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-a9b751f7-6d90-45cb-a802-4349071832ac tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:13:14.572320 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-a9b751f7-6d90-45cb-a802-4349071832ac tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] [instance: 41010ce2-2848-458a-b6da-5d2ac455bcc0] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:13:14.580800 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a9b751f7-6d90-45cb-a802-4349071832ac tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:13:14.581680 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a9b751f7-6d90-45cb-a802-4349071832ac tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:13:14.582873 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a9b751f7-6d90-45cb-a802-4349071832ac tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:13:14.583358 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a9b751f7-6d90-45cb-a802-4349071832ac tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:13:14.729419 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-a9b751f7-6d90-45cb-a802-4349071832ac tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd749cd77cf5c489e8c4e57e02ce768fd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'deb1e913bedd4fb5b286e21c60e27a41', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:13:14.739025 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-a9b751f7-6d90-45cb-a802-4349071832ac tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] 10.222.0.16 "GET /compute/v2.1/servers/41010ce2-2848-458a-b6da-5d2ac455bcc0" status: 200 len: 1355 microversion: 2.1 time: 0.172548 Jul 16 22:13:14.740494 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3426/6843] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:13:14 2022] GET /compute/v2.1/servers/41010ce2-2848-458a-b6da-5d2ac455bcc0 => generated 1355 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:13:15.756555 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-d5788e5a-9577-4085-bb69-707a1986cf25 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:13:15.757226 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-d5788e5a-9577-4085-bb69-707a1986cf25 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] [instance: 41010ce2-2848-458a-b6da-5d2ac455bcc0] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:13:15.764838 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d5788e5a-9577-4085-bb69-707a1986cf25 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:13:15.765303 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d5788e5a-9577-4085-bb69-707a1986cf25 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:13:15.766085 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d5788e5a-9577-4085-bb69-707a1986cf25 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:13:15.766534 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d5788e5a-9577-4085-bb69-707a1986cf25 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:13:15.905045 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-d5788e5a-9577-4085-bb69-707a1986cf25 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd749cd77cf5c489e8c4e57e02ce768fd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'deb1e913bedd4fb5b286e21c60e27a41', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:13:15.913366 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-d5788e5a-9577-4085-bb69-707a1986cf25 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] 10.222.0.16 "GET /compute/v2.1/servers/41010ce2-2848-458a-b6da-5d2ac455bcc0" status: 200 len: 1355 microversion: 2.1 time: 0.160588 Jul 16 22:13:15.925074 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3418/6844] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:13:15 2022] GET /compute/v2.1/servers/41010ce2-2848-458a-b6da-5d2ac455bcc0 => generated 1355 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:13:16.933026 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-e03648e1-6468-4d8d-b804-91071e86ddc7 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:13:16.933752 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-e03648e1-6468-4d8d-b804-91071e86ddc7 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] [instance: 41010ce2-2848-458a-b6da-5d2ac455bcc0] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:13:16.945489 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e03648e1-6468-4d8d-b804-91071e86ddc7 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:13:16.945956 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e03648e1-6468-4d8d-b804-91071e86ddc7 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:13:16.951340 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e03648e1-6468-4d8d-b804-91071e86ddc7 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:13:16.952083 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e03648e1-6468-4d8d-b804-91071e86ddc7 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:13:17.179808 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-e03648e1-6468-4d8d-b804-91071e86ddc7 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd749cd77cf5c489e8c4e57e02ce768fd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'deb1e913bedd4fb5b286e21c60e27a41', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:13:17.190423 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-e03648e1-6468-4d8d-b804-91071e86ddc7 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] 10.222.0.16 "GET /compute/v2.1/servers/41010ce2-2848-458a-b6da-5d2ac455bcc0" status: 200 len: 1355 microversion: 2.1 time: 0.261946 Jul 16 22:13:17.191229 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3427/6845] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:13:16 2022] GET /compute/v2.1/servers/41010ce2-2848-458a-b6da-5d2ac455bcc0 => generated 1355 bytes in 263 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 22:13:17.250145 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-b6fafbc7-cdf7-49ae-8187-75486edb4896 req-cf4e7035-14f9-46d7-812e-77d254b6019f service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "41010ce2-2848-458a-b6da-5d2ac455bcc0", "tag": "fff60fc8-0659-4ea8-bf1d-6bec19f459a0"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:13:17.265132 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-b6fafbc7-cdf7-49ae-8187-75486edb4896 req-cf4e7035-14f9-46d7-812e-77d254b6019f service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:13:17.265884 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-b6fafbc7-cdf7-49ae-8187-75486edb4896 req-cf4e7035-14f9-46d7-812e-77d254b6019f service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:13:17.310503 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-b6fafbc7-cdf7-49ae-8187-75486edb4896 req-cf4e7035-14f9-46d7-812e-77d254b6019f service nova] Creating event network-changed:fff60fc8-0659-4ea8-bf1d-6bec19f459a0 for instance 41010ce2-2848-458a-b6da-5d2ac455bcc0 on nch1-master-1 Jul 16 22:13:17.316453 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-b6fafbc7-cdf7-49ae-8187-75486edb4896 req-cf4e7035-14f9-46d7-812e-77d254b6019f service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.069812 Jul 16 22:13:17.320451 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3419/6846] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 22:13:17 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 75 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 22:13:18.210269 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-bcc61331-36cd-41d5-868b-c7f12f8dbf68 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:13:18.210883 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-bcc61331-36cd-41d5-868b-c7f12f8dbf68 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] [instance: 41010ce2-2848-458a-b6da-5d2ac455bcc0] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:13:18.217885 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-bcc61331-36cd-41d5-868b-c7f12f8dbf68 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:13:18.218373 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-bcc61331-36cd-41d5-868b-c7f12f8dbf68 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:13:18.219197 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-bcc61331-36cd-41d5-868b-c7f12f8dbf68 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:13:18.219656 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-bcc61331-36cd-41d5-868b-c7f12f8dbf68 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:13:18.370028 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-bcc61331-36cd-41d5-868b-c7f12f8dbf68 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd749cd77cf5c489e8c4e57e02ce768fd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'deb1e913bedd4fb5b286e21c60e27a41', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:13:18.378556 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-bcc61331-36cd-41d5-868b-c7f12f8dbf68 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] 10.222.0.16 "GET /compute/v2.1/servers/41010ce2-2848-458a-b6da-5d2ac455bcc0" status: 200 len: 1355 microversion: 2.1 time: 0.173314 Jul 16 22:13:18.387258 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3428/6847] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:13:18 2022] GET /compute/v2.1/servers/41010ce2-2848-458a-b6da-5d2ac455bcc0 => generated 1355 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:13:19.395479 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-a4b57c0c-1f8f-49fd-a7c1-e683aba0ef93 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:13:19.396941 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-a4b57c0c-1f8f-49fd-a7c1-e683aba0ef93 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] [instance: 41010ce2-2848-458a-b6da-5d2ac455bcc0] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:13:19.405299 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a4b57c0c-1f8f-49fd-a7c1-e683aba0ef93 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:13:19.406085 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a4b57c0c-1f8f-49fd-a7c1-e683aba0ef93 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:13:19.407139 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a4b57c0c-1f8f-49fd-a7c1-e683aba0ef93 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:13:19.407787 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a4b57c0c-1f8f-49fd-a7c1-e683aba0ef93 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:13:19.560690 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-a4b57c0c-1f8f-49fd-a7c1-e683aba0ef93 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd749cd77cf5c489e8c4e57e02ce768fd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'deb1e913bedd4fb5b286e21c60e27a41', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:13:19.568637 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-a4b57c0c-1f8f-49fd-a7c1-e683aba0ef93 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] 10.222.0.16 "GET /compute/v2.1/servers/41010ce2-2848-458a-b6da-5d2ac455bcc0" status: 200 len: 1355 microversion: 2.1 time: 0.177415 Jul 16 22:13:19.569212 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3420/6848] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:13:19 2022] GET /compute/v2.1/servers/41010ce2-2848-458a-b6da-5d2ac455bcc0 => generated 1355 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:13:20.587786 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-8d384f05-e532-494f-97e9-77c042360b2e tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:13:20.588379 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-8d384f05-e532-494f-97e9-77c042360b2e tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] [instance: 41010ce2-2848-458a-b6da-5d2ac455bcc0] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:13:20.596018 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8d384f05-e532-494f-97e9-77c042360b2e tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:13:20.603798 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8d384f05-e532-494f-97e9-77c042360b2e tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:13:20.607296 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8d384f05-e532-494f-97e9-77c042360b2e tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:13:20.607296 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8d384f05-e532-494f-97e9-77c042360b2e tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:13:20.761620 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-8d384f05-e532-494f-97e9-77c042360b2e tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd749cd77cf5c489e8c4e57e02ce768fd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'deb1e913bedd4fb5b286e21c60e27a41', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:13:20.775677 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-8d384f05-e532-494f-97e9-77c042360b2e tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] 10.222.0.16 "GET /compute/v2.1/servers/41010ce2-2848-458a-b6da-5d2ac455bcc0" status: 200 len: 1355 microversion: 2.1 time: 0.193079 Jul 16 22:13:20.776485 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3429/6849] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:13:20 2022] GET /compute/v2.1/servers/41010ce2-2848-458a-b6da-5d2ac455bcc0 => generated 1355 bytes in 195 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:13:21.792083 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-8771706d-cb46-44ec-996b-12d15d3f52cb tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:13:21.792715 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-8771706d-cb46-44ec-996b-12d15d3f52cb tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] [instance: 41010ce2-2848-458a-b6da-5d2ac455bcc0] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:13:21.794868 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell0:SELECT=6 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:13:21.799804 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8771706d-cb46-44ec-996b-12d15d3f52cb tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:13:21.800265 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8771706d-cb46-44ec-996b-12d15d3f52cb tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:13:21.801043 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8771706d-cb46-44ec-996b-12d15d3f52cb tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:13:21.801489 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8771706d-cb46-44ec-996b-12d15d3f52cb tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:13:21.948015 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-8771706d-cb46-44ec-996b-12d15d3f52cb tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd749cd77cf5c489e8c4e57e02ce768fd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'deb1e913bedd4fb5b286e21c60e27a41', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:13:21.956062 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-8771706d-cb46-44ec-996b-12d15d3f52cb tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] 10.222.0.16 "GET /compute/v2.1/servers/41010ce2-2848-458a-b6da-5d2ac455bcc0" status: 200 len: 1355 microversion: 2.1 time: 0.168311 Jul 16 22:13:21.956665 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3421/6850] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:13:21 2022] GET /compute/v2.1/servers/41010ce2-2848-458a-b6da-5d2ac455bcc0 => generated 1355 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:13:22.973755 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-a9c06713-f547-4322-92b2-9747a6352b15 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:13:22.974381 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-a9c06713-f547-4322-92b2-9747a6352b15 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] [instance: 41010ce2-2848-458a-b6da-5d2ac455bcc0] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:13:22.976339 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell0:SELECT=2 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:13:22.977024 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell0:SELECT=6 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:13:22.983653 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a9c06713-f547-4322-92b2-9747a6352b15 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:13:22.986547 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a9c06713-f547-4322-92b2-9747a6352b15 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:13:22.986547 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a9c06713-f547-4322-92b2-9747a6352b15 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:13:22.986547 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a9c06713-f547-4322-92b2-9747a6352b15 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:13:23.120568 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-a9c06713-f547-4322-92b2-9747a6352b15 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd749cd77cf5c489e8c4e57e02ce768fd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'deb1e913bedd4fb5b286e21c60e27a41', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:13:23.128185 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-a9c06713-f547-4322-92b2-9747a6352b15 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] 10.222.0.16 "GET /compute/v2.1/servers/41010ce2-2848-458a-b6da-5d2ac455bcc0" status: 200 len: 1355 microversion: 2.1 time: 0.158386 Jul 16 22:13:23.130287 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3430/6851] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:13:22 2022] GET /compute/v2.1/servers/41010ce2-2848-458a-b6da-5d2ac455bcc0 => generated 1355 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:13:24.145698 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-42c812b9-35ba-4ae8-a252-fc026f19a573 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:13:24.146322 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-42c812b9-35ba-4ae8-a252-fc026f19a573 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] [instance: 41010ce2-2848-458a-b6da-5d2ac455bcc0] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:13:24.153890 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-42c812b9-35ba-4ae8-a252-fc026f19a573 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:13:24.154346 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-42c812b9-35ba-4ae8-a252-fc026f19a573 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:13:24.155150 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-42c812b9-35ba-4ae8-a252-fc026f19a573 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:13:24.155596 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-42c812b9-35ba-4ae8-a252-fc026f19a573 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:13:24.292611 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-42c812b9-35ba-4ae8-a252-fc026f19a573 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd749cd77cf5c489e8c4e57e02ce768fd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'deb1e913bedd4fb5b286e21c60e27a41', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:13:24.300421 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-42c812b9-35ba-4ae8-a252-fc026f19a573 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] 10.222.0.16 "GET /compute/v2.1/servers/41010ce2-2848-458a-b6da-5d2ac455bcc0" status: 200 len: 1355 microversion: 2.1 time: 0.158554 Jul 16 22:13:24.301001 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3422/6852] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:13:24 2022] GET /compute/v2.1/servers/41010ce2-2848-458a-b6da-5d2ac455bcc0 => generated 1355 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:13:25.318651 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-1bb0c911-20fa-4c31-a20b-d365912220ac tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:13:25.319307 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-1bb0c911-20fa-4c31-a20b-d365912220ac tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] [instance: 41010ce2-2848-458a-b6da-5d2ac455bcc0] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:13:25.327439 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1bb0c911-20fa-4c31-a20b-d365912220ac tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:13:25.327904 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1bb0c911-20fa-4c31-a20b-d365912220ac tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:13:25.328684 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1bb0c911-20fa-4c31-a20b-d365912220ac tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:13:25.329136 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1bb0c911-20fa-4c31-a20b-d365912220ac tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:13:25.461213 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-1bb0c911-20fa-4c31-a20b-d365912220ac tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd749cd77cf5c489e8c4e57e02ce768fd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'deb1e913bedd4fb5b286e21c60e27a41', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:13:25.468777 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-1bb0c911-20fa-4c31-a20b-d365912220ac tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] 10.222.0.16 "GET /compute/v2.1/servers/41010ce2-2848-458a-b6da-5d2ac455bcc0" status: 200 len: 1355 microversion: 2.1 time: 0.154427 Jul 16 22:13:25.469360 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3431/6853] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:13:25 2022] GET /compute/v2.1/servers/41010ce2-2848-458a-b6da-5d2ac455bcc0 => generated 1355 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:13:26.486294 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-07f45695-0084-41d5-8b77-a7245e3a739e tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:13:26.486857 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-07f45695-0084-41d5-8b77-a7245e3a739e tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] [instance: 41010ce2-2848-458a-b6da-5d2ac455bcc0] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:13:26.494565 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-07f45695-0084-41d5-8b77-a7245e3a739e tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:13:26.495041 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-07f45695-0084-41d5-8b77-a7245e3a739e tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:13:26.495807 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-07f45695-0084-41d5-8b77-a7245e3a739e tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:13:26.496248 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-07f45695-0084-41d5-8b77-a7245e3a739e tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:13:26.630720 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-07f45695-0084-41d5-8b77-a7245e3a739e tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd749cd77cf5c489e8c4e57e02ce768fd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'deb1e913bedd4fb5b286e21c60e27a41', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:13:26.639107 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-07f45695-0084-41d5-8b77-a7245e3a739e tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] 10.222.0.16 "GET /compute/v2.1/servers/41010ce2-2848-458a-b6da-5d2ac455bcc0" status: 200 len: 1355 microversion: 2.1 time: 0.156739 Jul 16 22:13:26.639684 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3423/6854] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:13:26 2022] GET /compute/v2.1/servers/41010ce2-2848-458a-b6da-5d2ac455bcc0 => generated 1355 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:13:27.172626 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-27cab44e-5a0e-4f72-834a-617057e6d08d req-f33cf256-01c1-4f49-85b8-c1f345a0a8c8 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "41010ce2-2848-458a-b6da-5d2ac455bcc0", "name": "network-vif-plugged", "status": "completed", "tag": "fff60fc8-0659-4ea8-bf1d-6bec19f459a0"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:13:27.186292 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-27cab44e-5a0e-4f72-834a-617057e6d08d req-f33cf256-01c1-4f49-85b8-c1f345a0a8c8 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:13:27.186292 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-27cab44e-5a0e-4f72-834a-617057e6d08d req-f33cf256-01c1-4f49-85b8-c1f345a0a8c8 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:13:27.193583 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-27cab44e-5a0e-4f72-834a-617057e6d08d req-f33cf256-01c1-4f49-85b8-c1f345a0a8c8 service nova] Creating event network-vif-plugged:fff60fc8-0659-4ea8-bf1d-6bec19f459a0 for instance 41010ce2-2848-458a-b6da-5d2ac455bcc0 on nch1-master-1 Jul 16 22:13:27.199178 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-27cab44e-5a0e-4f72-834a-617057e6d08d req-f33cf256-01c1-4f49-85b8-c1f345a0a8c8 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.088678 Jul 16 22:13:27.200438 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3432/6855] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 22:13:27 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 22:13:27.657096 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-fe50a6cf-80de-44da-a543-6af98c32b258 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:13:27.657640 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-fe50a6cf-80de-44da-a543-6af98c32b258 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] [instance: 41010ce2-2848-458a-b6da-5d2ac455bcc0] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:13:27.664660 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-fe50a6cf-80de-44da-a543-6af98c32b258 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:13:27.665123 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-fe50a6cf-80de-44da-a543-6af98c32b258 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:13:27.665925 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-fe50a6cf-80de-44da-a543-6af98c32b258 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:13:27.666456 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-fe50a6cf-80de-44da-a543-6af98c32b258 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:13:27.795730 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-fe50a6cf-80de-44da-a543-6af98c32b258 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd749cd77cf5c489e8c4e57e02ce768fd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'deb1e913bedd4fb5b286e21c60e27a41', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:13:27.803220 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-fe50a6cf-80de-44da-a543-6af98c32b258 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] 10.222.0.16 "GET /compute/v2.1/servers/41010ce2-2848-458a-b6da-5d2ac455bcc0" status: 200 len: 1355 microversion: 2.1 time: 0.150020 Jul 16 22:13:27.803835 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3424/6856] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:13:27 2022] GET /compute/v2.1/servers/41010ce2-2848-458a-b6da-5d2ac455bcc0 => generated 1355 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:13:28.821861 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-5b4b9656-b7c9-43ff-8a84-ac2462905242 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:13:28.823156 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-5b4b9656-b7c9-43ff-8a84-ac2462905242 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] [instance: 41010ce2-2848-458a-b6da-5d2ac455bcc0] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:13:28.829536 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5b4b9656-b7c9-43ff-8a84-ac2462905242 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:13:28.829997 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5b4b9656-b7c9-43ff-8a84-ac2462905242 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:13:28.830770 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5b4b9656-b7c9-43ff-8a84-ac2462905242 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:13:28.831257 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5b4b9656-b7c9-43ff-8a84-ac2462905242 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:13:28.964672 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-5b4b9656-b7c9-43ff-8a84-ac2462905242 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd749cd77cf5c489e8c4e57e02ce768fd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'deb1e913bedd4fb5b286e21c60e27a41', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:13:28.972353 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-5b4b9656-b7c9-43ff-8a84-ac2462905242 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] 10.222.0.16 "GET /compute/v2.1/servers/41010ce2-2848-458a-b6da-5d2ac455bcc0" status: 200 len: 1545 microversion: 2.1 time: 0.155945 Jul 16 22:13:28.972932 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3433/6857] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:13:28 2022] GET /compute/v2.1/servers/41010ce2-2848-458a-b6da-5d2ac455bcc0 => generated 1545 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:13:28.989058 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-e070f5bf-6b4e-4422-90c5-664487150408 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:13:28.989683 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-e070f5bf-6b4e-4422-90c5-664487150408 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] [instance: 41010ce2-2848-458a-b6da-5d2ac455bcc0] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:13:28.996898 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e070f5bf-6b4e-4422-90c5-664487150408 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:13:28.997358 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e070f5bf-6b4e-4422-90c5-664487150408 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:13:28.998179 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e070f5bf-6b4e-4422-90c5-664487150408 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:13:28.998627 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e070f5bf-6b4e-4422-90c5-664487150408 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:13:29.138345 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-e070f5bf-6b4e-4422-90c5-664487150408 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd749cd77cf5c489e8c4e57e02ce768fd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'deb1e913bedd4fb5b286e21c60e27a41', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:13:29.145888 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-e070f5bf-6b4e-4422-90c5-664487150408 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] 10.222.0.16 "GET /compute/v2.1/servers/41010ce2-2848-458a-b6da-5d2ac455bcc0" status: 200 len: 1545 microversion: 2.1 time: 0.160885 Jul 16 22:13:29.146454 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3425/6858] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:13:28 2022] GET /compute/v2.1/servers/41010ce2-2848-458a-b6da-5d2ac455bcc0 => generated 1545 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:13:29.162610 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-b38e5d1d-7044-429b-8386-dd82a9e9bad8 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:13:29.166446 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-b38e5d1d-7044-429b-8386-dd82a9e9bad8 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Policy check for os_compute_api:servers:allow_all_filters failed with credentials {'is_admin': False, 'user_id': 'd749cd77cf5c489e8c4e57e02ce768fd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'deb1e913bedd4fb5b286e21c60e27a41', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:13:29.167295 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-b38e5d1d-7044-429b-8386-dd82a9e9bad8 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Searching by: {'deleted': False, 'project_id': 'deb1e913bedd4fb5b286e21c60e27a41'} {{(pid=138681) get_all /opt/stack/nova/nova/compute/api.py:2962}} Jul 16 22:13:29.172719 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b38e5d1d-7044-429b-8386-dd82a9e9bad8 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:13:29.173204 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b38e5d1d-7044-429b-8386-dd82a9e9bad8 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:13:29.174007 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b38e5d1d-7044-429b-8386-dd82a9e9bad8 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:13:29.174457 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b38e5d1d-7044-429b-8386-dd82a9e9bad8 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:13:29.199728 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.multi_cell_list [None req-b38e5d1d-7044-429b-8386-dd82a9e9bad8 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Listed batch of 1 results from cell out of 1000 limit. Returned 1 total so far. {{(pid=138681) do_query /opt/stack/nova/nova/compute/multi_cell_list.py:373}} Jul 16 22:13:29.227415 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-b38e5d1d-7044-429b-8386-dd82a9e9bad8 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] 10.222.0.16 "GET /compute/v2.1/servers" status: 200 len: 340 microversion: 2.1 time: 0.068556 Jul 16 22:13:29.228002 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3434/6859] 10.222.0.16 () {58 vars in 1124 bytes} [Sat Jul 16 22:13:29 2022] GET /compute/v2.1/servers => generated 340 bytes in 70 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 22:13:29.238696 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-a12a7c8e-618f-4c2f-ab3a-39739198f5a2 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:13:29.239363 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-a12a7c8e-618f-4c2f-ab3a-39739198f5a2 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] [instance: 41010ce2-2848-458a-b6da-5d2ac455bcc0] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:13:29.245816 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a12a7c8e-618f-4c2f-ab3a-39739198f5a2 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:13:29.246263 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a12a7c8e-618f-4c2f-ab3a-39739198f5a2 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:13:29.247049 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a12a7c8e-618f-4c2f-ab3a-39739198f5a2 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:13:29.247495 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a12a7c8e-618f-4c2f-ab3a-39739198f5a2 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:13:29.375226 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-a12a7c8e-618f-4c2f-ab3a-39739198f5a2 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd749cd77cf5c489e8c4e57e02ce768fd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'deb1e913bedd4fb5b286e21c60e27a41', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:13:29.382739 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-a12a7c8e-618f-4c2f-ab3a-39739198f5a2 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] 10.222.0.16 "GET /compute/v2.1/servers/41010ce2-2848-458a-b6da-5d2ac455bcc0" status: 200 len: 1545 microversion: 2.1 time: 0.147826 Jul 16 22:13:29.383370 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3426/6860] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:13:29 2022] GET /compute/v2.1/servers/41010ce2-2848-458a-b6da-5d2ac455bcc0 => generated 1545 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:13:31.158535 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-d0e5e3b7-9fc7-4e85-b96f-7b59014f8bf6 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "36056bf8-8e40-4e6d-8dbd-37c6058c9b1f"}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:13:31.159669 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-d0e5e3b7-9fc7-4e85-b96f-7b59014f8bf6 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] [instance: 41010ce2-2848-458a-b6da-5d2ac455bcc0] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:13:31.167166 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d0e5e3b7-9fc7-4e85-b96f-7b59014f8bf6 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:13:31.167647 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d0e5e3b7-9fc7-4e85-b96f-7b59014f8bf6 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:13:31.168442 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d0e5e3b7-9fc7-4e85-b96f-7b59014f8bf6 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:13:31.168903 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d0e5e3b7-9fc7-4e85-b96f-7b59014f8bf6 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:13:31.762500 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-d0e5e3b7-9fc7-4e85-b96f-7b59014f8bf6 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] 10.222.0.16 "POST /compute/v2.1/servers/41010ce2-2848-458a-b6da-5d2ac455bcc0/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.607789 Jul 16 22:13:31.763279 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3435/6861] 10.222.0.16 () {60 vars in 1322 bytes} [Sat Jul 16 22:13:31 2022] POST /compute/v2.1/servers/41010ce2-2848-458a-b6da-5d2ac455bcc0/os-volume_attachments => generated 194 bytes in 609 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 22:13:35.085775 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-3f56f60d-06ba-4e7e-9c15-eae9d5660d8b tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:13:35.086399 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-3f56f60d-06ba-4e7e-9c15-eae9d5660d8b tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] [instance: 41010ce2-2848-458a-b6da-5d2ac455bcc0] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:13:35.093668 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3f56f60d-06ba-4e7e-9c15-eae9d5660d8b tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:13:35.094126 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3f56f60d-06ba-4e7e-9c15-eae9d5660d8b tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:13:35.094953 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3f56f60d-06ba-4e7e-9c15-eae9d5660d8b tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:13:35.095423 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3f56f60d-06ba-4e7e-9c15-eae9d5660d8b tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:13:35.230414 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-3f56f60d-06ba-4e7e-9c15-eae9d5660d8b tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd749cd77cf5c489e8c4e57e02ce768fd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'deb1e913bedd4fb5b286e21c60e27a41', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:13:35.238522 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-3f56f60d-06ba-4e7e-9c15-eae9d5660d8b tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] 10.222.0.16 "GET /compute/v2.1/servers/41010ce2-2848-458a-b6da-5d2ac455bcc0" status: 200 len: 1591 microversion: 2.1 time: 0.156549 Jul 16 22:13:35.239116 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3427/6862] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:13:35 2022] GET /compute/v2.1/servers/41010ce2-2848-458a-b6da-5d2ac455bcc0 => generated 1591 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:13:37.109623 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-9e595a3c-bd59-4639-8443-e851122efff6 req-3084be0a-0701-40d3-ad74-dd5e296823d4 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "41010ce2-2848-458a-b6da-5d2ac455bcc0", "tag": "fff60fc8-0659-4ea8-bf1d-6bec19f459a0"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:13:37.118644 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-9e595a3c-bd59-4639-8443-e851122efff6 req-3084be0a-0701-40d3-ad74-dd5e296823d4 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:13:37.119210 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-9e595a3c-bd59-4639-8443-e851122efff6 req-3084be0a-0701-40d3-ad74-dd5e296823d4 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:13:37.132729 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-9e595a3c-bd59-4639-8443-e851122efff6 req-3084be0a-0701-40d3-ad74-dd5e296823d4 service nova] Creating event network-changed:fff60fc8-0659-4ea8-bf1d-6bec19f459a0 for instance 41010ce2-2848-458a-b6da-5d2ac455bcc0 on nch1-master-1 Jul 16 22:13:37.139437 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-9e595a3c-bd59-4639-8443-e851122efff6 req-3084be0a-0701-40d3-ad74-dd5e296823d4 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.037623 Jul 16 22:13:37.140367 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3436/6863] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 22:13:37 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 22:13:39.443594 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-e2ead8dc-43d8-45e3-b4b1-bb1208a828cc tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Action: 'action', calling method: >, body: {"addSecurityGroup": {"name": "tempest-secgroup-smoke-1525359978"}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:13:39.444168 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-e2ead8dc-43d8-45e3-b4b1-bb1208a828cc tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] [instance: 41010ce2-2848-458a-b6da-5d2ac455bcc0] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:13:39.451546 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e2ead8dc-43d8-45e3-b4b1-bb1208a828cc tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:13:39.451989 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e2ead8dc-43d8-45e3-b4b1-bb1208a828cc tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:13:39.452757 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e2ead8dc-43d8-45e3-b4b1-bb1208a828cc tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:13:39.453262 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e2ead8dc-43d8-45e3-b4b1-bb1208a828cc tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:13:39.583912 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.network.security_group_api [None req-e2ead8dc-43d8-45e3-b4b1-bb1208a828cc tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Adding security group 2e49ecad-30f5-4187-bef2-16aed93fc29a to port fff60fc8-0659-4ea8-bf1d-6bec19f459a0 Jul 16 22:13:40.004520 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-e2ead8dc-43d8-45e3-b4b1-bb1208a828cc tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] 10.222.0.16 "POST /compute/v2.1/servers/41010ce2-2848-458a-b6da-5d2ac455bcc0/action" status: 202 len: 0 microversion: 2.1 time: 0.565093 Jul 16 22:13:40.005341 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3428/6864] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 22:13:39 2022] POST /compute/v2.1/servers/41010ce2-2848-458a-b6da-5d2ac455bcc0/action => generated 0 bytes in 567 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 22:13:40.015713 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-17d7e65f-925d-4f2c-998e-f9954b5891be tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:13:40.016363 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-5fc59b85-1d37-40b6-be5a-a2645f9eb13f req-30a9bcc7-cb8c-46a9-ae99-336491c3b150 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "41010ce2-2848-458a-b6da-5d2ac455bcc0", "tag": "fff60fc8-0659-4ea8-bf1d-6bec19f459a0"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:13:40.017417 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-17d7e65f-925d-4f2c-998e-f9954b5891be tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] [instance: 41010ce2-2848-458a-b6da-5d2ac455bcc0] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:13:40.024375 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell0:SELECT=2 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:13:40.024637 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-5fc59b85-1d37-40b6-be5a-a2645f9eb13f req-30a9bcc7-cb8c-46a9-ae99-336491c3b150 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:13:40.025290 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-5fc59b85-1d37-40b6-be5a-a2645f9eb13f req-30a9bcc7-cb8c-46a9-ae99-336491c3b150 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:13:40.027149 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-17d7e65f-925d-4f2c-998e-f9954b5891be tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:13:40.028908 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-17d7e65f-925d-4f2c-998e-f9954b5891be tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:13:40.030980 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-17d7e65f-925d-4f2c-998e-f9954b5891be tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:13:40.039195 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-17d7e65f-925d-4f2c-998e-f9954b5891be tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:13:40.041007 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-5fc59b85-1d37-40b6-be5a-a2645f9eb13f req-30a9bcc7-cb8c-46a9-ae99-336491c3b150 service nova] Creating event network-changed:fff60fc8-0659-4ea8-bf1d-6bec19f459a0 for instance 41010ce2-2848-458a-b6da-5d2ac455bcc0 on nch1-master-1 Jul 16 22:13:40.048817 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-5fc59b85-1d37-40b6-be5a-a2645f9eb13f req-30a9bcc7-cb8c-46a9-ae99-336491c3b150 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.036129 Jul 16 22:13:40.050266 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3429/6865] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 22:13:40 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 22:13:40.260587 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-17d7e65f-925d-4f2c-998e-f9954b5891be tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd749cd77cf5c489e8c4e57e02ce768fd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'deb1e913bedd4fb5b286e21c60e27a41', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:13:40.270772 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-17d7e65f-925d-4f2c-998e-f9954b5891be tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] 10.222.0.16 "GET /compute/v2.1/servers/41010ce2-2848-458a-b6da-5d2ac455bcc0" status: 200 len: 1757 microversion: 2.1 time: 0.260526 Jul 16 22:13:40.271929 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3437/6866] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:13:40 2022] GET /compute/v2.1/servers/41010ce2-2848-458a-b6da-5d2ac455bcc0 => generated 1757 bytes in 263 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:14:02.639104 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_api:SELECT=41,nova_api:INSERT=1 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:14:02.640046 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=68 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:14:02.644147 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-f458d91a-68f7-45d5-9ccd-0686754548a3 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Action: 'action', calling method: >, body: {"reboot": {"type": "SOFT"}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:14:02.645196 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-f458d91a-68f7-45d5-9ccd-0686754548a3 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] [instance: 41010ce2-2848-458a-b6da-5d2ac455bcc0] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:14:02.651325 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f458d91a-68f7-45d5-9ccd-0686754548a3 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:14:02.653549 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f458d91a-68f7-45d5-9ccd-0686754548a3 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:14:02.653549 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f458d91a-68f7-45d5-9ccd-0686754548a3 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:14:02.653549 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f458d91a-68f7-45d5-9ccd-0686754548a3 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:14:02.813750 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-f458d91a-68f7-45d5-9ccd-0686754548a3 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] 10.222.0.16 "POST /compute/v2.1/servers/41010ce2-2848-458a-b6da-5d2ac455bcc0/action" status: 202 len: 0 microversion: 2.1 time: 0.177329 Jul 16 22:14:02.814513 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3430/6867] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 22:14:02 2022] POST /compute/v2.1/servers/41010ce2-2848-458a-b6da-5d2ac455bcc0/action => generated 0 bytes in 179 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 22:14:02.823876 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-a32abbcb-6ead-4062-994c-3479e2bf8523 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:14:02.825079 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-a32abbcb-6ead-4062-994c-3479e2bf8523 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] [instance: 41010ce2-2848-458a-b6da-5d2ac455bcc0] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:14:02.827858 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_api:SELECT=41,nova_api:INSERT=3 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:14:02.828686 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=79,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:14:02.833392 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a32abbcb-6ead-4062-994c-3479e2bf8523 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:14:02.837003 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a32abbcb-6ead-4062-994c-3479e2bf8523 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:14:02.837003 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a32abbcb-6ead-4062-994c-3479e2bf8523 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:14:02.837003 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a32abbcb-6ead-4062-994c-3479e2bf8523 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:14:02.989622 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-a32abbcb-6ead-4062-994c-3479e2bf8523 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd749cd77cf5c489e8c4e57e02ce768fd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'deb1e913bedd4fb5b286e21c60e27a41', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:14:02.999747 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-a32abbcb-6ead-4062-994c-3479e2bf8523 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] 10.222.0.16 "GET /compute/v2.1/servers/41010ce2-2848-458a-b6da-5d2ac455bcc0" status: 200 len: 1749 microversion: 2.1 time: 0.180629 Jul 16 22:14:03.000338 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3438/6868] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:14:02 2022] GET /compute/v2.1/servers/41010ce2-2848-458a-b6da-5d2ac455bcc0 => generated 1749 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:14:04.019469 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-e762426c-51cd-4a73-a989-293d854780f7 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:14:04.020171 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-e762426c-51cd-4a73-a989-293d854780f7 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] [instance: 41010ce2-2848-458a-b6da-5d2ac455bcc0] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:14:04.026480 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e762426c-51cd-4a73-a989-293d854780f7 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:14:04.026933 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e762426c-51cd-4a73-a989-293d854780f7 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:14:04.027713 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e762426c-51cd-4a73-a989-293d854780f7 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:14:04.028267 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e762426c-51cd-4a73-a989-293d854780f7 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:14:04.178925 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-e762426c-51cd-4a73-a989-293d854780f7 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd749cd77cf5c489e8c4e57e02ce768fd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'deb1e913bedd4fb5b286e21c60e27a41', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:14:04.187938 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-e762426c-51cd-4a73-a989-293d854780f7 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] 10.222.0.16 "GET /compute/v2.1/servers/41010ce2-2848-458a-b6da-5d2ac455bcc0" status: 200 len: 1754 microversion: 2.1 time: 0.173715 Jul 16 22:14:04.188507 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3431/6869] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:14:04 2022] GET /compute/v2.1/servers/41010ce2-2848-458a-b6da-5d2ac455bcc0 => generated 1754 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:14:05.206242 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-509d27a3-d644-4ec6-b96e-082e65c3bbec tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:14:05.206880 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-509d27a3-d644-4ec6-b96e-082e65c3bbec tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] [instance: 41010ce2-2848-458a-b6da-5d2ac455bcc0] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:14:05.214548 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-509d27a3-d644-4ec6-b96e-082e65c3bbec tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:14:05.215030 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-509d27a3-d644-4ec6-b96e-082e65c3bbec tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:14:05.216472 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-509d27a3-d644-4ec6-b96e-082e65c3bbec tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:14:05.216927 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-509d27a3-d644-4ec6-b96e-082e65c3bbec tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:14:05.360999 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-509d27a3-d644-4ec6-b96e-082e65c3bbec tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd749cd77cf5c489e8c4e57e02ce768fd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'deb1e913bedd4fb5b286e21c60e27a41', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:14:05.369179 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-509d27a3-d644-4ec6-b96e-082e65c3bbec tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] 10.222.0.16 "GET /compute/v2.1/servers/41010ce2-2848-458a-b6da-5d2ac455bcc0" status: 200 len: 1754 microversion: 2.1 time: 0.166862 Jul 16 22:14:05.369753 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3439/6870] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:14:05 2022] GET /compute/v2.1/servers/41010ce2-2848-458a-b6da-5d2ac455bcc0 => generated 1754 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:14:06.387289 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-51a6e746-5152-4f7e-99de-b7acb81d553b tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:14:06.388042 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-51a6e746-5152-4f7e-99de-b7acb81d553b tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] [instance: 41010ce2-2848-458a-b6da-5d2ac455bcc0] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:14:06.394867 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-51a6e746-5152-4f7e-99de-b7acb81d553b tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:14:06.395347 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-51a6e746-5152-4f7e-99de-b7acb81d553b tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:14:06.396103 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-51a6e746-5152-4f7e-99de-b7acb81d553b tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:14:06.396541 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-51a6e746-5152-4f7e-99de-b7acb81d553b tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:14:06.543157 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-51a6e746-5152-4f7e-99de-b7acb81d553b tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd749cd77cf5c489e8c4e57e02ce768fd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'deb1e913bedd4fb5b286e21c60e27a41', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:14:06.551225 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-51a6e746-5152-4f7e-99de-b7acb81d553b tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] 10.222.0.16 "GET /compute/v2.1/servers/41010ce2-2848-458a-b6da-5d2ac455bcc0" status: 200 len: 1754 microversion: 2.1 time: 0.167885 Jul 16 22:14:06.551786 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3432/6871] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:14:06 2022] GET /compute/v2.1/servers/41010ce2-2848-458a-b6da-5d2ac455bcc0 => generated 1754 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:14:07.569882 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-9c1f777f-6f7a-4ccb-98dd-846f0db6a095 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:14:07.570519 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-9c1f777f-6f7a-4ccb-98dd-846f0db6a095 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] [instance: 41010ce2-2848-458a-b6da-5d2ac455bcc0] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:14:07.577685 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9c1f777f-6f7a-4ccb-98dd-846f0db6a095 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:14:07.578159 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9c1f777f-6f7a-4ccb-98dd-846f0db6a095 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:14:07.578945 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9c1f777f-6f7a-4ccb-98dd-846f0db6a095 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:14:07.579428 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9c1f777f-6f7a-4ccb-98dd-846f0db6a095 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:14:07.722859 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-9c1f777f-6f7a-4ccb-98dd-846f0db6a095 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd749cd77cf5c489e8c4e57e02ce768fd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'deb1e913bedd4fb5b286e21c60e27a41', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:14:07.731826 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-9c1f777f-6f7a-4ccb-98dd-846f0db6a095 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] 10.222.0.16 "GET /compute/v2.1/servers/41010ce2-2848-458a-b6da-5d2ac455bcc0" status: 200 len: 1754 microversion: 2.1 time: 0.165438 Jul 16 22:14:07.732410 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3440/6872] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:14:07 2022] GET /compute/v2.1/servers/41010ce2-2848-458a-b6da-5d2ac455bcc0 => generated 1754 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:14:08.749530 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-e98a1bc1-5f25-4ffb-b110-f0cbb7c50614 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:14:08.750182 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-e98a1bc1-5f25-4ffb-b110-f0cbb7c50614 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] [instance: 41010ce2-2848-458a-b6da-5d2ac455bcc0] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:14:08.757213 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e98a1bc1-5f25-4ffb-b110-f0cbb7c50614 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:14:08.757673 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e98a1bc1-5f25-4ffb-b110-f0cbb7c50614 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:14:08.758437 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e98a1bc1-5f25-4ffb-b110-f0cbb7c50614 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:14:08.758881 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e98a1bc1-5f25-4ffb-b110-f0cbb7c50614 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:14:08.903243 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-e98a1bc1-5f25-4ffb-b110-f0cbb7c50614 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd749cd77cf5c489e8c4e57e02ce768fd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'deb1e913bedd4fb5b286e21c60e27a41', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:14:08.911304 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-e98a1bc1-5f25-4ffb-b110-f0cbb7c50614 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] 10.222.0.16 "GET /compute/v2.1/servers/41010ce2-2848-458a-b6da-5d2ac455bcc0" status: 200 len: 1754 microversion: 2.1 time: 0.166569 Jul 16 22:14:08.911876 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3433/6873] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:14:08 2022] GET /compute/v2.1/servers/41010ce2-2848-458a-b6da-5d2ac455bcc0 => generated 1754 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:14:09.929108 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-1127c191-be5c-40a0-91fe-bce2b13cc6ff tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:14:09.929748 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-1127c191-be5c-40a0-91fe-bce2b13cc6ff tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] [instance: 41010ce2-2848-458a-b6da-5d2ac455bcc0] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:14:09.937052 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1127c191-be5c-40a0-91fe-bce2b13cc6ff tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:14:09.937526 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1127c191-be5c-40a0-91fe-bce2b13cc6ff tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:14:09.938313 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1127c191-be5c-40a0-91fe-bce2b13cc6ff tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:14:09.938780 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1127c191-be5c-40a0-91fe-bce2b13cc6ff tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:14:10.091587 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-1127c191-be5c-40a0-91fe-bce2b13cc6ff tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd749cd77cf5c489e8c4e57e02ce768fd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'deb1e913bedd4fb5b286e21c60e27a41', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:14:10.099907 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-1127c191-be5c-40a0-91fe-bce2b13cc6ff tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] 10.222.0.16 "GET /compute/v2.1/servers/41010ce2-2848-458a-b6da-5d2ac455bcc0" status: 200 len: 1754 microversion: 2.1 time: 0.174751 Jul 16 22:14:10.103127 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3441/6874] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:14:09 2022] GET /compute/v2.1/servers/41010ce2-2848-458a-b6da-5d2ac455bcc0 => generated 1754 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:14:11.117866 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-64a41058-5621-4d1b-bc43-c3b888050dfa tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:14:11.118434 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-64a41058-5621-4d1b-bc43-c3b888050dfa tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] [instance: 41010ce2-2848-458a-b6da-5d2ac455bcc0] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:14:11.126071 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-64a41058-5621-4d1b-bc43-c3b888050dfa tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:14:11.126531 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-64a41058-5621-4d1b-bc43-c3b888050dfa tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:14:11.127334 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-64a41058-5621-4d1b-bc43-c3b888050dfa tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:14:11.127779 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-64a41058-5621-4d1b-bc43-c3b888050dfa tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:14:11.278887 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-64a41058-5621-4d1b-bc43-c3b888050dfa tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd749cd77cf5c489e8c4e57e02ce768fd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'deb1e913bedd4fb5b286e21c60e27a41', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:14:11.280445 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-64a41058-5621-4d1b-bc43-c3b888050dfa tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] 10.222.0.16 "GET /compute/v2.1/servers/41010ce2-2848-458a-b6da-5d2ac455bcc0" status: 200 len: 1754 microversion: 2.1 time: 0.166622 Jul 16 22:14:11.281012 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3434/6875] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:14:11 2022] GET /compute/v2.1/servers/41010ce2-2848-458a-b6da-5d2ac455bcc0 => generated 1754 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:14:12.298022 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-21b0d6da-8b5d-4e49-903f-c14a77dddccf tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:14:12.298654 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-21b0d6da-8b5d-4e49-903f-c14a77dddccf tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] [instance: 41010ce2-2848-458a-b6da-5d2ac455bcc0] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:14:12.305697 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-21b0d6da-8b5d-4e49-903f-c14a77dddccf tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:14:12.306231 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-21b0d6da-8b5d-4e49-903f-c14a77dddccf tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:14:12.307029 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-21b0d6da-8b5d-4e49-903f-c14a77dddccf tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:14:12.307479 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-21b0d6da-8b5d-4e49-903f-c14a77dddccf tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:14:12.456370 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-21b0d6da-8b5d-4e49-903f-c14a77dddccf tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd749cd77cf5c489e8c4e57e02ce768fd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'deb1e913bedd4fb5b286e21c60e27a41', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:14:12.464478 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-21b0d6da-8b5d-4e49-903f-c14a77dddccf tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] 10.222.0.16 "GET /compute/v2.1/servers/41010ce2-2848-458a-b6da-5d2ac455bcc0" status: 200 len: 1754 microversion: 2.1 time: 0.170556 Jul 16 22:14:12.465044 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3442/6876] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:14:12 2022] GET /compute/v2.1/servers/41010ce2-2848-458a-b6da-5d2ac455bcc0 => generated 1754 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 22:14:13.482085 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-a2ef6e6a-c1a1-4af0-b1ee-3a0feb553295 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:14:13.482728 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-a2ef6e6a-c1a1-4af0-b1ee-3a0feb553295 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] [instance: 41010ce2-2848-458a-b6da-5d2ac455bcc0] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:14:13.489772 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a2ef6e6a-c1a1-4af0-b1ee-3a0feb553295 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:14:13.490222 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a2ef6e6a-c1a1-4af0-b1ee-3a0feb553295 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:14:13.491023 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a2ef6e6a-c1a1-4af0-b1ee-3a0feb553295 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:14:13.491467 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a2ef6e6a-c1a1-4af0-b1ee-3a0feb553295 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:14:13.636028 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-a2ef6e6a-c1a1-4af0-b1ee-3a0feb553295 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd749cd77cf5c489e8c4e57e02ce768fd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'deb1e913bedd4fb5b286e21c60e27a41', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:14:13.644848 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-a2ef6e6a-c1a1-4af0-b1ee-3a0feb553295 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] 10.222.0.16 "GET /compute/v2.1/servers/41010ce2-2848-458a-b6da-5d2ac455bcc0" status: 200 len: 1754 microversion: 2.1 time: 0.166991 Jul 16 22:14:13.645425 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3435/6877] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:14:13 2022] GET /compute/v2.1/servers/41010ce2-2848-458a-b6da-5d2ac455bcc0 => generated 1754 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:14:14.662247 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-c175c938-06a5-4086-96ae-49d29b4a1054 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:14:14.662880 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-c175c938-06a5-4086-96ae-49d29b4a1054 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] [instance: 41010ce2-2848-458a-b6da-5d2ac455bcc0] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:14:14.670819 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c175c938-06a5-4086-96ae-49d29b4a1054 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:14:14.671352 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c175c938-06a5-4086-96ae-49d29b4a1054 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:14:14.672133 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c175c938-06a5-4086-96ae-49d29b4a1054 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:14:14.672588 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c175c938-06a5-4086-96ae-49d29b4a1054 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:14:14.818011 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-c175c938-06a5-4086-96ae-49d29b4a1054 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd749cd77cf5c489e8c4e57e02ce768fd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'deb1e913bedd4fb5b286e21c60e27a41', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:14:14.827660 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-c175c938-06a5-4086-96ae-49d29b4a1054 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] 10.222.0.16 "GET /compute/v2.1/servers/41010ce2-2848-458a-b6da-5d2ac455bcc0" status: 200 len: 1757 microversion: 2.1 time: 0.169291 Jul 16 22:14:14.828249 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3443/6878] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:14:14 2022] GET /compute/v2.1/servers/41010ce2-2848-458a-b6da-5d2ac455bcc0 => generated 1757 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:14:47.185619 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-b08b30ac-c892-424f-9fdb-602d061ff86f tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Action: 'action', calling method: >, body: {"removeSecurityGroup": {"name": "tempest-secgroup-smoke-1525359978"}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:14:47.186691 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-b08b30ac-c892-424f-9fdb-602d061ff86f tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] [instance: 41010ce2-2848-458a-b6da-5d2ac455bcc0] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:14:47.188889 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_api:SELECT=12 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:14:47.189594 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=40,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:14:47.194656 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b08b30ac-c892-424f-9fdb-602d061ff86f tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:14:47.195141 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b08b30ac-c892-424f-9fdb-602d061ff86f tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:14:47.195905 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b08b30ac-c892-424f-9fdb-602d061ff86f tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:14:47.196348 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b08b30ac-c892-424f-9fdb-602d061ff86f tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:14:47.328990 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.network.security_group_api [None req-b08b30ac-c892-424f-9fdb-602d061ff86f tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Removing security group 2e49ecad-30f5-4187-bef2-16aed93fc29a from port fff60fc8-0659-4ea8-bf1d-6bec19f459a0 Jul 16 22:14:47.705728 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-b08b30ac-c892-424f-9fdb-602d061ff86f tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] 10.222.0.16 "POST /compute/v2.1/servers/41010ce2-2848-458a-b6da-5d2ac455bcc0/action" status: 202 len: 0 microversion: 2.1 time: 0.524290 Jul 16 22:14:47.723467 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3436/6879] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 22:14:47 2022] POST /compute/v2.1/servers/41010ce2-2848-458a-b6da-5d2ac455bcc0/action => generated 0 bytes in 543 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 22:14:47.730815 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-3cbd7b17-4228-41f6-8656-670530dc190e req-3aee0c49-9205-4dce-b20e-13720573d577 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "41010ce2-2848-458a-b6da-5d2ac455bcc0", "tag": "fff60fc8-0659-4ea8-bf1d-6bec19f459a0"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:14:47.739364 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_api:SELECT=12 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:14:47.744524 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=36 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:14:47.757924 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-3cbd7b17-4228-41f6-8656-670530dc190e req-3aee0c49-9205-4dce-b20e-13720573d577 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:14:47.757924 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-3cbd7b17-4228-41f6-8656-670530dc190e req-3aee0c49-9205-4dce-b20e-13720573d577 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:14:47.779466 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-3cbd7b17-4228-41f6-8656-670530dc190e req-3aee0c49-9205-4dce-b20e-13720573d577 service nova] Creating event network-changed:fff60fc8-0659-4ea8-bf1d-6bec19f459a0 for instance 41010ce2-2848-458a-b6da-5d2ac455bcc0 on nch1-master-1 Jul 16 22:14:47.787703 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-3cbd7b17-4228-41f6-8656-670530dc190e req-3aee0c49-9205-4dce-b20e-13720573d577 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.070714 Jul 16 22:14:47.794468 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3444/6880] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 22:14:47 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 73 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 22:14:49.391531 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-a75343a8-5186-4bac-8f63-96b84050973b tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:14:49.392055 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-a75343a8-5186-4bac-8f63-96b84050973b tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] [instance: 41010ce2-2848-458a-b6da-5d2ac455bcc0] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:14:49.398088 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a75343a8-5186-4bac-8f63-96b84050973b tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:14:49.398539 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a75343a8-5186-4bac-8f63-96b84050973b tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:14:49.399328 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a75343a8-5186-4bac-8f63-96b84050973b tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:14:49.399768 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a75343a8-5186-4bac-8f63-96b84050973b tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:14:49.481512 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.objects.instance [None req-a75343a8-5186-4bac-8f63-96b84050973b tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lazy-loading 'services' on Instance uuid 41010ce2-2848-458a-b6da-5d2ac455bcc0 {{(pid=138680) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 22:14:49.647291 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-a75343a8-5186-4bac-8f63-96b84050973b tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] 10.222.0.16 "DELETE /compute/v2.1/servers/41010ce2-2848-458a-b6da-5d2ac455bcc0/os-volume_attachments/36056bf8-8e40-4e6d-8dbd-37c6058c9b1f" status: 202 len: 0 microversion: 2.1 time: 0.260572 Jul 16 22:14:49.648119 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3437/6881] 10.222.0.16 () {58 vars in 1415 bytes} [Sat Jul 16 22:14:49 2022] DELETE /compute/v2.1/servers/41010ce2-2848-458a-b6da-5d2ac455bcc0/os-volume_attachments/36056bf8-8e40-4e6d-8dbd-37c6058c9b1f => generated 0 bytes in 262 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 22:14:49.667853 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-6a189931-f4bd-4a52-8d3b-e7635f315c1d tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:14:49.669022 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-6a189931-f4bd-4a52-8d3b-e7635f315c1d tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] [instance: 41010ce2-2848-458a-b6da-5d2ac455bcc0] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:14:49.676025 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6a189931-f4bd-4a52-8d3b-e7635f315c1d tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:14:49.676622 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6a189931-f4bd-4a52-8d3b-e7635f315c1d tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:14:49.677689 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6a189931-f4bd-4a52-8d3b-e7635f315c1d tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:14:49.677926 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6a189931-f4bd-4a52-8d3b-e7635f315c1d tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:14:49.710038 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-6a189931-f4bd-4a52-8d3b-e7635f315c1d tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] 10.222.0.16 "GET /compute/v2.1/servers/41010ce2-2848-458a-b6da-5d2ac455bcc0/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.057009 Jul 16 22:14:49.714363 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3445/6882] 10.222.0.16 () {58 vars in 1301 bytes} [Sat Jul 16 22:14:49 2022] GET /compute/v2.1/servers/41010ce2-2848-458a-b6da-5d2ac455bcc0/os-volume_attachments => generated 197 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 22:14:49.805514 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-078d77d3-98d4-4080-a040-9db8e935050c req-b05701e7-0ed7-410a-aa56-e447d0637985 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "41010ce2-2848-458a-b6da-5d2ac455bcc0", "tag": "fff60fc8-0659-4ea8-bf1d-6bec19f459a0"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:14:49.814467 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-078d77d3-98d4-4080-a040-9db8e935050c req-b05701e7-0ed7-410a-aa56-e447d0637985 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:14:49.815199 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-078d77d3-98d4-4080-a040-9db8e935050c req-b05701e7-0ed7-410a-aa56-e447d0637985 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:14:49.827222 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-078d77d3-98d4-4080-a040-9db8e935050c req-b05701e7-0ed7-410a-aa56-e447d0637985 service nova] Creating event network-changed:fff60fc8-0659-4ea8-bf1d-6bec19f459a0 for instance 41010ce2-2848-458a-b6da-5d2ac455bcc0 on nch1-master-1 Jul 16 22:14:49.832347 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-078d77d3-98d4-4080-a040-9db8e935050c req-b05701e7-0ed7-410a-aa56-e447d0637985 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.030914 Jul 16 22:14:49.832935 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3438/6883] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 22:14:49 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 22:14:50.722583 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-93220e86-6573-426b-8aa0-0bba6abaa454 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:14:50.723581 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-93220e86-6573-426b-8aa0-0bba6abaa454 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] [instance: 41010ce2-2848-458a-b6da-5d2ac455bcc0] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:14:50.732946 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-93220e86-6573-426b-8aa0-0bba6abaa454 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:14:50.734163 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-93220e86-6573-426b-8aa0-0bba6abaa454 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:14:50.734748 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-93220e86-6573-426b-8aa0-0bba6abaa454 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:14:50.735247 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-93220e86-6573-426b-8aa0-0bba6abaa454 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:14:50.764235 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-93220e86-6573-426b-8aa0-0bba6abaa454 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] 10.222.0.16 "GET /compute/v2.1/servers/41010ce2-2848-458a-b6da-5d2ac455bcc0/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.045702 Jul 16 22:14:50.764805 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3446/6884] 10.222.0.16 () {58 vars in 1301 bytes} [Sat Jul 16 22:14:50 2022] GET /compute/v2.1/servers/41010ce2-2848-458a-b6da-5d2ac455bcc0/os-volume_attachments => generated 25 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Jul 16 22:14:50.776314 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-f1d0ebc0-94a7-410b-b0aa-a74e43ff5619 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:14:50.777035 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-f1d0ebc0-94a7-410b-b0aa-a74e43ff5619 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] [instance: 41010ce2-2848-458a-b6da-5d2ac455bcc0] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:14:50.784556 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f1d0ebc0-94a7-410b-b0aa-a74e43ff5619 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:14:50.785022 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f1d0ebc0-94a7-410b-b0aa-a74e43ff5619 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:14:50.785840 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f1d0ebc0-94a7-410b-b0aa-a74e43ff5619 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:14:50.786289 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f1d0ebc0-94a7-410b-b0aa-a74e43ff5619 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:14:50.864583 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-f1d0ebc0-94a7-410b-b0aa-a74e43ff5619 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] HTTP exception thrown: Instance 41010ce2-2848-458a-b6da-5d2ac455bcc0 is not attached to volume 36056bf8-8e40-4e6d-8dbd-37c6058c9b1f Jul 16 22:14:50.866187 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-f1d0ebc0-94a7-410b-b0aa-a74e43ff5619 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Request method failure captured: Jul 16 22:14:50.866187 ncdv-master-1 devstack@n-api.service[138680]: request: DELETE /compute/v2.1/servers/41010ce2-2848-458a-b6da-5d2ac455bcc0/os-volume_attachments/36056bf8-8e40-4e6d-8dbd-37c6058c9b1f HTTP/1.1 Jul 16 22:14:50.866187 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 22:14:50.866187 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 22:14:50.866187 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 22:14:50.866187 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 0 Jul 16 22:14:50.866187 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 22:14:50.866187 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 22:14:50.866187 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 22:14:50.866187 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 22:14:50.866187 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0zbe3rCsn-3QaicbUyd_l2yeeYdFw5j3pI0YNVDnyDW1adHPbvFyygFfhKMkLZKzSd5wCIHBh53_Ahvryl0lX_fYnOhaAfRD_aC5xNe-jnygMGiPOaIiG3WVsRGIF7WYX8xbkLiWHdzOeuosVkQ9hfVKA2k7v4UqqZHZ5ysM0snBRFk Jul 16 22:14:50.866187 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 22:14:50.866187 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 22:14:50.866187 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 22:14:50.866187 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 22:14:50.866187 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 22:14:50.866187 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 22:14:50.866187 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: deb1e913bedd4fb5b286e21c60e27a41 Jul 16 22:14:50.866187 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-TestMinimumBasicScenario-1144486311 Jul 16 22:14:50.866187 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 22:14:50.866187 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 22:14:50.866187 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/deb1e913bedd4fb5b286e21c60e27a41"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/deb1e913bedd4fb5b286e21c60e27a41"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_deb1e913bedd4fb5b286e21c60e27a41"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/deb1e913bedd4fb5b286e21c60e27a41"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 22:14:50.866187 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-TestMinimumBasicScenario-1144486311 Jul 16 22:14:50.866187 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: deb1e913bedd4fb5b286e21c60e27a41 Jul 16 22:14:50.866187 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-TestMinimumBasicScenario-1144486311 Jul 16 22:14:50.866187 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-TestMinimumBasicScenario-1144486311-project Jul 16 22:14:50.866187 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 22:14:50.866187 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 22:14:50.866187 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: d749cd77cf5c489e8c4e57e02ce768fd Jul 16 22:14:50.866187 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-TestMinimumBasicScenario-1144486311-project Jul 16 22:14:50.866187 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 22:14:50.866187 ncdv-master-1 devstack@n-api.service[138680]: exception: Instance 41010ce2-2848-458a-b6da-5d2ac455bcc0 is not attached to volume 36056bf8-8e40-4e6d-8dbd-37c6058c9b1f Jul 16 22:14:50.866187 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'server_id': '41010ce2-2848-458a-b6da-5d2ac455bcc0', 'id': '36056bf8-8e40-4e6d-8dbd-37c6058c9b1f'} Jul 16 22:14:50.866187 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 22:14:50.866187 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 22:14:50.866187 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 552, in delete Jul 16 22:14:50.866187 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi bdm = objects.BlockDeviceMapping.get_by_volume_and_instance( Jul 16 22:14:50.866187 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Jul 16 22:14:50.866187 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Jul 16 22:14:50.866187 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/block_device.py", line 281, in get_by_volume_and_instance Jul 16 22:14:50.866187 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.VolumeBDMNotFound(volume_id=volume_id) Jul 16 22:14:50.866187 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.VolumeBDMNotFound: No volume Block Device Mapping with id 36056bf8-8e40-4e6d-8dbd-37c6058c9b1f. Jul 16 22:14:50.866187 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 22:14:50.866187 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 22:14:50.866187 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 22:14:50.866187 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 22:14:50.866187 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 22:14:50.866187 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 22:14:50.866187 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 22:14:50.866187 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 22:14:50.866187 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 22:14:50.866187 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 22:14:50.866187 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 558, in delete Jul 16 22:14:50.866187 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=msg) Jul 16 22:14:50.866187 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 41010ce2-2848-458a-b6da-5d2ac455bcc0 is not attached to volume 36056bf8-8e40-4e6d-8dbd-37c6058c9b1f Jul 16 22:14:50.866187 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 22:14:50.866187 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 22:14:50.866187 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 22:14:50.866187 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 22:14:50.866187 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 22:14:50.866187 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 22:14:50.870990 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 22:14:50.870990 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 22:14:50.870990 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 41010ce2-2848-458a-b6da-5d2ac455bcc0 is not attached to volume 36056bf8-8e40-4e6d-8dbd-37c6058c9b1f Jul 16 22:14:50.870990 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 22:14:50.870990 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-f1d0ebc0-94a7-410b-b0aa-a74e43ff5619 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Returning 404 to user: Instance 41010ce2-2848-458a-b6da-5d2ac455bcc0 is not attached to volume 36056bf8-8e40-4e6d-8dbd-37c6058c9b1f {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 22:14:50.870990 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-f1d0ebc0-94a7-410b-b0aa-a74e43ff5619 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] 10.222.0.16 "DELETE /compute/v2.1/servers/41010ce2-2848-458a-b6da-5d2ac455bcc0/os-volume_attachments/36056bf8-8e40-4e6d-8dbd-37c6058c9b1f" status: 404 len: 154 microversion: 2.1 time: 0.096710 Jul 16 22:14:50.870990 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3439/6885] 10.222.0.16 () {58 vars in 1415 bytes} [Sat Jul 16 22:14:50 2022] DELETE /compute/v2.1/servers/41010ce2-2848-458a-b6da-5d2ac455bcc0/os-volume_attachments/36056bf8-8e40-4e6d-8dbd-37c6058c9b1f => generated 154 bytes in 98 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 22:14:54.204075 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-a16ecb1a-a150-4ba2-8d57-fff6f538d7b2 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:14:54.204624 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-a16ecb1a-a150-4ba2-8d57-fff6f538d7b2 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] [instance: 41010ce2-2848-458a-b6da-5d2ac455bcc0] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:14:54.211663 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a16ecb1a-a150-4ba2-8d57-fff6f538d7b2 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:14:54.212129 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a16ecb1a-a150-4ba2-8d57-fff6f538d7b2 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:14:54.212904 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a16ecb1a-a150-4ba2-8d57-fff6f538d7b2 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:14:54.213358 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a16ecb1a-a150-4ba2-8d57-fff6f538d7b2 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:14:54.238773 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-a16ecb1a-a150-4ba2-8d57-fff6f538d7b2 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] [instance: 41010ce2-2848-458a-b6da-5d2ac455bcc0] Going to try to terminate instance {{(pid=138681) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 16 22:14:54.522966 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-a16ecb1a-a150-4ba2-8d57-fff6f538d7b2 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] 10.222.0.16 "DELETE /compute/v2.1/servers/41010ce2-2848-458a-b6da-5d2ac455bcc0" status: 204 len: 0 microversion: 2.1 time: 0.323090 Jul 16 22:14:54.523818 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3447/6886] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 22:14:54 2022] DELETE /compute/v2.1/servers/41010ce2-2848-458a-b6da-5d2ac455bcc0 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 16 22:14:54.532033 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-7d3e621a-1a33-4d35-9dd4-9f0b73861448 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:14:54.532857 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-7d3e621a-1a33-4d35-9dd4-9f0b73861448 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] [instance: 41010ce2-2848-458a-b6da-5d2ac455bcc0] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:14:54.540868 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7d3e621a-1a33-4d35-9dd4-9f0b73861448 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:14:54.541314 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7d3e621a-1a33-4d35-9dd4-9f0b73861448 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:14:54.542075 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7d3e621a-1a33-4d35-9dd4-9f0b73861448 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:14:54.542540 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7d3e621a-1a33-4d35-9dd4-9f0b73861448 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:14:54.673174 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-7d3e621a-1a33-4d35-9dd4-9f0b73861448 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd749cd77cf5c489e8c4e57e02ce768fd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'deb1e913bedd4fb5b286e21c60e27a41', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:14:54.680631 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-7d3e621a-1a33-4d35-9dd4-9f0b73861448 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] 10.222.0.16 "GET /compute/v2.1/servers/41010ce2-2848-458a-b6da-5d2ac455bcc0" status: 200 len: 1551 microversion: 2.1 time: 0.152621 Jul 16 22:14:54.681193 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3440/6887] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:14:54 2022] GET /compute/v2.1/servers/41010ce2-2848-458a-b6da-5d2ac455bcc0 => generated 1551 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:14:55.699466 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-f719add0-60c6-4264-aaaa-155de5c2b237 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:14:55.700188 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-f719add0-60c6-4264-aaaa-155de5c2b237 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] [instance: 41010ce2-2848-458a-b6da-5d2ac455bcc0] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:14:55.708257 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f719add0-60c6-4264-aaaa-155de5c2b237 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:14:55.708722 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f719add0-60c6-4264-aaaa-155de5c2b237 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:14:55.709495 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f719add0-60c6-4264-aaaa-155de5c2b237 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:14:55.709944 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f719add0-60c6-4264-aaaa-155de5c2b237 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:14:55.843641 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-f719add0-60c6-4264-aaaa-155de5c2b237 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd749cd77cf5c489e8c4e57e02ce768fd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'deb1e913bedd4fb5b286e21c60e27a41', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:14:55.851889 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-f719add0-60c6-4264-aaaa-155de5c2b237 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] 10.222.0.16 "GET /compute/v2.1/servers/41010ce2-2848-458a-b6da-5d2ac455bcc0" status: 200 len: 1551 microversion: 2.1 time: 0.156764 Jul 16 22:14:55.852456 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3448/6888] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:14:55 2022] GET /compute/v2.1/servers/41010ce2-2848-458a-b6da-5d2ac455bcc0 => generated 1551 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:14:56.608095 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-8dd60c89-8f28-4754-a706-1795457d3c4d req-6c23c539-adee-4310-bbd9-e9c5515dd13d service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "41010ce2-2848-458a-b6da-5d2ac455bcc0", "name": "network-vif-deleted", "tag": "fff60fc8-0659-4ea8-bf1d-6bec19f459a0"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:14:56.616987 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-8dd60c89-8f28-4754-a706-1795457d3c4d req-6c23c539-adee-4310-bbd9-e9c5515dd13d service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:14:56.620677 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-8dd60c89-8f28-4754-a706-1795457d3c4d req-6c23c539-adee-4310-bbd9-e9c5515dd13d service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:14:56.631441 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-8dd60c89-8f28-4754-a706-1795457d3c4d req-6c23c539-adee-4310-bbd9-e9c5515dd13d service nova] Creating event network-vif-deleted:fff60fc8-0659-4ea8-bf1d-6bec19f459a0 for instance 41010ce2-2848-458a-b6da-5d2ac455bcc0 on nch1-master-1 Jul 16 22:14:56.636510 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-8dd60c89-8f28-4754-a706-1795457d3c4d req-6c23c539-adee-4310-bbd9-e9c5515dd13d service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.032092 Jul 16 22:14:56.637987 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3441/6889] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 22:14:56 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 22:14:56.870914 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-88b8d3bd-1da9-4437-a87a-3b7a67f0f0ec tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:14:56.871677 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-88b8d3bd-1da9-4437-a87a-3b7a67f0f0ec tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] [instance: 41010ce2-2848-458a-b6da-5d2ac455bcc0] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:14:56.880029 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-88b8d3bd-1da9-4437-a87a-3b7a67f0f0ec tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:14:56.881237 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-88b8d3bd-1da9-4437-a87a-3b7a67f0f0ec tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:14:56.882056 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-88b8d3bd-1da9-4437-a87a-3b7a67f0f0ec tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:14:56.882539 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-88b8d3bd-1da9-4437-a87a-3b7a67f0f0ec tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:14:56.921051 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-88b8d3bd-1da9-4437-a87a-3b7a67f0f0ec tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:14:56.921520 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-88b8d3bd-1da9-4437-a87a-3b7a67f0f0ec tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:14:56.922140 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.objects.instance [None req-88b8d3bd-1da9-4437-a87a-3b7a67f0f0ec tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lazy-loading 'fault' on Instance uuid 41010ce2-2848-458a-b6da-5d2ac455bcc0 {{(pid=138681) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 22:14:56.966346 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-88b8d3bd-1da9-4437-a87a-3b7a67f0f0ec tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd749cd77cf5c489e8c4e57e02ce768fd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'deb1e913bedd4fb5b286e21c60e27a41', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:14:56.974729 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-88b8d3bd-1da9-4437-a87a-3b7a67f0f0ec tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] 10.222.0.16 "GET /compute/v2.1/servers/41010ce2-2848-458a-b6da-5d2ac455bcc0" status: 200 len: 1345 microversion: 2.1 time: 0.107892 Jul 16 22:14:56.975634 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3449/6890] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:14:56 2022] GET /compute/v2.1/servers/41010ce2-2848-458a-b6da-5d2ac455bcc0 => generated 1345 bytes in 110 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:14:57.995247 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-0b276d40-4381-401a-bf3a-ce9290bf49f7 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:14:57.995954 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-0b276d40-4381-401a-bf3a-ce9290bf49f7 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] [instance: 41010ce2-2848-458a-b6da-5d2ac455bcc0] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:14:58.002927 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0b276d40-4381-401a-bf3a-ce9290bf49f7 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:14:58.003412 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0b276d40-4381-401a-bf3a-ce9290bf49f7 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:14:58.004173 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0b276d40-4381-401a-bf3a-ce9290bf49f7 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:14:58.004647 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0b276d40-4381-401a-bf3a-ce9290bf49f7 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:14:58.017897 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-0b276d40-4381-401a-bf3a-ce9290bf49f7 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] HTTP exception thrown: Instance 41010ce2-2848-458a-b6da-5d2ac455bcc0 could not be found. Jul 16 22:14:58.019538 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-0b276d40-4381-401a-bf3a-ce9290bf49f7 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Request method failure captured: Jul 16 22:14:58.019538 ncdv-master-1 devstack@n-api.service[138680]: request: GET /compute/v2.1/servers/41010ce2-2848-458a-b6da-5d2ac455bcc0 HTTP/1.1 Jul 16 22:14:58.019538 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 22:14:58.019538 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 22:14:58.019538 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 22:14:58.019538 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 0 Jul 16 22:14:58.019538 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 22:14:58.019538 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 22:14:58.019538 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 22:14:58.019538 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 22:14:58.019538 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0zbe3rCsn-3QaicbUyd_l2yeeYdFw5j3pI0YNVDnyDW1adHPbvFyygFfhKMkLZKzSd5wCIHBh53_Ahvryl0lX_fYnOhaAfRD_aC5xNe-jnygMGiPOaIiG3WVsRGIF7WYX8xbkLiWHdzOeuosVkQ9hfVKA2k7v4UqqZHZ5ysM0snBRFk Jul 16 22:14:58.019538 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 22:14:58.019538 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 22:14:58.019538 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 22:14:58.019538 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 22:14:58.019538 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 22:14:58.019538 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 22:14:58.019538 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: deb1e913bedd4fb5b286e21c60e27a41 Jul 16 22:14:58.019538 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-TestMinimumBasicScenario-1144486311 Jul 16 22:14:58.019538 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 22:14:58.019538 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 22:14:58.019538 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/deb1e913bedd4fb5b286e21c60e27a41"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/deb1e913bedd4fb5b286e21c60e27a41"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_deb1e913bedd4fb5b286e21c60e27a41"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/deb1e913bedd4fb5b286e21c60e27a41"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 22:14:58.019538 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-TestMinimumBasicScenario-1144486311 Jul 16 22:14:58.019538 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: deb1e913bedd4fb5b286e21c60e27a41 Jul 16 22:14:58.019538 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-TestMinimumBasicScenario-1144486311 Jul 16 22:14:58.019538 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-TestMinimumBasicScenario-1144486311-project Jul 16 22:14:58.019538 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 22:14:58.019538 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 22:14:58.019538 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: d749cd77cf5c489e8c4e57e02ce768fd Jul 16 22:14:58.019538 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-TestMinimumBasicScenario-1144486311-project Jul 16 22:14:58.019538 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 22:14:58.019538 ncdv-master-1 devstack@n-api.service[138680]: exception: Instance 41010ce2-2848-458a-b6da-5d2ac455bcc0 could not be found. Jul 16 22:14:58.019538 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': '41010ce2-2848-458a-b6da-5d2ac455bcc0'} Jul 16 22:14:58.019538 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 22:14:58.019538 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 22:14:58.019538 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 22:14:58.019538 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 22:14:58.019538 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 22:14:58.019538 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 22:14:58.019538 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 22:14:58.019538 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 22:14:58.019538 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 22:14:58.019538 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 22:14:58.019538 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 41010ce2-2848-458a-b6da-5d2ac455bcc0 could not be found. Jul 16 22:14:58.019538 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 22:14:58.019538 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 22:14:58.019538 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 22:14:58.019538 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 22:14:58.019538 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 22:14:58.019538 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 22:14:58.019538 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 22:14:58.019538 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 22:14:58.019538 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 22:14:58.019538 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 22:14:58.019538 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 22:14:58.019538 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 22:14:58.019538 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 22:14:58.019538 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 22:14:58.019538 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 22:14:58.019538 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 22:14:58.019538 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 41010ce2-2848-458a-b6da-5d2ac455bcc0 could not be found. Jul 16 22:14:58.019538 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 22:14:58.019538 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 22:14:58.019538 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 22:14:58.026697 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 22:14:58.026697 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 22:14:58.026697 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 22:14:58.026697 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 22:14:58.026697 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 22:14:58.026697 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 41010ce2-2848-458a-b6da-5d2ac455bcc0 could not be found. Jul 16 22:14:58.026697 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 22:14:58.026697 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-0b276d40-4381-401a-bf3a-ce9290bf49f7 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Returning 404 to user: Instance 41010ce2-2848-458a-b6da-5d2ac455bcc0 could not be found. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 22:14:58.026697 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-0b276d40-4381-401a-bf3a-ce9290bf49f7 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] 10.222.0.16 "GET /compute/v2.1/servers/41010ce2-2848-458a-b6da-5d2ac455bcc0" status: 404 len: 111 microversion: 2.1 time: 0.030998 Jul 16 22:14:58.026697 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3442/6891] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:14:57 2022] GET /compute/v2.1/servers/41010ce2-2848-458a-b6da-5d2ac455bcc0 => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 22:14:58.029861 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-96f6b4b0-45b9-4a52-8d6f-fa0248bc3082 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] Calling method '.version_select at 0x7fdd9d257f70>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:14:58.135397 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-96f6b4b0-45b9-4a52-8d6f-fa0248bc3082 tempest-TestMinimumBasicScenario-1144486311 tempest-TestMinimumBasicScenario-1144486311-project] 10.222.0.16 "DELETE /compute/v2.1/os-keypairs/tempest-TestMinimumBasicScenario-772695359" status: 202 len: 0 microversion: 2.1 time: 0.109325 Jul 16 22:14:58.135929 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3450/6892] 10.222.0.16 () {58 vars in 1268 bytes} [Sat Jul 16 22:14:58 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestMinimumBasicScenario-772695359 => generated 0 bytes in 111 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 22:15:10.084193 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_api:SELECT=14 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:15:10.085053 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=31,nova_cell1:INSERT=1 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:15:10.253013 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-83ddfbd7-b4cb-4700-8b58-c5ba61c3ecd6 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Action: 'create', calling method: .version_select at 0x7fdd8fb391f0>, body: {"keypair": {"name": "tempest-TestNetworkAdvancedServerOps-202973646", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBFExzX4xA6RcPSrSWEbQw9lxAEB+oAOyWLZeRLYqj9N2pNXcsHmOAU8nAyM85DRUdpP78c4tqzQwydBcr05UEbIqfJpYBEOfOzBMYwQQnNHmgHSyHyLp4FgllUZNFjjm6w=="}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:15:10.266929 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-83ddfbd7-b4cb-4700-8b58-c5ba61c3ecd6 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Getting quotas for project a45a8ce6687c427d8f76e4205a076a93. Resources: {'key_pairs'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 22:15:10.272088 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-83ddfbd7-b4cb-4700-8b58-c5ba61c3ecd6 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Getting quotas for user 4b8c7ad3cc6240c096c2c7cf31d8f74f and project a45a8ce6687c427d8f76e4205a076a93. Resources: {'key_pairs'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 22:15:10.375482 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-83ddfbd7-b4cb-4700-8b58-c5ba61c3ecd6 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "POST /compute/v2.1/os-keypairs" status: 200 len: 406 microversion: 2.1 time: 0.297045 Jul 16 22:15:10.376390 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3443/6893] 10.222.0.16 () {60 vars in 1158 bytes} [Sat Jul 16 22:15:10 2022] POST /compute/v2.1/os-keypairs => generated 406 bytes in 299 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 22:15:18.368649 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=42,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:15:18.370103 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_api:SELECT=22,nova_api:UPDATE=1,nova_api:DELETE=1 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:15:18.376172 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-9d8ec619-2c07-4224-9421-d394f6d9d533 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestNetworkAdvancedServerOps-server-524887633", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452", "networks": [{"uuid": "10dfd059-9ae6-4cbd-a824-df6a79b3db56"}], "key_name": "tempest-TestNetworkAdvancedServerOps-202973646", "security_groups": [{"name": "tempest-secgroup-smoke-1937894983"}]}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:15:18.497238 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.network.neutron [None req-9d8ec619-2c07-4224-9421-d394f6d9d533 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] validate_networks() for [('10dfd059-9ae6-4cbd-a824-df6a79b3db56', None, None, None, None, None)] {{(pid=138681) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 16 22:15:19.025071 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-9d8ec619-2c07-4224-9421-d394f6d9d533 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Flavor limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 16 22:15:19.025509 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-9d8ec619-2c07-4224-9421-d394f6d9d533 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Image limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 16 22:15:19.025998 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-9d8ec619-2c07-4224-9421-d394f6d9d533 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Flavor pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 16 22:15:19.026422 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-9d8ec619-2c07-4224-9421-d394f6d9d533 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Image pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 16 22:15:19.026847 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-9d8ec619-2c07-4224-9421-d394f6d9d533 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 16 22:15:19.039320 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9d8ec619-2c07-4224-9421-d394f6d9d533 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:15:19.039320 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9d8ec619-2c07-4224-9421-d394f6d9d533 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:15:19.040100 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9d8ec619-2c07-4224-9421-d394f6d9d533 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:15:19.040873 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9d8ec619-2c07-4224-9421-d394f6d9d533 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:15:19.074257 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-9d8ec619-2c07-4224-9421-d394f6d9d533 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Getting quotas for project a45a8ce6687c427d8f76e4205a076a93. Resources: {'ram', 'instances', 'cores'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 22:15:19.082294 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-9d8ec619-2c07-4224-9421-d394f6d9d533 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Getting quotas for user 4b8c7ad3cc6240c096c2c7cf31d8f74f and project a45a8ce6687c427d8f76e4205a076a93. Resources: {'ram', 'instances', 'cores'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 22:15:19.099226 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-9d8ec619-2c07-4224-9421-d394f6d9d533 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Going to run 1 instances... {{(pid=138681) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 16 22:15:19.101927 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-9d8ec619-2c07-4224-9421-d394f6d9d533 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] [instance: 8f799adb-6c06-4b1f-88ea-318088c8ede0] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138681) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 16 22:15:19.222885 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-9d8ec619-2c07-4224-9421-d394f6d9d533 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 406 microversion: 2.1 time: 0.855385 Jul 16 22:15:19.223930 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3451/6894] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 22:15:18 2022] POST /compute/v2.1/servers => generated 406 bytes in 858 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 16 22:15:19.235600 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-bd2a8422-6de9-4d07-ad2e-1430bbe7a6f3 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:15:19.236409 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-bd2a8422-6de9-4d07-ad2e-1430bbe7a6f3 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] [instance: 8f799adb-6c06-4b1f-88ea-318088c8ede0] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:15:19.306008 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-bd2a8422-6de9-4d07-ad2e-1430bbe7a6f3 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4b8c7ad3cc6240c096c2c7cf31d8f74f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a45a8ce6687c427d8f76e4205a076a93', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:15:19.320492 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-bd2a8422-6de9-4d07-ad2e-1430bbe7a6f3 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "GET /compute/v2.1/servers/8f799adb-6c06-4b1f-88ea-318088c8ede0" status: 200 len: 1265 microversion: 2.1 time: 0.090528 Jul 16 22:15:19.321266 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3444/6895] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:15:19 2022] GET /compute/v2.1/servers/8f799adb-6c06-4b1f-88ea-318088c8ede0 => generated 1265 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:15:20.340039 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-cb86da7c-e01d-4d62-aa53-8ba70a3fd825 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:15:20.340800 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-cb86da7c-e01d-4d62-aa53-8ba70a3fd825 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] [instance: 8f799adb-6c06-4b1f-88ea-318088c8ede0] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:15:20.431353 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-cb86da7c-e01d-4d62-aa53-8ba70a3fd825 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4b8c7ad3cc6240c096c2c7cf31d8f74f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a45a8ce6687c427d8f76e4205a076a93', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:15:20.439356 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-cb86da7c-e01d-4d62-aa53-8ba70a3fd825 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "GET /compute/v2.1/servers/8f799adb-6c06-4b1f-88ea-318088c8ede0" status: 200 len: 1265 microversion: 2.1 time: 0.104103 Jul 16 22:15:20.440600 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3452/6896] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:15:20 2022] GET /compute/v2.1/servers/8f799adb-6c06-4b1f-88ea-318088c8ede0 => generated 1265 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:15:21.457583 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-63804972-6139-4a04-88f9-5f9a69211ea8 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:15:21.458173 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-63804972-6139-4a04-88f9-5f9a69211ea8 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] [instance: 8f799adb-6c06-4b1f-88ea-318088c8ede0] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:15:21.466800 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-63804972-6139-4a04-88f9-5f9a69211ea8 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:15:21.467435 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-63804972-6139-4a04-88f9-5f9a69211ea8 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:15:21.468623 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-63804972-6139-4a04-88f9-5f9a69211ea8 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:15:21.469636 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-63804972-6139-4a04-88f9-5f9a69211ea8 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:15:21.545543 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-63804972-6139-4a04-88f9-5f9a69211ea8 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4b8c7ad3cc6240c096c2c7cf31d8f74f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a45a8ce6687c427d8f76e4205a076a93', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:15:21.554566 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-63804972-6139-4a04-88f9-5f9a69211ea8 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "GET /compute/v2.1/servers/8f799adb-6c06-4b1f-88ea-318088c8ede0" status: 200 len: 1325 microversion: 2.1 time: 0.101611 Jul 16 22:15:21.555164 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3445/6897] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:15:21 2022] GET /compute/v2.1/servers/8f799adb-6c06-4b1f-88ea-318088c8ede0 => generated 1325 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:15:22.572913 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-9f5b550d-5bff-4079-bac9-6051aee89268 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:15:22.573541 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-9f5b550d-5bff-4079-bac9-6051aee89268 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] [instance: 8f799adb-6c06-4b1f-88ea-318088c8ede0] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:15:22.581040 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9f5b550d-5bff-4079-bac9-6051aee89268 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:15:22.581516 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9f5b550d-5bff-4079-bac9-6051aee89268 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:15:22.582377 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9f5b550d-5bff-4079-bac9-6051aee89268 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:15:22.582834 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9f5b550d-5bff-4079-bac9-6051aee89268 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:15:22.731936 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-9f5b550d-5bff-4079-bac9-6051aee89268 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4b8c7ad3cc6240c096c2c7cf31d8f74f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a45a8ce6687c427d8f76e4205a076a93', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:15:22.741057 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-9f5b550d-5bff-4079-bac9-6051aee89268 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "GET /compute/v2.1/servers/8f799adb-6c06-4b1f-88ea-318088c8ede0" status: 200 len: 1391 microversion: 2.1 time: 0.172121 Jul 16 22:15:22.741735 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3453/6898] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:15:22 2022] GET /compute/v2.1/servers/8f799adb-6c06-4b1f-88ea-318088c8ede0 => generated 1391 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:15:23.567704 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-10b78f42-5226-4e49-bf1a-f2385aacdc0b req-8cdaea81-88e9-47b4-b723-1d48b97f8d02 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "8f799adb-6c06-4b1f-88ea-318088c8ede0", "tag": "159fb859-9182-4d67-ad75-d342c4afb984"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:15:23.576739 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-10b78f42-5226-4e49-bf1a-f2385aacdc0b req-8cdaea81-88e9-47b4-b723-1d48b97f8d02 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:15:23.577181 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-10b78f42-5226-4e49-bf1a-f2385aacdc0b req-8cdaea81-88e9-47b4-b723-1d48b97f8d02 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:15:23.605267 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-10b78f42-5226-4e49-bf1a-f2385aacdc0b req-8cdaea81-88e9-47b4-b723-1d48b97f8d02 service nova] Creating event network-changed:159fb859-9182-4d67-ad75-d342c4afb984 for instance 8f799adb-6c06-4b1f-88ea-318088c8ede0 on nch1-master-1 Jul 16 22:15:23.605267 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-10b78f42-5226-4e49-bf1a-f2385aacdc0b req-8cdaea81-88e9-47b4-b723-1d48b97f8d02 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.032554 Jul 16 22:15:23.605267 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3446/6899] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 22:15:23 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 22:15:23.759096 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-782b9f84-dcc9-433c-b935-019d209a3ee5 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:15:23.760549 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-782b9f84-dcc9-433c-b935-019d209a3ee5 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] [instance: 8f799adb-6c06-4b1f-88ea-318088c8ede0] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:15:23.769489 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-782b9f84-dcc9-433c-b935-019d209a3ee5 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:15:23.770013 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-782b9f84-dcc9-433c-b935-019d209a3ee5 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:15:23.770861 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-782b9f84-dcc9-433c-b935-019d209a3ee5 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:15:23.771350 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-782b9f84-dcc9-433c-b935-019d209a3ee5 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:15:23.921710 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-782b9f84-dcc9-433c-b935-019d209a3ee5 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4b8c7ad3cc6240c096c2c7cf31d8f74f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a45a8ce6687c427d8f76e4205a076a93', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:15:23.930737 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-782b9f84-dcc9-433c-b935-019d209a3ee5 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "GET /compute/v2.1/servers/8f799adb-6c06-4b1f-88ea-318088c8ede0" status: 200 len: 1391 microversion: 2.1 time: 0.175520 Jul 16 22:15:23.931475 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3454/6900] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:15:23 2022] GET /compute/v2.1/servers/8f799adb-6c06-4b1f-88ea-318088c8ede0 => generated 1391 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 22:15:24.950053 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-7eaf353a-6f1f-474c-9eea-163f826f0d1e tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:15:24.950759 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-7eaf353a-6f1f-474c-9eea-163f826f0d1e tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] [instance: 8f799adb-6c06-4b1f-88ea-318088c8ede0] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:15:24.957344 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7eaf353a-6f1f-474c-9eea-163f826f0d1e tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:15:24.957805 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7eaf353a-6f1f-474c-9eea-163f826f0d1e tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:15:24.958608 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7eaf353a-6f1f-474c-9eea-163f826f0d1e tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:15:24.959073 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7eaf353a-6f1f-474c-9eea-163f826f0d1e tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:15:25.095812 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-7eaf353a-6f1f-474c-9eea-163f826f0d1e tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4b8c7ad3cc6240c096c2c7cf31d8f74f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a45a8ce6687c427d8f76e4205a076a93', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:15:25.103269 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-7eaf353a-6f1f-474c-9eea-163f826f0d1e tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "GET /compute/v2.1/servers/8f799adb-6c06-4b1f-88ea-318088c8ede0" status: 200 len: 1391 microversion: 2.1 time: 0.158347 Jul 16 22:15:25.103841 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3447/6901] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:15:24 2022] GET /compute/v2.1/servers/8f799adb-6c06-4b1f-88ea-318088c8ede0 => generated 1391 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:15:26.121176 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-24266182-80da-4dcc-b797-18420f1e4ba7 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:15:26.121713 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-24266182-80da-4dcc-b797-18420f1e4ba7 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] [instance: 8f799adb-6c06-4b1f-88ea-318088c8ede0] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:15:26.129259 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-24266182-80da-4dcc-b797-18420f1e4ba7 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:15:26.129727 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-24266182-80da-4dcc-b797-18420f1e4ba7 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:15:26.130532 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-24266182-80da-4dcc-b797-18420f1e4ba7 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:15:26.130977 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-24266182-80da-4dcc-b797-18420f1e4ba7 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:15:26.270411 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-24266182-80da-4dcc-b797-18420f1e4ba7 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4b8c7ad3cc6240c096c2c7cf31d8f74f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a45a8ce6687c427d8f76e4205a076a93', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:15:26.278339 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-24266182-80da-4dcc-b797-18420f1e4ba7 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "GET /compute/v2.1/servers/8f799adb-6c06-4b1f-88ea-318088c8ede0" status: 200 len: 1391 microversion: 2.1 time: 0.161198 Jul 16 22:15:26.279291 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3455/6902] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:15:26 2022] GET /compute/v2.1/servers/8f799adb-6c06-4b1f-88ea-318088c8ede0 => generated 1391 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:15:27.295812 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-92240d0a-a6fd-48d2-bf5a-e6c80b95b5e1 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:15:27.296432 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-92240d0a-a6fd-48d2-bf5a-e6c80b95b5e1 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] [instance: 8f799adb-6c06-4b1f-88ea-318088c8ede0] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:15:27.303417 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-92240d0a-a6fd-48d2-bf5a-e6c80b95b5e1 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:15:27.303881 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-92240d0a-a6fd-48d2-bf5a-e6c80b95b5e1 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:15:27.304682 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-92240d0a-a6fd-48d2-bf5a-e6c80b95b5e1 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:15:27.305131 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-92240d0a-a6fd-48d2-bf5a-e6c80b95b5e1 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:15:27.446288 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-92240d0a-a6fd-48d2-bf5a-e6c80b95b5e1 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4b8c7ad3cc6240c096c2c7cf31d8f74f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a45a8ce6687c427d8f76e4205a076a93', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:15:27.454952 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-92240d0a-a6fd-48d2-bf5a-e6c80b95b5e1 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "GET /compute/v2.1/servers/8f799adb-6c06-4b1f-88ea-318088c8ede0" status: 200 len: 1391 microversion: 2.1 time: 0.162931 Jul 16 22:15:27.455762 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3448/6903] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:15:27 2022] GET /compute/v2.1/servers/8f799adb-6c06-4b1f-88ea-318088c8ede0 => generated 1391 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:15:28.478354 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-9c3dcefe-0c4e-4194-b9de-b5bc9a4db5f4 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:15:28.478993 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-9c3dcefe-0c4e-4194-b9de-b5bc9a4db5f4 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] [instance: 8f799adb-6c06-4b1f-88ea-318088c8ede0] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:15:28.486852 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9c3dcefe-0c4e-4194-b9de-b5bc9a4db5f4 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:15:28.487360 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9c3dcefe-0c4e-4194-b9de-b5bc9a4db5f4 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:15:28.488153 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9c3dcefe-0c4e-4194-b9de-b5bc9a4db5f4 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:15:28.488614 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9c3dcefe-0c4e-4194-b9de-b5bc9a4db5f4 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:15:28.621606 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-9c3dcefe-0c4e-4194-b9de-b5bc9a4db5f4 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4b8c7ad3cc6240c096c2c7cf31d8f74f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a45a8ce6687c427d8f76e4205a076a93', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:15:28.629359 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-9c3dcefe-0c4e-4194-b9de-b5bc9a4db5f4 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "GET /compute/v2.1/servers/8f799adb-6c06-4b1f-88ea-318088c8ede0" status: 200 len: 1391 microversion: 2.1 time: 0.154957 Jul 16 22:15:28.629957 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3456/6904] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:15:28 2022] GET /compute/v2.1/servers/8f799adb-6c06-4b1f-88ea-318088c8ede0 => generated 1391 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:15:29.521975 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-f7af7192-8253-4fe8-8975-44124a3ab5b6 req-b2b9e439-f595-4f1f-952f-efadfafb7fa0 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "8f799adb-6c06-4b1f-88ea-318088c8ede0", "name": "network-vif-plugged", "status": "completed", "tag": "159fb859-9182-4d67-ad75-d342c4afb984"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:15:29.526427 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell0:SELECT=6 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:15:29.531903 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-f7af7192-8253-4fe8-8975-44124a3ab5b6 req-b2b9e439-f595-4f1f-952f-efadfafb7fa0 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:15:29.532392 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-f7af7192-8253-4fe8-8975-44124a3ab5b6 req-b2b9e439-f595-4f1f-952f-efadfafb7fa0 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:15:29.543462 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-f7af7192-8253-4fe8-8975-44124a3ab5b6 req-b2b9e439-f595-4f1f-952f-efadfafb7fa0 service nova] Creating event network-vif-plugged:159fb859-9182-4d67-ad75-d342c4afb984 for instance 8f799adb-6c06-4b1f-88ea-318088c8ede0 on nch1-master-1 Jul 16 22:15:29.549173 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-f7af7192-8253-4fe8-8975-44124a3ab5b6 req-b2b9e439-f595-4f1f-952f-efadfafb7fa0 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.031083 Jul 16 22:15:29.549816 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3449/6905] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 22:15:29 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 22:15:29.645894 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-86903e11-c5d2-4e39-bc44-f2c83082b9ee tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:15:29.647325 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-86903e11-c5d2-4e39-bc44-f2c83082b9ee tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] [instance: 8f799adb-6c06-4b1f-88ea-318088c8ede0] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:15:29.649359 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell0:SELECT=2 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:15:29.655724 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-86903e11-c5d2-4e39-bc44-f2c83082b9ee tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:15:29.656769 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-86903e11-c5d2-4e39-bc44-f2c83082b9ee tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:15:29.657682 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-86903e11-c5d2-4e39-bc44-f2c83082b9ee tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:15:29.658270 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-86903e11-c5d2-4e39-bc44-f2c83082b9ee tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:15:29.802406 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-86903e11-c5d2-4e39-bc44-f2c83082b9ee tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4b8c7ad3cc6240c096c2c7cf31d8f74f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a45a8ce6687c427d8f76e4205a076a93', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:15:29.810019 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-86903e11-c5d2-4e39-bc44-f2c83082b9ee tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "GET /compute/v2.1/servers/8f799adb-6c06-4b1f-88ea-318088c8ede0" status: 200 len: 1391 microversion: 2.1 time: 0.168080 Jul 16 22:15:29.810608 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3457/6906] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:15:29 2022] GET /compute/v2.1/servers/8f799adb-6c06-4b1f-88ea-318088c8ede0 => generated 1391 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:15:30.829566 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-dae9c24a-4c0a-4d99-b873-95e6dc72c97b tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:15:30.830279 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-dae9c24a-4c0a-4d99-b873-95e6dc72c97b tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] [instance: 8f799adb-6c06-4b1f-88ea-318088c8ede0] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:15:30.837140 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-dae9c24a-4c0a-4d99-b873-95e6dc72c97b tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:15:30.838126 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-dae9c24a-4c0a-4d99-b873-95e6dc72c97b tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:15:30.840730 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-dae9c24a-4c0a-4d99-b873-95e6dc72c97b tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:15:30.840730 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-dae9c24a-4c0a-4d99-b873-95e6dc72c97b tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:15:30.980852 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-dae9c24a-4c0a-4d99-b873-95e6dc72c97b tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4b8c7ad3cc6240c096c2c7cf31d8f74f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a45a8ce6687c427d8f76e4205a076a93', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:15:30.989211 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-dae9c24a-4c0a-4d99-b873-95e6dc72c97b tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "GET /compute/v2.1/servers/8f799adb-6c06-4b1f-88ea-318088c8ede0" status: 200 len: 1563 microversion: 2.1 time: 0.165491 Jul 16 22:15:30.989855 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3450/6907] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:15:30 2022] GET /compute/v2.1/servers/8f799adb-6c06-4b1f-88ea-318088c8ede0 => generated 1563 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:15:31.006169 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-0d175aff-d916-4dca-98d0-530c65fa55aa tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:15:31.006803 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-0d175aff-d916-4dca-98d0-530c65fa55aa tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] [instance: 8f799adb-6c06-4b1f-88ea-318088c8ede0] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:15:31.008715 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell0:SELECT=8 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:15:31.014483 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0d175aff-d916-4dca-98d0-530c65fa55aa tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:15:31.018418 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0d175aff-d916-4dca-98d0-530c65fa55aa tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:15:31.018418 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0d175aff-d916-4dca-98d0-530c65fa55aa tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:15:31.018418 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0d175aff-d916-4dca-98d0-530c65fa55aa tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:15:31.149298 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-0d175aff-d916-4dca-98d0-530c65fa55aa tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4b8c7ad3cc6240c096c2c7cf31d8f74f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a45a8ce6687c427d8f76e4205a076a93', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:15:31.157469 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-0d175aff-d916-4dca-98d0-530c65fa55aa tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "GET /compute/v2.1/servers/8f799adb-6c06-4b1f-88ea-318088c8ede0" status: 200 len: 1563 microversion: 2.1 time: 0.155193 Jul 16 22:15:31.158048 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3458/6908] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:15:31 2022] GET /compute/v2.1/servers/8f799adb-6c06-4b1f-88ea-318088c8ede0 => generated 1563 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:15:33.385175 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-14f033a7-46fa-4849-9ec3-0cec3388ce59 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:15:33.386683 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-14f033a7-46fa-4849-9ec3-0cec3388ce59 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] [instance: 8f799adb-6c06-4b1f-88ea-318088c8ede0] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:15:33.392295 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-19cd5f58-51de-4936-bf18-9f2978b17cf2 req-3ddf7343-02a6-4003-923a-603b25aea027 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "8f799adb-6c06-4b1f-88ea-318088c8ede0", "tag": "159fb859-9182-4d67-ad75-d342c4afb984"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:15:33.399168 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-14f033a7-46fa-4849-9ec3-0cec3388ce59 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:15:33.399632 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-14f033a7-46fa-4849-9ec3-0cec3388ce59 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:15:33.400416 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-14f033a7-46fa-4849-9ec3-0cec3388ce59 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:15:33.400858 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-14f033a7-46fa-4849-9ec3-0cec3388ce59 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:15:33.403395 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-19cd5f58-51de-4936-bf18-9f2978b17cf2 req-3ddf7343-02a6-4003-923a-603b25aea027 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:15:33.404133 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-19cd5f58-51de-4936-bf18-9f2978b17cf2 req-3ddf7343-02a6-4003-923a-603b25aea027 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:15:33.417292 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-19cd5f58-51de-4936-bf18-9f2978b17cf2 req-3ddf7343-02a6-4003-923a-603b25aea027 service nova] Creating event network-changed:159fb859-9182-4d67-ad75-d342c4afb984 for instance 8f799adb-6c06-4b1f-88ea-318088c8ede0 on nch1-master-1 Jul 16 22:15:33.423960 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-19cd5f58-51de-4936-bf18-9f2978b17cf2 req-3ddf7343-02a6-4003-923a-603b25aea027 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.035918 Jul 16 22:15:33.424826 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3459/6909] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 22:15:33 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 16 22:15:33.545069 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-14f033a7-46fa-4849-9ec3-0cec3388ce59 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4b8c7ad3cc6240c096c2c7cf31d8f74f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a45a8ce6687c427d8f76e4205a076a93', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:15:33.556835 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-14f033a7-46fa-4849-9ec3-0cec3388ce59 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "GET /compute/v2.1/servers/8f799adb-6c06-4b1f-88ea-318088c8ede0" status: 200 len: 1563 microversion: 2.1 time: 0.175487 Jul 16 22:15:33.560208 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3451/6910] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:15:33 2022] GET /compute/v2.1/servers/8f799adb-6c06-4b1f-88ea-318088c8ede0 => generated 1563 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:15:56.812484 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_api:SELECT=33,nova_api:INSERT=3 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:15:56.813745 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=40 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:15:56.818007 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-9d4d81d8-d641-414d-aa0e-68dccfc6e35a tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Action: 'action', calling method: >, body: {"os-getConsoleOutput": {}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:15:56.819060 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-9d4d81d8-d641-414d-aa0e-68dccfc6e35a tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] [instance: 8f799adb-6c06-4b1f-88ea-318088c8ede0] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:15:56.825528 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9d4d81d8-d641-414d-aa0e-68dccfc6e35a tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:15:56.825996 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9d4d81d8-d641-414d-aa0e-68dccfc6e35a tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:15:56.826825 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9d4d81d8-d641-414d-aa0e-68dccfc6e35a tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:15:56.827309 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9d4d81d8-d641-414d-aa0e-68dccfc6e35a tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:15:56.884042 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-9d4d81d8-d641-414d-aa0e-68dccfc6e35a tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "POST /compute/v2.1/servers/8f799adb-6c06-4b1f-88ea-318088c8ede0/action" status: 200 len: 24559 microversion: 2.1 time: 0.073476 Jul 16 22:15:56.884708 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3460/6911] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 22:15:56 2022] POST /compute/v2.1/servers/8f799adb-6c06-4b1f-88ea-318088c8ede0/action => generated 24559 bytes in 75 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 16 22:16:03.325756 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-1ce6bcfb-8504-440d-ac75-fdbffdc70786 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Action: 'action', calling method: >, body: {"pause": {}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:16:03.326351 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-1ce6bcfb-8504-440d-ac75-fdbffdc70786 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] [instance: 8f799adb-6c06-4b1f-88ea-318088c8ede0] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:16:03.328450 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_api:SELECT=29,nova_api:INSERT=1 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:16:03.329692 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=34 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:16:03.334436 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1ce6bcfb-8504-440d-ac75-fdbffdc70786 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:16:03.334895 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1ce6bcfb-8504-440d-ac75-fdbffdc70786 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:16:03.335695 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1ce6bcfb-8504-440d-ac75-fdbffdc70786 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:16:03.336142 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1ce6bcfb-8504-440d-ac75-fdbffdc70786 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:16:03.559204 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-1ce6bcfb-8504-440d-ac75-fdbffdc70786 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "POST /compute/v2.1/servers/8f799adb-6c06-4b1f-88ea-318088c8ede0/action" status: 202 len: 0 microversion: 2.1 time: 0.237376 Jul 16 22:16:03.559878 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3452/6912] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 22:16:03 2022] POST /compute/v2.1/servers/8f799adb-6c06-4b1f-88ea-318088c8ede0/action => generated 0 bytes in 239 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 22:16:03.570398 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-6c7cb63b-a8f4-4bc6-8598-0a588a3fb567 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:16:03.571145 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-6c7cb63b-a8f4-4bc6-8598-0a588a3fb567 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] [instance: 8f799adb-6c06-4b1f-88ea-318088c8ede0] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:16:03.580895 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6c7cb63b-a8f4-4bc6-8598-0a588a3fb567 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:16:03.580895 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6c7cb63b-a8f4-4bc6-8598-0a588a3fb567 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:16:03.580895 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6c7cb63b-a8f4-4bc6-8598-0a588a3fb567 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:16:03.580895 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6c7cb63b-a8f4-4bc6-8598-0a588a3fb567 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:16:03.721399 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-6c7cb63b-a8f4-4bc6-8598-0a588a3fb567 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4b8c7ad3cc6240c096c2c7cf31d8f74f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a45a8ce6687c427d8f76e4205a076a93', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:16:03.729066 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-6c7cb63b-a8f4-4bc6-8598-0a588a3fb567 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "GET /compute/v2.1/servers/8f799adb-6c06-4b1f-88ea-318088c8ede0" status: 200 len: 1686 microversion: 2.1 time: 0.163713 Jul 16 22:16:03.729646 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3461/6913] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:16:03 2022] GET /compute/v2.1/servers/8f799adb-6c06-4b1f-88ea-318088c8ede0 => generated 1686 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 22:16:04.747754 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-a2c3028b-cb85-445e-b579-63ec4e4c0155 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:16:04.748561 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-a2c3028b-cb85-445e-b579-63ec4e4c0155 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] [instance: 8f799adb-6c06-4b1f-88ea-318088c8ede0] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:16:04.756193 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a2c3028b-cb85-445e-b579-63ec4e4c0155 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:16:04.757330 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a2c3028b-cb85-445e-b579-63ec4e4c0155 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:16:04.758132 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a2c3028b-cb85-445e-b579-63ec4e4c0155 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:16:04.758655 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a2c3028b-cb85-445e-b579-63ec4e4c0155 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:16:04.896953 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-a2c3028b-cb85-445e-b579-63ec4e4c0155 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4b8c7ad3cc6240c096c2c7cf31d8f74f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a45a8ce6687c427d8f76e4205a076a93', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:16:04.904637 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-a2c3028b-cb85-445e-b579-63ec4e4c0155 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "GET /compute/v2.1/servers/8f799adb-6c06-4b1f-88ea-318088c8ede0" status: 200 len: 1666 microversion: 2.1 time: 0.161314 Jul 16 22:16:04.905210 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3453/6914] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:16:04 2022] GET /compute/v2.1/servers/8f799adb-6c06-4b1f-88ea-318088c8ede0 => generated 1666 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:16:05.992624 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-9b45cfd8-5f4c-4178-afb8-9aee00fc3ffb tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Action: 'action', calling method: >, body: {"os-getConsoleOutput": {}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:16:05.993575 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-9b45cfd8-5f4c-4178-afb8-9aee00fc3ffb tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] [instance: 8f799adb-6c06-4b1f-88ea-318088c8ede0] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:16:06.001695 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9b45cfd8-5f4c-4178-afb8-9aee00fc3ffb tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:16:06.002167 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9b45cfd8-5f4c-4178-afb8-9aee00fc3ffb tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:16:06.002998 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9b45cfd8-5f4c-4178-afb8-9aee00fc3ffb tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:16:06.003472 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9b45cfd8-5f4c-4178-afb8-9aee00fc3ffb tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:16:06.053506 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-9b45cfd8-5f4c-4178-afb8-9aee00fc3ffb tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "POST /compute/v2.1/servers/8f799adb-6c06-4b1f-88ea-318088c8ede0/action" status: 200 len: 32631 microversion: 2.1 time: 0.064564 Jul 16 22:16:06.054169 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3462/6915] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 22:16:05 2022] POST /compute/v2.1/servers/8f799adb-6c06-4b1f-88ea-318088c8ede0/action => generated 32631 bytes in 66 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 16 22:16:06.066009 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-2510ac4f-b433-449f-89dc-efd1dd43cbbb tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Action: 'action', calling method: >, body: {"unpause": {}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:16:06.066542 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-2510ac4f-b433-449f-89dc-efd1dd43cbbb tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] [instance: 8f799adb-6c06-4b1f-88ea-318088c8ede0] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:16:06.073518 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2510ac4f-b433-449f-89dc-efd1dd43cbbb tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:16:06.073974 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2510ac4f-b433-449f-89dc-efd1dd43cbbb tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:16:06.074737 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2510ac4f-b433-449f-89dc-efd1dd43cbbb tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:16:06.075205 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2510ac4f-b433-449f-89dc-efd1dd43cbbb tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:16:06.285236 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-2510ac4f-b433-449f-89dc-efd1dd43cbbb tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "POST /compute/v2.1/servers/8f799adb-6c06-4b1f-88ea-318088c8ede0/action" status: 202 len: 0 microversion: 2.1 time: 0.223858 Jul 16 22:16:06.285880 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3454/6916] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 22:16:06 2022] POST /compute/v2.1/servers/8f799adb-6c06-4b1f-88ea-318088c8ede0/action => generated 0 bytes in 225 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Jul 16 22:16:06.294890 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-8728d626-85e9-4fef-84c8-506f79072958 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:16:06.295659 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-8728d626-85e9-4fef-84c8-506f79072958 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] [instance: 8f799adb-6c06-4b1f-88ea-318088c8ede0] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:16:06.302827 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8728d626-85e9-4fef-84c8-506f79072958 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:16:06.303337 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8728d626-85e9-4fef-84c8-506f79072958 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:16:06.304125 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8728d626-85e9-4fef-84c8-506f79072958 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:16:06.304789 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8728d626-85e9-4fef-84c8-506f79072958 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:16:06.446638 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-8728d626-85e9-4fef-84c8-506f79072958 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4b8c7ad3cc6240c096c2c7cf31d8f74f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a45a8ce6687c427d8f76e4205a076a93', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:16:06.454515 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-8728d626-85e9-4fef-84c8-506f79072958 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "GET /compute/v2.1/servers/8f799adb-6c06-4b1f-88ea-318088c8ede0" status: 200 len: 1673 microversion: 2.1 time: 0.163969 Jul 16 22:16:06.455127 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3463/6917] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:16:06 2022] GET /compute/v2.1/servers/8f799adb-6c06-4b1f-88ea-318088c8ede0 => generated 1673 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:16:07.473153 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-425289a1-2e36-4ee8-bd9a-f9f7a020f16d tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:16:07.473876 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-425289a1-2e36-4ee8-bd9a-f9f7a020f16d tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] [instance: 8f799adb-6c06-4b1f-88ea-318088c8ede0] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:16:07.480227 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-425289a1-2e36-4ee8-bd9a-f9f7a020f16d tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:16:07.480683 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-425289a1-2e36-4ee8-bd9a-f9f7a020f16d tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:16:07.481452 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-425289a1-2e36-4ee8-bd9a-f9f7a020f16d tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:16:07.481905 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-425289a1-2e36-4ee8-bd9a-f9f7a020f16d tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:16:07.614770 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-425289a1-2e36-4ee8-bd9a-f9f7a020f16d tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4b8c7ad3cc6240c096c2c7cf31d8f74f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a45a8ce6687c427d8f76e4205a076a93', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:16:07.622257 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-425289a1-2e36-4ee8-bd9a-f9f7a020f16d tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "GET /compute/v2.1/servers/8f799adb-6c06-4b1f-88ea-318088c8ede0" status: 200 len: 1673 microversion: 2.1 time: 0.154236 Jul 16 22:16:07.622832 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3455/6918] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:16:07 2022] GET /compute/v2.1/servers/8f799adb-6c06-4b1f-88ea-318088c8ede0 => generated 1673 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:16:08.640372 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-45b3f2e8-7ac1-4ffb-a32c-b4b6f3aa6722 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:16:08.640933 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-45b3f2e8-7ac1-4ffb-a32c-b4b6f3aa6722 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] [instance: 8f799adb-6c06-4b1f-88ea-318088c8ede0] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:16:08.652427 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-45b3f2e8-7ac1-4ffb-a32c-b4b6f3aa6722 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:16:08.652896 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-45b3f2e8-7ac1-4ffb-a32c-b4b6f3aa6722 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:16:08.653686 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-45b3f2e8-7ac1-4ffb-a32c-b4b6f3aa6722 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:16:08.654143 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-45b3f2e8-7ac1-4ffb-a32c-b4b6f3aa6722 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:16:08.789868 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-45b3f2e8-7ac1-4ffb-a32c-b4b6f3aa6722 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4b8c7ad3cc6240c096c2c7cf31d8f74f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a45a8ce6687c427d8f76e4205a076a93', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:16:08.797706 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-45b3f2e8-7ac1-4ffb-a32c-b4b6f3aa6722 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "GET /compute/v2.1/servers/8f799adb-6c06-4b1f-88ea-318088c8ede0" status: 200 len: 1673 microversion: 2.1 time: 0.161435 Jul 16 22:16:08.798288 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3464/6919] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:16:08 2022] GET /compute/v2.1/servers/8f799adb-6c06-4b1f-88ea-318088c8ede0 => generated 1673 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:16:09.815884 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-1db4425a-a2c0-42a2-8278-f2f7992113b0 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:16:09.816499 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-1db4425a-a2c0-42a2-8278-f2f7992113b0 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] [instance: 8f799adb-6c06-4b1f-88ea-318088c8ede0] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:16:09.823465 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1db4425a-a2c0-42a2-8278-f2f7992113b0 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:16:09.823919 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1db4425a-a2c0-42a2-8278-f2f7992113b0 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:16:09.824684 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1db4425a-a2c0-42a2-8278-f2f7992113b0 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:16:09.825126 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1db4425a-a2c0-42a2-8278-f2f7992113b0 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:16:09.960548 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-1db4425a-a2c0-42a2-8278-f2f7992113b0 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4b8c7ad3cc6240c096c2c7cf31d8f74f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a45a8ce6687c427d8f76e4205a076a93', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:16:09.968282 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-1db4425a-a2c0-42a2-8278-f2f7992113b0 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "GET /compute/v2.1/servers/8f799adb-6c06-4b1f-88ea-318088c8ede0" status: 200 len: 1673 microversion: 2.1 time: 0.157505 Jul 16 22:16:09.968860 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3456/6920] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:16:09 2022] GET /compute/v2.1/servers/8f799adb-6c06-4b1f-88ea-318088c8ede0 => generated 1673 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:16:10.986713 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-952c33a2-6b3f-43c8-9df6-b97134aaf1cf tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:16:10.987366 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-952c33a2-6b3f-43c8-9df6-b97134aaf1cf tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] [instance: 8f799adb-6c06-4b1f-88ea-318088c8ede0] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:16:10.994337 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-952c33a2-6b3f-43c8-9df6-b97134aaf1cf tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:16:10.994805 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-952c33a2-6b3f-43c8-9df6-b97134aaf1cf tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:16:10.995611 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-952c33a2-6b3f-43c8-9df6-b97134aaf1cf tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:16:10.996071 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-952c33a2-6b3f-43c8-9df6-b97134aaf1cf tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:16:11.128462 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-952c33a2-6b3f-43c8-9df6-b97134aaf1cf tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4b8c7ad3cc6240c096c2c7cf31d8f74f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a45a8ce6687c427d8f76e4205a076a93', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:16:11.136045 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-952c33a2-6b3f-43c8-9df6-b97134aaf1cf tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "GET /compute/v2.1/servers/8f799adb-6c06-4b1f-88ea-318088c8ede0" status: 200 len: 1673 microversion: 2.1 time: 0.153369 Jul 16 22:16:11.136625 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3465/6921] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:16:10 2022] GET /compute/v2.1/servers/8f799adb-6c06-4b1f-88ea-318088c8ede0 => generated 1673 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:16:12.153735 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-8181c489-b03a-473e-8c75-6011408bd99e tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:16:12.154271 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-8181c489-b03a-473e-8c75-6011408bd99e tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] [instance: 8f799adb-6c06-4b1f-88ea-318088c8ede0] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:16:12.162005 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8181c489-b03a-473e-8c75-6011408bd99e tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:16:12.162458 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8181c489-b03a-473e-8c75-6011408bd99e tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:16:12.163252 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8181c489-b03a-473e-8c75-6011408bd99e tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:16:12.163688 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8181c489-b03a-473e-8c75-6011408bd99e tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:16:12.298442 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-8181c489-b03a-473e-8c75-6011408bd99e tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4b8c7ad3cc6240c096c2c7cf31d8f74f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a45a8ce6687c427d8f76e4205a076a93', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:16:12.306120 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-8181c489-b03a-473e-8c75-6011408bd99e tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "GET /compute/v2.1/servers/8f799adb-6c06-4b1f-88ea-318088c8ede0" status: 200 len: 1681 microversion: 2.1 time: 0.156340 Jul 16 22:16:12.308920 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3457/6922] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:16:12 2022] GET /compute/v2.1/servers/8f799adb-6c06-4b1f-88ea-318088c8ede0 => generated 1681 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:16:12.397316 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-a0bc7a69-fc57-404d-a3d5-aa2159e0b883 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Action: 'action', calling method: >, body: {"os-getConsoleOutput": {}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:16:12.398274 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-a0bc7a69-fc57-404d-a3d5-aa2159e0b883 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] [instance: 8f799adb-6c06-4b1f-88ea-318088c8ede0] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:16:12.405555 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a0bc7a69-fc57-404d-a3d5-aa2159e0b883 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:16:12.405949 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a0bc7a69-fc57-404d-a3d5-aa2159e0b883 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:16:12.406841 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a0bc7a69-fc57-404d-a3d5-aa2159e0b883 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:16:12.407344 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a0bc7a69-fc57-404d-a3d5-aa2159e0b883 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:16:12.459583 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-a0bc7a69-fc57-404d-a3d5-aa2159e0b883 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "POST /compute/v2.1/servers/8f799adb-6c06-4b1f-88ea-318088c8ede0/action" status: 200 len: 38438 microversion: 2.1 time: 0.066387 Jul 16 22:16:12.460230 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3466/6923] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 22:16:12 2022] POST /compute/v2.1/servers/8f799adb-6c06-4b1f-88ea-318088c8ede0/action => generated 38438 bytes in 68 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 16 22:16:13.503297 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-1a9ee495-432e-4a6d-9aca-5442bfd8f1bc tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:16:13.503807 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-1a9ee495-432e-4a6d-9aca-5442bfd8f1bc tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] [instance: 8f799adb-6c06-4b1f-88ea-318088c8ede0] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:16:13.512605 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-47c88a29-87b1-424a-aa32-1672cae6e919 req-def0da51-ccd0-4279-870a-f4b875c4985d service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "8f799adb-6c06-4b1f-88ea-318088c8ede0", "tag": "159fb859-9182-4d67-ad75-d342c4afb984"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:16:13.522187 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1a9ee495-432e-4a6d-9aca-5442bfd8f1bc tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:16:13.522481 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-47c88a29-87b1-424a-aa32-1672cae6e919 req-def0da51-ccd0-4279-870a-f4b875c4985d service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:16:13.522798 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-47c88a29-87b1-424a-aa32-1672cae6e919 req-def0da51-ccd0-4279-870a-f4b875c4985d service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:16:13.523029 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1a9ee495-432e-4a6d-9aca-5442bfd8f1bc tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:16:13.523972 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1a9ee495-432e-4a6d-9aca-5442bfd8f1bc tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:16:13.524504 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1a9ee495-432e-4a6d-9aca-5442bfd8f1bc tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:16:13.534702 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-47c88a29-87b1-424a-aa32-1672cae6e919 req-def0da51-ccd0-4279-870a-f4b875c4985d service nova] Creating event network-changed:159fb859-9182-4d67-ad75-d342c4afb984 for instance 8f799adb-6c06-4b1f-88ea-318088c8ede0 on nch1-master-1 Jul 16 22:16:13.539734 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-47c88a29-87b1-424a-aa32-1672cae6e919 req-def0da51-ccd0-4279-870a-f4b875c4985d service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.031228 Jul 16 22:16:13.540339 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3467/6924] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 22:16:13 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 22:16:13.557917 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-1a9ee495-432e-4a6d-9aca-5442bfd8f1bc tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] [instance: 8f799adb-6c06-4b1f-88ea-318088c8ede0] Going to try to terminate instance {{(pid=138680) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 16 22:16:13.925292 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-1a9ee495-432e-4a6d-9aca-5442bfd8f1bc tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "DELETE /compute/v2.1/servers/8f799adb-6c06-4b1f-88ea-318088c8ede0" status: 204 len: 0 microversion: 2.1 time: 0.426226 Jul 16 22:16:13.926545 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3458/6925] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 22:16:13 2022] DELETE /compute/v2.1/servers/8f799adb-6c06-4b1f-88ea-318088c8ede0 => generated 0 bytes in 428 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 16 22:16:13.939497 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-5fc81fcf-7097-4146-ae5e-bdb6aee58962 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:16:13.940689 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-5fc81fcf-7097-4146-ae5e-bdb6aee58962 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] [instance: 8f799adb-6c06-4b1f-88ea-318088c8ede0] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:16:13.947453 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5fc81fcf-7097-4146-ae5e-bdb6aee58962 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:16:13.947929 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5fc81fcf-7097-4146-ae5e-bdb6aee58962 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:16:13.948709 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5fc81fcf-7097-4146-ae5e-bdb6aee58962 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:16:13.949343 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5fc81fcf-7097-4146-ae5e-bdb6aee58962 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:16:14.093269 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-5fc81fcf-7097-4146-ae5e-bdb6aee58962 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4b8c7ad3cc6240c096c2c7cf31d8f74f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a45a8ce6687c427d8f76e4205a076a93', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:16:14.101556 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-5fc81fcf-7097-4146-ae5e-bdb6aee58962 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "GET /compute/v2.1/servers/8f799adb-6c06-4b1f-88ea-318088c8ede0" status: 200 len: 1687 microversion: 2.1 time: 0.170075 Jul 16 22:16:14.102147 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3468/6926] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:16:13 2022] GET /compute/v2.1/servers/8f799adb-6c06-4b1f-88ea-318088c8ede0 => generated 1687 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:16:15.120284 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-a1271646-7ccf-43ad-9365-a1d31fa8463e tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:16:15.120990 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-a1271646-7ccf-43ad-9365-a1d31fa8463e tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] [instance: 8f799adb-6c06-4b1f-88ea-318088c8ede0] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:16:15.128117 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a1271646-7ccf-43ad-9365-a1d31fa8463e tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:16:15.128569 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a1271646-7ccf-43ad-9365-a1d31fa8463e tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:16:15.129332 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a1271646-7ccf-43ad-9365-a1d31fa8463e tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:16:15.129770 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a1271646-7ccf-43ad-9365-a1d31fa8463e tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:16:15.275172 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-a1271646-7ccf-43ad-9365-a1d31fa8463e tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4b8c7ad3cc6240c096c2c7cf31d8f74f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a45a8ce6687c427d8f76e4205a076a93', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:16:15.283615 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-a1271646-7ccf-43ad-9365-a1d31fa8463e tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "GET /compute/v2.1/servers/8f799adb-6c06-4b1f-88ea-318088c8ede0" status: 200 len: 1687 microversion: 2.1 time: 0.167534 Jul 16 22:16:15.284183 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3459/6927] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:16:15 2022] GET /compute/v2.1/servers/8f799adb-6c06-4b1f-88ea-318088c8ede0 => generated 1687 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:16:15.921417 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-3844a9a0-2e61-42d0-a2d8-6b8ed604ac93 req-200a72ac-abdc-4bcb-990b-0351c94cfa4c service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "8f799adb-6c06-4b1f-88ea-318088c8ede0", "name": "network-vif-deleted", "tag": "159fb859-9182-4d67-ad75-d342c4afb984"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:16:15.930521 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-3844a9a0-2e61-42d0-a2d8-6b8ed604ac93 req-200a72ac-abdc-4bcb-990b-0351c94cfa4c service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:16:15.930985 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-3844a9a0-2e61-42d0-a2d8-6b8ed604ac93 req-200a72ac-abdc-4bcb-990b-0351c94cfa4c service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:16:15.942905 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-3844a9a0-2e61-42d0-a2d8-6b8ed604ac93 req-200a72ac-abdc-4bcb-990b-0351c94cfa4c service nova] Creating event network-vif-deleted:159fb859-9182-4d67-ad75-d342c4afb984 for instance 8f799adb-6c06-4b1f-88ea-318088c8ede0 on nch1-master-1 Jul 16 22:16:15.948035 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-3844a9a0-2e61-42d0-a2d8-6b8ed604ac93 req-200a72ac-abdc-4bcb-990b-0351c94cfa4c service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.030693 Jul 16 22:16:15.949001 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3469/6928] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 22:16:15 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 22:16:16.302222 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-85d9e768-f914-4471-979a-87a1169781e5 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:16:16.302842 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-85d9e768-f914-4471-979a-87a1169781e5 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] [instance: 8f799adb-6c06-4b1f-88ea-318088c8ede0] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:16:16.310422 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-85d9e768-f914-4471-979a-87a1169781e5 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:16:16.310890 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-85d9e768-f914-4471-979a-87a1169781e5 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:16:16.311719 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-85d9e768-f914-4471-979a-87a1169781e5 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:16:16.312165 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-85d9e768-f914-4471-979a-87a1169781e5 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:16:16.348652 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-85d9e768-f914-4471-979a-87a1169781e5 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:16:16.349321 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-85d9e768-f914-4471-979a-87a1169781e5 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:16:16.349554 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.objects.instance [None req-85d9e768-f914-4471-979a-87a1169781e5 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lazy-loading 'fault' on Instance uuid 8f799adb-6c06-4b1f-88ea-318088c8ede0 {{(pid=138680) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 22:16:16.395168 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-85d9e768-f914-4471-979a-87a1169781e5 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4b8c7ad3cc6240c096c2c7cf31d8f74f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a45a8ce6687c427d8f76e4205a076a93', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:16:16.399915 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-85d9e768-f914-4471-979a-87a1169781e5 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "GET /compute/v2.1/servers/8f799adb-6c06-4b1f-88ea-318088c8ede0" status: 200 len: 1355 microversion: 2.1 time: 0.102441 Jul 16 22:16:16.400476 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3460/6929] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:16:16 2022] GET /compute/v2.1/servers/8f799adb-6c06-4b1f-88ea-318088c8ede0 => generated 1355 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:16:17.417640 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-10e3f0b8-b202-4209-b90f-e60d619523ef tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:16:17.418400 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-10e3f0b8-b202-4209-b90f-e60d619523ef tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] [instance: 8f799adb-6c06-4b1f-88ea-318088c8ede0] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:16:17.426305 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-10e3f0b8-b202-4209-b90f-e60d619523ef tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:16:17.426776 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-10e3f0b8-b202-4209-b90f-e60d619523ef tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:16:17.427587 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-10e3f0b8-b202-4209-b90f-e60d619523ef tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:16:17.428047 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-10e3f0b8-b202-4209-b90f-e60d619523ef tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:16:17.441689 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-10e3f0b8-b202-4209-b90f-e60d619523ef tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] HTTP exception thrown: Instance 8f799adb-6c06-4b1f-88ea-318088c8ede0 could not be found. Jul 16 22:16:17.443342 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-10e3f0b8-b202-4209-b90f-e60d619523ef tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Request method failure captured: Jul 16 22:16:17.443342 ncdv-master-1 devstack@n-api.service[138681]: request: GET /compute/v2.1/servers/8f799adb-6c06-4b1f-88ea-318088c8ede0 HTTP/1.1 Jul 16 22:16:17.443342 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 22:16:17.443342 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 22:16:17.443342 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 22:16:17.443342 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 22:16:17.443342 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 22:16:17.443342 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 22:16:17.443342 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 22:16:17.443342 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 22:16:17.443342 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0zhtDg4n0Cr97EpGJw_kphX8HozEdWitmpiSsnN8liwN7JwqYkEqcaIM-47rjt-yK-e5LgZo6pFGCSkRgNS-xnBjvFiClGjWelkYApUjhWHqw4F9MI58oj4QDcEbMKfIlL82tbwRwh1LdKROeHuRqnWXe9oskU-rprKG2IX2dbA4fe4 Jul 16 22:16:17.443342 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 22:16:17.443342 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 22:16:17.443342 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 22:16:17.443342 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 22:16:17.443342 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 22:16:17.443342 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 22:16:17.443342 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: a45a8ce6687c427d8f76e4205a076a93 Jul 16 22:16:17.443342 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-TestNetworkAdvancedServerOps-234580283 Jul 16 22:16:17.443342 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 22:16:17.443342 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 22:16:17.443342 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/a45a8ce6687c427d8f76e4205a076a93"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/a45a8ce6687c427d8f76e4205a076a93"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_a45a8ce6687c427d8f76e4205a076a93"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/a45a8ce6687c427d8f76e4205a076a93"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 22:16:17.443342 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-TestNetworkAdvancedServerOps-234580283 Jul 16 22:16:17.443342 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: a45a8ce6687c427d8f76e4205a076a93 Jul 16 22:16:17.443342 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-TestNetworkAdvancedServerOps-234580283 Jul 16 22:16:17.443342 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-TestNetworkAdvancedServerOps-234580283-project Jul 16 22:16:17.443342 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 22:16:17.443342 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 22:16:17.443342 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: 4b8c7ad3cc6240c096c2c7cf31d8f74f Jul 16 22:16:17.443342 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-TestNetworkAdvancedServerOps-234580283-project Jul 16 22:16:17.443342 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 22:16:17.443342 ncdv-master-1 devstack@n-api.service[138681]: exception: Instance 8f799adb-6c06-4b1f-88ea-318088c8ede0 could not be found. Jul 16 22:16:17.443342 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': '8f799adb-6c06-4b1f-88ea-318088c8ede0'} Jul 16 22:16:17.443342 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 22:16:17.443342 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 22:16:17.443342 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 22:16:17.443342 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 22:16:17.443342 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 22:16:17.443342 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 22:16:17.443342 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 22:16:17.443342 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 22:16:17.443342 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 22:16:17.443342 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 22:16:17.443342 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 8f799adb-6c06-4b1f-88ea-318088c8ede0 could not be found. Jul 16 22:16:17.443342 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 22:16:17.443342 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 22:16:17.443342 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 22:16:17.443342 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 22:16:17.443342 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 22:16:17.443342 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 22:16:17.443342 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 22:16:17.443342 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 22:16:17.443342 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 22:16:17.443342 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 22:16:17.443342 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 22:16:17.443342 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 22:16:17.443342 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 22:16:17.443342 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 22:16:17.443342 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 22:16:17.443342 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 22:16:17.443342 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 8f799adb-6c06-4b1f-88ea-318088c8ede0 could not be found. Jul 16 22:16:17.443342 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 22:16:17.443342 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 22:16:17.443342 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 22:16:17.448384 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 22:16:17.448384 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 22:16:17.448384 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 22:16:17.448384 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 22:16:17.448384 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 22:16:17.448384 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 8f799adb-6c06-4b1f-88ea-318088c8ede0 could not be found. Jul 16 22:16:17.448384 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 22:16:17.448384 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-10e3f0b8-b202-4209-b90f-e60d619523ef tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Returning 404 to user: Instance 8f799adb-6c06-4b1f-88ea-318088c8ede0 could not be found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 22:16:17.448384 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-10e3f0b8-b202-4209-b90f-e60d619523ef tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "GET /compute/v2.1/servers/8f799adb-6c06-4b1f-88ea-318088c8ede0" status: 404 len: 111 microversion: 2.1 time: 0.031484 Jul 16 22:16:17.448384 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3470/6930] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:16:17 2022] GET /compute/v2.1/servers/8f799adb-6c06-4b1f-88ea-318088c8ede0 => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 22:16:21.269223 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-b77fc4be-3f1e-4136-8ccf-e616d65f9be8 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Calling method '.version_select at 0x7fdd8faddee0>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:16:21.333630 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-b77fc4be-3f1e-4136-8ccf-e616d65f9be8 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "DELETE /compute/v2.1/os-keypairs/tempest-TestNetworkAdvancedServerOps-202973646" status: 202 len: 0 microversion: 2.1 time: 0.069031 Jul 16 22:16:21.334233 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3461/6931] 10.222.0.16 () {58 vars in 1280 bytes} [Sat Jul 16 22:16:21 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestNetworkAdvancedServerOps-202973646 => generated 0 bytes in 70 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 22:16:21.363323 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-aee00c32-6eb3-4872-8c0f-8c04f3279c1f tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Action: 'create', calling method: .version_select at 0x7fdd8ff63f70>, body: {"keypair": {"name": "tempest-TestNetworkAdvancedServerOps-690605879", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBBmo2jx5cacec0nFqqiPlz2Zjwx2pyHC8qbm/Cte95CQxXlRQxd9dGlTTLcyr2SBLF9CbaC1cNfEWufHOpq3FcgY3+ByclEu+LshgyRu7/u7F1NJc/yxvAGSXMbb+g/+YA=="}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:16:21.378875 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-aee00c32-6eb3-4872-8c0f-8c04f3279c1f tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Getting quotas for project a45a8ce6687c427d8f76e4205a076a93. Resources: {'key_pairs'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 22:16:21.384818 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-aee00c32-6eb3-4872-8c0f-8c04f3279c1f tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Getting quotas for user 4b8c7ad3cc6240c096c2c7cf31d8f74f and project a45a8ce6687c427d8f76e4205a076a93. Resources: {'key_pairs'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 22:16:21.463016 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-aee00c32-6eb3-4872-8c0f-8c04f3279c1f tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "POST /compute/v2.1/os-keypairs" status: 200 len: 406 microversion: 2.1 time: 0.104368 Jul 16 22:16:21.463795 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3471/6932] 10.222.0.16 () {60 vars in 1158 bytes} [Sat Jul 16 22:16:21 2022] POST /compute/v2.1/os-keypairs => generated 406 bytes in 106 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 16 22:16:25.001726 ncdv-master-1 devstack@n-api.service[138680]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jul 16 22:16:25.012337 ncdv-master-1 devstack@n-api.service[138680]: Sat Jul 16 22:16:25 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Jul 16 22:16:29.504504 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=72,nova_cell1:UPDATE=3,nova_cell1:INSERT=3 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:16:29.509080 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-ac7973cd-46d1-4b06-86e2-fe9877d22d8f tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestNetworkAdvancedServerOps-server-1351841812", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452", "networks": [{"uuid": "d911a1fe-f5d0-4dc7-b745-b8fa4e8a1605"}], "key_name": "tempest-TestNetworkAdvancedServerOps-690605879", "security_groups": [{"name": "tempest-secgroup-smoke-1937295247"}]}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:16:29.632666 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.network.neutron [None req-ac7973cd-46d1-4b06-86e2-fe9877d22d8f tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] validate_networks() for [('d911a1fe-f5d0-4dc7-b745-b8fa4e8a1605', None, None, None, None, None)] {{(pid=138680) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 16 22:16:30.140571 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-ac7973cd-46d1-4b06-86e2-fe9877d22d8f tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Flavor limits 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 16 22:16:30.141003 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-ac7973cd-46d1-4b06-86e2-fe9877d22d8f tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Image limits 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 16 22:16:30.141684 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-ac7973cd-46d1-4b06-86e2-fe9877d22d8f tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Flavor pref 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 16 22:16:30.142002 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-ac7973cd-46d1-4b06-86e2-fe9877d22d8f tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Image pref 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 16 22:16:30.142747 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-ac7973cd-46d1-4b06-86e2-fe9877d22d8f tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 16 22:16:30.153406 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ac7973cd-46d1-4b06-86e2-fe9877d22d8f tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:16:30.153897 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ac7973cd-46d1-4b06-86e2-fe9877d22d8f tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:16:30.154758 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ac7973cd-46d1-4b06-86e2-fe9877d22d8f tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:16:30.155254 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ac7973cd-46d1-4b06-86e2-fe9877d22d8f tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:16:30.174894 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-ac7973cd-46d1-4b06-86e2-fe9877d22d8f tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Getting quotas for project a45a8ce6687c427d8f76e4205a076a93. Resources: {'ram', 'instances', 'cores'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 22:16:30.183595 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-ac7973cd-46d1-4b06-86e2-fe9877d22d8f tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Getting quotas for user 4b8c7ad3cc6240c096c2c7cf31d8f74f and project a45a8ce6687c427d8f76e4205a076a93. Resources: {'ram', 'instances', 'cores'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 22:16:30.195147 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-ac7973cd-46d1-4b06-86e2-fe9877d22d8f tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Going to run 1 instances... {{(pid=138680) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 16 22:16:30.198189 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-ac7973cd-46d1-4b06-86e2-fe9877d22d8f tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] [instance: 52817115-f50c-4d1b-a4af-b9fddc045246] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138680) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 16 22:16:30.295154 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-ac7973cd-46d1-4b06-86e2-fe9877d22d8f tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 406 microversion: 2.1 time: 0.792444 Jul 16 22:16:30.299174 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3462/6933] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 22:16:29 2022] POST /compute/v2.1/servers => generated 406 bytes in 797 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 16 22:16:30.306996 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-445bf5fe-92ae-4ef0-826a-d4cf980c9ab8 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:16:30.315622 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-445bf5fe-92ae-4ef0-826a-d4cf980c9ab8 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] [instance: 52817115-f50c-4d1b-a4af-b9fddc045246] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:16:30.328622 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=48 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:16:30.411662 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-445bf5fe-92ae-4ef0-826a-d4cf980c9ab8 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4b8c7ad3cc6240c096c2c7cf31d8f74f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a45a8ce6687c427d8f76e4205a076a93', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:16:30.423229 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-445bf5fe-92ae-4ef0-826a-d4cf980c9ab8 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "GET /compute/v2.1/servers/52817115-f50c-4d1b-a4af-b9fddc045246" status: 200 len: 1266 microversion: 2.1 time: 0.119933 Jul 16 22:16:30.427282 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3472/6934] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:16:30 2022] GET /compute/v2.1/servers/52817115-f50c-4d1b-a4af-b9fddc045246 => generated 1266 bytes in 123 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 22:16:31.441938 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-0b0d8c1b-b516-44e4-9273-aa6ebf85d62f tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:16:31.443226 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-0b0d8c1b-b516-44e4-9273-aa6ebf85d62f tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] [instance: 52817115-f50c-4d1b-a4af-b9fddc045246] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:16:31.449606 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0b0d8c1b-b516-44e4-9273-aa6ebf85d62f tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:16:31.454210 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0b0d8c1b-b516-44e4-9273-aa6ebf85d62f tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:16:31.454210 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0b0d8c1b-b516-44e4-9273-aa6ebf85d62f tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:16:31.454210 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0b0d8c1b-b516-44e4-9273-aa6ebf85d62f tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:16:31.519772 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-0b0d8c1b-b516-44e4-9273-aa6ebf85d62f tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4b8c7ad3cc6240c096c2c7cf31d8f74f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a45a8ce6687c427d8f76e4205a076a93', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:16:31.534471 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-0b0d8c1b-b516-44e4-9273-aa6ebf85d62f tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "GET /compute/v2.1/servers/52817115-f50c-4d1b-a4af-b9fddc045246" status: 200 len: 1270 microversion: 2.1 time: 0.098331 Jul 16 22:16:31.535410 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3463/6935] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:16:31 2022] GET /compute/v2.1/servers/52817115-f50c-4d1b-a4af-b9fddc045246 => generated 1270 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:16:32.554045 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-e321a17e-7301-4d5d-9832-5f91e0439f12 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:16:32.554840 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-e321a17e-7301-4d5d-9832-5f91e0439f12 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] [instance: 52817115-f50c-4d1b-a4af-b9fddc045246] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:16:32.570343 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e321a17e-7301-4d5d-9832-5f91e0439f12 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:16:32.570817 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e321a17e-7301-4d5d-9832-5f91e0439f12 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:16:32.577570 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e321a17e-7301-4d5d-9832-5f91e0439f12 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:16:32.577570 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e321a17e-7301-4d5d-9832-5f91e0439f12 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:16:32.638557 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-e321a17e-7301-4d5d-9832-5f91e0439f12 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4b8c7ad3cc6240c096c2c7cf31d8f74f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a45a8ce6687c427d8f76e4205a076a93', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:16:32.647871 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-e321a17e-7301-4d5d-9832-5f91e0439f12 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "GET /compute/v2.1/servers/52817115-f50c-4d1b-a4af-b9fddc045246" status: 200 len: 1336 microversion: 2.1 time: 0.099997 Jul 16 22:16:32.649020 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3473/6936] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:16:32 2022] GET /compute/v2.1/servers/52817115-f50c-4d1b-a4af-b9fddc045246 => generated 1336 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:16:33.665989 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-97ce5a7d-0ba9-42b0-9b77-228496cb9f1f tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:16:33.666609 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-97ce5a7d-0ba9-42b0-9b77-228496cb9f1f tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] [instance: 52817115-f50c-4d1b-a4af-b9fddc045246] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:16:33.669390 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_api:SELECT=46,nova_api:UPDATE=1,nova_api:DELETE=1,nova_api:INSERT=3 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:16:33.674377 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-97ce5a7d-0ba9-42b0-9b77-228496cb9f1f tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:16:33.674840 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-97ce5a7d-0ba9-42b0-9b77-228496cb9f1f tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:16:33.675655 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-97ce5a7d-0ba9-42b0-9b77-228496cb9f1f tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:16:33.676103 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-97ce5a7d-0ba9-42b0-9b77-228496cb9f1f tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:16:33.879184 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-97ce5a7d-0ba9-42b0-9b77-228496cb9f1f tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4b8c7ad3cc6240c096c2c7cf31d8f74f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a45a8ce6687c427d8f76e4205a076a93', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:16:33.888103 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-97ce5a7d-0ba9-42b0-9b77-228496cb9f1f tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "GET /compute/v2.1/servers/52817115-f50c-4d1b-a4af-b9fddc045246" status: 200 len: 1392 microversion: 2.1 time: 0.226181 Jul 16 22:16:33.888693 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3464/6937] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:16:33 2022] GET /compute/v2.1/servers/52817115-f50c-4d1b-a4af-b9fddc045246 => generated 1392 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:16:34.124464 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-65aaa64b-5b36-4f1c-89b7-0b7df1e7b8c8 req-90716122-c9bc-493b-a4d7-91d7d9709891 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "52817115-f50c-4d1b-a4af-b9fddc045246", "tag": "6f7e4fe6-52ba-43e9-8b99-5718848c427a"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:16:34.131166 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_api:SELECT=40,nova_api:INSERT=1 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:16:34.135136 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-65aaa64b-5b36-4f1c-89b7-0b7df1e7b8c8 req-90716122-c9bc-493b-a4d7-91d7d9709891 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:16:34.135606 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-65aaa64b-5b36-4f1c-89b7-0b7df1e7b8c8 req-90716122-c9bc-493b-a4d7-91d7d9709891 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:16:34.155743 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-65aaa64b-5b36-4f1c-89b7-0b7df1e7b8c8 req-90716122-c9bc-493b-a4d7-91d7d9709891 service nova] Creating event network-changed:6f7e4fe6-52ba-43e9-8b99-5718848c427a for instance 52817115-f50c-4d1b-a4af-b9fddc045246 on nch1-master-1 Jul 16 22:16:34.161612 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-65aaa64b-5b36-4f1c-89b7-0b7df1e7b8c8 req-90716122-c9bc-493b-a4d7-91d7d9709891 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.041820 Jul 16 22:16:34.162473 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3474/6938] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 22:16:34 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 16 22:16:34.906313 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-dabe7d3a-33c8-444b-a6ab-8aa05a146c56 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:16:34.907379 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-dabe7d3a-33c8-444b-a6ab-8aa05a146c56 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] [instance: 52817115-f50c-4d1b-a4af-b9fddc045246] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:16:34.914472 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-dabe7d3a-33c8-444b-a6ab-8aa05a146c56 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:16:34.914925 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-dabe7d3a-33c8-444b-a6ab-8aa05a146c56 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:16:34.915710 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-dabe7d3a-33c8-444b-a6ab-8aa05a146c56 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:16:34.916153 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-dabe7d3a-33c8-444b-a6ab-8aa05a146c56 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:16:35.055990 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-dabe7d3a-33c8-444b-a6ab-8aa05a146c56 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4b8c7ad3cc6240c096c2c7cf31d8f74f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a45a8ce6687c427d8f76e4205a076a93', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:16:35.064988 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-dabe7d3a-33c8-444b-a6ab-8aa05a146c56 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "GET /compute/v2.1/servers/52817115-f50c-4d1b-a4af-b9fddc045246" status: 200 len: 1392 microversion: 2.1 time: 0.162935 Jul 16 22:16:35.065562 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3465/6939] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:16:34 2022] GET /compute/v2.1/servers/52817115-f50c-4d1b-a4af-b9fddc045246 => generated 1392 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:16:36.084346 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-f97126e4-8fe5-4560-aa89-b727773eae54 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:16:36.085066 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-f97126e4-8fe5-4560-aa89-b727773eae54 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] [instance: 52817115-f50c-4d1b-a4af-b9fddc045246] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:16:36.092078 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f97126e4-8fe5-4560-aa89-b727773eae54 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:16:36.092543 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f97126e4-8fe5-4560-aa89-b727773eae54 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:16:36.093314 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f97126e4-8fe5-4560-aa89-b727773eae54 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:16:36.093769 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f97126e4-8fe5-4560-aa89-b727773eae54 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:16:36.278757 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-f97126e4-8fe5-4560-aa89-b727773eae54 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4b8c7ad3cc6240c096c2c7cf31d8f74f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a45a8ce6687c427d8f76e4205a076a93', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:16:36.286549 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-f97126e4-8fe5-4560-aa89-b727773eae54 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "GET /compute/v2.1/servers/52817115-f50c-4d1b-a4af-b9fddc045246" status: 200 len: 1392 microversion: 2.1 time: 0.208536 Jul 16 22:16:36.287145 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3475/6940] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:16:36 2022] GET /compute/v2.1/servers/52817115-f50c-4d1b-a4af-b9fddc045246 => generated 1392 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:16:37.303528 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-6ef69b6e-61cc-40f3-9ff5-abba6cde550b tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:16:37.304275 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-6ef69b6e-61cc-40f3-9ff5-abba6cde550b tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] [instance: 52817115-f50c-4d1b-a4af-b9fddc045246] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:16:37.311502 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6ef69b6e-61cc-40f3-9ff5-abba6cde550b tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:16:37.311961 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6ef69b6e-61cc-40f3-9ff5-abba6cde550b tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:16:37.313629 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6ef69b6e-61cc-40f3-9ff5-abba6cde550b tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:16:37.314078 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6ef69b6e-61cc-40f3-9ff5-abba6cde550b tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:16:37.451618 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-6ef69b6e-61cc-40f3-9ff5-abba6cde550b tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4b8c7ad3cc6240c096c2c7cf31d8f74f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a45a8ce6687c427d8f76e4205a076a93', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:16:37.459433 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-6ef69b6e-61cc-40f3-9ff5-abba6cde550b tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "GET /compute/v2.1/servers/52817115-f50c-4d1b-a4af-b9fddc045246" status: 200 len: 1392 microversion: 2.1 time: 0.159854 Jul 16 22:16:37.460006 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3466/6941] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:16:37 2022] GET /compute/v2.1/servers/52817115-f50c-4d1b-a4af-b9fddc045246 => generated 1392 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 22:16:38.476592 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-47917a8a-44f0-4a33-8062-e3e146dbd7c1 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:16:38.477252 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-47917a8a-44f0-4a33-8062-e3e146dbd7c1 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] [instance: 52817115-f50c-4d1b-a4af-b9fddc045246] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:16:38.485453 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-47917a8a-44f0-4a33-8062-e3e146dbd7c1 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:16:38.492043 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-47917a8a-44f0-4a33-8062-e3e146dbd7c1 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:16:38.492043 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-47917a8a-44f0-4a33-8062-e3e146dbd7c1 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:16:38.492043 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-47917a8a-44f0-4a33-8062-e3e146dbd7c1 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:16:38.622323 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-47917a8a-44f0-4a33-8062-e3e146dbd7c1 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4b8c7ad3cc6240c096c2c7cf31d8f74f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a45a8ce6687c427d8f76e4205a076a93', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:16:38.630930 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-47917a8a-44f0-4a33-8062-e3e146dbd7c1 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "GET /compute/v2.1/servers/52817115-f50c-4d1b-a4af-b9fddc045246" status: 200 len: 1392 microversion: 2.1 time: 0.158684 Jul 16 22:16:38.631539 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3476/6942] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:16:38 2022] GET /compute/v2.1/servers/52817115-f50c-4d1b-a4af-b9fddc045246 => generated 1392 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 22:16:39.648209 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-d330d309-e8b3-4343-ab53-6401ca16e839 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:16:39.648819 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-d330d309-e8b3-4343-ab53-6401ca16e839 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] [instance: 52817115-f50c-4d1b-a4af-b9fddc045246] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:16:39.657381 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d330d309-e8b3-4343-ab53-6401ca16e839 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:16:39.658208 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d330d309-e8b3-4343-ab53-6401ca16e839 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:16:39.658987 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d330d309-e8b3-4343-ab53-6401ca16e839 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:16:39.659452 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d330d309-e8b3-4343-ab53-6401ca16e839 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:16:39.799981 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-d330d309-e8b3-4343-ab53-6401ca16e839 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4b8c7ad3cc6240c096c2c7cf31d8f74f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a45a8ce6687c427d8f76e4205a076a93', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:16:39.807625 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-d330d309-e8b3-4343-ab53-6401ca16e839 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "GET /compute/v2.1/servers/52817115-f50c-4d1b-a4af-b9fddc045246" status: 200 len: 1392 microversion: 2.1 time: 0.163280 Jul 16 22:16:39.808198 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3467/6943] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:16:39 2022] GET /compute/v2.1/servers/52817115-f50c-4d1b-a4af-b9fddc045246 => generated 1392 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:16:39.874695 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-6d845191-2ade-48fe-afe6-9a802842fad0 req-60eafe7b-879c-43a2-8f26-546fec947cc5 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "52817115-f50c-4d1b-a4af-b9fddc045246", "name": "network-vif-plugged", "status": "completed", "tag": "6f7e4fe6-52ba-43e9-8b99-5718848c427a"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:16:39.884669 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-6d845191-2ade-48fe-afe6-9a802842fad0 req-60eafe7b-879c-43a2-8f26-546fec947cc5 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:16:39.885015 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-6d845191-2ade-48fe-afe6-9a802842fad0 req-60eafe7b-879c-43a2-8f26-546fec947cc5 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:16:39.896891 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-6d845191-2ade-48fe-afe6-9a802842fad0 req-60eafe7b-879c-43a2-8f26-546fec947cc5 service nova] Creating event network-vif-plugged:6f7e4fe6-52ba-43e9-8b99-5718848c427a for instance 52817115-f50c-4d1b-a4af-b9fddc045246 on nch1-master-1 Jul 16 22:16:39.901715 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-6d845191-2ade-48fe-afe6-9a802842fad0 req-60eafe7b-879c-43a2-8f26-546fec947cc5 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.030846 Jul 16 22:16:39.902302 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3477/6944] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 22:16:39 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 22:16:40.825675 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-7c78247d-b242-4426-b7b1-6c923ac6dabb tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:16:40.826296 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-7c78247d-b242-4426-b7b1-6c923ac6dabb tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] [instance: 52817115-f50c-4d1b-a4af-b9fddc045246] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:16:40.828217 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell0:SELECT=2 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:16:40.829412 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell0:SELECT=6 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:16:40.835181 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7c78247d-b242-4426-b7b1-6c923ac6dabb tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:16:40.835640 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7c78247d-b242-4426-b7b1-6c923ac6dabb tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:16:40.836411 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7c78247d-b242-4426-b7b1-6c923ac6dabb tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:16:40.836855 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7c78247d-b242-4426-b7b1-6c923ac6dabb tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:16:40.972225 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-7c78247d-b242-4426-b7b1-6c923ac6dabb tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4b8c7ad3cc6240c096c2c7cf31d8f74f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a45a8ce6687c427d8f76e4205a076a93', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:16:40.979880 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-7c78247d-b242-4426-b7b1-6c923ac6dabb tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "GET /compute/v2.1/servers/52817115-f50c-4d1b-a4af-b9fddc045246" status: 200 len: 1392 microversion: 2.1 time: 0.158267 Jul 16 22:16:40.980440 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3468/6945] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:16:40 2022] GET /compute/v2.1/servers/52817115-f50c-4d1b-a4af-b9fddc045246 => generated 1392 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:16:41.993850 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell0:SELECT=6 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:16:41.999365 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-5052e738-441d-4c17-a9da-3c32baa7ad0a tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:16:42.000070 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-5052e738-441d-4c17-a9da-3c32baa7ad0a tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] [instance: 52817115-f50c-4d1b-a4af-b9fddc045246] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:16:42.006461 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5052e738-441d-4c17-a9da-3c32baa7ad0a tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:16:42.006942 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5052e738-441d-4c17-a9da-3c32baa7ad0a tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:16:42.007722 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5052e738-441d-4c17-a9da-3c32baa7ad0a tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:16:42.008167 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5052e738-441d-4c17-a9da-3c32baa7ad0a tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:16:42.144077 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-5052e738-441d-4c17-a9da-3c32baa7ad0a tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4b8c7ad3cc6240c096c2c7cf31d8f74f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a45a8ce6687c427d8f76e4205a076a93', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:16:42.151742 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-5052e738-441d-4c17-a9da-3c32baa7ad0a tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "GET /compute/v2.1/servers/52817115-f50c-4d1b-a4af-b9fddc045246" status: 200 len: 1564 microversion: 2.1 time: 0.158904 Jul 16 22:16:42.152439 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3478/6946] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:16:41 2022] GET /compute/v2.1/servers/52817115-f50c-4d1b-a4af-b9fddc045246 => generated 1564 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 22:16:42.169740 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-49a5ca06-6a9f-46aa-8a98-13635e39d3ef tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:16:42.170421 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-49a5ca06-6a9f-46aa-8a98-13635e39d3ef tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] [instance: 52817115-f50c-4d1b-a4af-b9fddc045246] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:16:42.178048 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-49a5ca06-6a9f-46aa-8a98-13635e39d3ef tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:16:42.178505 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-49a5ca06-6a9f-46aa-8a98-13635e39d3ef tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:16:42.179289 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-49a5ca06-6a9f-46aa-8a98-13635e39d3ef tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:16:42.179774 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-49a5ca06-6a9f-46aa-8a98-13635e39d3ef tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:16:42.321975 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-49a5ca06-6a9f-46aa-8a98-13635e39d3ef tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4b8c7ad3cc6240c096c2c7cf31d8f74f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a45a8ce6687c427d8f76e4205a076a93', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:16:42.330286 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-49a5ca06-6a9f-46aa-8a98-13635e39d3ef tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "GET /compute/v2.1/servers/52817115-f50c-4d1b-a4af-b9fddc045246" status: 200 len: 1564 microversion: 2.1 time: 0.164247 Jul 16 22:16:42.330878 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3469/6947] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:16:42 2022] GET /compute/v2.1/servers/52817115-f50c-4d1b-a4af-b9fddc045246 => generated 1564 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:16:44.244579 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-a07c5ea3-65a1-48ce-bdfb-1c73458e6c14 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:16:44.247205 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-a07c5ea3-65a1-48ce-bdfb-1c73458e6c14 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] [instance: 52817115-f50c-4d1b-a4af-b9fddc045246] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:16:44.247330 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-3632ec0a-a177-41be-84c2-150fb37f2310 req-1f13682a-103d-4347-ab91-2fb84a413bcd service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "52817115-f50c-4d1b-a4af-b9fddc045246", "tag": "6f7e4fe6-52ba-43e9-8b99-5718848c427a"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:16:44.254385 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a07c5ea3-65a1-48ce-bdfb-1c73458e6c14 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:16:44.255174 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a07c5ea3-65a1-48ce-bdfb-1c73458e6c14 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:16:44.255981 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a07c5ea3-65a1-48ce-bdfb-1c73458e6c14 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:16:44.256437 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a07c5ea3-65a1-48ce-bdfb-1c73458e6c14 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:16:44.256591 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-3632ec0a-a177-41be-84c2-150fb37f2310 req-1f13682a-103d-4347-ab91-2fb84a413bcd service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:16:44.257285 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-3632ec0a-a177-41be-84c2-150fb37f2310 req-1f13682a-103d-4347-ab91-2fb84a413bcd service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:16:44.269864 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-3632ec0a-a177-41be-84c2-150fb37f2310 req-1f13682a-103d-4347-ab91-2fb84a413bcd service nova] Creating event network-changed:6f7e4fe6-52ba-43e9-8b99-5718848c427a for instance 52817115-f50c-4d1b-a4af-b9fddc045246 on nch1-master-1 Jul 16 22:16:44.276165 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-3632ec0a-a177-41be-84c2-150fb37f2310 req-1f13682a-103d-4347-ab91-2fb84a413bcd service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.033212 Jul 16 22:16:44.277737 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3470/6948] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 22:16:44 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 22:16:44.758402 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-a07c5ea3-65a1-48ce-bdfb-1c73458e6c14 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4b8c7ad3cc6240c096c2c7cf31d8f74f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a45a8ce6687c427d8f76e4205a076a93', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:16:44.768038 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-a07c5ea3-65a1-48ce-bdfb-1c73458e6c14 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "GET /compute/v2.1/servers/52817115-f50c-4d1b-a4af-b9fddc045246" status: 200 len: 1564 microversion: 2.1 time: 0.527925 Jul 16 22:16:44.768747 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3479/6949] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:16:44 2022] GET /compute/v2.1/servers/52817115-f50c-4d1b-a4af-b9fddc045246 => generated 1564 bytes in 529 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:17:06.021775 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_api:SELECT=13 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:17:06.022542 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=46 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:17:06.026805 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-8e042ab3-ed31-428f-aeee-fcecdccdd205 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Action: 'action', calling method: >, body: {"os-getConsoleOutput": {}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:17:06.034216 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-8e042ab3-ed31-428f-aeee-fcecdccdd205 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] [instance: 52817115-f50c-4d1b-a4af-b9fddc045246] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:17:06.034704 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8e042ab3-ed31-428f-aeee-fcecdccdd205 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:17:06.035190 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8e042ab3-ed31-428f-aeee-fcecdccdd205 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:17:06.035967 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8e042ab3-ed31-428f-aeee-fcecdccdd205 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:17:06.036417 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8e042ab3-ed31-428f-aeee-fcecdccdd205 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:17:06.092074 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-8e042ab3-ed31-428f-aeee-fcecdccdd205 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "POST /compute/v2.1/servers/52817115-f50c-4d1b-a4af-b9fddc045246/action" status: 200 len: 24573 microversion: 2.1 time: 0.071980 Jul 16 22:17:06.092789 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3471/6950] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 22:17:06 2022] POST /compute/v2.1/servers/52817115-f50c-4d1b-a4af-b9fddc045246/action => generated 24573 bytes in 74 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 16 22:17:17.102280 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-28820ac4-168f-47f6-95f5-3bbc3bb2b4a4 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Action: 'action', calling method: >, body: {"reboot": {"type": "SOFT"}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:17:17.103281 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-28820ac4-168f-47f6-95f5-3bbc3bb2b4a4 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] [instance: 52817115-f50c-4d1b-a4af-b9fddc045246] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:17:17.105463 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_api:SELECT=11 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:17:17.106176 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=34 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:17:17.111123 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-28820ac4-168f-47f6-95f5-3bbc3bb2b4a4 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:17:17.111613 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-28820ac4-168f-47f6-95f5-3bbc3bb2b4a4 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:17:17.112662 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-28820ac4-168f-47f6-95f5-3bbc3bb2b4a4 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:17:17.113119 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-28820ac4-168f-47f6-95f5-3bbc3bb2b4a4 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:17:17.290203 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-28820ac4-168f-47f6-95f5-3bbc3bb2b4a4 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "POST /compute/v2.1/servers/52817115-f50c-4d1b-a4af-b9fddc045246/action" status: 202 len: 0 microversion: 2.1 time: 0.191990 Jul 16 22:17:17.290963 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3480/6951] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 22:17:17 2022] POST /compute/v2.1/servers/52817115-f50c-4d1b-a4af-b9fddc045246/action => generated 0 bytes in 194 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 22:17:17.299834 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-e1f35cc8-e976-443d-ad3b-97d7a4299042 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:17:17.300799 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-e1f35cc8-e976-443d-ad3b-97d7a4299042 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] [instance: 52817115-f50c-4d1b-a4af-b9fddc045246] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:17:17.303829 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_api:SELECT=2 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:17:17.304498 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=4 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:17:17.309604 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e1f35cc8-e976-443d-ad3b-97d7a4299042 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:17:17.310059 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e1f35cc8-e976-443d-ad3b-97d7a4299042 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:17:17.310834 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e1f35cc8-e976-443d-ad3b-97d7a4299042 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:17:17.311553 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e1f35cc8-e976-443d-ad3b-97d7a4299042 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:17:17.451715 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-e1f35cc8-e976-443d-ad3b-97d7a4299042 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4b8c7ad3cc6240c096c2c7cf31d8f74f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a45a8ce6687c427d8f76e4205a076a93', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:17:17.460041 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-e1f35cc8-e976-443d-ad3b-97d7a4299042 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "GET /compute/v2.1/servers/52817115-f50c-4d1b-a4af-b9fddc045246" status: 200 len: 1675 microversion: 2.1 time: 0.164659 Jul 16 22:17:17.460597 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3472/6952] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:17:17 2022] GET /compute/v2.1/servers/52817115-f50c-4d1b-a4af-b9fddc045246 => generated 1675 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:17:18.478577 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-bdeab2c6-52f6-4b42-b200-78f6f9f931fe tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:17:18.479260 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-bdeab2c6-52f6-4b42-b200-78f6f9f931fe tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] [instance: 52817115-f50c-4d1b-a4af-b9fddc045246] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:17:18.485746 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-bdeab2c6-52f6-4b42-b200-78f6f9f931fe tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:17:18.486212 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-bdeab2c6-52f6-4b42-b200-78f6f9f931fe tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:17:18.487619 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-bdeab2c6-52f6-4b42-b200-78f6f9f931fe tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:17:18.488321 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-bdeab2c6-52f6-4b42-b200-78f6f9f931fe tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:17:18.620983 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-bdeab2c6-52f6-4b42-b200-78f6f9f931fe tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4b8c7ad3cc6240c096c2c7cf31d8f74f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a45a8ce6687c427d8f76e4205a076a93', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:17:18.628861 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-bdeab2c6-52f6-4b42-b200-78f6f9f931fe tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "GET /compute/v2.1/servers/52817115-f50c-4d1b-a4af-b9fddc045246" status: 200 len: 1680 microversion: 2.1 time: 0.154892 Jul 16 22:17:18.629430 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3481/6953] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:17:18 2022] GET /compute/v2.1/servers/52817115-f50c-4d1b-a4af-b9fddc045246 => generated 1680 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:17:19.646554 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-73c204b6-1169-41c6-aba0-3b42af2b95d8 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:17:19.647301 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-73c204b6-1169-41c6-aba0-3b42af2b95d8 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] [instance: 52817115-f50c-4d1b-a4af-b9fddc045246] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:17:19.653983 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-73c204b6-1169-41c6-aba0-3b42af2b95d8 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:17:19.654444 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-73c204b6-1169-41c6-aba0-3b42af2b95d8 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:17:19.655245 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-73c204b6-1169-41c6-aba0-3b42af2b95d8 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:17:19.655718 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-73c204b6-1169-41c6-aba0-3b42af2b95d8 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:17:19.792839 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-73c204b6-1169-41c6-aba0-3b42af2b95d8 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4b8c7ad3cc6240c096c2c7cf31d8f74f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a45a8ce6687c427d8f76e4205a076a93', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:17:19.800635 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-73c204b6-1169-41c6-aba0-3b42af2b95d8 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "GET /compute/v2.1/servers/52817115-f50c-4d1b-a4af-b9fddc045246" status: 200 len: 1680 microversion: 2.1 time: 0.158187 Jul 16 22:17:19.801198 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3473/6954] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:17:19 2022] GET /compute/v2.1/servers/52817115-f50c-4d1b-a4af-b9fddc045246 => generated 1680 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:17:20.818088 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-b437723c-2d17-444f-bb9e-686b30b5ba6b tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:17:20.818727 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-b437723c-2d17-444f-bb9e-686b30b5ba6b tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] [instance: 52817115-f50c-4d1b-a4af-b9fddc045246] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:17:20.826200 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b437723c-2d17-444f-bb9e-686b30b5ba6b tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:17:20.826667 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b437723c-2d17-444f-bb9e-686b30b5ba6b tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:17:20.827485 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b437723c-2d17-444f-bb9e-686b30b5ba6b tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:17:20.827942 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b437723c-2d17-444f-bb9e-686b30b5ba6b tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:17:20.962320 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-b437723c-2d17-444f-bb9e-686b30b5ba6b tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4b8c7ad3cc6240c096c2c7cf31d8f74f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a45a8ce6687c427d8f76e4205a076a93', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:17:20.969885 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-b437723c-2d17-444f-bb9e-686b30b5ba6b tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "GET /compute/v2.1/servers/52817115-f50c-4d1b-a4af-b9fddc045246" status: 200 len: 1680 microversion: 2.1 time: 0.155897 Jul 16 22:17:20.970456 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3482/6955] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:17:20 2022] GET /compute/v2.1/servers/52817115-f50c-4d1b-a4af-b9fddc045246 => generated 1680 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:17:21.987178 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-06c5177e-115b-4f4f-bb56-a184247af0a9 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:17:21.987794 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-06c5177e-115b-4f4f-bb56-a184247af0a9 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] [instance: 52817115-f50c-4d1b-a4af-b9fddc045246] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:17:21.994700 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-06c5177e-115b-4f4f-bb56-a184247af0a9 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:17:21.995191 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-06c5177e-115b-4f4f-bb56-a184247af0a9 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:17:21.995958 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-06c5177e-115b-4f4f-bb56-a184247af0a9 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:17:21.996402 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-06c5177e-115b-4f4f-bb56-a184247af0a9 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:17:22.133460 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-06c5177e-115b-4f4f-bb56-a184247af0a9 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4b8c7ad3cc6240c096c2c7cf31d8f74f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a45a8ce6687c427d8f76e4205a076a93', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:17:22.141105 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-06c5177e-115b-4f4f-bb56-a184247af0a9 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "GET /compute/v2.1/servers/52817115-f50c-4d1b-a4af-b9fddc045246" status: 200 len: 1680 microversion: 2.1 time: 0.157970 Jul 16 22:17:22.141668 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3474/6956] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:17:21 2022] GET /compute/v2.1/servers/52817115-f50c-4d1b-a4af-b9fddc045246 => generated 1680 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:17:23.159679 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-6f9ff908-b360-4d06-bd69-c2838f5190fd tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:17:23.160207 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-6f9ff908-b360-4d06-bd69-c2838f5190fd tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] [instance: 52817115-f50c-4d1b-a4af-b9fddc045246] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:17:23.167156 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6f9ff908-b360-4d06-bd69-c2838f5190fd tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:17:23.167626 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6f9ff908-b360-4d06-bd69-c2838f5190fd tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:17:23.168406 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6f9ff908-b360-4d06-bd69-c2838f5190fd tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:17:23.168861 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6f9ff908-b360-4d06-bd69-c2838f5190fd tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:17:23.298478 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-6f9ff908-b360-4d06-bd69-c2838f5190fd tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4b8c7ad3cc6240c096c2c7cf31d8f74f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a45a8ce6687c427d8f76e4205a076a93', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:17:23.306659 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-6f9ff908-b360-4d06-bd69-c2838f5190fd tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "GET /compute/v2.1/servers/52817115-f50c-4d1b-a4af-b9fddc045246" status: 200 len: 1680 microversion: 2.1 time: 0.151600 Jul 16 22:17:23.307278 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3483/6957] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:17:23 2022] GET /compute/v2.1/servers/52817115-f50c-4d1b-a4af-b9fddc045246 => generated 1680 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:17:24.324541 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-8593252f-14cb-463b-88b4-8c1cae5abfa2 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:17:24.325161 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-8593252f-14cb-463b-88b4-8c1cae5abfa2 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] [instance: 52817115-f50c-4d1b-a4af-b9fddc045246] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:17:24.332902 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8593252f-14cb-463b-88b4-8c1cae5abfa2 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:17:24.333360 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8593252f-14cb-463b-88b4-8c1cae5abfa2 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:17:24.334130 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8593252f-14cb-463b-88b4-8c1cae5abfa2 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:17:24.334577 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8593252f-14cb-463b-88b4-8c1cae5abfa2 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:17:24.467240 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-8593252f-14cb-463b-88b4-8c1cae5abfa2 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4b8c7ad3cc6240c096c2c7cf31d8f74f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a45a8ce6687c427d8f76e4205a076a93', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:17:24.476085 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-8593252f-14cb-463b-88b4-8c1cae5abfa2 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "GET /compute/v2.1/servers/52817115-f50c-4d1b-a4af-b9fddc045246" status: 200 len: 1680 microversion: 2.1 time: 0.155433 Jul 16 22:17:24.476652 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3475/6958] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:17:24 2022] GET /compute/v2.1/servers/52817115-f50c-4d1b-a4af-b9fddc045246 => generated 1680 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:17:25.493777 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-04f5bb8f-4cd9-4ef9-b4e1-f63fe01cea11 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:17:25.494483 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-04f5bb8f-4cd9-4ef9-b4e1-f63fe01cea11 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] [instance: 52817115-f50c-4d1b-a4af-b9fddc045246] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:17:25.501400 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-04f5bb8f-4cd9-4ef9-b4e1-f63fe01cea11 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:17:25.501861 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-04f5bb8f-4cd9-4ef9-b4e1-f63fe01cea11 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:17:25.502875 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-04f5bb8f-4cd9-4ef9-b4e1-f63fe01cea11 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:17:25.503524 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-04f5bb8f-4cd9-4ef9-b4e1-f63fe01cea11 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:17:25.640413 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-04f5bb8f-4cd9-4ef9-b4e1-f63fe01cea11 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4b8c7ad3cc6240c096c2c7cf31d8f74f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a45a8ce6687c427d8f76e4205a076a93', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:17:25.648549 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-04f5bb8f-4cd9-4ef9-b4e1-f63fe01cea11 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "GET /compute/v2.1/servers/52817115-f50c-4d1b-a4af-b9fddc045246" status: 200 len: 1680 microversion: 2.1 time: 0.158751 Jul 16 22:17:25.649137 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3484/6959] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:17:25 2022] GET /compute/v2.1/servers/52817115-f50c-4d1b-a4af-b9fddc045246 => generated 1680 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:17:26.665877 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-c509d63d-8767-4ffc-a052-334cec042208 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:17:26.666537 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-c509d63d-8767-4ffc-a052-334cec042208 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] [instance: 52817115-f50c-4d1b-a4af-b9fddc045246] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:17:26.673369 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c509d63d-8767-4ffc-a052-334cec042208 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:17:26.673820 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c509d63d-8767-4ffc-a052-334cec042208 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:17:26.675100 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c509d63d-8767-4ffc-a052-334cec042208 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:17:26.675581 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c509d63d-8767-4ffc-a052-334cec042208 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:17:26.807904 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-c509d63d-8767-4ffc-a052-334cec042208 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4b8c7ad3cc6240c096c2c7cf31d8f74f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a45a8ce6687c427d8f76e4205a076a93', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:17:26.816925 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-c509d63d-8767-4ffc-a052-334cec042208 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "GET /compute/v2.1/servers/52817115-f50c-4d1b-a4af-b9fddc045246" status: 200 len: 1680 microversion: 2.1 time: 0.155035 Jul 16 22:17:26.817506 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3476/6960] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:17:26 2022] GET /compute/v2.1/servers/52817115-f50c-4d1b-a4af-b9fddc045246 => generated 1680 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:17:27.835927 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-4964324f-dcee-4d70-8df0-3c125ea11f71 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:17:27.836563 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-4964324f-dcee-4d70-8df0-3c125ea11f71 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] [instance: 52817115-f50c-4d1b-a4af-b9fddc045246] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:17:27.843585 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4964324f-dcee-4d70-8df0-3c125ea11f71 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:17:27.844048 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4964324f-dcee-4d70-8df0-3c125ea11f71 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:17:27.844819 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4964324f-dcee-4d70-8df0-3c125ea11f71 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:17:27.845271 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4964324f-dcee-4d70-8df0-3c125ea11f71 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:17:27.982189 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-4964324f-dcee-4d70-8df0-3c125ea11f71 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4b8c7ad3cc6240c096c2c7cf31d8f74f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a45a8ce6687c427d8f76e4205a076a93', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:17:27.990426 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-4964324f-dcee-4d70-8df0-3c125ea11f71 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "GET /compute/v2.1/servers/52817115-f50c-4d1b-a4af-b9fddc045246" status: 200 len: 1680 microversion: 2.1 time: 0.159505 Jul 16 22:17:27.991001 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3485/6961] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:17:27 2022] GET /compute/v2.1/servers/52817115-f50c-4d1b-a4af-b9fddc045246 => generated 1680 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:17:29.007782 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-642ac7e2-3df5-47fe-9885-7dba6ca5e068 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:17:29.008396 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-642ac7e2-3df5-47fe-9885-7dba6ca5e068 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] [instance: 52817115-f50c-4d1b-a4af-b9fddc045246] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:17:29.015533 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-642ac7e2-3df5-47fe-9885-7dba6ca5e068 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:17:29.015991 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-642ac7e2-3df5-47fe-9885-7dba6ca5e068 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:17:29.016759 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-642ac7e2-3df5-47fe-9885-7dba6ca5e068 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:17:29.017204 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-642ac7e2-3df5-47fe-9885-7dba6ca5e068 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:17:29.155610 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-642ac7e2-3df5-47fe-9885-7dba6ca5e068 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4b8c7ad3cc6240c096c2c7cf31d8f74f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a45a8ce6687c427d8f76e4205a076a93', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:17:29.163283 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-642ac7e2-3df5-47fe-9885-7dba6ca5e068 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "GET /compute/v2.1/servers/52817115-f50c-4d1b-a4af-b9fddc045246" status: 200 len: 1680 microversion: 2.1 time: 0.159408 Jul 16 22:17:29.163852 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3477/6962] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:17:29 2022] GET /compute/v2.1/servers/52817115-f50c-4d1b-a4af-b9fddc045246 => generated 1680 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:17:30.180330 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-599c6a4d-af2a-488f-a017-e8faf67304b2 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:17:30.180940 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-599c6a4d-af2a-488f-a017-e8faf67304b2 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] [instance: 52817115-f50c-4d1b-a4af-b9fddc045246] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:17:30.188254 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-599c6a4d-af2a-488f-a017-e8faf67304b2 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:17:30.188721 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-599c6a4d-af2a-488f-a017-e8faf67304b2 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:17:30.189539 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-599c6a4d-af2a-488f-a017-e8faf67304b2 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:17:30.189996 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-599c6a4d-af2a-488f-a017-e8faf67304b2 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:17:30.329993 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-599c6a4d-af2a-488f-a017-e8faf67304b2 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4b8c7ad3cc6240c096c2c7cf31d8f74f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a45a8ce6687c427d8f76e4205a076a93', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:17:30.338262 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-599c6a4d-af2a-488f-a017-e8faf67304b2 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "GET /compute/v2.1/servers/52817115-f50c-4d1b-a4af-b9fddc045246" status: 200 len: 1680 microversion: 2.1 time: 0.161946 Jul 16 22:17:30.338865 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3486/6963] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:17:30 2022] GET /compute/v2.1/servers/52817115-f50c-4d1b-a4af-b9fddc045246 => generated 1680 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:17:31.356184 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-d1147edf-ad64-4063-bdaa-99974beab568 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:17:31.356799 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-d1147edf-ad64-4063-bdaa-99974beab568 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] [instance: 52817115-f50c-4d1b-a4af-b9fddc045246] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:17:31.364288 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d1147edf-ad64-4063-bdaa-99974beab568 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:17:31.364738 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d1147edf-ad64-4063-bdaa-99974beab568 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:17:31.365499 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d1147edf-ad64-4063-bdaa-99974beab568 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:17:31.365935 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d1147edf-ad64-4063-bdaa-99974beab568 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:17:31.497952 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-d1147edf-ad64-4063-bdaa-99974beab568 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4b8c7ad3cc6240c096c2c7cf31d8f74f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a45a8ce6687c427d8f76e4205a076a93', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:17:31.505787 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-d1147edf-ad64-4063-bdaa-99974beab568 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "GET /compute/v2.1/servers/52817115-f50c-4d1b-a4af-b9fddc045246" status: 200 len: 1680 microversion: 2.1 time: 0.153536 Jul 16 22:17:31.506350 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3478/6964] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:17:31 2022] GET /compute/v2.1/servers/52817115-f50c-4d1b-a4af-b9fddc045246 => generated 1680 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:17:32.522894 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-94b63b3d-41a5-44b3-b6d1-3861e9ea1ac0 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:17:32.523596 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-94b63b3d-41a5-44b3-b6d1-3861e9ea1ac0 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] [instance: 52817115-f50c-4d1b-a4af-b9fddc045246] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:17:32.530484 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-94b63b3d-41a5-44b3-b6d1-3861e9ea1ac0 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:17:32.530948 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-94b63b3d-41a5-44b3-b6d1-3861e9ea1ac0 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:17:32.531748 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-94b63b3d-41a5-44b3-b6d1-3861e9ea1ac0 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:17:32.532227 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-94b63b3d-41a5-44b3-b6d1-3861e9ea1ac0 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:17:32.663756 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-94b63b3d-41a5-44b3-b6d1-3861e9ea1ac0 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4b8c7ad3cc6240c096c2c7cf31d8f74f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a45a8ce6687c427d8f76e4205a076a93', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:17:32.671625 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-94b63b3d-41a5-44b3-b6d1-3861e9ea1ac0 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "GET /compute/v2.1/servers/52817115-f50c-4d1b-a4af-b9fddc045246" status: 200 len: 1680 microversion: 2.1 time: 0.152768 Jul 16 22:17:32.672188 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3487/6965] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:17:32 2022] GET /compute/v2.1/servers/52817115-f50c-4d1b-a4af-b9fddc045246 => generated 1680 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:17:33.689639 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-682ee648-9e6f-49de-aa0a-3b871656a765 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:17:33.690289 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-682ee648-9e6f-49de-aa0a-3b871656a765 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] [instance: 52817115-f50c-4d1b-a4af-b9fddc045246] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:17:33.698399 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-682ee648-9e6f-49de-aa0a-3b871656a765 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:17:33.698852 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-682ee648-9e6f-49de-aa0a-3b871656a765 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:17:33.699655 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-682ee648-9e6f-49de-aa0a-3b871656a765 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:17:33.700126 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-682ee648-9e6f-49de-aa0a-3b871656a765 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:17:33.839255 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-682ee648-9e6f-49de-aa0a-3b871656a765 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4b8c7ad3cc6240c096c2c7cf31d8f74f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a45a8ce6687c427d8f76e4205a076a93', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:17:33.846789 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-682ee648-9e6f-49de-aa0a-3b871656a765 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "GET /compute/v2.1/servers/52817115-f50c-4d1b-a4af-b9fddc045246" status: 200 len: 1680 microversion: 2.1 time: 0.161320 Jul 16 22:17:33.847807 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3479/6966] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:17:33 2022] GET /compute/v2.1/servers/52817115-f50c-4d1b-a4af-b9fddc045246 => generated 1680 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:17:34.864622 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-a7654972-ba9f-4411-9054-5d89daed1c8f tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:17:34.865278 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-a7654972-ba9f-4411-9054-5d89daed1c8f tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] [instance: 52817115-f50c-4d1b-a4af-b9fddc045246] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:17:34.872852 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a7654972-ba9f-4411-9054-5d89daed1c8f tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:17:34.873317 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a7654972-ba9f-4411-9054-5d89daed1c8f tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:17:34.874099 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a7654972-ba9f-4411-9054-5d89daed1c8f tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:17:34.874554 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a7654972-ba9f-4411-9054-5d89daed1c8f tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:17:35.007412 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-a7654972-ba9f-4411-9054-5d89daed1c8f tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4b8c7ad3cc6240c096c2c7cf31d8f74f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a45a8ce6687c427d8f76e4205a076a93', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:17:35.015059 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-a7654972-ba9f-4411-9054-5d89daed1c8f tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "GET /compute/v2.1/servers/52817115-f50c-4d1b-a4af-b9fddc045246" status: 200 len: 1683 microversion: 2.1 time: 0.154480 Jul 16 22:17:35.015630 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3488/6967] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:17:34 2022] GET /compute/v2.1/servers/52817115-f50c-4d1b-a4af-b9fddc045246 => generated 1683 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:17:59.200122 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-c6fb23d2-4e83-48d7-bae0-ab07cdf0103e tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Action: 'action', calling method: >, body: {"os-getConsoleOutput": {}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:17:59.200982 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-c6fb23d2-4e83-48d7-bae0-ab07cdf0103e tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] [instance: 52817115-f50c-4d1b-a4af-b9fddc045246] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:17:59.203220 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_api:SELECT=16 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:17:59.204418 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=48 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:17:59.208660 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c6fb23d2-4e83-48d7-bae0-ab07cdf0103e tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:17:59.209115 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c6fb23d2-4e83-48d7-bae0-ab07cdf0103e tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:17:59.209885 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c6fb23d2-4e83-48d7-bae0-ab07cdf0103e tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:17:59.210368 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c6fb23d2-4e83-48d7-bae0-ab07cdf0103e tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:17:59.264773 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-c6fb23d2-4e83-48d7-bae0-ab07cdf0103e tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "POST /compute/v2.1/servers/52817115-f50c-4d1b-a4af-b9fddc045246/action" status: 200 len: 68878 microversion: 2.1 time: 0.068652 Jul 16 22:17:59.265754 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3480/6968] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 22:17:59 2022] POST /compute/v2.1/servers/52817115-f50c-4d1b-a4af-b9fddc045246/action => generated 68878 bytes in 70 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 16 22:18:06.640196 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-66e20b7e-bf49-4b90-b075-ec4fa407fe8f tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:18:06.640717 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-66e20b7e-bf49-4b90-b075-ec4fa407fe8f tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] [instance: 52817115-f50c-4d1b-a4af-b9fddc045246] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:18:06.643518 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_api:SELECT=18 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:18:06.644438 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=58,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:18:06.649974 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-66e20b7e-bf49-4b90-b075-ec4fa407fe8f tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:18:06.650689 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-66e20b7e-bf49-4b90-b075-ec4fa407fe8f tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:18:06.651513 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-66e20b7e-bf49-4b90-b075-ec4fa407fe8f tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:18:06.652155 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-66e20b7e-bf49-4b90-b075-ec4fa407fe8f tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:18:06.681900 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-66e20b7e-bf49-4b90-b075-ec4fa407fe8f tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] [instance: 52817115-f50c-4d1b-a4af-b9fddc045246] Going to try to terminate instance {{(pid=138681) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 16 22:18:06.706281 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-4c77bc3a-e96e-4421-8363-7676baebf745 req-dd1f88e5-1979-4f1b-9cc9-0fd518a7b88e service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "52817115-f50c-4d1b-a4af-b9fddc045246", "tag": "6f7e4fe6-52ba-43e9-8b99-5718848c427a"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:18:06.717418 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-4c77bc3a-e96e-4421-8363-7676baebf745 req-dd1f88e5-1979-4f1b-9cc9-0fd518a7b88e service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:18:06.717880 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-4c77bc3a-e96e-4421-8363-7676baebf745 req-dd1f88e5-1979-4f1b-9cc9-0fd518a7b88e service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:18:06.729648 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-4c77bc3a-e96e-4421-8363-7676baebf745 req-dd1f88e5-1979-4f1b-9cc9-0fd518a7b88e service nova] Creating event network-changed:6f7e4fe6-52ba-43e9-8b99-5718848c427a for instance 52817115-f50c-4d1b-a4af-b9fddc045246 on nch1-master-1 Jul 16 22:18:06.734400 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-4c77bc3a-e96e-4421-8363-7676baebf745 req-dd1f88e5-1979-4f1b-9cc9-0fd518a7b88e service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.095278 Jul 16 22:18:06.735099 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3481/6969] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 22:18:06 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 22:18:06.877663 ncdv-master-1 devstack@n-api.service[138681]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jul 16 22:18:06.880419 ncdv-master-1 devstack@n-api.service[138681]: Sat Jul 16 22:18:06 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/52817115-f50c-4d1b-a4af-b9fddc045246 (ip 10.222.0.16) !!! Jul 16 22:18:06.976986 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-66e20b7e-bf49-4b90-b075-ec4fa407fe8f tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "DELETE /compute/v2.1/servers/52817115-f50c-4d1b-a4af-b9fddc045246" status: 204 len: 0 microversion: 2.1 time: 0.340582 Jul 16 22:18:06.981002 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3489/6970] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 22:18:06 2022] DELETE /compute/v2.1/servers/52817115-f50c-4d1b-a4af-b9fddc045246 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 16 22:18:06.987987 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-f485d660-00d2-4a3e-952a-6a0dd6f5181f tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:18:06.988981 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-f485d660-00d2-4a3e-952a-6a0dd6f5181f tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] [instance: 52817115-f50c-4d1b-a4af-b9fddc045246] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:18:06.995638 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f485d660-00d2-4a3e-952a-6a0dd6f5181f tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:18:06.996099 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f485d660-00d2-4a3e-952a-6a0dd6f5181f tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:18:06.996883 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f485d660-00d2-4a3e-952a-6a0dd6f5181f tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:18:06.997584 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f485d660-00d2-4a3e-952a-6a0dd6f5181f tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:18:07.134994 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-f485d660-00d2-4a3e-952a-6a0dd6f5181f tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4b8c7ad3cc6240c096c2c7cf31d8f74f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a45a8ce6687c427d8f76e4205a076a93', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:18:07.142618 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-f485d660-00d2-4a3e-952a-6a0dd6f5181f tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "GET /compute/v2.1/servers/52817115-f50c-4d1b-a4af-b9fddc045246" status: 200 len: 1689 microversion: 2.1 time: 0.160276 Jul 16 22:18:07.143449 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3482/6971] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:18:06 2022] GET /compute/v2.1/servers/52817115-f50c-4d1b-a4af-b9fddc045246 => generated 1689 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:18:08.162084 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-401eba11-f952-4e6f-9448-bf4c8d5723d8 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:18:08.163084 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-401eba11-f952-4e6f-9448-bf4c8d5723d8 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] [instance: 52817115-f50c-4d1b-a4af-b9fddc045246] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:18:08.170837 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-401eba11-f952-4e6f-9448-bf4c8d5723d8 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:18:08.171328 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-401eba11-f952-4e6f-9448-bf4c8d5723d8 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:18:08.172199 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-401eba11-f952-4e6f-9448-bf4c8d5723d8 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:18:08.172656 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-401eba11-f952-4e6f-9448-bf4c8d5723d8 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:18:08.307457 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-401eba11-f952-4e6f-9448-bf4c8d5723d8 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4b8c7ad3cc6240c096c2c7cf31d8f74f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a45a8ce6687c427d8f76e4205a076a93', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:18:08.315544 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-401eba11-f952-4e6f-9448-bf4c8d5723d8 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "GET /compute/v2.1/servers/52817115-f50c-4d1b-a4af-b9fddc045246" status: 200 len: 1689 microversion: 2.1 time: 0.158525 Jul 16 22:18:08.316117 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3490/6972] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:18:08 2022] GET /compute/v2.1/servers/52817115-f50c-4d1b-a4af-b9fddc045246 => generated 1689 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:18:09.003128 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-1c6cc90d-6712-439c-8434-691776180701 req-42928ee8-b36a-4fd7-b3ad-4360058de903 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "52817115-f50c-4d1b-a4af-b9fddc045246", "name": "network-vif-deleted", "tag": "6f7e4fe6-52ba-43e9-8b99-5718848c427a"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:18:09.012055 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-1c6cc90d-6712-439c-8434-691776180701 req-42928ee8-b36a-4fd7-b3ad-4360058de903 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:18:09.012506 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-1c6cc90d-6712-439c-8434-691776180701 req-42928ee8-b36a-4fd7-b3ad-4360058de903 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:18:09.024685 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-1c6cc90d-6712-439c-8434-691776180701 req-42928ee8-b36a-4fd7-b3ad-4360058de903 service nova] Creating event network-vif-deleted:6f7e4fe6-52ba-43e9-8b99-5718848c427a for instance 52817115-f50c-4d1b-a4af-b9fddc045246 on nch1-master-1 Jul 16 22:18:09.029491 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-1c6cc90d-6712-439c-8434-691776180701 req-42928ee8-b36a-4fd7-b3ad-4360058de903 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.083746 Jul 16 22:18:09.030101 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3483/6973] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 22:18:08 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 22:18:09.334684 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-ea42e001-7f91-4c31-9d1e-a3b371691673 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:18:09.335392 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-ea42e001-7f91-4c31-9d1e-a3b371691673 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] [instance: 52817115-f50c-4d1b-a4af-b9fddc045246] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:18:09.342511 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ea42e001-7f91-4c31-9d1e-a3b371691673 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:18:09.342983 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ea42e001-7f91-4c31-9d1e-a3b371691673 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:18:09.343835 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ea42e001-7f91-4c31-9d1e-a3b371691673 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:18:09.344294 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ea42e001-7f91-4c31-9d1e-a3b371691673 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:18:09.379031 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ea42e001-7f91-4c31-9d1e-a3b371691673 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:18:09.379504 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ea42e001-7f91-4c31-9d1e-a3b371691673 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:18:09.380528 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.objects.instance [None req-ea42e001-7f91-4c31-9d1e-a3b371691673 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lazy-loading 'fault' on Instance uuid 52817115-f50c-4d1b-a4af-b9fddc045246 {{(pid=138681) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 22:18:09.422599 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-ea42e001-7f91-4c31-9d1e-a3b371691673 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4b8c7ad3cc6240c096c2c7cf31d8f74f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a45a8ce6687c427d8f76e4205a076a93', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:18:09.431306 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-ea42e001-7f91-4c31-9d1e-a3b371691673 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "GET /compute/v2.1/servers/52817115-f50c-4d1b-a4af-b9fddc045246" status: 200 len: 1356 microversion: 2.1 time: 0.101366 Jul 16 22:18:09.431879 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3491/6974] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:18:09 2022] GET /compute/v2.1/servers/52817115-f50c-4d1b-a4af-b9fddc045246 => generated 1356 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:18:10.451072 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-f9a1d2fd-9883-438a-9e90-66bb1b819663 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:18:10.451794 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-f9a1d2fd-9883-438a-9e90-66bb1b819663 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] [instance: 52817115-f50c-4d1b-a4af-b9fddc045246] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:18:10.458856 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f9a1d2fd-9883-438a-9e90-66bb1b819663 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:18:10.459367 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f9a1d2fd-9883-438a-9e90-66bb1b819663 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:18:10.460138 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f9a1d2fd-9883-438a-9e90-66bb1b819663 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:18:10.460582 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f9a1d2fd-9883-438a-9e90-66bb1b819663 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:18:10.476762 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-f9a1d2fd-9883-438a-9e90-66bb1b819663 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] HTTP exception thrown: Instance 52817115-f50c-4d1b-a4af-b9fddc045246 could not be found. Jul 16 22:18:10.478581 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-f9a1d2fd-9883-438a-9e90-66bb1b819663 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Request method failure captured: Jul 16 22:18:10.478581 ncdv-master-1 devstack@n-api.service[138680]: request: GET /compute/v2.1/servers/52817115-f50c-4d1b-a4af-b9fddc045246 HTTP/1.1 Jul 16 22:18:10.478581 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 22:18:10.478581 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 22:18:10.478581 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 22:18:10.478581 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 0 Jul 16 22:18:10.478581 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 22:18:10.478581 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 22:18:10.478581 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 22:18:10.478581 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 22:18:10.478581 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0zhtDg4n0Cr97EpGJw_kphX8HozEdWitmpiSsnN8liwN7JwqYkEqcaIM-47rjt-yK-e5LgZo6pFGCSkRgNS-xnBjvFiClGjWelkYApUjhWHqw4F9MI58oj4QDcEbMKfIlL82tbwRwh1LdKROeHuRqnWXe9oskU-rprKG2IX2dbA4fe4 Jul 16 22:18:10.478581 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 22:18:10.478581 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 22:18:10.478581 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 22:18:10.478581 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 22:18:10.478581 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 22:18:10.478581 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 22:18:10.478581 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: a45a8ce6687c427d8f76e4205a076a93 Jul 16 22:18:10.478581 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-TestNetworkAdvancedServerOps-234580283 Jul 16 22:18:10.478581 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 22:18:10.478581 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 22:18:10.478581 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/a45a8ce6687c427d8f76e4205a076a93"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/a45a8ce6687c427d8f76e4205a076a93"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_a45a8ce6687c427d8f76e4205a076a93"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/a45a8ce6687c427d8f76e4205a076a93"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 22:18:10.478581 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-TestNetworkAdvancedServerOps-234580283 Jul 16 22:18:10.478581 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: a45a8ce6687c427d8f76e4205a076a93 Jul 16 22:18:10.478581 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-TestNetworkAdvancedServerOps-234580283 Jul 16 22:18:10.478581 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-TestNetworkAdvancedServerOps-234580283-project Jul 16 22:18:10.478581 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 22:18:10.478581 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 22:18:10.478581 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: 4b8c7ad3cc6240c096c2c7cf31d8f74f Jul 16 22:18:10.478581 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-TestNetworkAdvancedServerOps-234580283-project Jul 16 22:18:10.478581 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 22:18:10.478581 ncdv-master-1 devstack@n-api.service[138680]: exception: Instance 52817115-f50c-4d1b-a4af-b9fddc045246 could not be found. Jul 16 22:18:10.478581 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': '52817115-f50c-4d1b-a4af-b9fddc045246'} Jul 16 22:18:10.478581 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 22:18:10.478581 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 22:18:10.478581 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 22:18:10.478581 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 22:18:10.478581 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 22:18:10.478581 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 22:18:10.478581 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 22:18:10.478581 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 22:18:10.478581 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 22:18:10.478581 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 22:18:10.478581 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 52817115-f50c-4d1b-a4af-b9fddc045246 could not be found. Jul 16 22:18:10.478581 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 22:18:10.478581 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 22:18:10.478581 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 22:18:10.478581 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 22:18:10.478581 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 22:18:10.478581 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 22:18:10.478581 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 22:18:10.478581 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 22:18:10.478581 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 22:18:10.478581 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 22:18:10.478581 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 22:18:10.478581 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 22:18:10.478581 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 22:18:10.478581 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 22:18:10.478581 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 22:18:10.478581 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 22:18:10.478581 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 52817115-f50c-4d1b-a4af-b9fddc045246 could not be found. Jul 16 22:18:10.478581 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 22:18:10.478581 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 22:18:10.478581 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 22:18:10.483459 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 22:18:10.483459 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 22:18:10.483459 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 22:18:10.483459 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 22:18:10.483459 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 22:18:10.483459 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 52817115-f50c-4d1b-a4af-b9fddc045246 could not be found. Jul 16 22:18:10.483459 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 22:18:10.483459 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-f9a1d2fd-9883-438a-9e90-66bb1b819663 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Returning 404 to user: Instance 52817115-f50c-4d1b-a4af-b9fddc045246 could not be found. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 22:18:10.483459 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-f9a1d2fd-9883-438a-9e90-66bb1b819663 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "GET /compute/v2.1/servers/52817115-f50c-4d1b-a4af-b9fddc045246" status: 404 len: 111 microversion: 2.1 time: 0.034595 Jul 16 22:18:10.483459 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3484/6975] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:18:10 2022] GET /compute/v2.1/servers/52817115-f50c-4d1b-a4af-b9fddc045246 => generated 111 bytes in 36 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 22:18:14.358390 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-14d2c87f-454a-45f7-8d63-efdcc4ec2c66 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Calling method '.version_select at 0x7fdda2ac0c10>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:18:14.419552 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-14d2c87f-454a-45f7-8d63-efdcc4ec2c66 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "DELETE /compute/v2.1/os-keypairs/tempest-TestNetworkAdvancedServerOps-690605879" status: 202 len: 0 microversion: 2.1 time: 0.063198 Jul 16 22:18:14.419552 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3492/6976] 10.222.0.16 () {58 vars in 1280 bytes} [Sat Jul 16 22:18:14 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestNetworkAdvancedServerOps-690605879 => generated 0 bytes in 65 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 22:18:14.451818 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-d96522a3-5182-4eee-94ec-c3f451d6eb7a tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Action: 'create', calling method: .version_select at 0x7fdd8fa48280>, body: {"keypair": {"name": "tempest-TestNetworkAdvancedServerOps-1845463979", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBHq/x4P/nFX48xsSXdpNzfBMP+qz/HrIuqyA4WuHfKru5T5mV+x2Pf77o32qTN8WLt1JAoFTGIlPZ7I6QslDJ7Bebh+43Em8KclBkn6kv/BTihgdpYjVkvXpkfXaqIP0Gw=="}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:18:14.467424 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-d96522a3-5182-4eee-94ec-c3f451d6eb7a tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Getting quotas for project a45a8ce6687c427d8f76e4205a076a93. Resources: {'key_pairs'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 22:18:14.474341 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-d96522a3-5182-4eee-94ec-c3f451d6eb7a tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Getting quotas for user 4b8c7ad3cc6240c096c2c7cf31d8f74f and project a45a8ce6687c427d8f76e4205a076a93. Resources: {'key_pairs'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 22:18:14.570277 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-d96522a3-5182-4eee-94ec-c3f451d6eb7a tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "POST /compute/v2.1/os-keypairs" status: 200 len: 407 microversion: 2.1 time: 0.122395 Jul 16 22:18:14.570964 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3485/6977] 10.222.0.16 () {60 vars in 1158 bytes} [Sat Jul 16 22:18:14 2022] POST /compute/v2.1/os-keypairs => generated 407 bytes in 124 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 22:18:22.955679 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-dbe57d7c-2274-4b4b-8c90-5a1eed7eb5dc tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestNetworkAdvancedServerOps-server-1042764939", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452", "networks": [{"uuid": "d706f8be-abaa-4fac-b1c6-ad63d4e53cde"}], "key_name": "tempest-TestNetworkAdvancedServerOps-1845463979", "security_groups": [{"name": "tempest-secgroup-smoke-302578737"}]}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:18:22.967853 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=28,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:18:23.116345 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.network.neutron [None req-dbe57d7c-2274-4b4b-8c90-5a1eed7eb5dc tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] validate_networks() for [('d706f8be-abaa-4fac-b1c6-ad63d4e53cde', None, None, None, None, None)] {{(pid=138681) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 16 22:18:23.533002 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-dbe57d7c-2274-4b4b-8c90-5a1eed7eb5dc tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Flavor limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 16 22:18:23.533438 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-dbe57d7c-2274-4b4b-8c90-5a1eed7eb5dc tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Image limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 16 22:18:23.534336 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-dbe57d7c-2274-4b4b-8c90-5a1eed7eb5dc tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Flavor pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 16 22:18:23.534773 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-dbe57d7c-2274-4b4b-8c90-5a1eed7eb5dc tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Image pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 16 22:18:23.535345 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-dbe57d7c-2274-4b4b-8c90-5a1eed7eb5dc tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 16 22:18:23.545455 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-dbe57d7c-2274-4b4b-8c90-5a1eed7eb5dc tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:18:23.545925 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-dbe57d7c-2274-4b4b-8c90-5a1eed7eb5dc tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:18:23.546739 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-dbe57d7c-2274-4b4b-8c90-5a1eed7eb5dc tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:18:23.547219 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-dbe57d7c-2274-4b4b-8c90-5a1eed7eb5dc tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:18:23.566076 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-dbe57d7c-2274-4b4b-8c90-5a1eed7eb5dc tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Getting quotas for project a45a8ce6687c427d8f76e4205a076a93. Resources: {'ram', 'instances', 'cores'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 22:18:23.571928 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-dbe57d7c-2274-4b4b-8c90-5a1eed7eb5dc tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Getting quotas for user 4b8c7ad3cc6240c096c2c7cf31d8f74f and project a45a8ce6687c427d8f76e4205a076a93. Resources: {'ram', 'instances', 'cores'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 22:18:23.588117 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-dbe57d7c-2274-4b4b-8c90-5a1eed7eb5dc tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Going to run 1 instances... {{(pid=138681) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 16 22:18:23.590753 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-dbe57d7c-2274-4b4b-8c90-5a1eed7eb5dc tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] [instance: a0d12e80-5ef8-408c-a4d3-92da4c085d85] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138681) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 16 22:18:23.726203 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-dbe57d7c-2274-4b4b-8c90-5a1eed7eb5dc tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 405 microversion: 2.1 time: 0.774399 Jul 16 22:18:23.731844 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3493/6978] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 22:18:22 2022] POST /compute/v2.1/servers => generated 405 bytes in 776 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 16 22:18:23.739194 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-be5c8c7b-6fe2-4a6c-80c9-151eed5fc471 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:18:23.739194 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-be5c8c7b-6fe2-4a6c-80c9-151eed5fc471 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] [instance: a0d12e80-5ef8-408c-a4d3-92da4c085d85] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:18:23.745945 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=16 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:18:23.808824 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-be5c8c7b-6fe2-4a6c-80c9-151eed5fc471 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4b8c7ad3cc6240c096c2c7cf31d8f74f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a45a8ce6687c427d8f76e4205a076a93', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:18:23.818157 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-be5c8c7b-6fe2-4a6c-80c9-151eed5fc471 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "GET /compute/v2.1/servers/a0d12e80-5ef8-408c-a4d3-92da4c085d85" status: 200 len: 1267 microversion: 2.1 time: 0.084405 Jul 16 22:18:23.823525 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3486/6979] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:18:23 2022] GET /compute/v2.1/servers/a0d12e80-5ef8-408c-a4d3-92da4c085d85 => generated 1267 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:18:24.838531 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-cad77cf9-e414-4335-bfa4-46b7683b1e51 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:18:24.839205 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-cad77cf9-e414-4335-bfa4-46b7683b1e51 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] [instance: a0d12e80-5ef8-408c-a4d3-92da4c085d85] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:18:24.846856 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cad77cf9-e414-4335-bfa4-46b7683b1e51 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:18:24.847627 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cad77cf9-e414-4335-bfa4-46b7683b1e51 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:18:24.850139 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cad77cf9-e414-4335-bfa4-46b7683b1e51 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:18:24.850602 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cad77cf9-e414-4335-bfa4-46b7683b1e51 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:18:24.929794 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-cad77cf9-e414-4335-bfa4-46b7683b1e51 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4b8c7ad3cc6240c096c2c7cf31d8f74f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a45a8ce6687c427d8f76e4205a076a93', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:18:24.938255 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-cad77cf9-e414-4335-bfa4-46b7683b1e51 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "GET /compute/v2.1/servers/a0d12e80-5ef8-408c-a4d3-92da4c085d85" status: 200 len: 1271 microversion: 2.1 time: 0.106861 Jul 16 22:18:24.938909 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3494/6980] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:18:24 2022] GET /compute/v2.1/servers/a0d12e80-5ef8-408c-a4d3-92da4c085d85 => generated 1271 bytes in 108 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 22:18:25.956355 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-a6e35ece-196f-4664-8cfe-8f4accfba43f tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:18:25.956981 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-a6e35ece-196f-4664-8cfe-8f4accfba43f tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] [instance: a0d12e80-5ef8-408c-a4d3-92da4c085d85] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:18:25.964212 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a6e35ece-196f-4664-8cfe-8f4accfba43f tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:18:25.964680 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a6e35ece-196f-4664-8cfe-8f4accfba43f tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:18:25.965463 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a6e35ece-196f-4664-8cfe-8f4accfba43f tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:18:25.965912 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a6e35ece-196f-4664-8cfe-8f4accfba43f tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:18:26.054241 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-a6e35ece-196f-4664-8cfe-8f4accfba43f tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4b8c7ad3cc6240c096c2c7cf31d8f74f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a45a8ce6687c427d8f76e4205a076a93', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:18:26.062211 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-a6e35ece-196f-4664-8cfe-8f4accfba43f tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "GET /compute/v2.1/servers/a0d12e80-5ef8-408c-a4d3-92da4c085d85" status: 200 len: 1327 microversion: 2.1 time: 0.109690 Jul 16 22:18:26.062797 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3487/6981] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:18:25 2022] GET /compute/v2.1/servers/a0d12e80-5ef8-408c-a4d3-92da4c085d85 => generated 1327 bytes in 111 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:18:27.078765 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-1cfe5f79-b51f-466f-a216-5828b8d666a4 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:18:27.079581 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-1cfe5f79-b51f-466f-a216-5828b8d666a4 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] [instance: a0d12e80-5ef8-408c-a4d3-92da4c085d85] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:18:27.087668 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1cfe5f79-b51f-466f-a216-5828b8d666a4 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:18:27.088137 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1cfe5f79-b51f-466f-a216-5828b8d666a4 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:18:27.088923 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1cfe5f79-b51f-466f-a216-5828b8d666a4 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:18:27.089412 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1cfe5f79-b51f-466f-a216-5828b8d666a4 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:18:27.527501 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-1cfe5f79-b51f-466f-a216-5828b8d666a4 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4b8c7ad3cc6240c096c2c7cf31d8f74f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a45a8ce6687c427d8f76e4205a076a93', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:18:27.535136 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-1cfe5f79-b51f-466f-a216-5828b8d666a4 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "GET /compute/v2.1/servers/a0d12e80-5ef8-408c-a4d3-92da4c085d85" status: 200 len: 1392 microversion: 2.1 time: 0.460296 Jul 16 22:18:27.536701 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3495/6982] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:18:27 2022] GET /compute/v2.1/servers/a0d12e80-5ef8-408c-a4d3-92da4c085d85 => generated 1392 bytes in 462 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:18:28.131401 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-e182a31a-86f3-4108-aec9-6250b848d6c4 req-b55fa038-168f-4da3-b71e-cb6d3761de63 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "a0d12e80-5ef8-408c-a4d3-92da4c085d85", "tag": "35c4a2dc-4c5a-4052-bb2c-1cbc5e17f6ea"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:18:28.145927 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-e182a31a-86f3-4108-aec9-6250b848d6c4 req-b55fa038-168f-4da3-b71e-cb6d3761de63 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:18:28.146687 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-e182a31a-86f3-4108-aec9-6250b848d6c4 req-b55fa038-168f-4da3-b71e-cb6d3761de63 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:18:28.161718 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-e182a31a-86f3-4108-aec9-6250b848d6c4 req-b55fa038-168f-4da3-b71e-cb6d3761de63 service nova] Creating event network-changed:35c4a2dc-4c5a-4052-bb2c-1cbc5e17f6ea for instance a0d12e80-5ef8-408c-a4d3-92da4c085d85 on nch1-master-1 Jul 16 22:18:28.167421 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-e182a31a-86f3-4108-aec9-6250b848d6c4 req-b55fa038-168f-4da3-b71e-cb6d3761de63 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.040421 Jul 16 22:18:28.168118 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3488/6983] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 22:18:28 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 22:18:28.554815 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-86d07f2e-95aa-4d65-bd95-cfb518443a81 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:18:28.555480 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-86d07f2e-95aa-4d65-bd95-cfb518443a81 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] [instance: a0d12e80-5ef8-408c-a4d3-92da4c085d85] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:18:28.562725 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-86d07f2e-95aa-4d65-bd95-cfb518443a81 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:18:28.563315 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-86d07f2e-95aa-4d65-bd95-cfb518443a81 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:18:28.564127 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-86d07f2e-95aa-4d65-bd95-cfb518443a81 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:18:28.564988 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-86d07f2e-95aa-4d65-bd95-cfb518443a81 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:18:28.749035 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-86d07f2e-95aa-4d65-bd95-cfb518443a81 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4b8c7ad3cc6240c096c2c7cf31d8f74f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a45a8ce6687c427d8f76e4205a076a93', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:18:28.757150 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-86d07f2e-95aa-4d65-bd95-cfb518443a81 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "GET /compute/v2.1/servers/a0d12e80-5ef8-408c-a4d3-92da4c085d85" status: 200 len: 1392 microversion: 2.1 time: 0.208088 Jul 16 22:18:28.757731 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3496/6984] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:18:28 2022] GET /compute/v2.1/servers/a0d12e80-5ef8-408c-a4d3-92da4c085d85 => generated 1392 bytes in 209 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:18:29.774629 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-8ca24c46-bf3b-49ae-8fe5-fa589465738b tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:18:29.775363 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-8ca24c46-bf3b-49ae-8fe5-fa589465738b tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] [instance: a0d12e80-5ef8-408c-a4d3-92da4c085d85] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:18:29.778159 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_api:SELECT=30,nova_api:INSERT=1 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:18:29.783301 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8ca24c46-bf3b-49ae-8fe5-fa589465738b tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:18:29.783763 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8ca24c46-bf3b-49ae-8fe5-fa589465738b tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:18:29.784538 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8ca24c46-bf3b-49ae-8fe5-fa589465738b tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:18:29.784988 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8ca24c46-bf3b-49ae-8fe5-fa589465738b tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:18:29.918745 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-8ca24c46-bf3b-49ae-8fe5-fa589465738b tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4b8c7ad3cc6240c096c2c7cf31d8f74f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a45a8ce6687c427d8f76e4205a076a93', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:18:29.927120 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-8ca24c46-bf3b-49ae-8fe5-fa589465738b tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "GET /compute/v2.1/servers/a0d12e80-5ef8-408c-a4d3-92da4c085d85" status: 200 len: 1392 microversion: 2.1 time: 0.156627 Jul 16 22:18:29.927703 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3489/6985] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:18:29 2022] GET /compute/v2.1/servers/a0d12e80-5ef8-408c-a4d3-92da4c085d85 => generated 1392 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:18:30.945156 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-393c531a-ebd2-4f7d-8c71-a48df8651c56 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:18:30.945735 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-393c531a-ebd2-4f7d-8c71-a48df8651c56 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] [instance: a0d12e80-5ef8-408c-a4d3-92da4c085d85] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:18:30.952724 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-393c531a-ebd2-4f7d-8c71-a48df8651c56 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:18:30.953191 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-393c531a-ebd2-4f7d-8c71-a48df8651c56 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:18:30.953961 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-393c531a-ebd2-4f7d-8c71-a48df8651c56 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:18:30.954412 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-393c531a-ebd2-4f7d-8c71-a48df8651c56 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:18:31.091641 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-393c531a-ebd2-4f7d-8c71-a48df8651c56 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4b8c7ad3cc6240c096c2c7cf31d8f74f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a45a8ce6687c427d8f76e4205a076a93', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:18:31.099653 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-393c531a-ebd2-4f7d-8c71-a48df8651c56 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "GET /compute/v2.1/servers/a0d12e80-5ef8-408c-a4d3-92da4c085d85" status: 200 len: 1392 microversion: 2.1 time: 0.158890 Jul 16 22:18:31.100229 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3497/6986] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:18:30 2022] GET /compute/v2.1/servers/a0d12e80-5ef8-408c-a4d3-92da4c085d85 => generated 1392 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:18:32.117171 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-f0173c5c-031a-4f24-8bcf-938eda3022ce tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:18:32.117809 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-f0173c5c-031a-4f24-8bcf-938eda3022ce tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] [instance: a0d12e80-5ef8-408c-a4d3-92da4c085d85] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:18:32.124755 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f0173c5c-031a-4f24-8bcf-938eda3022ce tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:18:32.125214 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f0173c5c-031a-4f24-8bcf-938eda3022ce tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:18:32.125970 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f0173c5c-031a-4f24-8bcf-938eda3022ce tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:18:32.126407 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f0173c5c-031a-4f24-8bcf-938eda3022ce tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:18:32.261854 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-f0173c5c-031a-4f24-8bcf-938eda3022ce tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4b8c7ad3cc6240c096c2c7cf31d8f74f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a45a8ce6687c427d8f76e4205a076a93', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:18:32.269430 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-f0173c5c-031a-4f24-8bcf-938eda3022ce tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "GET /compute/v2.1/servers/a0d12e80-5ef8-408c-a4d3-92da4c085d85" status: 200 len: 1392 microversion: 2.1 time: 0.156758 Jul 16 22:18:32.270084 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3490/6987] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:18:32 2022] GET /compute/v2.1/servers/a0d12e80-5ef8-408c-a4d3-92da4c085d85 => generated 1392 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:18:33.286770 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-3a9061c0-6ce8-41f5-87d7-fde8a0519965 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:18:33.287345 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-3a9061c0-6ce8-41f5-87d7-fde8a0519965 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] [instance: a0d12e80-5ef8-408c-a4d3-92da4c085d85] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:18:33.294315 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3a9061c0-6ce8-41f5-87d7-fde8a0519965 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:18:33.294784 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3a9061c0-6ce8-41f5-87d7-fde8a0519965 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:18:33.295664 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3a9061c0-6ce8-41f5-87d7-fde8a0519965 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:18:33.296122 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3a9061c0-6ce8-41f5-87d7-fde8a0519965 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:18:33.428744 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-3a9061c0-6ce8-41f5-87d7-fde8a0519965 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4b8c7ad3cc6240c096c2c7cf31d8f74f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a45a8ce6687c427d8f76e4205a076a93', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:18:33.436641 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-3a9061c0-6ce8-41f5-87d7-fde8a0519965 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "GET /compute/v2.1/servers/a0d12e80-5ef8-408c-a4d3-92da4c085d85" status: 200 len: 1392 microversion: 2.1 time: 0.153885 Jul 16 22:18:33.450048 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3498/6988] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:18:33 2022] GET /compute/v2.1/servers/a0d12e80-5ef8-408c-a4d3-92da4c085d85 => generated 1392 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:18:34.350855 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell0:SELECT=6 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:18:34.408467 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-bc85eaa6-a8df-477d-b049-81516457674e req-16854c17-a046-4206-a330-70ad659acca9 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "a0d12e80-5ef8-408c-a4d3-92da4c085d85", "name": "network-vif-plugged", "status": "completed", "tag": "35c4a2dc-4c5a-4052-bb2c-1cbc5e17f6ea"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:18:34.413255 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-bc85eaa6-a8df-477d-b049-81516457674e req-16854c17-a046-4206-a330-70ad659acca9 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:18:34.413779 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-bc85eaa6-a8df-477d-b049-81516457674e req-16854c17-a046-4206-a330-70ad659acca9 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:18:34.424638 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-bc85eaa6-a8df-477d-b049-81516457674e req-16854c17-a046-4206-a330-70ad659acca9 service nova] Creating event network-vif-plugged:35c4a2dc-4c5a-4052-bb2c-1cbc5e17f6ea for instance a0d12e80-5ef8-408c-a4d3-92da4c085d85 on nch1-master-1 Jul 16 22:18:34.430250 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-bc85eaa6-a8df-477d-b049-81516457674e req-16854c17-a046-4206-a330-70ad659acca9 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.084712 Jul 16 22:18:34.430912 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3491/6989] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 22:18:34 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 16 22:18:34.454025 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-96bc399b-8624-4fd1-a9b2-83e50248e25d tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:18:34.455533 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-96bc399b-8624-4fd1-a9b2-83e50248e25d tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] [instance: a0d12e80-5ef8-408c-a4d3-92da4c085d85] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:18:34.462037 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell0:SELECT=2 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:18:34.465925 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell0:SELECT=6 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:18:34.465925 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-96bc399b-8624-4fd1-a9b2-83e50248e25d tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:18:34.466611 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-96bc399b-8624-4fd1-a9b2-83e50248e25d tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:18:34.467679 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-96bc399b-8624-4fd1-a9b2-83e50248e25d tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:18:34.468144 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-96bc399b-8624-4fd1-a9b2-83e50248e25d tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:18:34.677520 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-96bc399b-8624-4fd1-a9b2-83e50248e25d tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4b8c7ad3cc6240c096c2c7cf31d8f74f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a45a8ce6687c427d8f76e4205a076a93', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:18:34.686357 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-96bc399b-8624-4fd1-a9b2-83e50248e25d tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "GET /compute/v2.1/servers/a0d12e80-5ef8-408c-a4d3-92da4c085d85" status: 200 len: 1392 microversion: 2.1 time: 0.236429 Jul 16 22:18:34.686944 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3499/6990] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:18:34 2022] GET /compute/v2.1/servers/a0d12e80-5ef8-408c-a4d3-92da4c085d85 => generated 1392 bytes in 238 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:18:35.705820 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-f74b0b86-e2b0-4a95-8f2b-c1394b89f3a8 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:18:35.706530 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-f74b0b86-e2b0-4a95-8f2b-c1394b89f3a8 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] [instance: a0d12e80-5ef8-408c-a4d3-92da4c085d85] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:18:35.713539 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f74b0b86-e2b0-4a95-8f2b-c1394b89f3a8 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:18:35.713998 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f74b0b86-e2b0-4a95-8f2b-c1394b89f3a8 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:18:35.714766 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f74b0b86-e2b0-4a95-8f2b-c1394b89f3a8 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:18:35.715247 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f74b0b86-e2b0-4a95-8f2b-c1394b89f3a8 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:18:35.856873 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-f74b0b86-e2b0-4a95-8f2b-c1394b89f3a8 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4b8c7ad3cc6240c096c2c7cf31d8f74f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a45a8ce6687c427d8f76e4205a076a93', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:18:35.864605 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-f74b0b86-e2b0-4a95-8f2b-c1394b89f3a8 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "GET /compute/v2.1/servers/a0d12e80-5ef8-408c-a4d3-92da4c085d85" status: 200 len: 1392 microversion: 2.1 time: 0.163885 Jul 16 22:18:35.865190 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3492/6991] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:18:35 2022] GET /compute/v2.1/servers/a0d12e80-5ef8-408c-a4d3-92da4c085d85 => generated 1392 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 22:18:36.881424 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-280b420b-5b7a-42ed-968f-cee575697bdf tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:18:36.882189 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-280b420b-5b7a-42ed-968f-cee575697bdf tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] [instance: a0d12e80-5ef8-408c-a4d3-92da4c085d85] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:18:36.884913 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_api:SELECT=44,nova_api:UPDATE=1,nova_api:DELETE=1,nova_api:INSERT=3 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:18:36.889700 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-280b420b-5b7a-42ed-968f-cee575697bdf tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:18:36.890163 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-280b420b-5b7a-42ed-968f-cee575697bdf tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:18:36.890930 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-280b420b-5b7a-42ed-968f-cee575697bdf tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:18:36.891407 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-280b420b-5b7a-42ed-968f-cee575697bdf tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:18:37.024928 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-280b420b-5b7a-42ed-968f-cee575697bdf tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4b8c7ad3cc6240c096c2c7cf31d8f74f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a45a8ce6687c427d8f76e4205a076a93', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:18:37.032483 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-280b420b-5b7a-42ed-968f-cee575697bdf tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "GET /compute/v2.1/servers/a0d12e80-5ef8-408c-a4d3-92da4c085d85" status: 200 len: 1563 microversion: 2.1 time: 0.155254 Jul 16 22:18:37.033064 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3500/6992] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:18:36 2022] GET /compute/v2.1/servers/a0d12e80-5ef8-408c-a4d3-92da4c085d85 => generated 1563 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:18:37.049324 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-717ca7de-fdad-4337-bcdb-5d32a616a9e3 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:18:37.049957 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-717ca7de-fdad-4337-bcdb-5d32a616a9e3 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] [instance: a0d12e80-5ef8-408c-a4d3-92da4c085d85] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:18:37.056869 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-717ca7de-fdad-4337-bcdb-5d32a616a9e3 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:18:37.057331 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-717ca7de-fdad-4337-bcdb-5d32a616a9e3 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:18:37.058099 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-717ca7de-fdad-4337-bcdb-5d32a616a9e3 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:18:37.058545 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-717ca7de-fdad-4337-bcdb-5d32a616a9e3 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:18:37.466358 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-717ca7de-fdad-4337-bcdb-5d32a616a9e3 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4b8c7ad3cc6240c096c2c7cf31d8f74f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a45a8ce6687c427d8f76e4205a076a93', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:18:37.474210 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-717ca7de-fdad-4337-bcdb-5d32a616a9e3 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "GET /compute/v2.1/servers/a0d12e80-5ef8-408c-a4d3-92da4c085d85" status: 200 len: 1563 microversion: 2.1 time: 0.428749 Jul 16 22:18:37.474787 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3493/6993] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:18:37 2022] GET /compute/v2.1/servers/a0d12e80-5ef8-408c-a4d3-92da4c085d85 => generated 1563 bytes in 430 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:18:39.565983 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-5523d00e-1c08-4ade-8abb-d97cc6773e3c tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:18:39.566722 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-5523d00e-1c08-4ade-8abb-d97cc6773e3c tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] [instance: a0d12e80-5ef8-408c-a4d3-92da4c085d85] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:18:39.570980 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-7df11c32-0ab0-4cdb-9c72-cb288b9f442e req-221367c5-ea43-4aba-bec1-ee80562b5027 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "a0d12e80-5ef8-408c-a4d3-92da4c085d85", "tag": "35c4a2dc-4c5a-4052-bb2c-1cbc5e17f6ea"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:18:39.574603 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5523d00e-1c08-4ade-8abb-d97cc6773e3c tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:18:39.575101 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5523d00e-1c08-4ade-8abb-d97cc6773e3c tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:18:39.576098 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5523d00e-1c08-4ade-8abb-d97cc6773e3c tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:18:39.576549 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5523d00e-1c08-4ade-8abb-d97cc6773e3c tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:18:39.580285 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-7df11c32-0ab0-4cdb-9c72-cb288b9f442e req-221367c5-ea43-4aba-bec1-ee80562b5027 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:18:39.580819 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-7df11c32-0ab0-4cdb-9c72-cb288b9f442e req-221367c5-ea43-4aba-bec1-ee80562b5027 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:18:39.592244 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-7df11c32-0ab0-4cdb-9c72-cb288b9f442e req-221367c5-ea43-4aba-bec1-ee80562b5027 service nova] Creating event network-changed:35c4a2dc-4c5a-4052-bb2c-1cbc5e17f6ea for instance a0d12e80-5ef8-408c-a4d3-92da4c085d85 on nch1-master-1 Jul 16 22:18:39.597521 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-7df11c32-0ab0-4cdb-9c72-cb288b9f442e req-221367c5-ea43-4aba-bec1-ee80562b5027 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.031226 Jul 16 22:18:39.598454 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3494/6994] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 22:18:39 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 22:18:39.867148 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-5523d00e-1c08-4ade-8abb-d97cc6773e3c tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4b8c7ad3cc6240c096c2c7cf31d8f74f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a45a8ce6687c427d8f76e4205a076a93', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:18:39.874857 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-5523d00e-1c08-4ade-8abb-d97cc6773e3c tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "GET /compute/v2.1/servers/a0d12e80-5ef8-408c-a4d3-92da4c085d85" status: 200 len: 1563 microversion: 2.1 time: 0.312656 Jul 16 22:18:39.875843 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3501/6995] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:18:39 2022] GET /compute/v2.1/servers/a0d12e80-5ef8-408c-a4d3-92da4c085d85 => generated 1563 bytes in 314 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:19:01.104328 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_api:SELECT=11 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:19:01.105119 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=36 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:19:01.109371 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-12a3082e-6fa2-4c1e-98d4-2baec9e79e59 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Action: 'action', calling method: >, body: {"os-getConsoleOutput": {}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:19:01.128242 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-12a3082e-6fa2-4c1e-98d4-2baec9e79e59 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] [instance: a0d12e80-5ef8-408c-a4d3-92da4c085d85] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:19:01.128242 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-12a3082e-6fa2-4c1e-98d4-2baec9e79e59 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:19:01.128242 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-12a3082e-6fa2-4c1e-98d4-2baec9e79e59 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:19:01.128242 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-12a3082e-6fa2-4c1e-98d4-2baec9e79e59 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:19:01.128242 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-12a3082e-6fa2-4c1e-98d4-2baec9e79e59 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:19:01.167539 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-12a3082e-6fa2-4c1e-98d4-2baec9e79e59 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "POST /compute/v2.1/servers/a0d12e80-5ef8-408c-a4d3-92da4c085d85/action" status: 200 len: 24567 microversion: 2.1 time: 0.065104 Jul 16 22:19:01.168196 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3495/6996] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 22:19:01 2022] POST /compute/v2.1/servers/a0d12e80-5ef8-408c-a4d3-92da4c085d85/action => generated 24567 bytes in 67 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 16 22:19:07.576559 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-096bb6c6-3202-41a8-96a0-0f7b90a2f0cc tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Action: 'action', calling method: >, body: {"resize": {"flavorRef": "451"}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:19:07.577636 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-096bb6c6-3202-41a8-96a0-0f7b90a2f0cc tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] [instance: a0d12e80-5ef8-408c-a4d3-92da4c085d85] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:19:07.579873 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_api:SELECT=3 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:19:07.580654 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=50 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:19:07.585825 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-096bb6c6-3202-41a8-96a0-0f7b90a2f0cc tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:19:07.589515 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-096bb6c6-3202-41a8-96a0-0f7b90a2f0cc tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:19:07.589515 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-096bb6c6-3202-41a8-96a0-0f7b90a2f0cc tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:19:07.589515 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-096bb6c6-3202-41a8-96a0-0f7b90a2f0cc tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:19:07.627258 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-096bb6c6-3202-41a8-96a0-0f7b90a2f0cc tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Policy check for compute:servers:resize:cross_cell failed with credentials {'is_admin': False, 'user_id': '4b8c7ad3cc6240c096c2c7cf31d8f74f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a45a8ce6687c427d8f76e4205a076a93', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:19:07.634360 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-096bb6c6-3202-41a8-96a0-0f7b90a2f0cc tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] [instance: a0d12e80-5ef8-408c-a4d3-92da4c085d85] Old instance type m1.heat2, new instance type m1.heat {{(pid=138681) resize /opt/stack/nova/nova/compute/api.py:4166}} Jul 16 22:19:07.640973 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-096bb6c6-3202-41a8-96a0-0f7b90a2f0cc tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Flavor limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 16 22:19:07.641406 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-096bb6c6-3202-41a8-96a0-0f7b90a2f0cc tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Image limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 16 22:19:07.641889 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-096bb6c6-3202-41a8-96a0-0f7b90a2f0cc tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Flavor pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 16 22:19:07.642311 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-096bb6c6-3202-41a8-96a0-0f7b90a2f0cc tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Image pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 16 22:19:07.642733 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-096bb6c6-3202-41a8-96a0-0f7b90a2f0cc tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 16 22:19:07.830036 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-096bb6c6-3202-41a8-96a0-0f7b90a2f0cc tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "POST /compute/v2.1/servers/a0d12e80-5ef8-408c-a4d3-92da4c085d85/action" status: 202 len: 0 microversion: 2.1 time: 0.257351 Jul 16 22:19:07.830687 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3502/6997] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 22:19:07 2022] POST /compute/v2.1/servers/a0d12e80-5ef8-408c-a4d3-92da4c085d85/action => generated 0 bytes in 259 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 22:19:07.840555 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-34b7ade2-0509-436d-99e8-00c4a397381a tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:19:07.843199 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-34b7ade2-0509-436d-99e8-00c4a397381a tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] [instance: a0d12e80-5ef8-408c-a4d3-92da4c085d85] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:19:07.849401 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-34b7ade2-0509-436d-99e8-00c4a397381a tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:19:07.850443 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-34b7ade2-0509-436d-99e8-00c4a397381a tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:19:07.851621 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-34b7ade2-0509-436d-99e8-00c4a397381a tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:19:07.852069 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-34b7ade2-0509-436d-99e8-00c4a397381a tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:19:07.991943 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-34b7ade2-0509-436d-99e8-00c4a397381a tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4b8c7ad3cc6240c096c2c7cf31d8f74f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a45a8ce6687c427d8f76e4205a076a93', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:19:07.999931 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-34b7ade2-0509-436d-99e8-00c4a397381a tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "GET /compute/v2.1/servers/a0d12e80-5ef8-408c-a4d3-92da4c085d85" status: 200 len: 1691 microversion: 2.1 time: 0.163965 Jul 16 22:19:08.000655 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3496/6998] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:19:07 2022] GET /compute/v2.1/servers/a0d12e80-5ef8-408c-a4d3-92da4c085d85 => generated 1691 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:19:09.017804 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-393b5b4c-0d7d-4384-afa5-2664a572c94b tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:19:09.018450 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-393b5b4c-0d7d-4384-afa5-2664a572c94b tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] [instance: a0d12e80-5ef8-408c-a4d3-92da4c085d85] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:19:09.026462 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-393b5b4c-0d7d-4384-afa5-2664a572c94b tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:19:09.026870 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-393b5b4c-0d7d-4384-afa5-2664a572c94b tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:19:09.027831 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-393b5b4c-0d7d-4384-afa5-2664a572c94b tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:19:09.028140 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-393b5b4c-0d7d-4384-afa5-2664a572c94b tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:19:09.168009 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-393b5b4c-0d7d-4384-afa5-2664a572c94b tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4b8c7ad3cc6240c096c2c7cf31d8f74f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a45a8ce6687c427d8f76e4205a076a93', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:19:09.175695 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-393b5b4c-0d7d-4384-afa5-2664a572c94b tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "GET /compute/v2.1/servers/a0d12e80-5ef8-408c-a4d3-92da4c085d85" status: 200 len: 1691 microversion: 2.1 time: 0.161854 Jul 16 22:19:09.176277 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3503/6999] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:19:09 2022] GET /compute/v2.1/servers/a0d12e80-5ef8-408c-a4d3-92da4c085d85 => generated 1691 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:19:10.193992 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-355cc53a-2694-442a-87d5-db232cd82a16 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:19:10.194555 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-355cc53a-2694-442a-87d5-db232cd82a16 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] [instance: a0d12e80-5ef8-408c-a4d3-92da4c085d85] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:19:10.202654 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-355cc53a-2694-442a-87d5-db232cd82a16 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:19:10.203150 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-355cc53a-2694-442a-87d5-db232cd82a16 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:19:10.203959 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-355cc53a-2694-442a-87d5-db232cd82a16 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:19:10.204415 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-355cc53a-2694-442a-87d5-db232cd82a16 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:19:10.344365 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-355cc53a-2694-442a-87d5-db232cd82a16 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4b8c7ad3cc6240c096c2c7cf31d8f74f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a45a8ce6687c427d8f76e4205a076a93', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:19:10.353203 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-355cc53a-2694-442a-87d5-db232cd82a16 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "GET /compute/v2.1/servers/a0d12e80-5ef8-408c-a4d3-92da4c085d85" status: 200 len: 1691 microversion: 2.1 time: 0.163317 Jul 16 22:19:10.353771 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3497/7000] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:19:10 2022] GET /compute/v2.1/servers/a0d12e80-5ef8-408c-a4d3-92da4c085d85 => generated 1691 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:19:11.371139 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-170e641a-c06a-4803-8419-541cb87655ae tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:19:11.371761 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-170e641a-c06a-4803-8419-541cb87655ae tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] [instance: a0d12e80-5ef8-408c-a4d3-92da4c085d85] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:19:11.378744 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-170e641a-c06a-4803-8419-541cb87655ae tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:19:11.379251 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-170e641a-c06a-4803-8419-541cb87655ae tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:19:11.380031 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-170e641a-c06a-4803-8419-541cb87655ae tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:19:11.380488 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-170e641a-c06a-4803-8419-541cb87655ae tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:19:11.513577 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-170e641a-c06a-4803-8419-541cb87655ae tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4b8c7ad3cc6240c096c2c7cf31d8f74f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a45a8ce6687c427d8f76e4205a076a93', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:19:11.521123 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-170e641a-c06a-4803-8419-541cb87655ae tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "GET /compute/v2.1/servers/a0d12e80-5ef8-408c-a4d3-92da4c085d85" status: 200 len: 1696 microversion: 2.1 time: 0.154613 Jul 16 22:19:11.521739 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3504/7001] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:19:11 2022] GET /compute/v2.1/servers/a0d12e80-5ef8-408c-a4d3-92da4c085d85 => generated 1696 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:19:12.538793 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-30c4a680-baa3-4cb9-a439-e49d5f2f6d75 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:19:12.539523 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-30c4a680-baa3-4cb9-a439-e49d5f2f6d75 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] [instance: a0d12e80-5ef8-408c-a4d3-92da4c085d85] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:19:12.546503 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-30c4a680-baa3-4cb9-a439-e49d5f2f6d75 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:19:12.546959 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-30c4a680-baa3-4cb9-a439-e49d5f2f6d75 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:19:12.547751 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-30c4a680-baa3-4cb9-a439-e49d5f2f6d75 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:19:12.548193 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-30c4a680-baa3-4cb9-a439-e49d5f2f6d75 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:19:12.678374 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-30c4a680-baa3-4cb9-a439-e49d5f2f6d75 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4b8c7ad3cc6240c096c2c7cf31d8f74f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a45a8ce6687c427d8f76e4205a076a93', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:19:12.686769 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-30c4a680-baa3-4cb9-a439-e49d5f2f6d75 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "GET /compute/v2.1/servers/a0d12e80-5ef8-408c-a4d3-92da4c085d85" status: 200 len: 1696 microversion: 2.1 time: 0.151989 Jul 16 22:19:12.687385 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3498/7002] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:19:12 2022] GET /compute/v2.1/servers/a0d12e80-5ef8-408c-a4d3-92da4c085d85 => generated 1696 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:19:13.703817 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-2a544757-b3d8-4cb7-9b56-d37d9e8c9701 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:19:13.704482 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-2a544757-b3d8-4cb7-9b56-d37d9e8c9701 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] [instance: a0d12e80-5ef8-408c-a4d3-92da4c085d85] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:19:13.711868 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2a544757-b3d8-4cb7-9b56-d37d9e8c9701 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:19:13.712372 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2a544757-b3d8-4cb7-9b56-d37d9e8c9701 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:19:13.713153 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2a544757-b3d8-4cb7-9b56-d37d9e8c9701 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:19:13.713635 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-2a544757-b3d8-4cb7-9b56-d37d9e8c9701 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:19:13.856365 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-2a544757-b3d8-4cb7-9b56-d37d9e8c9701 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4b8c7ad3cc6240c096c2c7cf31d8f74f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a45a8ce6687c427d8f76e4205a076a93', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:19:13.864478 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-2a544757-b3d8-4cb7-9b56-d37d9e8c9701 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "GET /compute/v2.1/servers/a0d12e80-5ef8-408c-a4d3-92da4c085d85" status: 200 len: 1696 microversion: 2.1 time: 0.164570 Jul 16 22:19:13.865073 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3505/7003] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:19:13 2022] GET /compute/v2.1/servers/a0d12e80-5ef8-408c-a4d3-92da4c085d85 => generated 1696 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:19:14.881724 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-3bee1346-2dcc-4ecc-a58a-86841edc5a4f tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:19:14.882345 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-3bee1346-2dcc-4ecc-a58a-86841edc5a4f tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] [instance: a0d12e80-5ef8-408c-a4d3-92da4c085d85] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:19:14.889498 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3bee1346-2dcc-4ecc-a58a-86841edc5a4f tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:19:14.889957 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3bee1346-2dcc-4ecc-a58a-86841edc5a4f tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:19:14.890728 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3bee1346-2dcc-4ecc-a58a-86841edc5a4f tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:19:14.891205 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-3bee1346-2dcc-4ecc-a58a-86841edc5a4f tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:19:15.026676 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-3bee1346-2dcc-4ecc-a58a-86841edc5a4f tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4b8c7ad3cc6240c096c2c7cf31d8f74f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a45a8ce6687c427d8f76e4205a076a93', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:19:15.034170 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-3bee1346-2dcc-4ecc-a58a-86841edc5a4f tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "GET /compute/v2.1/servers/a0d12e80-5ef8-408c-a4d3-92da4c085d85" status: 200 len: 1696 microversion: 2.1 time: 0.156401 Jul 16 22:19:15.034759 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3499/7004] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:19:14 2022] GET /compute/v2.1/servers/a0d12e80-5ef8-408c-a4d3-92da4c085d85 => generated 1696 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:19:16.052295 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-ca9d5e2c-97eb-4a33-8379-cac2b5b8fc8b tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:19:16.052935 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-ca9d5e2c-97eb-4a33-8379-cac2b5b8fc8b tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] [instance: a0d12e80-5ef8-408c-a4d3-92da4c085d85] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:19:16.060828 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ca9d5e2c-97eb-4a33-8379-cac2b5b8fc8b tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:19:16.061333 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ca9d5e2c-97eb-4a33-8379-cac2b5b8fc8b tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:19:16.062121 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ca9d5e2c-97eb-4a33-8379-cac2b5b8fc8b tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:19:16.062582 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ca9d5e2c-97eb-4a33-8379-cac2b5b8fc8b tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:19:16.201145 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-ca9d5e2c-97eb-4a33-8379-cac2b5b8fc8b tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4b8c7ad3cc6240c096c2c7cf31d8f74f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a45a8ce6687c427d8f76e4205a076a93', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:19:16.208869 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-ca9d5e2c-97eb-4a33-8379-cac2b5b8fc8b tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "GET /compute/v2.1/servers/a0d12e80-5ef8-408c-a4d3-92da4c085d85" status: 200 len: 1696 microversion: 2.1 time: 0.160611 Jul 16 22:19:16.209469 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3506/7005] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:19:16 2022] GET /compute/v2.1/servers/a0d12e80-5ef8-408c-a4d3-92da4c085d85 => generated 1696 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:19:17.225752 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-6d289520-3e08-4da1-a83d-11dbbb442303 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:19:17.226359 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-6d289520-3e08-4da1-a83d-11dbbb442303 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] [instance: a0d12e80-5ef8-408c-a4d3-92da4c085d85] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:19:17.233208 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6d289520-3e08-4da1-a83d-11dbbb442303 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:19:17.233670 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6d289520-3e08-4da1-a83d-11dbbb442303 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:19:17.234437 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6d289520-3e08-4da1-a83d-11dbbb442303 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:19:17.234881 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6d289520-3e08-4da1-a83d-11dbbb442303 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:19:17.367638 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-6d289520-3e08-4da1-a83d-11dbbb442303 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4b8c7ad3cc6240c096c2c7cf31d8f74f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a45a8ce6687c427d8f76e4205a076a93', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:19:17.375328 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-6d289520-3e08-4da1-a83d-11dbbb442303 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "GET /compute/v2.1/servers/a0d12e80-5ef8-408c-a4d3-92da4c085d85" status: 200 len: 1696 microversion: 2.1 time: 0.153583 Jul 16 22:19:17.375904 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3500/7006] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:19:17 2022] GET /compute/v2.1/servers/a0d12e80-5ef8-408c-a4d3-92da4c085d85 => generated 1696 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:19:18.393841 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-115673ed-6df0-4862-834d-ff390e41b9a0 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:19:18.394477 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-115673ed-6df0-4862-834d-ff390e41b9a0 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] [instance: a0d12e80-5ef8-408c-a4d3-92da4c085d85] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:19:18.401800 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-115673ed-6df0-4862-834d-ff390e41b9a0 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:19:18.402270 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-115673ed-6df0-4862-834d-ff390e41b9a0 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:19:18.403093 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-115673ed-6df0-4862-834d-ff390e41b9a0 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:19:18.403554 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-115673ed-6df0-4862-834d-ff390e41b9a0 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:19:18.535928 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-115673ed-6df0-4862-834d-ff390e41b9a0 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4b8c7ad3cc6240c096c2c7cf31d8f74f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a45a8ce6687c427d8f76e4205a076a93', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:19:18.543971 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-115673ed-6df0-4862-834d-ff390e41b9a0 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "GET /compute/v2.1/servers/a0d12e80-5ef8-408c-a4d3-92da4c085d85" status: 200 len: 1696 microversion: 2.1 time: 0.154328 Jul 16 22:19:18.544550 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3507/7007] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:19:18 2022] GET /compute/v2.1/servers/a0d12e80-5ef8-408c-a4d3-92da4c085d85 => generated 1696 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:19:19.561385 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-6244264e-0a07-4d9f-95b5-3eca43a4db9a tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:19:19.562010 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-6244264e-0a07-4d9f-95b5-3eca43a4db9a tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] [instance: a0d12e80-5ef8-408c-a4d3-92da4c085d85] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:19:19.569064 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6244264e-0a07-4d9f-95b5-3eca43a4db9a tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:19:19.569531 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6244264e-0a07-4d9f-95b5-3eca43a4db9a tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:19:19.570479 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6244264e-0a07-4d9f-95b5-3eca43a4db9a tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:19:19.570929 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6244264e-0a07-4d9f-95b5-3eca43a4db9a tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:19:19.706210 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-6244264e-0a07-4d9f-95b5-3eca43a4db9a tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4b8c7ad3cc6240c096c2c7cf31d8f74f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a45a8ce6687c427d8f76e4205a076a93', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:19:19.714663 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-6244264e-0a07-4d9f-95b5-3eca43a4db9a tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "GET /compute/v2.1/servers/a0d12e80-5ef8-408c-a4d3-92da4c085d85" status: 200 len: 1696 microversion: 2.1 time: 0.157499 Jul 16 22:19:19.715330 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3501/7008] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:19:19 2022] GET /compute/v2.1/servers/a0d12e80-5ef8-408c-a4d3-92da4c085d85 => generated 1696 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:19:20.732305 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-3771266c-0326-48ca-a140-accf4ef5c6a5 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:19:20.734863 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-3771266c-0326-48ca-a140-accf4ef5c6a5 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] [instance: a0d12e80-5ef8-408c-a4d3-92da4c085d85] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:19:20.739885 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3771266c-0326-48ca-a140-accf4ef5c6a5 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:19:20.740362 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3771266c-0326-48ca-a140-accf4ef5c6a5 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:19:20.741749 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3771266c-0326-48ca-a140-accf4ef5c6a5 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:19:20.742211 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-3771266c-0326-48ca-a140-accf4ef5c6a5 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:19:20.875505 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-3771266c-0326-48ca-a140-accf4ef5c6a5 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4b8c7ad3cc6240c096c2c7cf31d8f74f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a45a8ce6687c427d8f76e4205a076a93', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:19:20.883156 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-3771266c-0326-48ca-a140-accf4ef5c6a5 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "GET /compute/v2.1/servers/a0d12e80-5ef8-408c-a4d3-92da4c085d85" status: 200 len: 1696 microversion: 2.1 time: 0.154768 Jul 16 22:19:20.883715 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3508/7009] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:19:20 2022] GET /compute/v2.1/servers/a0d12e80-5ef8-408c-a4d3-92da4c085d85 => generated 1696 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:19:21.900936 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-4ed412fe-bdfc-4edc-9f99-d472467d05b3 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:19:21.901553 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-4ed412fe-bdfc-4edc-9f99-d472467d05b3 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] [instance: a0d12e80-5ef8-408c-a4d3-92da4c085d85] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:19:21.908572 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4ed412fe-bdfc-4edc-9f99-d472467d05b3 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:19:21.909027 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4ed412fe-bdfc-4edc-9f99-d472467d05b3 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:19:21.909796 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4ed412fe-bdfc-4edc-9f99-d472467d05b3 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:19:21.910261 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4ed412fe-bdfc-4edc-9f99-d472467d05b3 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:19:22.045908 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-4ed412fe-bdfc-4edc-9f99-d472467d05b3 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4b8c7ad3cc6240c096c2c7cf31d8f74f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a45a8ce6687c427d8f76e4205a076a93', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:19:22.053732 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-4ed412fe-bdfc-4edc-9f99-d472467d05b3 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "GET /compute/v2.1/servers/a0d12e80-5ef8-408c-a4d3-92da4c085d85" status: 200 len: 1696 microversion: 2.1 time: 0.157062 Jul 16 22:19:22.054309 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3502/7010] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:19:21 2022] GET /compute/v2.1/servers/a0d12e80-5ef8-408c-a4d3-92da4c085d85 => generated 1696 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:19:23.072269 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-54274848-d7a0-47d6-bb3f-9d538e877114 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:19:23.072916 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-54274848-d7a0-47d6-bb3f-9d538e877114 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] [instance: a0d12e80-5ef8-408c-a4d3-92da4c085d85] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:19:23.080061 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-54274848-d7a0-47d6-bb3f-9d538e877114 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:19:23.080664 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-54274848-d7a0-47d6-bb3f-9d538e877114 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:19:23.081455 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-54274848-d7a0-47d6-bb3f-9d538e877114 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:19:23.081914 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-54274848-d7a0-47d6-bb3f-9d538e877114 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:19:23.219965 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-54274848-d7a0-47d6-bb3f-9d538e877114 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4b8c7ad3cc6240c096c2c7cf31d8f74f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a45a8ce6687c427d8f76e4205a076a93', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:19:23.228421 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-54274848-d7a0-47d6-bb3f-9d538e877114 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "GET /compute/v2.1/servers/a0d12e80-5ef8-408c-a4d3-92da4c085d85" status: 200 len: 1696 microversion: 2.1 time: 0.160491 Jul 16 22:19:23.229072 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3509/7011] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:19:23 2022] GET /compute/v2.1/servers/a0d12e80-5ef8-408c-a4d3-92da4c085d85 => generated 1696 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:19:24.112478 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-6b1e07ec-99c6-461e-9f8c-12294dc70450 req-59e62a84-6546-4c75-9a20-4974165e2f75 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "a0d12e80-5ef8-408c-a4d3-92da4c085d85", "name": "network-vif-unplugged", "status": "completed", "tag": "35c4a2dc-4c5a-4052-bb2c-1cbc5e17f6ea"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:19:24.122543 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-6b1e07ec-99c6-461e-9f8c-12294dc70450 req-59e62a84-6546-4c75-9a20-4974165e2f75 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:19:24.122797 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-6b1e07ec-99c6-461e-9f8c-12294dc70450 req-59e62a84-6546-4c75-9a20-4974165e2f75 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:19:24.135833 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-6b1e07ec-99c6-461e-9f8c-12294dc70450 req-59e62a84-6546-4c75-9a20-4974165e2f75 service nova] Creating event network-vif-unplugged:35c4a2dc-4c5a-4052-bb2c-1cbc5e17f6ea for instance a0d12e80-5ef8-408c-a4d3-92da4c085d85 on nch2-master-1 Jul 16 22:19:24.141320 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [req-6b1e07ec-99c6-461e-9f8c-12294dc70450 req-59e62a84-6546-4c75-9a20-4974165e2f75 service nova] Instance a0d12e80-5ef8-408c-a4d3-92da4c085d85 is migrating within the same cell, copying events to all relevant hosts: {'nch2-master-1', 'nch1-master-1'} {{(pid=138680) _get_relevant_hosts /opt/stack/nova/nova/compute/api.py:5807}} Jul 16 22:19:24.152234 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-6b1e07ec-99c6-461e-9f8c-12294dc70450 req-59e62a84-6546-4c75-9a20-4974165e2f75 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.043706 Jul 16 22:19:24.152848 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3503/7012] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 22:19:24 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 22:19:24.246624 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-d412376c-b0f0-4df7-84e3-46c82d43f74b tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:19:24.247368 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-d412376c-b0f0-4df7-84e3-46c82d43f74b tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] [instance: a0d12e80-5ef8-408c-a4d3-92da4c085d85] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:19:24.254982 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d412376c-b0f0-4df7-84e3-46c82d43f74b tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:19:24.256846 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d412376c-b0f0-4df7-84e3-46c82d43f74b tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:19:24.256846 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d412376c-b0f0-4df7-84e3-46c82d43f74b tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:19:24.257439 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d412376c-b0f0-4df7-84e3-46c82d43f74b tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:19:24.542267 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-d412376c-b0f0-4df7-84e3-46c82d43f74b tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4b8c7ad3cc6240c096c2c7cf31d8f74f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a45a8ce6687c427d8f76e4205a076a93', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:19:24.550148 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-d412376c-b0f0-4df7-84e3-46c82d43f74b tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "GET /compute/v2.1/servers/a0d12e80-5ef8-408c-a4d3-92da4c085d85" status: 200 len: 1695 microversion: 2.1 time: 0.308035 Jul 16 22:19:24.550855 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3510/7013] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:19:24 2022] GET /compute/v2.1/servers/a0d12e80-5ef8-408c-a4d3-92da4c085d85 => generated 1695 bytes in 310 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:19:25.570255 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-63e782e9-414c-4408-bb0f-afdbd3058393 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:19:25.571027 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-63e782e9-414c-4408-bb0f-afdbd3058393 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] [instance: a0d12e80-5ef8-408c-a4d3-92da4c085d85] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:19:25.577510 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-63e782e9-414c-4408-bb0f-afdbd3058393 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:19:25.577969 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-63e782e9-414c-4408-bb0f-afdbd3058393 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:19:25.578747 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-63e782e9-414c-4408-bb0f-afdbd3058393 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:19:25.579234 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-63e782e9-414c-4408-bb0f-afdbd3058393 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:19:25.715429 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-63e782e9-414c-4408-bb0f-afdbd3058393 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4b8c7ad3cc6240c096c2c7cf31d8f74f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a45a8ce6687c427d8f76e4205a076a93', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:19:25.723028 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-63e782e9-414c-4408-bb0f-afdbd3058393 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "GET /compute/v2.1/servers/a0d12e80-5ef8-408c-a4d3-92da4c085d85" status: 200 len: 1695 microversion: 2.1 time: 0.157937 Jul 16 22:19:25.723609 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3504/7014] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:19:25 2022] GET /compute/v2.1/servers/a0d12e80-5ef8-408c-a4d3-92da4c085d85 => generated 1695 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:19:26.167308 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-954f8df4-ebbb-4506-b6b7-d81bed3bab2f req-6a6fd5e8-604a-4757-a1ac-db65b10d0f6c service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "a0d12e80-5ef8-408c-a4d3-92da4c085d85", "tag": "35c4a2dc-4c5a-4052-bb2c-1cbc5e17f6ea"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:19:26.177604 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-954f8df4-ebbb-4506-b6b7-d81bed3bab2f req-6a6fd5e8-604a-4757-a1ac-db65b10d0f6c service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:19:26.178064 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-954f8df4-ebbb-4506-b6b7-d81bed3bab2f req-6a6fd5e8-604a-4757-a1ac-db65b10d0f6c service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:19:26.190524 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-954f8df4-ebbb-4506-b6b7-d81bed3bab2f req-6a6fd5e8-604a-4757-a1ac-db65b10d0f6c service nova] Creating event network-changed:35c4a2dc-4c5a-4052-bb2c-1cbc5e17f6ea for instance a0d12e80-5ef8-408c-a4d3-92da4c085d85 on nch2-master-1 Jul 16 22:19:26.195914 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [req-954f8df4-ebbb-4506-b6b7-d81bed3bab2f req-6a6fd5e8-604a-4757-a1ac-db65b10d0f6c service nova] Instance a0d12e80-5ef8-408c-a4d3-92da4c085d85 is migrating within the same cell, copying events to all relevant hosts: {'nch2-master-1', 'nch1-master-1'} {{(pid=138681) _get_relevant_hosts /opt/stack/nova/nova/compute/api.py:5807}} Jul 16 22:19:26.207342 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-954f8df4-ebbb-4506-b6b7-d81bed3bab2f req-6a6fd5e8-604a-4757-a1ac-db65b10d0f6c service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.043888 Jul 16 22:19:26.207952 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3511/7015] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 22:19:26 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 22:19:26.740241 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-c8042c23-3639-4a78-9e7e-09b8a9628905 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:19:26.740869 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-c8042c23-3639-4a78-9e7e-09b8a9628905 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] [instance: a0d12e80-5ef8-408c-a4d3-92da4c085d85] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:19:26.747721 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c8042c23-3639-4a78-9e7e-09b8a9628905 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:19:26.748179 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c8042c23-3639-4a78-9e7e-09b8a9628905 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:19:26.748949 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c8042c23-3639-4a78-9e7e-09b8a9628905 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:19:26.749394 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c8042c23-3639-4a78-9e7e-09b8a9628905 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:19:26.882785 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-c8042c23-3639-4a78-9e7e-09b8a9628905 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4b8c7ad3cc6240c096c2c7cf31d8f74f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a45a8ce6687c427d8f76e4205a076a93', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:19:26.891094 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-c8042c23-3639-4a78-9e7e-09b8a9628905 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "GET /compute/v2.1/servers/a0d12e80-5ef8-408c-a4d3-92da4c085d85" status: 200 len: 1693 microversion: 2.1 time: 0.154752 Jul 16 22:19:26.891674 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3505/7016] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:19:26 2022] GET /compute/v2.1/servers/a0d12e80-5ef8-408c-a4d3-92da4c085d85 => generated 1693 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:19:27.911782 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-14a87e6b-cece-49da-a5f0-50ecf8c43940 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:19:27.912447 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-14a87e6b-cece-49da-a5f0-50ecf8c43940 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] [instance: a0d12e80-5ef8-408c-a4d3-92da4c085d85] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:19:27.918677 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-14a87e6b-cece-49da-a5f0-50ecf8c43940 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:19:27.920278 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-14a87e6b-cece-49da-a5f0-50ecf8c43940 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:19:27.921057 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-14a87e6b-cece-49da-a5f0-50ecf8c43940 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:19:27.921514 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-14a87e6b-cece-49da-a5f0-50ecf8c43940 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:19:28.066766 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-14a87e6b-cece-49da-a5f0-50ecf8c43940 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4b8c7ad3cc6240c096c2c7cf31d8f74f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a45a8ce6687c427d8f76e4205a076a93', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:19:28.074768 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-14a87e6b-cece-49da-a5f0-50ecf8c43940 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "GET /compute/v2.1/servers/a0d12e80-5ef8-408c-a4d3-92da4c085d85" status: 200 len: 1690 microversion: 2.1 time: 0.168419 Jul 16 22:19:28.075878 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3512/7017] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:19:27 2022] GET /compute/v2.1/servers/a0d12e80-5ef8-408c-a4d3-92da4c085d85 => generated 1690 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:19:28.092646 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-7d17f659-0ab2-4031-966b-27e6385714e8 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Action: 'action', calling method: >, body: {"confirmResize": {}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:19:28.098442 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-7d17f659-0ab2-4031-966b-27e6385714e8 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] [instance: a0d12e80-5ef8-408c-a4d3-92da4c085d85] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:19:28.100229 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7d17f659-0ab2-4031-966b-27e6385714e8 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:19:28.100691 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7d17f659-0ab2-4031-966b-27e6385714e8 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:19:28.101454 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7d17f659-0ab2-4031-966b-27e6385714e8 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:19:28.101896 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7d17f659-0ab2-4031-966b-27e6385714e8 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:19:28.365270 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-7d17f659-0ab2-4031-966b-27e6385714e8 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "POST /compute/v2.1/servers/a0d12e80-5ef8-408c-a4d3-92da4c085d85/action" status: 204 len: 0 microversion: 2.1 time: 0.277049 Jul 16 22:19:28.365964 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3506/7018] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 22:19:28 2022] POST /compute/v2.1/servers/a0d12e80-5ef8-408c-a4d3-92da4c085d85/action => generated 0 bytes in 279 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 16 22:19:28.375192 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-e4325f52-1183-4246-9483-cceb32fa1113 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:19:28.375822 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-e4325f52-1183-4246-9483-cceb32fa1113 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] [instance: a0d12e80-5ef8-408c-a4d3-92da4c085d85] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:19:28.381904 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e4325f52-1183-4246-9483-cceb32fa1113 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:19:28.382365 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e4325f52-1183-4246-9483-cceb32fa1113 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:19:28.383189 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e4325f52-1183-4246-9483-cceb32fa1113 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:19:28.383644 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e4325f52-1183-4246-9483-cceb32fa1113 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:19:28.618670 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-e4325f52-1183-4246-9483-cceb32fa1113 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4b8c7ad3cc6240c096c2c7cf31d8f74f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a45a8ce6687c427d8f76e4205a076a93', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:19:28.626770 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-e4325f52-1183-4246-9483-cceb32fa1113 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "GET /compute/v2.1/servers/a0d12e80-5ef8-408c-a4d3-92da4c085d85" status: 200 len: 1690 microversion: 2.1 time: 0.256673 Jul 16 22:19:28.627385 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3513/7019] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:19:28 2022] GET /compute/v2.1/servers/a0d12e80-5ef8-408c-a4d3-92da4c085d85 => generated 1690 bytes in 258 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:19:28.644344 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-5a30afdc-6096-4995-9dc3-15e62e6cce87 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:19:28.645053 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-5a30afdc-6096-4995-9dc3-15e62e6cce87 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] [instance: a0d12e80-5ef8-408c-a4d3-92da4c085d85] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:19:28.651368 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5a30afdc-6096-4995-9dc3-15e62e6cce87 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:19:28.651827 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5a30afdc-6096-4995-9dc3-15e62e6cce87 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:19:28.652598 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5a30afdc-6096-4995-9dc3-15e62e6cce87 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:19:28.653043 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5a30afdc-6096-4995-9dc3-15e62e6cce87 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:19:28.890951 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-5a30afdc-6096-4995-9dc3-15e62e6cce87 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4b8c7ad3cc6240c096c2c7cf31d8f74f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a45a8ce6687c427d8f76e4205a076a93', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:19:28.898630 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-5a30afdc-6096-4995-9dc3-15e62e6cce87 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "GET /compute/v2.1/servers/a0d12e80-5ef8-408c-a4d3-92da4c085d85" status: 200 len: 1690 microversion: 2.1 time: 0.258516 Jul 16 22:19:28.899242 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3507/7020] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:19:28 2022] GET /compute/v2.1/servers/a0d12e80-5ef8-408c-a4d3-92da4c085d85 => generated 1690 bytes in 260 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:19:29.779591 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-344c1fed-d590-48f3-a763-1c7f8bd9e254 req-6850b032-969e-42ee-b38d-0553843d5520 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "a0d12e80-5ef8-408c-a4d3-92da4c085d85", "name": "network-vif-plugged", "status": "completed", "tag": "35c4a2dc-4c5a-4052-bb2c-1cbc5e17f6ea"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:19:29.789275 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-344c1fed-d590-48f3-a763-1c7f8bd9e254 req-6850b032-969e-42ee-b38d-0553843d5520 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:19:29.789735 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-344c1fed-d590-48f3-a763-1c7f8bd9e254 req-6850b032-969e-42ee-b38d-0553843d5520 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:19:29.802733 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-344c1fed-d590-48f3-a763-1c7f8bd9e254 req-6850b032-969e-42ee-b38d-0553843d5520 service nova] Creating event network-vif-plugged:35c4a2dc-4c5a-4052-bb2c-1cbc5e17f6ea for instance a0d12e80-5ef8-408c-a4d3-92da4c085d85 on nch2-master-1 Jul 16 22:19:29.808363 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [req-344c1fed-d590-48f3-a763-1c7f8bd9e254 req-6850b032-969e-42ee-b38d-0553843d5520 service nova] Instance a0d12e80-5ef8-408c-a4d3-92da4c085d85 is migrating within the same cell, copying events to all relevant hosts: {'nch2-master-1', 'nch1-master-1'} {{(pid=138681) _get_relevant_hosts /opt/stack/nova/nova/compute/api.py:5807}} Jul 16 22:19:29.820508 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-344c1fed-d590-48f3-a763-1c7f8bd9e254 req-6850b032-969e-42ee-b38d-0553843d5520 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.044943 Jul 16 22:19:29.827785 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3514/7021] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 22:19:29 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 22:19:29.916345 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-13ab9330-8372-413b-af96-a5b2b734404b tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:19:29.916968 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-13ab9330-8372-413b-af96-a5b2b734404b tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] [instance: a0d12e80-5ef8-408c-a4d3-92da4c085d85] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:19:29.924588 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-13ab9330-8372-413b-af96-a5b2b734404b tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:19:29.926887 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-13ab9330-8372-413b-af96-a5b2b734404b tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:19:29.926887 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-13ab9330-8372-413b-af96-a5b2b734404b tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:19:29.926887 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-13ab9330-8372-413b-af96-a5b2b734404b tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:19:30.065175 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-13ab9330-8372-413b-af96-a5b2b734404b tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4b8c7ad3cc6240c096c2c7cf31d8f74f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a45a8ce6687c427d8f76e4205a076a93', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:19:30.072830 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-13ab9330-8372-413b-af96-a5b2b734404b tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "GET /compute/v2.1/servers/a0d12e80-5ef8-408c-a4d3-92da4c085d85" status: 200 len: 1690 microversion: 2.1 time: 0.160457 Jul 16 22:19:30.073414 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3508/7022] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:19:29 2022] GET /compute/v2.1/servers/a0d12e80-5ef8-408c-a4d3-92da4c085d85 => generated 1690 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:19:31.096215 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-8b9c1f12-bd38-4ff5-b4d7-10fa81e317ef tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:19:31.096963 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-8b9c1f12-bd38-4ff5-b4d7-10fa81e317ef tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] [instance: a0d12e80-5ef8-408c-a4d3-92da4c085d85] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:19:31.103352 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8b9c1f12-bd38-4ff5-b4d7-10fa81e317ef tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:19:31.103894 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8b9c1f12-bd38-4ff5-b4d7-10fa81e317ef tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:19:31.104661 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8b9c1f12-bd38-4ff5-b4d7-10fa81e317ef tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:19:31.105114 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8b9c1f12-bd38-4ff5-b4d7-10fa81e317ef tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:19:31.242306 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-8b9c1f12-bd38-4ff5-b4d7-10fa81e317ef tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4b8c7ad3cc6240c096c2c7cf31d8f74f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a45a8ce6687c427d8f76e4205a076a93', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:19:31.250132 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-8b9c1f12-bd38-4ff5-b4d7-10fa81e317ef tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "GET /compute/v2.1/servers/a0d12e80-5ef8-408c-a4d3-92da4c085d85" status: 200 len: 1682 microversion: 2.1 time: 0.159759 Jul 16 22:19:31.250703 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3515/7023] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:19:31 2022] GET /compute/v2.1/servers/a0d12e80-5ef8-408c-a4d3-92da4c085d85 => generated 1682 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:19:45.012102 ncdv-master-1 devstack@n-api.service[138680]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jul 16 22:19:45.012630 ncdv-master-1 devstack@n-api.service[138680]: Sat Jul 16 22:19:45 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Jul 16 22:19:55.429097 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-2a1a2464-77bf-4646-8d6d-94d9a5d0d10f tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Action: 'action', calling method: >, body: {"os-getConsoleOutput": {}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:19:55.430048 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-2a1a2464-77bf-4646-8d6d-94d9a5d0d10f tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] [instance: a0d12e80-5ef8-408c-a4d3-92da4c085d85] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:19:55.432779 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_api:SELECT=28 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:19:55.433471 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=85,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:19:55.438404 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2a1a2464-77bf-4646-8d6d-94d9a5d0d10f tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:19:55.440384 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2a1a2464-77bf-4646-8d6d-94d9a5d0d10f tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:19:55.440384 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2a1a2464-77bf-4646-8d6d-94d9a5d0d10f tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:19:55.440384 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2a1a2464-77bf-4646-8d6d-94d9a5d0d10f tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:19:55.492619 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-2a1a2464-77bf-4646-8d6d-94d9a5d0d10f tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "POST /compute/v2.1/servers/a0d12e80-5ef8-408c-a4d3-92da4c085d85/action" status: 200 len: 24582 microversion: 2.1 time: 0.067354 Jul 16 22:19:55.493245 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3509/7024] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 22:19:55 2022] POST /compute/v2.1/servers/a0d12e80-5ef8-408c-a4d3-92da4c085d85/action => generated 24582 bytes in 69 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 16 22:20:02.712491 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-8098a35f-39d4-454f-8b7b-995b0b30ca43 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:20:02.713104 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-8098a35f-39d4-454f-8b7b-995b0b30ca43 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] [instance: a0d12e80-5ef8-408c-a4d3-92da4c085d85] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:20:02.715267 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-95a78d3b-7816-4a67-b4c4-f87b652dd843 req-3f91d0a1-3fc4-46d3-b874-09a52eaef592 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "a0d12e80-5ef8-408c-a4d3-92da4c085d85", "tag": "35c4a2dc-4c5a-4052-bb2c-1cbc5e17f6ea"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:20:02.715752 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_api:SELECT=32 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:20:02.716289 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=86,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:20:02.721492 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8098a35f-39d4-454f-8b7b-995b0b30ca43 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:20:02.721967 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8098a35f-39d4-454f-8b7b-995b0b30ca43 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:20:02.722766 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8098a35f-39d4-454f-8b7b-995b0b30ca43 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:20:02.723292 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8098a35f-39d4-454f-8b7b-995b0b30ca43 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:20:02.727410 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-95a78d3b-7816-4a67-b4c4-f87b652dd843 req-3f91d0a1-3fc4-46d3-b874-09a52eaef592 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:20:02.728271 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-95a78d3b-7816-4a67-b4c4-f87b652dd843 req-3f91d0a1-3fc4-46d3-b874-09a52eaef592 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:20:02.742388 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-95a78d3b-7816-4a67-b4c4-f87b652dd843 req-3f91d0a1-3fc4-46d3-b874-09a52eaef592 service nova] Creating event network-changed:35c4a2dc-4c5a-4052-bb2c-1cbc5e17f6ea for instance a0d12e80-5ef8-408c-a4d3-92da4c085d85 on nch2-master-1 Jul 16 22:20:02.748403 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-95a78d3b-7816-4a67-b4c4-f87b652dd843 req-3f91d0a1-3fc4-46d3-b874-09a52eaef592 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.038032 Jul 16 22:20:02.749339 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3510/7025] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 22:20:02 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 22:20:02.753875 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-8098a35f-39d4-454f-8b7b-995b0b30ca43 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] [instance: a0d12e80-5ef8-408c-a4d3-92da4c085d85] Going to try to terminate instance {{(pid=138681) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 16 22:20:03.184402 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-8098a35f-39d4-454f-8b7b-995b0b30ca43 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "DELETE /compute/v2.1/servers/a0d12e80-5ef8-408c-a4d3-92da4c085d85" status: 204 len: 0 microversion: 2.1 time: 0.476665 Jul 16 22:20:03.185790 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3516/7026] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 22:20:02 2022] DELETE /compute/v2.1/servers/a0d12e80-5ef8-408c-a4d3-92da4c085d85 => generated 0 bytes in 479 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 16 22:20:03.194383 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-c8d2ccb0-e927-42a2-97ca-6a80dd1bcc3f tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:20:03.195444 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-c8d2ccb0-e927-42a2-97ca-6a80dd1bcc3f tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] [instance: a0d12e80-5ef8-408c-a4d3-92da4c085d85] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:20:03.204166 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c8d2ccb0-e927-42a2-97ca-6a80dd1bcc3f tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:20:03.204633 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c8d2ccb0-e927-42a2-97ca-6a80dd1bcc3f tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:20:03.205432 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c8d2ccb0-e927-42a2-97ca-6a80dd1bcc3f tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:20:03.206195 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c8d2ccb0-e927-42a2-97ca-6a80dd1bcc3f tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:20:03.355704 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-c8d2ccb0-e927-42a2-97ca-6a80dd1bcc3f tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4b8c7ad3cc6240c096c2c7cf31d8f74f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a45a8ce6687c427d8f76e4205a076a93', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:20:03.363678 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-c8d2ccb0-e927-42a2-97ca-6a80dd1bcc3f tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "GET /compute/v2.1/servers/a0d12e80-5ef8-408c-a4d3-92da4c085d85" status: 200 len: 1688 microversion: 2.1 time: 0.173407 Jul 16 22:20:03.364262 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3511/7027] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:20:03 2022] GET /compute/v2.1/servers/a0d12e80-5ef8-408c-a4d3-92da4c085d85 => generated 1688 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:20:04.383043 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-0de15523-fd6e-45a1-bd04-339591250822 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:20:04.384010 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-0de15523-fd6e-45a1-bd04-339591250822 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] [instance: a0d12e80-5ef8-408c-a4d3-92da4c085d85] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:20:04.392560 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0de15523-fd6e-45a1-bd04-339591250822 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:20:04.393038 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0de15523-fd6e-45a1-bd04-339591250822 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:20:04.393933 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0de15523-fd6e-45a1-bd04-339591250822 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:20:04.394402 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0de15523-fd6e-45a1-bd04-339591250822 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:20:04.561540 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-0de15523-fd6e-45a1-bd04-339591250822 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4b8c7ad3cc6240c096c2c7cf31d8f74f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a45a8ce6687c427d8f76e4205a076a93', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:20:04.569676 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-0de15523-fd6e-45a1-bd04-339591250822 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "GET /compute/v2.1/servers/a0d12e80-5ef8-408c-a4d3-92da4c085d85" status: 200 len: 1688 microversion: 2.1 time: 0.191121 Jul 16 22:20:04.570251 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3517/7028] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:20:04 2022] GET /compute/v2.1/servers/a0d12e80-5ef8-408c-a4d3-92da4c085d85 => generated 1688 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:20:05.133196 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-2de16475-f763-4bdb-ab3a-4061e27cbbb3 req-4ff4c735-5e2f-4b96-9590-d44f844f6821 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "a0d12e80-5ef8-408c-a4d3-92da4c085d85", "name": "network-vif-deleted", "tag": "35c4a2dc-4c5a-4052-bb2c-1cbc5e17f6ea"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:20:05.141268 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-2de16475-f763-4bdb-ab3a-4061e27cbbb3 req-4ff4c735-5e2f-4b96-9590-d44f844f6821 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:20:05.147727 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-2de16475-f763-4bdb-ab3a-4061e27cbbb3 req-4ff4c735-5e2f-4b96-9590-d44f844f6821 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:20:05.153462 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-2de16475-f763-4bdb-ab3a-4061e27cbbb3 req-4ff4c735-5e2f-4b96-9590-d44f844f6821 service nova] Creating event network-vif-deleted:35c4a2dc-4c5a-4052-bb2c-1cbc5e17f6ea for instance a0d12e80-5ef8-408c-a4d3-92da4c085d85 on nch2-master-1 Jul 16 22:20:05.158246 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-2de16475-f763-4bdb-ab3a-4061e27cbbb3 req-4ff4c735-5e2f-4b96-9590-d44f844f6821 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.030128 Jul 16 22:20:05.158835 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3512/7029] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 22:20:05 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 22:20:05.587991 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-132db526-f5f9-4716-ae26-75b39acec928 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:20:05.588836 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-132db526-f5f9-4716-ae26-75b39acec928 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] [instance: a0d12e80-5ef8-408c-a4d3-92da4c085d85] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:20:05.595812 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-132db526-f5f9-4716-ae26-75b39acec928 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:20:05.596280 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-132db526-f5f9-4716-ae26-75b39acec928 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:20:05.597203 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-132db526-f5f9-4716-ae26-75b39acec928 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:20:05.597745 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-132db526-f5f9-4716-ae26-75b39acec928 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:20:05.632002 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-132db526-f5f9-4716-ae26-75b39acec928 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:20:05.632464 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-132db526-f5f9-4716-ae26-75b39acec928 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:20:05.632921 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.objects.instance [None req-132db526-f5f9-4716-ae26-75b39acec928 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lazy-loading 'fault' on Instance uuid a0d12e80-5ef8-408c-a4d3-92da4c085d85 {{(pid=138681) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 22:20:05.674469 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-132db526-f5f9-4716-ae26-75b39acec928 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4b8c7ad3cc6240c096c2c7cf31d8f74f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a45a8ce6687c427d8f76e4205a076a93', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:20:05.682776 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-132db526-f5f9-4716-ae26-75b39acec928 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "GET /compute/v2.1/servers/a0d12e80-5ef8-408c-a4d3-92da4c085d85" status: 200 len: 1357 microversion: 2.1 time: 0.099600 Jul 16 22:20:05.683384 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3518/7030] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:20:05 2022] GET /compute/v2.1/servers/a0d12e80-5ef8-408c-a4d3-92da4c085d85 => generated 1357 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:20:06.701090 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-97fb64d8-9dc0-4707-bdd9-26902502f1dc tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:20:06.701801 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-97fb64d8-9dc0-4707-bdd9-26902502f1dc tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] [instance: a0d12e80-5ef8-408c-a4d3-92da4c085d85] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:20:06.709081 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-97fb64d8-9dc0-4707-bdd9-26902502f1dc tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:20:06.709543 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-97fb64d8-9dc0-4707-bdd9-26902502f1dc tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:20:06.710322 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-97fb64d8-9dc0-4707-bdd9-26902502f1dc tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:20:06.710765 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-97fb64d8-9dc0-4707-bdd9-26902502f1dc tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:20:06.724338 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-97fb64d8-9dc0-4707-bdd9-26902502f1dc tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] HTTP exception thrown: Instance a0d12e80-5ef8-408c-a4d3-92da4c085d85 could not be found. Jul 16 22:20:06.725997 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-97fb64d8-9dc0-4707-bdd9-26902502f1dc tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Request method failure captured: Jul 16 22:20:06.725997 ncdv-master-1 devstack@n-api.service[138680]: request: GET /compute/v2.1/servers/a0d12e80-5ef8-408c-a4d3-92da4c085d85 HTTP/1.1 Jul 16 22:20:06.725997 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 22:20:06.725997 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 22:20:06.725997 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 22:20:06.725997 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 0 Jul 16 22:20:06.725997 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 22:20:06.725997 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 22:20:06.725997 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 22:20:06.725997 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 22:20:06.725997 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0zhtDg4n0Cr97EpGJw_kphX8HozEdWitmpiSsnN8liwN7JwqYkEqcaIM-47rjt-yK-e5LgZo6pFGCSkRgNS-xnBjvFiClGjWelkYApUjhWHqw4F9MI58oj4QDcEbMKfIlL82tbwRwh1LdKROeHuRqnWXe9oskU-rprKG2IX2dbA4fe4 Jul 16 22:20:06.725997 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 22:20:06.725997 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 22:20:06.725997 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 22:20:06.725997 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 22:20:06.725997 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 22:20:06.725997 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 22:20:06.725997 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: a45a8ce6687c427d8f76e4205a076a93 Jul 16 22:20:06.725997 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-TestNetworkAdvancedServerOps-234580283 Jul 16 22:20:06.725997 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 22:20:06.725997 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 22:20:06.725997 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/a45a8ce6687c427d8f76e4205a076a93"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/a45a8ce6687c427d8f76e4205a076a93"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_a45a8ce6687c427d8f76e4205a076a93"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/a45a8ce6687c427d8f76e4205a076a93"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 22:20:06.725997 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-TestNetworkAdvancedServerOps-234580283 Jul 16 22:20:06.725997 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: a45a8ce6687c427d8f76e4205a076a93 Jul 16 22:20:06.725997 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-TestNetworkAdvancedServerOps-234580283 Jul 16 22:20:06.725997 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-TestNetworkAdvancedServerOps-234580283-project Jul 16 22:20:06.725997 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 22:20:06.725997 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 22:20:06.725997 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: 4b8c7ad3cc6240c096c2c7cf31d8f74f Jul 16 22:20:06.725997 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-TestNetworkAdvancedServerOps-234580283-project Jul 16 22:20:06.725997 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 22:20:06.725997 ncdv-master-1 devstack@n-api.service[138680]: exception: Instance a0d12e80-5ef8-408c-a4d3-92da4c085d85 could not be found. Jul 16 22:20:06.725997 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': 'a0d12e80-5ef8-408c-a4d3-92da4c085d85'} Jul 16 22:20:06.725997 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 22:20:06.725997 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 22:20:06.725997 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 22:20:06.725997 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 22:20:06.725997 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 22:20:06.725997 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 22:20:06.725997 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 22:20:06.725997 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 22:20:06.725997 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 22:20:06.725997 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 22:20:06.725997 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance a0d12e80-5ef8-408c-a4d3-92da4c085d85 could not be found. Jul 16 22:20:06.725997 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 22:20:06.725997 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 22:20:06.725997 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 22:20:06.725997 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 22:20:06.725997 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 22:20:06.725997 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 22:20:06.725997 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 22:20:06.725997 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 22:20:06.725997 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 22:20:06.725997 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 22:20:06.725997 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 22:20:06.725997 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 22:20:06.725997 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 22:20:06.725997 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 22:20:06.725997 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 22:20:06.725997 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 22:20:06.725997 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance a0d12e80-5ef8-408c-a4d3-92da4c085d85 could not be found. Jul 16 22:20:06.725997 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 22:20:06.725997 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 22:20:06.725997 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 22:20:06.731738 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 22:20:06.731738 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 22:20:06.731738 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 22:20:06.731738 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 22:20:06.731738 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 22:20:06.731738 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance a0d12e80-5ef8-408c-a4d3-92da4c085d85 could not be found. Jul 16 22:20:06.731738 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 22:20:06.731738 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-97fb64d8-9dc0-4707-bdd9-26902502f1dc tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Returning 404 to user: Instance a0d12e80-5ef8-408c-a4d3-92da4c085d85 could not be found. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 22:20:06.731738 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-97fb64d8-9dc0-4707-bdd9-26902502f1dc tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "GET /compute/v2.1/servers/a0d12e80-5ef8-408c-a4d3-92da4c085d85" status: 404 len: 111 microversion: 2.1 time: 0.031592 Jul 16 22:20:06.731738 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3513/7031] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:20:06 2022] GET /compute/v2.1/servers/a0d12e80-5ef8-408c-a4d3-92da4c085d85 => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 22:20:10.508195 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-8c937a08-44bb-4f05-84bd-940235023ec0 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Calling method '.version_select at 0x7fdda2af1d30>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:20:10.592719 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-8c937a08-44bb-4f05-84bd-940235023ec0 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "DELETE /compute/v2.1/os-keypairs/tempest-TestNetworkAdvancedServerOps-1845463979" status: 202 len: 0 microversion: 2.1 time: 0.088814 Jul 16 22:20:10.593326 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3519/7032] 10.222.0.16 () {58 vars in 1283 bytes} [Sat Jul 16 22:20:10 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestNetworkAdvancedServerOps-1845463979 => generated 0 bytes in 90 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 22:20:10.620881 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-e8ddb3b0-1835-4cb8-a4c6-c91d19f931a9 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Action: 'create', calling method: .version_select at 0x7fdd8fb48c10>, body: {"keypair": {"name": "tempest-TestNetworkAdvancedServerOps-564397440", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBE6zdTkNgTlFv+mMh1LK0Lt9b2qbkNsiGem8kfJDToKiZkTzfrMmADBdynMvYrGpa5G4KKZ/m1MfDEJq1uYDo1TO+jWCZJm8qXsGDVVCcASHPaVSOp+SRCH73TTz+LDpKQ=="}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:20:10.639307 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-e8ddb3b0-1835-4cb8-a4c6-c91d19f931a9 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Getting quotas for project a45a8ce6687c427d8f76e4205a076a93. Resources: {'key_pairs'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 22:20:10.644898 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-e8ddb3b0-1835-4cb8-a4c6-c91d19f931a9 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Getting quotas for user 4b8c7ad3cc6240c096c2c7cf31d8f74f and project a45a8ce6687c427d8f76e4205a076a93. Resources: {'key_pairs'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 22:20:10.726724 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-e8ddb3b0-1835-4cb8-a4c6-c91d19f931a9 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "POST /compute/v2.1/os-keypairs" status: 200 len: 406 microversion: 2.1 time: 0.109698 Jul 16 22:20:10.727519 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3514/7033] 10.222.0.16 () {60 vars in 1158 bytes} [Sat Jul 16 22:20:10 2022] POST /compute/v2.1/os-keypairs => generated 406 bytes in 111 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 22:20:18.953214 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=28,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:20:18.957827 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-7bbec25c-cdce-4de5-a304-7f3ff507751d tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestNetworkAdvancedServerOps-server-677851093", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452", "networks": [{"uuid": "4e0f0c59-b0ce-4dc9-b83d-7a48cb94e578"}], "key_name": "tempest-TestNetworkAdvancedServerOps-564397440", "security_groups": [{"name": "tempest-secgroup-smoke-1338535962"}]}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:20:19.083638 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.network.neutron [None req-7bbec25c-cdce-4de5-a304-7f3ff507751d tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] validate_networks() for [('4e0f0c59-b0ce-4dc9-b83d-7a48cb94e578', None, None, None, None, None)] {{(pid=138681) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 16 22:20:19.544018 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-7bbec25c-cdce-4de5-a304-7f3ff507751d tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Flavor limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 16 22:20:19.544467 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-7bbec25c-cdce-4de5-a304-7f3ff507751d tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Image limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 16 22:20:19.544959 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-7bbec25c-cdce-4de5-a304-7f3ff507751d tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Flavor pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 16 22:20:19.545386 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-7bbec25c-cdce-4de5-a304-7f3ff507751d tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Image pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 16 22:20:19.545815 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-7bbec25c-cdce-4de5-a304-7f3ff507751d tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 16 22:20:19.555853 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7bbec25c-cdce-4de5-a304-7f3ff507751d tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:20:19.556322 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7bbec25c-cdce-4de5-a304-7f3ff507751d tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:20:19.557389 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7bbec25c-cdce-4de5-a304-7f3ff507751d tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:20:19.557844 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7bbec25c-cdce-4de5-a304-7f3ff507751d tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:20:19.575919 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-7bbec25c-cdce-4de5-a304-7f3ff507751d tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Getting quotas for project a45a8ce6687c427d8f76e4205a076a93. Resources: {'ram', 'instances', 'cores'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 22:20:19.581583 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-7bbec25c-cdce-4de5-a304-7f3ff507751d tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Getting quotas for user 4b8c7ad3cc6240c096c2c7cf31d8f74f and project a45a8ce6687c427d8f76e4205a076a93. Resources: {'ram', 'instances', 'cores'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 22:20:19.594605 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-7bbec25c-cdce-4de5-a304-7f3ff507751d tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Going to run 1 instances... {{(pid=138681) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 16 22:20:19.597360 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-7bbec25c-cdce-4de5-a304-7f3ff507751d tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] [instance: d7dd236d-4a79-4885-843d-e280807faed7] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138681) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 16 22:20:19.714308 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-7bbec25c-cdce-4de5-a304-7f3ff507751d tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 406 microversion: 2.1 time: 0.763090 Jul 16 22:20:19.715152 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3520/7034] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 22:20:18 2022] POST /compute/v2.1/servers => generated 406 bytes in 765 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 16 22:20:19.726131 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-283d3d1f-574a-4567-bee3-0c8dfbe911ee tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:20:19.727929 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-283d3d1f-574a-4567-bee3-0c8dfbe911ee tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] [instance: d7dd236d-4a79-4885-843d-e280807faed7] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:20:19.734150 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=16 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:20:19.817916 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-283d3d1f-574a-4567-bee3-0c8dfbe911ee tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4b8c7ad3cc6240c096c2c7cf31d8f74f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a45a8ce6687c427d8f76e4205a076a93', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:20:19.830081 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-283d3d1f-574a-4567-bee3-0c8dfbe911ee tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "GET /compute/v2.1/servers/d7dd236d-4a79-4885-843d-e280807faed7" status: 200 len: 1265 microversion: 2.1 time: 0.108721 Jul 16 22:20:19.831404 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3515/7035] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:20:19 2022] GET /compute/v2.1/servers/d7dd236d-4a79-4885-843d-e280807faed7 => generated 1265 bytes in 111 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:20:20.849502 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-c08bc843-9ddc-463d-ad6f-10866a102dc6 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:20:20.850137 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-c08bc843-9ddc-463d-ad6f-10866a102dc6 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] [instance: d7dd236d-4a79-4885-843d-e280807faed7] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:20:20.858291 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c08bc843-9ddc-463d-ad6f-10866a102dc6 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:20:20.868361 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c08bc843-9ddc-463d-ad6f-10866a102dc6 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:20:20.868870 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c08bc843-9ddc-463d-ad6f-10866a102dc6 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:20:20.869563 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c08bc843-9ddc-463d-ad6f-10866a102dc6 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:20:20.940485 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-c08bc843-9ddc-463d-ad6f-10866a102dc6 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4b8c7ad3cc6240c096c2c7cf31d8f74f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a45a8ce6687c427d8f76e4205a076a93', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:20:20.954149 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-c08bc843-9ddc-463d-ad6f-10866a102dc6 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "GET /compute/v2.1/servers/d7dd236d-4a79-4885-843d-e280807faed7" status: 200 len: 1269 microversion: 2.1 time: 0.109948 Jul 16 22:20:20.955174 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3521/7036] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:20:20 2022] GET /compute/v2.1/servers/d7dd236d-4a79-4885-843d-e280807faed7 => generated 1269 bytes in 112 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:20:21.972609 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-c7119154-7dc3-463a-9bb9-a5b2ed1f7a73 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:20:21.974665 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-c7119154-7dc3-463a-9bb9-a5b2ed1f7a73 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] [instance: d7dd236d-4a79-4885-843d-e280807faed7] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:20:21.983139 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c7119154-7dc3-463a-9bb9-a5b2ed1f7a73 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:20:21.983599 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c7119154-7dc3-463a-9bb9-a5b2ed1f7a73 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:20:21.984420 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c7119154-7dc3-463a-9bb9-a5b2ed1f7a73 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:20:21.984946 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c7119154-7dc3-463a-9bb9-a5b2ed1f7a73 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:20:22.076162 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-c7119154-7dc3-463a-9bb9-a5b2ed1f7a73 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4b8c7ad3cc6240c096c2c7cf31d8f74f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a45a8ce6687c427d8f76e4205a076a93', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:20:22.086399 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-c7119154-7dc3-463a-9bb9-a5b2ed1f7a73 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "GET /compute/v2.1/servers/d7dd236d-4a79-4885-843d-e280807faed7" status: 200 len: 1325 microversion: 2.1 time: 0.118664 Jul 16 22:20:22.087761 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3516/7037] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:20:21 2022] GET /compute/v2.1/servers/d7dd236d-4a79-4885-843d-e280807faed7 => generated 1325 bytes in 121 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:20:23.105187 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-321cb602-89c2-4ec0-ab85-645cfd234f74 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:20:23.105827 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-321cb602-89c2-4ec0-ab85-645cfd234f74 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] [instance: d7dd236d-4a79-4885-843d-e280807faed7] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:20:23.112980 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-321cb602-89c2-4ec0-ab85-645cfd234f74 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:20:23.113450 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-321cb602-89c2-4ec0-ab85-645cfd234f74 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:20:23.114309 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-321cb602-89c2-4ec0-ab85-645cfd234f74 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:20:23.114769 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-321cb602-89c2-4ec0-ab85-645cfd234f74 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:20:23.317053 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-321cb602-89c2-4ec0-ab85-645cfd234f74 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4b8c7ad3cc6240c096c2c7cf31d8f74f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a45a8ce6687c427d8f76e4205a076a93', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:20:23.324774 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-321cb602-89c2-4ec0-ab85-645cfd234f74 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "GET /compute/v2.1/servers/d7dd236d-4a79-4885-843d-e280807faed7" status: 200 len: 1391 microversion: 2.1 time: 0.223530 Jul 16 22:20:23.325354 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3522/7038] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:20:23 2022] GET /compute/v2.1/servers/d7dd236d-4a79-4885-843d-e280807faed7 => generated 1391 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:20:23.711715 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-2b5648bb-81c5-447e-b9e7-915d64b975df req-2de6bbe1-2026-4b6c-81e5-ecf1d77bc648 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "d7dd236d-4a79-4885-843d-e280807faed7", "tag": "2a69c584-a331-4d4e-b886-014d2e9250d4"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:20:23.723199 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-2b5648bb-81c5-447e-b9e7-915d64b975df req-2de6bbe1-2026-4b6c-81e5-ecf1d77bc648 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:20:23.726604 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-2b5648bb-81c5-447e-b9e7-915d64b975df req-2de6bbe1-2026-4b6c-81e5-ecf1d77bc648 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:20:23.735449 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-2b5648bb-81c5-447e-b9e7-915d64b975df req-2de6bbe1-2026-4b6c-81e5-ecf1d77bc648 service nova] Creating event network-changed:2a69c584-a331-4d4e-b886-014d2e9250d4 for instance d7dd236d-4a79-4885-843d-e280807faed7 on nch1-master-1 Jul 16 22:20:23.742320 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-2b5648bb-81c5-447e-b9e7-915d64b975df req-2de6bbe1-2026-4b6c-81e5-ecf1d77bc648 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.034592 Jul 16 22:20:23.742935 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3517/7039] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 22:20:23 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 22:20:24.341890 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-b3099bad-dca9-46f4-98e4-11efc2151882 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:20:24.342521 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-b3099bad-dca9-46f4-98e4-11efc2151882 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] [instance: d7dd236d-4a79-4885-843d-e280807faed7] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:20:24.350595 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b3099bad-dca9-46f4-98e4-11efc2151882 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:20:24.350996 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b3099bad-dca9-46f4-98e4-11efc2151882 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:20:24.351803 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b3099bad-dca9-46f4-98e4-11efc2151882 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:20:24.352258 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b3099bad-dca9-46f4-98e4-11efc2151882 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:20:24.530311 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-b3099bad-dca9-46f4-98e4-11efc2151882 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4b8c7ad3cc6240c096c2c7cf31d8f74f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a45a8ce6687c427d8f76e4205a076a93', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:20:24.538039 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-b3099bad-dca9-46f4-98e4-11efc2151882 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "GET /compute/v2.1/servers/d7dd236d-4a79-4885-843d-e280807faed7" status: 200 len: 1391 microversion: 2.1 time: 0.200069 Jul 16 22:20:24.538619 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3523/7040] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:20:24 2022] GET /compute/v2.1/servers/d7dd236d-4a79-4885-843d-e280807faed7 => generated 1391 bytes in 201 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:20:25.556841 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-a4755dbb-8f1e-47d3-a736-1b86b2c86e84 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:20:25.557595 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-a4755dbb-8f1e-47d3-a736-1b86b2c86e84 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] [instance: d7dd236d-4a79-4885-843d-e280807faed7] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:20:25.559601 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_api:SELECT=30,nova_api:INSERT=1 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:20:25.564596 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a4755dbb-8f1e-47d3-a736-1b86b2c86e84 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:20:25.565124 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a4755dbb-8f1e-47d3-a736-1b86b2c86e84 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:20:25.565994 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a4755dbb-8f1e-47d3-a736-1b86b2c86e84 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:20:25.566445 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a4755dbb-8f1e-47d3-a736-1b86b2c86e84 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:20:25.706451 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-a4755dbb-8f1e-47d3-a736-1b86b2c86e84 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4b8c7ad3cc6240c096c2c7cf31d8f74f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a45a8ce6687c427d8f76e4205a076a93', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:20:25.714097 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-a4755dbb-8f1e-47d3-a736-1b86b2c86e84 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "GET /compute/v2.1/servers/d7dd236d-4a79-4885-843d-e280807faed7" status: 200 len: 1391 microversion: 2.1 time: 0.162670 Jul 16 22:20:25.715879 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3518/7041] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:20:25 2022] GET /compute/v2.1/servers/d7dd236d-4a79-4885-843d-e280807faed7 => generated 1391 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:20:26.732061 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-5be0daf8-7dd6-48d1-b180-1cd3576e2e04 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:20:26.732635 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-5be0daf8-7dd6-48d1-b180-1cd3576e2e04 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] [instance: d7dd236d-4a79-4885-843d-e280807faed7] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:20:26.740246 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5be0daf8-7dd6-48d1-b180-1cd3576e2e04 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:20:26.740717 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5be0daf8-7dd6-48d1-b180-1cd3576e2e04 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:20:26.741506 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5be0daf8-7dd6-48d1-b180-1cd3576e2e04 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:20:26.741962 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5be0daf8-7dd6-48d1-b180-1cd3576e2e04 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:20:26.875356 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-5be0daf8-7dd6-48d1-b180-1cd3576e2e04 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4b8c7ad3cc6240c096c2c7cf31d8f74f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a45a8ce6687c427d8f76e4205a076a93', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:20:26.883332 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-5be0daf8-7dd6-48d1-b180-1cd3576e2e04 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "GET /compute/v2.1/servers/d7dd236d-4a79-4885-843d-e280807faed7" status: 200 len: 1391 microversion: 2.1 time: 0.155230 Jul 16 22:20:26.883926 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3524/7042] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:20:26 2022] GET /compute/v2.1/servers/d7dd236d-4a79-4885-843d-e280807faed7 => generated 1391 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:20:27.900484 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-c0ca1949-81c9-451b-bc10-9014c0600915 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:20:27.901098 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-c0ca1949-81c9-451b-bc10-9014c0600915 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] [instance: d7dd236d-4a79-4885-843d-e280807faed7] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:20:27.908025 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c0ca1949-81c9-451b-bc10-9014c0600915 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:20:27.908826 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c0ca1949-81c9-451b-bc10-9014c0600915 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:20:27.909697 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c0ca1949-81c9-451b-bc10-9014c0600915 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:20:27.910130 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c0ca1949-81c9-451b-bc10-9014c0600915 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:20:28.044888 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-c0ca1949-81c9-451b-bc10-9014c0600915 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4b8c7ad3cc6240c096c2c7cf31d8f74f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a45a8ce6687c427d8f76e4205a076a93', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:20:28.053081 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-c0ca1949-81c9-451b-bc10-9014c0600915 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "GET /compute/v2.1/servers/d7dd236d-4a79-4885-843d-e280807faed7" status: 200 len: 1391 microversion: 2.1 time: 0.156644 Jul 16 22:20:28.053645 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3519/7043] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:20:27 2022] GET /compute/v2.1/servers/d7dd236d-4a79-4885-843d-e280807faed7 => generated 1391 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:20:28.864526 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-1128ef85-98e9-4f74-8588-c7a098c81120 req-2f16b801-6cba-4e6f-b9d0-2e0397fdb369 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "d7dd236d-4a79-4885-843d-e280807faed7", "name": "network-vif-plugged", "status": "completed", "tag": "2a69c584-a331-4d4e-b886-014d2e9250d4"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:20:28.873929 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-1128ef85-98e9-4f74-8588-c7a098c81120 req-2f16b801-6cba-4e6f-b9d0-2e0397fdb369 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:20:28.874401 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-1128ef85-98e9-4f74-8588-c7a098c81120 req-2f16b801-6cba-4e6f-b9d0-2e0397fdb369 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:20:28.885521 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-1128ef85-98e9-4f74-8588-c7a098c81120 req-2f16b801-6cba-4e6f-b9d0-2e0397fdb369 service nova] Creating event network-vif-plugged:2a69c584-a331-4d4e-b886-014d2e9250d4 for instance d7dd236d-4a79-4885-843d-e280807faed7 on nch1-master-1 Jul 16 22:20:28.890655 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-1128ef85-98e9-4f74-8588-c7a098c81120 req-2f16b801-6cba-4e6f-b9d0-2e0397fdb369 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.030090 Jul 16 22:20:28.891289 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3525/7044] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 22:20:28 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 16 22:20:29.070602 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-326d2aa6-064d-496d-aac6-4f653d02e890 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:20:29.071256 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-326d2aa6-064d-496d-aac6-4f653d02e890 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] [instance: d7dd236d-4a79-4885-843d-e280807faed7] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:20:29.078346 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-326d2aa6-064d-496d-aac6-4f653d02e890 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:20:29.078802 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-326d2aa6-064d-496d-aac6-4f653d02e890 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:20:29.079817 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-326d2aa6-064d-496d-aac6-4f653d02e890 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:20:29.080874 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-326d2aa6-064d-496d-aac6-4f653d02e890 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:20:29.214646 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-326d2aa6-064d-496d-aac6-4f653d02e890 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4b8c7ad3cc6240c096c2c7cf31d8f74f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a45a8ce6687c427d8f76e4205a076a93', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:20:29.222932 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-326d2aa6-064d-496d-aac6-4f653d02e890 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "GET /compute/v2.1/servers/d7dd236d-4a79-4885-843d-e280807faed7" status: 200 len: 1391 microversion: 2.1 time: 0.156299 Jul 16 22:20:29.223532 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3520/7045] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:20:29 2022] GET /compute/v2.1/servers/d7dd236d-4a79-4885-843d-e280807faed7 => generated 1391 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:20:30.237397 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell0:SELECT=2 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:20:30.238236 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell0:SELECT=6 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:20:30.243906 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-99cfd2b0-0993-4983-b3d1-a9eab0994f63 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:20:30.244632 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-99cfd2b0-0993-4983-b3d1-a9eab0994f63 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] [instance: d7dd236d-4a79-4885-843d-e280807faed7] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:20:30.251256 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-99cfd2b0-0993-4983-b3d1-a9eab0994f63 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:20:30.251726 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-99cfd2b0-0993-4983-b3d1-a9eab0994f63 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:20:30.253572 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-99cfd2b0-0993-4983-b3d1-a9eab0994f63 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:20:30.254030 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-99cfd2b0-0993-4983-b3d1-a9eab0994f63 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:20:30.396088 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-99cfd2b0-0993-4983-b3d1-a9eab0994f63 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4b8c7ad3cc6240c096c2c7cf31d8f74f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a45a8ce6687c427d8f76e4205a076a93', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:20:30.403970 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-99cfd2b0-0993-4983-b3d1-a9eab0994f63 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "GET /compute/v2.1/servers/d7dd236d-4a79-4885-843d-e280807faed7" status: 200 len: 1564 microversion: 2.1 time: 0.168079 Jul 16 22:20:30.404572 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3526/7046] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:20:30 2022] GET /compute/v2.1/servers/d7dd236d-4a79-4885-843d-e280807faed7 => generated 1564 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:20:30.421128 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-4d969617-1b92-4b03-970d-34d70683fe15 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:20:30.421757 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-4d969617-1b92-4b03-970d-34d70683fe15 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] [instance: d7dd236d-4a79-4885-843d-e280807faed7] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:20:30.424395 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell0:SELECT=6 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:20:30.432795 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4d969617-1b92-4b03-970d-34d70683fe15 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:20:30.432795 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4d969617-1b92-4b03-970d-34d70683fe15 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:20:30.432795 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4d969617-1b92-4b03-970d-34d70683fe15 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:20:30.432795 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4d969617-1b92-4b03-970d-34d70683fe15 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:20:30.565578 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-4d969617-1b92-4b03-970d-34d70683fe15 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4b8c7ad3cc6240c096c2c7cf31d8f74f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a45a8ce6687c427d8f76e4205a076a93', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:20:30.573559 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-4d969617-1b92-4b03-970d-34d70683fe15 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "GET /compute/v2.1/servers/d7dd236d-4a79-4885-843d-e280807faed7" status: 200 len: 1564 microversion: 2.1 time: 0.156357 Jul 16 22:20:30.574142 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3521/7047] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:20:30 2022] GET /compute/v2.1/servers/d7dd236d-4a79-4885-843d-e280807faed7 => generated 1564 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:20:32.810494 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-eb5e0784-7546-4710-be9c-7098c939d364 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:20:32.811325 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-eb5e0784-7546-4710-be9c-7098c939d364 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] [instance: d7dd236d-4a79-4885-843d-e280807faed7] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:20:32.813207 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-18f89ac6-22e7-4d9c-8089-d04acea559ef req-15a52703-1638-498b-b051-87a10b1ab64b service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "d7dd236d-4a79-4885-843d-e280807faed7", "tag": "2a69c584-a331-4d4e-b886-014d2e9250d4"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:20:32.819568 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-eb5e0784-7546-4710-be9c-7098c939d364 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:20:32.820042 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-eb5e0784-7546-4710-be9c-7098c939d364 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:20:32.820838 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-eb5e0784-7546-4710-be9c-7098c939d364 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:20:32.821695 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-eb5e0784-7546-4710-be9c-7098c939d364 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:20:32.829033 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-18f89ac6-22e7-4d9c-8089-d04acea559ef req-15a52703-1638-498b-b051-87a10b1ab64b service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:20:32.829565 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-18f89ac6-22e7-4d9c-8089-d04acea559ef req-15a52703-1638-498b-b051-87a10b1ab64b service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:20:32.845750 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-18f89ac6-22e7-4d9c-8089-d04acea559ef req-15a52703-1638-498b-b051-87a10b1ab64b service nova] Creating event network-changed:2a69c584-a331-4d4e-b886-014d2e9250d4 for instance d7dd236d-4a79-4885-843d-e280807faed7 on nch1-master-1 Jul 16 22:20:32.851928 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-18f89ac6-22e7-4d9c-8089-d04acea559ef req-15a52703-1638-498b-b051-87a10b1ab64b service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.043036 Jul 16 22:20:32.852671 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3522/7048] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 22:20:32 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 16 22:20:33.057131 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-eb5e0784-7546-4710-be9c-7098c939d364 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4b8c7ad3cc6240c096c2c7cf31d8f74f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a45a8ce6687c427d8f76e4205a076a93', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:20:33.068347 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-eb5e0784-7546-4710-be9c-7098c939d364 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "GET /compute/v2.1/servers/d7dd236d-4a79-4885-843d-e280807faed7" status: 200 len: 1564 microversion: 2.1 time: 0.262638 Jul 16 22:20:33.069803 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3527/7049] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:20:32 2022] GET /compute/v2.1/servers/d7dd236d-4a79-4885-843d-e280807faed7 => generated 1564 bytes in 265 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:20:56.360468 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-4e5ca789-2087-48c8-96d0-9d44b115054c tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Action: 'action', calling method: >, body: {"os-getConsoleOutput": {}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:20:56.361487 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-4e5ca789-2087-48c8-96d0-9d44b115054c tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] [instance: d7dd236d-4a79-4885-843d-e280807faed7] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:20:56.364303 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_api:SELECT=9 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:20:56.365126 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=34 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:20:56.370085 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4e5ca789-2087-48c8-96d0-9d44b115054c tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:20:56.370536 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4e5ca789-2087-48c8-96d0-9d44b115054c tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:20:56.371340 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4e5ca789-2087-48c8-96d0-9d44b115054c tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:20:56.371784 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4e5ca789-2087-48c8-96d0-9d44b115054c tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:20:56.423100 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-4e5ca789-2087-48c8-96d0-9d44b115054c tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "POST /compute/v2.1/servers/d7dd236d-4a79-4885-843d-e280807faed7/action" status: 200 len: 24566 microversion: 2.1 time: 0.067016 Jul 16 22:20:56.423863 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3523/7050] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 22:20:56 2022] POST /compute/v2.1/servers/d7dd236d-4a79-4885-843d-e280807faed7/action => generated 24566 bytes in 69 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 16 22:21:02.802328 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-10e2d793-4db0-489d-aa95-0ba3ebb06c35 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Action: 'action', calling method: >, body: {"os-stop": {}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:21:02.804603 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_api:SELECT=45,nova_api:UPDATE=1,nova_api:DELETE=1,nova_api:INSERT=3 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:21:02.805636 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=40 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:21:02.811782 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-10e2d793-4db0-489d-aa95-0ba3ebb06c35 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:21:02.812255 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-10e2d793-4db0-489d-aa95-0ba3ebb06c35 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:21:02.835745 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-10e2d793-4db0-489d-aa95-0ba3ebb06c35 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] [instance: d7dd236d-4a79-4885-843d-e280807faed7] Going to try to stop instance {{(pid=138681) force_stop /opt/stack/nova/nova/compute/api.py:2707}} Jul 16 22:21:03.022069 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-10e2d793-4db0-489d-aa95-0ba3ebb06c35 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "POST /compute/v2.1/servers/d7dd236d-4a79-4885-843d-e280807faed7/action" status: 202 len: 0 microversion: 2.1 time: 0.224028 Jul 16 22:21:03.022824 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3528/7051] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 22:21:02 2022] POST /compute/v2.1/servers/d7dd236d-4a79-4885-843d-e280807faed7/action => generated 0 bytes in 226 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 22:21:03.032177 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-9ed596b9-4324-4a11-8198-c66319be2390 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:21:03.033018 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-9ed596b9-4324-4a11-8198-c66319be2390 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] [instance: d7dd236d-4a79-4885-843d-e280807faed7] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:21:03.041648 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9ed596b9-4324-4a11-8198-c66319be2390 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:21:03.042110 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9ed596b9-4324-4a11-8198-c66319be2390 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:21:03.042889 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9ed596b9-4324-4a11-8198-c66319be2390 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:21:03.043480 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9ed596b9-4324-4a11-8198-c66319be2390 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:21:03.185639 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-9ed596b9-4324-4a11-8198-c66319be2390 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4b8c7ad3cc6240c096c2c7cf31d8f74f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a45a8ce6687c427d8f76e4205a076a93', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:21:03.193620 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-9ed596b9-4324-4a11-8198-c66319be2390 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "GET /compute/v2.1/servers/d7dd236d-4a79-4885-843d-e280807faed7" status: 200 len: 1692 microversion: 2.1 time: 0.166276 Jul 16 22:21:03.194180 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3524/7052] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:21:03 2022] GET /compute/v2.1/servers/d7dd236d-4a79-4885-843d-e280807faed7 => generated 1692 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 22:21:04.212608 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-a823ee95-03f4-429a-b31c-fb6e7973650a tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:21:04.213273 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-a823ee95-03f4-429a-b31c-fb6e7973650a tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] [instance: d7dd236d-4a79-4885-843d-e280807faed7] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:21:04.220152 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a823ee95-03f4-429a-b31c-fb6e7973650a tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:21:04.220623 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a823ee95-03f4-429a-b31c-fb6e7973650a tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:21:04.221410 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a823ee95-03f4-429a-b31c-fb6e7973650a tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:21:04.221869 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-a823ee95-03f4-429a-b31c-fb6e7973650a tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:21:04.371901 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-a823ee95-03f4-429a-b31c-fb6e7973650a tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4b8c7ad3cc6240c096c2c7cf31d8f74f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a45a8ce6687c427d8f76e4205a076a93', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:21:04.379740 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-a823ee95-03f4-429a-b31c-fb6e7973650a tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "GET /compute/v2.1/servers/d7dd236d-4a79-4885-843d-e280807faed7" status: 200 len: 1692 microversion: 2.1 time: 0.173047 Jul 16 22:21:04.380334 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3529/7053] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:21:04 2022] GET /compute/v2.1/servers/d7dd236d-4a79-4885-843d-e280807faed7 => generated 1692 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:21:05.396900 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-c9f16cce-0427-4570-9a9e-2b502875afec tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:21:05.397531 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-c9f16cce-0427-4570-9a9e-2b502875afec tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] [instance: d7dd236d-4a79-4885-843d-e280807faed7] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:21:05.404635 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c9f16cce-0427-4570-9a9e-2b502875afec tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:21:05.405097 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c9f16cce-0427-4570-9a9e-2b502875afec tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:21:05.405872 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c9f16cce-0427-4570-9a9e-2b502875afec tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:21:05.406329 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c9f16cce-0427-4570-9a9e-2b502875afec tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:21:05.539732 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-c9f16cce-0427-4570-9a9e-2b502875afec tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4b8c7ad3cc6240c096c2c7cf31d8f74f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a45a8ce6687c427d8f76e4205a076a93', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:21:05.547282 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-c9f16cce-0427-4570-9a9e-2b502875afec tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "GET /compute/v2.1/servers/d7dd236d-4a79-4885-843d-e280807faed7" status: 200 len: 1692 microversion: 2.1 time: 0.154610 Jul 16 22:21:05.547885 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3525/7054] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:21:05 2022] GET /compute/v2.1/servers/d7dd236d-4a79-4885-843d-e280807faed7 => generated 1692 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:21:06.565249 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-aa65baf9-1888-41f5-a0f5-a54393de3987 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:21:06.565967 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-aa65baf9-1888-41f5-a0f5-a54393de3987 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] [instance: d7dd236d-4a79-4885-843d-e280807faed7] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:21:06.573603 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-aa65baf9-1888-41f5-a0f5-a54393de3987 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:21:06.574068 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-aa65baf9-1888-41f5-a0f5-a54393de3987 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:21:06.574847 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-aa65baf9-1888-41f5-a0f5-a54393de3987 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:21:06.575325 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-aa65baf9-1888-41f5-a0f5-a54393de3987 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:21:06.711624 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-aa65baf9-1888-41f5-a0f5-a54393de3987 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4b8c7ad3cc6240c096c2c7cf31d8f74f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a45a8ce6687c427d8f76e4205a076a93', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:21:06.719228 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-aa65baf9-1888-41f5-a0f5-a54393de3987 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "GET /compute/v2.1/servers/d7dd236d-4a79-4885-843d-e280807faed7" status: 200 len: 1692 microversion: 2.1 time: 0.157953 Jul 16 22:21:06.719744 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3530/7055] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:21:06 2022] GET /compute/v2.1/servers/d7dd236d-4a79-4885-843d-e280807faed7 => generated 1692 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:21:07.737162 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-30fd66a7-c29e-4e11-82d3-3af5fdc2c092 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:21:07.737818 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-30fd66a7-c29e-4e11-82d3-3af5fdc2c092 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] [instance: d7dd236d-4a79-4885-843d-e280807faed7] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:21:07.744832 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-30fd66a7-c29e-4e11-82d3-3af5fdc2c092 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:21:07.745293 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-30fd66a7-c29e-4e11-82d3-3af5fdc2c092 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:21:07.746062 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-30fd66a7-c29e-4e11-82d3-3af5fdc2c092 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:21:07.746513 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-30fd66a7-c29e-4e11-82d3-3af5fdc2c092 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:21:07.879936 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-30fd66a7-c29e-4e11-82d3-3af5fdc2c092 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4b8c7ad3cc6240c096c2c7cf31d8f74f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a45a8ce6687c427d8f76e4205a076a93', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:21:07.887659 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-30fd66a7-c29e-4e11-82d3-3af5fdc2c092 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "GET /compute/v2.1/servers/d7dd236d-4a79-4885-843d-e280807faed7" status: 200 len: 1692 microversion: 2.1 time: 0.154461 Jul 16 22:21:07.894021 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3526/7056] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:21:07 2022] GET /compute/v2.1/servers/d7dd236d-4a79-4885-843d-e280807faed7 => generated 1692 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:21:08.905597 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-9d19e09f-c49f-4af9-80d5-b9730f48edf9 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:21:08.906230 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-9d19e09f-c49f-4af9-80d5-b9730f48edf9 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] [instance: d7dd236d-4a79-4885-843d-e280807faed7] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:21:08.914321 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9d19e09f-c49f-4af9-80d5-b9730f48edf9 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:21:08.916603 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9d19e09f-c49f-4af9-80d5-b9730f48edf9 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:21:08.916603 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9d19e09f-c49f-4af9-80d5-b9730f48edf9 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:21:08.916603 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9d19e09f-c49f-4af9-80d5-b9730f48edf9 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:21:09.054280 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-9d19e09f-c49f-4af9-80d5-b9730f48edf9 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4b8c7ad3cc6240c096c2c7cf31d8f74f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a45a8ce6687c427d8f76e4205a076a93', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:21:09.062093 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-9d19e09f-c49f-4af9-80d5-b9730f48edf9 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "GET /compute/v2.1/servers/d7dd236d-4a79-4885-843d-e280807faed7" status: 200 len: 1692 microversion: 2.1 time: 0.160571 Jul 16 22:21:09.062660 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3531/7057] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:21:08 2022] GET /compute/v2.1/servers/d7dd236d-4a79-4885-843d-e280807faed7 => generated 1692 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:21:10.079996 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-5bfdd87c-17ce-43f5-915c-e72555695f7e tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:21:10.080657 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-5bfdd87c-17ce-43f5-915c-e72555695f7e tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] [instance: d7dd236d-4a79-4885-843d-e280807faed7] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:21:10.087760 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5bfdd87c-17ce-43f5-915c-e72555695f7e tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:21:10.088229 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5bfdd87c-17ce-43f5-915c-e72555695f7e tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:21:10.089005 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5bfdd87c-17ce-43f5-915c-e72555695f7e tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:21:10.089454 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5bfdd87c-17ce-43f5-915c-e72555695f7e tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:21:10.223331 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-5bfdd87c-17ce-43f5-915c-e72555695f7e tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4b8c7ad3cc6240c096c2c7cf31d8f74f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a45a8ce6687c427d8f76e4205a076a93', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:21:10.231037 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-5bfdd87c-17ce-43f5-915c-e72555695f7e tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "GET /compute/v2.1/servers/d7dd236d-4a79-4885-843d-e280807faed7" status: 200 len: 1692 microversion: 2.1 time: 0.155130 Jul 16 22:21:10.245077 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3527/7058] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:21:10 2022] GET /compute/v2.1/servers/d7dd236d-4a79-4885-843d-e280807faed7 => generated 1692 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:21:11.248613 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-98dcc3ad-ffd2-4544-b36a-22a9b025c841 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:21:11.249146 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-98dcc3ad-ffd2-4544-b36a-22a9b025c841 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] [instance: d7dd236d-4a79-4885-843d-e280807faed7] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:21:11.256780 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-98dcc3ad-ffd2-4544-b36a-22a9b025c841 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:21:11.257251 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-98dcc3ad-ffd2-4544-b36a-22a9b025c841 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:21:11.258030 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-98dcc3ad-ffd2-4544-b36a-22a9b025c841 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:21:11.258482 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-98dcc3ad-ffd2-4544-b36a-22a9b025c841 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:21:11.397821 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-98dcc3ad-ffd2-4544-b36a-22a9b025c841 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4b8c7ad3cc6240c096c2c7cf31d8f74f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a45a8ce6687c427d8f76e4205a076a93', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:21:11.405300 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-98dcc3ad-ffd2-4544-b36a-22a9b025c841 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "GET /compute/v2.1/servers/d7dd236d-4a79-4885-843d-e280807faed7" status: 200 len: 1692 microversion: 2.1 time: 0.160722 Jul 16 22:21:11.405871 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3532/7059] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:21:11 2022] GET /compute/v2.1/servers/d7dd236d-4a79-4885-843d-e280807faed7 => generated 1692 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:21:12.422659 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-2179caf9-945b-41e1-b9cf-1020149cd7f0 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:21:12.423312 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-2179caf9-945b-41e1-b9cf-1020149cd7f0 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] [instance: d7dd236d-4a79-4885-843d-e280807faed7] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:21:12.431161 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2179caf9-945b-41e1-b9cf-1020149cd7f0 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:21:12.431624 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2179caf9-945b-41e1-b9cf-1020149cd7f0 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:21:12.432401 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2179caf9-945b-41e1-b9cf-1020149cd7f0 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:21:12.432875 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2179caf9-945b-41e1-b9cf-1020149cd7f0 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:21:12.565177 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-2179caf9-945b-41e1-b9cf-1020149cd7f0 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4b8c7ad3cc6240c096c2c7cf31d8f74f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a45a8ce6687c427d8f76e4205a076a93', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:21:12.573653 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-2179caf9-945b-41e1-b9cf-1020149cd7f0 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "GET /compute/v2.1/servers/d7dd236d-4a79-4885-843d-e280807faed7" status: 200 len: 1692 microversion: 2.1 time: 0.154992 Jul 16 22:21:12.574216 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3528/7060] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:21:12 2022] GET /compute/v2.1/servers/d7dd236d-4a79-4885-843d-e280807faed7 => generated 1692 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:21:13.590973 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-0fc34c89-0ad4-405f-9692-56ce62cd6b1e tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:21:13.591612 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-0fc34c89-0ad4-405f-9692-56ce62cd6b1e tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] [instance: d7dd236d-4a79-4885-843d-e280807faed7] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:21:13.598688 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0fc34c89-0ad4-405f-9692-56ce62cd6b1e tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:21:13.599199 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0fc34c89-0ad4-405f-9692-56ce62cd6b1e tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:21:13.599979 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0fc34c89-0ad4-405f-9692-56ce62cd6b1e tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:21:13.600430 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0fc34c89-0ad4-405f-9692-56ce62cd6b1e tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:21:13.732112 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-0fc34c89-0ad4-405f-9692-56ce62cd6b1e tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4b8c7ad3cc6240c096c2c7cf31d8f74f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a45a8ce6687c427d8f76e4205a076a93', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:21:13.739826 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-0fc34c89-0ad4-405f-9692-56ce62cd6b1e tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "GET /compute/v2.1/servers/d7dd236d-4a79-4885-843d-e280807faed7" status: 200 len: 1669 microversion: 2.1 time: 0.152821 Jul 16 22:21:13.740404 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3533/7061] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:21:13 2022] GET /compute/v2.1/servers/d7dd236d-4a79-4885-843d-e280807faed7 => generated 1669 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:21:14.831501 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-0f62c283-8b61-43d7-8fa0-833673de4b68 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Action: 'action', calling method: >, body: {"os-getConsoleOutput": {}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:21:14.833417 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-0f62c283-8b61-43d7-8fa0-833673de4b68 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] [instance: d7dd236d-4a79-4885-843d-e280807faed7] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:21:14.840305 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0f62c283-8b61-43d7-8fa0-833673de4b68 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:21:14.840760 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0f62c283-8b61-43d7-8fa0-833673de4b68 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:21:14.841521 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0f62c283-8b61-43d7-8fa0-833673de4b68 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:21:14.841968 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-0f62c283-8b61-43d7-8fa0-833673de4b68 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:21:14.893202 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-0f62c283-8b61-43d7-8fa0-833673de4b68 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "POST /compute/v2.1/servers/d7dd236d-4a79-4885-843d-e280807faed7/action" status: 200 len: 35972 microversion: 2.1 time: 0.065452 Jul 16 22:21:14.894658 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3529/7062] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 22:21:14 2022] POST /compute/v2.1/servers/d7dd236d-4a79-4885-843d-e280807faed7/action => generated 35972 bytes in 67 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 16 22:21:14.906044 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-bd9373df-35f6-4ac8-81a5-e85422127766 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Action: 'action', calling method: >, body: {"os-start": {}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:21:14.914018 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-bd9373df-35f6-4ac8-81a5-e85422127766 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:21:14.914627 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-bd9373df-35f6-4ac8-81a5-e85422127766 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:21:14.937609 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-bd9373df-35f6-4ac8-81a5-e85422127766 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] [instance: d7dd236d-4a79-4885-843d-e280807faed7] Going to try to start instance {{(pid=138681) start /opt/stack/nova/nova/compute/api.py:2730}} Jul 16 22:21:15.120714 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-bd9373df-35f6-4ac8-81a5-e85422127766 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "POST /compute/v2.1/servers/d7dd236d-4a79-4885-843d-e280807faed7/action" status: 202 len: 0 microversion: 2.1 time: 0.218493 Jul 16 22:21:15.121294 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3534/7063] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 22:21:14 2022] POST /compute/v2.1/servers/d7dd236d-4a79-4885-843d-e280807faed7/action => generated 0 bytes in 220 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 22:21:15.131422 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-e30f1e4a-ce6e-4626-9a6d-f8664a5fd15a tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:21:15.132142 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-e30f1e4a-ce6e-4626-9a6d-f8664a5fd15a tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] [instance: d7dd236d-4a79-4885-843d-e280807faed7] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:21:15.139458 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e30f1e4a-ce6e-4626-9a6d-f8664a5fd15a tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:21:15.139916 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e30f1e4a-ce6e-4626-9a6d-f8664a5fd15a tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:21:15.140684 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e30f1e4a-ce6e-4626-9a6d-f8664a5fd15a tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:21:15.141127 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e30f1e4a-ce6e-4626-9a6d-f8664a5fd15a tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:21:15.295327 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-e30f1e4a-ce6e-4626-9a6d-f8664a5fd15a tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4b8c7ad3cc6240c096c2c7cf31d8f74f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a45a8ce6687c427d8f76e4205a076a93', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:21:15.305913 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-e30f1e4a-ce6e-4626-9a6d-f8664a5fd15a tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "GET /compute/v2.1/servers/d7dd236d-4a79-4885-843d-e280807faed7" status: 200 len: 1678 microversion: 2.1 time: 0.179659 Jul 16 22:21:15.306519 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3530/7064] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:21:15 2022] GET /compute/v2.1/servers/d7dd236d-4a79-4885-843d-e280807faed7 => generated 1678 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:21:16.324490 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-49472855-231c-4f35-893d-a823cd0e6958 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:21:16.325213 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-49472855-231c-4f35-893d-a823cd0e6958 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] [instance: d7dd236d-4a79-4885-843d-e280807faed7] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:21:16.333491 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-49472855-231c-4f35-893d-a823cd0e6958 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:21:16.333957 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-49472855-231c-4f35-893d-a823cd0e6958 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:21:16.334785 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-49472855-231c-4f35-893d-a823cd0e6958 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:21:16.335265 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-49472855-231c-4f35-893d-a823cd0e6958 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:21:16.474008 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-49472855-231c-4f35-893d-a823cd0e6958 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4b8c7ad3cc6240c096c2c7cf31d8f74f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a45a8ce6687c427d8f76e4205a076a93', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:21:16.481602 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-49472855-231c-4f35-893d-a823cd0e6958 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "GET /compute/v2.1/servers/d7dd236d-4a79-4885-843d-e280807faed7" status: 200 len: 1682 microversion: 2.1 time: 0.161631 Jul 16 22:21:16.482174 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3535/7065] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:21:16 2022] GET /compute/v2.1/servers/d7dd236d-4a79-4885-843d-e280807faed7 => generated 1682 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:21:40.672151 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-093aa41a-fc1f-4b21-996f-1d03971d62f5 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Action: 'action', calling method: >, body: {"os-getConsoleOutput": {}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:21:40.673050 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-093aa41a-fc1f-4b21-996f-1d03971d62f5 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] [instance: d7dd236d-4a79-4885-843d-e280807faed7] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:21:40.675809 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_api:SELECT=16 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:21:40.676520 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=44 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:21:40.682074 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-093aa41a-fc1f-4b21-996f-1d03971d62f5 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:21:40.682529 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-093aa41a-fc1f-4b21-996f-1d03971d62f5 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:21:40.683320 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-093aa41a-fc1f-4b21-996f-1d03971d62f5 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:21:40.683763 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-093aa41a-fc1f-4b21-996f-1d03971d62f5 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:21:40.738651 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-093aa41a-fc1f-4b21-996f-1d03971d62f5 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "POST /compute/v2.1/servers/d7dd236d-4a79-4885-843d-e280807faed7/action" status: 200 len: 60527 microversion: 2.1 time: 0.070317 Jul 16 22:21:40.743307 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3531/7066] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 22:21:40 2022] POST /compute/v2.1/servers/d7dd236d-4a79-4885-843d-e280807faed7/action => generated 60527 bytes in 72 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 16 22:21:47.845098 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-e3048478-defe-4b29-9e3d-f561a5cde5f9 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:21:47.856467 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-e3048478-defe-4b29-9e3d-f561a5cde5f9 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] [instance: d7dd236d-4a79-4885-843d-e280807faed7] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:21:47.856644 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-6343300f-1ede-45a7-9a9c-8ed56655aea3 req-30381f65-cdd3-4ea1-aa8c-36ffe2cf2dd7 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "d7dd236d-4a79-4885-843d-e280807faed7", "tag": "2a69c584-a331-4d4e-b886-014d2e9250d4"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:21:47.857112 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_api:SELECT=16 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:21:47.857112 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=56,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:21:47.857112 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e3048478-defe-4b29-9e3d-f561a5cde5f9 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:21:47.857112 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e3048478-defe-4b29-9e3d-f561a5cde5f9 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:21:47.857112 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e3048478-defe-4b29-9e3d-f561a5cde5f9 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:21:47.857112 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e3048478-defe-4b29-9e3d-f561a5cde5f9 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:21:47.858775 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-6343300f-1ede-45a7-9a9c-8ed56655aea3 req-30381f65-cdd3-4ea1-aa8c-36ffe2cf2dd7 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:21:47.859483 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-6343300f-1ede-45a7-9a9c-8ed56655aea3 req-30381f65-cdd3-4ea1-aa8c-36ffe2cf2dd7 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:21:47.872859 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-6343300f-1ede-45a7-9a9c-8ed56655aea3 req-30381f65-cdd3-4ea1-aa8c-36ffe2cf2dd7 service nova] Creating event network-changed:2a69c584-a331-4d4e-b886-014d2e9250d4 for instance d7dd236d-4a79-4885-843d-e280807faed7 on nch1-master-1 Jul 16 22:21:47.879794 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-6343300f-1ede-45a7-9a9c-8ed56655aea3 req-30381f65-cdd3-4ea1-aa8c-36ffe2cf2dd7 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.036152 Jul 16 22:21:47.880869 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3532/7067] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 22:21:47 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 16 22:21:47.892258 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-e3048478-defe-4b29-9e3d-f561a5cde5f9 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] [instance: d7dd236d-4a79-4885-843d-e280807faed7] Going to try to terminate instance {{(pid=138681) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 16 22:21:48.214725 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-e3048478-defe-4b29-9e3d-f561a5cde5f9 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "DELETE /compute/v2.1/servers/d7dd236d-4a79-4885-843d-e280807faed7" status: 204 len: 0 microversion: 2.1 time: 0.373508 Jul 16 22:21:48.216487 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3536/7068] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 22:21:47 2022] DELETE /compute/v2.1/servers/d7dd236d-4a79-4885-843d-e280807faed7 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 16 22:21:48.224628 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-1df350e4-8abc-4cb4-8005-fd1e75e5b9a5 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:21:48.225442 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-1df350e4-8abc-4cb4-8005-fd1e75e5b9a5 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] [instance: d7dd236d-4a79-4885-843d-e280807faed7] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:21:48.231956 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1df350e4-8abc-4cb4-8005-fd1e75e5b9a5 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:21:48.232443 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1df350e4-8abc-4cb4-8005-fd1e75e5b9a5 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:21:48.233201 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1df350e4-8abc-4cb4-8005-fd1e75e5b9a5 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:21:48.233653 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1df350e4-8abc-4cb4-8005-fd1e75e5b9a5 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:21:48.416878 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-1df350e4-8abc-4cb4-8005-fd1e75e5b9a5 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4b8c7ad3cc6240c096c2c7cf31d8f74f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a45a8ce6687c427d8f76e4205a076a93', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:21:48.424993 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-1df350e4-8abc-4cb4-8005-fd1e75e5b9a5 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "GET /compute/v2.1/servers/d7dd236d-4a79-4885-843d-e280807faed7" status: 200 len: 1688 microversion: 2.1 time: 0.205093 Jul 16 22:21:48.425580 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3533/7069] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:21:48 2022] GET /compute/v2.1/servers/d7dd236d-4a79-4885-843d-e280807faed7 => generated 1688 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:21:49.444104 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-7985db5a-c061-4e8c-a9c1-732ad4149265 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:21:49.444835 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-7985db5a-c061-4e8c-a9c1-732ad4149265 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] [instance: d7dd236d-4a79-4885-843d-e280807faed7] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:21:49.452054 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7985db5a-c061-4e8c-a9c1-732ad4149265 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:21:49.452529 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7985db5a-c061-4e8c-a9c1-732ad4149265 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:21:49.453319 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7985db5a-c061-4e8c-a9c1-732ad4149265 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:21:49.453780 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7985db5a-c061-4e8c-a9c1-732ad4149265 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:21:49.684590 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-7985db5a-c061-4e8c-a9c1-732ad4149265 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4b8c7ad3cc6240c096c2c7cf31d8f74f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a45a8ce6687c427d8f76e4205a076a93', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:21:49.692391 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-7985db5a-c061-4e8c-a9c1-732ad4149265 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "GET /compute/v2.1/servers/d7dd236d-4a79-4885-843d-e280807faed7" status: 200 len: 1688 microversion: 2.1 time: 0.252595 Jul 16 22:21:49.694376 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3537/7070] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:21:49 2022] GET /compute/v2.1/servers/d7dd236d-4a79-4885-843d-e280807faed7 => generated 1688 bytes in 255 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:21:50.167807 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-18de6b2d-7fa2-4538-aa57-d9d143e82c41 req-bc977f2d-0d8f-4c5a-bb90-229988444342 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "d7dd236d-4a79-4885-843d-e280807faed7", "name": "network-vif-deleted", "tag": "2a69c584-a331-4d4e-b886-014d2e9250d4"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:21:50.177765 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-18de6b2d-7fa2-4538-aa57-d9d143e82c41 req-bc977f2d-0d8f-4c5a-bb90-229988444342 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:21:50.183529 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-18de6b2d-7fa2-4538-aa57-d9d143e82c41 req-bc977f2d-0d8f-4c5a-bb90-229988444342 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:21:50.189939 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-18de6b2d-7fa2-4538-aa57-d9d143e82c41 req-bc977f2d-0d8f-4c5a-bb90-229988444342 service nova] Creating event network-vif-deleted:2a69c584-a331-4d4e-b886-014d2e9250d4 for instance d7dd236d-4a79-4885-843d-e280807faed7 on nch1-master-1 Jul 16 22:21:50.195872 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-18de6b2d-7fa2-4538-aa57-d9d143e82c41 req-bc977f2d-0d8f-4c5a-bb90-229988444342 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.031638 Jul 16 22:21:50.196461 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3534/7071] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 22:21:50 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 22:21:50.711389 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-6a4c5d41-6326-430f-a179-cf90cf091364 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:21:50.711928 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-6a4c5d41-6326-430f-a179-cf90cf091364 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] [instance: d7dd236d-4a79-4885-843d-e280807faed7] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:21:50.718835 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6a4c5d41-6326-430f-a179-cf90cf091364 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:21:50.719339 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6a4c5d41-6326-430f-a179-cf90cf091364 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:21:50.720153 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6a4c5d41-6326-430f-a179-cf90cf091364 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:21:50.720611 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6a4c5d41-6326-430f-a179-cf90cf091364 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:21:50.759598 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6a4c5d41-6326-430f-a179-cf90cf091364 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:21:50.759598 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6a4c5d41-6326-430f-a179-cf90cf091364 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:21:50.759830 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.objects.instance [None req-6a4c5d41-6326-430f-a179-cf90cf091364 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lazy-loading 'fault' on Instance uuid d7dd236d-4a79-4885-843d-e280807faed7 {{(pid=138681) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 22:21:50.802639 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-6a4c5d41-6326-430f-a179-cf90cf091364 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4b8c7ad3cc6240c096c2c7cf31d8f74f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a45a8ce6687c427d8f76e4205a076a93', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:21:50.810645 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-6a4c5d41-6326-430f-a179-cf90cf091364 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "GET /compute/v2.1/servers/d7dd236d-4a79-4885-843d-e280807faed7" status: 200 len: 1355 microversion: 2.1 time: 0.104062 Jul 16 22:21:50.811623 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3538/7072] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:21:50 2022] GET /compute/v2.1/servers/d7dd236d-4a79-4885-843d-e280807faed7 => generated 1355 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:21:51.829901 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-9ee60088-c4c3-45a2-a7e3-bddfadf54aea tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:21:51.830598 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-9ee60088-c4c3-45a2-a7e3-bddfadf54aea tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] [instance: d7dd236d-4a79-4885-843d-e280807faed7] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:21:51.838181 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9ee60088-c4c3-45a2-a7e3-bddfadf54aea tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:21:51.838635 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9ee60088-c4c3-45a2-a7e3-bddfadf54aea tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:21:51.839456 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9ee60088-c4c3-45a2-a7e3-bddfadf54aea tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:21:51.839903 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9ee60088-c4c3-45a2-a7e3-bddfadf54aea tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:21:51.854021 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-9ee60088-c4c3-45a2-a7e3-bddfadf54aea tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] HTTP exception thrown: Instance d7dd236d-4a79-4885-843d-e280807faed7 could not be found. Jul 16 22:21:51.855712 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-9ee60088-c4c3-45a2-a7e3-bddfadf54aea tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Request method failure captured: Jul 16 22:21:51.855712 ncdv-master-1 devstack@n-api.service[138680]: request: GET /compute/v2.1/servers/d7dd236d-4a79-4885-843d-e280807faed7 HTTP/1.1 Jul 16 22:21:51.855712 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 22:21:51.855712 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 22:21:51.855712 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 22:21:51.855712 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 0 Jul 16 22:21:51.855712 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 22:21:51.855712 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 22:21:51.855712 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 22:21:51.855712 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 22:21:51.855712 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0zhtDg4n0Cr97EpGJw_kphX8HozEdWitmpiSsnN8liwN7JwqYkEqcaIM-47rjt-yK-e5LgZo6pFGCSkRgNS-xnBjvFiClGjWelkYApUjhWHqw4F9MI58oj4QDcEbMKfIlL82tbwRwh1LdKROeHuRqnWXe9oskU-rprKG2IX2dbA4fe4 Jul 16 22:21:51.855712 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 22:21:51.855712 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 22:21:51.855712 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 22:21:51.855712 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 22:21:51.855712 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 22:21:51.855712 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 22:21:51.855712 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: a45a8ce6687c427d8f76e4205a076a93 Jul 16 22:21:51.855712 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-TestNetworkAdvancedServerOps-234580283 Jul 16 22:21:51.855712 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 22:21:51.855712 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 22:21:51.855712 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/a45a8ce6687c427d8f76e4205a076a93"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/a45a8ce6687c427d8f76e4205a076a93"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_a45a8ce6687c427d8f76e4205a076a93"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/a45a8ce6687c427d8f76e4205a076a93"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 22:21:51.855712 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-TestNetworkAdvancedServerOps-234580283 Jul 16 22:21:51.855712 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: a45a8ce6687c427d8f76e4205a076a93 Jul 16 22:21:51.855712 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-TestNetworkAdvancedServerOps-234580283 Jul 16 22:21:51.855712 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-TestNetworkAdvancedServerOps-234580283-project Jul 16 22:21:51.855712 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 22:21:51.855712 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 22:21:51.855712 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: 4b8c7ad3cc6240c096c2c7cf31d8f74f Jul 16 22:21:51.855712 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-TestNetworkAdvancedServerOps-234580283-project Jul 16 22:21:51.855712 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 22:21:51.855712 ncdv-master-1 devstack@n-api.service[138680]: exception: Instance d7dd236d-4a79-4885-843d-e280807faed7 could not be found. Jul 16 22:21:51.855712 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': 'd7dd236d-4a79-4885-843d-e280807faed7'} Jul 16 22:21:51.855712 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 22:21:51.855712 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 22:21:51.855712 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 22:21:51.855712 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 22:21:51.855712 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 22:21:51.855712 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 22:21:51.855712 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 22:21:51.855712 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 22:21:51.855712 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 22:21:51.855712 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 22:21:51.855712 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance d7dd236d-4a79-4885-843d-e280807faed7 could not be found. Jul 16 22:21:51.855712 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 22:21:51.855712 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 22:21:51.855712 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 22:21:51.855712 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 22:21:51.855712 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 22:21:51.855712 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 22:21:51.855712 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 22:21:51.855712 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 22:21:51.855712 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 22:21:51.855712 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 22:21:51.855712 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 22:21:51.855712 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 22:21:51.855712 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 22:21:51.855712 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 22:21:51.855712 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 22:21:51.855712 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 22:21:51.855712 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance d7dd236d-4a79-4885-843d-e280807faed7 could not be found. Jul 16 22:21:51.855712 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 22:21:51.855712 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 22:21:51.855712 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 22:21:51.861232 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 22:21:51.861232 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 22:21:51.861232 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 22:21:51.861232 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 22:21:51.861232 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 22:21:51.861232 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance d7dd236d-4a79-4885-843d-e280807faed7 could not be found. Jul 16 22:21:51.861232 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 22:21:51.861232 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-9ee60088-c4c3-45a2-a7e3-bddfadf54aea tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Returning 404 to user: Instance d7dd236d-4a79-4885-843d-e280807faed7 could not be found. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 22:21:51.861232 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-9ee60088-c4c3-45a2-a7e3-bddfadf54aea tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "GET /compute/v2.1/servers/d7dd236d-4a79-4885-843d-e280807faed7" status: 404 len: 111 microversion: 2.1 time: 0.031506 Jul 16 22:21:51.861232 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3535/7073] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:21:51 2022] GET /compute/v2.1/servers/d7dd236d-4a79-4885-843d-e280807faed7 => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 22:21:55.686928 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-2b0329ec-35bc-4b63-a533-86c4eacc0ce9 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Calling method '.version_select at 0x7fdd9c26fd30>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:21:55.768879 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-2b0329ec-35bc-4b63-a533-86c4eacc0ce9 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "DELETE /compute/v2.1/os-keypairs/tempest-TestNetworkAdvancedServerOps-564397440" status: 202 len: 0 microversion: 2.1 time: 0.085714 Jul 16 22:21:55.769477 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3539/7074] 10.222.0.16 () {58 vars in 1280 bytes} [Sat Jul 16 22:21:55 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestNetworkAdvancedServerOps-564397440 => generated 0 bytes in 87 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 22:21:55.800993 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-b7a37961-3588-4ec9-b4d8-ae2568dbc860 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Action: 'create', calling method: .version_select at 0x7fdd8fce0a60>, body: {"keypair": {"name": "tempest-TestNetworkAdvancedServerOps-2116104084", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBCS3To7Njs6OLyJO1X+4WxaiBCS+i8w2/CgpOUG1MCmGwVyfjHyl+vTrKl0VvP91BQsnFs9KF/dj5x9gXg75Qz9+qfBVKvcaof8O7tZfQvXUrktMlqM1vK8Ou1qt7LNnpg=="}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:21:55.816678 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-b7a37961-3588-4ec9-b4d8-ae2568dbc860 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Getting quotas for project a45a8ce6687c427d8f76e4205a076a93. Resources: {'key_pairs'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 22:21:55.822724 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-b7a37961-3588-4ec9-b4d8-ae2568dbc860 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Getting quotas for user 4b8c7ad3cc6240c096c2c7cf31d8f74f and project a45a8ce6687c427d8f76e4205a076a93. Resources: {'key_pairs'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 22:21:55.947682 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-b7a37961-3588-4ec9-b4d8-ae2568dbc860 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "POST /compute/v2.1/os-keypairs" status: 200 len: 407 microversion: 2.1 time: 0.150487 Jul 16 22:21:55.948339 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3536/7075] 10.222.0.16 () {60 vars in 1158 bytes} [Sat Jul 16 22:21:55 2022] POST /compute/v2.1/os-keypairs => generated 407 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 22:22:03.915943 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-6e090fb6-1670-469a-9521-da61fa9417ac tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestNetworkAdvancedServerOps-server-983544830", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452", "networks": [{"uuid": "c7353e54-6629-4de4-91d4-0d775b7bdb33"}], "key_name": "tempest-TestNetworkAdvancedServerOps-2116104084", "security_groups": [{"name": "tempest-secgroup-smoke-2111927935"}]}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:22:03.924776 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=28,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:22:04.050791 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.network.neutron [None req-6e090fb6-1670-469a-9521-da61fa9417ac tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] validate_networks() for [('c7353e54-6629-4de4-91d4-0d775b7bdb33', None, None, None, None, None)] {{(pid=138681) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 16 22:22:04.550397 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-6e090fb6-1670-469a-9521-da61fa9417ac tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Flavor limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 16 22:22:04.550847 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-6e090fb6-1670-469a-9521-da61fa9417ac tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Image limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 16 22:22:04.551434 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-6e090fb6-1670-469a-9521-da61fa9417ac tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Flavor pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 16 22:22:04.551900 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-6e090fb6-1670-469a-9521-da61fa9417ac tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Image pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 16 22:22:04.552272 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-6e090fb6-1670-469a-9521-da61fa9417ac tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 16 22:22:04.578659 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6e090fb6-1670-469a-9521-da61fa9417ac tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:22:04.579406 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6e090fb6-1670-469a-9521-da61fa9417ac tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:22:04.580463 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6e090fb6-1670-469a-9521-da61fa9417ac tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:22:04.580927 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6e090fb6-1670-469a-9521-da61fa9417ac tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:22:04.630789 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-6e090fb6-1670-469a-9521-da61fa9417ac tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Getting quotas for project a45a8ce6687c427d8f76e4205a076a93. Resources: {'ram', 'instances', 'cores'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 22:22:04.643963 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-6e090fb6-1670-469a-9521-da61fa9417ac tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Getting quotas for user 4b8c7ad3cc6240c096c2c7cf31d8f74f and project a45a8ce6687c427d8f76e4205a076a93. Resources: {'ram', 'instances', 'cores'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 22:22:04.660183 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-6e090fb6-1670-469a-9521-da61fa9417ac tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Going to run 1 instances... {{(pid=138681) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 16 22:22:04.662573 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-6e090fb6-1670-469a-9521-da61fa9417ac tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] [instance: 373f324d-27e7-42ba-90b0-6ed693d9cfec] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138681) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 16 22:22:04.831430 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-6e090fb6-1670-469a-9521-da61fa9417ac tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 406 microversion: 2.1 time: 0.919981 Jul 16 22:22:04.832748 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3540/7076] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 22:22:03 2022] POST /compute/v2.1/servers => generated 406 bytes in 922 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Jul 16 22:22:04.845004 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-ca7e3e78-e465-4008-a512-8368dfc998bf tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:22:04.846439 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-ca7e3e78-e465-4008-a512-8368dfc998bf tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] [instance: 373f324d-27e7-42ba-90b0-6ed693d9cfec] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:22:04.849920 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=16 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:22:04.927757 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-ca7e3e78-e465-4008-a512-8368dfc998bf tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4b8c7ad3cc6240c096c2c7cf31d8f74f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a45a8ce6687c427d8f76e4205a076a93', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:22:04.941645 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-ca7e3e78-e465-4008-a512-8368dfc998bf tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "GET /compute/v2.1/servers/373f324d-27e7-42ba-90b0-6ed693d9cfec" status: 200 len: 1266 microversion: 2.1 time: 0.101047 Jul 16 22:22:04.943790 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3537/7077] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:22:04 2022] GET /compute/v2.1/servers/373f324d-27e7-42ba-90b0-6ed693d9cfec => generated 1266 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:22:05.962561 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-7a684d19-9569-4502-be1e-531d93a9aef2 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:22:05.963407 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-7a684d19-9569-4502-be1e-531d93a9aef2 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] [instance: 373f324d-27e7-42ba-90b0-6ed693d9cfec] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:22:05.972198 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7a684d19-9569-4502-be1e-531d93a9aef2 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:22:05.977317 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7a684d19-9569-4502-be1e-531d93a9aef2 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:22:05.977317 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7a684d19-9569-4502-be1e-531d93a9aef2 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:22:05.977317 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7a684d19-9569-4502-be1e-531d93a9aef2 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:22:06.057892 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-7a684d19-9569-4502-be1e-531d93a9aef2 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4b8c7ad3cc6240c096c2c7cf31d8f74f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a45a8ce6687c427d8f76e4205a076a93', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:22:06.067560 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-7a684d19-9569-4502-be1e-531d93a9aef2 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "GET /compute/v2.1/servers/373f324d-27e7-42ba-90b0-6ed693d9cfec" status: 200 len: 1270 microversion: 2.1 time: 0.109852 Jul 16 22:22:06.068775 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3541/7078] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:22:05 2022] GET /compute/v2.1/servers/373f324d-27e7-42ba-90b0-6ed693d9cfec => generated 1270 bytes in 112 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:22:07.090580 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-4ae4d6cd-0435-4626-b7ec-d0c1865d739e tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:22:07.091256 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-4ae4d6cd-0435-4626-b7ec-d0c1865d739e tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] [instance: 373f324d-27e7-42ba-90b0-6ed693d9cfec] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:22:07.100773 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4ae4d6cd-0435-4626-b7ec-d0c1865d739e tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:22:07.103002 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4ae4d6cd-0435-4626-b7ec-d0c1865d739e tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:22:07.103002 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4ae4d6cd-0435-4626-b7ec-d0c1865d739e tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:22:07.103869 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4ae4d6cd-0435-4626-b7ec-d0c1865d739e tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:22:07.187098 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-4ae4d6cd-0435-4626-b7ec-d0c1865d739e tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4b8c7ad3cc6240c096c2c7cf31d8f74f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a45a8ce6687c427d8f76e4205a076a93', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:22:07.196134 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-4ae4d6cd-0435-4626-b7ec-d0c1865d739e tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "GET /compute/v2.1/servers/373f324d-27e7-42ba-90b0-6ed693d9cfec" status: 200 len: 1326 microversion: 2.1 time: 0.110145 Jul 16 22:22:07.196733 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3538/7079] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:22:07 2022] GET /compute/v2.1/servers/373f324d-27e7-42ba-90b0-6ed693d9cfec => generated 1326 bytes in 112 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:22:08.213989 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-65de88ed-2baf-4969-924d-06330bcf2082 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:22:08.214666 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-65de88ed-2baf-4969-924d-06330bcf2082 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] [instance: 373f324d-27e7-42ba-90b0-6ed693d9cfec] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:22:08.221960 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-65de88ed-2baf-4969-924d-06330bcf2082 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:22:08.222432 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-65de88ed-2baf-4969-924d-06330bcf2082 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:22:08.223269 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-65de88ed-2baf-4969-924d-06330bcf2082 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:22:08.223732 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-65de88ed-2baf-4969-924d-06330bcf2082 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:22:08.386868 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-65de88ed-2baf-4969-924d-06330bcf2082 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4b8c7ad3cc6240c096c2c7cf31d8f74f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a45a8ce6687c427d8f76e4205a076a93', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:22:08.394808 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-65de88ed-2baf-4969-924d-06330bcf2082 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "GET /compute/v2.1/servers/373f324d-27e7-42ba-90b0-6ed693d9cfec" status: 200 len: 1392 microversion: 2.1 time: 0.185032 Jul 16 22:22:08.395427 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3542/7080] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:22:08 2022] GET /compute/v2.1/servers/373f324d-27e7-42ba-90b0-6ed693d9cfec => generated 1392 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:22:09.210175 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-9c55c9b0-b7bf-42d8-b7fd-b86b03bd11c8 req-efac5874-f3e7-4479-b936-db9856e44a70 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "373f324d-27e7-42ba-90b0-6ed693d9cfec", "tag": "bf1e8fb7-4d69-4b1c-b183-64e36384e222"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:22:09.220258 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-9c55c9b0-b7bf-42d8-b7fd-b86b03bd11c8 req-efac5874-f3e7-4479-b936-db9856e44a70 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:22:09.220710 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-9c55c9b0-b7bf-42d8-b7fd-b86b03bd11c8 req-efac5874-f3e7-4479-b936-db9856e44a70 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:22:09.239364 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-9c55c9b0-b7bf-42d8-b7fd-b86b03bd11c8 req-efac5874-f3e7-4479-b936-db9856e44a70 service nova] Creating event network-changed:bf1e8fb7-4d69-4b1c-b183-64e36384e222 for instance 373f324d-27e7-42ba-90b0-6ed693d9cfec on nch1-master-1 Jul 16 22:22:09.245998 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-9c55c9b0-b7bf-42d8-b7fd-b86b03bd11c8 req-efac5874-f3e7-4479-b936-db9856e44a70 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.040563 Jul 16 22:22:09.246540 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3539/7081] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 22:22:09 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 16 22:22:09.413828 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-40133f30-30f0-49d6-8d02-c7d029f5d403 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:22:09.415830 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-40133f30-30f0-49d6-8d02-c7d029f5d403 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] [instance: 373f324d-27e7-42ba-90b0-6ed693d9cfec] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:22:09.422861 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-40133f30-30f0-49d6-8d02-c7d029f5d403 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:22:09.423525 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-40133f30-30f0-49d6-8d02-c7d029f5d403 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:22:09.424245 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-40133f30-30f0-49d6-8d02-c7d029f5d403 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:22:09.424741 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-40133f30-30f0-49d6-8d02-c7d029f5d403 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:22:09.854164 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-40133f30-30f0-49d6-8d02-c7d029f5d403 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4b8c7ad3cc6240c096c2c7cf31d8f74f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a45a8ce6687c427d8f76e4205a076a93', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:22:09.861895 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-40133f30-30f0-49d6-8d02-c7d029f5d403 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "GET /compute/v2.1/servers/373f324d-27e7-42ba-90b0-6ed693d9cfec" status: 200 len: 1392 microversion: 2.1 time: 0.451972 Jul 16 22:22:09.862490 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3543/7082] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:22:09 2022] GET /compute/v2.1/servers/373f324d-27e7-42ba-90b0-6ed693d9cfec => generated 1392 bytes in 453 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:22:10.881230 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-55482184-f939-414b-bc19-6a34664a0efc tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:22:10.881964 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-55482184-f939-414b-bc19-6a34664a0efc tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] [instance: 373f324d-27e7-42ba-90b0-6ed693d9cfec] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:22:10.884606 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_api:SELECT=30,nova_api:INSERT=1 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:22:10.889770 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-55482184-f939-414b-bc19-6a34664a0efc tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:22:10.890259 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-55482184-f939-414b-bc19-6a34664a0efc tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:22:10.891067 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-55482184-f939-414b-bc19-6a34664a0efc tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:22:10.891561 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-55482184-f939-414b-bc19-6a34664a0efc tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:22:11.029437 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-55482184-f939-414b-bc19-6a34664a0efc tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4b8c7ad3cc6240c096c2c7cf31d8f74f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a45a8ce6687c427d8f76e4205a076a93', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:22:11.037117 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-55482184-f939-414b-bc19-6a34664a0efc tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "GET /compute/v2.1/servers/373f324d-27e7-42ba-90b0-6ed693d9cfec" status: 200 len: 1392 microversion: 2.1 time: 0.161007 Jul 16 22:22:11.037693 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3540/7083] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:22:10 2022] GET /compute/v2.1/servers/373f324d-27e7-42ba-90b0-6ed693d9cfec => generated 1392 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:22:12.054414 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-bb7e04b0-4346-48d5-a04d-2a22b6ba10df tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:22:12.055078 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-bb7e04b0-4346-48d5-a04d-2a22b6ba10df tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] [instance: 373f324d-27e7-42ba-90b0-6ed693d9cfec] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:22:12.062295 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-bb7e04b0-4346-48d5-a04d-2a22b6ba10df tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:22:12.062764 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-bb7e04b0-4346-48d5-a04d-2a22b6ba10df tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:22:12.063572 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-bb7e04b0-4346-48d5-a04d-2a22b6ba10df tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:22:12.064028 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-bb7e04b0-4346-48d5-a04d-2a22b6ba10df tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:22:12.204850 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-bb7e04b0-4346-48d5-a04d-2a22b6ba10df tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4b8c7ad3cc6240c096c2c7cf31d8f74f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a45a8ce6687c427d8f76e4205a076a93', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:22:12.213475 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-bb7e04b0-4346-48d5-a04d-2a22b6ba10df tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "GET /compute/v2.1/servers/373f324d-27e7-42ba-90b0-6ed693d9cfec" status: 200 len: 1392 microversion: 2.1 time: 0.163296 Jul 16 22:22:12.213987 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3544/7084] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:22:12 2022] GET /compute/v2.1/servers/373f324d-27e7-42ba-90b0-6ed693d9cfec => generated 1392 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:22:13.231130 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-adde8f1b-7900-491d-8748-336bc5cd2ba7 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:22:13.231752 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-adde8f1b-7900-491d-8748-336bc5cd2ba7 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] [instance: 373f324d-27e7-42ba-90b0-6ed693d9cfec] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:22:13.238766 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-adde8f1b-7900-491d-8748-336bc5cd2ba7 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:22:13.239268 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-adde8f1b-7900-491d-8748-336bc5cd2ba7 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:22:13.240046 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-adde8f1b-7900-491d-8748-336bc5cd2ba7 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:22:13.240536 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-adde8f1b-7900-491d-8748-336bc5cd2ba7 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:22:13.388122 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-adde8f1b-7900-491d-8748-336bc5cd2ba7 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4b8c7ad3cc6240c096c2c7cf31d8f74f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a45a8ce6687c427d8f76e4205a076a93', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:22:13.396214 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-adde8f1b-7900-491d-8748-336bc5cd2ba7 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "GET /compute/v2.1/servers/373f324d-27e7-42ba-90b0-6ed693d9cfec" status: 200 len: 1392 microversion: 2.1 time: 0.169839 Jul 16 22:22:13.399611 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3541/7085] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:22:13 2022] GET /compute/v2.1/servers/373f324d-27e7-42ba-90b0-6ed693d9cfec => generated 1392 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:22:14.415249 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-f621885b-1803-4998-9dec-7013745e0747 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:22:14.415983 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-f621885b-1803-4998-9dec-7013745e0747 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] [instance: 373f324d-27e7-42ba-90b0-6ed693d9cfec] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:22:14.423115 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f621885b-1803-4998-9dec-7013745e0747 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:22:14.423587 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f621885b-1803-4998-9dec-7013745e0747 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:22:14.424598 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f621885b-1803-4998-9dec-7013745e0747 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:22:14.424975 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f621885b-1803-4998-9dec-7013745e0747 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:22:14.566541 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-f621885b-1803-4998-9dec-7013745e0747 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4b8c7ad3cc6240c096c2c7cf31d8f74f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a45a8ce6687c427d8f76e4205a076a93', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:22:14.574463 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-f621885b-1803-4998-9dec-7013745e0747 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "GET /compute/v2.1/servers/373f324d-27e7-42ba-90b0-6ed693d9cfec" status: 200 len: 1392 microversion: 2.1 time: 0.164145 Jul 16 22:22:14.575139 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3545/7086] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:22:14 2022] GET /compute/v2.1/servers/373f324d-27e7-42ba-90b0-6ed693d9cfec => generated 1392 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:22:15.321331 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-0c23cda6-3594-4ca5-b8ba-9e25e9d43ed9 req-f600e4a8-8664-4b08-ad44-04f544541ad0 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "373f324d-27e7-42ba-90b0-6ed693d9cfec", "name": "network-vif-plugged", "status": "completed", "tag": "bf1e8fb7-4d69-4b1c-b183-64e36384e222"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:22:15.325996 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell0:SELECT=6 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:22:15.331484 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-0c23cda6-3594-4ca5-b8ba-9e25e9d43ed9 req-f600e4a8-8664-4b08-ad44-04f544541ad0 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:22:15.331938 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-0c23cda6-3594-4ca5-b8ba-9e25e9d43ed9 req-f600e4a8-8664-4b08-ad44-04f544541ad0 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:22:15.343856 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-0c23cda6-3594-4ca5-b8ba-9e25e9d43ed9 req-f600e4a8-8664-4b08-ad44-04f544541ad0 service nova] Creating event network-vif-plugged:bf1e8fb7-4d69-4b1c-b183-64e36384e222 for instance 373f324d-27e7-42ba-90b0-6ed693d9cfec on nch1-master-1 Jul 16 22:22:15.349158 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-0c23cda6-3594-4ca5-b8ba-9e25e9d43ed9 req-f600e4a8-8664-4b08-ad44-04f544541ad0 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.032326 Jul 16 22:22:15.349767 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3542/7087] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 22:22:15 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 22:22:15.591690 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-b7a704c8-ac71-4caf-9a84-e7e49b80caa7 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:22:15.592491 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-b7a704c8-ac71-4caf-9a84-e7e49b80caa7 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] [instance: 373f324d-27e7-42ba-90b0-6ed693d9cfec] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:22:15.594353 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell0:SELECT=2 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:22:15.595081 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell0:SELECT=6 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:22:15.600841 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b7a704c8-ac71-4caf-9a84-e7e49b80caa7 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:22:15.601304 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b7a704c8-ac71-4caf-9a84-e7e49b80caa7 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:22:15.602444 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b7a704c8-ac71-4caf-9a84-e7e49b80caa7 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:22:15.602983 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b7a704c8-ac71-4caf-9a84-e7e49b80caa7 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:22:15.740304 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-b7a704c8-ac71-4caf-9a84-e7e49b80caa7 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4b8c7ad3cc6240c096c2c7cf31d8f74f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a45a8ce6687c427d8f76e4205a076a93', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:22:15.749138 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-b7a704c8-ac71-4caf-9a84-e7e49b80caa7 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "GET /compute/v2.1/servers/373f324d-27e7-42ba-90b0-6ed693d9cfec" status: 200 len: 1392 microversion: 2.1 time: 0.161448 Jul 16 22:22:15.749725 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3546/7088] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:22:15 2022] GET /compute/v2.1/servers/373f324d-27e7-42ba-90b0-6ed693d9cfec => generated 1392 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:22:16.766722 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-004f981a-7760-4675-84db-c10392ff55a9 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:22:16.767473 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-004f981a-7760-4675-84db-c10392ff55a9 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] [instance: 373f324d-27e7-42ba-90b0-6ed693d9cfec] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:22:16.775385 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-004f981a-7760-4675-84db-c10392ff55a9 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:22:16.775847 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-004f981a-7760-4675-84db-c10392ff55a9 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:22:16.776621 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-004f981a-7760-4675-84db-c10392ff55a9 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:22:16.777070 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-004f981a-7760-4675-84db-c10392ff55a9 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:22:16.913000 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-004f981a-7760-4675-84db-c10392ff55a9 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4b8c7ad3cc6240c096c2c7cf31d8f74f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a45a8ce6687c427d8f76e4205a076a93', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:22:16.921351 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-004f981a-7760-4675-84db-c10392ff55a9 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "GET /compute/v2.1/servers/373f324d-27e7-42ba-90b0-6ed693d9cfec" status: 200 len: 1563 microversion: 2.1 time: 0.159380 Jul 16 22:22:16.921924 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3543/7089] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:22:16 2022] GET /compute/v2.1/servers/373f324d-27e7-42ba-90b0-6ed693d9cfec => generated 1563 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:22:16.938245 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-6b174a28-3181-431e-bf76-cf01f0748ec6 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:22:16.938881 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-6b174a28-3181-431e-bf76-cf01f0748ec6 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] [instance: 373f324d-27e7-42ba-90b0-6ed693d9cfec] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:22:16.946641 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6b174a28-3181-431e-bf76-cf01f0748ec6 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:22:16.947140 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6b174a28-3181-431e-bf76-cf01f0748ec6 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:22:16.947938 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6b174a28-3181-431e-bf76-cf01f0748ec6 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:22:16.948398 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6b174a28-3181-431e-bf76-cf01f0748ec6 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:22:17.079550 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-6b174a28-3181-431e-bf76-cf01f0748ec6 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4b8c7ad3cc6240c096c2c7cf31d8f74f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a45a8ce6687c427d8f76e4205a076a93', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:22:17.087701 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-6b174a28-3181-431e-bf76-cf01f0748ec6 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "GET /compute/v2.1/servers/373f324d-27e7-42ba-90b0-6ed693d9cfec" status: 200 len: 1563 microversion: 2.1 time: 0.153524 Jul 16 22:22:17.088281 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3547/7090] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:22:16 2022] GET /compute/v2.1/servers/373f324d-27e7-42ba-90b0-6ed693d9cfec => generated 1563 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:22:19.037802 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-7fb06a91-3b8c-45ab-88d3-8ad12cc13fbb tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:22:19.038411 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-7fb06a91-3b8c-45ab-88d3-8ad12cc13fbb tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] [instance: 373f324d-27e7-42ba-90b0-6ed693d9cfec] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:22:19.041758 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-ec40f914-2ded-4b68-b718-7a030fc703e4 req-63525d71-8b7e-42f4-9f4a-54e4d95e465e service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "373f324d-27e7-42ba-90b0-6ed693d9cfec", "tag": "bf1e8fb7-4d69-4b1c-b183-64e36384e222"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:22:19.045710 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_api:SELECT=46,nova_api:UPDATE=1,nova_api:DELETE=1,nova_api:INSERT=3 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:22:19.054471 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-ec40f914-2ded-4b68-b718-7a030fc703e4 req-63525d71-8b7e-42f4-9f4a-54e4d95e465e service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:22:19.054836 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7fb06a91-3b8c-45ab-88d3-8ad12cc13fbb tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:22:19.054996 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-ec40f914-2ded-4b68-b718-7a030fc703e4 req-63525d71-8b7e-42f4-9f4a-54e4d95e465e service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:22:19.056225 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7fb06a91-3b8c-45ab-88d3-8ad12cc13fbb tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:22:19.057155 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7fb06a91-3b8c-45ab-88d3-8ad12cc13fbb tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:22:19.057705 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7fb06a91-3b8c-45ab-88d3-8ad12cc13fbb tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:22:19.068151 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-ec40f914-2ded-4b68-b718-7a030fc703e4 req-63525d71-8b7e-42f4-9f4a-54e4d95e465e service nova] Creating event network-changed:bf1e8fb7-4d69-4b1c-b183-64e36384e222 for instance 373f324d-27e7-42ba-90b0-6ed693d9cfec on nch1-master-1 Jul 16 22:22:19.075518 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-ec40f914-2ded-4b68-b718-7a030fc703e4 req-63525d71-8b7e-42f4-9f4a-54e4d95e465e service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.042279 Jul 16 22:22:19.076564 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3548/7091] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 22:22:19 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 22:22:19.204192 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-7fb06a91-3b8c-45ab-88d3-8ad12cc13fbb tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4b8c7ad3cc6240c096c2c7cf31d8f74f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a45a8ce6687c427d8f76e4205a076a93', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:22:19.212151 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-7fb06a91-3b8c-45ab-88d3-8ad12cc13fbb tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "GET /compute/v2.1/servers/373f324d-27e7-42ba-90b0-6ed693d9cfec" status: 200 len: 1563 microversion: 2.1 time: 0.181795 Jul 16 22:22:19.212784 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3544/7092] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:22:19 2022] GET /compute/v2.1/servers/373f324d-27e7-42ba-90b0-6ed693d9cfec => generated 1563 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:22:42.463355 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=46 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:22:42.468102 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-13880a44-8d28-4f1b-9daa-589694adc1da tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Action: 'action', calling method: >, body: {"os-getConsoleOutput": {}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:22:42.469133 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-13880a44-8d28-4f1b-9daa-589694adc1da tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] [instance: 373f324d-27e7-42ba-90b0-6ed693d9cfec] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:22:42.475633 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-13880a44-8d28-4f1b-9daa-589694adc1da tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:22:42.476106 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-13880a44-8d28-4f1b-9daa-589694adc1da tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:22:42.476887 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-13880a44-8d28-4f1b-9daa-589694adc1da tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:22:42.477345 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-13880a44-8d28-4f1b-9daa-589694adc1da tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:22:42.530059 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-13880a44-8d28-4f1b-9daa-589694adc1da tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "POST /compute/v2.1/servers/373f324d-27e7-42ba-90b0-6ed693d9cfec/action" status: 200 len: 24568 microversion: 2.1 time: 0.068398 Jul 16 22:22:42.530708 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3549/7093] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 22:22:42 2022] POST /compute/v2.1/servers/373f324d-27e7-42ba-90b0-6ed693d9cfec/action => generated 24568 bytes in 70 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 16 22:22:48.923831 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-b64545b0-2452-4f78-a042-f88322f82d88 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Action: 'action', calling method: >, body: {"suspend": {}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:22:48.924391 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-b64545b0-2452-4f78-a042-f88322f82d88 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] [instance: 373f324d-27e7-42ba-90b0-6ed693d9cfec] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:22:48.926485 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_api:SELECT=9 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:22:48.927178 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=34 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:22:48.932223 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b64545b0-2452-4f78-a042-f88322f82d88 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:22:48.932680 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b64545b0-2452-4f78-a042-f88322f82d88 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:22:48.933440 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b64545b0-2452-4f78-a042-f88322f82d88 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:22:48.933886 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-b64545b0-2452-4f78-a042-f88322f82d88 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:22:48.959105 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.objects.instance [None req-b64545b0-2452-4f78-a042-f88322f82d88 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lazy-loading 'flavor' on Instance uuid 373f324d-27e7-42ba-90b0-6ed693d9cfec {{(pid=138680) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 22:22:49.198587 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-b64545b0-2452-4f78-a042-f88322f82d88 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "POST /compute/v2.1/servers/373f324d-27e7-42ba-90b0-6ed693d9cfec/action" status: 202 len: 0 microversion: 2.1 time: 0.278987 Jul 16 22:22:49.199203 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3545/7094] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 22:22:48 2022] POST /compute/v2.1/servers/373f324d-27e7-42ba-90b0-6ed693d9cfec/action => generated 0 bytes in 281 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 22:22:49.208962 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-814c52bc-2ced-4e31-bf1a-34101af720ec tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:22:49.209765 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-814c52bc-2ced-4e31-bf1a-34101af720ec tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] [instance: 373f324d-27e7-42ba-90b0-6ed693d9cfec] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:22:49.217539 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-814c52bc-2ced-4e31-bf1a-34101af720ec tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:22:49.221665 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-814c52bc-2ced-4e31-bf1a-34101af720ec tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:22:49.221665 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-814c52bc-2ced-4e31-bf1a-34101af720ec tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:22:49.221665 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-814c52bc-2ced-4e31-bf1a-34101af720ec tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:22:49.353872 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-814c52bc-2ced-4e31-bf1a-34101af720ec tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4b8c7ad3cc6240c096c2c7cf31d8f74f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a45a8ce6687c427d8f76e4205a076a93', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:22:49.362398 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-814c52bc-2ced-4e31-bf1a-34101af720ec tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "GET /compute/v2.1/servers/373f324d-27e7-42ba-90b0-6ed693d9cfec" status: 200 len: 1690 microversion: 2.1 time: 0.158120 Jul 16 22:22:49.362981 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3550/7095] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:22:49 2022] GET /compute/v2.1/servers/373f324d-27e7-42ba-90b0-6ed693d9cfec => generated 1690 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:22:50.382756 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-50d081df-6f33-4c1c-b21f-2ed7c335e2a7 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:22:50.383546 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-50d081df-6f33-4c1c-b21f-2ed7c335e2a7 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] [instance: 373f324d-27e7-42ba-90b0-6ed693d9cfec] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:22:50.390021 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-50d081df-6f33-4c1c-b21f-2ed7c335e2a7 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:22:50.390491 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-50d081df-6f33-4c1c-b21f-2ed7c335e2a7 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:22:50.391300 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-50d081df-6f33-4c1c-b21f-2ed7c335e2a7 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:22:50.391798 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-50d081df-6f33-4c1c-b21f-2ed7c335e2a7 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:22:50.525967 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-50d081df-6f33-4c1c-b21f-2ed7c335e2a7 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4b8c7ad3cc6240c096c2c7cf31d8f74f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a45a8ce6687c427d8f76e4205a076a93', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:22:50.534120 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-50d081df-6f33-4c1c-b21f-2ed7c335e2a7 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "GET /compute/v2.1/servers/373f324d-27e7-42ba-90b0-6ed693d9cfec" status: 200 len: 1673 microversion: 2.1 time: 0.156826 Jul 16 22:22:50.534693 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3546/7096] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:22:50 2022] GET /compute/v2.1/servers/373f324d-27e7-42ba-90b0-6ed693d9cfec => generated 1673 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:22:51.625185 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-83d1d382-caab-4c86-a699-9c503923a287 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Action: 'action', calling method: >, body: {"os-getConsoleOutput": {}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:22:51.626146 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-83d1d382-caab-4c86-a699-9c503923a287 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] [instance: 373f324d-27e7-42ba-90b0-6ed693d9cfec] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:22:51.633503 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-83d1d382-caab-4c86-a699-9c503923a287 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:22:51.633979 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-83d1d382-caab-4c86-a699-9c503923a287 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:22:51.634767 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-83d1d382-caab-4c86-a699-9c503923a287 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:22:51.635257 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-83d1d382-caab-4c86-a699-9c503923a287 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:22:51.686373 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-83d1d382-caab-4c86-a699-9c503923a287 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "POST /compute/v2.1/servers/373f324d-27e7-42ba-90b0-6ed693d9cfec/action" status: 200 len: 35687 microversion: 2.1 time: 0.064996 Jul 16 22:22:51.687023 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3551/7097] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 22:22:51 2022] POST /compute/v2.1/servers/373f324d-27e7-42ba-90b0-6ed693d9cfec/action => generated 35687 bytes in 67 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 16 22:22:51.698245 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-7712fb6c-4a73-4d04-903f-82b6b36f1f63 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Action: 'action', calling method: >, body: {"resume": {}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:22:51.698787 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-7712fb6c-4a73-4d04-903f-82b6b36f1f63 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] [instance: 373f324d-27e7-42ba-90b0-6ed693d9cfec] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:22:51.706196 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7712fb6c-4a73-4d04-903f-82b6b36f1f63 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:22:51.706654 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7712fb6c-4a73-4d04-903f-82b6b36f1f63 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:22:51.707447 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7712fb6c-4a73-4d04-903f-82b6b36f1f63 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:22:51.707893 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7712fb6c-4a73-4d04-903f-82b6b36f1f63 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:22:51.929834 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-7712fb6c-4a73-4d04-903f-82b6b36f1f63 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "POST /compute/v2.1/servers/373f324d-27e7-42ba-90b0-6ed693d9cfec/action" status: 202 len: 0 microversion: 2.1 time: 0.235170 Jul 16 22:22:51.930468 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3547/7098] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 22:22:51 2022] POST /compute/v2.1/servers/373f324d-27e7-42ba-90b0-6ed693d9cfec/action => generated 0 bytes in 237 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 22:22:51.940960 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-6849448d-4b4b-4027-ab18-32b961c792ef tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:22:51.941728 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-6849448d-4b4b-4027-ab18-32b961c792ef tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] [instance: 373f324d-27e7-42ba-90b0-6ed693d9cfec] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:22:51.949103 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6849448d-4b4b-4027-ab18-32b961c792ef tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:22:51.950672 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6849448d-4b4b-4027-ab18-32b961c792ef tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:22:51.951324 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6849448d-4b4b-4027-ab18-32b961c792ef tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:22:51.951797 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6849448d-4b4b-4027-ab18-32b961c792ef tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:22:52.092688 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-6849448d-4b4b-4027-ab18-32b961c792ef tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4b8c7ad3cc6240c096c2c7cf31d8f74f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a45a8ce6687c427d8f76e4205a076a93', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:22:52.100629 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-6849448d-4b4b-4027-ab18-32b961c792ef tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "GET /compute/v2.1/servers/373f324d-27e7-42ba-90b0-6ed693d9cfec" status: 200 len: 1679 microversion: 2.1 time: 0.164392 Jul 16 22:22:52.101599 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3552/7099] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:22:51 2022] GET /compute/v2.1/servers/373f324d-27e7-42ba-90b0-6ed693d9cfec => generated 1679 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:22:53.120288 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-fabdcd77-c8e5-48a4-a61e-fe144d217817 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:22:53.120921 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-fabdcd77-c8e5-48a4-a61e-fe144d217817 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] [instance: 373f324d-27e7-42ba-90b0-6ed693d9cfec] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:22:53.128320 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-fabdcd77-c8e5-48a4-a61e-fe144d217817 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:22:53.128753 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-fabdcd77-c8e5-48a4-a61e-fe144d217817 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:22:53.129575 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-fabdcd77-c8e5-48a4-a61e-fe144d217817 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:22:53.130018 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-fabdcd77-c8e5-48a4-a61e-fe144d217817 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:22:53.267761 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-fabdcd77-c8e5-48a4-a61e-fe144d217817 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4b8c7ad3cc6240c096c2c7cf31d8f74f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a45a8ce6687c427d8f76e4205a076a93', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:22:53.275390 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-fabdcd77-c8e5-48a4-a61e-fe144d217817 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "GET /compute/v2.1/servers/373f324d-27e7-42ba-90b0-6ed693d9cfec" status: 200 len: 1679 microversion: 2.1 time: 0.160555 Jul 16 22:22:53.275959 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3548/7100] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:22:53 2022] GET /compute/v2.1/servers/373f324d-27e7-42ba-90b0-6ed693d9cfec => generated 1679 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:22:54.293311 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-e388eecc-0028-4629-bf95-baf826baf635 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:22:54.293883 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-e388eecc-0028-4629-bf95-baf826baf635 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] [instance: 373f324d-27e7-42ba-90b0-6ed693d9cfec] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:22:54.304707 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e388eecc-0028-4629-bf95-baf826baf635 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:22:54.305177 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e388eecc-0028-4629-bf95-baf826baf635 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:22:54.305963 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e388eecc-0028-4629-bf95-baf826baf635 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:22:54.306422 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e388eecc-0028-4629-bf95-baf826baf635 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:22:54.437927 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-e388eecc-0028-4629-bf95-baf826baf635 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4b8c7ad3cc6240c096c2c7cf31d8f74f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a45a8ce6687c427d8f76e4205a076a93', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:22:54.445779 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-e388eecc-0028-4629-bf95-baf826baf635 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "GET /compute/v2.1/servers/373f324d-27e7-42ba-90b0-6ed693d9cfec" status: 200 len: 1682 microversion: 2.1 time: 0.156537 Jul 16 22:22:54.446448 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3553/7101] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:22:54 2022] GET /compute/v2.1/servers/373f324d-27e7-42ba-90b0-6ed693d9cfec => generated 1682 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:22:54.539829 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-1a9df72f-93a3-4d99-89d7-e002d955d37e tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Action: 'action', calling method: >, body: {"os-getConsoleOutput": {}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:22:54.540798 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-1a9df72f-93a3-4d99-89d7-e002d955d37e tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] [instance: 373f324d-27e7-42ba-90b0-6ed693d9cfec] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:22:54.548938 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1a9df72f-93a3-4d99-89d7-e002d955d37e tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:22:54.549332 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1a9df72f-93a3-4d99-89d7-e002d955d37e tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:22:54.550099 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1a9df72f-93a3-4d99-89d7-e002d955d37e tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:22:54.550540 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1a9df72f-93a3-4d99-89d7-e002d955d37e tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:22:54.592525 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-1a9df72f-93a3-4d99-89d7-e002d955d37e tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "POST /compute/v2.1/servers/373f324d-27e7-42ba-90b0-6ed693d9cfec/action" status: 200 len: 35687 microversion: 2.1 time: 0.056413 Jul 16 22:22:54.593429 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3549/7102] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 22:22:54 2022] POST /compute/v2.1/servers/373f324d-27e7-42ba-90b0-6ed693d9cfec/action => generated 35687 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 16 22:22:55.678182 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-7610ce94-7571-4c8f-ae69-a74efd52d6ca tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:22:55.678675 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-7610ce94-7571-4c8f-ae69-a74efd52d6ca tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] [instance: 373f324d-27e7-42ba-90b0-6ed693d9cfec] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:22:55.684453 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-a8f8f87f-16c5-4875-9715-635f183cd4f3 req-16442c2e-203f-4464-9e31-ac7baf668e00 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "373f324d-27e7-42ba-90b0-6ed693d9cfec", "tag": "bf1e8fb7-4d69-4b1c-b183-64e36384e222"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:22:55.690976 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7610ce94-7571-4c8f-ae69-a74efd52d6ca tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:22:55.696657 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7610ce94-7571-4c8f-ae69-a74efd52d6ca tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:22:55.696657 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7610ce94-7571-4c8f-ae69-a74efd52d6ca tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:22:55.696657 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7610ce94-7571-4c8f-ae69-a74efd52d6ca tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:22:55.697103 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-a8f8f87f-16c5-4875-9715-635f183cd4f3 req-16442c2e-203f-4464-9e31-ac7baf668e00 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:22:55.697103 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-a8f8f87f-16c5-4875-9715-635f183cd4f3 req-16442c2e-203f-4464-9e31-ac7baf668e00 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:22:55.708239 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-a8f8f87f-16c5-4875-9715-635f183cd4f3 req-16442c2e-203f-4464-9e31-ac7baf668e00 service nova] Creating event network-changed:bf1e8fb7-4d69-4b1c-b183-64e36384e222 for instance 373f324d-27e7-42ba-90b0-6ed693d9cfec on nch1-master-1 Jul 16 22:22:55.713412 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-a8f8f87f-16c5-4875-9715-635f183cd4f3 req-16442c2e-203f-4464-9e31-ac7baf668e00 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.032906 Jul 16 22:22:55.714442 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3550/7103] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 22:22:55 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 16 22:22:55.723068 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-7610ce94-7571-4c8f-ae69-a74efd52d6ca tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] [instance: 373f324d-27e7-42ba-90b0-6ed693d9cfec] Going to try to terminate instance {{(pid=138681) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 16 22:22:56.050621 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-7610ce94-7571-4c8f-ae69-a74efd52d6ca tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "DELETE /compute/v2.1/servers/373f324d-27e7-42ba-90b0-6ed693d9cfec" status: 204 len: 0 microversion: 2.1 time: 0.376377 Jul 16 22:22:56.053380 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3554/7104] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 22:22:55 2022] DELETE /compute/v2.1/servers/373f324d-27e7-42ba-90b0-6ed693d9cfec => generated 0 bytes in 378 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 16 22:22:56.060935 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-83007484-c1ef-4c17-b711-af8003781890 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:22:56.061875 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-83007484-c1ef-4c17-b711-af8003781890 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] [instance: 373f324d-27e7-42ba-90b0-6ed693d9cfec] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:22:56.069616 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-83007484-c1ef-4c17-b711-af8003781890 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:22:56.070077 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-83007484-c1ef-4c17-b711-af8003781890 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:22:56.070882 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-83007484-c1ef-4c17-b711-af8003781890 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:22:56.071338 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-83007484-c1ef-4c17-b711-af8003781890 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:22:56.214871 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-83007484-c1ef-4c17-b711-af8003781890 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4b8c7ad3cc6240c096c2c7cf31d8f74f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a45a8ce6687c427d8f76e4205a076a93', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:22:56.223778 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-83007484-c1ef-4c17-b711-af8003781890 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "GET /compute/v2.1/servers/373f324d-27e7-42ba-90b0-6ed693d9cfec" status: 200 len: 1569 microversion: 2.1 time: 0.167457 Jul 16 22:22:56.228821 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3551/7105] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:22:56 2022] GET /compute/v2.1/servers/373f324d-27e7-42ba-90b0-6ed693d9cfec => generated 1569 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:22:57.243786 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-819e1515-04a5-4cdc-8a88-7ec104ebd958 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:22:57.244421 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-819e1515-04a5-4cdc-8a88-7ec104ebd958 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] [instance: 373f324d-27e7-42ba-90b0-6ed693d9cfec] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:22:57.251407 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-819e1515-04a5-4cdc-8a88-7ec104ebd958 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:22:57.251874 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-819e1515-04a5-4cdc-8a88-7ec104ebd958 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:22:57.252643 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-819e1515-04a5-4cdc-8a88-7ec104ebd958 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:22:57.253093 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-819e1515-04a5-4cdc-8a88-7ec104ebd958 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:22:57.387574 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-819e1515-04a5-4cdc-8a88-7ec104ebd958 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4b8c7ad3cc6240c096c2c7cf31d8f74f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a45a8ce6687c427d8f76e4205a076a93', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:22:57.395348 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-819e1515-04a5-4cdc-8a88-7ec104ebd958 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "GET /compute/v2.1/servers/373f324d-27e7-42ba-90b0-6ed693d9cfec" status: 200 len: 1569 microversion: 2.1 time: 0.155935 Jul 16 22:22:57.396025 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3555/7106] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:22:57 2022] GET /compute/v2.1/servers/373f324d-27e7-42ba-90b0-6ed693d9cfec => generated 1569 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:22:58.334840 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-69d6dd71-d70e-49df-9c41-bba9e5a657f6 req-0a5fdef4-04a4-4b9d-a7cc-c56cf1f4a54b service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "373f324d-27e7-42ba-90b0-6ed693d9cfec", "name": "network-vif-deleted", "tag": "bf1e8fb7-4d69-4b1c-b183-64e36384e222"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:22:58.348281 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-69d6dd71-d70e-49df-9c41-bba9e5a657f6 req-0a5fdef4-04a4-4b9d-a7cc-c56cf1f4a54b service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:22:58.348741 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-69d6dd71-d70e-49df-9c41-bba9e5a657f6 req-0a5fdef4-04a4-4b9d-a7cc-c56cf1f4a54b service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:22:58.359066 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-69d6dd71-d70e-49df-9c41-bba9e5a657f6 req-0a5fdef4-04a4-4b9d-a7cc-c56cf1f4a54b service nova] Creating event network-vif-deleted:bf1e8fb7-4d69-4b1c-b183-64e36384e222 for instance 373f324d-27e7-42ba-90b0-6ed693d9cfec on nch1-master-1 Jul 16 22:22:58.363953 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-69d6dd71-d70e-49df-9c41-bba9e5a657f6 req-0a5fdef4-04a4-4b9d-a7cc-c56cf1f4a54b service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.032868 Jul 16 22:22:58.367451 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3552/7107] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 22:22:58 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 22:22:58.413590 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-f7533fbd-c236-4ed7-9fe8-3c4d959b9580 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:22:58.414217 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-f7533fbd-c236-4ed7-9fe8-3c4d959b9580 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] [instance: 373f324d-27e7-42ba-90b0-6ed693d9cfec] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:22:58.421522 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f7533fbd-c236-4ed7-9fe8-3c4d959b9580 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:22:58.422109 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f7533fbd-c236-4ed7-9fe8-3c4d959b9580 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:22:58.422895 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f7533fbd-c236-4ed7-9fe8-3c4d959b9580 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:22:58.423377 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f7533fbd-c236-4ed7-9fe8-3c4d959b9580 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:22:58.488940 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-f7533fbd-c236-4ed7-9fe8-3c4d959b9580 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4b8c7ad3cc6240c096c2c7cf31d8f74f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a45a8ce6687c427d8f76e4205a076a93', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:22:58.496860 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-f7533fbd-c236-4ed7-9fe8-3c4d959b9580 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "GET /compute/v2.1/servers/373f324d-27e7-42ba-90b0-6ed693d9cfec" status: 200 len: 1501 microversion: 2.1 time: 0.087275 Jul 16 22:22:58.500618 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3556/7108] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:22:58 2022] GET /compute/v2.1/servers/373f324d-27e7-42ba-90b0-6ed693d9cfec => generated 1501 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:22:59.516879 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-239cb95d-5185-4fdb-bdd6-9c1ed86a127d tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:22:59.517589 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-239cb95d-5185-4fdb-bdd6-9c1ed86a127d tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] [instance: 373f324d-27e7-42ba-90b0-6ed693d9cfec] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:22:59.523870 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-239cb95d-5185-4fdb-bdd6-9c1ed86a127d tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:22:59.524328 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-239cb95d-5185-4fdb-bdd6-9c1ed86a127d tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:22:59.525098 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-239cb95d-5185-4fdb-bdd6-9c1ed86a127d tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:22:59.525547 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-239cb95d-5185-4fdb-bdd6-9c1ed86a127d tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:22:59.540217 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-239cb95d-5185-4fdb-bdd6-9c1ed86a127d tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] HTTP exception thrown: Instance 373f324d-27e7-42ba-90b0-6ed693d9cfec could not be found. Jul 16 22:22:59.541840 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-239cb95d-5185-4fdb-bdd6-9c1ed86a127d tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Request method failure captured: Jul 16 22:22:59.541840 ncdv-master-1 devstack@n-api.service[138680]: request: GET /compute/v2.1/servers/373f324d-27e7-42ba-90b0-6ed693d9cfec HTTP/1.1 Jul 16 22:22:59.541840 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 22:22:59.541840 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 22:22:59.541840 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 22:22:59.541840 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 0 Jul 16 22:22:59.541840 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 22:22:59.541840 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 22:22:59.541840 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 22:22:59.541840 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 22:22:59.541840 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0zhtDg4n0Cr97EpGJw_kphX8HozEdWitmpiSsnN8liwN7JwqYkEqcaIM-47rjt-yK-e5LgZo6pFGCSkRgNS-xnBjvFiClGjWelkYApUjhWHqw4F9MI58oj4QDcEbMKfIlL82tbwRwh1LdKROeHuRqnWXe9oskU-rprKG2IX2dbA4fe4 Jul 16 22:22:59.541840 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 22:22:59.541840 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 22:22:59.541840 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 22:22:59.541840 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 22:22:59.541840 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 22:22:59.541840 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 22:22:59.541840 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: a45a8ce6687c427d8f76e4205a076a93 Jul 16 22:22:59.541840 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-TestNetworkAdvancedServerOps-234580283 Jul 16 22:22:59.541840 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 22:22:59.541840 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 22:22:59.541840 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/a45a8ce6687c427d8f76e4205a076a93"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/a45a8ce6687c427d8f76e4205a076a93"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_a45a8ce6687c427d8f76e4205a076a93"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/a45a8ce6687c427d8f76e4205a076a93"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 22:22:59.541840 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-TestNetworkAdvancedServerOps-234580283 Jul 16 22:22:59.541840 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: a45a8ce6687c427d8f76e4205a076a93 Jul 16 22:22:59.541840 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-TestNetworkAdvancedServerOps-234580283 Jul 16 22:22:59.541840 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-TestNetworkAdvancedServerOps-234580283-project Jul 16 22:22:59.541840 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 22:22:59.541840 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 22:22:59.541840 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: 4b8c7ad3cc6240c096c2c7cf31d8f74f Jul 16 22:22:59.541840 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-TestNetworkAdvancedServerOps-234580283-project Jul 16 22:22:59.541840 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 22:22:59.541840 ncdv-master-1 devstack@n-api.service[138680]: exception: Instance 373f324d-27e7-42ba-90b0-6ed693d9cfec could not be found. Jul 16 22:22:59.541840 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': '373f324d-27e7-42ba-90b0-6ed693d9cfec'} Jul 16 22:22:59.541840 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 22:22:59.541840 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 22:22:59.541840 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 22:22:59.541840 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 22:22:59.541840 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 22:22:59.541840 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 22:22:59.541840 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 22:22:59.541840 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 22:22:59.541840 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 22:22:59.541840 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 22:22:59.541840 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 373f324d-27e7-42ba-90b0-6ed693d9cfec could not be found. Jul 16 22:22:59.541840 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 22:22:59.541840 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 22:22:59.541840 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 22:22:59.541840 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 22:22:59.541840 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 22:22:59.541840 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 22:22:59.541840 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 22:22:59.541840 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 22:22:59.541840 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 22:22:59.541840 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 22:22:59.541840 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 22:22:59.541840 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 22:22:59.541840 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 22:22:59.541840 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 22:22:59.541840 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 22:22:59.541840 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 22:22:59.541840 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 373f324d-27e7-42ba-90b0-6ed693d9cfec could not be found. Jul 16 22:22:59.541840 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 22:22:59.541840 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 22:22:59.541840 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 22:22:59.547829 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 22:22:59.547829 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 22:22:59.547829 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 22:22:59.547829 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 22:22:59.547829 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 22:22:59.547829 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 373f324d-27e7-42ba-90b0-6ed693d9cfec could not be found. Jul 16 22:22:59.547829 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 22:22:59.547829 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-239cb95d-5185-4fdb-bdd6-9c1ed86a127d tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Returning 404 to user: Instance 373f324d-27e7-42ba-90b0-6ed693d9cfec could not be found. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 22:22:59.547829 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-239cb95d-5185-4fdb-bdd6-9c1ed86a127d tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "GET /compute/v2.1/servers/373f324d-27e7-42ba-90b0-6ed693d9cfec" status: 404 len: 111 microversion: 2.1 time: 0.031883 Jul 16 22:22:59.547829 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3553/7109] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:22:59 2022] GET /compute/v2.1/servers/373f324d-27e7-42ba-90b0-6ed693d9cfec => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 22:23:03.697277 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-65cb8f3b-90d7-4c5a-9531-80b97cdb2884 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] Calling method '.version_select at 0x7fdda2b89160>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:23:03.794150 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-65cb8f3b-90d7-4c5a-9531-80b97cdb2884 tempest-TestNetworkAdvancedServerOps-234580283 tempest-TestNetworkAdvancedServerOps-234580283-project] 10.222.0.16 "DELETE /compute/v2.1/os-keypairs/tempest-TestNetworkAdvancedServerOps-2116104084" status: 202 len: 0 microversion: 2.1 time: 0.100777 Jul 16 22:23:03.794712 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3557/7110] 10.222.0.16 () {58 vars in 1283 bytes} [Sat Jul 16 22:23:03 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestNetworkAdvancedServerOps-2116104084 => generated 0 bytes in 102 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 22:23:05.021392 ncdv-master-1 devstack@n-api.service[138680]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jul 16 22:23:05.022002 ncdv-master-1 devstack@n-api.service[138680]: Sat Jul 16 22:23:05 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Jul 16 22:23:18.717057 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_api:SELECT=18 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:23:18.718229 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=51,nova_cell1:UPDATE=3,nova_cell1:INSERT=2 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:23:18.914493 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-8f5ec107-3ad5-44a3-bedd-b132372760c7 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Action: 'create', calling method: .version_select at 0x7fdd8fd99e50>, body: {"keypair": {"name": "tempest-TestShelveInstance-1336585092", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBFBVzaeKdpXbn96V5tnz3JbXhF2/x4DjlppZRBcVLDiIawWmgW+iKHZsR+0O8ygOInIhw3jWM8lz0wImT7hvBg+UuLyzFn/C8D+S6xjW7Vb1BtjdY3ZPQmD6ybErXwcKTw=="}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:23:18.929912 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-8f5ec107-3ad5-44a3-bedd-b132372760c7 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Getting quotas for project f1d60fe3fdc841998824b803809f9e49. Resources: {'key_pairs'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 22:23:18.936029 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-8f5ec107-3ad5-44a3-bedd-b132372760c7 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Getting quotas for user 45b6f8ac55464fc99331221b6c67a85d and project f1d60fe3fdc841998824b803809f9e49. Resources: {'key_pairs'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 22:23:19.005842 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-8f5ec107-3ad5-44a3-bedd-b132372760c7 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] 10.222.0.16 "POST /compute/v2.1/os-keypairs" status: 200 len: 397 microversion: 2.1 time: 0.288958 Jul 16 22:23:19.005842 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3554/7111] 10.222.0.16 () {60 vars in 1158 bytes} [Sat Jul 16 22:23:18 2022] POST /compute/v2.1/os-keypairs => generated 397 bytes in 291 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 22:23:21.497481 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=52,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:23:21.498616 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_api:SELECT=25,nova_api:UPDATE=1,nova_api:DELETE=1 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:23:21.502963 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-4fbfe49f-b3be-4cd5-ad56-7ef0726e39e0 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestShelveInstance-server-861718690", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452", "key_name": "tempest-TestShelveInstance-1336585092", "security_groups": [{"name": "tempest-secgroup-smoke-1317406157"}], "networks": [{"uuid": "b5314ebf-c1d9-472e-b991-557ed5cf9d11"}]}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:23:21.620882 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.network.neutron [None req-4fbfe49f-b3be-4cd5-ad56-7ef0726e39e0 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] validate_networks() for [('b5314ebf-c1d9-472e-b991-557ed5cf9d11', None, None, None, None, None)] {{(pid=138681) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 16 22:23:22.014186 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-4fbfe49f-b3be-4cd5-ad56-7ef0726e39e0 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Flavor limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 16 22:23:22.014186 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-4fbfe49f-b3be-4cd5-ad56-7ef0726e39e0 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Image limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 16 22:23:22.014186 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-4fbfe49f-b3be-4cd5-ad56-7ef0726e39e0 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Flavor pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 16 22:23:22.014186 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-4fbfe49f-b3be-4cd5-ad56-7ef0726e39e0 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Image pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 16 22:23:22.014186 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-4fbfe49f-b3be-4cd5-ad56-7ef0726e39e0 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 16 22:23:22.023370 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4fbfe49f-b3be-4cd5-ad56-7ef0726e39e0 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:23:22.023833 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4fbfe49f-b3be-4cd5-ad56-7ef0726e39e0 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:23:22.024638 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4fbfe49f-b3be-4cd5-ad56-7ef0726e39e0 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:23:22.025091 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4fbfe49f-b3be-4cd5-ad56-7ef0726e39e0 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:23:22.042577 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-4fbfe49f-b3be-4cd5-ad56-7ef0726e39e0 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Getting quotas for project f1d60fe3fdc841998824b803809f9e49. Resources: {'ram', 'instances', 'cores'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 22:23:22.048116 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-4fbfe49f-b3be-4cd5-ad56-7ef0726e39e0 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Getting quotas for user 45b6f8ac55464fc99331221b6c67a85d and project f1d60fe3fdc841998824b803809f9e49. Resources: {'ram', 'instances', 'cores'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 22:23:22.060587 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-4fbfe49f-b3be-4cd5-ad56-7ef0726e39e0 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Going to run 1 instances... {{(pid=138681) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 16 22:23:22.064316 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-4fbfe49f-b3be-4cd5-ad56-7ef0726e39e0 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] [instance: 4f08056e-5402-4d5a-88e3-c561d4ef9160] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138681) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 16 22:23:22.200547 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-4fbfe49f-b3be-4cd5-ad56-7ef0726e39e0 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 406 microversion: 2.1 time: 0.704819 Jul 16 22:23:22.201118 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3558/7112] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 22:23:21 2022] POST /compute/v2.1/servers => generated 406 bytes in 706 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 16 22:23:22.212118 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-cff1c2d7-e76e-47a7-ba93-5391124417c9 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:23:22.212736 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-cff1c2d7-e76e-47a7-ba93-5391124417c9 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] [instance: 4f08056e-5402-4d5a-88e3-c561d4ef9160] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:23:22.272576 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-cff1c2d7-e76e-47a7-ba93-5391124417c9 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '45b6f8ac55464fc99331221b6c67a85d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1d60fe3fdc841998824b803809f9e49', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:23:22.279540 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-cff1c2d7-e76e-47a7-ba93-5391124417c9 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] 10.222.0.16 "GET /compute/v2.1/servers/4f08056e-5402-4d5a-88e3-c561d4ef9160" status: 200 len: 1246 microversion: 2.1 time: 0.071334 Jul 16 22:23:22.280326 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3555/7113] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:23:22 2022] GET /compute/v2.1/servers/4f08056e-5402-4d5a-88e3-c561d4ef9160 => generated 1246 bytes in 73 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:23:23.297766 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-62ae60d7-9dc7-483f-9c09-ea4b671121b6 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:23:23.298541 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-62ae60d7-9dc7-483f-9c09-ea4b671121b6 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] [instance: 4f08056e-5402-4d5a-88e3-c561d4ef9160] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:23:23.306527 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-62ae60d7-9dc7-483f-9c09-ea4b671121b6 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:23:23.307001 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-62ae60d7-9dc7-483f-9c09-ea4b671121b6 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:23:23.307837 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-62ae60d7-9dc7-483f-9c09-ea4b671121b6 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:23:23.308293 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-62ae60d7-9dc7-483f-9c09-ea4b671121b6 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:23:23.378015 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-62ae60d7-9dc7-483f-9c09-ea4b671121b6 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '45b6f8ac55464fc99331221b6c67a85d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1d60fe3fdc841998824b803809f9e49', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:23:23.386052 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-62ae60d7-9dc7-483f-9c09-ea4b671121b6 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] 10.222.0.16 "GET /compute/v2.1/servers/4f08056e-5402-4d5a-88e3-c561d4ef9160" status: 200 len: 1250 microversion: 2.1 time: 0.093293 Jul 16 22:23:23.388212 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3559/7114] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:23:23 2022] GET /compute/v2.1/servers/4f08056e-5402-4d5a-88e3-c561d4ef9160 => generated 1250 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:23:24.404208 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-cfbadcf9-ad11-4685-be13-507fe5e574a4 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:23:24.404960 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-cfbadcf9-ad11-4685-be13-507fe5e574a4 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] [instance: 4f08056e-5402-4d5a-88e3-c561d4ef9160] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:23:24.413913 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cfbadcf9-ad11-4685-be13-507fe5e574a4 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:23:24.414372 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cfbadcf9-ad11-4685-be13-507fe5e574a4 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:23:24.415176 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cfbadcf9-ad11-4685-be13-507fe5e574a4 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:23:24.415618 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cfbadcf9-ad11-4685-be13-507fe5e574a4 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:23:24.483054 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-cfbadcf9-ad11-4685-be13-507fe5e574a4 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '45b6f8ac55464fc99331221b6c67a85d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1d60fe3fdc841998824b803809f9e49', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:23:24.491878 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-cfbadcf9-ad11-4685-be13-507fe5e574a4 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] 10.222.0.16 "GET /compute/v2.1/servers/4f08056e-5402-4d5a-88e3-c561d4ef9160" status: 200 len: 1316 microversion: 2.1 time: 0.092152 Jul 16 22:23:24.501023 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3556/7115] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:23:24 2022] GET /compute/v2.1/servers/4f08056e-5402-4d5a-88e3-c561d4ef9160 => generated 1316 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:23:25.509183 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-36b62f6a-2cfc-4fe7-baaf-bfa1069398bd tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:23:25.509937 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-36b62f6a-2cfc-4fe7-baaf-bfa1069398bd tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] [instance: 4f08056e-5402-4d5a-88e3-c561d4ef9160] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:23:25.518183 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-36b62f6a-2cfc-4fe7-baaf-bfa1069398bd tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:23:25.518729 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-36b62f6a-2cfc-4fe7-baaf-bfa1069398bd tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:23:25.519642 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-36b62f6a-2cfc-4fe7-baaf-bfa1069398bd tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:23:25.520100 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-36b62f6a-2cfc-4fe7-baaf-bfa1069398bd tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:23:25.658155 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-36b62f6a-2cfc-4fe7-baaf-bfa1069398bd tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '45b6f8ac55464fc99331221b6c67a85d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1d60fe3fdc841998824b803809f9e49', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:23:25.665831 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-36b62f6a-2cfc-4fe7-baaf-bfa1069398bd tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] 10.222.0.16 "GET /compute/v2.1/servers/4f08056e-5402-4d5a-88e3-c561d4ef9160" status: 200 len: 1372 microversion: 2.1 time: 0.160629 Jul 16 22:23:25.666410 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3560/7116] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:23:25 2022] GET /compute/v2.1/servers/4f08056e-5402-4d5a-88e3-c561d4ef9160 => generated 1372 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:23:26.147405 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-d63ca4e3-201f-4404-9440-90f0dab6c30e req-79991152-607d-4da7-9496-0adc3c50f394 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "4f08056e-5402-4d5a-88e3-c561d4ef9160", "tag": "e1ce8fbe-b51a-4c42-9649-f0936b53cf4b"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:23:26.156812 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-d63ca4e3-201f-4404-9440-90f0dab6c30e req-79991152-607d-4da7-9496-0adc3c50f394 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:23:26.157406 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-d63ca4e3-201f-4404-9440-90f0dab6c30e req-79991152-607d-4da7-9496-0adc3c50f394 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:23:26.169549 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-d63ca4e3-201f-4404-9440-90f0dab6c30e req-79991152-607d-4da7-9496-0adc3c50f394 service nova] Creating event network-changed:e1ce8fbe-b51a-4c42-9649-f0936b53cf4b for instance 4f08056e-5402-4d5a-88e3-c561d4ef9160 on nch1-master-1 Jul 16 22:23:26.176924 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-d63ca4e3-201f-4404-9440-90f0dab6c30e req-79991152-607d-4da7-9496-0adc3c50f394 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.095710 Jul 16 22:23:26.177997 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3557/7117] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 22:23:26 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 22:23:26.683655 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-b30cd36a-7826-4931-afd4-5b5f21a60320 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:23:26.684313 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-b30cd36a-7826-4931-afd4-5b5f21a60320 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] [instance: 4f08056e-5402-4d5a-88e3-c561d4ef9160] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:23:26.691537 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b30cd36a-7826-4931-afd4-5b5f21a60320 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:23:26.692004 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b30cd36a-7826-4931-afd4-5b5f21a60320 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:23:26.692790 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b30cd36a-7826-4931-afd4-5b5f21a60320 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:23:26.693251 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b30cd36a-7826-4931-afd4-5b5f21a60320 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:23:26.828402 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-b30cd36a-7826-4931-afd4-5b5f21a60320 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '45b6f8ac55464fc99331221b6c67a85d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1d60fe3fdc841998824b803809f9e49', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:23:26.836088 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-b30cd36a-7826-4931-afd4-5b5f21a60320 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] 10.222.0.16 "GET /compute/v2.1/servers/4f08056e-5402-4d5a-88e3-c561d4ef9160" status: 200 len: 1372 microversion: 2.1 time: 0.156417 Jul 16 22:23:26.836668 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3561/7118] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:23:26 2022] GET /compute/v2.1/servers/4f08056e-5402-4d5a-88e3-c561d4ef9160 => generated 1372 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:23:27.854250 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-ec5e36c1-fa7e-47cd-be49-9a1a90562827 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:23:27.854956 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-ec5e36c1-fa7e-47cd-be49-9a1a90562827 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] [instance: 4f08056e-5402-4d5a-88e3-c561d4ef9160] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:23:27.864106 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ec5e36c1-fa7e-47cd-be49-9a1a90562827 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:23:27.865023 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ec5e36c1-fa7e-47cd-be49-9a1a90562827 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:23:27.865558 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ec5e36c1-fa7e-47cd-be49-9a1a90562827 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:23:27.866085 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ec5e36c1-fa7e-47cd-be49-9a1a90562827 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:23:28.011910 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-ec5e36c1-fa7e-47cd-be49-9a1a90562827 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '45b6f8ac55464fc99331221b6c67a85d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1d60fe3fdc841998824b803809f9e49', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:23:28.015700 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-ec5e36c1-fa7e-47cd-be49-9a1a90562827 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] 10.222.0.16 "GET /compute/v2.1/servers/4f08056e-5402-4d5a-88e3-c561d4ef9160" status: 200 len: 1372 microversion: 2.1 time: 0.165101 Jul 16 22:23:28.016279 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3558/7119] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:23:27 2022] GET /compute/v2.1/servers/4f08056e-5402-4d5a-88e3-c561d4ef9160 => generated 1372 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:23:29.032455 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-5ae84d1e-710e-4b5c-a218-9463aa3af1e6 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:23:29.033094 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-5ae84d1e-710e-4b5c-a218-9463aa3af1e6 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] [instance: 4f08056e-5402-4d5a-88e3-c561d4ef9160] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:23:29.040152 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5ae84d1e-710e-4b5c-a218-9463aa3af1e6 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:23:29.040621 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5ae84d1e-710e-4b5c-a218-9463aa3af1e6 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:23:29.041408 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5ae84d1e-710e-4b5c-a218-9463aa3af1e6 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:23:29.041889 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5ae84d1e-710e-4b5c-a218-9463aa3af1e6 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:23:29.180122 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-5ae84d1e-710e-4b5c-a218-9463aa3af1e6 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '45b6f8ac55464fc99331221b6c67a85d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1d60fe3fdc841998824b803809f9e49', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:23:29.188650 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-5ae84d1e-710e-4b5c-a218-9463aa3af1e6 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] 10.222.0.16 "GET /compute/v2.1/servers/4f08056e-5402-4d5a-88e3-c561d4ef9160" status: 200 len: 1372 microversion: 2.1 time: 0.160326 Jul 16 22:23:29.189230 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3562/7120] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:23:29 2022] GET /compute/v2.1/servers/4f08056e-5402-4d5a-88e3-c561d4ef9160 => generated 1372 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:23:30.206451 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-f58785fd-fc01-4f32-9d25-20f6b26950e8 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:23:30.207111 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-f58785fd-fc01-4f32-9d25-20f6b26950e8 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] [instance: 4f08056e-5402-4d5a-88e3-c561d4ef9160] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:23:30.214234 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f58785fd-fc01-4f32-9d25-20f6b26950e8 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:23:30.214701 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f58785fd-fc01-4f32-9d25-20f6b26950e8 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:23:30.215495 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f58785fd-fc01-4f32-9d25-20f6b26950e8 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:23:30.215945 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f58785fd-fc01-4f32-9d25-20f6b26950e8 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:23:30.349188 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-f58785fd-fc01-4f32-9d25-20f6b26950e8 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '45b6f8ac55464fc99331221b6c67a85d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1d60fe3fdc841998824b803809f9e49', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:23:30.356796 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-f58785fd-fc01-4f32-9d25-20f6b26950e8 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] 10.222.0.16 "GET /compute/v2.1/servers/4f08056e-5402-4d5a-88e3-c561d4ef9160" status: 200 len: 1372 microversion: 2.1 time: 0.155013 Jul 16 22:23:30.357427 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3559/7121] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:23:30 2022] GET /compute/v2.1/servers/4f08056e-5402-4d5a-88e3-c561d4ef9160 => generated 1372 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:23:31.374047 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-797cf59c-0bc4-458c-a9ab-75afae9ab4ea tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:23:31.375209 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-797cf59c-0bc4-458c-a9ab-75afae9ab4ea tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] [instance: 4f08056e-5402-4d5a-88e3-c561d4ef9160] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:23:31.382353 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-797cf59c-0bc4-458c-a9ab-75afae9ab4ea tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:23:31.382831 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-797cf59c-0bc4-458c-a9ab-75afae9ab4ea tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:23:31.383656 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-797cf59c-0bc4-458c-a9ab-75afae9ab4ea tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:23:31.384115 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-797cf59c-0bc4-458c-a9ab-75afae9ab4ea tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:23:31.522972 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-797cf59c-0bc4-458c-a9ab-75afae9ab4ea tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '45b6f8ac55464fc99331221b6c67a85d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1d60fe3fdc841998824b803809f9e49', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:23:31.531421 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-797cf59c-0bc4-458c-a9ab-75afae9ab4ea tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] 10.222.0.16 "GET /compute/v2.1/servers/4f08056e-5402-4d5a-88e3-c561d4ef9160" status: 200 len: 1372 microversion: 2.1 time: 0.160535 Jul 16 22:23:31.532020 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3563/7122] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:23:31 2022] GET /compute/v2.1/servers/4f08056e-5402-4d5a-88e3-c561d4ef9160 => generated 1372 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:23:31.653846 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-beac3ef0-9ce0-4687-8b46-a4c0177935d7 req-c6dc9ea7-d69d-469d-bb4d-fb41ca018d0c service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "4f08056e-5402-4d5a-88e3-c561d4ef9160", "name": "network-vif-plugged", "status": "completed", "tag": "e1ce8fbe-b51a-4c42-9649-f0936b53cf4b"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:23:31.663176 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-beac3ef0-9ce0-4687-8b46-a4c0177935d7 req-c6dc9ea7-d69d-469d-bb4d-fb41ca018d0c service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:23:31.663637 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-beac3ef0-9ce0-4687-8b46-a4c0177935d7 req-c6dc9ea7-d69d-469d-bb4d-fb41ca018d0c service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:23:31.674912 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-beac3ef0-9ce0-4687-8b46-a4c0177935d7 req-c6dc9ea7-d69d-469d-bb4d-fb41ca018d0c service nova] Creating event network-vif-plugged:e1ce8fbe-b51a-4c42-9649-f0936b53cf4b for instance 4f08056e-5402-4d5a-88e3-c561d4ef9160 on nch1-master-1 Jul 16 22:23:31.680617 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-beac3ef0-9ce0-4687-8b46-a4c0177935d7 req-c6dc9ea7-d69d-469d-bb4d-fb41ca018d0c service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.030400 Jul 16 22:23:31.681211 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3560/7123] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 22:23:31 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 22:23:32.549084 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-c16652c2-9530-43c1-beb9-91b3e0f9c789 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:23:32.550121 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-c16652c2-9530-43c1-beb9-91b3e0f9c789 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] [instance: 4f08056e-5402-4d5a-88e3-c561d4ef9160] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:23:32.553116 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell0:SELECT=2 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:23:32.553820 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell0:SELECT=6 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:23:32.559644 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c16652c2-9530-43c1-beb9-91b3e0f9c789 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:23:32.560107 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c16652c2-9530-43c1-beb9-91b3e0f9c789 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:23:32.560939 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c16652c2-9530-43c1-beb9-91b3e0f9c789 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:23:32.561839 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c16652c2-9530-43c1-beb9-91b3e0f9c789 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:23:32.695730 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-c16652c2-9530-43c1-beb9-91b3e0f9c789 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '45b6f8ac55464fc99331221b6c67a85d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1d60fe3fdc841998824b803809f9e49', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:23:32.703594 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-c16652c2-9530-43c1-beb9-91b3e0f9c789 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] 10.222.0.16 "GET /compute/v2.1/servers/4f08056e-5402-4d5a-88e3-c561d4ef9160" status: 200 len: 1372 microversion: 2.1 time: 0.159127 Jul 16 22:23:32.704177 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3564/7124] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:23:32 2022] GET /compute/v2.1/servers/4f08056e-5402-4d5a-88e3-c561d4ef9160 => generated 1372 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:23:33.720944 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-d7476427-19c2-4aa3-bde2-bd186971ce7b tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:23:33.721650 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-d7476427-19c2-4aa3-bde2-bd186971ce7b tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] [instance: 4f08056e-5402-4d5a-88e3-c561d4ef9160] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:23:33.723651 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell0:SELECT=6 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:23:33.729795 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d7476427-19c2-4aa3-bde2-bd186971ce7b tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:23:33.731286 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d7476427-19c2-4aa3-bde2-bd186971ce7b tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:23:33.731286 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d7476427-19c2-4aa3-bde2-bd186971ce7b tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:23:33.731554 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d7476427-19c2-4aa3-bde2-bd186971ce7b tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:23:33.872374 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-d7476427-19c2-4aa3-bde2-bd186971ce7b tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '45b6f8ac55464fc99331221b6c67a85d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1d60fe3fdc841998824b803809f9e49', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:23:33.880700 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-d7476427-19c2-4aa3-bde2-bd186971ce7b tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] 10.222.0.16 "GET /compute/v2.1/servers/4f08056e-5402-4d5a-88e3-c561d4ef9160" status: 200 len: 1556 microversion: 2.1 time: 0.164093 Jul 16 22:23:33.881279 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3561/7125] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:23:33 2022] GET /compute/v2.1/servers/4f08056e-5402-4d5a-88e3-c561d4ef9160 => generated 1556 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:23:33.897010 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-64338d86-a119-424c-86ea-1e4dc6a0937e tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:23:33.897653 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-64338d86-a119-424c-86ea-1e4dc6a0937e tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] [instance: 4f08056e-5402-4d5a-88e3-c561d4ef9160] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:23:33.904722 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-64338d86-a119-424c-86ea-1e4dc6a0937e tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:23:33.905763 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-64338d86-a119-424c-86ea-1e4dc6a0937e tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:23:33.906670 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-64338d86-a119-424c-86ea-1e4dc6a0937e tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:23:33.907165 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-64338d86-a119-424c-86ea-1e4dc6a0937e tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:23:34.044760 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-64338d86-a119-424c-86ea-1e4dc6a0937e tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '45b6f8ac55464fc99331221b6c67a85d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1d60fe3fdc841998824b803809f9e49', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:23:34.053638 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-64338d86-a119-424c-86ea-1e4dc6a0937e tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] 10.222.0.16 "GET /compute/v2.1/servers/4f08056e-5402-4d5a-88e3-c561d4ef9160" status: 200 len: 1556 microversion: 2.1 time: 0.160588 Jul 16 22:23:34.054293 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3565/7126] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:23:33 2022] GET /compute/v2.1/servers/4f08056e-5402-4d5a-88e3-c561d4ef9160 => generated 1556 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:23:36.403917 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-765e3ce0-1277-4dd8-9809-afc6ed3d0d8b req-a85157e8-2a3b-4ab9-88b2-c5d5ac559498 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "4f08056e-5402-4d5a-88e3-c561d4ef9160", "tag": "e1ce8fbe-b51a-4c42-9649-f0936b53cf4b"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:23:36.412364 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-765e3ce0-1277-4dd8-9809-afc6ed3d0d8b req-a85157e8-2a3b-4ab9-88b2-c5d5ac559498 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:23:36.412815 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-765e3ce0-1277-4dd8-9809-afc6ed3d0d8b req-a85157e8-2a3b-4ab9-88b2-c5d5ac559498 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:23:36.424062 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-765e3ce0-1277-4dd8-9809-afc6ed3d0d8b req-a85157e8-2a3b-4ab9-88b2-c5d5ac559498 service nova] Creating event network-changed:e1ce8fbe-b51a-4c42-9649-f0936b53cf4b for instance 4f08056e-5402-4d5a-88e3-c561d4ef9160 on nch1-master-1 Jul 16 22:23:36.429352 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-765e3ce0-1277-4dd8-9809-afc6ed3d0d8b req-a85157e8-2a3b-4ab9-88b2-c5d5ac559498 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.083988 Jul 16 22:23:36.430239 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3562/7127] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 22:23:36 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 22:25:30.546392 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_api:SELECT=29,nova_api:INSERT=3 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:25:30.547239 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=44 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:25:30.551266 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-ecee76e3-195c-46dd-8acf-20fdc44667f5 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Action: 'action', calling method: >, body: {"shelve": {}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:25:30.551815 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-ecee76e3-195c-46dd-8acf-20fdc44667f5 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] [instance: 4f08056e-5402-4d5a-88e3-c561d4ef9160] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:25:30.558186 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ecee76e3-195c-46dd-8acf-20fdc44667f5 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:25:30.558654 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ecee76e3-195c-46dd-8acf-20fdc44667f5 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:25:30.559481 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ecee76e3-195c-46dd-8acf-20fdc44667f5 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:25:30.559936 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ecee76e3-195c-46dd-8acf-20fdc44667f5 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:25:30.583249 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.objects.instance [None req-ecee76e3-195c-46dd-8acf-20fdc44667f5 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lazy-loading 'flavor' on Instance uuid 4f08056e-5402-4d5a-88e3-c561d4ef9160 {{(pid=138681) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 22:25:31.197070 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-ecee76e3-195c-46dd-8acf-20fdc44667f5 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] 10.222.0.16 "POST /compute/v2.1/servers/4f08056e-5402-4d5a-88e3-c561d4ef9160/action" status: 202 len: 0 microversion: 2.1 time: 0.651853 Jul 16 22:25:31.197753 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3566/7128] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 22:25:30 2022] POST /compute/v2.1/servers/4f08056e-5402-4d5a-88e3-c561d4ef9160/action => generated 0 bytes in 654 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Jul 16 22:25:31.205157 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_api:SELECT=29,nova_api:INSERT=1 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:25:31.205918 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=30 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:25:31.209985 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-acc29a37-a718-4f2c-b8c3-e48f29ab79c3 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:25:31.210689 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-acc29a37-a718-4f2c-b8c3-e48f29ab79c3 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] [instance: 4f08056e-5402-4d5a-88e3-c561d4ef9160] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:25:31.217167 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-acc29a37-a718-4f2c-b8c3-e48f29ab79c3 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:25:31.218583 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-acc29a37-a718-4f2c-b8c3-e48f29ab79c3 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:25:31.218583 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-acc29a37-a718-4f2c-b8c3-e48f29ab79c3 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:25:31.218881 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-acc29a37-a718-4f2c-b8c3-e48f29ab79c3 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:25:31.356079 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-acc29a37-a718-4f2c-b8c3-e48f29ab79c3 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '45b6f8ac55464fc99331221b6c67a85d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1d60fe3fdc841998824b803809f9e49', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:25:31.364011 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-acc29a37-a718-4f2c-b8c3-e48f29ab79c3 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] 10.222.0.16 "GET /compute/v2.1/servers/4f08056e-5402-4d5a-88e3-c561d4ef9160" status: 200 len: 1680 microversion: 2.1 time: 0.161444 Jul 16 22:25:31.364571 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3563/7129] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:25:31 2022] GET /compute/v2.1/servers/4f08056e-5402-4d5a-88e3-c561d4ef9160 => generated 1680 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:25:32.383366 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-cbf4b92d-32aa-4f1d-b2d5-4b4a930bd27d tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:25:32.384315 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-cbf4b92d-32aa-4f1d-b2d5-4b4a930bd27d tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] [instance: 4f08056e-5402-4d5a-88e3-c561d4ef9160] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:25:32.390707 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cbf4b92d-32aa-4f1d-b2d5-4b4a930bd27d tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:25:32.391221 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cbf4b92d-32aa-4f1d-b2d5-4b4a930bd27d tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:25:32.392027 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cbf4b92d-32aa-4f1d-b2d5-4b4a930bd27d tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:25:32.392485 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-cbf4b92d-32aa-4f1d-b2d5-4b4a930bd27d tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:25:32.526641 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-cbf4b92d-32aa-4f1d-b2d5-4b4a930bd27d tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '45b6f8ac55464fc99331221b6c67a85d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1d60fe3fdc841998824b803809f9e49', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:25:32.534253 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-cbf4b92d-32aa-4f1d-b2d5-4b4a930bd27d tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] 10.222.0.16 "GET /compute/v2.1/servers/4f08056e-5402-4d5a-88e3-c561d4ef9160" status: 200 len: 1680 microversion: 2.1 time: 0.156273 Jul 16 22:25:32.542617 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3567/7130] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:25:32 2022] GET /compute/v2.1/servers/4f08056e-5402-4d5a-88e3-c561d4ef9160 => generated 1680 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:25:33.551926 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-31343ef7-ae92-48ac-a817-32781c0ea11f tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:25:33.552788 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-31343ef7-ae92-48ac-a817-32781c0ea11f tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] [instance: 4f08056e-5402-4d5a-88e3-c561d4ef9160] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:25:33.560393 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-31343ef7-ae92-48ac-a817-32781c0ea11f tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:25:33.560839 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-31343ef7-ae92-48ac-a817-32781c0ea11f tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:25:33.561594 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-31343ef7-ae92-48ac-a817-32781c0ea11f tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:25:33.562032 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-31343ef7-ae92-48ac-a817-32781c0ea11f tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:25:33.702486 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-31343ef7-ae92-48ac-a817-32781c0ea11f tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '45b6f8ac55464fc99331221b6c67a85d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1d60fe3fdc841998824b803809f9e49', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:25:33.710134 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-31343ef7-ae92-48ac-a817-32781c0ea11f tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] 10.222.0.16 "GET /compute/v2.1/servers/4f08056e-5402-4d5a-88e3-c561d4ef9160" status: 200 len: 1680 microversion: 2.1 time: 0.162152 Jul 16 22:25:33.710688 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3564/7131] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:25:33 2022] GET /compute/v2.1/servers/4f08056e-5402-4d5a-88e3-c561d4ef9160 => generated 1680 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:25:34.728871 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-498e7018-8678-4f59-9809-4615aa061be6 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:25:34.729558 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-498e7018-8678-4f59-9809-4615aa061be6 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] [instance: 4f08056e-5402-4d5a-88e3-c561d4ef9160] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:25:34.736640 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-498e7018-8678-4f59-9809-4615aa061be6 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:25:34.737690 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-498e7018-8678-4f59-9809-4615aa061be6 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:25:34.737909 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-498e7018-8678-4f59-9809-4615aa061be6 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:25:34.738365 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-498e7018-8678-4f59-9809-4615aa061be6 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:25:34.872600 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-498e7018-8678-4f59-9809-4615aa061be6 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '45b6f8ac55464fc99331221b6c67a85d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1d60fe3fdc841998824b803809f9e49', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:25:34.880216 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-498e7018-8678-4f59-9809-4615aa061be6 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] 10.222.0.16 "GET /compute/v2.1/servers/4f08056e-5402-4d5a-88e3-c561d4ef9160" status: 200 len: 1680 microversion: 2.1 time: 0.155861 Jul 16 22:25:34.880792 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3568/7132] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:25:34 2022] GET /compute/v2.1/servers/4f08056e-5402-4d5a-88e3-c561d4ef9160 => generated 1680 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:25:35.897793 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-9f8f7bbf-e38c-4b4b-b293-68ca6a9ee704 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:25:35.898345 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-9f8f7bbf-e38c-4b4b-b293-68ca6a9ee704 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] [instance: 4f08056e-5402-4d5a-88e3-c561d4ef9160] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:25:35.906226 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9f8f7bbf-e38c-4b4b-b293-68ca6a9ee704 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:25:35.906678 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9f8f7bbf-e38c-4b4b-b293-68ca6a9ee704 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:25:35.907473 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9f8f7bbf-e38c-4b4b-b293-68ca6a9ee704 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:25:35.907914 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9f8f7bbf-e38c-4b4b-b293-68ca6a9ee704 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:25:36.357400 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-9f8f7bbf-e38c-4b4b-b293-68ca6a9ee704 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '45b6f8ac55464fc99331221b6c67a85d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1d60fe3fdc841998824b803809f9e49', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:25:36.364967 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-9f8f7bbf-e38c-4b4b-b293-68ca6a9ee704 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] 10.222.0.16 "GET /compute/v2.1/servers/4f08056e-5402-4d5a-88e3-c561d4ef9160" status: 200 len: 1680 microversion: 2.1 time: 0.471168 Jul 16 22:25:36.365535 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3565/7133] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:25:35 2022] GET /compute/v2.1/servers/4f08056e-5402-4d5a-88e3-c561d4ef9160 => generated 1680 bytes in 472 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:25:37.383235 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-13ffd524-3e5d-4a9b-b56e-d5a7e0645c52 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:25:37.384583 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-13ffd524-3e5d-4a9b-b56e-d5a7e0645c52 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] [instance: 4f08056e-5402-4d5a-88e3-c561d4ef9160] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:25:37.391565 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-13ffd524-3e5d-4a9b-b56e-d5a7e0645c52 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:25:37.392030 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-13ffd524-3e5d-4a9b-b56e-d5a7e0645c52 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:25:37.392803 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-13ffd524-3e5d-4a9b-b56e-d5a7e0645c52 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:25:37.393254 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-13ffd524-3e5d-4a9b-b56e-d5a7e0645c52 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:25:37.525893 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-13ffd524-3e5d-4a9b-b56e-d5a7e0645c52 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '45b6f8ac55464fc99331221b6c67a85d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1d60fe3fdc841998824b803809f9e49', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:25:37.533385 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-13ffd524-3e5d-4a9b-b56e-d5a7e0645c52 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] 10.222.0.16 "GET /compute/v2.1/servers/4f08056e-5402-4d5a-88e3-c561d4ef9160" status: 200 len: 1680 microversion: 2.1 time: 0.154329 Jul 16 22:25:37.533952 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3569/7134] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:25:37 2022] GET /compute/v2.1/servers/4f08056e-5402-4d5a-88e3-c561d4ef9160 => generated 1680 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:25:38.551054 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-973c4c81-9393-4044-b578-f6dcdfabbb16 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:25:38.551683 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-973c4c81-9393-4044-b578-f6dcdfabbb16 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] [instance: 4f08056e-5402-4d5a-88e3-c561d4ef9160] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:25:38.559648 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-973c4c81-9393-4044-b578-f6dcdfabbb16 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:25:38.560052 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-973c4c81-9393-4044-b578-f6dcdfabbb16 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:25:38.560820 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-973c4c81-9393-4044-b578-f6dcdfabbb16 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:25:38.561267 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-973c4c81-9393-4044-b578-f6dcdfabbb16 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:25:38.698423 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-973c4c81-9393-4044-b578-f6dcdfabbb16 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '45b6f8ac55464fc99331221b6c67a85d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1d60fe3fdc841998824b803809f9e49', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:25:38.706183 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-973c4c81-9393-4044-b578-f6dcdfabbb16 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] 10.222.0.16 "GET /compute/v2.1/servers/4f08056e-5402-4d5a-88e3-c561d4ef9160" status: 200 len: 1680 microversion: 2.1 time: 0.159127 Jul 16 22:25:38.706790 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3566/7135] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:25:38 2022] GET /compute/v2.1/servers/4f08056e-5402-4d5a-88e3-c561d4ef9160 => generated 1680 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:25:39.723696 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-dfa2bfb8-82f8-4de2-aeb4-9ef6ad481842 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:25:39.724339 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-dfa2bfb8-82f8-4de2-aeb4-9ef6ad481842 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] [instance: 4f08056e-5402-4d5a-88e3-c561d4ef9160] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:25:39.732165 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-dfa2bfb8-82f8-4de2-aeb4-9ef6ad481842 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:25:39.732635 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-dfa2bfb8-82f8-4de2-aeb4-9ef6ad481842 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:25:39.733410 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-dfa2bfb8-82f8-4de2-aeb4-9ef6ad481842 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:25:39.733855 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-dfa2bfb8-82f8-4de2-aeb4-9ef6ad481842 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:25:39.865229 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-dfa2bfb8-82f8-4de2-aeb4-9ef6ad481842 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '45b6f8ac55464fc99331221b6c67a85d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1d60fe3fdc841998824b803809f9e49', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:25:39.873607 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-dfa2bfb8-82f8-4de2-aeb4-9ef6ad481842 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] 10.222.0.16 "GET /compute/v2.1/servers/4f08056e-5402-4d5a-88e3-c561d4ef9160" status: 200 len: 1680 microversion: 2.1 time: 0.153861 Jul 16 22:25:39.874174 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3570/7136] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:25:39 2022] GET /compute/v2.1/servers/4f08056e-5402-4d5a-88e3-c561d4ef9160 => generated 1680 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:25:40.890884 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-2950d2fb-c19f-4e6e-b989-58b9d646b171 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:25:40.891554 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-2950d2fb-c19f-4e6e-b989-58b9d646b171 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] [instance: 4f08056e-5402-4d5a-88e3-c561d4ef9160] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:25:40.898502 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2950d2fb-c19f-4e6e-b989-58b9d646b171 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:25:40.898965 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2950d2fb-c19f-4e6e-b989-58b9d646b171 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:25:40.899814 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2950d2fb-c19f-4e6e-b989-58b9d646b171 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:25:40.900343 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2950d2fb-c19f-4e6e-b989-58b9d646b171 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:25:41.031878 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-2950d2fb-c19f-4e6e-b989-58b9d646b171 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '45b6f8ac55464fc99331221b6c67a85d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1d60fe3fdc841998824b803809f9e49', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:25:41.039617 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-2950d2fb-c19f-4e6e-b989-58b9d646b171 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] 10.222.0.16 "GET /compute/v2.1/servers/4f08056e-5402-4d5a-88e3-c561d4ef9160" status: 200 len: 1680 microversion: 2.1 time: 0.152685 Jul 16 22:25:41.040351 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3567/7137] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:25:40 2022] GET /compute/v2.1/servers/4f08056e-5402-4d5a-88e3-c561d4ef9160 => generated 1680 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:25:42.056965 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-1ee89ffa-8575-4ded-b168-4e1f09652220 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:25:42.057603 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-1ee89ffa-8575-4ded-b168-4e1f09652220 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] [instance: 4f08056e-5402-4d5a-88e3-c561d4ef9160] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:25:42.064788 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1ee89ffa-8575-4ded-b168-4e1f09652220 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:25:42.065260 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1ee89ffa-8575-4ded-b168-4e1f09652220 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:25:42.066037 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1ee89ffa-8575-4ded-b168-4e1f09652220 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:25:42.066493 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1ee89ffa-8575-4ded-b168-4e1f09652220 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:25:42.208639 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-1ee89ffa-8575-4ded-b168-4e1f09652220 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '45b6f8ac55464fc99331221b6c67a85d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1d60fe3fdc841998824b803809f9e49', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:25:42.217809 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-1ee89ffa-8575-4ded-b168-4e1f09652220 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] 10.222.0.16 "GET /compute/v2.1/servers/4f08056e-5402-4d5a-88e3-c561d4ef9160" status: 200 len: 1701 microversion: 2.1 time: 0.165042 Jul 16 22:25:42.218401 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3571/7138] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:25:42 2022] GET /compute/v2.1/servers/4f08056e-5402-4d5a-88e3-c561d4ef9160 => generated 1701 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 22:25:43.234881 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-8daf74d2-f4f6-4b8e-a122-2b7f579a197e tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:25:43.235553 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-8daf74d2-f4f6-4b8e-a122-2b7f579a197e tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] [instance: 4f08056e-5402-4d5a-88e3-c561d4ef9160] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:25:43.242913 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8daf74d2-f4f6-4b8e-a122-2b7f579a197e tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:25:43.244441 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8daf74d2-f4f6-4b8e-a122-2b7f579a197e tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:25:43.244441 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8daf74d2-f4f6-4b8e-a122-2b7f579a197e tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:25:43.244739 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-8daf74d2-f4f6-4b8e-a122-2b7f579a197e tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:25:43.381792 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-8daf74d2-f4f6-4b8e-a122-2b7f579a197e tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '45b6f8ac55464fc99331221b6c67a85d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1d60fe3fdc841998824b803809f9e49', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:25:43.389548 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-8daf74d2-f4f6-4b8e-a122-2b7f579a197e tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] 10.222.0.16 "GET /compute/v2.1/servers/4f08056e-5402-4d5a-88e3-c561d4ef9160" status: 200 len: 1701 microversion: 2.1 time: 0.158894 Jul 16 22:25:43.390132 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3568/7139] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:25:43 2022] GET /compute/v2.1/servers/4f08056e-5402-4d5a-88e3-c561d4ef9160 => generated 1701 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:25:44.344506 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-00e582df-0277-4ee4-8de3-1a118c391406 req-d1f8edd5-ce9b-4dd2-9d62-5c318ca39b52 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "4f08056e-5402-4d5a-88e3-c561d4ef9160", "name": "network-vif-plugged", "status": "completed", "tag": "e1ce8fbe-b51a-4c42-9649-f0936b53cf4b"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:25:44.352725 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-00e582df-0277-4ee4-8de3-1a118c391406 req-d1f8edd5-ce9b-4dd2-9d62-5c318ca39b52 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:25:44.353210 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-00e582df-0277-4ee4-8de3-1a118c391406 req-d1f8edd5-ce9b-4dd2-9d62-5c318ca39b52 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:25:44.364806 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-00e582df-0277-4ee4-8de3-1a118c391406 req-d1f8edd5-ce9b-4dd2-9d62-5c318ca39b52 service nova] Creating event network-vif-plugged:e1ce8fbe-b51a-4c42-9649-f0936b53cf4b for instance 4f08056e-5402-4d5a-88e3-c561d4ef9160 on nch1-master-1 Jul 16 22:25:44.370858 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-00e582df-0277-4ee4-8de3-1a118c391406 req-d1f8edd5-ce9b-4dd2-9d62-5c318ca39b52 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.115088 Jul 16 22:25:44.373214 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3572/7140] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 22:25:44 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 117 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 16 22:25:44.406750 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-f57a422e-d48e-499c-9c4d-4459709ec67e tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:25:44.407544 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-f57a422e-d48e-499c-9c4d-4459709ec67e tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] [instance: 4f08056e-5402-4d5a-88e3-c561d4ef9160] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:25:44.415498 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f57a422e-d48e-499c-9c4d-4459709ec67e tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:25:44.416061 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f57a422e-d48e-499c-9c4d-4459709ec67e tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:25:44.416918 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f57a422e-d48e-499c-9c4d-4459709ec67e tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:25:44.417446 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f57a422e-d48e-499c-9c4d-4459709ec67e tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:25:44.559084 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-f57a422e-d48e-499c-9c4d-4459709ec67e tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '45b6f8ac55464fc99331221b6c67a85d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1d60fe3fdc841998824b803809f9e49', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:25:44.567055 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-f57a422e-d48e-499c-9c4d-4459709ec67e tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] 10.222.0.16 "GET /compute/v2.1/servers/4f08056e-5402-4d5a-88e3-c561d4ef9160" status: 200 len: 1696 microversion: 2.1 time: 0.164394 Jul 16 22:25:44.568056 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3569/7141] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:25:44 2022] GET /compute/v2.1/servers/4f08056e-5402-4d5a-88e3-c561d4ef9160 => generated 1696 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:25:45.585601 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-1c6afb50-9a2e-458c-8736-b9b7a9522854 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:25:45.587755 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-1c6afb50-9a2e-458c-8736-b9b7a9522854 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] [instance: 4f08056e-5402-4d5a-88e3-c561d4ef9160] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:25:45.597389 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1c6afb50-9a2e-458c-8736-b9b7a9522854 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:25:45.598140 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1c6afb50-9a2e-458c-8736-b9b7a9522854 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:25:45.598945 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1c6afb50-9a2e-458c-8736-b9b7a9522854 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:25:45.599459 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1c6afb50-9a2e-458c-8736-b9b7a9522854 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:25:45.750959 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-1c6afb50-9a2e-458c-8736-b9b7a9522854 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '45b6f8ac55464fc99331221b6c67a85d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1d60fe3fdc841998824b803809f9e49', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:25:45.759922 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-1c6afb50-9a2e-458c-8736-b9b7a9522854 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] 10.222.0.16 "GET /compute/v2.1/servers/4f08056e-5402-4d5a-88e3-c561d4ef9160" status: 200 len: 1696 microversion: 2.1 time: 0.178398 Jul 16 22:25:45.763249 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3573/7142] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:25:45 2022] GET /compute/v2.1/servers/4f08056e-5402-4d5a-88e3-c561d4ef9160 => generated 1696 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:25:46.788834 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-74990e80-6bdd-4640-8970-ab2949e3e499 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:25:46.789394 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-74990e80-6bdd-4640-8970-ab2949e3e499 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] [instance: 4f08056e-5402-4d5a-88e3-c561d4ef9160] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:25:46.796495 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-74990e80-6bdd-4640-8970-ab2949e3e499 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:25:46.797351 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-74990e80-6bdd-4640-8970-ab2949e3e499 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:25:46.798952 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-74990e80-6bdd-4640-8970-ab2949e3e499 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:25:46.799952 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-74990e80-6bdd-4640-8970-ab2949e3e499 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:25:46.964936 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-74990e80-6bdd-4640-8970-ab2949e3e499 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '45b6f8ac55464fc99331221b6c67a85d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1d60fe3fdc841998824b803809f9e49', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:25:46.973018 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-74990e80-6bdd-4640-8970-ab2949e3e499 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] 10.222.0.16 "GET /compute/v2.1/servers/4f08056e-5402-4d5a-88e3-c561d4ef9160" status: 200 len: 1696 microversion: 2.1 time: 0.188458 Jul 16 22:25:46.973768 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3570/7143] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:25:46 2022] GET /compute/v2.1/servers/4f08056e-5402-4d5a-88e3-c561d4ef9160 => generated 1696 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:25:47.990187 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-6bc51a18-818e-4786-937d-81fd96f6501d tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:25:47.990818 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-6bc51a18-818e-4786-937d-81fd96f6501d tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] [instance: 4f08056e-5402-4d5a-88e3-c561d4ef9160] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:25:47.998422 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6bc51a18-818e-4786-937d-81fd96f6501d tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:25:47.999252 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6bc51a18-818e-4786-937d-81fd96f6501d tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:25:48.000240 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6bc51a18-818e-4786-937d-81fd96f6501d tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:25:48.000691 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6bc51a18-818e-4786-937d-81fd96f6501d tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:25:48.154197 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-6bc51a18-818e-4786-937d-81fd96f6501d tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '45b6f8ac55464fc99331221b6c67a85d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1d60fe3fdc841998824b803809f9e49', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:25:48.162265 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-6bc51a18-818e-4786-937d-81fd96f6501d tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] 10.222.0.16 "GET /compute/v2.1/servers/4f08056e-5402-4d5a-88e3-c561d4ef9160" status: 200 len: 1696 microversion: 2.1 time: 0.176343 Jul 16 22:25:48.162860 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3574/7144] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:25:47 2022] GET /compute/v2.1/servers/4f08056e-5402-4d5a-88e3-c561d4ef9160 => generated 1696 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:25:49.180402 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-1c62d16a-5a28-4e6a-aa6e-1c8e6dedde3f tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:25:49.181805 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-1c62d16a-5a28-4e6a-aa6e-1c8e6dedde3f tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] [instance: 4f08056e-5402-4d5a-88e3-c561d4ef9160] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:25:49.190563 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1c62d16a-5a28-4e6a-aa6e-1c8e6dedde3f tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:25:49.191040 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1c62d16a-5a28-4e6a-aa6e-1c8e6dedde3f tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:25:49.192039 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1c62d16a-5a28-4e6a-aa6e-1c8e6dedde3f tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:25:49.192495 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1c62d16a-5a28-4e6a-aa6e-1c8e6dedde3f tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:25:49.341043 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-1c62d16a-5a28-4e6a-aa6e-1c8e6dedde3f tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '45b6f8ac55464fc99331221b6c67a85d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1d60fe3fdc841998824b803809f9e49', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:25:49.349178 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-1c62d16a-5a28-4e6a-aa6e-1c8e6dedde3f tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] 10.222.0.16 "GET /compute/v2.1/servers/4f08056e-5402-4d5a-88e3-c561d4ef9160" status: 200 len: 1696 microversion: 2.1 time: 0.173280 Jul 16 22:25:49.351336 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3571/7145] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:25:49 2022] GET /compute/v2.1/servers/4f08056e-5402-4d5a-88e3-c561d4ef9160 => generated 1696 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:25:50.369296 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-bd3e5e0a-1ebf-4b4e-aac1-b1c0997ecde1 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:25:50.370803 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-bd3e5e0a-1ebf-4b4e-aac1-b1c0997ecde1 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] [instance: 4f08056e-5402-4d5a-88e3-c561d4ef9160] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:25:50.379035 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-bd3e5e0a-1ebf-4b4e-aac1-b1c0997ecde1 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:25:50.379580 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-bd3e5e0a-1ebf-4b4e-aac1-b1c0997ecde1 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:25:50.380483 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-bd3e5e0a-1ebf-4b4e-aac1-b1c0997ecde1 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:25:50.381045 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-bd3e5e0a-1ebf-4b4e-aac1-b1c0997ecde1 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:25:50.527420 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-bd3e5e0a-1ebf-4b4e-aac1-b1c0997ecde1 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '45b6f8ac55464fc99331221b6c67a85d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1d60fe3fdc841998824b803809f9e49', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:25:50.535306 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-bd3e5e0a-1ebf-4b4e-aac1-b1c0997ecde1 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] 10.222.0.16 "GET /compute/v2.1/servers/4f08056e-5402-4d5a-88e3-c561d4ef9160" status: 200 len: 1696 microversion: 2.1 time: 0.171090 Jul 16 22:25:50.535895 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3575/7146] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:25:50 2022] GET /compute/v2.1/servers/4f08056e-5402-4d5a-88e3-c561d4ef9160 => generated 1696 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:25:51.554354 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-6aabc3d2-df1e-4ee7-9f1e-204b53a351ab tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:25:51.554998 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-6aabc3d2-df1e-4ee7-9f1e-204b53a351ab tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] [instance: 4f08056e-5402-4d5a-88e3-c561d4ef9160] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:25:51.562281 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6aabc3d2-df1e-4ee7-9f1e-204b53a351ab tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:25:51.562985 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6aabc3d2-df1e-4ee7-9f1e-204b53a351ab tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:25:51.564057 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6aabc3d2-df1e-4ee7-9f1e-204b53a351ab tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:25:51.564507 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-6aabc3d2-df1e-4ee7-9f1e-204b53a351ab tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:25:51.707887 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-6aabc3d2-df1e-4ee7-9f1e-204b53a351ab tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '45b6f8ac55464fc99331221b6c67a85d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1d60fe3fdc841998824b803809f9e49', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:25:51.717157 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-6aabc3d2-df1e-4ee7-9f1e-204b53a351ab tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] 10.222.0.16 "GET /compute/v2.1/servers/4f08056e-5402-4d5a-88e3-c561d4ef9160" status: 200 len: 1696 microversion: 2.1 time: 0.167303 Jul 16 22:25:51.717961 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3572/7147] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:25:51 2022] GET /compute/v2.1/servers/4f08056e-5402-4d5a-88e3-c561d4ef9160 => generated 1696 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:25:52.735257 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-b2ebc7e8-bb8b-4cf6-ac66-8bccd00e6cd5 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:25:52.736544 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-b2ebc7e8-bb8b-4cf6-ac66-8bccd00e6cd5 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] [instance: 4f08056e-5402-4d5a-88e3-c561d4ef9160] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:25:52.744288 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b2ebc7e8-bb8b-4cf6-ac66-8bccd00e6cd5 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:25:52.745862 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b2ebc7e8-bb8b-4cf6-ac66-8bccd00e6cd5 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:25:52.746736 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b2ebc7e8-bb8b-4cf6-ac66-8bccd00e6cd5 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:25:52.747286 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b2ebc7e8-bb8b-4cf6-ac66-8bccd00e6cd5 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:25:52.910732 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-b2ebc7e8-bb8b-4cf6-ac66-8bccd00e6cd5 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '45b6f8ac55464fc99331221b6c67a85d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1d60fe3fdc841998824b803809f9e49', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:25:52.921293 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-b2ebc7e8-bb8b-4cf6-ac66-8bccd00e6cd5 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] 10.222.0.16 "GET /compute/v2.1/servers/4f08056e-5402-4d5a-88e3-c561d4ef9160" status: 200 len: 1696 microversion: 2.1 time: 0.188872 Jul 16 22:25:52.923538 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3576/7148] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:25:52 2022] GET /compute/v2.1/servers/4f08056e-5402-4d5a-88e3-c561d4ef9160 => generated 1696 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:25:53.937680 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-46a9d6f8-283f-4e28-94c5-db047b79f088 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:25:53.938321 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-46a9d6f8-283f-4e28-94c5-db047b79f088 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] [instance: 4f08056e-5402-4d5a-88e3-c561d4ef9160] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:25:53.946031 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-46a9d6f8-283f-4e28-94c5-db047b79f088 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:25:53.946443 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-46a9d6f8-283f-4e28-94c5-db047b79f088 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:25:53.947249 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-46a9d6f8-283f-4e28-94c5-db047b79f088 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:25:53.948400 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-46a9d6f8-283f-4e28-94c5-db047b79f088 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:25:54.093638 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-46a9d6f8-283f-4e28-94c5-db047b79f088 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '45b6f8ac55464fc99331221b6c67a85d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1d60fe3fdc841998824b803809f9e49', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:25:54.104657 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-46a9d6f8-283f-4e28-94c5-db047b79f088 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] 10.222.0.16 "GET /compute/v2.1/servers/4f08056e-5402-4d5a-88e3-c561d4ef9160" status: 200 len: 1696 microversion: 2.1 time: 0.171024 Jul 16 22:25:54.105247 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3573/7149] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:25:53 2022] GET /compute/v2.1/servers/4f08056e-5402-4d5a-88e3-c561d4ef9160 => generated 1696 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:25:55.122635 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-bfc6ec6e-eeb0-463c-9454-b180341f7efb tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:25:55.123242 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-bfc6ec6e-eeb0-463c-9454-b180341f7efb tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] [instance: 4f08056e-5402-4d5a-88e3-c561d4ef9160] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:25:55.130312 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-bfc6ec6e-eeb0-463c-9454-b180341f7efb tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:25:55.130788 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-bfc6ec6e-eeb0-463c-9454-b180341f7efb tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:25:55.131605 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-bfc6ec6e-eeb0-463c-9454-b180341f7efb tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:25:55.132064 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-bfc6ec6e-eeb0-463c-9454-b180341f7efb tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:25:55.271816 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-bfc6ec6e-eeb0-463c-9454-b180341f7efb tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '45b6f8ac55464fc99331221b6c67a85d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1d60fe3fdc841998824b803809f9e49', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:25:55.280124 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-bfc6ec6e-eeb0-463c-9454-b180341f7efb tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] 10.222.0.16 "GET /compute/v2.1/servers/4f08056e-5402-4d5a-88e3-c561d4ef9160" status: 200 len: 1678 microversion: 2.1 time: 0.162263 Jul 16 22:25:55.280725 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3577/7150] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:25:55 2022] GET /compute/v2.1/servers/4f08056e-5402-4d5a-88e3-c561d4ef9160 => generated 1678 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:25:56.298588 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-cfaca259-38d5-48ca-9af9-94d4b438cfe4 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:25:56.299244 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-cfaca259-38d5-48ca-9af9-94d4b438cfe4 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] [instance: 4f08056e-5402-4d5a-88e3-c561d4ef9160] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:25:56.306509 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cfaca259-38d5-48ca-9af9-94d4b438cfe4 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:25:56.307675 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cfaca259-38d5-48ca-9af9-94d4b438cfe4 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:25:56.308453 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cfaca259-38d5-48ca-9af9-94d4b438cfe4 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:25:56.308898 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cfaca259-38d5-48ca-9af9-94d4b438cfe4 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:25:56.448420 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-cfaca259-38d5-48ca-9af9-94d4b438cfe4 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '45b6f8ac55464fc99331221b6c67a85d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1d60fe3fdc841998824b803809f9e49', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:25:56.456168 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-cfaca259-38d5-48ca-9af9-94d4b438cfe4 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] 10.222.0.16 "GET /compute/v2.1/servers/4f08056e-5402-4d5a-88e3-c561d4ef9160" status: 200 len: 1678 microversion: 2.1 time: 0.161685 Jul 16 22:25:56.456747 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3574/7151] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:25:56 2022] GET /compute/v2.1/servers/4f08056e-5402-4d5a-88e3-c561d4ef9160 => generated 1678 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:25:57.473557 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-6526f0e2-5c3f-403a-b85b-d600c7a4db24 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:25:57.474110 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-6526f0e2-5c3f-403a-b85b-d600c7a4db24 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] [instance: 4f08056e-5402-4d5a-88e3-c561d4ef9160] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:25:57.481472 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6526f0e2-5c3f-403a-b85b-d600c7a4db24 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:25:57.481932 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6526f0e2-5c3f-403a-b85b-d600c7a4db24 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:25:57.482702 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6526f0e2-5c3f-403a-b85b-d600c7a4db24 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:25:57.483186 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-6526f0e2-5c3f-403a-b85b-d600c7a4db24 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:25:57.621988 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-6526f0e2-5c3f-403a-b85b-d600c7a4db24 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '45b6f8ac55464fc99331221b6c67a85d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1d60fe3fdc841998824b803809f9e49', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:25:57.629588 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-6526f0e2-5c3f-403a-b85b-d600c7a4db24 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] 10.222.0.16 "GET /compute/v2.1/servers/4f08056e-5402-4d5a-88e3-c561d4ef9160" status: 200 len: 1681 microversion: 2.1 time: 0.160123 Jul 16 22:25:57.630162 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3578/7152] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:25:57 2022] GET /compute/v2.1/servers/4f08056e-5402-4d5a-88e3-c561d4ef9160 => generated 1681 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:25:57.646690 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-c37664c2-a3e7-42a8-a1a1-334e8ac1618b tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Action: 'action', calling method: >, body: {"unshelve": null} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:25:57.647410 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-c37664c2-a3e7-42a8-a1a1-334e8ac1618b tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] [instance: 4f08056e-5402-4d5a-88e3-c561d4ef9160] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:25:57.655001 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c37664c2-a3e7-42a8-a1a1-334e8ac1618b tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:25:57.655540 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c37664c2-a3e7-42a8-a1a1-334e8ac1618b tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:25:57.656316 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c37664c2-a3e7-42a8-a1a1-334e8ac1618b tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:25:57.656756 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c37664c2-a3e7-42a8-a1a1-334e8ac1618b tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:25:57.924044 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-c37664c2-a3e7-42a8-a1a1-334e8ac1618b tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] 10.222.0.16 "POST /compute/v2.1/servers/4f08056e-5402-4d5a-88e3-c561d4ef9160/action" status: 202 len: 0 microversion: 2.1 time: 0.281243 Jul 16 22:25:57.924825 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3575/7153] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 22:25:57 2022] POST /compute/v2.1/servers/4f08056e-5402-4d5a-88e3-c561d4ef9160/action => generated 0 bytes in 283 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 22:25:57.933794 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-b6b2f274-9bc9-4d03-b360-40c6161bd72a tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:25:57.934453 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-b6b2f274-9bc9-4d03-b360-40c6161bd72a tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] [instance: 4f08056e-5402-4d5a-88e3-c561d4ef9160] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:25:57.940659 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b6b2f274-9bc9-4d03-b360-40c6161bd72a tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:25:57.941170 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b6b2f274-9bc9-4d03-b360-40c6161bd72a tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:25:57.941958 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b6b2f274-9bc9-4d03-b360-40c6161bd72a tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:25:57.942404 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-b6b2f274-9bc9-4d03-b360-40c6161bd72a tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:25:58.089753 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-b6b2f274-9bc9-4d03-b360-40c6161bd72a tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '45b6f8ac55464fc99331221b6c67a85d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1d60fe3fdc841998824b803809f9e49', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:25:58.097536 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-b6b2f274-9bc9-4d03-b360-40c6161bd72a tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] 10.222.0.16 "GET /compute/v2.1/servers/4f08056e-5402-4d5a-88e3-c561d4ef9160" status: 200 len: 1629 microversion: 2.1 time: 0.168307 Jul 16 22:25:58.098171 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3579/7154] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:25:57 2022] GET /compute/v2.1/servers/4f08056e-5402-4d5a-88e3-c561d4ef9160 => generated 1629 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:25:58.288323 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-8d6a5718-65f0-4f44-b66e-2a5c10a4f165 req-ad462925-adcc-4731-b6d2-4a7fe1cc1353 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "4f08056e-5402-4d5a-88e3-c561d4ef9160", "name": "network-vif-unplugged", "status": "completed", "tag": "e1ce8fbe-b51a-4c42-9649-f0936b53cf4b"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:25:58.297166 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-8d6a5718-65f0-4f44-b66e-2a5c10a4f165 req-ad462925-adcc-4731-b6d2-4a7fe1cc1353 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:25:58.297617 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-8d6a5718-65f0-4f44-b66e-2a5c10a4f165 req-ad462925-adcc-4731-b6d2-4a7fe1cc1353 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:25:58.309513 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.compute.server_external_events [req-8d6a5718-65f0-4f44-b66e-2a5c10a4f165 req-ad462925-adcc-4731-b6d2-4a7fe1cc1353 service nova] Unable to find a host for instance 4f08056e-5402-4d5a-88e3-c561d4ef9160. Dropping event network-vif-unplugged {{(pid=138680) create /opt/stack/nova/nova/api/openstack/compute/server_external_events.py:143}} Jul 16 22:25:58.310388 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-8d6a5718-65f0-4f44-b66e-2a5c10a4f165 req-ad462925-adcc-4731-b6d2-4a7fe1cc1353 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 207 len: 182 microversion: 2.1 time: 0.026052 Jul 16 22:25:58.310951 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3576/7155] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 22:25:58 2022] POST /compute/v2.1/os-server-external-events => generated 182 bytes in 27 msecs (HTTP/1.1 207) 9 headers in 367 bytes (2 switches on core 0) Jul 16 22:25:59.117643 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-c93c7862-52a3-4b59-8d81-99dd853a3673 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:25:59.118283 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-c93c7862-52a3-4b59-8d81-99dd853a3673 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] [instance: 4f08056e-5402-4d5a-88e3-c561d4ef9160] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:25:59.125741 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c93c7862-52a3-4b59-8d81-99dd853a3673 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:25:59.126210 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c93c7862-52a3-4b59-8d81-99dd853a3673 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:25:59.127075 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c93c7862-52a3-4b59-8d81-99dd853a3673 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:25:59.127530 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c93c7862-52a3-4b59-8d81-99dd853a3673 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:25:59.265159 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-c93c7862-52a3-4b59-8d81-99dd853a3673 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '45b6f8ac55464fc99331221b6c67a85d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1d60fe3fdc841998824b803809f9e49', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:25:59.273089 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-c93c7862-52a3-4b59-8d81-99dd853a3673 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] 10.222.0.16 "GET /compute/v2.1/servers/4f08056e-5402-4d5a-88e3-c561d4ef9160" status: 200 len: 1629 microversion: 2.1 time: 0.161036 Jul 16 22:25:59.273673 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3580/7156] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:25:59 2022] GET /compute/v2.1/servers/4f08056e-5402-4d5a-88e3-c561d4ef9160 => generated 1629 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:26:00.291087 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-97242517-5c0a-4436-bf6f-ee0d9ed2f8c1 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:26:00.291716 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-97242517-5c0a-4436-bf6f-ee0d9ed2f8c1 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] [instance: 4f08056e-5402-4d5a-88e3-c561d4ef9160] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:26:00.299531 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-97242517-5c0a-4436-bf6f-ee0d9ed2f8c1 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:26:00.299993 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-97242517-5c0a-4436-bf6f-ee0d9ed2f8c1 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:26:00.307369 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-97242517-5c0a-4436-bf6f-ee0d9ed2f8c1 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:26:00.307369 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-97242517-5c0a-4436-bf6f-ee0d9ed2f8c1 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:26:00.447339 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-97242517-5c0a-4436-bf6f-ee0d9ed2f8c1 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '45b6f8ac55464fc99331221b6c67a85d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1d60fe3fdc841998824b803809f9e49', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:26:00.456222 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-97242517-5c0a-4436-bf6f-ee0d9ed2f8c1 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] 10.222.0.16 "GET /compute/v2.1/servers/4f08056e-5402-4d5a-88e3-c561d4ef9160" status: 200 len: 1687 microversion: 2.1 time: 0.169340 Jul 16 22:26:00.469322 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3577/7157] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:26:00 2022] GET /compute/v2.1/servers/4f08056e-5402-4d5a-88e3-c561d4ef9160 => generated 1687 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:26:01.475777 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-714487b8-d4f2-49de-bd4e-d5d59a33d44f tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:26:01.476572 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-714487b8-d4f2-49de-bd4e-d5d59a33d44f tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] [instance: 4f08056e-5402-4d5a-88e3-c561d4ef9160] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:26:01.480555 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_api:SELECT=31 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:26:01.486323 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-714487b8-d4f2-49de-bd4e-d5d59a33d44f tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:26:01.486942 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-714487b8-d4f2-49de-bd4e-d5d59a33d44f tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:26:01.487868 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-714487b8-d4f2-49de-bd4e-d5d59a33d44f tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:26:01.488450 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-714487b8-d4f2-49de-bd4e-d5d59a33d44f tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:26:01.491669 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=96,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:26:01.641986 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-714487b8-d4f2-49de-bd4e-d5d59a33d44f tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '45b6f8ac55464fc99331221b6c67a85d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1d60fe3fdc841998824b803809f9e49', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:26:01.650153 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-714487b8-d4f2-49de-bd4e-d5d59a33d44f tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] 10.222.0.16 "GET /compute/v2.1/servers/4f08056e-5402-4d5a-88e3-c561d4ef9160" status: 200 len: 1687 microversion: 2.1 time: 0.178125 Jul 16 22:26:01.650740 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3581/7158] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:26:01 2022] GET /compute/v2.1/servers/4f08056e-5402-4d5a-88e3-c561d4ef9160 => generated 1687 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:26:02.669198 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-34f6340e-148e-4060-ad17-06a42a0c41b1 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:26:02.669921 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-34f6340e-148e-4060-ad17-06a42a0c41b1 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] [instance: 4f08056e-5402-4d5a-88e3-c561d4ef9160] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:26:02.672886 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_api:SELECT=33 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:26:02.677948 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-34f6340e-148e-4060-ad17-06a42a0c41b1 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:26:02.678735 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-34f6340e-148e-4060-ad17-06a42a0c41b1 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:26:02.679634 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-34f6340e-148e-4060-ad17-06a42a0c41b1 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:26:02.680163 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-34f6340e-148e-4060-ad17-06a42a0c41b1 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:26:02.682996 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=91,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:26:02.835233 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-34f6340e-148e-4060-ad17-06a42a0c41b1 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '45b6f8ac55464fc99331221b6c67a85d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1d60fe3fdc841998824b803809f9e49', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:26:02.844402 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-34f6340e-148e-4060-ad17-06a42a0c41b1 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] 10.222.0.16 "GET /compute/v2.1/servers/4f08056e-5402-4d5a-88e3-c561d4ef9160" status: 200 len: 1687 microversion: 2.1 time: 0.178592 Jul 16 22:26:02.854715 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3578/7159] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:26:02 2022] GET /compute/v2.1/servers/4f08056e-5402-4d5a-88e3-c561d4ef9160 => generated 1687 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:26:03.862690 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-fcad3656-8ff9-43d7-a1a0-43d9f18594d9 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:26:03.863364 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-fcad3656-8ff9-43d7-a1a0-43d9f18594d9 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] [instance: 4f08056e-5402-4d5a-88e3-c561d4ef9160] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:26:03.870716 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fcad3656-8ff9-43d7-a1a0-43d9f18594d9 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:26:03.871227 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fcad3656-8ff9-43d7-a1a0-43d9f18594d9 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:26:03.872018 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fcad3656-8ff9-43d7-a1a0-43d9f18594d9 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:26:03.872470 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fcad3656-8ff9-43d7-a1a0-43d9f18594d9 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:26:04.025294 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-fcad3656-8ff9-43d7-a1a0-43d9f18594d9 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '45b6f8ac55464fc99331221b6c67a85d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1d60fe3fdc841998824b803809f9e49', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:26:04.039103 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-fcad3656-8ff9-43d7-a1a0-43d9f18594d9 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] 10.222.0.16 "GET /compute/v2.1/servers/4f08056e-5402-4d5a-88e3-c561d4ef9160" status: 200 len: 1687 microversion: 2.1 time: 0.178244 Jul 16 22:26:04.051798 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3582/7160] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:26:03 2022] GET /compute/v2.1/servers/4f08056e-5402-4d5a-88e3-c561d4ef9160 => generated 1687 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:26:05.058016 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-2d999762-e444-4ccc-af5f-cee541af2bba tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:26:05.058650 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-2d999762-e444-4ccc-af5f-cee541af2bba tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] [instance: 4f08056e-5402-4d5a-88e3-c561d4ef9160] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:26:05.067363 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2d999762-e444-4ccc-af5f-cee541af2bba tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:26:05.067819 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2d999762-e444-4ccc-af5f-cee541af2bba tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:26:05.068598 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2d999762-e444-4ccc-af5f-cee541af2bba tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:26:05.069035 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-2d999762-e444-4ccc-af5f-cee541af2bba tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:26:05.228501 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-2d999762-e444-4ccc-af5f-cee541af2bba tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '45b6f8ac55464fc99331221b6c67a85d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1d60fe3fdc841998824b803809f9e49', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:26:05.237024 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-2d999762-e444-4ccc-af5f-cee541af2bba tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] 10.222.0.16 "GET /compute/v2.1/servers/4f08056e-5402-4d5a-88e3-c561d4ef9160" status: 200 len: 1687 microversion: 2.1 time: 0.182768 Jul 16 22:26:05.247165 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3579/7161] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:26:05 2022] GET /compute/v2.1/servers/4f08056e-5402-4d5a-88e3-c561d4ef9160 => generated 1687 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:26:06.254147 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-9c44a1d9-34f0-46c6-91d0-34e4f598dcf2 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:26:06.254760 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-9c44a1d9-34f0-46c6-91d0-34e4f598dcf2 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] [instance: 4f08056e-5402-4d5a-88e3-c561d4ef9160] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:26:06.261982 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9c44a1d9-34f0-46c6-91d0-34e4f598dcf2 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:26:06.262458 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9c44a1d9-34f0-46c6-91d0-34e4f598dcf2 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:26:06.263338 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9c44a1d9-34f0-46c6-91d0-34e4f598dcf2 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:26:06.263754 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-9c44a1d9-34f0-46c6-91d0-34e4f598dcf2 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:26:06.414373 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-9c44a1d9-34f0-46c6-91d0-34e4f598dcf2 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '45b6f8ac55464fc99331221b6c67a85d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1d60fe3fdc841998824b803809f9e49', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:26:06.422386 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-9c44a1d9-34f0-46c6-91d0-34e4f598dcf2 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] 10.222.0.16 "GET /compute/v2.1/servers/4f08056e-5402-4d5a-88e3-c561d4ef9160" status: 200 len: 1687 microversion: 2.1 time: 0.172485 Jul 16 22:26:06.427060 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3583/7162] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:26:06 2022] GET /compute/v2.1/servers/4f08056e-5402-4d5a-88e3-c561d4ef9160 => generated 1687 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:26:07.441057 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-fa6cb705-50fd-4415-a76a-c9c53b5111e7 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:26:07.441716 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-fa6cb705-50fd-4415-a76a-c9c53b5111e7 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] [instance: 4f08056e-5402-4d5a-88e3-c561d4ef9160] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:26:07.448562 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-fa6cb705-50fd-4415-a76a-c9c53b5111e7 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:26:07.449052 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-fa6cb705-50fd-4415-a76a-c9c53b5111e7 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:26:07.449809 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-fa6cb705-50fd-4415-a76a-c9c53b5111e7 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:26:07.450249 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-fa6cb705-50fd-4415-a76a-c9c53b5111e7 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:26:07.593964 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-fa6cb705-50fd-4415-a76a-c9c53b5111e7 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '45b6f8ac55464fc99331221b6c67a85d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1d60fe3fdc841998824b803809f9e49', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:26:07.604056 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-fa6cb705-50fd-4415-a76a-c9c53b5111e7 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] 10.222.0.16 "GET /compute/v2.1/servers/4f08056e-5402-4d5a-88e3-c561d4ef9160" status: 200 len: 1687 microversion: 2.1 time: 0.166503 Jul 16 22:26:07.604056 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3580/7163] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:26:07 2022] GET /compute/v2.1/servers/4f08056e-5402-4d5a-88e3-c561d4ef9160 => generated 1687 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:26:08.622073 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-60fb5e07-242d-4fa6-89cb-6e0a843e93cd tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:26:08.623399 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-60fb5e07-242d-4fa6-89cb-6e0a843e93cd tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] [instance: 4f08056e-5402-4d5a-88e3-c561d4ef9160] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:26:08.643045 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-60fb5e07-242d-4fa6-89cb-6e0a843e93cd tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:26:08.643045 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-60fb5e07-242d-4fa6-89cb-6e0a843e93cd tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:26:08.643045 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-60fb5e07-242d-4fa6-89cb-6e0a843e93cd tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:26:08.643045 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-60fb5e07-242d-4fa6-89cb-6e0a843e93cd tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:26:08.782086 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-60fb5e07-242d-4fa6-89cb-6e0a843e93cd tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '45b6f8ac55464fc99331221b6c67a85d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1d60fe3fdc841998824b803809f9e49', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:26:08.790284 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-60fb5e07-242d-4fa6-89cb-6e0a843e93cd tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] 10.222.0.16 "GET /compute/v2.1/servers/4f08056e-5402-4d5a-88e3-c561d4ef9160" status: 200 len: 1687 microversion: 2.1 time: 0.172362 Jul 16 22:26:08.790889 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3584/7164] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:26:08 2022] GET /compute/v2.1/servers/4f08056e-5402-4d5a-88e3-c561d4ef9160 => generated 1687 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:26:09.807387 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-c7c8823f-a85d-46c8-9df7-aee157fd621e tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:26:09.808031 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-c7c8823f-a85d-46c8-9df7-aee157fd621e tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] [instance: 4f08056e-5402-4d5a-88e3-c561d4ef9160] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:26:09.816129 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c7c8823f-a85d-46c8-9df7-aee157fd621e tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:26:09.816592 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c7c8823f-a85d-46c8-9df7-aee157fd621e tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:26:09.817371 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c7c8823f-a85d-46c8-9df7-aee157fd621e tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:26:09.817815 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c7c8823f-a85d-46c8-9df7-aee157fd621e tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:26:09.955295 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-c7c8823f-a85d-46c8-9df7-aee157fd621e tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '45b6f8ac55464fc99331221b6c67a85d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1d60fe3fdc841998824b803809f9e49', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:26:09.962924 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-c7c8823f-a85d-46c8-9df7-aee157fd621e tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] 10.222.0.16 "GET /compute/v2.1/servers/4f08056e-5402-4d5a-88e3-c561d4ef9160" status: 200 len: 1687 microversion: 2.1 time: 0.159764 Jul 16 22:26:09.963544 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3581/7165] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:26:09 2022] GET /compute/v2.1/servers/4f08056e-5402-4d5a-88e3-c561d4ef9160 => generated 1687 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:26:10.979877 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-8a4d7f64-df25-43a2-95fa-292a8e2a8426 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:26:10.980488 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-8a4d7f64-df25-43a2-95fa-292a8e2a8426 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] [instance: 4f08056e-5402-4d5a-88e3-c561d4ef9160] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:26:10.987718 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8a4d7f64-df25-43a2-95fa-292a8e2a8426 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:26:10.988181 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8a4d7f64-df25-43a2-95fa-292a8e2a8426 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:26:10.988957 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8a4d7f64-df25-43a2-95fa-292a8e2a8426 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:26:10.989405 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-8a4d7f64-df25-43a2-95fa-292a8e2a8426 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:26:11.124752 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-8a4d7f64-df25-43a2-95fa-292a8e2a8426 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '45b6f8ac55464fc99331221b6c67a85d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1d60fe3fdc841998824b803809f9e49', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:26:11.133019 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-8a4d7f64-df25-43a2-95fa-292a8e2a8426 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] 10.222.0.16 "GET /compute/v2.1/servers/4f08056e-5402-4d5a-88e3-c561d4ef9160" status: 200 len: 1687 microversion: 2.1 time: 0.157097 Jul 16 22:26:11.133591 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3585/7166] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:26:10 2022] GET /compute/v2.1/servers/4f08056e-5402-4d5a-88e3-c561d4ef9160 => generated 1687 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:26:12.151197 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-41ebef21-a60d-4b4d-b858-3511753d3d05 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:26:12.151802 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-41ebef21-a60d-4b4d-b858-3511753d3d05 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] [instance: 4f08056e-5402-4d5a-88e3-c561d4ef9160] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:26:12.159103 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-41ebef21-a60d-4b4d-b858-3511753d3d05 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:26:12.159569 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-41ebef21-a60d-4b4d-b858-3511753d3d05 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:26:12.160399 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-41ebef21-a60d-4b4d-b858-3511753d3d05 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:26:12.160846 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-41ebef21-a60d-4b4d-b858-3511753d3d05 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:26:12.309153 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-41ebef21-a60d-4b4d-b858-3511753d3d05 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '45b6f8ac55464fc99331221b6c67a85d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1d60fe3fdc841998824b803809f9e49', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:26:12.315277 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-41ebef21-a60d-4b4d-b858-3511753d3d05 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] 10.222.0.16 "GET /compute/v2.1/servers/4f08056e-5402-4d5a-88e3-c561d4ef9160" status: 200 len: 1687 microversion: 2.1 time: 0.168116 Jul 16 22:26:12.315866 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3582/7167] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:26:12 2022] GET /compute/v2.1/servers/4f08056e-5402-4d5a-88e3-c561d4ef9160 => generated 1687 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:26:13.332620 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-ba4ff851-bce5-405b-9da9-347c839a7660 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:26:13.333248 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-ba4ff851-bce5-405b-9da9-347c839a7660 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] [instance: 4f08056e-5402-4d5a-88e3-c561d4ef9160] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:26:13.340259 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ba4ff851-bce5-405b-9da9-347c839a7660 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:26:13.340718 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ba4ff851-bce5-405b-9da9-347c839a7660 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:26:13.341489 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ba4ff851-bce5-405b-9da9-347c839a7660 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:26:13.341935 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ba4ff851-bce5-405b-9da9-347c839a7660 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:26:13.475714 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-ba4ff851-bce5-405b-9da9-347c839a7660 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '45b6f8ac55464fc99331221b6c67a85d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1d60fe3fdc841998824b803809f9e49', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:26:13.483263 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-ba4ff851-bce5-405b-9da9-347c839a7660 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] 10.222.0.16 "GET /compute/v2.1/servers/4f08056e-5402-4d5a-88e3-c561d4ef9160" status: 200 len: 1687 microversion: 2.1 time: 0.154869 Jul 16 22:26:13.483977 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3586/7168] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:26:13 2022] GET /compute/v2.1/servers/4f08056e-5402-4d5a-88e3-c561d4ef9160 => generated 1687 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:26:14.451133 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-e1038f2b-bf0a-4df6-a87e-912b2d88af31 req-1390e35c-91d1-4973-9c9e-af138121561c service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "4f08056e-5402-4d5a-88e3-c561d4ef9160", "name": "network-vif-plugged", "status": "completed", "tag": "e1ce8fbe-b51a-4c42-9649-f0936b53cf4b"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:26:14.461579 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-e1038f2b-bf0a-4df6-a87e-912b2d88af31 req-1390e35c-91d1-4973-9c9e-af138121561c service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:26:14.465997 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-e1038f2b-bf0a-4df6-a87e-912b2d88af31 req-1390e35c-91d1-4973-9c9e-af138121561c service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:26:14.474710 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-e1038f2b-bf0a-4df6-a87e-912b2d88af31 req-1390e35c-91d1-4973-9c9e-af138121561c service nova] Creating event network-vif-plugged:e1ce8fbe-b51a-4c42-9649-f0936b53cf4b for instance 4f08056e-5402-4d5a-88e3-c561d4ef9160 on nch1-master-1 Jul 16 22:26:14.479923 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-e1038f2b-bf0a-4df6-a87e-912b2d88af31 req-1390e35c-91d1-4973-9c9e-af138121561c service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.032596 Jul 16 22:26:14.480525 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3583/7169] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 22:26:14 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 22:26:14.501284 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-656bb382-425e-472d-9212-706d6fccfce9 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:26:14.502281 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-656bb382-425e-472d-9212-706d6fccfce9 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] [instance: 4f08056e-5402-4d5a-88e3-c561d4ef9160] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:26:14.510033 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-656bb382-425e-472d-9212-706d6fccfce9 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:26:14.510505 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-656bb382-425e-472d-9212-706d6fccfce9 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:26:14.511334 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-656bb382-425e-472d-9212-706d6fccfce9 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:26:14.511792 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-656bb382-425e-472d-9212-706d6fccfce9 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:26:14.661348 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-656bb382-425e-472d-9212-706d6fccfce9 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '45b6f8ac55464fc99331221b6c67a85d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1d60fe3fdc841998824b803809f9e49', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:26:14.670054 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-656bb382-425e-472d-9212-706d6fccfce9 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] 10.222.0.16 "GET /compute/v2.1/servers/4f08056e-5402-4d5a-88e3-c561d4ef9160" status: 200 len: 1687 microversion: 2.1 time: 0.172507 Jul 16 22:26:14.671063 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3587/7170] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:26:14 2022] GET /compute/v2.1/servers/4f08056e-5402-4d5a-88e3-c561d4ef9160 => generated 1687 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:26:15.689681 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-05776545-6ec8-4f4d-9ed5-a9de4b14c945 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:26:15.691047 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-05776545-6ec8-4f4d-9ed5-a9de4b14c945 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] [instance: 4f08056e-5402-4d5a-88e3-c561d4ef9160] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:26:15.697651 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-05776545-6ec8-4f4d-9ed5-a9de4b14c945 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:26:15.698106 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-05776545-6ec8-4f4d-9ed5-a9de4b14c945 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:26:15.698874 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-05776545-6ec8-4f4d-9ed5-a9de4b14c945 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:26:15.699356 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-05776545-6ec8-4f4d-9ed5-a9de4b14c945 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:26:15.854926 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-05776545-6ec8-4f4d-9ed5-a9de4b14c945 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '45b6f8ac55464fc99331221b6c67a85d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1d60fe3fdc841998824b803809f9e49', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:26:15.863894 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-05776545-6ec8-4f4d-9ed5-a9de4b14c945 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] 10.222.0.16 "GET /compute/v2.1/servers/4f08056e-5402-4d5a-88e3-c561d4ef9160" status: 200 len: 1687 microversion: 2.1 time: 0.179420 Jul 16 22:26:15.864487 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3584/7171] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:26:15 2022] GET /compute/v2.1/servers/4f08056e-5402-4d5a-88e3-c561d4ef9160 => generated 1687 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:26:16.880864 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-36993ad9-5d13-4ff6-86bd-49c1090142f5 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:26:16.881429 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-36993ad9-5d13-4ff6-86bd-49c1090142f5 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] [instance: 4f08056e-5402-4d5a-88e3-c561d4ef9160] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:26:16.888286 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-36993ad9-5d13-4ff6-86bd-49c1090142f5 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:26:16.888792 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-36993ad9-5d13-4ff6-86bd-49c1090142f5 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:26:16.889577 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-36993ad9-5d13-4ff6-86bd-49c1090142f5 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:26:16.890071 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-36993ad9-5d13-4ff6-86bd-49c1090142f5 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:26:17.029453 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-36993ad9-5d13-4ff6-86bd-49c1090142f5 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '45b6f8ac55464fc99331221b6c67a85d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1d60fe3fdc841998824b803809f9e49', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:26:17.037962 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-36993ad9-5d13-4ff6-86bd-49c1090142f5 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] 10.222.0.16 "GET /compute/v2.1/servers/4f08056e-5402-4d5a-88e3-c561d4ef9160" status: 200 len: 1687 microversion: 2.1 time: 0.161340 Jul 16 22:26:17.038539 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3588/7172] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:26:16 2022] GET /compute/v2.1/servers/4f08056e-5402-4d5a-88e3-c561d4ef9160 => generated 1687 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:26:18.055049 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-a458b6ce-8d81-4739-ad35-dbefd94590e9 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:26:18.055707 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-a458b6ce-8d81-4739-ad35-dbefd94590e9 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] [instance: 4f08056e-5402-4d5a-88e3-c561d4ef9160] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:26:18.062733 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a458b6ce-8d81-4739-ad35-dbefd94590e9 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:26:18.063727 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a458b6ce-8d81-4739-ad35-dbefd94590e9 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:26:18.064596 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a458b6ce-8d81-4739-ad35-dbefd94590e9 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:26:18.065047 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a458b6ce-8d81-4739-ad35-dbefd94590e9 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:26:18.199027 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-a458b6ce-8d81-4739-ad35-dbefd94590e9 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '45b6f8ac55464fc99331221b6c67a85d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1d60fe3fdc841998824b803809f9e49', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:26:18.206944 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-a458b6ce-8d81-4739-ad35-dbefd94590e9 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] 10.222.0.16 "GET /compute/v2.1/servers/4f08056e-5402-4d5a-88e3-c561d4ef9160" status: 200 len: 1674 microversion: 2.1 time: 0.156067 Jul 16 22:26:18.207542 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3585/7173] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:26:18 2022] GET /compute/v2.1/servers/4f08056e-5402-4d5a-88e3-c561d4ef9160 => generated 1674 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:26:24.810580 ncdv-master-1 devstack@n-api.service[138680]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jul 16 22:26:24.811148 ncdv-master-1 devstack@n-api.service[138680]: Sat Jul 16 22:26:24 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Jul 16 22:26:26.907270 ncdv-master-1 devstack@n-api.service[138681]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jul 16 22:26:26.907746 ncdv-master-1 devstack@n-api.service[138681]: Sat Jul 16 22:26:26 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Jul 16 22:27:30.059742 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_api:SELECT=15 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:27:30.060440 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=47 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:27:30.061484 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_api:SELECT=15 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:27:30.062266 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=43 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:27:30.064662 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-7c98d929-9876-41dc-be86-8c33ad2d2d30 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:27:30.065891 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-7c98d929-9876-41dc-be86-8c33ad2d2d30 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] [instance: 4f08056e-5402-4d5a-88e3-c561d4ef9160] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:27:30.067264 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-439ea77e-f25b-4128-bf9d-e36fb5f0f55a req-5561981c-a085-4c2c-8652-51255fdb0b3a service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "4f08056e-5402-4d5a-88e3-c561d4ef9160", "tag": "e1ce8fbe-b51a-4c42-9649-f0936b53cf4b"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:27:30.072825 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7c98d929-9876-41dc-be86-8c33ad2d2d30 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:27:30.073866 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7c98d929-9876-41dc-be86-8c33ad2d2d30 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:27:30.075093 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7c98d929-9876-41dc-be86-8c33ad2d2d30 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:27:30.076253 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7c98d929-9876-41dc-be86-8c33ad2d2d30 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:27:30.076406 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-439ea77e-f25b-4128-bf9d-e36fb5f0f55a req-5561981c-a085-4c2c-8652-51255fdb0b3a service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:27:30.076712 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-439ea77e-f25b-4128-bf9d-e36fb5f0f55a req-5561981c-a085-4c2c-8652-51255fdb0b3a service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:27:30.089666 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-439ea77e-f25b-4128-bf9d-e36fb5f0f55a req-5561981c-a085-4c2c-8652-51255fdb0b3a service nova] Creating event network-changed:e1ce8fbe-b51a-4c42-9649-f0936b53cf4b for instance 4f08056e-5402-4d5a-88e3-c561d4ef9160 on nch1-master-1 Jul 16 22:27:30.095424 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-439ea77e-f25b-4128-bf9d-e36fb5f0f55a req-5561981c-a085-4c2c-8652-51255fdb0b3a service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.035519 Jul 16 22:27:30.096435 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3586/7174] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 22:27:30 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 22:27:30.112849 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-7c98d929-9876-41dc-be86-8c33ad2d2d30 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] [instance: 4f08056e-5402-4d5a-88e3-c561d4ef9160] Going to try to terminate instance {{(pid=138681) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 16 22:27:30.351051 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-7c98d929-9876-41dc-be86-8c33ad2d2d30 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] 10.222.0.16 "DELETE /compute/v2.1/servers/4f08056e-5402-4d5a-88e3-c561d4ef9160" status: 204 len: 0 microversion: 2.1 time: 0.293928 Jul 16 22:27:30.351683 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3589/7175] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 22:27:30 2022] DELETE /compute/v2.1/servers/4f08056e-5402-4d5a-88e3-c561d4ef9160 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 16 22:27:30.363533 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-cf3184d1-fb9b-44b3-8c9f-f25daa12614a tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:27:30.364793 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-cf3184d1-fb9b-44b3-8c9f-f25daa12614a tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] [instance: 4f08056e-5402-4d5a-88e3-c561d4ef9160] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:27:30.371804 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cf3184d1-fb9b-44b3-8c9f-f25daa12614a tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:27:30.372270 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cf3184d1-fb9b-44b3-8c9f-f25daa12614a tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:27:30.373057 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cf3184d1-fb9b-44b3-8c9f-f25daa12614a tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:27:30.373509 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cf3184d1-fb9b-44b3-8c9f-f25daa12614a tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:27:30.566541 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-cf3184d1-fb9b-44b3-8c9f-f25daa12614a tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '45b6f8ac55464fc99331221b6c67a85d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1d60fe3fdc841998824b803809f9e49', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:27:30.574427 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-cf3184d1-fb9b-44b3-8c9f-f25daa12614a tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] 10.222.0.16 "GET /compute/v2.1/servers/4f08056e-5402-4d5a-88e3-c561d4ef9160" status: 200 len: 1680 microversion: 2.1 time: 0.218208 Jul 16 22:27:30.575035 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3587/7176] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:27:30 2022] GET /compute/v2.1/servers/4f08056e-5402-4d5a-88e3-c561d4ef9160 => generated 1680 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:27:31.600480 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-fc2dd799-8acf-4d29-9cfe-c090ab6a789c tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:27:31.601207 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-fc2dd799-8acf-4d29-9cfe-c090ab6a789c tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] [instance: 4f08056e-5402-4d5a-88e3-c561d4ef9160] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:27:31.608276 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fc2dd799-8acf-4d29-9cfe-c090ab6a789c tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:27:31.613945 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fc2dd799-8acf-4d29-9cfe-c090ab6a789c tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:27:31.613945 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fc2dd799-8acf-4d29-9cfe-c090ab6a789c tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:27:31.613945 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fc2dd799-8acf-4d29-9cfe-c090ab6a789c tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:27:31.810975 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-fc2dd799-8acf-4d29-9cfe-c090ab6a789c tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '45b6f8ac55464fc99331221b6c67a85d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1d60fe3fdc841998824b803809f9e49', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:27:31.819056 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-fc2dd799-8acf-4d29-9cfe-c090ab6a789c tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] 10.222.0.16 "GET /compute/v2.1/servers/4f08056e-5402-4d5a-88e3-c561d4ef9160" status: 200 len: 1680 microversion: 2.1 time: 0.223745 Jul 16 22:27:31.819856 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3590/7177] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:27:31 2022] GET /compute/v2.1/servers/4f08056e-5402-4d5a-88e3-c561d4ef9160 => generated 1680 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:27:32.152029 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-ded88cae-cced-4f15-a36a-c9171f2d10c9 req-a10cae9b-c98a-4553-9d77-230b69d43d6d service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "4f08056e-5402-4d5a-88e3-c561d4ef9160", "name": "network-vif-deleted", "tag": "e1ce8fbe-b51a-4c42-9649-f0936b53cf4b"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:27:32.161795 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-ded88cae-cced-4f15-a36a-c9171f2d10c9 req-a10cae9b-c98a-4553-9d77-230b69d43d6d service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:27:32.162986 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-ded88cae-cced-4f15-a36a-c9171f2d10c9 req-a10cae9b-c98a-4553-9d77-230b69d43d6d service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:27:32.177343 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-ded88cae-cced-4f15-a36a-c9171f2d10c9 req-a10cae9b-c98a-4553-9d77-230b69d43d6d service nova] Creating event network-vif-deleted:e1ce8fbe-b51a-4c42-9649-f0936b53cf4b for instance 4f08056e-5402-4d5a-88e3-c561d4ef9160 on nch1-master-1 Jul 16 22:27:32.182475 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-ded88cae-cced-4f15-a36a-c9171f2d10c9 req-a10cae9b-c98a-4553-9d77-230b69d43d6d service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.034212 Jul 16 22:27:32.189152 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3588/7178] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 22:27:32 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 22:27:32.838034 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-5e98bbdd-d3e5-4a28-b768-c593f76fb15d tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:27:32.838702 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-5e98bbdd-d3e5-4a28-b768-c593f76fb15d tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] [instance: 4f08056e-5402-4d5a-88e3-c561d4ef9160] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:27:32.846227 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5e98bbdd-d3e5-4a28-b768-c593f76fb15d tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:27:32.846702 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5e98bbdd-d3e5-4a28-b768-c593f76fb15d tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:27:32.848469 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5e98bbdd-d3e5-4a28-b768-c593f76fb15d tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:27:32.848921 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5e98bbdd-d3e5-4a28-b768-c593f76fb15d tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:27:32.883781 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5e98bbdd-d3e5-4a28-b768-c593f76fb15d tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:27:32.884286 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-5e98bbdd-d3e5-4a28-b768-c593f76fb15d tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:27:32.884774 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.objects.instance [None req-5e98bbdd-d3e5-4a28-b768-c593f76fb15d tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lazy-loading 'fault' on Instance uuid 4f08056e-5402-4d5a-88e3-c561d4ef9160 {{(pid=138681) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 22:27:32.926771 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-5e98bbdd-d3e5-4a28-b768-c593f76fb15d tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '45b6f8ac55464fc99331221b6c67a85d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1d60fe3fdc841998824b803809f9e49', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:27:32.934737 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-5e98bbdd-d3e5-4a28-b768-c593f76fb15d tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] 10.222.0.16 "GET /compute/v2.1/servers/4f08056e-5402-4d5a-88e3-c561d4ef9160" status: 200 len: 1336 microversion: 2.1 time: 0.101295 Jul 16 22:27:32.935354 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3591/7179] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:27:32 2022] GET /compute/v2.1/servers/4f08056e-5402-4d5a-88e3-c561d4ef9160 => generated 1336 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:27:33.952433 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-14f9aead-58eb-43f0-9322-34e7c76c953b tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:27:33.953151 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-14f9aead-58eb-43f0-9322-34e7c76c953b tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] [instance: 4f08056e-5402-4d5a-88e3-c561d4ef9160] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:27:33.960914 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-14f9aead-58eb-43f0-9322-34e7c76c953b tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:27:33.961374 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-14f9aead-58eb-43f0-9322-34e7c76c953b tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:27:33.962166 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-14f9aead-58eb-43f0-9322-34e7c76c953b tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:27:33.962605 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-14f9aead-58eb-43f0-9322-34e7c76c953b tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:27:33.976378 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.wsgi [None req-14f9aead-58eb-43f0-9322-34e7c76c953b tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] HTTP exception thrown: Instance 4f08056e-5402-4d5a-88e3-c561d4ef9160 could not be found. Jul 16 22:27:33.977975 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-14f9aead-58eb-43f0-9322-34e7c76c953b tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Request method failure captured: Jul 16 22:27:33.977975 ncdv-master-1 devstack@n-api.service[138680]: request: GET /compute/v2.1/servers/4f08056e-5402-4d5a-88e3-c561d4ef9160 HTTP/1.1 Jul 16 22:27:33.977975 ncdv-master-1 devstack@n-api.service[138680]: Accept: application/json Jul 16 22:27:33.977975 ncdv-master-1 devstack@n-api.service[138680]: Accept-Encoding: identity Jul 16 22:27:33.977975 ncdv-master-1 devstack@n-api.service[138680]: Connection: close Jul 16 22:27:33.977975 ncdv-master-1 devstack@n-api.service[138680]: Content-Length: 0 Jul 16 22:27:33.977975 ncdv-master-1 devstack@n-api.service[138680]: Content-Type: application/json Jul 16 22:27:33.977975 ncdv-master-1 devstack@n-api.service[138680]: Host: 10.222.0.16 Jul 16 22:27:33.977975 ncdv-master-1 devstack@n-api.service[138680]: Openstack-System-Scope: None Jul 16 22:27:33.977975 ncdv-master-1 devstack@n-api.service[138680]: User-Agent: python-urllib3/1.26.9 Jul 16 22:27:33.977975 ncdv-master-1 devstack@n-api.service[138680]: X-Auth-Token: gAAAAABi0zpPBdeAo_vWtVXYIi-bSOM-tiLa8OvUoyegMfchbuqV5mTfCQcK6IxjM8bhh6FRbRh1XKEgJ7doP6PFcG_RKQJx30U_p-OV9wOEn7wZR-2sYFOm4bFBgwpqdQDDW6gjHkjOuDHceK7lTEWp7k0jHu-PGFew0xD4wOdVBDnoW0YD768 Jul 16 22:27:33.977975 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Id: None Jul 16 22:27:33.977975 ncdv-master-1 devstack@n-api.service[138680]: X-Domain-Name: None Jul 16 22:27:33.977975 ncdv-master-1 devstack@n-api.service[138680]: X-Identity-Status: Confirmed Jul 16 22:27:33.977975 ncdv-master-1 devstack@n-api.service[138680]: X-Is-Admin-Project: True Jul 16 22:27:33.977975 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Id: default Jul 16 22:27:33.977975 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Domain-Name: Default Jul 16 22:27:33.977975 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Id: f1d60fe3fdc841998824b803809f9e49 Jul 16 22:27:33.977975 ncdv-master-1 devstack@n-api.service[138680]: X-Project-Name: tempest-TestShelveInstance-574938005 Jul 16 22:27:33.977975 ncdv-master-1 devstack@n-api.service[138680]: X-Role: member,reader Jul 16 22:27:33.977975 ncdv-master-1 devstack@n-api.service[138680]: X-Roles: member,reader Jul 16 22:27:33.977975 ncdv-master-1 devstack@n-api.service[138680]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/f1d60fe3fdc841998824b803809f9e49"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/f1d60fe3fdc841998824b803809f9e49"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_f1d60fe3fdc841998824b803809f9e49"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/f1d60fe3fdc841998824b803809f9e49"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 22:27:33.977975 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant: tempest-TestShelveInstance-574938005 Jul 16 22:27:33.977975 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Id: f1d60fe3fdc841998824b803809f9e49 Jul 16 22:27:33.977975 ncdv-master-1 devstack@n-api.service[138680]: X-Tenant-Name: tempest-TestShelveInstance-574938005 Jul 16 22:27:33.977975 ncdv-master-1 devstack@n-api.service[138680]: X-User: tempest-TestShelveInstance-574938005-project Jul 16 22:27:33.977975 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Id: default Jul 16 22:27:33.977975 ncdv-master-1 devstack@n-api.service[138680]: X-User-Domain-Name: Default Jul 16 22:27:33.977975 ncdv-master-1 devstack@n-api.service[138680]: X-User-Id: 45b6f8ac55464fc99331221b6c67a85d Jul 16 22:27:33.977975 ncdv-master-1 devstack@n-api.service[138680]: X-User-Name: tempest-TestShelveInstance-574938005-project Jul 16 22:27:33.977975 ncdv-master-1 devstack@n-api.service[138680]: method: > Jul 16 22:27:33.977975 ncdv-master-1 devstack@n-api.service[138680]: exception: Instance 4f08056e-5402-4d5a-88e3-c561d4ef9160 could not be found. Jul 16 22:27:33.977975 ncdv-master-1 devstack@n-api.service[138680]: action_args: {'id': '4f08056e-5402-4d5a-88e3-c561d4ef9160'} Jul 16 22:27:33.977975 ncdv-master-1 devstack@n-api.service[138680]: {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 22:27:33.977975 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 22:27:33.977975 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 22:27:33.977975 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 22:27:33.977975 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 22:27:33.977975 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 22:27:33.977975 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 22:27:33.977975 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 22:27:33.977975 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 22:27:33.977975 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 22:27:33.977975 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 4f08056e-5402-4d5a-88e3-c561d4ef9160 could not be found. Jul 16 22:27:33.977975 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 22:27:33.977975 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 22:27:33.977975 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 22:27:33.977975 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 22:27:33.977975 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 22:27:33.977975 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 22:27:33.977975 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 22:27:33.977975 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 22:27:33.977975 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 22:27:33.977975 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 22:27:33.977975 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 22:27:33.977975 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 22:27:33.977975 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 22:27:33.977975 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 22:27:33.977975 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 22:27:33.977975 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 22:27:33.977975 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 4f08056e-5402-4d5a-88e3-c561d4ef9160 could not be found. Jul 16 22:27:33.977975 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 22:27:33.977975 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 22:27:33.977975 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 22:27:33.983597 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 22:27:33.983597 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 22:27:33.983597 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 22:27:33.983597 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 22:27:33.983597 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 22:27:33.983597 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 4f08056e-5402-4d5a-88e3-c561d4ef9160 could not be found. Jul 16 22:27:33.983597 ncdv-master-1 devstack@n-api.service[138680]: ERROR nova.api.openstack.wsgi Jul 16 22:27:33.983597 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-14f9aead-58eb-43f0-9322-34e7c76c953b tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Returning 404 to user: Instance 4f08056e-5402-4d5a-88e3-c561d4ef9160 could not be found. {{(pid=138680) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 22:27:33.983597 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-14f9aead-58eb-43f0-9322-34e7c76c953b tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] 10.222.0.16 "GET /compute/v2.1/servers/4f08056e-5402-4d5a-88e3-c561d4ef9160" status: 404 len: 111 microversion: 2.1 time: 0.031297 Jul 16 22:27:33.983597 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3589/7180] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:27:33 2022] GET /compute/v2.1/servers/4f08056e-5402-4d5a-88e3-c561d4ef9160 => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 22:27:34.213091 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-4a5d3754-5b92-44ec-804e-a35a96a88c6e tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Calling method '.version_select at 0x7fdd9c238550>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:27:34.216406 ncdv-master-1 devstack@n-api.service[138681]: ERROR oslo.messaging._drivers.impl_rabbit [None req-4a5d3754-5b92-44ec-804e-a35a96a88c6e tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] [0cc6ec90-aeaa-4138-9056-1296518c79df] AMQP server on 10.222.0.16:5672 is unreachable: [Errno 104] Connection reset by peer. Trying again in 1 seconds.: ConnectionResetError: [Errno 104] Connection reset by peer Jul 16 22:27:35.224444 ncdv-master-1 devstack@n-api.service[138681]: INFO oslo.messaging._drivers.impl_rabbit [None req-4a5d3754-5b92-44ec-804e-a35a96a88c6e tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] [0cc6ec90-aeaa-4138-9056-1296518c79df] Reconnected to AMQP server on 10.222.0.16:5672 via [amqp] client with port 34294. Jul 16 22:27:35.300755 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-4a5d3754-5b92-44ec-804e-a35a96a88c6e tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] 10.222.0.16 "DELETE /compute/v2.1/os-keypairs/tempest-TestShelveInstance-1336585092" status: 202 len: 0 microversion: 2.1 time: 1.091492 Jul 16 22:27:35.301373 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3592/7181] 10.222.0.16 () {58 vars in 1253 bytes} [Sat Jul 16 22:27:34 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestShelveInstance-1336585092 => generated 0 bytes in 1094 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 22:27:35.326331 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-4e93a8ef-2c86-489d-a482-da1ea8e9fdd6 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Action: 'create', calling method: .version_select at 0x7fdd8fc498b0>, body: {"keypair": {"name": "tempest-TestShelveInstance-729410316", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBLBn2utFT9VfMwJZnBWZJ2P07IeTwPMxVddxHqmYu1kKyaIQyOYfR7Ajk53rvV9Q7X4B2u/jCYC82VzP7TracP4xxyVK9E6YWWuNkiNj4nk44MNHMK4Zl1B5Xn/cq7EwJg=="}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:27:35.341890 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-4e93a8ef-2c86-489d-a482-da1ea8e9fdd6 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Getting quotas for project f1d60fe3fdc841998824b803809f9e49. Resources: {'key_pairs'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 22:27:35.347993 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-4e93a8ef-2c86-489d-a482-da1ea8e9fdd6 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Getting quotas for user 45b6f8ac55464fc99331221b6c67a85d and project f1d60fe3fdc841998824b803809f9e49. Resources: {'key_pairs'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 22:27:35.415706 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-4e93a8ef-2c86-489d-a482-da1ea8e9fdd6 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] 10.222.0.16 "POST /compute/v2.1/os-keypairs" status: 200 len: 396 microversion: 2.1 time: 0.093393 Jul 16 22:27:35.416315 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3590/7182] 10.222.0.16 () {60 vars in 1158 bytes} [Sat Jul 16 22:27:35 2022] POST /compute/v2.1/os-keypairs => generated 396 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 22:27:41.298721 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-f6a519c7-c13a-4881-abac-6d623af8b809 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestShelveInstance-server-1222160494", "imageRef": "", "flavorRef": "452", "key_name": "tempest-TestShelveInstance-729410316", "security_groups": [{"name": "tempest-secgroup-smoke-352484793"}], "networks": [{"uuid": "b5314ebf-c1d9-472e-b991-557ed5cf9d11"}], "block_device_mapping_v2": [{"uuid": "51002b9e-3891-4493-ad02-66fefc771dc2", "source_type": "volume", "destination_type": "volume", "boot_index": 0, "delete_on_termination": true}]}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:27:41.409183 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.network.neutron [None req-f6a519c7-c13a-4881-abac-6d623af8b809 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] validate_networks() for [('b5314ebf-c1d9-472e-b991-557ed5cf9d11', None, None, None, None, None)] {{(pid=138681) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 16 22:27:41.832412 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-f6a519c7-c13a-4881-abac-6d623af8b809 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Flavor limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 16 22:27:41.833585 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-f6a519c7-c13a-4881-abac-6d623af8b809 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Image limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 16 22:27:41.834039 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-f6a519c7-c13a-4881-abac-6d623af8b809 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Flavor pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 16 22:27:41.834497 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-f6a519c7-c13a-4881-abac-6d623af8b809 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Image pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 16 22:27:41.834924 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-f6a519c7-c13a-4881-abac-6d623af8b809 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 16 22:27:41.844008 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f6a519c7-c13a-4881-abac-6d623af8b809 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:27:41.844475 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f6a519c7-c13a-4881-abac-6d623af8b809 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:27:41.845331 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f6a519c7-c13a-4881-abac-6d623af8b809 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:27:41.845775 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f6a519c7-c13a-4881-abac-6d623af8b809 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:27:41.862657 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-f6a519c7-c13a-4881-abac-6d623af8b809 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Getting quotas for project f1d60fe3fdc841998824b803809f9e49. Resources: {'ram', 'instances', 'cores'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 22:27:41.868153 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-f6a519c7-c13a-4881-abac-6d623af8b809 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Getting quotas for user 45b6f8ac55464fc99331221b6c67a85d and project f1d60fe3fdc841998824b803809f9e49. Resources: {'ram', 'instances', 'cores'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 22:27:41.925970 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-f6a519c7-c13a-4881-abac-6d623af8b809 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Going to run 1 instances... {{(pid=138681) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 16 22:27:41.928286 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-f6a519c7-c13a-4881-abac-6d623af8b809 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] [instance: 468c7efb-80de-4061-86c6-dd1eadbec954] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag=None,updated_at=,uuid=,volume_id='51002b9e-3891-4493-ad02-66fefc771dc2',volume_size=None,volume_type=None)] {{(pid=138681) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 16 22:27:42.227531 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-f6a519c7-c13a-4881-abac-6d623af8b809 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 405 microversion: 2.1 time: 0.933598 Jul 16 22:27:42.228269 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3593/7183] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 22:27:41 2022] POST /compute/v2.1/servers => generated 405 bytes in 935 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 16 22:27:42.238502 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-e20321df-29ea-4e90-8944-f881f78562bd tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:27:42.239697 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-e20321df-29ea-4e90-8944-f881f78562bd tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] [instance: 468c7efb-80de-4061-86c6-dd1eadbec954] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:27:42.297599 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-e20321df-29ea-4e90-8944-f881f78562bd tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '45b6f8ac55464fc99331221b6c67a85d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1d60fe3fdc841998824b803809f9e49', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:27:42.305879 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-e20321df-29ea-4e90-8944-f881f78562bd tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] 10.222.0.16 "GET /compute/v2.1/servers/468c7efb-80de-4061-86c6-dd1eadbec954" status: 200 len: 1088 microversion: 2.1 time: 0.071311 Jul 16 22:27:42.306449 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3591/7184] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:27:42 2022] GET /compute/v2.1/servers/468c7efb-80de-4061-86c6-dd1eadbec954 => generated 1088 bytes in 73 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:27:43.324382 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-59c99c4a-7a18-42e1-b2d1-e7e38c33bf73 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:27:43.324991 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-59c99c4a-7a18-42e1-b2d1-e7e38c33bf73 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] [instance: 468c7efb-80de-4061-86c6-dd1eadbec954] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:27:43.331349 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-59c99c4a-7a18-42e1-b2d1-e7e38c33bf73 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:27:43.331812 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-59c99c4a-7a18-42e1-b2d1-e7e38c33bf73 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:27:43.332608 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-59c99c4a-7a18-42e1-b2d1-e7e38c33bf73 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:27:43.333862 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-59c99c4a-7a18-42e1-b2d1-e7e38c33bf73 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:27:43.399258 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-59c99c4a-7a18-42e1-b2d1-e7e38c33bf73 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '45b6f8ac55464fc99331221b6c67a85d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1d60fe3fdc841998824b803809f9e49', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:27:43.407096 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-59c99c4a-7a18-42e1-b2d1-e7e38c33bf73 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] 10.222.0.16 "GET /compute/v2.1/servers/468c7efb-80de-4061-86c6-dd1eadbec954" status: 200 len: 1130 microversion: 2.1 time: 0.087634 Jul 16 22:27:43.407899 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3594/7185] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:27:43 2022] GET /compute/v2.1/servers/468c7efb-80de-4061-86c6-dd1eadbec954 => generated 1130 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 22:27:44.424955 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-afbf8508-209d-4e69-93a6-0a59fdbd71fb tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:27:44.426349 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-afbf8508-209d-4e69-93a6-0a59fdbd71fb tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] [instance: 468c7efb-80de-4061-86c6-dd1eadbec954] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:27:44.430054 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=12 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:27:44.436473 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-afbf8508-209d-4e69-93a6-0a59fdbd71fb tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:27:44.436950 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-afbf8508-209d-4e69-93a6-0a59fdbd71fb tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:27:44.437727 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-afbf8508-209d-4e69-93a6-0a59fdbd71fb tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:27:44.438169 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-afbf8508-209d-4e69-93a6-0a59fdbd71fb tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:27:44.503981 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-afbf8508-209d-4e69-93a6-0a59fdbd71fb tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '45b6f8ac55464fc99331221b6c67a85d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1d60fe3fdc841998824b803809f9e49', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:27:44.512270 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-afbf8508-209d-4e69-93a6-0a59fdbd71fb tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] 10.222.0.16 "GET /compute/v2.1/servers/468c7efb-80de-4061-86c6-dd1eadbec954" status: 200 len: 1204 microversion: 2.1 time: 0.091318 Jul 16 22:27:44.512969 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3592/7186] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:27:44 2022] GET /compute/v2.1/servers/468c7efb-80de-4061-86c6-dd1eadbec954 => generated 1204 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:27:45.530519 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-ec4d2e08-962a-4f08-acca-c3c3250f4666 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:27:45.531373 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-ec4d2e08-962a-4f08-acca-c3c3250f4666 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] [instance: 468c7efb-80de-4061-86c6-dd1eadbec954] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:27:45.539231 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ec4d2e08-962a-4f08-acca-c3c3250f4666 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:27:45.539708 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ec4d2e08-962a-4f08-acca-c3c3250f4666 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:27:45.542079 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ec4d2e08-962a-4f08-acca-c3c3250f4666 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:27:45.542079 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ec4d2e08-962a-4f08-acca-c3c3250f4666 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:27:45.691186 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-ec4d2e08-962a-4f08-acca-c3c3250f4666 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '45b6f8ac55464fc99331221b6c67a85d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1d60fe3fdc841998824b803809f9e49', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:27:45.699580 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-ec4d2e08-962a-4f08-acca-c3c3250f4666 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] 10.222.0.16 "GET /compute/v2.1/servers/468c7efb-80de-4061-86c6-dd1eadbec954" status: 200 len: 1271 microversion: 2.1 time: 0.172990 Jul 16 22:27:45.700198 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3595/7187] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:27:45 2022] GET /compute/v2.1/servers/468c7efb-80de-4061-86c6-dd1eadbec954 => generated 1271 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:27:46.233163 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-6e66eb02-4e80-4d63-9aa1-c6bd42d081f0 req-fe1d2d8e-e99c-4ed1-9754-c5243a9523ff service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "468c7efb-80de-4061-86c6-dd1eadbec954", "tag": "2b47bb9a-6a79-4e7e-a496-20c05d58d97f"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:27:46.246546 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-6e66eb02-4e80-4d63-9aa1-c6bd42d081f0 req-fe1d2d8e-e99c-4ed1-9754-c5243a9523ff service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:27:46.248468 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-6e66eb02-4e80-4d63-9aa1-c6bd42d081f0 req-fe1d2d8e-e99c-4ed1-9754-c5243a9523ff service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:27:46.276729 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-6e66eb02-4e80-4d63-9aa1-c6bd42d081f0 req-fe1d2d8e-e99c-4ed1-9754-c5243a9523ff service nova] Creating event network-changed:2b47bb9a-6a79-4e7e-a496-20c05d58d97f for instance 468c7efb-80de-4061-86c6-dd1eadbec954 on nch2-master-1 Jul 16 22:27:46.281851 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-6e66eb02-4e80-4d63-9aa1-c6bd42d081f0 req-fe1d2d8e-e99c-4ed1-9754-c5243a9523ff service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.056303 Jul 16 22:27:46.282531 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3593/7188] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 22:27:46 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 22:27:46.716810 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-1d3d0bcb-1ba5-4c6c-81d1-ac9d40c36282 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:27:46.718014 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-1d3d0bcb-1ba5-4c6c-81d1-ac9d40c36282 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] [instance: 468c7efb-80de-4061-86c6-dd1eadbec954] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:27:46.726388 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1d3d0bcb-1ba5-4c6c-81d1-ac9d40c36282 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:27:46.727153 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1d3d0bcb-1ba5-4c6c-81d1-ac9d40c36282 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:27:46.728324 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1d3d0bcb-1ba5-4c6c-81d1-ac9d40c36282 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:27:46.729066 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1d3d0bcb-1ba5-4c6c-81d1-ac9d40c36282 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:27:46.880523 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-1d3d0bcb-1ba5-4c6c-81d1-ac9d40c36282 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '45b6f8ac55464fc99331221b6c67a85d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1d60fe3fdc841998824b803809f9e49', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:27:46.888302 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-1d3d0bcb-1ba5-4c6c-81d1-ac9d40c36282 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] 10.222.0.16 "GET /compute/v2.1/servers/468c7efb-80de-4061-86c6-dd1eadbec954" status: 200 len: 1259 microversion: 2.1 time: 0.175449 Jul 16 22:27:46.888874 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3596/7189] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:27:46 2022] GET /compute/v2.1/servers/468c7efb-80de-4061-86c6-dd1eadbec954 => generated 1259 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:27:47.906817 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-acbff44c-22c3-451b-bd98-c9ee1c8999a6 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:27:47.907680 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-acbff44c-22c3-451b-bd98-c9ee1c8999a6 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] [instance: 468c7efb-80de-4061-86c6-dd1eadbec954] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:27:47.914500 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-acbff44c-22c3-451b-bd98-c9ee1c8999a6 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:27:47.914963 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-acbff44c-22c3-451b-bd98-c9ee1c8999a6 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:27:47.915796 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-acbff44c-22c3-451b-bd98-c9ee1c8999a6 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:27:47.916304 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-acbff44c-22c3-451b-bd98-c9ee1c8999a6 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:27:48.050718 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-acbff44c-22c3-451b-bd98-c9ee1c8999a6 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '45b6f8ac55464fc99331221b6c67a85d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1d60fe3fdc841998824b803809f9e49', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:27:48.059610 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-acbff44c-22c3-451b-bd98-c9ee1c8999a6 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] 10.222.0.16 "GET /compute/v2.1/servers/468c7efb-80de-4061-86c6-dd1eadbec954" status: 200 len: 1259 microversion: 2.1 time: 0.157921 Jul 16 22:27:48.060185 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3594/7190] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:27:47 2022] GET /compute/v2.1/servers/468c7efb-80de-4061-86c6-dd1eadbec954 => generated 1259 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:27:49.077658 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-62026f56-6123-4d09-8a69-343cab3d0641 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:27:49.078281 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-62026f56-6123-4d09-8a69-343cab3d0641 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] [instance: 468c7efb-80de-4061-86c6-dd1eadbec954] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:27:49.084468 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-62026f56-6123-4d09-8a69-343cab3d0641 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:27:49.084931 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-62026f56-6123-4d09-8a69-343cab3d0641 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:27:49.085706 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-62026f56-6123-4d09-8a69-343cab3d0641 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:27:49.086155 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-62026f56-6123-4d09-8a69-343cab3d0641 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:27:49.222363 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-62026f56-6123-4d09-8a69-343cab3d0641 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '45b6f8ac55464fc99331221b6c67a85d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1d60fe3fdc841998824b803809f9e49', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:27:49.230477 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-62026f56-6123-4d09-8a69-343cab3d0641 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] 10.222.0.16 "GET /compute/v2.1/servers/468c7efb-80de-4061-86c6-dd1eadbec954" status: 200 len: 1259 microversion: 2.1 time: 0.157026 Jul 16 22:27:49.231065 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3597/7191] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:27:49 2022] GET /compute/v2.1/servers/468c7efb-80de-4061-86c6-dd1eadbec954 => generated 1259 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:27:50.248328 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-5ecd6343-47cb-45e6-b58d-ff2f6d344592 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:27:50.253319 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-5ecd6343-47cb-45e6-b58d-ff2f6d344592 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] [instance: 468c7efb-80de-4061-86c6-dd1eadbec954] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:27:50.255925 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5ecd6343-47cb-45e6-b58d-ff2f6d344592 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:27:50.256383 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5ecd6343-47cb-45e6-b58d-ff2f6d344592 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:27:50.257149 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5ecd6343-47cb-45e6-b58d-ff2f6d344592 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:27:50.257592 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5ecd6343-47cb-45e6-b58d-ff2f6d344592 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:27:50.392079 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-5ecd6343-47cb-45e6-b58d-ff2f6d344592 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '45b6f8ac55464fc99331221b6c67a85d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1d60fe3fdc841998824b803809f9e49', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:27:50.400441 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-5ecd6343-47cb-45e6-b58d-ff2f6d344592 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] 10.222.0.16 "GET /compute/v2.1/servers/468c7efb-80de-4061-86c6-dd1eadbec954" status: 200 len: 1259 microversion: 2.1 time: 0.156236 Jul 16 22:27:50.401022 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3595/7192] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:27:50 2022] GET /compute/v2.1/servers/468c7efb-80de-4061-86c6-dd1eadbec954 => generated 1259 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:27:51.417246 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-ee7d1e42-4873-4fd3-8b80-cc943b4368ae tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:27:51.417871 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-ee7d1e42-4873-4fd3-8b80-cc943b4368ae tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] [instance: 468c7efb-80de-4061-86c6-dd1eadbec954] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:27:51.424825 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ee7d1e42-4873-4fd3-8b80-cc943b4368ae tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:27:51.425285 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ee7d1e42-4873-4fd3-8b80-cc943b4368ae tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:27:51.426060 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ee7d1e42-4873-4fd3-8b80-cc943b4368ae tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:27:51.426537 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ee7d1e42-4873-4fd3-8b80-cc943b4368ae tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:27:51.560602 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-ee7d1e42-4873-4fd3-8b80-cc943b4368ae tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '45b6f8ac55464fc99331221b6c67a85d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1d60fe3fdc841998824b803809f9e49', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:27:51.568355 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-ee7d1e42-4873-4fd3-8b80-cc943b4368ae tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] 10.222.0.16 "GET /compute/v2.1/servers/468c7efb-80de-4061-86c6-dd1eadbec954" status: 200 len: 1259 microversion: 2.1 time: 0.155145 Jul 16 22:27:51.568927 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3598/7193] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:27:51 2022] GET /compute/v2.1/servers/468c7efb-80de-4061-86c6-dd1eadbec954 => generated 1259 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:27:52.179691 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-b1d6e809-7ac4-4153-affa-f9c59d388f0b req-f183d5c2-cacb-4823-89a6-a1aa550cd882 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "468c7efb-80de-4061-86c6-dd1eadbec954", "name": "network-vif-plugged", "status": "completed", "tag": "2b47bb9a-6a79-4e7e-a496-20c05d58d97f"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:27:52.189589 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-b1d6e809-7ac4-4153-affa-f9c59d388f0b req-f183d5c2-cacb-4823-89a6-a1aa550cd882 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:27:52.190037 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-b1d6e809-7ac4-4153-affa-f9c59d388f0b req-f183d5c2-cacb-4823-89a6-a1aa550cd882 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:27:52.202425 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-b1d6e809-7ac4-4153-affa-f9c59d388f0b req-f183d5c2-cacb-4823-89a6-a1aa550cd882 service nova] Creating event network-vif-plugged:2b47bb9a-6a79-4e7e-a496-20c05d58d97f for instance 468c7efb-80de-4061-86c6-dd1eadbec954 on nch2-master-1 Jul 16 22:27:52.208366 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-b1d6e809-7ac4-4153-affa-f9c59d388f0b req-f183d5c2-cacb-4823-89a6-a1aa550cd882 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.032261 Jul 16 22:27:52.208997 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3596/7194] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 22:27:52 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 22:27:52.585984 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-01a307a1-fc25-47c6-a353-a9390d543b9e tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:27:52.586665 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-01a307a1-fc25-47c6-a353-a9390d543b9e tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] [instance: 468c7efb-80de-4061-86c6-dd1eadbec954] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:27:52.588670 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell0:SELECT=2 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:27:52.589399 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell0:SELECT=6 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:27:52.595316 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-01a307a1-fc25-47c6-a353-a9390d543b9e tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:27:52.602220 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-01a307a1-fc25-47c6-a353-a9390d543b9e tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:27:52.602220 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-01a307a1-fc25-47c6-a353-a9390d543b9e tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:27:52.602220 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-01a307a1-fc25-47c6-a353-a9390d543b9e tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:27:52.734614 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-01a307a1-fc25-47c6-a353-a9390d543b9e tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '45b6f8ac55464fc99331221b6c67a85d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1d60fe3fdc841998824b803809f9e49', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:27:52.742831 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-01a307a1-fc25-47c6-a353-a9390d543b9e tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] 10.222.0.16 "GET /compute/v2.1/servers/468c7efb-80de-4061-86c6-dd1eadbec954" status: 200 len: 1259 microversion: 2.1 time: 0.161239 Jul 16 22:27:52.743444 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3599/7195] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:27:52 2022] GET /compute/v2.1/servers/468c7efb-80de-4061-86c6-dd1eadbec954 => generated 1259 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:27:53.757827 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell0:SELECT=6 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:27:53.762910 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-74c87493-fb23-4fec-a54a-ba092faf7b7f tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:27:53.763655 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-74c87493-fb23-4fec-a54a-ba092faf7b7f tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] [instance: 468c7efb-80de-4061-86c6-dd1eadbec954] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:27:53.769805 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-74c87493-fb23-4fec-a54a-ba092faf7b7f tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:27:53.771847 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-74c87493-fb23-4fec-a54a-ba092faf7b7f tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:27:53.771847 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-74c87493-fb23-4fec-a54a-ba092faf7b7f tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:27:53.772152 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-74c87493-fb23-4fec-a54a-ba092faf7b7f tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:27:53.910970 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-74c87493-fb23-4fec-a54a-ba092faf7b7f tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '45b6f8ac55464fc99331221b6c67a85d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1d60fe3fdc841998824b803809f9e49', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:27:53.918755 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-74c87493-fb23-4fec-a54a-ba092faf7b7f tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] 10.222.0.16 "GET /compute/v2.1/servers/468c7efb-80de-4061-86c6-dd1eadbec954" status: 200 len: 1259 microversion: 2.1 time: 0.162020 Jul 16 22:27:53.919367 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3597/7196] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:27:53 2022] GET /compute/v2.1/servers/468c7efb-80de-4061-86c6-dd1eadbec954 => generated 1259 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:27:54.936442 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-f8e94761-563b-4c5d-8a50-1f41c281e6ad tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:27:54.936976 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-f8e94761-563b-4c5d-8a50-1f41c281e6ad tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] [instance: 468c7efb-80de-4061-86c6-dd1eadbec954] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:27:54.944584 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f8e94761-563b-4c5d-8a50-1f41c281e6ad tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:27:54.945048 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f8e94761-563b-4c5d-8a50-1f41c281e6ad tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:27:54.945819 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f8e94761-563b-4c5d-8a50-1f41c281e6ad tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:27:54.946269 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-f8e94761-563b-4c5d-8a50-1f41c281e6ad tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:27:55.082099 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-f8e94761-563b-4c5d-8a50-1f41c281e6ad tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '45b6f8ac55464fc99331221b6c67a85d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1d60fe3fdc841998824b803809f9e49', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:27:55.090900 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-f8e94761-563b-4c5d-8a50-1f41c281e6ad tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] 10.222.0.16 "GET /compute/v2.1/servers/468c7efb-80de-4061-86c6-dd1eadbec954" status: 200 len: 1443 microversion: 2.1 time: 0.158685 Jul 16 22:27:55.091504 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3600/7197] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:27:54 2022] GET /compute/v2.1/servers/468c7efb-80de-4061-86c6-dd1eadbec954 => generated 1443 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:27:55.107533 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-717b14c0-dbcc-45b6-ac78-a4c82a32b49f tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:27:55.108151 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-717b14c0-dbcc-45b6-ac78-a4c82a32b49f tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] [instance: 468c7efb-80de-4061-86c6-dd1eadbec954] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:27:55.114978 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-717b14c0-dbcc-45b6-ac78-a4c82a32b49f tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:27:55.115469 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-717b14c0-dbcc-45b6-ac78-a4c82a32b49f tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:27:55.116242 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-717b14c0-dbcc-45b6-ac78-a4c82a32b49f tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:27:55.116686 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-717b14c0-dbcc-45b6-ac78-a4c82a32b49f tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:27:55.252026 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-717b14c0-dbcc-45b6-ac78-a4c82a32b49f tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '45b6f8ac55464fc99331221b6c67a85d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1d60fe3fdc841998824b803809f9e49', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:27:55.260193 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-717b14c0-dbcc-45b6-ac78-a4c82a32b49f tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] 10.222.0.16 "GET /compute/v2.1/servers/468c7efb-80de-4061-86c6-dd1eadbec954" status: 200 len: 1443 microversion: 2.1 time: 0.156526 Jul 16 22:27:55.260781 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3598/7198] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:27:55 2022] GET /compute/v2.1/servers/468c7efb-80de-4061-86c6-dd1eadbec954 => generated 1443 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 22:27:57.301110 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-35043751-2cf1-4bd2-8636-6c12b3b652ae req-1897ba3c-2cfd-41c3-b942-9ee560c065d3 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "468c7efb-80de-4061-86c6-dd1eadbec954", "tag": "2b47bb9a-6a79-4e7e-a496-20c05d58d97f"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:27:57.311070 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-35043751-2cf1-4bd2-8636-6c12b3b652ae req-1897ba3c-2cfd-41c3-b942-9ee560c065d3 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:27:57.311539 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-35043751-2cf1-4bd2-8636-6c12b3b652ae req-1897ba3c-2cfd-41c3-b942-9ee560c065d3 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:27:57.323601 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-35043751-2cf1-4bd2-8636-6c12b3b652ae req-1897ba3c-2cfd-41c3-b942-9ee560c065d3 service nova] Creating event network-changed:2b47bb9a-6a79-4e7e-a496-20c05d58d97f for instance 468c7efb-80de-4061-86c6-dd1eadbec954 on nch2-master-1 Jul 16 22:27:57.329020 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-35043751-2cf1-4bd2-8636-6c12b3b652ae req-1897ba3c-2cfd-41c3-b942-9ee560c065d3 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.031669 Jul 16 22:27:57.331839 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3601/7199] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 22:27:57 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 22:28:05.031867 ncdv-master-1 devstack@n-api.service[138680]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jul 16 22:28:05.032425 ncdv-master-1 devstack@n-api.service[138680]: Sat Jul 16 22:28:05 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Jul 16 22:30:19.878172 ncdv-master-1 devstack@n-api.service[138680]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jul 16 22:30:19.878784 ncdv-master-1 devstack@n-api.service[138680]: Sat Jul 16 22:30:19 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Jul 16 22:31:25.040441 ncdv-master-1 devstack@n-api.service[138680]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jul 16 22:31:25.041255 ncdv-master-1 devstack@n-api.service[138680]: Sat Jul 16 22:31:25 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Jul 16 22:31:27.489939 ncdv-master-1 devstack@n-api.service[138680]: ERROR oslo.messaging._drivers.impl_rabbit [-] [7d526b5e-7cab-48f3-be00-1b242d865b86] AMQP server on 10.222.0.16:5672 is unreachable: . Trying again in 1 seconds.: amqp.exceptions.RecoverableConnectionError: Jul 16 22:31:27.491133 ncdv-master-1 devstack@n-api.service[138680]: ERROR oslo.messaging._drivers.impl_rabbit [-] [a302809d-3282-48f5-9662-a738982e3059] AMQP server on 10.222.0.16:5672 is unreachable: . Trying again in 1 seconds.: amqp.exceptions.RecoverableConnectionError: Jul 16 22:31:27.491700 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_api:SELECT=37,nova_api:INSERT=1 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:31:27.492454 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=34 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:31:27.549696 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-703be9d6-6ac4-4fb7-b73a-cea92f376f39 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Action: 'action', calling method: >, body: {"os-getConsoleOutput": {}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:31:27.551606 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-703be9d6-6ac4-4fb7-b73a-cea92f376f39 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] [instance: 468c7efb-80de-4061-86c6-dd1eadbec954] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:31:27.558918 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-703be9d6-6ac4-4fb7-b73a-cea92f376f39 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:31:27.559398 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-703be9d6-6ac4-4fb7-b73a-cea92f376f39 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:31:27.560166 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-703be9d6-6ac4-4fb7-b73a-cea92f376f39 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:31:27.560606 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-703be9d6-6ac4-4fb7-b73a-cea92f376f39 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:31:28.499577 ncdv-master-1 devstack@n-api.service[138680]: INFO oslo.messaging._drivers.impl_rabbit [-] [a302809d-3282-48f5-9662-a738982e3059] Reconnected to AMQP server on 10.222.0.16:5672 via [amqp] client with port 34608. Jul 16 22:31:28.500337 ncdv-master-1 devstack@n-api.service[138680]: INFO oslo.messaging._drivers.impl_rabbit [-] [7d526b5e-7cab-48f3-be00-1b242d865b86] Reconnected to AMQP server on 10.222.0.16:5672 via [amqp] client with port 34606. Jul 16 22:31:28.504209 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-703be9d6-6ac4-4fb7-b73a-cea92f376f39 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] 10.222.0.16 "POST /compute/v2.1/servers/468c7efb-80de-4061-86c6-dd1eadbec954/action" status: 200 len: 24570 microversion: 2.1 time: 1.016720 Jul 16 22:31:28.504836 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3599/7200] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 22:31:27 2022] POST /compute/v2.1/servers/468c7efb-80de-4061-86c6-dd1eadbec954/action => generated 24570 bytes in 1018 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 16 22:31:29.479907 ncdv-master-1 devstack@n-api.service[138681]: ERROR oslo.messaging._drivers.impl_rabbit [-] [e4397b8a-69d0-4b1f-ae48-d345afeb4816] AMQP server on 10.222.0.16:5672 is unreachable: [Errno 104] Connection reset by peer. Trying again in 1 seconds.: ConnectionResetError: [Errno 104] Connection reset by peer Jul 16 22:31:29.481436 ncdv-master-1 devstack@n-api.service[138681]: ERROR oslo.messaging._drivers.impl_rabbit [-] [4de8847c-05a2-4cd6-bd0e-256d7a754675] AMQP server on 10.222.0.16:5672 is unreachable: [Errno 104] Connection reset by peer. Trying again in 1 seconds.: ConnectionResetError: [Errno 104] Connection reset by peer Jul 16 22:31:29.482059 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_api:SELECT=47,nova_api:UPDATE=1,nova_api:DELETE=1,nova_api:INSERT=3 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:31:29.482828 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=74,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:31:29.494172 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-1fdca95d-8a4d-4cce-8566-496a0503a41f tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:31:29.494172 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-1fdca95d-8a4d-4cce-8566-496a0503a41f tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] [instance: 468c7efb-80de-4061-86c6-dd1eadbec954] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:31:29.498299 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1fdca95d-8a4d-4cce-8566-496a0503a41f tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:31:29.498841 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1fdca95d-8a4d-4cce-8566-496a0503a41f tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:31:29.499641 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1fdca95d-8a4d-4cce-8566-496a0503a41f tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:31:29.500103 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1fdca95d-8a4d-4cce-8566-496a0503a41f tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:31:29.542224 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-1fdca95d-8a4d-4cce-8566-496a0503a41f tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] [instance: 468c7efb-80de-4061-86c6-dd1eadbec954] Going to try to terminate instance {{(pid=138681) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 16 22:31:29.543684 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-b307906e-1e8c-4125-9449-83e321cdcf73 req-2f5e9feb-c86e-42a6-a553-54a53442ef99 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "468c7efb-80de-4061-86c6-dd1eadbec954", "tag": "2b47bb9a-6a79-4e7e-a496-20c05d58d97f"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:31:29.553974 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-b307906e-1e8c-4125-9449-83e321cdcf73 req-2f5e9feb-c86e-42a6-a553-54a53442ef99 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:31:29.555032 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-b307906e-1e8c-4125-9449-83e321cdcf73 req-2f5e9feb-c86e-42a6-a553-54a53442ef99 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:31:29.566928 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-b307906e-1e8c-4125-9449-83e321cdcf73 req-2f5e9feb-c86e-42a6-a553-54a53442ef99 service nova] Creating event network-changed:2b47bb9a-6a79-4e7e-a496-20c05d58d97f for instance 468c7efb-80de-4061-86c6-dd1eadbec954 on nch2-master-1 Jul 16 22:31:29.576625 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-b307906e-1e8c-4125-9449-83e321cdcf73 req-2f5e9feb-c86e-42a6-a553-54a53442ef99 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.095642 Jul 16 22:31:29.583538 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3600/7201] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 22:31:29 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 22:31:29.873328 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-1fdca95d-8a4d-4cce-8566-496a0503a41f tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] 10.222.0.16 "DELETE /compute/v2.1/servers/468c7efb-80de-4061-86c6-dd1eadbec954" status: 204 len: 0 microversion: 2.1 time: 0.395298 Jul 16 22:31:29.877351 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3602/7202] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 22:31:29 2022] DELETE /compute/v2.1/servers/468c7efb-80de-4061-86c6-dd1eadbec954 => generated 0 bytes in 397 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 16 22:31:29.884390 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-1e0e1b8e-f891-47ac-8b03-666055425eac tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:31:29.885385 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-1e0e1b8e-f891-47ac-8b03-666055425eac tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] [instance: 468c7efb-80de-4061-86c6-dd1eadbec954] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:31:29.892149 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1e0e1b8e-f891-47ac-8b03-666055425eac tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:31:29.892874 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1e0e1b8e-f891-47ac-8b03-666055425eac tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:31:29.894203 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1e0e1b8e-f891-47ac-8b03-666055425eac tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:31:29.894954 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1e0e1b8e-f891-47ac-8b03-666055425eac tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:31:30.038646 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-1e0e1b8e-f891-47ac-8b03-666055425eac tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '45b6f8ac55464fc99331221b6c67a85d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1d60fe3fdc841998824b803809f9e49', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:31:30.046560 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-1e0e1b8e-f891-47ac-8b03-666055425eac tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] 10.222.0.16 "GET /compute/v2.1/servers/468c7efb-80de-4061-86c6-dd1eadbec954" status: 200 len: 1567 microversion: 2.1 time: 0.167845 Jul 16 22:31:30.047233 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3601/7203] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:31:29 2022] GET /compute/v2.1/servers/468c7efb-80de-4061-86c6-dd1eadbec954 => generated 1567 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:31:31.066288 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-514ff952-0c67-4228-824b-0b5b49b51281 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:31:31.067026 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-514ff952-0c67-4228-824b-0b5b49b51281 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] [instance: 468c7efb-80de-4061-86c6-dd1eadbec954] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:31:31.081015 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-514ff952-0c67-4228-824b-0b5b49b51281 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:31:31.081415 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-514ff952-0c67-4228-824b-0b5b49b51281 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:31:31.082202 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-514ff952-0c67-4228-824b-0b5b49b51281 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:31:31.082658 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-514ff952-0c67-4228-824b-0b5b49b51281 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:31:31.087231 ncdv-master-1 devstack@n-api.service[138681]: INFO oslo.messaging._drivers.impl_rabbit [-] [e4397b8a-69d0-4b1f-ae48-d345afeb4816] Reconnected to AMQP server on 10.222.0.16:5672 via [amqp] client with port 34626. Jul 16 22:31:31.089483 ncdv-master-1 devstack@n-api.service[138681]: INFO oslo.messaging._drivers.impl_rabbit [-] [4de8847c-05a2-4cd6-bd0e-256d7a754675] Reconnected to AMQP server on 10.222.0.16:5672 via [amqp] client with port 34628. Jul 16 22:31:31.221236 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-514ff952-0c67-4228-824b-0b5b49b51281 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '45b6f8ac55464fc99331221b6c67a85d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1d60fe3fdc841998824b803809f9e49', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:31:31.230328 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-514ff952-0c67-4228-824b-0b5b49b51281 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] 10.222.0.16 "GET /compute/v2.1/servers/468c7efb-80de-4061-86c6-dd1eadbec954" status: 200 len: 1567 microversion: 2.1 time: 0.169242 Jul 16 22:31:31.230920 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3603/7204] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:31:31 2022] GET /compute/v2.1/servers/468c7efb-80de-4061-86c6-dd1eadbec954 => generated 1567 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:31:32.248264 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-c73ef4b0-f278-4c7c-a1d2-2b1ca9d120fb tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:31:32.248886 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-c73ef4b0-f278-4c7c-a1d2-2b1ca9d120fb tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] [instance: 468c7efb-80de-4061-86c6-dd1eadbec954] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:31:32.256111 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c73ef4b0-f278-4c7c-a1d2-2b1ca9d120fb tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:31:32.257234 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c73ef4b0-f278-4c7c-a1d2-2b1ca9d120fb tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:31:32.258023 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c73ef4b0-f278-4c7c-a1d2-2b1ca9d120fb tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:31:32.258469 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-c73ef4b0-f278-4c7c-a1d2-2b1ca9d120fb tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:31:32.398152 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-c73ef4b0-f278-4c7c-a1d2-2b1ca9d120fb tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '45b6f8ac55464fc99331221b6c67a85d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1d60fe3fdc841998824b803809f9e49', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:31:32.405878 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-c73ef4b0-f278-4c7c-a1d2-2b1ca9d120fb tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] 10.222.0.16 "GET /compute/v2.1/servers/468c7efb-80de-4061-86c6-dd1eadbec954" status: 200 len: 1449 microversion: 2.1 time: 0.161666 Jul 16 22:31:32.411189 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3602/7205] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:31:32 2022] GET /compute/v2.1/servers/468c7efb-80de-4061-86c6-dd1eadbec954 => generated 1449 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:31:32.525667 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-6af77d40-c5e6-4ea2-a041-b162629c6824 req-f7ed3fd7-6609-4c47-a778-3878d75789d7 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "468c7efb-80de-4061-86c6-dd1eadbec954", "name": "network-vif-deleted", "tag": "2b47bb9a-6a79-4e7e-a496-20c05d58d97f"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:31:32.534629 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-6af77d40-c5e6-4ea2-a041-b162629c6824 req-f7ed3fd7-6609-4c47-a778-3878d75789d7 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:31:32.538369 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-6af77d40-c5e6-4ea2-a041-b162629c6824 req-f7ed3fd7-6609-4c47-a778-3878d75789d7 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:31:32.547143 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-6af77d40-c5e6-4ea2-a041-b162629c6824 req-f7ed3fd7-6609-4c47-a778-3878d75789d7 service nova] Creating event network-vif-deleted:2b47bb9a-6a79-4e7e-a496-20c05d58d97f for instance 468c7efb-80de-4061-86c6-dd1eadbec954 on nch2-master-1 Jul 16 22:31:32.552403 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-6af77d40-c5e6-4ea2-a041-b162629c6824 req-f7ed3fd7-6609-4c47-a778-3878d75789d7 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.031244 Jul 16 22:31:32.553006 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3604/7206] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 22:31:32 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 22:31:33.423308 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-95bf8628-2dbb-4415-b7e7-2cfb0e75f866 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:31:33.423889 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-95bf8628-2dbb-4415-b7e7-2cfb0e75f866 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] [instance: 468c7efb-80de-4061-86c6-dd1eadbec954] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:31:33.431299 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-95bf8628-2dbb-4415-b7e7-2cfb0e75f866 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:31:33.431768 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-95bf8628-2dbb-4415-b7e7-2cfb0e75f866 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:31:33.432549 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-95bf8628-2dbb-4415-b7e7-2cfb0e75f866 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:31:33.432997 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-95bf8628-2dbb-4415-b7e7-2cfb0e75f866 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:31:33.464551 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-95bf8628-2dbb-4415-b7e7-2cfb0e75f866 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:31:33.465012 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-95bf8628-2dbb-4415-b7e7-2cfb0e75f866 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:31:33.465472 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.objects.instance [None req-95bf8628-2dbb-4415-b7e7-2cfb0e75f866 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lazy-loading 'fault' on Instance uuid 468c7efb-80de-4061-86c6-dd1eadbec954 {{(pid=138680) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 22:31:33.507260 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-95bf8628-2dbb-4415-b7e7-2cfb0e75f866 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '45b6f8ac55464fc99331221b6c67a85d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f1d60fe3fdc841998824b803809f9e49', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:31:33.516007 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-95bf8628-2dbb-4415-b7e7-2cfb0e75f866 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] 10.222.0.16 "GET /compute/v2.1/servers/468c7efb-80de-4061-86c6-dd1eadbec954" status: 200 len: 1224 microversion: 2.1 time: 0.096744 Jul 16 22:31:33.521053 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3603/7207] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:31:33 2022] GET /compute/v2.1/servers/468c7efb-80de-4061-86c6-dd1eadbec954 => generated 1224 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:31:34.538892 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-437f1449-0d2e-464a-9444-8f9078e3ec59 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:31:34.540446 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-437f1449-0d2e-464a-9444-8f9078e3ec59 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] [instance: 468c7efb-80de-4061-86c6-dd1eadbec954] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:31:34.547199 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-437f1449-0d2e-464a-9444-8f9078e3ec59 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:31:34.547668 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-437f1449-0d2e-464a-9444-8f9078e3ec59 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:31:34.548453 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-437f1449-0d2e-464a-9444-8f9078e3ec59 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:31:34.548909 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-437f1449-0d2e-464a-9444-8f9078e3ec59 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:31:34.562465 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-437f1449-0d2e-464a-9444-8f9078e3ec59 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] HTTP exception thrown: Instance 468c7efb-80de-4061-86c6-dd1eadbec954 could not be found. Jul 16 22:31:34.564143 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-437f1449-0d2e-464a-9444-8f9078e3ec59 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Request method failure captured: Jul 16 22:31:34.564143 ncdv-master-1 devstack@n-api.service[138681]: request: GET /compute/v2.1/servers/468c7efb-80de-4061-86c6-dd1eadbec954 HTTP/1.1 Jul 16 22:31:34.564143 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 22:31:34.564143 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 22:31:34.564143 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 22:31:34.564143 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 22:31:34.564143 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 22:31:34.564143 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 22:31:34.564143 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 22:31:34.564143 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 22:31:34.564143 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0zpPBdeAo_vWtVXYIi-bSOM-tiLa8OvUoyegMfchbuqV5mTfCQcK6IxjM8bhh6FRbRh1XKEgJ7doP6PFcG_RKQJx30U_p-OV9wOEn7wZR-2sYFOm4bFBgwpqdQDDW6gjHkjOuDHceK7lTEWp7k0jHu-PGFew0xD4wOdVBDnoW0YD768 Jul 16 22:31:34.564143 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 22:31:34.564143 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 22:31:34.564143 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 22:31:34.564143 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 22:31:34.564143 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 22:31:34.564143 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 22:31:34.564143 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: f1d60fe3fdc841998824b803809f9e49 Jul 16 22:31:34.564143 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-TestShelveInstance-574938005 Jul 16 22:31:34.564143 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 22:31:34.564143 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 22:31:34.564143 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/f1d60fe3fdc841998824b803809f9e49"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/f1d60fe3fdc841998824b803809f9e49"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_f1d60fe3fdc841998824b803809f9e49"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/f1d60fe3fdc841998824b803809f9e49"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 22:31:34.564143 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-TestShelveInstance-574938005 Jul 16 22:31:34.564143 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: f1d60fe3fdc841998824b803809f9e49 Jul 16 22:31:34.564143 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-TestShelveInstance-574938005 Jul 16 22:31:34.564143 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-TestShelveInstance-574938005-project Jul 16 22:31:34.564143 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 22:31:34.564143 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 22:31:34.564143 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: 45b6f8ac55464fc99331221b6c67a85d Jul 16 22:31:34.564143 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-TestShelveInstance-574938005-project Jul 16 22:31:34.564143 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 22:31:34.564143 ncdv-master-1 devstack@n-api.service[138681]: exception: Instance 468c7efb-80de-4061-86c6-dd1eadbec954 could not be found. Jul 16 22:31:34.564143 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': '468c7efb-80de-4061-86c6-dd1eadbec954'} Jul 16 22:31:34.564143 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 22:31:34.564143 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 22:31:34.564143 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 22:31:34.564143 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 22:31:34.564143 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 22:31:34.564143 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 22:31:34.564143 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 22:31:34.564143 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 22:31:34.564143 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 22:31:34.564143 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 22:31:34.564143 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 468c7efb-80de-4061-86c6-dd1eadbec954 could not be found. Jul 16 22:31:34.564143 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 22:31:34.564143 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 22:31:34.564143 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 22:31:34.564143 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 22:31:34.564143 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 22:31:34.564143 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 22:31:34.564143 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 22:31:34.564143 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 22:31:34.564143 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 22:31:34.564143 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 22:31:34.564143 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 22:31:34.564143 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 22:31:34.564143 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 22:31:34.564143 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 22:31:34.564143 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 22:31:34.564143 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 22:31:34.564143 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 468c7efb-80de-4061-86c6-dd1eadbec954 could not be found. Jul 16 22:31:34.564143 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 22:31:34.564143 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 22:31:34.564143 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 22:31:34.574201 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 22:31:34.574201 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 22:31:34.574201 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 22:31:34.574201 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 22:31:34.574201 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 22:31:34.574201 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 468c7efb-80de-4061-86c6-dd1eadbec954 could not be found. Jul 16 22:31:34.574201 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 22:31:34.574201 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-437f1449-0d2e-464a-9444-8f9078e3ec59 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Returning 404 to user: Instance 468c7efb-80de-4061-86c6-dd1eadbec954 could not be found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 22:31:34.574201 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-437f1449-0d2e-464a-9444-8f9078e3ec59 tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] 10.222.0.16 "GET /compute/v2.1/servers/468c7efb-80de-4061-86c6-dd1eadbec954" status: 404 len: 111 microversion: 2.1 time: 0.033635 Jul 16 22:31:34.574201 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3605/7208] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:31:34 2022] GET /compute/v2.1/servers/468c7efb-80de-4061-86c6-dd1eadbec954 => generated 111 bytes in 35 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 22:31:34.791829 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-a892765e-084f-47da-9611-f994c8ce087b tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] Calling method '.version_select at 0x7fdd9ce76430>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:31:34.868712 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-a892765e-084f-47da-9611-f994c8ce087b tempest-TestShelveInstance-574938005 tempest-TestShelveInstance-574938005-project] 10.222.0.16 "DELETE /compute/v2.1/os-keypairs/tempest-TestShelveInstance-729410316" status: 202 len: 0 microversion: 2.1 time: 0.080519 Jul 16 22:31:34.869358 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3604/7209] 10.222.0.16 () {58 vars in 1250 bytes} [Sat Jul 16 22:31:34 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestShelveInstance-729410316 => generated 0 bytes in 82 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 22:31:57.557276 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_api:SELECT=13,nova_api:UPDATE=1 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:31:57.558040 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=24,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:31:57.719879 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-02029e72-9b63-4091-88a7-583f4babaf09 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Action: 'create', calling method: .version_select at 0x7fdd8c5a8e50>, body: {"keypair": {"name": "tempest-TestSnapshotPattern-1466442200", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBM3X+VmspyujJbiOGAzbag4clSTh3GVADpnVChPCdv/dA/vxJSSzdkQpkQn1VAr6iSqCSBeNXQQP+WWk9LFYqU9ndfl1HUzbIpGTuzfEGMbU5hCsUKXlkbivGAkmTPm0bA=="}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:31:57.735223 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-02029e72-9b63-4091-88a7-583f4babaf09 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Getting quotas for project 4490e1139a4046228fe39fbdc7973d20. Resources: {'key_pairs'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 22:31:57.740502 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-02029e72-9b63-4091-88a7-583f4babaf09 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Getting quotas for user 77a17d04ccb1477cafd82ca23fd65328 and project 4490e1139a4046228fe39fbdc7973d20. Resources: {'key_pairs'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 22:31:57.828704 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-02029e72-9b63-4091-88a7-583f4babaf09 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] 10.222.0.16 "POST /compute/v2.1/os-keypairs" status: 200 len: 398 microversion: 2.1 time: 0.277400 Jul 16 22:31:57.829401 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3606/7210] 10.222.0.16 () {60 vars in 1158 bytes} [Sat Jul 16 22:31:57 2022] POST /compute/v2.1/os-keypairs => generated 398 bytes in 279 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 16 22:32:00.356566 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=26 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:32:00.357405 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_api:SELECT=15,nova_api:DELETE=1 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:32:00.361555 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-dda50cff-7c75-4669-ad15-366e43cbcdfb tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestSnapshotPattern-server-150117362", "imageRef": "1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9", "flavorRef": "452", "key_name": "tempest-TestSnapshotPattern-1466442200", "security_groups": [{"name": "tempest-secgroup-smoke-1333268523"}], "networks": [{"uuid": "496ad3c9-f791-43dc-a218-50a7eb7daba2"}]}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:32:00.479942 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.network.neutron [None req-dda50cff-7c75-4669-ad15-366e43cbcdfb tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] validate_networks() for [('496ad3c9-f791-43dc-a218-50a7eb7daba2', None, None, None, None, None)] {{(pid=138680) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 16 22:32:01.007280 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-dda50cff-7c75-4669-ad15-366e43cbcdfb tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Flavor limits 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 16 22:32:01.007718 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-dda50cff-7c75-4669-ad15-366e43cbcdfb tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Image limits 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 16 22:32:01.008203 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-dda50cff-7c75-4669-ad15-366e43cbcdfb tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Flavor pref 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 16 22:32:01.008614 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-dda50cff-7c75-4669-ad15-366e43cbcdfb tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Image pref 0:0:0 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 16 22:32:01.009290 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.virt.hardware [None req-dda50cff-7c75-4669-ad15-366e43cbcdfb tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138680) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 16 22:32:01.020556 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-dda50cff-7c75-4669-ad15-366e43cbcdfb tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:32:01.021016 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-dda50cff-7c75-4669-ad15-366e43cbcdfb tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:32:01.021826 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-dda50cff-7c75-4669-ad15-366e43cbcdfb tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:32:01.022268 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-dda50cff-7c75-4669-ad15-366e43cbcdfb tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:32:01.043570 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-dda50cff-7c75-4669-ad15-366e43cbcdfb tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Getting quotas for project 4490e1139a4046228fe39fbdc7973d20. Resources: {'ram', 'instances', 'cores'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 22:32:01.049561 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.quota [None req-dda50cff-7c75-4669-ad15-366e43cbcdfb tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Getting quotas for user 77a17d04ccb1477cafd82ca23fd65328 and project 4490e1139a4046228fe39fbdc7973d20. Resources: {'ram', 'instances', 'cores'} {{(pid=138680) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 22:32:01.063895 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-dda50cff-7c75-4669-ad15-366e43cbcdfb tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Going to run 1 instances... {{(pid=138680) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 16 22:32:01.066622 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-dda50cff-7c75-4669-ad15-366e43cbcdfb tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] [instance: d9d4d400-1fed-4996-9d09-dd0dddd7fc42] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='1bb9fddd-5eec-4ac9-86bd-3afdc2144cb9',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138680) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 16 22:32:01.220128 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-dda50cff-7c75-4669-ad15-366e43cbcdfb tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 406 microversion: 2.1 time: 0.865483 Jul 16 22:32:01.220996 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3605/7211] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 22:32:00 2022] POST /compute/v2.1/servers => generated 406 bytes in 867 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 16 22:32:01.232374 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-adf3de88-aaf7-4a00-8fc1-f4297636d5eb tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:32:01.233595 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-adf3de88-aaf7-4a00-8fc1-f4297636d5eb tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] [instance: d9d4d400-1fed-4996-9d09-dd0dddd7fc42] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:32:01.299775 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-adf3de88-aaf7-4a00-8fc1-f4297636d5eb tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '77a17d04ccb1477cafd82ca23fd65328', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4490e1139a4046228fe39fbdc7973d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:32:01.307741 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-adf3de88-aaf7-4a00-8fc1-f4297636d5eb tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] 10.222.0.16 "GET /compute/v2.1/servers/d9d4d400-1fed-4996-9d09-dd0dddd7fc42" status: 200 len: 1248 microversion: 2.1 time: 0.080196 Jul 16 22:32:01.308336 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3607/7212] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:32:01 2022] GET /compute/v2.1/servers/d9d4d400-1fed-4996-9d09-dd0dddd7fc42 => generated 1248 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 22:32:02.326248 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-9e960042-63c3-4f3e-927e-7735edd6e847 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:32:02.326880 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-9e960042-63c3-4f3e-927e-7735edd6e847 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] [instance: d9d4d400-1fed-4996-9d09-dd0dddd7fc42] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:32:02.333390 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9e960042-63c3-4f3e-927e-7735edd6e847 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:32:02.333840 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9e960042-63c3-4f3e-927e-7735edd6e847 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:32:02.334611 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9e960042-63c3-4f3e-927e-7735edd6e847 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:32:02.335079 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-9e960042-63c3-4f3e-927e-7735edd6e847 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:32:02.400634 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-9e960042-63c3-4f3e-927e-7735edd6e847 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '77a17d04ccb1477cafd82ca23fd65328', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4490e1139a4046228fe39fbdc7973d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:32:02.409008 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-9e960042-63c3-4f3e-927e-7735edd6e847 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] 10.222.0.16 "GET /compute/v2.1/servers/d9d4d400-1fed-4996-9d09-dd0dddd7fc42" status: 200 len: 1252 microversion: 2.1 time: 0.088293 Jul 16 22:32:02.409580 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3606/7213] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:32:02 2022] GET /compute/v2.1/servers/d9d4d400-1fed-4996-9d09-dd0dddd7fc42 => generated 1252 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:32:03.426040 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-77cc1f51-4c7e-4d49-9441-deddee8705fb tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:32:03.426672 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-77cc1f51-4c7e-4d49-9441-deddee8705fb tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] [instance: d9d4d400-1fed-4996-9d09-dd0dddd7fc42] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:32:03.435326 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-77cc1f51-4c7e-4d49-9441-deddee8705fb tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:32:03.435796 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-77cc1f51-4c7e-4d49-9441-deddee8705fb tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:32:03.436590 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-77cc1f51-4c7e-4d49-9441-deddee8705fb tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:32:03.437075 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-77cc1f51-4c7e-4d49-9441-deddee8705fb tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:32:03.511447 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-77cc1f51-4c7e-4d49-9441-deddee8705fb tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '77a17d04ccb1477cafd82ca23fd65328', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4490e1139a4046228fe39fbdc7973d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:32:03.519919 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-77cc1f51-4c7e-4d49-9441-deddee8705fb tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] 10.222.0.16 "GET /compute/v2.1/servers/d9d4d400-1fed-4996-9d09-dd0dddd7fc42" status: 200 len: 1308 microversion: 2.1 time: 0.097796 Jul 16 22:32:03.520618 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3608/7214] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:32:03 2022] GET /compute/v2.1/servers/d9d4d400-1fed-4996-9d09-dd0dddd7fc42 => generated 1308 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:32:04.544830 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-1762725b-53ef-4cd9-83b2-c8633ae07025 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:32:04.545469 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-1762725b-53ef-4cd9-83b2-c8633ae07025 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] [instance: d9d4d400-1fed-4996-9d09-dd0dddd7fc42] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:32:04.552209 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1762725b-53ef-4cd9-83b2-c8633ae07025 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:32:04.552655 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1762725b-53ef-4cd9-83b2-c8633ae07025 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:32:04.553623 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1762725b-53ef-4cd9-83b2-c8633ae07025 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:32:04.554233 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1762725b-53ef-4cd9-83b2-c8633ae07025 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:32:04.694072 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-1762725b-53ef-4cd9-83b2-c8633ae07025 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '77a17d04ccb1477cafd82ca23fd65328', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4490e1139a4046228fe39fbdc7973d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:32:04.702018 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-1762725b-53ef-4cd9-83b2-c8633ae07025 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] 10.222.0.16 "GET /compute/v2.1/servers/d9d4d400-1fed-4996-9d09-dd0dddd7fc42" status: 200 len: 1374 microversion: 2.1 time: 0.161789 Jul 16 22:32:04.702593 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3607/7215] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:32:04 2022] GET /compute/v2.1/servers/d9d4d400-1fed-4996-9d09-dd0dddd7fc42 => generated 1374 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:32:05.084040 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-dd9f44a4-a2fd-4fbf-a1b8-abd72d934952 req-3d9fc052-1093-488a-bb8f-ed308a1c4d06 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "d9d4d400-1fed-4996-9d09-dd0dddd7fc42", "tag": "f6f604bc-d380-4063-b8fa-414495aada0b"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:32:05.094185 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-dd9f44a4-a2fd-4fbf-a1b8-abd72d934952 req-3d9fc052-1093-488a-bb8f-ed308a1c4d06 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:32:05.094658 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-dd9f44a4-a2fd-4fbf-a1b8-abd72d934952 req-3d9fc052-1093-488a-bb8f-ed308a1c4d06 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:32:05.107195 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-dd9f44a4-a2fd-4fbf-a1b8-abd72d934952 req-3d9fc052-1093-488a-bb8f-ed308a1c4d06 service nova] Creating event network-changed:f6f604bc-d380-4063-b8fa-414495aada0b for instance d9d4d400-1fed-4996-9d09-dd0dddd7fc42 on nch1-master-1 Jul 16 22:32:05.113413 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-dd9f44a4-a2fd-4fbf-a1b8-abd72d934952 req-3d9fc052-1093-488a-bb8f-ed308a1c4d06 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.036133 Jul 16 22:32:05.114032 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3609/7216] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 22:32:05 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 22:32:05.719911 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-a9b0c386-ab52-4f5c-b6dc-dba0c9abb1b9 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:32:05.720536 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-a9b0c386-ab52-4f5c-b6dc-dba0c9abb1b9 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] [instance: d9d4d400-1fed-4996-9d09-dd0dddd7fc42] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:32:05.727531 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a9b0c386-ab52-4f5c-b6dc-dba0c9abb1b9 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:32:05.727989 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a9b0c386-ab52-4f5c-b6dc-dba0c9abb1b9 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:32:05.728760 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a9b0c386-ab52-4f5c-b6dc-dba0c9abb1b9 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:32:05.729204 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-a9b0c386-ab52-4f5c-b6dc-dba0c9abb1b9 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:32:05.867662 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-a9b0c386-ab52-4f5c-b6dc-dba0c9abb1b9 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '77a17d04ccb1477cafd82ca23fd65328', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4490e1139a4046228fe39fbdc7973d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:32:05.876374 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-a9b0c386-ab52-4f5c-b6dc-dba0c9abb1b9 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] 10.222.0.16 "GET /compute/v2.1/servers/d9d4d400-1fed-4996-9d09-dd0dddd7fc42" status: 200 len: 1374 microversion: 2.1 time: 0.160425 Jul 16 22:32:05.876955 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3608/7217] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:32:05 2022] GET /compute/v2.1/servers/d9d4d400-1fed-4996-9d09-dd0dddd7fc42 => generated 1374 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:32:06.894854 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-976224ee-b2e4-40e1-891b-0a35d3521abc tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:32:06.895673 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-976224ee-b2e4-40e1-891b-0a35d3521abc tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] [instance: d9d4d400-1fed-4996-9d09-dd0dddd7fc42] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:32:06.902745 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-976224ee-b2e4-40e1-891b-0a35d3521abc tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:32:06.903221 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-976224ee-b2e4-40e1-891b-0a35d3521abc tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:32:06.904003 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-976224ee-b2e4-40e1-891b-0a35d3521abc tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:32:06.904459 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-976224ee-b2e4-40e1-891b-0a35d3521abc tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:32:07.040317 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-976224ee-b2e4-40e1-891b-0a35d3521abc tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '77a17d04ccb1477cafd82ca23fd65328', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4490e1139a4046228fe39fbdc7973d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:32:07.048030 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-976224ee-b2e4-40e1-891b-0a35d3521abc tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] 10.222.0.16 "GET /compute/v2.1/servers/d9d4d400-1fed-4996-9d09-dd0dddd7fc42" status: 200 len: 1374 microversion: 2.1 time: 0.158282 Jul 16 22:32:07.048804 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3610/7218] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:32:06 2022] GET /compute/v2.1/servers/d9d4d400-1fed-4996-9d09-dd0dddd7fc42 => generated 1374 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:32:08.065478 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-7fbabfcc-df9d-4886-843a-8549423a0ba8 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:32:08.066132 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-7fbabfcc-df9d-4886-843a-8549423a0ba8 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] [instance: d9d4d400-1fed-4996-9d09-dd0dddd7fc42] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:32:08.073138 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7fbabfcc-df9d-4886-843a-8549423a0ba8 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:32:08.074102 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7fbabfcc-df9d-4886-843a-8549423a0ba8 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:32:08.074963 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7fbabfcc-df9d-4886-843a-8549423a0ba8 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:32:08.075442 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7fbabfcc-df9d-4886-843a-8549423a0ba8 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:32:08.209296 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-7fbabfcc-df9d-4886-843a-8549423a0ba8 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '77a17d04ccb1477cafd82ca23fd65328', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4490e1139a4046228fe39fbdc7973d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:32:08.219596 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-7fbabfcc-df9d-4886-843a-8549423a0ba8 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] 10.222.0.16 "GET /compute/v2.1/servers/d9d4d400-1fed-4996-9d09-dd0dddd7fc42" status: 200 len: 1374 microversion: 2.1 time: 0.157854 Jul 16 22:32:08.222908 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3609/7219] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:32:08 2022] GET /compute/v2.1/servers/d9d4d400-1fed-4996-9d09-dd0dddd7fc42 => generated 1374 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:32:09.237198 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-08e2aeac-37b4-4c56-8fd7-8634bf76b438 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:32:09.237831 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-08e2aeac-37b4-4c56-8fd7-8634bf76b438 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] [instance: d9d4d400-1fed-4996-9d09-dd0dddd7fc42] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:32:09.245137 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-08e2aeac-37b4-4c56-8fd7-8634bf76b438 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:32:09.245628 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-08e2aeac-37b4-4c56-8fd7-8634bf76b438 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:32:09.246462 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-08e2aeac-37b4-4c56-8fd7-8634bf76b438 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:32:09.246910 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-08e2aeac-37b4-4c56-8fd7-8634bf76b438 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:32:09.386135 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-08e2aeac-37b4-4c56-8fd7-8634bf76b438 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '77a17d04ccb1477cafd82ca23fd65328', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4490e1139a4046228fe39fbdc7973d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:32:09.393642 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-08e2aeac-37b4-4c56-8fd7-8634bf76b438 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] 10.222.0.16 "GET /compute/v2.1/servers/d9d4d400-1fed-4996-9d09-dd0dddd7fc42" status: 200 len: 1374 microversion: 2.1 time: 0.160438 Jul 16 22:32:09.396859 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3611/7220] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:32:09 2022] GET /compute/v2.1/servers/d9d4d400-1fed-4996-9d09-dd0dddd7fc42 => generated 1374 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:32:10.411342 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-ece27c8f-d7cc-4a4f-98a6-9e6145bfd1eb tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:32:10.412004 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-ece27c8f-d7cc-4a4f-98a6-9e6145bfd1eb tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] [instance: d9d4d400-1fed-4996-9d09-dd0dddd7fc42] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:32:10.420277 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ece27c8f-d7cc-4a4f-98a6-9e6145bfd1eb tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:32:10.420740 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ece27c8f-d7cc-4a4f-98a6-9e6145bfd1eb tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:32:10.421536 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ece27c8f-d7cc-4a4f-98a6-9e6145bfd1eb tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:32:10.421980 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ece27c8f-d7cc-4a4f-98a6-9e6145bfd1eb tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:32:10.525318 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-f0911be3-9f48-4d70-8086-568394069310 req-bcbb6d85-a8a5-4308-9741-ec8dbf0081bf service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "d9d4d400-1fed-4996-9d09-dd0dddd7fc42", "name": "network-vif-plugged", "status": "completed", "tag": "f6f604bc-d380-4063-b8fa-414495aada0b"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:32:10.534288 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-f0911be3-9f48-4d70-8086-568394069310 req-bcbb6d85-a8a5-4308-9741-ec8dbf0081bf service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:32:10.535044 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-f0911be3-9f48-4d70-8086-568394069310 req-bcbb6d85-a8a5-4308-9741-ec8dbf0081bf service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:32:10.547355 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-f0911be3-9f48-4d70-8086-568394069310 req-bcbb6d85-a8a5-4308-9741-ec8dbf0081bf service nova] Creating event network-vif-plugged:f6f604bc-d380-4063-b8fa-414495aada0b for instance d9d4d400-1fed-4996-9d09-dd0dddd7fc42 on nch1-master-1 Jul 16 22:32:10.552780 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-f0911be3-9f48-4d70-8086-568394069310 req-bcbb6d85-a8a5-4308-9741-ec8dbf0081bf service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.089089 Jul 16 22:32:10.553411 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3612/7221] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 22:32:10 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 16 22:32:10.578795 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-ece27c8f-d7cc-4a4f-98a6-9e6145bfd1eb tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '77a17d04ccb1477cafd82ca23fd65328', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4490e1139a4046228fe39fbdc7973d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:32:10.588955 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-ece27c8f-d7cc-4a4f-98a6-9e6145bfd1eb tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] 10.222.0.16 "GET /compute/v2.1/servers/d9d4d400-1fed-4996-9d09-dd0dddd7fc42" status: 200 len: 1374 microversion: 2.1 time: 0.181284 Jul 16 22:32:10.589825 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3610/7222] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:32:10 2022] GET /compute/v2.1/servers/d9d4d400-1fed-4996-9d09-dd0dddd7fc42 => generated 1374 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:32:11.603333 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell0:SELECT=6 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:32:11.608411 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-4beb9eb3-5e76-482d-a875-bbdb8cd258c3 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:32:11.609137 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-4beb9eb3-5e76-482d-a875-bbdb8cd258c3 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] [instance: d9d4d400-1fed-4996-9d09-dd0dddd7fc42] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:32:11.615420 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4beb9eb3-5e76-482d-a875-bbdb8cd258c3 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:32:11.615887 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4beb9eb3-5e76-482d-a875-bbdb8cd258c3 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:32:11.616666 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4beb9eb3-5e76-482d-a875-bbdb8cd258c3 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:32:11.617118 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-4beb9eb3-5e76-482d-a875-bbdb8cd258c3 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:32:11.754124 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-4beb9eb3-5e76-482d-a875-bbdb8cd258c3 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '77a17d04ccb1477cafd82ca23fd65328', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4490e1139a4046228fe39fbdc7973d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:32:11.762581 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-4beb9eb3-5e76-482d-a875-bbdb8cd258c3 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] 10.222.0.16 "GET /compute/v2.1/servers/d9d4d400-1fed-4996-9d09-dd0dddd7fc42" status: 200 len: 1374 microversion: 2.1 time: 0.160203 Jul 16 22:32:11.763237 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3613/7223] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:32:11 2022] GET /compute/v2.1/servers/d9d4d400-1fed-4996-9d09-dd0dddd7fc42 => generated 1374 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:32:12.780650 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-44916701-f990-4180-b664-7dda918077e1 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:32:12.781250 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-44916701-f990-4180-b664-7dda918077e1 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] [instance: d9d4d400-1fed-4996-9d09-dd0dddd7fc42] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:32:12.783144 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell0:SELECT=2 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:32:12.783828 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell0:SELECT=6 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:32:12.789477 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-44916701-f990-4180-b664-7dda918077e1 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:32:12.789932 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-44916701-f990-4180-b664-7dda918077e1 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:32:12.790689 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-44916701-f990-4180-b664-7dda918077e1 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:32:12.791157 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-44916701-f990-4180-b664-7dda918077e1 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:32:12.931213 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-44916701-f990-4180-b664-7dda918077e1 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '77a17d04ccb1477cafd82ca23fd65328', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4490e1139a4046228fe39fbdc7973d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:32:12.939665 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-44916701-f990-4180-b664-7dda918077e1 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] 10.222.0.16 "GET /compute/v2.1/servers/d9d4d400-1fed-4996-9d09-dd0dddd7fc42" status: 200 len: 1560 microversion: 2.1 time: 0.163946 Jul 16 22:32:12.940251 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3611/7224] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:32:12 2022] GET /compute/v2.1/servers/d9d4d400-1fed-4996-9d09-dd0dddd7fc42 => generated 1560 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:32:12.956865 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-ebe93044-40ba-437d-bcd1-7d0a486a8791 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:32:12.957559 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-ebe93044-40ba-437d-bcd1-7d0a486a8791 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] [instance: d9d4d400-1fed-4996-9d09-dd0dddd7fc42] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:32:12.964574 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ebe93044-40ba-437d-bcd1-7d0a486a8791 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:32:12.965201 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ebe93044-40ba-437d-bcd1-7d0a486a8791 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:32:12.965989 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ebe93044-40ba-437d-bcd1-7d0a486a8791 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:32:12.966455 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ebe93044-40ba-437d-bcd1-7d0a486a8791 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:32:13.103548 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-ebe93044-40ba-437d-bcd1-7d0a486a8791 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '77a17d04ccb1477cafd82ca23fd65328', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4490e1139a4046228fe39fbdc7973d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:32:13.112004 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-ebe93044-40ba-437d-bcd1-7d0a486a8791 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] 10.222.0.16 "GET /compute/v2.1/servers/d9d4d400-1fed-4996-9d09-dd0dddd7fc42" status: 200 len: 1560 microversion: 2.1 time: 0.158538 Jul 16 22:32:13.112004 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3614/7225] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:32:12 2022] GET /compute/v2.1/servers/d9d4d400-1fed-4996-9d09-dd0dddd7fc42 => generated 1560 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:32:15.495925 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-76b34f8a-d20c-4aa7-946c-5ecb38a67e0d req-729577c0-4136-4b36-b643-42261065d0b5 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "d9d4d400-1fed-4996-9d09-dd0dddd7fc42", "tag": "f6f604bc-d380-4063-b8fa-414495aada0b"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:32:15.504174 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-76b34f8a-d20c-4aa7-946c-5ecb38a67e0d req-729577c0-4136-4b36-b643-42261065d0b5 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:32:15.504626 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-76b34f8a-d20c-4aa7-946c-5ecb38a67e0d req-729577c0-4136-4b36-b643-42261065d0b5 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:32:15.516654 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-76b34f8a-d20c-4aa7-946c-5ecb38a67e0d req-729577c0-4136-4b36-b643-42261065d0b5 service nova] Creating event network-changed:f6f604bc-d380-4063-b8fa-414495aada0b for instance d9d4d400-1fed-4996-9d09-dd0dddd7fc42 on nch1-master-1 Jul 16 22:32:15.522062 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-76b34f8a-d20c-4aa7-946c-5ecb38a67e0d req-729577c0-4136-4b36-b643-42261065d0b5 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.089852 Jul 16 22:32:15.522686 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3612/7226] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 22:32:15 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 22:33:06.918925 ncdv-master-1 devstack@n-api.service[138681]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jul 16 22:33:06.919974 ncdv-master-1 devstack@n-api.service[138681]: Sat Jul 16 22:33:06 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Jul 16 22:34:10.551982 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_api:SELECT=29,nova_api:INSERT=1 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:34:10.552758 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=34 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:34:10.557140 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-88ef7e9e-fe81-4f9b-8fbb-7b3d8e504111 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Action: 'action', calling method: >, body: {"createImage": {"name": "tempest-TestSnapshotPatternsnapshot-1373808034"}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:34:10.558189 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-88ef7e9e-fe81-4f9b-8fbb-7b3d8e504111 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] [instance: d9d4d400-1fed-4996-9d09-dd0dddd7fc42] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:34:10.564249 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-88ef7e9e-fe81-4f9b-8fbb-7b3d8e504111 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:34:10.564703 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-88ef7e9e-fe81-4f9b-8fbb-7b3d8e504111 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:34:10.565476 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-88ef7e9e-fe81-4f9b-8fbb-7b3d8e504111 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:34:10.565919 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-88ef7e9e-fe81-4f9b-8fbb-7b3d8e504111 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:34:11.218814 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-88ef7e9e-fe81-4f9b-8fbb-7b3d8e504111 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] 10.222.0.16 "POST /compute/v2.1/servers/d9d4d400-1fed-4996-9d09-dd0dddd7fc42/action" status: 202 len: 0 microversion: 2.1 time: 0.668039 Jul 16 22:34:11.225767 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3615/7227] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 22:34:10 2022] POST /compute/v2.1/servers/d9d4d400-1fed-4996-9d09-dd0dddd7fc42/action => generated 0 bytes in 671 msecs (HTTP/1.1 202) 10 headers in 450 bytes (1 switches on core 0) Jul 16 22:34:16.940568 ncdv-master-1 devstack@n-api.service[138680]: ERROR oslo.messaging._drivers.impl_rabbit [-] [a302809d-3282-48f5-9662-a738982e3059] AMQP server on 10.222.0.16:5672 is unreachable: Too many heartbeats missed. Trying again in 1 seconds.: amqp.exceptions.ConnectionForced: Too many heartbeats missed Jul 16 22:34:16.941949 ncdv-master-1 devstack@n-api.service[138680]: ERROR oslo.messaging._drivers.impl_rabbit [-] [7d526b5e-7cab-48f3-be00-1b242d865b86] AMQP server on 10.222.0.16:5672 is unreachable: Too many heartbeats missed. Trying again in 1 seconds.: amqp.exceptions.ConnectionForced: Too many heartbeats missed Jul 16 22:34:16.942353 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_api:SELECT=29,nova_api:INSERT=3 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:34:16.943447 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=40 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:34:16.949727 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-ae84564b-3881-4433-a098-4c5255ec8621 req-612f6b8a-e0dc-48e4-a7e6-bb31fcb5b15d service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "d9d4d400-1fed-4996-9d09-dd0dddd7fc42", "name": "network-vif-plugged", "status": "completed", "tag": "f6f604bc-d380-4063-b8fa-414495aada0b"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:34:16.959263 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-ae84564b-3881-4433-a098-4c5255ec8621 req-612f6b8a-e0dc-48e4-a7e6-bb31fcb5b15d service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:34:16.960002 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-ae84564b-3881-4433-a098-4c5255ec8621 req-612f6b8a-e0dc-48e4-a7e6-bb31fcb5b15d service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:34:16.971894 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-ae84564b-3881-4433-a098-4c5255ec8621 req-612f6b8a-e0dc-48e4-a7e6-bb31fcb5b15d service nova] Creating event network-vif-plugged:f6f604bc-d380-4063-b8fa-414495aada0b for instance d9d4d400-1fed-4996-9d09-dd0dddd7fc42 on nch1-master-1 Jul 16 22:34:16.977461 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-ae84564b-3881-4433-a098-4c5255ec8621 req-612f6b8a-e0dc-48e4-a7e6-bb31fcb5b15d service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.039478 Jul 16 22:34:16.978066 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3613/7228] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 22:34:16 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 22:34:25.858716 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-dd91bba1-813f-4549-a24a-9c3b3e0c80d7 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestSnapshotPattern-server-1387194992", "imageRef": "5bd35c9f-f30e-44b1-bf6e-dbe1feb40ebf", "flavorRef": "452", "key_name": "tempest-TestSnapshotPattern-1466442200", "security_groups": [{"name": "tempest-secgroup-smoke-1333268523"}], "networks": [{"uuid": "496ad3c9-f791-43dc-a218-50a7eb7daba2"}]}} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:34:25.867305 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_api:SELECT=2 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:34:25.867915 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=12,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:34:25.970142 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.network.neutron [None req-dd91bba1-813f-4549-a24a-9c3b3e0c80d7 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] validate_networks() for [('496ad3c9-f791-43dc-a218-50a7eb7daba2', None, None, None, None, None)] {{(pid=138681) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 16 22:34:26.383517 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-dd91bba1-813f-4549-a24a-9c3b3e0c80d7 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Flavor limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 16 22:34:26.383957 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-dd91bba1-813f-4549-a24a-9c3b3e0c80d7 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Image limits 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 16 22:34:26.384448 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-dd91bba1-813f-4549-a24a-9c3b3e0c80d7 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Flavor pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 16 22:34:26.384866 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-dd91bba1-813f-4549-a24a-9c3b3e0c80d7 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Image pref 0:0:0 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 16 22:34:26.385289 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.virt.hardware [None req-dd91bba1-813f-4549-a24a-9c3b3e0c80d7 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=138681) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 16 22:34:26.394918 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-dd91bba1-813f-4549-a24a-9c3b3e0c80d7 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:34:26.395417 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-dd91bba1-813f-4549-a24a-9c3b3e0c80d7 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:34:26.398369 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-dd91bba1-813f-4549-a24a-9c3b3e0c80d7 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:34:26.398369 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-dd91bba1-813f-4549-a24a-9c3b3e0c80d7 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:34:26.413373 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-dd91bba1-813f-4549-a24a-9c3b3e0c80d7 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Getting quotas for project 4490e1139a4046228fe39fbdc7973d20. Resources: {'ram', 'instances', 'cores'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 16 22:34:26.418782 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.quota [None req-dd91bba1-813f-4549-a24a-9c3b3e0c80d7 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Getting quotas for user 77a17d04ccb1477cafd82ca23fd65328 and project 4490e1139a4046228fe39fbdc7973d20. Resources: {'ram', 'instances', 'cores'} {{(pid=138681) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 16 22:34:26.431639 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-dd91bba1-813f-4549-a24a-9c3b3e0c80d7 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Going to run 1 instances... {{(pid=138681) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 16 22:34:26.434123 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-dd91bba1-813f-4549-a24a-9c3b3e0c80d7 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] [instance: 2cd73456-1037-4243-8aa3-2a93c28b8f96] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='5bd35c9f-f30e-44b1-bf6e-dbe1feb40ebf',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=138681) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 16 22:34:26.552734 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-dd91bba1-813f-4549-a24a-9c3b3e0c80d7 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] 10.222.0.16 "POST /compute/v2.1/servers" status: 202 len: 406 microversion: 2.1 time: 0.698292 Jul 16 22:34:26.553395 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3616/7229] 10.222.0.16 () {60 vars in 1146 bytes} [Sat Jul 16 22:34:25 2022] POST /compute/v2.1/servers => generated 406 bytes in 700 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 16 22:34:26.566857 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-d5360183-1d37-4acb-ab72-1a02a85186c7 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:34:26.567603 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-d5360183-1d37-4acb-ab72-1a02a85186c7 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] [instance: 2cd73456-1037-4243-8aa3-2a93c28b8f96] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:34:26.579745 ncdv-master-1 devstack@n-api.service[138680]: INFO oslo.messaging._drivers.impl_rabbit [-] [7d526b5e-7cab-48f3-be00-1b242d865b86] Reconnected to AMQP server on 10.222.0.16:5672 via [amqp] client with port 35120. Jul 16 22:34:26.581698 ncdv-master-1 devstack@n-api.service[138680]: INFO oslo.messaging._drivers.impl_rabbit [-] [a302809d-3282-48f5-9662-a738982e3059] Reconnected to AMQP server on 10.222.0.16:5672 via [amqp] client with port 35118. Jul 16 22:34:26.680559 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-d5360183-1d37-4acb-ab72-1a02a85186c7 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '77a17d04ccb1477cafd82ca23fd65328', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4490e1139a4046228fe39fbdc7973d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:34:26.687357 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-d5360183-1d37-4acb-ab72-1a02a85186c7 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] 10.222.0.16 "GET /compute/v2.1/servers/2cd73456-1037-4243-8aa3-2a93c28b8f96" status: 200 len: 1249 microversion: 2.1 time: 0.127403 Jul 16 22:34:26.688109 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3614/7230] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:34:26 2022] GET /compute/v2.1/servers/2cd73456-1037-4243-8aa3-2a93c28b8f96 => generated 1249 bytes in 129 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:34:27.708729 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-93406034-8506-40b4-9a16-d465d5026842 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:34:27.712972 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-93406034-8506-40b4-9a16-d465d5026842 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] [instance: 2cd73456-1037-4243-8aa3-2a93c28b8f96] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:34:27.719691 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-93406034-8506-40b4-9a16-d465d5026842 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:34:27.720153 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-93406034-8506-40b4-9a16-d465d5026842 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:34:27.720937 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-93406034-8506-40b4-9a16-d465d5026842 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:34:27.721389 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-93406034-8506-40b4-9a16-d465d5026842 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:34:27.788677 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-93406034-8506-40b4-9a16-d465d5026842 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '77a17d04ccb1477cafd82ca23fd65328', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4490e1139a4046228fe39fbdc7973d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:34:27.796346 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-93406034-8506-40b4-9a16-d465d5026842 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] 10.222.0.16 "GET /compute/v2.1/servers/2cd73456-1037-4243-8aa3-2a93c28b8f96" status: 200 len: 1253 microversion: 2.1 time: 0.095216 Jul 16 22:34:27.796907 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3617/7231] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:34:27 2022] GET /compute/v2.1/servers/2cd73456-1037-4243-8aa3-2a93c28b8f96 => generated 1253 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:34:28.813917 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-1b715537-d95a-45da-83ca-b47c014382be tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:34:28.814660 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-1b715537-d95a-45da-83ca-b47c014382be tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] [instance: 2cd73456-1037-4243-8aa3-2a93c28b8f96] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:34:28.827431 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1b715537-d95a-45da-83ca-b47c014382be tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:34:28.827911 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1b715537-d95a-45da-83ca-b47c014382be tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:34:28.828681 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1b715537-d95a-45da-83ca-b47c014382be tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:34:28.829126 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-1b715537-d95a-45da-83ca-b47c014382be tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:34:28.905605 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-1b715537-d95a-45da-83ca-b47c014382be tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '77a17d04ccb1477cafd82ca23fd65328', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4490e1139a4046228fe39fbdc7973d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:34:28.914730 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-1b715537-d95a-45da-83ca-b47c014382be tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] 10.222.0.16 "GET /compute/v2.1/servers/2cd73456-1037-4243-8aa3-2a93c28b8f96" status: 200 len: 1309 microversion: 2.1 time: 0.104724 Jul 16 22:34:28.916204 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3615/7232] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:34:28 2022] GET /compute/v2.1/servers/2cd73456-1037-4243-8aa3-2a93c28b8f96 => generated 1309 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:34:28.970878 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-6adb24ef-dc21-4edc-9a75-508ef5b2f40e req-16d0c1e7-68f4-4232-a9b8-8f8146ca7ef4 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "d9d4d400-1fed-4996-9d09-dd0dddd7fc42", "name": "network-vif-unplugged", "status": "completed", "tag": "f6f604bc-d380-4063-b8fa-414495aada0b"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:34:28.981088 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-6adb24ef-dc21-4edc-9a75-508ef5b2f40e req-16d0c1e7-68f4-4232-a9b8-8f8146ca7ef4 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:34:28.981654 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-6adb24ef-dc21-4edc-9a75-508ef5b2f40e req-16d0c1e7-68f4-4232-a9b8-8f8146ca7ef4 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:34:28.993566 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-6adb24ef-dc21-4edc-9a75-508ef5b2f40e req-16d0c1e7-68f4-4232-a9b8-8f8146ca7ef4 service nova] Creating event network-vif-unplugged:f6f604bc-d380-4063-b8fa-414495aada0b for instance d9d4d400-1fed-4996-9d09-dd0dddd7fc42 on nch1-master-1 Jul 16 22:34:28.998843 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-6adb24ef-dc21-4edc-9a75-508ef5b2f40e req-16d0c1e7-68f4-4232-a9b8-8f8146ca7ef4 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.032382 Jul 16 22:34:28.999502 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3618/7233] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 22:34:28 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 22:34:29.932916 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-29b10820-142c-4974-8069-8edfc55e38e4 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:34:29.933540 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-29b10820-142c-4974-8069-8edfc55e38e4 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] [instance: 2cd73456-1037-4243-8aa3-2a93c28b8f96] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:34:29.941782 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-29b10820-142c-4974-8069-8edfc55e38e4 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:34:29.942328 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-29b10820-142c-4974-8069-8edfc55e38e4 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:34:29.943391 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-29b10820-142c-4974-8069-8edfc55e38e4 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:34:29.944076 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-29b10820-142c-4974-8069-8edfc55e38e4 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:34:30.094165 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-29b10820-142c-4974-8069-8edfc55e38e4 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '77a17d04ccb1477cafd82ca23fd65328', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4490e1139a4046228fe39fbdc7973d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:34:30.100920 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-29b10820-142c-4974-8069-8edfc55e38e4 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] 10.222.0.16 "GET /compute/v2.1/servers/2cd73456-1037-4243-8aa3-2a93c28b8f96" status: 200 len: 1375 microversion: 2.1 time: 0.172145 Jul 16 22:34:30.101520 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3616/7234] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:34:29 2022] GET /compute/v2.1/servers/2cd73456-1037-4243-8aa3-2a93c28b8f96 => generated 1375 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:34:31.119437 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-499f69cf-5ead-47fa-a04b-6f88add54dc0 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:34:31.120081 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-499f69cf-5ead-47fa-a04b-6f88add54dc0 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] [instance: 2cd73456-1037-4243-8aa3-2a93c28b8f96] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:34:31.127245 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-499f69cf-5ead-47fa-a04b-6f88add54dc0 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:34:31.127798 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-499f69cf-5ead-47fa-a04b-6f88add54dc0 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:34:31.128936 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-499f69cf-5ead-47fa-a04b-6f88add54dc0 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:34:31.129401 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-499f69cf-5ead-47fa-a04b-6f88add54dc0 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:34:31.282981 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-499f69cf-5ead-47fa-a04b-6f88add54dc0 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '77a17d04ccb1477cafd82ca23fd65328', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4490e1139a4046228fe39fbdc7973d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:34:31.293160 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-499f69cf-5ead-47fa-a04b-6f88add54dc0 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] 10.222.0.16 "GET /compute/v2.1/servers/2cd73456-1037-4243-8aa3-2a93c28b8f96" status: 200 len: 1375 microversion: 2.1 time: 0.178939 Jul 16 22:34:31.293904 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3619/7235] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:34:31 2022] GET /compute/v2.1/servers/2cd73456-1037-4243-8aa3-2a93c28b8f96 => generated 1375 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:34:32.311602 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-4690f932-847e-423b-bf99-2033140e489c tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:34:32.312342 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-4690f932-847e-423b-bf99-2033140e489c tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] [instance: 2cd73456-1037-4243-8aa3-2a93c28b8f96] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:34:32.320514 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4690f932-847e-423b-bf99-2033140e489c tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:34:32.321082 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4690f932-847e-423b-bf99-2033140e489c tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:34:32.321955 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4690f932-847e-423b-bf99-2033140e489c tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:34:32.322480 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4690f932-847e-423b-bf99-2033140e489c tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:34:32.473110 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-4690f932-847e-423b-bf99-2033140e489c tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '77a17d04ccb1477cafd82ca23fd65328', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4490e1139a4046228fe39fbdc7973d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:34:32.482382 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-4690f932-847e-423b-bf99-2033140e489c tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] 10.222.0.16 "GET /compute/v2.1/servers/2cd73456-1037-4243-8aa3-2a93c28b8f96" status: 200 len: 1375 microversion: 2.1 time: 0.174867 Jul 16 22:34:32.483044 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3617/7236] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:34:32 2022] GET /compute/v2.1/servers/2cd73456-1037-4243-8aa3-2a93c28b8f96 => generated 1375 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:34:33.504786 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-1c0bc6df-ca24-4179-8d7f-f9e105b21239 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:34:33.506179 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-1c0bc6df-ca24-4179-8d7f-f9e105b21239 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] [instance: 2cd73456-1037-4243-8aa3-2a93c28b8f96] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:34:33.525288 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1c0bc6df-ca24-4179-8d7f-f9e105b21239 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:34:33.525764 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1c0bc6df-ca24-4179-8d7f-f9e105b21239 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:34:33.526564 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1c0bc6df-ca24-4179-8d7f-f9e105b21239 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:34:33.527062 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-1c0bc6df-ca24-4179-8d7f-f9e105b21239 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:34:33.698768 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-1c0bc6df-ca24-4179-8d7f-f9e105b21239 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '77a17d04ccb1477cafd82ca23fd65328', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4490e1139a4046228fe39fbdc7973d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:34:33.706767 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-1c0bc6df-ca24-4179-8d7f-f9e105b21239 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] 10.222.0.16 "GET /compute/v2.1/servers/2cd73456-1037-4243-8aa3-2a93c28b8f96" status: 200 len: 1375 microversion: 2.1 time: 0.206292 Jul 16 22:34:33.707382 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3620/7237] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:34:33 2022] GET /compute/v2.1/servers/2cd73456-1037-4243-8aa3-2a93c28b8f96 => generated 1375 bytes in 208 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:34:34.732249 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-f8d3b46a-1155-486d-b819-28b9d2df0268 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:34:34.741325 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-f8d3b46a-1155-486d-b819-28b9d2df0268 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] [instance: 2cd73456-1037-4243-8aa3-2a93c28b8f96] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:34:34.744991 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f8d3b46a-1155-486d-b819-28b9d2df0268 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:34:34.748902 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f8d3b46a-1155-486d-b819-28b9d2df0268 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:34:34.750117 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f8d3b46a-1155-486d-b819-28b9d2df0268 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:34:34.750810 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f8d3b46a-1155-486d-b819-28b9d2df0268 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:34:34.947769 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-362fade6-b2de-45da-b60b-3217642489d9 req-720b702e-883c-4638-97dc-f14c1d5eacef service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "2cd73456-1037-4243-8aa3-2a93c28b8f96", "tag": "6a501eec-2053-4dda-96da-d7a37de33e06"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:34:34.965429 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-362fade6-b2de-45da-b60b-3217642489d9 req-720b702e-883c-4638-97dc-f14c1d5eacef service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:34:34.966229 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-362fade6-b2de-45da-b60b-3217642489d9 req-720b702e-883c-4638-97dc-f14c1d5eacef service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:34:34.982557 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-362fade6-b2de-45da-b60b-3217642489d9 req-720b702e-883c-4638-97dc-f14c1d5eacef service nova] Creating event network-changed:6a501eec-2053-4dda-96da-d7a37de33e06 for instance 2cd73456-1037-4243-8aa3-2a93c28b8f96 on nch2-master-1 Jul 16 22:34:34.988591 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-f8d3b46a-1155-486d-b819-28b9d2df0268 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '77a17d04ccb1477cafd82ca23fd65328', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4490e1139a4046228fe39fbdc7973d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:34:34.996442 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-362fade6-b2de-45da-b60b-3217642489d9 req-720b702e-883c-4638-97dc-f14c1d5eacef service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.054790 Jul 16 22:34:34.997813 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3621/7238] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 22:34:34 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 16 22:34:34.999819 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-f8d3b46a-1155-486d-b819-28b9d2df0268 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] 10.222.0.16 "GET /compute/v2.1/servers/2cd73456-1037-4243-8aa3-2a93c28b8f96" status: 200 len: 1375 microversion: 2.1 time: 0.271008 Jul 16 22:34:35.000407 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3618/7239] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:34:34 2022] GET /compute/v2.1/servers/2cd73456-1037-4243-8aa3-2a93c28b8f96 => generated 1375 bytes in 273 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:34:36.030358 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-e9f9d59c-0ab2-4f59-a352-0e438b328201 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:34:36.031321 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-e9f9d59c-0ab2-4f59-a352-0e438b328201 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] [instance: 2cd73456-1037-4243-8aa3-2a93c28b8f96] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:34:36.040942 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e9f9d59c-0ab2-4f59-a352-0e438b328201 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:34:36.041410 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e9f9d59c-0ab2-4f59-a352-0e438b328201 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:34:36.042197 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e9f9d59c-0ab2-4f59-a352-0e438b328201 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:34:36.042646 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e9f9d59c-0ab2-4f59-a352-0e438b328201 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:34:36.217327 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-e9f9d59c-0ab2-4f59-a352-0e438b328201 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '77a17d04ccb1477cafd82ca23fd65328', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4490e1139a4046228fe39fbdc7973d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:34:36.225711 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-e9f9d59c-0ab2-4f59-a352-0e438b328201 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] 10.222.0.16 "GET /compute/v2.1/servers/2cd73456-1037-4243-8aa3-2a93c28b8f96" status: 200 len: 1375 microversion: 2.1 time: 0.200306 Jul 16 22:34:36.226301 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3622/7240] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:34:36 2022] GET /compute/v2.1/servers/2cd73456-1037-4243-8aa3-2a93c28b8f96 => generated 1375 bytes in 202 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:34:37.250690 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-5278703e-d5db-48f2-a137-17ccfd298f10 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:34:37.252280 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-5278703e-d5db-48f2-a137-17ccfd298f10 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] [instance: 2cd73456-1037-4243-8aa3-2a93c28b8f96] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:34:37.262129 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell0:SELECT=2 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:34:37.265091 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5278703e-d5db-48f2-a137-17ccfd298f10 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:34:37.265948 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5278703e-d5db-48f2-a137-17ccfd298f10 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:34:37.268492 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5278703e-d5db-48f2-a137-17ccfd298f10 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:34:37.270738 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-5278703e-d5db-48f2-a137-17ccfd298f10 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:34:37.450210 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-5278703e-d5db-48f2-a137-17ccfd298f10 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '77a17d04ccb1477cafd82ca23fd65328', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4490e1139a4046228fe39fbdc7973d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:34:37.458944 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-5278703e-d5db-48f2-a137-17ccfd298f10 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] 10.222.0.16 "GET /compute/v2.1/servers/2cd73456-1037-4243-8aa3-2a93c28b8f96" status: 200 len: 1375 microversion: 2.1 time: 0.213810 Jul 16 22:34:37.460754 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3619/7241] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:34:37 2022] GET /compute/v2.1/servers/2cd73456-1037-4243-8aa3-2a93c28b8f96 => generated 1375 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:34:38.476642 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-fdaceed6-a894-4916-b9d4-14b4f4e87fed tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:34:38.477344 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-fdaceed6-a894-4916-b9d4-14b4f4e87fed tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] [instance: 2cd73456-1037-4243-8aa3-2a93c28b8f96] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:34:38.479376 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell0:SELECT=2 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:34:38.480080 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell0:SELECT=6 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:34:38.486158 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fdaceed6-a894-4916-b9d4-14b4f4e87fed tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:34:38.486634 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fdaceed6-a894-4916-b9d4-14b4f4e87fed tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:34:38.487576 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fdaceed6-a894-4916-b9d4-14b4f4e87fed tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:34:38.488041 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-fdaceed6-a894-4916-b9d4-14b4f4e87fed tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:34:38.623795 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-fdaceed6-a894-4916-b9d4-14b4f4e87fed tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '77a17d04ccb1477cafd82ca23fd65328', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4490e1139a4046228fe39fbdc7973d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:34:38.632290 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-fdaceed6-a894-4916-b9d4-14b4f4e87fed tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] 10.222.0.16 "GET /compute/v2.1/servers/2cd73456-1037-4243-8aa3-2a93c28b8f96" status: 200 len: 1375 microversion: 2.1 time: 0.159700 Jul 16 22:34:38.633143 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3623/7242] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:34:38 2022] GET /compute/v2.1/servers/2cd73456-1037-4243-8aa3-2a93c28b8f96 => generated 1375 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:34:39.649706 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-f759b6f0-62a9-4796-bec6-0aa909f06878 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:34:39.650387 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-f759b6f0-62a9-4796-bec6-0aa909f06878 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] [instance: 2cd73456-1037-4243-8aa3-2a93c28b8f96] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:34:39.657261 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f759b6f0-62a9-4796-bec6-0aa909f06878 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:34:39.657712 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f759b6f0-62a9-4796-bec6-0aa909f06878 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:34:39.658508 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f759b6f0-62a9-4796-bec6-0aa909f06878 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:34:39.658943 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f759b6f0-62a9-4796-bec6-0aa909f06878 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:34:39.793799 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-f759b6f0-62a9-4796-bec6-0aa909f06878 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '77a17d04ccb1477cafd82ca23fd65328', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4490e1139a4046228fe39fbdc7973d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:34:39.801281 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-f759b6f0-62a9-4796-bec6-0aa909f06878 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] 10.222.0.16 "GET /compute/v2.1/servers/2cd73456-1037-4243-8aa3-2a93c28b8f96" status: 200 len: 1375 microversion: 2.1 time: 0.156154 Jul 16 22:34:39.801899 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3620/7243] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:34:39 2022] GET /compute/v2.1/servers/2cd73456-1037-4243-8aa3-2a93c28b8f96 => generated 1375 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:34:40.818510 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-0890f683-e87d-4a58-9c1b-58df23e48d97 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:34:40.819183 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-0890f683-e87d-4a58-9c1b-58df23e48d97 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] [instance: 2cd73456-1037-4243-8aa3-2a93c28b8f96] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:34:40.826310 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0890f683-e87d-4a58-9c1b-58df23e48d97 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:34:40.826778 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0890f683-e87d-4a58-9c1b-58df23e48d97 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:34:40.827594 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0890f683-e87d-4a58-9c1b-58df23e48d97 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:34:40.828056 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-0890f683-e87d-4a58-9c1b-58df23e48d97 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:34:40.966787 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-0890f683-e87d-4a58-9c1b-58df23e48d97 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '77a17d04ccb1477cafd82ca23fd65328', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4490e1139a4046228fe39fbdc7973d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:34:40.975172 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-0890f683-e87d-4a58-9c1b-58df23e48d97 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] 10.222.0.16 "GET /compute/v2.1/servers/2cd73456-1037-4243-8aa3-2a93c28b8f96" status: 200 len: 1375 microversion: 2.1 time: 0.160528 Jul 16 22:34:40.975772 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3624/7244] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:34:40 2022] GET /compute/v2.1/servers/2cd73456-1037-4243-8aa3-2a93c28b8f96 => generated 1375 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:34:41.991736 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-cea22251-1c66-45aa-a2ee-92f7f7e8a606 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:34:41.992355 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-cea22251-1c66-45aa-a2ee-92f7f7e8a606 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] [instance: 2cd73456-1037-4243-8aa3-2a93c28b8f96] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:34:41.999305 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cea22251-1c66-45aa-a2ee-92f7f7e8a606 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:34:41.999764 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cea22251-1c66-45aa-a2ee-92f7f7e8a606 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:34:42.000531 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cea22251-1c66-45aa-a2ee-92f7f7e8a606 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:34:42.000976 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cea22251-1c66-45aa-a2ee-92f7f7e8a606 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:34:42.141433 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-cea22251-1c66-45aa-a2ee-92f7f7e8a606 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '77a17d04ccb1477cafd82ca23fd65328', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4490e1139a4046228fe39fbdc7973d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:34:42.149201 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-cea22251-1c66-45aa-a2ee-92f7f7e8a606 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] 10.222.0.16 "GET /compute/v2.1/servers/2cd73456-1037-4243-8aa3-2a93c28b8f96" status: 200 len: 1375 microversion: 2.1 time: 0.162003 Jul 16 22:34:42.149875 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3621/7245] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:34:41 2022] GET /compute/v2.1/servers/2cd73456-1037-4243-8aa3-2a93c28b8f96 => generated 1375 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:34:43.166136 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-615f84ce-0c81-4f75-9d71-3ff2657147cb tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:34:43.166851 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-615f84ce-0c81-4f75-9d71-3ff2657147cb tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] [instance: 2cd73456-1037-4243-8aa3-2a93c28b8f96] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:34:43.173786 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-615f84ce-0c81-4f75-9d71-3ff2657147cb tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:34:43.174253 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-615f84ce-0c81-4f75-9d71-3ff2657147cb tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:34:43.175062 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-615f84ce-0c81-4f75-9d71-3ff2657147cb tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:34:43.175554 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-615f84ce-0c81-4f75-9d71-3ff2657147cb tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:34:43.310125 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-615f84ce-0c81-4f75-9d71-3ff2657147cb tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '77a17d04ccb1477cafd82ca23fd65328', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4490e1139a4046228fe39fbdc7973d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:34:43.318665 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-615f84ce-0c81-4f75-9d71-3ff2657147cb tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] 10.222.0.16 "GET /compute/v2.1/servers/2cd73456-1037-4243-8aa3-2a93c28b8f96" status: 200 len: 1375 microversion: 2.1 time: 0.156667 Jul 16 22:34:43.319725 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3625/7246] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:34:43 2022] GET /compute/v2.1/servers/2cd73456-1037-4243-8aa3-2a93c28b8f96 => generated 1375 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:34:44.294550 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [req-796ae47b-a9ba-459e-85c0-f6f662596499 req-85abfbbc-18a9-4606-9e1c-9da470464d4b service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "2cd73456-1037-4243-8aa3-2a93c28b8f96", "name": "network-vif-plugged", "status": "completed", "tag": "6a501eec-2053-4dda-96da-d7a37de33e06"}]} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:34:44.304282 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-796ae47b-a9ba-459e-85c0-f6f662596499 req-85abfbbc-18a9-4606-9e1c-9da470464d4b service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:34:44.304883 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [req-796ae47b-a9ba-459e-85c0-f6f662596499 req-85abfbbc-18a9-4606-9e1c-9da470464d4b service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:34:44.315893 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.compute.server_external_events [req-796ae47b-a9ba-459e-85c0-f6f662596499 req-85abfbbc-18a9-4606-9e1c-9da470464d4b service nova] Creating event network-vif-plugged:6a501eec-2053-4dda-96da-d7a37de33e06 for instance 2cd73456-1037-4243-8aa3-2a93c28b8f96 on nch2-master-1 Jul 16 22:34:44.322247 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [req-796ae47b-a9ba-459e-85c0-f6f662596499 req-85abfbbc-18a9-4606-9e1c-9da470464d4b service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.031525 Jul 16 22:34:44.322809 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3622/7247] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 22:34:44 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 22:34:44.336477 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-7832f190-4cbe-479d-8583-562f94e61f08 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:34:44.337136 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-7832f190-4cbe-479d-8583-562f94e61f08 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] [instance: 2cd73456-1037-4243-8aa3-2a93c28b8f96] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:34:44.345128 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7832f190-4cbe-479d-8583-562f94e61f08 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:34:44.345682 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7832f190-4cbe-479d-8583-562f94e61f08 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:34:44.346922 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7832f190-4cbe-479d-8583-562f94e61f08 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:34:44.347611 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-7832f190-4cbe-479d-8583-562f94e61f08 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:34:44.496742 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-7832f190-4cbe-479d-8583-562f94e61f08 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '77a17d04ccb1477cafd82ca23fd65328', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4490e1139a4046228fe39fbdc7973d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:34:44.504505 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-7832f190-4cbe-479d-8583-562f94e61f08 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] 10.222.0.16 "GET /compute/v2.1/servers/2cd73456-1037-4243-8aa3-2a93c28b8f96" status: 200 len: 1375 microversion: 2.1 time: 0.173129 Jul 16 22:34:44.505087 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3626/7248] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:34:44 2022] GET /compute/v2.1/servers/2cd73456-1037-4243-8aa3-2a93c28b8f96 => generated 1375 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:34:44.826237 ncdv-master-1 devstack@n-api.service[138680]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jul 16 22:34:44.826464 ncdv-master-1 devstack@n-api.service[138680]: Sat Jul 16 22:34:44 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Jul 16 22:34:45.048309 ncdv-master-1 devstack@n-api.service[138680]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jul 16 22:34:45.048505 ncdv-master-1 devstack@n-api.service[138680]: Sat Jul 16 22:34:45 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Jul 16 22:34:45.524126 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-d2b2601f-4142-4ade-a1d4-34ff2de66f46 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:34:45.524832 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-d2b2601f-4142-4ade-a1d4-34ff2de66f46 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] [instance: 2cd73456-1037-4243-8aa3-2a93c28b8f96] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:34:45.531872 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d2b2601f-4142-4ade-a1d4-34ff2de66f46 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:34:45.532331 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d2b2601f-4142-4ade-a1d4-34ff2de66f46 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:34:45.533096 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d2b2601f-4142-4ade-a1d4-34ff2de66f46 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:34:45.533538 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-d2b2601f-4142-4ade-a1d4-34ff2de66f46 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:34:45.676672 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-d2b2601f-4142-4ade-a1d4-34ff2de66f46 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '77a17d04ccb1477cafd82ca23fd65328', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4490e1139a4046228fe39fbdc7973d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:34:45.681343 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-d2b2601f-4142-4ade-a1d4-34ff2de66f46 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] 10.222.0.16 "GET /compute/v2.1/servers/2cd73456-1037-4243-8aa3-2a93c28b8f96" status: 200 len: 1375 microversion: 2.1 time: 0.163031 Jul 16 22:34:45.681990 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3623/7249] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:34:45 2022] GET /compute/v2.1/servers/2cd73456-1037-4243-8aa3-2a93c28b8f96 => generated 1375 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:34:46.174125 ncdv-master-1 devstack@n-api.service[138681]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jul 16 22:34:46.174692 ncdv-master-1 devstack@n-api.service[138681]: Sat Jul 16 22:34:46 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Jul 16 22:34:46.699053 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-c7f70073-e073-4161-8323-5fcd602c2897 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:34:46.699683 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-c7f70073-e073-4161-8323-5fcd602c2897 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] [instance: 2cd73456-1037-4243-8aa3-2a93c28b8f96] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:34:46.706554 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c7f70073-e073-4161-8323-5fcd602c2897 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:34:46.707033 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c7f70073-e073-4161-8323-5fcd602c2897 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:34:46.707824 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c7f70073-e073-4161-8323-5fcd602c2897 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:34:46.708273 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-c7f70073-e073-4161-8323-5fcd602c2897 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:34:46.841930 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-c7f70073-e073-4161-8323-5fcd602c2897 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '77a17d04ccb1477cafd82ca23fd65328', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4490e1139a4046228fe39fbdc7973d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:34:46.854762 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-c7f70073-e073-4161-8323-5fcd602c2897 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] 10.222.0.16 "GET /compute/v2.1/servers/2cd73456-1037-4243-8aa3-2a93c28b8f96" status: 200 len: 1561 microversion: 2.1 time: 0.160314 Jul 16 22:34:46.855442 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3627/7250] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:34:46 2022] GET /compute/v2.1/servers/2cd73456-1037-4243-8aa3-2a93c28b8f96 => generated 1561 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:34:46.870810 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-e781c84a-d615-4c48-a6bc-c522d2170ecd tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:34:46.871460 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-e781c84a-d615-4c48-a6bc-c522d2170ecd tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] [instance: 2cd73456-1037-4243-8aa3-2a93c28b8f96] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:34:46.878391 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e781c84a-d615-4c48-a6bc-c522d2170ecd tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:34:46.878845 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e781c84a-d615-4c48-a6bc-c522d2170ecd tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:34:46.879665 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e781c84a-d615-4c48-a6bc-c522d2170ecd tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:34:46.880112 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e781c84a-d615-4c48-a6bc-c522d2170ecd tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:34:47.015667 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-e781c84a-d615-4c48-a6bc-c522d2170ecd tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '77a17d04ccb1477cafd82ca23fd65328', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4490e1139a4046228fe39fbdc7973d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:34:47.023764 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-e781c84a-d615-4c48-a6bc-c522d2170ecd tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] 10.222.0.16 "GET /compute/v2.1/servers/2cd73456-1037-4243-8aa3-2a93c28b8f96" status: 200 len: 1561 microversion: 2.1 time: 0.156901 Jul 16 22:34:47.024338 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3624/7251] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:34:46 2022] GET /compute/v2.1/servers/2cd73456-1037-4243-8aa3-2a93c28b8f96 => generated 1561 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:34:48.989463 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-8d2d61ba-7d83-4516-bc12-21fed76e8ee9 req-65f7fce1-d9f4-4898-9f32-a02754021dad service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "2cd73456-1037-4243-8aa3-2a93c28b8f96", "tag": "6a501eec-2053-4dda-96da-d7a37de33e06"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:34:48.999176 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-8d2d61ba-7d83-4516-bc12-21fed76e8ee9 req-65f7fce1-d9f4-4898-9f32-a02754021dad service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:34:48.999644 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-8d2d61ba-7d83-4516-bc12-21fed76e8ee9 req-65f7fce1-d9f4-4898-9f32-a02754021dad service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:34:49.012058 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-8d2d61ba-7d83-4516-bc12-21fed76e8ee9 req-65f7fce1-d9f4-4898-9f32-a02754021dad service nova] Creating event network-changed:6a501eec-2053-4dda-96da-d7a37de33e06 for instance 2cd73456-1037-4243-8aa3-2a93c28b8f96 on nch2-master-1 Jul 16 22:34:49.017359 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-8d2d61ba-7d83-4516-bc12-21fed76e8ee9 req-65f7fce1-d9f4-4898-9f32-a02754021dad service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.031594 Jul 16 22:34:49.018335 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3628/7252] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 22:34:48 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 22:36:17.500129 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_api:SELECT=26 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:36:17.500897 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=58 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:36:17.504967 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-cc060142-69aa-42f1-ae94-e3565184775d tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Action: 'action', calling method: >, body: {"createImage": {"name": "tempest-TestSnapshotPatternsnapshot-1622758898"}} {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:36:17.506008 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-cc060142-69aa-42f1-ae94-e3565184775d tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] [instance: 2cd73456-1037-4243-8aa3-2a93c28b8f96] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:36:17.511991 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cc060142-69aa-42f1-ae94-e3565184775d tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:36:17.516103 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cc060142-69aa-42f1-ae94-e3565184775d tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:36:17.516103 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cc060142-69aa-42f1-ae94-e3565184775d tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:36:17.516103 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-cc060142-69aa-42f1-ae94-e3565184775d tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:36:17.994368 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-cc060142-69aa-42f1-ae94-e3565184775d tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] 10.222.0.16 "POST /compute/v2.1/servers/2cd73456-1037-4243-8aa3-2a93c28b8f96/action" status: 202 len: 0 microversion: 2.1 time: 0.496079 Jul 16 22:36:17.995118 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3625/7253] 10.222.0.16 () {60 vars in 1277 bytes} [Sat Jul 16 22:36:17 2022] POST /compute/v2.1/servers/2cd73456-1037-4243-8aa3-2a93c28b8f96/action => generated 0 bytes in 498 msecs (HTTP/1.1 202) 10 headers in 450 bytes (1 switches on core 0) Jul 16 22:36:24.852846 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-6495ff5a-6f0a-4eff-855e-a1ff946953f0 req-e4fe645b-2dd9-4f42-b84f-f04452109ffb service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "2cd73456-1037-4243-8aa3-2a93c28b8f96", "name": "network-vif-plugged", "status": "completed", "tag": "6a501eec-2053-4dda-96da-d7a37de33e06"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:36:24.863681 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_api:SELECT=39,nova_api:INSERT=3 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:36:24.864937 ncdv-master-1 devstack@n-api.service[138681]: DEBUG dbcounter [-] [138681] Writing DB stats nova_cell1:SELECT=62 {{(pid=138681) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:36:24.868567 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-6495ff5a-6f0a-4eff-855e-a1ff946953f0 req-e4fe645b-2dd9-4f42-b84f-f04452109ffb service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:36:24.869752 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-6495ff5a-6f0a-4eff-855e-a1ff946953f0 req-e4fe645b-2dd9-4f42-b84f-f04452109ffb service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:36:24.884140 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-6495ff5a-6f0a-4eff-855e-a1ff946953f0 req-e4fe645b-2dd9-4f42-b84f-f04452109ffb service nova] Creating event network-vif-plugged:6a501eec-2053-4dda-96da-d7a37de33e06 for instance 2cd73456-1037-4243-8aa3-2a93c28b8f96 on nch2-master-1 Jul 16 22:36:24.891657 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-6495ff5a-6f0a-4eff-855e-a1ff946953f0 req-e4fe645b-2dd9-4f42-b84f-f04452109ffb service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.046147 Jul 16 22:36:24.893210 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3629/7254] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 22:36:24 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 22:36:34.655103 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_api:SELECT=2 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:36:34.655908 ncdv-master-1 devstack@n-api.service[138680]: DEBUG dbcounter [-] [138680] Writing DB stats nova_cell1:SELECT=12,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=138680) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 16 22:36:34.658254 ncdv-master-1 devstack@n-api.service[138681]: ERROR oslo.messaging._drivers.impl_rabbit [-] [e4397b8a-69d0-4b1f-ae48-d345afeb4816] AMQP server on 10.222.0.16:5672 is unreachable: . Trying again in 1 seconds.: amqp.exceptions.RecoverableConnectionError: Jul 16 22:36:34.659963 ncdv-master-1 devstack@n-api.service[138681]: ERROR oslo.messaging._drivers.impl_rabbit [-] [4de8847c-05a2-4cd6-bd0e-256d7a754675] AMQP server on 10.222.0.16:5672 is unreachable: . Trying again in 1 seconds.: amqp.exceptions.RecoverableConnectionError: Jul 16 22:36:34.660095 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-4106894a-f39b-4aac-965f-3877f6a53049 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:36:34.660601 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-4106894a-f39b-4aac-965f-3877f6a53049 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] [instance: 2cd73456-1037-4243-8aa3-2a93c28b8f96] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:36:34.667251 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4106894a-f39b-4aac-965f-3877f6a53049 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:36:34.667478 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4106894a-f39b-4aac-965f-3877f6a53049 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:36:34.668237 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4106894a-f39b-4aac-965f-3877f6a53049 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:36:34.668678 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-4106894a-f39b-4aac-965f-3877f6a53049 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:36:34.697431 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-4106894a-f39b-4aac-965f-3877f6a53049 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] [instance: 2cd73456-1037-4243-8aa3-2a93c28b8f96] Going to try to terminate instance {{(pid=138680) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 16 22:36:34.758308 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-6c3526e4-11b1-49ce-8374-51b271c0e378 req-1fcce42f-9478-4353-8d79-ab12e39c0384 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "2cd73456-1037-4243-8aa3-2a93c28b8f96", "tag": "6a501eec-2053-4dda-96da-d7a37de33e06"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:36:34.771167 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-6c3526e4-11b1-49ce-8374-51b271c0e378 req-1fcce42f-9478-4353-8d79-ab12e39c0384 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:36:34.775321 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-6c3526e4-11b1-49ce-8374-51b271c0e378 req-1fcce42f-9478-4353-8d79-ab12e39c0384 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:36:34.788460 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-6c3526e4-11b1-49ce-8374-51b271c0e378 req-1fcce42f-9478-4353-8d79-ab12e39c0384 service nova] Creating event network-changed:6a501eec-2053-4dda-96da-d7a37de33e06 for instance 2cd73456-1037-4243-8aa3-2a93c28b8f96 on nch2-master-1 Jul 16 22:36:34.794131 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-6c3526e4-11b1-49ce-8374-51b271c0e378 req-1fcce42f-9478-4353-8d79-ab12e39c0384 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.137810 Jul 16 22:36:34.795032 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3630/7255] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 22:36:34 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 140 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 22:36:34.900755 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-b7a81524-b717-4a15-bfe8-92df3d1ea50d req-8aece3d6-15d5-4915-bff2-4c60cb256d7a service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "2cd73456-1037-4243-8aa3-2a93c28b8f96", "name": "network-vif-unplugged", "status": "completed", "tag": "6a501eec-2053-4dda-96da-d7a37de33e06"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:36:34.914782 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-b7a81524-b717-4a15-bfe8-92df3d1ea50d req-8aece3d6-15d5-4915-bff2-4c60cb256d7a service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:36:34.915342 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-b7a81524-b717-4a15-bfe8-92df3d1ea50d req-8aece3d6-15d5-4915-bff2-4c60cb256d7a service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:36:34.928116 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-b7a81524-b717-4a15-bfe8-92df3d1ea50d req-8aece3d6-15d5-4915-bff2-4c60cb256d7a service nova] Creating event network-vif-unplugged:6a501eec-2053-4dda-96da-d7a37de33e06 for instance 2cd73456-1037-4243-8aa3-2a93c28b8f96 on nch2-master-1 Jul 16 22:36:34.933530 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-b7a81524-b717-4a15-bfe8-92df3d1ea50d req-8aece3d6-15d5-4915-bff2-4c60cb256d7a service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.037449 Jul 16 22:36:34.934225 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3631/7256] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 22:36:34 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 16 22:36:35.073424 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-4106894a-f39b-4aac-965f-3877f6a53049 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] 10.222.0.16 "DELETE /compute/v2.1/servers/2cd73456-1037-4243-8aa3-2a93c28b8f96" status: 204 len: 0 microversion: 2.1 time: 0.420367 Jul 16 22:36:35.074081 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3626/7257] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 22:36:34 2022] DELETE /compute/v2.1/servers/2cd73456-1037-4243-8aa3-2a93c28b8f96 => generated 0 bytes in 422 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 16 22:36:35.083800 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-e5c0dd2f-4ed7-4e27-94d0-b31cdcf749e9 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:36:35.084521 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-e5c0dd2f-4ed7-4e27-94d0-b31cdcf749e9 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] [instance: 2cd73456-1037-4243-8aa3-2a93c28b8f96] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:36:35.091145 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e5c0dd2f-4ed7-4e27-94d0-b31cdcf749e9 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:36:35.091630 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e5c0dd2f-4ed7-4e27-94d0-b31cdcf749e9 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:36:35.092451 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e5c0dd2f-4ed7-4e27-94d0-b31cdcf749e9 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:36:35.093442 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-e5c0dd2f-4ed7-4e27-94d0-b31cdcf749e9 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:36:35.246399 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-e5c0dd2f-4ed7-4e27-94d0-b31cdcf749e9 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '77a17d04ccb1477cafd82ca23fd65328', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4490e1139a4046228fe39fbdc7973d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:36:35.254407 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-e5c0dd2f-4ed7-4e27-94d0-b31cdcf749e9 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] 10.222.0.16 "GET /compute/v2.1/servers/2cd73456-1037-4243-8aa3-2a93c28b8f96" status: 200 len: 1686 microversion: 2.1 time: 0.175455 Jul 16 22:36:35.254985 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3632/7258] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:36:35 2022] GET /compute/v2.1/servers/2cd73456-1037-4243-8aa3-2a93c28b8f96 => generated 1686 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:36:36.272648 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-7340ec17-bd86-4b51-9d39-0884d989e0ab tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:36:36.273562 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-7340ec17-bd86-4b51-9d39-0884d989e0ab tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] [instance: 2cd73456-1037-4243-8aa3-2a93c28b8f96] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:36:36.280487 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7340ec17-bd86-4b51-9d39-0884d989e0ab tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:36:36.280934 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7340ec17-bd86-4b51-9d39-0884d989e0ab tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:36:36.281759 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7340ec17-bd86-4b51-9d39-0884d989e0ab tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:36:36.282203 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-7340ec17-bd86-4b51-9d39-0884d989e0ab tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:36:36.417270 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-7340ec17-bd86-4b51-9d39-0884d989e0ab tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '77a17d04ccb1477cafd82ca23fd65328', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4490e1139a4046228fe39fbdc7973d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:36:36.425420 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-7340ec17-bd86-4b51-9d39-0884d989e0ab tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] 10.222.0.16 "GET /compute/v2.1/servers/2cd73456-1037-4243-8aa3-2a93c28b8f96" status: 200 len: 1686 microversion: 2.1 time: 0.157095 Jul 16 22:36:36.425989 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3627/7259] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:36:36 2022] GET /compute/v2.1/servers/2cd73456-1037-4243-8aa3-2a93c28b8f96 => generated 1686 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:36:37.408949 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-176812e0-4217-489c-8eb5-0b1f3c5e5f3d req-fa06d07c-1e96-431d-9847-c2d176416626 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "2cd73456-1037-4243-8aa3-2a93c28b8f96", "name": "network-vif-deleted", "tag": "6a501eec-2053-4dda-96da-d7a37de33e06"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:36:37.423826 ncdv-master-1 devstack@n-api.service[138681]: INFO oslo.messaging._drivers.impl_rabbit [-] [e4397b8a-69d0-4b1f-ae48-d345afeb4816] Reconnected to AMQP server on 10.222.0.16:5672 via [amqp] client with port 35466. Jul 16 22:36:37.430769 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-176812e0-4217-489c-8eb5-0b1f3c5e5f3d req-fa06d07c-1e96-431d-9847-c2d176416626 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:36:37.431252 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-176812e0-4217-489c-8eb5-0b1f3c5e5f3d req-fa06d07c-1e96-431d-9847-c2d176416626 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:36:37.441671 ncdv-master-1 devstack@n-api.service[138681]: INFO oslo.messaging._drivers.impl_rabbit [-] [4de8847c-05a2-4cd6-bd0e-256d7a754675] Reconnected to AMQP server on 10.222.0.16:5672 via [amqp] client with port 35468. Jul 16 22:36:37.443287 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-53b159e1-dd85-4021-9098-7ec0fadfee43 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:36:37.448683 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-53b159e1-dd85-4021-9098-7ec0fadfee43 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] [instance: 2cd73456-1037-4243-8aa3-2a93c28b8f96] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:36:37.448820 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-176812e0-4217-489c-8eb5-0b1f3c5e5f3d req-fa06d07c-1e96-431d-9847-c2d176416626 service nova] Creating event network-vif-deleted:6a501eec-2053-4dda-96da-d7a37de33e06 for instance 2cd73456-1037-4243-8aa3-2a93c28b8f96 on nch2-master-1 Jul 16 22:36:37.448820 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-176812e0-4217-489c-8eb5-0b1f3c5e5f3d req-fa06d07c-1e96-431d-9847-c2d176416626 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.044579 Jul 16 22:36:37.449195 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3633/7260] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 22:36:37 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 22:36:37.451216 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-53b159e1-dd85-4021-9098-7ec0fadfee43 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:36:37.451676 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-53b159e1-dd85-4021-9098-7ec0fadfee43 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:36:37.452651 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-53b159e1-dd85-4021-9098-7ec0fadfee43 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:36:37.453100 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-53b159e1-dd85-4021-9098-7ec0fadfee43 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:36:37.521815 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-53b159e1-dd85-4021-9098-7ec0fadfee43 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '77a17d04ccb1477cafd82ca23fd65328', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4490e1139a4046228fe39fbdc7973d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:36:37.529617 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-53b159e1-dd85-4021-9098-7ec0fadfee43 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] 10.222.0.16 "GET /compute/v2.1/servers/2cd73456-1037-4243-8aa3-2a93c28b8f96" status: 200 len: 1499 microversion: 2.1 time: 0.090781 Jul 16 22:36:37.530198 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3628/7261] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:36:37 2022] GET /compute/v2.1/servers/2cd73456-1037-4243-8aa3-2a93c28b8f96 => generated 1499 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 16 22:36:38.549122 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-d15f6bfa-de22-4d46-ae79-0fbd99626221 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:36:38.549836 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-d15f6bfa-de22-4d46-ae79-0fbd99626221 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] [instance: 2cd73456-1037-4243-8aa3-2a93c28b8f96] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:36:38.556530 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d15f6bfa-de22-4d46-ae79-0fbd99626221 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:36:38.556992 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d15f6bfa-de22-4d46-ae79-0fbd99626221 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:36:38.557770 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d15f6bfa-de22-4d46-ae79-0fbd99626221 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:36:38.558223 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-d15f6bfa-de22-4d46-ae79-0fbd99626221 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:36:38.571696 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-d15f6bfa-de22-4d46-ae79-0fbd99626221 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] HTTP exception thrown: Instance 2cd73456-1037-4243-8aa3-2a93c28b8f96 could not be found. Jul 16 22:36:38.573392 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-d15f6bfa-de22-4d46-ae79-0fbd99626221 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Request method failure captured: Jul 16 22:36:38.573392 ncdv-master-1 devstack@n-api.service[138681]: request: GET /compute/v2.1/servers/2cd73456-1037-4243-8aa3-2a93c28b8f96 HTTP/1.1 Jul 16 22:36:38.573392 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 22:36:38.573392 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 22:36:38.573392 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 22:36:38.573392 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 22:36:38.573392 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 22:36:38.573392 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 22:36:38.573392 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 22:36:38.573392 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 22:36:38.573392 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0zxWDR_i1rS0dfC59s6FogsdbTsIgJs04NRrUt_Ip6dK9Avdg_hazzWFc0MptINfxdMuB0Yr15DJlf4FYh7gRal53LdHgHSM0QinV58MdOjAYqrjw_lUyorm3rcpeifUpDRmBKnKiSiF8WQjHYVz9g6e9_9wfohGF0S4S9V1JhS0560 Jul 16 22:36:38.573392 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 22:36:38.573392 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 22:36:38.573392 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 22:36:38.573392 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 22:36:38.573392 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 22:36:38.573392 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 22:36:38.573392 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: 4490e1139a4046228fe39fbdc7973d20 Jul 16 22:36:38.573392 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-TestSnapshotPattern-959556747 Jul 16 22:36:38.573392 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 22:36:38.573392 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 22:36:38.573392 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/4490e1139a4046228fe39fbdc7973d20"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/4490e1139a4046228fe39fbdc7973d20"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_4490e1139a4046228fe39fbdc7973d20"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/4490e1139a4046228fe39fbdc7973d20"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 22:36:38.573392 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-TestSnapshotPattern-959556747 Jul 16 22:36:38.573392 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: 4490e1139a4046228fe39fbdc7973d20 Jul 16 22:36:38.573392 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-TestSnapshotPattern-959556747 Jul 16 22:36:38.573392 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-TestSnapshotPattern-959556747-project Jul 16 22:36:38.573392 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 22:36:38.573392 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 22:36:38.573392 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: 77a17d04ccb1477cafd82ca23fd65328 Jul 16 22:36:38.573392 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-TestSnapshotPattern-959556747-project Jul 16 22:36:38.573392 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 22:36:38.573392 ncdv-master-1 devstack@n-api.service[138681]: exception: Instance 2cd73456-1037-4243-8aa3-2a93c28b8f96 could not be found. Jul 16 22:36:38.573392 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': '2cd73456-1037-4243-8aa3-2a93c28b8f96'} Jul 16 22:36:38.573392 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 22:36:38.573392 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 22:36:38.573392 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 22:36:38.573392 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 22:36:38.573392 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 22:36:38.573392 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 22:36:38.573392 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 22:36:38.573392 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 22:36:38.573392 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 22:36:38.573392 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 22:36:38.573392 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 2cd73456-1037-4243-8aa3-2a93c28b8f96 could not be found. Jul 16 22:36:38.573392 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 22:36:38.573392 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 22:36:38.573392 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 22:36:38.573392 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 22:36:38.573392 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 22:36:38.573392 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 22:36:38.573392 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 22:36:38.573392 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 22:36:38.573392 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 22:36:38.573392 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 22:36:38.573392 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 22:36:38.573392 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 22:36:38.573392 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 22:36:38.573392 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 22:36:38.573392 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 22:36:38.573392 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 22:36:38.573392 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 2cd73456-1037-4243-8aa3-2a93c28b8f96 could not be found. Jul 16 22:36:38.573392 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 22:36:38.573392 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 22:36:38.573392 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 22:36:38.584259 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 22:36:38.584259 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 22:36:38.584259 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 22:36:38.584259 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 22:36:38.584259 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 22:36:38.584259 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 2cd73456-1037-4243-8aa3-2a93c28b8f96 could not be found. Jul 16 22:36:38.584259 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 22:36:38.584259 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-d15f6bfa-de22-4d46-ae79-0fbd99626221 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Returning 404 to user: Instance 2cd73456-1037-4243-8aa3-2a93c28b8f96 could not be found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 22:36:38.584259 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-d15f6bfa-de22-4d46-ae79-0fbd99626221 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] 10.222.0.16 "GET /compute/v2.1/servers/2cd73456-1037-4243-8aa3-2a93c28b8f96" status: 404 len: 111 microversion: 2.1 time: 0.031559 Jul 16 22:36:38.584259 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3634/7262] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:36:38 2022] GET /compute/v2.1/servers/2cd73456-1037-4243-8aa3-2a93c28b8f96 => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 22:36:40.847711 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-f512b4d4-a3d3-4661-a577-5fbd7b073028 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:36:40.848515 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-f512b4d4-a3d3-4661-a577-5fbd7b073028 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] [instance: d9d4d400-1fed-4996-9d09-dd0dddd7fc42] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:36:40.850041 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-6712d437-fafa-46c2-9acb-ab9fd5214c64 req-3431bf04-09a1-4067-b983-8f34306b74bc service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "d9d4d400-1fed-4996-9d09-dd0dddd7fc42", "tag": "f6f604bc-d380-4063-b8fa-414495aada0b"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:36:40.855803 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f512b4d4-a3d3-4661-a577-5fbd7b073028 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:36:40.856552 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f512b4d4-a3d3-4661-a577-5fbd7b073028 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:36:40.857570 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f512b4d4-a3d3-4661-a577-5fbd7b073028 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:36:40.858213 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-f512b4d4-a3d3-4661-a577-5fbd7b073028 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:36:40.864416 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-6712d437-fafa-46c2-9acb-ab9fd5214c64 req-3431bf04-09a1-4067-b983-8f34306b74bc service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:36:40.864416 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-6712d437-fafa-46c2-9acb-ab9fd5214c64 req-3431bf04-09a1-4067-b983-8f34306b74bc service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:36:40.874041 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-6712d437-fafa-46c2-9acb-ab9fd5214c64 req-3431bf04-09a1-4067-b983-8f34306b74bc service nova] Creating event network-changed:f6f604bc-d380-4063-b8fa-414495aada0b for instance d9d4d400-1fed-4996-9d09-dd0dddd7fc42 on nch1-master-1 Jul 16 22:36:40.879848 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-6712d437-fafa-46c2-9acb-ab9fd5214c64 req-3431bf04-09a1-4067-b983-8f34306b74bc service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.033513 Jul 16 22:36:40.880726 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3635/7263] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 22:36:40 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 22:36:40.895531 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-f512b4d4-a3d3-4661-a577-5fbd7b073028 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] [instance: d9d4d400-1fed-4996-9d09-dd0dddd7fc42] Going to try to terminate instance {{(pid=138680) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 16 22:36:41.258425 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-f512b4d4-a3d3-4661-a577-5fbd7b073028 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] 10.222.0.16 "DELETE /compute/v2.1/servers/d9d4d400-1fed-4996-9d09-dd0dddd7fc42" status: 204 len: 0 microversion: 2.1 time: 0.415109 Jul 16 22:36:41.262646 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3629/7264] 10.222.0.16 () {58 vars in 1238 bytes} [Sat Jul 16 22:36:40 2022] DELETE /compute/v2.1/servers/d9d4d400-1fed-4996-9d09-dd0dddd7fc42 => generated 0 bytes in 417 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 16 22:36:41.269416 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-74bdb9c5-eff3-451e-a476-6603a116bf1d tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:36:41.270147 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-74bdb9c5-eff3-451e-a476-6603a116bf1d tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] [instance: d9d4d400-1fed-4996-9d09-dd0dddd7fc42] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:36:41.277356 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-74bdb9c5-eff3-451e-a476-6603a116bf1d tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:36:41.278007 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-74bdb9c5-eff3-451e-a476-6603a116bf1d tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:36:41.279151 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-74bdb9c5-eff3-451e-a476-6603a116bf1d tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:36:41.279801 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-74bdb9c5-eff3-451e-a476-6603a116bf1d tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:36:41.427264 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.policy [None req-74bdb9c5-eff3-451e-a476-6603a116bf1d tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '77a17d04ccb1477cafd82ca23fd65328', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4490e1139a4046228fe39fbdc7973d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138681) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:36:41.436014 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-74bdb9c5-eff3-451e-a476-6603a116bf1d tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] 10.222.0.16 "GET /compute/v2.1/servers/d9d4d400-1fed-4996-9d09-dd0dddd7fc42" status: 200 len: 1685 microversion: 2.1 time: 0.171472 Jul 16 22:36:41.436590 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3636/7265] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:36:41 2022] GET /compute/v2.1/servers/d9d4d400-1fed-4996-9d09-dd0dddd7fc42 => generated 1685 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:36:42.454957 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-e4dcd7e3-9cce-4326-893b-8f55aa9cc718 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:36:42.455826 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-e4dcd7e3-9cce-4326-893b-8f55aa9cc718 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] [instance: d9d4d400-1fed-4996-9d09-dd0dddd7fc42] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:36:42.462821 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e4dcd7e3-9cce-4326-893b-8f55aa9cc718 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:36:42.463328 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e4dcd7e3-9cce-4326-893b-8f55aa9cc718 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:36:42.464106 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e4dcd7e3-9cce-4326-893b-8f55aa9cc718 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:36:42.464609 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-e4dcd7e3-9cce-4326-893b-8f55aa9cc718 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:36:42.606917 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-e4dcd7e3-9cce-4326-893b-8f55aa9cc718 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '77a17d04ccb1477cafd82ca23fd65328', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4490e1139a4046228fe39fbdc7973d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:36:42.615168 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-e4dcd7e3-9cce-4326-893b-8f55aa9cc718 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] 10.222.0.16 "GET /compute/v2.1/servers/d9d4d400-1fed-4996-9d09-dd0dddd7fc42" status: 200 len: 1685 microversion: 2.1 time: 0.165185 Jul 16 22:36:42.615826 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3630/7266] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:36:42 2022] GET /compute/v2.1/servers/d9d4d400-1fed-4996-9d09-dd0dddd7fc42 => generated 1685 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:36:43.275312 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [req-be79df14-6314-499c-a0ae-e390b65c3a5d req-2d24b933-5be6-4fc9-b565-1def82678f55 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "d9d4d400-1fed-4996-9d09-dd0dddd7fc42", "name": "network-vif-deleted", "tag": "f6f604bc-d380-4063-b8fa-414495aada0b"}]} {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 16 22:36:43.285124 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-be79df14-6314-499c-a0ae-e390b65c3a5d req-2d24b933-5be6-4fc9-b565-1def82678f55 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:36:43.285590 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [req-be79df14-6314-499c-a0ae-e390b65c3a5d req-2d24b933-5be6-4fc9-b565-1def82678f55 service nova] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:36:43.297055 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.compute.server_external_events [req-be79df14-6314-499c-a0ae-e390b65c3a5d req-2d24b933-5be6-4fc9-b565-1def82678f55 service nova] Creating event network-vif-deleted:f6f604bc-d380-4063-b8fa-414495aada0b for instance d9d4d400-1fed-4996-9d09-dd0dddd7fc42 on nch1-master-1 Jul 16 22:36:43.302050 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [req-be79df14-6314-499c-a0ae-e390b65c3a5d req-2d24b933-5be6-4fc9-b565-1def82678f55 service nova] 10.222.0.16 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.030726 Jul 16 22:36:43.302629 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3637/7267] 10.222.0.16 () {64 vars in 1317 bytes} [Sat Jul 16 22:36:43 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 16 22:36:43.633079 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-ae5db34a-6b8f-406a-b555-a47ac5b39f9a tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Calling method '>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:36:43.633701 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.compute.api [None req-ae5db34a-6b8f-406a-b555-a47ac5b39f9a tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] [instance: d9d4d400-1fed-4996-9d09-dd0dddd7fc42] Fetching instance by UUID {{(pid=138680) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:36:43.641207 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ae5db34a-6b8f-406a-b555-a47ac5b39f9a tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:36:43.641697 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ae5db34a-6b8f-406a-b555-a47ac5b39f9a tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:36:43.642492 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ae5db34a-6b8f-406a-b555-a47ac5b39f9a tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:36:43.642936 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ae5db34a-6b8f-406a-b555-a47ac5b39f9a tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:36:43.678498 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ae5db34a-6b8f-406a-b555-a47ac5b39f9a tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:36:43.678953 ncdv-master-1 devstack@n-api.service[138680]: DEBUG oslo_concurrency.lockutils [None req-ae5db34a-6b8f-406a-b555-a47ac5b39f9a tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138680) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:36:43.679463 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.objects.instance [None req-ae5db34a-6b8f-406a-b555-a47ac5b39f9a tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Lazy-loading 'fault' on Instance uuid d9d4d400-1fed-4996-9d09-dd0dddd7fc42 {{(pid=138680) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 16 22:36:43.727423 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.policy [None req-ae5db34a-6b8f-406a-b555-a47ac5b39f9a tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '77a17d04ccb1477cafd82ca23fd65328', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4490e1139a4046228fe39fbdc7973d20', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=138680) authorize /opt/stack/nova/nova/policy.py:199}} Jul 16 22:36:43.735164 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-ae5db34a-6b8f-406a-b555-a47ac5b39f9a tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] 10.222.0.16 "GET /compute/v2.1/servers/d9d4d400-1fed-4996-9d09-dd0dddd7fc42" status: 200 len: 1338 microversion: 2.1 time: 0.106106 Jul 16 22:36:43.736637 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3631/7268] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:36:43 2022] GET /compute/v2.1/servers/d9d4d400-1fed-4996-9d09-dd0dddd7fc42 => generated 1338 bytes in 108 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 16 22:36:44.754676 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-ccf1b565-5555-48b7-a3c0-fb1fb1c8c36c tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Calling method '>' {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:36:44.755663 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.compute.api [None req-ccf1b565-5555-48b7-a3c0-fb1fb1c8c36c tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] [instance: d9d4d400-1fed-4996-9d09-dd0dddd7fc42] Fetching instance by UUID {{(pid=138681) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 16 22:36:44.762170 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ccf1b565-5555-48b7-a3c0-fb1fb1c8c36c tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:36:44.762710 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ccf1b565-5555-48b7-a3c0-fb1fb1c8c36c tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:36:44.763527 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ccf1b565-5555-48b7-a3c0-fb1fb1c8c36c tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 16 22:36:44.763987 ncdv-master-1 devstack@n-api.service[138681]: DEBUG oslo_concurrency.lockutils [None req-ccf1b565-5555-48b7-a3c0-fb1fb1c8c36c tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Lock "327e8df1-39cc-4b36-9ea6-6a9b88ed54fc" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=138681) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 16 22:36:44.777374 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.wsgi [None req-ccf1b565-5555-48b7-a3c0-fb1fb1c8c36c tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] HTTP exception thrown: Instance d9d4d400-1fed-4996-9d09-dd0dddd7fc42 could not be found. Jul 16 22:36:44.779052 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-ccf1b565-5555-48b7-a3c0-fb1fb1c8c36c tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Request method failure captured: Jul 16 22:36:44.779052 ncdv-master-1 devstack@n-api.service[138681]: request: GET /compute/v2.1/servers/d9d4d400-1fed-4996-9d09-dd0dddd7fc42 HTTP/1.1 Jul 16 22:36:44.779052 ncdv-master-1 devstack@n-api.service[138681]: Accept: application/json Jul 16 22:36:44.779052 ncdv-master-1 devstack@n-api.service[138681]: Accept-Encoding: identity Jul 16 22:36:44.779052 ncdv-master-1 devstack@n-api.service[138681]: Connection: close Jul 16 22:36:44.779052 ncdv-master-1 devstack@n-api.service[138681]: Content-Length: 0 Jul 16 22:36:44.779052 ncdv-master-1 devstack@n-api.service[138681]: Content-Type: application/json Jul 16 22:36:44.779052 ncdv-master-1 devstack@n-api.service[138681]: Host: 10.222.0.16 Jul 16 22:36:44.779052 ncdv-master-1 devstack@n-api.service[138681]: Openstack-System-Scope: None Jul 16 22:36:44.779052 ncdv-master-1 devstack@n-api.service[138681]: User-Agent: python-urllib3/1.26.9 Jul 16 22:36:44.779052 ncdv-master-1 devstack@n-api.service[138681]: X-Auth-Token: gAAAAABi0zxWDR_i1rS0dfC59s6FogsdbTsIgJs04NRrUt_Ip6dK9Avdg_hazzWFc0MptINfxdMuB0Yr15DJlf4FYh7gRal53LdHgHSM0QinV58MdOjAYqrjw_lUyorm3rcpeifUpDRmBKnKiSiF8WQjHYVz9g6e9_9wfohGF0S4S9V1JhS0560 Jul 16 22:36:44.779052 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Id: None Jul 16 22:36:44.779052 ncdv-master-1 devstack@n-api.service[138681]: X-Domain-Name: None Jul 16 22:36:44.779052 ncdv-master-1 devstack@n-api.service[138681]: X-Identity-Status: Confirmed Jul 16 22:36:44.779052 ncdv-master-1 devstack@n-api.service[138681]: X-Is-Admin-Project: True Jul 16 22:36:44.779052 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Id: default Jul 16 22:36:44.779052 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Domain-Name: Default Jul 16 22:36:44.779052 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Id: 4490e1139a4046228fe39fbdc7973d20 Jul 16 22:36:44.779052 ncdv-master-1 devstack@n-api.service[138681]: X-Project-Name: tempest-TestSnapshotPattern-959556747 Jul 16 22:36:44.779052 ncdv-master-1 devstack@n-api.service[138681]: X-Role: member,reader Jul 16 22:36:44.779052 ncdv-master-1 devstack@n-api.service[138681]: X-Roles: member,reader Jul 16 22:36:44.779052 ncdv-master-1 devstack@n-api.service[138681]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/image"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/4490e1139a4046228fe39fbdc7973d20"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/volume/v3/4490e1139a4046228fe39fbdc7973d20"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.16:8080", "publicURL": "http://10.222.0.16:8080/v1/AUTH_4490e1139a4046228fe39fbdc7973d20"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2/4490e1139a4046228fe39fbdc7973d20"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/identity"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/compute/v2.1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.16/placement"}]}] Jul 16 22:36:44.779052 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant: tempest-TestSnapshotPattern-959556747 Jul 16 22:36:44.779052 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Id: 4490e1139a4046228fe39fbdc7973d20 Jul 16 22:36:44.779052 ncdv-master-1 devstack@n-api.service[138681]: X-Tenant-Name: tempest-TestSnapshotPattern-959556747 Jul 16 22:36:44.779052 ncdv-master-1 devstack@n-api.service[138681]: X-User: tempest-TestSnapshotPattern-959556747-project Jul 16 22:36:44.779052 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Id: default Jul 16 22:36:44.779052 ncdv-master-1 devstack@n-api.service[138681]: X-User-Domain-Name: Default Jul 16 22:36:44.779052 ncdv-master-1 devstack@n-api.service[138681]: X-User-Id: 77a17d04ccb1477cafd82ca23fd65328 Jul 16 22:36:44.779052 ncdv-master-1 devstack@n-api.service[138681]: X-User-Name: tempest-TestSnapshotPattern-959556747-project Jul 16 22:36:44.779052 ncdv-master-1 devstack@n-api.service[138681]: method: > Jul 16 22:36:44.779052 ncdv-master-1 devstack@n-api.service[138681]: exception: Instance d9d4d400-1fed-4996-9d09-dd0dddd7fc42 could not be found. Jul 16 22:36:44.779052 ncdv-master-1 devstack@n-api.service[138681]: action_args: {'id': 'd9d4d400-1fed-4996-9d09-dd0dddd7fc42'} Jul 16 22:36:44.779052 ncdv-master-1 devstack@n-api.service[138681]: {{(pid=138681) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 16 22:36:44.779052 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 22:36:44.779052 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 16 22:36:44.779052 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 16 22:36:44.779052 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 16 22:36:44.779052 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 16 22:36:44.779052 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 16 22:36:44.779052 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 16 22:36:44.779052 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 16 22:36:44.779052 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 16 22:36:44.779052 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance d9d4d400-1fed-4996-9d09-dd0dddd7fc42 could not be found. Jul 16 22:36:44.779052 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 22:36:44.779052 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 22:36:44.779052 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 22:36:44.779052 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 22:36:44.779052 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 22:36:44.779052 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 22:36:44.779052 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 16 22:36:44.779052 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 16 22:36:44.779052 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 16 22:36:44.779052 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 16 22:36:44.779052 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 16 22:36:44.779052 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 16 22:36:44.779052 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 16 22:36:44.779052 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 16 22:36:44.779052 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 16 22:36:44.779052 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 16 22:36:44.779052 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance d9d4d400-1fed-4996-9d09-dd0dddd7fc42 could not be found. Jul 16 22:36:44.779052 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 22:36:44.779052 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 16 22:36:44.779052 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 22:36:44.784872 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 16 22:36:44.784872 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 16 22:36:44.784872 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 16 22:36:44.784872 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 16 22:36:44.784872 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 16 22:36:44.784872 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance d9d4d400-1fed-4996-9d09-dd0dddd7fc42 could not be found. Jul 16 22:36:44.784872 ncdv-master-1 devstack@n-api.service[138681]: ERROR nova.api.openstack.wsgi Jul 16 22:36:44.784872 ncdv-master-1 devstack@n-api.service[138681]: DEBUG nova.api.openstack.wsgi [None req-ccf1b565-5555-48b7-a3c0-fb1fb1c8c36c tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Returning 404 to user: Instance d9d4d400-1fed-4996-9d09-dd0dddd7fc42 could not be found. {{(pid=138681) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 16 22:36:44.784872 ncdv-master-1 devstack@n-api.service[138681]: INFO nova.api.openstack.requestlog [None req-ccf1b565-5555-48b7-a3c0-fb1fb1c8c36c tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] 10.222.0.16 "GET /compute/v2.1/servers/d9d4d400-1fed-4996-9d09-dd0dddd7fc42" status: 404 len: 111 microversion: 2.1 time: 0.031667 Jul 16 22:36:44.784872 ncdv-master-1 devstack@n-api.service[138681]: [pid: 138681|app: 0|req: 3638/7269] 10.222.0.16 () {58 vars in 1235 bytes} [Sat Jul 16 22:36:44 2022] GET /compute/v2.1/servers/d9d4d400-1fed-4996-9d09-dd0dddd7fc42 => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 16 22:36:45.011638 ncdv-master-1 devstack@n-api.service[138680]: DEBUG nova.api.openstack.wsgi [None req-eede27aa-a03a-46a7-a6d8-0bf763d31ac2 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] Calling method '.version_select at 0x7fdd8fadcd30>' {{(pid=138680) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 16 22:36:45.116561 ncdv-master-1 devstack@n-api.service[138680]: INFO nova.api.openstack.requestlog [None req-eede27aa-a03a-46a7-a6d8-0bf763d31ac2 tempest-TestSnapshotPattern-959556747 tempest-TestSnapshotPattern-959556747-project] 10.222.0.16 "DELETE /compute/v2.1/os-keypairs/tempest-TestSnapshotPattern-1466442200" status: 202 len: 0 microversion: 2.1 time: 0.109015 Jul 16 22:36:45.117156 ncdv-master-1 devstack@n-api.service[138680]: [pid: 138680|app: 0|req: 3632/7270] 10.222.0.16 () {58 vars in 1256 bytes} [Sat Jul 16 22:36:45 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestSnapshotPattern-1466442200 => generated 0 bytes in 110 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 16 22:38:05.058187 ncdv-master-1 devstack@n-api.service[138680]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jul 16 22:38:05.058757 ncdv-master-1 devstack@n-api.service[138680]: Sat Jul 16 22:38:05 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Jul 16 22:38:06.183880 ncdv-master-1 devstack@n-api.service[138681]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jul 16 22:38:06.184493 ncdv-master-1 devstack@n-api.service[138681]: Sat Jul 16 22:38:06 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Jul 16 22:39:44.836121 ncdv-master-1 devstack@n-api.service[138680]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jul 16 22:39:44.836739 ncdv-master-1 devstack@n-api.service[138680]: Sat Jul 16 22:39:44 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Jul 16 22:39:46.931541 ncdv-master-1 devstack@n-api.service[138681]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jul 16 22:39:46.932062 ncdv-master-1 devstack@n-api.service[138681]: Sat Jul 16 22:39:46 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Jul 16 22:41:25.066787 ncdv-master-1 devstack@n-api.service[138680]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jul 16 22:41:25.066787 ncdv-master-1 devstack@n-api.service[138680]: Sat Jul 16 22:41:25 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Jul 16 22:41:26.192860 ncdv-master-1 devstack@n-api.service[138681]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jul 16 22:41:26.193897 ncdv-master-1 devstack@n-api.service[138681]: Sat Jul 16 22:41:26 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Jul 16 22:41:49.003697 ncdv-master-1 devstack@n-api.service[138681]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jul 16 22:41:49.004354 ncdv-master-1 devstack@n-api.service[138681]: Sat Jul 16 22:41:49 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Jul 16 22:41:59.895075 ncdv-master-1 devstack@n-api.service[138680]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jul 16 22:41:59.895540 ncdv-master-1 devstack@n-api.service[138680]: Sat Jul 16 22:41:59 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Jul 16 22:43:04.844133 ncdv-master-1 devstack@n-api.service[138680]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jul 16 22:43:04.844847 ncdv-master-1 devstack@n-api.service[138680]: Sat Jul 16 22:43:04 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Jul 16 22:43:06.939324 ncdv-master-1 devstack@n-api.service[138681]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jul 16 22:43:06.939828 ncdv-master-1 devstack@n-api.service[138681]: Sat Jul 16 22:43:06 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Jul 16 22:44:45.075907 ncdv-master-1 devstack@n-api.service[138680]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jul 16 22:44:45.076536 ncdv-master-1 devstack@n-api.service[138680]: Sat Jul 16 22:44:45 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Jul 16 22:44:46.202637 ncdv-master-1 devstack@n-api.service[138681]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jul 16 22:44:46.203253 ncdv-master-1 devstack@n-api.service[138681]: Sat Jul 16 22:44:46 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Jul 16 22:45:09.013285 ncdv-master-1 devstack@n-api.service[138681]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jul 16 22:45:09.013939 ncdv-master-1 devstack@n-api.service[138681]: Sat Jul 16 22:45:09 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!!